2023-10-06T02:04:06.158 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc 2023-10-06T02:04:06.158 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --owner scheduled_yuriw@teuthology --name yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi --description fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/misc}} --archive /home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412129 -- /home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412129/orig.config.yaml 2023-10-06T02:04:06.203 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-06T02:04:06.287 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412129 branch: wip-yuri-testing-2023-10-05-0730-quincy description: fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/misc}} email: yweinste@redhat.com first_in_suite: false job_id: '7412129' kernel: client: branch: testing kdb: true sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.4' overrides: admin_socket: branch: wip-yuri-testing-2023-10-05-0730-quincy ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: 736ac5ba72cb47af7d6f6666b1e100f7e20a20e2 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 736ac5ba72cb47af7d6f6666b1e100f7e20a20e2 kclient: syntax: v2 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuri-testing-2023-10-05-0730-quincy sha1: f1e315a9732d36c05ec316678e2397dc58561c7f owner: scheduled_yuriw@teuthology priority: 75 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 7519 sha1: 736ac5ba72cb47af7d6f6666b1e100f7e20a20e2 sleep_before_teardown: 0 subset: 111/12000 suite: fs suite_branch: wip-yuri-testing-2023-10-05-0730-quincy suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_f1e315a9732d36c05ec316678e2397dc58561c7f/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: f1e315a9732d36c05ec316678e2397dc58561c7f targets: smithi031.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLZXMnoxnVb2uZqUBuqUws24Kw6G5H9Ueh504kqdn3nWt5HsTr4MrTt+YmJAfIwE8R4h+T5Wag7WSIB7NhjqUsQ= smithi062.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJ9YBgmw9WCA672tKizZk9aWL2VzsFwWWtEBkVvpPLrsDmh+ytXxJq/7oZwtwZzaHXQZNuxRpkPQY6vUM4uonkY= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - ceph: null - kclient: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_volumes.TestMisc teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 54e62bcbac4e53d9685e08328b790d3b20d71cae timestamp: 2023-10-05_23:52:50 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2830 2023-10-06T02:04:06.287 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_f1e315a9732d36c05ec316678e2397dc58561c7f/qa; will attempt to use it 2023-10-06T02:04:06.288 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_f1e315a9732d36c05ec316678e2397dc58561c7f/qa/tasks 2023-10-06T02:04:06.288 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-10-06T02:04:06.290 INFO:teuthology.task.internal:Checking packages... 2023-10-06T02:04:06.312 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '736ac5ba72cb47af7d6f6666b1e100f7e20a20e2' 2023-10-06T02:04:06.312 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-10-06T02:04:06.312 INFO:teuthology.packaging:ref: None 2023-10-06T02:04:06.312 INFO:teuthology.packaging:tag: None 2023-10-06T02:04:06.312 INFO:teuthology.packaging:branch: wip-yuri-testing-2023-10-05-0730-quincy 2023-10-06T02:04:06.312 INFO:teuthology.packaging:sha1: 736ac5ba72cb47af7d6f6666b1e100f7e20a20e2 2023-10-06T02:04:06.312 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-yuri-testing-2023-10-05-0730-quincy 2023-10-06T02:04:06.457 INFO:teuthology.task.internal:Found packages for ceph version 17.2.6-1296.gf1e315a9 2023-10-06T02:04:06.458 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-10-06T02:04:06.466 INFO:teuthology.task.internal:no buildpackages task found 2023-10-06T02:04:06.466 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-10-06T02:04:06.520 INFO:teuthology.task.internal:Saving configuration 2023-10-06T02:04:06.537 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-10-06T02:04:06.578 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-06T02:04:06.597 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi031.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412129', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 01:58:52.479588', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLZXMnoxnVb2uZqUBuqUws24Kw6G5H9Ueh504kqdn3nWt5HsTr4MrTt+YmJAfIwE8R4h+T5Wag7WSIB7NhjqUsQ='} 2023-10-06T02:04:06.616 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi062.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412129', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 01:58:52.481061', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJ9YBgmw9WCA672tKizZk9aWL2VzsFwWWtEBkVvpPLrsDmh+ytXxJq/7oZwtwZzaHXQZNuxRpkPQY6vUM4uonkY='} 2023-10-06T02:04:06.616 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-10-06T02:04:06.628 INFO:teuthology.task.internal:roles: ubuntu@smithi031.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-10-06T02:04:06.628 INFO:teuthology.task.internal:roles: ubuntu@smithi062.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-10-06T02:04:06.628 INFO:teuthology.run_tasks:Running task console_log... 2023-10-06T02:04:06.731 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f5e3774bb80>, signals=[15]) 2023-10-06T02:04:06.731 INFO:teuthology.run_tasks:Running task internal.connect... 2023-10-06T02:04:06.743 INFO:teuthology.task.internal:Opening connections... 2023-10-06T02:04:06.743 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi031.front.sepia.ceph.com 2023-10-06T02:04:06.745 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T02:04:06.811 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi062.front.sepia.ceph.com 2023-10-06T02:04:06.812 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T02:04:06.879 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-10-06T02:04:06.891 DEBUG:teuthology.orchestra.run.smithi031:> uname -m 2023-10-06T02:04:06.908 INFO:teuthology.orchestra.run.smithi031.stdout:x86_64 2023-10-06T02:04:06.909 DEBUG:teuthology.orchestra.run.smithi031:> cat /etc/os-release 2023-10-06T02:04:06.966 INFO:teuthology.orchestra.run.smithi031.stdout:NAME="Red Hat Enterprise Linux" 2023-10-06T02:04:06.966 INFO:teuthology.orchestra.run.smithi031.stdout:VERSION="8.4 (Ootpa)" 2023-10-06T02:04:06.966 INFO:teuthology.orchestra.run.smithi031.stdout:ID="rhel" 2023-10-06T02:04:06.966 INFO:teuthology.orchestra.run.smithi031.stdout:ID_LIKE="fedora" 2023-10-06T02:04:06.966 INFO:teuthology.orchestra.run.smithi031.stdout:VERSION_ID="8.4" 2023-10-06T02:04:06.966 INFO:teuthology.orchestra.run.smithi031.stdout:PLATFORM_ID="platform:el8" 2023-10-06T02:04:06.966 INFO:teuthology.orchestra.run.smithi031.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-10-06T02:04:06.966 INFO:teuthology.orchestra.run.smithi031.stdout:ANSI_COLOR="0;31" 2023-10-06T02:04:06.966 INFO:teuthology.orchestra.run.smithi031.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-10-06T02:04:06.967 INFO:teuthology.orchestra.run.smithi031.stdout:HOME_URL="https://www.redhat.com/" 2023-10-06T02:04:06.967 INFO:teuthology.orchestra.run.smithi031.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-10-06T02:04:06.967 INFO:teuthology.orchestra.run.smithi031.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-06T02:04:06.967 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:04:06.967 INFO:teuthology.orchestra.run.smithi031.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-06T02:04:06.967 INFO:teuthology.orchestra.run.smithi031.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-10-06T02:04:06.967 INFO:teuthology.orchestra.run.smithi031.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-10-06T02:04:06.967 INFO:teuthology.orchestra.run.smithi031.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-10-06T02:04:06.968 INFO:teuthology.lock.ops:Updating smithi031.front.sepia.ceph.com on lock server 2023-10-06T02:04:06.994 DEBUG:teuthology.orchestra.run.smithi062:> uname -m 2023-10-06T02:04:07.011 INFO:teuthology.orchestra.run.smithi062.stdout:x86_64 2023-10-06T02:04:07.011 DEBUG:teuthology.orchestra.run.smithi062:> cat /etc/os-release 2023-10-06T02:04:07.069 INFO:teuthology.orchestra.run.smithi062.stdout:NAME="Red Hat Enterprise Linux" 2023-10-06T02:04:07.069 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION="8.4 (Ootpa)" 2023-10-06T02:04:07.070 INFO:teuthology.orchestra.run.smithi062.stdout:ID="rhel" 2023-10-06T02:04:07.070 INFO:teuthology.orchestra.run.smithi062.stdout:ID_LIKE="fedora" 2023-10-06T02:04:07.070 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION_ID="8.4" 2023-10-06T02:04:07.070 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM_ID="platform:el8" 2023-10-06T02:04:07.070 INFO:teuthology.orchestra.run.smithi062.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-10-06T02:04:07.071 INFO:teuthology.orchestra.run.smithi062.stdout:ANSI_COLOR="0;31" 2023-10-06T02:04:07.071 INFO:teuthology.orchestra.run.smithi062.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-10-06T02:04:07.071 INFO:teuthology.orchestra.run.smithi062.stdout:HOME_URL="https://www.redhat.com/" 2023-10-06T02:04:07.071 INFO:teuthology.orchestra.run.smithi062.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-10-06T02:04:07.071 INFO:teuthology.orchestra.run.smithi062.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-06T02:04:07.072 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-10-06T02:04:07.072 INFO:teuthology.orchestra.run.smithi062.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-06T02:04:07.072 INFO:teuthology.orchestra.run.smithi062.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-10-06T02:04:07.072 INFO:teuthology.orchestra.run.smithi062.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-10-06T02:04:07.072 INFO:teuthology.orchestra.run.smithi062.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-10-06T02:04:07.073 INFO:teuthology.lock.ops:Updating smithi062.front.sepia.ceph.com on lock server 2023-10-06T02:04:07.093 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-10-06T02:04:07.111 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-10-06T02:04:07.165 INFO:teuthology.task.internal:Checking for old test directory... 2023-10-06T02:04:07.166 DEBUG:teuthology.orchestra.run.smithi031:> test '!' -e /home/ubuntu/cephtest 2023-10-06T02:04:07.168 DEBUG:teuthology.orchestra.run.smithi062:> test '!' -e /home/ubuntu/cephtest 2023-10-06T02:04:07.184 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-10-06T02:04:07.208 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-10-06T02:04:07.208 DEBUG:teuthology.orchestra.run.smithi031:> test -z $(ls -A /var/lib/ceph) 2023-10-06T02:04:07.225 DEBUG:teuthology.orchestra.run.smithi062:> test -z $(ls -A /var/lib/ceph) 2023-10-06T02:04:07.240 INFO:teuthology.orchestra.run.smithi031.stderr:ls: cannot access '/var/lib/ceph': No such file or directory 2023-10-06T02:04:07.242 INFO:teuthology.orchestra.run.smithi062.stderr:ls: cannot access '/var/lib/ceph': No such file or directory 2023-10-06T02:04:07.243 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-10-06T02:04:07.308 INFO:teuthology.run_tasks:Running task kernel... 2023-10-06T02:04:07.329 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'} 2023-10-06T02:04:07.329 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2023-10-06T02:04:07.329 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2023-10-06T02:04:07.330 DEBUG:teuthology.orchestra.run.smithi031:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-06T02:04:07.330 DEBUG:teuthology.orchestra.run.smithi062:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-06T02:04:07.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T02:04:07.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T02:04:07.348 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&ref=testing 2023-10-06T02:04:07.349 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&ref=testing 2023-10-06T02:04:07.716 DEBUG:teuthology.task.kernel:sha1 for client.0 is fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-10-06T02:04:07.716 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c"... 2023-10-06T02:04:07.716 DEBUG:teuthology.orchestra.run.smithi031:> uname -r 2023-10-06T02:04:07.725 DEBUG:teuthology.task.kernel:sha1 for client.1 is fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-10-06T02:04:07.725 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c"... 2023-10-06T02:04:07.725 DEBUG:teuthology.orchestra.run.smithi062:> uname -r 2023-10-06T02:04:07.733 INFO:teuthology.orchestra.run.smithi031.stdout:4.18.0-305.25.1.el8_4.x86_64 2023-10-06T02:04:07.734 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-305.25.1.el8_4.x86_64 vs fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-10-06T02:04:07.734 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-10-06T02:04:07.734 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum upgrade -y linux-firmware 2023-10-06T02:04:07.742 INFO:teuthology.orchestra.run.smithi062.stdout:4.18.0-305.25.1.el8_4.x86_64 2023-10-06T02:04:07.742 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-305.25.1.el8_4.x86_64 vs fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-10-06T02:04:07.743 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-10-06T02:04:07.743 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum upgrade -y linux-firmware 2023-10-06T02:04:09.169 INFO:teuthology.orchestra.run.smithi062.stdout:Updating Subscription Management repositories. 2023-10-06T02:04:09.170 INFO:teuthology.orchestra.run.smithi062.stdout:Unable to read consumer identity 2023-10-06T02:04:09.225 INFO:teuthology.orchestra.run.smithi031.stdout:Updating Subscription Management repositories. 2023-10-06T02:04:09.226 INFO:teuthology.orchestra.run.smithi031.stdout:Unable to read consumer identity 2023-10-06T02:04:10.637 INFO:teuthology.orchestra.run.smithi062.stdout:Copr repo for python3-asyncssh owned by ceph 26 kB/s | 3.5 kB 00:00 2023-10-06T02:04:10.685 INFO:teuthology.orchestra.run.smithi031.stdout:Copr repo for python3-asyncssh owned by ceph 25 kB/s | 3.5 kB 00:00 2023-10-06T02:04:11.491 INFO:teuthology.orchestra.run.smithi062.stdout:Extra Packages for Enterprise Linux 19 MB/s | 16 MB 00:00 2023-10-06T02:04:12.437 INFO:teuthology.orchestra.run.smithi031.stdout:Extra Packages for Enterprise Linux 9.3 MB/s | 16 MB 00:01 2023-10-06T02:04:14.970 INFO:teuthology.orchestra.run.smithi062.stdout:lab-extras 4.7 MB/s | 24 kB 00:00 2023-10-06T02:04:15.891 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2023-10-06T02:04:15.892 INFO:teuthology.orchestra.run.smithi062.stdout:Nothing to do. 2023-10-06T02:04:15.892 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2023-10-06T02:04:15.950 INFO:teuthology.orchestra.run.smithi031.stdout:lab-extras 4.4 MB/s | 24 kB 00:00 2023-10-06T02:04:16.012 INFO:teuthology.task.kernel:Downloading kernel fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c on client.1... 2023-10-06T02:04:16.012 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-10-06T02:04:16.233 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/centos/8/flavors/default/x86_64/ 2023-10-06T02:04:16.233 DEBUG:teuthology.orchestra.run.smithi062:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.5.0_rc7_gfef64f93da2b-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/centos/8/flavors/default/x86_64/ --input-file=- 2023-10-06T02:04:16.236 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi062.front.sepia.ceph.com to complete... 2023-10-06T02:04:16.870 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2023-10-06T02:04:16.871 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2023-10-06T02:04:16.872 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2023-10-06T02:04:16.994 INFO:teuthology.task.kernel:Downloading kernel fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c on client.0... 2023-10-06T02:04:16.995 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-10-06T02:04:17.214 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/centos/8/flavors/default/x86_64/ 2023-10-06T02:04:17.215 DEBUG:teuthology.orchestra.run.smithi031:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.5.0_rc7_gfef64f93da2b-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/centos/8/flavors/default/x86_64/ --input-file=- 2023-10-06T02:04:17.217 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi031.front.sepia.ceph.com to complete... 2023-10-06T02:04:34.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06 02:04:34 URL:https://3.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/centos/8/flavors/default/x86_64/kernel-6.5.0_rc7_gfef64f93da2b-1.x86_64.rpm [380759012/380759012] -> "/tmp/kernel.x86_64.rpm" [1] 2023-10-06T02:04:34.102 INFO:teuthology.orchestra.run.smithi062.stderr:FINISHED --2023-10-06 02:04:34-- 2023-10-06T02:04:34.103 INFO:teuthology.orchestra.run.smithi062.stderr:Total wall clock time: 18s 2023-10-06T02:04:34.103 INFO:teuthology.orchestra.run.smithi062.stderr:Downloaded: 1 files, 363M in 18s (20.6 MB/s) 2023-10-06T02:04:34.104 INFO:teuthology.task.kernel:Installing kernel fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c on client.1... 2023-10-06T02:04:34.104 DEBUG:teuthology.orchestra.run.smithi062:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-10-06T02:04:35.950 INFO:teuthology.orchestra.run.smithi062.stdout:Verifying... ######################################## 2023-10-06T02:04:36.336 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing... ######################################## 2023-10-06T02:04:36.506 INFO:teuthology.orchestra.run.smithi062.stdout:Updating / installing... 2023-10-06T02:04:51.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06 02:04:51 URL:https://3.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/centos/8/flavors/default/x86_64/kernel-6.5.0_rc7_gfef64f93da2b-1.x86_64.rpm [380759012/380759012] -> "/tmp/kernel.x86_64.rpm" [1] 2023-10-06T02:04:51.036 INFO:teuthology.orchestra.run.smithi031.stderr:FINISHED --2023-10-06 02:04:51-- 2023-10-06T02:04:51.036 INFO:teuthology.orchestra.run.smithi031.stderr:Total wall clock time: 34s 2023-10-06T02:04:51.036 INFO:teuthology.orchestra.run.smithi031.stderr:Downloaded: 1 files, 363M in 33s (10.9 MB/s) 2023-10-06T02:04:51.100 INFO:teuthology.task.kernel:Installing kernel fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c on client.0... 2023-10-06T02:04:51.100 DEBUG:teuthology.orchestra.run.smithi031:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-10-06T02:04:53.763 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying... ######################################## 2023-10-06T02:04:54.144 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing... ######################################## 2023-10-06T02:04:54.289 INFO:teuthology.orchestra.run.smithi031.stdout:Updating / installing... 2023-10-06T02:05:17.080 INFO:teuthology.orchestra.run.smithi062.stdout:kernel-6.5.0_rc7_gfef64f93da2b-1 ######################################## 2023-10-06T02:05:34.717 INFO:teuthology.orchestra.run.smithi031.stdout:kernel-6.5.0_rc7_gfef64f93da2b-1 ######################################## 2023-10-06T02:05:39.523 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi062.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-10-06T02:05:39.523 DEBUG:teuthology.orchestra.run.smithi062:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-10-06T02:05:39.575 INFO:teuthology.orchestra.run.smithi062.stdout:/boot/System.map-6.5.0-rc7-gfef64f93da2b 2023-10-06T02:05:39.575 INFO:teuthology.orchestra.run.smithi062.stdout:/boot/config-6.5.0-rc7-gfef64f93da2b 2023-10-06T02:05:39.575 INFO:teuthology.orchestra.run.smithi062.stdout:/boot/vmlinuz-6.5.0-rc7-gfef64f93da2b 2023-10-06T02:05:39.575 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b 2023-10-06T02:05:39.575 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel 2023-10-06T02:05:39.576 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch 2023-10-06T02:05:39.576 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86 2023-10-06T02:05:39.576 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto 2023-10-06T02:05:39.576 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-10-06T02:05:39.576 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-10-06T02:05:39.576 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-10-06T02:05:39.576 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-10-06T02:05:39.576 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-10-06T02:05:39.576 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-10-06T02:05:39.576 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-10-06T02:05:39.576 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crc32c-intel.ko 2023-10-06T02:05:39.576 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-10-06T02:05:39.576 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-10-06T02:05:39.577 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-10-06T02:05:39.577 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-10-06T02:05:39.577 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-avx2.ko 2023-10-06T02:05:39.577 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-10-06T02:05:39.577 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-10-06T02:05:39.577 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-10-06T02:05:39.577 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-10-06T02:05:39.577 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-10-06T02:05:39.577 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-10-06T02:05:39.577 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-10-06T02:05:39.577 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel 2023-10-06T02:05:39.577 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu 2023-10-06T02:05:39.578 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu/mce 2023-10-06T02:05:39.578 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-10-06T02:05:39.578 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm 2023-10-06T02:05:39.578 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm-amd.ko 2023-10-06T02:05:39.578 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm-intel.ko 2023-10-06T02:05:39.578 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm.ko 2023-10-06T02:05:39.578 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto 2023-10-06T02:05:39.578 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/ansi_cprng.ko 2023-10-06T02:05:39.578 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/anubis.ko 2023-10-06T02:05:39.578 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/arc4.ko 2023-10-06T02:05:39.578 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx 2023-10-06T02:05:39.578 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_memcpy.ko 2023-10-06T02:05:39.579 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_pq.ko 2023-10-06T02:05:39.579 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_raid6_recov.ko 2023-10-06T02:05:39.579 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_tx.ko 2023-10-06T02:05:39.579 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_xor.ko 2023-10-06T02:05:39.579 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/raid6test.ko 2023-10-06T02:05:39.579 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/authenc.ko 2023-10-06T02:05:39.579 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/authencesn.ko 2023-10-06T02:05:39.579 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blake2b_generic.ko 2023-10-06T02:05:39.579 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blowfish_common.ko 2023-10-06T02:05:39.579 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blowfish_generic.ko 2023-10-06T02:05:39.579 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/camellia_generic.ko 2023-10-06T02:05:39.579 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast5_generic.ko 2023-10-06T02:05:39.579 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast6_generic.ko 2023-10-06T02:05:39.579 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast_common.ko 2023-10-06T02:05:39.580 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/ccm.ko 2023-10-06T02:05:39.580 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cmac.ko 2023-10-06T02:05:39.580 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crc32_generic.ko 2023-10-06T02:05:39.580 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crypto_engine.ko 2023-10-06T02:05:39.580 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crypto_user.ko 2023-10-06T02:05:39.580 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cts.ko 2023-10-06T02:05:39.580 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/des_generic.ko 2023-10-06T02:05:39.580 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/dh_generic.ko 2023-10-06T02:05:39.580 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/echainiv.ko 2023-10-06T02:05:39.580 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/essiv.ko 2023-10-06T02:05:39.580 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/fcrypt.ko 2023-10-06T02:05:39.580 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/khazad.ko 2023-10-06T02:05:39.581 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/lz4.ko 2023-10-06T02:05:39.581 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/lz4hc.ko 2023-10-06T02:05:39.581 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/md4.ko 2023-10-06T02:05:39.581 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/michael_mic.ko 2023-10-06T02:05:39.581 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/pcbc.ko 2023-10-06T02:05:39.581 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/pcrypt.ko 2023-10-06T02:05:39.581 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/rmd160.ko 2023-10-06T02:05:39.581 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/seed.ko 2023-10-06T02:05:39.581 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/serpent_generic.ko 2023-10-06T02:05:39.581 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/tcrypt.ko 2023-10-06T02:05:39.581 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/tea.ko 2023-10-06T02:05:39.581 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/twofish_common.ko 2023-10-06T02:05:39.581 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/twofish_generic.ko 2023-10-06T02:05:39.582 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/vmac.ko 2023-10-06T02:05:39.582 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/wp512.ko 2023-10-06T02:05:39.582 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xcbc.ko 2023-10-06T02:05:39.582 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xor.ko 2023-10-06T02:05:39.582 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xxhash_generic.ko 2023-10-06T02:05:39.582 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers 2023-10-06T02:05:39.582 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi 2023-10-06T02:05:39.582 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/acpi_ipmi.ko 2023-10-06T02:05:39.582 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/acpi_pad.ko 2023-10-06T02:05:39.582 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/custom_method.ko 2023-10-06T02:05:39.582 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/ec_sys.ko 2023-10-06T02:05:39.582 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/platform_profile.ko 2023-10-06T02:05:39.583 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/sbs.ko 2023-10-06T02:05:39.583 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/sbshc.ko 2023-10-06T02:05:39.583 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/video.ko 2023-10-06T02:05:39.583 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata 2023-10-06T02:05:39.583 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/acard-ahci.ko 2023-10-06T02:05:39.583 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/ahci_platform.ko 2023-10-06T02:05:39.583 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/ata_generic.ko 2023-10-06T02:05:39.583 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/libahci_platform.ko 2023-10-06T02:05:39.583 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_acpi.ko 2023-10-06T02:05:39.583 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ali.ko 2023-10-06T02:05:39.583 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_amd.ko 2023-10-06T02:05:39.583 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_artop.ko 2023-10-06T02:05:39.583 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_atiixp.ko 2023-10-06T02:05:39.584 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_atp867x.ko 2023-10-06T02:05:39.584 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cmd640.ko 2023-10-06T02:05:39.584 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cmd64x.ko 2023-10-06T02:05:39.584 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cypress.ko 2023-10-06T02:05:39.584 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_efar.ko 2023-10-06T02:05:39.584 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt366.ko 2023-10-06T02:05:39.584 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt37x.ko 2023-10-06T02:05:39.584 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt3x2n.ko 2023-10-06T02:05:39.584 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt3x3.ko 2023-10-06T02:05:39.584 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_it8213.ko 2023-10-06T02:05:39.584 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_it821x.ko 2023-10-06T02:05:39.584 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_jmicron.ko 2023-10-06T02:05:39.585 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_marvell.ko 2023-10-06T02:05:39.585 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_mpiix.ko 2023-10-06T02:05:39.585 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_netcell.ko 2023-10-06T02:05:39.585 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ninja32.ko 2023-10-06T02:05:39.585 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ns87410.ko 2023-10-06T02:05:39.585 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ns87415.ko 2023-10-06T02:05:39.585 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_oldpiix.ko 2023-10-06T02:05:39.585 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_opti.ko 2023-10-06T02:05:39.585 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_optidma.ko 2023-10-06T02:05:39.585 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pcmcia.ko 2023-10-06T02:05:39.585 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pdc2027x.ko 2023-10-06T02:05:39.585 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-10-06T02:05:39.586 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_piccolo.ko 2023-10-06T02:05:39.586 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_rdc.ko 2023-10-06T02:05:39.586 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sch.ko 2023-10-06T02:05:39.586 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_serverworks.ko 2023-10-06T02:05:39.586 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sil680.ko 2023-10-06T02:05:39.586 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sis.ko 2023-10-06T02:05:39.586 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sl82c105.ko 2023-10-06T02:05:39.586 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_triflex.ko 2023-10-06T02:05:39.586 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_via.ko 2023-10-06T02:05:39.586 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pdc_adma.ko 2023-10-06T02:05:39.586 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_inic162x.ko 2023-10-06T02:05:39.586 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_mv.ko 2023-10-06T02:05:39.587 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_nv.ko 2023-10-06T02:05:39.587 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_promise.ko 2023-10-06T02:05:39.587 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_qstor.ko 2023-10-06T02:05:39.587 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sil.ko 2023-10-06T02:05:39.587 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sil24.ko 2023-10-06T02:05:39.587 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sis.ko 2023-10-06T02:05:39.587 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_svw.ko 2023-10-06T02:05:39.587 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sx4.ko 2023-10-06T02:05:39.587 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_uli.ko 2023-10-06T02:05:39.587 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_via.ko 2023-10-06T02:05:39.587 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_vsc.ko 2023-10-06T02:05:39.587 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/auxdisplay 2023-10-06T02:05:39.587 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/auxdisplay/ks0108.ko 2023-10-06T02:05:39.588 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base 2023-10-06T02:05:39.588 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base/regmap 2023-10-06T02:05:39.588 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base/regmap/regmap-i2c.ko 2023-10-06T02:05:39.588 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base/regmap/regmap-mmio.ko 2023-10-06T02:05:39.588 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/bcma 2023-10-06T02:05:39.588 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/bcma/bcma.ko 2023-10-06T02:05:39.588 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block 2023-10-06T02:05:39.588 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/aoe 2023-10-06T02:05:39.588 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/aoe/aoe.ko 2023-10-06T02:05:39.588 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/brd.ko 2023-10-06T02:05:39.588 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/drbd 2023-10-06T02:05:39.588 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/drbd/drbd.ko 2023-10-06T02:05:39.588 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/floppy.ko 2023-10-06T02:05:39.589 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/loop.ko 2023-10-06T02:05:39.589 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/mtip32xx 2023-10-06T02:05:39.589 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-10-06T02:05:39.589 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/nbd.ko 2023-10-06T02:05:39.589 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/null_blk 2023-10-06T02:05:39.589 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/null_blk/null_blk.ko 2023-10-06T02:05:39.589 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/pktcdvd.ko 2023-10-06T02:05:39.589 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/rbd.ko 2023-10-06T02:05:39.589 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/virtio_blk.ko 2023-10-06T02:05:39.589 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkback 2023-10-06T02:05:39.589 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-10-06T02:05:39.589 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkfront.ko 2023-10-06T02:05:39.590 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/zram 2023-10-06T02:05:39.590 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/zram/zram.ko 2023-10-06T02:05:39.590 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char 2023-10-06T02:05:39.590 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hangcheck-timer.ko 2023-10-06T02:05:39.590 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random 2023-10-06T02:05:39.590 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/amd-rng.ko 2023-10-06T02:05:39.590 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/intel-rng.ko 2023-10-06T02:05:39.590 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-10-06T02:05:39.590 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/via-rng.ko 2023-10-06T02:05:39.590 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/virtio-rng.ko 2023-10-06T02:05:39.590 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi 2023-10-06T02:05:39.590 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-10-06T02:05:39.591 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-10-06T02:05:39.591 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-10-06T02:05:39.591 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_si.ko 2023-10-06T02:05:39.591 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-10-06T02:05:39.591 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-10-06T02:05:39.591 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/lp.ko 2023-10-06T02:05:39.591 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/mwave 2023-10-06T02:05:39.591 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/mwave/mwave.ko 2023-10-06T02:05:39.591 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ppdev.ko 2023-10-06T02:05:39.591 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tlclk.ko 2023-10-06T02:05:39.591 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm 2023-10-06T02:05:39.591 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm.ko 2023-10-06T02:05:39.591 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_atmel.ko 2023-10-06T02:05:39.591 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_infineon.ko 2023-10-06T02:05:39.592 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_nsc.ko 2023-10-06T02:05:39.592 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_tis.ko 2023-10-06T02:05:39.592 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-10-06T02:05:39.592 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/uv_mmtimer.ko 2023-10-06T02:05:39.592 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/virtio_console.ko 2023-10-06T02:05:39.592 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq 2023-10-06T02:05:39.592 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-10-06T02:05:39.592 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-10-06T02:05:39.592 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/p4-clockmod.ko 2023-10-06T02:05:39.592 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-10-06T02:05:39.592 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/powernow-k8.ko 2023-10-06T02:05:39.592 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/speedstep-lib.ko 2023-10-06T02:05:39.592 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto 2023-10-06T02:05:39.593 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp 2023-10-06T02:05:39.593 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-10-06T02:05:39.593 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp/ccp.ko 2023-10-06T02:05:39.593 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel 2023-10-06T02:05:39.593 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat 2023-10-06T02:05:39.593 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_common 2023-10-06T02:05:39.593 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-10-06T02:05:39.593 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-10-06T02:05:39.593 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-10-06T02:05:39.593 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/padlock-aes.ko 2023-10-06T02:05:39.593 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/padlock-sha.ko 2023-10-06T02:05:39.593 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/virtio 2023-10-06T02:05:39.593 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-10-06T02:05:39.594 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dca 2023-10-06T02:05:39.594 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dca/dca.ko 2023-10-06T02:05:39.594 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/devfreq 2023-10-06T02:05:39.594 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-10-06T02:05:39.594 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma 2023-10-06T02:05:39.594 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/dw 2023-10-06T02:05:39.594 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/dw/dw_dmac.ko 2023-10-06T02:05:39.594 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/ioat 2023-10-06T02:05:39.594 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/ioat/ioatdma.ko 2023-10-06T02:05:39.594 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac 2023-10-06T02:05:39.594 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/amd64_edac.ko 2023-10-06T02:05:39.595 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/e752x_edac.ko 2023-10-06T02:05:39.595 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/edac_mce_amd.ko 2023-10-06T02:05:39.595 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i3000_edac.ko 2023-10-06T02:05:39.595 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i3200_edac.ko 2023-10-06T02:05:39.595 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i5100_edac.ko 2023-10-06T02:05:39.595 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i5400_edac.ko 2023-10-06T02:05:39.595 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i7300_edac.ko 2023-10-06T02:05:39.595 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i7core_edac.ko 2023-10-06T02:05:39.595 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i82975x_edac.ko 2023-10-06T02:05:39.595 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/ie31200_edac.ko 2023-10-06T02:05:39.595 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/sb_edac.ko 2023-10-06T02:05:39.595 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/x38_edac.ko 2023-10-06T02:05:39.596 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware 2023-10-06T02:05:39.596 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware/edd.ko 2023-10-06T02:05:39.596 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware/iscsi_ibft.ko 2023-10-06T02:05:39.596 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio 2023-10-06T02:05:39.596 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-ich.ko 2023-10-06T02:05:39.596 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-viperboard.ko 2023-10-06T02:05:39.596 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid 2023-10-06T02:05:39.596 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-appleir.ko 2023-10-06T02:05:39.596 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-aureal.ko 2023-10-06T02:05:39.596 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-axff.ko 2023-10-06T02:05:39.596 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-dr.ko 2023-10-06T02:05:39.596 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-elecom.ko 2023-10-06T02:05:39.597 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-elo.ko 2023-10-06T02:05:39.597 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-emsff.ko 2023-10-06T02:05:39.597 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gaff.ko 2023-10-06T02:05:39.597 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gt683r.ko 2023-10-06T02:05:39.597 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gyration.ko 2023-10-06T02:05:39.597 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtek-kbd.ko 2023-10-06T02:05:39.597 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtek-mouse.ko 2023-10-06T02:05:39.597 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtekff.ko 2023-10-06T02:05:39.597 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-hyperv.ko 2023-10-06T02:05:39.597 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-icade.ko 2023-10-06T02:05:39.597 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-keytouch.ko 2023-10-06T02:05:39.597 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-kye.ko 2023-10-06T02:05:39.597 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-lcpower.ko 2023-10-06T02:05:39.598 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-led.ko 2023-10-06T02:05:39.598 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-lenovo.ko 2023-10-06T02:05:39.598 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-logitech-dj.ko 2023-10-06T02:05:39.598 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-10-06T02:05:39.598 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-multitouch.ko 2023-10-06T02:05:39.598 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-ortek.ko 2023-10-06T02:05:39.598 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-penmount.ko 2023-10-06T02:05:39.598 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-petalynx.ko 2023-10-06T02:05:39.598 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-picolcd.ko 2023-10-06T02:05:39.598 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-pl.ko 2023-10-06T02:05:39.598 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-plantronics.ko 2023-10-06T02:05:39.598 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-primax.ko 2023-10-06T02:05:39.599 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-rmi.ko 2023-10-06T02:05:39.599 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-arvo.ko 2023-10-06T02:05:39.599 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-common.ko 2023-10-06T02:05:39.599 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-isku.ko 2023-10-06T02:05:39.599 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-kone.ko 2023-10-06T02:05:39.599 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-10-06T02:05:39.599 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-konepure.ko 2023-10-06T02:05:39.599 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-10-06T02:05:39.599 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-lua.ko 2023-10-06T02:05:39.599 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-pyra.ko 2023-10-06T02:05:39.599 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-ryos.ko 2023-10-06T02:05:39.599 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-savu.ko 2023-10-06T02:05:39.600 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat.ko 2023-10-06T02:05:39.600 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-saitek.ko 2023-10-06T02:05:39.600 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-samsung.ko 2023-10-06T02:05:39.600 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sensor-hub.ko 2023-10-06T02:05:39.600 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sjoy.ko 2023-10-06T02:05:39.600 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sony.ko 2023-10-06T02:05:39.600 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-speedlink.ko 2023-10-06T02:05:39.600 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-steelseries.ko 2023-10-06T02:05:39.600 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sunplus.ko 2023-10-06T02:05:39.600 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-thrustmaster.ko 2023-10-06T02:05:39.600 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-tivo.ko 2023-10-06T02:05:39.600 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-tmff.ko 2023-10-06T02:05:39.600 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-topseed.ko 2023-10-06T02:05:39.601 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-twinhan.ko 2023-10-06T02:05:39.601 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-uclogic.ko 2023-10-06T02:05:39.601 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-waltop.ko 2023-10-06T02:05:39.601 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-wiimote.ko 2023-10-06T02:05:39.601 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-xinmo.ko 2023-10-06T02:05:39.601 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-zpff.ko 2023-10-06T02:05:39.601 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-zydacron.ko 2023-10-06T02:05:39.601 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/uhid.ko 2023-10-06T02:05:39.601 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/wacom.ko 2023-10-06T02:05:39.601 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv 2023-10-06T02:05:39.601 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_balloon.ko 2023-10-06T02:05:39.601 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_utils.ko 2023-10-06T02:05:39.602 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_vmbus.ko 2023-10-06T02:05:39.602 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon 2023-10-06T02:05:39.602 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/abituguru.ko 2023-10-06T02:05:39.602 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/abituguru3.ko 2023-10-06T02:05:39.602 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/acpi_power_meter.ko 2023-10-06T02:05:39.602 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ad7414.ko 2023-10-06T02:05:39.602 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ad7418.ko 2023-10-06T02:05:39.602 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adc128d818.ko 2023-10-06T02:05:39.602 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1025.ko 2023-10-06T02:05:39.602 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1026.ko 2023-10-06T02:05:39.602 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1029.ko 2023-10-06T02:05:39.602 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1031.ko 2023-10-06T02:05:39.603 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm9240.ko 2023-10-06T02:05:39.603 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ads7828.ko 2023-10-06T02:05:39.603 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7410.ko 2023-10-06T02:05:39.603 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7411.ko 2023-10-06T02:05:39.603 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7462.ko 2023-10-06T02:05:39.603 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7470.ko 2023-10-06T02:05:39.603 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7475.ko 2023-10-06T02:05:39.603 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7x10.ko 2023-10-06T02:05:39.603 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/amc6821.ko 2023-10-06T02:05:39.603 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/applesmc.ko 2023-10-06T02:05:39.603 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asb100.ko 2023-10-06T02:05:39.603 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asc7621.ko 2023-10-06T02:05:39.604 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asus_atk0110.ko 2023-10-06T02:05:39.604 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/atxp1.ko 2023-10-06T02:05:39.604 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/coretemp.ko 2023-10-06T02:05:39.604 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-10-06T02:05:39.604 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/dme1737.ko 2023-10-06T02:05:39.604 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ds1621.ko 2023-10-06T02:05:39.604 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ds620.ko 2023-10-06T02:05:39.604 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/emc1403.ko 2023-10-06T02:05:39.604 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/emc6w201.ko 2023-10-06T02:05:39.604 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f71805f.ko 2023-10-06T02:05:39.604 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f71882fg.ko 2023-10-06T02:05:39.604 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f75375s.ko 2023-10-06T02:05:39.605 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/fam15h_power.ko 2023-10-06T02:05:39.605 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/fschmd.ko 2023-10-06T02:05:39.605 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/g760a.ko 2023-10-06T02:05:39.605 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/g762.ko 2023-10-06T02:05:39.605 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/gl518sm.ko 2023-10-06T02:05:39.605 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/gl520sm.ko 2023-10-06T02:05:39.605 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/hwmon-vid.ko 2023-10-06T02:05:39.605 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/i5500_temp.ko 2023-10-06T02:05:39.605 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/i5k_amb.ko 2023-10-06T02:05:39.605 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ibmaem.ko 2023-10-06T02:05:39.605 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ibmpex.ko 2023-10-06T02:05:39.606 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ina209.ko 2023-10-06T02:05:39.606 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ina2xx.ko 2023-10-06T02:05:39.606 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/it87.ko 2023-10-06T02:05:39.606 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/k10temp.ko 2023-10-06T02:05:39.606 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/k8temp.ko 2023-10-06T02:05:39.606 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lineage-pem.ko 2023-10-06T02:05:39.606 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm63.ko 2023-10-06T02:05:39.606 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm73.ko 2023-10-06T02:05:39.606 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm75.ko 2023-10-06T02:05:39.606 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm77.ko 2023-10-06T02:05:39.606 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm78.ko 2023-10-06T02:05:39.606 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm80.ko 2023-10-06T02:05:39.607 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm83.ko 2023-10-06T02:05:39.607 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm85.ko 2023-10-06T02:05:39.607 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm87.ko 2023-10-06T02:05:39.607 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm90.ko 2023-10-06T02:05:39.607 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm92.ko 2023-10-06T02:05:39.607 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm93.ko 2023-10-06T02:05:39.607 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95234.ko 2023-10-06T02:05:39.607 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95241.ko 2023-10-06T02:05:39.607 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95245.ko 2023-10-06T02:05:39.607 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc2945.ko 2023-10-06T02:05:39.607 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4151.ko 2023-10-06T02:05:39.607 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4215.ko 2023-10-06T02:05:39.607 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4222.ko 2023-10-06T02:05:39.608 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4245.ko 2023-10-06T02:05:39.608 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4260.ko 2023-10-06T02:05:39.608 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4261.ko 2023-10-06T02:05:39.608 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max16065.ko 2023-10-06T02:05:39.608 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max1619.ko 2023-10-06T02:05:39.608 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max1668.ko 2023-10-06T02:05:39.608 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max197.ko 2023-10-06T02:05:39.608 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6639.ko 2023-10-06T02:05:39.608 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6650.ko 2023-10-06T02:05:39.608 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6697.ko 2023-10-06T02:05:39.608 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/mcp3021.ko 2023-10-06T02:05:39.608 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6683.ko 2023-10-06T02:05:39.608 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6775-core.ko 2023-10-06T02:05:39.609 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6775.ko 2023-10-06T02:05:39.609 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct7802.ko 2023-10-06T02:05:39.609 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pc87360.ko 2023-10-06T02:05:39.609 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pc87427.ko 2023-10-06T02:05:39.609 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pcf8591.ko 2023-10-06T02:05:39.609 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus 2023-10-06T02:05:39.609 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-10-06T02:05:39.609 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-10-06T02:05:39.609 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-10-06T02:05:39.609 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max16064.ko 2023-10-06T02:05:39.609 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max34440.ko 2023-10-06T02:05:39.609 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max8688.ko 2023-10-06T02:05:39.610 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-10-06T02:05:39.610 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-10-06T02:05:39.610 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-10-06T02:05:39.610 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-10-06T02:05:39.610 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-10-06T02:05:39.610 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-10-06T02:05:39.610 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/powr1220.ko 2023-10-06T02:05:39.610 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch5627.ko 2023-10-06T02:05:39.610 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch5636.ko 2023-10-06T02:05:39.610 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch56xx-common.ko 2023-10-06T02:05:39.610 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sht15.ko 2023-10-06T02:05:39.610 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sht21.ko 2023-10-06T02:05:39.611 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/shtc1.ko 2023-10-06T02:05:39.611 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sis5595.ko 2023-10-06T02:05:39.611 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47b397.ko 2023-10-06T02:05:39.611 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47m1.ko 2023-10-06T02:05:39.611 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47m192.ko 2023-10-06T02:05:39.611 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/thmc50.ko 2023-10-06T02:05:39.611 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp102.ko 2023-10-06T02:05:39.611 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp103.ko 2023-10-06T02:05:39.611 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp401.ko 2023-10-06T02:05:39.611 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp421.ko 2023-10-06T02:05:39.611 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/via-cputemp.ko 2023-10-06T02:05:39.611 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/via686a.ko 2023-10-06T02:05:39.611 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/vt1211.ko 2023-10-06T02:05:39.612 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/vt8231.ko 2023-10-06T02:05:39.612 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83627ehf.ko 2023-10-06T02:05:39.612 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83627hf.ko 2023-10-06T02:05:39.612 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83781d.ko 2023-10-06T02:05:39.612 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83791d.ko 2023-10-06T02:05:39.612 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83792d.ko 2023-10-06T02:05:39.612 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83793.ko 2023-10-06T02:05:39.612 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83795.ko 2023-10-06T02:05:39.612 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83l785ts.ko 2023-10-06T02:05:39.612 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83l786ng.ko 2023-10-06T02:05:39.612 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c 2023-10-06T02:05:39.612 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos 2023-10-06T02:05:39.612 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-10-06T02:05:39.612 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-10-06T02:05:39.613 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses 2023-10-06T02:05:39.613 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-10-06T02:05:39.613 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-10-06T02:05:39.613 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-10-06T02:05:39.613 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-10-06T02:05:39.613 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-10-06T02:05:39.613 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-10-06T02:05:39.613 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-10-06T02:05:39.613 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-10-06T02:05:39.613 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-i801.ko 2023-10-06T02:05:39.613 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-isch.ko 2023-10-06T02:05:39.613 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-10-06T02:05:39.613 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-10-06T02:05:39.614 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-10-06T02:05:39.614 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-parport.ko 2023-10-06T02:05:39.614 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-10-06T02:05:39.614 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-10-06T02:05:39.614 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-10-06T02:05:39.614 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-10-06T02:05:39.614 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-10-06T02:05:39.614 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-10-06T02:05:39.614 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-via.ko 2023-10-06T02:05:39.614 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-10-06T02:05:39.614 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-10-06T02:05:39.614 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-dev.ko 2023-10-06T02:05:39.615 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-mux.ko 2023-10-06T02:05:39.615 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-10-06T02:05:39.615 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-smbus.ko 2023-10-06T02:05:39.615 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-stub.ko 2023-10-06T02:05:39.615 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband 2023-10-06T02:05:39.615 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core 2023-10-06T02:05:39.615 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_cm.ko 2023-10-06T02:05:39.615 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_core.ko 2023-10-06T02:05:39.615 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_umad.ko 2023-10-06T02:05:39.615 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-10-06T02:05:39.615 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/iw_cm.ko 2023-10-06T02:05:39.615 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/rdma_cm.ko 2023-10-06T02:05:39.616 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-10-06T02:05:39.616 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw 2023-10-06T02:05:39.616 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/cxgb4 2023-10-06T02:05:39.616 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-10-06T02:05:39.616 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx4 2023-10-06T02:05:39.616 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-10-06T02:05:39.616 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx5 2023-10-06T02:05:39.616 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-10-06T02:05:39.616 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mthca 2023-10-06T02:05:39.616 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-10-06T02:05:39.616 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp 2023-10-06T02:05:39.616 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/ipoib 2023-10-06T02:05:39.616 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-10-06T02:05:39.617 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/iser 2023-10-06T02:05:39.617 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-10-06T02:05:39.617 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/isert 2023-10-06T02:05:39.617 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-10-06T02:05:39.617 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srp 2023-10-06T02:05:39.617 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-10-06T02:05:39.617 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srpt 2023-10-06T02:05:39.617 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-10-06T02:05:39.617 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input 2023-10-06T02:05:39.617 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport 2023-10-06T02:05:39.617 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-10-06T02:05:39.617 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/fm801-gp.ko 2023-10-06T02:05:39.617 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/gameport.ko 2023-10-06T02:05:39.618 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/lightning.ko 2023-10-06T02:05:39.618 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/ns558.ko 2023-10-06T02:05:39.618 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/joydev.ko 2023-10-06T02:05:39.618 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard 2023-10-06T02:05:39.618 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/gpio_keys.ko 2023-10-06T02:05:39.618 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse 2023-10-06T02:05:39.618 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/appletouch.ko 2023-10-06T02:05:39.618 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/bcm5974.ko 2023-10-06T02:05:39.618 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/cyapatp.ko 2023-10-06T02:05:39.618 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/elan_i2c.ko 2023-10-06T02:05:39.618 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/sermouse.ko 2023-10-06T02:05:39.618 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-10-06T02:05:39.619 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/synaptics_usb.ko 2023-10-06T02:05:39.619 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/vsxxxaa.ko 2023-10-06T02:05:39.619 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/rmi4 2023-10-06T02:05:39.619 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/rmi4/rmi_core.ko 2023-10-06T02:05:39.619 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio 2023-10-06T02:05:39.619 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/altera_ps2.ko 2023-10-06T02:05:39.619 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/arc_ps2.ko 2023-10-06T02:05:39.619 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-10-06T02:05:39.619 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/serio_raw.ko 2023-10-06T02:05:39.619 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/sparse-keymap.ko 2023-10-06T02:05:39.619 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu 2023-10-06T02:05:39.619 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu/amd 2023-10-06T02:05:39.619 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu/amd/iommu_v2.ko 2023-10-06T02:05:39.619 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds 2023-10-06T02:05:39.620 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-blinkm.ko 2023-10-06T02:05:39.620 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-lm3530.ko 2023-10-06T02:05:39.620 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-lp3944.ko 2023-10-06T02:05:39.620 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-ss4200.ko 2023-10-06T02:05:39.620 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger 2023-10-06T02:05:39.620 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-10-06T02:05:39.620 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-10-06T02:05:39.620 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-10-06T02:05:39.620 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-10-06T02:05:39.620 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-10-06T02:05:39.620 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-10-06T02:05:39.620 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-10-06T02:05:39.620 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-10-06T02:05:39.621 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md 2023-10-06T02:05:39.621 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/bcache 2023-10-06T02:05:39.621 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/bcache/bcache.ko 2023-10-06T02:05:39.621 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-bio-prison.ko 2023-10-06T02:05:39.621 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-cache-smq.ko 2023-10-06T02:05:39.621 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-cache.ko 2023-10-06T02:05:39.621 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-crypt.ko 2023-10-06T02:05:39.621 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-delay.ko 2023-10-06T02:05:39.621 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-flakey.ko 2023-10-06T02:05:39.621 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-log-userspace.ko 2023-10-06T02:05:39.621 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-multipath.ko 2023-10-06T02:05:39.621 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-queue-length.ko 2023-10-06T02:05:39.621 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-raid.ko 2023-10-06T02:05:39.621 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-round-robin.ko 2023-10-06T02:05:39.622 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-service-time.ko 2023-10-06T02:05:39.622 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-switch.ko 2023-10-06T02:05:39.622 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-thin-pool.ko 2023-10-06T02:05:39.622 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-verity.ko 2023-10-06T02:05:39.622 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/faulty.ko 2023-10-06T02:05:39.622 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/linear.ko 2023-10-06T02:05:39.622 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/multipath.ko 2023-10-06T02:05:39.622 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/persistent-data 2023-10-06T02:05:39.622 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-10-06T02:05:39.622 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid0.ko 2023-10-06T02:05:39.622 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid1.ko 2023-10-06T02:05:39.622 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid10.ko 2023-10-06T02:05:39.623 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid456.ko 2023-10-06T02:05:39.623 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message 2023-10-06T02:05:39.623 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion 2023-10-06T02:05:39.623 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptbase.ko 2023-10-06T02:05:39.623 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptctl.ko 2023-10-06T02:05:39.623 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptfc.ko 2023-10-06T02:05:39.623 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptlan.ko 2023-10-06T02:05:39.623 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptsas.ko 2023-10-06T02:05:39.623 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptscsih.ko 2023-10-06T02:05:39.623 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptspi.ko 2023-10-06T02:05:39.623 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd 2023-10-06T02:05:39.623 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/lpc_ich.ko 2023-10-06T02:05:39.623 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/lpc_sch.ko 2023-10-06T02:05:39.624 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/mfd-core.ko 2023-10-06T02:05:39.624 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/sm501.ko 2023-10-06T02:05:39.624 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/viperboard.ko 2023-10-06T02:05:39.624 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/vx855.ko 2023-10-06T02:05:39.624 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/wl1273-core.ko 2023-10-06T02:05:39.624 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc 2023-10-06T02:05:39.624 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/altera-stapl 2023-10-06T02:05:39.624 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-10-06T02:05:39.624 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/apds9802als.ko 2023-10-06T02:05:39.624 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/apds990x.ko 2023-10-06T02:05:39.624 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/bh1770glc.ko 2023-10-06T02:05:39.624 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/cb710 2023-10-06T02:05:39.624 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/cb710/cb710.ko 2023-10-06T02:05:39.625 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom 2023-10-06T02:05:39.625 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/at24.ko 2023-10-06T02:05:39.625 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/eeprom.ko 2023-10-06T02:05:39.625 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-10-06T02:05:39.625 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/max6875.ko 2023-10-06T02:05:39.625 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/enclosure.ko 2023-10-06T02:05:39.625 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/hpilo.ko 2023-10-06T02:05:39.625 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/isl29003.ko 2023-10-06T02:05:39.625 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/isl29020.ko 2023-10-06T02:05:39.625 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d 2023-10-06T02:05:39.625 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-10-06T02:05:39.625 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-10-06T02:05:39.626 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei 2023-10-06T02:05:39.626 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei-me.ko 2023-10-06T02:05:39.626 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei-txe.ko 2023-10-06T02:05:39.626 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei.ko 2023-10-06T02:05:39.626 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-gru 2023-10-06T02:05:39.626 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-gru/gru.ko 2023-10-06T02:05:39.626 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp 2023-10-06T02:05:39.626 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp/xp.ko 2023-10-06T02:05:39.626 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp/xpc.ko 2023-10-06T02:05:39.626 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-10-06T02:05:39.626 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tifm_7xx1.ko 2023-10-06T02:05:39.626 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tifm_core.ko 2023-10-06T02:05:39.627 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tsl2550.ko 2023-10-06T02:05:39.627 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_balloon.ko 2023-10-06T02:05:39.627 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_vmci 2023-10-06T02:05:39.627 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-10-06T02:05:39.627 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net 2023-10-06T02:05:39.627 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/bonding 2023-10-06T02:05:39.627 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/bonding/bonding.ko 2023-10-06T02:05:39.627 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa 2023-10-06T02:05:39.627 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/b53 2023-10-06T02:05:39.627 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/b53/b53_common.ko 2023-10-06T02:05:39.627 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/bcm-sf2.ko 2023-10-06T02:05:39.627 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6060.ko 2023-10-06T02:05:39.628 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6xxx 2023-10-06T02:05:39.628 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-10-06T02:05:39.628 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dummy.ko 2023-10-06T02:05:39.628 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/eql.ko 2023-10-06T02:05:39.628 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet 2023-10-06T02:05:39.628 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com 2023-10-06T02:05:39.628 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-10-06T02:05:39.628 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-10-06T02:05:39.628 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-10-06T02:05:39.628 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-10-06T02:05:39.628 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390 2023-10-06T02:05:39.628 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/8390.ko 2023-10-06T02:05:39.629 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-10-06T02:05:39.629 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-10-06T02:05:39.629 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-10-06T02:05:39.629 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/adaptec 2023-10-06T02:05:39.629 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-10-06T02:05:39.629 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/agere 2023-10-06T02:05:39.629 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/agere/et131x.ko 2023-10-06T02:05:39.629 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/alteon 2023-10-06T02:05:39.629 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-10-06T02:05:39.629 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/altera 2023-10-06T02:05:39.629 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-10-06T02:05:39.629 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd 2023-10-06T02:05:39.629 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-10-06T02:05:39.630 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-10-06T02:05:39.630 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-10-06T02:05:39.630 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros 2023-10-06T02:05:39.630 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/alx 2023-10-06T02:05:39.630 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-10-06T02:05:39.630 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1c 2023-10-06T02:05:39.630 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-10-06T02:05:39.630 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1e 2023-10-06T02:05:39.630 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-10-06T02:05:39.630 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx 2023-10-06T02:05:39.630 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-10-06T02:05:39.630 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-10-06T02:05:39.631 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom 2023-10-06T02:05:39.631 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-10-06T02:05:39.631 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-10-06T02:05:39.631 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-10-06T02:05:39.631 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-10-06T02:05:39.631 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-10-06T02:05:39.631 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/genet 2023-10-06T02:05:39.631 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-10-06T02:05:39.631 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-10-06T02:05:39.631 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade 2023-10-06T02:05:39.631 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade/bna 2023-10-06T02:05:39.631 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-10-06T02:05:39.632 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio 2023-10-06T02:05:39.632 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb 2023-10-06T02:05:39.632 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-10-06T02:05:39.632 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-10-06T02:05:39.632 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-10-06T02:05:39.632 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-10-06T02:05:39.632 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-10-06T02:05:39.632 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-10-06T02:05:39.632 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-10-06T02:05:39.632 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-10-06T02:05:39.632 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-10-06T02:05:39.632 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco 2023-10-06T02:05:39.633 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco/enic 2023-10-06T02:05:39.633 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-10-06T02:05:39.633 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec 2023-10-06T02:05:39.633 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip 2023-10-06T02:05:39.633 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-10-06T02:05:39.633 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-10-06T02:05:39.633 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-10-06T02:05:39.633 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-10-06T02:05:39.633 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-10-06T02:05:39.633 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-10-06T02:05:39.633 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink 2023-10-06T02:05:39.633 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-10-06T02:05:39.634 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-10-06T02:05:39.634 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dnet.ko 2023-10-06T02:05:39.634 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex 2023-10-06T02:05:39.634 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex/benet 2023-10-06T02:05:39.634 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-10-06T02:05:39.634 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ethoc.ko 2023-10-06T02:05:39.634 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/fealnx.ko 2023-10-06T02:05:39.634 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel 2023-10-06T02:05:39.634 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e100.ko 2023-10-06T02:05:39.634 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000 2023-10-06T02:05:39.634 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-10-06T02:05:39.634 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000e 2023-10-06T02:05:39.635 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-10-06T02:05:39.635 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/fm10k 2023-10-06T02:05:39.635 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-10-06T02:05:39.635 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/i40e 2023-10-06T02:05:39.635 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-10-06T02:05:39.635 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/iavf 2023-10-06T02:05:39.635 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-10-06T02:05:39.635 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igb 2023-10-06T02:05:39.635 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-10-06T02:05:39.635 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igbvf 2023-10-06T02:05:39.635 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-10-06T02:05:39.635 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbe 2023-10-06T02:05:39.636 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-10-06T02:05:39.636 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbevf 2023-10-06T02:05:39.636 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-10-06T02:05:39.636 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/jme.ko 2023-10-06T02:05:39.636 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell 2023-10-06T02:05:39.636 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-10-06T02:05:39.636 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/skge.ko 2023-10-06T02:05:39.636 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-10-06T02:05:39.636 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox 2023-10-06T02:05:39.636 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4 2023-10-06T02:05:39.636 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-10-06T02:05:39.636 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-10-06T02:05:39.636 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5 2023-10-06T02:05:39.637 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-10-06T02:05:39.637 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-10-06T02:05:39.637 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/micrel 2023-10-06T02:05:39.637 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-10-06T02:05:39.637 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom 2023-10-06T02:05:39.637 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom/myri10ge 2023-10-06T02:05:39.637 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-10-06T02:05:39.637 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi 2023-10-06T02:05:39.637 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-10-06T02:05:39.637 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-10-06T02:05:39.637 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/neterion 2023-10-06T02:05:39.637 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-10-06T02:05:39.637 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/nvidia 2023-10-06T02:05:39.638 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-10-06T02:05:39.638 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines 2023-10-06T02:05:39.638 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-10-06T02:05:39.638 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-10-06T02:05:39.638 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic 2023-10-06T02:05:39.638 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/netxen 2023-10-06T02:05:39.638 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-10-06T02:05:39.638 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-10-06T02:05:39.638 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-10-06T02:05:39.638 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-10-06T02:05:39.638 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rdc 2023-10-06T02:05:39.638 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-10-06T02:05:39.639 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek 2023-10-06T02:05:39.639 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-10-06T02:05:39.639 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-10-06T02:05:39.639 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/atp.ko 2023-10-06T02:05:39.639 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-10-06T02:05:39.639 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rocker 2023-10-06T02:05:39.639 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-10-06T02:05:39.639 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sfc 2023-10-06T02:05:39.639 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-10-06T02:05:39.639 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/silan 2023-10-06T02:05:39.639 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-10-06T02:05:39.639 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis 2023-10-06T02:05:39.639 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis/sis190.ko 2023-10-06T02:05:39.640 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis/sis900.ko 2023-10-06T02:05:39.640 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc 2023-10-06T02:05:39.640 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-10-06T02:05:39.640 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-10-06T02:05:39.640 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-10-06T02:05:39.640 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-10-06T02:05:39.640 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro 2023-10-06T02:05:39.640 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac 2023-10-06T02:05:39.640 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-10-06T02:05:39.640 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-10-06T02:05:39.640 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun 2023-10-06T02:05:39.640 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/cassini.ko 2023-10-06T02:05:39.641 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/niu.ko 2023-10-06T02:05:39.641 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/sungem.ko 2023-10-06T02:05:39.641 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-10-06T02:05:39.641 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/tehuti 2023-10-06T02:05:39.641 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-10-06T02:05:39.641 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ti 2023-10-06T02:05:39.641 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ti/tlan.ko 2023-10-06T02:05:39.641 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via 2023-10-06T02:05:39.641 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-10-06T02:05:39.641 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-10-06T02:05:39.641 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet 2023-10-06T02:05:39.642 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-10-06T02:05:39.642 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-10-06T02:05:39.642 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/xircom 2023-10-06T02:05:39.642 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-10-06T02:05:39.642 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/geneve.ko 2023-10-06T02:05:39.642 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/hyperv 2023-10-06T02:05:39.642 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-10-06T02:05:39.642 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ieee802154 2023-10-06T02:05:39.642 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ieee802154/fakelb.ko 2023-10-06T02:05:39.642 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ifb.ko 2023-10-06T02:05:39.642 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ipvlan 2023-10-06T02:05:39.642 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ipvlan/ipvlan.ko 2023-10-06T02:05:39.643 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/macvlan.ko 2023-10-06T02:05:39.643 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/macvtap.ko 2023-10-06T02:05:39.643 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio 2023-10-06T02:05:39.643 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio.ko 2023-10-06T02:05:39.643 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-10-06T02:05:39.643 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-10-06T02:05:39.643 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-regmap.ko 2023-10-06T02:05:39.643 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mii.ko 2023-10-06T02:05:39.643 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/net_failover.ko 2023-10-06T02:05:39.643 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/netconsole.ko 2023-10-06T02:05:39.643 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/nlmon.ko 2023-10-06T02:05:39.643 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs 2023-10-06T02:05:39.643 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs/pcs-lynx.ko 2023-10-06T02:05:39.643 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-10-06T02:05:39.644 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy 2023-10-06T02:05:39.644 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/amd.ko 2023-10-06T02:05:39.644 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-10-06T02:05:39.644 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-10-06T02:05:39.644 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm7xxx.ko 2023-10-06T02:05:39.644 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm87xx.ko 2023-10-06T02:05:39.644 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/broadcom.ko 2023-10-06T02:05:39.644 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/cicada.ko 2023-10-06T02:05:39.644 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/davicom.ko 2023-10-06T02:05:39.644 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/dp83640.ko 2023-10-06T02:05:39.644 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/et1011c.ko 2023-10-06T02:05:39.644 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/icplus.ko 2023-10-06T02:05:39.644 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/lxt.ko 2023-10-06T02:05:39.645 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/marvell.ko 2023-10-06T02:05:39.645 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/micrel.ko 2023-10-06T02:05:39.645 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/national.ko 2023-10-06T02:05:39.645 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/phylink.ko 2023-10-06T02:05:39.645 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/qsemi.ko 2023-10-06T02:05:39.645 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/realtek.ko 2023-10-06T02:05:39.645 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/smsc.ko 2023-10-06T02:05:39.645 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/ste10Xp.ko 2023-10-06T02:05:39.645 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/vitesse.ko 2023-10-06T02:05:39.645 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp 2023-10-06T02:05:39.645 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/bsd_comp.ko 2023-10-06T02:05:39.645 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_async.ko 2023-10-06T02:05:39.645 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_deflate.ko 2023-10-06T02:05:39.646 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_generic.ko 2023-10-06T02:05:39.646 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_mppe.ko 2023-10-06T02:05:39.646 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_synctty.ko 2023-10-06T02:05:39.646 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pppoe.ko 2023-10-06T02:05:39.646 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pppox.ko 2023-10-06T02:05:39.646 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pptp.ko 2023-10-06T02:05:39.646 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip 2023-10-06T02:05:39.646 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip/slhc.ko 2023-10-06T02:05:39.646 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip/slip.ko 2023-10-06T02:05:39.646 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/sungem_phy.ko 2023-10-06T02:05:39.646 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/tap.ko 2023-10-06T02:05:39.646 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team 2023-10-06T02:05:39.647 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team.ko 2023-10-06T02:05:39.647 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_activebackup.ko 2023-10-06T02:05:39.647 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_broadcast.ko 2023-10-06T02:05:39.647 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-10-06T02:05:39.647 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_random.ko 2023-10-06T02:05:39.647 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-10-06T02:05:39.647 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/tun.ko 2023-10-06T02:05:39.647 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/veth.ko 2023-10-06T02:05:39.647 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/virtio_net.ko 2023-10-06T02:05:39.647 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vmxnet3 2023-10-06T02:05:39.647 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-10-06T02:05:39.647 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vxlan 2023-10-06T02:05:39.647 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vxlan/vxlan.ko 2023-10-06T02:05:39.648 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netback 2023-10-06T02:05:39.648 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netback/xen-netback.ko 2023-10-06T02:05:39.648 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netfront.ko 2023-10-06T02:05:39.648 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ntb 2023-10-06T02:05:39.648 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ntb/ntb.ko 2023-10-06T02:05:39.648 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme 2023-10-06T02:05:39.648 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host 2023-10-06T02:05:39.648 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host/nvme-core.ko 2023-10-06T02:05:39.648 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host/nvme.ko 2023-10-06T02:05:39.648 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport 2023-10-06T02:05:39.648 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport.ko 2023-10-06T02:05:39.648 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_cs.ko 2023-10-06T02:05:39.648 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_pc.ko 2023-10-06T02:05:39.649 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_serial.ko 2023-10-06T02:05:39.649 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci 2023-10-06T02:05:39.649 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/hotplug 2023-10-06T02:05:39.649 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-10-06T02:05:39.649 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/pcie 2023-10-06T02:05:39.649 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/pcie/aer_inject.ko 2023-10-06T02:05:39.649 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/xen-pcifront.ko 2023-10-06T02:05:39.649 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia 2023-10-06T02:05:39.649 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/i82092.ko 2023-10-06T02:05:39.649 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/pd6729.ko 2023-10-06T02:05:39.649 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/yenta_socket.ko 2023-10-06T02:05:39.649 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl 2023-10-06T02:05:39.649 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl/intel 2023-10-06T02:05:39.649 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-10-06T02:05:39.650 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform 2023-10-06T02:05:39.650 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86 2023-10-06T02:05:39.650 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/acer-wmi.ko 2023-10-06T02:05:39.650 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/acerhdf.ko 2023-10-06T02:05:39.650 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/apple-gmux.ko 2023-10-06T02:05:39.650 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-laptop.ko 2023-10-06T02:05:39.650 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-10-06T02:05:39.650 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-wmi.ko 2023-10-06T02:05:39.650 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/classmate-laptop.ko 2023-10-06T02:05:39.650 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-10-06T02:05:39.650 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-10-06T02:05:39.650 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-10-06T02:05:39.651 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-10-06T02:05:39.651 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/hdaps.ko 2023-10-06T02:05:39.651 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel 2023-10-06T02:05:39.651 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-10-06T02:05:39.651 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel_ips.ko 2023-10-06T02:05:39.651 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/msi-wmi.ko 2023-10-06T02:05:39.651 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/mxm-wmi.ko 2023-10-06T02:05:39.651 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-10-06T02:05:39.651 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/samsung-laptop.ko 2023-10-06T02:05:39.651 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/samsung-q10.ko 2023-10-06T02:05:39.651 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/topstar-laptop.ko 2023-10-06T02:05:39.651 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-10-06T02:05:39.651 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/toshiba_haps.ko 2023-10-06T02:05:39.651 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/wmi-bmof.ko 2023-10-06T02:05:39.652 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/wmi.ko 2023-10-06T02:05:39.652 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap 2023-10-06T02:05:39.652 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap/intel_rapl_common.ko 2023-10-06T02:05:39.652 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap/intel_rapl_msr.ko 2023-10-06T02:05:39.652 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps 2023-10-06T02:05:39.652 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients 2023-10-06T02:05:39.652 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps-gpio.ko 2023-10-06T02:05:39.652 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps-ldisc.ko 2023-10-06T02:05:39.652 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps_parport.ko 2023-10-06T02:05:39.652 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/pps_core.ko 2023-10-06T02:05:39.652 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp 2023-10-06T02:05:39.652 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp/ptp.ko 2023-10-06T02:05:39.652 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp/ptp_kvm.ko 2023-10-06T02:05:39.653 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc 2023-10-06T02:05:39.653 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-bq32k.ko 2023-10-06T02:05:39.653 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-bq4802.ko 2023-10-06T02:05:39.653 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1286.ko 2023-10-06T02:05:39.653 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1307.ko 2023-10-06T02:05:39.653 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1374.ko 2023-10-06T02:05:39.653 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1511.ko 2023-10-06T02:05:39.653 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1553.ko 2023-10-06T02:05:39.653 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1672.ko 2023-10-06T02:05:39.653 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1742.ko 2023-10-06T02:05:39.653 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds2404.ko 2023-10-06T02:05:39.653 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds3232.ko 2023-10-06T02:05:39.653 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-em3027.ko 2023-10-06T02:05:39.654 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-fm3130.ko 2023-10-06T02:05:39.654 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-isl12022.ko 2023-10-06T02:05:39.654 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-isl1208.ko 2023-10-06T02:05:39.654 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m41t80.ko 2023-10-06T02:05:39.654 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m48t35.ko 2023-10-06T02:05:39.654 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m48t59.ko 2023-10-06T02:05:39.654 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-max6900.ko 2023-10-06T02:05:39.654 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-msm6242.ko 2023-10-06T02:05:39.654 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf2127.ko 2023-10-06T02:05:39.654 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf85063.ko 2023-10-06T02:05:39.654 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8523.ko 2023-10-06T02:05:39.654 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8563.ko 2023-10-06T02:05:39.655 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8583.ko 2023-10-06T02:05:39.655 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rp5c01.ko 2023-10-06T02:05:39.655 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rs5c372.ko 2023-10-06T02:05:39.655 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-10-06T02:05:39.655 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rx8025.ko 2023-10-06T02:05:39.655 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rx8581.ko 2023-10-06T02:05:39.655 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-10-06T02:05:39.655 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-x1205.ko 2023-10-06T02:05:39.655 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi 2023-10-06T02:05:39.655 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-9xxx.ko 2023-10-06T02:05:39.655 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-sas.ko 2023-10-06T02:05:39.655 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-xxxx.ko 2023-10-06T02:05:39.655 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/BusLogic.ko 2023-10-06T02:05:39.656 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/a100u2w.ko 2023-10-06T02:05:39.656 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aacraid 2023-10-06T02:05:39.656 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aacraid/aacraid.ko 2023-10-06T02:05:39.656 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/advansys.ko 2023-10-06T02:05:39.656 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx 2023-10-06T02:05:39.656 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-10-06T02:05:39.656 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-10-06T02:05:39.656 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/am53c974.ko 2023-10-06T02:05:39.656 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/arcmsr 2023-10-06T02:05:39.656 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-10-06T02:05:39.656 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/atp870u.ko 2023-10-06T02:05:39.656 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/be2iscsi 2023-10-06T02:05:39.656 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-10-06T02:05:39.656 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bfa 2023-10-06T02:05:39.657 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bfa/bfa.ko 2023-10-06T02:05:39.657 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2fc 2023-10-06T02:05:39.657 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-10-06T02:05:39.657 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2i 2023-10-06T02:05:39.657 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-10-06T02:05:39.657 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ch.ko 2023-10-06T02:05:39.657 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/csiostor 2023-10-06T02:05:39.657 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/csiostor/csiostor.ko 2023-10-06T02:05:39.657 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi 2023-10-06T02:05:39.657 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb3i 2023-10-06T02:05:39.657 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-10-06T02:05:39.657 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb4i 2023-10-06T02:05:39.657 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-10-06T02:05:39.658 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-10-06T02:05:39.658 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/dc395x.ko 2023-10-06T02:05:39.658 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler 2023-10-06T02:05:39.658 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-10-06T02:05:39.658 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-10-06T02:05:39.658 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-10-06T02:05:39.658 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-10-06T02:05:39.658 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/dmx3191d.ko 2023-10-06T02:05:39.658 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esas2r 2023-10-06T02:05:39.658 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esas2r/esas2r.ko 2023-10-06T02:05:39.658 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esp_scsi.ko 2023-10-06T02:05:39.658 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe 2023-10-06T02:05:39.658 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe/fcoe.ko 2023-10-06T02:05:39.659 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-10-06T02:05:39.659 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fnic 2023-10-06T02:05:39.659 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fnic/fnic.ko 2023-10-06T02:05:39.659 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hpsa.ko 2023-10-06T02:05:39.659 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hptiop.ko 2023-10-06T02:05:39.659 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hv_storvsc.ko 2023-10-06T02:05:39.659 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/initio.ko 2023-10-06T02:05:39.659 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ipr.ko 2023-10-06T02:05:39.659 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ips.ko 2023-10-06T02:05:39.659 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/isci 2023-10-06T02:05:39.659 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/isci/isci.ko 2023-10-06T02:05:39.659 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-10-06T02:05:39.659 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/iscsi_tcp.ko 2023-10-06T02:05:39.659 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libfc 2023-10-06T02:05:39.660 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libfc/libfc.ko 2023-10-06T02:05:39.660 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libiscsi.ko 2023-10-06T02:05:39.660 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libiscsi_tcp.ko 2023-10-06T02:05:39.660 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libsas 2023-10-06T02:05:39.660 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libsas/libsas.ko 2023-10-06T02:05:39.660 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/lpfc 2023-10-06T02:05:39.660 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/lpfc/lpfc.ko 2023-10-06T02:05:39.660 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid 2023-10-06T02:05:39.660 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid.ko 2023-10-06T02:05:39.660 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-10-06T02:05:39.660 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-10-06T02:05:39.660 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-10-06T02:05:39.660 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mpt3sas 2023-10-06T02:05:39.661 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-10-06T02:05:39.661 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvsas 2023-10-06T02:05:39.661 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvsas/mvsas.ko 2023-10-06T02:05:39.661 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvumi.ko 2023-10-06T02:05:39.661 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pm8001 2023-10-06T02:05:39.661 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-10-06T02:05:39.661 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pmcraid.ko 2023-10-06T02:05:39.661 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla1280.ko 2023-10-06T02:05:39.661 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx 2023-10-06T02:05:39.661 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-10-06T02:05:39.661 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-10-06T02:05:39.661 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla4xxx 2023-10-06T02:05:39.661 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-10-06T02:05:39.662 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/raid_class.ko 2023-10-06T02:05:39.662 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_debug.ko 2023-10-06T02:05:39.662 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_fc.ko 2023-10-06T02:05:39.662 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-10-06T02:05:39.662 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_sas.ko 2023-10-06T02:05:39.662 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_spi.ko 2023-10-06T02:05:39.662 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_srp.ko 2023-10-06T02:05:39.662 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ses.ko 2023-10-06T02:05:39.662 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/st.ko 2023-10-06T02:05:39.662 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/stex.ko 2023-10-06T02:05:39.662 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/sym53c8xx_2 2023-10-06T02:05:39.663 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-10-06T02:05:39.663 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/virtio_scsi.ko 2023-10-06T02:05:39.663 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/vmw_pvscsi.ko 2023-10-06T02:05:39.663 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/wd719x.ko 2023-10-06T02:05:39.663 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ssb 2023-10-06T02:05:39.663 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ssb/ssb.ko 2023-10-06T02:05:39.663 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target 2023-10-06T02:05:39.663 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/iscsi 2023-10-06T02:05:39.663 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-10-06T02:05:39.663 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/loopback 2023-10-06T02:05:39.663 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/loopback/tcm_loop.ko 2023-10-06T02:05:39.663 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_file.ko 2023-10-06T02:05:39.663 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_iblock.ko 2023-10-06T02:05:39.664 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_mod.ko 2023-10-06T02:05:39.664 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_pscsi.ko 2023-10-06T02:05:39.664 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_user.ko 2023-10-06T02:05:39.664 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/tcm_fc 2023-10-06T02:05:39.664 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-10-06T02:05:39.664 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal 2023-10-06T02:05:39.664 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel 2023-10-06T02:05:39.664 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal 2023-10-06T02:05:39.664 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-10-06T02:05:39.664 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-10-06T02:05:39.664 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-10-06T02:05:39.664 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-10-06T02:05:39.665 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-10-06T02:05:39.665 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-10-06T02:05:39.665 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-10-06T02:05:39.665 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-10-06T02:05:39.665 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-10-06T02:05:39.665 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-10-06T02:05:39.665 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-10-06T02:05:39.665 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-10-06T02:05:39.665 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-10-06T02:05:39.665 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-10-06T02:05:39.665 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-10-06T02:05:39.665 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty 2023-10-06T02:05:39.666 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/ipwireless 2023-10-06T02:05:39.666 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-10-06T02:05:39.666 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/n_gsm.ko 2023-10-06T02:05:39.666 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/n_hdlc.ko 2023-10-06T02:05:39.666 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/nozomi.ko 2023-10-06T02:05:39.666 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial 2023-10-06T02:05:39.666 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/8250 2023-10-06T02:05:39.666 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-10-06T02:05:39.666 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/arc_uart.ko 2023-10-06T02:05:39.666 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/jsm 2023-10-06T02:05:39.666 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/jsm/jsm.ko 2023-10-06T02:05:39.666 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/synclink_gt.ko 2023-10-06T02:05:39.667 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs 2023-10-06T02:05:39.667 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/core 2023-10-06T02:05:39.667 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/core/ufshcd-core.ko 2023-10-06T02:05:39.667 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/host 2023-10-06T02:05:39.667 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-10-06T02:05:39.667 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio 2023-10-06T02:05:39.667 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio.ko 2023-10-06T02:05:39.667 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_aec.ko 2023-10-06T02:05:39.667 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_cif.ko 2023-10-06T02:05:39.667 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_pci_generic.ko 2023-10-06T02:05:39.667 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_sercos3.ko 2023-10-06T02:05:39.667 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb 2023-10-06T02:05:39.668 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class 2023-10-06T02:05:39.668 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/cdc-acm.ko 2023-10-06T02:05:39.668 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/cdc-wdm.ko 2023-10-06T02:05:39.668 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/usblp.ko 2023-10-06T02:05:39.668 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/usbtmc.ko 2023-10-06T02:05:39.668 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host 2023-10-06T02:05:39.668 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host/sl811-hcd.ko 2023-10-06T02:05:39.668 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image 2023-10-06T02:05:39.668 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image/mdc800.ko 2023-10-06T02:05:39.668 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image/microtek.ko 2023-10-06T02:05:39.668 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc 2023-10-06T02:05:39.668 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/adutux.ko 2023-10-06T02:05:39.668 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/appledisplay.ko 2023-10-06T02:05:39.668 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/emi26.ko 2023-10-06T02:05:39.669 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/emi62.ko 2023-10-06T02:05:39.669 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/ezusb.ko 2023-10-06T02:05:39.669 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/idmouse.ko 2023-10-06T02:05:39.669 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/iowarrior.ko 2023-10-06T02:05:39.669 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/isight_firmware.ko 2023-10-06T02:05:39.669 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/ldusb.ko 2023-10-06T02:05:39.669 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/legousbtower.ko 2023-10-06T02:05:39.669 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/sisusbvga 2023-10-06T02:05:39.669 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-10-06T02:05:39.669 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/trancevibrator.ko 2023-10-06T02:05:39.669 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usb3503.ko 2023-10-06T02:05:39.669 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usblcd.ko 2023-10-06T02:05:39.669 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usbsevseg.ko 2023-10-06T02:05:39.670 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/uss720.ko 2023-10-06T02:05:39.670 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/yurex.ko 2023-10-06T02:05:39.670 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/phy 2023-10-06T02:05:39.670 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/phy/phy-generic.ko 2023-10-06T02:05:39.670 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial 2023-10-06T02:05:39.670 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/aircable.ko 2023-10-06T02:05:39.670 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ark3116.ko 2023-10-06T02:05:39.670 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/belkin_sa.ko 2023-10-06T02:05:39.670 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ch341.ko 2023-10-06T02:05:39.670 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cp210x.ko 2023-10-06T02:05:39.670 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cyberjack.ko 2023-10-06T02:05:39.670 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cypress_m8.ko 2023-10-06T02:05:39.671 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/digi_acceleport.ko 2023-10-06T02:05:39.671 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/empeg.ko 2023-10-06T02:05:39.671 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ftdi_sio.ko 2023-10-06T02:05:39.671 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/garmin_gps.ko 2023-10-06T02:05:39.671 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/io_edgeport.ko 2023-10-06T02:05:39.671 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/io_ti.ko 2023-10-06T02:05:39.671 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ipaq.ko 2023-10-06T02:05:39.671 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ipw.ko 2023-10-06T02:05:39.671 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ir-usb.ko 2023-10-06T02:05:39.671 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-10-06T02:05:39.671 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/keyspan.ko 2023-10-06T02:05:39.671 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/keyspan_pda.ko 2023-10-06T02:05:39.671 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/kl5kusb105.ko 2023-10-06T02:05:39.672 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/kobil_sct.ko 2023-10-06T02:05:39.672 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mct_u232.ko 2023-10-06T02:05:39.672 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mos7720.ko 2023-10-06T02:05:39.672 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mos7840.ko 2023-10-06T02:05:39.672 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/navman.ko 2023-10-06T02:05:39.672 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/omninet.ko 2023-10-06T02:05:39.672 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/opticon.ko 2023-10-06T02:05:39.672 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/option.ko 2023-10-06T02:05:39.672 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/oti6858.ko 2023-10-06T02:05:39.672 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/pl2303.ko 2023-10-06T02:05:39.672 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/qcaux.ko 2023-10-06T02:05:39.672 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/qcserial.ko 2023-10-06T02:05:39.672 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/quatech2.ko 2023-10-06T02:05:39.673 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/safe_serial.ko 2023-10-06T02:05:39.673 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/sierra.ko 2023-10-06T02:05:39.673 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/spcp8x5.ko 2023-10-06T02:05:39.673 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ssu100.ko 2023-10-06T02:05:39.673 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/symbolserial.ko 2023-10-06T02:05:39.673 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-10-06T02:05:39.673 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-10-06T02:05:39.673 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb_debug.ko 2023-10-06T02:05:39.673 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb_wwan.ko 2023-10-06T02:05:39.673 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/visor.ko 2023-10-06T02:05:39.673 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/whiteheat.ko 2023-10-06T02:05:39.673 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/xsens_mt.ko 2023-10-06T02:05:39.673 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage 2023-10-06T02:05:39.674 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/uas.ko 2023-10-06T02:05:39.674 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-alauda.ko 2023-10-06T02:05:39.674 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-cypress.ko 2023-10-06T02:05:39.674 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-datafab.ko 2023-10-06T02:05:39.674 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-10-06T02:05:39.674 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-freecom.ko 2023-10-06T02:05:39.674 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-isd200.ko 2023-10-06T02:05:39.674 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-10-06T02:05:39.674 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-karma.ko 2023-10-06T02:05:39.674 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-onetouch.ko 2023-10-06T02:05:39.674 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-realtek.ko 2023-10-06T02:05:39.674 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-sddr09.ko 2023-10-06T02:05:39.675 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-sddr55.ko 2023-10-06T02:05:39.675 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-usbat.ko 2023-10-06T02:05:39.675 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/usb-storage.ko 2023-10-06T02:05:39.675 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip 2023-10-06T02:05:39.675 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/usbip-core.ko 2023-10-06T02:05:39.675 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/usbip-host.ko 2023-10-06T02:05:39.675 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-10-06T02:05:39.675 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio 2023-10-06T02:05:39.675 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci 2023-10-06T02:05:39.675 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-10-06T02:05:39.675 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci/vfio-pci.ko 2023-10-06T02:05:39.675 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/vfio.ko 2023-10-06T02:05:39.676 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-10-06T02:05:39.676 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost 2023-10-06T02:05:39.676 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost.ko 2023-10-06T02:05:39.676 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_iotlb.ko 2023-10-06T02:05:39.676 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_net.ko 2023-10-06T02:05:39.676 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_scsi.ko 2023-10-06T02:05:39.676 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video 2023-10-06T02:05:39.676 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight 2023-10-06T02:05:39.676 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/apple_bl.ko 2023-10-06T02:05:39.676 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/lcd.ko 2023-10-06T02:05:39.676 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/platform_lcd.ko 2023-10-06T02:05:39.676 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio 2023-10-06T02:05:39.676 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio.ko 2023-10-06T02:05:39.677 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_balloon.ko 2023-10-06T02:05:39.677 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_mmio.ko 2023-10-06T02:05:39.677 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_pci.ko 2023-10-06T02:05:39.677 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-10-06T02:05:39.677 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-10-06T02:05:39.677 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_ring.ko 2023-10-06T02:05:39.677 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1 2023-10-06T02:05:39.677 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters 2023-10-06T02:05:39.677 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters/ds2482.ko 2023-10-06T02:05:39.677 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters/ds2490.ko 2023-10-06T02:05:39.677 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves 2023-10-06T02:05:39.677 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-10-06T02:05:39.677 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-10-06T02:05:39.678 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-10-06T02:05:39.678 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-10-06T02:05:39.678 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-10-06T02:05:39.678 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-10-06T02:05:39.678 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-10-06T02:05:39.678 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-10-06T02:05:39.678 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-10-06T02:05:39.678 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_smem.ko 2023-10-06T02:05:39.678 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_therm.ko 2023-10-06T02:05:39.678 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/wire.ko 2023-10-06T02:05:39.678 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog 2023-10-06T02:05:39.678 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/alim1535_wdt.ko 2023-10-06T02:05:39.679 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/alim7101_wdt.ko 2023-10-06T02:05:39.679 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/f71808e_wdt.ko 2023-10-06T02:05:39.679 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/hpwdt.ko 2023-10-06T02:05:39.679 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/i6300esb.ko 2023-10-06T02:05:39.679 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-10-06T02:05:39.679 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/iTCO_wdt.ko 2023-10-06T02:05:39.679 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ib700wdt.ko 2023-10-06T02:05:39.679 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ibmasr.ko 2023-10-06T02:05:39.679 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-10-06T02:05:39.679 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/it8712f_wdt.ko 2023-10-06T02:05:39.679 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/it87_wdt.ko 2023-10-06T02:05:39.679 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/machzwd.ko 2023-10-06T02:05:39.680 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/nv_tco.ko 2023-10-06T02:05:39.680 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/pcwd_pci.ko 2023-10-06T02:05:39.680 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/pcwd_usb.ko 2023-10-06T02:05:39.680 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-10-06T02:05:39.680 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sch311x_wdt.ko 2023-10-06T02:05:39.680 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/softdog.ko 2023-10-06T02:05:39.680 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sp5100_tco.ko 2023-10-06T02:05:39.680 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/via_wdt.ko 2023-10-06T02:05:39.680 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-10-06T02:05:39.680 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83877f_wdt.ko 2023-10-06T02:05:39.680 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83977f_wdt.ko 2023-10-06T02:05:39.680 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/wdt_pci.ko 2023-10-06T02:05:39.680 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/xen_wdt.ko 2023-10-06T02:05:39.680 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen 2023-10-06T02:05:39.681 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-acpi-processor.ko 2023-10-06T02:05:39.681 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-evtchn.ko 2023-10-06T02:05:39.681 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-gntalloc.ko 2023-10-06T02:05:39.681 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-gntdev.ko 2023-10-06T02:05:39.681 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-pciback 2023-10-06T02:05:39.681 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-10-06T02:05:39.681 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-privcmd.ko 2023-10-06T02:05:39.681 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xenfs 2023-10-06T02:05:39.681 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xenfs/xenfs.ko 2023-10-06T02:05:39.681 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs 2023-10-06T02:05:39.681 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/9p 2023-10-06T02:05:39.681 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/9p/9p.ko 2023-10-06T02:05:39.681 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/affs 2023-10-06T02:05:39.682 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/affs/affs.ko 2023-10-06T02:05:39.682 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/befs 2023-10-06T02:05:39.682 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/befs/befs.ko 2023-10-06T02:05:39.682 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/binfmt_misc.ko 2023-10-06T02:05:39.682 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/btrfs 2023-10-06T02:05:39.682 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/btrfs/btrfs.ko 2023-10-06T02:05:39.682 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cachefiles 2023-10-06T02:05:39.682 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cachefiles/cachefiles.ko 2023-10-06T02:05:39.682 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ceph 2023-10-06T02:05:39.682 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ceph/ceph.ko 2023-10-06T02:05:39.682 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/coda 2023-10-06T02:05:39.682 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/coda/coda.ko 2023-10-06T02:05:39.683 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cramfs 2023-10-06T02:05:39.683 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cramfs/cramfs.ko 2023-10-06T02:05:39.683 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/dlm 2023-10-06T02:05:39.683 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/dlm/dlm.ko 2023-10-06T02:05:39.683 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ecryptfs 2023-10-06T02:05:39.683 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ecryptfs/ecryptfs.ko 2023-10-06T02:05:39.683 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/f2fs 2023-10-06T02:05:39.683 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/f2fs/f2fs.ko 2023-10-06T02:05:39.683 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat 2023-10-06T02:05:39.683 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/fat.ko 2023-10-06T02:05:39.683 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/msdos.ko 2023-10-06T02:05:39.683 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/vfat.ko 2023-10-06T02:05:39.683 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fscache 2023-10-06T02:05:39.684 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fscache/fscache.ko 2023-10-06T02:05:39.684 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse 2023-10-06T02:05:39.684 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse/cuse.ko 2023-10-06T02:05:39.684 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse/fuse.ko 2023-10-06T02:05:39.684 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/gfs2 2023-10-06T02:05:39.684 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/gfs2/gfs2.ko 2023-10-06T02:05:39.684 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfs 2023-10-06T02:05:39.684 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfs/hfs.ko 2023-10-06T02:05:39.684 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfsplus 2023-10-06T02:05:39.684 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfsplus/hfsplus.ko 2023-10-06T02:05:39.684 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/isofs 2023-10-06T02:05:39.684 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/isofs/isofs.ko 2023-10-06T02:05:39.684 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/jfs 2023-10-06T02:05:39.685 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/jfs/jfs.ko 2023-10-06T02:05:39.685 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/lockd 2023-10-06T02:05:39.685 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/lockd/lockd.ko 2023-10-06T02:05:39.685 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/minix 2023-10-06T02:05:39.685 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/minix/minix.ko 2023-10-06T02:05:39.685 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/netfs 2023-10-06T02:05:39.685 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/netfs/netfs.ko 2023-10-06T02:05:39.685 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs 2023-10-06T02:05:39.685 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/blocklayout 2023-10-06T02:05:39.685 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-10-06T02:05:39.685 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/filelayout 2023-10-06T02:05:39.685 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-10-06T02:05:39.686 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/flexfilelayout 2023-10-06T02:05:39.686 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-10-06T02:05:39.686 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfs.ko 2023-10-06T02:05:39.686 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfsv3.ko 2023-10-06T02:05:39.686 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfsv4.ko 2023-10-06T02:05:39.686 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common 2023-10-06T02:05:39.686 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common/grace.ko 2023-10-06T02:05:39.686 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common/nfs_acl.ko 2023-10-06T02:05:39.686 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfsd 2023-10-06T02:05:39.686 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfsd/nfsd.ko 2023-10-06T02:05:39.686 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nilfs2 2023-10-06T02:05:39.686 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nilfs2/nilfs2.ko 2023-10-06T02:05:39.687 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls 2023-10-06T02:05:39.687 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-celtic.ko 2023-10-06T02:05:39.687 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-centeuro.ko 2023-10-06T02:05:39.687 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-croatian.ko 2023-10-06T02:05:39.687 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-cyrillic.ko 2023-10-06T02:05:39.687 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-gaelic.ko 2023-10-06T02:05:39.687 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-greek.ko 2023-10-06T02:05:39.687 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-iceland.ko 2023-10-06T02:05:39.687 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-inuit.ko 2023-10-06T02:05:39.687 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-roman.ko 2023-10-06T02:05:39.687 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-romanian.ko 2023-10-06T02:05:39.687 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-turkish.ko 2023-10-06T02:05:39.687 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1250.ko 2023-10-06T02:05:39.688 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1251.ko 2023-10-06T02:05:39.688 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1255.ko 2023-10-06T02:05:39.688 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp737.ko 2023-10-06T02:05:39.688 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp775.ko 2023-10-06T02:05:39.688 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp850.ko 2023-10-06T02:05:39.688 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp852.ko 2023-10-06T02:05:39.688 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp855.ko 2023-10-06T02:05:39.688 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp857.ko 2023-10-06T02:05:39.688 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp860.ko 2023-10-06T02:05:39.688 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp861.ko 2023-10-06T02:05:39.688 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp862.ko 2023-10-06T02:05:39.688 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp863.ko 2023-10-06T02:05:39.689 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp864.ko 2023-10-06T02:05:39.689 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp865.ko 2023-10-06T02:05:39.689 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp866.ko 2023-10-06T02:05:39.689 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp869.ko 2023-10-06T02:05:39.689 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp874.ko 2023-10-06T02:05:39.689 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp932.ko 2023-10-06T02:05:39.689 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp936.ko 2023-10-06T02:05:39.689 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp949.ko 2023-10-06T02:05:39.689 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp950.ko 2023-10-06T02:05:39.689 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_euc-jp.ko 2023-10-06T02:05:39.689 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-1.ko 2023-10-06T02:05:39.689 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-13.ko 2023-10-06T02:05:39.690 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-14.ko 2023-10-06T02:05:39.690 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-15.ko 2023-10-06T02:05:39.690 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-2.ko 2023-10-06T02:05:39.690 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-3.ko 2023-10-06T02:05:39.690 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-4.ko 2023-10-06T02:05:39.690 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-5.ko 2023-10-06T02:05:39.690 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-6.ko 2023-10-06T02:05:39.690 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-7.ko 2023-10-06T02:05:39.690 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-9.ko 2023-10-06T02:05:39.690 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-r.ko 2023-10-06T02:05:39.690 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-ru.ko 2023-10-06T02:05:39.690 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-u.ko 2023-10-06T02:05:39.691 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_utf8.ko 2023-10-06T02:05:39.691 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2 2023-10-06T02:05:39.691 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/cluster 2023-10-06T02:05:39.691 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-10-06T02:05:39.691 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlm 2023-10-06T02:05:39.691 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-10-06T02:05:39.691 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlmfs 2023-10-06T02:05:39.691 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-10-06T02:05:39.691 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2.ko 2023-10-06T02:05:39.691 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-10-06T02:05:39.691 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-10-06T02:05:39.691 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-10-06T02:05:39.691 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/overlayfs 2023-10-06T02:05:39.692 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/overlayfs/overlay.ko 2023-10-06T02:05:39.692 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/pstore 2023-10-06T02:05:39.692 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/pstore/ramoops.ko 2023-10-06T02:05:39.692 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/reiserfs 2023-10-06T02:05:39.692 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/reiserfs/reiserfs.ko 2023-10-06T02:05:39.692 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/romfs 2023-10-06T02:05:39.692 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/romfs/romfs.ko 2023-10-06T02:05:39.692 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb 2023-10-06T02:05:39.692 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/client 2023-10-06T02:05:39.692 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/client/cifs.ko 2023-10-06T02:05:39.692 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common 2023-10-06T02:05:39.692 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common/cifs_arc4.ko 2023-10-06T02:05:39.692 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common/cifs_md4.ko 2023-10-06T02:05:39.693 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/squashfs 2023-10-06T02:05:39.693 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/squashfs/squashfs.ko 2023-10-06T02:05:39.693 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/sysv 2023-10-06T02:05:39.693 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/sysv/sysv.ko 2023-10-06T02:05:39.693 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/udf 2023-10-06T02:05:39.693 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/udf/udf.ko 2023-10-06T02:05:39.693 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ufs 2023-10-06T02:05:39.693 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ufs/ufs.ko 2023-10-06T02:05:39.693 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/xfs 2023-10-06T02:05:39.693 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/xfs/xfs.ko 2023-10-06T02:05:39.693 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel 2023-10-06T02:05:39.693 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/rcu 2023-10-06T02:05:39.693 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/rcu/rcutorture.ko 2023-10-06T02:05:39.694 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/torture.ko 2023-10-06T02:05:39.694 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/trace 2023-10-06T02:05:39.694 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-10-06T02:05:39.694 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib 2023-10-06T02:05:39.694 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/asn1_encoder.ko 2023-10-06T02:05:39.694 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crc-itu-t.ko 2023-10-06T02:05:39.694 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crc8.ko 2023-10-06T02:05:39.694 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto 2023-10-06T02:05:39.694 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto/libarc4.ko 2023-10-06T02:05:39.694 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto/libdes.ko 2023-10-06T02:05:39.694 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/libcrc32c.ko 2023-10-06T02:05:39.694 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lru_cache.ko 2023-10-06T02:05:39.695 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4 2023-10-06T02:05:39.695 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4/lz4_compress.ko 2023-10-06T02:05:39.695 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4/lz4hc_compress.ko 2023-10-06T02:05:39.695 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/math 2023-10-06T02:05:39.695 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/math/cordic.ko 2023-10-06T02:05:39.695 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/raid6 2023-10-06T02:05:39.695 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/raid6/raid6_pq.ko 2023-10-06T02:05:39.695 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/reed_solomon 2023-10-06T02:05:39.695 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/reed_solomon/reed_solomon.ko 2023-10-06T02:05:39.695 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_bm.ko 2023-10-06T02:05:39.695 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_fsm.ko 2023-10-06T02:05:39.695 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_kmp.ko 2023-10-06T02:05:39.696 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/zstd 2023-10-06T02:05:39.696 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/zstd/zstd_compress.ko 2023-10-06T02:05:39.696 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/mm 2023-10-06T02:05:39.696 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/mm/hwpoison-inject.ko 2023-10-06T02:05:39.696 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net 2023-10-06T02:05:39.696 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan 2023-10-06T02:05:39.696 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/6lowpan.ko 2023-10-06T02:05:39.696 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_dest.ko 2023-10-06T02:05:39.696 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_fragment.ko 2023-10-06T02:05:39.696 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_hop.ko 2023-10-06T02:05:39.696 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_ipv6.ko 2023-10-06T02:05:39.696 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_mobility.ko 2023-10-06T02:05:39.696 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_routing.ko 2023-10-06T02:05:39.697 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_udp.ko 2023-10-06T02:05:39.697 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802 2023-10-06T02:05:39.697 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/garp.ko 2023-10-06T02:05:39.697 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/mrp.ko 2023-10-06T02:05:39.697 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/p8022.ko 2023-10-06T02:05:39.697 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/psnap.ko 2023-10-06T02:05:39.697 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/stp.ko 2023-10-06T02:05:39.697 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/8021q 2023-10-06T02:05:39.697 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/8021q/8021q.ko 2023-10-06T02:05:39.697 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p 2023-10-06T02:05:39.697 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet.ko 2023-10-06T02:05:39.697 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_fd.ko 2023-10-06T02:05:39.697 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_rdma.ko 2023-10-06T02:05:39.698 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_virtio.ko 2023-10-06T02:05:39.698 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/batman-adv 2023-10-06T02:05:39.698 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/batman-adv/batman-adv.ko 2023-10-06T02:05:39.698 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge 2023-10-06T02:05:39.698 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/br_netfilter.ko 2023-10-06T02:05:39.698 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/bridge.ko 2023-10-06T02:05:39.698 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter 2023-10-06T02:05:39.698 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-10-06T02:05:39.698 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_among.ko 2023-10-06T02:05:39.698 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_arp.ko 2023-10-06T02:05:39.698 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-10-06T02:05:39.698 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-10-06T02:05:39.699 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_ip.ko 2023-10-06T02:05:39.699 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-10-06T02:05:39.699 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_limit.ko 2023-10-06T02:05:39.699 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_log.ko 2023-10-06T02:05:39.699 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_mark.ko 2023-10-06T02:05:39.699 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-10-06T02:05:39.699 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-10-06T02:05:39.699 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-10-06T02:05:39.699 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-10-06T02:05:39.699 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_snat.ko 2023-10-06T02:05:39.699 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_stp.ko 2023-10-06T02:05:39.699 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-10-06T02:05:39.699 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-10-06T02:05:39.700 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-10-06T02:05:39.700 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-10-06T02:05:39.700 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtables.ko 2023-10-06T02:05:39.700 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-10-06T02:05:39.700 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-10-06T02:05:39.700 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-10-06T02:05:39.700 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ceph 2023-10-06T02:05:39.700 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ceph/libceph.ko 2023-10-06T02:05:39.700 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core 2023-10-06T02:05:39.700 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core/failover.ko 2023-10-06T02:05:39.700 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core/pktgen.ko 2023-10-06T02:05:39.700 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp 2023-10-06T02:05:39.700 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp.ko 2023-10-06T02:05:39.701 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_diag.ko 2023-10-06T02:05:39.701 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_ipv4.ko 2023-10-06T02:05:39.701 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_ipv6.ko 2023-10-06T02:05:39.701 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dns_resolver 2023-10-06T02:05:39.701 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dns_resolver/dns_resolver.ko 2023-10-06T02:05:39.701 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa 2023-10-06T02:05:39.701 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/dsa_core.ko 2023-10-06T02:05:39.701 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_brcm.ko 2023-10-06T02:05:39.701 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_dsa.ko 2023-10-06T02:05:39.701 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_none.ko 2023-10-06T02:05:39.701 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_trailer.ko 2023-10-06T02:05:39.701 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154 2023-10-06T02:05:39.701 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/6lowpan 2023-10-06T02:05:39.702 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-10-06T02:05:39.702 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/ieee802154.ko 2023-10-06T02:05:39.702 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/ieee802154_socket.ko 2023-10-06T02:05:39.702 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4 2023-10-06T02:05:39.702 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ah4.ko 2023-10-06T02:05:39.702 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/esp4.ko 2023-10-06T02:05:39.702 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/fou.ko 2023-10-06T02:05:39.702 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/gre.ko 2023-10-06T02:05:39.702 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/inet_diag.ko 2023-10-06T02:05:39.702 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_gre.ko 2023-10-06T02:05:39.702 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_tunnel.ko 2023-10-06T02:05:39.702 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_vti.ko 2023-10-06T02:05:39.703 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ipcomp.ko 2023-10-06T02:05:39.703 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ipip.ko 2023-10-06T02:05:39.703 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter 2023-10-06T02:05:39.703 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arp_tables.ko 2023-10-06T02:05:39.703 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-10-06T02:05:39.703 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-10-06T02:05:39.703 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-10-06T02:05:39.703 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-10-06T02:05:39.703 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-10-06T02:05:39.703 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-10-06T02:05:39.703 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-10-06T02:05:39.703 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-10-06T02:05:39.703 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-10-06T02:05:39.703 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_security.ko 2023-10-06T02:05:39.704 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-10-06T02:05:39.704 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-10-06T02:05:39.704 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-10-06T02:05:39.704 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-10-06T02:05:39.704 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-10-06T02:05:39.704 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-10-06T02:05:39.704 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-10-06T02:05:39.704 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-10-06T02:05:39.704 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-10-06T02:05:39.704 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-10-06T02:05:39.704 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_bic.ko 2023-10-06T02:05:39.704 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_dctcp.ko 2023-10-06T02:05:39.705 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_diag.ko 2023-10-06T02:05:39.705 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_highspeed.ko 2023-10-06T02:05:39.705 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_htcp.ko 2023-10-06T02:05:39.705 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_hybla.ko 2023-10-06T02:05:39.705 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_illinois.ko 2023-10-06T02:05:39.705 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_lp.ko 2023-10-06T02:05:39.705 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_scalable.ko 2023-10-06T02:05:39.705 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_vegas.ko 2023-10-06T02:05:39.705 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_veno.ko 2023-10-06T02:05:39.705 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_westwood.ko 2023-10-06T02:05:39.705 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_yeah.ko 2023-10-06T02:05:39.705 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tunnel4.ko 2023-10-06T02:05:39.706 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/udp_diag.ko 2023-10-06T02:05:39.706 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/udp_tunnel.ko 2023-10-06T02:05:39.706 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/xfrm4_tunnel.ko 2023-10-06T02:05:39.706 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6 2023-10-06T02:05:39.706 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ah6.ko 2023-10-06T02:05:39.706 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/esp6.ko 2023-10-06T02:05:39.706 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/fou6.ko 2023-10-06T02:05:39.706 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_tunnel.ko 2023-10-06T02:05:39.706 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-10-06T02:05:39.706 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_vti.ko 2023-10-06T02:05:39.706 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ipcomp6.ko 2023-10-06T02:05:39.706 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter 2023-10-06T02:05:39.706 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-10-06T02:05:39.707 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-10-06T02:05:39.707 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-10-06T02:05:39.707 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-10-06T02:05:39.707 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-10-06T02:05:39.707 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-10-06T02:05:39.707 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-10-06T02:05:39.707 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-10-06T02:05:39.707 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-10-06T02:05:39.707 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-10-06T02:05:39.707 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-10-06T02:05:39.707 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-10-06T02:05:39.707 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-10-06T02:05:39.707 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-10-06T02:05:39.708 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-10-06T02:05:39.708 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-10-06T02:05:39.708 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-10-06T02:05:39.708 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-10-06T02:05:39.708 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-10-06T02:05:39.708 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-10-06T02:05:39.708 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-10-06T02:05:39.708 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-10-06T02:05:39.708 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-10-06T02:05:39.708 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-10-06T02:05:39.708 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/sit.ko 2023-10-06T02:05:39.708 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/tunnel6.ko 2023-10-06T02:05:39.709 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/xfrm6_tunnel.ko 2023-10-06T02:05:39.709 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/key 2023-10-06T02:05:39.709 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/key/af_key.ko 2023-10-06T02:05:39.709 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp 2023-10-06T02:05:39.709 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_core.ko 2023-10-06T02:05:39.709 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_debugfs.ko 2023-10-06T02:05:39.709 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_eth.ko 2023-10-06T02:05:39.709 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ip.ko 2023-10-06T02:05:39.709 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ip6.ko 2023-10-06T02:05:39.709 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_netlink.ko 2023-10-06T02:05:39.709 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ppp.ko 2023-10-06T02:05:39.709 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/llc 2023-10-06T02:05:39.709 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/llc/llc.ko 2023-10-06T02:05:39.710 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mac802154 2023-10-06T02:05:39.710 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mac802154/mac802154.ko 2023-10-06T02:05:39.710 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mpls 2023-10-06T02:05:39.710 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mpls/mpls_gso.ko 2023-10-06T02:05:39.710 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter 2023-10-06T02:05:39.710 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset 2023-10-06T02:05:39.710 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set.ko 2023-10-06T02:05:39.710 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-10-06T02:05:39.710 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-10-06T02:05:39.710 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-10-06T02:05:39.710 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-10-06T02:05:39.711 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-10-06T02:05:39.711 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-10-06T02:05:39.711 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-10-06T02:05:39.711 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-10-06T02:05:39.711 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-10-06T02:05:39.711 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-10-06T02:05:39.711 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-10-06T02:05:39.711 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-10-06T02:05:39.711 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-10-06T02:05:39.711 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-10-06T02:05:39.711 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-10-06T02:05:39.711 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs 2023-10-06T02:05:39.711 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs.ko 2023-10-06T02:05:39.711 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-10-06T02:05:39.712 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-10-06T02:05:39.712 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-10-06T02:05:39.712 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-10-06T02:05:39.712 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-10-06T02:05:39.712 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-10-06T02:05:39.712 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-10-06T02:05:39.712 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-10-06T02:05:39.712 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-10-06T02:05:39.712 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-10-06T02:05:39.712 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-10-06T02:05:39.712 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-10-06T02:05:39.712 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-10-06T02:05:39.713 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conncount.ko 2023-10-06T02:05:39.713 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack.ko 2023-10-06T02:05:39.713 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-10-06T02:05:39.713 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-10-06T02:05:39.713 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-10-06T02:05:39.713 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_h323.ko 2023-10-06T02:05:39.713 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_irc.ko 2023-10-06T02:05:39.713 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-10-06T02:05:39.713 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-10-06T02:05:39.713 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-10-06T02:05:39.713 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_sane.ko 2023-10-06T02:05:39.713 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_sip.ko 2023-10-06T02:05:39.714 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-10-06T02:05:39.714 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-10-06T02:05:39.714 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_dup_netdev.ko 2023-10-06T02:05:39.714 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_flow_table.ko 2023-10-06T02:05:39.714 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_flow_table_inet.ko 2023-10-06T02:05:39.714 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_log_syslog.ko 2023-10-06T02:05:39.714 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat.ko 2023-10-06T02:05:39.714 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_amanda.ko 2023-10-06T02:05:39.714 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_ftp.ko 2023-10-06T02:05:39.714 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_irc.ko 2023-10-06T02:05:39.714 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_sip.ko 2023-10-06T02:05:39.714 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_tftp.ko 2023-10-06T02:05:39.714 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_synproxy_core.ko 2023-10-06T02:05:39.715 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_tables.ko 2023-10-06T02:05:39.715 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink.ko 2023-10-06T02:05:39.715 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_acct.ko 2023-10-06T02:05:39.715 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_log.ko 2023-10-06T02:05:39.715 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_osf.ko 2023-10-06T02:05:39.715 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_queue.ko 2023-10-06T02:05:39.715 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_chain_nat.ko 2023-10-06T02:05:39.715 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_compat.ko 2023-10-06T02:05:39.715 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_ct.ko 2023-10-06T02:05:39.715 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_dup_netdev.ko 2023-10-06T02:05:39.715 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_fib.ko 2023-10-06T02:05:39.715 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_fwd_netdev.ko 2023-10-06T02:05:39.715 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_hash.ko 2023-10-06T02:05:39.716 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_limit.ko 2023-10-06T02:05:39.716 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_log.ko 2023-10-06T02:05:39.716 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_masq.ko 2023-10-06T02:05:39.716 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_nat.ko 2023-10-06T02:05:39.716 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_numgen.ko 2023-10-06T02:05:39.716 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_queue.ko 2023-10-06T02:05:39.716 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_quota.ko 2023-10-06T02:05:39.716 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_redir.ko 2023-10-06T02:05:39.716 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_reject.ko 2023-10-06T02:05:39.716 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_reject_inet.ko 2023-10-06T02:05:39.716 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_socket.ko 2023-10-06T02:05:39.716 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_tunnel.ko 2023-10-06T02:05:39.717 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_AUDIT.ko 2023-10-06T02:05:39.717 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CHECKSUM.ko 2023-10-06T02:05:39.717 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CLASSIFY.ko 2023-10-06T02:05:39.717 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-10-06T02:05:39.717 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CT.ko 2023-10-06T02:05:39.717 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_DSCP.ko 2023-10-06T02:05:39.717 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_HL.ko 2023-10-06T02:05:39.717 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_HMARK.ko 2023-10-06T02:05:39.717 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_IDLETIMER.ko 2023-10-06T02:05:39.717 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_LED.ko 2023-10-06T02:05:39.717 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_LOG.ko 2023-10-06T02:05:39.717 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_MASQUERADE.ko 2023-10-06T02:05:39.717 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NETMAP.ko 2023-10-06T02:05:39.717 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NFLOG.ko 2023-10-06T02:05:39.718 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NFQUEUE.ko 2023-10-06T02:05:39.718 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_RATEEST.ko 2023-10-06T02:05:39.718 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_REDIRECT.ko 2023-10-06T02:05:39.718 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_SECMARK.ko 2023-10-06T02:05:39.718 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TCPMSS.ko 2023-10-06T02:05:39.718 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-10-06T02:05:39.718 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TEE.ko 2023-10-06T02:05:39.718 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TPROXY.ko 2023-10-06T02:05:39.718 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TRACE.ko 2023-10-06T02:05:39.718 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_addrtype.ko 2023-10-06T02:05:39.718 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_bpf.ko 2023-10-06T02:05:39.719 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cgroup.ko 2023-10-06T02:05:39.719 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cluster.ko 2023-10-06T02:05:39.719 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_comment.ko 2023-10-06T02:05:39.719 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connbytes.ko 2023-10-06T02:05:39.719 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connlabel.ko 2023-10-06T02:05:39.719 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connlimit.ko 2023-10-06T02:05:39.719 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connmark.ko 2023-10-06T02:05:39.719 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_conntrack.ko 2023-10-06T02:05:39.719 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cpu.ko 2023-10-06T02:05:39.719 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_dccp.ko 2023-10-06T02:05:39.719 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_devgroup.ko 2023-10-06T02:05:39.719 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_dscp.ko 2023-10-06T02:05:39.719 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ecn.ko 2023-10-06T02:05:39.720 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_esp.ko 2023-10-06T02:05:39.720 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_hashlimit.ko 2023-10-06T02:05:39.720 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_helper.ko 2023-10-06T02:05:39.720 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_hl.ko 2023-10-06T02:05:39.720 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ipcomp.ko 2023-10-06T02:05:39.720 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_iprange.ko 2023-10-06T02:05:39.720 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ipvs.ko 2023-10-06T02:05:39.720 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_l2tp.ko 2023-10-06T02:05:39.720 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_length.ko 2023-10-06T02:05:39.720 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_limit.ko 2023-10-06T02:05:39.720 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_mac.ko 2023-10-06T02:05:39.720 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_mark.ko 2023-10-06T02:05:39.721 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_multiport.ko 2023-10-06T02:05:39.721 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_nat.ko 2023-10-06T02:05:39.721 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_nfacct.ko 2023-10-06T02:05:39.721 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_osf.ko 2023-10-06T02:05:39.721 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_owner.ko 2023-10-06T02:05:39.721 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_physdev.ko 2023-10-06T02:05:39.721 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_pkttype.ko 2023-10-06T02:05:39.721 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_policy.ko 2023-10-06T02:05:39.721 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_quota.ko 2023-10-06T02:05:39.721 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_rateest.ko 2023-10-06T02:05:39.721 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_realm.ko 2023-10-06T02:05:39.721 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_recent.ko 2023-10-06T02:05:39.721 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_sctp.ko 2023-10-06T02:05:39.722 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_set.ko 2023-10-06T02:05:39.722 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_socket.ko 2023-10-06T02:05:39.722 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_state.ko 2023-10-06T02:05:39.722 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_statistic.ko 2023-10-06T02:05:39.722 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_string.ko 2023-10-06T02:05:39.722 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_tcpmss.ko 2023-10-06T02:05:39.722 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_time.ko 2023-10-06T02:05:39.722 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_u32.ko 2023-10-06T02:05:39.722 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netlink 2023-10-06T02:05:39.722 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netlink/netlink_diag.ko 2023-10-06T02:05:39.722 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/nsh 2023-10-06T02:05:39.722 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/nsh/nsh.ko 2023-10-06T02:05:39.723 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch 2023-10-06T02:05:39.723 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/openvswitch.ko 2023-10-06T02:05:39.723 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-geneve.ko 2023-10-06T02:05:39.723 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-gre.ko 2023-10-06T02:05:39.723 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-vxlan.ko 2023-10-06T02:05:39.723 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/packet 2023-10-06T02:05:39.723 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/packet/af_packet_diag.ko 2023-10-06T02:05:39.723 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds 2023-10-06T02:05:39.723 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds.ko 2023-10-06T02:05:39.723 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds_rdma.ko 2023-10-06T02:05:39.723 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds_tcp.ko 2023-10-06T02:05:39.723 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched 2023-10-06T02:05:39.723 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_csum.ko 2023-10-06T02:05:39.724 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_gact.ko 2023-10-06T02:05:39.724 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_ipt.ko 2023-10-06T02:05:39.724 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_mirred.ko 2023-10-06T02:05:39.724 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_nat.ko 2023-10-06T02:05:39.724 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_pedit.ko 2023-10-06T02:05:39.724 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_police.ko 2023-10-06T02:05:39.724 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_simple.ko 2023-10-06T02:05:39.724 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_skbedit.ko 2023-10-06T02:05:39.724 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_vlan.ko 2023-10-06T02:05:39.724 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_basic.ko 2023-10-06T02:05:39.724 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_bpf.ko 2023-10-06T02:05:39.724 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_flow.ko 2023-10-06T02:05:39.724 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_fw.ko 2023-10-06T02:05:39.725 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_route.ko 2023-10-06T02:05:39.725 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_u32.ko 2023-10-06T02:05:39.725 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_cmp.ko 2023-10-06T02:05:39.725 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_ipset.ko 2023-10-06T02:05:39.725 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_meta.ko 2023-10-06T02:05:39.725 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_nbyte.ko 2023-10-06T02:05:39.725 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_text.ko 2023-10-06T02:05:39.725 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_u32.ko 2023-10-06T02:05:39.725 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_choke.ko 2023-10-06T02:05:39.725 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_codel.ko 2023-10-06T02:05:39.725 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_drr.ko 2023-10-06T02:05:39.725 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_fq.ko 2023-10-06T02:05:39.726 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_gred.ko 2023-10-06T02:05:39.726 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_hfsc.ko 2023-10-06T02:05:39.726 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_hhf.ko 2023-10-06T02:05:39.726 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_htb.ko 2023-10-06T02:05:39.726 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_ingress.ko 2023-10-06T02:05:39.726 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_mqprio.ko 2023-10-06T02:05:39.726 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_mqprio_lib.ko 2023-10-06T02:05:39.726 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_multiq.ko 2023-10-06T02:05:39.726 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_netem.ko 2023-10-06T02:05:39.726 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_pie.ko 2023-10-06T02:05:39.726 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_plug.ko 2023-10-06T02:05:39.726 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_prio.ko 2023-10-06T02:05:39.727 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_qfq.ko 2023-10-06T02:05:39.727 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_red.ko 2023-10-06T02:05:39.727 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_sfb.ko 2023-10-06T02:05:39.727 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_sfq.ko 2023-10-06T02:05:39.727 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_tbf.ko 2023-10-06T02:05:39.727 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_teql.ko 2023-10-06T02:05:39.727 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp 2023-10-06T02:05:39.727 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp/sctp.ko 2023-10-06T02:05:39.727 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp/sctp_diag.ko 2023-10-06T02:05:39.727 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc 2023-10-06T02:05:39.727 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss 2023-10-06T02:05:39.727 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-10-06T02:05:39.727 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-10-06T02:05:39.727 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/sunrpc.ko 2023-10-06T02:05:39.728 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/xprtrdma 2023-10-06T02:05:39.728 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-10-06T02:05:39.728 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc 2023-10-06T02:05:39.728 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc/diag.ko 2023-10-06T02:05:39.728 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc/tipc.ko 2023-10-06T02:05:39.728 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/unix 2023-10-06T02:05:39.728 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/unix/unix_diag.ko 2023-10-06T02:05:39.728 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock 2023-10-06T02:05:39.728 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-10-06T02:05:39.728 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-10-06T02:05:39.728 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock.ko 2023-10-06T02:05:39.728 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock_diag.ko 2023-10-06T02:05:39.729 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock_loopback.ko 2023-10-06T02:05:39.729 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/xfrm 2023-10-06T02:05:39.729 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/xfrm/xfrm_ipcomp.ko 2023-10-06T02:05:39.729 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security 2023-10-06T02:05:39.729 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys 2023-10-06T02:05:39.729 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/encrypted-keys 2023-10-06T02:05:39.729 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-10-06T02:05:39.729 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/trusted-keys 2023-10-06T02:05:39.729 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/trusted-keys/trusted.ko 2023-10-06T02:05:39.729 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt 2023-10-06T02:05:39.729 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt/lib 2023-10-06T02:05:39.729 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt/lib/irqbypass.ko 2023-10-06T02:05:39.729 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.alias 2023-10-06T02:05:39.730 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.alias.bin 2023-10-06T02:05:39.730 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin 2023-10-06T02:05:39.730 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin.bin 2023-10-06T02:05:39.730 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin.modinfo 2023-10-06T02:05:39.730 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.dep 2023-10-06T02:05:39.730 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.dep.bin 2023-10-06T02:05:39.730 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.devname 2023-10-06T02:05:39.730 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.order 2023-10-06T02:05:39.730 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.softdep 2023-10-06T02:05:39.730 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.symbols 2023-10-06T02:05:39.730 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.symbols.bin 2023-10-06T02:05:39.731 DEBUG:teuthology.task.kernel:get_image_version: 6.5.0-rc7-gfef64f93da2b 2023-10-06T02:05:39.731 DEBUG:teuthology.orchestra.run.smithi062:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-10-06T02:05:39.754 INFO:teuthology.orchestra.run.smithi062.stdout:postinstall scriptlet (using /bin/sh): 2023-10-06T02:05:39.754 INFO:teuthology.orchestra.run.smithi062.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b -a -r /boot/System.map-6.5.0-rc7-gfef64f93da2b ]; then 2023-10-06T02:05:39.754 INFO:teuthology.orchestra.run.smithi062.stdout:cp /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b /boot/.vmlinuz-6.5.0-rc7-gfef64f93da2b-rpm 2023-10-06T02:05:39.754 INFO:teuthology.orchestra.run.smithi062.stdout:cp /boot/System.map-6.5.0-rc7-gfef64f93da2b /boot/.System.map-6.5.0-rc7-gfef64f93da2b-rpm 2023-10-06T02:05:39.754 INFO:teuthology.orchestra.run.smithi062.stdout:rm -f /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b /boot/System.map-6.5.0-rc7-gfef64f93da2b 2023-10-06T02:05:39.754 INFO:teuthology.orchestra.run.smithi062.stdout:/sbin/installkernel 6.5.0-rc7-gfef64f93da2b /boot/.vmlinuz-6.5.0-rc7-gfef64f93da2b-rpm /boot/.System.map-6.5.0-rc7-gfef64f93da2b-rpm 2023-10-06T02:05:39.754 INFO:teuthology.orchestra.run.smithi062.stdout:rm -f /boot/.vmlinuz-6.5.0-rc7-gfef64f93da2b-rpm /boot/.System.map-6.5.0-rc7-gfef64f93da2b-rpm 2023-10-06T02:05:39.754 INFO:teuthology.orchestra.run.smithi062.stdout:fi 2023-10-06T02:05:39.754 INFO:teuthology.orchestra.run.smithi062.stdout:preuninstall scriptlet (using /bin/sh): 2023-10-06T02:05:39.755 INFO:teuthology.orchestra.run.smithi062.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-10-06T02:05:39.755 INFO:teuthology.orchestra.run.smithi062.stdout:new-kernel-pkg --remove 6.5.0-rc7-gfef64f93da2b --rminitrd --initrdfile=/boot/initramfs-6.5.0-rc7-gfef64f93da2b.img 2023-10-06T02:05:39.755 INFO:teuthology.orchestra.run.smithi062.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-10-06T02:05:39.755 INFO:teuthology.orchestra.run.smithi062.stdout:kernel-install remove 6.5.0-rc7-gfef64f93da2b 2023-10-06T02:05:39.755 INFO:teuthology.orchestra.run.smithi062.stdout:fi 2023-10-06T02:05:39.755 INFO:teuthology.orchestra.run.smithi062.stdout:postuninstall scriptlet (using /bin/sh): 2023-10-06T02:05:39.755 INFO:teuthology.orchestra.run.smithi062.stdout:if [ -x /sbin/update-bootloader ]; then 2023-10-06T02:05:39.755 INFO:teuthology.orchestra.run.smithi062.stdout:/sbin/update-bootloader --remove 6.5.0-rc7-gfef64f93da2b 2023-10-06T02:05:39.755 INFO:teuthology.orchestra.run.smithi062.stdout:fi 2023-10-06T02:05:39.756 DEBUG:teuthology.orchestra.run.smithi062:> sudo rpm -qi grub2-tools 2023-10-06T02:05:39.831 INFO:teuthology.orchestra.run.smithi062.stdout:Name : grub2-tools 2023-10-06T02:05:39.831 INFO:teuthology.orchestra.run.smithi062.stdout:Epoch : 1 2023-10-06T02:05:39.832 INFO:teuthology.orchestra.run.smithi062.stdout:Version : 2.02 2023-10-06T02:05:39.832 INFO:teuthology.orchestra.run.smithi062.stdout:Release : 99.el8_4.1 2023-10-06T02:05:39.832 INFO:teuthology.orchestra.run.smithi062.stdout:Architecture: x86_64 2023-10-06T02:05:39.832 INFO:teuthology.orchestra.run.smithi062.stdout:Install Date: Wed 21 Dec 2022 01:02:32 PM UTC 2023-10-06T02:05:39.832 INFO:teuthology.orchestra.run.smithi062.stdout:Group : System Environment/Base 2023-10-06T02:05:39.832 INFO:teuthology.orchestra.run.smithi062.stdout:Size : 9264631 2023-10-06T02:05:39.832 INFO:teuthology.orchestra.run.smithi062.stdout:License : GPLv3+ 2023-10-06T02:05:39.832 INFO:teuthology.orchestra.run.smithi062.stdout:Signature : RSA/SHA256, Thu 09 Sep 2021 09:41:55 AM UTC, Key ID 199e2f91fd431d51 2023-10-06T02:05:39.832 INFO:teuthology.orchestra.run.smithi062.stdout:Source RPM : grub2-2.02-99.el8_4.1.src.rpm 2023-10-06T02:05:39.833 INFO:teuthology.orchestra.run.smithi062.stdout:Build Date : Tue 18 May 2021 07:46:34 AM UTC 2023-10-06T02:05:39.833 INFO:teuthology.orchestra.run.smithi062.stdout:Build Host : x86-vm-07.build.eng.bos.redhat.com 2023-10-06T02:05:39.833 INFO:teuthology.orchestra.run.smithi062.stdout:Relocations : (not relocatable) 2023-10-06T02:05:39.833 INFO:teuthology.orchestra.run.smithi062.stdout:Packager : Red Hat, Inc. 2023-10-06T02:05:39.833 INFO:teuthology.orchestra.run.smithi062.stdout:Vendor : Red Hat, Inc. 2023-10-06T02:05:39.833 INFO:teuthology.orchestra.run.smithi062.stdout:URL : http://www.gnu.org/software/grub/ 2023-10-06T02:05:39.833 INFO:teuthology.orchestra.run.smithi062.stdout:Summary : Support tools for GRUB. 2023-10-06T02:05:39.833 INFO:teuthology.orchestra.run.smithi062.stdout:Description : 2023-10-06T02:05:39.833 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-10-06T02:05:39.833 INFO:teuthology.orchestra.run.smithi062.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-10-06T02:05:39.833 INFO:teuthology.orchestra.run.smithi062.stdout:customizable bootloader with modular architecture. It supports a rich 2023-10-06T02:05:39.833 INFO:teuthology.orchestra.run.smithi062.stdout:variety of kernel formats, file systems, computer architectures and 2023-10-06T02:05:39.834 INFO:teuthology.orchestra.run.smithi062.stdout:hardware devices. 2023-10-06T02:05:39.834 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-10-06T02:05:39.834 INFO:teuthology.orchestra.run.smithi062.stdout:This subpackage provides tools for support of all platforms. 2023-10-06T02:05:39.835 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-10-06T02:05:39.835 INFO:teuthology.task.kernel:Updating grub on smithi062 to boot 6.5.0-rc7-gfef64f93da2b 2023-10-06T02:05:39.835 DEBUG:teuthology.orchestra.run.smithi062:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-10-06T02:05:44.913 INFO:teuthology.orchestra.run.smithi062.stderr:Generating grub configuration file ... 2023-10-06T02:05:47.086 INFO:teuthology.orchestra.run.smithi062.stderr:done 2023-10-06T02:05:47.088 DEBUG:teuthology.orchestra.run.smithi062:> mktemp 2023-10-06T02:05:47.118 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/tmp.7ww4JjV8Pa 2023-10-06T02:05:47.118 DEBUG:teuthology.orchestra.run.smithi062:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.7ww4JjV8Pa 2023-10-06T02:05:47.190 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 0666 /tmp/tmp.7ww4JjV8Pa 2023-10-06T02:05:47.348 DEBUG:teuthology.orchestra.remote:smithi062:/tmp/tmp.7ww4JjV8Pa is 6KB 2023-10-06T02:05:47.396 DEBUG:teuthology.orchestra.run.smithi062:> rm -fr /tmp/tmp.7ww4JjV8Pa 2023-10-06T02:05:47.410 DEBUG:teuthology.orchestra.run.smithi062:> sudo /bin/ls /boot/loader/entries || true 2023-10-06T02:05:47.488 INFO:teuthology.orchestra.run.smithi062.stdout:298e228f3b5e401e8ac0f5ae159a2e06-0-rescue.conf 2023-10-06T02:05:47.488 INFO:teuthology.orchestra.run.smithi062.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-305.25.1.el8_4.x86_64.conf 2023-10-06T02:05:47.488 INFO:teuthology.orchestra.run.smithi062.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-425.3.1.el8.x86_64.conf 2023-10-06T02:05:47.488 INFO:teuthology.orchestra.run.smithi062.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-80.el8.x86_64.conf 2023-10-06T02:05:47.488 INFO:teuthology.orchestra.run.smithi062.stdout:298e228f3b5e401e8ac0f5ae159a2e06-6.5.0-rc7-gfef64f93da2b.conf 2023-10-06T02:05:47.489 DEBUG:teuthology.orchestra.run.smithi062:> sudo grub2-set-default 298e228f3b5e401e8ac0f5ae159a2e06-6.5.0-rc7-gfef64f93da2b 2023-10-06T02:05:47.668 DEBUG:teuthology.orchestra.run.smithi062:> sudo shutdown -r now 2023-10-06T02:05:57.620 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi031.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-10-06T02:05:57.621 DEBUG:teuthology.orchestra.run.smithi031:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-10-06T02:05:57.675 INFO:teuthology.orchestra.run.smithi031.stdout:/boot/System.map-6.5.0-rc7-gfef64f93da2b 2023-10-06T02:05:57.676 INFO:teuthology.orchestra.run.smithi031.stdout:/boot/config-6.5.0-rc7-gfef64f93da2b 2023-10-06T02:05:57.676 INFO:teuthology.orchestra.run.smithi031.stdout:/boot/vmlinuz-6.5.0-rc7-gfef64f93da2b 2023-10-06T02:05:57.676 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b 2023-10-06T02:05:57.676 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel 2023-10-06T02:05:57.676 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch 2023-10-06T02:05:57.676 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86 2023-10-06T02:05:57.676 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto 2023-10-06T02:05:57.676 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-10-06T02:05:57.676 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-10-06T02:05:57.676 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-10-06T02:05:57.676 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-10-06T02:05:57.677 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-10-06T02:05:57.677 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-10-06T02:05:57.677 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-10-06T02:05:57.677 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crc32c-intel.ko 2023-10-06T02:05:57.677 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-10-06T02:05:57.677 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-10-06T02:05:57.677 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-10-06T02:05:57.677 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-10-06T02:05:57.677 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-avx2.ko 2023-10-06T02:05:57.677 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-10-06T02:05:57.677 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-10-06T02:05:57.677 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-10-06T02:05:57.678 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-10-06T02:05:57.678 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-10-06T02:05:57.678 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-10-06T02:05:57.678 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-10-06T02:05:57.678 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel 2023-10-06T02:05:57.678 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu 2023-10-06T02:05:57.678 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu/mce 2023-10-06T02:05:57.678 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-10-06T02:05:57.678 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm 2023-10-06T02:05:57.678 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm-amd.ko 2023-10-06T02:05:57.678 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm-intel.ko 2023-10-06T02:05:57.679 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm.ko 2023-10-06T02:05:57.679 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto 2023-10-06T02:05:57.679 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/ansi_cprng.ko 2023-10-06T02:05:57.679 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/anubis.ko 2023-10-06T02:05:57.679 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/arc4.ko 2023-10-06T02:05:57.679 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx 2023-10-06T02:05:57.679 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_memcpy.ko 2023-10-06T02:05:57.679 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_pq.ko 2023-10-06T02:05:57.679 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_raid6_recov.ko 2023-10-06T02:05:57.679 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_tx.ko 2023-10-06T02:05:57.679 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_xor.ko 2023-10-06T02:05:57.680 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/raid6test.ko 2023-10-06T02:05:57.680 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/authenc.ko 2023-10-06T02:05:57.680 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/authencesn.ko 2023-10-06T02:05:57.680 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blake2b_generic.ko 2023-10-06T02:05:57.680 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blowfish_common.ko 2023-10-06T02:05:57.680 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blowfish_generic.ko 2023-10-06T02:05:57.680 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/camellia_generic.ko 2023-10-06T02:05:57.680 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast5_generic.ko 2023-10-06T02:05:57.680 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast6_generic.ko 2023-10-06T02:05:57.680 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast_common.ko 2023-10-06T02:05:57.680 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/ccm.ko 2023-10-06T02:05:57.680 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cmac.ko 2023-10-06T02:05:57.681 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crc32_generic.ko 2023-10-06T02:05:57.681 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crypto_engine.ko 2023-10-06T02:05:57.681 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crypto_user.ko 2023-10-06T02:05:57.681 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cts.ko 2023-10-06T02:05:57.681 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/des_generic.ko 2023-10-06T02:05:57.681 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/dh_generic.ko 2023-10-06T02:05:57.681 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/echainiv.ko 2023-10-06T02:05:57.681 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/essiv.ko 2023-10-06T02:05:57.681 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/fcrypt.ko 2023-10-06T02:05:57.681 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/khazad.ko 2023-10-06T02:05:57.681 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/lz4.ko 2023-10-06T02:05:57.681 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/lz4hc.ko 2023-10-06T02:05:57.682 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/md4.ko 2023-10-06T02:05:57.682 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/michael_mic.ko 2023-10-06T02:05:57.682 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/pcbc.ko 2023-10-06T02:05:57.682 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/pcrypt.ko 2023-10-06T02:05:57.682 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/rmd160.ko 2023-10-06T02:05:57.682 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/seed.ko 2023-10-06T02:05:57.682 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/serpent_generic.ko 2023-10-06T02:05:57.682 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/tcrypt.ko 2023-10-06T02:05:57.682 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/tea.ko 2023-10-06T02:05:57.682 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/twofish_common.ko 2023-10-06T02:05:57.682 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/twofish_generic.ko 2023-10-06T02:05:57.682 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/vmac.ko 2023-10-06T02:05:57.683 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/wp512.ko 2023-10-06T02:05:57.683 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xcbc.ko 2023-10-06T02:05:57.683 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xor.ko 2023-10-06T02:05:57.683 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xxhash_generic.ko 2023-10-06T02:05:57.683 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers 2023-10-06T02:05:57.683 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi 2023-10-06T02:05:57.683 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/acpi_ipmi.ko 2023-10-06T02:05:57.683 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/acpi_pad.ko 2023-10-06T02:05:57.683 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/custom_method.ko 2023-10-06T02:05:57.683 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/ec_sys.ko 2023-10-06T02:05:57.683 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/platform_profile.ko 2023-10-06T02:05:57.683 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/sbs.ko 2023-10-06T02:05:57.684 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/sbshc.ko 2023-10-06T02:05:57.684 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/video.ko 2023-10-06T02:05:57.684 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata 2023-10-06T02:05:57.684 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/acard-ahci.ko 2023-10-06T02:05:57.684 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/ahci_platform.ko 2023-10-06T02:05:57.684 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/ata_generic.ko 2023-10-06T02:05:57.684 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/libahci_platform.ko 2023-10-06T02:05:57.684 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_acpi.ko 2023-10-06T02:05:57.684 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ali.ko 2023-10-06T02:05:57.684 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_amd.ko 2023-10-06T02:05:57.684 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_artop.ko 2023-10-06T02:05:57.684 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_atiixp.ko 2023-10-06T02:05:57.685 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_atp867x.ko 2023-10-06T02:05:57.685 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cmd640.ko 2023-10-06T02:05:57.685 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cmd64x.ko 2023-10-06T02:05:57.685 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cypress.ko 2023-10-06T02:05:57.685 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_efar.ko 2023-10-06T02:05:57.685 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt366.ko 2023-10-06T02:05:57.685 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt37x.ko 2023-10-06T02:05:57.685 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt3x2n.ko 2023-10-06T02:05:57.685 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt3x3.ko 2023-10-06T02:05:57.685 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_it8213.ko 2023-10-06T02:05:57.685 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_it821x.ko 2023-10-06T02:05:57.685 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_jmicron.ko 2023-10-06T02:05:57.686 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_marvell.ko 2023-10-06T02:05:57.686 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_mpiix.ko 2023-10-06T02:05:57.686 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_netcell.ko 2023-10-06T02:05:57.686 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ninja32.ko 2023-10-06T02:05:57.686 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ns87410.ko 2023-10-06T02:05:57.686 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ns87415.ko 2023-10-06T02:05:57.686 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_oldpiix.ko 2023-10-06T02:05:57.686 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_opti.ko 2023-10-06T02:05:57.686 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_optidma.ko 2023-10-06T02:05:57.686 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pcmcia.ko 2023-10-06T02:05:57.686 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pdc2027x.ko 2023-10-06T02:05:57.686 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-10-06T02:05:57.687 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_piccolo.ko 2023-10-06T02:05:57.687 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_rdc.ko 2023-10-06T02:05:57.687 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sch.ko 2023-10-06T02:05:57.687 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_serverworks.ko 2023-10-06T02:05:57.687 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sil680.ko 2023-10-06T02:05:57.687 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sis.ko 2023-10-06T02:05:57.687 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sl82c105.ko 2023-10-06T02:05:57.687 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_triflex.ko 2023-10-06T02:05:57.687 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_via.ko 2023-10-06T02:05:57.687 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pdc_adma.ko 2023-10-06T02:05:57.687 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_inic162x.ko 2023-10-06T02:05:57.687 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_mv.ko 2023-10-06T02:05:57.688 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_nv.ko 2023-10-06T02:05:57.688 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_promise.ko 2023-10-06T02:05:57.688 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_qstor.ko 2023-10-06T02:05:57.688 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sil.ko 2023-10-06T02:05:57.688 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sil24.ko 2023-10-06T02:05:57.688 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sis.ko 2023-10-06T02:05:57.688 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_svw.ko 2023-10-06T02:05:57.688 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sx4.ko 2023-10-06T02:05:57.688 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_uli.ko 2023-10-06T02:05:57.688 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_via.ko 2023-10-06T02:05:57.688 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_vsc.ko 2023-10-06T02:05:57.688 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/auxdisplay 2023-10-06T02:05:57.688 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/auxdisplay/ks0108.ko 2023-10-06T02:05:57.689 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base 2023-10-06T02:05:57.689 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base/regmap 2023-10-06T02:05:57.689 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base/regmap/regmap-i2c.ko 2023-10-06T02:05:57.689 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base/regmap/regmap-mmio.ko 2023-10-06T02:05:57.689 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/bcma 2023-10-06T02:05:57.689 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/bcma/bcma.ko 2023-10-06T02:05:57.689 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block 2023-10-06T02:05:57.689 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/aoe 2023-10-06T02:05:57.689 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/aoe/aoe.ko 2023-10-06T02:05:57.689 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/brd.ko 2023-10-06T02:05:57.689 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/drbd 2023-10-06T02:05:57.689 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/drbd/drbd.ko 2023-10-06T02:05:57.690 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/floppy.ko 2023-10-06T02:05:57.690 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/loop.ko 2023-10-06T02:05:57.690 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/mtip32xx 2023-10-06T02:05:57.690 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-10-06T02:05:57.690 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/nbd.ko 2023-10-06T02:05:57.690 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/null_blk 2023-10-06T02:05:57.690 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/null_blk/null_blk.ko 2023-10-06T02:05:57.690 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/pktcdvd.ko 2023-10-06T02:05:57.690 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/rbd.ko 2023-10-06T02:05:57.690 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/virtio_blk.ko 2023-10-06T02:05:57.690 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkback 2023-10-06T02:05:57.690 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-10-06T02:05:57.691 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkfront.ko 2023-10-06T02:05:57.691 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/zram 2023-10-06T02:05:57.691 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/zram/zram.ko 2023-10-06T02:05:57.691 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char 2023-10-06T02:05:57.691 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hangcheck-timer.ko 2023-10-06T02:05:57.691 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random 2023-10-06T02:05:57.691 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/amd-rng.ko 2023-10-06T02:05:57.691 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/intel-rng.ko 2023-10-06T02:05:57.691 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-10-06T02:05:57.691 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/via-rng.ko 2023-10-06T02:05:57.691 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/virtio-rng.ko 2023-10-06T02:05:57.691 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi 2023-10-06T02:05:57.691 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-10-06T02:05:57.692 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-10-06T02:05:57.692 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-10-06T02:05:57.692 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_si.ko 2023-10-06T02:05:57.692 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-10-06T02:05:57.692 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-10-06T02:05:57.692 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/lp.ko 2023-10-06T02:05:57.692 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/mwave 2023-10-06T02:05:57.692 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/mwave/mwave.ko 2023-10-06T02:05:57.692 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ppdev.ko 2023-10-06T02:05:57.692 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tlclk.ko 2023-10-06T02:05:57.692 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm 2023-10-06T02:05:57.692 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm.ko 2023-10-06T02:05:57.693 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_atmel.ko 2023-10-06T02:05:57.693 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_infineon.ko 2023-10-06T02:05:57.693 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_nsc.ko 2023-10-06T02:05:57.693 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_tis.ko 2023-10-06T02:05:57.693 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-10-06T02:05:57.693 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/uv_mmtimer.ko 2023-10-06T02:05:57.693 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/virtio_console.ko 2023-10-06T02:05:57.693 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq 2023-10-06T02:05:57.693 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-10-06T02:05:57.693 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-10-06T02:05:57.693 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/p4-clockmod.ko 2023-10-06T02:05:57.693 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-10-06T02:05:57.693 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/powernow-k8.ko 2023-10-06T02:05:57.694 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/speedstep-lib.ko 2023-10-06T02:05:57.694 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto 2023-10-06T02:05:57.694 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp 2023-10-06T02:05:57.694 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-10-06T02:05:57.694 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp/ccp.ko 2023-10-06T02:05:57.694 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel 2023-10-06T02:05:57.694 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat 2023-10-06T02:05:57.694 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_common 2023-10-06T02:05:57.694 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-10-06T02:05:57.694 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-10-06T02:05:57.694 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-10-06T02:05:57.694 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/padlock-aes.ko 2023-10-06T02:05:57.695 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/padlock-sha.ko 2023-10-06T02:05:57.695 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/virtio 2023-10-06T02:05:57.695 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-10-06T02:05:57.695 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dca 2023-10-06T02:05:57.695 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dca/dca.ko 2023-10-06T02:05:57.695 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/devfreq 2023-10-06T02:05:57.695 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-10-06T02:05:57.695 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma 2023-10-06T02:05:57.695 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/dw 2023-10-06T02:05:57.695 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/dw/dw_dmac.ko 2023-10-06T02:05:57.695 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/ioat 2023-10-06T02:05:57.695 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/ioat/ioatdma.ko 2023-10-06T02:05:57.696 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac 2023-10-06T02:05:57.696 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/amd64_edac.ko 2023-10-06T02:05:57.696 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/e752x_edac.ko 2023-10-06T02:05:57.696 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/edac_mce_amd.ko 2023-10-06T02:05:57.696 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i3000_edac.ko 2023-10-06T02:05:57.696 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i3200_edac.ko 2023-10-06T02:05:57.696 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i5100_edac.ko 2023-10-06T02:05:57.696 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i5400_edac.ko 2023-10-06T02:05:57.696 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i7300_edac.ko 2023-10-06T02:05:57.696 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i7core_edac.ko 2023-10-06T02:05:57.696 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i82975x_edac.ko 2023-10-06T02:05:57.696 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/ie31200_edac.ko 2023-10-06T02:05:57.697 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/sb_edac.ko 2023-10-06T02:05:57.697 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/x38_edac.ko 2023-10-06T02:05:57.697 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware 2023-10-06T02:05:57.697 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware/edd.ko 2023-10-06T02:05:57.697 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware/iscsi_ibft.ko 2023-10-06T02:05:57.697 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio 2023-10-06T02:05:57.697 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-ich.ko 2023-10-06T02:05:57.697 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-viperboard.ko 2023-10-06T02:05:57.697 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid 2023-10-06T02:05:57.697 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-appleir.ko 2023-10-06T02:05:57.697 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-aureal.ko 2023-10-06T02:05:57.697 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-axff.ko 2023-10-06T02:05:57.697 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-dr.ko 2023-10-06T02:05:57.698 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-elecom.ko 2023-10-06T02:05:57.698 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-elo.ko 2023-10-06T02:05:57.698 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-emsff.ko 2023-10-06T02:05:57.698 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gaff.ko 2023-10-06T02:05:57.698 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gt683r.ko 2023-10-06T02:05:57.698 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gyration.ko 2023-10-06T02:05:57.698 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtek-kbd.ko 2023-10-06T02:05:57.698 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtek-mouse.ko 2023-10-06T02:05:57.698 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtekff.ko 2023-10-06T02:05:57.698 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-hyperv.ko 2023-10-06T02:05:57.698 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-icade.ko 2023-10-06T02:05:57.698 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-keytouch.ko 2023-10-06T02:05:57.699 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-kye.ko 2023-10-06T02:05:57.699 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-lcpower.ko 2023-10-06T02:05:57.699 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-led.ko 2023-10-06T02:05:57.699 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-lenovo.ko 2023-10-06T02:05:57.699 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-logitech-dj.ko 2023-10-06T02:05:57.699 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-10-06T02:05:57.699 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-multitouch.ko 2023-10-06T02:05:57.699 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-ortek.ko 2023-10-06T02:05:57.699 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-penmount.ko 2023-10-06T02:05:57.699 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-petalynx.ko 2023-10-06T02:05:57.699 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-picolcd.ko 2023-10-06T02:05:57.699 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-pl.ko 2023-10-06T02:05:57.699 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-plantronics.ko 2023-10-06T02:05:57.700 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-primax.ko 2023-10-06T02:05:57.700 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-rmi.ko 2023-10-06T02:05:57.700 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-arvo.ko 2023-10-06T02:05:57.700 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-common.ko 2023-10-06T02:05:57.700 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-isku.ko 2023-10-06T02:05:57.700 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-kone.ko 2023-10-06T02:05:57.700 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-10-06T02:05:57.700 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-konepure.ko 2023-10-06T02:05:57.700 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-10-06T02:05:57.700 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-lua.ko 2023-10-06T02:05:57.700 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-pyra.ko 2023-10-06T02:05:57.700 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-ryos.ko 2023-10-06T02:05:57.701 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-savu.ko 2023-10-06T02:05:57.701 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat.ko 2023-10-06T02:05:57.701 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-saitek.ko 2023-10-06T02:05:57.701 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-samsung.ko 2023-10-06T02:05:57.701 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sensor-hub.ko 2023-10-06T02:05:57.701 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sjoy.ko 2023-10-06T02:05:57.701 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sony.ko 2023-10-06T02:05:57.701 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-speedlink.ko 2023-10-06T02:05:57.701 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-steelseries.ko 2023-10-06T02:05:57.701 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sunplus.ko 2023-10-06T02:05:57.701 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-thrustmaster.ko 2023-10-06T02:05:57.701 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-tivo.ko 2023-10-06T02:05:57.701 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-tmff.ko 2023-10-06T02:05:57.702 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-topseed.ko 2023-10-06T02:05:57.702 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-twinhan.ko 2023-10-06T02:05:57.702 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-uclogic.ko 2023-10-06T02:05:57.702 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-waltop.ko 2023-10-06T02:05:57.702 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-wiimote.ko 2023-10-06T02:05:57.702 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-xinmo.ko 2023-10-06T02:05:57.702 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-zpff.ko 2023-10-06T02:05:57.702 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-zydacron.ko 2023-10-06T02:05:57.702 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/uhid.ko 2023-10-06T02:05:57.702 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/wacom.ko 2023-10-06T02:05:57.702 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv 2023-10-06T02:05:57.702 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_balloon.ko 2023-10-06T02:05:57.703 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_utils.ko 2023-10-06T02:05:57.703 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_vmbus.ko 2023-10-06T02:05:57.703 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon 2023-10-06T02:05:57.703 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/abituguru.ko 2023-10-06T02:05:57.703 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/abituguru3.ko 2023-10-06T02:05:57.703 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/acpi_power_meter.ko 2023-10-06T02:05:57.703 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ad7414.ko 2023-10-06T02:05:57.703 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ad7418.ko 2023-10-06T02:05:57.703 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adc128d818.ko 2023-10-06T02:05:57.703 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1025.ko 2023-10-06T02:05:57.703 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1026.ko 2023-10-06T02:05:57.703 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1029.ko 2023-10-06T02:05:57.703 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1031.ko 2023-10-06T02:05:57.704 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm9240.ko 2023-10-06T02:05:57.704 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ads7828.ko 2023-10-06T02:05:57.704 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7410.ko 2023-10-06T02:05:57.704 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7411.ko 2023-10-06T02:05:57.704 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7462.ko 2023-10-06T02:05:57.704 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7470.ko 2023-10-06T02:05:57.704 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7475.ko 2023-10-06T02:05:57.704 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7x10.ko 2023-10-06T02:05:57.704 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/amc6821.ko 2023-10-06T02:05:57.704 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/applesmc.ko 2023-10-06T02:05:57.704 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asb100.ko 2023-10-06T02:05:57.704 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asc7621.ko 2023-10-06T02:05:57.704 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asus_atk0110.ko 2023-10-06T02:05:57.705 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/atxp1.ko 2023-10-06T02:05:57.705 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/coretemp.ko 2023-10-06T02:05:57.705 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-10-06T02:05:57.705 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/dme1737.ko 2023-10-06T02:05:57.705 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ds1621.ko 2023-10-06T02:05:57.705 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ds620.ko 2023-10-06T02:05:57.705 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/emc1403.ko 2023-10-06T02:05:57.705 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/emc6w201.ko 2023-10-06T02:05:57.705 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f71805f.ko 2023-10-06T02:05:57.705 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f71882fg.ko 2023-10-06T02:05:57.705 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f75375s.ko 2023-10-06T02:05:57.705 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/fam15h_power.ko 2023-10-06T02:05:57.706 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/fschmd.ko 2023-10-06T02:05:57.706 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/g760a.ko 2023-10-06T02:05:57.706 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/g762.ko 2023-10-06T02:05:57.706 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/gl518sm.ko 2023-10-06T02:05:57.706 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/gl520sm.ko 2023-10-06T02:05:57.706 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/hwmon-vid.ko 2023-10-06T02:05:57.706 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/i5500_temp.ko 2023-10-06T02:05:57.706 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/i5k_amb.ko 2023-10-06T02:05:57.706 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ibmaem.ko 2023-10-06T02:05:57.706 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ibmpex.ko 2023-10-06T02:05:57.706 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ina209.ko 2023-10-06T02:05:57.706 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ina2xx.ko 2023-10-06T02:05:57.707 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/it87.ko 2023-10-06T02:05:57.707 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/k10temp.ko 2023-10-06T02:05:57.707 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/k8temp.ko 2023-10-06T02:05:57.707 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lineage-pem.ko 2023-10-06T02:05:57.707 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm63.ko 2023-10-06T02:05:57.707 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm73.ko 2023-10-06T02:05:57.707 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm75.ko 2023-10-06T02:05:57.707 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm77.ko 2023-10-06T02:05:57.707 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm78.ko 2023-10-06T02:05:57.707 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm80.ko 2023-10-06T02:05:57.707 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm83.ko 2023-10-06T02:05:57.707 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm85.ko 2023-10-06T02:05:57.708 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm87.ko 2023-10-06T02:05:57.708 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm90.ko 2023-10-06T02:05:57.708 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm92.ko 2023-10-06T02:05:57.708 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm93.ko 2023-10-06T02:05:57.708 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95234.ko 2023-10-06T02:05:57.708 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95241.ko 2023-10-06T02:05:57.708 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95245.ko 2023-10-06T02:05:57.708 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc2945.ko 2023-10-06T02:05:57.708 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4151.ko 2023-10-06T02:05:57.708 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4215.ko 2023-10-06T02:05:57.708 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4222.ko 2023-10-06T02:05:57.708 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4245.ko 2023-10-06T02:05:57.709 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4260.ko 2023-10-06T02:05:57.709 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4261.ko 2023-10-06T02:05:57.709 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max16065.ko 2023-10-06T02:05:57.709 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max1619.ko 2023-10-06T02:05:57.709 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max1668.ko 2023-10-06T02:05:57.709 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max197.ko 2023-10-06T02:05:57.709 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6639.ko 2023-10-06T02:05:57.709 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6650.ko 2023-10-06T02:05:57.709 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6697.ko 2023-10-06T02:05:57.709 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/mcp3021.ko 2023-10-06T02:05:57.709 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6683.ko 2023-10-06T02:05:57.709 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6775-core.ko 2023-10-06T02:05:57.710 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6775.ko 2023-10-06T02:05:57.710 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct7802.ko 2023-10-06T02:05:57.710 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pc87360.ko 2023-10-06T02:05:57.710 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pc87427.ko 2023-10-06T02:05:57.710 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pcf8591.ko 2023-10-06T02:05:57.710 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus 2023-10-06T02:05:57.710 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-10-06T02:05:57.710 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-10-06T02:05:57.710 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-10-06T02:05:57.710 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max16064.ko 2023-10-06T02:05:57.710 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max34440.ko 2023-10-06T02:05:57.710 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max8688.ko 2023-10-06T02:05:57.711 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-10-06T02:05:57.711 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-10-06T02:05:57.711 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-10-06T02:05:57.711 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-10-06T02:05:57.711 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-10-06T02:05:57.711 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-10-06T02:05:57.711 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/powr1220.ko 2023-10-06T02:05:57.711 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch5627.ko 2023-10-06T02:05:57.711 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch5636.ko 2023-10-06T02:05:57.711 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch56xx-common.ko 2023-10-06T02:05:57.711 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sht15.ko 2023-10-06T02:05:57.711 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sht21.ko 2023-10-06T02:05:57.712 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/shtc1.ko 2023-10-06T02:05:57.712 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sis5595.ko 2023-10-06T02:05:57.712 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47b397.ko 2023-10-06T02:05:57.712 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47m1.ko 2023-10-06T02:05:57.712 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47m192.ko 2023-10-06T02:05:57.712 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/thmc50.ko 2023-10-06T02:05:57.712 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp102.ko 2023-10-06T02:05:57.712 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp103.ko 2023-10-06T02:05:57.712 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp401.ko 2023-10-06T02:05:57.712 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp421.ko 2023-10-06T02:05:57.712 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/via-cputemp.ko 2023-10-06T02:05:57.712 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/via686a.ko 2023-10-06T02:05:57.713 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/vt1211.ko 2023-10-06T02:05:57.713 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/vt8231.ko 2023-10-06T02:05:57.713 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83627ehf.ko 2023-10-06T02:05:57.713 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83627hf.ko 2023-10-06T02:05:57.713 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83781d.ko 2023-10-06T02:05:57.713 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83791d.ko 2023-10-06T02:05:57.713 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83792d.ko 2023-10-06T02:05:57.713 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83793.ko 2023-10-06T02:05:57.713 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83795.ko 2023-10-06T02:05:57.713 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83l785ts.ko 2023-10-06T02:05:57.713 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83l786ng.ko 2023-10-06T02:05:57.713 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c 2023-10-06T02:05:57.713 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos 2023-10-06T02:05:57.714 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-10-06T02:05:57.714 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-10-06T02:05:57.714 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses 2023-10-06T02:05:57.714 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-10-06T02:05:57.714 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-10-06T02:05:57.714 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-10-06T02:05:57.714 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-10-06T02:05:57.714 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-10-06T02:05:57.714 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-10-06T02:05:57.714 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-10-06T02:05:57.714 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-10-06T02:05:57.715 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-i801.ko 2023-10-06T02:05:57.715 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-isch.ko 2023-10-06T02:05:57.715 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-10-06T02:05:57.715 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-10-06T02:05:57.715 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-10-06T02:05:57.715 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-parport.ko 2023-10-06T02:05:57.715 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-10-06T02:05:57.715 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-10-06T02:05:57.715 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-10-06T02:05:57.715 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-10-06T02:05:57.715 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-10-06T02:05:57.715 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-10-06T02:05:57.715 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-via.ko 2023-10-06T02:05:57.716 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-10-06T02:05:57.716 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-10-06T02:05:57.716 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-dev.ko 2023-10-06T02:05:57.716 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-mux.ko 2023-10-06T02:05:57.716 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-10-06T02:05:57.716 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-smbus.ko 2023-10-06T02:05:57.716 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-stub.ko 2023-10-06T02:05:57.716 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband 2023-10-06T02:05:57.716 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core 2023-10-06T02:05:57.716 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_cm.ko 2023-10-06T02:05:57.716 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_core.ko 2023-10-06T02:05:57.716 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_umad.ko 2023-10-06T02:05:57.716 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-10-06T02:05:57.717 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/iw_cm.ko 2023-10-06T02:05:57.717 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/rdma_cm.ko 2023-10-06T02:05:57.717 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-10-06T02:05:57.717 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw 2023-10-06T02:05:57.717 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/cxgb4 2023-10-06T02:05:57.717 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-10-06T02:05:57.717 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx4 2023-10-06T02:05:57.717 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-10-06T02:05:57.717 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx5 2023-10-06T02:05:57.717 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-10-06T02:05:57.717 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mthca 2023-10-06T02:05:57.717 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-10-06T02:05:57.718 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp 2023-10-06T02:05:57.718 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/ipoib 2023-10-06T02:05:57.718 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-10-06T02:05:57.718 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/iser 2023-10-06T02:05:57.718 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-10-06T02:05:57.718 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/isert 2023-10-06T02:05:57.718 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-10-06T02:05:57.718 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srp 2023-10-06T02:05:57.718 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-10-06T02:05:57.718 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srpt 2023-10-06T02:05:57.718 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-10-06T02:05:57.718 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input 2023-10-06T02:05:57.718 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport 2023-10-06T02:05:57.719 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-10-06T02:05:57.719 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/fm801-gp.ko 2023-10-06T02:05:57.719 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/gameport.ko 2023-10-06T02:05:57.719 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/lightning.ko 2023-10-06T02:05:57.719 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/ns558.ko 2023-10-06T02:05:57.719 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/joydev.ko 2023-10-06T02:05:57.719 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard 2023-10-06T02:05:57.719 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/gpio_keys.ko 2023-10-06T02:05:57.719 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse 2023-10-06T02:05:57.719 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/appletouch.ko 2023-10-06T02:05:57.719 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/bcm5974.ko 2023-10-06T02:05:57.719 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/cyapatp.ko 2023-10-06T02:05:57.720 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/elan_i2c.ko 2023-10-06T02:05:57.720 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/sermouse.ko 2023-10-06T02:05:57.720 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-10-06T02:05:57.720 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/synaptics_usb.ko 2023-10-06T02:05:57.720 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/vsxxxaa.ko 2023-10-06T02:05:57.720 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/rmi4 2023-10-06T02:05:57.720 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/rmi4/rmi_core.ko 2023-10-06T02:05:57.720 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio 2023-10-06T02:05:57.720 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/altera_ps2.ko 2023-10-06T02:05:57.720 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/arc_ps2.ko 2023-10-06T02:05:57.720 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-10-06T02:05:57.720 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/serio_raw.ko 2023-10-06T02:05:57.721 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/sparse-keymap.ko 2023-10-06T02:05:57.721 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu 2023-10-06T02:05:57.721 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu/amd 2023-10-06T02:05:57.721 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu/amd/iommu_v2.ko 2023-10-06T02:05:57.721 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds 2023-10-06T02:05:57.721 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-blinkm.ko 2023-10-06T02:05:57.721 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-lm3530.ko 2023-10-06T02:05:57.721 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-lp3944.ko 2023-10-06T02:05:57.721 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-ss4200.ko 2023-10-06T02:05:57.721 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger 2023-10-06T02:05:57.721 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-10-06T02:05:57.721 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-10-06T02:05:57.721 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-10-06T02:05:57.722 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-10-06T02:05:57.722 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-10-06T02:05:57.722 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-10-06T02:05:57.722 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-10-06T02:05:57.722 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-10-06T02:05:57.722 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md 2023-10-06T02:05:57.722 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/bcache 2023-10-06T02:05:57.722 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/bcache/bcache.ko 2023-10-06T02:05:57.722 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-bio-prison.ko 2023-10-06T02:05:57.722 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-cache-smq.ko 2023-10-06T02:05:57.722 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-cache.ko 2023-10-06T02:05:57.722 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-crypt.ko 2023-10-06T02:05:57.722 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-delay.ko 2023-10-06T02:05:57.723 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-flakey.ko 2023-10-06T02:05:57.723 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-log-userspace.ko 2023-10-06T02:05:57.723 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-multipath.ko 2023-10-06T02:05:57.723 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-queue-length.ko 2023-10-06T02:05:57.723 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-raid.ko 2023-10-06T02:05:57.723 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-round-robin.ko 2023-10-06T02:05:57.723 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-service-time.ko 2023-10-06T02:05:57.723 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-switch.ko 2023-10-06T02:05:57.723 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-thin-pool.ko 2023-10-06T02:05:57.723 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-verity.ko 2023-10-06T02:05:57.723 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/faulty.ko 2023-10-06T02:05:57.723 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/linear.ko 2023-10-06T02:05:57.724 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/multipath.ko 2023-10-06T02:05:57.724 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/persistent-data 2023-10-06T02:05:57.724 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-10-06T02:05:57.724 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid0.ko 2023-10-06T02:05:57.724 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid1.ko 2023-10-06T02:05:57.724 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid10.ko 2023-10-06T02:05:57.724 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid456.ko 2023-10-06T02:05:57.724 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message 2023-10-06T02:05:57.724 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion 2023-10-06T02:05:57.724 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptbase.ko 2023-10-06T02:05:57.724 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptctl.ko 2023-10-06T02:05:57.724 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptfc.ko 2023-10-06T02:05:57.725 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptlan.ko 2023-10-06T02:05:57.725 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptsas.ko 2023-10-06T02:05:57.725 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptscsih.ko 2023-10-06T02:05:57.725 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptspi.ko 2023-10-06T02:05:57.725 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd 2023-10-06T02:05:57.725 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/lpc_ich.ko 2023-10-06T02:05:57.725 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/lpc_sch.ko 2023-10-06T02:05:57.725 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/mfd-core.ko 2023-10-06T02:05:57.725 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/sm501.ko 2023-10-06T02:05:57.725 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/viperboard.ko 2023-10-06T02:05:57.725 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/vx855.ko 2023-10-06T02:05:57.725 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/wl1273-core.ko 2023-10-06T02:05:57.725 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc 2023-10-06T02:05:57.726 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/altera-stapl 2023-10-06T02:05:57.726 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-10-06T02:05:57.726 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/apds9802als.ko 2023-10-06T02:05:57.726 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/apds990x.ko 2023-10-06T02:05:57.726 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/bh1770glc.ko 2023-10-06T02:05:57.726 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/cb710 2023-10-06T02:05:57.726 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/cb710/cb710.ko 2023-10-06T02:05:57.726 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom 2023-10-06T02:05:57.726 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/at24.ko 2023-10-06T02:05:57.726 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/eeprom.ko 2023-10-06T02:05:57.726 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-10-06T02:05:57.726 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/max6875.ko 2023-10-06T02:05:57.727 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/enclosure.ko 2023-10-06T02:05:57.727 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/hpilo.ko 2023-10-06T02:05:57.727 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/isl29003.ko 2023-10-06T02:05:57.727 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/isl29020.ko 2023-10-06T02:05:57.727 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d 2023-10-06T02:05:57.727 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-10-06T02:05:57.727 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-10-06T02:05:57.727 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei 2023-10-06T02:05:57.727 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei-me.ko 2023-10-06T02:05:57.727 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei-txe.ko 2023-10-06T02:05:57.727 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei.ko 2023-10-06T02:05:57.727 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-gru 2023-10-06T02:05:57.727 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-gru/gru.ko 2023-10-06T02:05:57.728 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp 2023-10-06T02:05:57.728 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp/xp.ko 2023-10-06T02:05:57.728 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp/xpc.ko 2023-10-06T02:05:57.728 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-10-06T02:05:57.728 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tifm_7xx1.ko 2023-10-06T02:05:57.728 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tifm_core.ko 2023-10-06T02:05:57.728 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tsl2550.ko 2023-10-06T02:05:57.728 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_balloon.ko 2023-10-06T02:05:57.728 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_vmci 2023-10-06T02:05:57.728 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-10-06T02:05:57.728 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net 2023-10-06T02:05:57.728 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/bonding 2023-10-06T02:05:57.729 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/bonding/bonding.ko 2023-10-06T02:05:57.729 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa 2023-10-06T02:05:57.729 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/b53 2023-10-06T02:05:57.729 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/b53/b53_common.ko 2023-10-06T02:05:57.729 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/bcm-sf2.ko 2023-10-06T02:05:57.729 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6060.ko 2023-10-06T02:05:57.729 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6xxx 2023-10-06T02:05:57.729 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-10-06T02:05:57.729 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dummy.ko 2023-10-06T02:05:57.729 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/eql.ko 2023-10-06T02:05:57.729 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet 2023-10-06T02:05:57.729 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com 2023-10-06T02:05:57.729 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-10-06T02:05:57.730 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-10-06T02:05:57.730 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-10-06T02:05:57.730 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-10-06T02:05:57.730 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390 2023-10-06T02:05:57.730 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/8390.ko 2023-10-06T02:05:57.730 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-10-06T02:05:57.730 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-10-06T02:05:57.730 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-10-06T02:05:57.730 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/adaptec 2023-10-06T02:05:57.730 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-10-06T02:05:57.730 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/agere 2023-10-06T02:05:57.730 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/agere/et131x.ko 2023-10-06T02:05:57.731 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/alteon 2023-10-06T02:05:57.731 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-10-06T02:05:57.731 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/altera 2023-10-06T02:05:57.731 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-10-06T02:05:57.731 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd 2023-10-06T02:05:57.731 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-10-06T02:05:57.731 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-10-06T02:05:57.731 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-10-06T02:05:57.731 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros 2023-10-06T02:05:57.731 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/alx 2023-10-06T02:05:57.731 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-10-06T02:05:57.731 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1c 2023-10-06T02:05:57.732 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-10-06T02:05:57.732 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1e 2023-10-06T02:05:57.732 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-10-06T02:05:57.732 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx 2023-10-06T02:05:57.732 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-10-06T02:05:57.732 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-10-06T02:05:57.732 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom 2023-10-06T02:05:57.732 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-10-06T02:05:57.732 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-10-06T02:05:57.732 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-10-06T02:05:57.732 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-10-06T02:05:57.732 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-10-06T02:05:57.733 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/genet 2023-10-06T02:05:57.733 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-10-06T02:05:57.733 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-10-06T02:05:57.733 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade 2023-10-06T02:05:57.733 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade/bna 2023-10-06T02:05:57.733 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-10-06T02:05:57.733 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio 2023-10-06T02:05:57.733 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb 2023-10-06T02:05:57.733 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-10-06T02:05:57.733 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-10-06T02:05:57.733 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-10-06T02:05:57.733 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-10-06T02:05:57.733 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-10-06T02:05:57.734 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-10-06T02:05:57.734 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-10-06T02:05:57.734 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-10-06T02:05:57.734 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-10-06T02:05:57.734 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco 2023-10-06T02:05:57.734 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco/enic 2023-10-06T02:05:57.734 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-10-06T02:05:57.734 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec 2023-10-06T02:05:57.734 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip 2023-10-06T02:05:57.734 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-10-06T02:05:57.734 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-10-06T02:05:57.734 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-10-06T02:05:57.735 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-10-06T02:05:57.735 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-10-06T02:05:57.735 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-10-06T02:05:57.735 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink 2023-10-06T02:05:57.735 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-10-06T02:05:57.735 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-10-06T02:05:57.735 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dnet.ko 2023-10-06T02:05:57.735 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex 2023-10-06T02:05:57.735 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex/benet 2023-10-06T02:05:57.735 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-10-06T02:05:57.735 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ethoc.ko 2023-10-06T02:05:57.735 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/fealnx.ko 2023-10-06T02:05:57.735 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel 2023-10-06T02:05:57.736 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e100.ko 2023-10-06T02:05:57.736 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000 2023-10-06T02:05:57.736 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-10-06T02:05:57.736 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000e 2023-10-06T02:05:57.736 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-10-06T02:05:57.736 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/fm10k 2023-10-06T02:05:57.736 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-10-06T02:05:57.736 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/i40e 2023-10-06T02:05:57.736 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-10-06T02:05:57.736 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/iavf 2023-10-06T02:05:57.736 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-10-06T02:05:57.736 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igb 2023-10-06T02:05:57.737 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-10-06T02:05:57.737 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igbvf 2023-10-06T02:05:57.737 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-10-06T02:05:57.737 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbe 2023-10-06T02:05:57.737 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-10-06T02:05:57.737 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbevf 2023-10-06T02:05:57.737 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-10-06T02:05:57.737 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/jme.ko 2023-10-06T02:05:57.737 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell 2023-10-06T02:05:57.737 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-10-06T02:05:57.737 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/skge.ko 2023-10-06T02:05:57.737 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-10-06T02:05:57.737 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox 2023-10-06T02:05:57.738 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4 2023-10-06T02:05:57.738 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-10-06T02:05:57.738 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-10-06T02:05:57.738 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5 2023-10-06T02:05:57.738 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-10-06T02:05:57.738 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-10-06T02:05:57.738 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/micrel 2023-10-06T02:05:57.738 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-10-06T02:05:57.738 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom 2023-10-06T02:05:57.738 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom/myri10ge 2023-10-06T02:05:57.738 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-10-06T02:05:57.738 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi 2023-10-06T02:05:57.739 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-10-06T02:05:57.739 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-10-06T02:05:57.739 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/neterion 2023-10-06T02:05:57.739 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-10-06T02:05:57.739 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/nvidia 2023-10-06T02:05:57.739 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-10-06T02:05:57.739 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines 2023-10-06T02:05:57.739 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-10-06T02:05:57.739 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-10-06T02:05:57.739 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic 2023-10-06T02:05:57.739 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/netxen 2023-10-06T02:05:57.739 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-10-06T02:05:57.739 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-10-06T02:05:57.740 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-10-06T02:05:57.740 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-10-06T02:05:57.740 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rdc 2023-10-06T02:05:57.740 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-10-06T02:05:57.740 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek 2023-10-06T02:05:57.740 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-10-06T02:05:57.740 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-10-06T02:05:57.740 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/atp.ko 2023-10-06T02:05:57.740 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-10-06T02:05:57.740 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rocker 2023-10-06T02:05:57.740 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-10-06T02:05:57.741 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sfc 2023-10-06T02:05:57.741 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-10-06T02:05:57.741 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/silan 2023-10-06T02:05:57.741 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-10-06T02:05:57.741 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis 2023-10-06T02:05:57.741 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis/sis190.ko 2023-10-06T02:05:57.741 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis/sis900.ko 2023-10-06T02:05:57.741 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc 2023-10-06T02:05:57.741 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-10-06T02:05:57.741 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-10-06T02:05:57.741 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-10-06T02:05:57.741 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-10-06T02:05:57.741 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro 2023-10-06T02:05:57.742 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac 2023-10-06T02:05:57.742 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-10-06T02:05:57.742 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-10-06T02:05:57.742 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun 2023-10-06T02:05:57.742 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/cassini.ko 2023-10-06T02:05:57.742 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/niu.ko 2023-10-06T02:05:57.742 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/sungem.ko 2023-10-06T02:05:57.742 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-10-06T02:05:57.742 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/tehuti 2023-10-06T02:05:57.742 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-10-06T02:05:57.742 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ti 2023-10-06T02:05:57.742 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ti/tlan.ko 2023-10-06T02:05:57.743 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via 2023-10-06T02:05:57.743 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-10-06T02:05:57.743 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-10-06T02:05:57.743 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet 2023-10-06T02:05:57.743 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-10-06T02:05:57.743 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-10-06T02:05:57.743 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/xircom 2023-10-06T02:05:57.743 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-10-06T02:05:57.743 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/geneve.ko 2023-10-06T02:05:57.743 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/hyperv 2023-10-06T02:05:57.743 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-10-06T02:05:57.743 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ieee802154 2023-10-06T02:05:57.743 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ieee802154/fakelb.ko 2023-10-06T02:05:57.744 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ifb.ko 2023-10-06T02:05:57.744 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ipvlan 2023-10-06T02:05:57.744 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ipvlan/ipvlan.ko 2023-10-06T02:05:57.744 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/macvlan.ko 2023-10-06T02:05:57.744 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/macvtap.ko 2023-10-06T02:05:57.744 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio 2023-10-06T02:05:57.744 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio.ko 2023-10-06T02:05:57.744 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-10-06T02:05:57.744 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-10-06T02:05:57.744 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-regmap.ko 2023-10-06T02:05:57.744 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mii.ko 2023-10-06T02:05:57.744 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/net_failover.ko 2023-10-06T02:05:57.744 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/netconsole.ko 2023-10-06T02:05:57.745 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/nlmon.ko 2023-10-06T02:05:57.745 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs 2023-10-06T02:05:57.745 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs/pcs-lynx.ko 2023-10-06T02:05:57.745 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-10-06T02:05:57.745 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy 2023-10-06T02:05:57.745 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/amd.ko 2023-10-06T02:05:57.745 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-10-06T02:05:57.745 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-10-06T02:05:57.745 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm7xxx.ko 2023-10-06T02:05:57.745 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm87xx.ko 2023-10-06T02:05:57.745 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/broadcom.ko 2023-10-06T02:05:57.745 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/cicada.ko 2023-10-06T02:05:57.746 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/davicom.ko 2023-10-06T02:05:57.746 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/dp83640.ko 2023-10-06T02:05:57.746 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/et1011c.ko 2023-10-06T02:05:57.746 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/icplus.ko 2023-10-06T02:05:57.746 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/lxt.ko 2023-10-06T02:05:57.746 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/marvell.ko 2023-10-06T02:05:57.746 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/micrel.ko 2023-10-06T02:05:57.746 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/national.ko 2023-10-06T02:05:57.746 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/phylink.ko 2023-10-06T02:05:57.746 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/qsemi.ko 2023-10-06T02:05:57.746 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/realtek.ko 2023-10-06T02:05:57.746 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/smsc.ko 2023-10-06T02:05:57.747 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/ste10Xp.ko 2023-10-06T02:05:57.747 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/vitesse.ko 2023-10-06T02:05:57.747 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp 2023-10-06T02:05:57.747 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/bsd_comp.ko 2023-10-06T02:05:57.747 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_async.ko 2023-10-06T02:05:57.747 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_deflate.ko 2023-10-06T02:05:57.747 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_generic.ko 2023-10-06T02:05:57.747 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_mppe.ko 2023-10-06T02:05:57.747 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_synctty.ko 2023-10-06T02:05:57.747 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pppoe.ko 2023-10-06T02:05:57.747 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pppox.ko 2023-10-06T02:05:57.748 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pptp.ko 2023-10-06T02:05:57.748 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip 2023-10-06T02:05:57.748 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip/slhc.ko 2023-10-06T02:05:57.748 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip/slip.ko 2023-10-06T02:05:57.748 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/sungem_phy.ko 2023-10-06T02:05:57.748 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/tap.ko 2023-10-06T02:05:57.748 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team 2023-10-06T02:05:57.748 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team.ko 2023-10-06T02:05:57.748 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_activebackup.ko 2023-10-06T02:05:57.748 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_broadcast.ko 2023-10-06T02:05:57.748 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-10-06T02:05:57.748 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_random.ko 2023-10-06T02:05:57.749 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-10-06T02:05:57.749 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/tun.ko 2023-10-06T02:05:57.749 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/veth.ko 2023-10-06T02:05:57.749 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/virtio_net.ko 2023-10-06T02:05:57.749 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vmxnet3 2023-10-06T02:05:57.749 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-10-06T02:05:57.749 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vxlan 2023-10-06T02:05:57.749 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vxlan/vxlan.ko 2023-10-06T02:05:57.749 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netback 2023-10-06T02:05:57.749 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netback/xen-netback.ko 2023-10-06T02:05:57.749 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netfront.ko 2023-10-06T02:05:57.749 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ntb 2023-10-06T02:05:57.750 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ntb/ntb.ko 2023-10-06T02:05:57.750 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme 2023-10-06T02:05:57.750 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host 2023-10-06T02:05:57.750 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host/nvme-core.ko 2023-10-06T02:05:57.750 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host/nvme.ko 2023-10-06T02:05:57.750 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport 2023-10-06T02:05:57.750 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport.ko 2023-10-06T02:05:57.750 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_cs.ko 2023-10-06T02:05:57.750 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_pc.ko 2023-10-06T02:05:57.750 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_serial.ko 2023-10-06T02:05:57.750 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci 2023-10-06T02:05:57.750 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/hotplug 2023-10-06T02:05:57.751 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-10-06T02:05:57.751 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/pcie 2023-10-06T02:05:57.751 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/pcie/aer_inject.ko 2023-10-06T02:05:57.751 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/xen-pcifront.ko 2023-10-06T02:05:57.751 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia 2023-10-06T02:05:57.751 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/i82092.ko 2023-10-06T02:05:57.751 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/pd6729.ko 2023-10-06T02:05:57.751 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/yenta_socket.ko 2023-10-06T02:05:57.751 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl 2023-10-06T02:05:57.751 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl/intel 2023-10-06T02:05:57.751 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-10-06T02:05:57.751 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform 2023-10-06T02:05:57.752 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86 2023-10-06T02:05:57.752 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/acer-wmi.ko 2023-10-06T02:05:57.752 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/acerhdf.ko 2023-10-06T02:05:57.752 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/apple-gmux.ko 2023-10-06T02:05:57.752 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-laptop.ko 2023-10-06T02:05:57.752 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-10-06T02:05:57.752 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-wmi.ko 2023-10-06T02:05:57.752 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/classmate-laptop.ko 2023-10-06T02:05:57.752 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-10-06T02:05:57.752 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-10-06T02:05:57.752 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-10-06T02:05:57.752 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-10-06T02:05:57.753 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/hdaps.ko 2023-10-06T02:05:57.753 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel 2023-10-06T02:05:57.753 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-10-06T02:05:57.753 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel_ips.ko 2023-10-06T02:05:57.753 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/msi-wmi.ko 2023-10-06T02:05:57.753 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/mxm-wmi.ko 2023-10-06T02:05:57.753 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-10-06T02:05:57.753 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/samsung-laptop.ko 2023-10-06T02:05:57.753 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/samsung-q10.ko 2023-10-06T02:05:57.753 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/topstar-laptop.ko 2023-10-06T02:05:57.753 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-10-06T02:05:57.753 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/toshiba_haps.ko 2023-10-06T02:05:57.754 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/wmi-bmof.ko 2023-10-06T02:05:57.754 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/wmi.ko 2023-10-06T02:05:57.754 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap 2023-10-06T02:05:57.754 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap/intel_rapl_common.ko 2023-10-06T02:05:57.754 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap/intel_rapl_msr.ko 2023-10-06T02:05:57.754 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps 2023-10-06T02:05:57.754 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients 2023-10-06T02:05:57.754 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps-gpio.ko 2023-10-06T02:05:57.754 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps-ldisc.ko 2023-10-06T02:05:57.754 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps_parport.ko 2023-10-06T02:05:57.754 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/pps_core.ko 2023-10-06T02:05:57.754 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp 2023-10-06T02:05:57.755 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp/ptp.ko 2023-10-06T02:05:57.755 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp/ptp_kvm.ko 2023-10-06T02:05:57.755 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc 2023-10-06T02:05:57.755 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-bq32k.ko 2023-10-06T02:05:57.755 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-bq4802.ko 2023-10-06T02:05:57.755 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1286.ko 2023-10-06T02:05:57.755 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1307.ko 2023-10-06T02:05:57.755 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1374.ko 2023-10-06T02:05:57.755 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1511.ko 2023-10-06T02:05:57.755 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1553.ko 2023-10-06T02:05:57.755 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1672.ko 2023-10-06T02:05:57.755 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1742.ko 2023-10-06T02:05:57.756 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds2404.ko 2023-10-06T02:05:57.756 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds3232.ko 2023-10-06T02:05:57.756 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-em3027.ko 2023-10-06T02:05:57.756 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-fm3130.ko 2023-10-06T02:05:57.756 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-isl12022.ko 2023-10-06T02:05:57.756 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-isl1208.ko 2023-10-06T02:05:57.756 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m41t80.ko 2023-10-06T02:05:57.756 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m48t35.ko 2023-10-06T02:05:57.756 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m48t59.ko 2023-10-06T02:05:57.756 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-max6900.ko 2023-10-06T02:05:57.756 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-msm6242.ko 2023-10-06T02:05:57.756 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf2127.ko 2023-10-06T02:05:57.757 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf85063.ko 2023-10-06T02:05:57.757 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8523.ko 2023-10-06T02:05:57.757 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8563.ko 2023-10-06T02:05:57.757 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8583.ko 2023-10-06T02:05:57.757 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rp5c01.ko 2023-10-06T02:05:57.757 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rs5c372.ko 2023-10-06T02:05:57.757 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-10-06T02:05:57.757 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rx8025.ko 2023-10-06T02:05:57.757 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rx8581.ko 2023-10-06T02:05:57.757 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-10-06T02:05:57.757 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-x1205.ko 2023-10-06T02:05:57.757 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi 2023-10-06T02:05:57.758 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-9xxx.ko 2023-10-06T02:05:57.758 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-sas.ko 2023-10-06T02:05:57.758 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-xxxx.ko 2023-10-06T02:05:57.758 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/BusLogic.ko 2023-10-06T02:05:57.758 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/a100u2w.ko 2023-10-06T02:05:57.758 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aacraid 2023-10-06T02:05:57.758 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aacraid/aacraid.ko 2023-10-06T02:05:57.758 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/advansys.ko 2023-10-06T02:05:57.758 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx 2023-10-06T02:05:57.758 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-10-06T02:05:57.758 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-10-06T02:05:57.759 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/am53c974.ko 2023-10-06T02:05:57.759 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/arcmsr 2023-10-06T02:05:57.759 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-10-06T02:05:57.759 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/atp870u.ko 2023-10-06T02:05:57.759 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/be2iscsi 2023-10-06T02:05:57.759 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-10-06T02:05:57.759 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bfa 2023-10-06T02:05:57.759 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bfa/bfa.ko 2023-10-06T02:05:57.759 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2fc 2023-10-06T02:05:57.759 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-10-06T02:05:57.759 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2i 2023-10-06T02:05:57.759 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-10-06T02:05:57.760 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ch.ko 2023-10-06T02:05:57.760 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/csiostor 2023-10-06T02:05:57.760 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/csiostor/csiostor.ko 2023-10-06T02:05:57.760 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi 2023-10-06T02:05:57.760 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb3i 2023-10-06T02:05:57.760 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-10-06T02:05:57.760 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb4i 2023-10-06T02:05:57.760 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-10-06T02:05:57.760 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-10-06T02:05:57.760 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/dc395x.ko 2023-10-06T02:05:57.760 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler 2023-10-06T02:05:57.760 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-10-06T02:05:57.760 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-10-06T02:05:57.761 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-10-06T02:05:57.761 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-10-06T02:05:57.761 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/dmx3191d.ko 2023-10-06T02:05:57.761 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esas2r 2023-10-06T02:05:57.761 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esas2r/esas2r.ko 2023-10-06T02:05:57.761 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esp_scsi.ko 2023-10-06T02:05:57.761 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe 2023-10-06T02:05:57.761 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe/fcoe.ko 2023-10-06T02:05:57.761 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-10-06T02:05:57.761 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fnic 2023-10-06T02:05:57.761 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fnic/fnic.ko 2023-10-06T02:05:57.761 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hpsa.ko 2023-10-06T02:05:57.762 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hptiop.ko 2023-10-06T02:05:57.762 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hv_storvsc.ko 2023-10-06T02:05:57.762 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/initio.ko 2023-10-06T02:05:57.762 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ipr.ko 2023-10-06T02:05:57.762 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ips.ko 2023-10-06T02:05:57.762 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/isci 2023-10-06T02:05:57.762 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/isci/isci.ko 2023-10-06T02:05:57.762 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-10-06T02:05:57.762 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/iscsi_tcp.ko 2023-10-06T02:05:57.762 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libfc 2023-10-06T02:05:57.762 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libfc/libfc.ko 2023-10-06T02:05:57.763 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libiscsi.ko 2023-10-06T02:05:57.763 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libiscsi_tcp.ko 2023-10-06T02:05:57.763 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libsas 2023-10-06T02:05:57.763 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libsas/libsas.ko 2023-10-06T02:05:57.763 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/lpfc 2023-10-06T02:05:57.763 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/lpfc/lpfc.ko 2023-10-06T02:05:57.763 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid 2023-10-06T02:05:57.763 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid.ko 2023-10-06T02:05:57.763 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-10-06T02:05:57.763 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-10-06T02:05:57.763 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-10-06T02:05:57.763 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mpt3sas 2023-10-06T02:05:57.763 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-10-06T02:05:57.764 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvsas 2023-10-06T02:05:57.764 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvsas/mvsas.ko 2023-10-06T02:05:57.764 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvumi.ko 2023-10-06T02:05:57.764 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pm8001 2023-10-06T02:05:57.764 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-10-06T02:05:57.764 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pmcraid.ko 2023-10-06T02:05:57.764 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla1280.ko 2023-10-06T02:05:57.764 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx 2023-10-06T02:05:57.764 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-10-06T02:05:57.764 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-10-06T02:05:57.764 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla4xxx 2023-10-06T02:05:57.764 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-10-06T02:05:57.764 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/raid_class.ko 2023-10-06T02:05:57.765 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_debug.ko 2023-10-06T02:05:57.765 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_fc.ko 2023-10-06T02:05:57.765 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-10-06T02:05:57.765 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_sas.ko 2023-10-06T02:05:57.765 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_spi.ko 2023-10-06T02:05:57.765 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_srp.ko 2023-10-06T02:05:57.765 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ses.ko 2023-10-06T02:05:57.765 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/st.ko 2023-10-06T02:05:57.765 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/stex.ko 2023-10-06T02:05:57.765 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/sym53c8xx_2 2023-10-06T02:05:57.765 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-10-06T02:05:57.765 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/virtio_scsi.ko 2023-10-06T02:05:57.766 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/vmw_pvscsi.ko 2023-10-06T02:05:57.766 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/wd719x.ko 2023-10-06T02:05:57.766 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ssb 2023-10-06T02:05:57.766 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ssb/ssb.ko 2023-10-06T02:05:57.766 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target 2023-10-06T02:05:57.766 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/iscsi 2023-10-06T02:05:57.766 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-10-06T02:05:57.766 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/loopback 2023-10-06T02:05:57.766 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/loopback/tcm_loop.ko 2023-10-06T02:05:57.766 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_file.ko 2023-10-06T02:05:57.766 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_iblock.ko 2023-10-06T02:05:57.766 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_mod.ko 2023-10-06T02:05:57.767 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_pscsi.ko 2023-10-06T02:05:57.767 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_user.ko 2023-10-06T02:05:57.767 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/tcm_fc 2023-10-06T02:05:57.767 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-10-06T02:05:57.767 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal 2023-10-06T02:05:57.767 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel 2023-10-06T02:05:57.767 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal 2023-10-06T02:05:57.767 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-10-06T02:05:57.767 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-10-06T02:05:57.767 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-10-06T02:05:57.767 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-10-06T02:05:57.767 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-10-06T02:05:57.767 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-10-06T02:05:57.768 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-10-06T02:05:57.768 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-10-06T02:05:57.768 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-10-06T02:05:57.768 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-10-06T02:05:57.768 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-10-06T02:05:57.768 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-10-06T02:05:57.768 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-10-06T02:05:57.768 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-10-06T02:05:57.768 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-10-06T02:05:57.768 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty 2023-10-06T02:05:57.768 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/ipwireless 2023-10-06T02:05:57.768 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-10-06T02:05:57.769 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/n_gsm.ko 2023-10-06T02:05:57.769 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/n_hdlc.ko 2023-10-06T02:05:57.769 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/nozomi.ko 2023-10-06T02:05:57.769 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial 2023-10-06T02:05:57.769 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/8250 2023-10-06T02:05:57.769 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-10-06T02:05:57.769 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/arc_uart.ko 2023-10-06T02:05:57.769 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/jsm 2023-10-06T02:05:57.769 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/jsm/jsm.ko 2023-10-06T02:05:57.769 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/synclink_gt.ko 2023-10-06T02:05:57.769 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs 2023-10-06T02:05:57.769 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/core 2023-10-06T02:05:57.769 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/core/ufshcd-core.ko 2023-10-06T02:05:57.770 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/host 2023-10-06T02:05:57.770 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-10-06T02:05:57.770 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio 2023-10-06T02:05:57.770 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio.ko 2023-10-06T02:05:57.770 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_aec.ko 2023-10-06T02:05:57.770 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_cif.ko 2023-10-06T02:05:57.770 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_pci_generic.ko 2023-10-06T02:05:57.770 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_sercos3.ko 2023-10-06T02:05:57.770 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb 2023-10-06T02:05:57.770 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class 2023-10-06T02:05:57.770 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/cdc-acm.ko 2023-10-06T02:05:57.770 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/cdc-wdm.ko 2023-10-06T02:05:57.771 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/usblp.ko 2023-10-06T02:05:57.771 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/usbtmc.ko 2023-10-06T02:05:57.771 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host 2023-10-06T02:05:57.771 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host/sl811-hcd.ko 2023-10-06T02:05:57.771 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image 2023-10-06T02:05:57.771 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image/mdc800.ko 2023-10-06T02:05:57.771 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image/microtek.ko 2023-10-06T02:05:57.771 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc 2023-10-06T02:05:57.771 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/adutux.ko 2023-10-06T02:05:57.771 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/appledisplay.ko 2023-10-06T02:05:57.771 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/emi26.ko 2023-10-06T02:05:57.771 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/emi62.ko 2023-10-06T02:05:57.771 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/ezusb.ko 2023-10-06T02:05:57.772 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/idmouse.ko 2023-10-06T02:05:57.772 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/iowarrior.ko 2023-10-06T02:05:57.772 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/isight_firmware.ko 2023-10-06T02:05:57.772 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/ldusb.ko 2023-10-06T02:05:57.772 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/legousbtower.ko 2023-10-06T02:05:57.772 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/sisusbvga 2023-10-06T02:05:57.772 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-10-06T02:05:57.772 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/trancevibrator.ko 2023-10-06T02:05:57.772 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usb3503.ko 2023-10-06T02:05:57.772 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usblcd.ko 2023-10-06T02:05:57.772 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usbsevseg.ko 2023-10-06T02:05:57.772 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/uss720.ko 2023-10-06T02:05:57.773 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/yurex.ko 2023-10-06T02:05:57.773 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/phy 2023-10-06T02:05:57.773 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/phy/phy-generic.ko 2023-10-06T02:05:57.773 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial 2023-10-06T02:05:57.773 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/aircable.ko 2023-10-06T02:05:57.773 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ark3116.ko 2023-10-06T02:05:57.773 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/belkin_sa.ko 2023-10-06T02:05:57.773 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ch341.ko 2023-10-06T02:05:57.773 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cp210x.ko 2023-10-06T02:05:57.773 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cyberjack.ko 2023-10-06T02:05:57.773 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cypress_m8.ko 2023-10-06T02:05:57.773 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/digi_acceleport.ko 2023-10-06T02:05:57.774 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/empeg.ko 2023-10-06T02:05:57.774 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ftdi_sio.ko 2023-10-06T02:05:57.774 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/garmin_gps.ko 2023-10-06T02:05:57.774 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/io_edgeport.ko 2023-10-06T02:05:57.774 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/io_ti.ko 2023-10-06T02:05:57.774 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ipaq.ko 2023-10-06T02:05:57.774 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ipw.ko 2023-10-06T02:05:57.774 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ir-usb.ko 2023-10-06T02:05:57.774 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-10-06T02:05:57.774 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/keyspan.ko 2023-10-06T02:05:57.774 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/keyspan_pda.ko 2023-10-06T02:05:57.774 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/kl5kusb105.ko 2023-10-06T02:05:57.775 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/kobil_sct.ko 2023-10-06T02:05:57.775 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mct_u232.ko 2023-10-06T02:05:57.775 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mos7720.ko 2023-10-06T02:05:57.775 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mos7840.ko 2023-10-06T02:05:57.775 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/navman.ko 2023-10-06T02:05:57.775 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/omninet.ko 2023-10-06T02:05:57.775 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/opticon.ko 2023-10-06T02:05:57.775 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/option.ko 2023-10-06T02:05:57.775 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/oti6858.ko 2023-10-06T02:05:57.775 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/pl2303.ko 2023-10-06T02:05:57.775 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/qcaux.ko 2023-10-06T02:05:57.775 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/qcserial.ko 2023-10-06T02:05:57.776 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/quatech2.ko 2023-10-06T02:05:57.776 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/safe_serial.ko 2023-10-06T02:05:57.776 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/sierra.ko 2023-10-06T02:05:57.776 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/spcp8x5.ko 2023-10-06T02:05:57.776 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ssu100.ko 2023-10-06T02:05:57.776 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/symbolserial.ko 2023-10-06T02:05:57.776 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-10-06T02:05:57.776 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-10-06T02:05:57.776 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb_debug.ko 2023-10-06T02:05:57.776 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb_wwan.ko 2023-10-06T02:05:57.776 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/visor.ko 2023-10-06T02:05:57.776 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/whiteheat.ko 2023-10-06T02:05:57.776 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/xsens_mt.ko 2023-10-06T02:05:57.777 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage 2023-10-06T02:05:57.777 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/uas.ko 2023-10-06T02:05:57.777 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-alauda.ko 2023-10-06T02:05:57.777 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-cypress.ko 2023-10-06T02:05:57.777 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-datafab.ko 2023-10-06T02:05:57.777 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-10-06T02:05:57.777 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-freecom.ko 2023-10-06T02:05:57.777 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-isd200.ko 2023-10-06T02:05:57.777 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-10-06T02:05:57.777 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-karma.ko 2023-10-06T02:05:57.777 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-onetouch.ko 2023-10-06T02:05:57.777 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-realtek.ko 2023-10-06T02:05:57.778 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-sddr09.ko 2023-10-06T02:05:57.778 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-sddr55.ko 2023-10-06T02:05:57.778 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-usbat.ko 2023-10-06T02:05:57.778 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/usb-storage.ko 2023-10-06T02:05:57.778 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip 2023-10-06T02:05:57.778 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/usbip-core.ko 2023-10-06T02:05:57.778 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/usbip-host.ko 2023-10-06T02:05:57.778 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-10-06T02:05:57.778 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio 2023-10-06T02:05:57.778 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci 2023-10-06T02:05:57.778 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-10-06T02:05:57.778 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci/vfio-pci.ko 2023-10-06T02:05:57.779 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/vfio.ko 2023-10-06T02:05:57.779 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-10-06T02:05:57.779 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost 2023-10-06T02:05:57.779 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost.ko 2023-10-06T02:05:57.779 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_iotlb.ko 2023-10-06T02:05:57.779 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_net.ko 2023-10-06T02:05:57.779 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_scsi.ko 2023-10-06T02:05:57.779 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video 2023-10-06T02:05:57.779 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight 2023-10-06T02:05:57.779 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/apple_bl.ko 2023-10-06T02:05:57.779 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/lcd.ko 2023-10-06T02:05:57.779 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/platform_lcd.ko 2023-10-06T02:05:57.779 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio 2023-10-06T02:05:57.780 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio.ko 2023-10-06T02:05:57.780 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_balloon.ko 2023-10-06T02:05:57.780 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_mmio.ko 2023-10-06T02:05:57.780 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_pci.ko 2023-10-06T02:05:57.780 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-10-06T02:05:57.780 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-10-06T02:05:57.780 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_ring.ko 2023-10-06T02:05:57.780 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1 2023-10-06T02:05:57.780 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters 2023-10-06T02:05:57.780 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters/ds2482.ko 2023-10-06T02:05:57.780 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters/ds2490.ko 2023-10-06T02:05:57.780 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves 2023-10-06T02:05:57.781 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-10-06T02:05:57.781 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-10-06T02:05:57.781 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-10-06T02:05:57.781 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-10-06T02:05:57.781 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-10-06T02:05:57.781 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-10-06T02:05:57.781 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-10-06T02:05:57.781 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-10-06T02:05:57.781 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-10-06T02:05:57.781 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_smem.ko 2023-10-06T02:05:57.781 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_therm.ko 2023-10-06T02:05:57.781 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/wire.ko 2023-10-06T02:05:57.781 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog 2023-10-06T02:05:57.782 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/alim1535_wdt.ko 2023-10-06T02:05:57.782 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/alim7101_wdt.ko 2023-10-06T02:05:57.782 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/f71808e_wdt.ko 2023-10-06T02:05:57.782 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/hpwdt.ko 2023-10-06T02:05:57.782 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/i6300esb.ko 2023-10-06T02:05:57.782 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-10-06T02:05:57.782 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/iTCO_wdt.ko 2023-10-06T02:05:57.782 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ib700wdt.ko 2023-10-06T02:05:57.782 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ibmasr.ko 2023-10-06T02:05:57.782 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-10-06T02:05:57.782 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/it8712f_wdt.ko 2023-10-06T02:05:57.782 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/it87_wdt.ko 2023-10-06T02:05:57.783 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/machzwd.ko 2023-10-06T02:05:57.783 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/nv_tco.ko 2023-10-06T02:05:57.783 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/pcwd_pci.ko 2023-10-06T02:05:57.783 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/pcwd_usb.ko 2023-10-06T02:05:57.783 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-10-06T02:05:57.783 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sch311x_wdt.ko 2023-10-06T02:05:57.783 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/softdog.ko 2023-10-06T02:05:57.783 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sp5100_tco.ko 2023-10-06T02:05:57.783 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/via_wdt.ko 2023-10-06T02:05:57.783 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-10-06T02:05:57.783 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83877f_wdt.ko 2023-10-06T02:05:57.783 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83977f_wdt.ko 2023-10-06T02:05:57.783 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/wdt_pci.ko 2023-10-06T02:05:57.784 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/xen_wdt.ko 2023-10-06T02:05:57.784 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen 2023-10-06T02:05:57.784 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-acpi-processor.ko 2023-10-06T02:05:57.784 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-evtchn.ko 2023-10-06T02:05:57.784 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-gntalloc.ko 2023-10-06T02:05:57.784 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-gntdev.ko 2023-10-06T02:05:57.784 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-pciback 2023-10-06T02:05:57.784 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-10-06T02:05:57.784 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-privcmd.ko 2023-10-06T02:05:57.784 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xenfs 2023-10-06T02:05:57.784 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xenfs/xenfs.ko 2023-10-06T02:05:57.784 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs 2023-10-06T02:05:57.784 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/9p 2023-10-06T02:05:57.785 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/9p/9p.ko 2023-10-06T02:05:57.785 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/affs 2023-10-06T02:05:57.785 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/affs/affs.ko 2023-10-06T02:05:57.785 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/befs 2023-10-06T02:05:57.785 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/befs/befs.ko 2023-10-06T02:05:57.785 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/binfmt_misc.ko 2023-10-06T02:05:57.785 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/btrfs 2023-10-06T02:05:57.785 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/btrfs/btrfs.ko 2023-10-06T02:05:57.785 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cachefiles 2023-10-06T02:05:57.785 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cachefiles/cachefiles.ko 2023-10-06T02:05:57.785 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ceph 2023-10-06T02:05:57.785 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ceph/ceph.ko 2023-10-06T02:05:57.786 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/coda 2023-10-06T02:05:57.786 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/coda/coda.ko 2023-10-06T02:05:57.786 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cramfs 2023-10-06T02:05:57.786 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cramfs/cramfs.ko 2023-10-06T02:05:57.786 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/dlm 2023-10-06T02:05:57.786 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/dlm/dlm.ko 2023-10-06T02:05:57.786 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ecryptfs 2023-10-06T02:05:57.786 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ecryptfs/ecryptfs.ko 2023-10-06T02:05:57.786 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/f2fs 2023-10-06T02:05:57.786 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/f2fs/f2fs.ko 2023-10-06T02:05:57.786 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat 2023-10-06T02:05:57.786 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/fat.ko 2023-10-06T02:05:57.787 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/msdos.ko 2023-10-06T02:05:57.787 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/vfat.ko 2023-10-06T02:05:57.787 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fscache 2023-10-06T02:05:57.787 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fscache/fscache.ko 2023-10-06T02:05:57.787 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse 2023-10-06T02:05:57.787 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse/cuse.ko 2023-10-06T02:05:57.787 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse/fuse.ko 2023-10-06T02:05:57.787 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/gfs2 2023-10-06T02:05:57.787 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/gfs2/gfs2.ko 2023-10-06T02:05:57.787 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfs 2023-10-06T02:05:57.787 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfs/hfs.ko 2023-10-06T02:05:57.787 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfsplus 2023-10-06T02:05:57.788 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfsplus/hfsplus.ko 2023-10-06T02:05:57.788 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/isofs 2023-10-06T02:05:57.788 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/isofs/isofs.ko 2023-10-06T02:05:57.788 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/jfs 2023-10-06T02:05:57.788 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/jfs/jfs.ko 2023-10-06T02:05:57.788 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/lockd 2023-10-06T02:05:57.788 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/lockd/lockd.ko 2023-10-06T02:05:57.788 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/minix 2023-10-06T02:05:57.788 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/minix/minix.ko 2023-10-06T02:05:57.788 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/netfs 2023-10-06T02:05:57.788 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/netfs/netfs.ko 2023-10-06T02:05:57.788 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs 2023-10-06T02:05:57.789 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/blocklayout 2023-10-06T02:05:57.789 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-10-06T02:05:57.789 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/filelayout 2023-10-06T02:05:57.789 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-10-06T02:05:57.789 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/flexfilelayout 2023-10-06T02:05:57.789 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-10-06T02:05:57.789 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfs.ko 2023-10-06T02:05:57.789 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfsv3.ko 2023-10-06T02:05:57.789 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfsv4.ko 2023-10-06T02:05:57.789 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common 2023-10-06T02:05:57.789 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common/grace.ko 2023-10-06T02:05:57.789 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common/nfs_acl.ko 2023-10-06T02:05:57.790 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfsd 2023-10-06T02:05:57.790 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfsd/nfsd.ko 2023-10-06T02:05:57.790 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nilfs2 2023-10-06T02:05:57.790 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nilfs2/nilfs2.ko 2023-10-06T02:05:57.790 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls 2023-10-06T02:05:57.790 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-celtic.ko 2023-10-06T02:05:57.790 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-centeuro.ko 2023-10-06T02:05:57.790 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-croatian.ko 2023-10-06T02:05:57.790 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-cyrillic.ko 2023-10-06T02:05:57.790 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-gaelic.ko 2023-10-06T02:05:57.790 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-greek.ko 2023-10-06T02:05:57.790 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-iceland.ko 2023-10-06T02:05:57.791 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-inuit.ko 2023-10-06T02:05:57.791 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-roman.ko 2023-10-06T02:05:57.791 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-romanian.ko 2023-10-06T02:05:57.791 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-turkish.ko 2023-10-06T02:05:57.791 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1250.ko 2023-10-06T02:05:57.791 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1251.ko 2023-10-06T02:05:57.791 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1255.ko 2023-10-06T02:05:57.791 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp737.ko 2023-10-06T02:05:57.791 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp775.ko 2023-10-06T02:05:57.791 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp850.ko 2023-10-06T02:05:57.791 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp852.ko 2023-10-06T02:05:57.791 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp855.ko 2023-10-06T02:05:57.791 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp857.ko 2023-10-06T02:05:57.791 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp860.ko 2023-10-06T02:05:57.792 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp861.ko 2023-10-06T02:05:57.792 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp862.ko 2023-10-06T02:05:57.792 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp863.ko 2023-10-06T02:05:57.792 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp864.ko 2023-10-06T02:05:57.792 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp865.ko 2023-10-06T02:05:57.792 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp866.ko 2023-10-06T02:05:57.792 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp869.ko 2023-10-06T02:05:57.792 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp874.ko 2023-10-06T02:05:57.792 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp932.ko 2023-10-06T02:05:57.792 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp936.ko 2023-10-06T02:05:57.792 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp949.ko 2023-10-06T02:05:57.793 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp950.ko 2023-10-06T02:05:57.793 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_euc-jp.ko 2023-10-06T02:05:57.793 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-1.ko 2023-10-06T02:05:57.793 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-13.ko 2023-10-06T02:05:57.793 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-14.ko 2023-10-06T02:05:57.793 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-15.ko 2023-10-06T02:05:57.793 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-2.ko 2023-10-06T02:05:57.793 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-3.ko 2023-10-06T02:05:57.793 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-4.ko 2023-10-06T02:05:57.793 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-5.ko 2023-10-06T02:05:57.793 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-6.ko 2023-10-06T02:05:57.793 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-7.ko 2023-10-06T02:05:57.794 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-9.ko 2023-10-06T02:05:57.794 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-r.ko 2023-10-06T02:05:57.794 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-ru.ko 2023-10-06T02:05:57.794 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-u.ko 2023-10-06T02:05:57.794 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_utf8.ko 2023-10-06T02:05:57.794 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2 2023-10-06T02:05:57.794 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/cluster 2023-10-06T02:05:57.794 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-10-06T02:05:57.794 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlm 2023-10-06T02:05:57.794 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-10-06T02:05:57.794 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlmfs 2023-10-06T02:05:57.794 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-10-06T02:05:57.795 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2.ko 2023-10-06T02:05:57.795 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-10-06T02:05:57.795 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-10-06T02:05:57.795 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-10-06T02:05:57.795 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/overlayfs 2023-10-06T02:05:57.795 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/overlayfs/overlay.ko 2023-10-06T02:05:57.795 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/pstore 2023-10-06T02:05:57.795 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/pstore/ramoops.ko 2023-10-06T02:05:57.795 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/reiserfs 2023-10-06T02:05:57.795 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/reiserfs/reiserfs.ko 2023-10-06T02:05:57.795 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/romfs 2023-10-06T02:05:57.795 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/romfs/romfs.ko 2023-10-06T02:05:57.795 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb 2023-10-06T02:05:57.796 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/client 2023-10-06T02:05:57.796 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/client/cifs.ko 2023-10-06T02:05:57.796 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common 2023-10-06T02:05:57.796 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common/cifs_arc4.ko 2023-10-06T02:05:57.796 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common/cifs_md4.ko 2023-10-06T02:05:57.796 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/squashfs 2023-10-06T02:05:57.796 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/squashfs/squashfs.ko 2023-10-06T02:05:57.796 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/sysv 2023-10-06T02:05:57.796 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/sysv/sysv.ko 2023-10-06T02:05:57.796 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/udf 2023-10-06T02:05:57.796 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/udf/udf.ko 2023-10-06T02:05:57.797 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ufs 2023-10-06T02:05:57.797 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ufs/ufs.ko 2023-10-06T02:05:57.797 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/xfs 2023-10-06T02:05:57.797 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/xfs/xfs.ko 2023-10-06T02:05:57.797 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel 2023-10-06T02:05:57.797 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/rcu 2023-10-06T02:05:57.797 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/rcu/rcutorture.ko 2023-10-06T02:05:57.797 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/torture.ko 2023-10-06T02:05:57.797 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/trace 2023-10-06T02:05:57.797 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-10-06T02:05:57.797 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib 2023-10-06T02:05:57.797 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/asn1_encoder.ko 2023-10-06T02:05:57.798 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crc-itu-t.ko 2023-10-06T02:05:57.798 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crc8.ko 2023-10-06T02:05:57.798 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto 2023-10-06T02:05:57.798 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto/libarc4.ko 2023-10-06T02:05:57.798 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto/libdes.ko 2023-10-06T02:05:57.798 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/libcrc32c.ko 2023-10-06T02:05:57.798 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lru_cache.ko 2023-10-06T02:05:57.798 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4 2023-10-06T02:05:57.798 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4/lz4_compress.ko 2023-10-06T02:05:57.798 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4/lz4hc_compress.ko 2023-10-06T02:05:57.798 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/math 2023-10-06T02:05:57.798 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/math/cordic.ko 2023-10-06T02:05:57.799 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/raid6 2023-10-06T02:05:57.799 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/raid6/raid6_pq.ko 2023-10-06T02:05:57.799 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/reed_solomon 2023-10-06T02:05:57.799 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/reed_solomon/reed_solomon.ko 2023-10-06T02:05:57.799 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_bm.ko 2023-10-06T02:05:57.799 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_fsm.ko 2023-10-06T02:05:57.799 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_kmp.ko 2023-10-06T02:05:57.799 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/zstd 2023-10-06T02:05:57.799 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/zstd/zstd_compress.ko 2023-10-06T02:05:57.799 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/mm 2023-10-06T02:05:57.799 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/mm/hwpoison-inject.ko 2023-10-06T02:05:57.799 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net 2023-10-06T02:05:57.800 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan 2023-10-06T02:05:57.800 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/6lowpan.ko 2023-10-06T02:05:57.800 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_dest.ko 2023-10-06T02:05:57.800 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_fragment.ko 2023-10-06T02:05:57.800 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_hop.ko 2023-10-06T02:05:57.800 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_ipv6.ko 2023-10-06T02:05:57.800 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_mobility.ko 2023-10-06T02:05:57.800 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_routing.ko 2023-10-06T02:05:57.800 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_udp.ko 2023-10-06T02:05:57.800 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802 2023-10-06T02:05:57.800 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/garp.ko 2023-10-06T02:05:57.800 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/mrp.ko 2023-10-06T02:05:57.801 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/p8022.ko 2023-10-06T02:05:57.801 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/psnap.ko 2023-10-06T02:05:57.801 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/stp.ko 2023-10-06T02:05:57.801 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/8021q 2023-10-06T02:05:57.801 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/8021q/8021q.ko 2023-10-06T02:05:57.801 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p 2023-10-06T02:05:57.801 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet.ko 2023-10-06T02:05:57.801 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_fd.ko 2023-10-06T02:05:57.801 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_rdma.ko 2023-10-06T02:05:57.801 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_virtio.ko 2023-10-06T02:05:57.801 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/batman-adv 2023-10-06T02:05:57.801 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/batman-adv/batman-adv.ko 2023-10-06T02:05:57.801 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge 2023-10-06T02:05:57.802 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/br_netfilter.ko 2023-10-06T02:05:57.802 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/bridge.ko 2023-10-06T02:05:57.802 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter 2023-10-06T02:05:57.802 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-10-06T02:05:57.802 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_among.ko 2023-10-06T02:05:57.802 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_arp.ko 2023-10-06T02:05:57.802 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-10-06T02:05:57.802 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-10-06T02:05:57.802 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_ip.ko 2023-10-06T02:05:57.802 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-10-06T02:05:57.802 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_limit.ko 2023-10-06T02:05:57.803 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_log.ko 2023-10-06T02:05:57.803 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_mark.ko 2023-10-06T02:05:57.803 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-10-06T02:05:57.803 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-10-06T02:05:57.803 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-10-06T02:05:57.803 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-10-06T02:05:57.803 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_snat.ko 2023-10-06T02:05:57.803 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_stp.ko 2023-10-06T02:05:57.803 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-10-06T02:05:57.803 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-10-06T02:05:57.803 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-10-06T02:05:57.803 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-10-06T02:05:57.803 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtables.ko 2023-10-06T02:05:57.804 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-10-06T02:05:57.804 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-10-06T02:05:57.804 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-10-06T02:05:57.804 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ceph 2023-10-06T02:05:57.804 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ceph/libceph.ko 2023-10-06T02:05:57.804 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core 2023-10-06T02:05:57.804 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core/failover.ko 2023-10-06T02:05:57.804 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core/pktgen.ko 2023-10-06T02:05:57.804 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp 2023-10-06T02:05:57.804 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp.ko 2023-10-06T02:05:57.804 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_diag.ko 2023-10-06T02:05:57.805 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_ipv4.ko 2023-10-06T02:05:57.805 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_ipv6.ko 2023-10-06T02:05:57.805 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dns_resolver 2023-10-06T02:05:57.805 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dns_resolver/dns_resolver.ko 2023-10-06T02:05:57.805 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa 2023-10-06T02:05:57.805 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/dsa_core.ko 2023-10-06T02:05:57.805 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_brcm.ko 2023-10-06T02:05:57.805 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_dsa.ko 2023-10-06T02:05:57.805 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_none.ko 2023-10-06T02:05:57.805 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_trailer.ko 2023-10-06T02:05:57.805 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154 2023-10-06T02:05:57.805 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/6lowpan 2023-10-06T02:05:57.805 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-10-06T02:05:57.806 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/ieee802154.ko 2023-10-06T02:05:57.806 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/ieee802154_socket.ko 2023-10-06T02:05:57.806 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4 2023-10-06T02:05:57.806 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ah4.ko 2023-10-06T02:05:57.806 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/esp4.ko 2023-10-06T02:05:57.806 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/fou.ko 2023-10-06T02:05:57.806 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/gre.ko 2023-10-06T02:05:57.806 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/inet_diag.ko 2023-10-06T02:05:57.806 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_gre.ko 2023-10-06T02:05:57.806 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_tunnel.ko 2023-10-06T02:05:57.806 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_vti.ko 2023-10-06T02:05:57.807 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ipcomp.ko 2023-10-06T02:05:57.807 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ipip.ko 2023-10-06T02:05:57.807 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter 2023-10-06T02:05:57.807 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arp_tables.ko 2023-10-06T02:05:57.807 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-10-06T02:05:57.807 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-10-06T02:05:57.807 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-10-06T02:05:57.807 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-10-06T02:05:57.807 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-10-06T02:05:57.807 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-10-06T02:05:57.807 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-10-06T02:05:57.807 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-10-06T02:05:57.808 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-10-06T02:05:57.808 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_security.ko 2023-10-06T02:05:57.808 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-10-06T02:05:57.808 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-10-06T02:05:57.808 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-10-06T02:05:57.808 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-10-06T02:05:57.808 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-10-06T02:05:57.808 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-10-06T02:05:57.808 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-10-06T02:05:57.808 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-10-06T02:05:57.808 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-10-06T02:05:57.808 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-10-06T02:05:57.808 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_bic.ko 2023-10-06T02:05:57.809 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_dctcp.ko 2023-10-06T02:05:57.809 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_diag.ko 2023-10-06T02:05:57.809 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_highspeed.ko 2023-10-06T02:05:57.809 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_htcp.ko 2023-10-06T02:05:57.809 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_hybla.ko 2023-10-06T02:05:57.809 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_illinois.ko 2023-10-06T02:05:57.809 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_lp.ko 2023-10-06T02:05:57.809 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_scalable.ko 2023-10-06T02:05:57.809 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_vegas.ko 2023-10-06T02:05:57.809 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_veno.ko 2023-10-06T02:05:57.809 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_westwood.ko 2023-10-06T02:05:57.809 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_yeah.ko 2023-10-06T02:05:57.809 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tunnel4.ko 2023-10-06T02:05:57.810 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/udp_diag.ko 2023-10-06T02:05:57.810 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/udp_tunnel.ko 2023-10-06T02:05:57.810 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/xfrm4_tunnel.ko 2023-10-06T02:05:57.810 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6 2023-10-06T02:05:57.810 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ah6.ko 2023-10-06T02:05:57.810 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/esp6.ko 2023-10-06T02:05:57.810 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/fou6.ko 2023-10-06T02:05:57.810 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_tunnel.ko 2023-10-06T02:05:57.810 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-10-06T02:05:57.810 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_vti.ko 2023-10-06T02:05:57.810 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ipcomp6.ko 2023-10-06T02:05:57.810 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter 2023-10-06T02:05:57.811 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-10-06T02:05:57.811 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-10-06T02:05:57.811 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-10-06T02:05:57.811 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-10-06T02:05:57.811 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-10-06T02:05:57.811 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-10-06T02:05:57.811 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-10-06T02:05:57.811 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-10-06T02:05:57.811 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-10-06T02:05:57.811 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-10-06T02:05:57.811 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-10-06T02:05:57.811 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-10-06T02:05:57.811 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-10-06T02:05:57.812 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-10-06T02:05:57.812 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-10-06T02:05:57.812 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-10-06T02:05:57.812 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-10-06T02:05:57.812 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-10-06T02:05:57.812 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-10-06T02:05:57.812 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-10-06T02:05:57.812 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-10-06T02:05:57.812 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-10-06T02:05:57.812 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-10-06T02:05:57.812 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-10-06T02:05:57.812 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/sit.ko 2023-10-06T02:05:57.813 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/tunnel6.ko 2023-10-06T02:05:57.813 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/xfrm6_tunnel.ko 2023-10-06T02:05:57.813 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/key 2023-10-06T02:05:57.813 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/key/af_key.ko 2023-10-06T02:05:57.813 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp 2023-10-06T02:05:57.813 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_core.ko 2023-10-06T02:05:57.813 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_debugfs.ko 2023-10-06T02:05:57.813 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_eth.ko 2023-10-06T02:05:57.813 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ip.ko 2023-10-06T02:05:57.813 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ip6.ko 2023-10-06T02:05:57.813 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_netlink.ko 2023-10-06T02:05:57.813 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ppp.ko 2023-10-06T02:05:57.813 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/llc 2023-10-06T02:05:57.814 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/llc/llc.ko 2023-10-06T02:05:57.814 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mac802154 2023-10-06T02:05:57.814 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mac802154/mac802154.ko 2023-10-06T02:05:57.814 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mpls 2023-10-06T02:05:57.814 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mpls/mpls_gso.ko 2023-10-06T02:05:57.814 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter 2023-10-06T02:05:57.814 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset 2023-10-06T02:05:57.814 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set.ko 2023-10-06T02:05:57.814 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-10-06T02:05:57.814 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-10-06T02:05:57.814 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-10-06T02:05:57.814 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-10-06T02:05:57.814 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-10-06T02:05:57.815 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-10-06T02:05:57.815 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-10-06T02:05:57.815 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-10-06T02:05:57.815 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-10-06T02:05:57.815 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-10-06T02:05:57.815 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-10-06T02:05:57.815 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-10-06T02:05:57.815 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-10-06T02:05:57.815 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-10-06T02:05:57.815 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-10-06T02:05:57.815 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs 2023-10-06T02:05:57.815 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs.ko 2023-10-06T02:05:57.816 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-10-06T02:05:57.816 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-10-06T02:05:57.816 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-10-06T02:05:57.816 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-10-06T02:05:57.816 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-10-06T02:05:57.816 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-10-06T02:05:57.816 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-10-06T02:05:57.816 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-10-06T02:05:57.816 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-10-06T02:05:57.816 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-10-06T02:05:57.816 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-10-06T02:05:57.816 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-10-06T02:05:57.816 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-10-06T02:05:57.817 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conncount.ko 2023-10-06T02:05:57.817 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack.ko 2023-10-06T02:05:57.817 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-10-06T02:05:57.817 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-10-06T02:05:57.817 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-10-06T02:05:57.817 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_h323.ko 2023-10-06T02:05:57.817 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_irc.ko 2023-10-06T02:05:57.817 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-10-06T02:05:57.817 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-10-06T02:05:57.817 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-10-06T02:05:57.817 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_sane.ko 2023-10-06T02:05:57.817 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_sip.ko 2023-10-06T02:05:57.817 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-10-06T02:05:57.818 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-10-06T02:05:57.818 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_dup_netdev.ko 2023-10-06T02:05:57.818 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_flow_table.ko 2023-10-06T02:05:57.818 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_flow_table_inet.ko 2023-10-06T02:05:57.818 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_log_syslog.ko 2023-10-06T02:05:57.818 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat.ko 2023-10-06T02:05:57.818 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_amanda.ko 2023-10-06T02:05:57.818 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_ftp.ko 2023-10-06T02:05:57.818 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_irc.ko 2023-10-06T02:05:57.818 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_sip.ko 2023-10-06T02:05:57.818 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_tftp.ko 2023-10-06T02:05:57.818 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_synproxy_core.ko 2023-10-06T02:05:57.819 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_tables.ko 2023-10-06T02:05:57.819 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink.ko 2023-10-06T02:05:57.819 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_acct.ko 2023-10-06T02:05:57.819 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_log.ko 2023-10-06T02:05:57.819 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_osf.ko 2023-10-06T02:05:57.819 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_queue.ko 2023-10-06T02:05:57.819 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_chain_nat.ko 2023-10-06T02:05:57.819 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_compat.ko 2023-10-06T02:05:57.819 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_ct.ko 2023-10-06T02:05:57.819 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_dup_netdev.ko 2023-10-06T02:05:57.819 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_fib.ko 2023-10-06T02:05:57.819 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_fwd_netdev.ko 2023-10-06T02:05:57.819 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_hash.ko 2023-10-06T02:05:57.820 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_limit.ko 2023-10-06T02:05:57.820 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_log.ko 2023-10-06T02:05:57.820 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_masq.ko 2023-10-06T02:05:57.820 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_nat.ko 2023-10-06T02:05:57.820 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_numgen.ko 2023-10-06T02:05:57.820 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_queue.ko 2023-10-06T02:05:57.820 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_quota.ko 2023-10-06T02:05:57.820 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_redir.ko 2023-10-06T02:05:57.820 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_reject.ko 2023-10-06T02:05:57.820 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_reject_inet.ko 2023-10-06T02:05:57.820 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_socket.ko 2023-10-06T02:05:57.820 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_tunnel.ko 2023-10-06T02:05:57.820 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_AUDIT.ko 2023-10-06T02:05:57.821 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CHECKSUM.ko 2023-10-06T02:05:57.821 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CLASSIFY.ko 2023-10-06T02:05:57.821 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-10-06T02:05:57.821 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CT.ko 2023-10-06T02:05:57.821 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_DSCP.ko 2023-10-06T02:05:57.821 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_HL.ko 2023-10-06T02:05:57.821 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_HMARK.ko 2023-10-06T02:05:57.821 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_IDLETIMER.ko 2023-10-06T02:05:57.821 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_LED.ko 2023-10-06T02:05:57.821 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_LOG.ko 2023-10-06T02:05:57.821 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_MASQUERADE.ko 2023-10-06T02:05:57.821 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NETMAP.ko 2023-10-06T02:05:57.821 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NFLOG.ko 2023-10-06T02:05:57.822 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NFQUEUE.ko 2023-10-06T02:05:57.822 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_RATEEST.ko 2023-10-06T02:05:57.822 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_REDIRECT.ko 2023-10-06T02:05:57.822 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_SECMARK.ko 2023-10-06T02:05:57.822 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TCPMSS.ko 2023-10-06T02:05:57.822 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-10-06T02:05:57.822 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TEE.ko 2023-10-06T02:05:57.822 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TPROXY.ko 2023-10-06T02:05:57.822 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TRACE.ko 2023-10-06T02:05:57.822 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_addrtype.ko 2023-10-06T02:05:57.822 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_bpf.ko 2023-10-06T02:05:57.822 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cgroup.ko 2023-10-06T02:05:57.822 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cluster.ko 2023-10-06T02:05:57.823 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_comment.ko 2023-10-06T02:05:57.823 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connbytes.ko 2023-10-06T02:05:57.823 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connlabel.ko 2023-10-06T02:05:57.823 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connlimit.ko 2023-10-06T02:05:57.823 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connmark.ko 2023-10-06T02:05:57.823 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_conntrack.ko 2023-10-06T02:05:57.823 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cpu.ko 2023-10-06T02:05:57.823 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_dccp.ko 2023-10-06T02:05:57.823 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_devgroup.ko 2023-10-06T02:05:57.823 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_dscp.ko 2023-10-06T02:05:57.823 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ecn.ko 2023-10-06T02:05:57.823 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_esp.ko 2023-10-06T02:05:57.823 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_hashlimit.ko 2023-10-06T02:05:57.824 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_helper.ko 2023-10-06T02:05:57.824 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_hl.ko 2023-10-06T02:05:57.824 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ipcomp.ko 2023-10-06T02:05:57.824 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_iprange.ko 2023-10-06T02:05:57.824 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ipvs.ko 2023-10-06T02:05:57.824 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_l2tp.ko 2023-10-06T02:05:57.824 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_length.ko 2023-10-06T02:05:57.824 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_limit.ko 2023-10-06T02:05:57.824 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_mac.ko 2023-10-06T02:05:57.824 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_mark.ko 2023-10-06T02:05:57.824 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_multiport.ko 2023-10-06T02:05:57.825 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_nat.ko 2023-10-06T02:05:57.825 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_nfacct.ko 2023-10-06T02:05:57.825 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_osf.ko 2023-10-06T02:05:57.825 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_owner.ko 2023-10-06T02:05:57.825 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_physdev.ko 2023-10-06T02:05:57.825 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_pkttype.ko 2023-10-06T02:05:57.825 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_policy.ko 2023-10-06T02:05:57.825 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_quota.ko 2023-10-06T02:05:57.825 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_rateest.ko 2023-10-06T02:05:57.825 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_realm.ko 2023-10-06T02:05:57.825 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_recent.ko 2023-10-06T02:05:57.825 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_sctp.ko 2023-10-06T02:05:57.826 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_set.ko 2023-10-06T02:05:57.826 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_socket.ko 2023-10-06T02:05:57.826 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_state.ko 2023-10-06T02:05:57.826 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_statistic.ko 2023-10-06T02:05:57.826 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_string.ko 2023-10-06T02:05:57.826 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_tcpmss.ko 2023-10-06T02:05:57.826 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_time.ko 2023-10-06T02:05:57.826 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_u32.ko 2023-10-06T02:05:57.826 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netlink 2023-10-06T02:05:57.826 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netlink/netlink_diag.ko 2023-10-06T02:05:57.826 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/nsh 2023-10-06T02:05:57.826 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/nsh/nsh.ko 2023-10-06T02:05:57.826 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch 2023-10-06T02:05:57.827 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/openvswitch.ko 2023-10-06T02:05:57.827 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-geneve.ko 2023-10-06T02:05:57.827 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-gre.ko 2023-10-06T02:05:57.827 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-vxlan.ko 2023-10-06T02:05:57.827 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/packet 2023-10-06T02:05:57.827 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/packet/af_packet_diag.ko 2023-10-06T02:05:57.827 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds 2023-10-06T02:05:57.827 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds.ko 2023-10-06T02:05:57.827 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds_rdma.ko 2023-10-06T02:05:57.827 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds_tcp.ko 2023-10-06T02:05:57.827 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched 2023-10-06T02:05:57.827 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_csum.ko 2023-10-06T02:05:57.828 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_gact.ko 2023-10-06T02:05:57.828 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_ipt.ko 2023-10-06T02:05:57.828 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_mirred.ko 2023-10-06T02:05:57.828 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_nat.ko 2023-10-06T02:05:57.828 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_pedit.ko 2023-10-06T02:05:57.828 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_police.ko 2023-10-06T02:05:57.828 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_simple.ko 2023-10-06T02:05:57.828 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_skbedit.ko 2023-10-06T02:05:57.828 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_vlan.ko 2023-10-06T02:05:57.828 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_basic.ko 2023-10-06T02:05:57.828 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_bpf.ko 2023-10-06T02:05:57.828 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_flow.ko 2023-10-06T02:05:57.828 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_fw.ko 2023-10-06T02:05:57.829 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_route.ko 2023-10-06T02:05:57.829 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_u32.ko 2023-10-06T02:05:57.829 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_cmp.ko 2023-10-06T02:05:57.829 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_ipset.ko 2023-10-06T02:05:57.829 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_meta.ko 2023-10-06T02:05:57.829 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_nbyte.ko 2023-10-06T02:05:57.829 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_text.ko 2023-10-06T02:05:57.829 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_u32.ko 2023-10-06T02:05:57.829 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_choke.ko 2023-10-06T02:05:57.829 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_codel.ko 2023-10-06T02:05:57.829 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_drr.ko 2023-10-06T02:05:57.829 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_fq.ko 2023-10-06T02:05:57.830 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_gred.ko 2023-10-06T02:05:57.830 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_hfsc.ko 2023-10-06T02:05:57.830 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_hhf.ko 2023-10-06T02:05:57.830 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_htb.ko 2023-10-06T02:05:57.830 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_ingress.ko 2023-10-06T02:05:57.830 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_mqprio.ko 2023-10-06T02:05:57.830 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_mqprio_lib.ko 2023-10-06T02:05:57.830 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_multiq.ko 2023-10-06T02:05:57.830 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_netem.ko 2023-10-06T02:05:57.830 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_pie.ko 2023-10-06T02:05:57.830 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_plug.ko 2023-10-06T02:05:57.830 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_prio.ko 2023-10-06T02:05:57.831 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_qfq.ko 2023-10-06T02:05:57.831 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_red.ko 2023-10-06T02:05:57.831 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_sfb.ko 2023-10-06T02:05:57.831 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_sfq.ko 2023-10-06T02:05:57.831 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_tbf.ko 2023-10-06T02:05:57.831 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_teql.ko 2023-10-06T02:05:57.831 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp 2023-10-06T02:05:57.831 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp/sctp.ko 2023-10-06T02:05:57.831 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp/sctp_diag.ko 2023-10-06T02:05:57.831 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc 2023-10-06T02:05:57.831 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss 2023-10-06T02:05:57.831 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-10-06T02:05:57.832 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-10-06T02:05:57.832 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/sunrpc.ko 2023-10-06T02:05:57.832 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/xprtrdma 2023-10-06T02:05:57.832 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-10-06T02:05:57.832 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc 2023-10-06T02:05:57.832 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc/diag.ko 2023-10-06T02:05:57.832 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc/tipc.ko 2023-10-06T02:05:57.832 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/unix 2023-10-06T02:05:57.832 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/unix/unix_diag.ko 2023-10-06T02:05:57.832 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock 2023-10-06T02:05:57.832 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-10-06T02:05:57.832 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-10-06T02:05:57.832 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock.ko 2023-10-06T02:05:57.833 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock_diag.ko 2023-10-06T02:05:57.833 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock_loopback.ko 2023-10-06T02:05:57.833 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/xfrm 2023-10-06T02:05:57.833 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/xfrm/xfrm_ipcomp.ko 2023-10-06T02:05:57.833 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security 2023-10-06T02:05:57.833 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys 2023-10-06T02:05:57.833 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/encrypted-keys 2023-10-06T02:05:57.833 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-10-06T02:05:57.833 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/trusted-keys 2023-10-06T02:05:57.833 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/trusted-keys/trusted.ko 2023-10-06T02:05:57.833 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt 2023-10-06T02:05:57.833 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt/lib 2023-10-06T02:05:57.834 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt/lib/irqbypass.ko 2023-10-06T02:05:57.834 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.alias 2023-10-06T02:05:57.834 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.alias.bin 2023-10-06T02:05:57.834 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin 2023-10-06T02:05:57.834 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin.bin 2023-10-06T02:05:57.834 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin.modinfo 2023-10-06T02:05:57.834 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.dep 2023-10-06T02:05:57.834 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.dep.bin 2023-10-06T02:05:57.834 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.devname 2023-10-06T02:05:57.834 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.order 2023-10-06T02:05:57.834 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.softdep 2023-10-06T02:05:57.835 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.symbols 2023-10-06T02:05:57.835 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.symbols.bin 2023-10-06T02:05:57.835 DEBUG:teuthology.task.kernel:get_image_version: 6.5.0-rc7-gfef64f93da2b 2023-10-06T02:05:57.835 DEBUG:teuthology.orchestra.run.smithi031:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-10-06T02:05:57.857 INFO:teuthology.orchestra.run.smithi031.stdout:postinstall scriptlet (using /bin/sh): 2023-10-06T02:05:57.857 INFO:teuthology.orchestra.run.smithi031.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b -a -r /boot/System.map-6.5.0-rc7-gfef64f93da2b ]; then 2023-10-06T02:05:57.857 INFO:teuthology.orchestra.run.smithi031.stdout:cp /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b /boot/.vmlinuz-6.5.0-rc7-gfef64f93da2b-rpm 2023-10-06T02:05:57.858 INFO:teuthology.orchestra.run.smithi031.stdout:cp /boot/System.map-6.5.0-rc7-gfef64f93da2b /boot/.System.map-6.5.0-rc7-gfef64f93da2b-rpm 2023-10-06T02:05:57.858 INFO:teuthology.orchestra.run.smithi031.stdout:rm -f /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b /boot/System.map-6.5.0-rc7-gfef64f93da2b 2023-10-06T02:05:57.858 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/installkernel 6.5.0-rc7-gfef64f93da2b /boot/.vmlinuz-6.5.0-rc7-gfef64f93da2b-rpm /boot/.System.map-6.5.0-rc7-gfef64f93da2b-rpm 2023-10-06T02:05:57.858 INFO:teuthology.orchestra.run.smithi031.stdout:rm -f /boot/.vmlinuz-6.5.0-rc7-gfef64f93da2b-rpm /boot/.System.map-6.5.0-rc7-gfef64f93da2b-rpm 2023-10-06T02:05:57.858 INFO:teuthology.orchestra.run.smithi031.stdout:fi 2023-10-06T02:05:57.858 INFO:teuthology.orchestra.run.smithi031.stdout:preuninstall scriptlet (using /bin/sh): 2023-10-06T02:05:57.858 INFO:teuthology.orchestra.run.smithi031.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-10-06T02:05:57.858 INFO:teuthology.orchestra.run.smithi031.stdout:new-kernel-pkg --remove 6.5.0-rc7-gfef64f93da2b --rminitrd --initrdfile=/boot/initramfs-6.5.0-rc7-gfef64f93da2b.img 2023-10-06T02:05:57.858 INFO:teuthology.orchestra.run.smithi031.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-10-06T02:05:57.858 INFO:teuthology.orchestra.run.smithi031.stdout:kernel-install remove 6.5.0-rc7-gfef64f93da2b 2023-10-06T02:05:57.858 INFO:teuthology.orchestra.run.smithi031.stdout:fi 2023-10-06T02:05:57.858 INFO:teuthology.orchestra.run.smithi031.stdout:postuninstall scriptlet (using /bin/sh): 2023-10-06T02:05:57.859 INFO:teuthology.orchestra.run.smithi031.stdout:if [ -x /sbin/update-bootloader ]; then 2023-10-06T02:05:57.859 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/update-bootloader --remove 6.5.0-rc7-gfef64f93da2b 2023-10-06T02:05:57.859 INFO:teuthology.orchestra.run.smithi031.stdout:fi 2023-10-06T02:05:57.861 DEBUG:teuthology.orchestra.run.smithi031:> sudo rpm -qi grub2-tools 2023-10-06T02:05:57.934 INFO:teuthology.orchestra.run.smithi031.stdout:Name : grub2-tools 2023-10-06T02:05:57.934 INFO:teuthology.orchestra.run.smithi031.stdout:Epoch : 1 2023-10-06T02:05:57.934 INFO:teuthology.orchestra.run.smithi031.stdout:Version : 2.02 2023-10-06T02:05:57.934 INFO:teuthology.orchestra.run.smithi031.stdout:Release : 99.el8_4.1 2023-10-06T02:05:57.934 INFO:teuthology.orchestra.run.smithi031.stdout:Architecture: x86_64 2023-10-06T02:05:57.934 INFO:teuthology.orchestra.run.smithi031.stdout:Install Date: Wed 21 Dec 2022 01:02:32 PM UTC 2023-10-06T02:05:57.934 INFO:teuthology.orchestra.run.smithi031.stdout:Group : System Environment/Base 2023-10-06T02:05:57.935 INFO:teuthology.orchestra.run.smithi031.stdout:Size : 9264631 2023-10-06T02:05:57.935 INFO:teuthology.orchestra.run.smithi031.stdout:License : GPLv3+ 2023-10-06T02:05:57.935 INFO:teuthology.orchestra.run.smithi031.stdout:Signature : RSA/SHA256, Thu 09 Sep 2021 09:41:55 AM UTC, Key ID 199e2f91fd431d51 2023-10-06T02:05:57.935 INFO:teuthology.orchestra.run.smithi031.stdout:Source RPM : grub2-2.02-99.el8_4.1.src.rpm 2023-10-06T02:05:57.935 INFO:teuthology.orchestra.run.smithi031.stdout:Build Date : Tue 18 May 2021 07:46:34 AM UTC 2023-10-06T02:05:57.935 INFO:teuthology.orchestra.run.smithi031.stdout:Build Host : x86-vm-07.build.eng.bos.redhat.com 2023-10-06T02:05:57.935 INFO:teuthology.orchestra.run.smithi031.stdout:Relocations : (not relocatable) 2023-10-06T02:05:57.935 INFO:teuthology.orchestra.run.smithi031.stdout:Packager : Red Hat, Inc. 2023-10-06T02:05:57.935 INFO:teuthology.orchestra.run.smithi031.stdout:Vendor : Red Hat, Inc. 2023-10-06T02:05:57.935 INFO:teuthology.orchestra.run.smithi031.stdout:URL : http://www.gnu.org/software/grub/ 2023-10-06T02:05:57.935 INFO:teuthology.orchestra.run.smithi031.stdout:Summary : Support tools for GRUB. 2023-10-06T02:05:57.935 INFO:teuthology.orchestra.run.smithi031.stdout:Description : 2023-10-06T02:05:57.935 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:05:57.936 INFO:teuthology.orchestra.run.smithi031.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-10-06T02:05:57.936 INFO:teuthology.orchestra.run.smithi031.stdout:customizable bootloader with modular architecture. It supports a rich 2023-10-06T02:05:57.936 INFO:teuthology.orchestra.run.smithi031.stdout:variety of kernel formats, file systems, computer architectures and 2023-10-06T02:05:57.936 INFO:teuthology.orchestra.run.smithi031.stdout:hardware devices. 2023-10-06T02:05:57.936 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:05:57.936 INFO:teuthology.orchestra.run.smithi031.stdout:This subpackage provides tools for support of all platforms. 2023-10-06T02:05:57.937 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-10-06T02:05:57.937 INFO:teuthology.task.kernel:Updating grub on smithi031 to boot 6.5.0-rc7-gfef64f93da2b 2023-10-06T02:05:57.937 DEBUG:teuthology.orchestra.run.smithi031:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-10-06T02:06:02.226 INFO:teuthology.orchestra.run.smithi031.stderr:Generating grub configuration file ... 2023-10-06T02:06:04.658 INFO:teuthology.orchestra.run.smithi031.stderr:done 2023-10-06T02:06:04.660 DEBUG:teuthology.orchestra.run.smithi031:> mktemp 2023-10-06T02:06:04.688 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/tmp.q0x32Ci17l 2023-10-06T02:06:04.689 DEBUG:teuthology.orchestra.run.smithi031:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.q0x32Ci17l 2023-10-06T02:06:04.759 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 0666 /tmp/tmp.q0x32Ci17l 2023-10-06T02:06:04.918 DEBUG:teuthology.orchestra.remote:smithi031:/tmp/tmp.q0x32Ci17l is 6KB 2023-10-06T02:06:04.966 DEBUG:teuthology.orchestra.run.smithi031:> rm -fr /tmp/tmp.q0x32Ci17l 2023-10-06T02:06:04.980 DEBUG:teuthology.orchestra.run.smithi031:> sudo /bin/ls /boot/loader/entries || true 2023-10-06T02:06:05.042 INFO:teuthology.orchestra.run.smithi031.stdout:298e228f3b5e401e8ac0f5ae159a2e06-0-rescue.conf 2023-10-06T02:06:05.043 INFO:teuthology.orchestra.run.smithi031.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-305.25.1.el8_4.x86_64.conf 2023-10-06T02:06:05.043 INFO:teuthology.orchestra.run.smithi031.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-425.3.1.el8.x86_64.conf 2023-10-06T02:06:05.043 INFO:teuthology.orchestra.run.smithi031.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-80.el8.x86_64.conf 2023-10-06T02:06:05.043 INFO:teuthology.orchestra.run.smithi031.stdout:298e228f3b5e401e8ac0f5ae159a2e06-6.5.0-rc7-gfef64f93da2b.conf 2023-10-06T02:06:05.044 DEBUG:teuthology.orchestra.run.smithi031:> sudo grub2-set-default 298e228f3b5e401e8ac0f5ae159a2e06-6.5.0-rc7-gfef64f93da2b 2023-10-06T02:06:05.198 DEBUG:teuthology.orchestra.run.smithi031:> sudo shutdown -r now 2023-10-06T02:06:17.682 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2023-10-06T02:06:17.682 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi062.front.sepia.ceph.com' 2023-10-06T02:06:17.683 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T02:06:35.218 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2023-10-06T02:06:35.218 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2023-10-06T02:06:35.219 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T02:06:36.158 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.62 2023-10-06T02:06:45.166 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi062.front.sepia.ceph.com' 2023-10-06T02:06:45.167 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T02:06:48.218 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.62 2023-10-06T02:06:53.566 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.31 2023-10-06T02:07:00.224 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi062.front.sepia.ceph.com' 2023-10-06T02:07:00.225 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T02:07:02.570 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2023-10-06T02:07:02.570 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T02:07:18.654 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.62 2023-10-06T02:07:20.958 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.31 2023-10-06T02:07:32.970 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2023-10-06T02:07:32.971 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T02:07:33.656 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi062.front.sepia.ceph.com' 2023-10-06T02:07:33.656 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T02:07:34.303 DEBUG:teuthology.orchestra.run.smithi062:> true 2023-10-06T02:07:34.702 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi062.front.sepia.ceph.com' 2023-10-06T02:07:34.702 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.5.0-rc7-gfef64f93da2b"... 2023-10-06T02:07:34.703 DEBUG:teuthology.orchestra.run.smithi062:> uname -r 2023-10-06T02:07:34.738 INFO:teuthology.orchestra.run.smithi062.stdout:6.5.0-rc7-gfef64f93da2b 2023-10-06T02:07:34.739 DEBUG:teuthology.task.kernel:current kernel version is 6.5.0-rc7-gfef64f93da2b vs 6.5.0-rc7-gfef64f93da2b 2023-10-06T02:07:34.739 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-06T02:07:34.739 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-10-06T02:07:35.739 DEBUG:teuthology.parallel:result is None 2023-10-06T02:07:35.889 DEBUG:teuthology.orchestra.run.smithi031:> true 2023-10-06T02:07:36.302 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi031.front.sepia.ceph.com' 2023-10-06T02:07:36.302 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.5.0-rc7-gfef64f93da2b"... 2023-10-06T02:07:36.303 DEBUG:teuthology.orchestra.run.smithi031:> uname -r 2023-10-06T02:07:36.339 INFO:teuthology.orchestra.run.smithi031.stdout:6.5.0-rc7-gfef64f93da2b 2023-10-06T02:07:36.339 DEBUG:teuthology.task.kernel:current kernel version is 6.5.0-rc7-gfef64f93da2b vs 6.5.0-rc7-gfef64f93da2b 2023-10-06T02:07:36.339 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-06T02:07:36.339 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-10-06T02:07:37.340 DEBUG:teuthology.parallel:result is None 2023-10-06T02:07:37.340 INFO:teuthology.run_tasks:Running task internal.base... 2023-10-06T02:07:37.348 INFO:teuthology.task.internal:Creating test directory... 2023-10-06T02:07:37.348 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-06T02:07:37.354 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-06T02:07:37.391 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-10-06T02:07:37.400 INFO:teuthology.run_tasks:Running task internal.archive... 2023-10-06T02:07:37.459 INFO:teuthology.task.internal:Creating archive directory... 2023-10-06T02:07:37.460 DEBUG:teuthology.orchestra.run.smithi031:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-06T02:07:37.464 DEBUG:teuthology.orchestra.run.smithi062:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-06T02:07:37.510 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-10-06T02:07:37.519 INFO:teuthology.task.internal:Enabling coredump saving... 2023-10-06T02:07:37.519 DEBUG:teuthology.orchestra.run.smithi031:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-06T02:07:37.549 DEBUG:teuthology.orchestra.run.smithi062:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-06T02:07:37.676 INFO:teuthology.orchestra.run.smithi031.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T02:07:37.726 INFO:teuthology.orchestra.run.smithi031.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T02:07:37.732 INFO:teuthology.orchestra.run.smithi062.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T02:07:37.769 INFO:teuthology.orchestra.run.smithi062.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T02:07:37.772 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-10-06T02:07:37.779 INFO:teuthology.task.internal:Configuring sudo... 2023-10-06T02:07:37.779 DEBUG:teuthology.orchestra.run.smithi031:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-06T02:07:37.784 DEBUG:teuthology.orchestra.run.smithi062:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-06T02:07:37.851 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-10-06T02:07:37.860 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-10-06T02:07:37.860 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-06T02:07:37.864 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-06T02:07:37.900 DEBUG:teuthology.orchestra.run.smithi031:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-06T02:07:38.000 DEBUG:teuthology.orchestra.run.smithi031:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-06T02:07:38.079 DEBUG:teuthology.orchestra.run.smithi031:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-06T02:07:38.146 DEBUG:teuthology.orchestra.run.smithi031:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-06T02:07:38.224 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T02:07:38.224 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-06T02:07:38.292 DEBUG:teuthology.orchestra.run.smithi062:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-06T02:07:38.360 DEBUG:teuthology.orchestra.run.smithi062:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-06T02:07:38.489 DEBUG:teuthology.orchestra.run.smithi062:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-06T02:07:38.554 DEBUG:teuthology.orchestra.run.smithi062:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-06T02:07:38.630 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-10-06T02:07:38.630 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-06T02:07:38.706 DEBUG:teuthology.orchestra.run.smithi031:> sudo service rsyslog restart 2023-10-06T02:07:38.711 DEBUG:teuthology.orchestra.run.smithi062:> sudo service rsyslog restart 2023-10-06T02:07:38.838 INFO:teuthology.orchestra.run.smithi031.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-06T02:07:38.862 INFO:teuthology.orchestra.run.smithi062.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-06T02:07:39.292 INFO:teuthology.run_tasks:Running task internal.timer... 2023-10-06T02:07:39.304 INFO:teuthology.task.internal:Starting timer... 2023-10-06T02:07:39.305 INFO:teuthology.run_tasks:Running task pcp... 2023-10-06T02:07:39.340 INFO:teuthology.run_tasks:Running task selinux... 2023-10-06T02:07:39.381 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-10-06T02:07:39.382 DEBUG:teuthology.orchestra.run.smithi031:> sudo service auditd rotate 2023-10-06T02:07:39.516 INFO:teuthology.orchestra.run.smithi031.stdout:Rotating logs: [ OK ] 2023-10-06T02:07:39.520 DEBUG:teuthology.orchestra.run.smithi062:> sudo service auditd rotate 2023-10-06T02:07:39.618 INFO:teuthology.orchestra.run.smithi062.stdout:Rotating logs: [ OK ] 2023-10-06T02:07:39.622 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-10-06T02:07:39.622 DEBUG:teuthology.orchestra.run.smithi031:> /usr/sbin/getenforce 2023-10-06T02:07:39.699 INFO:teuthology.orchestra.run.smithi031.stdout:Permissive 2023-10-06T02:07:39.699 DEBUG:teuthology.orchestra.run.smithi062:> /usr/sbin/getenforce 2023-10-06T02:07:39.748 INFO:teuthology.orchestra.run.smithi062.stdout:Permissive 2023-10-06T02:07:39.748 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi031.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi062.front.sepia.ceph.com': 'permissive'} 2023-10-06T02:07:39.748 DEBUG:teuthology.orchestra.run.smithi031:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-06T02:07:39.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T02:07:39.811 DEBUG:teuthology.orchestra.run.smithi062:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-06T02:07:39.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T02:07:39.874 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-10-06T02:07:39.874 DEBUG:teuthology.orchestra.run.smithi031:> sudo /usr/sbin/setenforce permissive 2023-10-06T02:07:39.937 DEBUG:teuthology.orchestra.run.smithi062:> sudo /usr/sbin/setenforce permissive 2023-10-06T02:07:39.998 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-10-06T02:07:40.009 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-10-06T02:07:40.009 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-10-06T02:07:40.025 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-10-06T02:07:40.026 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi031.front.sepia.ceph.com,smithi062.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-10-06T02:19:29.703 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi031.front.sepia.ceph.com'), Remote(name='ubuntu@smithi062.front.sepia.ceph.com')] 2023-10-06T02:19:29.704 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2023-10-06T02:19:29.705 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T02:19:29.793 DEBUG:teuthology.orchestra.run.smithi031:> true 2023-10-06T02:19:29.901 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi031.front.sepia.ceph.com' 2023-10-06T02:19:29.901 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi062.front.sepia.ceph.com' 2023-10-06T02:19:29.902 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T02:19:29.989 DEBUG:teuthology.orchestra.run.smithi062:> true 2023-10-06T02:19:30.094 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi062.front.sepia.ceph.com' 2023-10-06T02:19:30.095 INFO:teuthology.run_tasks:Running task clock... 2023-10-06T02:19:30.106 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-10-06T02:19:30.106 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-06T02:19:30.106 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-06T02:19:30.111 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-06T02:19:30.111 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-06T02:19:30.175 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-06T02:19:30.203 INFO:teuthology.orchestra.run.smithi062.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-06T02:19:30.210 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-06T02:19:30.243 INFO:teuthology.orchestra.run.smithi062.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-06T02:19:30.272 INFO:teuthology.orchestra.run.smithi031.stderr:sudo: ntpd: command not found 2023-10-06T02:19:30.299 INFO:teuthology.orchestra.run.smithi031.stdout:506 Cannot talk to daemon 2023-10-06T02:19:30.300 INFO:teuthology.orchestra.run.smithi062.stderr:sudo: ntpd: command not found 2023-10-06T02:19:30.328 INFO:teuthology.orchestra.run.smithi062.stdout:506 Cannot talk to daemon 2023-10-06T02:19:30.333 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-06T02:19:30.363 INFO:teuthology.orchestra.run.smithi062.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-06T02:19:30.368 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-06T02:19:30.397 INFO:teuthology.orchestra.run.smithi062.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-06T02:19:30.453 INFO:teuthology.orchestra.run.smithi031.stderr:bash: ntpq: command not found 2023-10-06T02:19:30.459 INFO:teuthology.orchestra.run.smithi031.stdout:210 Number of sources = 4 2023-10-06T02:19:30.459 INFO:teuthology.orchestra.run.smithi031.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-06T02:19:30.459 INFO:teuthology.orchestra.run.smithi031.stdout:=============================================================================== 2023-10-06T02:19:30.459 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T02:19:30.459 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T02:19:30.459 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T02:19:30.459 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T02:19:30.485 INFO:teuthology.orchestra.run.smithi062.stderr:bash: ntpq: command not found 2023-10-06T02:19:30.490 INFO:teuthology.orchestra.run.smithi062.stdout:210 Number of sources = 4 2023-10-06T02:19:30.490 INFO:teuthology.orchestra.run.smithi062.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-06T02:19:30.490 INFO:teuthology.orchestra.run.smithi062.stdout:=============================================================================== 2023-10-06T02:19:30.491 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T02:19:30.491 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T02:19:30.491 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T02:19:30.491 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T02:19:30.492 INFO:teuthology.run_tasks:Running task install... 2023-10-06T02:19:30.501 DEBUG:teuthology.task.install:project ceph 2023-10-06T02:19:30.501 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '736ac5ba72cb47af7d6f6666b1e100f7e20a20e2'}} 2023-10-06T02:19:30.501 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '736ac5ba72cb47af7d6f6666b1e100f7e20a20e2'} 2023-10-06T02:19:30.501 INFO:teuthology.task.install:Using flavor: default 2023-10-06T02:19:30.507 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-10-06T02:19:30.507 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-10-06T02:19:30.508 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '736ac5ba72cb47af7d6f6666b1e100f7e20a20e2', 'tag': None, 'wait_for_package': False} 2023-10-06T02:19:30.508 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=736ac5ba72cb47af7d6f6666b1e100f7e20a20e2 2023-10-06T02:19:30.510 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '736ac5ba72cb47af7d6f6666b1e100f7e20a20e2', 'tag': None, 'wait_for_package': False} 2023-10-06T02:19:30.510 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=736ac5ba72cb47af7d6f6666b1e100f7e20a20e2 2023-10-06T02:19:30.657 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/centos/8/flavors/default/ 2023-10-06T02:19:30.657 INFO:teuthology.task.install.rpm:Package version is 17.2.6-1296.g736ac5ba 2023-10-06T02:19:30.662 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/centos/8/flavors/default/ 2023-10-06T02:19:30.663 INFO:teuthology.task.install.rpm:Package version is 17.2.6-1296.g736ac5ba 2023-10-06T02:19:30.768 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-06T02:19:30.768 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T02:19:30.769 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-06T02:19:30.832 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2023-10-06T02:19:30.832 DEBUG:teuthology.orchestra.run.smithi031:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-06T02:19:30.922 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-06T02:19:30.922 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-10-06T02:19:30.922 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-06T02:19:30.935 DEBUG:teuthology.orchestra.run.smithi031:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-06T02:19:30.986 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2023-10-06T02:19:30.986 DEBUG:teuthology.orchestra.run.smithi062:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-06T02:19:31.025 DEBUG:teuthology.orchestra.run.smithi031:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-06T02:19:31.087 INFO:teuthology.orchestra.run.smithi031.stdout:check_obsoletes = 1 2023-10-06T02:19:31.089 DEBUG:teuthology.orchestra.run.smithi062:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-06T02:19:31.090 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum clean all 2023-10-06T02:19:31.180 DEBUG:teuthology.orchestra.run.smithi062:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-06T02:19:31.241 INFO:teuthology.orchestra.run.smithi062.stdout:check_obsoletes = 1 2023-10-06T02:19:31.244 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum clean all 2023-10-06T02:19:31.583 INFO:teuthology.orchestra.run.smithi031.stdout:Updating Subscription Management repositories. 2023-10-06T02:19:31.735 INFO:teuthology.orchestra.run.smithi062.stdout:Updating Subscription Management repositories. 2023-10-06T02:19:31.981 INFO:teuthology.orchestra.run.smithi031.stdout:44 files removed 2023-10-06T02:19:32.040 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2023-10-06T02:19:32.131 INFO:teuthology.orchestra.run.smithi062.stdout:44 files removed 2023-10-06T02:19:32.196 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2023-10-06T02:19:32.534 INFO:teuthology.orchestra.run.smithi031.stdout:Updating Subscription Management repositories. 2023-10-06T02:19:32.690 INFO:teuthology.orchestra.run.smithi062.stdout:Updating Subscription Management repositories. 2023-10-06T02:19:33.874 INFO:teuthology.orchestra.run.smithi031.stdout:ceph packages for x86_64 98 kB/s | 78 kB 00:00 2023-10-06T02:19:34.025 INFO:teuthology.orchestra.run.smithi062.stdout:ceph packages for x86_64 98 kB/s | 78 kB 00:00 2023-10-06T02:19:34.314 INFO:teuthology.orchestra.run.smithi031.stdout:ceph noarch packages 23 kB/s | 9.2 kB 00:00 2023-10-06T02:19:34.464 INFO:teuthology.orchestra.run.smithi062.stdout:ceph noarch packages 23 kB/s | 9.2 kB 00:00 2023-10-06T02:19:34.898 INFO:teuthology.orchestra.run.smithi031.stdout:ceph source packages 2.6 kB/s | 1.4 kB 00:00 2023-10-06T02:19:35.034 INFO:teuthology.orchestra.run.smithi031.stdout:Copr repo for python3-asyncssh owned by ceph 34 kB/s | 3.5 kB 00:00 2023-10-06T02:19:35.120 INFO:teuthology.orchestra.run.smithi062.stdout:ceph source packages 2.3 kB/s | 1.4 kB 00:00 2023-10-06T02:19:35.279 INFO:teuthology.orchestra.run.smithi062.stdout:Copr repo for python3-asyncssh owned by ceph 28 kB/s | 3.5 kB 00:00 2023-10-06T02:19:35.744 INFO:teuthology.orchestra.run.smithi031.stdout:Extra Packages for Enterprise Linux 24 MB/s | 16 MB 00:00 2023-10-06T02:19:38.407 INFO:teuthology.orchestra.run.smithi062.stdout:Extra Packages for Enterprise Linux 5.1 MB/s | 16 MB 00:03 2023-10-06T02:19:39.431 INFO:teuthology.orchestra.run.smithi031.stdout:lab-extras 4.2 MB/s | 24 kB 00:00 2023-10-06T02:19:39.960 INFO:teuthology.orchestra.run.smithi031.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 75 MB/s | 33 MB 00:00 2023-10-06T02:19:42.117 INFO:teuthology.orchestra.run.smithi062.stdout:lab-extras 4.2 MB/s | 24 kB 00:00 2023-10-06T02:19:42.639 INFO:teuthology.orchestra.run.smithi062.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 76 MB/s | 33 MB 00:00 2023-10-06T02:19:45.974 INFO:teuthology.orchestra.run.smithi031.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 75 MB/s | 38 MB 00:00 2023-10-06T02:19:48.527 INFO:teuthology.orchestra.run.smithi062.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 79 MB/s | 38 MB 00:00 2023-10-06T02:19:55.334 INFO:teuthology.orchestra.run.smithi031.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-06T02:19:55.334 INFO:teuthology.orchestra.run.smithi031.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-06T02:19:55.346 INFO:teuthology.orchestra.run.smithi031.stdout:Package NetworkManager-1:1.30.0-13.el8_4.x86_64 is already installed. 2023-10-06T02:19:55.346 INFO:teuthology.orchestra.run.smithi031.stdout:Package iproute-5.9.0-4.el8.x86_64 is already installed. 2023-10-06T02:19:55.347 INFO:teuthology.orchestra.run.smithi031.stdout:Package util-linux-2.32.1-27.el8.x86_64 is already installed. 2023-10-06T02:19:55.348 INFO:teuthology.orchestra.run.smithi031.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-10-06T02:19:55.350 INFO:teuthology.orchestra.run.smithi031.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-10-06T02:19:55.353 INFO:teuthology.orchestra.run.smithi031.stdout:Package xfsdump-3.1.8-2.el8.x86_64 is already installed. 2023-10-06T02:19:55.353 INFO:teuthology.orchestra.run.smithi031.stdout:Package xfsprogs-5.0.0-8.el8.x86_64 is already installed. 2023-10-06T02:19:55.909 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2023-10-06T02:19:55.921 INFO:teuthology.orchestra.run.smithi031.stdout:====================================================================================================================== 2023-10-06T02:19:55.921 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2023-10-06T02:19:55.921 INFO:teuthology.orchestra.run.smithi031.stdout:====================================================================================================================== 2023-10-06T02:19:55.921 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2023-10-06T02:19:55.921 INFO:teuthology.orchestra.run.smithi031.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-10-06T02:19:55.922 INFO:teuthology.orchestra.run.smithi031.stdout: ceph x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 6.4 k 2023-10-06T02:19:55.922 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-base x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 5.2 M 2023-10-06T02:19:55.922 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-fuse x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 876 k 2023-10-06T02:19:55.922 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-immutable-object-cache x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 162 k 2023-10-06T02:19:55.922 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 1.6 M 2023-10-06T02:19:55.922 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-cephadm noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 121 k 2023-10-06T02:19:55.922 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-dashboard noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 1.7 M 2023-10-06T02:19:55.922 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 7.4 M 2023-10-06T02:19:55.922 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-rook noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 47 k 2023-10-06T02:19:55.922 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-radosgw x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 17 M 2023-10-06T02:19:55.923 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-test x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 52 M 2023-10-06T02:19:55.923 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-volume noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 267 k 2023-10-06T02:19:55.923 INFO:teuthology.orchestra.run.smithi031.stdout: cephadm noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 80 k 2023-10-06T02:19:55.923 INFO:teuthology.orchestra.run.smithi031.stdout: cephfs-mirror x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 255 k 2023-10-06T02:19:55.923 INFO:teuthology.orchestra.run.smithi031.stdout: cephfs-top noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 18 k 2023-10-06T02:19:55.923 INFO:teuthology.orchestra.run.smithi031.stdout: elfutils-libelf-devel x86_64 0.182-3.el8 rhel-8-for-x86_64-baseos-rpms 59 k 2023-10-06T02:19:55.923 INFO:teuthology.orchestra.run.smithi031.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-10-06T02:19:55.923 INFO:teuthology.orchestra.run.smithi031.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-10-06T02:19:55.923 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs-devel x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 25 k 2023-10-06T02:19:55.923 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs2 x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 797 k 2023-10-06T02:19:55.923 INFO:teuthology.orchestra.run.smithi031.stdout: librados-devel x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 130 k 2023-10-06T02:19:55.924 INFO:teuthology.orchestra.run.smithi031.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-10-06T02:19:55.924 INFO:teuthology.orchestra.run.smithi031.stdout: libuuid-devel x86_64 2.32.1-27.el8 rhel-8-for-x86_64-baseos-rpms 97 k 2023-10-06T02:19:55.924 INFO:teuthology.orchestra.run.smithi031.stdout: openssl-devel x86_64 1:1.1.1g-15.el8_3 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-10-06T02:19:55.924 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cephfs x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 190 k 2023-10-06T02:19:55.924 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rados x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 371 k 2023-10-06T02:19:55.924 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rbd x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 358 k 2023-10-06T02:19:55.924 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rgw x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 107 k 2023-10-06T02:19:55.924 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-fuse x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 90 k 2023-10-06T02:19:55.924 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-mirror x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 3.5 M 2023-10-06T02:19:55.924 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-nbd x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 173 k 2023-10-06T02:19:55.924 INFO:teuthology.orchestra.run.smithi031.stdout: xfsprogs-devel x86_64 5.0.0-8.el8 rhel-8-for-x86_64-baseos-rpms 70 k 2023-10-06T02:19:55.925 INFO:teuthology.orchestra.run.smithi031.stdout:Upgrading: 2023-10-06T02:19:55.925 INFO:teuthology.orchestra.run.smithi031.stdout: librados2 x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 3.5 M 2023-10-06T02:19:55.925 INFO:teuthology.orchestra.run.smithi031.stdout: librbd1 x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 3.6 M 2023-10-06T02:19:55.925 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2023-10-06T02:19:55.925 INFO:teuthology.orchestra.run.smithi031.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-10-06T02:19:55.925 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-common x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 23 M 2023-10-06T02:19:55.925 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-grafana-dashboards noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 23 k 2023-10-06T02:19:55.925 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mds x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 2.3 M 2023-10-06T02:19:55.925 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-modules-core noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 238 k 2023-10-06T02:19:55.925 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mon x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 3.9 M 2023-10-06T02:19:55.925 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-osd x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 17 M 2023-10-06T02:19:55.925 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-prometheus-alerts noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 14 k 2023-10-06T02:19:55.926 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-selinux x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 24 k 2023-10-06T02:19:55.926 INFO:teuthology.orchestra.run.smithi031.stdout: fuse x86_64 2.9.7-12.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-10-06T02:19:55.926 INFO:teuthology.orchestra.run.smithi031.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-10-06T02:19:55.926 INFO:teuthology.orchestra.run.smithi031.stdout: jq x86_64 1.5-12.el8 rhel-8-for-x86_64-appstream-rpms 161 k 2023-10-06T02:19:55.926 INFO:teuthology.orchestra.run.smithi031.stdout: keyutils-libs-devel x86_64 1.5.10-6.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-10-06T02:19:55.926 INFO:teuthology.orchestra.run.smithi031.stdout: krb5-devel x86_64 1.18.2-8.3.el8_4 rhel-8-for-x86_64-baseos-rpms 559 k 2023-10-06T02:19:55.926 INFO:teuthology.orchestra.run.smithi031.stdout: libcephsqlite x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 174 k 2023-10-06T02:19:55.926 INFO:teuthology.orchestra.run.smithi031.stdout: libcom_err-devel x86_64 1.45.6-1.el8 rhel-8-for-x86_64-baseos-rpms 38 k 2023-10-06T02:19:55.926 INFO:teuthology.orchestra.run.smithi031.stdout: libgfortran x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 639 k 2023-10-06T02:19:55.926 INFO:teuthology.orchestra.run.smithi031.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-10-06T02:19:55.926 INFO:teuthology.orchestra.run.smithi031.stdout: libquadmath x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 169 k 2023-10-06T02:19:55.927 INFO:teuthology.orchestra.run.smithi031.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-10-06T02:19:55.927 INFO:teuthology.orchestra.run.smithi031.stdout: libradosstriper1 x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 507 k 2023-10-06T02:19:55.927 INFO:teuthology.orchestra.run.smithi031.stdout: librdkafka x86_64 0.11.4-1.el8 rhel-8-for-x86_64-appstream-rpms 353 k 2023-10-06T02:19:55.927 INFO:teuthology.orchestra.run.smithi031.stdout: librgw2 x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 5.6 M 2023-10-06T02:19:55.927 INFO:teuthology.orchestra.run.smithi031.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-10-06T02:19:55.927 INFO:teuthology.orchestra.run.smithi031.stdout: libsepol-devel x86_64 2.9-2.el8 rhel-8-for-x86_64-baseos-rpms 86 k 2023-10-06T02:19:55.927 INFO:teuthology.orchestra.run.smithi031.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-10-06T02:19:55.927 INFO:teuthology.orchestra.run.smithi031.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-10-06T02:19:55.927 INFO:teuthology.orchestra.run.smithi031.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-10-06T02:19:55.927 INFO:teuthology.orchestra.run.smithi031.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-10-06T02:19:55.927 INFO:teuthology.orchestra.run.smithi031.stdout: openblas x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.6 M 2023-10-06T02:19:55.927 INFO:teuthology.orchestra.run.smithi031.stdout: openblas-threads x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-10-06T02:19:55.928 INFO:teuthology.orchestra.run.smithi031.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-10-06T02:19:55.928 INFO:teuthology.orchestra.run.smithi031.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-10-06T02:19:55.928 INFO:teuthology.orchestra.run.smithi031.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-10-06T02:19:55.928 INFO:teuthology.orchestra.run.smithi031.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-10-06T02:19:55.928 INFO:teuthology.orchestra.run.smithi031.stdout: python3-babel noarch 2.5.1-5.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-10-06T02:19:55.928 INFO:teuthology.orchestra.run.smithi031.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-10-06T02:19:55.928 INFO:teuthology.orchestra.run.smithi031.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-10-06T02:19:55.928 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-10-06T02:19:55.928 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-argparse x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 45 k 2023-10-06T02:19:55.928 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-common x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 115 k 2023-10-06T02:19:55.928 INFO:teuthology.orchestra.run.smithi031.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-10-06T02:19:55.929 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-10-06T02:19:55.929 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-10-06T02:19:55.929 INFO:teuthology.orchestra.run.smithi031.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-10-06T02:19:55.929 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-10-06T02:19:55.929 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-10-06T02:19:55.929 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jinja2 noarch 2.10.1-2.el8_0 rhel-8-for-x86_64-appstream-rpms 538 k 2023-10-06T02:19:55.929 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-10-06T02:19:55.929 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-10-06T02:19:55.929 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-06T02:19:55.929 INFO:teuthology.orchestra.run.smithi031.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-10-06T02:19:55.929 INFO:teuthology.orchestra.run.smithi031.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-10-06T02:19:55.929 INFO:teuthology.orchestra.run.smithi031.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-10-06T02:19:55.929 INFO:teuthology.orchestra.run.smithi031.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-10-06T02:19:55.930 INFO:teuthology.orchestra.run.smithi031.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-10-06T02:19:55.930 INFO:teuthology.orchestra.run.smithi031.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-10-06T02:19:55.930 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-10-06T02:19:55.930 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy-f2py x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-10-06T02:19:55.930 INFO:teuthology.orchestra.run.smithi031.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-10-06T02:19:55.930 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-10-06T02:19:55.930 INFO:teuthology.orchestra.run.smithi031.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-10-06T02:19:55.930 INFO:teuthology.orchestra.run.smithi031.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-10-06T02:19:55.930 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-10-06T02:19:55.930 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-10-06T02:19:55.930 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-10-06T02:19:55.930 INFO:teuthology.orchestra.run.smithi031.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-10-06T02:19:55.931 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-06T02:19:55.931 INFO:teuthology.orchestra.run.smithi031.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-10-06T02:19:55.931 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-10-06T02:19:55.931 INFO:teuthology.orchestra.run.smithi031.stdout: python3-scipy x86_64 1.0.0-20.module+el8.1.0+3334+5cb623d7 rhel-8-for-x86_64-appstream-rpms 15 M 2023-10-06T02:19:55.931 INFO:teuthology.orchestra.run.smithi031.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-10-06T02:19:55.931 INFO:teuthology.orchestra.run.smithi031.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-10-06T02:19:55.931 INFO:teuthology.orchestra.run.smithi031.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-10-06T02:19:55.931 INFO:teuthology.orchestra.run.smithi031.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-10-06T02:19:55.931 INFO:teuthology.orchestra.run.smithi031.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-10-06T02:19:55.931 INFO:teuthology.orchestra.run.smithi031.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-10-06T02:19:55.931 INFO:teuthology.orchestra.run.smithi031.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-10-06T02:19:55.931 INFO:teuthology.orchestra.run.smithi031.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-10-06T02:19:55.931 INFO:teuthology.orchestra.run.smithi031.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-10-06T02:19:55.932 INFO:teuthology.orchestra.run.smithi031.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-10-06T02:19:55.932 INFO:teuthology.orchestra.run.smithi031.stdout: socat x86_64 1.7.3.3-2.el8 rhel-8-for-x86_64-appstream-rpms 302 k 2023-10-06T02:19:55.932 INFO:teuthology.orchestra.run.smithi031.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-10-06T02:19:55.932 INFO:teuthology.orchestra.run.smithi031.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-10-06T02:19:55.932 INFO:teuthology.orchestra.run.smithi031.stdout: zlib-devel x86_64 1.2.11-17.el8 rhel-8-for-x86_64-baseos-rpms 58 k 2023-10-06T02:19:55.932 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:19:55.932 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2023-10-06T02:19:55.932 INFO:teuthology.orchestra.run.smithi031.stdout:====================================================================================================================== 2023-10-06T02:19:55.932 INFO:teuthology.orchestra.run.smithi031.stdout:Install 118 Packages 2023-10-06T02:19:55.932 INFO:teuthology.orchestra.run.smithi031.stdout:Upgrade 2 Packages 2023-10-06T02:19:55.932 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:19:55.933 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 201 M 2023-10-06T02:19:55.933 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2023-10-06T02:19:57.767 INFO:teuthology.orchestra.run.smithi062.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-06T02:19:57.767 INFO:teuthology.orchestra.run.smithi062.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-06T02:19:57.779 INFO:teuthology.orchestra.run.smithi062.stdout:Package NetworkManager-1:1.30.0-13.el8_4.x86_64 is already installed. 2023-10-06T02:19:57.780 INFO:teuthology.orchestra.run.smithi062.stdout:Package iproute-5.9.0-4.el8.x86_64 is already installed. 2023-10-06T02:19:57.781 INFO:teuthology.orchestra.run.smithi062.stdout:Package util-linux-2.32.1-27.el8.x86_64 is already installed. 2023-10-06T02:19:57.782 INFO:teuthology.orchestra.run.smithi062.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-10-06T02:19:57.783 INFO:teuthology.orchestra.run.smithi062.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-10-06T02:19:57.786 INFO:teuthology.orchestra.run.smithi062.stdout:Package xfsdump-3.1.8-2.el8.x86_64 is already installed. 2023-10-06T02:19:57.787 INFO:teuthology.orchestra.run.smithi062.stdout:Package xfsprogs-5.0.0-8.el8.x86_64 is already installed. 2023-10-06T02:19:58.330 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2023-10-06T02:19:58.342 INFO:teuthology.orchestra.run.smithi062.stdout:====================================================================================================================== 2023-10-06T02:19:58.342 INFO:teuthology.orchestra.run.smithi062.stdout: Package Arch Version Repository Size 2023-10-06T02:19:58.342 INFO:teuthology.orchestra.run.smithi062.stdout:====================================================================================================================== 2023-10-06T02:19:58.342 INFO:teuthology.orchestra.run.smithi062.stdout:Installing: 2023-10-06T02:19:58.343 INFO:teuthology.orchestra.run.smithi062.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-10-06T02:19:58.343 INFO:teuthology.orchestra.run.smithi062.stdout: ceph x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 6.4 k 2023-10-06T02:19:58.343 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-base x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 5.2 M 2023-10-06T02:19:58.343 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-fuse x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 876 k 2023-10-06T02:19:58.343 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-immutable-object-cache x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 162 k 2023-10-06T02:19:58.343 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 1.6 M 2023-10-06T02:19:58.343 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-cephadm noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 121 k 2023-10-06T02:19:58.343 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-dashboard noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 1.7 M 2023-10-06T02:19:58.343 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 7.4 M 2023-10-06T02:19:58.343 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-rook noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 47 k 2023-10-06T02:19:58.343 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-radosgw x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 17 M 2023-10-06T02:19:58.344 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-test x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 52 M 2023-10-06T02:19:58.344 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-volume noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 267 k 2023-10-06T02:19:58.344 INFO:teuthology.orchestra.run.smithi062.stdout: cephadm noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 80 k 2023-10-06T02:19:58.344 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-mirror x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 255 k 2023-10-06T02:19:58.344 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-top noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 18 k 2023-10-06T02:19:58.344 INFO:teuthology.orchestra.run.smithi062.stdout: elfutils-libelf-devel x86_64 0.182-3.el8 rhel-8-for-x86_64-baseos-rpms 59 k 2023-10-06T02:19:58.344 INFO:teuthology.orchestra.run.smithi062.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-10-06T02:19:58.344 INFO:teuthology.orchestra.run.smithi062.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-10-06T02:19:58.344 INFO:teuthology.orchestra.run.smithi062.stdout: libcephfs-devel x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 25 k 2023-10-06T02:19:58.344 INFO:teuthology.orchestra.run.smithi062.stdout: libcephfs2 x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 797 k 2023-10-06T02:19:58.344 INFO:teuthology.orchestra.run.smithi062.stdout: librados-devel x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 130 k 2023-10-06T02:19:58.344 INFO:teuthology.orchestra.run.smithi062.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-10-06T02:19:58.345 INFO:teuthology.orchestra.run.smithi062.stdout: libuuid-devel x86_64 2.32.1-27.el8 rhel-8-for-x86_64-baseos-rpms 97 k 2023-10-06T02:19:58.345 INFO:teuthology.orchestra.run.smithi062.stdout: openssl-devel x86_64 1:1.1.1g-15.el8_3 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-10-06T02:19:58.345 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cephfs x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 190 k 2023-10-06T02:19:58.345 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rados x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 371 k 2023-10-06T02:19:58.345 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rbd x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 358 k 2023-10-06T02:19:58.345 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rgw x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 107 k 2023-10-06T02:19:58.345 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-fuse x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 90 k 2023-10-06T02:19:58.345 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-mirror x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 3.5 M 2023-10-06T02:19:58.345 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-nbd x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 173 k 2023-10-06T02:19:58.345 INFO:teuthology.orchestra.run.smithi062.stdout: xfsprogs-devel x86_64 5.0.0-8.el8 rhel-8-for-x86_64-baseos-rpms 70 k 2023-10-06T02:19:58.345 INFO:teuthology.orchestra.run.smithi062.stdout:Upgrading: 2023-10-06T02:19:58.345 INFO:teuthology.orchestra.run.smithi062.stdout: librados2 x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 3.5 M 2023-10-06T02:19:58.346 INFO:teuthology.orchestra.run.smithi062.stdout: librbd1 x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 3.6 M 2023-10-06T02:19:58.346 INFO:teuthology.orchestra.run.smithi062.stdout:Installing dependencies: 2023-10-06T02:19:58.346 INFO:teuthology.orchestra.run.smithi062.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-10-06T02:19:58.346 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-common x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 23 M 2023-10-06T02:19:58.346 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-grafana-dashboards noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 23 k 2023-10-06T02:19:58.346 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mds x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 2.3 M 2023-10-06T02:19:58.346 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-modules-core noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 238 k 2023-10-06T02:19:58.346 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mon x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 3.9 M 2023-10-06T02:19:58.346 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-osd x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 17 M 2023-10-06T02:19:58.346 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-prometheus-alerts noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 14 k 2023-10-06T02:19:58.346 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-selinux x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 24 k 2023-10-06T02:19:58.347 INFO:teuthology.orchestra.run.smithi062.stdout: fuse x86_64 2.9.7-12.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-10-06T02:19:58.347 INFO:teuthology.orchestra.run.smithi062.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-10-06T02:19:58.347 INFO:teuthology.orchestra.run.smithi062.stdout: jq x86_64 1.5-12.el8 rhel-8-for-x86_64-appstream-rpms 161 k 2023-10-06T02:19:58.347 INFO:teuthology.orchestra.run.smithi062.stdout: keyutils-libs-devel x86_64 1.5.10-6.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-10-06T02:19:58.347 INFO:teuthology.orchestra.run.smithi062.stdout: krb5-devel x86_64 1.18.2-8.3.el8_4 rhel-8-for-x86_64-baseos-rpms 559 k 2023-10-06T02:19:58.347 INFO:teuthology.orchestra.run.smithi062.stdout: libcephsqlite x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 174 k 2023-10-06T02:19:58.347 INFO:teuthology.orchestra.run.smithi062.stdout: libcom_err-devel x86_64 1.45.6-1.el8 rhel-8-for-x86_64-baseos-rpms 38 k 2023-10-06T02:19:58.347 INFO:teuthology.orchestra.run.smithi062.stdout: libgfortran x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 639 k 2023-10-06T02:19:58.347 INFO:teuthology.orchestra.run.smithi062.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-10-06T02:19:58.347 INFO:teuthology.orchestra.run.smithi062.stdout: libquadmath x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 169 k 2023-10-06T02:19:58.347 INFO:teuthology.orchestra.run.smithi062.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-10-06T02:19:58.347 INFO:teuthology.orchestra.run.smithi062.stdout: libradosstriper1 x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 507 k 2023-10-06T02:19:58.348 INFO:teuthology.orchestra.run.smithi062.stdout: librdkafka x86_64 0.11.4-1.el8 rhel-8-for-x86_64-appstream-rpms 353 k 2023-10-06T02:19:58.348 INFO:teuthology.orchestra.run.smithi062.stdout: librgw2 x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 5.6 M 2023-10-06T02:19:58.348 INFO:teuthology.orchestra.run.smithi062.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-10-06T02:19:58.348 INFO:teuthology.orchestra.run.smithi062.stdout: libsepol-devel x86_64 2.9-2.el8 rhel-8-for-x86_64-baseos-rpms 86 k 2023-10-06T02:19:58.348 INFO:teuthology.orchestra.run.smithi062.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-10-06T02:19:58.348 INFO:teuthology.orchestra.run.smithi062.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-10-06T02:19:58.348 INFO:teuthology.orchestra.run.smithi062.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-10-06T02:19:58.348 INFO:teuthology.orchestra.run.smithi062.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-10-06T02:19:58.348 INFO:teuthology.orchestra.run.smithi062.stdout: openblas x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.6 M 2023-10-06T02:19:58.348 INFO:teuthology.orchestra.run.smithi062.stdout: openblas-threads x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-10-06T02:19:58.348 INFO:teuthology.orchestra.run.smithi062.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-10-06T02:19:58.348 INFO:teuthology.orchestra.run.smithi062.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-10-06T02:19:58.349 INFO:teuthology.orchestra.run.smithi062.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-10-06T02:19:58.349 INFO:teuthology.orchestra.run.smithi062.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-10-06T02:19:58.349 INFO:teuthology.orchestra.run.smithi062.stdout: python3-babel noarch 2.5.1-5.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-10-06T02:19:58.349 INFO:teuthology.orchestra.run.smithi062.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-10-06T02:19:58.349 INFO:teuthology.orchestra.run.smithi062.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-10-06T02:19:58.349 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-10-06T02:19:58.349 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ceph-argparse x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 45 k 2023-10-06T02:19:58.349 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ceph-common x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 115 k 2023-10-06T02:19:58.349 INFO:teuthology.orchestra.run.smithi062.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-10-06T02:19:58.349 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-10-06T02:19:58.349 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-10-06T02:19:58.349 INFO:teuthology.orchestra.run.smithi062.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-10-06T02:19:58.350 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-10-06T02:19:58.350 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-10-06T02:19:58.350 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jinja2 noarch 2.10.1-2.el8_0 rhel-8-for-x86_64-appstream-rpms 538 k 2023-10-06T02:19:58.350 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-10-06T02:19:58.350 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-10-06T02:19:58.350 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-06T02:19:58.350 INFO:teuthology.orchestra.run.smithi062.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-10-06T02:19:58.350 INFO:teuthology.orchestra.run.smithi062.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-10-06T02:19:58.350 INFO:teuthology.orchestra.run.smithi062.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-10-06T02:19:58.350 INFO:teuthology.orchestra.run.smithi062.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-10-06T02:19:58.350 INFO:teuthology.orchestra.run.smithi062.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-10-06T02:19:58.350 INFO:teuthology.orchestra.run.smithi062.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-10-06T02:19:58.351 INFO:teuthology.orchestra.run.smithi062.stdout: python3-numpy x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-10-06T02:19:58.351 INFO:teuthology.orchestra.run.smithi062.stdout: python3-numpy-f2py x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-10-06T02:19:58.351 INFO:teuthology.orchestra.run.smithi062.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-10-06T02:19:58.351 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-10-06T02:19:58.351 INFO:teuthology.orchestra.run.smithi062.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-10-06T02:19:58.351 INFO:teuthology.orchestra.run.smithi062.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-10-06T02:19:58.351 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-10-06T02:19:58.351 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-10-06T02:19:58.351 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-10-06T02:19:58.351 INFO:teuthology.orchestra.run.smithi062.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-10-06T02:19:58.351 INFO:teuthology.orchestra.run.smithi062.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-06T02:19:58.351 INFO:teuthology.orchestra.run.smithi062.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-10-06T02:19:58.352 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-10-06T02:19:58.352 INFO:teuthology.orchestra.run.smithi062.stdout: python3-scipy x86_64 1.0.0-20.module+el8.1.0+3334+5cb623d7 rhel-8-for-x86_64-appstream-rpms 15 M 2023-10-06T02:19:58.352 INFO:teuthology.orchestra.run.smithi062.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-10-06T02:19:58.352 INFO:teuthology.orchestra.run.smithi062.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-10-06T02:19:58.352 INFO:teuthology.orchestra.run.smithi062.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-10-06T02:19:58.352 INFO:teuthology.orchestra.run.smithi062.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-10-06T02:19:58.352 INFO:teuthology.orchestra.run.smithi062.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-10-06T02:19:58.352 INFO:teuthology.orchestra.run.smithi062.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-10-06T02:19:58.352 INFO:teuthology.orchestra.run.smithi062.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-10-06T02:19:58.352 INFO:teuthology.orchestra.run.smithi062.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-10-06T02:19:58.352 INFO:teuthology.orchestra.run.smithi062.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-10-06T02:19:58.352 INFO:teuthology.orchestra.run.smithi062.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-10-06T02:19:58.352 INFO:teuthology.orchestra.run.smithi062.stdout: socat x86_64 1.7.3.3-2.el8 rhel-8-for-x86_64-appstream-rpms 302 k 2023-10-06T02:19:58.353 INFO:teuthology.orchestra.run.smithi062.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-10-06T02:19:58.353 INFO:teuthology.orchestra.run.smithi062.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-10-06T02:19:58.353 INFO:teuthology.orchestra.run.smithi062.stdout: zlib-devel x86_64 1.2.11-17.el8 rhel-8-for-x86_64-baseos-rpms 58 k 2023-10-06T02:19:58.353 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-10-06T02:19:58.353 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2023-10-06T02:19:58.353 INFO:teuthology.orchestra.run.smithi062.stdout:====================================================================================================================== 2023-10-06T02:19:58.353 INFO:teuthology.orchestra.run.smithi062.stdout:Install 118 Packages 2023-10-06T02:19:58.353 INFO:teuthology.orchestra.run.smithi062.stdout:Upgrade 2 Packages 2023-10-06T02:19:58.353 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-10-06T02:19:58.354 INFO:teuthology.orchestra.run.smithi062.stdout:Total download size: 201 M 2023-10-06T02:19:58.354 INFO:teuthology.orchestra.run.smithi062.stdout:Downloading Packages: 2023-10-06T02:19:58.678 INFO:teuthology.orchestra.run.smithi062.stdout:(1/120): ceph-17.2.6-1296.g736ac5ba.el8.x86_64. 39 kB/s | 6.4 kB 00:00 2023-10-06T02:19:58.974 INFO:teuthology.orchestra.run.smithi062.stdout:(2/120): ceph-fuse-17.2.6-1296.g736ac5ba.el8.x8 2.9 MB/s | 876 kB 00:00 2023-10-06T02:19:59.028 INFO:teuthology.orchestra.run.smithi062.stdout:(3/120): ceph-immutable-object-cache-17.2.6-129 3.0 MB/s | 162 kB 00:00 2023-10-06T02:19:59.134 INFO:teuthology.orchestra.run.smithi062.stdout:(4/120): ceph-base-17.2.6-1296.g736ac5ba.el8.x8 8.3 MB/s | 5.2 MB 00:00 2023-10-06T02:19:59.289 INFO:teuthology.orchestra.run.smithi062.stdout:(5/120): ceph-mgr-17.2.6-1296.g736ac5ba.el8.x86 10 MB/s | 1.6 MB 00:00 2023-10-06T02:19:59.562 INFO:teuthology.orchestra.run.smithi062.stdout:(6/120): ceph-mds-17.2.6-1296.g736ac5ba.el8.x86 4.3 MB/s | 2.3 MB 00:00 2023-10-06T02:19:59.590 INFO:teuthology.orchestra.run.smithi031.stdout:(1/120): ceph-17.2.6-1296.g736ac5ba.el8.x86_64. 31 kB/s | 6.4 kB 00:00 2023-10-06T02:19:59.667 INFO:teuthology.orchestra.run.smithi062.stdout:(7/120): ceph-mon-17.2.6-1296.g736ac5ba.el8.x86 10 MB/s | 3.9 MB 00:00 2023-10-06T02:20:00.236 INFO:teuthology.orchestra.run.smithi031.stdout:(2/120): ceph-fuse-17.2.6-1296.g736ac5ba.el8.x8 1.3 MB/s | 876 kB 00:00 2023-10-06T02:20:00.362 INFO:teuthology.orchestra.run.smithi031.stdout:(3/120): ceph-immutable-object-cache-17.2.6-129 1.3 MB/s | 162 kB 00:00 2023-10-06T02:20:00.715 INFO:teuthology.orchestra.run.smithi062.stdout:(8/120): ceph-common-17.2.6-1296.g736ac5ba.el8. 11 MB/s | 23 MB 00:02 2023-10-06T02:20:00.767 INFO:teuthology.orchestra.run.smithi062.stdout:(9/120): ceph-selinux-17.2.6-1296.g736ac5ba.el8 478 kB/s | 24 kB 00:00 2023-10-06T02:20:01.091 INFO:teuthology.orchestra.run.smithi031.stdout:(4/120): ceph-mds-17.2.6-1296.g736ac5ba.el8.x86 3.2 MB/s | 2.3 MB 00:00 2023-10-06T02:20:01.512 INFO:teuthology.orchestra.run.smithi031.stdout:(5/120): ceph-mgr-17.2.6-1296.g736ac5ba.el8.x86 3.7 MB/s | 1.6 MB 00:00 2023-10-06T02:20:01.833 INFO:teuthology.orchestra.run.smithi062.stdout:(10/120): ceph-radosgw-17.2.6-1296.g736ac5ba.el 7.7 MB/s | 17 MB 00:02 2023-10-06T02:20:01.898 INFO:teuthology.orchestra.run.smithi062.stdout:(11/120): cephfs-mirror-17.2.6-1296.g736ac5ba.e 3.8 MB/s | 255 kB 00:00 2023-10-06T02:20:01.956 INFO:teuthology.orchestra.run.smithi062.stdout:(12/120): libcephfs-devel-17.2.6-1296.g736ac5ba 431 kB/s | 25 kB 00:00 2023-10-06T02:20:02.104 INFO:teuthology.orchestra.run.smithi062.stdout:(13/120): libcephfs2-17.2.6-1296.g736ac5ba.el8. 5.2 MB/s | 797 kB 00:00 2023-10-06T02:20:02.172 INFO:teuthology.orchestra.run.smithi062.stdout:(14/120): libcephsqlite-17.2.6-1296.g736ac5ba.e 2.5 MB/s | 174 kB 00:00 2023-10-06T02:20:02.242 INFO:teuthology.orchestra.run.smithi062.stdout:(15/120): librados-devel-17.2.6-1296.g736ac5ba. 1.8 MB/s | 130 kB 00:00 2023-10-06T02:20:02.477 INFO:teuthology.orchestra.run.smithi031.stdout:(6/120): ceph-mon-17.2.6-1296.g736ac5ba.el8.x86 4.1 MB/s | 3.9 MB 00:00 2023-10-06T02:20:02.486 INFO:teuthology.orchestra.run.smithi062.stdout:(16/120): libradosstriper1-17.2.6-1296.g736ac5b 2.0 MB/s | 507 kB 00:00 2023-10-06T02:20:02.734 INFO:teuthology.orchestra.run.smithi031.stdout:(7/120): ceph-common-17.2.6-1296.g736ac5ba.el8. 7.0 MB/s | 23 MB 00:03 2023-10-06T02:20:03.527 INFO:teuthology.orchestra.run.smithi031.stdout:(8/120): ceph-base-17.2.6-1296.g736ac5ba.el8.x8 1.2 MB/s | 5.2 MB 00:04 2023-10-06T02:20:03.615 INFO:teuthology.orchestra.run.smithi031.stdout:(9/120): ceph-selinux-17.2.6-1296.g736ac5ba.el8 278 kB/s | 24 kB 00:00 2023-10-06T02:20:03.975 INFO:teuthology.orchestra.run.smithi031.stdout:(10/120): ceph-radosgw-17.2.6-1296.g736ac5ba.el 14 MB/s | 17 MB 00:01 2023-10-06T02:20:04.114 INFO:teuthology.orchestra.run.smithi031.stdout:(11/120): cephfs-mirror-17.2.6-1296.g736ac5ba.e 6.2 MB/s | 255 kB 00:00 2023-10-06T02:20:04.114 INFO:teuthology.orchestra.run.smithi031.stdout:(12/120): libcephfs-devel-17.2.6-1296.g736ac5ba 759 kB/s | 25 kB 00:00 2023-10-06T02:20:04.114 INFO:teuthology.orchestra.run.smithi031.stdout:(13/120): libcephfs2-17.2.6-1296.g736ac5ba.el8. 13 MB/s | 797 kB 00:00 2023-10-06T02:20:04.154 INFO:teuthology.orchestra.run.smithi031.stdout:(14/120): libcephsqlite-17.2.6-1296.g736ac5ba.e 3.8 MB/s | 174 kB 00:00 2023-10-06T02:20:04.188 INFO:teuthology.orchestra.run.smithi031.stdout:(15/120): librados-devel-17.2.6-1296.g736ac5ba. 3.8 MB/s | 130 kB 00:00 2023-10-06T02:20:04.241 INFO:teuthology.orchestra.run.smithi031.stdout:(16/120): libradosstriper1-17.2.6-1296.g736ac5b 9.4 MB/s | 507 kB 00:00 2023-10-06T02:20:04.537 INFO:teuthology.orchestra.run.smithi062.stdout:(17/120): librgw2-17.2.6-1296.g736ac5ba.el8.x86 2.7 MB/s | 5.6 MB 00:02 2023-10-06T02:20:04.598 INFO:teuthology.orchestra.run.smithi062.stdout:(18/120): python3-ceph-argparse-17.2.6-1296.g73 736 kB/s | 45 kB 00:00 2023-10-06T02:20:04.661 INFO:teuthology.orchestra.run.smithi062.stdout:(19/120): python3-ceph-common-17.2.6-1296.g736a 1.8 MB/s | 115 kB 00:00 2023-10-06T02:20:04.675 INFO:teuthology.orchestra.run.smithi031.stdout:(17/120): librgw2-17.2.6-1296.g736ac5ba.el8.x86 13 MB/s | 5.6 MB 00:00 2023-10-06T02:20:04.695 INFO:teuthology.orchestra.run.smithi062.stdout:(20/120): python3-cephfs-17.2.6-1296.g736ac5ba. 5.5 MB/s | 190 kB 00:00 2023-10-06T02:20:04.710 INFO:teuthology.orchestra.run.smithi031.stdout:(18/120): python3-ceph-argparse-17.2.6-1296.g73 1.3 MB/s | 45 kB 00:00 2023-10-06T02:20:04.751 INFO:teuthology.orchestra.run.smithi031.stdout:(19/120): python3-ceph-common-17.2.6-1296.g736a 2.8 MB/s | 115 kB 00:00 2023-10-06T02:20:04.767 INFO:teuthology.orchestra.run.smithi062.stdout:(21/120): python3-rados-17.2.6-1296.g736ac5ba.e 5.1 MB/s | 371 kB 00:00 2023-10-06T02:20:04.790 INFO:teuthology.orchestra.run.smithi031.stdout:(20/120): python3-cephfs-17.2.6-1296.g736ac5ba. 4.7 MB/s | 190 kB 00:00 2023-10-06T02:20:04.837 INFO:teuthology.orchestra.run.smithi031.stdout:(21/120): python3-rados-17.2.6-1296.g736ac5ba.e 7.8 MB/s | 371 kB 00:00 2023-10-06T02:20:04.853 INFO:teuthology.orchestra.run.smithi062.stdout:(22/120): python3-rbd-17.2.6-1296.g736ac5ba.el8 4.1 MB/s | 358 kB 00:00 2023-10-06T02:20:04.887 INFO:teuthology.orchestra.run.smithi031.stdout:(22/120): python3-rbd-17.2.6-1296.g736ac5ba.el8 7.0 MB/s | 358 kB 00:00 2023-10-06T02:20:04.897 INFO:teuthology.orchestra.run.smithi062.stdout:(23/120): python3-rgw-17.2.6-1296.g736ac5ba.el8 2.4 MB/s | 107 kB 00:00 2023-10-06T02:20:04.920 INFO:teuthology.orchestra.run.smithi031.stdout:(23/120): python3-rgw-17.2.6-1296.g736ac5ba.el8 3.2 MB/s | 107 kB 00:00 2023-10-06T02:20:04.932 INFO:teuthology.orchestra.run.smithi062.stdout:(24/120): rbd-fuse-17.2.6-1296.g736ac5ba.el8.x8 2.5 MB/s | 90 kB 00:00 2023-10-06T02:20:04.954 INFO:teuthology.orchestra.run.smithi031.stdout:(24/120): rbd-fuse-17.2.6-1296.g736ac5ba.el8.x8 2.6 MB/s | 90 kB 00:00 2023-10-06T02:20:05.228 INFO:teuthology.orchestra.run.smithi031.stdout:(25/120): rbd-mirror-17.2.6-1296.g736ac5ba.el8. 13 MB/s | 3.5 MB 00:00 2023-10-06T02:20:05.271 INFO:teuthology.orchestra.run.smithi031.stdout:(26/120): rbd-nbd-17.2.6-1296.g736ac5ba.el8.x86 4.0 MB/s | 173 kB 00:00 2023-10-06T02:20:05.307 INFO:teuthology.orchestra.run.smithi031.stdout:(27/120): ceph-grafana-dashboards-17.2.6-1296.g 635 kB/s | 23 kB 00:00 2023-10-06T02:20:05.346 INFO:teuthology.orchestra.run.smithi031.stdout:(28/120): ceph-mgr-cephadm-17.2.6-1296.g736ac5b 3.1 MB/s | 121 kB 00:00 2023-10-06T02:20:05.484 INFO:teuthology.orchestra.run.smithi031.stdout:(29/120): ceph-mgr-dashboard-17.2.6-1296.g736ac 12 MB/s | 1.7 MB 00:00 2023-10-06T02:20:05.713 INFO:teuthology.orchestra.run.smithi062.stdout:(25/120): rbd-mirror-17.2.6-1296.g736ac5ba.el8. 4.4 MB/s | 3.5 MB 00:00 2023-10-06T02:20:05.779 INFO:teuthology.orchestra.run.smithi062.stdout:(26/120): rbd-nbd-17.2.6-1296.g736ac5ba.el8.x86 2.6 MB/s | 173 kB 00:00 2023-10-06T02:20:05.840 INFO:teuthology.orchestra.run.smithi062.stdout:(27/120): ceph-grafana-dashboards-17.2.6-1296.g 384 kB/s | 23 kB 00:00 2023-10-06T02:20:05.904 INFO:teuthology.orchestra.run.smithi062.stdout:(28/120): ceph-mgr-cephadm-17.2.6-1296.g736ac5b 1.9 MB/s | 121 kB 00:00 2023-10-06T02:20:06.031 INFO:teuthology.orchestra.run.smithi031.stdout:(30/120): ceph-mgr-diskprediction-local-17.2.6- 14 MB/s | 7.4 MB 00:00 2023-10-06T02:20:06.078 INFO:teuthology.orchestra.run.smithi031.stdout:(31/120): ceph-mgr-modules-core-17.2.6-1296.g73 5.0 MB/s | 238 kB 00:00 2023-10-06T02:20:06.119 INFO:teuthology.orchestra.run.smithi031.stdout:(32/120): ceph-mgr-rook-17.2.6-1296.g736ac5ba.e 1.1 MB/s | 47 kB 00:00 2023-10-06T02:20:06.153 INFO:teuthology.orchestra.run.smithi031.stdout:(33/120): ceph-prometheus-alerts-17.2.6-1296.g7 429 kB/s | 14 kB 00:00 2023-10-06T02:20:06.228 INFO:teuthology.orchestra.run.smithi062.stdout:(29/120): ceph-osd-17.2.6-1296.g736ac5ba.el8.x8 2.5 MB/s | 17 MB 00:06 2023-10-06T02:20:06.233 INFO:teuthology.orchestra.run.smithi031.stdout:(34/120): ceph-osd-17.2.6-1296.g736ac5ba.el8.x8 4.5 MB/s | 17 MB 00:03 2023-10-06T02:20:06.237 INFO:teuthology.orchestra.run.smithi031.stdout:(35/120): ceph-volume-17.2.6-1296.g736ac5ba.el8 3.1 MB/s | 267 kB 00:00 2023-10-06T02:20:06.237 INFO:teuthology.orchestra.run.smithi062.stdout:(30/120): ceph-mgr-dashboard-17.2.6-1296.g736ac 5.1 MB/s | 1.7 MB 00:00 2023-10-06T02:20:06.260 INFO:teuthology.orchestra.run.smithi031.stdout:(36/120): cephadm-17.2.6-1296.g736ac5ba.el8.noa 3.0 MB/s | 80 kB 00:00 2023-10-06T02:20:06.261 INFO:teuthology.orchestra.run.smithi031.stdout:(37/120): cephfs-top-17.2.6-1296.g736ac5ba.el8. 758 kB/s | 18 kB 00:00 2023-10-06T02:20:06.269 INFO:teuthology.orchestra.run.smithi062.stdout:(31/120): ceph-mgr-modules-core-17.2.6-1296.g73 7.3 MB/s | 238 kB 00:00 2023-10-06T02:20:06.292 INFO:teuthology.orchestra.run.smithi062.stdout:(32/120): ceph-mgr-rook-17.2.6-1296.g736ac5ba.e 2.1 MB/s | 47 kB 00:00 2023-10-06T02:20:06.313 INFO:teuthology.orchestra.run.smithi062.stdout:(33/120): ceph-prometheus-alerts-17.2.6-1296.g7 688 kB/s | 14 kB 00:00 2023-10-06T02:20:06.337 INFO:teuthology.orchestra.run.smithi062.stdout:(34/120): ceph-volume-17.2.6-1296.g736ac5ba.el8 11 MB/s | 267 kB 00:00 2023-10-06T02:20:06.360 INFO:teuthology.orchestra.run.smithi062.stdout:(35/120): cephadm-17.2.6-1296.g736ac5ba.el8.noa 3.5 MB/s | 80 kB 00:00 2023-10-06T02:20:06.381 INFO:teuthology.orchestra.run.smithi062.stdout:(36/120): cephfs-top-17.2.6-1296.g736ac5ba.el8. 859 kB/s | 18 kB 00:00 2023-10-06T02:20:06.564 INFO:teuthology.orchestra.run.smithi062.stdout:(37/120): gperftools-libs-2.7-9.el8.x86_64.rpm 1.6 MB/s | 306 kB 00:00 2023-10-06T02:20:06.591 INFO:teuthology.orchestra.run.smithi062.stdout:(38/120): liboath-2.6.2-3.el8.x86_64.rpm 2.1 MB/s | 59 kB 00:00 2023-10-06T02:20:06.602 INFO:teuthology.orchestra.run.smithi031.stdout:(38/120): liboath-2.6.2-3.el8.x86_64.rpm 172 kB/s | 59 kB 00:00 2023-10-06T02:20:06.619 INFO:teuthology.orchestra.run.smithi062.stdout:(39/120): libunwind-1.3.1-3.el8.x86_64.rpm 2.6 MB/s | 75 kB 00:00 2023-10-06T02:20:06.677 INFO:teuthology.orchestra.run.smithi062.stdout:(40/120): python3-asyncssh-2.7.0-2.el8.noarch.r 8.1 MB/s | 476 kB 00:00 2023-10-06T02:20:06.705 INFO:teuthology.orchestra.run.smithi062.stdout:(41/120): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 1.5 MB/s | 44 kB 00:00 2023-10-06T02:20:06.736 INFO:teuthology.orchestra.run.smithi062.stdout:(42/120): python3-beautifulsoup4-4.6.3-2.el8.1. 5.9 MB/s | 185 kB 00:00 2023-10-06T02:20:06.748 INFO:teuthology.orchestra.run.smithi031.stdout:(39/120): gperftools-libs-2.7-9.el8.x86_64.rpm 629 kB/s | 306 kB 00:00 2023-10-06T02:20:06.762 INFO:teuthology.orchestra.run.smithi031.stdout:(40/120): libunwind-1.3.1-3.el8.x86_64.rpm 470 kB/s | 75 kB 00:00 2023-10-06T02:20:06.766 INFO:teuthology.orchestra.run.smithi062.stdout:(43/120): python3-cachetools-3.1.1-4.el8.noarch 1.1 MB/s | 33 kB 00:00 2023-10-06T02:20:06.797 INFO:teuthology.orchestra.run.smithi062.stdout:(44/120): python3-certifi-2018.10.15-7.el8.noar 529 kB/s | 16 kB 00:00 2023-10-06T02:20:06.833 INFO:teuthology.orchestra.run.smithi062.stdout:(45/120): python3-cheroot-8.5.2-1.el8.noarch.rp 4.8 MB/s | 173 kB 00:00 2023-10-06T02:20:06.844 INFO:teuthology.orchestra.run.smithi031.stdout:(41/120): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 536 kB/s | 44 kB 00:00 2023-10-06T02:20:06.877 INFO:teuthology.orchestra.run.smithi062.stdout:(46/120): python3-cherrypy-18.4.0-1.el8.noarch. 8.5 MB/s | 384 kB 00:00 2023-10-06T02:20:06.927 INFO:teuthology.orchestra.run.smithi062.stdout:(47/120): python3-google-auth-1.1.1-10.el8.noar 1.6 MB/s | 81 kB 00:00 2023-10-06T02:20:06.982 INFO:teuthology.orchestra.run.smithi062.stdout:(48/120): python3-jaraco-6.2-6.el8.noarch.rpm 202 kB/s | 11 kB 00:00 2023-10-06T02:20:06.985 INFO:teuthology.orchestra.run.smithi031.stdout:(42/120): python3-asyncssh-2.7.0-2.el8.noarch.r 2.0 MB/s | 476 kB 00:00 2023-10-06T02:20:07.006 INFO:teuthology.orchestra.run.smithi031.stdout:(43/120): python3-beautifulsoup4-4.6.3-2.el8.1. 1.1 MB/s | 185 kB 00:00 2023-10-06T02:20:07.023 INFO:teuthology.orchestra.run.smithi062.stdout:(49/120): python3-jaraco-functools-2.0-4.el8.no 426 kB/s | 18 kB 00:00 2023-10-06T02:20:07.086 INFO:teuthology.orchestra.run.smithi031.stdout:(44/120): python3-certifi-2018.10.15-7.el8.noar 203 kB/s | 16 kB 00:00 2023-10-06T02:20:07.087 INFO:teuthology.orchestra.run.smithi031.stdout:(45/120): python3-cachetools-3.1.1-4.el8.noarch 323 kB/s | 33 kB 00:00 2023-10-06T02:20:07.117 INFO:teuthology.orchestra.run.smithi062.stdout:(50/120): python3-kubernetes-11.0.0-6.el8.noarc 12 MB/s | 1.2 MB 00:00 2023-10-06T02:20:07.156 INFO:teuthology.orchestra.run.smithi062.stdout:(51/120): python3-logutils-0.3.5-11.el8.noarch. 1.2 MB/s | 49 kB 00:00 2023-10-06T02:20:07.169 INFO:teuthology.orchestra.run.smithi031.stdout:(46/120): python3-cheroot-8.5.2-1.el8.noarch.rp 2.0 MB/s | 173 kB 00:00 2023-10-06T02:20:07.183 INFO:teuthology.orchestra.run.smithi062.stdout:(52/120): python3-more-itertools-7.2.0-3.el8.no 2.1 MB/s | 59 kB 00:00 2023-10-06T02:20:07.211 INFO:teuthology.orchestra.run.smithi062.stdout:(53/120): python3-natsort-7.1.1-2.el8.noarch.rp 2.1 MB/s | 60 kB 00:00 2023-10-06T02:20:07.250 INFO:teuthology.orchestra.run.smithi031.stdout:(47/120): python3-cherrypy-18.4.0-1.el8.noarch. 2.3 MB/s | 384 kB 00:00 2023-10-06T02:20:07.274 INFO:teuthology.orchestra.run.smithi031.stdout:(48/120): python3-google-auth-1.1.1-10.el8.noar 778 kB/s | 81 kB 00:00 2023-10-06T02:20:07.332 INFO:teuthology.orchestra.run.smithi031.stdout:(49/120): python3-jaraco-6.2-6.el8.noarch.rpm 136 kB/s | 11 kB 00:00 2023-10-06T02:20:07.355 INFO:teuthology.orchestra.run.smithi031.stdout:(50/120): python3-jaraco-functools-2.0-4.el8.no 218 kB/s | 18 kB 00:00 2023-10-06T02:20:07.407 INFO:teuthology.orchestra.run.smithi062.stdout:(54/120): ceph-test-17.2.6-1296.g736ac5ba.el8.x 7.8 MB/s | 52 MB 00:06 2023-10-06T02:20:07.414 INFO:teuthology.orchestra.run.smithi062.stdout:(55/120): python3-pecan-1.3.2-9.el8.noarch.rpm 1.4 MB/s | 283 kB 00:00 2023-10-06T02:20:07.436 INFO:teuthology.orchestra.run.smithi031.stdout:(51/120): python3-logutils-0.3.5-11.el8.noarch. 605 kB/s | 49 kB 00:00 2023-10-06T02:20:07.441 INFO:teuthology.orchestra.run.smithi062.stdout:(56/120): python3-repoze-lru-0.7-6.el8.noarch.r 1.2 MB/s | 34 kB 00:00 2023-10-06T02:20:07.470 INFO:teuthology.orchestra.run.smithi062.stdout:(57/120): python3-routes-2.4.1-12.el8.noarch.rp 6.6 MB/s | 196 kB 00:00 2023-10-06T02:20:07.501 INFO:teuthology.orchestra.run.smithi062.stdout:(58/120): python3-rsa-4.9-2.el8.noarch.rpm 2.1 MB/s | 64 kB 00:00 2023-10-06T02:20:07.517 INFO:teuthology.orchestra.run.smithi031.stdout:(52/120): python3-more-itertools-7.2.0-3.el8.no 729 kB/s | 59 kB 00:00 2023-10-06T02:20:07.527 INFO:teuthology.orchestra.run.smithi062.stdout:(59/120): python3-simplegeneric-0.8.1-17.el8.no 723 kB/s | 19 kB 00:00 2023-10-06T02:20:07.532 INFO:teuthology.orchestra.run.smithi062.stdout:(60/120): python3-portend-2.6-1.el8.noarch.rpm 126 kB/s | 16 kB 00:00 2023-10-06T02:20:07.553 INFO:teuthology.orchestra.run.smithi062.stdout:(61/120): python3-singledispatch-3.4.0.3-18.el8 915 kB/s | 24 kB 00:00 2023-10-06T02:20:07.560 INFO:teuthology.orchestra.run.smithi062.stdout:(62/120): python3-tempora-1.14.1-5.el8.noarch.r 1.0 MB/s | 29 kB 00:00 2023-10-06T02:20:07.581 INFO:teuthology.orchestra.run.smithi062.stdout:(63/120): python3-trustme-0.6.0-4.el8.noarch.rp 1.0 MB/s | 27 kB 00:00 2023-10-06T02:20:07.611 INFO:teuthology.orchestra.run.smithi062.stdout:(64/120): python3-webob-1.8.5-1.el8.1.noarch.rp 8.3 MB/s | 251 kB 00:00 2023-10-06T02:20:07.617 INFO:teuthology.orchestra.run.smithi031.stdout:(53/120): python3-natsort-7.1.1-2.el8.noarch.rp 599 kB/s | 60 kB 00:00 2023-10-06T02:20:07.638 INFO:teuthology.orchestra.run.smithi062.stdout:(65/120): python3-websocket-client-0.56.0-5.el8 2.2 MB/s | 61 kB 00:00 2023-10-06T02:20:07.648 INFO:teuthology.orchestra.run.smithi062.stdout:(66/120): python3-waitress-1.4.3-1.el8.noarch.r 2.8 MB/s | 255 kB 00:00 2023-10-06T02:20:07.667 INFO:teuthology.orchestra.run.smithi062.stdout:(67/120): python3-webtest-2.0.33-1.el8.noarch.r 2.9 MB/s | 85 kB 00:00 2023-10-06T02:20:07.679 INFO:teuthology.orchestra.run.smithi062.stdout:(68/120): python3-zc-lockfile-2.0-2.el8.noarch. 761 kB/s | 23 kB 00:00 2023-10-06T02:20:07.683 INFO:teuthology.orchestra.run.smithi062.stdout:(69/120): xmlstarlet-1.6.1-20.el8.x86_64.rpm 17 MB/s | 70 kB 00:00 2023-10-06T02:20:07.695 INFO:teuthology.orchestra.run.smithi031.stdout:(54/120): python3-kubernetes-11.0.0-6.el8.noarc 3.2 MB/s | 1.2 MB 00:00 2023-10-06T02:20:07.702 INFO:teuthology.orchestra.run.smithi031.stdout:(55/120): python3-pecan-1.3.2-9.el8.noarch.rpm 3.3 MB/s | 283 kB 00:00 2023-10-06T02:20:07.750 INFO:teuthology.orchestra.run.smithi062.stdout:(70/120): thrift-0.13.0-2.el8.x86_64.rpm 21 MB/s | 1.7 MB 00:00 2023-10-06T02:20:07.776 INFO:teuthology.orchestra.run.smithi031.stdout:(56/120): python3-portend-2.6-1.el8.noarch.rpm 191 kB/s | 16 kB 00:00 2023-10-06T02:20:07.782 INFO:teuthology.orchestra.run.smithi062.stdout:(71/120): python3-prettytable-0.7.2-14.el8.noar 1.3 MB/s | 44 kB 00:00 2023-10-06T02:20:07.798 INFO:teuthology.orchestra.run.smithi062.stdout:(72/120): python3-jsonpatch-1.21-2.el8.noarch.r 1.8 MB/s | 27 kB 00:00 2023-10-06T02:20:07.804 INFO:teuthology.orchestra.run.smithi031.stdout:(57/120): python3-repoze-lru-0.7-6.el8.noarch.r 330 kB/s | 34 kB 00:00 2023-10-06T02:20:07.816 INFO:teuthology.orchestra.run.smithi062.stdout:(73/120): jq-1.5-12.el8.x86_64.rpm 8.9 MB/s | 161 kB 00:00 2023-10-06T02:20:07.836 INFO:teuthology.orchestra.run.smithi062.stdout:(74/120): flex-2.6.1-9.el8.x86_64.rpm 16 MB/s | 320 kB 00:00 2023-10-06T02:20:07.865 INFO:teuthology.orchestra.run.smithi062.stdout:(75/120): python3-werkzeug-0.12.2-4.el8.noarch. 16 MB/s | 457 kB 00:00 2023-10-06T02:20:07.902 INFO:teuthology.orchestra.run.smithi031.stdout:(58/120): python3-rsa-4.9-2.el8.noarch.rpm 657 kB/s | 64 kB 00:00 2023-10-06T02:20:07.920 INFO:teuthology.orchestra.run.smithi031.stdout:(59/120): python3-routes-2.4.1-12.el8.noarch.rp 1.3 MB/s | 196 kB 00:00 2023-10-06T02:20:07.983 INFO:teuthology.orchestra.run.smithi031.stdout:(60/120): python3-simplegeneric-0.8.1-17.el8.no 233 kB/s | 19 kB 00:00 2023-10-06T02:20:07.998 INFO:teuthology.orchestra.run.smithi062.stdout:(76/120): python3-babel-2.5.1-5.el8.noarch.rpm 36 MB/s | 4.8 MB 00:00 2023-10-06T02:20:08.002 INFO:teuthology.orchestra.run.smithi031.stdout:(61/120): python3-singledispatch-3.4.0.3-18.el8 293 kB/s | 24 kB 00:00 2023-10-06T02:20:08.061 INFO:teuthology.orchestra.run.smithi062.stdout:(77/120): python3-scipy-1.0.0-20.module+el8.1.0 39 MB/s | 15 MB 00:00 2023-10-06T02:20:08.063 INFO:teuthology.orchestra.run.smithi062.stdout:(78/120): python3-markupsafe-0.23-19.el8.x86_64 599 kB/s | 39 kB 00:00 2023-10-06T02:20:08.063 INFO:teuthology.orchestra.run.smithi031.stdout:(62/120): python3-tempora-1.14.1-5.el8.noarch.r 363 kB/s | 29 kB 00:00 2023-10-06T02:20:08.082 INFO:teuthology.orchestra.run.smithi062.stdout:(79/120): lttng-ust-2.8.1-11.el8.x86_64.rpm 13 MB/s | 259 kB 00:00 2023-10-06T02:20:08.084 INFO:teuthology.orchestra.run.smithi031.stdout:(63/120): python3-trustme-0.6.0-4.el8.noarch.rp 338 kB/s | 27 kB 00:00 2023-10-06T02:20:08.110 INFO:teuthology.orchestra.run.smithi062.stdout:(80/120): ceph-mgr-diskprediction-local-17.2.6- 3.9 MB/s | 7.4 MB 00:01 2023-10-06T02:20:08.116 INFO:teuthology.orchestra.run.smithi062.stdout:(81/120): python3-numpy-f2py-1.14.3-9.el8.x86_6 6.6 MB/s | 225 kB 00:00 2023-10-06T02:20:08.122 INFO:teuthology.orchestra.run.smithi062.stdout:(82/120): bison-3.0.4-10.el8.x86_64.rpm 11 MB/s | 688 kB 00:00 2023-10-06T02:20:08.142 INFO:teuthology.orchestra.run.smithi062.stdout:(83/120): oniguruma-6.8.2-2.el8.x86_64.rpm 9.3 MB/s | 187 kB 00:00 2023-10-06T02:20:08.147 INFO:teuthology.orchestra.run.smithi031.stdout:(64/120): python3-waitress-1.4.3-1.el8.noarch.r 3.0 MB/s | 255 kB 00:00 2023-10-06T02:20:08.153 INFO:teuthology.orchestra.run.smithi062.stdout:(84/120): python3-pyasn1-0.3.7-6.el8.noarch.rpm 2.9 MB/s | 126 kB 00:00 2023-10-06T02:20:08.167 INFO:teuthology.orchestra.run.smithi062.stdout:(85/120): python3-jinja2-2.10.1-2.el8_0.noarch. 21 MB/s | 538 kB 00:00 2023-10-06T02:20:08.177 INFO:teuthology.orchestra.run.smithi062.stdout:(86/120): socat-1.7.3.3-2.el8.x86_64.rpm 13 MB/s | 302 kB 00:00 2023-10-06T02:20:08.184 INFO:teuthology.orchestra.run.smithi031.stdout:(65/120): python3-webob-1.8.5-1.el8.1.noarch.rp 2.5 MB/s | 251 kB 00:00 2023-10-06T02:20:08.185 INFO:teuthology.orchestra.run.smithi062.stdout:(87/120): python3-mako-1.0.6-13.el8.noarch.rpm 9.0 MB/s | 157 kB 00:00 2023-10-06T02:20:08.196 INFO:teuthology.orchestra.run.smithi062.stdout:(88/120): python3-pyasn1-modules-0.3.7-6.el8.no 5.6 MB/s | 110 kB 00:00 2023-10-06T02:20:08.212 INFO:teuthology.orchestra.run.smithi062.stdout:(89/120): python3-numpy-1.14.3-9.el8.x86_64.rpm 39 MB/s | 3.7 MB 00:00 2023-10-06T02:20:08.213 INFO:teuthology.orchestra.run.smithi062.stdout:(90/120): python3-jsonpointer-1.10-11.el8.noarc 1.2 MB/s | 20 kB 00:00 2023-10-06T02:20:08.219 INFO:teuthology.orchestra.run.smithi062.stdout:(91/120): librdkafka-0.11.4-1.el8.x86_64.rpm 10 MB/s | 353 kB 00:00 2023-10-06T02:20:08.232 INFO:teuthology.orchestra.run.smithi062.stdout:(92/120): python3-pytz-2017.2-9.el8.noarch.rpm 2.9 MB/s | 54 kB 00:00 2023-10-06T02:20:08.242 INFO:teuthology.orchestra.run.smithi062.stdout:(93/120): libtool-2.4.6-25.el8.x86_64.rpm 23 MB/s | 709 kB 00:00 2023-10-06T02:20:08.253 INFO:teuthology.orchestra.run.smithi031.stdout:(66/120): python3-websocket-client-0.56.0-5.el8 582 kB/s | 61 kB 00:00 2023-10-06T02:20:08.267 INFO:teuthology.orchestra.run.smithi031.stdout:(67/120): python3-webtest-2.0.33-1.el8.noarch.r 1.0 MB/s | 85 kB 00:00 2023-10-06T02:20:08.282 INFO:teuthology.orchestra.run.smithi062.stdout:(94/120): automake-1.16.1-7.el8.noarch.rpm 18 MB/s | 713 kB 00:00 2023-10-06T02:20:08.297 INFO:teuthology.orchestra.run.smithi062.stdout:(95/120): libaio-devel-0.3.112-1.el8.x86_64.rpm 1.2 MB/s | 19 kB 00:00 2023-10-06T02:20:08.313 INFO:teuthology.orchestra.run.smithi062.stdout:(96/120): python3-jwt-1.6.1-2.el8.noarch.rpm 2.8 MB/s | 43 kB 00:00 2023-10-06T02:20:08.331 INFO:teuthology.orchestra.run.smithi062.stdout:(97/120): python3-oauthlib-2.1.0-1.el8.noarch.r 8.3 MB/s | 155 kB 00:00 2023-10-06T02:20:08.334 INFO:teuthology.orchestra.run.smithi031.stdout:(68/120): python3-zc-lockfile-2.0-2.el8.noarch. 285 kB/s | 23 kB 00:00 2023-10-06T02:20:08.339 INFO:teuthology.orchestra.run.smithi031.stdout:(69/120): xmlstarlet-1.6.1-20.el8.x86_64.rpm 16 MB/s | 70 kB 00:00 2023-10-06T02:20:08.360 INFO:teuthology.orchestra.run.smithi062.stdout:(98/120): openblas-threads-0.3.12-1.el8.x86_64. 34 MB/s | 4.7 MB 00:00 2023-10-06T02:20:08.366 INFO:teuthology.orchestra.run.smithi062.stdout:(99/120): pcre2-utf32-10.32-2.el8.x86_64.rpm 6.3 MB/s | 220 kB 00:00 2023-10-06T02:20:08.386 INFO:teuthology.orchestra.run.smithi062.stdout:(100/120): openblas-0.3.12-1.el8.x86_64.rpm 30 MB/s | 4.6 MB 00:00 2023-10-06T02:20:08.387 INFO:teuthology.orchestra.run.smithi062.stdout:(101/120): keyutils-libs-devel-1.5.10-6.el8.x86 1.8 MB/s | 48 kB 00:00 2023-10-06T02:20:08.391 INFO:teuthology.orchestra.run.smithi062.stdout:(102/120): pcre2-utf16-10.32-2.el8.x86_64.rpm 9.1 MB/s | 229 kB 00:00 2023-10-06T02:20:08.404 INFO:teuthology.orchestra.run.smithi062.stdout:(103/120): python3-requests-oauthlib-1.0.0-1.el 2.3 MB/s | 43 kB 00:00 2023-10-06T02:20:08.409 INFO:teuthology.orchestra.run.smithi062.stdout:(104/120): libverto-devel-0.3.0-5.el8.x86_64.rp 997 kB/s | 18 kB 00:00 2023-10-06T02:20:08.439 INFO:teuthology.orchestra.run.smithi062.stdout:(105/120): pcre2-devel-10.32-2.el8.x86_64.rpm 17 MB/s | 605 kB 00:00 2023-10-06T02:20:08.441 INFO:teuthology.orchestra.run.smithi062.stdout:(106/120): fuse-2.9.7-12.el8.x86_64.rpm 2.5 MB/s | 83 kB 00:00 2023-10-06T02:20:08.453 INFO:teuthology.orchestra.run.smithi062.stdout:(107/120): openssl-devel-1.1.1g-15.el8_3.x86_64 36 MB/s | 2.3 MB 00:00 2023-10-06T02:20:08.461 INFO:teuthology.orchestra.run.smithi062.stdout:(108/120): elfutils-libelf-devel-0.182-3.el8.x8 2.7 MB/s | 59 kB 00:00 2023-10-06T02:20:08.463 INFO:teuthology.orchestra.run.smithi062.stdout:(109/120): libquadmath-8.4.1-1.el8.x86_64.rpm 7.6 MB/s | 169 kB 00:00 2023-10-06T02:20:08.471 INFO:teuthology.orchestra.run.smithi062.stdout:(110/120): libcom_err-devel-1.45.6-1.el8.x86_64 2.0 MB/s | 38 kB 00:00 2023-10-06T02:20:08.483 INFO:teuthology.orchestra.run.smithi062.stdout:(111/120): libsepol-devel-2.9-2.el8.x86_64.rpm 3.8 MB/s | 86 kB 00:00 2023-10-06T02:20:08.484 INFO:teuthology.orchestra.run.smithi062.stdout:(112/120): zlib-devel-1.2.11-17.el8.x86_64.rpm 2.7 MB/s | 58 kB 00:00 2023-10-06T02:20:08.490 INFO:teuthology.orchestra.run.smithi062.stdout:(113/120): librabbitmq-0.9.0-3.el8.x86_64.rpm 2.4 MB/s | 47 kB 00:00 2023-10-06T02:20:08.518 INFO:teuthology.orchestra.run.smithi062.stdout:(114/120): krb5-devel-1.18.2-8.3.el8_4.x86_64.r 16 MB/s | 559 kB 00:00 2023-10-06T02:20:08.519 INFO:teuthology.orchestra.run.smithi062.stdout:(115/120): xfsprogs-devel-5.0.0-8.el8.x86_64.rp 2.4 MB/s | 70 kB 00:00 2023-10-06T02:20:08.524 INFO:teuthology.orchestra.run.smithi062.stdout:(116/120): libgfortran-8.4.1-1.el8.x86_64.rpm 16 MB/s | 639 kB 00:00 2023-10-06T02:20:08.537 INFO:teuthology.orchestra.run.smithi062.stdout:(117/120): libselinux-devel-2.9-5.el8.x86_64.rp 10 MB/s | 200 kB 00:00 2023-10-06T02:20:08.539 INFO:teuthology.orchestra.run.smithi062.stdout:(118/120): libuuid-devel-2.32.1-27.el8.x86_64.r 4.9 MB/s | 97 kB 00:00 2023-10-06T02:20:08.658 INFO:teuthology.orchestra.run.smithi031.stdout:(70/120): python3-scipy-1.0.0-20.module+el8.1.0 46 MB/s | 15 MB 00:00 2023-10-06T02:20:08.677 INFO:teuthology.orchestra.run.smithi031.stdout:(71/120): python3-prettytable-0.7.2-14.el8.noar 2.4 MB/s | 44 kB 00:00 2023-10-06T02:20:08.695 INFO:teuthology.orchestra.run.smithi031.stdout:(72/120): python3-jsonpatch-1.21-2.el8.noarch.r 1.5 MB/s | 27 kB 00:00 2023-10-06T02:20:08.715 INFO:teuthology.orchestra.run.smithi031.stdout:(73/120): jq-1.5-12.el8.x86_64.rpm 8.0 MB/s | 161 kB 00:00 2023-10-06T02:20:08.739 INFO:teuthology.orchestra.run.smithi031.stdout:(74/120): flex-2.6.1-9.el8.x86_64.rpm 13 MB/s | 320 kB 00:00 2023-10-06T02:20:08.746 INFO:teuthology.orchestra.run.smithi031.stdout:(75/120): thrift-0.13.0-2.el8.x86_64.rpm 3.6 MB/s | 1.7 MB 00:00 2023-10-06T02:20:08.772 INFO:teuthology.orchestra.run.smithi031.stdout:(76/120): python3-werkzeug-0.12.2-4.el8.noarch. 14 MB/s | 457 kB 00:00 2023-10-06T02:20:08.790 INFO:teuthology.orchestra.run.smithi031.stdout:(77/120): python3-markupsafe-0.23-19.el8.x86_64 2.1 MB/s | 39 kB 00:00 2023-10-06T02:20:08.824 INFO:teuthology.orchestra.run.smithi031.stdout:(78/120): bison-3.0.4-10.el8.x86_64.rpm 21 MB/s | 688 kB 00:00 2023-10-06T02:20:08.846 INFO:teuthology.orchestra.run.smithi031.stdout:(79/120): lttng-ust-2.8.1-11.el8.x86_64.rpm 11 MB/s | 259 kB 00:00 2023-10-06T02:20:08.866 INFO:teuthology.orchestra.run.smithi062.stdout:(119/120): librbd1-17.2.6-1296.g736ac5ba.el8.x8 11 MB/s | 3.6 MB 00:00 2023-10-06T02:20:08.867 INFO:teuthology.orchestra.run.smithi031.stdout:(80/120): python3-numpy-f2py-1.14.3-9.el8.x86_6 10 MB/s | 225 kB 00:00 2023-10-06T02:20:08.887 INFO:teuthology.orchestra.run.smithi031.stdout:(81/120): python3-babel-2.5.1-5.el8.noarch.rpm 34 MB/s | 4.8 MB 00:00 2023-10-06T02:20:08.890 INFO:teuthology.orchestra.run.smithi031.stdout:(82/120): python3-pyasn1-0.3.7-6.el8.noarch.rpm 5.6 MB/s | 126 kB 00:00 2023-10-06T02:20:08.911 INFO:teuthology.orchestra.run.smithi031.stdout:(83/120): oniguruma-6.8.2-2.el8.x86_64.rpm 8.7 MB/s | 187 kB 00:00 2023-10-06T02:20:08.938 INFO:teuthology.orchestra.run.smithi031.stdout:(84/120): python3-jinja2-2.10.1-2.el8_0.noarch. 20 MB/s | 538 kB 00:00 2023-10-06T02:20:08.959 INFO:teuthology.orchestra.run.smithi031.stdout:(85/120): socat-1.7.3.3-2.el8.x86_64.rpm 14 MB/s | 302 kB 00:00 2023-10-06T02:20:08.981 INFO:teuthology.orchestra.run.smithi031.stdout:(86/120): python3-numpy-1.14.3-9.el8.x86_64.rpm 40 MB/s | 3.7 MB 00:00 2023-10-06T02:20:08.985 INFO:teuthology.orchestra.run.smithi031.stdout:(87/120): python3-mako-1.0.6-13.el8.noarch.rpm 6.0 MB/s | 157 kB 00:00 2023-10-06T02:20:08.998 INFO:teuthology.orchestra.run.smithi031.stdout:(88/120): python3-pyasn1-modules-0.3.7-6.el8.no 6.8 MB/s | 110 kB 00:00 2023-10-06T02:20:09.005 INFO:teuthology.orchestra.run.smithi031.stdout:(89/120): librdkafka-0.11.4-1.el8.x86_64.rpm 18 MB/s | 353 kB 00:00 2023-10-06T02:20:09.014 INFO:teuthology.orchestra.run.smithi031.stdout:(90/120): python3-jsonpointer-1.10-11.el8.noarc 1.3 MB/s | 20 kB 00:00 2023-10-06T02:20:09.033 INFO:teuthology.orchestra.run.smithi031.stdout:(91/120): libtool-2.4.6-25.el8.x86_64.rpm 25 MB/s | 709 kB 00:00 2023-10-06T02:20:09.034 INFO:teuthology.orchestra.run.smithi031.stdout:(92/120): python3-pytz-2017.2-9.el8.noarch.rpm 2.6 MB/s | 54 kB 00:00 2023-10-06T02:20:09.160 INFO:teuthology.orchestra.run.smithi031.stdout:(93/120): openblas-0.3.12-1.el8.x86_64.rpm 37 MB/s | 4.6 MB 00:00 2023-10-06T02:20:09.177 INFO:teuthology.orchestra.run.smithi031.stdout:(94/120): openblas-threads-0.3.12-1.el8.x86_64. 33 MB/s | 4.7 MB 00:00 2023-10-06T02:20:09.190 INFO:teuthology.orchestra.run.smithi031.stdout:(95/120): automake-1.16.1-7.el8.noarch.rpm 23 MB/s | 713 kB 00:00 2023-10-06T02:20:09.191 INFO:teuthology.orchestra.run.smithi031.stdout:(96/120): libaio-devel-0.3.112-1.el8.x86_64.rpm 1.4 MB/s | 19 kB 00:00 2023-10-06T02:20:09.204 INFO:teuthology.orchestra.run.smithi031.stdout:(97/120): python3-jwt-1.6.1-2.el8.noarch.rpm 3.2 MB/s | 43 kB 00:00 2023-10-06T02:20:09.207 INFO:teuthology.orchestra.run.smithi031.stdout:(98/120): python3-oauthlib-2.1.0-1.el8.noarch.r 10 MB/s | 155 kB 00:00 2023-10-06T02:20:09.222 INFO:teuthology.orchestra.run.smithi031.stdout:(99/120): pcre2-utf32-10.32-2.el8.x86_64.rpm 12 MB/s | 220 kB 00:00 2023-10-06T02:20:09.225 INFO:teuthology.orchestra.run.smithi031.stdout:(100/120): keyutils-libs-devel-1.5.10-6.el8.x86 2.5 MB/s | 48 kB 00:00 2023-10-06T02:20:09.240 INFO:teuthology.orchestra.run.smithi031.stdout:(101/120): python3-requests-oauthlib-1.0.0-1.el 2.9 MB/s | 43 kB 00:00 2023-10-06T02:20:09.241 INFO:teuthology.orchestra.run.smithi031.stdout:(102/120): pcre2-utf16-10.32-2.el8.x86_64.rpm 12 MB/s | 229 kB 00:00 2023-10-06T02:20:09.255 INFO:teuthology.orchestra.run.smithi031.stdout:(103/120): libverto-devel-0.3.0-5.el8.x86_64.rp 1.3 MB/s | 18 kB 00:00 2023-10-06T02:20:09.266 INFO:teuthology.orchestra.run.smithi062.stdout:(120/120): librados2-17.2.6-1296.g736ac5ba.el8. 4.7 MB/s | 3.5 MB 00:00 2023-10-06T02:20:09.271 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2023-10-06T02:20:09.271 INFO:teuthology.orchestra.run.smithi062.stdout:Total 18 MB/s | 201 MB 00:10 2023-10-06T02:20:09.281 INFO:teuthology.orchestra.run.smithi031.stdout:(104/120): pcre2-devel-10.32-2.el8.x86_64.rpm 23 MB/s | 605 kB 00:00 2023-10-06T02:20:09.302 INFO:teuthology.orchestra.run.smithi031.stdout:(105/120): openssl-devel-1.1.1g-15.el8_3.x86_64 38 MB/s | 2.3 MB 00:00 2023-10-06T02:20:09.303 INFO:teuthology.orchestra.run.smithi031.stdout:(106/120): fuse-2.9.7-12.el8.x86_64.rpm 3.7 MB/s | 83 kB 00:00 2023-10-06T02:20:09.318 INFO:teuthology.orchestra.run.smithi031.stdout:(107/120): elfutils-libelf-devel-0.182-3.el8.x8 3.6 MB/s | 59 kB 00:00 2023-10-06T02:20:09.319 INFO:teuthology.orchestra.run.smithi031.stdout:(108/120): libquadmath-8.4.1-1.el8.x86_64.rpm 10 MB/s | 169 kB 00:00 2023-10-06T02:20:09.332 INFO:teuthology.orchestra.run.smithi031.stdout:(109/120): libcom_err-devel-1.45.6-1.el8.x86_64 2.7 MB/s | 38 kB 00:00 2023-10-06T02:20:09.334 INFO:teuthology.orchestra.run.smithi031.stdout:(110/120): libsepol-devel-2.9-2.el8.x86_64.rpm 5.9 MB/s | 86 kB 00:00 2023-10-06T02:20:09.346 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction check 2023-10-06T02:20:09.346 INFO:teuthology.orchestra.run.smithi031.stdout:(111/120): zlib-devel-1.2.11-17.el8.x86_64.rpm 4.0 MB/s | 58 kB 00:00 2023-10-06T02:20:09.349 INFO:teuthology.orchestra.run.smithi031.stdout:(112/120): librabbitmq-0.9.0-3.el8.x86_64.rpm 3.2 MB/s | 47 kB 00:00 2023-10-06T02:20:09.374 INFO:teuthology.orchestra.run.smithi031.stdout:(113/120): krb5-devel-1.18.2-8.3.el8_4.x86_64.r 20 MB/s | 559 kB 00:00 2023-10-06T02:20:09.380 INFO:teuthology.orchestra.run.smithi031.stdout:(114/120): libgfortran-8.4.1-1.el8.x86_64.rpm 20 MB/s | 639 kB 00:00 2023-10-06T02:20:09.389 INFO:teuthology.orchestra.run.smithi031.stdout:(115/120): xfsprogs-devel-5.0.0-8.el8.x86_64.rp 4.8 MB/s | 70 kB 00:00 2023-10-06T02:20:09.397 INFO:teuthology.orchestra.run.smithi031.stdout:(116/120): libselinux-devel-2.9-5.el8.x86_64.rp 12 MB/s | 200 kB 00:00 2023-10-06T02:20:09.406 INFO:teuthology.orchestra.run.smithi031.stdout:(117/120): libuuid-devel-2.32.1-27.el8.x86_64.r 5.6 MB/s | 97 kB 00:00 2023-10-06T02:20:09.481 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction check succeeded. 2023-10-06T02:20:09.481 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction test 2023-10-06T02:20:09.692 INFO:teuthology.orchestra.run.smithi031.stdout:(118/120): librbd1-17.2.6-1296.g736ac5ba.el8.x8 13 MB/s | 3.6 MB 00:00 2023-10-06T02:20:09.829 INFO:teuthology.orchestra.run.smithi031.stdout:(119/120): librados2-17.2.6-1296.g736ac5ba.el8. 8.0 MB/s | 3.5 MB 00:00 2023-10-06T02:20:11.348 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction test succeeded. 2023-10-06T02:20:11.353 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction 2023-10-06T02:20:16.290 INFO:teuthology.orchestra.run.smithi062.stdout: Preparing : 1/1 2023-10-06T02:20:16.518 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/122 2023-10-06T02:20:16.774 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 2/122 2023-10-06T02:20:16.937 INFO:teuthology.orchestra.run.smithi031.stdout:(120/120): ceph-test-17.2.6-1296.g736ac5ba.el8. 3.9 MB/s | 52 MB 00:13 2023-10-06T02:20:16.941 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2023-10-06T02:20:16.941 INFO:teuthology.orchestra.run.smithi031.stdout:Total 9.6 MB/s | 201 MB 00:21 2023-10-06T02:20:16.983 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : librdkafka-0.11.4-1.el8.x86_64 3/122 2023-10-06T02:20:17.015 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2023-10-06T02:20:17.149 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2023-10-06T02:20:17.149 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2023-10-06T02:20:17.158 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librdkafka-0.11.4-1.el8.x86_64 3/122 2023-10-06T02:20:17.350 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/122 2023-10-06T02:20:17.857 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/122 2023-10-06T02:20:18.066 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : librados2-2:17.2.6-1296.g736ac5ba.el8.x86_64 5/122 2023-10-06T02:20:18.556 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librados2-2:17.2.6-1296.g736ac5ba.el8.x86_64 5/122 2023-10-06T02:20:18.800 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : librbd1-2:17.2.6-1296.g736ac5ba.el8.x86_64 6/122 2023-10-06T02:20:18.965 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librbd1-2:17.2.6-1296.g736ac5ba.el8.x86_64 6/122 2023-10-06T02:20:19.158 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libcephfs2-2:17.2.6-1296.g736ac5ba.el8.x86_64 7/122 2023-10-06T02:20:19.305 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libcephfs2-2:17.2.6-1296.g736ac5ba.el8.x86_64 7/122 2023-10-06T02:20:19.419 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2023-10-06T02:20:19.424 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2023-10-06T02:20:19.704 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-rados-2:17.2.6-1296.g736ac5ba.el8.x86_64 8/122 2023-10-06T02:20:20.476 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : thrift-0.13.0-2.el8.x86_64 9/122 2023-10-06T02:20:20.641 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : librgw2-2:17.2.6-1296.g736ac5ba.el8.x86_64 10/122 2023-10-06T02:20:20.807 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librgw2-2:17.2.6-1296.g736ac5ba.el8.x86_64 10/122 2023-10-06T02:20:21.545 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/122 2023-10-06T02:20:21.756 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/122 2023-10-06T02:20:21.991 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libcephsqlite-2:17.2.6-1296.g736ac5ba.el8.x86_64 13/122 2023-10-06T02:20:22.132 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libcephsqlite-2:17.2.6-1296.g736ac5ba.el8.x86_64 13/122 2023-10-06T02:20:22.324 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libradosstriper1-2:17.2.6-1296.g736ac5ba.el8.x86 14/122 2023-10-06T02:20:22.425 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libradosstriper1-2:17.2.6-1296.g736ac5ba.el8.x86 14/122 2023-10-06T02:20:22.660 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : zlib-devel-1.2.11-17.el8.x86_64 15/122 2023-10-06T02:20:22.841 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libquadmath-8.4.1-1.el8.x86_64 16/122 2023-10-06T02:20:23.016 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libquadmath-8.4.1-1.el8.x86_64 16/122 2023-10-06T02:20:23.199 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libgfortran-8.4.1-1.el8.x86_64 17/122 2023-10-06T02:20:23.858 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libgfortran-8.4.1-1.el8.x86_64 17/122 2023-10-06T02:20:24.041 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : openblas-threads-0.3.12-1.el8.x86_64 18/122 2023-10-06T02:20:24.277 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2023-10-06T02:20:24.477 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/122 2023-10-06T02:20:24.674 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 2/122 2023-10-06T02:20:24.684 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: openblas-threads-0.3.12-1.el8.x86_64 18/122 2023-10-06T02:20:24.880 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librdkafka-0.11.4-1.el8.x86_64 3/122 2023-10-06T02:20:25.024 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : openblas-0.3.12-1.el8.x86_64 19/122 2023-10-06T02:20:25.039 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librdkafka-0.11.4-1.el8.x86_64 3/122 2023-10-06T02:20:25.280 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/122 2023-10-06T02:20:25.739 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/122 2023-10-06T02:20:25.945 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: openblas-0.3.12-1.el8.x86_64 19/122 2023-10-06T02:20:25.955 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : librados2-2:17.2.6-1296.g736ac5ba.el8.x86_64 5/122 2023-10-06T02:20:26.434 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librados2-2:17.2.6-1296.g736ac5ba.el8.x86_64 5/122 2023-10-06T02:20:26.505 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-numpy-1:1.14.3-9.el8.x86_64 20/122 2023-10-06T02:20:26.672 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : librbd1-2:17.2.6-1296.g736ac5ba.el8.x86_64 6/122 2023-10-06T02:20:26.730 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 21/122 2023-10-06T02:20:26.831 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librbd1-2:17.2.6-1296.g736ac5ba.el8.x86_64 6/122 2023-10-06T02:20:26.953 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 22/122 2023-10-06T02:20:27.030 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libcephfs2-2:17.2.6-1296.g736ac5ba.el8.x86_64 7/122 2023-10-06T02:20:27.154 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libcephfs2-2:17.2.6-1296.g736ac5ba.el8.x86_64 7/122 2023-10-06T02:20:27.275 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/122 2023-10-06T02:20:27.528 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rados-2:17.2.6-1296.g736ac5ba.el8.x86_64 8/122 2023-10-06T02:20:27.603 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 24/122 2023-10-06T02:20:27.993 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/122 2023-10-06T02:20:28.207 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-ceph-common-2:17.2.6-1296.g736ac5ba.el8. 26/122 2023-10-06T02:20:28.282 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : thrift-0.13.0-2.el8.x86_64 9/122 2023-10-06T02:20:28.338 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-ceph-argparse-2:17.2.6-1296.g736ac5ba.el 27/122 2023-10-06T02:20:28.438 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librgw2-2:17.2.6-1296.g736ac5ba.el8.x86_64 10/122 2023-10-06T02:20:28.616 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librgw2-2:17.2.6-1296.g736ac5ba.el8.x86_64 10/122 2023-10-06T02:20:28.619 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cephfs-2:17.2.6-1296.g736ac5ba.el8.x86_6 28/122 2023-10-06T02:20:28.854 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/122 2023-10-06T02:20:29.080 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/122 2023-10-06T02:20:29.263 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libcephsqlite-2:17.2.6-1296.g736ac5ba.el8.x86_64 13/122 2023-10-06T02:20:29.405 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libcephsqlite-2:17.2.6-1296.g736ac5ba.el8.x86_64 13/122 2023-10-06T02:20:29.546 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libradosstriper1-2:17.2.6-1296.g736ac5ba.el8.x86 14/122 2023-10-06T02:20:29.619 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libradosstriper1-2:17.2.6-1296.g736ac5ba.el8.x86 14/122 2023-10-06T02:20:29.855 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : zlib-devel-1.2.11-17.el8.x86_64 15/122 2023-10-06T02:20:29.949 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 29/122 2023-10-06T02:20:30.013 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libquadmath-8.4.1-1.el8.x86_64 16/122 2023-10-06T02:20:30.225 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libquadmath-8.4.1-1.el8.x86_64 16/122 2023-10-06T02:20:30.305 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-babel-2.5.1-5.el8.noarch 30/122 2023-10-06T02:20:30.387 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libgfortran-8.4.1-1.el8.x86_64 17/122 2023-10-06T02:20:30.647 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jinja2-2.10.1-2.el8_0.noarch 31/122 2023-10-06T02:20:30.954 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 32/122 2023-10-06T02:20:31.079 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libgfortran-8.4.1-1.el8.x86_64 17/122 2023-10-06T02:20:31.263 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : openblas-threads-0.3.12-1.el8.x86_64 18/122 2023-10-06T02:20:31.269 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 33/122 2023-10-06T02:20:31.523 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 34/122 2023-10-06T02:20:31.888 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: openblas-threads-0.3.12-1.el8.x86_64 18/122 2023-10-06T02:20:32.079 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : openblas-0.3.12-1.el8.x86_64 19/122 2023-10-06T02:20:32.990 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: openblas-0.3.12-1.el8.x86_64 19/122 2023-10-06T02:20:33.281 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-numpy-1:1.14.3-9.el8.x86_64 20/122 2023-10-06T02:20:33.449 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 21/122 2023-10-06T02:20:33.663 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 22/122 2023-10-06T02:20:33.929 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/122 2023-10-06T02:20:34.101 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 34/122 2023-10-06T02:20:34.224 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 24/122 2023-10-06T02:20:34.461 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 35/122 2023-10-06T02:20:34.506 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/122 2023-10-06T02:20:34.720 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-ceph-common-2:17.2.6-1296.g736ac5ba.el8. 26/122 2023-10-06T02:20:34.791 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-rsa-4.9-2.el8.noarch 36/122 2023-10-06T02:20:34.876 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-ceph-argparse-2:17.2.6-1296.g736ac5ba.el 27/122 2023-10-06T02:20:35.020 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 37/122 2023-10-06T02:20:35.181 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cephfs-2:17.2.6-1296.g736ac5ba.el8.x86_6 28/122 2023-10-06T02:20:35.213 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-rgw-2:17.2.6-1296.g736ac5ba.el8.x86_64 38/122 2023-10-06T02:20:35.417 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-rbd-2:17.2.6-1296.g736ac5ba.el8.x86_64 39/122 2023-10-06T02:20:35.641 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : librados-devel-2:17.2.6-1296.g736ac5ba.el8.x86_6 40/122 2023-10-06T02:20:35.808 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libuuid-devel-2.32.1-27.el8.x86_64 41/122 2023-10-06T02:20:36.004 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libsepol-devel-2.9-2.el8.x86_64 42/122 2023-10-06T02:20:36.203 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libcom_err-devel-1.45.6-1.el8.x86_64 43/122 2023-10-06T02:20:36.406 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 29/122 2023-10-06T02:20:36.418 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : fuse-2.9.7-12.el8.x86_64 44/122 2023-10-06T02:20:36.582 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 45/122 2023-10-06T02:20:36.793 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-babel-2.5.1-5.el8.noarch 30/122 2023-10-06T02:20:36.818 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 46/122 2023-10-06T02:20:36.989 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : keyutils-libs-devel-1.5.10-6.el8.x86_64 47/122 2023-10-06T02:20:37.127 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jinja2-2.10.1-2.el8_0.noarch 31/122 2023-10-06T02:20:37.369 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 48/122 2023-10-06T02:20:37.409 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 32/122 2023-10-06T02:20:37.662 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 49/122 2023-10-06T02:20:37.683 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 33/122 2023-10-06T02:20:37.953 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 34/122 2023-10-06T02:20:37.970 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 50/122 2023-10-06T02:20:38.349 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : krb5-devel-1.18.2-8.3.el8_4.x86_64 51/122 2023-10-06T02:20:38.598 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : automake-1.16.1-7.el8.noarch 52/122 2023-10-06T02:20:38.824 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 53/122 2023-10-06T02:20:39.026 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 54/122 2023-10-06T02:20:39.246 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : socat-1.7.3.3-2.el8.x86_64 55/122 2023-10-06T02:20:39.457 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 56/122 2023-10-06T02:20:39.557 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 56/122 2023-10-06T02:20:39.758 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : jq-1.5-12.el8.x86_64 57/122 2023-10-06T02:20:39.918 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 58/122 2023-10-06T02:20:40.109 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 59/122 2023-10-06T02:20:40.301 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 60/122 2023-10-06T02:20:40.539 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 34/122 2023-10-06T02:20:40.575 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 61/122 2023-10-06T02:20:40.791 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 62/122 2023-10-06T02:20:40.883 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 35/122 2023-10-06T02:20:41.031 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 63/122 2023-10-06T02:20:41.121 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rsa-4.9-2.el8.noarch 36/122 2023-10-06T02:20:41.212 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 64/122 2023-10-06T02:20:41.322 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 37/122 2023-10-06T02:20:41.370 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 65/122 2023-10-06T02:20:41.524 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rgw-2:17.2.6-1296.g736ac5ba.el8.x86_64 38/122 2023-10-06T02:20:41.676 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 66/122 2023-10-06T02:20:41.712 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rbd-2:17.2.6-1296.g736ac5ba.el8.x86_64 39/122 2023-10-06T02:20:41.907 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librados-devel-2:17.2.6-1296.g736ac5ba.el8.x86_6 40/122 2023-10-06T02:20:41.927 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 67/122 2023-10-06T02:20:42.095 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libuuid-devel-2.32.1-27.el8.x86_64 41/122 2023-10-06T02:20:42.176 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 68/122 2023-10-06T02:20:42.251 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libsepol-devel-2.9-2.el8.x86_64 42/122 2023-10-06T02:20:42.368 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 69/122 2023-10-06T02:20:42.441 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libcom_err-devel-1.45.6-1.el8.x86_64 43/122 2023-10-06T02:20:42.574 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 70/122 2023-10-06T02:20:42.631 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : fuse-2.9.7-12.el8.x86_64 44/122 2023-10-06T02:20:42.811 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 45/122 2023-10-06T02:20:42.854 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 71/122 2023-10-06T02:20:42.992 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 46/122 2023-10-06T02:20:43.114 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 72/122 2023-10-06T02:20:43.210 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : keyutils-libs-devel-1.5.10-6.el8.x86_64 47/122 2023-10-06T02:20:43.356 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 73/122 2023-10-06T02:20:43.546 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 48/122 2023-10-06T02:20:43.692 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-portend-2.6-1.el8.noarch 74/122 2023-10-06T02:20:43.843 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 49/122 2023-10-06T02:20:43.966 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 75/122 2023-10-06T02:20:44.130 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 50/122 2023-10-06T02:20:44.144 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 76/122 2023-10-06T02:20:44.447 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 77/122 2023-10-06T02:20:44.538 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : krb5-devel-1.18.2-8.3.el8_4.x86_64 51/122 2023-10-06T02:20:44.795 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : automake-1.16.1-7.el8.noarch 52/122 2023-10-06T02:20:44.962 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 53/122 2023-10-06T02:20:45.174 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 54/122 2023-10-06T02:20:45.359 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : socat-1.7.3.3-2.el8.x86_64 55/122 2023-10-06T02:20:45.535 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 56/122 2023-10-06T02:20:45.640 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 56/122 2023-10-06T02:20:45.880 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : jq-1.5-12.el8.x86_64 57/122 2023-10-06T02:20:45.954 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 78/122 2023-10-06T02:20:46.057 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 58/122 2023-10-06T02:20:46.265 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 59/122 2023-10-06T02:20:46.314 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 79/122 2023-10-06T02:20:46.459 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 60/122 2023-10-06T02:20:46.765 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 61/122 2023-10-06T02:20:46.884 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 80/122 2023-10-06T02:20:47.015 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 62/122 2023-10-06T02:20:47.220 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 63/122 2023-10-06T02:20:47.240 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 81/122 2023-10-06T02:20:47.401 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 64/122 2023-10-06T02:20:47.544 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 65/122 2023-10-06T02:20:47.554 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 82/122 2023-10-06T02:20:47.786 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 66/122 2023-10-06T02:20:47.901 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 83/122 2023-10-06T02:20:47.994 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 67/122 2023-10-06T02:20:48.190 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 84/122 2023-10-06T02:20:48.234 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 68/122 2023-10-06T02:20:48.370 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 85/122 2023-10-06T02:20:48.482 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 69/122 2023-10-06T02:20:48.580 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 86/122 2023-10-06T02:20:48.662 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 70/122 2023-10-06T02:20:48.958 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 71/122 2023-10-06T02:20:49.255 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 72/122 2023-10-06T02:20:49.820 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 73/122 2023-10-06T02:20:50.163 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-portend-2.6-1.el8.noarch 74/122 2023-10-06T02:20:50.429 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 75/122 2023-10-06T02:20:50.634 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 76/122 2023-10-06T02:20:50.925 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 77/122 2023-10-06T02:20:52.445 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 78/122 2023-10-06T02:20:52.777 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 79/122 2023-10-06T02:20:53.028 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 80/122 2023-10-06T02:20:53.389 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 81/122 2023-10-06T02:20:53.689 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 82/122 2023-10-06T02:20:54.082 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 83/122 2023-10-06T02:20:54.331 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 84/122 2023-10-06T02:20:54.558 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 85/122 2023-10-06T02:20:54.776 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 86/122 2023-10-06T02:20:56.269 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-common-2:17.2.6-1296.g736ac5ba.el8.x86_64 87/122 2023-10-06T02:20:56.521 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-common-2:17.2.6-1296.g736ac5ba.el8.x86_64 87/122 2023-10-06T02:20:57.568 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-common-2:17.2.6-1296.g736ac5ba.el8.x86_64 87/122 2023-10-06T02:20:57.869 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-base-2:17.2.6-1296.g736ac5ba.el8.x86_64 88/122 2023-10-06T02:20:58.066 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-base-2:17.2.6-1296.g736ac5ba.el8.x86_64 88/122 2023-10-06T02:20:58.244 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-selinux-2:17.2.6-1296.g736ac5ba.el8.x86_64 89/122 2023-10-06T02:21:02.513 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-common-2:17.2.6-1296.g736ac5ba.el8.x86_64 87/122 2023-10-06T02:21:02.758 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-common-2:17.2.6-1296.g736ac5ba.el8.x86_64 87/122 2023-10-06T02:21:03.477 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-common-2:17.2.6-1296.g736ac5ba.el8.x86_64 87/122 2023-10-06T02:21:03.757 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-base-2:17.2.6-1296.g736ac5ba.el8.x86_64 88/122 2023-10-06T02:21:03.911 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-base-2:17.2.6-1296.g736ac5ba.el8.x86_64 88/122 2023-10-06T02:21:04.099 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-selinux-2:17.2.6-1296.g736ac5ba.el8.x86_64 89/122 2023-10-06T02:21:18.309 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-selinux-2:17.2.6-1296.g736ac5ba.el8.x86_64 89/122 2023-10-06T02:21:18.309 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /sys 2023-10-06T02:21:18.309 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /proc 2023-10-06T02:21:18.309 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /dev 2023-10-06T02:21:18.309 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /run 2023-10-06T02:21:18.309 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /mnt 2023-10-06T02:21:18.309 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /var/tmp 2023-10-06T02:21:18.310 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /home 2023-10-06T02:21:18.310 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /tmp 2023-10-06T02:21:18.310 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /dev 2023-10-06T02:21:18.310 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-10-06T02:21:18.760 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-volume-2:17.2.6-1296.g736ac5ba.el8.noarch 90/122 2023-10-06T02:21:20.494 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-volume-2:17.2.6-1296.g736ac5ba.el8.noarch 90/122 2023-10-06T02:21:20.687 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-osd-2:17.2.6-1296.g736ac5ba.el8.x86_64 91/122 2023-10-06T02:21:20.975 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-osd-2:17.2.6-1296.g736ac5ba.el8.x86_64 91/122 2023-10-06T02:21:21.126 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mds-2:17.2.6-1296.g736ac5ba.el8.x86_64 92/122 2023-10-06T02:21:21.633 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mds-2:17.2.6-1296.g736ac5ba.el8.x86_64 92/122 2023-10-06T02:21:21.775 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mon-2:17.2.6-1296.g736ac5ba.el8.x86_64 93/122 2023-10-06T02:21:21.817 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mon-2:17.2.6-1296.g736ac5ba.el8.x86_64 93/122 2023-10-06T02:21:24.021 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-selinux-2:17.2.6-1296.g736ac5ba.el8.x86_64 89/122 2023-10-06T02:21:24.022 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /sys 2023-10-06T02:21:24.022 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /proc 2023-10-06T02:21:24.022 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /dev 2023-10-06T02:21:24.022 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /run 2023-10-06T02:21:24.022 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /mnt 2023-10-06T02:21:24.022 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /var/tmp 2023-10-06T02:21:24.022 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /home 2023-10-06T02:21:24.022 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /tmp 2023-10-06T02:21:24.022 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /dev 2023-10-06T02:21:24.022 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:21:24.446 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-volume-2:17.2.6-1296.g736ac5ba.el8.noarch 90/122 2023-10-06T02:21:26.184 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-volume-2:17.2.6-1296.g736ac5ba.el8.noarch 90/122 2023-10-06T02:21:26.345 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-osd-2:17.2.6-1296.g736ac5ba.el8.x86_64 91/122 2023-10-06T02:21:26.503 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: cephadm-2:17.2.6-1296.g736ac5ba.el8.noarch 94/122 2023-10-06T02:21:26.644 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-osd-2:17.2.6-1296.g736ac5ba.el8.x86_64 91/122 2023-10-06T02:21:26.728 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : cephadm-2:17.2.6-1296.g736ac5ba.el8.noarch 94/122 2023-10-06T02:21:26.804 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mds-2:17.2.6-1296.g736ac5ba.el8.x86_64 92/122 2023-10-06T02:21:26.975 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-prometheus-alerts-2:17.2.6-1296.g736ac5ba.e 95/122 2023-10-06T02:21:27.223 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-grafana-dashboards-2:17.2.6-1296.g736ac5ba. 96/122 2023-10-06T02:21:27.298 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mds-2:17.2.6-1296.g736ac5ba.el8.x86_64 92/122 2023-10-06T02:21:27.446 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mon-2:17.2.6-1296.g736ac5ba.el8.x86_64 93/122 2023-10-06T02:21:27.487 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mon-2:17.2.6-1296.g736ac5ba.el8.x86_64 93/122 2023-10-06T02:21:27.498 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-cephadm-2:17.2.6-1296.g736ac5ba.el8.noa 97/122 2023-10-06T02:21:27.781 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.6-1296.g736ac5ba.el8.noa 97/122 2023-10-06T02:21:27.982 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-dashboard-2:17.2.6-1296.g736ac5ba.el8.n 98/122 2023-10-06T02:21:29.170 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.6-1296.g736ac5ba.el8.n 98/122 2023-10-06T02:21:29.382 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.6-1296.g736 99/122 2023-10-06T02:21:29.528 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.6-1296.g736 99/122 2023-10-06T02:21:29.919 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-modules-core-2:17.2.6-1296.g736ac5ba.el 100/122 2023-10-06T02:21:30.140 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-2:17.2.6-1296.g736ac5ba.el8.x86_64 101/122 2023-10-06T02:21:30.211 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-2:17.2.6-1296.g736ac5ba.el8.x86_64 101/122 2023-10-06T02:21:30.399 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-rook-2:17.2.6-1296.g736ac5ba.el8.noarch 102/122 2023-10-06T02:21:30.427 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.6-1296.g736ac5ba.el8.noarch 102/122 2023-10-06T02:21:32.136 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: cephadm-2:17.2.6-1296.g736ac5ba.el8.noarch 94/122 2023-10-06T02:21:32.261 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-2:17.2.6-1296.g736ac5ba.el8.x86_64 103/122 2023-10-06T02:21:32.331 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : cephadm-2:17.2.6-1296.g736ac5ba.el8.noarch 94/122 2023-10-06T02:21:32.513 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-prometheus-alerts-2:17.2.6-1296.g736ac5ba.e 95/122 2023-10-06T02:21:32.540 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-radosgw-2:17.2.6-1296.g736ac5ba.el8.x86_64 104/122 2023-10-06T02:21:32.671 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-radosgw-2:17.2.6-1296.g736ac5ba.el8.x86_64 104/122 2023-10-06T02:21:32.725 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-grafana-dashboards-2:17.2.6-1296.g736ac5ba. 96/122 2023-10-06T02:21:32.866 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-immutable-object-cache-2:17.2.6-1296.g736ac 105/122 2023-10-06T02:21:32.940 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-cephadm-2:17.2.6-1296.g736ac5ba.el8.noa 97/122 2023-10-06T02:21:32.942 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.6-1296.g736ac 105/122 2023-10-06T02:21:33.132 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : cephfs-mirror-2:17.2.6-1296.g736ac5ba.el8.x86_64 106/122 2023-10-06T02:21:33.220 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.6-1296.g736ac5ba.el8.noa 97/122 2023-10-06T02:21:33.435 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-dashboard-2:17.2.6-1296.g736ac5ba.el8.n 98/122 2023-10-06T02:21:33.570 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: cephfs-mirror-2:17.2.6-1296.g736ac5ba.el8.x86_64 106/122 2023-10-06T02:21:33.765 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : rbd-mirror-2:17.2.6-1296.g736ac5ba.el8.x86_64 107/122 2023-10-06T02:21:34.623 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.6-1296.g736ac5ba.el8.n 98/122 2023-10-06T02:21:34.819 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.6-1296.g736 99/122 2023-10-06T02:21:34.965 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.6-1296.g736 99/122 2023-10-06T02:21:35.389 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-modules-core-2:17.2.6-1296.g736ac5ba.el 100/122 2023-10-06T02:21:35.577 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-2:17.2.6-1296.g736ac5ba.el8.x86_64 101/122 2023-10-06T02:21:35.648 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-2:17.2.6-1296.g736ac5ba.el8.x86_64 101/122 2023-10-06T02:21:35.777 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-rook-2:17.2.6-1296.g736ac5ba.el8.noarch 102/122 2023-10-06T02:21:35.805 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.6-1296.g736ac5ba.el8.noarch 102/122 2023-10-06T02:21:37.701 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-2:17.2.6-1296.g736ac5ba.el8.x86_64 103/122 2023-10-06T02:21:37.960 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-radosgw-2:17.2.6-1296.g736ac5ba.el8.x86_64 104/122 2023-10-06T02:21:38.088 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-radosgw-2:17.2.6-1296.g736ac5ba.el8.x86_64 104/122 2023-10-06T02:21:38.277 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-immutable-object-cache-2:17.2.6-1296.g736ac 105/122 2023-10-06T02:21:38.360 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.6-1296.g736ac 105/122 2023-10-06T02:21:38.502 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : cephfs-mirror-2:17.2.6-1296.g736ac5ba.el8.x86_64 106/122 2023-10-06T02:21:38.937 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: cephfs-mirror-2:17.2.6-1296.g736ac5ba.el8.x86_64 106/122 2023-10-06T02:21:39.102 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : rbd-mirror-2:17.2.6-1296.g736ac5ba.el8.x86_64 107/122 2023-10-06T02:21:39.616 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: rbd-mirror-2:17.2.6-1296.g736ac5ba.el8.x86_64 107/122 2023-10-06T02:21:40.019 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-test-2:17.2.6-1296.g736ac5ba.el8.x86_64 108/122 2023-10-06T02:21:40.214 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libtool-2.4.6-25.el8.x86_64 109/122 2023-10-06T02:21:41.377 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 109/122 2023-10-06T02:21:41.900 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : openssl-devel-1:1.1.1g-15.el8_3.x86_64 110/122 2023-10-06T02:21:42.087 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-fuse-2:17.2.6-1296.g736ac5ba.el8.x86_64 111/122 2023-10-06T02:21:42.302 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : xfsprogs-devel-5.0.0-8.el8.x86_64 112/122 2023-10-06T02:21:42.478 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libcephfs-devel-2:17.2.6-1296.g736ac5ba.el8.x86_ 113/122 2023-10-06T02:21:42.629 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : elfutils-libelf-devel-0.182-3.el8.x86_64 114/122 2023-10-06T02:21:42.832 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : cephfs-top-2:17.2.6-1296.g736ac5ba.el8.noarch 115/122 2023-10-06T02:21:43.008 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : rbd-fuse-2:17.2.6-1296.g736ac5ba.el8.x86_64 116/122 2023-10-06T02:21:43.166 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : rbd-nbd-2:17.2.6-1296.g736ac5ba.el8.x86_64 117/122 2023-10-06T02:21:43.413 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 118/122 2023-10-06T02:21:43.555 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : bison-3.0.4-10.el8.x86_64 119/122 2023-10-06T02:21:43.637 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 119/122 2023-10-06T02:21:43.847 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : flex-2.6.1-9.el8.x86_64 120/122 2023-10-06T02:21:43.871 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 120/122 2023-10-06T02:21:43.872 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-10-06T02:21:44.112 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-10-06T02:21:44.112 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 122/122 2023-10-06T02:21:44.469 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 122/122 2023-10-06T02:21:44.469 INFO:teuthology.orchestra.run.smithi062.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-10-06T02:21:44.469 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-10-06T02:21:44.831 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: rbd-mirror-2:17.2.6-1296.g736ac5ba.el8.x86_64 107/122 2023-10-06T02:21:45.195 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-test-2:17.2.6-1296.g736ac5ba.el8.x86_64 108/122 2023-10-06T02:21:45.384 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libtool-2.4.6-25.el8.x86_64 109/122 2023-10-06T02:21:45.503 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-2:17.2.6-1296.g736ac5ba.el8.x86_64 1/122 2023-10-06T02:21:45.503 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-base-2:17.2.6-1296.g736ac5ba.el8.x86_64 2/122 2023-10-06T02:21:45.503 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-common-2:17.2.6-1296.g736ac5ba.el8.x86_64 3/122 2023-10-06T02:21:45.503 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-fuse-2:17.2.6-1296.g736ac5ba.el8.x86_64 4/122 2023-10-06T02:21:45.503 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-immutable-object-cache-2:17.2.6-1296.g736ac 5/122 2023-10-06T02:21:45.503 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mds-2:17.2.6-1296.g736ac5ba.el8.x86_64 6/122 2023-10-06T02:21:45.503 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-2:17.2.6-1296.g736ac5ba.el8.x86_64 7/122 2023-10-06T02:21:45.503 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mon-2:17.2.6-1296.g736ac5ba.el8.x86_64 8/122 2023-10-06T02:21:45.504 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-osd-2:17.2.6-1296.g736ac5ba.el8.x86_64 9/122 2023-10-06T02:21:45.504 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-radosgw-2:17.2.6-1296.g736ac5ba.el8.x86_64 10/122 2023-10-06T02:21:45.504 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-selinux-2:17.2.6-1296.g736ac5ba.el8.x86_64 11/122 2023-10-06T02:21:45.504 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-test-2:17.2.6-1296.g736ac5ba.el8.x86_64 12/122 2023-10-06T02:21:45.504 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : cephfs-mirror-2:17.2.6-1296.g736ac5ba.el8.x86_64 13/122 2023-10-06T02:21:45.504 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libcephfs-devel-2:17.2.6-1296.g736ac5ba.el8.x86_ 14/122 2023-10-06T02:21:45.504 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libcephfs2-2:17.2.6-1296.g736ac5ba.el8.x86_64 15/122 2023-10-06T02:21:45.504 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libcephsqlite-2:17.2.6-1296.g736ac5ba.el8.x86_64 16/122 2023-10-06T02:21:45.504 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librados-devel-2:17.2.6-1296.g736ac5ba.el8.x86_6 17/122 2023-10-06T02:21:45.504 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libradosstriper1-2:17.2.6-1296.g736ac5ba.el8.x86 18/122 2023-10-06T02:21:45.504 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librgw2-2:17.2.6-1296.g736ac5ba.el8.x86_64 19/122 2023-10-06T02:21:45.504 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-ceph-argparse-2:17.2.6-1296.g736ac5ba.el 20/122 2023-10-06T02:21:45.505 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-ceph-common-2:17.2.6-1296.g736ac5ba.el8. 21/122 2023-10-06T02:21:45.505 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cephfs-2:17.2.6-1296.g736ac5ba.el8.x86_6 22/122 2023-10-06T02:21:45.505 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-rados-2:17.2.6-1296.g736ac5ba.el8.x86_64 23/122 2023-10-06T02:21:45.505 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-rbd-2:17.2.6-1296.g736ac5ba.el8.x86_64 24/122 2023-10-06T02:21:45.505 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-rgw-2:17.2.6-1296.g736ac5ba.el8.x86_64 25/122 2023-10-06T02:21:45.505 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : rbd-fuse-2:17.2.6-1296.g736ac5ba.el8.x86_64 26/122 2023-10-06T02:21:45.505 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : rbd-mirror-2:17.2.6-1296.g736ac5ba.el8.x86_64 27/122 2023-10-06T02:21:45.505 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : rbd-nbd-2:17.2.6-1296.g736ac5ba.el8.x86_64 28/122 2023-10-06T02:21:45.505 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-grafana-dashboards-2:17.2.6-1296.g736ac5ba. 29/122 2023-10-06T02:21:45.505 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-cephadm-2:17.2.6-1296.g736ac5ba.el8.noa 30/122 2023-10-06T02:21:45.505 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-dashboard-2:17.2.6-1296.g736ac5ba.el8.n 31/122 2023-10-06T02:21:45.505 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.6-1296.g736 32/122 2023-10-06T02:21:45.506 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-modules-core-2:17.2.6-1296.g736ac5ba.el 33/122 2023-10-06T02:21:45.506 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-rook-2:17.2.6-1296.g736ac5ba.el8.noarch 34/122 2023-10-06T02:21:45.506 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-prometheus-alerts-2:17.2.6-1296.g736ac5ba.e 35/122 2023-10-06T02:21:45.506 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-volume-2:17.2.6-1296.g736ac5ba.el8.noarch 36/122 2023-10-06T02:21:45.506 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : cephadm-2:17.2.6-1296.g736ac5ba.el8.noarch 37/122 2023-10-06T02:21:45.506 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : cephfs-top-2:17.2.6-1296.g736ac5ba.el8.noarch 38/122 2023-10-06T02:21:45.506 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/122 2023-10-06T02:21:45.506 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/122 2023-10-06T02:21:45.506 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/122 2023-10-06T02:21:45.506 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/122 2023-10-06T02:21:45.506 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/122 2023-10-06T02:21:45.506 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/122 2023-10-06T02:21:45.507 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/122 2023-10-06T02:21:45.507 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/122 2023-10-06T02:21:45.507 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/122 2023-10-06T02:21:45.507 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/122 2023-10-06T02:21:45.507 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/122 2023-10-06T02:21:45.507 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/122 2023-10-06T02:21:45.507 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/122 2023-10-06T02:21:45.507 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/122 2023-10-06T02:21:45.507 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/122 2023-10-06T02:21:45.507 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/122 2023-10-06T02:21:45.507 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/122 2023-10-06T02:21:45.507 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/122 2023-10-06T02:21:45.508 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/122 2023-10-06T02:21:45.508 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/122 2023-10-06T02:21:45.508 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/122 2023-10-06T02:21:45.508 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/122 2023-10-06T02:21:45.508 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/122 2023-10-06T02:21:45.508 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/122 2023-10-06T02:21:45.508 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/122 2023-10-06T02:21:45.508 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/122 2023-10-06T02:21:45.508 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/122 2023-10-06T02:21:45.508 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/122 2023-10-06T02:21:45.508 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/122 2023-10-06T02:21:45.508 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/122 2023-10-06T02:21:45.509 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/122 2023-10-06T02:21:45.509 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/122 2023-10-06T02:21:45.509 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/122 2023-10-06T02:21:45.509 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 72/122 2023-10-06T02:21:45.509 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 73/122 2023-10-06T02:21:45.509 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 74/122 2023-10-06T02:21:45.509 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : jq-1.5-12.el8.x86_64 75/122 2023-10-06T02:21:45.509 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : flex-2.6.1-9.el8.x86_64 76/122 2023-10-06T02:21:45.509 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 77/122 2023-10-06T02:21:45.509 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-babel-2.5.1-5.el8.noarch 78/122 2023-10-06T02:21:45.509 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 79/122 2023-10-06T02:21:45.509 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : bison-3.0.4-10.el8.x86_64 80/122 2023-10-06T02:21:45.509 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 81/122 2023-10-06T02:21:45.510 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 82/122 2023-10-06T02:21:45.510 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 83/122 2023-10-06T02:21:45.510 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-numpy-1:1.14.3-9.el8.x86_64 84/122 2023-10-06T02:21:45.510 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 85/122 2023-10-06T02:21:45.510 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jinja2-2.10.1-2.el8_0.noarch 86/122 2023-10-06T02:21:45.510 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : socat-1.7.3.3-2.el8.x86_64 87/122 2023-10-06T02:21:45.510 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 88/122 2023-10-06T02:21:45.510 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 89/122 2023-10-06T02:21:45.510 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librdkafka-0.11.4-1.el8.x86_64 90/122 2023-10-06T02:21:45.510 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 91/122 2023-10-06T02:21:45.510 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 92/122 2023-10-06T02:21:45.510 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 93/122 2023-10-06T02:21:45.511 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openblas-threads-0.3.12-1.el8.x86_64 94/122 2023-10-06T02:21:45.511 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openblas-0.3.12-1.el8.x86_64 95/122 2023-10-06T02:21:45.511 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : automake-1.16.1-7.el8.noarch 96/122 2023-10-06T02:21:45.511 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 97/122 2023-10-06T02:21:45.511 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 98/122 2023-10-06T02:21:45.511 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 99/122 2023-10-06T02:21:45.511 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 100/122 2023-10-06T02:21:45.511 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : keyutils-libs-devel-1.5.10-6.el8.x86_64 101/122 2023-10-06T02:21:45.511 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 102/122 2023-10-06T02:21:45.511 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 103/122 2023-10-06T02:21:45.511 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openssl-devel-1:1.1.1g-15.el8_3.x86_64 104/122 2023-10-06T02:21:45.511 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 105/122 2023-10-06T02:21:45.511 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 106/122 2023-10-06T02:21:45.512 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : fuse-2.9.7-12.el8.x86_64 107/122 2023-10-06T02:21:45.512 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : elfutils-libelf-devel-0.182-3.el8.x86_64 108/122 2023-10-06T02:21:45.512 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libquadmath-8.4.1-1.el8.x86_64 109/122 2023-10-06T02:21:45.512 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libcom_err-devel-1.45.6-1.el8.x86_64 110/122 2023-10-06T02:21:45.512 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libsepol-devel-2.9-2.el8.x86_64 111/122 2023-10-06T02:21:45.512 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : zlib-devel-1.2.11-17.el8.x86_64 112/122 2023-10-06T02:21:45.512 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 113/122 2023-10-06T02:21:45.512 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : krb5-devel-1.18.2-8.3.el8_4.x86_64 114/122 2023-10-06T02:21:45.512 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libgfortran-8.4.1-1.el8.x86_64 115/122 2023-10-06T02:21:45.512 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : xfsprogs-devel-5.0.0-8.el8.x86_64 116/122 2023-10-06T02:21:45.512 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 117/122 2023-10-06T02:21:45.512 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libuuid-devel-2.32.1-27.el8.x86_64 118/122 2023-10-06T02:21:45.513 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librados2-2:17.2.6-1296.g736ac5ba.el8.x86_64 119/122 2023-10-06T02:21:45.513 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 120/122 2023-10-06T02:21:45.513 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librbd1-2:17.2.6-1296.g736ac5ba.el8.x86_64 121/122 2023-10-06T02:21:46.534 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 109/122 2023-10-06T02:21:46.883 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : openssl-devel-1:1.1.1g-15.el8_3.x86_64 110/122 2023-10-06T02:21:47.085 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-fuse-2:17.2.6-1296.g736ac5ba.el8.x86_64 111/122 2023-10-06T02:21:47.280 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : xfsprogs-devel-5.0.0-8.el8.x86_64 112/122 2023-10-06T02:21:47.448 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libcephfs-devel-2:17.2.6-1296.g736ac5ba.el8.x86_ 113/122 2023-10-06T02:21:47.641 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : elfutils-libelf-devel-0.182-3.el8.x86_64 114/122 2023-10-06T02:21:47.929 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : cephfs-top-2:17.2.6-1296.g736ac5ba.el8.noarch 115/122 2023-10-06T02:21:47.981 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 122/122 2023-10-06T02:21:48.073 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:01:59 ago on Fri 06 Oct 2023 02:19:48 AM UTC. 2023-10-06T02:21:48.074 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : rbd-fuse-2:17.2.6-1296.g736ac5ba.el8.x86_64 116/122 2023-10-06T02:21:48.169 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : rbd-nbd-2:17.2.6-1296.g736ac5ba.el8.x86_64 117/122 2023-10-06T02:21:48.431 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 118/122 2023-10-06T02:21:48.617 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : bison-3.0.4-10.el8.x86_64 119/122 2023-10-06T02:21:48.703 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 119/122 2023-10-06T02:21:48.908 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : flex-2.6.1-9.el8.x86_64 120/122 2023-10-06T02:21:48.933 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 120/122 2023-10-06T02:21:48.933 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-10-06T02:21:49.207 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-10-06T02:21:49.207 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 122/122 2023-10-06T02:21:49.624 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 122/122 2023-10-06T02:21:49.624 INFO:teuthology.orchestra.run.smithi031.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-10-06T02:21:49.624 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:21:50.361 INFO:teuthology.orchestra.run.smithi062.stdout:Installed products updated. 2023-10-06T02:21:50.497 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-10-06T02:21:50.498 INFO:teuthology.orchestra.run.smithi062.stdout:Upgraded: 2023-10-06T02:21:50.498 INFO:teuthology.orchestra.run.smithi062.stdout: librados2-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T02:21:50.498 INFO:teuthology.orchestra.run.smithi062.stdout: librbd1-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T02:21:50.498 INFO:teuthology.orchestra.run.smithi062.stdout:Installed: 2023-10-06T02:21:50.498 INFO:teuthology.orchestra.run.smithi062.stdout: automake-1.16.1-7.el8.noarch 2023-10-06T02:21:50.498 INFO:teuthology.orchestra.run.smithi062.stdout: bison-3.0.4-10.el8.x86_64 2023-10-06T02:21:50.498 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T02:21:50.498 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-base-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T02:21:50.498 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-common-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T02:21:50.498 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-fuse-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T02:21:50.499 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-grafana-dashboards-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T02:21:50.499 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-immutable-object-cache-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T02:21:50.499 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mds-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T02:21:50.499 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T02:21:50.499 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-cephadm-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T02:21:50.499 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-dashboard-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T02:21:50.499 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-diskprediction-local-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T02:21:50.499 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-modules-core-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T02:21:50.499 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-rook-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T02:21:50.499 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mon-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T02:21:50.499 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-osd-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T02:21:50.499 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-prometheus-alerts-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T02:21:50.500 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-radosgw-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T02:21:50.500 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-selinux-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T02:21:50.500 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-test-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T02:21:50.500 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-volume-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T02:21:50.500 INFO:teuthology.orchestra.run.smithi062.stdout: cephadm-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T02:21:50.500 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-mirror-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T02:21:50.500 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-top-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T02:21:50.501 INFO:teuthology.orchestra.run.smithi062.stdout: elfutils-libelf-devel-0.182-3.el8.x86_64 2023-10-06T02:21:50.501 INFO:teuthology.orchestra.run.smithi062.stdout: flex-2.6.1-9.el8.x86_64 2023-10-06T02:21:50.501 INFO:teuthology.orchestra.run.smithi062.stdout: fuse-2.9.7-12.el8.x86_64 2023-10-06T02:21:50.501 INFO:teuthology.orchestra.run.smithi062.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-10-06T02:21:50.501 INFO:teuthology.orchestra.run.smithi062.stdout: jq-1.5-12.el8.x86_64 2023-10-06T02:21:50.501 INFO:teuthology.orchestra.run.smithi062.stdout: keyutils-libs-devel-1.5.10-6.el8.x86_64 2023-10-06T02:21:50.501 INFO:teuthology.orchestra.run.smithi062.stdout: krb5-devel-1.18.2-8.3.el8_4.x86_64 2023-10-06T02:21:50.501 INFO:teuthology.orchestra.run.smithi062.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-10-06T02:21:50.501 INFO:teuthology.orchestra.run.smithi062.stdout: libcephfs-devel-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T02:21:50.501 INFO:teuthology.orchestra.run.smithi062.stdout: libcephfs2-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T02:21:50.501 INFO:teuthology.orchestra.run.smithi062.stdout: libcephsqlite-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T02:21:50.501 INFO:teuthology.orchestra.run.smithi062.stdout: libcom_err-devel-1.45.6-1.el8.x86_64 2023-10-06T02:21:50.501 INFO:teuthology.orchestra.run.smithi062.stdout: libgfortran-8.4.1-1.el8.x86_64 2023-10-06T02:21:50.502 INFO:teuthology.orchestra.run.smithi062.stdout: liboath-2.6.2-3.el8.x86_64 2023-10-06T02:21:50.502 INFO:teuthology.orchestra.run.smithi062.stdout: libquadmath-8.4.1-1.el8.x86_64 2023-10-06T02:21:50.502 INFO:teuthology.orchestra.run.smithi062.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-10-06T02:21:50.502 INFO:teuthology.orchestra.run.smithi062.stdout: librados-devel-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T02:21:50.502 INFO:teuthology.orchestra.run.smithi062.stdout: libradosstriper1-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T02:21:50.502 INFO:teuthology.orchestra.run.smithi062.stdout: librdkafka-0.11.4-1.el8.x86_64 2023-10-06T02:21:50.502 INFO:teuthology.orchestra.run.smithi062.stdout: librgw2-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T02:21:50.503 INFO:teuthology.orchestra.run.smithi062.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-10-06T02:21:50.503 INFO:teuthology.orchestra.run.smithi062.stdout: libsepol-devel-2.9-2.el8.x86_64 2023-10-06T02:21:50.503 INFO:teuthology.orchestra.run.smithi062.stdout: libtool-2.4.6-25.el8.x86_64 2023-10-06T02:21:50.503 INFO:teuthology.orchestra.run.smithi062.stdout: libunwind-1.3.1-3.el8.x86_64 2023-10-06T02:21:50.503 INFO:teuthology.orchestra.run.smithi062.stdout: libuuid-devel-2.32.1-27.el8.x86_64 2023-10-06T02:21:50.503 INFO:teuthology.orchestra.run.smithi062.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-10-06T02:21:50.503 INFO:teuthology.orchestra.run.smithi062.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-10-06T02:21:50.503 INFO:teuthology.orchestra.run.smithi062.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-10-06T02:21:50.503 INFO:teuthology.orchestra.run.smithi062.stdout: openblas-0.3.12-1.el8.x86_64 2023-10-06T02:21:50.503 INFO:teuthology.orchestra.run.smithi062.stdout: openblas-threads-0.3.12-1.el8.x86_64 2023-10-06T02:21:50.503 INFO:teuthology.orchestra.run.smithi062.stdout: openssl-devel-1:1.1.1g-15.el8_3.x86_64 2023-10-06T02:21:50.503 INFO:teuthology.orchestra.run.smithi062.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-10-06T02:21:50.503 INFO:teuthology.orchestra.run.smithi062.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-10-06T02:21:50.504 INFO:teuthology.orchestra.run.smithi062.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-10-06T02:21:50.504 INFO:teuthology.orchestra.run.smithi062.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-10-06T02:21:50.504 INFO:teuthology.orchestra.run.smithi062.stdout: python3-babel-2.5.1-5.el8.noarch 2023-10-06T02:21:50.504 INFO:teuthology.orchestra.run.smithi062.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-10-06T02:21:50.504 INFO:teuthology.orchestra.run.smithi062.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-10-06T02:21:50.504 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-10-06T02:21:50.504 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ceph-argparse-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T02:21:50.505 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ceph-common-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T02:21:50.505 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cephfs-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T02:21:50.505 INFO:teuthology.orchestra.run.smithi062.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-10-06T02:21:50.505 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-10-06T02:21:50.505 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-10-06T02:21:50.505 INFO:teuthology.orchestra.run.smithi062.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-10-06T02:21:50.505 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-6.2-6.el8.noarch 2023-10-06T02:21:50.505 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-10-06T02:21:50.505 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jinja2-2.10.1-2.el8_0.noarch 2023-10-06T02:21:50.506 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-10-06T02:21:50.506 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-10-06T02:21:50.506 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-10-06T02:21:50.506 INFO:teuthology.orchestra.run.smithi062.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-10-06T02:21:50.506 INFO:teuthology.orchestra.run.smithi062.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-10-06T02:21:50.506 INFO:teuthology.orchestra.run.smithi062.stdout: python3-mako-1.0.6-13.el8.noarch 2023-10-06T02:21:50.506 INFO:teuthology.orchestra.run.smithi062.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-10-06T02:21:50.507 INFO:teuthology.orchestra.run.smithi062.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-10-06T02:21:50.507 INFO:teuthology.orchestra.run.smithi062.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-10-06T02:21:50.507 INFO:teuthology.orchestra.run.smithi062.stdout: python3-numpy-1:1.14.3-9.el8.x86_64 2023-10-06T02:21:50.507 INFO:teuthology.orchestra.run.smithi062.stdout: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 2023-10-06T02:21:50.507 INFO:teuthology.orchestra.run.smithi062.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-10-06T02:21:50.507 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-10-06T02:21:50.507 INFO:teuthology.orchestra.run.smithi062.stdout: python3-portend-2.6-1.el8.noarch 2023-10-06T02:21:50.507 INFO:teuthology.orchestra.run.smithi062.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-10-06T02:21:50.507 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-10-06T02:21:50.507 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-10-06T02:21:50.507 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pytz-2017.2-9.el8.noarch 2023-10-06T02:21:50.508 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rados-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T02:21:50.508 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rbd-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T02:21:50.508 INFO:teuthology.orchestra.run.smithi062.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-10-06T02:21:50.508 INFO:teuthology.orchestra.run.smithi062.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-10-06T02:21:50.508 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rgw-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T02:21:50.508 INFO:teuthology.orchestra.run.smithi062.stdout: python3-routes-2.4.1-12.el8.noarch 2023-10-06T02:21:50.508 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rsa-4.9-2.el8.noarch 2023-10-06T02:21:50.508 INFO:teuthology.orchestra.run.smithi062.stdout: python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7.x86_64 2023-10-06T02:21:50.509 INFO:teuthology.orchestra.run.smithi062.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-10-06T02:21:50.509 INFO:teuthology.orchestra.run.smithi062.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-10-06T02:21:50.509 INFO:teuthology.orchestra.run.smithi062.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-10-06T02:21:50.509 INFO:teuthology.orchestra.run.smithi062.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-10-06T02:21:50.509 INFO:teuthology.orchestra.run.smithi062.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-10-06T02:21:50.509 INFO:teuthology.orchestra.run.smithi062.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-10-06T02:21:50.509 INFO:teuthology.orchestra.run.smithi062.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-10-06T02:21:50.509 INFO:teuthology.orchestra.run.smithi062.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-10-06T02:21:50.510 INFO:teuthology.orchestra.run.smithi062.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-10-06T02:21:50.510 INFO:teuthology.orchestra.run.smithi062.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-10-06T02:21:50.510 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-fuse-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T02:21:50.510 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-mirror-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T02:21:50.510 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-nbd-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T02:21:50.510 INFO:teuthology.orchestra.run.smithi062.stdout: socat-1.7.3.3-2.el8.x86_64 2023-10-06T02:21:50.510 INFO:teuthology.orchestra.run.smithi062.stdout: thrift-0.13.0-2.el8.x86_64 2023-10-06T02:21:50.510 INFO:teuthology.orchestra.run.smithi062.stdout: xfsprogs-devel-5.0.0-8.el8.x86_64 2023-10-06T02:21:50.510 INFO:teuthology.orchestra.run.smithi062.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-10-06T02:21:50.511 INFO:teuthology.orchestra.run.smithi062.stdout: zlib-devel-1.2.11-17.el8.x86_64 2023-10-06T02:21:50.511 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-10-06T02:21:50.511 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2023-10-06T02:21:50.532 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-2:17.2.6-1296.g736ac5ba.el8.x86_64 1/122 2023-10-06T02:21:50.532 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-base-2:17.2.6-1296.g736ac5ba.el8.x86_64 2/122 2023-10-06T02:21:50.533 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-common-2:17.2.6-1296.g736ac5ba.el8.x86_64 3/122 2023-10-06T02:21:50.533 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-fuse-2:17.2.6-1296.g736ac5ba.el8.x86_64 4/122 2023-10-06T02:21:50.533 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-immutable-object-cache-2:17.2.6-1296.g736ac 5/122 2023-10-06T02:21:50.533 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mds-2:17.2.6-1296.g736ac5ba.el8.x86_64 6/122 2023-10-06T02:21:50.533 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-2:17.2.6-1296.g736ac5ba.el8.x86_64 7/122 2023-10-06T02:21:50.533 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mon-2:17.2.6-1296.g736ac5ba.el8.x86_64 8/122 2023-10-06T02:21:50.533 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-osd-2:17.2.6-1296.g736ac5ba.el8.x86_64 9/122 2023-10-06T02:21:50.533 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-radosgw-2:17.2.6-1296.g736ac5ba.el8.x86_64 10/122 2023-10-06T02:21:50.533 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-selinux-2:17.2.6-1296.g736ac5ba.el8.x86_64 11/122 2023-10-06T02:21:50.533 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-test-2:17.2.6-1296.g736ac5ba.el8.x86_64 12/122 2023-10-06T02:21:50.533 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : cephfs-mirror-2:17.2.6-1296.g736ac5ba.el8.x86_64 13/122 2023-10-06T02:21:50.533 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libcephfs-devel-2:17.2.6-1296.g736ac5ba.el8.x86_ 14/122 2023-10-06T02:21:50.534 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libcephfs2-2:17.2.6-1296.g736ac5ba.el8.x86_64 15/122 2023-10-06T02:21:50.534 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libcephsqlite-2:17.2.6-1296.g736ac5ba.el8.x86_64 16/122 2023-10-06T02:21:50.534 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librados-devel-2:17.2.6-1296.g736ac5ba.el8.x86_6 17/122 2023-10-06T02:21:50.534 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libradosstriper1-2:17.2.6-1296.g736ac5ba.el8.x86 18/122 2023-10-06T02:21:50.534 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librgw2-2:17.2.6-1296.g736ac5ba.el8.x86_64 19/122 2023-10-06T02:21:50.534 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-ceph-argparse-2:17.2.6-1296.g736ac5ba.el 20/122 2023-10-06T02:21:50.534 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-ceph-common-2:17.2.6-1296.g736ac5ba.el8. 21/122 2023-10-06T02:21:50.534 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cephfs-2:17.2.6-1296.g736ac5ba.el8.x86_6 22/122 2023-10-06T02:21:50.534 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rados-2:17.2.6-1296.g736ac5ba.el8.x86_64 23/122 2023-10-06T02:21:50.534 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rbd-2:17.2.6-1296.g736ac5ba.el8.x86_64 24/122 2023-10-06T02:21:50.534 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rgw-2:17.2.6-1296.g736ac5ba.el8.x86_64 25/122 2023-10-06T02:21:50.535 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : rbd-fuse-2:17.2.6-1296.g736ac5ba.el8.x86_64 26/122 2023-10-06T02:21:50.535 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : rbd-mirror-2:17.2.6-1296.g736ac5ba.el8.x86_64 27/122 2023-10-06T02:21:50.535 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : rbd-nbd-2:17.2.6-1296.g736ac5ba.el8.x86_64 28/122 2023-10-06T02:21:50.535 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-grafana-dashboards-2:17.2.6-1296.g736ac5ba. 29/122 2023-10-06T02:21:50.535 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-cephadm-2:17.2.6-1296.g736ac5ba.el8.noa 30/122 2023-10-06T02:21:50.535 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-dashboard-2:17.2.6-1296.g736ac5ba.el8.n 31/122 2023-10-06T02:21:50.535 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.6-1296.g736 32/122 2023-10-06T02:21:50.535 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-modules-core-2:17.2.6-1296.g736ac5ba.el 33/122 2023-10-06T02:21:50.535 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-rook-2:17.2.6-1296.g736ac5ba.el8.noarch 34/122 2023-10-06T02:21:50.535 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-prometheus-alerts-2:17.2.6-1296.g736ac5ba.e 35/122 2023-10-06T02:21:50.535 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-volume-2:17.2.6-1296.g736ac5ba.el8.noarch 36/122 2023-10-06T02:21:50.535 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : cephadm-2:17.2.6-1296.g736ac5ba.el8.noarch 37/122 2023-10-06T02:21:50.535 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : cephfs-top-2:17.2.6-1296.g736ac5ba.el8.noarch 38/122 2023-10-06T02:21:50.536 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/122 2023-10-06T02:21:50.536 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/122 2023-10-06T02:21:50.536 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/122 2023-10-06T02:21:50.536 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/122 2023-10-06T02:21:50.536 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/122 2023-10-06T02:21:50.536 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/122 2023-10-06T02:21:50.536 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/122 2023-10-06T02:21:50.536 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/122 2023-10-06T02:21:50.536 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/122 2023-10-06T02:21:50.536 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/122 2023-10-06T02:21:50.536 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/122 2023-10-06T02:21:50.536 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/122 2023-10-06T02:21:50.537 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/122 2023-10-06T02:21:50.537 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/122 2023-10-06T02:21:50.537 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/122 2023-10-06T02:21:50.537 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/122 2023-10-06T02:21:50.537 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/122 2023-10-06T02:21:50.537 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/122 2023-10-06T02:21:50.537 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/122 2023-10-06T02:21:50.537 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/122 2023-10-06T02:21:50.537 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/122 2023-10-06T02:21:50.537 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/122 2023-10-06T02:21:50.537 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/122 2023-10-06T02:21:50.537 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/122 2023-10-06T02:21:50.538 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/122 2023-10-06T02:21:50.538 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/122 2023-10-06T02:21:50.538 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/122 2023-10-06T02:21:50.538 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/122 2023-10-06T02:21:50.538 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/122 2023-10-06T02:21:50.538 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/122 2023-10-06T02:21:50.538 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/122 2023-10-06T02:21:50.538 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/122 2023-10-06T02:21:50.538 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/122 2023-10-06T02:21:50.538 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 72/122 2023-10-06T02:21:50.538 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 73/122 2023-10-06T02:21:50.539 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 74/122 2023-10-06T02:21:50.539 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : jq-1.5-12.el8.x86_64 75/122 2023-10-06T02:21:50.539 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : flex-2.6.1-9.el8.x86_64 76/122 2023-10-06T02:21:50.539 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 77/122 2023-10-06T02:21:50.539 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-babel-2.5.1-5.el8.noarch 78/122 2023-10-06T02:21:50.539 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 79/122 2023-10-06T02:21:50.539 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : bison-3.0.4-10.el8.x86_64 80/122 2023-10-06T02:21:50.539 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 81/122 2023-10-06T02:21:50.539 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 82/122 2023-10-06T02:21:50.539 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 83/122 2023-10-06T02:21:50.539 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-numpy-1:1.14.3-9.el8.x86_64 84/122 2023-10-06T02:21:50.539 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 85/122 2023-10-06T02:21:50.540 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jinja2-2.10.1-2.el8_0.noarch 86/122 2023-10-06T02:21:50.540 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : socat-1.7.3.3-2.el8.x86_64 87/122 2023-10-06T02:21:50.540 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 88/122 2023-10-06T02:21:50.540 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 89/122 2023-10-06T02:21:50.540 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librdkafka-0.11.4-1.el8.x86_64 90/122 2023-10-06T02:21:50.540 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 91/122 2023-10-06T02:21:50.540 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 92/122 2023-10-06T02:21:50.540 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 93/122 2023-10-06T02:21:50.540 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openblas-threads-0.3.12-1.el8.x86_64 94/122 2023-10-06T02:21:50.540 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openblas-0.3.12-1.el8.x86_64 95/122 2023-10-06T02:21:50.540 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : automake-1.16.1-7.el8.noarch 96/122 2023-10-06T02:21:50.540 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 97/122 2023-10-06T02:21:50.540 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 98/122 2023-10-06T02:21:50.541 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 99/122 2023-10-06T02:21:50.541 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 100/122 2023-10-06T02:21:50.541 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : keyutils-libs-devel-1.5.10-6.el8.x86_64 101/122 2023-10-06T02:21:50.541 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 102/122 2023-10-06T02:21:50.541 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 103/122 2023-10-06T02:21:50.541 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openssl-devel-1:1.1.1g-15.el8_3.x86_64 104/122 2023-10-06T02:21:50.541 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 105/122 2023-10-06T02:21:50.541 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 106/122 2023-10-06T02:21:50.541 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : fuse-2.9.7-12.el8.x86_64 107/122 2023-10-06T02:21:50.541 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : elfutils-libelf-devel-0.182-3.el8.x86_64 108/122 2023-10-06T02:21:50.541 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libquadmath-8.4.1-1.el8.x86_64 109/122 2023-10-06T02:21:50.541 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libcom_err-devel-1.45.6-1.el8.x86_64 110/122 2023-10-06T02:21:50.542 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libsepol-devel-2.9-2.el8.x86_64 111/122 2023-10-06T02:21:50.542 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : zlib-devel-1.2.11-17.el8.x86_64 112/122 2023-10-06T02:21:50.542 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 113/122 2023-10-06T02:21:50.542 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : krb5-devel-1.18.2-8.3.el8_4.x86_64 114/122 2023-10-06T02:21:50.542 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libgfortran-8.4.1-1.el8.x86_64 115/122 2023-10-06T02:21:50.542 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : xfsprogs-devel-5.0.0-8.el8.x86_64 116/122 2023-10-06T02:21:50.542 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 117/122 2023-10-06T02:21:50.542 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libuuid-devel-2.32.1-27.el8.x86_64 118/122 2023-10-06T02:21:50.542 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librados2-2:17.2.6-1296.g736ac5ba.el8.x86_64 119/122 2023-10-06T02:21:50.542 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 120/122 2023-10-06T02:21:50.542 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librbd1-2:17.2.6-1296.g736ac5ba.el8.x86_64 121/122 2023-10-06T02:21:50.885 DEBUG:teuthology.parallel:result is None 2023-10-06T02:21:52.965 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 122/122 2023-10-06T02:21:52.982 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:02:07 ago on Fri 06 Oct 2023 02:19:45 AM UTC. 2023-10-06T02:21:55.302 INFO:teuthology.orchestra.run.smithi031.stdout:Installed products updated. 2023-10-06T02:21:55.437 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:21:55.481 INFO:teuthology.orchestra.run.smithi031.stdout:Upgraded: 2023-10-06T02:21:55.481 INFO:teuthology.orchestra.run.smithi031.stdout: librados2-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T02:21:55.481 INFO:teuthology.orchestra.run.smithi031.stdout: librbd1-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T02:21:55.482 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2023-10-06T02:21:55.482 INFO:teuthology.orchestra.run.smithi031.stdout: automake-1.16.1-7.el8.noarch 2023-10-06T02:21:55.483 INFO:teuthology.orchestra.run.smithi031.stdout: bison-3.0.4-10.el8.x86_64 2023-10-06T02:21:55.483 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T02:21:55.483 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-base-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T02:21:55.484 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-common-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T02:21:55.484 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-fuse-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T02:21:55.484 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-grafana-dashboards-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T02:21:55.484 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-immutable-object-cache-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T02:21:55.484 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mds-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T02:21:55.485 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T02:21:55.485 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-cephadm-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T02:21:55.485 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-dashboard-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T02:21:55.485 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-diskprediction-local-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T02:21:55.485 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-modules-core-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T02:21:55.485 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-rook-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T02:21:55.485 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mon-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T02:21:55.485 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-osd-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T02:21:55.485 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-prometheus-alerts-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T02:21:55.485 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-radosgw-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T02:21:55.486 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-selinux-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T02:21:55.486 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-test-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T02:21:55.486 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-volume-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T02:21:55.486 INFO:teuthology.orchestra.run.smithi031.stdout: cephadm-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T02:21:55.486 INFO:teuthology.orchestra.run.smithi031.stdout: cephfs-mirror-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T02:21:55.486 INFO:teuthology.orchestra.run.smithi031.stdout: cephfs-top-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T02:21:55.486 INFO:teuthology.orchestra.run.smithi031.stdout: elfutils-libelf-devel-0.182-3.el8.x86_64 2023-10-06T02:21:55.486 INFO:teuthology.orchestra.run.smithi031.stdout: flex-2.6.1-9.el8.x86_64 2023-10-06T02:21:55.486 INFO:teuthology.orchestra.run.smithi031.stdout: fuse-2.9.7-12.el8.x86_64 2023-10-06T02:21:55.486 INFO:teuthology.orchestra.run.smithi031.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-10-06T02:21:55.486 INFO:teuthology.orchestra.run.smithi031.stdout: jq-1.5-12.el8.x86_64 2023-10-06T02:21:55.487 INFO:teuthology.orchestra.run.smithi031.stdout: keyutils-libs-devel-1.5.10-6.el8.x86_64 2023-10-06T02:21:55.487 INFO:teuthology.orchestra.run.smithi031.stdout: krb5-devel-1.18.2-8.3.el8_4.x86_64 2023-10-06T02:21:55.487 INFO:teuthology.orchestra.run.smithi031.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-10-06T02:21:55.487 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs-devel-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T02:21:55.487 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs2-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T02:21:55.487 INFO:teuthology.orchestra.run.smithi031.stdout: libcephsqlite-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T02:21:55.487 INFO:teuthology.orchestra.run.smithi031.stdout: libcom_err-devel-1.45.6-1.el8.x86_64 2023-10-06T02:21:55.487 INFO:teuthology.orchestra.run.smithi031.stdout: libgfortran-8.4.1-1.el8.x86_64 2023-10-06T02:21:55.487 INFO:teuthology.orchestra.run.smithi031.stdout: liboath-2.6.2-3.el8.x86_64 2023-10-06T02:21:55.487 INFO:teuthology.orchestra.run.smithi031.stdout: libquadmath-8.4.1-1.el8.x86_64 2023-10-06T02:21:55.487 INFO:teuthology.orchestra.run.smithi031.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-10-06T02:21:55.487 INFO:teuthology.orchestra.run.smithi031.stdout: librados-devel-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T02:21:55.488 INFO:teuthology.orchestra.run.smithi031.stdout: libradosstriper1-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T02:21:55.488 INFO:teuthology.orchestra.run.smithi031.stdout: librdkafka-0.11.4-1.el8.x86_64 2023-10-06T02:21:55.488 INFO:teuthology.orchestra.run.smithi031.stdout: librgw2-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T02:21:55.488 INFO:teuthology.orchestra.run.smithi031.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-10-06T02:21:55.488 INFO:teuthology.orchestra.run.smithi031.stdout: libsepol-devel-2.9-2.el8.x86_64 2023-10-06T02:21:55.488 INFO:teuthology.orchestra.run.smithi031.stdout: libtool-2.4.6-25.el8.x86_64 2023-10-06T02:21:55.488 INFO:teuthology.orchestra.run.smithi031.stdout: libunwind-1.3.1-3.el8.x86_64 2023-10-06T02:21:55.488 INFO:teuthology.orchestra.run.smithi031.stdout: libuuid-devel-2.32.1-27.el8.x86_64 2023-10-06T02:21:55.488 INFO:teuthology.orchestra.run.smithi031.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-10-06T02:21:55.488 INFO:teuthology.orchestra.run.smithi031.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-10-06T02:21:55.488 INFO:teuthology.orchestra.run.smithi031.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-10-06T02:21:55.489 INFO:teuthology.orchestra.run.smithi031.stdout: openblas-0.3.12-1.el8.x86_64 2023-10-06T02:21:55.489 INFO:teuthology.orchestra.run.smithi031.stdout: openblas-threads-0.3.12-1.el8.x86_64 2023-10-06T02:21:55.489 INFO:teuthology.orchestra.run.smithi031.stdout: openssl-devel-1:1.1.1g-15.el8_3.x86_64 2023-10-06T02:21:55.489 INFO:teuthology.orchestra.run.smithi031.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-10-06T02:21:55.489 INFO:teuthology.orchestra.run.smithi031.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-10-06T02:21:55.489 INFO:teuthology.orchestra.run.smithi031.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-10-06T02:21:55.489 INFO:teuthology.orchestra.run.smithi031.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-10-06T02:21:55.489 INFO:teuthology.orchestra.run.smithi031.stdout: python3-babel-2.5.1-5.el8.noarch 2023-10-06T02:21:55.489 INFO:teuthology.orchestra.run.smithi031.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-10-06T02:21:55.489 INFO:teuthology.orchestra.run.smithi031.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-10-06T02:21:55.489 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-10-06T02:21:55.489 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-argparse-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T02:21:55.489 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-common-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T02:21:55.490 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cephfs-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T02:21:55.490 INFO:teuthology.orchestra.run.smithi031.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-10-06T02:21:55.490 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-10-06T02:21:55.490 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-10-06T02:21:55.490 INFO:teuthology.orchestra.run.smithi031.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-10-06T02:21:55.490 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-6.2-6.el8.noarch 2023-10-06T02:21:55.490 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-10-06T02:21:55.490 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jinja2-2.10.1-2.el8_0.noarch 2023-10-06T02:21:55.490 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-10-06T02:21:55.490 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-10-06T02:21:55.490 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-10-06T02:21:55.491 INFO:teuthology.orchestra.run.smithi031.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-10-06T02:21:55.491 INFO:teuthology.orchestra.run.smithi031.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-10-06T02:21:55.491 INFO:teuthology.orchestra.run.smithi031.stdout: python3-mako-1.0.6-13.el8.noarch 2023-10-06T02:21:55.491 INFO:teuthology.orchestra.run.smithi031.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-10-06T02:21:55.491 INFO:teuthology.orchestra.run.smithi031.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-10-06T02:21:55.491 INFO:teuthology.orchestra.run.smithi031.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-10-06T02:21:55.491 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy-1:1.14.3-9.el8.x86_64 2023-10-06T02:21:55.491 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 2023-10-06T02:21:55.491 INFO:teuthology.orchestra.run.smithi031.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-10-06T02:21:55.491 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-10-06T02:21:55.491 INFO:teuthology.orchestra.run.smithi031.stdout: python3-portend-2.6-1.el8.noarch 2023-10-06T02:21:55.491 INFO:teuthology.orchestra.run.smithi031.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-10-06T02:21:55.492 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-10-06T02:21:55.492 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-10-06T02:21:55.492 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pytz-2017.2-9.el8.noarch 2023-10-06T02:21:55.492 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rados-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T02:21:55.492 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rbd-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T02:21:55.492 INFO:teuthology.orchestra.run.smithi031.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-10-06T02:21:55.492 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-10-06T02:21:55.492 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rgw-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T02:21:55.492 INFO:teuthology.orchestra.run.smithi031.stdout: python3-routes-2.4.1-12.el8.noarch 2023-10-06T02:21:55.493 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rsa-4.9-2.el8.noarch 2023-10-06T02:21:55.493 INFO:teuthology.orchestra.run.smithi031.stdout: python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7.x86_64 2023-10-06T02:21:55.493 INFO:teuthology.orchestra.run.smithi031.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-10-06T02:21:55.493 INFO:teuthology.orchestra.run.smithi031.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-10-06T02:21:55.493 INFO:teuthology.orchestra.run.smithi031.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-10-06T02:21:55.493 INFO:teuthology.orchestra.run.smithi031.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-10-06T02:21:55.493 INFO:teuthology.orchestra.run.smithi031.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-10-06T02:21:55.493 INFO:teuthology.orchestra.run.smithi031.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-10-06T02:21:55.493 INFO:teuthology.orchestra.run.smithi031.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-10-06T02:21:55.493 INFO:teuthology.orchestra.run.smithi031.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-10-06T02:21:55.493 INFO:teuthology.orchestra.run.smithi031.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-10-06T02:21:55.493 INFO:teuthology.orchestra.run.smithi031.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-10-06T02:21:55.494 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-fuse-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T02:21:55.494 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-mirror-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T02:21:55.494 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-nbd-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T02:21:55.494 INFO:teuthology.orchestra.run.smithi031.stdout: socat-1.7.3.3-2.el8.x86_64 2023-10-06T02:21:55.494 INFO:teuthology.orchestra.run.smithi031.stdout: thrift-0.13.0-2.el8.x86_64 2023-10-06T02:21:55.494 INFO:teuthology.orchestra.run.smithi031.stdout: xfsprogs-devel-5.0.0-8.el8.x86_64 2023-10-06T02:21:55.494 INFO:teuthology.orchestra.run.smithi031.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-10-06T02:21:55.495 INFO:teuthology.orchestra.run.smithi031.stdout: zlib-devel-1.2.11-17.el8.x86_64 2023-10-06T02:21:55.495 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:21:55.495 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2023-10-06T02:21:55.795 DEBUG:teuthology.parallel:result is None 2023-10-06T02:21:55.795 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=736ac5ba72cb47af7d6f6666b1e100f7e20a20e2 2023-10-06T02:21:55.939 DEBUG:teuthology.orchestra.run.smithi031:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-06T02:21:55.992 INFO:teuthology.orchestra.run.smithi031.stdout:17.2.6-1296.g736ac5ba.el8 2023-10-06T02:21:56.006 INFO:teuthology.packaging:The installed version of ceph is 17.2.6-1296.g736ac5ba.el8 2023-10-06T02:21:56.006 INFO:teuthology.task.install:The correct ceph version 17.2.6-1296.g736ac5ba is installed. 2023-10-06T02:21:56.006 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=736ac5ba72cb47af7d6f6666b1e100f7e20a20e2 2023-10-06T02:21:56.232 DEBUG:teuthology.orchestra.run.smithi062:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-06T02:21:56.284 INFO:teuthology.orchestra.run.smithi062.stdout:17.2.6-1296.g736ac5ba.el8 2023-10-06T02:21:56.285 INFO:teuthology.packaging:The installed version of ceph is 17.2.6-1296.g736ac5ba.el8 2023-10-06T02:21:56.285 INFO:teuthology.task.install:The correct ceph version 17.2.6-1296.g736ac5ba is installed. 2023-10-06T02:21:56.285 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-10-06T02:21:56.285 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T02:21:56.286 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-06T02:21:56.376 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-10-06T02:21:56.376 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-06T02:21:56.463 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-10-06T02:21:56.463 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T02:21:56.463 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/usr/bin/daemon-helper 2023-10-06T02:21:56.526 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-06T02:21:56.630 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-10-06T02:21:56.631 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/usr/bin/daemon-helper 2023-10-06T02:21:56.695 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-06T02:21:56.796 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-10-06T02:21:56.796 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T02:21:56.796 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-06T02:21:56.862 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-06T02:21:56.967 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-10-06T02:21:56.967 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-06T02:21:57.029 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-06T02:21:57.130 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-10-06T02:21:57.130 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T02:21:57.131 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/usr/bin/stdin-killer 2023-10-06T02:21:57.192 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-06T02:21:57.293 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-10-06T02:21:57.293 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/usr/bin/stdin-killer 2023-10-06T02:21:57.355 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-06T02:21:57.456 INFO:teuthology.run_tasks:Running task ceph... 2023-10-06T02:21:57.528 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-10-06T02:21:57.529 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 777 /var/log/ceph 2023-10-06T02:21:57.533 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 777 /var/log/ceph 2023-10-06T02:21:57.594 INFO:tasks.ceph:Disabling ceph logrotate... 2023-10-06T02:21:57.594 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-06T02:21:57.598 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-06T02:21:57.658 INFO:tasks.ceph:Creating extra log directories... 2023-10-06T02:21:57.659 DEBUG:teuthology.orchestra.run.smithi031:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-06T02:21:57.663 DEBUG:teuthology.orchestra.run.smithi062:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-06T02:21:57.735 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-10-06T02:21:57.736 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-10-06T02:21:57.736 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-10-06T02:21:57.736 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T02:21:57.736 DEBUG:teuthology.orchestra.run.smithi031:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-06T02:21:57.772 DEBUG:teuthology.orchestra.run.smithi031:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-06T02:21:58.675 DEBUG:teuthology.orchestra.run.smithi031:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-10-06T02:21:59.363 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-10-06T02:21:59.363 DEBUG:teuthology.orchestra.run.smithi062:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-06T02:21:59.404 DEBUG:teuthology.orchestra.run.smithi062:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-06T02:21:59.610 DEBUG:teuthology.orchestra.run.smithi062:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-10-06T02:21:59.830 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-10-06T02:21:59.830 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-10-06T02:21:59.830 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412129', 'branch': 'wip-yuri-testing-2023-10-05-0730-quincy', 'description': 'fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/misc}}', 'email': 'yweinste@redhat.com', 'first_in_suite': False, 'job_id': '7412129', 'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'rhel', 'os_version': '8.4', 'overrides': {'admin_socket': {'branch': 'wip-yuri-testing-2023-10-05-0730-quincy'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '736ac5ba72cb47af7d6f6666b1e100f7e20a20e2'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '736ac5ba72cb47af7d6f6666b1e100f7e20a20e2'}}, 'kclient': {'syntax': 'v2'}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-yuri-testing-2023-10-05-0730-quincy', 'sha1': 'f1e315a9732d36c05ec316678e2397dc58561c7f'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 75, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 7519, 'sha1': '736ac5ba72cb47af7d6f6666b1e100f7e20a20e2', 'sleep_before_teardown': 0, 'subset': '111/12000', 'suite': 'fs', 'suite_branch': 'wip-yuri-testing-2023-10-05-0730-quincy', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_f1e315a9732d36c05ec316678e2397dc58561c7f/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': 'f1e315a9732d36c05ec316678e2397dc58561c7f', 'targets': {'smithi031.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLZXMnoxnVb2uZqUBuqUws24Kw6G5H9Ueh504kqdn3nWt5HsTr4MrTt+YmJAfIwE8R4h+T5Wag7WSIB7NhjqUsQ=', 'smithi062.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJ9YBgmw9WCA672tKizZk9aWL2VzsFwWWtEBkVvpPLrsDmh+ytXxJq/7oZwtwZzaHXQZNuxRpkPQY6vUM4uonkY='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '736ac5ba72cb47af7d6f6666b1e100f7e20a20e2'}}, {'ceph': None}, {'kclient': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_volumes.TestMisc']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '54e62bcbac4e53d9685e08328b790d3b20d71cae', 'timestamp': '2023-10-05_23:52:50', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2830'} 2023-10-06T02:21:59.831 DEBUG:teuthology.orchestra.run.smithi031:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-06T02:21:59.835 DEBUG:teuthology.orchestra.run.smithi062:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-06T02:21:59.872 DEBUG:teuthology.orchestra.run.smithi031:> sudo install -d -m0777 -- /var/run/ceph 2023-10-06T02:21:59.916 DEBUG:teuthology.orchestra.run.smithi062:> sudo install -d -m0777 -- /var/run/ceph 2023-10-06T02:21:59.978 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T02:21:59.978 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/scratch_devs of=/dev/stdout 2023-10-06T02:22:00.016 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-06T02:22:00.016 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_1 2023-10-06T02:22:00.093 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-06T02:22:00.094 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T02:22:00.094 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 452 Links: 1 2023-10-06T02:22:00.094 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T02:22:00.094 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T02:22:00.094 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2023-10-06 02:21:49.952781257 +0000 2023-10-06T02:22:00.094 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2023-10-06 02:18:47.222207625 +0000 2023-10-06T02:22:00.094 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2023-10-06 02:18:47.222207625 +0000 2023-10-06T02:22:00.095 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: - 2023-10-06T02:22:00.095 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-06T02:22:00.194 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2023-10-06T02:22:00.194 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2023-10-06T02:22:00.194 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000391367 s, 1.3 MB/s 2023-10-06T02:22:00.197 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-06T02:22:00.235 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_2 2023-10-06T02:22:00.312 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-06T02:22:00.312 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T02:22:00.312 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 458 Links: 1 2023-10-06T02:22:00.312 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T02:22:00.312 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T02:22:00.312 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2023-10-06 02:21:49.952781257 +0000 2023-10-06T02:22:00.312 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2023-10-06 02:18:47.998196022 +0000 2023-10-06T02:22:00.312 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2023-10-06 02:18:47.998196022 +0000 2023-10-06T02:22:00.312 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: - 2023-10-06T02:22:00.313 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-06T02:22:00.410 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2023-10-06T02:22:00.410 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2023-10-06T02:22:00.410 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000284002 s, 1.8 MB/s 2023-10-06T02:22:00.413 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-06T02:22:00.450 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_3 2023-10-06T02:22:00.528 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-06T02:22:00.528 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T02:22:00.528 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 464 Links: 1 2023-10-06T02:22:00.528 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T02:22:00.528 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T02:22:00.528 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2023-10-06 02:21:49.952781257 +0000 2023-10-06T02:22:00.528 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2023-10-06 02:18:48.789184195 +0000 2023-10-06T02:22:00.528 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2023-10-06 02:18:48.789184195 +0000 2023-10-06T02:22:00.529 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: - 2023-10-06T02:22:00.529 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-06T02:22:00.626 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2023-10-06T02:22:00.626 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2023-10-06T02:22:00.626 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000318174 s, 1.6 MB/s 2023-10-06T02:22:00.629 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-06T02:22:00.667 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_4 2023-10-06T02:22:00.746 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-06T02:22:00.746 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T02:22:00.746 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 470 Links: 1 2023-10-06T02:22:00.746 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T02:22:00.746 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T02:22:00.746 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2023-10-06 02:21:49.953781242 +0000 2023-10-06T02:22:00.746 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2023-10-06 02:18:49.581172352 +0000 2023-10-06T02:22:00.746 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2023-10-06 02:18:49.581172352 +0000 2023-10-06T02:22:00.746 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: - 2023-10-06T02:22:00.747 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-06T02:22:00.844 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2023-10-06T02:22:00.895 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2023-10-06T02:22:00.895 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000285498 s, 1.8 MB/s 2023-10-06T02:22:00.896 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-06T02:22:00.934 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-06T02:22:00.934 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-10-06T02:22:00.934 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/scratch_devs of=/dev/stdout 2023-10-06T02:22:00.974 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-06T02:22:00.998 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_1 2023-10-06T02:22:01.054 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-06T02:22:01.054 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T02:22:01.054 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 5h/5d Inode: 452 Links: 1 2023-10-06T02:22:01.054 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T02:22:01.055 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T02:22:01.055 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2023-10-06 02:21:44.814430051 +0000 2023-10-06T02:22:01.055 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2023-10-06 02:18:48.967033831 +0000 2023-10-06T02:22:01.055 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2023-10-06 02:18:48.967033831 +0000 2023-10-06T02:22:01.055 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: - 2023-10-06T02:22:01.055 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-06T02:22:01.156 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2023-10-06T02:22:01.156 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2023-10-06T02:22:01.156 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.000373086 s, 1.4 MB/s 2023-10-06T02:22:01.158 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-06T02:22:01.198 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_2 2023-10-06T02:22:01.275 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-06T02:22:01.275 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T02:22:01.275 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 5h/5d Inode: 458 Links: 1 2023-10-06T02:22:01.275 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T02:22:01.275 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T02:22:01.275 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2023-10-06 02:21:44.814430051 +0000 2023-10-06T02:22:01.276 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2023-10-06 02:18:49.760022060 +0000 2023-10-06T02:22:01.276 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2023-10-06 02:18:49.760022060 +0000 2023-10-06T02:22:01.276 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: - 2023-10-06T02:22:01.276 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-06T02:22:01.374 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2023-10-06T02:22:01.374 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2023-10-06T02:22:01.375 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.000296666 s, 1.7 MB/s 2023-10-06T02:22:01.377 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-06T02:22:01.415 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_3 2023-10-06T02:22:01.491 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-06T02:22:01.492 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T02:22:01.492 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 5h/5d Inode: 464 Links: 1 2023-10-06T02:22:01.492 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T02:22:01.492 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T02:22:01.492 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2023-10-06 02:21:44.814430051 +0000 2023-10-06T02:22:01.492 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2023-10-06 02:18:50.542010453 +0000 2023-10-06T02:22:01.492 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2023-10-06 02:18:50.542010453 +0000 2023-10-06T02:22:01.492 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: - 2023-10-06T02:22:01.492 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-06T02:22:01.590 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2023-10-06T02:22:01.590 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2023-10-06T02:22:01.590 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.000302716 s, 1.7 MB/s 2023-10-06T02:22:01.592 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-06T02:22:01.630 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_4 2023-10-06T02:22:01.707 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-06T02:22:01.708 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T02:22:01.708 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 5h/5d Inode: 470 Links: 1 2023-10-06T02:22:01.708 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T02:22:01.708 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T02:22:01.708 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2023-10-06 02:21:44.815430035 +0000 2023-10-06T02:22:01.708 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2023-10-06 02:18:51.337998637 +0000 2023-10-06T02:22:01.708 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2023-10-06 02:18:51.337998637 +0000 2023-10-06T02:22:01.708 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: - 2023-10-06T02:22:01.708 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-06T02:22:01.806 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2023-10-06T02:22:01.807 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2023-10-06T02:22:01.807 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.000538264 s, 951 kB/s 2023-10-06T02:22:01.810 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-06T02:22:01.848 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-06T02:22:01.848 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi031.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi062.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-06T02:22:01.848 INFO:tasks.ceph:Generating config... 2023-10-06T02:22:01.849 INFO:tasks.ceph:[client] client mount timeout = 600 2023-10-06T02:22:01.849 INFO:tasks.ceph:[client] debug client = 20 2023-10-06T02:22:01.850 INFO:tasks.ceph:[client] debug ms = 1 2023-10-06T02:22:01.850 INFO:tasks.ceph:[client] pid file = 2023-10-06T02:22:01.850 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-10-06T02:22:01.850 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-10-06T02:22:01.850 INFO:tasks.ceph:[global] ms die on skipped message = False 2023-10-06T02:22:01.851 INFO:tasks.ceph:[mds] debug mds = 20 2023-10-06T02:22:01.851 INFO:tasks.ceph:[mds] debug ms = 1 2023-10-06T02:22:01.851 INFO:tasks.ceph:[mds] mds debug frag = True 2023-10-06T02:22:01.851 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-10-06T02:22:01.851 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-10-06T02:22:01.851 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-10-06T02:22:01.851 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-10-06T02:22:01.851 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-10-06T02:22:01.851 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-10-06T02:22:01.852 INFO:tasks.ceph:[mgr] debug client = 20 2023-10-06T02:22:01.852 INFO:tasks.ceph:[mgr] debug finisher = 20 2023-10-06T02:22:01.852 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-10-06T02:22:01.852 INFO:tasks.ceph:[mgr] debug ms = 1 2023-10-06T02:22:01.852 INFO:tasks.ceph:[mon] debug mon = 20 2023-10-06T02:22:01.853 INFO:tasks.ceph:[mon] debug ms = 1 2023-10-06T02:22:01.853 INFO:tasks.ceph:[mon] debug paxos = 20 2023-10-06T02:22:01.853 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-10-06T02:22:01.853 INFO:tasks.ceph:[osd] bdev async discard = True 2023-10-06T02:22:01.853 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-10-06T02:22:01.853 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-10-06T02:22:01.853 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-10-06T02:22:01.853 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-10-06T02:22:01.853 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-10-06T02:22:01.853 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-10-06T02:22:01.854 INFO:tasks.ceph:[osd] debug ms = 1 2023-10-06T02:22:01.854 INFO:tasks.ceph:[osd] debug osd = 20 2023-10-06T02:22:01.854 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-10-06T02:22:01.854 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-10-06T02:22:01.854 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-10-06T02:22:01.855 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-10-06T02:22:01.855 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-10-06T02:22:01.855 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-10-06T02:22:01.855 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-10-06T02:22:01.855 INFO:tasks.ceph:Setting up mon.a... 2023-10-06T02:22:01.855 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-10-06T02:22:01.978 INFO:teuthology.orchestra.run.smithi031.stdout:creating /etc/ceph/ceph.keyring 2023-10-06T02:22:01.983 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-10-06T02:22:02.073 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-06T02:22:02.134 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.31'), ('mon.b', '172.21.15.62'), ('mon.c', '[v2:172.21.15.62:3301,v1:172.21.15.62:6790]')] 2023-10-06T02:22:02.135 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.31,172.21.15.62,[v2:172.21.15.62:3301,v1:172.21.15.62:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-10-06T02:22:02.135 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T02:22:02.136 DEBUG:teuthology.orchestra.run.smithi031:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-10-06T02:22:02.174 DEBUG:teuthology.orchestra.run.smithi031:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.31 --add b 172.21.15.62 --addv c '[v2:172.21.15.62:3301,v1:172.21.15.62:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-10-06T02:22:02.288 INFO:teuthology.orchestra.run.smithi031.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-10-06T02:22:02.288 INFO:teuthology.orchestra.run.smithi031.stdout:monmaptool: generated fsid d6fcf512-874b-4259-b90a-30ae070f00a9 2023-10-06T02:22:02.288 INFO:teuthology.orchestra.run.smithi031.stdout:setting min_mon_release = octopus 2023-10-06T02:22:02.288 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 0 2023-10-06T02:22:02.288 INFO:teuthology.orchestra.run.smithi031.stdout:fsid d6fcf512-874b-4259-b90a-30ae070f00a9 2023-10-06T02:22:02.288 INFO:teuthology.orchestra.run.smithi031.stdout:last_changed 2023-10-06T02:22:02.288364+0000 2023-10-06T02:22:02.288 INFO:teuthology.orchestra.run.smithi031.stdout:created 2023-10-06T02:22:02.288364+0000 2023-10-06T02:22:02.288 INFO:teuthology.orchestra.run.smithi031.stdout:min_mon_release 15 (octopus) 2023-10-06T02:22:02.288 INFO:teuthology.orchestra.run.smithi031.stdout:election_strategy: 1 2023-10-06T02:22:02.288 INFO:teuthology.orchestra.run.smithi031.stdout:0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.a 2023-10-06T02:22:02.289 INFO:teuthology.orchestra.run.smithi031.stdout:1: [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] mon.b 2023-10-06T02:22:02.289 INFO:teuthology.orchestra.run.smithi031.stdout:2: [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] mon.c 2023-10-06T02:22:02.289 INFO:teuthology.orchestra.run.smithi031.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-10-06T02:22:02.290 DEBUG:teuthology.orchestra.run.smithi031:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-10-06T02:22:02.326 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID d6fcf512-874b-4259-b90a-30ae070f00a9... 2023-10-06T02:22:02.327 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-06T02:22:02.370 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-06T02:22:02.479 INFO:teuthology.orchestra.run.smithi031.stdout:[global] 2023-10-06T02:22:02.479 INFO:teuthology.orchestra.run.smithi031.stdout: chdir = "" 2023-10-06T02:22:02.479 INFO:teuthology.orchestra.run.smithi031.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-06T02:22:02.480 INFO:teuthology.orchestra.run.smithi031.stdout: auth supported = cephx 2023-10-06T02:22:02.480 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:22:02.480 INFO:teuthology.orchestra.run.smithi031.stdout: filestore xattr use omap = true 2023-10-06T02:22:02.480 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:22:02.480 INFO:teuthology.orchestra.run.smithi031.stdout: mon clock drift allowed = 1.000 2023-10-06T02:22:02.480 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:22:02.480 INFO:teuthology.orchestra.run.smithi031.stdout: osd crush chooseleaf type = 0 2023-10-06T02:22:02.481 INFO:teuthology.orchestra.run.smithi031.stdout: auth debug = true 2023-10-06T02:22:02.481 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:22:02.481 INFO:teuthology.orchestra.run.smithi031.stdout: ms die on old message = true 2023-10-06T02:22:02.481 INFO:teuthology.orchestra.run.smithi031.stdout: ms die on bug = true 2023-10-06T02:22:02.481 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:22:02.481 INFO:teuthology.orchestra.run.smithi031.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-06T02:22:02.481 INFO:teuthology.orchestra.run.smithi031.stdout: mon pg warn max object skew = 0 2023-10-06T02:22:02.481 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:22:02.482 INFO:teuthology.orchestra.run.smithi031.stdout: # disable pg_autoscaler by default for new pools 2023-10-06T02:22:02.482 INFO:teuthology.orchestra.run.smithi031.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-06T02:22:02.482 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:22:02.482 INFO:teuthology.orchestra.run.smithi031.stdout: osd pool default size = 2 2023-10-06T02:22:02.482 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:22:02.482 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd allow primary affinity = true 2023-10-06T02:22:02.482 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd allow pg remap = true 2023-10-06T02:22:02.482 INFO:teuthology.orchestra.run.smithi031.stdout: mon warn on legacy crush tunables = false 2023-10-06T02:22:02.482 INFO:teuthology.orchestra.run.smithi031.stdout: mon warn on crush straw calc version zero = false 2023-10-06T02:22:02.482 INFO:teuthology.orchestra.run.smithi031.stdout: mon warn on no sortbitwise = false 2023-10-06T02:22:02.482 INFO:teuthology.orchestra.run.smithi031.stdout: mon warn on osd down out interval zero = false 2023-10-06T02:22:02.483 INFO:teuthology.orchestra.run.smithi031.stdout: mon warn on too few osds = false 2023-10-06T02:22:02.483 INFO:teuthology.orchestra.run.smithi031.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-06T02:22:02.483 INFO:teuthology.orchestra.run.smithi031.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-06T02:22:02.483 INFO:teuthology.orchestra.run.smithi031.stdout: mon_allow_pool_size_one = true 2023-10-06T02:22:02.483 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:22:02.483 INFO:teuthology.orchestra.run.smithi031.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-06T02:22:02.483 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:22:02.483 INFO:teuthology.orchestra.run.smithi031.stdout: osd default data pool replay window = 5 2023-10-06T02:22:02.483 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:22:02.483 INFO:teuthology.orchestra.run.smithi031.stdout: mon allow pool delete = true 2023-10-06T02:22:02.483 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:22:02.483 INFO:teuthology.orchestra.run.smithi031.stdout: mon cluster log file level = debug 2023-10-06T02:22:02.483 INFO:teuthology.orchestra.run.smithi031.stdout: debug asserts on shutdown = true 2023-10-06T02:22:02.484 INFO:teuthology.orchestra.run.smithi031.stdout: mon health detail to clog = false 2023-10-06T02:22:02.484 INFO:teuthology.orchestra.run.smithi031.stdout: mon host = "172.21.15.31,172.21.15.62,[v2:172.21.15.62:3301,v1:172.21.15.62:6790]" 2023-10-06T02:22:02.484 INFO:teuthology.orchestra.run.smithi031.stdout: ms die on skipped message = False 2023-10-06T02:22:02.484 INFO:teuthology.orchestra.run.smithi031.stdout: fsid = d6fcf512-874b-4259-b90a-30ae070f00a9 2023-10-06T02:22:02.484 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:22:02.484 INFO:teuthology.orchestra.run.smithi031.stdout:[osd] 2023-10-06T02:22:02.484 INFO:teuthology.orchestra.run.smithi031.stdout: osd journal size = 100 2023-10-06T02:22:02.484 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:22:02.484 INFO:teuthology.orchestra.run.smithi031.stdout: osd scrub load threshold = 5.0 2023-10-06T02:22:02.484 INFO:teuthology.orchestra.run.smithi031.stdout: osd scrub max interval = 600 2023-10-06T02:22:02.484 INFO:teuthology.orchestra.run.smithi031.stdout: osd mclock profile = high_recovery_ops 2023-10-06T02:22:02.484 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:22:02.485 INFO:teuthology.orchestra.run.smithi031.stdout: osd recover clone overlap = true 2023-10-06T02:22:02.485 INFO:teuthology.orchestra.run.smithi031.stdout: osd recovery max chunk = 1048576 2023-10-06T02:22:02.485 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:22:02.485 INFO:teuthology.orchestra.run.smithi031.stdout: osd debug shutdown = true 2023-10-06T02:22:02.485 INFO:teuthology.orchestra.run.smithi031.stdout: osd debug op order = true 2023-10-06T02:22:02.485 INFO:teuthology.orchestra.run.smithi031.stdout: osd debug verify stray on activate = true 2023-10-06T02:22:02.485 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:22:02.485 INFO:teuthology.orchestra.run.smithi031.stdout: osd open classes on start = true 2023-10-06T02:22:02.485 INFO:teuthology.orchestra.run.smithi031.stdout: osd debug pg log writeout = true 2023-10-06T02:22:02.485 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:22:02.485 INFO:teuthology.orchestra.run.smithi031.stdout: osd deep scrub update digest min age = 30 2023-10-06T02:22:02.485 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:22:02.485 INFO:teuthology.orchestra.run.smithi031.stdout: osd map max advance = 10 2023-10-06T02:22:02.486 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:22:02.486 INFO:teuthology.orchestra.run.smithi031.stdout: journal zero on create = true 2023-10-06T02:22:02.486 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:22:02.486 INFO:teuthology.orchestra.run.smithi031.stdout: filestore ondisk finisher threads = 3 2023-10-06T02:22:02.486 INFO:teuthology.orchestra.run.smithi031.stdout: filestore apply finisher threads = 3 2023-10-06T02:22:02.486 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:22:02.486 INFO:teuthology.orchestra.run.smithi031.stdout: bdev debug aio = true 2023-10-06T02:22:02.486 INFO:teuthology.orchestra.run.smithi031.stdout: osd debug misdirected ops = true 2023-10-06T02:22:02.486 INFO:teuthology.orchestra.run.smithi031.stdout: bdev async discard = True 2023-10-06T02:22:02.486 INFO:teuthology.orchestra.run.smithi031.stdout: bdev enable discard = True 2023-10-06T02:22:02.486 INFO:teuthology.orchestra.run.smithi031.stdout: bluestore allocator = bitmap 2023-10-06T02:22:02.486 INFO:teuthology.orchestra.run.smithi031.stdout: bluestore block size = 96636764160 2023-10-06T02:22:02.487 INFO:teuthology.orchestra.run.smithi031.stdout: bluestore fsck on mount = True 2023-10-06T02:22:02.487 INFO:teuthology.orchestra.run.smithi031.stdout: debug bluefs = 1/20 2023-10-06T02:22:02.487 INFO:teuthology.orchestra.run.smithi031.stdout: debug bluestore = 1/20 2023-10-06T02:22:02.487 INFO:teuthology.orchestra.run.smithi031.stdout: debug ms = 1 2023-10-06T02:22:02.487 INFO:teuthology.orchestra.run.smithi031.stdout: debug osd = 20 2023-10-06T02:22:02.487 INFO:teuthology.orchestra.run.smithi031.stdout: debug rocksdb = 4/10 2023-10-06T02:22:02.487 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd backfillfull_ratio = 0.85 2023-10-06T02:22:02.487 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd full ratio = 0.9 2023-10-06T02:22:02.487 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd nearfull ratio = 0.8 2023-10-06T02:22:02.487 INFO:teuthology.orchestra.run.smithi031.stdout: osd failsafe full ratio = 0.95 2023-10-06T02:22:02.487 INFO:teuthology.orchestra.run.smithi031.stdout: osd objectstore = bluestore 2023-10-06T02:22:02.487 INFO:teuthology.orchestra.run.smithi031.stdout: osd op complaint time = 180 2023-10-06T02:22:02.487 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:22:02.488 INFO:teuthology.orchestra.run.smithi031.stdout:[mgr] 2023-10-06T02:22:02.488 INFO:teuthology.orchestra.run.smithi031.stdout: debug ms = 1 2023-10-06T02:22:02.488 INFO:teuthology.orchestra.run.smithi031.stdout: debug mgr = 20 2023-10-06T02:22:02.488 INFO:teuthology.orchestra.run.smithi031.stdout: debug mon = 20 2023-10-06T02:22:02.488 INFO:teuthology.orchestra.run.smithi031.stdout: debug auth = 20 2023-10-06T02:22:02.488 INFO:teuthology.orchestra.run.smithi031.stdout: mon reweight min pgs per osd = 4 2023-10-06T02:22:02.488 INFO:teuthology.orchestra.run.smithi031.stdout: mon reweight min bytes per osd = 10 2023-10-06T02:22:02.488 INFO:teuthology.orchestra.run.smithi031.stdout: mgr/telemetry/nag = false 2023-10-06T02:22:02.488 INFO:teuthology.orchestra.run.smithi031.stdout: debug client = 20 2023-10-06T02:22:02.488 INFO:teuthology.orchestra.run.smithi031.stdout: debug finisher = 20 2023-10-06T02:22:02.488 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:22:02.488 INFO:teuthology.orchestra.run.smithi031.stdout:[mon] 2023-10-06T02:22:02.488 INFO:teuthology.orchestra.run.smithi031.stdout: debug ms = 1 2023-10-06T02:22:02.489 INFO:teuthology.orchestra.run.smithi031.stdout: debug mon = 20 2023-10-06T02:22:02.489 INFO:teuthology.orchestra.run.smithi031.stdout: debug paxos = 20 2023-10-06T02:22:02.489 INFO:teuthology.orchestra.run.smithi031.stdout: debug auth = 20 2023-10-06T02:22:02.489 INFO:teuthology.orchestra.run.smithi031.stdout: mon data avail warn = 5 2023-10-06T02:22:02.489 INFO:teuthology.orchestra.run.smithi031.stdout: mon mgr mkfs grace = 240 2023-10-06T02:22:02.489 INFO:teuthology.orchestra.run.smithi031.stdout: mon reweight min pgs per osd = 4 2023-10-06T02:22:02.489 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd reporter subtree level = osd 2023-10-06T02:22:02.489 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd prime pg temp = true 2023-10-06T02:22:02.489 INFO:teuthology.orchestra.run.smithi031.stdout: mon reweight min bytes per osd = 10 2023-10-06T02:22:02.489 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:22:02.489 INFO:teuthology.orchestra.run.smithi031.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-06T02:22:02.489 INFO:teuthology.orchestra.run.smithi031.stdout: auth mon ticket ttl = 660 # 11m 2023-10-06T02:22:02.489 INFO:teuthology.orchestra.run.smithi031.stdout: auth service ticket ttl = 240 # 4m 2023-10-06T02:22:02.490 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:22:02.490 INFO:teuthology.orchestra.run.smithi031.stdout: # don't complain about insecure global_id in the test suite 2023-10-06T02:22:02.490 INFO:teuthology.orchestra.run.smithi031.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-06T02:22:02.490 INFO:teuthology.orchestra.run.smithi031.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-06T02:22:02.490 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:22:02.490 INFO:teuthology.orchestra.run.smithi031.stdout: # 1m isn't quite enough 2023-10-06T02:22:02.490 INFO:teuthology.orchestra.run.smithi031.stdout: mon_down_mkfs_grace = 2m 2023-10-06T02:22:02.490 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:22:02.490 INFO:teuthology.orchestra.run.smithi031.stdout: mon_warn_on_filestore_osds = false 2023-10-06T02:22:02.490 INFO:teuthology.orchestra.run.smithi031.stdout: mon op complaint time = 120 2023-10-06T02:22:02.490 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:22:02.490 INFO:teuthology.orchestra.run.smithi031.stdout:[client] 2023-10-06T02:22:02.490 INFO:teuthology.orchestra.run.smithi031.stdout: rgw cache enabled = true 2023-10-06T02:22:02.491 INFO:teuthology.orchestra.run.smithi031.stdout: rgw enable ops log = true 2023-10-06T02:22:02.491 INFO:teuthology.orchestra.run.smithi031.stdout: rgw enable usage log = true 2023-10-06T02:22:02.491 INFO:teuthology.orchestra.run.smithi031.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-06T02:22:02.491 INFO:teuthology.orchestra.run.smithi031.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-06T02:22:02.491 INFO:teuthology.orchestra.run.smithi031.stdout: client mount timeout = 600 2023-10-06T02:22:02.491 INFO:teuthology.orchestra.run.smithi031.stdout: debug client = 20 2023-10-06T02:22:02.491 INFO:teuthology.orchestra.run.smithi031.stdout: debug ms = 1 2023-10-06T02:22:02.491 INFO:teuthology.orchestra.run.smithi031.stdout: pid file = "" 2023-10-06T02:22:02.491 INFO:teuthology.orchestra.run.smithi031.stdout: rados mon op timeout = 900 2023-10-06T02:22:02.491 INFO:teuthology.orchestra.run.smithi031.stdout: rados osd op timeout = 900 2023-10-06T02:22:02.491 INFO:teuthology.orchestra.run.smithi031.stdout:[mon.a] 2023-10-06T02:22:02.491 INFO:teuthology.orchestra.run.smithi031.stdout:[mon.b] 2023-10-06T02:22:02.491 INFO:teuthology.orchestra.run.smithi031.stdout:[mon.c] 2023-10-06T02:22:02.492 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.a] 2023-10-06T02:22:02.492 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.c] 2023-10-06T02:22:02.492 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.b] 2023-10-06T02:22:02.492 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.d] 2023-10-06T02:22:02.492 INFO:teuthology.orchestra.run.smithi031.stdout:[mds] 2023-10-06T02:22:02.492 INFO:teuthology.orchestra.run.smithi031.stdout: debug mds = 20 2023-10-06T02:22:02.492 INFO:teuthology.orchestra.run.smithi031.stdout: debug ms = 1 2023-10-06T02:22:02.492 INFO:teuthology.orchestra.run.smithi031.stdout: mds debug frag = True 2023-10-06T02:22:02.492 INFO:teuthology.orchestra.run.smithi031.stdout: mds debug scatterstat = True 2023-10-06T02:22:02.492 INFO:teuthology.orchestra.run.smithi031.stdout: mds op complaint time = 180 2023-10-06T02:22:02.492 INFO:teuthology.orchestra.run.smithi031.stdout: mds verify scatter = True 2023-10-06T02:22:02.492 INFO:teuthology.orchestra.run.smithi031.stdout: osd op complaint time = 180 2023-10-06T02:22:02.492 INFO:teuthology.orchestra.run.smithi031.stdout: rados mon op timeout = 900 2023-10-06T02:22:02.493 INFO:teuthology.orchestra.run.smithi031.stdout: rados osd op timeout = 900 2023-10-06T02:22:02.493 INFO:teuthology.orchestra.run.smithi062.stdout:[global] 2023-10-06T02:22:02.493 INFO:teuthology.orchestra.run.smithi062.stdout: chdir = "" 2023-10-06T02:22:02.493 INFO:teuthology.orchestra.run.smithi062.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-06T02:22:02.493 INFO:teuthology.orchestra.run.smithi062.stdout: auth supported = cephx 2023-10-06T02:22:02.493 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-10-06T02:22:02.493 INFO:teuthology.orchestra.run.smithi062.stdout: filestore xattr use omap = true 2023-10-06T02:22:02.494 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-10-06T02:22:02.494 INFO:teuthology.orchestra.run.smithi062.stdout: mon clock drift allowed = 1.000 2023-10-06T02:22:02.494 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-10-06T02:22:02.494 INFO:teuthology.orchestra.run.smithi062.stdout: osd crush chooseleaf type = 0 2023-10-06T02:22:02.494 INFO:teuthology.orchestra.run.smithi062.stdout: auth debug = true 2023-10-06T02:22:02.494 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-10-06T02:22:02.494 INFO:teuthology.orchestra.run.smithi062.stdout: ms die on old message = true 2023-10-06T02:22:02.494 INFO:teuthology.orchestra.run.smithi062.stdout: ms die on bug = true 2023-10-06T02:22:02.494 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-10-06T02:22:02.494 INFO:teuthology.orchestra.run.smithi062.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-06T02:22:02.494 INFO:teuthology.orchestra.run.smithi062.stdout: mon pg warn max object skew = 0 2023-10-06T02:22:02.494 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-10-06T02:22:02.494 INFO:teuthology.orchestra.run.smithi062.stdout: # disable pg_autoscaler by default for new pools 2023-10-06T02:22:02.495 INFO:teuthology.orchestra.run.smithi062.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-06T02:22:02.495 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-10-06T02:22:02.495 INFO:teuthology.orchestra.run.smithi062.stdout: osd pool default size = 2 2023-10-06T02:22:02.495 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-10-06T02:22:02.495 INFO:teuthology.orchestra.run.smithi062.stdout: mon osd allow primary affinity = true 2023-10-06T02:22:02.495 INFO:teuthology.orchestra.run.smithi062.stdout: mon osd allow pg remap = true 2023-10-06T02:22:02.495 INFO:teuthology.orchestra.run.smithi062.stdout: mon warn on legacy crush tunables = false 2023-10-06T02:22:02.495 INFO:teuthology.orchestra.run.smithi062.stdout: mon warn on crush straw calc version zero = false 2023-10-06T02:22:02.495 INFO:teuthology.orchestra.run.smithi062.stdout: mon warn on no sortbitwise = false 2023-10-06T02:22:02.495 INFO:teuthology.orchestra.run.smithi062.stdout: mon warn on osd down out interval zero = false 2023-10-06T02:22:02.495 INFO:teuthology.orchestra.run.smithi062.stdout: mon warn on too few osds = false 2023-10-06T02:22:02.495 INFO:teuthology.orchestra.run.smithi062.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-06T02:22:02.496 INFO:teuthology.orchestra.run.smithi062.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-06T02:22:02.496 INFO:teuthology.orchestra.run.smithi062.stdout: mon_allow_pool_size_one = true 2023-10-06T02:22:02.496 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-10-06T02:22:02.496 INFO:teuthology.orchestra.run.smithi062.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-06T02:22:02.496 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-10-06T02:22:02.496 INFO:teuthology.orchestra.run.smithi062.stdout: osd default data pool replay window = 5 2023-10-06T02:22:02.496 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-10-06T02:22:02.496 INFO:teuthology.orchestra.run.smithi062.stdout: mon allow pool delete = true 2023-10-06T02:22:02.496 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-10-06T02:22:02.496 INFO:teuthology.orchestra.run.smithi062.stdout: mon cluster log file level = debug 2023-10-06T02:22:02.496 INFO:teuthology.orchestra.run.smithi062.stdout: debug asserts on shutdown = true 2023-10-06T02:22:02.496 INFO:teuthology.orchestra.run.smithi062.stdout: mon health detail to clog = false 2023-10-06T02:22:02.496 INFO:teuthology.orchestra.run.smithi062.stdout: mon host = "172.21.15.31,172.21.15.62,[v2:172.21.15.62:3301,v1:172.21.15.62:6790]" 2023-10-06T02:22:02.497 INFO:teuthology.orchestra.run.smithi062.stdout: ms die on skipped message = False 2023-10-06T02:22:02.497 INFO:teuthology.orchestra.run.smithi062.stdout: fsid = d6fcf512-874b-4259-b90a-30ae070f00a9 2023-10-06T02:22:02.497 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-10-06T02:22:02.497 INFO:teuthology.orchestra.run.smithi062.stdout:[osd] 2023-10-06T02:22:02.497 INFO:teuthology.orchestra.run.smithi062.stdout: osd journal size = 100 2023-10-06T02:22:02.497 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-10-06T02:22:02.497 INFO:teuthology.orchestra.run.smithi062.stdout: osd scrub load threshold = 5.0 2023-10-06T02:22:02.497 INFO:teuthology.orchestra.run.smithi062.stdout: osd scrub max interval = 600 2023-10-06T02:22:02.497 INFO:teuthology.orchestra.run.smithi062.stdout: osd mclock profile = high_recovery_ops 2023-10-06T02:22:02.497 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-10-06T02:22:02.497 INFO:teuthology.orchestra.run.smithi062.stdout: osd recover clone overlap = true 2023-10-06T02:22:02.497 INFO:teuthology.orchestra.run.smithi062.stdout: osd recovery max chunk = 1048576 2023-10-06T02:22:02.497 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-10-06T02:22:02.498 INFO:teuthology.orchestra.run.smithi062.stdout: osd debug shutdown = true 2023-10-06T02:22:02.498 INFO:teuthology.orchestra.run.smithi062.stdout: osd debug op order = true 2023-10-06T02:22:02.498 INFO:teuthology.orchestra.run.smithi062.stdout: osd debug verify stray on activate = true 2023-10-06T02:22:02.498 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-10-06T02:22:02.498 INFO:teuthology.orchestra.run.smithi062.stdout: osd open classes on start = true 2023-10-06T02:22:02.498 INFO:teuthology.orchestra.run.smithi062.stdout: osd debug pg log writeout = true 2023-10-06T02:22:02.498 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-10-06T02:22:02.498 INFO:teuthology.orchestra.run.smithi062.stdout: osd deep scrub update digest min age = 30 2023-10-06T02:22:02.498 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-10-06T02:22:02.498 INFO:teuthology.orchestra.run.smithi062.stdout: osd map max advance = 10 2023-10-06T02:22:02.498 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-10-06T02:22:02.498 INFO:teuthology.orchestra.run.smithi062.stdout: journal zero on create = true 2023-10-06T02:22:02.499 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-10-06T02:22:02.499 INFO:teuthology.orchestra.run.smithi062.stdout: filestore ondisk finisher threads = 3 2023-10-06T02:22:02.499 INFO:teuthology.orchestra.run.smithi062.stdout: filestore apply finisher threads = 3 2023-10-06T02:22:02.499 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-10-06T02:22:02.499 INFO:teuthology.orchestra.run.smithi062.stdout: bdev debug aio = true 2023-10-06T02:22:02.499 INFO:teuthology.orchestra.run.smithi062.stdout: osd debug misdirected ops = true 2023-10-06T02:22:02.499 INFO:teuthology.orchestra.run.smithi062.stdout: bdev async discard = True 2023-10-06T02:22:02.499 INFO:teuthology.orchestra.run.smithi062.stdout: bdev enable discard = True 2023-10-06T02:22:02.499 INFO:teuthology.orchestra.run.smithi062.stdout: bluestore allocator = bitmap 2023-10-06T02:22:02.499 INFO:teuthology.orchestra.run.smithi062.stdout: bluestore block size = 96636764160 2023-10-06T02:22:02.499 INFO:teuthology.orchestra.run.smithi062.stdout: bluestore fsck on mount = True 2023-10-06T02:22:02.499 INFO:teuthology.orchestra.run.smithi062.stdout: debug bluefs = 1/20 2023-10-06T02:22:02.499 INFO:teuthology.orchestra.run.smithi062.stdout: debug bluestore = 1/20 2023-10-06T02:22:02.500 INFO:teuthology.orchestra.run.smithi062.stdout: debug ms = 1 2023-10-06T02:22:02.500 INFO:teuthology.orchestra.run.smithi062.stdout: debug osd = 20 2023-10-06T02:22:02.500 INFO:teuthology.orchestra.run.smithi062.stdout: debug rocksdb = 4/10 2023-10-06T02:22:02.500 INFO:teuthology.orchestra.run.smithi062.stdout: mon osd backfillfull_ratio = 0.85 2023-10-06T02:22:02.500 INFO:teuthology.orchestra.run.smithi062.stdout: mon osd full ratio = 0.9 2023-10-06T02:22:02.500 INFO:teuthology.orchestra.run.smithi062.stdout: mon osd nearfull ratio = 0.8 2023-10-06T02:22:02.500 INFO:teuthology.orchestra.run.smithi062.stdout: osd failsafe full ratio = 0.95 2023-10-06T02:22:02.500 INFO:teuthology.orchestra.run.smithi062.stdout: osd objectstore = bluestore 2023-10-06T02:22:02.500 INFO:teuthology.orchestra.run.smithi062.stdout: osd op complaint time = 180 2023-10-06T02:22:02.500 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-10-06T02:22:02.500 INFO:teuthology.orchestra.run.smithi062.stdout:[mgr] 2023-10-06T02:22:02.500 INFO:teuthology.orchestra.run.smithi062.stdout: debug ms = 1 2023-10-06T02:22:02.501 INFO:teuthology.orchestra.run.smithi062.stdout: debug mgr = 20 2023-10-06T02:22:02.501 INFO:teuthology.orchestra.run.smithi062.stdout: debug mon = 20 2023-10-06T02:22:02.501 INFO:teuthology.orchestra.run.smithi062.stdout: debug auth = 20 2023-10-06T02:22:02.501 INFO:teuthology.orchestra.run.smithi062.stdout: mon reweight min pgs per osd = 4 2023-10-06T02:22:02.501 INFO:teuthology.orchestra.run.smithi062.stdout: mon reweight min bytes per osd = 10 2023-10-06T02:22:02.501 INFO:teuthology.orchestra.run.smithi062.stdout: mgr/telemetry/nag = false 2023-10-06T02:22:02.501 INFO:teuthology.orchestra.run.smithi062.stdout: debug client = 20 2023-10-06T02:22:02.501 INFO:teuthology.orchestra.run.smithi062.stdout: debug finisher = 20 2023-10-06T02:22:02.501 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-10-06T02:22:02.501 INFO:teuthology.orchestra.run.smithi062.stdout:[mon] 2023-10-06T02:22:02.501 INFO:teuthology.orchestra.run.smithi062.stdout: debug ms = 1 2023-10-06T02:22:02.501 INFO:teuthology.orchestra.run.smithi062.stdout: debug mon = 20 2023-10-06T02:22:02.502 INFO:teuthology.orchestra.run.smithi062.stdout: debug paxos = 20 2023-10-06T02:22:02.502 INFO:teuthology.orchestra.run.smithi062.stdout: debug auth = 20 2023-10-06T02:22:02.502 INFO:teuthology.orchestra.run.smithi062.stdout: mon data avail warn = 5 2023-10-06T02:22:02.502 INFO:teuthology.orchestra.run.smithi062.stdout: mon mgr mkfs grace = 240 2023-10-06T02:22:02.502 INFO:teuthology.orchestra.run.smithi062.stdout: mon reweight min pgs per osd = 4 2023-10-06T02:22:02.502 INFO:teuthology.orchestra.run.smithi062.stdout: mon osd reporter subtree level = osd 2023-10-06T02:22:02.502 INFO:teuthology.orchestra.run.smithi062.stdout: mon osd prime pg temp = true 2023-10-06T02:22:02.502 INFO:teuthology.orchestra.run.smithi062.stdout: mon reweight min bytes per osd = 10 2023-10-06T02:22:02.502 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-10-06T02:22:02.502 INFO:teuthology.orchestra.run.smithi062.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-06T02:22:02.503 INFO:teuthology.orchestra.run.smithi062.stdout: auth mon ticket ttl = 660 # 11m 2023-10-06T02:22:02.503 INFO:teuthology.orchestra.run.smithi062.stdout: auth service ticket ttl = 240 # 4m 2023-10-06T02:22:02.503 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-10-06T02:22:02.503 INFO:teuthology.orchestra.run.smithi062.stdout: # don't complain about insecure global_id in the test suite 2023-10-06T02:22:02.503 INFO:teuthology.orchestra.run.smithi062.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-06T02:22:02.503 INFO:teuthology.orchestra.run.smithi062.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-06T02:22:02.503 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-10-06T02:22:02.503 INFO:teuthology.orchestra.run.smithi062.stdout: # 1m isn't quite enough 2023-10-06T02:22:02.503 INFO:teuthology.orchestra.run.smithi062.stdout: mon_down_mkfs_grace = 2m 2023-10-06T02:22:02.503 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-10-06T02:22:02.503 INFO:teuthology.orchestra.run.smithi062.stdout: mon_warn_on_filestore_osds = false 2023-10-06T02:22:02.503 INFO:teuthology.orchestra.run.smithi062.stdout: mon op complaint time = 120 2023-10-06T02:22:02.504 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-10-06T02:22:02.504 INFO:teuthology.orchestra.run.smithi062.stdout:[client] 2023-10-06T02:22:02.504 INFO:teuthology.orchestra.run.smithi062.stdout: rgw cache enabled = true 2023-10-06T02:22:02.504 INFO:teuthology.orchestra.run.smithi062.stdout: rgw enable ops log = true 2023-10-06T02:22:02.504 INFO:teuthology.orchestra.run.smithi062.stdout: rgw enable usage log = true 2023-10-06T02:22:02.504 INFO:teuthology.orchestra.run.smithi062.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-06T02:22:02.504 INFO:teuthology.orchestra.run.smithi062.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-06T02:22:02.504 INFO:teuthology.orchestra.run.smithi062.stdout: client mount timeout = 600 2023-10-06T02:22:02.504 INFO:teuthology.orchestra.run.smithi062.stdout: debug client = 20 2023-10-06T02:22:02.504 INFO:teuthology.orchestra.run.smithi062.stdout: debug ms = 1 2023-10-06T02:22:02.505 INFO:teuthology.orchestra.run.smithi062.stdout: pid file = "" 2023-10-06T02:22:02.505 INFO:teuthology.orchestra.run.smithi062.stdout: rados mon op timeout = 900 2023-10-06T02:22:02.505 INFO:teuthology.orchestra.run.smithi062.stdout: rados osd op timeout = 900 2023-10-06T02:22:02.505 INFO:teuthology.orchestra.run.smithi062.stdout:[mon.a] 2023-10-06T02:22:02.505 INFO:teuthology.orchestra.run.smithi062.stdout:[mon.b] 2023-10-06T02:22:02.505 INFO:teuthology.orchestra.run.smithi062.stdout:[mon.c] 2023-10-06T02:22:02.505 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.a] 2023-10-06T02:22:02.505 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.c] 2023-10-06T02:22:02.505 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.b] 2023-10-06T02:22:02.505 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.d] 2023-10-06T02:22:02.506 INFO:teuthology.orchestra.run.smithi062.stdout:[mds] 2023-10-06T02:22:02.506 INFO:teuthology.orchestra.run.smithi062.stdout: debug mds = 20 2023-10-06T02:22:02.506 INFO:teuthology.orchestra.run.smithi062.stdout: debug ms = 1 2023-10-06T02:22:02.506 INFO:teuthology.orchestra.run.smithi062.stdout: mds debug frag = True 2023-10-06T02:22:02.506 INFO:teuthology.orchestra.run.smithi062.stdout: mds debug scatterstat = True 2023-10-06T02:22:02.506 INFO:teuthology.orchestra.run.smithi062.stdout: mds op complaint time = 180 2023-10-06T02:22:02.506 INFO:teuthology.orchestra.run.smithi062.stdout: mds verify scatter = True 2023-10-06T02:22:02.506 INFO:teuthology.orchestra.run.smithi062.stdout: osd op complaint time = 180 2023-10-06T02:22:02.506 INFO:teuthology.orchestra.run.smithi062.stdout: rados mon op timeout = 900 2023-10-06T02:22:02.506 INFO:teuthology.orchestra.run.smithi062.stdout: rados osd op timeout = 900 2023-10-06T02:22:02.515 INFO:tasks.ceph:Creating admin key on mon.a... 2023-10-06T02:22:02.515 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-10-06T02:22:02.616 INFO:tasks.ceph:Copying monmap to all nodes... 2023-10-06T02:22:02.616 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T02:22:02.616 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-10-06T02:22:02.652 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T02:22:02.652 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-10-06T02:22:02.728 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi031.front.sepia.ceph.com 2023-10-06T02:22:02.729 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T02:22:02.729 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-06T02:22:02.729 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-06T02:22:02.856 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T02:22:02.857 DEBUG:teuthology.orchestra.run.smithi031:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-06T02:22:02.933 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi062.front.sepia.ceph.com 2023-10-06T02:22:02.933 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-10-06T02:22:02.933 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-06T02:22:02.933 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-06T02:22:03.030 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-10-06T02:22:03.030 DEBUG:teuthology.orchestra.run.smithi062:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-06T02:22:03.106 INFO:tasks.ceph:Setting up mon nodes... 2023-10-06T02:22:03.107 INFO:tasks.ceph:Setting up mgr nodes... 2023-10-06T02:22:03.107 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-10-06T02:22:03.218 INFO:teuthology.orchestra.run.smithi031.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-10-06T02:22:03.223 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-10-06T02:22:03.348 INFO:teuthology.orchestra.run.smithi062.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-10-06T02:22:03.354 INFO:tasks.ceph:Setting up mds nodes... 2023-10-06T02:22:03.354 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-10-06T02:22:03.465 INFO:teuthology.orchestra.run.smithi031.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-10-06T02:22:03.470 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-10-06T02:22:03.532 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-10-06T02:22:03.644 INFO:teuthology.orchestra.run.smithi031.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-10-06T02:22:03.648 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-10-06T02:22:03.710 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-10-06T02:22:03.823 INFO:teuthology.orchestra.run.smithi062.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-10-06T02:22:03.827 DEBUG:teuthology.orchestra.run.smithi062:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-10-06T02:22:03.889 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-10-06T02:22:04.000 INFO:teuthology.orchestra.run.smithi062.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-10-06T02:22:04.005 DEBUG:teuthology.orchestra.run.smithi062:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-10-06T02:22:04.067 INFO:tasks.ceph_client:Setting up client nodes... 2023-10-06T02:22:04.067 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-10-06T02:22:04.152 INFO:teuthology.orchestra.run.smithi031.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-10-06T02:22:04.184 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-10-06T02:22:04.268 INFO:teuthology.orchestra.run.smithi031.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-10-06T02:22:04.299 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-10-06T02:22:04.383 INFO:teuthology.orchestra.run.smithi062.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-10-06T02:22:04.415 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-10-06T02:22:04.499 INFO:teuthology.orchestra.run.smithi062.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-10-06T02:22:04.531 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-10-06T02:22:04.531 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi031.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi062.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-06T02:22:04.531 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-10-06T02:22:04.593 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-06T02:22:04.593 INFO:tasks.ceph:role: osd.0 2023-10-06T02:22:04.593 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi031.front.sepia.ceph.com 2023-10-06T02:22:04.594 DEBUG:teuthology.orchestra.run.smithi031:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-06T02:22:04.662 INFO:teuthology.orchestra.run.smithi031.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T02:22:04.662 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T02:22:04.662 INFO:teuthology.orchestra.run.smithi031.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T02:22:04.663 INFO:teuthology.orchestra.run.smithi031.stdout: = reflink=1 2023-10-06T02:22:04.663 INFO:teuthology.orchestra.run.smithi031.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T02:22:04.663 INFO:teuthology.orchestra.run.smithi031.stdout: = sunit=0 swidth=0 blks 2023-10-06T02:22:04.663 INFO:teuthology.orchestra.run.smithi031.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T02:22:04.663 INFO:teuthology.orchestra.run.smithi031.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T02:22:04.663 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T02:22:04.663 INFO:teuthology.orchestra.run.smithi031.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T02:22:05.091 INFO:teuthology.orchestra.run.smithi031.stdout:Discarding blocks...Done. 2023-10-06T02:22:05.095 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi031.front.sepia.ceph.com -o noatime 2023-10-06T02:22:05.095 DEBUG:teuthology.orchestra.run.smithi031:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-10-06T02:22:05.286 DEBUG:teuthology.orchestra.run.smithi031:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-10-06T02:22:05.358 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-10-06T02:22:05.421 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-06T02:22:05.421 INFO:tasks.ceph:role: osd.1 2023-10-06T02:22:05.421 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi031.front.sepia.ceph.com 2023-10-06T02:22:05.421 DEBUG:teuthology.orchestra.run.smithi031:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-06T02:22:05.489 INFO:teuthology.orchestra.run.smithi031.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T02:22:05.489 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T02:22:05.489 INFO:teuthology.orchestra.run.smithi031.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T02:22:05.489 INFO:teuthology.orchestra.run.smithi031.stdout: = reflink=1 2023-10-06T02:22:05.490 INFO:teuthology.orchestra.run.smithi031.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T02:22:05.490 INFO:teuthology.orchestra.run.smithi031.stdout: = sunit=0 swidth=0 blks 2023-10-06T02:22:05.490 INFO:teuthology.orchestra.run.smithi031.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T02:22:05.490 INFO:teuthology.orchestra.run.smithi031.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T02:22:05.490 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T02:22:05.490 INFO:teuthology.orchestra.run.smithi031.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T02:22:05.899 INFO:teuthology.orchestra.run.smithi031.stdout:Discarding blocks...Done. 2023-10-06T02:22:05.902 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi031.front.sepia.ceph.com -o noatime 2023-10-06T02:22:05.902 DEBUG:teuthology.orchestra.run.smithi031:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-10-06T02:22:06.026 DEBUG:teuthology.orchestra.run.smithi031:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-10-06T02:22:06.094 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-10-06T02:22:06.155 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-06T02:22:06.155 INFO:tasks.ceph:role: osd.2 2023-10-06T02:22:06.155 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi031.front.sepia.ceph.com 2023-10-06T02:22:06.156 DEBUG:teuthology.orchestra.run.smithi031:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-06T02:22:06.224 INFO:teuthology.orchestra.run.smithi031.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T02:22:06.225 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T02:22:06.225 INFO:teuthology.orchestra.run.smithi031.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T02:22:06.225 INFO:teuthology.orchestra.run.smithi031.stdout: = reflink=1 2023-10-06T02:22:06.225 INFO:teuthology.orchestra.run.smithi031.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T02:22:06.225 INFO:teuthology.orchestra.run.smithi031.stdout: = sunit=0 swidth=0 blks 2023-10-06T02:22:06.225 INFO:teuthology.orchestra.run.smithi031.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T02:22:06.225 INFO:teuthology.orchestra.run.smithi031.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T02:22:06.225 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T02:22:06.225 INFO:teuthology.orchestra.run.smithi031.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T02:22:06.668 INFO:teuthology.orchestra.run.smithi031.stdout:Discarding blocks...Done. 2023-10-06T02:22:06.672 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi031.front.sepia.ceph.com -o noatime 2023-10-06T02:22:06.672 DEBUG:teuthology.orchestra.run.smithi031:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-10-06T02:22:06.843 DEBUG:teuthology.orchestra.run.smithi031:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-10-06T02:22:06.915 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-10-06T02:22:06.977 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-06T02:22:06.977 INFO:tasks.ceph:role: osd.3 2023-10-06T02:22:06.977 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi031.front.sepia.ceph.com 2023-10-06T02:22:06.977 DEBUG:teuthology.orchestra.run.smithi031:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-06T02:22:07.045 INFO:teuthology.orchestra.run.smithi031.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T02:22:07.046 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T02:22:07.046 INFO:teuthology.orchestra.run.smithi031.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T02:22:07.046 INFO:teuthology.orchestra.run.smithi031.stdout: = reflink=1 2023-10-06T02:22:07.046 INFO:teuthology.orchestra.run.smithi031.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T02:22:07.046 INFO:teuthology.orchestra.run.smithi031.stdout: = sunit=0 swidth=0 blks 2023-10-06T02:22:07.046 INFO:teuthology.orchestra.run.smithi031.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T02:22:07.046 INFO:teuthology.orchestra.run.smithi031.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T02:22:07.046 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T02:22:07.046 INFO:teuthology.orchestra.run.smithi031.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T02:22:07.472 INFO:teuthology.orchestra.run.smithi031.stdout:Discarding blocks...Done. 2023-10-06T02:22:07.476 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi031.front.sepia.ceph.com -o noatime 2023-10-06T02:22:07.476 DEBUG:teuthology.orchestra.run.smithi031:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-10-06T02:22:07.641 DEBUG:teuthology.orchestra.run.smithi031:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-10-06T02:22:07.712 DEBUG:teuthology.orchestra.run.smithi031:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T02:22:07.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:07.816+0000 7fa58f373540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-10-06T02:22:07.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:07.816+0000 7fa58f373540 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-10-06T02:22:07.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:07.816+0000 7fa58f373540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-06T02:22:07.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:07.816+0000 7fa58f373540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-06T02:22:07.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:07.816+0000 7fa58f373540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-06T02:22:07.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:07.818+0000 7fa58f373540 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-10-06T02:22:08.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:08.360+0000 7fa58f373540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T02:22:09.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:09.128+0000 7fa58f373540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T02:22:09.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:09.892+0000 7fa58f373540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T02:22:10.711 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-10-06T02:22:10.778 DEBUG:teuthology.orchestra.run.smithi031:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T02:22:10.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:10.873+0000 7f16a9450540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-10-06T02:22:10.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:10.873+0000 7f16a9450540 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-10-06T02:22:10.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:10.873+0000 7f16a9450540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-06T02:22:10.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:10.873+0000 7f16a9450540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-06T02:22:10.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:10.873+0000 7f16a9450540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-06T02:22:10.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:10.875+0000 7f16a9450540 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-10-06T02:22:11.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:11.414+0000 7f16a9450540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T02:22:12.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:12.174+0000 7f16a9450540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T02:22:12.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:12.936+0000 7f16a9450540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T02:22:13.754 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-10-06T02:22:13.822 DEBUG:teuthology.orchestra.run.smithi031:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T02:22:13.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:13.917+0000 7f9824ac8540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-10-06T02:22:13.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:13.917+0000 7f9824ac8540 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-10-06T02:22:13.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:13.917+0000 7f9824ac8540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-06T02:22:13.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:13.917+0000 7f9824ac8540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-06T02:22:13.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:13.917+0000 7f9824ac8540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-06T02:22:13.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:13.919+0000 7f9824ac8540 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-10-06T02:22:14.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:14.457+0000 7f9824ac8540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T02:22:15.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:15.223+0000 7f9824ac8540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T02:22:15.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:15.985+0000 7f9824ac8540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T02:22:16.802 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-10-06T02:22:16.870 DEBUG:teuthology.orchestra.run.smithi031:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T02:22:17.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:17.028+0000 7fafee62a540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-10-06T02:22:17.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:17.028+0000 7fafee62a540 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-10-06T02:22:17.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:17.028+0000 7fafee62a540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-06T02:22:17.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:17.029+0000 7fafee62a540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-06T02:22:17.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:17.029+0000 7fafee62a540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-06T02:22:17.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:17.030+0000 7fafee62a540 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-10-06T02:22:17.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:17.567+0000 7fafee62a540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T02:22:18.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:18.328+0000 7fafee62a540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T02:22:19.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:19.090+0000 7fafee62a540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T02:22:19.911 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-10-06T02:22:19.977 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-10-06T02:22:20.056 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-06T02:22:20.056 INFO:tasks.ceph:role: osd.4 2023-10-06T02:22:20.056 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi062.front.sepia.ceph.com 2023-10-06T02:22:20.056 DEBUG:teuthology.orchestra.run.smithi062:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-06T02:22:20.129 INFO:teuthology.orchestra.run.smithi062.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T02:22:20.129 INFO:teuthology.orchestra.run.smithi062.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T02:22:20.129 INFO:teuthology.orchestra.run.smithi062.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T02:22:20.130 INFO:teuthology.orchestra.run.smithi062.stdout: = reflink=1 2023-10-06T02:22:20.130 INFO:teuthology.orchestra.run.smithi062.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T02:22:20.130 INFO:teuthology.orchestra.run.smithi062.stdout: = sunit=0 swidth=0 blks 2023-10-06T02:22:20.130 INFO:teuthology.orchestra.run.smithi062.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T02:22:20.130 INFO:teuthology.orchestra.run.smithi062.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T02:22:20.130 INFO:teuthology.orchestra.run.smithi062.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T02:22:20.130 INFO:teuthology.orchestra.run.smithi062.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T02:22:20.571 INFO:teuthology.orchestra.run.smithi062.stdout:Discarding blocks...Done. 2023-10-06T02:22:20.575 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi062.front.sepia.ceph.com -o noatime 2023-10-06T02:22:20.575 DEBUG:teuthology.orchestra.run.smithi062:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-10-06T02:22:20.732 DEBUG:teuthology.orchestra.run.smithi062:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-10-06T02:22:20.804 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-10-06T02:22:20.866 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-06T02:22:20.866 INFO:tasks.ceph:role: osd.5 2023-10-06T02:22:20.866 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi062.front.sepia.ceph.com 2023-10-06T02:22:20.866 DEBUG:teuthology.orchestra.run.smithi062:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-06T02:22:20.934 INFO:teuthology.orchestra.run.smithi062.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T02:22:20.935 INFO:teuthology.orchestra.run.smithi062.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T02:22:20.935 INFO:teuthology.orchestra.run.smithi062.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T02:22:20.935 INFO:teuthology.orchestra.run.smithi062.stdout: = reflink=1 2023-10-06T02:22:20.935 INFO:teuthology.orchestra.run.smithi062.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T02:22:20.935 INFO:teuthology.orchestra.run.smithi062.stdout: = sunit=0 swidth=0 blks 2023-10-06T02:22:20.935 INFO:teuthology.orchestra.run.smithi062.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T02:22:20.935 INFO:teuthology.orchestra.run.smithi062.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T02:22:20.935 INFO:teuthology.orchestra.run.smithi062.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T02:22:20.936 INFO:teuthology.orchestra.run.smithi062.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T02:22:21.344 INFO:teuthology.orchestra.run.smithi062.stdout:Discarding blocks...Done. 2023-10-06T02:22:21.348 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi062.front.sepia.ceph.com -o noatime 2023-10-06T02:22:21.349 DEBUG:teuthology.orchestra.run.smithi062:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-10-06T02:22:21.473 DEBUG:teuthology.orchestra.run.smithi062:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-10-06T02:22:21.542 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-10-06T02:22:21.603 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-06T02:22:21.604 INFO:tasks.ceph:role: osd.6 2023-10-06T02:22:21.604 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi062.front.sepia.ceph.com 2023-10-06T02:22:21.604 DEBUG:teuthology.orchestra.run.smithi062:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-06T02:22:21.673 INFO:teuthology.orchestra.run.smithi062.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T02:22:21.673 INFO:teuthology.orchestra.run.smithi062.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T02:22:21.673 INFO:teuthology.orchestra.run.smithi062.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T02:22:21.673 INFO:teuthology.orchestra.run.smithi062.stdout: = reflink=1 2023-10-06T02:22:21.673 INFO:teuthology.orchestra.run.smithi062.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T02:22:21.673 INFO:teuthology.orchestra.run.smithi062.stdout: = sunit=0 swidth=0 blks 2023-10-06T02:22:21.673 INFO:teuthology.orchestra.run.smithi062.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T02:22:21.673 INFO:teuthology.orchestra.run.smithi062.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T02:22:21.674 INFO:teuthology.orchestra.run.smithi062.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T02:22:21.674 INFO:teuthology.orchestra.run.smithi062.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T02:22:22.126 INFO:teuthology.orchestra.run.smithi062.stdout:Discarding blocks...Done. 2023-10-06T02:22:22.130 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi062.front.sepia.ceph.com -o noatime 2023-10-06T02:22:22.130 DEBUG:teuthology.orchestra.run.smithi062:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-10-06T02:22:22.313 DEBUG:teuthology.orchestra.run.smithi062:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-10-06T02:22:22.383 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-10-06T02:22:22.444 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-06T02:22:22.444 INFO:tasks.ceph:role: osd.7 2023-10-06T02:22:22.444 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi062.front.sepia.ceph.com 2023-10-06T02:22:22.444 DEBUG:teuthology.orchestra.run.smithi062:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-06T02:22:22.512 INFO:teuthology.orchestra.run.smithi062.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T02:22:22.512 INFO:teuthology.orchestra.run.smithi062.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T02:22:22.513 INFO:teuthology.orchestra.run.smithi062.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T02:22:22.513 INFO:teuthology.orchestra.run.smithi062.stdout: = reflink=1 2023-10-06T02:22:22.513 INFO:teuthology.orchestra.run.smithi062.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T02:22:22.513 INFO:teuthology.orchestra.run.smithi062.stdout: = sunit=0 swidth=0 blks 2023-10-06T02:22:22.513 INFO:teuthology.orchestra.run.smithi062.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T02:22:22.513 INFO:teuthology.orchestra.run.smithi062.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T02:22:22.513 INFO:teuthology.orchestra.run.smithi062.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T02:22:22.513 INFO:teuthology.orchestra.run.smithi062.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T02:22:22.958 INFO:teuthology.orchestra.run.smithi062.stdout:Discarding blocks...Done. 2023-10-06T02:22:22.962 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi062.front.sepia.ceph.com -o noatime 2023-10-06T02:22:22.962 DEBUG:teuthology.orchestra.run.smithi062:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-10-06T02:22:23.129 DEBUG:teuthology.orchestra.run.smithi062:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-10-06T02:22:23.200 DEBUG:teuthology.orchestra.run.smithi062:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T02:22:23.311 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:23.310+0000 7f50f0826540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-10-06T02:22:23.311 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:23.310+0000 7f50f0826540 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-10-06T02:22:23.311 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:23.310+0000 7f50f0826540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-06T02:22:23.311 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:23.310+0000 7f50f0826540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-06T02:22:23.311 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:23.310+0000 7f50f0826540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-06T02:22:23.313 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:23.312+0000 7f50f0826540 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-10-06T02:22:23.850 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:23.849+0000 7f50f0826540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T02:22:24.619 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:24.618+0000 7f50f0826540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T02:22:25.382 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:25.381+0000 7f50f0826540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T02:22:26.201 DEBUG:teuthology.orchestra.run.smithi062:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-10-06T02:22:26.267 DEBUG:teuthology.orchestra.run.smithi062:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T02:22:26.360 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:26.360+0000 7ff942999540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-10-06T02:22:26.361 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:26.360+0000 7ff942999540 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-10-06T02:22:26.361 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:26.360+0000 7ff942999540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-06T02:22:26.361 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:26.360+0000 7ff942999540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-06T02:22:26.361 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:26.360+0000 7ff942999540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-06T02:22:26.363 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:26.362+0000 7ff942999540 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-10-06T02:22:26.899 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:26.898+0000 7ff942999540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T02:22:27.660 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:27.659+0000 7ff942999540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T02:22:28.424 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:28.423+0000 7ff942999540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T02:22:29.238 DEBUG:teuthology.orchestra.run.smithi062:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-10-06T02:22:29.304 DEBUG:teuthology.orchestra.run.smithi062:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T02:22:29.398 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:29.397+0000 7fbc75b04540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-10-06T02:22:29.398 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:29.397+0000 7fbc75b04540 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-10-06T02:22:29.398 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:29.397+0000 7fbc75b04540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-06T02:22:29.398 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:29.397+0000 7fbc75b04540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-06T02:22:29.399 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:29.397+0000 7fbc75b04540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-06T02:22:29.399 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:29.399+0000 7fbc75b04540 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-10-06T02:22:29.832 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T02:22:29.837 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T02:22:29.961 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:29.960+0000 7fbc75b04540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T02:22:30.723 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:30.723+0000 7fbc75b04540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T02:22:31.487 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:31.486+0000 7fbc75b04540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T02:22:32.301 DEBUG:teuthology.orchestra.run.smithi062:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-10-06T02:22:32.409 DEBUG:teuthology.orchestra.run.smithi062:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T02:22:32.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:32.503+0000 7f3246687540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-10-06T02:22:32.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:32.504+0000 7f3246687540 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-10-06T02:22:32.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:32.504+0000 7f3246687540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-06T02:22:32.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:32.504+0000 7f3246687540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-06T02:22:32.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:32.504+0000 7f3246687540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-06T02:22:32.506 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:32.506+0000 7f3246687540 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-10-06T02:22:33.044 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:33.043+0000 7f3246687540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T02:22:33.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:33.804+0000 7f3246687540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T02:22:34.568 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:34.567+0000 7f3246687540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T02:22:35.384 DEBUG:teuthology.orchestra.run.smithi062:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-10-06T02:22:35.450 INFO:tasks.ceph:Reading keys from all nodes... 2023-10-06T02:22:35.451 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T02:22:35.451 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-10-06T02:22:35.519 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T02:22:35.519 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-10-06T02:22:35.623 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T02:22:35.623 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-10-06T02:22:35.725 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T02:22:35.725 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-10-06T02:22:35.826 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T02:22:35.826 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-10-06T02:22:35.927 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T02:22:35.927 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-10-06T02:22:36.028 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T02:22:36.028 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-10-06T02:22:36.129 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-10-06T02:22:36.129 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-10-06T02:22:36.197 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-10-06T02:22:36.197 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-10-06T02:22:36.301 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-10-06T02:22:36.302 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-10-06T02:22:36.404 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-10-06T02:22:36.404 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-10-06T02:22:36.506 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-10-06T02:22:36.507 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-10-06T02:22:36.607 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-10-06T02:22:36.608 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-10-06T02:22:36.709 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-10-06T02:22:36.709 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-10-06T02:22:36.809 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T02:22:36.809 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-10-06T02:22:36.849 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T02:22:36.849 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-10-06T02:22:36.926 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-10-06T02:22:36.927 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-10-06T02:22:36.962 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-10-06T02:22:36.962 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-10-06T02:22:37.038 INFO:tasks.ceph:Adding keys to all mons... 2023-10-06T02:22:37.038 DEBUG:teuthology.orchestra.run.smithi031:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-06T02:22:37.042 DEBUG:teuthology.orchestra.run.smithi062:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-06T02:22:37.098 INFO:teuthology.orchestra.run.smithi031.stdout:[mgr.y] 2023-10-06T02:22:37.099 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQBLbx9l4+oLDRAA5YVo3q6FI9e9W/rJYROIsg== 2023-10-06T02:22:37.099 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.a] 2023-10-06T02:22:37.099 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQBLbx9lbjrOGxAAVoSOoDaZb1temL41F4Nong== 2023-10-06T02:22:37.099 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.c] 2023-10-06T02:22:37.099 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQBLbx9lezVtJhAA8wK7+dJ1OmIc+iXJkexK0w== 2023-10-06T02:22:37.099 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.0] 2023-10-06T02:22:37.099 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQBPbx9lgj6+MBAAXSUD0GCG+rO/9DQBZYt/8w== 2023-10-06T02:22:37.099 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.1] 2023-10-06T02:22:37.099 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQBSbx9lXhMTNBAAeEdPnMP71ZGb6k2Ddg5raQ== 2023-10-06T02:22:37.099 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.2] 2023-10-06T02:22:37.099 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQBVbx9l4Ly0NhAATYXYDAjYuP53zjqK0yMboA== 2023-10-06T02:22:37.100 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.3] 2023-10-06T02:22:37.100 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQBZbx9lzsi4ARAAbGnNV0QyL/7itzy827+chg== 2023-10-06T02:22:37.100 INFO:teuthology.orchestra.run.smithi031.stdout:[mgr.x] 2023-10-06T02:22:37.100 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQBLbx9lTBbIFBAAX2wsioKL4Nx4qz/qwsCsNA== 2023-10-06T02:22:37.100 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.b] 2023-10-06T02:22:37.100 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQBLbx9lFgsTMRAATTklzroiyQbTLiZc9j1BHQ== 2023-10-06T02:22:37.100 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.d] 2023-10-06T02:22:37.100 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQBMbx9l74gWABAApBu2nVv12rGA+OBfBnEc3A== 2023-10-06T02:22:37.100 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.4] 2023-10-06T02:22:37.100 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQBfbx9lW4OSEhAAj7HnVh/yq12zzCaXd6BHCA== 2023-10-06T02:22:37.100 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.5] 2023-10-06T02:22:37.100 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQBibx9lBcaIFRAAV8wq8QjsCOkI9Pzf/E+9bA== 2023-10-06T02:22:37.101 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.6] 2023-10-06T02:22:37.101 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQBlbx9lA7S+FxAA38mlSAurT+DpwMzH53zFcw== 2023-10-06T02:22:37.101 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.7] 2023-10-06T02:22:37.101 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQBobx9l5RkdHhAAxn0Z8IAE/o9oe2LRp3O9gg== 2023-10-06T02:22:37.101 INFO:teuthology.orchestra.run.smithi031.stdout:[client.0] 2023-10-06T02:22:37.101 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQBMbx9lh44fCRAA7sMISvk1t7dEJg4ZcBiMcA== 2023-10-06T02:22:37.101 INFO:teuthology.orchestra.run.smithi031.stdout:[client.2] 2023-10-06T02:22:37.101 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQBMbx9lpxYAEBAACtiI1XgQXM8yGGcpFyNK1w== 2023-10-06T02:22:37.101 INFO:teuthology.orchestra.run.smithi031.stdout:[client.1] 2023-10-06T02:22:37.101 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQBMbx9lP0zqFhAAlov2y7ipSvS3AkOdQkhg6g== 2023-10-06T02:22:37.101 INFO:teuthology.orchestra.run.smithi031.stdout:[client.3] 2023-10-06T02:22:37.101 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQBMbx9l81/PHRAAYFOoaezu0pwE0NdtnlYfVA== 2023-10-06T02:22:37.137 INFO:teuthology.orchestra.run.smithi062.stdout:[mgr.y] 2023-10-06T02:22:37.137 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQBLbx9l4+oLDRAA5YVo3q6FI9e9W/rJYROIsg== 2023-10-06T02:22:37.137 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.a] 2023-10-06T02:22:37.138 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQBLbx9lbjrOGxAAVoSOoDaZb1temL41F4Nong== 2023-10-06T02:22:37.138 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.c] 2023-10-06T02:22:37.138 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQBLbx9lezVtJhAA8wK7+dJ1OmIc+iXJkexK0w== 2023-10-06T02:22:37.138 INFO:teuthology.orchestra.run.smithi062.stdout:[osd.0] 2023-10-06T02:22:37.138 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQBPbx9lgj6+MBAAXSUD0GCG+rO/9DQBZYt/8w== 2023-10-06T02:22:37.138 INFO:teuthology.orchestra.run.smithi062.stdout:[osd.1] 2023-10-06T02:22:37.138 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQBSbx9lXhMTNBAAeEdPnMP71ZGb6k2Ddg5raQ== 2023-10-06T02:22:37.138 INFO:teuthology.orchestra.run.smithi062.stdout:[osd.2] 2023-10-06T02:22:37.138 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQBVbx9l4Ly0NhAATYXYDAjYuP53zjqK0yMboA== 2023-10-06T02:22:37.138 INFO:teuthology.orchestra.run.smithi062.stdout:[osd.3] 2023-10-06T02:22:37.138 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQBZbx9lzsi4ARAAbGnNV0QyL/7itzy827+chg== 2023-10-06T02:22:37.138 INFO:teuthology.orchestra.run.smithi062.stdout:[mgr.x] 2023-10-06T02:22:37.139 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQBLbx9lTBbIFBAAX2wsioKL4Nx4qz/qwsCsNA== 2023-10-06T02:22:37.139 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.b] 2023-10-06T02:22:37.139 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQBLbx9lFgsTMRAATTklzroiyQbTLiZc9j1BHQ== 2023-10-06T02:22:37.139 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.d] 2023-10-06T02:22:37.139 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQBMbx9l74gWABAApBu2nVv12rGA+OBfBnEc3A== 2023-10-06T02:22:37.139 INFO:teuthology.orchestra.run.smithi062.stdout:[osd.4] 2023-10-06T02:22:37.139 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQBfbx9lW4OSEhAAj7HnVh/yq12zzCaXd6BHCA== 2023-10-06T02:22:37.139 INFO:teuthology.orchestra.run.smithi062.stdout:[osd.5] 2023-10-06T02:22:37.139 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQBibx9lBcaIFRAAV8wq8QjsCOkI9Pzf/E+9bA== 2023-10-06T02:22:37.139 INFO:teuthology.orchestra.run.smithi062.stdout:[osd.6] 2023-10-06T02:22:37.139 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQBlbx9lA7S+FxAA38mlSAurT+DpwMzH53zFcw== 2023-10-06T02:22:37.139 INFO:teuthology.orchestra.run.smithi062.stdout:[osd.7] 2023-10-06T02:22:37.139 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQBobx9l5RkdHhAAxn0Z8IAE/o9oe2LRp3O9gg== 2023-10-06T02:22:37.140 INFO:teuthology.orchestra.run.smithi062.stdout:[client.0] 2023-10-06T02:22:37.140 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQBMbx9lh44fCRAA7sMISvk1t7dEJg4ZcBiMcA== 2023-10-06T02:22:37.140 INFO:teuthology.orchestra.run.smithi062.stdout:[client.2] 2023-10-06T02:22:37.140 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQBMbx9lpxYAEBAACtiI1XgQXM8yGGcpFyNK1w== 2023-10-06T02:22:37.140 INFO:teuthology.orchestra.run.smithi062.stdout:[client.1] 2023-10-06T02:22:37.140 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQBMbx9lP0zqFhAAlov2y7ipSvS3AkOdQkhg6g== 2023-10-06T02:22:37.140 INFO:teuthology.orchestra.run.smithi062.stdout:[client.3] 2023-10-06T02:22:37.140 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQBMbx9l81/PHRAAYFOoaezu0pwE0NdtnlYfVA== 2023-10-06T02:22:37.141 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-06T02:22:37.147 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-06T02:22:37.278 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T02:22:37.282 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T02:22:37.381 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T02:22:37.386 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T02:22:37.484 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T02:22:37.489 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T02:22:37.584 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T02:22:37.588 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T02:22:37.684 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T02:22:37.689 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T02:22:37.785 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T02:22:37.790 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T02:22:37.886 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-06T02:22:37.891 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-06T02:22:37.987 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T02:22:37.991 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T02:22:38.090 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T02:22:38.095 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T02:22:38.194 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T02:22:38.198 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T02:22:38.294 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T02:22:38.299 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T02:22:38.395 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T02:22:38.399 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T02:22:38.495 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T02:22:38.499 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T02:22:38.595 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T02:22:38.599 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T02:22:38.699 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T02:22:38.704 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T02:22:38.803 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T02:22:38.807 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T02:22:38.906 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T02:22:38.910 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T02:22:39.009 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-10-06T02:22:39.010 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-10-06T02:22:39.071 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-06T02:22:39.182 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-10-06T02:22:39.244 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-10-06T02:22:39.306 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-06T02:22:39.418 DEBUG:teuthology.orchestra.run.smithi062:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-10-06T02:22:39.481 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-10-06T02:22:39.543 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-06T02:22:39.652 DEBUG:teuthology.orchestra.run.smithi062:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-10-06T02:22:39.713 DEBUG:teuthology.orchestra.run.smithi031:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-06T02:22:39.718 DEBUG:teuthology.orchestra.run.smithi062:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-06T02:22:39.754 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-10-06T02:22:39.754 INFO:tasks.ceph.mon.a:Restarting daemon 2023-10-06T02:22:39.754 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-10-06T02:22:39.795 INFO:tasks.ceph.mon.a:Started 2023-10-06T02:22:39.795 INFO:tasks.ceph.mon.b:Restarting daemon 2023-10-06T02:22:39.795 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-10-06T02:22:39.800 INFO:tasks.ceph.mon.b:Started 2023-10-06T02:22:39.800 INFO:tasks.ceph.mon.c:Restarting daemon 2023-10-06T02:22:39.800 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-10-06T02:22:39.804 INFO:tasks.ceph.mon.c:Started 2023-10-06T02:22:39.804 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-10-06T02:22:39.804 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-10-06T02:22:39.804 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-10-06T02:22:39.808 INFO:tasks.ceph.mgr.y:Started 2023-10-06T02:22:39.808 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-10-06T02:22:39.809 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-10-06T02:22:39.812 INFO:tasks.ceph.mgr.x:Started 2023-10-06T02:22:39.813 INFO:tasks.ceph:Setting crush tunables to default 2023-10-06T02:22:39.813 DEBUG:teuthology.orchestra.run.smithi031:> sudo ceph --cluster ceph osd crush tunables default 2023-10-06T02:22:40.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:40.000+0000 7fcdf299d700 1 Processor -- start 2023-10-06T02:22:40.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:40.000+0000 7fcdf299d700 1 -- start start 2023-10-06T02:22:40.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:40.002+0000 7fcdf299d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fcdec153a00 0x7fcdec153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:22:40.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:40.002+0000 7fcdf299d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fcdec15de50 0x7fcdec15a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:22:40.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:40.002+0000 7fcdf299d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdec15af60 con 0x7fcdec0cea00 2023-10-06T02:22:40.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:40.002+0000 7fcdf299d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdec15d010 con 0x7fcdec15de50 2023-10-06T02:22:40.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:40.002+0000 7fcdf299d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdec15b240 con 0x7fcdec153a00 2023-10-06T02:22:40.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:40.002+0000 7fcdf119a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fcdec15de50 0x7fcdec15a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:22:40.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:40.002+0000 7fcdf119a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fcdec15de50 0x7fcdec15a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57598/0 (socket says 172.21.15.31:57598) 2023-10-06T02:22:40.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:40.002+0000 7fcdf119a700 1 -- 172.21.15.31:0/3572145167 learned_addr learned my addr 172.21.15.31:0/3572145167 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:22:40.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:40.002+0000 7fcdf199b700 1 --2- 172.21.15.31:0/3572145167 >> v2:172.21.15.62:3300/0 conn(0x7fcdec153a00 0x7fcdec153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:22:40.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:40.002+0000 7fcdf119a700 1 -- 172.21.15.31:0/3572145167 >> v2:172.21.15.31:3300/0 conn(0x7fcdec15de50 msgr2=0x7fcdec15a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-06T02:22:40.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:40.002+0000 7fcdf119a700 1 -- 172.21.15.31:0/3572145167 >> v2:172.21.15.31:3300/0 conn(0x7fcdec15de50 msgr2=0x7fcdec15a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T02:22:40.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:40.002+0000 7fcdf119a700 1 --2- 172.21.15.31:0/3572145167 >> v2:172.21.15.31:3300/0 conn(0x7fcdec15de50 0x7fcdec15a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T02:22:40.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:40.002+0000 7fcdf119a700 1 --2- 172.21.15.31:0/3572145167 >> v2:172.21.15.31:3300/0 conn(0x7fcdec15de50 0x7fcdec15a540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T02:22:40.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:40.002+0000 7fcdf199b700 1 -- 172.21.15.31:0/3572145167 >> v2:172.21.15.62:3300/0 conn(0x7fcdec153a00 msgr2=0x7fcdec153dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-06T02:22:40.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:40.002+0000 7fcdf199b700 1 -- 172.21.15.31:0/3572145167 >> v2:172.21.15.62:3300/0 conn(0x7fcdec153a00 msgr2=0x7fcdec153dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T02:22:40.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:40.002+0000 7fcdf199b700 1 --2- 172.21.15.31:0/3572145167 >> v2:172.21.15.62:3300/0 conn(0x7fcdec153a00 0x7fcdec153dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T02:22:40.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:40.002+0000 7fcdf199b700 1 --2- 172.21.15.31:0/3572145167 >> v2:172.21.15.62:3300/0 conn(0x7fcdec153a00 0x7fcdec153dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T02:22:40.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:40.203+0000 7fcdf119a700 1 --2- 172.21.15.31:0/3572145167 >> v2:172.21.15.31:3300/0 conn(0x7fcdec15de50 0x7fcdec15a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:22:40.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:40.203+0000 7fcdf199b700 1 --2- 172.21.15.31:0/3572145167 >> v2:172.21.15.62:3300/0 conn(0x7fcdec153a00 0x7fcdec153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:22:40.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:40.203+0000 7fcdf119a700 1 -- 172.21.15.31:0/3572145167 >> v2:172.21.15.62:3300/0 conn(0x7fcdec153a00 msgr2=0x7fcdec153dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:22:40.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:40.203+0000 7fcdf119a700 1 --2- 172.21.15.31:0/3572145167 >> v2:172.21.15.62:3300/0 conn(0x7fcdec153a00 0x7fcdec153dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:40.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:40.203+0000 7fcdf119a700 1 -- 172.21.15.31:0/3572145167 >> v1:172.21.15.31:6789/0 conn(0x7fcdec0cea00 legacy=0x7fcdec0cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:22:40.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:40.204+0000 7fcdf119a700 1 -- 172.21.15.31:0/3572145167 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdec15b520 con 0x7fcdec15de50 2023-10-06T02:22:40.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:40.204+0000 7fcdf119a700 1 --2- 172.21.15.31:0/3572145167 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcdec15de50 0x7fcdec15a540 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fcdec0cac20 tx=0x7fcddc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4858858a35ed88b2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:22:40.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:40.204+0000 7fcdf0999700 1 -- 172.21.15.31:0/3572145167 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcddc00e480 con 0x7fcdec15de50 2023-10-06T02:22:40.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:40.204+0000 7fcdf0999700 1 -- 172.21.15.31:0/3572145167 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcddc00e5e0 con 0x7fcdec15de50 2023-10-06T02:22:40.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:40.204+0000 7fcdf0999700 1 -- 172.21.15.31:0/3572145167 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcddc00e9a0 con 0x7fcdec15de50 2023-10-06T02:22:40.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:40.205+0000 7fcdf299d700 1 -- 172.21.15.31:0/3572145167 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcdec15de50 msgr2=0x7fcdec15a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:22:40.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:40.205+0000 7fcdf299d700 1 --2- 172.21.15.31:0/3572145167 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcdec15de50 0x7fcdec15a540 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fcdec0cac20 tx=0x7fcddc008040 comp rx=0 tx=0).stop 2023-10-06T02:22:40.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:40.205+0000 7fcdf299d700 1 -- 172.21.15.31:0/3572145167 shutdown_connections 2023-10-06T02:22:40.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:40.205+0000 7fcdf299d700 1 --2- 172.21.15.31:0/3572145167 >> v2:172.21.15.62:3300/0 conn(0x7fcdec153a00 0x7fcdec153dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:40.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:40.205+0000 7fcdf299d700 1 --2- 172.21.15.31:0/3572145167 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcdec15de50 0x7fcdec15a540 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:40.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:40.205+0000 7fcdf299d700 1 -- 172.21.15.31:0/3572145167 >> 172.21.15.31:0/3572145167 conn(0x7fcdec0703a0 msgr2=0x7fcdec0707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:22:40.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:40.205+0000 7fcdf299d700 1 -- 172.21.15.31:0/3572145167 shutdown_connections 2023-10-06T02:22:40.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:40.205+0000 7fcdf299d700 1 -- 172.21.15.31:0/3572145167 wait complete. 2023-10-06T02:22:40.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:40.207+0000 7fcdf299d700 1 Processor -- start 2023-10-06T02:22:40.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:40.207+0000 7fcdf299d700 1 -- start start 2023-10-06T02:22:40.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:40.208+0000 7fcdf299d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcdec0cea00 0x7fcdec0c6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:22:40.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:40.208+0000 7fcdf299d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcdec153a00 0x7fcdec157f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:22:40.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:40.208+0000 7fcdf299d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcdec15de50 0x7fcdec0c52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:22:40.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:40.208+0000 7fcdf299d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdec0c7220 con 0x7fcdec0cea00 2023-10-06T02:22:40.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:40.208+0000 7fcdf299d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdec0c6850 con 0x7fcdec15de50 2023-10-06T02:22:40.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:40.208+0000 7fcdf299d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdec1584b0 con 0x7fcdec153a00 2023-10-06T02:22:40.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:40.208+0000 7fcdf119a700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcdec153a00 0x7fcdec157f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:22:40.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:40.208+0000 7fcdf119a700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcdec153a00 0x7fcdec157f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:49620/0 (socket says 172.21.15.31:49620) 2023-10-06T02:22:40.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:40.208+0000 7fcdf119a700 1 -- 172.21.15.31:0/1704292838 learned_addr learned my addr 172.21.15.31:0/1704292838 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:22:40.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:40.208+0000 7fcdf219c700 1 --2- 172.21.15.31:0/1704292838 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcdec15de50 0x7fcdec0c52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:22:40.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:40.209+0000 7fcdf199b700 1 --2- 172.21.15.31:0/1704292838 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcdec0cea00 0x7fcdec0c6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:22:40.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:40.209+0000 7fcdf219c700 1 -- 172.21.15.31:0/1704292838 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcdec153a00 msgr2=0x7fcdec157f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:22:40.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:40.209+0000 7fcdf219c700 1 --2- 172.21.15.31:0/1704292838 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcdec153a00 0x7fcdec157f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:40.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:40.209+0000 7fcdf219c700 1 -- 172.21.15.31:0/1704292838 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcdec0cea00 msgr2=0x7fcdec0c6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:22:40.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:40.209+0000 7fcdf219c700 1 --2- 172.21.15.31:0/1704292838 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcdec0cea00 0x7fcdec0c6ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:40.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:40.209+0000 7fcdf219c700 1 -- 172.21.15.31:0/1704292838 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcddc009570 con 0x7fcdec15de50 2023-10-06T02:22:40.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:40.209+0000 7fcdf219c700 1 --2- 172.21.15.31:0/1704292838 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcdec15de50 0x7fcdec0c52d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcde400c790 tx=0x7fcde40015f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:22:40.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:40.210+0000 7fcde1ffb700 1 -- 172.21.15.31:0/1704292838 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde4012630 con 0x7fcdec15de50 2023-10-06T02:22:40.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:40.210+0000 7fcde1ffb700 1 -- 172.21.15.31:0/1704292838 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcde40127b0 con 0x7fcdec15de50 2023-10-06T02:22:40.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:40.210+0000 7fcde1ffb700 1 -- 172.21.15.31:0/1704292838 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde4012b70 con 0x7fcdec15de50 2023-10-06T02:22:40.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:40.210+0000 7fcdf299d700 1 -- 172.21.15.31:0/1704292838 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdec0c5c40 con 0x7fcdec15de50 2023-10-06T02:22:40.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:40.210+0000 7fcdf299d700 1 -- 172.21.15.31:0/1704292838 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdec0c6200 con 0x7fcdec15de50 2023-10-06T02:22:40.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:40.210+0000 7fcde1ffb700 1 -- 172.21.15.31:0/1704292838 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fcde4003030 con 0x7fcdec15de50 2023-10-06T02:22:40.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:40.211+0000 7fcde1ffb700 1 -- 172.21.15.31:0/1704292838 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fcde4011380 con 0x7fcdec15de50 2023-10-06T02:22:40.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:40.212+0000 7fcdf299d700 1 -- 172.21.15.31:0/1704292838 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdec04c3c0 con 0x7fcdec15de50 2023-10-06T02:22:40.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:40.215+0000 7fcde1ffb700 1 -- 172.21.15.31:0/1704292838 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fcde401b400 con 0x7fcdec15de50 2023-10-06T02:22:40.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:40.295+0000 7fcdf299d700 1 -- 172.21.15.31:0/1704292838 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fcdec0cf4e0 con 0x7fcdec15de50 2023-10-06T02:22:40.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:40.304+0000 7fcde1ffb700 1 -- 172.21.15.31:0/1704292838 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7fcde401b5a0 con 0x7fcdec15de50 2023-10-06T02:22:40.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:40.305+0000 7fcdf299d700 1 -- 172.21.15.31:0/1704292838 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcdec15de50 msgr2=0x7fcdec0c52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:22:40.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:40.305+0000 7fcdf299d700 1 --2- 172.21.15.31:0/1704292838 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcdec15de50 0x7fcdec0c52d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcde400c790 tx=0x7fcde40015f0 comp rx=0 tx=0).stop 2023-10-06T02:22:40.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:40.305+0000 7fcdf299d700 1 -- 172.21.15.31:0/1704292838 shutdown_connections 2023-10-06T02:22:40.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:40.305+0000 7fcdf299d700 1 --2- 172.21.15.31:0/1704292838 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcdec15de50 0x7fcdec0c52d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:40.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:40.305+0000 7fcdf299d700 1 --2- 172.21.15.31:0/1704292838 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcdec0cea00 0x7fcdec0c6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:40.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:40.305+0000 7fcdf299d700 1 --2- 172.21.15.31:0/1704292838 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcdec153a00 0x7fcdec157f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:40.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:40.305+0000 7fcdf299d700 1 -- 172.21.15.31:0/1704292838 >> 172.21.15.31:0/1704292838 conn(0x7fcdec0703a0 msgr2=0x7fcdec14ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:22:40.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:40.305+0000 7fcdf299d700 1 -- 172.21.15.31:0/1704292838 shutdown_connections 2023-10-06T02:22:40.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:40.305+0000 7fcdf299d700 1 -- 172.21.15.31:0/1704292838 wait complete. 2023-10-06T02:22:40.306 INFO:teuthology.orchestra.run.smithi031.stderr:adjusted tunables profile to default 2023-10-06T02:22:40.323 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-10-06T02:22:40.323 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T02:22:40.323 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-10-06T02:22:40.385 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T02:22:40.385 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-10-06T02:22:40.488 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T02:22:40.489 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-10-06T02:22:40.592 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T02:22:40.592 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-10-06T02:22:40.695 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-10-06T02:22:40.695 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-10-06T02:22:40.757 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-10-06T02:22:40.757 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-10-06T02:22:40.859 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-10-06T02:22:40.860 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-10-06T02:22:40.962 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-10-06T02:22:40.962 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-10-06T02:22:41.065 DEBUG:teuthology.orchestra.run.smithi062:> sudo ceph --cluster ceph osd new e9b19574-d60f-480a-bb2d-ab0672dcbccf 0 2023-10-06T02:22:41.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.285+0000 7efcbab9d700 1 Processor -- start 2023-10-06T02:22:41.287 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.286+0000 7efcbab9d700 1 -- start start 2023-10-06T02:22:41.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.287+0000 7efcbab9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7efcb4153dd0 0x7efcb415de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:22:41.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.287+0000 7efcbab9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcb415ac50 con 0x7efcb4153a00 2023-10-06T02:22:41.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.287+0000 7efcbab9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcb415d010 con 0x7efcb40cea00 2023-10-06T02:22:41.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.287+0000 7efcbab9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcb415af60 con 0x7efcb4153dd0 2023-10-06T02:22:41.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.287+0000 7efcb939a700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7efcb4153dd0 0x7efcb415de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:22:41.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.287+0000 7efcb939a700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7efcb4153dd0 0x7efcb415de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49938/0 (socket says 172.21.15.62:49938) 2023-10-06T02:22:41.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.287+0000 7efcb939a700 1 -- 172.21.15.62:0/1887822670 learned_addr learned my addr 172.21.15.62:0/1887822670 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-10-06T02:22:41.289 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.287+0000 7efcba39c700 1 --1- 172.21.15.62:0/1887822670 >> v1:172.21.15.62:6789/0 conn(0x7efcb40cea00 0x7efcb40ceea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:48254/0 (socket says 172.21.15.62:48254) 2023-10-06T02:22:41.290 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.288+0000 7efcb939a700 1 -- 172.21.15.62:0/1887822670 >> v1:172.21.15.62:6789/0 conn(0x7efcb40cea00 legacy=0x7efcb40ceea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:22:41.290 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.288+0000 7efcb8b99700 1 -- 172.21.15.62:0/1887822670 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3503692075 0 0) 0x7efcb415ac50 con 0x7efcb4153a00 2023-10-06T02:22:41.290 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.288+0000 7efcb939a700 1 -- 172.21.15.62:0/1887822670 >> v1:172.21.15.31:6789/0 conn(0x7efcb4153a00 legacy=0x7efcb415a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:22:41.290 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.288+0000 7efcb939a700 1 -- 172.21.15.62:0/1887822670 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcb415b240 con 0x7efcb4153dd0 2023-10-06T02:22:41.290 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.288+0000 7efcb939a700 1 --2- 172.21.15.62:0/1887822670 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efcb4153dd0 0x7efcb415de50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7efca40099b0 tx=0x7efca4009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4991c11fd9f68550 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:22:41.290 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.288+0000 7efcb8b99700 1 -- 172.21.15.62:0/1887822670 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca400b860 con 0x7efcb4153dd0 2023-10-06T02:22:41.290 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.288+0000 7efcb8b99700 1 -- 172.21.15.62:0/1887822670 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efca400b9c0 con 0x7efcb4153dd0 2023-10-06T02:22:41.290 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.288+0000 7efcb8b99700 1 -- 172.21.15.62:0/1887822670 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca400bd60 con 0x7efcb4153dd0 2023-10-06T02:22:41.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.289+0000 7efcbab9d700 1 -- 172.21.15.62:0/1887822670 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efcb4153dd0 msgr2=0x7efcb415de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:22:41.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.289+0000 7efcbab9d700 1 --2- 172.21.15.62:0/1887822670 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efcb4153dd0 0x7efcb415de50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7efca40099b0 tx=0x7efca4009250 comp rx=0 tx=0).stop 2023-10-06T02:22:41.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.289+0000 7efcbab9d700 1 -- 172.21.15.62:0/1887822670 shutdown_connections 2023-10-06T02:22:41.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.289+0000 7efcbab9d700 1 --2- 172.21.15.62:0/1887822670 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efcb4153dd0 0x7efcb415de50 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:41.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.289+0000 7efcbab9d700 1 -- 172.21.15.62:0/1887822670 >> 172.21.15.62:0/1887822670 conn(0x7efcb40703a0 msgr2=0x7efcb40707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:22:41.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.289+0000 7efcbab9d700 1 -- 172.21.15.62:0/1887822670 shutdown_connections 2023-10-06T02:22:41.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.289+0000 7efcbab9d700 1 -- 172.21.15.62:0/1887822670 wait complete. 2023-10-06T02:22:41.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.290+0000 7efcbab9d700 1 Processor -- start 2023-10-06T02:22:41.292 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.290+0000 7efcbab9d700 1 -- start start 2023-10-06T02:22:41.293 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.291+0000 7efcbab9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efcb40cea00 0x7efcb415c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:22:41.293 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.291+0000 7efcbab9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efcb4153a00 0x7efcb40c80f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:22:41.293 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.291+0000 7efcbab9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efcb4153dd0 0x7efcb40c8630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:22:41.293 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.291+0000 7efcbab9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcb40c8b70 con 0x7efcb4153dd0 2023-10-06T02:22:41.293 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.291+0000 7efcbab9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcb40c9600 con 0x7efcb4153a00 2023-10-06T02:22:41.293 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.291+0000 7efcbab9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcb4151cb0 con 0x7efcb40cea00 2023-10-06T02:22:41.293 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.292+0000 7efcb9b9b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efcb40cea00 0x7efcb415c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:22:41.294 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.292+0000 7efcb939a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efcb4153a00 0x7efcb40c80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:22:41.294 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.292+0000 7efcb939a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efcb4153a00 0x7efcb40c80f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49954/0 (socket says 172.21.15.62:49954) 2023-10-06T02:22:41.294 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.292+0000 7efcb939a700 1 -- 172.21.15.62:0/1349600333 learned_addr learned my addr 172.21.15.62:0/1349600333 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-10-06T02:22:41.294 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.292+0000 7efcba39c700 1 --2- 172.21.15.62:0/1349600333 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efcb4153dd0 0x7efcb40c8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:22:41.294 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.292+0000 7efcb939a700 1 -- 172.21.15.62:0/1349600333 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efcb40cea00 msgr2=0x7efcb415c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:22:41.294 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.292+0000 7efcb939a700 1 --2- 172.21.15.62:0/1349600333 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efcb40cea00 0x7efcb415c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:41.294 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.292+0000 7efcb939a700 1 -- 172.21.15.62:0/1349600333 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efcb4153dd0 msgr2=0x7efcb40c8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:22:41.294 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.292+0000 7efcb939a700 1 --2- 172.21.15.62:0/1349600333 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efcb4153dd0 0x7efcb40c8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:41.294 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.292+0000 7efcb939a700 1 -- 172.21.15.62:0/1349600333 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efca40094f0 con 0x7efcb4153a00 2023-10-06T02:22:41.294 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.292+0000 7efcb939a700 1 --2- 172.21.15.62:0/1349600333 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efcb4153a00 0x7efcb40c80f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7efca4006f50 tx=0x7efca4017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:22:41.295 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.293+0000 7efca9ffb700 1 -- 172.21.15.62:0/1349600333 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca4011e40 con 0x7efcb4153a00 2023-10-06T02:22:41.295 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.293+0000 7efca9ffb700 1 -- 172.21.15.62:0/1349600333 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efca4024440 con 0x7efcb4153a00 2023-10-06T02:22:41.295 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.293+0000 7efca9ffb700 1 -- 172.21.15.62:0/1349600333 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca4024800 con 0x7efcb4153a00 2023-10-06T02:22:41.295 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.293+0000 7efcbab9d700 1 -- 172.21.15.62:0/1349600333 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcb40c8d60 con 0x7efcb4153a00 2023-10-06T02:22:41.296 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.293+0000 7efcbab9d700 1 -- 172.21.15.62:0/1349600333 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcb40c9320 con 0x7efcb4153a00 2023-10-06T02:22:41.296 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.293+0000 7efca9ffb700 1 -- 172.21.15.62:0/1349600333 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7efca401a420 con 0x7efcb4153a00 2023-10-06T02:22:41.296 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.293+0000 7efca9ffb700 1 -- 172.21.15.62:0/1349600333 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 721+0+0 (secure 0 0 0) 0x7efca402d740 con 0x7efcb4153a00 2023-10-06T02:22:41.296 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.294+0000 7efcbab9d700 1 -- 172.21.15.62:0/1349600333 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcb404c3c0 con 0x7efcb4153a00 2023-10-06T02:22:41.297 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.296+0000 7efca9ffb700 1 -- 172.21.15.62:0/1349600333 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7efca4015020 con 0x7efcb4153a00 2023-10-06T02:22:41.384 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.383+0000 7efcbab9d700 1 -- 172.21.15.62:0/1349600333 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd new", "uuid": "e9b19574-d60f-480a-bb2d-ab0672dcbccf", "id": 0} v 0) v1 -- 0x7efcb415b520 con 0x7efcb4153a00 2023-10-06T02:22:41.392 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.391+0000 7efca9ffb700 1 -- 172.21.15.62:0/1349600333 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e9b19574-d60f-480a-bb2d-ab0672dcbccf", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7efca401a8b0 con 0x7efcb4153a00 2023-10-06T02:22:41.392 INFO:teuthology.orchestra.run.smithi062.stdout:0 2023-10-06T02:22:41.393 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.392+0000 7efcbab9d700 1 -- 172.21.15.62:0/1349600333 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efcb4153a00 msgr2=0x7efcb40c80f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:22:41.393 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.392+0000 7efcbab9d700 1 --2- 172.21.15.62:0/1349600333 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efcb4153a00 0x7efcb40c80f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7efca4006f50 tx=0x7efca4017000 comp rx=0 tx=0).stop 2023-10-06T02:22:41.394 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.392+0000 7efcbab9d700 1 -- 172.21.15.62:0/1349600333 shutdown_connections 2023-10-06T02:22:41.394 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.392+0000 7efcbab9d700 1 --2- 172.21.15.62:0/1349600333 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efcb4153dd0 0x7efcb40c8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:41.394 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.392+0000 7efcbab9d700 1 --2- 172.21.15.62:0/1349600333 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efcb40cea00 0x7efcb415c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:41.394 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.392+0000 7efcbab9d700 1 --2- 172.21.15.62:0/1349600333 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efcb4153a00 0x7efcb40c80f0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:41.394 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.392+0000 7efcbab9d700 1 -- 172.21.15.62:0/1349600333 >> 172.21.15.62:0/1349600333 conn(0x7efcb40703a0 msgr2=0x7efcb40ceea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:22:41.394 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.392+0000 7efcbab9d700 1 -- 172.21.15.62:0/1349600333 shutdown_connections 2023-10-06T02:22:41.394 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.392+0000 7efcbab9d700 1 -- 172.21.15.62:0/1349600333 wait complete. 2023-10-06T02:22:41.410 DEBUG:teuthology.orchestra.run.smithi062:> sudo ceph --cluster ceph osd new 5ac13881-4851-43f8-bfab-c9b162300451 1 2023-10-06T02:22:41.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.590+0000 7fb3f09fe700 1 Processor -- start 2023-10-06T02:22:41.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.591+0000 7fb3f09fe700 1 -- start start 2023-10-06T02:22:41.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.592+0000 7fb3f09fe700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb3ec153a00 0x7fb3ec153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:22:41.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.592+0000 7fb3f09fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb3ec0cea00 0x7fb3ec0cee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:22:41.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.592+0000 7fb3f09fe700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3ec15ac50 con 0x7fb3ec15de50 2023-10-06T02:22:41.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.592+0000 7fb3f09fe700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3ec15d010 con 0x7fb3ec0cea00 2023-10-06T02:22:41.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.592+0000 7fb3f09fe700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3ec0cf3c0 con 0x7fb3ec153a00 2023-10-06T02:22:41.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.592+0000 7fb3ead9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb3ec153a00 0x7fb3ec153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:22:41.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.592+0000 7fb3ea59c700 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fb3ec15de50 0x7fb3ec15a540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:48268/0 (socket says 172.21.15.62:48268) 2023-10-06T02:22:41.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.592+0000 7fb3ead9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb3ec153a00 0x7fb3ec153dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49966/0 (socket says 172.21.15.62:49966) 2023-10-06T02:22:41.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.592+0000 7fb3ea59c700 1 -- 172.21.15.62:0/3246142308 learned_addr learned my addr 172.21.15.62:0/3246142308 (peer_addr_for_me v1:172.21.15.62:0/0) 2023-10-06T02:22:41.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.592+0000 7fb3eb59e700 1 --2- 172.21.15.62:0/3246142308 >> v2:172.21.15.31:3300/0 conn(0x7fb3ec0cea00 0x7fb3ec0cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:22:41.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.593+0000 7fb3ead9d700 1 -- 172.21.15.62:0/3246142308 >> v2:172.21.15.31:3300/0 conn(0x7fb3ec0cea00 msgr2=0x7fb3ec0cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:22:41.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.593+0000 7fb3ead9d700 1 --2- 172.21.15.62:0/3246142308 >> v2:172.21.15.31:3300/0 conn(0x7fb3ec0cea00 0x7fb3ec0cee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:41.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.593+0000 7fb3ead9d700 1 -- 172.21.15.62:0/3246142308 >> v1:172.21.15.62:6789/0 conn(0x7fb3ec15de50 legacy=0x7fb3ec15a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:22:41.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.593+0000 7fb3ead9d700 1 -- 172.21.15.62:0/3246142308 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3ec15af60 con 0x7fb3ec153a00 2023-10-06T02:22:41.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.593+0000 7fb3eb59e700 1 --2- 172.21.15.62:0/3246142308 >> v2:172.21.15.31:3300/0 conn(0x7fb3ec0cea00 0x7fb3ec0cee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:22:41.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.593+0000 7fb3ead9d700 1 --2- 172.21.15.62:0/3246142308 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3ec153a00 0x7fb3ec153dd0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec0cac20 tx=0x7fb3dc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6bdf16acd718cbe7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:22:41.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.593+0000 7fb3e9d9b700 1 -- 172.21.15.62:0/3246142308 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3dc00b6b0 con 0x7fb3ec153a00 2023-10-06T02:22:41.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.593+0000 7fb3e9d9b700 1 -- 172.21.15.62:0/3246142308 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3dc00b810 con 0x7fb3ec153a00 2023-10-06T02:22:41.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.593+0000 7fb3e9d9b700 1 -- 172.21.15.62:0/3246142308 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3dc00bbb0 con 0x7fb3ec153a00 2023-10-06T02:22:41.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.594+0000 7fb3f09fe700 1 -- 172.21.15.62:0/3246142308 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3ec153a00 msgr2=0x7fb3ec153dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:22:41.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.594+0000 7fb3f09fe700 1 --2- 172.21.15.62:0/3246142308 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3ec153a00 0x7fb3ec153dd0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec0cac20 tx=0x7fb3dc0091a0 comp rx=0 tx=0).stop 2023-10-06T02:22:41.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.594+0000 7fb3f09fe700 1 -- 172.21.15.62:0/3246142308 shutdown_connections 2023-10-06T02:22:41.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.594+0000 7fb3f09fe700 1 --2- 172.21.15.62:0/3246142308 >> v2:172.21.15.31:3300/0 conn(0x7fb3ec0cea00 0x7fb3ec0cee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:41.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.594+0000 7fb3f09fe700 1 --2- 172.21.15.62:0/3246142308 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3ec153a00 0x7fb3ec153dd0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:41.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.594+0000 7fb3f09fe700 1 -- 172.21.15.62:0/3246142308 >> 172.21.15.62:0/3246142308 conn(0x7fb3ec0703a0 msgr2=0x7fb3ec0707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:22:41.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.594+0000 7fb3f09fe700 1 -- 172.21.15.62:0/3246142308 shutdown_connections 2023-10-06T02:22:41.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.594+0000 7fb3f09fe700 1 -- 172.21.15.62:0/3246142308 wait complete. 2023-10-06T02:22:41.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.596+0000 7fb3f09fe700 1 Processor -- start 2023-10-06T02:22:41.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.596+0000 7fb3f09fe700 1 -- start start 2023-10-06T02:22:41.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.597+0000 7fb3f09fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb3ec0cea00 0x7fb3ec156560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:22:41.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.597+0000 7fb3f09fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3ec153a00 0x7fb3ec152d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:22:41.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.597+0000 7fb3f09fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3ec15de50 0x7fb3ec151140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:22:41.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.597+0000 7fb3f09fe700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3ec0cda90 con 0x7fb3ec15de50 2023-10-06T02:22:41.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.597+0000 7fb3f09fe700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3ec16e0c0 con 0x7fb3ec153a00 2023-10-06T02:22:41.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.597+0000 7fb3f09fe700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3ec156aa0 con 0x7fb3ec0cea00 2023-10-06T02:22:41.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.597+0000 7fb3ea59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3ec153a00 0x7fb3ec152d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:22:41.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.597+0000 7fb3ead9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb3ec0cea00 0x7fb3ec156560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:22:41.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.597+0000 7fb3ea59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3ec153a00 0x7fb3ec152d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49978/0 (socket says 172.21.15.62:49978) 2023-10-06T02:22:41.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.597+0000 7fb3ea59c700 1 -- 172.21.15.62:0/1738804626 learned_addr learned my addr 172.21.15.62:0/1738804626 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-10-06T02:22:41.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.597+0000 7fb3eb59e700 1 --2- 172.21.15.62:0/1738804626 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3ec15de50 0x7fb3ec151140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:22:41.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.597+0000 7fb3ead9d700 1 -- 172.21.15.62:0/1738804626 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3ec153a00 msgr2=0x7fb3ec152d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:22:41.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.597+0000 7fb3ead9d700 1 --2- 172.21.15.62:0/1738804626 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3ec153a00 0x7fb3ec152d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:41.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.597+0000 7fb3ead9d700 1 -- 172.21.15.62:0/1738804626 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3ec15de50 msgr2=0x7fb3ec151140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:22:41.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.597+0000 7fb3ead9d700 1 --2- 172.21.15.62:0/1738804626 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3ec15de50 0x7fb3ec151140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:41.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.597+0000 7fb3ead9d700 1 -- 172.21.15.62:0/1738804626 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3dc009420 con 0x7fb3ec0cea00 2023-10-06T02:22:41.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.597+0000 7fb3ea59c700 1 --2- 172.21.15.62:0/1738804626 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3ec153a00 0x7fb3ec152d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:22:41.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.598+0000 7fb3ead9d700 1 --2- 172.21.15.62:0/1738804626 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb3ec0cea00 0x7fb3ec156560 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc016000 tx=0x7fb3dc017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:22:41.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.598+0000 7fb3d2ffd700 1 -- 172.21.15.62:0/1738804626 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3dc000d10 con 0x7fb3ec0cea00 2023-10-06T02:22:41.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.598+0000 7fb3d2ffd700 1 -- 172.21.15.62:0/1738804626 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3dc00bb90 con 0x7fb3ec0cea00 2023-10-06T02:22:41.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.598+0000 7fb3f09fe700 1 -- 172.21.15.62:0/1738804626 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3ec16d300 con 0x7fb3ec0cea00 2023-10-06T02:22:41.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.598+0000 7fb3d2ffd700 1 -- 172.21.15.62:0/1738804626 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3dc011650 con 0x7fb3ec0cea00 2023-10-06T02:22:41.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.598+0000 7fb3f09fe700 1 -- 172.21.15.62:0/1738804626 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3ec16d8c0 con 0x7fb3ec0cea00 2023-10-06T02:22:41.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.598+0000 7fb3d2ffd700 1 -- 172.21.15.62:0/1738804626 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fb3dc011c80 con 0x7fb3ec0cea00 2023-10-06T02:22:41.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.599+0000 7fb3d2ffd700 1 -- 172.21.15.62:0/1738804626 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 836+0+0 (secure 0 0 0) 0x7fb3dc02eda0 con 0x7fb3ec0cea00 2023-10-06T02:22:41.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.599+0000 7fb3f09fe700 1 -- 172.21.15.62:0/1738804626 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3ec04c3c0 con 0x7fb3ec0cea00 2023-10-06T02:22:41.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.602+0000 7fb3d2ffd700 1 -- 172.21.15.62:0/1738804626 <== mon.2 v2:172.21.15.62:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fb3dc015020 con 0x7fb3ec0cea00 2023-10-06T02:22:41.649 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:22:41.649+0000 7f3f028b9180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T02:22:41.690 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.689+0000 7fb3f09fe700 1 -- 172.21.15.62:0/1738804626 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "osd new", "uuid": "5ac13881-4851-43f8-bfab-c9b162300451", "id": 1} v 0) v1 -- 0x7fb3ec0c5a30 con 0x7fb3ec0cea00 2023-10-06T02:22:41.696 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.695+0000 7fb3d2ffd700 1 -- 172.21.15.62:0/1738804626 <== mon.2 v2:172.21.15.62:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5ac13881-4851-43f8-bfab-c9b162300451", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fb3dc02e400 con 0x7fb3ec0cea00 2023-10-06T02:22:41.696 INFO:teuthology.orchestra.run.smithi062.stdout:1 2023-10-06T02:22:41.697 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.696+0000 7fb3f09fe700 1 -- 172.21.15.62:0/1738804626 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb3ec0cea00 msgr2=0x7fb3ec156560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:22:41.697 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.696+0000 7fb3f09fe700 1 --2- 172.21.15.62:0/1738804626 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb3ec0cea00 0x7fb3ec156560 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc016000 tx=0x7fb3dc017000 comp rx=0 tx=0).stop 2023-10-06T02:22:41.697 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.696+0000 7fb3f09fe700 1 -- 172.21.15.62:0/1738804626 shutdown_connections 2023-10-06T02:22:41.697 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.696+0000 7fb3f09fe700 1 --2- 172.21.15.62:0/1738804626 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3ec15de50 0x7fb3ec151140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:41.697 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.696+0000 7fb3f09fe700 1 --2- 172.21.15.62:0/1738804626 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb3ec0cea00 0x7fb3ec156560 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:41.697 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.696+0000 7fb3f09fe700 1 --2- 172.21.15.62:0/1738804626 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3ec153a00 0x7fb3ec152d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:41.698 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.696+0000 7fb3f09fe700 1 -- 172.21.15.62:0/1738804626 >> 172.21.15.62:0/1738804626 conn(0x7fb3ec0703a0 msgr2=0x7fb3ec152050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:22:41.698 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.696+0000 7fb3f09fe700 1 -- 172.21.15.62:0/1738804626 shutdown_connections 2023-10-06T02:22:41.698 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.696+0000 7fb3f09fe700 1 -- 172.21.15.62:0/1738804626 wait complete. 2023-10-06T02:22:41.713 DEBUG:teuthology.orchestra.run.smithi062:> sudo ceph --cluster ceph osd new ad28fcc7-3408-4231-81cc-5a01e7a56ec2 2 2023-10-06T02:22:41.821 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:22:41.820+0000 7f3f028b9180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T02:22:41.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.892+0000 7f349db9d700 1 Processor -- start 2023-10-06T02:22:41.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.892+0000 7f349db9d700 1 -- start start 2023-10-06T02:22:41.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.893+0000 7f349db9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3498153a00 0x7f3498153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:22:41.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.893+0000 7f349db9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f349815de50 0x7f349815a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:22:41.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.893+0000 7f349db9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f349815af60 con 0x7f34980cea00 2023-10-06T02:22:41.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.893+0000 7f349db9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f349815d010 con 0x7f349815de50 2023-10-06T02:22:41.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.893+0000 7f349db9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f349815b240 con 0x7f3498153a00 2023-10-06T02:22:41.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.894+0000 7f349cb9b700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3498153a00 0x7f3498153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:22:41.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.894+0000 7f349cb9b700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3498153a00 0x7f3498153dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49988/0 (socket says 172.21.15.62:49988) 2023-10-06T02:22:41.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.894+0000 7f349cb9b700 1 -- 172.21.15.62:0/3357641795 learned_addr learned my addr 172.21.15.62:0/3357641795 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-10-06T02:22:41.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.894+0000 7f3497fff700 1 --2- 172.21.15.62:0/3357641795 >> v2:172.21.15.31:3300/0 conn(0x7f349815de50 0x7f349815a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:22:41.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.894+0000 7f349cb9b700 1 -- 172.21.15.62:0/3357641795 >> v2:172.21.15.31:3300/0 conn(0x7f349815de50 msgr2=0x7f349815a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:22:41.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.894+0000 7f349cb9b700 1 --2- 172.21.15.62:0/3357641795 >> v2:172.21.15.31:3300/0 conn(0x7f349815de50 0x7f349815a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:41.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.894+0000 7f349cb9b700 1 -- 172.21.15.62:0/3357641795 >> v1:172.21.15.62:6789/0 conn(0x7f34980cea00 legacy=0x7f34980cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:22:41.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.894+0000 7f349cb9b700 1 -- 172.21.15.62:0/3357641795 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f349815b520 con 0x7f3498153a00 2023-10-06T02:22:41.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.894+0000 7f34977fe700 1 -- 172.21.15.62:0/3357641795 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2133543374 0 0) 0x7f349815af60 con 0x7f34980cea00 2023-10-06T02:22:41.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.894+0000 7f349cb9b700 1 --2- 172.21.15.62:0/3357641795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3498153a00 0x7f3498153dd0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f34980cac20 tx=0x7f348c009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e53dff667da10469 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:22:41.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.895+0000 7f34977fe700 1 -- 172.21.15.62:0/3357641795 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f348c009e70 con 0x7f3498153a00 2023-10-06T02:22:41.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.895+0000 7f34977fe700 1 -- 172.21.15.62:0/3357641795 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f348c00b860 con 0x7f3498153a00 2023-10-06T02:22:41.898 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.895+0000 7f34977fe700 1 -- 172.21.15.62:0/3357641795 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f348c00bc00 con 0x7f3498153a00 2023-10-06T02:22:41.898 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.895+0000 7f349db9d700 1 -- 172.21.15.62:0/3357641795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3498153a00 msgr2=0x7f3498153dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:22:41.898 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.895+0000 7f349db9d700 1 --2- 172.21.15.62:0/3357641795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3498153a00 0x7f3498153dd0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f34980cac20 tx=0x7f348c009250 comp rx=0 tx=0).stop 2023-10-06T02:22:41.898 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.895+0000 7f349db9d700 1 -- 172.21.15.62:0/3357641795 shutdown_connections 2023-10-06T02:22:41.898 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.895+0000 7f349db9d700 1 --2- 172.21.15.62:0/3357641795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3498153a00 0x7f3498153dd0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:41.898 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.895+0000 7f349db9d700 1 --2- 172.21.15.62:0/3357641795 >> v2:172.21.15.31:3300/0 conn(0x7f349815de50 0x7f349815a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:41.898 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.895+0000 7f349db9d700 1 -- 172.21.15.62:0/3357641795 >> 172.21.15.62:0/3357641795 conn(0x7f34980703a0 msgr2=0x7f34980707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:22:41.898 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.895+0000 7f349db9d700 1 -- 172.21.15.62:0/3357641795 shutdown_connections 2023-10-06T02:22:41.898 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.895+0000 7f349db9d700 1 -- 172.21.15.62:0/3357641795 wait complete. 2023-10-06T02:22:41.900 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.897+0000 7f349db9d700 1 Processor -- start 2023-10-06T02:22:41.900 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.897+0000 7f349db9d700 1 -- start start 2023-10-06T02:22:41.900 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.898+0000 7f349db9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34980cea00 0x7f34980c9f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:22:41.900 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.898+0000 7f349db9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3498153a00 0x7f349814f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:22:41.900 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.898+0000 7f349db9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f349815de50 0x7f349814fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:22:41.900 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.898+0000 7f349db9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34981501a0 con 0x7f34980cea00 2023-10-06T02:22:41.900 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.898+0000 7f349db9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3498150430 con 0x7f349815de50 2023-10-06T02:22:41.900 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.898+0000 7f349db9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3498150710 con 0x7f3498153a00 2023-10-06T02:22:41.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.898+0000 7f3497fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3498153a00 0x7f349814f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:22:41.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.898+0000 7f349cb9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34980cea00 0x7f34980c9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:22:41.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.898+0000 7f349d39c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f349815de50 0x7f349814fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:22:41.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.898+0000 7f349cb9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34980cea00 0x7f34980c9f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.62:48866/0 (socket says 172.21.15.62:48866) 2023-10-06T02:22:41.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.898+0000 7f349cb9b700 1 -- 172.21.15.62:0/4278656884 learned_addr learned my addr 172.21.15.62:0/4278656884 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-10-06T02:22:41.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.899+0000 7f349cb9b700 1 -- 172.21.15.62:0/4278656884 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3498153a00 msgr2=0x7f349814f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:22:41.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.899+0000 7f349cb9b700 1 --2- 172.21.15.62:0/4278656884 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3498153a00 0x7f349814f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:41.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.899+0000 7f349cb9b700 1 -- 172.21.15.62:0/4278656884 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f349815de50 msgr2=0x7f349814fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:22:41.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.899+0000 7f349cb9b700 1 --2- 172.21.15.62:0/4278656884 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f349815de50 0x7f349814fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:41.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.899+0000 7f349cb9b700 1 -- 172.21.15.62:0/4278656884 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f348c0094f0 con 0x7f34980cea00 2023-10-06T02:22:41.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.899+0000 7f3497fff700 1 --2- 172.21.15.62:0/4278656884 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3498153a00 0x7f349814f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:22:41.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.899+0000 7f349d39c700 1 --2- 172.21.15.62:0/4278656884 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f349815de50 0x7f349814fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:22:41.902 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.899+0000 7f349cb9b700 1 --2- 172.21.15.62:0/4278656884 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34980cea00 0x7f34980c9f60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f348c0094c0 tx=0x7f348c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:22:41.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.899+0000 7f3494ff9700 1 -- 172.21.15.62:0/4278656884 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f348c00bbe0 con 0x7f34980cea00 2023-10-06T02:22:41.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.899+0000 7f3494ff9700 1 -- 172.21.15.62:0/4278656884 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f348c000c90 con 0x7f34980cea00 2023-10-06T02:22:41.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.899+0000 7f349db9d700 1 -- 172.21.15.62:0/4278656884 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34981509f0 con 0x7f34980cea00 2023-10-06T02:22:41.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.900+0000 7f3494ff9700 1 -- 172.21.15.62:0/4278656884 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f348c011650 con 0x7f34980cea00 2023-10-06T02:22:41.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.900+0000 7f349db9d700 1 -- 172.21.15.62:0/4278656884 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3498147fc0 con 0x7f34980cea00 2023-10-06T02:22:41.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.900+0000 7f3494ff9700 1 -- 172.21.15.62:0/4278656884 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f348c011cc0 con 0x7f34980cea00 2023-10-06T02:22:41.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.900+0000 7f3494ff9700 1 -- 172.21.15.62:0/4278656884 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 951+0+0 (secure 0 0 0) 0x7f348c02ebe0 con 0x7f34980cea00 2023-10-06T02:22:41.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.901+0000 7f349db9d700 1 -- 172.21.15.62:0/4278656884 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f349804c3c0 con 0x7f34980cea00 2023-10-06T02:22:41.904 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.903+0000 7f3494ff9700 1 -- 172.21.15.62:0/4278656884 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f348c033060 con 0x7f34980cea00 2023-10-06T02:22:41.920 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:22:41.919+0000 7f3f028b9180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T02:22:41.991 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.991+0000 7f349db9d700 1 -- 172.21.15.62:0/4278656884 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd new", "uuid": "ad28fcc7-3408-4231-81cc-5a01e7a56ec2", "id": 2} v 0) v1 -- 0x7f3498147be0 con 0x7f34980cea00 2023-10-06T02:22:41.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.994+0000 7f3494ff9700 1 -- 172.21.15.62:0/4278656884 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ad28fcc7-3408-4231-81cc-5a01e7a56ec2", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f348c015020 con 0x7f34980cea00 2023-10-06T02:22:41.995 INFO:teuthology.orchestra.run.smithi062.stdout:2 2023-10-06T02:22:41.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.995+0000 7f349db9d700 1 -- 172.21.15.62:0/4278656884 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34980cea00 msgr2=0x7f34980c9f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:22:41.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.995+0000 7f349db9d700 1 --2- 172.21.15.62:0/4278656884 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34980cea00 0x7f34980c9f60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f348c0094c0 tx=0x7f348c017000 comp rx=0 tx=0).stop 2023-10-06T02:22:41.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.995+0000 7f349db9d700 1 -- 172.21.15.62:0/4278656884 shutdown_connections 2023-10-06T02:22:41.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.995+0000 7f349db9d700 1 --2- 172.21.15.62:0/4278656884 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f349815de50 0x7f349814fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:41.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.995+0000 7f349db9d700 1 --2- 172.21.15.62:0/4278656884 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34980cea00 0x7f34980c9f60 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:41.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.995+0000 7f349db9d700 1 --2- 172.21.15.62:0/4278656884 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3498153a00 0x7f349814f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:41.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.995+0000 7f349db9d700 1 -- 172.21.15.62:0/4278656884 >> 172.21.15.62:0/4278656884 conn(0x7f34980703a0 msgr2=0x7f349814adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:22:41.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.995+0000 7f349db9d700 1 -- 172.21.15.62:0/4278656884 shutdown_connections 2023-10-06T02:22:41.997 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:41.995+0000 7f349db9d700 1 -- 172.21.15.62:0/4278656884 wait complete. 2023-10-06T02:22:42.012 DEBUG:teuthology.orchestra.run.smithi062:> sudo ceph --cluster ceph osd new afb6d8de-115b-4585-acdc-e5c2d6f38269 3 2023-10-06T02:22:42.031 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:22:42.030+0000 7f65ed07d180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T02:22:42.150 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:22:42.148+0000 7f3f028b9180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T02:22:42.192 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.191+0000 7f211e1ff700 1 Processor -- start 2023-10-06T02:22:42.192 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.191+0000 7f211e1ff700 1 -- start start 2023-10-06T02:22:42.195 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.193+0000 7f211e1ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f211813c9b0 0x7f2118069a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:22:42.195 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.193+0000 7f211e1ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f21180cc6f0 0x7f2118069f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:22:42.195 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.193+0000 7f211e1ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f211806d7b0 con 0x7f21180cc320 2023-10-06T02:22:42.195 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.193+0000 7f211e1ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f211806f7d0 con 0x7f211813c9b0 2023-10-06T02:22:42.195 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.193+0000 7f211e1ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f211806e330 con 0x7f21180cc6f0 2023-10-06T02:22:42.195 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.193+0000 7f21177fe700 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f21180cc320 0x7f211813c2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:48278/0 (socket says 172.21.15.62:48278) 2023-10-06T02:22:42.195 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.193+0000 7f21177fe700 1 -- 172.21.15.62:0/1369144893 learned_addr learned my addr 172.21.15.62:0/1369144893 (peer_addr_for_me v1:172.21.15.62:0/0) 2023-10-06T02:22:42.195 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.193+0000 7f2116ffd700 1 --2- 172.21.15.62:0/1369144893 >> v2:172.21.15.62:3300/0 conn(0x7f211813c9b0 0x7f2118069a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:22:42.195 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.193+0000 7f2117fff700 1 --2- 172.21.15.62:0/1369144893 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f21180cc6f0 0x7f2118069f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:22:42.195 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.193+0000 7f2116ffd700 1 -- 172.21.15.62:0/1369144893 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f21180cc6f0 msgr2=0x7f2118069f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:22:42.196 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.193+0000 7f2116ffd700 1 --2- 172.21.15.62:0/1369144893 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f21180cc6f0 0x7f2118069f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:42.196 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.193+0000 7f2116ffd700 1 -- 172.21.15.62:0/1369144893 >> v1:172.21.15.62:6789/0 conn(0x7f21180cc320 legacy=0x7f211813c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:22:42.196 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.193+0000 7f21167fc700 1 -- 172.21.15.62:0/1369144893 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3521826966 0 0) 0x7f211806d7b0 con 0x7f21180cc320 2023-10-06T02:22:42.196 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.193+0000 7f2116ffd700 1 -- 172.21.15.62:0/1369144893 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f211806da00 con 0x7f211813c9b0 2023-10-06T02:22:42.196 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.193+0000 7f2117fff700 1 --2- 172.21.15.62:0/1369144893 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f21180cc6f0 0x7f2118069f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:22:42.196 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.194+0000 7f2116ffd700 1 --2- 172.21.15.62:0/1369144893 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f211813c9b0 0x7f2118069a00 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f210c0098e0 tx=0x7f210c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4581220b2bfed50 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:22:42.196 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.194+0000 7f21167fc700 1 -- 172.21.15.62:0/1369144893 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f210c00b640 con 0x7f211813c9b0 2023-10-06T02:22:42.197 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.194+0000 7f21167fc700 1 -- 172.21.15.62:0/1369144893 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f210c00b7a0 con 0x7f211813c9b0 2023-10-06T02:22:42.197 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.194+0000 7f21167fc700 1 -- 172.21.15.62:0/1369144893 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f210c00bb40 con 0x7f211813c9b0 2023-10-06T02:22:42.197 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.195+0000 7f211e1ff700 1 -- 172.21.15.62:0/1369144893 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f211813c9b0 msgr2=0x7f2118069a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:22:42.197 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.195+0000 7f211e1ff700 1 --2- 172.21.15.62:0/1369144893 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f211813c9b0 0x7f2118069a00 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f210c0098e0 tx=0x7f210c0091a0 comp rx=0 tx=0).stop 2023-10-06T02:22:42.198 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.195+0000 7f211e1ff700 1 -- 172.21.15.62:0/1369144893 shutdown_connections 2023-10-06T02:22:42.198 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.195+0000 7f211e1ff700 1 --2- 172.21.15.62:0/1369144893 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f21180cc6f0 0x7f2118069f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:42.198 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.195+0000 7f211e1ff700 1 --2- 172.21.15.62:0/1369144893 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f211813c9b0 0x7f2118069a00 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:42.198 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.195+0000 7f211e1ff700 1 -- 172.21.15.62:0/1369144893 >> 172.21.15.62:0/1369144893 conn(0x7f2118067f00 msgr2=0x7f2118068300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:22:42.198 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.195+0000 7f211e1ff700 1 -- 172.21.15.62:0/1369144893 shutdown_connections 2023-10-06T02:22:42.198 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.195+0000 7f211e1ff700 1 -- 172.21.15.62:0/1369144893 wait complete. 2023-10-06T02:22:42.198 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.196+0000 7f211e1ff700 1 Processor -- start 2023-10-06T02:22:42.198 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.196+0000 7f211e1ff700 1 -- start start 2023-10-06T02:22:42.200 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.197+0000 7f211e1ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21180cc320 0x7f21180c7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:22:42.200 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.197+0000 7f211e1ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f21180cc6f0 0x7f211816c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:22:42.200 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.197+0000 7f211e1ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f211813c9b0 0x7f211816ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:22:42.200 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.197+0000 7f211e1ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2118153820 con 0x7f21180cc320 2023-10-06T02:22:42.200 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.197+0000 7f211e1ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21180b0730 con 0x7f211813c9b0 2023-10-06T02:22:42.201 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.197+0000 7f211e1ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f211816cee0 con 0x7f21180cc6f0 2023-10-06T02:22:42.201 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.198+0000 7f2117fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f211813c9b0 0x7f211816ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:22:42.201 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.198+0000 7f2116ffd700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f21180cc6f0 0x7f211816c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:22:42.201 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.198+0000 7f21177fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21180cc320 0x7f21180c7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:22:42.201 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.198+0000 7f2116ffd700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f21180cc6f0 0x7f211816c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.62:32888/0 (socket says 172.21.15.62:32888) 2023-10-06T02:22:42.201 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.198+0000 7f2116ffd700 1 -- 172.21.15.62:0/1493125523 learned_addr learned my addr 172.21.15.62:0/1493125523 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-10-06T02:22:42.201 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.198+0000 7f2116ffd700 1 -- 172.21.15.62:0/1493125523 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f211813c9b0 msgr2=0x7f211816ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:22:42.201 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.198+0000 7f2116ffd700 1 --2- 172.21.15.62:0/1493125523 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f211813c9b0 0x7f211816ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:42.201 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.198+0000 7f2116ffd700 1 -- 172.21.15.62:0/1493125523 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21180cc320 msgr2=0x7f21180c7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:22:42.201 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.198+0000 7f2116ffd700 1 --2- 172.21.15.62:0/1493125523 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21180cc320 0x7f21180c7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:42.201 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.198+0000 7f2116ffd700 1 -- 172.21.15.62:0/1493125523 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2110009a20 con 0x7f21180cc6f0 2023-10-06T02:22:42.201 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.198+0000 7f2117fff700 1 --2- 172.21.15.62:0/1493125523 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f211813c9b0 0x7f211816ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:22:42.202 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.198+0000 7f2116ffd700 1 --2- 172.21.15.62:0/1493125523 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f21180cc6f0 0x7f211816c9a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f210c013000 tx=0x7f210c017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:22:42.202 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.199+0000 7f20ff7fe700 1 -- 172.21.15.62:0/1493125523 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f210c00bb20 con 0x7f21180cc6f0 2023-10-06T02:22:42.202 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.199+0000 7f20ff7fe700 1 -- 172.21.15.62:0/1493125523 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f210c000c40 con 0x7f21180cc6f0 2023-10-06T02:22:42.202 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.199+0000 7f20ff7fe700 1 -- 172.21.15.62:0/1493125523 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f210c011650 con 0x7f21180cc6f0 2023-10-06T02:22:42.202 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.199+0000 7f211e1ff700 1 -- 172.21.15.62:0/1493125523 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f210c009420 con 0x7f21180cc6f0 2023-10-06T02:22:42.202 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.199+0000 7f211e1ff700 1 -- 172.21.15.62:0/1493125523 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f211816c510 con 0x7f21180cc6f0 2023-10-06T02:22:42.202 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.199+0000 7f20ff7fe700 1 -- 172.21.15.62:0/1493125523 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f210c011d00 con 0x7f21180cc6f0 2023-10-06T02:22:42.202 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.199+0000 7f20ff7fe700 1 -- 172.21.15.62:0/1493125523 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1066+0+0 (secure 0 0 0) 0x7f210c02ed80 con 0x7f21180cc6f0 2023-10-06T02:22:42.202 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.200+0000 7f211e1ff700 1 -- 172.21.15.62:0/1493125523 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f211816c6a0 con 0x7f21180cc6f0 2023-10-06T02:22:42.204 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.203+0000 7f20ff7fe700 1 -- 172.21.15.62:0/1493125523 <== mon.2 v2:172.21.15.62:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f210c0446d0 con 0x7f21180cc6f0 2023-10-06T02:22:42.210 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:22:42.210+0000 7f65ed07d180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T02:22:42.339 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.338+0000 7f211e1ff700 1 -- 172.21.15.62:0/1493125523 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "osd new", "uuid": "afb6d8de-115b-4585-acdc-e5c2d6f38269", "id": 3} v 0) v1 -- 0x7f211804c3c0 con 0x7f21180cc6f0 2023-10-06T02:22:42.342 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.342+0000 7f20ff7fe700 1 -- 172.21.15.62:0/1493125523 <== mon.2 v2:172.21.15.62:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "afb6d8de-115b-4585-acdc-e5c2d6f38269", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f210c033070 con 0x7f21180cc6f0 2023-10-06T02:22:42.343 INFO:teuthology.orchestra.run.smithi062.stdout:3 2023-10-06T02:22:42.343 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:22:42.342+0000 7f65ed07d180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T02:22:42.344 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.343+0000 7f211e1ff700 1 -- 172.21.15.62:0/1493125523 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f21180cc6f0 msgr2=0x7f211816c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:22:42.344 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.343+0000 7f211e1ff700 1 --2- 172.21.15.62:0/1493125523 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f21180cc6f0 0x7f211816c9a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f210c013000 tx=0x7f210c017000 comp rx=0 tx=0).stop 2023-10-06T02:22:42.344 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.343+0000 7f211e1ff700 1 -- 172.21.15.62:0/1493125523 shutdown_connections 2023-10-06T02:22:42.344 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.343+0000 7f211e1ff700 1 --2- 172.21.15.62:0/1493125523 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f211813c9b0 0x7f211816ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:42.344 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.343+0000 7f211e1ff700 1 --2- 172.21.15.62:0/1493125523 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21180cc320 0x7f21180c7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:42.345 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.343+0000 7f211e1ff700 1 --2- 172.21.15.62:0/1493125523 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f21180cc6f0 0x7f211816c9a0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:42.345 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.343+0000 7f211e1ff700 1 -- 172.21.15.62:0/1493125523 >> 172.21.15.62:0/1493125523 conn(0x7f2118067f00 msgr2=0x7f21180c4c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:22:42.345 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.343+0000 7f211e1ff700 1 -- 172.21.15.62:0/1493125523 shutdown_connections 2023-10-06T02:22:42.345 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.343+0000 7f211e1ff700 1 -- 172.21.15.62:0/1493125523 wait complete. 2023-10-06T02:22:42.367 DEBUG:teuthology.orchestra.run.smithi062:> sudo ceph --cluster ceph osd new 909bca1c-f5fd-4efd-b50e-79843e358fcd 4 2023-10-06T02:22:42.373 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:22:42.372+0000 7f3f028b9180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T02:22:42.472 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:22:42.470+0000 7f3f028b9180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T02:22:42.584 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:22:42.583+0000 7f65ed07d180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T02:22:42.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.588+0000 7f82ca59d700 1 Processor -- start 2023-10-06T02:22:42.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.588+0000 7f82ca59d700 1 -- start start 2023-10-06T02:22:42.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.589+0000 7f82ca59d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f82c4153a00 0x7f82c4153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:22:42.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.589+0000 7f82ca59d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f82c415de50 0x7f82c415a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:22:42.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.589+0000 7f82ca59d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f82c40cea00 0x7f82c40cee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:22:42.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.589+0000 7f82ca59d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82c40cf3c0 con 0x7f82c415de50 2023-10-06T02:22:42.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.589+0000 7f82ca59d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82c415af60 con 0x7f82c4153a00 2023-10-06T02:22:42.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.589+0000 7f82ca59d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82c415b240 con 0x7f82c40cea00 2023-10-06T02:22:42.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.590+0000 7f82c959b700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f82c4153a00 0x7f82c4153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:22:42.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.590+0000 7f82c9d9c700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f82c40cea00 0x7f82c40cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:22:42.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.590+0000 7f82c959b700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f82c4153a00 0x7f82c4153dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59964/0 (socket says 172.21.15.62:59964) 2023-10-06T02:22:42.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.590+0000 7f82c959b700 1 -- 172.21.15.62:0/572475921 learned_addr learned my addr 172.21.15.62:0/572475921 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-10-06T02:22:42.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.590+0000 7f82c8d9a700 1 --2- 172.21.15.62:0/572475921 >> v2:172.21.15.31:3300/0 conn(0x7f82c415de50 0x7f82c415a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:22:42.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.590+0000 7f82c9d9c700 1 -- 172.21.15.62:0/572475921 >> v2:172.21.15.62:3300/0 conn(0x7f82c4153a00 msgr2=0x7f82c4153dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:22:42.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.590+0000 7f82c9d9c700 1 --2- 172.21.15.62:0/572475921 >> v2:172.21.15.62:3300/0 conn(0x7f82c4153a00 0x7f82c4153dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:42.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.590+0000 7f82c9d9c700 1 -- 172.21.15.62:0/572475921 >> v2:172.21.15.31:3300/0 conn(0x7f82c415de50 msgr2=0x7f82c415a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:22:42.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.590+0000 7f82c9d9c700 1 --2- 172.21.15.62:0/572475921 >> v2:172.21.15.31:3300/0 conn(0x7f82c415de50 0x7f82c415a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:42.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.590+0000 7f82c9d9c700 1 -- 172.21.15.62:0/572475921 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82c415b520 con 0x7f82c40cea00 2023-10-06T02:22:42.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.590+0000 7f82c959b700 1 --2- 172.21.15.62:0/572475921 >> v2:172.21.15.62:3300/0 conn(0x7f82c4153a00 0x7f82c4153dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:22:42.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.591+0000 7f82c9d9c700 1 --2- 172.21.15.62:0/572475921 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f82c40cea00 0x7f82c40cee80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f82c40cac20 tx=0x7f82bc009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78fde390e4d8bf44 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:22:42.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.591+0000 7f82bbfff700 1 -- 172.21.15.62:0/572475921 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82bc00e070 con 0x7f82c40cea00 2023-10-06T02:22:42.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.591+0000 7f82bbfff700 1 -- 172.21.15.62:0/572475921 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82bc00bab0 con 0x7f82c40cea00 2023-10-06T02:22:42.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.591+0000 7f82bbfff700 1 -- 172.21.15.62:0/572475921 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82bc00be70 con 0x7f82c40cea00 2023-10-06T02:22:42.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.591+0000 7f82ca59d700 1 -- 172.21.15.62:0/572475921 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f82c40cea00 msgr2=0x7f82c40cee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:22:42.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.591+0000 7f82ca59d700 1 --2- 172.21.15.62:0/572475921 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f82c40cea00 0x7f82c40cee80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f82c40cac20 tx=0x7f82bc009780 comp rx=0 tx=0).stop 2023-10-06T02:22:42.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.591+0000 7f82ca59d700 1 -- 172.21.15.62:0/572475921 shutdown_connections 2023-10-06T02:22:42.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.591+0000 7f82ca59d700 1 --2- 172.21.15.62:0/572475921 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f82c40cea00 0x7f82c40cee80 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:42.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.591+0000 7f82ca59d700 1 --2- 172.21.15.62:0/572475921 >> v2:172.21.15.62:3300/0 conn(0x7f82c4153a00 0x7f82c4153dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:42.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.591+0000 7f82ca59d700 1 --2- 172.21.15.62:0/572475921 >> v2:172.21.15.31:3300/0 conn(0x7f82c415de50 0x7f82c415a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:42.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.591+0000 7f82ca59d700 1 -- 172.21.15.62:0/572475921 >> 172.21.15.62:0/572475921 conn(0x7f82c40703a0 msgr2=0x7f82c40707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:22:42.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.592+0000 7f82ca59d700 1 -- 172.21.15.62:0/572475921 shutdown_connections 2023-10-06T02:22:42.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.592+0000 7f82ca59d700 1 -- 172.21.15.62:0/572475921 wait complete. 2023-10-06T02:22:42.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.593+0000 7f82ca59d700 1 Processor -- start 2023-10-06T02:22:42.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.593+0000 7f82ca59d700 1 -- start start 2023-10-06T02:22:42.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.594+0000 7f82ca59d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f82c40cea00 0x7f82c414fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:22:42.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.594+0000 7f82ca59d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82c4153a00 0x7f82c415c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:22:42.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.594+0000 7f82ca59d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82c415de50 0x7f82c40c7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:22:42.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.594+0000 7f82ca59d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c40cdc70 con 0x7f82c4153a00 2023-10-06T02:22:42.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.594+0000 7f82ca59d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c40cd7e0 con 0x7f82c415de50 2023-10-06T02:22:42.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.594+0000 7f82ca59d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82c41501d0 con 0x7f82c40cea00 2023-10-06T02:22:42.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.594+0000 7f82c959b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f82c40cea00 0x7f82c414fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:22:42.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.594+0000 7f82c959b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f82c40cea00 0x7f82c414fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.62:51754/0 (socket says 172.21.15.62:51754) 2023-10-06T02:22:42.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.594+0000 7f82c959b700 1 -- 172.21.15.62:0/4253087214 learned_addr learned my addr 172.21.15.62:0/4253087214 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-10-06T02:22:42.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.594+0000 7f82c9d9c700 1 --2- 172.21.15.62:0/4253087214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82c415de50 0x7f82c40c7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:22:42.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.594+0000 7f82c8d9a700 1 --2- 172.21.15.62:0/4253087214 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82c4153a00 0x7f82c415c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:22:42.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.595+0000 7f82c959b700 1 -- 172.21.15.62:0/4253087214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82c415de50 msgr2=0x7f82c40c7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:22:42.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.595+0000 7f82c959b700 1 --2- 172.21.15.62:0/4253087214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82c415de50 0x7f82c40c7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:42.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.595+0000 7f82c959b700 1 -- 172.21.15.62:0/4253087214 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82c4153a00 msgr2=0x7f82c415c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:22:42.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.595+0000 7f82c959b700 1 --2- 172.21.15.62:0/4253087214 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82c4153a00 0x7f82c415c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:42.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.595+0000 7f82c959b700 1 -- 172.21.15.62:0/4253087214 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82c00094f0 con 0x7f82c40cea00 2023-10-06T02:22:42.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.595+0000 7f82c959b700 1 --2- 172.21.15.62:0/4253087214 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f82c40cea00 0x7f82c414fc90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f82c000ec80 tx=0x7f82c0009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:22:42.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.595+0000 7f82b97fa700 1 -- 172.21.15.62:0/4253087214 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c000c8e0 con 0x7f82c40cea00 2023-10-06T02:22:42.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.595+0000 7f82b97fa700 1 -- 172.21.15.62:0/4253087214 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82c000ca40 con 0x7f82c40cea00 2023-10-06T02:22:42.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.595+0000 7f82b97fa700 1 -- 172.21.15.62:0/4253087214 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c000cdc0 con 0x7f82c40cea00 2023-10-06T02:22:42.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.595+0000 7f82ca59d700 1 -- 172.21.15.62:0/4253087214 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82bc009a20 con 0x7f82c40cea00 2023-10-06T02:22:42.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.596+0000 7f82ca59d700 1 -- 172.21.15.62:0/4253087214 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82c40cceb0 con 0x7f82c40cea00 2023-10-06T02:22:42.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.596+0000 7f82b97fa700 1 -- 172.21.15.62:0/4253087214 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f82c0005c80 con 0x7f82c40cea00 2023-10-06T02:22:42.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.596+0000 7f82b97fa700 1 -- 172.21.15.62:0/4253087214 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1181+0+0 (secure 0 0 0) 0x7f82c0013680 con 0x7f82c40cea00 2023-10-06T02:22:42.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.597+0000 7f82ca59d700 1 -- 172.21.15.62:0/4253087214 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82c404c3c0 con 0x7f82c40cea00 2023-10-06T02:22:42.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.601+0000 7f82b97fa700 1 -- 172.21.15.62:0/4253087214 <== mon.2 v2:172.21.15.62:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f82c0021060 con 0x7f82c40cea00 2023-10-06T02:22:42.640 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:22:42.639+0000 7f3f028b9180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T02:22:42.691 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.690+0000 7f82ca59d700 1 -- 172.21.15.62:0/4253087214 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "osd new", "uuid": "909bca1c-f5fd-4efd-b50e-79843e358fcd", "id": 4} v 0) v1 -- 0x7f82c40b2d30 con 0x7f82c40cea00 2023-10-06T02:22:42.700 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.699+0000 7f82b97fa700 1 -- 172.21.15.62:0/4253087214 <== mon.2 v2:172.21.15.62:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "909bca1c-f5fd-4efd-b50e-79843e358fcd", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f82c0029020 con 0x7f82c40cea00 2023-10-06T02:22:42.700 INFO:teuthology.orchestra.run.smithi062.stdout:4 2023-10-06T02:22:42.701 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.700+0000 7f82ca59d700 1 -- 172.21.15.62:0/4253087214 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f82c40cea00 msgr2=0x7f82c414fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:22:42.701 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.700+0000 7f82ca59d700 1 --2- 172.21.15.62:0/4253087214 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f82c40cea00 0x7f82c414fc90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f82c000ec80 tx=0x7f82c0009250 comp rx=0 tx=0).stop 2023-10-06T02:22:42.701 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.700+0000 7f82ca59d700 1 -- 172.21.15.62:0/4253087214 shutdown_connections 2023-10-06T02:22:42.701 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.700+0000 7f82ca59d700 1 --2- 172.21.15.62:0/4253087214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82c415de50 0x7f82c40c7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:42.701 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.700+0000 7f82ca59d700 1 --2- 172.21.15.62:0/4253087214 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f82c40cea00 0x7f82c414fc90 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:42.701 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.700+0000 7f82ca59d700 1 --2- 172.21.15.62:0/4253087214 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82c4153a00 0x7f82c415c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:42.702 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.700+0000 7f82ca59d700 1 -- 172.21.15.62:0/4253087214 >> 172.21.15.62:0/4253087214 conn(0x7f82c40703a0 msgr2=0x7f82c4151110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:22:42.702 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.700+0000 7f82ca59d700 1 -- 172.21.15.62:0/4253087214 shutdown_connections 2023-10-06T02:22:42.702 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.700+0000 7f82ca59d700 1 -- 172.21.15.62:0/4253087214 wait complete. 2023-10-06T02:22:42.718 DEBUG:teuthology.orchestra.run.smithi062:> sudo ceph --cluster ceph osd new 48d582fb-7f96-4cd2-ae6e-95ab273a5907 5 2023-10-06T02:22:42.763 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:22:42.762+0000 7f3f028b9180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T02:22:42.816 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:22:42.815+0000 7f65ed07d180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T02:22:42.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.900+0000 7f1a2ef9d700 1 Processor -- start 2023-10-06T02:22:42.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.900+0000 7f1a2ef9d700 1 -- start start 2023-10-06T02:22:42.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.902+0000 7f1a2ef9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1a28153dd0 0x7f1a2815de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:22:42.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.902+0000 7f1a2ef9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a2815ac50 con 0x7f1a280cea00 2023-10-06T02:22:42.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.902+0000 7f1a2ef9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a2815d010 con 0x7f1a28153a00 2023-10-06T02:22:42.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.902+0000 7f1a2ef9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a2815af60 con 0x7f1a28153dd0 2023-10-06T02:22:42.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.902+0000 7f1a2df9b700 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f1a28153a00 0x7f1a2815a540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:40834/0 (socket says 172.21.15.62:40834) 2023-10-06T02:22:42.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.902+0000 7f1a2df9b700 1 -- 172.21.15.62:0/2137823821 learned_addr learned my addr 172.21.15.62:0/2137823821 (peer_addr_for_me v1:172.21.15.62:0/0) 2023-10-06T02:22:42.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.902+0000 7f1a2d79a700 1 --2- 172.21.15.62:0/2137823821 >> v2:172.21.15.31:3300/0 conn(0x7f1a28153dd0 0x7f1a2815de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:22:42.905 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.902+0000 7f1a2cf99700 1 -- 172.21.15.62:0/2137823821 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3577460155 0 0) 0x7f1a2815ac50 con 0x7f1a280cea00 2023-10-06T02:22:42.905 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.903+0000 7f1a2cf99700 1 -- 172.21.15.62:0/2137823821 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a14003570 con 0x7f1a280cea00 2023-10-06T02:22:42.905 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.903+0000 7f1a2cf99700 1 -- 172.21.15.62:0/2137823821 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3434920809 0 0) 0x7f1a2815d010 con 0x7f1a28153a00 2023-10-06T02:22:42.905 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.903+0000 7f1a2cf99700 1 -- 172.21.15.62:0/2137823821 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a2815ac50 con 0x7f1a28153a00 2023-10-06T02:22:42.905 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.903+0000 7f1a2d79a700 1 -- 172.21.15.62:0/2137823821 >> v1:172.21.15.62:6789/0 conn(0x7f1a28153a00 legacy=0x7f1a2815a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:22:42.906 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.903+0000 7f1a2d79a700 1 -- 172.21.15.62:0/2137823821 >> v1:172.21.15.31:6789/0 conn(0x7f1a280cea00 legacy=0x7f1a280ceea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:22:42.906 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.903+0000 7f1a2d79a700 1 -- 172.21.15.62:0/2137823821 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a2815b240 con 0x7f1a28153dd0 2023-10-06T02:22:42.906 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.903+0000 7f1a2d79a700 1 --2- 172.21.15.62:0/2137823821 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a28153dd0 0x7f1a2815de50 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1a180098e0 tx=0x7f1a180091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5738e99655fce912 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:22:42.906 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.903+0000 7f1a2cf99700 1 -- 172.21.15.62:0/2137823821 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a18011630 con 0x7f1a28153dd0 2023-10-06T02:22:42.906 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.903+0000 7f1a2cf99700 1 -- 172.21.15.62:0/2137823821 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a18011790 con 0x7f1a28153dd0 2023-10-06T02:22:42.906 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.903+0000 7f1a2cf99700 1 -- 172.21.15.62:0/2137823821 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a18011b30 con 0x7f1a28153dd0 2023-10-06T02:22:42.906 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.904+0000 7f1a2ef9d700 1 -- 172.21.15.62:0/2137823821 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a28153dd0 msgr2=0x7f1a2815de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:22:42.906 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.904+0000 7f1a2ef9d700 1 --2- 172.21.15.62:0/2137823821 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a28153dd0 0x7f1a2815de50 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1a180098e0 tx=0x7f1a180091a0 comp rx=0 tx=0).stop 2023-10-06T02:22:42.906 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.904+0000 7f1a2ef9d700 1 -- 172.21.15.62:0/2137823821 shutdown_connections 2023-10-06T02:22:42.906 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.904+0000 7f1a2ef9d700 1 --2- 172.21.15.62:0/2137823821 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a28153dd0 0x7f1a2815de50 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:42.906 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.904+0000 7f1a2ef9d700 1 -- 172.21.15.62:0/2137823821 >> 172.21.15.62:0/2137823821 conn(0x7f1a280703a0 msgr2=0x7f1a280707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:22:42.906 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.904+0000 7f1a2ef9d700 1 -- 172.21.15.62:0/2137823821 shutdown_connections 2023-10-06T02:22:42.907 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.904+0000 7f1a2ef9d700 1 -- 172.21.15.62:0/2137823821 wait complete. 2023-10-06T02:22:42.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.905+0000 7f1a2ef9d700 1 Processor -- start 2023-10-06T02:22:42.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.906+0000 7f1a2ef9d700 1 -- start start 2023-10-06T02:22:42.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.906+0000 7f1a2ef9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1a280cea00 0x7f1a281585f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:22:42.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.906+0000 7f1a2ef9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a28153a00 0x7f1a2814c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:22:42.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.907+0000 7f1a2ef9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a28153dd0 0x7f1a2806d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:22:42.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.907+0000 7f1a2ef9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a28158160 con 0x7f1a28153a00 2023-10-06T02:22:42.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.907+0000 7f1a2ef9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a28157550 con 0x7f1a28153dd0 2023-10-06T02:22:42.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.907+0000 7f1a2ef9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a28157830 con 0x7f1a280cea00 2023-10-06T02:22:42.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.907+0000 7f1a2df9b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1a280cea00 0x7f1a281585f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:22:42.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.907+0000 7f1a2df9b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1a280cea00 0x7f1a281585f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.62:51756/0 (socket says 172.21.15.62:51756) 2023-10-06T02:22:42.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.907+0000 7f1a2df9b700 1 -- 172.21.15.62:0/3857108427 learned_addr learned my addr 172.21.15.62:0/3857108427 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-10-06T02:22:42.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.907+0000 7f1a2d79a700 1 --2- 172.21.15.62:0/3857108427 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a28153a00 0x7f1a2814c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:22:42.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.907+0000 7f1a2e79c700 1 --2- 172.21.15.62:0/3857108427 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a28153dd0 0x7f1a2806d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:22:42.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.907+0000 7f1a2df9b700 1 -- 172.21.15.62:0/3857108427 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a28153dd0 msgr2=0x7f1a2806d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:22:42.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.907+0000 7f1a2df9b700 1 --2- 172.21.15.62:0/3857108427 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a28153dd0 0x7f1a2806d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:42.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.907+0000 7f1a2df9b700 1 -- 172.21.15.62:0/3857108427 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a28153a00 msgr2=0x7f1a2814c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:22:42.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.907+0000 7f1a2df9b700 1 --2- 172.21.15.62:0/3857108427 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a28153a00 0x7f1a2814c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:42.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.907+0000 7f1a2df9b700 1 -- 172.21.15.62:0/3857108427 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a18009420 con 0x7f1a280cea00 2023-10-06T02:22:42.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.907+0000 7f1a2d79a700 1 --2- 172.21.15.62:0/3857108427 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a28153a00 0x7f1a2814c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:22:42.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.908+0000 7f1a2df9b700 1 --2- 172.21.15.62:0/3857108427 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1a280cea00 0x7f1a281585f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1a2400c7c0 tx=0x7f1a24007e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:22:42.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.908+0000 7f1a1dffb700 1 -- 172.21.15.62:0/3857108427 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a24005cd0 con 0x7f1a280cea00 2023-10-06T02:22:42.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.908+0000 7f1a1dffb700 1 -- 172.21.15.62:0/3857108427 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a24005e30 con 0x7f1a280cea00 2023-10-06T02:22:42.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.908+0000 7f1a1dffb700 1 -- 172.21.15.62:0/3857108427 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a24007320 con 0x7f1a280cea00 2023-10-06T02:22:42.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.908+0000 7f1a2ef9d700 1 -- 172.21.15.62:0/3857108427 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a28157b10 con 0x7f1a280cea00 2023-10-06T02:22:42.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.908+0000 7f1a2ef9d700 1 -- 172.21.15.62:0/3857108427 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a2814ca40 con 0x7f1a280cea00 2023-10-06T02:22:42.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.908+0000 7f1a1dffb700 1 -- 172.21.15.62:0/3857108427 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f1a2401c6c0 con 0x7f1a280cea00 2023-10-06T02:22:42.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.908+0000 7f1a1dffb700 1 -- 172.21.15.62:0/3857108427 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1296+0+0 (secure 0 0 0) 0x7f1a24007720 con 0x7f1a280cea00 2023-10-06T02:22:42.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.909+0000 7f1a2ef9d700 1 -- 172.21.15.62:0/3857108427 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a2804c3c0 con 0x7f1a280cea00 2023-10-06T02:22:42.914 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:42.913+0000 7f1a1dffb700 1 -- 172.21.15.62:0/3857108427 <== mon.2 v2:172.21.15.62:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f1a24017070 con 0x7f1a280cea00 2023-10-06T02:22:42.919 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:22:42.918+0000 7f65ed07d180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T02:22:43.003 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.002+0000 7f1a2ef9d700 1 -- 172.21.15.62:0/3857108427 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "osd new", "uuid": "48d582fb-7f96-4cd2-ae6e-95ab273a5907", "id": 5} v 0) v1 -- 0x7f1a2815b520 con 0x7f1a280cea00 2023-10-06T02:22:43.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.009+0000 7f1a1dffb700 1 -- 172.21.15.62:0/3857108427 <== mon.2 v2:172.21.15.62:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "48d582fb-7f96-4cd2-ae6e-95ab273a5907", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f1a2401c970 con 0x7f1a280cea00 2023-10-06T02:22:43.010 INFO:teuthology.orchestra.run.smithi062.stdout:5 2023-10-06T02:22:43.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.010+0000 7f1a2ef9d700 1 -- 172.21.15.62:0/3857108427 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1a280cea00 msgr2=0x7f1a281585f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:22:43.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.010+0000 7f1a2ef9d700 1 --2- 172.21.15.62:0/3857108427 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1a280cea00 0x7f1a281585f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1a2400c7c0 tx=0x7f1a24007e00 comp rx=0 tx=0).stop 2023-10-06T02:22:43.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.010+0000 7f1a2ef9d700 1 -- 172.21.15.62:0/3857108427 shutdown_connections 2023-10-06T02:22:43.012 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.010+0000 7f1a2ef9d700 1 --2- 172.21.15.62:0/3857108427 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a28153dd0 0x7f1a2806d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:43.012 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.010+0000 7f1a2ef9d700 1 --2- 172.21.15.62:0/3857108427 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1a280cea00 0x7f1a281585f0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:43.012 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.010+0000 7f1a2ef9d700 1 --2- 172.21.15.62:0/3857108427 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a28153a00 0x7f1a2814c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:43.012 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.010+0000 7f1a2ef9d700 1 -- 172.21.15.62:0/3857108427 >> 172.21.15.62:0/3857108427 conn(0x7f1a280703a0 msgr2=0x7f1a280b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:22:43.012 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.010+0000 7f1a2ef9d700 1 -- 172.21.15.62:0/3857108427 shutdown_connections 2023-10-06T02:22:43.012 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.010+0000 7f1a2ef9d700 1 -- 172.21.15.62:0/3857108427 wait complete. 2023-10-06T02:22:43.028 DEBUG:teuthology.orchestra.run.smithi062:> sudo ceph --cluster ceph osd new 485507f8-ea88-45d4-a0fd-793ea2d7507b 6 2023-10-06T02:22:43.094 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:22:43.093+0000 7f65ed07d180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T02:22:43.215 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.214+0000 7f026bfff700 1 Processor -- start 2023-10-06T02:22:43.215 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.214+0000 7f026bfff700 1 -- start start 2023-10-06T02:22:43.216 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.215+0000 7f026bfff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f026c153a00 0x7f026c153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:22:43.217 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.215+0000 7f026bfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f026c0cea00 0x7f026c0cee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:22:43.217 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.215+0000 7f026bfff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f026c15ac50 con 0x7f026c15de50 2023-10-06T02:22:43.217 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.215+0000 7f026bfff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f026c15d010 con 0x7f026c0cea00 2023-10-06T02:22:43.217 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.215+0000 7f026bfff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f026c0cf3c0 con 0x7f026c153a00 2023-10-06T02:22:43.217 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.215+0000 7f026affd700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f026c153a00 0x7f026c153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:22:43.217 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.215+0000 7f026affd700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f026c153a00 0x7f026c153dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59996/0 (socket says 172.21.15.62:59996) 2023-10-06T02:22:43.217 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.215+0000 7f026affd700 1 -- 172.21.15.62:0/422182202 learned_addr learned my addr 172.21.15.62:0/422182202 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-10-06T02:22:43.217 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.215+0000 7f026b7fe700 1 --2- 172.21.15.62:0/422182202 >> v2:172.21.15.31:3300/0 conn(0x7f026c0cea00 0x7f026c0cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:22:43.217 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.216+0000 7f026affd700 1 -- 172.21.15.62:0/422182202 >> v2:172.21.15.31:3300/0 conn(0x7f026c0cea00 msgr2=0x7f026c0cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:22:43.217 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.216+0000 7f026affd700 1 --2- 172.21.15.62:0/422182202 >> v2:172.21.15.31:3300/0 conn(0x7f026c0cea00 0x7f026c0cee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:43.217 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.216+0000 7f026affd700 1 -- 172.21.15.62:0/422182202 >> v1:172.21.15.31:6789/0 conn(0x7f026c15de50 legacy=0x7f026c15a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:22:43.218 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.216+0000 7f026affd700 1 -- 172.21.15.62:0/422182202 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f026c15af60 con 0x7f026c153a00 2023-10-06T02:22:43.218 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.216+0000 7f026affd700 1 --2- 172.21.15.62:0/422182202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f026c153a00 0x7f026c153dd0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f026c0cac20 tx=0x7f025c009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7af2cabdc7ef59b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:22:43.219 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.216+0000 7f0269ffb700 1 -- 172.21.15.62:0/422182202 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f025c009e70 con 0x7f026c153a00 2023-10-06T02:22:43.219 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.216+0000 7f0269ffb700 1 -- 172.21.15.62:0/422182202 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f025c00b860 con 0x7f026c153a00 2023-10-06T02:22:43.219 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.216+0000 7f0269ffb700 1 -- 172.21.15.62:0/422182202 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f025c00bc00 con 0x7f026c153a00 2023-10-06T02:22:43.219 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.217+0000 7f026bfff700 1 -- 172.21.15.62:0/422182202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f026c153a00 msgr2=0x7f026c153dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:22:43.219 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.217+0000 7f026bfff700 1 --2- 172.21.15.62:0/422182202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f026c153a00 0x7f026c153dd0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f026c0cac20 tx=0x7f025c009250 comp rx=0 tx=0).stop 2023-10-06T02:22:43.219 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.217+0000 7f026bfff700 1 -- 172.21.15.62:0/422182202 shutdown_connections 2023-10-06T02:22:43.219 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.217+0000 7f026bfff700 1 --2- 172.21.15.62:0/422182202 >> v2:172.21.15.31:3300/0 conn(0x7f026c0cea00 0x7f026c0cee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:43.219 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.217+0000 7f026bfff700 1 --2- 172.21.15.62:0/422182202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f026c153a00 0x7f026c153dd0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:43.219 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.217+0000 7f026bfff700 1 -- 172.21.15.62:0/422182202 >> 172.21.15.62:0/422182202 conn(0x7f026c0703a0 msgr2=0x7f026c0707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:22:43.219 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.217+0000 7f026bfff700 1 -- 172.21.15.62:0/422182202 shutdown_connections 2023-10-06T02:22:43.220 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.217+0000 7f026bfff700 1 -- 172.21.15.62:0/422182202 wait complete. 2023-10-06T02:22:43.220 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.218+0000 7f026bfff700 1 Processor -- start 2023-10-06T02:22:43.221 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.219+0000 7f026bfff700 1 -- start start 2023-10-06T02:22:43.221 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.219+0000 7f026bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f026c0cea00 0x7f026c151290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:22:43.221 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.219+0000 7f026bfff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f026c153a00 0x7f026c0b55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:22:43.221 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.219+0000 7f026bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f026c15de50 0x7f026c0b5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:22:43.221 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.220+0000 7f026bfff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f026c1517d0 con 0x7f026c15de50 2023-10-06T02:22:43.221 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.220+0000 7f026bfff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f026c0b3de0 con 0x7f026c0cea00 2023-10-06T02:22:43.221 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.220+0000 7f026bfff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f026c0c7b80 con 0x7f026c153a00 2023-10-06T02:22:43.221 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:22:43.220+0000 7f65ed07d180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T02:22:43.223 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.220+0000 7f026affd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f026c0cea00 0x7f026c151290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:22:43.223 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.220+0000 7f026affd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f026c0cea00 0x7f026c151290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60008/0 (socket says 172.21.15.62:60008) 2023-10-06T02:22:43.223 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.220+0000 7f026a7fc700 1 --2- 172.21.15.62:0/2076124279 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f026c153a00 0x7f026c0b55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:22:43.223 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.220+0000 7f026affd700 1 -- 172.21.15.62:0/2076124279 learned_addr learned my addr 172.21.15.62:0/2076124279 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-10-06T02:22:43.223 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.220+0000 7f026b7fe700 1 --2- 172.21.15.62:0/2076124279 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f026c15de50 0x7f026c0b5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:22:43.223 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.220+0000 7f026affd700 1 -- 172.21.15.62:0/2076124279 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f026c153a00 msgr2=0x7f026c0b55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:22:43.223 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.220+0000 7f026affd700 1 --2- 172.21.15.62:0/2076124279 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f026c153a00 0x7f026c0b55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:43.223 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.220+0000 7f026affd700 1 -- 172.21.15.62:0/2076124279 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f026c15de50 msgr2=0x7f026c0b5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:22:43.224 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.220+0000 7f026affd700 1 --2- 172.21.15.62:0/2076124279 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f026c15de50 0x7f026c0b5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:43.224 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.220+0000 7f026affd700 1 -- 172.21.15.62:0/2076124279 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f025c0094f0 con 0x7f026c0cea00 2023-10-06T02:22:43.224 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.220+0000 7f026b7fe700 1 --2- 172.21.15.62:0/2076124279 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f026c15de50 0x7f026c0b5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:22:43.224 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.220+0000 7f026a7fc700 1 --2- 172.21.15.62:0/2076124279 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f026c153a00 0x7f026c0b55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:22:43.224 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.221+0000 7f026affd700 1 --2- 172.21.15.62:0/2076124279 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f026c0cea00 0x7f026c151290 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f026c0cf500 tx=0x7f025c017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:22:43.224 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.221+0000 7f0252ffd700 1 -- 172.21.15.62:0/2076124279 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f025c00bbe0 con 0x7f026c0cea00 2023-10-06T02:22:43.224 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.221+0000 7f0252ffd700 1 -- 172.21.15.62:0/2076124279 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f025c011420 con 0x7f026c0cea00 2023-10-06T02:22:43.224 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.221+0000 7f0252ffd700 1 -- 172.21.15.62:0/2076124279 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f025c0117c0 con 0x7f026c0cea00 2023-10-06T02:22:43.224 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.221+0000 7f026bfff700 1 -- 172.21.15.62:0/2076124279 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f026c0c7e60 con 0x7f026c0cea00 2023-10-06T02:22:43.224 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.221+0000 7f026bfff700 1 -- 172.21.15.62:0/2076124279 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f026c0c8420 con 0x7f026c0cea00 2023-10-06T02:22:43.224 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.221+0000 7f0252ffd700 1 -- 172.21.15.62:0/2076124279 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f025c011bf0 con 0x7f026c0cea00 2023-10-06T02:22:43.224 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.221+0000 7f0252ffd700 1 -- 172.21.15.62:0/2076124279 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1411+0+0 (secure 0 0 0) 0x7f025c0239f0 con 0x7f026c0cea00 2023-10-06T02:22:43.225 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.222+0000 7f026bfff700 1 -- 172.21.15.62:0/2076124279 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f026c0b28e0 con 0x7f026c0cea00 2023-10-06T02:22:43.225 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.225+0000 7f0252ffd700 1 -- 172.21.15.62:0/2076124279 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f026c0b28e0 con 0x7f026c0cea00 2023-10-06T02:22:43.327 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.326+0000 7f026bfff700 1 -- 172.21.15.62:0/2076124279 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd new", "uuid": "485507f8-ea88-45d4-a0fd-793ea2d7507b", "id": 6} v 0) v1 -- 0x7f026c04c3c0 con 0x7f026c0cea00 2023-10-06T02:22:43.331 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.330+0000 7f0252ffd700 1 -- 172.21.15.62:0/2076124279 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "485507f8-ea88-45d4-a0fd-793ea2d7507b", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f025c028070 con 0x7f026c0cea00 2023-10-06T02:22:43.331 INFO:teuthology.orchestra.run.smithi062.stdout:6 2023-10-06T02:22:43.332 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.331+0000 7f026bfff700 1 -- 172.21.15.62:0/2076124279 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f026c0cea00 msgr2=0x7f026c151290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:22:43.332 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.331+0000 7f026bfff700 1 --2- 172.21.15.62:0/2076124279 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f026c0cea00 0x7f026c151290 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f026c0cf500 tx=0x7f025c017000 comp rx=0 tx=0).stop 2023-10-06T02:22:43.333 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.331+0000 7f026bfff700 1 -- 172.21.15.62:0/2076124279 shutdown_connections 2023-10-06T02:22:43.333 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.331+0000 7f026bfff700 1 --2- 172.21.15.62:0/2076124279 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f026c15de50 0x7f026c0b5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:43.333 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.331+0000 7f026bfff700 1 --2- 172.21.15.62:0/2076124279 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f026c0cea00 0x7f026c151290 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:43.333 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.331+0000 7f026bfff700 1 --2- 172.21.15.62:0/2076124279 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f026c153a00 0x7f026c0b55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:43.333 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.331+0000 7f026bfff700 1 -- 172.21.15.62:0/2076124279 >> 172.21.15.62:0/2076124279 conn(0x7f026c0703a0 msgr2=0x7f026c0d0ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:22:43.333 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.332+0000 7f026bfff700 1 -- 172.21.15.62:0/2076124279 shutdown_connections 2023-10-06T02:22:43.333 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.332+0000 7f026bfff700 1 -- 172.21.15.62:0/2076124279 wait complete. 2023-10-06T02:22:43.350 DEBUG:teuthology.orchestra.run.smithi062:> sudo ceph --cluster ceph osd new ce5a1b03-1cb6-4cc3-a303-90859e26d517 7 2023-10-06T02:22:43.374 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:22:43.373+0000 7f3f028b9180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T02:22:43.473 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:22:43.472+0000 7f3f028b9180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T02:22:43.530 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.529+0000 7fde8ad9d700 1 Processor -- start 2023-10-06T02:22:43.530 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.529+0000 7fde8ad9d700 1 -- start start 2023-10-06T02:22:43.531 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.530+0000 7fde8ad9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fde84153a00 0x7fde84153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:22:43.531 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.530+0000 7fde8ad9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde8415ac50 con 0x7fde8415de50 2023-10-06T02:22:43.531 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.530+0000 7fde8ad9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde8415d010 con 0x7fde840cea00 2023-10-06T02:22:43.531 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.530+0000 7fde8ad9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde8415af60 con 0x7fde84153a00 2023-10-06T02:22:43.531 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.530+0000 7fde89d9b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fde84153a00 0x7fde84153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:22:43.532 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.530+0000 7fde8a59c700 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fde840cea00 0x7fde840cee40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:40846/0 (socket says 172.21.15.62:40846) 2023-10-06T02:22:43.532 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.530+0000 7fde8a59c700 1 -- 172.21.15.62:0/875297518 learned_addr learned my addr 172.21.15.62:0/875297518 (peer_addr_for_me v1:172.21.15.62:0/0) 2023-10-06T02:22:43.533 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.531+0000 7fde89d9b700 1 -- 172.21.15.62:0/875297518 >> v1:172.21.15.62:6789/0 conn(0x7fde840cea00 legacy=0x7fde840cee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:22:43.533 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.531+0000 7fde88d99700 1 -- 172.21.15.62:0/875297518 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3471191248 0 0) 0x7fde8415d010 con 0x7fde840cea00 2023-10-06T02:22:43.533 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.531+0000 7fde89d9b700 1 -- 172.21.15.62:0/875297518 >> v1:172.21.15.31:6789/0 conn(0x7fde8415de50 legacy=0x7fde8415a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:22:43.533 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.531+0000 7fde89d9b700 1 -- 172.21.15.62:0/875297518 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde8415b240 con 0x7fde84153a00 2023-10-06T02:22:43.533 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.531+0000 7fde89d9b700 1 --2- 172.21.15.62:0/875297518 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fde84153a00 0x7fde84153dd0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fde80009910 tx=0x7fde800091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eed231b298347c34 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:22:43.534 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.531+0000 7fde88d99700 1 -- 172.21.15.62:0/875297518 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde8000b760 con 0x7fde84153a00 2023-10-06T02:22:43.534 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.531+0000 7fde88d99700 1 -- 172.21.15.62:0/875297518 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde8000b8c0 con 0x7fde84153a00 2023-10-06T02:22:43.534 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.532+0000 7fde88d99700 1 -- 172.21.15.62:0/875297518 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde8000bc60 con 0x7fde84153a00 2023-10-06T02:22:43.534 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.532+0000 7fde8ad9d700 1 -- 172.21.15.62:0/875297518 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fde84153a00 msgr2=0x7fde84153dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:22:43.534 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.532+0000 7fde8ad9d700 1 --2- 172.21.15.62:0/875297518 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fde84153a00 0x7fde84153dd0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fde80009910 tx=0x7fde800091a0 comp rx=0 tx=0).stop 2023-10-06T02:22:43.534 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.532+0000 7fde8ad9d700 1 -- 172.21.15.62:0/875297518 shutdown_connections 2023-10-06T02:22:43.534 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.532+0000 7fde8ad9d700 1 --2- 172.21.15.62:0/875297518 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fde84153a00 0x7fde84153dd0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:43.534 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.532+0000 7fde8ad9d700 1 -- 172.21.15.62:0/875297518 >> 172.21.15.62:0/875297518 conn(0x7fde840703a0 msgr2=0x7fde840707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:22:43.534 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.532+0000 7fde8ad9d700 1 -- 172.21.15.62:0/875297518 shutdown_connections 2023-10-06T02:22:43.534 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.532+0000 7fde8ad9d700 1 -- 172.21.15.62:0/875297518 wait complete. 2023-10-06T02:22:43.535 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.534+0000 7fde8ad9d700 1 Processor -- start 2023-10-06T02:22:43.535 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.534+0000 7fde8ad9d700 1 -- start start 2023-10-06T02:22:43.536 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.535+0000 7fde8ad9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fde840cea00 0x7fde8413f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:22:43.536 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.535+0000 7fde8ad9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde84153a00 0x7fde840c7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:22:43.537 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.535+0000 7fde8ad9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde8415de50 0x7fde840c6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:22:43.537 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.535+0000 7fde8ad9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde840cd6c0 con 0x7fde84153a00 2023-10-06T02:22:43.537 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.535+0000 7fde8ad9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde840c8c60 con 0x7fde8415de50 2023-10-06T02:22:43.537 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.535+0000 7fde8ad9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde840cd230 con 0x7fde840cea00 2023-10-06T02:22:43.537 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.535+0000 7fde89d9b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fde840cea00 0x7fde8413f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:22:43.537 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.535+0000 7fde89d9b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fde840cea00 0x7fde8413f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.62:51776/0 (socket says 172.21.15.62:51776) 2023-10-06T02:22:43.537 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.535+0000 7fde89d9b700 1 -- 172.21.15.62:0/3422428051 learned_addr learned my addr 172.21.15.62:0/3422428051 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-10-06T02:22:43.537 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.535+0000 7fde8a59c700 1 --2- 172.21.15.62:0/3422428051 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde8415de50 0x7fde840c6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:22:43.537 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.535+0000 7fde8959a700 1 --2- 172.21.15.62:0/3422428051 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde84153a00 0x7fde840c7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:22:43.537 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.536+0000 7fde89d9b700 1 -- 172.21.15.62:0/3422428051 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde8415de50 msgr2=0x7fde840c6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:22:43.537 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.536+0000 7fde89d9b700 1 --2- 172.21.15.62:0/3422428051 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde8415de50 0x7fde840c6290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:43.537 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.536+0000 7fde89d9b700 1 -- 172.21.15.62:0/3422428051 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde84153a00 msgr2=0x7fde840c7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:22:43.538 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.536+0000 7fde89d9b700 1 --2- 172.21.15.62:0/3422428051 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde84153a00 0x7fde840c7ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:43.538 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.536+0000 7fde89d9b700 1 -- 172.21.15.62:0/3422428051 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde80009420 con 0x7fde840cea00 2023-10-06T02:22:43.538 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.536+0000 7fde89d9b700 1 --2- 172.21.15.62:0/3422428051 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fde840cea00 0x7fde8413f640 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fde800030c0 tx=0x7fde80016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:22:43.539 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.536+0000 7fde79ffb700 1 -- 172.21.15.62:0/3422428051 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde80011c70 con 0x7fde840cea00 2023-10-06T02:22:43.539 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.536+0000 7fde79ffb700 1 -- 172.21.15.62:0/3422428051 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde80011df0 con 0x7fde840cea00 2023-10-06T02:22:43.539 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.536+0000 7fde79ffb700 1 -- 172.21.15.62:0/3422428051 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde8001c6b0 con 0x7fde840cea00 2023-10-06T02:22:43.539 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.536+0000 7fde8ad9d700 1 -- 172.21.15.62:0/3422428051 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde840cc620 con 0x7fde840cea00 2023-10-06T02:22:43.539 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.536+0000 7fde8ad9d700 1 -- 172.21.15.62:0/3422428051 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde840ccbe0 con 0x7fde840cea00 2023-10-06T02:22:43.539 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.537+0000 7fde79ffb700 1 -- 172.21.15.62:0/3422428051 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fde8001cd20 con 0x7fde840cea00 2023-10-06T02:22:43.539 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.537+0000 7fde79ffb700 1 -- 172.21.15.62:0/3422428051 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1526+0+0 (secure 0 0 0) 0x7fde8001a450 con 0x7fde840cea00 2023-10-06T02:22:43.539 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.537+0000 7fde8ad9d700 1 -- 172.21.15.62:0/3422428051 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde8404c3c0 con 0x7fde840cea00 2023-10-06T02:22:43.540 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.540+0000 7fde79ffb700 1 -- 172.21.15.62:0/3422428051 <== mon.2 v2:172.21.15.62:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fde800456d0 con 0x7fde840cea00 2023-10-06T02:22:43.573 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:22:43.572+0000 7f3f028b9180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T02:22:43.629 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.628+0000 7fde8ad9d700 1 -- 172.21.15.62:0/3422428051 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "osd new", "uuid": "ce5a1b03-1cb6-4cc3-a303-90859e26d517", "id": 7} v 0) v1 -- 0x7fde8404c3c0 con 0x7fde840cea00 2023-10-06T02:22:43.637 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.636+0000 7fde79ffb700 1 -- 172.21.15.62:0/3422428051 <== mon.2 v2:172.21.15.62:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ce5a1b03-1cb6-4cc3-a303-90859e26d517", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7fde80034070 con 0x7fde840cea00 2023-10-06T02:22:43.637 INFO:teuthology.orchestra.run.smithi062.stdout:7 2023-10-06T02:22:43.638 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.637+0000 7fde8ad9d700 1 -- 172.21.15.62:0/3422428051 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fde840cea00 msgr2=0x7fde8413f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:22:43.638 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.637+0000 7fde8ad9d700 1 --2- 172.21.15.62:0/3422428051 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fde840cea00 0x7fde8413f640 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fde800030c0 tx=0x7fde80016000 comp rx=0 tx=0).stop 2023-10-06T02:22:43.639 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.637+0000 7fde8ad9d700 1 -- 172.21.15.62:0/3422428051 shutdown_connections 2023-10-06T02:22:43.639 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.637+0000 7fde8ad9d700 1 --2- 172.21.15.62:0/3422428051 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde8415de50 0x7fde840c6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:43.639 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.637+0000 7fde8ad9d700 1 --2- 172.21.15.62:0/3422428051 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fde840cea00 0x7fde8413f640 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:43.639 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.637+0000 7fde8ad9d700 1 --2- 172.21.15.62:0/3422428051 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde84153a00 0x7fde840c7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:43.639 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.637+0000 7fde8ad9d700 1 -- 172.21.15.62:0/3422428051 >> 172.21.15.62:0/3422428051 conn(0x7fde840703a0 msgr2=0x7fde8406d790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:22:43.639 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.637+0000 7fde8ad9d700 1 -- 172.21.15.62:0/3422428051 shutdown_connections 2023-10-06T02:22:43.639 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T02:22:43.637+0000 7fde8ad9d700 1 -- 172.21.15.62:0/3422428051 wait complete. 2023-10-06T02:22:43.654 INFO:tasks.ceph.osd.0:Restarting daemon 2023-10-06T02:22:43.654 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-10-06T02:22:43.659 INFO:tasks.ceph.osd.0:Started 2023-10-06T02:22:43.659 INFO:tasks.ceph.osd.1:Restarting daemon 2023-10-06T02:22:43.659 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-10-06T02:22:43.663 INFO:tasks.ceph.osd.1:Started 2023-10-06T02:22:43.663 INFO:tasks.ceph.osd.2:Restarting daemon 2023-10-06T02:22:43.663 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-10-06T02:22:43.668 INFO:tasks.ceph.osd.2:Started 2023-10-06T02:22:43.668 INFO:tasks.ceph.osd.3:Restarting daemon 2023-10-06T02:22:43.668 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-10-06T02:22:43.673 INFO:tasks.ceph.osd.3:Started 2023-10-06T02:22:43.674 INFO:tasks.ceph.osd.4:Restarting daemon 2023-10-06T02:22:43.674 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-10-06T02:22:43.678 INFO:tasks.ceph.osd.4:Started 2023-10-06T02:22:43.678 INFO:tasks.ceph.osd.5:Restarting daemon 2023-10-06T02:22:43.678 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-10-06T02:22:43.682 INFO:tasks.ceph.osd.5:Started 2023-10-06T02:22:43.682 INFO:tasks.ceph.osd.6:Restarting daemon 2023-10-06T02:22:43.683 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-10-06T02:22:43.687 INFO:tasks.ceph.osd.6:Started 2023-10-06T02:22:43.687 INFO:tasks.ceph.osd.7:Restarting daemon 2023-10-06T02:22:43.687 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-10-06T02:22:43.692 INFO:tasks.ceph.osd.7:Started 2023-10-06T02:22:43.692 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:22:43.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:43.912+0000 7f261df9d700 1 Processor -- start 2023-10-06T02:22:43.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:43.912+0000 7f261df9d700 1 -- start start 2023-10-06T02:22:43.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:43.914+0000 7f261df9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f261806dfa0 0x7f2618164620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:22:43.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:43.914+0000 7f261d79c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f261806dfa0 0x7f2618164620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:22:43.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:43.914+0000 7f261d79c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f261806dfa0 0x7f2618164620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57666/0 (socket says 172.21.15.31:57666) 2023-10-06T02:22:43.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:43.914+0000 7f261d79c700 1 -- 172.21.15.31:0/894743341 learned_addr learned my addr 172.21.15.31:0/894743341 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:22:43.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:43.914+0000 7f261df9d700 1 -- 172.21.15.31:0/894743341 --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26180cec30 con 0x7f26180b2980 2023-10-06T02:22:43.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:43.914+0000 7f261df9d700 1 -- 172.21.15.31:0/894743341 --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26180ce930 con 0x7f261815a8c0 2023-10-06T02:22:43.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:43.914+0000 7f261df9d700 1 -- 172.21.15.31:0/894743341 --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26180cd510 con 0x7f261806dfa0 2023-10-06T02:22:43.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:43.914+0000 7f26177fe700 1 -- 172.21.15.31:0/894743341 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 438227955 0 0) 0x7f26180ce930 con 0x7f261815a8c0 2023-10-06T02:22:43.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:43.914+0000 7f26177fe700 1 -- 172.21.15.31:0/894743341 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2608003570 con 0x7f261815a8c0 2023-10-06T02:22:43.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:43.915+0000 7f26177fe700 1 -- 172.21.15.31:0/894743341 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3978029929 0 0) 0x7f26180cec30 con 0x7f26180b2980 2023-10-06T02:22:43.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:43.915+0000 7f26177fe700 1 -- 172.21.15.31:0/894743341 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26180ce930 con 0x7f26180b2980 2023-10-06T02:22:43.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:43.915+0000 7f261d79c700 1 -- 172.21.15.31:0/894743341 >> v1:172.21.15.62:6789/0 conn(0x7f261815a8c0 legacy=0x7f261806d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:22:43.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:43.915+0000 7f261d79c700 1 -- 172.21.15.31:0/894743341 >> v1:172.21.15.31:6789/0 conn(0x7f26180b2980 legacy=0x7f26180cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:22:43.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:43.915+0000 7f261d79c700 1 -- 172.21.15.31:0/894743341 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26180d00d0 con 0x7f261806dfa0 2023-10-06T02:22:43.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:43.915+0000 7f261d79c700 1 --2- 172.21.15.31:0/894743341 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f261806dfa0 0x7f2618164620 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f2610009f20 tx=0x7f2610009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fa7bc42dbd40cac server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:22:43.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:43.915+0000 7f26177fe700 1 -- 172.21.15.31:0/894743341 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261000e070 con 0x7f261806dfa0 2023-10-06T02:22:43.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:43.915+0000 7f26177fe700 1 -- 172.21.15.31:0/894743341 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f261000b970 con 0x7f261806dfa0 2023-10-06T02:22:43.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:43.915+0000 7f26177fe700 1 -- 172.21.15.31:0/894743341 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26100092e0 con 0x7f261806dfa0 2023-10-06T02:22:43.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:43.916+0000 7f261df9d700 1 -- 172.21.15.31:0/894743341 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f261806dfa0 msgr2=0x7f2618164620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:22:43.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:43.916+0000 7f261df9d700 1 --2- 172.21.15.31:0/894743341 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f261806dfa0 0x7f2618164620 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f2610009f20 tx=0x7f2610009840 comp rx=0 tx=0).stop 2023-10-06T02:22:43.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:43.916+0000 7f261df9d700 1 -- 172.21.15.31:0/894743341 shutdown_connections 2023-10-06T02:22:43.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:43.916+0000 7f261df9d700 1 --2- 172.21.15.31:0/894743341 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f261806dfa0 0x7f2618164620 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:43.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:43.916+0000 7f261df9d700 1 -- 172.21.15.31:0/894743341 >> 172.21.15.31:0/894743341 conn(0x7f26180703a0 msgr2=0x7f26180707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:22:43.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:43.916+0000 7f261df9d700 1 -- 172.21.15.31:0/894743341 shutdown_connections 2023-10-06T02:22:43.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:43.916+0000 7f261df9d700 1 -- 172.21.15.31:0/894743341 wait complete. 2023-10-06T02:22:43.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:43.917+0000 7f261df9d700 1 Processor -- start 2023-10-06T02:22:43.920 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:22:43.917+0000 7f65ed07d180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T02:22:43.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:43.918+0000 7f261df9d700 1 -- start start 2023-10-06T02:22:43.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:43.918+0000 7f261df9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f261806dfa0 0x7f2618169510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:22:43.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:43.918+0000 7f261df9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f26180b2980 0x7f2618141fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:22:43.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:43.918+0000 7f261df9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f261815a8c0 0x7f26180b0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:22:43.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:43.919+0000 7f261df9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f261816a530 con 0x7f261806dfa0 2023-10-06T02:22:43.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:43.919+0000 7f261cf9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f261806dfa0 0x7f2618169510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:22:43.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:43.919+0000 7f261cf9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f261806dfa0 0x7f2618169510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57678/0 (socket says 172.21.15.31:57678) 2023-10-06T02:22:43.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:43.919+0000 7f261cf9b700 1 -- 172.21.15.31:0/921673057 learned_addr learned my addr 172.21.15.31:0/921673057 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:22:43.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:43.919+0000 7f261df9d700 1 -- 172.21.15.31:0/921673057 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2618169a50 con 0x7f261815a8c0 2023-10-06T02:22:43.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:43.919+0000 7f261df9d700 1 -- 172.21.15.31:0/921673057 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26180ca8a0 con 0x7f26180b2980 2023-10-06T02:22:43.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:43.919+0000 7f2617fff700 1 --2- 172.21.15.31:0/921673057 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f26180b2980 0x7f2618141fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:22:43.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:43.919+0000 7f261d79c700 1 --2- 172.21.15.31:0/921673057 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f261815a8c0 0x7f26180b0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:22:43.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:43.919+0000 7f261cf9b700 1 -- 172.21.15.31:0/921673057 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f26180b2980 msgr2=0x7f2618141fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:22:43.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:43.919+0000 7f261cf9b700 1 --2- 172.21.15.31:0/921673057 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f26180b2980 0x7f2618141fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:43.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:43.919+0000 7f261cf9b700 1 -- 172.21.15.31:0/921673057 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f261815a8c0 msgr2=0x7f26180b0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:22:43.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:43.919+0000 7f261cf9b700 1 --2- 172.21.15.31:0/921673057 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f261815a8c0 0x7f26180b0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:43.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:43.919+0000 7f261cf9b700 1 -- 172.21.15.31:0/921673057 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2610009ae0 con 0x7f261806dfa0 2023-10-06T02:22:43.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:43.920+0000 7f261cf9b700 1 --2- 172.21.15.31:0/921673057 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f261806dfa0 0x7f2618169510 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f260c00c790 tx=0x7f260c007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:22:43.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:43.920+0000 7f2614ff9700 1 -- 172.21.15.31:0/921673057 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f260c010510 con 0x7f261806dfa0 2023-10-06T02:22:43.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:43.920+0000 7f2614ff9700 1 -- 172.21.15.31:0/921673057 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f260c010670 con 0x7f261806dfa0 2023-10-06T02:22:43.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:43.920+0000 7f261df9d700 1 -- 172.21.15.31:0/921673057 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26181424f0 con 0x7f261806dfa0 2023-10-06T02:22:43.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:43.920+0000 7f2614ff9700 1 -- 172.21.15.31:0/921673057 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f260c010a10 con 0x7f261806dfa0 2023-10-06T02:22:43.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:43.920+0000 7f261df9d700 1 -- 172.21.15.31:0/921673057 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2618169f70 con 0x7f261806dfa0 2023-10-06T02:22:43.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:43.921+0000 7f2614ff9700 1 -- 172.21.15.31:0/921673057 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f260c01c6c0 con 0x7f261806dfa0 2023-10-06T02:22:43.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:43.921+0000 7f2614ff9700 1 -- 172.21.15.31:0/921673057 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7f260c012b20 con 0x7f261806dfa0 2023-10-06T02:22:43.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:43.921+0000 7f261df9d700 1 -- 172.21.15.31:0/921673057 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f261804c3c0 con 0x7f261806dfa0 2023-10-06T02:22:43.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:43.924+0000 7f2614ff9700 1 -- 172.21.15.31:0/921673057 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f260c017070 con 0x7f261806dfa0 2023-10-06T02:22:43.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:43.997+0000 7f261df9d700 1 -- 172.21.15.31:0/921673057 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f26180ce470 con 0x7f261806dfa0 2023-10-06T02:22:43.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:43.998+0000 7f2614ff9700 1 -- 172.21.15.31:0/921673057 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7013 (secure 0 0 0) 0x7f260c01cba0 con 0x7f261806dfa0 2023-10-06T02:22:43.999 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:22:44.000 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":10,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:22:43.630440+0000","last_up_change":"0.000000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:22:44.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:44.000+0000 7f261df9d700 1 -- 172.21.15.31:0/921673057 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f261806dfa0 msgr2=0x7f2618169510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:22:44.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:44.000+0000 7f261df9d700 1 --2- 172.21.15.31:0/921673057 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f261806dfa0 0x7f2618169510 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f260c00c790 tx=0x7f260c007e10 comp rx=0 tx=0).stop 2023-10-06T02:22:44.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:44.000+0000 7f261df9d700 1 -- 172.21.15.31:0/921673057 shutdown_connections 2023-10-06T02:22:44.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:44.000+0000 7f261df9d700 1 --2- 172.21.15.31:0/921673057 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f261815a8c0 0x7f26180b0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:44.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:44.000+0000 7f261df9d700 1 --2- 172.21.15.31:0/921673057 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f261806dfa0 0x7f2618169510 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:44.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:44.000+0000 7f261df9d700 1 --2- 172.21.15.31:0/921673057 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f26180b2980 0x7f2618141fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:44.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:44.000+0000 7f261df9d700 1 -- 172.21.15.31:0/921673057 >> 172.21.15.31:0/921673057 conn(0x7f26180703a0 msgr2=0x7f2618150070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:22:44.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:44.000+0000 7f261df9d700 1 -- 172.21.15.31:0/921673057 shutdown_connections 2023-10-06T02:22:44.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:44.000+0000 7f261df9d700 1 -- 172.21.15.31:0/921673057 wait complete. 2023-10-06T02:22:44.017 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:22:44.016+0000 7f65ed07d180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T02:22:44.018 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-10-06T02:22:44.019 INFO:tasks.ceph:Waiting for OSDs to come up 2023-10-06T02:22:44.117 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:22:44.116+0000 7f65ed07d180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T02:22:44.352 INFO:tasks.ceph.osd.4.smithi062.stderr:2023-10-06T02:22:44.351+0000 7f9ecf37f540 -1 Falling back to public interface 2023-10-06T02:22:44.354 INFO:tasks.ceph.osd.0.smithi031.stderr:2023-10-06T02:22:44.353+0000 7f4d30ade540 -1 Falling back to public interface 2023-10-06T02:22:44.354 INFO:tasks.ceph.osd.6.smithi062.stderr:2023-10-06T02:22:44.354+0000 7f002317a540 -1 Falling back to public interface 2023-10-06T02:22:44.363 INFO:tasks.ceph.osd.5.smithi062.stderr:2023-10-06T02:22:44.363+0000 7f86c4ee3540 -1 Falling back to public interface 2023-10-06T02:22:44.368 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:22:44.367+0000 7f3f028b9180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T02:22:44.383 INFO:tasks.ceph.osd.1.smithi031.stderr:2023-10-06T02:22:44.382+0000 7fa11d97e540 -1 Falling back to public interface 2023-10-06T02:22:44.387 INFO:tasks.ceph.osd.3.smithi031.stderr:2023-10-06T02:22:44.386+0000 7f1a31269540 -1 Falling back to public interface 2023-10-06T02:22:44.388 INFO:tasks.ceph.osd.1.smithi031.stderr:2023-10-06T02:22:44.387+0000 7fa114bfd700 -1 PosixStack listen unable to listen on v1:0.0.0.0:6815/0: (98) Address already in use 2023-10-06T02:22:44.392 INFO:tasks.ceph.osd.2.smithi031.stderr:2023-10-06T02:22:44.391+0000 7f19b081e540 -1 Falling back to public interface 2023-10-06T02:22:44.418 INFO:tasks.ceph.osd.7.smithi062.stderr:2023-10-06T02:22:44.417+0000 7fda7e346540 -1 Falling back to public interface 2023-10-06T02:22:44.554 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:22:44.553+0000 7f3f028b9180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T02:22:44.675 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:22:44.674+0000 7f3f028b9180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T02:22:44.823 DEBUG:teuthology.orchestra.run.smithi031:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-06T02:22:44.862 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:22:44.861+0000 7f65ed07d180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T02:22:44.984 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:22:44.983+0000 7f3f028b9180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T02:22:44.987 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:44.986+0000 7fe039b9d700 1 Processor -- start 2023-10-06T02:22:44.988 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:44.987+0000 7fe039b9d700 1 -- start start 2023-10-06T02:22:44.990 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:44.989+0000 7fe039b9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe0340d0bf0 0x7fe03415d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:22:44.990 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:44.989+0000 7fe039b9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe03415e280 con 0x7fe034157540 2023-10-06T02:22:44.990 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:44.989+0000 7fe039b9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0340cf050 con 0x7fe034157170 2023-10-06T02:22:44.991 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:44.989+0000 7fe039b9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0340cf330 con 0x7fe0340d0bf0 2023-10-06T02:22:44.991 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:44.989+0000 7fe038b9b700 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fe034157170 0x7fe0340cfb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:36342/0 (socket says 172.21.15.31:36342) 2023-10-06T02:22:44.991 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:44.989+0000 7fe03939c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe0340d0bf0 0x7fe03415d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:22:44.991 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:44.989+0000 7fe038b9b700 1 -- 172.21.15.31:0/582570034 learned_addr learned my addr 172.21.15.31:0/582570034 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:22:44.992 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:44.990+0000 7fe0337fe700 1 -- 172.21.15.31:0/582570034 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 595377977 0 0) 0x7fe0340cf050 con 0x7fe034157170 2023-10-06T02:22:44.992 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:44.990+0000 7fe0337fe700 1 -- 172.21.15.31:0/582570034 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe024003570 con 0x7fe034157170 2023-10-06T02:22:44.993 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:44.990+0000 7fe0337fe700 1 -- 172.21.15.31:0/582570034 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3133115972 0 0) 0x7fe03415e280 con 0x7fe034157540 2023-10-06T02:22:44.993 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:44.990+0000 7fe03939c700 1 -- 172.21.15.31:0/582570034 >> v1:172.21.15.62:6789/0 conn(0x7fe034157170 legacy=0x7fe0340cfb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:22:44.993 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:44.990+0000 7fe03939c700 1 -- 172.21.15.31:0/582570034 >> v1:172.21.15.31:6789/0 conn(0x7fe034157540 legacy=0x7fe0340d04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:22:44.993 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:44.990+0000 7fe03939c700 1 -- 172.21.15.31:0/582570034 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0340cf610 con 0x7fe0340d0bf0 2023-10-06T02:22:44.993 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:44.990+0000 7fe03939c700 1 --2- 172.21.15.31:0/582570034 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe0340d0bf0 0x7fe03415d550 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe02c009e60 tx=0x7fe02c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d20e684a178beeb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:22:44.993 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:44.990+0000 7fe0337fe700 1 -- 172.21.15.31:0/582570034 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe02c00e070 con 0x7fe0340d0bf0 2023-10-06T02:22:44.993 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:44.990+0000 7fe0337fe700 1 -- 172.21.15.31:0/582570034 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe02c00bb60 con 0x7fe0340d0bf0 2023-10-06T02:22:44.993 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:44.990+0000 7fe0337fe700 1 -- 172.21.15.31:0/582570034 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe02c012430 con 0x7fe0340d0bf0 2023-10-06T02:22:44.993 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:44.991+0000 7fe039b9d700 1 -- 172.21.15.31:0/582570034 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe0340d0bf0 msgr2=0x7fe03415d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:22:44.993 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:44.991+0000 7fe039b9d700 1 --2- 172.21.15.31:0/582570034 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe0340d0bf0 0x7fe03415d550 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe02c009e60 tx=0x7fe02c009780 comp rx=0 tx=0).stop 2023-10-06T02:22:44.993 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:44.991+0000 7fe039b9d700 1 -- 172.21.15.31:0/582570034 shutdown_connections 2023-10-06T02:22:44.993 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:44.991+0000 7fe039b9d700 1 --2- 172.21.15.31:0/582570034 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe0340d0bf0 0x7fe03415d550 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:44.994 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:44.991+0000 7fe039b9d700 1 -- 172.21.15.31:0/582570034 >> 172.21.15.31:0/582570034 conn(0x7fe03406da30 msgr2=0x7fe03406de30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:22:44.994 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:44.991+0000 7fe039b9d700 1 -- 172.21.15.31:0/582570034 shutdown_connections 2023-10-06T02:22:44.994 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:44.991+0000 7fe039b9d700 1 -- 172.21.15.31:0/582570034 wait complete. 2023-10-06T02:22:44.996 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:44.993+0000 7fe039b9d700 1 Processor -- start 2023-10-06T02:22:44.996 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:44.993+0000 7fe039b9d700 1 -- start start 2023-10-06T02:22:44.996 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:44.994+0000 7fe039b9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0340d0bf0 0x7fe0340b0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:22:44.996 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:44.994+0000 7fe039b9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe034157170 0x7fe034154390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:22:44.997 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:44.994+0000 7fe039b9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe034157540 0x7fe03415c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:22:44.997 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:44.994+0000 7fe039b9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0340b2090 con 0x7fe034157540 2023-10-06T02:22:44.997 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:44.994+0000 7fe039b9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0340b1db0 con 0x7fe0340d0bf0 2023-10-06T02:22:44.997 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:44.994+0000 7fe039b9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0340c53d0 con 0x7fe034157170 2023-10-06T02:22:44.997 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:44.994+0000 7fe03939c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe034157540 0x7fe03415c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:22:44.997 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:44.994+0000 7fe038b9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0340d0bf0 0x7fe0340b0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:22:44.997 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:44.994+0000 7fe03939c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe034157540 0x7fe03415c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57704/0 (socket says 172.21.15.31:57704) 2023-10-06T02:22:44.997 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:44.994+0000 7fe03939c700 1 -- 172.21.15.31:0/1176303544 learned_addr learned my addr 172.21.15.31:0/1176303544 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:22:44.997 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:44.994+0000 7fe033fff700 1 --2- 172.21.15.31:0/1176303544 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe034157170 0x7fe034154390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:22:44.997 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:44.994+0000 7fe03939c700 1 -- 172.21.15.31:0/1176303544 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe034157170 msgr2=0x7fe034154390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:22:44.997 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:44.994+0000 7fe03939c700 1 --2- 172.21.15.31:0/1176303544 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe034157170 0x7fe034154390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:44.997 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:44.994+0000 7fe03939c700 1 -- 172.21.15.31:0/1176303544 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0340d0bf0 msgr2=0x7fe0340b0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:22:44.998 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:44.994+0000 7fe03939c700 1 --2- 172.21.15.31:0/1176303544 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0340d0bf0 0x7fe0340b0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:44.998 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:44.994+0000 7fe03939c700 1 -- 172.21.15.31:0/1176303544 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe02c009a20 con 0x7fe034157540 2023-10-06T02:22:44.998 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:44.994+0000 7fe038b9b700 1 --2- 172.21.15.31:0/1176303544 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0340d0bf0 0x7fe0340b0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:22:44.998 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:44.995+0000 7fe03939c700 1 --2- 172.21.15.31:0/1176303544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe034157540 0x7fe03415c750 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe02c013000 tx=0x7fe02c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:22:44.998 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:44.995+0000 7fe030ff9700 1 -- 172.21.15.31:0/1176303544 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe02c026480 con 0x7fe034157540 2023-10-06T02:22:44.998 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:44.995+0000 7fe030ff9700 1 -- 172.21.15.31:0/1176303544 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe02c026640 con 0x7fe034157540 2023-10-06T02:22:44.998 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:44.995+0000 7fe030ff9700 1 -- 172.21.15.31:0/1176303544 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe02c026a00 con 0x7fe034157540 2023-10-06T02:22:44.998 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:44.995+0000 7fe039b9d700 1 -- 172.21.15.31:0/1176303544 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0340c56b0 con 0x7fe034157540 2023-10-06T02:22:44.998 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:44.995+0000 7fe039b9d700 1 -- 172.21.15.31:0/1176303544 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0340b0e80 con 0x7fe034157540 2023-10-06T02:22:44.998 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:44.995+0000 7fe030ff9700 1 -- 172.21.15.31:0/1176303544 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fe02c01c400 con 0x7fe034157540 2023-10-06T02:22:44.998 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:44.995+0000 7fe030ff9700 1 -- 172.21.15.31:0/1176303544 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7fe02c026db0 con 0x7fe034157540 2023-10-06T02:22:44.999 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:44.996+0000 7fe039b9d700 1 -- 172.21.15.31:0/1176303544 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe034088a50 con 0x7fe034157540 2023-10-06T02:22:44.999 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:44.999+0000 7fe030ff9700 1 -- 172.21.15.31:0/1176303544 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fe02c0466d0 con 0x7fe034157540 2023-10-06T02:22:45.040 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:22:45.039+0000 7f65ed07d180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T02:22:45.074 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:45.073+0000 7fe039b9d700 1 -- 172.21.15.31:0/1176303544 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe0340cec70 con 0x7fe034157540 2023-10-06T02:22:45.075 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:45.074+0000 7fe030ff9700 1 -- 172.21.15.31:0/1176303544 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7013 (secure 0 0 0) 0x7fe02c035030 con 0x7fe034157540 2023-10-06T02:22:45.075 INFO:teuthology.misc.health.smithi031.stdout: 2023-10-06T02:22:45.075 INFO:teuthology.misc.health.smithi031.stdout:{"epoch":10,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:22:43.630440+0000","last_up_change":"0.000000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:22:45.076 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:45.075+0000 7fe039b9d700 1 -- 172.21.15.31:0/1176303544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe034157540 msgr2=0x7fe03415c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:22:45.076 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:45.075+0000 7fe039b9d700 1 --2- 172.21.15.31:0/1176303544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe034157540 0x7fe03415c750 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe02c013000 tx=0x7fe02c019000 comp rx=0 tx=0).stop 2023-10-06T02:22:45.076 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:45.075+0000 7fe039b9d700 1 -- 172.21.15.31:0/1176303544 shutdown_connections 2023-10-06T02:22:45.076 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:45.075+0000 7fe039b9d700 1 --2- 172.21.15.31:0/1176303544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe034157540 0x7fe03415c750 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:45.076 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:45.075+0000 7fe039b9d700 1 --2- 172.21.15.31:0/1176303544 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0340d0bf0 0x7fe0340b0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:45.077 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:45.075+0000 7fe039b9d700 1 --2- 172.21.15.31:0/1176303544 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe034157170 0x7fe034154390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:45.077 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:45.075+0000 7fe039b9d700 1 -- 172.21.15.31:0/1176303544 >> 172.21.15.31:0/1176303544 conn(0x7fe03406da30 msgr2=0x7fe03415d390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:22:45.077 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:45.075+0000 7fe039b9d700 1 -- 172.21.15.31:0/1176303544 shutdown_connections 2023-10-06T02:22:45.077 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:45.075+0000 7fe039b9d700 1 -- 172.21.15.31:0/1176303544 wait complete. 2023-10-06T02:22:45.087 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:22:45.086+0000 7f3f028b9180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T02:22:45.090 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-10-06T02:22:45.160 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:22:45.159+0000 7f65ed07d180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T02:22:45.466 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:22:45.466+0000 7f65ed07d180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T02:22:45.567 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:22:45.567+0000 7f65ed07d180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T02:22:46.116 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:22:46.115+0000 7f3f028b9180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T02:22:46.212 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:22:46.211+0000 7f3f028b9180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T02:22:46.309 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:22:46.308+0000 7f3f028b9180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T02:22:46.478 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:22:46.476+0000 7f3f028b9180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T02:22:46.587 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:22:46.585+0000 7f3f028b9180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T02:22:46.600 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:22:46.599+0000 7f65ed07d180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T02:22:46.697 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:22:46.696+0000 7f65ed07d180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T02:22:46.783 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:22:46.782+0000 7f3f028b9180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T02:22:46.801 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:22:46.801+0000 7f65ed07d180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T02:22:46.905 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:22:46.903+0000 7f3f028b9180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T02:22:46.982 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:22:46.981+0000 7f65ed07d180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T02:22:47.002 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:22:47.001+0000 7f3f028b9180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T02:22:47.100 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:22:47.099+0000 7f65ed07d180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T02:22:47.101 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:22:47.100+0000 7f3f028b9180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T02:22:47.341 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:22:47.341+0000 7f65ed07d180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T02:22:47.441 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:22:47.440+0000 7f65ed07d180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T02:22:47.538 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:22:47.537+0000 7f65ed07d180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T02:22:47.643 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:22:47.643+0000 7f65ed07d180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T02:22:47.724 INFO:tasks.ceph.osd.7.smithi062.stderr:2023-10-06T02:22:47.723+0000 7fda7e346540 -1 osd.7 0 log_to_monitors true 2023-10-06T02:22:47.875 INFO:tasks.ceph.osd.4.smithi062.stderr:2023-10-06T02:22:47.874+0000 7f9ecf37f540 -1 osd.4 0 log_to_monitors true 2023-10-06T02:22:47.903 INFO:tasks.ceph.osd.0.smithi031.stderr:2023-10-06T02:22:47.902+0000 7f4d30ade540 -1 osd.0 0 log_to_monitors true 2023-10-06T02:22:47.939 INFO:tasks.ceph.osd.6.smithi062.stderr:2023-10-06T02:22:47.938+0000 7f002317a540 -1 osd.6 0 log_to_monitors true 2023-10-06T02:22:47.942 INFO:tasks.ceph.osd.5.smithi062.stderr:2023-10-06T02:22:47.941+0000 7f86c4ee3540 -1 osd.5 0 log_to_monitors true 2023-10-06T02:22:47.943 INFO:tasks.ceph.osd.1.smithi031.stderr:2023-10-06T02:22:47.942+0000 7fa11d97e540 -1 osd.1 0 log_to_monitors true 2023-10-06T02:22:47.967 INFO:tasks.ceph.osd.3.smithi031.stderr:2023-10-06T02:22:47.966+0000 7f1a31269540 -1 osd.3 0 log_to_monitors true 2023-10-06T02:22:47.971 INFO:tasks.ceph.osd.2.smithi031.stderr:2023-10-06T02:22:47.970+0000 7f19b081e540 -1 osd.2 0 log_to_monitors true 2023-10-06T02:22:48.120 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:22:48.119+0000 7f3eba9fa700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-10-06T02:22:48.121 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:22:48.119+0000 7f3eba9fa700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-10-06T02:22:51.295 INFO:tasks.ceph.osd.5.smithi062.stderr:2023-10-06T02:22:51.294+0000 7f86ba1f9700 -1 osd.5 0 waiting for initial osdmap 2023-10-06T02:22:51.302 INFO:tasks.ceph.osd.5.smithi062.stderr:2023-10-06T02:22:51.301+0000 7f86b19e8700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T02:22:51.314 INFO:tasks.ceph.osd.6.smithi062.stderr:2023-10-06T02:22:51.314+0000 7f0016bf6700 -1 osd.6 0 waiting for initial osdmap 2023-10-06T02:22:51.321 INFO:tasks.ceph.osd.6.smithi062.stderr:2023-10-06T02:22:51.320+0000 7f000fbe8700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T02:22:51.378 INFO:tasks.ceph.osd.7.smithi062.stderr:2023-10-06T02:22:51.377+0000 7fda735f9700 -1 osd.7 0 waiting for initial osdmap 2023-10-06T02:22:51.387 INFO:tasks.ceph.osd.7.smithi062.stderr:2023-10-06T02:22:51.386+0000 7fda6ade8700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T02:22:51.545 INFO:tasks.ceph.osd.3.smithi031.stderr:2023-10-06T02:22:51.544+0000 7f1a24df6700 -1 osd.3 0 waiting for initial osdmap 2023-10-06T02:22:51.553 INFO:tasks.ceph.osd.0.smithi031.stderr:2023-10-06T02:22:51.552+0000 7f4d245f6700 -1 osd.0 0 waiting for initial osdmap 2023-10-06T02:22:51.553 INFO:tasks.ceph.osd.3.smithi031.stderr:2023-10-06T02:22:51.552+0000 7f1a1dde8700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T02:22:51.558 INFO:tasks.ceph.osd.1.smithi031.stderr:2023-10-06T02:22:51.557+0000 7fa1113f6700 -1 osd.1 0 waiting for initial osdmap 2023-10-06T02:22:51.562 INFO:tasks.ceph.osd.0.smithi031.stderr:2023-10-06T02:22:51.561+0000 7f4d1d5e8700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T02:22:51.565 INFO:tasks.ceph.osd.1.smithi031.stderr:2023-10-06T02:22:51.564+0000 7fa10a3e8700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T02:22:51.587 INFO:tasks.ceph.osd.2.smithi031.stderr:2023-10-06T02:22:51.586+0000 7f19a43f6700 -1 osd.2 0 waiting for initial osdmap 2023-10-06T02:22:51.592 INFO:tasks.ceph.osd.4.smithi062.stderr:2023-10-06T02:22:51.592+0000 7f9ec45f9700 -1 osd.4 0 waiting for initial osdmap 2023-10-06T02:22:51.594 INFO:tasks.ceph.osd.2.smithi031.stderr:2023-10-06T02:22:51.593+0000 7f199d3e8700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T02:22:51.598 INFO:tasks.ceph.osd.4.smithi062.stderr:2023-10-06T02:22:51.597+0000 7f9ebbde8700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T02:22:51.896 DEBUG:teuthology.orchestra.run.smithi031:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-06T02:22:52.057 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:52.055+0000 7fda6239d700 1 Processor -- start 2023-10-06T02:22:52.057 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:52.055+0000 7fda6239d700 1 -- start start 2023-10-06T02:22:52.060 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:52.057+0000 7fda6239d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fda5c155a10 0x7fda5c06a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:22:52.060 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:52.057+0000 7fda6239d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fda5c155de0 0x7fda5c06a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:22:52.061 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:52.057+0000 7fda6239d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda5c06aef0 con 0x7fda5c06e350 2023-10-06T02:22:52.061 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:52.057+0000 7fda6239d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda5c06b1d0 con 0x7fda5c155a10 2023-10-06T02:22:52.061 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:52.057+0000 7fda6239d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda5c06b4b0 con 0x7fda5c155de0 2023-10-06T02:22:52.061 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:52.057+0000 7fda61b9c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fda5c06e350 0x7fda5c159830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56496/0 (socket says 172.21.15.31:56496) 2023-10-06T02:22:52.061 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:52.057+0000 7fda6139b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fda5c155a10 0x7fda5c06a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:22:52.061 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:52.057+0000 7fda61b9c700 1 -- 172.21.15.31:0/1481701156 learned_addr learned my addr 172.21.15.31:0/1481701156 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:22:52.061 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:52.057+0000 7fda60b9a700 1 --2- 172.21.15.31:0/1481701156 >> v2:172.21.15.62:3300/0 conn(0x7fda5c155de0 0x7fda5c06a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:22:52.061 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:52.058+0000 7fda53fff700 1 -- 172.21.15.31:0/1481701156 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1851866680 0 0) 0x7fda5c06aef0 con 0x7fda5c06e350 2023-10-06T02:22:52.062 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:52.058+0000 7fda6139b700 1 -- 172.21.15.31:0/1481701156 >> v2:172.21.15.62:3300/0 conn(0x7fda5c155de0 msgr2=0x7fda5c06a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:22:52.062 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:52.058+0000 7fda6139b700 1 --2- 172.21.15.31:0/1481701156 >> v2:172.21.15.62:3300/0 conn(0x7fda5c155de0 0x7fda5c06a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:52.062 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:52.058+0000 7fda6139b700 1 -- 172.21.15.31:0/1481701156 >> v1:172.21.15.31:6789/0 conn(0x7fda5c06e350 legacy=0x7fda5c159830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:22:52.062 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:52.058+0000 7fda6139b700 1 -- 172.21.15.31:0/1481701156 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda5c06b790 con 0x7fda5c155a10 2023-10-06T02:22:52.062 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:52.058+0000 7fda6139b700 1 --2- 172.21.15.31:0/1481701156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda5c155a10 0x7fda5c06a3a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fda5c0c31e0 tx=0x7fda580091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a16d22c58259d05d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:22:52.062 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:52.058+0000 7fda53fff700 1 -- 172.21.15.31:0/1481701156 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda58011630 con 0x7fda5c155a10 2023-10-06T02:22:52.062 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:52.058+0000 7fda53fff700 1 -- 172.21.15.31:0/1481701156 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda58011790 con 0x7fda5c155a10 2023-10-06T02:22:52.063 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:52.058+0000 7fda53fff700 1 -- 172.21.15.31:0/1481701156 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda58011b30 con 0x7fda5c155a10 2023-10-06T02:22:52.063 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:52.059+0000 7fda6239d700 1 -- 172.21.15.31:0/1481701156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda5c155a10 msgr2=0x7fda5c06a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:22:52.063 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:52.059+0000 7fda6239d700 1 --2- 172.21.15.31:0/1481701156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda5c155a10 0x7fda5c06a3a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fda5c0c31e0 tx=0x7fda580091a0 comp rx=0 tx=0).stop 2023-10-06T02:22:52.063 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:52.059+0000 7fda6239d700 1 -- 172.21.15.31:0/1481701156 shutdown_connections 2023-10-06T02:22:52.063 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:52.059+0000 7fda6239d700 1 --2- 172.21.15.31:0/1481701156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda5c155a10 0x7fda5c06a3a0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:52.063 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:52.059+0000 7fda6239d700 1 --2- 172.21.15.31:0/1481701156 >> v2:172.21.15.62:3300/0 conn(0x7fda5c155de0 0x7fda5c06a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:52.063 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:52.059+0000 7fda6239d700 1 -- 172.21.15.31:0/1481701156 >> 172.21.15.31:0/1481701156 conn(0x7fda5c06d060 msgr2=0x7fda5c0cc180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:22:52.063 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:52.059+0000 7fda6239d700 1 -- 172.21.15.31:0/1481701156 shutdown_connections 2023-10-06T02:22:52.063 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:52.059+0000 7fda6239d700 1 -- 172.21.15.31:0/1481701156 wait complete. 2023-10-06T02:22:52.067 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:52.060+0000 7fda6239d700 1 Processor -- start 2023-10-06T02:22:52.067 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:52.060+0000 7fda6239d700 1 -- start start 2023-10-06T02:22:52.067 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:52.061+0000 7fda6239d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda5c06e350 0x7fda5c0c5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:22:52.067 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:52.061+0000 7fda6239d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fda5c155a10 0x7fda5c144f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:22:52.067 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:52.061+0000 7fda6239d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda5c155de0 0x7fda5c148dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:22:52.068 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:52.061+0000 7fda6239d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda5c0c4140 con 0x7fda5c06e350 2023-10-06T02:22:52.068 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:52.061+0000 7fda6239d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda5c0c4420 con 0x7fda5c155de0 2023-10-06T02:22:52.068 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:52.061+0000 7fda6239d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda5c0c4700 con 0x7fda5c155a10 2023-10-06T02:22:52.068 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:52.062+0000 7fda6139b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda5c06e350 0x7fda5c0c5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:22:52.068 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:52.062+0000 7fda6139b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda5c06e350 0x7fda5c0c5e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51128/0 (socket says 172.21.15.31:51128) 2023-10-06T02:22:52.068 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:52.062+0000 7fda6139b700 1 -- 172.21.15.31:0/234756951 learned_addr learned my addr 172.21.15.31:0/234756951 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:22:52.068 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:52.062+0000 7fda60b9a700 1 --2- 172.21.15.31:0/234756951 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fda5c155a10 0x7fda5c144f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:22:52.068 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:52.062+0000 7fda61b9c700 1 --2- 172.21.15.31:0/234756951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda5c155de0 0x7fda5c148dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:22:52.068 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:52.062+0000 7fda6139b700 1 -- 172.21.15.31:0/234756951 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fda5c155a10 msgr2=0x7fda5c144f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:22:52.068 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:52.062+0000 7fda6139b700 1 --2- 172.21.15.31:0/234756951 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fda5c155a10 0x7fda5c144f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:52.069 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:52.062+0000 7fda6139b700 1 -- 172.21.15.31:0/234756951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda5c155de0 msgr2=0x7fda5c148dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:22:52.069 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:52.062+0000 7fda6139b700 1 --2- 172.21.15.31:0/234756951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda5c155de0 0x7fda5c148dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:52.069 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:52.062+0000 7fda6139b700 1 -- 172.21.15.31:0/234756951 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda58009420 con 0x7fda5c06e350 2023-10-06T02:22:52.069 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:52.062+0000 7fda6139b700 1 --2- 172.21.15.31:0/234756951 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda5c06e350 0x7fda5c0c5e80 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fda5c06d5a0 tx=0x7fda58017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:22:52.069 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:52.062+0000 7fda517fa700 1 -- 172.21.15.31:0/234756951 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda58000d10 con 0x7fda5c06e350 2023-10-06T02:22:52.069 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:52.063+0000 7fda517fa700 1 -- 172.21.15.31:0/234756951 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda58011b10 con 0x7fda5c06e350 2023-10-06T02:22:52.069 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:52.063+0000 7fda6239d700 1 -- 172.21.15.31:0/234756951 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda5c0c49e0 con 0x7fda5c06e350 2023-10-06T02:22:52.069 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:52.063+0000 7fda517fa700 1 -- 172.21.15.31:0/234756951 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda58025640 con 0x7fda5c06e350 2023-10-06T02:22:52.069 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:52.063+0000 7fda6239d700 1 -- 172.21.15.31:0/234756951 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda5c0c51e0 con 0x7fda5c06e350 2023-10-06T02:22:52.069 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:52.064+0000 7fda6239d700 1 -- 172.21.15.31:0/234756951 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda5c068a60 con 0x7fda5c06e350 2023-10-06T02:22:52.070 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:52.064+0000 7fda517fa700 1 -- 172.21.15.31:0/234756951 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7fda58025a30 con 0x7fda5c06e350 2023-10-06T02:22:52.070 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:52.065+0000 7fda517fa700 1 --2- 172.21.15.31:0/234756951 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fda40066ea0 0x7fda40069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:22:52.070 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:52.065+0000 7fda517fa700 1 -- 172.21.15.31:0/234756951 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2227+0+0 (secure 0 0 0) 0x7fda58084ed0 con 0x7fda5c06e350 2023-10-06T02:22:52.070 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:52.066+0000 7fda60b9a700 1 --2- 172.21.15.31:0/234756951 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fda40066ea0 0x7fda40069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:22:52.070 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:52.066+0000 7fda60b9a700 1 --2- 172.21.15.31:0/234756951 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fda40066ea0 0x7fda40069350 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fda5c06ae20 tx=0x7fda4c0091a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:22:52.071 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:52.070+0000 7fda517fa700 1 -- 172.21.15.31:0/234756951 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fda58052f10 con 0x7fda5c06e350 2023-10-06T02:22:52.278 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:52.277+0000 7fda6239d700 1 -- 172.21.15.31:0/234756951 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fda5c06f330 con 0x7fda5c06e350 2023-10-06T02:22:52.279 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:52.278+0000 7fda517fa700 1 -- 172.21.15.31:0/234756951 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7013 (secure 0 0 0) 0x7fda58015020 con 0x7fda5c06e350 2023-10-06T02:22:52.279 INFO:teuthology.misc.health.smithi031.stdout: 2023-10-06T02:22:52.279 INFO:teuthology.misc.health.smithi031.stdout:{"epoch":12,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:22:49.179921+0000","last_up_change":"0.000000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:22:52.282 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:52.280+0000 7fda6239d700 1 -- 172.21.15.31:0/234756951 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fda40066ea0 msgr2=0x7fda40069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:22:52.282 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:52.280+0000 7fda6239d700 1 --2- 172.21.15.31:0/234756951 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fda40066ea0 0x7fda40069350 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fda5c06ae20 tx=0x7fda4c0091a0 comp rx=0 tx=0).stop 2023-10-06T02:22:52.282 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:52.280+0000 7fda6239d700 1 -- 172.21.15.31:0/234756951 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda5c06e350 msgr2=0x7fda5c0c5e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:22:52.283 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:52.280+0000 7fda6239d700 1 --2- 172.21.15.31:0/234756951 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda5c06e350 0x7fda5c0c5e80 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fda5c06d5a0 tx=0x7fda58017000 comp rx=0 tx=0).stop 2023-10-06T02:22:52.283 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:52.280+0000 7fda6239d700 1 -- 172.21.15.31:0/234756951 shutdown_connections 2023-10-06T02:22:52.283 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:52.280+0000 7fda6239d700 1 --2- 172.21.15.31:0/234756951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda5c155de0 0x7fda5c148dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:52.283 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:52.280+0000 7fda6239d700 1 --2- 172.21.15.31:0/234756951 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda5c06e350 0x7fda5c0c5e80 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:52.283 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:52.280+0000 7fda6239d700 1 --2- 172.21.15.31:0/234756951 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fda40066ea0 0x7fda40069350 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:52.283 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:52.280+0000 7fda6239d700 1 --2- 172.21.15.31:0/234756951 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fda5c155a10 0x7fda5c144f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:52.283 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:52.280+0000 7fda6239d700 1 -- 172.21.15.31:0/234756951 >> 172.21.15.31:0/234756951 conn(0x7fda5c06d060 msgr2=0x7fda5c0c9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:22:52.283 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:52.280+0000 7fda6239d700 1 -- 172.21.15.31:0/234756951 shutdown_connections 2023-10-06T02:22:52.283 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:52.281+0000 7fda6239d700 1 -- 172.21.15.31:0/234756951 wait complete. 2023-10-06T02:22:52.296 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-10-06T02:22:59.193 DEBUG:teuthology.orchestra.run.smithi031:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-06T02:22:59.366 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:59.364+0000 7ff229f9d700 1 Processor -- start 2023-10-06T02:22:59.366 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:59.365+0000 7ff229f9d700 1 -- start start 2023-10-06T02:22:59.367 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:59.366+0000 7ff229f9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff224159b60 0x7ff2240cdb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:22:59.367 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:59.366+0000 7ff229f9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2240cd0d0 con 0x7ff224159790 2023-10-06T02:22:59.367 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:59.366+0000 7ff229f9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2240cd690 con 0x7ff22406c5c0 2023-10-06T02:22:59.368 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:59.366+0000 7ff229f9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff22406cb30 con 0x7ff224159b60 2023-10-06T02:22:59.368 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:59.366+0000 7ff223fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff224159b60 0x7ff2240cdb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:22:59.368 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:59.366+0000 7ff228f9b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7ff224159790 0x7ff2240b2a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:39200/0 (socket says 172.21.15.31:39200) 2023-10-06T02:22:59.368 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:59.366+0000 7ff223fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff224159b60 0x7ff2240cdb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56742/0 (socket says 172.21.15.31:56742) 2023-10-06T02:22:59.368 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:59.366+0000 7ff228f9b700 1 -- 172.21.15.31:0/3374058362 learned_addr learned my addr 172.21.15.31:0/3374058362 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:22:59.369 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:59.367+0000 7ff223fff700 1 -- 172.21.15.31:0/3374058362 >> v1:172.21.15.62:6789/0 conn(0x7ff22406c5c0 legacy=0x7ff224067c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:22:59.369 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:59.367+0000 7ff223fff700 1 -- 172.21.15.31:0/3374058362 >> v1:172.21.15.31:6789/0 conn(0x7ff224159790 legacy=0x7ff2240b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:22:59.370 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:59.367+0000 7ff223fff700 1 -- 172.21.15.31:0/3374058362 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff22406ce10 con 0x7ff224159b60 2023-10-06T02:22:59.370 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:59.367+0000 7ff2237fe700 1 -- 172.21.15.31:0/3374058362 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3392237409 0 0) 0x7ff2240cd0d0 con 0x7ff224159790 2023-10-06T02:22:59.370 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:59.367+0000 7ff223fff700 1 --2- 172.21.15.31:0/3374058362 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff224159b60 0x7ff2240cdb20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff210009910 tx=0x7ff2100091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b63769ee873c728a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:22:59.370 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:59.367+0000 7ff2237fe700 1 -- 172.21.15.31:0/3374058362 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff21000b820 con 0x7ff224159b60 2023-10-06T02:22:59.370 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:59.367+0000 7ff2237fe700 1 -- 172.21.15.31:0/3374058362 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff21000b980 con 0x7ff224159b60 2023-10-06T02:22:59.370 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:59.367+0000 7ff2237fe700 1 -- 172.21.15.31:0/3374058362 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff21000bd20 con 0x7ff224159b60 2023-10-06T02:22:59.370 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:59.368+0000 7ff229f9d700 1 -- 172.21.15.31:0/3374058362 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff224159b60 msgr2=0x7ff2240cdb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:22:59.370 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:59.368+0000 7ff229f9d700 1 --2- 172.21.15.31:0/3374058362 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff224159b60 0x7ff2240cdb20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff210009910 tx=0x7ff2100091a0 comp rx=0 tx=0).stop 2023-10-06T02:22:59.371 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:59.368+0000 7ff229f9d700 1 -- 172.21.15.31:0/3374058362 shutdown_connections 2023-10-06T02:22:59.371 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:59.368+0000 7ff229f9d700 1 --2- 172.21.15.31:0/3374058362 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff224159b60 0x7ff2240cdb20 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:59.371 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:59.368+0000 7ff229f9d700 1 -- 172.21.15.31:0/3374058362 >> 172.21.15.31:0/3374058362 conn(0x7ff224068810 msgr2=0x7ff224068c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:22:59.371 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:59.368+0000 7ff229f9d700 1 -- 172.21.15.31:0/3374058362 shutdown_connections 2023-10-06T02:22:59.371 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:59.368+0000 7ff229f9d700 1 -- 172.21.15.31:0/3374058362 wait complete. 2023-10-06T02:22:59.373 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:59.369+0000 7ff229f9d700 1 Processor -- start 2023-10-06T02:22:59.374 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:59.370+0000 7ff229f9d700 1 -- start start 2023-10-06T02:22:59.374 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:59.370+0000 7ff229f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff22406c5c0 0x7ff224141e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:22:59.374 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:59.371+0000 7ff229f9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff224159790 0x7ff22415b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:22:59.374 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:59.371+0000 7ff229f9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff224159b60 0x7ff2240ab550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:22:59.374 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:59.371+0000 7ff229f9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff224154bd0 con 0x7ff224159b60 2023-10-06T02:22:59.374 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:59.371+0000 7ff229f9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2241542a0 con 0x7ff224159790 2023-10-06T02:22:59.374 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:59.371+0000 7ff229f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff224154580 con 0x7ff22406c5c0 2023-10-06T02:22:59.375 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:59.371+0000 7ff22979c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff224159b60 0x7ff2240ab550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:22:59.375 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:59.371+0000 7ff22979c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff224159b60 0x7ff2240ab550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56744/0 (socket says 172.21.15.31:56744) 2023-10-06T02:22:59.375 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:59.371+0000 7ff22979c700 1 -- 172.21.15.31:0/3848005273 learned_addr learned my addr 172.21.15.31:0/3848005273 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:22:59.375 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:59.371+0000 7ff223fff700 1 --2- 172.21.15.31:0/3848005273 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff224159790 0x7ff22415b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:22:59.375 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:59.371+0000 7ff228f9b700 1 --2- 172.21.15.31:0/3848005273 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff22406c5c0 0x7ff224141e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:22:59.375 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:59.371+0000 7ff22979c700 1 -- 172.21.15.31:0/3848005273 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff22406c5c0 msgr2=0x7ff224141e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:22:59.375 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:59.371+0000 7ff22979c700 1 --2- 172.21.15.31:0/3848005273 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff22406c5c0 0x7ff224141e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:59.376 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:59.371+0000 7ff22979c700 1 -- 172.21.15.31:0/3848005273 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff224159790 msgr2=0x7ff22415b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:22:59.376 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:59.371+0000 7ff22979c700 1 --2- 172.21.15.31:0/3848005273 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff224159790 0x7ff22415b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:59.376 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:59.371+0000 7ff22979c700 1 -- 172.21.15.31:0/3848005273 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff210009420 con 0x7ff224159b60 2023-10-06T02:22:59.376 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:59.371+0000 7ff223fff700 1 --2- 172.21.15.31:0/3848005273 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff224159790 0x7ff22415b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:22:59.376 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:59.371+0000 7ff22979c700 1 --2- 172.21.15.31:0/3848005273 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff224159b60 0x7ff2240ab550 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff21c00c8e0 tx=0x7ff21c001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:22:59.376 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:59.372+0000 7ff220ff9700 1 -- 172.21.15.31:0/3848005273 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff21c00ce80 con 0x7ff224159b60 2023-10-06T02:22:59.376 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:59.372+0000 7ff220ff9700 1 -- 172.21.15.31:0/3848005273 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff21c00a7e0 con 0x7ff224159b60 2023-10-06T02:22:59.376 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:59.372+0000 7ff229f9d700 1 -- 172.21.15.31:0/3848005273 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff224154860 con 0x7ff224159b60 2023-10-06T02:22:59.376 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:59.372+0000 7ff220ff9700 1 -- 172.21.15.31:0/3848005273 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff21c00ab80 con 0x7ff224159b60 2023-10-06T02:22:59.377 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:59.372+0000 7ff229f9d700 1 -- 172.21.15.31:0/3848005273 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff224153f50 con 0x7ff224159b60 2023-10-06T02:22:59.377 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:59.373+0000 7ff229f9d700 1 -- 172.21.15.31:0/3848005273 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff224069ea0 con 0x7ff224159b60 2023-10-06T02:22:59.377 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:59.374+0000 7ff220ff9700 1 -- 172.21.15.31:0/3848005273 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff21c012420 con 0x7ff224159b60 2023-10-06T02:22:59.377 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:59.374+0000 7ff220ff9700 1 --2- 172.21.15.31:0/3848005273 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7ff208066f30 0x7ff2080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:22:59.378 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:59.375+0000 7ff220ff9700 1 -- 172.21.15.31:0/3848005273 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4853+0+0 (secure 0 0 0) 0x7ff21c084570 con 0x7ff224159b60 2023-10-06T02:22:59.378 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:59.375+0000 7ff228f9b700 1 --2- 172.21.15.31:0/3848005273 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7ff208066f30 0x7ff2080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:22:59.378 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:59.375+0000 7ff228f9b700 1 --2- 172.21.15.31:0/3848005273 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7ff208066f30 0x7ff2080693e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff2180011e0 tx=0x7ff218009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:22:59.381 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:59.380+0000 7ff220ff9700 1 -- 172.21.15.31:0/3848005273 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff21c051a90 con 0x7ff224159b60 2023-10-06T02:22:59.588 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:59.587+0000 7ff229f9d700 1 -- 172.21.15.31:0/3848005273 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff2240cd320 con 0x7ff224159b60 2023-10-06T02:22:59.589 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:59.588+0000 7ff220ff9700 1 -- 172.21.15.31:0/3848005273 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12079 (secure 0 0 0) 0x7ff21c0550b0 con 0x7ff224159b60 2023-10-06T02:22:59.590 INFO:teuthology.misc.health.smithi031.stdout: 2023-10-06T02:22:59.590 INFO:teuthology.misc.health.smithi031.stdout:{"epoch":16,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:22:55.967512+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:22:59.592 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:59.590+0000 7ff229f9d700 1 -- 172.21.15.31:0/3848005273 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7ff208066f30 msgr2=0x7ff2080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:22:59.592 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:59.590+0000 7ff229f9d700 1 --2- 172.21.15.31:0/3848005273 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7ff208066f30 0x7ff2080693e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff2180011e0 tx=0x7ff218009040 comp rx=0 tx=0).stop 2023-10-06T02:22:59.592 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:59.590+0000 7ff229f9d700 1 -- 172.21.15.31:0/3848005273 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff224159b60 msgr2=0x7ff2240ab550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:22:59.592 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:59.590+0000 7ff229f9d700 1 --2- 172.21.15.31:0/3848005273 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff224159b60 0x7ff2240ab550 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff21c00c8e0 tx=0x7ff21c001710 comp rx=0 tx=0).stop 2023-10-06T02:22:59.592 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:59.590+0000 7ff229f9d700 1 -- 172.21.15.31:0/3848005273 shutdown_connections 2023-10-06T02:22:59.592 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:59.590+0000 7ff229f9d700 1 --2- 172.21.15.31:0/3848005273 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff224159b60 0x7ff2240ab550 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:59.592 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:59.590+0000 7ff229f9d700 1 --2- 172.21.15.31:0/3848005273 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7ff208066f30 0x7ff2080693e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:59.593 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:59.590+0000 7ff229f9d700 1 --2- 172.21.15.31:0/3848005273 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff22406c5c0 0x7ff224141e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:59.593 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:59.590+0000 7ff229f9d700 1 --2- 172.21.15.31:0/3848005273 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff224159790 0x7ff22415b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:59.593 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:59.590+0000 7ff229f9d700 1 -- 172.21.15.31:0/3848005273 >> 172.21.15.31:0/3848005273 conn(0x7ff224068810 msgr2=0x7ff2240acb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:22:59.593 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:59.590+0000 7ff229f9d700 1 -- 172.21.15.31:0/3848005273 shutdown_connections 2023-10-06T02:22:59.593 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T02:22:59.590+0000 7ff229f9d700 1 -- 172.21.15.31:0/3848005273 wait complete. 2023-10-06T02:22:59.606 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-10-06T02:22:59.606 INFO:tasks.ceph:Creating RBD pool 2023-10-06T02:22:59.607 DEBUG:teuthology.orchestra.run.smithi031:> sudo ceph --cluster ceph osd pool create rbd 8 2023-10-06T02:22:59.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:59.786+0000 7fc6ca79d700 1 Processor -- start 2023-10-06T02:22:59.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:59.786+0000 7fc6ca79d700 1 -- start start 2023-10-06T02:22:59.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:59.787+0000 7fc6ca79d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc6c4159550 0x7fc6c40ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:22:59.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:59.787+0000 7fc6ca79d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc6c415dfd0 0x7fc6c40aedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:22:59.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:59.787+0000 7fc6ca79d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc6c4159920 0x7fc6c40ad090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:22:59.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:59.787+0000 7fc6ca79d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6c40acc00 con 0x7fc6c415dfd0 2023-10-06T02:22:59.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:59.787+0000 7fc6ca79d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6c40abff0 con 0x7fc6c4159920 2023-10-06T02:22:59.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:59.787+0000 7fc6ca79d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c40ac2d0 con 0x7fc6c4159550 2023-10-06T02:22:59.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:59.788+0000 7fc6c8f9a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc6c415dfd0 0x7fc6c40aedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:22:59.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:59.788+0000 7fc6c8f9a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc6c415dfd0 0x7fc6c40aedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56748/0 (socket says 172.21.15.31:56748) 2023-10-06T02:22:59.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:59.788+0000 7fc6c8f9a700 1 -- 172.21.15.31:0/32727634 learned_addr learned my addr 172.21.15.31:0/32727634 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:22:59.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:59.788+0000 7fc6c979b700 1 --2- 172.21.15.31:0/32727634 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc6c4159550 0x7fc6c40ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:22:59.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:59.788+0000 7fc6c9f9c700 1 --2- 172.21.15.31:0/32727634 >> v2:172.21.15.62:3300/0 conn(0x7fc6c4159920 0x7fc6c40ad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:22:59.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:59.788+0000 7fc6c8f9a700 1 -- 172.21.15.31:0/32727634 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc6c4159550 msgr2=0x7fc6c40ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:22:59.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:59.788+0000 7fc6c8f9a700 1 --2- 172.21.15.31:0/32727634 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc6c4159550 0x7fc6c40ceac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:59.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:59.788+0000 7fc6c8f9a700 1 -- 172.21.15.31:0/32727634 >> v2:172.21.15.62:3300/0 conn(0x7fc6c4159920 msgr2=0x7fc6c40ad090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:22:59.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:59.788+0000 7fc6c8f9a700 1 --2- 172.21.15.31:0/32727634 >> v2:172.21.15.62:3300/0 conn(0x7fc6c4159920 0x7fc6c40ad090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:59.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:59.788+0000 7fc6c8f9a700 1 -- 172.21.15.31:0/32727634 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6c40ac5b0 con 0x7fc6c415dfd0 2023-10-06T02:22:59.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:59.788+0000 7fc6c8f9a700 1 --2- 172.21.15.31:0/32727634 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6c415dfd0 0x7fc6c40aedb0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc6c406c180 tx=0x7fc6b4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=509bf8f81d7e38f8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:22:59.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:59.789+0000 7fc6bbfff700 1 -- 172.21.15.31:0/32727634 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b400b640 con 0x7fc6c415dfd0 2023-10-06T02:22:59.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:59.789+0000 7fc6bbfff700 1 -- 172.21.15.31:0/32727634 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6b400b7a0 con 0x7fc6c415dfd0 2023-10-06T02:22:59.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:59.789+0000 7fc6bbfff700 1 -- 172.21.15.31:0/32727634 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b400bb60 con 0x7fc6c415dfd0 2023-10-06T02:22:59.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:59.789+0000 7fc6ca79d700 1 -- 172.21.15.31:0/32727634 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6c415dfd0 msgr2=0x7fc6c40aedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:22:59.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:59.789+0000 7fc6ca79d700 1 --2- 172.21.15.31:0/32727634 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6c415dfd0 0x7fc6c40aedb0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc6c406c180 tx=0x7fc6b4009250 comp rx=0 tx=0).stop 2023-10-06T02:22:59.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:59.789+0000 7fc6ca79d700 1 -- 172.21.15.31:0/32727634 shutdown_connections 2023-10-06T02:22:59.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:59.789+0000 7fc6ca79d700 1 --2- 172.21.15.31:0/32727634 >> v2:172.21.15.62:3300/0 conn(0x7fc6c4159920 0x7fc6c40ad090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:59.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:59.789+0000 7fc6ca79d700 1 --2- 172.21.15.31:0/32727634 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc6c4159550 0x7fc6c40ceac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:59.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:59.789+0000 7fc6ca79d700 1 --2- 172.21.15.31:0/32727634 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6c415dfd0 0x7fc6c40aedb0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:59.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:59.789+0000 7fc6ca79d700 1 -- 172.21.15.31:0/32727634 >> 172.21.15.31:0/32727634 conn(0x7fc6c406ce80 msgr2=0x7fc6c40ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:22:59.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:59.789+0000 7fc6ca79d700 1 -- 172.21.15.31:0/32727634 shutdown_connections 2023-10-06T02:22:59.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:59.789+0000 7fc6ca79d700 1 -- 172.21.15.31:0/32727634 wait complete. 2023-10-06T02:22:59.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:59.791+0000 7fc6ca79d700 1 Processor -- start 2023-10-06T02:22:59.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:59.791+0000 7fc6ca79d700 1 -- start start 2023-10-06T02:22:59.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:59.792+0000 7fc6ca79d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6c4159550 0x7fc6c40cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:22:59.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:59.792+0000 7fc6ca79d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6c4159920 0x7fc6c416d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:22:59.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:59.792+0000 7fc6ca79d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc6c415dfd0 0x7fc6c416d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:22:59.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:59.792+0000 7fc6ca79d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c40ca950 con 0x7fc6c4159920 2023-10-06T02:22:59.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:59.792+0000 7fc6ca79d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c40cb2f0 con 0x7fc6c4159550 2023-10-06T02:22:59.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:59.792+0000 7fc6ca79d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c414b160 con 0x7fc6c415dfd0 2023-10-06T02:22:59.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:59.792+0000 7fc6c8f9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6c4159920 0x7fc6c416d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:22:59.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:59.792+0000 7fc6c8f9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6c4159920 0x7fc6c416d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56762/0 (socket says 172.21.15.31:56762) 2023-10-06T02:22:59.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:59.792+0000 7fc6c8f9a700 1 -- 172.21.15.31:0/1883768160 learned_addr learned my addr 172.21.15.31:0/1883768160 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:22:59.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:59.792+0000 7fc6c9f9c700 1 --2- 172.21.15.31:0/1883768160 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc6c415dfd0 0x7fc6c416d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:22:59.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:59.792+0000 7fc6c979b700 1 --2- 172.21.15.31:0/1883768160 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6c4159550 0x7fc6c40cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:22:59.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:59.793+0000 7fc6c8f9a700 1 -- 172.21.15.31:0/1883768160 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc6c415dfd0 msgr2=0x7fc6c416d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:22:59.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:59.793+0000 7fc6c8f9a700 1 --2- 172.21.15.31:0/1883768160 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc6c415dfd0 0x7fc6c416d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:59.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:59.793+0000 7fc6c8f9a700 1 -- 172.21.15.31:0/1883768160 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6c4159550 msgr2=0x7fc6c40cadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:22:59.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:59.793+0000 7fc6c8f9a700 1 --2- 172.21.15.31:0/1883768160 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6c4159550 0x7fc6c40cadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:22:59.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:59.793+0000 7fc6c8f9a700 1 -- 172.21.15.31:0/1883768160 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6b40094f0 con 0x7fc6c4159920 2023-10-06T02:22:59.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:59.793+0000 7fc6c979b700 1 --2- 172.21.15.31:0/1883768160 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6c4159550 0x7fc6c40cadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:22:59.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:59.793+0000 7fc6c8f9a700 1 --2- 172.21.15.31:0/1883768160 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6c4159920 0x7fc6c416d0c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc6c406ad70 tx=0x7fc6b4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:22:59.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:59.793+0000 7fc6b97fa700 1 -- 172.21.15.31:0/1883768160 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b4000d10 con 0x7fc6c4159920 2023-10-06T02:22:59.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:59.793+0000 7fc6b97fa700 1 -- 172.21.15.31:0/1883768160 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6b400bb20 con 0x7fc6c4159920 2023-10-06T02:22:59.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:59.793+0000 7fc6b97fa700 1 -- 172.21.15.31:0/1883768160 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b4011670 con 0x7fc6c4159920 2023-10-06T02:22:59.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:59.793+0000 7fc6ca79d700 1 -- 172.21.15.31:0/1883768160 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6c416dc60 con 0x7fc6c4159920 2023-10-06T02:22:59.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:59.793+0000 7fc6ca79d700 1 -- 172.21.15.31:0/1883768160 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6c4156280 con 0x7fc6c4159920 2023-10-06T02:22:59.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:59.794+0000 7fc6ca79d700 1 -- 172.21.15.31:0/1883768160 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6c404c3c0 con 0x7fc6c4159920 2023-10-06T02:22:59.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:59.795+0000 7fc6b97fa700 1 -- 172.21.15.31:0/1883768160 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc6b40119e0 con 0x7fc6c4159920 2023-10-06T02:22:59.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:59.796+0000 7fc6b97fa700 1 --2- 172.21.15.31:0/1883768160 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc6a8066f30 0x7fc6a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:22:59.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:59.796+0000 7fc6b97fa700 1 -- 172.21.15.31:0/1883768160 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fc6b40859e0 con 0x7fc6c4159920 2023-10-06T02:22:59.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:59.796+0000 7fc6c979b700 1 --2- 172.21.15.31:0/1883768160 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc6a8066f30 0x7fc6a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:22:59.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:59.797+0000 7fc6c979b700 1 --2- 172.21.15.31:0/1883768160 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc6a8066f30 0x7fc6a80693e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc6c0009d80 tx=0x7fc6c0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:22:59.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:22:59.800+0000 7fc6b97fa700 1 -- 172.21.15.31:0/1883768160 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc6b4052f00 con 0x7fc6c4159920 2023-10-06T02:22:59.964 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T02:22:59.968 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T02:23:00.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:00.005+0000 7fc6ca79d700 1 -- 172.21.15.31:0/1883768160 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7fc6c406e7c0 con 0x7fc6c4159920 2023-10-06T02:23:00.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:00.971+0000 7fc6b97fa700 1 -- 172.21.15.31:0/1883768160 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v17) v1 ==== 109+0+0 (secure 0 0 0) 0x7fc6b40150a0 con 0x7fc6c4159920 2023-10-06T02:23:00.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:00.975+0000 7fc6ca79d700 1 -- 172.21.15.31:0/1883768160 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc6a8066f30 msgr2=0x7fc6a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:00.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:00.975+0000 7fc6ca79d700 1 --2- 172.21.15.31:0/1883768160 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc6a8066f30 0x7fc6a80693e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc6c0009d80 tx=0x7fc6c0008040 comp rx=0 tx=0).stop 2023-10-06T02:23:00.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:00.975+0000 7fc6ca79d700 1 -- 172.21.15.31:0/1883768160 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6c4159920 msgr2=0x7fc6c416d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:00.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:00.975+0000 7fc6ca79d700 1 --2- 172.21.15.31:0/1883768160 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6c4159920 0x7fc6c416d0c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc6c406ad70 tx=0x7fc6b4017000 comp rx=0 tx=0).stop 2023-10-06T02:23:00.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:00.975+0000 7fc6ca79d700 1 -- 172.21.15.31:0/1883768160 shutdown_connections 2023-10-06T02:23:00.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:00.975+0000 7fc6ca79d700 1 --2- 172.21.15.31:0/1883768160 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc6a8066f30 0x7fc6a80693e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:00.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:00.975+0000 7fc6ca79d700 1 --2- 172.21.15.31:0/1883768160 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc6c415dfd0 0x7fc6c416d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:00.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:00.975+0000 7fc6ca79d700 1 --2- 172.21.15.31:0/1883768160 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6c4159550 0x7fc6c40cadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:00.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:00.975+0000 7fc6ca79d700 1 --2- 172.21.15.31:0/1883768160 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6c4159920 0x7fc6c416d0c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:00.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:00.975+0000 7fc6ca79d700 1 -- 172.21.15.31:0/1883768160 >> 172.21.15.31:0/1883768160 conn(0x7fc6c406ce80 msgr2=0x7fc6c40af170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:00.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:00.975+0000 7fc6ca79d700 1 -- 172.21.15.31:0/1883768160 shutdown_connections 2023-10-06T02:23:00.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:00.976+0000 7fc6ca79d700 1 -- 172.21.15.31:0/1883768160 wait complete. 2023-10-06T02:23:00.979 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'rbd' created 2023-10-06T02:23:00.999 DEBUG:teuthology.orchestra.run.smithi031:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-10-06T02:23:01.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.183+0000 7f4f0ffff700 1 Processor -- start 2023-10-06T02:23:01.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.184+0000 7f4f0ffff700 1 -- start start 2023-10-06T02:23:01.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.185+0000 7f4f0ffff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4f100cb450 0x7f4f1015dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:01.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.185+0000 7f4f0ffff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f1015d8d0 con 0x7f4f10142ca0 2023-10-06T02:23:01.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.185+0000 7f4f0ffff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f1015d2f0 con 0x7f4f10143070 2023-10-06T02:23:01.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.185+0000 7f4f0ffff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f100cc630 con 0x7f4f100cb450 2023-10-06T02:23:01.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.185+0000 7f4f0effd700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f4f10142ca0 0x7f4f100cad40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:39208/0 (socket says 172.21.15.31:39208) 2023-10-06T02:23:01.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.185+0000 7f4f0effd700 1 -- 172.21.15.31:0/2289005616 learned_addr learned my addr 172.21.15.31:0/2289005616 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:23:01.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.185+0000 7f4f0e7fc700 1 --2- 172.21.15.31:0/2289005616 >> v2:172.21.15.62:3300/0 conn(0x7f4f100cb450 0x7f4f1015dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:01.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.185+0000 7f4f0dffb700 1 -- 172.21.15.31:0/2289005616 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3292609823 0 0) 0x7f4f1015d8d0 con 0x7f4f10142ca0 2023-10-06T02:23:01.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.186+0000 7f4f0dffb700 1 -- 172.21.15.31:0/2289005616 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ef8003570 con 0x7f4f10142ca0 2023-10-06T02:23:01.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.186+0000 7f4f0dffb700 1 -- 172.21.15.31:0/2289005616 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3975308036 0 0) 0x7f4ef8003570 con 0x7f4f10142ca0 2023-10-06T02:23:01.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.186+0000 7f4f0dffb700 1 -- 172.21.15.31:0/2289005616 >> v2:172.21.15.62:3300/0 conn(0x7f4f100cb450 msgr2=0x7f4f1015dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:01.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.186+0000 7f4f0dffb700 1 --2- 172.21.15.31:0/2289005616 >> v2:172.21.15.62:3300/0 conn(0x7f4f100cb450 0x7f4f1015dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:01.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.186+0000 7f4f0dffb700 1 -- 172.21.15.31:0/2289005616 >> v1:172.21.15.62:6789/0 conn(0x7f4f10143070 legacy=0x7f4f1006dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:01.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.186+0000 7f4f0dffb700 1 -- 172.21.15.31:0/2289005616 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f1015d560 con 0x7f4f10142ca0 2023-10-06T02:23:01.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.186+0000 7f4f0dffb700 1 -- 172.21.15.31:0/2289005616 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7f4f00001470 con 0x7f4f10142ca0 2023-10-06T02:23:01.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.186+0000 7f4f0dffb700 1 -- 172.21.15.31:0/2289005616 >> v1:172.21.15.31:6789/0 conn(0x7f4f10142ca0 legacy=0x7f4f100cad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:01.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.186+0000 7f4f0dffb700 1 --2- 172.21.15.31:0/2289005616 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4ef8003a60 0x7f4ef8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:01.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.186+0000 7f4f0dffb700 1 --2- 172.21.15.31:0/2289005616 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ef8004810 0x7f4ef8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:01.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.186+0000 7f4f0dffb700 1 --2- 172.21.15.31:0/2289005616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ef80051c0 0x7f4ef800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:01.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.186+0000 7f4f0dffb700 1 -- 172.21.15.31:0/2289005616 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f100cc630 con 0x7f4ef8004810 2023-10-06T02:23:01.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.186+0000 7f4f0dffb700 1 -- 172.21.15.31:0/2289005616 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef800c050 con 0x7f4ef80051c0 2023-10-06T02:23:01.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.186+0000 7f4f0dffb700 1 -- 172.21.15.31:0/2289005616 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef800c210 con 0x7f4ef8003a60 2023-10-06T02:23:01.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.186+0000 7f4f0e7fc700 1 --2- 172.21.15.31:0/2289005616 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ef8004810 0x7f4ef8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:01.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.186+0000 7f4f0effd700 1 --2- 172.21.15.31:0/2289005616 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4ef8003a60 0x7f4ef8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:01.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.186+0000 7f4f0f7fe700 1 --2- 172.21.15.31:0/2289005616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ef80051c0 0x7f4ef800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:01.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.187+0000 7f4f0e7fc700 1 -- 172.21.15.31:0/2289005616 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4ef8003a60 msgr2=0x7f4ef8003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:01.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.187+0000 7f4f0e7fc700 1 --2- 172.21.15.31:0/2289005616 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4ef8003a60 0x7f4ef8003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:01.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.187+0000 7f4f0e7fc700 1 -- 172.21.15.31:0/2289005616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ef80051c0 msgr2=0x7f4ef800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:01.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.187+0000 7f4f0e7fc700 1 --2- 172.21.15.31:0/2289005616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ef80051c0 0x7f4ef800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:01.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.187+0000 7f4f0e7fc700 1 -- 172.21.15.31:0/2289005616 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ef8003570 con 0x7f4ef8004810 2023-10-06T02:23:01.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.187+0000 7f4f0f7fe700 1 -- 172.21.15.31:0/2289005616 reap_dead start 2023-10-06T02:23:01.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.187+0000 7f4f0e7fc700 1 --2- 172.21.15.31:0/2289005616 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ef8004810 0x7f4ef8004c80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4f04009c00 tx=0x7f4f040091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1cc787ea6c6e758 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:01.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.187+0000 7f4f0dffb700 1 -- 172.21.15.31:0/2289005616 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f0400e710 con 0x7f4ef8004810 2023-10-06T02:23:01.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.187+0000 7f4f0dffb700 1 -- 172.21.15.31:0/2289005616 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f0400e870 con 0x7f4ef8004810 2023-10-06T02:23:01.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.187+0000 7f4f0dffb700 1 -- 172.21.15.31:0/2289005616 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f0400ec30 con 0x7f4ef8004810 2023-10-06T02:23:01.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.188+0000 7f4f0ffff700 1 -- 172.21.15.31:0/2289005616 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ef8004810 msgr2=0x7f4ef8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:01.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.188+0000 7f4f0ffff700 1 --2- 172.21.15.31:0/2289005616 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ef8004810 0x7f4ef8004c80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4f04009c00 tx=0x7f4f040091a0 comp rx=0 tx=0).stop 2023-10-06T02:23:01.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.188+0000 7f4f0ffff700 1 -- 172.21.15.31:0/2289005616 shutdown_connections 2023-10-06T02:23:01.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.188+0000 7f4f0ffff700 1 --2- 172.21.15.31:0/2289005616 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ef8004810 0x7f4ef8004c80 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:01.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.188+0000 7f4f0ffff700 1 -- 172.21.15.31:0/2289005616 >> 172.21.15.31:0/2289005616 conn(0x7f4f10067c80 msgr2=0x7f4f10068080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:01.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.188+0000 7f4f0ffff700 1 -- 172.21.15.31:0/2289005616 shutdown_connections 2023-10-06T02:23:01.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.188+0000 7f4f0ffff700 1 -- 172.21.15.31:0/2289005616 wait complete. 2023-10-06T02:23:01.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.189+0000 7f4f0ffff700 1 Processor -- start 2023-10-06T02:23:01.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.189+0000 7f4f0ffff700 1 -- start start 2023-10-06T02:23:01.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.190+0000 7f4f0ffff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4ef8004810 0x7f4f1015b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:01.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.190+0000 7f4f0ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f1016b1b0 0x7f4f1015bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:01.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.190+0000 7f4f0ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f100afb40 0x7f4f100affe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:01.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.190+0000 7f4f0ffff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f100b1dc0 con 0x7f4f100afb40 2023-10-06T02:23:01.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.190+0000 7f4f0ffff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f100af6b0 con 0x7f4f1016b1b0 2023-10-06T02:23:01.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.190+0000 7f4f0ffff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1016a8e0 con 0x7f4ef8004810 2023-10-06T02:23:01.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.190+0000 7f4f0f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f100afb40 0x7f4f100affe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:01.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.190+0000 7f4f0f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f100afb40 0x7f4f100affe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56778/0 (socket says 172.21.15.31:56778) 2023-10-06T02:23:01.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.190+0000 7f4f0f7fe700 1 -- 172.21.15.31:0/1443671580 learned_addr learned my addr 172.21.15.31:0/1443671580 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:01.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.191+0000 7f4f0e7fc700 1 --2- 172.21.15.31:0/1443671580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f1016b1b0 0x7f4f1015bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:01.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.191+0000 7f4f0effd700 1 --2- 172.21.15.31:0/1443671580 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4ef8004810 0x7f4f1015b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:01.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.191+0000 7f4f0f7fe700 1 -- 172.21.15.31:0/1443671580 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4ef8004810 msgr2=0x7f4f1015b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:01.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.191+0000 7f4f0f7fe700 1 --2- 172.21.15.31:0/1443671580 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4ef8004810 0x7f4f1015b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:01.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.191+0000 7f4f0f7fe700 1 -- 172.21.15.31:0/1443671580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f1016b1b0 msgr2=0x7f4f1015bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:01.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.191+0000 7f4f0f7fe700 1 --2- 172.21.15.31:0/1443671580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f1016b1b0 0x7f4f1015bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:01.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.191+0000 7f4f0f7fe700 1 -- 172.21.15.31:0/1443671580 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f100af060 con 0x7f4f100afb40 2023-10-06T02:23:01.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.191+0000 7f4f0effd700 1 --2- 172.21.15.31:0/1443671580 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4ef8004810 0x7f4f1015b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:23:01.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.191+0000 7f4f0f7fe700 1 --2- 172.21.15.31:0/1443671580 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f100afb40 0x7f4f100affe0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4f1015d840 tx=0x7f4f08009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:01.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.191+0000 7f4ef6ffd700 1 -- 172.21.15.31:0/1443671580 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f08006db0 con 0x7f4f100afb40 2023-10-06T02:23:01.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.191+0000 7f4ef6ffd700 1 -- 172.21.15.31:0/1443671580 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f080030d0 con 0x7f4f100afb40 2023-10-06T02:23:01.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.191+0000 7f4ef6ffd700 1 -- 172.21.15.31:0/1443671580 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f08003490 con 0x7f4f100afb40 2023-10-06T02:23:01.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.192+0000 7f4f0ffff700 1 -- 172.21.15.31:0/1443671580 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f100af340 con 0x7f4f100afb40 2023-10-06T02:23:01.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.192+0000 7f4f0ffff700 1 -- 172.21.15.31:0/1443671580 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f100b0520 con 0x7f4f100afb40 2023-10-06T02:23:01.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.192+0000 7f4f0ffff700 1 -- 172.21.15.31:0/1443671580 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f1004c3c0 con 0x7f4f100afb40 2023-10-06T02:23:01.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.194+0000 7f4ef6ffd700 1 -- 172.21.15.31:0/1443671580 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4f08001920 con 0x7f4f100afb40 2023-10-06T02:23:01.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.195+0000 7f4ef6ffd700 1 --2- 172.21.15.31:0/1443671580 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f4efc066f30 0x7f4efc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:01.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.195+0000 7f4ef6ffd700 1 -- 172.21.15.31:0/1443671580 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5195+0+0 (secure 0 0 0) 0x7f4f08084220 con 0x7f4f100afb40 2023-10-06T02:23:01.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.195+0000 7f4f0effd700 1 --2- 172.21.15.31:0/1443671580 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f4efc066f30 0x7f4efc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:01.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.196+0000 7f4f0effd700 1 --2- 172.21.15.31:0/1443671580 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f4efc066f30 0x7f4efc0693e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4f00001050 tx=0x7f4f00009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:01.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.198+0000 7f4ef6ffd700 1 -- 172.21.15.31:0/1443671580 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4f08051670 con 0x7f4f100afb40 2023-10-06T02:23:01.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.398+0000 7f4f0ffff700 1 -- 172.21.15.31:0/1443671580 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f4f100af7f0 con 0x7f4f100afb40 2023-10-06T02:23:01.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.978+0000 7f4ef6ffd700 1 -- 172.21.15.31:0/1443671580 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v18) v1 ==== 173+0+0 (secure 0 0 0) 0x7f4f08054c90 con 0x7f4f100afb40 2023-10-06T02:23:01.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.980+0000 7f4f0ffff700 1 -- 172.21.15.31:0/1443671580 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f4efc066f30 msgr2=0x7f4efc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:01.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.980+0000 7f4f0ffff700 1 --2- 172.21.15.31:0/1443671580 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f4efc066f30 0x7f4efc0693e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4f00001050 tx=0x7f4f00009040 comp rx=0 tx=0).stop 2023-10-06T02:23:01.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.980+0000 7f4f0ffff700 1 -- 172.21.15.31:0/1443671580 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f100afb40 msgr2=0x7f4f100affe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:01.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.980+0000 7f4f0ffff700 1 --2- 172.21.15.31:0/1443671580 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f100afb40 0x7f4f100affe0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4f1015d840 tx=0x7f4f08009040 comp rx=0 tx=0).stop 2023-10-06T02:23:01.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.981+0000 7f4f0ffff700 1 -- 172.21.15.31:0/1443671580 shutdown_connections 2023-10-06T02:23:01.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.981+0000 7f4f0ffff700 1 --2- 172.21.15.31:0/1443671580 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f100afb40 0x7f4f100affe0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:01.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.981+0000 7f4f0ffff700 1 --2- 172.21.15.31:0/1443671580 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f4efc066f30 0x7f4efc0693e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:01.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.981+0000 7f4f0ffff700 1 --2- 172.21.15.31:0/1443671580 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4ef8004810 0x7f4f1015b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:01.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.981+0000 7f4f0ffff700 1 --2- 172.21.15.31:0/1443671580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f1016b1b0 0x7f4f1015bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:01.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.981+0000 7f4f0ffff700 1 -- 172.21.15.31:0/1443671580 >> 172.21.15.31:0/1443671580 conn(0x7f4f10067c80 msgr2=0x7f4f10068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:01.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.981+0000 7f4f0ffff700 1 -- 172.21.15.31:0/1443671580 shutdown_connections 2023-10-06T02:23:01.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:01.981+0000 7f4f0ffff700 1 -- 172.21.15.31:0/1443671580 wait complete. 2023-10-06T02:23:01.984 INFO:teuthology.orchestra.run.smithi031.stderr:enabled application 'rbd' on pool 'rbd' 2023-10-06T02:23:02.003 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-10-06T02:23:02.003 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T02:23:02.003 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T02:23:02.008 INFO:tasks.ceph.mds.a:Started 2023-10-06T02:23:02.008 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T02:23:02.008 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T02:23:02.012 INFO:tasks.ceph.mds.c:Started 2023-10-06T02:23:02.012 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T02:23:02.013 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T02:23:02.018 INFO:tasks.ceph.mds.b:Started 2023-10-06T02:23:02.018 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T02:23:02.018 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T02:23:02.022 INFO:tasks.ceph.mds.d:Started 2023-10-06T02:23:02.022 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-10-06T02:23:02.023 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:23:02.150 INFO:tasks.ceph.mds.a.smithi031.stdout:starting mds.a at 2023-10-06T02:23:02.150 INFO:tasks.ceph.mds.c.smithi031.stdout:starting mds.c at 2023-10-06T02:23:02.159 INFO:tasks.ceph.mds.b.smithi062.stdout:starting mds.b at 2023-10-06T02:23:02.161 INFO:tasks.ceph.mds.d.smithi062.stdout:starting mds.d at 2023-10-06T02:23:02.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.224+0000 7fbef359e700 1 Processor -- start 2023-10-06T02:23:02.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.224+0000 7fbef359e700 1 -- start start 2023-10-06T02:23:02.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.225+0000 7fbef359e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbef413fb20 0x7fbef415a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:02.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.225+0000 7fbef359e700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbef415ae70 con 0x7fbef413fef0 2023-10-06T02:23:02.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.225+0000 7fbef359e700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbef406af60 con 0x7fbef406aaa0 2023-10-06T02:23:02.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.225+0000 7fbef359e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbef406b240 con 0x7fbef413fb20 2023-10-06T02:23:02.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.225+0000 7fbef259c700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbef413fb20 0x7fbef415a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:02.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.225+0000 7fbef1d9b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fbef413fef0 0x7fbef40c7990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:39236/0 (socket says 172.21.15.31:39236) 2023-10-06T02:23:02.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.225+0000 7fbef1d9b700 1 -- 172.21.15.31:0/741799287 learned_addr learned my addr 172.21.15.31:0/741799287 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:23:02.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.226+0000 7fbef159a700 1 -- 172.21.15.31:0/741799287 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1997742291 0 0) 0x7fbee8001290 con 0x7fbef413fef0 2023-10-06T02:23:02.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.226+0000 7fbef259c700 1 -- 172.21.15.31:0/741799287 >> v1:172.21.15.62:6789/0 conn(0x7fbef406aaa0 legacy=0x7fbef40c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:02.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.226+0000 7fbef259c700 1 -- 172.21.15.31:0/741799287 >> v1:172.21.15.31:6789/0 conn(0x7fbef413fef0 legacy=0x7fbef40c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:02.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.226+0000 7fbef259c700 1 -- 172.21.15.31:0/741799287 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbef406b520 con 0x7fbef413fb20 2023-10-06T02:23:02.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.226+0000 7fbef259c700 1 --2- 172.21.15.31:0/741799287 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbef413fb20 0x7fbef415a930 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fbee4009890 tx=0x7fbee40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53a6d2420bd1e97b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:02.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.226+0000 7fbef159a700 1 -- 172.21.15.31:0/741799287 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee4011630 con 0x7fbef413fb20 2023-10-06T02:23:02.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.226+0000 7fbef159a700 1 -- 172.21.15.31:0/741799287 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbee4011790 con 0x7fbef413fb20 2023-10-06T02:23:02.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.227+0000 7fbef159a700 1 -- 172.21.15.31:0/741799287 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee4011b50 con 0x7fbef413fb20 2023-10-06T02:23:02.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.227+0000 7fbef359e700 1 -- 172.21.15.31:0/741799287 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbef413fb20 msgr2=0x7fbef415a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:02.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.227+0000 7fbef359e700 1 --2- 172.21.15.31:0/741799287 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbef413fb20 0x7fbef415a930 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fbee4009890 tx=0x7fbee40091a0 comp rx=0 tx=0).stop 2023-10-06T02:23:02.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.227+0000 7fbef359e700 1 -- 172.21.15.31:0/741799287 shutdown_connections 2023-10-06T02:23:02.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.227+0000 7fbef359e700 1 --2- 172.21.15.31:0/741799287 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbef413fb20 0x7fbef415a930 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:02.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.227+0000 7fbef359e700 1 -- 172.21.15.31:0/741799287 >> 172.21.15.31:0/741799287 conn(0x7fbef406d170 msgr2=0x7fbef40cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:02.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.227+0000 7fbef359e700 1 -- 172.21.15.31:0/741799287 shutdown_connections 2023-10-06T02:23:02.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.227+0000 7fbef359e700 1 -- 172.21.15.31:0/741799287 wait complete. 2023-10-06T02:23:02.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.229+0000 7fbef359e700 1 Processor -- start 2023-10-06T02:23:02.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.229+0000 7fbef359e700 1 -- start start 2023-10-06T02:23:02.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.230+0000 7fbef359e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbef406aaa0 0x7fbef41361f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:02.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.230+0000 7fbef359e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbef413fb20 0x7fbef416d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:02.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.230+0000 7fbef359e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbef413fef0 0x7fbef406de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:02.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.230+0000 7fbef359e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef416dbf0 con 0x7fbef413fef0 2023-10-06T02:23:02.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.230+0000 7fbef359e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef416ded0 con 0x7fbef413fb20 2023-10-06T02:23:02.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.230+0000 7fbef359e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbef4156920 con 0x7fbef406aaa0 2023-10-06T02:23:02.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.230+0000 7fbef2d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbef413fef0 0x7fbef406de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:02.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.230+0000 7fbef2d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbef413fef0 0x7fbef406de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56826/0 (socket says 172.21.15.31:56826) 2023-10-06T02:23:02.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.230+0000 7fbef2d9d700 1 -- 172.21.15.31:0/3654217668 learned_addr learned my addr 172.21.15.31:0/3654217668 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:02.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.230+0000 7fbef259c700 1 --2- 172.21.15.31:0/3654217668 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbef406aaa0 0x7fbef41361f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:02.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.230+0000 7fbef1d9b700 1 --2- 172.21.15.31:0/3654217668 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbef413fb20 0x7fbef416d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:02.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.230+0000 7fbef2d9d700 1 -- 172.21.15.31:0/3654217668 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbef406aaa0 msgr2=0x7fbef41361f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:02.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.230+0000 7fbef2d9d700 1 --2- 172.21.15.31:0/3654217668 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbef406aaa0 0x7fbef41361f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:02.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.230+0000 7fbef2d9d700 1 -- 172.21.15.31:0/3654217668 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbef413fb20 msgr2=0x7fbef416d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:02.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.230+0000 7fbef2d9d700 1 --2- 172.21.15.31:0/3654217668 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbef413fb20 0x7fbef416d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:02.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.230+0000 7fbef2d9d700 1 -- 172.21.15.31:0/3654217668 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbee4009420 con 0x7fbef413fef0 2023-10-06T02:23:02.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.231+0000 7fbef2d9d700 1 --2- 172.21.15.31:0/3654217668 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbef413fef0 0x7fbef406de50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbeec00cb40 tx=0x7fbeec0017b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:02.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.231+0000 7fbee27fc700 1 -- 172.21.15.31:0/3654217668 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeec0020a0 con 0x7fbef413fef0 2023-10-06T02:23:02.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.231+0000 7fbee27fc700 1 -- 172.21.15.31:0/3654217668 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbeec00a7e0 con 0x7fbef413fef0 2023-10-06T02:23:02.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.231+0000 7fbef359e700 1 -- 172.21.15.31:0/3654217668 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbef4156c00 con 0x7fbef413fef0 2023-10-06T02:23:02.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.231+0000 7fbee27fc700 1 -- 172.21.15.31:0/3654217668 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeec00aba0 con 0x7fbef413fef0 2023-10-06T02:23:02.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.231+0000 7fbef359e700 1 -- 172.21.15.31:0/3654217668 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbef40c2c50 con 0x7fbef413fef0 2023-10-06T02:23:02.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.232+0000 7fbef359e700 1 -- 172.21.15.31:0/3654217668 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbef4157120 con 0x7fbef413fef0 2023-10-06T02:23:02.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.233+0000 7fbee27fc700 1 -- 172.21.15.31:0/3654217668 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbeec003030 con 0x7fbef413fef0 2023-10-06T02:23:02.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.234+0000 7fbee27fc700 1 --2- 172.21.15.31:0/3654217668 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fbed4066f30 0x7fbed40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:02.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.234+0000 7fbee27fc700 1 -- 172.21.15.31:0/3654217668 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fbeec0843e0 con 0x7fbef413fef0 2023-10-06T02:23:02.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.234+0000 7fbef259c700 1 --2- 172.21.15.31:0/3654217668 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fbed4066f30 0x7fbed40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:02.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.235+0000 7fbef259c700 1 --2- 172.21.15.31:0/3654217668 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fbed4066f30 0x7fbed40693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbee4009860 tx=0x7fbee4016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:02.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.239+0000 7fbee27fc700 1 -- 172.21.15.31:0/3654217668 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbeec0517a0 con 0x7fbef413fef0 2023-10-06T02:23:02.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.442+0000 7fbef359e700 1 -- 172.21.15.31:0/3654217668 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbef404c3c0 con 0x7fbef413fef0 2023-10-06T02:23:02.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.443+0000 7fbee27fc700 1 -- 172.21.15.31:0/3654217668 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+13612 (secure 0 0 0) 0x7fbeec054dc0 con 0x7fbef413fef0 2023-10-06T02:23:02.446 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:23:02.446 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":18,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:23:01.970718+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:23:02.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.446+0000 7fbef359e700 1 -- 172.21.15.31:0/3654217668 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fbed4066f30 msgr2=0x7fbed40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:02.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.446+0000 7fbef359e700 1 --2- 172.21.15.31:0/3654217668 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fbed4066f30 0x7fbed40693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbee4009860 tx=0x7fbee4016000 comp rx=0 tx=0).stop 2023-10-06T02:23:02.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.446+0000 7fbef359e700 1 -- 172.21.15.31:0/3654217668 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbef413fef0 msgr2=0x7fbef406de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:02.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.446+0000 7fbef359e700 1 --2- 172.21.15.31:0/3654217668 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbef413fef0 0x7fbef406de50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbeec00cb40 tx=0x7fbeec0017b0 comp rx=0 tx=0).stop 2023-10-06T02:23:02.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.446+0000 7fbef359e700 1 -- 172.21.15.31:0/3654217668 shutdown_connections 2023-10-06T02:23:02.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.446+0000 7fbef359e700 1 --2- 172.21.15.31:0/3654217668 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbef413fef0 0x7fbef406de50 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:02.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.446+0000 7fbef359e700 1 --2- 172.21.15.31:0/3654217668 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fbed4066f30 0x7fbed40693e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:02.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.446+0000 7fbef359e700 1 --2- 172.21.15.31:0/3654217668 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbef406aaa0 0x7fbef41361f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:02.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.446+0000 7fbef359e700 1 --2- 172.21.15.31:0/3654217668 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbef413fb20 0x7fbef416d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:02.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.446+0000 7fbef359e700 1 -- 172.21.15.31:0/3654217668 >> 172.21.15.31:0/3654217668 conn(0x7fbef406d170 msgr2=0x7fbef40c75f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:02.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.446+0000 7fbef359e700 1 -- 172.21.15.31:0/3654217668 shutdown_connections 2023-10-06T02:23:02.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.446+0000 7fbef359e700 1 -- 172.21.15.31:0/3654217668 wait complete. 2023-10-06T02:23:02.465 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T02:22:53.183425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T02:23:00.046967+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-10-06T02:23:02.465 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T02:23:02.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.653+0000 7fb5f359d700 1 Processor -- start 2023-10-06T02:23:02.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.653+0000 7fb5f359d700 1 -- start start 2023-10-06T02:23:02.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.655+0000 7fb5f359d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb5ec155890 0x7fb5ec0c7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:02.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.655+0000 7fb5f359d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb5ec06de50 0x7fb5ec06e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:02.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.655+0000 7fb5f359d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5ec0c7e70 con 0x7fb5ec155c60 2023-10-06T02:23:02.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.655+0000 7fb5f359d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5ec06e7f0 con 0x7fb5ec155890 2023-10-06T02:23:02.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.655+0000 7fb5f359d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5ec06b790 con 0x7fb5ec06de50 2023-10-06T02:23:02.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.655+0000 7fb5f259b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb5ec155890 0x7fb5ec0c7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:02.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.655+0000 7fb5f259b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb5ec155890 0x7fb5ec0c7930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56842/0 (socket says 172.21.15.31:56842) 2023-10-06T02:23:02.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.655+0000 7fb5f259b700 1 -- 172.21.15.31:0/1577087577 learned_addr learned my addr 172.21.15.31:0/1577087577 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:02.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.655+0000 7fb5f2d9c700 1 --2- 172.21.15.31:0/1577087577 >> v2:172.21.15.62:3300/0 conn(0x7fb5ec06de50 0x7fb5ec06e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:02.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.655+0000 7fb5f259b700 1 -- 172.21.15.31:0/1577087577 >> v2:172.21.15.62:3300/0 conn(0x7fb5ec06de50 msgr2=0x7fb5ec06e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:02.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.655+0000 7fb5f259b700 1 --2- 172.21.15.31:0/1577087577 >> v2:172.21.15.62:3300/0 conn(0x7fb5ec06de50 0x7fb5ec06e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:02.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.655+0000 7fb5f259b700 1 -- 172.21.15.31:0/1577087577 >> v1:172.21.15.62:6789/0 conn(0x7fb5ec155c60 legacy=0x7fb5ec06a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:02.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.655+0000 7fb5f259b700 1 -- 172.21.15.31:0/1577087577 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5ec06ab80 con 0x7fb5ec155890 2023-10-06T02:23:02.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.656+0000 7fb5f259b700 1 --2- 172.21.15.31:0/1577087577 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5ec155890 0x7fb5ec0c7930 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb5e800ab50 tx=0x7fb5e800a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb047fd2dcb8b904 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:02.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.656+0000 7fb5f1599700 1 -- 172.21.15.31:0/1577087577 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e800ca20 con 0x7fb5ec155890 2023-10-06T02:23:02.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.656+0000 7fb5f1599700 1 -- 172.21.15.31:0/1577087577 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5e800cb80 con 0x7fb5ec155890 2023-10-06T02:23:02.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.656+0000 7fb5f1599700 1 -- 172.21.15.31:0/1577087577 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e80183f0 con 0x7fb5ec155890 2023-10-06T02:23:02.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.656+0000 7fb5f359d700 1 -- 172.21.15.31:0/1577087577 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5ec155890 msgr2=0x7fb5ec0c7930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:02.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.656+0000 7fb5f359d700 1 --2- 172.21.15.31:0/1577087577 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5ec155890 0x7fb5ec0c7930 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb5e800ab50 tx=0x7fb5e800a3f0 comp rx=0 tx=0).stop 2023-10-06T02:23:02.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.656+0000 7fb5f359d700 1 -- 172.21.15.31:0/1577087577 shutdown_connections 2023-10-06T02:23:02.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.656+0000 7fb5f359d700 1 --2- 172.21.15.31:0/1577087577 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5ec155890 0x7fb5ec0c7930 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:02.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.656+0000 7fb5f359d700 1 --2- 172.21.15.31:0/1577087577 >> v2:172.21.15.62:3300/0 conn(0x7fb5ec06de50 0x7fb5ec06e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:02.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.657+0000 7fb5f359d700 1 -- 172.21.15.31:0/1577087577 >> 172.21.15.31:0/1577087577 conn(0x7fb5ec06d110 msgr2=0x7fb5ec0cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:02.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.657+0000 7fb5f359d700 1 -- 172.21.15.31:0/1577087577 shutdown_connections 2023-10-06T02:23:02.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.657+0000 7fb5f359d700 1 -- 172.21.15.31:0/1577087577 wait complete. 2023-10-06T02:23:02.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.658+0000 7fb5f359d700 1 Processor -- start 2023-10-06T02:23:02.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.658+0000 7fb5f359d700 1 -- start start 2023-10-06T02:23:02.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.659+0000 7fb5f359d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5ec06de50 0x7fb5ec0adc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:02.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.659+0000 7fb5f359d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb5ec155890 0x7fb5ec15b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:02.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.659+0000 7fb5f359d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5ec155c60 0x7fb5ec1673b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:02.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.659+0000 7fb5f359d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5ec0ae140 con 0x7fb5ec06de50 2023-10-06T02:23:02.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.659+0000 7fb5f359d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5ec0c4250 con 0x7fb5ec155c60 2023-10-06T02:23:02.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.659+0000 7fb5f359d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5ec0c4530 con 0x7fb5ec155890 2023-10-06T02:23:02.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.659+0000 7fb5f259b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5ec06de50 0x7fb5ec0adc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:02.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.659+0000 7fb5f259b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5ec06de50 0x7fb5ec0adc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56850/0 (socket says 172.21.15.31:56850) 2023-10-06T02:23:02.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.659+0000 7fb5f259b700 1 -- 172.21.15.31:0/510368029 learned_addr learned my addr 172.21.15.31:0/510368029 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:02.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.660+0000 7fb5f2d9c700 1 --2- 172.21.15.31:0/510368029 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5ec155c60 0x7fb5ec1673b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:02.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.660+0000 7fb5f1d9a700 1 --2- 172.21.15.31:0/510368029 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb5ec155890 0x7fb5ec15b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:02.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.660+0000 7fb5f259b700 1 -- 172.21.15.31:0/510368029 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb5ec155890 msgr2=0x7fb5ec15b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:02.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.660+0000 7fb5f259b700 1 --2- 172.21.15.31:0/510368029 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb5ec155890 0x7fb5ec15b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:02.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.660+0000 7fb5f259b700 1 -- 172.21.15.31:0/510368029 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5ec155c60 msgr2=0x7fb5ec1673b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:02.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.660+0000 7fb5f259b700 1 --2- 172.21.15.31:0/510368029 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5ec155c60 0x7fb5ec1673b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:02.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.660+0000 7fb5f259b700 1 -- 172.21.15.31:0/510368029 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5e800a690 con 0x7fb5ec06de50 2023-10-06T02:23:02.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.660+0000 7fb5f1d9a700 1 --2- 172.21.15.31:0/510368029 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb5ec155890 0x7fb5ec15b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:23:02.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.660+0000 7fb5f259b700 1 --2- 172.21.15.31:0/510368029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5ec06de50 0x7fb5ec0adc00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb5e800a430 tx=0x7fb5e8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:02.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.660+0000 7fb5e27fc700 1 -- 172.21.15.31:0/510368029 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e8010600 con 0x7fb5ec06de50 2023-10-06T02:23:02.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.660+0000 7fb5e27fc700 1 -- 172.21.15.31:0/510368029 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5e8010760 con 0x7fb5ec06de50 2023-10-06T02:23:02.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.660+0000 7fb5e27fc700 1 -- 172.21.15.31:0/510368029 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e8010b20 con 0x7fb5ec06de50 2023-10-06T02:23:02.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.660+0000 7fb5f359d700 1 -- 172.21.15.31:0/510368029 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5ec0c4810 con 0x7fb5ec06de50 2023-10-06T02:23:02.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.660+0000 7fb5f359d700 1 -- 172.21.15.31:0/510368029 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5ec0c4dd0 con 0x7fb5ec06de50 2023-10-06T02:23:02.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.661+0000 7fb5f359d700 1 -- 172.21.15.31:0/510368029 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5ec04c3c0 con 0x7fb5ec06de50 2023-10-06T02:23:02.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.662+0000 7fb5e27fc700 1 -- 172.21.15.31:0/510368029 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb5e80173a0 con 0x7fb5ec06de50 2023-10-06T02:23:02.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.663+0000 7fb5e27fc700 1 --2- 172.21.15.31:0/510368029 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fb5d0066f30 0x7fb5d00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:02.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.663+0000 7fb5e27fc700 1 -- 172.21.15.31:0/510368029 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fb5e80858e0 con 0x7fb5ec06de50 2023-10-06T02:23:02.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.663+0000 7fb5f1d9a700 1 --2- 172.21.15.31:0/510368029 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fb5d0066f30 0x7fb5d00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:02.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.664+0000 7fb5f1d9a700 1 --2- 172.21.15.31:0/510368029 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fb5d0066f30 0x7fb5d00693e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec0c7e40 tx=0x7fb5dc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:02.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.667+0000 7fb5e27fc700 1 -- 172.21.15.31:0/510368029 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb5e8052d40 con 0x7fb5ec06de50 2023-10-06T02:23:02.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.869+0000 7fb5f359d700 1 -- 172.21.15.31:0/510368029 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb5ec06f170 con 0x7fb5ec06de50 2023-10-06T02:23:04.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.870+0000 7fb5e27fc700 1 -- 172.21.15.31:0/510368029 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v18) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb5e8026020 con 0x7fb5ec06de50 2023-10-06T02:23:04.186 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-10-06T02:23:04.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.872+0000 7fb5f359d700 1 -- 172.21.15.31:0/510368029 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fb5d0066f30 msgr2=0x7fb5d00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:04.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.872+0000 7fb5f359d700 1 --2- 172.21.15.31:0/510368029 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fb5d0066f30 0x7fb5d00693e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec0c7e40 tx=0x7fb5dc009040 comp rx=0 tx=0).stop 2023-10-06T02:23:04.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.872+0000 7fb5f359d700 1 -- 172.21.15.31:0/510368029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5ec06de50 msgr2=0x7fb5ec0adc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:04.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.872+0000 7fb5f359d700 1 --2- 172.21.15.31:0/510368029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5ec06de50 0x7fb5ec0adc00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb5e800a430 tx=0x7fb5e8019000 comp rx=0 tx=0).stop 2023-10-06T02:23:04.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.872+0000 7fb5f359d700 1 -- 172.21.15.31:0/510368029 shutdown_connections 2023-10-06T02:23:04.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.872+0000 7fb5f359d700 1 --2- 172.21.15.31:0/510368029 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5ec155c60 0x7fb5ec1673b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:04.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.872+0000 7fb5f359d700 1 --2- 172.21.15.31:0/510368029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5ec06de50 0x7fb5ec0adc00 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:04.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.872+0000 7fb5f359d700 1 --2- 172.21.15.31:0/510368029 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fb5d0066f30 0x7fb5d00693e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:04.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.872+0000 7fb5f359d700 1 --2- 172.21.15.31:0/510368029 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb5ec155890 0x7fb5ec15b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:04.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.872+0000 7fb5f359d700 1 -- 172.21.15.31:0/510368029 >> 172.21.15.31:0/510368029 conn(0x7fb5ec06d110 msgr2=0x7fb5ec0b0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:04.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.872+0000 7fb5f359d700 1 -- 172.21.15.31:0/510368029 shutdown_connections 2023-10-06T02:23:04.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:02.872+0000 7fb5f359d700 1 -- 172.21.15.31:0/510368029 wait complete. 2023-10-06T02:23:04.189 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T02:23:04.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.386+0000 7fc1c979d700 1 Processor -- start 2023-10-06T02:23:04.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.386+0000 7fc1c979d700 1 -- start start 2023-10-06T02:23:04.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.387+0000 7fc1c979d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc1c4153dd0 0x7fc1c415de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:04.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.387+0000 7fc1c979d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc1c40cea00 0x7fc1c40ceea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:04.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.387+0000 7fc1c979d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1c415ac50 con 0x7fc1c4153a00 2023-10-06T02:23:04.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.387+0000 7fc1c979d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1c415d010 con 0x7fc1c40cea00 2023-10-06T02:23:04.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.387+0000 7fc1c979d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1c40cf3e0 con 0x7fc1c4153dd0 2023-10-06T02:23:04.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.388+0000 7fc1c8f9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc1c40cea00 0x7fc1c40ceea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:04.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.388+0000 7fc1c8f9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc1c40cea00 0x7fc1c40ceea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56862/0 (socket says 172.21.15.31:56862) 2023-10-06T02:23:04.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.388+0000 7fc1c8f9c700 1 -- 172.21.15.31:0/3506553572 learned_addr learned my addr 172.21.15.31:0/3506553572 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:04.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.388+0000 7fc1c3fff700 1 --1- 172.21.15.31:0/3506553572 >> v1:172.21.15.31:6789/0 conn(0x7fc1c4153a00 0x7fc1c415a540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:39252/0 (socket says 172.21.15.31:39252) 2023-10-06T02:23:04.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.388+0000 7fc1c37fe700 1 --2- 172.21.15.31:0/3506553572 >> v2:172.21.15.62:3300/0 conn(0x7fc1c4153dd0 0x7fc1c415de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:04.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.388+0000 7fc1c8f9c700 1 -- 172.21.15.31:0/3506553572 >> v2:172.21.15.62:3300/0 conn(0x7fc1c4153dd0 msgr2=0x7fc1c415de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:04.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.388+0000 7fc1c8f9c700 1 --2- 172.21.15.31:0/3506553572 >> v2:172.21.15.62:3300/0 conn(0x7fc1c4153dd0 0x7fc1c415de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:04.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.388+0000 7fc1c8f9c700 1 -- 172.21.15.31:0/3506553572 >> v1:172.21.15.31:6789/0 conn(0x7fc1c4153a00 legacy=0x7fc1c415a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:04.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.388+0000 7fc1c8f9c700 1 -- 172.21.15.31:0/3506553572 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1c415af60 con 0x7fc1c40cea00 2023-10-06T02:23:04.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.388+0000 7fc1c2ffd700 1 -- 172.21.15.31:0/3506553572 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2263152328 0 0) 0x7fc1c415ac50 con 0x7fc1c4153a00 2023-10-06T02:23:04.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.388+0000 7fc1c8f9c700 1 --2- 172.21.15.31:0/3506553572 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1c40cea00 0x7fc1c40ceea0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc1c40b3070 tx=0x7fc1bc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=654765edcfcdaf48 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:04.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.389+0000 7fc1c2ffd700 1 -- 172.21.15.31:0/3506553572 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc00e070 con 0x7fc1c40cea00 2023-10-06T02:23:04.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.389+0000 7fc1c2ffd700 1 -- 172.21.15.31:0/3506553572 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1bc00ba20 con 0x7fc1c40cea00 2023-10-06T02:23:04.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.389+0000 7fc1c2ffd700 1 -- 172.21.15.31:0/3506553572 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc0092e0 con 0x7fc1c40cea00 2023-10-06T02:23:04.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.389+0000 7fc1c979d700 1 -- 172.21.15.31:0/3506553572 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1c40cea00 msgr2=0x7fc1c40ceea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:04.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.389+0000 7fc1c979d700 1 --2- 172.21.15.31:0/3506553572 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1c40cea00 0x7fc1c40ceea0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc1c40b3070 tx=0x7fc1bc009840 comp rx=0 tx=0).stop 2023-10-06T02:23:04.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.389+0000 7fc1c979d700 1 -- 172.21.15.31:0/3506553572 shutdown_connections 2023-10-06T02:23:04.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.389+0000 7fc1c979d700 1 --2- 172.21.15.31:0/3506553572 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1c40cea00 0x7fc1c40ceea0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:04.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.389+0000 7fc1c979d700 1 --2- 172.21.15.31:0/3506553572 >> v2:172.21.15.62:3300/0 conn(0x7fc1c4153dd0 0x7fc1c415de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:04.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.389+0000 7fc1c979d700 1 -- 172.21.15.31:0/3506553572 >> 172.21.15.31:0/3506553572 conn(0x7fc1c40703a0 msgr2=0x7fc1c40707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:04.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.389+0000 7fc1c979d700 1 -- 172.21.15.31:0/3506553572 shutdown_connections 2023-10-06T02:23:04.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.389+0000 7fc1c979d700 1 -- 172.21.15.31:0/3506553572 wait complete. 2023-10-06T02:23:04.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.391+0000 7fc1c979d700 1 Processor -- start 2023-10-06T02:23:04.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.391+0000 7fc1c979d700 1 -- start start 2023-10-06T02:23:04.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.392+0000 7fc1c979d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1c40cea00 0x7fc1c4144020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:04.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.392+0000 7fc1c979d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc1c4153a00 0x7fc1c4151270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:04.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.392+0000 7fc1c979d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1c4153dd0 0x7fc1c40c6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:04.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.392+0000 7fc1c979d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c4150d00 con 0x7fc1c40cea00 2023-10-06T02:23:04.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.392+0000 7fc1c979d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c41500f0 con 0x7fc1c4153dd0 2023-10-06T02:23:04.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.392+0000 7fc1c979d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c41503d0 con 0x7fc1c4153a00 2023-10-06T02:23:04.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.392+0000 7fc1c3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1c40cea00 0x7fc1c4144020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:04.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.392+0000 7fc1c3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1c40cea00 0x7fc1c4144020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56870/0 (socket says 172.21.15.31:56870) 2023-10-06T02:23:04.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.392+0000 7fc1c3fff700 1 -- 172.21.15.31:0/3168653976 learned_addr learned my addr 172.21.15.31:0/3168653976 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:04.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.392+0000 7fc1c8f9c700 1 --2- 172.21.15.31:0/3168653976 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1c4153dd0 0x7fc1c40c6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:04.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.392+0000 7fc1c37fe700 1 --2- 172.21.15.31:0/3168653976 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc1c4153a00 0x7fc1c4151270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:04.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.392+0000 7fc1c3fff700 1 -- 172.21.15.31:0/3168653976 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc1c4153a00 msgr2=0x7fc1c4151270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:04.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.392+0000 7fc1c3fff700 1 --2- 172.21.15.31:0/3168653976 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc1c4153a00 0x7fc1c4151270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:04.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.392+0000 7fc1c3fff700 1 -- 172.21.15.31:0/3168653976 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1c4153dd0 msgr2=0x7fc1c40c6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:04.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.392+0000 7fc1c3fff700 1 --2- 172.21.15.31:0/3168653976 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1c4153dd0 0x7fc1c40c6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:04.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.392+0000 7fc1c3fff700 1 -- 172.21.15.31:0/3168653976 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1bc009ae0 con 0x7fc1c40cea00 2023-10-06T02:23:04.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.393+0000 7fc1c3fff700 1 --2- 172.21.15.31:0/3168653976 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1c40cea00 0x7fc1c4144020 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc1b40011e0 tx=0x7fc1b4007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:04.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.393+0000 7fc1abfff700 1 -- 172.21.15.31:0/3168653976 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b4007300 con 0x7fc1c40cea00 2023-10-06T02:23:04.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.393+0000 7fc1abfff700 1 -- 172.21.15.31:0/3168653976 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1b4007460 con 0x7fc1c40cea00 2023-10-06T02:23:04.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.393+0000 7fc1abfff700 1 -- 172.21.15.31:0/3168653976 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b4004830 con 0x7fc1c40cea00 2023-10-06T02:23:04.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.393+0000 7fc1c979d700 1 -- 172.21.15.31:0/3168653976 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1c41506b0 con 0x7fc1c40cea00 2023-10-06T02:23:04.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.393+0000 7fc1c979d700 1 -- 172.21.15.31:0/3168653976 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1c414fde0 con 0x7fc1c40cea00 2023-10-06T02:23:04.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.394+0000 7fc1c979d700 1 -- 172.21.15.31:0/3168653976 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1c404c3c0 con 0x7fc1c40cea00 2023-10-06T02:23:04.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.395+0000 7fc1abfff700 1 -- 172.21.15.31:0/3168653976 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc1b4004be0 con 0x7fc1c40cea00 2023-10-06T02:23:04.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.396+0000 7fc1abfff700 1 --2- 172.21.15.31:0/3168653976 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc1b0066f60 0x7fc1b0069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:04.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.396+0000 7fc1abfff700 1 -- 172.21.15.31:0/3168653976 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fc1b40838e0 con 0x7fc1c40cea00 2023-10-06T02:23:04.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.396+0000 7fc1c37fe700 1 --2- 172.21.15.31:0/3168653976 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc1b0066f60 0x7fc1b0069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:04.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.396+0000 7fc1c37fe700 1 --2- 172.21.15.31:0/3168653976 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc1b0066f60 0x7fc1b0069410 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc1c40cd4e0 tx=0x7fc1b8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:04.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.400+0000 7fc1abfff700 1 -- 172.21.15.31:0/3168653976 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc1b4050d20 con 0x7fc1c40cea00 2023-10-06T02:23:04.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.602+0000 7fc1c979d700 1 -- 172.21.15.31:0/3168653976 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc1c415b240 con 0x7fc1c40cea00 2023-10-06T02:23:04.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.603+0000 7fc1abfff700 1 -- 172.21.15.31:0/3168653976 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v19) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc1b4018020 con 0x7fc1c40cea00 2023-10-06T02:23:04.661 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 8 2023-10-06T02:23:04.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.605+0000 7fc1c979d700 1 -- 172.21.15.31:0/3168653976 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc1b0066f60 msgr2=0x7fc1b0069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:04.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.605+0000 7fc1c979d700 1 --2- 172.21.15.31:0/3168653976 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc1b0066f60 0x7fc1b0069410 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc1c40cd4e0 tx=0x7fc1b8008040 comp rx=0 tx=0).stop 2023-10-06T02:23:04.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.605+0000 7fc1c979d700 1 -- 172.21.15.31:0/3168653976 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1c40cea00 msgr2=0x7fc1c4144020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:04.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.605+0000 7fc1c979d700 1 --2- 172.21.15.31:0/3168653976 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1c40cea00 0x7fc1c4144020 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc1b40011e0 tx=0x7fc1b4007e10 comp rx=0 tx=0).stop 2023-10-06T02:23:04.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.605+0000 7fc1c979d700 1 -- 172.21.15.31:0/3168653976 shutdown_connections 2023-10-06T02:23:04.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.605+0000 7fc1c979d700 1 --2- 172.21.15.31:0/3168653976 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1c4153dd0 0x7fc1c40c6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:04.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.605+0000 7fc1c979d700 1 --2- 172.21.15.31:0/3168653976 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1c40cea00 0x7fc1c4144020 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:04.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.605+0000 7fc1c979d700 1 --2- 172.21.15.31:0/3168653976 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc1b0066f60 0x7fc1b0069410 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:04.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.605+0000 7fc1c979d700 1 --2- 172.21.15.31:0/3168653976 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc1c4153a00 0x7fc1c4151270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:04.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.605+0000 7fc1c979d700 1 -- 172.21.15.31:0/3168653976 >> 172.21.15.31:0/3168653976 conn(0x7fc1c40703a0 msgr2=0x7fc1c415a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:04.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.605+0000 7fc1c979d700 1 -- 172.21.15.31:0/3168653976 shutdown_connections 2023-10-06T02:23:04.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.605+0000 7fc1c979d700 1 -- 172.21.15.31:0/3168653976 wait complete. 2023-10-06T02:23:04.664 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:23:04.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.849+0000 7ff2cf59e700 1 Processor -- start 2023-10-06T02:23:04.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.849+0000 7ff2cf59e700 1 -- start start 2023-10-06T02:23:04.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.850+0000 7ff2cf59e700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff2d015de50 0x7ff2d00cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:04.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.850+0000 7ff2cf59e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff2d00cef40 0x7ff2d015c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:04.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.850+0000 7ff2cf59e700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2d00cf3c0 con 0x7ff2d0153c30 2023-10-06T02:23:04.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.850+0000 7ff2cf59e700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2d015afd0 con 0x7ff2d00cef40 2023-10-06T02:23:04.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.850+0000 7ff2cf59e700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2d015b2b0 con 0x7ff2d015de50 2023-10-06T02:23:04.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.850+0000 7ff2ced9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff2d00cef40 0x7ff2d015c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:04.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.850+0000 7ff2ced9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff2d00cef40 0x7ff2d015c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56872/0 (socket says 172.21.15.31:56872) 2023-10-06T02:23:04.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.850+0000 7ff2ced9d700 1 -- 172.21.15.31:0/3091127943 learned_addr learned my addr 172.21.15.31:0/3091127943 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:04.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.851+0000 7ff2cdd9b700 1 --2- 172.21.15.31:0/3091127943 >> v2:172.21.15.62:3300/0 conn(0x7ff2d015de50 0x7ff2d00cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:04.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.851+0000 7ff2ced9d700 1 -- 172.21.15.31:0/3091127943 >> v2:172.21.15.62:3300/0 conn(0x7ff2d015de50 msgr2=0x7ff2d00cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:04.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.851+0000 7ff2ced9d700 1 --2- 172.21.15.31:0/3091127943 >> v2:172.21.15.62:3300/0 conn(0x7ff2d015de50 0x7ff2d00cea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:04.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.851+0000 7ff2ced9d700 1 -- 172.21.15.31:0/3091127943 >> v1:172.21.15.62:6789/0 conn(0x7ff2d0153c30 legacy=0x7ff2d015a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:04.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.851+0000 7ff2ced9d700 1 -- 172.21.15.31:0/3091127943 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2d015b590 con 0x7ff2d00cef40 2023-10-06T02:23:04.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.851+0000 7ff2ced9d700 1 --2- 172.21.15.31:0/3091127943 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2d00cef40 0x7ff2d015c070 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff2d00713a0 tx=0x7ff2c8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40924c75076b41a7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:04.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.851+0000 7ff2cd59a700 1 -- 172.21.15.31:0/3091127943 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c800e070 con 0x7ff2d00cef40 2023-10-06T02:23:04.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.851+0000 7ff2cd59a700 1 -- 172.21.15.31:0/3091127943 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2c800ba50 con 0x7ff2d00cef40 2023-10-06T02:23:04.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.851+0000 7ff2cd59a700 1 -- 172.21.15.31:0/3091127943 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c80092e0 con 0x7ff2d00cef40 2023-10-06T02:23:04.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.852+0000 7ff2cf59e700 1 -- 172.21.15.31:0/3091127943 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2d00cef40 msgr2=0x7ff2d015c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:04.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.852+0000 7ff2cf59e700 1 --2- 172.21.15.31:0/3091127943 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2d00cef40 0x7ff2d015c070 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff2d00713a0 tx=0x7ff2c8009840 comp rx=0 tx=0).stop 2023-10-06T02:23:04.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.852+0000 7ff2cf59e700 1 -- 172.21.15.31:0/3091127943 shutdown_connections 2023-10-06T02:23:04.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.852+0000 7ff2cf59e700 1 --2- 172.21.15.31:0/3091127943 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2d00cef40 0x7ff2d015c070 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:04.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.852+0000 7ff2cf59e700 1 --2- 172.21.15.31:0/3091127943 >> v2:172.21.15.62:3300/0 conn(0x7ff2d015de50 0x7ff2d00cea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:04.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.852+0000 7ff2cf59e700 1 -- 172.21.15.31:0/3091127943 >> 172.21.15.31:0/3091127943 conn(0x7ff2d00703a0 msgr2=0x7ff2d00707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:04.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.852+0000 7ff2cf59e700 1 -- 172.21.15.31:0/3091127943 shutdown_connections 2023-10-06T02:23:04.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.852+0000 7ff2cf59e700 1 -- 172.21.15.31:0/3091127943 wait complete. 2023-10-06T02:23:04.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.854+0000 7ff2cf59e700 1 Processor -- start 2023-10-06T02:23:04.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.854+0000 7ff2cf59e700 1 -- start start 2023-10-06T02:23:04.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.855+0000 7ff2cf59e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2d00cef40 0x7ff2d0146fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:04.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.855+0000 7ff2cf59e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2d0153c30 0x7ff2d00b4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:04.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.855+0000 7ff2cf59e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff2d015de50 0x7ff2d00c9c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:04.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.855+0000 7ff2cf59e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d00b3ee0 con 0x7ff2d00cef40 2023-10-06T02:23:04.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.855+0000 7ff2cf59e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d00b32d0 con 0x7ff2d0153c30 2023-10-06T02:23:04.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.855+0000 7ff2cf59e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d00b35b0 con 0x7ff2d015de50 2023-10-06T02:23:04.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.855+0000 7ff2ce59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2d00cef40 0x7ff2d0146fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:04.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.855+0000 7ff2ce59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2d00cef40 0x7ff2d0146fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56886/0 (socket says 172.21.15.31:56886) 2023-10-06T02:23:04.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.855+0000 7ff2ce59c700 1 -- 172.21.15.31:0/3780713872 learned_addr learned my addr 172.21.15.31:0/3780713872 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:04.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.855+0000 7ff2ced9d700 1 --2- 172.21.15.31:0/3780713872 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff2d015de50 0x7ff2d00c9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:04.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.855+0000 7ff2cdd9b700 1 --2- 172.21.15.31:0/3780713872 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2d0153c30 0x7ff2d00b4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:04.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.855+0000 7ff2ce59c700 1 -- 172.21.15.31:0/3780713872 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff2d015de50 msgr2=0x7ff2d00c9c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:04.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.855+0000 7ff2ce59c700 1 --2- 172.21.15.31:0/3780713872 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff2d015de50 0x7ff2d00c9c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:04.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.855+0000 7ff2ce59c700 1 -- 172.21.15.31:0/3780713872 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2d0153c30 msgr2=0x7ff2d00b4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:04.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.855+0000 7ff2ce59c700 1 --2- 172.21.15.31:0/3780713872 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2d0153c30 0x7ff2d00b4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:04.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.855+0000 7ff2ce59c700 1 -- 172.21.15.31:0/3780713872 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2c8009ae0 con 0x7ff2d00cef40 2023-10-06T02:23:04.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.856+0000 7ff2ce59c700 1 --2- 172.21.15.31:0/3780713872 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2d00cef40 0x7ff2d0146fd0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff2c000c4e0 tx=0x7ff2c000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:04.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.856+0000 7ff2be7fc700 1 -- 172.21.15.31:0/3780713872 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c0011590 con 0x7ff2d00cef40 2023-10-06T02:23:04.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.856+0000 7ff2be7fc700 1 -- 172.21.15.31:0/3780713872 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2c0011710 con 0x7ff2d00cef40 2023-10-06T02:23:04.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.856+0000 7ff2be7fc700 1 -- 172.21.15.31:0/3780713872 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c0011ad0 con 0x7ff2d00cef40 2023-10-06T02:23:04.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.856+0000 7ff2cf59e700 1 -- 172.21.15.31:0/3780713872 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2d00b3890 con 0x7ff2d00cef40 2023-10-06T02:23:04.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.856+0000 7ff2cf59e700 1 -- 172.21.15.31:0/3780713872 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2d0147510 con 0x7ff2d00cef40 2023-10-06T02:23:04.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.857+0000 7ff2cf59e700 1 -- 172.21.15.31:0/3780713872 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2d00b41c0 con 0x7ff2d00cef40 2023-10-06T02:23:04.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.858+0000 7ff2be7fc700 1 -- 172.21.15.31:0/3780713872 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff2c001b3b0 con 0x7ff2d00cef40 2023-10-06T02:23:04.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.859+0000 7ff2be7fc700 1 --2- 172.21.15.31:0/3780713872 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7ff2b0066f30 0x7ff2b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:04.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.859+0000 7ff2be7fc700 1 -- 172.21.15.31:0/3780713872 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5206+0+0 (secure 0 0 0) 0x7ff2c00836d0 con 0x7ff2d00cef40 2023-10-06T02:23:04.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.859+0000 7ff2cdd9b700 1 --2- 172.21.15.31:0/3780713872 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7ff2b0066f30 0x7ff2b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:04.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.860+0000 7ff2cdd9b700 1 --2- 172.21.15.31:0/3780713872 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7ff2b0066f30 0x7ff2b00693e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff2d0154cc0 tx=0x7ff2c4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:04.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:04.863+0000 7ff2be7fc700 1 -- 172.21.15.31:0/3780713872 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff2c0050b30 con 0x7ff2d00cef40 2023-10-06T02:23:05.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.113+0000 7ff2cf59e700 1 -- 172.21.15.31:0/3780713872 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff2d004c3c0 con 0x7ff2d00cef40 2023-10-06T02:23:05.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.114+0000 7ff2be7fc700 1 -- 172.21.15.31:0/3780713872 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3524 (secure 0 0 0) 0x7ff2c0017020 con 0x7ff2d00cef40 2023-10-06T02:23:05.115 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:23:05.115 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/1808327183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":1808327183},{"type":"v1","addr":"172.21.15.31:6837","nonce":1808327183}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/4263144517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":4263144517},{"type":"v1","addr":"172.21.15.62:6834","nonce":4263144517}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1489208568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":1489208568},{"type":"v1","addr":"172.21.15.62:6835","nonce":1489208568}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4297,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/3809284664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3809284664},{"type":"v1","addr":"172.21.15.31:6836","nonce":3809284664}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-10-06T02:23:05.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.116+0000 7ff2cf59e700 1 -- 172.21.15.31:0/3780713872 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7ff2b0066f30 msgr2=0x7ff2b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:05.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.116+0000 7ff2cf59e700 1 --2- 172.21.15.31:0/3780713872 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7ff2b0066f30 0x7ff2b00693e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff2d0154cc0 tx=0x7ff2c4008040 comp rx=0 tx=0).stop 2023-10-06T02:23:05.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.116+0000 7ff2cf59e700 1 -- 172.21.15.31:0/3780713872 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2d00cef40 msgr2=0x7ff2d0146fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:05.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.116+0000 7ff2cf59e700 1 --2- 172.21.15.31:0/3780713872 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2d00cef40 0x7ff2d0146fd0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff2c000c4e0 tx=0x7ff2c000c7f0 comp rx=0 tx=0).stop 2023-10-06T02:23:05.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.116+0000 7ff2cf59e700 1 -- 172.21.15.31:0/3780713872 shutdown_connections 2023-10-06T02:23:05.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.116+0000 7ff2cf59e700 1 --2- 172.21.15.31:0/3780713872 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7ff2b0066f30 0x7ff2b00693e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:05.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.116+0000 7ff2cf59e700 1 --2- 172.21.15.31:0/3780713872 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff2d015de50 0x7ff2d00c9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:05.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.116+0000 7ff2cf59e700 1 --2- 172.21.15.31:0/3780713872 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2d00cef40 0x7ff2d0146fd0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:05.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.116+0000 7ff2cf59e700 1 --2- 172.21.15.31:0/3780713872 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2d0153c30 0x7ff2d00b4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:05.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.116+0000 7ff2cf59e700 1 -- 172.21.15.31:0/3780713872 >> 172.21.15.31:0/3780713872 conn(0x7ff2d00703a0 msgr2=0x7ff2d0152fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:05.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.117+0000 7ff2cf59e700 1 -- 172.21.15.31:0/3780713872 shutdown_connections 2023-10-06T02:23:05.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.117+0000 7ff2cf59e700 1 -- 172.21.15.31:0/3780713872 wait complete. 2023-10-06T02:23:05.119 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 3 2023-10-06T02:23:05.135 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:23:05.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.319+0000 7fb41e19d700 1 Processor -- start 2023-10-06T02:23:05.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.320+0000 7fb41e19d700 1 -- start start 2023-10-06T02:23:05.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.321+0000 7fb41e19d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb41813fb20 0x7fb41815a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:05.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.321+0000 7fb41e19d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb41813fef0 0x7fb4180c7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:05.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.321+0000 7fb41e19d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb41806aaa0 0x7fb4180c7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:05.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.321+0000 7fb41e19d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4180c8410 con 0x7fb41806aaa0 2023-10-06T02:23:05.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.321+0000 7fb41e19d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb41815ae70 con 0x7fb41813fef0 2023-10-06T02:23:05.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.321+0000 7fb41e19d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb41806af60 con 0x7fb41813fb20 2023-10-06T02:23:05.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.321+0000 7fb41d99c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb41806aaa0 0x7fb4180c7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:05.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.321+0000 7fb41d99c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb41806aaa0 0x7fb4180c7ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56894/0 (socket says 172.21.15.31:56894) 2023-10-06T02:23:05.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.321+0000 7fb41d99c700 1 -- 172.21.15.31:0/1957875393 learned_addr learned my addr 172.21.15.31:0/1957875393 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:05.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.322+0000 7fb41d19b700 1 --2- 172.21.15.31:0/1957875393 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb41813fb20 0x7fb41815a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:05.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.322+0000 7fb41c99a700 1 --2- 172.21.15.31:0/1957875393 >> v2:172.21.15.62:3300/0 conn(0x7fb41813fef0 0x7fb4180c7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:05.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.322+0000 7fb41d99c700 1 -- 172.21.15.31:0/1957875393 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb41813fb20 msgr2=0x7fb41815a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:05.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.322+0000 7fb41d99c700 1 --2- 172.21.15.31:0/1957875393 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb41813fb20 0x7fb41815a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:05.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.322+0000 7fb41d99c700 1 -- 172.21.15.31:0/1957875393 >> v2:172.21.15.62:3300/0 conn(0x7fb41813fef0 msgr2=0x7fb4180c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:05.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.322+0000 7fb41d99c700 1 --2- 172.21.15.31:0/1957875393 >> v2:172.21.15.62:3300/0 conn(0x7fb41813fef0 0x7fb4180c7990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:05.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.322+0000 7fb41d99c700 1 -- 172.21.15.31:0/1957875393 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb41806b240 con 0x7fb41806aaa0 2023-10-06T02:23:05.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.322+0000 7fb41d19b700 1 --2- 172.21.15.31:0/1957875393 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb41813fb20 0x7fb41815a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:23:05.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.322+0000 7fb41d99c700 1 --2- 172.21.15.31:0/1957875393 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb41806aaa0 0x7fb4180c7ed0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb4180cc090 tx=0x7fb410009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=993a7a6eb79099ce server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:05.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.322+0000 7fb40ffff700 1 -- 172.21.15.31:0/1957875393 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb41000e070 con 0x7fb41806aaa0 2023-10-06T02:23:05.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.322+0000 7fb40ffff700 1 -- 172.21.15.31:0/1957875393 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb41000ba50 con 0x7fb41806aaa0 2023-10-06T02:23:05.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.322+0000 7fb40ffff700 1 -- 172.21.15.31:0/1957875393 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4100092e0 con 0x7fb41806aaa0 2023-10-06T02:23:05.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.323+0000 7fb41e19d700 1 -- 172.21.15.31:0/1957875393 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb41806aaa0 msgr2=0x7fb4180c7ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:05.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.323+0000 7fb41e19d700 1 --2- 172.21.15.31:0/1957875393 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb41806aaa0 0x7fb4180c7ed0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb4180cc090 tx=0x7fb410009840 comp rx=0 tx=0).stop 2023-10-06T02:23:05.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.323+0000 7fb41e19d700 1 -- 172.21.15.31:0/1957875393 shutdown_connections 2023-10-06T02:23:05.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.323+0000 7fb41e19d700 1 --2- 172.21.15.31:0/1957875393 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb41806aaa0 0x7fb4180c7ed0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:05.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.323+0000 7fb41e19d700 1 --2- 172.21.15.31:0/1957875393 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb41813fb20 0x7fb41815a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:05.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.323+0000 7fb41e19d700 1 --2- 172.21.15.31:0/1957875393 >> v2:172.21.15.62:3300/0 conn(0x7fb41813fef0 0x7fb4180c7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:05.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.323+0000 7fb41e19d700 1 -- 172.21.15.31:0/1957875393 >> 172.21.15.31:0/1957875393 conn(0x7fb41806d170 msgr2=0x7fb4180cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:05.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.323+0000 7fb41e19d700 1 -- 172.21.15.31:0/1957875393 shutdown_connections 2023-10-06T02:23:05.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.323+0000 7fb41e19d700 1 -- 172.21.15.31:0/1957875393 wait complete. 2023-10-06T02:23:05.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.324+0000 7fb41e19d700 1 Processor -- start 2023-10-06T02:23:05.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.325+0000 7fb41e19d700 1 -- start start 2023-10-06T02:23:05.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.325+0000 7fb41e19d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb41806aaa0 0x7fb41813d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:05.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.325+0000 7fb41e19d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb41813fb20 0x7fb418159390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:05.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.325+0000 7fb41e19d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb41813fef0 0x7fb4180c75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:05.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.325+0000 7fb41e19d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb418156be0 con 0x7fb41813fb20 2023-10-06T02:23:05.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.326+0000 7fb41e19d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb418169cd0 con 0x7fb41806aaa0 2023-10-06T02:23:05.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.326+0000 7fb41e19d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb418169fb0 con 0x7fb41813fef0 2023-10-06T02:23:05.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.326+0000 7fb41c99a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb41813fb20 0x7fb418159390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:05.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.326+0000 7fb41c99a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb41813fb20 0x7fb418159390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56906/0 (socket says 172.21.15.31:56906) 2023-10-06T02:23:05.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.326+0000 7fb41c99a700 1 -- 172.21.15.31:0/1886943077 learned_addr learned my addr 172.21.15.31:0/1886943077 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:05.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.326+0000 7fb41d19b700 1 --2- 172.21.15.31:0/1886943077 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb41806aaa0 0x7fb41813d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:05.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.326+0000 7fb41d99c700 1 --2- 172.21.15.31:0/1886943077 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb41813fef0 0x7fb4180c75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:05.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.326+0000 7fb41c99a700 1 -- 172.21.15.31:0/1886943077 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb41813fef0 msgr2=0x7fb4180c75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:05.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.326+0000 7fb41c99a700 1 --2- 172.21.15.31:0/1886943077 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb41813fef0 0x7fb4180c75f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:05.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.326+0000 7fb41c99a700 1 -- 172.21.15.31:0/1886943077 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb41806aaa0 msgr2=0x7fb41813d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:05.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.326+0000 7fb41c99a700 1 --2- 172.21.15.31:0/1886943077 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb41806aaa0 0x7fb41813d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:05.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.326+0000 7fb41c99a700 1 -- 172.21.15.31:0/1886943077 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb410009ae0 con 0x7fb41813fb20 2023-10-06T02:23:05.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.326+0000 7fb41c99a700 1 --2- 172.21.15.31:0/1886943077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb41813fb20 0x7fb418159390 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb40800d810 tx=0x7fb408008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:05.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.327+0000 7fb40d7fa700 1 -- 172.21.15.31:0/1886943077 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4080125f0 con 0x7fb41813fb20 2023-10-06T02:23:05.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.327+0000 7fb40d7fa700 1 -- 172.21.15.31:0/1886943077 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb408012750 con 0x7fb41813fb20 2023-10-06T02:23:05.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.327+0000 7fb40d7fa700 1 -- 172.21.15.31:0/1886943077 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb408012b10 con 0x7fb41813fb20 2023-10-06T02:23:05.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.327+0000 7fb41e19d700 1 -- 172.21.15.31:0/1886943077 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb41816a290 con 0x7fb41813fb20 2023-10-06T02:23:05.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.327+0000 7fb41e19d700 1 -- 172.21.15.31:0/1886943077 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb41816a850 con 0x7fb41813fb20 2023-10-06T02:23:05.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.328+0000 7fb41e19d700 1 -- 172.21.15.31:0/1886943077 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb41816a9e0 con 0x7fb41813fb20 2023-10-06T02:23:05.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.329+0000 7fb40d7fa700 1 -- 172.21.15.31:0/1886943077 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb40801e460 con 0x7fb41813fb20 2023-10-06T02:23:05.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.330+0000 7fb40d7fa700 1 --2- 172.21.15.31:0/1886943077 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fb3fc066f30 0x7fb3fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:05.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.330+0000 7fb40d7fa700 1 -- 172.21.15.31:0/1886943077 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fb4080849a0 con 0x7fb41813fb20 2023-10-06T02:23:05.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.330+0000 7fb41d19b700 1 --2- 172.21.15.31:0/1886943077 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fb3fc066f30 0x7fb3fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:05.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.330+0000 7fb41d19b700 1 --2- 172.21.15.31:0/1886943077 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fb3fc066f30 0x7fb3fc0693e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb4180ae150 tx=0x7fb414008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:05.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.334+0000 7fb40d7fa700 1 -- 172.21.15.31:0/1886943077 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb408051d60 con 0x7fb41813fb20 2023-10-06T02:23:05.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.536+0000 7fb41e19d700 1 -- 172.21.15.31:0/1886943077 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb41804c3c0 con 0x7fb41813fb20 2023-10-06T02:23:05.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.537+0000 7fb40d7fa700 1 -- 172.21.15.31:0/1886943077 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+13599 (secure 0 0 0) 0x7fb408055380 con 0x7fb41813fb20 2023-10-06T02:23:05.538 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:23:05.539 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":19,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:23:02.976797+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:23:05.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.539+0000 7fb41e19d700 1 -- 172.21.15.31:0/1886943077 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fb3fc066f30 msgr2=0x7fb3fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:05.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.539+0000 7fb41e19d700 1 --2- 172.21.15.31:0/1886943077 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fb3fc066f30 0x7fb3fc0693e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb4180ae150 tx=0x7fb414008040 comp rx=0 tx=0).stop 2023-10-06T02:23:05.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.539+0000 7fb41e19d700 1 -- 172.21.15.31:0/1886943077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb41813fb20 msgr2=0x7fb418159390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:05.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.539+0000 7fb41e19d700 1 --2- 172.21.15.31:0/1886943077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb41813fb20 0x7fb418159390 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb40800d810 tx=0x7fb408008040 comp rx=0 tx=0).stop 2023-10-06T02:23:05.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.539+0000 7fb41e19d700 1 -- 172.21.15.31:0/1886943077 shutdown_connections 2023-10-06T02:23:05.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.539+0000 7fb41e19d700 1 --2- 172.21.15.31:0/1886943077 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fb3fc066f30 0x7fb3fc0693e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:05.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.539+0000 7fb41e19d700 1 --2- 172.21.15.31:0/1886943077 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb41813fef0 0x7fb4180c75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:05.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.539+0000 7fb41e19d700 1 --2- 172.21.15.31:0/1886943077 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb41806aaa0 0x7fb41813d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:05.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.539+0000 7fb41e19d700 1 --2- 172.21.15.31:0/1886943077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb41813fb20 0x7fb418159390 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:05.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.539+0000 7fb41e19d700 1 -- 172.21.15.31:0/1886943077 >> 172.21.15.31:0/1886943077 conn(0x7fb41806d170 msgr2=0x7fb4180caf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:05.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.539+0000 7fb41e19d700 1 -- 172.21.15.31:0/1886943077 shutdown_connections 2023-10-06T02:23:05.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.539+0000 7fb41e19d700 1 -- 172.21.15.31:0/1886943077 wait complete. 2023-10-06T02:23:05.558 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T02:22:53.183425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T02:23:00.046967+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-10-06T02:23:05.558 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T02:23:05.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.747+0000 7fa7ff59e700 1 Processor -- start 2023-10-06T02:23:05.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.747+0000 7fa7ff59e700 1 -- start start 2023-10-06T02:23:05.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.748+0000 7fa7ff59e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa800159580 0x7fa8000b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:05.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.748+0000 7fa7ff59e700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa80006ded0 0x7fa8000b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:05.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.748+0000 7fa7ff59e700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa80015bc40 con 0x7fa8000cb1e0 2023-10-06T02:23:05.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.748+0000 7fa7ff59e700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa80015b070 con 0x7fa800159580 2023-10-06T02:23:05.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.748+0000 7fa7ff59e700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa80015b310 con 0x7fa80006ded0 2023-10-06T02:23:05.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.748+0000 7fa7fe59c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa800159580 0x7fa8000b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:05.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.748+0000 7fa7fe59c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa800159580 0x7fa8000b23f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56912/0 (socket says 172.21.15.31:56912) 2023-10-06T02:23:05.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.748+0000 7fa7fe59c700 1 -- 172.21.15.31:0/3717357114 learned_addr learned my addr 172.21.15.31:0/3717357114 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:05.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.749+0000 7fa7fed9d700 1 --2- 172.21.15.31:0/3717357114 >> v2:172.21.15.62:3300/0 conn(0x7fa80006ded0 0x7fa8000b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:05.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.749+0000 7fa7fe59c700 1 -- 172.21.15.31:0/3717357114 >> v2:172.21.15.62:3300/0 conn(0x7fa80006ded0 msgr2=0x7fa8000b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:05.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.749+0000 7fa7fe59c700 1 --2- 172.21.15.31:0/3717357114 >> v2:172.21.15.62:3300/0 conn(0x7fa80006ded0 0x7fa8000b2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:05.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.749+0000 7fa7fe59c700 1 -- 172.21.15.31:0/3717357114 >> v1:172.21.15.62:6789/0 conn(0x7fa8000cb1e0 legacy=0x7fa80006d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:05.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.749+0000 7fa7fe59c700 1 -- 172.21.15.31:0/3717357114 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa80015b5f0 con 0x7fa800159580 2023-10-06T02:23:05.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.749+0000 7fa7fe59c700 1 --2- 172.21.15.31:0/3717357114 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa800159580 0x7fa8000b23f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa8000b13e0 tx=0x7fa7f0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a16e191080a9630c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:05.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.749+0000 7fa7fd59a700 1 -- 172.21.15.31:0/3717357114 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f0009e70 con 0x7fa800159580 2023-10-06T02:23:05.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.750+0000 7fa7fd59a700 1 -- 172.21.15.31:0/3717357114 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7f000b860 con 0x7fa800159580 2023-10-06T02:23:05.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.750+0000 7fa7fd59a700 1 -- 172.21.15.31:0/3717357114 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f000bc00 con 0x7fa800159580 2023-10-06T02:23:05.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.750+0000 7fa7ff59e700 1 -- 172.21.15.31:0/3717357114 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa800159580 msgr2=0x7fa8000b23f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:05.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.750+0000 7fa7ff59e700 1 --2- 172.21.15.31:0/3717357114 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa800159580 0x7fa8000b23f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa8000b13e0 tx=0x7fa7f0009250 comp rx=0 tx=0).stop 2023-10-06T02:23:05.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.750+0000 7fa7ff59e700 1 -- 172.21.15.31:0/3717357114 shutdown_connections 2023-10-06T02:23:05.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.750+0000 7fa7ff59e700 1 --2- 172.21.15.31:0/3717357114 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa800159580 0x7fa8000b23f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:05.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.750+0000 7fa7ff59e700 1 --2- 172.21.15.31:0/3717357114 >> v2:172.21.15.62:3300/0 conn(0x7fa80006ded0 0x7fa8000b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:05.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.750+0000 7fa7ff59e700 1 -- 172.21.15.31:0/3717357114 >> 172.21.15.31:0/3717357114 conn(0x7fa8000703a0 msgr2=0x7fa8000707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:05.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.750+0000 7fa7ff59e700 1 -- 172.21.15.31:0/3717357114 shutdown_connections 2023-10-06T02:23:05.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.750+0000 7fa7ff59e700 1 -- 172.21.15.31:0/3717357114 wait complete. 2023-10-06T02:23:05.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.752+0000 7fa7ff59e700 1 Processor -- start 2023-10-06T02:23:05.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.752+0000 7fa7ff59e700 1 -- start start 2023-10-06T02:23:05.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.753+0000 7fa7ff59e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa80006ded0 0x7fa800169820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:05.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.753+0000 7fa7ff59e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8000cb1e0 0x7fa8000d0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:05.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.753+0000 7fa7ff59e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa800159580 0x7fa8000d0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:05.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.753+0000 7fa7ff59e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8000c6b80 con 0x7fa80006ded0 2023-10-06T02:23:05.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.753+0000 7fa7ff59e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8001475e0 con 0x7fa8000cb1e0 2023-10-06T02:23:05.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.753+0000 7fa7ff59e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8000c5540 con 0x7fa800159580 2023-10-06T02:23:05.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.753+0000 7fa7fe59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa80006ded0 0x7fa800169820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:05.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.753+0000 7fa7fe59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa80006ded0 0x7fa800169820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56922/0 (socket says 172.21.15.31:56922) 2023-10-06T02:23:05.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.753+0000 7fa7fe59c700 1 -- 172.21.15.31:0/2662481561 learned_addr learned my addr 172.21.15.31:0/2662481561 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:05.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.753+0000 7fa7fe59c700 1 -- 172.21.15.31:0/2662481561 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa800159580 msgr2=0x7fa8000d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:05.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.753+0000 7fa7fe59c700 1 --2- 172.21.15.31:0/2662481561 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa800159580 0x7fa8000d0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:05.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.753+0000 7fa7fe59c700 1 -- 172.21.15.31:0/2662481561 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8000cb1e0 msgr2=0x7fa8000d0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:05.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.753+0000 7fa7fe59c700 1 --2- 172.21.15.31:0/2662481561 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8000cb1e0 0x7fa8000d0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:05.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.753+0000 7fa7fe59c700 1 -- 172.21.15.31:0/2662481561 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7f00094f0 con 0x7fa80006ded0 2023-10-06T02:23:05.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.754+0000 7fa7fe59c700 1 --2- 172.21.15.31:0/2662481561 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa80006ded0 0x7fa800169820 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa80006ea10 tx=0x7fa7f0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:05.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.754+0000 7fa7ee7fc700 1 -- 172.21.15.31:0/2662481561 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f000baa0 con 0x7fa80006ded0 2023-10-06T02:23:05.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.754+0000 7fa7ee7fc700 1 -- 172.21.15.31:0/2662481561 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7f0025420 con 0x7fa80006ded0 2023-10-06T02:23:05.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.754+0000 7fa7ee7fc700 1 -- 172.21.15.31:0/2662481561 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f00257e0 con 0x7fa80006ded0 2023-10-06T02:23:05.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.754+0000 7fa7ff59e700 1 -- 172.21.15.31:0/2662481561 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8000c5820 con 0x7fa80006ded0 2023-10-06T02:23:05.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.754+0000 7fa7ff59e700 1 -- 172.21.15.31:0/2662481561 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8000c5de0 con 0x7fa80006ded0 2023-10-06T02:23:05.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.755+0000 7fa7ff59e700 1 -- 172.21.15.31:0/2662481561 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa800088b50 con 0x7fa80006ded0 2023-10-06T02:23:05.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.756+0000 7fa7ee7fc700 1 -- 172.21.15.31:0/2662481561 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa7f0023070 con 0x7fa80006ded0 2023-10-06T02:23:05.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.757+0000 7fa7ee7fc700 1 --2- 172.21.15.31:0/2662481561 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa7e0066f30 0x7fa7e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:05.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.757+0000 7fa7ee7fc700 1 -- 172.21.15.31:0/2662481561 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fa7f00858e0 con 0x7fa80006ded0 2023-10-06T02:23:05.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.757+0000 7fa7fdd9b700 1 --2- 172.21.15.31:0/2662481561 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa7e0066f30 0x7fa7e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:05.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.757+0000 7fa7fdd9b700 1 --2- 172.21.15.31:0/2662481561 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa7e0066f30 0x7fa7e00693e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa8000c73f0 tx=0x7fa7f4007b40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:05.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.761+0000 7fa7ee7fc700 1 -- 172.21.15.31:0/2662481561 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa7f0052d40 con 0x7fa80006ded0 2023-10-06T02:23:05.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.965+0000 7fa7ff59e700 1 -- 172.21.15.31:0/2662481561 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa80015b8d0 con 0x7fa80006ded0 2023-10-06T02:23:05.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.965+0000 7fa7ee7fc700 1 -- 172.21.15.31:0/2662481561 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v19) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa7f0056360 con 0x7fa80006ded0 2023-10-06T02:23:05.967 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-10-06T02:23:05.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.967+0000 7fa7ff59e700 1 -- 172.21.15.31:0/2662481561 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa7e0066f30 msgr2=0x7fa7e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:05.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.967+0000 7fa7ff59e700 1 --2- 172.21.15.31:0/2662481561 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa7e0066f30 0x7fa7e00693e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa8000c73f0 tx=0x7fa7f4007b40 comp rx=0 tx=0).stop 2023-10-06T02:23:05.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.967+0000 7fa7ff59e700 1 -- 172.21.15.31:0/2662481561 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa80006ded0 msgr2=0x7fa800169820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:05.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.967+0000 7fa7ff59e700 1 --2- 172.21.15.31:0/2662481561 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa80006ded0 0x7fa800169820 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa80006ea10 tx=0x7fa7f0017000 comp rx=0 tx=0).stop 2023-10-06T02:23:05.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.968+0000 7fa7ff59e700 1 -- 172.21.15.31:0/2662481561 shutdown_connections 2023-10-06T02:23:05.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.968+0000 7fa7ff59e700 1 --2- 172.21.15.31:0/2662481561 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa7e0066f30 0x7fa7e00693e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:05.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.968+0000 7fa7ff59e700 1 --2- 172.21.15.31:0/2662481561 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa800159580 0x7fa8000d0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:05.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.968+0000 7fa7ff59e700 1 --2- 172.21.15.31:0/2662481561 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa80006ded0 0x7fa800169820 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:05.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.968+0000 7fa7ff59e700 1 --2- 172.21.15.31:0/2662481561 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8000cb1e0 0x7fa8000d0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:05.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.968+0000 7fa7ff59e700 1 -- 172.21.15.31:0/2662481561 >> 172.21.15.31:0/2662481561 conn(0x7fa8000703a0 msgr2=0x7fa8000b5270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:05.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.968+0000 7fa7ff59e700 1 -- 172.21.15.31:0/2662481561 shutdown_connections 2023-10-06T02:23:05.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:05.968+0000 7fa7ff59e700 1 -- 172.21.15.31:0/2662481561 wait complete. 2023-10-06T02:23:05.987 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T02:23:06.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.176+0000 7f4a40f9d700 1 Processor -- start 2023-10-06T02:23:06.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.176+0000 7f4a40f9d700 1 -- start start 2023-10-06T02:23:06.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.177+0000 7f4a40f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4a3c140a70 0x7f4a3c069b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:06.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.177+0000 7f4a40f9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a3c15b640 con 0x7f4a3c06de50 2023-10-06T02:23:06.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.177+0000 7f4a40f9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a3c15b920 con 0x7f4a3c0c7c60 2023-10-06T02:23:06.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.177+0000 7f4a40f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3c0c6800 con 0x7f4a3c140a70 2023-10-06T02:23:06.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.177+0000 7f4a3affd700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f4a3c06de50 0x7f4a3c157a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:39266/0 (socket says 172.21.15.31:39266) 2023-10-06T02:23:06.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.177+0000 7f4a3affd700 1 -- 172.21.15.31:0/2456133479 learned_addr learned my addr 172.21.15.31:0/2456133479 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:23:06.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.178+0000 7f4a3b7fe700 1 --2- 172.21.15.31:0/2456133479 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4a3c140a70 0x7f4a3c069b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:06.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.178+0000 7f4a3a7fc700 1 -- 172.21.15.31:0/2456133479 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2646659359 0 0) 0x7f4a3c15b640 con 0x7f4a3c06de50 2023-10-06T02:23:06.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.178+0000 7f4a3a7fc700 1 -- 172.21.15.31:0/2456133479 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a24003570 con 0x7f4a3c06de50 2023-10-06T02:23:06.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.178+0000 7f4a3a7fc700 1 -- 172.21.15.31:0/2456133479 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2591469186 0 0) 0x7f4a3c15b920 con 0x7f4a3c0c7c60 2023-10-06T02:23:06.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.178+0000 7f4a3a7fc700 1 -- 172.21.15.31:0/2456133479 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a3c15b640 con 0x7f4a3c0c7c60 2023-10-06T02:23:06.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.178+0000 7f4a3a7fc700 1 -- 172.21.15.31:0/2456133479 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3875863549 0 0) 0x7f4a24003570 con 0x7f4a3c06de50 2023-10-06T02:23:06.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.178+0000 7f4a3a7fc700 1 -- 172.21.15.31:0/2456133479 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4a3c140a70 msgr2=0x7f4a3c069b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:06.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.178+0000 7f4a3a7fc700 1 --2- 172.21.15.31:0/2456133479 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4a3c140a70 0x7f4a3c069b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:06.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.178+0000 7f4a3a7fc700 1 -- 172.21.15.31:0/2456133479 >> v1:172.21.15.62:6789/0 conn(0x7f4a3c0c7c60 legacy=0x7f4a3c158190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:06.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.178+0000 7f4a3a7fc700 1 -- 172.21.15.31:0/2456133479 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a3c15bc00 con 0x7f4a3c06de50 2023-10-06T02:23:06.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.178+0000 7f4a3a7fc700 1 -- 172.21.15.31:0/2456133479 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7f4a30001470 con 0x7f4a3c06de50 2023-10-06T02:23:06.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.178+0000 7f4a3a7fc700 1 -- 172.21.15.31:0/2456133479 >> v1:172.21.15.31:6789/0 conn(0x7f4a3c06de50 legacy=0x7f4a3c157a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:06.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.178+0000 7f4a3a7fc700 1 --2- 172.21.15.31:0/2456133479 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a24003c20 0x7f4a24004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:06.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.178+0000 7f4a3a7fc700 1 --2- 172.21.15.31:0/2456133479 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4a3c140a70 0x7f4a24004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:06.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.178+0000 7f4a3a7fc700 1 --2- 172.21.15.31:0/2456133479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a24004d50 0x7f4a240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:06.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.178+0000 7f4a3a7fc700 1 -- 172.21.15.31:0/2456133479 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3c0c6800 con 0x7f4a24003c20 2023-10-06T02:23:06.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.178+0000 7f4a3a7fc700 1 -- 172.21.15.31:0/2456133479 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a2400bb70 con 0x7f4a24004d50 2023-10-06T02:23:06.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.178+0000 7f4a3a7fc700 1 -- 172.21.15.31:0/2456133479 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a2400bd30 con 0x7f4a3c140a70 2023-10-06T02:23:06.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.178+0000 7f4a3b7fe700 1 --2- 172.21.15.31:0/2456133479 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a24003c20 0x7f4a24004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:06.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.179+0000 7f4a3affd700 1 --2- 172.21.15.31:0/2456133479 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4a3c140a70 0x7f4a24004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:06.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.179+0000 7f4a3bfff700 1 --2- 172.21.15.31:0/2456133479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a24004d50 0x7f4a240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:06.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.179+0000 7f4a3b7fe700 1 -- 172.21.15.31:0/2456133479 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4a3c140a70 msgr2=0x7f4a24004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:06.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.179+0000 7f4a3b7fe700 1 --2- 172.21.15.31:0/2456133479 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4a3c140a70 0x7f4a24004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:06.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.179+0000 7f4a3b7fe700 1 -- 172.21.15.31:0/2456133479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a24004d50 msgr2=0x7f4a240051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:06.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.179+0000 7f4a3b7fe700 1 --2- 172.21.15.31:0/2456133479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a24004d50 0x7f4a240051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:06.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.179+0000 7f4a3b7fe700 1 -- 172.21.15.31:0/2456133479 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a24003570 con 0x7f4a24003c20 2023-10-06T02:23:06.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.179+0000 7f4a3affd700 1 --2- 172.21.15.31:0/2456133479 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4a3c140a70 0x7f4a24004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:23:06.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.179+0000 7f4a3b7fe700 1 --2- 172.21.15.31:0/2456133479 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a24003c20 0x7f4a24004030 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c009c00 tx=0x7f4a2c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b24e2024050a7a7e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:06.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.179+0000 7f4a3a7fc700 1 -- 172.21.15.31:0/2456133479 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a2c00e710 con 0x7f4a24003c20 2023-10-06T02:23:06.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.179+0000 7f4a3a7fc700 1 -- 172.21.15.31:0/2456133479 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a2c00e870 con 0x7f4a24003c20 2023-10-06T02:23:06.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.179+0000 7f4a3a7fc700 1 -- 172.21.15.31:0/2456133479 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a2c00ec30 con 0x7f4a24003c20 2023-10-06T02:23:06.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.180+0000 7f4a40f9d700 1 -- 172.21.15.31:0/2456133479 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a24003c20 msgr2=0x7f4a24004030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:06.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.180+0000 7f4a40f9d700 1 --2- 172.21.15.31:0/2456133479 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a24003c20 0x7f4a24004030 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c009c00 tx=0x7f4a2c0091a0 comp rx=0 tx=0).stop 2023-10-06T02:23:06.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.180+0000 7f4a3bfff700 1 -- 172.21.15.31:0/2456133479 reap_dead start 2023-10-06T02:23:06.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.180+0000 7f4a40f9d700 1 -- 172.21.15.31:0/2456133479 shutdown_connections 2023-10-06T02:23:06.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.180+0000 7f4a40f9d700 1 -- 172.21.15.31:0/2456133479 >> 172.21.15.31:0/2456133479 conn(0x7f4a3c06d110 msgr2=0x7f4a3c0cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:06.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.180+0000 7f4a40f9d700 1 -- 172.21.15.31:0/2456133479 shutdown_connections 2023-10-06T02:23:06.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.180+0000 7f4a40f9d700 1 -- 172.21.15.31:0/2456133479 wait complete. 2023-10-06T02:23:06.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.181+0000 7f4a40f9d700 1 Processor -- start 2023-10-06T02:23:06.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.182+0000 7f4a40f9d700 1 -- start start 2023-10-06T02:23:06.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.182+0000 7f4a40f9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a24003c20 0x7f4a3c168490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:06.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.182+0000 7f4a40f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4a3c0aeed0 0x7f4a3c13bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:06.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.182+0000 7f4a40f9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a3c069a00 0x7f4a3c069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:06.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.182+0000 7f4a40f9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3c06a3e0 con 0x7f4a24003c20 2023-10-06T02:23:06.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.182+0000 7f4a40f9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3c0b0060 con 0x7f4a3c069a00 2023-10-06T02:23:06.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.182+0000 7f4a40f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3c1689d0 con 0x7f4a3c0aeed0 2023-10-06T02:23:06.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.183+0000 7f4a3b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a24003c20 0x7f4a3c168490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:06.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.183+0000 7f4a3b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a24003c20 0x7f4a3c168490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56932/0 (socket says 172.21.15.31:56932) 2023-10-06T02:23:06.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.183+0000 7f4a3b7fe700 1 -- 172.21.15.31:0/3096981373 learned_addr learned my addr 172.21.15.31:0/3096981373 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:06.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.183+0000 7f4a3affd700 1 --2- 172.21.15.31:0/3096981373 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4a3c0aeed0 0x7f4a3c13bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:06.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.183+0000 7f4a3bfff700 1 --2- 172.21.15.31:0/3096981373 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a3c069a00 0x7f4a3c069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:06.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.183+0000 7f4a3b7fe700 1 -- 172.21.15.31:0/3096981373 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4a3c0aeed0 msgr2=0x7f4a3c13bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:06.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.183+0000 7f4a3b7fe700 1 --2- 172.21.15.31:0/3096981373 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4a3c0aeed0 0x7f4a3c13bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:06.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.183+0000 7f4a3b7fe700 1 -- 172.21.15.31:0/3096981373 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a3c069a00 msgr2=0x7f4a3c069ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:06.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.183+0000 7f4a3b7fe700 1 --2- 172.21.15.31:0/3096981373 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a3c069a00 0x7f4a3c069ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:06.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.183+0000 7f4a3b7fe700 1 -- 172.21.15.31:0/3096981373 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a3c0af450 con 0x7f4a24003c20 2023-10-06T02:23:06.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.183+0000 7f4a3bfff700 1 --2- 172.21.15.31:0/3096981373 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a3c069a00 0x7f4a3c069ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:23:06.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.183+0000 7f4a3b7fe700 1 --2- 172.21.15.31:0/3096981373 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a24003c20 0x7f4a3c168490 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4a2400bb40 tx=0x7f4a2c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:06.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.184+0000 7f4a237fe700 1 -- 172.21.15.31:0/3096981373 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a2c022070 con 0x7f4a24003c20 2023-10-06T02:23:06.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.184+0000 7f4a40f9d700 1 -- 172.21.15.31:0/3096981373 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a3c0af730 con 0x7f4a24003c20 2023-10-06T02:23:06.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.184+0000 7f4a237fe700 1 -- 172.21.15.31:0/3096981373 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a2c00e9e0 con 0x7f4a24003c20 2023-10-06T02:23:06.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.184+0000 7f4a237fe700 1 -- 172.21.15.31:0/3096981373 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a2c01c450 con 0x7f4a24003c20 2023-10-06T02:23:06.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.184+0000 7f4a40f9d700 1 -- 172.21.15.31:0/3096981373 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a3c0afcf0 con 0x7f4a24003c20 2023-10-06T02:23:06.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.185+0000 7f4a40f9d700 1 -- 172.21.15.31:0/3096981373 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a3c04c3c0 con 0x7f4a24003c20 2023-10-06T02:23:06.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.186+0000 7f4a237fe700 1 -- 172.21.15.31:0/3096981373 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4a2c01c8a0 con 0x7f4a24003c20 2023-10-06T02:23:06.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.186+0000 7f4a237fe700 1 --2- 172.21.15.31:0/3096981373 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f4a28066f30 0x7f4a280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:06.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.187+0000 7f4a237fe700 1 -- 172.21.15.31:0/3096981373 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f4a2c085c40 con 0x7f4a24003c20 2023-10-06T02:23:06.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.187+0000 7f4a3affd700 1 --2- 172.21.15.31:0/3096981373 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f4a28066f30 0x7f4a280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:06.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.187+0000 7f4a3affd700 1 --2- 172.21.15.31:0/3096981373 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f4a28066f30 0x7f4a280693e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4a30000e10 tx=0x7f4a30007040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:06.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.191+0000 7f4a237fe700 1 -- 172.21.15.31:0/3096981373 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4a2c053080 con 0x7f4a24003c20 2023-10-06T02:23:06.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.392+0000 7f4a40f9d700 1 -- 172.21.15.31:0/3096981373 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f4a3c15bee0 con 0x7f4a24003c20 2023-10-06T02:23:06.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.392+0000 7f4a237fe700 1 -- 172.21.15.31:0/3096981373 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v19) v1 ==== 92+0+10 (secure 0 0 0) 0x7f4a2c02b0a0 con 0x7f4a24003c20 2023-10-06T02:23:06.394 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 8 2023-10-06T02:23:06.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.394+0000 7f4a40f9d700 1 -- 172.21.15.31:0/3096981373 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f4a28066f30 msgr2=0x7f4a280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:06.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.394+0000 7f4a40f9d700 1 --2- 172.21.15.31:0/3096981373 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f4a28066f30 0x7f4a280693e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4a30000e10 tx=0x7f4a30007040 comp rx=0 tx=0).stop 2023-10-06T02:23:06.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.394+0000 7f4a40f9d700 1 -- 172.21.15.31:0/3096981373 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a24003c20 msgr2=0x7f4a3c168490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:06.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.394+0000 7f4a40f9d700 1 --2- 172.21.15.31:0/3096981373 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a24003c20 0x7f4a3c168490 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4a2400bb40 tx=0x7f4a2c01a000 comp rx=0 tx=0).stop 2023-10-06T02:23:06.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.394+0000 7f4a40f9d700 1 -- 172.21.15.31:0/3096981373 shutdown_connections 2023-10-06T02:23:06.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.394+0000 7f4a40f9d700 1 --2- 172.21.15.31:0/3096981373 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a3c069a00 0x7f4a3c069ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:06.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.394+0000 7f4a40f9d700 1 --2- 172.21.15.31:0/3096981373 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a24003c20 0x7f4a3c168490 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:06.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.394+0000 7f4a40f9d700 1 --2- 172.21.15.31:0/3096981373 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f4a28066f30 0x7f4a280693e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:06.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.394+0000 7f4a40f9d700 1 --2- 172.21.15.31:0/3096981373 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4a3c0aeed0 0x7f4a3c13bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:06.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.394+0000 7f4a40f9d700 1 -- 172.21.15.31:0/3096981373 >> 172.21.15.31:0/3096981373 conn(0x7f4a3c06d110 msgr2=0x7f4a3c0c93f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:06.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.394+0000 7f4a40f9d700 1 -- 172.21.15.31:0/3096981373 shutdown_connections 2023-10-06T02:23:06.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.394+0000 7f4a40f9d700 1 -- 172.21.15.31:0/3096981373 wait complete. 2023-10-06T02:23:06.413 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T02:23:06.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.599+0000 7fbb3599d700 1 Processor -- start 2023-10-06T02:23:06.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.599+0000 7fbb3599d700 1 -- start start 2023-10-06T02:23:06.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.600+0000 7fbb3599d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fbb3013fb20 0x7fbb3015a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:06.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.600+0000 7fbb3599d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fbb3013fef0 0x7fbb300c7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:06.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.600+0000 7fbb3599d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbb3006aaa0 0x7fbb300c7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:06.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.600+0000 7fbb3599d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb300c8410 con 0x7fbb3013fb20 2023-10-06T02:23:06.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.600+0000 7fbb3599d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb3015ae70 con 0x7fbb3013fef0 2023-10-06T02:23:06.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.600+0000 7fbb3599d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3006af60 con 0x7fbb3006aaa0 2023-10-06T02:23:06.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.601+0000 7fbb3499b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fbb3013fb20 0x7fbb3015a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:06.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.601+0000 7fbb3499b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fbb3013fb20 0x7fbb3015a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56936/0 (socket says 172.21.15.31:56936) 2023-10-06T02:23:06.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.601+0000 7fbb3499b700 1 -- 172.21.15.31:0/3302625732 learned_addr learned my addr 172.21.15.31:0/3302625732 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:06.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.601+0000 7fbb3519c700 1 --2- 172.21.15.31:0/3302625732 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbb3006aaa0 0x7fbb300c7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:06.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.601+0000 7fbb2ffff700 1 --2- 172.21.15.31:0/3302625732 >> v2:172.21.15.62:3300/0 conn(0x7fbb3013fef0 0x7fbb300c7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:06.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.601+0000 7fbb3499b700 1 -- 172.21.15.31:0/3302625732 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbb3006aaa0 msgr2=0x7fbb300c7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:06.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.601+0000 7fbb3499b700 1 --2- 172.21.15.31:0/3302625732 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbb3006aaa0 0x7fbb300c7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:06.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.601+0000 7fbb3499b700 1 -- 172.21.15.31:0/3302625732 >> v2:172.21.15.62:3300/0 conn(0x7fbb3013fef0 msgr2=0x7fbb300c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:06.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.601+0000 7fbb3499b700 1 --2- 172.21.15.31:0/3302625732 >> v2:172.21.15.62:3300/0 conn(0x7fbb3013fef0 0x7fbb300c7990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:06.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.601+0000 7fbb3499b700 1 -- 172.21.15.31:0/3302625732 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb3006b240 con 0x7fbb3013fb20 2023-10-06T02:23:06.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.601+0000 7fbb3499b700 1 --2- 172.21.15.31:0/3302625732 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb3013fb20 0x7fbb3015a930 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbb3006a5e0 tx=0x7fbb24009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f639175de379a9df server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:06.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.602+0000 7fbb2f7fe700 1 -- 172.21.15.31:0/3302625732 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2400b860 con 0x7fbb3013fb20 2023-10-06T02:23:06.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.602+0000 7fbb2f7fe700 1 -- 172.21.15.31:0/3302625732 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb2400b9c0 con 0x7fbb3013fb20 2023-10-06T02:23:06.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.602+0000 7fbb2f7fe700 1 -- 172.21.15.31:0/3302625732 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2400bd60 con 0x7fbb3013fb20 2023-10-06T02:23:06.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.602+0000 7fbb3599d700 1 -- 172.21.15.31:0/3302625732 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb3013fb20 msgr2=0x7fbb3015a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:06.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.602+0000 7fbb3599d700 1 --2- 172.21.15.31:0/3302625732 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb3013fb20 0x7fbb3015a930 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbb3006a5e0 tx=0x7fbb24009250 comp rx=0 tx=0).stop 2023-10-06T02:23:06.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.602+0000 7fbb3599d700 1 -- 172.21.15.31:0/3302625732 shutdown_connections 2023-10-06T02:23:06.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.602+0000 7fbb3599d700 1 --2- 172.21.15.31:0/3302625732 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbb3006aaa0 0x7fbb300c7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:06.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.602+0000 7fbb3599d700 1 --2- 172.21.15.31:0/3302625732 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb3013fb20 0x7fbb3015a930 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:06.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.602+0000 7fbb3599d700 1 --2- 172.21.15.31:0/3302625732 >> v2:172.21.15.62:3300/0 conn(0x7fbb3013fef0 0x7fbb300c7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:06.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.602+0000 7fbb3599d700 1 -- 172.21.15.31:0/3302625732 >> 172.21.15.31:0/3302625732 conn(0x7fbb3006d170 msgr2=0x7fbb300cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:06.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.602+0000 7fbb3599d700 1 -- 172.21.15.31:0/3302625732 shutdown_connections 2023-10-06T02:23:06.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.602+0000 7fbb3599d700 1 -- 172.21.15.31:0/3302625732 wait complete. 2023-10-06T02:23:06.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.604+0000 7fbb3599d700 1 Processor -- start 2023-10-06T02:23:06.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.604+0000 7fbb3599d700 1 -- start start 2023-10-06T02:23:06.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.605+0000 7fbb3599d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb3006aaa0 0x7fbb30155d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:06.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.605+0000 7fbb3599d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb3013fb20 0x7fbb300ab740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:06.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.605+0000 7fbb3599d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbb3013fef0 0x7fbb300cbd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:06.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.605+0000 7fbb3599d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb30157310 con 0x7fbb3006aaa0 2023-10-06T02:23:06.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.605+0000 7fbb3599d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb301562a0 con 0x7fbb3013fb20 2023-10-06T02:23:06.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.605+0000 7fbb3599d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb30158110 con 0x7fbb3013fef0 2023-10-06T02:23:06.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.605+0000 7fbb3499b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb3006aaa0 0x7fbb30155d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:06.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.605+0000 7fbb3499b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb3006aaa0 0x7fbb30155d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56946/0 (socket says 172.21.15.31:56946) 2023-10-06T02:23:06.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.605+0000 7fbb3499b700 1 -- 172.21.15.31:0/60623913 learned_addr learned my addr 172.21.15.31:0/60623913 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:06.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.605+0000 7fbb2ffff700 1 --2- 172.21.15.31:0/60623913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb3013fb20 0x7fbb300ab740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:06.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.605+0000 7fbb3519c700 1 --2- 172.21.15.31:0/60623913 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbb3013fef0 0x7fbb300cbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:06.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.605+0000 7fbb3499b700 1 -- 172.21.15.31:0/60623913 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbb3013fef0 msgr2=0x7fbb300cbd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:06.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.605+0000 7fbb3499b700 1 --2- 172.21.15.31:0/60623913 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbb3013fef0 0x7fbb300cbd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:06.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.606+0000 7fbb3499b700 1 -- 172.21.15.31:0/60623913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb3013fb20 msgr2=0x7fbb300ab740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:06.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.606+0000 7fbb3499b700 1 --2- 172.21.15.31:0/60623913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb3013fb20 0x7fbb300ab740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:06.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.606+0000 7fbb3499b700 1 -- 172.21.15.31:0/60623913 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb240094f0 con 0x7fbb3006aaa0 2023-10-06T02:23:06.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.606+0000 7fbb2ffff700 1 --2- 172.21.15.31:0/60623913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb3013fb20 0x7fbb300ab740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:23:06.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.606+0000 7fbb3519c700 1 --2- 172.21.15.31:0/60623913 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbb3013fef0 0x7fbb300cbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:23:06.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.606+0000 7fbb3499b700 1 --2- 172.21.15.31:0/60623913 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb3006aaa0 0x7fbb30155d60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbb24006fd0 tx=0x7fbb24017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:06.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.606+0000 7fbb2cff9700 1 -- 172.21.15.31:0/60623913 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb24024440 con 0x7fbb3006aaa0 2023-10-06T02:23:06.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.606+0000 7fbb2cff9700 1 -- 172.21.15.31:0/60623913 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb240245a0 con 0x7fbb3006aaa0 2023-10-06T02:23:06.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.606+0000 7fbb2cff9700 1 -- 172.21.15.31:0/60623913 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb24024960 con 0x7fbb3006aaa0 2023-10-06T02:23:06.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.606+0000 7fbb3599d700 1 -- 172.21.15.31:0/60623913 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb30157500 con 0x7fbb3006aaa0 2023-10-06T02:23:06.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.606+0000 7fbb3599d700 1 -- 172.21.15.31:0/60623913 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb30157ac0 con 0x7fbb3006aaa0 2023-10-06T02:23:06.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.607+0000 7fbb3599d700 1 -- 172.21.15.31:0/60623913 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb3004c3c0 con 0x7fbb3006aaa0 2023-10-06T02:23:06.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.608+0000 7fbb2cff9700 1 -- 172.21.15.31:0/60623913 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbb2401a420 con 0x7fbb3006aaa0 2023-10-06T02:23:06.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.609+0000 7fbb2cff9700 1 --2- 172.21.15.31:0/60623913 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fbb14066f30 0x7fbb140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:06.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.609+0000 7fbb2cff9700 1 -- 172.21.15.31:0/60623913 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fbb24086200 con 0x7fbb3006aaa0 2023-10-06T02:23:06.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.609+0000 7fbb2ffff700 1 --2- 172.21.15.31:0/60623913 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fbb14066f30 0x7fbb140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:06.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.610+0000 7fbb2ffff700 1 --2- 172.21.15.31:0/60623913 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fbb14066f30 0x7fbb140693e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c006aa0 tx=0x7fbb1c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:06.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.613+0000 7fbb2cff9700 1 -- 172.21.15.31:0/60623913 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbb240535c0 con 0x7fbb3006aaa0 2023-10-06T02:23:06.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.818+0000 7fbb3599d700 1 -- 172.21.15.31:0/60623913 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fbb300ca060 con 0x7fbb3006aaa0 2023-10-06T02:23:06.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.818+0000 7fbb2cff9700 1 -- 172.21.15.31:0/60623913 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v19) v1 ==== 84+0+126 (secure 0 0 0) 0x7fbb24056be0 con 0x7fbb3006aaa0 2023-10-06T02:23:06.819 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:23:06.819 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2023-10-06T02:23:06.820 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:23:06.820 INFO:teuthology.orchestra.run.smithi031.stdout: "poolnum": 1, 2023-10-06T02:23:06.820 INFO:teuthology.orchestra.run.smithi031.stdout: "poolname": ".mgr" 2023-10-06T02:23:06.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:23:06.820 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:23:06.820 INFO:teuthology.orchestra.run.smithi031.stdout: "poolnum": 2, 2023-10-06T02:23:06.820 INFO:teuthology.orchestra.run.smithi031.stdout: "poolname": "rbd" 2023-10-06T02:23:06.820 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:23:06.820 INFO:teuthology.orchestra.run.smithi031.stdout:] 2023-10-06T02:23:06.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.820+0000 7fbb3599d700 1 -- 172.21.15.31:0/60623913 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fbb14066f30 msgr2=0x7fbb140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:06.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.820+0000 7fbb3599d700 1 --2- 172.21.15.31:0/60623913 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fbb14066f30 0x7fbb140693e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c006aa0 tx=0x7fbb1c008040 comp rx=0 tx=0).stop 2023-10-06T02:23:06.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.820+0000 7fbb3599d700 1 -- 172.21.15.31:0/60623913 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb3006aaa0 msgr2=0x7fbb30155d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:06.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.820+0000 7fbb3599d700 1 --2- 172.21.15.31:0/60623913 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb3006aaa0 0x7fbb30155d60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbb24006fd0 tx=0x7fbb24017000 comp rx=0 tx=0).stop 2023-10-06T02:23:06.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.821+0000 7fbb3599d700 1 -- 172.21.15.31:0/60623913 shutdown_connections 2023-10-06T02:23:06.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.821+0000 7fbb3599d700 1 --2- 172.21.15.31:0/60623913 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fbb14066f30 0x7fbb140693e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:06.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.821+0000 7fbb3599d700 1 --2- 172.21.15.31:0/60623913 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbb3013fef0 0x7fbb300cbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:06.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.821+0000 7fbb3599d700 1 --2- 172.21.15.31:0/60623913 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb3006aaa0 0x7fbb30155d60 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:06.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.821+0000 7fbb3599d700 1 --2- 172.21.15.31:0/60623913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb3013fb20 0x7fbb300ab740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:06.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.821+0000 7fbb3599d700 1 -- 172.21.15.31:0/60623913 >> 172.21.15.31:0/60623913 conn(0x7fbb3006d170 msgr2=0x7fbb300ca580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:06.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.821+0000 7fbb3599d700 1 -- 172.21.15.31:0/60623913 shutdown_connections 2023-10-06T02:23:06.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:06.821+0000 7fbb3599d700 1 -- 172.21.15.31:0/60623913 wait complete. 2023-10-06T02:23:06.839 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T02:23:06.839 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T02:23:07.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:07.031+0000 7f0c6a59d700 1 Processor -- start 2023-10-06T02:23:07.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:07.031+0000 7f0c6a59d700 1 -- start start 2023-10-06T02:23:07.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:07.032+0000 7f0c6a59d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0c64157380 0x7f0c641585f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:07.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:07.032+0000 7f0c6a59d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0c64157750 0x7f0c640ae080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:07.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:07.032+0000 7f0c6a59d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c640c8680 con 0x7f0c6415b1a0 2023-10-06T02:23:07.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:07.032+0000 7f0c6a59d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c6406a7f0 con 0x7f0c64157380 2023-10-06T02:23:07.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:07.032+0000 7f0c6a59d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c640c7270 con 0x7f0c64157750 2023-10-06T02:23:07.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:07.032+0000 7f0c68d9a700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f0c6415b1a0 0x7f0c640c6b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:39276/0 (socket says 172.21.15.31:39276) 2023-10-06T02:23:07.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:07.032+0000 7f0c68d9a700 1 -- 172.21.15.31:0/761703709 learned_addr learned my addr 172.21.15.31:0/761703709 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:23:07.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:07.033+0000 7f0c6959b700 1 --2- 172.21.15.31:0/761703709 >> v2:172.21.15.62:3300/0 conn(0x7f0c64157380 0x7f0c641585f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:07.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:07.033+0000 7f0c69d9c700 1 --2- 172.21.15.31:0/761703709 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0c64157750 0x7f0c640ae080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:07.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:07.033+0000 7f0c5bfff700 1 -- 172.21.15.31:0/761703709 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 472535944 0 0) 0x7f0c640c8680 con 0x7f0c6415b1a0 2023-10-06T02:23:07.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:07.033+0000 7f0c5bfff700 1 -- 172.21.15.31:0/761703709 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c50003570 con 0x7f0c6415b1a0 2023-10-06T02:23:07.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:07.033+0000 7f0c69d9c700 1 -- 172.21.15.31:0/761703709 >> v2:172.21.15.62:3300/0 conn(0x7f0c64157380 msgr2=0x7f0c641585f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:07.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:07.033+0000 7f0c69d9c700 1 --2- 172.21.15.31:0/761703709 >> v2:172.21.15.62:3300/0 conn(0x7f0c64157380 0x7f0c641585f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:07.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:07.033+0000 7f0c69d9c700 1 -- 172.21.15.31:0/761703709 >> v1:172.21.15.31:6789/0 conn(0x7f0c6415b1a0 legacy=0x7f0c640c6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:07.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:07.033+0000 7f0c69d9c700 1 -- 172.21.15.31:0/761703709 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c64158b30 con 0x7f0c64157750 2023-10-06T02:23:07.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:07.033+0000 7f0c6959b700 1 --2- 172.21.15.31:0/761703709 >> v2:172.21.15.62:3300/0 conn(0x7f0c64157380 0x7f0c641585f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:23:07.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:07.033+0000 7f0c69d9c700 1 --2- 172.21.15.31:0/761703709 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0c64157750 0x7f0c640ae080 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c009e60 tx=0x7f0c5c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5db33e4dceb35342 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:07.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:07.034+0000 7f0c5bfff700 1 -- 172.21.15.31:0/761703709 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c5c00e070 con 0x7f0c64157750 2023-10-06T02:23:07.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:07.034+0000 7f0c5bfff700 1 -- 172.21.15.31:0/761703709 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c5c00bab0 con 0x7f0c64157750 2023-10-06T02:23:07.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:07.034+0000 7f0c5bfff700 1 -- 172.21.15.31:0/761703709 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c5c00be70 con 0x7f0c64157750 2023-10-06T02:23:07.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:07.034+0000 7f0c6a59d700 1 -- 172.21.15.31:0/761703709 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0c64157750 msgr2=0x7f0c640ae080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:07.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:07.034+0000 7f0c6a59d700 1 --2- 172.21.15.31:0/761703709 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0c64157750 0x7f0c640ae080 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c009e60 tx=0x7f0c5c009780 comp rx=0 tx=0).stop 2023-10-06T02:23:07.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:07.034+0000 7f0c6a59d700 1 -- 172.21.15.31:0/761703709 shutdown_connections 2023-10-06T02:23:07.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:07.034+0000 7f0c6a59d700 1 --2- 172.21.15.31:0/761703709 >> v2:172.21.15.62:3300/0 conn(0x7f0c64157380 0x7f0c641585f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:07.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:07.034+0000 7f0c6a59d700 1 --2- 172.21.15.31:0/761703709 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0c64157750 0x7f0c640ae080 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:07.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:07.034+0000 7f0c6a59d700 1 -- 172.21.15.31:0/761703709 >> 172.21.15.31:0/761703709 conn(0x7f0c6406c6a0 msgr2=0x7f0c6406caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:07.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:07.034+0000 7f0c6a59d700 1 -- 172.21.15.31:0/761703709 shutdown_connections 2023-10-06T02:23:07.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:07.034+0000 7f0c6a59d700 1 -- 172.21.15.31:0/761703709 wait complete. 2023-10-06T02:23:07.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:07.036+0000 7f0c6a59d700 1 Processor -- start 2023-10-06T02:23:07.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:07.036+0000 7f0c6a59d700 1 -- start start 2023-10-06T02:23:07.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:07.037+0000 7f0c6a59d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c64157380 0x7f0c640c2480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:07.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:07.037+0000 7f0c6a59d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c64157750 0x7f0c641566e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:07.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:07.037+0000 7f0c6a59d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0c6415b1a0 0x7f0c6416a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:07.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:07.037+0000 7f0c6a59d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c6414a750 con 0x7f0c64157750 2023-10-06T02:23:07.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:07.037+0000 7f0c6a59d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c641518a0 con 0x7f0c64157380 2023-10-06T02:23:07.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:07.037+0000 7f0c6a59d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c6416a5e0 con 0x7f0c6415b1a0 2023-10-06T02:23:07.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:07.037+0000 7f0c68d9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c64157750 0x7f0c641566e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:07.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:07.037+0000 7f0c68d9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c64157750 0x7f0c641566e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56954/0 (socket says 172.21.15.31:56954) 2023-10-06T02:23:07.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:07.037+0000 7f0c68d9a700 1 -- 172.21.15.31:0/3961159803 learned_addr learned my addr 172.21.15.31:0/3961159803 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:07.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:07.037+0000 7f0c6959b700 1 --2- 172.21.15.31:0/3961159803 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c64157380 0x7f0c640c2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:07.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:07.037+0000 7f0c69d9c700 1 --2- 172.21.15.31:0/3961159803 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0c6415b1a0 0x7f0c6416a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:07.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:07.037+0000 7f0c68d9a700 1 -- 172.21.15.31:0/3961159803 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0c6415b1a0 msgr2=0x7f0c6416a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:07.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:07.037+0000 7f0c68d9a700 1 --2- 172.21.15.31:0/3961159803 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0c6415b1a0 0x7f0c6416a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:07.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:07.037+0000 7f0c68d9a700 1 -- 172.21.15.31:0/3961159803 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c64157380 msgr2=0x7f0c640c2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:07.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:07.037+0000 7f0c68d9a700 1 --2- 172.21.15.31:0/3961159803 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c64157380 0x7f0c640c2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:07.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:07.037+0000 7f0c68d9a700 1 -- 172.21.15.31:0/3961159803 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c60009420 con 0x7f0c64157750 2023-10-06T02:23:07.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:07.037+0000 7f0c6959b700 1 --2- 172.21.15.31:0/3961159803 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c64157380 0x7f0c640c2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:23:07.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:07.038+0000 7f0c68d9a700 1 --2- 172.21.15.31:0/3961159803 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c64157750 0x7f0c641566e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0c5400c7f0 tx=0x7f0c54007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:07.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:07.038+0000 7f0c597fa700 1 -- 172.21.15.31:0/3961159803 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c54005b30 con 0x7f0c64157750 2023-10-06T02:23:07.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:07.038+0000 7f0c597fa700 1 -- 172.21.15.31:0/3961159803 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c54005c90 con 0x7f0c64157750 2023-10-06T02:23:07.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:07.038+0000 7f0c597fa700 1 -- 172.21.15.31:0/3961159803 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c54007320 con 0x7f0c64157750 2023-10-06T02:23:07.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:07.038+0000 7f0c6a59d700 1 -- 172.21.15.31:0/3961159803 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c5c009a20 con 0x7f0c64157750 2023-10-06T02:23:07.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:07.038+0000 7f0c6a59d700 1 -- 172.21.15.31:0/3961159803 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c64151d30 con 0x7f0c64157750 2023-10-06T02:23:07.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:07.039+0000 7f0c6a59d700 1 -- 172.21.15.31:0/3961159803 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c64151a30 con 0x7f0c64157750 2023-10-06T02:23:07.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:07.040+0000 7f0c597fa700 1 -- 172.21.15.31:0/3961159803 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0c5401b400 con 0x7f0c64157750 2023-10-06T02:23:07.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:07.041+0000 7f0c597fa700 1 --2- 172.21.15.31:0/3961159803 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f0c48066f30 0x7f0c480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:07.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:07.041+0000 7f0c597fa700 1 -- 172.21.15.31:0/3961159803 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f0c54083e50 con 0x7f0c64157750 2023-10-06T02:23:07.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:07.041+0000 7f0c6959b700 1 --2- 172.21.15.31:0/3961159803 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f0c48066f30 0x7f0c480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:07.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:07.042+0000 7f0c6959b700 1 --2- 172.21.15.31:0/3961159803 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f0c48066f30 0x7f0c480693e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0c600093f0 tx=0x7f0c60008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:07.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:07.045+0000 7f0c597fa700 1 -- 172.21.15.31:0/3961159803 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0c54051290 con 0x7f0c64157750 2023-10-06T02:23:07.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:07.250+0000 7f0c6a59d700 1 -- 172.21.15.31:0/3961159803 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f0c6404c3c0 con 0x7f0c64157750 2023-10-06T02:23:08.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:08.016+0000 7f0c597fa700 1 -- 172.21.15.31:0/3961159803 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v20) v1 ==== 138+0+0 (secure 0 0 0) 0x7f0c540548b0 con 0x7f0c64157750 2023-10-06T02:23:08.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:08.023+0000 7f0c6a59d700 1 -- 172.21.15.31:0/3961159803 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f0c48066f30 msgr2=0x7f0c480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:08.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:08.024+0000 7f0c6a59d700 1 --2- 172.21.15.31:0/3961159803 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f0c48066f30 0x7f0c480693e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0c600093f0 tx=0x7f0c60008040 comp rx=0 tx=0).stop 2023-10-06T02:23:08.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:08.024+0000 7f0c6a59d700 1 -- 172.21.15.31:0/3961159803 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c64157750 msgr2=0x7f0c641566e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:08.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:08.024+0000 7f0c6a59d700 1 --2- 172.21.15.31:0/3961159803 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c64157750 0x7f0c641566e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0c5400c7f0 tx=0x7f0c54007e00 comp rx=0 tx=0).stop 2023-10-06T02:23:08.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:08.024+0000 7f0c6a59d700 1 -- 172.21.15.31:0/3961159803 shutdown_connections 2023-10-06T02:23:08.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:08.024+0000 7f0c6a59d700 1 --2- 172.21.15.31:0/3961159803 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f0c48066f30 0x7f0c480693e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:08.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:08.024+0000 7f0c6a59d700 1 --2- 172.21.15.31:0/3961159803 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0c6415b1a0 0x7f0c6416a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:08.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:08.024+0000 7f0c6a59d700 1 --2- 172.21.15.31:0/3961159803 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c64157380 0x7f0c640c2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:08.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:08.024+0000 7f0c6a59d700 1 --2- 172.21.15.31:0/3961159803 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c64157750 0x7f0c641566e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:08.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:08.024+0000 7f0c6a59d700 1 -- 172.21.15.31:0/3961159803 >> 172.21.15.31:0/3961159803 conn(0x7f0c6406c6a0 msgr2=0x7f0c641532f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:08.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:08.024+0000 7f0c6a59d700 1 -- 172.21.15.31:0/3961159803 shutdown_connections 2023-10-06T02:23:08.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:08.024+0000 7f0c6a59d700 1 -- 172.21.15.31:0/3961159803 wait complete. 2023-10-06T02:23:08.027 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' created 2023-10-06T02:23:08.046 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T02:23:08.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:08.245+0000 7f4bf299d700 1 Processor -- start 2023-10-06T02:23:08.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:08.245+0000 7f4bf299d700 1 -- start start 2023-10-06T02:23:08.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:08.246+0000 7f4bf299d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4bec159380 0x7f4bec159750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:08.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:08.246+0000 7f4bf299d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4bec0cb3f0 0x7f4bec15dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:08.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:08.246+0000 7f4bf299d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bec0ccfa0 con 0x7f4bec0cea00 2023-10-06T02:23:08.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:08.246+0000 7f4bf299d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bec0cb880 con 0x7f4bec159380 2023-10-06T02:23:08.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:08.246+0000 7f4bf299d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bec0cbb60 con 0x7f4bec0cb3f0 2023-10-06T02:23:08.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:08.247+0000 7f4bf199b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4bec159380 0x7f4bec159750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:08.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:08.247+0000 7f4bf199b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4bec159380 0x7f4bec159750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44208/0 (socket says 172.21.15.31:44208) 2023-10-06T02:23:08.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:08.247+0000 7f4bf219c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f4bec0cea00 0x7f4bec0cedf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56088/0 (socket says 172.21.15.31:56088) 2023-10-06T02:23:08.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:08.247+0000 7f4bf199b700 1 -- 172.21.15.31:0/1092357379 learned_addr learned my addr 172.21.15.31:0/1092357379 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:08.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:08.247+0000 7f4bf119a700 1 --2- 172.21.15.31:0/1092357379 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4bec0cb3f0 0x7f4bec15dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:08.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:08.247+0000 7f4bf199b700 1 -- 172.21.15.31:0/1092357379 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4bec0cb3f0 msgr2=0x7f4bec15dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:08.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:08.247+0000 7f4bf199b700 1 --2- 172.21.15.31:0/1092357379 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4bec0cb3f0 0x7f4bec15dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:08.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:08.247+0000 7f4bf199b700 1 -- 172.21.15.31:0/1092357379 >> v1:172.21.15.31:6789/0 conn(0x7f4bec0cea00 legacy=0x7f4bec0cedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:08.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:08.247+0000 7f4bf199b700 1 -- 172.21.15.31:0/1092357379 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bec0cbe40 con 0x7f4bec159380 2023-10-06T02:23:08.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:08.247+0000 7f4bf0999700 1 -- 172.21.15.31:0/1092357379 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4058001561 0 0) 0x7f4bec0ccfa0 con 0x7f4bec0cea00 2023-10-06T02:23:08.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:08.247+0000 7f4bf199b700 1 --2- 172.21.15.31:0/1092357379 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4bec159380 0x7f4bec159750 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4be80099b0 tx=0x7f4be8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d89558bb07b798c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:08.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:08.248+0000 7f4bf0999700 1 -- 172.21.15.31:0/1092357379 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be8009ea0 con 0x7f4bec159380 2023-10-06T02:23:08.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:08.248+0000 7f4bf0999700 1 -- 172.21.15.31:0/1092357379 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4be800b860 con 0x7f4bec159380 2023-10-06T02:23:08.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:08.248+0000 7f4bf0999700 1 -- 172.21.15.31:0/1092357379 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be800bc00 con 0x7f4bec159380 2023-10-06T02:23:08.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:08.248+0000 7f4bf299d700 1 -- 172.21.15.31:0/1092357379 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4bec159380 msgr2=0x7f4bec159750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:08.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:08.248+0000 7f4bf299d700 1 --2- 172.21.15.31:0/1092357379 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4bec159380 0x7f4bec159750 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4be80099b0 tx=0x7f4be8009250 comp rx=0 tx=0).stop 2023-10-06T02:23:08.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:08.248+0000 7f4bf299d700 1 -- 172.21.15.31:0/1092357379 shutdown_connections 2023-10-06T02:23:08.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:08.248+0000 7f4bf299d700 1 --2- 172.21.15.31:0/1092357379 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4bec159380 0x7f4bec159750 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:08.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:08.248+0000 7f4bf299d700 1 --2- 172.21.15.31:0/1092357379 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4bec0cb3f0 0x7f4bec15dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:08.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:08.248+0000 7f4bf299d700 1 -- 172.21.15.31:0/1092357379 >> 172.21.15.31:0/1092357379 conn(0x7f4bec0703a0 msgr2=0x7f4bec0707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:08.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:08.248+0000 7f4bf299d700 1 -- 172.21.15.31:0/1092357379 shutdown_connections 2023-10-06T02:23:08.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:08.248+0000 7f4bf299d700 1 -- 172.21.15.31:0/1092357379 wait complete. 2023-10-06T02:23:08.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:08.250+0000 7f4bf299d700 1 Processor -- start 2023-10-06T02:23:08.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:08.250+0000 7f4bf299d700 1 -- start start 2023-10-06T02:23:08.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:08.251+0000 7f4bf299d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4bec0cb3f0 0x7f4bec1532f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:08.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:08.251+0000 7f4bf299d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4bec0cea00 0x7f4bec15c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:08.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:08.251+0000 7f4bf299d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4bec159380 0x7f4bec16d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:08.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:08.251+0000 7f4bf299d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bec0c8ef0 con 0x7f4bec0cea00 2023-10-06T02:23:08.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:08.251+0000 7f4bf299d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bec0c91d0 con 0x7f4bec0cb3f0 2023-10-06T02:23:08.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:08.251+0000 7f4bf299d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bec0c94b0 con 0x7f4bec159380 2023-10-06T02:23:08.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:08.251+0000 7f4bf119a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4bec0cea00 0x7f4bec15c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:08.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:08.251+0000 7f4bf119a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4bec0cea00 0x7f4bec15c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44220/0 (socket says 172.21.15.31:44220) 2023-10-06T02:23:08.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:08.251+0000 7f4bf119a700 1 -- 172.21.15.31:0/2243263693 learned_addr learned my addr 172.21.15.31:0/2243263693 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:08.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:08.251+0000 7f4bf219c700 1 --2- 172.21.15.31:0/2243263693 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4bec159380 0x7f4bec16d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:08.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:08.251+0000 7f4bf199b700 1 --2- 172.21.15.31:0/2243263693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4bec0cb3f0 0x7f4bec1532f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:08.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:08.251+0000 7f4bf119a700 1 -- 172.21.15.31:0/2243263693 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4bec159380 msgr2=0x7f4bec16d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:08.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:08.251+0000 7f4bf119a700 1 --2- 172.21.15.31:0/2243263693 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4bec159380 0x7f4bec16d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:08.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:08.251+0000 7f4bf119a700 1 -- 172.21.15.31:0/2243263693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4bec0cb3f0 msgr2=0x7f4bec1532f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:08.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:08.251+0000 7f4bf119a700 1 --2- 172.21.15.31:0/2243263693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4bec0cb3f0 0x7f4bec1532f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:08.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:08.251+0000 7f4bf119a700 1 -- 172.21.15.31:0/2243263693 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4be80094f0 con 0x7f4bec0cea00 2023-10-06T02:23:08.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:08.251+0000 7f4bf199b700 1 --2- 172.21.15.31:0/2243263693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4bec0cb3f0 0x7f4bec1532f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:23:08.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:08.252+0000 7f4bf119a700 1 --2- 172.21.15.31:0/2243263693 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4bec0cea00 0x7f4bec15c440 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc00ec80 tx=0x7f4bdc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:08.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:08.252+0000 7f4be1ffb700 1 -- 172.21.15.31:0/2243263693 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bdc00c850 con 0x7f4bec0cea00 2023-10-06T02:23:08.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:08.252+0000 7f4be1ffb700 1 -- 172.21.15.31:0/2243263693 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bdc00c9b0 con 0x7f4bec0cea00 2023-10-06T02:23:08.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:08.252+0000 7f4be1ffb700 1 -- 172.21.15.31:0/2243263693 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bdc00cd50 con 0x7f4bec0cea00 2023-10-06T02:23:08.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:08.252+0000 7f4bf299d700 1 -- 172.21.15.31:0/2243263693 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bec0c9790 con 0x7f4bec0cea00 2023-10-06T02:23:08.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:08.252+0000 7f4bf299d700 1 -- 172.21.15.31:0/2243263693 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bec0c9d50 con 0x7f4bec0cea00 2023-10-06T02:23:08.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:08.253+0000 7f4bf299d700 1 -- 172.21.15.31:0/2243263693 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bec0ca8d0 con 0x7f4bec0cea00 2023-10-06T02:23:08.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:08.254+0000 7f4be1ffb700 1 -- 172.21.15.31:0/2243263693 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4bdc013680 con 0x7f4bec0cea00 2023-10-06T02:23:08.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:08.255+0000 7f4be1ffb700 1 --2- 172.21.15.31:0/2243263693 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f4bd0066f30 0x7f4bd00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:08.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:08.255+0000 7f4be1ffb700 1 -- 172.21.15.31:0/2243263693 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5576+0+0 (secure 0 0 0) 0x7f4bdc0847f0 con 0x7f4bec0cea00 2023-10-06T02:23:08.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:08.255+0000 7f4bf199b700 1 --2- 172.21.15.31:0/2243263693 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f4bd0066f30 0x7f4bd00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:08.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:08.256+0000 7f4bf199b700 1 --2- 172.21.15.31:0/2243263693 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f4bd0066f30 0x7f4bd00693e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4be80099b0 tx=0x7f4be8017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:08.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:08.260+0000 7f4be1ffb700 1 -- 172.21.15.31:0/2243263693 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4bdc051a40 con 0x7f4bec0cea00 2023-10-06T02:23:08.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:08.464+0000 7f4bf299d700 1 -- 172.21.15.31:0/2243263693 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f4bec04c3c0 con 0x7f4bec0cea00 2023-10-06T02:23:09.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:09.018+0000 7f4be1ffb700 1 -- 172.21.15.31:0/2243263693 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v21) v1 ==== 170+0+0 (secure 0 0 0) 0x7f4bdc055060 con 0x7f4bec0cea00 2023-10-06T02:23:09.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:09.032+0000 7f4bf299d700 1 -- 172.21.15.31:0/2243263693 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f4bd0066f30 msgr2=0x7f4bd00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:09.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:09.032+0000 7f4bf299d700 1 --2- 172.21.15.31:0/2243263693 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f4bd0066f30 0x7f4bd00693e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4be80099b0 tx=0x7f4be8017000 comp rx=0 tx=0).stop 2023-10-06T02:23:09.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:09.032+0000 7f4bf299d700 1 -- 172.21.15.31:0/2243263693 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4bec0cea00 msgr2=0x7f4bec15c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:09.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:09.032+0000 7f4bf299d700 1 --2- 172.21.15.31:0/2243263693 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4bec0cea00 0x7f4bec15c440 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc00ec80 tx=0x7f4bdc0091a0 comp rx=0 tx=0).stop 2023-10-06T02:23:09.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:09.035+0000 7f4bf299d700 1 -- 172.21.15.31:0/2243263693 shutdown_connections 2023-10-06T02:23:09.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:09.035+0000 7f4bf299d700 1 --2- 172.21.15.31:0/2243263693 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f4bd0066f30 0x7f4bd00693e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:09.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:09.035+0000 7f4bf299d700 1 --2- 172.21.15.31:0/2243263693 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4bec159380 0x7f4bec16d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:09.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:09.035+0000 7f4bf299d700 1 --2- 172.21.15.31:0/2243263693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4bec0cb3f0 0x7f4bec1532f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:09.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:09.035+0000 7f4bf299d700 1 --2- 172.21.15.31:0/2243263693 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4bec0cea00 0x7f4bec15c440 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:09.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:09.035+0000 7f4bf299d700 1 -- 172.21.15.31:0/2243263693 >> 172.21.15.31:0/2243263693 conn(0x7f4bec0703a0 msgr2=0x7f4bec14acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:09.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:09.037+0000 7f4bf299d700 1 -- 172.21.15.31:0/2243263693 shutdown_connections 2023-10-06T02:23:09.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:09.037+0000 7f4bf299d700 1 -- 172.21.15.31:0/2243263693 wait complete. 2023-10-06T02:23:09.045 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' created 2023-10-06T02:23:09.065 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T02:23:09.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:09.256+0000 7fc9ebfff700 1 Processor -- start 2023-10-06T02:23:09.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:09.256+0000 7fc9ebfff700 1 -- start start 2023-10-06T02:23:09.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:09.257+0000 7fc9ebfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc9ec153a00 0x7fc9ec153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:09.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:09.257+0000 7fc9ebfff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc9ec15de50 0x7fc9ec15a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:09.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:09.257+0000 7fc9ebfff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9ec15af60 con 0x7fc9ec0cea00 2023-10-06T02:23:09.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:09.257+0000 7fc9ebfff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9ec15d010 con 0x7fc9ec153a00 2023-10-06T02:23:09.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:09.257+0000 7fc9ebfff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9ec15b240 con 0x7fc9ec15de50 2023-10-06T02:23:09.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:09.257+0000 7fc9eaffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc9ec153a00 0x7fc9ec153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:09.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:09.257+0000 7fc9eb7fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fc9ec0cea00 0x7fc9ec0cee80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56096/0 (socket says 172.21.15.31:56096) 2023-10-06T02:23:09.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:09.257+0000 7fc9eaffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc9ec153a00 0x7fc9ec153dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44222/0 (socket says 172.21.15.31:44222) 2023-10-06T02:23:09.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:09.257+0000 7fc9eb7fe700 1 -- 172.21.15.31:0/3103074756 learned_addr learned my addr 172.21.15.31:0/3103074756 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:23:09.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:09.258+0000 7fc9ea7fc700 1 --2- 172.21.15.31:0/3103074756 >> v2:172.21.15.62:3300/0 conn(0x7fc9ec15de50 0x7fc9ec15a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:09.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:09.258+0000 7fc9eaffd700 1 -- 172.21.15.31:0/3103074756 >> v2:172.21.15.62:3300/0 conn(0x7fc9ec15de50 msgr2=0x7fc9ec15a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:09.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:09.258+0000 7fc9eaffd700 1 --2- 172.21.15.31:0/3103074756 >> v2:172.21.15.62:3300/0 conn(0x7fc9ec15de50 0x7fc9ec15a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:09.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:09.258+0000 7fc9eaffd700 1 -- 172.21.15.31:0/3103074756 >> v1:172.21.15.31:6789/0 conn(0x7fc9ec0cea00 legacy=0x7fc9ec0cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:09.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:09.258+0000 7fc9eaffd700 1 -- 172.21.15.31:0/3103074756 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9ec15b520 con 0x7fc9ec153a00 2023-10-06T02:23:09.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:09.258+0000 7fc9e9ffb700 1 -- 172.21.15.31:0/3103074756 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2113133283 0 0) 0x7fc9ec15af60 con 0x7fc9ec0cea00 2023-10-06T02:23:09.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:09.258+0000 7fc9ea7fc700 1 --2- 172.21.15.31:0/3103074756 >> v2:172.21.15.62:3300/0 conn(0x7fc9ec15de50 0x7fc9ec15a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:23:09.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:09.258+0000 7fc9eaffd700 1 --2- 172.21.15.31:0/3103074756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9ec153a00 0x7fc9ec153dd0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec0b3070 tx=0x7fc9dc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1fad74b4eb1a8ac server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:09.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:09.258+0000 7fc9e9ffb700 1 -- 172.21.15.31:0/3103074756 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc00b6b0 con 0x7fc9ec153a00 2023-10-06T02:23:09.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:09.258+0000 7fc9e9ffb700 1 -- 172.21.15.31:0/3103074756 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9dc00b810 con 0x7fc9ec153a00 2023-10-06T02:23:09.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:09.258+0000 7fc9e9ffb700 1 -- 172.21.15.31:0/3103074756 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc00bbb0 con 0x7fc9ec153a00 2023-10-06T02:23:09.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:09.259+0000 7fc9ebfff700 1 -- 172.21.15.31:0/3103074756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9ec153a00 msgr2=0x7fc9ec153dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:09.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:09.259+0000 7fc9ebfff700 1 --2- 172.21.15.31:0/3103074756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9ec153a00 0x7fc9ec153dd0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec0b3070 tx=0x7fc9dc0091a0 comp rx=0 tx=0).stop 2023-10-06T02:23:09.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:09.259+0000 7fc9ebfff700 1 -- 172.21.15.31:0/3103074756 shutdown_connections 2023-10-06T02:23:09.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:09.259+0000 7fc9ebfff700 1 --2- 172.21.15.31:0/3103074756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9ec153a00 0x7fc9ec153dd0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:09.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:09.259+0000 7fc9ebfff700 1 --2- 172.21.15.31:0/3103074756 >> v2:172.21.15.62:3300/0 conn(0x7fc9ec15de50 0x7fc9ec15a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:09.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:09.259+0000 7fc9ebfff700 1 -- 172.21.15.31:0/3103074756 >> 172.21.15.31:0/3103074756 conn(0x7fc9ec0703a0 msgr2=0x7fc9ec0707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:09.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:09.259+0000 7fc9ebfff700 1 -- 172.21.15.31:0/3103074756 shutdown_connections 2023-10-06T02:23:09.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:09.259+0000 7fc9ebfff700 1 -- 172.21.15.31:0/3103074756 wait complete. 2023-10-06T02:23:09.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:09.260+0000 7fc9ebfff700 1 Processor -- start 2023-10-06T02:23:09.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:09.261+0000 7fc9ebfff700 1 -- start start 2023-10-06T02:23:09.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:09.262+0000 7fc9ebfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9ec0cea00 0x7fc9ec0c67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:09.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:09.262+0000 7fc9ebfff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc9ec153a00 0x7fc9ec150870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:09.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:09.262+0000 7fc9ebfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9ec15de50 0x7fc9ec14e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:09.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:09.262+0000 7fc9ebfff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ec0cb4d0 con 0x7fc9ec0cea00 2023-10-06T02:23:09.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:09.262+0000 7fc9ebfff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ec16db10 con 0x7fc9ec15de50 2023-10-06T02:23:09.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:09.262+0000 7fc9ebfff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ec150f50 con 0x7fc9ec153a00 2023-10-06T02:23:09.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:09.262+0000 7fc9eaffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9ec0cea00 0x7fc9ec0c67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:09.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:09.262+0000 7fc9eaffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9ec0cea00 0x7fc9ec0c67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44234/0 (socket says 172.21.15.31:44234) 2023-10-06T02:23:09.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:09.262+0000 7fc9eaffd700 1 -- 172.21.15.31:0/809862178 learned_addr learned my addr 172.21.15.31:0/809862178 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:09.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:09.262+0000 7fc9eb7fe700 1 --2- 172.21.15.31:0/809862178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9ec15de50 0x7fc9ec14e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:09.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:09.262+0000 7fc9ea7fc700 1 --2- 172.21.15.31:0/809862178 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc9ec153a00 0x7fc9ec150870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:09.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:09.262+0000 7fc9eaffd700 1 -- 172.21.15.31:0/809862178 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc9ec153a00 msgr2=0x7fc9ec150870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:09.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:09.262+0000 7fc9eaffd700 1 --2- 172.21.15.31:0/809862178 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc9ec153a00 0x7fc9ec150870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:09.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:09.262+0000 7fc9eaffd700 1 -- 172.21.15.31:0/809862178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9ec15de50 msgr2=0x7fc9ec14e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:09.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:09.262+0000 7fc9eaffd700 1 --2- 172.21.15.31:0/809862178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9ec15de50 0x7fc9ec14e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:09.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:09.262+0000 7fc9eaffd700 1 -- 172.21.15.31:0/809862178 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9dc009420 con 0x7fc9ec0cea00 2023-10-06T02:23:09.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:09.263+0000 7fc9eaffd700 1 --2- 172.21.15.31:0/809862178 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9ec0cea00 0x7fc9ec0c67f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc00b0e0 tx=0x7fc9dc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:09.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:09.263+0000 7fc9d2ffd700 1 -- 172.21.15.31:0/809862178 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc000d10 con 0x7fc9ec0cea00 2023-10-06T02:23:09.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:09.263+0000 7fc9d2ffd700 1 -- 172.21.15.31:0/809862178 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9dc00bb90 con 0x7fc9ec0cea00 2023-10-06T02:23:09.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:09.263+0000 7fc9ebfff700 1 -- 172.21.15.31:0/809862178 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9ec16d1e0 con 0x7fc9ec0cea00 2023-10-06T02:23:09.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:09.263+0000 7fc9d2ffd700 1 -- 172.21.15.31:0/809862178 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc026680 con 0x7fc9ec0cea00 2023-10-06T02:23:09.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:09.263+0000 7fc9ebfff700 1 -- 172.21.15.31:0/809862178 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9ec16d7a0 con 0x7fc9ec0cea00 2023-10-06T02:23:09.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:09.264+0000 7fc9ebfff700 1 -- 172.21.15.31:0/809862178 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9ec16ddf0 con 0x7fc9ec0cea00 2023-10-06T02:23:09.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:09.265+0000 7fc9d2ffd700 1 -- 172.21.15.31:0/809862178 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc9dc026a70 con 0x7fc9ec0cea00 2023-10-06T02:23:09.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:09.266+0000 7fc9d2ffd700 1 --2- 172.21.15.31:0/809862178 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc9d8066f30 0x7fc9d80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:09.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:09.266+0000 7fc9d2ffd700 1 -- 172.21.15.31:0/809862178 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5958+0+0 (secure 0 0 0) 0x7fc9dc086060 con 0x7fc9ec0cea00 2023-10-06T02:23:09.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:09.266+0000 7fc9ea7fc700 1 --2- 172.21.15.31:0/809862178 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc9d8066f30 0x7fc9d80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:09.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:09.267+0000 7fc9ea7fc700 1 --2- 172.21.15.31:0/809862178 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc9d8066f30 0x7fc9d80693e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec1604e0 tx=0x7fc9e0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:09.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:09.271+0000 7fc9d2ffd700 1 -- 172.21.15.31:0/809862178 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc9dc0b75a0 con 0x7fc9ec0cea00 2023-10-06T02:23:09.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:09.519+0000 7fc9ebfff700 1 -- 172.21.15.31:0/809862178 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fc9ec16e000 con 0x7fc9ec0cea00 2023-10-06T02:23:10.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.034+0000 7fc9d2ffd700 1 -- 172.21.15.31:0/809862178 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7fc9dc053130 con 0x7fc9ec0cea00 2023-10-06T02:23:10.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.039+0000 7fc9ebfff700 1 -- 172.21.15.31:0/809862178 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc9d8066f30 msgr2=0x7fc9d80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:10.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.039+0000 7fc9ebfff700 1 --2- 172.21.15.31:0/809862178 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc9d8066f30 0x7fc9d80693e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec1604e0 tx=0x7fc9e0008040 comp rx=0 tx=0).stop 2023-10-06T02:23:10.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.039+0000 7fc9ebfff700 1 -- 172.21.15.31:0/809862178 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9ec0cea00 msgr2=0x7fc9ec0c67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:10.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.039+0000 7fc9ebfff700 1 --2- 172.21.15.31:0/809862178 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9ec0cea00 0x7fc9ec0c67f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc00b0e0 tx=0x7fc9dc017000 comp rx=0 tx=0).stop 2023-10-06T02:23:10.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.039+0000 7fc9ebfff700 1 -- 172.21.15.31:0/809862178 shutdown_connections 2023-10-06T02:23:10.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.039+0000 7fc9ebfff700 1 --2- 172.21.15.31:0/809862178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9ec15de50 0x7fc9ec14e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:10.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.039+0000 7fc9ebfff700 1 --2- 172.21.15.31:0/809862178 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9ec0cea00 0x7fc9ec0c67f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:10.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.039+0000 7fc9ebfff700 1 --2- 172.21.15.31:0/809862178 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc9d8066f30 0x7fc9d80693e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:10.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.039+0000 7fc9ebfff700 1 --2- 172.21.15.31:0/809862178 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc9ec153a00 0x7fc9ec150870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:10.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.039+0000 7fc9ebfff700 1 -- 172.21.15.31:0/809862178 >> 172.21.15.31:0/809862178 conn(0x7fc9ec0703a0 msgr2=0x7fc9ec15bf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:10.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.039+0000 7fc9ebfff700 1 -- 172.21.15.31:0/809862178 shutdown_connections 2023-10-06T02:23:10.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.039+0000 7fc9ebfff700 1 -- 172.21.15.31:0/809862178 wait complete. 2023-10-06T02:23:10.042 INFO:teuthology.orchestra.run.smithi031.stderr:new fs with metadata pool 3 and data pool 4 2023-10-06T02:23:10.061 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:23:10.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.247+0000 7f7e8ffff700 1 Processor -- start 2023-10-06T02:23:10.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.247+0000 7f7e8ffff700 1 -- start start 2023-10-06T02:23:10.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.248+0000 7f7e8ffff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7e90153c30 0x7f7e9015a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:10.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.248+0000 7f7e8ffff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7e900cf110 0x7f7e9015c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:10.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.248+0000 7f7e8ffff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e9015afd0 con 0x7f7e9015de50 2023-10-06T02:23:10.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.248+0000 7f7e8ffff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e9015b2b0 con 0x7f7e90153c30 2023-10-06T02:23:10.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.248+0000 7f7e8ffff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9015b590 con 0x7f7e900cf110 2023-10-06T02:23:10.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.249+0000 7f7e8effd700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7e90153c30 0x7f7e9015a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:10.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.249+0000 7f7e8f7fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7e900cf110 0x7f7e9015c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:10.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.249+0000 7f7e8effd700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7e90153c30 0x7f7e9015a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:46810/0 (socket says 172.21.15.31:46810) 2023-10-06T02:23:10.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.249+0000 7f7e8f7fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7e900cf110 0x7f7e9015c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:33166/0 (socket says 172.21.15.31:33166) 2023-10-06T02:23:10.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.249+0000 7f7e8effd700 1 -- 172.21.15.31:0/177419884 learned_addr learned my addr 172.21.15.31:0/177419884 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:10.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.249+0000 7f7e8e7fc700 1 --1- 172.21.15.31:0/177419884 >> v1:172.21.15.62:6789/0 conn(0x7f7e9015de50 0x7f7e900cea00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:56818/0 (socket says 172.21.15.31:56818) 2023-10-06T02:23:10.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.249+0000 7f7e8effd700 1 -- 172.21.15.31:0/177419884 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7e900cf110 msgr2=0x7f7e9015c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:10.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.249+0000 7f7e8effd700 1 --2- 172.21.15.31:0/177419884 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7e900cf110 0x7f7e9015c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:10.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.249+0000 7f7e8effd700 1 -- 172.21.15.31:0/177419884 >> v1:172.21.15.62:6789/0 conn(0x7f7e9015de50 legacy=0x7f7e900cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:10.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.249+0000 7f7e8effd700 1 -- 172.21.15.31:0/177419884 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e9015b870 con 0x7f7e90153c30 2023-10-06T02:23:10.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.249+0000 7f7e8f7fe700 1 --2- 172.21.15.31:0/177419884 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7e900cf110 0x7f7e9015c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:23:10.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.250+0000 7f7e8effd700 1 --2- 172.21.15.31:0/177419884 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e90153c30 0x7f7e9015a5e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7e800099b0 tx=0x7f7e80009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8bb278a235541faa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:10.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.250+0000 7f7e8dffb700 1 -- 172.21.15.31:0/177419884 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e80009ea0 con 0x7f7e90153c30 2023-10-06T02:23:10.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.250+0000 7f7e8dffb700 1 -- 172.21.15.31:0/177419884 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e8000b860 con 0x7f7e90153c30 2023-10-06T02:23:10.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.250+0000 7f7e8dffb700 1 -- 172.21.15.31:0/177419884 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e8000bc00 con 0x7f7e90153c30 2023-10-06T02:23:10.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.250+0000 7f7e8ffff700 1 -- 172.21.15.31:0/177419884 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e90153c30 msgr2=0x7f7e9015a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:10.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.250+0000 7f7e8ffff700 1 --2- 172.21.15.31:0/177419884 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e90153c30 0x7f7e9015a5e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7e800099b0 tx=0x7f7e80009250 comp rx=0 tx=0).stop 2023-10-06T02:23:10.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.250+0000 7f7e8ffff700 1 -- 172.21.15.31:0/177419884 shutdown_connections 2023-10-06T02:23:10.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.250+0000 7f7e8ffff700 1 --2- 172.21.15.31:0/177419884 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7e900cf110 0x7f7e9015c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:10.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.250+0000 7f7e8ffff700 1 --2- 172.21.15.31:0/177419884 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e90153c30 0x7f7e9015a5e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:10.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.250+0000 7f7e8ffff700 1 -- 172.21.15.31:0/177419884 >> 172.21.15.31:0/177419884 conn(0x7f7e900703a0 msgr2=0x7f7e900707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:10.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.251+0000 7f7e8ffff700 1 -- 172.21.15.31:0/177419884 shutdown_connections 2023-10-06T02:23:10.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.251+0000 7f7e8ffff700 1 -- 172.21.15.31:0/177419884 wait complete. 2023-10-06T02:23:10.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.252+0000 7f7e8ffff700 1 Processor -- start 2023-10-06T02:23:10.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.252+0000 7f7e8ffff700 1 -- start start 2023-10-06T02:23:10.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.253+0000 7f7e8ffff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7e900cf110 0x7f7e900b2340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:10.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.253+0000 7f7e8ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e90153c30 0x7f7e90149f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:10.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.253+0000 7f7e8ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e9015de50 0x7f7e9016cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:10.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.253+0000 7f7e8ffff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9016da00 con 0x7f7e90153c30 2023-10-06T02:23:10.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.253+0000 7f7e8ffff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9016dce0 con 0x7f7e9015de50 2023-10-06T02:23:10.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.253+0000 7f7e8ffff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e900b1860 con 0x7f7e900cf110 2023-10-06T02:23:10.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.253+0000 7f7e8e7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e90153c30 0x7f7e90149f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:10.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.253+0000 7f7e8e7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e90153c30 0x7f7e90149f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44242/0 (socket says 172.21.15.31:44242) 2023-10-06T02:23:10.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.253+0000 7f7e8e7fc700 1 -- 172.21.15.31:0/4140134344 learned_addr learned my addr 172.21.15.31:0/4140134344 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:10.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.254+0000 7f7e8effd700 1 --2- 172.21.15.31:0/4140134344 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7e900cf110 0x7f7e900b2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:10.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.254+0000 7f7e8f7fe700 1 --2- 172.21.15.31:0/4140134344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e9015de50 0x7f7e9016cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:10.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.254+0000 7f7e8e7fc700 1 -- 172.21.15.31:0/4140134344 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7e900cf110 msgr2=0x7f7e900b2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:10.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.254+0000 7f7e8e7fc700 1 --2- 172.21.15.31:0/4140134344 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7e900cf110 0x7f7e900b2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:10.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.254+0000 7f7e8e7fc700 1 -- 172.21.15.31:0/4140134344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e9015de50 msgr2=0x7f7e9016cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:10.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.254+0000 7f7e8e7fc700 1 --2- 172.21.15.31:0/4140134344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e9015de50 0x7f7e9016cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:10.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.254+0000 7f7e8e7fc700 1 -- 172.21.15.31:0/4140134344 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e88009a20 con 0x7f7e90153c30 2023-10-06T02:23:10.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.254+0000 7f7e8effd700 1 --2- 172.21.15.31:0/4140134344 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7e900cf110 0x7f7e900b2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:23:10.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.254+0000 7f7e8f7fe700 1 --2- 172.21.15.31:0/4140134344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e9015de50 0x7f7e9016cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:23:10.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.254+0000 7f7e8e7fc700 1 --2- 172.21.15.31:0/4140134344 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e90153c30 0x7f7e90149f80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7e8400c560 tx=0x7f7e84007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:10.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.254+0000 7f7e76ffd700 1 -- 172.21.15.31:0/4140134344 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e84005a70 con 0x7f7e90153c30 2023-10-06T02:23:10.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.254+0000 7f7e76ffd700 1 -- 172.21.15.31:0/4140134344 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e84005bd0 con 0x7f7e90153c30 2023-10-06T02:23:10.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.254+0000 7f7e76ffd700 1 -- 172.21.15.31:0/4140134344 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e840070d0 con 0x7f7e90153c30 2023-10-06T02:23:10.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.254+0000 7f7e8ffff700 1 -- 172.21.15.31:0/4140134344 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e800094f0 con 0x7f7e90153c30 2023-10-06T02:23:10.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.254+0000 7f7e8ffff700 1 -- 172.21.15.31:0/4140134344 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e900b1ea0 con 0x7f7e90153c30 2023-10-06T02:23:10.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.255+0000 7f7e8ffff700 1 -- 172.21.15.31:0/4140134344 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e900b2060 con 0x7f7e90153c30 2023-10-06T02:23:10.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.256+0000 7f7e76ffd700 1 -- 172.21.15.31:0/4140134344 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7e84007480 con 0x7f7e90153c30 2023-10-06T02:23:10.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.258+0000 7f7e76ffd700 1 --2- 172.21.15.31:0/4140134344 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f7e7c066f30 0x7f7e7c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:10.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.258+0000 7f7e76ffd700 1 -- 172.21.15.31:0/4140134344 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f7e84083c40 con 0x7f7e90153c30 2023-10-06T02:23:10.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.258+0000 7f7e8effd700 1 --2- 172.21.15.31:0/4140134344 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f7e7c066f30 0x7f7e7c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:10.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.259+0000 7f7e8effd700 1 --2- 172.21.15.31:0/4140134344 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f7e7c066f30 0x7f7e7c0693e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7e800099b0 tx=0x7f7e80017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:10.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.263+0000 7f7e76ffd700 1 -- 172.21.15.31:0/4140134344 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7e900b2060 con 0x7f7e90153c30 2023-10-06T02:23:10.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.467+0000 7f7e8ffff700 1 -- 172.21.15.31:0/4140134344 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7e900b2060 con 0x7f7e90153c30 2023-10-06T02:23:10.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.468+0000 7f7e76ffd700 1 -- 172.21.15.31:0/4140134344 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16831 (secure 0 0 0) 0x7f7e900b2060 con 0x7f7e90153c30 2023-10-06T02:23:10.469 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:23:10.469 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":22,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:23:09.520882+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:23:07.290269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T02:23:08.504679+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:23:10.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.470+0000 7f7e8ffff700 1 -- 172.21.15.31:0/4140134344 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f7e7c066f30 msgr2=0x7f7e7c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:10.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.470+0000 7f7e8ffff700 1 --2- 172.21.15.31:0/4140134344 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f7e7c066f30 0x7f7e7c0693e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7e800099b0 tx=0x7f7e80017000 comp rx=0 tx=0).stop 2023-10-06T02:23:10.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.470+0000 7f7e8ffff700 1 -- 172.21.15.31:0/4140134344 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e90153c30 msgr2=0x7f7e90149f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:10.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.470+0000 7f7e8ffff700 1 --2- 172.21.15.31:0/4140134344 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e90153c30 0x7f7e90149f80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7e8400c560 tx=0x7f7e84007f70 comp rx=0 tx=0).stop 2023-10-06T02:23:10.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.470+0000 7f7e8ffff700 1 -- 172.21.15.31:0/4140134344 shutdown_connections 2023-10-06T02:23:10.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.470+0000 7f7e8ffff700 1 --2- 172.21.15.31:0/4140134344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e9015de50 0x7f7e9016cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:10.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.470+0000 7f7e8ffff700 1 --2- 172.21.15.31:0/4140134344 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f7e7c066f30 0x7f7e7c0693e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:10.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.470+0000 7f7e8ffff700 1 --2- 172.21.15.31:0/4140134344 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7e900cf110 0x7f7e900b2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:10.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.470+0000 7f7e8ffff700 1 --2- 172.21.15.31:0/4140134344 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e90153c30 0x7f7e90149f80 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:10.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.470+0000 7f7e8ffff700 1 -- 172.21.15.31:0/4140134344 >> 172.21.15.31:0/4140134344 conn(0x7f7e900703a0 msgr2=0x7f7e900c66b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:10.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.470+0000 7f7e8ffff700 1 -- 172.21.15.31:0/4140134344 shutdown_connections 2023-10-06T02:23:10.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.470+0000 7f7e8ffff700 1 -- 172.21.15.31:0/4140134344 wait complete. 2023-10-06T02:23:10.489 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:23:10.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.674+0000 7f4017fff700 1 Processor -- start 2023-10-06T02:23:10.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.674+0000 7f4017fff700 1 -- start start 2023-10-06T02:23:10.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.675+0000 7f4017fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4018153c30 0x7f401815a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:10.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.675+0000 7f4017fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f401815de50 0x7f40180cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:10.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.675+0000 7f4017fff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f401815afd0 con 0x7f40180cef40 2023-10-06T02:23:10.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.675+0000 7f4017fff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f401815b2b0 con 0x7f401815de50 2023-10-06T02:23:10.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.675+0000 7f4017fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f401815b590 con 0x7f4018153c30 2023-10-06T02:23:10.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.675+0000 7f400ffff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f401815de50 0x7f40180cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:10.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.675+0000 7f40177fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f40180cef40 0x7f401806d790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56106/0 (socket says 172.21.15.31:56106) 2023-10-06T02:23:10.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.676+0000 7f40177fe700 1 -- 172.21.15.31:0/1501973773 learned_addr learned my addr 172.21.15.31:0/1501973773 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:23:10.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.676+0000 7f4016ffd700 1 --2- 172.21.15.31:0/1501973773 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4018153c30 0x7f401815a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:10.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.676+0000 7f400ffff700 1 -- 172.21.15.31:0/1501973773 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4018153c30 msgr2=0x7f401815a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:10.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.676+0000 7f400ffff700 1 --2- 172.21.15.31:0/1501973773 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4018153c30 0x7f401815a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:10.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.676+0000 7f400ffff700 1 -- 172.21.15.31:0/1501973773 >> v1:172.21.15.31:6789/0 conn(0x7f40180cef40 legacy=0x7f401806d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:10.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.676+0000 7f400ffff700 1 -- 172.21.15.31:0/1501973773 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f401815b870 con 0x7f401815de50 2023-10-06T02:23:10.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.676+0000 7f40167fc700 1 -- 172.21.15.31:0/1501973773 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3457066851 0 0) 0x7f401815afd0 con 0x7f40180cef40 2023-10-06T02:23:10.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.676+0000 7f400ffff700 1 --2- 172.21.15.31:0/1501973773 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f401815de50 0x7f40180cea00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f40000098e0 tx=0x7f40000091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ddfddc37133b36 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:10.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.676+0000 7f40167fc700 1 -- 172.21.15.31:0/1501973773 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4000009e80 con 0x7f401815de50 2023-10-06T02:23:10.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.677+0000 7f40167fc700 1 -- 172.21.15.31:0/1501973773 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f400000b860 con 0x7f401815de50 2023-10-06T02:23:10.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.677+0000 7f40167fc700 1 -- 172.21.15.31:0/1501973773 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f400000bbe0 con 0x7f401815de50 2023-10-06T02:23:10.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.677+0000 7f4017fff700 1 -- 172.21.15.31:0/1501973773 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f401815de50 msgr2=0x7f40180cea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:10.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.677+0000 7f4017fff700 1 --2- 172.21.15.31:0/1501973773 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f401815de50 0x7f40180cea00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f40000098e0 tx=0x7f40000091a0 comp rx=0 tx=0).stop 2023-10-06T02:23:10.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.677+0000 7f4017fff700 1 -- 172.21.15.31:0/1501973773 shutdown_connections 2023-10-06T02:23:10.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.677+0000 7f4017fff700 1 --2- 172.21.15.31:0/1501973773 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4018153c30 0x7f401815a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:10.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.677+0000 7f4017fff700 1 --2- 172.21.15.31:0/1501973773 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f401815de50 0x7f40180cea00 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:10.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.677+0000 7f4017fff700 1 -- 172.21.15.31:0/1501973773 >> 172.21.15.31:0/1501973773 conn(0x7f40180703a0 msgr2=0x7f40180707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:10.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.677+0000 7f4017fff700 1 -- 172.21.15.31:0/1501973773 shutdown_connections 2023-10-06T02:23:10.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.677+0000 7f4017fff700 1 -- 172.21.15.31:0/1501973773 wait complete. 2023-10-06T02:23:10.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.679+0000 7f4017fff700 1 Processor -- start 2023-10-06T02:23:10.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.679+0000 7f4017fff700 1 -- start start 2023-10-06T02:23:10.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.680+0000 7f4017fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f40180cef40 0x7f4018152b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:10.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.680+0000 7f4017fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4018153c30 0x7f40180cc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:10.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.680+0000 7f4017fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f401815de50 0x7f40180cc620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:10.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.680+0000 7f4017fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40180c9ed0 con 0x7f401815de50 2023-10-06T02:23:10.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.680+0000 7f4017fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40180ca360 con 0x7f4018153c30 2023-10-06T02:23:10.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.680+0000 7f4017fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40180c92c0 con 0x7f40180cef40 2023-10-06T02:23:10.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.680+0000 7f40177fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f401815de50 0x7f40180cc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:10.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.680+0000 7f40177fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f401815de50 0x7f40180cc620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44268/0 (socket says 172.21.15.31:44268) 2023-10-06T02:23:10.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.680+0000 7f40177fe700 1 -- 172.21.15.31:0/2405399609 learned_addr learned my addr 172.21.15.31:0/2405399609 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:10.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.680+0000 7f4016ffd700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f40180cef40 0x7f4018152b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:10.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.680+0000 7f400ffff700 1 --2- 172.21.15.31:0/2405399609 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4018153c30 0x7f40180cc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:10.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.680+0000 7f40177fe700 1 -- 172.21.15.31:0/2405399609 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f40180cef40 msgr2=0x7f4018152b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:10.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.680+0000 7f40177fe700 1 --2- 172.21.15.31:0/2405399609 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f40180cef40 0x7f4018152b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:10.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.680+0000 7f40177fe700 1 -- 172.21.15.31:0/2405399609 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4018153c30 msgr2=0x7f40180cc0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:10.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.680+0000 7f40177fe700 1 --2- 172.21.15.31:0/2405399609 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4018153c30 0x7f40180cc0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:10.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.680+0000 7f40177fe700 1 -- 172.21.15.31:0/2405399609 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4000009420 con 0x7f401815de50 2023-10-06T02:23:10.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.681+0000 7f40177fe700 1 --2- 172.21.15.31:0/2405399609 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f401815de50 0x7f40180cc620 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f401000cbf0 tx=0x7f40100019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:10.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.681+0000 7f400effd700 1 -- 172.21.15.31:0/2405399609 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f401000a8b0 con 0x7f401815de50 2023-10-06T02:23:10.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.681+0000 7f400effd700 1 -- 172.21.15.31:0/2405399609 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f401000aa10 con 0x7f401815de50 2023-10-06T02:23:10.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.681+0000 7f400effd700 1 -- 172.21.15.31:0/2405399609 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f401000add0 con 0x7f401815de50 2023-10-06T02:23:10.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.681+0000 7f4017fff700 1 -- 172.21.15.31:0/2405399609 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40180c95a0 con 0x7f401815de50 2023-10-06T02:23:10.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.681+0000 7f4017fff700 1 -- 172.21.15.31:0/2405399609 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40180c9b60 con 0x7f401815de50 2023-10-06T02:23:10.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.682+0000 7f4017fff700 1 -- 172.21.15.31:0/2405399609 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40180ca040 con 0x7f401815de50 2023-10-06T02:23:10.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.683+0000 7f400effd700 1 -- 172.21.15.31:0/2405399609 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4010007300 con 0x7f401815de50 2023-10-06T02:23:10.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.684+0000 7f400effd700 1 --2- 172.21.15.31:0/2405399609 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f3ff8066f30 0x7f3ff80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:10.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.684+0000 7f400effd700 1 -- 172.21.15.31:0/2405399609 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f4010084a00 con 0x7f401815de50 2023-10-06T02:23:10.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.684+0000 7f4016ffd700 1 --2- 172.21.15.31:0/2405399609 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f3ff8066f30 0x7f3ff80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:10.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.685+0000 7f4016ffd700 1 --2- 172.21.15.31:0/2405399609 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f3ff8066f30 0x7f3ff80693e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4008006fd0 tx=0x7f4008008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:10.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.688+0000 7f400effd700 1 -- 172.21.15.31:0/2405399609 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4010051af0 con 0x7f401815de50 2023-10-06T02:23:10.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.892+0000 7f4017fff700 1 -- 172.21.15.31:0/2405399609 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f401804c3c0 con 0x7f401815de50 2023-10-06T02:23:10.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.893+0000 7f400effd700 1 -- 172.21.15.31:0/2405399609 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16831 (secure 0 0 0) 0x7f4010055110 con 0x7f401815de50 2023-10-06T02:23:10.895 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:23:10.895 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":22,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:23:09.520882+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:23:07.290269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T02:23:08.504679+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:23:10.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.896+0000 7f4017fff700 1 -- 172.21.15.31:0/2405399609 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f3ff8066f30 msgr2=0x7f3ff80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:10.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.896+0000 7f4017fff700 1 --2- 172.21.15.31:0/2405399609 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f3ff8066f30 0x7f3ff80693e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4008006fd0 tx=0x7f4008008040 comp rx=0 tx=0).stop 2023-10-06T02:23:10.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.896+0000 7f4017fff700 1 -- 172.21.15.31:0/2405399609 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f401815de50 msgr2=0x7f40180cc620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:10.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.896+0000 7f4017fff700 1 --2- 172.21.15.31:0/2405399609 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f401815de50 0x7f40180cc620 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f401000cbf0 tx=0x7f40100019b0 comp rx=0 tx=0).stop 2023-10-06T02:23:10.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.896+0000 7f4017fff700 1 -- 172.21.15.31:0/2405399609 shutdown_connections 2023-10-06T02:23:10.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.896+0000 7f4017fff700 1 --2- 172.21.15.31:0/2405399609 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f401815de50 0x7f40180cc620 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:10.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.896+0000 7f4017fff700 1 --2- 172.21.15.31:0/2405399609 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f3ff8066f30 0x7f3ff80693e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:10.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.896+0000 7f4017fff700 1 --2- 172.21.15.31:0/2405399609 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f40180cef40 0x7f4018152b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:10.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.896+0000 7f4017fff700 1 --2- 172.21.15.31:0/2405399609 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4018153c30 0x7f40180cc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:10.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.896+0000 7f4017fff700 1 -- 172.21.15.31:0/2405399609 >> 172.21.15.31:0/2405399609 conn(0x7f40180703a0 msgr2=0x7f4018070770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:10.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.896+0000 7f4017fff700 1 -- 172.21.15.31:0/2405399609 shutdown_connections 2023-10-06T02:23:10.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:10.896+0000 7f4017fff700 1 -- 172.21.15.31:0/2405399609 wait complete. 2023-10-06T02:23:10.915 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T02:23:11.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:11.109+0000 7fb5be59d700 1 Processor -- start 2023-10-06T02:23:11.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:11.109+0000 7fb5be59d700 1 -- start start 2023-10-06T02:23:11.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:11.111+0000 7fb5be59d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb5b8155890 0x7fb5b80c7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:11.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:11.111+0000 7fb5be59d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb5b806de50 0x7fb5b806e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:11.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:11.111+0000 7fb5be59d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5b80c7e70 con 0x7fb5b8155c60 2023-10-06T02:23:11.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:11.111+0000 7fb5be59d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5b806e7f0 con 0x7fb5b8155890 2023-10-06T02:23:11.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:11.111+0000 7fb5be59d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5b806b790 con 0x7fb5b806de50 2023-10-06T02:23:11.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:11.111+0000 7fb5bd59b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb5b8155890 0x7fb5b80c7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:11.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:11.111+0000 7fb5bd59b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb5b8155890 0x7fb5b80c7930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44282/0 (socket says 172.21.15.31:44282) 2023-10-06T02:23:11.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:11.111+0000 7fb5bd59b700 1 -- 172.21.15.31:0/3806284770 learned_addr learned my addr 172.21.15.31:0/3806284770 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:11.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:11.111+0000 7fb5bdd9c700 1 --2- 172.21.15.31:0/3806284770 >> v2:172.21.15.62:3300/0 conn(0x7fb5b806de50 0x7fb5b806e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:11.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:11.112+0000 7fb5bd59b700 1 -- 172.21.15.31:0/3806284770 >> v2:172.21.15.62:3300/0 conn(0x7fb5b806de50 msgr2=0x7fb5b806e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:11.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:11.112+0000 7fb5bd59b700 1 --2- 172.21.15.31:0/3806284770 >> v2:172.21.15.62:3300/0 conn(0x7fb5b806de50 0x7fb5b806e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:11.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:11.112+0000 7fb5bd59b700 1 -- 172.21.15.31:0/3806284770 >> v1:172.21.15.62:6789/0 conn(0x7fb5b8155c60 legacy=0x7fb5b806a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:11.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:11.112+0000 7fb5bd59b700 1 -- 172.21.15.31:0/3806284770 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5b806ab80 con 0x7fb5b8155890 2023-10-06T02:23:11.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:11.112+0000 7fb5bd59b700 1 --2- 172.21.15.31:0/3806284770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5b8155890 0x7fb5b80c7930 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb5b40099b0 tx=0x7fb5b4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12f56f2892d4304e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:11.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:11.112+0000 7fb5affff700 1 -- 172.21.15.31:0/3806284770 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5b400b7e0 con 0x7fb5b8155890 2023-10-06T02:23:11.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:11.112+0000 7fb5affff700 1 -- 172.21.15.31:0/3806284770 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5b400b940 con 0x7fb5b8155890 2023-10-06T02:23:11.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:11.112+0000 7fb5affff700 1 -- 172.21.15.31:0/3806284770 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5b400bce0 con 0x7fb5b8155890 2023-10-06T02:23:11.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:11.113+0000 7fb5be59d700 1 -- 172.21.15.31:0/3806284770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5b8155890 msgr2=0x7fb5b80c7930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:11.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:11.113+0000 7fb5be59d700 1 --2- 172.21.15.31:0/3806284770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5b8155890 0x7fb5b80c7930 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb5b40099b0 tx=0x7fb5b4009250 comp rx=0 tx=0).stop 2023-10-06T02:23:11.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:11.113+0000 7fb5be59d700 1 -- 172.21.15.31:0/3806284770 shutdown_connections 2023-10-06T02:23:11.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:11.113+0000 7fb5be59d700 1 --2- 172.21.15.31:0/3806284770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5b8155890 0x7fb5b80c7930 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:11.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:11.113+0000 7fb5be59d700 1 --2- 172.21.15.31:0/3806284770 >> v2:172.21.15.62:3300/0 conn(0x7fb5b806de50 0x7fb5b806e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:11.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:11.113+0000 7fb5be59d700 1 -- 172.21.15.31:0/3806284770 >> 172.21.15.31:0/3806284770 conn(0x7fb5b806d110 msgr2=0x7fb5b80cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:11.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:11.113+0000 7fb5be59d700 1 -- 172.21.15.31:0/3806284770 shutdown_connections 2023-10-06T02:23:11.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:11.113+0000 7fb5be59d700 1 -- 172.21.15.31:0/3806284770 wait complete. 2023-10-06T02:23:11.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:11.114+0000 7fb5be59d700 1 Processor -- start 2023-10-06T02:23:11.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:11.114+0000 7fb5be59d700 1 -- start start 2023-10-06T02:23:11.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:11.115+0000 7fb5be59d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5b806de50 0x7fb5b80c4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:11.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:11.115+0000 7fb5be59d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5b8155890 0x7fb5b8156190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:11.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:11.115+0000 7fb5be59d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb5b8155c60 0x7fb5b80c5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:11.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:11.115+0000 7fb5be59d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b81566d0 con 0x7fb5b806de50 2023-10-06T02:23:11.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:11.115+0000 7fb5be59d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b81574d0 con 0x7fb5b8155890 2023-10-06T02:23:11.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:11.115+0000 7fb5be59d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b81568c0 con 0x7fb5b8155c60 2023-10-06T02:23:11.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:11.115+0000 7fb5bd59b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5b806de50 0x7fb5b80c4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:11.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:11.115+0000 7fb5bd59b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5b806de50 0x7fb5b80c4620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44290/0 (socket says 172.21.15.31:44290) 2023-10-06T02:23:11.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:11.115+0000 7fb5bd59b700 1 -- 172.21.15.31:0/1053742955 learned_addr learned my addr 172.21.15.31:0/1053742955 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:11.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:11.116+0000 7fb5bdd9c700 1 --2- 172.21.15.31:0/1053742955 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb5b8155c60 0x7fb5b80c5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:11.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:11.116+0000 7fb5bcd9a700 1 --2- 172.21.15.31:0/1053742955 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5b8155890 0x7fb5b8156190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:11.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:11.116+0000 7fb5bd59b700 1 -- 172.21.15.31:0/1053742955 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb5b8155c60 msgr2=0x7fb5b80c5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:11.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:11.116+0000 7fb5bd59b700 1 --2- 172.21.15.31:0/1053742955 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb5b8155c60 0x7fb5b80c5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:11.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:11.116+0000 7fb5bd59b700 1 -- 172.21.15.31:0/1053742955 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5b8155890 msgr2=0x7fb5b8156190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:11.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:11.116+0000 7fb5bd59b700 1 --2- 172.21.15.31:0/1053742955 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5b8155890 0x7fb5b8156190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:11.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:11.116+0000 7fb5bd59b700 1 -- 172.21.15.31:0/1053742955 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5b40094f0 con 0x7fb5b806de50 2023-10-06T02:23:11.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:11.116+0000 7fb5bd59b700 1 --2- 172.21.15.31:0/1053742955 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5b806de50 0x7fb5b80c4620 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb5b4009290 tx=0x7fb5b4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:11.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:11.117+0000 7fb5ad7fa700 1 -- 172.21.15.31:0/1053742955 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5b401b470 con 0x7fb5b806de50 2023-10-06T02:23:11.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:11.117+0000 7fb5ad7fa700 1 -- 172.21.15.31:0/1053742955 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5b401b5d0 con 0x7fb5b806de50 2023-10-06T02:23:11.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:11.117+0000 7fb5be59d700 1 -- 172.21.15.31:0/1053742955 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5b8156ba0 con 0x7fb5b806de50 2023-10-06T02:23:11.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:11.117+0000 7fb5ad7fa700 1 -- 172.21.15.31:0/1053742955 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5b401b990 con 0x7fb5b806de50 2023-10-06T02:23:11.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:11.117+0000 7fb5be59d700 1 -- 172.21.15.31:0/1053742955 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5b8157160 con 0x7fb5b806de50 2023-10-06T02:23:11.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:11.118+0000 7fb5be59d700 1 -- 172.21.15.31:0/1053742955 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5b806bc20 con 0x7fb5b806de50 2023-10-06T02:23:11.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:11.119+0000 7fb5ad7fa700 1 -- 172.21.15.31:0/1053742955 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb5b401c450 con 0x7fb5b806de50 2023-10-06T02:23:11.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:11.119+0000 7fb5ad7fa700 1 --2- 172.21.15.31:0/1053742955 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fb59c066f30 0x7fb59c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:11.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:11.120+0000 7fb5ad7fa700 1 -- 172.21.15.31:0/1053742955 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb5b40863f0 con 0x7fb5b806de50 2023-10-06T02:23:11.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:11.120+0000 7fb5bcd9a700 1 --2- 172.21.15.31:0/1053742955 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fb59c066f30 0x7fb59c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:11.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:11.120+0000 7fb5bcd9a700 1 --2- 172.21.15.31:0/1053742955 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fb59c066f30 0x7fb59c0693e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb5a8006fd0 tx=0x7fb5a8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:11.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:11.124+0000 7fb5ad7fa700 1 -- 172.21.15.31:0/1053742955 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb5b806bc20 con 0x7fb5b806de50 2023-10-06T02:23:11.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:11.371+0000 7fb5be59d700 1 -- 172.21.15.31:0/1053742955 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fb5b806bc20 con 0x7fb5b806de50 2023-10-06T02:23:12.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:12.049+0000 7fb5ad7fa700 1 -- 172.21.15.31:0/1053742955 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7fb5b806bc20 con 0x7fb5b806de50 2023-10-06T02:23:12.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:12.052+0000 7fb5be59d700 1 -- 172.21.15.31:0/1053742955 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fb59c066f30 msgr2=0x7fb59c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:12.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:12.052+0000 7fb5be59d700 1 --2- 172.21.15.31:0/1053742955 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fb59c066f30 0x7fb59c0693e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb5a8006fd0 tx=0x7fb5a8008040 comp rx=0 tx=0).stop 2023-10-06T02:23:12.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:12.052+0000 7fb5be59d700 1 -- 172.21.15.31:0/1053742955 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5b806de50 msgr2=0x7fb5b80c4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:12.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:12.052+0000 7fb5be59d700 1 --2- 172.21.15.31:0/1053742955 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5b806de50 0x7fb5b80c4620 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb5b4009290 tx=0x7fb5b4016000 comp rx=0 tx=0).stop 2023-10-06T02:23:12.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:12.053+0000 7fb5be59d700 1 -- 172.21.15.31:0/1053742955 shutdown_connections 2023-10-06T02:23:12.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:12.053+0000 7fb5be59d700 1 --2- 172.21.15.31:0/1053742955 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fb59c066f30 0x7fb59c0693e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:12.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:12.053+0000 7fb5be59d700 1 --2- 172.21.15.31:0/1053742955 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb5b8155c60 0x7fb5b80c5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:12.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:12.053+0000 7fb5be59d700 1 --2- 172.21.15.31:0/1053742955 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5b806de50 0x7fb5b80c4620 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:12.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:12.053+0000 7fb5be59d700 1 --2- 172.21.15.31:0/1053742955 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5b8155890 0x7fb5b8156190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:12.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:12.053+0000 7fb5be59d700 1 -- 172.21.15.31:0/1053742955 >> 172.21.15.31:0/1053742955 conn(0x7fb5b806d110 msgr2=0x7fb5b81588d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:12.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:12.053+0000 7fb5be59d700 1 -- 172.21.15.31:0/1053742955 shutdown_connections 2023-10-06T02:23:12.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:12.053+0000 7fb5be59d700 1 -- 172.21.15.31:0/1053742955 wait complete. 2023-10-06T02:23:12.078 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T02:23:12.167 INFO:tasks.ceph.mds.a.smithi031.stderr:2023-10-06T02:23:12.166+0000 7f8f2fff1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T02:23:12.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:12.270+0000 7f1e689fe700 1 Processor -- start 2023-10-06T02:23:12.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:12.270+0000 7f1e689fe700 1 -- start start 2023-10-06T02:23:12.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:12.272+0000 7f1e689fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1e640c7be0 0x7f1e6406abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:12.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:12.272+0000 7f1e689fe700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e6406b290 con 0x7f1e640c7fb0 2023-10-06T02:23:12.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:12.272+0000 7f1e689fe700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e6406b570 con 0x7f1e6413f780 2023-10-06T02:23:12.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:12.272+0000 7f1e689fe700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e6406bf10 con 0x7f1e640c7be0 2023-10-06T02:23:12.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:12.272+0000 7f1e62d9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1e640c7be0 0x7f1e6406abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:12.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:12.272+0000 7f1e62d9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1e640c7be0 0x7f1e6406abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44296/0 (socket says 172.21.15.31:44296) 2023-10-06T02:23:12.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:12.272+0000 7f1e6259c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f1e640c7fb0 0x7f1e6415a990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56112/0 (socket says 172.21.15.31:56112) 2023-10-06T02:23:12.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:12.272+0000 7f1e62d9d700 1 -- 172.21.15.31:0/398994165 learned_addr learned my addr 172.21.15.31:0/398994165 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:12.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:12.272+0000 7f1e62d9d700 1 -- 172.21.15.31:0/398994165 >> v1:172.21.15.62:6789/0 conn(0x7f1e6413f780 legacy=0x7f1e6413fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:12.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:12.272+0000 7f1e62d9d700 1 -- 172.21.15.31:0/398994165 >> v1:172.21.15.31:6789/0 conn(0x7f1e640c7fb0 legacy=0x7f1e6415a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:12.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:12.272+0000 7f1e62d9d700 1 -- 172.21.15.31:0/398994165 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e6406b850 con 0x7f1e640c7be0 2023-10-06T02:23:12.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:12.273+0000 7f1e62d9d700 1 --2- 172.21.15.31:0/398994165 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e640c7be0 0x7f1e6406abd0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1e540099b0 tx=0x7f1e54009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52a6426d780e9787 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:12.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:12.273+0000 7f1e61d9b700 1 -- 172.21.15.31:0/398994165 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e5400b860 con 0x7f1e640c7be0 2023-10-06T02:23:12.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:12.273+0000 7f1e61d9b700 1 -- 172.21.15.31:0/398994165 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e5400b9c0 con 0x7f1e640c7be0 2023-10-06T02:23:12.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:12.273+0000 7f1e61d9b700 1 -- 172.21.15.31:0/398994165 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e5400bd60 con 0x7f1e640c7be0 2023-10-06T02:23:12.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:12.273+0000 7f1e689fe700 1 -- 172.21.15.31:0/398994165 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e640c7be0 msgr2=0x7f1e6406abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:12.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:12.273+0000 7f1e689fe700 1 --2- 172.21.15.31:0/398994165 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e640c7be0 0x7f1e6406abd0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1e540099b0 tx=0x7f1e54009250 comp rx=0 tx=0).stop 2023-10-06T02:23:12.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:12.274+0000 7f1e689fe700 1 -- 172.21.15.31:0/398994165 shutdown_connections 2023-10-06T02:23:12.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:12.274+0000 7f1e689fe700 1 --2- 172.21.15.31:0/398994165 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e640c7be0 0x7f1e6406abd0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:12.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:12.274+0000 7f1e689fe700 1 -- 172.21.15.31:0/398994165 >> 172.21.15.31:0/398994165 conn(0x7f1e6406d110 msgr2=0x7f1e640cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:12.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:12.274+0000 7f1e689fe700 1 -- 172.21.15.31:0/398994165 shutdown_connections 2023-10-06T02:23:12.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:12.274+0000 7f1e689fe700 1 -- 172.21.15.31:0/398994165 wait complete. 2023-10-06T02:23:12.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:12.275+0000 7f1e689fe700 1 Processor -- start 2023-10-06T02:23:12.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:12.275+0000 7f1e689fe700 1 -- start start 2023-10-06T02:23:12.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:12.276+0000 7f1e689fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e640c7be0 0x7f1e640c55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:12.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:12.276+0000 7f1e689fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e640c7fb0 0x7f1e640c41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:12.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:12.276+0000 7f1e689fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e6413f780 0x7f1e640c46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:12.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:12.276+0000 7f1e689fe700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e64159b20 con 0x7f1e640c7be0 2023-10-06T02:23:12.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:12.276+0000 7f1e689fe700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e64158fa0 con 0x7f1e6413f780 2023-10-06T02:23:12.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:12.276+0000 7f1e689fe700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e64159280 con 0x7f1e640c7fb0 2023-10-06T02:23:12.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:12.276+0000 7f1e62d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e640c7be0 0x7f1e640c55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:12.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:12.276+0000 7f1e62d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e640c7be0 0x7f1e640c55e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44304/0 (socket says 172.21.15.31:44304) 2023-10-06T02:23:12.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:12.276+0000 7f1e62d9d700 1 -- 172.21.15.31:0/3657063398 learned_addr learned my addr 172.21.15.31:0/3657063398 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:12.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:12.276+0000 7f1e6259c700 1 --2- 172.21.15.31:0/3657063398 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e640c7fb0 0x7f1e640c41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:12.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:12.276+0000 7f1e6359e700 1 --2- 172.21.15.31:0/3657063398 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e6413f780 0x7f1e640c46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:12.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:12.277+0000 7f1e62d9d700 1 -- 172.21.15.31:0/3657063398 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e640c7fb0 msgr2=0x7f1e640c41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:12.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:12.277+0000 7f1e62d9d700 1 --2- 172.21.15.31:0/3657063398 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e640c7fb0 0x7f1e640c41b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:12.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:12.277+0000 7f1e62d9d700 1 -- 172.21.15.31:0/3657063398 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e6413f780 msgr2=0x7f1e640c46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:12.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:12.277+0000 7f1e62d9d700 1 --2- 172.21.15.31:0/3657063398 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e6413f780 0x7f1e640c46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:12.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:12.277+0000 7f1e62d9d700 1 -- 172.21.15.31:0/3657063398 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e540094f0 con 0x7f1e640c7be0 2023-10-06T02:23:12.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:12.277+0000 7f1e6259c700 1 --2- 172.21.15.31:0/3657063398 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e640c7fb0 0x7f1e640c41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:23:12.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:12.277+0000 7f1e6359e700 1 --2- 172.21.15.31:0/3657063398 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e6413f780 0x7f1e640c46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:23:12.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:12.277+0000 7f1e62d9d700 1 --2- 172.21.15.31:0/3657063398 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e640c7be0 0x7f1e640c55e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1e640693f0 tx=0x7f1e54017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:12.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:12.277+0000 7f1e4affd700 1 -- 172.21.15.31:0/3657063398 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e54011ea0 con 0x7f1e640c7be0 2023-10-06T02:23:12.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:12.277+0000 7f1e4affd700 1 -- 172.21.15.31:0/3657063398 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e5401a630 con 0x7f1e640c7be0 2023-10-06T02:23:12.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:12.277+0000 7f1e689fe700 1 -- 172.21.15.31:0/3657063398 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e64159560 con 0x7f1e640c7be0 2023-10-06T02:23:12.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:12.277+0000 7f1e4affd700 1 -- 172.21.15.31:0/3657063398 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e5401a9d0 con 0x7f1e640c7be0 2023-10-06T02:23:12.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:12.277+0000 7f1e689fe700 1 -- 172.21.15.31:0/3657063398 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e640c5b20 con 0x7f1e640c7be0 2023-10-06T02:23:12.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:12.278+0000 7f1e689fe700 1 -- 172.21.15.31:0/3657063398 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e640aeed0 con 0x7f1e640c7be0 2023-10-06T02:23:12.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:12.279+0000 7f1e4affd700 1 -- 172.21.15.31:0/3657063398 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1e5402d440 con 0x7f1e640c7be0 2023-10-06T02:23:12.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:12.280+0000 7f1e4affd700 1 --2- 172.21.15.31:0/3657063398 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f1e50066f30 0x7f1e500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:12.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:12.280+0000 7f1e4affd700 1 -- 172.21.15.31:0/3657063398 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1e540862e0 con 0x7f1e640c7be0 2023-10-06T02:23:12.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:12.280+0000 7f1e6259c700 1 --2- 172.21.15.31:0/3657063398 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f1e50066f30 0x7f1e500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:12.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:12.281+0000 7f1e6259c700 1 --2- 172.21.15.31:0/3657063398 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f1e50066f30 0x7f1e500693e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1e58007fd0 tx=0x7f1e58007e10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:12.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:12.284+0000 7f1e4affd700 1 -- 172.21.15.31:0/3657063398 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1e640aeed0 con 0x7f1e640c7be0 2023-10-06T02:23:12.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:12.533+0000 7f1e689fe700 1 -- 172.21.15.31:0/3657063398 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f1e640aeed0 con 0x7f1e640c7be0 2023-10-06T02:23:13.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.049+0000 7f1e4affd700 1 -- 172.21.15.31:0/3657063398 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1e640aeed0 con 0x7f1e640c7be0 2023-10-06T02:23:13.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.052+0000 7f1e689fe700 1 -- 172.21.15.31:0/3657063398 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f1e50066f30 msgr2=0x7f1e500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:13.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.052+0000 7f1e689fe700 1 --2- 172.21.15.31:0/3657063398 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f1e50066f30 0x7f1e500693e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1e58007fd0 tx=0x7f1e58007e10 comp rx=0 tx=0).stop 2023-10-06T02:23:13.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.052+0000 7f1e689fe700 1 -- 172.21.15.31:0/3657063398 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e640c7be0 msgr2=0x7f1e640c55e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:13.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.052+0000 7f1e689fe700 1 --2- 172.21.15.31:0/3657063398 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e640c7be0 0x7f1e640c55e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1e640693f0 tx=0x7f1e54017000 comp rx=0 tx=0).stop 2023-10-06T02:23:13.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.053+0000 7f1e689fe700 1 -- 172.21.15.31:0/3657063398 shutdown_connections 2023-10-06T02:23:13.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.053+0000 7f1e689fe700 1 --2- 172.21.15.31:0/3657063398 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e6413f780 0x7f1e640c46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:13.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.053+0000 7f1e689fe700 1 --2- 172.21.15.31:0/3657063398 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e640c7be0 0x7f1e640c55e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:13.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.053+0000 7f1e689fe700 1 --2- 172.21.15.31:0/3657063398 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f1e50066f30 0x7f1e500693e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:13.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.053+0000 7f1e689fe700 1 --2- 172.21.15.31:0/3657063398 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e640c7fb0 0x7f1e640c41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:13.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.053+0000 7f1e689fe700 1 -- 172.21.15.31:0/3657063398 >> 172.21.15.31:0/3657063398 conn(0x7f1e6406d110 msgr2=0x7f1e640c2890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:13.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.053+0000 7f1e689fe700 1 -- 172.21.15.31:0/3657063398 shutdown_connections 2023-10-06T02:23:13.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.053+0000 7f1e689fe700 1 -- 172.21.15.31:0/3657063398 wait complete. 2023-10-06T02:23:13.073 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:23:13.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.262+0000 7f35d619d700 1 Processor -- start 2023-10-06T02:23:13.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.262+0000 7f35d619d700 1 -- start start 2023-10-06T02:23:13.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.263+0000 7f35d619d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f35d015dfd0 0x7f35d00aedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:13.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.263+0000 7f35d619d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f35d0159920 0x7f35d00ad090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:13.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.263+0000 7f35d619d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35d00acc00 con 0x7f35d0159550 2023-10-06T02:23:13.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.263+0000 7f35d619d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35d00abff0 con 0x7f35d0159920 2023-10-06T02:23:13.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.263+0000 7f35d619d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35d00ac2d0 con 0x7f35d015dfd0 2023-10-06T02:23:13.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.263+0000 7f35d599c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f35d0159920 0x7f35d00ad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:13.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.263+0000 7f35d599c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f35d0159920 0x7f35d00ad090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44308/0 (socket says 172.21.15.31:44308) 2023-10-06T02:23:13.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.263+0000 7f35d519b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f35d0159550 0x7f35d00ceac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56114/0 (socket says 172.21.15.31:56114) 2023-10-06T02:23:13.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.263+0000 7f35d599c700 1 -- 172.21.15.31:0/2091096502 learned_addr learned my addr 172.21.15.31:0/2091096502 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:13.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.264+0000 7f35d499a700 1 --2- 172.21.15.31:0/2091096502 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f35d015dfd0 0x7f35d00aedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:13.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.264+0000 7f35d599c700 1 -- 172.21.15.31:0/2091096502 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f35d015dfd0 msgr2=0x7f35d00aedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:13.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.264+0000 7f35d599c700 1 --2- 172.21.15.31:0/2091096502 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f35d015dfd0 0x7f35d00aedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:13.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.264+0000 7f35d599c700 1 -- 172.21.15.31:0/2091096502 >> v1:172.21.15.31:6789/0 conn(0x7f35d0159550 legacy=0x7f35d00ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:13.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.264+0000 7f35d599c700 1 -- 172.21.15.31:0/2091096502 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d00ac5b0 con 0x7f35d0159920 2023-10-06T02:23:13.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.264+0000 7f35c7fff700 1 -- 172.21.15.31:0/2091096502 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2781489981 0 0) 0x7f35d00acc00 con 0x7f35d0159550 2023-10-06T02:23:13.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.264+0000 7f35d499a700 1 --2- 172.21.15.31:0/2091096502 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f35d015dfd0 0x7f35d00aedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:23:13.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.264+0000 7f35d599c700 1 --2- 172.21.15.31:0/2091096502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35d0159920 0x7f35d00ad090 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f35c8009f20 tx=0x7f35c8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5934e6a3db51f982 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:13.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.264+0000 7f35c7fff700 1 -- 172.21.15.31:0/2091096502 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c800e070 con 0x7f35d0159920 2023-10-06T02:23:13.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.264+0000 7f35c7fff700 1 -- 172.21.15.31:0/2091096502 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35c800ba80 con 0x7f35d0159920 2023-10-06T02:23:13.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.264+0000 7f35c7fff700 1 -- 172.21.15.31:0/2091096502 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c80092e0 con 0x7f35d0159920 2023-10-06T02:23:13.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.265+0000 7f35d619d700 1 -- 172.21.15.31:0/2091096502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35d0159920 msgr2=0x7f35d00ad090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:13.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.265+0000 7f35d619d700 1 --2- 172.21.15.31:0/2091096502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35d0159920 0x7f35d00ad090 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f35c8009f20 tx=0x7f35c8009840 comp rx=0 tx=0).stop 2023-10-06T02:23:13.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.265+0000 7f35d619d700 1 -- 172.21.15.31:0/2091096502 shutdown_connections 2023-10-06T02:23:13.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.265+0000 7f35d619d700 1 --2- 172.21.15.31:0/2091096502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35d0159920 0x7f35d00ad090 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:13.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.265+0000 7f35d619d700 1 --2- 172.21.15.31:0/2091096502 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f35d015dfd0 0x7f35d00aedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:13.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.265+0000 7f35d619d700 1 -- 172.21.15.31:0/2091096502 >> 172.21.15.31:0/2091096502 conn(0x7f35d006ce80 msgr2=0x7f35d00ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:13.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.265+0000 7f35d619d700 1 -- 172.21.15.31:0/2091096502 shutdown_connections 2023-10-06T02:23:13.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.265+0000 7f35d619d700 1 -- 172.21.15.31:0/2091096502 wait complete. 2023-10-06T02:23:13.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.266+0000 7f35d619d700 1 Processor -- start 2023-10-06T02:23:13.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.267+0000 7f35d619d700 1 -- start start 2023-10-06T02:23:13.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.267+0000 7f35d619d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35d0159550 0x7f35d014da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:13.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.267+0000 7f35d619d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35d0159920 0x7f35d014cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:13.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.267+0000 7f35d619d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f35d015dfd0 0x7f35d0151d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:13.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.268+0000 7f35d619d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d00c7560 con 0x7f35d0159550 2023-10-06T02:23:13.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.268+0000 7f35d619d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d016d4e0 con 0x7f35d0159920 2023-10-06T02:23:13.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.268+0000 7f35d619d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35d016d7c0 con 0x7f35d015dfd0 2023-10-06T02:23:13.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.268+0000 7f35d519b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35d0159550 0x7f35d014da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:13.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.268+0000 7f35d519b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35d0159550 0x7f35d014da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44318/0 (socket says 172.21.15.31:44318) 2023-10-06T02:23:13.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.268+0000 7f35d519b700 1 -- 172.21.15.31:0/1863588416 learned_addr learned my addr 172.21.15.31:0/1863588416 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:13.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.268+0000 7f35d499a700 1 --2- 172.21.15.31:0/1863588416 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35d0159920 0x7f35d014cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:13.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.268+0000 7f35d599c700 1 --2- 172.21.15.31:0/1863588416 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f35d015dfd0 0x7f35d0151d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:13.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.268+0000 7f35d519b700 1 -- 172.21.15.31:0/1863588416 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f35d015dfd0 msgr2=0x7f35d0151d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:13.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.268+0000 7f35d519b700 1 --2- 172.21.15.31:0/1863588416 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f35d015dfd0 0x7f35d0151d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:13.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.268+0000 7f35d519b700 1 -- 172.21.15.31:0/1863588416 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35d0159920 msgr2=0x7f35d014cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:13.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.268+0000 7f35d519b700 1 --2- 172.21.15.31:0/1863588416 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35d0159920 0x7f35d014cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:13.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.268+0000 7f35d519b700 1 -- 172.21.15.31:0/1863588416 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35c8009ae0 con 0x7f35d0159550 2023-10-06T02:23:13.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.268+0000 7f35d599c700 1 --2- 172.21.15.31:0/1863588416 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f35d015dfd0 0x7f35d0151d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:23:13.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.268+0000 7f35d519b700 1 --2- 172.21.15.31:0/1863588416 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35d0159550 0x7f35d014da60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f35cc0011e0 tx=0x7f35cc007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:13.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.269+0000 7f35c57fa700 1 -- 172.21.15.31:0/1863588416 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35cc005a70 con 0x7f35d0159550 2023-10-06T02:23:13.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.269+0000 7f35c57fa700 1 -- 172.21.15.31:0/1863588416 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35cc005bd0 con 0x7f35d0159550 2023-10-06T02:23:13.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.269+0000 7f35d619d700 1 -- 172.21.15.31:0/1863588416 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35d016daa0 con 0x7f35d0159550 2023-10-06T02:23:13.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.269+0000 7f35c57fa700 1 -- 172.21.15.31:0/1863588416 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35cc0070d0 con 0x7f35d0159550 2023-10-06T02:23:13.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.269+0000 7f35d619d700 1 -- 172.21.15.31:0/1863588416 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35d014dfa0 con 0x7f35d0159550 2023-10-06T02:23:13.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.270+0000 7f35d619d700 1 -- 172.21.15.31:0/1863588416 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35d004c3c0 con 0x7f35d0159550 2023-10-06T02:23:13.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.271+0000 7f35c57fa700 1 -- 172.21.15.31:0/1863588416 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f35cc007480 con 0x7f35d0159550 2023-10-06T02:23:13.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.271+0000 7f35c57fa700 1 --2- 172.21.15.31:0/1863588416 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f35b4066f30 0x7f35b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:13.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.272+0000 7f35c57fa700 1 -- 172.21.15.31:0/1863588416 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f35cc083b90 con 0x7f35d0159550 2023-10-06T02:23:13.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.272+0000 7f35d499a700 1 --2- 172.21.15.31:0/1863588416 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f35b4066f30 0x7f35b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:13.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.272+0000 7f35d499a700 1 --2- 172.21.15.31:0/1863588416 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f35b4066f30 0x7f35b40693e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f35d00b2a80 tx=0x7f35c0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:13.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.275+0000 7f35c57fa700 1 -- 172.21.15.31:0/1863588416 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f35cc04b5a0 con 0x7f35d0159550 2023-10-06T02:23:13.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.522+0000 7f35d619d700 1 -- 172.21.15.31:0/1863588416 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f35d006ef90 con 0x7f35d0159550 2023-10-06T02:23:13.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.523+0000 7f35c57fa700 1 -- 172.21.15.31:0/1863588416 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4591 (secure 0 0 0) 0x7f35cc017020 con 0x7f35d0159550 2023-10-06T02:23:13.524 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:23:13.524 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/1808327183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":1808327183},{"type":"v1","addr":"172.21.15.31:6837","nonce":1808327183}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/4263144517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":4263144517},{"type":"v1","addr":"172.21.15.62:6834","nonce":4263144517}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1489208568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":1489208568},{"type":"v1","addr":"172.21.15.62:6835","nonce":1489208568}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:23:09.521236+0000","modified":"2023-10-06T02:23:13.047274+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6836/3809284664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3809284664},{"type":"v1","addr":"172.21.15.31:6836","nonce":3809284664}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T02:23:13.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.525+0000 7f35d619d700 1 -- 172.21.15.31:0/1863588416 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f35b4066f30 msgr2=0x7f35b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:13.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.525+0000 7f35d619d700 1 --2- 172.21.15.31:0/1863588416 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f35b4066f30 0x7f35b40693e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f35d00b2a80 tx=0x7f35c0008040 comp rx=0 tx=0).stop 2023-10-06T02:23:13.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.525+0000 7f35d619d700 1 -- 172.21.15.31:0/1863588416 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35d0159550 msgr2=0x7f35d014da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:13.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.525+0000 7f35d619d700 1 --2- 172.21.15.31:0/1863588416 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35d0159550 0x7f35d014da60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f35cc0011e0 tx=0x7f35cc007e10 comp rx=0 tx=0).stop 2023-10-06T02:23:13.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.525+0000 7f35d619d700 1 -- 172.21.15.31:0/1863588416 shutdown_connections 2023-10-06T02:23:13.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.525+0000 7f35d619d700 1 --2- 172.21.15.31:0/1863588416 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f35b4066f30 0x7f35b40693e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:13.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.525+0000 7f35d619d700 1 --2- 172.21.15.31:0/1863588416 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f35d015dfd0 0x7f35d0151d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:13.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.525+0000 7f35d619d700 1 --2- 172.21.15.31:0/1863588416 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35d0159550 0x7f35d014da60 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:13.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.525+0000 7f35d619d700 1 --2- 172.21.15.31:0/1863588416 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35d0159920 0x7f35d014cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:13.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.525+0000 7f35d619d700 1 -- 172.21.15.31:0/1863588416 >> 172.21.15.31:0/1863588416 conn(0x7f35d006ce80 msgr2=0x7f35d01537a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:13.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.526+0000 7f35d619d700 1 -- 172.21.15.31:0/1863588416 shutdown_connections 2023-10-06T02:23:13.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.526+0000 7f35d619d700 1 -- 172.21.15.31:0/1863588416 wait complete. 2023-10-06T02:23:13.528 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 8 2023-10-06T02:23:13.544 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:23:13.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.729+0000 7fc87a99d700 1 Processor -- start 2023-10-06T02:23:13.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.729+0000 7fc87a99d700 1 -- start start 2023-10-06T02:23:13.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.730+0000 7fc87a99d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc874142c00 0x7fc87406b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:13.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.730+0000 7fc87a99d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8740ca480 con 0x7fc87406bd20 2023-10-06T02:23:13.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.730+0000 7fc87a99d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc87415e240 con 0x7fc8740c7f90 2023-10-06T02:23:13.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.730+0000 7fc87a99d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc87406b350 con 0x7fc874142c00 2023-10-06T02:23:13.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.730+0000 7fc87919a700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fc87406bd20 0x7fc8740c7880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56128/0 (socket says 172.21.15.31:56128) 2023-10-06T02:23:13.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.730+0000 7fc87919a700 1 -- 172.21.15.31:0/2841190263 learned_addr learned my addr 172.21.15.31:0/2841190263 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:23:13.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.731+0000 7fc87999b700 1 --2- 172.21.15.31:0/2841190263 >> v2:172.21.15.62:3300/0 conn(0x7fc874142c00 0x7fc87406b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:13.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.731+0000 7fc878999700 1 -- 172.21.15.31:0/2841190263 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3777563011 0 0) 0x7fc8740ca480 con 0x7fc87406bd20 2023-10-06T02:23:13.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.731+0000 7fc878999700 1 -- 172.21.15.31:0/2841190263 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc860003570 con 0x7fc87406bd20 2023-10-06T02:23:13.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.731+0000 7fc878999700 1 -- 172.21.15.31:0/2841190263 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 743272575 0 0) 0x7fc87415e240 con 0x7fc8740c7f90 2023-10-06T02:23:13.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.731+0000 7fc878999700 1 -- 172.21.15.31:0/2841190263 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8740ca480 con 0x7fc8740c7f90 2023-10-06T02:23:13.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.731+0000 7fc878999700 1 -- 172.21.15.31:0/2841190263 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2225792141 0 0) 0x7fc860003570 con 0x7fc87406bd20 2023-10-06T02:23:13.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.731+0000 7fc87999b700 1 -- 172.21.15.31:0/2841190263 >> v1:172.21.15.62:6789/0 conn(0x7fc8740c7f90 legacy=0x7fc87406c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:13.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.731+0000 7fc87999b700 1 -- 172.21.15.31:0/2841190263 >> v1:172.21.15.31:6789/0 conn(0x7fc87406bd20 legacy=0x7fc8740c7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:13.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.731+0000 7fc87999b700 1 -- 172.21.15.31:0/2841190263 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc87415df60 con 0x7fc874142c00 2023-10-06T02:23:13.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.731+0000 7fc87999b700 1 --2- 172.21.15.31:0/2841190263 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc874142c00 0x7fc87406b7e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc8700098e0 tx=0x7fc8700091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b2922b5c6a94340 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:13.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.732+0000 7fc878999700 1 -- 172.21.15.31:0/2841190263 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc864001470 con 0x7fc874142c00 2023-10-06T02:23:13.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.732+0000 7fc878999700 1 -- 172.21.15.31:0/2841190263 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc870011420 con 0x7fc874142c00 2023-10-06T02:23:13.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.732+0000 7fc878999700 1 -- 172.21.15.31:0/2841190263 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8700117c0 con 0x7fc874142c00 2023-10-06T02:23:13.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.732+0000 7fc87a99d700 1 -- 172.21.15.31:0/2841190263 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc874142c00 msgr2=0x7fc87406b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:13.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.732+0000 7fc87a99d700 1 --2- 172.21.15.31:0/2841190263 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc874142c00 0x7fc87406b7e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc8700098e0 tx=0x7fc8700091a0 comp rx=0 tx=0).stop 2023-10-06T02:23:13.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.732+0000 7fc87a99d700 1 -- 172.21.15.31:0/2841190263 shutdown_connections 2023-10-06T02:23:13.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.732+0000 7fc87a99d700 1 --2- 172.21.15.31:0/2841190263 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc874142c00 0x7fc87406b7e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:13.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.732+0000 7fc87a99d700 1 -- 172.21.15.31:0/2841190263 >> 172.21.15.31:0/2841190263 conn(0x7fc8740675a0 msgr2=0x7fc8740679a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:13.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.732+0000 7fc87a99d700 1 -- 172.21.15.31:0/2841190263 shutdown_connections 2023-10-06T02:23:13.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.732+0000 7fc87a99d700 1 -- 172.21.15.31:0/2841190263 wait complete. 2023-10-06T02:23:13.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.734+0000 7fc87a99d700 1 Processor -- start 2023-10-06T02:23:13.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.734+0000 7fc87a99d700 1 -- start start 2023-10-06T02:23:13.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.735+0000 7fc87a99d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc87406bd20 0x7fc8740b24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:13.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.735+0000 7fc87a99d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8740c7f90 0x7fc8741590e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:13.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.735+0000 7fc87a99d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc874142c00 0x7fc874140070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:13.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.735+0000 7fc87a99d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8740b2f10 con 0x7fc8740c7f90 2023-10-06T02:23:13.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.735+0000 7fc87a99d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8740b2a20 con 0x7fc87406bd20 2023-10-06T02:23:13.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.735+0000 7fc87a99d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8740b02e0 con 0x7fc874142c00 2023-10-06T02:23:13.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.735+0000 7fc87919a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8740c7f90 0x7fc8741590e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:13.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.735+0000 7fc87919a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8740c7f90 0x7fc8741590e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44322/0 (socket says 172.21.15.31:44322) 2023-10-06T02:23:13.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.735+0000 7fc87919a700 1 -- 172.21.15.31:0/3393658862 learned_addr learned my addr 172.21.15.31:0/3393658862 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:13.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.735+0000 7fc87999b700 1 --2- 172.21.15.31:0/3393658862 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc87406bd20 0x7fc8740b24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:13.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.735+0000 7fc87a19c700 1 --2- 172.21.15.31:0/3393658862 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc874142c00 0x7fc874140070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:13.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.736+0000 7fc87919a700 1 -- 172.21.15.31:0/3393658862 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc874142c00 msgr2=0x7fc874140070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:13.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.736+0000 7fc87919a700 1 --2- 172.21.15.31:0/3393658862 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc874142c00 0x7fc874140070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:13.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.736+0000 7fc87919a700 1 -- 172.21.15.31:0/3393658862 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc87406bd20 msgr2=0x7fc8740b24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:13.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.736+0000 7fc87919a700 1 --2- 172.21.15.31:0/3393658862 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc87406bd20 0x7fc8740b24e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:13.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.736+0000 7fc87919a700 1 -- 172.21.15.31:0/3393658862 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc870009420 con 0x7fc8740c7f90 2023-10-06T02:23:13.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.736+0000 7fc87919a700 1 --2- 172.21.15.31:0/3393658862 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8740c7f90 0x7fc8741590e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc86400cb70 tx=0x7fc864001a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:13.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.736+0000 7fc869ffb700 1 -- 172.21.15.31:0/3393658862 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc864005920 con 0x7fc8740c7f90 2023-10-06T02:23:13.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.736+0000 7fc869ffb700 1 -- 172.21.15.31:0/3393658862 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc864005a80 con 0x7fc8740c7f90 2023-10-06T02:23:13.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.736+0000 7fc87a99d700 1 -- 172.21.15.31:0/3393658862 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8740c58d0 con 0x7fc8740c7f90 2023-10-06T02:23:13.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.736+0000 7fc869ffb700 1 -- 172.21.15.31:0/3393658862 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc864005e20 con 0x7fc8740c7f90 2023-10-06T02:23:13.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.736+0000 7fc87a99d700 1 -- 172.21.15.31:0/3393658862 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8740c5b70 con 0x7fc8740c7f90 2023-10-06T02:23:13.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.737+0000 7fc87a99d700 1 -- 172.21.15.31:0/3393658862 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc87404c3c0 con 0x7fc8740c7f90 2023-10-06T02:23:13.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.738+0000 7fc869ffb700 1 -- 172.21.15.31:0/3393658862 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc864007510 con 0x7fc8740c7f90 2023-10-06T02:23:13.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.739+0000 7fc869ffb700 1 --2- 172.21.15.31:0/3393658862 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc858066f30 0x7fc8580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:13.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.740+0000 7fc869ffb700 1 -- 172.21.15.31:0/3393658862 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc864084620 con 0x7fc8740c7f90 2023-10-06T02:23:13.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.740+0000 7fc87999b700 1 --2- 172.21.15.31:0/3393658862 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc858066f30 0x7fc8580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:13.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.740+0000 7fc87999b700 1 --2- 172.21.15.31:0/3393658862 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc858066f30 0x7fc8580693e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc8700093f0 tx=0x7fc870016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:13.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.744+0000 7fc869ffb700 1 -- 172.21.15.31:0/3393658862 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc864051710 con 0x7fc8740c7f90 2023-10-06T02:23:13.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.945+0000 7fc87a99d700 1 -- 172.21.15.31:0/3393658862 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc87406ad30 con 0x7fc8740c7f90 2023-10-06T02:23:13.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.946+0000 7fc869ffb700 1 -- 172.21.15.31:0/3393658862 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16818 (secure 0 0 0) 0x7fc864054d30 con 0x7fc8740c7f90 2023-10-06T02:23:13.948 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:23:13.948 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":23,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:23:10.970534+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:23:07.290269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T02:23:08.504679+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:23:13.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.948+0000 7fc87a99d700 1 -- 172.21.15.31:0/3393658862 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc858066f30 msgr2=0x7fc8580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:13.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.948+0000 7fc87a99d700 1 --2- 172.21.15.31:0/3393658862 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc858066f30 0x7fc8580693e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc8700093f0 tx=0x7fc870016000 comp rx=0 tx=0).stop 2023-10-06T02:23:13.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.948+0000 7fc87a99d700 1 -- 172.21.15.31:0/3393658862 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8740c7f90 msgr2=0x7fc8741590e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:13.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.948+0000 7fc87a99d700 1 --2- 172.21.15.31:0/3393658862 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8740c7f90 0x7fc8741590e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc86400cb70 tx=0x7fc864001a50 comp rx=0 tx=0).stop 2023-10-06T02:23:13.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.948+0000 7fc87a99d700 1 -- 172.21.15.31:0/3393658862 shutdown_connections 2023-10-06T02:23:13.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.948+0000 7fc87a99d700 1 --2- 172.21.15.31:0/3393658862 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc858066f30 0x7fc8580693e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:13.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.948+0000 7fc87a99d700 1 --2- 172.21.15.31:0/3393658862 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc874142c00 0x7fc874140070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:13.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.948+0000 7fc87a99d700 1 --2- 172.21.15.31:0/3393658862 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc87406bd20 0x7fc8740b24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:13.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.948+0000 7fc87a99d700 1 --2- 172.21.15.31:0/3393658862 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8740c7f90 0x7fc8741590e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:13.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.949+0000 7fc87a99d700 1 -- 172.21.15.31:0/3393658862 >> 172.21.15.31:0/3393658862 conn(0x7fc8740675a0 msgr2=0x7fc8740c2090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:13.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.949+0000 7fc87a99d700 1 -- 172.21.15.31:0/3393658862 shutdown_connections 2023-10-06T02:23:13.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:13.949+0000 7fc87a99d700 1 -- 172.21.15.31:0/3393658862 wait complete. 2023-10-06T02:23:13.967 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T02:23:13.968 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T02:23:14.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.154+0000 7fbe1e19d700 1 Processor -- start 2023-10-06T02:23:14.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.154+0000 7fbe1e19d700 1 -- start start 2023-10-06T02:23:14.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.155+0000 7fbe1e19d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbe18159640 0x7fbe1806d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:14.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.155+0000 7fbe1e19d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fbe18159a10 0x7fbe180ca7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:14.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.155+0000 7fbe1e19d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe180c7420 con 0x7fbe18070410 2023-10-06T02:23:14.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.155+0000 7fbe1e19d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe1806d5a0 con 0x7fbe18159a10 2023-10-06T02:23:14.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.155+0000 7fbe1e19d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1806d820 con 0x7fbe18159640 2023-10-06T02:23:14.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.155+0000 7fbe1c99a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fbe18159a10 0x7fbe180ca7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:14.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.155+0000 7fbe1d99c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fbe18070410 0x7fbe1806a2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56142/0 (socket says 172.21.15.31:56142) 2023-10-06T02:23:14.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.155+0000 7fbe1c99a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fbe18159a10 0x7fbe180ca7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44332/0 (socket says 172.21.15.31:44332) 2023-10-06T02:23:14.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.155+0000 7fbe1d99c700 1 -- 172.21.15.31:0/295468790 learned_addr learned my addr 172.21.15.31:0/295468790 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:23:14.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.156+0000 7fbe1d19b700 1 --2- 172.21.15.31:0/295468790 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbe18159640 0x7fbe1806d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:14.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.156+0000 7fbe1c99a700 1 -- 172.21.15.31:0/295468790 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbe18159640 msgr2=0x7fbe1806d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:14.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.156+0000 7fbe1c99a700 1 --2- 172.21.15.31:0/295468790 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbe18159640 0x7fbe1806d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:14.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.156+0000 7fbe1c99a700 1 -- 172.21.15.31:0/295468790 >> v1:172.21.15.31:6789/0 conn(0x7fbe18070410 legacy=0x7fbe1806a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:14.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.156+0000 7fbe1c99a700 1 -- 172.21.15.31:0/295468790 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe1806db00 con 0x7fbe18159a10 2023-10-06T02:23:14.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.156+0000 7fbe0ffff700 1 -- 172.21.15.31:0/295468790 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1206393785 0 0) 0x7fbe180c7420 con 0x7fbe18070410 2023-10-06T02:23:14.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.156+0000 7fbe1c99a700 1 --2- 172.21.15.31:0/295468790 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe18159a10 0x7fbe180ca7b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbe080098e0 tx=0x7fbe080091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=135f529460d031f3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:14.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.156+0000 7fbe0ffff700 1 -- 172.21.15.31:0/295468790 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe08009e80 con 0x7fbe18159a10 2023-10-06T02:23:14.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.156+0000 7fbe0ffff700 1 -- 172.21.15.31:0/295468790 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe0800b860 con 0x7fbe18159a10 2023-10-06T02:23:14.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.156+0000 7fbe0ffff700 1 -- 172.21.15.31:0/295468790 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe0800bbe0 con 0x7fbe18159a10 2023-10-06T02:23:14.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.157+0000 7fbe1e19d700 1 -- 172.21.15.31:0/295468790 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe18159a10 msgr2=0x7fbe180ca7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:14.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.157+0000 7fbe1e19d700 1 --2- 172.21.15.31:0/295468790 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe18159a10 0x7fbe180ca7b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbe080098e0 tx=0x7fbe080091a0 comp rx=0 tx=0).stop 2023-10-06T02:23:14.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.157+0000 7fbe1e19d700 1 -- 172.21.15.31:0/295468790 shutdown_connections 2023-10-06T02:23:14.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.157+0000 7fbe1e19d700 1 --2- 172.21.15.31:0/295468790 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbe18159640 0x7fbe1806d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:14.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.157+0000 7fbe1e19d700 1 --2- 172.21.15.31:0/295468790 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe18159a10 0x7fbe180ca7b0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:14.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.157+0000 7fbe1e19d700 1 -- 172.21.15.31:0/295468790 >> 172.21.15.31:0/295468790 conn(0x7fbe1806e720 msgr2=0x7fbe1806eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:14.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.157+0000 7fbe1e19d700 1 -- 172.21.15.31:0/295468790 shutdown_connections 2023-10-06T02:23:14.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.157+0000 7fbe1e19d700 1 -- 172.21.15.31:0/295468790 wait complete. 2023-10-06T02:23:14.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.159+0000 7fbe1e19d700 1 Processor -- start 2023-10-06T02:23:14.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.159+0000 7fbe1e19d700 1 -- start start 2023-10-06T02:23:14.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.160+0000 7fbe1e19d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe18070410 0x7fbe180c64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:14.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.160+0000 7fbe1e19d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbe18159640 0x7fbe180af7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:14.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.160+0000 7fbe1e19d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe18159a10 0x7fbe180b0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:14.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.160+0000 7fbe1e19d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe180afd30 con 0x7fbe18159a10 2023-10-06T02:23:14.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.160+0000 7fbe1e19d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe180b0b30 con 0x7fbe18070410 2023-10-06T02:23:14.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.160+0000 7fbe1e19d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe180b1500 con 0x7fbe18159640 2023-10-06T02:23:14.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.160+0000 7fbe1d99c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe18159a10 0x7fbe180b0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:14.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.160+0000 7fbe1d99c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe18159a10 0x7fbe180b0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44348/0 (socket says 172.21.15.31:44348) 2023-10-06T02:23:14.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.160+0000 7fbe1d99c700 1 -- 172.21.15.31:0/1573879339 learned_addr learned my addr 172.21.15.31:0/1573879339 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:14.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.160+0000 7fbe1d19b700 1 --2- 172.21.15.31:0/1573879339 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe18070410 0x7fbe180c64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:14.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.160+0000 7fbe1c99a700 1 --2- 172.21.15.31:0/1573879339 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbe18159640 0x7fbe180af7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:14.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.160+0000 7fbe1d99c700 1 -- 172.21.15.31:0/1573879339 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbe18159640 msgr2=0x7fbe180af7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:14.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.160+0000 7fbe1d99c700 1 --2- 172.21.15.31:0/1573879339 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbe18159640 0x7fbe180af7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:14.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.160+0000 7fbe1d99c700 1 -- 172.21.15.31:0/1573879339 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe18070410 msgr2=0x7fbe180c64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:14.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.160+0000 7fbe1d99c700 1 --2- 172.21.15.31:0/1573879339 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe18070410 0x7fbe180c64d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:14.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.160+0000 7fbe1d99c700 1 -- 172.21.15.31:0/1573879339 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe08009420 con 0x7fbe18159a10 2023-10-06T02:23:14.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.161+0000 7fbe1d99c700 1 --2- 172.21.15.31:0/1573879339 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe18159a10 0x7fbe180b0fc0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fbe1000cc90 tx=0x7fbe10001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:14.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.161+0000 7fbe0d7fa700 1 -- 172.21.15.31:0/1573879339 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe1000a8b0 con 0x7fbe18159a10 2023-10-06T02:23:14.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.161+0000 7fbe0d7fa700 1 -- 172.21.15.31:0/1573879339 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe1000aa10 con 0x7fbe18159a10 2023-10-06T02:23:14.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.161+0000 7fbe0d7fa700 1 -- 172.21.15.31:0/1573879339 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe1000add0 con 0x7fbe18159a10 2023-10-06T02:23:14.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.161+0000 7fbe1e19d700 1 -- 172.21.15.31:0/1573879339 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe180aff20 con 0x7fbe18159a10 2023-10-06T02:23:14.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.161+0000 7fbe1e19d700 1 -- 172.21.15.31:0/1573879339 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe180b04e0 con 0x7fbe18159a10 2023-10-06T02:23:14.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.162+0000 7fbe1e19d700 1 -- 172.21.15.31:0/1573879339 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe1804c3c0 con 0x7fbe18159a10 2023-10-06T02:23:14.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.163+0000 7fbe0d7fa700 1 -- 172.21.15.31:0/1573879339 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbe10007300 con 0x7fbe18159a10 2023-10-06T02:23:14.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.164+0000 7fbe0d7fa700 1 --2- 172.21.15.31:0/1573879339 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fbdfc066f30 0x7fbdfc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:14.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.164+0000 7fbe0d7fa700 1 -- 172.21.15.31:0/1573879339 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fbe10084a10 con 0x7fbe18159a10 2023-10-06T02:23:14.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.164+0000 7fbe1d19b700 1 --2- 172.21.15.31:0/1573879339 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fbdfc066f30 0x7fbdfc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:14.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.165+0000 7fbe1d19b700 1 --2- 172.21.15.31:0/1573879339 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fbdfc066f30 0x7fbdfc0693e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbe14006fd0 tx=0x7fbe14008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:14.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.168+0000 7fbe0d7fa700 1 -- 172.21.15.31:0/1573879339 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbe10051b00 con 0x7fbe18159a10 2023-10-06T02:23:14.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.365+0000 7fbe1e19d700 1 -- 172.21.15.31:0/1573879339 --> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbe180cb2f0 con 0x7fbdfc066f30 2023-10-06T02:23:14.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.375+0000 7fbe0d7fa700 1 -- 172.21.15.31:0/1573879339 <== mgr.4107 v2:172.21.15.31:6832/85292 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+275878 (secure 0 0 0) 0x7fbe180cb2f0 con 0x7fbdfc066f30 2023-10-06T02:23:14.380 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:23:14.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.379+0000 7fbe1e19d700 1 -- 172.21.15.31:0/1573879339 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fbdfc066f30 msgr2=0x7fbdfc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:14.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.379+0000 7fbe1e19d700 1 --2- 172.21.15.31:0/1573879339 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fbdfc066f30 0x7fbdfc0693e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbe14006fd0 tx=0x7fbe14008040 comp rx=0 tx=0).stop 2023-10-06T02:23:14.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.379+0000 7fbe1e19d700 1 -- 172.21.15.31:0/1573879339 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe18159a10 msgr2=0x7fbe180b0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:14.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.379+0000 7fbe1e19d700 1 --2- 172.21.15.31:0/1573879339 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe18159a10 0x7fbe180b0fc0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fbe1000cc90 tx=0x7fbe10001a80 comp rx=0 tx=0).stop 2023-10-06T02:23:14.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.379+0000 7fbe1e19d700 1 -- 172.21.15.31:0/1573879339 shutdown_connections 2023-10-06T02:23:14.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.379+0000 7fbe1e19d700 1 --2- 172.21.15.31:0/1573879339 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe18159a10 0x7fbe180b0fc0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:14.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.379+0000 7fbe1e19d700 1 --2- 172.21.15.31:0/1573879339 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe18070410 0x7fbe180c64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:14.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.379+0000 7fbe1e19d700 1 --2- 172.21.15.31:0/1573879339 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fbdfc066f30 0x7fbdfc0693e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:14.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.379+0000 7fbe1e19d700 1 --2- 172.21.15.31:0/1573879339 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbe18159640 0x7fbe180af7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:14.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.379+0000 7fbe1e19d700 1 -- 172.21.15.31:0/1573879339 >> 172.21.15.31:0/1573879339 conn(0x7fbe1806e720 msgr2=0x7fbe18156a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:14.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.379+0000 7fbe1e19d700 1 -- 172.21.15.31:0/1573879339 shutdown_connections 2023-10-06T02:23:14.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.379+0000 7fbe1e19d700 1 -- 172.21.15.31:0/1573879339 wait complete. 2023-10-06T02:23:14.383 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-10-06T02:23:14.397 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2023-10-06T02:23:13.117997+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2380968,"kb_used_data":2664,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752593752,"statfs":{"total":773094113280,"available":770656002048,"internally_reserved":0,"allocated":2727936,"data_stored":1683104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.103923"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:23:08.003076+0000","last_change":"2023-10-06T02:23:08.022651+0000","last_active":"2023-10-06T02:23:08.003076+0000","last_peered":"2023-10-06T02:23:08.003076+0000","last_clean":"2023-10-06T02:23:08.003076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:08.003076+0000","last_undegraded":"2023-10-06T02:23:08.003076+0000","last_fullsized":"2023-10-06T02:23:08.003076+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:07:19.307413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:23:08.003076+0000","last_change":"2023-10-06T02:23:08.015353+0000","last_active":"2023-10-06T02:23:08.003076+0000","last_peered":"2023-10-06T02:23:08.003076+0000","last_clean":"2023-10-06T02:23:08.003076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:08.003076+0000","last_undegraded":"2023-10-06T02:23:08.003076+0000","last_fullsized":"2023-10-06T02:23:08.003076+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:44:41.034030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:23:08.003076+0000","last_change":"2023-10-06T02:23:08.020162+0000","last_active":"2023-10-06T02:23:08.003076+0000","last_peered":"2023-10-06T02:23:08.003076+0000","last_clean":"2023-10-06T02:23:08.003076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:08.003076+0000","last_undegraded":"2023-10-06T02:23:08.003076+0000","last_fullsized":"2023-10-06T02:23:08.003076+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:07:56.722823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:23:08.003076+0000","last_change":"2023-10-06T02:23:08.023620+0000","last_active":"2023-10-06T02:23:08.003076+0000","last_peered":"2023-10-06T02:23:08.003076+0000","last_clean":"2023-10-06T02:23:08.003076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:08.003076+0000","last_undegraded":"2023-10-06T02:23:08.003076+0000","last_fullsized":"2023-10-06T02:23:08.003076+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:34:42.916959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:23:08.003076+0000","last_change":"2023-10-06T02:23:08.014971+0000","last_active":"2023-10-06T02:23:08.003076+0000","last_peered":"2023-10-06T02:23:08.003076+0000","last_clean":"2023-10-06T02:23:08.003076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:08.003076+0000","last_undegraded":"2023-10-06T02:23:08.003076+0000","last_fullsized":"2023-10-06T02:23:08.003076+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:58.976414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:23:08.003076+0000","last_change":"2023-10-06T02:23:08.018899+0000","last_active":"2023-10-06T02:23:08.003076+0000","last_peered":"2023-10-06T02:23:08.003076+0000","last_clean":"2023-10-06T02:23:08.003076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:08.003076+0000","last_undegraded":"2023-10-06T02:23:08.003076+0000","last_fullsized":"2023-10-06T02:23:08.003076+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:57:40.559177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:23:08.003076+0000","last_change":"2023-10-06T02:23:08.022152+0000","last_active":"2023-10-06T02:23:08.003076+0000","last_peered":"2023-10-06T02:23:08.003076+0000","last_clean":"2023-10-06T02:23:08.003076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:08.003076+0000","last_undegraded":"2023-10-06T02:23:08.003076+0000","last_fullsized":"2023-10-06T02:23:08.003076+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:34:42.916959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:23:08.003076+0000","last_change":"2023-10-06T02:23:08.018330+0000","last_active":"2023-10-06T02:23:08.003076+0000","last_peered":"2023-10-06T02:23:08.003076+0000","last_clean":"2023-10-06T02:23:08.003076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:08.003076+0000","last_undegraded":"2023-10-06T02:23:08.003076+0000","last_fullsized":"2023-10-06T02:23:08.003076+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:57.514051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:23:08.003076+0000","last_change":"2023-10-06T02:23:08.022963+0000","last_active":"2023-10-06T02:23:08.003076+0000","last_peered":"2023-10-06T02:23:08.003076+0000","last_clean":"2023-10-06T02:23:08.003076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:08.003076+0000","last_undegraded":"2023-10-06T02:23:08.003076+0000","last_fullsized":"2023-10-06T02:23:08.003076+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:44:41.034030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:23:08.003076+0000","last_change":"2023-10-06T02:23:08.018090+0000","last_active":"2023-10-06T02:23:08.003076+0000","last_peered":"2023-10-06T02:23:08.003076+0000","last_clean":"2023-10-06T02:23:08.003076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:08.003076+0000","last_undegraded":"2023-10-06T02:23:08.003076+0000","last_fullsized":"2023-10-06T02:23:08.003076+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:17:32.434412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:23:08.003076+0000","last_change":"2023-10-06T02:23:08.023612+0000","last_active":"2023-10-06T02:23:08.003076+0000","last_peered":"2023-10-06T02:23:08.003076+0000","last_clean":"2023-10-06T02:23:08.003076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:08.003076+0000","last_undegraded":"2023-10-06T02:23:08.003076+0000","last_fullsized":"2023-10-06T02:23:08.003076+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:44:41.034030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:23:08.003076+0000","last_change":"2023-10-06T02:23:08.018175+0000","last_active":"2023-10-06T02:23:08.003076+0000","last_peered":"2023-10-06T02:23:08.003076+0000","last_clean":"2023-10-06T02:23:08.003076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:08.003076+0000","last_undegraded":"2023-10-06T02:23:08.003076+0000","last_fullsized":"2023-10-06T02:23:08.003076+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:36:27.554258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:23:08.003076+0000","last_change":"2023-10-06T02:23:08.018151+0000","last_active":"2023-10-06T02:23:08.003076+0000","last_peered":"2023-10-06T02:23:08.003076+0000","last_clean":"2023-10-06T02:23:08.003076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:08.003076+0000","last_undegraded":"2023-10-06T02:23:08.003076+0000","last_fullsized":"2023-10-06T02:23:08.003076+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:07:56.722823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:23:08.003076+0000","last_change":"2023-10-06T02:23:08.018149+0000","last_active":"2023-10-06T02:23:08.003076+0000","last_peered":"2023-10-06T02:23:08.003076+0000","last_clean":"2023-10-06T02:23:08.003076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:08.003076+0000","last_undegraded":"2023-10-06T02:23:08.003076+0000","last_fullsized":"2023-10-06T02:23:08.003076+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:44:15.545672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:23:08.003076+0000","last_change":"2023-10-06T02:23:08.014974+0000","last_active":"2023-10-06T02:23:08.003076+0000","last_peered":"2023-10-06T02:23:08.003076+0000","last_clean":"2023-10-06T02:23:08.003076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:08.003076+0000","last_undegraded":"2023-10-06T02:23:08.003076+0000","last_fullsized":"2023-10-06T02:23:08.003076+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:38.263723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:23:08.003076+0000","last_change":"2023-10-06T02:23:08.019362+0000","last_active":"2023-10-06T02:23:08.003076+0000","last_peered":"2023-10-06T02:23:08.003076+0000","last_clean":"2023-10-06T02:23:08.003076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:08.003076+0000","last_undegraded":"2023-10-06T02:23:08.003076+0000","last_fullsized":"2023-10-06T02:23:08.003076+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:17:32.434412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:23:08.003076+0000","last_change":"2023-10-06T02:23:08.023679+0000","last_active":"2023-10-06T02:23:08.003076+0000","last_peered":"2023-10-06T02:23:08.003076+0000","last_clean":"2023-10-06T02:23:08.003076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:08.003076+0000","last_undegraded":"2023-10-06T02:23:08.003076+0000","last_fullsized":"2023-10-06T02:23:08.003076+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:34:42.916959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:23:08.003076+0000","last_change":"2023-10-06T02:23:08.022582+0000","last_active":"2023-10-06T02:23:08.003076+0000","last_peered":"2023-10-06T02:23:08.003076+0000","last_clean":"2023-10-06T02:23:08.003076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:08.003076+0000","last_undegraded":"2023-10-06T02:23:08.003076+0000","last_fullsized":"2023-10-06T02:23:08.003076+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:44:41.034030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:23:08.003076+0000","last_change":"2023-10-06T02:23:08.019257+0000","last_active":"2023-10-06T02:23:08.003076+0000","last_peered":"2023-10-06T02:23:08.003076+0000","last_clean":"2023-10-06T02:23:08.003076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:08.003076+0000","last_undegraded":"2023-10-06T02:23:08.003076+0000","last_fullsized":"2023-10-06T02:23:08.003076+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:38.263723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T02:23:04.712407+0000","last_change":"2023-10-06T02:23:02.054039+0000","last_active":"2023-10-06T02:23:04.712407+0000","last_peered":"2023-10-06T02:23:04.712407+0000","last_clean":"2023-10-06T02:23:04.712407+0000","last_became_active":"2023-10-06T02:23:02.053742+0000","last_became_peered":"2023-10-06T02:23:02.053742+0000","last_unstale":"2023-10-06T02:23:04.712407+0000","last_undegraded":"2023-10-06T02:23:04.712407+0000","last_fullsized":"2023-10-06T02:23:04.712407+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:58:18.846475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:23:08.003076+0000","last_change":"2023-10-06T02:23:08.014835+0000","last_active":"2023-10-06T02:23:08.003076+0000","last_peered":"2023-10-06T02:23:08.003076+0000","last_clean":"2023-10-06T02:23:08.003076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:08.003076+0000","last_undegraded":"2023-10-06T02:23:08.003076+0000","last_fullsized":"2023-10-06T02:23:08.003076+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:41.039005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T02:23:02.992320+0000","last_change":"2023-10-06T02:23:01.988631+0000","last_active":"2023-10-06T02:23:02.992320+0000","last_peered":"2023-10-06T02:23:02.992320+0000","last_clean":"2023-10-06T02:23:02.992320+0000","last_became_active":"2023-10-06T02:23:01.988303+0000","last_became_peered":"2023-10-06T02:23:01.988303+0000","last_unstale":"2023-10-06T02:23:02.992320+0000","last_undegraded":"2023-10-06T02:23:02.992320+0000","last_fullsized":"2023-10-06T02:23:02.992320+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:38.211466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:23:08.003076+0000","last_change":"2023-10-06T02:23:08.020989+0000","last_active":"2023-10-06T02:23:08.003076+0000","last_peered":"2023-10-06T02:23:08.003076+0000","last_clean":"2023-10-06T02:23:08.003076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:08.003076+0000","last_undegraded":"2023-10-06T02:23:08.003076+0000","last_fullsized":"2023-10-06T02:23:08.003076+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:35:05.782473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T02:23:04.712320+0000","last_change":"2023-10-06T02:23:02.054338+0000","last_active":"2023-10-06T02:23:04.712320+0000","last_peered":"2023-10-06T02:23:04.712320+0000","last_clean":"2023-10-06T02:23:04.712320+0000","last_became_active":"2023-10-06T02:23:02.053991+0000","last_became_peered":"2023-10-06T02:23:02.053991+0000","last_unstale":"2023-10-06T02:23:04.712320+0000","last_undegraded":"2023-10-06T02:23:04.712320+0000","last_fullsized":"2023-10-06T02:23:04.712320+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:39:30.192838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:23:08.003076+0000","last_change":"2023-10-06T02:23:08.015382+0000","last_active":"2023-10-06T02:23:08.003076+0000","last_peered":"2023-10-06T02:23:08.003076+0000","last_clean":"2023-10-06T02:23:08.003076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:08.003076+0000","last_undegraded":"2023-10-06T02:23:08.003076+0000","last_fullsized":"2023-10-06T02:23:08.003076+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:34:42.916959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T02:23:02.992302+0000","last_change":"2023-10-06T02:23:01.988140+0000","last_active":"2023-10-06T02:23:02.992302+0000","last_peered":"2023-10-06T02:23:02.992302+0000","last_clean":"2023-10-06T02:23:02.992302+0000","last_became_active":"2023-10-06T02:23:01.987769+0000","last_became_peered":"2023-10-06T02:23:01.987769+0000","last_unstale":"2023-10-06T02:23:02.992302+0000","last_undegraded":"2023-10-06T02:23:02.992302+0000","last_fullsized":"2023-10-06T02:23:02.992302+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:20:08.391912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":163,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T02:23:03.813375+0000","last_change":"2023-10-06T02:22:55.032519+0000","last_active":"2023-10-06T02:23:03.813375+0000","last_peered":"2023-10-06T02:23:03.813375+0000","last_clean":"2023-10-06T02:23:03.813375+0000","last_became_active":"2023-10-06T02:22:55.031367+0000","last_became_peered":"2023-10-06T02:22:55.031367+0000","last_unstale":"2023-10-06T02:23:03.813375+0000","last_undegraded":"2023-10-06T02:23:03.813375+0000","last_fullsized":"2023-10-06T02:23:03.813375+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:22:53.309097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:22:53.309097+0000","last_clean_scrub_stamp":"2023-10-06T02:22:53.309097+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:53:46.520909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:23:08.003076+0000","last_change":"2023-10-06T02:23:08.018672+0000","last_active":"2023-10-06T02:23:08.003076+0000","last_peered":"2023-10-06T02:23:08.003076+0000","last_clean":"2023-10-06T02:23:08.003076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:08.003076+0000","last_undegraded":"2023-10-06T02:23:08.003076+0000","last_fullsized":"2023-10-06T02:23:08.003076+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:34:42.916959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T02:23:03.813389+0000","last_change":"2023-10-06T02:23:01.990773+0000","last_active":"2023-10-06T02:23:03.813389+0000","last_peered":"2023-10-06T02:23:03.813389+0000","last_clean":"2023-10-06T02:23:03.813389+0000","last_became_active":"2023-10-06T02:23:01.990511+0000","last_became_peered":"2023-10-06T02:23:01.990511+0000","last_unstale":"2023-10-06T02:23:03.813389+0000","last_undegraded":"2023-10-06T02:23:03.813389+0000","last_fullsized":"2023-10-06T02:23:03.813389+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:42:54.797731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:23:08.003076+0000","last_change":"2023-10-06T02:23:08.023749+0000","last_active":"2023-10-06T02:23:08.003076+0000","last_peered":"2023-10-06T02:23:08.003076+0000","last_clean":"2023-10-06T02:23:08.003076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:08.003076+0000","last_undegraded":"2023-10-06T02:23:08.003076+0000","last_fullsized":"2023-10-06T02:23:08.003076+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:07:19.307413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T02:23:03.813384+0000","last_change":"2023-10-06T02:23:02.054069+0000","last_active":"2023-10-06T02:23:03.813384+0000","last_peered":"2023-10-06T02:23:03.813384+0000","last_clean":"2023-10-06T02:23:03.813384+0000","last_became_active":"2023-10-06T02:23:02.053739+0000","last_became_peered":"2023-10-06T02:23:02.053739+0000","last_unstale":"2023-10-06T02:23:03.813384+0000","last_undegraded":"2023-10-06T02:23:03.813384+0000","last_fullsized":"2023-10-06T02:23:03.813384+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:57:48.822608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:23:08.003076+0000","last_change":"2023-10-06T02:23:08.015425+0000","last_active":"2023-10-06T02:23:08.003076+0000","last_peered":"2023-10-06T02:23:08.003076+0000","last_clean":"2023-10-06T02:23:08.003076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:08.003076+0000","last_undegraded":"2023-10-06T02:23:08.003076+0000","last_fullsized":"2023-10-06T02:23:08.003076+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:41.039005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:23:08.003076+0000","last_change":"2023-10-06T02:23:08.019208+0000","last_active":"2023-10-06T02:23:08.003076+0000","last_peered":"2023-10-06T02:23:08.003076+0000","last_clean":"2023-10-06T02:23:08.003076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:08.003076+0000","last_undegraded":"2023-10-06T02:23:08.003076+0000","last_fullsized":"2023-10-06T02:23:08.003076+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:41.039005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T02:23:04.712344+0000","last_change":"2023-10-06T02:23:01.989934+0000","last_active":"2023-10-06T02:23:04.712344+0000","last_peered":"2023-10-06T02:23:04.712344+0000","last_clean":"2023-10-06T02:23:04.712344+0000","last_became_active":"2023-10-06T02:23:01.989706+0000","last_became_peered":"2023-10-06T02:23:01.989706+0000","last_unstale":"2023-10-06T02:23:04.712344+0000","last_undegraded":"2023-10-06T02:23:04.712344+0000","last_fullsized":"2023-10-06T02:23:04.712344+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:06.212251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T02:23:04.712944+0000","last_change":"2023-10-06T02:23:01.992935+0000","last_active":"2023-10-06T02:23:04.712944+0000","last_peered":"2023-10-06T02:23:04.712944+0000","last_clean":"2023-10-06T02:23:04.712944+0000","last_became_active":"2023-10-06T02:23:01.992568+0000","last_became_peered":"2023-10-06T02:23:01.992568+0000","last_unstale":"2023-10-06T02:23:04.712944+0000","last_undegraded":"2023-10-06T02:23:04.712944+0000","last_fullsized":"2023-10-06T02:23:04.712944+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:06.212251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:23:08.003076+0000","last_change":"2023-10-06T02:23:08.023179+0000","last_active":"2023-10-06T02:23:08.003076+0000","last_peered":"2023-10-06T02:23:08.003076+0000","last_clean":"2023-10-06T02:23:08.003076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:08.003076+0000","last_undegraded":"2023-10-06T02:23:08.003076+0000","last_fullsized":"2023-10-06T02:23:08.003076+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:41.039005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:23:08.003076+0000","last_change":"2023-10-06T02:23:08.023091+0000","last_active":"2023-10-06T02:23:08.003076+0000","last_peered":"2023-10-06T02:23:08.003076+0000","last_clean":"2023-10-06T02:23:08.003076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:08.003076+0000","last_undegraded":"2023-10-06T02:23:08.003076+0000","last_fullsized":"2023-10-06T02:23:08.003076+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:58.976414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:23:08.003076+0000","last_change":"2023-10-06T02:23:08.015775+0000","last_active":"2023-10-06T02:23:08.003076+0000","last_peered":"2023-10-06T02:23:08.003076+0000","last_clean":"2023-10-06T02:23:08.003076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:08.003076+0000","last_undegraded":"2023-10-06T02:23:08.003076+0000","last_fullsized":"2023-10-06T02:23:08.003076+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:07:56.722823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:23:08.003076+0000","last_change":"2023-10-06T02:23:08.015831+0000","last_active":"2023-10-06T02:23:08.003076+0000","last_peered":"2023-10-06T02:23:08.003076+0000","last_clean":"2023-10-06T02:23:08.003076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:08.003076+0000","last_undegraded":"2023-10-06T02:23:08.003076+0000","last_fullsized":"2023-10-06T02:23:08.003076+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:17:32.434412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:23:08.003076+0000","last_change":"2023-10-06T02:23:08.019223+0000","last_active":"2023-10-06T02:23:08.003076+0000","last_peered":"2023-10-06T02:23:08.003076+0000","last_clean":"2023-10-06T02:23:08.003076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:08.003076+0000","last_undegraded":"2023-10-06T02:23:08.003076+0000","last_fullsized":"2023-10-06T02:23:08.003076+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:58.976414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:23:08.003076+0000","last_change":"2023-10-06T02:23:08.024513+0000","last_active":"2023-10-06T02:23:08.003076+0000","last_peered":"2023-10-06T02:23:08.003076+0000","last_clean":"2023-10-06T02:23:08.003076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:08.003076+0000","last_undegraded":"2023-10-06T02:23:08.003076+0000","last_fullsized":"2023-10-06T02:23:08.003076+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:17:32.434412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298056,"kb_used_data":768,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073784,"statfs":{"total":96636764160,"available":96331554816,"internally_reserved":0,"allocated":786432,"data_stored":653164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297476,"kb_used_data":188,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074364,"statfs":{"total":96636764160,"available":96332148736,"internally_reserved":0,"allocated":192512,"data_stored":62796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86199999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74299999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297476,"kb_used_data":188,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074364,"statfs":{"total":96636764160,"available":96332148736,"internally_reserved":0,"allocated":192512,"data_stored":62796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0409999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.014}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1299999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0049999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298056,"kb_used_data":768,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073784,"statfs":{"total":96636764160,"available":96331554816,"internally_reserved":0,"allocated":786432,"data_stored":653164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84599999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0309999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99199999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96099999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297476,"kb_used_data":188,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074364,"statfs":{"total":96636764160,"available":96332148736,"internally_reserved":0,"allocated":192512,"data_stored":62796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35099999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78900000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297476,"kb_used_data":188,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074364,"statfs":{"total":96636764160,"available":96332148736,"internally_reserved":0,"allocated":192512,"data_stored":62796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66500000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297476,"kb_used_data":188,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074364,"statfs":{"total":96636764160,"available":96332148736,"internally_reserved":0,"allocated":192512,"data_stored":62796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71499999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83199999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63300000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297476,"kb_used_data":188,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074364,"statfs":{"total":96636764160,"available":96332148736,"internally_reserved":0,"allocated":192512,"data_stored":62796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77900000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91400000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.024}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T02:23:14.405 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T02:23:14.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.623+0000 7f621f19d700 1 Processor -- start 2023-10-06T02:23:14.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.623+0000 7f621f19d700 1 -- start start 2023-10-06T02:23:14.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.625+0000 7f621f19d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f621815dfd0 0x7f62180aedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:14.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.625+0000 7f621f19d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62180acc00 con 0x7f6218159920 2023-10-06T02:23:14.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.625+0000 7f621f19d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62180abff0 con 0x7f6218159550 2023-10-06T02:23:14.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.625+0000 7f621f19d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62180ac2d0 con 0x7f621815dfd0 2023-10-06T02:23:14.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.625+0000 7f621d99a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f621815dfd0 0x7f62180aedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:14.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.625+0000 7f621d99a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f621815dfd0 0x7f62180aedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44360/0 (socket says 172.21.15.31:44360) 2023-10-06T02:23:14.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.625+0000 7f621d99a700 1 -- 172.21.15.31:0/2795770796 learned_addr learned my addr 172.21.15.31:0/2795770796 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:14.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.625+0000 7f621e99c700 1 --1- 172.21.15.31:0/2795770796 >> v1:172.21.15.31:6789/0 conn(0x7f6218159920 0x7f62180ad090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56154/0 (socket says 172.21.15.31:56154) 2023-10-06T02:23:14.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.625+0000 7f621d99a700 1 -- 172.21.15.31:0/2795770796 >> v1:172.21.15.62:6789/0 conn(0x7f6218159550 legacy=0x7f62180ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:14.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.625+0000 7f621d99a700 1 -- 172.21.15.31:0/2795770796 >> v1:172.21.15.31:6789/0 conn(0x7f6218159920 legacy=0x7f62180ad090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:14.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.625+0000 7f621d99a700 1 -- 172.21.15.31:0/2795770796 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62180ac5b0 con 0x7f621815dfd0 2023-10-06T02:23:14.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.625+0000 7f621d199700 1 -- 172.21.15.31:0/2795770796 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3615351993 0 0) 0x7f62180acc00 con 0x7f6218159920 2023-10-06T02:23:14.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.626+0000 7f621d99a700 1 --2- 172.21.15.31:0/2795770796 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f621815dfd0 0x7f62180aedb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f62080099b0 tx=0x7f6208009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c675dd9e2e7d45bb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:14.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.626+0000 7f621d199700 1 -- 172.21.15.31:0/2795770796 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f620800b820 con 0x7f621815dfd0 2023-10-06T02:23:14.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.626+0000 7f621d199700 1 -- 172.21.15.31:0/2795770796 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f620800b980 con 0x7f621815dfd0 2023-10-06T02:23:14.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.626+0000 7f621d199700 1 -- 172.21.15.31:0/2795770796 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f620800bd20 con 0x7f621815dfd0 2023-10-06T02:23:14.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.626+0000 7f621f19d700 1 -- 172.21.15.31:0/2795770796 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f621815dfd0 msgr2=0x7f62180aedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:14.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.626+0000 7f621f19d700 1 --2- 172.21.15.31:0/2795770796 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f621815dfd0 0x7f62180aedb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f62080099b0 tx=0x7f6208009250 comp rx=0 tx=0).stop 2023-10-06T02:23:14.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.626+0000 7f621f19d700 1 -- 172.21.15.31:0/2795770796 shutdown_connections 2023-10-06T02:23:14.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.626+0000 7f621f19d700 1 --2- 172.21.15.31:0/2795770796 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f621815dfd0 0x7f62180aedb0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:14.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.626+0000 7f621f19d700 1 -- 172.21.15.31:0/2795770796 >> 172.21.15.31:0/2795770796 conn(0x7f621806ce80 msgr2=0x7f62180ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:14.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.627+0000 7f621f19d700 1 -- 172.21.15.31:0/2795770796 shutdown_connections 2023-10-06T02:23:14.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.627+0000 7f621f19d700 1 -- 172.21.15.31:0/2795770796 wait complete. 2023-10-06T02:23:14.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.628+0000 7f621f19d700 1 Processor -- start 2023-10-06T02:23:14.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.628+0000 7f621f19d700 1 -- start start 2023-10-06T02:23:14.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.629+0000 7f621f19d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6218159550 0x7f621816a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:14.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.629+0000 7f621f19d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6218159920 0x7f621814ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:14.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.629+0000 7f621f19d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f621815dfd0 0x7f62180b2140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:14.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.629+0000 7f621f19d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62180c8730 con 0x7f6218159920 2023-10-06T02:23:14.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.629+0000 7f621f19d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f621816ae90 con 0x7f6218159550 2023-10-06T02:23:14.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.629+0000 7f621f19d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6218150a30 con 0x7f621815dfd0 2023-10-06T02:23:14.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.629+0000 7f621d99a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6218159920 0x7f621814ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:14.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.629+0000 7f621d99a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6218159920 0x7f621814ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44366/0 (socket says 172.21.15.31:44366) 2023-10-06T02:23:14.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.629+0000 7f621d99a700 1 -- 172.21.15.31:0/2255291488 learned_addr learned my addr 172.21.15.31:0/2255291488 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:14.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.630+0000 7f621d99a700 1 -- 172.21.15.31:0/2255291488 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f621815dfd0 msgr2=0x7f62180b2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:14.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.630+0000 7f621d99a700 1 --2- 172.21.15.31:0/2255291488 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f621815dfd0 0x7f62180b2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:14.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.630+0000 7f621d99a700 1 -- 172.21.15.31:0/2255291488 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6218159550 msgr2=0x7f621816a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:14.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.630+0000 7f621d99a700 1 --2- 172.21.15.31:0/2255291488 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6218159550 0x7f621816a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:14.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.630+0000 7f621d99a700 1 -- 172.21.15.31:0/2255291488 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62080094f0 con 0x7f6218159920 2023-10-06T02:23:14.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.630+0000 7f621d99a700 1 --2- 172.21.15.31:0/2255291488 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6218159920 0x7f621814ce20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6208009290 tx=0x7f6208016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:14.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.630+0000 7f620e7fc700 1 -- 172.21.15.31:0/2255291488 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f620800bc40 con 0x7f6218159920 2023-10-06T02:23:14.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.630+0000 7f620e7fc700 1 -- 172.21.15.31:0/2255291488 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6208020070 con 0x7f6218159920 2023-10-06T02:23:14.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.630+0000 7f621f19d700 1 -- 172.21.15.31:0/2255291488 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62180c7b20 con 0x7f6218159920 2023-10-06T02:23:14.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.630+0000 7f620e7fc700 1 -- 172.21.15.31:0/2255291488 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f620801b6a0 con 0x7f6218159920 2023-10-06T02:23:14.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.630+0000 7f621f19d700 1 -- 172.21.15.31:0/2255291488 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62180c80e0 con 0x7f6218159920 2023-10-06T02:23:14.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.631+0000 7f621f19d700 1 -- 172.21.15.31:0/2255291488 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f621804c3c0 con 0x7f6218159920 2023-10-06T02:23:14.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.632+0000 7f620e7fc700 1 -- 172.21.15.31:0/2255291488 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6208022030 con 0x7f6218159920 2023-10-06T02:23:14.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.634+0000 7f620e7fc700 1 --2- 172.21.15.31:0/2255291488 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f61fc066f30 0x7f61fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:14.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.634+0000 7f620e7fc700 1 -- 172.21.15.31:0/2255291488 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6208086620 con 0x7f6218159920 2023-10-06T02:23:14.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.634+0000 7f621e19b700 1 --2- 172.21.15.31:0/2255291488 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f61fc066f30 0x7f61fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:14.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.634+0000 7f621e19b700 1 --2- 172.21.15.31:0/2255291488 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f61fc066f30 0x7f61fc0693e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6214007fd0 tx=0x7f6214007b20 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:14.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.638+0000 7f620e7fc700 1 -- 172.21.15.31:0/2255291488 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6208053690 con 0x7f6218159920 2023-10-06T02:23:14.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.838+0000 7f621f19d700 1 -- 172.21.15.31:0/2255291488 --> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f621804c3c0 con 0x7f61fc066f30 2023-10-06T02:23:14.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.846+0000 7f620e7fc700 1 -- 172.21.15.31:0/2255291488 <== mgr.4107 v2:172.21.15.31:6832/85292 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+275878 (secure 0 0 0) 0x7f621804c3c0 con 0x7f61fc066f30 2023-10-06T02:23:14.851 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:23:14.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.850+0000 7f621f19d700 1 -- 172.21.15.31:0/2255291488 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f61fc066f30 msgr2=0x7f61fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:14.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.850+0000 7f621f19d700 1 --2- 172.21.15.31:0/2255291488 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f61fc066f30 0x7f61fc0693e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6214007fd0 tx=0x7f6214007b20 comp rx=0 tx=0).stop 2023-10-06T02:23:14.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.850+0000 7f621f19d700 1 -- 172.21.15.31:0/2255291488 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6218159920 msgr2=0x7f621814ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:14.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.850+0000 7f621f19d700 1 --2- 172.21.15.31:0/2255291488 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6218159920 0x7f621814ce20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6208009290 tx=0x7f6208016000 comp rx=0 tx=0).stop 2023-10-06T02:23:14.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.851+0000 7f621f19d700 1 -- 172.21.15.31:0/2255291488 shutdown_connections 2023-10-06T02:23:14.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.851+0000 7f621f19d700 1 --2- 172.21.15.31:0/2255291488 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f61fc066f30 0x7f61fc0693e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:14.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.851+0000 7f621f19d700 1 --2- 172.21.15.31:0/2255291488 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f621815dfd0 0x7f62180b2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:14.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.851+0000 7f621f19d700 1 --2- 172.21.15.31:0/2255291488 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6218159550 0x7f621816a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:14.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.851+0000 7f621f19d700 1 --2- 172.21.15.31:0/2255291488 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6218159920 0x7f621814ce20 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:14.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.851+0000 7f621f19d700 1 -- 172.21.15.31:0/2255291488 >> 172.21.15.31:0/2255291488 conn(0x7f621806ce80 msgr2=0x7f621806e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:14.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.851+0000 7f621f19d700 1 -- 172.21.15.31:0/2255291488 shutdown_connections 2023-10-06T02:23:14.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:14.851+0000 7f621f19d700 1 -- 172.21.15.31:0/2255291488 wait complete. 2023-10-06T02:23:14.854 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-10-06T02:23:14.869 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2023-10-06T02:23:13.117997+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2380968,"kb_used_data":2664,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752593752,"statfs":{"total":773094113280,"available":770656002048,"internally_reserved":0,"allocated":2727936,"data_stored":1683104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.103923"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:23:08.003076+0000","last_change":"2023-10-06T02:23:08.022651+0000","last_active":"2023-10-06T02:23:08.003076+0000","last_peered":"2023-10-06T02:23:08.003076+0000","last_clean":"2023-10-06T02:23:08.003076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:08.003076+0000","last_undegraded":"2023-10-06T02:23:08.003076+0000","last_fullsized":"2023-10-06T02:23:08.003076+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:07:19.307413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:23:08.003076+0000","last_change":"2023-10-06T02:23:08.015353+0000","last_active":"2023-10-06T02:23:08.003076+0000","last_peered":"2023-10-06T02:23:08.003076+0000","last_clean":"2023-10-06T02:23:08.003076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:08.003076+0000","last_undegraded":"2023-10-06T02:23:08.003076+0000","last_fullsized":"2023-10-06T02:23:08.003076+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:44:41.034030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:23:08.003076+0000","last_change":"2023-10-06T02:23:08.020162+0000","last_active":"2023-10-06T02:23:08.003076+0000","last_peered":"2023-10-06T02:23:08.003076+0000","last_clean":"2023-10-06T02:23:08.003076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:08.003076+0000","last_undegraded":"2023-10-06T02:23:08.003076+0000","last_fullsized":"2023-10-06T02:23:08.003076+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:07:56.722823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:23:08.003076+0000","last_change":"2023-10-06T02:23:08.023620+0000","last_active":"2023-10-06T02:23:08.003076+0000","last_peered":"2023-10-06T02:23:08.003076+0000","last_clean":"2023-10-06T02:23:08.003076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:08.003076+0000","last_undegraded":"2023-10-06T02:23:08.003076+0000","last_fullsized":"2023-10-06T02:23:08.003076+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:34:42.916959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:23:08.003076+0000","last_change":"2023-10-06T02:23:08.014971+0000","last_active":"2023-10-06T02:23:08.003076+0000","last_peered":"2023-10-06T02:23:08.003076+0000","last_clean":"2023-10-06T02:23:08.003076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:08.003076+0000","last_undegraded":"2023-10-06T02:23:08.003076+0000","last_fullsized":"2023-10-06T02:23:08.003076+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:58.976414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:23:08.003076+0000","last_change":"2023-10-06T02:23:08.018899+0000","last_active":"2023-10-06T02:23:08.003076+0000","last_peered":"2023-10-06T02:23:08.003076+0000","last_clean":"2023-10-06T02:23:08.003076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:08.003076+0000","last_undegraded":"2023-10-06T02:23:08.003076+0000","last_fullsized":"2023-10-06T02:23:08.003076+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:57:40.559177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:23:08.003076+0000","last_change":"2023-10-06T02:23:08.022152+0000","last_active":"2023-10-06T02:23:08.003076+0000","last_peered":"2023-10-06T02:23:08.003076+0000","last_clean":"2023-10-06T02:23:08.003076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:08.003076+0000","last_undegraded":"2023-10-06T02:23:08.003076+0000","last_fullsized":"2023-10-06T02:23:08.003076+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:34:42.916959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:23:08.003076+0000","last_change":"2023-10-06T02:23:08.018330+0000","last_active":"2023-10-06T02:23:08.003076+0000","last_peered":"2023-10-06T02:23:08.003076+0000","last_clean":"2023-10-06T02:23:08.003076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:08.003076+0000","last_undegraded":"2023-10-06T02:23:08.003076+0000","last_fullsized":"2023-10-06T02:23:08.003076+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:57.514051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:23:08.003076+0000","last_change":"2023-10-06T02:23:08.022963+0000","last_active":"2023-10-06T02:23:08.003076+0000","last_peered":"2023-10-06T02:23:08.003076+0000","last_clean":"2023-10-06T02:23:08.003076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:08.003076+0000","last_undegraded":"2023-10-06T02:23:08.003076+0000","last_fullsized":"2023-10-06T02:23:08.003076+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:44:41.034030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:23:08.003076+0000","last_change":"2023-10-06T02:23:08.018090+0000","last_active":"2023-10-06T02:23:08.003076+0000","last_peered":"2023-10-06T02:23:08.003076+0000","last_clean":"2023-10-06T02:23:08.003076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:08.003076+0000","last_undegraded":"2023-10-06T02:23:08.003076+0000","last_fullsized":"2023-10-06T02:23:08.003076+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:17:32.434412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:23:08.003076+0000","last_change":"2023-10-06T02:23:08.023612+0000","last_active":"2023-10-06T02:23:08.003076+0000","last_peered":"2023-10-06T02:23:08.003076+0000","last_clean":"2023-10-06T02:23:08.003076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:08.003076+0000","last_undegraded":"2023-10-06T02:23:08.003076+0000","last_fullsized":"2023-10-06T02:23:08.003076+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:44:41.034030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:23:08.003076+0000","last_change":"2023-10-06T02:23:08.018175+0000","last_active":"2023-10-06T02:23:08.003076+0000","last_peered":"2023-10-06T02:23:08.003076+0000","last_clean":"2023-10-06T02:23:08.003076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:08.003076+0000","last_undegraded":"2023-10-06T02:23:08.003076+0000","last_fullsized":"2023-10-06T02:23:08.003076+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:36:27.554258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:23:08.003076+0000","last_change":"2023-10-06T02:23:08.018151+0000","last_active":"2023-10-06T02:23:08.003076+0000","last_peered":"2023-10-06T02:23:08.003076+0000","last_clean":"2023-10-06T02:23:08.003076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:08.003076+0000","last_undegraded":"2023-10-06T02:23:08.003076+0000","last_fullsized":"2023-10-06T02:23:08.003076+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:07:56.722823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:23:08.003076+0000","last_change":"2023-10-06T02:23:08.018149+0000","last_active":"2023-10-06T02:23:08.003076+0000","last_peered":"2023-10-06T02:23:08.003076+0000","last_clean":"2023-10-06T02:23:08.003076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:08.003076+0000","last_undegraded":"2023-10-06T02:23:08.003076+0000","last_fullsized":"2023-10-06T02:23:08.003076+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:44:15.545672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:23:08.003076+0000","last_change":"2023-10-06T02:23:08.014974+0000","last_active":"2023-10-06T02:23:08.003076+0000","last_peered":"2023-10-06T02:23:08.003076+0000","last_clean":"2023-10-06T02:23:08.003076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:08.003076+0000","last_undegraded":"2023-10-06T02:23:08.003076+0000","last_fullsized":"2023-10-06T02:23:08.003076+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:38.263723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:23:08.003076+0000","last_change":"2023-10-06T02:23:08.019362+0000","last_active":"2023-10-06T02:23:08.003076+0000","last_peered":"2023-10-06T02:23:08.003076+0000","last_clean":"2023-10-06T02:23:08.003076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:08.003076+0000","last_undegraded":"2023-10-06T02:23:08.003076+0000","last_fullsized":"2023-10-06T02:23:08.003076+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:17:32.434412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:23:08.003076+0000","last_change":"2023-10-06T02:23:08.023679+0000","last_active":"2023-10-06T02:23:08.003076+0000","last_peered":"2023-10-06T02:23:08.003076+0000","last_clean":"2023-10-06T02:23:08.003076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:08.003076+0000","last_undegraded":"2023-10-06T02:23:08.003076+0000","last_fullsized":"2023-10-06T02:23:08.003076+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:34:42.916959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:23:08.003076+0000","last_change":"2023-10-06T02:23:08.022582+0000","last_active":"2023-10-06T02:23:08.003076+0000","last_peered":"2023-10-06T02:23:08.003076+0000","last_clean":"2023-10-06T02:23:08.003076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:08.003076+0000","last_undegraded":"2023-10-06T02:23:08.003076+0000","last_fullsized":"2023-10-06T02:23:08.003076+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:44:41.034030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:23:08.003076+0000","last_change":"2023-10-06T02:23:08.019257+0000","last_active":"2023-10-06T02:23:08.003076+0000","last_peered":"2023-10-06T02:23:08.003076+0000","last_clean":"2023-10-06T02:23:08.003076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:08.003076+0000","last_undegraded":"2023-10-06T02:23:08.003076+0000","last_fullsized":"2023-10-06T02:23:08.003076+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:38.263723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T02:23:04.712407+0000","last_change":"2023-10-06T02:23:02.054039+0000","last_active":"2023-10-06T02:23:04.712407+0000","last_peered":"2023-10-06T02:23:04.712407+0000","last_clean":"2023-10-06T02:23:04.712407+0000","last_became_active":"2023-10-06T02:23:02.053742+0000","last_became_peered":"2023-10-06T02:23:02.053742+0000","last_unstale":"2023-10-06T02:23:04.712407+0000","last_undegraded":"2023-10-06T02:23:04.712407+0000","last_fullsized":"2023-10-06T02:23:04.712407+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:58:18.846475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:23:08.003076+0000","last_change":"2023-10-06T02:23:08.014835+0000","last_active":"2023-10-06T02:23:08.003076+0000","last_peered":"2023-10-06T02:23:08.003076+0000","last_clean":"2023-10-06T02:23:08.003076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:08.003076+0000","last_undegraded":"2023-10-06T02:23:08.003076+0000","last_fullsized":"2023-10-06T02:23:08.003076+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:41.039005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T02:23:02.992320+0000","last_change":"2023-10-06T02:23:01.988631+0000","last_active":"2023-10-06T02:23:02.992320+0000","last_peered":"2023-10-06T02:23:02.992320+0000","last_clean":"2023-10-06T02:23:02.992320+0000","last_became_active":"2023-10-06T02:23:01.988303+0000","last_became_peered":"2023-10-06T02:23:01.988303+0000","last_unstale":"2023-10-06T02:23:02.992320+0000","last_undegraded":"2023-10-06T02:23:02.992320+0000","last_fullsized":"2023-10-06T02:23:02.992320+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:38.211466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:23:08.003076+0000","last_change":"2023-10-06T02:23:08.020989+0000","last_active":"2023-10-06T02:23:08.003076+0000","last_peered":"2023-10-06T02:23:08.003076+0000","last_clean":"2023-10-06T02:23:08.003076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:08.003076+0000","last_undegraded":"2023-10-06T02:23:08.003076+0000","last_fullsized":"2023-10-06T02:23:08.003076+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:35:05.782473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T02:23:04.712320+0000","last_change":"2023-10-06T02:23:02.054338+0000","last_active":"2023-10-06T02:23:04.712320+0000","last_peered":"2023-10-06T02:23:04.712320+0000","last_clean":"2023-10-06T02:23:04.712320+0000","last_became_active":"2023-10-06T02:23:02.053991+0000","last_became_peered":"2023-10-06T02:23:02.053991+0000","last_unstale":"2023-10-06T02:23:04.712320+0000","last_undegraded":"2023-10-06T02:23:04.712320+0000","last_fullsized":"2023-10-06T02:23:04.712320+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:39:30.192838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:23:08.003076+0000","last_change":"2023-10-06T02:23:08.015382+0000","last_active":"2023-10-06T02:23:08.003076+0000","last_peered":"2023-10-06T02:23:08.003076+0000","last_clean":"2023-10-06T02:23:08.003076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:08.003076+0000","last_undegraded":"2023-10-06T02:23:08.003076+0000","last_fullsized":"2023-10-06T02:23:08.003076+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:34:42.916959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T02:23:02.992302+0000","last_change":"2023-10-06T02:23:01.988140+0000","last_active":"2023-10-06T02:23:02.992302+0000","last_peered":"2023-10-06T02:23:02.992302+0000","last_clean":"2023-10-06T02:23:02.992302+0000","last_became_active":"2023-10-06T02:23:01.987769+0000","last_became_peered":"2023-10-06T02:23:01.987769+0000","last_unstale":"2023-10-06T02:23:02.992302+0000","last_undegraded":"2023-10-06T02:23:02.992302+0000","last_fullsized":"2023-10-06T02:23:02.992302+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:20:08.391912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":163,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T02:23:03.813375+0000","last_change":"2023-10-06T02:22:55.032519+0000","last_active":"2023-10-06T02:23:03.813375+0000","last_peered":"2023-10-06T02:23:03.813375+0000","last_clean":"2023-10-06T02:23:03.813375+0000","last_became_active":"2023-10-06T02:22:55.031367+0000","last_became_peered":"2023-10-06T02:22:55.031367+0000","last_unstale":"2023-10-06T02:23:03.813375+0000","last_undegraded":"2023-10-06T02:23:03.813375+0000","last_fullsized":"2023-10-06T02:23:03.813375+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:22:53.309097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:22:53.309097+0000","last_clean_scrub_stamp":"2023-10-06T02:22:53.309097+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:53:46.520909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:23:08.003076+0000","last_change":"2023-10-06T02:23:08.018672+0000","last_active":"2023-10-06T02:23:08.003076+0000","last_peered":"2023-10-06T02:23:08.003076+0000","last_clean":"2023-10-06T02:23:08.003076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:08.003076+0000","last_undegraded":"2023-10-06T02:23:08.003076+0000","last_fullsized":"2023-10-06T02:23:08.003076+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:34:42.916959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T02:23:03.813389+0000","last_change":"2023-10-06T02:23:01.990773+0000","last_active":"2023-10-06T02:23:03.813389+0000","last_peered":"2023-10-06T02:23:03.813389+0000","last_clean":"2023-10-06T02:23:03.813389+0000","last_became_active":"2023-10-06T02:23:01.990511+0000","last_became_peered":"2023-10-06T02:23:01.990511+0000","last_unstale":"2023-10-06T02:23:03.813389+0000","last_undegraded":"2023-10-06T02:23:03.813389+0000","last_fullsized":"2023-10-06T02:23:03.813389+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:42:54.797731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:23:08.003076+0000","last_change":"2023-10-06T02:23:08.023749+0000","last_active":"2023-10-06T02:23:08.003076+0000","last_peered":"2023-10-06T02:23:08.003076+0000","last_clean":"2023-10-06T02:23:08.003076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:08.003076+0000","last_undegraded":"2023-10-06T02:23:08.003076+0000","last_fullsized":"2023-10-06T02:23:08.003076+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:07:19.307413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T02:23:03.813384+0000","last_change":"2023-10-06T02:23:02.054069+0000","last_active":"2023-10-06T02:23:03.813384+0000","last_peered":"2023-10-06T02:23:03.813384+0000","last_clean":"2023-10-06T02:23:03.813384+0000","last_became_active":"2023-10-06T02:23:02.053739+0000","last_became_peered":"2023-10-06T02:23:02.053739+0000","last_unstale":"2023-10-06T02:23:03.813384+0000","last_undegraded":"2023-10-06T02:23:03.813384+0000","last_fullsized":"2023-10-06T02:23:03.813384+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:57:48.822608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:23:08.003076+0000","last_change":"2023-10-06T02:23:08.015425+0000","last_active":"2023-10-06T02:23:08.003076+0000","last_peered":"2023-10-06T02:23:08.003076+0000","last_clean":"2023-10-06T02:23:08.003076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:08.003076+0000","last_undegraded":"2023-10-06T02:23:08.003076+0000","last_fullsized":"2023-10-06T02:23:08.003076+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:41.039005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:23:08.003076+0000","last_change":"2023-10-06T02:23:08.019208+0000","last_active":"2023-10-06T02:23:08.003076+0000","last_peered":"2023-10-06T02:23:08.003076+0000","last_clean":"2023-10-06T02:23:08.003076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:08.003076+0000","last_undegraded":"2023-10-06T02:23:08.003076+0000","last_fullsized":"2023-10-06T02:23:08.003076+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:41.039005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T02:23:04.712344+0000","last_change":"2023-10-06T02:23:01.989934+0000","last_active":"2023-10-06T02:23:04.712344+0000","last_peered":"2023-10-06T02:23:04.712344+0000","last_clean":"2023-10-06T02:23:04.712344+0000","last_became_active":"2023-10-06T02:23:01.989706+0000","last_became_peered":"2023-10-06T02:23:01.989706+0000","last_unstale":"2023-10-06T02:23:04.712344+0000","last_undegraded":"2023-10-06T02:23:04.712344+0000","last_fullsized":"2023-10-06T02:23:04.712344+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:06.212251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T02:23:04.712944+0000","last_change":"2023-10-06T02:23:01.992935+0000","last_active":"2023-10-06T02:23:04.712944+0000","last_peered":"2023-10-06T02:23:04.712944+0000","last_clean":"2023-10-06T02:23:04.712944+0000","last_became_active":"2023-10-06T02:23:01.992568+0000","last_became_peered":"2023-10-06T02:23:01.992568+0000","last_unstale":"2023-10-06T02:23:04.712944+0000","last_undegraded":"2023-10-06T02:23:04.712944+0000","last_fullsized":"2023-10-06T02:23:04.712944+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:06.212251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:23:08.003076+0000","last_change":"2023-10-06T02:23:08.023179+0000","last_active":"2023-10-06T02:23:08.003076+0000","last_peered":"2023-10-06T02:23:08.003076+0000","last_clean":"2023-10-06T02:23:08.003076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:08.003076+0000","last_undegraded":"2023-10-06T02:23:08.003076+0000","last_fullsized":"2023-10-06T02:23:08.003076+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:41.039005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:23:08.003076+0000","last_change":"2023-10-06T02:23:08.023091+0000","last_active":"2023-10-06T02:23:08.003076+0000","last_peered":"2023-10-06T02:23:08.003076+0000","last_clean":"2023-10-06T02:23:08.003076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:08.003076+0000","last_undegraded":"2023-10-06T02:23:08.003076+0000","last_fullsized":"2023-10-06T02:23:08.003076+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:58.976414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:23:08.003076+0000","last_change":"2023-10-06T02:23:08.015775+0000","last_active":"2023-10-06T02:23:08.003076+0000","last_peered":"2023-10-06T02:23:08.003076+0000","last_clean":"2023-10-06T02:23:08.003076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:08.003076+0000","last_undegraded":"2023-10-06T02:23:08.003076+0000","last_fullsized":"2023-10-06T02:23:08.003076+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:07:56.722823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:23:08.003076+0000","last_change":"2023-10-06T02:23:08.015831+0000","last_active":"2023-10-06T02:23:08.003076+0000","last_peered":"2023-10-06T02:23:08.003076+0000","last_clean":"2023-10-06T02:23:08.003076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:08.003076+0000","last_undegraded":"2023-10-06T02:23:08.003076+0000","last_fullsized":"2023-10-06T02:23:08.003076+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:17:32.434412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:23:08.003076+0000","last_change":"2023-10-06T02:23:08.019223+0000","last_active":"2023-10-06T02:23:08.003076+0000","last_peered":"2023-10-06T02:23:08.003076+0000","last_clean":"2023-10-06T02:23:08.003076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:08.003076+0000","last_undegraded":"2023-10-06T02:23:08.003076+0000","last_fullsized":"2023-10-06T02:23:08.003076+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:58.976414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:23:08.003076+0000","last_change":"2023-10-06T02:23:08.024513+0000","last_active":"2023-10-06T02:23:08.003076+0000","last_peered":"2023-10-06T02:23:08.003076+0000","last_clean":"2023-10-06T02:23:08.003076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:08.003076+0000","last_undegraded":"2023-10-06T02:23:08.003076+0000","last_fullsized":"2023-10-06T02:23:08.003076+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:17:32.434412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:23:09.010636+0000","last_change":"2023-10-06T02:23:09.010636+0000","last_active":"2023-10-06T02:23:09.010636+0000","last_peered":"2023-10-06T02:23:09.010636+0000","last_clean":"2023-10-06T02:23:09.010636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:23:09.010636+0000","last_undegraded":"2023-10-06T02:23:09.010636+0000","last_fullsized":"2023-10-06T02:23:09.010636+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298056,"kb_used_data":768,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073784,"statfs":{"total":96636764160,"available":96331554816,"internally_reserved":0,"allocated":786432,"data_stored":653164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297476,"kb_used_data":188,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074364,"statfs":{"total":96636764160,"available":96332148736,"internally_reserved":0,"allocated":192512,"data_stored":62796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86199999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74299999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297476,"kb_used_data":188,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074364,"statfs":{"total":96636764160,"available":96332148736,"internally_reserved":0,"allocated":192512,"data_stored":62796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0409999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.014}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1299999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0049999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298056,"kb_used_data":768,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073784,"statfs":{"total":96636764160,"available":96331554816,"internally_reserved":0,"allocated":786432,"data_stored":653164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84599999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0309999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99199999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96099999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297476,"kb_used_data":188,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074364,"statfs":{"total":96636764160,"available":96332148736,"internally_reserved":0,"allocated":192512,"data_stored":62796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35099999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78900000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297476,"kb_used_data":188,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074364,"statfs":{"total":96636764160,"available":96332148736,"internally_reserved":0,"allocated":192512,"data_stored":62796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66500000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297476,"kb_used_data":188,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074364,"statfs":{"total":96636764160,"available":96332148736,"internally_reserved":0,"allocated":192512,"data_stored":62796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71499999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83199999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63300000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297476,"kb_used_data":188,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074364,"statfs":{"total":96636764160,"available":96332148736,"internally_reserved":0,"allocated":192512,"data_stored":62796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77900000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91400000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.024}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T02:23:14.875 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T02:23:15.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.096+0000 7f09fb59e700 1 Processor -- start 2023-10-06T02:23:15.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.096+0000 7f09fb59e700 1 -- start start 2023-10-06T02:23:15.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.096+0000 7f09fb59e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f09fc13fb20 0x7f09fc15a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:15.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.096+0000 7f09fb59e700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f09fc13fef0 0x7f09fc0c7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:15.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.096+0000 7f09fb59e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f09fc06aaa0 0x7f09fc0c7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:15.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.096+0000 7f09fb59e700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09fc0c8410 con 0x7f09fc06aaa0 2023-10-06T02:23:15.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.096+0000 7f09fb59e700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09fc15ae70 con 0x7f09fc13fef0 2023-10-06T02:23:15.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.096+0000 7f09fb59e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09fc06af60 con 0x7f09fc13fb20 2023-10-06T02:23:15.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.096+0000 7f09fad9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f09fc06aaa0 0x7f09fc0c7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:15.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.097+0000 7f09fad9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f09fc06aaa0 0x7f09fc0c7ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44368/0 (socket says 172.21.15.31:44368) 2023-10-06T02:23:15.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.097+0000 7f09fad9d700 1 -- 172.21.15.31:0/203898878 learned_addr learned my addr 172.21.15.31:0/203898878 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:15.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.097+0000 7f09f9d9b700 1 --2- 172.21.15.31:0/203898878 >> v2:172.21.15.62:3300/0 conn(0x7f09fc13fef0 0x7f09fc0c7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:15.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.097+0000 7f09fa59c700 1 --2- 172.21.15.31:0/203898878 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f09fc13fb20 0x7f09fc15a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:15.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.097+0000 7f09fad9d700 1 -- 172.21.15.31:0/203898878 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f09fc13fb20 msgr2=0x7f09fc15a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:15.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.097+0000 7f09fad9d700 1 --2- 172.21.15.31:0/203898878 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f09fc13fb20 0x7f09fc15a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:15.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.097+0000 7f09fad9d700 1 -- 172.21.15.31:0/203898878 >> v2:172.21.15.62:3300/0 conn(0x7f09fc13fef0 msgr2=0x7f09fc0c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:15.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.097+0000 7f09fad9d700 1 --2- 172.21.15.31:0/203898878 >> v2:172.21.15.62:3300/0 conn(0x7f09fc13fef0 0x7f09fc0c7990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:15.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.097+0000 7f09fad9d700 1 -- 172.21.15.31:0/203898878 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09fc06b240 con 0x7f09fc06aaa0 2023-10-06T02:23:15.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.097+0000 7f09fad9d700 1 --2- 172.21.15.31:0/203898878 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09fc06aaa0 0x7f09fc0c7ed0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f09fc0cc090 tx=0x7f09f4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a22c728f1f7afde0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:15.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.098+0000 7f09f959a700 1 -- 172.21.15.31:0/203898878 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f400e070 con 0x7f09fc06aaa0 2023-10-06T02:23:15.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.098+0000 7f09f959a700 1 -- 172.21.15.31:0/203898878 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09f400b970 con 0x7f09fc06aaa0 2023-10-06T02:23:15.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.098+0000 7f09f959a700 1 -- 172.21.15.31:0/203898878 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f40092e0 con 0x7f09fc06aaa0 2023-10-06T02:23:15.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.098+0000 7f09fb59e700 1 -- 172.21.15.31:0/203898878 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09fc06aaa0 msgr2=0x7f09fc0c7ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:15.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.098+0000 7f09fb59e700 1 --2- 172.21.15.31:0/203898878 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09fc06aaa0 0x7f09fc0c7ed0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f09fc0cc090 tx=0x7f09f4009840 comp rx=0 tx=0).stop 2023-10-06T02:23:15.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.098+0000 7f09fb59e700 1 -- 172.21.15.31:0/203898878 shutdown_connections 2023-10-06T02:23:15.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.098+0000 7f09fb59e700 1 --2- 172.21.15.31:0/203898878 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09fc06aaa0 0x7f09fc0c7ed0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:15.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.098+0000 7f09fb59e700 1 --2- 172.21.15.31:0/203898878 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f09fc13fb20 0x7f09fc15a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:15.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.098+0000 7f09fb59e700 1 --2- 172.21.15.31:0/203898878 >> v2:172.21.15.62:3300/0 conn(0x7f09fc13fef0 0x7f09fc0c7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:15.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.098+0000 7f09fb59e700 1 -- 172.21.15.31:0/203898878 >> 172.21.15.31:0/203898878 conn(0x7f09fc06d170 msgr2=0x7f09fc0cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:15.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.098+0000 7f09fb59e700 1 -- 172.21.15.31:0/203898878 shutdown_connections 2023-10-06T02:23:15.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.098+0000 7f09fb59e700 1 -- 172.21.15.31:0/203898878 wait complete. 2023-10-06T02:23:15.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.101+0000 7f09fb59e700 1 Processor -- start 2023-10-06T02:23:15.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.101+0000 7f09fb59e700 1 -- start start 2023-10-06T02:23:15.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.101+0000 7f09fb59e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09fc06aaa0 0x7f09fc158650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:15.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.101+0000 7f09fb59e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09fc13fb20 0x7f09fc0ae160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:15.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.101+0000 7f09fb59e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f09fc13fef0 0x7f09fc0abaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:15.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.101+0000 7f09fb59e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09fc158230 con 0x7f09fc06aaa0 2023-10-06T02:23:15.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.101+0000 7f09fb59e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09fc1575b0 con 0x7f09fc13fb20 2023-10-06T02:23:15.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.101+0000 7f09fb59e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09fc157890 con 0x7f09fc13fef0 2023-10-06T02:23:15.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.101+0000 7f09fa59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09fc06aaa0 0x7f09fc158650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:15.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.101+0000 7f09fa59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09fc06aaa0 0x7f09fc158650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44384/0 (socket says 172.21.15.31:44384) 2023-10-06T02:23:15.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.101+0000 7f09fa59c700 1 -- 172.21.15.31:0/4021729726 learned_addr learned my addr 172.21.15.31:0/4021729726 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:15.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.101+0000 7f09fad9d700 1 --2- 172.21.15.31:0/4021729726 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f09fc13fef0 0x7f09fc0abaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:15.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.101+0000 7f09f9d9b700 1 --2- 172.21.15.31:0/4021729726 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09fc13fb20 0x7f09fc0ae160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:15.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.101+0000 7f09fa59c700 1 -- 172.21.15.31:0/4021729726 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f09fc13fef0 msgr2=0x7f09fc0abaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:15.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.101+0000 7f09fa59c700 1 --2- 172.21.15.31:0/4021729726 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f09fc13fef0 0x7f09fc0abaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:15.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.101+0000 7f09fa59c700 1 -- 172.21.15.31:0/4021729726 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09fc13fb20 msgr2=0x7f09fc0ae160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:15.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.101+0000 7f09fa59c700 1 --2- 172.21.15.31:0/4021729726 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09fc13fb20 0x7f09fc0ae160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:15.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.101+0000 7f09fa59c700 1 -- 172.21.15.31:0/4021729726 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09f4009ae0 con 0x7f09fc06aaa0 2023-10-06T02:23:15.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.102+0000 7f09fa59c700 1 --2- 172.21.15.31:0/4021729726 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09fc06aaa0 0x7f09fc158650 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f09ec00ec80 tx=0x7f09ec0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:15.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.102+0000 7f09ea7fc700 1 -- 172.21.15.31:0/4021729726 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09ec00c9d0 con 0x7f09fc06aaa0 2023-10-06T02:23:15.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.102+0000 7f09ea7fc700 1 -- 172.21.15.31:0/4021729726 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09ec00cb50 con 0x7f09fc06aaa0 2023-10-06T02:23:15.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.102+0000 7f09fb59e700 1 -- 172.21.15.31:0/4021729726 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09fc157b70 con 0x7f09fc06aaa0 2023-10-06T02:23:15.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.102+0000 7f09ea7fc700 1 -- 172.21.15.31:0/4021729726 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09ec013470 con 0x7f09fc06aaa0 2023-10-06T02:23:15.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.102+0000 7f09fb59e700 1 -- 172.21.15.31:0/4021729726 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09fc157240 con 0x7f09fc06aaa0 2023-10-06T02:23:15.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.103+0000 7f09fb59e700 1 -- 172.21.15.31:0/4021729726 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09fc04c3c0 con 0x7f09fc06aaa0 2023-10-06T02:23:15.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.104+0000 7f09ea7fc700 1 -- 172.21.15.31:0/4021729726 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f09ec013860 con 0x7f09fc06aaa0 2023-10-06T02:23:15.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.106+0000 7f09ea7fc700 1 --2- 172.21.15.31:0/4021729726 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f09dc066f00 0x7f09dc0693b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:15.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.106+0000 7f09ea7fc700 1 -- 172.21.15.31:0/4021729726 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f09ec084b10 con 0x7f09fc06aaa0 2023-10-06T02:23:15.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.106+0000 7f09f9d9b700 1 --2- 172.21.15.31:0/4021729726 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f09dc066f00 0x7f09dc0693b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:15.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.107+0000 7f09f9d9b700 1 --2- 172.21.15.31:0/4021729726 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f09dc066f00 0x7f09dc0693b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f09fc156170 tx=0x7f09f0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:15.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.110+0000 7f09ea7fc700 1 -- 172.21.15.31:0/4021729726 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f09ec051b80 con 0x7f09fc06aaa0 2023-10-06T02:23:15.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.308+0000 7f09fb59e700 1 -- 172.21.15.31:0/4021729726 --> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f09fc158010 con 0x7f09dc066f00 2023-10-06T02:23:15.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.317+0000 7f09ea7fc700 1 -- 172.21.15.31:0/4021729726 <== mgr.4107 v2:172.21.15.31:6832/85292 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286739 (secure 0 0 0) 0x7f09fc158010 con 0x7f09dc066f00 2023-10-06T02:23:15.321 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:23:15.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.322+0000 7f09fb59e700 1 -- 172.21.15.31:0/4021729726 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f09dc066f00 msgr2=0x7f09dc0693b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:15.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.322+0000 7f09fb59e700 1 --2- 172.21.15.31:0/4021729726 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f09dc066f00 0x7f09dc0693b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f09fc156170 tx=0x7f09f0008040 comp rx=0 tx=0).stop 2023-10-06T02:23:15.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.322+0000 7f09fb59e700 1 -- 172.21.15.31:0/4021729726 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09fc06aaa0 msgr2=0x7f09fc158650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:15.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.322+0000 7f09fb59e700 1 --2- 172.21.15.31:0/4021729726 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09fc06aaa0 0x7f09fc158650 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f09ec00ec80 tx=0x7f09ec0091a0 comp rx=0 tx=0).stop 2023-10-06T02:23:15.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.322+0000 7f09fb59e700 1 -- 172.21.15.31:0/4021729726 shutdown_connections 2023-10-06T02:23:15.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.322+0000 7f09fb59e700 1 --2- 172.21.15.31:0/4021729726 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f09dc066f00 0x7f09dc0693b0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:15.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.322+0000 7f09fb59e700 1 --2- 172.21.15.31:0/4021729726 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f09fc13fef0 0x7f09fc0abaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:15.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.322+0000 7f09fb59e700 1 --2- 172.21.15.31:0/4021729726 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09fc06aaa0 0x7f09fc158650 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:15.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.322+0000 7f09fb59e700 1 --2- 172.21.15.31:0/4021729726 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09fc13fb20 0x7f09fc0ae160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:15.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.322+0000 7f09fb59e700 1 -- 172.21.15.31:0/4021729726 >> 172.21.15.31:0/4021729726 conn(0x7f09fc06d170 msgr2=0x7f09fc0c0ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:15.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.322+0000 7f09fb59e700 1 -- 172.21.15.31:0/4021729726 shutdown_connections 2023-10-06T02:23:15.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.322+0000 7f09fb59e700 1 -- 172.21.15.31:0/4021729726 wait complete. 2023-10-06T02:23:15.325 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-10-06T02:23:15.339 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2023-10-06T02:23:15.118363+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":203,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381416,"kb_used_data":3112,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752593304,"statfs":{"total":773094113280,"available":770655543296,"internally_reserved":0,"allocated":3186688,"data_stored":1899900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.101512"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.985694+0000","last_change":"2023-10-06T02:23:09.982458+0000","last_active":"2023-10-06T02:23:09.985694+0000","last_peered":"2023-10-06T02:23:09.985694+0000","last_clean":"2023-10-06T02:23:09.985694+0000","last_became_active":"2023-10-06T02:23:09.982140+0000","last_became_peered":"2023-10-06T02:23:09.982140+0000","last_unstale":"2023-10-06T02:23:09.985694+0000","last_undegraded":"2023-10-06T02:23:09.985694+0000","last_fullsized":"2023-10-06T02:23:09.985694+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:07:57.730382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.999489+0000","last_change":"2023-10-06T02:23:09.996757+0000","last_active":"2023-10-06T02:23:09.999489+0000","last_peered":"2023-10-06T02:23:09.999489+0000","last_clean":"2023-10-06T02:23:09.999489+0000","last_became_active":"2023-10-06T02:23:09.996404+0000","last_became_peered":"2023-10-06T02:23:09.996404+0000","last_unstale":"2023-10-06T02:23:09.999489+0000","last_undegraded":"2023-10-06T02:23:09.999489+0000","last_fullsized":"2023-10-06T02:23:09.999489+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:39:15.275881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.010136+0000","last_change":"2023-10-06T02:23:09.999568+0000","last_active":"2023-10-06T02:23:10.010136+0000","last_peered":"2023-10-06T02:23:10.010136+0000","last_clean":"2023-10-06T02:23:10.010136+0000","last_became_active":"2023-10-06T02:23:09.999033+0000","last_became_peered":"2023-10-06T02:23:09.999033+0000","last_unstale":"2023-10-06T02:23:10.010136+0000","last_undegraded":"2023-10-06T02:23:10.010136+0000","last_fullsized":"2023-10-06T02:23:10.010136+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:23:26.580218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.996019+0000","last_change":"2023-10-06T02:23:09.995088+0000","last_active":"2023-10-06T02:23:09.996019+0000","last_peered":"2023-10-06T02:23:09.996019+0000","last_clean":"2023-10-06T02:23:09.996019+0000","last_became_active":"2023-10-06T02:23:09.994785+0000","last_became_peered":"2023-10-06T02:23:09.994785+0000","last_unstale":"2023-10-06T02:23:09.996019+0000","last_undegraded":"2023-10-06T02:23:09.996019+0000","last_fullsized":"2023-10-06T02:23:09.996019+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:21:33.737708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.994230+0000","last_change":"2023-10-06T02:23:09.986669+0000","last_active":"2023-10-06T02:23:09.994230+0000","last_peered":"2023-10-06T02:23:09.994230+0000","last_clean":"2023-10-06T02:23:09.994230+0000","last_became_active":"2023-10-06T02:23:09.986141+0000","last_became_peered":"2023-10-06T02:23:09.986141+0000","last_unstale":"2023-10-06T02:23:09.994230+0000","last_undegraded":"2023-10-06T02:23:09.994230+0000","last_fullsized":"2023-10-06T02:23:09.994230+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:39:15.275881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.010199+0000","last_change":"2023-10-06T02:23:10.000405+0000","last_active":"2023-10-06T02:23:10.010199+0000","last_peered":"2023-10-06T02:23:10.010199+0000","last_clean":"2023-10-06T02:23:10.010199+0000","last_became_active":"2023-10-06T02:23:10.000015+0000","last_became_peered":"2023-10-06T02:23:10.000015+0000","last_unstale":"2023-10-06T02:23:10.010199+0000","last_undegraded":"2023-10-06T02:23:10.010199+0000","last_fullsized":"2023-10-06T02:23:10.010199+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:39:15.275881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.009997+0000","last_change":"2023-10-06T02:23:10.004359+0000","last_active":"2023-10-06T02:23:10.009997+0000","last_peered":"2023-10-06T02:23:10.009997+0000","last_clean":"2023-10-06T02:23:10.009997+0000","last_became_active":"2023-10-06T02:23:10.003873+0000","last_became_peered":"2023-10-06T02:23:10.003873+0000","last_unstale":"2023-10-06T02:23:10.009997+0000","last_undegraded":"2023-10-06T02:23:10.009997+0000","last_fullsized":"2023-10-06T02:23:10.009997+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:44:41.608418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.998258+0000","last_change":"2023-10-06T02:23:09.994276+0000","last_active":"2023-10-06T02:23:09.998258+0000","last_peered":"2023-10-06T02:23:09.998258+0000","last_clean":"2023-10-06T02:23:09.998258+0000","last_became_active":"2023-10-06T02:23:09.993946+0000","last_became_peered":"2023-10-06T02:23:09.993946+0000","last_unstale":"2023-10-06T02:23:09.998258+0000","last_undegraded":"2023-10-06T02:23:09.998258+0000","last_fullsized":"2023-10-06T02:23:09.998258+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:19:29.462977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.997243+0000","last_change":"2023-10-06T02:23:09.993971+0000","last_active":"2023-10-06T02:23:09.997243+0000","last_peered":"2023-10-06T02:23:09.997243+0000","last_clean":"2023-10-06T02:23:09.997243+0000","last_became_active":"2023-10-06T02:23:09.993547+0000","last_became_peered":"2023-10-06T02:23:09.993547+0000","last_unstale":"2023-10-06T02:23:09.997243+0000","last_undegraded":"2023-10-06T02:23:09.997243+0000","last_fullsized":"2023-10-06T02:23:09.997243+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:19:29.462977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.999469+0000","last_change":"2023-10-06T02:23:09.995426+0000","last_active":"2023-10-06T02:23:09.999469+0000","last_peered":"2023-10-06T02:23:09.999469+0000","last_clean":"2023-10-06T02:23:09.999469+0000","last_became_active":"2023-10-06T02:23:09.995067+0000","last_became_peered":"2023-10-06T02:23:09.995067+0000","last_unstale":"2023-10-06T02:23:09.999469+0000","last_undegraded":"2023-10-06T02:23:09.999469+0000","last_fullsized":"2023-10-06T02:23:09.999469+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:50:19.168882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.010257+0000","last_change":"2023-10-06T02:23:10.001093+0000","last_active":"2023-10-06T02:23:10.010257+0000","last_peered":"2023-10-06T02:23:10.010257+0000","last_clean":"2023-10-06T02:23:10.010257+0000","last_became_active":"2023-10-06T02:23:10.000755+0000","last_became_peered":"2023-10-06T02:23:10.000755+0000","last_unstale":"2023-10-06T02:23:10.010257+0000","last_undegraded":"2023-10-06T02:23:10.010257+0000","last_fullsized":"2023-10-06T02:23:10.010257+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:43:02.156237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.998611+0000","last_change":"2023-10-06T02:23:09.993328+0000","last_active":"2023-10-06T02:23:09.998611+0000","last_peered":"2023-10-06T02:23:09.998611+0000","last_clean":"2023-10-06T02:23:09.998611+0000","last_became_active":"2023-10-06T02:23:09.993076+0000","last_became_peered":"2023-10-06T02:23:09.993076+0000","last_unstale":"2023-10-06T02:23:09.998611+0000","last_undegraded":"2023-10-06T02:23:09.998611+0000","last_fullsized":"2023-10-06T02:23:09.998611+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:58.521611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.986182+0000","last_change":"2023-10-06T02:23:09.982673+0000","last_active":"2023-10-06T02:23:09.986182+0000","last_peered":"2023-10-06T02:23:09.986182+0000","last_clean":"2023-10-06T02:23:09.986182+0000","last_became_active":"2023-10-06T02:23:09.982370+0000","last_became_peered":"2023-10-06T02:23:09.982370+0000","last_unstale":"2023-10-06T02:23:09.986182+0000","last_undegraded":"2023-10-06T02:23:09.986182+0000","last_fullsized":"2023-10-06T02:23:09.986182+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:17:33.441972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.985297+0000","last_change":"2023-10-06T02:23:09.982769+0000","last_active":"2023-10-06T02:23:09.985297+0000","last_peered":"2023-10-06T02:23:09.985297+0000","last_clean":"2023-10-06T02:23:09.985297+0000","last_became_active":"2023-10-06T02:23:09.982450+0000","last_became_peered":"2023-10-06T02:23:09.982450+0000","last_unstale":"2023-10-06T02:23:09.985297+0000","last_undegraded":"2023-10-06T02:23:09.985297+0000","last_fullsized":"2023-10-06T02:23:09.985297+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:12:16.306131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.010017+0000","last_change":"2023-10-06T02:23:10.004452+0000","last_active":"2023-10-06T02:23:10.010017+0000","last_peered":"2023-10-06T02:23:10.010017+0000","last_clean":"2023-10-06T02:23:10.010017+0000","last_became_active":"2023-10-06T02:23:10.004184+0000","last_became_peered":"2023-10-06T02:23:10.004184+0000","last_unstale":"2023-10-06T02:23:10.010017+0000","last_undegraded":"2023-10-06T02:23:10.010017+0000","last_fullsized":"2023-10-06T02:23:10.010017+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:50:19.168882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.010069+0000","last_change":"2023-10-06T02:23:09.998790+0000","last_active":"2023-10-06T02:23:10.010069+0000","last_peered":"2023-10-06T02:23:10.010069+0000","last_clean":"2023-10-06T02:23:10.010069+0000","last_became_active":"2023-10-06T02:23:09.998476+0000","last_became_peered":"2023-10-06T02:23:09.998476+0000","last_unstale":"2023-10-06T02:23:10.010069+0000","last_undegraded":"2023-10-06T02:23:10.010069+0000","last_fullsized":"2023-10-06T02:23:10.010069+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:19:29.462977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.997263+0000","last_change":"2023-10-06T02:23:09.994379+0000","last_active":"2023-10-06T02:23:09.997263+0000","last_peered":"2023-10-06T02:23:09.997263+0000","last_clean":"2023-10-06T02:23:09.997263+0000","last_became_active":"2023-10-06T02:23:09.994031+0000","last_became_peered":"2023-10-06T02:23:09.994031+0000","last_unstale":"2023-10-06T02:23:09.997263+0000","last_undegraded":"2023-10-06T02:23:09.997263+0000","last_fullsized":"2023-10-06T02:23:09.997263+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:12.998563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.994477+0000","last_change":"2023-10-06T02:23:09.988087+0000","last_active":"2023-10-06T02:23:09.994477+0000","last_peered":"2023-10-06T02:23:09.994477+0000","last_clean":"2023-10-06T02:23:09.994477+0000","last_became_active":"2023-10-06T02:23:09.987741+0000","last_became_peered":"2023-10-06T02:23:09.987741+0000","last_unstale":"2023-10-06T02:23:09.994477+0000","last_undegraded":"2023-10-06T02:23:09.994477+0000","last_fullsized":"2023-10-06T02:23:09.994477+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:05:24.047504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.987646+0000","last_change":"2023-10-06T02:23:09.983568+0000","last_active":"2023-10-06T02:23:09.987646+0000","last_peered":"2023-10-06T02:23:09.987646+0000","last_clean":"2023-10-06T02:23:09.987646+0000","last_became_active":"2023-10-06T02:23:09.983273+0000","last_became_peered":"2023-10-06T02:23:09.983273+0000","last_unstale":"2023-10-06T02:23:09.987646+0000","last_undegraded":"2023-10-06T02:23:09.987646+0000","last_fullsized":"2023-10-06T02:23:09.987646+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:44:16.553231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.009997+0000","last_change":"2023-10-06T02:23:09.997248+0000","last_active":"2023-10-06T02:23:10.009997+0000","last_peered":"2023-10-06T02:23:10.009997+0000","last_clean":"2023-10-06T02:23:10.009997+0000","last_became_active":"2023-10-06T02:23:09.996897+0000","last_became_peered":"2023-10-06T02:23:09.996897+0000","last_unstale":"2023-10-06T02:23:10.009997+0000","last_undegraded":"2023-10-06T02:23:10.009997+0000","last_fullsized":"2023-10-06T02:23:10.009997+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:00:45.442861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.994041+0000","last_change":"2023-10-06T02:23:09.991324+0000","last_active":"2023-10-06T02:23:09.994041+0000","last_peered":"2023-10-06T02:23:09.994041+0000","last_clean":"2023-10-06T02:23:09.994041+0000","last_became_active":"2023-10-06T02:23:09.991018+0000","last_became_peered":"2023-10-06T02:23:09.991018+0000","last_unstale":"2023-10-06T02:23:09.994041+0000","last_undegraded":"2023-10-06T02:23:09.994041+0000","last_fullsized":"2023-10-06T02:23:09.994041+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:43:01.259350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.004545+0000","last_change":"2023-10-06T02:23:10.000917+0000","last_active":"2023-10-06T02:23:10.004545+0000","last_peered":"2023-10-06T02:23:10.004545+0000","last_clean":"2023-10-06T02:23:10.004545+0000","last_became_active":"2023-10-06T02:23:10.000580+0000","last_became_peered":"2023-10-06T02:23:10.000580+0000","last_unstale":"2023-10-06T02:23:10.004545+0000","last_undegraded":"2023-10-06T02:23:10.004545+0000","last_fullsized":"2023-10-06T02:23:10.004545+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:50:19.168882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.998825+0000","last_change":"2023-10-06T02:23:09.995748+0000","last_active":"2023-10-06T02:23:09.998825+0000","last_peered":"2023-10-06T02:23:09.998825+0000","last_clean":"2023-10-06T02:23:09.998825+0000","last_became_active":"2023-10-06T02:23:09.995433+0000","last_became_peered":"2023-10-06T02:23:09.995433+0000","last_unstale":"2023-10-06T02:23:09.998825+0000","last_undegraded":"2023-10-06T02:23:09.998825+0000","last_fullsized":"2023-10-06T02:23:09.998825+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:21.438589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.996641+0000","last_change":"2023-10-06T02:23:09.992834+0000","last_active":"2023-10-06T02:23:09.996641+0000","last_peered":"2023-10-06T02:23:09.996641+0000","last_clean":"2023-10-06T02:23:09.996641+0000","last_became_active":"2023-10-06T02:23:09.991971+0000","last_became_peered":"2023-10-06T02:23:09.991971+0000","last_unstale":"2023-10-06T02:23:09.996641+0000","last_undegraded":"2023-10-06T02:23:09.996641+0000","last_fullsized":"2023-10-06T02:23:09.996641+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:12.998563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.007651+0000","last_change":"2023-10-06T02:23:10.005469+0000","last_active":"2023-10-06T02:23:10.007651+0000","last_peered":"2023-10-06T02:23:10.007651+0000","last_clean":"2023-10-06T02:23:10.007651+0000","last_became_active":"2023-10-06T02:23:10.005132+0000","last_became_peered":"2023-10-06T02:23:10.005132+0000","last_unstale":"2023-10-06T02:23:10.007651+0000","last_undegraded":"2023-10-06T02:23:10.007651+0000","last_fullsized":"2023-10-06T02:23:10.007651+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:19:18.748586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.005833+0000","last_change":"2023-10-06T02:23:09.999977+0000","last_active":"2023-10-06T02:23:10.005833+0000","last_peered":"2023-10-06T02:23:10.005833+0000","last_clean":"2023-10-06T02:23:10.005833+0000","last_became_active":"2023-10-06T02:23:09.999733+0000","last_became_peered":"2023-10-06T02:23:09.999733+0000","last_unstale":"2023-10-06T02:23:10.005833+0000","last_undegraded":"2023-10-06T02:23:10.005833+0000","last_fullsized":"2023-10-06T02:23:10.005833+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:58:41.925466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.999279+0000","last_change":"2023-10-06T02:23:09.994580+0000","last_active":"2023-10-06T02:23:09.999279+0000","last_peered":"2023-10-06T02:23:09.999279+0000","last_clean":"2023-10-06T02:23:09.999279+0000","last_became_active":"2023-10-06T02:23:09.993941+0000","last_became_peered":"2023-10-06T02:23:09.993941+0000","last_unstale":"2023-10-06T02:23:09.999279+0000","last_undegraded":"2023-10-06T02:23:09.999279+0000","last_fullsized":"2023-10-06T02:23:09.999279+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:43:01.259350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.043858+0000","last_change":"2023-10-06T02:23:09.048996+0000","last_active":"2023-10-06T02:23:10.043858+0000","last_peered":"2023-10-06T02:23:10.043858+0000","last_clean":"2023-10-06T02:23:10.043858+0000","last_became_active":"2023-10-06T02:23:09.048662+0000","last_became_peered":"2023-10-06T02:23:09.048662+0000","last_unstale":"2023-10-06T02:23:10.043858+0000","last_undegraded":"2023-10-06T02:23:10.043858+0000","last_fullsized":"2023-10-06T02:23:10.043858+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:58.976414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.004959+0000","last_change":"2023-10-06T02:23:10.001176+0000","last_active":"2023-10-06T02:23:10.004959+0000","last_peered":"2023-10-06T02:23:10.004959+0000","last_clean":"2023-10-06T02:23:10.004959+0000","last_became_active":"2023-10-06T02:23:10.000887+0000","last_became_peered":"2023-10-06T02:23:10.000887+0000","last_unstale":"2023-10-06T02:23:10.004959+0000","last_undegraded":"2023-10-06T02:23:10.004959+0000","last_fullsized":"2023-10-06T02:23:10.004959+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:43:02.156237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.998513+0000","last_change":"2023-10-06T02:23:09.048083+0000","last_active":"2023-10-06T02:23:09.998513+0000","last_peered":"2023-10-06T02:23:09.998513+0000","last_clean":"2023-10-06T02:23:09.998513+0000","last_became_active":"2023-10-06T02:23:09.047701+0000","last_became_peered":"2023-10-06T02:23:09.047701+0000","last_unstale":"2023-10-06T02:23:09.998513+0000","last_undegraded":"2023-10-06T02:23:09.998513+0000","last_fullsized":"2023-10-06T02:23:09.998513+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:17:32.434412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.994267+0000","last_change":"2023-10-06T02:23:09.988810+0000","last_active":"2023-10-06T02:23:09.994267+0000","last_peered":"2023-10-06T02:23:09.994267+0000","last_clean":"2023-10-06T02:23:09.994267+0000","last_became_active":"2023-10-06T02:23:09.988419+0000","last_became_peered":"2023-10-06T02:23:09.988419+0000","last_unstale":"2023-10-06T02:23:09.994267+0000","last_undegraded":"2023-10-06T02:23:09.994267+0000","last_fullsized":"2023-10-06T02:23:09.994267+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:43:01.259350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.976287+0000","last_change":"2023-10-06T02:23:09.044642+0000","last_active":"2023-10-06T02:23:09.976287+0000","last_peered":"2023-10-06T02:23:09.976287+0000","last_clean":"2023-10-06T02:23:09.976287+0000","last_became_active":"2023-10-06T02:23:09.044224+0000","last_became_peered":"2023-10-06T02:23:09.044224+0000","last_unstale":"2023-10-06T02:23:09.976287+0000","last_undegraded":"2023-10-06T02:23:09.976287+0000","last_fullsized":"2023-10-06T02:23:09.976287+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:31:52.666890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.986940+0000","last_change":"2023-10-06T02:23:09.985837+0000","last_active":"2023-10-06T02:23:09.986940+0000","last_peered":"2023-10-06T02:23:09.986940+0000","last_clean":"2023-10-06T02:23:09.986940+0000","last_became_active":"2023-10-06T02:23:09.985447+0000","last_became_peered":"2023-10-06T02:23:09.985447+0000","last_unstale":"2023-10-06T02:23:09.986940+0000","last_undegraded":"2023-10-06T02:23:09.986940+0000","last_fullsized":"2023-10-06T02:23:09.986940+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:31:53.674450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.979513+0000","last_change":"2023-10-06T02:23:09.040730+0000","last_active":"2023-10-06T02:23:09.979513+0000","last_peered":"2023-10-06T02:23:09.979513+0000","last_clean":"2023-10-06T02:23:09.979513+0000","last_became_active":"2023-10-06T02:23:09.040327+0000","last_became_peered":"2023-10-06T02:23:09.040327+0000","last_unstale":"2023-10-06T02:23:09.979513+0000","last_undegraded":"2023-10-06T02:23:09.979513+0000","last_fullsized":"2023-10-06T02:23:09.979513+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:07:56.722823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.998851+0000","last_change":"2023-10-06T02:23:09.995950+0000","last_active":"2023-10-06T02:23:09.998851+0000","last_peered":"2023-10-06T02:23:09.998851+0000","last_clean":"2023-10-06T02:23:09.998851+0000","last_became_active":"2023-10-06T02:23:09.995679+0000","last_became_peered":"2023-10-06T02:23:09.995679+0000","last_unstale":"2023-10-06T02:23:09.998851+0000","last_undegraded":"2023-10-06T02:23:09.998851+0000","last_fullsized":"2023-10-06T02:23:09.998851+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:23:26.580218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.976082+0000","last_change":"2023-10-06T02:23:09.040245+0000","last_active":"2023-10-06T02:23:09.976082+0000","last_peered":"2023-10-06T02:23:09.976082+0000","last_clean":"2023-10-06T02:23:09.976082+0000","last_became_active":"2023-10-06T02:23:09.039660+0000","last_became_peered":"2023-10-06T02:23:09.039660+0000","last_unstale":"2023-10-06T02:23:09.976082+0000","last_undegraded":"2023-10-06T02:23:09.976082+0000","last_fullsized":"2023-10-06T02:23:09.976082+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:57.514051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.994925+0000","last_change":"2023-10-06T02:23:09.992417+0000","last_active":"2023-10-06T02:23:09.994925+0000","last_peered":"2023-10-06T02:23:09.994925+0000","last_clean":"2023-10-06T02:23:09.994925+0000","last_became_active":"2023-10-06T02:23:09.992155+0000","last_became_peered":"2023-10-06T02:23:09.992155+0000","last_unstale":"2023-10-06T02:23:09.994925+0000","last_undegraded":"2023-10-06T02:23:09.994925+0000","last_fullsized":"2023-10-06T02:23:09.994925+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:39:15.275881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"22'3","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.044297+0000","last_change":"2023-10-06T02:23:09.046655+0000","last_active":"2023-10-06T02:23:10.044297+0000","last_peered":"2023-10-06T02:23:10.044297+0000","last_clean":"2023-10-06T02:23:10.044297+0000","last_became_active":"2023-10-06T02:23:09.046335+0000","last_became_peered":"2023-10-06T02:23:09.046335+0000","last_unstale":"2023-10-06T02:23:10.044297+0000","last_undegraded":"2023-10-06T02:23:10.044297+0000","last_fullsized":"2023-10-06T02:23:10.044297+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:53:00.225521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.007635+0000","last_change":"2023-10-06T02:23:10.005359+0000","last_active":"2023-10-06T02:23:10.007635+0000","last_peered":"2023-10-06T02:23:10.007635+0000","last_clean":"2023-10-06T02:23:10.007635+0000","last_became_active":"2023-10-06T02:23:10.005115+0000","last_became_peered":"2023-10-06T02:23:10.005115+0000","last_unstale":"2023-10-06T02:23:10.007635+0000","last_undegraded":"2023-10-06T02:23:10.007635+0000","last_fullsized":"2023-10-06T02:23:10.007635+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:29:00.771363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"22'6","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.052236+0000","last_change":"2023-10-06T02:23:09.046958+0000","last_active":"2023-10-06T02:23:10.052236+0000","last_peered":"2023-10-06T02:23:10.052236+0000","last_clean":"2023-10-06T02:23:10.052236+0000","last_became_active":"2023-10-06T02:23:09.044018+0000","last_became_peered":"2023-10-06T02:23:09.044018+0000","last_unstale":"2023-10-06T02:23:10.052236+0000","last_undegraded":"2023-10-06T02:23:10.052236+0000","last_fullsized":"2023-10-06T02:23:10.052236+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:41.039005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.992467+0000","last_change":"2023-10-06T02:23:09.987559+0000","last_active":"2023-10-06T02:23:09.992467+0000","last_peered":"2023-10-06T02:23:09.992467+0000","last_clean":"2023-10-06T02:23:09.992467+0000","last_became_active":"2023-10-06T02:23:09.987264+0000","last_became_peered":"2023-10-06T02:23:09.987264+0000","last_unstale":"2023-10-06T02:23:09.992467+0000","last_undegraded":"2023-10-06T02:23:09.992467+0000","last_fullsized":"2023-10-06T02:23:09.992467+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:44:41.608418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.049676+0000","last_change":"2023-10-06T02:23:09.048307+0000","last_active":"2023-10-06T02:23:10.049676+0000","last_peered":"2023-10-06T02:23:10.049676+0000","last_clean":"2023-10-06T02:23:10.049676+0000","last_became_active":"2023-10-06T02:23:09.047919+0000","last_became_peered":"2023-10-06T02:23:09.047919+0000","last_unstale":"2023-10-06T02:23:10.049676+0000","last_undegraded":"2023-10-06T02:23:10.049676+0000","last_fullsized":"2023-10-06T02:23:10.049676+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:28:59.763803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.004797+0000","last_change":"2023-10-06T02:23:10.001689+0000","last_active":"2023-10-06T02:23:10.004797+0000","last_peered":"2023-10-06T02:23:10.004797+0000","last_clean":"2023-10-06T02:23:10.004797+0000","last_became_active":"2023-10-06T02:23:10.001418+0000","last_became_peered":"2023-10-06T02:23:10.001418+0000","last_unstale":"2023-10-06T02:23:10.004797+0000","last_undegraded":"2023-10-06T02:23:10.004797+0000","last_fullsized":"2023-10-06T02:23:10.004797+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:53:54.469523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.977377+0000","last_change":"2023-10-06T02:23:09.041968+0000","last_active":"2023-10-06T02:23:09.977377+0000","last_peered":"2023-10-06T02:23:09.977377+0000","last_clean":"2023-10-06T02:23:09.977377+0000","last_became_active":"2023-10-06T02:23:09.041659+0000","last_became_peered":"2023-10-06T02:23:09.041659+0000","last_unstale":"2023-10-06T02:23:09.977377+0000","last_undegraded":"2023-10-06T02:23:09.977377+0000","last_fullsized":"2023-10-06T02:23:09.977377+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:44:15.545672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.992112+0000","last_change":"2023-10-06T02:23:09.989200+0000","last_active":"2023-10-06T02:23:09.992112+0000","last_peered":"2023-10-06T02:23:09.992112+0000","last_clean":"2023-10-06T02:23:09.992112+0000","last_became_active":"2023-10-06T02:23:09.988895+0000","last_became_peered":"2023-10-06T02:23:09.988895+0000","last_unstale":"2023-10-06T02:23:09.992112+0000","last_undegraded":"2023-10-06T02:23:09.992112+0000","last_fullsized":"2023-10-06T02:23:09.992112+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:44:41.608418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.984229+0000","last_change":"2023-10-06T02:23:09.048119+0000","last_active":"2023-10-06T02:23:09.984229+0000","last_peered":"2023-10-06T02:23:09.984229+0000","last_clean":"2023-10-06T02:23:09.984229+0000","last_became_active":"2023-10-06T02:23:09.047678+0000","last_became_peered":"2023-10-06T02:23:09.047678+0000","last_unstale":"2023-10-06T02:23:09.984229+0000","last_undegraded":"2023-10-06T02:23:09.984229+0000","last_fullsized":"2023-10-06T02:23:09.984229+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:42:11.693138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.997921+0000","last_change":"2023-10-06T02:23:09.994722+0000","last_active":"2023-10-06T02:23:09.997921+0000","last_peered":"2023-10-06T02:23:09.997921+0000","last_clean":"2023-10-06T02:23:09.997921+0000","last_became_active":"2023-10-06T02:23:09.994392+0000","last_became_peered":"2023-10-06T02:23:09.994392+0000","last_unstale":"2023-10-06T02:23:09.997921+0000","last_undegraded":"2023-10-06T02:23:09.997921+0000","last_fullsized":"2023-10-06T02:23:09.997921+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:21.438589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"22'8","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.049205+0000","last_change":"2023-10-06T02:23:09.034244+0000","last_active":"2023-10-06T02:23:10.049205+0000","last_peered":"2023-10-06T02:23:10.049205+0000","last_clean":"2023-10-06T02:23:10.049205+0000","last_became_active":"2023-10-06T02:23:09.033861+0000","last_became_peered":"2023-10-06T02:23:09.033861+0000","last_unstale":"2023-10-06T02:23:10.049205+0000","last_undegraded":"2023-10-06T02:23:10.049205+0000","last_fullsized":"2023-10-06T02:23:10.049205+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:41.039005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.990663+0000","last_change":"2023-10-06T02:23:09.988390+0000","last_active":"2023-10-06T02:23:09.990663+0000","last_peered":"2023-10-06T02:23:09.990663+0000","last_clean":"2023-10-06T02:23:09.990663+0000","last_became_active":"2023-10-06T02:23:09.988060+0000","last_became_peered":"2023-10-06T02:23:09.988060+0000","last_unstale":"2023-10-06T02:23:09.990663+0000","last_undegraded":"2023-10-06T02:23:09.990663+0000","last_fullsized":"2023-10-06T02:23:09.990663+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:43:01.259350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.044194+0000","last_change":"2023-10-06T02:23:09.048150+0000","last_active":"2023-10-06T02:23:10.044194+0000","last_peered":"2023-10-06T02:23:10.044194+0000","last_clean":"2023-10-06T02:23:10.044194+0000","last_became_active":"2023-10-06T02:23:09.047768+0000","last_became_peered":"2023-10-06T02:23:09.047768+0000","last_unstale":"2023-10-06T02:23:10.044194+0000","last_undegraded":"2023-10-06T02:23:10.044194+0000","last_fullsized":"2023-10-06T02:23:10.044194+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:03:50.725863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.992100+0000","last_change":"2023-10-06T02:23:09.990032+0000","last_active":"2023-10-06T02:23:09.992100+0000","last_peered":"2023-10-06T02:23:09.992100+0000","last_clean":"2023-10-06T02:23:09.992100+0000","last_became_active":"2023-10-06T02:23:09.988801+0000","last_became_peered":"2023-10-06T02:23:09.988801+0000","last_unstale":"2023-10-06T02:23:09.992100+0000","last_undegraded":"2023-10-06T02:23:09.992100+0000","last_fullsized":"2023-10-06T02:23:09.992100+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:12.998563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.044129+0000","last_change":"2023-10-06T02:23:09.048438+0000","last_active":"2023-10-06T02:23:10.044129+0000","last_peered":"2023-10-06T02:23:10.044129+0000","last_clean":"2023-10-06T02:23:10.044129+0000","last_became_active":"2023-10-06T02:23:09.048051+0000","last_became_peered":"2023-10-06T02:23:09.048051+0000","last_unstale":"2023-10-06T02:23:10.044129+0000","last_undegraded":"2023-10-06T02:23:10.044129+0000","last_fullsized":"2023-10-06T02:23:10.044129+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:21:32.730148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.010287+0000","last_change":"2023-10-06T02:23:10.002031+0000","last_active":"2023-10-06T02:23:10.010287+0000","last_peered":"2023-10-06T02:23:10.010287+0000","last_clean":"2023-10-06T02:23:10.010287+0000","last_became_active":"2023-10-06T02:23:10.001754+0000","last_became_peered":"2023-10-06T02:23:10.001754+0000","last_unstale":"2023-10-06T02:23:10.010287+0000","last_undegraded":"2023-10-06T02:23:10.010287+0000","last_fullsized":"2023-10-06T02:23:10.010287+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:45:34.162627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.044320+0000","last_change":"2023-10-06T02:23:09.038304+0000","last_active":"2023-10-06T02:23:10.044320+0000","last_peered":"2023-10-06T02:23:10.044320+0000","last_clean":"2023-10-06T02:23:10.044320+0000","last_became_active":"2023-10-06T02:23:09.038031+0000","last_became_peered":"2023-10-06T02:23:09.038031+0000","last_unstale":"2023-10-06T02:23:10.044320+0000","last_undegraded":"2023-10-06T02:23:10.044320+0000","last_fullsized":"2023-10-06T02:23:10.044320+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:31:52.666890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.004414+0000","last_change":"2023-10-06T02:23:10.000692+0000","last_active":"2023-10-06T02:23:10.004414+0000","last_peered":"2023-10-06T02:23:10.004414+0000","last_clean":"2023-10-06T02:23:10.004414+0000","last_became_active":"2023-10-06T02:23:10.000368+0000","last_became_peered":"2023-10-06T02:23:10.000368+0000","last_unstale":"2023-10-06T02:23:10.004414+0000","last_undegraded":"2023-10-06T02:23:10.004414+0000","last_fullsized":"2023-10-06T02:23:10.004414+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:53:01.233080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.974959+0000","last_change":"2023-10-06T02:23:09.037238+0000","last_active":"2023-10-06T02:23:09.974959+0000","last_peered":"2023-10-06T02:23:09.974959+0000","last_clean":"2023-10-06T02:23:09.974959+0000","last_became_active":"2023-10-06T02:23:09.036871+0000","last_became_peered":"2023-10-06T02:23:09.036871+0000","last_unstale":"2023-10-06T02:23:09.974959+0000","last_undegraded":"2023-10-06T02:23:09.974959+0000","last_fullsized":"2023-10-06T02:23:09.974959+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:44:15.545672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.002856+0000","last_change":"2023-10-06T02:23:10.000807+0000","last_active":"2023-10-06T02:23:10.002856+0000","last_peered":"2023-10-06T02:23:10.002856+0000","last_clean":"2023-10-06T02:23:10.002856+0000","last_became_active":"2023-10-06T02:23:10.000502+0000","last_became_peered":"2023-10-06T02:23:10.000502+0000","last_unstale":"2023-10-06T02:23:10.002856+0000","last_undegraded":"2023-10-06T02:23:10.002856+0000","last_fullsized":"2023-10-06T02:23:10.002856+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:31:27.242353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.042047+0000","last_change":"2023-10-06T02:23:09.044437+0000","last_active":"2023-10-06T02:23:10.042047+0000","last_peered":"2023-10-06T02:23:10.042047+0000","last_clean":"2023-10-06T02:23:10.042047+0000","last_became_active":"2023-10-06T02:23:09.042657+0000","last_became_peered":"2023-10-06T02:23:09.042657+0000","last_unstale":"2023-10-06T02:23:10.042047+0000","last_undegraded":"2023-10-06T02:23:10.042047+0000","last_fullsized":"2023-10-06T02:23:10.042047+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:57:40.559177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.994640+0000","last_change":"2023-10-06T02:23:09.990834+0000","last_active":"2023-10-06T02:23:09.994640+0000","last_peered":"2023-10-06T02:23:09.994640+0000","last_clean":"2023-10-06T02:23:09.994640+0000","last_became_active":"2023-10-06T02:23:09.990192+0000","last_became_peered":"2023-10-06T02:23:09.990192+0000","last_unstale":"2023-10-06T02:23:09.994640+0000","last_undegraded":"2023-10-06T02:23:09.994640+0000","last_fullsized":"2023-10-06T02:23:09.994640+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:18:32.416904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.980739+0000","last_change":"2023-10-06T02:23:09.044880+0000","last_active":"2023-10-06T02:23:09.980739+0000","last_peered":"2023-10-06T02:23:09.980739+0000","last_clean":"2023-10-06T02:23:09.980739+0000","last_became_active":"2023-10-06T02:23:09.044556+0000","last_became_peered":"2023-10-06T02:23:09.044556+0000","last_unstale":"2023-10-06T02:23:09.980739+0000","last_undegraded":"2023-10-06T02:23:09.980739+0000","last_fullsized":"2023-10-06T02:23:09.980739+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:58.976414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.994703+0000","last_change":"2023-10-06T02:23:09.992094+0000","last_active":"2023-10-06T02:23:09.994703+0000","last_peered":"2023-10-06T02:23:09.994703+0000","last_clean":"2023-10-06T02:23:09.994703+0000","last_became_active":"2023-10-06T02:23:09.991699+0000","last_became_peered":"2023-10-06T02:23:09.991699+0000","last_unstale":"2023-10-06T02:23:09.994703+0000","last_undegraded":"2023-10-06T02:23:09.994703+0000","last_fullsized":"2023-10-06T02:23:09.994703+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:21.438589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.986400+0000","last_change":"2023-10-06T02:23:09.041591+0000","last_active":"2023-10-06T02:23:09.986400+0000","last_peered":"2023-10-06T02:23:09.986400+0000","last_clean":"2023-10-06T02:23:09.986400+0000","last_became_active":"2023-10-06T02:23:09.041239+0000","last_became_peered":"2023-10-06T02:23:09.041239+0000","last_unstale":"2023-10-06T02:23:09.986400+0000","last_undegraded":"2023-10-06T02:23:09.986400+0000","last_fullsized":"2023-10-06T02:23:09.986400+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:17:32.434412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.994672+0000","last_change":"2023-10-06T02:23:09.991765+0000","last_active":"2023-10-06T02:23:09.994672+0000","last_peered":"2023-10-06T02:23:09.994672+0000","last_clean":"2023-10-06T02:23:09.994672+0000","last_became_active":"2023-10-06T02:23:09.990991+0000","last_became_peered":"2023-10-06T02:23:09.990991+0000","last_unstale":"2023-10-06T02:23:09.994672+0000","last_undegraded":"2023-10-06T02:23:09.994672+0000","last_fullsized":"2023-10-06T02:23:09.994672+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:18:23.992787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.976060+0000","last_change":"2023-10-06T02:23:09.046400+0000","last_active":"2023-10-06T02:23:09.976060+0000","last_peered":"2023-10-06T02:23:09.976060+0000","last_clean":"2023-10-06T02:23:09.976060+0000","last_became_active":"2023-10-06T02:23:09.046039+0000","last_became_peered":"2023-10-06T02:23:09.046039+0000","last_unstale":"2023-10-06T02:23:09.976060+0000","last_undegraded":"2023-10-06T02:23:09.976060+0000","last_fullsized":"2023-10-06T02:23:09.976060+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:44:15.545672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.005866+0000","last_change":"2023-10-06T02:23:10.001037+0000","last_active":"2023-10-06T02:23:10.005866+0000","last_peered":"2023-10-06T02:23:10.005866+0000","last_clean":"2023-10-06T02:23:10.005866+0000","last_became_active":"2023-10-06T02:23:10.000479+0000","last_became_peered":"2023-10-06T02:23:10.000479+0000","last_unstale":"2023-10-06T02:23:10.005866+0000","last_undegraded":"2023-10-06T02:23:10.005866+0000","last_fullsized":"2023-10-06T02:23:10.005866+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:50:19.168882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.996789+0000","last_change":"2023-10-06T02:23:02.054039+0000","last_active":"2023-10-06T02:23:09.996789+0000","last_peered":"2023-10-06T02:23:09.996789+0000","last_clean":"2023-10-06T02:23:09.996789+0000","last_became_active":"2023-10-06T02:23:02.053742+0000","last_became_peered":"2023-10-06T02:23:02.053742+0000","last_unstale":"2023-10-06T02:23:09.996789+0000","last_undegraded":"2023-10-06T02:23:09.996789+0000","last_fullsized":"2023-10-06T02:23:09.996789+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:58:18.846475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.975329+0000","last_change":"2023-10-06T02:23:09.037244+0000","last_active":"2023-10-06T02:23:09.975329+0000","last_peered":"2023-10-06T02:23:09.975329+0000","last_clean":"2023-10-06T02:23:09.975329+0000","last_became_active":"2023-10-06T02:23:09.036880+0000","last_became_peered":"2023-10-06T02:23:09.036880+0000","last_unstale":"2023-10-06T02:23:09.975329+0000","last_undegraded":"2023-10-06T02:23:09.975329+0000","last_fullsized":"2023-10-06T02:23:09.975329+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:36:27.554258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.005453+0000","last_change":"2023-10-06T02:23:09.999775+0000","last_active":"2023-10-06T02:23:10.005453+0000","last_peered":"2023-10-06T02:23:10.005453+0000","last_clean":"2023-10-06T02:23:10.005453+0000","last_became_active":"2023-10-06T02:23:09.999472+0000","last_became_peered":"2023-10-06T02:23:09.999472+0000","last_unstale":"2023-10-06T02:23:10.005453+0000","last_undegraded":"2023-10-06T02:23:10.005453+0000","last_fullsized":"2023-10-06T02:23:10.005453+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:44:41.608418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.977354+0000","last_change":"2023-10-06T02:23:01.988631+0000","last_active":"2023-10-06T02:23:09.977354+0000","last_peered":"2023-10-06T02:23:09.977354+0000","last_clean":"2023-10-06T02:23:09.977354+0000","last_became_active":"2023-10-06T02:23:01.988303+0000","last_became_peered":"2023-10-06T02:23:01.988303+0000","last_unstale":"2023-10-06T02:23:09.977354+0000","last_undegraded":"2023-10-06T02:23:09.977354+0000","last_fullsized":"2023-10-06T02:23:09.977354+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:38.211466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.986211+0000","last_change":"2023-10-06T02:23:09.041835+0000","last_active":"2023-10-06T02:23:09.986211+0000","last_peered":"2023-10-06T02:23:09.986211+0000","last_clean":"2023-10-06T02:23:09.986211+0000","last_became_active":"2023-10-06T02:23:09.041521+0000","last_became_peered":"2023-10-06T02:23:09.041521+0000","last_unstale":"2023-10-06T02:23:09.986211+0000","last_undegraded":"2023-10-06T02:23:09.986211+0000","last_fullsized":"2023-10-06T02:23:09.986211+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:38.263723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.003455+0000","last_change":"2023-10-06T02:23:09.995055+0000","last_active":"2023-10-06T02:23:10.003455+0000","last_peered":"2023-10-06T02:23:10.003455+0000","last_clean":"2023-10-06T02:23:10.003455+0000","last_became_active":"2023-10-06T02:23:09.994216+0000","last_became_peered":"2023-10-06T02:23:09.994216+0000","last_unstale":"2023-10-06T02:23:10.003455+0000","last_undegraded":"2023-10-06T02:23:10.003455+0000","last_fullsized":"2023-10-06T02:23:10.003455+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:47:34.184576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.972958+0000","last_change":"2023-10-06T02:23:02.054338+0000","last_active":"2023-10-06T02:23:09.972958+0000","last_peered":"2023-10-06T02:23:09.972958+0000","last_clean":"2023-10-06T02:23:09.972958+0000","last_became_active":"2023-10-06T02:23:02.053991+0000","last_became_peered":"2023-10-06T02:23:02.053991+0000","last_unstale":"2023-10-06T02:23:09.972958+0000","last_undegraded":"2023-10-06T02:23:09.972958+0000","last_fullsized":"2023-10-06T02:23:09.972958+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:39:30.192838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.996619+0000","last_change":"2023-10-06T02:23:09.039360+0000","last_active":"2023-10-06T02:23:09.996619+0000","last_peered":"2023-10-06T02:23:09.996619+0000","last_clean":"2023-10-06T02:23:09.996619+0000","last_became_active":"2023-10-06T02:23:09.038571+0000","last_became_peered":"2023-10-06T02:23:09.038571+0000","last_unstale":"2023-10-06T02:23:09.996619+0000","last_undegraded":"2023-10-06T02:23:09.996619+0000","last_fullsized":"2023-10-06T02:23:09.996619+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:58.976414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.005846+0000","last_change":"2023-10-06T02:23:10.000821+0000","last_active":"2023-10-06T02:23:10.005846+0000","last_peered":"2023-10-06T02:23:10.005846+0000","last_clean":"2023-10-06T02:23:10.005846+0000","last_became_active":"2023-10-06T02:23:10.000203+0000","last_became_peered":"2023-10-06T02:23:10.000203+0000","last_unstale":"2023-10-06T02:23:10.005846+0000","last_undegraded":"2023-10-06T02:23:10.005846+0000","last_fullsized":"2023-10-06T02:23:10.005846+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:43:01.259350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.975054+0000","last_change":"2023-10-06T02:23:01.988140+0000","last_active":"2023-10-06T02:23:09.975054+0000","last_peered":"2023-10-06T02:23:09.975054+0000","last_clean":"2023-10-06T02:23:09.975054+0000","last_became_active":"2023-10-06T02:23:01.987769+0000","last_became_peered":"2023-10-06T02:23:01.987769+0000","last_unstale":"2023-10-06T02:23:09.975054+0000","last_undegraded":"2023-10-06T02:23:09.975054+0000","last_fullsized":"2023-10-06T02:23:09.975054+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:20:08.391912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":166,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.983731+0000","last_change":"2023-10-06T02:22:55.032519+0000","last_active":"2023-10-06T02:23:09.983731+0000","last_peered":"2023-10-06T02:23:09.983731+0000","last_clean":"2023-10-06T02:23:09.983731+0000","last_became_active":"2023-10-06T02:22:55.031367+0000","last_became_peered":"2023-10-06T02:22:55.031367+0000","last_unstale":"2023-10-06T02:23:09.983731+0000","last_undegraded":"2023-10-06T02:23:09.983731+0000","last_fullsized":"2023-10-06T02:23:09.983731+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:22:53.309097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:22:53.309097+0000","last_clean_scrub_stamp":"2023-10-06T02:22:53.309097+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:53:46.520909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.975628+0000","last_change":"2023-10-06T02:23:09.041907+0000","last_active":"2023-10-06T02:23:09.975628+0000","last_peered":"2023-10-06T02:23:09.975628+0000","last_clean":"2023-10-06T02:23:09.975628+0000","last_became_active":"2023-10-06T02:23:09.036698+0000","last_became_peered":"2023-10-06T02:23:09.036698+0000","last_unstale":"2023-10-06T02:23:09.975628+0000","last_undegraded":"2023-10-06T02:23:09.975628+0000","last_fullsized":"2023-10-06T02:23:09.975628+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:57.514051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.003862+0000","last_change":"2023-10-06T02:23:09.997846+0000","last_active":"2023-10-06T02:23:10.003862+0000","last_peered":"2023-10-06T02:23:10.003862+0000","last_clean":"2023-10-06T02:23:10.003862+0000","last_became_active":"2023-10-06T02:23:09.997558+0000","last_became_peered":"2023-10-06T02:23:09.997558+0000","last_unstale":"2023-10-06T02:23:10.003862+0000","last_undegraded":"2023-10-06T02:23:10.003862+0000","last_fullsized":"2023-10-06T02:23:10.003862+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:29:00.771363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.984381+0000","last_change":"2023-10-06T02:23:01.990773+0000","last_active":"2023-10-06T02:23:09.984381+0000","last_peered":"2023-10-06T02:23:09.984381+0000","last_clean":"2023-10-06T02:23:09.984381+0000","last_became_active":"2023-10-06T02:23:01.990511+0000","last_became_peered":"2023-10-06T02:23:01.990511+0000","last_unstale":"2023-10-06T02:23:09.984381+0000","last_undegraded":"2023-10-06T02:23:09.984381+0000","last_fullsized":"2023-10-06T02:23:09.984381+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:42:54.797731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.977043+0000","last_change":"2023-10-06T02:23:09.047067+0000","last_active":"2023-10-06T02:23:09.977043+0000","last_peered":"2023-10-06T02:23:09.977043+0000","last_clean":"2023-10-06T02:23:09.977043+0000","last_became_active":"2023-10-06T02:23:09.046735+0000","last_became_peered":"2023-10-06T02:23:09.046735+0000","last_unstale":"2023-10-06T02:23:09.977043+0000","last_undegraded":"2023-10-06T02:23:09.977043+0000","last_fullsized":"2023-10-06T02:23:09.977043+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:57.514051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.005223+0000","last_change":"2023-10-06T02:23:10.001635+0000","last_active":"2023-10-06T02:23:10.005223+0000","last_peered":"2023-10-06T02:23:10.005223+0000","last_clean":"2023-10-06T02:23:10.005223+0000","last_became_active":"2023-10-06T02:23:10.001341+0000","last_became_peered":"2023-10-06T02:23:10.001341+0000","last_unstale":"2023-10-06T02:23:10.005223+0000","last_undegraded":"2023-10-06T02:23:10.005223+0000","last_fullsized":"2023-10-06T02:23:10.005223+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:44:41.608418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.984779+0000","last_change":"2023-10-06T02:23:02.054069+0000","last_active":"2023-10-06T02:23:09.984779+0000","last_peered":"2023-10-06T02:23:09.984779+0000","last_clean":"2023-10-06T02:23:09.984779+0000","last_became_active":"2023-10-06T02:23:02.053739+0000","last_became_peered":"2023-10-06T02:23:02.053739+0000","last_unstale":"2023-10-06T02:23:09.984779+0000","last_undegraded":"2023-10-06T02:23:09.984779+0000","last_fullsized":"2023-10-06T02:23:09.984779+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:57:48.822608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.996931+0000","last_change":"2023-10-06T02:23:09.037195+0000","last_active":"2023-10-06T02:23:09.996931+0000","last_peered":"2023-10-06T02:23:09.996931+0000","last_clean":"2023-10-06T02:23:09.996931+0000","last_became_active":"2023-10-06T02:23:09.036925+0000","last_became_peered":"2023-10-06T02:23:09.036925+0000","last_unstale":"2023-10-06T02:23:09.996931+0000","last_undegraded":"2023-10-06T02:23:09.996931+0000","last_fullsized":"2023-10-06T02:23:09.996931+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:38.263723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.994210+0000","last_change":"2023-10-06T02:23:09.986467+0000","last_active":"2023-10-06T02:23:09.994210+0000","last_peered":"2023-10-06T02:23:09.994210+0000","last_clean":"2023-10-06T02:23:09.994210+0000","last_became_active":"2023-10-06T02:23:09.985750+0000","last_became_peered":"2023-10-06T02:23:09.985750+0000","last_unstale":"2023-10-06T02:23:09.994210+0000","last_undegraded":"2023-10-06T02:23:09.994210+0000","last_fullsized":"2023-10-06T02:23:09.994210+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:18:23.992787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.044047+0000","last_change":"2023-10-06T02:23:09.041622+0000","last_active":"2023-10-06T02:23:10.044047+0000","last_peered":"2023-10-06T02:23:10.044047+0000","last_clean":"2023-10-06T02:23:10.044047+0000","last_became_active":"2023-10-06T02:23:09.036680+0000","last_became_peered":"2023-10-06T02:23:09.036680+0000","last_unstale":"2023-10-06T02:23:10.044047+0000","last_undegraded":"2023-10-06T02:23:10.044047+0000","last_fullsized":"2023-10-06T02:23:10.044047+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:36:27.554258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.993558+0000","last_change":"2023-10-06T02:23:01.989934+0000","last_active":"2023-10-06T02:23:09.993558+0000","last_peered":"2023-10-06T02:23:09.993558+0000","last_clean":"2023-10-06T02:23:09.993558+0000","last_became_active":"2023-10-06T02:23:01.989706+0000","last_became_peered":"2023-10-06T02:23:01.989706+0000","last_unstale":"2023-10-06T02:23:09.993558+0000","last_undegraded":"2023-10-06T02:23:09.993558+0000","last_fullsized":"2023-10-06T02:23:09.993558+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:06.212251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.989446+0000","last_change":"2023-10-06T02:23:09.987681+0000","last_active":"2023-10-06T02:23:09.989446+0000","last_peered":"2023-10-06T02:23:09.989446+0000","last_clean":"2023-10-06T02:23:09.989446+0000","last_became_active":"2023-10-06T02:23:09.987439+0000","last_became_peered":"2023-10-06T02:23:09.987439+0000","last_unstale":"2023-10-06T02:23:09.989446+0000","last_undegraded":"2023-10-06T02:23:09.989446+0000","last_fullsized":"2023-10-06T02:23:09.989446+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:31:27.242353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.980493+0000","last_change":"2023-10-06T02:23:01.992935+0000","last_active":"2023-10-06T02:23:09.980493+0000","last_peered":"2023-10-06T02:23:09.980493+0000","last_clean":"2023-10-06T02:23:09.980493+0000","last_became_active":"2023-10-06T02:23:01.992568+0000","last_became_peered":"2023-10-06T02:23:01.992568+0000","last_unstale":"2023-10-06T02:23:09.980493+0000","last_undegraded":"2023-10-06T02:23:09.980493+0000","last_fullsized":"2023-10-06T02:23:09.980493+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:06.212251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"23'1","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T02:23:11.091780+0000","last_change":"2023-10-06T02:23:09.039504+0000","last_active":"2023-10-06T02:23:11.091780+0000","last_peered":"2023-10-06T02:23:11.091780+0000","last_clean":"2023-10-06T02:23:11.091780+0000","last_became_active":"2023-10-06T02:23:09.039056+0000","last_became_peered":"2023-10-06T02:23:09.039056+0000","last_unstale":"2023-10-06T02:23:11.091780+0000","last_undegraded":"2023-10-06T02:23:11.091780+0000","last_fullsized":"2023-10-06T02:23:11.091780+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:38.263723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.995987+0000","last_change":"2023-10-06T02:23:09.990800+0000","last_active":"2023-10-06T02:23:09.995987+0000","last_peered":"2023-10-06T02:23:09.995987+0000","last_clean":"2023-10-06T02:23:09.995987+0000","last_became_active":"2023-10-06T02:23:09.990531+0000","last_became_peered":"2023-10-06T02:23:09.990531+0000","last_unstale":"2023-10-06T02:23:09.995987+0000","last_undegraded":"2023-10-06T02:23:09.995987+0000","last_fullsized":"2023-10-06T02:23:09.995987+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:39:39.959479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.045165+0000","last_change":"2023-10-06T02:23:09.046230+0000","last_active":"2023-10-06T02:23:10.045165+0000","last_peered":"2023-10-06T02:23:10.045165+0000","last_clean":"2023-10-06T02:23:10.045165+0000","last_became_active":"2023-10-06T02:23:09.045976+0000","last_became_peered":"2023-10-06T02:23:09.045976+0000","last_unstale":"2023-10-06T02:23:10.045165+0000","last_undegraded":"2023-10-06T02:23:10.045165+0000","last_fullsized":"2023-10-06T02:23:10.045165+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:36:27.554258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.998669+0000","last_change":"2023-10-06T02:23:09.995680+0000","last_active":"2023-10-06T02:23:09.998669+0000","last_peered":"2023-10-06T02:23:09.998669+0000","last_clean":"2023-10-06T02:23:09.998669+0000","last_became_active":"2023-10-06T02:23:09.995375+0000","last_became_peered":"2023-10-06T02:23:09.995375+0000","last_unstale":"2023-10-06T02:23:09.998669+0000","last_undegraded":"2023-10-06T02:23:09.998669+0000","last_fullsized":"2023-10-06T02:23:09.998669+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:36:28.561818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.046988+0000","last_change":"2023-10-06T02:23:09.043361+0000","last_active":"2023-10-06T02:23:10.046988+0000","last_peered":"2023-10-06T02:23:10.046988+0000","last_clean":"2023-10-06T02:23:10.046988+0000","last_became_active":"2023-10-06T02:23:09.042973+0000","last_became_peered":"2023-10-06T02:23:09.042973+0000","last_unstale":"2023-10-06T02:23:10.046988+0000","last_undegraded":"2023-10-06T02:23:10.046988+0000","last_fullsized":"2023-10-06T02:23:10.046988+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:31:52.666890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.992487+0000","last_change":"2023-10-06T02:23:09.988158+0000","last_active":"2023-10-06T02:23:09.992487+0000","last_peered":"2023-10-06T02:23:09.992487+0000","last_clean":"2023-10-06T02:23:09.992487+0000","last_became_active":"2023-10-06T02:23:09.987844+0000","last_became_peered":"2023-10-06T02:23:09.987844+0000","last_unstale":"2023-10-06T02:23:09.992487+0000","last_undegraded":"2023-10-06T02:23:09.992487+0000","last_fullsized":"2023-10-06T02:23:09.992487+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:10:59.616168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.045352+0000","last_change":"2023-10-06T02:23:09.044069+0000","last_active":"2023-10-06T02:23:10.045352+0000","last_peered":"2023-10-06T02:23:10.045352+0000","last_clean":"2023-10-06T02:23:10.045352+0000","last_became_active":"2023-10-06T02:23:09.043700+0000","last_became_peered":"2023-10-06T02:23:09.043700+0000","last_unstale":"2023-10-06T02:23:10.045352+0000","last_undegraded":"2023-10-06T02:23:10.045352+0000","last_fullsized":"2023-10-06T02:23:10.045352+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:57:40.559177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.010156+0000","last_change":"2023-10-06T02:23:09.999700+0000","last_active":"2023-10-06T02:23:10.010156+0000","last_peered":"2023-10-06T02:23:10.010156+0000","last_clean":"2023-10-06T02:23:10.010156+0000","last_became_active":"2023-10-06T02:23:09.997418+0000","last_became_peered":"2023-10-06T02:23:09.997418+0000","last_unstale":"2023-10-06T02:23:10.010156+0000","last_undegraded":"2023-10-06T02:23:10.010156+0000","last_fullsized":"2023-10-06T02:23:10.010156+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:53:54.469523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.041904+0000","last_change":"2023-10-06T02:23:09.034024+0000","last_active":"2023-10-06T02:23:10.041904+0000","last_peered":"2023-10-06T02:23:10.041904+0000","last_clean":"2023-10-06T02:23:10.041904+0000","last_became_active":"2023-10-06T02:23:09.033389+0000","last_became_peered":"2023-10-06T02:23:09.033389+0000","last_unstale":"2023-10-06T02:23:10.041904+0000","last_undegraded":"2023-10-06T02:23:10.041904+0000","last_fullsized":"2023-10-06T02:23:10.041904+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:03:50.725863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.991920+0000","last_change":"2023-10-06T02:23:09.984080+0000","last_active":"2023-10-06T02:23:09.991920+0000","last_peered":"2023-10-06T02:23:09.991920+0000","last_clean":"2023-10-06T02:23:09.991920+0000","last_became_active":"2023-10-06T02:23:09.983808+0000","last_became_peered":"2023-10-06T02:23:09.983808+0000","last_unstale":"2023-10-06T02:23:09.991920+0000","last_undegraded":"2023-10-06T02:23:09.991920+0000","last_fullsized":"2023-10-06T02:23:09.991920+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:12.998563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"22'4","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.045612+0000","last_change":"2023-10-06T02:23:09.037190+0000","last_active":"2023-10-06T02:23:10.045612+0000","last_peered":"2023-10-06T02:23:10.045612+0000","last_clean":"2023-10-06T02:23:10.045612+0000","last_became_active":"2023-10-06T02:23:09.036872+0000","last_became_peered":"2023-10-06T02:23:09.036872+0000","last_unstale":"2023-10-06T02:23:10.045612+0000","last_undegraded":"2023-10-06T02:23:10.045612+0000","last_fullsized":"2023-10-06T02:23:10.045612+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:57:40.559177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.994436+0000","last_change":"2023-10-06T02:23:09.989896+0000","last_active":"2023-10-06T02:23:09.994436+0000","last_peered":"2023-10-06T02:23:09.994436+0000","last_clean":"2023-10-06T02:23:09.994436+0000","last_became_active":"2023-10-06T02:23:09.989525+0000","last_became_peered":"2023-10-06T02:23:09.989525+0000","last_unstale":"2023-10-06T02:23:09.994436+0000","last_undegraded":"2023-10-06T02:23:09.994436+0000","last_fullsized":"2023-10-06T02:23:09.994436+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:58:41.925466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.992536+0000","last_change":"2023-10-06T02:23:09.988852+0000","last_active":"2023-10-06T02:23:09.992536+0000","last_peered":"2023-10-06T02:23:09.992536+0000","last_clean":"2023-10-06T02:23:09.992536+0000","last_became_active":"2023-10-06T02:23:09.988527+0000","last_became_peered":"2023-10-06T02:23:09.988527+0000","last_unstale":"2023-10-06T02:23:09.992536+0000","last_undegraded":"2023-10-06T02:23:09.992536+0000","last_fullsized":"2023-10-06T02:23:09.992536+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:03:51.733423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.001202+0000","last_change":"2023-10-06T02:23:09.999698+0000","last_active":"2023-10-06T02:23:10.001202+0000","last_peered":"2023-10-06T02:23:10.001202+0000","last_clean":"2023-10-06T02:23:10.001202+0000","last_became_active":"2023-10-06T02:23:09.999418+0000","last_became_peered":"2023-10-06T02:23:09.999418+0000","last_unstale":"2023-10-06T02:23:10.001202+0000","last_undegraded":"2023-10-06T02:23:10.001202+0000","last_fullsized":"2023-10-06T02:23:10.001202+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:10:59.616168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.010180+0000","last_change":"2023-10-06T02:23:10.000203+0000","last_active":"2023-10-06T02:23:10.010180+0000","last_peered":"2023-10-06T02:23:10.010180+0000","last_clean":"2023-10-06T02:23:10.010180+0000","last_became_active":"2023-10-06T02:23:09.999397+0000","last_became_peered":"2023-10-06T02:23:09.999397+0000","last_unstale":"2023-10-06T02:23:10.010180+0000","last_undegraded":"2023-10-06T02:23:10.010180+0000","last_fullsized":"2023-10-06T02:23:10.010180+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:31:27.242353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.989526+0000","last_change":"2023-10-06T02:23:09.986996+0000","last_active":"2023-10-06T02:23:09.989526+0000","last_peered":"2023-10-06T02:23:09.989526+0000","last_clean":"2023-10-06T02:23:09.989526+0000","last_became_active":"2023-10-06T02:23:09.986723+0000","last_became_peered":"2023-10-06T02:23:09.986723+0000","last_unstale":"2023-10-06T02:23:09.989526+0000","last_undegraded":"2023-10-06T02:23:09.989526+0000","last_fullsized":"2023-10-06T02:23:09.989526+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:19:29.462977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.007171+0000","last_change":"2023-10-06T02:23:10.003345+0000","last_active":"2023-10-06T02:23:10.007171+0000","last_peered":"2023-10-06T02:23:10.007171+0000","last_clean":"2023-10-06T02:23:10.007171+0000","last_became_active":"2023-10-06T02:23:10.003023+0000","last_became_peered":"2023-10-06T02:23:10.003023+0000","last_unstale":"2023-10-06T02:23:10.007171+0000","last_undegraded":"2023-10-06T02:23:10.007171+0000","last_fullsized":"2023-10-06T02:23:10.007171+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:18:23.992787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298132,"kb_used_data":844,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073708,"statfs":{"total":96636764160,"available":96331476992,"internally_reserved":0,"allocated":864256,"data_stored":681439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81299999999999994}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297524,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074316,"statfs":{"total":96636764160,"available":96332099584,"internally_reserved":0,"allocated":241664,"data_stored":89319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78500000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79600000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297524,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074316,"statfs":{"total":96636764160,"available":96332099584,"internally_reserved":0,"allocated":241664,"data_stored":89319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68500000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94399999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77300000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298108,"kb_used_data":820,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073732,"statfs":{"total":96636764160,"available":96331501568,"internally_reserved":0,"allocated":839680,"data_stored":680241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.755}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70599999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69199999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297524,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074316,"statfs":{"total":96636764160,"available":96332099584,"internally_reserved":0,"allocated":241664,"data_stored":89319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41599999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82699999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85199999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297528,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074312,"statfs":{"total":96636764160,"available":96332095488,"internally_reserved":0,"allocated":245760,"data_stored":89409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297544,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074296,"statfs":{"total":96636764160,"available":96332079104,"internally_reserved":0,"allocated":262144,"data_stored":90911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69199999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89600000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86799999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97999999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297532,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074308,"statfs":{"total":96636764160,"available":96332091392,"internally_reserved":0,"allocated":249856,"data_stored":89943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82199999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78300000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T02:23:15.346 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-06T02:23:15.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.566+0000 7f69cad9d700 1 Processor -- start 2023-10-06T02:23:15.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.566+0000 7f69cad9d700 1 -- start start 2023-10-06T02:23:15.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.567+0000 7f69cad9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f69c415dfd0 0x7f69c40aedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:15.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.567+0000 7f69cad9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69c40acc00 con 0x7f69c4159920 2023-10-06T02:23:15.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.567+0000 7f69cad9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69c40abff0 con 0x7f69c4159550 2023-10-06T02:23:15.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.567+0000 7f69cad9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69c40ac2d0 con 0x7f69c415dfd0 2023-10-06T02:23:15.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.567+0000 7f69ca59c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f69c4159920 0x7f69c40ad090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56162/0 (socket says 172.21.15.31:56162) 2023-10-06T02:23:15.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.567+0000 7f69ca59c700 1 -- 172.21.15.31:0/2475343842 learned_addr learned my addr 172.21.15.31:0/2475343842 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:23:15.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.567+0000 7f69c959a700 1 --2- 172.21.15.31:0/2475343842 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f69c415dfd0 0x7f69c40aedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:15.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.567+0000 7f69c8d99700 1 -- 172.21.15.31:0/2475343842 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3654934899 0 0) 0x7f69c40acc00 con 0x7f69c4159920 2023-10-06T02:23:15.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.567+0000 7f69c8d99700 1 -- 172.21.15.31:0/2475343842 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69b0003570 con 0x7f69c4159920 2023-10-06T02:23:15.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.567+0000 7f69c8d99700 1 -- 172.21.15.31:0/2475343842 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 611721818 0 0) 0x7f69c40abff0 con 0x7f69c4159550 2023-10-06T02:23:15.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.567+0000 7f69c8d99700 1 -- 172.21.15.31:0/2475343842 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69c40acc00 con 0x7f69c4159550 2023-10-06T02:23:15.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.567+0000 7f69c8d99700 1 -- 172.21.15.31:0/2475343842 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2192863681 0 0) 0x7f69b0003570 con 0x7f69c4159920 2023-10-06T02:23:15.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.567+0000 7f69c959a700 1 -- 172.21.15.31:0/2475343842 >> v1:172.21.15.62:6789/0 conn(0x7f69c4159550 legacy=0x7f69c40ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:15.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.567+0000 7f69c959a700 1 -- 172.21.15.31:0/2475343842 >> v1:172.21.15.31:6789/0 conn(0x7f69c4159920 legacy=0x7f69c40ad090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:15.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.567+0000 7f69c959a700 1 -- 172.21.15.31:0/2475343842 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69c40ac5b0 con 0x7f69c415dfd0 2023-10-06T02:23:15.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.568+0000 7f69c959a700 1 --2- 172.21.15.31:0/2475343842 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f69c415dfd0 0x7f69c40aedb0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f69b4009910 tx=0x7f69b40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b92a9e05d5318adc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:15.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.568+0000 7f69c8d99700 1 -- 172.21.15.31:0/2475343842 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69bc0017f0 con 0x7f69c415dfd0 2023-10-06T02:23:15.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.568+0000 7f69c8d99700 1 -- 172.21.15.31:0/2475343842 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69b4011420 con 0x7f69c415dfd0 2023-10-06T02:23:15.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.568+0000 7f69c8d99700 1 -- 172.21.15.31:0/2475343842 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b40117c0 con 0x7f69c415dfd0 2023-10-06T02:23:15.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.568+0000 7f69cad9d700 1 -- 172.21.15.31:0/2475343842 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f69c415dfd0 msgr2=0x7f69c40aedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:15.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.568+0000 7f69cad9d700 1 --2- 172.21.15.31:0/2475343842 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f69c415dfd0 0x7f69c40aedb0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f69b4009910 tx=0x7f69b40091a0 comp rx=0 tx=0).stop 2023-10-06T02:23:15.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.568+0000 7f69cad9d700 1 -- 172.21.15.31:0/2475343842 shutdown_connections 2023-10-06T02:23:15.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.568+0000 7f69cad9d700 1 --2- 172.21.15.31:0/2475343842 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f69c415dfd0 0x7f69c40aedb0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:15.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.568+0000 7f69cad9d700 1 -- 172.21.15.31:0/2475343842 >> 172.21.15.31:0/2475343842 conn(0x7f69c406ce80 msgr2=0x7f69c40ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:15.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.568+0000 7f69cad9d700 1 -- 172.21.15.31:0/2475343842 shutdown_connections 2023-10-06T02:23:15.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.568+0000 7f69cad9d700 1 -- 172.21.15.31:0/2475343842 wait complete. 2023-10-06T02:23:15.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.570+0000 7f69cad9d700 1 Processor -- start 2023-10-06T02:23:15.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.570+0000 7f69cad9d700 1 -- start start 2023-10-06T02:23:15.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.571+0000 7f69cad9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69c4159550 0x7f69c4149440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:15.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.571+0000 7f69cad9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69c4159920 0x7f69c414dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:15.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.571+0000 7f69cad9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f69c415dfd0 0x7f69c414d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:15.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.571+0000 7f69cad9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69c414bcc0 con 0x7f69c4159550 2023-10-06T02:23:15.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.571+0000 7f69cad9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69c40c95b0 con 0x7f69c4159920 2023-10-06T02:23:15.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.571+0000 7f69cad9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69c4149980 con 0x7f69c415dfd0 2023-10-06T02:23:15.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.571+0000 7f69c9d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69c4159550 0x7f69c4149440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:15.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.571+0000 7f69c9d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69c4159550 0x7f69c4149440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44398/0 (socket says 172.21.15.31:44398) 2023-10-06T02:23:15.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.571+0000 7f69c9d9b700 1 -- 172.21.15.31:0/4056218179 learned_addr learned my addr 172.21.15.31:0/4056218179 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:15.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.571+0000 7f69c959a700 1 --2- 172.21.15.31:0/4056218179 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69c4159920 0x7f69c414dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:15.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.571+0000 7f69ca59c700 1 --2- 172.21.15.31:0/4056218179 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f69c415dfd0 0x7f69c414d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:15.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.572+0000 7f69c9d9b700 1 -- 172.21.15.31:0/4056218179 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f69c415dfd0 msgr2=0x7f69c414d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:15.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.572+0000 7f69c9d9b700 1 --2- 172.21.15.31:0/4056218179 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f69c415dfd0 0x7f69c414d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:15.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.572+0000 7f69c9d9b700 1 -- 172.21.15.31:0/4056218179 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69c4159920 msgr2=0x7f69c414dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:15.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.572+0000 7f69c9d9b700 1 --2- 172.21.15.31:0/4056218179 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69c4159920 0x7f69c414dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:15.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.572+0000 7f69c9d9b700 1 -- 172.21.15.31:0/4056218179 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69b4009420 con 0x7f69c4159550 2023-10-06T02:23:15.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.572+0000 7f69c9d9b700 1 --2- 172.21.15.31:0/4056218179 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69c4159550 0x7f69c4149440 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f69c000c7c0 tx=0x7f69c0007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:15.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.572+0000 7f69b9ffb700 1 -- 172.21.15.31:0/4056218179 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c0001520 con 0x7f69c4159550 2023-10-06T02:23:15.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.572+0000 7f69b9ffb700 1 -- 172.21.15.31:0/4056218179 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69c00016c0 con 0x7f69c4159550 2023-10-06T02:23:15.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.572+0000 7f69cad9d700 1 -- 172.21.15.31:0/4056218179 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69c414b700 con 0x7f69c4159550 2023-10-06T02:23:15.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.572+0000 7f69b9ffb700 1 -- 172.21.15.31:0/4056218179 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c0001a80 con 0x7f69c4159550 2023-10-06T02:23:15.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.572+0000 7f69cad9d700 1 -- 172.21.15.31:0/4056218179 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69c40c9240 con 0x7f69c4159550 2023-10-06T02:23:15.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.573+0000 7f69cad9d700 1 -- 172.21.15.31:0/4056218179 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69c404c3c0 con 0x7f69c4159550 2023-10-06T02:23:15.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.574+0000 7f69b9ffb700 1 -- 172.21.15.31:0/4056218179 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f69c001b400 con 0x7f69c4159550 2023-10-06T02:23:15.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.575+0000 7f69b9ffb700 1 --2- 172.21.15.31:0/4056218179 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f69a8066f30 0x7f69a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:15.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.575+0000 7f69b9ffb700 1 -- 172.21.15.31:0/4056218179 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f69c0084470 con 0x7f69c4159550 2023-10-06T02:23:15.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.575+0000 7f69c959a700 1 --2- 172.21.15.31:0/4056218179 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f69a8066f30 0x7f69a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:15.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.576+0000 7f69c959a700 1 --2- 172.21.15.31:0/4056218179 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f69a8066f30 0x7f69a80693e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f69b40030c0 tx=0x7f69b4016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:15.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.579+0000 7f69b9ffb700 1 -- 172.21.15.31:0/4056218179 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f69c00514e0 con 0x7f69c4159550 2023-10-06T02:23:15.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.864+0000 7f69cad9d700 1 -- 172.21.15.31:0/4056218179 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f69c406e7c0 con 0x7f69c4159550 2023-10-06T02:23:15.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.864+0000 7f69b9ffb700 1 -- 172.21.15.31:0/4056218179 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1110 (secure 0 0 0) 0x7f69c0054b00 con 0x7f69c4159550 2023-10-06T02:23:15.865 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:23:15.865 INFO:teuthology.orchestra.run.smithi031.stdout:{"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":35,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":23,"num_osds":8,"num_up_osds":8,"osd_up_since":1696558972,"num_in_osds":8,"osd_in_since":1696558963,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":64},{"state_name":"creating+peering","count":32},{"state_name":"active+clean","count":9}],"num_pgs":105,"num_pools":4,"num_objects":2,"data_bytes":590368,"bytes_used":2438111232,"bytes_avail":770656002048,"bytes_total":773094113280,"unknown_pgs_ratio":0.60952383279800415,"inactive_pgs_ratio":0.30476191639900208},"fsmap":{"epoch":8,"id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"a","status":"up:active","gid":4297}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2023-10-06T02:22:40.183828+0000","services":{}},"progress_events":{}} 2023-10-06T02:23:15.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.867+0000 7f69cad9d700 1 -- 172.21.15.31:0/4056218179 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f69a8066f30 msgr2=0x7f69a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:15.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.867+0000 7f69cad9d700 1 --2- 172.21.15.31:0/4056218179 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f69a8066f30 0x7f69a80693e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f69b40030c0 tx=0x7f69b4016000 comp rx=0 tx=0).stop 2023-10-06T02:23:15.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.867+0000 7f69cad9d700 1 -- 172.21.15.31:0/4056218179 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69c4159550 msgr2=0x7f69c4149440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:15.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.867+0000 7f69cad9d700 1 --2- 172.21.15.31:0/4056218179 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69c4159550 0x7f69c4149440 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f69c000c7c0 tx=0x7f69c0007dd0 comp rx=0 tx=0).stop 2023-10-06T02:23:15.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.867+0000 7f69cad9d700 1 -- 172.21.15.31:0/4056218179 shutdown_connections 2023-10-06T02:23:15.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.867+0000 7f69cad9d700 1 --2- 172.21.15.31:0/4056218179 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f69a8066f30 0x7f69a80693e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:15.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.867+0000 7f69cad9d700 1 --2- 172.21.15.31:0/4056218179 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f69c415dfd0 0x7f69c414d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:15.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.867+0000 7f69cad9d700 1 --2- 172.21.15.31:0/4056218179 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69c4159550 0x7f69c4149440 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:15.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.867+0000 7f69cad9d700 1 --2- 172.21.15.31:0/4056218179 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69c4159920 0x7f69c414dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:15.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.867+0000 7f69cad9d700 1 -- 172.21.15.31:0/4056218179 >> 172.21.15.31:0/4056218179 conn(0x7f69c406ce80 msgr2=0x7f69c406e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:15.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.867+0000 7f69cad9d700 1 -- 172.21.15.31:0/4056218179 shutdown_connections 2023-10-06T02:23:15.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:15.867+0000 7f69cad9d700 1 -- 172.21.15.31:0/4056218179 wait complete. 2023-10-06T02:23:15.884 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-06T02:23:15.885 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T02:23:16.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:16.072+0000 7fa75f59d700 1 Processor -- start 2023-10-06T02:23:16.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:16.072+0000 7fa75f59d700 1 -- start start 2023-10-06T02:23:16.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:16.073+0000 7fa75f59d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa75806aaa0 0x7fa75815a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:16.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:16.073+0000 7fa75f59d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa75815ae70 con 0x7fa75813fb20 2023-10-06T02:23:16.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:16.074+0000 7fa75f59d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa75806af60 con 0x7fa75813fef0 2023-10-06T02:23:16.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:16.074+0000 7fa75f59d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa75806b240 con 0x7fa75806aaa0 2023-10-06T02:23:16.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:16.074+0000 7fa75e59b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fa75813fb20 0x7fa7580c7990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56170/0 (socket says 172.21.15.31:56170) 2023-10-06T02:23:16.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:16.074+0000 7fa75e59b700 1 -- 172.21.15.31:0/2460032831 learned_addr learned my addr 172.21.15.31:0/2460032831 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:23:16.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:16.074+0000 7fa75ed9c700 1 --2- 172.21.15.31:0/2460032831 >> v2:172.21.15.31:3300/0 conn(0x7fa75806aaa0 0x7fa75815a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:16.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:16.074+0000 7fa75d599700 1 -- 172.21.15.31:0/2460032831 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 83427091 0 0) 0x7fa754001110 con 0x7fa75813fb20 2023-10-06T02:23:16.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:16.074+0000 7fa75d599700 1 -- 172.21.15.31:0/2460032831 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa744003570 con 0x7fa75813fb20 2023-10-06T02:23:16.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:16.074+0000 7fa75ed9c700 1 -- 172.21.15.31:0/2460032831 >> v1:172.21.15.62:6789/0 conn(0x7fa75813fef0 legacy=0x7fa7580c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:16.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:16.074+0000 7fa75ed9c700 1 -- 172.21.15.31:0/2460032831 >> v1:172.21.15.31:6789/0 conn(0x7fa75813fb20 legacy=0x7fa7580c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:16.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:16.074+0000 7fa75ed9c700 1 -- 172.21.15.31:0/2460032831 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa75806b520 con 0x7fa75806aaa0 2023-10-06T02:23:16.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:16.075+0000 7fa75ed9c700 1 --2- 172.21.15.31:0/2460032831 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa75806aaa0 0x7fa75815a930 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa750009e60 tx=0x7fa750009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=831a7a5c7246c49d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:16.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:16.075+0000 7fa75d599700 1 -- 172.21.15.31:0/2460032831 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa75000e070 con 0x7fa75806aaa0 2023-10-06T02:23:16.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:16.075+0000 7fa75d599700 1 -- 172.21.15.31:0/2460032831 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa75000bb60 con 0x7fa75806aaa0 2023-10-06T02:23:16.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:16.075+0000 7fa75d599700 1 -- 172.21.15.31:0/2460032831 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa750012430 con 0x7fa75806aaa0 2023-10-06T02:23:16.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:16.075+0000 7fa75f59d700 1 -- 172.21.15.31:0/2460032831 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa75806aaa0 msgr2=0x7fa75815a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:16.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:16.075+0000 7fa75f59d700 1 --2- 172.21.15.31:0/2460032831 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa75806aaa0 0x7fa75815a930 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa750009e60 tx=0x7fa750009780 comp rx=0 tx=0).stop 2023-10-06T02:23:16.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:16.075+0000 7fa75f59d700 1 -- 172.21.15.31:0/2460032831 shutdown_connections 2023-10-06T02:23:16.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:16.075+0000 7fa75f59d700 1 --2- 172.21.15.31:0/2460032831 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa75806aaa0 0x7fa75815a930 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:16.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:16.075+0000 7fa75f59d700 1 -- 172.21.15.31:0/2460032831 >> 172.21.15.31:0/2460032831 conn(0x7fa75806d170 msgr2=0x7fa7580cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:16.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:16.075+0000 7fa75f59d700 1 -- 172.21.15.31:0/2460032831 shutdown_connections 2023-10-06T02:23:16.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:16.075+0000 7fa75f59d700 1 -- 172.21.15.31:0/2460032831 wait complete. 2023-10-06T02:23:16.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:16.077+0000 7fa75f59d700 1 Processor -- start 2023-10-06T02:23:16.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:16.077+0000 7fa75f59d700 1 -- start start 2023-10-06T02:23:16.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:16.078+0000 7fa75f59d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa75806aaa0 0x7fa7581590b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:16.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:16.078+0000 7fa75f59d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa75813fb20 0x7fa7580ae030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:16.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:16.078+0000 7fa75f59d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa75813fef0 0x7fa758138470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:16.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:16.078+0000 7fa75f59d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7581562a0 con 0x7fa75806aaa0 2023-10-06T02:23:16.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:16.078+0000 7fa75f59d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa758158c20 con 0x7fa75813fb20 2023-10-06T02:23:16.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:16.078+0000 7fa75f59d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7580ad370 con 0x7fa75813fef0 2023-10-06T02:23:16.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:16.078+0000 7fa75e59b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa75806aaa0 0x7fa7581590b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:16.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:16.078+0000 7fa75e59b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa75806aaa0 0x7fa7581590b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44418/0 (socket says 172.21.15.31:44418) 2023-10-06T02:23:16.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:16.078+0000 7fa75e59b700 1 -- 172.21.15.31:0/3315789682 learned_addr learned my addr 172.21.15.31:0/3315789682 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:16.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:16.078+0000 7fa75dd9a700 1 --2- 172.21.15.31:0/3315789682 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa75813fb20 0x7fa7580ae030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:16.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:16.078+0000 7fa75ed9c700 1 --2- 172.21.15.31:0/3315789682 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa75813fef0 0x7fa758138470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:16.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:16.078+0000 7fa75e59b700 1 -- 172.21.15.31:0/3315789682 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa75813fef0 msgr2=0x7fa758138470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:16.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:16.078+0000 7fa75e59b700 1 --2- 172.21.15.31:0/3315789682 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa75813fef0 0x7fa758138470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:16.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:16.078+0000 7fa75e59b700 1 -- 172.21.15.31:0/3315789682 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa75813fb20 msgr2=0x7fa7580ae030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:16.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:16.078+0000 7fa75e59b700 1 --2- 172.21.15.31:0/3315789682 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa75813fb20 0x7fa7580ae030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:16.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:16.079+0000 7fa75e59b700 1 -- 172.21.15.31:0/3315789682 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa750009a20 con 0x7fa75806aaa0 2023-10-06T02:23:16.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:16.079+0000 7fa75ed9c700 1 --2- 172.21.15.31:0/3315789682 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa75813fef0 0x7fa758138470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:23:16.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:16.079+0000 7fa75e59b700 1 --2- 172.21.15.31:0/3315789682 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa75806aaa0 0x7fa7581590b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa75400c7c0 tx=0x7fa754007c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:16.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:16.079+0000 7fa74e7fc700 1 -- 172.21.15.31:0/3315789682 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa754005b30 con 0x7fa75806aaa0 2023-10-06T02:23:16.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:16.079+0000 7fa74e7fc700 1 -- 172.21.15.31:0/3315789682 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa754005cb0 con 0x7fa75806aaa0 2023-10-06T02:23:16.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:16.079+0000 7fa75f59d700 1 -- 172.21.15.31:0/3315789682 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa758158610 con 0x7fa75806aaa0 2023-10-06T02:23:16.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:16.079+0000 7fa74e7fc700 1 -- 172.21.15.31:0/3315789682 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa754007320 con 0x7fa75806aaa0 2023-10-06T02:23:16.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:16.079+0000 7fa75f59d700 1 -- 172.21.15.31:0/3315789682 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7581588b0 con 0x7fa75806aaa0 2023-10-06T02:23:16.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:16.080+0000 7fa75f59d700 1 -- 172.21.15.31:0/3315789682 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa758158480 con 0x7fa75806aaa0 2023-10-06T02:23:16.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:16.081+0000 7fa74e7fc700 1 -- 172.21.15.31:0/3315789682 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa75401b3b0 con 0x7fa75806aaa0 2023-10-06T02:23:16.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:16.082+0000 7fa74e7fc700 1 --2- 172.21.15.31:0/3315789682 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa73c066f30 0x7fa73c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:16.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:16.083+0000 7fa74e7fc700 1 -- 172.21.15.31:0/3315789682 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa754083a80 con 0x7fa75806aaa0 2023-10-06T02:23:16.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:16.083+0000 7fa75dd9a700 1 --2- 172.21.15.31:0/3315789682 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa73c066f30 0x7fa73c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:16.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:16.083+0000 7fa75dd9a700 1 --2- 172.21.15.31:0/3315789682 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa73c066f30 0x7fa73c0693e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa75816aad0 tx=0x7fa748007eb0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:16.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:16.087+0000 7fa74e7fc700 1 -- 172.21.15.31:0/3315789682 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa754050af0 con 0x7fa75806aaa0 2023-10-06T02:23:16.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:16.283+0000 7fa75f59d700 1 -- 172.21.15.31:0/3315789682 --> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa758158d60 con 0x7fa73c066f30 2023-10-06T02:23:16.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:16.292+0000 7fa74e7fc700 1 -- 172.21.15.31:0/3315789682 <== mgr.4107 v2:172.21.15.31:6832/85292 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286739 (secure 0 0 0) 0x7fa748002770 con 0x7fa73c066f30 2023-10-06T02:23:16.296 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:23:16.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:16.296+0000 7fa75f59d700 1 -- 172.21.15.31:0/3315789682 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa73c066f30 msgr2=0x7fa73c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:16.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:16.296+0000 7fa75f59d700 1 --2- 172.21.15.31:0/3315789682 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa73c066f30 0x7fa73c0693e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa75816aad0 tx=0x7fa748007eb0 comp rx=0 tx=0).stop 2023-10-06T02:23:16.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:16.296+0000 7fa75f59d700 1 -- 172.21.15.31:0/3315789682 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa75806aaa0 msgr2=0x7fa7581590b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:16.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:16.296+0000 7fa75f59d700 1 --2- 172.21.15.31:0/3315789682 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa75806aaa0 0x7fa7581590b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa75400c7c0 tx=0x7fa754007c70 comp rx=0 tx=0).stop 2023-10-06T02:23:16.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:16.296+0000 7fa75f59d700 1 -- 172.21.15.31:0/3315789682 shutdown_connections 2023-10-06T02:23:16.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:16.296+0000 7fa75f59d700 1 --2- 172.21.15.31:0/3315789682 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa73c066f30 0x7fa73c0693e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:16.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:16.296+0000 7fa75f59d700 1 --2- 172.21.15.31:0/3315789682 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa75813fef0 0x7fa758138470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:16.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:16.296+0000 7fa75f59d700 1 --2- 172.21.15.31:0/3315789682 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa75806aaa0 0x7fa7581590b0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:16.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:16.296+0000 7fa75f59d700 1 --2- 172.21.15.31:0/3315789682 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa75813fb20 0x7fa7580ae030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:16.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:16.296+0000 7fa75f59d700 1 -- 172.21.15.31:0/3315789682 >> 172.21.15.31:0/3315789682 conn(0x7fa75806d170 msgr2=0x7fa75813e780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:16.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:16.296+0000 7fa75f59d700 1 -- 172.21.15.31:0/3315789682 shutdown_connections 2023-10-06T02:23:16.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:16.296+0000 7fa75f59d700 1 -- 172.21.15.31:0/3315789682 wait complete. 2023-10-06T02:23:16.299 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-10-06T02:23:16.314 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2023-10-06T02:23:15.118363+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":203,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381416,"kb_used_data":3112,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752593304,"statfs":{"total":773094113280,"available":770655543296,"internally_reserved":0,"allocated":3186688,"data_stored":1899900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.101512"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.985694+0000","last_change":"2023-10-06T02:23:09.982458+0000","last_active":"2023-10-06T02:23:09.985694+0000","last_peered":"2023-10-06T02:23:09.985694+0000","last_clean":"2023-10-06T02:23:09.985694+0000","last_became_active":"2023-10-06T02:23:09.982140+0000","last_became_peered":"2023-10-06T02:23:09.982140+0000","last_unstale":"2023-10-06T02:23:09.985694+0000","last_undegraded":"2023-10-06T02:23:09.985694+0000","last_fullsized":"2023-10-06T02:23:09.985694+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:07:57.730382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.999489+0000","last_change":"2023-10-06T02:23:09.996757+0000","last_active":"2023-10-06T02:23:09.999489+0000","last_peered":"2023-10-06T02:23:09.999489+0000","last_clean":"2023-10-06T02:23:09.999489+0000","last_became_active":"2023-10-06T02:23:09.996404+0000","last_became_peered":"2023-10-06T02:23:09.996404+0000","last_unstale":"2023-10-06T02:23:09.999489+0000","last_undegraded":"2023-10-06T02:23:09.999489+0000","last_fullsized":"2023-10-06T02:23:09.999489+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:39:15.275881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.010136+0000","last_change":"2023-10-06T02:23:09.999568+0000","last_active":"2023-10-06T02:23:10.010136+0000","last_peered":"2023-10-06T02:23:10.010136+0000","last_clean":"2023-10-06T02:23:10.010136+0000","last_became_active":"2023-10-06T02:23:09.999033+0000","last_became_peered":"2023-10-06T02:23:09.999033+0000","last_unstale":"2023-10-06T02:23:10.010136+0000","last_undegraded":"2023-10-06T02:23:10.010136+0000","last_fullsized":"2023-10-06T02:23:10.010136+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:23:26.580218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.996019+0000","last_change":"2023-10-06T02:23:09.995088+0000","last_active":"2023-10-06T02:23:09.996019+0000","last_peered":"2023-10-06T02:23:09.996019+0000","last_clean":"2023-10-06T02:23:09.996019+0000","last_became_active":"2023-10-06T02:23:09.994785+0000","last_became_peered":"2023-10-06T02:23:09.994785+0000","last_unstale":"2023-10-06T02:23:09.996019+0000","last_undegraded":"2023-10-06T02:23:09.996019+0000","last_fullsized":"2023-10-06T02:23:09.996019+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:21:33.737708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.994230+0000","last_change":"2023-10-06T02:23:09.986669+0000","last_active":"2023-10-06T02:23:09.994230+0000","last_peered":"2023-10-06T02:23:09.994230+0000","last_clean":"2023-10-06T02:23:09.994230+0000","last_became_active":"2023-10-06T02:23:09.986141+0000","last_became_peered":"2023-10-06T02:23:09.986141+0000","last_unstale":"2023-10-06T02:23:09.994230+0000","last_undegraded":"2023-10-06T02:23:09.994230+0000","last_fullsized":"2023-10-06T02:23:09.994230+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:39:15.275881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.010199+0000","last_change":"2023-10-06T02:23:10.000405+0000","last_active":"2023-10-06T02:23:10.010199+0000","last_peered":"2023-10-06T02:23:10.010199+0000","last_clean":"2023-10-06T02:23:10.010199+0000","last_became_active":"2023-10-06T02:23:10.000015+0000","last_became_peered":"2023-10-06T02:23:10.000015+0000","last_unstale":"2023-10-06T02:23:10.010199+0000","last_undegraded":"2023-10-06T02:23:10.010199+0000","last_fullsized":"2023-10-06T02:23:10.010199+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:39:15.275881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.009997+0000","last_change":"2023-10-06T02:23:10.004359+0000","last_active":"2023-10-06T02:23:10.009997+0000","last_peered":"2023-10-06T02:23:10.009997+0000","last_clean":"2023-10-06T02:23:10.009997+0000","last_became_active":"2023-10-06T02:23:10.003873+0000","last_became_peered":"2023-10-06T02:23:10.003873+0000","last_unstale":"2023-10-06T02:23:10.009997+0000","last_undegraded":"2023-10-06T02:23:10.009997+0000","last_fullsized":"2023-10-06T02:23:10.009997+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:44:41.608418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.998258+0000","last_change":"2023-10-06T02:23:09.994276+0000","last_active":"2023-10-06T02:23:09.998258+0000","last_peered":"2023-10-06T02:23:09.998258+0000","last_clean":"2023-10-06T02:23:09.998258+0000","last_became_active":"2023-10-06T02:23:09.993946+0000","last_became_peered":"2023-10-06T02:23:09.993946+0000","last_unstale":"2023-10-06T02:23:09.998258+0000","last_undegraded":"2023-10-06T02:23:09.998258+0000","last_fullsized":"2023-10-06T02:23:09.998258+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:19:29.462977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.997243+0000","last_change":"2023-10-06T02:23:09.993971+0000","last_active":"2023-10-06T02:23:09.997243+0000","last_peered":"2023-10-06T02:23:09.997243+0000","last_clean":"2023-10-06T02:23:09.997243+0000","last_became_active":"2023-10-06T02:23:09.993547+0000","last_became_peered":"2023-10-06T02:23:09.993547+0000","last_unstale":"2023-10-06T02:23:09.997243+0000","last_undegraded":"2023-10-06T02:23:09.997243+0000","last_fullsized":"2023-10-06T02:23:09.997243+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:19:29.462977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.999469+0000","last_change":"2023-10-06T02:23:09.995426+0000","last_active":"2023-10-06T02:23:09.999469+0000","last_peered":"2023-10-06T02:23:09.999469+0000","last_clean":"2023-10-06T02:23:09.999469+0000","last_became_active":"2023-10-06T02:23:09.995067+0000","last_became_peered":"2023-10-06T02:23:09.995067+0000","last_unstale":"2023-10-06T02:23:09.999469+0000","last_undegraded":"2023-10-06T02:23:09.999469+0000","last_fullsized":"2023-10-06T02:23:09.999469+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:50:19.168882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.010257+0000","last_change":"2023-10-06T02:23:10.001093+0000","last_active":"2023-10-06T02:23:10.010257+0000","last_peered":"2023-10-06T02:23:10.010257+0000","last_clean":"2023-10-06T02:23:10.010257+0000","last_became_active":"2023-10-06T02:23:10.000755+0000","last_became_peered":"2023-10-06T02:23:10.000755+0000","last_unstale":"2023-10-06T02:23:10.010257+0000","last_undegraded":"2023-10-06T02:23:10.010257+0000","last_fullsized":"2023-10-06T02:23:10.010257+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:43:02.156237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.998611+0000","last_change":"2023-10-06T02:23:09.993328+0000","last_active":"2023-10-06T02:23:09.998611+0000","last_peered":"2023-10-06T02:23:09.998611+0000","last_clean":"2023-10-06T02:23:09.998611+0000","last_became_active":"2023-10-06T02:23:09.993076+0000","last_became_peered":"2023-10-06T02:23:09.993076+0000","last_unstale":"2023-10-06T02:23:09.998611+0000","last_undegraded":"2023-10-06T02:23:09.998611+0000","last_fullsized":"2023-10-06T02:23:09.998611+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:58.521611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.986182+0000","last_change":"2023-10-06T02:23:09.982673+0000","last_active":"2023-10-06T02:23:09.986182+0000","last_peered":"2023-10-06T02:23:09.986182+0000","last_clean":"2023-10-06T02:23:09.986182+0000","last_became_active":"2023-10-06T02:23:09.982370+0000","last_became_peered":"2023-10-06T02:23:09.982370+0000","last_unstale":"2023-10-06T02:23:09.986182+0000","last_undegraded":"2023-10-06T02:23:09.986182+0000","last_fullsized":"2023-10-06T02:23:09.986182+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:17:33.441972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.985297+0000","last_change":"2023-10-06T02:23:09.982769+0000","last_active":"2023-10-06T02:23:09.985297+0000","last_peered":"2023-10-06T02:23:09.985297+0000","last_clean":"2023-10-06T02:23:09.985297+0000","last_became_active":"2023-10-06T02:23:09.982450+0000","last_became_peered":"2023-10-06T02:23:09.982450+0000","last_unstale":"2023-10-06T02:23:09.985297+0000","last_undegraded":"2023-10-06T02:23:09.985297+0000","last_fullsized":"2023-10-06T02:23:09.985297+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:12:16.306131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.010017+0000","last_change":"2023-10-06T02:23:10.004452+0000","last_active":"2023-10-06T02:23:10.010017+0000","last_peered":"2023-10-06T02:23:10.010017+0000","last_clean":"2023-10-06T02:23:10.010017+0000","last_became_active":"2023-10-06T02:23:10.004184+0000","last_became_peered":"2023-10-06T02:23:10.004184+0000","last_unstale":"2023-10-06T02:23:10.010017+0000","last_undegraded":"2023-10-06T02:23:10.010017+0000","last_fullsized":"2023-10-06T02:23:10.010017+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:50:19.168882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.010069+0000","last_change":"2023-10-06T02:23:09.998790+0000","last_active":"2023-10-06T02:23:10.010069+0000","last_peered":"2023-10-06T02:23:10.010069+0000","last_clean":"2023-10-06T02:23:10.010069+0000","last_became_active":"2023-10-06T02:23:09.998476+0000","last_became_peered":"2023-10-06T02:23:09.998476+0000","last_unstale":"2023-10-06T02:23:10.010069+0000","last_undegraded":"2023-10-06T02:23:10.010069+0000","last_fullsized":"2023-10-06T02:23:10.010069+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:19:29.462977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.997263+0000","last_change":"2023-10-06T02:23:09.994379+0000","last_active":"2023-10-06T02:23:09.997263+0000","last_peered":"2023-10-06T02:23:09.997263+0000","last_clean":"2023-10-06T02:23:09.997263+0000","last_became_active":"2023-10-06T02:23:09.994031+0000","last_became_peered":"2023-10-06T02:23:09.994031+0000","last_unstale":"2023-10-06T02:23:09.997263+0000","last_undegraded":"2023-10-06T02:23:09.997263+0000","last_fullsized":"2023-10-06T02:23:09.997263+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:12.998563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.994477+0000","last_change":"2023-10-06T02:23:09.988087+0000","last_active":"2023-10-06T02:23:09.994477+0000","last_peered":"2023-10-06T02:23:09.994477+0000","last_clean":"2023-10-06T02:23:09.994477+0000","last_became_active":"2023-10-06T02:23:09.987741+0000","last_became_peered":"2023-10-06T02:23:09.987741+0000","last_unstale":"2023-10-06T02:23:09.994477+0000","last_undegraded":"2023-10-06T02:23:09.994477+0000","last_fullsized":"2023-10-06T02:23:09.994477+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:05:24.047504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.987646+0000","last_change":"2023-10-06T02:23:09.983568+0000","last_active":"2023-10-06T02:23:09.987646+0000","last_peered":"2023-10-06T02:23:09.987646+0000","last_clean":"2023-10-06T02:23:09.987646+0000","last_became_active":"2023-10-06T02:23:09.983273+0000","last_became_peered":"2023-10-06T02:23:09.983273+0000","last_unstale":"2023-10-06T02:23:09.987646+0000","last_undegraded":"2023-10-06T02:23:09.987646+0000","last_fullsized":"2023-10-06T02:23:09.987646+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:44:16.553231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.009997+0000","last_change":"2023-10-06T02:23:09.997248+0000","last_active":"2023-10-06T02:23:10.009997+0000","last_peered":"2023-10-06T02:23:10.009997+0000","last_clean":"2023-10-06T02:23:10.009997+0000","last_became_active":"2023-10-06T02:23:09.996897+0000","last_became_peered":"2023-10-06T02:23:09.996897+0000","last_unstale":"2023-10-06T02:23:10.009997+0000","last_undegraded":"2023-10-06T02:23:10.009997+0000","last_fullsized":"2023-10-06T02:23:10.009997+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:00:45.442861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.994041+0000","last_change":"2023-10-06T02:23:09.991324+0000","last_active":"2023-10-06T02:23:09.994041+0000","last_peered":"2023-10-06T02:23:09.994041+0000","last_clean":"2023-10-06T02:23:09.994041+0000","last_became_active":"2023-10-06T02:23:09.991018+0000","last_became_peered":"2023-10-06T02:23:09.991018+0000","last_unstale":"2023-10-06T02:23:09.994041+0000","last_undegraded":"2023-10-06T02:23:09.994041+0000","last_fullsized":"2023-10-06T02:23:09.994041+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:43:01.259350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.004545+0000","last_change":"2023-10-06T02:23:10.000917+0000","last_active":"2023-10-06T02:23:10.004545+0000","last_peered":"2023-10-06T02:23:10.004545+0000","last_clean":"2023-10-06T02:23:10.004545+0000","last_became_active":"2023-10-06T02:23:10.000580+0000","last_became_peered":"2023-10-06T02:23:10.000580+0000","last_unstale":"2023-10-06T02:23:10.004545+0000","last_undegraded":"2023-10-06T02:23:10.004545+0000","last_fullsized":"2023-10-06T02:23:10.004545+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:50:19.168882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.998825+0000","last_change":"2023-10-06T02:23:09.995748+0000","last_active":"2023-10-06T02:23:09.998825+0000","last_peered":"2023-10-06T02:23:09.998825+0000","last_clean":"2023-10-06T02:23:09.998825+0000","last_became_active":"2023-10-06T02:23:09.995433+0000","last_became_peered":"2023-10-06T02:23:09.995433+0000","last_unstale":"2023-10-06T02:23:09.998825+0000","last_undegraded":"2023-10-06T02:23:09.998825+0000","last_fullsized":"2023-10-06T02:23:09.998825+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:21.438589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.996641+0000","last_change":"2023-10-06T02:23:09.992834+0000","last_active":"2023-10-06T02:23:09.996641+0000","last_peered":"2023-10-06T02:23:09.996641+0000","last_clean":"2023-10-06T02:23:09.996641+0000","last_became_active":"2023-10-06T02:23:09.991971+0000","last_became_peered":"2023-10-06T02:23:09.991971+0000","last_unstale":"2023-10-06T02:23:09.996641+0000","last_undegraded":"2023-10-06T02:23:09.996641+0000","last_fullsized":"2023-10-06T02:23:09.996641+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:12.998563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.007651+0000","last_change":"2023-10-06T02:23:10.005469+0000","last_active":"2023-10-06T02:23:10.007651+0000","last_peered":"2023-10-06T02:23:10.007651+0000","last_clean":"2023-10-06T02:23:10.007651+0000","last_became_active":"2023-10-06T02:23:10.005132+0000","last_became_peered":"2023-10-06T02:23:10.005132+0000","last_unstale":"2023-10-06T02:23:10.007651+0000","last_undegraded":"2023-10-06T02:23:10.007651+0000","last_fullsized":"2023-10-06T02:23:10.007651+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:19:18.748586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.005833+0000","last_change":"2023-10-06T02:23:09.999977+0000","last_active":"2023-10-06T02:23:10.005833+0000","last_peered":"2023-10-06T02:23:10.005833+0000","last_clean":"2023-10-06T02:23:10.005833+0000","last_became_active":"2023-10-06T02:23:09.999733+0000","last_became_peered":"2023-10-06T02:23:09.999733+0000","last_unstale":"2023-10-06T02:23:10.005833+0000","last_undegraded":"2023-10-06T02:23:10.005833+0000","last_fullsized":"2023-10-06T02:23:10.005833+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:58:41.925466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.999279+0000","last_change":"2023-10-06T02:23:09.994580+0000","last_active":"2023-10-06T02:23:09.999279+0000","last_peered":"2023-10-06T02:23:09.999279+0000","last_clean":"2023-10-06T02:23:09.999279+0000","last_became_active":"2023-10-06T02:23:09.993941+0000","last_became_peered":"2023-10-06T02:23:09.993941+0000","last_unstale":"2023-10-06T02:23:09.999279+0000","last_undegraded":"2023-10-06T02:23:09.999279+0000","last_fullsized":"2023-10-06T02:23:09.999279+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:43:01.259350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.043858+0000","last_change":"2023-10-06T02:23:09.048996+0000","last_active":"2023-10-06T02:23:10.043858+0000","last_peered":"2023-10-06T02:23:10.043858+0000","last_clean":"2023-10-06T02:23:10.043858+0000","last_became_active":"2023-10-06T02:23:09.048662+0000","last_became_peered":"2023-10-06T02:23:09.048662+0000","last_unstale":"2023-10-06T02:23:10.043858+0000","last_undegraded":"2023-10-06T02:23:10.043858+0000","last_fullsized":"2023-10-06T02:23:10.043858+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:58.976414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.004959+0000","last_change":"2023-10-06T02:23:10.001176+0000","last_active":"2023-10-06T02:23:10.004959+0000","last_peered":"2023-10-06T02:23:10.004959+0000","last_clean":"2023-10-06T02:23:10.004959+0000","last_became_active":"2023-10-06T02:23:10.000887+0000","last_became_peered":"2023-10-06T02:23:10.000887+0000","last_unstale":"2023-10-06T02:23:10.004959+0000","last_undegraded":"2023-10-06T02:23:10.004959+0000","last_fullsized":"2023-10-06T02:23:10.004959+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:43:02.156237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.998513+0000","last_change":"2023-10-06T02:23:09.048083+0000","last_active":"2023-10-06T02:23:09.998513+0000","last_peered":"2023-10-06T02:23:09.998513+0000","last_clean":"2023-10-06T02:23:09.998513+0000","last_became_active":"2023-10-06T02:23:09.047701+0000","last_became_peered":"2023-10-06T02:23:09.047701+0000","last_unstale":"2023-10-06T02:23:09.998513+0000","last_undegraded":"2023-10-06T02:23:09.998513+0000","last_fullsized":"2023-10-06T02:23:09.998513+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:17:32.434412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.994267+0000","last_change":"2023-10-06T02:23:09.988810+0000","last_active":"2023-10-06T02:23:09.994267+0000","last_peered":"2023-10-06T02:23:09.994267+0000","last_clean":"2023-10-06T02:23:09.994267+0000","last_became_active":"2023-10-06T02:23:09.988419+0000","last_became_peered":"2023-10-06T02:23:09.988419+0000","last_unstale":"2023-10-06T02:23:09.994267+0000","last_undegraded":"2023-10-06T02:23:09.994267+0000","last_fullsized":"2023-10-06T02:23:09.994267+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:43:01.259350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.976287+0000","last_change":"2023-10-06T02:23:09.044642+0000","last_active":"2023-10-06T02:23:09.976287+0000","last_peered":"2023-10-06T02:23:09.976287+0000","last_clean":"2023-10-06T02:23:09.976287+0000","last_became_active":"2023-10-06T02:23:09.044224+0000","last_became_peered":"2023-10-06T02:23:09.044224+0000","last_unstale":"2023-10-06T02:23:09.976287+0000","last_undegraded":"2023-10-06T02:23:09.976287+0000","last_fullsized":"2023-10-06T02:23:09.976287+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:31:52.666890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.986940+0000","last_change":"2023-10-06T02:23:09.985837+0000","last_active":"2023-10-06T02:23:09.986940+0000","last_peered":"2023-10-06T02:23:09.986940+0000","last_clean":"2023-10-06T02:23:09.986940+0000","last_became_active":"2023-10-06T02:23:09.985447+0000","last_became_peered":"2023-10-06T02:23:09.985447+0000","last_unstale":"2023-10-06T02:23:09.986940+0000","last_undegraded":"2023-10-06T02:23:09.986940+0000","last_fullsized":"2023-10-06T02:23:09.986940+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:31:53.674450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.979513+0000","last_change":"2023-10-06T02:23:09.040730+0000","last_active":"2023-10-06T02:23:09.979513+0000","last_peered":"2023-10-06T02:23:09.979513+0000","last_clean":"2023-10-06T02:23:09.979513+0000","last_became_active":"2023-10-06T02:23:09.040327+0000","last_became_peered":"2023-10-06T02:23:09.040327+0000","last_unstale":"2023-10-06T02:23:09.979513+0000","last_undegraded":"2023-10-06T02:23:09.979513+0000","last_fullsized":"2023-10-06T02:23:09.979513+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:07:56.722823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.998851+0000","last_change":"2023-10-06T02:23:09.995950+0000","last_active":"2023-10-06T02:23:09.998851+0000","last_peered":"2023-10-06T02:23:09.998851+0000","last_clean":"2023-10-06T02:23:09.998851+0000","last_became_active":"2023-10-06T02:23:09.995679+0000","last_became_peered":"2023-10-06T02:23:09.995679+0000","last_unstale":"2023-10-06T02:23:09.998851+0000","last_undegraded":"2023-10-06T02:23:09.998851+0000","last_fullsized":"2023-10-06T02:23:09.998851+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:23:26.580218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.976082+0000","last_change":"2023-10-06T02:23:09.040245+0000","last_active":"2023-10-06T02:23:09.976082+0000","last_peered":"2023-10-06T02:23:09.976082+0000","last_clean":"2023-10-06T02:23:09.976082+0000","last_became_active":"2023-10-06T02:23:09.039660+0000","last_became_peered":"2023-10-06T02:23:09.039660+0000","last_unstale":"2023-10-06T02:23:09.976082+0000","last_undegraded":"2023-10-06T02:23:09.976082+0000","last_fullsized":"2023-10-06T02:23:09.976082+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:57.514051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.994925+0000","last_change":"2023-10-06T02:23:09.992417+0000","last_active":"2023-10-06T02:23:09.994925+0000","last_peered":"2023-10-06T02:23:09.994925+0000","last_clean":"2023-10-06T02:23:09.994925+0000","last_became_active":"2023-10-06T02:23:09.992155+0000","last_became_peered":"2023-10-06T02:23:09.992155+0000","last_unstale":"2023-10-06T02:23:09.994925+0000","last_undegraded":"2023-10-06T02:23:09.994925+0000","last_fullsized":"2023-10-06T02:23:09.994925+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:39:15.275881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"22'3","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.044297+0000","last_change":"2023-10-06T02:23:09.046655+0000","last_active":"2023-10-06T02:23:10.044297+0000","last_peered":"2023-10-06T02:23:10.044297+0000","last_clean":"2023-10-06T02:23:10.044297+0000","last_became_active":"2023-10-06T02:23:09.046335+0000","last_became_peered":"2023-10-06T02:23:09.046335+0000","last_unstale":"2023-10-06T02:23:10.044297+0000","last_undegraded":"2023-10-06T02:23:10.044297+0000","last_fullsized":"2023-10-06T02:23:10.044297+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:53:00.225521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.007635+0000","last_change":"2023-10-06T02:23:10.005359+0000","last_active":"2023-10-06T02:23:10.007635+0000","last_peered":"2023-10-06T02:23:10.007635+0000","last_clean":"2023-10-06T02:23:10.007635+0000","last_became_active":"2023-10-06T02:23:10.005115+0000","last_became_peered":"2023-10-06T02:23:10.005115+0000","last_unstale":"2023-10-06T02:23:10.007635+0000","last_undegraded":"2023-10-06T02:23:10.007635+0000","last_fullsized":"2023-10-06T02:23:10.007635+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:29:00.771363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"22'6","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.052236+0000","last_change":"2023-10-06T02:23:09.046958+0000","last_active":"2023-10-06T02:23:10.052236+0000","last_peered":"2023-10-06T02:23:10.052236+0000","last_clean":"2023-10-06T02:23:10.052236+0000","last_became_active":"2023-10-06T02:23:09.044018+0000","last_became_peered":"2023-10-06T02:23:09.044018+0000","last_unstale":"2023-10-06T02:23:10.052236+0000","last_undegraded":"2023-10-06T02:23:10.052236+0000","last_fullsized":"2023-10-06T02:23:10.052236+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:41.039005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.992467+0000","last_change":"2023-10-06T02:23:09.987559+0000","last_active":"2023-10-06T02:23:09.992467+0000","last_peered":"2023-10-06T02:23:09.992467+0000","last_clean":"2023-10-06T02:23:09.992467+0000","last_became_active":"2023-10-06T02:23:09.987264+0000","last_became_peered":"2023-10-06T02:23:09.987264+0000","last_unstale":"2023-10-06T02:23:09.992467+0000","last_undegraded":"2023-10-06T02:23:09.992467+0000","last_fullsized":"2023-10-06T02:23:09.992467+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:44:41.608418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.049676+0000","last_change":"2023-10-06T02:23:09.048307+0000","last_active":"2023-10-06T02:23:10.049676+0000","last_peered":"2023-10-06T02:23:10.049676+0000","last_clean":"2023-10-06T02:23:10.049676+0000","last_became_active":"2023-10-06T02:23:09.047919+0000","last_became_peered":"2023-10-06T02:23:09.047919+0000","last_unstale":"2023-10-06T02:23:10.049676+0000","last_undegraded":"2023-10-06T02:23:10.049676+0000","last_fullsized":"2023-10-06T02:23:10.049676+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:28:59.763803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.004797+0000","last_change":"2023-10-06T02:23:10.001689+0000","last_active":"2023-10-06T02:23:10.004797+0000","last_peered":"2023-10-06T02:23:10.004797+0000","last_clean":"2023-10-06T02:23:10.004797+0000","last_became_active":"2023-10-06T02:23:10.001418+0000","last_became_peered":"2023-10-06T02:23:10.001418+0000","last_unstale":"2023-10-06T02:23:10.004797+0000","last_undegraded":"2023-10-06T02:23:10.004797+0000","last_fullsized":"2023-10-06T02:23:10.004797+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:53:54.469523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.977377+0000","last_change":"2023-10-06T02:23:09.041968+0000","last_active":"2023-10-06T02:23:09.977377+0000","last_peered":"2023-10-06T02:23:09.977377+0000","last_clean":"2023-10-06T02:23:09.977377+0000","last_became_active":"2023-10-06T02:23:09.041659+0000","last_became_peered":"2023-10-06T02:23:09.041659+0000","last_unstale":"2023-10-06T02:23:09.977377+0000","last_undegraded":"2023-10-06T02:23:09.977377+0000","last_fullsized":"2023-10-06T02:23:09.977377+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:44:15.545672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.992112+0000","last_change":"2023-10-06T02:23:09.989200+0000","last_active":"2023-10-06T02:23:09.992112+0000","last_peered":"2023-10-06T02:23:09.992112+0000","last_clean":"2023-10-06T02:23:09.992112+0000","last_became_active":"2023-10-06T02:23:09.988895+0000","last_became_peered":"2023-10-06T02:23:09.988895+0000","last_unstale":"2023-10-06T02:23:09.992112+0000","last_undegraded":"2023-10-06T02:23:09.992112+0000","last_fullsized":"2023-10-06T02:23:09.992112+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:44:41.608418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.984229+0000","last_change":"2023-10-06T02:23:09.048119+0000","last_active":"2023-10-06T02:23:09.984229+0000","last_peered":"2023-10-06T02:23:09.984229+0000","last_clean":"2023-10-06T02:23:09.984229+0000","last_became_active":"2023-10-06T02:23:09.047678+0000","last_became_peered":"2023-10-06T02:23:09.047678+0000","last_unstale":"2023-10-06T02:23:09.984229+0000","last_undegraded":"2023-10-06T02:23:09.984229+0000","last_fullsized":"2023-10-06T02:23:09.984229+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:42:11.693138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.997921+0000","last_change":"2023-10-06T02:23:09.994722+0000","last_active":"2023-10-06T02:23:09.997921+0000","last_peered":"2023-10-06T02:23:09.997921+0000","last_clean":"2023-10-06T02:23:09.997921+0000","last_became_active":"2023-10-06T02:23:09.994392+0000","last_became_peered":"2023-10-06T02:23:09.994392+0000","last_unstale":"2023-10-06T02:23:09.997921+0000","last_undegraded":"2023-10-06T02:23:09.997921+0000","last_fullsized":"2023-10-06T02:23:09.997921+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:21.438589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"22'8","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.049205+0000","last_change":"2023-10-06T02:23:09.034244+0000","last_active":"2023-10-06T02:23:10.049205+0000","last_peered":"2023-10-06T02:23:10.049205+0000","last_clean":"2023-10-06T02:23:10.049205+0000","last_became_active":"2023-10-06T02:23:09.033861+0000","last_became_peered":"2023-10-06T02:23:09.033861+0000","last_unstale":"2023-10-06T02:23:10.049205+0000","last_undegraded":"2023-10-06T02:23:10.049205+0000","last_fullsized":"2023-10-06T02:23:10.049205+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:41.039005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.990663+0000","last_change":"2023-10-06T02:23:09.988390+0000","last_active":"2023-10-06T02:23:09.990663+0000","last_peered":"2023-10-06T02:23:09.990663+0000","last_clean":"2023-10-06T02:23:09.990663+0000","last_became_active":"2023-10-06T02:23:09.988060+0000","last_became_peered":"2023-10-06T02:23:09.988060+0000","last_unstale":"2023-10-06T02:23:09.990663+0000","last_undegraded":"2023-10-06T02:23:09.990663+0000","last_fullsized":"2023-10-06T02:23:09.990663+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:43:01.259350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.044194+0000","last_change":"2023-10-06T02:23:09.048150+0000","last_active":"2023-10-06T02:23:10.044194+0000","last_peered":"2023-10-06T02:23:10.044194+0000","last_clean":"2023-10-06T02:23:10.044194+0000","last_became_active":"2023-10-06T02:23:09.047768+0000","last_became_peered":"2023-10-06T02:23:09.047768+0000","last_unstale":"2023-10-06T02:23:10.044194+0000","last_undegraded":"2023-10-06T02:23:10.044194+0000","last_fullsized":"2023-10-06T02:23:10.044194+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:03:50.725863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.992100+0000","last_change":"2023-10-06T02:23:09.990032+0000","last_active":"2023-10-06T02:23:09.992100+0000","last_peered":"2023-10-06T02:23:09.992100+0000","last_clean":"2023-10-06T02:23:09.992100+0000","last_became_active":"2023-10-06T02:23:09.988801+0000","last_became_peered":"2023-10-06T02:23:09.988801+0000","last_unstale":"2023-10-06T02:23:09.992100+0000","last_undegraded":"2023-10-06T02:23:09.992100+0000","last_fullsized":"2023-10-06T02:23:09.992100+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:12.998563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.044129+0000","last_change":"2023-10-06T02:23:09.048438+0000","last_active":"2023-10-06T02:23:10.044129+0000","last_peered":"2023-10-06T02:23:10.044129+0000","last_clean":"2023-10-06T02:23:10.044129+0000","last_became_active":"2023-10-06T02:23:09.048051+0000","last_became_peered":"2023-10-06T02:23:09.048051+0000","last_unstale":"2023-10-06T02:23:10.044129+0000","last_undegraded":"2023-10-06T02:23:10.044129+0000","last_fullsized":"2023-10-06T02:23:10.044129+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:21:32.730148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.010287+0000","last_change":"2023-10-06T02:23:10.002031+0000","last_active":"2023-10-06T02:23:10.010287+0000","last_peered":"2023-10-06T02:23:10.010287+0000","last_clean":"2023-10-06T02:23:10.010287+0000","last_became_active":"2023-10-06T02:23:10.001754+0000","last_became_peered":"2023-10-06T02:23:10.001754+0000","last_unstale":"2023-10-06T02:23:10.010287+0000","last_undegraded":"2023-10-06T02:23:10.010287+0000","last_fullsized":"2023-10-06T02:23:10.010287+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:45:34.162627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.044320+0000","last_change":"2023-10-06T02:23:09.038304+0000","last_active":"2023-10-06T02:23:10.044320+0000","last_peered":"2023-10-06T02:23:10.044320+0000","last_clean":"2023-10-06T02:23:10.044320+0000","last_became_active":"2023-10-06T02:23:09.038031+0000","last_became_peered":"2023-10-06T02:23:09.038031+0000","last_unstale":"2023-10-06T02:23:10.044320+0000","last_undegraded":"2023-10-06T02:23:10.044320+0000","last_fullsized":"2023-10-06T02:23:10.044320+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:31:52.666890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.004414+0000","last_change":"2023-10-06T02:23:10.000692+0000","last_active":"2023-10-06T02:23:10.004414+0000","last_peered":"2023-10-06T02:23:10.004414+0000","last_clean":"2023-10-06T02:23:10.004414+0000","last_became_active":"2023-10-06T02:23:10.000368+0000","last_became_peered":"2023-10-06T02:23:10.000368+0000","last_unstale":"2023-10-06T02:23:10.004414+0000","last_undegraded":"2023-10-06T02:23:10.004414+0000","last_fullsized":"2023-10-06T02:23:10.004414+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:53:01.233080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.974959+0000","last_change":"2023-10-06T02:23:09.037238+0000","last_active":"2023-10-06T02:23:09.974959+0000","last_peered":"2023-10-06T02:23:09.974959+0000","last_clean":"2023-10-06T02:23:09.974959+0000","last_became_active":"2023-10-06T02:23:09.036871+0000","last_became_peered":"2023-10-06T02:23:09.036871+0000","last_unstale":"2023-10-06T02:23:09.974959+0000","last_undegraded":"2023-10-06T02:23:09.974959+0000","last_fullsized":"2023-10-06T02:23:09.974959+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:44:15.545672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.002856+0000","last_change":"2023-10-06T02:23:10.000807+0000","last_active":"2023-10-06T02:23:10.002856+0000","last_peered":"2023-10-06T02:23:10.002856+0000","last_clean":"2023-10-06T02:23:10.002856+0000","last_became_active":"2023-10-06T02:23:10.000502+0000","last_became_peered":"2023-10-06T02:23:10.000502+0000","last_unstale":"2023-10-06T02:23:10.002856+0000","last_undegraded":"2023-10-06T02:23:10.002856+0000","last_fullsized":"2023-10-06T02:23:10.002856+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:31:27.242353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.042047+0000","last_change":"2023-10-06T02:23:09.044437+0000","last_active":"2023-10-06T02:23:10.042047+0000","last_peered":"2023-10-06T02:23:10.042047+0000","last_clean":"2023-10-06T02:23:10.042047+0000","last_became_active":"2023-10-06T02:23:09.042657+0000","last_became_peered":"2023-10-06T02:23:09.042657+0000","last_unstale":"2023-10-06T02:23:10.042047+0000","last_undegraded":"2023-10-06T02:23:10.042047+0000","last_fullsized":"2023-10-06T02:23:10.042047+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:57:40.559177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.994640+0000","last_change":"2023-10-06T02:23:09.990834+0000","last_active":"2023-10-06T02:23:09.994640+0000","last_peered":"2023-10-06T02:23:09.994640+0000","last_clean":"2023-10-06T02:23:09.994640+0000","last_became_active":"2023-10-06T02:23:09.990192+0000","last_became_peered":"2023-10-06T02:23:09.990192+0000","last_unstale":"2023-10-06T02:23:09.994640+0000","last_undegraded":"2023-10-06T02:23:09.994640+0000","last_fullsized":"2023-10-06T02:23:09.994640+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:18:32.416904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.980739+0000","last_change":"2023-10-06T02:23:09.044880+0000","last_active":"2023-10-06T02:23:09.980739+0000","last_peered":"2023-10-06T02:23:09.980739+0000","last_clean":"2023-10-06T02:23:09.980739+0000","last_became_active":"2023-10-06T02:23:09.044556+0000","last_became_peered":"2023-10-06T02:23:09.044556+0000","last_unstale":"2023-10-06T02:23:09.980739+0000","last_undegraded":"2023-10-06T02:23:09.980739+0000","last_fullsized":"2023-10-06T02:23:09.980739+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:58.976414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.994703+0000","last_change":"2023-10-06T02:23:09.992094+0000","last_active":"2023-10-06T02:23:09.994703+0000","last_peered":"2023-10-06T02:23:09.994703+0000","last_clean":"2023-10-06T02:23:09.994703+0000","last_became_active":"2023-10-06T02:23:09.991699+0000","last_became_peered":"2023-10-06T02:23:09.991699+0000","last_unstale":"2023-10-06T02:23:09.994703+0000","last_undegraded":"2023-10-06T02:23:09.994703+0000","last_fullsized":"2023-10-06T02:23:09.994703+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:21.438589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.986400+0000","last_change":"2023-10-06T02:23:09.041591+0000","last_active":"2023-10-06T02:23:09.986400+0000","last_peered":"2023-10-06T02:23:09.986400+0000","last_clean":"2023-10-06T02:23:09.986400+0000","last_became_active":"2023-10-06T02:23:09.041239+0000","last_became_peered":"2023-10-06T02:23:09.041239+0000","last_unstale":"2023-10-06T02:23:09.986400+0000","last_undegraded":"2023-10-06T02:23:09.986400+0000","last_fullsized":"2023-10-06T02:23:09.986400+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:17:32.434412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.994672+0000","last_change":"2023-10-06T02:23:09.991765+0000","last_active":"2023-10-06T02:23:09.994672+0000","last_peered":"2023-10-06T02:23:09.994672+0000","last_clean":"2023-10-06T02:23:09.994672+0000","last_became_active":"2023-10-06T02:23:09.990991+0000","last_became_peered":"2023-10-06T02:23:09.990991+0000","last_unstale":"2023-10-06T02:23:09.994672+0000","last_undegraded":"2023-10-06T02:23:09.994672+0000","last_fullsized":"2023-10-06T02:23:09.994672+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:18:23.992787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.976060+0000","last_change":"2023-10-06T02:23:09.046400+0000","last_active":"2023-10-06T02:23:09.976060+0000","last_peered":"2023-10-06T02:23:09.976060+0000","last_clean":"2023-10-06T02:23:09.976060+0000","last_became_active":"2023-10-06T02:23:09.046039+0000","last_became_peered":"2023-10-06T02:23:09.046039+0000","last_unstale":"2023-10-06T02:23:09.976060+0000","last_undegraded":"2023-10-06T02:23:09.976060+0000","last_fullsized":"2023-10-06T02:23:09.976060+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:44:15.545672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.005866+0000","last_change":"2023-10-06T02:23:10.001037+0000","last_active":"2023-10-06T02:23:10.005866+0000","last_peered":"2023-10-06T02:23:10.005866+0000","last_clean":"2023-10-06T02:23:10.005866+0000","last_became_active":"2023-10-06T02:23:10.000479+0000","last_became_peered":"2023-10-06T02:23:10.000479+0000","last_unstale":"2023-10-06T02:23:10.005866+0000","last_undegraded":"2023-10-06T02:23:10.005866+0000","last_fullsized":"2023-10-06T02:23:10.005866+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:50:19.168882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.996789+0000","last_change":"2023-10-06T02:23:02.054039+0000","last_active":"2023-10-06T02:23:09.996789+0000","last_peered":"2023-10-06T02:23:09.996789+0000","last_clean":"2023-10-06T02:23:09.996789+0000","last_became_active":"2023-10-06T02:23:02.053742+0000","last_became_peered":"2023-10-06T02:23:02.053742+0000","last_unstale":"2023-10-06T02:23:09.996789+0000","last_undegraded":"2023-10-06T02:23:09.996789+0000","last_fullsized":"2023-10-06T02:23:09.996789+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:58:18.846475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.975329+0000","last_change":"2023-10-06T02:23:09.037244+0000","last_active":"2023-10-06T02:23:09.975329+0000","last_peered":"2023-10-06T02:23:09.975329+0000","last_clean":"2023-10-06T02:23:09.975329+0000","last_became_active":"2023-10-06T02:23:09.036880+0000","last_became_peered":"2023-10-06T02:23:09.036880+0000","last_unstale":"2023-10-06T02:23:09.975329+0000","last_undegraded":"2023-10-06T02:23:09.975329+0000","last_fullsized":"2023-10-06T02:23:09.975329+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:36:27.554258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.005453+0000","last_change":"2023-10-06T02:23:09.999775+0000","last_active":"2023-10-06T02:23:10.005453+0000","last_peered":"2023-10-06T02:23:10.005453+0000","last_clean":"2023-10-06T02:23:10.005453+0000","last_became_active":"2023-10-06T02:23:09.999472+0000","last_became_peered":"2023-10-06T02:23:09.999472+0000","last_unstale":"2023-10-06T02:23:10.005453+0000","last_undegraded":"2023-10-06T02:23:10.005453+0000","last_fullsized":"2023-10-06T02:23:10.005453+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:44:41.608418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.977354+0000","last_change":"2023-10-06T02:23:01.988631+0000","last_active":"2023-10-06T02:23:09.977354+0000","last_peered":"2023-10-06T02:23:09.977354+0000","last_clean":"2023-10-06T02:23:09.977354+0000","last_became_active":"2023-10-06T02:23:01.988303+0000","last_became_peered":"2023-10-06T02:23:01.988303+0000","last_unstale":"2023-10-06T02:23:09.977354+0000","last_undegraded":"2023-10-06T02:23:09.977354+0000","last_fullsized":"2023-10-06T02:23:09.977354+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:38.211466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.986211+0000","last_change":"2023-10-06T02:23:09.041835+0000","last_active":"2023-10-06T02:23:09.986211+0000","last_peered":"2023-10-06T02:23:09.986211+0000","last_clean":"2023-10-06T02:23:09.986211+0000","last_became_active":"2023-10-06T02:23:09.041521+0000","last_became_peered":"2023-10-06T02:23:09.041521+0000","last_unstale":"2023-10-06T02:23:09.986211+0000","last_undegraded":"2023-10-06T02:23:09.986211+0000","last_fullsized":"2023-10-06T02:23:09.986211+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:38.263723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.003455+0000","last_change":"2023-10-06T02:23:09.995055+0000","last_active":"2023-10-06T02:23:10.003455+0000","last_peered":"2023-10-06T02:23:10.003455+0000","last_clean":"2023-10-06T02:23:10.003455+0000","last_became_active":"2023-10-06T02:23:09.994216+0000","last_became_peered":"2023-10-06T02:23:09.994216+0000","last_unstale":"2023-10-06T02:23:10.003455+0000","last_undegraded":"2023-10-06T02:23:10.003455+0000","last_fullsized":"2023-10-06T02:23:10.003455+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:47:34.184576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.972958+0000","last_change":"2023-10-06T02:23:02.054338+0000","last_active":"2023-10-06T02:23:09.972958+0000","last_peered":"2023-10-06T02:23:09.972958+0000","last_clean":"2023-10-06T02:23:09.972958+0000","last_became_active":"2023-10-06T02:23:02.053991+0000","last_became_peered":"2023-10-06T02:23:02.053991+0000","last_unstale":"2023-10-06T02:23:09.972958+0000","last_undegraded":"2023-10-06T02:23:09.972958+0000","last_fullsized":"2023-10-06T02:23:09.972958+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:39:30.192838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.996619+0000","last_change":"2023-10-06T02:23:09.039360+0000","last_active":"2023-10-06T02:23:09.996619+0000","last_peered":"2023-10-06T02:23:09.996619+0000","last_clean":"2023-10-06T02:23:09.996619+0000","last_became_active":"2023-10-06T02:23:09.038571+0000","last_became_peered":"2023-10-06T02:23:09.038571+0000","last_unstale":"2023-10-06T02:23:09.996619+0000","last_undegraded":"2023-10-06T02:23:09.996619+0000","last_fullsized":"2023-10-06T02:23:09.996619+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:58.976414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.005846+0000","last_change":"2023-10-06T02:23:10.000821+0000","last_active":"2023-10-06T02:23:10.005846+0000","last_peered":"2023-10-06T02:23:10.005846+0000","last_clean":"2023-10-06T02:23:10.005846+0000","last_became_active":"2023-10-06T02:23:10.000203+0000","last_became_peered":"2023-10-06T02:23:10.000203+0000","last_unstale":"2023-10-06T02:23:10.005846+0000","last_undegraded":"2023-10-06T02:23:10.005846+0000","last_fullsized":"2023-10-06T02:23:10.005846+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:43:01.259350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.975054+0000","last_change":"2023-10-06T02:23:01.988140+0000","last_active":"2023-10-06T02:23:09.975054+0000","last_peered":"2023-10-06T02:23:09.975054+0000","last_clean":"2023-10-06T02:23:09.975054+0000","last_became_active":"2023-10-06T02:23:01.987769+0000","last_became_peered":"2023-10-06T02:23:01.987769+0000","last_unstale":"2023-10-06T02:23:09.975054+0000","last_undegraded":"2023-10-06T02:23:09.975054+0000","last_fullsized":"2023-10-06T02:23:09.975054+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:20:08.391912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":166,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.983731+0000","last_change":"2023-10-06T02:22:55.032519+0000","last_active":"2023-10-06T02:23:09.983731+0000","last_peered":"2023-10-06T02:23:09.983731+0000","last_clean":"2023-10-06T02:23:09.983731+0000","last_became_active":"2023-10-06T02:22:55.031367+0000","last_became_peered":"2023-10-06T02:22:55.031367+0000","last_unstale":"2023-10-06T02:23:09.983731+0000","last_undegraded":"2023-10-06T02:23:09.983731+0000","last_fullsized":"2023-10-06T02:23:09.983731+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:22:53.309097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:22:53.309097+0000","last_clean_scrub_stamp":"2023-10-06T02:22:53.309097+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:53:46.520909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.975628+0000","last_change":"2023-10-06T02:23:09.041907+0000","last_active":"2023-10-06T02:23:09.975628+0000","last_peered":"2023-10-06T02:23:09.975628+0000","last_clean":"2023-10-06T02:23:09.975628+0000","last_became_active":"2023-10-06T02:23:09.036698+0000","last_became_peered":"2023-10-06T02:23:09.036698+0000","last_unstale":"2023-10-06T02:23:09.975628+0000","last_undegraded":"2023-10-06T02:23:09.975628+0000","last_fullsized":"2023-10-06T02:23:09.975628+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:57.514051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.003862+0000","last_change":"2023-10-06T02:23:09.997846+0000","last_active":"2023-10-06T02:23:10.003862+0000","last_peered":"2023-10-06T02:23:10.003862+0000","last_clean":"2023-10-06T02:23:10.003862+0000","last_became_active":"2023-10-06T02:23:09.997558+0000","last_became_peered":"2023-10-06T02:23:09.997558+0000","last_unstale":"2023-10-06T02:23:10.003862+0000","last_undegraded":"2023-10-06T02:23:10.003862+0000","last_fullsized":"2023-10-06T02:23:10.003862+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:29:00.771363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.984381+0000","last_change":"2023-10-06T02:23:01.990773+0000","last_active":"2023-10-06T02:23:09.984381+0000","last_peered":"2023-10-06T02:23:09.984381+0000","last_clean":"2023-10-06T02:23:09.984381+0000","last_became_active":"2023-10-06T02:23:01.990511+0000","last_became_peered":"2023-10-06T02:23:01.990511+0000","last_unstale":"2023-10-06T02:23:09.984381+0000","last_undegraded":"2023-10-06T02:23:09.984381+0000","last_fullsized":"2023-10-06T02:23:09.984381+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:42:54.797731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.977043+0000","last_change":"2023-10-06T02:23:09.047067+0000","last_active":"2023-10-06T02:23:09.977043+0000","last_peered":"2023-10-06T02:23:09.977043+0000","last_clean":"2023-10-06T02:23:09.977043+0000","last_became_active":"2023-10-06T02:23:09.046735+0000","last_became_peered":"2023-10-06T02:23:09.046735+0000","last_unstale":"2023-10-06T02:23:09.977043+0000","last_undegraded":"2023-10-06T02:23:09.977043+0000","last_fullsized":"2023-10-06T02:23:09.977043+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:57.514051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.005223+0000","last_change":"2023-10-06T02:23:10.001635+0000","last_active":"2023-10-06T02:23:10.005223+0000","last_peered":"2023-10-06T02:23:10.005223+0000","last_clean":"2023-10-06T02:23:10.005223+0000","last_became_active":"2023-10-06T02:23:10.001341+0000","last_became_peered":"2023-10-06T02:23:10.001341+0000","last_unstale":"2023-10-06T02:23:10.005223+0000","last_undegraded":"2023-10-06T02:23:10.005223+0000","last_fullsized":"2023-10-06T02:23:10.005223+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:44:41.608418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.984779+0000","last_change":"2023-10-06T02:23:02.054069+0000","last_active":"2023-10-06T02:23:09.984779+0000","last_peered":"2023-10-06T02:23:09.984779+0000","last_clean":"2023-10-06T02:23:09.984779+0000","last_became_active":"2023-10-06T02:23:02.053739+0000","last_became_peered":"2023-10-06T02:23:02.053739+0000","last_unstale":"2023-10-06T02:23:09.984779+0000","last_undegraded":"2023-10-06T02:23:09.984779+0000","last_fullsized":"2023-10-06T02:23:09.984779+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:57:48.822608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.996931+0000","last_change":"2023-10-06T02:23:09.037195+0000","last_active":"2023-10-06T02:23:09.996931+0000","last_peered":"2023-10-06T02:23:09.996931+0000","last_clean":"2023-10-06T02:23:09.996931+0000","last_became_active":"2023-10-06T02:23:09.036925+0000","last_became_peered":"2023-10-06T02:23:09.036925+0000","last_unstale":"2023-10-06T02:23:09.996931+0000","last_undegraded":"2023-10-06T02:23:09.996931+0000","last_fullsized":"2023-10-06T02:23:09.996931+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:38.263723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.994210+0000","last_change":"2023-10-06T02:23:09.986467+0000","last_active":"2023-10-06T02:23:09.994210+0000","last_peered":"2023-10-06T02:23:09.994210+0000","last_clean":"2023-10-06T02:23:09.994210+0000","last_became_active":"2023-10-06T02:23:09.985750+0000","last_became_peered":"2023-10-06T02:23:09.985750+0000","last_unstale":"2023-10-06T02:23:09.994210+0000","last_undegraded":"2023-10-06T02:23:09.994210+0000","last_fullsized":"2023-10-06T02:23:09.994210+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:18:23.992787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.044047+0000","last_change":"2023-10-06T02:23:09.041622+0000","last_active":"2023-10-06T02:23:10.044047+0000","last_peered":"2023-10-06T02:23:10.044047+0000","last_clean":"2023-10-06T02:23:10.044047+0000","last_became_active":"2023-10-06T02:23:09.036680+0000","last_became_peered":"2023-10-06T02:23:09.036680+0000","last_unstale":"2023-10-06T02:23:10.044047+0000","last_undegraded":"2023-10-06T02:23:10.044047+0000","last_fullsized":"2023-10-06T02:23:10.044047+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:36:27.554258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.993558+0000","last_change":"2023-10-06T02:23:01.989934+0000","last_active":"2023-10-06T02:23:09.993558+0000","last_peered":"2023-10-06T02:23:09.993558+0000","last_clean":"2023-10-06T02:23:09.993558+0000","last_became_active":"2023-10-06T02:23:01.989706+0000","last_became_peered":"2023-10-06T02:23:01.989706+0000","last_unstale":"2023-10-06T02:23:09.993558+0000","last_undegraded":"2023-10-06T02:23:09.993558+0000","last_fullsized":"2023-10-06T02:23:09.993558+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:06.212251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.989446+0000","last_change":"2023-10-06T02:23:09.987681+0000","last_active":"2023-10-06T02:23:09.989446+0000","last_peered":"2023-10-06T02:23:09.989446+0000","last_clean":"2023-10-06T02:23:09.989446+0000","last_became_active":"2023-10-06T02:23:09.987439+0000","last_became_peered":"2023-10-06T02:23:09.987439+0000","last_unstale":"2023-10-06T02:23:09.989446+0000","last_undegraded":"2023-10-06T02:23:09.989446+0000","last_fullsized":"2023-10-06T02:23:09.989446+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:31:27.242353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.980493+0000","last_change":"2023-10-06T02:23:01.992935+0000","last_active":"2023-10-06T02:23:09.980493+0000","last_peered":"2023-10-06T02:23:09.980493+0000","last_clean":"2023-10-06T02:23:09.980493+0000","last_became_active":"2023-10-06T02:23:01.992568+0000","last_became_peered":"2023-10-06T02:23:01.992568+0000","last_unstale":"2023-10-06T02:23:09.980493+0000","last_undegraded":"2023-10-06T02:23:09.980493+0000","last_fullsized":"2023-10-06T02:23:09.980493+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:06.212251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"23'1","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T02:23:11.091780+0000","last_change":"2023-10-06T02:23:09.039504+0000","last_active":"2023-10-06T02:23:11.091780+0000","last_peered":"2023-10-06T02:23:11.091780+0000","last_clean":"2023-10-06T02:23:11.091780+0000","last_became_active":"2023-10-06T02:23:09.039056+0000","last_became_peered":"2023-10-06T02:23:09.039056+0000","last_unstale":"2023-10-06T02:23:11.091780+0000","last_undegraded":"2023-10-06T02:23:11.091780+0000","last_fullsized":"2023-10-06T02:23:11.091780+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:38.263723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.995987+0000","last_change":"2023-10-06T02:23:09.990800+0000","last_active":"2023-10-06T02:23:09.995987+0000","last_peered":"2023-10-06T02:23:09.995987+0000","last_clean":"2023-10-06T02:23:09.995987+0000","last_became_active":"2023-10-06T02:23:09.990531+0000","last_became_peered":"2023-10-06T02:23:09.990531+0000","last_unstale":"2023-10-06T02:23:09.995987+0000","last_undegraded":"2023-10-06T02:23:09.995987+0000","last_fullsized":"2023-10-06T02:23:09.995987+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:39:39.959479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.045165+0000","last_change":"2023-10-06T02:23:09.046230+0000","last_active":"2023-10-06T02:23:10.045165+0000","last_peered":"2023-10-06T02:23:10.045165+0000","last_clean":"2023-10-06T02:23:10.045165+0000","last_became_active":"2023-10-06T02:23:09.045976+0000","last_became_peered":"2023-10-06T02:23:09.045976+0000","last_unstale":"2023-10-06T02:23:10.045165+0000","last_undegraded":"2023-10-06T02:23:10.045165+0000","last_fullsized":"2023-10-06T02:23:10.045165+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:36:27.554258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.998669+0000","last_change":"2023-10-06T02:23:09.995680+0000","last_active":"2023-10-06T02:23:09.998669+0000","last_peered":"2023-10-06T02:23:09.998669+0000","last_clean":"2023-10-06T02:23:09.998669+0000","last_became_active":"2023-10-06T02:23:09.995375+0000","last_became_peered":"2023-10-06T02:23:09.995375+0000","last_unstale":"2023-10-06T02:23:09.998669+0000","last_undegraded":"2023-10-06T02:23:09.998669+0000","last_fullsized":"2023-10-06T02:23:09.998669+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:36:28.561818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.046988+0000","last_change":"2023-10-06T02:23:09.043361+0000","last_active":"2023-10-06T02:23:10.046988+0000","last_peered":"2023-10-06T02:23:10.046988+0000","last_clean":"2023-10-06T02:23:10.046988+0000","last_became_active":"2023-10-06T02:23:09.042973+0000","last_became_peered":"2023-10-06T02:23:09.042973+0000","last_unstale":"2023-10-06T02:23:10.046988+0000","last_undegraded":"2023-10-06T02:23:10.046988+0000","last_fullsized":"2023-10-06T02:23:10.046988+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:31:52.666890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.992487+0000","last_change":"2023-10-06T02:23:09.988158+0000","last_active":"2023-10-06T02:23:09.992487+0000","last_peered":"2023-10-06T02:23:09.992487+0000","last_clean":"2023-10-06T02:23:09.992487+0000","last_became_active":"2023-10-06T02:23:09.987844+0000","last_became_peered":"2023-10-06T02:23:09.987844+0000","last_unstale":"2023-10-06T02:23:09.992487+0000","last_undegraded":"2023-10-06T02:23:09.992487+0000","last_fullsized":"2023-10-06T02:23:09.992487+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:10:59.616168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.045352+0000","last_change":"2023-10-06T02:23:09.044069+0000","last_active":"2023-10-06T02:23:10.045352+0000","last_peered":"2023-10-06T02:23:10.045352+0000","last_clean":"2023-10-06T02:23:10.045352+0000","last_became_active":"2023-10-06T02:23:09.043700+0000","last_became_peered":"2023-10-06T02:23:09.043700+0000","last_unstale":"2023-10-06T02:23:10.045352+0000","last_undegraded":"2023-10-06T02:23:10.045352+0000","last_fullsized":"2023-10-06T02:23:10.045352+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:57:40.559177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.010156+0000","last_change":"2023-10-06T02:23:09.999700+0000","last_active":"2023-10-06T02:23:10.010156+0000","last_peered":"2023-10-06T02:23:10.010156+0000","last_clean":"2023-10-06T02:23:10.010156+0000","last_became_active":"2023-10-06T02:23:09.997418+0000","last_became_peered":"2023-10-06T02:23:09.997418+0000","last_unstale":"2023-10-06T02:23:10.010156+0000","last_undegraded":"2023-10-06T02:23:10.010156+0000","last_fullsized":"2023-10-06T02:23:10.010156+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:53:54.469523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.041904+0000","last_change":"2023-10-06T02:23:09.034024+0000","last_active":"2023-10-06T02:23:10.041904+0000","last_peered":"2023-10-06T02:23:10.041904+0000","last_clean":"2023-10-06T02:23:10.041904+0000","last_became_active":"2023-10-06T02:23:09.033389+0000","last_became_peered":"2023-10-06T02:23:09.033389+0000","last_unstale":"2023-10-06T02:23:10.041904+0000","last_undegraded":"2023-10-06T02:23:10.041904+0000","last_fullsized":"2023-10-06T02:23:10.041904+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:03:50.725863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.991920+0000","last_change":"2023-10-06T02:23:09.984080+0000","last_active":"2023-10-06T02:23:09.991920+0000","last_peered":"2023-10-06T02:23:09.991920+0000","last_clean":"2023-10-06T02:23:09.991920+0000","last_became_active":"2023-10-06T02:23:09.983808+0000","last_became_peered":"2023-10-06T02:23:09.983808+0000","last_unstale":"2023-10-06T02:23:09.991920+0000","last_undegraded":"2023-10-06T02:23:09.991920+0000","last_fullsized":"2023-10-06T02:23:09.991920+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:12.998563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"22'4","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.045612+0000","last_change":"2023-10-06T02:23:09.037190+0000","last_active":"2023-10-06T02:23:10.045612+0000","last_peered":"2023-10-06T02:23:10.045612+0000","last_clean":"2023-10-06T02:23:10.045612+0000","last_became_active":"2023-10-06T02:23:09.036872+0000","last_became_peered":"2023-10-06T02:23:09.036872+0000","last_unstale":"2023-10-06T02:23:10.045612+0000","last_undegraded":"2023-10-06T02:23:10.045612+0000","last_fullsized":"2023-10-06T02:23:10.045612+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:57:40.559177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.994436+0000","last_change":"2023-10-06T02:23:09.989896+0000","last_active":"2023-10-06T02:23:09.994436+0000","last_peered":"2023-10-06T02:23:09.994436+0000","last_clean":"2023-10-06T02:23:09.994436+0000","last_became_active":"2023-10-06T02:23:09.989525+0000","last_became_peered":"2023-10-06T02:23:09.989525+0000","last_unstale":"2023-10-06T02:23:09.994436+0000","last_undegraded":"2023-10-06T02:23:09.994436+0000","last_fullsized":"2023-10-06T02:23:09.994436+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:58:41.925466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.992536+0000","last_change":"2023-10-06T02:23:09.988852+0000","last_active":"2023-10-06T02:23:09.992536+0000","last_peered":"2023-10-06T02:23:09.992536+0000","last_clean":"2023-10-06T02:23:09.992536+0000","last_became_active":"2023-10-06T02:23:09.988527+0000","last_became_peered":"2023-10-06T02:23:09.988527+0000","last_unstale":"2023-10-06T02:23:09.992536+0000","last_undegraded":"2023-10-06T02:23:09.992536+0000","last_fullsized":"2023-10-06T02:23:09.992536+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:03:51.733423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.001202+0000","last_change":"2023-10-06T02:23:09.999698+0000","last_active":"2023-10-06T02:23:10.001202+0000","last_peered":"2023-10-06T02:23:10.001202+0000","last_clean":"2023-10-06T02:23:10.001202+0000","last_became_active":"2023-10-06T02:23:09.999418+0000","last_became_peered":"2023-10-06T02:23:09.999418+0000","last_unstale":"2023-10-06T02:23:10.001202+0000","last_undegraded":"2023-10-06T02:23:10.001202+0000","last_fullsized":"2023-10-06T02:23:10.001202+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:10:59.616168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.010180+0000","last_change":"2023-10-06T02:23:10.000203+0000","last_active":"2023-10-06T02:23:10.010180+0000","last_peered":"2023-10-06T02:23:10.010180+0000","last_clean":"2023-10-06T02:23:10.010180+0000","last_became_active":"2023-10-06T02:23:09.999397+0000","last_became_peered":"2023-10-06T02:23:09.999397+0000","last_unstale":"2023-10-06T02:23:10.010180+0000","last_undegraded":"2023-10-06T02:23:10.010180+0000","last_fullsized":"2023-10-06T02:23:10.010180+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:31:27.242353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.989526+0000","last_change":"2023-10-06T02:23:09.986996+0000","last_active":"2023-10-06T02:23:09.989526+0000","last_peered":"2023-10-06T02:23:09.989526+0000","last_clean":"2023-10-06T02:23:09.989526+0000","last_became_active":"2023-10-06T02:23:09.986723+0000","last_became_peered":"2023-10-06T02:23:09.986723+0000","last_unstale":"2023-10-06T02:23:09.989526+0000","last_undegraded":"2023-10-06T02:23:09.989526+0000","last_fullsized":"2023-10-06T02:23:09.989526+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:19:29.462977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.007171+0000","last_change":"2023-10-06T02:23:10.003345+0000","last_active":"2023-10-06T02:23:10.007171+0000","last_peered":"2023-10-06T02:23:10.007171+0000","last_clean":"2023-10-06T02:23:10.007171+0000","last_became_active":"2023-10-06T02:23:10.003023+0000","last_became_peered":"2023-10-06T02:23:10.003023+0000","last_unstale":"2023-10-06T02:23:10.007171+0000","last_undegraded":"2023-10-06T02:23:10.007171+0000","last_fullsized":"2023-10-06T02:23:10.007171+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:18:23.992787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298132,"kb_used_data":844,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073708,"statfs":{"total":96636764160,"available":96331476992,"internally_reserved":0,"allocated":864256,"data_stored":681439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81299999999999994}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297524,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074316,"statfs":{"total":96636764160,"available":96332099584,"internally_reserved":0,"allocated":241664,"data_stored":89319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78500000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79600000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297524,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074316,"statfs":{"total":96636764160,"available":96332099584,"internally_reserved":0,"allocated":241664,"data_stored":89319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68500000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94399999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77300000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298108,"kb_used_data":820,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073732,"statfs":{"total":96636764160,"available":96331501568,"internally_reserved":0,"allocated":839680,"data_stored":680241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.755}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70599999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69199999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297524,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074316,"statfs":{"total":96636764160,"available":96332099584,"internally_reserved":0,"allocated":241664,"data_stored":89319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41599999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82699999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85199999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297528,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074312,"statfs":{"total":96636764160,"available":96332095488,"internally_reserved":0,"allocated":245760,"data_stored":89409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297544,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074296,"statfs":{"total":96636764160,"available":96332079104,"internally_reserved":0,"allocated":262144,"data_stored":90911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69199999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89600000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86799999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97999999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297532,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074308,"statfs":{"total":96636764160,"available":96332091392,"internally_reserved":0,"allocated":249856,"data_stored":89943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82199999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78300000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T02:23:19.320 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T02:23:19.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.517+0000 7fa8eb59e700 1 Processor -- start 2023-10-06T02:23:19.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.517+0000 7fa8eb59e700 1 -- start start 2023-10-06T02:23:19.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.519+0000 7fa8eb59e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa8ec140e40 0x7fa8ec0c6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:19.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.519+0000 7fa8eb59e700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa8ec06de50 0x7fa8ec0aeef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:19.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.519+0000 7fa8eb59e700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8ec0c6d50 con 0x7fa8ec140a70 2023-10-06T02:23:19.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.519+0000 7fa8eb59e700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8ec0c7030 con 0x7fa8ec140e40 2023-10-06T02:23:19.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.519+0000 7fa8eb59e700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8ec0c7310 con 0x7fa8ec06de50 2023-10-06T02:23:19.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.519+0000 7fa8e9d9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa8ec140e40 0x7fa8ec0c6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:19.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.519+0000 7fa8e9d9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa8ec140e40 0x7fa8ec0c6770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37396/0 (socket says 172.21.15.31:37396) 2023-10-06T02:23:19.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.519+0000 7fa8e9d9b700 1 -- 172.21.15.31:0/4189881465 learned_addr learned my addr 172.21.15.31:0/4189881465 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:19.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.519+0000 7fa8ead9d700 1 --2- 172.21.15.31:0/4189881465 >> v2:172.21.15.62:3300/0 conn(0x7fa8ec06de50 0x7fa8ec0aeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:19.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.520+0000 7fa8e9d9b700 1 -- 172.21.15.31:0/4189881465 >> v2:172.21.15.62:3300/0 conn(0x7fa8ec06de50 msgr2=0x7fa8ec0aeef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:19.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.520+0000 7fa8e9d9b700 1 --2- 172.21.15.31:0/4189881465 >> v2:172.21.15.62:3300/0 conn(0x7fa8ec06de50 0x7fa8ec0aeef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:19.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.520+0000 7fa8e9d9b700 1 -- 172.21.15.31:0/4189881465 >> v1:172.21.15.62:6789/0 conn(0x7fa8ec140a70 legacy=0x7fa8ec0c7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:19.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.520+0000 7fa8e9d9b700 1 -- 172.21.15.31:0/4189881465 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8ec0c75f0 con 0x7fa8ec140e40 2023-10-06T02:23:19.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.520+0000 7fa8e9d9b700 1 --2- 172.21.15.31:0/4189881465 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8ec140e40 0x7fa8ec0c6770 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec0aec50 tx=0x7fa8e0009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9be2958a820784fa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:19.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.520+0000 7fa8e959a700 1 -- 172.21.15.31:0/4189881465 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e0005c10 con 0x7fa8ec140e40 2023-10-06T02:23:19.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.520+0000 7fa8e959a700 1 -- 172.21.15.31:0/4189881465 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8e0005d70 con 0x7fa8ec140e40 2023-10-06T02:23:19.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.520+0000 7fa8e959a700 1 -- 172.21.15.31:0/4189881465 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e0006260 con 0x7fa8ec140e40 2023-10-06T02:23:19.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.521+0000 7fa8eb59e700 1 -- 172.21.15.31:0/4189881465 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8ec140e40 msgr2=0x7fa8ec0c6770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:19.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.521+0000 7fa8eb59e700 1 --2- 172.21.15.31:0/4189881465 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8ec140e40 0x7fa8ec0c6770 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec0aec50 tx=0x7fa8e0009220 comp rx=0 tx=0).stop 2023-10-06T02:23:19.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.521+0000 7fa8eb59e700 1 -- 172.21.15.31:0/4189881465 shutdown_connections 2023-10-06T02:23:19.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.521+0000 7fa8eb59e700 1 --2- 172.21.15.31:0/4189881465 >> v2:172.21.15.62:3300/0 conn(0x7fa8ec06de50 0x7fa8ec0aeef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:19.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.521+0000 7fa8eb59e700 1 --2- 172.21.15.31:0/4189881465 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8ec140e40 0x7fa8ec0c6770 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:19.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.521+0000 7fa8eb59e700 1 -- 172.21.15.31:0/4189881465 >> 172.21.15.31:0/4189881465 conn(0x7fa8ec06d170 msgr2=0x7fa8ec0cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:19.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.521+0000 7fa8eb59e700 1 -- 172.21.15.31:0/4189881465 shutdown_connections 2023-10-06T02:23:19.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.521+0000 7fa8eb59e700 1 -- 172.21.15.31:0/4189881465 wait complete. 2023-10-06T02:23:19.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.522+0000 7fa8eb59e700 1 Processor -- start 2023-10-06T02:23:19.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.522+0000 7fa8eb59e700 1 -- start start 2023-10-06T02:23:19.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.523+0000 7fa8eb59e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8ec06de50 0x7fa8ec0ae060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:19.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.523+0000 7fa8eb59e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8ec140a70 0x7fa8ec0c4130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:19.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.523+0000 7fa8eb59e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa8ec140e40 0x7fa8ec13e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:19.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.523+0000 7fa8eb59e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8ec16db90 con 0x7fa8ec06de50 2023-10-06T02:23:19.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.523+0000 7fa8eb59e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8ec0ae5a0 con 0x7fa8ec140a70 2023-10-06T02:23:19.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.523+0000 7fa8eb59e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8ec16d1f0 con 0x7fa8ec140e40 2023-10-06T02:23:19.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.523+0000 7fa8ea59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8ec06de50 0x7fa8ec0ae060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:19.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.524+0000 7fa8ea59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8ec06de50 0x7fa8ec0ae060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37412/0 (socket says 172.21.15.31:37412) 2023-10-06T02:23:19.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.524+0000 7fa8ea59c700 1 -- 172.21.15.31:0/591589347 learned_addr learned my addr 172.21.15.31:0/591589347 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:19.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.524+0000 7fa8e9d9b700 1 --2- 172.21.15.31:0/591589347 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8ec140a70 0x7fa8ec0c4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:19.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.524+0000 7fa8ead9d700 1 --2- 172.21.15.31:0/591589347 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa8ec140e40 0x7fa8ec13e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:19.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.524+0000 7fa8ea59c700 1 -- 172.21.15.31:0/591589347 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa8ec140e40 msgr2=0x7fa8ec13e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:19.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.524+0000 7fa8ea59c700 1 --2- 172.21.15.31:0/591589347 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa8ec140e40 0x7fa8ec13e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:19.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.524+0000 7fa8ea59c700 1 -- 172.21.15.31:0/591589347 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8ec140a70 msgr2=0x7fa8ec0c4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:19.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.524+0000 7fa8ea59c700 1 --2- 172.21.15.31:0/591589347 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8ec140a70 0x7fa8ec0c4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:19.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.524+0000 7fa8ea59c700 1 -- 172.21.15.31:0/591589347 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8e0009480 con 0x7fa8ec06de50 2023-10-06T02:23:19.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.524+0000 7fa8ea59c700 1 --2- 172.21.15.31:0/591589347 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8ec06de50 0x7fa8ec0ae060 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc00c590 tx=0x7fa8dc00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:19.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.524+0000 7fa8da7fc700 1 -- 172.21.15.31:0/591589347 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8dc011590 con 0x7fa8ec06de50 2023-10-06T02:23:19.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.524+0000 7fa8da7fc700 1 -- 172.21.15.31:0/591589347 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8dc011710 con 0x7fa8ec06de50 2023-10-06T02:23:19.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.525+0000 7fa8eb59e700 1 -- 172.21.15.31:0/591589347 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8ec16d4d0 con 0x7fa8ec06de50 2023-10-06T02:23:19.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.525+0000 7fa8da7fc700 1 -- 172.21.15.31:0/591589347 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8dc011ad0 con 0x7fa8ec06de50 2023-10-06T02:23:19.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.525+0000 7fa8eb59e700 1 -- 172.21.15.31:0/591589347 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8ec0c6490 con 0x7fa8ec06de50 2023-10-06T02:23:19.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.526+0000 7fa8eb59e700 1 -- 172.21.15.31:0/591589347 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8ec16de00 con 0x7fa8ec06de50 2023-10-06T02:23:19.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.527+0000 7fa8da7fc700 1 -- 172.21.15.31:0/591589347 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa8dc01b3b0 con 0x7fa8ec06de50 2023-10-06T02:23:19.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.528+0000 7fa8da7fc700 1 --2- 172.21.15.31:0/591589347 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa8cc066f30 0x7fa8cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:19.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.529+0000 7fa8da7fc700 1 -- 172.21.15.31:0/591589347 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa8dc083ba0 con 0x7fa8ec06de50 2023-10-06T02:23:19.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.529+0000 7fa8e9d9b700 1 --2- 172.21.15.31:0/591589347 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa8cc066f30 0x7fa8cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:19.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.529+0000 7fa8e9d9b700 1 --2- 172.21.15.31:0/591589347 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa8cc066f30 0x7fa8cc0693e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec1412a0 tx=0x7fa8e0016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:19.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.532+0000 7fa8da7fc700 1 -- 172.21.15.31:0/591589347 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa8dc050c10 con 0x7fa8ec06de50 2023-10-06T02:23:19.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.733+0000 7fa8eb59e700 1 -- 172.21.15.31:0/591589347 --> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa8ec04c3c0 con 0x7fa8cc066f30 2023-10-06T02:23:19.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.743+0000 7fa8da7fc700 1 -- 172.21.15.31:0/591589347 <== mgr.4107 v2:172.21.15.31:6832/85292 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286626 (secure 0 0 0) 0x7fa8ec04c3c0 con 0x7fa8cc066f30 2023-10-06T02:23:19.746 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:23:19.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.746+0000 7fa8eb59e700 1 -- 172.21.15.31:0/591589347 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa8cc066f30 msgr2=0x7fa8cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:19.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.746+0000 7fa8eb59e700 1 --2- 172.21.15.31:0/591589347 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa8cc066f30 0x7fa8cc0693e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec1412a0 tx=0x7fa8e0016000 comp rx=0 tx=0).stop 2023-10-06T02:23:19.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.746+0000 7fa8eb59e700 1 -- 172.21.15.31:0/591589347 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8ec06de50 msgr2=0x7fa8ec0ae060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:19.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.746+0000 7fa8eb59e700 1 --2- 172.21.15.31:0/591589347 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8ec06de50 0x7fa8ec0ae060 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc00c590 tx=0x7fa8dc00c8a0 comp rx=0 tx=0).stop 2023-10-06T02:23:19.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.746+0000 7fa8eb59e700 1 -- 172.21.15.31:0/591589347 shutdown_connections 2023-10-06T02:23:19.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.746+0000 7fa8eb59e700 1 --2- 172.21.15.31:0/591589347 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa8cc066f30 0x7fa8cc0693e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:19.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.746+0000 7fa8eb59e700 1 --2- 172.21.15.31:0/591589347 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa8ec140e40 0x7fa8ec13e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:19.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.746+0000 7fa8eb59e700 1 --2- 172.21.15.31:0/591589347 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8ec06de50 0x7fa8ec0ae060 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:19.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.746+0000 7fa8eb59e700 1 --2- 172.21.15.31:0/591589347 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8ec140a70 0x7fa8ec0c4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:19.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.746+0000 7fa8eb59e700 1 -- 172.21.15.31:0/591589347 >> 172.21.15.31:0/591589347 conn(0x7fa8ec06d170 msgr2=0x7fa8ec0c7c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:19.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.747+0000 7fa8eb59e700 1 -- 172.21.15.31:0/591589347 shutdown_connections 2023-10-06T02:23:19.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.747+0000 7fa8eb59e700 1 -- 172.21.15.31:0/591589347 wait complete. 2023-10-06T02:23:19.750 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-10-06T02:23:19.764 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":29,"stamp":"2023-10-06T02:23:19.119430+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381416,"kb_used_data":3112,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752593304,"statfs":{"total":773094113280,"available":770655543296,"internally_reserved":0,"allocated":3186688,"data_stored":1899900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.148183"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.985694+0000","last_change":"2023-10-06T02:23:09.982458+0000","last_active":"2023-10-06T02:23:09.985694+0000","last_peered":"2023-10-06T02:23:09.985694+0000","last_clean":"2023-10-06T02:23:09.985694+0000","last_became_active":"2023-10-06T02:23:09.982140+0000","last_became_peered":"2023-10-06T02:23:09.982140+0000","last_unstale":"2023-10-06T02:23:09.985694+0000","last_undegraded":"2023-10-06T02:23:09.985694+0000","last_fullsized":"2023-10-06T02:23:09.985694+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:07:57.730382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.999489+0000","last_change":"2023-10-06T02:23:09.996757+0000","last_active":"2023-10-06T02:23:09.999489+0000","last_peered":"2023-10-06T02:23:09.999489+0000","last_clean":"2023-10-06T02:23:09.999489+0000","last_became_active":"2023-10-06T02:23:09.996404+0000","last_became_peered":"2023-10-06T02:23:09.996404+0000","last_unstale":"2023-10-06T02:23:09.999489+0000","last_undegraded":"2023-10-06T02:23:09.999489+0000","last_fullsized":"2023-10-06T02:23:09.999489+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:39:15.275881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.010136+0000","last_change":"2023-10-06T02:23:09.999568+0000","last_active":"2023-10-06T02:23:10.010136+0000","last_peered":"2023-10-06T02:23:10.010136+0000","last_clean":"2023-10-06T02:23:10.010136+0000","last_became_active":"2023-10-06T02:23:09.999033+0000","last_became_peered":"2023-10-06T02:23:09.999033+0000","last_unstale":"2023-10-06T02:23:10.010136+0000","last_undegraded":"2023-10-06T02:23:10.010136+0000","last_fullsized":"2023-10-06T02:23:10.010136+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:23:26.580218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.996019+0000","last_change":"2023-10-06T02:23:09.995088+0000","last_active":"2023-10-06T02:23:09.996019+0000","last_peered":"2023-10-06T02:23:09.996019+0000","last_clean":"2023-10-06T02:23:09.996019+0000","last_became_active":"2023-10-06T02:23:09.994785+0000","last_became_peered":"2023-10-06T02:23:09.994785+0000","last_unstale":"2023-10-06T02:23:09.996019+0000","last_undegraded":"2023-10-06T02:23:09.996019+0000","last_fullsized":"2023-10-06T02:23:09.996019+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:21:33.737708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.994230+0000","last_change":"2023-10-06T02:23:09.986669+0000","last_active":"2023-10-06T02:23:09.994230+0000","last_peered":"2023-10-06T02:23:09.994230+0000","last_clean":"2023-10-06T02:23:09.994230+0000","last_became_active":"2023-10-06T02:23:09.986141+0000","last_became_peered":"2023-10-06T02:23:09.986141+0000","last_unstale":"2023-10-06T02:23:09.994230+0000","last_undegraded":"2023-10-06T02:23:09.994230+0000","last_fullsized":"2023-10-06T02:23:09.994230+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:39:15.275881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.010199+0000","last_change":"2023-10-06T02:23:10.000405+0000","last_active":"2023-10-06T02:23:10.010199+0000","last_peered":"2023-10-06T02:23:10.010199+0000","last_clean":"2023-10-06T02:23:10.010199+0000","last_became_active":"2023-10-06T02:23:10.000015+0000","last_became_peered":"2023-10-06T02:23:10.000015+0000","last_unstale":"2023-10-06T02:23:10.010199+0000","last_undegraded":"2023-10-06T02:23:10.010199+0000","last_fullsized":"2023-10-06T02:23:10.010199+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:39:15.275881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.009997+0000","last_change":"2023-10-06T02:23:10.004359+0000","last_active":"2023-10-06T02:23:10.009997+0000","last_peered":"2023-10-06T02:23:10.009997+0000","last_clean":"2023-10-06T02:23:10.009997+0000","last_became_active":"2023-10-06T02:23:10.003873+0000","last_became_peered":"2023-10-06T02:23:10.003873+0000","last_unstale":"2023-10-06T02:23:10.009997+0000","last_undegraded":"2023-10-06T02:23:10.009997+0000","last_fullsized":"2023-10-06T02:23:10.009997+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:44:41.608418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.998258+0000","last_change":"2023-10-06T02:23:09.994276+0000","last_active":"2023-10-06T02:23:09.998258+0000","last_peered":"2023-10-06T02:23:09.998258+0000","last_clean":"2023-10-06T02:23:09.998258+0000","last_became_active":"2023-10-06T02:23:09.993946+0000","last_became_peered":"2023-10-06T02:23:09.993946+0000","last_unstale":"2023-10-06T02:23:09.998258+0000","last_undegraded":"2023-10-06T02:23:09.998258+0000","last_fullsized":"2023-10-06T02:23:09.998258+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:19:29.462977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.997243+0000","last_change":"2023-10-06T02:23:09.993971+0000","last_active":"2023-10-06T02:23:09.997243+0000","last_peered":"2023-10-06T02:23:09.997243+0000","last_clean":"2023-10-06T02:23:09.997243+0000","last_became_active":"2023-10-06T02:23:09.993547+0000","last_became_peered":"2023-10-06T02:23:09.993547+0000","last_unstale":"2023-10-06T02:23:09.997243+0000","last_undegraded":"2023-10-06T02:23:09.997243+0000","last_fullsized":"2023-10-06T02:23:09.997243+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:19:29.462977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.999469+0000","last_change":"2023-10-06T02:23:09.995426+0000","last_active":"2023-10-06T02:23:09.999469+0000","last_peered":"2023-10-06T02:23:09.999469+0000","last_clean":"2023-10-06T02:23:09.999469+0000","last_became_active":"2023-10-06T02:23:09.995067+0000","last_became_peered":"2023-10-06T02:23:09.995067+0000","last_unstale":"2023-10-06T02:23:09.999469+0000","last_undegraded":"2023-10-06T02:23:09.999469+0000","last_fullsized":"2023-10-06T02:23:09.999469+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:50:19.168882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.010257+0000","last_change":"2023-10-06T02:23:10.001093+0000","last_active":"2023-10-06T02:23:10.010257+0000","last_peered":"2023-10-06T02:23:10.010257+0000","last_clean":"2023-10-06T02:23:10.010257+0000","last_became_active":"2023-10-06T02:23:10.000755+0000","last_became_peered":"2023-10-06T02:23:10.000755+0000","last_unstale":"2023-10-06T02:23:10.010257+0000","last_undegraded":"2023-10-06T02:23:10.010257+0000","last_fullsized":"2023-10-06T02:23:10.010257+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:43:02.156237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.998611+0000","last_change":"2023-10-06T02:23:09.993328+0000","last_active":"2023-10-06T02:23:09.998611+0000","last_peered":"2023-10-06T02:23:09.998611+0000","last_clean":"2023-10-06T02:23:09.998611+0000","last_became_active":"2023-10-06T02:23:09.993076+0000","last_became_peered":"2023-10-06T02:23:09.993076+0000","last_unstale":"2023-10-06T02:23:09.998611+0000","last_undegraded":"2023-10-06T02:23:09.998611+0000","last_fullsized":"2023-10-06T02:23:09.998611+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:58.521611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.986182+0000","last_change":"2023-10-06T02:23:09.982673+0000","last_active":"2023-10-06T02:23:09.986182+0000","last_peered":"2023-10-06T02:23:09.986182+0000","last_clean":"2023-10-06T02:23:09.986182+0000","last_became_active":"2023-10-06T02:23:09.982370+0000","last_became_peered":"2023-10-06T02:23:09.982370+0000","last_unstale":"2023-10-06T02:23:09.986182+0000","last_undegraded":"2023-10-06T02:23:09.986182+0000","last_fullsized":"2023-10-06T02:23:09.986182+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:17:33.441972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.985297+0000","last_change":"2023-10-06T02:23:09.982769+0000","last_active":"2023-10-06T02:23:09.985297+0000","last_peered":"2023-10-06T02:23:09.985297+0000","last_clean":"2023-10-06T02:23:09.985297+0000","last_became_active":"2023-10-06T02:23:09.982450+0000","last_became_peered":"2023-10-06T02:23:09.982450+0000","last_unstale":"2023-10-06T02:23:09.985297+0000","last_undegraded":"2023-10-06T02:23:09.985297+0000","last_fullsized":"2023-10-06T02:23:09.985297+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:12:16.306131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.010017+0000","last_change":"2023-10-06T02:23:10.004452+0000","last_active":"2023-10-06T02:23:10.010017+0000","last_peered":"2023-10-06T02:23:10.010017+0000","last_clean":"2023-10-06T02:23:10.010017+0000","last_became_active":"2023-10-06T02:23:10.004184+0000","last_became_peered":"2023-10-06T02:23:10.004184+0000","last_unstale":"2023-10-06T02:23:10.010017+0000","last_undegraded":"2023-10-06T02:23:10.010017+0000","last_fullsized":"2023-10-06T02:23:10.010017+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:50:19.168882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.010069+0000","last_change":"2023-10-06T02:23:09.998790+0000","last_active":"2023-10-06T02:23:10.010069+0000","last_peered":"2023-10-06T02:23:10.010069+0000","last_clean":"2023-10-06T02:23:10.010069+0000","last_became_active":"2023-10-06T02:23:09.998476+0000","last_became_peered":"2023-10-06T02:23:09.998476+0000","last_unstale":"2023-10-06T02:23:10.010069+0000","last_undegraded":"2023-10-06T02:23:10.010069+0000","last_fullsized":"2023-10-06T02:23:10.010069+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:19:29.462977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.997263+0000","last_change":"2023-10-06T02:23:09.994379+0000","last_active":"2023-10-06T02:23:09.997263+0000","last_peered":"2023-10-06T02:23:09.997263+0000","last_clean":"2023-10-06T02:23:09.997263+0000","last_became_active":"2023-10-06T02:23:09.994031+0000","last_became_peered":"2023-10-06T02:23:09.994031+0000","last_unstale":"2023-10-06T02:23:09.997263+0000","last_undegraded":"2023-10-06T02:23:09.997263+0000","last_fullsized":"2023-10-06T02:23:09.997263+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:12.998563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.994477+0000","last_change":"2023-10-06T02:23:09.988087+0000","last_active":"2023-10-06T02:23:09.994477+0000","last_peered":"2023-10-06T02:23:09.994477+0000","last_clean":"2023-10-06T02:23:09.994477+0000","last_became_active":"2023-10-06T02:23:09.987741+0000","last_became_peered":"2023-10-06T02:23:09.987741+0000","last_unstale":"2023-10-06T02:23:09.994477+0000","last_undegraded":"2023-10-06T02:23:09.994477+0000","last_fullsized":"2023-10-06T02:23:09.994477+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:05:24.047504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.987646+0000","last_change":"2023-10-06T02:23:09.983568+0000","last_active":"2023-10-06T02:23:09.987646+0000","last_peered":"2023-10-06T02:23:09.987646+0000","last_clean":"2023-10-06T02:23:09.987646+0000","last_became_active":"2023-10-06T02:23:09.983273+0000","last_became_peered":"2023-10-06T02:23:09.983273+0000","last_unstale":"2023-10-06T02:23:09.987646+0000","last_undegraded":"2023-10-06T02:23:09.987646+0000","last_fullsized":"2023-10-06T02:23:09.987646+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:44:16.553231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.009997+0000","last_change":"2023-10-06T02:23:09.997248+0000","last_active":"2023-10-06T02:23:10.009997+0000","last_peered":"2023-10-06T02:23:10.009997+0000","last_clean":"2023-10-06T02:23:10.009997+0000","last_became_active":"2023-10-06T02:23:09.996897+0000","last_became_peered":"2023-10-06T02:23:09.996897+0000","last_unstale":"2023-10-06T02:23:10.009997+0000","last_undegraded":"2023-10-06T02:23:10.009997+0000","last_fullsized":"2023-10-06T02:23:10.009997+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:00:45.442861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.994041+0000","last_change":"2023-10-06T02:23:09.991324+0000","last_active":"2023-10-06T02:23:09.994041+0000","last_peered":"2023-10-06T02:23:09.994041+0000","last_clean":"2023-10-06T02:23:09.994041+0000","last_became_active":"2023-10-06T02:23:09.991018+0000","last_became_peered":"2023-10-06T02:23:09.991018+0000","last_unstale":"2023-10-06T02:23:09.994041+0000","last_undegraded":"2023-10-06T02:23:09.994041+0000","last_fullsized":"2023-10-06T02:23:09.994041+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:43:01.259350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.004545+0000","last_change":"2023-10-06T02:23:10.000917+0000","last_active":"2023-10-06T02:23:10.004545+0000","last_peered":"2023-10-06T02:23:10.004545+0000","last_clean":"2023-10-06T02:23:10.004545+0000","last_became_active":"2023-10-06T02:23:10.000580+0000","last_became_peered":"2023-10-06T02:23:10.000580+0000","last_unstale":"2023-10-06T02:23:10.004545+0000","last_undegraded":"2023-10-06T02:23:10.004545+0000","last_fullsized":"2023-10-06T02:23:10.004545+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:50:19.168882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.998825+0000","last_change":"2023-10-06T02:23:09.995748+0000","last_active":"2023-10-06T02:23:09.998825+0000","last_peered":"2023-10-06T02:23:09.998825+0000","last_clean":"2023-10-06T02:23:09.998825+0000","last_became_active":"2023-10-06T02:23:09.995433+0000","last_became_peered":"2023-10-06T02:23:09.995433+0000","last_unstale":"2023-10-06T02:23:09.998825+0000","last_undegraded":"2023-10-06T02:23:09.998825+0000","last_fullsized":"2023-10-06T02:23:09.998825+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:21.438589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.996641+0000","last_change":"2023-10-06T02:23:09.992834+0000","last_active":"2023-10-06T02:23:09.996641+0000","last_peered":"2023-10-06T02:23:09.996641+0000","last_clean":"2023-10-06T02:23:09.996641+0000","last_became_active":"2023-10-06T02:23:09.991971+0000","last_became_peered":"2023-10-06T02:23:09.991971+0000","last_unstale":"2023-10-06T02:23:09.996641+0000","last_undegraded":"2023-10-06T02:23:09.996641+0000","last_fullsized":"2023-10-06T02:23:09.996641+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:12.998563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.007651+0000","last_change":"2023-10-06T02:23:10.005469+0000","last_active":"2023-10-06T02:23:10.007651+0000","last_peered":"2023-10-06T02:23:10.007651+0000","last_clean":"2023-10-06T02:23:10.007651+0000","last_became_active":"2023-10-06T02:23:10.005132+0000","last_became_peered":"2023-10-06T02:23:10.005132+0000","last_unstale":"2023-10-06T02:23:10.007651+0000","last_undegraded":"2023-10-06T02:23:10.007651+0000","last_fullsized":"2023-10-06T02:23:10.007651+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:19:18.748586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.005833+0000","last_change":"2023-10-06T02:23:09.999977+0000","last_active":"2023-10-06T02:23:10.005833+0000","last_peered":"2023-10-06T02:23:10.005833+0000","last_clean":"2023-10-06T02:23:10.005833+0000","last_became_active":"2023-10-06T02:23:09.999733+0000","last_became_peered":"2023-10-06T02:23:09.999733+0000","last_unstale":"2023-10-06T02:23:10.005833+0000","last_undegraded":"2023-10-06T02:23:10.005833+0000","last_fullsized":"2023-10-06T02:23:10.005833+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:58:41.925466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.999279+0000","last_change":"2023-10-06T02:23:09.994580+0000","last_active":"2023-10-06T02:23:09.999279+0000","last_peered":"2023-10-06T02:23:09.999279+0000","last_clean":"2023-10-06T02:23:09.999279+0000","last_became_active":"2023-10-06T02:23:09.993941+0000","last_became_peered":"2023-10-06T02:23:09.993941+0000","last_unstale":"2023-10-06T02:23:09.999279+0000","last_undegraded":"2023-10-06T02:23:09.999279+0000","last_fullsized":"2023-10-06T02:23:09.999279+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:43:01.259350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.043858+0000","last_change":"2023-10-06T02:23:09.048996+0000","last_active":"2023-10-06T02:23:10.043858+0000","last_peered":"2023-10-06T02:23:10.043858+0000","last_clean":"2023-10-06T02:23:10.043858+0000","last_became_active":"2023-10-06T02:23:09.048662+0000","last_became_peered":"2023-10-06T02:23:09.048662+0000","last_unstale":"2023-10-06T02:23:10.043858+0000","last_undegraded":"2023-10-06T02:23:10.043858+0000","last_fullsized":"2023-10-06T02:23:10.043858+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:58.976414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.004959+0000","last_change":"2023-10-06T02:23:10.001176+0000","last_active":"2023-10-06T02:23:10.004959+0000","last_peered":"2023-10-06T02:23:10.004959+0000","last_clean":"2023-10-06T02:23:10.004959+0000","last_became_active":"2023-10-06T02:23:10.000887+0000","last_became_peered":"2023-10-06T02:23:10.000887+0000","last_unstale":"2023-10-06T02:23:10.004959+0000","last_undegraded":"2023-10-06T02:23:10.004959+0000","last_fullsized":"2023-10-06T02:23:10.004959+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:43:02.156237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.998513+0000","last_change":"2023-10-06T02:23:09.048083+0000","last_active":"2023-10-06T02:23:09.998513+0000","last_peered":"2023-10-06T02:23:09.998513+0000","last_clean":"2023-10-06T02:23:09.998513+0000","last_became_active":"2023-10-06T02:23:09.047701+0000","last_became_peered":"2023-10-06T02:23:09.047701+0000","last_unstale":"2023-10-06T02:23:09.998513+0000","last_undegraded":"2023-10-06T02:23:09.998513+0000","last_fullsized":"2023-10-06T02:23:09.998513+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:17:32.434412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.994267+0000","last_change":"2023-10-06T02:23:09.988810+0000","last_active":"2023-10-06T02:23:09.994267+0000","last_peered":"2023-10-06T02:23:09.994267+0000","last_clean":"2023-10-06T02:23:09.994267+0000","last_became_active":"2023-10-06T02:23:09.988419+0000","last_became_peered":"2023-10-06T02:23:09.988419+0000","last_unstale":"2023-10-06T02:23:09.994267+0000","last_undegraded":"2023-10-06T02:23:09.994267+0000","last_fullsized":"2023-10-06T02:23:09.994267+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:43:01.259350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.976287+0000","last_change":"2023-10-06T02:23:09.044642+0000","last_active":"2023-10-06T02:23:09.976287+0000","last_peered":"2023-10-06T02:23:09.976287+0000","last_clean":"2023-10-06T02:23:09.976287+0000","last_became_active":"2023-10-06T02:23:09.044224+0000","last_became_peered":"2023-10-06T02:23:09.044224+0000","last_unstale":"2023-10-06T02:23:09.976287+0000","last_undegraded":"2023-10-06T02:23:09.976287+0000","last_fullsized":"2023-10-06T02:23:09.976287+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:31:52.666890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.986940+0000","last_change":"2023-10-06T02:23:09.985837+0000","last_active":"2023-10-06T02:23:09.986940+0000","last_peered":"2023-10-06T02:23:09.986940+0000","last_clean":"2023-10-06T02:23:09.986940+0000","last_became_active":"2023-10-06T02:23:09.985447+0000","last_became_peered":"2023-10-06T02:23:09.985447+0000","last_unstale":"2023-10-06T02:23:09.986940+0000","last_undegraded":"2023-10-06T02:23:09.986940+0000","last_fullsized":"2023-10-06T02:23:09.986940+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:31:53.674450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.979513+0000","last_change":"2023-10-06T02:23:09.040730+0000","last_active":"2023-10-06T02:23:09.979513+0000","last_peered":"2023-10-06T02:23:09.979513+0000","last_clean":"2023-10-06T02:23:09.979513+0000","last_became_active":"2023-10-06T02:23:09.040327+0000","last_became_peered":"2023-10-06T02:23:09.040327+0000","last_unstale":"2023-10-06T02:23:09.979513+0000","last_undegraded":"2023-10-06T02:23:09.979513+0000","last_fullsized":"2023-10-06T02:23:09.979513+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:07:56.722823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.998851+0000","last_change":"2023-10-06T02:23:09.995950+0000","last_active":"2023-10-06T02:23:09.998851+0000","last_peered":"2023-10-06T02:23:09.998851+0000","last_clean":"2023-10-06T02:23:09.998851+0000","last_became_active":"2023-10-06T02:23:09.995679+0000","last_became_peered":"2023-10-06T02:23:09.995679+0000","last_unstale":"2023-10-06T02:23:09.998851+0000","last_undegraded":"2023-10-06T02:23:09.998851+0000","last_fullsized":"2023-10-06T02:23:09.998851+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:23:26.580218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.976082+0000","last_change":"2023-10-06T02:23:09.040245+0000","last_active":"2023-10-06T02:23:09.976082+0000","last_peered":"2023-10-06T02:23:09.976082+0000","last_clean":"2023-10-06T02:23:09.976082+0000","last_became_active":"2023-10-06T02:23:09.039660+0000","last_became_peered":"2023-10-06T02:23:09.039660+0000","last_unstale":"2023-10-06T02:23:09.976082+0000","last_undegraded":"2023-10-06T02:23:09.976082+0000","last_fullsized":"2023-10-06T02:23:09.976082+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:57.514051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.994925+0000","last_change":"2023-10-06T02:23:09.992417+0000","last_active":"2023-10-06T02:23:09.994925+0000","last_peered":"2023-10-06T02:23:09.994925+0000","last_clean":"2023-10-06T02:23:09.994925+0000","last_became_active":"2023-10-06T02:23:09.992155+0000","last_became_peered":"2023-10-06T02:23:09.992155+0000","last_unstale":"2023-10-06T02:23:09.994925+0000","last_undegraded":"2023-10-06T02:23:09.994925+0000","last_fullsized":"2023-10-06T02:23:09.994925+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:39:15.275881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"22'3","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.044297+0000","last_change":"2023-10-06T02:23:09.046655+0000","last_active":"2023-10-06T02:23:10.044297+0000","last_peered":"2023-10-06T02:23:10.044297+0000","last_clean":"2023-10-06T02:23:10.044297+0000","last_became_active":"2023-10-06T02:23:09.046335+0000","last_became_peered":"2023-10-06T02:23:09.046335+0000","last_unstale":"2023-10-06T02:23:10.044297+0000","last_undegraded":"2023-10-06T02:23:10.044297+0000","last_fullsized":"2023-10-06T02:23:10.044297+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:53:00.225521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.007635+0000","last_change":"2023-10-06T02:23:10.005359+0000","last_active":"2023-10-06T02:23:10.007635+0000","last_peered":"2023-10-06T02:23:10.007635+0000","last_clean":"2023-10-06T02:23:10.007635+0000","last_became_active":"2023-10-06T02:23:10.005115+0000","last_became_peered":"2023-10-06T02:23:10.005115+0000","last_unstale":"2023-10-06T02:23:10.007635+0000","last_undegraded":"2023-10-06T02:23:10.007635+0000","last_fullsized":"2023-10-06T02:23:10.007635+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:29:00.771363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"22'6","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.052236+0000","last_change":"2023-10-06T02:23:09.046958+0000","last_active":"2023-10-06T02:23:10.052236+0000","last_peered":"2023-10-06T02:23:10.052236+0000","last_clean":"2023-10-06T02:23:10.052236+0000","last_became_active":"2023-10-06T02:23:09.044018+0000","last_became_peered":"2023-10-06T02:23:09.044018+0000","last_unstale":"2023-10-06T02:23:10.052236+0000","last_undegraded":"2023-10-06T02:23:10.052236+0000","last_fullsized":"2023-10-06T02:23:10.052236+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:41.039005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.992467+0000","last_change":"2023-10-06T02:23:09.987559+0000","last_active":"2023-10-06T02:23:09.992467+0000","last_peered":"2023-10-06T02:23:09.992467+0000","last_clean":"2023-10-06T02:23:09.992467+0000","last_became_active":"2023-10-06T02:23:09.987264+0000","last_became_peered":"2023-10-06T02:23:09.987264+0000","last_unstale":"2023-10-06T02:23:09.992467+0000","last_undegraded":"2023-10-06T02:23:09.992467+0000","last_fullsized":"2023-10-06T02:23:09.992467+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:44:41.608418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.049676+0000","last_change":"2023-10-06T02:23:09.048307+0000","last_active":"2023-10-06T02:23:10.049676+0000","last_peered":"2023-10-06T02:23:10.049676+0000","last_clean":"2023-10-06T02:23:10.049676+0000","last_became_active":"2023-10-06T02:23:09.047919+0000","last_became_peered":"2023-10-06T02:23:09.047919+0000","last_unstale":"2023-10-06T02:23:10.049676+0000","last_undegraded":"2023-10-06T02:23:10.049676+0000","last_fullsized":"2023-10-06T02:23:10.049676+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:28:59.763803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.004797+0000","last_change":"2023-10-06T02:23:10.001689+0000","last_active":"2023-10-06T02:23:10.004797+0000","last_peered":"2023-10-06T02:23:10.004797+0000","last_clean":"2023-10-06T02:23:10.004797+0000","last_became_active":"2023-10-06T02:23:10.001418+0000","last_became_peered":"2023-10-06T02:23:10.001418+0000","last_unstale":"2023-10-06T02:23:10.004797+0000","last_undegraded":"2023-10-06T02:23:10.004797+0000","last_fullsized":"2023-10-06T02:23:10.004797+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:53:54.469523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.977377+0000","last_change":"2023-10-06T02:23:09.041968+0000","last_active":"2023-10-06T02:23:09.977377+0000","last_peered":"2023-10-06T02:23:09.977377+0000","last_clean":"2023-10-06T02:23:09.977377+0000","last_became_active":"2023-10-06T02:23:09.041659+0000","last_became_peered":"2023-10-06T02:23:09.041659+0000","last_unstale":"2023-10-06T02:23:09.977377+0000","last_undegraded":"2023-10-06T02:23:09.977377+0000","last_fullsized":"2023-10-06T02:23:09.977377+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:44:15.545672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.992112+0000","last_change":"2023-10-06T02:23:09.989200+0000","last_active":"2023-10-06T02:23:09.992112+0000","last_peered":"2023-10-06T02:23:09.992112+0000","last_clean":"2023-10-06T02:23:09.992112+0000","last_became_active":"2023-10-06T02:23:09.988895+0000","last_became_peered":"2023-10-06T02:23:09.988895+0000","last_unstale":"2023-10-06T02:23:09.992112+0000","last_undegraded":"2023-10-06T02:23:09.992112+0000","last_fullsized":"2023-10-06T02:23:09.992112+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:44:41.608418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.984229+0000","last_change":"2023-10-06T02:23:09.048119+0000","last_active":"2023-10-06T02:23:09.984229+0000","last_peered":"2023-10-06T02:23:09.984229+0000","last_clean":"2023-10-06T02:23:09.984229+0000","last_became_active":"2023-10-06T02:23:09.047678+0000","last_became_peered":"2023-10-06T02:23:09.047678+0000","last_unstale":"2023-10-06T02:23:09.984229+0000","last_undegraded":"2023-10-06T02:23:09.984229+0000","last_fullsized":"2023-10-06T02:23:09.984229+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:42:11.693138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.997921+0000","last_change":"2023-10-06T02:23:09.994722+0000","last_active":"2023-10-06T02:23:09.997921+0000","last_peered":"2023-10-06T02:23:09.997921+0000","last_clean":"2023-10-06T02:23:09.997921+0000","last_became_active":"2023-10-06T02:23:09.994392+0000","last_became_peered":"2023-10-06T02:23:09.994392+0000","last_unstale":"2023-10-06T02:23:09.997921+0000","last_undegraded":"2023-10-06T02:23:09.997921+0000","last_fullsized":"2023-10-06T02:23:09.997921+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:21.438589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"22'8","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.049205+0000","last_change":"2023-10-06T02:23:09.034244+0000","last_active":"2023-10-06T02:23:10.049205+0000","last_peered":"2023-10-06T02:23:10.049205+0000","last_clean":"2023-10-06T02:23:10.049205+0000","last_became_active":"2023-10-06T02:23:09.033861+0000","last_became_peered":"2023-10-06T02:23:09.033861+0000","last_unstale":"2023-10-06T02:23:10.049205+0000","last_undegraded":"2023-10-06T02:23:10.049205+0000","last_fullsized":"2023-10-06T02:23:10.049205+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:41.039005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.990663+0000","last_change":"2023-10-06T02:23:09.988390+0000","last_active":"2023-10-06T02:23:09.990663+0000","last_peered":"2023-10-06T02:23:09.990663+0000","last_clean":"2023-10-06T02:23:09.990663+0000","last_became_active":"2023-10-06T02:23:09.988060+0000","last_became_peered":"2023-10-06T02:23:09.988060+0000","last_unstale":"2023-10-06T02:23:09.990663+0000","last_undegraded":"2023-10-06T02:23:09.990663+0000","last_fullsized":"2023-10-06T02:23:09.990663+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:43:01.259350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.044194+0000","last_change":"2023-10-06T02:23:09.048150+0000","last_active":"2023-10-06T02:23:10.044194+0000","last_peered":"2023-10-06T02:23:10.044194+0000","last_clean":"2023-10-06T02:23:10.044194+0000","last_became_active":"2023-10-06T02:23:09.047768+0000","last_became_peered":"2023-10-06T02:23:09.047768+0000","last_unstale":"2023-10-06T02:23:10.044194+0000","last_undegraded":"2023-10-06T02:23:10.044194+0000","last_fullsized":"2023-10-06T02:23:10.044194+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:03:50.725863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.992100+0000","last_change":"2023-10-06T02:23:09.990032+0000","last_active":"2023-10-06T02:23:09.992100+0000","last_peered":"2023-10-06T02:23:09.992100+0000","last_clean":"2023-10-06T02:23:09.992100+0000","last_became_active":"2023-10-06T02:23:09.988801+0000","last_became_peered":"2023-10-06T02:23:09.988801+0000","last_unstale":"2023-10-06T02:23:09.992100+0000","last_undegraded":"2023-10-06T02:23:09.992100+0000","last_fullsized":"2023-10-06T02:23:09.992100+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:12.998563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.044129+0000","last_change":"2023-10-06T02:23:09.048438+0000","last_active":"2023-10-06T02:23:10.044129+0000","last_peered":"2023-10-06T02:23:10.044129+0000","last_clean":"2023-10-06T02:23:10.044129+0000","last_became_active":"2023-10-06T02:23:09.048051+0000","last_became_peered":"2023-10-06T02:23:09.048051+0000","last_unstale":"2023-10-06T02:23:10.044129+0000","last_undegraded":"2023-10-06T02:23:10.044129+0000","last_fullsized":"2023-10-06T02:23:10.044129+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:21:32.730148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.010287+0000","last_change":"2023-10-06T02:23:10.002031+0000","last_active":"2023-10-06T02:23:10.010287+0000","last_peered":"2023-10-06T02:23:10.010287+0000","last_clean":"2023-10-06T02:23:10.010287+0000","last_became_active":"2023-10-06T02:23:10.001754+0000","last_became_peered":"2023-10-06T02:23:10.001754+0000","last_unstale":"2023-10-06T02:23:10.010287+0000","last_undegraded":"2023-10-06T02:23:10.010287+0000","last_fullsized":"2023-10-06T02:23:10.010287+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:45:34.162627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.044320+0000","last_change":"2023-10-06T02:23:09.038304+0000","last_active":"2023-10-06T02:23:10.044320+0000","last_peered":"2023-10-06T02:23:10.044320+0000","last_clean":"2023-10-06T02:23:10.044320+0000","last_became_active":"2023-10-06T02:23:09.038031+0000","last_became_peered":"2023-10-06T02:23:09.038031+0000","last_unstale":"2023-10-06T02:23:10.044320+0000","last_undegraded":"2023-10-06T02:23:10.044320+0000","last_fullsized":"2023-10-06T02:23:10.044320+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:31:52.666890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.004414+0000","last_change":"2023-10-06T02:23:10.000692+0000","last_active":"2023-10-06T02:23:10.004414+0000","last_peered":"2023-10-06T02:23:10.004414+0000","last_clean":"2023-10-06T02:23:10.004414+0000","last_became_active":"2023-10-06T02:23:10.000368+0000","last_became_peered":"2023-10-06T02:23:10.000368+0000","last_unstale":"2023-10-06T02:23:10.004414+0000","last_undegraded":"2023-10-06T02:23:10.004414+0000","last_fullsized":"2023-10-06T02:23:10.004414+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:53:01.233080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.974959+0000","last_change":"2023-10-06T02:23:09.037238+0000","last_active":"2023-10-06T02:23:09.974959+0000","last_peered":"2023-10-06T02:23:09.974959+0000","last_clean":"2023-10-06T02:23:09.974959+0000","last_became_active":"2023-10-06T02:23:09.036871+0000","last_became_peered":"2023-10-06T02:23:09.036871+0000","last_unstale":"2023-10-06T02:23:09.974959+0000","last_undegraded":"2023-10-06T02:23:09.974959+0000","last_fullsized":"2023-10-06T02:23:09.974959+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:44:15.545672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.002856+0000","last_change":"2023-10-06T02:23:10.000807+0000","last_active":"2023-10-06T02:23:10.002856+0000","last_peered":"2023-10-06T02:23:10.002856+0000","last_clean":"2023-10-06T02:23:10.002856+0000","last_became_active":"2023-10-06T02:23:10.000502+0000","last_became_peered":"2023-10-06T02:23:10.000502+0000","last_unstale":"2023-10-06T02:23:10.002856+0000","last_undegraded":"2023-10-06T02:23:10.002856+0000","last_fullsized":"2023-10-06T02:23:10.002856+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:31:27.242353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.042047+0000","last_change":"2023-10-06T02:23:09.044437+0000","last_active":"2023-10-06T02:23:10.042047+0000","last_peered":"2023-10-06T02:23:10.042047+0000","last_clean":"2023-10-06T02:23:10.042047+0000","last_became_active":"2023-10-06T02:23:09.042657+0000","last_became_peered":"2023-10-06T02:23:09.042657+0000","last_unstale":"2023-10-06T02:23:10.042047+0000","last_undegraded":"2023-10-06T02:23:10.042047+0000","last_fullsized":"2023-10-06T02:23:10.042047+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:57:40.559177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.994640+0000","last_change":"2023-10-06T02:23:09.990834+0000","last_active":"2023-10-06T02:23:09.994640+0000","last_peered":"2023-10-06T02:23:09.994640+0000","last_clean":"2023-10-06T02:23:09.994640+0000","last_became_active":"2023-10-06T02:23:09.990192+0000","last_became_peered":"2023-10-06T02:23:09.990192+0000","last_unstale":"2023-10-06T02:23:09.994640+0000","last_undegraded":"2023-10-06T02:23:09.994640+0000","last_fullsized":"2023-10-06T02:23:09.994640+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:18:32.416904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.980739+0000","last_change":"2023-10-06T02:23:09.044880+0000","last_active":"2023-10-06T02:23:09.980739+0000","last_peered":"2023-10-06T02:23:09.980739+0000","last_clean":"2023-10-06T02:23:09.980739+0000","last_became_active":"2023-10-06T02:23:09.044556+0000","last_became_peered":"2023-10-06T02:23:09.044556+0000","last_unstale":"2023-10-06T02:23:09.980739+0000","last_undegraded":"2023-10-06T02:23:09.980739+0000","last_fullsized":"2023-10-06T02:23:09.980739+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:58.976414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.994703+0000","last_change":"2023-10-06T02:23:09.992094+0000","last_active":"2023-10-06T02:23:09.994703+0000","last_peered":"2023-10-06T02:23:09.994703+0000","last_clean":"2023-10-06T02:23:09.994703+0000","last_became_active":"2023-10-06T02:23:09.991699+0000","last_became_peered":"2023-10-06T02:23:09.991699+0000","last_unstale":"2023-10-06T02:23:09.994703+0000","last_undegraded":"2023-10-06T02:23:09.994703+0000","last_fullsized":"2023-10-06T02:23:09.994703+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:21.438589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.986400+0000","last_change":"2023-10-06T02:23:09.041591+0000","last_active":"2023-10-06T02:23:09.986400+0000","last_peered":"2023-10-06T02:23:09.986400+0000","last_clean":"2023-10-06T02:23:09.986400+0000","last_became_active":"2023-10-06T02:23:09.041239+0000","last_became_peered":"2023-10-06T02:23:09.041239+0000","last_unstale":"2023-10-06T02:23:09.986400+0000","last_undegraded":"2023-10-06T02:23:09.986400+0000","last_fullsized":"2023-10-06T02:23:09.986400+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:17:32.434412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.994672+0000","last_change":"2023-10-06T02:23:09.991765+0000","last_active":"2023-10-06T02:23:09.994672+0000","last_peered":"2023-10-06T02:23:09.994672+0000","last_clean":"2023-10-06T02:23:09.994672+0000","last_became_active":"2023-10-06T02:23:09.990991+0000","last_became_peered":"2023-10-06T02:23:09.990991+0000","last_unstale":"2023-10-06T02:23:09.994672+0000","last_undegraded":"2023-10-06T02:23:09.994672+0000","last_fullsized":"2023-10-06T02:23:09.994672+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:18:23.992787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.976060+0000","last_change":"2023-10-06T02:23:09.046400+0000","last_active":"2023-10-06T02:23:09.976060+0000","last_peered":"2023-10-06T02:23:09.976060+0000","last_clean":"2023-10-06T02:23:09.976060+0000","last_became_active":"2023-10-06T02:23:09.046039+0000","last_became_peered":"2023-10-06T02:23:09.046039+0000","last_unstale":"2023-10-06T02:23:09.976060+0000","last_undegraded":"2023-10-06T02:23:09.976060+0000","last_fullsized":"2023-10-06T02:23:09.976060+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:44:15.545672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.005866+0000","last_change":"2023-10-06T02:23:10.001037+0000","last_active":"2023-10-06T02:23:10.005866+0000","last_peered":"2023-10-06T02:23:10.005866+0000","last_clean":"2023-10-06T02:23:10.005866+0000","last_became_active":"2023-10-06T02:23:10.000479+0000","last_became_peered":"2023-10-06T02:23:10.000479+0000","last_unstale":"2023-10-06T02:23:10.005866+0000","last_undegraded":"2023-10-06T02:23:10.005866+0000","last_fullsized":"2023-10-06T02:23:10.005866+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:50:19.168882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.996789+0000","last_change":"2023-10-06T02:23:02.054039+0000","last_active":"2023-10-06T02:23:09.996789+0000","last_peered":"2023-10-06T02:23:09.996789+0000","last_clean":"2023-10-06T02:23:09.996789+0000","last_became_active":"2023-10-06T02:23:02.053742+0000","last_became_peered":"2023-10-06T02:23:02.053742+0000","last_unstale":"2023-10-06T02:23:09.996789+0000","last_undegraded":"2023-10-06T02:23:09.996789+0000","last_fullsized":"2023-10-06T02:23:09.996789+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:58:18.846475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.975329+0000","last_change":"2023-10-06T02:23:09.037244+0000","last_active":"2023-10-06T02:23:09.975329+0000","last_peered":"2023-10-06T02:23:09.975329+0000","last_clean":"2023-10-06T02:23:09.975329+0000","last_became_active":"2023-10-06T02:23:09.036880+0000","last_became_peered":"2023-10-06T02:23:09.036880+0000","last_unstale":"2023-10-06T02:23:09.975329+0000","last_undegraded":"2023-10-06T02:23:09.975329+0000","last_fullsized":"2023-10-06T02:23:09.975329+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:36:27.554258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.005453+0000","last_change":"2023-10-06T02:23:09.999775+0000","last_active":"2023-10-06T02:23:10.005453+0000","last_peered":"2023-10-06T02:23:10.005453+0000","last_clean":"2023-10-06T02:23:10.005453+0000","last_became_active":"2023-10-06T02:23:09.999472+0000","last_became_peered":"2023-10-06T02:23:09.999472+0000","last_unstale":"2023-10-06T02:23:10.005453+0000","last_undegraded":"2023-10-06T02:23:10.005453+0000","last_fullsized":"2023-10-06T02:23:10.005453+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:44:41.608418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.977354+0000","last_change":"2023-10-06T02:23:01.988631+0000","last_active":"2023-10-06T02:23:09.977354+0000","last_peered":"2023-10-06T02:23:09.977354+0000","last_clean":"2023-10-06T02:23:09.977354+0000","last_became_active":"2023-10-06T02:23:01.988303+0000","last_became_peered":"2023-10-06T02:23:01.988303+0000","last_unstale":"2023-10-06T02:23:09.977354+0000","last_undegraded":"2023-10-06T02:23:09.977354+0000","last_fullsized":"2023-10-06T02:23:09.977354+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:38.211466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.986211+0000","last_change":"2023-10-06T02:23:09.041835+0000","last_active":"2023-10-06T02:23:09.986211+0000","last_peered":"2023-10-06T02:23:09.986211+0000","last_clean":"2023-10-06T02:23:09.986211+0000","last_became_active":"2023-10-06T02:23:09.041521+0000","last_became_peered":"2023-10-06T02:23:09.041521+0000","last_unstale":"2023-10-06T02:23:09.986211+0000","last_undegraded":"2023-10-06T02:23:09.986211+0000","last_fullsized":"2023-10-06T02:23:09.986211+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:38.263723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.003455+0000","last_change":"2023-10-06T02:23:09.995055+0000","last_active":"2023-10-06T02:23:10.003455+0000","last_peered":"2023-10-06T02:23:10.003455+0000","last_clean":"2023-10-06T02:23:10.003455+0000","last_became_active":"2023-10-06T02:23:09.994216+0000","last_became_peered":"2023-10-06T02:23:09.994216+0000","last_unstale":"2023-10-06T02:23:10.003455+0000","last_undegraded":"2023-10-06T02:23:10.003455+0000","last_fullsized":"2023-10-06T02:23:10.003455+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:47:34.184576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.972958+0000","last_change":"2023-10-06T02:23:02.054338+0000","last_active":"2023-10-06T02:23:09.972958+0000","last_peered":"2023-10-06T02:23:09.972958+0000","last_clean":"2023-10-06T02:23:09.972958+0000","last_became_active":"2023-10-06T02:23:02.053991+0000","last_became_peered":"2023-10-06T02:23:02.053991+0000","last_unstale":"2023-10-06T02:23:09.972958+0000","last_undegraded":"2023-10-06T02:23:09.972958+0000","last_fullsized":"2023-10-06T02:23:09.972958+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:39:30.192838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.996619+0000","last_change":"2023-10-06T02:23:09.039360+0000","last_active":"2023-10-06T02:23:09.996619+0000","last_peered":"2023-10-06T02:23:09.996619+0000","last_clean":"2023-10-06T02:23:09.996619+0000","last_became_active":"2023-10-06T02:23:09.038571+0000","last_became_peered":"2023-10-06T02:23:09.038571+0000","last_unstale":"2023-10-06T02:23:09.996619+0000","last_undegraded":"2023-10-06T02:23:09.996619+0000","last_fullsized":"2023-10-06T02:23:09.996619+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:58.976414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.005846+0000","last_change":"2023-10-06T02:23:10.000821+0000","last_active":"2023-10-06T02:23:10.005846+0000","last_peered":"2023-10-06T02:23:10.005846+0000","last_clean":"2023-10-06T02:23:10.005846+0000","last_became_active":"2023-10-06T02:23:10.000203+0000","last_became_peered":"2023-10-06T02:23:10.000203+0000","last_unstale":"2023-10-06T02:23:10.005846+0000","last_undegraded":"2023-10-06T02:23:10.005846+0000","last_fullsized":"2023-10-06T02:23:10.005846+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:43:01.259350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.975054+0000","last_change":"2023-10-06T02:23:01.988140+0000","last_active":"2023-10-06T02:23:09.975054+0000","last_peered":"2023-10-06T02:23:09.975054+0000","last_clean":"2023-10-06T02:23:09.975054+0000","last_became_active":"2023-10-06T02:23:01.987769+0000","last_became_peered":"2023-10-06T02:23:01.987769+0000","last_unstale":"2023-10-06T02:23:09.975054+0000","last_undegraded":"2023-10-06T02:23:09.975054+0000","last_fullsized":"2023-10-06T02:23:09.975054+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:20:08.391912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":166,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.983731+0000","last_change":"2023-10-06T02:22:55.032519+0000","last_active":"2023-10-06T02:23:09.983731+0000","last_peered":"2023-10-06T02:23:09.983731+0000","last_clean":"2023-10-06T02:23:09.983731+0000","last_became_active":"2023-10-06T02:22:55.031367+0000","last_became_peered":"2023-10-06T02:22:55.031367+0000","last_unstale":"2023-10-06T02:23:09.983731+0000","last_undegraded":"2023-10-06T02:23:09.983731+0000","last_fullsized":"2023-10-06T02:23:09.983731+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:22:53.309097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:22:53.309097+0000","last_clean_scrub_stamp":"2023-10-06T02:22:53.309097+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:53:46.520909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.975628+0000","last_change":"2023-10-06T02:23:09.041907+0000","last_active":"2023-10-06T02:23:09.975628+0000","last_peered":"2023-10-06T02:23:09.975628+0000","last_clean":"2023-10-06T02:23:09.975628+0000","last_became_active":"2023-10-06T02:23:09.036698+0000","last_became_peered":"2023-10-06T02:23:09.036698+0000","last_unstale":"2023-10-06T02:23:09.975628+0000","last_undegraded":"2023-10-06T02:23:09.975628+0000","last_fullsized":"2023-10-06T02:23:09.975628+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:57.514051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.003862+0000","last_change":"2023-10-06T02:23:09.997846+0000","last_active":"2023-10-06T02:23:10.003862+0000","last_peered":"2023-10-06T02:23:10.003862+0000","last_clean":"2023-10-06T02:23:10.003862+0000","last_became_active":"2023-10-06T02:23:09.997558+0000","last_became_peered":"2023-10-06T02:23:09.997558+0000","last_unstale":"2023-10-06T02:23:10.003862+0000","last_undegraded":"2023-10-06T02:23:10.003862+0000","last_fullsized":"2023-10-06T02:23:10.003862+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:29:00.771363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.984381+0000","last_change":"2023-10-06T02:23:01.990773+0000","last_active":"2023-10-06T02:23:09.984381+0000","last_peered":"2023-10-06T02:23:09.984381+0000","last_clean":"2023-10-06T02:23:09.984381+0000","last_became_active":"2023-10-06T02:23:01.990511+0000","last_became_peered":"2023-10-06T02:23:01.990511+0000","last_unstale":"2023-10-06T02:23:09.984381+0000","last_undegraded":"2023-10-06T02:23:09.984381+0000","last_fullsized":"2023-10-06T02:23:09.984381+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:42:54.797731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.977043+0000","last_change":"2023-10-06T02:23:09.047067+0000","last_active":"2023-10-06T02:23:09.977043+0000","last_peered":"2023-10-06T02:23:09.977043+0000","last_clean":"2023-10-06T02:23:09.977043+0000","last_became_active":"2023-10-06T02:23:09.046735+0000","last_became_peered":"2023-10-06T02:23:09.046735+0000","last_unstale":"2023-10-06T02:23:09.977043+0000","last_undegraded":"2023-10-06T02:23:09.977043+0000","last_fullsized":"2023-10-06T02:23:09.977043+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:57.514051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.005223+0000","last_change":"2023-10-06T02:23:10.001635+0000","last_active":"2023-10-06T02:23:10.005223+0000","last_peered":"2023-10-06T02:23:10.005223+0000","last_clean":"2023-10-06T02:23:10.005223+0000","last_became_active":"2023-10-06T02:23:10.001341+0000","last_became_peered":"2023-10-06T02:23:10.001341+0000","last_unstale":"2023-10-06T02:23:10.005223+0000","last_undegraded":"2023-10-06T02:23:10.005223+0000","last_fullsized":"2023-10-06T02:23:10.005223+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:44:41.608418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.984779+0000","last_change":"2023-10-06T02:23:02.054069+0000","last_active":"2023-10-06T02:23:09.984779+0000","last_peered":"2023-10-06T02:23:09.984779+0000","last_clean":"2023-10-06T02:23:09.984779+0000","last_became_active":"2023-10-06T02:23:02.053739+0000","last_became_peered":"2023-10-06T02:23:02.053739+0000","last_unstale":"2023-10-06T02:23:09.984779+0000","last_undegraded":"2023-10-06T02:23:09.984779+0000","last_fullsized":"2023-10-06T02:23:09.984779+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:57:48.822608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.996931+0000","last_change":"2023-10-06T02:23:09.037195+0000","last_active":"2023-10-06T02:23:09.996931+0000","last_peered":"2023-10-06T02:23:09.996931+0000","last_clean":"2023-10-06T02:23:09.996931+0000","last_became_active":"2023-10-06T02:23:09.036925+0000","last_became_peered":"2023-10-06T02:23:09.036925+0000","last_unstale":"2023-10-06T02:23:09.996931+0000","last_undegraded":"2023-10-06T02:23:09.996931+0000","last_fullsized":"2023-10-06T02:23:09.996931+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:38.263723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.994210+0000","last_change":"2023-10-06T02:23:09.986467+0000","last_active":"2023-10-06T02:23:09.994210+0000","last_peered":"2023-10-06T02:23:09.994210+0000","last_clean":"2023-10-06T02:23:09.994210+0000","last_became_active":"2023-10-06T02:23:09.985750+0000","last_became_peered":"2023-10-06T02:23:09.985750+0000","last_unstale":"2023-10-06T02:23:09.994210+0000","last_undegraded":"2023-10-06T02:23:09.994210+0000","last_fullsized":"2023-10-06T02:23:09.994210+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:18:23.992787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.044047+0000","last_change":"2023-10-06T02:23:09.041622+0000","last_active":"2023-10-06T02:23:10.044047+0000","last_peered":"2023-10-06T02:23:10.044047+0000","last_clean":"2023-10-06T02:23:10.044047+0000","last_became_active":"2023-10-06T02:23:09.036680+0000","last_became_peered":"2023-10-06T02:23:09.036680+0000","last_unstale":"2023-10-06T02:23:10.044047+0000","last_undegraded":"2023-10-06T02:23:10.044047+0000","last_fullsized":"2023-10-06T02:23:10.044047+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:36:27.554258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.993558+0000","last_change":"2023-10-06T02:23:01.989934+0000","last_active":"2023-10-06T02:23:09.993558+0000","last_peered":"2023-10-06T02:23:09.993558+0000","last_clean":"2023-10-06T02:23:09.993558+0000","last_became_active":"2023-10-06T02:23:01.989706+0000","last_became_peered":"2023-10-06T02:23:01.989706+0000","last_unstale":"2023-10-06T02:23:09.993558+0000","last_undegraded":"2023-10-06T02:23:09.993558+0000","last_fullsized":"2023-10-06T02:23:09.993558+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:06.212251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.989446+0000","last_change":"2023-10-06T02:23:09.987681+0000","last_active":"2023-10-06T02:23:09.989446+0000","last_peered":"2023-10-06T02:23:09.989446+0000","last_clean":"2023-10-06T02:23:09.989446+0000","last_became_active":"2023-10-06T02:23:09.987439+0000","last_became_peered":"2023-10-06T02:23:09.987439+0000","last_unstale":"2023-10-06T02:23:09.989446+0000","last_undegraded":"2023-10-06T02:23:09.989446+0000","last_fullsized":"2023-10-06T02:23:09.989446+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:31:27.242353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.980493+0000","last_change":"2023-10-06T02:23:01.992935+0000","last_active":"2023-10-06T02:23:09.980493+0000","last_peered":"2023-10-06T02:23:09.980493+0000","last_clean":"2023-10-06T02:23:09.980493+0000","last_became_active":"2023-10-06T02:23:01.992568+0000","last_became_peered":"2023-10-06T02:23:01.992568+0000","last_unstale":"2023-10-06T02:23:09.980493+0000","last_undegraded":"2023-10-06T02:23:09.980493+0000","last_fullsized":"2023-10-06T02:23:09.980493+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:06.212251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"23'1","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T02:23:11.091780+0000","last_change":"2023-10-06T02:23:09.039504+0000","last_active":"2023-10-06T02:23:11.091780+0000","last_peered":"2023-10-06T02:23:11.091780+0000","last_clean":"2023-10-06T02:23:11.091780+0000","last_became_active":"2023-10-06T02:23:09.039056+0000","last_became_peered":"2023-10-06T02:23:09.039056+0000","last_unstale":"2023-10-06T02:23:11.091780+0000","last_undegraded":"2023-10-06T02:23:11.091780+0000","last_fullsized":"2023-10-06T02:23:11.091780+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:38.263723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.995987+0000","last_change":"2023-10-06T02:23:09.990800+0000","last_active":"2023-10-06T02:23:09.995987+0000","last_peered":"2023-10-06T02:23:09.995987+0000","last_clean":"2023-10-06T02:23:09.995987+0000","last_became_active":"2023-10-06T02:23:09.990531+0000","last_became_peered":"2023-10-06T02:23:09.990531+0000","last_unstale":"2023-10-06T02:23:09.995987+0000","last_undegraded":"2023-10-06T02:23:09.995987+0000","last_fullsized":"2023-10-06T02:23:09.995987+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:39:39.959479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.045165+0000","last_change":"2023-10-06T02:23:09.046230+0000","last_active":"2023-10-06T02:23:10.045165+0000","last_peered":"2023-10-06T02:23:10.045165+0000","last_clean":"2023-10-06T02:23:10.045165+0000","last_became_active":"2023-10-06T02:23:09.045976+0000","last_became_peered":"2023-10-06T02:23:09.045976+0000","last_unstale":"2023-10-06T02:23:10.045165+0000","last_undegraded":"2023-10-06T02:23:10.045165+0000","last_fullsized":"2023-10-06T02:23:10.045165+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:36:27.554258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.998669+0000","last_change":"2023-10-06T02:23:09.995680+0000","last_active":"2023-10-06T02:23:09.998669+0000","last_peered":"2023-10-06T02:23:09.998669+0000","last_clean":"2023-10-06T02:23:09.998669+0000","last_became_active":"2023-10-06T02:23:09.995375+0000","last_became_peered":"2023-10-06T02:23:09.995375+0000","last_unstale":"2023-10-06T02:23:09.998669+0000","last_undegraded":"2023-10-06T02:23:09.998669+0000","last_fullsized":"2023-10-06T02:23:09.998669+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:36:28.561818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.046988+0000","last_change":"2023-10-06T02:23:09.043361+0000","last_active":"2023-10-06T02:23:10.046988+0000","last_peered":"2023-10-06T02:23:10.046988+0000","last_clean":"2023-10-06T02:23:10.046988+0000","last_became_active":"2023-10-06T02:23:09.042973+0000","last_became_peered":"2023-10-06T02:23:09.042973+0000","last_unstale":"2023-10-06T02:23:10.046988+0000","last_undegraded":"2023-10-06T02:23:10.046988+0000","last_fullsized":"2023-10-06T02:23:10.046988+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:31:52.666890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.992487+0000","last_change":"2023-10-06T02:23:09.988158+0000","last_active":"2023-10-06T02:23:09.992487+0000","last_peered":"2023-10-06T02:23:09.992487+0000","last_clean":"2023-10-06T02:23:09.992487+0000","last_became_active":"2023-10-06T02:23:09.987844+0000","last_became_peered":"2023-10-06T02:23:09.987844+0000","last_unstale":"2023-10-06T02:23:09.992487+0000","last_undegraded":"2023-10-06T02:23:09.992487+0000","last_fullsized":"2023-10-06T02:23:09.992487+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:10:59.616168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.045352+0000","last_change":"2023-10-06T02:23:09.044069+0000","last_active":"2023-10-06T02:23:10.045352+0000","last_peered":"2023-10-06T02:23:10.045352+0000","last_clean":"2023-10-06T02:23:10.045352+0000","last_became_active":"2023-10-06T02:23:09.043700+0000","last_became_peered":"2023-10-06T02:23:09.043700+0000","last_unstale":"2023-10-06T02:23:10.045352+0000","last_undegraded":"2023-10-06T02:23:10.045352+0000","last_fullsized":"2023-10-06T02:23:10.045352+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:57:40.559177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.010156+0000","last_change":"2023-10-06T02:23:09.999700+0000","last_active":"2023-10-06T02:23:10.010156+0000","last_peered":"2023-10-06T02:23:10.010156+0000","last_clean":"2023-10-06T02:23:10.010156+0000","last_became_active":"2023-10-06T02:23:09.997418+0000","last_became_peered":"2023-10-06T02:23:09.997418+0000","last_unstale":"2023-10-06T02:23:10.010156+0000","last_undegraded":"2023-10-06T02:23:10.010156+0000","last_fullsized":"2023-10-06T02:23:10.010156+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:53:54.469523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.041904+0000","last_change":"2023-10-06T02:23:09.034024+0000","last_active":"2023-10-06T02:23:10.041904+0000","last_peered":"2023-10-06T02:23:10.041904+0000","last_clean":"2023-10-06T02:23:10.041904+0000","last_became_active":"2023-10-06T02:23:09.033389+0000","last_became_peered":"2023-10-06T02:23:09.033389+0000","last_unstale":"2023-10-06T02:23:10.041904+0000","last_undegraded":"2023-10-06T02:23:10.041904+0000","last_fullsized":"2023-10-06T02:23:10.041904+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:03:50.725863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.991920+0000","last_change":"2023-10-06T02:23:09.984080+0000","last_active":"2023-10-06T02:23:09.991920+0000","last_peered":"2023-10-06T02:23:09.991920+0000","last_clean":"2023-10-06T02:23:09.991920+0000","last_became_active":"2023-10-06T02:23:09.983808+0000","last_became_peered":"2023-10-06T02:23:09.983808+0000","last_unstale":"2023-10-06T02:23:09.991920+0000","last_undegraded":"2023-10-06T02:23:09.991920+0000","last_fullsized":"2023-10-06T02:23:09.991920+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:12.998563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"22'4","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.045612+0000","last_change":"2023-10-06T02:23:09.037190+0000","last_active":"2023-10-06T02:23:10.045612+0000","last_peered":"2023-10-06T02:23:10.045612+0000","last_clean":"2023-10-06T02:23:10.045612+0000","last_became_active":"2023-10-06T02:23:09.036872+0000","last_became_peered":"2023-10-06T02:23:09.036872+0000","last_unstale":"2023-10-06T02:23:10.045612+0000","last_undegraded":"2023-10-06T02:23:10.045612+0000","last_fullsized":"2023-10-06T02:23:10.045612+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:57:40.559177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.994436+0000","last_change":"2023-10-06T02:23:09.989896+0000","last_active":"2023-10-06T02:23:09.994436+0000","last_peered":"2023-10-06T02:23:09.994436+0000","last_clean":"2023-10-06T02:23:09.994436+0000","last_became_active":"2023-10-06T02:23:09.989525+0000","last_became_peered":"2023-10-06T02:23:09.989525+0000","last_unstale":"2023-10-06T02:23:09.994436+0000","last_undegraded":"2023-10-06T02:23:09.994436+0000","last_fullsized":"2023-10-06T02:23:09.994436+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:58:41.925466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.992536+0000","last_change":"2023-10-06T02:23:09.988852+0000","last_active":"2023-10-06T02:23:09.992536+0000","last_peered":"2023-10-06T02:23:09.992536+0000","last_clean":"2023-10-06T02:23:09.992536+0000","last_became_active":"2023-10-06T02:23:09.988527+0000","last_became_peered":"2023-10-06T02:23:09.988527+0000","last_unstale":"2023-10-06T02:23:09.992536+0000","last_undegraded":"2023-10-06T02:23:09.992536+0000","last_fullsized":"2023-10-06T02:23:09.992536+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:03:51.733423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.001202+0000","last_change":"2023-10-06T02:23:09.999698+0000","last_active":"2023-10-06T02:23:10.001202+0000","last_peered":"2023-10-06T02:23:10.001202+0000","last_clean":"2023-10-06T02:23:10.001202+0000","last_became_active":"2023-10-06T02:23:09.999418+0000","last_became_peered":"2023-10-06T02:23:09.999418+0000","last_unstale":"2023-10-06T02:23:10.001202+0000","last_undegraded":"2023-10-06T02:23:10.001202+0000","last_fullsized":"2023-10-06T02:23:10.001202+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:10:59.616168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.010180+0000","last_change":"2023-10-06T02:23:10.000203+0000","last_active":"2023-10-06T02:23:10.010180+0000","last_peered":"2023-10-06T02:23:10.010180+0000","last_clean":"2023-10-06T02:23:10.010180+0000","last_became_active":"2023-10-06T02:23:09.999397+0000","last_became_peered":"2023-10-06T02:23:09.999397+0000","last_unstale":"2023-10-06T02:23:10.010180+0000","last_undegraded":"2023-10-06T02:23:10.010180+0000","last_fullsized":"2023-10-06T02:23:10.010180+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:31:27.242353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.989526+0000","last_change":"2023-10-06T02:23:09.986996+0000","last_active":"2023-10-06T02:23:09.989526+0000","last_peered":"2023-10-06T02:23:09.989526+0000","last_clean":"2023-10-06T02:23:09.989526+0000","last_became_active":"2023-10-06T02:23:09.986723+0000","last_became_peered":"2023-10-06T02:23:09.986723+0000","last_unstale":"2023-10-06T02:23:09.989526+0000","last_undegraded":"2023-10-06T02:23:09.989526+0000","last_fullsized":"2023-10-06T02:23:09.989526+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:19:29.462977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.007171+0000","last_change":"2023-10-06T02:23:10.003345+0000","last_active":"2023-10-06T02:23:10.007171+0000","last_peered":"2023-10-06T02:23:10.007171+0000","last_clean":"2023-10-06T02:23:10.007171+0000","last_became_active":"2023-10-06T02:23:10.003023+0000","last_became_peered":"2023-10-06T02:23:10.003023+0000","last_unstale":"2023-10-06T02:23:10.007171+0000","last_undegraded":"2023-10-06T02:23:10.007171+0000","last_fullsized":"2023-10-06T02:23:10.007171+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:18:23.992787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298132,"kb_used_data":844,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073708,"statfs":{"total":96636764160,"available":96331476992,"internally_reserved":0,"allocated":864256,"data_stored":681439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57399999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68899999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297524,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074316,"statfs":{"total":96636764160,"available":96332099584,"internally_reserved":0,"allocated":241664,"data_stored":89319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66500000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79900000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297524,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074316,"statfs":{"total":96636764160,"available":96332099584,"internally_reserved":0,"allocated":241664,"data_stored":89319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36699999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59799999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.62}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298108,"kb_used_data":820,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073732,"statfs":{"total":96636764160,"available":96331501568,"internally_reserved":0,"allocated":839680,"data_stored":680241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70799999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93200000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84799999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297524,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074316,"statfs":{"total":96636764160,"available":96332099584,"internally_reserved":0,"allocated":241664,"data_stored":89319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.625}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77200000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85599999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92800000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98899999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297528,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074312,"statfs":{"total":96636764160,"available":96332095488,"internally_reserved":0,"allocated":245760,"data_stored":89409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91100000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79600000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83699999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297544,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074296,"statfs":{"total":96636764160,"available":96332079104,"internally_reserved":0,"allocated":262144,"data_stored":90911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63300000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297532,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074308,"statfs":{"total":96636764160,"available":96332091392,"internally_reserved":0,"allocated":249856,"data_stored":89943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57799999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68700000000000006}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81000000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T02:23:19.771 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T02:23:19.771 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-10-06T02:23:19.771 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-10-06T02:23:19.771 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-10-06T02:23:19.771 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-10-06T02:23:19.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.992+0000 7f98b999d700 1 Processor -- start 2023-10-06T02:23:19.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.992+0000 7f98b999d700 1 -- start start 2023-10-06T02:23:19.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.993+0000 7f98b999d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f98b406e0a0 0x7f98b40c6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:19.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.993+0000 7f98b999d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98b406a490 con 0x7f98b4150380 2023-10-06T02:23:19.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.993+0000 7f98b999d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98b40c7490 con 0x7f98b406f3a0 2023-10-06T02:23:19.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.993+0000 7f98b999d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98b40c7770 con 0x7f98b406e0a0 2023-10-06T02:23:19.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.993+0000 7f98b919c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f98b4150380 0x7f98b406a920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49224/0 (socket says 172.21.15.31:49224) 2023-10-06T02:23:19.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.993+0000 7f98b919c700 1 -- 172.21.15.31:0/2583466013 learned_addr learned my addr 172.21.15.31:0/2583466013 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:23:19.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.994+0000 7f98b899b700 1 --2- 172.21.15.31:0/2583466013 >> v2:172.21.15.62:3300/0 conn(0x7f98b406e0a0 0x7f98b40c6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:19.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.994+0000 7f98b37fe700 1 -- 172.21.15.31:0/2583466013 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2781366892 0 0) 0x7f98b406a490 con 0x7f98b4150380 2023-10-06T02:23:19.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.994+0000 7f98b37fe700 1 -- 172.21.15.31:0/2583466013 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98a4004680 con 0x7f98b4150380 2023-10-06T02:23:19.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.994+0000 7f98b37fe700 1 -- 172.21.15.31:0/2583466013 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2296726998 0 0) 0x7f98a4004680 con 0x7f98b4150380 2023-10-06T02:23:19.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.994+0000 7f98b37fe700 1 -- 172.21.15.31:0/2583466013 >> v2:172.21.15.62:3300/0 conn(0x7f98b406e0a0 msgr2=0x7f98b40c6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:19.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.994+0000 7f98b37fe700 1 --2- 172.21.15.31:0/2583466013 >> v2:172.21.15.62:3300/0 conn(0x7f98b406e0a0 0x7f98b40c6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:19.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.994+0000 7f98b37fe700 1 -- 172.21.15.31:0/2583466013 >> v1:172.21.15.62:6789/0 conn(0x7f98b406f3a0 legacy=0x7f98b414fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:19.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.994+0000 7f98b37fe700 1 -- 172.21.15.31:0/2583466013 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98b40c7a50 con 0x7f98b4150380 2023-10-06T02:23:19.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.994+0000 7f98b37fe700 1 -- 172.21.15.31:0/2583466013 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7f98ac0017f0 con 0x7f98b4150380 2023-10-06T02:23:19.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.994+0000 7f98b37fe700 1 -- 172.21.15.31:0/2583466013 >> v1:172.21.15.31:6789/0 conn(0x7f98b4150380 legacy=0x7f98b406a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:19.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.994+0000 7f98b37fe700 1 --2- 172.21.15.31:0/2583466013 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98a4004bc0 0x7f98a4004fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:19.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.994+0000 7f98b37fe700 1 --2- 172.21.15.31:0/2583466013 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98a4005920 0x7f98a4005d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:19.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.994+0000 7f98b37fe700 1 --2- 172.21.15.31:0/2583466013 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f98a4006320 0x7f98a4003490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:19.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.994+0000 7f98b37fe700 1 -- 172.21.15.31:0/2583466013 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98b40c7770 con 0x7f98a4004bc0 2023-10-06T02:23:19.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.994+0000 7f98b37fe700 1 -- 172.21.15.31:0/2583466013 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a4003be0 con 0x7f98a4005920 2023-10-06T02:23:19.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.994+0000 7f98b37fe700 1 -- 172.21.15.31:0/2583466013 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98a4003da0 con 0x7f98a4006320 2023-10-06T02:23:19.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.994+0000 7f98b899b700 1 --2- 172.21.15.31:0/2583466013 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98a4004bc0 0x7f98a4004fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:19.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.995+0000 7f98b3fff700 1 --2- 172.21.15.31:0/2583466013 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98a4005920 0x7f98a4005d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:19.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.995+0000 7f98b919c700 1 --2- 172.21.15.31:0/2583466013 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f98a4006320 0x7f98a4003490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:19.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.995+0000 7f98b899b700 1 -- 172.21.15.31:0/2583466013 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f98a4006320 msgr2=0x7f98a4003490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:19.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.995+0000 7f98b899b700 1 --2- 172.21.15.31:0/2583466013 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f98a4006320 0x7f98a4003490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:19.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.995+0000 7f98b899b700 1 -- 172.21.15.31:0/2583466013 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98a4005920 msgr2=0x7f98a4005d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:19.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.995+0000 7f98b899b700 1 --2- 172.21.15.31:0/2583466013 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98a4005920 0x7f98a4005d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:19.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.995+0000 7f98b899b700 1 -- 172.21.15.31:0/2583466013 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98a4004680 con 0x7f98a4004bc0 2023-10-06T02:23:19.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.995+0000 7f98b919c700 1 --2- 172.21.15.31:0/2583466013 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f98a4006320 0x7f98a4003490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:23:19.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.995+0000 7f98b919c700 1 -- 172.21.15.31:0/2583466013 reap_dead start 2023-10-06T02:23:19.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.995+0000 7f98b899b700 1 --2- 172.21.15.31:0/2583466013 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98a4004bc0 0x7f98a4004fd0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f98a8009c00 tx=0x7f98a80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed3711463fc595b1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:19.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.995+0000 7f98b37fe700 1 -- 172.21.15.31:0/2583466013 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a800e710 con 0x7f98a4004bc0 2023-10-06T02:23:19.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.995+0000 7f98b37fe700 1 -- 172.21.15.31:0/2583466013 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98a800e870 con 0x7f98a4004bc0 2023-10-06T02:23:19.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.995+0000 7f98b37fe700 1 -- 172.21.15.31:0/2583466013 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a800ec30 con 0x7f98a4004bc0 2023-10-06T02:23:19.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.996+0000 7f98b999d700 1 -- 172.21.15.31:0/2583466013 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98a4004bc0 msgr2=0x7f98a4004fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:19.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.996+0000 7f98b999d700 1 --2- 172.21.15.31:0/2583466013 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98a4004bc0 0x7f98a4004fd0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f98a8009c00 tx=0x7f98a80091a0 comp rx=0 tx=0).stop 2023-10-06T02:23:19.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.996+0000 7f98b999d700 1 -- 172.21.15.31:0/2583466013 shutdown_connections 2023-10-06T02:23:19.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.996+0000 7f98b999d700 1 --2- 172.21.15.31:0/2583466013 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98a4004bc0 0x7f98a4004fd0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:19.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.996+0000 7f98b999d700 1 -- 172.21.15.31:0/2583466013 >> 172.21.15.31:0/2583466013 conn(0x7f98b406d170 msgr2=0x7f98b40cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:19.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.996+0000 7f98b999d700 1 -- 172.21.15.31:0/2583466013 shutdown_connections 2023-10-06T02:23:20.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.996+0000 7f98b999d700 1 -- 172.21.15.31:0/2583466013 wait complete. 2023-10-06T02:23:20.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.997+0000 7f98b999d700 1 Processor -- start 2023-10-06T02:23:20.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.998+0000 7f98b999d700 1 -- start start 2023-10-06T02:23:20.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.999+0000 7f98b999d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98a4004bc0 0x7f98b4158410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:20.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.999+0000 7f98b999d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f98b40b0860 0x7f98b414bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:20.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.999+0000 7f98b999d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98b4154ec0 0x7f98b4155360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:20.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.999+0000 7f98b999d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98b40c40a0 con 0x7f98a4004bc0 2023-10-06T02:23:20.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.999+0000 7f98b999d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98b41558a0 con 0x7f98b4154ec0 2023-10-06T02:23:20.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.999+0000 7f98b999d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98b414c310 con 0x7f98b40b0860 2023-10-06T02:23:20.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.999+0000 7f98b899b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98a4004bc0 0x7f98b4158410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:20.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.999+0000 7f98b899b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98a4004bc0 0x7f98b4158410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37430/0 (socket says 172.21.15.31:37430) 2023-10-06T02:23:20.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.999+0000 7f98b899b700 1 -- 172.21.15.31:0/1068669115 learned_addr learned my addr 172.21.15.31:0/1068669115 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:20.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.999+0000 7f98b3fff700 1 --2- 172.21.15.31:0/1068669115 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f98b40b0860 0x7f98b414bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:20.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.999+0000 7f98b919c700 1 --2- 172.21.15.31:0/1068669115 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98b4154ec0 0x7f98b4155360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:20.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.999+0000 7f98b899b700 1 -- 172.21.15.31:0/1068669115 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f98b40b0860 msgr2=0x7f98b414bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:20.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.999+0000 7f98b899b700 1 --2- 172.21.15.31:0/1068669115 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f98b40b0860 0x7f98b414bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:20.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.999+0000 7f98b899b700 1 -- 172.21.15.31:0/1068669115 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98b4154ec0 msgr2=0x7f98b4155360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:20.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.999+0000 7f98b899b700 1 --2- 172.21.15.31:0/1068669115 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98b4154ec0 0x7f98b4155360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:20.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.999+0000 7f98b899b700 1 -- 172.21.15.31:0/1068669115 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98b40b18c0 con 0x7f98a4004bc0 2023-10-06T02:23:20.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:19.999+0000 7f98b899b700 1 --2- 172.21.15.31:0/1068669115 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98a4004bc0 0x7f98b4158410 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f98a800e4b0 tx=0x7f98a801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:20.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.000+0000 7f98b0ff9700 1 -- 172.21.15.31:0/1068669115 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a8022070 con 0x7f98a4004bc0 2023-10-06T02:23:20.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.000+0000 7f98b0ff9700 1 -- 172.21.15.31:0/1068669115 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98a800e9e0 con 0x7f98a4004bc0 2023-10-06T02:23:20.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.000+0000 7f98b0ff9700 1 -- 172.21.15.31:0/1068669115 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a801c450 con 0x7f98a4004bc0 2023-10-06T02:23:20.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.000+0000 7f98b999d700 1 -- 172.21.15.31:0/1068669115 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98b40b1020 con 0x7f98a4004bc0 2023-10-06T02:23:20.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.000+0000 7f98b999d700 1 -- 172.21.15.31:0/1068669115 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98b40b15e0 con 0x7f98a4004bc0 2023-10-06T02:23:20.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.001+0000 7f98b999d700 1 -- 172.21.15.31:0/1068669115 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98b404c3c0 con 0x7f98a4004bc0 2023-10-06T02:23:20.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.002+0000 7f98b0ff9700 1 -- 172.21.15.31:0/1068669115 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f98a801c8a0 con 0x7f98a4004bc0 2023-10-06T02:23:20.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.003+0000 7f98b0ff9700 1 --2- 172.21.15.31:0/1068669115 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f9898066f20 0x7f98980693d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:20.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.003+0000 7f98b0ff9700 1 -- 172.21.15.31:0/1068669115 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f98a8085e20 con 0x7f98a4004bc0 2023-10-06T02:23:20.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.003+0000 7f98b3fff700 1 --2- 172.21.15.31:0/1068669115 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f9898066f20 0x7f98980693d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:20.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.004+0000 7f98b3fff700 1 --2- 172.21.15.31:0/1068669115 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f9898066f20 0x7f98980693d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f98b416d490 tx=0x7f98a0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:20.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.008+0000 7f98b0ff9700 1 -- 172.21.15.31:0/1068669115 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f98a8052f30 con 0x7f98a4004bc0 2023-10-06T02:23:20.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.270+0000 7f98b999d700 1 -- 172.21.15.31:0/1068669115 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f98b40c7d30 con 0x7f98a4004bc0 2023-10-06T02:23:20.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.273+0000 7f98b0ff9700 1 -- 172.21.15.31:0/1068669115 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+163699 (secure 0 0 0) 0x7f98a8006cb0 con 0x7f98a4004bc0 2023-10-06T02:23:20.275 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:23:20.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.276+0000 7f98b999d700 1 -- 172.21.15.31:0/1068669115 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f9898066f20 msgr2=0x7f98980693d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:20.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.276+0000 7f98b999d700 1 --2- 172.21.15.31:0/1068669115 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f9898066f20 0x7f98980693d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f98b416d490 tx=0x7f98a0009040 comp rx=0 tx=0).stop 2023-10-06T02:23:20.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.276+0000 7f98b999d700 1 -- 172.21.15.31:0/1068669115 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98a4004bc0 msgr2=0x7f98b4158410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:20.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.276+0000 7f98b999d700 1 --2- 172.21.15.31:0/1068669115 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98a4004bc0 0x7f98b4158410 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f98a800e4b0 tx=0x7f98a801a000 comp rx=0 tx=0).stop 2023-10-06T02:23:20.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.276+0000 7f98b999d700 1 -- 172.21.15.31:0/1068669115 shutdown_connections 2023-10-06T02:23:20.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.276+0000 7f98b999d700 1 --2- 172.21.15.31:0/1068669115 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98b4154ec0 0x7f98b4155360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:20.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.276+0000 7f98b999d700 1 --2- 172.21.15.31:0/1068669115 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98a4004bc0 0x7f98b4158410 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:20.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.276+0000 7f98b999d700 1 --2- 172.21.15.31:0/1068669115 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f9898066f20 0x7f98980693d0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:20.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.276+0000 7f98b999d700 1 --2- 172.21.15.31:0/1068669115 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f98b40b0860 0x7f98b414bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:20.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.276+0000 7f98b999d700 1 -- 172.21.15.31:0/1068669115 >> 172.21.15.31:0/1068669115 conn(0x7f98b406d170 msgr2=0x7f98b414dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:20.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.276+0000 7f98b999d700 1 -- 172.21.15.31:0/1068669115 shutdown_connections 2023-10-06T02:23:20.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.276+0000 7f98b999d700 1 -- 172.21.15.31:0/1068669115 wait complete. 2023-10-06T02:23:20.293 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":5,"active_gid":4107,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":85292},{"type":"v1","addr":"172.21.15.31:6833","nonce":85292}]},"active_addr":"172.21.15.31:6833/85292","active_change":"2023-10-06T02:22:47.104009+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":4109,"name":"x","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.31:0","nonce":2607073168}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.31:0","nonce":3265960029}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.31:0","nonce":2878303473}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.31:0","nonce":3769130347}]}]} 2023-10-06T02:23:20.297 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-10-06T02:23:20.297 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-10-06T02:23:20.297 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:23:20.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.485+0000 7f108bfff700 1 Processor -- start 2023-10-06T02:23:20.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.485+0000 7f108bfff700 1 -- start start 2023-10-06T02:23:20.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.486+0000 7f108bfff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f108c15de50 0x7f108c0cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:20.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.486+0000 7f108bfff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f108c0cef40 0x7f108c15c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:20.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.486+0000 7f108bfff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f108c0cf3c0 con 0x7f108c153c30 2023-10-06T02:23:20.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.486+0000 7f108bfff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f108c15afd0 con 0x7f108c15de50 2023-10-06T02:23:20.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.486+0000 7f108bfff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f108c15b2b0 con 0x7f108c0cef40 2023-10-06T02:23:20.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.487+0000 7f108a7fc700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f108c15de50 0x7f108c0cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:20.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.487+0000 7f108b7fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f108c0cef40 0x7f108c15c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:20.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.487+0000 7f108a7fc700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f108c15de50 0x7f108c0cea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:47172/0 (socket says 172.21.15.31:47172) 2023-10-06T02:23:20.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.487+0000 7f108a7fc700 1 -- 172.21.15.31:0/2334779848 learned_addr learned my addr 172.21.15.31:0/2334779848 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:20.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.487+0000 7f108affd700 1 --1- 172.21.15.31:0/2334779848 >> v1:172.21.15.62:6789/0 conn(0x7f108c153c30 0x7f108c15a5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:33866/0 (socket says 172.21.15.31:33866) 2023-10-06T02:23:20.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.487+0000 7f1089ffb700 1 -- 172.21.15.31:0/2334779848 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3769649090 0 0) 0x7f108c0cf3c0 con 0x7f108c153c30 2023-10-06T02:23:20.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.487+0000 7f1089ffb700 1 -- 172.21.15.31:0/2334779848 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1074003570 con 0x7f108c153c30 2023-10-06T02:23:20.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.487+0000 7f108b7fe700 1 -- 172.21.15.31:0/2334779848 >> v2:172.21.15.62:3300/0 conn(0x7f108c15de50 msgr2=0x7f108c0cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:20.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.487+0000 7f108b7fe700 1 --2- 172.21.15.31:0/2334779848 >> v2:172.21.15.62:3300/0 conn(0x7f108c15de50 0x7f108c0cea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:20.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.487+0000 7f108b7fe700 1 -- 172.21.15.31:0/2334779848 >> v1:172.21.15.62:6789/0 conn(0x7f108c153c30 legacy=0x7f108c15a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:20.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.487+0000 7f108b7fe700 1 -- 172.21.15.31:0/2334779848 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f108c15b590 con 0x7f108c0cef40 2023-10-06T02:23:20.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.487+0000 7f108a7fc700 1 --2- 172.21.15.31:0/2334779848 >> v2:172.21.15.62:3300/0 conn(0x7f108c15de50 0x7f108c0cea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:23:20.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.488+0000 7f108b7fe700 1 --2- 172.21.15.31:0/2334779848 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f108c0cef40 0x7f108c15c070 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f1084009e60 tx=0x7f1084009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b83461680d11a0f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:20.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.488+0000 7f1089ffb700 1 -- 172.21.15.31:0/2334779848 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108400e070 con 0x7f108c0cef40 2023-10-06T02:23:20.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.488+0000 7f1089ffb700 1 -- 172.21.15.31:0/2334779848 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f108400bab0 con 0x7f108c0cef40 2023-10-06T02:23:20.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.488+0000 7f1089ffb700 1 -- 172.21.15.31:0/2334779848 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108400be70 con 0x7f108c0cef40 2023-10-06T02:23:20.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.488+0000 7f108bfff700 1 -- 172.21.15.31:0/2334779848 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f108c0cef40 msgr2=0x7f108c15c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:20.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.488+0000 7f108bfff700 1 --2- 172.21.15.31:0/2334779848 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f108c0cef40 0x7f108c15c070 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f1084009e60 tx=0x7f1084009780 comp rx=0 tx=0).stop 2023-10-06T02:23:20.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.488+0000 7f108bfff700 1 -- 172.21.15.31:0/2334779848 shutdown_connections 2023-10-06T02:23:20.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.488+0000 7f108bfff700 1 --2- 172.21.15.31:0/2334779848 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f108c0cef40 0x7f108c15c070 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:20.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.488+0000 7f108bfff700 1 --2- 172.21.15.31:0/2334779848 >> v2:172.21.15.62:3300/0 conn(0x7f108c15de50 0x7f108c0cea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:20.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.488+0000 7f108bfff700 1 -- 172.21.15.31:0/2334779848 >> 172.21.15.31:0/2334779848 conn(0x7f108c0703a0 msgr2=0x7f108c0707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:20.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.489+0000 7f108bfff700 1 -- 172.21.15.31:0/2334779848 shutdown_connections 2023-10-06T02:23:20.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.489+0000 7f108bfff700 1 -- 172.21.15.31:0/2334779848 wait complete. 2023-10-06T02:23:20.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.490+0000 7f108bfff700 1 Processor -- start 2023-10-06T02:23:20.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.490+0000 7f108bfff700 1 -- start start 2023-10-06T02:23:20.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.491+0000 7f108bfff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f108c0cef40 0x7f108c141db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:20.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.491+0000 7f108bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f108c153c30 0x7f108c0c9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:20.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.491+0000 7f108bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f108c15de50 0x7f108c0c7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:20.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.491+0000 7f108bfff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f108c0cc4e0 con 0x7f108c15de50 2023-10-06T02:23:20.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.491+0000 7f108bfff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f108c0cb910 con 0x7f108c153c30 2023-10-06T02:23:20.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.491+0000 7f108bfff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f108c0cbbb0 con 0x7f108c0cef40 2023-10-06T02:23:20.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.491+0000 7f108b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f108c15de50 0x7f108c0c7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:20.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.491+0000 7f108affd700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f108c0cef40 0x7f108c141db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:20.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.492+0000 7f108a7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f108c153c30 0x7f108c0c9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:20.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.492+0000 7f108b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f108c15de50 0x7f108c0c7e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37442/0 (socket says 172.21.15.31:37442) 2023-10-06T02:23:20.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.492+0000 7f108b7fe700 1 -- 172.21.15.31:0/3702339536 learned_addr learned my addr 172.21.15.31:0/3702339536 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:20.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.492+0000 7f108affd700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f108c0cef40 0x7f108c141db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:56944/0 (socket says 172.21.15.31:56944) 2023-10-06T02:23:20.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.492+0000 7f108a7fc700 1 -- 172.21.15.31:0/3702339536 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f108c0cef40 msgr2=0x7f108c141db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:20.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.492+0000 7f108a7fc700 1 --2- 172.21.15.31:0/3702339536 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f108c0cef40 0x7f108c141db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:20.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.492+0000 7f108a7fc700 1 -- 172.21.15.31:0/3702339536 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f108c15de50 msgr2=0x7f108c0c7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:20.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.492+0000 7f108a7fc700 1 --2- 172.21.15.31:0/3702339536 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f108c15de50 0x7f108c0c7e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:20.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.492+0000 7f108a7fc700 1 -- 172.21.15.31:0/3702339536 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10800094f0 con 0x7f108c153c30 2023-10-06T02:23:20.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.492+0000 7f108affd700 1 --2- 172.21.15.31:0/3702339536 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f108c0cef40 0x7f108c141db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:23:20.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.492+0000 7f108b7fe700 1 --2- 172.21.15.31:0/3702339536 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f108c15de50 0x7f108c0c7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:23:20.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.492+0000 7f108a7fc700 1 --2- 172.21.15.31:0/3702339536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f108c153c30 0x7f108c0c9cd0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1080006fd0 tx=0x7f1080009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:20.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.493+0000 7f1072ffd700 1 -- 172.21.15.31:0/3702339536 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108000c850 con 0x7f108c153c30 2023-10-06T02:23:20.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.493+0000 7f1072ffd700 1 -- 172.21.15.31:0/3702339536 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f108000c9b0 con 0x7f108c153c30 2023-10-06T02:23:20.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.493+0000 7f108bfff700 1 -- 172.21.15.31:0/3702339536 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1084009a20 con 0x7f108c153c30 2023-10-06T02:23:20.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.493+0000 7f1072ffd700 1 -- 172.21.15.31:0/3702339536 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108000cd30 con 0x7f108c153c30 2023-10-06T02:23:20.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.493+0000 7f108bfff700 1 -- 172.21.15.31:0/3702339536 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f108c0cc170 con 0x7f108c153c30 2023-10-06T02:23:20.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.494+0000 7f108bfff700 1 -- 172.21.15.31:0/3702339536 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f108c04c3c0 con 0x7f108c153c30 2023-10-06T02:23:20.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.495+0000 7f1072ffd700 1 -- 172.21.15.31:0/3702339536 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1080013680 con 0x7f108c153c30 2023-10-06T02:23:20.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.496+0000 7f1072ffd700 1 --2- 172.21.15.31:0/3702339536 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f1078066f30 0x7f10780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:20.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.496+0000 7f1072ffd700 1 -- 172.21.15.31:0/3702339536 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1080084840 con 0x7f108c153c30 2023-10-06T02:23:20.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.496+0000 7f108affd700 1 --2- 172.21.15.31:0/3702339536 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f1078066f30 0x7f10780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:20.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.496+0000 7f108affd700 1 --2- 172.21.15.31:0/3702339536 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f1078066f30 0x7f10780693e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f107c004fb0 tx=0x7f107c007d40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:20.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.502+0000 7f1072ffd700 1 -- 172.21.15.31:0/3702339536 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f10800518b0 con 0x7f108c153c30 2023-10-06T02:23:20.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.705+0000 7f108bfff700 1 -- 172.21.15.31:0/3702339536 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f108c0caef0 con 0x7f108c153c30 2023-10-06T02:23:20.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.706+0000 7f1072ffd700 1 -- 172.21.15.31:0/3702339536 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16818 (secure 0 0 0) 0x7f1080054ed0 con 0x7f108c153c30 2023-10-06T02:23:20.707 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:23:20.707 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":23,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:23:10.970534+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:23:07.290269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T02:23:08.504679+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:23:20.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.709+0000 7f108bfff700 1 -- 172.21.15.31:0/3702339536 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f1078066f30 msgr2=0x7f10780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:20.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.709+0000 7f108bfff700 1 --2- 172.21.15.31:0/3702339536 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f1078066f30 0x7f10780693e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f107c004fb0 tx=0x7f107c007d40 comp rx=0 tx=0).stop 2023-10-06T02:23:20.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.709+0000 7f108bfff700 1 -- 172.21.15.31:0/3702339536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f108c153c30 msgr2=0x7f108c0c9cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:20.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.709+0000 7f108bfff700 1 --2- 172.21.15.31:0/3702339536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f108c153c30 0x7f108c0c9cd0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1080006fd0 tx=0x7f1080009250 comp rx=0 tx=0).stop 2023-10-06T02:23:20.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.709+0000 7f108bfff700 1 -- 172.21.15.31:0/3702339536 shutdown_connections 2023-10-06T02:23:20.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.709+0000 7f108bfff700 1 --2- 172.21.15.31:0/3702339536 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f108c15de50 0x7f108c0c7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:20.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.709+0000 7f108bfff700 1 --2- 172.21.15.31:0/3702339536 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f1078066f30 0x7f10780693e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:20.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.709+0000 7f108bfff700 1 --2- 172.21.15.31:0/3702339536 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f108c0cef40 0x7f108c141db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:20.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.709+0000 7f108bfff700 1 --2- 172.21.15.31:0/3702339536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f108c153c30 0x7f108c0c9cd0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:20.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.709+0000 7f108bfff700 1 -- 172.21.15.31:0/3702339536 >> 172.21.15.31:0/3702339536 conn(0x7f108c0703a0 msgr2=0x7f108c15bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:20.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.709+0000 7f108bfff700 1 -- 172.21.15.31:0/3702339536 shutdown_connections 2023-10-06T02:23:20.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.709+0000 7f108bfff700 1 -- 172.21.15.31:0/3702339536 wait complete. 2023-10-06T02:23:20.727 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-10-06T02:23:20.727 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:23:20.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.913+0000 7fb82239d700 1 Processor -- start 2023-10-06T02:23:20.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.914+0000 7fb82239d700 1 -- start start 2023-10-06T02:23:20.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.915+0000 7fb82239d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb81c13fb20 0x7fb81c15a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:20.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.915+0000 7fb82239d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb81c06aaa0 0x7fb81c0c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:20.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.915+0000 7fb82239d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb81c15ae70 con 0x7fb81c13fef0 2023-10-06T02:23:20.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.915+0000 7fb82239d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb81c06af60 con 0x7fb81c06aaa0 2023-10-06T02:23:20.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.915+0000 7fb82239d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb81c06b240 con 0x7fb81c13fb20 2023-10-06T02:23:20.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.915+0000 7fb82139b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb81c13fb20 0x7fb81c15a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:20.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.915+0000 7fb821b9c700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb81c06aaa0 0x7fb81c0c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:20.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.915+0000 7fb821b9c700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb81c06aaa0 0x7fb81c0c9d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:47186/0 (socket says 172.21.15.31:47186) 2023-10-06T02:23:20.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.916+0000 7fb821b9c700 1 -- 172.21.15.31:0/1946953449 learned_addr learned my addr 172.21.15.31:0/1946953449 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:20.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.916+0000 7fb813fff700 1 -- 172.21.15.31:0/1946953449 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 394513730 0 0) 0x7fb80c001160 con 0x7fb81c13fef0 2023-10-06T02:23:20.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.916+0000 7fb813fff700 1 -- 172.21.15.31:0/1946953449 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb808003570 con 0x7fb81c13fef0 2023-10-06T02:23:20.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.916+0000 7fb82139b700 1 -- 172.21.15.31:0/1946953449 >> v2:172.21.15.62:3300/0 conn(0x7fb81c06aaa0 msgr2=0x7fb81c0c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:20.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.916+0000 7fb82139b700 1 --2- 172.21.15.31:0/1946953449 >> v2:172.21.15.62:3300/0 conn(0x7fb81c06aaa0 0x7fb81c0c9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:20.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.916+0000 7fb82139b700 1 -- 172.21.15.31:0/1946953449 >> v1:172.21.15.62:6789/0 conn(0x7fb81c13fef0 legacy=0x7fb81c0c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:20.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.916+0000 7fb82139b700 1 -- 172.21.15.31:0/1946953449 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb81c06b520 con 0x7fb81c13fb20 2023-10-06T02:23:20.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.916+0000 7fb821b9c700 1 --2- 172.21.15.31:0/1946953449 >> v2:172.21.15.62:3300/0 conn(0x7fb81c06aaa0 0x7fb81c0c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:23:20.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.916+0000 7fb82139b700 1 --2- 172.21.15.31:0/1946953449 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb81c13fb20 0x7fb81c15a930 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb8180098e0 tx=0x7fb8180091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=444e0cca2f1c5d0d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:20.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.917+0000 7fb813fff700 1 -- 172.21.15.31:0/1946953449 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb818011630 con 0x7fb81c13fb20 2023-10-06T02:23:20.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.917+0000 7fb813fff700 1 -- 172.21.15.31:0/1946953449 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb818011790 con 0x7fb81c13fb20 2023-10-06T02:23:20.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.917+0000 7fb813fff700 1 -- 172.21.15.31:0/1946953449 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb818011b30 con 0x7fb81c13fb20 2023-10-06T02:23:20.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.917+0000 7fb82239d700 1 -- 172.21.15.31:0/1946953449 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb81c13fb20 msgr2=0x7fb81c15a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:20.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.917+0000 7fb82239d700 1 --2- 172.21.15.31:0/1946953449 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb81c13fb20 0x7fb81c15a930 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb8180098e0 tx=0x7fb8180091a0 comp rx=0 tx=0).stop 2023-10-06T02:23:20.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.917+0000 7fb82239d700 1 -- 172.21.15.31:0/1946953449 shutdown_connections 2023-10-06T02:23:20.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.917+0000 7fb82239d700 1 --2- 172.21.15.31:0/1946953449 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb81c13fb20 0x7fb81c15a930 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:20.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.917+0000 7fb82239d700 1 --2- 172.21.15.31:0/1946953449 >> v2:172.21.15.62:3300/0 conn(0x7fb81c06aaa0 0x7fb81c0c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:20.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.917+0000 7fb82239d700 1 -- 172.21.15.31:0/1946953449 >> 172.21.15.31:0/1946953449 conn(0x7fb81c06d170 msgr2=0x7fb81c0cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:20.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.917+0000 7fb82239d700 1 -- 172.21.15.31:0/1946953449 shutdown_connections 2023-10-06T02:23:20.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.917+0000 7fb82239d700 1 -- 172.21.15.31:0/1946953449 wait complete. 2023-10-06T02:23:20.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.919+0000 7fb82239d700 1 Processor -- start 2023-10-06T02:23:20.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.919+0000 7fb82239d700 1 -- start start 2023-10-06T02:23:20.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.920+0000 7fb82239d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb81c06aaa0 0x7fb81c155b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:20.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.920+0000 7fb82239d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb81c13fb20 0x7fb81c0c75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:20.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.920+0000 7fb82239d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb81c13fef0 0x7fb81c0c7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:20.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.920+0000 7fb82239d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb81c16cf50 con 0x7fb81c13fef0 2023-10-06T02:23:20.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.920+0000 7fb82239d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb81c16dad0 con 0x7fb81c13fb20 2023-10-06T02:23:20.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.920+0000 7fb82239d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb81c16d1a0 con 0x7fb81c06aaa0 2023-10-06T02:23:20.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.920+0000 7fb821b9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb81c13fef0 0x7fb81c0c7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:20.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.920+0000 7fb82139b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb81c06aaa0 0x7fb81c155b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:20.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.920+0000 7fb821b9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb81c13fef0 0x7fb81c0c7b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37456/0 (socket says 172.21.15.31:37456) 2023-10-06T02:23:20.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.920+0000 7fb821b9c700 1 -- 172.21.15.31:0/2008076262 learned_addr learned my addr 172.21.15.31:0/2008076262 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:20.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.920+0000 7fb820b9a700 1 --2- 172.21.15.31:0/2008076262 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb81c13fb20 0x7fb81c0c75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:20.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.921+0000 7fb821b9c700 1 -- 172.21.15.31:0/2008076262 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb81c06aaa0 msgr2=0x7fb81c155b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:20.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.921+0000 7fb821b9c700 1 --2- 172.21.15.31:0/2008076262 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb81c06aaa0 0x7fb81c155b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:20.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.921+0000 7fb821b9c700 1 -- 172.21.15.31:0/2008076262 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb81c13fb20 msgr2=0x7fb81c0c75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:20.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.921+0000 7fb821b9c700 1 --2- 172.21.15.31:0/2008076262 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb81c13fb20 0x7fb81c0c75f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:20.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.921+0000 7fb821b9c700 1 -- 172.21.15.31:0/2008076262 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb814009ae0 con 0x7fb81c13fef0 2023-10-06T02:23:20.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.921+0000 7fb82139b700 1 --2- 172.21.15.31:0/2008076262 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb81c06aaa0 0x7fb81c155b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:23:20.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.921+0000 7fb821b9c700 1 --2- 172.21.15.31:0/2008076262 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb81c13fef0 0x7fb81c0c7b30 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb814009fa0 tx=0x7fb81400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:20.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.921+0000 7fb8117fa700 1 -- 172.21.15.31:0/2008076262 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb814018070 con 0x7fb81c13fef0 2023-10-06T02:23:20.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.921+0000 7fb8117fa700 1 -- 172.21.15.31:0/2008076262 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb81400f040 con 0x7fb81c13fef0 2023-10-06T02:23:20.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.921+0000 7fb8117fa700 1 -- 172.21.15.31:0/2008076262 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8140092e0 con 0x7fb81c13fef0 2023-10-06T02:23:20.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.921+0000 7fb82239d700 1 -- 172.21.15.31:0/2008076262 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb818009420 con 0x7fb81c13fef0 2023-10-06T02:23:20.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.921+0000 7fb82239d700 1 -- 172.21.15.31:0/2008076262 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb81c16d760 con 0x7fb81c13fef0 2023-10-06T02:23:20.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.922+0000 7fb82239d700 1 -- 172.21.15.31:0/2008076262 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb81c16de60 con 0x7fb81c13fef0 2023-10-06T02:23:20.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.923+0000 7fb8117fa700 1 -- 172.21.15.31:0/2008076262 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb814003020 con 0x7fb81c13fef0 2023-10-06T02:23:20.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.924+0000 7fb8117fa700 1 --2- 172.21.15.31:0/2008076262 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fb800066f30 0x7fb8000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:20.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.924+0000 7fb8117fa700 1 -- 172.21.15.31:0/2008076262 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb814084b80 con 0x7fb81c13fef0 2023-10-06T02:23:20.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.924+0000 7fb82139b700 1 --2- 172.21.15.31:0/2008076262 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fb800066f30 0x7fb8000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:20.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.925+0000 7fb82139b700 1 --2- 172.21.15.31:0/2008076262 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fb800066f30 0x7fb8000693e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb8180091e0 tx=0x7fb818017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:20.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:20.929+0000 7fb8117fa700 1 -- 172.21.15.31:0/2008076262 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb814051bf0 con 0x7fb81c13fef0 2023-10-06T02:23:21.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.131+0000 7fb82239d700 1 -- 172.21.15.31:0/2008076262 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb81c16dc40 con 0x7fb81c13fef0 2023-10-06T02:23:21.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.132+0000 7fb8117fa700 1 -- 172.21.15.31:0/2008076262 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16818 (secure 0 0 0) 0x7fb814084e60 con 0x7fb81c13fef0 2023-10-06T02:23:21.132 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:23:21.133 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":23,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:23:10.970534+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:23:07.290269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T02:23:08.504679+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:23:21.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.134+0000 7fb82239d700 1 -- 172.21.15.31:0/2008076262 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fb800066f30 msgr2=0x7fb8000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:21.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.134+0000 7fb82239d700 1 --2- 172.21.15.31:0/2008076262 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fb800066f30 0x7fb8000693e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb8180091e0 tx=0x7fb818017000 comp rx=0 tx=0).stop 2023-10-06T02:23:21.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.134+0000 7fb82239d700 1 -- 172.21.15.31:0/2008076262 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb81c13fef0 msgr2=0x7fb81c0c7b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:21.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.134+0000 7fb82239d700 1 --2- 172.21.15.31:0/2008076262 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb81c13fef0 0x7fb81c0c7b30 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb814009fa0 tx=0x7fb81400b040 comp rx=0 tx=0).stop 2023-10-06T02:23:21.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.134+0000 7fb82239d700 1 -- 172.21.15.31:0/2008076262 shutdown_connections 2023-10-06T02:23:21.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.134+0000 7fb82239d700 1 --2- 172.21.15.31:0/2008076262 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb81c13fef0 0x7fb81c0c7b30 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:21.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.134+0000 7fb82239d700 1 --2- 172.21.15.31:0/2008076262 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fb800066f30 0x7fb8000693e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:21.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.134+0000 7fb82239d700 1 --2- 172.21.15.31:0/2008076262 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb81c06aaa0 0x7fb81c155b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:21.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.134+0000 7fb82239d700 1 --2- 172.21.15.31:0/2008076262 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb81c13fb20 0x7fb81c0c75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:21.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.134+0000 7fb82239d700 1 -- 172.21.15.31:0/2008076262 >> 172.21.15.31:0/2008076262 conn(0x7fb81c06d170 msgr2=0x7fb81c0c2490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:21.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.135+0000 7fb82239d700 1 -- 172.21.15.31:0/2008076262 shutdown_connections 2023-10-06T02:23:21.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.135+0000 7fb82239d700 1 -- 172.21.15.31:0/2008076262 wait complete. 2023-10-06T02:23:21.152 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-10-06T02:23:21.153 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-10-06T02:23:21.153 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-10-06T02:23:21.153 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-10-06T02:23:21.154 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-10-06T02:23:21.154 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-10-06T02:23:21.155 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-10-06T02:23:21.155 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-10-06T02:23:21.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.494+0000 7f7e2e79d700 1 Processor -- start 2023-10-06T02:23:21.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.494+0000 7f7e2e79d700 1 -- start start 2023-10-06T02:23:21.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.494+0000 7f65f8b9d700 1 Processor -- start 2023-10-06T02:23:21.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.495+0000 7f65f8b9d700 1 -- start start 2023-10-06T02:23:21.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.496+0000 7f7e2e79d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7e200dc600 0x7f7e20054290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:21.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.496+0000 7f7e2e79d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7e200e6570 0x7f7e20057810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:21.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.496+0000 7f7e2e79d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e20007090 con 0x7f7e200547d0 2023-10-06T02:23:21.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.496+0000 7f7e2e79d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e20007370 con 0x7f7e200e6570 2023-10-06T02:23:21.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.496+0000 7f7e2e79d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e20007650 con 0x7f7e200dc600 2023-10-06T02:23:21.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.497+0000 7f7e2df9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7e200e6570 0x7f7e20057810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:21.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.497+0000 7f7e2d79b700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7e200dc600 0x7f7e20054290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:21.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.497+0000 7f65f8b9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f65ec0dc0e0 0x7f65ec033a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:21.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.497+0000 7f7e2d79b700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7e200dc600 0x7f7e20054290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:47206/0 (socket says 172.21.15.31:47206) 2023-10-06T02:23:21.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.497+0000 7f7e2d79b700 1 -- 172.21.15.31:0/1026630092 learned_addr learned my addr 172.21.15.31:0/1026630092 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:21.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.495+0000 7ff0e03ff700 1 Processor -- start 2023-10-06T02:23:21.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.496+0000 7ff0e03ff700 1 -- start start 2023-10-06T02:23:21.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.498+0000 7f65f3fff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f65ec033fd0 0x7f65ec0de980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49246/0 (socket says 172.21.15.31:49246) 2023-10-06T02:23:21.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.498+0000 7f65f3fff700 1 -- 172.21.15.31:0/4097885263 learned_addr learned my addr 172.21.15.31:0/4097885263 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:23:21.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.498+0000 7f7e2d79b700 1 -- 172.21.15.31:0/1026630092 >> v2:172.21.15.31:3300/0 conn(0x7f7e200e6570 msgr2=0x7f7e20057810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:21.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.498+0000 7f7e2d79b700 1 --2- 172.21.15.31:0/1026630092 >> v2:172.21.15.31:3300/0 conn(0x7f7e200e6570 0x7f7e20057810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:21.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.498+0000 7f7e2d79b700 1 -- 172.21.15.31:0/1026630092 >> v1:172.21.15.31:6789/0 conn(0x7f7e200547d0 legacy=0x7f7e200e5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:21.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.498+0000 7f7e2d79b700 1 -- 172.21.15.31:0/1026630092 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e20007930 con 0x7f7e200dc600 2023-10-06T02:23:21.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.498+0000 7ff0e03ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff0d004ff80 0x7ff0d0050350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:21.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.498+0000 7ff0e03ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff0d0033ac0 0x7ff0d0033f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:21.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.498+0000 7ff0e03ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff0d00e96a0 0x7ff0d00e9b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:21.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.498+0000 7ff0e03ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0d0053e20 con 0x7ff0d004ff80 2023-10-06T02:23:21.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.498+0000 7ff0e03ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0d0054100 con 0x7ff0d00e96a0 2023-10-06T02:23:21.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.498+0000 7ff0e03ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d00543e0 con 0x7ff0d0033ac0 2023-10-06T02:23:21.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.498+0000 7f65f8b9d700 1 -- 172.21.15.31:0/4097885263 --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65ec053310 con 0x7f65ec033fd0 2023-10-06T02:23:21.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.499+0000 7f65f8b9d700 1 -- 172.21.15.31:0/4097885263 --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65ec053bb0 con 0x7f65ec0dbd10 2023-10-06T02:23:21.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.499+0000 7f65f8b9d700 1 -- 172.21.15.31:0/4097885263 --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65ec034430 con 0x7f65ec0dc0e0 2023-10-06T02:23:21.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.499+0000 7f7e2d79b700 1 --2- 172.21.15.31:0/1026630092 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e200dc600 0x7f7e20054290 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7e28065460 tx=0x7f7e280673b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3b63c3c9383141b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:21.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.499+0000 7ff0de99c700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff0d00e96a0 0x7ff0d00e9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:21.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.499+0000 7ff0de99c700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff0d00e96a0 0x7ff0d00e9b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:47214/0 (socket says 172.21.15.31:47214) 2023-10-06T02:23:21.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.499+0000 7ff0de99c700 1 -- 172.21.15.31:0/1380640533 learned_addr learned my addr 172.21.15.31:0/1380640533 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:21.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.499+0000 7ff0de19b700 1 --2- 172.21.15.31:0/1380640533 >> v2:172.21.15.31:3300/0 conn(0x7ff0d004ff80 0x7ff0d0050350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:21.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.500+0000 7ff0de19b700 1 -- 172.21.15.31:0/1380640533 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff0d0033ac0 msgr2=0x7ff0d0033f10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T02:23:21.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.500+0000 7ff0de19b700 1 --2- 172.21.15.31:0/1380640533 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff0d0033ac0 0x7ff0d0033f10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:21.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.500+0000 7ff0de19b700 1 -- 172.21.15.31:0/1380640533 >> v2:172.21.15.62:3300/0 conn(0x7ff0d00e96a0 msgr2=0x7ff0d00e9b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:21.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.500+0000 7ff0de19b700 1 --2- 172.21.15.31:0/1380640533 >> v2:172.21.15.62:3300/0 conn(0x7ff0d00e96a0 0x7ff0d00e9b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:21.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.500+0000 7ff0de19b700 1 -- 172.21.15.31:0/1380640533 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0d00546c0 con 0x7ff0d004ff80 2023-10-06T02:23:21.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.500+0000 7f65f27fc700 1 -- 172.21.15.31:0/4097885263 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3860331576 0 0) 0x7f65ec053bb0 con 0x7f65ec0dbd10 2023-10-06T02:23:21.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.500+0000 7f65f27fc700 1 -- 172.21.15.31:0/4097885263 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65e4003570 con 0x7f65ec0dbd10 2023-10-06T02:23:21.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.500+0000 7f65f2ffd700 1 --2- 172.21.15.31:0/4097885263 >> v2:172.21.15.62:3300/0 conn(0x7f65ec0dc0e0 0x7f65ec033a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:21.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.500+0000 7f65f27fc700 1 -- 172.21.15.31:0/4097885263 <== mon.1 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3277693491 0 0) 0x7f65e4003570 con 0x7f65ec0dbd10 2023-10-06T02:23:21.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.500+0000 7f65f27fc700 1 -- 172.21.15.31:0/4097885263 >> v2:172.21.15.62:3300/0 conn(0x7f65ec0dc0e0 msgr2=0x7f65ec033a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:21.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.500+0000 7f65f27fc700 1 --2- 172.21.15.31:0/4097885263 >> v2:172.21.15.62:3300/0 conn(0x7f65ec0dc0e0 0x7f65ec033a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:21.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.500+0000 7f65f27fc700 1 -- 172.21.15.31:0/4097885263 >> v1:172.21.15.31:6789/0 conn(0x7f65ec033fd0 legacy=0x7f65ec0de980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:21.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.500+0000 7f65f27fc700 1 -- 172.21.15.31:0/4097885263 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65ec0df6b0 con 0x7f65ec0dbd10 2023-10-06T02:23:21.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.501+0000 7f65f27fc700 1 -- 172.21.15.31:0/4097885263 <== mon.1 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7f65e8002420 con 0x7f65ec0dbd10 2023-10-06T02:23:21.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.500+0000 7f7e1ffff700 1 -- 172.21.15.31:0/1026630092 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e280008d0 con 0x7f7e200dc600 2023-10-06T02:23:21.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.500+0000 7f7e1ffff700 1 -- 172.21.15.31:0/1026630092 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e28064550 con 0x7f7e200dc600 2023-10-06T02:23:21.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.501+0000 7f65f27fc700 1 -- 172.21.15.31:0/4097885263 >> v1:172.21.15.62:6789/0 conn(0x7f65ec0dbd10 legacy=0x7f65ec005f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:21.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.501+0000 7f7e2e79d700 1 -- 172.21.15.31:0/1026630092 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e200dc600 msgr2=0x7f7e20054290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:21.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.501+0000 7f65f27fc700 1 --2- 172.21.15.31:0/4097885263 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f65e4003b60 0x7f65e4003f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:21.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.501+0000 7ff0de19b700 1 --2- 172.21.15.31:0/1380640533 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0d004ff80 0x7ff0d0050350 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff0d000c950 tx=0x7ff0c8009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea1bd47569c74cbd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:21.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.501+0000 7f7e2e79d700 1 --2- 172.21.15.31:0/1026630092 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e200dc600 0x7f7e20054290 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7e28065460 tx=0x7f7e280673b0 comp rx=0 tx=0).stop 2023-10-06T02:23:21.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.501+0000 7f65f27fc700 1 --2- 172.21.15.31:0/4097885263 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65e4004810 0x7f65e4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:21.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.501+0000 7ff0dd199700 1 -- 172.21.15.31:0/1380640533 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c8005c10 con 0x7ff0d004ff80 2023-10-06T02:23:21.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.502+0000 7f65f27fc700 1 --2- 172.21.15.31:0/4097885263 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65e4005310 0x7f65e400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:21.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.502+0000 7f65f37fe700 1 --2- 172.21.15.31:0/4097885263 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f65e4003b60 0x7f65e4003f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:21.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.502+0000 7f65f3fff700 1 --2- 172.21.15.31:0/4097885263 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65e4005310 0x7f65e400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:21.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.502+0000 7f65f27fc700 1 -- 172.21.15.31:0/4097885263 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65ec034430 con 0x7f65e4005310 2023-10-06T02:23:21.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.503+0000 7f65f27fc700 1 -- 172.21.15.31:0/4097885263 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65e400c240 con 0x7f65e4004810 2023-10-06T02:23:21.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.502+0000 7ff0dd199700 1 -- 172.21.15.31:0/1380640533 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0c8005d70 con 0x7ff0d004ff80 2023-10-06T02:23:21.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.503+0000 7f65f27fc700 1 -- 172.21.15.31:0/4097885263 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65e400c400 con 0x7f65e4003b60 2023-10-06T02:23:21.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.503+0000 7ff0e03ff700 1 -- 172.21.15.31:0/1380640533 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0d004ff80 msgr2=0x7ff0d0050350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:21.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.503+0000 7ff0dd199700 1 -- 172.21.15.31:0/1380640533 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c8005c10 con 0x7ff0d004ff80 2023-10-06T02:23:21.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.503+0000 7f65f2ffd700 1 --2- 172.21.15.31:0/4097885263 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65e4004810 0x7f65e4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:21.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.503+0000 7f7e2e79d700 1 -- 172.21.15.31:0/1026630092 shutdown_connections 2023-10-06T02:23:21.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.503+0000 7f7e2e79d700 1 --2- 172.21.15.31:0/1026630092 >> v2:172.21.15.31:3300/0 conn(0x7f7e200e6570 0x7f7e20057810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:21.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.503+0000 7f7e2e79d700 1 --2- 172.21.15.31:0/1026630092 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e200dc600 0x7f7e20054290 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:21.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.503+0000 7f7e2e79d700 1 -- 172.21.15.31:0/1026630092 >> 172.21.15.31:0/1026630092 conn(0x7f7e20055ba0 msgr2=0x7f7e20055fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:21.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.503+0000 7ff0e03ff700 1 --2- 172.21.15.31:0/1380640533 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0d004ff80 0x7ff0d0050350 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff0d000c950 tx=0x7ff0c8009120 comp rx=0 tx=0).stop 2023-10-06T02:23:21.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.503+0000 7ff0e03ff700 1 -- 172.21.15.31:0/1380640533 shutdown_connections 2023-10-06T02:23:21.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.503+0000 7f7e2e79d700 1 -- 172.21.15.31:0/1026630092 shutdown_connections 2023-10-06T02:23:21.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.503+0000 7ff0e03ff700 1 --2- 172.21.15.31:0/1380640533 >> v2:172.21.15.62:3300/0 conn(0x7ff0d00e96a0 0x7ff0d00e9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:21.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.503+0000 7ff0e03ff700 1 --2- 172.21.15.31:0/1380640533 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0d004ff80 0x7ff0d0050350 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:21.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.503+0000 7ff0e03ff700 1 --2- 172.21.15.31:0/1380640533 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff0d0033ac0 0x7ff0d0033f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:21.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.504+0000 7f65f3fff700 1 -- 172.21.15.31:0/4097885263 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f65e4003b60 msgr2=0x7f65e4003f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:21.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.503+0000 7ff0e03ff700 1 -- 172.21.15.31:0/1380640533 >> 172.21.15.31:0/1380640533 conn(0x7ff0d00517e0 msgr2=0x7ff0d0051be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:21.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.504+0000 7ff0e03ff700 1 -- 172.21.15.31:0/1380640533 shutdown_connections 2023-10-06T02:23:21.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.504+0000 7f65f3fff700 1 --2- 172.21.15.31:0/4097885263 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f65e4003b60 0x7f65e4003f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:21.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.504+0000 7f65f3fff700 1 -- 172.21.15.31:0/4097885263 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65e4004810 msgr2=0x7f65e4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:21.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.504+0000 7f6168bff700 1 Processor -- start 2023-10-06T02:23:21.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.504+0000 7f6168bff700 1 -- start start 2023-10-06T02:23:21.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.505+0000 7f65f3fff700 1 --2- 172.21.15.31:0/4097885263 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65e4004810 0x7f65e4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:21.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.505+0000 7f65f3fff700 1 -- 172.21.15.31:0/4097885263 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65e4003570 con 0x7f65e4005310 2023-10-06T02:23:21.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.504+0000 7ff0e03ff700 1 -- 172.21.15.31:0/1380640533 wait complete. 2023-10-06T02:23:21.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.505+0000 7f65f2ffd700 1 --2- 172.21.15.31:0/4097885263 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65e4004810 0x7f65e4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:23:21.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.505+0000 7f65f3fff700 1 --2- 172.21.15.31:0/4097885263 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65e4005310 0x7f65e400b9d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f65f4000eb0 tx=0x7f65f40673b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfb2d1eeab217209 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:21.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.506+0000 7f65f3fff700 1 -- 172.21.15.31:0/4097885263 reap_dead start 2023-10-06T02:23:21.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.506+0000 7fd9d159d700 1 Processor -- start 2023-10-06T02:23:21.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.504+0000 7f7e2e79d700 1 -- 172.21.15.31:0/1026630092 wait complete. 2023-10-06T02:23:21.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.506+0000 7f7e2e79d700 1 Processor -- start 2023-10-06T02:23:21.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.506+0000 7f65f27fc700 1 -- 172.21.15.31:0/4097885263 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f40009a0 con 0x7f65e4005310 2023-10-06T02:23:21.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.506+0000 7ff0e03ff700 1 Processor -- start 2023-10-06T02:23:21.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.507+0000 7f65f27fc700 1 -- 172.21.15.31:0/4097885263 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65f408b070 con 0x7f65e4005310 2023-10-06T02:23:21.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.507+0000 7ff0e03ff700 1 -- start start 2023-10-06T02:23:21.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.506+0000 7f6168bff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f61640cb5a0 0x7f61640cb970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:21.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.508+0000 7f65f8b9d700 1 -- 172.21.15.31:0/4097885263 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65e4005310 msgr2=0x7f65e400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:21.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.506+0000 7f7e2e79d700 1 -- start start 2023-10-06T02:23:21.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.508+0000 7f65f8b9d700 1 --2- 172.21.15.31:0/4097885263 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65e4005310 0x7f65e400b9d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f65f4000eb0 tx=0x7f65f40673b0 comp rx=0 tx=0).stop 2023-10-06T02:23:21.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.508+0000 7f7e2e79d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e200547d0 0x7f7e200d8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:21.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.508+0000 7f7e2e79d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e200dc600 0x7f7e2000fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:21.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.508+0000 7f65f8b9d700 1 -- 172.21.15.31:0/4097885263 shutdown_connections 2023-10-06T02:23:21.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.508+0000 7f65f8b9d700 1 --2- 172.21.15.31:0/4097885263 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65e4005310 0x7f65e400b9d0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:21.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.506+0000 7fd9d159d700 1 -- start start 2023-10-06T02:23:21.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.508+0000 7f65f8b9d700 1 -- 172.21.15.31:0/4097885263 >> 172.21.15.31:0/4097885263 conn(0x7f65ec0517b0 msgr2=0x7f65ec051bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:21.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.508+0000 7fd9d159d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd9c4006b50 0x7fd9c40e5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:21.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.508+0000 7fd9d159d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9c40549a0 con 0x7fd9c40dc9d0 2023-10-06T02:23:21.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.508+0000 7fd9d159d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9c4007090 con 0x7fd9c40dc600 2023-10-06T02:23:21.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.508+0000 7fd9d159d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c4007370 con 0x7fd9c4006b50 2023-10-06T02:23:21.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.508+0000 7f7e2d79b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e200547d0 0x7f7e200d8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:21.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.509+0000 7f65f8b9d700 1 -- 172.21.15.31:0/4097885263 shutdown_connections 2023-10-06T02:23:21.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.506+0000 7f6168bff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f61640cd9e0 0x7f6164159ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:21.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.506+0000 7f6168bff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f616415ae40 con 0x7f616415a400 2023-10-06T02:23:21.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.506+0000 7f6168bff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61640d13f0 con 0x7f61640cd9e0 2023-10-06T02:23:21.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.506+0000 7f6168bff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f616415ab60 con 0x7f61640cb5a0 2023-10-06T02:23:21.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.507+0000 7f6161d9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f61640cd9e0 0x7f6164159ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:21.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.507+0000 7f6161d9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f61640cd9e0 0x7f6164159ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37494/0 (socket says 172.21.15.31:37494) 2023-10-06T02:23:21.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.507+0000 7f6161d9b700 1 -- 172.21.15.31:0/1507641052 learned_addr learned my addr 172.21.15.31:0/1507641052 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:21.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.509+0000 7fd9cb7fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd9c4006b50 0x7fd9c40e5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:21.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.509+0000 7fd9cb7fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd9c4006b50 0x7fd9c40e5e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:56996/0 (socket says 172.21.15.31:56996) 2023-10-06T02:23:21.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.509+0000 7fd9cb7fe700 1 -- 172.21.15.31:0/3182716281 learned_addr learned my addr 172.21.15.31:0/3182716281 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:21.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.508+0000 7f6161d9b700 1 -- 172.21.15.31:0/1507641052 >> v2:172.21.15.62:3300/0 conn(0x7f61640cb5a0 msgr2=0x7f61640cb970 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T02:23:21.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.508+0000 7f6161d9b700 1 --2- 172.21.15.31:0/1507641052 >> v2:172.21.15.62:3300/0 conn(0x7f61640cb5a0 0x7f61640cb970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:21.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.508+0000 7f6161d9b700 1 -- 172.21.15.31:0/1507641052 >> v1:172.21.15.62:6789/0 conn(0x7f616415a400 legacy=0x7f6164156e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:21.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.508+0000 7f6161d9b700 1 -- 172.21.15.31:0/1507641052 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61640ccf00 con 0x7f61640cd9e0 2023-10-06T02:23:21.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.509+0000 7f6161d9b700 1 --2- 172.21.15.31:0/1507641052 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61640cd9e0 0x7f6164159ec0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f615c009f20 tx=0x7f615c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a49ffde6752f0e2f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:21.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.509+0000 7f65f8b9d700 1 -- 172.21.15.31:0/4097885263 wait complete. 2023-10-06T02:23:21.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.509+0000 7ff0e03ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff0d0033ac0 0x7ff0d00d9f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:21.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.509+0000 7ff0e03ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0d00e96a0 0x7ff0d0037b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:21.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.510+0000 7ff0e03ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0d00396b0 0x7ff0d00064f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:21.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.509+0000 7fd9cb7fe700 1 -- 172.21.15.31:0/3182716281 >> v1:172.21.15.62:6789/0 conn(0x7fd9c40dc600 legacy=0x7fd9c4054290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:21.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.509+0000 7fd9cb7fe700 1 -- 172.21.15.31:0/3182716281 >> v1:172.21.15.31:6789/0 conn(0x7fd9c40dc9d0 legacy=0x7fd9c4057810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:21.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.509+0000 7fd9cb7fe700 1 -- 172.21.15.31:0/3182716281 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9c4007650 con 0x7fd9c4006b50 2023-10-06T02:23:21.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.509+0000 7f616159a700 1 -- 172.21.15.31:0/1507641052 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f615c00e070 con 0x7f61640cd9e0 2023-10-06T02:23:21.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.510+0000 7f616159a700 1 -- 172.21.15.31:0/1507641052 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f615c00ba00 con 0x7f61640cd9e0 2023-10-06T02:23:21.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.510+0000 7f616159a700 1 -- 172.21.15.31:0/1507641052 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f615c0092e0 con 0x7f61640cd9e0 2023-10-06T02:23:21.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.510+0000 7fd9cb7fe700 1 --2- 172.21.15.31:0/3182716281 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd9c4006b50 0x7fd9c40e5e60 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc065460 tx=0x7fd9cc0673b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca5af26a0859b0b2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:21.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.510+0000 7ff0e03ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d0038610 con 0x7ff0d00e96a0 2023-10-06T02:23:21.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.510+0000 7ff0e03ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d00388f0 con 0x7ff0d00396b0 2023-10-06T02:23:21.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.510+0000 7f7e2e79d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7e200e6570 0x7f7e20010190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:21.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.510+0000 7f7e2e79d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e20051b00 con 0x7f7e200547d0 2023-10-06T02:23:21.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.510+0000 7f7e2d79b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e200547d0 0x7f7e200d8cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37496/0 (socket says 172.21.15.31:37496) 2023-10-06T02:23:21.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.510+0000 7ff0e03ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d0038bd0 con 0x7ff0d0033ac0 2023-10-06T02:23:21.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.510+0000 7ff0dd99a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0d00e96a0 0x7ff0d0037b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:21.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.510+0000 7ff0de19b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff0d0033ac0 0x7ff0d00d9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:21.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.510+0000 7f7e2d79b700 1 -- 172.21.15.31:0/4143711347 learned_addr learned my addr 172.21.15.31:0/4143711347 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:21.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.510+0000 7f7e2e79d700 1 -- 172.21.15.31:0/4143711347 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e200d9200 con 0x7f7e200dc600 2023-10-06T02:23:21.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.510+0000 7f7e2e79d700 1 -- 172.21.15.31:0/4143711347 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e20050ef0 con 0x7f7e200e6570 2023-10-06T02:23:21.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.510+0000 7ff0de19b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff0d0033ac0 0x7ff0d00d9f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:57008/0 (socket says 172.21.15.31:57008) 2023-10-06T02:23:21.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.510+0000 7ff0de19b700 1 -- 172.21.15.31:0/2040363347 learned_addr learned my addr 172.21.15.31:0/2040363347 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:21.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.510+0000 7ff0de99c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0d00396b0 0x7ff0d00064f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:21.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.511+0000 7f7e2df9c700 1 --2- 172.21.15.31:0/4143711347 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7e200e6570 0x7f7e20010190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:21.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.511+0000 7f7e2d79b700 1 -- 172.21.15.31:0/4143711347 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7e200e6570 msgr2=0x7f7e20010190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:21.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.511+0000 7f7e2d79b700 1 --2- 172.21.15.31:0/4143711347 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7e200e6570 0x7f7e20010190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:21.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.511+0000 7f7e2d79b700 1 -- 172.21.15.31:0/4143711347 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e200dc600 msgr2=0x7f7e2000fc50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T02:23:21.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.511+0000 7f7e2d79b700 1 --2- 172.21.15.31:0/4143711347 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e200dc600 0x7f7e2000fc50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:21.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.511+0000 7f7e2d79b700 1 -- 172.21.15.31:0/4143711347 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e200511d0 con 0x7f7e200547d0 2023-10-06T02:23:21.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.511+0000 7fd9caffd700 1 -- 172.21.15.31:0/3182716281 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9cc0008d0 con 0x7fd9c4006b50 2023-10-06T02:23:21.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.511+0000 7fd9caffd700 1 -- 172.21.15.31:0/3182716281 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9cc064550 con 0x7fd9c4006b50 2023-10-06T02:23:21.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.512+0000 7fd9caffd700 1 -- 172.21.15.31:0/3182716281 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9cc0646e0 con 0x7fd9c4006b50 2023-10-06T02:23:21.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.511+0000 7f6168bff700 1 -- 172.21.15.31:0/1507641052 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61640cd9e0 msgr2=0x7f6164159ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:21.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.511+0000 7f6168bff700 1 --2- 172.21.15.31:0/1507641052 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61640cd9e0 0x7f6164159ec0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f615c009f20 tx=0x7f615c009840 comp rx=0 tx=0).stop 2023-10-06T02:23:21.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.512+0000 7fd9d159d700 1 -- 172.21.15.31:0/3182716281 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd9c4006b50 msgr2=0x7fd9c40e5e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:21.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.512+0000 7fd9d159d700 1 --2- 172.21.15.31:0/3182716281 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd9c4006b50 0x7fd9c40e5e60 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc065460 tx=0x7fd9cc0673b0 comp rx=0 tx=0).stop 2023-10-06T02:23:21.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.511+0000 7ff0dd99a700 1 -- 172.21.15.31:0/2040363347 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff0d0033ac0 msgr2=0x7ff0d00d9f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:21.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.511+0000 7ff0dd99a700 1 --2- 172.21.15.31:0/2040363347 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff0d0033ac0 0x7ff0d00d9f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:21.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.511+0000 7ff0dd99a700 1 -- 172.21.15.31:0/2040363347 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0d00396b0 msgr2=0x7ff0d00064f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:21.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.511+0000 7ff0dd99a700 1 --2- 172.21.15.31:0/2040363347 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0d00396b0 0x7ff0d00064f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:21.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.512+0000 7f7e2d79b700 1 --2- 172.21.15.31:0/4143711347 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e200547d0 0x7f7e200d8cc0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7e28066da0 tx=0x7f7e28068000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:21.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.511+0000 7ff0dd99a700 1 -- 172.21.15.31:0/2040363347 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0c80093c0 con 0x7ff0d00e96a0 2023-10-06T02:23:21.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.511+0000 7ff0dd99a700 1 --2- 172.21.15.31:0/2040363347 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0d00e96a0 0x7ff0d0037b10 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff0d400b5b0 tx=0x7ff0d40090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:21.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.512+0000 7fd9d159d700 1 -- 172.21.15.31:0/3182716281 shutdown_connections 2023-10-06T02:23:21.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.512+0000 7fd9d159d700 1 --2- 172.21.15.31:0/3182716281 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd9c4006b50 0x7fd9c40e5e60 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:21.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.512+0000 7fd9d159d700 1 -- 172.21.15.31:0/3182716281 >> 172.21.15.31:0/3182716281 conn(0x7fd9c4055ba0 msgr2=0x7fd9c4055fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:21.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.512+0000 7fd9d159d700 1 -- 172.21.15.31:0/3182716281 shutdown_connections 2023-10-06T02:23:21.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.512+0000 7fd9d159d700 1 -- 172.21.15.31:0/3182716281 wait complete. 2023-10-06T02:23:21.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.512+0000 7f7e1d7fa700 1 -- 172.21.15.31:0/4143711347 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e280675a0 con 0x7f7e200547d0 2023-10-06T02:23:21.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.512+0000 7ff0de19b700 1 --2- 172.21.15.31:0/2040363347 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff0d0033ac0 0x7ff0d00d9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:23:21.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.512+0000 7ff0de99c700 1 --2- 172.21.15.31:0/2040363347 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0d00396b0 0x7ff0d00064f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:23:21.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.513+0000 7ff0ce7fc700 1 -- 172.21.15.31:0/2040363347 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d400f510 con 0x7ff0d00e96a0 2023-10-06T02:23:21.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.513+0000 7f65f8b9d700 1 Processor -- start 2023-10-06T02:23:21.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.513+0000 7f7e2e79d700 1 -- 172.21.15.31:0/4143711347 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e200514b0 con 0x7f7e200547d0 2023-10-06T02:23:21.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.513+0000 7f7e1d7fa700 1 -- 172.21.15.31:0/4143711347 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e28067700 con 0x7f7e200547d0 2023-10-06T02:23:21.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.513+0000 7f65f8b9d700 1 -- start start 2023-10-06T02:23:21.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.513+0000 7f7e2e79d700 1 -- 172.21.15.31:0/4143711347 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e200f65f0 con 0x7f7e200547d0 2023-10-06T02:23:21.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.513+0000 7f7e1d7fa700 1 -- 172.21.15.31:0/4143711347 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e28067a80 con 0x7f7e200547d0 2023-10-06T02:23:21.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.513+0000 7f6af9dff700 1 Processor -- start 2023-10-06T02:23:21.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.513+0000 7f6af9dff700 1 -- start start 2023-10-06T02:23:21.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.515+0000 7fd9d159d700 1 Processor -- start 2023-10-06T02:23:21.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.515+0000 7f7e2e79d700 1 -- 172.21.15.31:0/4143711347 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7e08000fd0 con 0x7f7e200547d0 2023-10-06T02:23:21.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.515+0000 7fd9d159d700 1 -- start start 2023-10-06T02:23:21.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.515+0000 7f6168bff700 1 -- 172.21.15.31:0/1507641052 shutdown_connections 2023-10-06T02:23:21.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.515+0000 7f6168bff700 1 --2- 172.21.15.31:0/1507641052 >> v2:172.21.15.62:3300/0 conn(0x7f61640cb5a0 0x7f61640cb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:21.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.515+0000 7f6168bff700 1 --2- 172.21.15.31:0/1507641052 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61640cd9e0 0x7f6164159ec0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:21.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.515+0000 7f6168bff700 1 -- 172.21.15.31:0/1507641052 >> 172.21.15.31:0/1507641052 conn(0x7f6164070e30 msgr2=0x7f61640d0710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:21.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.515+0000 7f65f8b9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65ec0d4740 0x7f65ec0dd060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:21.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.515+0000 7f65f8b9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f65ec0dd5a0 0x7f65ec038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:21.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.515+0000 7f65f37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65ec0d4740 0x7f65ec0dd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:21.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.516+0000 7ff0ce7fc700 1 -- 172.21.15.31:0/2040363347 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0d400f670 con 0x7ff0d00e96a0 2023-10-06T02:23:21.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.516+0000 7f65f37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65ec0d4740 0x7f65ec0dd060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37512/0 (socket says 172.21.15.31:37512) 2023-10-06T02:23:21.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.516+0000 7f6af9dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6af40b1e00 0x7f6af41555e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:21.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.516+0000 7ff0e03ff700 1 -- 172.21.15.31:0/2040363347 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0d0038eb0 con 0x7ff0d00e96a0 2023-10-06T02:23:21.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.516+0000 7f65f37fe700 1 -- 172.21.15.31:0/2958606099 learned_addr learned my addr 172.21.15.31:0/2958606099 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:21.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.516+0000 7f65f8b9d700 1 --2- 172.21.15.31:0/2958606099 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65ec0d4b10 0x7f65ec039180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:21.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.516+0000 7f65f8b9d700 1 -- 172.21.15.31:0/2958606099 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65ec0396c0 con 0x7f65ec0d4740 2023-10-06T02:23:21.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.516+0000 7f6168bff700 1 -- 172.21.15.31:0/1507641052 shutdown_connections 2023-10-06T02:23:21.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.516+0000 7f65f2ffd700 1 --2- 172.21.15.31:0/2958606099 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f65ec0dd5a0 0x7f65ec038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:21.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.517+0000 7f65f3fff700 1 --2- 172.21.15.31:0/2958606099 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65ec0d4b10 0x7f65ec039180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:21.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.517+0000 7f65f8b9d700 1 -- 172.21.15.31:0/2958606099 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65ec049b70 con 0x7f65ec0d4b10 2023-10-06T02:23:21.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.517+0000 7fd9d159d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9c4006b50 0x7fd9c40da1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:21.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.517+0000 7f65f8b9d700 1 -- 172.21.15.31:0/2958606099 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65ec0dda20 con 0x7f65ec0dd5a0 2023-10-06T02:23:21.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.517+0000 7fd9d159d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd9c40dc600 0x7fd9c403dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:21.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.517+0000 7f6af37fe700 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f6af4067fb0 0x7f6af4156950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:33920/0 (socket says 172.21.15.31:33920) 2023-10-06T02:23:21.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.517+0000 7f6af37fe700 1 -- 172.21.15.31:0/1087454585 learned_addr learned my addr 172.21.15.31:0/1087454585 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:23:21.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.517+0000 7fd9cbfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9c4006b50 0x7fd9c40da1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:21.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.517+0000 7fd9cbfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9c4006b50 0x7fd9c40da1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:47254/0 (socket says 172.21.15.31:47254) 2023-10-06T02:23:21.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.517+0000 7f7e1d7fa700 1 -- 172.21.15.31:0/4143711347 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7e2808d020 con 0x7f7e200547d0 2023-10-06T02:23:21.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.518+0000 7fd9cbfff700 1 -- 172.21.15.31:0/3492459277 learned_addr learned my addr 172.21.15.31:0/3492459277 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:21.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.518+0000 7f65f3fff700 1 -- 172.21.15.31:0/2958606099 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f65ec0dd5a0 msgr2=0x7f65ec038c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:21.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.518+0000 7f65f3fff700 1 --2- 172.21.15.31:0/2958606099 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f65ec0dd5a0 0x7f65ec038c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:21.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.518+0000 7f65f3fff700 1 -- 172.21.15.31:0/2958606099 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65ec0d4740 msgr2=0x7f65ec0dd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:21.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.518+0000 7fd9d159d700 1 --2- 172.21.15.31:0/3492459277 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9c40dc9d0 0x7fd9c40f3360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:21.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.518+0000 7f65f3fff700 1 --2- 172.21.15.31:0/2958606099 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65ec0d4740 0x7f65ec0dd060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:21.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.518+0000 7f65f3fff700 1 -- 172.21.15.31:0/2958606099 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65ec0ddd00 con 0x7f65ec0d4b10 2023-10-06T02:23:21.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.518+0000 7f65f37fe700 1 --2- 172.21.15.31:0/2958606099 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65ec0d4740 0x7f65ec0dd060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:23:21.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.518+0000 7fd9cb7fe700 1 --2- 172.21.15.31:0/3492459277 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd9c40dc600 0x7fd9c403dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:21.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.518+0000 7ff0ce7fc700 1 -- 172.21.15.31:0/2040363347 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d400fa30 con 0x7ff0d00e96a0 2023-10-06T02:23:21.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.518+0000 7f65f3fff700 1 --2- 172.21.15.31:0/2958606099 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65ec0d4b10 0x7f65ec039180 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f65f40669b0 tx=0x7f65f406d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:21.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.518+0000 7fd9d0d9c700 1 --2- 172.21.15.31:0/3492459277 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9c40dc9d0 0x7fd9c40f3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:21.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.519+0000 7fd9d159d700 1 -- 172.21.15.31:0/3492459277 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c403cb80 con 0x7fd9c40dc9d0 2023-10-06T02:23:21.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.516+0000 7f6168bff700 1 -- 172.21.15.31:0/1507641052 wait complete. 2023-10-06T02:23:21.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.519+0000 7fd9d159d700 1 -- 172.21.15.31:0/3492459277 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c40f38a0 con 0x7fd9c4006b50 2023-10-06T02:23:21.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.519+0000 7fd9d159d700 1 -- 172.21.15.31:0/3492459277 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c403bf70 con 0x7fd9c40dc600 2023-10-06T02:23:21.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.519+0000 7f6168bff700 1 Processor -- start 2023-10-06T02:23:21.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.519+0000 7f6af2ffd700 1 --2- 172.21.15.31:0/1087454585 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6af40b1e00 0x7f6af41555e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:21.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.519+0000 7f9205dff700 1 Processor -- start 2023-10-06T02:23:21.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.519+0000 7f9205dff700 1 -- start start 2023-10-06T02:23:21.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.519+0000 7ff0e03ff700 1 -- 172.21.15.31:0/2040363347 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0d004d8e0 con 0x7ff0d00e96a0 2023-10-06T02:23:21.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.519+0000 7ff0e03ff700 1 -- 172.21.15.31:0/2040363347 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff0d0005420 con 0x7ff0d00e96a0 2023-10-06T02:23:21.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.520+0000 7fd9cbfff700 1 -- 172.21.15.31:0/3492459277 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd9c40dc600 msgr2=0x7fd9c403dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:21.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.520+0000 7fd9cbfff700 1 --2- 172.21.15.31:0/3492459277 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd9c40dc600 0x7fd9c403dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:21.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.520+0000 7fd9cbfff700 1 -- 172.21.15.31:0/3492459277 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9c40dc9d0 msgr2=0x7fd9c40f3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:21.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.520+0000 7fe9ca5ff700 1 Processor -- start 2023-10-06T02:23:21.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.520+0000 7fd9cbfff700 1 --2- 172.21.15.31:0/3492459277 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9c40dc9d0 0x7fd9c40f3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:21.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.520+0000 7f7e1d7fa700 1 --2- 172.21.15.31:0/4143711347 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f7e0c066f30 0x7f7e0c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:21.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.520+0000 7fd9cbfff700 1 -- 172.21.15.31:0/3492459277 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9c403c250 con 0x7fd9c4006b50 2023-10-06T02:23:21.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.520+0000 7f7e1d7fa700 1 -- 172.21.15.31:0/4143711347 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f7e280fb7d0 con 0x7f7e200547d0 2023-10-06T02:23:21.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.520+0000 7f7e2cf9a700 1 --2- 172.21.15.31:0/4143711347 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f7e0c066f30 0x7f7e0c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:21.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.520+0000 7f6af9dff700 1 -- 172.21.15.31:0/1087454585 --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6af4157fe0 con 0x7f6af4155b20 2023-10-06T02:23:21.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.520+0000 7f6af9dff700 1 -- 172.21.15.31:0/1087454585 --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6af4157060 con 0x7f6af4067fb0 2023-10-06T02:23:21.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.520+0000 7f6af9dff700 1 -- 172.21.15.31:0/1087454585 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6af41576b0 con 0x7f6af40b1e00 2023-10-06T02:23:21.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.519+0000 7f6168bff700 1 -- start start 2023-10-06T02:23:21.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.520+0000 7f6168bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f61640cb5a0 0x7f616414d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:21.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.520+0000 7f6168bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f61640cd9e0 0x7f6164158ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:21.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.520+0000 7fe9ca5ff700 1 -- start start 2023-10-06T02:23:21.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.520+0000 7fd9d0d9c700 1 --2- 172.21.15.31:0/3492459277 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9c40dc9d0 0x7fd9c40f3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:23:21.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.520+0000 7fd9cbfff700 1 --2- 172.21.15.31:0/3492459277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9c4006b50 0x7fd9c40da1a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd9b800b5e0 tx=0x7fd9b8009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:21.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.520+0000 7f7e1d7fa700 1 --2- 172.21.15.31:0/4143711347 >> [v2:172.21.15.31:6822/85579,v1:172.21.15.31:6825/85579] conn(0x7f7e0c06f7e0 0x7f7e0c071bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:21.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.521+0000 7f6168bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f616415a400 0x7f6164159010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:21.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.521+0000 7f7e1d7fa700 1 -- 172.21.15.31:0/4143711347 --> [v2:172.21.15.31:6822/85579,v1:172.21.15.31:6825/85579] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7e0c0722a0 con 0x7f7e0c06f7e0 2023-10-06T02:23:21.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.521+0000 7f7e1d7fa700 1 -- 172.21.15.31:0/4143711347 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e280fba10 con 0x7f7e200547d0 2023-10-06T02:23:21.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.521+0000 7fd9b7fff700 1 -- 172.21.15.31:0/3492459277 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b800f510 con 0x7fd9c4006b50 2023-10-06T02:23:21.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.521+0000 7fd9b7fff700 1 -- 172.21.15.31:0/3492459277 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9b800f670 con 0x7fd9c4006b50 2023-10-06T02:23:21.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.521+0000 7fd9b7fff700 1 -- 172.21.15.31:0/3492459277 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b800fa10 con 0x7fd9c4006b50 2023-10-06T02:23:21.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.521+0000 7f65df7fe700 1 -- 172.21.15.31:0/2958606099 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f4096070 con 0x7f65ec0d4b10 2023-10-06T02:23:21.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.521+0000 7f65df7fe700 1 -- 172.21.15.31:0/2958606099 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65f408b070 con 0x7f65ec0d4b10 2023-10-06T02:23:21.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.521+0000 7f65df7fe700 1 -- 172.21.15.31:0/2958606099 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f406cb40 con 0x7f65ec0d4b10 2023-10-06T02:23:21.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.522+0000 7f65f8b9d700 1 -- 172.21.15.31:0/2958606099 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65ec0ddfe0 con 0x7f65ec0d4b10 2023-10-06T02:23:21.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.522+0000 7f65f8b9d700 1 -- 172.21.15.31:0/2958606099 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65ec037c30 con 0x7f65ec0d4b10 2023-10-06T02:23:21.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.522+0000 7f7e2df9c700 1 --2- 172.21.15.31:0/4143711347 >> [v2:172.21.15.31:6822/85579,v1:172.21.15.31:6825/85579] conn(0x7f7e0c06f7e0 0x7f7e0c071bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:21.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.521+0000 7f6168bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61640b5500 con 0x7f616415a400 2023-10-06T02:23:21.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.521+0000 7f6168bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f616414ce20 con 0x7f61640cb5a0 2023-10-06T02:23:21.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.521+0000 7f6168bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61640b3ed0 con 0x7f61640cd9e0 2023-10-06T02:23:21.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.522+0000 7f616259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f61640cb5a0 0x7f616414d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:21.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.522+0000 7f616259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f61640cb5a0 0x7f616414d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:47264/0 (socket says 172.21.15.31:47264) 2023-10-06T02:23:21.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.522+0000 7f616259c700 1 -- 172.21.15.31:0/729517814 learned_addr learned my addr 172.21.15.31:0/729517814 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:21.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.521+0000 7f6af27fc700 1 -- 172.21.15.31:0/1087454585 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 163966823 0 0) 0x7f6af4157060 con 0x7f6af4067fb0 2023-10-06T02:23:21.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.521+0000 7f6af27fc700 1 -- 172.21.15.31:0/1087454585 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6adc003570 con 0x7f6af4067fb0 2023-10-06T02:23:21.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.521+0000 7f6af2ffd700 1 -- 172.21.15.31:0/1087454585 >> v1:172.21.15.62:6789/0 conn(0x7f6af4067fb0 legacy=0x7f6af4156950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:21.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.521+0000 7f6af2ffd700 1 -- 172.21.15.31:0/1087454585 >> v1:172.21.15.31:6789/0 conn(0x7f6af4155b20 legacy=0x7f6af40afc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:21.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.522+0000 7f6af2ffd700 1 -- 172.21.15.31:0/1087454585 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6af4157990 con 0x7f6af40b1e00 2023-10-06T02:23:21.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.522+0000 7fe9ca5ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe9c40cb8e0 0x7fe9c40cbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:21.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.522+0000 7fe9c3fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe9c40cb8e0 0x7fe9c40cbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:21.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.522+0000 7fe9c3fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe9c40cb8e0 0x7fe9c40cbcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:57064/0 (socket says 172.21.15.31:57064) 2023-10-06T02:23:21.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.522+0000 7fe9ca5ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe9c406eb00 0x7fe9c40ad6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:21.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.522+0000 7fe9c3fff700 1 -- 172.21.15.31:0/2045576265 learned_addr learned my addr 172.21.15.31:0/2045576265 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:21.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.522+0000 7f9205dff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f92000cb5a0 0x7f92000cb970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:21.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.522+0000 7f9205dff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f920015a5d0 0x7f9200156e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:21.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.522+0000 7f9205dff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f920015ae40 con 0x7f92000cd9e0 2023-10-06T02:23:21.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.522+0000 7f9205dff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92000d13f0 con 0x7f920015a5d0 2023-10-06T02:23:21.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.522+0000 7f9205dff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f920015ab60 con 0x7f92000cb5a0 2023-10-06T02:23:21.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.522+0000 7ff0ce7fc700 1 -- 172.21.15.31:0/2040363347 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff0d401a3b0 con 0x7ff0d00e96a0 2023-10-06T02:23:21.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.523+0000 7fe9ca5ff700 1 -- 172.21.15.31:0/2045576265 --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9c4156c00 con 0x7fe9c40ac830 2023-10-06T02:23:21.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.523+0000 7fe9ca5ff700 1 -- 172.21.15.31:0/2045576265 --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9c40b20a0 con 0x7fe9c406eb00 2023-10-06T02:23:21.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.523+0000 7f7e2cf9a700 1 --2- 172.21.15.31:0/4143711347 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f7e0c066f30 0x7f7e0c0693e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7e24007fd0 tx=0x7f7e24007b20 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:21.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.523+0000 7f6162d9d700 1 --2- 172.21.15.31:0/729517814 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f616415a400 0x7f6164159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:21.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.523+0000 7fe9ca5ff700 1 -- 172.21.15.31:0/2045576265 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c4156ee0 con 0x7fe9c40cb8e0 2023-10-06T02:23:21.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.523+0000 7f91ff7fe700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f92000cb5a0 0x7f92000cb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:21.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.523+0000 7f91ff7fe700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f92000cb5a0 0x7f92000cb970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:47268/0 (socket says 172.21.15.31:47268) 2023-10-06T02:23:21.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.523+0000 7f91ff7fe700 1 -- 172.21.15.31:0/2654540350 learned_addr learned my addr 172.21.15.31:0/2654540350 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:21.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.524+0000 7f7e2df9c700 1 --2- 172.21.15.31:0/4143711347 >> [v2:172.21.15.31:6822/85579,v1:172.21.15.31:6825/85579] conn(0x7f7e0c06f7e0 0x7f7e0c071bf0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:21.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.523+0000 7fd9d159d700 1 -- 172.21.15.31:0/3492459277 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9c403c530 con 0x7fd9c4006b50 2023-10-06T02:23:21.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.524+0000 7fd9d159d700 1 -- 172.21.15.31:0/3492459277 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9c403d010 con 0x7fd9c4006b50 2023-10-06T02:23:21.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.523+0000 7f6161d9b700 1 --2- 172.21.15.31:0/729517814 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f61640cd9e0 0x7f6164158ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:21.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.523+0000 7f616259c700 1 -- 172.21.15.31:0/729517814 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f61640cd9e0 msgr2=0x7f6164158ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:21.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.523+0000 7f616259c700 1 --2- 172.21.15.31:0/729517814 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f61640cd9e0 0x7f6164158ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:21.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.523+0000 7f616259c700 1 -- 172.21.15.31:0/729517814 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f616415a400 msgr2=0x7f6164159010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:21.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.523+0000 7f616259c700 1 --2- 172.21.15.31:0/729517814 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f616415a400 0x7f6164159010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:21.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.524+0000 7f616259c700 1 -- 172.21.15.31:0/729517814 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f615c009ae0 con 0x7f61640cb5a0 2023-10-06T02:23:21.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.523+0000 7fe9c8b9c700 1 --2- 172.21.15.31:0/2045576265 >> v2:172.21.15.31:3300/0 conn(0x7fe9c406eb00 0x7fe9c40ad6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:21.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.523+0000 7fe9c3fff700 1 -- 172.21.15.31:0/2045576265 >> v2:172.21.15.31:3300/0 conn(0x7fe9c406eb00 msgr2=0x7fe9c40ad6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:21.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.523+0000 7fe9c3fff700 1 --2- 172.21.15.31:0/2045576265 >> v2:172.21.15.31:3300/0 conn(0x7fe9c406eb00 0x7fe9c40ad6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:21.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.524+0000 7fe9c3fff700 1 -- 172.21.15.31:0/2045576265 >> v1:172.21.15.62:6789/0 conn(0x7fe9c40ac830 legacy=0x7fe9c406e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:21.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.524+0000 7fe9c3fff700 1 -- 172.21.15.31:0/2045576265 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9c40abd50 con 0x7fe9c40cb8e0 2023-10-06T02:23:21.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.523+0000 7f91ff7fe700 1 -- 172.21.15.31:0/2654540350 >> v2:172.21.15.31:3300/0 conn(0x7f920015a5d0 msgr2=0x7f9200156e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:21.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.523+0000 7f91ff7fe700 1 --2- 172.21.15.31:0/2654540350 >> v2:172.21.15.31:3300/0 conn(0x7f920015a5d0 0x7f9200156e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:21.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.523+0000 7f91ff7fe700 1 -- 172.21.15.31:0/2654540350 >> v1:172.21.15.62:6789/0 conn(0x7f92000cd9e0 legacy=0x7f9200159ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:21.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.523+0000 7f91ff7fe700 1 -- 172.21.15.31:0/2654540350 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92000ccf00 con 0x7f92000cb5a0 2023-10-06T02:23:21.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.524+0000 7f65f8b9d700 1 -- 172.21.15.31:0/2958606099 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f65ec037dc0 con 0x7f65ec0d4b10 2023-10-06T02:23:21.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.523+0000 7f6af2ffd700 1 --2- 172.21.15.31:0/1087454585 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6af40b1e00 0x7f6af41555e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8009910 tx=0x7f6ae80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8ee1758aac8ca25 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:21.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.524+0000 7f6af27fc700 1 -- 172.21.15.31:0/1087454585 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae800b680 con 0x7f6af40b1e00 2023-10-06T02:23:21.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.524+0000 7f6af27fc700 1 -- 172.21.15.31:0/1087454585 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ae800b7e0 con 0x7f6af40b1e00 2023-10-06T02:23:21.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.524+0000 7f6af27fc700 1 -- 172.21.15.31:0/1087454585 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae800bba0 con 0x7f6af40b1e00 2023-10-06T02:23:21.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.524+0000 7f91ff7fe700 1 --2- 172.21.15.31:0/2654540350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92000cb5a0 0x7f92000cb970 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f91f8009f20 tx=0x7f91f8009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=162718cde8feedec server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:21.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.524+0000 7ff0ce7fc700 1 --2- 172.21.15.31:0/2040363347 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7ff0bc067000 0x7ff0bc0694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:21.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.524+0000 7f6161d9b700 1 --2- 172.21.15.31:0/729517814 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f61640cd9e0 0x7f6164158ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:23:21.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.524+0000 7ff0ce7fc700 1 -- 172.21.15.31:0/2040363347 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff0d40838f0 con 0x7ff0d00e96a0 2023-10-06T02:23:21.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.525+0000 7f616259c700 1 --2- 172.21.15.31:0/729517814 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f61640cb5a0 0x7f616414d1b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f615800b600 tx=0x7f61580091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:21.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.525+0000 7f91fe7fc700 1 -- 172.21.15.31:0/2654540350 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f800e070 con 0x7f92000cb5a0 2023-10-06T02:23:21.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.525+0000 7f91fe7fc700 1 -- 172.21.15.31:0/2654540350 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91f800ba40 con 0x7f92000cb5a0 2023-10-06T02:23:21.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.525+0000 7ff0ce7fc700 1 --2- 172.21.15.31:0/2040363347 >> [v2:172.21.15.31:6800/85567,v1:172.21.15.31:6801/85567] conn(0x7ff0bc06f8b0 0x7ff0bc071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:21.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.525+0000 7f6af9dff700 1 -- 172.21.15.31:0/1087454585 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6af40b1e00 msgr2=0x7f6af41555e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:21.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.525+0000 7f6af9dff700 1 --2- 172.21.15.31:0/1087454585 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6af40b1e00 0x7f6af41555e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8009910 tx=0x7f6ae80091a0 comp rx=0 tx=0).stop 2023-10-06T02:23:21.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.525+0000 7f91fe7fc700 1 -- 172.21.15.31:0/2654540350 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f800e070 con 0x7f92000cb5a0 2023-10-06T02:23:21.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.525+0000 7f7e1d7fa700 1 -- 172.21.15.31:0/4143711347 <== osd.2 v2:172.21.15.31:6822/85579 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f7e0c0722a0 con 0x7f7e0c06f7e0 2023-10-06T02:23:21.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.525+0000 7ff0de99c700 1 --2- 172.21.15.31:0/2040363347 >> [v2:172.21.15.31:6800/85567,v1:172.21.15.31:6801/85567] conn(0x7ff0bc06f8b0 0x7ff0bc071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:21.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.525+0000 7ff0ce7fc700 1 -- 172.21.15.31:0/2040363347 --> [v2:172.21.15.31:6800/85567,v1:172.21.15.31:6801/85567] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff0bc072370 con 0x7ff0bc06f8b0 2023-10-06T02:23:21.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.525+0000 7ff0ce7fc700 1 -- 172.21.15.31:0/2040363347 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff0d4083bf0 con 0x7ff0d00e96a0 2023-10-06T02:23:21.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.525+0000 7f6af9dff700 1 -- 172.21.15.31:0/1087454585 shutdown_connections 2023-10-06T02:23:21.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.525+0000 7f6af9dff700 1 --2- 172.21.15.31:0/1087454585 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6af40b1e00 0x7f6af41555e0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:21.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.525+0000 7f6af9dff700 1 -- 172.21.15.31:0/1087454585 >> 172.21.15.31:0/1087454585 conn(0x7f6af406d170 msgr2=0x7f6af40cc300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:21.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.525+0000 7f6af9dff700 1 -- 172.21.15.31:0/1087454585 shutdown_connections 2023-10-06T02:23:21.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.525+0000 7f6af9dff700 1 -- 172.21.15.31:0/1087454585 wait complete. 2023-10-06T02:23:21.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.526+0000 7fe9c3fff700 1 --2- 172.21.15.31:0/2045576265 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe9c40cb8e0 0x7fe9c40cbcb0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe9b4009f20 tx=0x7fe9b4009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3c89184fa74db3b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:21.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.526+0000 7fe9c37fe700 1 -- 172.21.15.31:0/2045576265 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b400e070 con 0x7fe9c40cb8e0 2023-10-06T02:23:21.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.526+0000 7fe9c37fe700 1 -- 172.21.15.31:0/2045576265 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9b400ba80 con 0x7fe9c40cb8e0 2023-10-06T02:23:21.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.526+0000 7fe9c37fe700 1 -- 172.21.15.31:0/2045576265 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b400e070 con 0x7fe9c40cb8e0 2023-10-06T02:23:21.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.526+0000 7f9205dff700 1 -- 172.21.15.31:0/2654540350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92000cb5a0 msgr2=0x7f92000cb970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:21.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.526+0000 7f9205dff700 1 --2- 172.21.15.31:0/2654540350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92000cb5a0 0x7f92000cb970 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f91f8009f20 tx=0x7f91f8009840 comp rx=0 tx=0).stop 2023-10-06T02:23:21.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.526+0000 7f9205dff700 1 -- 172.21.15.31:0/2654540350 shutdown_connections 2023-10-06T02:23:21.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.526+0000 7f9205dff700 1 --2- 172.21.15.31:0/2654540350 >> v2:172.21.15.31:3300/0 conn(0x7f920015a5d0 0x7f9200156e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:21.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.526+0000 7f9205dff700 1 --2- 172.21.15.31:0/2654540350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92000cb5a0 0x7f92000cb970 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:21.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.526+0000 7f9205dff700 1 -- 172.21.15.31:0/2654540350 >> 172.21.15.31:0/2654540350 conn(0x7f9200070e30 msgr2=0x7f92000d0710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:21.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.526+0000 7ff0de19b700 1 --2- 172.21.15.31:0/2040363347 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7ff0bc067000 0x7ff0bc0694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:21.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.526+0000 7f9205dff700 1 -- 172.21.15.31:0/2654540350 shutdown_connections 2023-10-06T02:23:21.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.527+0000 7fe9ca5ff700 1 -- 172.21.15.31:0/2045576265 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe9c40cb8e0 msgr2=0x7fe9c40cbcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:21.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.527+0000 7fe9ca5ff700 1 --2- 172.21.15.31:0/2045576265 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe9c40cb8e0 0x7fe9c40cbcb0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe9b4009f20 tx=0x7fe9b4009840 comp rx=0 tx=0).stop 2023-10-06T02:23:21.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.527+0000 7fe9ca5ff700 1 -- 172.21.15.31:0/2045576265 shutdown_connections 2023-10-06T02:23:21.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.527+0000 7fe9ca5ff700 1 --2- 172.21.15.31:0/2045576265 >> v2:172.21.15.31:3300/0 conn(0x7fe9c406eb00 0x7fe9c40ad6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:21.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.527+0000 7fe9ca5ff700 1 --2- 172.21.15.31:0/2045576265 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe9c40cb8e0 0x7fe9c40cbcb0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:21.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.527+0000 7fe9ca5ff700 1 -- 172.21.15.31:0/2045576265 >> 172.21.15.31:0/2045576265 conn(0x7fe9c40cc300 msgr2=0x7fe9c40cc700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:21.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.527+0000 7f61527fc700 1 -- 172.21.15.31:0/729517814 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f615800f510 con 0x7f61640cb5a0 2023-10-06T02:23:21.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.527+0000 7f61527fc700 1 -- 172.21.15.31:0/729517814 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f615800f670 con 0x7f61640cb5a0 2023-10-06T02:23:21.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.527+0000 7f61527fc700 1 -- 172.21.15.31:0/729517814 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f615800fa30 con 0x7f61640cb5a0 2023-10-06T02:23:21.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.527+0000 7fe9ca5ff700 1 -- 172.21.15.31:0/2045576265 shutdown_connections 2023-10-06T02:23:21.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.527+0000 7fe9ca5ff700 1 -- 172.21.15.31:0/2045576265 wait complete. 2023-10-06T02:23:21.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.525+0000 7fd9b5ffb700 1 -- 172.21.15.31:0/3492459277 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd9a4000fd0 con 0x7fd9c4006b50 2023-10-06T02:23:21.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.527+0000 7fd9b7fff700 1 -- 172.21.15.31:0/3492459277 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd9b801a400 con 0x7fd9c4006b50 2023-10-06T02:23:21.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.528+0000 7f65df7fe700 1 -- 172.21.15.31:0/2958606099 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f65f40675a0 con 0x7f65ec0d4b10 2023-10-06T02:23:21.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.526+0000 7f9205dff700 1 -- 172.21.15.31:0/2654540350 wait complete. 2023-10-06T02:23:21.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.529+0000 7f9205dff700 1 Processor -- start 2023-10-06T02:23:21.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.529+0000 7fe9ca5ff700 1 Processor -- start 2023-10-06T02:23:21.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.529+0000 7fe9ca5ff700 1 -- start start 2023-10-06T02:23:21.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.529+0000 7ff0de99c700 1 --2- 172.21.15.31:0/2040363347 >> [v2:172.21.15.31:6800/85567,v1:172.21.15.31:6801/85567] conn(0x7ff0bc06f8b0 0x7ff0bc071cc0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:21.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.528+0000 7f6af9dff700 1 Processor -- start 2023-10-06T02:23:21.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.528+0000 7f6af9dff700 1 -- start start 2023-10-06T02:23:21.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.530+0000 7f6af9dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6af4067fb0 0x7f6af40ab0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:21.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.529+0000 7ff0de19b700 1 --2- 172.21.15.31:0/2040363347 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7ff0bc067000 0x7ff0bc0694b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8009390 tx=0x7ff0c8016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:21.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.529+0000 7f6168bff700 1 -- 172.21.15.31:0/729517814 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f616414d8b0 con 0x7f61640cb5a0 2023-10-06T02:23:21.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.530+0000 7f65df7fe700 1 --2- 172.21.15.31:0/2958606099 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f65d8067080 0x7f65d8069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:21.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.530+0000 7f6168bff700 1 -- 172.21.15.31:0/729517814 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61640b3880 con 0x7f61640cb5a0 2023-10-06T02:23:21.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.531+0000 7f65df7fe700 1 -- 172.21.15.31:0/2958606099 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f65f40fb840 con 0x7f65ec0d4b10 2023-10-06T02:23:21.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.530+0000 7fd9b7fff700 1 --2- 172.21.15.31:0/3492459277 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fd9b0067080 0x7fd9b0069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:21.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.530+0000 7fd9b7fff700 1 -- 172.21.15.31:0/3492459277 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd9b8084100 con 0x7fd9c4006b50 2023-10-06T02:23:21.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.530+0000 7fd9b7fff700 1 --2- 172.21.15.31:0/3492459277 >> [v2:172.21.15.31:6815/85584,v1:172.21.15.31:6817/85584] conn(0x7fd9b006f8b0 0x7fd9b0071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:21.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.530+0000 7fd9b7fff700 1 -- 172.21.15.31:0/3492459277 --> [v2:172.21.15.31:6815/85584,v1:172.21.15.31:6817/85584] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd9b0072370 con 0x7fd9b006f8b0 2023-10-06T02:23:21.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.531+0000 7fd9cb7fe700 1 --2- 172.21.15.31:0/3492459277 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fd9b0067080 0x7fd9b0069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:21.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.531+0000 7fd9b7fff700 1 -- 172.21.15.31:0/3492459277 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd9b8009240 con 0x7fd9c4006b50 2023-10-06T02:23:21.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.531+0000 7f65df7fe700 1 --2- 172.21.15.31:0/2958606099 >> [v2:172.21.15.31:6808/85568,v1:172.21.15.31:6809/85568] conn(0x7f65d806f930 0x7f65d8071d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:21.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.531+0000 7fe9ca5ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9c406eb00 0x7fe9c4154e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:21.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.531+0000 7f65df7fe700 1 -- 172.21.15.31:0/2958606099 --> [v2:172.21.15.31:6808/85568,v1:172.21.15.31:6809/85568] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f65d80723f0 con 0x7f65d806f930 2023-10-06T02:23:21.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.531+0000 7f65df7fe700 1 -- 172.21.15.31:0/2958606099 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f65f40fbb70 con 0x7f65ec0d4b10 2023-10-06T02:23:21.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.531+0000 7fe9ca5ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe9c40ac830 0x7fe9c414aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:21.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.531+0000 7fe9ca5ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9c40cb8e0 0x7fe9c414b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:21.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.531+0000 7fd9d0d9c700 1 --2- 172.21.15.31:0/3492459277 >> [v2:172.21.15.31:6815/85584,v1:172.21.15.31:6817/85584] conn(0x7fd9b006f8b0 0x7fd9b0071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:21.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.530+0000 7f6af9dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6af40b1e00 0x7f6af4150ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:21.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.530+0000 7f6af9dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6af4155b20 0x7f6af4151410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:21.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.530+0000 7f6af3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6af4155b20 0x7f6af4151410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:21.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.530+0000 7f6af3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6af4155b20 0x7f6af4151410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37540/0 (socket says 172.21.15.31:37540) 2023-10-06T02:23:21.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.530+0000 7f6af3fff700 1 -- 172.21.15.31:0/1058530425 learned_addr learned my addr 172.21.15.31:0/1058530425 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:21.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.530+0000 7f6af9dff700 1 -- 172.21.15.31:0/1058530425 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af4170b40 con 0x7f6af4155b20 2023-10-06T02:23:21.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.531+0000 7f6af9dff700 1 -- 172.21.15.31:0/1058530425 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af417ab30 con 0x7f6af4067fb0 2023-10-06T02:23:21.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.531+0000 7f6af9dff700 1 -- 172.21.15.31:0/1058530425 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6af417a850 con 0x7f6af40b1e00 2023-10-06T02:23:21.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.531+0000 7fe9c3fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9c406eb00 0x7fe9c4154e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:21.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.531+0000 7fe9c3fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9c406eb00 0x7fe9c4154e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:47308/0 (socket says 172.21.15.31:47308) 2023-10-06T02:23:21.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.532+0000 7fe9c8b9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9c40cb8e0 0x7fe9c414b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:21.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.532+0000 7fe9c8b9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9c40cb8e0 0x7fe9c414b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37546/0 (socket says 172.21.15.31:37546) 2023-10-06T02:23:21.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.532+0000 7fe9c3fff700 1 -- 172.21.15.31:0/3222514863 learned_addr learned my addr 172.21.15.31:0/3222514863 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:21.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.532+0000 7fe9ca5ff700 1 -- 172.21.15.31:0/3222514863 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c4158370 con 0x7fe9c40cb8e0 2023-10-06T02:23:21.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.532+0000 7fe9ca5ff700 1 -- 172.21.15.31:0/3222514863 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c414a660 con 0x7fe9c406eb00 2023-10-06T02:23:21.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.531+0000 7f6af37fe700 1 --2- 172.21.15.31:0/1058530425 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6af4067fb0 0x7f6af40ab0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:21.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.532+0000 7f6af3fff700 1 -- 172.21.15.31:0/1058530425 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6af40b1e00 msgr2=0x7f6af4150ed0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T02:23:21.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.532+0000 7f6af3fff700 1 --2- 172.21.15.31:0/1058530425 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6af40b1e00 0x7f6af4150ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:21.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.532+0000 7f6af3fff700 1 -- 172.21.15.31:0/1058530425 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6af4067fb0 msgr2=0x7f6af40ab0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:21.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.532+0000 7f6af3fff700 1 --2- 172.21.15.31:0/1058530425 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6af4067fb0 0x7f6af40ab0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:21.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.532+0000 7fe9ca5ff700 1 -- 172.21.15.31:0/3222514863 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c4149a50 con 0x7fe9c40ac830 2023-10-06T02:23:21.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.529+0000 7f9205dff700 1 -- start start 2023-10-06T02:23:21.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.532+0000 7f6af3fff700 1 -- 172.21.15.31:0/1058530425 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ae8009420 con 0x7f6af4155b20 2023-10-06T02:23:21.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.531+0000 7f9205dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92000cb5a0 0x7f9200156300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:21.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.531+0000 7f9205dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92000cd9e0 0x7f92000b4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:21.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.531+0000 7f9205dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f920015a5d0 0x7f92000b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:21.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.532+0000 7ff0ce7fc700 1 -- 172.21.15.31:0/2040363347 <== osd.0 v2:172.21.15.31:6800/85567 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7ff0bc072370 con 0x7ff0bc06f8b0 2023-10-06T02:23:21.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.532+0000 7fe9bbfff700 1 --2- 172.21.15.31:0/3222514863 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe9c40ac830 0x7fe9c414aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:21.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.531+0000 7f9205dff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9200156840 con 0x7f92000cb5a0 2023-10-06T02:23:21.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.531+0000 7f91feffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92000cd9e0 0x7f92000b4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:21.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.531+0000 7f91feffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92000cd9e0 0x7f92000b4800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:47296/0 (socket says 172.21.15.31:47296) 2023-10-06T02:23:21.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.531+0000 7f91feffd700 1 -- 172.21.15.31:0/326909324 learned_addr learned my addr 172.21.15.31:0/326909324 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:21.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.531+0000 7f9205dff700 1 -- 172.21.15.31:0/326909324 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f920015c9f0 con 0x7f92000cd9e0 2023-10-06T02:23:21.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.531+0000 7f9205dff700 1 -- 172.21.15.31:0/326909324 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f920015c6a0 con 0x7f920015a5d0 2023-10-06T02:23:21.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.532+0000 7f6168bff700 1 -- 172.21.15.31:0/729517814 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f616404c3c0 con 0x7f61640cb5a0 2023-10-06T02:23:21.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.532+0000 7fd9d0d9c700 1 --2- 172.21.15.31:0/3492459277 >> [v2:172.21.15.31:6815/85584,v1:172.21.15.31:6817/85584] conn(0x7fd9b006f8b0 0x7fd9b0071cc0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:21.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.532+0000 7f6af3fff700 1 --2- 172.21.15.31:0/1058530425 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6af4155b20 0x7f6af4151410 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6aec00c8e0 tx=0x7f6aec001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:21.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.533+0000 7f6adb7fe700 1 -- 172.21.15.31:0/1058530425 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aec00ce80 con 0x7f6af4155b20 2023-10-06T02:23:21.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.533+0000 7f6adb7fe700 1 -- 172.21.15.31:0/1058530425 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6aec00a7e0 con 0x7f6af4155b20 2023-10-06T02:23:21.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.533+0000 7f6adb7fe700 1 -- 172.21.15.31:0/1058530425 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aec00ab80 con 0x7f6af4155b20 2023-10-06T02:23:21.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.533+0000 7f6af9dff700 1 -- 172.21.15.31:0/1058530425 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6af4176770 con 0x7f6af4155b20 2023-10-06T02:23:21.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.533+0000 7f65f37fe700 1 --2- 172.21.15.31:0/2958606099 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f65d8067080 0x7f65d8069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:21.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.533+0000 7f65f2ffd700 1 --2- 172.21.15.31:0/2958606099 >> [v2:172.21.15.31:6808/85568,v1:172.21.15.31:6809/85568] conn(0x7f65d806f930 0x7f65d8071d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:21.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.533+0000 7fd9cb7fe700 1 --2- 172.21.15.31:0/3492459277 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fd9b0067080 0x7fd9b0069530 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd9c4011ff0 tx=0x7fd9cc068000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:21.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.533+0000 7fe9c8b9c700 1 -- 172.21.15.31:0/3222514863 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe9c40ac830 msgr2=0x7fe9c414aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:21.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.533+0000 7fe9c8b9c700 1 --2- 172.21.15.31:0/3222514863 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe9c40ac830 0x7fe9c414aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:21.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.533+0000 7fe9c8b9c700 1 -- 172.21.15.31:0/3222514863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9c406eb00 msgr2=0x7fe9c4154e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:21.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.533+0000 7fe9c8b9c700 1 --2- 172.21.15.31:0/3222514863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9c406eb00 0x7fe9c4154e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:21.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.533+0000 7fe9c8b9c700 1 -- 172.21.15.31:0/3222514863 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9b4009ae0 con 0x7fe9c40cb8e0 2023-10-06T02:23:21.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.533+0000 7fe9c3fff700 1 --2- 172.21.15.31:0/3222514863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9c406eb00 0x7fe9c4154e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:23:21.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.533+0000 7fe9bbfff700 1 --2- 172.21.15.31:0/3222514863 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe9c40ac830 0x7fe9c414aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:23:21.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.532+0000 7f91feffd700 1 -- 172.21.15.31:0/326909324 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f920015a5d0 msgr2=0x7f92000b4d40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T02:23:21.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.532+0000 7f91fffff700 1 --2- 172.21.15.31:0/326909324 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f920015a5d0 0x7f92000b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:21.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.533+0000 7f6af9dff700 1 -- 172.21.15.31:0/1058530425 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6af4176d30 con 0x7f6af4155b20 2023-10-06T02:23:21.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.534+0000 7f91feffd700 1 --2- 172.21.15.31:0/326909324 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f920015a5d0 0x7f92000b4d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:21.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.534+0000 7f91feffd700 1 -- 172.21.15.31:0/326909324 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92000cb5a0 msgr2=0x7f9200156300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:21.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.534+0000 7f91ff7fe700 1 --2- 172.21.15.31:0/326909324 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92000cb5a0 0x7f9200156300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:21.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.534+0000 7f6af9dff700 1 -- 172.21.15.31:0/1058530425 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6af404c3c0 con 0x7f6af4155b20 2023-10-06T02:23:21.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.534+0000 7f91feffd700 1 --2- 172.21.15.31:0/326909324 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92000cb5a0 0x7f9200156300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:21.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.534+0000 7f91feffd700 1 -- 172.21.15.31:0/326909324 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91f8009ae0 con 0x7f92000cd9e0 2023-10-06T02:23:21.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.534+0000 7f91ff7fe700 1 --2- 172.21.15.31:0/326909324 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92000cb5a0 0x7f9200156300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:23:21.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.534+0000 7f91feffd700 1 --2- 172.21.15.31:0/326909324 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92000cd9e0 0x7f92000b4800 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f91f000c790 tx=0x7f91f0007f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:21.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.535+0000 7f65f2ffd700 1 --2- 172.21.15.31:0/2958606099 >> [v2:172.21.15.31:6808/85568,v1:172.21.15.31:6809/85568] conn(0x7f65d806f930 0x7f65d8071d40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:21.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.535+0000 7fd9b7fff700 1 -- 172.21.15.31:0/3492459277 <== osd.3 v2:172.21.15.31:6815/85584 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fd9b0072370 con 0x7fd9b006f8b0 2023-10-06T02:23:21.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.535+0000 7f65f37fe700 1 --2- 172.21.15.31:0/2958606099 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f65d8067080 0x7f65d8069530 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f65e8008dc0 tx=0x7f65e8008cb0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:21.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.535+0000 7fe9c8b9c700 1 --2- 172.21.15.31:0/3222514863 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9c40cb8e0 0x7fe9c414b030 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe9c406ef00 tx=0x7fe9bc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:21.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.535+0000 7f61527fc700 1 -- 172.21.15.31:0/729517814 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f615801a400 con 0x7f61640cb5a0 2023-10-06T02:23:21.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.535+0000 7f91e77fe700 1 -- 172.21.15.31:0/326909324 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f0011590 con 0x7f92000cd9e0 2023-10-06T02:23:21.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.535+0000 7f91e77fe700 1 -- 172.21.15.31:0/326909324 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91f00116f0 con 0x7f92000cd9e0 2023-10-06T02:23:21.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.535+0000 7f91e77fe700 1 -- 172.21.15.31:0/326909324 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f0011ab0 con 0x7f92000cd9e0 2023-10-06T02:23:21.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.535+0000 7f9205dff700 1 -- 172.21.15.31:0/326909324 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9200158070 con 0x7f92000cd9e0 2023-10-06T02:23:21.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.536+0000 7f9205dff700 1 -- 172.21.15.31:0/326909324 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9200158630 con 0x7f92000cd9e0 2023-10-06T02:23:21.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.536+0000 7fe9c0ff9700 1 -- 172.21.15.31:0/3222514863 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc014430 con 0x7fe9c40cb8e0 2023-10-06T02:23:21.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.536+0000 7fe9c0ff9700 1 -- 172.21.15.31:0/3222514863 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9bc014590 con 0x7fe9c40cb8e0 2023-10-06T02:23:21.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.536+0000 7fe9c0ff9700 1 -- 172.21.15.31:0/3222514863 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc014930 con 0x7fe9c40cb8e0 2023-10-06T02:23:21.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.536+0000 7fe9ca5ff700 1 -- 172.21.15.31:0/3222514863 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9c4149d30 con 0x7fe9c40cb8e0 2023-10-06T02:23:21.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.536+0000 7fe9ca5ff700 1 -- 172.21.15.31:0/3222514863 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9c414a2f0 con 0x7fe9c40cb8e0 2023-10-06T02:23:21.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.536+0000 7f61527fc700 1 --2- 172.21.15.31:0/729517814 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6144067080 0x7f6144069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:21.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.537+0000 7f61527fc700 1 -- 172.21.15.31:0/729517814 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6158084550 con 0x7f61640cb5a0 2023-10-06T02:23:21.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.537+0000 7f6161d9b700 1 --2- 172.21.15.31:0/729517814 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6144067080 0x7f6144069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:21.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.537+0000 7f61527fc700 1 --2- 172.21.15.31:0/729517814 >> [v2:172.21.15.62:6800/85799,v1:172.21.15.62:6801/85799] conn(0x7f614406f930 0x7f6144071d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:21.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.537+0000 7f61527fc700 1 -- 172.21.15.31:0/729517814 --> [v2:172.21.15.62:6800/85799,v1:172.21.15.62:6801/85799] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f61440723f0 con 0x7f614406f930 2023-10-06T02:23:21.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.537+0000 7f61527fc700 1 -- 172.21.15.31:0/729517814 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6158084880 con 0x7f61640cb5a0 2023-10-06T02:23:21.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.537+0000 7fe9ca5ff700 1 -- 172.21.15.31:0/3222514863 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe9c404c3c0 con 0x7fe9c40cb8e0 2023-10-06T02:23:21.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.537+0000 7f6adb7fe700 1 -- 172.21.15.31:0/1058530425 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6aec024430 con 0x7f6af4155b20 2023-10-06T02:23:21.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.538+0000 7f6162d9d700 1 --2- 172.21.15.31:0/729517814 >> [v2:172.21.15.62:6800/85799,v1:172.21.15.62:6801/85799] conn(0x7f614406f930 0x7f6144071d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:21.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.538+0000 7f9205dff700 1 -- 172.21.15.31:0/326909324 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f920004c3c0 con 0x7f92000cd9e0 2023-10-06T02:23:21.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.538+0000 7f6161d9b700 1 --2- 172.21.15.31:0/729517814 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6144067080 0x7f6144069530 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f615c009880 tx=0x7f615c017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:21.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.539+0000 7f65df7fe700 1 -- 172.21.15.31:0/2958606099 <== osd.1 v2:172.21.15.31:6808/85568 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f65d80723f0 con 0x7f65d806f930 2023-10-06T02:23:21.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.539+0000 7fe9c0ff9700 1 -- 172.21.15.31:0/3222514863 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe9bc014ca0 con 0x7fe9c40cb8e0 2023-10-06T02:23:21.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.539+0000 7f91e77fe700 1 -- 172.21.15.31:0/326909324 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f91f0012420 con 0x7f92000cd9e0 2023-10-06T02:23:21.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.540+0000 7f6adb7fe700 1 --2- 172.21.15.31:0/1058530425 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6ae0067000 0x7f6ae00694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:21.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.540+0000 7f6adb7fe700 1 -- 172.21.15.31:0/1058530425 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6aec085060 con 0x7f6af4155b20 2023-10-06T02:23:21.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.540+0000 7f6adb7fe700 1 --2- 172.21.15.31:0/1058530425 >> [v2:172.21.15.62:6816/85800,v1:172.21.15.62:6817/85800] conn(0x7f6ae006f8b0 0x7f6ae0071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:21.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.540+0000 7f6adb7fe700 1 -- 172.21.15.31:0/1058530425 --> [v2:172.21.15.62:6816/85800,v1:172.21.15.62:6817/85800] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6ae0072370 con 0x7f6ae006f8b0 2023-10-06T02:23:21.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.540+0000 7f6adb7fe700 1 -- 172.21.15.31:0/1058530425 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6aec085430 con 0x7f6af4155b20 2023-10-06T02:23:21.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.541+0000 7f6af37fe700 1 --2- 172.21.15.31:0/1058530425 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6ae0067000 0x7f6ae00694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:21.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.541+0000 7fe9c0ff9700 1 --2- 172.21.15.31:0/3222514863 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe9a406b490 0x7fe9a406d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:21.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.541+0000 7fe9c0ff9700 1 -- 172.21.15.31:0/3222514863 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe9bc084c20 con 0x7fe9c40cb8e0 2023-10-06T02:23:21.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.541+0000 7fe9c0ff9700 1 --2- 172.21.15.31:0/3222514863 >> [v2:172.21.15.62:6808/85798,v1:172.21.15.62:6809/85798] conn(0x7fe9a4073d40 0x7fe9a4076150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:21.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.541+0000 7fe9c0ff9700 1 -- 172.21.15.31:0/3222514863 --> [v2:172.21.15.62:6808/85798,v1:172.21.15.62:6809/85798] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe9a4076800 con 0x7fe9a4073d40 2023-10-06T02:23:21.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.541+0000 7fe9c0ff9700 1 -- 172.21.15.31:0/3222514863 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9bc089220 con 0x7fe9c40cb8e0 2023-10-06T02:23:21.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.541+0000 7f91e77fe700 1 --2- 172.21.15.31:0/326909324 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f91ec067000 0x7f91ec0694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:21.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.541+0000 7f91e77fe700 1 -- 172.21.15.31:0/326909324 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f91f0083a70 con 0x7f92000cd9e0 2023-10-06T02:23:21.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.541+0000 7f91e77fe700 1 --2- 172.21.15.31:0/326909324 >> [v2:172.21.15.62:6824/85843,v1:172.21.15.62:6825/85843] conn(0x7f91ec06f8b0 0x7f91ec071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:21.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.541+0000 7f91e77fe700 1 -- 172.21.15.31:0/326909324 --> [v2:172.21.15.62:6824/85843,v1:172.21.15.62:6825/85843] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f91ec072370 con 0x7f91ec06f8b0 2023-10-06T02:23:21.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.541+0000 7f91e77fe700 1 -- 172.21.15.31:0/326909324 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f91f0088220 con 0x7f92000cd9e0 2023-10-06T02:23:21.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.542+0000 7f91ff7fe700 1 --2- 172.21.15.31:0/326909324 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f91ec067000 0x7f91ec0694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:21.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.542+0000 7f6162d9d700 1 --2- 172.21.15.31:0/729517814 >> [v2:172.21.15.62:6800/85799,v1:172.21.15.62:6801/85799] conn(0x7f614406f930 0x7f6144071d40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:21.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.542+0000 7f6af37fe700 1 --2- 172.21.15.31:0/1058530425 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6ae0067000 0x7f6ae00694b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6ae40058d0 tx=0x7f6ae4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:21.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.542+0000 7fe9bbfff700 1 --2- 172.21.15.31:0/3222514863 >> [v2:172.21.15.62:6808/85798,v1:172.21.15.62:6809/85798] conn(0x7fe9a4073d40 0x7fe9a4076150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:21.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.542+0000 7fe9c3fff700 1 --2- 172.21.15.31:0/3222514863 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe9a406b490 0x7fe9a406d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:21.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.542+0000 7f91fffff700 1 --2- 172.21.15.31:0/326909324 >> [v2:172.21.15.62:6824/85843,v1:172.21.15.62:6825/85843] conn(0x7f91ec06f8b0 0x7f91ec071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:21.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.542+0000 7f6af2ffd700 1 --2- 172.21.15.31:0/1058530425 >> [v2:172.21.15.62:6816/85800,v1:172.21.15.62:6817/85800] conn(0x7f6ae006f8b0 0x7f6ae0071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:21.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.543+0000 7f91fffff700 1 --2- 172.21.15.31:0/326909324 >> [v2:172.21.15.62:6824/85843,v1:172.21.15.62:6825/85843] conn(0x7f91ec06f8b0 0x7f91ec071cc0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:21.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.543+0000 7f91ff7fe700 1 --2- 172.21.15.31:0/326909324 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f91ec067000 0x7f91ec0694b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f91f8009ab0 tx=0x7f91f8018000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:21.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.543+0000 7f61527fc700 1 -- 172.21.15.31:0/729517814 <== osd.4 v2:172.21.15.62:6800/85799 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f61440723f0 con 0x7f614406f930 2023-10-06T02:23:21.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.543+0000 7fe9bbfff700 1 --2- 172.21.15.31:0/3222514863 >> [v2:172.21.15.62:6808/85798,v1:172.21.15.62:6809/85798] conn(0x7fe9a4073d40 0x7fe9a4076150 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:21.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.543+0000 7f6af2ffd700 1 --2- 172.21.15.31:0/1058530425 >> [v2:172.21.15.62:6816/85800,v1:172.21.15.62:6817/85800] conn(0x7f6ae006f8b0 0x7f6ae0071cc0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:21.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.544+0000 7fe9c3fff700 1 --2- 172.21.15.31:0/3222514863 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe9a406b490 0x7fe9a406d940 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe9b4000f80 tx=0x7fe9b4018000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:21.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.544+0000 7fe9c0ff9700 1 -- 172.21.15.31:0/3222514863 <== osd.6 v2:172.21.15.62:6808/85798 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fe9a4076800 con 0x7fe9a4073d40 2023-10-06T02:23:21.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.545+0000 7f91e77fe700 1 -- 172.21.15.31:0/326909324 <== osd.7 v2:172.21.15.62:6824/85843 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f91ec072370 con 0x7f91ec06f8b0 2023-10-06T02:23:21.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.544+0000 7f6adb7fe700 1 -- 172.21.15.31:0/1058530425 <== osd.5 v2:172.21.15.62:6816/85800 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f6ae0072370 con 0x7f6ae006f8b0 2023-10-06T02:23:21.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.561+0000 7f7e2e79d700 1 -- 172.21.15.31:0/4143711347 --> [v2:172.21.15.31:6822/85579,v1:172.21.15.31:6825/85579] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7e08002d30 con 0x7f7e0c06f7e0 2023-10-06T02:23:21.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.562+0000 7f7e1d7fa700 1 -- 172.21.15.31:0/4143711347 <== osd.2 v2:172.21.15.31:6822/85579 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f7e08002d30 con 0x7f7e0c06f7e0 2023-10-06T02:23:21.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.563+0000 7f7e12ffd700 1 -- 172.21.15.31:0/4143711347 >> [v2:172.21.15.31:6822/85579,v1:172.21.15.31:6825/85579] conn(0x7f7e0c06f7e0 msgr2=0x7f7e0c071bf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:21.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.563+0000 7f7e12ffd700 1 --2- 172.21.15.31:0/4143711347 >> [v2:172.21.15.31:6822/85579,v1:172.21.15.31:6825/85579] conn(0x7f7e0c06f7e0 0x7f7e0c071bf0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:21.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.564+0000 7f7e12ffd700 1 -- 172.21.15.31:0/4143711347 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f7e0c066f30 msgr2=0x7f7e0c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:21.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.564+0000 7f7e12ffd700 1 --2- 172.21.15.31:0/4143711347 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f7e0c066f30 0x7f7e0c0693e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7e24007fd0 tx=0x7f7e24007b20 comp rx=0 tx=0).stop 2023-10-06T02:23:21.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.564+0000 7f7e12ffd700 1 -- 172.21.15.31:0/4143711347 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e200547d0 msgr2=0x7f7e200d8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:21.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.564+0000 7f7e12ffd700 1 --2- 172.21.15.31:0/4143711347 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e200547d0 0x7f7e200d8cc0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7e28066da0 tx=0x7f7e28068000 comp rx=0 tx=0).stop 2023-10-06T02:23:21.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.564+0000 7f7e2df9c700 1 -- 172.21.15.31:0/4143711347 reap_dead start 2023-10-06T02:23:21.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.565+0000 7f7e12ffd700 1 -- 172.21.15.31:0/4143711347 shutdown_connections 2023-10-06T02:23:21.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.565+0000 7f7e12ffd700 1 -- 172.21.15.31:0/4143711347 >> 172.21.15.31:0/4143711347 conn(0x7f7e20055ba0 msgr2=0x7f7e2004f890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:21.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.565+0000 7ff0e03ff700 1 -- 172.21.15.31:0/2040363347 --> [v2:172.21.15.31:6800/85567,v1:172.21.15.31:6801/85567] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff0d003ab40 con 0x7ff0bc06f8b0 2023-10-06T02:23:21.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.566+0000 7f7e12ffd700 1 -- 172.21.15.31:0/4143711347 shutdown_connections 2023-10-06T02:23:21.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.566+0000 7ff0ce7fc700 1 -- 172.21.15.31:0/2040363347 <== osd.0 v2:172.21.15.31:6800/85567 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff0d003ab40 con 0x7ff0bc06f8b0 2023-10-06T02:23:21.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.566+0000 7f7e12ffd700 1 -- 172.21.15.31:0/4143711347 wait complete. 2023-10-06T02:23:21.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.568+0000 7ff0e03ff700 1 -- 172.21.15.31:0/2040363347 >> [v2:172.21.15.31:6800/85567,v1:172.21.15.31:6801/85567] conn(0x7ff0bc06f8b0 msgr2=0x7ff0bc071cc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:21.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.568+0000 7ff0e03ff700 1 --2- 172.21.15.31:0/2040363347 >> [v2:172.21.15.31:6800/85567,v1:172.21.15.31:6801/85567] conn(0x7ff0bc06f8b0 0x7ff0bc071cc0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:21.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.568+0000 7ff0e03ff700 1 -- 172.21.15.31:0/2040363347 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7ff0bc067000 msgr2=0x7ff0bc0694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:21.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.568+0000 7ff0e03ff700 1 --2- 172.21.15.31:0/2040363347 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7ff0bc067000 0x7ff0bc0694b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8009390 tx=0x7ff0c8016000 comp rx=0 tx=0).stop 2023-10-06T02:23:21.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.568+0000 7ff0e03ff700 1 -- 172.21.15.31:0/2040363347 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0d00e96a0 msgr2=0x7ff0d0037b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:21.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.568+0000 7ff0e03ff700 1 --2- 172.21.15.31:0/2040363347 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0d00e96a0 0x7ff0d0037b10 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff0d400b5b0 tx=0x7ff0d40090f0 comp rx=0 tx=0).stop 2023-10-06T02:23:21.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.570+0000 7ff0de99c700 1 -- 172.21.15.31:0/2040363347 reap_dead start 2023-10-06T02:23:21.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.570+0000 7ff0e03ff700 1 -- 172.21.15.31:0/2040363347 shutdown_connections 2023-10-06T02:23:21.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.570+0000 7ff0e03ff700 1 -- 172.21.15.31:0/2040363347 >> 172.21.15.31:0/2040363347 conn(0x7ff0d00517e0 msgr2=0x7ff0d00e3030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:21.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.571+0000 7ff0e03ff700 1 -- 172.21.15.31:0/2040363347 shutdown_connections 2023-10-06T02:23:21.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.571+0000 7ff0e03ff700 1 -- 172.21.15.31:0/2040363347 wait complete. 2023-10-06T02:23:21.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.573+0000 7f65f8b9d700 1 -- 172.21.15.31:0/2958606099 --> [v2:172.21.15.31:6808/85568,v1:172.21.15.31:6809/85568] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f65ec037dc0 con 0x7f65d806f930 2023-10-06T02:23:21.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.574+0000 7f65df7fe700 1 -- 172.21.15.31:0/2958606099 <== osd.1 v2:172.21.15.31:6808/85568 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f65ec037dc0 con 0x7f65d806f930 2023-10-06T02:23:21.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.574+0000 7fd9b5ffb700 1 -- 172.21.15.31:0/3492459277 --> [v2:172.21.15.31:6815/85584,v1:172.21.15.31:6817/85584] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd9a4002d30 con 0x7fd9b006f8b0 2023-10-06T02:23:21.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.575+0000 7fd9b7fff700 1 -- 172.21.15.31:0/3492459277 <== osd.3 v2:172.21.15.31:6815/85584 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd9a4002d30 con 0x7fd9b006f8b0 2023-10-06T02:23:21.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.575+0000 7f65dd7fa700 1 -- 172.21.15.31:0/2958606099 >> [v2:172.21.15.31:6808/85568,v1:172.21.15.31:6809/85568] conn(0x7f65d806f930 msgr2=0x7f65d8071d40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:21.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.575+0000 7f65dd7fa700 1 --2- 172.21.15.31:0/2958606099 >> [v2:172.21.15.31:6808/85568,v1:172.21.15.31:6809/85568] conn(0x7f65d806f930 0x7f65d8071d40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:21.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.576+0000 7fd9d159d700 1 -- 172.21.15.31:0/3492459277 >> [v2:172.21.15.31:6815/85584,v1:172.21.15.31:6817/85584] conn(0x7fd9b006f8b0 msgr2=0x7fd9b0071cc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:21.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.576+0000 7f65dd7fa700 1 -- 172.21.15.31:0/2958606099 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f65d8067080 msgr2=0x7f65d8069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:21.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.576+0000 7f65dd7fa700 1 --2- 172.21.15.31:0/2958606099 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f65d8067080 0x7f65d8069530 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f65e8008dc0 tx=0x7f65e8008cb0 comp rx=0 tx=0).stop 2023-10-06T02:23:21.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.576+0000 7fe9ca5ff700 1 -- 172.21.15.31:0/3222514863 --> [v2:172.21.15.62:6808/85798,v1:172.21.15.62:6809/85798] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe9c40ac030 con 0x7fe9a4073d40 2023-10-06T02:23:21.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.576+0000 7f65dd7fa700 1 -- 172.21.15.31:0/2958606099 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65ec0d4b10 msgr2=0x7f65ec039180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:21.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.576+0000 7f65dd7fa700 1 --2- 172.21.15.31:0/2958606099 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65ec0d4b10 0x7f65ec039180 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f65f40669b0 tx=0x7f65f406d000 comp rx=0 tx=0).stop 2023-10-06T02:23:21.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.576+0000 7fd9d159d700 1 --2- 172.21.15.31:0/3492459277 >> [v2:172.21.15.31:6815/85584,v1:172.21.15.31:6817/85584] conn(0x7fd9b006f8b0 0x7fd9b0071cc0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:21.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.576+0000 7f65f3fff700 1 -- 172.21.15.31:0/2958606099 reap_dead start 2023-10-06T02:23:21.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.576+0000 7f65dd7fa700 1 -- 172.21.15.31:0/2958606099 shutdown_connections 2023-10-06T02:23:21.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.576+0000 7f65dd7fa700 1 -- 172.21.15.31:0/2958606099 >> 172.21.15.31:0/2958606099 conn(0x7f65ec0517b0 msgr2=0x7f65ec0057f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:21.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.577+0000 7f6af9dff700 1 -- 172.21.15.31:0/1058530425 --> [v2:172.21.15.62:6816/85800,v1:172.21.15.62:6817/85800] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6af40ccc10 con 0x7f6ae006f8b0 2023-10-06T02:23:21.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.577+0000 7fe9c0ff9700 1 -- 172.21.15.31:0/3222514863 <== osd.6 v2:172.21.15.62:6808/85798 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe9c40ac030 con 0x7fe9a4073d40 2023-10-06T02:23:21.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.577+0000 7f65dd7fa700 1 -- 172.21.15.31:0/2958606099 shutdown_connections 2023-10-06T02:23:21.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.577+0000 7f65dd7fa700 1 -- 172.21.15.31:0/2958606099 wait complete. 2023-10-06T02:23:21.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.577+0000 7fd9d159d700 1 -- 172.21.15.31:0/3492459277 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fd9b0067080 msgr2=0x7fd9b0069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:21.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.577+0000 7fd9d159d700 1 --2- 172.21.15.31:0/3492459277 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fd9b0067080 0x7fd9b0069530 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd9c4011ff0 tx=0x7fd9cc068000 comp rx=0 tx=0).stop 2023-10-06T02:23:21.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.577+0000 7fd9d159d700 1 -- 172.21.15.31:0/3492459277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9c4006b50 msgr2=0x7fd9c40da1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:21.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.577+0000 7fd9d159d700 1 --2- 172.21.15.31:0/3492459277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9c4006b50 0x7fd9c40da1a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd9b800b5e0 tx=0x7fd9b8009120 comp rx=0 tx=0).stop 2023-10-06T02:23:21.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.577+0000 7fd9d0d9c700 1 -- 172.21.15.31:0/3492459277 reap_dead start 2023-10-06T02:23:21.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.577+0000 7f6adb7fe700 1 -- 172.21.15.31:0/1058530425 <== osd.5 v2:172.21.15.62:6816/85800 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f6af40ccc10 con 0x7f6ae006f8b0 2023-10-06T02:23:21.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.578+0000 7fe9b9ffb700 1 -- 172.21.15.31:0/3222514863 >> [v2:172.21.15.62:6808/85798,v1:172.21.15.62:6809/85798] conn(0x7fe9a4073d40 msgr2=0x7fe9a4076150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:21.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.578+0000 7fd9d159d700 1 -- 172.21.15.31:0/3492459277 shutdown_connections 2023-10-06T02:23:21.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.578+0000 7fd9d159d700 1 -- 172.21.15.31:0/3492459277 >> 172.21.15.31:0/3492459277 conn(0x7fd9c4055ba0 msgr2=0x7fd9c404f9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:21.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.578+0000 7fe9b9ffb700 1 --2- 172.21.15.31:0/3222514863 >> [v2:172.21.15.62:6808/85798,v1:172.21.15.62:6809/85798] conn(0x7fe9a4073d40 0x7fe9a4076150 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:21.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.578+0000 7fd9d159d700 1 -- 172.21.15.31:0/3492459277 shutdown_connections 2023-10-06T02:23:21.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.578+0000 7fd9d159d700 1 -- 172.21.15.31:0/3492459277 wait complete. 2023-10-06T02:23:21.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.578+0000 7f6168bff700 1 -- 172.21.15.31:0/729517814 --> [v2:172.21.15.62:6800/85799,v1:172.21.15.62:6801/85799] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f61640cd1e0 con 0x7f614406f930 2023-10-06T02:23:21.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.579+0000 7f6ad97fa700 1 -- 172.21.15.31:0/1058530425 >> [v2:172.21.15.62:6816/85800,v1:172.21.15.62:6817/85800] conn(0x7f6ae006f8b0 msgr2=0x7f6ae0071cc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:21.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.579+0000 7f6ad97fa700 1 --2- 172.21.15.31:0/1058530425 >> [v2:172.21.15.62:6816/85800,v1:172.21.15.62:6817/85800] conn(0x7f6ae006f8b0 0x7f6ae0071cc0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:21.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.579+0000 7fe9b9ffb700 1 -- 172.21.15.31:0/3222514863 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe9a406b490 msgr2=0x7fe9a406d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:21.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.579+0000 7fe9b9ffb700 1 --2- 172.21.15.31:0/3222514863 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe9a406b490 0x7fe9a406d940 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe9b4000f80 tx=0x7fe9b4018000 comp rx=0 tx=0).stop 2023-10-06T02:23:21.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.579+0000 7f61527fc700 1 -- 172.21.15.31:0/729517814 <== osd.4 v2:172.21.15.62:6800/85799 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f61640cd1e0 con 0x7f614406f930 2023-10-06T02:23:21.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.579+0000 7fe9b9ffb700 1 -- 172.21.15.31:0/3222514863 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9c40cb8e0 msgr2=0x7fe9c414b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:21.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.579+0000 7f9205dff700 1 -- 172.21.15.31:0/326909324 --> [v2:172.21.15.62:6824/85843,v1:172.21.15.62:6825/85843] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9200158850 con 0x7f91ec06f8b0 2023-10-06T02:23:21.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.579+0000 7f6ad97fa700 1 -- 172.21.15.31:0/1058530425 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6ae0067000 msgr2=0x7f6ae00694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:21.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.579+0000 7f6ad97fa700 1 --2- 172.21.15.31:0/1058530425 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6ae0067000 0x7f6ae00694b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6ae40058d0 tx=0x7f6ae4009040 comp rx=0 tx=0).stop 2023-10-06T02:23:21.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.579+0000 7fe9b9ffb700 1 --2- 172.21.15.31:0/3222514863 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9c40cb8e0 0x7fe9c414b030 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe9c406ef00 tx=0x7fe9bc008040 comp rx=0 tx=0).stop 2023-10-06T02:23:21.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.580+0000 7f6ad97fa700 1 -- 172.21.15.31:0/1058530425 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6af4155b20 msgr2=0x7f6af4151410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:21.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.580+0000 7f6ad97fa700 1 --2- 172.21.15.31:0/1058530425 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6af4155b20 0x7f6af4151410 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6aec00c8e0 tx=0x7f6aec001710 comp rx=0 tx=0).stop 2023-10-06T02:23:21.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.580+0000 7f6168bff700 1 -- 172.21.15.31:0/729517814 >> [v2:172.21.15.62:6800/85799,v1:172.21.15.62:6801/85799] conn(0x7f614406f930 msgr2=0x7f6144071d40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:21.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.580+0000 7f6af3fff700 1 -- 172.21.15.31:0/1058530425 reap_dead start 2023-10-06T02:23:21.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.580+0000 7f6168bff700 1 --2- 172.21.15.31:0/729517814 >> [v2:172.21.15.62:6800/85799,v1:172.21.15.62:6801/85799] conn(0x7f614406f930 0x7f6144071d40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:21.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.580+0000 7f6ad97fa700 1 -- 172.21.15.31:0/1058530425 shutdown_connections 2023-10-06T02:23:21.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.580+0000 7f6ad97fa700 1 -- 172.21.15.31:0/1058530425 >> 172.21.15.31:0/1058530425 conn(0x7f6af406d170 msgr2=0x7f6af406e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:21.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.580+0000 7fe9c8b9c700 1 -- 172.21.15.31:0/3222514863 reap_dead start 2023-10-06T02:23:21.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.580+0000 7f6ad97fa700 1 -- 172.21.15.31:0/1058530425 shutdown_connections 2023-10-06T02:23:21.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.580+0000 7f6168bff700 1 -- 172.21.15.31:0/729517814 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6144067080 msgr2=0x7f6144069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:21.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.580+0000 7f6168bff700 1 --2- 172.21.15.31:0/729517814 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6144067080 0x7f6144069530 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f615c009880 tx=0x7f615c017000 comp rx=0 tx=0).stop 2023-10-06T02:23:21.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.580+0000 7f6ad97fa700 1 -- 172.21.15.31:0/1058530425 wait complete. 2023-10-06T02:23:21.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.580+0000 7f91e77fe700 1 -- 172.21.15.31:0/326909324 <== osd.7 v2:172.21.15.62:6824/85843 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f91ec072370 con 0x7f91ec06f8b0 2023-10-06T02:23:21.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.580+0000 7fe9b9ffb700 1 -- 172.21.15.31:0/3222514863 shutdown_connections 2023-10-06T02:23:21.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.581+0000 7f6168bff700 1 -- 172.21.15.31:0/729517814 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f61640cb5a0 msgr2=0x7f616414d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:21.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.581+0000 7f6168bff700 1 --2- 172.21.15.31:0/729517814 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f61640cb5a0 0x7f616414d1b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f615800b600 tx=0x7f61580091a0 comp rx=0 tx=0).stop 2023-10-06T02:23:21.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.581+0000 7f6162d9d700 1 -- 172.21.15.31:0/729517814 reap_dead start 2023-10-06T02:23:21.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.581+0000 7f9205dff700 1 -- 172.21.15.31:0/326909324 >> [v2:172.21.15.62:6824/85843,v1:172.21.15.62:6825/85843] conn(0x7f91ec06f8b0 msgr2=0x7f91ec071cc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:21.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.581+0000 7f9205dff700 1 --2- 172.21.15.31:0/326909324 >> [v2:172.21.15.62:6824/85843,v1:172.21.15.62:6825/85843] conn(0x7f91ec06f8b0 0x7f91ec071cc0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:21.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.581+0000 7f6168bff700 1 -- 172.21.15.31:0/729517814 shutdown_connections 2023-10-06T02:23:21.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.581+0000 7f6168bff700 1 -- 172.21.15.31:0/729517814 >> 172.21.15.31:0/729517814 conn(0x7f6164070e30 msgr2=0x7f61640b2410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:21.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.582+0000 7f6168bff700 1 -- 172.21.15.31:0/729517814 shutdown_connections 2023-10-06T02:23:21.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.582+0000 7f6168bff700 1 -- 172.21.15.31:0/729517814 wait complete. 2023-10-06T02:23:21.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.581+0000 7fe9b9ffb700 1 -- 172.21.15.31:0/3222514863 >> 172.21.15.31:0/3222514863 conn(0x7fe9c40cc300 msgr2=0x7fe9c40638f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:21.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.581+0000 7f9205dff700 1 -- 172.21.15.31:0/326909324 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f91ec067000 msgr2=0x7f91ec0694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:21.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.582+0000 7f9205dff700 1 --2- 172.21.15.31:0/326909324 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f91ec067000 0x7f91ec0694b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f91f8009ab0 tx=0x7f91f8018000 comp rx=0 tx=0).stop 2023-10-06T02:23:21.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.582+0000 7f9205dff700 1 -- 172.21.15.31:0/326909324 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92000cd9e0 msgr2=0x7f92000b4800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:21.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.582+0000 7f9205dff700 1 --2- 172.21.15.31:0/326909324 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92000cd9e0 0x7f92000b4800 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f91f000c790 tx=0x7f91f0007f40 comp rx=0 tx=0).stop 2023-10-06T02:23:21.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.582+0000 7f91fffff700 1 -- 172.21.15.31:0/326909324 reap_dead start 2023-10-06T02:23:21.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.582+0000 7f9205dff700 1 -- 172.21.15.31:0/326909324 shutdown_connections 2023-10-06T02:23:21.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.582+0000 7f9205dff700 1 -- 172.21.15.31:0/326909324 >> 172.21.15.31:0/326909324 conn(0x7f9200070e30 msgr2=0x7f92000b2720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:21.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.582+0000 7f9205dff700 1 -- 172.21.15.31:0/326909324 shutdown_connections 2023-10-06T02:23:21.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.582+0000 7f9205dff700 1 -- 172.21.15.31:0/326909324 wait complete. 2023-10-06T02:23:21.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.583+0000 7fe9b9ffb700 1 -- 172.21.15.31:0/3222514863 shutdown_connections 2023-10-06T02:23:21.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.583+0000 7fe9b9ffb700 1 -- 172.21.15.31:0/3222514863 wait complete. 2023-10-06T02:23:21.632 INFO:teuthology.orchestra.run.smithi031.stdout:55834574858 2023-10-06T02:23:21.632 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-06T02:23:21.633 INFO:teuthology.orchestra.run.smithi031.stdout:55834574857 2023-10-06T02:23:21.633 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-06T02:23:21.634 INFO:teuthology.orchestra.run.smithi031.stdout:55834574856 2023-10-06T02:23:21.634 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-06T02:23:21.635 INFO:teuthology.orchestra.run.smithi031.stdout:55834574855 2023-10-06T02:23:21.635 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-06T02:23:21.636 INFO:teuthology.orchestra.run.smithi031.stdout:55834574857 2023-10-06T02:23:21.636 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-06T02:23:21.637 INFO:teuthology.orchestra.run.smithi031.stdout:55834574857 2023-10-06T02:23:21.637 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-06T02:23:21.638 INFO:teuthology.orchestra.run.smithi031.stdout:55834574857 2023-10-06T02:23:21.638 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-06T02:23:21.639 INFO:teuthology.orchestra.run.smithi031.stdout:55834574857 2023-10-06T02:23:21.639 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-06T02:23:21.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.996+0000 7f384679d700 1 Processor -- start 2023-10-06T02:23:21.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.997+0000 7f384679d700 1 -- start start 2023-10-06T02:23:21.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.999+0000 7f384679d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f38380dbd10 0x7f3838005f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:22.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.999+0000 7f384679d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f38380dc0e0 0x7f3838033a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:22.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.999+0000 7f384679d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3838033fd0 0x7f38380de980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:22.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.999+0000 7f384679d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3838053310 con 0x7f3838033fd0 2023-10-06T02:23:22.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.999+0000 7f384679d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3838053bb0 con 0x7f38380dbd10 2023-10-06T02:23:22.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:21.999+0000 7f384679d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38380343f0 con 0x7f38380dc0e0 2023-10-06T02:23:22.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.000+0000 7f3845f9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3838033fd0 0x7f38380de980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:22.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.000+0000 7f3845f9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3838033fd0 0x7f38380de980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37568/0 (socket says 172.21.15.31:37568) 2023-10-06T02:23:22.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.000+0000 7f3845f9c700 1 -- 172.21.15.31:0/1247566039 learned_addr learned my addr 172.21.15.31:0/1247566039 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:22.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.001+0000 7f3845f9c700 1 -- 172.21.15.31:0/1247566039 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f38380dc0e0 msgr2=0x7f3838033a90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T02:23:22.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.001+0000 7f3845f9c700 1 --2- 172.21.15.31:0/1247566039 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f38380dc0e0 0x7f3838033a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:22.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.001+0000 7f3845f9c700 1 -- 172.21.15.31:0/1247566039 >> v2:172.21.15.62:3300/0 conn(0x7f38380dbd10 msgr2=0x7f3838005f10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T02:23:22.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.001+0000 7f3845f9c700 1 --2- 172.21.15.31:0/1247566039 >> v2:172.21.15.62:3300/0 conn(0x7f38380dbd10 0x7f3838005f10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:22.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.001+0000 7f3845f9c700 1 -- 172.21.15.31:0/1247566039 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38380df6b0 con 0x7f3838033fd0 2023-10-06T02:23:22.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.002+0000 7f3845f9c700 1 --2- 172.21.15.31:0/1247566039 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3838033fd0 0x7f38380de980 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f383800d550 tx=0x7f3830009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a956a274482f0c6b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:22.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.002+0000 7f3837fff700 1 -- 172.21.15.31:0/1247566039 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f383000b7d0 con 0x7f3838033fd0 2023-10-06T02:23:22.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.003+0000 7f3837fff700 1 -- 172.21.15.31:0/1247566039 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f383000b930 con 0x7f3838033fd0 2023-10-06T02:23:22.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.004+0000 7f384679d700 1 -- 172.21.15.31:0/1247566039 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3838033fd0 msgr2=0x7f38380de980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:22.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.004+0000 7f384679d700 1 --2- 172.21.15.31:0/1247566039 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3838033fd0 0x7f38380de980 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f383800d550 tx=0x7f3830009250 comp rx=0 tx=0).stop 2023-10-06T02:23:22.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.004+0000 7f384679d700 1 -- 172.21.15.31:0/1247566039 shutdown_connections 2023-10-06T02:23:22.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.004+0000 7f384679d700 1 --2- 172.21.15.31:0/1247566039 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3838033fd0 0x7f38380de980 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:22.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.004+0000 7f384679d700 1 --2- 172.21.15.31:0/1247566039 >> v2:172.21.15.62:3300/0 conn(0x7f38380dbd10 0x7f3838005f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:22.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.004+0000 7f384679d700 1 --2- 172.21.15.31:0/1247566039 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f38380dc0e0 0x7f3838033a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:22.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.004+0000 7f384679d700 1 -- 172.21.15.31:0/1247566039 >> 172.21.15.31:0/1247566039 conn(0x7f38380517b0 msgr2=0x7f3838051bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:22.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.004+0000 7f384679d700 1 -- 172.21.15.31:0/1247566039 shutdown_connections 2023-10-06T02:23:22.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.004+0000 7f384679d700 1 -- 172.21.15.31:0/1247566039 wait complete. 2023-10-06T02:23:22.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.007+0000 7f384679d700 1 Processor -- start 2023-10-06T02:23:22.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.007+0000 7f384679d700 1 -- start start 2023-10-06T02:23:22.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.008+0000 7fe1c719d700 1 Processor -- start 2023-10-06T02:23:22.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.009+0000 7f384679d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3838033fd0 0x7f38380359e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:22.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.009+0000 7f384679d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38380dbd10 0x7f383804c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:22.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.009+0000 7f384679d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f383804da60 0x7f383804dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:22.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.010+0000 7f3844f9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38380dbd10 0x7f383804c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:22.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.010+0000 7f3844f9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38380dbd10 0x7f383804c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:47328/0 (socket says 172.21.15.31:47328) 2023-10-06T02:23:22.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.010+0000 7f384579b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3838033fd0 0x7f38380359e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:22.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.010+0000 7f384579b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3838033fd0 0x7f38380359e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:57108/0 (socket says 172.21.15.31:57108) 2023-10-06T02:23:22.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.010+0000 7f3845f9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f383804da60 0x7f383804dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:22.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.010+0000 7f3845f9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f383804da60 0x7f383804dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37570/0 (socket says 172.21.15.31:37570) 2023-10-06T02:23:22.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.009+0000 7fe1c719d700 1 -- start start 2023-10-06T02:23:22.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.010+0000 7f3844f9a700 1 -- 172.21.15.31:0/3321121793 learned_addr learned my addr 172.21.15.31:0/3321121793 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:22.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.010+0000 7f384679d700 1 -- 172.21.15.31:0/3321121793 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f383804c540 con 0x7f383804da60 2023-10-06T02:23:22.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.010+0000 7f384679d700 1 -- 172.21.15.31:0/3321121793 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3838037ef0 con 0x7f38380dbd10 2023-10-06T02:23:22.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.010+0000 7f384679d700 1 -- 172.21.15.31:0/3321121793 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3838036000 con 0x7f3838033fd0 2023-10-06T02:23:22.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.011+0000 7f3845f9c700 1 -- 172.21.15.31:0/3321121793 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3838033fd0 msgr2=0x7f38380359e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:22.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.011+0000 7fe1c719d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe1b80df180 0x7fe1b8050e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:22.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.011+0000 7fe1c719d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe1b8051340 0x7fe1b80551a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:22.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.011+0000 7fe1c719d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1b8008910 con 0x7fe1b80dedb0 2023-10-06T02:23:22.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.011+0000 7fe1c719d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1b80517c0 con 0x7fe1b8051340 2023-10-06T02:23:22.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.011+0000 7fe1c719d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b80094b0 con 0x7fe1b80df180 2023-10-06T02:23:22.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.011+0000 7f3845f9c700 1 --2- 172.21.15.31:0/3321121793 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3838033fd0 0x7f38380359e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:22.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.011+0000 7f3845f9c700 1 -- 172.21.15.31:0/3321121793 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38380dbd10 msgr2=0x7f383804c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:22.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.011+0000 7f3845f9c700 1 --2- 172.21.15.31:0/3321121793 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38380dbd10 0x7f383804c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:22.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.011+0000 7f3845f9c700 1 -- 172.21.15.31:0/3321121793 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38300094f0 con 0x7f383804da60 2023-10-06T02:23:22.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.012+0000 7f3845f9c700 1 --2- 172.21.15.31:0/3321121793 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f383804da60 0x7f383804dee0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f383800e090 tx=0x7f3830017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:22.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.012+0000 7fe1c619b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fe1b80dedb0 0x7fe1b8008da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49272/0 (socket says 172.21.15.31:49272) 2023-10-06T02:23:22.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.012+0000 7fe1c619b700 1 -- 172.21.15.31:0/832321840 learned_addr learned my addr 172.21.15.31:0/832321840 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:23:22.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.012+0000 7f38357fa700 1 -- 172.21.15.31:0/3321121793 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f383001a630 con 0x7f383804da60 2023-10-06T02:23:22.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.013+0000 7fe1c699c700 1 --2- 172.21.15.31:0/832321840 >> v2:172.21.15.62:3300/0 conn(0x7fe1b8051340 0x7fe1b80551a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:22.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.013+0000 7fe1c5199700 1 -- 172.21.15.31:0/832321840 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1187730454 0 0) 0x7fe1b8008910 con 0x7fe1b80dedb0 2023-10-06T02:23:22.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.013+0000 7fe1c5199700 1 -- 172.21.15.31:0/832321840 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1ac003570 con 0x7fe1b80dedb0 2023-10-06T02:23:22.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.013+0000 7fe1c599a700 1 --2- 172.21.15.31:0/832321840 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe1b80df180 0x7fe1b8050e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:22.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.014+0000 7fe1c599a700 1 -- 172.21.15.31:0/832321840 >> v2:172.21.15.62:3300/0 conn(0x7fe1b8051340 msgr2=0x7fe1b80551a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:22.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.014+0000 7fe1c599a700 1 --2- 172.21.15.31:0/832321840 >> v2:172.21.15.62:3300/0 conn(0x7fe1b8051340 0x7fe1b80551a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:22.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.014+0000 7fe1c599a700 1 -- 172.21.15.31:0/832321840 >> v1:172.21.15.31:6789/0 conn(0x7fe1b80dedb0 legacy=0x7fe1b8008da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:22.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.014+0000 7fe1c599a700 1 -- 172.21.15.31:0/832321840 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1b8007d00 con 0x7fe1b80df180 2023-10-06T02:23:22.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.014+0000 7fe1c599a700 1 --2- 172.21.15.31:0/832321840 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe1b80df180 0x7fe1b8050e00 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc00bb90 tx=0x7fe1bc00b430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77042150843f9518 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:22.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.015+0000 7fe1c5199700 1 -- 172.21.15.31:0/832321840 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1bc00da70 con 0x7fe1b80df180 2023-10-06T02:23:22.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.015+0000 7fe1c5199700 1 -- 172.21.15.31:0/832321840 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1bc00dbd0 con 0x7fe1b80df180 2023-10-06T02:23:22.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.015+0000 7fe1c5199700 1 -- 172.21.15.31:0/832321840 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1bc018640 con 0x7fe1b80df180 2023-10-06T02:23:22.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.013+0000 7f384679d700 1 -- 172.21.15.31:0/3321121793 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38380362e0 con 0x7f383804da60 2023-10-06T02:23:22.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.013+0000 7f384679d700 1 -- 172.21.15.31:0/3321121793 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38380368a0 con 0x7f383804da60 2023-10-06T02:23:22.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.014+0000 7f38357fa700 1 -- 172.21.15.31:0/3321121793 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f383001a790 con 0x7f383804da60 2023-10-06T02:23:22.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.014+0000 7f38357fa700 1 -- 172.21.15.31:0/3321121793 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f383001ab30 con 0x7f383804da60 2023-10-06T02:23:22.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.016+0000 7fe1c719d700 1 -- 172.21.15.31:0/832321840 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe1b80df180 msgr2=0x7fe1b8050e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:22.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.016+0000 7fe1c719d700 1 --2- 172.21.15.31:0/832321840 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe1b80df180 0x7fe1b8050e00 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc00bb90 tx=0x7fe1bc00b430 comp rx=0 tx=0).stop 2023-10-06T02:23:22.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.017+0000 7fe1c719d700 1 -- 172.21.15.31:0/832321840 shutdown_connections 2023-10-06T02:23:22.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.017+0000 7fe1c719d700 1 --2- 172.21.15.31:0/832321840 >> v2:172.21.15.62:3300/0 conn(0x7fe1b8051340 0x7fe1b80551a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:22.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.017+0000 7fe1c719d700 1 --2- 172.21.15.31:0/832321840 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe1b80df180 0x7fe1b8050e00 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:22.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.017+0000 7fe1c719d700 1 -- 172.21.15.31:0/832321840 >> 172.21.15.31:0/832321840 conn(0x7fe1b80528c0 msgr2=0x7fe1b8052cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:22.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.016+0000 7f384679d700 1 -- 172.21.15.31:0/3321121793 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38200052b0 con 0x7f383804da60 2023-10-06T02:23:22.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.018+0000 7f2251bff700 1 Processor -- start 2023-10-06T02:23:22.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.019+0000 7f2251bff700 1 -- start start 2023-10-06T02:23:22.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.019+0000 7f38357fa700 1 -- 172.21.15.31:0/3321121793 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f383002e450 con 0x7f383804da60 2023-10-06T02:23:22.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.020+0000 7f38357fa700 1 --2- 172.21.15.31:0/3321121793 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f3824067080 0x7f3824069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:22.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.019+0000 7f9ee01ff700 1 Processor -- start 2023-10-06T02:23:22.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.019+0000 7f9ee01ff700 1 -- start start 2023-10-06T02:23:22.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.018+0000 7fe1c719d700 1 -- 172.21.15.31:0/832321840 shutdown_connections 2023-10-06T02:23:22.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.021+0000 7f2251bff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f224c0cb970 0x7f224c159ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:22.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.022+0000 7f2251bff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f224c15ae40 con 0x7f224c0cb5a0 2023-10-06T02:23:22.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.021+0000 7f9ee01ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9ed815dc00 0x7f9ed80af7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:22.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.022+0000 7f9ee01ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9ed80cb460 0x7f9ed80b2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:22.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.022+0000 7f9ee01ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ed80cde10 con 0x7f9ed80b27a0 2023-10-06T02:23:22.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.022+0000 7f9ee01ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ed80ce410 con 0x7f9ed815dc00 2023-10-06T02:23:22.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.022+0000 7f9ee01ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed80ce6f0 con 0x7f9ed80cb460 2023-10-06T02:23:22.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.022+0000 7f2251bff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f224c15ab60 con 0x7f224c15a400 2023-10-06T02:23:22.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.022+0000 7f2251bff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f224c0ce0f0 con 0x7f224c0cb970 2023-10-06T02:23:22.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.022+0000 7f224bfff700 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f224c15a400 0x7f224c156e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:33944/0 (socket says 172.21.15.31:33944) 2023-10-06T02:23:22.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.022+0000 7f224bfff700 1 -- 172.21.15.31:0/2258491497 learned_addr learned my addr 172.21.15.31:0/2258491497 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:23:22.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.022+0000 7f9ede79c700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9ed80cb460 0x7f9ed80b2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:22.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.022+0000 7f9ede79c700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9ed80cb460 0x7f9ed80b2c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:57114/0 (socket says 172.21.15.31:57114) 2023-10-06T02:23:22.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.022+0000 7f9ede79c700 1 -- 172.21.15.31:0/1786992288 learned_addr learned my addr 172.21.15.31:0/1786992288 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:22.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.022+0000 7fa262b9d700 1 Processor -- start 2023-10-06T02:23:22.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.023+0000 7fa262b9d700 1 -- start start 2023-10-06T02:23:22.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.023+0000 7f9ede79c700 1 -- 172.21.15.31:0/1786992288 >> v2:172.21.15.62:3300/0 conn(0x7f9ed815dc00 msgr2=0x7f9ed80af7d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T02:23:22.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.023+0000 7f9ede79c700 1 --2- 172.21.15.31:0/1786992288 >> v2:172.21.15.62:3300/0 conn(0x7f9ed815dc00 0x7f9ed80af7d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:22.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.023+0000 7f9ede79c700 1 -- 172.21.15.31:0/1786992288 >> v1:172.21.15.62:6789/0 conn(0x7f9ed80b27a0 legacy=0x7f9ed80cad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:22.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.023+0000 7f9ede79c700 1 -- 172.21.15.31:0/1786992288 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ed80ce9d0 con 0x7f9ed80cb460 2023-10-06T02:23:22.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.022+0000 7f224affd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f224c0cb970 0x7f224c159ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:22.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.021+0000 7fe1c719d700 1 -- 172.21.15.31:0/832321840 wait complete. 2023-10-06T02:23:22.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.023+0000 7fe1c719d700 1 Processor -- start 2023-10-06T02:23:22.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.024+0000 7fe1c719d700 1 -- start start 2023-10-06T02:23:22.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.024+0000 7f224affd700 1 -- 172.21.15.31:0/2258491497 >> v1:172.21.15.62:6789/0 conn(0x7f224c15a400 legacy=0x7f224c156e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:22.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.024+0000 7f224affd700 1 -- 172.21.15.31:0/2258491497 >> v1:172.21.15.31:6789/0 conn(0x7f224c0cb5a0 legacy=0x7f224c0cd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:22.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.024+0000 7f224affd700 1 -- 172.21.15.31:0/2258491497 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f224c0ccf00 con 0x7f224c0cb970 2023-10-06T02:23:22.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.021+0000 7f384579b700 1 --2- 172.21.15.31:0/3321121793 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f3824067080 0x7f3824069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:22.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.024+0000 7f38357fa700 1 -- 172.21.15.31:0/3321121793 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3830085fe0 con 0x7f383804da60 2023-10-06T02:23:22.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.025+0000 7f224affd700 1 --2- 172.21.15.31:0/2258491497 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f224c0cb970 0x7f224c159ec0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f223c009de0 tx=0x7f223c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8260b8e92ebfc5c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:22.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.025+0000 7f9ede79c700 1 --2- 172.21.15.31:0/1786992288 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9ed80cb460 0x7f9ed80b2c40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0009f20 tx=0x7f9ed0009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac291cd34da50bc6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:22.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.025+0000 7fa262b9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa25c06ef30 0x7fa25c0a98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:22.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.025+0000 7fa262b9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa25c16dff0 0x7fa25c0a9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:22.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.025+0000 7fa262b9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa25c0c6950 con 0x7fa25c0ca690 2023-10-06T02:23:22.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.025+0000 7fa262b9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa25c0ab820 con 0x7fa25c06ef30 2023-10-06T02:23:22.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.025+0000 7fa262b9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa25c0ab260 con 0x7fa25c16dff0 2023-10-06T02:23:22.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.026+0000 7fe1c719d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe1b8051340 0x7fe1b80dd830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:22.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.026+0000 7fe1c719d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1b80dedb0 0x7fe1b80db220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:22.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.026+0000 7fe1c719d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1b80df180 0x7fe1b80549d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:22.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.026+0000 7fe1c719d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b80ee1d0 con 0x7fe1b80dedb0 2023-10-06T02:23:22.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.026+0000 7fa261b9b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fa25c0ca690 0x7fa25c16d8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49276/0 (socket says 172.21.15.31:49276) 2023-10-06T02:23:22.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.026+0000 7fa261b9b700 1 -- 172.21.15.31:0/3639577362 learned_addr learned my addr 172.21.15.31:0/3639577362 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:23:22.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.026+0000 7fa26239c700 1 --2- 172.21.15.31:0/3639577362 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa25c16dff0 0x7fa25c0a9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:22.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.026+0000 7fe1c719d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b804e550 con 0x7fe1b80df180 2023-10-06T02:23:22.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.026+0000 7fe1c719d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b804e830 con 0x7fe1b8051340 2023-10-06T02:23:22.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.026+0000 7f384579b700 1 --2- 172.21.15.31:0/3321121793 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f3824067080 0x7f3824069530 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f384004d1b0 tx=0x7f384006b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:22.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.026+0000 7fa26139a700 1 --2- 172.21.15.31:0/3639577362 >> v2:172.21.15.31:3300/0 conn(0x7fa25c06ef30 0x7fa25c0a98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:22.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.027+0000 7fa26239c700 1 -- 172.21.15.31:0/3639577362 >> v2:172.21.15.31:3300/0 conn(0x7fa25c06ef30 msgr2=0x7fa25c0a98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:22.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.027+0000 7fe1c619b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe1b8051340 0x7fe1b80dd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:22.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.027+0000 7fe1c619b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe1b8051340 0x7fe1b80dd830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:57126/0 (socket says 172.21.15.31:57126) 2023-10-06T02:23:22.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.027+0000 7fa26239c700 1 --2- 172.21.15.31:0/3639577362 >> v2:172.21.15.31:3300/0 conn(0x7fa25c06ef30 0x7fa25c0a98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:22.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.027+0000 7fa26239c700 1 -- 172.21.15.31:0/3639577362 >> v1:172.21.15.31:6789/0 conn(0x7fa25c0ca690 legacy=0x7fa25c16d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:22.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.027+0000 7fa26239c700 1 -- 172.21.15.31:0/3639577362 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa25c0ab540 con 0x7fa25c16dff0 2023-10-06T02:23:22.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.027+0000 7fe1c619b700 1 -- 172.21.15.31:0/703424535 learned_addr learned my addr 172.21.15.31:0/703424535 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:22.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.027+0000 7fa26139a700 1 --2- 172.21.15.31:0/3639577362 >> v2:172.21.15.31:3300/0 conn(0x7fa25c06ef30 0x7fa25c0a98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:23:22.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.027+0000 7fe1c599a700 1 --2- 172.21.15.31:0/703424535 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1b80dedb0 0x7fe1b80db220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:22.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.027+0000 7fe1c619b700 1 -- 172.21.15.31:0/703424535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1b80df180 msgr2=0x7fe1b80549d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:22.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.027+0000 7fe1c699c700 1 --2- 172.21.15.31:0/703424535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1b80df180 0x7fe1b80549d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:22.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.027+0000 7f9edcf99700 1 -- 172.21.15.31:0/1786992288 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed000e070 con 0x7f9ed80cb460 2023-10-06T02:23:22.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.027+0000 7f9edcf99700 1 -- 172.21.15.31:0/1786992288 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ed000ba40 con 0x7f9ed80cb460 2023-10-06T02:23:22.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.027+0000 7f224a7fc700 1 -- 172.21.15.31:0/2258491497 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f223c00e070 con 0x7f224c0cb970 2023-10-06T02:23:22.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.027+0000 7fa26239c700 1 --2- 172.21.15.31:0/3639577362 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa25c16dff0 0x7fa25c0a9e20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa250009e60 tx=0x7fa250009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a82867ed48eef17 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:22.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.027+0000 7fe1c619b700 1 --2- 172.21.15.31:0/703424535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1b80df180 0x7fe1b80549d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:22.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.027+0000 7fe1c619b700 1 -- 172.21.15.31:0/703424535 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1b80dedb0 msgr2=0x7fe1b80db220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:22.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.027+0000 7fe1c619b700 1 --2- 172.21.15.31:0/703424535 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1b80dedb0 0x7fe1b80db220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:22.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.027+0000 7fe1c619b700 1 -- 172.21.15.31:0/703424535 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1bc00b6d0 con 0x7fe1b8051340 2023-10-06T02:23:22.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.027+0000 7fe1c699c700 1 --2- 172.21.15.31:0/703424535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1b80df180 0x7fe1b80549d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:23:22.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.028+0000 7fe1c619b700 1 --2- 172.21.15.31:0/703424535 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe1b8051340 0x7fe1b80dd830 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe1b000c7f0 tx=0x7fe1b0007e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:22.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.028+0000 7f38357fa700 1 -- 172.21.15.31:0/3321121793 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f38300530d0 con 0x7f383804da60 2023-10-06T02:23:22.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.028+0000 7f1cdb5ff700 1 Processor -- start 2023-10-06T02:23:22.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.028+0000 7fa260b99700 1 -- 172.21.15.31:0/3639577362 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa25000e070 con 0x7fa25c16dff0 2023-10-06T02:23:22.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.028+0000 7fa260b99700 1 -- 172.21.15.31:0/3639577362 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa25000bcc0 con 0x7fa25c16dff0 2023-10-06T02:23:22.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.028+0000 7fa260b99700 1 -- 172.21.15.31:0/3639577362 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa250012680 con 0x7fa25c16dff0 2023-10-06T02:23:22.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.028+0000 7f9ee01ff700 1 -- 172.21.15.31:0/1786992288 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9ed80cb460 msgr2=0x7f9ed80b2c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:22.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.028+0000 7f9ee01ff700 1 --2- 172.21.15.31:0/1786992288 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9ed80cb460 0x7f9ed80b2c40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0009f20 tx=0x7f9ed0009840 comp rx=0 tx=0).stop 2023-10-06T02:23:22.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.028+0000 7f1cdb5ff700 1 -- start start 2023-10-06T02:23:22.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.029+0000 7fa262b9d700 1 -- 172.21.15.31:0/3639577362 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa25c16dff0 msgr2=0x7fa25c0a9e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:22.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.029+0000 7f224a7fc700 1 -- 172.21.15.31:0/2258491497 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f223c00bbb0 con 0x7f224c0cb970 2023-10-06T02:23:22.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.029+0000 7f224a7fc700 1 -- 172.21.15.31:0/2258491497 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f223c012430 con 0x7f224c0cb970 2023-10-06T02:23:22.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.029+0000 7f9ee01ff700 1 -- 172.21.15.31:0/1786992288 shutdown_connections 2023-10-06T02:23:22.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.029+0000 7f9ee01ff700 1 --2- 172.21.15.31:0/1786992288 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9ed80cb460 0x7f9ed80b2c40 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:22.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.029+0000 7f9ee01ff700 1 --2- 172.21.15.31:0/1786992288 >> v2:172.21.15.62:3300/0 conn(0x7f9ed815dc00 0x7f9ed80af7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:22.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.029+0000 7f9ee01ff700 1 -- 172.21.15.31:0/1786992288 >> 172.21.15.31:0/1786992288 conn(0x7f9ed8070e30 msgr2=0x7f9ed80d0710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:22.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.029+0000 7fa262b9d700 1 --2- 172.21.15.31:0/3639577362 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa25c16dff0 0x7fa25c0a9e20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa250009e60 tx=0x7fa250009780 comp rx=0 tx=0).stop 2023-10-06T02:23:22.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.029+0000 7fe1b67fc700 1 -- 172.21.15.31:0/703424535 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b0005b30 con 0x7fe1b8051340 2023-10-06T02:23:22.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.029+0000 7fe1b67fc700 1 -- 172.21.15.31:0/703424535 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1b0005c90 con 0x7fe1b8051340 2023-10-06T02:23:22.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.029+0000 7fe1b67fc700 1 -- 172.21.15.31:0/703424535 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b0007340 con 0x7fe1b8051340 2023-10-06T02:23:22.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.029+0000 7fe1c719d700 1 -- 172.21.15.31:0/703424535 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1b80dde80 con 0x7fe1b8051340 2023-10-06T02:23:22.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.029+0000 7fe1c719d700 1 -- 172.21.15.31:0/703424535 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1b80de440 con 0x7fe1b8051340 2023-10-06T02:23:22.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.029+0000 7fa262b9d700 1 -- 172.21.15.31:0/3639577362 shutdown_connections 2023-10-06T02:23:22.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.029+0000 7fa262b9d700 1 --2- 172.21.15.31:0/3639577362 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa25c16dff0 0x7fa25c0a9e20 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:22.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.029+0000 7fa262b9d700 1 --2- 172.21.15.31:0/3639577362 >> v2:172.21.15.31:3300/0 conn(0x7fa25c06ef30 0x7fa25c0a98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:22.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.029+0000 7fa262b9d700 1 -- 172.21.15.31:0/3639577362 >> 172.21.15.31:0/3639577362 conn(0x7fa25c06d130 msgr2=0x7fa25c0cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:22.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.029+0000 7f2251bff700 1 -- 172.21.15.31:0/2258491497 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f224c0cb970 msgr2=0x7f224c159ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:22.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.029+0000 7f2251bff700 1 --2- 172.21.15.31:0/2258491497 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f224c0cb970 0x7f224c159ec0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f223c009de0 tx=0x7f223c009780 comp rx=0 tx=0).stop 2023-10-06T02:23:22.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.030+0000 7f2251bff700 1 -- 172.21.15.31:0/2258491497 shutdown_connections 2023-10-06T02:23:22.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.030+0000 7fa262b9d700 1 -- 172.21.15.31:0/3639577362 shutdown_connections 2023-10-06T02:23:22.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.030+0000 7f2251bff700 1 --2- 172.21.15.31:0/2258491497 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f224c0cb970 0x7f224c159ec0 secure :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f223c009de0 tx=0x7f223c009780 comp rx=0 tx=0).stop 2023-10-06T02:23:22.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.030+0000 7f2251bff700 1 -- 172.21.15.31:0/2258491497 >> 172.21.15.31:0/2258491497 conn(0x7f224c070e30 msgr2=0x7f224c0d0710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:22.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.030+0000 7f9ee01ff700 1 -- 172.21.15.31:0/1786992288 shutdown_connections 2023-10-06T02:23:22.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.030+0000 7f1cdb5ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f1cd406cdb0 0x7f1cd41592d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:22.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.030+0000 7f1cd939b700 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f1cd40cbc80 0x7f1cd4152fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:33968/0 (socket says 172.21.15.31:33968) 2023-10-06T02:23:22.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.031+0000 7f2251bff700 1 -- 172.21.15.31:0/2258491497 shutdown_connections 2023-10-06T02:23:22.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.031+0000 7f2251bff700 1 -- 172.21.15.31:0/2258491497 wait complete. 2023-10-06T02:23:22.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.031+0000 7f1cd939b700 1 -- 172.21.15.31:0/1429024526 learned_addr learned my addr 172.21.15.31:0/1429024526 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:23:22.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.031+0000 7f1cd9b9c700 1 --2- 172.21.15.31:0/1429024526 >> v2:172.21.15.62:3300/0 conn(0x7f1cd406cdb0 0x7f1cd41592d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:22.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.032+0000 7f1cdb5ff700 1 -- 172.21.15.31:0/1429024526 --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cd4159ee0 con 0x7f1cd41568c0 2023-10-06T02:23:22.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.032+0000 7f1cdb5ff700 1 -- 172.21.15.31:0/1429024526 --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cd4155de0 con 0x7f1cd40cbc80 2023-10-06T02:23:22.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.032+0000 7f1cdb5ff700 1 -- 172.21.15.31:0/1429024526 --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cd406a0a0 con 0x7f1cd406cdb0 2023-10-06T02:23:22.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.030+0000 7fa262b9d700 1 -- 172.21.15.31:0/3639577362 wait complete. 2023-10-06T02:23:22.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.032+0000 7fa262b9d700 1 Processor -- start 2023-10-06T02:23:22.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.032+0000 7f1ccbfff700 1 -- 172.21.15.31:0/1429024526 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2205124104 0 0) 0x7f1cd4155de0 con 0x7f1cd40cbc80 2023-10-06T02:23:22.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.032+0000 7f1ccbfff700 1 -- 172.21.15.31:0/1429024526 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1cc0003570 con 0x7f1cd40cbc80 2023-10-06T02:23:22.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.033+0000 7f1cd9b9c700 1 -- 172.21.15.31:0/1429024526 >> v1:172.21.15.62:6789/0 conn(0x7f1cd40cbc80 legacy=0x7f1cd4152fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:22.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.033+0000 7f1cd9b9c700 1 -- 172.21.15.31:0/1429024526 >> v1:172.21.15.31:6789/0 conn(0x7f1cd41568c0 legacy=0x7f1cd406c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:22.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.033+0000 7f1cd9b9c700 1 -- 172.21.15.31:0/1429024526 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cd41560c0 con 0x7f1cd406cdb0 2023-10-06T02:23:22.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.033+0000 7f1cd9b9c700 1 --2- 172.21.15.31:0/1429024526 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1cd406cdb0 0x7f1cd41592d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc0098e0 tx=0x7f1ccc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c86abf228eb2a83 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:22.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.034+0000 7f2251bff700 1 Processor -- start 2023-10-06T02:23:22.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.034+0000 7f1ccbfff700 1 -- 172.21.15.31:0/1429024526 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ccc011630 con 0x7f1cd406cdb0 2023-10-06T02:23:22.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.034+0000 7f1ccbfff700 1 -- 172.21.15.31:0/1429024526 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ccc011790 con 0x7f1cd406cdb0 2023-10-06T02:23:22.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.034+0000 7f1ccbfff700 1 -- 172.21.15.31:0/1429024526 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ccc011b30 con 0x7f1cd406cdb0 2023-10-06T02:23:22.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.034+0000 7f2251bff700 1 -- start start 2023-10-06T02:23:22.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.035+0000 7fe1b67fc700 1 -- 172.21.15.31:0/703424535 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe1b001b400 con 0x7fe1b8051340 2023-10-06T02:23:22.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.033+0000 7fa262b9d700 1 -- start start 2023-10-06T02:23:22.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.034+0000 7fa262b9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa25c06ef30 0x7fa25c06b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:22.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.034+0000 7fa262b9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa25c0ca690 0x7fa25c06b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:22.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.034+0000 7fa262b9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa25c16dff0 0x7fa25c187090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:22.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.034+0000 7fa262b9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa25c0c3b40 con 0x7fa25c16dff0 2023-10-06T02:23:22.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.034+0000 7fa262b9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa25c0c3280 con 0x7fa25c06ef30 2023-10-06T02:23:22.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.034+0000 7fa262b9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa25c0c34f0 con 0x7fa25c0ca690 2023-10-06T02:23:22.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.035+0000 7fa26139a700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa25c0ca690 0x7fa25c06b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:22.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.035+0000 7fa26139a700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa25c0ca690 0x7fa25c06b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:57128/0 (socket says 172.21.15.31:57128) 2023-10-06T02:23:22.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.035+0000 7fa26139a700 1 -- 172.21.15.31:0/2262830846 learned_addr learned my addr 172.21.15.31:0/2262830846 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:22.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.035+0000 7f1cdb5ff700 1 -- 172.21.15.31:0/1429024526 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1cd406cdb0 msgr2=0x7f1cd41592d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:22.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.035+0000 7f1cdb5ff700 1 --2- 172.21.15.31:0/1429024526 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1cd406cdb0 0x7f1cd41592d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc0098e0 tx=0x7f1ccc0091a0 comp rx=0 tx=0).stop 2023-10-06T02:23:22.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.035+0000 7f1cdb5ff700 1 -- 172.21.15.31:0/1429024526 shutdown_connections 2023-10-06T02:23:22.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.035+0000 7f1cdb5ff700 1 --2- 172.21.15.31:0/1429024526 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1cd406cdb0 0x7f1cd41592d0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:22.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.035+0000 7f1cdb5ff700 1 -- 172.21.15.31:0/1429024526 >> 172.21.15.31:0/1429024526 conn(0x7f1cd40c8bb0 msgr2=0x7f1cd40c8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:22.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.035+0000 7f891f9ff700 1 Processor -- start 2023-10-06T02:23:22.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.030+0000 7f9ee01ff700 1 -- 172.21.15.31:0/1786992288 wait complete. 2023-10-06T02:23:22.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.035+0000 7fa26239c700 1 --2- 172.21.15.31:0/2262830846 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa25c16dff0 0x7fa25c187090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:22.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.032+0000 7f9ee01ff700 1 Processor -- start 2023-10-06T02:23:22.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.033+0000 7f9ee01ff700 1 -- start start 2023-10-06T02:23:22.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.034+0000 7f9ee01ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9ed80b27a0 0x7f9ed815dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:22.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.034+0000 7f9ee01ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ed80cb460 0x7f9ed8087830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:22.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.034+0000 7f9ee01ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ed815dc00 0x7f9ed81821e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:22.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.035+0000 7f9edd79a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ed80cb460 0x7f9ed8087830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:22.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.035+0000 7f9edd79a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ed80cb460 0x7f9ed8087830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37604/0 (socket says 172.21.15.31:37604) 2023-10-06T02:23:22.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.035+0000 7f9ee01ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed8087d70 con 0x7f9ed80cb460 2023-10-06T02:23:22.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.035+0000 7f9ede79c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ed815dc00 0x7f9ed81821e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:22.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.035+0000 7f9ede79c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ed815dc00 0x7f9ed81821e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:47366/0 (socket says 172.21.15.31:47366) 2023-10-06T02:23:22.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.035+0000 7f9ede79c700 1 -- 172.21.15.31:0/4248208831 learned_addr learned my addr 172.21.15.31:0/4248208831 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:22.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.035+0000 7f9ee01ff700 1 -- 172.21.15.31:0/4248208831 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed80b1150 con 0x7f9ed815dc00 2023-10-06T02:23:22.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.035+0000 7f9ee01ff700 1 -- 172.21.15.31:0/4248208831 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed80b5dc0 con 0x7f9ed80b27a0 2023-10-06T02:23:22.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.036+0000 7f891f9ff700 1 -- start start 2023-10-06T02:23:22.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.036+0000 7f9eddf9b700 1 --2- 172.21.15.31:0/4248208831 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9ed80b27a0 0x7f9ed815dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:22.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.036+0000 7fa261b9b700 1 --2- 172.21.15.31:0/2262830846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa25c06ef30 0x7fa25c06b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:22.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.036+0000 7fa26139a700 1 -- 172.21.15.31:0/2262830846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa25c06ef30 msgr2=0x7fa25c06b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:22.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.036+0000 7f1cdb5ff700 1 -- 172.21.15.31:0/1429024526 shutdown_connections 2023-10-06T02:23:22.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.036+0000 7f1cdb5ff700 1 -- 172.21.15.31:0/1429024526 wait complete. 2023-10-06T02:23:22.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.036+0000 7fa26139a700 1 --2- 172.21.15.31:0/2262830846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa25c06ef30 0x7fa25c06b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:22.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.036+0000 7fa26139a700 1 -- 172.21.15.31:0/2262830846 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa25c16dff0 msgr2=0x7fa25c187090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:22.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.036+0000 7fa26139a700 1 --2- 172.21.15.31:0/2262830846 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa25c16dff0 0x7fa25c187090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:22.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.036+0000 7fa26139a700 1 -- 172.21.15.31:0/2262830846 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa250009a20 con 0x7fa25c0ca690 2023-10-06T02:23:22.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.036+0000 7f9ede79c700 1 -- 172.21.15.31:0/4248208831 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9ed80b27a0 msgr2=0x7f9ed815dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:22.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.036+0000 7fa26139a700 1 --2- 172.21.15.31:0/2262830846 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa25c0ca690 0x7fa25c06b6f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa24c00e8d0 tx=0x7fa24c008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:22.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.036+0000 7f9ede79c700 1 --2- 172.21.15.31:0/4248208831 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9ed80b27a0 0x7f9ed815dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:22.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.036+0000 7fe1b67fc700 1 --2- 172.21.15.31:0/703424535 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe1a4067080 0x7fe1a4069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:22.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.036+0000 7fe1b67fc700 1 -- 172.21.15.31:0/703424535 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe1b00841b0 con 0x7fe1b8051340 2023-10-06T02:23:22.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.036+0000 7f9ede79c700 1 -- 172.21.15.31:0/4248208831 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ed80cb460 msgr2=0x7f9ed8087830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:22.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.037+0000 7f9ede79c700 1 --2- 172.21.15.31:0/4248208831 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ed80cb460 0x7f9ed8087830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:22.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.037+0000 7fe1c599a700 1 --2- 172.21.15.31:0/703424535 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe1a4067080 0x7fe1a4069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:22.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.037+0000 7f9ede79c700 1 -- 172.21.15.31:0/4248208831 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ed0009ae0 con 0x7f9ed815dc00 2023-10-06T02:23:22.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.037+0000 7f9edd79a700 1 --2- 172.21.15.31:0/4248208831 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ed80cb460 0x7f9ed8087830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:23:22.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.037+0000 7fa259ffb700 1 -- 172.21.15.31:0/2262830846 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa24c00ee70 con 0x7fa25c0ca690 2023-10-06T02:23:22.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.037+0000 7fa262b9d700 1 -- 172.21.15.31:0/2262830846 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa25c0c37d0 con 0x7fa25c0ca690 2023-10-06T02:23:22.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.037+0000 7fa262b9d700 1 -- 172.21.15.31:0/2262830846 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa25c0c3fd0 con 0x7fa25c0ca690 2023-10-06T02:23:22.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.038+0000 7f9ede79c700 1 --2- 172.21.15.31:0/4248208831 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ed815dc00 0x7f9ed81821e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9ed000b6c0 tx=0x7f9ed0017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:22.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.038+0000 7f891f9ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8918159b40 0x7f89180ab250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:22.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.038+0000 7f891f9ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f89180ab790 0x7f89180af410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:22.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.037+0000 7f2251bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f224c0cb5a0 0x7f224c153390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:22.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.037+0000 7f2251bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f224c15a400 0x7f224c0b34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:22.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.038+0000 7f891f9ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89180ac8e0 con 0x7f891806e3f0 2023-10-06T02:23:22.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.037+0000 7f2251bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f224c1554b0 0x7f224c155950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:22.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.038+0000 7f224affd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f224c15a400 0x7f224c0b34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:22.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.038+0000 7f2251bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f224c1538d0 con 0x7f224c1554b0 2023-10-06T02:23:22.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.038+0000 7f2251bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f224c155e90 con 0x7f224c15a400 2023-10-06T02:23:22.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.038+0000 7f2251bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f224c0b63b0 con 0x7f224c0cb5a0 2023-10-06T02:23:22.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.038+0000 7f224affd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f224c15a400 0x7f224c0b34b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:47372/0 (socket says 172.21.15.31:47372) 2023-10-06T02:23:22.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.038+0000 7f224affd700 1 -- 172.21.15.31:0/1200253084 learned_addr learned my addr 172.21.15.31:0/1200253084 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:22.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.038+0000 7f224bfff700 1 --2- 172.21.15.31:0/1200253084 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f224c1554b0 0x7f224c155950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:22.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.038+0000 7f224b7fe700 1 --2- 172.21.15.31:0/1200253084 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f224c0cb5a0 0x7f224c153390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:22.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.038+0000 7f891f9ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89180abcd0 con 0x7f8918159b40 2023-10-06T02:23:22.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.038+0000 7f891f9ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89180abfb0 con 0x7f89180ab790 2023-10-06T02:23:22.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.039+0000 7f224affd700 1 -- 172.21.15.31:0/1200253084 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f224c0cb5a0 msgr2=0x7f224c153390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:22.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.039+0000 7f224affd700 1 --2- 172.21.15.31:0/1200253084 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f224c0cb5a0 0x7f224c153390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:22.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.039+0000 7f224affd700 1 -- 172.21.15.31:0/1200253084 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f224c1554b0 msgr2=0x7f224c155950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:22.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.039+0000 7f224affd700 1 --2- 172.21.15.31:0/1200253084 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f224c1554b0 0x7f224c155950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:22.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.039+0000 7f224affd700 1 -- 172.21.15.31:0/1200253084 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f223c009a20 con 0x7f224c15a400 2023-10-06T02:23:22.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.039+0000 7f891df9c700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f89180ab790 0x7f89180af410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:22.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.039+0000 7f891df9c700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f89180ab790 0x7f89180af410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:57164/0 (socket says 172.21.15.31:57164) 2023-10-06T02:23:22.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.039+0000 7f891df9c700 1 -- 172.21.15.31:0/1625259117 learned_addr learned my addr 172.21.15.31:0/1625259117 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:22.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.039+0000 7fe1c719d700 1 -- 172.21.15.31:0/703424535 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1a80052b0 con 0x7fe1b8051340 2023-10-06T02:23:22.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.039+0000 7fa262b9d700 1 -- 172.21.15.31:0/2262830846 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa25c0c3cb0 con 0x7fa25c0ca690 2023-10-06T02:23:22.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.039+0000 7f891df9c700 1 -- 172.21.15.31:0/1625259117 >> v2:172.21.15.62:3300/0 conn(0x7f8918159b40 msgr2=0x7f89180ab250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:22.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.039+0000 7f9ecdffb700 1 -- 172.21.15.31:0/4248208831 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed0012de0 con 0x7f9ed815dc00 2023-10-06T02:23:22.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.039+0000 7f891cf9a700 1 --2- 172.21.15.31:0/1625259117 >> v2:172.21.15.62:3300/0 conn(0x7f8918159b40 0x7f89180ab250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:22.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.038+0000 7f1cdb5ff700 1 Processor -- start 2023-10-06T02:23:22.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.039+0000 7f891df9c700 1 --2- 172.21.15.31:0/1625259117 >> v2:172.21.15.62:3300/0 conn(0x7f8918159b40 0x7f89180ab250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:22.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.040+0000 7f891df9c700 1 -- 172.21.15.31:0/1625259117 >> v1:172.21.15.31:6789/0 conn(0x7f891806e3f0 legacy=0x7f891806e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:22.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.040+0000 7f891df9c700 1 -- 172.21.15.31:0/1625259117 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89180ac290 con 0x7f89180ab790 2023-10-06T02:23:22.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.040+0000 7fefbeb9d700 1 Processor -- start 2023-10-06T02:23:22.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.039+0000 7f9ee01ff700 1 -- 172.21.15.31:0/4248208831 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ed80b0b00 con 0x7f9ed815dc00 2023-10-06T02:23:22.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.040+0000 7f9ee01ff700 1 -- 172.21.15.31:0/4248208831 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ed80b07f0 con 0x7f9ed815dc00 2023-10-06T02:23:22.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.040+0000 7f9ecdffb700 1 -- 172.21.15.31:0/4248208831 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ed001d420 con 0x7f9ed815dc00 2023-10-06T02:23:22.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.040+0000 7f9ecdffb700 1 -- 172.21.15.31:0/4248208831 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed001d7e0 con 0x7f9ed815dc00 2023-10-06T02:23:22.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.040+0000 7fe1c599a700 1 --2- 172.21.15.31:0/703424535 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe1a4067080 0x7fe1a4069530 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc00b6a0 tx=0x7fe1bc01b000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:22.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.040+0000 7f1cdb5ff700 1 -- start start 2023-10-06T02:23:22.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.040+0000 7f891df9c700 1 --2- 172.21.15.31:0/1625259117 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f89180ab790 0x7f89180af410 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8910009f50 tx=0x7f8910009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c6f0572e7e3c7d6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:22.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.040+0000 7fefbeb9d700 1 -- start start 2023-10-06T02:23:22.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.040+0000 7f224affd700 1 --2- 172.21.15.31:0/1200253084 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f224c15a400 0x7f224c0b34b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f223c00b6c0 tx=0x7f223c017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:22.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.040+0000 7fa259ffb700 1 -- 172.21.15.31:0/2262830846 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa24c00c5e0 con 0x7fa25c0ca690 2023-10-06T02:23:22.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.040+0000 7fa259ffb700 1 -- 172.21.15.31:0/2262830846 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa24c00ee70 con 0x7fa25c0ca690 2023-10-06T02:23:22.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.041+0000 7f22437fe700 1 -- 172.21.15.31:0/1200253084 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f223c00be70 con 0x7f224c15a400 2023-10-06T02:23:22.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.041+0000 7f22437fe700 1 -- 172.21.15.31:0/1200253084 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f223c01e450 con 0x7f224c15a400 2023-10-06T02:23:22.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.041+0000 7f22437fe700 1 -- 172.21.15.31:0/1200253084 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f223c00be70 con 0x7f224c15a400 2023-10-06T02:23:22.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.042+0000 7f1cdb5ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1cd406cdb0 0x7f1cd4154cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:22.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.042+0000 7f9ee01ff700 1 -- 172.21.15.31:0/4248208831 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9eb80052b0 con 0x7f9ed815dc00 2023-10-06T02:23:22.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.042+0000 7f1cdb5ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1cd40cbc80 0x7f1cd40c58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:22.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.042+0000 7fefbeb9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fefb00dc0e0 0x7fefb0033a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:22.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.042+0000 7f1cdb5ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1cd41568c0 0x7f1cd40c25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:22.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.042+0000 7f891cf9a700 1 --2- 172.21.15.31:0/1625259117 >> v2:172.21.15.62:3300/0 conn(0x7f8918159b40 0x7f89180ab250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:23:22.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.042+0000 7f1cdb5ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd40afdc0 con 0x7f1cd406cdb0 2023-10-06T02:23:22.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.042+0000 7fefbeb9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fefb0033fd0 0x7fefb00de980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:22.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.042+0000 7fefbeb9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefb0053310 con 0x7fefb00dbd10 2023-10-06T02:23:22.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.042+0000 7fefbeb9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefb0053bb0 con 0x7fefb0033fd0 2023-10-06T02:23:22.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.042+0000 7fefbeb9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefb0034430 con 0x7fefb00dc0e0 2023-10-06T02:23:22.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.042+0000 7f1cdb5ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd40b0620 con 0x7f1cd41568c0 2023-10-06T02:23:22.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.043+0000 7f2251bff700 1 -- 172.21.15.31:0/1200253084 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f224c16f840 con 0x7f224c15a400 2023-10-06T02:23:22.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.042+0000 7f1cdb5ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd40c62f0 con 0x7f1cd40cbc80 2023-10-06T02:23:22.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.042+0000 7fa259ffb700 1 -- 172.21.15.31:0/2262830846 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa24c00c9e0 con 0x7fa25c0ca690 2023-10-06T02:23:22.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.043+0000 7f2251bff700 1 -- 172.21.15.31:0/1200253084 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f224c16ef10 con 0x7f224c15a400 2023-10-06T02:23:22.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.043+0000 7fefbd39a700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fefb00dc0e0 0x7fefb0033a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:22.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.043+0000 7fefbdb9b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fefb00dbd10 0x7fefb0005f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49314/0 (socket says 172.21.15.31:49314) 2023-10-06T02:23:22.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.043+0000 7fefbdb9b700 1 -- 172.21.15.31:0/4175229670 learned_addr learned my addr 172.21.15.31:0/4175229670 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:23:22.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.043+0000 7fefbe39c700 1 --2- 172.21.15.31:0/4175229670 >> v2:172.21.15.31:3300/0 conn(0x7fefb0033fd0 0x7fefb00de980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:22.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.043+0000 7f1cd8b9a700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1cd40cbc80 0x7f1cd40c58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:22.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.043+0000 7f1cd8b9a700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1cd40cbc80 0x7f1cd40c58f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:57168/0 (socket says 172.21.15.31:57168) 2023-10-06T02:23:22.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.043+0000 7f1cd8b9a700 1 -- 172.21.15.31:0/1563654420 learned_addr learned my addr 172.21.15.31:0/1563654420 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:22.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.043+0000 7f1cd939b700 1 --2- 172.21.15.31:0/1563654420 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1cd406cdb0 0x7f1cd4154cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:22.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.044+0000 7fefbd39a700 1 -- 172.21.15.31:0/4175229670 >> v2:172.21.15.31:3300/0 conn(0x7fefb0033fd0 msgr2=0x7fefb00de980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:22.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.044+0000 7f1cd9b9c700 1 --2- 172.21.15.31:0/1563654420 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1cd41568c0 0x7f1cd40c25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:22.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.044+0000 7fefbd39a700 1 --2- 172.21.15.31:0/4175229670 >> v2:172.21.15.31:3300/0 conn(0x7fefb0033fd0 0x7fefb00de980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:22.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.044+0000 7f1cd8b9a700 1 -- 172.21.15.31:0/1563654420 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1cd41568c0 msgr2=0x7f1cd40c25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:22.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.044+0000 7f1cd8b9a700 1 --2- 172.21.15.31:0/1563654420 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1cd41568c0 0x7f1cd40c25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:22.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.044+0000 7f1cd8b9a700 1 -- 172.21.15.31:0/1563654420 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1cd406cdb0 msgr2=0x7f1cd4154cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:22.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.044+0000 7f1cd8b9a700 1 --2- 172.21.15.31:0/1563654420 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1cd406cdb0 0x7f1cd4154cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:22.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.044+0000 7f1cd8b9a700 1 -- 172.21.15.31:0/1563654420 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ccc009420 con 0x7f1cd40cbc80 2023-10-06T02:23:22.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.044+0000 7f1cd8b9a700 1 --2- 172.21.15.31:0/1563654420 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1cd40cbc80 0x7f1cd40c58f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1cd000bbb0 tx=0x7f1cd0009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:22.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.044+0000 7fa259ffb700 1 --2- 172.21.15.31:0/2262830846 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa248066f90 0x7fa248069440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:22.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.044+0000 7fa259ffb700 1 -- 172.21.15.31:0/2262830846 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa24c0840c0 con 0x7fa25c0ca690 2023-10-06T02:23:22.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.044+0000 7fefbd39a700 1 -- 172.21.15.31:0/4175229670 >> v1:172.21.15.31:6789/0 conn(0x7fefb00dbd10 legacy=0x7fefb0005f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:22.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.045+0000 7fefbcb99700 1 -- 172.21.15.31:0/4175229670 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1852888917 0 0) 0x7fefb0053310 con 0x7fefb00dbd10 2023-10-06T02:23:22.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.045+0000 7fefbd39a700 1 -- 172.21.15.31:0/4175229670 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefb00df6b0 con 0x7fefb00dc0e0 2023-10-06T02:23:22.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.044+0000 7f9ecdffb700 1 -- 172.21.15.31:0/4248208831 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9ed001dc10 con 0x7f9ed815dc00 2023-10-06T02:23:22.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.045+0000 7f2251bff700 1 -- 172.21.15.31:0/1200253084 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f224c0c8880 con 0x7f224c15a400 2023-10-06T02:23:22.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.045+0000 7fa261b9b700 1 --2- 172.21.15.31:0/2262830846 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa248066f90 0x7fa248069440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:22.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.045+0000 7f9ecdffb700 1 --2- 172.21.15.31:0/4248208831 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f9ebc067080 0x7f9ebc069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:22.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.045+0000 7fefbd39a700 1 --2- 172.21.15.31:0/4175229670 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefb00dc0e0 0x7fefb0033a90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fefb000ddf0 tx=0x7fefac0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=813be6ea827d0f64 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:22.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.045+0000 7f9ecdffb700 1 -- 172.21.15.31:0/4248208831 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9ed0086030 con 0x7f9ed815dc00 2023-10-06T02:23:22.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.046+0000 7f1cc97fa700 1 -- 172.21.15.31:0/1563654420 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd00092e0 con 0x7f1cd40cbc80 2023-10-06T02:23:22.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.046+0000 7f890ffff700 1 -- 172.21.15.31:0/1625259117 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f891000e070 con 0x7f89180ab790 2023-10-06T02:23:22.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.046+0000 7fe1b67fc700 1 -- 172.21.15.31:0/703424535 <== mon.2 v2:172.21.15.62:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe1b00513d0 con 0x7fe1b8051340 2023-10-06T02:23:22.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.046+0000 7f1cdb5ff700 1 -- 172.21.15.31:0/1563654420 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cd40affd0 con 0x7f1cd40cbc80 2023-10-06T02:23:22.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.046+0000 7f1cdb5ff700 1 -- 172.21.15.31:0/1563654420 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cd4155200 con 0x7f1cd40cbc80 2023-10-06T02:23:22.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.046+0000 7fa261b9b700 1 --2- 172.21.15.31:0/2262830846 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa248066f90 0x7fa248069440 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa25c069370 tx=0x7fa254009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:22.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.046+0000 7f9eddf9b700 1 --2- 172.21.15.31:0/4248208831 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f9ebc067080 0x7f9ebc069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:22.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.047+0000 7f9eddf9b700 1 --2- 172.21.15.31:0/4248208831 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f9ebc067080 0x7f9ebc069530 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4006fd0 tx=0x7f9ed4000f90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:22.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.047+0000 7f22437fe700 1 -- 172.21.15.31:0/1200253084 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f223c01e5c0 con 0x7f224c15a400 2023-10-06T02:23:22.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.047+0000 7f1cdb5ff700 1 -- 172.21.15.31:0/1563654420 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cd40b07b0 con 0x7f1cd40cbc80 2023-10-06T02:23:22.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.048+0000 7f22437fe700 1 --2- 172.21.15.31:0/1200253084 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f222c066ff0 0x7f222c0694a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:22.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.048+0000 7f22437fe700 1 -- 172.21.15.31:0/1200253084 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f223c086500 con 0x7f224c15a400 2023-10-06T02:23:22.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.049+0000 7f224b7fe700 1 --2- 172.21.15.31:0/1200253084 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f222c066ff0 0x7f222c0694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:22.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.049+0000 7fefbcb99700 1 -- 172.21.15.31:0/4175229670 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefac011630 con 0x7fefb00dc0e0 2023-10-06T02:23:22.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.049+0000 7f224b7fe700 1 --2- 172.21.15.31:0/1200253084 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f222c066ff0 0x7f222c0694a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2244006fd0 tx=0x7f2244008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:22.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.050+0000 7f9ecdffb700 1 -- 172.21.15.31:0/4248208831 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9ed00b75a0 con 0x7f9ed815dc00 2023-10-06T02:23:22.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.051+0000 7f1cc97fa700 1 -- 172.21.15.31:0/1563654420 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cd000c9b0 con 0x7f1cd40cbc80 2023-10-06T02:23:22.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.051+0000 7f1cc97fa700 1 -- 172.21.15.31:0/1563654420 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd000cb40 con 0x7f1cd40cbc80 2023-10-06T02:23:22.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.051+0000 7fa259ffb700 1 -- 172.21.15.31:0/2262830846 <== mon.2 v2:172.21.15.62:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa24c054c60 con 0x7fa25c0ca690 2023-10-06T02:23:22.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.051+0000 7f890ffff700 1 -- 172.21.15.31:0/1625259117 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f891000bac0 con 0x7f89180ab790 2023-10-06T02:23:22.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.051+0000 7f890ffff700 1 -- 172.21.15.31:0/1625259117 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89100092e0 con 0x7f89180ab790 2023-10-06T02:23:22.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.052+0000 7f891f9ff700 1 -- 172.21.15.31:0/1625259117 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f89180ab790 msgr2=0x7f89180af410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:22.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.052+0000 7f891f9ff700 1 --2- 172.21.15.31:0/1625259117 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f89180ab790 0x7f89180af410 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8910009f50 tx=0x7f8910009840 comp rx=0 tx=0).stop 2023-10-06T02:23:22.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.052+0000 7f891f9ff700 1 -- 172.21.15.31:0/1625259117 shutdown_connections 2023-10-06T02:23:22.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.052+0000 7f891f9ff700 1 --2- 172.21.15.31:0/1625259117 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f89180ab790 0x7f89180af410 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:22.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.053+0000 7f891f9ff700 1 --2- 172.21.15.31:0/1625259117 >> v2:172.21.15.62:3300/0 conn(0x7f8918159b40 0x7f89180ab250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:22.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.053+0000 7f891f9ff700 1 -- 172.21.15.31:0/1625259117 >> 172.21.15.31:0/1625259117 conn(0x7f89180cc300 msgr2=0x7f89180cc700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:22.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.053+0000 7f891f9ff700 1 -- 172.21.15.31:0/1625259117 shutdown_connections 2023-10-06T02:23:22.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.053+0000 7f1cc97fa700 1 -- 172.21.15.31:0/1563654420 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1cd0012070 con 0x7f1cd40cbc80 2023-10-06T02:23:22.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.053+0000 7f891f9ff700 1 -- 172.21.15.31:0/1625259117 wait complete. 2023-10-06T02:23:22.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.054+0000 7fefbcb99700 1 -- 172.21.15.31:0/4175229670 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefac011790 con 0x7fefb00dc0e0 2023-10-06T02:23:22.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.054+0000 7f22437fe700 1 -- 172.21.15.31:0/1200253084 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f224c0c8880 con 0x7f224c15a400 2023-10-06T02:23:22.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.055+0000 7fefbcb99700 1 -- 172.21.15.31:0/4175229670 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefac011b30 con 0x7fefb00dc0e0 2023-10-06T02:23:22.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.055+0000 7fefbeb9d700 1 -- 172.21.15.31:0/4175229670 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefb00dc0e0 msgr2=0x7fefb0033a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:22.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.055+0000 7fefbeb9d700 1 --2- 172.21.15.31:0/4175229670 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefb00dc0e0 0x7fefb0033a90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fefb000ddf0 tx=0x7fefac0091a0 comp rx=0 tx=0).stop 2023-10-06T02:23:22.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.055+0000 7fefbeb9d700 1 -- 172.21.15.31:0/4175229670 shutdown_connections 2023-10-06T02:23:22.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.055+0000 7fefbeb9d700 1 --2- 172.21.15.31:0/4175229670 >> v2:172.21.15.31:3300/0 conn(0x7fefb0033fd0 0x7fefb00de980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:22.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.055+0000 7fefbeb9d700 1 --2- 172.21.15.31:0/4175229670 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefb00dc0e0 0x7fefb0033a90 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:22.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.055+0000 7fefbeb9d700 1 -- 172.21.15.31:0/4175229670 >> 172.21.15.31:0/4175229670 conn(0x7fefb00517b0 msgr2=0x7fefb0051bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:22.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.056+0000 7f1cc97fa700 1 --2- 172.21.15.31:0/1563654420 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f1cb8066f00 0x7f1cb80693b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:22.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.056+0000 7f1cc97fa700 1 -- 172.21.15.31:0/1563654420 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1cd0084880 con 0x7f1cd40cbc80 2023-10-06T02:23:22.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.056+0000 7f891f9ff700 1 Processor -- start 2023-10-06T02:23:22.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.056+0000 7f891f9ff700 1 -- start start 2023-10-06T02:23:22.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.055+0000 7fefbeb9d700 1 -- 172.21.15.31:0/4175229670 shutdown_connections 2023-10-06T02:23:22.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.056+0000 7fefbeb9d700 1 -- 172.21.15.31:0/4175229670 wait complete. 2023-10-06T02:23:22.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.057+0000 7f1cd939b700 1 --2- 172.21.15.31:0/1563654420 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f1cb8066f00 0x7f1cb80693b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:22.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.057+0000 7f1cc97fa700 1 -- 172.21.15.31:0/1563654420 <== mon.2 v2:172.21.15.62:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1cd0051950 con 0x7f1cd40cbc80 2023-10-06T02:23:22.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.058+0000 7fefbeb9d700 1 Processor -- start 2023-10-06T02:23:22.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.058+0000 7f1cd939b700 1 --2- 172.21.15.31:0/1563654420 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f1cb8066f00 0x7f1cb80693b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1cc40058d0 tx=0x7f1cc4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:22.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.058+0000 7fefbeb9d700 1 -- start start 2023-10-06T02:23:22.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.058+0000 7f891f9ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f891806e3f0 0x7f891814ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:22.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.059+0000 7f891f9ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89180ab790 0x7f8918160ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:22.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.059+0000 7f891d79b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f891806e3f0 0x7f891814ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:22.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.059+0000 7f891d79b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f891806e3f0 0x7f891814ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:57182/0 (socket says 172.21.15.31:57182) 2023-10-06T02:23:22.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.059+0000 7f891d79b700 1 -- 172.21.15.31:0/2662857315 learned_addr learned my addr 172.21.15.31:0/2662857315 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:22.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.060+0000 7f891cf9a700 1 --2- 172.21.15.31:0/2662857315 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89180ab790 0x7f8918160ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:22.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.060+0000 7f891f9ff700 1 --2- 172.21.15.31:0/2662857315 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8918161020 0x7f891815d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:22.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.060+0000 7f891df9c700 1 --2- 172.21.15.31:0/2662857315 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8918161020 0x7f891815d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:22.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.061+0000 7f891f9ff700 1 -- 172.21.15.31:0/2662857315 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89181614c0 con 0x7f8918161020 2023-10-06T02:23:22.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.061+0000 7f891f9ff700 1 -- 172.21.15.31:0/2662857315 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f891815ee80 con 0x7f89180ab790 2023-10-06T02:23:22.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.061+0000 7f891f9ff700 1 -- 172.21.15.31:0/2662857315 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f891815e270 con 0x7f891806e3f0 2023-10-06T02:23:22.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.060+0000 7fefbeb9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fefb0033fd0 0x7fefb0035c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:22.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.060+0000 7fefbeb9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fefb00dbd10 0x7fefb00361d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:22.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.060+0000 7fefbeb9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefb00cc590 0x7fefb0036710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:22.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.060+0000 7fefbeb9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefb00cafc0 con 0x7fefb00dbd10 2023-10-06T02:23:22.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.060+0000 7fefbeb9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefb00351b0 con 0x7fefb00cc590 2023-10-06T02:23:22.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.060+0000 7fefbeb9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefb0035490 con 0x7fefb0033fd0 2023-10-06T02:23:22.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.060+0000 7fefbe39c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefb00cc590 0x7fefb0036710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:22.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.060+0000 7fefbe39c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefb00cc590 0x7fefb0036710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:47420/0 (socket says 172.21.15.31:47420) 2023-10-06T02:23:22.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.060+0000 7fefbe39c700 1 -- 172.21.15.31:0/3393926690 learned_addr learned my addr 172.21.15.31:0/3393926690 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:22.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.061+0000 7fefbdb9b700 1 --2- 172.21.15.31:0/3393926690 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fefb0033fd0 0x7fefb0035c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:22.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.061+0000 7fefbe39c700 1 -- 172.21.15.31:0/3393926690 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fefb0033fd0 msgr2=0x7fefb0035c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:22.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.061+0000 7fefbe39c700 1 --2- 172.21.15.31:0/3393926690 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fefb0033fd0 0x7fefb0035c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:22.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.061+0000 7fefbe39c700 1 -- 172.21.15.31:0/3393926690 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fefb00dbd10 msgr2=0x7fefb00361d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T02:23:22.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.061+0000 7fefbe39c700 1 --2- 172.21.15.31:0/3393926690 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fefb00dbd10 0x7fefb00361d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:22.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.061+0000 7fefbe39c700 1 -- 172.21.15.31:0/3393926690 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefac009420 con 0x7fefb00cc590 2023-10-06T02:23:22.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.062+0000 7fefbe39c700 1 --2- 172.21.15.31:0/3393926690 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefb00cc590 0x7fefb0036710 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fefb8062fd0 tx=0x7fefb806b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:22.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.062+0000 7f891df9c700 1 -- 172.21.15.31:0/2662857315 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f891806e3f0 msgr2=0x7f891814ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:22.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.062+0000 7f891df9c700 1 --2- 172.21.15.31:0/2662857315 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f891806e3f0 0x7f891814ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:22.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.062+0000 7fefa9ffb700 1 -- 172.21.15.31:0/3393926690 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb808c070 con 0x7fefb00cc590 2023-10-06T02:23:22.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.062+0000 7f891df9c700 1 -- 172.21.15.31:0/2662857315 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89180ab790 msgr2=0x7f8918160ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:22.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.062+0000 7f891df9c700 1 --2- 172.21.15.31:0/2662857315 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89180ab790 0x7f8918160ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:22.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.062+0000 7fefbeb9d700 1 -- 172.21.15.31:0/3393926690 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefb0035770 con 0x7fefb00cc590 2023-10-06T02:23:22.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.062+0000 7f891df9c700 1 -- 172.21.15.31:0/2662857315 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8910009ae0 con 0x7f8918161020 2023-10-06T02:23:22.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.062+0000 7f891d79b700 1 --2- 172.21.15.31:0/2662857315 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f891806e3f0 0x7f891814ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:23:22.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.063+0000 7f891df9c700 1 --2- 172.21.15.31:0/2662857315 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8918161020 0x7f891815d530 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f89180649d0 tx=0x7f8910017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:22.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.063+0000 7fefbeb9d700 1 -- 172.21.15.31:0/3393926690 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefb00cabe0 con 0x7fefb00cc590 2023-10-06T02:23:22.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.063+0000 7fefa9ffb700 1 -- 172.21.15.31:0/3393926690 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefb806d820 con 0x7fefb00cc590 2023-10-06T02:23:22.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.063+0000 7fefa9ffb700 1 -- 172.21.15.31:0/3393926690 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb806dbe0 con 0x7fefb00cc590 2023-10-06T02:23:22.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.064+0000 7fefbeb9d700 1 -- 172.21.15.31:0/3393926690 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefb00053d0 con 0x7fefb00cc590 2023-10-06T02:23:22.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.065+0000 7f890d7fa700 1 -- 172.21.15.31:0/2662857315 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89100032f0 con 0x7f8918161020 2023-10-06T02:23:22.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.065+0000 7f890d7fa700 1 -- 172.21.15.31:0/2662857315 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f891001c620 con 0x7f8918161020 2023-10-06T02:23:22.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.065+0000 7f890d7fa700 1 -- 172.21.15.31:0/2662857315 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f891001c9e0 con 0x7f8918161020 2023-10-06T02:23:22.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.066+0000 7f891f9ff700 1 -- 172.21.15.31:0/2662857315 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f891815e550 con 0x7f8918161020 2023-10-06T02:23:22.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.066+0000 7f891f9ff700 1 -- 172.21.15.31:0/2662857315 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f891815eb10 con 0x7f8918161020 2023-10-06T02:23:22.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.065+0000 7fefa9ffb700 1 -- 172.21.15.31:0/3393926690 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fefb806dd40 con 0x7fefb00cc590 2023-10-06T02:23:22.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.067+0000 7fefa9ffb700 1 --2- 172.21.15.31:0/3393926690 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fef9c067080 0x7fef9c069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:22.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.067+0000 7fefa9ffb700 1 -- 172.21.15.31:0/3393926690 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fefb80fb5e0 con 0x7fefb00cc590 2023-10-06T02:23:22.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.068+0000 7fefbdb9b700 1 --2- 172.21.15.31:0/3393926690 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fef9c067080 0x7fef9c069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:22.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.069+0000 7fefbdb9b700 1 --2- 172.21.15.31:0/3393926690 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fef9c067080 0x7fef9c069530 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fefb4007fd0 tx=0x7fefb4007ad0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:22.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.070+0000 7f890d7fa700 1 -- 172.21.15.31:0/2662857315 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8910019070 con 0x7f8918161020 2023-10-06T02:23:22.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.071+0000 7f890d7fa700 1 --2- 172.21.15.31:0/2662857315 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f88fc067080 0x7f88fc069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:22.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.076+0000 7fefa9ffb700 1 -- 172.21.15.31:0/3393926690 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fefb8096020 con 0x7fefb00cc590 2023-10-06T02:23:22.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.071+0000 7f8902ffd700 1 -- 172.21.15.31:0/2662857315 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f891804c3c0 con 0x7f8918161020 2023-10-06T02:23:22.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.076+0000 7f890d7fa700 1 -- 172.21.15.31:0/2662857315 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8910085dc0 con 0x7f8918161020 2023-10-06T02:23:22.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.081+0000 7f891d79b700 1 --2- 172.21.15.31:0/2662857315 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f88fc067080 0x7f88fc069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:22.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.082+0000 7f891d79b700 1 --2- 172.21.15.31:0/2662857315 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f88fc067080 0x7f88fc069530 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8908007fd0 tx=0x7f8908007f60 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:22.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.083+0000 7f890d7fa700 1 -- 172.21.15.31:0/2662857315 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8910052eb0 con 0x7f8918161020 2023-10-06T02:23:22.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.482+0000 7f384679d700 1 -- 172.21.15.31:0/3321121793 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f3820005560 con 0x7f383804da60 2023-10-06T02:23:22.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.483+0000 7f38357fa700 1 -- 172.21.15.31:0/3321121793 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3830009350 con 0x7f383804da60 2023-10-06T02:23:22.484 INFO:teuthology.orchestra.run.smithi031.stdout:55834574857 2023-10-06T02:23:22.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.487+0000 7f382affd700 1 -- 172.21.15.31:0/3321121793 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f3824067080 msgr2=0x7f3824069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:22.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.488+0000 7f382affd700 1 --2- 172.21.15.31:0/3321121793 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f3824067080 0x7f3824069530 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f384004d1b0 tx=0x7f384006b040 comp rx=0 tx=0).stop 2023-10-06T02:23:22.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.488+0000 7f382affd700 1 -- 172.21.15.31:0/3321121793 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f383804da60 msgr2=0x7f383804dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:22.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.488+0000 7f382affd700 1 --2- 172.21.15.31:0/3321121793 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f383804da60 0x7f383804dee0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f383800e090 tx=0x7f3830017000 comp rx=0 tx=0).stop 2023-10-06T02:23:22.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.489+0000 7f382affd700 1 -- 172.21.15.31:0/3321121793 shutdown_connections 2023-10-06T02:23:22.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.489+0000 7f382affd700 1 --2- 172.21.15.31:0/3321121793 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f383804da60 0x7f383804dee0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:22.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.489+0000 7f382affd700 1 --2- 172.21.15.31:0/3321121793 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f3824067080 0x7f3824069530 secure :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f384004d1b0 tx=0x7f384006b040 comp rx=0 tx=0).stop 2023-10-06T02:23:22.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.489+0000 7f382affd700 1 --2- 172.21.15.31:0/3321121793 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3838033fd0 0x7f38380359e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:22.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.489+0000 7f382affd700 1 --2- 172.21.15.31:0/3321121793 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38380dbd10 0x7f383804c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:22.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.489+0000 7f382affd700 1 -- 172.21.15.31:0/3321121793 >> 172.21.15.31:0/3321121793 conn(0x7f38380517b0 msgr2=0x7f38380069b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:22.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.491+0000 7f382affd700 1 -- 172.21.15.31:0/3321121793 shutdown_connections 2023-10-06T02:23:22.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.491+0000 7f382affd700 1 -- 172.21.15.31:0/3321121793 wait complete. 2023-10-06T02:23:22.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.500+0000 7fa262b9d700 1 -- 172.21.15.31:0/2262830846 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fa25c0c3630 con 0x7fa25c0ca690 2023-10-06T02:23:22.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.501+0000 7fa259ffb700 1 -- 172.21.15.31:0/2262830846 <== mon.2 v2:172.21.15.62:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa25c0c3630 con 0x7fa25c0ca690 2023-10-06T02:23:22.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.501+0000 7fe1c719d700 1 -- 172.21.15.31:0/703424535 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fe1a8005560 con 0x7fe1b8051340 2023-10-06T02:23:22.501 INFO:teuthology.orchestra.run.smithi031.stdout:55834574856 2023-10-06T02:23:22.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.502+0000 7fe1b67fc700 1 -- 172.21.15.31:0/703424535 <== mon.2 v2:172.21.15.62:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe1b00076f0 con 0x7fe1b8051340 2023-10-06T02:23:22.502 INFO:teuthology.orchestra.run.smithi031.stdout:55834574856 2023-10-06T02:23:22.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.504+0000 7f9ee01ff700 1 -- 172.21.15.31:0/4248208831 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f9eb8005560 con 0x7f9ed815dc00 2023-10-06T02:23:22.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.505+0000 7fa2437fe700 1 -- 172.21.15.31:0/2262830846 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa248066f90 msgr2=0x7fa248069440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:22.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.505+0000 7fa2437fe700 1 --2- 172.21.15.31:0/2262830846 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa248066f90 0x7fa248069440 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa25c069370 tx=0x7fa254009040 comp rx=0 tx=0).stop 2023-10-06T02:23:22.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.505+0000 7fa2437fe700 1 -- 172.21.15.31:0/2262830846 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa25c0ca690 msgr2=0x7fa25c06b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:22.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.505+0000 7fa2437fe700 1 --2- 172.21.15.31:0/2262830846 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa25c0ca690 0x7fa25c06b6f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa24c00e8d0 tx=0x7fa24c008040 comp rx=0 tx=0).stop 2023-10-06T02:23:22.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.505+0000 7f9ecdffb700 1 -- 172.21.15.31:0/4248208831 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9ed0053120 con 0x7f9ed815dc00 2023-10-06T02:23:22.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.506+0000 7fe1c719d700 1 -- 172.21.15.31:0/703424535 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe1a4067080 msgr2=0x7fe1a4069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:22.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.506+0000 7fe1c719d700 1 --2- 172.21.15.31:0/703424535 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe1a4067080 0x7fe1a4069530 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc00b6a0 tx=0x7fe1bc01b000 comp rx=0 tx=0).stop 2023-10-06T02:23:22.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.506+0000 7fa2437fe700 1 -- 172.21.15.31:0/2262830846 shutdown_connections 2023-10-06T02:23:22.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.506+0000 7fa2437fe700 1 --2- 172.21.15.31:0/2262830846 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa25c16dff0 0x7fa25c187090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:22.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.506+0000 7fa2437fe700 1 --2- 172.21.15.31:0/2262830846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa25c06ef30 0x7fa25c06b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:22.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.506+0000 7fa2437fe700 1 --2- 172.21.15.31:0/2262830846 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa248066f90 0x7fa248069440 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:22.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.506+0000 7fa2437fe700 1 --2- 172.21.15.31:0/2262830846 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa25c0ca690 0x7fa25c06b6f0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:22.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.506+0000 7fa2437fe700 1 -- 172.21.15.31:0/2262830846 >> 172.21.15.31:0/2262830846 conn(0x7fa25c06d130 msgr2=0x7fa25c0c7280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:22.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.506+0000 7fe1c719d700 1 -- 172.21.15.31:0/703424535 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe1b8051340 msgr2=0x7fe1b80dd830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:22.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.506+0000 7fe1c719d700 1 --2- 172.21.15.31:0/703424535 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe1b8051340 0x7fe1b80dd830 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe1b000c7f0 tx=0x7fe1b0007e00 comp rx=0 tx=0).stop 2023-10-06T02:23:22.508 INFO:teuthology.orchestra.run.smithi031.stdout:55834574855 2023-10-06T02:23:22.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.506+0000 7fe1c719d700 1 -- 172.21.15.31:0/703424535 shutdown_connections 2023-10-06T02:23:22.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.506+0000 7fe1c719d700 1 --2- 172.21.15.31:0/703424535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1b80df180 0x7fe1b80549d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:22.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.506+0000 7fe1c719d700 1 --2- 172.21.15.31:0/703424535 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe1a4067080 0x7fe1a4069530 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:22.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.507+0000 7fe1c719d700 1 --2- 172.21.15.31:0/703424535 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe1b8051340 0x7fe1b80dd830 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:22.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.507+0000 7fe1c719d700 1 --2- 172.21.15.31:0/703424535 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1b80dedb0 0x7fe1b80db220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:22.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.507+0000 7fe1c719d700 1 -- 172.21.15.31:0/703424535 >> 172.21.15.31:0/703424535 conn(0x7fe1b80528c0 msgr2=0x7fe1b804c500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:22.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.507+0000 7fe1c719d700 1 -- 172.21.15.31:0/703424535 shutdown_connections 2023-10-06T02:23:22.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.507+0000 7fe1c719d700 1 -- 172.21.15.31:0/703424535 wait complete. 2023-10-06T02:23:22.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.507+0000 7fa2437fe700 1 -- 172.21.15.31:0/2262830846 shutdown_connections 2023-10-06T02:23:22.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.507+0000 7fa2437fe700 1 -- 172.21.15.31:0/2262830846 wait complete. 2023-10-06T02:23:22.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.510+0000 7f9ee01ff700 1 -- 172.21.15.31:0/4248208831 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f9ebc067080 msgr2=0x7f9ebc069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:22.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.510+0000 7f9ee01ff700 1 --2- 172.21.15.31:0/4248208831 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f9ebc067080 0x7f9ebc069530 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4006fd0 tx=0x7f9ed4000f90 comp rx=0 tx=0).stop 2023-10-06T02:23:22.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.510+0000 7f9ee01ff700 1 -- 172.21.15.31:0/4248208831 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ed815dc00 msgr2=0x7f9ed81821e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:22.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.510+0000 7f9ee01ff700 1 --2- 172.21.15.31:0/4248208831 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ed815dc00 0x7f9ed81821e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9ed000b6c0 tx=0x7f9ed0017000 comp rx=0 tx=0).stop 2023-10-06T02:23:22.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.511+0000 7f9ee01ff700 1 -- 172.21.15.31:0/4248208831 shutdown_connections 2023-10-06T02:23:22.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.511+0000 7f9ee01ff700 1 --2- 172.21.15.31:0/4248208831 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ed815dc00 0x7f9ed81821e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:22.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.511+0000 7f9ee01ff700 1 --2- 172.21.15.31:0/4248208831 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f9ebc067080 0x7f9ebc069530 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:22.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.511+0000 7f9ee01ff700 1 --2- 172.21.15.31:0/4248208831 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9ed80b27a0 0x7f9ed815dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:22.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.511+0000 7f9ee01ff700 1 --2- 172.21.15.31:0/4248208831 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ed80cb460 0x7f9ed8087830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:22.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.511+0000 7f9ee01ff700 1 -- 172.21.15.31:0/4248208831 >> 172.21.15.31:0/4248208831 conn(0x7f9ed8070e30 msgr2=0x7f9ed815a900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:22.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.511+0000 7f9ee01ff700 1 -- 172.21.15.31:0/4248208831 shutdown_connections 2023-10-06T02:23:22.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.512+0000 7f9ee01ff700 1 -- 172.21.15.31:0/4248208831 wait complete. 2023-10-06T02:23:22.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.521+0000 7f1cdb5ff700 1 -- 172.21.15.31:0/1563654420 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f1cd404c3c0 con 0x7f1cd40cbc80 2023-10-06T02:23:22.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.522+0000 7f1cc97fa700 1 -- 172.21.15.31:0/1563654420 <== mon.2 v2:172.21.15.62:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1cd0054f70 con 0x7f1cd40cbc80 2023-10-06T02:23:22.522 INFO:teuthology.orchestra.run.smithi031.stdout:55834574856 2023-10-06T02:23:22.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.524+0000 7f2251bff700 1 -- 172.21.15.31:0/1200253084 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f224c04c3c0 con 0x7f224c15a400 2023-10-06T02:23:22.525 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.2 2023-10-06T02:23:22.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.525+0000 7f22437fe700 1 -- 172.21.15.31:0/1200253084 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f223c016090 con 0x7f224c15a400 2023-10-06T02:23:22.526 INFO:teuthology.orchestra.run.smithi031.stdout:55834574854 2023-10-06T02:23:22.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.525+0000 7f1cdb5ff700 1 -- 172.21.15.31:0/1563654420 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f1cb8066f00 msgr2=0x7f1cb80693b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:22.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.525+0000 7f1cdb5ff700 1 --2- 172.21.15.31:0/1563654420 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f1cb8066f00 0x7f1cb80693b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1cc40058d0 tx=0x7f1cc4009040 comp rx=0 tx=0).stop 2023-10-06T02:23:22.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.525+0000 7f1cdb5ff700 1 -- 172.21.15.31:0/1563654420 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1cd40cbc80 msgr2=0x7f1cd40c58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:22.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.525+0000 7f1cdb5ff700 1 --2- 172.21.15.31:0/1563654420 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1cd40cbc80 0x7f1cd40c58f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1cd000bbb0 tx=0x7f1cd0009840 comp rx=0 tx=0).stop 2023-10-06T02:23:22.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.526+0000 7f1cdb5ff700 1 -- 172.21.15.31:0/1563654420 shutdown_connections 2023-10-06T02:23:22.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.526+0000 7f1cdb5ff700 1 --2- 172.21.15.31:0/1563654420 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1cd41568c0 0x7f1cd40c25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:22.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.526+0000 7f1cdb5ff700 1 --2- 172.21.15.31:0/1563654420 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1cd406cdb0 0x7f1cd4154cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:22.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.526+0000 7f1cdb5ff700 1 --2- 172.21.15.31:0/1563654420 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f1cb8066f00 0x7f1cb80693b0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:22.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.526+0000 7f1cdb5ff700 1 --2- 172.21.15.31:0/1563654420 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1cd40cbc80 0x7f1cd40c58f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:22.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.526+0000 7f1cdb5ff700 1 -- 172.21.15.31:0/1563654420 >> 172.21.15.31:0/1563654420 conn(0x7f1cd40c8bb0 msgr2=0x7f1cd40a98f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:22.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.526+0000 7f1cdb5ff700 1 -- 172.21.15.31:0/1563654420 shutdown_connections 2023-10-06T02:23:22.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.527+0000 7f1cdb5ff700 1 -- 172.21.15.31:0/1563654420 wait complete. 2023-10-06T02:23:22.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.529+0000 7f2251bff700 1 -- 172.21.15.31:0/1200253084 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f222c066ff0 msgr2=0x7f222c0694a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:22.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.529+0000 7f2251bff700 1 --2- 172.21.15.31:0/1200253084 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f222c066ff0 0x7f222c0694a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2244006fd0 tx=0x7f2244008040 comp rx=0 tx=0).stop 2023-10-06T02:23:22.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.529+0000 7f2251bff700 1 -- 172.21.15.31:0/1200253084 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f224c15a400 msgr2=0x7f224c0b34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:22.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.529+0000 7f2251bff700 1 --2- 172.21.15.31:0/1200253084 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f224c15a400 0x7f224c0b34b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f223c00b6c0 tx=0x7f223c017000 comp rx=0 tx=0).stop 2023-10-06T02:23:22.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.530+0000 7f2251bff700 1 -- 172.21.15.31:0/1200253084 shutdown_connections 2023-10-06T02:23:22.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.530+0000 7f2251bff700 1 --2- 172.21.15.31:0/1200253084 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f224c1554b0 0x7f224c155950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:22.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.530+0000 7f2251bff700 1 --2- 172.21.15.31:0/1200253084 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f222c066ff0 0x7f222c0694a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:22.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.530+0000 7f2251bff700 1 --2- 172.21.15.31:0/1200253084 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f224c0cb5a0 0x7f224c153390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:22.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.530+0000 7f2251bff700 1 --2- 172.21.15.31:0/1200253084 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f224c15a400 0x7f224c0b34b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:22.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.530+0000 7f2251bff700 1 -- 172.21.15.31:0/1200253084 >> 172.21.15.31:0/1200253084 conn(0x7f224c070e30 msgr2=0x7f224c153a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:22.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.530+0000 7f2251bff700 1 -- 172.21.15.31:0/1200253084 shutdown_connections 2023-10-06T02:23:22.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.531+0000 7f2251bff700 1 -- 172.21.15.31:0/1200253084 wait complete. 2023-10-06T02:23:22.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.531+0000 7fefbeb9d700 1 -- 172.21.15.31:0/3393926690 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fefb0035960 con 0x7fefb00cc590 2023-10-06T02:23:22.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.532+0000 7fefa9ffb700 1 -- 172.21.15.31:0/3393926690 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fefb80cc160 con 0x7fefb00cc590 2023-10-06T02:23:22.532 INFO:teuthology.orchestra.run.smithi031.stdout:55834574856 2023-10-06T02:23:22.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.535+0000 7fefa37fe700 1 -- 172.21.15.31:0/3393926690 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fef9c067080 msgr2=0x7fef9c069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:22.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.535+0000 7fefa37fe700 1 --2- 172.21.15.31:0/3393926690 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fef9c067080 0x7fef9c069530 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fefb4007fd0 tx=0x7fefb4007ad0 comp rx=0 tx=0).stop 2023-10-06T02:23:22.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.535+0000 7f8902ffd700 1 -- 172.21.15.31:0/2662857315 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f89180ac570 con 0x7f8918161020 2023-10-06T02:23:22.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.536+0000 7fefa37fe700 1 -- 172.21.15.31:0/3393926690 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefb00cc590 msgr2=0x7fefb0036710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:22.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.536+0000 7fefa37fe700 1 --2- 172.21.15.31:0/3393926690 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefb00cc590 0x7fefb0036710 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fefb8062fd0 tx=0x7fefb806b040 comp rx=0 tx=0).stop 2023-10-06T02:23:22.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.536+0000 7fefa37fe700 1 -- 172.21.15.31:0/3393926690 shutdown_connections 2023-10-06T02:23:22.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.536+0000 7f890d7fa700 1 -- 172.21.15.31:0/2662857315 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8910029020 con 0x7f8918161020 2023-10-06T02:23:22.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.537+0000 7fefa37fe700 1 --2- 172.21.15.31:0/3393926690 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefb00cc590 0x7fefb0036710 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:22.538 INFO:teuthology.orchestra.run.smithi031.stdout:55834574856 2023-10-06T02:23:22.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.537+0000 7fefa37fe700 1 --2- 172.21.15.31:0/3393926690 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fef9c067080 0x7fef9c069530 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:22.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.537+0000 7fefa37fe700 1 --2- 172.21.15.31:0/3393926690 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fefb0033fd0 0x7fefb0035c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:22.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.537+0000 7fefa37fe700 1 --2- 172.21.15.31:0/3393926690 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fefb00dbd10 0x7fefb00361d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:22.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.537+0000 7fefa37fe700 1 -- 172.21.15.31:0/3393926690 >> 172.21.15.31:0/3393926690 conn(0x7fefb00517b0 msgr2=0x7fefb00cedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:22.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.538+0000 7fefa37fe700 1 -- 172.21.15.31:0/3393926690 shutdown_connections 2023-10-06T02:23:22.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.539+0000 7fefa37fe700 1 -- 172.21.15.31:0/3393926690 wait complete. 2023-10-06T02:23:22.541 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.5 2023-10-06T02:23:22.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.539+0000 7f8902ffd700 1 -- 172.21.15.31:0/2662857315 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f88fc067080 msgr2=0x7f88fc069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:22.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.539+0000 7f8902ffd700 1 --2- 172.21.15.31:0/2662857315 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f88fc067080 0x7f88fc069530 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8908007fd0 tx=0x7f8908007f60 comp rx=0 tx=0).stop 2023-10-06T02:23:22.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.539+0000 7f8902ffd700 1 -- 172.21.15.31:0/2662857315 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8918161020 msgr2=0x7f891815d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:22.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.539+0000 7f8902ffd700 1 --2- 172.21.15.31:0/2662857315 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8918161020 0x7f891815d530 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f89180649d0 tx=0x7f8910017000 comp rx=0 tx=0).stop 2023-10-06T02:23:22.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.540+0000 7f8902ffd700 1 -- 172.21.15.31:0/2662857315 shutdown_connections 2023-10-06T02:23:22.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.540+0000 7f8902ffd700 1 --2- 172.21.15.31:0/2662857315 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8918161020 0x7f891815d530 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:22.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.540+0000 7f8902ffd700 1 --2- 172.21.15.31:0/2662857315 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f88fc067080 0x7f88fc069530 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:22.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.540+0000 7f8902ffd700 1 --2- 172.21.15.31:0/2662857315 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f891806e3f0 0x7f891814ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:22.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.540+0000 7f8902ffd700 1 --2- 172.21.15.31:0/2662857315 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89180ab790 0x7f8918160ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:22.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.540+0000 7f8902ffd700 1 -- 172.21.15.31:0/2662857315 >> 172.21.15.31:0/2662857315 conn(0x7f89180cc300 msgr2=0x7f89180acd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:22.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.540+0000 7f8902ffd700 1 -- 172.21.15.31:0/2662857315 shutdown_connections 2023-10-06T02:23:22.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:22.540+0000 7f8902ffd700 1 -- 172.21.15.31:0/2662857315 wait complete. 2023-10-06T02:23:22.544 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.0 2023-10-06T02:23:22.545 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.1 2023-10-06T02:23:22.562 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.4 2023-10-06T02:23:22.563 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.3 2023-10-06T02:23:22.569 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.6 2023-10-06T02:23:22.570 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.7 2023-10-06T02:23:23.527 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-06T02:23:23.542 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-06T02:23:23.545 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-06T02:23:23.545 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-06T02:23:23.563 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-06T02:23:23.564 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-06T02:23:23.570 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-06T02:23:23.570 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-06T02:23:23.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.848+0000 7f029bfff700 1 Processor -- start 2023-10-06T02:23:23.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.849+0000 7f029bfff700 1 -- start start 2023-10-06T02:23:23.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.851+0000 7f029bfff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f02940dbd10 0x7f0294005f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:23.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.851+0000 7f029bfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f02940341a0 0x7f02940de980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:23.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.851+0000 7f029bfff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0294053310 con 0x7f02940dc0e0 2023-10-06T02:23:23.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.851+0000 7f029bfff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0294053bb0 con 0x7f02940341a0 2023-10-06T02:23:23.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.851+0000 7f029bfff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02940df6b0 con 0x7f02940dbd10 2023-10-06T02:23:23.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.852+0000 7f029b7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f02940341a0 0x7f02940de980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:23.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.852+0000 7f029b7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f02940341a0 0x7f02940de980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37664/0 (socket says 172.21.15.31:37664) 2023-10-06T02:23:23.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.852+0000 7f029b7fe700 1 -- 172.21.15.31:0/2082154788 learned_addr learned my addr 172.21.15.31:0/2082154788 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:23.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.853+0000 7f029b7fe700 1 -- 172.21.15.31:0/2082154788 >> v2:172.21.15.62:3300/0 conn(0x7f02940dbd10 msgr2=0x7f0294005f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:23.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.853+0000 7f029b7fe700 1 --2- 172.21.15.31:0/2082154788 >> v2:172.21.15.62:3300/0 conn(0x7f02940dbd10 0x7f0294005f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:23.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.853+0000 7f029b7fe700 1 -- 172.21.15.31:0/2082154788 >> v1:172.21.15.62:6789/0 conn(0x7f02940dc0e0 legacy=0x7f0294033a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:23.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.853+0000 7f029b7fe700 1 -- 172.21.15.31:0/2082154788 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0294053560 con 0x7f02940341a0 2023-10-06T02:23:23.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.854+0000 7f029b7fe700 1 --2- 172.21.15.31:0/2082154788 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02940341a0 0x7f02940de980 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f029400d550 tx=0x7f0290009ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac0319e0cf261a06 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:23.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.855+0000 7f0299ffb700 1 -- 172.21.15.31:0/2082154788 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f029000b730 con 0x7f02940341a0 2023-10-06T02:23:23.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.855+0000 7f0299ffb700 1 -- 172.21.15.31:0/2082154788 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f029000b890 con 0x7f02940341a0 2023-10-06T02:23:23.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.855+0000 7f0299ffb700 1 -- 172.21.15.31:0/2082154788 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f029000bc30 con 0x7f02940341a0 2023-10-06T02:23:23.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.856+0000 7f3bd09fe700 1 Processor -- start 2023-10-06T02:23:23.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.856+0000 7f029bfff700 1 -- 172.21.15.31:0/2082154788 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02940341a0 msgr2=0x7f02940de980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:23.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.856+0000 7f029bfff700 1 --2- 172.21.15.31:0/2082154788 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02940341a0 0x7f02940de980 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f029400d550 tx=0x7f0290009ae0 comp rx=0 tx=0).stop 2023-10-06T02:23:23.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.856+0000 7f029bfff700 1 -- 172.21.15.31:0/2082154788 shutdown_connections 2023-10-06T02:23:23.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.857+0000 7f029bfff700 1 --2- 172.21.15.31:0/2082154788 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02940341a0 0x7f02940de980 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:23.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.857+0000 7f029bfff700 1 --2- 172.21.15.31:0/2082154788 >> v2:172.21.15.62:3300/0 conn(0x7f02940dbd10 0x7f0294005f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:23.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.857+0000 7f029bfff700 1 -- 172.21.15.31:0/2082154788 >> 172.21.15.31:0/2082154788 conn(0x7f02940517b0 msgr2=0x7f0294051bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:23.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.856+0000 7f3bd09fe700 1 -- start start 2023-10-06T02:23:23.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.858+0000 7f3ae89fe700 1 Processor -- start 2023-10-06T02:23:23.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.859+0000 7f3bd09fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3bc40e0170 0x7f3bc4035a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:23.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.859+0000 7f3bd09fe700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3bc400cc40 0x7f3bc4051fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:23.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.859+0000 7f3bd09fe700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bc4054e50 con 0x7f3bc40524e0 2023-10-06T02:23:23.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.859+0000 7f3bd09fe700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bc4054b20 con 0x7f3bc40e0170 2023-10-06T02:23:23.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.859+0000 7f3bd09fe700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bc4036030 con 0x7f3bc400cc40 2023-10-06T02:23:23.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.860+0000 7f3bcb59e700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f3bc40524e0 0x7f3bc40e0540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49328/0 (socket says 172.21.15.31:49328) 2023-10-06T02:23:23.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.860+0000 7f3bcb59e700 1 -- 172.21.15.31:0/1120058010 learned_addr learned my addr 172.21.15.31:0/1120058010 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:23:23.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.860+0000 7f3bcad9d700 1 --2- 172.21.15.31:0/1120058010 >> v2:172.21.15.31:3300/0 conn(0x7f3bc40e0170 0x7f3bc4035a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:23.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.861+0000 7f3bca59c700 1 --2- 172.21.15.31:0/1120058010 >> v2:172.21.15.62:3300/0 conn(0x7f3bc400cc40 0x7f3bc4051fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:23.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.860+0000 7f3ae89fe700 1 -- start start 2023-10-06T02:23:23.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.862+0000 7f3bca59c700 1 -- 172.21.15.31:0/1120058010 >> v2:172.21.15.31:3300/0 conn(0x7f3bc40e0170 msgr2=0x7f3bc4035a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:23.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.862+0000 7f3bc9d9b700 1 -- 172.21.15.31:0/1120058010 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4269567902 0 0) 0x7f3bc4054e50 con 0x7f3bc40524e0 2023-10-06T02:23:23.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.857+0000 7f029bfff700 1 -- 172.21.15.31:0/2082154788 shutdown_connections 2023-10-06T02:23:23.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.857+0000 7f029bfff700 1 -- 172.21.15.31:0/2082154788 wait complete. 2023-10-06T02:23:23.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.859+0000 7f029bfff700 1 Processor -- start 2023-10-06T02:23:23.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.860+0000 7f029bfff700 1 -- start start 2023-10-06T02:23:23.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.862+0000 7f029bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02940341a0 0x7f0294109c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:23.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.862+0000 7f029bfff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f02940dbd10 0x7f02940e2e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:23.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.862+0000 7f029bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02940e3380 0x7f02940dcae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:23.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.862+0000 7f029bfff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0294038af0 con 0x7f02940e3380 2023-10-06T02:23:23.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.862+0000 7f029bfff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02940e29b0 con 0x7f02940341a0 2023-10-06T02:23:23.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.862+0000 7f029bfff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02940e2080 con 0x7f02940dbd10 2023-10-06T02:23:23.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.862+0000 7f3bca59c700 1 --2- 172.21.15.31:0/1120058010 >> v2:172.21.15.31:3300/0 conn(0x7f3bc40e0170 0x7f3bc4035a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:23.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.862+0000 7f3bca59c700 1 -- 172.21.15.31:0/1120058010 >> v1:172.21.15.31:6789/0 conn(0x7f3bc40524e0 legacy=0x7f3bc40e0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:23.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.862+0000 7f3bca59c700 1 -- 172.21.15.31:0/1120058010 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bc4036310 con 0x7f3bc400cc40 2023-10-06T02:23:23.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.862+0000 7f3bcad9d700 1 --2- 172.21.15.31:0/1120058010 >> v2:172.21.15.31:3300/0 conn(0x7f3bc40e0170 0x7f3bc4035a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:23:23.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.862+0000 7f3ae89fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3adc0dff60 0x7f3adc035f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:23.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.862+0000 7f3ae89fe700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3adc055ec0 0x7f3adc051e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:23.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.862+0000 7f3ae89fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3adc0e0330 0x7f3adc052370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:23.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.862+0000 7f3ae89fe700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3adc005f10 con 0x7f3adc0dff60 2023-10-06T02:23:23.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.862+0000 7f3ae89fe700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3adc0061b0 con 0x7f3adc055ec0 2023-10-06T02:23:23.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.862+0000 7f3ae89fe700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3adc036dd0 con 0x7f3adc0e0330 2023-10-06T02:23:23.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.863+0000 7f029affd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02940341a0 0x7f0294109c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:23.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.863+0000 7f3ae2d9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3adc0dff60 0x7f3adc035f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:23.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.863+0000 7f3ae2d9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3adc0dff60 0x7f3adc035f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37694/0 (socket says 172.21.15.31:37694) 2023-10-06T02:23:23.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.863+0000 7f3ae2d9d700 1 -- 172.21.15.31:0/12151072 learned_addr learned my addr 172.21.15.31:0/12151072 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:23.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.863+0000 7f3ae359e700 1 --2- 172.21.15.31:0/12151072 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3adc0e0330 0x7f3adc052370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:23.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.863+0000 7f029affd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02940341a0 0x7f0294109c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:47440/0 (socket says 172.21.15.31:47440) 2023-10-06T02:23:23.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.863+0000 7f029affd700 1 -- 172.21.15.31:0/241787915 learned_addr learned my addr 172.21.15.31:0/241787915 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:23.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.863+0000 7f3bca59c700 1 --2- 172.21.15.31:0/1120058010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3bc400cc40 0x7f3bc4051fa0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3bc400c050 tx=0x7f3bc00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92fd2ce50b2a8b83 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:23.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.863+0000 7f029b7fe700 1 --2- 172.21.15.31:0/241787915 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02940e3380 0x7f02940dcae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:23.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.864+0000 7f029a7fc700 1 --2- 172.21.15.31:0/241787915 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f02940dbd10 0x7f02940e2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:23.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.864+0000 7f3ae259c700 1 --2- 172.21.15.31:0/12151072 >> v2:172.21.15.62:3300/0 conn(0x7f3adc055ec0 0x7f3adc051e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:23.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.864+0000 7f3ae359e700 1 -- 172.21.15.31:0/12151072 >> v2:172.21.15.62:3300/0 conn(0x7f3adc055ec0 msgr2=0x7f3adc051e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:23.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.864+0000 7f3ae359e700 1 --2- 172.21.15.31:0/12151072 >> v2:172.21.15.62:3300/0 conn(0x7f3adc055ec0 0x7f3adc051e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:23.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.864+0000 7f3ae359e700 1 -- 172.21.15.31:0/12151072 >> v2:172.21.15.31:3300/0 conn(0x7f3adc0dff60 msgr2=0x7f3adc035f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:23.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.864+0000 7f3ae359e700 1 --2- 172.21.15.31:0/12151072 >> v2:172.21.15.31:3300/0 conn(0x7f3adc0dff60 0x7f3adc035f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:23.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.864+0000 7f3ae359e700 1 -- 172.21.15.31:0/12151072 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3adc0563a0 con 0x7f3adc0e0330 2023-10-06T02:23:23.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.864+0000 7f3bc9d9b700 1 -- 172.21.15.31:0/1120058010 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc0011630 con 0x7f3bc400cc40 2023-10-06T02:23:23.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.865+0000 7f029b7fe700 1 -- 172.21.15.31:0/241787915 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f02940dbd10 msgr2=0x7f02940e2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:23.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.865+0000 7f029b7fe700 1 --2- 172.21.15.31:0/241787915 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f02940dbd10 0x7f02940e2e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:23.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.865+0000 7f3bc9d9b700 1 -- 172.21.15.31:0/1120058010 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bc0011790 con 0x7f3bc400cc40 2023-10-06T02:23:23.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.865+0000 7f029b7fe700 1 -- 172.21.15.31:0/241787915 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02940341a0 msgr2=0x7f0294109c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:23.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.865+0000 7f029b7fe700 1 --2- 172.21.15.31:0/241787915 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02940341a0 0x7f0294109c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:23.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.865+0000 7f029b7fe700 1 -- 172.21.15.31:0/241787915 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0290009d80 con 0x7f02940e3380 2023-10-06T02:23:23.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.865+0000 7f3ae359e700 1 --2- 172.21.15.31:0/12151072 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3adc0e0330 0x7f3adc052370 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3ae4000eb0 tx=0x7f3ae40673b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab91eb79c3affacc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:23.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.865+0000 7f3bc9d9b700 1 -- 172.21.15.31:0/1120058010 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc0011b30 con 0x7f3bc400cc40 2023-10-06T02:23:23.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.865+0000 7f029b7fe700 1 --2- 172.21.15.31:0/241787915 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02940e3380 0x7f02940dcae0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f029400ddf0 tx=0x7f0290018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:23.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.865+0000 7f3ae1d9b700 1 -- 172.21.15.31:0/12151072 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ae40008d0 con 0x7f3adc0e0330 2023-10-06T02:23:23.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.865+0000 7f3ae1d9b700 1 -- 172.21.15.31:0/12151072 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ae4064550 con 0x7f3adc0e0330 2023-10-06T02:23:23.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.866+0000 7f3bd09fe700 1 -- 172.21.15.31:0/1120058010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3bc400cc40 msgr2=0x7f3bc4051fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:23.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.866+0000 7f3bd09fe700 1 --2- 172.21.15.31:0/1120058010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3bc400cc40 0x7f3bc4051fa0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3bc400c050 tx=0x7f3bc00091a0 comp rx=0 tx=0).stop 2023-10-06T02:23:23.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.866+0000 7f3bd09fe700 1 -- 172.21.15.31:0/1120058010 shutdown_connections 2023-10-06T02:23:23.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.866+0000 7f3bd09fe700 1 --2- 172.21.15.31:0/1120058010 >> v2:172.21.15.31:3300/0 conn(0x7f3bc40e0170 0x7f3bc4035a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:23.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.866+0000 7f3bd09fe700 1 --2- 172.21.15.31:0/1120058010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3bc400cc40 0x7f3bc4051fa0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:23.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.866+0000 7f3bd09fe700 1 -- 172.21.15.31:0/1120058010 >> 172.21.15.31:0/1120058010 conn(0x7f3bc4057a20 msgr2=0x7f3bc400c320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:23.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.866+0000 7f3ae89fe700 1 -- 172.21.15.31:0/12151072 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3adc0e0330 msgr2=0x7f3adc052370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:23.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.866+0000 7f3ae89fe700 1 --2- 172.21.15.31:0/12151072 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3adc0e0330 0x7f3adc052370 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3ae4000eb0 tx=0x7f3ae40673b0 comp rx=0 tx=0).stop 2023-10-06T02:23:23.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.867+0000 7f3ae89fe700 1 -- 172.21.15.31:0/12151072 shutdown_connections 2023-10-06T02:23:23.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.867+0000 7f3bd09fe700 1 -- 172.21.15.31:0/1120058010 shutdown_connections 2023-10-06T02:23:23.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.867+0000 7f3ae89fe700 1 --2- 172.21.15.31:0/12151072 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3adc0e0330 0x7f3adc052370 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:23.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.867+0000 7f3ae89fe700 1 --2- 172.21.15.31:0/12151072 >> v2:172.21.15.31:3300/0 conn(0x7f3adc0dff60 0x7f3adc035f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:23.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.867+0000 7f3ae89fe700 1 --2- 172.21.15.31:0/12151072 >> v2:172.21.15.62:3300/0 conn(0x7f3adc055ec0 0x7f3adc051e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:23.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.867+0000 7f3ae89fe700 1 -- 172.21.15.31:0/12151072 >> 172.21.15.31:0/12151072 conn(0x7f3adc0539c0 msgr2=0x7f3adc053dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:23.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.867+0000 7f3ae89fe700 1 -- 172.21.15.31:0/12151072 shutdown_connections 2023-10-06T02:23:23.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.867+0000 7f3bd09fe700 1 -- 172.21.15.31:0/1120058010 wait complete. 2023-10-06T02:23:23.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.870+0000 7f3bd09fe700 1 Processor -- start 2023-10-06T02:23:23.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.870+0000 7f3bd09fe700 1 -- start start 2023-10-06T02:23:23.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.872+0000 7f3bd09fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3bc400cc40 0x7f3bc4039cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:23.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.872+0000 7f3bd09fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3bc40524e0 0x7f3bc40dcad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:23.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.873+0000 7f3bcad9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3bc400cc40 0x7f3bc4039cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:23.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.873+0000 7f3bcad9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3bc400cc40 0x7f3bc4039cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:57214/0 (socket says 172.21.15.31:57214) 2023-10-06T02:23:23.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.873+0000 7f3bcad9d700 1 -- 172.21.15.31:0/2028734760 learned_addr learned my addr 172.21.15.31:0/2028734760 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:23.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.873+0000 7f3bd09fe700 1 --2- 172.21.15.31:0/2028734760 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3bc40e0170 0x7f3bc40ff690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:23.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.873+0000 7f3bca59c700 1 --2- 172.21.15.31:0/2028734760 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3bc40524e0 0x7f3bc40dcad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:23.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.868+0000 7f3ae89fe700 1 -- 172.21.15.31:0/12151072 wait complete. 2023-10-06T02:23:23.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.870+0000 7f3ae89fe700 1 Processor -- start 2023-10-06T02:23:23.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.870+0000 7f3ae89fe700 1 -- start start 2023-10-06T02:23:23.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.872+0000 7f3ae89fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3adc055ec0 0x7f3adc006890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:23.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.872+0000 7f3ae89fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3adc0dff60 0x7f3adc10efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:23.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.872+0000 7f3ae89fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3adc10f510 0x7f3adc04e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:23.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.872+0000 7f3ae89fe700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3adc04f6b0 con 0x7f3adc055ec0 2023-10-06T02:23:23.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.872+0000 7f3ae89fe700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3adc04ee10 con 0x7f3adc0dff60 2023-10-06T02:23:23.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.873+0000 7f3ae89fe700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3adc04f0f0 con 0x7f3adc10f510 2023-10-06T02:23:23.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.873+0000 7f3ae259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3adc0dff60 0x7f3adc10efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:23.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.873+0000 7f3ae259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3adc0dff60 0x7f3adc10efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:47454/0 (socket says 172.21.15.31:47454) 2023-10-06T02:23:23.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.873+0000 7f3ae259c700 1 -- 172.21.15.31:0/2496158553 learned_addr learned my addr 172.21.15.31:0/2496158553 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:23.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.874+0000 7f3bd09fe700 1 -- 172.21.15.31:0/2028734760 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc40dd010 con 0x7f3bc40e0170 2023-10-06T02:23:23.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.874+0000 7f3bd09fe700 1 -- 172.21.15.31:0/2028734760 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc40ffbd0 con 0x7f3bc40524e0 2023-10-06T02:23:23.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.874+0000 7f3bd09fe700 1 -- 172.21.15.31:0/2028734760 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc403a2d0 con 0x7f3bc400cc40 2023-10-06T02:23:23.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.874+0000 7f3bcb59e700 1 --2- 172.21.15.31:0/2028734760 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3bc40e0170 0x7f3bc40ff690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:23.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.874+0000 7f3bcad9d700 1 -- 172.21.15.31:0/2028734760 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3bc40524e0 msgr2=0x7f3bc40dcad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:23.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.874+0000 7f3bcad9d700 1 --2- 172.21.15.31:0/2028734760 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3bc40524e0 0x7f3bc40dcad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:23.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.875+0000 7f3ae359e700 1 --2- 172.21.15.31:0/2496158553 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3adc10f510 0x7f3adc04e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:23.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.874+0000 7f3bcad9d700 1 -- 172.21.15.31:0/2028734760 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3bc40e0170 msgr2=0x7f3bc40ff690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:23.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.874+0000 7f3bcad9d700 1 --2- 172.21.15.31:0/2028734760 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3bc40e0170 0x7f3bc40ff690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:23.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.867+0000 7f0282ffd700 1 -- 172.21.15.31:0/241787915 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0290025070 con 0x7f02940e3380 2023-10-06T02:23:23.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.867+0000 7f0282ffd700 1 -- 172.21.15.31:0/241787915 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f029000bbd0 con 0x7f02940e3380 2023-10-06T02:23:23.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.867+0000 7f0282ffd700 1 -- 172.21.15.31:0/241787915 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0290016800 con 0x7f02940e3380 2023-10-06T02:23:23.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.868+0000 7f029bfff700 1 -- 172.21.15.31:0/241787915 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02940e2360 con 0x7f02940e3380 2023-10-06T02:23:23.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.869+0000 7f029bfff700 1 -- 172.21.15.31:0/241787915 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f029410a180 con 0x7f02940e3380 2023-10-06T02:23:23.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.873+0000 7f0282ffd700 1 -- 172.21.15.31:0/241787915 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0290016bf0 con 0x7f02940e3380 2023-10-06T02:23:23.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.875+0000 7f0282ffd700 1 --2- 172.21.15.31:0/241787915 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f0288067080 0x7f0288069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:23.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.875+0000 7f0282ffd700 1 -- 172.21.15.31:0/241787915 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0290013070 con 0x7f02940e3380 2023-10-06T02:23:23.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.875+0000 7f3ae2d9d700 1 --2- 172.21.15.31:0/2496158553 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3adc055ec0 0x7f3adc006890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:23.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.874+0000 7f3bcad9d700 1 -- 172.21.15.31:0/2028734760 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bbc009420 con 0x7f3bc400cc40 2023-10-06T02:23:23.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.875+0000 7f3bcad9d700 1 --2- 172.21.15.31:0/2028734760 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3bc400cc40 0x7f3bc4039cf0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc00ecb0 tx=0x7f3bbc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:23.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.876+0000 7f029affd700 1 --2- 172.21.15.31:0/241787915 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f0288067080 0x7f0288069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:23.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.876+0000 7f3ae359e700 1 -- 172.21.15.31:0/2496158553 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3adc0dff60 msgr2=0x7f3adc10efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:23.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.876+0000 7f3ae359e700 1 --2- 172.21.15.31:0/2496158553 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3adc0dff60 0x7f3adc10efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:23.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.876+0000 7f3ae359e700 1 -- 172.21.15.31:0/2496158553 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3adc055ec0 msgr2=0x7f3adc006890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:23.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.876+0000 7f3ae359e700 1 --2- 172.21.15.31:0/2496158553 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3adc055ec0 0x7f3adc006890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:23.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.876+0000 7f3bb2ffd700 1 -- 172.21.15.31:0/2028734760 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc00c950 con 0x7f3bc400cc40 2023-10-06T02:23:23.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.876+0000 7f3ae359e700 1 -- 172.21.15.31:0/2496158553 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3adc04f3d0 con 0x7f3adc10f510 2023-10-06T02:23:23.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.876+0000 7f3bb2ffd700 1 -- 172.21.15.31:0/2028734760 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bbc00cab0 con 0x7f3bc400cc40 2023-10-06T02:23:23.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.877+0000 7f3ae2d9d700 1 --2- 172.21.15.31:0/2496158553 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3adc055ec0 0x7f3adc006890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:23:23.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.877+0000 7f3bb2ffd700 1 -- 172.21.15.31:0/2028734760 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc00ce30 con 0x7f3bc400cc40 2023-10-06T02:23:23.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.877+0000 7f3bd09fe700 1 -- 172.21.15.31:0/2028734760 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bc0009420 con 0x7f3bc400cc40 2023-10-06T02:23:23.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.877+0000 7f029bfff700 1 -- 172.21.15.31:0/241787915 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02940e21c0 con 0x7f02940e3380 2023-10-06T02:23:23.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.877+0000 7f3bd09fe700 1 -- 172.21.15.31:0/2028734760 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bc403a890 con 0x7f3bc400cc40 2023-10-06T02:23:23.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.878+0000 7f029affd700 1 --2- 172.21.15.31:0/241787915 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f0288067080 0x7f0288069530 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f028c006fd0 tx=0x7f028c009250 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:23.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.878+0000 7fa5ff9ff700 1 Processor -- start 2023-10-06T02:23:23.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.878+0000 7fa5ff9ff700 1 -- start start 2023-10-06T02:23:23.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.877+0000 7f3ae359e700 1 --2- 172.21.15.31:0/2496158553 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3adc10f510 0x7f3adc04e0e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3ae40614f0 tx=0x7f3ae4068000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:23.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.877+0000 7f3acaffd700 1 -- 172.21.15.31:0/2496158553 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ae406db00 con 0x7f3adc10f510 2023-10-06T02:23:23.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.877+0000 7f3acaffd700 1 -- 172.21.15.31:0/2496158553 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ae406dc60 con 0x7f3adc10f510 2023-10-06T02:23:23.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.877+0000 7f3acaffd700 1 -- 172.21.15.31:0/2496158553 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ae406db00 con 0x7f3adc10f510 2023-10-06T02:23:23.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.879+0000 7f3ae89fe700 1 -- 172.21.15.31:0/2496158553 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3adc0dd900 con 0x7f3adc10f510 2023-10-06T02:23:23.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.879+0000 7f3ae89fe700 1 -- 172.21.15.31:0/2496158553 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3adc006dd0 con 0x7f3adc10f510 2023-10-06T02:23:23.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.879+0000 7f3bb0ff9700 1 -- 172.21.15.31:0/2028734760 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bc4005820 con 0x7f3bc400cc40 2023-10-06T02:23:23.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.881+0000 7f3ac8ff9700 1 -- 172.21.15.31:0/2496158553 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3adc0dd760 con 0x7f3adc10f510 2023-10-06T02:23:23.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.881+0000 7fa5ff9ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa5f806bc20 0x7fa5f80ac830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:23.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.881+0000 7fa5ff9ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa5f806eb00 0x7fa5f80cbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:23.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.881+0000 7fa5ff9ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5f80abd50 con 0x7fa5f80cb720 2023-10-06T02:23:23.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.881+0000 7fa5ff9ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5f80cc100 con 0x7fa5f806bc20 2023-10-06T02:23:23.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.881+0000 7fa5ff9ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f80b20a0 con 0x7fa5f806eb00 2023-10-06T02:23:23.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.881+0000 7f3bb2ffd700 1 -- 172.21.15.31:0/2028734760 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3bbc005a00 con 0x7f3bc400cc40 2023-10-06T02:23:23.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.882+0000 7fa5fd79b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa5f806bc20 0x7fa5f80ac830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:23.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.882+0000 7fa5fd79b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa5f806bc20 0x7fa5f80ac830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37714/0 (socket says 172.21.15.31:37714) 2023-10-06T02:23:23.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.882+0000 7fa5fd79b700 1 -- 172.21.15.31:0/4290381141 learned_addr learned my addr 172.21.15.31:0/4290381141 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:23.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.882+0000 7fa5fcf9a700 1 --1- 172.21.15.31:0/4290381141 >> v1:172.21.15.31:6789/0 conn(0x7fa5f80cb720 0x7fa5f806e3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49330/0 (socket says 172.21.15.31:49330) 2023-10-06T02:23:23.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.883+0000 7fa5fd79b700 1 -- 172.21.15.31:0/4290381141 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa5f806eb00 msgr2=0x7fa5f80cbbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:23.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.883+0000 7fa5fdf9c700 1 --2- 172.21.15.31:0/4290381141 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa5f806eb00 0x7fa5f80cbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:23.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.883+0000 7fa5fd79b700 1 --2- 172.21.15.31:0/4290381141 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa5f806eb00 0x7fa5f80cbbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:23.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.883+0000 7fa5fd79b700 1 -- 172.21.15.31:0/4290381141 >> v1:172.21.15.31:6789/0 conn(0x7fa5f80cb720 legacy=0x7fa5f806e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:23.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.883+0000 7fa5fd79b700 1 -- 172.21.15.31:0/4290381141 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5f80ac030 con 0x7fa5f806bc20 2023-10-06T02:23:23.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.883+0000 7f3bb2ffd700 1 --2- 172.21.15.31:0/2028734760 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f3bb8067080 0x7f3bb8069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:23.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.883+0000 7f3bb2ffd700 1 -- 172.21.15.31:0/2028734760 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3bbc084b10 con 0x7f3bc400cc40 2023-10-06T02:23:23.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.883+0000 7f3bca59c700 1 --2- 172.21.15.31:0/2028734760 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f3bb8067080 0x7f3bb8069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:23.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.884+0000 7fa5fdf9c700 1 --2- 172.21.15.31:0/4290381141 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa5f806eb00 0x7fa5f80cbbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:23:23.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.884+0000 7fa5fd79b700 1 --2- 172.21.15.31:0/4290381141 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5f806bc20 0x7fa5f80ac830 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa5f0009f20 tx=0x7fa5f0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfda3a09c0cb4a96 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:23.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.885+0000 7f3bca59c700 1 --2- 172.21.15.31:0/2028734760 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f3bb8067080 0x7f3bb8069530 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3bc00093f0 tx=0x7f3bc0017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:23.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.885+0000 7f3bb2ffd700 1 -- 172.21.15.31:0/2028734760 <== mon.2 v2:172.21.15.62:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3bbc055710 con 0x7f3bc400cc40 2023-10-06T02:23:23.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.886+0000 7fa5effff700 1 -- 172.21.15.31:0/4290381141 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f000e070 con 0x7fa5f806bc20 2023-10-06T02:23:23.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.887+0000 7fa5effff700 1 -- 172.21.15.31:0/4290381141 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5f000ba80 con 0x7fa5f806bc20 2023-10-06T02:23:23.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.887+0000 7fa5effff700 1 -- 172.21.15.31:0/4290381141 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f00092e0 con 0x7fa5f806bc20 2023-10-06T02:23:23.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.887+0000 7f3acaffd700 1 -- 172.21.15.31:0/2496158553 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3ae4096020 con 0x7f3adc10f510 2023-10-06T02:23:23.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.888+0000 7fa5ff9ff700 1 -- 172.21.15.31:0/4290381141 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5f806bc20 msgr2=0x7fa5f80ac830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:23.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.888+0000 7fa5ff9ff700 1 --2- 172.21.15.31:0/4290381141 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5f806bc20 0x7fa5f80ac830 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa5f0009f20 tx=0x7fa5f0009840 comp rx=0 tx=0).stop 2023-10-06T02:23:23.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.888+0000 7f3acaffd700 1 --2- 172.21.15.31:0/2496158553 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f3ad0066ff0 0x7f3ad00694a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:23.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.888+0000 7f3acaffd700 1 -- 172.21.15.31:0/2496158553 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3ae40f1f90 con 0x7f3adc10f510 2023-10-06T02:23:23.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.887+0000 7f0282ffd700 1 -- 172.21.15.31:0/241787915 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f02940e21c0 con 0x7f02940e3380 2023-10-06T02:23:23.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.888+0000 7fa5ff9ff700 1 -- 172.21.15.31:0/4290381141 shutdown_connections 2023-10-06T02:23:23.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.888+0000 7fa5ff9ff700 1 --2- 172.21.15.31:0/4290381141 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5f806bc20 0x7fa5f80ac830 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:23.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.888+0000 7fa5ff9ff700 1 --2- 172.21.15.31:0/4290381141 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa5f806eb00 0x7fa5f80cbbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:23.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.888+0000 7fa5ff9ff700 1 -- 172.21.15.31:0/4290381141 >> 172.21.15.31:0/4290381141 conn(0x7fa5f80cc300 msgr2=0x7fa5f80cc700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:23.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.889+0000 7fa5ff9ff700 1 -- 172.21.15.31:0/4290381141 shutdown_connections 2023-10-06T02:23:23.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.889+0000 7fa5ff9ff700 1 -- 172.21.15.31:0/4290381141 wait complete. 2023-10-06T02:23:23.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.889+0000 7f3ae2d9d700 1 --2- 172.21.15.31:0/2496158553 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f3ad0066ff0 0x7f3ad00694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:23.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.890+0000 7f3ae2d9d700 1 --2- 172.21.15.31:0/2496158553 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f3ad0066ff0 0x7f3ad00694a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3ad8006ce0 tx=0x7f3ad8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:23.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.890+0000 7f3acaffd700 1 -- 172.21.15.31:0/2496158553 <== mon.2 v2:172.21.15.62:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3ae40bf000 con 0x7f3adc10f510 2023-10-06T02:23:23.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.891+0000 7fa5ff9ff700 1 Processor -- start 2023-10-06T02:23:23.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.892+0000 7fa5ff9ff700 1 -- start start 2023-10-06T02:23:23.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.893+0000 7fa5ff9ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa5f806bc20 0x7fa5f80a9130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:23.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.893+0000 7fa5ff9ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa5f806eb00 0x7fa5f80a7670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:23.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.893+0000 7fa5ff9ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5f80cb720 0x7fa5f8170a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:23.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.893+0000 7fa5ff9ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f81535a0 con 0x7fa5f80cb720 2023-10-06T02:23:23.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.893+0000 7fa5fd79b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa5f806bc20 0x7fa5f80a9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:23.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.893+0000 7fa5fd79b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa5f806bc20 0x7fa5f80a9130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:47470/0 (socket says 172.21.15.31:47470) 2023-10-06T02:23:23.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.894+0000 7fa5fd79b700 1 -- 172.21.15.31:0/2877535116 learned_addr learned my addr 172.21.15.31:0/2877535116 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:23.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.894+0000 7fa5fcf9a700 1 --2- 172.21.15.31:0/2877535116 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa5f806eb00 0x7fa5f80a7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:23.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.893+0000 7fa5ff9ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f81537b0 con 0x7fa5f806bc20 2023-10-06T02:23:23.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.894+0000 7fa5ff9ff700 1 -- 172.21.15.31:0/2877535116 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f8153a90 con 0x7fa5f806eb00 2023-10-06T02:23:23.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.894+0000 7fa5fd79b700 1 -- 172.21.15.31:0/2877535116 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa5f806eb00 msgr2=0x7fa5f80a7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:23.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.894+0000 7fa5fd79b700 1 --2- 172.21.15.31:0/2877535116 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa5f806eb00 0x7fa5f80a7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:23.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.894+0000 7fa5fd79b700 1 -- 172.21.15.31:0/2877535116 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5f80cb720 msgr2=0x7fa5f8170a40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T02:23:23.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.894+0000 7fa5fd79b700 1 --2- 172.21.15.31:0/2877535116 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5f80cb720 0x7fa5f8170a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:23.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.894+0000 7fa5fd79b700 1 -- 172.21.15.31:0/2877535116 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5f0009ae0 con 0x7fa5f806bc20 2023-10-06T02:23:23.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.895+0000 7fa5fd79b700 1 --2- 172.21.15.31:0/2877535116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa5f806bc20 0x7fa5f80a9130 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa5f0009410 tx=0x7fa5f0018000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:23.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.896+0000 7fa5ed7fa700 1 -- 172.21.15.31:0/2877535116 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f0008ac0 con 0x7fa5f806bc20 2023-10-06T02:23:23.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.896+0000 7fa5ff9ff700 1 -- 172.21.15.31:0/2877535116 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5f8153d70 con 0x7fa5f806bc20 2023-10-06T02:23:23.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.896+0000 7fa5ff9ff700 1 -- 172.21.15.31:0/2877535116 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5f8154100 con 0x7fa5f806bc20 2023-10-06T02:23:23.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.898+0000 7fa5ed7fa700 1 -- 172.21.15.31:0/2877535116 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5f0008c20 con 0x7fa5f806bc20 2023-10-06T02:23:23.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.898+0000 7fa5ff9ff700 1 -- 172.21.15.31:0/2877535116 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5f8154570 con 0x7fa5f806bc20 2023-10-06T02:23:23.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.899+0000 7fa5ed7fa700 1 -- 172.21.15.31:0/2877535116 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f0025460 con 0x7fa5f806bc20 2023-10-06T02:23:23.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.900+0000 7fa5ed7fa700 1 -- 172.21.15.31:0/2877535116 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa5f0003040 con 0x7fa5f806bc20 2023-10-06T02:23:23.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.902+0000 7fa5ed7fa700 1 --2- 172.21.15.31:0/2877535116 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa5dc067080 0x7fa5dc069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:23.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.903+0000 7fa5fcf9a700 1 --2- 172.21.15.31:0/2877535116 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa5dc067080 0x7fa5dc069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:23.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.904+0000 7fa5fcf9a700 1 --2- 172.21.15.31:0/2877535116 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa5dc067080 0x7fa5dc069530 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa5f81449b0 tx=0x7fa5f4007eb0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:23.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.905+0000 7fa5ed7fa700 1 -- 172.21.15.31:0/2877535116 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa5f00862a0 con 0x7fa5f806bc20 2023-10-06T02:23:23.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.906+0000 7fa5ed7fa700 1 -- 172.21.15.31:0/2877535116 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa5f0086690 con 0x7fa5f806bc20 2023-10-06T02:23:23.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.944+0000 7f929e99d700 1 Processor -- start 2023-10-06T02:23:23.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.945+0000 7f929e99d700 1 -- start start 2023-10-06T02:23:23.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.946+0000 7fa742d9d700 1 Processor -- start 2023-10-06T02:23:23.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.946+0000 7fa742d9d700 1 -- start start 2023-10-06T02:23:23.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.948+0000 7f929d99b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f9298153c30 0x7f929815a5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49334/0 (socket says 172.21.15.31:49334) 2023-10-06T02:23:23.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.945+0000 7fbb8d59d700 1 Processor -- start 2023-10-06T02:23:23.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.946+0000 7fbb8d59d700 1 -- start start 2023-10-06T02:23:23.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.948+0000 7fbb8d59d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fbb80053e80 0x7fbb8003b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:23.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.948+0000 7fbb8d59d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb80006b70 con 0x7fbb800dd620 2023-10-06T02:23:23.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.948+0000 7fbb8d59d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb8003ba10 con 0x7fbb8003af50 2023-10-06T02:23:23.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.948+0000 7fbb8d59d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb80006240 con 0x7fbb80053e80 2023-10-06T02:23:23.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.948+0000 7f929d99b700 1 -- 172.21.15.31:0/3929058211 learned_addr learned my addr 172.21.15.31:0/3929058211 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:23:23.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.949+0000 7f929e99d700 1 --2- 172.21.15.31:0/3929058211 >> v2:172.21.15.31:3300/0 conn(0x7f929815de50 0x7f92980cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:23.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.949+0000 7f929d19a700 1 --2- 172.21.15.31:0/3929058211 >> v2:172.21.15.31:3300/0 conn(0x7f929815de50 0x7f92980cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:23.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.950+0000 7f929e99d700 1 --2- 172.21.15.31:0/3929058211 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f92980cef40 0x7f929815c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:23.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.950+0000 7f929e99d700 1 -- 172.21.15.31:0/3929058211 --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92980cf3c0 con 0x7f9298153c30 2023-10-06T02:23:23.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.950+0000 7f929e99d700 1 -- 172.21.15.31:0/3929058211 --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f929815afd0 con 0x7f929815de50 2023-10-06T02:23:23.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.950+0000 7f929e19c700 1 --2- 172.21.15.31:0/3929058211 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f92980cef40 0x7f929815c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:23.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.950+0000 7f929e99d700 1 -- 172.21.15.31:0/3929058211 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f929815b2b0 con 0x7f92980cef40 2023-10-06T02:23:23.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.949+0000 7fbb8cd9c700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fbb80053e80 0x7fbb8003b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:23.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.949+0000 7fbb8cd9c700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fbb80053e80 0x7fbb8003b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:47486/0 (socket says 172.21.15.31:47486) 2023-10-06T02:23:23.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.949+0000 7fbb8cd9c700 1 -- 172.21.15.31:0/1332014186 learned_addr learned my addr 172.21.15.31:0/1332014186 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:23.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.949+0000 7fbb8cd9c700 1 -- 172.21.15.31:0/1332014186 >> v1:172.21.15.62:6789/0 conn(0x7fbb8003af50 legacy=0x7fbb80053770 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T02:23:23.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.949+0000 7fbb8cd9c700 1 -- 172.21.15.31:0/1332014186 >> v1:172.21.15.31:6789/0 conn(0x7fbb800dd620 legacy=0x7fbb80056d00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T02:23:23.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.950+0000 7fbb8cd9c700 1 -- 172.21.15.31:0/1332014186 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb80006520 con 0x7fbb80053e80 2023-10-06T02:23:23.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.951+0000 7fbb8cd9c700 1 --2- 172.21.15.31:0/1332014186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb80053e80 0x7fbb8003b3f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbb88065460 tx=0x7fbb880673b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4f5cc8916ba0169 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:23.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.948+0000 7fa742d9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa73c06e060 0x7fa73c0ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:23.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.948+0000 7fa742d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa73c0cabd0 0x7fa73c0caff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:23.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.948+0000 7fa742d9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa73c0ccf40 con 0x7fa73c1549e0 2023-10-06T02:23:23.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.948+0000 7fa742d9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa73c0cb530 con 0x7fa73c06e060 2023-10-06T02:23:23.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.948+0000 7fa742d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa73c15e8d0 con 0x7fa73c0cabd0 2023-10-06T02:23:23.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.949+0000 7fa74159a700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa73c06e060 0x7fa73c0ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:23.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.950+0000 7fa74159a700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa73c06e060 0x7fa73c0ce160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:47492/0 (socket says 172.21.15.31:47492) 2023-10-06T02:23:23.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.950+0000 7fa74159a700 1 -- 172.21.15.31:0/2840491951 learned_addr learned my addr 172.21.15.31:0/2840491951 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:23.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.950+0000 7fa74259c700 1 --2- 172.21.15.31:0/2840491951 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa73c0cabd0 0x7fa73c0caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:23.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.951+0000 7fa74259c700 1 -- 172.21.15.31:0/2840491951 >> v2:172.21.15.62:3300/0 conn(0x7fa73c06e060 msgr2=0x7fa73c0ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:23.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.951+0000 7fa74259c700 1 --2- 172.21.15.31:0/2840491951 >> v2:172.21.15.62:3300/0 conn(0x7fa73c06e060 0x7fa73c0ce160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:23.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.951+0000 7fa74259c700 1 -- 172.21.15.31:0/2840491951 >> v1:172.21.15.62:6789/0 conn(0x7fa73c1549e0 legacy=0x7fa73c15e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:23.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.951+0000 7fa74259c700 1 -- 172.21.15.31:0/2840491951 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa73c15ebb0 con 0x7fa73c0cabd0 2023-10-06T02:23:23.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.951+0000 7fa74259c700 1 --2- 172.21.15.31:0/2840491951 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa73c0cabd0 0x7fa73c0caff0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa734009e60 tx=0x7fa734009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7be4446a9f004f42 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:23.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.951+0000 7f929c999700 1 -- 172.21.15.31:0/3929058211 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 444058586 0 0) 0x7f92980cf3c0 con 0x7f9298153c30 2023-10-06T02:23:23.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.951+0000 7faec299d700 1 Processor -- start 2023-10-06T02:23:23.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.951+0000 7fbb86ffd700 1 -- 172.21.15.31:0/1332014186 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb880008d0 con 0x7fbb80053e80 2023-10-06T02:23:23.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.951+0000 7fbb86ffd700 1 -- 172.21.15.31:0/1332014186 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb88064550 con 0x7fbb80053e80 2023-10-06T02:23:23.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.952+0000 7f929c999700 1 -- 172.21.15.31:0/3929058211 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9284003570 con 0x7f9298153c30 2023-10-06T02:23:23.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.952+0000 7fbb8d59d700 1 -- 172.21.15.31:0/1332014186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb80053e80 msgr2=0x7fbb8003b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:23.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.952+0000 7fbb8d59d700 1 --2- 172.21.15.31:0/1332014186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb80053e80 0x7fbb8003b3f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbb88065460 tx=0x7fbb880673b0 comp rx=0 tx=0).stop 2023-10-06T02:23:23.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.953+0000 7f929d19a700 1 -- 172.21.15.31:0/3929058211 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f92980cef40 msgr2=0x7f929815c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:23.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.953+0000 7f929d19a700 1 --2- 172.21.15.31:0/3929058211 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f92980cef40 0x7f929815c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:23.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.953+0000 7f929d19a700 1 -- 172.21.15.31:0/3929058211 >> v1:172.21.15.31:6789/0 conn(0x7f9298153c30 legacy=0x7f929815a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:23.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.953+0000 7f929d19a700 1 -- 172.21.15.31:0/3929058211 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f929815b590 con 0x7f929815de50 2023-10-06T02:23:23.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.954+0000 7f929d19a700 1 --2- 172.21.15.31:0/3929058211 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f929815de50 0x7f92980cea00 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f92880098e0 tx=0x7f92880091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e284c7e4ecaf7009 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:23.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.954+0000 7fbb8d59d700 1 -- 172.21.15.31:0/1332014186 shutdown_connections 2023-10-06T02:23:23.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.954+0000 7fbb8d59d700 1 --2- 172.21.15.31:0/1332014186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb80053e80 0x7fbb8003b3f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:23.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.954+0000 7fbb8d59d700 1 -- 172.21.15.31:0/1332014186 >> 172.21.15.31:0/1332014186 conn(0x7fbb80055ba0 msgr2=0x7fbb80055fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:23.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.952+0000 7faec299d700 1 -- start start 2023-10-06T02:23:23.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.954+0000 7faec299d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faebc15b5d0 0x7faebc0c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:23.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.954+0000 7faec299d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7faebc0ca240 0x7faebc0c6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:23.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.954+0000 7faec299d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faebc0ca6b0 con 0x7faebc0c6ec0 2023-10-06T02:23:23.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.954+0000 7faec299d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faebc0cb4f0 con 0x7faebc0ca240 2023-10-06T02:23:23.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.954+0000 7faec299d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faebc1601e0 con 0x7faebc15b5d0 2023-10-06T02:23:23.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.952+0000 7fa740d99700 1 -- 172.21.15.31:0/2840491951 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa73400e070 con 0x7fa73c0cabd0 2023-10-06T02:23:23.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.952+0000 7fa740d99700 1 -- 172.21.15.31:0/2840491951 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa73400bc30 con 0x7fa73c0cabd0 2023-10-06T02:23:23.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.952+0000 7fa740d99700 1 -- 172.21.15.31:0/2840491951 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa734012430 con 0x7fa73c0cabd0 2023-10-06T02:23:23.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.954+0000 7fa742d9d700 1 -- 172.21.15.31:0/2840491951 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa73c0cabd0 msgr2=0x7fa73c0caff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:23.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.954+0000 7fa742d9d700 1 --2- 172.21.15.31:0/2840491951 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa73c0cabd0 0x7fa73c0caff0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa734009e60 tx=0x7fa734009780 comp rx=0 tx=0).stop 2023-10-06T02:23:23.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.954+0000 7fa742d9d700 1 -- 172.21.15.31:0/2840491951 shutdown_connections 2023-10-06T02:23:23.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.954+0000 7fa742d9d700 1 --2- 172.21.15.31:0/2840491951 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa73c0cabd0 0x7fa73c0caff0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:23.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.954+0000 7fa742d9d700 1 --2- 172.21.15.31:0/2840491951 >> v2:172.21.15.62:3300/0 conn(0x7fa73c06e060 0x7fa73c0ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:23.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.954+0000 7fa742d9d700 1 -- 172.21.15.31:0/2840491951 >> 172.21.15.31:0/2840491951 conn(0x7fa73c0703a0 msgr2=0x7fa73c0707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:23.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.954+0000 7faec119a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7faebc0ca240 0x7faebc0c6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:23.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.955+0000 7faec119a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7faebc0ca240 0x7faebc0c6980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37718/0 (socket says 172.21.15.31:37718) 2023-10-06T02:23:23.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.955+0000 7faec119a700 1 -- 172.21.15.31:0/793157466 learned_addr learned my addr 172.21.15.31:0/793157466 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:23.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.955+0000 7faec119a700 1 -- 172.21.15.31:0/793157466 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faebc15b5d0 msgr2=0x7faebc0c9d00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T02:23:23.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.955+0000 7faec119a700 1 --2- 172.21.15.31:0/793157466 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faebc15b5d0 0x7faebc0c9d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:23.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.955+0000 7faec119a700 1 -- 172.21.15.31:0/793157466 >> v1:172.21.15.62:6789/0 conn(0x7faebc0c6ec0 legacy=0x7faebc06de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:23.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.955+0000 7faec119a700 1 -- 172.21.15.31:0/793157466 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faebc15e390 con 0x7faebc0ca240 2023-10-06T02:23:23.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.955+0000 7f929c999700 1 -- 172.21.15.31:0/3929058211 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f928800b640 con 0x7f929815de50 2023-10-06T02:23:23.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.955+0000 7fbb8d59d700 1 -- 172.21.15.31:0/1332014186 shutdown_connections 2023-10-06T02:23:23.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.956+0000 7f929c999700 1 -- 172.21.15.31:0/3929058211 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f928800b7a0 con 0x7f929815de50 2023-10-06T02:23:23.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.955+0000 7faec119a700 1 --2- 172.21.15.31:0/793157466 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faebc0ca240 0x7faebc0c6980 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7faeb4009f70 tx=0x7faeb4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cce4bd3d46f84d4a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:23.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.956+0000 7f929c999700 1 -- 172.21.15.31:0/3929058211 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f928800b640 con 0x7f929815de50 2023-10-06T02:23:23.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.956+0000 7fa742d9d700 1 -- 172.21.15.31:0/2840491951 shutdown_connections 2023-10-06T02:23:23.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.957+0000 7faec0999700 1 -- 172.21.15.31:0/793157466 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb400e070 con 0x7faebc0ca240 2023-10-06T02:23:23.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.957+0000 7faec0999700 1 -- 172.21.15.31:0/793157466 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faeb400ba90 con 0x7faebc0ca240 2023-10-06T02:23:23.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.957+0000 7f929e99d700 1 -- 172.21.15.31:0/3929058211 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f929815de50 msgr2=0x7f92980cea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:23.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.957+0000 7f929e99d700 1 --2- 172.21.15.31:0/3929058211 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f929815de50 0x7f92980cea00 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f92880098e0 tx=0x7f92880091a0 comp rx=0 tx=0).stop 2023-10-06T02:23:23.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.958+0000 7faec0999700 1 -- 172.21.15.31:0/793157466 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb40092e0 con 0x7faebc0ca240 2023-10-06T02:23:23.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.957+0000 7f929e99d700 1 -- 172.21.15.31:0/3929058211 shutdown_connections 2023-10-06T02:23:23.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.958+0000 7faec299d700 1 -- 172.21.15.31:0/793157466 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faebc0ca240 msgr2=0x7faebc0c6980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:23.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.958+0000 7faec299d700 1 --2- 172.21.15.31:0/793157466 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faebc0ca240 0x7faebc0c6980 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7faeb4009f70 tx=0x7faeb4009840 comp rx=0 tx=0).stop 2023-10-06T02:23:23.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.958+0000 7f929e99d700 1 --2- 172.21.15.31:0/3929058211 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f92980cef40 0x7f929815c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:23.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.958+0000 7f929e99d700 1 --2- 172.21.15.31:0/3929058211 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f929815de50 0x7f92980cea00 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:23.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.958+0000 7faec299d700 1 -- 172.21.15.31:0/793157466 shutdown_connections 2023-10-06T02:23:23.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.958+0000 7faec299d700 1 --2- 172.21.15.31:0/793157466 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faebc15b5d0 0x7faebc0c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:23.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.958+0000 7faec299d700 1 --2- 172.21.15.31:0/793157466 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faebc0ca240 0x7faebc0c6980 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:23.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.958+0000 7faec299d700 1 -- 172.21.15.31:0/793157466 >> 172.21.15.31:0/793157466 conn(0x7faebc06d170 msgr2=0x7faebc0cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:23.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.958+0000 7f929e99d700 1 -- 172.21.15.31:0/3929058211 >> 172.21.15.31:0/3929058211 conn(0x7f92980703a0 msgr2=0x7f92980707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:23.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.958+0000 7faec299d700 1 -- 172.21.15.31:0/793157466 shutdown_connections 2023-10-06T02:23:23.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.958+0000 7faec299d700 1 -- 172.21.15.31:0/793157466 wait complete. 2023-10-06T02:23:23.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.956+0000 7fbb8d59d700 1 -- 172.21.15.31:0/1332014186 wait complete. 2023-10-06T02:23:23.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.958+0000 7fbb8d59d700 1 Processor -- start 2023-10-06T02:23:23.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.958+0000 7fbb8d59d700 1 -- start start 2023-10-06T02:23:23.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.959+0000 7fa742d9d700 1 -- 172.21.15.31:0/2840491951 wait complete. 2023-10-06T02:23:23.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.959+0000 7f929e99d700 1 -- 172.21.15.31:0/3929058211 shutdown_connections 2023-10-06T02:23:23.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.959+0000 7f929e99d700 1 -- 172.21.15.31:0/3929058211 wait complete. 2023-10-06T02:23:23.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.961+0000 7faec299d700 1 Processor -- start 2023-10-06T02:23:23.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.961+0000 7fa742d9d700 1 Processor -- start 2023-10-06T02:23:23.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.960+0000 7fbb8d59d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb8003af50 0x7fbb800b2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:23.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.960+0000 7fbb8d59d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbb80053e80 0x7fbb80051100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:23.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.960+0000 7fbb8d59d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb800dd620 0x7fbb800b79d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:23.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.960+0000 7fbb8d59d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb800dee80 con 0x7fbb8003af50 2023-10-06T02:23:23.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.960+0000 7fbb8d59d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb80051b30 con 0x7fbb800dd620 2023-10-06T02:23:23.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.960+0000 7fbb8d59d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb800df9e0 con 0x7fbb80053e80 2023-10-06T02:23:23.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.960+0000 7fbb8cd9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb800dd620 0x7fbb800b79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:23.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.960+0000 7fbb8cd9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb800dd620 0x7fbb800b79d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:47500/0 (socket says 172.21.15.31:47500) 2023-10-06T02:23:23.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.960+0000 7fbb8cd9c700 1 -- 172.21.15.31:0/3441150159 learned_addr learned my addr 172.21.15.31:0/3441150159 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:23.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.961+0000 7fbb8cd9c700 1 -- 172.21.15.31:0/3441150159 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbb80053e80 msgr2=0x7fbb80051100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:23.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.960+0000 7fbb877fe700 1 --2- 172.21.15.31:0/3441150159 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbb80053e80 0x7fbb80051100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:23.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.961+0000 7fbb8cd9c700 1 --2- 172.21.15.31:0/3441150159 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbb80053e80 0x7fbb80051100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:23.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.961+0000 7fbb8cd9c700 1 -- 172.21.15.31:0/3441150159 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb8003af50 msgr2=0x7fbb800b2110 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T02:23:23.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.961+0000 7fbb87fff700 1 --2- 172.21.15.31:0/3441150159 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb8003af50 0x7fbb800b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:23.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.961+0000 7fbb8cd9c700 1 --2- 172.21.15.31:0/3441150159 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb8003af50 0x7fbb800b2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:23.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.961+0000 7fbb8cd9c700 1 -- 172.21.15.31:0/3441150159 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb800df0b0 con 0x7fbb800dd620 2023-10-06T02:23:23.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.961+0000 7fbb877fe700 1 --2- 172.21.15.31:0/3441150159 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbb80053e80 0x7fbb80051100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:23:23.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.961+0000 7fbb8cd9c700 1 --2- 172.21.15.31:0/3441150159 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb800dd620 0x7fbb800b79d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fbb88000eb0 tx=0x7fbb88068000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:23.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.962+0000 7fbb6ffff700 1 -- 172.21.15.31:0/3441150159 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb8806de90 con 0x7fbb800dd620 2023-10-06T02:23:23.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.962+0000 7fbb8d59d700 1 -- 172.21.15.31:0/3441150159 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb800df390 con 0x7fbb800dd620 2023-10-06T02:23:23.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.962+0000 7f929e99d700 1 Processor -- start 2023-10-06T02:23:23.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.962+0000 7f929e99d700 1 -- start start 2023-10-06T02:23:23.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.962+0000 7fbb8d59d700 1 -- 172.21.15.31:0/3441150159 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb800503a0 con 0x7fbb800dd620 2023-10-06T02:23:23.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.962+0000 7fbb6ffff700 1 -- 172.21.15.31:0/3441150159 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb880675a0 con 0x7fbb800dd620 2023-10-06T02:23:23.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.962+0000 7fbb6ffff700 1 -- 172.21.15.31:0/3441150159 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb88067920 con 0x7fbb800dd620 2023-10-06T02:23:23.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.961+0000 7fa742d9d700 1 -- start start 2023-10-06T02:23:23.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.963+0000 7fa742d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa73c06e060 0x7fa73c140930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:23.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.963+0000 7f929e99d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92980cef40 0x7f9298177560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:23.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.963+0000 7fa742d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa73c0cabd0 0x7fa73c160230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:23.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.964+0000 7f929e99d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9298153c30 0x7f92980d0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:23.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.963+0000 7fa742d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa73c1549e0 0x7fa73c160770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:23.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.963+0000 7fa742d9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa73c0c9760 con 0x7fa73c1549e0 2023-10-06T02:23:23.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.963+0000 7fa742d9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa73c144070 con 0x7fa73c0cabd0 2023-10-06T02:23:23.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.963+0000 7fa742d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa73c161a20 con 0x7fa73c06e060 2023-10-06T02:23:23.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.964+0000 7fa74259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa73c1549e0 0x7fa73c160770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:23.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.964+0000 7fa74259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa73c1549e0 0x7fa73c160770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37736/0 (socket says 172.21.15.31:37736) 2023-10-06T02:23:23.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.964+0000 7fa74259c700 1 -- 172.21.15.31:0/3828772127 learned_addr learned my addr 172.21.15.31:0/3828772127 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:23.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.962+0000 7faec299d700 1 -- start start 2023-10-06T02:23:23.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.963+0000 7faec299d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faebc0c6ec0 0x7faebc0aeed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:23.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.963+0000 7faec299d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faebc15b5d0 0x7faebc0b1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:23.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.964+0000 7faec119a700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faebc15b5d0 0x7faebc0b1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:23.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.964+0000 7faec299d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faebc13b240 0x7faebc15e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:23.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.964+0000 7faec299d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faebc0af590 con 0x7faebc13b240 2023-10-06T02:23:23.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.964+0000 7faec299d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faebc0b0fd0 con 0x7faebc0c6ec0 2023-10-06T02:23:23.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.964+0000 7faec299d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faebc0b12b0 con 0x7faebc15b5d0 2023-10-06T02:23:23.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.964+0000 7faec119a700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faebc15b5d0 0x7faebc0b1ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:57306/0 (socket says 172.21.15.31:57306) 2023-10-06T02:23:23.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.964+0000 7faec119a700 1 -- 172.21.15.31:0/1868953954 learned_addr learned my addr 172.21.15.31:0/1868953954 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:23.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.964+0000 7faec199b700 1 --2- 172.21.15.31:0/1868953954 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faebc0c6ec0 0x7faebc0aeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:23.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.965+0000 7faec119a700 1 -- 172.21.15.31:0/1868953954 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faebc0c6ec0 msgr2=0x7faebc0aeed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:23.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.965+0000 7f929e99d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f929815de50 0x7f92980d0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:23.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.965+0000 7f929d19a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9298153c30 0x7f92980d0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:23.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.965+0000 7f929d19a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9298153c30 0x7f92980d0450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37740/0 (socket says 172.21.15.31:37740) 2023-10-06T02:23:23.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.966+0000 7f929d19a700 1 -- 172.21.15.31:0/812676491 learned_addr learned my addr 172.21.15.31:0/812676491 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:23.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.966+0000 7f929e99d700 1 -- 172.21.15.31:0/812676491 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9298153850 con 0x7f9298153c30 2023-10-06T02:23:23.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.965+0000 7fbb6ffff700 1 -- 172.21.15.31:0/3441150159 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbb8808d020 con 0x7fbb800dd620 2023-10-06T02:23:23.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.966+0000 7fbb8d59d700 1 -- 172.21.15.31:0/3441150159 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb680052b0 con 0x7fbb800dd620 2023-10-06T02:23:23.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.964+0000 7fa74259c700 1 -- 172.21.15.31:0/3828772127 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa73c06e060 msgr2=0x7fa73c140930 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T02:23:23.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.964+0000 7fa74259c700 1 --2- 172.21.15.31:0/3828772127 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa73c06e060 0x7fa73c140930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:23.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.964+0000 7fa74259c700 1 -- 172.21.15.31:0/3828772127 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa73c0cabd0 msgr2=0x7fa73c160230 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T02:23:23.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.964+0000 7fa74259c700 1 --2- 172.21.15.31:0/3828772127 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa73c0cabd0 0x7fa73c160230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:23.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.964+0000 7fa74259c700 1 -- 172.21.15.31:0/3828772127 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa734009a20 con 0x7fa73c1549e0 2023-10-06T02:23:23.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.965+0000 7fa74259c700 1 --2- 172.21.15.31:0/3828772127 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa73c1549e0 0x7fa73c160770 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa734012b10 tx=0x7fa734019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:23.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.966+0000 7fa731ffb700 1 -- 172.21.15.31:0/3828772127 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa734012e50 con 0x7fa73c1549e0 2023-10-06T02:23:23.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.966+0000 7fa731ffb700 1 -- 172.21.15.31:0/3828772127 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa734007000 con 0x7fa73c1549e0 2023-10-06T02:23:23.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.965+0000 7faec119a700 1 --2- 172.21.15.31:0/1868953954 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faebc0c6ec0 0x7faebc0aeed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:23.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.965+0000 7faec119a700 1 -- 172.21.15.31:0/1868953954 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faebc13b240 msgr2=0x7faebc15e550 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T02:23:23.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.966+0000 7faec219c700 1 --2- 172.21.15.31:0/1868953954 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faebc13b240 0x7faebc15e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:23.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.967+0000 7fbb6ffff700 1 --2- 172.21.15.31:0/3441150159 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fbb74067080 0x7fbb74069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:23.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.967+0000 7faec119a700 1 --2- 172.21.15.31:0/1868953954 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faebc13b240 0x7faebc15e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:23.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.967+0000 7faec119a700 1 -- 172.21.15.31:0/1868953954 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeb4009ae0 con 0x7faebc15b5d0 2023-10-06T02:23:23.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.967+0000 7fa731ffb700 1 -- 172.21.15.31:0/3828772127 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa73401b830 con 0x7fa73c1549e0 2023-10-06T02:23:23.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.967+0000 7fa742d9d700 1 -- 172.21.15.31:0/3828772127 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa73c160e10 con 0x7fa73c1549e0 2023-10-06T02:23:23.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.966+0000 7f929e99d700 1 -- 172.21.15.31:0/812676491 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92980b3d90 con 0x7f92980cef40 2023-10-06T02:23:23.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.967+0000 7fbb6ffff700 1 -- 172.21.15.31:0/3441150159 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fbb880fb940 con 0x7fbb800dd620 2023-10-06T02:23:23.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.967+0000 7fbb87fff700 1 --2- 172.21.15.31:0/3441150159 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fbb74067080 0x7fbb74069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:23.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.968+0000 7faec119a700 1 --2- 172.21.15.31:0/1868953954 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faebc15b5d0 0x7faebc0b1ab0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7faeb4009ab0 tx=0x7faeb4017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:23.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.966+0000 7f929e99d700 1 -- 172.21.15.31:0/812676491 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9298177250 con 0x7f929815de50 2023-10-06T02:23:23.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.966+0000 7f929e19c700 1 --2- 172.21.15.31:0/812676491 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f929815de50 0x7f92980d0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:23.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.968+0000 7faeb1ffb700 1 -- 172.21.15.31:0/1868953954 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb4022070 con 0x7faebc15b5d0 2023-10-06T02:23:23.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.966+0000 7f929d99b700 1 --2- 172.21.15.31:0/812676491 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92980cef40 0x7f9298177560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:23.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.967+0000 7f929d99b700 1 -- 172.21.15.31:0/812676491 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f929815de50 msgr2=0x7f92980d0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:23.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.967+0000 7f929d99b700 1 --2- 172.21.15.31:0/812676491 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f929815de50 0x7f92980d0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:23.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.967+0000 7f929d99b700 1 -- 172.21.15.31:0/812676491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9298153c30 msgr2=0x7f92980d0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:23.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.967+0000 7f929d99b700 1 --2- 172.21.15.31:0/812676491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9298153c30 0x7f92980d0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:23.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.967+0000 7f929d99b700 1 -- 172.21.15.31:0/812676491 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9288009420 con 0x7f92980cef40 2023-10-06T02:23:23.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.969+0000 7f929d19a700 1 --2- 172.21.15.31:0/812676491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9298153c30 0x7f92980d0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:23:23.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.969+0000 7fbb87fff700 1 --2- 172.21.15.31:0/3441150159 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fbb74067080 0x7fbb74069530 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbb800dec70 tx=0x7fbb78008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:23.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.969+0000 7faeb1ffb700 1 -- 172.21.15.31:0/1868953954 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faeb4012ac0 con 0x7faebc15b5d0 2023-10-06T02:23:23.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.968+0000 7fa742d9d700 1 -- 172.21.15.31:0/3828772127 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa73c1613d0 con 0x7fa73c1549e0 2023-10-06T02:23:23.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.970+0000 7f929d99b700 1 --2- 172.21.15.31:0/812676491 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92980cef40 0x7f9298177560 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f929400c560 tx=0x7f9294007cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:23.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.970+0000 7fa742d9d700 1 -- 172.21.15.31:0/3828772127 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa73c04c3c0 con 0x7fa73c1549e0 2023-10-06T02:23:23.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.971+0000 7faec199b700 1 --2- 172.21.15.31:0/1868953954 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faebc0c6ec0 0x7faebc0aeed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:23:23.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.971+0000 7faec219c700 1 --2- 172.21.15.31:0/1868953954 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faebc13b240 0x7faebc15e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:23:23.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.971+0000 7faec299d700 1 -- 172.21.15.31:0/1868953954 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faebc0b1590 con 0x7faebc15b5d0 2023-10-06T02:23:23.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.971+0000 7faeb1ffb700 1 -- 172.21.15.31:0/1868953954 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb4012e80 con 0x7faebc15b5d0 2023-10-06T02:23:23.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.972+0000 7f928dffb700 1 -- 172.21.15.31:0/812676491 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9294005b30 con 0x7f92980cef40 2023-10-06T02:23:23.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.972+0000 7f928dffb700 1 -- 172.21.15.31:0/812676491 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9294005cb0 con 0x7f92980cef40 2023-10-06T02:23:23.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.972+0000 7f928dffb700 1 -- 172.21.15.31:0/812676491 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9294007320 con 0x7f92980cef40 2023-10-06T02:23:23.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.972+0000 7fbb6ffff700 1 -- 172.21.15.31:0/3441150159 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbb880c8a30 con 0x7fbb800dd620 2023-10-06T02:23:23.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.973+0000 7faec299d700 1 -- 172.21.15.31:0/1868953954 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faebc15ea90 con 0x7faebc15b5d0 2023-10-06T02:23:23.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.973+0000 7f929e99d700 1 -- 172.21.15.31:0/812676491 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92980b3460 con 0x7f92980cef40 2023-10-06T02:23:23.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.974+0000 7f929e99d700 1 -- 172.21.15.31:0/812676491 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92980b3a20 con 0x7f92980cef40 2023-10-06T02:23:23.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.974+0000 7fa731ffb700 1 -- 172.21.15.31:0/3828772127 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa73401bc20 con 0x7fa73c1549e0 2023-10-06T02:23:23.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.976+0000 7fa731ffb700 1 --2- 172.21.15.31:0/3828772127 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa720066f30 0x7fa7200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:23.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.976+0000 7fa731ffb700 1 -- 172.21.15.31:0/3828772127 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa734086e80 con 0x7fa73c1549e0 2023-10-06T02:23:23.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.976+0000 7f929e99d700 1 -- 172.21.15.31:0/812676491 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f929804c3c0 con 0x7f92980cef40 2023-10-06T02:23:23.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.977+0000 7fa741d9b700 1 --2- 172.21.15.31:0/3828772127 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa720066f30 0x7fa7200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:23.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.978+0000 7fa741d9b700 1 --2- 172.21.15.31:0/3828772127 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa720066f30 0x7fa7200693e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa738001050 tx=0x7fa738007ae0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:23.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.977+0000 7faec299d700 1 -- 172.21.15.31:0/1868953954 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faebc0b17a0 con 0x7faebc15b5d0 2023-10-06T02:23:23.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.978+0000 7faeb1ffb700 1 -- 172.21.15.31:0/1868953954 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7faeb401e640 con 0x7faebc15b5d0 2023-10-06T02:23:23.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.979+0000 7faeb1ffb700 1 --2- 172.21.15.31:0/1868953954 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7faea0066f30 0x7faea00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:23.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.980+0000 7faec199b700 1 --2- 172.21.15.31:0/1868953954 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7faea0066f30 0x7faea00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:23.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.981+0000 7faec199b700 1 --2- 172.21.15.31:0/1868953954 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7faea0066f30 0x7faea00693e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7faeb8006fd0 tx=0x7faeb8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:23.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.981+0000 7f928dffb700 1 -- 172.21.15.31:0/812676491 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9294007750 con 0x7f92980cef40 2023-10-06T02:23:23.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.982+0000 7faeb1ffb700 1 -- 172.21.15.31:0/1868953954 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7faeb4085d70 con 0x7faebc15b5d0 2023-10-06T02:23:23.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.982+0000 7f928dffb700 1 --2- 172.21.15.31:0/812676491 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f927c067080 0x7f927c069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:23.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.982+0000 7fa731ffb700 1 -- 172.21.15.31:0/3828772127 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa734053f70 con 0x7fa73c1549e0 2023-10-06T02:23:23.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.982+0000 7f928dffb700 1 -- 172.21.15.31:0/812676491 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9294083ac0 con 0x7f92980cef40 2023-10-06T02:23:23.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.983+0000 7f929d19a700 1 --2- 172.21.15.31:0/812676491 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f927c067080 0x7f927c069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:23.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.984+0000 7faeb1ffb700 1 -- 172.21.15.31:0/1868953954 <== mon.2 v2:172.21.15.62:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faeb408a070 con 0x7faebc15b5d0 2023-10-06T02:23:23.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.986+0000 7f929d19a700 1 --2- 172.21.15.31:0/812676491 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f927c067080 0x7f927c069530 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f928800a010 tx=0x7f9288017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:23.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:23.986+0000 7f928dffb700 1 -- 172.21.15.31:0/812676491 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9294085e60 con 0x7f92980cef40 2023-10-06T02:23:24.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.333+0000 7f3bb0ff9700 1 -- 172.21.15.31:0/2028734760 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f3bc40e1670 con 0x7f3bc400cc40 2023-10-06T02:23:24.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.334+0000 7f3bb2ffd700 1 -- 172.21.15.31:0/2028734760 <== mon.2 v2:172.21.15.62:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3bbc021020 con 0x7f3bc400cc40 2023-10-06T02:23:24.335 INFO:teuthology.orchestra.run.smithi031.stdout:55834574857 2023-10-06T02:23:24.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.336+0000 7fa5ff9ff700 1 -- 172.21.15.31:0/2877535116 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fa5f80a7d00 con 0x7fa5f806bc20 2023-10-06T02:23:24.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.337+0000 7fa5ed7fa700 1 -- 172.21.15.31:0/2877535116 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa5f0053410 con 0x7fa5f806bc20 2023-10-06T02:23:24.337 INFO:teuthology.orchestra.run.smithi031.stdout:55834574856 2023-10-06T02:23:24.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.338+0000 7f3bb0ff9700 1 -- 172.21.15.31:0/2028734760 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f3bb8067080 msgr2=0x7f3bb8069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:24.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.338+0000 7f3bb0ff9700 1 --2- 172.21.15.31:0/2028734760 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f3bb8067080 0x7f3bb8069530 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3bc00093f0 tx=0x7f3bc0017000 comp rx=0 tx=0).stop 2023-10-06T02:23:24.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.339+0000 7f029bfff700 1 -- 172.21.15.31:0/241787915 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f02940e21c0 con 0x7f02940e3380 2023-10-06T02:23:24.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.339+0000 7f3bb0ff9700 1 -- 172.21.15.31:0/2028734760 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3bc400cc40 msgr2=0x7f3bc4039cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:24.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.339+0000 7f3bb0ff9700 1 --2- 172.21.15.31:0/2028734760 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3bc400cc40 0x7f3bc4039cf0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc00ecb0 tx=0x7f3bbc0091a0 comp rx=0 tx=0).stop 2023-10-06T02:23:24.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.339+0000 7f3bb0ff9700 1 -- 172.21.15.31:0/2028734760 shutdown_connections 2023-10-06T02:23:24.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.339+0000 7f3bb0ff9700 1 --2- 172.21.15.31:0/2028734760 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3bc40e0170 0x7f3bc40ff690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:24.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.340+0000 7f3bb0ff9700 1 --2- 172.21.15.31:0/2028734760 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f3bb8067080 0x7f3bb8069530 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:24.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.340+0000 7f3bb0ff9700 1 --2- 172.21.15.31:0/2028734760 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3bc400cc40 0x7f3bc4039cf0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:24.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.340+0000 7f0282ffd700 1 -- 172.21.15.31:0/241787915 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f02940e21c0 con 0x7f02940e3380 2023-10-06T02:23:24.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.340+0000 7f3bb0ff9700 1 --2- 172.21.15.31:0/2028734760 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3bc40524e0 0x7f3bc40dcad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:24.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.340+0000 7f3bb0ff9700 1 -- 172.21.15.31:0/2028734760 >> 172.21.15.31:0/2028734760 conn(0x7f3bc4057a20 msgr2=0x7f3bc40deb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:24.341 INFO:teuthology.orchestra.run.smithi031.stdout:55834574858 2023-10-06T02:23:24.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.340+0000 7f3bb0ff9700 1 -- 172.21.15.31:0/2028734760 shutdown_connections 2023-10-06T02:23:24.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.341+0000 7fa5e2ffd700 1 -- 172.21.15.31:0/2877535116 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa5dc067080 msgr2=0x7fa5dc069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:24.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.341+0000 7fa5e2ffd700 1 --2- 172.21.15.31:0/2877535116 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa5dc067080 0x7fa5dc069530 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa5f81449b0 tx=0x7fa5f4007eb0 comp rx=0 tx=0).stop 2023-10-06T02:23:24.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.341+0000 7f3bb0ff9700 1 -- 172.21.15.31:0/2028734760 wait complete. 2023-10-06T02:23:24.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.341+0000 7fa5e2ffd700 1 -- 172.21.15.31:0/2877535116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa5f806bc20 msgr2=0x7fa5f80a9130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:24.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.341+0000 7fa5e2ffd700 1 --2- 172.21.15.31:0/2877535116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa5f806bc20 0x7fa5f80a9130 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa5f0009410 tx=0x7fa5f0018000 comp rx=0 tx=0).stop 2023-10-06T02:23:24.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.341+0000 7fa5e2ffd700 1 -- 172.21.15.31:0/2877535116 shutdown_connections 2023-10-06T02:23:24.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.341+0000 7fa5e2ffd700 1 --2- 172.21.15.31:0/2877535116 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5f80cb720 0x7fa5f8170a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:24.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.341+0000 7fa5e2ffd700 1 --2- 172.21.15.31:0/2877535116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa5f806bc20 0x7fa5f80a9130 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:24.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.341+0000 7fa5e2ffd700 1 --2- 172.21.15.31:0/2877535116 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa5dc067080 0x7fa5dc069530 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:24.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.341+0000 7fa5e2ffd700 1 --2- 172.21.15.31:0/2877535116 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa5f806eb00 0x7fa5f80a7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:24.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.341+0000 7fa5e2ffd700 1 -- 172.21.15.31:0/2877535116 >> 172.21.15.31:0/2877535116 conn(0x7fa5f80cc300 msgr2=0x7fa5f8155ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:24.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.342+0000 7fa5e2ffd700 1 -- 172.21.15.31:0/2877535116 shutdown_connections 2023-10-06T02:23:24.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.342+0000 7fa5e2ffd700 1 -- 172.21.15.31:0/2877535116 wait complete. 2023-10-06T02:23:24.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.345+0000 7f0280ff9700 1 -- 172.21.15.31:0/241787915 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f0288067080 msgr2=0x7f0288069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:24.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.345+0000 7f3ac8ff9700 1 -- 172.21.15.31:0/2496158553 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f3adc0dd760 con 0x7f3adc10f510 2023-10-06T02:23:24.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.345+0000 7f0280ff9700 1 --2- 172.21.15.31:0/241787915 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f0288067080 0x7f0288069530 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f028c006fd0 tx=0x7f028c009250 comp rx=0 tx=0).stop 2023-10-06T02:23:24.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.345+0000 7f0280ff9700 1 -- 172.21.15.31:0/241787915 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02940e3380 msgr2=0x7f02940dcae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:24.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.345+0000 7f0280ff9700 1 --2- 172.21.15.31:0/241787915 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02940e3380 0x7f02940dcae0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f029400ddf0 tx=0x7f0290018000 comp rx=0 tx=0).stop 2023-10-06T02:23:24.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.346+0000 7f0280ff9700 1 -- 172.21.15.31:0/241787915 shutdown_connections 2023-10-06T02:23:24.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.346+0000 7f3acaffd700 1 -- 172.21.15.31:0/2496158553 <== mon.2 v2:172.21.15.62:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3adc0dd760 con 0x7f3adc10f510 2023-10-06T02:23:24.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.346+0000 7f0280ff9700 1 --2- 172.21.15.31:0/241787915 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02940e3380 0x7f02940dcae0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:24.347 INFO:teuthology.orchestra.run.smithi031.stdout:55834574857 2023-10-06T02:23:24.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.346+0000 7f0280ff9700 1 --2- 172.21.15.31:0/241787915 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02940341a0 0x7f0294109c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:24.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.346+0000 7f0280ff9700 1 --2- 172.21.15.31:0/241787915 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f0288067080 0x7f0288069530 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:24.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.346+0000 7f0280ff9700 1 --2- 172.21.15.31:0/241787915 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f02940dbd10 0x7f02940e2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:24.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.346+0000 7f0280ff9700 1 -- 172.21.15.31:0/241787915 >> 172.21.15.31:0/241787915 conn(0x7f02940517b0 msgr2=0x7f0294054040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:24.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.347+0000 7f0280ff9700 1 -- 172.21.15.31:0/241787915 shutdown_connections 2023-10-06T02:23:24.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.347+0000 7f0280ff9700 1 -- 172.21.15.31:0/241787915 wait complete. 2023-10-06T02:23:24.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.350+0000 7f3ac8ff9700 1 -- 172.21.15.31:0/2496158553 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f3ad0066ff0 msgr2=0x7f3ad00694a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:24.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.350+0000 7f3ac8ff9700 1 --2- 172.21.15.31:0/2496158553 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f3ad0066ff0 0x7f3ad00694a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3ad8006ce0 tx=0x7f3ad8008040 comp rx=0 tx=0).stop 2023-10-06T02:23:24.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.351+0000 7f3ac8ff9700 1 -- 172.21.15.31:0/2496158553 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3adc10f510 msgr2=0x7f3adc04e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:24.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.351+0000 7f3ac8ff9700 1 --2- 172.21.15.31:0/2496158553 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3adc10f510 0x7f3adc04e0e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3ae40614f0 tx=0x7f3ae4068000 comp rx=0 tx=0).stop 2023-10-06T02:23:24.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.351+0000 7f3ac8ff9700 1 -- 172.21.15.31:0/2496158553 shutdown_connections 2023-10-06T02:23:24.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.351+0000 7f3ac8ff9700 1 --2- 172.21.15.31:0/2496158553 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f3ad0066ff0 0x7f3ad00694a0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:24.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.352+0000 7f3ac8ff9700 1 --2- 172.21.15.31:0/2496158553 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3adc10f510 0x7f3adc04e0e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:24.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.352+0000 7f3ac8ff9700 1 --2- 172.21.15.31:0/2496158553 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3adc055ec0 0x7f3adc006890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:24.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.352+0000 7f3ac8ff9700 1 --2- 172.21.15.31:0/2496158553 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3adc0dff60 0x7f3adc10efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:24.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.352+0000 7f3ac8ff9700 1 -- 172.21.15.31:0/2496158553 >> 172.21.15.31:0/2496158553 conn(0x7f3adc0539c0 msgr2=0x7f3adc0db5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:24.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.352+0000 7f3ac8ff9700 1 -- 172.21.15.31:0/2496158553 shutdown_connections 2023-10-06T02:23:24.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.353+0000 7f3ac8ff9700 1 -- 172.21.15.31:0/2496158553 wait complete. 2023-10-06T02:23:24.379 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.0 2023-10-06T02:23:24.380 DEBUG:teuthology.parallel:result is None 2023-10-06T02:23:24.380 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.1 2023-10-06T02:23:24.380 DEBUG:teuthology.parallel:result is None 2023-10-06T02:23:24.382 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.2 2023-10-06T02:23:24.383 DEBUG:teuthology.parallel:result is None 2023-10-06T02:23:24.386 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.5 2023-10-06T02:23:24.386 DEBUG:teuthology.parallel:result is None 2023-10-06T02:23:24.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.401+0000 7fbb8d59d700 1 -- 172.21.15.31:0/3441150159 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fbb68005560 con 0x7fbb800dd620 2023-10-06T02:23:24.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.401+0000 7faec299d700 1 -- 172.21.15.31:0/1868953954 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7faebc04c3c0 con 0x7faebc15b5d0 2023-10-06T02:23:24.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.401+0000 7faeb1ffb700 1 -- 172.21.15.31:0/1868953954 <== mon.2 v2:172.21.15.62:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7faeb4052de0 con 0x7faebc15b5d0 2023-10-06T02:23:24.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.401+0000 7fbb6ffff700 1 -- 172.21.15.31:0/3441150159 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbb880cc050 con 0x7fbb800dd620 2023-10-06T02:23:24.402 INFO:teuthology.orchestra.run.smithi031.stdout:55834574855 2023-10-06T02:23:24.402 INFO:teuthology.orchestra.run.smithi031.stdout:55834574857 2023-10-06T02:23:24.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.403+0000 7fa742d9d700 1 -- 172.21.15.31:0/3828772127 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fa73c15ee90 con 0x7fa73c1549e0 2023-10-06T02:23:24.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.403+0000 7fa731ffb700 1 -- 172.21.15.31:0/3828772127 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa734057590 con 0x7fa73c1549e0 2023-10-06T02:23:24.404 INFO:teuthology.orchestra.run.smithi031.stdout:55834574857 2023-10-06T02:23:24.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.404+0000 7faec299d700 1 -- 172.21.15.31:0/1868953954 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7faea0066f30 msgr2=0x7faea00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:24.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.404+0000 7faec299d700 1 --2- 172.21.15.31:0/1868953954 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7faea0066f30 0x7faea00693e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7faeb8006fd0 tx=0x7faeb8008040 comp rx=0 tx=0).stop 2023-10-06T02:23:24.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.404+0000 7faec299d700 1 -- 172.21.15.31:0/1868953954 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faebc15b5d0 msgr2=0x7faebc0b1ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:24.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.404+0000 7faec299d700 1 --2- 172.21.15.31:0/1868953954 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faebc15b5d0 0x7faebc0b1ab0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7faeb4009ab0 tx=0x7faeb4017000 comp rx=0 tx=0).stop 2023-10-06T02:23:24.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.404+0000 7faec299d700 1 -- 172.21.15.31:0/1868953954 shutdown_connections 2023-10-06T02:23:24.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.404+0000 7faec299d700 1 --2- 172.21.15.31:0/1868953954 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faebc13b240 0x7faebc15e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:24.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.404+0000 7faec299d700 1 --2- 172.21.15.31:0/1868953954 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faebc0c6ec0 0x7faebc0aeed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:24.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.404+0000 7faec299d700 1 --2- 172.21.15.31:0/1868953954 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7faea0066f30 0x7faea00693e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:24.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.404+0000 7faec299d700 1 --2- 172.21.15.31:0/1868953954 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faebc15b5d0 0x7faebc0b1ab0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:24.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.404+0000 7faec299d700 1 -- 172.21.15.31:0/1868953954 >> 172.21.15.31:0/1868953954 conn(0x7faebc06d170 msgr2=0x7faebc160ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:24.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.404+0000 7faec299d700 1 -- 172.21.15.31:0/1868953954 shutdown_connections 2023-10-06T02:23:24.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.404+0000 7faec299d700 1 -- 172.21.15.31:0/1868953954 wait complete. 2023-10-06T02:23:24.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.405+0000 7fbb8d59d700 1 -- 172.21.15.31:0/3441150159 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fbb74067080 msgr2=0x7fbb74069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:24.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.405+0000 7fbb8d59d700 1 --2- 172.21.15.31:0/3441150159 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fbb74067080 0x7fbb74069530 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbb800dec70 tx=0x7fbb78008040 comp rx=0 tx=0).stop 2023-10-06T02:23:24.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.405+0000 7fbb8d59d700 1 -- 172.21.15.31:0/3441150159 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb800dd620 msgr2=0x7fbb800b79d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:24.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.405+0000 7fbb8d59d700 1 --2- 172.21.15.31:0/3441150159 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb800dd620 0x7fbb800b79d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fbb88000eb0 tx=0x7fbb88068000 comp rx=0 tx=0).stop 2023-10-06T02:23:24.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.405+0000 7fbb8d59d700 1 -- 172.21.15.31:0/3441150159 shutdown_connections 2023-10-06T02:23:24.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.405+0000 7fbb8d59d700 1 --2- 172.21.15.31:0/3441150159 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb800dd620 0x7fbb800b79d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:24.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.405+0000 7fbb8d59d700 1 --2- 172.21.15.31:0/3441150159 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb8003af50 0x7fbb800b2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:24.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.405+0000 7fbb8d59d700 1 --2- 172.21.15.31:0/3441150159 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fbb74067080 0x7fbb74069530 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:24.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.405+0000 7fbb8d59d700 1 --2- 172.21.15.31:0/3441150159 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbb80053e80 0x7fbb80051100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:24.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.405+0000 7fbb8d59d700 1 -- 172.21.15.31:0/3441150159 >> 172.21.15.31:0/3441150159 conn(0x7fbb80055ba0 msgr2=0x7fbb80038e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:24.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.405+0000 7fbb8d59d700 1 -- 172.21.15.31:0/3441150159 shutdown_connections 2023-10-06T02:23:24.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.405+0000 7fbb8d59d700 1 -- 172.21.15.31:0/3441150159 wait complete. 2023-10-06T02:23:24.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.406+0000 7fa742d9d700 1 -- 172.21.15.31:0/3828772127 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa720066f30 msgr2=0x7fa7200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:24.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.406+0000 7fa742d9d700 1 --2- 172.21.15.31:0/3828772127 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa720066f30 0x7fa7200693e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa738001050 tx=0x7fa738007ae0 comp rx=0 tx=0).stop 2023-10-06T02:23:24.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.406+0000 7fa742d9d700 1 -- 172.21.15.31:0/3828772127 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa73c1549e0 msgr2=0x7fa73c160770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:24.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.406+0000 7fa742d9d700 1 --2- 172.21.15.31:0/3828772127 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa73c1549e0 0x7fa73c160770 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa734012b10 tx=0x7fa734019000 comp rx=0 tx=0).stop 2023-10-06T02:23:24.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.406+0000 7fa742d9d700 1 -- 172.21.15.31:0/3828772127 shutdown_connections 2023-10-06T02:23:24.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.406+0000 7fa742d9d700 1 --2- 172.21.15.31:0/3828772127 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa73c1549e0 0x7fa73c160770 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:24.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.406+0000 7fa742d9d700 1 --2- 172.21.15.31:0/3828772127 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa720066f30 0x7fa7200693e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:24.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.406+0000 7fa742d9d700 1 --2- 172.21.15.31:0/3828772127 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa73c06e060 0x7fa73c140930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:24.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.406+0000 7fa742d9d700 1 --2- 172.21.15.31:0/3828772127 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa73c0cabd0 0x7fa73c160230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:24.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.406+0000 7fa742d9d700 1 -- 172.21.15.31:0/3828772127 >> 172.21.15.31:0/3828772127 conn(0x7fa73c0703a0 msgr2=0x7fa73c0c6980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:24.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.406+0000 7fa742d9d700 1 -- 172.21.15.31:0/3828772127 shutdown_connections 2023-10-06T02:23:24.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.406+0000 7fa742d9d700 1 -- 172.21.15.31:0/3828772127 wait complete. 2023-10-06T02:23:24.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.415+0000 7f929e99d700 1 -- 172.21.15.31:0/812676491 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f92980b3bb0 con 0x7f92980cef40 2023-10-06T02:23:24.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.415+0000 7f928dffb700 1 -- 172.21.15.31:0/812676491 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9294017090 con 0x7f92980cef40 2023-10-06T02:23:24.416 INFO:teuthology.orchestra.run.smithi031.stdout:55834574857 2023-10-06T02:23:24.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.420+0000 7f929e99d700 1 -- 172.21.15.31:0/812676491 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f927c067080 msgr2=0x7f927c069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:24.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.420+0000 7f929e99d700 1 --2- 172.21.15.31:0/812676491 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f927c067080 0x7f927c069530 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f928800a010 tx=0x7f9288017000 comp rx=0 tx=0).stop 2023-10-06T02:23:24.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.420+0000 7f929e99d700 1 -- 172.21.15.31:0/812676491 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92980cef40 msgr2=0x7f9298177560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:24.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.420+0000 7f929e99d700 1 --2- 172.21.15.31:0/812676491 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92980cef40 0x7f9298177560 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f929400c560 tx=0x7f9294007cd0 comp rx=0 tx=0).stop 2023-10-06T02:23:24.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.420+0000 7f929e99d700 1 -- 172.21.15.31:0/812676491 shutdown_connections 2023-10-06T02:23:24.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.420+0000 7f929e99d700 1 --2- 172.21.15.31:0/812676491 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f927c067080 0x7f927c069530 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:24.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.420+0000 7f929e99d700 1 --2- 172.21.15.31:0/812676491 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f929815de50 0x7f92980d0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:24.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.420+0000 7f929e99d700 1 --2- 172.21.15.31:0/812676491 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92980cef40 0x7f9298177560 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:24.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.420+0000 7f929e99d700 1 --2- 172.21.15.31:0/812676491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9298153c30 0x7f92980d0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:24.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.420+0000 7f929e99d700 1 -- 172.21.15.31:0/812676491 >> 172.21.15.31:0/812676491 conn(0x7f92980703a0 msgr2=0x7f92980c6710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:24.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.420+0000 7f929e99d700 1 -- 172.21.15.31:0/812676491 shutdown_connections 2023-10-06T02:23:24.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.420+0000 7f929e99d700 1 -- 172.21.15.31:0/812676491 wait complete. 2023-10-06T02:23:24.427 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.7 2023-10-06T02:23:24.427 DEBUG:teuthology.parallel:result is None 2023-10-06T02:23:24.428 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.6 2023-10-06T02:23:24.428 DEBUG:teuthology.parallel:result is None 2023-10-06T02:23:24.431 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.3 2023-10-06T02:23:24.432 DEBUG:teuthology.parallel:result is None 2023-10-06T02:23:24.440 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.4 2023-10-06T02:23:24.440 DEBUG:teuthology.parallel:result is None 2023-10-06T02:23:24.440 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-06T02:23:24.441 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T02:23:24.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.665+0000 7f78af59e700 1 Processor -- start 2023-10-06T02:23:24.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.665+0000 7f78af59e700 1 -- start start 2023-10-06T02:23:24.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.666+0000 7f78af59e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f78b0159550 0x7f78b00ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:24.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.666+0000 7f78af59e700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f78b0159920 0x7f78b00ad090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:24.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.667+0000 7f78af59e700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78b00acc00 con 0x7f78b015dfd0 2023-10-06T02:23:24.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.667+0000 7f78af59e700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78b00abff0 con 0x7f78b0159920 2023-10-06T02:23:24.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.667+0000 7f78af59e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78b00ac2d0 con 0x7f78b0159550 2023-10-06T02:23:24.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.667+0000 7f78ae59c700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f78b0159550 0x7f78b00ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:24.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.667+0000 7f78aed9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f78b0159920 0x7f78b00ad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:24.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.667+0000 7f78aed9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f78b0159920 0x7f78b00ad090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:47522/0 (socket says 172.21.15.31:47522) 2023-10-06T02:23:24.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.667+0000 7f78aed9d700 1 -- 172.21.15.31:0/1946614325 learned_addr learned my addr 172.21.15.31:0/1946614325 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:24.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.667+0000 7f78add9b700 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f78b015dfd0 0x7f78b00aedb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:34004/0 (socket says 172.21.15.31:34004) 2023-10-06T02:23:24.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.667+0000 7f78ae59c700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f78b0159550 0x7f78b00ceac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:57344/0 (socket says 172.21.15.31:57344) 2023-10-06T02:23:24.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.668+0000 7f78aed9d700 1 -- 172.21.15.31:0/1946614325 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f78b0159550 msgr2=0x7f78b00ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:24.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.668+0000 7f78aed9d700 1 --2- 172.21.15.31:0/1946614325 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f78b0159550 0x7f78b00ceac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:24.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.668+0000 7f78aed9d700 1 -- 172.21.15.31:0/1946614325 >> v1:172.21.15.62:6789/0 conn(0x7f78b015dfd0 legacy=0x7f78b00aedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:24.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.668+0000 7f78aed9d700 1 -- 172.21.15.31:0/1946614325 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78b00ac5b0 con 0x7f78b0159920 2023-10-06T02:23:24.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.668+0000 7f78ad59a700 1 -- 172.21.15.31:0/1946614325 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3657986141 0 0) 0x7f78b00acc00 con 0x7f78b015dfd0 2023-10-06T02:23:24.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.668+0000 7f78aed9d700 1 --2- 172.21.15.31:0/1946614325 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78b0159920 0x7f78b00ad090 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f78a8009f20 tx=0x7f78a8009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6428446d8ea97f3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:24.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.668+0000 7f78ad59a700 1 -- 172.21.15.31:0/1946614325 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a800e070 con 0x7f78b0159920 2023-10-06T02:23:24.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.668+0000 7f78ad59a700 1 -- 172.21.15.31:0/1946614325 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78a800b9d0 con 0x7f78b0159920 2023-10-06T02:23:24.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.668+0000 7f78ad59a700 1 -- 172.21.15.31:0/1946614325 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a80092e0 con 0x7f78b0159920 2023-10-06T02:23:24.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.669+0000 7f78af59e700 1 -- 172.21.15.31:0/1946614325 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78b0159920 msgr2=0x7f78b00ad090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:24.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.669+0000 7f78af59e700 1 --2- 172.21.15.31:0/1946614325 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78b0159920 0x7f78b00ad090 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f78a8009f20 tx=0x7f78a8009840 comp rx=0 tx=0).stop 2023-10-06T02:23:24.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.669+0000 7f78af59e700 1 -- 172.21.15.31:0/1946614325 shutdown_connections 2023-10-06T02:23:24.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.669+0000 7f78af59e700 1 --2- 172.21.15.31:0/1946614325 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f78b0159550 0x7f78b00ceac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:24.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.669+0000 7f78af59e700 1 --2- 172.21.15.31:0/1946614325 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78b0159920 0x7f78b00ad090 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:24.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.669+0000 7f78af59e700 1 -- 172.21.15.31:0/1946614325 >> 172.21.15.31:0/1946614325 conn(0x7f78b006ce80 msgr2=0x7f78b00ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:24.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.669+0000 7f78af59e700 1 -- 172.21.15.31:0/1946614325 shutdown_connections 2023-10-06T02:23:24.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.669+0000 7f78af59e700 1 -- 172.21.15.31:0/1946614325 wait complete. 2023-10-06T02:23:24.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.670+0000 7f78af59e700 1 Processor -- start 2023-10-06T02:23:24.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.670+0000 7f78af59e700 1 -- start start 2023-10-06T02:23:24.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.671+0000 7f78af59e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78b0159550 0x7f78b0151870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:24.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.671+0000 7f78af59e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78b0159920 0x7f78b01559a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:24.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.671+0000 7f78af59e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f78b015dfd0 0x7f78b0155ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:24.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.671+0000 7f78af59e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b0157120 con 0x7f78b0159920 2023-10-06T02:23:24.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.671+0000 7f78af59e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b0152390 con 0x7f78b0159550 2023-10-06T02:23:24.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.671+0000 7f78af59e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78b0156510 con 0x7f78b015dfd0 2023-10-06T02:23:24.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.672+0000 7f78ae59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78b0159550 0x7f78b0151870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:24.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.672+0000 7f78add9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78b0159920 0x7f78b01559a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:24.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.672+0000 7f78ae59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78b0159550 0x7f78b0151870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:47524/0 (socket says 172.21.15.31:47524) 2023-10-06T02:23:24.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.672+0000 7f78ae59c700 1 -- 172.21.15.31:0/1974468591 learned_addr learned my addr 172.21.15.31:0/1974468591 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:24.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.672+0000 7f78aed9d700 1 --2- 172.21.15.31:0/1974468591 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f78b015dfd0 0x7f78b0155ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:24.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.672+0000 7f78add9b700 1 -- 172.21.15.31:0/1974468591 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f78b015dfd0 msgr2=0x7f78b0155ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:24.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.672+0000 7f78add9b700 1 --2- 172.21.15.31:0/1974468591 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f78b015dfd0 0x7f78b0155ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:24.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.672+0000 7f78add9b700 1 -- 172.21.15.31:0/1974468591 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78b0159550 msgr2=0x7f78b0151870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:24.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.672+0000 7f78add9b700 1 --2- 172.21.15.31:0/1974468591 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78b0159550 0x7f78b0151870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:24.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.672+0000 7f78add9b700 1 -- 172.21.15.31:0/1974468591 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78a8009ae0 con 0x7f78b0159920 2023-10-06T02:23:24.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.672+0000 7f78ae59c700 1 --2- 172.21.15.31:0/1974468591 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78b0159550 0x7f78b0151870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:23:24.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.672+0000 7f78add9b700 1 --2- 172.21.15.31:0/1974468591 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78b0159920 0x7f78b01559a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f78a400c4f0 tx=0x7f78a4007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:24.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.673+0000 7f789e7fc700 1 -- 172.21.15.31:0/1974468591 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a400cea0 con 0x7f78b0159920 2023-10-06T02:23:24.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.673+0000 7f789e7fc700 1 -- 172.21.15.31:0/1974468591 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78a4005a70 con 0x7f78b0159920 2023-10-06T02:23:24.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.673+0000 7f789e7fc700 1 -- 172.21.15.31:0/1974468591 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a4005e30 con 0x7f78b0159920 2023-10-06T02:23:24.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.673+0000 7f78af59e700 1 -- 172.21.15.31:0/1974468591 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78b01567f0 con 0x7f78b0159920 2023-10-06T02:23:24.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.673+0000 7f78af59e700 1 -- 172.21.15.31:0/1974468591 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78b0156db0 con 0x7f78b0159920 2023-10-06T02:23:24.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.674+0000 7f78af59e700 1 -- 172.21.15.31:0/1974468591 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78b01572d0 con 0x7f78b0159920 2023-10-06T02:23:24.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.675+0000 7f789e7fc700 1 -- 172.21.15.31:0/1974468591 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f78a40072e0 con 0x7f78b0159920 2023-10-06T02:23:24.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.675+0000 7f789e7fc700 1 --2- 172.21.15.31:0/1974468591 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f7890066f30 0x7f78900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:24.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.676+0000 7f789e7fc700 1 -- 172.21.15.31:0/1974468591 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f78a4083ac0 con 0x7f78b0159920 2023-10-06T02:23:24.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.676+0000 7f78ae59c700 1 --2- 172.21.15.31:0/1974468591 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f7890066f30 0x7f78900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:24.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.676+0000 7f78ae59c700 1 --2- 172.21.15.31:0/1974468591 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f7890066f30 0x7f78900693e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f78b009c150 tx=0x7f78a00091a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:24.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.679+0000 7f789e7fc700 1 -- 172.21.15.31:0/1974468591 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f78b01572d0 con 0x7f78b0159920 2023-10-06T02:23:24.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.877+0000 7f78af59e700 1 -- 172.21.15.31:0/1974468591 --> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f78b004c3c0 con 0x7f7890066f30 2023-10-06T02:23:24.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.886+0000 7f789e7fc700 1 -- 172.21.15.31:0/1974468591 <== mgr.4107 v2:172.21.15.31:6832/85292 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286495 (secure 0 0 0) 0x7f78b004c3c0 con 0x7f7890066f30 2023-10-06T02:23:24.889 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:23:24.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.890+0000 7f78af59e700 1 -- 172.21.15.31:0/1974468591 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f7890066f30 msgr2=0x7f78900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:24.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.890+0000 7f78af59e700 1 --2- 172.21.15.31:0/1974468591 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f7890066f30 0x7f78900693e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f78b009c150 tx=0x7f78a00091a0 comp rx=0 tx=0).stop 2023-10-06T02:23:24.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.890+0000 7f78af59e700 1 -- 172.21.15.31:0/1974468591 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78b0159920 msgr2=0x7f78b01559a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:24.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.890+0000 7f78af59e700 1 --2- 172.21.15.31:0/1974468591 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78b0159920 0x7f78b01559a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f78a400c4f0 tx=0x7f78a4007ef0 comp rx=0 tx=0).stop 2023-10-06T02:23:24.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.890+0000 7f78af59e700 1 -- 172.21.15.31:0/1974468591 shutdown_connections 2023-10-06T02:23:24.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.890+0000 7f78af59e700 1 --2- 172.21.15.31:0/1974468591 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f7890066f30 0x7f78900693e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:24.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.890+0000 7f78af59e700 1 --2- 172.21.15.31:0/1974468591 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f78b015dfd0 0x7f78b0155ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:24.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.890+0000 7f78af59e700 1 --2- 172.21.15.31:0/1974468591 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78b0159550 0x7f78b0151870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:24.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.890+0000 7f78af59e700 1 --2- 172.21.15.31:0/1974468591 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78b0159920 0x7f78b01559a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:24.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.890+0000 7f78af59e700 1 -- 172.21.15.31:0/1974468591 >> 172.21.15.31:0/1974468591 conn(0x7f78b006ce80 msgr2=0x7f78b00b3800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:24.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.890+0000 7f78af59e700 1 -- 172.21.15.31:0/1974468591 shutdown_connections 2023-10-06T02:23:24.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:24.890+0000 7f78af59e700 1 -- 172.21.15.31:0/1974468591 wait complete. 2023-10-06T02:23:24.893 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-10-06T02:23:24.908 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":31,"stamp":"2023-10-06T02:23:23.120405+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381416,"kb_used_data":3112,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752593304,"statfs":{"total":773094113280,"available":770655543296,"internally_reserved":0,"allocated":3186688,"data_stored":1899900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002766"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.985694+0000","last_change":"2023-10-06T02:23:09.982458+0000","last_active":"2023-10-06T02:23:09.985694+0000","last_peered":"2023-10-06T02:23:09.985694+0000","last_clean":"2023-10-06T02:23:09.985694+0000","last_became_active":"2023-10-06T02:23:09.982140+0000","last_became_peered":"2023-10-06T02:23:09.982140+0000","last_unstale":"2023-10-06T02:23:09.985694+0000","last_undegraded":"2023-10-06T02:23:09.985694+0000","last_fullsized":"2023-10-06T02:23:09.985694+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:07:57.730382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.999489+0000","last_change":"2023-10-06T02:23:09.996757+0000","last_active":"2023-10-06T02:23:09.999489+0000","last_peered":"2023-10-06T02:23:09.999489+0000","last_clean":"2023-10-06T02:23:09.999489+0000","last_became_active":"2023-10-06T02:23:09.996404+0000","last_became_peered":"2023-10-06T02:23:09.996404+0000","last_unstale":"2023-10-06T02:23:09.999489+0000","last_undegraded":"2023-10-06T02:23:09.999489+0000","last_fullsized":"2023-10-06T02:23:09.999489+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:39:15.275881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.010136+0000","last_change":"2023-10-06T02:23:09.999568+0000","last_active":"2023-10-06T02:23:10.010136+0000","last_peered":"2023-10-06T02:23:10.010136+0000","last_clean":"2023-10-06T02:23:10.010136+0000","last_became_active":"2023-10-06T02:23:09.999033+0000","last_became_peered":"2023-10-06T02:23:09.999033+0000","last_unstale":"2023-10-06T02:23:10.010136+0000","last_undegraded":"2023-10-06T02:23:10.010136+0000","last_fullsized":"2023-10-06T02:23:10.010136+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:23:26.580218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.996019+0000","last_change":"2023-10-06T02:23:09.995088+0000","last_active":"2023-10-06T02:23:09.996019+0000","last_peered":"2023-10-06T02:23:09.996019+0000","last_clean":"2023-10-06T02:23:09.996019+0000","last_became_active":"2023-10-06T02:23:09.994785+0000","last_became_peered":"2023-10-06T02:23:09.994785+0000","last_unstale":"2023-10-06T02:23:09.996019+0000","last_undegraded":"2023-10-06T02:23:09.996019+0000","last_fullsized":"2023-10-06T02:23:09.996019+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:21:33.737708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.994230+0000","last_change":"2023-10-06T02:23:09.986669+0000","last_active":"2023-10-06T02:23:09.994230+0000","last_peered":"2023-10-06T02:23:09.994230+0000","last_clean":"2023-10-06T02:23:09.994230+0000","last_became_active":"2023-10-06T02:23:09.986141+0000","last_became_peered":"2023-10-06T02:23:09.986141+0000","last_unstale":"2023-10-06T02:23:09.994230+0000","last_undegraded":"2023-10-06T02:23:09.994230+0000","last_fullsized":"2023-10-06T02:23:09.994230+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:39:15.275881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.010199+0000","last_change":"2023-10-06T02:23:10.000405+0000","last_active":"2023-10-06T02:23:10.010199+0000","last_peered":"2023-10-06T02:23:10.010199+0000","last_clean":"2023-10-06T02:23:10.010199+0000","last_became_active":"2023-10-06T02:23:10.000015+0000","last_became_peered":"2023-10-06T02:23:10.000015+0000","last_unstale":"2023-10-06T02:23:10.010199+0000","last_undegraded":"2023-10-06T02:23:10.010199+0000","last_fullsized":"2023-10-06T02:23:10.010199+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:39:15.275881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.009997+0000","last_change":"2023-10-06T02:23:10.004359+0000","last_active":"2023-10-06T02:23:10.009997+0000","last_peered":"2023-10-06T02:23:10.009997+0000","last_clean":"2023-10-06T02:23:10.009997+0000","last_became_active":"2023-10-06T02:23:10.003873+0000","last_became_peered":"2023-10-06T02:23:10.003873+0000","last_unstale":"2023-10-06T02:23:10.009997+0000","last_undegraded":"2023-10-06T02:23:10.009997+0000","last_fullsized":"2023-10-06T02:23:10.009997+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:44:41.608418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.998258+0000","last_change":"2023-10-06T02:23:09.994276+0000","last_active":"2023-10-06T02:23:09.998258+0000","last_peered":"2023-10-06T02:23:09.998258+0000","last_clean":"2023-10-06T02:23:09.998258+0000","last_became_active":"2023-10-06T02:23:09.993946+0000","last_became_peered":"2023-10-06T02:23:09.993946+0000","last_unstale":"2023-10-06T02:23:09.998258+0000","last_undegraded":"2023-10-06T02:23:09.998258+0000","last_fullsized":"2023-10-06T02:23:09.998258+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:19:29.462977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.997243+0000","last_change":"2023-10-06T02:23:09.993971+0000","last_active":"2023-10-06T02:23:09.997243+0000","last_peered":"2023-10-06T02:23:09.997243+0000","last_clean":"2023-10-06T02:23:09.997243+0000","last_became_active":"2023-10-06T02:23:09.993547+0000","last_became_peered":"2023-10-06T02:23:09.993547+0000","last_unstale":"2023-10-06T02:23:09.997243+0000","last_undegraded":"2023-10-06T02:23:09.997243+0000","last_fullsized":"2023-10-06T02:23:09.997243+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:19:29.462977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.999469+0000","last_change":"2023-10-06T02:23:09.995426+0000","last_active":"2023-10-06T02:23:09.999469+0000","last_peered":"2023-10-06T02:23:09.999469+0000","last_clean":"2023-10-06T02:23:09.999469+0000","last_became_active":"2023-10-06T02:23:09.995067+0000","last_became_peered":"2023-10-06T02:23:09.995067+0000","last_unstale":"2023-10-06T02:23:09.999469+0000","last_undegraded":"2023-10-06T02:23:09.999469+0000","last_fullsized":"2023-10-06T02:23:09.999469+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:50:19.168882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.010257+0000","last_change":"2023-10-06T02:23:10.001093+0000","last_active":"2023-10-06T02:23:10.010257+0000","last_peered":"2023-10-06T02:23:10.010257+0000","last_clean":"2023-10-06T02:23:10.010257+0000","last_became_active":"2023-10-06T02:23:10.000755+0000","last_became_peered":"2023-10-06T02:23:10.000755+0000","last_unstale":"2023-10-06T02:23:10.010257+0000","last_undegraded":"2023-10-06T02:23:10.010257+0000","last_fullsized":"2023-10-06T02:23:10.010257+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:43:02.156237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.998611+0000","last_change":"2023-10-06T02:23:09.993328+0000","last_active":"2023-10-06T02:23:09.998611+0000","last_peered":"2023-10-06T02:23:09.998611+0000","last_clean":"2023-10-06T02:23:09.998611+0000","last_became_active":"2023-10-06T02:23:09.993076+0000","last_became_peered":"2023-10-06T02:23:09.993076+0000","last_unstale":"2023-10-06T02:23:09.998611+0000","last_undegraded":"2023-10-06T02:23:09.998611+0000","last_fullsized":"2023-10-06T02:23:09.998611+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:58.521611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.986182+0000","last_change":"2023-10-06T02:23:09.982673+0000","last_active":"2023-10-06T02:23:09.986182+0000","last_peered":"2023-10-06T02:23:09.986182+0000","last_clean":"2023-10-06T02:23:09.986182+0000","last_became_active":"2023-10-06T02:23:09.982370+0000","last_became_peered":"2023-10-06T02:23:09.982370+0000","last_unstale":"2023-10-06T02:23:09.986182+0000","last_undegraded":"2023-10-06T02:23:09.986182+0000","last_fullsized":"2023-10-06T02:23:09.986182+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:17:33.441972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.985297+0000","last_change":"2023-10-06T02:23:09.982769+0000","last_active":"2023-10-06T02:23:09.985297+0000","last_peered":"2023-10-06T02:23:09.985297+0000","last_clean":"2023-10-06T02:23:09.985297+0000","last_became_active":"2023-10-06T02:23:09.982450+0000","last_became_peered":"2023-10-06T02:23:09.982450+0000","last_unstale":"2023-10-06T02:23:09.985297+0000","last_undegraded":"2023-10-06T02:23:09.985297+0000","last_fullsized":"2023-10-06T02:23:09.985297+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:12:16.306131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.010017+0000","last_change":"2023-10-06T02:23:10.004452+0000","last_active":"2023-10-06T02:23:10.010017+0000","last_peered":"2023-10-06T02:23:10.010017+0000","last_clean":"2023-10-06T02:23:10.010017+0000","last_became_active":"2023-10-06T02:23:10.004184+0000","last_became_peered":"2023-10-06T02:23:10.004184+0000","last_unstale":"2023-10-06T02:23:10.010017+0000","last_undegraded":"2023-10-06T02:23:10.010017+0000","last_fullsized":"2023-10-06T02:23:10.010017+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:50:19.168882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.010069+0000","last_change":"2023-10-06T02:23:09.998790+0000","last_active":"2023-10-06T02:23:10.010069+0000","last_peered":"2023-10-06T02:23:10.010069+0000","last_clean":"2023-10-06T02:23:10.010069+0000","last_became_active":"2023-10-06T02:23:09.998476+0000","last_became_peered":"2023-10-06T02:23:09.998476+0000","last_unstale":"2023-10-06T02:23:10.010069+0000","last_undegraded":"2023-10-06T02:23:10.010069+0000","last_fullsized":"2023-10-06T02:23:10.010069+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:19:29.462977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.997263+0000","last_change":"2023-10-06T02:23:09.994379+0000","last_active":"2023-10-06T02:23:09.997263+0000","last_peered":"2023-10-06T02:23:09.997263+0000","last_clean":"2023-10-06T02:23:09.997263+0000","last_became_active":"2023-10-06T02:23:09.994031+0000","last_became_peered":"2023-10-06T02:23:09.994031+0000","last_unstale":"2023-10-06T02:23:09.997263+0000","last_undegraded":"2023-10-06T02:23:09.997263+0000","last_fullsized":"2023-10-06T02:23:09.997263+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:12.998563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.994477+0000","last_change":"2023-10-06T02:23:09.988087+0000","last_active":"2023-10-06T02:23:09.994477+0000","last_peered":"2023-10-06T02:23:09.994477+0000","last_clean":"2023-10-06T02:23:09.994477+0000","last_became_active":"2023-10-06T02:23:09.987741+0000","last_became_peered":"2023-10-06T02:23:09.987741+0000","last_unstale":"2023-10-06T02:23:09.994477+0000","last_undegraded":"2023-10-06T02:23:09.994477+0000","last_fullsized":"2023-10-06T02:23:09.994477+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:05:24.047504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.987646+0000","last_change":"2023-10-06T02:23:09.983568+0000","last_active":"2023-10-06T02:23:09.987646+0000","last_peered":"2023-10-06T02:23:09.987646+0000","last_clean":"2023-10-06T02:23:09.987646+0000","last_became_active":"2023-10-06T02:23:09.983273+0000","last_became_peered":"2023-10-06T02:23:09.983273+0000","last_unstale":"2023-10-06T02:23:09.987646+0000","last_undegraded":"2023-10-06T02:23:09.987646+0000","last_fullsized":"2023-10-06T02:23:09.987646+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:44:16.553231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.009997+0000","last_change":"2023-10-06T02:23:09.997248+0000","last_active":"2023-10-06T02:23:10.009997+0000","last_peered":"2023-10-06T02:23:10.009997+0000","last_clean":"2023-10-06T02:23:10.009997+0000","last_became_active":"2023-10-06T02:23:09.996897+0000","last_became_peered":"2023-10-06T02:23:09.996897+0000","last_unstale":"2023-10-06T02:23:10.009997+0000","last_undegraded":"2023-10-06T02:23:10.009997+0000","last_fullsized":"2023-10-06T02:23:10.009997+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:00:45.442861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.994041+0000","last_change":"2023-10-06T02:23:09.991324+0000","last_active":"2023-10-06T02:23:09.994041+0000","last_peered":"2023-10-06T02:23:09.994041+0000","last_clean":"2023-10-06T02:23:09.994041+0000","last_became_active":"2023-10-06T02:23:09.991018+0000","last_became_peered":"2023-10-06T02:23:09.991018+0000","last_unstale":"2023-10-06T02:23:09.994041+0000","last_undegraded":"2023-10-06T02:23:09.994041+0000","last_fullsized":"2023-10-06T02:23:09.994041+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:43:01.259350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.004545+0000","last_change":"2023-10-06T02:23:10.000917+0000","last_active":"2023-10-06T02:23:10.004545+0000","last_peered":"2023-10-06T02:23:10.004545+0000","last_clean":"2023-10-06T02:23:10.004545+0000","last_became_active":"2023-10-06T02:23:10.000580+0000","last_became_peered":"2023-10-06T02:23:10.000580+0000","last_unstale":"2023-10-06T02:23:10.004545+0000","last_undegraded":"2023-10-06T02:23:10.004545+0000","last_fullsized":"2023-10-06T02:23:10.004545+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:50:19.168882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.998825+0000","last_change":"2023-10-06T02:23:09.995748+0000","last_active":"2023-10-06T02:23:09.998825+0000","last_peered":"2023-10-06T02:23:09.998825+0000","last_clean":"2023-10-06T02:23:09.998825+0000","last_became_active":"2023-10-06T02:23:09.995433+0000","last_became_peered":"2023-10-06T02:23:09.995433+0000","last_unstale":"2023-10-06T02:23:09.998825+0000","last_undegraded":"2023-10-06T02:23:09.998825+0000","last_fullsized":"2023-10-06T02:23:09.998825+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:21.438589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.996641+0000","last_change":"2023-10-06T02:23:09.992834+0000","last_active":"2023-10-06T02:23:09.996641+0000","last_peered":"2023-10-06T02:23:09.996641+0000","last_clean":"2023-10-06T02:23:09.996641+0000","last_became_active":"2023-10-06T02:23:09.991971+0000","last_became_peered":"2023-10-06T02:23:09.991971+0000","last_unstale":"2023-10-06T02:23:09.996641+0000","last_undegraded":"2023-10-06T02:23:09.996641+0000","last_fullsized":"2023-10-06T02:23:09.996641+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:12.998563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.007651+0000","last_change":"2023-10-06T02:23:10.005469+0000","last_active":"2023-10-06T02:23:10.007651+0000","last_peered":"2023-10-06T02:23:10.007651+0000","last_clean":"2023-10-06T02:23:10.007651+0000","last_became_active":"2023-10-06T02:23:10.005132+0000","last_became_peered":"2023-10-06T02:23:10.005132+0000","last_unstale":"2023-10-06T02:23:10.007651+0000","last_undegraded":"2023-10-06T02:23:10.007651+0000","last_fullsized":"2023-10-06T02:23:10.007651+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:19:18.748586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.005833+0000","last_change":"2023-10-06T02:23:09.999977+0000","last_active":"2023-10-06T02:23:10.005833+0000","last_peered":"2023-10-06T02:23:10.005833+0000","last_clean":"2023-10-06T02:23:10.005833+0000","last_became_active":"2023-10-06T02:23:09.999733+0000","last_became_peered":"2023-10-06T02:23:09.999733+0000","last_unstale":"2023-10-06T02:23:10.005833+0000","last_undegraded":"2023-10-06T02:23:10.005833+0000","last_fullsized":"2023-10-06T02:23:10.005833+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:58:41.925466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.999279+0000","last_change":"2023-10-06T02:23:09.994580+0000","last_active":"2023-10-06T02:23:09.999279+0000","last_peered":"2023-10-06T02:23:09.999279+0000","last_clean":"2023-10-06T02:23:09.999279+0000","last_became_active":"2023-10-06T02:23:09.993941+0000","last_became_peered":"2023-10-06T02:23:09.993941+0000","last_unstale":"2023-10-06T02:23:09.999279+0000","last_undegraded":"2023-10-06T02:23:09.999279+0000","last_fullsized":"2023-10-06T02:23:09.999279+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:43:01.259350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.043858+0000","last_change":"2023-10-06T02:23:09.048996+0000","last_active":"2023-10-06T02:23:10.043858+0000","last_peered":"2023-10-06T02:23:10.043858+0000","last_clean":"2023-10-06T02:23:10.043858+0000","last_became_active":"2023-10-06T02:23:09.048662+0000","last_became_peered":"2023-10-06T02:23:09.048662+0000","last_unstale":"2023-10-06T02:23:10.043858+0000","last_undegraded":"2023-10-06T02:23:10.043858+0000","last_fullsized":"2023-10-06T02:23:10.043858+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:58.976414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.004959+0000","last_change":"2023-10-06T02:23:10.001176+0000","last_active":"2023-10-06T02:23:10.004959+0000","last_peered":"2023-10-06T02:23:10.004959+0000","last_clean":"2023-10-06T02:23:10.004959+0000","last_became_active":"2023-10-06T02:23:10.000887+0000","last_became_peered":"2023-10-06T02:23:10.000887+0000","last_unstale":"2023-10-06T02:23:10.004959+0000","last_undegraded":"2023-10-06T02:23:10.004959+0000","last_fullsized":"2023-10-06T02:23:10.004959+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:43:02.156237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.998513+0000","last_change":"2023-10-06T02:23:09.048083+0000","last_active":"2023-10-06T02:23:09.998513+0000","last_peered":"2023-10-06T02:23:09.998513+0000","last_clean":"2023-10-06T02:23:09.998513+0000","last_became_active":"2023-10-06T02:23:09.047701+0000","last_became_peered":"2023-10-06T02:23:09.047701+0000","last_unstale":"2023-10-06T02:23:09.998513+0000","last_undegraded":"2023-10-06T02:23:09.998513+0000","last_fullsized":"2023-10-06T02:23:09.998513+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:17:32.434412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.994267+0000","last_change":"2023-10-06T02:23:09.988810+0000","last_active":"2023-10-06T02:23:09.994267+0000","last_peered":"2023-10-06T02:23:09.994267+0000","last_clean":"2023-10-06T02:23:09.994267+0000","last_became_active":"2023-10-06T02:23:09.988419+0000","last_became_peered":"2023-10-06T02:23:09.988419+0000","last_unstale":"2023-10-06T02:23:09.994267+0000","last_undegraded":"2023-10-06T02:23:09.994267+0000","last_fullsized":"2023-10-06T02:23:09.994267+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:43:01.259350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.976287+0000","last_change":"2023-10-06T02:23:09.044642+0000","last_active":"2023-10-06T02:23:09.976287+0000","last_peered":"2023-10-06T02:23:09.976287+0000","last_clean":"2023-10-06T02:23:09.976287+0000","last_became_active":"2023-10-06T02:23:09.044224+0000","last_became_peered":"2023-10-06T02:23:09.044224+0000","last_unstale":"2023-10-06T02:23:09.976287+0000","last_undegraded":"2023-10-06T02:23:09.976287+0000","last_fullsized":"2023-10-06T02:23:09.976287+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:31:52.666890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.986940+0000","last_change":"2023-10-06T02:23:09.985837+0000","last_active":"2023-10-06T02:23:09.986940+0000","last_peered":"2023-10-06T02:23:09.986940+0000","last_clean":"2023-10-06T02:23:09.986940+0000","last_became_active":"2023-10-06T02:23:09.985447+0000","last_became_peered":"2023-10-06T02:23:09.985447+0000","last_unstale":"2023-10-06T02:23:09.986940+0000","last_undegraded":"2023-10-06T02:23:09.986940+0000","last_fullsized":"2023-10-06T02:23:09.986940+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:31:53.674450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.979513+0000","last_change":"2023-10-06T02:23:09.040730+0000","last_active":"2023-10-06T02:23:09.979513+0000","last_peered":"2023-10-06T02:23:09.979513+0000","last_clean":"2023-10-06T02:23:09.979513+0000","last_became_active":"2023-10-06T02:23:09.040327+0000","last_became_peered":"2023-10-06T02:23:09.040327+0000","last_unstale":"2023-10-06T02:23:09.979513+0000","last_undegraded":"2023-10-06T02:23:09.979513+0000","last_fullsized":"2023-10-06T02:23:09.979513+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:07:56.722823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.998851+0000","last_change":"2023-10-06T02:23:09.995950+0000","last_active":"2023-10-06T02:23:09.998851+0000","last_peered":"2023-10-06T02:23:09.998851+0000","last_clean":"2023-10-06T02:23:09.998851+0000","last_became_active":"2023-10-06T02:23:09.995679+0000","last_became_peered":"2023-10-06T02:23:09.995679+0000","last_unstale":"2023-10-06T02:23:09.998851+0000","last_undegraded":"2023-10-06T02:23:09.998851+0000","last_fullsized":"2023-10-06T02:23:09.998851+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:23:26.580218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.976082+0000","last_change":"2023-10-06T02:23:09.040245+0000","last_active":"2023-10-06T02:23:09.976082+0000","last_peered":"2023-10-06T02:23:09.976082+0000","last_clean":"2023-10-06T02:23:09.976082+0000","last_became_active":"2023-10-06T02:23:09.039660+0000","last_became_peered":"2023-10-06T02:23:09.039660+0000","last_unstale":"2023-10-06T02:23:09.976082+0000","last_undegraded":"2023-10-06T02:23:09.976082+0000","last_fullsized":"2023-10-06T02:23:09.976082+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:57.514051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.994925+0000","last_change":"2023-10-06T02:23:09.992417+0000","last_active":"2023-10-06T02:23:09.994925+0000","last_peered":"2023-10-06T02:23:09.994925+0000","last_clean":"2023-10-06T02:23:09.994925+0000","last_became_active":"2023-10-06T02:23:09.992155+0000","last_became_peered":"2023-10-06T02:23:09.992155+0000","last_unstale":"2023-10-06T02:23:09.994925+0000","last_undegraded":"2023-10-06T02:23:09.994925+0000","last_fullsized":"2023-10-06T02:23:09.994925+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:39:15.275881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"22'3","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.044297+0000","last_change":"2023-10-06T02:23:09.046655+0000","last_active":"2023-10-06T02:23:10.044297+0000","last_peered":"2023-10-06T02:23:10.044297+0000","last_clean":"2023-10-06T02:23:10.044297+0000","last_became_active":"2023-10-06T02:23:09.046335+0000","last_became_peered":"2023-10-06T02:23:09.046335+0000","last_unstale":"2023-10-06T02:23:10.044297+0000","last_undegraded":"2023-10-06T02:23:10.044297+0000","last_fullsized":"2023-10-06T02:23:10.044297+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:53:00.225521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.007635+0000","last_change":"2023-10-06T02:23:10.005359+0000","last_active":"2023-10-06T02:23:10.007635+0000","last_peered":"2023-10-06T02:23:10.007635+0000","last_clean":"2023-10-06T02:23:10.007635+0000","last_became_active":"2023-10-06T02:23:10.005115+0000","last_became_peered":"2023-10-06T02:23:10.005115+0000","last_unstale":"2023-10-06T02:23:10.007635+0000","last_undegraded":"2023-10-06T02:23:10.007635+0000","last_fullsized":"2023-10-06T02:23:10.007635+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:29:00.771363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"22'6","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.052236+0000","last_change":"2023-10-06T02:23:09.046958+0000","last_active":"2023-10-06T02:23:10.052236+0000","last_peered":"2023-10-06T02:23:10.052236+0000","last_clean":"2023-10-06T02:23:10.052236+0000","last_became_active":"2023-10-06T02:23:09.044018+0000","last_became_peered":"2023-10-06T02:23:09.044018+0000","last_unstale":"2023-10-06T02:23:10.052236+0000","last_undegraded":"2023-10-06T02:23:10.052236+0000","last_fullsized":"2023-10-06T02:23:10.052236+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:41.039005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.992467+0000","last_change":"2023-10-06T02:23:09.987559+0000","last_active":"2023-10-06T02:23:09.992467+0000","last_peered":"2023-10-06T02:23:09.992467+0000","last_clean":"2023-10-06T02:23:09.992467+0000","last_became_active":"2023-10-06T02:23:09.987264+0000","last_became_peered":"2023-10-06T02:23:09.987264+0000","last_unstale":"2023-10-06T02:23:09.992467+0000","last_undegraded":"2023-10-06T02:23:09.992467+0000","last_fullsized":"2023-10-06T02:23:09.992467+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:44:41.608418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.049676+0000","last_change":"2023-10-06T02:23:09.048307+0000","last_active":"2023-10-06T02:23:10.049676+0000","last_peered":"2023-10-06T02:23:10.049676+0000","last_clean":"2023-10-06T02:23:10.049676+0000","last_became_active":"2023-10-06T02:23:09.047919+0000","last_became_peered":"2023-10-06T02:23:09.047919+0000","last_unstale":"2023-10-06T02:23:10.049676+0000","last_undegraded":"2023-10-06T02:23:10.049676+0000","last_fullsized":"2023-10-06T02:23:10.049676+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:28:59.763803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.004797+0000","last_change":"2023-10-06T02:23:10.001689+0000","last_active":"2023-10-06T02:23:10.004797+0000","last_peered":"2023-10-06T02:23:10.004797+0000","last_clean":"2023-10-06T02:23:10.004797+0000","last_became_active":"2023-10-06T02:23:10.001418+0000","last_became_peered":"2023-10-06T02:23:10.001418+0000","last_unstale":"2023-10-06T02:23:10.004797+0000","last_undegraded":"2023-10-06T02:23:10.004797+0000","last_fullsized":"2023-10-06T02:23:10.004797+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:53:54.469523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.977377+0000","last_change":"2023-10-06T02:23:09.041968+0000","last_active":"2023-10-06T02:23:09.977377+0000","last_peered":"2023-10-06T02:23:09.977377+0000","last_clean":"2023-10-06T02:23:09.977377+0000","last_became_active":"2023-10-06T02:23:09.041659+0000","last_became_peered":"2023-10-06T02:23:09.041659+0000","last_unstale":"2023-10-06T02:23:09.977377+0000","last_undegraded":"2023-10-06T02:23:09.977377+0000","last_fullsized":"2023-10-06T02:23:09.977377+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:44:15.545672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.992112+0000","last_change":"2023-10-06T02:23:09.989200+0000","last_active":"2023-10-06T02:23:09.992112+0000","last_peered":"2023-10-06T02:23:09.992112+0000","last_clean":"2023-10-06T02:23:09.992112+0000","last_became_active":"2023-10-06T02:23:09.988895+0000","last_became_peered":"2023-10-06T02:23:09.988895+0000","last_unstale":"2023-10-06T02:23:09.992112+0000","last_undegraded":"2023-10-06T02:23:09.992112+0000","last_fullsized":"2023-10-06T02:23:09.992112+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:44:41.608418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.984229+0000","last_change":"2023-10-06T02:23:09.048119+0000","last_active":"2023-10-06T02:23:09.984229+0000","last_peered":"2023-10-06T02:23:09.984229+0000","last_clean":"2023-10-06T02:23:09.984229+0000","last_became_active":"2023-10-06T02:23:09.047678+0000","last_became_peered":"2023-10-06T02:23:09.047678+0000","last_unstale":"2023-10-06T02:23:09.984229+0000","last_undegraded":"2023-10-06T02:23:09.984229+0000","last_fullsized":"2023-10-06T02:23:09.984229+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:42:11.693138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.997921+0000","last_change":"2023-10-06T02:23:09.994722+0000","last_active":"2023-10-06T02:23:09.997921+0000","last_peered":"2023-10-06T02:23:09.997921+0000","last_clean":"2023-10-06T02:23:09.997921+0000","last_became_active":"2023-10-06T02:23:09.994392+0000","last_became_peered":"2023-10-06T02:23:09.994392+0000","last_unstale":"2023-10-06T02:23:09.997921+0000","last_undegraded":"2023-10-06T02:23:09.997921+0000","last_fullsized":"2023-10-06T02:23:09.997921+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:21.438589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"22'8","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.049205+0000","last_change":"2023-10-06T02:23:09.034244+0000","last_active":"2023-10-06T02:23:10.049205+0000","last_peered":"2023-10-06T02:23:10.049205+0000","last_clean":"2023-10-06T02:23:10.049205+0000","last_became_active":"2023-10-06T02:23:09.033861+0000","last_became_peered":"2023-10-06T02:23:09.033861+0000","last_unstale":"2023-10-06T02:23:10.049205+0000","last_undegraded":"2023-10-06T02:23:10.049205+0000","last_fullsized":"2023-10-06T02:23:10.049205+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:41.039005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.990663+0000","last_change":"2023-10-06T02:23:09.988390+0000","last_active":"2023-10-06T02:23:09.990663+0000","last_peered":"2023-10-06T02:23:09.990663+0000","last_clean":"2023-10-06T02:23:09.990663+0000","last_became_active":"2023-10-06T02:23:09.988060+0000","last_became_peered":"2023-10-06T02:23:09.988060+0000","last_unstale":"2023-10-06T02:23:09.990663+0000","last_undegraded":"2023-10-06T02:23:09.990663+0000","last_fullsized":"2023-10-06T02:23:09.990663+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:43:01.259350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.044194+0000","last_change":"2023-10-06T02:23:09.048150+0000","last_active":"2023-10-06T02:23:10.044194+0000","last_peered":"2023-10-06T02:23:10.044194+0000","last_clean":"2023-10-06T02:23:10.044194+0000","last_became_active":"2023-10-06T02:23:09.047768+0000","last_became_peered":"2023-10-06T02:23:09.047768+0000","last_unstale":"2023-10-06T02:23:10.044194+0000","last_undegraded":"2023-10-06T02:23:10.044194+0000","last_fullsized":"2023-10-06T02:23:10.044194+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:03:50.725863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.992100+0000","last_change":"2023-10-06T02:23:09.990032+0000","last_active":"2023-10-06T02:23:09.992100+0000","last_peered":"2023-10-06T02:23:09.992100+0000","last_clean":"2023-10-06T02:23:09.992100+0000","last_became_active":"2023-10-06T02:23:09.988801+0000","last_became_peered":"2023-10-06T02:23:09.988801+0000","last_unstale":"2023-10-06T02:23:09.992100+0000","last_undegraded":"2023-10-06T02:23:09.992100+0000","last_fullsized":"2023-10-06T02:23:09.992100+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:12.998563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.044129+0000","last_change":"2023-10-06T02:23:09.048438+0000","last_active":"2023-10-06T02:23:10.044129+0000","last_peered":"2023-10-06T02:23:10.044129+0000","last_clean":"2023-10-06T02:23:10.044129+0000","last_became_active":"2023-10-06T02:23:09.048051+0000","last_became_peered":"2023-10-06T02:23:09.048051+0000","last_unstale":"2023-10-06T02:23:10.044129+0000","last_undegraded":"2023-10-06T02:23:10.044129+0000","last_fullsized":"2023-10-06T02:23:10.044129+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:21:32.730148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.010287+0000","last_change":"2023-10-06T02:23:10.002031+0000","last_active":"2023-10-06T02:23:10.010287+0000","last_peered":"2023-10-06T02:23:10.010287+0000","last_clean":"2023-10-06T02:23:10.010287+0000","last_became_active":"2023-10-06T02:23:10.001754+0000","last_became_peered":"2023-10-06T02:23:10.001754+0000","last_unstale":"2023-10-06T02:23:10.010287+0000","last_undegraded":"2023-10-06T02:23:10.010287+0000","last_fullsized":"2023-10-06T02:23:10.010287+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:45:34.162627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.044320+0000","last_change":"2023-10-06T02:23:09.038304+0000","last_active":"2023-10-06T02:23:10.044320+0000","last_peered":"2023-10-06T02:23:10.044320+0000","last_clean":"2023-10-06T02:23:10.044320+0000","last_became_active":"2023-10-06T02:23:09.038031+0000","last_became_peered":"2023-10-06T02:23:09.038031+0000","last_unstale":"2023-10-06T02:23:10.044320+0000","last_undegraded":"2023-10-06T02:23:10.044320+0000","last_fullsized":"2023-10-06T02:23:10.044320+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:31:52.666890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.004414+0000","last_change":"2023-10-06T02:23:10.000692+0000","last_active":"2023-10-06T02:23:10.004414+0000","last_peered":"2023-10-06T02:23:10.004414+0000","last_clean":"2023-10-06T02:23:10.004414+0000","last_became_active":"2023-10-06T02:23:10.000368+0000","last_became_peered":"2023-10-06T02:23:10.000368+0000","last_unstale":"2023-10-06T02:23:10.004414+0000","last_undegraded":"2023-10-06T02:23:10.004414+0000","last_fullsized":"2023-10-06T02:23:10.004414+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:53:01.233080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.974959+0000","last_change":"2023-10-06T02:23:09.037238+0000","last_active":"2023-10-06T02:23:09.974959+0000","last_peered":"2023-10-06T02:23:09.974959+0000","last_clean":"2023-10-06T02:23:09.974959+0000","last_became_active":"2023-10-06T02:23:09.036871+0000","last_became_peered":"2023-10-06T02:23:09.036871+0000","last_unstale":"2023-10-06T02:23:09.974959+0000","last_undegraded":"2023-10-06T02:23:09.974959+0000","last_fullsized":"2023-10-06T02:23:09.974959+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:44:15.545672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.002856+0000","last_change":"2023-10-06T02:23:10.000807+0000","last_active":"2023-10-06T02:23:10.002856+0000","last_peered":"2023-10-06T02:23:10.002856+0000","last_clean":"2023-10-06T02:23:10.002856+0000","last_became_active":"2023-10-06T02:23:10.000502+0000","last_became_peered":"2023-10-06T02:23:10.000502+0000","last_unstale":"2023-10-06T02:23:10.002856+0000","last_undegraded":"2023-10-06T02:23:10.002856+0000","last_fullsized":"2023-10-06T02:23:10.002856+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:31:27.242353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.042047+0000","last_change":"2023-10-06T02:23:09.044437+0000","last_active":"2023-10-06T02:23:10.042047+0000","last_peered":"2023-10-06T02:23:10.042047+0000","last_clean":"2023-10-06T02:23:10.042047+0000","last_became_active":"2023-10-06T02:23:09.042657+0000","last_became_peered":"2023-10-06T02:23:09.042657+0000","last_unstale":"2023-10-06T02:23:10.042047+0000","last_undegraded":"2023-10-06T02:23:10.042047+0000","last_fullsized":"2023-10-06T02:23:10.042047+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:57:40.559177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.994640+0000","last_change":"2023-10-06T02:23:09.990834+0000","last_active":"2023-10-06T02:23:09.994640+0000","last_peered":"2023-10-06T02:23:09.994640+0000","last_clean":"2023-10-06T02:23:09.994640+0000","last_became_active":"2023-10-06T02:23:09.990192+0000","last_became_peered":"2023-10-06T02:23:09.990192+0000","last_unstale":"2023-10-06T02:23:09.994640+0000","last_undegraded":"2023-10-06T02:23:09.994640+0000","last_fullsized":"2023-10-06T02:23:09.994640+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:18:32.416904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.980739+0000","last_change":"2023-10-06T02:23:09.044880+0000","last_active":"2023-10-06T02:23:09.980739+0000","last_peered":"2023-10-06T02:23:09.980739+0000","last_clean":"2023-10-06T02:23:09.980739+0000","last_became_active":"2023-10-06T02:23:09.044556+0000","last_became_peered":"2023-10-06T02:23:09.044556+0000","last_unstale":"2023-10-06T02:23:09.980739+0000","last_undegraded":"2023-10-06T02:23:09.980739+0000","last_fullsized":"2023-10-06T02:23:09.980739+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:58.976414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.994703+0000","last_change":"2023-10-06T02:23:09.992094+0000","last_active":"2023-10-06T02:23:09.994703+0000","last_peered":"2023-10-06T02:23:09.994703+0000","last_clean":"2023-10-06T02:23:09.994703+0000","last_became_active":"2023-10-06T02:23:09.991699+0000","last_became_peered":"2023-10-06T02:23:09.991699+0000","last_unstale":"2023-10-06T02:23:09.994703+0000","last_undegraded":"2023-10-06T02:23:09.994703+0000","last_fullsized":"2023-10-06T02:23:09.994703+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:21.438589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.986400+0000","last_change":"2023-10-06T02:23:09.041591+0000","last_active":"2023-10-06T02:23:09.986400+0000","last_peered":"2023-10-06T02:23:09.986400+0000","last_clean":"2023-10-06T02:23:09.986400+0000","last_became_active":"2023-10-06T02:23:09.041239+0000","last_became_peered":"2023-10-06T02:23:09.041239+0000","last_unstale":"2023-10-06T02:23:09.986400+0000","last_undegraded":"2023-10-06T02:23:09.986400+0000","last_fullsized":"2023-10-06T02:23:09.986400+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:17:32.434412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.994672+0000","last_change":"2023-10-06T02:23:09.991765+0000","last_active":"2023-10-06T02:23:09.994672+0000","last_peered":"2023-10-06T02:23:09.994672+0000","last_clean":"2023-10-06T02:23:09.994672+0000","last_became_active":"2023-10-06T02:23:09.990991+0000","last_became_peered":"2023-10-06T02:23:09.990991+0000","last_unstale":"2023-10-06T02:23:09.994672+0000","last_undegraded":"2023-10-06T02:23:09.994672+0000","last_fullsized":"2023-10-06T02:23:09.994672+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:18:23.992787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.976060+0000","last_change":"2023-10-06T02:23:09.046400+0000","last_active":"2023-10-06T02:23:09.976060+0000","last_peered":"2023-10-06T02:23:09.976060+0000","last_clean":"2023-10-06T02:23:09.976060+0000","last_became_active":"2023-10-06T02:23:09.046039+0000","last_became_peered":"2023-10-06T02:23:09.046039+0000","last_unstale":"2023-10-06T02:23:09.976060+0000","last_undegraded":"2023-10-06T02:23:09.976060+0000","last_fullsized":"2023-10-06T02:23:09.976060+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:44:15.545672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.005866+0000","last_change":"2023-10-06T02:23:10.001037+0000","last_active":"2023-10-06T02:23:10.005866+0000","last_peered":"2023-10-06T02:23:10.005866+0000","last_clean":"2023-10-06T02:23:10.005866+0000","last_became_active":"2023-10-06T02:23:10.000479+0000","last_became_peered":"2023-10-06T02:23:10.000479+0000","last_unstale":"2023-10-06T02:23:10.005866+0000","last_undegraded":"2023-10-06T02:23:10.005866+0000","last_fullsized":"2023-10-06T02:23:10.005866+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:50:19.168882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.996789+0000","last_change":"2023-10-06T02:23:02.054039+0000","last_active":"2023-10-06T02:23:09.996789+0000","last_peered":"2023-10-06T02:23:09.996789+0000","last_clean":"2023-10-06T02:23:09.996789+0000","last_became_active":"2023-10-06T02:23:02.053742+0000","last_became_peered":"2023-10-06T02:23:02.053742+0000","last_unstale":"2023-10-06T02:23:09.996789+0000","last_undegraded":"2023-10-06T02:23:09.996789+0000","last_fullsized":"2023-10-06T02:23:09.996789+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:58:18.846475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.975329+0000","last_change":"2023-10-06T02:23:09.037244+0000","last_active":"2023-10-06T02:23:09.975329+0000","last_peered":"2023-10-06T02:23:09.975329+0000","last_clean":"2023-10-06T02:23:09.975329+0000","last_became_active":"2023-10-06T02:23:09.036880+0000","last_became_peered":"2023-10-06T02:23:09.036880+0000","last_unstale":"2023-10-06T02:23:09.975329+0000","last_undegraded":"2023-10-06T02:23:09.975329+0000","last_fullsized":"2023-10-06T02:23:09.975329+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:36:27.554258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.005453+0000","last_change":"2023-10-06T02:23:09.999775+0000","last_active":"2023-10-06T02:23:10.005453+0000","last_peered":"2023-10-06T02:23:10.005453+0000","last_clean":"2023-10-06T02:23:10.005453+0000","last_became_active":"2023-10-06T02:23:09.999472+0000","last_became_peered":"2023-10-06T02:23:09.999472+0000","last_unstale":"2023-10-06T02:23:10.005453+0000","last_undegraded":"2023-10-06T02:23:10.005453+0000","last_fullsized":"2023-10-06T02:23:10.005453+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:44:41.608418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.977354+0000","last_change":"2023-10-06T02:23:01.988631+0000","last_active":"2023-10-06T02:23:09.977354+0000","last_peered":"2023-10-06T02:23:09.977354+0000","last_clean":"2023-10-06T02:23:09.977354+0000","last_became_active":"2023-10-06T02:23:01.988303+0000","last_became_peered":"2023-10-06T02:23:01.988303+0000","last_unstale":"2023-10-06T02:23:09.977354+0000","last_undegraded":"2023-10-06T02:23:09.977354+0000","last_fullsized":"2023-10-06T02:23:09.977354+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:38.211466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.986211+0000","last_change":"2023-10-06T02:23:09.041835+0000","last_active":"2023-10-06T02:23:09.986211+0000","last_peered":"2023-10-06T02:23:09.986211+0000","last_clean":"2023-10-06T02:23:09.986211+0000","last_became_active":"2023-10-06T02:23:09.041521+0000","last_became_peered":"2023-10-06T02:23:09.041521+0000","last_unstale":"2023-10-06T02:23:09.986211+0000","last_undegraded":"2023-10-06T02:23:09.986211+0000","last_fullsized":"2023-10-06T02:23:09.986211+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:38.263723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.003455+0000","last_change":"2023-10-06T02:23:09.995055+0000","last_active":"2023-10-06T02:23:10.003455+0000","last_peered":"2023-10-06T02:23:10.003455+0000","last_clean":"2023-10-06T02:23:10.003455+0000","last_became_active":"2023-10-06T02:23:09.994216+0000","last_became_peered":"2023-10-06T02:23:09.994216+0000","last_unstale":"2023-10-06T02:23:10.003455+0000","last_undegraded":"2023-10-06T02:23:10.003455+0000","last_fullsized":"2023-10-06T02:23:10.003455+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:47:34.184576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.972958+0000","last_change":"2023-10-06T02:23:02.054338+0000","last_active":"2023-10-06T02:23:09.972958+0000","last_peered":"2023-10-06T02:23:09.972958+0000","last_clean":"2023-10-06T02:23:09.972958+0000","last_became_active":"2023-10-06T02:23:02.053991+0000","last_became_peered":"2023-10-06T02:23:02.053991+0000","last_unstale":"2023-10-06T02:23:09.972958+0000","last_undegraded":"2023-10-06T02:23:09.972958+0000","last_fullsized":"2023-10-06T02:23:09.972958+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:39:30.192838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.996619+0000","last_change":"2023-10-06T02:23:09.039360+0000","last_active":"2023-10-06T02:23:09.996619+0000","last_peered":"2023-10-06T02:23:09.996619+0000","last_clean":"2023-10-06T02:23:09.996619+0000","last_became_active":"2023-10-06T02:23:09.038571+0000","last_became_peered":"2023-10-06T02:23:09.038571+0000","last_unstale":"2023-10-06T02:23:09.996619+0000","last_undegraded":"2023-10-06T02:23:09.996619+0000","last_fullsized":"2023-10-06T02:23:09.996619+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:58.976414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.005846+0000","last_change":"2023-10-06T02:23:10.000821+0000","last_active":"2023-10-06T02:23:10.005846+0000","last_peered":"2023-10-06T02:23:10.005846+0000","last_clean":"2023-10-06T02:23:10.005846+0000","last_became_active":"2023-10-06T02:23:10.000203+0000","last_became_peered":"2023-10-06T02:23:10.000203+0000","last_unstale":"2023-10-06T02:23:10.005846+0000","last_undegraded":"2023-10-06T02:23:10.005846+0000","last_fullsized":"2023-10-06T02:23:10.005846+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:43:01.259350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.975054+0000","last_change":"2023-10-06T02:23:01.988140+0000","last_active":"2023-10-06T02:23:09.975054+0000","last_peered":"2023-10-06T02:23:09.975054+0000","last_clean":"2023-10-06T02:23:09.975054+0000","last_became_active":"2023-10-06T02:23:01.987769+0000","last_became_peered":"2023-10-06T02:23:01.987769+0000","last_unstale":"2023-10-06T02:23:09.975054+0000","last_undegraded":"2023-10-06T02:23:09.975054+0000","last_fullsized":"2023-10-06T02:23:09.975054+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:20:08.391912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":166,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.983731+0000","last_change":"2023-10-06T02:22:55.032519+0000","last_active":"2023-10-06T02:23:09.983731+0000","last_peered":"2023-10-06T02:23:09.983731+0000","last_clean":"2023-10-06T02:23:09.983731+0000","last_became_active":"2023-10-06T02:22:55.031367+0000","last_became_peered":"2023-10-06T02:22:55.031367+0000","last_unstale":"2023-10-06T02:23:09.983731+0000","last_undegraded":"2023-10-06T02:23:09.983731+0000","last_fullsized":"2023-10-06T02:23:09.983731+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:22:53.309097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:22:53.309097+0000","last_clean_scrub_stamp":"2023-10-06T02:22:53.309097+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:53:46.520909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.975628+0000","last_change":"2023-10-06T02:23:09.041907+0000","last_active":"2023-10-06T02:23:09.975628+0000","last_peered":"2023-10-06T02:23:09.975628+0000","last_clean":"2023-10-06T02:23:09.975628+0000","last_became_active":"2023-10-06T02:23:09.036698+0000","last_became_peered":"2023-10-06T02:23:09.036698+0000","last_unstale":"2023-10-06T02:23:09.975628+0000","last_undegraded":"2023-10-06T02:23:09.975628+0000","last_fullsized":"2023-10-06T02:23:09.975628+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:57.514051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.003862+0000","last_change":"2023-10-06T02:23:09.997846+0000","last_active":"2023-10-06T02:23:10.003862+0000","last_peered":"2023-10-06T02:23:10.003862+0000","last_clean":"2023-10-06T02:23:10.003862+0000","last_became_active":"2023-10-06T02:23:09.997558+0000","last_became_peered":"2023-10-06T02:23:09.997558+0000","last_unstale":"2023-10-06T02:23:10.003862+0000","last_undegraded":"2023-10-06T02:23:10.003862+0000","last_fullsized":"2023-10-06T02:23:10.003862+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:29:00.771363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.984381+0000","last_change":"2023-10-06T02:23:01.990773+0000","last_active":"2023-10-06T02:23:09.984381+0000","last_peered":"2023-10-06T02:23:09.984381+0000","last_clean":"2023-10-06T02:23:09.984381+0000","last_became_active":"2023-10-06T02:23:01.990511+0000","last_became_peered":"2023-10-06T02:23:01.990511+0000","last_unstale":"2023-10-06T02:23:09.984381+0000","last_undegraded":"2023-10-06T02:23:09.984381+0000","last_fullsized":"2023-10-06T02:23:09.984381+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:42:54.797731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.977043+0000","last_change":"2023-10-06T02:23:09.047067+0000","last_active":"2023-10-06T02:23:09.977043+0000","last_peered":"2023-10-06T02:23:09.977043+0000","last_clean":"2023-10-06T02:23:09.977043+0000","last_became_active":"2023-10-06T02:23:09.046735+0000","last_became_peered":"2023-10-06T02:23:09.046735+0000","last_unstale":"2023-10-06T02:23:09.977043+0000","last_undegraded":"2023-10-06T02:23:09.977043+0000","last_fullsized":"2023-10-06T02:23:09.977043+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:57.514051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.005223+0000","last_change":"2023-10-06T02:23:10.001635+0000","last_active":"2023-10-06T02:23:10.005223+0000","last_peered":"2023-10-06T02:23:10.005223+0000","last_clean":"2023-10-06T02:23:10.005223+0000","last_became_active":"2023-10-06T02:23:10.001341+0000","last_became_peered":"2023-10-06T02:23:10.001341+0000","last_unstale":"2023-10-06T02:23:10.005223+0000","last_undegraded":"2023-10-06T02:23:10.005223+0000","last_fullsized":"2023-10-06T02:23:10.005223+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:44:41.608418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.984779+0000","last_change":"2023-10-06T02:23:02.054069+0000","last_active":"2023-10-06T02:23:09.984779+0000","last_peered":"2023-10-06T02:23:09.984779+0000","last_clean":"2023-10-06T02:23:09.984779+0000","last_became_active":"2023-10-06T02:23:02.053739+0000","last_became_peered":"2023-10-06T02:23:02.053739+0000","last_unstale":"2023-10-06T02:23:09.984779+0000","last_undegraded":"2023-10-06T02:23:09.984779+0000","last_fullsized":"2023-10-06T02:23:09.984779+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:57:48.822608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.996931+0000","last_change":"2023-10-06T02:23:09.037195+0000","last_active":"2023-10-06T02:23:09.996931+0000","last_peered":"2023-10-06T02:23:09.996931+0000","last_clean":"2023-10-06T02:23:09.996931+0000","last_became_active":"2023-10-06T02:23:09.036925+0000","last_became_peered":"2023-10-06T02:23:09.036925+0000","last_unstale":"2023-10-06T02:23:09.996931+0000","last_undegraded":"2023-10-06T02:23:09.996931+0000","last_fullsized":"2023-10-06T02:23:09.996931+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:38.263723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.994210+0000","last_change":"2023-10-06T02:23:09.986467+0000","last_active":"2023-10-06T02:23:09.994210+0000","last_peered":"2023-10-06T02:23:09.994210+0000","last_clean":"2023-10-06T02:23:09.994210+0000","last_became_active":"2023-10-06T02:23:09.985750+0000","last_became_peered":"2023-10-06T02:23:09.985750+0000","last_unstale":"2023-10-06T02:23:09.994210+0000","last_undegraded":"2023-10-06T02:23:09.994210+0000","last_fullsized":"2023-10-06T02:23:09.994210+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:18:23.992787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.044047+0000","last_change":"2023-10-06T02:23:09.041622+0000","last_active":"2023-10-06T02:23:10.044047+0000","last_peered":"2023-10-06T02:23:10.044047+0000","last_clean":"2023-10-06T02:23:10.044047+0000","last_became_active":"2023-10-06T02:23:09.036680+0000","last_became_peered":"2023-10-06T02:23:09.036680+0000","last_unstale":"2023-10-06T02:23:10.044047+0000","last_undegraded":"2023-10-06T02:23:10.044047+0000","last_fullsized":"2023-10-06T02:23:10.044047+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:36:27.554258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.993558+0000","last_change":"2023-10-06T02:23:01.989934+0000","last_active":"2023-10-06T02:23:09.993558+0000","last_peered":"2023-10-06T02:23:09.993558+0000","last_clean":"2023-10-06T02:23:09.993558+0000","last_became_active":"2023-10-06T02:23:01.989706+0000","last_became_peered":"2023-10-06T02:23:01.989706+0000","last_unstale":"2023-10-06T02:23:09.993558+0000","last_undegraded":"2023-10-06T02:23:09.993558+0000","last_fullsized":"2023-10-06T02:23:09.993558+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:06.212251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.989446+0000","last_change":"2023-10-06T02:23:09.987681+0000","last_active":"2023-10-06T02:23:09.989446+0000","last_peered":"2023-10-06T02:23:09.989446+0000","last_clean":"2023-10-06T02:23:09.989446+0000","last_became_active":"2023-10-06T02:23:09.987439+0000","last_became_peered":"2023-10-06T02:23:09.987439+0000","last_unstale":"2023-10-06T02:23:09.989446+0000","last_undegraded":"2023-10-06T02:23:09.989446+0000","last_fullsized":"2023-10-06T02:23:09.989446+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:31:27.242353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.980493+0000","last_change":"2023-10-06T02:23:01.992935+0000","last_active":"2023-10-06T02:23:09.980493+0000","last_peered":"2023-10-06T02:23:09.980493+0000","last_clean":"2023-10-06T02:23:09.980493+0000","last_became_active":"2023-10-06T02:23:01.992568+0000","last_became_peered":"2023-10-06T02:23:01.992568+0000","last_unstale":"2023-10-06T02:23:09.980493+0000","last_undegraded":"2023-10-06T02:23:09.980493+0000","last_fullsized":"2023-10-06T02:23:09.980493+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:06.212251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"23'1","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T02:23:11.091780+0000","last_change":"2023-10-06T02:23:09.039504+0000","last_active":"2023-10-06T02:23:11.091780+0000","last_peered":"2023-10-06T02:23:11.091780+0000","last_clean":"2023-10-06T02:23:11.091780+0000","last_became_active":"2023-10-06T02:23:09.039056+0000","last_became_peered":"2023-10-06T02:23:09.039056+0000","last_unstale":"2023-10-06T02:23:11.091780+0000","last_undegraded":"2023-10-06T02:23:11.091780+0000","last_fullsized":"2023-10-06T02:23:11.091780+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:38.263723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.995987+0000","last_change":"2023-10-06T02:23:09.990800+0000","last_active":"2023-10-06T02:23:09.995987+0000","last_peered":"2023-10-06T02:23:09.995987+0000","last_clean":"2023-10-06T02:23:09.995987+0000","last_became_active":"2023-10-06T02:23:09.990531+0000","last_became_peered":"2023-10-06T02:23:09.990531+0000","last_unstale":"2023-10-06T02:23:09.995987+0000","last_undegraded":"2023-10-06T02:23:09.995987+0000","last_fullsized":"2023-10-06T02:23:09.995987+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:39:39.959479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.045165+0000","last_change":"2023-10-06T02:23:09.046230+0000","last_active":"2023-10-06T02:23:10.045165+0000","last_peered":"2023-10-06T02:23:10.045165+0000","last_clean":"2023-10-06T02:23:10.045165+0000","last_became_active":"2023-10-06T02:23:09.045976+0000","last_became_peered":"2023-10-06T02:23:09.045976+0000","last_unstale":"2023-10-06T02:23:10.045165+0000","last_undegraded":"2023-10-06T02:23:10.045165+0000","last_fullsized":"2023-10-06T02:23:10.045165+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:36:27.554258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.998669+0000","last_change":"2023-10-06T02:23:09.995680+0000","last_active":"2023-10-06T02:23:09.998669+0000","last_peered":"2023-10-06T02:23:09.998669+0000","last_clean":"2023-10-06T02:23:09.998669+0000","last_became_active":"2023-10-06T02:23:09.995375+0000","last_became_peered":"2023-10-06T02:23:09.995375+0000","last_unstale":"2023-10-06T02:23:09.998669+0000","last_undegraded":"2023-10-06T02:23:09.998669+0000","last_fullsized":"2023-10-06T02:23:09.998669+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:36:28.561818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.046988+0000","last_change":"2023-10-06T02:23:09.043361+0000","last_active":"2023-10-06T02:23:10.046988+0000","last_peered":"2023-10-06T02:23:10.046988+0000","last_clean":"2023-10-06T02:23:10.046988+0000","last_became_active":"2023-10-06T02:23:09.042973+0000","last_became_peered":"2023-10-06T02:23:09.042973+0000","last_unstale":"2023-10-06T02:23:10.046988+0000","last_undegraded":"2023-10-06T02:23:10.046988+0000","last_fullsized":"2023-10-06T02:23:10.046988+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:31:52.666890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.992487+0000","last_change":"2023-10-06T02:23:09.988158+0000","last_active":"2023-10-06T02:23:09.992487+0000","last_peered":"2023-10-06T02:23:09.992487+0000","last_clean":"2023-10-06T02:23:09.992487+0000","last_became_active":"2023-10-06T02:23:09.987844+0000","last_became_peered":"2023-10-06T02:23:09.987844+0000","last_unstale":"2023-10-06T02:23:09.992487+0000","last_undegraded":"2023-10-06T02:23:09.992487+0000","last_fullsized":"2023-10-06T02:23:09.992487+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:10:59.616168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.045352+0000","last_change":"2023-10-06T02:23:09.044069+0000","last_active":"2023-10-06T02:23:10.045352+0000","last_peered":"2023-10-06T02:23:10.045352+0000","last_clean":"2023-10-06T02:23:10.045352+0000","last_became_active":"2023-10-06T02:23:09.043700+0000","last_became_peered":"2023-10-06T02:23:09.043700+0000","last_unstale":"2023-10-06T02:23:10.045352+0000","last_undegraded":"2023-10-06T02:23:10.045352+0000","last_fullsized":"2023-10-06T02:23:10.045352+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:57:40.559177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.010156+0000","last_change":"2023-10-06T02:23:09.999700+0000","last_active":"2023-10-06T02:23:10.010156+0000","last_peered":"2023-10-06T02:23:10.010156+0000","last_clean":"2023-10-06T02:23:10.010156+0000","last_became_active":"2023-10-06T02:23:09.997418+0000","last_became_peered":"2023-10-06T02:23:09.997418+0000","last_unstale":"2023-10-06T02:23:10.010156+0000","last_undegraded":"2023-10-06T02:23:10.010156+0000","last_fullsized":"2023-10-06T02:23:10.010156+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:53:54.469523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.041904+0000","last_change":"2023-10-06T02:23:09.034024+0000","last_active":"2023-10-06T02:23:10.041904+0000","last_peered":"2023-10-06T02:23:10.041904+0000","last_clean":"2023-10-06T02:23:10.041904+0000","last_became_active":"2023-10-06T02:23:09.033389+0000","last_became_peered":"2023-10-06T02:23:09.033389+0000","last_unstale":"2023-10-06T02:23:10.041904+0000","last_undegraded":"2023-10-06T02:23:10.041904+0000","last_fullsized":"2023-10-06T02:23:10.041904+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:03:50.725863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.991920+0000","last_change":"2023-10-06T02:23:09.984080+0000","last_active":"2023-10-06T02:23:09.991920+0000","last_peered":"2023-10-06T02:23:09.991920+0000","last_clean":"2023-10-06T02:23:09.991920+0000","last_became_active":"2023-10-06T02:23:09.983808+0000","last_became_peered":"2023-10-06T02:23:09.983808+0000","last_unstale":"2023-10-06T02:23:09.991920+0000","last_undegraded":"2023-10-06T02:23:09.991920+0000","last_fullsized":"2023-10-06T02:23:09.991920+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:12.998563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"22'4","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.045612+0000","last_change":"2023-10-06T02:23:09.037190+0000","last_active":"2023-10-06T02:23:10.045612+0000","last_peered":"2023-10-06T02:23:10.045612+0000","last_clean":"2023-10-06T02:23:10.045612+0000","last_became_active":"2023-10-06T02:23:09.036872+0000","last_became_peered":"2023-10-06T02:23:09.036872+0000","last_unstale":"2023-10-06T02:23:10.045612+0000","last_undegraded":"2023-10-06T02:23:10.045612+0000","last_fullsized":"2023-10-06T02:23:10.045612+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:57:40.559177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.994436+0000","last_change":"2023-10-06T02:23:09.989896+0000","last_active":"2023-10-06T02:23:09.994436+0000","last_peered":"2023-10-06T02:23:09.994436+0000","last_clean":"2023-10-06T02:23:09.994436+0000","last_became_active":"2023-10-06T02:23:09.989525+0000","last_became_peered":"2023-10-06T02:23:09.989525+0000","last_unstale":"2023-10-06T02:23:09.994436+0000","last_undegraded":"2023-10-06T02:23:09.994436+0000","last_fullsized":"2023-10-06T02:23:09.994436+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:58:41.925466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.992536+0000","last_change":"2023-10-06T02:23:09.988852+0000","last_active":"2023-10-06T02:23:09.992536+0000","last_peered":"2023-10-06T02:23:09.992536+0000","last_clean":"2023-10-06T02:23:09.992536+0000","last_became_active":"2023-10-06T02:23:09.988527+0000","last_became_peered":"2023-10-06T02:23:09.988527+0000","last_unstale":"2023-10-06T02:23:09.992536+0000","last_undegraded":"2023-10-06T02:23:09.992536+0000","last_fullsized":"2023-10-06T02:23:09.992536+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:03:51.733423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.001202+0000","last_change":"2023-10-06T02:23:09.999698+0000","last_active":"2023-10-06T02:23:10.001202+0000","last_peered":"2023-10-06T02:23:10.001202+0000","last_clean":"2023-10-06T02:23:10.001202+0000","last_became_active":"2023-10-06T02:23:09.999418+0000","last_became_peered":"2023-10-06T02:23:09.999418+0000","last_unstale":"2023-10-06T02:23:10.001202+0000","last_undegraded":"2023-10-06T02:23:10.001202+0000","last_fullsized":"2023-10-06T02:23:10.001202+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:10:59.616168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.010180+0000","last_change":"2023-10-06T02:23:10.000203+0000","last_active":"2023-10-06T02:23:10.010180+0000","last_peered":"2023-10-06T02:23:10.010180+0000","last_clean":"2023-10-06T02:23:10.010180+0000","last_became_active":"2023-10-06T02:23:09.999397+0000","last_became_peered":"2023-10-06T02:23:09.999397+0000","last_unstale":"2023-10-06T02:23:10.010180+0000","last_undegraded":"2023-10-06T02:23:10.010180+0000","last_fullsized":"2023-10-06T02:23:10.010180+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:31:27.242353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.989526+0000","last_change":"2023-10-06T02:23:09.986996+0000","last_active":"2023-10-06T02:23:09.989526+0000","last_peered":"2023-10-06T02:23:09.989526+0000","last_clean":"2023-10-06T02:23:09.989526+0000","last_became_active":"2023-10-06T02:23:09.986723+0000","last_became_peered":"2023-10-06T02:23:09.986723+0000","last_unstale":"2023-10-06T02:23:09.989526+0000","last_undegraded":"2023-10-06T02:23:09.989526+0000","last_fullsized":"2023-10-06T02:23:09.989526+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:19:29.462977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.007171+0000","last_change":"2023-10-06T02:23:10.003345+0000","last_active":"2023-10-06T02:23:10.007171+0000","last_peered":"2023-10-06T02:23:10.007171+0000","last_clean":"2023-10-06T02:23:10.007171+0000","last_became_active":"2023-10-06T02:23:10.003023+0000","last_became_peered":"2023-10-06T02:23:10.003023+0000","last_unstale":"2023-10-06T02:23:10.007171+0000","last_undegraded":"2023-10-06T02:23:10.007171+0000","last_fullsized":"2023-10-06T02:23:10.007171+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:18:23.992787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298132,"kb_used_data":844,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073708,"statfs":{"total":96636764160,"available":96331476992,"internally_reserved":0,"allocated":864256,"data_stored":681439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41399999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297524,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074316,"statfs":{"total":96636764160,"available":96332099584,"internally_reserved":0,"allocated":241664,"data_stored":89319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66500000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79900000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297524,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074316,"statfs":{"total":96636764160,"available":96332099584,"internally_reserved":0,"allocated":241664,"data_stored":89319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2230000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0840000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.266}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9279999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9830000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.829}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3340000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298108,"kb_used_data":820,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073732,"statfs":{"total":96636764160,"available":96331501568,"internally_reserved":0,"allocated":839680,"data_stored":680241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69299999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86699999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96499999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297524,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074316,"statfs":{"total":96636764160,"available":96332099584,"internally_reserved":0,"allocated":241664,"data_stored":89319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79200000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83699999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297528,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074312,"statfs":{"total":96636764160,"available":96332095488,"internally_reserved":0,"allocated":245760,"data_stored":89409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.024}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.395}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0870000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.214}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.131}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.357}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.27}]}]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297544,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074296,"statfs":{"total":96636764160,"available":96332079104,"internally_reserved":0,"allocated":262144,"data_stored":90911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38500000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64200000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297532,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074308,"statfs":{"total":96636764160,"available":96332091392,"internally_reserved":0,"allocated":249856,"data_stored":89943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75800000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T02:23:24.915 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T02:23:25.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.136+0000 7f4400b9d700 1 Processor -- start 2023-10-06T02:23:25.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.136+0000 7f4400b9d700 1 -- start start 2023-10-06T02:23:25.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.138+0000 7f4400b9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f43fc13fb20 0x7f43fc15a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:25.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.138+0000 7f4400b9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43fc15ae70 con 0x7f43fc06aaa0 2023-10-06T02:23:25.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.138+0000 7f4400b9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43fc06af60 con 0x7f43fc13fef0 2023-10-06T02:23:25.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.138+0000 7f4400b9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43fc06b240 con 0x7f43fc13fb20 2023-10-06T02:23:25.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.138+0000 7f43fb7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f43fc13fb20 0x7f43fc15a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:25.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.138+0000 7f43fb7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f43fc13fb20 0x7f43fc15a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37760/0 (socket says 172.21.15.31:37760) 2023-10-06T02:23:25.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.138+0000 7f43fb7fe700 1 -- 172.21.15.31:0/3864165244 learned_addr learned my addr 172.21.15.31:0/3864165244 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:25.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.138+0000 7f43fbfff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f43fc06aaa0 0x7f43fc0c9d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49360/0 (socket says 172.21.15.31:49360) 2023-10-06T02:23:25.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.138+0000 7f43fb7fe700 1 -- 172.21.15.31:0/3864165244 >> v1:172.21.15.62:6789/0 conn(0x7f43fc13fef0 legacy=0x7f43fc0c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:25.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.138+0000 7f43fb7fe700 1 -- 172.21.15.31:0/3864165244 >> v1:172.21.15.31:6789/0 conn(0x7f43fc06aaa0 legacy=0x7f43fc0c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:25.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.138+0000 7f43fb7fe700 1 -- 172.21.15.31:0/3864165244 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43fc06b520 con 0x7f43fc13fb20 2023-10-06T02:23:25.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.139+0000 7f43fb7fe700 1 --2- 172.21.15.31:0/3864165244 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43fc13fb20 0x7f43fc15a930 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f43ec0099b0 tx=0x7f43ec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a8832523d20db58 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:25.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.139+0000 7f43fa7fc700 1 -- 172.21.15.31:0/3864165244 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43ec00b860 con 0x7f43fc13fb20 2023-10-06T02:23:25.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.139+0000 7f43fa7fc700 1 -- 172.21.15.31:0/3864165244 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43ec00b9c0 con 0x7f43fc13fb20 2023-10-06T02:23:25.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.139+0000 7f43fa7fc700 1 -- 172.21.15.31:0/3864165244 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43ec00bd60 con 0x7f43fc13fb20 2023-10-06T02:23:25.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.139+0000 7f4400b9d700 1 -- 172.21.15.31:0/3864165244 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43fc13fb20 msgr2=0x7f43fc15a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:25.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.139+0000 7f4400b9d700 1 --2- 172.21.15.31:0/3864165244 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43fc13fb20 0x7f43fc15a930 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f43ec0099b0 tx=0x7f43ec009250 comp rx=0 tx=0).stop 2023-10-06T02:23:25.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.139+0000 7f4400b9d700 1 -- 172.21.15.31:0/3864165244 shutdown_connections 2023-10-06T02:23:25.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.139+0000 7f4400b9d700 1 --2- 172.21.15.31:0/3864165244 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43fc13fb20 0x7f43fc15a930 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:25.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.140+0000 7f4400b9d700 1 -- 172.21.15.31:0/3864165244 >> 172.21.15.31:0/3864165244 conn(0x7f43fc06d170 msgr2=0x7f43fc0cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:25.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.140+0000 7f4400b9d700 1 -- 172.21.15.31:0/3864165244 shutdown_connections 2023-10-06T02:23:25.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.140+0000 7f4400b9d700 1 -- 172.21.15.31:0/3864165244 wait complete. 2023-10-06T02:23:25.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.141+0000 7f4400b9d700 1 Processor -- start 2023-10-06T02:23:25.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.141+0000 7f4400b9d700 1 -- start start 2023-10-06T02:23:25.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.142+0000 7f4400b9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f43fc06aaa0 0x7f43fc148b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:25.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.142+0000 7f4400b9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43fc13fb20 0x7f43fc0aeed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:25.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.142+0000 7f4400b9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43fc13fef0 0x7f43fc0c75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:25.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.142+0000 7f4400b9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43fc1490d0 con 0x7f43fc13fb20 2023-10-06T02:23:25.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.142+0000 7f4400b9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43fc0abc50 con 0x7f43fc13fef0 2023-10-06T02:23:25.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.142+0000 7f4400b9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43fc0af920 con 0x7f43fc06aaa0 2023-10-06T02:23:25.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.142+0000 7f43faffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43fc13fb20 0x7f43fc0aeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:25.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.142+0000 7f43faffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43fc13fb20 0x7f43fc0aeed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37774/0 (socket says 172.21.15.31:37774) 2023-10-06T02:23:25.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.142+0000 7f43faffd700 1 -- 172.21.15.31:0/1923738718 learned_addr learned my addr 172.21.15.31:0/1923738718 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:25.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.143+0000 7f43fbfff700 1 --2- 172.21.15.31:0/1923738718 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43fc13fef0 0x7f43fc0c75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:25.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.143+0000 7f43fb7fe700 1 --2- 172.21.15.31:0/1923738718 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f43fc06aaa0 0x7f43fc148b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:25.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.143+0000 7f43faffd700 1 -- 172.21.15.31:0/1923738718 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f43fc06aaa0 msgr2=0x7f43fc148b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:25.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.143+0000 7f43faffd700 1 --2- 172.21.15.31:0/1923738718 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f43fc06aaa0 0x7f43fc148b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:25.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.143+0000 7f43faffd700 1 -- 172.21.15.31:0/1923738718 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43fc13fef0 msgr2=0x7f43fc0c75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:25.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.143+0000 7f43faffd700 1 --2- 172.21.15.31:0/1923738718 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43fc13fef0 0x7f43fc0c75f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:25.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.143+0000 7f43faffd700 1 -- 172.21.15.31:0/1923738718 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43ec0094f0 con 0x7f43fc13fb20 2023-10-06T02:23:25.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.143+0000 7f43fbfff700 1 --2- 172.21.15.31:0/1923738718 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43fc13fef0 0x7f43fc0c75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:23:25.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.143+0000 7f43fb7fe700 1 --2- 172.21.15.31:0/1923738718 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f43fc06aaa0 0x7f43fc148b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:23:25.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.143+0000 7f43faffd700 1 --2- 172.21.15.31:0/1923738718 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43fc13fb20 0x7f43fc0aeed0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f43f000c5c0 tx=0x7f43f0007f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:25.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.143+0000 7f43e37fe700 1 -- 172.21.15.31:0/1923738718 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f0011590 con 0x7f43fc13fb20 2023-10-06T02:23:25.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.143+0000 7f43e37fe700 1 -- 172.21.15.31:0/1923738718 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43f00116f0 con 0x7f43fc13fb20 2023-10-06T02:23:25.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.143+0000 7f43e37fe700 1 -- 172.21.15.31:0/1923738718 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f0011ab0 con 0x7f43fc13fb20 2023-10-06T02:23:25.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.143+0000 7f4400b9d700 1 -- 172.21.15.31:0/1923738718 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43fc0afc00 con 0x7f43fc13fb20 2023-10-06T02:23:25.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.143+0000 7f4400b9d700 1 -- 172.21.15.31:0/1923738718 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43fc155c20 con 0x7f43fc13fb20 2023-10-06T02:23:25.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.144+0000 7f4400b9d700 1 -- 172.21.15.31:0/1923738718 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43fc0ab740 con 0x7f43fc13fb20 2023-10-06T02:23:25.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.145+0000 7f43e37fe700 1 -- 172.21.15.31:0/1923738718 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f43f001b3b0 con 0x7f43fc13fb20 2023-10-06T02:23:25.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.146+0000 7f43e37fe700 1 --2- 172.21.15.31:0/1923738718 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f43e8066f30 0x7f43e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:25.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.146+0000 7f43e37fe700 1 -- 172.21.15.31:0/1923738718 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f43f0083bc0 con 0x7f43fc13fb20 2023-10-06T02:23:25.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.147+0000 7f43fb7fe700 1 --2- 172.21.15.31:0/1923738718 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f43e8066f30 0x7f43e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:25.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.147+0000 7f43fb7fe700 1 --2- 172.21.15.31:0/1923738718 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f43e8066f30 0x7f43e80693e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f43ec0099b0 tx=0x7f43ec017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:25.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.151+0000 7f43e37fe700 1 -- 172.21.15.31:0/1923738718 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f43fc0ab740 con 0x7f43fc13fb20 2023-10-06T02:23:25.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.348+0000 7f4400b9d700 1 -- 172.21.15.31:0/1923738718 --> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f43fc0ab740 con 0x7f43e8066f30 2023-10-06T02:23:25.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.357+0000 7f43e37fe700 1 -- 172.21.15.31:0/1923738718 <== mgr.4107 v2:172.21.15.31:6832/85292 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286537 (secure 0 0 0) 0x7f43fc0ab740 con 0x7f43e8066f30 2023-10-06T02:23:25.361 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:23:25.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.361+0000 7f4400b9d700 1 -- 172.21.15.31:0/1923738718 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f43e8066f30 msgr2=0x7f43e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:25.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.361+0000 7f4400b9d700 1 --2- 172.21.15.31:0/1923738718 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f43e8066f30 0x7f43e80693e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f43ec0099b0 tx=0x7f43ec017000 comp rx=0 tx=0).stop 2023-10-06T02:23:25.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.361+0000 7f4400b9d700 1 -- 172.21.15.31:0/1923738718 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43fc13fb20 msgr2=0x7f43fc0aeed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:25.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.361+0000 7f4400b9d700 1 --2- 172.21.15.31:0/1923738718 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43fc13fb20 0x7f43fc0aeed0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f43f000c5c0 tx=0x7f43f0007f10 comp rx=0 tx=0).stop 2023-10-06T02:23:25.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.361+0000 7f4400b9d700 1 -- 172.21.15.31:0/1923738718 shutdown_connections 2023-10-06T02:23:25.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.361+0000 7f4400b9d700 1 --2- 172.21.15.31:0/1923738718 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43fc13fef0 0x7f43fc0c75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:25.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.361+0000 7f4400b9d700 1 --2- 172.21.15.31:0/1923738718 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f43e8066f30 0x7f43e80693e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:25.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.361+0000 7f4400b9d700 1 --2- 172.21.15.31:0/1923738718 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f43fc06aaa0 0x7f43fc148b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:25.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.361+0000 7f4400b9d700 1 --2- 172.21.15.31:0/1923738718 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43fc13fb20 0x7f43fc0aeed0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:25.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.361+0000 7f4400b9d700 1 -- 172.21.15.31:0/1923738718 >> 172.21.15.31:0/1923738718 conn(0x7f43fc06d170 msgr2=0x7f43fc0cc990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:25.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.361+0000 7f4400b9d700 1 -- 172.21.15.31:0/1923738718 shutdown_connections 2023-10-06T02:23:25.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.361+0000 7f4400b9d700 1 -- 172.21.15.31:0/1923738718 wait complete. 2023-10-06T02:23:25.365 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-10-06T02:23:25.378 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":32,"stamp":"2023-10-06T02:23:25.120976+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381416,"kb_used_data":3112,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752593304,"statfs":{"total":773094113280,"available":770655543296,"internally_reserved":0,"allocated":3186688,"data_stored":1899900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002979"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.985694+0000","last_change":"2023-10-06T02:23:09.982458+0000","last_active":"2023-10-06T02:23:09.985694+0000","last_peered":"2023-10-06T02:23:09.985694+0000","last_clean":"2023-10-06T02:23:09.985694+0000","last_became_active":"2023-10-06T02:23:09.982140+0000","last_became_peered":"2023-10-06T02:23:09.982140+0000","last_unstale":"2023-10-06T02:23:09.985694+0000","last_undegraded":"2023-10-06T02:23:09.985694+0000","last_fullsized":"2023-10-06T02:23:09.985694+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:07:57.730382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.999489+0000","last_change":"2023-10-06T02:23:09.996757+0000","last_active":"2023-10-06T02:23:09.999489+0000","last_peered":"2023-10-06T02:23:09.999489+0000","last_clean":"2023-10-06T02:23:09.999489+0000","last_became_active":"2023-10-06T02:23:09.996404+0000","last_became_peered":"2023-10-06T02:23:09.996404+0000","last_unstale":"2023-10-06T02:23:09.999489+0000","last_undegraded":"2023-10-06T02:23:09.999489+0000","last_fullsized":"2023-10-06T02:23:09.999489+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:39:15.275881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.010136+0000","last_change":"2023-10-06T02:23:09.999568+0000","last_active":"2023-10-06T02:23:10.010136+0000","last_peered":"2023-10-06T02:23:10.010136+0000","last_clean":"2023-10-06T02:23:10.010136+0000","last_became_active":"2023-10-06T02:23:09.999033+0000","last_became_peered":"2023-10-06T02:23:09.999033+0000","last_unstale":"2023-10-06T02:23:10.010136+0000","last_undegraded":"2023-10-06T02:23:10.010136+0000","last_fullsized":"2023-10-06T02:23:10.010136+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:23:26.580218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.996019+0000","last_change":"2023-10-06T02:23:09.995088+0000","last_active":"2023-10-06T02:23:09.996019+0000","last_peered":"2023-10-06T02:23:09.996019+0000","last_clean":"2023-10-06T02:23:09.996019+0000","last_became_active":"2023-10-06T02:23:09.994785+0000","last_became_peered":"2023-10-06T02:23:09.994785+0000","last_unstale":"2023-10-06T02:23:09.996019+0000","last_undegraded":"2023-10-06T02:23:09.996019+0000","last_fullsized":"2023-10-06T02:23:09.996019+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:21:33.737708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.994230+0000","last_change":"2023-10-06T02:23:09.986669+0000","last_active":"2023-10-06T02:23:09.994230+0000","last_peered":"2023-10-06T02:23:09.994230+0000","last_clean":"2023-10-06T02:23:09.994230+0000","last_became_active":"2023-10-06T02:23:09.986141+0000","last_became_peered":"2023-10-06T02:23:09.986141+0000","last_unstale":"2023-10-06T02:23:09.994230+0000","last_undegraded":"2023-10-06T02:23:09.994230+0000","last_fullsized":"2023-10-06T02:23:09.994230+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:39:15.275881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.010199+0000","last_change":"2023-10-06T02:23:10.000405+0000","last_active":"2023-10-06T02:23:10.010199+0000","last_peered":"2023-10-06T02:23:10.010199+0000","last_clean":"2023-10-06T02:23:10.010199+0000","last_became_active":"2023-10-06T02:23:10.000015+0000","last_became_peered":"2023-10-06T02:23:10.000015+0000","last_unstale":"2023-10-06T02:23:10.010199+0000","last_undegraded":"2023-10-06T02:23:10.010199+0000","last_fullsized":"2023-10-06T02:23:10.010199+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:39:15.275881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.009997+0000","last_change":"2023-10-06T02:23:10.004359+0000","last_active":"2023-10-06T02:23:10.009997+0000","last_peered":"2023-10-06T02:23:10.009997+0000","last_clean":"2023-10-06T02:23:10.009997+0000","last_became_active":"2023-10-06T02:23:10.003873+0000","last_became_peered":"2023-10-06T02:23:10.003873+0000","last_unstale":"2023-10-06T02:23:10.009997+0000","last_undegraded":"2023-10-06T02:23:10.009997+0000","last_fullsized":"2023-10-06T02:23:10.009997+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:44:41.608418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.998258+0000","last_change":"2023-10-06T02:23:09.994276+0000","last_active":"2023-10-06T02:23:09.998258+0000","last_peered":"2023-10-06T02:23:09.998258+0000","last_clean":"2023-10-06T02:23:09.998258+0000","last_became_active":"2023-10-06T02:23:09.993946+0000","last_became_peered":"2023-10-06T02:23:09.993946+0000","last_unstale":"2023-10-06T02:23:09.998258+0000","last_undegraded":"2023-10-06T02:23:09.998258+0000","last_fullsized":"2023-10-06T02:23:09.998258+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:19:29.462977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.997243+0000","last_change":"2023-10-06T02:23:09.993971+0000","last_active":"2023-10-06T02:23:09.997243+0000","last_peered":"2023-10-06T02:23:09.997243+0000","last_clean":"2023-10-06T02:23:09.997243+0000","last_became_active":"2023-10-06T02:23:09.993547+0000","last_became_peered":"2023-10-06T02:23:09.993547+0000","last_unstale":"2023-10-06T02:23:09.997243+0000","last_undegraded":"2023-10-06T02:23:09.997243+0000","last_fullsized":"2023-10-06T02:23:09.997243+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:19:29.462977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.999469+0000","last_change":"2023-10-06T02:23:09.995426+0000","last_active":"2023-10-06T02:23:09.999469+0000","last_peered":"2023-10-06T02:23:09.999469+0000","last_clean":"2023-10-06T02:23:09.999469+0000","last_became_active":"2023-10-06T02:23:09.995067+0000","last_became_peered":"2023-10-06T02:23:09.995067+0000","last_unstale":"2023-10-06T02:23:09.999469+0000","last_undegraded":"2023-10-06T02:23:09.999469+0000","last_fullsized":"2023-10-06T02:23:09.999469+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:50:19.168882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.010257+0000","last_change":"2023-10-06T02:23:10.001093+0000","last_active":"2023-10-06T02:23:10.010257+0000","last_peered":"2023-10-06T02:23:10.010257+0000","last_clean":"2023-10-06T02:23:10.010257+0000","last_became_active":"2023-10-06T02:23:10.000755+0000","last_became_peered":"2023-10-06T02:23:10.000755+0000","last_unstale":"2023-10-06T02:23:10.010257+0000","last_undegraded":"2023-10-06T02:23:10.010257+0000","last_fullsized":"2023-10-06T02:23:10.010257+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:43:02.156237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.998611+0000","last_change":"2023-10-06T02:23:09.993328+0000","last_active":"2023-10-06T02:23:09.998611+0000","last_peered":"2023-10-06T02:23:09.998611+0000","last_clean":"2023-10-06T02:23:09.998611+0000","last_became_active":"2023-10-06T02:23:09.993076+0000","last_became_peered":"2023-10-06T02:23:09.993076+0000","last_unstale":"2023-10-06T02:23:09.998611+0000","last_undegraded":"2023-10-06T02:23:09.998611+0000","last_fullsized":"2023-10-06T02:23:09.998611+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:58.521611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.986182+0000","last_change":"2023-10-06T02:23:09.982673+0000","last_active":"2023-10-06T02:23:09.986182+0000","last_peered":"2023-10-06T02:23:09.986182+0000","last_clean":"2023-10-06T02:23:09.986182+0000","last_became_active":"2023-10-06T02:23:09.982370+0000","last_became_peered":"2023-10-06T02:23:09.982370+0000","last_unstale":"2023-10-06T02:23:09.986182+0000","last_undegraded":"2023-10-06T02:23:09.986182+0000","last_fullsized":"2023-10-06T02:23:09.986182+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:17:33.441972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.985297+0000","last_change":"2023-10-06T02:23:09.982769+0000","last_active":"2023-10-06T02:23:09.985297+0000","last_peered":"2023-10-06T02:23:09.985297+0000","last_clean":"2023-10-06T02:23:09.985297+0000","last_became_active":"2023-10-06T02:23:09.982450+0000","last_became_peered":"2023-10-06T02:23:09.982450+0000","last_unstale":"2023-10-06T02:23:09.985297+0000","last_undegraded":"2023-10-06T02:23:09.985297+0000","last_fullsized":"2023-10-06T02:23:09.985297+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:12:16.306131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.010017+0000","last_change":"2023-10-06T02:23:10.004452+0000","last_active":"2023-10-06T02:23:10.010017+0000","last_peered":"2023-10-06T02:23:10.010017+0000","last_clean":"2023-10-06T02:23:10.010017+0000","last_became_active":"2023-10-06T02:23:10.004184+0000","last_became_peered":"2023-10-06T02:23:10.004184+0000","last_unstale":"2023-10-06T02:23:10.010017+0000","last_undegraded":"2023-10-06T02:23:10.010017+0000","last_fullsized":"2023-10-06T02:23:10.010017+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:50:19.168882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.010069+0000","last_change":"2023-10-06T02:23:09.998790+0000","last_active":"2023-10-06T02:23:10.010069+0000","last_peered":"2023-10-06T02:23:10.010069+0000","last_clean":"2023-10-06T02:23:10.010069+0000","last_became_active":"2023-10-06T02:23:09.998476+0000","last_became_peered":"2023-10-06T02:23:09.998476+0000","last_unstale":"2023-10-06T02:23:10.010069+0000","last_undegraded":"2023-10-06T02:23:10.010069+0000","last_fullsized":"2023-10-06T02:23:10.010069+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:19:29.462977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.997263+0000","last_change":"2023-10-06T02:23:09.994379+0000","last_active":"2023-10-06T02:23:09.997263+0000","last_peered":"2023-10-06T02:23:09.997263+0000","last_clean":"2023-10-06T02:23:09.997263+0000","last_became_active":"2023-10-06T02:23:09.994031+0000","last_became_peered":"2023-10-06T02:23:09.994031+0000","last_unstale":"2023-10-06T02:23:09.997263+0000","last_undegraded":"2023-10-06T02:23:09.997263+0000","last_fullsized":"2023-10-06T02:23:09.997263+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:12.998563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.994477+0000","last_change":"2023-10-06T02:23:09.988087+0000","last_active":"2023-10-06T02:23:09.994477+0000","last_peered":"2023-10-06T02:23:09.994477+0000","last_clean":"2023-10-06T02:23:09.994477+0000","last_became_active":"2023-10-06T02:23:09.987741+0000","last_became_peered":"2023-10-06T02:23:09.987741+0000","last_unstale":"2023-10-06T02:23:09.994477+0000","last_undegraded":"2023-10-06T02:23:09.994477+0000","last_fullsized":"2023-10-06T02:23:09.994477+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:05:24.047504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.987646+0000","last_change":"2023-10-06T02:23:09.983568+0000","last_active":"2023-10-06T02:23:09.987646+0000","last_peered":"2023-10-06T02:23:09.987646+0000","last_clean":"2023-10-06T02:23:09.987646+0000","last_became_active":"2023-10-06T02:23:09.983273+0000","last_became_peered":"2023-10-06T02:23:09.983273+0000","last_unstale":"2023-10-06T02:23:09.987646+0000","last_undegraded":"2023-10-06T02:23:09.987646+0000","last_fullsized":"2023-10-06T02:23:09.987646+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:44:16.553231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.009997+0000","last_change":"2023-10-06T02:23:09.997248+0000","last_active":"2023-10-06T02:23:10.009997+0000","last_peered":"2023-10-06T02:23:10.009997+0000","last_clean":"2023-10-06T02:23:10.009997+0000","last_became_active":"2023-10-06T02:23:09.996897+0000","last_became_peered":"2023-10-06T02:23:09.996897+0000","last_unstale":"2023-10-06T02:23:10.009997+0000","last_undegraded":"2023-10-06T02:23:10.009997+0000","last_fullsized":"2023-10-06T02:23:10.009997+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:00:45.442861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.994041+0000","last_change":"2023-10-06T02:23:09.991324+0000","last_active":"2023-10-06T02:23:09.994041+0000","last_peered":"2023-10-06T02:23:09.994041+0000","last_clean":"2023-10-06T02:23:09.994041+0000","last_became_active":"2023-10-06T02:23:09.991018+0000","last_became_peered":"2023-10-06T02:23:09.991018+0000","last_unstale":"2023-10-06T02:23:09.994041+0000","last_undegraded":"2023-10-06T02:23:09.994041+0000","last_fullsized":"2023-10-06T02:23:09.994041+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:43:01.259350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.004545+0000","last_change":"2023-10-06T02:23:10.000917+0000","last_active":"2023-10-06T02:23:10.004545+0000","last_peered":"2023-10-06T02:23:10.004545+0000","last_clean":"2023-10-06T02:23:10.004545+0000","last_became_active":"2023-10-06T02:23:10.000580+0000","last_became_peered":"2023-10-06T02:23:10.000580+0000","last_unstale":"2023-10-06T02:23:10.004545+0000","last_undegraded":"2023-10-06T02:23:10.004545+0000","last_fullsized":"2023-10-06T02:23:10.004545+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:50:19.168882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.998825+0000","last_change":"2023-10-06T02:23:09.995748+0000","last_active":"2023-10-06T02:23:09.998825+0000","last_peered":"2023-10-06T02:23:09.998825+0000","last_clean":"2023-10-06T02:23:09.998825+0000","last_became_active":"2023-10-06T02:23:09.995433+0000","last_became_peered":"2023-10-06T02:23:09.995433+0000","last_unstale":"2023-10-06T02:23:09.998825+0000","last_undegraded":"2023-10-06T02:23:09.998825+0000","last_fullsized":"2023-10-06T02:23:09.998825+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:21.438589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.996641+0000","last_change":"2023-10-06T02:23:09.992834+0000","last_active":"2023-10-06T02:23:09.996641+0000","last_peered":"2023-10-06T02:23:09.996641+0000","last_clean":"2023-10-06T02:23:09.996641+0000","last_became_active":"2023-10-06T02:23:09.991971+0000","last_became_peered":"2023-10-06T02:23:09.991971+0000","last_unstale":"2023-10-06T02:23:09.996641+0000","last_undegraded":"2023-10-06T02:23:09.996641+0000","last_fullsized":"2023-10-06T02:23:09.996641+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:12.998563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.007651+0000","last_change":"2023-10-06T02:23:10.005469+0000","last_active":"2023-10-06T02:23:10.007651+0000","last_peered":"2023-10-06T02:23:10.007651+0000","last_clean":"2023-10-06T02:23:10.007651+0000","last_became_active":"2023-10-06T02:23:10.005132+0000","last_became_peered":"2023-10-06T02:23:10.005132+0000","last_unstale":"2023-10-06T02:23:10.007651+0000","last_undegraded":"2023-10-06T02:23:10.007651+0000","last_fullsized":"2023-10-06T02:23:10.007651+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:19:18.748586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.005833+0000","last_change":"2023-10-06T02:23:09.999977+0000","last_active":"2023-10-06T02:23:10.005833+0000","last_peered":"2023-10-06T02:23:10.005833+0000","last_clean":"2023-10-06T02:23:10.005833+0000","last_became_active":"2023-10-06T02:23:09.999733+0000","last_became_peered":"2023-10-06T02:23:09.999733+0000","last_unstale":"2023-10-06T02:23:10.005833+0000","last_undegraded":"2023-10-06T02:23:10.005833+0000","last_fullsized":"2023-10-06T02:23:10.005833+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:58:41.925466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.999279+0000","last_change":"2023-10-06T02:23:09.994580+0000","last_active":"2023-10-06T02:23:09.999279+0000","last_peered":"2023-10-06T02:23:09.999279+0000","last_clean":"2023-10-06T02:23:09.999279+0000","last_became_active":"2023-10-06T02:23:09.993941+0000","last_became_peered":"2023-10-06T02:23:09.993941+0000","last_unstale":"2023-10-06T02:23:09.999279+0000","last_undegraded":"2023-10-06T02:23:09.999279+0000","last_fullsized":"2023-10-06T02:23:09.999279+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:43:01.259350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.043858+0000","last_change":"2023-10-06T02:23:09.048996+0000","last_active":"2023-10-06T02:23:10.043858+0000","last_peered":"2023-10-06T02:23:10.043858+0000","last_clean":"2023-10-06T02:23:10.043858+0000","last_became_active":"2023-10-06T02:23:09.048662+0000","last_became_peered":"2023-10-06T02:23:09.048662+0000","last_unstale":"2023-10-06T02:23:10.043858+0000","last_undegraded":"2023-10-06T02:23:10.043858+0000","last_fullsized":"2023-10-06T02:23:10.043858+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:58.976414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.004959+0000","last_change":"2023-10-06T02:23:10.001176+0000","last_active":"2023-10-06T02:23:10.004959+0000","last_peered":"2023-10-06T02:23:10.004959+0000","last_clean":"2023-10-06T02:23:10.004959+0000","last_became_active":"2023-10-06T02:23:10.000887+0000","last_became_peered":"2023-10-06T02:23:10.000887+0000","last_unstale":"2023-10-06T02:23:10.004959+0000","last_undegraded":"2023-10-06T02:23:10.004959+0000","last_fullsized":"2023-10-06T02:23:10.004959+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:43:02.156237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.998513+0000","last_change":"2023-10-06T02:23:09.048083+0000","last_active":"2023-10-06T02:23:09.998513+0000","last_peered":"2023-10-06T02:23:09.998513+0000","last_clean":"2023-10-06T02:23:09.998513+0000","last_became_active":"2023-10-06T02:23:09.047701+0000","last_became_peered":"2023-10-06T02:23:09.047701+0000","last_unstale":"2023-10-06T02:23:09.998513+0000","last_undegraded":"2023-10-06T02:23:09.998513+0000","last_fullsized":"2023-10-06T02:23:09.998513+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:17:32.434412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.994267+0000","last_change":"2023-10-06T02:23:09.988810+0000","last_active":"2023-10-06T02:23:09.994267+0000","last_peered":"2023-10-06T02:23:09.994267+0000","last_clean":"2023-10-06T02:23:09.994267+0000","last_became_active":"2023-10-06T02:23:09.988419+0000","last_became_peered":"2023-10-06T02:23:09.988419+0000","last_unstale":"2023-10-06T02:23:09.994267+0000","last_undegraded":"2023-10-06T02:23:09.994267+0000","last_fullsized":"2023-10-06T02:23:09.994267+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:43:01.259350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.976287+0000","last_change":"2023-10-06T02:23:09.044642+0000","last_active":"2023-10-06T02:23:09.976287+0000","last_peered":"2023-10-06T02:23:09.976287+0000","last_clean":"2023-10-06T02:23:09.976287+0000","last_became_active":"2023-10-06T02:23:09.044224+0000","last_became_peered":"2023-10-06T02:23:09.044224+0000","last_unstale":"2023-10-06T02:23:09.976287+0000","last_undegraded":"2023-10-06T02:23:09.976287+0000","last_fullsized":"2023-10-06T02:23:09.976287+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:31:52.666890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.986940+0000","last_change":"2023-10-06T02:23:09.985837+0000","last_active":"2023-10-06T02:23:09.986940+0000","last_peered":"2023-10-06T02:23:09.986940+0000","last_clean":"2023-10-06T02:23:09.986940+0000","last_became_active":"2023-10-06T02:23:09.985447+0000","last_became_peered":"2023-10-06T02:23:09.985447+0000","last_unstale":"2023-10-06T02:23:09.986940+0000","last_undegraded":"2023-10-06T02:23:09.986940+0000","last_fullsized":"2023-10-06T02:23:09.986940+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:31:53.674450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.979513+0000","last_change":"2023-10-06T02:23:09.040730+0000","last_active":"2023-10-06T02:23:09.979513+0000","last_peered":"2023-10-06T02:23:09.979513+0000","last_clean":"2023-10-06T02:23:09.979513+0000","last_became_active":"2023-10-06T02:23:09.040327+0000","last_became_peered":"2023-10-06T02:23:09.040327+0000","last_unstale":"2023-10-06T02:23:09.979513+0000","last_undegraded":"2023-10-06T02:23:09.979513+0000","last_fullsized":"2023-10-06T02:23:09.979513+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:07:56.722823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.998851+0000","last_change":"2023-10-06T02:23:09.995950+0000","last_active":"2023-10-06T02:23:09.998851+0000","last_peered":"2023-10-06T02:23:09.998851+0000","last_clean":"2023-10-06T02:23:09.998851+0000","last_became_active":"2023-10-06T02:23:09.995679+0000","last_became_peered":"2023-10-06T02:23:09.995679+0000","last_unstale":"2023-10-06T02:23:09.998851+0000","last_undegraded":"2023-10-06T02:23:09.998851+0000","last_fullsized":"2023-10-06T02:23:09.998851+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:23:26.580218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.976082+0000","last_change":"2023-10-06T02:23:09.040245+0000","last_active":"2023-10-06T02:23:09.976082+0000","last_peered":"2023-10-06T02:23:09.976082+0000","last_clean":"2023-10-06T02:23:09.976082+0000","last_became_active":"2023-10-06T02:23:09.039660+0000","last_became_peered":"2023-10-06T02:23:09.039660+0000","last_unstale":"2023-10-06T02:23:09.976082+0000","last_undegraded":"2023-10-06T02:23:09.976082+0000","last_fullsized":"2023-10-06T02:23:09.976082+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:57.514051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.994925+0000","last_change":"2023-10-06T02:23:09.992417+0000","last_active":"2023-10-06T02:23:09.994925+0000","last_peered":"2023-10-06T02:23:09.994925+0000","last_clean":"2023-10-06T02:23:09.994925+0000","last_became_active":"2023-10-06T02:23:09.992155+0000","last_became_peered":"2023-10-06T02:23:09.992155+0000","last_unstale":"2023-10-06T02:23:09.994925+0000","last_undegraded":"2023-10-06T02:23:09.994925+0000","last_fullsized":"2023-10-06T02:23:09.994925+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:39:15.275881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"22'3","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.044297+0000","last_change":"2023-10-06T02:23:09.046655+0000","last_active":"2023-10-06T02:23:10.044297+0000","last_peered":"2023-10-06T02:23:10.044297+0000","last_clean":"2023-10-06T02:23:10.044297+0000","last_became_active":"2023-10-06T02:23:09.046335+0000","last_became_peered":"2023-10-06T02:23:09.046335+0000","last_unstale":"2023-10-06T02:23:10.044297+0000","last_undegraded":"2023-10-06T02:23:10.044297+0000","last_fullsized":"2023-10-06T02:23:10.044297+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:53:00.225521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.007635+0000","last_change":"2023-10-06T02:23:10.005359+0000","last_active":"2023-10-06T02:23:10.007635+0000","last_peered":"2023-10-06T02:23:10.007635+0000","last_clean":"2023-10-06T02:23:10.007635+0000","last_became_active":"2023-10-06T02:23:10.005115+0000","last_became_peered":"2023-10-06T02:23:10.005115+0000","last_unstale":"2023-10-06T02:23:10.007635+0000","last_undegraded":"2023-10-06T02:23:10.007635+0000","last_fullsized":"2023-10-06T02:23:10.007635+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:29:00.771363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"22'6","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.052236+0000","last_change":"2023-10-06T02:23:09.046958+0000","last_active":"2023-10-06T02:23:10.052236+0000","last_peered":"2023-10-06T02:23:10.052236+0000","last_clean":"2023-10-06T02:23:10.052236+0000","last_became_active":"2023-10-06T02:23:09.044018+0000","last_became_peered":"2023-10-06T02:23:09.044018+0000","last_unstale":"2023-10-06T02:23:10.052236+0000","last_undegraded":"2023-10-06T02:23:10.052236+0000","last_fullsized":"2023-10-06T02:23:10.052236+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:41.039005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.992467+0000","last_change":"2023-10-06T02:23:09.987559+0000","last_active":"2023-10-06T02:23:09.992467+0000","last_peered":"2023-10-06T02:23:09.992467+0000","last_clean":"2023-10-06T02:23:09.992467+0000","last_became_active":"2023-10-06T02:23:09.987264+0000","last_became_peered":"2023-10-06T02:23:09.987264+0000","last_unstale":"2023-10-06T02:23:09.992467+0000","last_undegraded":"2023-10-06T02:23:09.992467+0000","last_fullsized":"2023-10-06T02:23:09.992467+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:44:41.608418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.049676+0000","last_change":"2023-10-06T02:23:09.048307+0000","last_active":"2023-10-06T02:23:10.049676+0000","last_peered":"2023-10-06T02:23:10.049676+0000","last_clean":"2023-10-06T02:23:10.049676+0000","last_became_active":"2023-10-06T02:23:09.047919+0000","last_became_peered":"2023-10-06T02:23:09.047919+0000","last_unstale":"2023-10-06T02:23:10.049676+0000","last_undegraded":"2023-10-06T02:23:10.049676+0000","last_fullsized":"2023-10-06T02:23:10.049676+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:28:59.763803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.004797+0000","last_change":"2023-10-06T02:23:10.001689+0000","last_active":"2023-10-06T02:23:10.004797+0000","last_peered":"2023-10-06T02:23:10.004797+0000","last_clean":"2023-10-06T02:23:10.004797+0000","last_became_active":"2023-10-06T02:23:10.001418+0000","last_became_peered":"2023-10-06T02:23:10.001418+0000","last_unstale":"2023-10-06T02:23:10.004797+0000","last_undegraded":"2023-10-06T02:23:10.004797+0000","last_fullsized":"2023-10-06T02:23:10.004797+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:53:54.469523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.977377+0000","last_change":"2023-10-06T02:23:09.041968+0000","last_active":"2023-10-06T02:23:09.977377+0000","last_peered":"2023-10-06T02:23:09.977377+0000","last_clean":"2023-10-06T02:23:09.977377+0000","last_became_active":"2023-10-06T02:23:09.041659+0000","last_became_peered":"2023-10-06T02:23:09.041659+0000","last_unstale":"2023-10-06T02:23:09.977377+0000","last_undegraded":"2023-10-06T02:23:09.977377+0000","last_fullsized":"2023-10-06T02:23:09.977377+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:44:15.545672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.992112+0000","last_change":"2023-10-06T02:23:09.989200+0000","last_active":"2023-10-06T02:23:09.992112+0000","last_peered":"2023-10-06T02:23:09.992112+0000","last_clean":"2023-10-06T02:23:09.992112+0000","last_became_active":"2023-10-06T02:23:09.988895+0000","last_became_peered":"2023-10-06T02:23:09.988895+0000","last_unstale":"2023-10-06T02:23:09.992112+0000","last_undegraded":"2023-10-06T02:23:09.992112+0000","last_fullsized":"2023-10-06T02:23:09.992112+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:44:41.608418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.984229+0000","last_change":"2023-10-06T02:23:09.048119+0000","last_active":"2023-10-06T02:23:09.984229+0000","last_peered":"2023-10-06T02:23:09.984229+0000","last_clean":"2023-10-06T02:23:09.984229+0000","last_became_active":"2023-10-06T02:23:09.047678+0000","last_became_peered":"2023-10-06T02:23:09.047678+0000","last_unstale":"2023-10-06T02:23:09.984229+0000","last_undegraded":"2023-10-06T02:23:09.984229+0000","last_fullsized":"2023-10-06T02:23:09.984229+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:42:11.693138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.997921+0000","last_change":"2023-10-06T02:23:09.994722+0000","last_active":"2023-10-06T02:23:09.997921+0000","last_peered":"2023-10-06T02:23:09.997921+0000","last_clean":"2023-10-06T02:23:09.997921+0000","last_became_active":"2023-10-06T02:23:09.994392+0000","last_became_peered":"2023-10-06T02:23:09.994392+0000","last_unstale":"2023-10-06T02:23:09.997921+0000","last_undegraded":"2023-10-06T02:23:09.997921+0000","last_fullsized":"2023-10-06T02:23:09.997921+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:21.438589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"22'8","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.049205+0000","last_change":"2023-10-06T02:23:09.034244+0000","last_active":"2023-10-06T02:23:10.049205+0000","last_peered":"2023-10-06T02:23:10.049205+0000","last_clean":"2023-10-06T02:23:10.049205+0000","last_became_active":"2023-10-06T02:23:09.033861+0000","last_became_peered":"2023-10-06T02:23:09.033861+0000","last_unstale":"2023-10-06T02:23:10.049205+0000","last_undegraded":"2023-10-06T02:23:10.049205+0000","last_fullsized":"2023-10-06T02:23:10.049205+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:41.039005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.990663+0000","last_change":"2023-10-06T02:23:09.988390+0000","last_active":"2023-10-06T02:23:09.990663+0000","last_peered":"2023-10-06T02:23:09.990663+0000","last_clean":"2023-10-06T02:23:09.990663+0000","last_became_active":"2023-10-06T02:23:09.988060+0000","last_became_peered":"2023-10-06T02:23:09.988060+0000","last_unstale":"2023-10-06T02:23:09.990663+0000","last_undegraded":"2023-10-06T02:23:09.990663+0000","last_fullsized":"2023-10-06T02:23:09.990663+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:43:01.259350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.044194+0000","last_change":"2023-10-06T02:23:09.048150+0000","last_active":"2023-10-06T02:23:10.044194+0000","last_peered":"2023-10-06T02:23:10.044194+0000","last_clean":"2023-10-06T02:23:10.044194+0000","last_became_active":"2023-10-06T02:23:09.047768+0000","last_became_peered":"2023-10-06T02:23:09.047768+0000","last_unstale":"2023-10-06T02:23:10.044194+0000","last_undegraded":"2023-10-06T02:23:10.044194+0000","last_fullsized":"2023-10-06T02:23:10.044194+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:03:50.725863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.992100+0000","last_change":"2023-10-06T02:23:09.990032+0000","last_active":"2023-10-06T02:23:09.992100+0000","last_peered":"2023-10-06T02:23:09.992100+0000","last_clean":"2023-10-06T02:23:09.992100+0000","last_became_active":"2023-10-06T02:23:09.988801+0000","last_became_peered":"2023-10-06T02:23:09.988801+0000","last_unstale":"2023-10-06T02:23:09.992100+0000","last_undegraded":"2023-10-06T02:23:09.992100+0000","last_fullsized":"2023-10-06T02:23:09.992100+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:12.998563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.044129+0000","last_change":"2023-10-06T02:23:09.048438+0000","last_active":"2023-10-06T02:23:10.044129+0000","last_peered":"2023-10-06T02:23:10.044129+0000","last_clean":"2023-10-06T02:23:10.044129+0000","last_became_active":"2023-10-06T02:23:09.048051+0000","last_became_peered":"2023-10-06T02:23:09.048051+0000","last_unstale":"2023-10-06T02:23:10.044129+0000","last_undegraded":"2023-10-06T02:23:10.044129+0000","last_fullsized":"2023-10-06T02:23:10.044129+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:21:32.730148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.010287+0000","last_change":"2023-10-06T02:23:10.002031+0000","last_active":"2023-10-06T02:23:10.010287+0000","last_peered":"2023-10-06T02:23:10.010287+0000","last_clean":"2023-10-06T02:23:10.010287+0000","last_became_active":"2023-10-06T02:23:10.001754+0000","last_became_peered":"2023-10-06T02:23:10.001754+0000","last_unstale":"2023-10-06T02:23:10.010287+0000","last_undegraded":"2023-10-06T02:23:10.010287+0000","last_fullsized":"2023-10-06T02:23:10.010287+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:45:34.162627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.044320+0000","last_change":"2023-10-06T02:23:09.038304+0000","last_active":"2023-10-06T02:23:10.044320+0000","last_peered":"2023-10-06T02:23:10.044320+0000","last_clean":"2023-10-06T02:23:10.044320+0000","last_became_active":"2023-10-06T02:23:09.038031+0000","last_became_peered":"2023-10-06T02:23:09.038031+0000","last_unstale":"2023-10-06T02:23:10.044320+0000","last_undegraded":"2023-10-06T02:23:10.044320+0000","last_fullsized":"2023-10-06T02:23:10.044320+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:31:52.666890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.004414+0000","last_change":"2023-10-06T02:23:10.000692+0000","last_active":"2023-10-06T02:23:10.004414+0000","last_peered":"2023-10-06T02:23:10.004414+0000","last_clean":"2023-10-06T02:23:10.004414+0000","last_became_active":"2023-10-06T02:23:10.000368+0000","last_became_peered":"2023-10-06T02:23:10.000368+0000","last_unstale":"2023-10-06T02:23:10.004414+0000","last_undegraded":"2023-10-06T02:23:10.004414+0000","last_fullsized":"2023-10-06T02:23:10.004414+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:53:01.233080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.974959+0000","last_change":"2023-10-06T02:23:09.037238+0000","last_active":"2023-10-06T02:23:09.974959+0000","last_peered":"2023-10-06T02:23:09.974959+0000","last_clean":"2023-10-06T02:23:09.974959+0000","last_became_active":"2023-10-06T02:23:09.036871+0000","last_became_peered":"2023-10-06T02:23:09.036871+0000","last_unstale":"2023-10-06T02:23:09.974959+0000","last_undegraded":"2023-10-06T02:23:09.974959+0000","last_fullsized":"2023-10-06T02:23:09.974959+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:44:15.545672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.002856+0000","last_change":"2023-10-06T02:23:10.000807+0000","last_active":"2023-10-06T02:23:10.002856+0000","last_peered":"2023-10-06T02:23:10.002856+0000","last_clean":"2023-10-06T02:23:10.002856+0000","last_became_active":"2023-10-06T02:23:10.000502+0000","last_became_peered":"2023-10-06T02:23:10.000502+0000","last_unstale":"2023-10-06T02:23:10.002856+0000","last_undegraded":"2023-10-06T02:23:10.002856+0000","last_fullsized":"2023-10-06T02:23:10.002856+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:31:27.242353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.042047+0000","last_change":"2023-10-06T02:23:09.044437+0000","last_active":"2023-10-06T02:23:10.042047+0000","last_peered":"2023-10-06T02:23:10.042047+0000","last_clean":"2023-10-06T02:23:10.042047+0000","last_became_active":"2023-10-06T02:23:09.042657+0000","last_became_peered":"2023-10-06T02:23:09.042657+0000","last_unstale":"2023-10-06T02:23:10.042047+0000","last_undegraded":"2023-10-06T02:23:10.042047+0000","last_fullsized":"2023-10-06T02:23:10.042047+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:57:40.559177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.994640+0000","last_change":"2023-10-06T02:23:09.990834+0000","last_active":"2023-10-06T02:23:09.994640+0000","last_peered":"2023-10-06T02:23:09.994640+0000","last_clean":"2023-10-06T02:23:09.994640+0000","last_became_active":"2023-10-06T02:23:09.990192+0000","last_became_peered":"2023-10-06T02:23:09.990192+0000","last_unstale":"2023-10-06T02:23:09.994640+0000","last_undegraded":"2023-10-06T02:23:09.994640+0000","last_fullsized":"2023-10-06T02:23:09.994640+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:18:32.416904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.980739+0000","last_change":"2023-10-06T02:23:09.044880+0000","last_active":"2023-10-06T02:23:09.980739+0000","last_peered":"2023-10-06T02:23:09.980739+0000","last_clean":"2023-10-06T02:23:09.980739+0000","last_became_active":"2023-10-06T02:23:09.044556+0000","last_became_peered":"2023-10-06T02:23:09.044556+0000","last_unstale":"2023-10-06T02:23:09.980739+0000","last_undegraded":"2023-10-06T02:23:09.980739+0000","last_fullsized":"2023-10-06T02:23:09.980739+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:58.976414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.994703+0000","last_change":"2023-10-06T02:23:09.992094+0000","last_active":"2023-10-06T02:23:09.994703+0000","last_peered":"2023-10-06T02:23:09.994703+0000","last_clean":"2023-10-06T02:23:09.994703+0000","last_became_active":"2023-10-06T02:23:09.991699+0000","last_became_peered":"2023-10-06T02:23:09.991699+0000","last_unstale":"2023-10-06T02:23:09.994703+0000","last_undegraded":"2023-10-06T02:23:09.994703+0000","last_fullsized":"2023-10-06T02:23:09.994703+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:21.438589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.986400+0000","last_change":"2023-10-06T02:23:09.041591+0000","last_active":"2023-10-06T02:23:09.986400+0000","last_peered":"2023-10-06T02:23:09.986400+0000","last_clean":"2023-10-06T02:23:09.986400+0000","last_became_active":"2023-10-06T02:23:09.041239+0000","last_became_peered":"2023-10-06T02:23:09.041239+0000","last_unstale":"2023-10-06T02:23:09.986400+0000","last_undegraded":"2023-10-06T02:23:09.986400+0000","last_fullsized":"2023-10-06T02:23:09.986400+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:17:32.434412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.994672+0000","last_change":"2023-10-06T02:23:09.991765+0000","last_active":"2023-10-06T02:23:09.994672+0000","last_peered":"2023-10-06T02:23:09.994672+0000","last_clean":"2023-10-06T02:23:09.994672+0000","last_became_active":"2023-10-06T02:23:09.990991+0000","last_became_peered":"2023-10-06T02:23:09.990991+0000","last_unstale":"2023-10-06T02:23:09.994672+0000","last_undegraded":"2023-10-06T02:23:09.994672+0000","last_fullsized":"2023-10-06T02:23:09.994672+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:18:23.992787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.976060+0000","last_change":"2023-10-06T02:23:09.046400+0000","last_active":"2023-10-06T02:23:09.976060+0000","last_peered":"2023-10-06T02:23:09.976060+0000","last_clean":"2023-10-06T02:23:09.976060+0000","last_became_active":"2023-10-06T02:23:09.046039+0000","last_became_peered":"2023-10-06T02:23:09.046039+0000","last_unstale":"2023-10-06T02:23:09.976060+0000","last_undegraded":"2023-10-06T02:23:09.976060+0000","last_fullsized":"2023-10-06T02:23:09.976060+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:44:15.545672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.005866+0000","last_change":"2023-10-06T02:23:10.001037+0000","last_active":"2023-10-06T02:23:10.005866+0000","last_peered":"2023-10-06T02:23:10.005866+0000","last_clean":"2023-10-06T02:23:10.005866+0000","last_became_active":"2023-10-06T02:23:10.000479+0000","last_became_peered":"2023-10-06T02:23:10.000479+0000","last_unstale":"2023-10-06T02:23:10.005866+0000","last_undegraded":"2023-10-06T02:23:10.005866+0000","last_fullsized":"2023-10-06T02:23:10.005866+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:50:19.168882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.996789+0000","last_change":"2023-10-06T02:23:02.054039+0000","last_active":"2023-10-06T02:23:09.996789+0000","last_peered":"2023-10-06T02:23:09.996789+0000","last_clean":"2023-10-06T02:23:09.996789+0000","last_became_active":"2023-10-06T02:23:02.053742+0000","last_became_peered":"2023-10-06T02:23:02.053742+0000","last_unstale":"2023-10-06T02:23:09.996789+0000","last_undegraded":"2023-10-06T02:23:09.996789+0000","last_fullsized":"2023-10-06T02:23:09.996789+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:58:18.846475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.975329+0000","last_change":"2023-10-06T02:23:09.037244+0000","last_active":"2023-10-06T02:23:09.975329+0000","last_peered":"2023-10-06T02:23:09.975329+0000","last_clean":"2023-10-06T02:23:09.975329+0000","last_became_active":"2023-10-06T02:23:09.036880+0000","last_became_peered":"2023-10-06T02:23:09.036880+0000","last_unstale":"2023-10-06T02:23:09.975329+0000","last_undegraded":"2023-10-06T02:23:09.975329+0000","last_fullsized":"2023-10-06T02:23:09.975329+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:36:27.554258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.005453+0000","last_change":"2023-10-06T02:23:09.999775+0000","last_active":"2023-10-06T02:23:10.005453+0000","last_peered":"2023-10-06T02:23:10.005453+0000","last_clean":"2023-10-06T02:23:10.005453+0000","last_became_active":"2023-10-06T02:23:09.999472+0000","last_became_peered":"2023-10-06T02:23:09.999472+0000","last_unstale":"2023-10-06T02:23:10.005453+0000","last_undegraded":"2023-10-06T02:23:10.005453+0000","last_fullsized":"2023-10-06T02:23:10.005453+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:44:41.608418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.977354+0000","last_change":"2023-10-06T02:23:01.988631+0000","last_active":"2023-10-06T02:23:09.977354+0000","last_peered":"2023-10-06T02:23:09.977354+0000","last_clean":"2023-10-06T02:23:09.977354+0000","last_became_active":"2023-10-06T02:23:01.988303+0000","last_became_peered":"2023-10-06T02:23:01.988303+0000","last_unstale":"2023-10-06T02:23:09.977354+0000","last_undegraded":"2023-10-06T02:23:09.977354+0000","last_fullsized":"2023-10-06T02:23:09.977354+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:38.211466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.986211+0000","last_change":"2023-10-06T02:23:09.041835+0000","last_active":"2023-10-06T02:23:09.986211+0000","last_peered":"2023-10-06T02:23:09.986211+0000","last_clean":"2023-10-06T02:23:09.986211+0000","last_became_active":"2023-10-06T02:23:09.041521+0000","last_became_peered":"2023-10-06T02:23:09.041521+0000","last_unstale":"2023-10-06T02:23:09.986211+0000","last_undegraded":"2023-10-06T02:23:09.986211+0000","last_fullsized":"2023-10-06T02:23:09.986211+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:38.263723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.003455+0000","last_change":"2023-10-06T02:23:09.995055+0000","last_active":"2023-10-06T02:23:10.003455+0000","last_peered":"2023-10-06T02:23:10.003455+0000","last_clean":"2023-10-06T02:23:10.003455+0000","last_became_active":"2023-10-06T02:23:09.994216+0000","last_became_peered":"2023-10-06T02:23:09.994216+0000","last_unstale":"2023-10-06T02:23:10.003455+0000","last_undegraded":"2023-10-06T02:23:10.003455+0000","last_fullsized":"2023-10-06T02:23:10.003455+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:47:34.184576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.972958+0000","last_change":"2023-10-06T02:23:02.054338+0000","last_active":"2023-10-06T02:23:09.972958+0000","last_peered":"2023-10-06T02:23:09.972958+0000","last_clean":"2023-10-06T02:23:09.972958+0000","last_became_active":"2023-10-06T02:23:02.053991+0000","last_became_peered":"2023-10-06T02:23:02.053991+0000","last_unstale":"2023-10-06T02:23:09.972958+0000","last_undegraded":"2023-10-06T02:23:09.972958+0000","last_fullsized":"2023-10-06T02:23:09.972958+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:39:30.192838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.996619+0000","last_change":"2023-10-06T02:23:09.039360+0000","last_active":"2023-10-06T02:23:09.996619+0000","last_peered":"2023-10-06T02:23:09.996619+0000","last_clean":"2023-10-06T02:23:09.996619+0000","last_became_active":"2023-10-06T02:23:09.038571+0000","last_became_peered":"2023-10-06T02:23:09.038571+0000","last_unstale":"2023-10-06T02:23:09.996619+0000","last_undegraded":"2023-10-06T02:23:09.996619+0000","last_fullsized":"2023-10-06T02:23:09.996619+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:58.976414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.005846+0000","last_change":"2023-10-06T02:23:10.000821+0000","last_active":"2023-10-06T02:23:10.005846+0000","last_peered":"2023-10-06T02:23:10.005846+0000","last_clean":"2023-10-06T02:23:10.005846+0000","last_became_active":"2023-10-06T02:23:10.000203+0000","last_became_peered":"2023-10-06T02:23:10.000203+0000","last_unstale":"2023-10-06T02:23:10.005846+0000","last_undegraded":"2023-10-06T02:23:10.005846+0000","last_fullsized":"2023-10-06T02:23:10.005846+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:43:01.259350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.975054+0000","last_change":"2023-10-06T02:23:01.988140+0000","last_active":"2023-10-06T02:23:09.975054+0000","last_peered":"2023-10-06T02:23:09.975054+0000","last_clean":"2023-10-06T02:23:09.975054+0000","last_became_active":"2023-10-06T02:23:01.987769+0000","last_became_peered":"2023-10-06T02:23:01.987769+0000","last_unstale":"2023-10-06T02:23:09.975054+0000","last_undegraded":"2023-10-06T02:23:09.975054+0000","last_fullsized":"2023-10-06T02:23:09.975054+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:20:08.391912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":166,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.983731+0000","last_change":"2023-10-06T02:22:55.032519+0000","last_active":"2023-10-06T02:23:09.983731+0000","last_peered":"2023-10-06T02:23:09.983731+0000","last_clean":"2023-10-06T02:23:09.983731+0000","last_became_active":"2023-10-06T02:22:55.031367+0000","last_became_peered":"2023-10-06T02:22:55.031367+0000","last_unstale":"2023-10-06T02:23:09.983731+0000","last_undegraded":"2023-10-06T02:23:09.983731+0000","last_fullsized":"2023-10-06T02:23:09.983731+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:22:53.309097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:22:53.309097+0000","last_clean_scrub_stamp":"2023-10-06T02:22:53.309097+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:53:46.520909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.975628+0000","last_change":"2023-10-06T02:23:09.041907+0000","last_active":"2023-10-06T02:23:09.975628+0000","last_peered":"2023-10-06T02:23:09.975628+0000","last_clean":"2023-10-06T02:23:09.975628+0000","last_became_active":"2023-10-06T02:23:09.036698+0000","last_became_peered":"2023-10-06T02:23:09.036698+0000","last_unstale":"2023-10-06T02:23:09.975628+0000","last_undegraded":"2023-10-06T02:23:09.975628+0000","last_fullsized":"2023-10-06T02:23:09.975628+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:57.514051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.003862+0000","last_change":"2023-10-06T02:23:09.997846+0000","last_active":"2023-10-06T02:23:10.003862+0000","last_peered":"2023-10-06T02:23:10.003862+0000","last_clean":"2023-10-06T02:23:10.003862+0000","last_became_active":"2023-10-06T02:23:09.997558+0000","last_became_peered":"2023-10-06T02:23:09.997558+0000","last_unstale":"2023-10-06T02:23:10.003862+0000","last_undegraded":"2023-10-06T02:23:10.003862+0000","last_fullsized":"2023-10-06T02:23:10.003862+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:29:00.771363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.984381+0000","last_change":"2023-10-06T02:23:01.990773+0000","last_active":"2023-10-06T02:23:09.984381+0000","last_peered":"2023-10-06T02:23:09.984381+0000","last_clean":"2023-10-06T02:23:09.984381+0000","last_became_active":"2023-10-06T02:23:01.990511+0000","last_became_peered":"2023-10-06T02:23:01.990511+0000","last_unstale":"2023-10-06T02:23:09.984381+0000","last_undegraded":"2023-10-06T02:23:09.984381+0000","last_fullsized":"2023-10-06T02:23:09.984381+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:42:54.797731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.977043+0000","last_change":"2023-10-06T02:23:09.047067+0000","last_active":"2023-10-06T02:23:09.977043+0000","last_peered":"2023-10-06T02:23:09.977043+0000","last_clean":"2023-10-06T02:23:09.977043+0000","last_became_active":"2023-10-06T02:23:09.046735+0000","last_became_peered":"2023-10-06T02:23:09.046735+0000","last_unstale":"2023-10-06T02:23:09.977043+0000","last_undegraded":"2023-10-06T02:23:09.977043+0000","last_fullsized":"2023-10-06T02:23:09.977043+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:57.514051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.005223+0000","last_change":"2023-10-06T02:23:10.001635+0000","last_active":"2023-10-06T02:23:10.005223+0000","last_peered":"2023-10-06T02:23:10.005223+0000","last_clean":"2023-10-06T02:23:10.005223+0000","last_became_active":"2023-10-06T02:23:10.001341+0000","last_became_peered":"2023-10-06T02:23:10.001341+0000","last_unstale":"2023-10-06T02:23:10.005223+0000","last_undegraded":"2023-10-06T02:23:10.005223+0000","last_fullsized":"2023-10-06T02:23:10.005223+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:44:41.608418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.984779+0000","last_change":"2023-10-06T02:23:02.054069+0000","last_active":"2023-10-06T02:23:09.984779+0000","last_peered":"2023-10-06T02:23:09.984779+0000","last_clean":"2023-10-06T02:23:09.984779+0000","last_became_active":"2023-10-06T02:23:02.053739+0000","last_became_peered":"2023-10-06T02:23:02.053739+0000","last_unstale":"2023-10-06T02:23:09.984779+0000","last_undegraded":"2023-10-06T02:23:09.984779+0000","last_fullsized":"2023-10-06T02:23:09.984779+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:57:48.822608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.996931+0000","last_change":"2023-10-06T02:23:09.037195+0000","last_active":"2023-10-06T02:23:09.996931+0000","last_peered":"2023-10-06T02:23:09.996931+0000","last_clean":"2023-10-06T02:23:09.996931+0000","last_became_active":"2023-10-06T02:23:09.036925+0000","last_became_peered":"2023-10-06T02:23:09.036925+0000","last_unstale":"2023-10-06T02:23:09.996931+0000","last_undegraded":"2023-10-06T02:23:09.996931+0000","last_fullsized":"2023-10-06T02:23:09.996931+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:38.263723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.994210+0000","last_change":"2023-10-06T02:23:09.986467+0000","last_active":"2023-10-06T02:23:09.994210+0000","last_peered":"2023-10-06T02:23:09.994210+0000","last_clean":"2023-10-06T02:23:09.994210+0000","last_became_active":"2023-10-06T02:23:09.985750+0000","last_became_peered":"2023-10-06T02:23:09.985750+0000","last_unstale":"2023-10-06T02:23:09.994210+0000","last_undegraded":"2023-10-06T02:23:09.994210+0000","last_fullsized":"2023-10-06T02:23:09.994210+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:18:23.992787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.044047+0000","last_change":"2023-10-06T02:23:09.041622+0000","last_active":"2023-10-06T02:23:10.044047+0000","last_peered":"2023-10-06T02:23:10.044047+0000","last_clean":"2023-10-06T02:23:10.044047+0000","last_became_active":"2023-10-06T02:23:09.036680+0000","last_became_peered":"2023-10-06T02:23:09.036680+0000","last_unstale":"2023-10-06T02:23:10.044047+0000","last_undegraded":"2023-10-06T02:23:10.044047+0000","last_fullsized":"2023-10-06T02:23:10.044047+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:36:27.554258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.993558+0000","last_change":"2023-10-06T02:23:01.989934+0000","last_active":"2023-10-06T02:23:09.993558+0000","last_peered":"2023-10-06T02:23:09.993558+0000","last_clean":"2023-10-06T02:23:09.993558+0000","last_became_active":"2023-10-06T02:23:01.989706+0000","last_became_peered":"2023-10-06T02:23:01.989706+0000","last_unstale":"2023-10-06T02:23:09.993558+0000","last_undegraded":"2023-10-06T02:23:09.993558+0000","last_fullsized":"2023-10-06T02:23:09.993558+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:06.212251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.989446+0000","last_change":"2023-10-06T02:23:09.987681+0000","last_active":"2023-10-06T02:23:09.989446+0000","last_peered":"2023-10-06T02:23:09.989446+0000","last_clean":"2023-10-06T02:23:09.989446+0000","last_became_active":"2023-10-06T02:23:09.987439+0000","last_became_peered":"2023-10-06T02:23:09.987439+0000","last_unstale":"2023-10-06T02:23:09.989446+0000","last_undegraded":"2023-10-06T02:23:09.989446+0000","last_fullsized":"2023-10-06T02:23:09.989446+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:31:27.242353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.980493+0000","last_change":"2023-10-06T02:23:01.992935+0000","last_active":"2023-10-06T02:23:09.980493+0000","last_peered":"2023-10-06T02:23:09.980493+0000","last_clean":"2023-10-06T02:23:09.980493+0000","last_became_active":"2023-10-06T02:23:01.992568+0000","last_became_peered":"2023-10-06T02:23:01.992568+0000","last_unstale":"2023-10-06T02:23:09.980493+0000","last_undegraded":"2023-10-06T02:23:09.980493+0000","last_fullsized":"2023-10-06T02:23:09.980493+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:06.212251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"23'1","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T02:23:11.091780+0000","last_change":"2023-10-06T02:23:09.039504+0000","last_active":"2023-10-06T02:23:11.091780+0000","last_peered":"2023-10-06T02:23:11.091780+0000","last_clean":"2023-10-06T02:23:11.091780+0000","last_became_active":"2023-10-06T02:23:09.039056+0000","last_became_peered":"2023-10-06T02:23:09.039056+0000","last_unstale":"2023-10-06T02:23:11.091780+0000","last_undegraded":"2023-10-06T02:23:11.091780+0000","last_fullsized":"2023-10-06T02:23:11.091780+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:38.263723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.995987+0000","last_change":"2023-10-06T02:23:09.990800+0000","last_active":"2023-10-06T02:23:09.995987+0000","last_peered":"2023-10-06T02:23:09.995987+0000","last_clean":"2023-10-06T02:23:09.995987+0000","last_became_active":"2023-10-06T02:23:09.990531+0000","last_became_peered":"2023-10-06T02:23:09.990531+0000","last_unstale":"2023-10-06T02:23:09.995987+0000","last_undegraded":"2023-10-06T02:23:09.995987+0000","last_fullsized":"2023-10-06T02:23:09.995987+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:39:39.959479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.045165+0000","last_change":"2023-10-06T02:23:09.046230+0000","last_active":"2023-10-06T02:23:10.045165+0000","last_peered":"2023-10-06T02:23:10.045165+0000","last_clean":"2023-10-06T02:23:10.045165+0000","last_became_active":"2023-10-06T02:23:09.045976+0000","last_became_peered":"2023-10-06T02:23:09.045976+0000","last_unstale":"2023-10-06T02:23:10.045165+0000","last_undegraded":"2023-10-06T02:23:10.045165+0000","last_fullsized":"2023-10-06T02:23:10.045165+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:36:27.554258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.998669+0000","last_change":"2023-10-06T02:23:09.995680+0000","last_active":"2023-10-06T02:23:09.998669+0000","last_peered":"2023-10-06T02:23:09.998669+0000","last_clean":"2023-10-06T02:23:09.998669+0000","last_became_active":"2023-10-06T02:23:09.995375+0000","last_became_peered":"2023-10-06T02:23:09.995375+0000","last_unstale":"2023-10-06T02:23:09.998669+0000","last_undegraded":"2023-10-06T02:23:09.998669+0000","last_fullsized":"2023-10-06T02:23:09.998669+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:36:28.561818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.046988+0000","last_change":"2023-10-06T02:23:09.043361+0000","last_active":"2023-10-06T02:23:10.046988+0000","last_peered":"2023-10-06T02:23:10.046988+0000","last_clean":"2023-10-06T02:23:10.046988+0000","last_became_active":"2023-10-06T02:23:09.042973+0000","last_became_peered":"2023-10-06T02:23:09.042973+0000","last_unstale":"2023-10-06T02:23:10.046988+0000","last_undegraded":"2023-10-06T02:23:10.046988+0000","last_fullsized":"2023-10-06T02:23:10.046988+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:31:52.666890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.992487+0000","last_change":"2023-10-06T02:23:09.988158+0000","last_active":"2023-10-06T02:23:09.992487+0000","last_peered":"2023-10-06T02:23:09.992487+0000","last_clean":"2023-10-06T02:23:09.992487+0000","last_became_active":"2023-10-06T02:23:09.987844+0000","last_became_peered":"2023-10-06T02:23:09.987844+0000","last_unstale":"2023-10-06T02:23:09.992487+0000","last_undegraded":"2023-10-06T02:23:09.992487+0000","last_fullsized":"2023-10-06T02:23:09.992487+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:10:59.616168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.045352+0000","last_change":"2023-10-06T02:23:09.044069+0000","last_active":"2023-10-06T02:23:10.045352+0000","last_peered":"2023-10-06T02:23:10.045352+0000","last_clean":"2023-10-06T02:23:10.045352+0000","last_became_active":"2023-10-06T02:23:09.043700+0000","last_became_peered":"2023-10-06T02:23:09.043700+0000","last_unstale":"2023-10-06T02:23:10.045352+0000","last_undegraded":"2023-10-06T02:23:10.045352+0000","last_fullsized":"2023-10-06T02:23:10.045352+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:57:40.559177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.010156+0000","last_change":"2023-10-06T02:23:09.999700+0000","last_active":"2023-10-06T02:23:10.010156+0000","last_peered":"2023-10-06T02:23:10.010156+0000","last_clean":"2023-10-06T02:23:10.010156+0000","last_became_active":"2023-10-06T02:23:09.997418+0000","last_became_peered":"2023-10-06T02:23:09.997418+0000","last_unstale":"2023-10-06T02:23:10.010156+0000","last_undegraded":"2023-10-06T02:23:10.010156+0000","last_fullsized":"2023-10-06T02:23:10.010156+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:53:54.469523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.041904+0000","last_change":"2023-10-06T02:23:09.034024+0000","last_active":"2023-10-06T02:23:10.041904+0000","last_peered":"2023-10-06T02:23:10.041904+0000","last_clean":"2023-10-06T02:23:10.041904+0000","last_became_active":"2023-10-06T02:23:09.033389+0000","last_became_peered":"2023-10-06T02:23:09.033389+0000","last_unstale":"2023-10-06T02:23:10.041904+0000","last_undegraded":"2023-10-06T02:23:10.041904+0000","last_fullsized":"2023-10-06T02:23:10.041904+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:03:50.725863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.991920+0000","last_change":"2023-10-06T02:23:09.984080+0000","last_active":"2023-10-06T02:23:09.991920+0000","last_peered":"2023-10-06T02:23:09.991920+0000","last_clean":"2023-10-06T02:23:09.991920+0000","last_became_active":"2023-10-06T02:23:09.983808+0000","last_became_peered":"2023-10-06T02:23:09.983808+0000","last_unstale":"2023-10-06T02:23:09.991920+0000","last_undegraded":"2023-10-06T02:23:09.991920+0000","last_fullsized":"2023-10-06T02:23:09.991920+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:12.998563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"22'4","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.045612+0000","last_change":"2023-10-06T02:23:09.037190+0000","last_active":"2023-10-06T02:23:10.045612+0000","last_peered":"2023-10-06T02:23:10.045612+0000","last_clean":"2023-10-06T02:23:10.045612+0000","last_became_active":"2023-10-06T02:23:09.036872+0000","last_became_peered":"2023-10-06T02:23:09.036872+0000","last_unstale":"2023-10-06T02:23:10.045612+0000","last_undegraded":"2023-10-06T02:23:10.045612+0000","last_fullsized":"2023-10-06T02:23:10.045612+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:08.003076+0000","last_clean_scrub_stamp":"2023-10-06T02:23:08.003076+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:57:40.559177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.994436+0000","last_change":"2023-10-06T02:23:09.989896+0000","last_active":"2023-10-06T02:23:09.994436+0000","last_peered":"2023-10-06T02:23:09.994436+0000","last_clean":"2023-10-06T02:23:09.994436+0000","last_became_active":"2023-10-06T02:23:09.989525+0000","last_became_peered":"2023-10-06T02:23:09.989525+0000","last_unstale":"2023-10-06T02:23:09.994436+0000","last_undegraded":"2023-10-06T02:23:09.994436+0000","last_fullsized":"2023-10-06T02:23:09.994436+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:58:41.925466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.992536+0000","last_change":"2023-10-06T02:23:09.988852+0000","last_active":"2023-10-06T02:23:09.992536+0000","last_peered":"2023-10-06T02:23:09.992536+0000","last_clean":"2023-10-06T02:23:09.992536+0000","last_became_active":"2023-10-06T02:23:09.988527+0000","last_became_peered":"2023-10-06T02:23:09.988527+0000","last_unstale":"2023-10-06T02:23:09.992536+0000","last_undegraded":"2023-10-06T02:23:09.992536+0000","last_fullsized":"2023-10-06T02:23:09.992536+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:03:51.733423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.001202+0000","last_change":"2023-10-06T02:23:09.999698+0000","last_active":"2023-10-06T02:23:10.001202+0000","last_peered":"2023-10-06T02:23:10.001202+0000","last_clean":"2023-10-06T02:23:10.001202+0000","last_became_active":"2023-10-06T02:23:09.999418+0000","last_became_peered":"2023-10-06T02:23:09.999418+0000","last_unstale":"2023-10-06T02:23:10.001202+0000","last_undegraded":"2023-10-06T02:23:10.001202+0000","last_fullsized":"2023-10-06T02:23:10.001202+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:10:59.616168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.010180+0000","last_change":"2023-10-06T02:23:10.000203+0000","last_active":"2023-10-06T02:23:10.010180+0000","last_peered":"2023-10-06T02:23:10.010180+0000","last_clean":"2023-10-06T02:23:10.010180+0000","last_became_active":"2023-10-06T02:23:09.999397+0000","last_became_peered":"2023-10-06T02:23:09.999397+0000","last_unstale":"2023-10-06T02:23:10.010180+0000","last_undegraded":"2023-10-06T02:23:10.010180+0000","last_fullsized":"2023-10-06T02:23:10.010180+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:31:27.242353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:09.989526+0000","last_change":"2023-10-06T02:23:09.986996+0000","last_active":"2023-10-06T02:23:09.989526+0000","last_peered":"2023-10-06T02:23:09.989526+0000","last_clean":"2023-10-06T02:23:09.989526+0000","last_became_active":"2023-10-06T02:23:09.986723+0000","last_became_peered":"2023-10-06T02:23:09.986723+0000","last_unstale":"2023-10-06T02:23:09.989526+0000","last_undegraded":"2023-10-06T02:23:09.989526+0000","last_fullsized":"2023-10-06T02:23:09.989526+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:19:29.462977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T02:23:10.007171+0000","last_change":"2023-10-06T02:23:10.003345+0000","last_active":"2023-10-06T02:23:10.007171+0000","last_peered":"2023-10-06T02:23:10.007171+0000","last_clean":"2023-10-06T02:23:10.007171+0000","last_became_active":"2023-10-06T02:23:10.003023+0000","last_became_peered":"2023-10-06T02:23:10.003023+0000","last_unstale":"2023-10-06T02:23:10.007171+0000","last_undegraded":"2023-10-06T02:23:10.007171+0000","last_fullsized":"2023-10-06T02:23:10.007171+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:09.010636+0000","last_clean_scrub_stamp":"2023-10-06T02:23:09.010636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:18:23.992787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298132,"kb_used_data":844,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073708,"statfs":{"total":96636764160,"available":96331476992,"internally_reserved":0,"allocated":864256,"data_stored":681439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41399999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297524,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074316,"statfs":{"total":96636764160,"available":96332099584,"internally_reserved":0,"allocated":241664,"data_stored":89319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81499999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297524,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074316,"statfs":{"total":96636764160,"available":96332099584,"internally_reserved":0,"allocated":241664,"data_stored":89319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2230000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0840000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.266}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9279999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9830000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.829}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3340000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298108,"kb_used_data":820,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073732,"statfs":{"total":96636764160,"available":96331501568,"internally_reserved":0,"allocated":839680,"data_stored":680241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69299999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86699999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96499999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574859,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297524,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074316,"statfs":{"total":96636764160,"available":96332099584,"internally_reserved":0,"allocated":241664,"data_stored":89319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79200000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83699999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297528,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074312,"statfs":{"total":96636764160,"available":96332095488,"internally_reserved":0,"allocated":245760,"data_stored":89409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.024}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.395}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0870000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.214}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.131}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.357}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.27}]}]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297544,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074296,"statfs":{"total":96636764160,"available":96332079104,"internally_reserved":0,"allocated":262144,"data_stored":90911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79700000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88600000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297532,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074308,"statfs":{"total":96636764160,"available":96332091392,"internally_reserved":0,"allocated":249856,"data_stored":89943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T02:23:25.386 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-06T02:23:25.386 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-10-06T02:23:25.386 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-10-06T02:23:25.387 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-06T02:23:25.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.603+0000 7fc3e5f9d700 1 Processor -- start 2023-10-06T02:23:25.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.604+0000 7fc3e5f9d700 1 -- start start 2023-10-06T02:23:25.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.605+0000 7fc3e5f9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc3e0159920 0x7fc3e00ad090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:25.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.605+0000 7fc3e5f9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3e00acc00 con 0x7fc3e0159550 2023-10-06T02:23:25.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.605+0000 7fc3e5f9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3e00abff0 con 0x7fc3e015dfd0 2023-10-06T02:23:25.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.605+0000 7fc3e5f9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3e00ac2d0 con 0x7fc3e0159920 2023-10-06T02:23:25.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.605+0000 7fc3e4f9b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fc3e0159550 0x7fc3e00ceac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49366/0 (socket says 172.21.15.31:49366) 2023-10-06T02:23:25.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.605+0000 7fc3e4f9b700 1 -- 172.21.15.31:0/3981360149 learned_addr learned my addr 172.21.15.31:0/3981360149 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:23:25.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.605+0000 7fc3e579c700 1 --2- 172.21.15.31:0/3981360149 >> v2:172.21.15.62:3300/0 conn(0x7fc3e0159920 0x7fc3e00ad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:25.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.606+0000 7fc3df7fe700 1 -- 172.21.15.31:0/3981360149 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1009887080 0 0) 0x7fc3e00acc00 con 0x7fc3e0159550 2023-10-06T02:23:25.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.606+0000 7fc3df7fe700 1 -- 172.21.15.31:0/3981360149 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3d0003570 con 0x7fc3e0159550 2023-10-06T02:23:25.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.606+0000 7fc3df7fe700 1 -- 172.21.15.31:0/3981360149 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 581324639 0 0) 0x7fc3e00abff0 con 0x7fc3e015dfd0 2023-10-06T02:23:25.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.606+0000 7fc3df7fe700 1 -- 172.21.15.31:0/3981360149 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3e00acc00 con 0x7fc3e015dfd0 2023-10-06T02:23:25.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.606+0000 7fc3e579c700 1 -- 172.21.15.31:0/3981360149 >> v1:172.21.15.62:6789/0 conn(0x7fc3e015dfd0 legacy=0x7fc3e00aedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:25.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.606+0000 7fc3e579c700 1 -- 172.21.15.31:0/3981360149 >> v1:172.21.15.31:6789/0 conn(0x7fc3e0159550 legacy=0x7fc3e00ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:25.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.606+0000 7fc3df7fe700 1 -- 172.21.15.31:0/3981360149 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1376196611 0 0) 0x7fc3d0003570 con 0x7fc3e0159550 2023-10-06T02:23:25.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.606+0000 7fc3e579c700 1 -- 172.21.15.31:0/3981360149 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3e00ac5b0 con 0x7fc3e0159920 2023-10-06T02:23:25.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.606+0000 7fc3e579c700 1 --2- 172.21.15.31:0/3981360149 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3e0159920 0x7fc3e00ad090 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc3d8009e60 tx=0x7fc3d8009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc5f53ccdb738262 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:25.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.606+0000 7fc3df7fe700 1 -- 172.21.15.31:0/3981360149 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3d800e070 con 0x7fc3e0159920 2023-10-06T02:23:25.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.606+0000 7fc3df7fe700 1 -- 172.21.15.31:0/3981360149 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3d800b7a0 con 0x7fc3e0159920 2023-10-06T02:23:25.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.606+0000 7fc3df7fe700 1 -- 172.21.15.31:0/3981360149 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3d4001470 con 0x7fc3e0159920 2023-10-06T02:23:25.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.607+0000 7fc3e5f9d700 1 -- 172.21.15.31:0/3981360149 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3e0159920 msgr2=0x7fc3e00ad090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:25.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.607+0000 7fc3e5f9d700 1 --2- 172.21.15.31:0/3981360149 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3e0159920 0x7fc3e00ad090 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc3d8009e60 tx=0x7fc3d8009780 comp rx=0 tx=0).stop 2023-10-06T02:23:25.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.607+0000 7fc3e5f9d700 1 -- 172.21.15.31:0/3981360149 shutdown_connections 2023-10-06T02:23:25.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.607+0000 7fc3e5f9d700 1 --2- 172.21.15.31:0/3981360149 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3e0159920 0x7fc3e00ad090 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:25.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.607+0000 7fc3e5f9d700 1 -- 172.21.15.31:0/3981360149 >> 172.21.15.31:0/3981360149 conn(0x7fc3e006ce80 msgr2=0x7fc3e00ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:25.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.607+0000 7fc3e5f9d700 1 -- 172.21.15.31:0/3981360149 shutdown_connections 2023-10-06T02:23:25.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.607+0000 7fc3e5f9d700 1 -- 172.21.15.31:0/3981360149 wait complete. 2023-10-06T02:23:25.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.609+0000 7fc3e5f9d700 1 Processor -- start 2023-10-06T02:23:25.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.609+0000 7fc3e5f9d700 1 -- start start 2023-10-06T02:23:25.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.610+0000 7fc3e5f9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3e0159550 0x7fc3e00c8380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:25.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.610+0000 7fc3e5f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc3e0159920 0x7fc3e014c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:25.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.610+0000 7fc3e5f9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3e015dfd0 0x7fc3e014fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:25.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.610+0000 7fc3e5f9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e00ca4e0 con 0x7fc3e0159550 2023-10-06T02:23:25.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.610+0000 7fc3e5f9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e00b1ab0 con 0x7fc3e015dfd0 2023-10-06T02:23:25.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.610+0000 7fc3e5f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e00b1d90 con 0x7fc3e0159920 2023-10-06T02:23:25.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.610+0000 7fc3e4f9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3e0159550 0x7fc3e00c8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:25.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.610+0000 7fc3e4f9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3e0159550 0x7fc3e00c8380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37780/0 (socket says 172.21.15.31:37780) 2023-10-06T02:23:25.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.610+0000 7fc3e4f9b700 1 -- 172.21.15.31:0/1247980037 learned_addr learned my addr 172.21.15.31:0/1247980037 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:25.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.610+0000 7fc3e579c700 1 --2- 172.21.15.31:0/1247980037 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3e015dfd0 0x7fc3e014fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:25.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.610+0000 7fc3dffff700 1 --2- 172.21.15.31:0/1247980037 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc3e0159920 0x7fc3e014c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:25.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.610+0000 7fc3e4f9b700 1 -- 172.21.15.31:0/1247980037 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc3e0159920 msgr2=0x7fc3e014c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:25.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.610+0000 7fc3e4f9b700 1 --2- 172.21.15.31:0/1247980037 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc3e0159920 0x7fc3e014c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:25.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.610+0000 7fc3e4f9b700 1 -- 172.21.15.31:0/1247980037 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3e015dfd0 msgr2=0x7fc3e014fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:25.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.610+0000 7fc3e4f9b700 1 --2- 172.21.15.31:0/1247980037 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3e015dfd0 0x7fc3e014fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:25.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.610+0000 7fc3e4f9b700 1 -- 172.21.15.31:0/1247980037 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3d8009a20 con 0x7fc3e0159550 2023-10-06T02:23:25.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.610+0000 7fc3dffff700 1 --2- 172.21.15.31:0/1247980037 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc3e0159920 0x7fc3e014c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:23:25.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.611+0000 7fc3e4f9b700 1 --2- 172.21.15.31:0/1247980037 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3e0159550 0x7fc3e00c8380 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc3d400cb90 tx=0x7fc3d4001ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:25.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.611+0000 7fc3dcff9700 1 -- 172.21.15.31:0/1247980037 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3d4001d00 con 0x7fc3e0159550 2023-10-06T02:23:25.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.611+0000 7fc3dcff9700 1 -- 172.21.15.31:0/1247980037 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3d4007360 con 0x7fc3e0159550 2023-10-06T02:23:25.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.611+0000 7fc3dcff9700 1 -- 172.21.15.31:0/1247980037 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3d4004830 con 0x7fc3e0159550 2023-10-06T02:23:25.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.611+0000 7fc3e5f9d700 1 -- 172.21.15.31:0/1247980037 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3e00b2070 con 0x7fc3e0159550 2023-10-06T02:23:25.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.611+0000 7fc3e5f9d700 1 -- 172.21.15.31:0/1247980037 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3e016df80 con 0x7fc3e0159550 2023-10-06T02:23:25.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.612+0000 7fc3e5f9d700 1 -- 172.21.15.31:0/1247980037 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3e004c3c0 con 0x7fc3e0159550 2023-10-06T02:23:25.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.613+0000 7fc3dcff9700 1 -- 172.21.15.31:0/1247980037 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc3d4004be0 con 0x7fc3e0159550 2023-10-06T02:23:25.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.614+0000 7fc3dcff9700 1 --2- 172.21.15.31:0/1247980037 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc3c4066f30 0x7fc3c40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:25.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.614+0000 7fc3dcff9700 1 -- 172.21.15.31:0/1247980037 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc3d4083fd0 con 0x7fc3e0159550 2023-10-06T02:23:25.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.614+0000 7fc3dffff700 1 --2- 172.21.15.31:0/1247980037 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc3c4066f30 0x7fc3c40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:25.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.614+0000 7fc3dffff700 1 --2- 172.21.15.31:0/1247980037 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc3c4066f30 0x7fc3c40693e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc007fd0 tx=0x7fc3cc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:25.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.618+0000 7fc3dcff9700 1 -- 172.21.15.31:0/1247980037 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc3d40510c0 con 0x7fc3e0159550 2023-10-06T02:23:25.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.862+0000 7fc3e5f9d700 1 -- 172.21.15.31:0/1247980037 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fc3e006e7c0 con 0x7fc3e0159550 2023-10-06T02:23:25.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.862+0000 7fc3dcff9700 1 -- 172.21.15.31:0/1247980037 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fc3d4018080 con 0x7fc3e0159550 2023-10-06T02:23:25.863 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:23:25.863 INFO:teuthology.orchestra.run.smithi031.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-10-06T02:23:25.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.865+0000 7fc3e5f9d700 1 -- 172.21.15.31:0/1247980037 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc3c4066f30 msgr2=0x7fc3c40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:25.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.865+0000 7fc3e5f9d700 1 --2- 172.21.15.31:0/1247980037 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc3c4066f30 0x7fc3c40693e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc007fd0 tx=0x7fc3cc009040 comp rx=0 tx=0).stop 2023-10-06T02:23:25.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.865+0000 7fc3e5f9d700 1 -- 172.21.15.31:0/1247980037 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3e0159550 msgr2=0x7fc3e00c8380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:25.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.865+0000 7fc3e5f9d700 1 --2- 172.21.15.31:0/1247980037 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3e0159550 0x7fc3e00c8380 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc3d400cb90 tx=0x7fc3d4001ab0 comp rx=0 tx=0).stop 2023-10-06T02:23:25.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.865+0000 7fc3e5f9d700 1 -- 172.21.15.31:0/1247980037 shutdown_connections 2023-10-06T02:23:25.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.865+0000 7fc3e5f9d700 1 --2- 172.21.15.31:0/1247980037 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3e015dfd0 0x7fc3e014fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:25.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.865+0000 7fc3e5f9d700 1 --2- 172.21.15.31:0/1247980037 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3e0159550 0x7fc3e00c8380 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:25.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.865+0000 7fc3e5f9d700 1 --2- 172.21.15.31:0/1247980037 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc3c4066f30 0x7fc3c40693e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:25.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.865+0000 7fc3e5f9d700 1 --2- 172.21.15.31:0/1247980037 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc3e0159920 0x7fc3e014c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:25.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.865+0000 7fc3e5f9d700 1 -- 172.21.15.31:0/1247980037 >> 172.21.15.31:0/1247980037 conn(0x7fc3e006ce80 msgr2=0x7fc3e0154f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:25.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.865+0000 7fc3e5f9d700 1 -- 172.21.15.31:0/1247980037 shutdown_connections 2023-10-06T02:23:25.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:25.865+0000 7fc3e5f9d700 1 -- 172.21.15.31:0/1247980037 wait complete. 2023-10-06T02:23:25.882 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-10-06T02:23:25.883 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:23:26.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.072+0000 7f5570b9d700 1 Processor -- start 2023-10-06T02:23:26.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.072+0000 7f5570b9d700 1 -- start start 2023-10-06T02:23:26.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.074+0000 7f5570b9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f556c13fb20 0x7f556c15a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:26.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.074+0000 7f5570b9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f556c13fef0 0x7f556c0c7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:26.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.074+0000 7f5570b9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f556c15ae70 con 0x7f556c06aaa0 2023-10-06T02:23:26.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.074+0000 7f5570b9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f556c06af60 con 0x7f556c13fef0 2023-10-06T02:23:26.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.074+0000 7f5570b9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f556c06b240 con 0x7f556c13fb20 2023-10-06T02:23:26.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.074+0000 7f556affd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f556c13fef0 0x7f556c0c7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:26.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.074+0000 7f556affd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f556c13fef0 0x7f556c0c7990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37788/0 (socket says 172.21.15.31:37788) 2023-10-06T02:23:26.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.074+0000 7f556affd700 1 -- 172.21.15.31:0/2896400235 learned_addr learned my addr 172.21.15.31:0/2896400235 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:26.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.074+0000 7f556b7fe700 1 --2- 172.21.15.31:0/2896400235 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f556c13fb20 0x7f556c15a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:26.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.074+0000 7f556affd700 1 -- 172.21.15.31:0/2896400235 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f556c13fb20 msgr2=0x7f556c15a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:26.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.074+0000 7f556affd700 1 --2- 172.21.15.31:0/2896400235 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f556c13fb20 0x7f556c15a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:26.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.074+0000 7f556affd700 1 -- 172.21.15.31:0/2896400235 >> v1:172.21.15.62:6789/0 conn(0x7f556c06aaa0 legacy=0x7f556c0c6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:26.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.075+0000 7f556affd700 1 -- 172.21.15.31:0/2896400235 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f556c06b520 con 0x7f556c13fef0 2023-10-06T02:23:26.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.075+0000 7f556affd700 1 --2- 172.21.15.31:0/2896400235 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f556c13fef0 0x7f556c0c7990 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5560009990 tx=0x7f5560009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa160fba749150 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:26.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.075+0000 7f556a7fc700 1 -- 172.21.15.31:0/2896400235 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5560005c10 con 0x7f556c13fef0 2023-10-06T02:23:26.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.075+0000 7f556a7fc700 1 -- 172.21.15.31:0/2896400235 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5560005d70 con 0x7f556c13fef0 2023-10-06T02:23:26.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.075+0000 7f556a7fc700 1 -- 172.21.15.31:0/2896400235 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f556000b880 con 0x7f556c13fef0 2023-10-06T02:23:26.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.075+0000 7f5570b9d700 1 -- 172.21.15.31:0/2896400235 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f556c13fef0 msgr2=0x7f556c0c7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:26.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.075+0000 7f5570b9d700 1 --2- 172.21.15.31:0/2896400235 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f556c13fef0 0x7f556c0c7990 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5560009990 tx=0x7f5560009220 comp rx=0 tx=0).stop 2023-10-06T02:23:26.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.076+0000 7f5570b9d700 1 -- 172.21.15.31:0/2896400235 shutdown_connections 2023-10-06T02:23:26.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.076+0000 7f5570b9d700 1 --2- 172.21.15.31:0/2896400235 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f556c13fb20 0x7f556c15a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:26.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.076+0000 7f5570b9d700 1 --2- 172.21.15.31:0/2896400235 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f556c13fef0 0x7f556c0c7990 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:26.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.076+0000 7f5570b9d700 1 -- 172.21.15.31:0/2896400235 >> 172.21.15.31:0/2896400235 conn(0x7f556c06d170 msgr2=0x7f556c0cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:26.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.076+0000 7f5570b9d700 1 -- 172.21.15.31:0/2896400235 shutdown_connections 2023-10-06T02:23:26.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.076+0000 7f5570b9d700 1 -- 172.21.15.31:0/2896400235 wait complete. 2023-10-06T02:23:26.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.077+0000 7f5570b9d700 1 Processor -- start 2023-10-06T02:23:26.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.077+0000 7f5570b9d700 1 -- start start 2023-10-06T02:23:26.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.078+0000 7f5570b9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f556c06aaa0 0x7f556c0accd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:26.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.078+0000 7f5570b9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f556c13fb20 0x7f556c155d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:26.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.078+0000 7f5570b9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f556c13fef0 0x7f556c167740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:26.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.078+0000 7f5570b9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f556c0ad210 con 0x7f556c13fb20 2023-10-06T02:23:26.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.078+0000 7f5570b9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f556c1562c0 con 0x7f556c06aaa0 2023-10-06T02:23:26.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.078+0000 7f5570b9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f556c0aba70 con 0x7f556c13fef0 2023-10-06T02:23:26.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.078+0000 7f556affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f556c13fb20 0x7f556c155d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:26.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.078+0000 7f556affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f556c13fb20 0x7f556c155d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37794/0 (socket says 172.21.15.31:37794) 2023-10-06T02:23:26.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.078+0000 7f556affd700 1 -- 172.21.15.31:0/3138088678 learned_addr learned my addr 172.21.15.31:0/3138088678 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:26.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.078+0000 7f556b7fe700 1 --2- 172.21.15.31:0/3138088678 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f556c06aaa0 0x7f556c0accd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:26.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.078+0000 7f556bfff700 1 --2- 172.21.15.31:0/3138088678 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f556c13fef0 0x7f556c167740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:26.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.079+0000 7f556affd700 1 -- 172.21.15.31:0/3138088678 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f556c13fef0 msgr2=0x7f556c167740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:26.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.079+0000 7f556affd700 1 --2- 172.21.15.31:0/3138088678 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f556c13fef0 0x7f556c167740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:26.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.079+0000 7f556affd700 1 -- 172.21.15.31:0/3138088678 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f556c06aaa0 msgr2=0x7f556c0accd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:26.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.079+0000 7f556affd700 1 --2- 172.21.15.31:0/3138088678 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f556c06aaa0 0x7f556c0accd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:26.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.079+0000 7f556affd700 1 -- 172.21.15.31:0/3138088678 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5560009480 con 0x7f556c13fb20 2023-10-06T02:23:26.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.079+0000 7f556affd700 1 --2- 172.21.15.31:0/3138088678 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f556c13fb20 0x7f556c155d80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5560005d40 tx=0x7f5560016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:26.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.079+0000 7f55537fe700 1 -- 172.21.15.31:0/3138088678 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5560006b60 con 0x7f556c13fb20 2023-10-06T02:23:26.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.079+0000 7f55537fe700 1 -- 172.21.15.31:0/3138088678 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55600118c0 con 0x7f556c13fb20 2023-10-06T02:23:26.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.079+0000 7f5570b9d700 1 -- 172.21.15.31:0/3138088678 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f556c156950 con 0x7f556c13fb20 2023-10-06T02:23:26.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.079+0000 7f55537fe700 1 -- 172.21.15.31:0/3138088678 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5560011c40 con 0x7f556c13fb20 2023-10-06T02:23:26.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.080+0000 7f5570b9d700 1 -- 172.21.15.31:0/3138088678 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f556c156e80 con 0x7f556c13fb20 2023-10-06T02:23:26.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.080+0000 7f5570b9d700 1 -- 172.21.15.31:0/3138088678 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f556c04c3c0 con 0x7f556c13fb20 2023-10-06T02:23:26.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.081+0000 7f55537fe700 1 -- 172.21.15.31:0/3138088678 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f556001b640 con 0x7f556c13fb20 2023-10-06T02:23:26.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.082+0000 7f55537fe700 1 --2- 172.21.15.31:0/3138088678 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5558066f30 0x7f55580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:26.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.082+0000 7f55537fe700 1 -- 172.21.15.31:0/3138088678 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f5560085fb0 con 0x7f556c13fb20 2023-10-06T02:23:26.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.082+0000 7f556b7fe700 1 --2- 172.21.15.31:0/3138088678 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5558066f30 0x7f55580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:26.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.083+0000 7f556b7fe700 1 --2- 172.21.15.31:0/3138088678 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5558066f30 0x7f55580693e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f555c006fd0 tx=0x7f555c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:26.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.086+0000 7f55537fe700 1 -- 172.21.15.31:0/3138088678 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5560053020 con 0x7f556c13fb20 2023-10-06T02:23:26.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.289+0000 7f5570b9d700 1 -- 172.21.15.31:0/3138088678 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f556c1575e0 con 0x7f556c13fb20 2023-10-06T02:23:26.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.289+0000 7f55537fe700 1 -- 172.21.15.31:0/3138088678 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16818 (secure 0 0 0) 0x7f556002a090 con 0x7f556c13fb20 2023-10-06T02:23:26.290 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:23:26.290 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":23,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:23:10.970534+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:23:07.290269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T02:23:08.504679+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:23:26.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.292+0000 7f5570b9d700 1 -- 172.21.15.31:0/3138088678 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5558066f30 msgr2=0x7f55580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:26.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.292+0000 7f5570b9d700 1 --2- 172.21.15.31:0/3138088678 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5558066f30 0x7f55580693e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f555c006fd0 tx=0x7f555c008040 comp rx=0 tx=0).stop 2023-10-06T02:23:26.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.292+0000 7f5570b9d700 1 -- 172.21.15.31:0/3138088678 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f556c13fb20 msgr2=0x7f556c155d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:26.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.292+0000 7f5570b9d700 1 --2- 172.21.15.31:0/3138088678 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f556c13fb20 0x7f556c155d80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5560005d40 tx=0x7f5560016000 comp rx=0 tx=0).stop 2023-10-06T02:23:26.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.292+0000 7f5570b9d700 1 -- 172.21.15.31:0/3138088678 shutdown_connections 2023-10-06T02:23:26.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.292+0000 7f5570b9d700 1 --2- 172.21.15.31:0/3138088678 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5558066f30 0x7f55580693e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:26.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.292+0000 7f5570b9d700 1 --2- 172.21.15.31:0/3138088678 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f556c13fef0 0x7f556c167740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:26.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.292+0000 7f5570b9d700 1 --2- 172.21.15.31:0/3138088678 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f556c06aaa0 0x7f556c0accd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:26.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.292+0000 7f5570b9d700 1 --2- 172.21.15.31:0/3138088678 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f556c13fb20 0x7f556c155d80 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:26.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.292+0000 7f5570b9d700 1 -- 172.21.15.31:0/3138088678 >> 172.21.15.31:0/3138088678 conn(0x7f556c06d170 msgr2=0x7f556c0aeed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:26.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.292+0000 7f5570b9d700 1 -- 172.21.15.31:0/3138088678 shutdown_connections 2023-10-06T02:23:26.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.292+0000 7f5570b9d700 1 -- 172.21.15.31:0/3138088678 wait complete. 2023-10-06T02:23:26.310 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T02:22:53.183425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T02:23:00.046967+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T02:23:07.290269+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T02:23:08.504679+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T02:23:26.310 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T02:23:26.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.499+0000 7fc5d6b9d700 1 Processor -- start 2023-10-06T02:23:26.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.500+0000 7fc5d6b9d700 1 -- start start 2023-10-06T02:23:26.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.501+0000 7fc5d6b9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc5d0159550 0x7fc5d00ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:26.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.501+0000 7fc5d6b9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc5d0159920 0x7fc5d00ad090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:26.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.501+0000 7fc5d6b9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5d00acc00 con 0x7fc5d015dfd0 2023-10-06T02:23:26.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.501+0000 7fc5d6b9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5d00abff0 con 0x7fc5d0159550 2023-10-06T02:23:26.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.501+0000 7fc5d6b9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5d00ac2d0 con 0x7fc5d0159920 2023-10-06T02:23:26.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.501+0000 7fc5d5b9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc5d0159550 0x7fc5d00ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:26.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.501+0000 7fc5d5b9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc5d0159550 0x7fc5d00ceac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37808/0 (socket says 172.21.15.31:37808) 2023-10-06T02:23:26.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.501+0000 7fc5d5b9b700 1 -- 172.21.15.31:0/299812962 learned_addr learned my addr 172.21.15.31:0/299812962 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:26.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.501+0000 7fc5d639c700 1 --2- 172.21.15.31:0/299812962 >> v2:172.21.15.62:3300/0 conn(0x7fc5d0159920 0x7fc5d00ad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:26.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.502+0000 7fc5d5b9b700 1 -- 172.21.15.31:0/299812962 >> v2:172.21.15.62:3300/0 conn(0x7fc5d0159920 msgr2=0x7fc5d00ad090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:26.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.502+0000 7fc5d5b9b700 1 --2- 172.21.15.31:0/299812962 >> v2:172.21.15.62:3300/0 conn(0x7fc5d0159920 0x7fc5d00ad090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:26.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.502+0000 7fc5d5b9b700 1 -- 172.21.15.31:0/299812962 >> v1:172.21.15.62:6789/0 conn(0x7fc5d015dfd0 legacy=0x7fc5d00aedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:26.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.502+0000 7fc5d5b9b700 1 -- 172.21.15.31:0/299812962 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5d00ac5b0 con 0x7fc5d0159550 2023-10-06T02:23:26.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.502+0000 7fc5d5b9b700 1 --2- 172.21.15.31:0/299812962 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5d0159550 0x7fc5d00ceac0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc5d006b0f0 tx=0x7fc5cc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47b8e16056c9a9c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:26.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.502+0000 7fc5d4b99700 1 -- 172.21.15.31:0/299812962 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc009e70 con 0x7fc5d0159550 2023-10-06T02:23:26.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.502+0000 7fc5d4b99700 1 -- 172.21.15.31:0/299812962 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5cc00b860 con 0x7fc5d0159550 2023-10-06T02:23:26.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.502+0000 7fc5d4b99700 1 -- 172.21.15.31:0/299812962 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc00bc00 con 0x7fc5d0159550 2023-10-06T02:23:26.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.503+0000 7fc5d6b9d700 1 -- 172.21.15.31:0/299812962 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5d0159550 msgr2=0x7fc5d00ceac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:26.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.503+0000 7fc5d6b9d700 1 --2- 172.21.15.31:0/299812962 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5d0159550 0x7fc5d00ceac0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc5d006b0f0 tx=0x7fc5cc009250 comp rx=0 tx=0).stop 2023-10-06T02:23:26.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.503+0000 7fc5d6b9d700 1 -- 172.21.15.31:0/299812962 shutdown_connections 2023-10-06T02:23:26.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.503+0000 7fc5d6b9d700 1 --2- 172.21.15.31:0/299812962 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5d0159550 0x7fc5d00ceac0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:26.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.503+0000 7fc5d6b9d700 1 --2- 172.21.15.31:0/299812962 >> v2:172.21.15.62:3300/0 conn(0x7fc5d0159920 0x7fc5d00ad090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:26.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.503+0000 7fc5d6b9d700 1 -- 172.21.15.31:0/299812962 >> 172.21.15.31:0/299812962 conn(0x7fc5d006ce80 msgr2=0x7fc5d00ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:26.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.503+0000 7fc5d6b9d700 1 -- 172.21.15.31:0/299812962 shutdown_connections 2023-10-06T02:23:26.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.503+0000 7fc5d6b9d700 1 -- 172.21.15.31:0/299812962 wait complete. 2023-10-06T02:23:26.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.504+0000 7fc5d6b9d700 1 Processor -- start 2023-10-06T02:23:26.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.504+0000 7fc5d6b9d700 1 -- start start 2023-10-06T02:23:26.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.505+0000 7fc5d6b9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5d0159550 0x7fc5d0152110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:26.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.505+0000 7fc5d6b9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc5d0159920 0x7fc5d0098d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:26.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.505+0000 7fc5d6b9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc5d015dfd0 0x7fc5d00cda40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:26.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.505+0000 7fc5d6b9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d00ce5b0 con 0x7fc5d0159550 2023-10-06T02:23:26.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.505+0000 7fc5d6b9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d00cc7c0 con 0x7fc5d015dfd0 2023-10-06T02:23:26.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.505+0000 7fc5d6b9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d00b1850 con 0x7fc5d0159920 2023-10-06T02:23:26.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.506+0000 7fc5d5b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5d0159550 0x7fc5d0152110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:26.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.506+0000 7fc5d5b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5d0159550 0x7fc5d0152110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37820/0 (socket says 172.21.15.31:37820) 2023-10-06T02:23:26.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.506+0000 7fc5d639c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc5d015dfd0 0x7fc5d00cda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:26.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.506+0000 7fc5d5b9b700 1 -- 172.21.15.31:0/4187183717 learned_addr learned my addr 172.21.15.31:0/4187183717 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:26.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.506+0000 7fc5d539a700 1 --2- 172.21.15.31:0/4187183717 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc5d0159920 0x7fc5d0098d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:26.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.506+0000 7fc5d5b9b700 1 -- 172.21.15.31:0/4187183717 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc5d0159920 msgr2=0x7fc5d0098d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:26.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.506+0000 7fc5d5b9b700 1 --2- 172.21.15.31:0/4187183717 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc5d0159920 0x7fc5d0098d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:26.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.506+0000 7fc5d5b9b700 1 -- 172.21.15.31:0/4187183717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc5d015dfd0 msgr2=0x7fc5d00cda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:26.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.506+0000 7fc5d5b9b700 1 --2- 172.21.15.31:0/4187183717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc5d015dfd0 0x7fc5d00cda40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:26.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.506+0000 7fc5d5b9b700 1 -- 172.21.15.31:0/4187183717 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5cc0094f0 con 0x7fc5d0159550 2023-10-06T02:23:26.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.506+0000 7fc5d5b9b700 1 --2- 172.21.15.31:0/4187183717 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5d0159550 0x7fc5d0152110 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc5d006ad70 tx=0x7fc5cc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:26.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.506+0000 7fc5c5ffb700 1 -- 172.21.15.31:0/4187183717 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc00bbe0 con 0x7fc5d0159550 2023-10-06T02:23:26.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.507+0000 7fc5c5ffb700 1 -- 172.21.15.31:0/4187183717 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5cc011420 con 0x7fc5d0159550 2023-10-06T02:23:26.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.507+0000 7fc5c5ffb700 1 -- 172.21.15.31:0/4187183717 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc0117e0 con 0x7fc5d0159550 2023-10-06T02:23:26.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.507+0000 7fc5d6b9d700 1 -- 172.21.15.31:0/4187183717 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5d00b1b10 con 0x7fc5d0159550 2023-10-06T02:23:26.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.507+0000 7fc5d6b9d700 1 -- 172.21.15.31:0/4187183717 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5d00b20d0 con 0x7fc5d0159550 2023-10-06T02:23:26.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.508+0000 7fc5d6b9d700 1 -- 172.21.15.31:0/4187183717 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5d004c3c0 con 0x7fc5d0159550 2023-10-06T02:23:26.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.508+0000 7fc5c5ffb700 1 -- 172.21.15.31:0/4187183717 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc5cc011c10 con 0x7fc5d0159550 2023-10-06T02:23:26.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.509+0000 7fc5c5ffb700 1 --2- 172.21.15.31:0/4187183717 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc5b4066f30 0x7fc5b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:26.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.509+0000 7fc5c5ffb700 1 -- 172.21.15.31:0/4187183717 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc5cc085f80 con 0x7fc5d0159550 2023-10-06T02:23:26.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.510+0000 7fc5d539a700 1 --2- 172.21.15.31:0/4187183717 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc5b4066f30 0x7fc5b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:26.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.510+0000 7fc5d539a700 1 --2- 172.21.15.31:0/4187183717 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc5b4066f30 0x7fc5b40693e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc5d009be60 tx=0x7fc5c0007ec0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:26.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.514+0000 7fc5c5ffb700 1 -- 172.21.15.31:0/4187183717 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc5cc052ff0 con 0x7fc5d0159550 2023-10-06T02:23:26.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.713+0000 7fc5d6b9d700 1 -- 172.21.15.31:0/4187183717 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc5d006e7c0 con 0x7fc5d0159550 2023-10-06T02:23:26.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.714+0000 7fc5c5ffb700 1 -- 172.21.15.31:0/4187183717 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc5cc056610 con 0x7fc5d0159550 2023-10-06T02:23:26.714 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-10-06T02:23:26.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.716+0000 7fc5d6b9d700 1 -- 172.21.15.31:0/4187183717 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc5b4066f30 msgr2=0x7fc5b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:26.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.716+0000 7fc5d6b9d700 1 --2- 172.21.15.31:0/4187183717 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc5b4066f30 0x7fc5b40693e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc5d009be60 tx=0x7fc5c0007ec0 comp rx=0 tx=0).stop 2023-10-06T02:23:26.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.716+0000 7fc5d6b9d700 1 -- 172.21.15.31:0/4187183717 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5d0159550 msgr2=0x7fc5d0152110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:26.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.716+0000 7fc5d6b9d700 1 --2- 172.21.15.31:0/4187183717 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5d0159550 0x7fc5d0152110 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc5d006ad70 tx=0x7fc5cc017000 comp rx=0 tx=0).stop 2023-10-06T02:23:26.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.716+0000 7fc5d6b9d700 1 -- 172.21.15.31:0/4187183717 shutdown_connections 2023-10-06T02:23:26.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.716+0000 7fc5d6b9d700 1 --2- 172.21.15.31:0/4187183717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc5d015dfd0 0x7fc5d00cda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:26.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.716+0000 7fc5d6b9d700 1 --2- 172.21.15.31:0/4187183717 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5d0159550 0x7fc5d0152110 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:26.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.716+0000 7fc5d6b9d700 1 --2- 172.21.15.31:0/4187183717 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc5b4066f30 0x7fc5b40693e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:26.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.716+0000 7fc5d6b9d700 1 --2- 172.21.15.31:0/4187183717 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc5d0159920 0x7fc5d0098d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:26.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.716+0000 7fc5d6b9d700 1 -- 172.21.15.31:0/4187183717 >> 172.21.15.31:0/4187183717 conn(0x7fc5d006ce80 msgr2=0x7fc5d00ad090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:26.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.716+0000 7fc5d6b9d700 1 -- 172.21.15.31:0/4187183717 shutdown_connections 2023-10-06T02:23:26.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.716+0000 7fc5d6b9d700 1 -- 172.21.15.31:0/4187183717 wait complete. 2023-10-06T02:23:26.734 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T02:23:26.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.922+0000 7f8f11b9d700 1 Processor -- start 2023-10-06T02:23:26.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.922+0000 7f8f11b9d700 1 -- start start 2023-10-06T02:23:26.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.923+0000 7f8f11b9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8f0c142c00 0x7f8f0c06b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:26.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.923+0000 7f8f11b9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8f0c0c7f90 0x7f8f0c06c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:26.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.924+0000 7f8f11b9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f0c0ca480 con 0x7f8f0c06bd20 2023-10-06T02:23:26.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.924+0000 7f8f11b9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f0c15e240 con 0x7f8f0c0c7f90 2023-10-06T02:23:26.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.924+0000 7f8f11b9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f0c06b350 con 0x7f8f0c142c00 2023-10-06T02:23:26.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.924+0000 7f8f1139c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8f0c0c7f90 0x7f8f0c06c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:26.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.924+0000 7f8f1139c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8f0c0c7f90 0x7f8f0c06c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37824/0 (socket says 172.21.15.31:37824) 2023-10-06T02:23:26.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.924+0000 7f8f1139c700 1 -- 172.21.15.31:0/1654116824 learned_addr learned my addr 172.21.15.31:0/1654116824 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:26.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.924+0000 7f8f10b9b700 1 --2- 172.21.15.31:0/1654116824 >> v2:172.21.15.62:3300/0 conn(0x7f8f0c142c00 0x7f8f0c06b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:26.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.924+0000 7f8f1139c700 1 -- 172.21.15.31:0/1654116824 >> v2:172.21.15.62:3300/0 conn(0x7f8f0c142c00 msgr2=0x7f8f0c06b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:26.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.924+0000 7f8f1139c700 1 --2- 172.21.15.31:0/1654116824 >> v2:172.21.15.62:3300/0 conn(0x7f8f0c142c00 0x7f8f0c06b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:26.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.924+0000 7f8f1139c700 1 -- 172.21.15.31:0/1654116824 >> v1:172.21.15.31:6789/0 conn(0x7f8f0c06bd20 legacy=0x7f8f0c0c7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:26.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.924+0000 7f8f1139c700 1 -- 172.21.15.31:0/1654116824 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f0c15df60 con 0x7f8f0c0c7f90 2023-10-06T02:23:26.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.924+0000 7f8f0b7fe700 1 -- 172.21.15.31:0/1654116824 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2024049654 0 0) 0x7f8f0c0ca480 con 0x7f8f0c06bd20 2023-10-06T02:23:26.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.925+0000 7f8f1139c700 1 --2- 172.21.15.31:0/1654116824 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f0c0c7f90 0x7f8f0c06c6a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c06f490 tx=0x7f8f04009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8295b8ec5d222f7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:26.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.925+0000 7f8f0b7fe700 1 -- 172.21.15.31:0/1654116824 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f0400e070 con 0x7f8f0c0c7f90 2023-10-06T02:23:26.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.925+0000 7f8f0b7fe700 1 -- 172.21.15.31:0/1654116824 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f0400ba20 con 0x7f8f0c0c7f90 2023-10-06T02:23:26.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.925+0000 7f8f0b7fe700 1 -- 172.21.15.31:0/1654116824 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f040092e0 con 0x7f8f0c0c7f90 2023-10-06T02:23:26.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.925+0000 7f8f11b9d700 1 -- 172.21.15.31:0/1654116824 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f0c0c7f90 msgr2=0x7f8f0c06c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:26.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.925+0000 7f8f11b9d700 1 --2- 172.21.15.31:0/1654116824 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f0c0c7f90 0x7f8f0c06c6a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c06f490 tx=0x7f8f04009840 comp rx=0 tx=0).stop 2023-10-06T02:23:26.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.925+0000 7f8f11b9d700 1 -- 172.21.15.31:0/1654116824 shutdown_connections 2023-10-06T02:23:26.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.925+0000 7f8f11b9d700 1 --2- 172.21.15.31:0/1654116824 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f0c0c7f90 0x7f8f0c06c6a0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:26.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.925+0000 7f8f11b9d700 1 --2- 172.21.15.31:0/1654116824 >> v2:172.21.15.62:3300/0 conn(0x7f8f0c142c00 0x7f8f0c06b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:26.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.925+0000 7f8f11b9d700 1 -- 172.21.15.31:0/1654116824 >> 172.21.15.31:0/1654116824 conn(0x7f8f0c0675a0 msgr2=0x7f8f0c0679a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:26.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.925+0000 7f8f11b9d700 1 -- 172.21.15.31:0/1654116824 shutdown_connections 2023-10-06T02:23:26.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.926+0000 7f8f11b9d700 1 -- 172.21.15.31:0/1654116824 wait complete. 2023-10-06T02:23:26.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.927+0000 7f8f11b9d700 1 Processor -- start 2023-10-06T02:23:26.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.927+0000 7f8f11b9d700 1 -- start start 2023-10-06T02:23:26.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.928+0000 7f8f11b9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8f0c06bd20 0x7f8f0c158eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:26.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.928+0000 7f8f11b9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f0c0c7f90 0x7f8f0c140ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:26.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.928+0000 7f8f11b9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f0c142c00 0x7f8f0c141000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:26.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.928+0000 7f8f11b9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f0c0b2810 con 0x7f8f0c142c00 2023-10-06T02:23:26.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.928+0000 7f8f11b9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f0c0b1c00 con 0x7f8f0c0c7f90 2023-10-06T02:23:26.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.928+0000 7f8f11b9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f0c0b1ee0 con 0x7f8f0c06bd20 2023-10-06T02:23:26.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.928+0000 7f8f1139c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f0c142c00 0x7f8f0c141000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:26.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.928+0000 7f8f1139c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f0c142c00 0x7f8f0c141000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37838/0 (socket says 172.21.15.31:37838) 2023-10-06T02:23:26.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.928+0000 7f8f1139c700 1 -- 172.21.15.31:0/2556991276 learned_addr learned my addr 172.21.15.31:0/2556991276 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:26.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.928+0000 7f8f10b9b700 1 --2- 172.21.15.31:0/2556991276 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8f0c06bd20 0x7f8f0c158eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:26.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.928+0000 7f8f0bfff700 1 --2- 172.21.15.31:0/2556991276 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f0c0c7f90 0x7f8f0c140ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:26.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.929+0000 7f8f1139c700 1 -- 172.21.15.31:0/2556991276 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8f0c06bd20 msgr2=0x7f8f0c158eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:26.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.929+0000 7f8f1139c700 1 --2- 172.21.15.31:0/2556991276 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8f0c06bd20 0x7f8f0c158eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:26.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.929+0000 7f8f1139c700 1 -- 172.21.15.31:0/2556991276 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f0c0c7f90 msgr2=0x7f8f0c140ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:26.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.929+0000 7f8f1139c700 1 --2- 172.21.15.31:0/2556991276 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f0c0c7f90 0x7f8f0c140ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:26.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.929+0000 7f8f1139c700 1 -- 172.21.15.31:0/2556991276 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f04009ae0 con 0x7f8f0c142c00 2023-10-06T02:23:26.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.929+0000 7f8f1139c700 1 --2- 172.21.15.31:0/2556991276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f0c142c00 0x7f8f0c141000 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8f0400b6c0 tx=0x7f8f04018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:26.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.929+0000 7f8f08ff9700 1 -- 172.21.15.31:0/2556991276 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f040079a0 con 0x7f8f0c142c00 2023-10-06T02:23:26.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.929+0000 7f8f08ff9700 1 -- 172.21.15.31:0/2556991276 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f04007b00 con 0x7f8f0c142c00 2023-10-06T02:23:26.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.929+0000 7f8f08ff9700 1 -- 172.21.15.31:0/2556991276 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f04026460 con 0x7f8f0c142c00 2023-10-06T02:23:26.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.929+0000 7f8f11b9d700 1 -- 172.21.15.31:0/2556991276 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f0c0b21c0 con 0x7f8f0c142c00 2023-10-06T02:23:26.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.929+0000 7f8f11b9d700 1 -- 172.21.15.31:0/2556991276 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f0c1593f0 con 0x7f8f0c142c00 2023-10-06T02:23:26.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.930+0000 7f8f11b9d700 1 -- 172.21.15.31:0/2556991276 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f0c04c3c0 con 0x7f8f0c142c00 2023-10-06T02:23:26.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.931+0000 7f8f08ff9700 1 -- 172.21.15.31:0/2556991276 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8f04026890 con 0x7f8f0c142c00 2023-10-06T02:23:26.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.933+0000 7f8f08ff9700 1 --2- 172.21.15.31:0/2556991276 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f8ef0066f30 0x7f8ef00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:26.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.933+0000 7f8f08ff9700 1 -- 172.21.15.31:0/2556991276 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8f04086410 con 0x7f8f0c142c00 2023-10-06T02:23:26.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.933+0000 7f8f10b9b700 1 --2- 172.21.15.31:0/2556991276 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f8ef0066f30 0x7f8ef00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:26.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.933+0000 7f8f10b9b700 1 --2- 172.21.15.31:0/2556991276 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f8ef0066f30 0x7f8ef00693e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8f00009d10 tx=0x7f8f000091a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:26.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:26.937+0000 7f8f08ff9700 1 -- 172.21.15.31:0/2556991276 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8f04053480 con 0x7f8f0c142c00 2023-10-06T02:23:27.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.144+0000 7f8f11b9d700 1 -- 172.21.15.31:0/2556991276 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f8f0c0b2950 con 0x7f8f0c142c00 2023-10-06T02:23:27.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.144+0000 7f8f08ff9700 1 -- 172.21.15.31:0/2556991276 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8f04056aa0 con 0x7f8f0c142c00 2023-10-06T02:23:27.144 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 8 2023-10-06T02:23:27.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.146+0000 7f8f11b9d700 1 -- 172.21.15.31:0/2556991276 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f8ef0066f30 msgr2=0x7f8ef00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:27.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.146+0000 7f8f11b9d700 1 --2- 172.21.15.31:0/2556991276 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f8ef0066f30 0x7f8ef00693e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8f00009d10 tx=0x7f8f000091a0 comp rx=0 tx=0).stop 2023-10-06T02:23:27.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.146+0000 7f8f11b9d700 1 -- 172.21.15.31:0/2556991276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f0c142c00 msgr2=0x7f8f0c141000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:27.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.146+0000 7f8f11b9d700 1 --2- 172.21.15.31:0/2556991276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f0c142c00 0x7f8f0c141000 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8f0400b6c0 tx=0x7f8f04018000 comp rx=0 tx=0).stop 2023-10-06T02:23:27.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.146+0000 7f8f11b9d700 1 -- 172.21.15.31:0/2556991276 shutdown_connections 2023-10-06T02:23:27.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.147+0000 7f8f11b9d700 1 --2- 172.21.15.31:0/2556991276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f0c142c00 0x7f8f0c141000 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:27.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.147+0000 7f8f11b9d700 1 --2- 172.21.15.31:0/2556991276 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f8ef0066f30 0x7f8ef00693e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:27.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.147+0000 7f8f11b9d700 1 --2- 172.21.15.31:0/2556991276 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8f0c06bd20 0x7f8f0c158eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:27.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.147+0000 7f8f11b9d700 1 --2- 172.21.15.31:0/2556991276 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f0c0c7f90 0x7f8f0c140ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:27.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.147+0000 7f8f11b9d700 1 -- 172.21.15.31:0/2556991276 >> 172.21.15.31:0/2556991276 conn(0x7f8f0c0675a0 msgr2=0x7f8f0c06c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:27.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.147+0000 7f8f11b9d700 1 -- 172.21.15.31:0/2556991276 shutdown_connections 2023-10-06T02:23:27.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.147+0000 7f8f11b9d700 1 -- 172.21.15.31:0/2556991276 wait complete. 2023-10-06T02:23:27.165 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T02:23:27.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.354+0000 7f7270d9d700 1 Processor -- start 2023-10-06T02:23:27.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.354+0000 7f7270d9d700 1 -- start start 2023-10-06T02:23:27.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.356+0000 7f7270d9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f726c159550 0x7f726c0ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:27.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.356+0000 7f7270d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f726c15dfd0 0x7f726c0aedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:27.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.356+0000 7f7270d9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f726c0acc00 con 0x7f726c159920 2023-10-06T02:23:27.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.356+0000 7f7270d9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f726c0abff0 con 0x7f726c159550 2023-10-06T02:23:27.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.356+0000 7f7270d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f726c0ac2d0 con 0x7f726c15dfd0 2023-10-06T02:23:27.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.356+0000 7f726b7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f726c159550 0x7f726c0ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:27.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.356+0000 7f726b7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f726c159550 0x7f726c0ceac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37852/0 (socket says 172.21.15.31:37852) 2023-10-06T02:23:27.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.356+0000 7f726b7fe700 1 -- 172.21.15.31:0/1378436373 learned_addr learned my addr 172.21.15.31:0/1378436373 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:27.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.356+0000 7f726bfff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f726c159920 0x7f726c0ad090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49374/0 (socket says 172.21.15.31:49374) 2023-10-06T02:23:27.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.356+0000 7f726affd700 1 --2- 172.21.15.31:0/1378436373 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f726c15dfd0 0x7f726c0aedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:27.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.356+0000 7f726b7fe700 1 -- 172.21.15.31:0/1378436373 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f726c15dfd0 msgr2=0x7f726c0aedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:27.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.356+0000 7f726b7fe700 1 --2- 172.21.15.31:0/1378436373 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f726c15dfd0 0x7f726c0aedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:27.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.356+0000 7f726b7fe700 1 -- 172.21.15.31:0/1378436373 >> v1:172.21.15.31:6789/0 conn(0x7f726c159920 legacy=0x7f726c0ad090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:27.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.356+0000 7f726b7fe700 1 -- 172.21.15.31:0/1378436373 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f726c0ac5b0 con 0x7f726c159550 2023-10-06T02:23:27.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.356+0000 7f726a7fc700 1 -- 172.21.15.31:0/1378436373 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4110892864 0 0) 0x7f726c0acc00 con 0x7f726c159920 2023-10-06T02:23:27.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.357+0000 7f726b7fe700 1 --2- 172.21.15.31:0/1378436373 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f726c159550 0x7f726c0ceac0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f725c0099b0 tx=0x7f725c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9be1ce809bf41fbb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:27.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.357+0000 7f726a7fc700 1 -- 172.21.15.31:0/1378436373 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725c009ea0 con 0x7f726c159550 2023-10-06T02:23:27.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.357+0000 7f726a7fc700 1 -- 172.21.15.31:0/1378436373 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f725c00b860 con 0x7f726c159550 2023-10-06T02:23:27.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.357+0000 7f726a7fc700 1 -- 172.21.15.31:0/1378436373 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725c00bc00 con 0x7f726c159550 2023-10-06T02:23:27.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.357+0000 7f7270d9d700 1 -- 172.21.15.31:0/1378436373 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f726c159550 msgr2=0x7f726c0ceac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:27.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.357+0000 7f7270d9d700 1 --2- 172.21.15.31:0/1378436373 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f726c159550 0x7f726c0ceac0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f725c0099b0 tx=0x7f725c009250 comp rx=0 tx=0).stop 2023-10-06T02:23:27.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.358+0000 7f7270d9d700 1 -- 172.21.15.31:0/1378436373 shutdown_connections 2023-10-06T02:23:27.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.358+0000 7f7270d9d700 1 --2- 172.21.15.31:0/1378436373 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f726c159550 0x7f726c0ceac0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:27.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.358+0000 7f7270d9d700 1 --2- 172.21.15.31:0/1378436373 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f726c15dfd0 0x7f726c0aedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:27.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.358+0000 7f7270d9d700 1 -- 172.21.15.31:0/1378436373 >> 172.21.15.31:0/1378436373 conn(0x7f726c06ce80 msgr2=0x7f726c0ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:27.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.358+0000 7f7270d9d700 1 -- 172.21.15.31:0/1378436373 shutdown_connections 2023-10-06T02:23:27.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.358+0000 7f7270d9d700 1 -- 172.21.15.31:0/1378436373 wait complete. 2023-10-06T02:23:27.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.359+0000 7f7270d9d700 1 Processor -- start 2023-10-06T02:23:27.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.359+0000 7f7270d9d700 1 -- start start 2023-10-06T02:23:27.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.360+0000 7f7270d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f726c159550 0x7f726c151d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:27.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.360+0000 7f7270d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f726c159920 0x7f726c0b22d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:27.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.360+0000 7f7270d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f726c15dfd0 0x7f726c0c9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:27.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.360+0000 7f7270d9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f726c0c9a30 con 0x7f726c159550 2023-10-06T02:23:27.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.360+0000 7f7270d9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f726c0b3330 con 0x7f726c159920 2023-10-06T02:23:27.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.360+0000 7f7270d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f726c0b2d70 con 0x7f726c15dfd0 2023-10-06T02:23:27.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.360+0000 7f726b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f726c159550 0x7f726c151d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:27.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.360+0000 7f726b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f726c159550 0x7f726c151d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37858/0 (socket says 172.21.15.31:37858) 2023-10-06T02:23:27.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.360+0000 7f726b7fe700 1 -- 172.21.15.31:0/2603579265 learned_addr learned my addr 172.21.15.31:0/2603579265 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:27.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.361+0000 7f726b7fe700 1 -- 172.21.15.31:0/2603579265 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f726c15dfd0 msgr2=0x7f726c0c9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:27.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.361+0000 7f726b7fe700 1 --2- 172.21.15.31:0/2603579265 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f726c15dfd0 0x7f726c0c9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:27.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.361+0000 7f726b7fe700 1 -- 172.21.15.31:0/2603579265 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f726c159920 msgr2=0x7f726c0b22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:27.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.361+0000 7f726b7fe700 1 --2- 172.21.15.31:0/2603579265 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f726c159920 0x7f726c0b22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:27.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.361+0000 7f726b7fe700 1 -- 172.21.15.31:0/2603579265 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f725c0094f0 con 0x7f726c159550 2023-10-06T02:23:27.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.361+0000 7f726b7fe700 1 --2- 172.21.15.31:0/2603579265 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f726c159550 0x7f726c151d50 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f726c06ad70 tx=0x7f725c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:27.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.361+0000 7f72537fe700 1 -- 172.21.15.31:0/2603579265 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725c00bbe0 con 0x7f726c159550 2023-10-06T02:23:27.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.361+0000 7f72537fe700 1 -- 172.21.15.31:0/2603579265 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f725c011420 con 0x7f726c159550 2023-10-06T02:23:27.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.361+0000 7f72537fe700 1 -- 172.21.15.31:0/2603579265 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725c0117e0 con 0x7f726c159550 2023-10-06T02:23:27.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.361+0000 7f7270d9d700 1 -- 172.21.15.31:0/2603579265 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f726c0b3050 con 0x7f726c159550 2023-10-06T02:23:27.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.361+0000 7f7270d9d700 1 -- 172.21.15.31:0/2603579265 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f726c0c7760 con 0x7f726c159550 2023-10-06T02:23:27.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.363+0000 7f7270d9d700 1 -- 172.21.15.31:0/2603579265 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f726c04c3c0 con 0x7f726c159550 2023-10-06T02:23:27.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.363+0000 7f72537fe700 1 -- 172.21.15.31:0/2603579265 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f725c011c10 con 0x7f726c159550 2023-10-06T02:23:27.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.364+0000 7f72537fe700 1 --2- 172.21.15.31:0/2603579265 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f7258066f30 0x7f72580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:27.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.364+0000 7f72537fe700 1 -- 172.21.15.31:0/2603579265 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f725c0860a0 con 0x7f726c159550 2023-10-06T02:23:27.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.364+0000 7f726affd700 1 --2- 172.21.15.31:0/2603579265 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f7258066f30 0x7f72580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:27.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.365+0000 7f726affd700 1 --2- 172.21.15.31:0/2603579265 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f7258066f30 0x7f72580693e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f726c0b39f0 tx=0x7f72600091a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:27.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.368+0000 7f72537fe700 1 -- 172.21.15.31:0/2603579265 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f725c053110 con 0x7f726c159550 2023-10-06T02:23:27.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.572+0000 7f7270d9d700 1 -- 172.21.15.31:0/2603579265 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f726c158d80 con 0x7f726c159550 2023-10-06T02:23:27.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.572+0000 7f72537fe700 1 -- 172.21.15.31:0/2603579265 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f725c056730 con 0x7f726c159550 2023-10-06T02:23:27.573 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-10-06T02:23:27.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.575+0000 7f7270d9d700 1 -- 172.21.15.31:0/2603579265 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f7258066f30 msgr2=0x7f72580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:27.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.575+0000 7f7270d9d700 1 --2- 172.21.15.31:0/2603579265 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f7258066f30 0x7f72580693e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f726c0b39f0 tx=0x7f72600091a0 comp rx=0 tx=0).stop 2023-10-06T02:23:27.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.575+0000 7f7270d9d700 1 -- 172.21.15.31:0/2603579265 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f726c159550 msgr2=0x7f726c151d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:27.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.575+0000 7f7270d9d700 1 --2- 172.21.15.31:0/2603579265 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f726c159550 0x7f726c151d50 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f726c06ad70 tx=0x7f725c017000 comp rx=0 tx=0).stop 2023-10-06T02:23:27.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.575+0000 7f7270d9d700 1 -- 172.21.15.31:0/2603579265 shutdown_connections 2023-10-06T02:23:27.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.575+0000 7f7270d9d700 1 --2- 172.21.15.31:0/2603579265 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f7258066f30 0x7f72580693e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:27.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.575+0000 7f7270d9d700 1 --2- 172.21.15.31:0/2603579265 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f726c15dfd0 0x7f726c0c9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:27.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.575+0000 7f7270d9d700 1 --2- 172.21.15.31:0/2603579265 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f726c159550 0x7f726c151d50 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:27.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.575+0000 7f7270d9d700 1 --2- 172.21.15.31:0/2603579265 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f726c159920 0x7f726c0b22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:27.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.575+0000 7f7270d9d700 1 -- 172.21.15.31:0/2603579265 >> 172.21.15.31:0/2603579265 conn(0x7f726c06ce80 msgr2=0x7f726c06e040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:27.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.575+0000 7f7270d9d700 1 -- 172.21.15.31:0/2603579265 shutdown_connections 2023-10-06T02:23:27.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.575+0000 7f7270d9d700 1 -- 172.21.15.31:0/2603579265 wait complete. 2023-10-06T02:23:27.592 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T02:23:27.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.783+0000 7f1ed659d700 1 Processor -- start 2023-10-06T02:23:27.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.783+0000 7f1ed659d700 1 -- start start 2023-10-06T02:23:27.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.784+0000 7f1ed659d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f1ed0153c90 0x7f1ed015a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:27.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.784+0000 7f1ed659d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1ed006db20 0x7f1ed00ca7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:27.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.784+0000 7f1ed659d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ed015ada0 con 0x7f1ed0154060 2023-10-06T02:23:27.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.784+0000 7f1ed659d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ed006ca80 con 0x7f1ed0153c90 2023-10-06T02:23:27.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.784+0000 7f1ed659d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed006cd60 con 0x7f1ed006db20 2023-10-06T02:23:27.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.784+0000 7f1ed5d9c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f1ed0154060 0x7f1ed006a2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49376/0 (socket says 172.21.15.31:49376) 2023-10-06T02:23:27.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.784+0000 7f1ed5d9c700 1 -- 172.21.15.31:0/737715688 learned_addr learned my addr 172.21.15.31:0/737715688 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:23:27.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.784+0000 7f1ed559b700 1 --2- 172.21.15.31:0/737715688 >> v2:172.21.15.62:3300/0 conn(0x7f1ed0153c90 0x7f1ed015a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:27.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.784+0000 7f1ed4d9a700 1 --2- 172.21.15.31:0/737715688 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1ed006db20 0x7f1ed00ca7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:27.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.785+0000 7f1ec7fff700 1 -- 172.21.15.31:0/737715688 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1608480728 0 0) 0x7f1ec8001490 con 0x7f1ed0154060 2023-10-06T02:23:27.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.785+0000 7f1ec7fff700 1 -- 172.21.15.31:0/737715688 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ebc003570 con 0x7f1ed0154060 2023-10-06T02:23:27.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.785+0000 7f1ed4d9a700 1 -- 172.21.15.31:0/737715688 >> v2:172.21.15.62:3300/0 conn(0x7f1ed0153c90 msgr2=0x7f1ed015a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:27.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.785+0000 7f1ed4d9a700 1 --2- 172.21.15.31:0/737715688 >> v2:172.21.15.62:3300/0 conn(0x7f1ed0153c90 0x7f1ed015a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:27.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.785+0000 7f1ed4d9a700 1 -- 172.21.15.31:0/737715688 >> v1:172.21.15.31:6789/0 conn(0x7f1ed0154060 legacy=0x7f1ed006a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:27.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.785+0000 7f1ed4d9a700 1 -- 172.21.15.31:0/737715688 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ed006d040 con 0x7f1ed006db20 2023-10-06T02:23:27.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.785+0000 7f1ed559b700 1 --2- 172.21.15.31:0/737715688 >> v2:172.21.15.62:3300/0 conn(0x7f1ed0153c90 0x7f1ed015a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:23:27.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.785+0000 7f1ed4d9a700 1 --2- 172.21.15.31:0/737715688 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1ed006db20 0x7f1ed00ca7b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1ec00098e0 tx=0x7f1ec00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95ad761707944cb9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:27.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.786+0000 7f1ec7fff700 1 -- 172.21.15.31:0/737715688 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec0011630 con 0x7f1ed006db20 2023-10-06T02:23:27.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.786+0000 7f1ec7fff700 1 -- 172.21.15.31:0/737715688 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ec0011790 con 0x7f1ed006db20 2023-10-06T02:23:27.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.786+0000 7f1ec7fff700 1 -- 172.21.15.31:0/737715688 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec0011b30 con 0x7f1ed006db20 2023-10-06T02:23:27.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.786+0000 7f1ed659d700 1 -- 172.21.15.31:0/737715688 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1ed006db20 msgr2=0x7f1ed00ca7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:27.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.786+0000 7f1ed659d700 1 --2- 172.21.15.31:0/737715688 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1ed006db20 0x7f1ed00ca7b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1ec00098e0 tx=0x7f1ec00091a0 comp rx=0 tx=0).stop 2023-10-06T02:23:27.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.786+0000 7f1ed659d700 1 -- 172.21.15.31:0/737715688 shutdown_connections 2023-10-06T02:23:27.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.786+0000 7f1ed659d700 1 --2- 172.21.15.31:0/737715688 >> v2:172.21.15.62:3300/0 conn(0x7f1ed0153c90 0x7f1ed015a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:27.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.786+0000 7f1ed659d700 1 --2- 172.21.15.31:0/737715688 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1ed006db20 0x7f1ed00ca7b0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:27.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.786+0000 7f1ed659d700 1 -- 172.21.15.31:0/737715688 >> 172.21.15.31:0/737715688 conn(0x7f1ed006e720 msgr2=0x7f1ed006eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:27.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.786+0000 7f1ed659d700 1 -- 172.21.15.31:0/737715688 shutdown_connections 2023-10-06T02:23:27.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.786+0000 7f1ed659d700 1 -- 172.21.15.31:0/737715688 wait complete. 2023-10-06T02:23:27.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.788+0000 7f1ed659d700 1 Processor -- start 2023-10-06T02:23:27.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.788+0000 7f1ed659d700 1 -- start start 2023-10-06T02:23:27.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.789+0000 7f1ed659d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ed006db20 0x7f1ed00b2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:27.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.789+0000 7f1ed659d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1ed0153c90 0x7f1ed00c6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:27.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.789+0000 7f1ed659d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ed0154060 0x7f1ed00c6640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:27.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.789+0000 7f1ed659d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed00c7c70 con 0x7f1ed0154060 2023-10-06T02:23:27.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.789+0000 7f1ed659d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed00c6b80 con 0x7f1ed006db20 2023-10-06T02:23:27.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.789+0000 7f1ed659d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed00c4ca0 con 0x7f1ed0153c90 2023-10-06T02:23:27.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.789+0000 7f1ed5d9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ed0154060 0x7f1ed00c6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:27.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.789+0000 7f1ed5d9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ed0154060 0x7f1ed00c6640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37864/0 (socket says 172.21.15.31:37864) 2023-10-06T02:23:27.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.789+0000 7f1ed5d9c700 1 -- 172.21.15.31:0/2361951121 learned_addr learned my addr 172.21.15.31:0/2361951121 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:27.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.789+0000 7f1ed4d9a700 1 --2- 172.21.15.31:0/2361951121 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1ed0153c90 0x7f1ed00c6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:27.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.789+0000 7f1ed559b700 1 --2- 172.21.15.31:0/2361951121 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ed006db20 0x7f1ed00b2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:27.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.789+0000 7f1ed5d9c700 1 -- 172.21.15.31:0/2361951121 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1ed0153c90 msgr2=0x7f1ed00c6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:27.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.789+0000 7f1ed5d9c700 1 --2- 172.21.15.31:0/2361951121 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1ed0153c90 0x7f1ed00c6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:27.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.789+0000 7f1ed5d9c700 1 -- 172.21.15.31:0/2361951121 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ed006db20 msgr2=0x7f1ed00b2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:27.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.789+0000 7f1ed5d9c700 1 --2- 172.21.15.31:0/2361951121 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ed006db20 0x7f1ed00b2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:27.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.789+0000 7f1ed5d9c700 1 -- 172.21.15.31:0/2361951121 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ecc009420 con 0x7f1ed0154060 2023-10-06T02:23:27.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.790+0000 7f1ed5d9c700 1 --2- 172.21.15.31:0/2361951121 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ed0154060 0x7f1ed00c6640 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1ec80015f0 tx=0x7f1ec8001ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:27.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.790+0000 7f1ec57fa700 1 -- 172.21.15.31:0/2361951121 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec800a8b0 con 0x7f1ed0154060 2023-10-06T02:23:27.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.790+0000 7f1ec57fa700 1 -- 172.21.15.31:0/2361951121 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ec800aa10 con 0x7f1ed0154060 2023-10-06T02:23:27.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.790+0000 7f1ed659d700 1 -- 172.21.15.31:0/2361951121 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ec0009420 con 0x7f1ed0154060 2023-10-06T02:23:27.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.790+0000 7f1ec57fa700 1 -- 172.21.15.31:0/2361951121 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec800ad90 con 0x7f1ed0154060 2023-10-06T02:23:27.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.790+0000 7f1ed659d700 1 -- 172.21.15.31:0/2361951121 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ed00c49c0 con 0x7f1ed0154060 2023-10-06T02:23:27.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.791+0000 7f1ed659d700 1 -- 172.21.15.31:0/2361951121 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ed006f240 con 0x7f1ed0154060 2023-10-06T02:23:27.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.792+0000 7f1ec57fa700 1 -- 172.21.15.31:0/2361951121 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1ec8007300 con 0x7f1ed0154060 2023-10-06T02:23:27.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.793+0000 7f1ec57fa700 1 --2- 172.21.15.31:0/2361951121 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f1eb4066f30 0x7f1eb40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:27.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.793+0000 7f1ec57fa700 1 -- 172.21.15.31:0/2361951121 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1ec8084880 con 0x7f1ed0154060 2023-10-06T02:23:27.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.793+0000 7f1ed559b700 1 --2- 172.21.15.31:0/2361951121 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f1eb4066f30 0x7f1eb40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:27.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.793+0000 7f1ed559b700 1 --2- 172.21.15.31:0/2361951121 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f1eb4066f30 0x7f1eb40693e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1ed0167c30 tx=0x7f1ecc0091a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:27.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.797+0000 7f1ec57fa700 1 -- 172.21.15.31:0/2361951121 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1ec8051970 con 0x7f1ed0154060 2023-10-06T02:23:27.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.997+0000 7f1ed659d700 1 -- 172.21.15.31:0/2361951121 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1ed015ada0 con 0x7f1ed0154060 2023-10-06T02:23:27.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:27.997+0000 7f1ec57fa700 1 -- 172.21.15.31:0/2361951121 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1ed015ada0 con 0x7f1ed0154060 2023-10-06T02:23:27.998 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-10-06T02:23:28.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.000+0000 7f1ed659d700 1 -- 172.21.15.31:0/2361951121 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f1eb4066f30 msgr2=0x7f1eb40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:28.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.000+0000 7f1ed659d700 1 --2- 172.21.15.31:0/2361951121 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f1eb4066f30 0x7f1eb40693e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1ed0167c30 tx=0x7f1ecc0091a0 comp rx=0 tx=0).stop 2023-10-06T02:23:28.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.000+0000 7f1ed659d700 1 -- 172.21.15.31:0/2361951121 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ed0154060 msgr2=0x7f1ed00c6640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:28.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.000+0000 7f1ed659d700 1 --2- 172.21.15.31:0/2361951121 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ed0154060 0x7f1ed00c6640 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1ec80015f0 tx=0x7f1ec8001ab0 comp rx=0 tx=0).stop 2023-10-06T02:23:28.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.000+0000 7f1ed659d700 1 -- 172.21.15.31:0/2361951121 shutdown_connections 2023-10-06T02:23:28.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.000+0000 7f1ed659d700 1 --2- 172.21.15.31:0/2361951121 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ed0154060 0x7f1ed00c6640 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:28.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.000+0000 7f1ed659d700 1 --2- 172.21.15.31:0/2361951121 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ed006db20 0x7f1ed00b2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:28.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.000+0000 7f1ed659d700 1 --2- 172.21.15.31:0/2361951121 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f1eb4066f30 0x7f1eb40693e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:28.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.000+0000 7f1ed659d700 1 --2- 172.21.15.31:0/2361951121 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1ed0153c90 0x7f1ed00c6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:28.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.000+0000 7f1ed659d700 1 -- 172.21.15.31:0/2361951121 >> 172.21.15.31:0/2361951121 conn(0x7f1ed006e720 msgr2=0x7f1ed015cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:28.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.000+0000 7f1ed659d700 1 -- 172.21.15.31:0/2361951121 shutdown_connections 2023-10-06T02:23:28.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.000+0000 7f1ed659d700 1 -- 172.21.15.31:0/2361951121 wait complete. 2023-10-06T02:23:28.018 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:23:28.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.205+0000 7fe8cddff700 1 Processor -- start 2023-10-06T02:23:28.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.205+0000 7fe8cddff700 1 -- start start 2023-10-06T02:23:28.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.206+0000 7fe8cddff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe8c80cb5a0 0x7fe8c80cb970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:28.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.206+0000 7fe8cddff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8c815ae40 con 0x7fe8c80cd9e0 2023-10-06T02:23:28.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.206+0000 7fe8cddff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8c815ab60 con 0x7fe8c815a5d0 2023-10-06T02:23:28.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.206+0000 7fe8cddff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8c80d13f0 con 0x7fe8c80cb5a0 2023-10-06T02:23:28.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.206+0000 7fe8c77fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe8c80cb5a0 0x7fe8c80cb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:28.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.206+0000 7fe8c6ffd700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fe8c80cd9e0 0x7fe8c8159ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:41750/0 (socket says 172.21.15.31:41750) 2023-10-06T02:23:28.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.206+0000 7fe8c6ffd700 1 -- 172.21.15.31:0/3597642803 learned_addr learned my addr 172.21.15.31:0/3597642803 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:23:28.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.206+0000 7fe8c77fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe8c80cb5a0 0x7fe8c80cb970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58766/0 (socket says 172.21.15.31:58766) 2023-10-06T02:23:28.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.207+0000 7fe8c67fc700 1 -- 172.21.15.31:0/3597642803 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2072705688 0 0) 0x7fe8c815ae40 con 0x7fe8c80cd9e0 2023-10-06T02:23:28.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.207+0000 7fe8c77fe700 1 -- 172.21.15.31:0/3597642803 >> v1:172.21.15.62:6789/0 conn(0x7fe8c815a5d0 legacy=0x7fe8c8156e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:28.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.207+0000 7fe8c77fe700 1 -- 172.21.15.31:0/3597642803 >> v1:172.21.15.31:6789/0 conn(0x7fe8c80cd9e0 legacy=0x7fe8c8159ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:28.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.207+0000 7fe8c77fe700 1 -- 172.21.15.31:0/3597642803 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8c80ccf00 con 0x7fe8c80cb5a0 2023-10-06T02:23:28.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.207+0000 7fe8c77fe700 1 --2- 172.21.15.31:0/3597642803 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8c80cb5a0 0x7fe8c80cb970 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8009910 tx=0x7fe8b80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f8cb06dbcf4c2d1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:28.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.207+0000 7fe8c67fc700 1 -- 172.21.15.31:0/3597642803 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b800b820 con 0x7fe8c80cb5a0 2023-10-06T02:23:28.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.207+0000 7fe8c67fc700 1 -- 172.21.15.31:0/3597642803 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8b800b980 con 0x7fe8c80cb5a0 2023-10-06T02:23:28.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.207+0000 7fe8c67fc700 1 -- 172.21.15.31:0/3597642803 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b800bd20 con 0x7fe8c80cb5a0 2023-10-06T02:23:28.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.208+0000 7fe8cddff700 1 -- 172.21.15.31:0/3597642803 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8c80cb5a0 msgr2=0x7fe8c80cb970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:28.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.208+0000 7fe8cddff700 1 --2- 172.21.15.31:0/3597642803 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8c80cb5a0 0x7fe8c80cb970 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8009910 tx=0x7fe8b80091a0 comp rx=0 tx=0).stop 2023-10-06T02:23:28.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.208+0000 7fe8cddff700 1 -- 172.21.15.31:0/3597642803 shutdown_connections 2023-10-06T02:23:28.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.208+0000 7fe8cddff700 1 --2- 172.21.15.31:0/3597642803 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8c80cb5a0 0x7fe8c80cb970 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:28.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.208+0000 7fe8cddff700 1 -- 172.21.15.31:0/3597642803 >> 172.21.15.31:0/3597642803 conn(0x7fe8c8070e30 msgr2=0x7fe8c80d0710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:28.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.208+0000 7fe8cddff700 1 -- 172.21.15.31:0/3597642803 shutdown_connections 2023-10-06T02:23:28.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.208+0000 7fe8cddff700 1 -- 172.21.15.31:0/3597642803 wait complete. 2023-10-06T02:23:28.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.209+0000 7fe8cddff700 1 Processor -- start 2023-10-06T02:23:28.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.210+0000 7fe8cddff700 1 -- start start 2023-10-06T02:23:28.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.210+0000 7fe8cddff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8c80cb5a0 0x7fe8c814fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:28.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.210+0000 7fe8cddff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8c80cd9e0 0x7fe8c80c9e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:28.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.210+0000 7fe8cddff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe8c815a5d0 0x7fe8c80c7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:28.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.210+0000 7fe8cddff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c81513c0 con 0x7fe8c80cb5a0 2023-10-06T02:23:28.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.210+0000 7fe8cddff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c81521c0 con 0x7fe8c80cd9e0 2023-10-06T02:23:28.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.210+0000 7fe8cddff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c81515b0 con 0x7fe8c815a5d0 2023-10-06T02:23:28.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.211+0000 7fe8c77fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8c80cb5a0 0x7fe8c814fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:28.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.211+0000 7fe8c77fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8c80cb5a0 0x7fe8c814fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58778/0 (socket says 172.21.15.31:58778) 2023-10-06T02:23:28.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.211+0000 7fe8c77fe700 1 -- 172.21.15.31:0/78686996 learned_addr learned my addr 172.21.15.31:0/78686996 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:28.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.211+0000 7fe8c7fff700 1 --2- 172.21.15.31:0/78686996 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe8c815a5d0 0x7fe8c80c7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:28.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.211+0000 7fe8c6ffd700 1 --2- 172.21.15.31:0/78686996 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8c80cd9e0 0x7fe8c80c9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:28.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.211+0000 7fe8c77fe700 1 -- 172.21.15.31:0/78686996 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe8c815a5d0 msgr2=0x7fe8c80c7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:28.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.211+0000 7fe8c77fe700 1 --2- 172.21.15.31:0/78686996 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe8c815a5d0 0x7fe8c80c7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:28.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.211+0000 7fe8c77fe700 1 -- 172.21.15.31:0/78686996 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8c80cd9e0 msgr2=0x7fe8c80c9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:28.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.211+0000 7fe8c77fe700 1 --2- 172.21.15.31:0/78686996 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8c80cd9e0 0x7fe8c80c9e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:28.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.211+0000 7fe8c77fe700 1 -- 172.21.15.31:0/78686996 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8b8009420 con 0x7fe8c80cb5a0 2023-10-06T02:23:28.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.211+0000 7fe8c77fe700 1 --2- 172.21.15.31:0/78686996 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8c80cb5a0 0x7fe8c814fab0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe8b80093f0 tx=0x7fe8b8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:28.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.212+0000 7fe8af7fe700 1 -- 172.21.15.31:0/78686996 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b801f070 con 0x7fe8c80cb5a0 2023-10-06T02:23:28.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.212+0000 7fe8af7fe700 1 -- 172.21.15.31:0/78686996 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8b8002f70 con 0x7fe8c80cb5a0 2023-10-06T02:23:28.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.212+0000 7fe8af7fe700 1 -- 172.21.15.31:0/78686996 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b8025420 con 0x7fe8c80cb5a0 2023-10-06T02:23:28.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.212+0000 7fe8cddff700 1 -- 172.21.15.31:0/78686996 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8c8151890 con 0x7fe8c80cb5a0 2023-10-06T02:23:28.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.212+0000 7fe8cddff700 1 -- 172.21.15.31:0/78686996 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8c8151e50 con 0x7fe8c80cb5a0 2023-10-06T02:23:28.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.213+0000 7fe8cddff700 1 -- 172.21.15.31:0/78686996 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8c804c3c0 con 0x7fe8c80cb5a0 2023-10-06T02:23:28.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.214+0000 7fe8af7fe700 1 -- 172.21.15.31:0/78686996 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe8b8023070 con 0x7fe8c80cb5a0 2023-10-06T02:23:28.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.215+0000 7fe8af7fe700 1 --2- 172.21.15.31:0/78686996 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe8b4066f30 0x7fe8b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:28.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.215+0000 7fe8af7fe700 1 -- 172.21.15.31:0/78686996 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe8b8085c00 con 0x7fe8c80cb5a0 2023-10-06T02:23:28.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.215+0000 7fe8c6ffd700 1 --2- 172.21.15.31:0/78686996 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe8b4066f30 0x7fe8b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:28.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.215+0000 7fe8c6ffd700 1 --2- 172.21.15.31:0/78686996 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe8b4066f30 0x7fe8b40693e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc007fd0 tx=0x7fe8bc007d10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:28.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.219+0000 7fe8af7fe700 1 -- 172.21.15.31:0/78686996 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe8b808c080 con 0x7fe8c80cb5a0 2023-10-06T02:23:28.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.463+0000 7fe8cddff700 1 -- 172.21.15.31:0/78686996 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe8c80cd1e0 con 0x7fe8c80cb5a0 2023-10-06T02:23:28.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.464+0000 7fe8af7fe700 1 -- 172.21.15.31:0/78686996 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4591 (secure 0 0 0) 0x7fe8b8015090 con 0x7fe8c80cb5a0 2023-10-06T02:23:28.464 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:23:28.464 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/1808327183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":1808327183},{"type":"v1","addr":"172.21.15.31:6837","nonce":1808327183}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/4263144517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":4263144517},{"type":"v1","addr":"172.21.15.62:6834","nonce":4263144517}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1489208568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":1489208568},{"type":"v1","addr":"172.21.15.62:6835","nonce":1489208568}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:23:09.521236+0000","modified":"2023-10-06T02:23:13.047274+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6836/3809284664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3809284664},{"type":"v1","addr":"172.21.15.31:6836","nonce":3809284664}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T02:23:28.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.466+0000 7fe8cddff700 1 -- 172.21.15.31:0/78686996 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe8b4066f30 msgr2=0x7fe8b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:28.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.466+0000 7fe8cddff700 1 --2- 172.21.15.31:0/78686996 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe8b4066f30 0x7fe8b40693e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc007fd0 tx=0x7fe8bc007d10 comp rx=0 tx=0).stop 2023-10-06T02:23:28.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.466+0000 7fe8cddff700 1 -- 172.21.15.31:0/78686996 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8c80cb5a0 msgr2=0x7fe8c814fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:28.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.466+0000 7fe8cddff700 1 --2- 172.21.15.31:0/78686996 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8c80cb5a0 0x7fe8c814fab0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe8b80093f0 tx=0x7fe8b8017000 comp rx=0 tx=0).stop 2023-10-06T02:23:28.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.466+0000 7fe8cddff700 1 -- 172.21.15.31:0/78686996 shutdown_connections 2023-10-06T02:23:28.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.466+0000 7fe8cddff700 1 --2- 172.21.15.31:0/78686996 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe8b4066f30 0x7fe8b40693e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:28.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.466+0000 7fe8cddff700 1 --2- 172.21.15.31:0/78686996 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe8c815a5d0 0x7fe8c80c7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:28.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.466+0000 7fe8cddff700 1 --2- 172.21.15.31:0/78686996 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8c80cb5a0 0x7fe8c814fab0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:28.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.466+0000 7fe8cddff700 1 --2- 172.21.15.31:0/78686996 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8c80cd9e0 0x7fe8c80c9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:28.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.466+0000 7fe8cddff700 1 -- 172.21.15.31:0/78686996 >> 172.21.15.31:0/78686996 conn(0x7fe8c8070e30 msgr2=0x7fe8c80b0b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:28.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.466+0000 7fe8cddff700 1 -- 172.21.15.31:0/78686996 shutdown_connections 2023-10-06T02:23:28.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.466+0000 7fe8cddff700 1 -- 172.21.15.31:0/78686996 wait complete. 2023-10-06T02:23:28.468 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 8 2023-10-06T02:23:28.484 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:23:28.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.671+0000 7fc50d39d700 1 Processor -- start 2023-10-06T02:23:28.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.671+0000 7fc50d39d700 1 -- start start 2023-10-06T02:23:28.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.672+0000 7fc50d39d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc50815de50 0x7fc5080cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:28.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.672+0000 7fc50d39d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc5080cef40 0x7fc50815c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:28.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.672+0000 7fc50d39d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc50815afd0 con 0x7fc508153c30 2023-10-06T02:23:28.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.672+0000 7fc50d39d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc50815b2b0 con 0x7fc5080cef40 2023-10-06T02:23:28.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.672+0000 7fc50d39d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc50815b590 con 0x7fc50815de50 2023-10-06T02:23:28.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.673+0000 7fc50cb9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc5080cef40 0x7fc50815c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:28.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.673+0000 7fc50cb9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc5080cef40 0x7fc50815c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58788/0 (socket says 172.21.15.31:58788) 2023-10-06T02:23:28.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.673+0000 7fc50cb9c700 1 -- 172.21.15.31:0/667301882 learned_addr learned my addr 172.21.15.31:0/667301882 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:28.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.673+0000 7fc507fff700 1 --1- 172.21.15.31:0/667301882 >> v1:172.21.15.31:6789/0 conn(0x7fc508153c30 0x7fc50815a5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:41760/0 (socket says 172.21.15.31:41760) 2023-10-06T02:23:28.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.673+0000 7fc5077fe700 1 --2- 172.21.15.31:0/667301882 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc50815de50 0x7fc5080cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:28.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.673+0000 7fc50cb9c700 1 -- 172.21.15.31:0/667301882 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc50815de50 msgr2=0x7fc5080cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:28.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.673+0000 7fc50cb9c700 1 --2- 172.21.15.31:0/667301882 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc50815de50 0x7fc5080cea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:28.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.673+0000 7fc50cb9c700 1 -- 172.21.15.31:0/667301882 >> v1:172.21.15.31:6789/0 conn(0x7fc508153c30 legacy=0x7fc50815a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:28.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.673+0000 7fc506ffd700 1 -- 172.21.15.31:0/667301882 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3034738626 0 0) 0x7fc50815afd0 con 0x7fc508153c30 2023-10-06T02:23:28.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.673+0000 7fc50cb9c700 1 -- 172.21.15.31:0/667301882 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc50815b870 con 0x7fc5080cef40 2023-10-06T02:23:28.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.673+0000 7fc50cb9c700 1 --2- 172.21.15.31:0/667301882 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5080cef40 0x7fc50815c070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc500009f20 tx=0x7fc500009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee18d2a586fb5ab4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:28.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.674+0000 7fc506ffd700 1 -- 172.21.15.31:0/667301882 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc50000e070 con 0x7fc5080cef40 2023-10-06T02:23:28.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.674+0000 7fc506ffd700 1 -- 172.21.15.31:0/667301882 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc50000b9d0 con 0x7fc5080cef40 2023-10-06T02:23:28.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.674+0000 7fc506ffd700 1 -- 172.21.15.31:0/667301882 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5000092e0 con 0x7fc5080cef40 2023-10-06T02:23:28.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.674+0000 7fc50d39d700 1 -- 172.21.15.31:0/667301882 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5080cef40 msgr2=0x7fc50815c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:28.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.674+0000 7fc50d39d700 1 --2- 172.21.15.31:0/667301882 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5080cef40 0x7fc50815c070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc500009f20 tx=0x7fc500009840 comp rx=0 tx=0).stop 2023-10-06T02:23:28.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.674+0000 7fc50d39d700 1 -- 172.21.15.31:0/667301882 shutdown_connections 2023-10-06T02:23:28.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.674+0000 7fc50d39d700 1 --2- 172.21.15.31:0/667301882 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5080cef40 0x7fc50815c070 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:28.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.674+0000 7fc50d39d700 1 --2- 172.21.15.31:0/667301882 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc50815de50 0x7fc5080cea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:28.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.674+0000 7fc50d39d700 1 -- 172.21.15.31:0/667301882 >> 172.21.15.31:0/667301882 conn(0x7fc5080703a0 msgr2=0x7fc5080707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:28.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.674+0000 7fc50d39d700 1 -- 172.21.15.31:0/667301882 shutdown_connections 2023-10-06T02:23:28.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.674+0000 7fc50d39d700 1 -- 172.21.15.31:0/667301882 wait complete. 2023-10-06T02:23:28.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.676+0000 7fc50d39d700 1 Processor -- start 2023-10-06T02:23:28.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.676+0000 7fc50d39d700 1 -- start start 2023-10-06T02:23:28.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.677+0000 7fc50d39d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc5080cef40 0x7fc5080c7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:28.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.677+0000 7fc50d39d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc508153c30 0x7fc5080b19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:28.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.677+0000 7fc50d39d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc50815de50 0x7fc5080b1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:28.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.677+0000 7fc50d39d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5080c8160 con 0x7fc508153c30 2023-10-06T02:23:28.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.677+0000 7fc50d39d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc50816da40 con 0x7fc5080cef40 2023-10-06T02:23:28.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.677+0000 7fc50d39d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc50816d110 con 0x7fc50815de50 2023-10-06T02:23:28.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.677+0000 7fc5077fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc508153c30 0x7fc5080b19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:28.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.677+0000 7fc5077fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc508153c30 0x7fc5080b19e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58804/0 (socket says 172.21.15.31:58804) 2023-10-06T02:23:28.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.677+0000 7fc5077fe700 1 -- 172.21.15.31:0/35068650 learned_addr learned my addr 172.21.15.31:0/35068650 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:28.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.677+0000 7fc50cb9c700 1 --2- 172.21.15.31:0/35068650 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc50815de50 0x7fc5080b1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:28.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.677+0000 7fc5077fe700 1 -- 172.21.15.31:0/35068650 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc50815de50 msgr2=0x7fc5080b1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:28.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.677+0000 7fc507fff700 1 --2- 172.21.15.31:0/35068650 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc5080cef40 0x7fc5080c7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:28.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.677+0000 7fc5077fe700 1 --2- 172.21.15.31:0/35068650 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc50815de50 0x7fc5080b1f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:28.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.677+0000 7fc5077fe700 1 -- 172.21.15.31:0/35068650 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc5080cef40 msgr2=0x7fc5080c7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:28.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.677+0000 7fc5077fe700 1 --2- 172.21.15.31:0/35068650 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc5080cef40 0x7fc5080c7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:28.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.677+0000 7fc5077fe700 1 -- 172.21.15.31:0/35068650 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc500009ae0 con 0x7fc508153c30 2023-10-06T02:23:28.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.677+0000 7fc507fff700 1 --2- 172.21.15.31:0/35068650 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc5080cef40 0x7fc5080c7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:23:28.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.678+0000 7fc5077fe700 1 --2- 172.21.15.31:0/35068650 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc508153c30 0x7fc5080b19e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc00ec80 tx=0x7fc4fc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:28.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.678+0000 7fc4effff700 1 -- 172.21.15.31:0/35068650 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4fc00c9d0 con 0x7fc508153c30 2023-10-06T02:23:28.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.678+0000 7fc4effff700 1 -- 172.21.15.31:0/35068650 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4fc00cb30 con 0x7fc508153c30 2023-10-06T02:23:28.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.678+0000 7fc50d39d700 1 -- 172.21.15.31:0/35068650 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc50816d3f0 con 0x7fc508153c30 2023-10-06T02:23:28.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.678+0000 7fc4effff700 1 -- 172.21.15.31:0/35068650 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4fc013470 con 0x7fc508153c30 2023-10-06T02:23:28.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.678+0000 7fc50d39d700 1 -- 172.21.15.31:0/35068650 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc50816aa80 con 0x7fc508153c30 2023-10-06T02:23:28.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.679+0000 7fc50d39d700 1 -- 172.21.15.31:0/35068650 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc50816dbb0 con 0x7fc508153c30 2023-10-06T02:23:28.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.680+0000 7fc4effff700 1 -- 172.21.15.31:0/35068650 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc4fc013820 con 0x7fc508153c30 2023-10-06T02:23:28.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.681+0000 7fc4effff700 1 --2- 172.21.15.31:0/35068650 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc4f4066f30 0x7fc4f40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:28.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.681+0000 7fc4effff700 1 -- 172.21.15.31:0/35068650 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc4fc084af0 con 0x7fc508153c30 2023-10-06T02:23:28.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.681+0000 7fc507fff700 1 --2- 172.21.15.31:0/35068650 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc4f4066f30 0x7fc4f40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:28.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.682+0000 7fc507fff700 1 --2- 172.21.15.31:0/35068650 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc4f4066f30 0x7fc4f40693e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc5080cda50 tx=0x7fc4f8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:28.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.685+0000 7fc4effff700 1 -- 172.21.15.31:0/35068650 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc4fc051b60 con 0x7fc508153c30 2023-10-06T02:23:28.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.889+0000 7fc50d39d700 1 -- 172.21.15.31:0/35068650 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc50816d890 con 0x7fc508153c30 2023-10-06T02:23:28.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.890+0000 7fc4effff700 1 -- 172.21.15.31:0/35068650 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16818 (secure 0 0 0) 0x7fc4fc055180 con 0x7fc508153c30 2023-10-06T02:23:28.890 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:23:28.891 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":23,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:23:10.970534+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:23:07.290269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T02:23:08.504679+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:23:28.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.893+0000 7fc50d39d700 1 -- 172.21.15.31:0/35068650 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc4f4066f30 msgr2=0x7fc4f40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:28.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.893+0000 7fc50d39d700 1 --2- 172.21.15.31:0/35068650 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc4f4066f30 0x7fc4f40693e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc5080cda50 tx=0x7fc4f8009040 comp rx=0 tx=0).stop 2023-10-06T02:23:28.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.893+0000 7fc50d39d700 1 -- 172.21.15.31:0/35068650 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc508153c30 msgr2=0x7fc5080b19e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:28.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.893+0000 7fc50d39d700 1 --2- 172.21.15.31:0/35068650 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc508153c30 0x7fc5080b19e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc00ec80 tx=0x7fc4fc0091a0 comp rx=0 tx=0).stop 2023-10-06T02:23:28.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.893+0000 7fc50d39d700 1 -- 172.21.15.31:0/35068650 shutdown_connections 2023-10-06T02:23:28.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.893+0000 7fc50d39d700 1 --2- 172.21.15.31:0/35068650 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc4f4066f30 0x7fc4f40693e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:28.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.893+0000 7fc50d39d700 1 --2- 172.21.15.31:0/35068650 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc50815de50 0x7fc5080b1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:28.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.893+0000 7fc50d39d700 1 --2- 172.21.15.31:0/35068650 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc5080cef40 0x7fc5080c7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:28.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.893+0000 7fc50d39d700 1 --2- 172.21.15.31:0/35068650 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc508153c30 0x7fc5080b19e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:28.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.893+0000 7fc50d39d700 1 -- 172.21.15.31:0/35068650 >> 172.21.15.31:0/35068650 conn(0x7fc5080703a0 msgr2=0x7fc508070770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:28.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.893+0000 7fc50d39d700 1 -- 172.21.15.31:0/35068650 shutdown_connections 2023-10-06T02:23:28.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:28.893+0000 7fc50d39d700 1 -- 172.21.15.31:0/35068650 wait complete. 2023-10-06T02:23:28.911 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T02:22:53.183425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T02:23:00.046967+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T02:23:07.290269+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T02:23:08.504679+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T02:23:28.911 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T02:23:29.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.100+0000 7fc442b9d700 1 Processor -- start 2023-10-06T02:23:29.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.100+0000 7fc442b9d700 1 -- start start 2023-10-06T02:23:29.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.101+0000 7fc442b9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc43c153dd0 0x7fc43c15de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:29.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.101+0000 7fc442b9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc43c0cea00 0x7fc43c0cee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:29.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.101+0000 7fc442b9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc43c15ac50 con 0x7fc43c153a00 2023-10-06T02:23:29.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.101+0000 7fc442b9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc43c15d010 con 0x7fc43c0cea00 2023-10-06T02:23:29.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.101+0000 7fc442b9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc43c15af60 con 0x7fc43c153dd0 2023-10-06T02:23:29.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.101+0000 7fc441b9b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fc43c153a00 0x7fc43c15a540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:41776/0 (socket says 172.21.15.31:41776) 2023-10-06T02:23:29.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.101+0000 7fc441b9b700 1 -- 172.21.15.31:0/4107524444 learned_addr learned my addr 172.21.15.31:0/4107524444 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:23:29.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.102+0000 7fc44239c700 1 --2- 172.21.15.31:0/4107524444 >> v2:172.21.15.62:3300/0 conn(0x7fc43c0cea00 0x7fc43c0cee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:29.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.102+0000 7fc44139a700 1 --2- 172.21.15.31:0/4107524444 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc43c153dd0 0x7fc43c15de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:29.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.102+0000 7fc440b99700 1 -- 172.21.15.31:0/4107524444 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1759635195 0 0) 0x7fc43c15ac50 con 0x7fc43c153a00 2023-10-06T02:23:29.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.102+0000 7fc440b99700 1 -- 172.21.15.31:0/4107524444 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc428003570 con 0x7fc43c153a00 2023-10-06T02:23:29.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.102+0000 7fc44139a700 1 -- 172.21.15.31:0/4107524444 >> v2:172.21.15.62:3300/0 conn(0x7fc43c0cea00 msgr2=0x7fc43c0cee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:29.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.102+0000 7fc44139a700 1 --2- 172.21.15.31:0/4107524444 >> v2:172.21.15.62:3300/0 conn(0x7fc43c0cea00 0x7fc43c0cee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:29.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.102+0000 7fc44139a700 1 -- 172.21.15.31:0/4107524444 >> v1:172.21.15.31:6789/0 conn(0x7fc43c153a00 legacy=0x7fc43c15a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:29.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.102+0000 7fc44139a700 1 -- 172.21.15.31:0/4107524444 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc43c15b240 con 0x7fc43c153dd0 2023-10-06T02:23:29.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.102+0000 7fc44139a700 1 --2- 172.21.15.31:0/4107524444 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc43c153dd0 0x7fc43c15de50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc42c009910 tx=0x7fc42c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0313e6f01329130 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:29.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.103+0000 7fc440b99700 1 -- 172.21.15.31:0/4107524444 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc42c011630 con 0x7fc43c153dd0 2023-10-06T02:23:29.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.103+0000 7fc440b99700 1 -- 172.21.15.31:0/4107524444 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc42c011790 con 0x7fc43c153dd0 2023-10-06T02:23:29.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.103+0000 7fc440b99700 1 -- 172.21.15.31:0/4107524444 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc42c011b30 con 0x7fc43c153dd0 2023-10-06T02:23:29.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.103+0000 7fc442b9d700 1 -- 172.21.15.31:0/4107524444 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc43c153dd0 msgr2=0x7fc43c15de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:29.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.103+0000 7fc442b9d700 1 --2- 172.21.15.31:0/4107524444 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc43c153dd0 0x7fc43c15de50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc42c009910 tx=0x7fc42c0091a0 comp rx=0 tx=0).stop 2023-10-06T02:23:29.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.103+0000 7fc442b9d700 1 -- 172.21.15.31:0/4107524444 shutdown_connections 2023-10-06T02:23:29.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.103+0000 7fc442b9d700 1 --2- 172.21.15.31:0/4107524444 >> v2:172.21.15.62:3300/0 conn(0x7fc43c0cea00 0x7fc43c0cee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:29.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.103+0000 7fc442b9d700 1 --2- 172.21.15.31:0/4107524444 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc43c153dd0 0x7fc43c15de50 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:29.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.103+0000 7fc442b9d700 1 -- 172.21.15.31:0/4107524444 >> 172.21.15.31:0/4107524444 conn(0x7fc43c0703a0 msgr2=0x7fc43c0707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:29.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.104+0000 7fc442b9d700 1 -- 172.21.15.31:0/4107524444 shutdown_connections 2023-10-06T02:23:29.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.104+0000 7fc442b9d700 1 -- 172.21.15.31:0/4107524444 wait complete. 2023-10-06T02:23:29.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.105+0000 7fc442b9d700 1 Processor -- start 2023-10-06T02:23:29.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.105+0000 7fc442b9d700 1 -- start start 2023-10-06T02:23:29.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.106+0000 7fc442b9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc43c0cea00 0x7fc43c1544d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:29.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.106+0000 7fc442b9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc43c153a00 0x7fc43c0cc330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:29.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.106+0000 7fc442b9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc43c153dd0 0x7fc43c150b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:29.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.106+0000 7fc442b9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc43c154b20 con 0x7fc43c153dd0 2023-10-06T02:23:29.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.106+0000 7fc442b9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc43c154e00 con 0x7fc43c153a00 2023-10-06T02:23:29.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.106+0000 7fc442b9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc43c1550e0 con 0x7fc43c0cea00 2023-10-06T02:23:29.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.106+0000 7fc44239c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc43c153dd0 0x7fc43c150b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:29.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.106+0000 7fc44239c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc43c153dd0 0x7fc43c150b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58810/0 (socket says 172.21.15.31:58810) 2023-10-06T02:23:29.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.106+0000 7fc44239c700 1 -- 172.21.15.31:0/3529083281 learned_addr learned my addr 172.21.15.31:0/3529083281 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:29.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.106+0000 7fc44139a700 1 --2- 172.21.15.31:0/3529083281 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc43c153a00 0x7fc43c0cc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:29.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.106+0000 7fc441b9b700 1 --2- 172.21.15.31:0/3529083281 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc43c0cea00 0x7fc43c1544d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:29.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.107+0000 7fc44239c700 1 -- 172.21.15.31:0/3529083281 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc43c0cea00 msgr2=0x7fc43c1544d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:29.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.107+0000 7fc44239c700 1 --2- 172.21.15.31:0/3529083281 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc43c0cea00 0x7fc43c1544d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:29.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.107+0000 7fc44239c700 1 -- 172.21.15.31:0/3529083281 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc43c153a00 msgr2=0x7fc43c0cc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:29.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.107+0000 7fc44239c700 1 --2- 172.21.15.31:0/3529083281 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc43c153a00 0x7fc43c0cc330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:29.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.107+0000 7fc44239c700 1 -- 172.21.15.31:0/3529083281 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc42c009420 con 0x7fc43c153dd0 2023-10-06T02:23:29.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.107+0000 7fc44239c700 1 --2- 172.21.15.31:0/3529083281 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc43c153dd0 0x7fc43c150b80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc434009e90 tx=0x7fc43400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:29.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.107+0000 7fc431ffb700 1 -- 172.21.15.31:0/3529083281 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43400cc00 con 0x7fc43c153dd0 2023-10-06T02:23:29.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.107+0000 7fc431ffb700 1 -- 172.21.15.31:0/3529083281 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc43400f040 con 0x7fc43c153dd0 2023-10-06T02:23:29.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.107+0000 7fc442b9d700 1 -- 172.21.15.31:0/3529083281 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc43c1553c0 con 0x7fc43c153dd0 2023-10-06T02:23:29.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.107+0000 7fc431ffb700 1 -- 172.21.15.31:0/3529083281 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc434014480 con 0x7fc43c153dd0 2023-10-06T02:23:29.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.107+0000 7fc442b9d700 1 -- 172.21.15.31:0/3529083281 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc43c0cc870 con 0x7fc43c153dd0 2023-10-06T02:23:29.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.108+0000 7fc442b9d700 1 -- 172.21.15.31:0/3529083281 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc43c1558e0 con 0x7fc43c153dd0 2023-10-06T02:23:29.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.109+0000 7fc431ffb700 1 -- 172.21.15.31:0/3529083281 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc434014830 con 0x7fc43c153dd0 2023-10-06T02:23:29.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.110+0000 7fc431ffb700 1 --2- 172.21.15.31:0/3529083281 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc420066f30 0x7fc4200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:29.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.110+0000 7fc431ffb700 1 -- 172.21.15.31:0/3529083281 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc4340851e0 con 0x7fc43c153dd0 2023-10-06T02:23:29.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.110+0000 7fc441b9b700 1 --2- 172.21.15.31:0/3529083281 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc420066f30 0x7fc4200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:29.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.111+0000 7fc441b9b700 1 --2- 172.21.15.31:0/3529083281 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc420066f30 0x7fc4200693e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc438007fd0 tx=0x7fc438009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:29.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.114+0000 7fc431ffb700 1 -- 172.21.15.31:0/3529083281 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc43c1558e0 con 0x7fc43c153dd0 2023-10-06T02:23:29.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.316+0000 7fc442b9d700 1 -- 172.21.15.31:0/3529083281 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc43c1558e0 con 0x7fc43c153dd0 2023-10-06T02:23:29.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.317+0000 7fc431ffb700 1 -- 172.21.15.31:0/3529083281 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc43c1558e0 con 0x7fc43c153dd0 2023-10-06T02:23:29.317 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-10-06T02:23:29.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.319+0000 7fc442b9d700 1 -- 172.21.15.31:0/3529083281 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc420066f30 msgr2=0x7fc4200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:29.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.319+0000 7fc442b9d700 1 --2- 172.21.15.31:0/3529083281 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc420066f30 0x7fc4200693e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc438007fd0 tx=0x7fc438009040 comp rx=0 tx=0).stop 2023-10-06T02:23:29.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.319+0000 7fc442b9d700 1 -- 172.21.15.31:0/3529083281 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc43c153dd0 msgr2=0x7fc43c150b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:29.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.319+0000 7fc442b9d700 1 --2- 172.21.15.31:0/3529083281 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc43c153dd0 0x7fc43c150b80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc434009e90 tx=0x7fc43400b040 comp rx=0 tx=0).stop 2023-10-06T02:23:29.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.319+0000 7fc442b9d700 1 -- 172.21.15.31:0/3529083281 shutdown_connections 2023-10-06T02:23:29.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.319+0000 7fc442b9d700 1 --2- 172.21.15.31:0/3529083281 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc43c153dd0 0x7fc43c150b80 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:29.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.319+0000 7fc442b9d700 1 --2- 172.21.15.31:0/3529083281 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc420066f30 0x7fc4200693e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:29.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.319+0000 7fc442b9d700 1 --2- 172.21.15.31:0/3529083281 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc43c0cea00 0x7fc43c1544d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:29.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.319+0000 7fc442b9d700 1 --2- 172.21.15.31:0/3529083281 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc43c153a00 0x7fc43c0cc330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:29.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.319+0000 7fc442b9d700 1 -- 172.21.15.31:0/3529083281 >> 172.21.15.31:0/3529083281 conn(0x7fc43c0703a0 msgr2=0x7fc43c0b1a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:29.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.320+0000 7fc442b9d700 1 -- 172.21.15.31:0/3529083281 shutdown_connections 2023-10-06T02:23:29.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.320+0000 7fc442b9d700 1 -- 172.21.15.31:0/3529083281 wait complete. 2023-10-06T02:23:29.337 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T02:23:29.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.526+0000 7f0e4759e700 1 Processor -- start 2023-10-06T02:23:29.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.527+0000 7f0e4759e700 1 -- start start 2023-10-06T02:23:29.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.528+0000 7f0e4759e700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0e480c7fb0 0x7f0e4806abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:29.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.528+0000 7f0e4759e700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e4806b290 con 0x7f0e4813f780 2023-10-06T02:23:29.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.528+0000 7f0e4759e700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e4806b570 con 0x7f0e480c7be0 2023-10-06T02:23:29.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.528+0000 7f0e4759e700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e4806bf10 con 0x7f0e480c7fb0 2023-10-06T02:23:29.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.528+0000 7f0e46d9d700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f0e4813f780 0x7f0e4813fba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:41786/0 (socket says 172.21.15.31:41786) 2023-10-06T02:23:29.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.528+0000 7f0e46d9d700 1 -- 172.21.15.31:0/2562701676 learned_addr learned my addr 172.21.15.31:0/2562701676 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:23:29.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.528+0000 7f0e45d9b700 1 --2- 172.21.15.31:0/2562701676 >> v2:172.21.15.62:3300/0 conn(0x7f0e480c7fb0 0x7f0e4806abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:29.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.529+0000 7f0e4559a700 1 -- 172.21.15.31:0/2562701676 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3784515965 0 0) 0x7f0e4806b290 con 0x7f0e4813f780 2023-10-06T02:23:29.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.529+0000 7f0e4559a700 1 -- 172.21.15.31:0/2562701676 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e30003570 con 0x7f0e4813f780 2023-10-06T02:23:29.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.529+0000 7f0e4559a700 1 -- 172.21.15.31:0/2562701676 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 105816632 0 0) 0x7f0e4806b570 con 0x7f0e480c7be0 2023-10-06T02:23:29.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.529+0000 7f0e4559a700 1 -- 172.21.15.31:0/2562701676 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e4806b290 con 0x7f0e480c7be0 2023-10-06T02:23:29.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.529+0000 7f0e4559a700 1 -- 172.21.15.31:0/2562701676 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4021307469 0 0) 0x7f0e30003570 con 0x7f0e4813f780 2023-10-06T02:23:29.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.529+0000 7f0e4559a700 1 -- 172.21.15.31:0/2562701676 >> v2:172.21.15.62:3300/0 conn(0x7f0e480c7fb0 msgr2=0x7f0e4806abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:29.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.529+0000 7f0e4559a700 1 --2- 172.21.15.31:0/2562701676 >> v2:172.21.15.62:3300/0 conn(0x7f0e480c7fb0 0x7f0e4806abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:29.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.529+0000 7f0e4559a700 1 -- 172.21.15.31:0/2562701676 >> v1:172.21.15.62:6789/0 conn(0x7f0e480c7be0 legacy=0x7f0e4815a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:29.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.529+0000 7f0e4559a700 1 -- 172.21.15.31:0/2562701676 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e4806b850 con 0x7f0e4813f780 2023-10-06T02:23:29.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.529+0000 7f0e4559a700 1 -- 172.21.15.31:0/2562701676 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7f0e400017f0 con 0x7f0e4813f780 2023-10-06T02:23:29.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.529+0000 7f0e4559a700 1 -- 172.21.15.31:0/2562701676 >> v1:172.21.15.31:6789/0 conn(0x7f0e4813f780 legacy=0x7f0e4813fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:29.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.529+0000 7f0e4559a700 1 --2- 172.21.15.31:0/2562701676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e30003bf0 0x7f0e30003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:29.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.529+0000 7f0e4559a700 1 --2- 172.21.15.31:0/2562701676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e30004810 0x7f0e30004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:29.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.529+0000 7f0e4559a700 1 --2- 172.21.15.31:0/2562701676 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0e300051c0 0x7f0e3000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:29.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.529+0000 7f0e4559a700 1 -- 172.21.15.31:0/2562701676 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e4806bf10 con 0x7f0e30004810 2023-10-06T02:23:29.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.529+0000 7f0e4559a700 1 -- 172.21.15.31:0/2562701676 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3000c0b0 con 0x7f0e30003bf0 2023-10-06T02:23:29.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.529+0000 7f0e4559a700 1 -- 172.21.15.31:0/2562701676 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3000c270 con 0x7f0e300051c0 2023-10-06T02:23:29.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.529+0000 7f0e45d9b700 1 --2- 172.21.15.31:0/2562701676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e30004810 0x7f0e30004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:29.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.530+0000 7f0e46d9d700 1 --2- 172.21.15.31:0/2562701676 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0e300051c0 0x7f0e3000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:29.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.530+0000 7f0e4659c700 1 --2- 172.21.15.31:0/2562701676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e30003bf0 0x7f0e30003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:29.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.530+0000 7f0e45d9b700 1 -- 172.21.15.31:0/2562701676 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0e300051c0 msgr2=0x7f0e3000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:29.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.530+0000 7f0e45d9b700 1 --2- 172.21.15.31:0/2562701676 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0e300051c0 0x7f0e3000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:29.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.530+0000 7f0e45d9b700 1 -- 172.21.15.31:0/2562701676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e30003bf0 msgr2=0x7f0e30003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:29.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.530+0000 7f0e45d9b700 1 --2- 172.21.15.31:0/2562701676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e30003bf0 0x7f0e30003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:29.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.530+0000 7f0e45d9b700 1 -- 172.21.15.31:0/2562701676 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e30003570 con 0x7f0e30004810 2023-10-06T02:23:29.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.530+0000 7f0e4659c700 1 --2- 172.21.15.31:0/2562701676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e30003bf0 0x7f0e30003fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:23:29.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.530+0000 7f0e46d9d700 1 -- 172.21.15.31:0/2562701676 reap_dead start 2023-10-06T02:23:29.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.530+0000 7f0e45d9b700 1 --2- 172.21.15.31:0/2562701676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e30004810 0x7f0e30004c80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c009c00 tx=0x7f0e3c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14aaf91076dfc71f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:29.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.530+0000 7f0e4559a700 1 -- 172.21.15.31:0/2562701676 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e3c00e710 con 0x7f0e30004810 2023-10-06T02:23:29.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.530+0000 7f0e4559a700 1 -- 172.21.15.31:0/2562701676 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e3c00e870 con 0x7f0e30004810 2023-10-06T02:23:29.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.530+0000 7f0e4559a700 1 -- 172.21.15.31:0/2562701676 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e3c00ec30 con 0x7f0e30004810 2023-10-06T02:23:29.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.531+0000 7f0e4759e700 1 -- 172.21.15.31:0/2562701676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e30004810 msgr2=0x7f0e30004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:29.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.531+0000 7f0e4759e700 1 --2- 172.21.15.31:0/2562701676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e30004810 0x7f0e30004c80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c009c00 tx=0x7f0e3c0091a0 comp rx=0 tx=0).stop 2023-10-06T02:23:29.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.531+0000 7f0e4759e700 1 -- 172.21.15.31:0/2562701676 shutdown_connections 2023-10-06T02:23:29.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.531+0000 7f0e4759e700 1 --2- 172.21.15.31:0/2562701676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e30004810 0x7f0e30004c80 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:29.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.531+0000 7f0e4759e700 1 -- 172.21.15.31:0/2562701676 >> 172.21.15.31:0/2562701676 conn(0x7f0e4806d110 msgr2=0x7f0e480cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:29.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.531+0000 7f0e4759e700 1 -- 172.21.15.31:0/2562701676 shutdown_connections 2023-10-06T02:23:29.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.531+0000 7f0e4759e700 1 -- 172.21.15.31:0/2562701676 wait complete. 2023-10-06T02:23:29.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.533+0000 7f0e4759e700 1 Processor -- start 2023-10-06T02:23:29.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.533+0000 7f0e4759e700 1 -- start start 2023-10-06T02:23:29.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.534+0000 7f0e4759e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e30004810 0x7f0e480c3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:29.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.534+0000 7f0e4759e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e4813e4f0 0x7f0e480c54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:29.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.534+0000 7f0e4759e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0e4813e9a0 0x7f0e480c5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:29.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.534+0000 7f0e4759e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e480c4100 con 0x7f0e30004810 2023-10-06T02:23:29.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.534+0000 7f0e4759e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e480c5f60 con 0x7f0e4813e4f0 2023-10-06T02:23:29.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.534+0000 7f0e4759e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e480c2910 con 0x7f0e4813e9a0 2023-10-06T02:23:29.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.534+0000 7f0e4659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e30004810 0x7f0e480c3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:29.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.534+0000 7f0e4659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e30004810 0x7f0e480c3bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58824/0 (socket says 172.21.15.31:58824) 2023-10-06T02:23:29.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.534+0000 7f0e4659c700 1 -- 172.21.15.31:0/143723590 learned_addr learned my addr 172.21.15.31:0/143723590 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:29.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.534+0000 7f0e46d9d700 1 --2- 172.21.15.31:0/143723590 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0e4813e9a0 0x7f0e480c5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:29.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.534+0000 7f0e45d9b700 1 --2- 172.21.15.31:0/143723590 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e4813e4f0 0x7f0e480c54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:29.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.534+0000 7f0e4659c700 1 -- 172.21.15.31:0/143723590 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0e4813e9a0 msgr2=0x7f0e480c5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:29.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.534+0000 7f0e4659c700 1 --2- 172.21.15.31:0/143723590 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0e4813e9a0 0x7f0e480c5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:29.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.534+0000 7f0e4659c700 1 -- 172.21.15.31:0/143723590 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e4813e4f0 msgr2=0x7f0e480c54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:29.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.534+0000 7f0e4659c700 1 --2- 172.21.15.31:0/143723590 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e4813e4f0 0x7f0e480c54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:29.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.534+0000 7f0e4659c700 1 -- 172.21.15.31:0/143723590 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e4816a7b0 con 0x7f0e30004810 2023-10-06T02:23:29.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.534+0000 7f0e46d9d700 1 --2- 172.21.15.31:0/143723590 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0e4813e9a0 0x7f0e480c5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:23:29.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.535+0000 7f0e4659c700 1 --2- 172.21.15.31:0/143723590 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e30004810 0x7f0e480c3bc0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0e3800e5f0 tx=0x7f0e38009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:29.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.535+0000 7f0e367fc700 1 -- 172.21.15.31:0/143723590 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e38012590 con 0x7f0e30004810 2023-10-06T02:23:29.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.535+0000 7f0e367fc700 1 -- 172.21.15.31:0/143723590 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e380126f0 con 0x7f0e30004810 2023-10-06T02:23:29.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.535+0000 7f0e367fc700 1 -- 172.21.15.31:0/143723590 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e38012a90 con 0x7f0e30004810 2023-10-06T02:23:29.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.535+0000 7f0e4759e700 1 -- 172.21.15.31:0/143723590 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e48169c10 con 0x7f0e30004810 2023-10-06T02:23:29.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.535+0000 7f0e4759e700 1 -- 172.21.15.31:0/143723590 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e4816a160 con 0x7f0e30004810 2023-10-06T02:23:29.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.536+0000 7f0e4759e700 1 -- 172.21.15.31:0/143723590 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e4804c3c0 con 0x7f0e30004810 2023-10-06T02:23:29.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.537+0000 7f0e367fc700 1 -- 172.21.15.31:0/143723590 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0e3801d460 con 0x7f0e30004810 2023-10-06T02:23:29.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.537+0000 7f0e367fc700 1 --2- 172.21.15.31:0/143723590 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f0e28066f30 0x7f0e280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:29.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.538+0000 7f0e367fc700 1 -- 172.21.15.31:0/143723590 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0e380849b0 con 0x7f0e30004810 2023-10-06T02:23:29.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.538+0000 7f0e45d9b700 1 --2- 172.21.15.31:0/143723590 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f0e28066f30 0x7f0e280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:29.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.538+0000 7f0e45d9b700 1 --2- 172.21.15.31:0/143723590 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f0e28066f30 0x7f0e280693e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c0091e0 tx=0x7f0e3c01a000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:29.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.542+0000 7f0e367fc700 1 -- 172.21.15.31:0/143723590 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0e38051a20 con 0x7f0e30004810 2023-10-06T02:23:29.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.744+0000 7f0e4759e700 1 -- 172.21.15.31:0/143723590 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f0e4806bb30 con 0x7f0e30004810 2023-10-06T02:23:29.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.745+0000 7f0e367fc700 1 -- 172.21.15.31:0/143723590 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0e38055040 con 0x7f0e30004810 2023-10-06T02:23:29.745 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 8 2023-10-06T02:23:29.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.747+0000 7f0e4759e700 1 -- 172.21.15.31:0/143723590 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f0e28066f30 msgr2=0x7f0e280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:29.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.747+0000 7f0e4759e700 1 --2- 172.21.15.31:0/143723590 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f0e28066f30 0x7f0e280693e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c0091e0 tx=0x7f0e3c01a000 comp rx=0 tx=0).stop 2023-10-06T02:23:29.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.747+0000 7f0e4759e700 1 -- 172.21.15.31:0/143723590 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e30004810 msgr2=0x7f0e480c3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:29.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.747+0000 7f0e4759e700 1 --2- 172.21.15.31:0/143723590 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e30004810 0x7f0e480c3bc0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0e3800e5f0 tx=0x7f0e38009040 comp rx=0 tx=0).stop 2023-10-06T02:23:29.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.747+0000 7f0e4759e700 1 -- 172.21.15.31:0/143723590 shutdown_connections 2023-10-06T02:23:29.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.747+0000 7f0e4759e700 1 --2- 172.21.15.31:0/143723590 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f0e28066f30 0x7f0e280693e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:29.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.747+0000 7f0e4759e700 1 --2- 172.21.15.31:0/143723590 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0e4813e9a0 0x7f0e480c5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:29.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.747+0000 7f0e4759e700 1 --2- 172.21.15.31:0/143723590 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e30004810 0x7f0e480c3bc0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:29.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.747+0000 7f0e4759e700 1 --2- 172.21.15.31:0/143723590 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e4813e4f0 0x7f0e480c54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:29.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.747+0000 7f0e4759e700 1 -- 172.21.15.31:0/143723590 >> 172.21.15.31:0/143723590 conn(0x7f0e4806d110 msgr2=0x7f0e4806de50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:29.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.747+0000 7f0e4759e700 1 -- 172.21.15.31:0/143723590 shutdown_connections 2023-10-06T02:23:29.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.747+0000 7f0e4759e700 1 -- 172.21.15.31:0/143723590 wait complete. 2023-10-06T02:23:29.764 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T02:23:29.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.954+0000 7ff6ebfff700 1 Processor -- start 2023-10-06T02:23:29.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.954+0000 7ff6ebfff700 1 -- start start 2023-10-06T02:23:29.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.956+0000 7ff6ebfff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6ec153a00 0x7ff6ec153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:29.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.956+0000 7ff6ebfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff6ec15de50 0x7ff6ec15a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:29.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.956+0000 7ff6ebfff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff6ec0cea00 0x7ff6ec0cee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:29.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.956+0000 7ff6ebfff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6ec15af60 con 0x7ff6ec15de50 2023-10-06T02:23:29.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.956+0000 7ff6ebfff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6ec15b240 con 0x7ff6ec0cea00 2023-10-06T02:23:29.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.956+0000 7ff6ebfff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ec15b520 con 0x7ff6ec153a00 2023-10-06T02:23:29.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.956+0000 7ff6ea7fc700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff6ec15de50 0x7ff6ec15a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:29.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.956+0000 7ff6ea7fc700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff6ec15de50 0x7ff6ec15a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58826/0 (socket says 172.21.15.31:58826) 2023-10-06T02:23:29.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.956+0000 7ff6ea7fc700 1 -- 172.21.15.31:0/174078113 learned_addr learned my addr 172.21.15.31:0/174078113 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:29.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.956+0000 7ff6eb7fe700 1 --2- 172.21.15.31:0/174078113 >> v2:172.21.15.62:3300/0 conn(0x7ff6ec0cea00 0x7ff6ec0cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:29.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.956+0000 7ff6eaffd700 1 --2- 172.21.15.31:0/174078113 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6ec153a00 0x7ff6ec153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:29.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.956+0000 7ff6ea7fc700 1 -- 172.21.15.31:0/174078113 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6ec153a00 msgr2=0x7ff6ec153dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:29.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.956+0000 7ff6ea7fc700 1 --2- 172.21.15.31:0/174078113 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6ec153a00 0x7ff6ec153dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:29.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.956+0000 7ff6ea7fc700 1 -- 172.21.15.31:0/174078113 >> v2:172.21.15.62:3300/0 conn(0x7ff6ec0cea00 msgr2=0x7ff6ec0cee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:29.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.956+0000 7ff6ea7fc700 1 --2- 172.21.15.31:0/174078113 >> v2:172.21.15.62:3300/0 conn(0x7ff6ec0cea00 0x7ff6ec0cee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:29.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.956+0000 7ff6ea7fc700 1 -- 172.21.15.31:0/174078113 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6ec15b800 con 0x7ff6ec15de50 2023-10-06T02:23:29.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.957+0000 7ff6ea7fc700 1 --2- 172.21.15.31:0/174078113 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6ec15de50 0x7ff6ec15a540 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec159c60 tx=0x7ff6e0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbc7a919f34303ea server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:29.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.957+0000 7ff6e9ffb700 1 -- 172.21.15.31:0/174078113 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e000b860 con 0x7ff6ec15de50 2023-10-06T02:23:29.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.957+0000 7ff6e9ffb700 1 -- 172.21.15.31:0/174078113 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6e000b9c0 con 0x7ff6ec15de50 2023-10-06T02:23:29.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.957+0000 7ff6e9ffb700 1 -- 172.21.15.31:0/174078113 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e000bd60 con 0x7ff6ec15de50 2023-10-06T02:23:29.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.957+0000 7ff6ebfff700 1 -- 172.21.15.31:0/174078113 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6ec15de50 msgr2=0x7ff6ec15a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:29.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.957+0000 7ff6ebfff700 1 --2- 172.21.15.31:0/174078113 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6ec15de50 0x7ff6ec15a540 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec159c60 tx=0x7ff6e0009250 comp rx=0 tx=0).stop 2023-10-06T02:23:29.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.957+0000 7ff6ebfff700 1 -- 172.21.15.31:0/174078113 shutdown_connections 2023-10-06T02:23:29.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.957+0000 7ff6ebfff700 1 --2- 172.21.15.31:0/174078113 >> v2:172.21.15.62:3300/0 conn(0x7ff6ec0cea00 0x7ff6ec0cee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:29.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.957+0000 7ff6ebfff700 1 --2- 172.21.15.31:0/174078113 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6ec153a00 0x7ff6ec153dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:29.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.957+0000 7ff6ebfff700 1 --2- 172.21.15.31:0/174078113 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6ec15de50 0x7ff6ec15a540 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:29.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.957+0000 7ff6ebfff700 1 -- 172.21.15.31:0/174078113 >> 172.21.15.31:0/174078113 conn(0x7ff6ec0703a0 msgr2=0x7ff6ec0707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:29.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.958+0000 7ff6ebfff700 1 -- 172.21.15.31:0/174078113 shutdown_connections 2023-10-06T02:23:29.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.958+0000 7ff6ebfff700 1 -- 172.21.15.31:0/174078113 wait complete. 2023-10-06T02:23:29.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.959+0000 7ff6ebfff700 1 Processor -- start 2023-10-06T02:23:29.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.959+0000 7ff6ebfff700 1 -- start start 2023-10-06T02:23:29.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.960+0000 7ff6ebfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6ec0cea00 0x7ff6ec150810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:29.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.960+0000 7ff6ebfff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6ec153a00 0x7ff6ec14af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:29.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.960+0000 7ff6ebfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6ec15de50 0x7ff6ec0d0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:29.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.960+0000 7ff6ebfff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ec150d50 con 0x7ff6ec15de50 2023-10-06T02:23:29.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.960+0000 7ff6ebfff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ec16de10 con 0x7ff6ec0cea00 2023-10-06T02:23:29.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.960+0000 7ff6ebfff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ec16d200 con 0x7ff6ec153a00 2023-10-06T02:23:29.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.960+0000 7ff6eb7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6ec15de50 0x7ff6ec0d0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:29.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.960+0000 7ff6eb7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6ec15de50 0x7ff6ec0d0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58842/0 (socket says 172.21.15.31:58842) 2023-10-06T02:23:29.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.960+0000 7ff6eb7fe700 1 -- 172.21.15.31:0/3575273286 learned_addr learned my addr 172.21.15.31:0/3575273286 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:29.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.961+0000 7ff6ea7fc700 1 --2- 172.21.15.31:0/3575273286 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6ec153a00 0x7ff6ec14af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:29.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.961+0000 7ff6eaffd700 1 --2- 172.21.15.31:0/3575273286 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6ec0cea00 0x7ff6ec150810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:29.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.961+0000 7ff6eb7fe700 1 -- 172.21.15.31:0/3575273286 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6ec153a00 msgr2=0x7ff6ec14af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:29.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.961+0000 7ff6eb7fe700 1 --2- 172.21.15.31:0/3575273286 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6ec153a00 0x7ff6ec14af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:29.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.961+0000 7ff6eb7fe700 1 -- 172.21.15.31:0/3575273286 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6ec0cea00 msgr2=0x7ff6ec150810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:29.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.961+0000 7ff6eb7fe700 1 --2- 172.21.15.31:0/3575273286 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6ec0cea00 0x7ff6ec150810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:29.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.961+0000 7ff6eb7fe700 1 -- 172.21.15.31:0/3575273286 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6e00094f0 con 0x7ff6ec15de50 2023-10-06T02:23:29.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.961+0000 7ff6eb7fe700 1 --2- 172.21.15.31:0/3575273286 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6ec15de50 0x7ff6ec0d0170 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff6e4009fa0 tx=0x7ff6e4009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:29.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.961+0000 7ff6d2ffd700 1 -- 172.21.15.31:0/3575273286 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e400ed20 con 0x7ff6ec15de50 2023-10-06T02:23:29.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.961+0000 7ff6d2ffd700 1 -- 172.21.15.31:0/3575273286 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6e400ee80 con 0x7ff6ec15de50 2023-10-06T02:23:29.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.961+0000 7ff6d2ffd700 1 -- 172.21.15.31:0/3575273286 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e4014670 con 0x7ff6ec15de50 2023-10-06T02:23:29.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.961+0000 7ff6ebfff700 1 -- 172.21.15.31:0/3575273286 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6ec16d4e0 con 0x7ff6ec15de50 2023-10-06T02:23:29.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.961+0000 7ff6ebfff700 1 -- 172.21.15.31:0/3575273286 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6ec16daa0 con 0x7ff6ec15de50 2023-10-06T02:23:29.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.962+0000 7ff6ebfff700 1 -- 172.21.15.31:0/3575273286 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6ec157790 con 0x7ff6ec15de50 2023-10-06T02:23:29.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.963+0000 7ff6d2ffd700 1 -- 172.21.15.31:0/3575273286 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff6e40149e0 con 0x7ff6ec15de50 2023-10-06T02:23:29.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.964+0000 7ff6d2ffd700 1 --2- 172.21.15.31:0/3575273286 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7ff6d8066f30 0x7ff6d80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:29.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.964+0000 7ff6d2ffd700 1 -- 172.21.15.31:0/3575273286 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff6e4084f20 con 0x7ff6ec15de50 2023-10-06T02:23:29.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.965+0000 7ff6eaffd700 1 --2- 172.21.15.31:0/3575273286 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7ff6d8066f30 0x7ff6d80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:29.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.965+0000 7ff6eaffd700 1 --2- 172.21.15.31:0/3575273286 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7ff6d8066f30 0x7ff6d80693e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc006fd0 tx=0x7ff6dc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:29.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:29.968+0000 7ff6d2ffd700 1 -- 172.21.15.31:0/3575273286 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff6ec157790 con 0x7ff6ec15de50 2023-10-06T02:23:30.073 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T02:23:30.078 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T02:23:30.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.171+0000 7ff6ebfff700 1 -- 172.21.15.31:0/3575273286 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff6ec157790 con 0x7ff6ec15de50 2023-10-06T02:23:30.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.171+0000 7ff6d2ffd700 1 -- 172.21.15.31:0/3575273286 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff6ec157790 con 0x7ff6ec15de50 2023-10-06T02:23:30.172 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-10-06T02:23:30.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.174+0000 7ff6ebfff700 1 -- 172.21.15.31:0/3575273286 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7ff6d8066f30 msgr2=0x7ff6d80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:30.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.174+0000 7ff6ebfff700 1 --2- 172.21.15.31:0/3575273286 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7ff6d8066f30 0x7ff6d80693e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc006fd0 tx=0x7ff6dc008040 comp rx=0 tx=0).stop 2023-10-06T02:23:30.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.174+0000 7ff6ebfff700 1 -- 172.21.15.31:0/3575273286 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6ec15de50 msgr2=0x7ff6ec0d0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:30.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.174+0000 7ff6ebfff700 1 --2- 172.21.15.31:0/3575273286 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6ec15de50 0x7ff6ec0d0170 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff6e4009fa0 tx=0x7ff6e4009900 comp rx=0 tx=0).stop 2023-10-06T02:23:30.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.174+0000 7ff6ebfff700 1 -- 172.21.15.31:0/3575273286 shutdown_connections 2023-10-06T02:23:30.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.174+0000 7ff6ebfff700 1 --2- 172.21.15.31:0/3575273286 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6ec15de50 0x7ff6ec0d0170 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:30.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.174+0000 7ff6ebfff700 1 --2- 172.21.15.31:0/3575273286 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6ec0cea00 0x7ff6ec150810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:30.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.174+0000 7ff6ebfff700 1 --2- 172.21.15.31:0/3575273286 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7ff6d8066f30 0x7ff6d80693e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:30.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.174+0000 7ff6ebfff700 1 --2- 172.21.15.31:0/3575273286 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6ec153a00 0x7ff6ec14af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:30.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.174+0000 7ff6ebfff700 1 -- 172.21.15.31:0/3575273286 >> 172.21.15.31:0/3575273286 conn(0x7ff6ec0703a0 msgr2=0x7ff6ec15cc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:30.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.174+0000 7ff6ebfff700 1 -- 172.21.15.31:0/3575273286 shutdown_connections 2023-10-06T02:23:30.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.174+0000 7ff6ebfff700 1 -- 172.21.15.31:0/3575273286 wait complete. 2023-10-06T02:23:30.192 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T02:23:30.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.421+0000 7f6e66d9d700 1 Processor -- start 2023-10-06T02:23:30.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.421+0000 7f6e66d9d700 1 -- start start 2023-10-06T02:23:30.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.422+0000 7f6e66d9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f6e60153a00 0x7f6e60153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:30.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.422+0000 7f6e66d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6e600cea00 0x7f6e600cee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:30.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.422+0000 7f6e66d9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e6015ac50 con 0x7f6e6015de50 2023-10-06T02:23:30.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.422+0000 7f6e66d9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e6015d010 con 0x7f6e60153a00 2023-10-06T02:23:30.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.422+0000 7f6e66d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e600cf3c0 con 0x7f6e600cea00 2023-10-06T02:23:30.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.423+0000 7f6e6659c700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6e600cea00 0x7f6e600cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:30.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.423+0000 7f6e65d9b700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f6e60153a00 0x7f6e60153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:30.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.423+0000 7f6e65d9b700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f6e60153a00 0x7f6e60153dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:33760/0 (socket says 172.21.15.31:33760) 2023-10-06T02:23:30.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.423+0000 7f6e65d9b700 1 -- 172.21.15.31:0/2111514667 learned_addr learned my addr 172.21.15.31:0/2111514667 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:30.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.423+0000 7f6e6559a700 1 --1- 172.21.15.31:0/2111514667 >> v1:172.21.15.62:6789/0 conn(0x7f6e6015de50 0x7f6e6015a540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:38036/0 (socket says 172.21.15.31:38036) 2023-10-06T02:23:30.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.423+0000 7f6e65d9b700 1 -- 172.21.15.31:0/2111514667 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6e600cea00 msgr2=0x7f6e600cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:30.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.423+0000 7f6e65d9b700 1 --2- 172.21.15.31:0/2111514667 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6e600cea00 0x7f6e600cee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:30.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.423+0000 7f6e65d9b700 1 -- 172.21.15.31:0/2111514667 >> v1:172.21.15.62:6789/0 conn(0x7f6e6015de50 legacy=0x7f6e6015a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:30.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.423+0000 7f6e65d9b700 1 -- 172.21.15.31:0/2111514667 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e6015af60 con 0x7f6e60153a00 2023-10-06T02:23:30.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.424+0000 7f6e65d9b700 1 --2- 172.21.15.31:0/2111514667 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e60153a00 0x7f6e60153dd0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c0099b0 tx=0x7f6e5c009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78695990374a9907 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:30.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.424+0000 7f6e64d99700 1 -- 172.21.15.31:0/2111514667 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e5c009ea0 con 0x7f6e60153a00 2023-10-06T02:23:30.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.424+0000 7f6e64d99700 1 -- 172.21.15.31:0/2111514667 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e5c00b860 con 0x7f6e60153a00 2023-10-06T02:23:30.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.424+0000 7f6e64d99700 1 -- 172.21.15.31:0/2111514667 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e5c00bc00 con 0x7f6e60153a00 2023-10-06T02:23:30.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.424+0000 7f6e66d9d700 1 -- 172.21.15.31:0/2111514667 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e60153a00 msgr2=0x7f6e60153dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:30.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.424+0000 7f6e66d9d700 1 --2- 172.21.15.31:0/2111514667 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e60153a00 0x7f6e60153dd0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c0099b0 tx=0x7f6e5c009250 comp rx=0 tx=0).stop 2023-10-06T02:23:30.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.425+0000 7f6e66d9d700 1 -- 172.21.15.31:0/2111514667 shutdown_connections 2023-10-06T02:23:30.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.425+0000 7f6e66d9d700 1 --2- 172.21.15.31:0/2111514667 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6e600cea00 0x7f6e600cee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:30.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.425+0000 7f6e66d9d700 1 --2- 172.21.15.31:0/2111514667 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e60153a00 0x7f6e60153dd0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:30.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.425+0000 7f6e66d9d700 1 -- 172.21.15.31:0/2111514667 >> 172.21.15.31:0/2111514667 conn(0x7f6e600703a0 msgr2=0x7f6e600707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:30.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.425+0000 7f6e66d9d700 1 -- 172.21.15.31:0/2111514667 shutdown_connections 2023-10-06T02:23:30.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.425+0000 7f6e66d9d700 1 -- 172.21.15.31:0/2111514667 wait complete. 2023-10-06T02:23:30.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.426+0000 7f6e66d9d700 1 Processor -- start 2023-10-06T02:23:30.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.426+0000 7f6e66d9d700 1 -- start start 2023-10-06T02:23:30.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.427+0000 7f6e66d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e600cea00 0x7f6e600ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:30.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.427+0000 7f6e66d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6e60153a00 0x7f6e600c8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:30.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.427+0000 7f6e66d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e6015de50 0x7f6e6006d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:30.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.427+0000 7f6e66d9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e601565a0 con 0x7f6e600cea00 2023-10-06T02:23:30.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.427+0000 7f6e66d9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e600cd030 con 0x7f6e6015de50 2023-10-06T02:23:30.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.427+0000 7f6e66d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e60155990 con 0x7f6e60153a00 2023-10-06T02:23:30.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.427+0000 7f6e65d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e600cea00 0x7f6e600ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:30.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.427+0000 7f6e65d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e600cea00 0x7f6e600ccaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58850/0 (socket says 172.21.15.31:58850) 2023-10-06T02:23:30.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.427+0000 7f6e65d9b700 1 -- 172.21.15.31:0/3794308658 learned_addr learned my addr 172.21.15.31:0/3794308658 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:30.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.428+0000 7f6e6659c700 1 --2- 172.21.15.31:0/3794308658 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e6015de50 0x7f6e6006d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:30.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.428+0000 7f6e6559a700 1 --2- 172.21.15.31:0/3794308658 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6e60153a00 0x7f6e600c8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:30.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.428+0000 7f6e65d9b700 1 -- 172.21.15.31:0/3794308658 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6e60153a00 msgr2=0x7f6e600c8ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:30.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.428+0000 7f6e65d9b700 1 --2- 172.21.15.31:0/3794308658 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6e60153a00 0x7f6e600c8ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:30.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.428+0000 7f6e65d9b700 1 -- 172.21.15.31:0/3794308658 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e6015de50 msgr2=0x7f6e6006d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:30.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.428+0000 7f6e65d9b700 1 --2- 172.21.15.31:0/3794308658 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e6015de50 0x7f6e6006d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:30.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.428+0000 7f6e65d9b700 1 -- 172.21.15.31:0/3794308658 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e5c0094f0 con 0x7f6e600cea00 2023-10-06T02:23:30.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.428+0000 7f6e65d9b700 1 --2- 172.21.15.31:0/3794308658 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e600cea00 0x7f6e600ccaf0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6e6006eb00 tx=0x7f6e5c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:30.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.428+0000 7f6e55ffb700 1 -- 172.21.15.31:0/3794308658 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e5c000d10 con 0x7f6e600cea00 2023-10-06T02:23:30.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.428+0000 7f6e55ffb700 1 -- 172.21.15.31:0/3794308658 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e5c00bbe0 con 0x7f6e600cea00 2023-10-06T02:23:30.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.428+0000 7f6e55ffb700 1 -- 172.21.15.31:0/3794308658 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e5c011670 con 0x7f6e600cea00 2023-10-06T02:23:30.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.428+0000 7f6e66d9d700 1 -- 172.21.15.31:0/3794308658 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e60155c70 con 0x7f6e600cea00 2023-10-06T02:23:30.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.429+0000 7f6e66d9d700 1 -- 172.21.15.31:0/3794308658 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e60156230 con 0x7f6e600cea00 2023-10-06T02:23:30.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.430+0000 7f6e66d9d700 1 -- 172.21.15.31:0/3794308658 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e6004c3c0 con 0x7f6e600cea00 2023-10-06T02:23:30.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.430+0000 7f6e55ffb700 1 -- 172.21.15.31:0/3794308658 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6e5c011aa0 con 0x7f6e600cea00 2023-10-06T02:23:30.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.432+0000 7f6e55ffb700 1 --2- 172.21.15.31:0/3794308658 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6e44066f30 0x7f6e440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:30.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.432+0000 7f6e55ffb700 1 -- 172.21.15.31:0/3794308658 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6e5c085b70 con 0x7f6e600cea00 2023-10-06T02:23:30.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.432+0000 7f6e6559a700 1 --2- 172.21.15.31:0/3794308658 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6e44066f30 0x7f6e440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:30.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.432+0000 7f6e6559a700 1 --2- 172.21.15.31:0/3794308658 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6e44066f30 0x7f6e440693e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6e600ca5e0 tx=0x7f6e50007eb0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:30.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.436+0000 7f6e55ffb700 1 -- 172.21.15.31:0/3794308658 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6e5c052c60 con 0x7f6e600cea00 2023-10-06T02:23:30.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.635+0000 7f6e66d9d700 1 -- 172.21.15.31:0/3794308658 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6e600caef0 con 0x7f6e600cea00 2023-10-06T02:23:30.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.636+0000 7f6e55ffb700 1 -- 172.21.15.31:0/3794308658 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6e5c0150a0 con 0x7f6e600cea00 2023-10-06T02:23:30.636 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-10-06T02:23:30.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.638+0000 7f6e66d9d700 1 -- 172.21.15.31:0/3794308658 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6e44066f30 msgr2=0x7f6e440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:30.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.638+0000 7f6e66d9d700 1 --2- 172.21.15.31:0/3794308658 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6e44066f30 0x7f6e440693e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6e600ca5e0 tx=0x7f6e50007eb0 comp rx=0 tx=0).stop 2023-10-06T02:23:30.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.638+0000 7f6e66d9d700 1 -- 172.21.15.31:0/3794308658 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e600cea00 msgr2=0x7f6e600ccaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:30.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.638+0000 7f6e66d9d700 1 --2- 172.21.15.31:0/3794308658 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e600cea00 0x7f6e600ccaf0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6e6006eb00 tx=0x7f6e5c017000 comp rx=0 tx=0).stop 2023-10-06T02:23:30.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.638+0000 7f6e66d9d700 1 -- 172.21.15.31:0/3794308658 shutdown_connections 2023-10-06T02:23:30.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.638+0000 7f6e66d9d700 1 --2- 172.21.15.31:0/3794308658 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e6015de50 0x7f6e6006d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:30.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.638+0000 7f6e66d9d700 1 --2- 172.21.15.31:0/3794308658 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e600cea00 0x7f6e600ccaf0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:30.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.638+0000 7f6e66d9d700 1 --2- 172.21.15.31:0/3794308658 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6e44066f30 0x7f6e440693e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:30.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.638+0000 7f6e66d9d700 1 --2- 172.21.15.31:0/3794308658 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6e60153a00 0x7f6e600c8ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:30.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.638+0000 7f6e66d9d700 1 -- 172.21.15.31:0/3794308658 >> 172.21.15.31:0/3794308658 conn(0x7f6e600703a0 msgr2=0x7f6e600c6a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:30.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.638+0000 7f6e66d9d700 1 -- 172.21.15.31:0/3794308658 shutdown_connections 2023-10-06T02:23:30.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.638+0000 7f6e66d9d700 1 -- 172.21.15.31:0/3794308658 wait complete. 2023-10-06T02:23:30.656 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:23:30.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.841+0000 7f55c159d700 1 Processor -- start 2023-10-06T02:23:30.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.841+0000 7f55c159d700 1 -- start start 2023-10-06T02:23:30.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.842+0000 7f55c159d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f55bc159550 0x7f55bc0ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:30.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.842+0000 7f55c159d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55bc15dfd0 0x7f55bc0aedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:30.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.842+0000 7f55c159d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55bc0acc00 con 0x7f55bc159920 2023-10-06T02:23:30.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.842+0000 7f55c159d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55bc0abff0 con 0x7f55bc159550 2023-10-06T02:23:30.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.842+0000 7f55c159d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55bc0ac2d0 con 0x7f55bc15dfd0 2023-10-06T02:23:30.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.843+0000 7f55bbfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f55bc159550 0x7f55bc0ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:30.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.843+0000 7f55bbfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f55bc159550 0x7f55bc0ceac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58866/0 (socket says 172.21.15.31:58866) 2023-10-06T02:23:30.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.843+0000 7f55bbfff700 1 -- 172.21.15.31:0/4281875721 learned_addr learned my addr 172.21.15.31:0/4281875721 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:30.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.843+0000 7f55bb7fe700 1 --2- 172.21.15.31:0/4281875721 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55bc15dfd0 0x7f55bc0aedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:30.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.843+0000 7f55bbfff700 1 -- 172.21.15.31:0/4281875721 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55bc15dfd0 msgr2=0x7f55bc0aedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:30.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.843+0000 7f55bbfff700 1 --2- 172.21.15.31:0/4281875721 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55bc15dfd0 0x7f55bc0aedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:30.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.843+0000 7f55bbfff700 1 -- 172.21.15.31:0/4281875721 >> v1:172.21.15.62:6789/0 conn(0x7f55bc159920 legacy=0x7f55bc0ad090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:30.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.843+0000 7f55bbfff700 1 -- 172.21.15.31:0/4281875721 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55bc0ac5b0 con 0x7f55bc159550 2023-10-06T02:23:30.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.843+0000 7f55bbfff700 1 --2- 172.21.15.31:0/4281875721 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55bc159550 0x7f55bc0ceac0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f55ac0099b0 tx=0x7f55ac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=621ed1f9225cc9de server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:30.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.844+0000 7f55baffd700 1 -- 172.21.15.31:0/4281875721 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ac00b860 con 0x7f55bc159550 2023-10-06T02:23:30.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.844+0000 7f55baffd700 1 -- 172.21.15.31:0/4281875721 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55ac00b9c0 con 0x7f55bc159550 2023-10-06T02:23:30.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.844+0000 7f55baffd700 1 -- 172.21.15.31:0/4281875721 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ac00bd60 con 0x7f55bc159550 2023-10-06T02:23:30.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.844+0000 7f55c159d700 1 -- 172.21.15.31:0/4281875721 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55bc159550 msgr2=0x7f55bc0ceac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:30.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.844+0000 7f55c159d700 1 --2- 172.21.15.31:0/4281875721 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55bc159550 0x7f55bc0ceac0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f55ac0099b0 tx=0x7f55ac009250 comp rx=0 tx=0).stop 2023-10-06T02:23:30.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.844+0000 7f55c159d700 1 -- 172.21.15.31:0/4281875721 shutdown_connections 2023-10-06T02:23:30.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.844+0000 7f55c159d700 1 --2- 172.21.15.31:0/4281875721 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55bc159550 0x7f55bc0ceac0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:30.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.844+0000 7f55c159d700 1 --2- 172.21.15.31:0/4281875721 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55bc15dfd0 0x7f55bc0aedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:30.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.844+0000 7f55c159d700 1 -- 172.21.15.31:0/4281875721 >> 172.21.15.31:0/4281875721 conn(0x7f55bc06ce80 msgr2=0x7f55bc0ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:30.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.844+0000 7f55c159d700 1 -- 172.21.15.31:0/4281875721 shutdown_connections 2023-10-06T02:23:30.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.844+0000 7f55c159d700 1 -- 172.21.15.31:0/4281875721 wait complete. 2023-10-06T02:23:30.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.846+0000 7f55c159d700 1 Processor -- start 2023-10-06T02:23:30.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.846+0000 7f55c159d700 1 -- start start 2023-10-06T02:23:30.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.847+0000 7f55c159d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55bc159550 0x7f55bc142530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:30.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.847+0000 7f55c159d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55bc159920 0x7f55bc0cdee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:30.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.847+0000 7f55c159d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55bc15dfd0 0x7f55bc16ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:30.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.847+0000 7f55c159d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55bc0cda50 con 0x7f55bc15dfd0 2023-10-06T02:23:30.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.847+0000 7f55c159d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55bc16aa50 con 0x7f55bc159550 2023-10-06T02:23:30.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.847+0000 7f55c159d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55bc0cd400 con 0x7f55bc159920 2023-10-06T02:23:30.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.847+0000 7f55c0d9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55bc15dfd0 0x7f55bc16ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:30.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.847+0000 7f55c0d9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55bc15dfd0 0x7f55bc16ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58872/0 (socket says 172.21.15.31:58872) 2023-10-06T02:23:30.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.847+0000 7f55c0d9c700 1 -- 172.21.15.31:0/3625685356 learned_addr learned my addr 172.21.15.31:0/3625685356 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:30.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.847+0000 7f55bb7fe700 1 --2- 172.21.15.31:0/3625685356 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55bc159920 0x7f55bc0cdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:30.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.847+0000 7f55bbfff700 1 --2- 172.21.15.31:0/3625685356 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55bc159550 0x7f55bc142530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:30.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.847+0000 7f55c0d9c700 1 -- 172.21.15.31:0/3625685356 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55bc159920 msgr2=0x7f55bc0cdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:30.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.847+0000 7f55c0d9c700 1 --2- 172.21.15.31:0/3625685356 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55bc159920 0x7f55bc0cdee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:30.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.847+0000 7f55c0d9c700 1 -- 172.21.15.31:0/3625685356 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55bc159550 msgr2=0x7f55bc142530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:30.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.847+0000 7f55c0d9c700 1 --2- 172.21.15.31:0/3625685356 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55bc159550 0x7f55bc142530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:30.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.847+0000 7f55c0d9c700 1 -- 172.21.15.31:0/3625685356 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55ac0094f0 con 0x7f55bc15dfd0 2023-10-06T02:23:30.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.847+0000 7f55bb7fe700 1 --2- 172.21.15.31:0/3625685356 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55bc159920 0x7f55bc0cdee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:23:30.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.848+0000 7f55c0d9c700 1 --2- 172.21.15.31:0/3625685356 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55bc15dfd0 0x7f55bc16ad30 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f55b400c8e0 tx=0x7f55b4001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:30.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.848+0000 7f55a3fff700 1 -- 172.21.15.31:0/3625685356 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b400ce80 con 0x7f55bc15dfd0 2023-10-06T02:23:30.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.848+0000 7f55a3fff700 1 -- 172.21.15.31:0/3625685356 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55b400a7e0 con 0x7f55bc15dfd0 2023-10-06T02:23:30.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.848+0000 7f55a3fff700 1 -- 172.21.15.31:0/3625685356 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b400ab80 con 0x7f55bc15dfd0 2023-10-06T02:23:30.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.848+0000 7f55c159d700 1 -- 172.21.15.31:0/3625685356 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55bc0cd6e0 con 0x7f55bc15dfd0 2023-10-06T02:23:30.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.848+0000 7f55c159d700 1 -- 172.21.15.31:0/3625685356 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55bc142a70 con 0x7f55bc15dfd0 2023-10-06T02:23:30.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.849+0000 7f55c159d700 1 -- 172.21.15.31:0/3625685356 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55bc04c3c0 con 0x7f55bc15dfd0 2023-10-06T02:23:30.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.850+0000 7f55a3fff700 1 -- 172.21.15.31:0/3625685356 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f55b4012420 con 0x7f55bc15dfd0 2023-10-06T02:23:30.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.851+0000 7f55a3fff700 1 --2- 172.21.15.31:0/3625685356 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f55a8066f30 0x7f55a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:30.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.851+0000 7f55a3fff700 1 -- 172.21.15.31:0/3625685356 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f55b4084a80 con 0x7f55bc15dfd0 2023-10-06T02:23:30.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.852+0000 7f55bbfff700 1 --2- 172.21.15.31:0/3625685356 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f55a8066f30 0x7f55a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:30.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.852+0000 7f55bbfff700 1 --2- 172.21.15.31:0/3625685356 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f55a8066f30 0x7f55a80693e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f55ac0099b0 tx=0x7f55ac017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:30.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:30.855+0000 7f55a3fff700 1 -- 172.21.15.31:0/3625685356 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f55b4051af0 con 0x7f55bc15dfd0 2023-10-06T02:23:31.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.105+0000 7f55c159d700 1 -- 172.21.15.31:0/3625685356 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f55bc0cdb90 con 0x7f55bc15dfd0 2023-10-06T02:23:31.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.105+0000 7f55a3fff700 1 -- 172.21.15.31:0/3625685356 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4591 (secure 0 0 0) 0x7f55b4055110 con 0x7f55bc15dfd0 2023-10-06T02:23:31.106 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:23:31.106 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/1808327183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":1808327183},{"type":"v1","addr":"172.21.15.31:6837","nonce":1808327183}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/4263144517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":4263144517},{"type":"v1","addr":"172.21.15.62:6834","nonce":4263144517}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1489208568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":1489208568},{"type":"v1","addr":"172.21.15.62:6835","nonce":1489208568}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:23:09.521236+0000","modified":"2023-10-06T02:23:13.047274+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6836/3809284664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3809284664},{"type":"v1","addr":"172.21.15.31:6836","nonce":3809284664}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T02:23:31.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.108+0000 7f55c159d700 1 -- 172.21.15.31:0/3625685356 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f55a8066f30 msgr2=0x7f55a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:31.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.108+0000 7f55c159d700 1 --2- 172.21.15.31:0/3625685356 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f55a8066f30 0x7f55a80693e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f55ac0099b0 tx=0x7f55ac017000 comp rx=0 tx=0).stop 2023-10-06T02:23:31.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.108+0000 7f55c159d700 1 -- 172.21.15.31:0/3625685356 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55bc15dfd0 msgr2=0x7f55bc16ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:31.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.108+0000 7f55c159d700 1 --2- 172.21.15.31:0/3625685356 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55bc15dfd0 0x7f55bc16ad30 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f55b400c8e0 tx=0x7f55b4001710 comp rx=0 tx=0).stop 2023-10-06T02:23:31.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.108+0000 7f55c159d700 1 -- 172.21.15.31:0/3625685356 shutdown_connections 2023-10-06T02:23:31.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.108+0000 7f55c159d700 1 --2- 172.21.15.31:0/3625685356 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55bc15dfd0 0x7f55bc16ad30 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:31.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.108+0000 7f55c159d700 1 --2- 172.21.15.31:0/3625685356 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55bc159550 0x7f55bc142530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:31.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.108+0000 7f55c159d700 1 --2- 172.21.15.31:0/3625685356 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f55a8066f30 0x7f55a80693e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:31.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.108+0000 7f55c159d700 1 --2- 172.21.15.31:0/3625685356 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55bc159920 0x7f55bc0cdee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:31.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.108+0000 7f55c159d700 1 -- 172.21.15.31:0/3625685356 >> 172.21.15.31:0/3625685356 conn(0x7f55bc06ce80 msgr2=0x7f55bc0afb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:31.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.108+0000 7f55c159d700 1 -- 172.21.15.31:0/3625685356 shutdown_connections 2023-10-06T02:23:31.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.108+0000 7f55c159d700 1 -- 172.21.15.31:0/3625685356 wait complete. 2023-10-06T02:23:31.109 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 8 2023-10-06T02:23:31.126 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:23:31.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.313+0000 7f5d6f59d700 1 Processor -- start 2023-10-06T02:23:31.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.313+0000 7f5d6f59d700 1 -- start start 2023-10-06T02:23:31.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.315+0000 7f5d6f59d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f5d6813fef0 0x7f5d6815a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:31.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.315+0000 7f5d6f59d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5d6806aaa0 0x7f5d680c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:31.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.315+0000 7f5d6f59d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d6815ae70 con 0x7f5d6813fb20 2023-10-06T02:23:31.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.315+0000 7f5d6f59d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d6806af60 con 0x7f5d6813fef0 2023-10-06T02:23:31.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.315+0000 7f5d6f59d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d6806b240 con 0x7f5d6806aaa0 2023-10-06T02:23:31.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.315+0000 7f5d6ed9c700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5d6806aaa0 0x7f5d680c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:31.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.315+0000 7f5d6dd9a700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f5d6813fef0 0x7f5d6815a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:31.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.315+0000 7f5d6ed9c700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5d6806aaa0 0x7f5d680c9d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:43238/0 (socket says 172.21.15.31:43238) 2023-10-06T02:23:31.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.315+0000 7f5d6dd9a700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f5d6813fef0 0x7f5d6815a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:33770/0 (socket says 172.21.15.31:33770) 2023-10-06T02:23:31.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.315+0000 7f5d6ed9c700 1 -- 172.21.15.31:0/3052124948 learned_addr learned my addr 172.21.15.31:0/3052124948 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:31.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.315+0000 7f5d6e59b700 1 --1- 172.21.15.31:0/3052124948 >> v1:172.21.15.62:6789/0 conn(0x7f5d6813fb20 0x7f5d680c7990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:38058/0 (socket says 172.21.15.31:38058) 2023-10-06T02:23:31.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.316+0000 7f5d6d599700 1 -- 172.21.15.31:0/3052124948 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2962385512 0 0) 0x7f5d64001110 con 0x7f5d6813fb20 2023-10-06T02:23:31.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.316+0000 7f5d6d599700 1 -- 172.21.15.31:0/3052124948 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d54003570 con 0x7f5d6813fb20 2023-10-06T02:23:31.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.316+0000 7f5d6ed9c700 1 -- 172.21.15.31:0/3052124948 >> v2:172.21.15.62:3300/0 conn(0x7f5d6813fef0 msgr2=0x7f5d6815a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:31.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.316+0000 7f5d6ed9c700 1 --2- 172.21.15.31:0/3052124948 >> v2:172.21.15.62:3300/0 conn(0x7f5d6813fef0 0x7f5d6815a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:31.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.316+0000 7f5d6ed9c700 1 -- 172.21.15.31:0/3052124948 >> v1:172.21.15.62:6789/0 conn(0x7f5d6813fb20 legacy=0x7f5d680c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:31.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.316+0000 7f5d6ed9c700 1 -- 172.21.15.31:0/3052124948 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d6806b520 con 0x7f5d6806aaa0 2023-10-06T02:23:31.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.316+0000 7f5d6dd9a700 1 --2- 172.21.15.31:0/3052124948 >> v2:172.21.15.62:3300/0 conn(0x7f5d6813fef0 0x7f5d6815a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:23:31.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.316+0000 7f5d6ed9c700 1 --2- 172.21.15.31:0/3052124948 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5d6806aaa0 0x7f5d680c9d00 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5d60009f20 tx=0x7f5d60009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fea6f41d8b31c205 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:31.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.316+0000 7f5d6d599700 1 -- 172.21.15.31:0/3052124948 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d6000e070 con 0x7f5d6806aaa0 2023-10-06T02:23:31.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.316+0000 7f5d6d599700 1 -- 172.21.15.31:0/3052124948 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d6000b970 con 0x7f5d6806aaa0 2023-10-06T02:23:31.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.316+0000 7f5d6d599700 1 -- 172.21.15.31:0/3052124948 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d600092e0 con 0x7f5d6806aaa0 2023-10-06T02:23:31.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.317+0000 7f5d6f59d700 1 -- 172.21.15.31:0/3052124948 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5d6806aaa0 msgr2=0x7f5d680c9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:31.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.317+0000 7f5d6f59d700 1 --2- 172.21.15.31:0/3052124948 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5d6806aaa0 0x7f5d680c9d00 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5d60009f20 tx=0x7f5d60009840 comp rx=0 tx=0).stop 2023-10-06T02:23:31.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.317+0000 7f5d6f59d700 1 -- 172.21.15.31:0/3052124948 shutdown_connections 2023-10-06T02:23:31.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.317+0000 7f5d6f59d700 1 --2- 172.21.15.31:0/3052124948 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5d6806aaa0 0x7f5d680c9d00 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:31.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.317+0000 7f5d6f59d700 1 --2- 172.21.15.31:0/3052124948 >> v2:172.21.15.62:3300/0 conn(0x7f5d6813fef0 0x7f5d6815a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:31.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.317+0000 7f5d6f59d700 1 -- 172.21.15.31:0/3052124948 >> 172.21.15.31:0/3052124948 conn(0x7f5d6806d170 msgr2=0x7f5d680cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:31.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.317+0000 7f5d6f59d700 1 -- 172.21.15.31:0/3052124948 shutdown_connections 2023-10-06T02:23:31.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.317+0000 7f5d6f59d700 1 -- 172.21.15.31:0/3052124948 wait complete. 2023-10-06T02:23:31.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.318+0000 7f5d6f59d700 1 Processor -- start 2023-10-06T02:23:31.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.318+0000 7f5d6f59d700 1 -- start start 2023-10-06T02:23:31.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.319+0000 7f5d6f59d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d6806aaa0 0x7f5d6815b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:31.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.319+0000 7f5d6f59d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5d6813fb20 0x7f5d6813ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:31.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.319+0000 7f5d6f59d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d6813fef0 0x7f5d6813b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:31.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.319+0000 7f5d6f59d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d68168710 con 0x7f5d6813fef0 2023-10-06T02:23:31.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.319+0000 7f5d6f59d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d681655a0 con 0x7f5d6806aaa0 2023-10-06T02:23:31.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.319+0000 7f5d6f59d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d68165880 con 0x7f5d6813fb20 2023-10-06T02:23:31.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.320+0000 7f5d6ed9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d6813fef0 0x7f5d6813b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:31.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.320+0000 7f5d6ed9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d6813fef0 0x7f5d6813b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58878/0 (socket says 172.21.15.31:58878) 2023-10-06T02:23:31.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.320+0000 7f5d6ed9c700 1 -- 172.21.15.31:0/337124472 learned_addr learned my addr 172.21.15.31:0/337124472 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:31.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.320+0000 7f5d6e59b700 1 --2- 172.21.15.31:0/337124472 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d6806aaa0 0x7f5d6815b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:31.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.320+0000 7f5d6dd9a700 1 --2- 172.21.15.31:0/337124472 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5d6813fb20 0x7f5d6813ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:31.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.320+0000 7f5d6ed9c700 1 -- 172.21.15.31:0/337124472 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5d6813fb20 msgr2=0x7f5d6813ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:31.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.320+0000 7f5d6ed9c700 1 --2- 172.21.15.31:0/337124472 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5d6813fb20 0x7f5d6813ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:31.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.320+0000 7f5d6ed9c700 1 -- 172.21.15.31:0/337124472 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d6806aaa0 msgr2=0x7f5d6815b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:31.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.320+0000 7f5d6ed9c700 1 --2- 172.21.15.31:0/337124472 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d6806aaa0 0x7f5d6815b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:31.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.320+0000 7f5d6ed9c700 1 -- 172.21.15.31:0/337124472 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d5800d860 con 0x7f5d6813fef0 2023-10-06T02:23:31.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.320+0000 7f5d6ed9c700 1 --2- 172.21.15.31:0/337124472 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d6813fef0 0x7f5d6813b280 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5d6806b380 tx=0x7f5d60018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:31.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.321+0000 7f5d5e7fc700 1 -- 172.21.15.31:0/337124472 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d60008ad0 con 0x7f5d6813fef0 2023-10-06T02:23:31.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.321+0000 7f5d5e7fc700 1 -- 172.21.15.31:0/337124472 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d60008c30 con 0x7f5d6813fef0 2023-10-06T02:23:31.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.321+0000 7f5d6f59d700 1 -- 172.21.15.31:0/337124472 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d60009ae0 con 0x7f5d6813fef0 2023-10-06T02:23:31.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.321+0000 7f5d5e7fc700 1 -- 172.21.15.31:0/337124472 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d6001c410 con 0x7f5d6813fef0 2023-10-06T02:23:31.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.321+0000 7f5d6f59d700 1 -- 172.21.15.31:0/337124472 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d68165e40 con 0x7f5d6813fef0 2023-10-06T02:23:31.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.322+0000 7f5d6f59d700 1 -- 172.21.15.31:0/337124472 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d6804c3c0 con 0x7f5d6813fef0 2023-10-06T02:23:31.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.323+0000 7f5d5e7fc700 1 -- 172.21.15.31:0/337124472 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5d60003040 con 0x7f5d6813fef0 2023-10-06T02:23:31.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.324+0000 7f5d5e7fc700 1 --2- 172.21.15.31:0/337124472 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5d4c066f30 0x7f5d4c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:31.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.324+0000 7f5d5e7fc700 1 -- 172.21.15.31:0/337124472 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f5d60085da0 con 0x7f5d6813fef0 2023-10-06T02:23:31.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.324+0000 7f5d6e59b700 1 --2- 172.21.15.31:0/337124472 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5d4c066f30 0x7f5d4c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:31.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.325+0000 7f5d6e59b700 1 --2- 172.21.15.31:0/337124472 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5d4c066f30 0x7f5d4c0693e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5d64004e10 tx=0x7f5d64007bb0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:31.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.328+0000 7f5d5e7fc700 1 -- 172.21.15.31:0/337124472 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5d60052e10 con 0x7f5d6813fef0 2023-10-06T02:23:31.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.531+0000 7f5d6f59d700 1 -- 172.21.15.31:0/337124472 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5d680cc870 con 0x7f5d6813fef0 2023-10-06T02:23:31.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.532+0000 7f5d5e7fc700 1 -- 172.21.15.31:0/337124472 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16818 (secure 0 0 0) 0x7f5d60021090 con 0x7f5d6813fef0 2023-10-06T02:23:31.533 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:23:31.533 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":23,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:23:10.970534+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:23:07.290269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T02:23:08.504679+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:23:31.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.535+0000 7f5d6f59d700 1 -- 172.21.15.31:0/337124472 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5d4c066f30 msgr2=0x7f5d4c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:31.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.535+0000 7f5d6f59d700 1 --2- 172.21.15.31:0/337124472 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5d4c066f30 0x7f5d4c0693e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5d64004e10 tx=0x7f5d64007bb0 comp rx=0 tx=0).stop 2023-10-06T02:23:31.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.535+0000 7f5d6f59d700 1 -- 172.21.15.31:0/337124472 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d6813fef0 msgr2=0x7f5d6813b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:31.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.535+0000 7f5d6f59d700 1 --2- 172.21.15.31:0/337124472 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d6813fef0 0x7f5d6813b280 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5d6806b380 tx=0x7f5d60018000 comp rx=0 tx=0).stop 2023-10-06T02:23:31.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.535+0000 7f5d6f59d700 1 -- 172.21.15.31:0/337124472 shutdown_connections 2023-10-06T02:23:31.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.535+0000 7f5d6f59d700 1 --2- 172.21.15.31:0/337124472 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d6813fef0 0x7f5d6813b280 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:31.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.535+0000 7f5d6f59d700 1 --2- 172.21.15.31:0/337124472 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d6806aaa0 0x7f5d6815b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:31.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.535+0000 7f5d6f59d700 1 --2- 172.21.15.31:0/337124472 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5d4c066f30 0x7f5d4c0693e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:31.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.535+0000 7f5d6f59d700 1 --2- 172.21.15.31:0/337124472 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5d6813fb20 0x7f5d6813ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:31.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.535+0000 7f5d6f59d700 1 -- 172.21.15.31:0/337124472 >> 172.21.15.31:0/337124472 conn(0x7f5d6806d170 msgr2=0x7f5d680cbd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:31.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.535+0000 7f5d6f59d700 1 -- 172.21.15.31:0/337124472 shutdown_connections 2023-10-06T02:23:31.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.535+0000 7f5d6f59d700 1 -- 172.21.15.31:0/337124472 wait complete. 2023-10-06T02:23:31.552 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:23:31.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.739+0000 7f7de579d700 1 Processor -- start 2023-10-06T02:23:31.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.739+0000 7f7de579d700 1 -- start start 2023-10-06T02:23:31.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.740+0000 7f7de579d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7de0159550 0x7f7de00ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:31.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.740+0000 7f7de579d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7de015dfd0 0x7f7de00aedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:31.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.740+0000 7f7de579d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7de00acc00 con 0x7f7de0159920 2023-10-06T02:23:31.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.740+0000 7f7de579d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7de00abff0 con 0x7f7de0159550 2023-10-06T02:23:31.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.740+0000 7f7de579d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7de00ac2d0 con 0x7f7de015dfd0 2023-10-06T02:23:31.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.740+0000 7f7ddffff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7de0159550 0x7f7de00ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:31.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.740+0000 7f7de4f9c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f7de0159920 0x7f7de00ad090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:41800/0 (socket says 172.21.15.31:41800) 2023-10-06T02:23:31.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.740+0000 7f7de4f9c700 1 -- 172.21.15.31:0/1114584561 learned_addr learned my addr 172.21.15.31:0/1114584561 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:23:31.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.740+0000 7f7ddffff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7de0159550 0x7f7de00ceac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58886/0 (socket says 172.21.15.31:58886) 2023-10-06T02:23:31.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.741+0000 7f7ddf7fe700 1 --2- 172.21.15.31:0/1114584561 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7de015dfd0 0x7f7de00aedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:31.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.741+0000 7f7ddffff700 1 -- 172.21.15.31:0/1114584561 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7de015dfd0 msgr2=0x7f7de00aedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:31.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.741+0000 7f7ddffff700 1 --2- 172.21.15.31:0/1114584561 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7de015dfd0 0x7f7de00aedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:31.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.741+0000 7f7ddffff700 1 -- 172.21.15.31:0/1114584561 >> v1:172.21.15.31:6789/0 conn(0x7f7de0159920 legacy=0x7f7de00ad090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:31.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.741+0000 7f7ddffff700 1 -- 172.21.15.31:0/1114584561 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7de00ac5b0 con 0x7f7de0159550 2023-10-06T02:23:31.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.741+0000 7f7ddf7fe700 1 --2- 172.21.15.31:0/1114584561 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7de015dfd0 0x7f7de00aedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:23:31.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.741+0000 7f7ddeffd700 1 -- 172.21.15.31:0/1114584561 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2830270788 0 0) 0x7f7de00acc00 con 0x7f7de0159920 2023-10-06T02:23:31.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.741+0000 7f7ddffff700 1 --2- 172.21.15.31:0/1114584561 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7de0159550 0x7f7de00ceac0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7dd00098e0 tx=0x7f7dd00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99d78fcdc6e864e5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:31.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.741+0000 7f7ddeffd700 1 -- 172.21.15.31:0/1114584561 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd0009e80 con 0x7f7de0159550 2023-10-06T02:23:31.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.741+0000 7f7ddeffd700 1 -- 172.21.15.31:0/1114584561 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dd000b860 con 0x7f7de0159550 2023-10-06T02:23:31.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.741+0000 7f7ddeffd700 1 -- 172.21.15.31:0/1114584561 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd000bbe0 con 0x7f7de0159550 2023-10-06T02:23:31.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.742+0000 7f7de579d700 1 -- 172.21.15.31:0/1114584561 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7de0159550 msgr2=0x7f7de00ceac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:31.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.742+0000 7f7de579d700 1 --2- 172.21.15.31:0/1114584561 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7de0159550 0x7f7de00ceac0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7dd00098e0 tx=0x7f7dd00091a0 comp rx=0 tx=0).stop 2023-10-06T02:23:31.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.742+0000 7f7de579d700 1 -- 172.21.15.31:0/1114584561 shutdown_connections 2023-10-06T02:23:31.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.742+0000 7f7de579d700 1 --2- 172.21.15.31:0/1114584561 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7de0159550 0x7f7de00ceac0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:31.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.742+0000 7f7de579d700 1 --2- 172.21.15.31:0/1114584561 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7de015dfd0 0x7f7de00aedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:31.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.742+0000 7f7de579d700 1 -- 172.21.15.31:0/1114584561 >> 172.21.15.31:0/1114584561 conn(0x7f7de006ce80 msgr2=0x7f7de00ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:31.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.742+0000 7f7de579d700 1 -- 172.21.15.31:0/1114584561 shutdown_connections 2023-10-06T02:23:31.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.742+0000 7f7de579d700 1 -- 172.21.15.31:0/1114584561 wait complete. 2023-10-06T02:23:31.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.743+0000 7f7de579d700 1 Processor -- start 2023-10-06T02:23:31.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.743+0000 7f7de579d700 1 -- start start 2023-10-06T02:23:31.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.744+0000 7f7de579d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7de0159550 0x7f7de00ce210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:31.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.744+0000 7f7de579d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7de0159920 0x7f7de00b35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:31.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.744+0000 7f7de579d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7de015dfd0 0x7f7de00b3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:31.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.744+0000 7f7de579d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7de00ceb40 con 0x7f7de015dfd0 2023-10-06T02:23:31.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.744+0000 7f7de579d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7de016aaf0 con 0x7f7de0159550 2023-10-06T02:23:31.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.744+0000 7f7de579d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7de016a930 con 0x7f7de0159920 2023-10-06T02:23:31.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.745+0000 7f7ddffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7de0159550 0x7f7de00ce210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:31.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.745+0000 7f7de4f9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7de015dfd0 0x7f7de00b3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:31.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.745+0000 7f7ddffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7de0159550 0x7f7de00ce210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:33784/0 (socket says 172.21.15.31:33784) 2023-10-06T02:23:31.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.745+0000 7f7ddffff700 1 -- 172.21.15.31:0/3232275311 learned_addr learned my addr 172.21.15.31:0/3232275311 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:31.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.745+0000 7f7ddf7fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7de0159920 0x7f7de00b35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:31.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.745+0000 7f7de4f9c700 1 -- 172.21.15.31:0/3232275311 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7de0159920 msgr2=0x7f7de00b35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:31.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.745+0000 7f7de4f9c700 1 --2- 172.21.15.31:0/3232275311 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7de0159920 0x7f7de00b35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:31.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.745+0000 7f7de4f9c700 1 -- 172.21.15.31:0/3232275311 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7de0159550 msgr2=0x7f7de00ce210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:31.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.745+0000 7f7de4f9c700 1 --2- 172.21.15.31:0/3232275311 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7de0159550 0x7f7de00ce210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:31.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.745+0000 7f7de4f9c700 1 -- 172.21.15.31:0/3232275311 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dd0009420 con 0x7f7de015dfd0 2023-10-06T02:23:31.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.745+0000 7f7de4f9c700 1 --2- 172.21.15.31:0/3232275311 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7de015dfd0 0x7f7de00b3b20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7dd800cbc0 tx=0x7f7dd8001980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:31.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.746+0000 7f7dc7fff700 1 -- 172.21.15.31:0/3232275311 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd800a740 con 0x7f7de015dfd0 2023-10-06T02:23:31.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.746+0000 7f7dc7fff700 1 -- 172.21.15.31:0/3232275311 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dd800a8a0 con 0x7f7de015dfd0 2023-10-06T02:23:31.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.746+0000 7f7dc7fff700 1 -- 172.21.15.31:0/3232275311 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd800ac60 con 0x7f7de015dfd0 2023-10-06T02:23:31.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.746+0000 7f7de579d700 1 -- 172.21.15.31:0/3232275311 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7de0169920 con 0x7f7de015dfd0 2023-10-06T02:23:31.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.746+0000 7f7de579d700 1 -- 172.21.15.31:0/3232275311 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7de016a4a0 con 0x7f7de015dfd0 2023-10-06T02:23:31.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.747+0000 7f7de579d700 1 -- 172.21.15.31:0/3232275311 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7de016a630 con 0x7f7de015dfd0 2023-10-06T02:23:31.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.748+0000 7f7dc7fff700 1 -- 172.21.15.31:0/3232275311 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7dd8007300 con 0x7f7de015dfd0 2023-10-06T02:23:31.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.748+0000 7f7dc7fff700 1 --2- 172.21.15.31:0/3232275311 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f7dcc066f30 0x7f7dcc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:31.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.749+0000 7f7dc7fff700 1 -- 172.21.15.31:0/3232275311 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f7dd8084970 con 0x7f7de015dfd0 2023-10-06T02:23:31.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.749+0000 7f7ddffff700 1 --2- 172.21.15.31:0/3232275311 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f7dcc066f30 0x7f7dcc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:31.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.749+0000 7f7ddffff700 1 --2- 172.21.15.31:0/3232275311 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f7dcc066f30 0x7f7dcc0693e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7dd0003080 tx=0x7f7dd0017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:31.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.753+0000 7f7dc7fff700 1 -- 172.21.15.31:0/3232275311 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7dd80519d0 con 0x7f7de015dfd0 2023-10-06T02:23:31.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.997+0000 7f7de579d700 1 -- 172.21.15.31:0/3232275311 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7de004c3c0 con 0x7f7de015dfd0 2023-10-06T02:23:31.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:31.998+0000 7f7dc7fff700 1 -- 172.21.15.31:0/3232275311 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4591 (secure 0 0 0) 0x7f7dd8054ff0 con 0x7f7de015dfd0 2023-10-06T02:23:31.998 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:23:31.998 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/1808327183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":1808327183},{"type":"v1","addr":"172.21.15.31:6837","nonce":1808327183}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/4263144517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":4263144517},{"type":"v1","addr":"172.21.15.62:6834","nonce":4263144517}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1489208568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":1489208568},{"type":"v1","addr":"172.21.15.62:6835","nonce":1489208568}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:23:09.521236+0000","modified":"2023-10-06T02:23:13.047274+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6836/3809284664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3809284664},{"type":"v1","addr":"172.21.15.31:6836","nonce":3809284664}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T02:23:32.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.000+0000 7f7de579d700 1 -- 172.21.15.31:0/3232275311 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f7dcc066f30 msgr2=0x7f7dcc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:32.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.000+0000 7f7de579d700 1 --2- 172.21.15.31:0/3232275311 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f7dcc066f30 0x7f7dcc0693e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7dd0003080 tx=0x7f7dd0017000 comp rx=0 tx=0).stop 2023-10-06T02:23:32.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.000+0000 7f7de579d700 1 -- 172.21.15.31:0/3232275311 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7de015dfd0 msgr2=0x7f7de00b3b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:32.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.000+0000 7f7de579d700 1 --2- 172.21.15.31:0/3232275311 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7de015dfd0 0x7f7de00b3b20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7dd800cbc0 tx=0x7f7dd8001980 comp rx=0 tx=0).stop 2023-10-06T02:23:32.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.000+0000 7f7de579d700 1 -- 172.21.15.31:0/3232275311 shutdown_connections 2023-10-06T02:23:32.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.000+0000 7f7de579d700 1 --2- 172.21.15.31:0/3232275311 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7de015dfd0 0x7f7de00b3b20 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:32.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.000+0000 7f7de579d700 1 --2- 172.21.15.31:0/3232275311 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7de0159550 0x7f7de00ce210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:32.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.000+0000 7f7de579d700 1 --2- 172.21.15.31:0/3232275311 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f7dcc066f30 0x7f7dcc0693e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:32.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.000+0000 7f7de579d700 1 --2- 172.21.15.31:0/3232275311 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7de0159920 0x7f7de00b35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:32.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.000+0000 7f7de579d700 1 -- 172.21.15.31:0/3232275311 >> 172.21.15.31:0/3232275311 conn(0x7f7de006ce80 msgr2=0x7f7de00af980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:32.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.000+0000 7f7de579d700 1 -- 172.21.15.31:0/3232275311 shutdown_connections 2023-10-06T02:23:32.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.000+0000 7f7de579d700 1 -- 172.21.15.31:0/3232275311 wait complete. 2023-10-06T02:23:32.002 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 8 2023-10-06T02:23:32.420 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T02:23:09.521236+0000', 'modified': '2023-10-06T02:23:13.047274+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4297}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4297': {'gid': 4297, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.31:6836/3809284664', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6834', 'nonce': 3809284664}, {'type': 'v1', 'addr': '172.21.15.31:6836', 'nonce': 3809284664}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T02:23:32.420 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T02:23:32.420 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-06T02:23:32.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.609+0000 7f386319d700 1 Processor -- start 2023-10-06T02:23:32.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.609+0000 7f386319d700 1 -- start start 2023-10-06T02:23:32.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.611+0000 7f386319d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f385c142c00 0x7f385c06b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:32.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.611+0000 7f386319d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f385c06bd20 0x7f385c0c7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:32.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.611+0000 7f386319d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f385c142fd0 0x7f385c0c7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:32.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.611+0000 7f386319d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f385c0ca480 con 0x7f385c06bd20 2023-10-06T02:23:32.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.611+0000 7f386319d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f385c15e240 con 0x7f385c142fd0 2023-10-06T02:23:32.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.611+0000 7f386319d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f385c06b350 con 0x7f385c142c00 2023-10-06T02:23:32.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.611+0000 7f386199a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f385c06bd20 0x7f385c0c7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:32.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.611+0000 7f386219b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f385c142c00 0x7f385c06b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:32.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.611+0000 7f386199a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f385c06bd20 0x7f385c0c7880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58918/0 (socket says 172.21.15.31:58918) 2023-10-06T02:23:32.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.611+0000 7f386199a700 1 -- 172.21.15.31:0/3693083805 learned_addr learned my addr 172.21.15.31:0/3693083805 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:32.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.611+0000 7f386299c700 1 --2- 172.21.15.31:0/3693083805 >> v2:172.21.15.62:3300/0 conn(0x7f385c142fd0 0x7f385c0c7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:32.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.611+0000 7f386199a700 1 -- 172.21.15.31:0/3693083805 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f385c142c00 msgr2=0x7f385c06b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:32.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.611+0000 7f386199a700 1 --2- 172.21.15.31:0/3693083805 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f385c142c00 0x7f385c06b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:32.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.611+0000 7f386199a700 1 -- 172.21.15.31:0/3693083805 >> v2:172.21.15.62:3300/0 conn(0x7f385c142fd0 msgr2=0x7f385c0c7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:32.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.611+0000 7f386199a700 1 --2- 172.21.15.31:0/3693083805 >> v2:172.21.15.62:3300/0 conn(0x7f385c142fd0 0x7f385c0c7dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:32.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.611+0000 7f386199a700 1 -- 172.21.15.31:0/3693083805 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f385c15df60 con 0x7f385c06bd20 2023-10-06T02:23:32.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.612+0000 7f386199a700 1 --2- 172.21.15.31:0/3693083805 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f385c06bd20 0x7f385c0c7880 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f385c0ca8c0 tx=0x7f384c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca3da92003632d2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:32.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.612+0000 7f3861199700 1 -- 172.21.15.31:0/3693083805 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f384c00b640 con 0x7f385c06bd20 2023-10-06T02:23:32.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.612+0000 7f3861199700 1 -- 172.21.15.31:0/3693083805 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f384c00b7a0 con 0x7f385c06bd20 2023-10-06T02:23:32.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.612+0000 7f3861199700 1 -- 172.21.15.31:0/3693083805 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f384c00bb40 con 0x7f385c06bd20 2023-10-06T02:23:32.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.612+0000 7f386319d700 1 -- 172.21.15.31:0/3693083805 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f385c06bd20 msgr2=0x7f385c0c7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:32.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.613+0000 7f386319d700 1 --2- 172.21.15.31:0/3693083805 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f385c06bd20 0x7f385c0c7880 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f385c0ca8c0 tx=0x7f384c009250 comp rx=0 tx=0).stop 2023-10-06T02:23:32.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.613+0000 7f386319d700 1 -- 172.21.15.31:0/3693083805 shutdown_connections 2023-10-06T02:23:32.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.613+0000 7f386319d700 1 --2- 172.21.15.31:0/3693083805 >> v2:172.21.15.62:3300/0 conn(0x7f385c142fd0 0x7f385c0c7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:32.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.613+0000 7f386319d700 1 --2- 172.21.15.31:0/3693083805 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f385c142c00 0x7f385c06b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:32.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.613+0000 7f386319d700 1 --2- 172.21.15.31:0/3693083805 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f385c06bd20 0x7f385c0c7880 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:32.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.613+0000 7f386319d700 1 -- 172.21.15.31:0/3693083805 >> 172.21.15.31:0/3693083805 conn(0x7f385c0675a0 msgr2=0x7f385c0679a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:32.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.613+0000 7f386319d700 1 -- 172.21.15.31:0/3693083805 shutdown_connections 2023-10-06T02:23:32.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.613+0000 7f386319d700 1 -- 172.21.15.31:0/3693083805 wait complete. 2023-10-06T02:23:32.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.614+0000 7f386319d700 1 Processor -- start 2023-10-06T02:23:32.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.614+0000 7f386319d700 1 -- start start 2023-10-06T02:23:32.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.615+0000 7f386319d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f385c06bd20 0x7f385c0c6b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:32.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.615+0000 7f386319d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f385c142c00 0x7f385c0c5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:32.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.615+0000 7f386319d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f385c142fd0 0x7f385c16c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:32.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.615+0000 7f386319d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f385c0c54c0 con 0x7f385c06bd20 2023-10-06T02:23:32.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.615+0000 7f386319d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f385c0c70c0 con 0x7f385c142fd0 2023-10-06T02:23:32.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.615+0000 7f386319d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f385c0c48b0 con 0x7f385c142c00 2023-10-06T02:23:32.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.615+0000 7f386219b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f385c06bd20 0x7f385c0c6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:32.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.615+0000 7f386219b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f385c06bd20 0x7f385c0c6b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58932/0 (socket says 172.21.15.31:58932) 2023-10-06T02:23:32.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.615+0000 7f386219b700 1 -- 172.21.15.31:0/1844846549 learned_addr learned my addr 172.21.15.31:0/1844846549 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:32.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.616+0000 7f386299c700 1 --2- 172.21.15.31:0/1844846549 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f385c142fd0 0x7f385c16c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:32.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.616+0000 7f386199a700 1 --2- 172.21.15.31:0/1844846549 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f385c142c00 0x7f385c0c5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:32.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.616+0000 7f386219b700 1 -- 172.21.15.31:0/1844846549 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f385c142c00 msgr2=0x7f385c0c5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:32.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.616+0000 7f386219b700 1 --2- 172.21.15.31:0/1844846549 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f385c142c00 0x7f385c0c5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:32.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.616+0000 7f386219b700 1 -- 172.21.15.31:0/1844846549 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f385c142fd0 msgr2=0x7f385c16c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:32.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.616+0000 7f386219b700 1 --2- 172.21.15.31:0/1844846549 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f385c142fd0 0x7f385c16c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:32.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.616+0000 7f386219b700 1 -- 172.21.15.31:0/1844846549 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f384c0094f0 con 0x7f385c06bd20 2023-10-06T02:23:32.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.616+0000 7f386219b700 1 --2- 172.21.15.31:0/1844846549 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f385c06bd20 0x7f385c0c6b80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f385800ec80 tx=0x7f38580091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:32.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.616+0000 7f38527fc700 1 -- 172.21.15.31:0/1844846549 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385800c850 con 0x7f385c06bd20 2023-10-06T02:23:32.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.616+0000 7f38527fc700 1 -- 172.21.15.31:0/1844846549 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f385800c9d0 con 0x7f385c06bd20 2023-10-06T02:23:32.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.616+0000 7f38527fc700 1 -- 172.21.15.31:0/1844846549 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385800cd90 con 0x7f385c06bd20 2023-10-06T02:23:32.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.616+0000 7f386319d700 1 -- 172.21.15.31:0/1844846549 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f385c0c4b90 con 0x7f385c06bd20 2023-10-06T02:23:32.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.616+0000 7f386319d700 1 -- 172.21.15.31:0/1844846549 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f385c0c5150 con 0x7f385c06bd20 2023-10-06T02:23:32.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.618+0000 7f38527fc700 1 -- 172.21.15.31:0/1844846549 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3858013680 con 0x7f385c06bd20 2023-10-06T02:23:32.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.619+0000 7f38527fc700 1 --2- 172.21.15.31:0/1844846549 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f3840066f30 0x7f38400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:32.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.619+0000 7f38527fc700 1 -- 172.21.15.31:0/1844846549 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f38580849c0 con 0x7f385c06bd20 2023-10-06T02:23:32.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.620+0000 7f386199a700 1 --2- 172.21.15.31:0/1844846549 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f3840066f30 0x7f38400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:32.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.620+0000 7f386199a700 1 --2- 172.21.15.31:0/1844846549 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f3840066f30 0x7f38400693e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f385c0ca8c0 tx=0x7f384c017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:32.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.621+0000 7f3872b4fb80 1 Processor -- start 2023-10-06T02:23:32.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.621+0000 7f3872b4fb80 1 -- start start 2023-10-06T02:23:32.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.622+0000 7f3872b4fb80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562e7af04340 0x562e7aedb670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:32.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.622+0000 7f3872b4fb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562e7af04710 0x562e7af250e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:32.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.622+0000 7f3872b4fb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e7af25620 0x562e7af25a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:32.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.622+0000 7f3872b4fb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e7ae31a70 con 0x562e7af04710 2023-10-06T02:23:32.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.622+0000 7f3872b4fb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e7ae2f270 con 0x562e7af25620 2023-10-06T02:23:32.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.622+0000 7f3872b4fb80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x562e7ae2b060 con 0x562e7af04340 2023-10-06T02:23:32.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.622+0000 7f386199a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562e7af04710 0x562e7af250e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:32.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.623+0000 7f386199a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562e7af04710 0x562e7af250e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58946/0 (socket says 172.21.15.31:58946) 2023-10-06T02:23:32.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.623+0000 7f386199a700 1 -- 172.21.15.31:0/3814823721 learned_addr learned my addr 172.21.15.31:0/3814823721 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:32.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.623+0000 7f386299c700 1 --2- 172.21.15.31:0/3814823721 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e7af25620 0x562e7af25a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:32.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.623+0000 7f386219b700 1 --2- 172.21.15.31:0/3814823721 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562e7af04340 0x562e7aedb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:32.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.623+0000 7f386199a700 1 -- 172.21.15.31:0/3814823721 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562e7af04340 msgr2=0x562e7aedb670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:32.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.623+0000 7f386199a700 1 --2- 172.21.15.31:0/3814823721 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562e7af04340 0x562e7aedb670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:32.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.623+0000 7f386199a700 1 -- 172.21.15.31:0/3814823721 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e7af25620 msgr2=0x562e7af25a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:32.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.623+0000 7f386199a700 1 --2- 172.21.15.31:0/3814823721 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e7af25620 0x562e7af25a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:32.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.623+0000 7f386199a700 1 -- 172.21.15.31:0/3814823721 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e7acc5270 con 0x562e7af04710 2023-10-06T02:23:32.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.623+0000 7f386219b700 1 --2- 172.21.15.31:0/3814823721 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562e7af04340 0x562e7aedb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:23:32.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.623+0000 7f386299c700 1 --2- 172.21.15.31:0/3814823721 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e7af25620 0x562e7af25a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:23:32.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.623+0000 7f386199a700 1 --2- 172.21.15.31:0/3814823721 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562e7af04710 0x562e7af250e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f384c009f70 tx=0x7f384c021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ca8b46a9d184d93 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:32.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.623+0000 7f383f7fe700 1 -- 172.21.15.31:0/3814823721 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f384c02d070 con 0x562e7af04710 2023-10-06T02:23:32.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.623+0000 7f383f7fe700 1 -- 172.21.15.31:0/3814823721 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f384c01a8b0 con 0x562e7af04710 2023-10-06T02:23:32.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.623+0000 7f383f7fe700 1 -- 172.21.15.31:0/3814823721 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f384c021070 con 0x562e7af04710 2023-10-06T02:23:32.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.624+0000 7f3872b4fb80 1 -- 172.21.15.31:0/3814823721 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562e7af04710 msgr2=0x562e7af250e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:32.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.624+0000 7f3872b4fb80 1 --2- 172.21.15.31:0/3814823721 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562e7af04710 0x562e7af250e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f384c009f70 tx=0x7f384c021000 comp rx=0 tx=0).stop 2023-10-06T02:23:32.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.624+0000 7f3872b4fb80 1 -- 172.21.15.31:0/3814823721 shutdown_connections 2023-10-06T02:23:32.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.624+0000 7f3872b4fb80 1 --2- 172.21.15.31:0/3814823721 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e7af25620 0x562e7af25a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:32.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.624+0000 7f3872b4fb80 1 --2- 172.21.15.31:0/3814823721 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562e7af04340 0x562e7aedb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:32.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.624+0000 7f3872b4fb80 1 --2- 172.21.15.31:0/3814823721 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562e7af04710 0x562e7af250e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:32.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.624+0000 7f3872b4fb80 1 -- 172.21.15.31:0/3814823721 >> 172.21.15.31:0/3814823721 conn(0x562e7aedbe80 msgr2=0x562e7aedc6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:32.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.624+0000 7f3872b4fb80 1 -- 172.21.15.31:0/3814823721 shutdown_connections 2023-10-06T02:23:32.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.624+0000 7f3872b4fb80 1 -- 172.21.15.31:0/3814823721 wait complete. 2023-10-06T02:23:32.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.625+0000 7f3872b4fb80 1 -- start start 2023-10-06T02:23:32.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.625+0000 7f3872b4fb80 1 Processor -- start 2023-10-06T02:23:32.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.626+0000 7f3872b4fb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e7af04340 0x562e7af24dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:32.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.626+0000 7f3872b4fb80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562e7af04710 0x562e7aed7590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:32.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.626+0000 7f3872b4fb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562e7af25620 0x562e7af264f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:32.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.626+0000 7f3872b4fb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e7add6890 con 0x562e7af25620 2023-10-06T02:23:32.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.626+0000 7f3872b4fb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e7adf1ea0 con 0x562e7af04340 2023-10-06T02:23:32.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.626+0000 7f3872b4fb80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x562e7acade30 con 0x562e7af04710 2023-10-06T02:23:32.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.626+0000 7f3872b4fb80 10 client.0 _ll_register_callbacks cb 0x562e7ae9e020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T02:23:32.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.626+0000 7f386299c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562e7af25620 0x562e7af264f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:32.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.626+0000 7f386299c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562e7af25620 0x562e7af264f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58950/0 (socket says 172.21.15.31:58950) 2023-10-06T02:23:32.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.626+0000 7f386299c700 1 -- 172.21.15.31:0/1195385128 learned_addr learned my addr 172.21.15.31:0/1195385128 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:32.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.627+0000 7f386219b700 1 --2- 172.21.15.31:0/1195385128 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e7af04340 0x562e7af24dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:32.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.627+0000 7f386199a700 1 --2- 172.21.15.31:0/1195385128 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562e7af04710 0x562e7aed7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:32.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.627+0000 7f386299c700 1 -- 172.21.15.31:0/1195385128 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562e7af04710 msgr2=0x562e7aed7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:32.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.627+0000 7f386299c700 1 --2- 172.21.15.31:0/1195385128 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562e7af04710 0x562e7aed7590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:32.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.627+0000 7f386299c700 1 -- 172.21.15.31:0/1195385128 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e7af04340 msgr2=0x562e7af24dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:32.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.627+0000 7f386299c700 1 --2- 172.21.15.31:0/1195385128 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e7af04340 0x562e7af24dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:32.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.627+0000 7f386299c700 1 -- 172.21.15.31:0/1195385128 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f385c15df60 con 0x562e7af25620 2023-10-06T02:23:32.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.627+0000 7f386199a700 1 --2- 172.21.15.31:0/1195385128 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562e7af04710 0x562e7aed7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:23:32.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.627+0000 7f386299c700 1 --2- 172.21.15.31:0/1195385128 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562e7af25620 0x562e7af264f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3854007e50 tx=0x7f385400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60a0a3d722480c82 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:32.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.627+0000 7f383cff9700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T02:23:32.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.627+0000 7f383cff9700 1 -- 172.21.15.31:0/1195385128 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385400e040 con 0x562e7af25620 2023-10-06T02:23:32.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.627+0000 7f383cff9700 1 -- 172.21.15.31:0/1195385128 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3854012da0 con 0x562e7af25620 2023-10-06T02:23:32.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.627+0000 7f3872b4fb80 1 -- 172.21.15.31:0/1195385128 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562e7aecf940 con 0x562e7af25620 2023-10-06T02:23:32.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.628+0000 7f383cff9700 1 -- 172.21.15.31:0/1195385128 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3854018680 con 0x562e7af25620 2023-10-06T02:23:32.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.628+0000 7f3872b4fb80 10 client.4722 fetch_fsmap learned FSMap version 8 2023-10-06T02:23:32.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.628+0000 7f3872b4fb80 1 -- 172.21.15.31:0/1195385128 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x562e7ac67fe0 con 0x562e7af25620 2023-10-06T02:23:32.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.628+0000 7f383cff9700 1 -- 172.21.15.31:0/1195385128 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562e7aecf940 con 0x562e7af25620 2023-10-06T02:23:32.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.628+0000 7f383cff9700 1 -- 172.21.15.31:0/1195385128 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 8) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f3854014040 con 0x562e7af25620 2023-10-06T02:23:32.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.628+0000 7f3872b4fb80 10 client.4722 fetch_fsmap finished waiting for FSMap version 8 2023-10-06T02:23:32.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.628+0000 7f3872b4fb80 10 client.4722 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T02:23:32.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.628+0000 7f3872b4fb80 20 client.4722 populate_metadata read hostname 'smithi031' 2023-10-06T02:23:32.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.628+0000 7f3872b4fb80 1 --2- 172.21.15.31:0/1195385128 >> [v2:172.21.15.31:6834/3809284664,v1:172.21.15.31:6836/3809284664] conn(0x562e7af26c10 0x562e7af06700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:32.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.628+0000 7f3872b4fb80 4 client.4722 mds_command: new command op to 4297 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T02:23:32.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.628+0000 7f3872b4fb80 1 -- 172.21.15.31:0/1195385128 --> [v2:172.21.15.31:6834/3809284664,v1:172.21.15.31:6836/3809284664] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562e7aeac870 con 0x562e7af26c10 2023-10-06T02:23:32.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.628+0000 7f386219b700 1 --2- 172.21.15.31:0/1195385128 >> [v2:172.21.15.31:6834/3809284664,v1:172.21.15.31:6836/3809284664] conn(0x562e7af26c10 0x562e7af06700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:32.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.629+0000 7f386219b700 1 --2- 172.21.15.31:0/1195385128 >> [v2:172.21.15.31:6834/3809284664,v1:172.21.15.31:6836/3809284664] conn(0x562e7af26c10 0x562e7af06700 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33470f262f4b8ae1 server_cookie=de97799b5b072e43 in_seq=0 out_seq=0 2023-10-06T02:23:32.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.629+0000 7f383cff9700 10 client.4722 ms_handle_connect on v2:172.21.15.31:6834/3809284664 2023-10-06T02:23:32.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.629+0000 7f383cff9700 1 -- 172.21.15.31:0/1195385128 <== mds.0 v2:172.21.15.31:6834/3809284664 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x562e7aeac870 con 0x562e7af26c10 2023-10-06T02:23:32.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.629+0000 7f383cff9700 10 client.4722 handle_command_reply: tid=0 2023-10-06T02:23:32.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.629+0000 7f3872b4fb80 1 client.4722 shutdown 2023-10-06T02:23:32.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.630+0000 7f3872b4fb80 1 -- 172.21.15.31:0/1195385128 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562e7af25620 msgr2=0x562e7af264f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:32.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.630+0000 7f3872b4fb80 1 --2- 172.21.15.31:0/1195385128 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562e7af25620 0x562e7af264f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3854007e50 tx=0x7f385400b040 comp rx=0 tx=0).stop 2023-10-06T02:23:32.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.630+0000 7f3872b4fb80 1 -- 172.21.15.31:0/1195385128 shutdown_connections 2023-10-06T02:23:32.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.630+0000 7f3872b4fb80 1 --2- 172.21.15.31:0/1195385128 >> [v2:172.21.15.31:6834/3809284664,v1:172.21.15.31:6836/3809284664] conn(0x562e7af26c10 0x562e7af06700 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:32.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.630+0000 7f3872b4fb80 1 --2- 172.21.15.31:0/1195385128 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562e7af25620 0x562e7af264f0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:32.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.630+0000 7f3872b4fb80 1 --2- 172.21.15.31:0/1195385128 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e7af04340 0x562e7af24dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:32.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.630+0000 7f3872b4fb80 1 --2- 172.21.15.31:0/1195385128 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562e7af04710 0x562e7aed7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:32.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.630+0000 7f383cff9700 0 client.4722 ms_handle_reset on v2:172.21.15.31:6834/3809284664 2023-10-06T02:23:32.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.630+0000 7f3872b4fb80 1 -- 172.21.15.31:0/1195385128 >> 172.21.15.31:0/1195385128 conn(0x562e7aedbe80 msgr2=0x562e7aeda9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:32.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.630+0000 7f3872b4fb80 1 -- 172.21.15.31:0/1195385128 shutdown_connections 2023-10-06T02:23:32.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.630+0000 7f3872b4fb80 1 -- 172.21.15.31:0/1195385128 wait complete. 2023-10-06T02:23:32.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.630+0000 7f3872b4fb80 20 client.4722 trim_cache size 0 max 16384 2023-10-06T02:23:32.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.647+0000 7f3872b4fb80 1 Processor -- start 2023-10-06T02:23:32.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.647+0000 7f3872b4fb80 1 -- start start 2023-10-06T02:23:32.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.648+0000 7f3872b4fb80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562e7aedbe80 0x562e7af066d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:32.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.648+0000 7f3872b4fb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e7af04340 0x562e7af06c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:32.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.648+0000 7f3872b4fb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562e7af04710 0x562e7af23d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:32.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.648+0000 7f3872b4fb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e7af03cf0 con 0x562e7af04710 2023-10-06T02:23:32.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.648+0000 7f3872b4fb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e7af4b430 con 0x562e7af04340 2023-10-06T02:23:32.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.648+0000 7f3872b4fb80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x562e7aedcf20 con 0x562e7aedbe80 2023-10-06T02:23:32.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.648+0000 7f386299c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562e7af04710 0x562e7af23d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:32.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.649+0000 7f386299c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562e7af04710 0x562e7af23d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58966/0 (socket says 172.21.15.31:58966) 2023-10-06T02:23:32.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.649+0000 7f386299c700 1 -- 172.21.15.31:0/597817344 learned_addr learned my addr 172.21.15.31:0/597817344 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:32.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.649+0000 7f386299c700 1 -- 172.21.15.31:0/597817344 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562e7aedbe80 msgr2=0x562e7af066d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:32.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.649+0000 7f386299c700 1 --2- 172.21.15.31:0/597817344 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562e7aedbe80 0x562e7af066d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:32.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.649+0000 7f386299c700 1 -- 172.21.15.31:0/597817344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e7af04340 msgr2=0x562e7af06c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:32.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.649+0000 7f386299c700 1 --2- 172.21.15.31:0/597817344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e7af04340 0x562e7af06c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:32.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.649+0000 7f386299c700 1 -- 172.21.15.31:0/597817344 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e7af04df0 con 0x562e7af04710 2023-10-06T02:23:32.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.649+0000 7f386299c700 1 --2- 172.21.15.31:0/597817344 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562e7af04710 0x562e7af23d40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f38540070b0 tx=0x7f385401e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf07d7cbbbb4961c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:32.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.649+0000 7f383ffff700 1 -- 172.21.15.31:0/597817344 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385400e070 con 0x562e7af04710 2023-10-06T02:23:32.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.649+0000 7f383ffff700 1 -- 172.21.15.31:0/597817344 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f385400f040 con 0x562e7af04710 2023-10-06T02:23:32.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.649+0000 7f383ffff700 1 -- 172.21.15.31:0/597817344 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3854012da0 con 0x562e7af04710 2023-10-06T02:23:32.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.650+0000 7f3872b4fb80 1 -- 172.21.15.31:0/597817344 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562e7af04710 msgr2=0x562e7af23d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:32.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.650+0000 7f3872b4fb80 1 --2- 172.21.15.31:0/597817344 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562e7af04710 0x562e7af23d40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f38540070b0 tx=0x7f385401e000 comp rx=0 tx=0).stop 2023-10-06T02:23:32.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.650+0000 7f3872b4fb80 1 -- 172.21.15.31:0/597817344 shutdown_connections 2023-10-06T02:23:32.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.650+0000 7f3872b4fb80 1 --2- 172.21.15.31:0/597817344 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562e7af04710 0x562e7af23d40 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:32.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.650+0000 7f3872b4fb80 1 --2- 172.21.15.31:0/597817344 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562e7aedbe80 0x562e7af066d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:32.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.650+0000 7f3872b4fb80 1 --2- 172.21.15.31:0/597817344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e7af04340 0x562e7af06c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:32.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.650+0000 7f3872b4fb80 1 -- 172.21.15.31:0/597817344 >> 172.21.15.31:0/597817344 conn(0x562e7af08530 msgr2=0x562e7aed8ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:32.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.650+0000 7f3872b4fb80 1 -- 172.21.15.31:0/597817344 shutdown_connections 2023-10-06T02:23:32.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.650+0000 7f3872b4fb80 1 -- 172.21.15.31:0/597817344 wait complete. 2023-10-06T02:23:32.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.650+0000 7f3872b4fb80 1 -- start start 2023-10-06T02:23:32.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.651+0000 7f3872b4fb80 1 Processor -- start 2023-10-06T02:23:32.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.652+0000 7f3872b4fb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562e7aedbe80 0x562e7af8f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:32.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.652+0000 7f3872b4fb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e7af04340 0x562e7aef8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:32.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.652+0000 7f3872b4fb80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562e7af04710 0x562e7aefb850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:32.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.652+0000 7f3872b4fb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e7aefc490 con 0x562e7aedbe80 2023-10-06T02:23:32.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.652+0000 7f3872b4fb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e7af27150 con 0x562e7af04340 2023-10-06T02:23:32.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.652+0000 7f3872b4fb80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x562e7af8fb70 con 0x562e7af04710 2023-10-06T02:23:32.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.652+0000 7f3872b4fb80 10 client.0 _ll_register_callbacks cb 0x562e7ae9e020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T02:23:32.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.652+0000 7f386219b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562e7aedbe80 0x562e7af8f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:32.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.652+0000 7f386219b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562e7aedbe80 0x562e7af8f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58978/0 (socket says 172.21.15.31:58978) 2023-10-06T02:23:32.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.652+0000 7f386219b700 1 -- 172.21.15.31:0/398425052 learned_addr learned my addr 172.21.15.31:0/398425052 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:32.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.652+0000 7f386199a700 1 --2- 172.21.15.31:0/398425052 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e7af04340 0x562e7aef8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:32.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.652+0000 7f386219b700 1 -- 172.21.15.31:0/398425052 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562e7af04710 msgr2=0x562e7aefb850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:32.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.652+0000 7f386299c700 1 --2- 172.21.15.31:0/398425052 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562e7af04710 0x562e7aefb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:32.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.652+0000 7f386219b700 1 --2- 172.21.15.31:0/398425052 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562e7af04710 0x562e7aefb850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:32.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.652+0000 7f386219b700 1 -- 172.21.15.31:0/398425052 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e7af04340 msgr2=0x562e7aef8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:32.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.652+0000 7f386219b700 1 --2- 172.21.15.31:0/398425052 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e7af04340 0x562e7aef8f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:32.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.652+0000 7f386219b700 1 -- 172.21.15.31:0/398425052 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562e7ac67fe0 con 0x562e7aedbe80 2023-10-06T02:23:32.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.652+0000 7f386199a700 1 --2- 172.21.15.31:0/398425052 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e7af04340 0x562e7aef8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:23:32.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.652+0000 7f386299c700 1 --2- 172.21.15.31:0/398425052 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562e7af04710 0x562e7aefb850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:23:32.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.652+0000 7f386219b700 1 --2- 172.21.15.31:0/398425052 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562e7aedbe80 0x562e7af8f630 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3858051bf0 tx=0x7f3858082c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eabbe6fb0186f878 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:32.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.652+0000 7f383cff9700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T02:23:32.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.653+0000 7f383cff9700 1 -- 172.21.15.31:0/398425052 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3858082df0 con 0x562e7aedbe80 2023-10-06T02:23:32.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.653+0000 7f383cff9700 1 -- 172.21.15.31:0/398425052 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3858053a90 con 0x562e7aedbe80 2023-10-06T02:23:32.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.653+0000 7f3872b4fb80 1 -- 172.21.15.31:0/398425052 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f384c01a8b0 con 0x562e7aedbe80 2023-10-06T02:23:32.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.653+0000 7f383cff9700 1 -- 172.21.15.31:0/398425052 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3858055b60 con 0x562e7aedbe80 2023-10-06T02:23:32.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.653+0000 7f3872b4fb80 10 client.4728 fetch_fsmap learned FSMap version 8 2023-10-06T02:23:32.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.653+0000 7f3872b4fb80 1 -- 172.21.15.31:0/398425052 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x562e7adf9780 con 0x562e7aedbe80 2023-10-06T02:23:32.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.653+0000 7f383cff9700 1 -- 172.21.15.31:0/398425052 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f384c01a8b0 con 0x562e7aedbe80 2023-10-06T02:23:32.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.653+0000 7f383cff9700 1 -- 172.21.15.31:0/398425052 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 8) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f3858054be0 con 0x562e7aedbe80 2023-10-06T02:23:32.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.653+0000 7f3872b4fb80 10 client.4728 fetch_fsmap finished waiting for FSMap version 8 2023-10-06T02:23:32.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.653+0000 7f3872b4fb80 10 client.4728 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T02:23:32.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.653+0000 7f3872b4fb80 20 client.4728 populate_metadata read hostname 'smithi031' 2023-10-06T02:23:32.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.653+0000 7f3872b4fb80 1 --2- 172.21.15.31:0/398425052 >> [v2:172.21.15.31:6834/3809284664,v1:172.21.15.31:6836/3809284664] conn(0x562e7af25620 0x562e7af022c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:32.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.653+0000 7f3872b4fb80 4 client.4728 mds_command: new command op to 4297 tid=0[{"prefix": "status"}] 2023-10-06T02:23:32.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.653+0000 7f3872b4fb80 1 -- 172.21.15.31:0/398425052 --> [v2:172.21.15.31:6834/3809284664,v1:172.21.15.31:6836/3809284664] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562e7adcde00 con 0x562e7af25620 2023-10-06T02:23:32.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.653+0000 7f386199a700 1 --2- 172.21.15.31:0/398425052 >> [v2:172.21.15.31:6834/3809284664,v1:172.21.15.31:6836/3809284664] conn(0x562e7af25620 0x562e7af022c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:32.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.654+0000 7f386199a700 1 --2- 172.21.15.31:0/398425052 >> [v2:172.21.15.31:6834/3809284664,v1:172.21.15.31:6836/3809284664] conn(0x562e7af25620 0x562e7af022c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8428aa0482c3c5e8 server_cookie=fe0a736348151bfb in_seq=0 out_seq=0 2023-10-06T02:23:32.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.654+0000 7f383cff9700 10 client.4728 ms_handle_connect on v2:172.21.15.31:6834/3809284664 2023-10-06T02:23:32.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.654+0000 7f383cff9700 1 -- 172.21.15.31:0/398425052 <== mds.0 v2:172.21.15.31:6834/3809284664 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+323 (crc 0 0 0) 0x562e7adcde00 con 0x562e7af25620 2023-10-06T02:23:32.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.654+0000 7f383cff9700 10 client.4728 handle_command_reply: tid=0 2023-10-06T02:23:32.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.654+0000 7f3872b4fb80 1 client.4728 shutdown 2023-10-06T02:23:32.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.654+0000 7f3872b4fb80 1 -- 172.21.15.31:0/398425052 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562e7aedbe80 msgr2=0x562e7af8f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:32.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.654+0000 7f3872b4fb80 1 --2- 172.21.15.31:0/398425052 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562e7aedbe80 0x562e7af8f630 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3858051bf0 tx=0x7f3858082c40 comp rx=0 tx=0).stop 2023-10-06T02:23:32.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.654+0000 7f3872b4fb80 1 -- 172.21.15.31:0/398425052 shutdown_connections 2023-10-06T02:23:32.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.654+0000 7f3872b4fb80 1 --2- 172.21.15.31:0/398425052 >> [v2:172.21.15.31:6834/3809284664,v1:172.21.15.31:6836/3809284664] conn(0x562e7af25620 0x562e7af022c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:32.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.654+0000 7f3872b4fb80 1 --2- 172.21.15.31:0/398425052 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562e7af04710 0x562e7aefb850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:32.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.654+0000 7f3872b4fb80 1 --2- 172.21.15.31:0/398425052 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562e7aedbe80 0x562e7af8f630 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:32.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.654+0000 7f3872b4fb80 1 --2- 172.21.15.31:0/398425052 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e7af04340 0x562e7aef8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:32.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.654+0000 7f383cff9700 0 client.4728 ms_handle_reset on v2:172.21.15.31:6834/3809284664 2023-10-06T02:23:32.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.654+0000 7f3872b4fb80 1 -- 172.21.15.31:0/398425052 >> 172.21.15.31:0/398425052 conn(0x562e7af08530 msgr2=0x562e7aed8ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:32.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.654+0000 7f3872b4fb80 1 -- 172.21.15.31:0/398425052 shutdown_connections 2023-10-06T02:23:32.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.654+0000 7f3872b4fb80 1 -- 172.21.15.31:0/398425052 wait complete. 2023-10-06T02:23:32.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.655+0000 7f3872b4fb80 20 client.4728 trim_cache size 0 max 16384 2023-10-06T02:23:32.661 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T02:23:32.661 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "d6fcf512-874b-4259-b90a-30ae070f00a9", 2023-10-06T02:23:32.661 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 0, 2023-10-06T02:23:32.661 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 4297, 2023-10-06T02:23:32.661 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2023-10-06T02:23:32.661 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2023-10-06T02:23:32.661 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2023-10-06T02:23:32.661 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 22.622501424999999, 2023-10-06T02:23:32.661 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 8, 2023-10-06T02:23:32.661 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 23, 2023-10-06T02:23:32.661 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 23, 2023-10-06T02:23:32.662 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 30.50002542 2023-10-06T02:23:32.662 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T02:23:32.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.655+0000 7f386319d700 1 -- 172.21.15.31:0/1844846549 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f3840066f30 msgr2=0x7f38400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:32.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.655+0000 7f386319d700 1 --2- 172.21.15.31:0/1844846549 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f3840066f30 0x7f38400693e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f385c0ca8c0 tx=0x7f384c017000 comp rx=0 tx=0).stop 2023-10-06T02:23:32.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.655+0000 7f386319d700 1 -- 172.21.15.31:0/1844846549 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f385c06bd20 msgr2=0x7f385c0c6b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:32.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.655+0000 7f386319d700 1 --2- 172.21.15.31:0/1844846549 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f385c06bd20 0x7f385c0c6b80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f385800ec80 tx=0x7f38580091a0 comp rx=0 tx=0).stop 2023-10-06T02:23:32.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.655+0000 7f386319d700 1 -- 172.21.15.31:0/1844846549 shutdown_connections 2023-10-06T02:23:32.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.655+0000 7f386319d700 1 --2- 172.21.15.31:0/1844846549 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f385c142fd0 0x7f385c16c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:32.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.655+0000 7f386319d700 1 --2- 172.21.15.31:0/1844846549 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f385c06bd20 0x7f385c0c6b80 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:32.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.655+0000 7f386319d700 1 --2- 172.21.15.31:0/1844846549 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f3840066f30 0x7f38400693e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:32.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.655+0000 7f386319d700 1 --2- 172.21.15.31:0/1844846549 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f385c142c00 0x7f385c0c5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:32.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.655+0000 7f386319d700 1 -- 172.21.15.31:0/1844846549 >> 172.21.15.31:0/1844846549 conn(0x7f385c0675a0 msgr2=0x7f385c0c2480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:32.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.655+0000 7f386319d700 1 -- 172.21.15.31:0/1844846549 shutdown_connections 2023-10-06T02:23:32.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:32.655+0000 7f386319d700 1 -- 172.21.15.31:0/1844846549 wait complete. 2023-10-06T02:23:32.672 INFO:teuthology.run_tasks:Running task kclient... 2023-10-06T02:23:32.695 INFO:tasks.kclient:Mounting kernel clients... 2023-10-06T02:23:32.696 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-10-06T02:23:32.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:23:32.696 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2023-10-06T02:23:32.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:23:32.733 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link delete ceph-brx 2023-10-06T02:23:32.837 INFO:teuthology.orchestra.run.smithi031.stderr:Cannot find device "ceph-brx" 2023-10-06T02:23:32.840 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T02:23:32.841 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:23:32.841 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2023-10-06T02:23:32.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:23:32.877 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link delete ceph-brx 2023-10-06T02:23:32.981 INFO:teuthology.orchestra.run.smithi031.stderr:Cannot find device "ceph-brx" 2023-10-06T02:23:32.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T02:23:32.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:23:32.984 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2023-10-06T02:23:33.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:23:33.025 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link delete ceph-brx 2023-10-06T02:23:33.135 INFO:teuthology.orchestra.run.smithi062.stderr:Cannot find device "ceph-brx" 2023-10-06T02:23:33.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T02:23:33.138 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:23:33.139 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2023-10-06T02:23:33.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:23:33.176 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link delete ceph-brx 2023-10-06T02:23:33.281 INFO:teuthology.orchestra.run.smithi062.stderr:Cannot find device "ceph-brx" 2023-10-06T02:23:33.283 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T02:23:33.284 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2023-10-06T02:23:33.284 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T02:23:33.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:33.471+0000 7fc18359e700 1 Processor -- start 2023-10-06T02:23:33.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:33.472+0000 7fc18359e700 1 -- start start 2023-10-06T02:23:33.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:33.473+0000 7fc18359e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc184159550 0x7fc1840ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:33.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:33.473+0000 7fc18359e700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc184159920 0x7fc1840ad090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:33.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:33.473+0000 7fc18359e700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1840acc00 con 0x7fc18415dfd0 2023-10-06T02:23:33.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:33.473+0000 7fc18359e700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1840abff0 con 0x7fc184159550 2023-10-06T02:23:33.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:33.473+0000 7fc18359e700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1840ac2d0 con 0x7fc184159920 2023-10-06T02:23:33.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:33.473+0000 7fc18259c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc184159550 0x7fc1840ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:33.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:33.473+0000 7fc18259c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc184159550 0x7fc1840ceac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58984/0 (socket says 172.21.15.31:58984) 2023-10-06T02:23:33.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:33.473+0000 7fc18259c700 1 -- 172.21.15.31:0/4170637125 learned_addr learned my addr 172.21.15.31:0/4170637125 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:33.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:33.473+0000 7fc182d9d700 1 --2- 172.21.15.31:0/4170637125 >> v2:172.21.15.62:3300/0 conn(0x7fc184159920 0x7fc1840ad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:33.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:33.473+0000 7fc18259c700 1 -- 172.21.15.31:0/4170637125 >> v2:172.21.15.62:3300/0 conn(0x7fc184159920 msgr2=0x7fc1840ad090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:33.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:33.473+0000 7fc18259c700 1 --2- 172.21.15.31:0/4170637125 >> v2:172.21.15.62:3300/0 conn(0x7fc184159920 0x7fc1840ad090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:33.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:33.474+0000 7fc18259c700 1 -- 172.21.15.31:0/4170637125 >> v1:172.21.15.62:6789/0 conn(0x7fc18415dfd0 legacy=0x7fc1840aedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:33.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:33.474+0000 7fc18259c700 1 -- 172.21.15.31:0/4170637125 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1840ac5b0 con 0x7fc184159550 2023-10-06T02:23:33.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:33.474+0000 7fc182d9d700 1 --2- 172.21.15.31:0/4170637125 >> v2:172.21.15.62:3300/0 conn(0x7fc184159920 0x7fc1840ad090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:23:33.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:33.474+0000 7fc18259c700 1 --2- 172.21.15.31:0/4170637125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc184159550 0x7fc1840ceac0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc18406b0f0 tx=0x7fc174009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fa63fb7b8ac027e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:33.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:33.474+0000 7fc18159a700 1 -- 172.21.15.31:0/4170637125 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17400b860 con 0x7fc184159550 2023-10-06T02:23:33.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:33.474+0000 7fc18159a700 1 -- 172.21.15.31:0/4170637125 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc17400b9c0 con 0x7fc184159550 2023-10-06T02:23:33.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:33.474+0000 7fc18159a700 1 -- 172.21.15.31:0/4170637125 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17400bd60 con 0x7fc184159550 2023-10-06T02:23:33.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:33.475+0000 7fc18359e700 1 -- 172.21.15.31:0/4170637125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc184159550 msgr2=0x7fc1840ceac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:33.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:33.475+0000 7fc18359e700 1 --2- 172.21.15.31:0/4170637125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc184159550 0x7fc1840ceac0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc18406b0f0 tx=0x7fc174009250 comp rx=0 tx=0).stop 2023-10-06T02:23:33.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:33.475+0000 7fc18359e700 1 -- 172.21.15.31:0/4170637125 shutdown_connections 2023-10-06T02:23:33.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:33.475+0000 7fc18359e700 1 --2- 172.21.15.31:0/4170637125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc184159550 0x7fc1840ceac0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:33.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:33.475+0000 7fc18359e700 1 --2- 172.21.15.31:0/4170637125 >> v2:172.21.15.62:3300/0 conn(0x7fc184159920 0x7fc1840ad090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:33.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:33.475+0000 7fc18359e700 1 -- 172.21.15.31:0/4170637125 >> 172.21.15.31:0/4170637125 conn(0x7fc18406ce80 msgr2=0x7fc1840ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:33.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:33.475+0000 7fc18359e700 1 -- 172.21.15.31:0/4170637125 shutdown_connections 2023-10-06T02:23:33.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:33.475+0000 7fc18359e700 1 -- 172.21.15.31:0/4170637125 wait complete. 2023-10-06T02:23:33.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:33.476+0000 7fc18359e700 1 Processor -- start 2023-10-06T02:23:33.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:33.476+0000 7fc18359e700 1 -- start start 2023-10-06T02:23:33.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:33.477+0000 7fc18359e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc184159550 0x7fc1840cc180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:33.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:33.477+0000 7fc18359e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc184159920 0x7fc18416d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:33.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:33.477+0000 7fc18359e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc18415dfd0 0x7fc18416d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:33.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:33.477+0000 7fc18359e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1840caed0 con 0x7fc184159920 2023-10-06T02:23:33.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:33.477+0000 7fc18359e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1840cbcf0 con 0x7fc18415dfd0 2023-10-06T02:23:33.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:33.477+0000 7fc18359e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1840cc6c0 con 0x7fc184159550 2023-10-06T02:23:33.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:33.477+0000 7fc181d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc184159920 0x7fc18416d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:33.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:33.478+0000 7fc18259c700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc184159550 0x7fc1840cc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:33.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:33.478+0000 7fc181d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc184159920 0x7fc18416d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58990/0 (socket says 172.21.15.31:58990) 2023-10-06T02:23:33.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:33.478+0000 7fc181d9b700 1 -- 172.21.15.31:0/2802369123 learned_addr learned my addr 172.21.15.31:0/2802369123 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:33.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:33.478+0000 7fc182d9d700 1 --2- 172.21.15.31:0/2802369123 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc18415dfd0 0x7fc18416d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:33.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:33.478+0000 7fc181d9b700 1 -- 172.21.15.31:0/2802369123 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc184159550 msgr2=0x7fc1840cc180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:33.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:33.478+0000 7fc181d9b700 1 --2- 172.21.15.31:0/2802369123 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc184159550 0x7fc1840cc180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:33.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:33.478+0000 7fc181d9b700 1 -- 172.21.15.31:0/2802369123 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc18415dfd0 msgr2=0x7fc18416d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:33.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:33.478+0000 7fc181d9b700 1 --2- 172.21.15.31:0/2802369123 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc18415dfd0 0x7fc18416d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:33.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:33.478+0000 7fc181d9b700 1 -- 172.21.15.31:0/2802369123 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1740094f0 con 0x7fc184159920 2023-10-06T02:23:33.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:33.478+0000 7fc181d9b700 1 --2- 172.21.15.31:0/2802369123 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc184159920 0x7fc18416d380 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc17800c4e0 tx=0x7fc17800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:33.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:33.478+0000 7fc1727fc700 1 -- 172.21.15.31:0/2802369123 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc178011590 con 0x7fc184159920 2023-10-06T02:23:33.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:33.478+0000 7fc1727fc700 1 -- 172.21.15.31:0/2802369123 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc178011710 con 0x7fc184159920 2023-10-06T02:23:33.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:33.478+0000 7fc1727fc700 1 -- 172.21.15.31:0/2802369123 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc178011ad0 con 0x7fc184159920 2023-10-06T02:23:33.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:33.478+0000 7fc18359e700 1 -- 172.21.15.31:0/2802369123 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1840cb0e0 con 0x7fc184159920 2023-10-06T02:23:33.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:33.479+0000 7fc18359e700 1 -- 172.21.15.31:0/2802369123 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1840cb6a0 con 0x7fc184159920 2023-10-06T02:23:33.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:33.479+0000 7fc18359e700 1 -- 172.21.15.31:0/2802369123 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc18404c3c0 con 0x7fc184159920 2023-10-06T02:23:33.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:33.480+0000 7fc1727fc700 1 -- 172.21.15.31:0/2802369123 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc17801b3b0 con 0x7fc184159920 2023-10-06T02:23:33.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:33.481+0000 7fc1727fc700 1 --2- 172.21.15.31:0/2802369123 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc164066f30 0x7fc1640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:33.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:33.482+0000 7fc1727fc700 1 -- 172.21.15.31:0/2802369123 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc178083af0 con 0x7fc184159920 2023-10-06T02:23:33.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:33.482+0000 7fc18259c700 1 --2- 172.21.15.31:0/2802369123 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc164066f30 0x7fc1640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:33.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:33.482+0000 7fc18259c700 1 --2- 172.21.15.31:0/2802369123 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc164066f30 0x7fc1640693e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc18406b0f0 tx=0x7fc174017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:33.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:33.485+0000 7fc1727fc700 1 -- 172.21.15.31:0/2802369123 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc178050b60 con 0x7fc184159920 2023-10-06T02:23:33.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:33.734+0000 7fc18359e700 1 -- 172.21.15.31:0/2802369123 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc1840cbe30 con 0x7fc184159920 2023-10-06T02:23:33.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:33.734+0000 7fc1727fc700 1 -- 172.21.15.31:0/2802369123 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7fc178017070 con 0x7fc184159920 2023-10-06T02:23:33.734 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T02:23:33.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:33.736+0000 7fc18359e700 1 -- 172.21.15.31:0/2802369123 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc164066f30 msgr2=0x7fc1640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:33.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:33.736+0000 7fc18359e700 1 --2- 172.21.15.31:0/2802369123 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc164066f30 0x7fc1640693e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc18406b0f0 tx=0x7fc174017000 comp rx=0 tx=0).stop 2023-10-06T02:23:33.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:33.736+0000 7fc18359e700 1 -- 172.21.15.31:0/2802369123 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc184159920 msgr2=0x7fc18416d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:33.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:33.736+0000 7fc18359e700 1 --2- 172.21.15.31:0/2802369123 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc184159920 0x7fc18416d380 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc17800c4e0 tx=0x7fc17800c7f0 comp rx=0 tx=0).stop 2023-10-06T02:23:33.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:33.736+0000 7fc18359e700 1 -- 172.21.15.31:0/2802369123 shutdown_connections 2023-10-06T02:23:33.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:33.736+0000 7fc18359e700 1 --2- 172.21.15.31:0/2802369123 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc18415dfd0 0x7fc18416d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:33.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:33.736+0000 7fc18359e700 1 --2- 172.21.15.31:0/2802369123 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc164066f30 0x7fc1640693e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:33.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:33.736+0000 7fc18359e700 1 --2- 172.21.15.31:0/2802369123 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc184159550 0x7fc1840cc180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:33.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:33.736+0000 7fc18359e700 1 --2- 172.21.15.31:0/2802369123 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc184159920 0x7fc18416d380 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:33.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:33.736+0000 7fc18359e700 1 -- 172.21.15.31:0/2802369123 >> 172.21.15.31:0/2802369123 conn(0x7fc18406ce80 msgr2=0x7fc184152cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:33.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:33.736+0000 7fc18359e700 1 -- 172.21.15.31:0/2802369123 shutdown_connections 2023-10-06T02:23:33.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:33.736+0000 7fc18359e700 1 -- 172.21.15.31:0/2802369123 wait complete. 2023-10-06T02:23:33.754 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T02:23:33.754 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T02:23:33.754 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi031.front.sepia.ceph.com 2023-10-06T02:23:33.754 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T02:23:33.754 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T02:23:33.754 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T02:23:33.754 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T02:23:33.754 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T02:23:33.754 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T02:23:33.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:23:33.754 DEBUG:teuthology.orchestra.run.smithi031:> ip addr 2023-10-06T02:23:33.791 INFO:teuthology.orchestra.run.smithi031.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T02:23:33.791 INFO:teuthology.orchestra.run.smithi031.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T02:23:33.791 INFO:teuthology.orchestra.run.smithi031.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T02:23:33.791 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:23:33.791 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 ::1/128 scope host 2023-10-06T02:23:33.791 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:23:33.791 INFO:teuthology.orchestra.run.smithi031.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T02:23:33.792 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cc brd ff:ff:ff:ff:ff:ff 2023-10-06T02:23:33.792 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f0 2023-10-06T02:23:33.792 INFO:teuthology.orchestra.run.smithi031.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T02:23:33.792 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cd brd ff:ff:ff:ff:ff:ff 2023-10-06T02:23:33.792 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f1 2023-10-06T02:23:33.792 INFO:teuthology.orchestra.run.smithi031.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T02:23:33.792 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:ba brd ff:ff:ff:ff:ff:ff 2023-10-06T02:23:33.792 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f0 2023-10-06T02:23:33.792 INFO:teuthology.orchestra.run.smithi031.stdout: inet 172.21.15.31/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T02:23:33.792 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft 42242sec preferred_lft 42242sec 2023-10-06T02:23:33.792 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::ae1f:6bff:fea5:caba/64 scope link 2023-10-06T02:23:33.793 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:23:33.793 INFO:teuthology.orchestra.run.smithi031.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T02:23:33.793 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:bb brd ff:ff:ff:ff:ff:ff 2023-10-06T02:23:33.793 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f1 2023-10-06T02:23:33.794 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-06T02:23:33.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:23:33.794 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2023-10-06T02:23:33.794 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2023-10-06T02:23:33.794 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link add name ceph-brx type bridge 2023-10-06T02:23:33.794 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip addr flush dev ceph-brx 2023-10-06T02:23:33.794 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set ceph-brx up 2023-10-06T02:23:33.794 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-06T02:23:33.795 DEBUG:teuthology.orchestra.run.smithi031:> ') 2023-10-06T02:23:34.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:23:34.231 DEBUG:teuthology.orchestra.run.smithi031:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T02:23:34.297 INFO:teuthology.orchestra.run.smithi031.stdout:1 2023-10-06T02:23:34.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:23:34.300 DEBUG:teuthology.orchestra.run.smithi031:> route 2023-10-06T02:23:34.357 INFO:teuthology.orchestra.run.smithi031.stdout:Kernel IP routing table 2023-10-06T02:23:34.357 INFO:teuthology.orchestra.run.smithi031.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T02:23:34.357 INFO:teuthology.orchestra.run.smithi031.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T02:23:34.357 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T02:23:34.357 INFO:teuthology.orchestra.run.smithi031.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T02:23:34.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:23:34.359 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2023-10-06T02:23:34.359 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2023-10-06T02:23:34.359 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T02:23:34.359 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T02:23:34.359 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T02:23:34.359 DEBUG:teuthology.orchestra.run.smithi031:> ') 2023-10-06T02:23:35.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:23:35.098 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2023-10-06T02:23:35.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:23:35.135 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list-id 2023-10-06T02:23:35.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:23:35.212 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2023-10-06T02:23:35.212 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2023-10-06T02:23:35.212 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T02:23:35.212 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-10-06T02:23:35.212 DEBUG:teuthology.orchestra.run.smithi031:> ') 2023-10-06T02:23:35.382 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-10-06T02:23:35.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:23:35.383 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2023-10-06T02:23:35.383 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2023-10-06T02:23:35.383 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-10-06T02:23:35.383 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-06T02:23:35.383 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-10-06T02:23:35.383 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-10-06T02:23:35.383 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-10-06T02:23:35.383 DEBUG:teuthology.orchestra.run.smithi031:> ') 2023-10-06T02:23:35.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:23:35.642 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2023-10-06T02:23:35.642 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2023-10-06T02:23:35.642 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set brx.0 up 2023-10-06T02:23:35.642 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set dev brx.0 master ceph-brx 2023-10-06T02:23:35.642 DEBUG:teuthology.orchestra.run.smithi031:> ') 2023-10-06T02:23:35.802 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T02:23:35.802 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T02:23:35.838 INFO:teuthology.orchestra.run.smithi031.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T02:23:35.838 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T02:23:35.838 DEBUG:teuthology.orchestra.run.smithi031:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T02:23:35.914 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-06T02:23:35.914 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T02:23:35.915 DEBUG:teuthology.orchestra.run.smithi031:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T02:23:36.071 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T02:23:36.072 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T02:23:36.503 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T02:23:36.503 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T02:23:36.503 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.62:3300,172.21.15.62:3301" 2023-10-06T02:23:36.503 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: trying mount with new device syntax: 0@d6fcf512-874b-4259-b90a-30ae070f00a9.cephfs=/ 2023-10-06T02:23:36.503 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.31:3300/172.21.15.62:3300/172.21.15.62:3301" will pass to kernel 2023-10-06T02:23:36.508 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T02:23:36.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:23:36.508 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T02:23:36.571 INFO:tasks.kclient:client.2 config is {'syntax': 'v2'} 2023-10-06T02:23:36.571 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T02:23:36.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:36.757+0000 7f05cb19d700 1 Processor -- start 2023-10-06T02:23:36.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:36.757+0000 7f05cb19d700 1 -- start start 2023-10-06T02:23:36.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:36.759+0000 7f05cb19d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f05c406f3a0 0x7f05c40cc840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:36.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:36.759+0000 7f05cb19d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f05c40c6b20 0x7f05c406e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:36.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:36.759+0000 7f05cb19d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05c406a490 con 0x7f05c40c6f70 2023-10-06T02:23:36.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:36.759+0000 7f05cb19d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05c40c7490 con 0x7f05c406f3a0 2023-10-06T02:23:36.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:36.759+0000 7f05cb19d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05c40c7770 con 0x7f05c40c6b20 2023-10-06T02:23:36.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:36.759+0000 7f05ca19b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f05c406f3a0 0x7f05c40cc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:36.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:36.759+0000 7f05ca99c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f05c40c6f70 0x7f05c414fc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:41802/0 (socket says 172.21.15.31:41802) 2023-10-06T02:23:36.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:36.759+0000 7f05ca99c700 1 -- 172.21.15.31:0/574647223 learned_addr learned my addr 172.21.15.31:0/574647223 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:23:36.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:36.759+0000 7f05c999a700 1 --2- 172.21.15.31:0/574647223 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f05c40c6b20 0x7f05c406e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:36.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:36.759+0000 7f05c9199700 1 -- 172.21.15.31:0/574647223 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 26764422 0 0) 0x7f05c406a490 con 0x7f05c40c6f70 2023-10-06T02:23:36.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:36.759+0000 7f05ca19b700 1 -- 172.21.15.31:0/574647223 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f05c40c6b20 msgr2=0x7f05c406e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:36.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:36.759+0000 7f05ca19b700 1 --2- 172.21.15.31:0/574647223 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f05c40c6b20 0x7f05c406e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:36.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:36.759+0000 7f05ca19b700 1 -- 172.21.15.31:0/574647223 >> v1:172.21.15.31:6789/0 conn(0x7f05c40c6f70 legacy=0x7f05c414fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:36.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:36.759+0000 7f05ca19b700 1 -- 172.21.15.31:0/574647223 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05c40c7a50 con 0x7f05c406f3a0 2023-10-06T02:23:36.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:36.760+0000 7f05ca19b700 1 --2- 172.21.15.31:0/574647223 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05c406f3a0 0x7f05c40cc840 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f05c00098e0 tx=0x7f05c00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a01f41b847acd89f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:36.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:36.760+0000 7f05c9199700 1 -- 172.21.15.31:0/574647223 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05c000b640 con 0x7f05c406f3a0 2023-10-06T02:23:36.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:36.760+0000 7f05c9199700 1 -- 172.21.15.31:0/574647223 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05c000b7a0 con 0x7f05c406f3a0 2023-10-06T02:23:36.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:36.760+0000 7f05c9199700 1 -- 172.21.15.31:0/574647223 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05c000bb40 con 0x7f05c406f3a0 2023-10-06T02:23:36.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:36.760+0000 7f05cb19d700 1 -- 172.21.15.31:0/574647223 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05c406f3a0 msgr2=0x7f05c40cc840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:36.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:36.760+0000 7f05cb19d700 1 --2- 172.21.15.31:0/574647223 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05c406f3a0 0x7f05c40cc840 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f05c00098e0 tx=0x7f05c00091a0 comp rx=0 tx=0).stop 2023-10-06T02:23:36.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:36.761+0000 7f05cb19d700 1 -- 172.21.15.31:0/574647223 shutdown_connections 2023-10-06T02:23:36.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:36.761+0000 7f05cb19d700 1 --2- 172.21.15.31:0/574647223 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05c406f3a0 0x7f05c40cc840 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:36.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:36.761+0000 7f05cb19d700 1 --2- 172.21.15.31:0/574647223 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f05c40c6b20 0x7f05c406e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:36.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:36.761+0000 7f05cb19d700 1 -- 172.21.15.31:0/574647223 >> 172.21.15.31:0/574647223 conn(0x7f05c406d170 msgr2=0x7f05c40cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:36.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:36.761+0000 7f05cb19d700 1 -- 172.21.15.31:0/574647223 shutdown_connections 2023-10-06T02:23:36.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:36.761+0000 7f05cb19d700 1 -- 172.21.15.31:0/574647223 wait complete. 2023-10-06T02:23:36.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:36.762+0000 7f05cb19d700 1 Processor -- start 2023-10-06T02:23:36.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:36.762+0000 7f05cb19d700 1 -- start start 2023-10-06T02:23:36.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:36.763+0000 7f05cb19d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05c406f3a0 0x7f05c40afdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:36.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:36.763+0000 7f05cb19d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f05c40c6b20 0x7f05c4140f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:36.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:36.763+0000 7f05cb19d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05c40c6f70 0x7f05c41577a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:36.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:36.763+0000 7f05cb19d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05c40c0ed0 con 0x7f05c406f3a0 2023-10-06T02:23:36.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:36.763+0000 7f05cb19d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05c40c1a50 con 0x7f05c40c6f70 2023-10-06T02:23:36.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:36.763+0000 7f05cb19d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05c40c1120 con 0x7f05c40c6b20 2023-10-06T02:23:36.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:36.764+0000 7f05ca19b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05c406f3a0 0x7f05c40afdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:36.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:36.764+0000 7f05ca19b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05c406f3a0 0x7f05c40afdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59012/0 (socket says 172.21.15.31:59012) 2023-10-06T02:23:36.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:36.764+0000 7f05ca19b700 1 -- 172.21.15.31:0/607626658 learned_addr learned my addr 172.21.15.31:0/607626658 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:36.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:36.764+0000 7f05c999a700 1 --2- 172.21.15.31:0/607626658 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f05c40c6b20 0x7f05c4140f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:36.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:36.764+0000 7f05ca99c700 1 --2- 172.21.15.31:0/607626658 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05c40c6f70 0x7f05c41577a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:36.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:36.764+0000 7f05ca19b700 1 -- 172.21.15.31:0/607626658 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f05c40c6b20 msgr2=0x7f05c4140f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:36.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:36.764+0000 7f05ca19b700 1 --2- 172.21.15.31:0/607626658 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f05c40c6b20 0x7f05c4140f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:36.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:36.764+0000 7f05ca19b700 1 -- 172.21.15.31:0/607626658 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05c40c6f70 msgr2=0x7f05c41577a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:36.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:36.764+0000 7f05ca19b700 1 --2- 172.21.15.31:0/607626658 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05c40c6f70 0x7f05c41577a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:36.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:36.764+0000 7f05ca19b700 1 -- 172.21.15.31:0/607626658 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05c0009420 con 0x7f05c406f3a0 2023-10-06T02:23:36.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:36.764+0000 7f05c999a700 1 --2- 172.21.15.31:0/607626658 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f05c40c6b20 0x7f05c4140f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:23:36.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:36.764+0000 7f05ca99c700 1 --2- 172.21.15.31:0/607626658 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05c40c6f70 0x7f05c41577a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:23:36.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:36.764+0000 7f05ca19b700 1 --2- 172.21.15.31:0/607626658 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05c406f3a0 0x7f05c40afdf0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f05c0009f80 tx=0x7f05c0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:36.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:36.764+0000 7f05ba7fc700 1 -- 172.21.15.31:0/607626658 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05c000bb20 con 0x7f05c406f3a0 2023-10-06T02:23:36.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:36.765+0000 7f05ba7fc700 1 -- 172.21.15.31:0/607626658 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05c0011420 con 0x7f05c406f3a0 2023-10-06T02:23:36.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:36.765+0000 7f05cb19d700 1 -- 172.21.15.31:0/607626658 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05c40c1400 con 0x7f05c406f3a0 2023-10-06T02:23:36.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:36.765+0000 7f05ba7fc700 1 -- 172.21.15.31:0/607626658 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05c00117a0 con 0x7f05c406f3a0 2023-10-06T02:23:36.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:36.765+0000 7f05cb19d700 1 -- 172.21.15.31:0/607626658 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05c40b0330 con 0x7f05c406f3a0 2023-10-06T02:23:36.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:36.766+0000 7f05cb19d700 1 -- 172.21.15.31:0/607626658 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05c404c3c0 con 0x7f05c406f3a0 2023-10-06T02:23:36.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:36.767+0000 7f05ba7fc700 1 -- 172.21.15.31:0/607626658 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f05c0011b50 con 0x7f05c406f3a0 2023-10-06T02:23:36.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:36.767+0000 7f05ba7fc700 1 --2- 172.21.15.31:0/607626658 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f05a8066f30 0x7f05a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:36.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:36.768+0000 7f05ba7fc700 1 -- 172.21.15.31:0/607626658 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f05c0085d50 con 0x7f05c406f3a0 2023-10-06T02:23:36.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:36.768+0000 7f05c999a700 1 --2- 172.21.15.31:0/607626658 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f05a8066f30 0x7f05a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:36.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:36.768+0000 7f05c999a700 1 --2- 172.21.15.31:0/607626658 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f05a8066f30 0x7f05a80693e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f05b4006ea0 tx=0x7f05b4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:36.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:36.772+0000 7f05ba7fc700 1 -- 172.21.15.31:0/607626658 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f05c0052dc0 con 0x7f05c406f3a0 2023-10-06T02:23:37.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:37.021+0000 7f05cb19d700 1 -- 172.21.15.31:0/607626658 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f05c40c7d30 con 0x7f05c406f3a0 2023-10-06T02:23:37.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:37.022+0000 7f05ba7fc700 1 -- 172.21.15.31:0/607626658 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f05c00563e0 con 0x7f05c406f3a0 2023-10-06T02:23:37.022 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T02:23:37.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:37.024+0000 7f05cb19d700 1 -- 172.21.15.31:0/607626658 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f05a8066f30 msgr2=0x7f05a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:37.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:37.024+0000 7f05cb19d700 1 --2- 172.21.15.31:0/607626658 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f05a8066f30 0x7f05a80693e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f05b4006ea0 tx=0x7f05b4008040 comp rx=0 tx=0).stop 2023-10-06T02:23:37.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:37.024+0000 7f05cb19d700 1 -- 172.21.15.31:0/607626658 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05c406f3a0 msgr2=0x7f05c40afdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:37.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:37.024+0000 7f05cb19d700 1 --2- 172.21.15.31:0/607626658 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05c406f3a0 0x7f05c40afdf0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f05c0009f80 tx=0x7f05c0017000 comp rx=0 tx=0).stop 2023-10-06T02:23:37.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:37.024+0000 7f05cb19d700 1 -- 172.21.15.31:0/607626658 shutdown_connections 2023-10-06T02:23:37.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:37.024+0000 7f05cb19d700 1 --2- 172.21.15.31:0/607626658 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05c40c6f70 0x7f05c41577a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:37.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:37.024+0000 7f05cb19d700 1 --2- 172.21.15.31:0/607626658 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05c406f3a0 0x7f05c40afdf0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:37.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:37.024+0000 7f05cb19d700 1 --2- 172.21.15.31:0/607626658 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f05a8066f30 0x7f05a80693e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:37.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:37.024+0000 7f05cb19d700 1 --2- 172.21.15.31:0/607626658 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f05c40c6b20 0x7f05c4140f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:37.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:37.024+0000 7f05cb19d700 1 -- 172.21.15.31:0/607626658 >> 172.21.15.31:0/607626658 conn(0x7f05c406d170 msgr2=0x7f05c414aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:37.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:37.024+0000 7f05cb19d700 1 -- 172.21.15.31:0/607626658 shutdown_connections 2023-10-06T02:23:37.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:37.024+0000 7f05cb19d700 1 -- 172.21.15.31:0/607626658 wait complete. 2023-10-06T02:23:37.042 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T02:23:37.042 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T02:23:37.042 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi031.front.sepia.ceph.com 2023-10-06T02:23:37.042 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-10-06T02:23:37.042 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-10-06T02:23:37.042 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T02:23:37.042 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T02:23:37.042 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T02:23:37.043 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-10-06T02:23:37.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:23:37.043 DEBUG:teuthology.orchestra.run.smithi031:> ip addr 2023-10-06T02:23:37.080 INFO:teuthology.orchestra.run.smithi031.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T02:23:37.080 INFO:teuthology.orchestra.run.smithi031.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T02:23:37.080 INFO:teuthology.orchestra.run.smithi031.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T02:23:37.080 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:23:37.080 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 ::1/128 scope host 2023-10-06T02:23:37.080 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:23:37.081 INFO:teuthology.orchestra.run.smithi031.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T02:23:37.081 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cc brd ff:ff:ff:ff:ff:ff 2023-10-06T02:23:37.081 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f0 2023-10-06T02:23:37.081 INFO:teuthology.orchestra.run.smithi031.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T02:23:37.081 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cd brd ff:ff:ff:ff:ff:ff 2023-10-06T02:23:37.081 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f1 2023-10-06T02:23:37.081 INFO:teuthology.orchestra.run.smithi031.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T02:23:37.081 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:ba brd ff:ff:ff:ff:ff:ff 2023-10-06T02:23:37.081 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f0 2023-10-06T02:23:37.081 INFO:teuthology.orchestra.run.smithi031.stdout: inet 172.21.15.31/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T02:23:37.081 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft 42239sec preferred_lft 42239sec 2023-10-06T02:23:37.081 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::ae1f:6bff:fea5:caba/64 scope link 2023-10-06T02:23:37.082 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:23:37.082 INFO:teuthology.orchestra.run.smithi031.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T02:23:37.082 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:bb brd ff:ff:ff:ff:ff:ff 2023-10-06T02:23:37.082 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f1 2023-10-06T02:23:37.082 INFO:teuthology.orchestra.run.smithi031.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T02:23:37.082 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 16:d7:88:2d:d0:c2 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:23:37.082 INFO:teuthology.orchestra.run.smithi031.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T02:23:37.082 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:23:37.082 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::408:52ff:fe3c:ba8f/64 scope link 2023-10-06T02:23:37.082 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:23:37.083 INFO:teuthology.orchestra.run.smithi031.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T02:23:37.083 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 16:d7:88:2d:d0:c2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T02:23:37.083 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::14d7:88ff:fe2d:d0c2/64 scope link tentative 2023-10-06T02:23:37.083 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:23:37.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:23:37.083 DEBUG:teuthology.orchestra.run.smithi031:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T02:23:37.180 INFO:teuthology.orchestra.run.smithi031.stdout:1 2023-10-06T02:23:37.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:23:37.183 DEBUG:teuthology.orchestra.run.smithi031:> route 2023-10-06T02:23:37.222 INFO:teuthology.orchestra.run.smithi031.stdout:Kernel IP routing table 2023-10-06T02:23:37.222 INFO:teuthology.orchestra.run.smithi031.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T02:23:37.222 INFO:teuthology.orchestra.run.smithi031.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T02:23:37.222 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T02:23:37.222 INFO:teuthology.orchestra.run.smithi031.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T02:23:37.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:23:37.223 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2023-10-06T02:23:37.223 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2023-10-06T02:23:37.223 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T02:23:37.223 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T02:23:37.224 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T02:23:37.224 DEBUG:teuthology.orchestra.run.smithi031:> ') 2023-10-06T02:23:37.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:23:37.421 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2023-10-06T02:23:37.457 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T02:23:37.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:23:37.458 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list-id 2023-10-06T02:23:37.533 INFO:teuthology.orchestra.run.smithi031.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-10-06T02:23:37.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:23:37.534 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2023-10-06T02:23:37.534 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2023-10-06T02:23:37.534 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T02:23:37.534 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-10-06T02:23:37.535 DEBUG:teuthology.orchestra.run.smithi031:> ') 2023-10-06T02:23:37.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:23:37.702 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-10-06T02:23:37.764 INFO:teuthology.orchestra.run.smithi031.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T02:23:37.764 INFO:teuthology.orchestra.run.smithi031.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T02:23:37.764 INFO:teuthology.orchestra.run.smithi031.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T02:23:37.764 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:23:37.764 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 ::1/128 scope host 2023-10-06T02:23:37.764 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:23:37.765 INFO:teuthology.orchestra.run.smithi031.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T02:23:37.765 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 6a:7f:ca:93:81:ed brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-06T02:23:37.765 INFO:teuthology.orchestra.run.smithi031.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-06T02:23:37.765 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:23:37.765 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::687f:caff:fe93:81ed/64 scope link 2023-10-06T02:23:37.765 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:23:37.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:23:37.767 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-10-06T02:23:37.829 INFO:teuthology.orchestra.run.smithi031.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T02:23:37.829 INFO:teuthology.orchestra.run.smithi031.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T02:23:37.829 INFO:teuthology.orchestra.run.smithi031.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T02:23:37.829 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:23:37.829 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 ::1/128 scope host 2023-10-06T02:23:37.829 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:23:37.829 INFO:teuthology.orchestra.run.smithi031.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T02:23:37.829 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 6a:7f:ca:93:81:ed brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-06T02:23:37.829 INFO:teuthology.orchestra.run.smithi031.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-06T02:23:37.829 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:23:37.830 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::687f:caff:fe93:81ed/64 scope link 2023-10-06T02:23:37.830 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:23:37.832 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-10-06T02:23:37.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:23:37.832 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2023-10-06T02:23:37.832 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2023-10-06T02:23:37.832 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-10-06T02:23:37.832 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-10-06T02:23:37.832 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-10-06T02:23:37.832 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-10-06T02:23:37.833 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-10-06T02:23:37.833 DEBUG:teuthology.orchestra.run.smithi031:> ') 2023-10-06T02:23:38.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:23:38.063 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2023-10-06T02:23:38.064 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2023-10-06T02:23:38.064 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set brx.1 up 2023-10-06T02:23:38.064 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set dev brx.1 master ceph-brx 2023-10-06T02:23:38.064 DEBUG:teuthology.orchestra.run.smithi031:> ') 2023-10-06T02:23:38.225 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T02:23:38.225 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-10-06T02:23:38.261 INFO:teuthology.orchestra.run.smithi031.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-10-06T02:23:38.261 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T02:23:38.261 DEBUG:teuthology.orchestra.run.smithi031:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-10-06T02:23:38.336 INFO:tasks.cephfs.kernel_mount:mounting using device: 2@.cephfs=/ 2023-10-06T02:23:38.336 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T02:23:38.336 DEBUG:teuthology.orchestra.run.smithi031:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 2@.cephfs=/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T02:23:38.468 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T02:23:38.468 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T02:23:38.497 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T02:23:38.497 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T02:23:38.497 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.62:3300,172.21.15.62:3301" 2023-10-06T02:23:38.497 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: trying mount with new device syntax: 2@d6fcf512-874b-4259-b90a-30ae070f00a9.cephfs=/ 2023-10-06T02:23:38.497 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.31:3300/172.21.15.62:3300/172.21.15.62:3301" will pass to kernel 2023-10-06T02:23:38.502 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T02:23:38.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:23:38.502 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-10-06T02:23:38.564 INFO:tasks.kclient:client.1 config is {'syntax': 'v2'} 2023-10-06T02:23:38.564 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T02:23:38.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:38.749+0000 7f029339d700 1 Processor -- start 2023-10-06T02:23:38.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:38.749+0000 7f029339d700 1 -- start start 2023-10-06T02:23:38.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:38.750+0000 7f029339d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f028c159a10 0x7f028c0ca7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:38.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:38.750+0000 7f029339d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f028c0c7420 con 0x7f028c070410 2023-10-06T02:23:38.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:38.750+0000 7f029339d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f028c06d820 con 0x7f028c159640 2023-10-06T02:23:38.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:38.750+0000 7f029339d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f028c06db00 con 0x7f028c159a10 2023-10-06T02:23:38.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:38.751+0000 7f0291b9a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f028c159a10 0x7f028c0ca7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:38.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:38.751+0000 7f0292b9c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f028c070410 0x7f028c0c78b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:51326/0 (socket says 172.21.15.31:51326) 2023-10-06T02:23:38.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:38.751+0000 7f0291b9a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f028c159a10 0x7f028c0ca7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45658/0 (socket says 172.21.15.31:45658) 2023-10-06T02:23:38.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:38.751+0000 7f0292b9c700 1 -- 172.21.15.31:0/1853080647 learned_addr learned my addr 172.21.15.31:0/1853080647 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:23:38.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:38.751+0000 7f029239b700 1 --1- 172.21.15.31:0/1853080647 >> v1:172.21.15.62:6789/0 conn(0x7f028c159640 0x7f028c06d060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:39808/0 (socket says 172.21.15.31:39808) 2023-10-06T02:23:38.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:38.751+0000 7f0291b9a700 1 -- 172.21.15.31:0/1853080647 >> v1:172.21.15.62:6789/0 conn(0x7f028c159640 legacy=0x7f028c06d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:38.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:38.751+0000 7f0291b9a700 1 -- 172.21.15.31:0/1853080647 >> v1:172.21.15.31:6789/0 conn(0x7f028c070410 legacy=0x7f028c0c78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:38.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:38.751+0000 7f0291b9a700 1 -- 172.21.15.31:0/1853080647 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f028c06dde0 con 0x7f028c159a10 2023-10-06T02:23:38.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:38.751+0000 7f0291399700 1 -- 172.21.15.31:0/1853080647 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 923544578 0 0) 0x7f028c0c7420 con 0x7f028c070410 2023-10-06T02:23:38.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:38.752+0000 7f0291b9a700 1 --2- 172.21.15.31:0/1853080647 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f028c159a10 0x7f028c0ca7b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f027c0098e0 tx=0x7f027c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4af5ee3203fbc3f1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:38.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:38.752+0000 7f0291399700 1 -- 172.21.15.31:0/1853080647 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027c009e80 con 0x7f028c159a10 2023-10-06T02:23:38.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:38.752+0000 7f0291399700 1 -- 172.21.15.31:0/1853080647 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f027c00b860 con 0x7f028c159a10 2023-10-06T02:23:38.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:38.752+0000 7f0291399700 1 -- 172.21.15.31:0/1853080647 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027c00bbe0 con 0x7f028c159a10 2023-10-06T02:23:38.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:38.752+0000 7f029339d700 1 -- 172.21.15.31:0/1853080647 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f028c159a10 msgr2=0x7f028c0ca7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:38.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:38.752+0000 7f029339d700 1 --2- 172.21.15.31:0/1853080647 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f028c159a10 0x7f028c0ca7b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f027c0098e0 tx=0x7f027c0091a0 comp rx=0 tx=0).stop 2023-10-06T02:23:38.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:38.752+0000 7f029339d700 1 -- 172.21.15.31:0/1853080647 shutdown_connections 2023-10-06T02:23:38.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:38.752+0000 7f029339d700 1 --2- 172.21.15.31:0/1853080647 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f028c159a10 0x7f028c0ca7b0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:38.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:38.752+0000 7f029339d700 1 -- 172.21.15.31:0/1853080647 >> 172.21.15.31:0/1853080647 conn(0x7f028c06e720 msgr2=0x7f028c06eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:38.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:38.752+0000 7f029339d700 1 -- 172.21.15.31:0/1853080647 shutdown_connections 2023-10-06T02:23:38.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:38.752+0000 7f029339d700 1 -- 172.21.15.31:0/1853080647 wait complete. 2023-10-06T02:23:38.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:38.754+0000 7f029339d700 1 Processor -- start 2023-10-06T02:23:38.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:38.754+0000 7f029339d700 1 -- start start 2023-10-06T02:23:38.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:38.755+0000 7f029339d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f028c070410 0x7f028c0b1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:38.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:38.755+0000 7f029339d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f028c159640 0x7f028c06eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:38.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:38.755+0000 7f029339d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f028c159a10 0x7f028c06f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:38.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:38.755+0000 7f029339d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f028c0b0880 con 0x7f028c070410 2023-10-06T02:23:38.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:38.755+0000 7f029339d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f028c0aff50 con 0x7f028c159640 2023-10-06T02:23:38.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:38.755+0000 7f029339d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f028c0b0230 con 0x7f028c159a10 2023-10-06T02:23:38.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:38.755+0000 7f029239b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f028c070410 0x7f028c0b1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:38.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:38.755+0000 7f029239b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f028c070410 0x7f028c0b1dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45660/0 (socket says 172.21.15.31:45660) 2023-10-06T02:23:38.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:38.755+0000 7f029239b700 1 -- 172.21.15.31:0/38992057 learned_addr learned my addr 172.21.15.31:0/38992057 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:38.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:38.755+0000 7f0292b9c700 1 --2- 172.21.15.31:0/38992057 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f028c159a10 0x7f028c06f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:38.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:38.755+0000 7f0291b9a700 1 --2- 172.21.15.31:0/38992057 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f028c159640 0x7f028c06eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:38.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:38.756+0000 7f029239b700 1 -- 172.21.15.31:0/38992057 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f028c159a10 msgr2=0x7f028c06f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:38.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:38.756+0000 7f029239b700 1 --2- 172.21.15.31:0/38992057 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f028c159a10 0x7f028c06f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:38.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:38.756+0000 7f029239b700 1 -- 172.21.15.31:0/38992057 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f028c159640 msgr2=0x7f028c06eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:38.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:38.756+0000 7f029239b700 1 --2- 172.21.15.31:0/38992057 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f028c159640 0x7f028c06eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:38.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:38.756+0000 7f029239b700 1 -- 172.21.15.31:0/38992057 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f027c009420 con 0x7f028c070410 2023-10-06T02:23:38.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:38.756+0000 7f0291b9a700 1 --2- 172.21.15.31:0/38992057 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f028c159640 0x7f028c06eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:23:38.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:38.756+0000 7f029239b700 1 --2- 172.21.15.31:0/38992057 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f028c070410 0x7f028c0b1dd0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f028800c7c0 tx=0x7f0288007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:38.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:38.756+0000 7f02827fc700 1 -- 172.21.15.31:0/38992057 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0288005a70 con 0x7f028c070410 2023-10-06T02:23:38.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:38.756+0000 7f02827fc700 1 -- 172.21.15.31:0/38992057 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0288005bf0 con 0x7f028c070410 2023-10-06T02:23:38.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:38.756+0000 7f02827fc700 1 -- 172.21.15.31:0/38992057 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02880070f0 con 0x7f028c070410 2023-10-06T02:23:38.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:38.756+0000 7f029339d700 1 -- 172.21.15.31:0/38992057 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f028c0b0510 con 0x7f028c070410 2023-10-06T02:23:38.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:38.756+0000 7f029339d700 1 -- 172.21.15.31:0/38992057 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f028c0af860 con 0x7f028c070410 2023-10-06T02:23:38.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:38.757+0000 7f029339d700 1 -- 172.21.15.31:0/38992057 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f028c0afca0 con 0x7f028c070410 2023-10-06T02:23:38.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:38.758+0000 7f02827fc700 1 -- 172.21.15.31:0/38992057 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0288007460 con 0x7f028c070410 2023-10-06T02:23:38.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:38.759+0000 7f02827fc700 1 --2- 172.21.15.31:0/38992057 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f0270066f30 0x7f02700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:38.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:38.759+0000 7f02827fc700 1 -- 172.21.15.31:0/38992057 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f02880841e0 con 0x7f028c070410 2023-10-06T02:23:38.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:38.759+0000 7f0291b9a700 1 --2- 172.21.15.31:0/38992057 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f0270066f30 0x7f02700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:38.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:38.760+0000 7f0291b9a700 1 --2- 172.21.15.31:0/38992057 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f0270066f30 0x7f02700693e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f028c159e50 tx=0x7f027c017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:38.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:38.763+0000 7f02827fc700 1 -- 172.21.15.31:0/38992057 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f02880512d0 con 0x7f028c070410 2023-10-06T02:23:39.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:39.010+0000 7f029339d700 1 -- 172.21.15.31:0/38992057 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f028c04c3c0 con 0x7f028c070410 2023-10-06T02:23:39.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:39.010+0000 7f02827fc700 1 -- 172.21.15.31:0/38992057 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f02880548f0 con 0x7f028c070410 2023-10-06T02:23:39.011 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T02:23:39.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:39.013+0000 7f029339d700 1 -- 172.21.15.31:0/38992057 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f0270066f30 msgr2=0x7f02700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:39.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:39.013+0000 7f029339d700 1 --2- 172.21.15.31:0/38992057 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f0270066f30 0x7f02700693e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f028c159e50 tx=0x7f027c017000 comp rx=0 tx=0).stop 2023-10-06T02:23:39.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:39.013+0000 7f029339d700 1 -- 172.21.15.31:0/38992057 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f028c070410 msgr2=0x7f028c0b1dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:39.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:39.013+0000 7f029339d700 1 --2- 172.21.15.31:0/38992057 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f028c070410 0x7f028c0b1dd0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f028800c7c0 tx=0x7f0288007f70 comp rx=0 tx=0).stop 2023-10-06T02:23:39.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:39.013+0000 7f029339d700 1 -- 172.21.15.31:0/38992057 shutdown_connections 2023-10-06T02:23:39.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:39.013+0000 7f029339d700 1 --2- 172.21.15.31:0/38992057 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f0270066f30 0x7f02700693e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:39.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:39.013+0000 7f029339d700 1 --2- 172.21.15.31:0/38992057 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f028c159a10 0x7f028c06f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:39.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:39.013+0000 7f029339d700 1 --2- 172.21.15.31:0/38992057 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f028c070410 0x7f028c0b1dd0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:39.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:39.013+0000 7f029339d700 1 --2- 172.21.15.31:0/38992057 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f028c159640 0x7f028c06eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:39.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:39.013+0000 7f029339d700 1 -- 172.21.15.31:0/38992057 >> 172.21.15.31:0/38992057 conn(0x7f028c06e720 msgr2=0x7f028c157d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:39.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:39.013+0000 7f029339d700 1 -- 172.21.15.31:0/38992057 shutdown_connections 2023-10-06T02:23:39.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:39.013+0000 7f029339d700 1 -- 172.21.15.31:0/38992057 wait complete. 2023-10-06T02:23:39.030 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T02:23:39.030 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T02:23:39.030 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi062.front.sepia.ceph.com 2023-10-06T02:23:39.030 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-06T02:23:39.030 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-06T02:23:39.030 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T02:23:39.030 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T02:23:39.030 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T02:23:39.031 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T02:23:39.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:23:39.031 DEBUG:teuthology.orchestra.run.smithi062:> ip addr 2023-10-06T02:23:39.072 INFO:teuthology.orchestra.run.smithi062.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T02:23:39.072 INFO:teuthology.orchestra.run.smithi062.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T02:23:39.072 INFO:teuthology.orchestra.run.smithi062.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T02:23:39.073 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:23:39.073 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 ::1/128 scope host 2023-10-06T02:23:39.073 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:23:39.073 INFO:teuthology.orchestra.run.smithi062.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T02:23:39.073 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:14 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:23:39.073 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f0 2023-10-06T02:23:39.073 INFO:teuthology.orchestra.run.smithi062.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T02:23:39.073 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:15 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:23:39.073 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f1 2023-10-06T02:23:39.073 INFO:teuthology.orchestra.run.smithi062.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T02:23:39.073 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c2 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:23:39.073 INFO:teuthology.orchestra.run.smithi062.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T02:23:39.073 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c3 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:23:39.074 INFO:teuthology.orchestra.run.smithi062.stdout: inet 172.21.15.62/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T02:23:39.074 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft 42230sec preferred_lft 42230sec 2023-10-06T02:23:39.074 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::ec4:7aff:febb:d1c3/64 scope link 2023-10-06T02:23:39.074 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:23:39.074 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-06T02:23:39.074 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:23:39.074 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2023-10-06T02:23:39.074 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2023-10-06T02:23:39.075 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link add name ceph-brx type bridge 2023-10-06T02:23:39.075 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip addr flush dev ceph-brx 2023-10-06T02:23:39.075 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set ceph-brx up 2023-10-06T02:23:39.075 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-06T02:23:39.075 DEBUG:teuthology.orchestra.run.smithi062:> ') 2023-10-06T02:23:39.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:23:39.527 DEBUG:teuthology.orchestra.run.smithi062:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T02:23:39.687 INFO:teuthology.orchestra.run.smithi062.stdout:1 2023-10-06T02:23:39.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:23:39.691 DEBUG:teuthology.orchestra.run.smithi062:> route 2023-10-06T02:23:39.746 INFO:teuthology.orchestra.run.smithi062.stdout:Kernel IP routing table 2023-10-06T02:23:39.747 INFO:teuthology.orchestra.run.smithi062.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T02:23:39.747 INFO:teuthology.orchestra.run.smithi062.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T02:23:39.747 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T02:23:39.747 INFO:teuthology.orchestra.run.smithi062.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T02:23:39.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:23:39.748 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2023-10-06T02:23:39.748 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2023-10-06T02:23:39.748 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T02:23:39.749 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T02:23:39.749 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T02:23:39.749 DEBUG:teuthology.orchestra.run.smithi062:> ') 2023-10-06T02:23:40.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:23:40.535 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2023-10-06T02:23:40.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:23:40.572 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list-id 2023-10-06T02:23:40.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:23:40.649 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2023-10-06T02:23:40.649 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2023-10-06T02:23:40.649 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T02:23:40.649 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-10-06T02:23:40.649 DEBUG:teuthology.orchestra.run.smithi062:> ') 2023-10-06T02:23:40.815 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-10-06T02:23:40.816 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:23:40.816 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2023-10-06T02:23:40.816 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2023-10-06T02:23:40.816 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-10-06T02:23:40.816 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-06T02:23:40.816 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-10-06T02:23:40.816 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-10-06T02:23:40.816 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-10-06T02:23:40.816 DEBUG:teuthology.orchestra.run.smithi062:> ') 2023-10-06T02:23:41.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:23:41.072 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2023-10-06T02:23:41.072 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2023-10-06T02:23:41.072 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set brx.0 up 2023-10-06T02:23:41.072 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set dev brx.0 master ceph-brx 2023-10-06T02:23:41.072 DEBUG:teuthology.orchestra.run.smithi062:> ') 2023-10-06T02:23:41.231 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T02:23:41.231 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-06T02:23:41.268 INFO:teuthology.orchestra.run.smithi062.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-06T02:23:41.269 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T02:23:41.269 DEBUG:teuthology.orchestra.run.smithi062:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-06T02:23:41.344 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-06T02:23:41.344 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T02:23:41.344 DEBUG:teuthology.orchestra.run.smithi062:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T02:23:41.489 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T02:23:41.489 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T02:23:41.943 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T02:23:41.943 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T02:23:41.943 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.62:3300,172.21.15.62:3301" 2023-10-06T02:23:41.943 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: trying mount with new device syntax: 1@d6fcf512-874b-4259-b90a-30ae070f00a9.cephfs=/ 2023-10-06T02:23:41.943 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.31:3300/172.21.15.62:3300/172.21.15.62:3301" will pass to kernel 2023-10-06T02:23:41.948 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T02:23:41.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:23:41.948 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-06T02:23:42.012 INFO:tasks.kclient:client.3 config is {'syntax': 'v2'} 2023-10-06T02:23:42.013 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T02:23:42.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:42.210+0000 7fba74b9d700 1 Processor -- start 2023-10-06T02:23:42.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:42.210+0000 7fba74b9d700 1 -- start start 2023-10-06T02:23:42.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:42.211+0000 7fba74b9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fba700cf110 0x7fba70164600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:42.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:42.211+0000 7fba74b9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba7015afd0 con 0x7fba70153c30 2023-10-06T02:23:42.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:42.211+0000 7fba74b9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba7015b2b0 con 0x7fba7015de50 2023-10-06T02:23:42.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:42.211+0000 7fba74b9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba7015b590 con 0x7fba700cf110 2023-10-06T02:23:42.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:42.212+0000 7fba6f7fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fba70153c30 0x7fba7015a5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:51336/0 (socket says 172.21.15.31:51336) 2023-10-06T02:23:42.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:42.212+0000 7fba6f7fe700 1 -- 172.21.15.31:0/3528864911 learned_addr learned my addr 172.21.15.31:0/3528864911 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:23:42.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:42.212+0000 7fba6ffff700 1 --2- 172.21.15.31:0/3528864911 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fba700cf110 0x7fba70164600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:42.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:42.212+0000 7fba6e7fc700 1 -- 172.21.15.31:0/3528864911 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2259932818 0 0) 0x7fba7015afd0 con 0x7fba70153c30 2023-10-06T02:23:42.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:42.212+0000 7fba6e7fc700 1 -- 172.21.15.31:0/3528864911 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba58003570 con 0x7fba70153c30 2023-10-06T02:23:42.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:42.212+0000 7fba6e7fc700 1 -- 172.21.15.31:0/3528864911 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2659084324 0 0) 0x7fba7015b2b0 con 0x7fba7015de50 2023-10-06T02:23:42.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:42.212+0000 7fba6e7fc700 1 -- 172.21.15.31:0/3528864911 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba7015afd0 con 0x7fba7015de50 2023-10-06T02:23:42.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:42.212+0000 7fba6ffff700 1 -- 172.21.15.31:0/3528864911 >> v1:172.21.15.62:6789/0 conn(0x7fba7015de50 legacy=0x7fba700cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:42.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:42.212+0000 7fba6ffff700 1 -- 172.21.15.31:0/3528864911 >> v1:172.21.15.31:6789/0 conn(0x7fba70153c30 legacy=0x7fba7015a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:42.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:42.212+0000 7fba6e7fc700 1 -- 172.21.15.31:0/3528864911 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1820138252 0 0) 0x7fba58003570 con 0x7fba70153c30 2023-10-06T02:23:42.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:42.212+0000 7fba6ffff700 1 -- 172.21.15.31:0/3528864911 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba7015b870 con 0x7fba700cf110 2023-10-06T02:23:42.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:42.213+0000 7fba6ffff700 1 --2- 172.21.15.31:0/3528864911 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fba700cf110 0x7fba70164600 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fba68009e90 tx=0x7fba68009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b8c05ad3d61daf9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:42.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:42.213+0000 7fba6e7fc700 1 -- 172.21.15.31:0/3528864911 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6800e070 con 0x7fba700cf110 2023-10-06T02:23:42.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:42.213+0000 7fba6e7fc700 1 -- 172.21.15.31:0/3528864911 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba6800bb80 con 0x7fba700cf110 2023-10-06T02:23:42.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:42.213+0000 7fba6e7fc700 1 -- 172.21.15.31:0/3528864911 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba68012430 con 0x7fba700cf110 2023-10-06T02:23:42.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:42.213+0000 7fba74b9d700 1 -- 172.21.15.31:0/3528864911 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fba700cf110 msgr2=0x7fba70164600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:42.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:42.213+0000 7fba74b9d700 1 --2- 172.21.15.31:0/3528864911 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fba700cf110 0x7fba70164600 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fba68009e90 tx=0x7fba68009780 comp rx=0 tx=0).stop 2023-10-06T02:23:42.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:42.214+0000 7fba74b9d700 1 -- 172.21.15.31:0/3528864911 shutdown_connections 2023-10-06T02:23:42.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:42.214+0000 7fba74b9d700 1 --2- 172.21.15.31:0/3528864911 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fba700cf110 0x7fba70164600 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:42.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:42.214+0000 7fba74b9d700 1 -- 172.21.15.31:0/3528864911 >> 172.21.15.31:0/3528864911 conn(0x7fba700703a0 msgr2=0x7fba700707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:42.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:42.214+0000 7fba74b9d700 1 -- 172.21.15.31:0/3528864911 shutdown_connections 2023-10-06T02:23:42.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:42.214+0000 7fba74b9d700 1 -- 172.21.15.31:0/3528864911 wait complete. 2023-10-06T02:23:42.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:42.215+0000 7fba74b9d700 1 Processor -- start 2023-10-06T02:23:42.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:42.215+0000 7fba74b9d700 1 -- start start 2023-10-06T02:23:42.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:42.216+0000 7fba74b9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fba700cf110 0x7fba70151d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:42.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:42.216+0000 7fba74b9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba70153c30 0x7fba700c9f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:42.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:42.216+0000 7fba74b9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba7015de50 0x7fba7015ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:42.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:42.216+0000 7fba74b9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba7015eab0 con 0x7fba7015de50 2023-10-06T02:23:42.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:42.216+0000 7fba74b9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba701526d0 con 0x7fba70153c30 2023-10-06T02:23:42.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:42.216+0000 7fba74b9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba701523f0 con 0x7fba700cf110 2023-10-06T02:23:42.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:42.216+0000 7fba6ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba7015de50 0x7fba7015ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:42.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:42.216+0000 7fba6ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba7015de50 0x7fba7015ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45662/0 (socket says 172.21.15.31:45662) 2023-10-06T02:23:42.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:42.216+0000 7fba6effd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba70153c30 0x7fba700c9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:42.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:42.216+0000 7fba6ffff700 1 -- 172.21.15.31:0/4149523061 learned_addr learned my addr 172.21.15.31:0/4149523061 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:42.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:42.216+0000 7fba6f7fe700 1 --2- 172.21.15.31:0/4149523061 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fba700cf110 0x7fba70151d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:42.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:42.217+0000 7fba6effd700 1 -- 172.21.15.31:0/4149523061 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fba700cf110 msgr2=0x7fba70151d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:42.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:42.217+0000 7fba6effd700 1 --2- 172.21.15.31:0/4149523061 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fba700cf110 0x7fba70151d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:42.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:42.217+0000 7fba6effd700 1 -- 172.21.15.31:0/4149523061 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba7015de50 msgr2=0x7fba7015ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:42.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:42.217+0000 7fba6effd700 1 --2- 172.21.15.31:0/4149523061 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba7015de50 0x7fba7015ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:42.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:42.217+0000 7fba6effd700 1 -- 172.21.15.31:0/4149523061 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba68009a20 con 0x7fba70153c30 2023-10-06T02:23:42.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:42.217+0000 7fba6ffff700 1 --2- 172.21.15.31:0/4149523061 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba7015de50 0x7fba7015ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:23:42.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:42.217+0000 7fba6effd700 1 --2- 172.21.15.31:0/4149523061 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba70153c30 0x7fba700c9f20 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fba6400c4b0 tx=0x7fba64007d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:42.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:42.217+0000 7fba577fe700 1 -- 172.21.15.31:0/4149523061 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6400ce60 con 0x7fba70153c30 2023-10-06T02:23:42.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:42.217+0000 7fba577fe700 1 -- 172.21.15.31:0/4149523061 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba64011590 con 0x7fba70153c30 2023-10-06T02:23:42.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:42.217+0000 7fba74b9d700 1 -- 172.21.15.31:0/4149523061 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba7015e460 con 0x7fba70153c30 2023-10-06T02:23:42.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:42.218+0000 7fba577fe700 1 -- 172.21.15.31:0/4149523061 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba64011950 con 0x7fba70153c30 2023-10-06T02:23:42.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:42.218+0000 7fba74b9d700 1 -- 172.21.15.31:0/4149523061 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba70142440 con 0x7fba70153c30 2023-10-06T02:23:42.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:42.219+0000 7fba74b9d700 1 -- 172.21.15.31:0/4149523061 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba7004c3c0 con 0x7fba70153c30 2023-10-06T02:23:42.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:42.219+0000 7fba577fe700 1 -- 172.21.15.31:0/4149523061 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fba64011d00 con 0x7fba70153c30 2023-10-06T02:23:42.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:42.220+0000 7fba577fe700 1 --2- 172.21.15.31:0/4149523061 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fba5c066f30 0x7fba5c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:42.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:42.220+0000 7fba577fe700 1 -- 172.21.15.31:0/4149523061 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fba640841e0 con 0x7fba70153c30 2023-10-06T02:23:42.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:42.221+0000 7fba6f7fe700 1 --2- 172.21.15.31:0/4149523061 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fba5c066f30 0x7fba5c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:42.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:42.221+0000 7fba6f7fe700 1 --2- 172.21.15.31:0/4149523061 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fba5c066f30 0x7fba5c0693e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fba600011e0 tx=0x7fba60007ad0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:42.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:42.225+0000 7fba577fe700 1 -- 172.21.15.31:0/4149523061 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fba64051250 con 0x7fba70153c30 2023-10-06T02:23:42.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:42.473+0000 7fba74b9d700 1 -- 172.21.15.31:0/4149523061 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fba7015ebf0 con 0x7fba70153c30 2023-10-06T02:23:42.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:42.474+0000 7fba577fe700 1 -- 172.21.15.31:0/4149523061 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7fba64054870 con 0x7fba70153c30 2023-10-06T02:23:42.474 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T02:23:42.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:42.476+0000 7fba74b9d700 1 -- 172.21.15.31:0/4149523061 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fba5c066f30 msgr2=0x7fba5c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:42.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:42.476+0000 7fba74b9d700 1 --2- 172.21.15.31:0/4149523061 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fba5c066f30 0x7fba5c0693e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fba600011e0 tx=0x7fba60007ad0 comp rx=0 tx=0).stop 2023-10-06T02:23:42.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:42.476+0000 7fba74b9d700 1 -- 172.21.15.31:0/4149523061 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba70153c30 msgr2=0x7fba700c9f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:42.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:42.476+0000 7fba74b9d700 1 --2- 172.21.15.31:0/4149523061 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba70153c30 0x7fba700c9f20 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fba6400c4b0 tx=0x7fba64007d50 comp rx=0 tx=0).stop 2023-10-06T02:23:42.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:42.477+0000 7fba74b9d700 1 -- 172.21.15.31:0/4149523061 shutdown_connections 2023-10-06T02:23:42.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:42.477+0000 7fba74b9d700 1 --2- 172.21.15.31:0/4149523061 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba7015de50 0x7fba7015ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:42.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:42.477+0000 7fba74b9d700 1 --2- 172.21.15.31:0/4149523061 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fba5c066f30 0x7fba5c0693e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:42.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:42.477+0000 7fba74b9d700 1 --2- 172.21.15.31:0/4149523061 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fba700cf110 0x7fba70151d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:42.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:42.477+0000 7fba74b9d700 1 --2- 172.21.15.31:0/4149523061 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba70153c30 0x7fba700c9f20 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:42.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:42.477+0000 7fba74b9d700 1 -- 172.21.15.31:0/4149523061 >> 172.21.15.31:0/4149523061 conn(0x7fba700703a0 msgr2=0x7fba70070770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:42.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:42.477+0000 7fba74b9d700 1 -- 172.21.15.31:0/4149523061 shutdown_connections 2023-10-06T02:23:42.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:42.477+0000 7fba74b9d700 1 -- 172.21.15.31:0/4149523061 wait complete. 2023-10-06T02:23:42.494 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T02:23:42.494 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T02:23:42.494 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi062.front.sepia.ceph.com 2023-10-06T02:23:42.494 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-10-06T02:23:42.494 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-10-06T02:23:42.494 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T02:23:42.494 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T02:23:42.495 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T02:23:42.495 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-10-06T02:23:42.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:23:42.495 DEBUG:teuthology.orchestra.run.smithi062:> ip addr 2023-10-06T02:23:42.533 INFO:teuthology.orchestra.run.smithi062.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T02:23:42.533 INFO:teuthology.orchestra.run.smithi062.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T02:23:42.533 INFO:teuthology.orchestra.run.smithi062.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T02:23:42.534 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:23:42.534 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 ::1/128 scope host 2023-10-06T02:23:42.534 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:23:42.534 INFO:teuthology.orchestra.run.smithi062.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T02:23:42.534 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:14 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:23:42.534 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f0 2023-10-06T02:23:42.534 INFO:teuthology.orchestra.run.smithi062.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T02:23:42.534 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:15 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:23:42.534 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f1 2023-10-06T02:23:42.534 INFO:teuthology.orchestra.run.smithi062.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T02:23:42.534 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c2 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:23:42.535 INFO:teuthology.orchestra.run.smithi062.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T02:23:42.535 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c3 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:23:42.535 INFO:teuthology.orchestra.run.smithi062.stdout: inet 172.21.15.62/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T02:23:42.535 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft 42226sec preferred_lft 42226sec 2023-10-06T02:23:42.535 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::ec4:7aff:febb:d1c3/64 scope link 2023-10-06T02:23:42.535 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:23:42.535 INFO:teuthology.orchestra.run.smithi062.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T02:23:42.535 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether fe:60:2b:8e:af:28 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:23:42.535 INFO:teuthology.orchestra.run.smithi062.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T02:23:42.535 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:23:42.535 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::1098:a4ff:fe0d:389/64 scope link 2023-10-06T02:23:42.535 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:23:42.536 INFO:teuthology.orchestra.run.smithi062.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T02:23:42.536 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether fe:60:2b:8e:af:28 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T02:23:42.536 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::fc60:2bff:fe8e:af28/64 scope link tentative 2023-10-06T02:23:42.536 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:23:42.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:23:42.536 DEBUG:teuthology.orchestra.run.smithi062:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T02:23:42.634 INFO:teuthology.orchestra.run.smithi062.stdout:1 2023-10-06T02:23:42.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:23:42.637 DEBUG:teuthology.orchestra.run.smithi062:> route 2023-10-06T02:23:42.675 INFO:teuthology.orchestra.run.smithi062.stdout:Kernel IP routing table 2023-10-06T02:23:42.675 INFO:teuthology.orchestra.run.smithi062.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T02:23:42.675 INFO:teuthology.orchestra.run.smithi062.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T02:23:42.676 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T02:23:42.676 INFO:teuthology.orchestra.run.smithi062.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T02:23:42.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:23:42.677 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2023-10-06T02:23:42.677 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2023-10-06T02:23:42.677 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T02:23:42.677 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T02:23:42.677 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T02:23:42.677 DEBUG:teuthology.orchestra.run.smithi062:> ') 2023-10-06T02:23:42.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:23:42.874 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2023-10-06T02:23:42.910 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T02:23:42.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:23:42.911 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list-id 2023-10-06T02:23:42.986 INFO:teuthology.orchestra.run.smithi062.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-10-06T02:23:42.987 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:23:42.987 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2023-10-06T02:23:42.987 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2023-10-06T02:23:42.987 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T02:23:42.987 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-10-06T02:23:42.987 DEBUG:teuthology.orchestra.run.smithi062:> ') 2023-10-06T02:23:43.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:23:43.155 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-10-06T02:23:43.217 INFO:teuthology.orchestra.run.smithi062.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T02:23:43.218 INFO:teuthology.orchestra.run.smithi062.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T02:23:43.218 INFO:teuthology.orchestra.run.smithi062.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T02:23:43.218 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:23:43.218 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 ::1/128 scope host 2023-10-06T02:23:43.218 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:23:43.218 INFO:teuthology.orchestra.run.smithi062.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T02:23:43.218 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 32:c8:55:04:f4:9f brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-06T02:23:43.218 INFO:teuthology.orchestra.run.smithi062.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-06T02:23:43.218 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:23:43.218 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::30c8:55ff:fe04:f49f/64 scope link 2023-10-06T02:23:43.218 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:23:43.220 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:23:43.220 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-10-06T02:23:43.282 INFO:teuthology.orchestra.run.smithi062.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T02:23:43.282 INFO:teuthology.orchestra.run.smithi062.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T02:23:43.282 INFO:teuthology.orchestra.run.smithi062.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T02:23:43.282 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:23:43.282 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 ::1/128 scope host 2023-10-06T02:23:43.282 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:23:43.282 INFO:teuthology.orchestra.run.smithi062.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T02:23:43.283 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 32:c8:55:04:f4:9f brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-06T02:23:43.283 INFO:teuthology.orchestra.run.smithi062.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-06T02:23:43.283 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:23:43.283 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::30c8:55ff:fe04:f49f/64 scope link 2023-10-06T02:23:43.283 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:23:43.285 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-10-06T02:23:43.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:23:43.285 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2023-10-06T02:23:43.285 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2023-10-06T02:23:43.285 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-10-06T02:23:43.285 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-10-06T02:23:43.285 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-10-06T02:23:43.286 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-10-06T02:23:43.286 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-10-06T02:23:43.286 DEBUG:teuthology.orchestra.run.smithi062:> ') 2023-10-06T02:23:43.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:23:43.519 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2023-10-06T02:23:43.519 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2023-10-06T02:23:43.519 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set brx.1 up 2023-10-06T02:23:43.519 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set dev brx.1 master ceph-brx 2023-10-06T02:23:43.519 DEBUG:teuthology.orchestra.run.smithi062:> ') 2023-10-06T02:23:43.677 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T02:23:43.677 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-10-06T02:23:43.714 INFO:teuthology.orchestra.run.smithi062.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-10-06T02:23:43.714 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T02:23:43.714 DEBUG:teuthology.orchestra.run.smithi062:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-10-06T02:23:43.789 INFO:tasks.cephfs.kernel_mount:mounting using device: 3@.cephfs=/ 2023-10-06T02:23:43.789 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T02:23:43.789 DEBUG:teuthology.orchestra.run.smithi062:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 3@.cephfs=/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T02:23:43.920 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T02:23:43.921 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T02:23:43.949 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T02:23:43.949 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T02:23:43.949 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.62:3300,172.21.15.62:3301" 2023-10-06T02:23:43.950 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: trying mount with new device syntax: 3@d6fcf512-874b-4259-b90a-30ae070f00a9.cephfs=/ 2023-10-06T02:23:43.950 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.31:3300/172.21.15.62:3300/172.21.15.62:3301" will pass to kernel 2023-10-06T02:23:43.955 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T02:23:43.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:23:43.955 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-10-06T02:23:44.017 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-10-06T02:23:44.033 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:23:44.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.232+0000 7fb92159d700 1 Processor -- start 2023-10-06T02:23:44.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.232+0000 7fb92159d700 1 -- start start 2023-10-06T02:23:44.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.233+0000 7fb92159d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb91c159640 0x7fb91c06d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:44.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.233+0000 7fb92159d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb91c0caec0 0x7fb91c0c78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:44.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.233+0000 7fb92159d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb91c0c7420 con 0x7fb91c159a10 2023-10-06T02:23:44.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.233+0000 7fb92159d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb91c06d5a0 con 0x7fb91c159640 2023-10-06T02:23:44.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.234+0000 7fb92159d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb91c06d820 con 0x7fb91c0caec0 2023-10-06T02:23:44.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.234+0000 7fb91bfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb91c159640 0x7fb91c06d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:44.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.234+0000 7fb91bfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb91c159640 0x7fb91c06d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45674/0 (socket says 172.21.15.31:45674) 2023-10-06T02:23:44.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.234+0000 7fb91bfff700 1 -- 172.21.15.31:0/1709930524 learned_addr learned my addr 172.21.15.31:0/1709930524 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:44.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.234+0000 7fb920d9c700 1 --2- 172.21.15.31:0/1709930524 >> v2:172.21.15.62:3300/0 conn(0x7fb91c0caec0 0x7fb91c0c78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:44.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.234+0000 7fb91bfff700 1 -- 172.21.15.31:0/1709930524 >> v2:172.21.15.62:3300/0 conn(0x7fb91c0caec0 msgr2=0x7fb91c0c78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:44.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.234+0000 7fb91bfff700 1 --2- 172.21.15.31:0/1709930524 >> v2:172.21.15.62:3300/0 conn(0x7fb91c0caec0 0x7fb91c0c78b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:44.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.234+0000 7fb91bfff700 1 -- 172.21.15.31:0/1709930524 >> v1:172.21.15.62:6789/0 conn(0x7fb91c159a10 legacy=0x7fb91c0ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:44.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.234+0000 7fb91bfff700 1 -- 172.21.15.31:0/1709930524 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb91c06db00 con 0x7fb91c159640 2023-10-06T02:23:44.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.234+0000 7fb91bfff700 1 --2- 172.21.15.31:0/1709930524 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb91c159640 0x7fb91c06d060 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb90c0099b0 tx=0x7fb90c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad96068672b775a0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:44.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.235+0000 7fb91affd700 1 -- 172.21.15.31:0/1709930524 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb90c00b820 con 0x7fb91c159640 2023-10-06T02:23:44.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.235+0000 7fb91affd700 1 -- 172.21.15.31:0/1709930524 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb90c00b980 con 0x7fb91c159640 2023-10-06T02:23:44.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.235+0000 7fb91affd700 1 -- 172.21.15.31:0/1709930524 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb90c00bd20 con 0x7fb91c159640 2023-10-06T02:23:44.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.235+0000 7fb92159d700 1 -- 172.21.15.31:0/1709930524 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb91c159640 msgr2=0x7fb91c06d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:44.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.235+0000 7fb92159d700 1 --2- 172.21.15.31:0/1709930524 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb91c159640 0x7fb91c06d060 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb90c0099b0 tx=0x7fb90c009250 comp rx=0 tx=0).stop 2023-10-06T02:23:44.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.235+0000 7fb92159d700 1 -- 172.21.15.31:0/1709930524 shutdown_connections 2023-10-06T02:23:44.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.235+0000 7fb92159d700 1 --2- 172.21.15.31:0/1709930524 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb91c159640 0x7fb91c06d060 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:44.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.235+0000 7fb92159d700 1 --2- 172.21.15.31:0/1709930524 >> v2:172.21.15.62:3300/0 conn(0x7fb91c0caec0 0x7fb91c0c78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:44.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.235+0000 7fb92159d700 1 -- 172.21.15.31:0/1709930524 >> 172.21.15.31:0/1709930524 conn(0x7fb91c06e720 msgr2=0x7fb91c06eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:44.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.235+0000 7fb92159d700 1 -- 172.21.15.31:0/1709930524 shutdown_connections 2023-10-06T02:23:44.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.235+0000 7fb92159d700 1 -- 172.21.15.31:0/1709930524 wait complete. 2023-10-06T02:23:44.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.237+0000 7fb92159d700 1 Processor -- start 2023-10-06T02:23:44.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.237+0000 7fb92159d700 1 -- start start 2023-10-06T02:23:44.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.238+0000 7fb92159d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb91c0caec0 0x7fb91c0c84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:44.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.238+0000 7fb92159d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb91c159640 0x7fb91c0b0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:44.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.238+0000 7fb92159d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb91c159a10 0x7fb91c1524e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:44.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.238+0000 7fb92159d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb91c0b10f0 con 0x7fb91c159640 2023-10-06T02:23:44.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.238+0000 7fb92159d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb91c0b1ef0 con 0x7fb91c159a10 2023-10-06T02:23:44.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.238+0000 7fb92159d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb91c152a20 con 0x7fb91c0caec0 2023-10-06T02:23:44.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.238+0000 7fb91b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb91c159640 0x7fb91c0b0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:44.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.238+0000 7fb91bfff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb91c0caec0 0x7fb91c0c84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:44.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.238+0000 7fb91b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb91c159640 0x7fb91c0b0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45680/0 (socket says 172.21.15.31:45680) 2023-10-06T02:23:44.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.238+0000 7fb91b7fe700 1 -- 172.21.15.31:0/2017920569 learned_addr learned my addr 172.21.15.31:0/2017920569 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:44.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.238+0000 7fb920d9c700 1 --2- 172.21.15.31:0/2017920569 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb91c159a10 0x7fb91c1524e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:44.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.239+0000 7fb91b7fe700 1 -- 172.21.15.31:0/2017920569 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb91c0caec0 msgr2=0x7fb91c0c84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:44.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.239+0000 7fb91b7fe700 1 --2- 172.21.15.31:0/2017920569 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb91c0caec0 0x7fb91c0c84f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:44.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.239+0000 7fb91b7fe700 1 -- 172.21.15.31:0/2017920569 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb91c159a10 msgr2=0x7fb91c1524e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:44.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.239+0000 7fb91b7fe700 1 --2- 172.21.15.31:0/2017920569 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb91c159a10 0x7fb91c1524e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:44.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.239+0000 7fb91b7fe700 1 -- 172.21.15.31:0/2017920569 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb90c0094f0 con 0x7fb91c159640 2023-10-06T02:23:44.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.239+0000 7fb91b7fe700 1 --2- 172.21.15.31:0/2017920569 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb91c159640 0x7fb91c0b0bb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb91000c590 tx=0x7fb910007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:44.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.239+0000 7fb903fff700 1 -- 172.21.15.31:0/2017920569 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb910011590 con 0x7fb91c159640 2023-10-06T02:23:44.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.239+0000 7fb903fff700 1 -- 172.21.15.31:0/2017920569 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb910011710 con 0x7fb91c159640 2023-10-06T02:23:44.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.239+0000 7fb903fff700 1 -- 172.21.15.31:0/2017920569 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb910011ad0 con 0x7fb91c159640 2023-10-06T02:23:44.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.239+0000 7fb92159d700 1 -- 172.21.15.31:0/2017920569 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb91c0b12e0 con 0x7fb91c159640 2023-10-06T02:23:44.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.239+0000 7fb92159d700 1 -- 172.21.15.31:0/2017920569 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb91c0b18a0 con 0x7fb91c159640 2023-10-06T02:23:44.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.241+0000 7fb92159d700 1 -- 172.21.15.31:0/2017920569 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb91c06a430 con 0x7fb91c159640 2023-10-06T02:23:44.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.241+0000 7fb903fff700 1 -- 172.21.15.31:0/2017920569 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb91001a360 con 0x7fb91c159640 2023-10-06T02:23:44.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.242+0000 7fb903fff700 1 --2- 172.21.15.31:0/2017920569 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fb908066f30 0x7fb9080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:44.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.242+0000 7fb903fff700 1 -- 172.21.15.31:0/2017920569 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb9100834e0 con 0x7fb91c159640 2023-10-06T02:23:44.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.242+0000 7fb91bfff700 1 --2- 172.21.15.31:0/2017920569 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fb908066f30 0x7fb9080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:44.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.243+0000 7fb91bfff700 1 --2- 172.21.15.31:0/2017920569 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fb908066f30 0x7fb9080693e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb90c0099b0 tx=0x7fb90c017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:44.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.247+0000 7fb903fff700 1 -- 172.21.15.31:0/2017920569 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb9100505f0 con 0x7fb91c159640 2023-10-06T02:23:44.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.452+0000 7fb92159d700 1 -- 172.21.15.31:0/2017920569 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb91c0b2060 con 0x7fb91c159640 2023-10-06T02:23:44.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.453+0000 7fb903fff700 1 -- 172.21.15.31:0/2017920569 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16818 (secure 0 0 0) 0x7fb910053c10 con 0x7fb91c159640 2023-10-06T02:23:44.454 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:23:44.454 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":23,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:23:10.970534+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:23:07.290269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T02:23:08.504679+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:23:44.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.456+0000 7fb92159d700 1 -- 172.21.15.31:0/2017920569 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fb908066f30 msgr2=0x7fb9080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:44.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.456+0000 7fb92159d700 1 --2- 172.21.15.31:0/2017920569 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fb908066f30 0x7fb9080693e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb90c0099b0 tx=0x7fb90c017000 comp rx=0 tx=0).stop 2023-10-06T02:23:44.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.456+0000 7fb92159d700 1 -- 172.21.15.31:0/2017920569 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb91c159640 msgr2=0x7fb91c0b0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:44.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.456+0000 7fb92159d700 1 --2- 172.21.15.31:0/2017920569 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb91c159640 0x7fb91c0b0bb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb91000c590 tx=0x7fb910007f50 comp rx=0 tx=0).stop 2023-10-06T02:23:44.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.456+0000 7fb92159d700 1 -- 172.21.15.31:0/2017920569 shutdown_connections 2023-10-06T02:23:44.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.456+0000 7fb92159d700 1 --2- 172.21.15.31:0/2017920569 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb91c159a10 0x7fb91c1524e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:44.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.456+0000 7fb92159d700 1 --2- 172.21.15.31:0/2017920569 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fb908066f30 0x7fb9080693e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:44.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.456+0000 7fb92159d700 1 --2- 172.21.15.31:0/2017920569 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb91c0caec0 0x7fb91c0c84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:44.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.456+0000 7fb92159d700 1 --2- 172.21.15.31:0/2017920569 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb91c159640 0x7fb91c0b0bb0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:44.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.456+0000 7fb92159d700 1 -- 172.21.15.31:0/2017920569 >> 172.21.15.31:0/2017920569 conn(0x7fb91c06e720 msgr2=0x7fb91c15bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:44.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.456+0000 7fb92159d700 1 -- 172.21.15.31:0/2017920569 shutdown_connections 2023-10-06T02:23:44.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.456+0000 7fb92159d700 1 -- 172.21.15.31:0/2017920569 wait complete. 2023-10-06T02:23:44.474 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T02:22:53.183425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T02:23:00.046967+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T02:23:07.290269+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T02:23:08.504679+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T02:23:44.474 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T02:23:44.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.663+0000 7f879bfff700 1 Processor -- start 2023-10-06T02:23:44.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.663+0000 7f879bfff700 1 -- start start 2023-10-06T02:23:44.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.665+0000 7f879bfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f879c159550 0x7f879c0ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:44.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.665+0000 7f879bfff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f879c15dfd0 0x7f879c0aedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:44.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.665+0000 7f879bfff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f879c0acc00 con 0x7f879c159920 2023-10-06T02:23:44.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.665+0000 7f879bfff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f879c0abff0 con 0x7f879c159550 2023-10-06T02:23:44.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.665+0000 7f879bfff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f879c0ac2d0 con 0x7f879c15dfd0 2023-10-06T02:23:44.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.665+0000 7f879affd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f879c159550 0x7f879c0ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:44.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.665+0000 7f879b7fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f879c159920 0x7f879c0ad090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:51340/0 (socket says 172.21.15.31:51340) 2023-10-06T02:23:44.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.665+0000 7f879b7fe700 1 -- 172.21.15.31:0/3778955930 learned_addr learned my addr 172.21.15.31:0/3778955930 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:23:44.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.665+0000 7f879a7fc700 1 --2- 172.21.15.31:0/3778955930 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f879c15dfd0 0x7f879c0aedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:44.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.665+0000 7f879affd700 1 -- 172.21.15.31:0/3778955930 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f879c15dfd0 msgr2=0x7f879c0aedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:44.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.665+0000 7f879affd700 1 --2- 172.21.15.31:0/3778955930 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f879c15dfd0 0x7f879c0aedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:44.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.665+0000 7f879affd700 1 -- 172.21.15.31:0/3778955930 >> v1:172.21.15.31:6789/0 conn(0x7f879c159920 legacy=0x7f879c0ad090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:44.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.665+0000 7f879affd700 1 -- 172.21.15.31:0/3778955930 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f879c0ac5b0 con 0x7f879c159550 2023-10-06T02:23:44.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.666+0000 7f879affd700 1 --2- 172.21.15.31:0/3778955930 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f879c159550 0x7f879c0ceac0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f878c0098e0 tx=0x7f878c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a499709c1912a1a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:44.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.666+0000 7f8799ffb700 1 -- 172.21.15.31:0/3778955930 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f878c009e80 con 0x7f879c159550 2023-10-06T02:23:44.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.666+0000 7f8799ffb700 1 -- 172.21.15.31:0/3778955930 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f878c00b860 con 0x7f879c159550 2023-10-06T02:23:44.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.666+0000 7f8799ffb700 1 -- 172.21.15.31:0/3778955930 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f878c00bbe0 con 0x7f879c159550 2023-10-06T02:23:44.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.666+0000 7f879bfff700 1 -- 172.21.15.31:0/3778955930 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f879c159550 msgr2=0x7f879c0ceac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:44.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.666+0000 7f879bfff700 1 --2- 172.21.15.31:0/3778955930 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f879c159550 0x7f879c0ceac0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f878c0098e0 tx=0x7f878c0091a0 comp rx=0 tx=0).stop 2023-10-06T02:23:44.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.667+0000 7f879bfff700 1 -- 172.21.15.31:0/3778955930 shutdown_connections 2023-10-06T02:23:44.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.667+0000 7f879bfff700 1 --2- 172.21.15.31:0/3778955930 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f879c159550 0x7f879c0ceac0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:44.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.667+0000 7f879bfff700 1 --2- 172.21.15.31:0/3778955930 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f879c15dfd0 0x7f879c0aedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:44.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.667+0000 7f879bfff700 1 -- 172.21.15.31:0/3778955930 >> 172.21.15.31:0/3778955930 conn(0x7f879c06ce80 msgr2=0x7f879c0ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:44.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.667+0000 7f879bfff700 1 -- 172.21.15.31:0/3778955930 shutdown_connections 2023-10-06T02:23:44.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.667+0000 7f879bfff700 1 -- 172.21.15.31:0/3778955930 wait complete. 2023-10-06T02:23:44.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.668+0000 7f879bfff700 1 Processor -- start 2023-10-06T02:23:44.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.668+0000 7f879bfff700 1 -- start start 2023-10-06T02:23:44.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.669+0000 7f879bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f879c159550 0x7f879c151de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:44.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.669+0000 7f879bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f879c159920 0x7f879c0c8980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:44.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.669+0000 7f879bfff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f879c15dfd0 0x7f879c0b2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:44.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.669+0000 7f879bfff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f879c0b1250 con 0x7f879c159550 2023-10-06T02:23:44.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.669+0000 7f879bfff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f879c0b1dd0 con 0x7f879c159920 2023-10-06T02:23:44.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.669+0000 7f879bfff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f879c152320 con 0x7f879c15dfd0 2023-10-06T02:23:44.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.669+0000 7f879affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f879c159550 0x7f879c151de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:44.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.669+0000 7f879affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f879c159550 0x7f879c151de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45696/0 (socket says 172.21.15.31:45696) 2023-10-06T02:23:44.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.669+0000 7f879affd700 1 -- 172.21.15.31:0/4206482976 learned_addr learned my addr 172.21.15.31:0/4206482976 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:44.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.670+0000 7f879a7fc700 1 --2- 172.21.15.31:0/4206482976 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f879c159920 0x7f879c0c8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:44.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.670+0000 7f879b7fe700 1 --2- 172.21.15.31:0/4206482976 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f879c15dfd0 0x7f879c0b2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:44.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.670+0000 7f879affd700 1 -- 172.21.15.31:0/4206482976 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f879c15dfd0 msgr2=0x7f879c0b2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:44.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.670+0000 7f879affd700 1 --2- 172.21.15.31:0/4206482976 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f879c15dfd0 0x7f879c0b2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:44.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.670+0000 7f879affd700 1 -- 172.21.15.31:0/4206482976 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f879c159920 msgr2=0x7f879c0c8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:44.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.670+0000 7f879affd700 1 --2- 172.21.15.31:0/4206482976 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f879c159920 0x7f879c0c8980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:44.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.670+0000 7f879affd700 1 -- 172.21.15.31:0/4206482976 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f878c009420 con 0x7f879c159550 2023-10-06T02:23:44.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.670+0000 7f879b7fe700 1 --2- 172.21.15.31:0/4206482976 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f879c15dfd0 0x7f879c0b2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:23:44.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.670+0000 7f879affd700 1 --2- 172.21.15.31:0/4206482976 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f879c159550 0x7f879c151de0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f878c00b3c0 tx=0x7f878c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:44.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.670+0000 7f8782ffd700 1 -- 172.21.15.31:0/4206482976 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f878c00bbe0 con 0x7f879c159550 2023-10-06T02:23:44.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.670+0000 7f8782ffd700 1 -- 172.21.15.31:0/4206482976 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f878c011420 con 0x7f879c159550 2023-10-06T02:23:44.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.670+0000 7f8782ffd700 1 -- 172.21.15.31:0/4206482976 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f878c0117c0 con 0x7f879c159550 2023-10-06T02:23:44.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.670+0000 7f879bfff700 1 -- 172.21.15.31:0/4206482976 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f879c0b14a0 con 0x7f879c159550 2023-10-06T02:23:44.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.671+0000 7f879bfff700 1 -- 172.21.15.31:0/4206482976 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f879c0b1a60 con 0x7f879c159550 2023-10-06T02:23:44.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.672+0000 7f879bfff700 1 -- 172.21.15.31:0/4206482976 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f879c0b20b0 con 0x7f879c159550 2023-10-06T02:23:44.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.673+0000 7f8782ffd700 1 -- 172.21.15.31:0/4206482976 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f878c011bb0 con 0x7f879c159550 2023-10-06T02:23:44.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.674+0000 7f8782ffd700 1 --2- 172.21.15.31:0/4206482976 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f8788066f30 0x7f87880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:44.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.674+0000 7f8782ffd700 1 -- 172.21.15.31:0/4206482976 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f878c085c30 con 0x7f879c159550 2023-10-06T02:23:44.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.674+0000 7f879a7fc700 1 --2- 172.21.15.31:0/4206482976 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f8788066f30 0x7f87880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:44.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.675+0000 7f879a7fc700 1 --2- 172.21.15.31:0/4206482976 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f8788066f30 0x7f87880693e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f879c06d360 tx=0x7f8790008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:44.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.678+0000 7f8782ffd700 1 -- 172.21.15.31:0/4206482976 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f878c052d20 con 0x7f879c159550 2023-10-06T02:23:44.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.880+0000 7f879bfff700 1 -- 172.21.15.31:0/4206482976 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f879c04c3c0 con 0x7f879c159550 2023-10-06T02:23:44.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.880+0000 7f8782ffd700 1 -- 172.21.15.31:0/4206482976 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f878c015020 con 0x7f879c159550 2023-10-06T02:23:44.880 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-10-06T02:23:44.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.882+0000 7f879bfff700 1 -- 172.21.15.31:0/4206482976 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f8788066f30 msgr2=0x7f87880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:44.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.882+0000 7f879bfff700 1 --2- 172.21.15.31:0/4206482976 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f8788066f30 0x7f87880693e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f879c06d360 tx=0x7f8790008040 comp rx=0 tx=0).stop 2023-10-06T02:23:44.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.882+0000 7f879bfff700 1 -- 172.21.15.31:0/4206482976 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f879c159550 msgr2=0x7f879c151de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:44.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.882+0000 7f879bfff700 1 --2- 172.21.15.31:0/4206482976 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f879c159550 0x7f879c151de0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f878c00b3c0 tx=0x7f878c017000 comp rx=0 tx=0).stop 2023-10-06T02:23:44.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.882+0000 7f879bfff700 1 -- 172.21.15.31:0/4206482976 shutdown_connections 2023-10-06T02:23:44.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.882+0000 7f879bfff700 1 --2- 172.21.15.31:0/4206482976 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f8788066f30 0x7f87880693e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:44.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.882+0000 7f879bfff700 1 --2- 172.21.15.31:0/4206482976 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f879c15dfd0 0x7f879c0b2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:44.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.882+0000 7f879bfff700 1 --2- 172.21.15.31:0/4206482976 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f879c159550 0x7f879c151de0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:44.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.882+0000 7f879bfff700 1 --2- 172.21.15.31:0/4206482976 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f879c159920 0x7f879c0c8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:44.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.882+0000 7f879bfff700 1 -- 172.21.15.31:0/4206482976 >> 172.21.15.31:0/4206482976 conn(0x7f879c06ce80 msgr2=0x7f879c06d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:44.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.882+0000 7f879bfff700 1 -- 172.21.15.31:0/4206482976 shutdown_connections 2023-10-06T02:23:44.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:44.883+0000 7f879bfff700 1 -- 172.21.15.31:0/4206482976 wait complete. 2023-10-06T02:23:44.900 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T02:23:45.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.091+0000 7f571eb9d700 1 Processor -- start 2023-10-06T02:23:45.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.091+0000 7f571eb9d700 1 -- start start 2023-10-06T02:23:45.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.093+0000 7f571eb9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5718140a70 0x7f5718069b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:45.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.093+0000 7f571eb9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f57180c7c60 0x7f571806a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:45.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.093+0000 7f571eb9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f571815b640 con 0x7f571806de50 2023-10-06T02:23:45.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.093+0000 7f571eb9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57180c6800 con 0x7f5718140a70 2023-10-06T02:23:45.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.093+0000 7f571eb9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f571815b920 con 0x7f57180c7c60 2023-10-06T02:23:45.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.093+0000 7f571db9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5718140a70 0x7f5718069b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:45.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.093+0000 7f571d39a700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f571806de50 0x7f5718157a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:51346/0 (socket says 172.21.15.31:51346) 2023-10-06T02:23:45.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.093+0000 7f571d39a700 1 -- 172.21.15.31:0/3636092088 learned_addr learned my addr 172.21.15.31:0/3636092088 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:23:45.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.093+0000 7f571e39c700 1 --2- 172.21.15.31:0/3636092088 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f57180c7c60 0x7f571806a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:45.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.094+0000 7f571db9b700 1 -- 172.21.15.31:0/3636092088 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f57180c7c60 msgr2=0x7f571806a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:45.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.094+0000 7f571db9b700 1 --2- 172.21.15.31:0/3636092088 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f57180c7c60 0x7f571806a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:45.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.094+0000 7f571db9b700 1 -- 172.21.15.31:0/3636092088 >> v1:172.21.15.31:6789/0 conn(0x7f571806de50 legacy=0x7f5718157a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:45.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.094+0000 7f571cb99700 1 -- 172.21.15.31:0/3636092088 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3988726475 0 0) 0x7f571815b640 con 0x7f571806de50 2023-10-06T02:23:45.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.094+0000 7f571db9b700 1 -- 172.21.15.31:0/3636092088 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f571815bc00 con 0x7f5718140a70 2023-10-06T02:23:45.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.094+0000 7f571e39c700 1 --2- 172.21.15.31:0/3636092088 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f57180c7c60 0x7f571806a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:23:45.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.094+0000 7f571db9b700 1 --2- 172.21.15.31:0/3636092088 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5718140a70 0x7f5718069b00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f57140098e0 tx=0x7f57140091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92ef35ace19f6d45 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:45.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.094+0000 7f571cb99700 1 -- 172.21.15.31:0/3636092088 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5714009e80 con 0x7f5718140a70 2023-10-06T02:23:45.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.094+0000 7f571cb99700 1 -- 172.21.15.31:0/3636092088 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f571400b860 con 0x7f5718140a70 2023-10-06T02:23:45.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.094+0000 7f571cb99700 1 -- 172.21.15.31:0/3636092088 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f571400bbe0 con 0x7f5718140a70 2023-10-06T02:23:45.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.095+0000 7f571eb9d700 1 -- 172.21.15.31:0/3636092088 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5718140a70 msgr2=0x7f5718069b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:45.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.095+0000 7f571eb9d700 1 --2- 172.21.15.31:0/3636092088 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5718140a70 0x7f5718069b00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f57140098e0 tx=0x7f57140091a0 comp rx=0 tx=0).stop 2023-10-06T02:23:45.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.095+0000 7f571eb9d700 1 -- 172.21.15.31:0/3636092088 shutdown_connections 2023-10-06T02:23:45.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.095+0000 7f571eb9d700 1 --2- 172.21.15.31:0/3636092088 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5718140a70 0x7f5718069b00 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:45.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.095+0000 7f571eb9d700 1 --2- 172.21.15.31:0/3636092088 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f57180c7c60 0x7f571806a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:45.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.095+0000 7f571eb9d700 1 -- 172.21.15.31:0/3636092088 >> 172.21.15.31:0/3636092088 conn(0x7f571806d110 msgr2=0x7f57180cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:45.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.095+0000 7f571eb9d700 1 -- 172.21.15.31:0/3636092088 shutdown_connections 2023-10-06T02:23:45.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.095+0000 7f571eb9d700 1 -- 172.21.15.31:0/3636092088 wait complete. 2023-10-06T02:23:45.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.096+0000 7f571eb9d700 1 Processor -- start 2023-10-06T02:23:45.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.096+0000 7f571eb9d700 1 -- start start 2023-10-06T02:23:45.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.097+0000 7f571eb9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f571806de50 0x7f571815c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:45.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.097+0000 7f571eb9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f57180c7c60 0x7f571813cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:45.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.097+0000 7f571eb9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5718140a70 0x7f57180b0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:45.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.097+0000 7f571eb9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f571815d3f0 con 0x7f571806de50 2023-10-06T02:23:45.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.097+0000 7f571eb9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57180b10f0 con 0x7f5718140a70 2023-10-06T02:23:45.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.097+0000 7f571eb9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f571815c7e0 con 0x7f57180c7c60 2023-10-06T02:23:45.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.097+0000 7f571db9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f571806de50 0x7f571815c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:45.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.098+0000 7f571db9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f571806de50 0x7f571815c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45718/0 (socket says 172.21.15.31:45718) 2023-10-06T02:23:45.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.098+0000 7f571db9b700 1 -- 172.21.15.31:0/2422357026 learned_addr learned my addr 172.21.15.31:0/2422357026 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:45.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.098+0000 7f571d39a700 1 --2- 172.21.15.31:0/2422357026 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f57180c7c60 0x7f571813cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:45.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.098+0000 7f571e39c700 1 --2- 172.21.15.31:0/2422357026 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5718140a70 0x7f57180b0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:45.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.098+0000 7f571db9b700 1 -- 172.21.15.31:0/2422357026 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f57180c7c60 msgr2=0x7f571813cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:45.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.098+0000 7f571db9b700 1 --2- 172.21.15.31:0/2422357026 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f57180c7c60 0x7f571813cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:45.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.098+0000 7f571db9b700 1 -- 172.21.15.31:0/2422357026 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5718140a70 msgr2=0x7f57180b0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:45.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.098+0000 7f571db9b700 1 --2- 172.21.15.31:0/2422357026 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5718140a70 0x7f57180b0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:45.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.098+0000 7f571db9b700 1 -- 172.21.15.31:0/2422357026 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5714009420 con 0x7f571806de50 2023-10-06T02:23:45.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.098+0000 7f571db9b700 1 --2- 172.21.15.31:0/2422357026 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f571806de50 0x7f571815c200 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f571806e300 tx=0x7f5714017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:45.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.098+0000 7f570dffb700 1 -- 172.21.15.31:0/2422357026 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f571400bbe0 con 0x7f571806de50 2023-10-06T02:23:45.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.099+0000 7f570dffb700 1 -- 172.21.15.31:0/2422357026 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5714011420 con 0x7f571806de50 2023-10-06T02:23:45.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.099+0000 7f571eb9d700 1 -- 172.21.15.31:0/2422357026 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f571815cac0 con 0x7f571806de50 2023-10-06T02:23:45.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.099+0000 7f570dffb700 1 -- 172.21.15.31:0/2422357026 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57140117e0 con 0x7f571806de50 2023-10-06T02:23:45.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.099+0000 7f571eb9d700 1 -- 172.21.15.31:0/2422357026 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f571815d080 con 0x7f571806de50 2023-10-06T02:23:45.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.100+0000 7f571eb9d700 1 -- 172.21.15.31:0/2422357026 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f571804c3c0 con 0x7f571806de50 2023-10-06T02:23:45.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.101+0000 7f570dffb700 1 -- 172.21.15.31:0/2422357026 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5714011bd0 con 0x7f571806de50 2023-10-06T02:23:45.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.101+0000 7f570dffb700 1 --2- 172.21.15.31:0/2422357026 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f56fc066f30 0x7f56fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:45.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.101+0000 7f570dffb700 1 -- 172.21.15.31:0/2422357026 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f5714086160 con 0x7f571806de50 2023-10-06T02:23:45.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.102+0000 7f571d39a700 1 --2- 172.21.15.31:0/2422357026 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f56fc066f30 0x7f56fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:45.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.102+0000 7f571d39a700 1 --2- 172.21.15.31:0/2422357026 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f56fc066f30 0x7f56fc0693e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f57181588f0 tx=0x7f5708009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:45.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.106+0000 7f570dffb700 1 -- 172.21.15.31:0/2422357026 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5714053270 con 0x7f571806de50 2023-10-06T02:23:45.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.307+0000 7f571eb9d700 1 -- 172.21.15.31:0/2422357026 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f571815bee0 con 0x7f571806de50 2023-10-06T02:23:45.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.307+0000 7f570dffb700 1 -- 172.21.15.31:0/2422357026 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5714056890 con 0x7f571806de50 2023-10-06T02:23:45.308 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 8 2023-10-06T02:23:45.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.309+0000 7f571eb9d700 1 -- 172.21.15.31:0/2422357026 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f56fc066f30 msgr2=0x7f56fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:45.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.309+0000 7f571eb9d700 1 --2- 172.21.15.31:0/2422357026 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f56fc066f30 0x7f56fc0693e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f57181588f0 tx=0x7f5708009040 comp rx=0 tx=0).stop 2023-10-06T02:23:45.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.309+0000 7f571eb9d700 1 -- 172.21.15.31:0/2422357026 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f571806de50 msgr2=0x7f571815c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:45.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.309+0000 7f571eb9d700 1 --2- 172.21.15.31:0/2422357026 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f571806de50 0x7f571815c200 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f571806e300 tx=0x7f5714017000 comp rx=0 tx=0).stop 2023-10-06T02:23:45.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.310+0000 7f571eb9d700 1 -- 172.21.15.31:0/2422357026 shutdown_connections 2023-10-06T02:23:45.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.310+0000 7f571eb9d700 1 --2- 172.21.15.31:0/2422357026 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5718140a70 0x7f57180b0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:45.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.310+0000 7f571eb9d700 1 --2- 172.21.15.31:0/2422357026 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f571806de50 0x7f571815c200 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:45.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.310+0000 7f571eb9d700 1 --2- 172.21.15.31:0/2422357026 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f56fc066f30 0x7f56fc0693e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:45.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.310+0000 7f571eb9d700 1 --2- 172.21.15.31:0/2422357026 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f57180c7c60 0x7f571813cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:45.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.310+0000 7f571eb9d700 1 -- 172.21.15.31:0/2422357026 >> 172.21.15.31:0/2422357026 conn(0x7f571806d110 msgr2=0x7f57180c2700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:45.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.310+0000 7f571eb9d700 1 -- 172.21.15.31:0/2422357026 shutdown_connections 2023-10-06T02:23:45.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.310+0000 7f571eb9d700 1 -- 172.21.15.31:0/2422357026 wait complete. 2023-10-06T02:23:45.327 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T02:23:45.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.518+0000 7fe8bcd9d700 1 Processor -- start 2023-10-06T02:23:45.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.518+0000 7fe8bcd9d700 1 -- start start 2023-10-06T02:23:45.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.519+0000 7fe8bcd9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe8b8153c90 0x7fe8b815a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:45.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.519+0000 7fe8bcd9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe8b80caec0 0x7fe8b815afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:45.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.519+0000 7fe8bcd9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8b815ada0 con 0x7fe8b806db20 2023-10-06T02:23:45.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.519+0000 7fe8bcd9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8b806ca80 con 0x7fe8b80caec0 2023-10-06T02:23:45.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.519+0000 7fe8bcd9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8b806cd60 con 0x7fe8b8153c90 2023-10-06T02:23:45.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.520+0000 7fe8b7fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe8b80caec0 0x7fe8b815afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:45.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.520+0000 7fe8b7fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe8b80caec0 0x7fe8b815afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45720/0 (socket says 172.21.15.31:45720) 2023-10-06T02:23:45.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.520+0000 7fe8b7fff700 1 -- 172.21.15.31:0/3306750333 learned_addr learned my addr 172.21.15.31:0/3306750333 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:45.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.520+0000 7fe8b77fe700 1 --2- 172.21.15.31:0/3306750333 >> v2:172.21.15.62:3300/0 conn(0x7fe8b8153c90 0x7fe8b815a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:45.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.520+0000 7fe8b7fff700 1 -- 172.21.15.31:0/3306750333 >> v2:172.21.15.62:3300/0 conn(0x7fe8b8153c90 msgr2=0x7fe8b815a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:45.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.520+0000 7fe8b7fff700 1 --2- 172.21.15.31:0/3306750333 >> v2:172.21.15.62:3300/0 conn(0x7fe8b8153c90 0x7fe8b815a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:45.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.520+0000 7fe8b7fff700 1 -- 172.21.15.31:0/3306750333 >> v1:172.21.15.31:6789/0 conn(0x7fe8b806db20 legacy=0x7fe8b80ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:45.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.520+0000 7fe8b67fc700 1 -- 172.21.15.31:0/3306750333 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3956025243 0 0) 0x7fe8ac001110 con 0x7fe8b806db20 2023-10-06T02:23:45.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.520+0000 7fe8b7fff700 1 -- 172.21.15.31:0/3306750333 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8b806d040 con 0x7fe8b80caec0 2023-10-06T02:23:45.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.520+0000 7fe8b7fff700 1 --2- 172.21.15.31:0/3306750333 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8b80caec0 0x7fe8b815afb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe8b80c37f0 tx=0x7fe8b0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3c8a5aa44c93a74 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:45.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.521+0000 7fe8b67fc700 1 -- 172.21.15.31:0/3306750333 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b000e070 con 0x7fe8b80caec0 2023-10-06T02:23:45.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.521+0000 7fe8b67fc700 1 -- 172.21.15.31:0/3306750333 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8b000ba50 con 0x7fe8b80caec0 2023-10-06T02:23:45.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.521+0000 7fe8b67fc700 1 -- 172.21.15.31:0/3306750333 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b00092e0 con 0x7fe8b80caec0 2023-10-06T02:23:45.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.521+0000 7fe8bcd9d700 1 -- 172.21.15.31:0/3306750333 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8b80caec0 msgr2=0x7fe8b815afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:45.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.521+0000 7fe8bcd9d700 1 --2- 172.21.15.31:0/3306750333 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8b80caec0 0x7fe8b815afb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe8b80c37f0 tx=0x7fe8b0009840 comp rx=0 tx=0).stop 2023-10-06T02:23:45.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.521+0000 7fe8bcd9d700 1 -- 172.21.15.31:0/3306750333 shutdown_connections 2023-10-06T02:23:45.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.521+0000 7fe8bcd9d700 1 --2- 172.21.15.31:0/3306750333 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8b80caec0 0x7fe8b815afb0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:45.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.521+0000 7fe8bcd9d700 1 --2- 172.21.15.31:0/3306750333 >> v2:172.21.15.62:3300/0 conn(0x7fe8b8153c90 0x7fe8b815a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:45.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.521+0000 7fe8bcd9d700 1 -- 172.21.15.31:0/3306750333 >> 172.21.15.31:0/3306750333 conn(0x7fe8b806e720 msgr2=0x7fe8b806eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:45.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.521+0000 7fe8bcd9d700 1 -- 172.21.15.31:0/3306750333 shutdown_connections 2023-10-06T02:23:45.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.521+0000 7fe8bcd9d700 1 -- 172.21.15.31:0/3306750333 wait complete. 2023-10-06T02:23:45.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.523+0000 7fe8bcd9d700 1 Processor -- start 2023-10-06T02:23:45.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.523+0000 7fe8bcd9d700 1 -- start start 2023-10-06T02:23:45.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.524+0000 7fe8bcd9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8b806db20 0x7fe8b816a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:45.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.524+0000 7fe8bcd9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe8b80caec0 0x7fe8b816b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:45.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.524+0000 7fe8bcd9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8b8153c90 0x7fe8b814f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:45.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.524+0000 7fe8bcd9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b816b160 con 0x7fe8b8153c90 2023-10-06T02:23:45.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.524+0000 7fe8bcd9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b814edc0 con 0x7fe8b806db20 2023-10-06T02:23:45.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.524+0000 7fe8bcd9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b816ae90 con 0x7fe8b80caec0 2023-10-06T02:23:45.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.524+0000 7fe8b7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8b8153c90 0x7fe8b814f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:45.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.524+0000 7fe8b7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8b8153c90 0x7fe8b814f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45724/0 (socket says 172.21.15.31:45724) 2023-10-06T02:23:45.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.524+0000 7fe8b7fff700 1 -- 172.21.15.31:0/1493173865 learned_addr learned my addr 172.21.15.31:0/1493173865 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:45.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.524+0000 7fe8b77fe700 1 --2- 172.21.15.31:0/1493173865 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8b806db20 0x7fe8b816a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:45.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.524+0000 7fe8b6ffd700 1 --2- 172.21.15.31:0/1493173865 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe8b80caec0 0x7fe8b816b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:45.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.524+0000 7fe8b7fff700 1 -- 172.21.15.31:0/1493173865 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe8b80caec0 msgr2=0x7fe8b816b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:45.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.524+0000 7fe8b7fff700 1 --2- 172.21.15.31:0/1493173865 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe8b80caec0 0x7fe8b816b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:45.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.524+0000 7fe8b7fff700 1 -- 172.21.15.31:0/1493173865 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8b806db20 msgr2=0x7fe8b816a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:45.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.524+0000 7fe8b7fff700 1 --2- 172.21.15.31:0/1493173865 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8b806db20 0x7fe8b816a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:45.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.524+0000 7fe8b7fff700 1 -- 172.21.15.31:0/1493173865 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8b0009ae0 con 0x7fe8b8153c90 2023-10-06T02:23:45.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.524+0000 7fe8b77fe700 1 --2- 172.21.15.31:0/1493173865 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8b806db20 0x7fe8b816a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:23:45.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.525+0000 7fe8b7fff700 1 --2- 172.21.15.31:0/1493173865 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8b8153c90 0x7fe8b814f250 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe8b80c3820 tx=0x7fe8b0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:45.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.525+0000 7fe89f7fe700 1 -- 172.21.15.31:0/1493173865 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b0008e80 con 0x7fe8b8153c90 2023-10-06T02:23:45.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.525+0000 7fe89f7fe700 1 -- 172.21.15.31:0/1493173865 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8b0026460 con 0x7fe8b8153c90 2023-10-06T02:23:45.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.525+0000 7fe89f7fe700 1 -- 172.21.15.31:0/1493173865 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b0026800 con 0x7fe8b8153c90 2023-10-06T02:23:45.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.525+0000 7fe8bcd9d700 1 -- 172.21.15.31:0/1493173865 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8b814e490 con 0x7fe8b8153c90 2023-10-06T02:23:45.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.525+0000 7fe8bcd9d700 1 -- 172.21.15.31:0/1493173865 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8b814ea50 con 0x7fe8b8153c90 2023-10-06T02:23:45.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.526+0000 7fe8bcd9d700 1 -- 172.21.15.31:0/1493173865 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8b814ef50 con 0x7fe8b8153c90 2023-10-06T02:23:45.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.527+0000 7fe89f7fe700 1 -- 172.21.15.31:0/1493173865 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe8b0003040 con 0x7fe8b8153c90 2023-10-06T02:23:45.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.528+0000 7fe89f7fe700 1 --2- 172.21.15.31:0/1493173865 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe8a4066f30 0x7fe8a40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:45.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.528+0000 7fe89f7fe700 1 -- 172.21.15.31:0/1493173865 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe8b0086830 con 0x7fe8b8153c90 2023-10-06T02:23:45.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.528+0000 7fe8b77fe700 1 --2- 172.21.15.31:0/1493173865 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe8a4066f30 0x7fe8a40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:45.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.529+0000 7fe8b77fe700 1 --2- 172.21.15.31:0/1493173865 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe8a4066f30 0x7fe8a40693e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8006fd0 tx=0x7fe8a8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:45.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.532+0000 7fe89f7fe700 1 -- 172.21.15.31:0/1493173865 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe8b00539f0 con 0x7fe8b8153c90 2023-10-06T02:23:45.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.733+0000 7fe8bcd9d700 1 -- 172.21.15.31:0/1493173865 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe8b804c3c0 con 0x7fe8b8153c90 2023-10-06T02:23:45.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.733+0000 7fe89f7fe700 1 -- 172.21.15.31:0/1493173865 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe8b0057010 con 0x7fe8b8153c90 2023-10-06T02:23:45.734 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-10-06T02:23:45.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.736+0000 7fe8bcd9d700 1 -- 172.21.15.31:0/1493173865 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe8a4066f30 msgr2=0x7fe8a40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:45.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.736+0000 7fe8bcd9d700 1 --2- 172.21.15.31:0/1493173865 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe8a4066f30 0x7fe8a40693e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8006fd0 tx=0x7fe8a8008040 comp rx=0 tx=0).stop 2023-10-06T02:23:45.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.736+0000 7fe8bcd9d700 1 -- 172.21.15.31:0/1493173865 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8b8153c90 msgr2=0x7fe8b814f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:45.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.736+0000 7fe8bcd9d700 1 --2- 172.21.15.31:0/1493173865 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8b8153c90 0x7fe8b814f250 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe8b80c3820 tx=0x7fe8b0018000 comp rx=0 tx=0).stop 2023-10-06T02:23:45.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.736+0000 7fe8bcd9d700 1 -- 172.21.15.31:0/1493173865 shutdown_connections 2023-10-06T02:23:45.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.736+0000 7fe8bcd9d700 1 --2- 172.21.15.31:0/1493173865 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8b8153c90 0x7fe8b814f250 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:45.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.736+0000 7fe8bcd9d700 1 --2- 172.21.15.31:0/1493173865 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8b806db20 0x7fe8b816a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:45.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.736+0000 7fe8bcd9d700 1 --2- 172.21.15.31:0/1493173865 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe8a4066f30 0x7fe8a40693e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:45.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.736+0000 7fe8bcd9d700 1 --2- 172.21.15.31:0/1493173865 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe8b80caec0 0x7fe8b816b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:45.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.736+0000 7fe8bcd9d700 1 -- 172.21.15.31:0/1493173865 >> 172.21.15.31:0/1493173865 conn(0x7fe8b806e720 msgr2=0x7fe8b80c2810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:45.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.736+0000 7fe8bcd9d700 1 -- 172.21.15.31:0/1493173865 shutdown_connections 2023-10-06T02:23:45.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.736+0000 7fe8bcd9d700 1 -- 172.21.15.31:0/1493173865 wait complete. 2023-10-06T02:23:45.753 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T02:23:45.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.941+0000 7f701759e700 1 Processor -- start 2023-10-06T02:23:45.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.941+0000 7f701759e700 1 -- start start 2023-10-06T02:23:45.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.943+0000 7f701759e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7018142c00 0x7f701806b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:45.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.943+0000 7f701759e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f701806bd20 0x7f70180c7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:45.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.943+0000 7f701759e700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70180ca480 con 0x7f7018142fd0 2023-10-06T02:23:45.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.943+0000 7f701759e700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f701815e240 con 0x7f7018142c00 2023-10-06T02:23:45.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.943+0000 7f701759e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f701806b350 con 0x7f701806bd20 2023-10-06T02:23:45.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.943+0000 7f701659c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7018142c00 0x7f701806b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:45.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.943+0000 7f701659c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7018142c00 0x7f701806b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45740/0 (socket says 172.21.15.31:45740) 2023-10-06T02:23:45.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.943+0000 7f701659c700 1 -- 172.21.15.31:0/2740122661 learned_addr learned my addr 172.21.15.31:0/2740122661 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:45.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.943+0000 7f7015d9b700 1 --2- 172.21.15.31:0/2740122661 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f701806bd20 0x7f70180c7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:45.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.943+0000 7f701659c700 1 -- 172.21.15.31:0/2740122661 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f701806bd20 msgr2=0x7f70180c7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:45.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.943+0000 7f701659c700 1 --2- 172.21.15.31:0/2740122661 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f701806bd20 0x7f70180c7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:45.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.943+0000 7f701659c700 1 -- 172.21.15.31:0/2740122661 >> v1:172.21.15.31:6789/0 conn(0x7f7018142fd0 legacy=0x7f701806c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:45.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.944+0000 7f701659c700 1 -- 172.21.15.31:0/2740122661 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f701815df60 con 0x7f7018142c00 2023-10-06T02:23:45.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.944+0000 7f701559a700 1 -- 172.21.15.31:0/2740122661 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4228495615 0 0) 0x7f70180ca480 con 0x7f7018142fd0 2023-10-06T02:23:45.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.944+0000 7f701659c700 1 --2- 172.21.15.31:0/2740122661 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7018142c00 0x7f701806b7e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f70080099b0 tx=0x7f7008009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdfd036263e1320b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:45.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.944+0000 7f701559a700 1 -- 172.21.15.31:0/2740122661 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7008009ea0 con 0x7f7018142c00 2023-10-06T02:23:45.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.944+0000 7f701559a700 1 -- 172.21.15.31:0/2740122661 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f700800b860 con 0x7f7018142c00 2023-10-06T02:23:45.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.944+0000 7f701559a700 1 -- 172.21.15.31:0/2740122661 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f700800bc20 con 0x7f7018142c00 2023-10-06T02:23:45.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.945+0000 7f701759e700 1 -- 172.21.15.31:0/2740122661 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7018142c00 msgr2=0x7f701806b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:45.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.945+0000 7f701759e700 1 --2- 172.21.15.31:0/2740122661 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7018142c00 0x7f701806b7e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f70080099b0 tx=0x7f7008009250 comp rx=0 tx=0).stop 2023-10-06T02:23:45.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.945+0000 7f701759e700 1 -- 172.21.15.31:0/2740122661 shutdown_connections 2023-10-06T02:23:45.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.945+0000 7f701759e700 1 --2- 172.21.15.31:0/2740122661 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7018142c00 0x7f701806b7e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:45.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.945+0000 7f701759e700 1 --2- 172.21.15.31:0/2740122661 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f701806bd20 0x7f70180c7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:45.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.945+0000 7f701759e700 1 -- 172.21.15.31:0/2740122661 >> 172.21.15.31:0/2740122661 conn(0x7f70180675a0 msgr2=0x7f70180679a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:45.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.945+0000 7f701759e700 1 -- 172.21.15.31:0/2740122661 shutdown_connections 2023-10-06T02:23:45.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.945+0000 7f701759e700 1 -- 172.21.15.31:0/2740122661 wait complete. 2023-10-06T02:23:45.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.946+0000 7f701759e700 1 Processor -- start 2023-10-06T02:23:45.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.946+0000 7f701759e700 1 -- start start 2023-10-06T02:23:45.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.947+0000 7f701759e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f701806bd20 0x7f7018139350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:45.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.947+0000 7f701759e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7018142c00 0x7f70180c42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:45.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.947+0000 7f701759e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7018142fd0 0x7f70180c60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:45.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.947+0000 7f701759e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70180c4c60 con 0x7f7018142c00 2023-10-06T02:23:45.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.947+0000 7f701759e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f701813eac0 con 0x7f7018142fd0 2023-10-06T02:23:45.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.947+0000 7f701759e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f701813eda0 con 0x7f701806bd20 2023-10-06T02:23:45.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.948+0000 7f7015d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7018142c00 0x7f70180c42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:45.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.948+0000 7f7015d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7018142c00 0x7f70180c42f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45746/0 (socket says 172.21.15.31:45746) 2023-10-06T02:23:45.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.948+0000 7f7015d9b700 1 -- 172.21.15.31:0/207007805 learned_addr learned my addr 172.21.15.31:0/207007805 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:45.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.948+0000 7f701659c700 1 --2- 172.21.15.31:0/207007805 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f701806bd20 0x7f7018139350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:45.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.948+0000 7f7016d9d700 1 --2- 172.21.15.31:0/207007805 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7018142fd0 0x7f70180c60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:45.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.948+0000 7f7015d9b700 1 -- 172.21.15.31:0/207007805 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f701806bd20 msgr2=0x7f7018139350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:45.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.948+0000 7f7015d9b700 1 --2- 172.21.15.31:0/207007805 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f701806bd20 0x7f7018139350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:45.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.948+0000 7f7015d9b700 1 -- 172.21.15.31:0/207007805 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7018142fd0 msgr2=0x7f70180c60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:45.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.948+0000 7f7015d9b700 1 --2- 172.21.15.31:0/207007805 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7018142fd0 0x7f70180c60f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:45.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.948+0000 7f7015d9b700 1 -- 172.21.15.31:0/207007805 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70080094f0 con 0x7f7018142c00 2023-10-06T02:23:45.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.948+0000 7f7015d9b700 1 --2- 172.21.15.31:0/207007805 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7018142c00 0x7f70180c42f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f700c00ec80 tx=0x7f700c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:45.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.948+0000 7f70067fc700 1 -- 172.21.15.31:0/207007805 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f700c00c850 con 0x7f7018142c00 2023-10-06T02:23:45.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.949+0000 7f70067fc700 1 -- 172.21.15.31:0/207007805 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f700c00c9b0 con 0x7f7018142c00 2023-10-06T02:23:45.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.949+0000 7f701759e700 1 -- 172.21.15.31:0/207007805 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f701813f080 con 0x7f7018142c00 2023-10-06T02:23:45.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.949+0000 7f70067fc700 1 -- 172.21.15.31:0/207007805 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f700c00cd50 con 0x7f7018142c00 2023-10-06T02:23:45.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.949+0000 7f701759e700 1 -- 172.21.15.31:0/207007805 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70180c4830 con 0x7f7018142c00 2023-10-06T02:23:45.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.950+0000 7f701759e700 1 -- 172.21.15.31:0/207007805 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70180c4a20 con 0x7f7018142c00 2023-10-06T02:23:45.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.951+0000 7f70067fc700 1 -- 172.21.15.31:0/207007805 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f700c013680 con 0x7f7018142c00 2023-10-06T02:23:45.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.952+0000 7f70067fc700 1 --2- 172.21.15.31:0/207007805 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6ff8066f30 0x7f6ff80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:45.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.952+0000 7f70067fc700 1 -- 172.21.15.31:0/207007805 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f700c084950 con 0x7f7018142c00 2023-10-06T02:23:45.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.952+0000 7f701659c700 1 --2- 172.21.15.31:0/207007805 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6ff8066f30 0x7f6ff80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:45.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.953+0000 7f701659c700 1 --2- 172.21.15.31:0/207007805 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6ff8066f30 0x7f6ff80693e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f70080099b0 tx=0x7f7008017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:45.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:45.956+0000 7f70067fc700 1 -- 172.21.15.31:0/207007805 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f70180c4a20 con 0x7f7018142c00 2023-10-06T02:23:46.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.156+0000 7f701759e700 1 -- 172.21.15.31:0/207007805 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f70180c4a20 con 0x7f7018142c00 2023-10-06T02:23:46.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.156+0000 7f70067fc700 1 -- 172.21.15.31:0/207007805 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f70180c4a20 con 0x7f7018142c00 2023-10-06T02:23:46.157 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-10-06T02:23:46.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.158+0000 7f701759e700 1 -- 172.21.15.31:0/207007805 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6ff8066f30 msgr2=0x7f6ff80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:46.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.158+0000 7f701759e700 1 --2- 172.21.15.31:0/207007805 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6ff8066f30 0x7f6ff80693e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f70080099b0 tx=0x7f7008017000 comp rx=0 tx=0).stop 2023-10-06T02:23:46.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.158+0000 7f701759e700 1 -- 172.21.15.31:0/207007805 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7018142c00 msgr2=0x7f70180c42f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:46.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.158+0000 7f701759e700 1 --2- 172.21.15.31:0/207007805 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7018142c00 0x7f70180c42f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f700c00ec80 tx=0x7f700c0091a0 comp rx=0 tx=0).stop 2023-10-06T02:23:46.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.158+0000 7f701759e700 1 -- 172.21.15.31:0/207007805 shutdown_connections 2023-10-06T02:23:46.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.158+0000 7f701759e700 1 --2- 172.21.15.31:0/207007805 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7018142fd0 0x7f70180c60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:46.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.158+0000 7f701759e700 1 --2- 172.21.15.31:0/207007805 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6ff8066f30 0x7f6ff80693e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:46.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.158+0000 7f701759e700 1 --2- 172.21.15.31:0/207007805 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f701806bd20 0x7f7018139350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:46.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.158+0000 7f701759e700 1 --2- 172.21.15.31:0/207007805 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7018142c00 0x7f70180c42f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:46.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.158+0000 7f701759e700 1 -- 172.21.15.31:0/207007805 >> 172.21.15.31:0/207007805 conn(0x7f70180675a0 msgr2=0x7f70180c7aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:46.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.159+0000 7f701759e700 1 -- 172.21.15.31:0/207007805 shutdown_connections 2023-10-06T02:23:46.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.159+0000 7f701759e700 1 -- 172.21.15.31:0/207007805 wait complete. 2023-10-06T02:23:46.176 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:23:46.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.362+0000 7f83cf59e700 1 Processor -- start 2023-10-06T02:23:46.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.363+0000 7f83cf59e700 1 -- start start 2023-10-06T02:23:46.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.364+0000 7f83cf59e700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f83d0153c30 0x7f83d015a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:46.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.364+0000 7f83cf59e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f83d015de50 0x7f83d00cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:46.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.364+0000 7f83cf59e700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83d015afd0 con 0x7f83d00cef40 2023-10-06T02:23:46.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.364+0000 7f83cf59e700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83d015b2b0 con 0x7f83d0153c30 2023-10-06T02:23:46.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.364+0000 7f83cf59e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83d015b590 con 0x7f83d015de50 2023-10-06T02:23:46.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.364+0000 7f83cdd9b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f83d015de50 0x7f83d00cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:46.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.364+0000 7f83ce59c700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f83d0153c30 0x7f83d015a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:46.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.364+0000 7f83ced9d700 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f83d00cef40 0x7f83d006d790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:39844/0 (socket says 172.21.15.31:39844) 2023-10-06T02:23:46.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.364+0000 7f83ced9d700 1 -- 172.21.15.31:0/1867827112 learned_addr learned my addr 172.21.15.31:0/1867827112 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:23:46.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.365+0000 7f83cd59a700 1 -- 172.21.15.31:0/1867827112 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 359746171 0 0) 0x7f83d015afd0 con 0x7f83d00cef40 2023-10-06T02:23:46.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.365+0000 7f83cdd9b700 1 -- 172.21.15.31:0/1867827112 >> v2:172.21.15.62:3300/0 conn(0x7f83d0153c30 msgr2=0x7f83d015a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:46.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.365+0000 7f83cdd9b700 1 --2- 172.21.15.31:0/1867827112 >> v2:172.21.15.62:3300/0 conn(0x7f83d0153c30 0x7f83d015a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:46.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.365+0000 7f83cdd9b700 1 -- 172.21.15.31:0/1867827112 >> v1:172.21.15.62:6789/0 conn(0x7f83d00cef40 legacy=0x7f83d006d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:46.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.365+0000 7f83cdd9b700 1 -- 172.21.15.31:0/1867827112 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83d015b870 con 0x7f83d015de50 2023-10-06T02:23:46.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.365+0000 7f83cdd9b700 1 --2- 172.21.15.31:0/1867827112 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f83d015de50 0x7f83d00cea00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f83c40098e0 tx=0x7f83c40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=803f2f3a6770a3db server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:46.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.366+0000 7f83cd59a700 1 -- 172.21.15.31:0/1867827112 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c400b640 con 0x7f83d015de50 2023-10-06T02:23:46.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.366+0000 7f83cd59a700 1 -- 172.21.15.31:0/1867827112 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83c400b7a0 con 0x7f83d015de50 2023-10-06T02:23:46.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.366+0000 7f83cd59a700 1 -- 172.21.15.31:0/1867827112 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c400bb40 con 0x7f83d015de50 2023-10-06T02:23:46.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.366+0000 7f83cf59e700 1 -- 172.21.15.31:0/1867827112 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f83d015de50 msgr2=0x7f83d00cea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:46.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.366+0000 7f83cf59e700 1 --2- 172.21.15.31:0/1867827112 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f83d015de50 0x7f83d00cea00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f83c40098e0 tx=0x7f83c40091a0 comp rx=0 tx=0).stop 2023-10-06T02:23:46.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.366+0000 7f83cf59e700 1 -- 172.21.15.31:0/1867827112 shutdown_connections 2023-10-06T02:23:46.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.366+0000 7f83cf59e700 1 --2- 172.21.15.31:0/1867827112 >> v2:172.21.15.62:3300/0 conn(0x7f83d0153c30 0x7f83d015a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:46.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.366+0000 7f83cf59e700 1 --2- 172.21.15.31:0/1867827112 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f83d015de50 0x7f83d00cea00 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:46.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.366+0000 7f83cf59e700 1 -- 172.21.15.31:0/1867827112 >> 172.21.15.31:0/1867827112 conn(0x7f83d00703a0 msgr2=0x7f83d00707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:46.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.366+0000 7f83cf59e700 1 -- 172.21.15.31:0/1867827112 shutdown_connections 2023-10-06T02:23:46.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.366+0000 7f83cf59e700 1 -- 172.21.15.31:0/1867827112 wait complete. 2023-10-06T02:23:46.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.368+0000 7f83cf59e700 1 Processor -- start 2023-10-06T02:23:46.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.368+0000 7f83cf59e700 1 -- start start 2023-10-06T02:23:46.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.369+0000 7f83cf59e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f83d00cef40 0x7f83d0148e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:46.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.369+0000 7f83cf59e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f83d0153c30 0x7f83d015f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:46.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.369+0000 7f83cf59e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83d015de50 0x7f83d015fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:46.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.369+0000 7f83cf59e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83d01521d0 con 0x7f83d0153c30 2023-10-06T02:23:46.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.369+0000 7f83cf59e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83d0151320 con 0x7f83d015de50 2023-10-06T02:23:46.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.369+0000 7f83cf59e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83d01515c0 con 0x7f83d00cef40 2023-10-06T02:23:46.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.369+0000 7f83cdd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f83d0153c30 0x7f83d015f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:46.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.369+0000 7f83cdd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f83d0153c30 0x7f83d015f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45762/0 (socket says 172.21.15.31:45762) 2023-10-06T02:23:46.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.369+0000 7f83cdd9b700 1 -- 172.21.15.31:0/978650982 learned_addr learned my addr 172.21.15.31:0/978650982 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:46.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.369+0000 7f83ce59c700 1 --2- 172.21.15.31:0/978650982 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f83d00cef40 0x7f83d0148e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:46.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.369+0000 7f83ced9d700 1 --2- 172.21.15.31:0/978650982 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83d015de50 0x7f83d015fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:46.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.369+0000 7f83cdd9b700 1 -- 172.21.15.31:0/978650982 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f83d00cef40 msgr2=0x7f83d0148e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:46.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.369+0000 7f83cdd9b700 1 --2- 172.21.15.31:0/978650982 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f83d00cef40 0x7f83d0148e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:46.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.370+0000 7f83cdd9b700 1 -- 172.21.15.31:0/978650982 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83d015de50 msgr2=0x7f83d015fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:46.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.370+0000 7f83cdd9b700 1 --2- 172.21.15.31:0/978650982 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83d015de50 0x7f83d015fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:46.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.370+0000 7f83cdd9b700 1 -- 172.21.15.31:0/978650982 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83c4009420 con 0x7f83d0153c30 2023-10-06T02:23:46.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.370+0000 7f83ced9d700 1 --2- 172.21.15.31:0/978650982 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83d015de50 0x7f83d015fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:23:46.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.370+0000 7f83cdd9b700 1 --2- 172.21.15.31:0/978650982 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f83d0153c30 0x7f83d015f790 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f83c40093f0 tx=0x7f83c4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:46.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.370+0000 7f83be7fc700 1 -- 172.21.15.31:0/978650982 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c400bb20 con 0x7f83d0153c30 2023-10-06T02:23:46.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.370+0000 7f83be7fc700 1 -- 172.21.15.31:0/978650982 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83c4011420 con 0x7f83d0153c30 2023-10-06T02:23:46.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.370+0000 7f83be7fc700 1 -- 172.21.15.31:0/978650982 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c40117e0 con 0x7f83d0153c30 2023-10-06T02:23:46.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.370+0000 7f83cf59e700 1 -- 172.21.15.31:0/978650982 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83d01518a0 con 0x7f83d0153c30 2023-10-06T02:23:46.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.370+0000 7f83cf59e700 1 -- 172.21.15.31:0/978650982 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83d0151e60 con 0x7f83d0153c30 2023-10-06T02:23:46.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.371+0000 7f83cf59e700 1 -- 172.21.15.31:0/978650982 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83d0152340 con 0x7f83d0153c30 2023-10-06T02:23:46.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.372+0000 7f83be7fc700 1 -- 172.21.15.31:0/978650982 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f83c4011c10 con 0x7f83d0153c30 2023-10-06T02:23:46.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.373+0000 7f83be7fc700 1 --2- 172.21.15.31:0/978650982 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f83b0066f30 0x7f83b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:46.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.373+0000 7f83be7fc700 1 -- 172.21.15.31:0/978650982 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f83c4085fe0 con 0x7f83d0153c30 2023-10-06T02:23:46.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.374+0000 7f83ce59c700 1 --2- 172.21.15.31:0/978650982 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f83b0066f30 0x7f83b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:46.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.374+0000 7f83ce59c700 1 --2- 172.21.15.31:0/978650982 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f83b0066f30 0x7f83b00693e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f83c0009db0 tx=0x7f83c00091a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:46.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.377+0000 7f83be7fc700 1 -- 172.21.15.31:0/978650982 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f83c4053090 con 0x7f83d0153c30 2023-10-06T02:23:46.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.579+0000 7f83cf59e700 1 -- 172.21.15.31:0/978650982 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f83d004c3c0 con 0x7f83d0153c30 2023-10-06T02:23:46.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.580+0000 7f83be7fc700 1 -- 172.21.15.31:0/978650982 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16818 (secure 0 0 0) 0x7f83c40566b0 con 0x7f83d0153c30 2023-10-06T02:23:46.580 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:23:46.581 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":23,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:23:10.970534+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:23:07.290269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T02:23:08.504679+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:23:46.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.582+0000 7f83cf59e700 1 -- 172.21.15.31:0/978650982 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f83b0066f30 msgr2=0x7f83b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:46.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.582+0000 7f83cf59e700 1 --2- 172.21.15.31:0/978650982 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f83b0066f30 0x7f83b00693e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f83c0009db0 tx=0x7f83c00091a0 comp rx=0 tx=0).stop 2023-10-06T02:23:46.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.582+0000 7f83cf59e700 1 -- 172.21.15.31:0/978650982 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f83d0153c30 msgr2=0x7f83d015f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:46.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.582+0000 7f83cf59e700 1 --2- 172.21.15.31:0/978650982 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f83d0153c30 0x7f83d015f790 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f83c40093f0 tx=0x7f83c4017000 comp rx=0 tx=0).stop 2023-10-06T02:23:46.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.583+0000 7f83cf59e700 1 -- 172.21.15.31:0/978650982 shutdown_connections 2023-10-06T02:23:46.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.583+0000 7f83cf59e700 1 --2- 172.21.15.31:0/978650982 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83d015de50 0x7f83d015fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:46.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.583+0000 7f83cf59e700 1 --2- 172.21.15.31:0/978650982 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f83b0066f30 0x7f83b00693e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:46.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.583+0000 7f83cf59e700 1 --2- 172.21.15.31:0/978650982 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f83d00cef40 0x7f83d0148e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:46.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.583+0000 7f83cf59e700 1 --2- 172.21.15.31:0/978650982 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f83d0153c30 0x7f83d015f790 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:46.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.583+0000 7f83cf59e700 1 -- 172.21.15.31:0/978650982 >> 172.21.15.31:0/978650982 conn(0x7f83d00703a0 msgr2=0x7f83d00c6ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:46.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.583+0000 7f83cf59e700 1 -- 172.21.15.31:0/978650982 shutdown_connections 2023-10-06T02:23:46.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.583+0000 7f83cf59e700 1 -- 172.21.15.31:0/978650982 wait complete. 2023-10-06T02:23:46.601 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T02:22:53.183425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T02:23:00.046967+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T02:23:07.290269+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T02:23:08.504679+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T02:23:46.601 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T02:23:46.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.790+0000 7f7afe39d700 1 Processor -- start 2023-10-06T02:23:46.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.790+0000 7f7afe39d700 1 -- start start 2023-10-06T02:23:46.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.791+0000 7f7afe39d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7af8142fd0 0x7f7af80c7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:46.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.791+0000 7f7afe39d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7af80c7dc0 0x7f7af806c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:46.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.791+0000 7f7afe39d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7af80ca480 con 0x7f7af8142c00 2023-10-06T02:23:46.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.791+0000 7f7afe39d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7af80c8240 con 0x7f7af80c7dc0 2023-10-06T02:23:46.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.791+0000 7f7afe39d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7af815e240 con 0x7f7af8142fd0 2023-10-06T02:23:46.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.792+0000 7f7afdb9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7af80c7dc0 0x7f7af806c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:46.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.792+0000 7f7afdb9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7af80c7dc0 0x7f7af806c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45774/0 (socket says 172.21.15.31:45774) 2023-10-06T02:23:46.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.792+0000 7f7afdb9c700 1 -- 172.21.15.31:0/2938447638 learned_addr learned my addr 172.21.15.31:0/2938447638 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:46.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.792+0000 7f7afcb9a700 1 --2- 172.21.15.31:0/2938447638 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7af8142fd0 0x7f7af80c7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:46.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.792+0000 7f7afdb9c700 1 -- 172.21.15.31:0/2938447638 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7af8142fd0 msgr2=0x7f7af80c7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:46.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.792+0000 7f7afdb9c700 1 --2- 172.21.15.31:0/2938447638 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7af8142fd0 0x7f7af80c7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:46.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.792+0000 7f7afdb9c700 1 -- 172.21.15.31:0/2938447638 >> v1:172.21.15.62:6789/0 conn(0x7f7af8142c00 legacy=0x7f7af806b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:46.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.792+0000 7f7afdb9c700 1 -- 172.21.15.31:0/2938447638 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7af806b350 con 0x7f7af80c7dc0 2023-10-06T02:23:46.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.792+0000 7f7aeffff700 1 -- 172.21.15.31:0/2938447638 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1321316537 0 0) 0x7f7af80ca480 con 0x7f7af8142c00 2023-10-06T02:23:46.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.792+0000 7f7afdb9c700 1 --2- 172.21.15.31:0/2938447638 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7af80c7dc0 0x7f7af806c6a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7af0009f20 tx=0x7f7af0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52419545d6595335 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:46.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.793+0000 7f7aeffff700 1 -- 172.21.15.31:0/2938447638 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af000e070 con 0x7f7af80c7dc0 2023-10-06T02:23:46.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.793+0000 7f7aeffff700 1 -- 172.21.15.31:0/2938447638 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7af000b9d0 con 0x7f7af80c7dc0 2023-10-06T02:23:46.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.793+0000 7f7aeffff700 1 -- 172.21.15.31:0/2938447638 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af00092e0 con 0x7f7af80c7dc0 2023-10-06T02:23:46.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.793+0000 7f7afe39d700 1 -- 172.21.15.31:0/2938447638 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7af80c7dc0 msgr2=0x7f7af806c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:46.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.793+0000 7f7afe39d700 1 --2- 172.21.15.31:0/2938447638 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7af80c7dc0 0x7f7af806c6a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7af0009f20 tx=0x7f7af0009840 comp rx=0 tx=0).stop 2023-10-06T02:23:46.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.793+0000 7f7afe39d700 1 -- 172.21.15.31:0/2938447638 shutdown_connections 2023-10-06T02:23:46.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.793+0000 7f7afe39d700 1 --2- 172.21.15.31:0/2938447638 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7af80c7dc0 0x7f7af806c6a0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:46.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.793+0000 7f7afe39d700 1 --2- 172.21.15.31:0/2938447638 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7af8142fd0 0x7f7af80c7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:46.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.793+0000 7f7afe39d700 1 -- 172.21.15.31:0/2938447638 >> 172.21.15.31:0/2938447638 conn(0x7f7af80675a0 msgr2=0x7f7af80679a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:46.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.793+0000 7f7afe39d700 1 -- 172.21.15.31:0/2938447638 shutdown_connections 2023-10-06T02:23:46.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.793+0000 7f7afe39d700 1 -- 172.21.15.31:0/2938447638 wait complete. 2023-10-06T02:23:46.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.795+0000 7f7afe39d700 1 Processor -- start 2023-10-06T02:23:46.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.795+0000 7f7afe39d700 1 -- start start 2023-10-06T02:23:46.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.796+0000 7f7afe39d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7af80c7dc0 0x7f7af8159050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:46.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.796+0000 7f7afe39d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7af8142c00 0x7f7af80b2050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:46.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.796+0000 7f7afe39d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7af8142fd0 0x7f7af8141360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:46.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.796+0000 7f7afe39d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af8159590 con 0x7f7af80c7dc0 2023-10-06T02:23:46.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.796+0000 7f7afe39d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af80b04a0 con 0x7f7af8142fd0 2023-10-06T02:23:46.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.796+0000 7f7afe39d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7af80b0780 con 0x7f7af8142c00 2023-10-06T02:23:46.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.796+0000 7f7afd39b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7af80c7dc0 0x7f7af8159050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:46.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.796+0000 7f7afd39b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7af80c7dc0 0x7f7af8159050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45790/0 (socket says 172.21.15.31:45790) 2023-10-06T02:23:46.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.796+0000 7f7afd39b700 1 -- 172.21.15.31:0/2093768881 learned_addr learned my addr 172.21.15.31:0/2093768881 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:46.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.796+0000 7f7afcb9a700 1 --2- 172.21.15.31:0/2093768881 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7af8142c00 0x7f7af80b2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:46.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.796+0000 7f7afdb9c700 1 --2- 172.21.15.31:0/2093768881 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7af8142fd0 0x7f7af8141360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:46.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.796+0000 7f7afd39b700 1 -- 172.21.15.31:0/2093768881 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7af8142c00 msgr2=0x7f7af80b2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:46.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.796+0000 7f7afd39b700 1 --2- 172.21.15.31:0/2093768881 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7af8142c00 0x7f7af80b2050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:46.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.796+0000 7f7afd39b700 1 -- 172.21.15.31:0/2093768881 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7af8142fd0 msgr2=0x7f7af8141360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:46.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.796+0000 7f7afd39b700 1 --2- 172.21.15.31:0/2093768881 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7af8142fd0 0x7f7af8141360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:46.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.796+0000 7f7afd39b700 1 -- 172.21.15.31:0/2093768881 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7af0009ae0 con 0x7f7af80c7dc0 2023-10-06T02:23:46.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.796+0000 7f7afcb9a700 1 --2- 172.21.15.31:0/2093768881 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7af8142c00 0x7f7af80b2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:23:46.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.797+0000 7f7afd39b700 1 --2- 172.21.15.31:0/2093768881 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7af80c7dc0 0x7f7af8159050 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7af400c7e0 tx=0x7f7af4007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:46.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.797+0000 7f7aed7fa700 1 -- 172.21.15.31:0/2093768881 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af4012700 con 0x7f7af80c7dc0 2023-10-06T02:23:46.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.797+0000 7f7aed7fa700 1 -- 172.21.15.31:0/2093768881 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7af4012880 con 0x7f7af80c7dc0 2023-10-06T02:23:46.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.797+0000 7f7aed7fa700 1 -- 172.21.15.31:0/2093768881 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af4012c40 con 0x7f7af80c7dc0 2023-10-06T02:23:46.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.797+0000 7f7afe39d700 1 -- 172.21.15.31:0/2093768881 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7af80b0a60 con 0x7f7af80c7dc0 2023-10-06T02:23:46.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.797+0000 7f7afe39d700 1 -- 172.21.15.31:0/2093768881 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7af80b2590 con 0x7f7af80c7dc0 2023-10-06T02:23:46.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.798+0000 7f7afe39d700 1 -- 172.21.15.31:0/2093768881 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7af806a610 con 0x7f7af80c7dc0 2023-10-06T02:23:46.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.799+0000 7f7aed7fa700 1 -- 172.21.15.31:0/2093768881 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7af401b400 con 0x7f7af80c7dc0 2023-10-06T02:23:46.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.800+0000 7f7aed7fa700 1 --2- 172.21.15.31:0/2093768881 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f7adc066f30 0x7f7adc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:46.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.800+0000 7f7aed7fa700 1 -- 172.21.15.31:0/2093768881 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f7af4084300 con 0x7f7af80c7dc0 2023-10-06T02:23:46.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.800+0000 7f7afcb9a700 1 --2- 172.21.15.31:0/2093768881 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f7adc066f30 0x7f7adc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:46.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.800+0000 7f7afcb9a700 1 --2- 172.21.15.31:0/2093768881 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f7adc066f30 0x7f7adc0693e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7ae8006f00 tx=0x7f7ae8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:46.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:46.804+0000 7f7aed7fa700 1 -- 172.21.15.31:0/2093768881 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7af4051370 con 0x7f7af80c7dc0 2023-10-06T02:23:47.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.003+0000 7f7afe39d700 1 -- 172.21.15.31:0/2093768881 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7af804c3c0 con 0x7f7af80c7dc0 2023-10-06T02:23:47.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.004+0000 7f7aed7fa700 1 -- 172.21.15.31:0/2093768881 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7af4054990 con 0x7f7af80c7dc0 2023-10-06T02:23:47.004 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-10-06T02:23:47.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.006+0000 7f7afe39d700 1 -- 172.21.15.31:0/2093768881 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f7adc066f30 msgr2=0x7f7adc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:47.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.006+0000 7f7afe39d700 1 --2- 172.21.15.31:0/2093768881 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f7adc066f30 0x7f7adc0693e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7ae8006f00 tx=0x7f7ae8008040 comp rx=0 tx=0).stop 2023-10-06T02:23:47.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.006+0000 7f7afe39d700 1 -- 172.21.15.31:0/2093768881 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7af80c7dc0 msgr2=0x7f7af8159050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:47.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.006+0000 7f7afe39d700 1 --2- 172.21.15.31:0/2093768881 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7af80c7dc0 0x7f7af8159050 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7af400c7e0 tx=0x7f7af4007df0 comp rx=0 tx=0).stop 2023-10-06T02:23:47.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.006+0000 7f7afe39d700 1 -- 172.21.15.31:0/2093768881 shutdown_connections 2023-10-06T02:23:47.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.006+0000 7f7afe39d700 1 --2- 172.21.15.31:0/2093768881 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7af8142fd0 0x7f7af8141360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:47.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.006+0000 7f7afe39d700 1 --2- 172.21.15.31:0/2093768881 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7af80c7dc0 0x7f7af8159050 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:47.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.006+0000 7f7afe39d700 1 --2- 172.21.15.31:0/2093768881 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f7adc066f30 0x7f7adc0693e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:47.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.006+0000 7f7afe39d700 1 --2- 172.21.15.31:0/2093768881 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7af8142c00 0x7f7af80b2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:47.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.006+0000 7f7afe39d700 1 -- 172.21.15.31:0/2093768881 >> 172.21.15.31:0/2093768881 conn(0x7f7af80675a0 msgr2=0x7f7af80c8690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:47.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.006+0000 7f7afe39d700 1 -- 172.21.15.31:0/2093768881 shutdown_connections 2023-10-06T02:23:47.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.006+0000 7f7afe39d700 1 -- 172.21.15.31:0/2093768881 wait complete. 2023-10-06T02:23:47.024 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T02:23:47.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.218+0000 7fd320b9d700 1 Processor -- start 2023-10-06T02:23:47.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.219+0000 7fd320b9d700 1 -- start start 2023-10-06T02:23:47.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.220+0000 7fd320b9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd31c153a00 0x7fd31c153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:47.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.220+0000 7fd320b9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd31c0cea00 0x7fd31c0cee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:47.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.220+0000 7fd320b9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd31c15ac50 con 0x7fd31c15de50 2023-10-06T02:23:47.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.220+0000 7fd320b9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd31c15d010 con 0x7fd31c0cea00 2023-10-06T02:23:47.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.220+0000 7fd320b9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd31c15af60 con 0x7fd31c153a00 2023-10-06T02:23:47.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.220+0000 7fd31bfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd31c0cea00 0x7fd31c0cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:47.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.220+0000 7fd31bfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd31c0cea00 0x7fd31c0cee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45802/0 (socket says 172.21.15.31:45802) 2023-10-06T02:23:47.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.220+0000 7fd31bfff700 1 -- 172.21.15.31:0/3667207610 learned_addr learned my addr 172.21.15.31:0/3667207610 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:47.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.220+0000 7fd31b7fe700 1 --2- 172.21.15.31:0/3667207610 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd31c153a00 0x7fd31c153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:47.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.221+0000 7fd31bfff700 1 -- 172.21.15.31:0/3667207610 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd31c153a00 msgr2=0x7fd31c153dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:47.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.221+0000 7fd31bfff700 1 --2- 172.21.15.31:0/3667207610 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd31c153a00 0x7fd31c153dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:47.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.221+0000 7fd31bfff700 1 -- 172.21.15.31:0/3667207610 >> v1:172.21.15.62:6789/0 conn(0x7fd31c15de50 legacy=0x7fd31c15a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:47.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.221+0000 7fd31bfff700 1 -- 172.21.15.31:0/3667207610 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd31c15b240 con 0x7fd31c0cea00 2023-10-06T02:23:47.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.221+0000 7fd31bfff700 1 --2- 172.21.15.31:0/3667207610 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd31c0cea00 0x7fd31c0cee40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd314009f20 tx=0x7fd314009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a1dafd3bd8fb251 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:47.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.221+0000 7fd31a7fc700 1 -- 172.21.15.31:0/3667207610 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd31400e070 con 0x7fd31c0cea00 2023-10-06T02:23:47.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.221+0000 7fd31a7fc700 1 -- 172.21.15.31:0/3667207610 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd31400ba80 con 0x7fd31c0cea00 2023-10-06T02:23:47.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.221+0000 7fd31a7fc700 1 -- 172.21.15.31:0/3667207610 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3140092e0 con 0x7fd31c0cea00 2023-10-06T02:23:47.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.222+0000 7fd320b9d700 1 -- 172.21.15.31:0/3667207610 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd31c0cea00 msgr2=0x7fd31c0cee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:47.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.222+0000 7fd320b9d700 1 --2- 172.21.15.31:0/3667207610 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd31c0cea00 0x7fd31c0cee40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd314009f20 tx=0x7fd314009840 comp rx=0 tx=0).stop 2023-10-06T02:23:47.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.222+0000 7fd320b9d700 1 -- 172.21.15.31:0/3667207610 shutdown_connections 2023-10-06T02:23:47.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.222+0000 7fd320b9d700 1 --2- 172.21.15.31:0/3667207610 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd31c0cea00 0x7fd31c0cee40 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:47.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.222+0000 7fd320b9d700 1 --2- 172.21.15.31:0/3667207610 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd31c153a00 0x7fd31c153dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:47.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.222+0000 7fd320b9d700 1 -- 172.21.15.31:0/3667207610 >> 172.21.15.31:0/3667207610 conn(0x7fd31c0703a0 msgr2=0x7fd31c0707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:47.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.222+0000 7fd320b9d700 1 -- 172.21.15.31:0/3667207610 shutdown_connections 2023-10-06T02:23:47.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.222+0000 7fd320b9d700 1 -- 172.21.15.31:0/3667207610 wait complete. 2023-10-06T02:23:47.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.223+0000 7fd320b9d700 1 Processor -- start 2023-10-06T02:23:47.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.224+0000 7fd320b9d700 1 -- start start 2023-10-06T02:23:47.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.224+0000 7fd320b9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd31c0cea00 0x7fd31c152190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:47.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.224+0000 7fd320b9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd31c153a00 0x7fd31c150aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:47.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.224+0000 7fd320b9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd31c15de50 0x7fd31c0c7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:47.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.224+0000 7fd320b9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd31c0c8820 con 0x7fd31c15de50 2023-10-06T02:23:47.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.224+0000 7fd320b9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd31c0c8b00 con 0x7fd31c153a00 2023-10-06T02:23:47.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.224+0000 7fd320b9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd31c0c8de0 con 0x7fd31c0cea00 2023-10-06T02:23:47.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.225+0000 7fd31bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd31c15de50 0x7fd31c0c7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:47.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.225+0000 7fd31bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd31c15de50 0x7fd31c0c7ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45810/0 (socket says 172.21.15.31:45810) 2023-10-06T02:23:47.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.225+0000 7fd31bfff700 1 -- 172.21.15.31:0/3003189764 learned_addr learned my addr 172.21.15.31:0/3003189764 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:47.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.225+0000 7fd31b7fe700 1 --2- 172.21.15.31:0/3003189764 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd31c0cea00 0x7fd31c152190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:47.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.225+0000 7fd31affd700 1 --2- 172.21.15.31:0/3003189764 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd31c153a00 0x7fd31c150aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:47.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.225+0000 7fd31bfff700 1 -- 172.21.15.31:0/3003189764 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd31c0cea00 msgr2=0x7fd31c152190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:47.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.225+0000 7fd31bfff700 1 --2- 172.21.15.31:0/3003189764 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd31c0cea00 0x7fd31c152190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:47.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.225+0000 7fd31bfff700 1 -- 172.21.15.31:0/3003189764 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd31c153a00 msgr2=0x7fd31c150aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:47.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.225+0000 7fd31bfff700 1 --2- 172.21.15.31:0/3003189764 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd31c153a00 0x7fd31c150aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:47.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.225+0000 7fd31bfff700 1 -- 172.21.15.31:0/3003189764 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd314009ae0 con 0x7fd31c15de50 2023-10-06T02:23:47.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.225+0000 7fd31bfff700 1 --2- 172.21.15.31:0/3003189764 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd31c15de50 0x7fd31c0c7ff0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd31400b6c0 tx=0x7fd314018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:47.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.226+0000 7fd3037fe700 1 -- 172.21.15.31:0/3003189764 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3140032f0 con 0x7fd31c15de50 2023-10-06T02:23:47.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.226+0000 7fd3037fe700 1 -- 172.21.15.31:0/3003189764 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd314008e80 con 0x7fd31c15de50 2023-10-06T02:23:47.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.226+0000 7fd3037fe700 1 -- 172.21.15.31:0/3003189764 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd314026670 con 0x7fd31c15de50 2023-10-06T02:23:47.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.226+0000 7fd320b9d700 1 -- 172.21.15.31:0/3003189764 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd31c0c90c0 con 0x7fd31c15de50 2023-10-06T02:23:47.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.226+0000 7fd320b9d700 1 -- 172.21.15.31:0/3003189764 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd31c0b33e0 con 0x7fd31c15de50 2023-10-06T02:23:47.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.227+0000 7fd320b9d700 1 -- 172.21.15.31:0/3003189764 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd31c0b3600 con 0x7fd31c15de50 2023-10-06T02:23:47.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.228+0000 7fd3037fe700 1 -- 172.21.15.31:0/3003189764 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd314003040 con 0x7fd31c15de50 2023-10-06T02:23:47.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.228+0000 7fd3037fe700 1 --2- 172.21.15.31:0/3003189764 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fd308066f30 0x7fd3080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:47.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.228+0000 7fd3037fe700 1 -- 172.21.15.31:0/3003189764 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6116+0+0 (secure 0 0 0) 0x7fd3140867b0 con 0x7fd31c15de50 2023-10-06T02:23:47.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.229+0000 7fd31b7fe700 1 --2- 172.21.15.31:0/3003189764 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fd308066f30 0x7fd3080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:47.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.229+0000 7fd31b7fe700 1 --2- 172.21.15.31:0/3003189764 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fd308066f30 0x7fd3080693e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd30c006fd0 tx=0x7fd30c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:47.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.233+0000 7fd3037fe700 1 -- 172.21.15.31:0/3003189764 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd3140537e0 con 0x7fd31c15de50 2023-10-06T02:23:47.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.434+0000 7fd320b9d700 1 -- 172.21.15.31:0/3003189764 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd31c04c3c0 con 0x7fd31c15de50 2023-10-06T02:23:47.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.434+0000 7fd3037fe700 1 -- 172.21.15.31:0/3003189764 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd314056e00 con 0x7fd31c15de50 2023-10-06T02:23:47.435 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 8 2023-10-06T02:23:47.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.437+0000 7fd320b9d700 1 -- 172.21.15.31:0/3003189764 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fd308066f30 msgr2=0x7fd3080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:47.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.437+0000 7fd320b9d700 1 --2- 172.21.15.31:0/3003189764 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fd308066f30 0x7fd3080693e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd30c006fd0 tx=0x7fd30c008040 comp rx=0 tx=0).stop 2023-10-06T02:23:47.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.437+0000 7fd320b9d700 1 -- 172.21.15.31:0/3003189764 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd31c15de50 msgr2=0x7fd31c0c7ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:47.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.437+0000 7fd320b9d700 1 --2- 172.21.15.31:0/3003189764 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd31c15de50 0x7fd31c0c7ff0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd31400b6c0 tx=0x7fd314018000 comp rx=0 tx=0).stop 2023-10-06T02:23:47.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.437+0000 7fd320b9d700 1 -- 172.21.15.31:0/3003189764 shutdown_connections 2023-10-06T02:23:47.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.437+0000 7fd320b9d700 1 --2- 172.21.15.31:0/3003189764 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd31c15de50 0x7fd31c0c7ff0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:47.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.437+0000 7fd320b9d700 1 --2- 172.21.15.31:0/3003189764 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fd308066f30 0x7fd3080693e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:47.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.437+0000 7fd320b9d700 1 --2- 172.21.15.31:0/3003189764 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd31c0cea00 0x7fd31c152190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:47.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.437+0000 7fd320b9d700 1 --2- 172.21.15.31:0/3003189764 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd31c153a00 0x7fd31c150aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:47.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.437+0000 7fd320b9d700 1 -- 172.21.15.31:0/3003189764 >> 172.21.15.31:0/3003189764 conn(0x7fd31c0703a0 msgr2=0x7fd31c0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:47.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.437+0000 7fd320b9d700 1 -- 172.21.15.31:0/3003189764 shutdown_connections 2023-10-06T02:23:47.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.437+0000 7fd320b9d700 1 -- 172.21.15.31:0/3003189764 wait complete. 2023-10-06T02:23:47.455 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T02:23:47.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.644+0000 7f40bbfff700 1 Processor -- start 2023-10-06T02:23:47.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.644+0000 7f40bbfff700 1 -- start start 2023-10-06T02:23:47.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.645+0000 7f40bbfff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f40bc153c90 0x7f40bc15a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:47.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.645+0000 7f40bbfff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f40bc06db20 0x7f40bc0ca7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:47.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.645+0000 7f40bbfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f40bc154060 0x7f40bc0cacf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:47.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.645+0000 7f40bbfff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40bc15ada0 con 0x7f40bc154060 2023-10-06T02:23:47.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.645+0000 7f40bbfff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40bc06ca80 con 0x7f40bc153c90 2023-10-06T02:23:47.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.645+0000 7f40bbfff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40bc06cd60 con 0x7f40bc06db20 2023-10-06T02:23:47.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.645+0000 7f40bb7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f40bc154060 0x7f40bc0cacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:47.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.645+0000 7f40bb7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f40bc154060 0x7f40bc0cacf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45822/0 (socket says 172.21.15.31:45822) 2023-10-06T02:23:47.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.646+0000 7f40bb7fe700 1 -- 172.21.15.31:0/185287605 learned_addr learned my addr 172.21.15.31:0/185287605 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:47.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.646+0000 7f40baffd700 1 --2- 172.21.15.31:0/185287605 >> v2:172.21.15.62:3300/0 conn(0x7f40bc153c90 0x7f40bc15a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:47.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.646+0000 7f40ba7fc700 1 --2- 172.21.15.31:0/185287605 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f40bc06db20 0x7f40bc0ca7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:47.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.646+0000 7f40bb7fe700 1 -- 172.21.15.31:0/185287605 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f40bc06db20 msgr2=0x7f40bc0ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:47.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.646+0000 7f40bb7fe700 1 --2- 172.21.15.31:0/185287605 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f40bc06db20 0x7f40bc0ca7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:47.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.646+0000 7f40bb7fe700 1 -- 172.21.15.31:0/185287605 >> v2:172.21.15.62:3300/0 conn(0x7f40bc153c90 msgr2=0x7f40bc15a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:47.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.646+0000 7f40bb7fe700 1 --2- 172.21.15.31:0/185287605 >> v2:172.21.15.62:3300/0 conn(0x7f40bc153c90 0x7f40bc15a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:47.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.646+0000 7f40bb7fe700 1 -- 172.21.15.31:0/185287605 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40bc06d040 con 0x7f40bc154060 2023-10-06T02:23:47.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.646+0000 7f40ba7fc700 1 --2- 172.21.15.31:0/185287605 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f40bc06db20 0x7f40bc0ca7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:23:47.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.646+0000 7f40bb7fe700 1 --2- 172.21.15.31:0/185287605 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40bc154060 0x7f40bc0cacf0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f40bc0c37f0 tx=0x7f40b4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a68b7b34846032b3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:47.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.646+0000 7f40b9ffb700 1 -- 172.21.15.31:0/185287605 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b400e070 con 0x7f40bc154060 2023-10-06T02:23:47.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.647+0000 7f40b9ffb700 1 -- 172.21.15.31:0/185287605 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40b400ba20 con 0x7f40bc154060 2023-10-06T02:23:47.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.647+0000 7f40b9ffb700 1 -- 172.21.15.31:0/185287605 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b40092e0 con 0x7f40bc154060 2023-10-06T02:23:47.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.647+0000 7f40bbfff700 1 -- 172.21.15.31:0/185287605 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40bc154060 msgr2=0x7f40bc0cacf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:47.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.647+0000 7f40bbfff700 1 --2- 172.21.15.31:0/185287605 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40bc154060 0x7f40bc0cacf0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f40bc0c37f0 tx=0x7f40b4009840 comp rx=0 tx=0).stop 2023-10-06T02:23:47.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.647+0000 7f40bbfff700 1 -- 172.21.15.31:0/185287605 shutdown_connections 2023-10-06T02:23:47.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.647+0000 7f40bbfff700 1 --2- 172.21.15.31:0/185287605 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40bc154060 0x7f40bc0cacf0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:47.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.647+0000 7f40bbfff700 1 --2- 172.21.15.31:0/185287605 >> v2:172.21.15.62:3300/0 conn(0x7f40bc153c90 0x7f40bc15a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:47.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.647+0000 7f40bbfff700 1 --2- 172.21.15.31:0/185287605 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f40bc06db20 0x7f40bc0ca7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:47.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.647+0000 7f40bbfff700 1 -- 172.21.15.31:0/185287605 >> 172.21.15.31:0/185287605 conn(0x7f40bc06e720 msgr2=0x7f40bc06eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:47.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.647+0000 7f40bbfff700 1 -- 172.21.15.31:0/185287605 shutdown_connections 2023-10-06T02:23:47.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.647+0000 7f40bbfff700 1 -- 172.21.15.31:0/185287605 wait complete. 2023-10-06T02:23:47.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.649+0000 7f40bbfff700 1 Processor -- start 2023-10-06T02:23:47.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.649+0000 7f40bbfff700 1 -- start start 2023-10-06T02:23:47.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.650+0000 7f40bbfff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f40bc06db20 0x7f40bc151c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:47.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.650+0000 7f40bbfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40bc153c90 0x7f40bc16a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:47.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.650+0000 7f40bbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40bc154060 0x7f40bc16b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:47.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.650+0000 7f40bbfff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40bc152180 con 0x7f40bc154060 2023-10-06T02:23:47.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.650+0000 7f40bbfff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40bc16a550 con 0x7f40bc153c90 2023-10-06T02:23:47.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.650+0000 7f40bbfff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40bc152b60 con 0x7f40bc06db20 2023-10-06T02:23:47.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.650+0000 7f40bb7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40bc154060 0x7f40bc16b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:47.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.650+0000 7f40bb7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40bc154060 0x7f40bc16b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45832/0 (socket says 172.21.15.31:45832) 2023-10-06T02:23:47.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.650+0000 7f40bb7fe700 1 -- 172.21.15.31:0/1496247299 learned_addr learned my addr 172.21.15.31:0/1496247299 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:47.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.650+0000 7f40ba7fc700 1 --2- 172.21.15.31:0/1496247299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40bc153c90 0x7f40bc16a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:47.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.650+0000 7f40baffd700 1 --2- 172.21.15.31:0/1496247299 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f40bc06db20 0x7f40bc151c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:47.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.651+0000 7f40bb7fe700 1 -- 172.21.15.31:0/1496247299 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f40bc06db20 msgr2=0x7f40bc151c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:47.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.651+0000 7f40bb7fe700 1 --2- 172.21.15.31:0/1496247299 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f40bc06db20 0x7f40bc151c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:47.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.651+0000 7f40bb7fe700 1 -- 172.21.15.31:0/1496247299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40bc153c90 msgr2=0x7f40bc16a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:47.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.651+0000 7f40bb7fe700 1 --2- 172.21.15.31:0/1496247299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40bc153c90 0x7f40bc16a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:47.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.651+0000 7f40bb7fe700 1 -- 172.21.15.31:0/1496247299 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40b4009ae0 con 0x7f40bc154060 2023-10-06T02:23:47.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.651+0000 7f40baffd700 1 --2- 172.21.15.31:0/1496247299 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f40bc06db20 0x7f40bc151c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:23:47.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.651+0000 7f40bb7fe700 1 --2- 172.21.15.31:0/1496247299 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40bc154060 0x7f40bc16b7e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f40b4012000 tx=0x7f40b4018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:47.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.651+0000 7f40a2ffd700 1 -- 172.21.15.31:0/1496247299 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b40032f0 con 0x7f40bc154060 2023-10-06T02:23:47.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.651+0000 7f40a2ffd700 1 -- 172.21.15.31:0/1496247299 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40b4008e60 con 0x7f40bc154060 2023-10-06T02:23:47.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.651+0000 7f40bbfff700 1 -- 172.21.15.31:0/1496247299 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40bc0c4210 con 0x7f40bc154060 2023-10-06T02:23:47.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.651+0000 7f40a2ffd700 1 -- 172.21.15.31:0/1496247299 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b4026690 con 0x7f40bc154060 2023-10-06T02:23:47.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.651+0000 7f40bbfff700 1 -- 172.21.15.31:0/1496247299 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40bc16aa20 con 0x7f40bc154060 2023-10-06T02:23:47.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.652+0000 7f40bbfff700 1 -- 172.21.15.31:0/1496247299 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40bc04c3c0 con 0x7f40bc154060 2023-10-06T02:23:47.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.653+0000 7f40a2ffd700 1 -- 172.21.15.31:0/1496247299 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f40b4003040 con 0x7f40bc154060 2023-10-06T02:23:47.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.654+0000 7f40a2ffd700 1 --2- 172.21.15.31:0/1496247299 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f40a8066f30 0x7f40a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:47.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.654+0000 7f40a2ffd700 1 -- 172.21.15.31:0/1496247299 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f40b4086830 con 0x7f40bc154060 2023-10-06T02:23:47.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.654+0000 7f40baffd700 1 --2- 172.21.15.31:0/1496247299 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f40a8066f30 0x7f40a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:47.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.655+0000 7f40baffd700 1 --2- 172.21.15.31:0/1496247299 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f40a8066f30 0x7f40a80693e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f40ac009d80 tx=0x7f40ac008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:47.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.658+0000 7f40a2ffd700 1 -- 172.21.15.31:0/1496247299 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f40b4053910 con 0x7f40bc154060 2023-10-06T02:23:47.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.862+0000 7f40bbfff700 1 -- 172.21.15.31:0/1496247299 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f40bc0ca720 con 0x7f40bc154060 2023-10-06T02:23:47.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.862+0000 7f40a2ffd700 1 -- 172.21.15.31:0/1496247299 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f40b4056f30 con 0x7f40bc154060 2023-10-06T02:23:47.864 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-10-06T02:23:47.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.865+0000 7f40bbfff700 1 -- 172.21.15.31:0/1496247299 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f40a8066f30 msgr2=0x7f40a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:47.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.865+0000 7f40bbfff700 1 --2- 172.21.15.31:0/1496247299 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f40a8066f30 0x7f40a80693e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f40ac009d80 tx=0x7f40ac008040 comp rx=0 tx=0).stop 2023-10-06T02:23:47.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.865+0000 7f40bbfff700 1 -- 172.21.15.31:0/1496247299 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40bc154060 msgr2=0x7f40bc16b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:47.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.865+0000 7f40bbfff700 1 --2- 172.21.15.31:0/1496247299 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40bc154060 0x7f40bc16b7e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f40b4012000 tx=0x7f40b4018000 comp rx=0 tx=0).stop 2023-10-06T02:23:47.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.865+0000 7f40bbfff700 1 -- 172.21.15.31:0/1496247299 shutdown_connections 2023-10-06T02:23:47.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.865+0000 7f40bbfff700 1 --2- 172.21.15.31:0/1496247299 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40bc154060 0x7f40bc16b7e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:47.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.865+0000 7f40bbfff700 1 --2- 172.21.15.31:0/1496247299 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f40a8066f30 0x7f40a80693e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:47.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.865+0000 7f40bbfff700 1 --2- 172.21.15.31:0/1496247299 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f40bc06db20 0x7f40bc151c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:47.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.865+0000 7f40bbfff700 1 --2- 172.21.15.31:0/1496247299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40bc153c90 0x7f40bc16a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:47.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.865+0000 7f40bbfff700 1 -- 172.21.15.31:0/1496247299 >> 172.21.15.31:0/1496247299 conn(0x7f40bc06e720 msgr2=0x7f40bc15b020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:47.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.865+0000 7f40bbfff700 1 -- 172.21.15.31:0/1496247299 shutdown_connections 2023-10-06T02:23:47.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:47.865+0000 7f40bbfff700 1 -- 172.21.15.31:0/1496247299 wait complete. 2023-10-06T02:23:47.883 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T02:23:48.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.072+0000 7f294b59e700 1 Processor -- start 2023-10-06T02:23:48.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.072+0000 7f294b59e700 1 -- start start 2023-10-06T02:23:48.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.073+0000 7f294b59e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f294c153a00 0x7f294c153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:48.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.073+0000 7f294b59e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f294c0cea00 0x7f294c0cee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:48.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.073+0000 7f294b59e700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f294c15ac50 con 0x7f294c15de50 2023-10-06T02:23:48.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.073+0000 7f294b59e700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f294c15d010 con 0x7f294c0cea00 2023-10-06T02:23:48.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.073+0000 7f294b59e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f294c15af60 con 0x7f294c153a00 2023-10-06T02:23:48.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.073+0000 7f294ad9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f294c0cea00 0x7f294c0cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:48.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.074+0000 7f294a59c700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f294c153a00 0x7f294c153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:48.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.074+0000 7f2949d9b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f294c15de50 0x7f294c15a540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49324/0 (socket says 172.21.15.31:49324) 2023-10-06T02:23:48.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.074+0000 7f2949d9b700 1 -- 172.21.15.31:0/778648664 learned_addr learned my addr 172.21.15.31:0/778648664 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:23:48.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.074+0000 7f294ad9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f294c0cea00 0x7f294c0cee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38870/0 (socket says 172.21.15.31:38870) 2023-10-06T02:23:48.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.074+0000 7f294ad9d700 1 -- 172.21.15.31:0/778648664 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f294c153a00 msgr2=0x7f294c153dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:48.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.074+0000 7f294ad9d700 1 --2- 172.21.15.31:0/778648664 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f294c153a00 0x7f294c153dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:48.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.074+0000 7f294ad9d700 1 -- 172.21.15.31:0/778648664 >> v1:172.21.15.31:6789/0 conn(0x7f294c15de50 legacy=0x7f294c15a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:48.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.074+0000 7f294ad9d700 1 -- 172.21.15.31:0/778648664 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f294c15b240 con 0x7f294c0cea00 2023-10-06T02:23:48.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.074+0000 7f294ad9d700 1 --2- 172.21.15.31:0/778648664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f294c0cea00 0x7f294c0cee40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2944009e60 tx=0x7f2944009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=828a536604ab9eee server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:48.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.074+0000 7f294959a700 1 -- 172.21.15.31:0/778648664 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294400e070 con 0x7f294c0cea00 2023-10-06T02:23:48.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.075+0000 7f294959a700 1 -- 172.21.15.31:0/778648664 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f294400bb60 con 0x7f294c0cea00 2023-10-06T02:23:48.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.075+0000 7f294959a700 1 -- 172.21.15.31:0/778648664 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2944012430 con 0x7f294c0cea00 2023-10-06T02:23:48.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.075+0000 7f294b59e700 1 -- 172.21.15.31:0/778648664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f294c0cea00 msgr2=0x7f294c0cee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:48.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.075+0000 7f294b59e700 1 --2- 172.21.15.31:0/778648664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f294c0cea00 0x7f294c0cee40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2944009e60 tx=0x7f2944009780 comp rx=0 tx=0).stop 2023-10-06T02:23:48.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.075+0000 7f294b59e700 1 -- 172.21.15.31:0/778648664 shutdown_connections 2023-10-06T02:23:48.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.075+0000 7f294b59e700 1 --2- 172.21.15.31:0/778648664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f294c0cea00 0x7f294c0cee40 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:48.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.075+0000 7f294b59e700 1 --2- 172.21.15.31:0/778648664 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f294c153a00 0x7f294c153dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:48.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.075+0000 7f294b59e700 1 -- 172.21.15.31:0/778648664 >> 172.21.15.31:0/778648664 conn(0x7f294c0703a0 msgr2=0x7f294c0707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:48.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.075+0000 7f294b59e700 1 -- 172.21.15.31:0/778648664 shutdown_connections 2023-10-06T02:23:48.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.075+0000 7f294b59e700 1 -- 172.21.15.31:0/778648664 wait complete. 2023-10-06T02:23:48.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.077+0000 7f294b59e700 1 Processor -- start 2023-10-06T02:23:48.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.077+0000 7f294b59e700 1 -- start start 2023-10-06T02:23:48.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.078+0000 7f294b59e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f294c0cea00 0x7f294c152190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:48.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.078+0000 7f294b59e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f294c153a00 0x7f294c150aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:48.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.078+0000 7f294b59e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f294c15de50 0x7f294c0c7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:48.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.078+0000 7f294b59e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f294c0c8820 con 0x7f294c15de50 2023-10-06T02:23:48.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.078+0000 7f294b59e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f294c0c8b00 con 0x7f294c153a00 2023-10-06T02:23:48.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.078+0000 7f294b59e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f294c0c8de0 con 0x7f294c0cea00 2023-10-06T02:23:48.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.078+0000 7f294ad9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f294c15de50 0x7f294c0c7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:48.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.078+0000 7f294ad9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f294c15de50 0x7f294c0c7ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38884/0 (socket says 172.21.15.31:38884) 2023-10-06T02:23:48.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.078+0000 7f294ad9d700 1 -- 172.21.15.31:0/2324434042 learned_addr learned my addr 172.21.15.31:0/2324434042 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:48.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.078+0000 7f294a59c700 1 --2- 172.21.15.31:0/2324434042 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f294c0cea00 0x7f294c152190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:48.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.078+0000 7f2949d9b700 1 --2- 172.21.15.31:0/2324434042 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f294c153a00 0x7f294c150aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:48.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.079+0000 7f294ad9d700 1 -- 172.21.15.31:0/2324434042 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f294c0cea00 msgr2=0x7f294c152190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:48.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.079+0000 7f294ad9d700 1 --2- 172.21.15.31:0/2324434042 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f294c0cea00 0x7f294c152190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:48.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.079+0000 7f294ad9d700 1 -- 172.21.15.31:0/2324434042 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f294c153a00 msgr2=0x7f294c150aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:48.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.079+0000 7f294ad9d700 1 --2- 172.21.15.31:0/2324434042 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f294c153a00 0x7f294c150aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:48.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.079+0000 7f294ad9d700 1 -- 172.21.15.31:0/2324434042 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2944009a20 con 0x7f294c15de50 2023-10-06T02:23:48.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.079+0000 7f294ad9d700 1 --2- 172.21.15.31:0/2324434042 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f294c15de50 0x7f294c0c7ff0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f294c06eb00 tx=0x7f2944019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:48.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.079+0000 7f293a7fc700 1 -- 172.21.15.31:0/2324434042 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2944012ea0 con 0x7f294c15de50 2023-10-06T02:23:48.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.079+0000 7f293a7fc700 1 -- 172.21.15.31:0/2324434042 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2944027470 con 0x7f294c15de50 2023-10-06T02:23:48.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.079+0000 7f294b59e700 1 -- 172.21.15.31:0/2324434042 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f294c0c90c0 con 0x7f294c15de50 2023-10-06T02:23:48.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.079+0000 7f293a7fc700 1 -- 172.21.15.31:0/2324434042 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2944027850 con 0x7f294c15de50 2023-10-06T02:23:48.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.079+0000 7f294b59e700 1 -- 172.21.15.31:0/2324434042 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f294c0b33e0 con 0x7f294c15de50 2023-10-06T02:23:48.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.080+0000 7f294b59e700 1 -- 172.21.15.31:0/2324434042 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f294c0b3600 con 0x7f294c15de50 2023-10-06T02:23:48.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.081+0000 7f293a7fc700 1 -- 172.21.15.31:0/2324434042 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2944024070 con 0x7f294c15de50 2023-10-06T02:23:48.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.082+0000 7f293a7fc700 1 --2- 172.21.15.31:0/2324434042 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f292c066f30 0x7f292c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:48.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.082+0000 7f293a7fc700 1 -- 172.21.15.31:0/2324434042 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f29440871a0 con 0x7f294c15de50 2023-10-06T02:23:48.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.083+0000 7f294a59c700 1 --2- 172.21.15.31:0/2324434042 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f292c066f30 0x7f292c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:48.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.083+0000 7f294a59c700 1 --2- 172.21.15.31:0/2324434042 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f292c066f30 0x7f292c0693e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f293c009db0 tx=0x7f293c0091a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:48.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.086+0000 7f293a7fc700 1 -- 172.21.15.31:0/2324434042 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f29440541d0 con 0x7f294c15de50 2023-10-06T02:23:48.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.293+0000 7f294b59e700 1 -- 172.21.15.31:0/2324434042 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f294c04c3c0 con 0x7f294c15de50 2023-10-06T02:23:48.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.293+0000 7f293a7fc700 1 -- 172.21.15.31:0/2324434042 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f29440577f0 con 0x7f294c15de50 2023-10-06T02:23:48.295 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-10-06T02:23:48.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.296+0000 7f294b59e700 1 -- 172.21.15.31:0/2324434042 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f292c066f30 msgr2=0x7f292c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:48.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.296+0000 7f294b59e700 1 --2- 172.21.15.31:0/2324434042 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f292c066f30 0x7f292c0693e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f293c009db0 tx=0x7f293c0091a0 comp rx=0 tx=0).stop 2023-10-06T02:23:48.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.296+0000 7f294b59e700 1 -- 172.21.15.31:0/2324434042 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f294c15de50 msgr2=0x7f294c0c7ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:48.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.296+0000 7f294b59e700 1 --2- 172.21.15.31:0/2324434042 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f294c15de50 0x7f294c0c7ff0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f294c06eb00 tx=0x7f2944019000 comp rx=0 tx=0).stop 2023-10-06T02:23:48.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.296+0000 7f294b59e700 1 -- 172.21.15.31:0/2324434042 shutdown_connections 2023-10-06T02:23:48.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.296+0000 7f294b59e700 1 --2- 172.21.15.31:0/2324434042 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f294c15de50 0x7f294c0c7ff0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:48.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.296+0000 7f294b59e700 1 --2- 172.21.15.31:0/2324434042 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f292c066f30 0x7f292c0693e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:48.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.296+0000 7f294b59e700 1 --2- 172.21.15.31:0/2324434042 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f294c0cea00 0x7f294c152190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:48.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.296+0000 7f294b59e700 1 --2- 172.21.15.31:0/2324434042 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f294c153a00 0x7f294c150aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:48.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.296+0000 7f294b59e700 1 -- 172.21.15.31:0/2324434042 >> 172.21.15.31:0/2324434042 conn(0x7f294c0703a0 msgr2=0x7f294c0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:48.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.296+0000 7f294b59e700 1 -- 172.21.15.31:0/2324434042 shutdown_connections 2023-10-06T02:23:48.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.296+0000 7f294b59e700 1 -- 172.21.15.31:0/2324434042 wait complete. 2023-10-06T02:23:48.314 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:23:48.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.500+0000 7f26faf9d700 1 Processor -- start 2023-10-06T02:23:48.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.500+0000 7f26faf9d700 1 -- start start 2023-10-06T02:23:48.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.501+0000 7f26faf9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f26f40cf110 0x7f26f4164600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:48.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.501+0000 7f26faf9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26f415afd0 con 0x7f26f4153c30 2023-10-06T02:23:48.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.501+0000 7f26faf9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26f415b2b0 con 0x7f26f415de50 2023-10-06T02:23:48.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.501+0000 7f26faf9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26f415b590 con 0x7f26f40cf110 2023-10-06T02:23:48.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.502+0000 7f26f9f9b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f26f4153c30 0x7f26f415a5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49326/0 (socket says 172.21.15.31:49326) 2023-10-06T02:23:48.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.502+0000 7f26f9f9b700 1 -- 172.21.15.31:0/3531760337 learned_addr learned my addr 172.21.15.31:0/3531760337 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:23:48.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.502+0000 7f26fa79c700 1 --2- 172.21.15.31:0/3531760337 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f26f40cf110 0x7f26f4164600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:48.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.502+0000 7f26f8f99700 1 -- 172.21.15.31:0/3531760337 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 734591385 0 0) 0x7f26f415afd0 con 0x7f26f4153c30 2023-10-06T02:23:48.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.502+0000 7f26f8f99700 1 -- 172.21.15.31:0/3531760337 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26e0003570 con 0x7f26f4153c30 2023-10-06T02:23:48.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.502+0000 7f26f8f99700 1 -- 172.21.15.31:0/3531760337 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 152449575 0 0) 0x7f26f415b2b0 con 0x7f26f415de50 2023-10-06T02:23:48.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.502+0000 7f26f8f99700 1 -- 172.21.15.31:0/3531760337 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26f415afd0 con 0x7f26f415de50 2023-10-06T02:23:48.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.502+0000 7f26f8f99700 1 -- 172.21.15.31:0/3531760337 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4094214310 0 0) 0x7f26e0003570 con 0x7f26f4153c30 2023-10-06T02:23:48.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.502+0000 7f26f8f99700 1 -- 172.21.15.31:0/3531760337 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f26f40cf110 msgr2=0x7f26f4164600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:48.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.502+0000 7f26f8f99700 1 --2- 172.21.15.31:0/3531760337 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f26f40cf110 0x7f26f4164600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:48.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.502+0000 7f26f8f99700 1 -- 172.21.15.31:0/3531760337 >> v1:172.21.15.62:6789/0 conn(0x7f26f415de50 legacy=0x7f26f40cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:48.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.502+0000 7f26f8f99700 1 -- 172.21.15.31:0/3531760337 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26f415b870 con 0x7f26f4153c30 2023-10-06T02:23:48.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.502+0000 7f26f8f99700 1 -- 172.21.15.31:0/3531760337 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7f26f0001470 con 0x7f26f4153c30 2023-10-06T02:23:48.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.503+0000 7f26f8f99700 1 -- 172.21.15.31:0/3531760337 >> v1:172.21.15.31:6789/0 conn(0x7f26f4153c30 legacy=0x7f26f415a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:48.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.503+0000 7f26f8f99700 1 --2- 172.21.15.31:0/3531760337 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26e0003c20 0x7f26e0004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:48.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.503+0000 7f26f8f99700 1 --2- 172.21.15.31:0/3531760337 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f26f40cf110 0x7f26e0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:48.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.503+0000 7f26f8f99700 1 --2- 172.21.15.31:0/3531760337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26e0004d50 0x7f26e00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:48.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.503+0000 7f26f8f99700 1 -- 172.21.15.31:0/3531760337 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f415b590 con 0x7f26e0004d50 2023-10-06T02:23:48.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.503+0000 7f26f8f99700 1 -- 172.21.15.31:0/3531760337 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e000bb70 con 0x7f26e0003c20 2023-10-06T02:23:48.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.503+0000 7f26f8f99700 1 -- 172.21.15.31:0/3531760337 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26e000bd30 con 0x7f26f40cf110 2023-10-06T02:23:48.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.503+0000 7f26fa79c700 1 --2- 172.21.15.31:0/3531760337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26e0004d50 0x7f26e00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:48.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.503+0000 7f26f979a700 1 --2- 172.21.15.31:0/3531760337 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f26f40cf110 0x7f26e0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:48.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.503+0000 7f26f9f9b700 1 --2- 172.21.15.31:0/3531760337 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26e0003c20 0x7f26e0004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:48.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.503+0000 7f26fa79c700 1 -- 172.21.15.31:0/3531760337 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f26f40cf110 msgr2=0x7f26e0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:48.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.503+0000 7f26fa79c700 1 --2- 172.21.15.31:0/3531760337 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f26f40cf110 0x7f26e0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:48.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.503+0000 7f26fa79c700 1 -- 172.21.15.31:0/3531760337 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26e0003c20 msgr2=0x7f26e0004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:48.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.503+0000 7f26fa79c700 1 --2- 172.21.15.31:0/3531760337 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26e0003c20 0x7f26e0004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:48.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.503+0000 7f26fa79c700 1 -- 172.21.15.31:0/3531760337 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26e0003570 con 0x7f26e0004d50 2023-10-06T02:23:48.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.504+0000 7f26fa79c700 1 --2- 172.21.15.31:0/3531760337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26e0004d50 0x7f26e00051c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f26ec00e210 tx=0x7f26ec00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a204ff0964196a8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:48.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.504+0000 7f26f8f99700 1 -- 172.21.15.31:0/3531760337 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ec00eac0 con 0x7f26e0004d50 2023-10-06T02:23:48.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.504+0000 7f26f8f99700 1 -- 172.21.15.31:0/3531760337 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26ec00f040 con 0x7f26e0004d50 2023-10-06T02:23:48.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.504+0000 7f26f8f99700 1 -- 172.21.15.31:0/3531760337 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ec00ee70 con 0x7f26e0004d50 2023-10-06T02:23:48.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.504+0000 7f26faf9d700 1 -- 172.21.15.31:0/3531760337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26e0004d50 msgr2=0x7f26e00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:48.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.504+0000 7f26faf9d700 1 --2- 172.21.15.31:0/3531760337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26e0004d50 0x7f26e00051c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f26ec00e210 tx=0x7f26ec00b040 comp rx=0 tx=0).stop 2023-10-06T02:23:48.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.504+0000 7f26fa79c700 1 -- 172.21.15.31:0/3531760337 reap_dead start 2023-10-06T02:23:48.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.504+0000 7f26faf9d700 1 -- 172.21.15.31:0/3531760337 shutdown_connections 2023-10-06T02:23:48.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.504+0000 7f26faf9d700 1 -- 172.21.15.31:0/3531760337 >> 172.21.15.31:0/3531760337 conn(0x7f26f40703a0 msgr2=0x7f26f40707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:48.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.505+0000 7f26faf9d700 1 -- 172.21.15.31:0/3531760337 shutdown_connections 2023-10-06T02:23:48.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.505+0000 7f26faf9d700 1 -- 172.21.15.31:0/3531760337 wait complete. 2023-10-06T02:23:48.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.506+0000 7f26faf9d700 1 Processor -- start 2023-10-06T02:23:48.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.506+0000 7f26faf9d700 1 -- start start 2023-10-06T02:23:48.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.507+0000 7f26faf9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26e0004d50 0x7f26f4143e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:48.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.507+0000 7f26faf9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26f40b44f0 0x7f26f40b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:48.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.507+0000 7f26faf9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f26f40b2160 0x7f26f40b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:48.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.507+0000 7f26faf9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f40b4060 con 0x7f26f40b44f0 2023-10-06T02:23:48.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.507+0000 7f26faf9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f40b25b0 con 0x7f26e0004d50 2023-10-06T02:23:48.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.507+0000 7f26faf9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26f40b3450 con 0x7f26f40b2160 2023-10-06T02:23:48.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.507+0000 7f26f979a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26f40b44f0 0x7f26f40b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:48.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.507+0000 7f26f979a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26f40b44f0 0x7f26f40b1c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38908/0 (socket says 172.21.15.31:38908) 2023-10-06T02:23:48.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.507+0000 7f26f979a700 1 -- 172.21.15.31:0/2239536798 learned_addr learned my addr 172.21.15.31:0/2239536798 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:48.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.508+0000 7f26fa79c700 1 --2- 172.21.15.31:0/2239536798 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f26f40b2160 0x7f26f40b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:48.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.508+0000 7f26f9f9b700 1 --2- 172.21.15.31:0/2239536798 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26e0004d50 0x7f26f4143e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:48.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.508+0000 7f26f979a700 1 -- 172.21.15.31:0/2239536798 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f26f40b2160 msgr2=0x7f26f40b4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:48.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.508+0000 7f26f979a700 1 --2- 172.21.15.31:0/2239536798 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f26f40b2160 0x7f26f40b4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:48.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.508+0000 7f26f979a700 1 -- 172.21.15.31:0/2239536798 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26e0004d50 msgr2=0x7f26f4143e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:48.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.508+0000 7f26f979a700 1 --2- 172.21.15.31:0/2239536798 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26e0004d50 0x7f26f4143e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:48.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.508+0000 7f26f979a700 1 -- 172.21.15.31:0/2239536798 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26f40b3730 con 0x7f26f40b44f0 2023-10-06T02:23:48.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.508+0000 7f26f9f9b700 1 --2- 172.21.15.31:0/2239536798 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26e0004d50 0x7f26f4143e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:23:48.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.508+0000 7f26f979a700 1 --2- 172.21.15.31:0/2239536798 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26f40b44f0 0x7f26f40b1c20 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f26e4002bf0 tx=0x7f26e4007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:48.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.508+0000 7f26e9ffb700 1 -- 172.21.15.31:0/2239536798 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e400fab0 con 0x7f26f40b44f0 2023-10-06T02:23:48.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.508+0000 7f26e9ffb700 1 -- 172.21.15.31:0/2239536798 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26e400fc10 con 0x7f26f40b44f0 2023-10-06T02:23:48.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.508+0000 7f26e9ffb700 1 -- 172.21.15.31:0/2239536798 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e40133f0 con 0x7f26f40b44f0 2023-10-06T02:23:48.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.508+0000 7f26faf9d700 1 -- 172.21.15.31:0/2239536798 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26f40b3a10 con 0x7f26f40b44f0 2023-10-06T02:23:48.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.508+0000 7f26faf9d700 1 -- 172.21.15.31:0/2239536798 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26f41443a0 con 0x7f26f40b44f0 2023-10-06T02:23:48.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.509+0000 7f26faf9d700 1 -- 172.21.15.31:0/2239536798 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26f404c3c0 con 0x7f26f40b44f0 2023-10-06T02:23:48.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.510+0000 7f26e9ffb700 1 -- 172.21.15.31:0/2239536798 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f26e4013760 con 0x7f26f40b44f0 2023-10-06T02:23:48.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.511+0000 7f26e9ffb700 1 --2- 172.21.15.31:0/2239536798 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f26d8066f30 0x7f26d80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:48.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.511+0000 7f26e9ffb700 1 -- 172.21.15.31:0/2239536798 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f26e4084fe0 con 0x7f26f40b44f0 2023-10-06T02:23:48.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.511+0000 7f26f9f9b700 1 --2- 172.21.15.31:0/2239536798 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f26d8066f30 0x7f26d80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:48.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.512+0000 7f26f9f9b700 1 --2- 172.21.15.31:0/2239536798 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f26d8066f30 0x7f26d80693e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f26f0005a50 tx=0x7f26f4159c60 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:48.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.515+0000 7f26e9ffb700 1 -- 172.21.15.31:0/2239536798 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f26e4052050 con 0x7f26f40b44f0 2023-10-06T02:23:48.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.718+0000 7f26faf9d700 1 -- 172.21.15.31:0/2239536798 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f26f40b3eb0 con 0x7f26f40b44f0 2023-10-06T02:23:48.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.719+0000 7f26e9ffb700 1 -- 172.21.15.31:0/2239536798 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+16914 (secure 0 0 0) 0x7f26e4055670 con 0x7f26f40b44f0 2023-10-06T02:23:48.721 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:23:48.721 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":25,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:23:48.196317+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:23:07.290269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T02:23:08.504679+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.2f","mappings":[{"from":3,"to":2}]},{"pgid":"4.31","mappings":[{"from":1,"to":5}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:23:48.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.721+0000 7f26faf9d700 1 -- 172.21.15.31:0/2239536798 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f26d8066f30 msgr2=0x7f26d80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:48.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.721+0000 7f26faf9d700 1 --2- 172.21.15.31:0/2239536798 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f26d8066f30 0x7f26d80693e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f26f0005a50 tx=0x7f26f4159c60 comp rx=0 tx=0).stop 2023-10-06T02:23:48.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.721+0000 7f26faf9d700 1 -- 172.21.15.31:0/2239536798 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26f40b44f0 msgr2=0x7f26f40b1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:48.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.721+0000 7f26faf9d700 1 --2- 172.21.15.31:0/2239536798 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26f40b44f0 0x7f26f40b1c20 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f26e4002bf0 tx=0x7f26e4007d30 comp rx=0 tx=0).stop 2023-10-06T02:23:48.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.722+0000 7f26faf9d700 1 -- 172.21.15.31:0/2239536798 shutdown_connections 2023-10-06T02:23:48.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.722+0000 7f26faf9d700 1 --2- 172.21.15.31:0/2239536798 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f26d8066f30 0x7f26d80693e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:48.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.722+0000 7f26faf9d700 1 --2- 172.21.15.31:0/2239536798 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f26f40b2160 0x7f26f40b4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:48.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.722+0000 7f26faf9d700 1 --2- 172.21.15.31:0/2239536798 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26e0004d50 0x7f26f4143e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:48.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.722+0000 7f26faf9d700 1 --2- 172.21.15.31:0/2239536798 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26f40b44f0 0x7f26f40b1c20 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:48.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.722+0000 7f26faf9d700 1 -- 172.21.15.31:0/2239536798 >> 172.21.15.31:0/2239536798 conn(0x7f26f40703a0 msgr2=0x7f26f4153030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:48.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.722+0000 7f26faf9d700 1 -- 172.21.15.31:0/2239536798 shutdown_connections 2023-10-06T02:23:48.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.722+0000 7f26faf9d700 1 -- 172.21.15.31:0/2239536798 wait complete. 2023-10-06T02:23:48.740 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T02:22:53.183425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T02:23:00.046967+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T02:23:07.290269+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T02:23:08.504679+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T02:23:48.741 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T02:23:48.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.932+0000 7f1d5b59e700 1 Processor -- start 2023-10-06T02:23:48.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.932+0000 7f1d5b59e700 1 -- start start 2023-10-06T02:23:48.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.933+0000 7f1d5b59e700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f1d5c153a00 0x7f1d5c153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:48.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.933+0000 7f1d5b59e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1d5c0cea00 0x7f1d5c0cee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:48.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.933+0000 7f1d5b59e700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d5c15ac50 con 0x7f1d5c15de50 2023-10-06T02:23:48.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.933+0000 7f1d5b59e700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d5c15d010 con 0x7f1d5c153a00 2023-10-06T02:23:48.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.933+0000 7f1d5b59e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d5c0cf3c0 con 0x7f1d5c0cea00 2023-10-06T02:23:48.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.934+0000 7f1d5ad9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1d5c0cea00 0x7f1d5c0cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:48.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.934+0000 7f1d59d9b700 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f1d5c15de50 0x7f1d5c15a540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:59826/0 (socket says 172.21.15.31:59826) 2023-10-06T02:23:48.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.934+0000 7f1d59d9b700 1 -- 172.21.15.31:0/2571596229 learned_addr learned my addr 172.21.15.31:0/2571596229 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:23:48.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.934+0000 7f1d5a59c700 1 --2- 172.21.15.31:0/2571596229 >> v2:172.21.15.62:3300/0 conn(0x7f1d5c153a00 0x7f1d5c153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:48.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.934+0000 7f1d5ad9d700 1 -- 172.21.15.31:0/2571596229 >> v2:172.21.15.62:3300/0 conn(0x7f1d5c153a00 msgr2=0x7f1d5c153dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:48.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.934+0000 7f1d5ad9d700 1 --2- 172.21.15.31:0/2571596229 >> v2:172.21.15.62:3300/0 conn(0x7f1d5c153a00 0x7f1d5c153dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:48.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.934+0000 7f1d5ad9d700 1 -- 172.21.15.31:0/2571596229 >> v1:172.21.15.62:6789/0 conn(0x7f1d5c15de50 legacy=0x7f1d5c15a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:48.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.934+0000 7f1d5ad9d700 1 -- 172.21.15.31:0/2571596229 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d5c15af60 con 0x7f1d5c0cea00 2023-10-06T02:23:48.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.934+0000 7f1d5959a700 1 -- 172.21.15.31:0/2571596229 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2305621646 0 0) 0x7f1d5c15ac50 con 0x7f1d5c15de50 2023-10-06T02:23:48.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.935+0000 7f1d5ad9d700 1 --2- 172.21.15.31:0/2571596229 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1d5c0cea00 0x7f1d5c0cee80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1d54009e60 tx=0x7f1d54009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28593bfe537efe1b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:48.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.935+0000 7f1d5959a700 1 -- 172.21.15.31:0/2571596229 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d5400e070 con 0x7f1d5c0cea00 2023-10-06T02:23:48.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.935+0000 7f1d5959a700 1 -- 172.21.15.31:0/2571596229 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d5400bb60 con 0x7f1d5c0cea00 2023-10-06T02:23:48.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.935+0000 7f1d5959a700 1 -- 172.21.15.31:0/2571596229 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d54012430 con 0x7f1d5c0cea00 2023-10-06T02:23:48.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.935+0000 7f1d5b59e700 1 -- 172.21.15.31:0/2571596229 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1d5c0cea00 msgr2=0x7f1d5c0cee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:48.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.935+0000 7f1d5b59e700 1 --2- 172.21.15.31:0/2571596229 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1d5c0cea00 0x7f1d5c0cee80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1d54009e60 tx=0x7f1d54009780 comp rx=0 tx=0).stop 2023-10-06T02:23:48.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.935+0000 7f1d5b59e700 1 -- 172.21.15.31:0/2571596229 shutdown_connections 2023-10-06T02:23:48.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.935+0000 7f1d5b59e700 1 --2- 172.21.15.31:0/2571596229 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1d5c0cea00 0x7f1d5c0cee80 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:48.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.935+0000 7f1d5b59e700 1 --2- 172.21.15.31:0/2571596229 >> v2:172.21.15.62:3300/0 conn(0x7f1d5c153a00 0x7f1d5c153dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:48.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.935+0000 7f1d5b59e700 1 -- 172.21.15.31:0/2571596229 >> 172.21.15.31:0/2571596229 conn(0x7f1d5c0703a0 msgr2=0x7f1d5c0707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:48.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.936+0000 7f1d5b59e700 1 -- 172.21.15.31:0/2571596229 shutdown_connections 2023-10-06T02:23:48.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.936+0000 7f1d5b59e700 1 -- 172.21.15.31:0/2571596229 wait complete. 2023-10-06T02:23:48.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.937+0000 7f1d5b59e700 1 Processor -- start 2023-10-06T02:23:48.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.937+0000 7f1d5b59e700 1 -- start start 2023-10-06T02:23:48.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.938+0000 7f1d5b59e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1d5c0cea00 0x7f1d5c0b34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:48.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.938+0000 7f1d5b59e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d5c153a00 0x7f1d5c0cd460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:48.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.938+0000 7f1d5b59e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d5c15de50 0x7f1d5c153080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:48.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.938+0000 7f1d5b59e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d5c0c7fb0 con 0x7f1d5c15de50 2023-10-06T02:23:48.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.938+0000 7f1d5b59e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d5c1535c0 con 0x7f1d5c153a00 2023-10-06T02:23:48.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.938+0000 7f1d5b59e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d5c0cc370 con 0x7f1d5c0cea00 2023-10-06T02:23:48.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.938+0000 7f1d5ad9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d5c15de50 0x7f1d5c153080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:48.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.938+0000 7f1d5ad9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d5c15de50 0x7f1d5c153080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38924/0 (socket says 172.21.15.31:38924) 2023-10-06T02:23:48.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.938+0000 7f1d5ad9d700 1 -- 172.21.15.31:0/2537206007 learned_addr learned my addr 172.21.15.31:0/2537206007 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:48.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.938+0000 7f1d5a59c700 1 --2- 172.21.15.31:0/2537206007 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1d5c0cea00 0x7f1d5c0b34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:48.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.939+0000 7f1d59d9b700 1 --2- 172.21.15.31:0/2537206007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d5c153a00 0x7f1d5c0cd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:48.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.939+0000 7f1d5ad9d700 1 -- 172.21.15.31:0/2537206007 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1d5c0cea00 msgr2=0x7f1d5c0b34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:48.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.939+0000 7f1d5ad9d700 1 --2- 172.21.15.31:0/2537206007 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1d5c0cea00 0x7f1d5c0b34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:48.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.939+0000 7f1d5ad9d700 1 -- 172.21.15.31:0/2537206007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d5c153a00 msgr2=0x7f1d5c0cd460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:48.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.939+0000 7f1d5ad9d700 1 --2- 172.21.15.31:0/2537206007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d5c153a00 0x7f1d5c0cd460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:48.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.939+0000 7f1d5ad9d700 1 -- 172.21.15.31:0/2537206007 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d54009a20 con 0x7f1d5c15de50 2023-10-06T02:23:48.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.939+0000 7f1d59d9b700 1 --2- 172.21.15.31:0/2537206007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d5c153a00 0x7f1d5c0cd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:23:48.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.939+0000 7f1d5a59c700 1 --2- 172.21.15.31:0/2537206007 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1d5c0cea00 0x7f1d5c0b34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:23:48.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.939+0000 7f1d5ad9d700 1 --2- 172.21.15.31:0/2537206007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d5c15de50 0x7f1d5c153080 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c0cf500 tx=0x7f1d54019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:48.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.940+0000 7f1d4a7fc700 1 -- 172.21.15.31:0/2537206007 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d54012ea0 con 0x7f1d5c15de50 2023-10-06T02:23:48.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.940+0000 7f1d4a7fc700 1 -- 172.21.15.31:0/2537206007 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d54027470 con 0x7f1d5c15de50 2023-10-06T02:23:48.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.940+0000 7f1d4a7fc700 1 -- 172.21.15.31:0/2537206007 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d54027830 con 0x7f1d5c15de50 2023-10-06T02:23:48.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.940+0000 7f1d5b59e700 1 -- 172.21.15.31:0/2537206007 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d5c0c7680 con 0x7f1d5c15de50 2023-10-06T02:23:48.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.940+0000 7f1d5b59e700 1 -- 172.21.15.31:0/2537206007 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d5c0c7c40 con 0x7f1d5c15de50 2023-10-06T02:23:48.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.941+0000 7f1d5b59e700 1 -- 172.21.15.31:0/2537206007 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d5c0c7dd0 con 0x7f1d5c15de50 2023-10-06T02:23:48.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.942+0000 7f1d4a7fc700 1 -- 172.21.15.31:0/2537206007 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1d54024070 con 0x7f1d5c15de50 2023-10-06T02:23:48.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.942+0000 7f1d4a7fc700 1 --2- 172.21.15.31:0/2537206007 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f1d3c066f30 0x7f1d3c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:48.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.943+0000 7f1d4a7fc700 1 -- 172.21.15.31:0/2537206007 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f1d54086dd0 con 0x7f1d5c15de50 2023-10-06T02:23:48.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.943+0000 7f1d5a59c700 1 --2- 172.21.15.31:0/2537206007 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f1d3c066f30 0x7f1d3c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:48.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.943+0000 7f1d5a59c700 1 --2- 172.21.15.31:0/2537206007 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f1d3c066f30 0x7f1d3c0693e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c011480 tx=0x7f1d4c0091a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:48.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:48.947+0000 7f1d4a7fc700 1 -- 172.21.15.31:0/2537206007 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1d54053e00 con 0x7f1d5c15de50 2023-10-06T02:23:49.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.152+0000 7f1d5b59e700 1 -- 172.21.15.31:0/2537206007 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1d5c04c3c0 con 0x7f1d5c15de50 2023-10-06T02:23:49.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.153+0000 7f1d4a7fc700 1 -- 172.21.15.31:0/2537206007 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1d5402c0a0 con 0x7f1d5c15de50 2023-10-06T02:23:49.154 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-10-06T02:23:49.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.155+0000 7f1d5b59e700 1 -- 172.21.15.31:0/2537206007 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f1d3c066f30 msgr2=0x7f1d3c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:49.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.155+0000 7f1d5b59e700 1 --2- 172.21.15.31:0/2537206007 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f1d3c066f30 0x7f1d3c0693e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c011480 tx=0x7f1d4c0091a0 comp rx=0 tx=0).stop 2023-10-06T02:23:49.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.155+0000 7f1d5b59e700 1 -- 172.21.15.31:0/2537206007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d5c15de50 msgr2=0x7f1d5c153080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:49.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.155+0000 7f1d5b59e700 1 --2- 172.21.15.31:0/2537206007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d5c15de50 0x7f1d5c153080 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c0cf500 tx=0x7f1d54019000 comp rx=0 tx=0).stop 2023-10-06T02:23:49.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.155+0000 7f1d5b59e700 1 -- 172.21.15.31:0/2537206007 shutdown_connections 2023-10-06T02:23:49.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.155+0000 7f1d5b59e700 1 --2- 172.21.15.31:0/2537206007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d5c15de50 0x7f1d5c153080 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:49.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.155+0000 7f1d5b59e700 1 --2- 172.21.15.31:0/2537206007 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f1d3c066f30 0x7f1d3c0693e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:49.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.155+0000 7f1d5b59e700 1 --2- 172.21.15.31:0/2537206007 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1d5c0cea00 0x7f1d5c0b34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:49.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.155+0000 7f1d5b59e700 1 --2- 172.21.15.31:0/2537206007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d5c153a00 0x7f1d5c0cd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:49.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.155+0000 7f1d5b59e700 1 -- 172.21.15.31:0/2537206007 >> 172.21.15.31:0/2537206007 conn(0x7f1d5c0703a0 msgr2=0x7f1d5c159c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:49.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.155+0000 7f1d5b59e700 1 -- 172.21.15.31:0/2537206007 shutdown_connections 2023-10-06T02:23:49.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.155+0000 7f1d5b59e700 1 -- 172.21.15.31:0/2537206007 wait complete. 2023-10-06T02:23:49.174 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T02:23:49.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.363+0000 7fb40759e700 1 Processor -- start 2023-10-06T02:23:49.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.363+0000 7fb40759e700 1 -- start start 2023-10-06T02:23:49.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.364+0000 7fb40759e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb4080c7fb0 0x7fb40806abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:49.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.365+0000 7fb40759e700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb40813f780 0x7fb40813fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:49.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.365+0000 7fb40759e700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb408140120 con 0x7fb4080c7be0 2023-10-06T02:23:49.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.365+0000 7fb40759e700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb40806bf10 con 0x7fb40813f780 2023-10-06T02:23:49.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.365+0000 7fb40759e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb40806b290 con 0x7fb4080c7fb0 2023-10-06T02:23:49.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.365+0000 7fb40659c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fb4080c7be0 0x7fb40815a990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49332/0 (socket says 172.21.15.31:49332) 2023-10-06T02:23:49.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.365+0000 7fb40659c700 1 -- 172.21.15.31:0/1332103693 learned_addr learned my addr 172.21.15.31:0/1332103693 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:23:49.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.365+0000 7fb406d9d700 1 --2- 172.21.15.31:0/1332103693 >> v2:172.21.15.62:3300/0 conn(0x7fb40813f780 0x7fb40813fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:49.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.365+0000 7fb405d9b700 1 --2- 172.21.15.31:0/1332103693 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb4080c7fb0 0x7fb40806abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:49.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.365+0000 7fb40559a700 1 -- 172.21.15.31:0/1332103693 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3852596265 0 0) 0x7fb3f8001110 con 0x7fb4080c7be0 2023-10-06T02:23:49.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.365+0000 7fb40559a700 1 -- 172.21.15.31:0/1332103693 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3f0003570 con 0x7fb4080c7be0 2023-10-06T02:23:49.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.365+0000 7fb40559a700 1 -- 172.21.15.31:0/1332103693 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4060274844 0 0) 0x7fb3f0003570 con 0x7fb4080c7be0 2023-10-06T02:23:49.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.366+0000 7fb40559a700 1 -- 172.21.15.31:0/1332103693 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb4080c7fb0 msgr2=0x7fb40806abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:49.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.366+0000 7fb40559a700 1 --2- 172.21.15.31:0/1332103693 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb4080c7fb0 0x7fb40806abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:49.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.366+0000 7fb40559a700 1 -- 172.21.15.31:0/1332103693 >> v2:172.21.15.62:3300/0 conn(0x7fb40813f780 msgr2=0x7fb40813fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:49.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.366+0000 7fb40559a700 1 --2- 172.21.15.31:0/1332103693 >> v2:172.21.15.62:3300/0 conn(0x7fb40813f780 0x7fb40813fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:49.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.366+0000 7fb40559a700 1 -- 172.21.15.31:0/1332103693 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb40806b570 con 0x7fb4080c7be0 2023-10-06T02:23:49.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.366+0000 7fb40559a700 1 -- 172.21.15.31:0/1332103693 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7fb3f8001600 con 0x7fb4080c7be0 2023-10-06T02:23:49.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.366+0000 7fb40559a700 1 -- 172.21.15.31:0/1332103693 >> v1:172.21.15.31:6789/0 conn(0x7fb4080c7be0 legacy=0x7fb40815a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:49.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.366+0000 7fb40559a700 1 --2- 172.21.15.31:0/1332103693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3f0003a60 0x7fb3f0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:49.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.366+0000 7fb40559a700 1 --2- 172.21.15.31:0/1332103693 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb4080c7fb0 0x7fb3f0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:49.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.366+0000 7fb40559a700 1 --2- 172.21.15.31:0/1332103693 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3f0004d50 0x7fb3f00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:49.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.366+0000 7fb40559a700 1 -- 172.21.15.31:0/1332103693 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb40806bf10 con 0x7fb3f0004d50 2023-10-06T02:23:49.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.366+0000 7fb40559a700 1 -- 172.21.15.31:0/1332103693 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb40806b290 con 0x7fb3f0003a60 2023-10-06T02:23:49.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.366+0000 7fb40559a700 1 -- 172.21.15.31:0/1332103693 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f000dc70 con 0x7fb4080c7fb0 2023-10-06T02:23:49.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.366+0000 7fb406d9d700 1 --2- 172.21.15.31:0/1332103693 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3f0004d50 0x7fb3f00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:49.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.366+0000 7fb405d9b700 1 --2- 172.21.15.31:0/1332103693 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb4080c7fb0 0x7fb3f0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:49.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.366+0000 7fb40659c700 1 --2- 172.21.15.31:0/1332103693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3f0003a60 0x7fb3f0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:49.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.366+0000 7fb406d9d700 1 -- 172.21.15.31:0/1332103693 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb4080c7fb0 msgr2=0x7fb3f0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:49.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.366+0000 7fb406d9d700 1 --2- 172.21.15.31:0/1332103693 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb4080c7fb0 0x7fb3f0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:49.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.366+0000 7fb406d9d700 1 -- 172.21.15.31:0/1332103693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3f0003a60 msgr2=0x7fb3f0003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:49.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.366+0000 7fb406d9d700 1 --2- 172.21.15.31:0/1332103693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3f0003a60 0x7fb3f0003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:49.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.367+0000 7fb406d9d700 1 -- 172.21.15.31:0/1332103693 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3f0003570 con 0x7fb3f0004d50 2023-10-06T02:23:49.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.367+0000 7fb40659c700 1 --2- 172.21.15.31:0/1332103693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3f0003a60 0x7fb3f0003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:23:49.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.367+0000 7fb406d9d700 1 --2- 172.21.15.31:0/1332103693 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3f0004d50 0x7fb3f00051c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb40000e210 tx=0x7fb40000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53f59b0df35858c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:49.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.367+0000 7fb40559a700 1 -- 172.21.15.31:0/1332103693 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb40000ea60 con 0x7fb3f0004d50 2023-10-06T02:23:49.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.367+0000 7fb40559a700 1 -- 172.21.15.31:0/1332103693 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb40000f040 con 0x7fb3f0004d50 2023-10-06T02:23:49.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.367+0000 7fb40559a700 1 -- 172.21.15.31:0/1332103693 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb40000ee10 con 0x7fb3f0004d50 2023-10-06T02:23:49.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.368+0000 7fb40759e700 1 -- 172.21.15.31:0/1332103693 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3f0004d50 msgr2=0x7fb3f00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:49.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.368+0000 7fb40759e700 1 --2- 172.21.15.31:0/1332103693 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3f0004d50 0x7fb3f00051c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb40000e210 tx=0x7fb40000b040 comp rx=0 tx=0).stop 2023-10-06T02:23:49.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.368+0000 7fb406d9d700 1 -- 172.21.15.31:0/1332103693 reap_dead start 2023-10-06T02:23:49.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.368+0000 7fb40759e700 1 -- 172.21.15.31:0/1332103693 shutdown_connections 2023-10-06T02:23:49.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.368+0000 7fb40759e700 1 -- 172.21.15.31:0/1332103693 >> 172.21.15.31:0/1332103693 conn(0x7fb40806d110 msgr2=0x7fb4080cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:49.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.368+0000 7fb40759e700 1 -- 172.21.15.31:0/1332103693 shutdown_connections 2023-10-06T02:23:49.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.368+0000 7fb40759e700 1 -- 172.21.15.31:0/1332103693 wait complete. 2023-10-06T02:23:49.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.369+0000 7fb40759e700 1 Processor -- start 2023-10-06T02:23:49.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.369+0000 7fb40759e700 1 -- start start 2023-10-06T02:23:49.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.370+0000 7fb40759e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3f0004d50 0x7fb40813ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:49.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.370+0000 7fb40759e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb40813dd90 0x7fb40813e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:49.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.370+0000 7fb40759e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb4080c5440 0x7fb4080c5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:49.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.370+0000 7fb40759e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb40813e740 con 0x7fb3f0004d50 2023-10-06T02:23:49.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.370+0000 7fb40759e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4080af780 con 0x7fb40813dd90 2023-10-06T02:23:49.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.370+0000 7fb40759e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4080b0300 con 0x7fb4080c5440 2023-10-06T02:23:49.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.371+0000 7fb40659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3f0004d50 0x7fb40813ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:49.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.371+0000 7fb40659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3f0004d50 0x7fb40813ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38938/0 (socket says 172.21.15.31:38938) 2023-10-06T02:23:49.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.371+0000 7fb40659c700 1 -- 172.21.15.31:0/3367220255 learned_addr learned my addr 172.21.15.31:0/3367220255 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:49.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.371+0000 7fb406d9d700 1 --2- 172.21.15.31:0/3367220255 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb4080c5440 0x7fb4080c5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:49.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.371+0000 7fb405d9b700 1 --2- 172.21.15.31:0/3367220255 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb40813dd90 0x7fb40813e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:49.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.371+0000 7fb40659c700 1 -- 172.21.15.31:0/3367220255 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb4080c5440 msgr2=0x7fb4080c5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:49.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.371+0000 7fb40659c700 1 --2- 172.21.15.31:0/3367220255 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb4080c5440 0x7fb4080c5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:49.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.371+0000 7fb40659c700 1 -- 172.21.15.31:0/3367220255 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb40813dd90 msgr2=0x7fb40813e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:49.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.371+0000 7fb40659c700 1 --2- 172.21.15.31:0/3367220255 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb40813dd90 0x7fb40813e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:49.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.371+0000 7fb40659c700 1 -- 172.21.15.31:0/3367220255 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4080af9d0 con 0x7fb3f0004d50 2023-10-06T02:23:49.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.371+0000 7fb406d9d700 1 --2- 172.21.15.31:0/3367220255 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb4080c5440 0x7fb4080c5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:23:49.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.371+0000 7fb40659c700 1 --2- 172.21.15.31:0/3367220255 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3f0004d50 0x7fb40813ae00 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb3f8007fd0 tx=0x7fb40806be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:49.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.371+0000 7fb3f67fc700 1 -- 172.21.15.31:0/3367220255 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f800e7d0 con 0x7fb3f0004d50 2023-10-06T02:23:49.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.371+0000 7fb3f67fc700 1 -- 172.21.15.31:0/3367220255 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3f8010040 con 0x7fb3f0004d50 2023-10-06T02:23:49.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.371+0000 7fb3f67fc700 1 -- 172.21.15.31:0/3367220255 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f800eb40 con 0x7fb3f0004d50 2023-10-06T02:23:49.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.371+0000 7fb40759e700 1 -- 172.21.15.31:0/3367220255 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4080afcb0 con 0x7fb3f0004d50 2023-10-06T02:23:49.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.372+0000 7fb40759e700 1 -- 172.21.15.31:0/3367220255 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb40813b340 con 0x7fb3f0004d50 2023-10-06T02:23:49.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.372+0000 7fb40759e700 1 -- 172.21.15.31:0/3367220255 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4080cccc0 con 0x7fb3f0004d50 2023-10-06T02:23:49.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.373+0000 7fb3f67fc700 1 -- 172.21.15.31:0/3367220255 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb3f801d400 con 0x7fb3f0004d50 2023-10-06T02:23:49.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.375+0000 7fb3f67fc700 1 --2- 172.21.15.31:0/3367220255 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fb3e8066f30 0x7fb3e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:49.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.375+0000 7fb3f67fc700 1 -- 172.21.15.31:0/3367220255 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6116+0+0 (secure 0 0 0) 0x7fb3f80841f0 con 0x7fb3f0004d50 2023-10-06T02:23:49.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.375+0000 7fb405d9b700 1 --2- 172.21.15.31:0/3367220255 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fb3e8066f30 0x7fb3e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:49.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.375+0000 7fb405d9b700 1 --2- 172.21.15.31:0/3367220255 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fb3e8066f30 0x7fb3e80693e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb40816cd80 tx=0x7fb3fc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:49.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.378+0000 7fb3f67fc700 1 -- 172.21.15.31:0/3367220255 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb3f8051220 con 0x7fb3f0004d50 2023-10-06T02:23:49.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.578+0000 7fb40759e700 1 -- 172.21.15.31:0/3367220255 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb408140120 con 0x7fb3f0004d50 2023-10-06T02:23:49.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.578+0000 7fb3f67fc700 1 -- 172.21.15.31:0/3367220255 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb408140120 con 0x7fb3f0004d50 2023-10-06T02:23:49.580 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 8 2023-10-06T02:23:49.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.580+0000 7fb40759e700 1 -- 172.21.15.31:0/3367220255 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fb3e8066f30 msgr2=0x7fb3e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:49.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.580+0000 7fb40759e700 1 --2- 172.21.15.31:0/3367220255 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fb3e8066f30 0x7fb3e80693e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb40816cd80 tx=0x7fb3fc008040 comp rx=0 tx=0).stop 2023-10-06T02:23:49.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.580+0000 7fb40759e700 1 -- 172.21.15.31:0/3367220255 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3f0004d50 msgr2=0x7fb40813ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:49.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.580+0000 7fb40759e700 1 --2- 172.21.15.31:0/3367220255 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3f0004d50 0x7fb40813ae00 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb3f8007fd0 tx=0x7fb40806be10 comp rx=0 tx=0).stop 2023-10-06T02:23:49.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.580+0000 7fb40759e700 1 -- 172.21.15.31:0/3367220255 shutdown_connections 2023-10-06T02:23:49.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.580+0000 7fb40759e700 1 --2- 172.21.15.31:0/3367220255 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fb3e8066f30 0x7fb3e80693e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:49.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.580+0000 7fb40759e700 1 --2- 172.21.15.31:0/3367220255 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb4080c5440 0x7fb4080c5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:49.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.580+0000 7fb40759e700 1 --2- 172.21.15.31:0/3367220255 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3f0004d50 0x7fb40813ae00 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:49.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.580+0000 7fb40759e700 1 --2- 172.21.15.31:0/3367220255 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb40813dd90 0x7fb40813e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:49.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.580+0000 7fb40759e700 1 -- 172.21.15.31:0/3367220255 >> 172.21.15.31:0/3367220255 conn(0x7fb40806d110 msgr2=0x7fb4080ca000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:49.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.581+0000 7fb40759e700 1 -- 172.21.15.31:0/3367220255 shutdown_connections 2023-10-06T02:23:49.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.581+0000 7fb40759e700 1 -- 172.21.15.31:0/3367220255 wait complete. 2023-10-06T02:23:49.599 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T02:23:49.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.787+0000 7fc0cb59d700 1 Processor -- start 2023-10-06T02:23:49.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.788+0000 7fc0cb59d700 1 -- start start 2023-10-06T02:23:49.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.789+0000 7fc0cb59d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc0c415dfd0 0x7fc0c40aedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:49.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.789+0000 7fc0cb59d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc0c4159920 0x7fc0c40ad090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:49.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.789+0000 7fc0cb59d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0c40acc00 con 0x7fc0c4159550 2023-10-06T02:23:49.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.789+0000 7fc0cb59d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0c40abff0 con 0x7fc0c4159920 2023-10-06T02:23:49.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.789+0000 7fc0cb59d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c40ac2d0 con 0x7fc0c415dfd0 2023-10-06T02:23:49.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.789+0000 7fc0ca59b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fc0c4159550 0x7fc0c40ceac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49340/0 (socket says 172.21.15.31:49340) 2023-10-06T02:23:49.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.789+0000 7fc0ca59b700 1 -- 172.21.15.31:0/4101749328 learned_addr learned my addr 172.21.15.31:0/4101749328 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:23:49.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.789+0000 7fc0cad9c700 1 --2- 172.21.15.31:0/4101749328 >> v2:172.21.15.31:3300/0 conn(0x7fc0c4159920 0x7fc0c40ad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:49.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.789+0000 7fc0c9d9a700 1 --2- 172.21.15.31:0/4101749328 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc0c415dfd0 0x7fc0c40aedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:49.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.790+0000 7fc0cad9c700 1 -- 172.21.15.31:0/4101749328 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc0c415dfd0 msgr2=0x7fc0c40aedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:49.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.790+0000 7fc0cad9c700 1 --2- 172.21.15.31:0/4101749328 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc0c415dfd0 0x7fc0c40aedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:49.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.790+0000 7fc0c9599700 1 -- 172.21.15.31:0/4101749328 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1003803865 0 0) 0x7fc0c40acc00 con 0x7fc0c4159550 2023-10-06T02:23:49.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.790+0000 7fc0cad9c700 1 -- 172.21.15.31:0/4101749328 >> v1:172.21.15.31:6789/0 conn(0x7fc0c4159550 legacy=0x7fc0c40ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:49.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.790+0000 7fc0cad9c700 1 -- 172.21.15.31:0/4101749328 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0c40ac5b0 con 0x7fc0c4159920 2023-10-06T02:23:49.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.790+0000 7fc0cad9c700 1 --2- 172.21.15.31:0/4101749328 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0c4159920 0x7fc0c40ad090 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc009e60 tx=0x7fc0bc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be231bdeed6bfbaf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:49.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.790+0000 7fc0c9599700 1 -- 172.21.15.31:0/4101749328 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0bc00e070 con 0x7fc0c4159920 2023-10-06T02:23:49.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.790+0000 7fc0c9599700 1 -- 172.21.15.31:0/4101749328 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0bc00bc30 con 0x7fc0c4159920 2023-10-06T02:23:49.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.790+0000 7fc0c9599700 1 -- 172.21.15.31:0/4101749328 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0bc012430 con 0x7fc0c4159920 2023-10-06T02:23:49.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.791+0000 7fc0cb59d700 1 -- 172.21.15.31:0/4101749328 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0c4159920 msgr2=0x7fc0c40ad090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:49.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.791+0000 7fc0cb59d700 1 --2- 172.21.15.31:0/4101749328 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0c4159920 0x7fc0c40ad090 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc009e60 tx=0x7fc0bc009780 comp rx=0 tx=0).stop 2023-10-06T02:23:49.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.791+0000 7fc0cb59d700 1 -- 172.21.15.31:0/4101749328 shutdown_connections 2023-10-06T02:23:49.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.791+0000 7fc0cb59d700 1 --2- 172.21.15.31:0/4101749328 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0c4159920 0x7fc0c40ad090 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:49.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.791+0000 7fc0cb59d700 1 --2- 172.21.15.31:0/4101749328 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc0c415dfd0 0x7fc0c40aedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:49.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.791+0000 7fc0cb59d700 1 -- 172.21.15.31:0/4101749328 >> 172.21.15.31:0/4101749328 conn(0x7fc0c406ce80 msgr2=0x7fc0c40ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:49.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.791+0000 7fc0cb59d700 1 -- 172.21.15.31:0/4101749328 shutdown_connections 2023-10-06T02:23:49.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.791+0000 7fc0cb59d700 1 -- 172.21.15.31:0/4101749328 wait complete. 2023-10-06T02:23:49.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.792+0000 7fc0cb59d700 1 Processor -- start 2023-10-06T02:23:49.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.792+0000 7fc0cb59d700 1 -- start start 2023-10-06T02:23:49.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.793+0000 7fc0cb59d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0c4159550 0x7fc0c40c9820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:49.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.793+0000 7fc0cb59d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc0c4159920 0x7fc0c406d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:49.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.793+0000 7fc0cb59d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0c415dfd0 0x7fc0c406dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:49.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.793+0000 7fc0cb59d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c416d720 con 0x7fc0c415dfd0 2023-10-06T02:23:49.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.793+0000 7fc0cb59d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c4150030 con 0x7fc0c4159550 2023-10-06T02:23:49.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.793+0000 7fc0cb59d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c416a640 con 0x7fc0c4159920 2023-10-06T02:23:49.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.794+0000 7fc0cad9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0c415dfd0 0x7fc0c406dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:49.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.794+0000 7fc0cad9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0c415dfd0 0x7fc0c406dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38948/0 (socket says 172.21.15.31:38948) 2023-10-06T02:23:49.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.794+0000 7fc0cad9c700 1 -- 172.21.15.31:0/552899243 learned_addr learned my addr 172.21.15.31:0/552899243 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:49.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.794+0000 7fc0ca59b700 1 --2- 172.21.15.31:0/552899243 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0c4159550 0x7fc0c40c9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:49.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.794+0000 7fc0c9d9a700 1 --2- 172.21.15.31:0/552899243 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc0c4159920 0x7fc0c406d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:49.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.794+0000 7fc0cad9c700 1 -- 172.21.15.31:0/552899243 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc0c4159920 msgr2=0x7fc0c406d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:49.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.794+0000 7fc0cad9c700 1 --2- 172.21.15.31:0/552899243 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc0c4159920 0x7fc0c406d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:49.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.794+0000 7fc0cad9c700 1 -- 172.21.15.31:0/552899243 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0c4159550 msgr2=0x7fc0c40c9820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:49.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.794+0000 7fc0cad9c700 1 --2- 172.21.15.31:0/552899243 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0c4159550 0x7fc0c40c9820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:49.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.794+0000 7fc0cad9c700 1 -- 172.21.15.31:0/552899243 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0bc009a20 con 0x7fc0c415dfd0 2023-10-06T02:23:49.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.794+0000 7fc0cad9c700 1 --2- 172.21.15.31:0/552899243 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0c415dfd0 0x7fc0c406dcf0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc013000 tx=0x7fc0bc019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:49.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.795+0000 7fc0ba7fc700 1 -- 172.21.15.31:0/552899243 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0bc027470 con 0x7fc0c415dfd0 2023-10-06T02:23:49.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.795+0000 7fc0ba7fc700 1 -- 172.21.15.31:0/552899243 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0bc027610 con 0x7fc0c415dfd0 2023-10-06T02:23:49.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.795+0000 7fc0ba7fc700 1 -- 172.21.15.31:0/552899243 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0bc0279d0 con 0x7fc0c415dfd0 2023-10-06T02:23:49.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.795+0000 7fc0cb59d700 1 -- 172.21.15.31:0/552899243 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0c416a920 con 0x7fc0c415dfd0 2023-10-06T02:23:49.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.795+0000 7fc0cb59d700 1 -- 172.21.15.31:0/552899243 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0c416d3b0 con 0x7fc0c415dfd0 2023-10-06T02:23:49.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.796+0000 7fc0cb59d700 1 -- 172.21.15.31:0/552899243 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0c404c3c0 con 0x7fc0c415dfd0 2023-10-06T02:23:49.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.797+0000 7fc0ba7fc700 1 -- 172.21.15.31:0/552899243 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc0bc024070 con 0x7fc0c415dfd0 2023-10-06T02:23:49.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.798+0000 7fc0ba7fc700 1 --2- 172.21.15.31:0/552899243 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc0a8066f30 0x7fc0a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:49.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.798+0000 7fc0ba7fc700 1 -- 172.21.15.31:0/552899243 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6116+0+0 (secure 0 0 0) 0x7fc0bc086e00 con 0x7fc0c415dfd0 2023-10-06T02:23:49.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.798+0000 7fc0ca59b700 1 --2- 172.21.15.31:0/552899243 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc0a8066f30 0x7fc0a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:49.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.799+0000 7fc0ca59b700 1 --2- 172.21.15.31:0/552899243 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc0a8066f30 0x7fc0a80693e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc0c41538f0 tx=0x7fc0c0007d30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:49.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:49.802+0000 7fc0ba7fc700 1 -- 172.21.15.31:0/552899243 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc0bc053ed0 con 0x7fc0c415dfd0 2023-10-06T02:23:50.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.008+0000 7fc0cb59d700 1 -- 172.21.15.31:0/552899243 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc0c4155780 con 0x7fc0c415dfd0 2023-10-06T02:23:50.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.009+0000 7fc0ba7fc700 1 -- 172.21.15.31:0/552899243 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc0bc0574f0 con 0x7fc0c415dfd0 2023-10-06T02:23:50.010 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-10-06T02:23:50.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.011+0000 7fc0cb59d700 1 -- 172.21.15.31:0/552899243 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc0a8066f30 msgr2=0x7fc0a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:50.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.011+0000 7fc0cb59d700 1 --2- 172.21.15.31:0/552899243 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc0a8066f30 0x7fc0a80693e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc0c41538f0 tx=0x7fc0c0007d30 comp rx=0 tx=0).stop 2023-10-06T02:23:50.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.011+0000 7fc0cb59d700 1 -- 172.21.15.31:0/552899243 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0c415dfd0 msgr2=0x7fc0c406dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:50.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.011+0000 7fc0cb59d700 1 --2- 172.21.15.31:0/552899243 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0c415dfd0 0x7fc0c406dcf0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc013000 tx=0x7fc0bc019000 comp rx=0 tx=0).stop 2023-10-06T02:23:50.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.011+0000 7fc0cb59d700 1 -- 172.21.15.31:0/552899243 shutdown_connections 2023-10-06T02:23:50.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.011+0000 7fc0cb59d700 1 --2- 172.21.15.31:0/552899243 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0c415dfd0 0x7fc0c406dcf0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:50.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.011+0000 7fc0cb59d700 1 --2- 172.21.15.31:0/552899243 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0c4159550 0x7fc0c40c9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:50.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.011+0000 7fc0cb59d700 1 --2- 172.21.15.31:0/552899243 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc0a8066f30 0x7fc0a80693e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:50.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.011+0000 7fc0cb59d700 1 --2- 172.21.15.31:0/552899243 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc0c4159920 0x7fc0c406d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:50.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.011+0000 7fc0cb59d700 1 -- 172.21.15.31:0/552899243 >> 172.21.15.31:0/552899243 conn(0x7fc0c406ce80 msgr2=0x7fc0c4154e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:50.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.011+0000 7fc0cb59d700 1 -- 172.21.15.31:0/552899243 shutdown_connections 2023-10-06T02:23:50.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.011+0000 7fc0cb59d700 1 -- 172.21.15.31:0/552899243 wait complete. 2023-10-06T02:23:50.030 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T02:23:50.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.219+0000 7fcf4759e700 1 Processor -- start 2023-10-06T02:23:50.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.219+0000 7fcf4759e700 1 -- start start 2023-10-06T02:23:50.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.221+0000 7fcf4759e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcf48153c90 0x7fcf4815a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:50.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.221+0000 7fcf4759e700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf4815ada0 con 0x7fcf480caec0 2023-10-06T02:23:50.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.221+0000 7fcf4759e700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf4806ca80 con 0x7fcf4806db20 2023-10-06T02:23:50.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.221+0000 7fcf4759e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf4806cd60 con 0x7fcf48153c90 2023-10-06T02:23:50.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.221+0000 7fcf46d9d700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fcf480caec0 0x7fcf4815afb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49354/0 (socket says 172.21.15.31:49354) 2023-10-06T02:23:50.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.221+0000 7fcf46d9d700 1 -- 172.21.15.31:0/2162727975 learned_addr learned my addr 172.21.15.31:0/2162727975 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:23:50.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.221+0000 7fcf4659c700 1 --2- 172.21.15.31:0/2162727975 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcf48153c90 0x7fcf4815a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:50.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.221+0000 7fcf4559a700 1 -- 172.21.15.31:0/2162727975 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1810866111 0 0) 0x7fcf40001490 con 0x7fcf480caec0 2023-10-06T02:23:50.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.221+0000 7fcf4559a700 1 -- 172.21.15.31:0/2162727975 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf30003570 con 0x7fcf480caec0 2023-10-06T02:23:50.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.222+0000 7fcf4559a700 1 -- 172.21.15.31:0/2162727975 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2089377635 0 0) 0x7fcf4806ca80 con 0x7fcf4806db20 2023-10-06T02:23:50.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.222+0000 7fcf4559a700 1 -- 172.21.15.31:0/2162727975 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf40001490 con 0x7fcf4806db20 2023-10-06T02:23:50.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.222+0000 7fcf4659c700 1 -- 172.21.15.31:0/2162727975 >> v1:172.21.15.62:6789/0 conn(0x7fcf4806db20 legacy=0x7fcf480ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:50.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.222+0000 7fcf4659c700 1 -- 172.21.15.31:0/2162727975 >> v1:172.21.15.31:6789/0 conn(0x7fcf480caec0 legacy=0x7fcf4815afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:50.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.222+0000 7fcf4659c700 1 -- 172.21.15.31:0/2162727975 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf4806d040 con 0x7fcf48153c90 2023-10-06T02:23:50.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.222+0000 7fcf4659c700 1 --2- 172.21.15.31:0/2162727975 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcf48153c90 0x7fcf4815a860 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fcf38009910 tx=0x7fcf380091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62b2afeaf7adc86a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:50.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.222+0000 7fcf4559a700 1 -- 172.21.15.31:0/2162727975 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf38011630 con 0x7fcf48153c90 2023-10-06T02:23:50.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.222+0000 7fcf4559a700 1 -- 172.21.15.31:0/2162727975 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf38011790 con 0x7fcf48153c90 2023-10-06T02:23:50.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.222+0000 7fcf4559a700 1 -- 172.21.15.31:0/2162727975 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf38011b30 con 0x7fcf48153c90 2023-10-06T02:23:50.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.223+0000 7fcf4759e700 1 -- 172.21.15.31:0/2162727975 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcf48153c90 msgr2=0x7fcf4815a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:50.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.223+0000 7fcf4759e700 1 --2- 172.21.15.31:0/2162727975 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcf48153c90 0x7fcf4815a860 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fcf38009910 tx=0x7fcf380091a0 comp rx=0 tx=0).stop 2023-10-06T02:23:50.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.223+0000 7fcf4759e700 1 -- 172.21.15.31:0/2162727975 shutdown_connections 2023-10-06T02:23:50.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.223+0000 7fcf4759e700 1 --2- 172.21.15.31:0/2162727975 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcf48153c90 0x7fcf4815a860 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:50.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.223+0000 7fcf4759e700 1 -- 172.21.15.31:0/2162727975 >> 172.21.15.31:0/2162727975 conn(0x7fcf4806e720 msgr2=0x7fcf4806eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:50.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.223+0000 7fcf4759e700 1 -- 172.21.15.31:0/2162727975 shutdown_connections 2023-10-06T02:23:50.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.223+0000 7fcf4759e700 1 -- 172.21.15.31:0/2162727975 wait complete. 2023-10-06T02:23:50.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.224+0000 7fcf4759e700 1 Processor -- start 2023-10-06T02:23:50.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.225+0000 7fcf4759e700 1 -- start start 2023-10-06T02:23:50.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.225+0000 7fcf4759e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf4806db20 0x7fcf48151770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:50.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.225+0000 7fcf4759e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcf480caec0 0x7fcf480b01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:50.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.225+0000 7fcf4759e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcf48153c90 0x7fcf48152410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:50.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.225+0000 7fcf4759e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf480c3a80 con 0x7fcf4806db20 2023-10-06T02:23:50.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.225+0000 7fcf4759e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf480c3fd0 con 0x7fcf48153c90 2023-10-06T02:23:50.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.225+0000 7fcf4759e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf48151cb0 con 0x7fcf480caec0 2023-10-06T02:23:50.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.226+0000 7fcf4659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf4806db20 0x7fcf48151770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:50.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.226+0000 7fcf4659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf4806db20 0x7fcf48151770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38950/0 (socket says 172.21.15.31:38950) 2023-10-06T02:23:50.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.226+0000 7fcf4659c700 1 -- 172.21.15.31:0/221772360 learned_addr learned my addr 172.21.15.31:0/221772360 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:50.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.226+0000 7fcf45d9b700 1 --2- 172.21.15.31:0/221772360 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcf480caec0 0x7fcf480b01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:50.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.226+0000 7fcf46d9d700 1 --2- 172.21.15.31:0/221772360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcf48153c90 0x7fcf48152410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:50.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.226+0000 7fcf4659c700 1 -- 172.21.15.31:0/221772360 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcf480caec0 msgr2=0x7fcf480b01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:50.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.226+0000 7fcf4659c700 1 --2- 172.21.15.31:0/221772360 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcf480caec0 0x7fcf480b01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:50.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.226+0000 7fcf4659c700 1 -- 172.21.15.31:0/221772360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcf48153c90 msgr2=0x7fcf48152410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:50.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.226+0000 7fcf4659c700 1 --2- 172.21.15.31:0/221772360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcf48153c90 0x7fcf48152410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:50.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.226+0000 7fcf4659c700 1 -- 172.21.15.31:0/221772360 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf38009420 con 0x7fcf4806db20 2023-10-06T02:23:50.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.226+0000 7fcf4659c700 1 --2- 172.21.15.31:0/221772360 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf4806db20 0x7fcf48151770 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fcf38009f00 tx=0x7fcf38017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:50.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.227+0000 7fcf367fc700 1 -- 172.21.15.31:0/221772360 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf3801f070 con 0x7fcf4806db20 2023-10-06T02:23:50.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.227+0000 7fcf367fc700 1 -- 172.21.15.31:0/221772360 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf38000bc0 con 0x7fcf4806db20 2023-10-06T02:23:50.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.227+0000 7fcf367fc700 1 -- 172.21.15.31:0/221772360 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf38025420 con 0x7fcf4806db20 2023-10-06T02:23:50.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.227+0000 7fcf4759e700 1 -- 172.21.15.31:0/221772360 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf480b12d0 con 0x7fcf4806db20 2023-10-06T02:23:50.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.227+0000 7fcf4759e700 1 -- 172.21.15.31:0/221772360 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf480b1890 con 0x7fcf4806db20 2023-10-06T02:23:50.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.228+0000 7fcf4759e700 1 -- 172.21.15.31:0/221772360 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf4806f240 con 0x7fcf4806db20 2023-10-06T02:23:50.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.229+0000 7fcf367fc700 1 -- 172.21.15.31:0/221772360 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcf38023070 con 0x7fcf4806db20 2023-10-06T02:23:50.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.230+0000 7fcf367fc700 1 --2- 172.21.15.31:0/221772360 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fcf28066f30 0x7fcf280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:50.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.230+0000 7fcf367fc700 1 -- 172.21.15.31:0/221772360 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6116+0+0 (secure 0 0 0) 0x7fcf380859f0 con 0x7fcf4806db20 2023-10-06T02:23:50.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.230+0000 7fcf45d9b700 1 --2- 172.21.15.31:0/221772360 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fcf28066f30 0x7fcf280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:50.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.230+0000 7fcf45d9b700 1 --2- 172.21.15.31:0/221772360 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fcf28066f30 0x7fcf280693e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fcf48152f70 tx=0x7fcf3c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:50.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.234+0000 7fcf367fc700 1 -- 172.21.15.31:0/221772360 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcf38052b00 con 0x7fcf4806db20 2023-10-06T02:23:50.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.434+0000 7fcf4759e700 1 -- 172.21.15.31:0/221772360 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcf480b1a80 con 0x7fcf4806db20 2023-10-06T02:23:50.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.434+0000 7fcf367fc700 1 -- 172.21.15.31:0/221772360 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcf3802a0a0 con 0x7fcf4806db20 2023-10-06T02:23:50.436 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-10-06T02:23:50.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.437+0000 7fcf4759e700 1 -- 172.21.15.31:0/221772360 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fcf28066f30 msgr2=0x7fcf280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:50.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.437+0000 7fcf4759e700 1 --2- 172.21.15.31:0/221772360 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fcf28066f30 0x7fcf280693e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fcf48152f70 tx=0x7fcf3c009040 comp rx=0 tx=0).stop 2023-10-06T02:23:50.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.437+0000 7fcf4759e700 1 -- 172.21.15.31:0/221772360 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf4806db20 msgr2=0x7fcf48151770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:50.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.437+0000 7fcf4759e700 1 --2- 172.21.15.31:0/221772360 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf4806db20 0x7fcf48151770 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fcf38009f00 tx=0x7fcf38017000 comp rx=0 tx=0).stop 2023-10-06T02:23:50.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.437+0000 7fcf4759e700 1 -- 172.21.15.31:0/221772360 shutdown_connections 2023-10-06T02:23:50.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.437+0000 7fcf4759e700 1 --2- 172.21.15.31:0/221772360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcf48153c90 0x7fcf48152410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:50.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.437+0000 7fcf4759e700 1 --2- 172.21.15.31:0/221772360 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf4806db20 0x7fcf48151770 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:50.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.437+0000 7fcf4759e700 1 --2- 172.21.15.31:0/221772360 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fcf28066f30 0x7fcf280693e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:50.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.437+0000 7fcf4759e700 1 --2- 172.21.15.31:0/221772360 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcf480caec0 0x7fcf480b01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:50.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.437+0000 7fcf4759e700 1 -- 172.21.15.31:0/221772360 >> 172.21.15.31:0/221772360 conn(0x7fcf4806e720 msgr2=0x7fcf48159de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:50.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.437+0000 7fcf4759e700 1 -- 172.21.15.31:0/221772360 shutdown_connections 2023-10-06T02:23:50.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.437+0000 7fcf4759e700 1 -- 172.21.15.31:0/221772360 wait complete. 2023-10-06T02:23:50.455 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:23:50.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.641+0000 7fc775d9d700 1 Processor -- start 2023-10-06T02:23:50.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.641+0000 7fc775d9d700 1 -- start start 2023-10-06T02:23:50.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.642+0000 7fc775d9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc77013fb20 0x7fc77015a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:50.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.642+0000 7fc775d9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc77006aaa0 0x7fc7700c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:50.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.642+0000 7fc775d9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc77015ae70 con 0x7fc77013fef0 2023-10-06T02:23:50.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.642+0000 7fc775d9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc77006af60 con 0x7fc77013fb20 2023-10-06T02:23:50.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.642+0000 7fc775d9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc77006b240 con 0x7fc77006aaa0 2023-10-06T02:23:50.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.643+0000 7fc774d9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc77013fb20 0x7fc77015a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:50.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.643+0000 7fc774d9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc77013fb20 0x7fc77015a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38962/0 (socket says 172.21.15.31:38962) 2023-10-06T02:23:50.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.643+0000 7fc774d9b700 1 -- 172.21.15.31:0/3064370439 learned_addr learned my addr 172.21.15.31:0/3064370439 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:50.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.643+0000 7fc76ffff700 1 --1- 172.21.15.31:0/3064370439 >> v1:172.21.15.31:6789/0 conn(0x7fc77013fef0 0x7fc7700c7990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49370/0 (socket says 172.21.15.31:49370) 2023-10-06T02:23:50.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.643+0000 7fc77559c700 1 --2- 172.21.15.31:0/3064370439 >> v2:172.21.15.62:3300/0 conn(0x7fc77006aaa0 0x7fc7700c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:50.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.643+0000 7fc774d9b700 1 -- 172.21.15.31:0/3064370439 >> v2:172.21.15.62:3300/0 conn(0x7fc77006aaa0 msgr2=0x7fc7700c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:50.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.643+0000 7fc774d9b700 1 --2- 172.21.15.31:0/3064370439 >> v2:172.21.15.62:3300/0 conn(0x7fc77006aaa0 0x7fc7700c9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:50.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.643+0000 7fc774d9b700 1 -- 172.21.15.31:0/3064370439 >> v1:172.21.15.31:6789/0 conn(0x7fc77013fef0 legacy=0x7fc7700c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:50.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.643+0000 7fc774d9b700 1 -- 172.21.15.31:0/3064370439 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc77006b520 con 0x7fc77013fb20 2023-10-06T02:23:50.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.643+0000 7fc76f7fe700 1 -- 172.21.15.31:0/3064370439 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2677593090 0 0) 0x7fc75c0011d0 con 0x7fc77013fef0 2023-10-06T02:23:50.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.644+0000 7fc774d9b700 1 --2- 172.21.15.31:0/3064370439 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc77013fb20 0x7fc77015a930 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc770069d40 tx=0x7fc764009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a9d1a29d5ced428 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:50.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.644+0000 7fc76f7fe700 1 -- 172.21.15.31:0/3064370439 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc764009e70 con 0x7fc77013fb20 2023-10-06T02:23:50.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.644+0000 7fc76f7fe700 1 -- 172.21.15.31:0/3064370439 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc76400b860 con 0x7fc77013fb20 2023-10-06T02:23:50.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.644+0000 7fc76f7fe700 1 -- 172.21.15.31:0/3064370439 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc76400bc00 con 0x7fc77013fb20 2023-10-06T02:23:50.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.644+0000 7fc775d9d700 1 -- 172.21.15.31:0/3064370439 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc77013fb20 msgr2=0x7fc77015a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:50.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.644+0000 7fc775d9d700 1 --2- 172.21.15.31:0/3064370439 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc77013fb20 0x7fc77015a930 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc770069d40 tx=0x7fc764009250 comp rx=0 tx=0).stop 2023-10-06T02:23:50.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.644+0000 7fc775d9d700 1 -- 172.21.15.31:0/3064370439 shutdown_connections 2023-10-06T02:23:50.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.644+0000 7fc775d9d700 1 --2- 172.21.15.31:0/3064370439 >> v2:172.21.15.62:3300/0 conn(0x7fc77006aaa0 0x7fc7700c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:50.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.644+0000 7fc775d9d700 1 --2- 172.21.15.31:0/3064370439 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc77013fb20 0x7fc77015a930 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:50.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.645+0000 7fc775d9d700 1 -- 172.21.15.31:0/3064370439 >> 172.21.15.31:0/3064370439 conn(0x7fc77006d170 msgr2=0x7fc7700cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:50.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.645+0000 7fc775d9d700 1 -- 172.21.15.31:0/3064370439 shutdown_connections 2023-10-06T02:23:50.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.645+0000 7fc775d9d700 1 -- 172.21.15.31:0/3064370439 wait complete. 2023-10-06T02:23:50.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.646+0000 7fc775d9d700 1 Processor -- start 2023-10-06T02:23:50.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.646+0000 7fc775d9d700 1 -- start start 2023-10-06T02:23:50.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.647+0000 7fc775d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc77006aaa0 0x7fc7700abe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:50.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.647+0000 7fc775d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc77013fb20 0x7fc77015f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:50.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.647+0000 7fc775d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc77013fef0 0x7fc77015f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:50.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.647+0000 7fc775d9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc770157b20 con 0x7fc77013fb20 2023-10-06T02:23:50.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.647+0000 7fc775d9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7700aea20 con 0x7fc77006aaa0 2023-10-06T02:23:50.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.647+0000 7fc775d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7700ae740 con 0x7fc77013fef0 2023-10-06T02:23:50.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.647+0000 7fc76ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc77013fb20 0x7fc77015f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:50.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.647+0000 7fc76ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc77013fb20 0x7fc77015f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38970/0 (socket says 172.21.15.31:38970) 2023-10-06T02:23:50.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.647+0000 7fc76ffff700 1 -- 172.21.15.31:0/3476162812 learned_addr learned my addr 172.21.15.31:0/3476162812 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:50.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.647+0000 7fc77559c700 1 --2- 172.21.15.31:0/3476162812 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc77013fef0 0x7fc77015f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:50.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.648+0000 7fc774d9b700 1 --2- 172.21.15.31:0/3476162812 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc77006aaa0 0x7fc7700abe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:50.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.648+0000 7fc76ffff700 1 -- 172.21.15.31:0/3476162812 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc77013fef0 msgr2=0x7fc77015f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:50.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.648+0000 7fc76ffff700 1 --2- 172.21.15.31:0/3476162812 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc77013fef0 0x7fc77015f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:50.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.648+0000 7fc76ffff700 1 -- 172.21.15.31:0/3476162812 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc77006aaa0 msgr2=0x7fc7700abe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:50.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.648+0000 7fc76ffff700 1 --2- 172.21.15.31:0/3476162812 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc77006aaa0 0x7fc7700abe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:50.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.648+0000 7fc76ffff700 1 -- 172.21.15.31:0/3476162812 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7640094f0 con 0x7fc77013fb20 2023-10-06T02:23:50.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.648+0000 7fc774d9b700 1 --2- 172.21.15.31:0/3476162812 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc77006aaa0 0x7fc7700abe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:23:50.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.648+0000 7fc76ffff700 1 --2- 172.21.15.31:0/3476162812 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc77013fb20 0x7fc77015f2d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc75c00c5e0 tx=0x7fc75c007ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:50.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.648+0000 7fc76cff9700 1 -- 172.21.15.31:0/3476162812 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc75c0072e0 con 0x7fc77013fb20 2023-10-06T02:23:50.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.648+0000 7fc76cff9700 1 -- 172.21.15.31:0/3476162812 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc75c007460 con 0x7fc77013fb20 2023-10-06T02:23:50.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.648+0000 7fc76cff9700 1 -- 172.21.15.31:0/3476162812 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc75c0057f0 con 0x7fc77013fb20 2023-10-06T02:23:50.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.648+0000 7fc775d9d700 1 -- 172.21.15.31:0/3476162812 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7701574d0 con 0x7fc77013fb20 2023-10-06T02:23:50.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.648+0000 7fc775d9d700 1 -- 172.21.15.31:0/3476162812 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7700ac3a0 con 0x7fc77013fb20 2023-10-06T02:23:50.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.649+0000 7fc775d9d700 1 -- 172.21.15.31:0/3476162812 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc770157cd0 con 0x7fc77013fb20 2023-10-06T02:23:50.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.650+0000 7fc76cff9700 1 -- 172.21.15.31:0/3476162812 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc75c005bc0 con 0x7fc77013fb20 2023-10-06T02:23:50.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.651+0000 7fc76cff9700 1 --2- 172.21.15.31:0/3476162812 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc754066f30 0x7fc7540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:50.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.651+0000 7fc76cff9700 1 -- 172.21.15.31:0/3476162812 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6116+0+0 (secure 0 0 0) 0x7fc75c083a50 con 0x7fc77013fb20 2023-10-06T02:23:50.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.651+0000 7fc774d9b700 1 --2- 172.21.15.31:0/3476162812 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc754066f30 0x7fc7540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:50.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.652+0000 7fc774d9b700 1 --2- 172.21.15.31:0/3476162812 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc754066f30 0x7fc7540693e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc770069d40 tx=0x7fc764017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:50.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.655+0000 7fc76cff9700 1 -- 172.21.15.31:0/3476162812 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc770157cd0 con 0x7fc77013fb20 2023-10-06T02:23:50.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.858+0000 7fc775d9d700 1 -- 172.21.15.31:0/3476162812 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc77015ae70 con 0x7fc77013fb20 2023-10-06T02:23:50.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.859+0000 7fc76cff9700 1 -- 172.21.15.31:0/3476162812 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+16914 (secure 0 0 0) 0x7fc77015ae70 con 0x7fc77013fb20 2023-10-06T02:23:50.862 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:23:50.862 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":25,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:23:48.196317+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:23:07.290269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T02:23:08.504679+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.2f","mappings":[{"from":3,"to":2}]},{"pgid":"4.31","mappings":[{"from":1,"to":5}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:23:50.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.861+0000 7fc775d9d700 1 -- 172.21.15.31:0/3476162812 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc754066f30 msgr2=0x7fc7540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:50.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.861+0000 7fc775d9d700 1 --2- 172.21.15.31:0/3476162812 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc754066f30 0x7fc7540693e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc770069d40 tx=0x7fc764017000 comp rx=0 tx=0).stop 2023-10-06T02:23:50.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.861+0000 7fc775d9d700 1 -- 172.21.15.31:0/3476162812 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc77013fb20 msgr2=0x7fc77015f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:50.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.861+0000 7fc775d9d700 1 --2- 172.21.15.31:0/3476162812 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc77013fb20 0x7fc77015f2d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc75c00c5e0 tx=0x7fc75c007ee0 comp rx=0 tx=0).stop 2023-10-06T02:23:50.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.861+0000 7fc775d9d700 1 -- 172.21.15.31:0/3476162812 shutdown_connections 2023-10-06T02:23:50.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.861+0000 7fc775d9d700 1 --2- 172.21.15.31:0/3476162812 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc754066f30 0x7fc7540693e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:50.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.861+0000 7fc775d9d700 1 --2- 172.21.15.31:0/3476162812 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc77013fef0 0x7fc77015f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:50.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.861+0000 7fc775d9d700 1 --2- 172.21.15.31:0/3476162812 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc77006aaa0 0x7fc7700abe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:50.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.861+0000 7fc775d9d700 1 --2- 172.21.15.31:0/3476162812 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc77013fb20 0x7fc77015f2d0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:50.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.861+0000 7fc775d9d700 1 -- 172.21.15.31:0/3476162812 >> 172.21.15.31:0/3476162812 conn(0x7fc77006d170 msgr2=0x7fc770136930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:50.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.861+0000 7fc775d9d700 1 -- 172.21.15.31:0/3476162812 shutdown_connections 2023-10-06T02:23:50.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:50.861+0000 7fc775d9d700 1 -- 172.21.15.31:0/3476162812 wait complete. 2023-10-06T02:23:50.880 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T02:22:53.183425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T02:23:00.046967+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T02:23:07.290269+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T02:23:08.504679+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T02:23:50.880 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T02:23:51.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.071+0000 7fe102f9d700 1 Processor -- start 2023-10-06T02:23:51.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.071+0000 7fe102f9d700 1 -- start start 2023-10-06T02:23:51.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.072+0000 7fe102f9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe0fc153a00 0x7fe0fc153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:51.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.072+0000 7fe102f9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe0fc15de50 0x7fe0fc15a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:51.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.072+0000 7fe102f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe0fc0cea00 0x7fe0fc0cee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:51.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.072+0000 7fe102f9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0fc0cf3c0 con 0x7fe0fc15de50 2023-10-06T02:23:51.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.072+0000 7fe102f9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0fc15af60 con 0x7fe0fc153a00 2023-10-06T02:23:51.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.072+0000 7fe102f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0fc15b240 con 0x7fe0fc0cea00 2023-10-06T02:23:51.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.073+0000 7fe10179a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe0fc15de50 0x7fe0fc15a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:51.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.073+0000 7fe10179a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe0fc15de50 0x7fe0fc15a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38976/0 (socket says 172.21.15.31:38976) 2023-10-06T02:23:51.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.073+0000 7fe10179a700 1 -- 172.21.15.31:0/2233411431 learned_addr learned my addr 172.21.15.31:0/2233411431 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:51.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.073+0000 7fe101f9b700 1 --2- 172.21.15.31:0/2233411431 >> v2:172.21.15.62:3300/0 conn(0x7fe0fc153a00 0x7fe0fc153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:51.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.073+0000 7fe10279c700 1 --2- 172.21.15.31:0/2233411431 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe0fc0cea00 0x7fe0fc0cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:51.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.073+0000 7fe10179a700 1 -- 172.21.15.31:0/2233411431 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe0fc0cea00 msgr2=0x7fe0fc0cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:51.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.073+0000 7fe10179a700 1 --2- 172.21.15.31:0/2233411431 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe0fc0cea00 0x7fe0fc0cee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:51.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.073+0000 7fe10179a700 1 -- 172.21.15.31:0/2233411431 >> v2:172.21.15.62:3300/0 conn(0x7fe0fc153a00 msgr2=0x7fe0fc153dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:51.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.073+0000 7fe10179a700 1 --2- 172.21.15.31:0/2233411431 >> v2:172.21.15.62:3300/0 conn(0x7fe0fc153a00 0x7fe0fc153dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:51.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.073+0000 7fe10179a700 1 -- 172.21.15.31:0/2233411431 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0fc15b520 con 0x7fe0fc15de50 2023-10-06T02:23:51.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.073+0000 7fe101f9b700 1 --2- 172.21.15.31:0/2233411431 >> v2:172.21.15.62:3300/0 conn(0x7fe0fc153a00 0x7fe0fc153dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:23:51.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.073+0000 7fe10179a700 1 --2- 172.21.15.31:0/2233411431 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe0fc15de50 0x7fe0fc15a540 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc0cac20 tx=0x7fe0ec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac82a012b5ae7afd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:51.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.074+0000 7fe100f99700 1 -- 172.21.15.31:0/2233411431 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0ec00b860 con 0x7fe0fc15de50 2023-10-06T02:23:51.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.074+0000 7fe100f99700 1 -- 172.21.15.31:0/2233411431 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0ec00b9c0 con 0x7fe0fc15de50 2023-10-06T02:23:51.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.074+0000 7fe100f99700 1 -- 172.21.15.31:0/2233411431 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0ec00bd60 con 0x7fe0fc15de50 2023-10-06T02:23:51.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.074+0000 7fe102f9d700 1 -- 172.21.15.31:0/2233411431 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe0fc15de50 msgr2=0x7fe0fc15a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:51.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.074+0000 7fe102f9d700 1 --2- 172.21.15.31:0/2233411431 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe0fc15de50 0x7fe0fc15a540 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc0cac20 tx=0x7fe0ec009250 comp rx=0 tx=0).stop 2023-10-06T02:23:51.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.074+0000 7fe102f9d700 1 -- 172.21.15.31:0/2233411431 shutdown_connections 2023-10-06T02:23:51.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.074+0000 7fe102f9d700 1 --2- 172.21.15.31:0/2233411431 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe0fc0cea00 0x7fe0fc0cee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:51.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.074+0000 7fe102f9d700 1 --2- 172.21.15.31:0/2233411431 >> v2:172.21.15.62:3300/0 conn(0x7fe0fc153a00 0x7fe0fc153dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:51.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.074+0000 7fe102f9d700 1 --2- 172.21.15.31:0/2233411431 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe0fc15de50 0x7fe0fc15a540 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:51.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.074+0000 7fe102f9d700 1 -- 172.21.15.31:0/2233411431 >> 172.21.15.31:0/2233411431 conn(0x7fe0fc0703a0 msgr2=0x7fe0fc0707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:51.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.074+0000 7fe102f9d700 1 -- 172.21.15.31:0/2233411431 shutdown_connections 2023-10-06T02:23:51.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.074+0000 7fe102f9d700 1 -- 172.21.15.31:0/2233411431 wait complete. 2023-10-06T02:23:51.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.076+0000 7fe102f9d700 1 Processor -- start 2023-10-06T02:23:51.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.076+0000 7fe102f9d700 1 -- start start 2023-10-06T02:23:51.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.077+0000 7fe102f9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe0fc0cea00 0x7fe0fc0c8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:51.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.077+0000 7fe102f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe0fc153a00 0x7fe0fc0cbd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:51.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.077+0000 7fe102f9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0fc15de50 0x7fe0fc150ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:51.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.077+0000 7fe102f9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0fc0cc240 con 0x7fe0fc0cea00 2023-10-06T02:23:51.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.077+0000 7fe102f9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0fc1513e0 con 0x7fe0fc15de50 2023-10-06T02:23:51.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.077+0000 7fe102f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0fc0cc470 con 0x7fe0fc153a00 2023-10-06T02:23:51.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.077+0000 7fe101f9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe0fc0cea00 0x7fe0fc0c8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:51.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.077+0000 7fe101f9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe0fc0cea00 0x7fe0fc0c8180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38982/0 (socket says 172.21.15.31:38982) 2023-10-06T02:23:51.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.077+0000 7fe101f9b700 1 -- 172.21.15.31:0/3539072927 learned_addr learned my addr 172.21.15.31:0/3539072927 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:51.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.077+0000 7fe101f9b700 1 -- 172.21.15.31:0/3539072927 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe0fc153a00 msgr2=0x7fe0fc0cbd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:51.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.077+0000 7fe10179a700 1 --2- 172.21.15.31:0/3539072927 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe0fc153a00 0x7fe0fc0cbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:51.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.078+0000 7fe10279c700 1 --2- 172.21.15.31:0/3539072927 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0fc15de50 0x7fe0fc150ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:51.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.078+0000 7fe101f9b700 1 --2- 172.21.15.31:0/3539072927 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe0fc153a00 0x7fe0fc0cbd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:51.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.078+0000 7fe101f9b700 1 -- 172.21.15.31:0/3539072927 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0fc15de50 msgr2=0x7fe0fc150ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:51.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.078+0000 7fe101f9b700 1 --2- 172.21.15.31:0/3539072927 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0fc15de50 0x7fe0fc150ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:51.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.078+0000 7fe101f9b700 1 -- 172.21.15.31:0/3539072927 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0ec0094f0 con 0x7fe0fc0cea00 2023-10-06T02:23:51.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.078+0000 7fe10179a700 1 --2- 172.21.15.31:0/3539072927 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe0fc153a00 0x7fe0fc0cbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:23:51.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.078+0000 7fe101f9b700 1 --2- 172.21.15.31:0/3539072927 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe0fc0cea00 0x7fe0fc0c8180 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe0f800d860 tx=0x7fe0f8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:51.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.078+0000 7fe0f1ffb700 1 -- 172.21.15.31:0/3539072927 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f80125f0 con 0x7fe0fc0cea00 2023-10-06T02:23:51.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.078+0000 7fe0f1ffb700 1 -- 172.21.15.31:0/3539072927 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0f8012750 con 0x7fe0fc0cea00 2023-10-06T02:23:51.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.078+0000 7fe0f1ffb700 1 -- 172.21.15.31:0/3539072927 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f8012ad0 con 0x7fe0fc0cea00 2023-10-06T02:23:51.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.078+0000 7fe102f9d700 1 -- 172.21.15.31:0/3539072927 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0fc0cc750 con 0x7fe0fc0cea00 2023-10-06T02:23:51.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.078+0000 7fe102f9d700 1 -- 172.21.15.31:0/3539072927 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0fc0ccd10 con 0x7fe0fc0cea00 2023-10-06T02:23:51.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.079+0000 7fe102f9d700 1 -- 172.21.15.31:0/3539072927 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0fc04c3c0 con 0x7fe0fc0cea00 2023-10-06T02:23:51.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.080+0000 7fe0f1ffb700 1 -- 172.21.15.31:0/3539072927 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe0f8014430 con 0x7fe0fc0cea00 2023-10-06T02:23:51.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.081+0000 7fe0f1ffb700 1 --2- 172.21.15.31:0/3539072927 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe0e0066f30 0x7fe0e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:51.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.081+0000 7fe0f1ffb700 1 -- 172.21.15.31:0/3539072927 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6116+0+0 (secure 0 0 0) 0x7fe0f8084bc0 con 0x7fe0fc0cea00 2023-10-06T02:23:51.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.081+0000 7fe10179a700 1 --2- 172.21.15.31:0/3539072927 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe0e0066f30 0x7fe0e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:51.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.082+0000 7fe10179a700 1 --2- 172.21.15.31:0/3539072927 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe0e0066f30 0x7fe0e00693e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc0cac20 tx=0x7fe0ec017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:51.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.085+0000 7fe0f1ffb700 1 -- 172.21.15.31:0/3539072927 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe0f8051bf0 con 0x7fe0fc0cea00 2023-10-06T02:23:51.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.289+0000 7fe102f9d700 1 -- 172.21.15.31:0/3539072927 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe0fc0caef0 con 0x7fe0fc0cea00 2023-10-06T02:23:51.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.290+0000 7fe0f1ffb700 1 -- 172.21.15.31:0/3539072927 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe0f8055210 con 0x7fe0fc0cea00 2023-10-06T02:23:51.291 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-10-06T02:23:51.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.292+0000 7fe102f9d700 1 -- 172.21.15.31:0/3539072927 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe0e0066f30 msgr2=0x7fe0e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:51.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.292+0000 7fe102f9d700 1 --2- 172.21.15.31:0/3539072927 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe0e0066f30 0x7fe0e00693e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc0cac20 tx=0x7fe0ec017000 comp rx=0 tx=0).stop 2023-10-06T02:23:51.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.292+0000 7fe102f9d700 1 -- 172.21.15.31:0/3539072927 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe0fc0cea00 msgr2=0x7fe0fc0c8180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:51.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.292+0000 7fe102f9d700 1 --2- 172.21.15.31:0/3539072927 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe0fc0cea00 0x7fe0fc0c8180 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe0f800d860 tx=0x7fe0f8008040 comp rx=0 tx=0).stop 2023-10-06T02:23:51.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.292+0000 7fe102f9d700 1 -- 172.21.15.31:0/3539072927 shutdown_connections 2023-10-06T02:23:51.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.292+0000 7fe102f9d700 1 --2- 172.21.15.31:0/3539072927 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0fc15de50 0x7fe0fc150ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:51.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.292+0000 7fe102f9d700 1 --2- 172.21.15.31:0/3539072927 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe0fc0cea00 0x7fe0fc0c8180 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:51.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.292+0000 7fe102f9d700 1 --2- 172.21.15.31:0/3539072927 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe0e0066f30 0x7fe0e00693e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:51.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.292+0000 7fe102f9d700 1 --2- 172.21.15.31:0/3539072927 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe0fc153a00 0x7fe0fc0cbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:51.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.292+0000 7fe102f9d700 1 -- 172.21.15.31:0/3539072927 >> 172.21.15.31:0/3539072927 conn(0x7fe0fc0703a0 msgr2=0x7fe0fc15ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:51.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.292+0000 7fe102f9d700 1 -- 172.21.15.31:0/3539072927 shutdown_connections 2023-10-06T02:23:51.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.292+0000 7fe102f9d700 1 -- 172.21.15.31:0/3539072927 wait complete. 2023-10-06T02:23:51.311 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T02:23:51.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.498+0000 7fc2b539d700 1 Processor -- start 2023-10-06T02:23:51.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.499+0000 7fc2b539d700 1 -- start start 2023-10-06T02:23:51.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.500+0000 7fc2b539d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc2b0142c00 0x7fc2b006b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:51.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.500+0000 7fc2b539d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc2b006bd20 0x7fc2b00c7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:51.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.500+0000 7fc2b539d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2b00ca480 con 0x7fc2b0142fd0 2023-10-06T02:23:51.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.500+0000 7fc2b539d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2b015e240 con 0x7fc2b006bd20 2023-10-06T02:23:51.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.500+0000 7fc2b539d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2b006b350 con 0x7fc2b0142c00 2023-10-06T02:23:51.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.500+0000 7fc2af7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc2b006bd20 0x7fc2b00c7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:51.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.500+0000 7fc2af7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc2b006bd20 0x7fc2b00c7880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38990/0 (socket says 172.21.15.31:38990) 2023-10-06T02:23:51.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.500+0000 7fc2af7fe700 1 -- 172.21.15.31:0/3206426190 learned_addr learned my addr 172.21.15.31:0/3206426190 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:51.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.500+0000 7fc2b4b9c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fc2b0142fd0 0x7fc2b006c6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49376/0 (socket says 172.21.15.31:49376) 2023-10-06T02:23:51.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.500+0000 7fc2affff700 1 --2- 172.21.15.31:0/3206426190 >> v2:172.21.15.62:3300/0 conn(0x7fc2b0142c00 0x7fc2b006b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:51.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.501+0000 7fc2af7fe700 1 -- 172.21.15.31:0/3206426190 >> v2:172.21.15.62:3300/0 conn(0x7fc2b0142c00 msgr2=0x7fc2b006b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:51.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.501+0000 7fc2af7fe700 1 --2- 172.21.15.31:0/3206426190 >> v2:172.21.15.62:3300/0 conn(0x7fc2b0142c00 0x7fc2b006b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:51.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.501+0000 7fc2af7fe700 1 -- 172.21.15.31:0/3206426190 >> v1:172.21.15.31:6789/0 conn(0x7fc2b0142fd0 legacy=0x7fc2b006c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:51.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.501+0000 7fc2af7fe700 1 -- 172.21.15.31:0/3206426190 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2b015df60 con 0x7fc2b006bd20 2023-10-06T02:23:51.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.501+0000 7fc2aeffd700 1 -- 172.21.15.31:0/3206426190 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3575336668 0 0) 0x7fc2b00ca480 con 0x7fc2b0142fd0 2023-10-06T02:23:51.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.501+0000 7fc2af7fe700 1 --2- 172.21.15.31:0/3206426190 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2b006bd20 0x7fc2b00c7880 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc2b006f490 tx=0x7fc2a4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3ae2e4d120d8ec1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:51.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.501+0000 7fc2aeffd700 1 -- 172.21.15.31:0/3206426190 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a4009e70 con 0x7fc2b006bd20 2023-10-06T02:23:51.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.501+0000 7fc2aeffd700 1 -- 172.21.15.31:0/3206426190 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2a400b860 con 0x7fc2b006bd20 2023-10-06T02:23:51.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.501+0000 7fc2aeffd700 1 -- 172.21.15.31:0/3206426190 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a400bc00 con 0x7fc2b006bd20 2023-10-06T02:23:51.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.502+0000 7fc2b539d700 1 -- 172.21.15.31:0/3206426190 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2b006bd20 msgr2=0x7fc2b00c7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:51.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.502+0000 7fc2b539d700 1 --2- 172.21.15.31:0/3206426190 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2b006bd20 0x7fc2b00c7880 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc2b006f490 tx=0x7fc2a4009250 comp rx=0 tx=0).stop 2023-10-06T02:23:51.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.502+0000 7fc2b539d700 1 -- 172.21.15.31:0/3206426190 shutdown_connections 2023-10-06T02:23:51.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.502+0000 7fc2b539d700 1 --2- 172.21.15.31:0/3206426190 >> v2:172.21.15.62:3300/0 conn(0x7fc2b0142c00 0x7fc2b006b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:51.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.502+0000 7fc2b539d700 1 --2- 172.21.15.31:0/3206426190 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2b006bd20 0x7fc2b00c7880 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:51.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.502+0000 7fc2b539d700 1 -- 172.21.15.31:0/3206426190 >> 172.21.15.31:0/3206426190 conn(0x7fc2b00675a0 msgr2=0x7fc2b00679a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:51.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.502+0000 7fc2b539d700 1 -- 172.21.15.31:0/3206426190 shutdown_connections 2023-10-06T02:23:51.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.502+0000 7fc2b539d700 1 -- 172.21.15.31:0/3206426190 wait complete. 2023-10-06T02:23:51.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.503+0000 7fc2b539d700 1 Processor -- start 2023-10-06T02:23:51.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.503+0000 7fc2b539d700 1 -- start start 2023-10-06T02:23:51.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.504+0000 7fc2b539d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc2b006bd20 0x7fc2b0169d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:51.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.504+0000 7fc2b539d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc2b0142c00 0x7fc2b0158b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:51.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.504+0000 7fc2b539d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2b0142fd0 0x7fc2b00c55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:51.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.504+0000 7fc2b539d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b00affe0 con 0x7fc2b0142fd0 2023-10-06T02:23:51.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.504+0000 7fc2b539d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b016a350 con 0x7fc2b006bd20 2023-10-06T02:23:51.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.504+0000 7fc2b539d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b016a630 con 0x7fc2b0142c00 2023-10-06T02:23:51.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.505+0000 7fc2b4b9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2b0142fd0 0x7fc2b00c55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:51.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.505+0000 7fc2b4b9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2b0142fd0 0x7fc2b00c55d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38994/0 (socket says 172.21.15.31:38994) 2023-10-06T02:23:51.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.505+0000 7fc2b4b9c700 1 -- 172.21.15.31:0/2314738887 learned_addr learned my addr 172.21.15.31:0/2314738887 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:51.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.505+0000 7fc2affff700 1 --2- 172.21.15.31:0/2314738887 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc2b006bd20 0x7fc2b0169d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:51.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.505+0000 7fc2af7fe700 1 --2- 172.21.15.31:0/2314738887 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc2b0142c00 0x7fc2b0158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:51.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.505+0000 7fc2b4b9c700 1 -- 172.21.15.31:0/2314738887 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc2b0142c00 msgr2=0x7fc2b0158b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:51.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.505+0000 7fc2b4b9c700 1 --2- 172.21.15.31:0/2314738887 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc2b0142c00 0x7fc2b0158b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:51.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.505+0000 7fc2b4b9c700 1 -- 172.21.15.31:0/2314738887 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc2b006bd20 msgr2=0x7fc2b0169d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:51.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.505+0000 7fc2b4b9c700 1 --2- 172.21.15.31:0/2314738887 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc2b006bd20 0x7fc2b0169d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:51.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.505+0000 7fc2b4b9c700 1 -- 172.21.15.31:0/2314738887 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2a40094f0 con 0x7fc2b0142fd0 2023-10-06T02:23:51.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.505+0000 7fc2af7fe700 1 --2- 172.21.15.31:0/2314738887 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc2b0142c00 0x7fc2b0158b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:23:51.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.505+0000 7fc2b4b9c700 1 --2- 172.21.15.31:0/2314738887 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2b0142fd0 0x7fc2b00c55d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc2a800ccc0 tx=0x7fc2a8001a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:51.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.505+0000 7fc297fff700 1 -- 172.21.15.31:0/2314738887 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a8005df0 con 0x7fc2b0142fd0 2023-10-06T02:23:51.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.505+0000 7fc297fff700 1 -- 172.21.15.31:0/2314738887 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2a800a6b0 con 0x7fc2b0142fd0 2023-10-06T02:23:51.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.505+0000 7fc297fff700 1 -- 172.21.15.31:0/2314738887 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a800aa70 con 0x7fc2b0142fd0 2023-10-06T02:23:51.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.506+0000 7fc2b539d700 1 -- 172.21.15.31:0/2314738887 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2b016a910 con 0x7fc2b0142fd0 2023-10-06T02:23:51.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.506+0000 7fc2b539d700 1 -- 172.21.15.31:0/2314738887 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2b016dc70 con 0x7fc2b0142fd0 2023-10-06T02:23:51.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.507+0000 7fc2b539d700 1 -- 172.21.15.31:0/2314738887 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2b004c3c0 con 0x7fc2b0142fd0 2023-10-06T02:23:51.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.507+0000 7fc297fff700 1 -- 172.21.15.31:0/2314738887 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc2a80070f0 con 0x7fc2b0142fd0 2023-10-06T02:23:51.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.508+0000 7fc297fff700 1 --2- 172.21.15.31:0/2314738887 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc29c066f30 0x7fc29c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:51.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.508+0000 7fc297fff700 1 -- 172.21.15.31:0/2314738887 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6116+0+0 (secure 0 0 0) 0x7fc2a80847a0 con 0x7fc2b0142fd0 2023-10-06T02:23:51.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.508+0000 7fc2affff700 1 --2- 172.21.15.31:0/2314738887 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc29c066f30 0x7fc29c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:51.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.509+0000 7fc2affff700 1 --2- 172.21.15.31:0/2314738887 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc29c066f30 0x7fc29c0693e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc2b0167930 tx=0x7fc2a00091a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:51.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.513+0000 7fc297fff700 1 -- 172.21.15.31:0/2314738887 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc2a8051900 con 0x7fc2b0142fd0 2023-10-06T02:23:51.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.715+0000 7fc2b539d700 1 -- 172.21.15.31:0/2314738887 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc2b006ad00 con 0x7fc2b0142fd0 2023-10-06T02:23:51.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.715+0000 7fc297fff700 1 -- 172.21.15.31:0/2314738887 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc2a800ae60 con 0x7fc2b0142fd0 2023-10-06T02:23:51.717 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 8 2023-10-06T02:23:51.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.718+0000 7fc2b539d700 1 -- 172.21.15.31:0/2314738887 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc29c066f30 msgr2=0x7fc29c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:51.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.718+0000 7fc2b539d700 1 --2- 172.21.15.31:0/2314738887 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc29c066f30 0x7fc29c0693e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc2b0167930 tx=0x7fc2a00091a0 comp rx=0 tx=0).stop 2023-10-06T02:23:51.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.718+0000 7fc2b539d700 1 -- 172.21.15.31:0/2314738887 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2b0142fd0 msgr2=0x7fc2b00c55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:51.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.718+0000 7fc2b539d700 1 --2- 172.21.15.31:0/2314738887 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2b0142fd0 0x7fc2b00c55d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc2a800ccc0 tx=0x7fc2a8001a70 comp rx=0 tx=0).stop 2023-10-06T02:23:51.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.718+0000 7fc2b539d700 1 -- 172.21.15.31:0/2314738887 shutdown_connections 2023-10-06T02:23:51.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.718+0000 7fc2b539d700 1 --2- 172.21.15.31:0/2314738887 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2b0142fd0 0x7fc2b00c55d0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:51.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.718+0000 7fc2b539d700 1 --2- 172.21.15.31:0/2314738887 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc2b006bd20 0x7fc2b0169d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:51.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.718+0000 7fc2b539d700 1 --2- 172.21.15.31:0/2314738887 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc29c066f30 0x7fc29c0693e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:51.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.718+0000 7fc2b539d700 1 --2- 172.21.15.31:0/2314738887 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc2b0142c00 0x7fc2b0158b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:51.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.718+0000 7fc2b539d700 1 -- 172.21.15.31:0/2314738887 >> 172.21.15.31:0/2314738887 conn(0x7fc2b00675a0 msgr2=0x7fc2b00cd010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:51.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.718+0000 7fc2b539d700 1 -- 172.21.15.31:0/2314738887 shutdown_connections 2023-10-06T02:23:51.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.718+0000 7fc2b539d700 1 -- 172.21.15.31:0/2314738887 wait complete. 2023-10-06T02:23:51.736 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T02:23:51.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.924+0000 7f3b2cb9d700 1 Processor -- start 2023-10-06T02:23:51.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.924+0000 7f3b2cb9d700 1 -- start start 2023-10-06T02:23:51.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.926+0000 7f3b2cb9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3b280cf110 0x7f3b2815de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:51.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.926+0000 7f3b2cb9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b2815ac50 con 0x7f3b28153dd0 2023-10-06T02:23:51.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.926+0000 7f3b2cb9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b2815d010 con 0x7f3b28153a00 2023-10-06T02:23:51.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.926+0000 7f3b2cb9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2815af60 con 0x7f3b280cf110 2023-10-06T02:23:51.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.926+0000 7f3b26ffd700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f3b28153dd0 0x7f3b280cea00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49382/0 (socket says 172.21.15.31:49382) 2023-10-06T02:23:51.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.926+0000 7f3b26ffd700 1 -- 172.21.15.31:0/920078164 learned_addr learned my addr 172.21.15.31:0/920078164 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:23:51.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.926+0000 7f3b267fc700 1 -- 172.21.15.31:0/920078164 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1271947928 0 0) 0x7f3b2815ac50 con 0x7f3b28153dd0 2023-10-06T02:23:51.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.926+0000 7f3b27fff700 1 --2- 172.21.15.31:0/920078164 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3b280cf110 0x7f3b2815de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:51.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.926+0000 7f3b267fc700 1 -- 172.21.15.31:0/920078164 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b10003570 con 0x7f3b28153dd0 2023-10-06T02:23:51.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.927+0000 7f3b267fc700 1 -- 172.21.15.31:0/920078164 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3906041414 0 0) 0x7f3b10003570 con 0x7f3b28153dd0 2023-10-06T02:23:51.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.927+0000 7f3b267fc700 1 -- 172.21.15.31:0/920078164 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3b280cf110 msgr2=0x7f3b2815de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:51.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.927+0000 7f3b267fc700 1 --2- 172.21.15.31:0/920078164 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3b280cf110 0x7f3b2815de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:51.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.927+0000 7f3b267fc700 1 -- 172.21.15.31:0/920078164 >> v1:172.21.15.62:6789/0 conn(0x7f3b28153a00 legacy=0x7f3b2815a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:51.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.927+0000 7f3b267fc700 1 -- 172.21.15.31:0/920078164 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b2815b240 con 0x7f3b28153dd0 2023-10-06T02:23:51.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.927+0000 7f3b267fc700 1 -- 172.21.15.31:0/920078164 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7f3b1c001470 con 0x7f3b28153dd0 2023-10-06T02:23:51.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.927+0000 7f3b267fc700 1 -- 172.21.15.31:0/920078164 >> v1:172.21.15.31:6789/0 conn(0x7f3b28153dd0 legacy=0x7f3b280cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:51.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.927+0000 7f3b267fc700 1 --2- 172.21.15.31:0/920078164 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3b280cf110 0x7f3b10003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:51.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.927+0000 7f3b267fc700 1 --2- 172.21.15.31:0/920078164 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b10003fd0 0x7f3b10004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:51.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.927+0000 7f3b267fc700 1 --2- 172.21.15.31:0/920078164 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b10004d50 0x7f3b100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:51.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.927+0000 7f3b267fc700 1 -- 172.21.15.31:0/920078164 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2815af60 con 0x7f3b10003fd0 2023-10-06T02:23:51.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.927+0000 7f3b267fc700 1 -- 172.21.15.31:0/920078164 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1000bb40 con 0x7f3b10004d50 2023-10-06T02:23:51.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.927+0000 7f3b267fc700 1 -- 172.21.15.31:0/920078164 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1000bd00 con 0x7f3b280cf110 2023-10-06T02:23:51.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.927+0000 7f3b26ffd700 1 --2- 172.21.15.31:0/920078164 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b10003fd0 0x7f3b10004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:51.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.927+0000 7f3b277fe700 1 --2- 172.21.15.31:0/920078164 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3b280cf110 0x7f3b10003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:51.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.927+0000 7f3b27fff700 1 --2- 172.21.15.31:0/920078164 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b10004d50 0x7f3b100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:51.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.927+0000 7f3b26ffd700 1 -- 172.21.15.31:0/920078164 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3b280cf110 msgr2=0x7f3b10003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:51.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.927+0000 7f3b26ffd700 1 --2- 172.21.15.31:0/920078164 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3b280cf110 0x7f3b10003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:51.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.927+0000 7f3b26ffd700 1 -- 172.21.15.31:0/920078164 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b10004d50 msgr2=0x7f3b100051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:51.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.927+0000 7f3b26ffd700 1 --2- 172.21.15.31:0/920078164 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b10004d50 0x7f3b100051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:51.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.927+0000 7f3b26ffd700 1 -- 172.21.15.31:0/920078164 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b10003570 con 0x7f3b10003fd0 2023-10-06T02:23:51.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.928+0000 7f3b26ffd700 1 --2- 172.21.15.31:0/920078164 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b10003fd0 0x7f3b10004810 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c00c3c0 tx=0x7f3b1c001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8073a0784cbfa56 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:51.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.928+0000 7f3b267fc700 1 -- 172.21.15.31:0/920078164 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1c00cc90 con 0x7f3b10003fd0 2023-10-06T02:23:51.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.928+0000 7f3b267fc700 1 -- 172.21.15.31:0/920078164 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b1c00cdf0 con 0x7f3b10003fd0 2023-10-06T02:23:51.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.928+0000 7f3b267fc700 1 -- 172.21.15.31:0/920078164 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1c005b70 con 0x7f3b10003fd0 2023-10-06T02:23:51.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.928+0000 7f3b2cb9d700 1 -- 172.21.15.31:0/920078164 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b10003fd0 msgr2=0x7f3b10004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:51.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.928+0000 7f3b2cb9d700 1 --2- 172.21.15.31:0/920078164 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b10003fd0 0x7f3b10004810 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c00c3c0 tx=0x7f3b1c001e70 comp rx=0 tx=0).stop 2023-10-06T02:23:51.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.928+0000 7f3b27fff700 1 -- 172.21.15.31:0/920078164 reap_dead start 2023-10-06T02:23:51.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.929+0000 7f3b2cb9d700 1 -- 172.21.15.31:0/920078164 shutdown_connections 2023-10-06T02:23:51.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.929+0000 7f3b2cb9d700 1 -- 172.21.15.31:0/920078164 >> 172.21.15.31:0/920078164 conn(0x7f3b280703a0 msgr2=0x7f3b280707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:51.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.929+0000 7f3b2cb9d700 1 -- 172.21.15.31:0/920078164 shutdown_connections 2023-10-06T02:23:51.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.929+0000 7f3b2cb9d700 1 -- 172.21.15.31:0/920078164 wait complete. 2023-10-06T02:23:51.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.930+0000 7f3b2cb9d700 1 Processor -- start 2023-10-06T02:23:51.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.930+0000 7f3b2cb9d700 1 -- start start 2023-10-06T02:23:51.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.931+0000 7f3b2cb9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b10003fd0 0x7f3b2816d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:51.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.931+0000 7f3b2cb9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3b280c7b20 0x7f3b281508e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:51.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.931+0000 7f3b2cb9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b280b14f0 0x7f3b280b1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:51.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.931+0000 7f3b2cb9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b280c8e80 con 0x7f3b10003fd0 2023-10-06T02:23:51.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.931+0000 7f3b2cb9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b280b1ea0 con 0x7f3b280b14f0 2023-10-06T02:23:51.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.931+0000 7f3b2cb9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b280c8300 con 0x7f3b280c7b20 2023-10-06T02:23:51.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.931+0000 7f3b277fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b10003fd0 0x7f3b2816d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:51.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.931+0000 7f3b277fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b10003fd0 0x7f3b2816d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39006/0 (socket says 172.21.15.31:39006) 2023-10-06T02:23:51.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.931+0000 7f3b277fe700 1 -- 172.21.15.31:0/520069175 learned_addr learned my addr 172.21.15.31:0/520069175 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:51.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.931+0000 7f3b26ffd700 1 --2- 172.21.15.31:0/520069175 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3b280c7b20 0x7f3b281508e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:51.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.931+0000 7f3b27fff700 1 --2- 172.21.15.31:0/520069175 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b280b14f0 0x7f3b280b1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:51.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.932+0000 7f3b277fe700 1 -- 172.21.15.31:0/520069175 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3b280c7b20 msgr2=0x7f3b281508e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:51.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.932+0000 7f3b277fe700 1 --2- 172.21.15.31:0/520069175 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3b280c7b20 0x7f3b281508e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:51.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.932+0000 7f3b277fe700 1 -- 172.21.15.31:0/520069175 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b280b14f0 msgr2=0x7f3b280b1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:51.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.932+0000 7f3b277fe700 1 --2- 172.21.15.31:0/520069175 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b280b14f0 0x7f3b280b1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:51.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.932+0000 7f3b277fe700 1 -- 172.21.15.31:0/520069175 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b280c85e0 con 0x7f3b10003fd0 2023-10-06T02:23:51.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.932+0000 7f3b27fff700 1 --2- 172.21.15.31:0/520069175 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b280b14f0 0x7f3b280b1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:23:51.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.932+0000 7f3b277fe700 1 --2- 172.21.15.31:0/520069175 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b10003fd0 0x7f3b2816d920 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3b18002ae0 tx=0x7f3b18007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:51.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.932+0000 7f3b0f7fe700 1 -- 172.21.15.31:0/520069175 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1800f8d0 con 0x7f3b10003fd0 2023-10-06T02:23:51.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.932+0000 7f3b0f7fe700 1 -- 172.21.15.31:0/520069175 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b1800fa30 con 0x7f3b10003fd0 2023-10-06T02:23:51.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.932+0000 7f3b2cb9d700 1 -- 172.21.15.31:0/520069175 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b280c88c0 con 0x7f3b10003fd0 2023-10-06T02:23:51.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.932+0000 7f3b0f7fe700 1 -- 172.21.15.31:0/520069175 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1800fe30 con 0x7f3b10003fd0 2023-10-06T02:23:51.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.932+0000 7f3b2cb9d700 1 -- 172.21.15.31:0/520069175 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b28150e20 con 0x7f3b10003fd0 2023-10-06T02:23:51.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.933+0000 7f3b2cb9d700 1 -- 172.21.15.31:0/520069175 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b2804c3c0 con 0x7f3b10003fd0 2023-10-06T02:23:51.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.934+0000 7f3b0f7fe700 1 -- 172.21.15.31:0/520069175 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3b18013600 con 0x7f3b10003fd0 2023-10-06T02:23:51.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.935+0000 7f3b0f7fe700 1 --2- 172.21.15.31:0/520069175 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f3b14066f30 0x7f3b140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:51.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.935+0000 7f3b0f7fe700 1 -- 172.21.15.31:0/520069175 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f3b18084660 con 0x7f3b10003fd0 2023-10-06T02:23:51.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.935+0000 7f3b26ffd700 1 --2- 172.21.15.31:0/520069175 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f3b14066f30 0x7f3b140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:51.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.936+0000 7f3b26ffd700 1 --2- 172.21.15.31:0/520069175 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f3b14066f30 0x7f3b140693e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c007cf0 tx=0x7f3b1c015000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:51.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:51.939+0000 7f3b0f7fe700 1 -- 172.21.15.31:0/520069175 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3b18051710 con 0x7f3b10003fd0 2023-10-06T02:23:52.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.140+0000 7f3b2cb9d700 1 -- 172.21.15.31:0/520069175 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3b2815b520 con 0x7f3b10003fd0 2023-10-06T02:23:52.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.140+0000 7f3b0f7fe700 1 -- 172.21.15.31:0/520069175 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3b18054d30 con 0x7f3b10003fd0 2023-10-06T02:23:52.142 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-10-06T02:23:52.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.142+0000 7f3b2cb9d700 1 -- 172.21.15.31:0/520069175 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f3b14066f30 msgr2=0x7f3b140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:52.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.142+0000 7f3b2cb9d700 1 --2- 172.21.15.31:0/520069175 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f3b14066f30 0x7f3b140693e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c007cf0 tx=0x7f3b1c015000 comp rx=0 tx=0).stop 2023-10-06T02:23:52.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.142+0000 7f3b2cb9d700 1 -- 172.21.15.31:0/520069175 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b10003fd0 msgr2=0x7f3b2816d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:52.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.142+0000 7f3b2cb9d700 1 --2- 172.21.15.31:0/520069175 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b10003fd0 0x7f3b2816d920 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3b18002ae0 tx=0x7f3b18007d10 comp rx=0 tx=0).stop 2023-10-06T02:23:52.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.142+0000 7f3b2cb9d700 1 -- 172.21.15.31:0/520069175 shutdown_connections 2023-10-06T02:23:52.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.142+0000 7f3b2cb9d700 1 --2- 172.21.15.31:0/520069175 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b280b14f0 0x7f3b280b1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:52.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.142+0000 7f3b2cb9d700 1 --2- 172.21.15.31:0/520069175 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b10003fd0 0x7f3b2816d920 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:52.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.142+0000 7f3b2cb9d700 1 --2- 172.21.15.31:0/520069175 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f3b14066f30 0x7f3b140693e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:52.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.142+0000 7f3b2cb9d700 1 --2- 172.21.15.31:0/520069175 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3b280c7b20 0x7f3b281508e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:52.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.142+0000 7f3b2cb9d700 1 -- 172.21.15.31:0/520069175 >> 172.21.15.31:0/520069175 conn(0x7f3b280703a0 msgr2=0x7f3b280707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:52.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.143+0000 7f3b2cb9d700 1 -- 172.21.15.31:0/520069175 shutdown_connections 2023-10-06T02:23:52.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.143+0000 7f3b2cb9d700 1 -- 172.21.15.31:0/520069175 wait complete. 2023-10-06T02:23:52.161 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T02:23:52.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.351+0000 7f360f39d700 1 Processor -- start 2023-10-06T02:23:52.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.351+0000 7f360f39d700 1 -- start start 2023-10-06T02:23:52.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.352+0000 7f360f39d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f36080c7fb0 0x7f360806abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:52.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.352+0000 7f360f39d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f360813f780 0x7f360813fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:52.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.352+0000 7f360f39d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36081400e0 con 0x7f36080c7be0 2023-10-06T02:23:52.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.352+0000 7f360f39d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f360806bf10 con 0x7f36080c7fb0 2023-10-06T02:23:52.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.352+0000 7f360f39d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f360806b290 con 0x7f360813f780 2023-10-06T02:23:52.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.352+0000 7f360db9a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f36080c7fb0 0x7f360806abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:52.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.352+0000 7f360db9a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f36080c7fb0 0x7f360806abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39012/0 (socket says 172.21.15.31:39012) 2023-10-06T02:23:52.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.352+0000 7f360db9a700 1 -- 172.21.15.31:0/1469219842 learned_addr learned my addr 172.21.15.31:0/1469219842 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:52.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.353+0000 7f360eb9c700 1 --2- 172.21.15.31:0/1469219842 >> v2:172.21.15.62:3300/0 conn(0x7f360813f780 0x7f360813fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:52.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.353+0000 7f360db9a700 1 -- 172.21.15.31:0/1469219842 >> v2:172.21.15.62:3300/0 conn(0x7f360813f780 msgr2=0x7f360813fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:52.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.353+0000 7f360db9a700 1 --2- 172.21.15.31:0/1469219842 >> v2:172.21.15.62:3300/0 conn(0x7f360813f780 0x7f360813fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:52.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.353+0000 7f360db9a700 1 -- 172.21.15.31:0/1469219842 >> v1:172.21.15.62:6789/0 conn(0x7f36080c7be0 legacy=0x7f360815a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:52.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.353+0000 7f360db9a700 1 -- 172.21.15.31:0/1469219842 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f360806b570 con 0x7f36080c7fb0 2023-10-06T02:23:52.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.353+0000 7f360db9a700 1 --2- 172.21.15.31:0/1469219842 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f36080c7fb0 0x7f360806abd0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f36080aec50 tx=0x7f35f800b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fe9f22dd08fea76 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:52.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.353+0000 7f360d399700 1 -- 172.21.15.31:0/1469219842 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f800dc50 con 0x7f36080c7fb0 2023-10-06T02:23:52.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.353+0000 7f360d399700 1 -- 172.21.15.31:0/1469219842 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35f800ddb0 con 0x7f36080c7fb0 2023-10-06T02:23:52.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.353+0000 7f360d399700 1 -- 172.21.15.31:0/1469219842 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f80196d0 con 0x7f36080c7fb0 2023-10-06T02:23:52.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.354+0000 7f360f39d700 1 -- 172.21.15.31:0/1469219842 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f36080c7fb0 msgr2=0x7f360806abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:52.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.354+0000 7f360f39d700 1 --2- 172.21.15.31:0/1469219842 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f36080c7fb0 0x7f360806abd0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f36080aec50 tx=0x7f35f800b4f0 comp rx=0 tx=0).stop 2023-10-06T02:23:52.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.354+0000 7f360f39d700 1 -- 172.21.15.31:0/1469219842 shutdown_connections 2023-10-06T02:23:52.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.354+0000 7f360f39d700 1 --2- 172.21.15.31:0/1469219842 >> v2:172.21.15.62:3300/0 conn(0x7f360813f780 0x7f360813fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:52.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.354+0000 7f360f39d700 1 --2- 172.21.15.31:0/1469219842 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f36080c7fb0 0x7f360806abd0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:52.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.354+0000 7f360f39d700 1 -- 172.21.15.31:0/1469219842 >> 172.21.15.31:0/1469219842 conn(0x7f360806d110 msgr2=0x7f36080cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:52.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.354+0000 7f360f39d700 1 -- 172.21.15.31:0/1469219842 shutdown_connections 2023-10-06T02:23:52.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.354+0000 7f360f39d700 1 -- 172.21.15.31:0/1469219842 wait complete. 2023-10-06T02:23:52.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.355+0000 7f360f39d700 1 Processor -- start 2023-10-06T02:23:52.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.356+0000 7f360f39d700 1 -- start start 2023-10-06T02:23:52.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.356+0000 7f360f39d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f36080c7be0 0x7f360816d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:52.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.356+0000 7f360f39d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f36080c7fb0 0x7f3608159270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:52.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.357+0000 7f360f39d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f360813f780 0x7f360813e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:52.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.357+0000 7f360f39d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3608158960 con 0x7f36080c7fb0 2023-10-06T02:23:52.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.357+0000 7f360f39d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f360816dc20 con 0x7f360813f780 2023-10-06T02:23:52.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.357+0000 7f360f39d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36081597b0 con 0x7f36080c7be0 2023-10-06T02:23:52.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.357+0000 7f360db9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f36080c7fb0 0x7f3608159270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:52.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.357+0000 7f360db9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f36080c7fb0 0x7f3608159270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39020/0 (socket says 172.21.15.31:39020) 2023-10-06T02:23:52.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.357+0000 7f360db9a700 1 -- 172.21.15.31:0/717561246 learned_addr learned my addr 172.21.15.31:0/717561246 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:52.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.357+0000 7f360db9a700 1 -- 172.21.15.31:0/717561246 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f36080c7be0 msgr2=0x7f360816d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:52.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.357+0000 7f360db9a700 1 --2- 172.21.15.31:0/717561246 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f36080c7be0 0x7f360816d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:52.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.357+0000 7f360db9a700 1 -- 172.21.15.31:0/717561246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f360813f780 msgr2=0x7f360813e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:52.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.357+0000 7f360db9a700 1 --2- 172.21.15.31:0/717561246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f360813f780 0x7f360813e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:52.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.357+0000 7f360db9a700 1 -- 172.21.15.31:0/717561246 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35f800b790 con 0x7f36080c7fb0 2023-10-06T02:23:52.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.357+0000 7f360db9a700 1 --2- 172.21.15.31:0/717561246 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f36080c7fb0 0x7f3608159270 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f36080693f0 tx=0x7f35f801b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:52.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.358+0000 7f35fe7fc700 1 -- 172.21.15.31:0/717561246 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f8010680 con 0x7f36080c7fb0 2023-10-06T02:23:52.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.358+0000 7f35fe7fc700 1 -- 172.21.15.31:0/717561246 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35f80107e0 con 0x7f36080c7fb0 2023-10-06T02:23:52.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.358+0000 7f35fe7fc700 1 -- 172.21.15.31:0/717561246 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f8010ba0 con 0x7f36080c7fb0 2023-10-06T02:23:52.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.358+0000 7f360f39d700 1 -- 172.21.15.31:0/717561246 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36081583a0 con 0x7f36080c7fb0 2023-10-06T02:23:52.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.358+0000 7f360f39d700 1 -- 172.21.15.31:0/717561246 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3608156c60 con 0x7f36080c7fb0 2023-10-06T02:23:52.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.359+0000 7f360f39d700 1 -- 172.21.15.31:0/717561246 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3608156ac0 con 0x7f36080c7fb0 2023-10-06T02:23:52.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.360+0000 7f35fe7fc700 1 -- 172.21.15.31:0/717561246 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f35f80233f0 con 0x7f36080c7fb0 2023-10-06T02:23:52.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.360+0000 7f35fe7fc700 1 --2- 172.21.15.31:0/717561246 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f35ec066f30 0x7f35ec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:52.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.360+0000 7f35fe7fc700 1 -- 172.21.15.31:0/717561246 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f35f8014070 con 0x7f36080c7fb0 2023-10-06T02:23:52.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.361+0000 7f360e39b700 1 --2- 172.21.15.31:0/717561246 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f35ec066f30 0x7f35ec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:52.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.361+0000 7f360e39b700 1 --2- 172.21.15.31:0/717561246 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f35ec066f30 0x7f35ec0693e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f36080ae570 tx=0x7f3604007b40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:52.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.365+0000 7f35fe7fc700 1 -- 172.21.15.31:0/717561246 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3608156ac0 con 0x7f36080c7fb0 2023-10-06T02:23:52.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.564+0000 7f360f39d700 1 -- 172.21.15.31:0/717561246 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3608156ac0 con 0x7f36080c7fb0 2023-10-06T02:23:52.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.564+0000 7f35fe7fc700 1 -- 172.21.15.31:0/717561246 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3608156ac0 con 0x7f36080c7fb0 2023-10-06T02:23:52.566 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-10-06T02:23:52.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.567+0000 7f360f39d700 1 -- 172.21.15.31:0/717561246 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f35ec066f30 msgr2=0x7f35ec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:52.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.567+0000 7f360f39d700 1 --2- 172.21.15.31:0/717561246 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f35ec066f30 0x7f35ec0693e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f36080ae570 tx=0x7f3604007b40 comp rx=0 tx=0).stop 2023-10-06T02:23:52.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.567+0000 7f360f39d700 1 -- 172.21.15.31:0/717561246 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f36080c7fb0 msgr2=0x7f3608159270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:52.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.567+0000 7f360f39d700 1 --2- 172.21.15.31:0/717561246 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f36080c7fb0 0x7f3608159270 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f36080693f0 tx=0x7f35f801b000 comp rx=0 tx=0).stop 2023-10-06T02:23:52.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.567+0000 7f360f39d700 1 -- 172.21.15.31:0/717561246 shutdown_connections 2023-10-06T02:23:52.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.567+0000 7f360f39d700 1 --2- 172.21.15.31:0/717561246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f360813f780 0x7f360813e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:52.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.567+0000 7f360f39d700 1 --2- 172.21.15.31:0/717561246 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f35ec066f30 0x7f35ec0693e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:52.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.567+0000 7f360f39d700 1 --2- 172.21.15.31:0/717561246 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f36080c7be0 0x7f360816d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:52.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.567+0000 7f360f39d700 1 --2- 172.21.15.31:0/717561246 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f36080c7fb0 0x7f3608159270 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:52.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.567+0000 7f360f39d700 1 -- 172.21.15.31:0/717561246 >> 172.21.15.31:0/717561246 conn(0x7f360806d110 msgr2=0x7f360806e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:52.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.567+0000 7f360f39d700 1 -- 172.21.15.31:0/717561246 shutdown_connections 2023-10-06T02:23:52.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.567+0000 7f360f39d700 1 -- 172.21.15.31:0/717561246 wait complete. 2023-10-06T02:23:52.649 INFO:tasks.cephfs_test_runner:Starting test: test_binary_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) Validate binary .meta file on legacy subvol root doesn't break the system 2023-10-06T02:23:52.649 INFO:tasks.cephfs_test_runner:test_binary_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) 2023-10-06T02:23:52.650 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade' 2023-10-06T02:23:52.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.835+0000 7fc44ffff700 1 Processor -- start 2023-10-06T02:23:52.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.836+0000 7fc44ffff700 1 -- start start 2023-10-06T02:23:52.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.837+0000 7fc44ffff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc450153d70 0x7fc4500acba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:52.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.837+0000 7fc44ffff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc45006ed40 0x7fc45006f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:52.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.837+0000 7fc44ffff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4500c8ce0 con 0x7fc450156860 2023-10-06T02:23:52.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.837+0000 7fc44ffff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc45006f720 con 0x7fc450153d70 2023-10-06T02:23:52.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.837+0000 7fc44ffff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4500ad0e0 con 0x7fc45006ed40 2023-10-06T02:23:52.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.838+0000 7fc44f7fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc45006ed40 0x7fc45006f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:52.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.838+0000 7fc44e7fc700 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fc450156860 0x7fc4500ae300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:59854/0 (socket says 172.21.15.31:59854) 2023-10-06T02:23:52.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.838+0000 7fc44e7fc700 1 -- 172.21.15.31:0/3744310119 learned_addr learned my addr 172.21.15.31:0/3744310119 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:23:52.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.838+0000 7fc44effd700 1 --2- 172.21.15.31:0/3744310119 >> v2:172.21.15.62:3300/0 conn(0x7fc450153d70 0x7fc4500acba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:52.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.838+0000 7fc44dffb700 1 -- 172.21.15.31:0/3744310119 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2782814163 0 0) 0x7fc4500c8ce0 con 0x7fc450156860 2023-10-06T02:23:52.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.838+0000 7fc44dffb700 1 -- 172.21.15.31:0/3744310119 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc438003570 con 0x7fc450156860 2023-10-06T02:23:52.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.838+0000 7fc44f7fe700 1 -- 172.21.15.31:0/3744310119 >> v2:172.21.15.62:3300/0 conn(0x7fc450153d70 msgr2=0x7fc4500acba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:52.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.838+0000 7fc44f7fe700 1 --2- 172.21.15.31:0/3744310119 >> v2:172.21.15.62:3300/0 conn(0x7fc450153d70 0x7fc4500acba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:52.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.838+0000 7fc44f7fe700 1 -- 172.21.15.31:0/3744310119 >> v1:172.21.15.62:6789/0 conn(0x7fc450156860 legacy=0x7fc4500ae300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:52.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.838+0000 7fc44f7fe700 1 -- 172.21.15.31:0/3744310119 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4500aea10 con 0x7fc45006ed40 2023-10-06T02:23:52.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.838+0000 7fc44effd700 1 --2- 172.21.15.31:0/3744310119 >> v2:172.21.15.62:3300/0 conn(0x7fc450153d70 0x7fc4500acba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:23:52.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.839+0000 7fc44f7fe700 1 --2- 172.21.15.31:0/3744310119 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc45006ed40 0x7fc45006f1e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc448009e60 tx=0x7fc448009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=573d628aa12cbb9e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:52.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.839+0000 7fc44dffb700 1 -- 172.21.15.31:0/3744310119 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc44800e070 con 0x7fc45006ed40 2023-10-06T02:23:52.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.839+0000 7fc44dffb700 1 -- 172.21.15.31:0/3744310119 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc44800bb40 con 0x7fc45006ed40 2023-10-06T02:23:52.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.839+0000 7fc44dffb700 1 -- 172.21.15.31:0/3744310119 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc448012430 con 0x7fc45006ed40 2023-10-06T02:23:52.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.839+0000 7fc44ffff700 1 -- 172.21.15.31:0/3744310119 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc45006ed40 msgr2=0x7fc45006f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:52.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.839+0000 7fc44ffff700 1 --2- 172.21.15.31:0/3744310119 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc45006ed40 0x7fc45006f1e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc448009e60 tx=0x7fc448009780 comp rx=0 tx=0).stop 2023-10-06T02:23:52.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.839+0000 7fc44ffff700 1 -- 172.21.15.31:0/3744310119 shutdown_connections 2023-10-06T02:23:52.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.839+0000 7fc44ffff700 1 --2- 172.21.15.31:0/3744310119 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc45006ed40 0x7fc45006f1e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:52.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.839+0000 7fc44ffff700 1 --2- 172.21.15.31:0/3744310119 >> v2:172.21.15.62:3300/0 conn(0x7fc450153d70 0x7fc4500acba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:52.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.839+0000 7fc44ffff700 1 -- 172.21.15.31:0/3744310119 >> 172.21.15.31:0/3744310119 conn(0x7fc45006d170 msgr2=0x7fc4500cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:52.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.839+0000 7fc44ffff700 1 -- 172.21.15.31:0/3744310119 shutdown_connections 2023-10-06T02:23:52.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.839+0000 7fc44ffff700 1 -- 172.21.15.31:0/3744310119 wait complete. 2023-10-06T02:23:52.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.841+0000 7fc44ffff700 1 Processor -- start 2023-10-06T02:23:52.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.841+0000 7fc44ffff700 1 -- start start 2023-10-06T02:23:52.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.842+0000 7fc44ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc45006ed40 0x7fc4501594a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:52.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.842+0000 7fc44ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc450153d70 0x7fc45006ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:52.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.842+0000 7fc44ffff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc450156860 0x7fc450150ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:52.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.842+0000 7fc44ffff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc45016e2f0 con 0x7fc45006ed40 2023-10-06T02:23:52.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.842+0000 7fc44ffff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc45016d6e0 con 0x7fc450153d70 2023-10-06T02:23:52.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.842+0000 7fc44ffff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc45016d9c0 con 0x7fc450156860 2023-10-06T02:23:52.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.842+0000 7fc44effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc45006ed40 0x7fc4501594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:52.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.842+0000 7fc44effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc45006ed40 0x7fc4501594a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39028/0 (socket says 172.21.15.31:39028) 2023-10-06T02:23:52.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.842+0000 7fc44effd700 1 -- 172.21.15.31:0/3384178582 learned_addr learned my addr 172.21.15.31:0/3384178582 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:52.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.842+0000 7fc44f7fe700 1 --2- 172.21.15.31:0/3384178582 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc450156860 0x7fc450150ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:52.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.842+0000 7fc44e7fc700 1 --2- 172.21.15.31:0/3384178582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc450153d70 0x7fc45006ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:52.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.843+0000 7fc44effd700 1 -- 172.21.15.31:0/3384178582 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc450156860 msgr2=0x7fc450150ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:52.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.843+0000 7fc44effd700 1 --2- 172.21.15.31:0/3384178582 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc450156860 0x7fc450150ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:52.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.843+0000 7fc44effd700 1 -- 172.21.15.31:0/3384178582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc450153d70 msgr2=0x7fc45006ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:52.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.843+0000 7fc44effd700 1 --2- 172.21.15.31:0/3384178582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc450153d70 0x7fc45006ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:52.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.843+0000 7fc44effd700 1 -- 172.21.15.31:0/3384178582 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc440009420 con 0x7fc45006ed40 2023-10-06T02:23:52.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.843+0000 7fc44e7fc700 1 --2- 172.21.15.31:0/3384178582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc450153d70 0x7fc45006ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:23:52.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.843+0000 7fc44effd700 1 --2- 172.21.15.31:0/3384178582 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc45006ed40 0x7fc4501594a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc44000ec80 tx=0x7fc4400091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:52.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.843+0000 7fc436ffd700 1 -- 172.21.15.31:0/3384178582 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc44000c8f0 con 0x7fc45006ed40 2023-10-06T02:23:52.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.843+0000 7fc436ffd700 1 -- 172.21.15.31:0/3384178582 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc44000ca50 con 0x7fc45006ed40 2023-10-06T02:23:52.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.843+0000 7fc436ffd700 1 -- 172.21.15.31:0/3384178582 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc44000cdd0 con 0x7fc45006ed40 2023-10-06T02:23:52.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.843+0000 7fc44ffff700 1 -- 172.21.15.31:0/3384178582 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc448009a20 con 0x7fc45006ed40 2023-10-06T02:23:52.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.843+0000 7fc44ffff700 1 -- 172.21.15.31:0/3384178582 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc45016df80 con 0x7fc45006ed40 2023-10-06T02:23:52.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.844+0000 7fc44ffff700 1 -- 172.21.15.31:0/3384178582 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc45004c3c0 con 0x7fc45006ed40 2023-10-06T02:23:52.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.845+0000 7fc436ffd700 1 -- 172.21.15.31:0/3384178582 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc440013680 con 0x7fc45006ed40 2023-10-06T02:23:52.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.846+0000 7fc436ffd700 1 --2- 172.21.15.31:0/3384178582 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc43c066f30 0x7fc43c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:52.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.846+0000 7fc436ffd700 1 -- 172.21.15.31:0/3384178582 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6116+0+0 (secure 0 0 0) 0x7fc440084b20 con 0x7fc45006ed40 2023-10-06T02:23:52.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.847+0000 7fc44e7fc700 1 --2- 172.21.15.31:0/3384178582 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc43c066f30 0x7fc43c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:52.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.847+0000 7fc44e7fc700 1 --2- 172.21.15.31:0/3384178582 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc43c066f30 0x7fc43c0693e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc4500cbf30 tx=0x7fc444009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:52.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:52.850+0000 7fc436ffd700 1 -- 172.21.15.31:0/3384178582 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc440051b50 con 0x7fc45006ed40 2023-10-06T02:23:53.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.135+0000 7fc44ffff700 1 -- 172.21.15.31:0/3384178582 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade"]} v 0) v1 -- 0x7fc45016e5d0 con 0x7fc45006ed40 2023-10-06T02:23:53.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.224+0000 7fc436ffd700 1 -- 172.21.15.31:0/3384178582 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade"]}]=0 v69) v1 ==== 161+0+0 (secure 0 0 0) 0x7fc440055170 con 0x7fc45006ed40 2023-10-06T02:23:53.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.226+0000 7fc44ffff700 1 -- 172.21.15.31:0/3384178582 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc43c066f30 msgr2=0x7fc43c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:53.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.226+0000 7fc44ffff700 1 --2- 172.21.15.31:0/3384178582 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc43c066f30 0x7fc43c0693e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc4500cbf30 tx=0x7fc444009040 comp rx=0 tx=0).stop 2023-10-06T02:23:53.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.226+0000 7fc44ffff700 1 -- 172.21.15.31:0/3384178582 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc45006ed40 msgr2=0x7fc4501594a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:53.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.226+0000 7fc44ffff700 1 --2- 172.21.15.31:0/3384178582 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc45006ed40 0x7fc4501594a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc44000ec80 tx=0x7fc4400091a0 comp rx=0 tx=0).stop 2023-10-06T02:23:53.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.226+0000 7fc44ffff700 1 -- 172.21.15.31:0/3384178582 shutdown_connections 2023-10-06T02:23:53.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.226+0000 7fc44ffff700 1 --2- 172.21.15.31:0/3384178582 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc43c066f30 0x7fc43c0693e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:53.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.226+0000 7fc44ffff700 1 --2- 172.21.15.31:0/3384178582 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc450156860 0x7fc450150ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:53.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.226+0000 7fc44ffff700 1 --2- 172.21.15.31:0/3384178582 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc45006ed40 0x7fc4501594a0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:53.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.226+0000 7fc44ffff700 1 --2- 172.21.15.31:0/3384178582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc450153d70 0x7fc45006ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:53.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.226+0000 7fc44ffff700 1 -- 172.21.15.31:0/3384178582 >> 172.21.15.31:0/3384178582 conn(0x7fc45006d170 msgr2=0x7fc4500ac9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:53.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.226+0000 7fc44ffff700 1 -- 172.21.15.31:0/3384178582 shutdown_connections 2023-10-06T02:23:53.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.226+0000 7fc44ffff700 1 -- 172.21.15.31:0/3384178582 wait complete. 2023-10-06T02:23:53.244 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T02:23:53.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.433+0000 7f453d19d700 1 Processor -- start 2023-10-06T02:23:53.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.433+0000 7f453d19d700 1 -- start start 2023-10-06T02:23:53.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.434+0000 7f453d19d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f45380cf110 0x7f453815de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:53.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.434+0000 7f453d19d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f453815ac50 con 0x7f4538153dd0 2023-10-06T02:23:53.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.434+0000 7f453d19d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f453815d010 con 0x7f4538153a00 2023-10-06T02:23:53.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.434+0000 7f453d19d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f453815af60 con 0x7f45380cf110 2023-10-06T02:23:53.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.435+0000 7f452f7fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f4538153dd0 0x7f45380cea00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49392/0 (socket says 172.21.15.31:49392) 2023-10-06T02:23:53.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.435+0000 7f452f7fe700 1 -- 172.21.15.31:0/1861040834 learned_addr learned my addr 172.21.15.31:0/1861040834 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:23:53.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.435+0000 7f453c99c700 1 --2- 172.21.15.31:0/1861040834 >> v2:172.21.15.62:3300/0 conn(0x7f45380cf110 0x7f453815de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:53.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.435+0000 7f45377fe700 1 -- 172.21.15.31:0/1861040834 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1063042131 0 0) 0x7f453815ac50 con 0x7f4538153dd0 2023-10-06T02:23:53.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.435+0000 7f45377fe700 1 -- 172.21.15.31:0/1861040834 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4524003570 con 0x7f4538153dd0 2023-10-06T02:23:53.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.435+0000 7f45377fe700 1 -- 172.21.15.31:0/1861040834 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1935339203 0 0) 0x7f453815d010 con 0x7f4538153a00 2023-10-06T02:23:53.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.435+0000 7f45377fe700 1 -- 172.21.15.31:0/1861040834 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f453815ac50 con 0x7f4538153a00 2023-10-06T02:23:53.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.435+0000 7f45377fe700 1 -- 172.21.15.31:0/1861040834 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4065063437 0 0) 0x7f4524003570 con 0x7f4538153dd0 2023-10-06T02:23:53.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.435+0000 7f45377fe700 1 -- 172.21.15.31:0/1861040834 >> v2:172.21.15.62:3300/0 conn(0x7f45380cf110 msgr2=0x7f453815de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:53.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.435+0000 7f45377fe700 1 --2- 172.21.15.31:0/1861040834 >> v2:172.21.15.62:3300/0 conn(0x7f45380cf110 0x7f453815de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:53.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.435+0000 7f45377fe700 1 -- 172.21.15.31:0/1861040834 >> v1:172.21.15.62:6789/0 conn(0x7f4538153a00 legacy=0x7f453815a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:53.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.435+0000 7f45377fe700 1 -- 172.21.15.31:0/1861040834 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f453815b240 con 0x7f4538153dd0 2023-10-06T02:23:53.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.435+0000 7f45377fe700 1 -- 172.21.15.31:0/1861040834 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7f4520001470 con 0x7f4538153dd0 2023-10-06T02:23:53.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.435+0000 7f45377fe700 1 -- 172.21.15.31:0/1861040834 >> v1:172.21.15.31:6789/0 conn(0x7f4538153dd0 legacy=0x7f45380cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:53.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.436+0000 7f45377fe700 1 --2- 172.21.15.31:0/1861040834 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4524003c40 0x7f4524004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:53.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.436+0000 7f45377fe700 1 --2- 172.21.15.31:0/1861040834 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4524004810 0x7f4524004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:53.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.436+0000 7f45377fe700 1 --2- 172.21.15.31:0/1861040834 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4524005210 0x7f452400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:53.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.436+0000 7f45377fe700 1 -- 172.21.15.31:0/1861040834 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f453815af60 con 0x7f4524004810 2023-10-06T02:23:53.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.436+0000 7f45377fe700 1 -- 172.21.15.31:0/1861040834 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f452400c1b0 con 0x7f4524005210 2023-10-06T02:23:53.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.436+0000 7f45377fe700 1 -- 172.21.15.31:0/1861040834 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f452400c370 con 0x7f4524003c40 2023-10-06T02:23:53.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.436+0000 7f452f7fe700 1 --2- 172.21.15.31:0/1861040834 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4524004810 0x7f4524004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:53.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.436+0000 7f4537fff700 1 --2- 172.21.15.31:0/1861040834 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4524003c40 0x7f4524004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:53.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.436+0000 7f453c99c700 1 --2- 172.21.15.31:0/1861040834 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4524005210 0x7f452400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:53.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.436+0000 7f452f7fe700 1 -- 172.21.15.31:0/1861040834 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4524003c40 msgr2=0x7f4524004030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:53.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.436+0000 7f452f7fe700 1 --2- 172.21.15.31:0/1861040834 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4524003c40 0x7f4524004030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:53.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.436+0000 7f452f7fe700 1 -- 172.21.15.31:0/1861040834 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4524005210 msgr2=0x7f452400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:53.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.436+0000 7f452f7fe700 1 --2- 172.21.15.31:0/1861040834 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4524005210 0x7f452400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:53.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.436+0000 7f452f7fe700 1 -- 172.21.15.31:0/1861040834 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4524003570 con 0x7f4524004810 2023-10-06T02:23:53.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.436+0000 7f453c99c700 1 -- 172.21.15.31:0/1861040834 reap_dead start 2023-10-06T02:23:53.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.437+0000 7f452f7fe700 1 --2- 172.21.15.31:0/1861040834 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4524004810 0x7f4524004c80 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f452000c1e0 tx=0x7f4520001d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d34ec6e1e48e8c91 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:53.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.437+0000 7f45377fe700 1 -- 172.21.15.31:0/1861040834 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f452000cba0 con 0x7f4524004810 2023-10-06T02:23:53.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.437+0000 7f45377fe700 1 -- 172.21.15.31:0/1861040834 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f452000cd00 con 0x7f4524004810 2023-10-06T02:23:53.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.437+0000 7f45377fe700 1 -- 172.21.15.31:0/1861040834 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4520005b70 con 0x7f4524004810 2023-10-06T02:23:53.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.437+0000 7f453d19d700 1 -- 172.21.15.31:0/1861040834 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4524004810 msgr2=0x7f4524004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:53.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.437+0000 7f453d19d700 1 --2- 172.21.15.31:0/1861040834 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4524004810 0x7f4524004c80 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f452000c1e0 tx=0x7f4520001d40 comp rx=0 tx=0).stop 2023-10-06T02:23:53.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.437+0000 7f453d19d700 1 -- 172.21.15.31:0/1861040834 shutdown_connections 2023-10-06T02:23:53.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.437+0000 7f453d19d700 1 --2- 172.21.15.31:0/1861040834 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4524004810 0x7f4524004c80 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:53.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.437+0000 7f453d19d700 1 -- 172.21.15.31:0/1861040834 >> 172.21.15.31:0/1861040834 conn(0x7f45380703a0 msgr2=0x7f45380707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:53.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.437+0000 7f453d19d700 1 -- 172.21.15.31:0/1861040834 shutdown_connections 2023-10-06T02:23:53.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.438+0000 7f453d19d700 1 -- 172.21.15.31:0/1861040834 wait complete. 2023-10-06T02:23:53.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.439+0000 7f453d19d700 1 Processor -- start 2023-10-06T02:23:53.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.439+0000 7f453d19d700 1 -- start start 2023-10-06T02:23:53.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.440+0000 7f453d19d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4524004810 0x7f4538158770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:53.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.440+0000 7f453d19d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45380c75e0 0x7f4538152300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:53.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.440+0000 7f453d19d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45380c7a50 0x7f4538152840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:53.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.440+0000 7f453d19d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4538152d80 con 0x7f45380c7a50 2023-10-06T02:23:53.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.440+0000 7f453d19d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4538151260 con 0x7f45380c75e0 2023-10-06T02:23:53.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.440+0000 7f453d19d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4538151540 con 0x7f4524004810 2023-10-06T02:23:53.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.440+0000 7f453c99c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45380c7a50 0x7f4538152840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:53.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.440+0000 7f453c99c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45380c7a50 0x7f4538152840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39058/0 (socket says 172.21.15.31:39058) 2023-10-06T02:23:53.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.440+0000 7f453c99c700 1 -- 172.21.15.31:0/3561823927 learned_addr learned my addr 172.21.15.31:0/3561823927 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:53.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.441+0000 7f452f7fe700 1 --2- 172.21.15.31:0/3561823927 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45380c75e0 0x7f4538152300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:53.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.441+0000 7f4537fff700 1 --2- 172.21.15.31:0/3561823927 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4524004810 0x7f4538158770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:53.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.441+0000 7f453c99c700 1 -- 172.21.15.31:0/3561823927 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4524004810 msgr2=0x7f4538158770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:53.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.441+0000 7f453c99c700 1 --2- 172.21.15.31:0/3561823927 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4524004810 0x7f4538158770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:53.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.441+0000 7f453c99c700 1 -- 172.21.15.31:0/3561823927 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45380c75e0 msgr2=0x7f4538152300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:53.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.441+0000 7f453c99c700 1 --2- 172.21.15.31:0/3561823927 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45380c75e0 0x7f4538152300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:53.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.441+0000 7f453c99c700 1 -- 172.21.15.31:0/3561823927 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4538151820 con 0x7f45380c7a50 2023-10-06T02:23:53.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.441+0000 7f453c99c700 1 --2- 172.21.15.31:0/3561823927 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45380c7a50 0x7f4538152840 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4530006f00 tx=0x7f45300091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:53.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.441+0000 7f4534ff9700 1 -- 172.21.15.31:0/3561823927 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4530011e80 con 0x7f45380c7a50 2023-10-06T02:23:53.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.441+0000 7f4534ff9700 1 -- 172.21.15.31:0/3561823927 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f453000f910 con 0x7f45380c7a50 2023-10-06T02:23:53.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.441+0000 7f4534ff9700 1 -- 172.21.15.31:0/3561823927 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453000fcd0 con 0x7f45380c7a50 2023-10-06T02:23:53.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.441+0000 7f453d19d700 1 -- 172.21.15.31:0/3561823927 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4538151b00 con 0x7f45380c7a50 2023-10-06T02:23:53.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.441+0000 7f453d19d700 1 -- 172.21.15.31:0/3561823927 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4538158cb0 con 0x7f45380c7a50 2023-10-06T02:23:53.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.442+0000 7f453d19d700 1 -- 172.21.15.31:0/3561823927 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4538152020 con 0x7f45380c7a50 2023-10-06T02:23:53.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.443+0000 7f4534ff9700 1 -- 172.21.15.31:0/3561823927 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4530014650 con 0x7f45380c7a50 2023-10-06T02:23:53.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.444+0000 7f4534ff9700 1 --2- 172.21.15.31:0/3561823927 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f4518066f30 0x7f45180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:53.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.444+0000 7f4534ff9700 1 -- 172.21.15.31:0/3561823927 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f4530084c90 con 0x7f45380c7a50 2023-10-06T02:23:53.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.444+0000 7f4537fff700 1 --2- 172.21.15.31:0/3561823927 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f4518066f30 0x7f45180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:53.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.445+0000 7f4537fff700 1 --2- 172.21.15.31:0/3561823927 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f4518066f30 0x7f45180693e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4528000f80 tx=0x7f45280017a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:53.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.449+0000 7f4534ff9700 1 -- 172.21.15.31:0/3561823927 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4538152020 con 0x7f45380c7a50 2023-10-06T02:23:53.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.651+0000 7f453d19d700 1 -- 172.21.15.31:0/3561823927 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f4538152020 con 0x7f45380c7a50 2023-10-06T02:23:53.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.655+0000 7f4534ff9700 1 -- 172.21.15.31:0/3561823927 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4538152020 con 0x7f45380c7a50 2023-10-06T02:23:53.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.657+0000 7f453d19d700 1 -- 172.21.15.31:0/3561823927 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f4518066f30 msgr2=0x7f45180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:53.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.657+0000 7f453d19d700 1 --2- 172.21.15.31:0/3561823927 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f4518066f30 0x7f45180693e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4528000f80 tx=0x7f45280017a0 comp rx=0 tx=0).stop 2023-10-06T02:23:53.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.657+0000 7f453d19d700 1 -- 172.21.15.31:0/3561823927 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45380c7a50 msgr2=0x7f4538152840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:53.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.657+0000 7f453d19d700 1 --2- 172.21.15.31:0/3561823927 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45380c7a50 0x7f4538152840 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4530006f00 tx=0x7f45300091a0 comp rx=0 tx=0).stop 2023-10-06T02:23:53.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.657+0000 7f453d19d700 1 -- 172.21.15.31:0/3561823927 shutdown_connections 2023-10-06T02:23:53.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.657+0000 7f453d19d700 1 --2- 172.21.15.31:0/3561823927 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45380c7a50 0x7f4538152840 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:53.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.657+0000 7f453d19d700 1 --2- 172.21.15.31:0/3561823927 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f4518066f30 0x7f45180693e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:53.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.657+0000 7f453d19d700 1 --2- 172.21.15.31:0/3561823927 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4524004810 0x7f4538158770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:53.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.657+0000 7f453d19d700 1 --2- 172.21.15.31:0/3561823927 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45380c75e0 0x7f4538152300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:53.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.657+0000 7f453d19d700 1 -- 172.21.15.31:0/3561823927 >> 172.21.15.31:0/3561823927 conn(0x7f45380703a0 msgr2=0x7f453815ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:53.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.657+0000 7f453d19d700 1 -- 172.21.15.31:0/3561823927 shutdown_connections 2023-10-06T02:23:53.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:53.658+0000 7f453d19d700 1 -- 172.21.15.31:0/3561823927 wait complete. 2023-10-06T02:23:53.676 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T02:23:53.676 DEBUG:teuthology.orchestra.run.smithi031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T02:23:53.757 DEBUG:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T02:23:53.855 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T02:23:53.855 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T02:23:53.855 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:23:53.855 DEBUG:teuthology.orchestra.run.smithi031:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2023-10-06T02:23:53.951 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi031.front.sepia.ceph.com 2023-10-06T02:23:53.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:23:53.951 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T02:23:53.993 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-06T02:23:53.993 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:23:53.993 DEBUG:teuthology.orchestra.run.smithi062:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2023-10-06T02:23:54.100 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi062.front.sepia.ceph.com 2023-10-06T02:23:54.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:23:54.100 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T02:23:54.277 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2023-10-06T02:23:54.277 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:23:54.277 DEBUG:teuthology.orchestra.run.smithi031:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2023-10-06T02:23:54.357 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi031.front.sepia.ceph.com 2023-10-06T02:23:54.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:23:54.357 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-06T02:23:54.393 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2023-10-06T02:23:54.394 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:23:54.394 DEBUG:teuthology.orchestra.run.smithi062:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2023-10-06T02:23:54.474 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi062.front.sepia.ceph.com 2023-10-06T02:23:54.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:23:54.474 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-06T02:23:54.511 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T02:23:54.511 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:23:54.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:54.697+0000 7f59e959d700 1 Processor -- start 2023-10-06T02:23:54.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:54.697+0000 7f59e959d700 1 -- start start 2023-10-06T02:23:54.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:54.698+0000 7f59e959d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f59e415a8c0 0x7f59e406d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:54.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:54.698+0000 7f59e959d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f59e406ddd0 0x7f59e40cf0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:54.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:54.698+0000 7f59e959d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59e406e240 con 0x7f59e40b2980 2023-10-06T02:23:54.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:54.698+0000 7f59e959d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59e40cec30 con 0x7f59e415a8c0 2023-10-06T02:23:54.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:54.698+0000 7f59e959d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59e40ce930 con 0x7f59e406ddd0 2023-10-06T02:23:54.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:54.699+0000 7f59e37fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f59e415a8c0 0x7f59e406d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:54.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:54.699+0000 7f59e3fff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f59e40b2980 0x7f59e40cce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49396/0 (socket says 172.21.15.31:49396) 2023-10-06T02:23:54.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:54.699+0000 7f59e37fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f59e415a8c0 0x7f59e406d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39074/0 (socket says 172.21.15.31:39074) 2023-10-06T02:23:54.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:54.699+0000 7f59e3fff700 1 -- 172.21.15.31:0/3352439012 learned_addr learned my addr 172.21.15.31:0/3352439012 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:23:54.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:54.699+0000 7f59e8d9c700 1 --2- 172.21.15.31:0/3352439012 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f59e406ddd0 0x7f59e40cf0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:54.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:54.699+0000 7f59e37fe700 1 -- 172.21.15.31:0/3352439012 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f59e406ddd0 msgr2=0x7f59e40cf0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:54.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:54.699+0000 7f59e37fe700 1 --2- 172.21.15.31:0/3352439012 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f59e406ddd0 0x7f59e40cf0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:54.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:54.699+0000 7f59e37fe700 1 -- 172.21.15.31:0/3352439012 >> v1:172.21.15.31:6789/0 conn(0x7f59e40b2980 legacy=0x7f59e40cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:54.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:54.699+0000 7f59e37fe700 1 -- 172.21.15.31:0/3352439012 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59e40cd510 con 0x7f59e415a8c0 2023-10-06T02:23:54.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:54.699+0000 7f59e2ffd700 1 -- 172.21.15.31:0/3352439012 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 756975091 0 0) 0x7f59e406e240 con 0x7f59e40b2980 2023-10-06T02:23:54.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:54.699+0000 7f59e37fe700 1 --2- 172.21.15.31:0/3352439012 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59e415a8c0 0x7f59e406d890 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f59d80098e0 tx=0x7f59d80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e17dc6f08b43e42 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:54.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:54.700+0000 7f59e2ffd700 1 -- 172.21.15.31:0/3352439012 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d800b640 con 0x7f59e415a8c0 2023-10-06T02:23:54.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:54.700+0000 7f59e2ffd700 1 -- 172.21.15.31:0/3352439012 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59d800b7a0 con 0x7f59e415a8c0 2023-10-06T02:23:54.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:54.700+0000 7f59e2ffd700 1 -- 172.21.15.31:0/3352439012 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d800bb40 con 0x7f59e415a8c0 2023-10-06T02:23:54.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:54.700+0000 7f59e959d700 1 -- 172.21.15.31:0/3352439012 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59e415a8c0 msgr2=0x7f59e406d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:54.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:54.700+0000 7f59e959d700 1 --2- 172.21.15.31:0/3352439012 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59e415a8c0 0x7f59e406d890 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f59d80098e0 tx=0x7f59d80091a0 comp rx=0 tx=0).stop 2023-10-06T02:23:54.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:54.700+0000 7f59e959d700 1 -- 172.21.15.31:0/3352439012 shutdown_connections 2023-10-06T02:23:54.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:54.700+0000 7f59e959d700 1 --2- 172.21.15.31:0/3352439012 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f59e406ddd0 0x7f59e40cf0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:54.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:54.700+0000 7f59e959d700 1 --2- 172.21.15.31:0/3352439012 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59e415a8c0 0x7f59e406d890 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:54.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:54.700+0000 7f59e959d700 1 -- 172.21.15.31:0/3352439012 >> 172.21.15.31:0/3352439012 conn(0x7f59e40703a0 msgr2=0x7f59e40707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:54.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:54.700+0000 7f59e959d700 1 -- 172.21.15.31:0/3352439012 shutdown_connections 2023-10-06T02:23:54.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:54.700+0000 7f59e959d700 1 -- 172.21.15.31:0/3352439012 wait complete. 2023-10-06T02:23:54.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:54.702+0000 7f59e959d700 1 Processor -- start 2023-10-06T02:23:54.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:54.702+0000 7f59e959d700 1 -- start start 2023-10-06T02:23:54.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:54.703+0000 7f59e959d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59e406ddd0 0x7f59e416cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:54.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:54.703+0000 7f59e959d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f59e40b2980 0x7f59e40b5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:54.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:54.703+0000 7f59e959d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59e415a8c0 0x7f59e40c9870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:54.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:54.703+0000 7f59e959d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e40c9db0 con 0x7f59e406ddd0 2023-10-06T02:23:54.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:54.703+0000 7f59e959d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e416ddf0 con 0x7f59e415a8c0 2023-10-06T02:23:54.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:54.703+0000 7f59e959d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59e416d550 con 0x7f59e40b2980 2023-10-06T02:23:54.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:54.703+0000 7f59e3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59e406ddd0 0x7f59e416cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:54.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:54.703+0000 7f59e3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59e406ddd0 0x7f59e416cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39080/0 (socket says 172.21.15.31:39080) 2023-10-06T02:23:54.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:54.703+0000 7f59e3fff700 1 -- 172.21.15.31:0/3676006053 learned_addr learned my addr 172.21.15.31:0/3676006053 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:54.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:54.703+0000 7f59e8d9c700 1 --2- 172.21.15.31:0/3676006053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59e415a8c0 0x7f59e40c9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:54.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:54.703+0000 7f59e37fe700 1 --2- 172.21.15.31:0/3676006053 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f59e40b2980 0x7f59e40b5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:54.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:54.703+0000 7f59e3fff700 1 -- 172.21.15.31:0/3676006053 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f59e40b2980 msgr2=0x7f59e40b5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:54.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:54.703+0000 7f59e3fff700 1 --2- 172.21.15.31:0/3676006053 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f59e40b2980 0x7f59e40b5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:54.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:54.704+0000 7f59e3fff700 1 -- 172.21.15.31:0/3676006053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59e415a8c0 msgr2=0x7f59e40c9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:54.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:54.704+0000 7f59e3fff700 1 --2- 172.21.15.31:0/3676006053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59e415a8c0 0x7f59e40c9870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:54.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:54.704+0000 7f59e3fff700 1 -- 172.21.15.31:0/3676006053 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59d8009420 con 0x7f59e406ddd0 2023-10-06T02:23:54.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:54.704+0000 7f59e37fe700 1 --2- 172.21.15.31:0/3676006053 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f59e40b2980 0x7f59e40b5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:23:54.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:54.704+0000 7f59e3fff700 1 --2- 172.21.15.31:0/3676006053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59e406ddd0 0x7f59e416cf70 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f59d400c7e0 tx=0x7f59d4007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:54.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:54.704+0000 7f59cbfff700 1 -- 172.21.15.31:0/3676006053 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d4005b30 con 0x7f59e406ddd0 2023-10-06T02:23:54.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:54.704+0000 7f59cbfff700 1 -- 172.21.15.31:0/3676006053 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59d4005c90 con 0x7f59e406ddd0 2023-10-06T02:23:54.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:54.704+0000 7f59cbfff700 1 -- 172.21.15.31:0/3676006053 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d4007300 con 0x7f59e406ddd0 2023-10-06T02:23:54.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:54.704+0000 7f59e959d700 1 -- 172.21.15.31:0/3676006053 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59e416d830 con 0x7f59e406ddd0 2023-10-06T02:23:54.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:54.704+0000 7f59e959d700 1 -- 172.21.15.31:0/3676006053 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59e40b43a0 con 0x7f59e406ddd0 2023-10-06T02:23:54.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:54.705+0000 7f59e959d700 1 -- 172.21.15.31:0/3676006053 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59e40b5930 con 0x7f59e406ddd0 2023-10-06T02:23:54.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:54.706+0000 7f59cbfff700 1 -- 172.21.15.31:0/3676006053 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f59d401b400 con 0x7f59e406ddd0 2023-10-06T02:23:54.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:54.707+0000 7f59cbfff700 1 --2- 172.21.15.31:0/3676006053 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f59d0066f30 0x7f59d00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:54.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:54.707+0000 7f59cbfff700 1 -- 172.21.15.31:0/3676006053 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f59d4084270 con 0x7f59e406ddd0 2023-10-06T02:23:54.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:54.707+0000 7f59e37fe700 1 --2- 172.21.15.31:0/3676006053 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f59d0066f30 0x7f59d00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:54.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:54.708+0000 7f59e37fe700 1 --2- 172.21.15.31:0/3676006053 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f59d0066f30 0x7f59d00693e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f59d80093f0 tx=0x7f59d8017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:54.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:54.711+0000 7f59cbfff700 1 -- 172.21.15.31:0/3676006053 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f59e40b5930 con 0x7f59e406ddd0 2023-10-06T02:23:54.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:54.961+0000 7f59e959d700 1 -- 172.21.15.31:0/3676006053 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f59e40b5930 con 0x7f59e406ddd0 2023-10-06T02:23:54.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:54.961+0000 7f59cbfff700 1 -- 172.21.15.31:0/3676006053 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4591 (secure 0 0 0) 0x7f59e40b5930 con 0x7f59e406ddd0 2023-10-06T02:23:54.963 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:23:54.963 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/1808327183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":1808327183},{"type":"v1","addr":"172.21.15.31:6837","nonce":1808327183}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/4263144517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":4263144517},{"type":"v1","addr":"172.21.15.62:6834","nonce":4263144517}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1489208568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":1489208568},{"type":"v1","addr":"172.21.15.62:6835","nonce":1489208568}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:23:09.521236+0000","modified":"2023-10-06T02:23:13.047274+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6836/3809284664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3809284664},{"type":"v1","addr":"172.21.15.31:6836","nonce":3809284664}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T02:23:54.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:54.964+0000 7f59e959d700 1 -- 172.21.15.31:0/3676006053 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f59d0066f30 msgr2=0x7f59d00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:54.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:54.964+0000 7f59e959d700 1 --2- 172.21.15.31:0/3676006053 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f59d0066f30 0x7f59d00693e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f59d80093f0 tx=0x7f59d8017000 comp rx=0 tx=0).stop 2023-10-06T02:23:54.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:54.964+0000 7f59e959d700 1 -- 172.21.15.31:0/3676006053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59e406ddd0 msgr2=0x7f59e416cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:54.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:54.964+0000 7f59e959d700 1 --2- 172.21.15.31:0/3676006053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59e406ddd0 0x7f59e416cf70 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f59d400c7e0 tx=0x7f59d4007df0 comp rx=0 tx=0).stop 2023-10-06T02:23:54.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:54.964+0000 7f59e959d700 1 -- 172.21.15.31:0/3676006053 shutdown_connections 2023-10-06T02:23:54.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:54.964+0000 7f59e959d700 1 --2- 172.21.15.31:0/3676006053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59e415a8c0 0x7f59e40c9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:54.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:54.964+0000 7f59e959d700 1 --2- 172.21.15.31:0/3676006053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59e406ddd0 0x7f59e416cf70 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:54.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:54.964+0000 7f59e959d700 1 --2- 172.21.15.31:0/3676006053 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f59d0066f30 0x7f59d00693e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:54.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:54.964+0000 7f59e959d700 1 --2- 172.21.15.31:0/3676006053 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f59e40b2980 0x7f59e40b5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:54.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:54.964+0000 7f59e959d700 1 -- 172.21.15.31:0/3676006053 >> 172.21.15.31:0/3676006053 conn(0x7f59e40703a0 msgr2=0x7f59e40cb130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:54.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:54.964+0000 7f59e959d700 1 -- 172.21.15.31:0/3676006053 shutdown_connections 2023-10-06T02:23:54.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:54.964+0000 7f59e959d700 1 -- 172.21.15.31:0/3676006053 wait complete. 2023-10-06T02:23:54.966 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 8 2023-10-06T02:23:54.983 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:23:55.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.169+0000 7f30389fe700 1 Processor -- start 2023-10-06T02:23:55.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.169+0000 7f30389fe700 1 -- start start 2023-10-06T02:23:55.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.170+0000 7f30389fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f30340ca410 0x7f30340c6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:55.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.170+0000 7f30389fe700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30340cb4f0 con 0x7f303415b5d0 2023-10-06T02:23:55.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.170+0000 7f30389fe700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f303415e390 con 0x7f30340c6ec0 2023-10-06T02:23:55.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.170+0000 7f30389fe700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30340c7340 con 0x7f30340ca410 2023-10-06T02:23:55.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.171+0000 7f303259c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f30340ca410 0x7f30340c6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:55.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.171+0000 7f3032d9d700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f303415b5d0 0x7f30340c9d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49412/0 (socket says 172.21.15.31:49412) 2023-10-06T02:23:55.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.171+0000 7f3032d9d700 1 -- 172.21.15.31:0/3745838838 learned_addr learned my addr 172.21.15.31:0/3745838838 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:23:55.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.171+0000 7f303259c700 1 -- 172.21.15.31:0/3745838838 >> v1:172.21.15.62:6789/0 conn(0x7f30340c6ec0 legacy=0x7f303415f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:55.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.171+0000 7f303259c700 1 -- 172.21.15.31:0/3745838838 >> v1:172.21.15.31:6789/0 conn(0x7f303415b5d0 legacy=0x7f30340c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:55.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.171+0000 7f303259c700 1 -- 172.21.15.31:0/3745838838 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30340ca8e0 con 0x7f30340ca410 2023-10-06T02:23:55.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.171+0000 7f3031d9b700 1 -- 172.21.15.31:0/3745838838 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 400667606 0 0) 0x7f30340cb4f0 con 0x7f303415b5d0 2023-10-06T02:23:55.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.171+0000 7f303259c700 1 --2- 172.21.15.31:0/3745838838 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30340ca410 0x7f30340c6980 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3028009900 tx=0x7f30280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94c9fea1dfb2e4fe server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:55.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.172+0000 7f3031d9b700 1 -- 172.21.15.31:0/3745838838 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3028009ea0 con 0x7f30340ca410 2023-10-06T02:23:55.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.172+0000 7f3031d9b700 1 -- 172.21.15.31:0/3745838838 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f302800b820 con 0x7f30340ca410 2023-10-06T02:23:55.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.172+0000 7f3031d9b700 1 -- 172.21.15.31:0/3745838838 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f302800bbc0 con 0x7f30340ca410 2023-10-06T02:23:55.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.172+0000 7f30389fe700 1 -- 172.21.15.31:0/3745838838 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30340ca410 msgr2=0x7f30340c6980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:55.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.172+0000 7f30389fe700 1 --2- 172.21.15.31:0/3745838838 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30340ca410 0x7f30340c6980 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3028009900 tx=0x7f30280091a0 comp rx=0 tx=0).stop 2023-10-06T02:23:55.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.172+0000 7f30389fe700 1 -- 172.21.15.31:0/3745838838 shutdown_connections 2023-10-06T02:23:55.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.172+0000 7f30389fe700 1 --2- 172.21.15.31:0/3745838838 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30340ca410 0x7f30340c6980 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:55.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.172+0000 7f30389fe700 1 -- 172.21.15.31:0/3745838838 >> 172.21.15.31:0/3745838838 conn(0x7f303406d170 msgr2=0x7f30340cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:55.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.172+0000 7f30389fe700 1 -- 172.21.15.31:0/3745838838 shutdown_connections 2023-10-06T02:23:55.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.172+0000 7f30389fe700 1 -- 172.21.15.31:0/3745838838 wait complete. 2023-10-06T02:23:55.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.174+0000 7f30389fe700 1 Processor -- start 2023-10-06T02:23:55.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.174+0000 7f30389fe700 1 -- start start 2023-10-06T02:23:55.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.175+0000 7f30389fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f30340c6ec0 0x7f303416ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:55.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.175+0000 7f30389fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30340ca410 0x7f30340ae6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:55.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.175+0000 7f30389fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f303415b5d0 0x7f3034158740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:55.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.175+0000 7f30389fe700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3034158c80 con 0x7f30340ca410 2023-10-06T02:23:55.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.175+0000 7f30389fe700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30340c65b0 con 0x7f303415b5d0 2023-10-06T02:23:55.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.175+0000 7f30389fe700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30340c62d0 con 0x7f30340c6ec0 2023-10-06T02:23:55.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.175+0000 7f303259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30340ca410 0x7f30340ae6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:55.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.175+0000 7f303259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30340ca410 0x7f30340ae6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39112/0 (socket says 172.21.15.31:39112) 2023-10-06T02:23:55.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.175+0000 7f303259c700 1 -- 172.21.15.31:0/3547708431 learned_addr learned my addr 172.21.15.31:0/3547708431 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:55.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.175+0000 7f3032d9d700 1 --2- 172.21.15.31:0/3547708431 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f30340c6ec0 0x7f303416ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:55.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.175+0000 7f303359e700 1 --2- 172.21.15.31:0/3547708431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f303415b5d0 0x7f3034158740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:55.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.176+0000 7f303259c700 1 -- 172.21.15.31:0/3547708431 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f30340c6ec0 msgr2=0x7f303416ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:55.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.176+0000 7f303259c700 1 --2- 172.21.15.31:0/3547708431 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f30340c6ec0 0x7f303416ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:55.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.176+0000 7f303259c700 1 -- 172.21.15.31:0/3547708431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f303415b5d0 msgr2=0x7f3034158740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:55.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.176+0000 7f303259c700 1 --2- 172.21.15.31:0/3547708431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f303415b5d0 0x7f3034158740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:55.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.176+0000 7f303259c700 1 -- 172.21.15.31:0/3547708431 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30280093c0 con 0x7f30340ca410 2023-10-06T02:23:55.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.176+0000 7f303259c700 1 --2- 172.21.15.31:0/3547708431 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30340ca410 0x7f30340ae6c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f3028013000 tx=0x7f3028017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:55.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.176+0000 7f301affd700 1 -- 172.21.15.31:0/3547708431 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f302800bba0 con 0x7f30340ca410 2023-10-06T02:23:55.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.176+0000 7f301affd700 1 -- 172.21.15.31:0/3547708431 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3028011420 con 0x7f30340ca410 2023-10-06T02:23:55.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.176+0000 7f301affd700 1 -- 172.21.15.31:0/3547708431 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30280117e0 con 0x7f30340ca410 2023-10-06T02:23:55.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.176+0000 7f30389fe700 1 -- 172.21.15.31:0/3547708431 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f303416e1f0 con 0x7f30340ca410 2023-10-06T02:23:55.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.176+0000 7f30389fe700 1 -- 172.21.15.31:0/3547708431 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f303416e7b0 con 0x7f30340ca410 2023-10-06T02:23:55.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.178+0000 7f30389fe700 1 -- 172.21.15.31:0/3547708431 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f303416e9a0 con 0x7f30340ca410 2023-10-06T02:23:55.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.179+0000 7f301affd700 1 -- 172.21.15.31:0/3547708431 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3028011b90 con 0x7f30340ca410 2023-10-06T02:23:55.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.179+0000 7f301affd700 1 --2- 172.21.15.31:0/3547708431 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f3020066f30 0x7f30200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:55.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.180+0000 7f301affd700 1 -- 172.21.15.31:0/3547708431 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f3028086160 con 0x7f30340ca410 2023-10-06T02:23:55.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.180+0000 7f3032d9d700 1 --2- 172.21.15.31:0/3547708431 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f3020066f30 0x7f30200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:55.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.180+0000 7f3032d9d700 1 --2- 172.21.15.31:0/3547708431 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f3020066f30 0x7f30200693e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f303406bb20 tx=0x7f3024007d10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:55.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.185+0000 7f301affd700 1 -- 172.21.15.31:0/3547708431 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3028053190 con 0x7f30340ca410 2023-10-06T02:23:55.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.390+0000 7f30389fe700 1 -- 172.21.15.31:0/3547708431 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f303404c3c0 con 0x7f30340ca410 2023-10-06T02:23:55.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.391+0000 7f301affd700 1 -- 172.21.15.31:0/3547708431 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+16914 (secure 0 0 0) 0x7f30280567b0 con 0x7f30340ca410 2023-10-06T02:23:55.393 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:23:55.393 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":25,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:23:48.196317+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:23:07.290269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T02:23:08.504679+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.2f","mappings":[{"from":3,"to":2}]},{"pgid":"4.31","mappings":[{"from":1,"to":5}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:23:55.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.393+0000 7f30389fe700 1 -- 172.21.15.31:0/3547708431 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f3020066f30 msgr2=0x7f30200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:55.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.393+0000 7f30389fe700 1 --2- 172.21.15.31:0/3547708431 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f3020066f30 0x7f30200693e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f303406bb20 tx=0x7f3024007d10 comp rx=0 tx=0).stop 2023-10-06T02:23:55.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.393+0000 7f30389fe700 1 -- 172.21.15.31:0/3547708431 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30340ca410 msgr2=0x7f30340ae6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:55.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.393+0000 7f30389fe700 1 --2- 172.21.15.31:0/3547708431 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30340ca410 0x7f30340ae6c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f3028013000 tx=0x7f3028017000 comp rx=0 tx=0).stop 2023-10-06T02:23:55.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.394+0000 7f30389fe700 1 -- 172.21.15.31:0/3547708431 shutdown_connections 2023-10-06T02:23:55.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.394+0000 7f30389fe700 1 --2- 172.21.15.31:0/3547708431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f303415b5d0 0x7f3034158740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:55.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.394+0000 7f30389fe700 1 --2- 172.21.15.31:0/3547708431 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f3020066f30 0x7f30200693e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:55.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.394+0000 7f30389fe700 1 --2- 172.21.15.31:0/3547708431 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f30340c6ec0 0x7f303416ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:55.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.394+0000 7f30389fe700 1 --2- 172.21.15.31:0/3547708431 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30340ca410 0x7f30340ae6c0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:55.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.394+0000 7f30389fe700 1 -- 172.21.15.31:0/3547708431 >> 172.21.15.31:0/3547708431 conn(0x7f303406d170 msgr2=0x7f30340c26b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:55.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.394+0000 7f30389fe700 1 -- 172.21.15.31:0/3547708431 shutdown_connections 2023-10-06T02:23:55.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.394+0000 7f30389fe700 1 -- 172.21.15.31:0/3547708431 wait complete. 2023-10-06T02:23:55.413 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T02:22:53.183425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T02:23:00.046967+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T02:23:07.290269+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T02:23:08.504679+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T02:23:55.413 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T02:23:55.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.601+0000 7f2ecf59e700 1 Processor -- start 2023-10-06T02:23:55.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.601+0000 7f2ecf59e700 1 -- start start 2023-10-06T02:23:55.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.602+0000 7f2ecf59e700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2ed0153c90 0x7f2ed015a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:55.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.602+0000 7f2ecf59e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2ed00caec0 0x7f2ed015afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:55.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.602+0000 7f2ecf59e700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ed015ada0 con 0x7f2ed006db20 2023-10-06T02:23:55.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.602+0000 7f2ecf59e700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ed006ca80 con 0x7f2ed0153c90 2023-10-06T02:23:55.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.602+0000 7f2ecf59e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed006cd60 con 0x7f2ed00caec0 2023-10-06T02:23:55.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.603+0000 7f2ecdd9b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f2ed006db20 0x7f2ed00ca7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49422/0 (socket says 172.21.15.31:49422) 2023-10-06T02:23:55.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.603+0000 7f2ecdd9b700 1 -- 172.21.15.31:0/396644455 learned_addr learned my addr 172.21.15.31:0/396644455 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:23:55.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.603+0000 7f2ece59c700 1 --2- 172.21.15.31:0/396644455 >> v2:172.21.15.62:3300/0 conn(0x7f2ed0153c90 0x7f2ed015a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:55.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.603+0000 7f2eced9d700 1 --2- 172.21.15.31:0/396644455 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2ed00caec0 0x7f2ed015afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:55.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.603+0000 7f2ecd59a700 1 -- 172.21.15.31:0/396644455 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2157680660 0 0) 0x7f2ec4001110 con 0x7f2ed006db20 2023-10-06T02:23:55.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.603+0000 7f2ecd59a700 1 -- 172.21.15.31:0/396644455 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2eb8003570 con 0x7f2ed006db20 2023-10-06T02:23:55.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.603+0000 7f2ecd59a700 1 -- 172.21.15.31:0/396644455 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4029976093 0 0) 0x7f2eb8003570 con 0x7f2ed006db20 2023-10-06T02:23:55.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.603+0000 7f2ecd59a700 1 -- 172.21.15.31:0/396644455 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2ed00caec0 msgr2=0x7f2ed015afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:55.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.603+0000 7f2ecd59a700 1 --2- 172.21.15.31:0/396644455 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2ed00caec0 0x7f2ed015afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:55.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.603+0000 7f2ecd59a700 1 -- 172.21.15.31:0/396644455 >> v2:172.21.15.62:3300/0 conn(0x7f2ed0153c90 msgr2=0x7f2ed015a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:55.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.603+0000 7f2ecd59a700 1 --2- 172.21.15.31:0/396644455 >> v2:172.21.15.62:3300/0 conn(0x7f2ed0153c90 0x7f2ed015a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:55.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.603+0000 7f2ecd59a700 1 -- 172.21.15.31:0/396644455 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ed006d040 con 0x7f2ed006db20 2023-10-06T02:23:55.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.603+0000 7f2ecd59a700 1 -- 172.21.15.31:0/396644455 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7f2ec4001600 con 0x7f2ed006db20 2023-10-06T02:23:55.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.603+0000 7f2ece59c700 1 --2- 172.21.15.31:0/396644455 >> v2:172.21.15.62:3300/0 conn(0x7f2ed0153c90 0x7f2ed015a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:23:55.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.604+0000 7f2ecd59a700 1 -- 172.21.15.31:0/396644455 >> v1:172.21.15.31:6789/0 conn(0x7f2ed006db20 legacy=0x7f2ed00ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:55.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.604+0000 7f2ecd59a700 1 --2- 172.21.15.31:0/396644455 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2eb8003a60 0x7f2eb8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:55.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.604+0000 7f2ecd59a700 1 --2- 172.21.15.31:0/396644455 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2ed00caec0 0x7f2eb8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:55.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.604+0000 7f2ecd59a700 1 --2- 172.21.15.31:0/396644455 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2eb8004d50 0x7f2eb80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:55.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.604+0000 7f2ecd59a700 1 -- 172.21.15.31:0/396644455 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed006ca80 con 0x7f2eb8004d50 2023-10-06T02:23:55.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.604+0000 7f2ecd59a700 1 -- 172.21.15.31:0/396644455 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed006cd60 con 0x7f2eb8003a60 2023-10-06T02:23:55.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.604+0000 7f2ecd59a700 1 -- 172.21.15.31:0/396644455 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb800ba60 con 0x7f2ed00caec0 2023-10-06T02:23:55.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.604+0000 7f2ece59c700 1 --2- 172.21.15.31:0/396644455 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2eb8003a60 0x7f2eb8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:55.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.604+0000 7f2eced9d700 1 --2- 172.21.15.31:0/396644455 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2eb8004d50 0x7f2eb80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:55.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.604+0000 7f2ecdd9b700 1 --2- 172.21.15.31:0/396644455 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2ed00caec0 0x7f2eb8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:55.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.604+0000 7f2eced9d700 1 -- 172.21.15.31:0/396644455 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2ed00caec0 msgr2=0x7f2eb8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:55.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.604+0000 7f2eced9d700 1 --2- 172.21.15.31:0/396644455 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2ed00caec0 0x7f2eb8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:55.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.604+0000 7f2eced9d700 1 -- 172.21.15.31:0/396644455 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2eb8003a60 msgr2=0x7f2eb8003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:55.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.604+0000 7f2eced9d700 1 --2- 172.21.15.31:0/396644455 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2eb8003a60 0x7f2eb8003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:55.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.604+0000 7f2eced9d700 1 -- 172.21.15.31:0/396644455 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eb8003570 con 0x7f2eb8004d50 2023-10-06T02:23:55.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.605+0000 7f2eced9d700 1 --2- 172.21.15.31:0/396644455 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2eb8004d50 0x7f2eb80051c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2ed00cc370 tx=0x7f2ec800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acbc8739e7b48324 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:55.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.605+0000 7f2ecd59a700 1 -- 172.21.15.31:0/396644455 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec800ea00 con 0x7f2eb8004d50 2023-10-06T02:23:55.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.605+0000 7f2ecd59a700 1 -- 172.21.15.31:0/396644455 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ec800f040 con 0x7f2eb8004d50 2023-10-06T02:23:55.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.605+0000 7f2ecd59a700 1 -- 172.21.15.31:0/396644455 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec800edb0 con 0x7f2eb8004d50 2023-10-06T02:23:55.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.605+0000 7f2ecf59e700 1 -- 172.21.15.31:0/396644455 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2eb8004d50 msgr2=0x7f2eb80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:55.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.605+0000 7f2ecf59e700 1 --2- 172.21.15.31:0/396644455 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2eb8004d50 0x7f2eb80051c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2ed00cc370 tx=0x7f2ec800b040 comp rx=0 tx=0).stop 2023-10-06T02:23:55.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.605+0000 7f2eced9d700 1 -- 172.21.15.31:0/396644455 reap_dead start 2023-10-06T02:23:55.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.605+0000 7f2ecf59e700 1 -- 172.21.15.31:0/396644455 shutdown_connections 2023-10-06T02:23:55.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.605+0000 7f2ecf59e700 1 -- 172.21.15.31:0/396644455 >> 172.21.15.31:0/396644455 conn(0x7f2ed006e720 msgr2=0x7f2ed006eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:55.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.606+0000 7f2ecf59e700 1 -- 172.21.15.31:0/396644455 shutdown_connections 2023-10-06T02:23:55.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.606+0000 7f2ecf59e700 1 -- 172.21.15.31:0/396644455 wait complete. 2023-10-06T02:23:55.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.607+0000 7f2ecf59e700 1 Processor -- start 2023-10-06T02:23:55.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.607+0000 7f2ecf59e700 1 -- start start 2023-10-06T02:23:55.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.608+0000 7f2ecf59e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2eb8004d50 0x7f2ed016aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:55.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.608+0000 7f2ecf59e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ed0158770 0x7f2ed0152220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:55.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.608+0000 7f2ecf59e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ed00c28b0 0x7f2ed00c2d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:55.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.608+0000 7f2ecf59e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed016b030 con 0x7f2ed0158770 2023-10-06T02:23:55.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.608+0000 7f2ecf59e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed0152c90 con 0x7f2ed00c28b0 2023-10-06T02:23:55.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.608+0000 7f2ecf59e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed00c3260 con 0x7f2eb8004d50 2023-10-06T02:23:55.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.608+0000 7f2ecdd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ed0158770 0x7f2ed0152220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:55.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.608+0000 7f2ecdd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ed0158770 0x7f2ed0152220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39120/0 (socket says 172.21.15.31:39120) 2023-10-06T02:23:55.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.608+0000 7f2ecdd9b700 1 -- 172.21.15.31:0/3138639784 learned_addr learned my addr 172.21.15.31:0/3138639784 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:55.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.609+0000 7f2ece59c700 1 --2- 172.21.15.31:0/3138639784 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2eb8004d50 0x7f2ed016aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:55.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.609+0000 7f2eced9d700 1 --2- 172.21.15.31:0/3138639784 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ed00c28b0 0x7f2ed00c2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:55.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.609+0000 7f2ecdd9b700 1 -- 172.21.15.31:0/3138639784 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2eb8004d50 msgr2=0x7f2ed016aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:55.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.609+0000 7f2ecdd9b700 1 --2- 172.21.15.31:0/3138639784 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2eb8004d50 0x7f2ed016aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:55.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.609+0000 7f2ecdd9b700 1 -- 172.21.15.31:0/3138639784 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ed00c28b0 msgr2=0x7f2ed00c2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:55.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.609+0000 7f2ecdd9b700 1 --2- 172.21.15.31:0/3138639784 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ed00c28b0 0x7f2ed00c2d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:55.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.609+0000 7f2ecdd9b700 1 -- 172.21.15.31:0/3138639784 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ed00b04a0 con 0x7f2ed0158770 2023-10-06T02:23:55.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.609+0000 7f2ece59c700 1 --2- 172.21.15.31:0/3138639784 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2eb8004d50 0x7f2ed016aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:23:55.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.609+0000 7f2ecdd9b700 1 --2- 172.21.15.31:0/3138639784 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ed0158770 0x7f2ed0152220 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2ec400e880 tx=0x7f2ec4000d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:55.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.609+0000 7f2ebe7fc700 1 -- 172.21.15.31:0/3138639784 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec400a580 con 0x7f2ed0158770 2023-10-06T02:23:55.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.609+0000 7f2ebe7fc700 1 -- 172.21.15.31:0/3138639784 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ec4010040 con 0x7f2ed0158770 2023-10-06T02:23:55.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.609+0000 7f2ecf59e700 1 -- 172.21.15.31:0/3138639784 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ed00af890 con 0x7f2ed0158770 2023-10-06T02:23:55.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.609+0000 7f2ebe7fc700 1 -- 172.21.15.31:0/3138639784 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec400a950 con 0x7f2ed0158770 2023-10-06T02:23:55.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.609+0000 7f2ecf59e700 1 -- 172.21.15.31:0/3138639784 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ed00afe50 con 0x7f2ed0158770 2023-10-06T02:23:55.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.610+0000 7f2ecf59e700 1 -- 172.21.15.31:0/3138639784 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ed00b0630 con 0x7f2ed0158770 2023-10-06T02:23:55.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.611+0000 7f2ebe7fc700 1 -- 172.21.15.31:0/3138639784 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2ec400acc0 con 0x7f2ed0158770 2023-10-06T02:23:55.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.613+0000 7f2ebe7fc700 1 --2- 172.21.15.31:0/3138639784 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f2eb0066f60 0x7f2eb0069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:55.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.613+0000 7f2ebe7fc700 1 -- 172.21.15.31:0/3138639784 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f2ec4083a50 con 0x7f2ed0158770 2023-10-06T02:23:55.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.613+0000 7f2ece59c700 1 --2- 172.21.15.31:0/3138639784 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f2eb0066f60 0x7f2eb0069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:55.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.614+0000 7f2ece59c700 1 --2- 172.21.15.31:0/3138639784 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f2eb0066f60 0x7f2eb0069410 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2ec0006f00 tx=0x7f2ec0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:55.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.617+0000 7f2ebe7fc700 1 -- 172.21.15.31:0/3138639784 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2ec4050a80 con 0x7f2ed0158770 2023-10-06T02:23:55.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.823+0000 7f2ecf59e700 1 -- 172.21.15.31:0/3138639784 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2ed015ada0 con 0x7f2ed0158770 2023-10-06T02:23:55.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.824+0000 7f2ebe7fc700 1 -- 172.21.15.31:0/3138639784 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2ed015ada0 con 0x7f2ed0158770 2023-10-06T02:23:55.825 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-10-06T02:23:55.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.826+0000 7f2ecf59e700 1 -- 172.21.15.31:0/3138639784 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f2eb0066f60 msgr2=0x7f2eb0069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:55.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.826+0000 7f2ecf59e700 1 --2- 172.21.15.31:0/3138639784 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f2eb0066f60 0x7f2eb0069410 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2ec0006f00 tx=0x7f2ec0008040 comp rx=0 tx=0).stop 2023-10-06T02:23:55.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.826+0000 7f2ecf59e700 1 -- 172.21.15.31:0/3138639784 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ed0158770 msgr2=0x7f2ed0152220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:55.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.826+0000 7f2ecf59e700 1 --2- 172.21.15.31:0/3138639784 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ed0158770 0x7f2ed0152220 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2ec400e880 tx=0x7f2ec4000d70 comp rx=0 tx=0).stop 2023-10-06T02:23:55.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.826+0000 7f2ecf59e700 1 -- 172.21.15.31:0/3138639784 shutdown_connections 2023-10-06T02:23:55.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.826+0000 7f2ecf59e700 1 --2- 172.21.15.31:0/3138639784 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ed00c28b0 0x7f2ed00c2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:55.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.826+0000 7f2ecf59e700 1 --2- 172.21.15.31:0/3138639784 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f2eb0066f60 0x7f2eb0069410 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:55.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.826+0000 7f2ecf59e700 1 --2- 172.21.15.31:0/3138639784 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2eb8004d50 0x7f2ed016aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:55.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.826+0000 7f2ecf59e700 1 --2- 172.21.15.31:0/3138639784 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ed0158770 0x7f2ed0152220 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:55.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.827+0000 7f2ecf59e700 1 -- 172.21.15.31:0/3138639784 >> 172.21.15.31:0/3138639784 conn(0x7f2ed006e720 msgr2=0x7f2ed015c230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:55.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.827+0000 7f2ecf59e700 1 -- 172.21.15.31:0/3138639784 shutdown_connections 2023-10-06T02:23:55.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:55.827+0000 7f2ecf59e700 1 -- 172.21.15.31:0/3138639784 wait complete. 2023-10-06T02:23:55.845 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T02:23:56.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.033+0000 7f3fe979d700 1 Processor -- start 2023-10-06T02:23:56.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.033+0000 7f3fe979d700 1 -- start start 2023-10-06T02:23:56.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.035+0000 7f3fe979d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3fe415dfd0 0x7f3fe40aedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:56.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.035+0000 7f3fe979d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3fe4159920 0x7f3fe40ad090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:56.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.035+0000 7f3fe979d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fe40acc00 con 0x7f3fe4159550 2023-10-06T02:23:56.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.035+0000 7f3fe979d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fe40abff0 con 0x7f3fe415dfd0 2023-10-06T02:23:56.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.035+0000 7f3fe979d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe40ac2d0 con 0x7f3fe4159920 2023-10-06T02:23:56.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.035+0000 7f3fe3fff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f3fe4159550 0x7f3fe40ceac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49426/0 (socket says 172.21.15.31:49426) 2023-10-06T02:23:56.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.035+0000 7f3fe3fff700 1 -- 172.21.15.31:0/636777730 learned_addr learned my addr 172.21.15.31:0/636777730 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:23:56.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.035+0000 7f3fe37fe700 1 --2- 172.21.15.31:0/636777730 >> v2:172.21.15.62:3300/0 conn(0x7f3fe415dfd0 0x7f3fe40aedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:56.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.035+0000 7f3fe8f9c700 1 --2- 172.21.15.31:0/636777730 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3fe4159920 0x7f3fe40ad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:56.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.035+0000 7f3fe2ffd700 1 -- 172.21.15.31:0/636777730 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1737169014 0 0) 0x7f3fe40acc00 con 0x7f3fe4159550 2023-10-06T02:23:56.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.036+0000 7f3fe2ffd700 1 -- 172.21.15.31:0/636777730 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3fcc003570 con 0x7f3fe4159550 2023-10-06T02:23:56.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.036+0000 7f3fe8f9c700 1 -- 172.21.15.31:0/636777730 >> v2:172.21.15.62:3300/0 conn(0x7f3fe415dfd0 msgr2=0x7f3fe40aedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:56.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.036+0000 7f3fe8f9c700 1 --2- 172.21.15.31:0/636777730 >> v2:172.21.15.62:3300/0 conn(0x7f3fe415dfd0 0x7f3fe40aedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:56.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.036+0000 7f3fe8f9c700 1 -- 172.21.15.31:0/636777730 >> v1:172.21.15.31:6789/0 conn(0x7f3fe4159550 legacy=0x7f3fe40ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:56.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.036+0000 7f3fe8f9c700 1 -- 172.21.15.31:0/636777730 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fe40ac5b0 con 0x7f3fe4159920 2023-10-06T02:23:56.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.036+0000 7f3fe37fe700 1 --2- 172.21.15.31:0/636777730 >> v2:172.21.15.62:3300/0 conn(0x7f3fe415dfd0 0x7f3fe40aedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:23:56.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.036+0000 7f3fe8f9c700 1 --2- 172.21.15.31:0/636777730 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3fe4159920 0x7f3fe40ad090 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc009e60 tx=0x7f3fdc009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19a79ac3aea564a0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:56.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.036+0000 7f3fe2ffd700 1 -- 172.21.15.31:0/636777730 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc00e070 con 0x7f3fe4159920 2023-10-06T02:23:56.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.037+0000 7f3fe2ffd700 1 -- 172.21.15.31:0/636777730 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fdc00bab0 con 0x7f3fe4159920 2023-10-06T02:23:56.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.037+0000 7f3fe2ffd700 1 -- 172.21.15.31:0/636777730 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc00be70 con 0x7f3fe4159920 2023-10-06T02:23:56.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.037+0000 7f3fe979d700 1 -- 172.21.15.31:0/636777730 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3fe4159920 msgr2=0x7f3fe40ad090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:56.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.037+0000 7f3fe979d700 1 --2- 172.21.15.31:0/636777730 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3fe4159920 0x7f3fe40ad090 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc009e60 tx=0x7f3fdc009780 comp rx=0 tx=0).stop 2023-10-06T02:23:56.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.037+0000 7f3fe979d700 1 -- 172.21.15.31:0/636777730 shutdown_connections 2023-10-06T02:23:56.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.037+0000 7f3fe979d700 1 --2- 172.21.15.31:0/636777730 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3fe4159920 0x7f3fe40ad090 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:56.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.037+0000 7f3fe979d700 1 --2- 172.21.15.31:0/636777730 >> v2:172.21.15.62:3300/0 conn(0x7f3fe415dfd0 0x7f3fe40aedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:56.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.037+0000 7f3fe979d700 1 -- 172.21.15.31:0/636777730 >> 172.21.15.31:0/636777730 conn(0x7f3fe406ce80 msgr2=0x7f3fe40ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:56.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.037+0000 7f3fe979d700 1 -- 172.21.15.31:0/636777730 shutdown_connections 2023-10-06T02:23:56.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.037+0000 7f3fe979d700 1 -- 172.21.15.31:0/636777730 wait complete. 2023-10-06T02:23:56.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.039+0000 7f3fe979d700 1 Processor -- start 2023-10-06T02:23:56.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.039+0000 7f3fe979d700 1 -- start start 2023-10-06T02:23:56.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.040+0000 7f3fe979d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3fe4159550 0x7f3fe4158310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:56.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.040+0000 7f3fe979d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3fe4159920 0x7f3fe4157760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:56.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.040+0000 7f3fe979d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3fe415dfd0 0x7f3fe4157ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:56.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.040+0000 7f3fe979d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe40c94b0 con 0x7f3fe4159550 2023-10-06T02:23:56.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.040+0000 7f3fe979d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe40c9790 con 0x7f3fe4159920 2023-10-06T02:23:56.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.040+0000 7f3fe979d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe40c9a70 con 0x7f3fe415dfd0 2023-10-06T02:23:56.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.040+0000 7f3fe3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3fe4159550 0x7f3fe4158310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:56.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.040+0000 7f3fe3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3fe4159550 0x7f3fe4158310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39132/0 (socket says 172.21.15.31:39132) 2023-10-06T02:23:56.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.040+0000 7f3fe3fff700 1 -- 172.21.15.31:0/1849174054 learned_addr learned my addr 172.21.15.31:0/1849174054 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:56.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.040+0000 7f3fe8f9c700 1 --2- 172.21.15.31:0/1849174054 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3fe415dfd0 0x7f3fe4157ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:56.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.040+0000 7f3fe37fe700 1 --2- 172.21.15.31:0/1849174054 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3fe4159920 0x7f3fe4157760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:56.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.040+0000 7f3fe3fff700 1 -- 172.21.15.31:0/1849174054 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3fe415dfd0 msgr2=0x7f3fe4157ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:56.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.040+0000 7f3fe3fff700 1 --2- 172.21.15.31:0/1849174054 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3fe415dfd0 0x7f3fe4157ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:56.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.040+0000 7f3fe3fff700 1 -- 172.21.15.31:0/1849174054 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3fe4159920 msgr2=0x7f3fe4157760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:56.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.040+0000 7f3fe3fff700 1 --2- 172.21.15.31:0/1849174054 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3fe4159920 0x7f3fe4157760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:56.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.040+0000 7f3fe3fff700 1 -- 172.21.15.31:0/1849174054 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fd8009420 con 0x7f3fe4159550 2023-10-06T02:23:56.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.040+0000 7f3fe37fe700 1 --2- 172.21.15.31:0/1849174054 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3fe4159920 0x7f3fe4157760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:23:56.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.041+0000 7f3fe3fff700 1 --2- 172.21.15.31:0/1849174054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3fe4159550 0x7f3fe4158310 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3fd400c7c0 tx=0x7f3fd4007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:56.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.041+0000 7f3fcbfff700 1 -- 172.21.15.31:0/1849174054 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd4005b30 con 0x7f3fe4159550 2023-10-06T02:23:56.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.041+0000 7f3fcbfff700 1 -- 172.21.15.31:0/1849174054 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fd4005cb0 con 0x7f3fe4159550 2023-10-06T02:23:56.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.041+0000 7f3fcbfff700 1 -- 172.21.15.31:0/1849174054 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd4007320 con 0x7f3fe4159550 2023-10-06T02:23:56.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.041+0000 7f3fe979d700 1 -- 172.21.15.31:0/1849174054 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fdc009a20 con 0x7f3fe4159550 2023-10-06T02:23:56.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.041+0000 7f3fe979d700 1 -- 172.21.15.31:0/1849174054 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fe40cbc50 con 0x7f3fe4159550 2023-10-06T02:23:56.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.042+0000 7f3fe979d700 1 -- 172.21.15.31:0/1849174054 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fe404c3c0 con 0x7f3fe4159550 2023-10-06T02:23:56.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.043+0000 7f3fcbfff700 1 -- 172.21.15.31:0/1849174054 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3fd401b3b0 con 0x7f3fe4159550 2023-10-06T02:23:56.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.044+0000 7f3fcbfff700 1 --2- 172.21.15.31:0/1849174054 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f3fd0066f30 0x7f3fd00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:56.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.044+0000 7f3fcbfff700 1 -- 172.21.15.31:0/1849174054 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f3fd4083c60 con 0x7f3fe4159550 2023-10-06T02:23:56.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.044+0000 7f3fe37fe700 1 --2- 172.21.15.31:0/1849174054 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f3fd0066f30 0x7f3fd00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:56.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.045+0000 7f3fe37fe700 1 --2- 172.21.15.31:0/1849174054 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f3fd0066f30 0x7f3fd00693e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3fd80093f0 tx=0x7f3fd8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:56.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.048+0000 7f3fcbfff700 1 -- 172.21.15.31:0/1849174054 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3fd4050c90 con 0x7f3fe4159550 2023-10-06T02:23:56.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.253+0000 7f3fe979d700 1 -- 172.21.15.31:0/1849174054 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f3fe406e7c0 con 0x7f3fe4159550 2023-10-06T02:23:56.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.254+0000 7f3fcbfff700 1 -- 172.21.15.31:0/1849174054 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f3fd4017020 con 0x7f3fe4159550 2023-10-06T02:23:56.255 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 8 2023-10-06T02:23:56.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.256+0000 7f3fe979d700 1 -- 172.21.15.31:0/1849174054 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f3fd0066f30 msgr2=0x7f3fd00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:56.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.256+0000 7f3fe979d700 1 --2- 172.21.15.31:0/1849174054 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f3fd0066f30 0x7f3fd00693e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3fd80093f0 tx=0x7f3fd8008040 comp rx=0 tx=0).stop 2023-10-06T02:23:56.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.256+0000 7f3fe979d700 1 -- 172.21.15.31:0/1849174054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3fe4159550 msgr2=0x7f3fe4158310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:56.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.256+0000 7f3fe979d700 1 --2- 172.21.15.31:0/1849174054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3fe4159550 0x7f3fe4158310 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3fd400c7c0 tx=0x7f3fd4007e00 comp rx=0 tx=0).stop 2023-10-06T02:23:56.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.256+0000 7f3fe979d700 1 -- 172.21.15.31:0/1849174054 shutdown_connections 2023-10-06T02:23:56.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.256+0000 7f3fe979d700 1 --2- 172.21.15.31:0/1849174054 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f3fd0066f30 0x7f3fd00693e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:56.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.256+0000 7f3fe979d700 1 --2- 172.21.15.31:0/1849174054 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3fe415dfd0 0x7f3fe4157ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:56.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.256+0000 7f3fe979d700 1 --2- 172.21.15.31:0/1849174054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3fe4159550 0x7f3fe4158310 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:56.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.256+0000 7f3fe979d700 1 --2- 172.21.15.31:0/1849174054 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3fe4159920 0x7f3fe4157760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:56.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.256+0000 7f3fe979d700 1 -- 172.21.15.31:0/1849174054 >> 172.21.15.31:0/1849174054 conn(0x7f3fe406ce80 msgr2=0x7f3fe40afe80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:56.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.256+0000 7f3fe979d700 1 -- 172.21.15.31:0/1849174054 shutdown_connections 2023-10-06T02:23:56.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.256+0000 7f3fe979d700 1 -- 172.21.15.31:0/1849174054 wait complete. 2023-10-06T02:23:56.274 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T02:23:56.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.463+0000 7fe6f099d700 1 Processor -- start 2023-10-06T02:23:56.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.463+0000 7fe6f099d700 1 -- start start 2023-10-06T02:23:56.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.464+0000 7fe6f099d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe6ec159550 0x7fe6ec0ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:56.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.464+0000 7fe6f099d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe6ec15dfd0 0x7fe6ec0aedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:56.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.464+0000 7fe6f099d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe6ec159920 0x7fe6ec0ad090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:56.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.464+0000 7fe6f099d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6ec0acc00 con 0x7fe6ec159920 2023-10-06T02:23:56.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.464+0000 7fe6f099d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6ec0abff0 con 0x7fe6ec15dfd0 2023-10-06T02:23:56.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.464+0000 7fe6f099d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6ec0ac2d0 con 0x7fe6ec159550 2023-10-06T02:23:56.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.465+0000 7fe6ebfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe6ec159920 0x7fe6ec0ad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:56.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.465+0000 7fe6ebfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe6ec159920 0x7fe6ec0ad090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39142/0 (socket says 172.21.15.31:39142) 2023-10-06T02:23:56.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.465+0000 7fe6ebfff700 1 -- 172.21.15.31:0/4136472317 learned_addr learned my addr 172.21.15.31:0/4136472317 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:56.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.465+0000 7fe6eaffd700 1 --2- 172.21.15.31:0/4136472317 >> v2:172.21.15.62:3300/0 conn(0x7fe6ec15dfd0 0x7fe6ec0aedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:56.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.465+0000 7fe6eb7fe700 1 --2- 172.21.15.31:0/4136472317 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe6ec159550 0x7fe6ec0ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:56.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.465+0000 7fe6ebfff700 1 -- 172.21.15.31:0/4136472317 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe6ec159550 msgr2=0x7fe6ec0ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:56.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.465+0000 7fe6ebfff700 1 --2- 172.21.15.31:0/4136472317 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe6ec159550 0x7fe6ec0ceac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:56.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.465+0000 7fe6ebfff700 1 -- 172.21.15.31:0/4136472317 >> v2:172.21.15.62:3300/0 conn(0x7fe6ec15dfd0 msgr2=0x7fe6ec0aedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:56.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.465+0000 7fe6ebfff700 1 --2- 172.21.15.31:0/4136472317 >> v2:172.21.15.62:3300/0 conn(0x7fe6ec15dfd0 0x7fe6ec0aedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:56.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.465+0000 7fe6ebfff700 1 -- 172.21.15.31:0/4136472317 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6ec0ac5b0 con 0x7fe6ec159920 2023-10-06T02:23:56.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.465+0000 7fe6eaffd700 1 --2- 172.21.15.31:0/4136472317 >> v2:172.21.15.62:3300/0 conn(0x7fe6ec15dfd0 0x7fe6ec0aedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:23:56.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.465+0000 7fe6eb7fe700 1 --2- 172.21.15.31:0/4136472317 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe6ec159550 0x7fe6ec0ceac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:23:56.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.466+0000 7fe6ebfff700 1 --2- 172.21.15.31:0/4136472317 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe6ec159920 0x7fe6ec0ad090 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec06b120 tx=0x7fe6e4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=159e4ec3421126d7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:56.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.466+0000 7fe6ea7fc700 1 -- 172.21.15.31:0/4136472317 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e400e070 con 0x7fe6ec159920 2023-10-06T02:23:56.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.466+0000 7fe6ea7fc700 1 -- 172.21.15.31:0/4136472317 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6e400ba50 con 0x7fe6ec159920 2023-10-06T02:23:56.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.466+0000 7fe6ea7fc700 1 -- 172.21.15.31:0/4136472317 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e40092e0 con 0x7fe6ec159920 2023-10-06T02:23:56.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.466+0000 7fe6f099d700 1 -- 172.21.15.31:0/4136472317 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe6ec159920 msgr2=0x7fe6ec0ad090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:56.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.466+0000 7fe6f099d700 1 --2- 172.21.15.31:0/4136472317 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe6ec159920 0x7fe6ec0ad090 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec06b120 tx=0x7fe6e4009840 comp rx=0 tx=0).stop 2023-10-06T02:23:56.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.466+0000 7fe6f099d700 1 -- 172.21.15.31:0/4136472317 shutdown_connections 2023-10-06T02:23:56.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.466+0000 7fe6f099d700 1 --2- 172.21.15.31:0/4136472317 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe6ec159920 0x7fe6ec0ad090 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:56.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.466+0000 7fe6f099d700 1 --2- 172.21.15.31:0/4136472317 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe6ec159550 0x7fe6ec0ceac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:56.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.466+0000 7fe6f099d700 1 --2- 172.21.15.31:0/4136472317 >> v2:172.21.15.62:3300/0 conn(0x7fe6ec15dfd0 0x7fe6ec0aedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:56.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.466+0000 7fe6f099d700 1 -- 172.21.15.31:0/4136472317 >> 172.21.15.31:0/4136472317 conn(0x7fe6ec06ce80 msgr2=0x7fe6ec0ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:56.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.466+0000 7fe6f099d700 1 -- 172.21.15.31:0/4136472317 shutdown_connections 2023-10-06T02:23:56.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.467+0000 7fe6f099d700 1 -- 172.21.15.31:0/4136472317 wait complete. 2023-10-06T02:23:56.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.468+0000 7fe6f099d700 1 Processor -- start 2023-10-06T02:23:56.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.468+0000 7fe6f099d700 1 -- start start 2023-10-06T02:23:56.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.469+0000 7fe6f099d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6ec159550 0x7fe6ec16b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:56.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.469+0000 7fe6f099d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe6ec159920 0x7fe6ec16cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:56.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.469+0000 7fe6f099d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe6ec15dfd0 0x7fe6ec157920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:56.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.469+0000 7fe6f099d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6ec0c7560 con 0x7fe6ec15dfd0 2023-10-06T02:23:56.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.469+0000 7fe6f099d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6ec16ac90 con 0x7fe6ec159550 2023-10-06T02:23:56.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.469+0000 7fe6f099d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6ec0c77b0 con 0x7fe6ec159920 2023-10-06T02:23:56.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.469+0000 7fe6ebfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe6ec15dfd0 0x7fe6ec157920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:56.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.469+0000 7fe6ebfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe6ec15dfd0 0x7fe6ec157920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39152/0 (socket says 172.21.15.31:39152) 2023-10-06T02:23:56.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.469+0000 7fe6ebfff700 1 -- 172.21.15.31:0/1606179180 learned_addr learned my addr 172.21.15.31:0/1606179180 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:56.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.469+0000 7fe6eaffd700 1 --2- 172.21.15.31:0/1606179180 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe6ec159920 0x7fe6ec16cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:56.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.469+0000 7fe6eb7fe700 1 --2- 172.21.15.31:0/1606179180 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6ec159550 0x7fe6ec16b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:56.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.470+0000 7fe6ebfff700 1 -- 172.21.15.31:0/1606179180 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe6ec159920 msgr2=0x7fe6ec16cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:56.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.470+0000 7fe6ebfff700 1 --2- 172.21.15.31:0/1606179180 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe6ec159920 0x7fe6ec16cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:56.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.470+0000 7fe6ebfff700 1 -- 172.21.15.31:0/1606179180 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6ec159550 msgr2=0x7fe6ec16b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:56.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.470+0000 7fe6ebfff700 1 --2- 172.21.15.31:0/1606179180 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6ec159550 0x7fe6ec16b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:56.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.470+0000 7fe6ebfff700 1 -- 172.21.15.31:0/1606179180 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6e4009ae0 con 0x7fe6ec15dfd0 2023-10-06T02:23:56.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.470+0000 7fe6ebfff700 1 --2- 172.21.15.31:0/1606179180 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe6ec15dfd0 0x7fe6ec157920 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec06ad70 tx=0x7fe6e4018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:56.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.470+0000 7fe6d37fe700 1 -- 172.21.15.31:0/1606179180 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e40032f0 con 0x7fe6ec15dfd0 2023-10-06T02:23:56.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.470+0000 7fe6d37fe700 1 -- 172.21.15.31:0/1606179180 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6e4008ce0 con 0x7fe6ec15dfd0 2023-10-06T02:23:56.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.470+0000 7fe6d37fe700 1 -- 172.21.15.31:0/1606179180 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e40266b0 con 0x7fe6ec15dfd0 2023-10-06T02:23:56.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.470+0000 7fe6f099d700 1 -- 172.21.15.31:0/1606179180 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6ec0c7a90 con 0x7fe6ec15dfd0 2023-10-06T02:23:56.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.471+0000 7fe6f099d700 1 -- 172.21.15.31:0/1606179180 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6ec16d170 con 0x7fe6ec15dfd0 2023-10-06T02:23:56.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.472+0000 7fe6f099d700 1 -- 172.21.15.31:0/1606179180 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6ec06d480 con 0x7fe6ec15dfd0 2023-10-06T02:23:56.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.473+0000 7fe6d37fe700 1 -- 172.21.15.31:0/1606179180 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe6e4003040 con 0x7fe6ec15dfd0 2023-10-06T02:23:56.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.473+0000 7fe6d37fe700 1 --2- 172.21.15.31:0/1606179180 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe6d8066f30 0x7fe6d80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:56.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.474+0000 7fe6d37fe700 1 -- 172.21.15.31:0/1606179180 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6116+0+0 (secure 0 0 0) 0x7fe6e4085fa0 con 0x7fe6ec15dfd0 2023-10-06T02:23:56.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.474+0000 7fe6eb7fe700 1 --2- 172.21.15.31:0/1606179180 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe6d8066f30 0x7fe6d80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:56.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.474+0000 7fe6eb7fe700 1 --2- 172.21.15.31:0/1606179180 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe6d8066f30 0x7fe6d80693e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc006f00 tx=0x7fe6dc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:56.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.478+0000 7fe6d37fe700 1 -- 172.21.15.31:0/1606179180 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe6e4052fd0 con 0x7fe6ec15dfd0 2023-10-06T02:23:56.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.680+0000 7fe6f099d700 1 -- 172.21.15.31:0/1606179180 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe6ec0c7f90 con 0x7fe6ec15dfd0 2023-10-06T02:23:56.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.681+0000 7fe6d37fe700 1 -- 172.21.15.31:0/1606179180 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe6e40565f0 con 0x7fe6ec15dfd0 2023-10-06T02:23:56.682 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-10-06T02:23:56.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.683+0000 7fe6f099d700 1 -- 172.21.15.31:0/1606179180 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe6d8066f30 msgr2=0x7fe6d80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:56.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.683+0000 7fe6f099d700 1 --2- 172.21.15.31:0/1606179180 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe6d8066f30 0x7fe6d80693e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc006f00 tx=0x7fe6dc008040 comp rx=0 tx=0).stop 2023-10-06T02:23:56.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.683+0000 7fe6f099d700 1 -- 172.21.15.31:0/1606179180 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe6ec15dfd0 msgr2=0x7fe6ec157920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:56.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.683+0000 7fe6f099d700 1 --2- 172.21.15.31:0/1606179180 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe6ec15dfd0 0x7fe6ec157920 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec06ad70 tx=0x7fe6e4018000 comp rx=0 tx=0).stop 2023-10-06T02:23:56.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.683+0000 7fe6f099d700 1 -- 172.21.15.31:0/1606179180 shutdown_connections 2023-10-06T02:23:56.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.683+0000 7fe6f099d700 1 --2- 172.21.15.31:0/1606179180 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe6ec15dfd0 0x7fe6ec157920 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:56.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.683+0000 7fe6f099d700 1 --2- 172.21.15.31:0/1606179180 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6ec159550 0x7fe6ec16b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:56.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.683+0000 7fe6f099d700 1 --2- 172.21.15.31:0/1606179180 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe6d8066f30 0x7fe6d80693e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:56.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.683+0000 7fe6f099d700 1 --2- 172.21.15.31:0/1606179180 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe6ec159920 0x7fe6ec16cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:56.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.683+0000 7fe6f099d700 1 -- 172.21.15.31:0/1606179180 >> 172.21.15.31:0/1606179180 conn(0x7fe6ec06ce80 msgr2=0x7fe6ec152b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:56.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.683+0000 7fe6f099d700 1 -- 172.21.15.31:0/1606179180 shutdown_connections 2023-10-06T02:23:56.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.683+0000 7fe6f099d700 1 -- 172.21.15.31:0/1606179180 wait complete. 2023-10-06T02:23:56.702 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T02:23:56.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.890+0000 7f0a8bfff700 1 Processor -- start 2023-10-06T02:23:56.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.890+0000 7f0a8bfff700 1 -- start start 2023-10-06T02:23:56.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.892+0000 7f0a8bfff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0a8c0c7fb0 0x7f0a8c06abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:56.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.892+0000 7f0a8bfff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0a8c13f780 0x7f0a8c13fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:56.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.892+0000 7f0a8bfff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a8c140120 con 0x7f0a8c0c7be0 2023-10-06T02:23:56.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.892+0000 7f0a8bfff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a8c06bf10 con 0x7f0a8c0c7fb0 2023-10-06T02:23:56.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.892+0000 7f0a8bfff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a8c06b290 con 0x7f0a8c13f780 2023-10-06T02:23:56.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.892+0000 7f0a8affd700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f0a8c0c7be0 0x7f0a8c15a990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49430/0 (socket says 172.21.15.31:49430) 2023-10-06T02:23:56.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.892+0000 7f0a8affd700 1 -- 172.21.15.31:0/3269356260 learned_addr learned my addr 172.21.15.31:0/3269356260 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:23:56.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.892+0000 7f0a8b7fe700 1 --2- 172.21.15.31:0/3269356260 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0a8c13f780 0x7f0a8c13fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:56.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.892+0000 7f0a8a7fc700 1 --2- 172.21.15.31:0/3269356260 >> v2:172.21.15.62:3300/0 conn(0x7f0a8c0c7fb0 0x7f0a8c06abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:56.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.892+0000 7f0a89ffb700 1 -- 172.21.15.31:0/3269356260 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 982588122 0 0) 0x7f0a7c001110 con 0x7f0a8c0c7be0 2023-10-06T02:23:56.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.893+0000 7f0a89ffb700 1 -- 172.21.15.31:0/3269356260 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a74003570 con 0x7f0a8c0c7be0 2023-10-06T02:23:56.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.893+0000 7f0a8b7fe700 1 -- 172.21.15.31:0/3269356260 >> v2:172.21.15.62:3300/0 conn(0x7f0a8c0c7fb0 msgr2=0x7f0a8c06abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:56.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.893+0000 7f0a8b7fe700 1 --2- 172.21.15.31:0/3269356260 >> v2:172.21.15.62:3300/0 conn(0x7f0a8c0c7fb0 0x7f0a8c06abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:56.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.893+0000 7f0a8b7fe700 1 -- 172.21.15.31:0/3269356260 >> v1:172.21.15.31:6789/0 conn(0x7f0a8c0c7be0 legacy=0x7f0a8c15a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:56.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.893+0000 7f0a8b7fe700 1 -- 172.21.15.31:0/3269356260 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a8c06b570 con 0x7f0a8c13f780 2023-10-06T02:23:56.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.893+0000 7f0a8b7fe700 1 --2- 172.21.15.31:0/3269356260 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0a8c13f780 0x7f0a8c13fbe0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0a84009e60 tx=0x7f0a84009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eaaaccf33c435a16 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:56.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.893+0000 7f0a89ffb700 1 -- 172.21.15.31:0/3269356260 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a8400e070 con 0x7f0a8c13f780 2023-10-06T02:23:56.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.893+0000 7f0a89ffb700 1 -- 172.21.15.31:0/3269356260 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a8400bab0 con 0x7f0a8c13f780 2023-10-06T02:23:56.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.893+0000 7f0a89ffb700 1 -- 172.21.15.31:0/3269356260 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a8400be30 con 0x7f0a8c13f780 2023-10-06T02:23:56.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.894+0000 7f0a8bfff700 1 -- 172.21.15.31:0/3269356260 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0a8c13f780 msgr2=0x7f0a8c13fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:56.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.894+0000 7f0a8bfff700 1 --2- 172.21.15.31:0/3269356260 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0a8c13f780 0x7f0a8c13fbe0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0a84009e60 tx=0x7f0a84009780 comp rx=0 tx=0).stop 2023-10-06T02:23:56.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.894+0000 7f0a8bfff700 1 -- 172.21.15.31:0/3269356260 shutdown_connections 2023-10-06T02:23:56.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.894+0000 7f0a8bfff700 1 --2- 172.21.15.31:0/3269356260 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0a8c13f780 0x7f0a8c13fbe0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:56.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.894+0000 7f0a8bfff700 1 --2- 172.21.15.31:0/3269356260 >> v2:172.21.15.62:3300/0 conn(0x7f0a8c0c7fb0 0x7f0a8c06abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:56.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.894+0000 7f0a8bfff700 1 -- 172.21.15.31:0/3269356260 >> 172.21.15.31:0/3269356260 conn(0x7f0a8c06d110 msgr2=0x7f0a8c0cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:56.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.894+0000 7f0a8bfff700 1 -- 172.21.15.31:0/3269356260 shutdown_connections 2023-10-06T02:23:56.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.894+0000 7f0a8bfff700 1 -- 172.21.15.31:0/3269356260 wait complete. 2023-10-06T02:23:56.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.895+0000 7f0a8bfff700 1 Processor -- start 2023-10-06T02:23:56.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.895+0000 7f0a8bfff700 1 -- start start 2023-10-06T02:23:56.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.896+0000 7f0a8bfff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0a8c0c7be0 0x7f0a8c155bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:56.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.896+0000 7f0a8bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a8c0c7fb0 0x7f0a8c0c5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:56.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.896+0000 7f0a8bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a8c13f780 0x7f0a8c1568c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:56.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.896+0000 7f0a8bfff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a8c0c5470 con 0x7f0a8c13f780 2023-10-06T02:23:56.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.896+0000 7f0a8bfff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a8c159bb0 con 0x7f0a8c0c7fb0 2023-10-06T02:23:56.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.896+0000 7f0a8bfff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a8c0c4860 con 0x7f0a8c0c7be0 2023-10-06T02:23:56.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.897+0000 7f0a8b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a8c13f780 0x7f0a8c1568c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:56.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.897+0000 7f0a8b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a8c13f780 0x7f0a8c1568c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39158/0 (socket says 172.21.15.31:39158) 2023-10-06T02:23:56.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.897+0000 7f0a8b7fe700 1 -- 172.21.15.31:0/3490332442 learned_addr learned my addr 172.21.15.31:0/3490332442 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:56.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.897+0000 7f0a8b7fe700 1 -- 172.21.15.31:0/3490332442 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0a8c0c7be0 msgr2=0x7f0a8c155bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:56.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.897+0000 7f0a8b7fe700 1 --2- 172.21.15.31:0/3490332442 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0a8c0c7be0 0x7f0a8c155bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:56.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.897+0000 7f0a8b7fe700 1 -- 172.21.15.31:0/3490332442 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a8c0c7fb0 msgr2=0x7f0a8c0c5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:56.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.897+0000 7f0a8b7fe700 1 --2- 172.21.15.31:0/3490332442 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a8c0c7fb0 0x7f0a8c0c5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:56.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.897+0000 7f0a8b7fe700 1 -- 172.21.15.31:0/3490332442 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a84009a20 con 0x7f0a8c13f780 2023-10-06T02:23:56.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.897+0000 7f0a8b7fe700 1 --2- 172.21.15.31:0/3490332442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a8c13f780 0x7f0a8c1568c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c06b3d0 tx=0x7f0a84019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:56.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.898+0000 7f0a72ffd700 1 -- 172.21.15.31:0/3490332442 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a84027470 con 0x7f0a8c13f780 2023-10-06T02:23:56.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.898+0000 7f0a72ffd700 1 -- 172.21.15.31:0/3490332442 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a840275f0 con 0x7f0a8c13f780 2023-10-06T02:23:56.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.898+0000 7f0a8bfff700 1 -- 172.21.15.31:0/3490332442 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a8c0c4b40 con 0x7f0a8c13f780 2023-10-06T02:23:56.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.898+0000 7f0a72ffd700 1 -- 172.21.15.31:0/3490332442 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a840279b0 con 0x7f0a8c13f780 2023-10-06T02:23:56.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.898+0000 7f0a8bfff700 1 -- 172.21.15.31:0/3490332442 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a8c0c5100 con 0x7f0a8c13f780 2023-10-06T02:23:56.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.899+0000 7f0a8bfff700 1 -- 172.21.15.31:0/3490332442 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a8c04c3c0 con 0x7f0a8c13f780 2023-10-06T02:23:56.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.900+0000 7f0a72ffd700 1 -- 172.21.15.31:0/3490332442 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0a84024070 con 0x7f0a8c13f780 2023-10-06T02:23:56.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.901+0000 7f0a72ffd700 1 --2- 172.21.15.31:0/3490332442 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f0a78066f60 0x7f0a78069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:56.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.901+0000 7f0a72ffd700 1 -- 172.21.15.31:0/3490332442 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f0a840871d0 con 0x7f0a8c13f780 2023-10-06T02:23:56.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.901+0000 7f0a8affd700 1 --2- 172.21.15.31:0/3490332442 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f0a78066f60 0x7f0a78069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:56.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.901+0000 7f0a8affd700 1 --2- 172.21.15.31:0/3490332442 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f0a78066f60 0x7f0a78069410 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c0058d0 tx=0x7f0a7c001160 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:56.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:56.905+0000 7f0a72ffd700 1 -- 172.21.15.31:0/3490332442 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0a84054200 con 0x7f0a8c13f780 2023-10-06T02:23:57.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.110+0000 7f0a8bfff700 1 -- 172.21.15.31:0/3490332442 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0a8c0c55b0 con 0x7f0a8c13f780 2023-10-06T02:23:57.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.111+0000 7f0a72ffd700 1 -- 172.21.15.31:0/3490332442 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0a84057820 con 0x7f0a8c13f780 2023-10-06T02:23:57.112 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-10-06T02:23:57.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.113+0000 7f0a8bfff700 1 -- 172.21.15.31:0/3490332442 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f0a78066f60 msgr2=0x7f0a78069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:57.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.113+0000 7f0a8bfff700 1 --2- 172.21.15.31:0/3490332442 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f0a78066f60 0x7f0a78069410 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c0058d0 tx=0x7f0a7c001160 comp rx=0 tx=0).stop 2023-10-06T02:23:57.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.113+0000 7f0a8bfff700 1 -- 172.21.15.31:0/3490332442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a8c13f780 msgr2=0x7f0a8c1568c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:57.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.113+0000 7f0a8bfff700 1 --2- 172.21.15.31:0/3490332442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a8c13f780 0x7f0a8c1568c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c06b3d0 tx=0x7f0a84019000 comp rx=0 tx=0).stop 2023-10-06T02:23:57.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.113+0000 7f0a8bfff700 1 -- 172.21.15.31:0/3490332442 shutdown_connections 2023-10-06T02:23:57.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.113+0000 7f0a8bfff700 1 --2- 172.21.15.31:0/3490332442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a8c13f780 0x7f0a8c1568c0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:57.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.113+0000 7f0a8bfff700 1 --2- 172.21.15.31:0/3490332442 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f0a78066f60 0x7f0a78069410 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:57.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.113+0000 7f0a8bfff700 1 --2- 172.21.15.31:0/3490332442 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0a8c0c7be0 0x7f0a8c155bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:57.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.113+0000 7f0a8bfff700 1 --2- 172.21.15.31:0/3490332442 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a8c0c7fb0 0x7f0a8c0c5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:57.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.113+0000 7f0a8bfff700 1 -- 172.21.15.31:0/3490332442 >> 172.21.15.31:0/3490332442 conn(0x7f0a8c06d110 msgr2=0x7f0a8c0c8b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:57.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.113+0000 7f0a8bfff700 1 -- 172.21.15.31:0/3490332442 shutdown_connections 2023-10-06T02:23:57.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.113+0000 7f0a8bfff700 1 -- 172.21.15.31:0/3490332442 wait complete. 2023-10-06T02:23:57.132 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:23:57.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.318+0000 7fa6bc9fe700 1 Processor -- start 2023-10-06T02:23:57.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.318+0000 7fa6bc9fe700 1 -- start start 2023-10-06T02:23:57.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.319+0000 7fa6bc9fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa6b8153c30 0x7fa6b815a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:57.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.319+0000 7fa6bc9fe700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6b815afd0 con 0x7fa6b80cf110 2023-10-06T02:23:57.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.319+0000 7fa6bc9fe700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6b815b2b0 con 0x7fa6b815de50 2023-10-06T02:23:57.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.319+0000 7fa6bc9fe700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b815b590 con 0x7fa6b8153c30 2023-10-06T02:23:57.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.320+0000 7fa6b759e700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fa6b80cf110 0x7fa6b815c070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49432/0 (socket says 172.21.15.31:49432) 2023-10-06T02:23:57.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.320+0000 7fa6b6d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa6b8153c30 0x7fa6b815a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:57.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.320+0000 7fa6b759e700 1 -- 172.21.15.31:0/2794149390 learned_addr learned my addr 172.21.15.31:0/2794149390 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:23:57.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.320+0000 7fa6b659c700 1 --1- 172.21.15.31:0/2794149390 >> v1:172.21.15.62:6789/0 conn(0x7fa6b815de50 0x7fa6b80cea00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:59888/0 (socket says 172.21.15.31:59888) 2023-10-06T02:23:57.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.320+0000 7fa6b6d9d700 1 -- 172.21.15.31:0/2794149390 >> v1:172.21.15.62:6789/0 conn(0x7fa6b815de50 legacy=0x7fa6b80cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:57.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.320+0000 7fa6b6d9d700 1 -- 172.21.15.31:0/2794149390 >> v1:172.21.15.31:6789/0 conn(0x7fa6b80cf110 legacy=0x7fa6b815c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:57.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.320+0000 7fa6b5d9b700 1 -- 172.21.15.31:0/2794149390 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3268019034 0 0) 0x7fa6b815b2b0 con 0x7fa6b815de50 2023-10-06T02:23:57.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.320+0000 7fa6b6d9d700 1 -- 172.21.15.31:0/2794149390 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6b815b870 con 0x7fa6b8153c30 2023-10-06T02:23:57.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.321+0000 7fa6b6d9d700 1 --2- 172.21.15.31:0/2794149390 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa6b8153c30 0x7fa6b815a5e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa6a8009910 tx=0x7fa6a80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b7f0d1f5673443d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:57.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.321+0000 7fa6b5d9b700 1 -- 172.21.15.31:0/2794149390 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a800b760 con 0x7fa6b8153c30 2023-10-06T02:23:57.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.321+0000 7fa6b5d9b700 1 -- 172.21.15.31:0/2794149390 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6a800b8c0 con 0x7fa6b8153c30 2023-10-06T02:23:57.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.321+0000 7fa6b5d9b700 1 -- 172.21.15.31:0/2794149390 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a800bc60 con 0x7fa6b8153c30 2023-10-06T02:23:57.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.321+0000 7fa6bc9fe700 1 -- 172.21.15.31:0/2794149390 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa6b8153c30 msgr2=0x7fa6b815a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:57.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.321+0000 7fa6bc9fe700 1 --2- 172.21.15.31:0/2794149390 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa6b8153c30 0x7fa6b815a5e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa6a8009910 tx=0x7fa6a80091a0 comp rx=0 tx=0).stop 2023-10-06T02:23:57.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.321+0000 7fa6bc9fe700 1 -- 172.21.15.31:0/2794149390 shutdown_connections 2023-10-06T02:23:57.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.321+0000 7fa6bc9fe700 1 --2- 172.21.15.31:0/2794149390 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa6b8153c30 0x7fa6b815a5e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:57.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.321+0000 7fa6bc9fe700 1 -- 172.21.15.31:0/2794149390 >> 172.21.15.31:0/2794149390 conn(0x7fa6b80703a0 msgr2=0x7fa6b80707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:57.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.322+0000 7fa6bc9fe700 1 -- 172.21.15.31:0/2794149390 shutdown_connections 2023-10-06T02:23:57.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.322+0000 7fa6bc9fe700 1 -- 172.21.15.31:0/2794149390 wait complete. 2023-10-06T02:23:57.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.323+0000 7fa6bc9fe700 1 Processor -- start 2023-10-06T02:23:57.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.323+0000 7fa6bc9fe700 1 -- start start 2023-10-06T02:23:57.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.324+0000 7fa6bc9fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6b80cf110 0x7fa6b816c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:57.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.324+0000 7fa6bc9fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6b8153c30 0x7fa6b80b32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:57.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.324+0000 7fa6bc9fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa6b815de50 0x7fa6b80b3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:57.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.324+0000 7fa6bc9fe700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b80ca250 con 0x7fa6b80cf110 2023-10-06T02:23:57.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.324+0000 7fa6bc9fe700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b80b4a80 con 0x7fa6b8153c30 2023-10-06T02:23:57.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.324+0000 7fa6bc9fe700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b816ce60 con 0x7fa6b815de50 2023-10-06T02:23:57.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.324+0000 7fa6b6d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6b80cf110 0x7fa6b816c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:57.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.324+0000 7fa6b6d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6b80cf110 0x7fa6b816c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39168/0 (socket says 172.21.15.31:39168) 2023-10-06T02:23:57.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.324+0000 7fa6b6d9d700 1 -- 172.21.15.31:0/1772081149 learned_addr learned my addr 172.21.15.31:0/1772081149 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:57.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.325+0000 7fa6b659c700 1 --2- 172.21.15.31:0/1772081149 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6b8153c30 0x7fa6b80b32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:57.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.325+0000 7fa6b759e700 1 --2- 172.21.15.31:0/1772081149 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa6b815de50 0x7fa6b80b3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:57.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.325+0000 7fa6b6d9d700 1 -- 172.21.15.31:0/1772081149 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa6b815de50 msgr2=0x7fa6b80b3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:57.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.325+0000 7fa6b6d9d700 1 --2- 172.21.15.31:0/1772081149 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa6b815de50 0x7fa6b80b3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:57.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.325+0000 7fa6b6d9d700 1 -- 172.21.15.31:0/1772081149 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6b8153c30 msgr2=0x7fa6b80b32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:57.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.325+0000 7fa6b6d9d700 1 --2- 172.21.15.31:0/1772081149 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6b8153c30 0x7fa6b80b32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:57.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.325+0000 7fa6b6d9d700 1 -- 172.21.15.31:0/1772081149 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6a8009420 con 0x7fa6b80cf110 2023-10-06T02:23:57.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.325+0000 7fa6b6d9d700 1 --2- 172.21.15.31:0/1772081149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6b80cf110 0x7fa6b816c920 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8159ca0 tx=0x7fa6a8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:57.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.325+0000 7fa69effd700 1 -- 172.21.15.31:0/1772081149 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a8011dd0 con 0x7fa6b80cf110 2023-10-06T02:23:57.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.325+0000 7fa69effd700 1 -- 172.21.15.31:0/1772081149 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6a801c460 con 0x7fa6b80cf110 2023-10-06T02:23:57.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.325+0000 7fa6bc9fe700 1 -- 172.21.15.31:0/1772081149 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6b80b3e70 con 0x7fa6b80cf110 2023-10-06T02:23:57.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.325+0000 7fa69effd700 1 -- 172.21.15.31:0/1772081149 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a801c820 con 0x7fa6b80cf110 2023-10-06T02:23:57.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.326+0000 7fa6bc9fe700 1 -- 172.21.15.31:0/1772081149 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6b80b4430 con 0x7fa6b80cf110 2023-10-06T02:23:57.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.326+0000 7fa6bc9fe700 1 -- 172.21.15.31:0/1772081149 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6b804c3c0 con 0x7fa6b80cf110 2023-10-06T02:23:57.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.327+0000 7fa69effd700 1 -- 172.21.15.31:0/1772081149 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa6a801cc10 con 0x7fa6b80cf110 2023-10-06T02:23:57.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.328+0000 7fa69effd700 1 --2- 172.21.15.31:0/1772081149 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa6a4066f30 0x7fa6a40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:57.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.328+0000 7fa69effd700 1 -- 172.21.15.31:0/1772081149 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6116+0+0 (secure 0 0 0) 0x7fa6a8086c00 con 0x7fa6b80cf110 2023-10-06T02:23:57.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.328+0000 7fa6b659c700 1 --2- 172.21.15.31:0/1772081149 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa6a4066f30 0x7fa6a40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:57.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.329+0000 7fa6b659c700 1 --2- 172.21.15.31:0/1772081149 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa6a4066f30 0x7fa6a40693e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac007fd0 tx=0x7fa6ac007db0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:57.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.332+0000 7fa69effd700 1 -- 172.21.15.31:0/1772081149 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa6a8053c30 con 0x7fa6b80cf110 2023-10-06T02:23:57.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.579+0000 7fa6bc9fe700 1 -- 172.21.15.31:0/1772081149 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa6b804c3c0 con 0x7fa6b80cf110 2023-10-06T02:23:57.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.579+0000 7fa69effd700 1 -- 172.21.15.31:0/1772081149 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4591 (secure 0 0 0) 0x7fa6a8057250 con 0x7fa6b80cf110 2023-10-06T02:23:57.685 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:23:57.685 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/1808327183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":1808327183},{"type":"v1","addr":"172.21.15.31:6837","nonce":1808327183}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/4263144517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":4263144517},{"type":"v1","addr":"172.21.15.62:6834","nonce":4263144517}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1489208568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":1489208568},{"type":"v1","addr":"172.21.15.62:6835","nonce":1489208568}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:23:09.521236+0000","modified":"2023-10-06T02:23:13.047274+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6836/3809284664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3809284664},{"type":"v1","addr":"172.21.15.31:6836","nonce":3809284664}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T02:23:57.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.582+0000 7fa6bc9fe700 1 -- 172.21.15.31:0/1772081149 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa6a4066f30 msgr2=0x7fa6a40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:57.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.582+0000 7fa6bc9fe700 1 --2- 172.21.15.31:0/1772081149 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa6a4066f30 0x7fa6a40693e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac007fd0 tx=0x7fa6ac007db0 comp rx=0 tx=0).stop 2023-10-06T02:23:57.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.582+0000 7fa6bc9fe700 1 -- 172.21.15.31:0/1772081149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6b80cf110 msgr2=0x7fa6b816c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:57.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.582+0000 7fa6bc9fe700 1 --2- 172.21.15.31:0/1772081149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6b80cf110 0x7fa6b816c920 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8159ca0 tx=0x7fa6a8016000 comp rx=0 tx=0).stop 2023-10-06T02:23:57.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.582+0000 7fa6bc9fe700 1 -- 172.21.15.31:0/1772081149 shutdown_connections 2023-10-06T02:23:57.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.582+0000 7fa6bc9fe700 1 --2- 172.21.15.31:0/1772081149 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa6a4066f30 0x7fa6a40693e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:57.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.582+0000 7fa6bc9fe700 1 --2- 172.21.15.31:0/1772081149 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa6b815de50 0x7fa6b80b3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:57.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.582+0000 7fa6bc9fe700 1 --2- 172.21.15.31:0/1772081149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6b80cf110 0x7fa6b816c920 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:57.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.582+0000 7fa6bc9fe700 1 --2- 172.21.15.31:0/1772081149 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6b8153c30 0x7fa6b80b32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:57.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.582+0000 7fa6bc9fe700 1 -- 172.21.15.31:0/1772081149 >> 172.21.15.31:0/1772081149 conn(0x7fa6b80703a0 msgr2=0x7fa6b8153030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:57.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.582+0000 7fa6bc9fe700 1 -- 172.21.15.31:0/1772081149 shutdown_connections 2023-10-06T02:23:57.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.582+0000 7fa6bc9fe700 1 -- 172.21.15.31:0/1772081149 wait complete. 2023-10-06T02:23:57.717 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 8 2023-10-06T02:23:57.717 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:23:57.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.902+0000 7f7cfe99d700 1 Processor -- start 2023-10-06T02:23:57.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.903+0000 7f7cfe99d700 1 -- start start 2023-10-06T02:23:57.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.904+0000 7f7cfe99d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7cf8159550 0x7f7cf80ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:57.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.904+0000 7f7cfe99d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7cf815dfd0 0x7f7cf80aedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:57.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.904+0000 7f7cfe99d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7cf8159920 0x7f7cf80ad090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:57.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.904+0000 7f7cfe99d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cf80acc00 con 0x7f7cf8159920 2023-10-06T02:23:57.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.904+0000 7f7cfe99d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cf80abff0 con 0x7f7cf8159550 2023-10-06T02:23:57.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.904+0000 7f7cfe99d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf80ac2d0 con 0x7f7cf815dfd0 2023-10-06T02:23:57.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.904+0000 7f7cfe19c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7cf8159920 0x7f7cf80ad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:57.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.904+0000 7f7cfe19c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7cf8159920 0x7f7cf80ad090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49658/0 (socket says 172.21.15.31:49658) 2023-10-06T02:23:57.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.904+0000 7f7cfe19c700 1 -- 172.21.15.31:0/2877775869 learned_addr learned my addr 172.21.15.31:0/2877775869 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:57.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.905+0000 7f7cfd99b700 1 --2- 172.21.15.31:0/2877775869 >> v2:172.21.15.62:3300/0 conn(0x7f7cf8159550 0x7f7cf80ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:57.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.905+0000 7f7cfd19a700 1 --2- 172.21.15.31:0/2877775869 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7cf815dfd0 0x7f7cf80aedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:57.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.905+0000 7f7cfe19c700 1 -- 172.21.15.31:0/2877775869 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7cf815dfd0 msgr2=0x7f7cf80aedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:57.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.905+0000 7f7cfe19c700 1 --2- 172.21.15.31:0/2877775869 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7cf815dfd0 0x7f7cf80aedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:57.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.905+0000 7f7cfe19c700 1 -- 172.21.15.31:0/2877775869 >> v2:172.21.15.62:3300/0 conn(0x7f7cf8159550 msgr2=0x7f7cf80ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:57.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.905+0000 7f7cfe19c700 1 --2- 172.21.15.31:0/2877775869 >> v2:172.21.15.62:3300/0 conn(0x7f7cf8159550 0x7f7cf80ceac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:57.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.905+0000 7f7cfe19c700 1 -- 172.21.15.31:0/2877775869 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cf80ac5b0 con 0x7f7cf8159920 2023-10-06T02:23:57.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.905+0000 7f7cfd99b700 1 --2- 172.21.15.31:0/2877775869 >> v2:172.21.15.62:3300/0 conn(0x7f7cf8159550 0x7f7cf80ceac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:23:57.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.905+0000 7f7cfe19c700 1 --2- 172.21.15.31:0/2877775869 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7cf8159920 0x7f7cf80ad090 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8064270 tx=0x7f7cf0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36404ed816f00d62 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:57.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.905+0000 7f7cfc999700 1 -- 172.21.15.31:0/2877775869 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf000e070 con 0x7f7cf8159920 2023-10-06T02:23:57.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.905+0000 7f7cfc999700 1 -- 172.21.15.31:0/2877775869 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cf000ba50 con 0x7f7cf8159920 2023-10-06T02:23:57.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.905+0000 7f7cfc999700 1 -- 172.21.15.31:0/2877775869 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf00092e0 con 0x7f7cf8159920 2023-10-06T02:23:57.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.906+0000 7f7cfe99d700 1 -- 172.21.15.31:0/2877775869 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7cf8159920 msgr2=0x7f7cf80ad090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:57.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.906+0000 7f7cfe99d700 1 --2- 172.21.15.31:0/2877775869 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7cf8159920 0x7f7cf80ad090 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8064270 tx=0x7f7cf0009840 comp rx=0 tx=0).stop 2023-10-06T02:23:57.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.906+0000 7f7cfe99d700 1 -- 172.21.15.31:0/2877775869 shutdown_connections 2023-10-06T02:23:57.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.906+0000 7f7cfe99d700 1 --2- 172.21.15.31:0/2877775869 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7cf8159920 0x7f7cf80ad090 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:57.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.906+0000 7f7cfe99d700 1 --2- 172.21.15.31:0/2877775869 >> v2:172.21.15.62:3300/0 conn(0x7f7cf8159550 0x7f7cf80ceac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:57.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.906+0000 7f7cfe99d700 1 --2- 172.21.15.31:0/2877775869 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7cf815dfd0 0x7f7cf80aedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:57.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.906+0000 7f7cfe99d700 1 -- 172.21.15.31:0/2877775869 >> 172.21.15.31:0/2877775869 conn(0x7f7cf806ce80 msgr2=0x7f7cf80ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:57.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.906+0000 7f7cfe99d700 1 -- 172.21.15.31:0/2877775869 shutdown_connections 2023-10-06T02:23:57.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.906+0000 7f7cfe99d700 1 -- 172.21.15.31:0/2877775869 wait complete. 2023-10-06T02:23:57.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.908+0000 7f7cfe99d700 1 Processor -- start 2023-10-06T02:23:57.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.908+0000 7f7cfe99d700 1 -- start start 2023-10-06T02:23:57.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.909+0000 7f7cfe99d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7cf8159550 0x7f7cf8146d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:57.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.909+0000 7f7cfe99d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7cf8159920 0x7f7cf80c7560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:57.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.909+0000 7f7cfe99d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7cf815dfd0 0x7f7cf80c9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:57.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.909+0000 7f7cfe99d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf80c7f40 con 0x7f7cf8159550 2023-10-06T02:23:57.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.909+0000 7f7cfe99d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf80c8220 con 0x7f7cf8159920 2023-10-06T02:23:57.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.909+0000 7f7cfe99d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf80c8500 con 0x7f7cf815dfd0 2023-10-06T02:23:57.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.909+0000 7f7cfd99b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7cf8159550 0x7f7cf8146d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:57.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.909+0000 7f7cfd99b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7cf8159550 0x7f7cf8146d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49664/0 (socket says 172.21.15.31:49664) 2023-10-06T02:23:57.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.909+0000 7f7cfd99b700 1 -- 172.21.15.31:0/932052500 learned_addr learned my addr 172.21.15.31:0/932052500 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:57.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.909+0000 7f7cfd99b700 1 -- 172.21.15.31:0/932052500 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7cf815dfd0 msgr2=0x7f7cf80c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:57.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.909+0000 7f7cfe19c700 1 --2- 172.21.15.31:0/932052500 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7cf815dfd0 0x7f7cf80c9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:57.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.909+0000 7f7cfd19a700 1 --2- 172.21.15.31:0/932052500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7cf8159920 0x7f7cf80c7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:57.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.909+0000 7f7cfd99b700 1 --2- 172.21.15.31:0/932052500 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7cf815dfd0 0x7f7cf80c9cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:57.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.909+0000 7f7cfd99b700 1 -- 172.21.15.31:0/932052500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7cf8159920 msgr2=0x7f7cf80c7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:57.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.909+0000 7f7cfd99b700 1 --2- 172.21.15.31:0/932052500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7cf8159920 0x7f7cf80c7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:57.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.909+0000 7f7cfd99b700 1 -- 172.21.15.31:0/932052500 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cf0009ae0 con 0x7f7cf8159550 2023-10-06T02:23:57.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.909+0000 7f7cfd19a700 1 --2- 172.21.15.31:0/932052500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7cf8159920 0x7f7cf80c7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:23:57.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.909+0000 7f7cfd99b700 1 --2- 172.21.15.31:0/932052500 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7cf8159550 0x7f7cf8146d20 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7cf400e8d0 tx=0x7f7cf4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:57.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.910+0000 7f7cedffb700 1 -- 172.21.15.31:0/932052500 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf400ee70 con 0x7f7cf8159550 2023-10-06T02:23:57.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.910+0000 7f7cedffb700 1 -- 172.21.15.31:0/932052500 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cf400c5e0 con 0x7f7cf8159550 2023-10-06T02:23:57.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.910+0000 7f7cedffb700 1 -- 172.21.15.31:0/932052500 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf400c980 con 0x7f7cf8159550 2023-10-06T02:23:57.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.910+0000 7f7cfe99d700 1 -- 172.21.15.31:0/932052500 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cf80c87e0 con 0x7f7cf8159550 2023-10-06T02:23:57.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.910+0000 7f7cfe99d700 1 -- 172.21.15.31:0/932052500 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cf8147260 con 0x7f7cf8159550 2023-10-06T02:23:57.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.911+0000 7f7cfe99d700 1 -- 172.21.15.31:0/932052500 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cf80c8e30 con 0x7f7cf8159550 2023-10-06T02:23:57.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.912+0000 7f7cedffb700 1 -- 172.21.15.31:0/932052500 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7cf4013410 con 0x7f7cf8159550 2023-10-06T02:23:57.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.913+0000 7f7cedffb700 1 --2- 172.21.15.31:0/932052500 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f7cdc066f30 0x7f7cdc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:57.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.913+0000 7f7cedffb700 1 -- 172.21.15.31:0/932052500 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f7cf4084800 con 0x7f7cf8159550 2023-10-06T02:23:57.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.913+0000 7f7cfd19a700 1 --2- 172.21.15.31:0/932052500 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f7cdc066f30 0x7f7cdc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:57.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.914+0000 7f7cfd19a700 1 --2- 172.21.15.31:0/932052500 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f7cdc066f30 0x7f7cdc0693e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7cf815de90 tx=0x7f7ce8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:57.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:57.917+0000 7f7cedffb700 1 -- 172.21.15.31:0/932052500 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7cf4051830 con 0x7f7cf8159550 2023-10-06T02:23:58.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.127+0000 7f7cfe99d700 1 -- 172.21.15.31:0/932052500 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7cf804c3c0 con 0x7f7cf8159550 2023-10-06T02:23:58.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.128+0000 7f7cedffb700 1 -- 172.21.15.31:0/932052500 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+16914 (secure 0 0 0) 0x7f7cf4054e50 con 0x7f7cf8159550 2023-10-06T02:23:58.187 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:23:58.187 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":25,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:23:48.196317+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:23:07.290269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T02:23:08.504679+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.2f","mappings":[{"from":3,"to":2}]},{"pgid":"4.31","mappings":[{"from":1,"to":5}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:23:58.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.130+0000 7f7cfe99d700 1 -- 172.21.15.31:0/932052500 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f7cdc066f30 msgr2=0x7f7cdc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:58.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.130+0000 7f7cfe99d700 1 --2- 172.21.15.31:0/932052500 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f7cdc066f30 0x7f7cdc0693e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7cf815de90 tx=0x7f7ce8008040 comp rx=0 tx=0).stop 2023-10-06T02:23:58.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.130+0000 7f7cfe99d700 1 -- 172.21.15.31:0/932052500 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7cf8159550 msgr2=0x7f7cf8146d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:58.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.130+0000 7f7cfe99d700 1 --2- 172.21.15.31:0/932052500 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7cf8159550 0x7f7cf8146d20 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7cf400e8d0 tx=0x7f7cf4008040 comp rx=0 tx=0).stop 2023-10-06T02:23:58.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.130+0000 7f7cfe99d700 1 -- 172.21.15.31:0/932052500 shutdown_connections 2023-10-06T02:23:58.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.130+0000 7f7cfe99d700 1 --2- 172.21.15.31:0/932052500 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f7cdc066f30 0x7f7cdc0693e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:58.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.130+0000 7f7cfe99d700 1 --2- 172.21.15.31:0/932052500 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7cf815dfd0 0x7f7cf80c9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:58.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.130+0000 7f7cfe99d700 1 --2- 172.21.15.31:0/932052500 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7cf8159550 0x7f7cf8146d20 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:58.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.130+0000 7f7cfe99d700 1 --2- 172.21.15.31:0/932052500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7cf8159920 0x7f7cf80c7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:58.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.130+0000 7f7cfe99d700 1 -- 172.21.15.31:0/932052500 >> 172.21.15.31:0/932052500 conn(0x7f7cf806ce80 msgr2=0x7f7cf80b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:58.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.131+0000 7f7cfe99d700 1 -- 172.21.15.31:0/932052500 shutdown_connections 2023-10-06T02:23:58.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.131+0000 7f7cfe99d700 1 -- 172.21.15.31:0/932052500 wait complete. 2023-10-06T02:23:58.190 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T02:23:58.196 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:23:58.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.382+0000 7f67b319d700 1 Processor -- start 2023-10-06T02:23:58.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.383+0000 7f67b319d700 1 -- start start 2023-10-06T02:23:58.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.384+0000 7f67b319d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f67ac0aaed0 0x7f67ac06e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:58.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.384+0000 7f67b319d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67ac0ca980 con 0x7f67ac0cc5b0 2023-10-06T02:23:58.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.384+0000 7f67b319d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67ac0ad780 con 0x7f67ac0accb0 2023-10-06T02:23:58.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.384+0000 7f67b319d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67ac0ab2d0 con 0x7f67ac0aaed0 2023-10-06T02:23:58.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.384+0000 7f67b199a700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f67ac0cc5b0 0x7f67ac0ca270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:38610/0 (socket says 172.21.15.31:38610) 2023-10-06T02:23:58.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.384+0000 7f67b199a700 1 -- 172.21.15.31:0/3673941355 learned_addr learned my addr 172.21.15.31:0/3673941355 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:23:58.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.384+0000 7f67b299c700 1 --2- 172.21.15.31:0/3673941355 >> v2:172.21.15.62:3300/0 conn(0x7f67ac0aaed0 0x7f67ac06e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:58.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.385+0000 7f67b1199700 1 -- 172.21.15.31:0/3673941355 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2153788892 0 0) 0x7f67ac0ca980 con 0x7f67ac0cc5b0 2023-10-06T02:23:58.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.385+0000 7f67b1199700 1 -- 172.21.15.31:0/3673941355 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6798003570 con 0x7f67ac0cc5b0 2023-10-06T02:23:58.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.385+0000 7f67b1199700 1 -- 172.21.15.31:0/3673941355 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1090172293 0 0) 0x7f67ac0ad780 con 0x7f67ac0accb0 2023-10-06T02:23:58.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.385+0000 7f67b1199700 1 -- 172.21.15.31:0/3673941355 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67ac0ca980 con 0x7f67ac0accb0 2023-10-06T02:23:58.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.385+0000 7f67b1199700 1 -- 172.21.15.31:0/3673941355 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4118366180 0 0) 0x7f6798003570 con 0x7f67ac0cc5b0 2023-10-06T02:23:58.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.385+0000 7f67b1199700 1 -- 172.21.15.31:0/3673941355 >> v2:172.21.15.62:3300/0 conn(0x7f67ac0aaed0 msgr2=0x7f67ac06e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:58.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.385+0000 7f67b1199700 1 --2- 172.21.15.31:0/3673941355 >> v2:172.21.15.62:3300/0 conn(0x7f67ac0aaed0 0x7f67ac06e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:58.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.385+0000 7f67b1199700 1 -- 172.21.15.31:0/3673941355 >> v1:172.21.15.62:6789/0 conn(0x7f67ac0accb0 legacy=0x7f67ac0cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:58.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.385+0000 7f67b1199700 1 -- 172.21.15.31:0/3673941355 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67ac0cee60 con 0x7f67ac0cc5b0 2023-10-06T02:23:58.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.385+0000 7f67b1199700 1 -- 172.21.15.31:0/3673941355 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7f679c001470 con 0x7f67ac0cc5b0 2023-10-06T02:23:58.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.385+0000 7f67b1199700 1 -- 172.21.15.31:0/3673941355 >> v1:172.21.15.31:6789/0 conn(0x7f67ac0cc5b0 legacy=0x7f67ac0ca270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:58.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.385+0000 7f67b1199700 1 --2- 172.21.15.31:0/3673941355 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6798003c40 0x7f6798004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:58.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.385+0000 7f67b1199700 1 --2- 172.21.15.31:0/3673941355 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6798004810 0x7f6798004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:58.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.385+0000 7f67b1199700 1 --2- 172.21.15.31:0/3673941355 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6798005210 0x7f679800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:58.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.385+0000 7f67b1199700 1 -- 172.21.15.31:0/3673941355 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67ac0ab2d0 con 0x7f6798003c40 2023-10-06T02:23:58.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.385+0000 7f67b1199700 1 -- 172.21.15.31:0/3673941355 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f679800c130 con 0x7f6798005210 2023-10-06T02:23:58.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.385+0000 7f67b1199700 1 -- 172.21.15.31:0/3673941355 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f679800c2c0 con 0x7f6798004810 2023-10-06T02:23:58.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.385+0000 7f67b219b700 1 --2- 172.21.15.31:0/3673941355 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6798003c40 0x7f6798004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:58.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.386+0000 7f67b199a700 1 --2- 172.21.15.31:0/3673941355 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6798004810 0x7f6798004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:58.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.386+0000 7f67b299c700 1 --2- 172.21.15.31:0/3673941355 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6798005210 0x7f679800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:58.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.386+0000 7f67b219b700 1 -- 172.21.15.31:0/3673941355 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6798004810 msgr2=0x7f6798004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:58.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.386+0000 7f67b219b700 1 --2- 172.21.15.31:0/3673941355 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6798004810 0x7f6798004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:58.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.386+0000 7f67b219b700 1 -- 172.21.15.31:0/3673941355 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6798005210 msgr2=0x7f679800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:58.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.386+0000 7f67b219b700 1 --2- 172.21.15.31:0/3673941355 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6798005210 0x7f679800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:58.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.386+0000 7f67b219b700 1 -- 172.21.15.31:0/3673941355 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6798003570 con 0x7f6798003c40 2023-10-06T02:23:58.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.386+0000 7f67b299c700 1 --2- 172.21.15.31:0/3673941355 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6798005210 0x7f679800ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:23:58.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.386+0000 7f67b299c700 1 -- 172.21.15.31:0/3673941355 reap_dead start 2023-10-06T02:23:58.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.386+0000 7f67b219b700 1 --2- 172.21.15.31:0/3673941355 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6798003c40 0x7f6798004030 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f67a8001a40 tx=0x7f67a8007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=267fa7a6f3b19610 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:58.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.386+0000 7f67b1199700 1 -- 172.21.15.31:0/3673941355 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a800c590 con 0x7f6798003c40 2023-10-06T02:23:58.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.386+0000 7f67b1199700 1 -- 172.21.15.31:0/3673941355 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67a800c6f0 con 0x7f6798003c40 2023-10-06T02:23:58.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.386+0000 7f67b1199700 1 -- 172.21.15.31:0/3673941355 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a800cab0 con 0x7f6798003c40 2023-10-06T02:23:58.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.387+0000 7f67b319d700 1 -- 172.21.15.31:0/3673941355 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6798003c40 msgr2=0x7f6798004030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:58.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.387+0000 7f67b319d700 1 --2- 172.21.15.31:0/3673941355 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6798003c40 0x7f6798004030 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f67a8001a40 tx=0x7f67a8007d30 comp rx=0 tx=0).stop 2023-10-06T02:23:58.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.387+0000 7f67b319d700 1 -- 172.21.15.31:0/3673941355 shutdown_connections 2023-10-06T02:23:58.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.387+0000 7f67b319d700 1 --2- 172.21.15.31:0/3673941355 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6798003c40 0x7f6798004030 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:58.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.387+0000 7f67b319d700 1 -- 172.21.15.31:0/3673941355 >> 172.21.15.31:0/3673941355 conn(0x7f67ac06ce80 msgr2=0x7f67ac0ce370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:58.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.387+0000 7f67b319d700 1 -- 172.21.15.31:0/3673941355 shutdown_connections 2023-10-06T02:23:58.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.387+0000 7f67b319d700 1 -- 172.21.15.31:0/3673941355 wait complete. 2023-10-06T02:23:58.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.388+0000 7f67b319d700 1 Processor -- start 2023-10-06T02:23:58.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.389+0000 7f67b319d700 1 -- start start 2023-10-06T02:23:58.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.389+0000 7f67b319d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6798003c40 0x7f67ac158690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:58.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.389+0000 7f67b319d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f67ac1695a0 0x7f67ac0b3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:58.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.389+0000 7f67b319d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67ac0c7390 0x7f67ac0c7810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:58.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.389+0000 7f67b319d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67ac0c7d50 con 0x7f6798003c40 2023-10-06T02:23:58.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.389+0000 7f67b319d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67ac0c1ed0 con 0x7f67ac0c7390 2023-10-06T02:23:58.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.389+0000 7f67b319d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67ac169a10 con 0x7f67ac1695a0 2023-10-06T02:23:58.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.390+0000 7f67b219b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6798003c40 0x7f67ac158690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:58.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.390+0000 7f67b219b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6798003c40 0x7f67ac158690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49680/0 (socket says 172.21.15.31:49680) 2023-10-06T02:23:58.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.390+0000 7f67b219b700 1 -- 172.21.15.31:0/388150799 learned_addr learned my addr 172.21.15.31:0/388150799 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:58.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.390+0000 7f67b199a700 1 --2- 172.21.15.31:0/388150799 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f67ac1695a0 0x7f67ac0b3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:58.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.390+0000 7f67b299c700 1 --2- 172.21.15.31:0/388150799 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67ac0c7390 0x7f67ac0c7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:58.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.390+0000 7f67b219b700 1 -- 172.21.15.31:0/388150799 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f67ac1695a0 msgr2=0x7f67ac0b3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:58.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.390+0000 7f67b219b700 1 --2- 172.21.15.31:0/388150799 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f67ac1695a0 0x7f67ac0b3a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:58.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.390+0000 7f67b219b700 1 -- 172.21.15.31:0/388150799 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67ac0c7390 msgr2=0x7f67ac0c7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:58.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.390+0000 7f67b219b700 1 --2- 172.21.15.31:0/388150799 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67ac0c7390 0x7f67ac0c7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:58.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.390+0000 7f67b219b700 1 -- 172.21.15.31:0/388150799 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67ac0b3fa0 con 0x7f6798003c40 2023-10-06T02:23:58.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.390+0000 7f67b219b700 1 --2- 172.21.15.31:0/388150799 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6798003c40 0x7f67ac158690 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f67a800c3c0 tx=0x7f67a8015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:58.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.391+0000 7f67a27fc700 1 -- 172.21.15.31:0/388150799 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a800c590 con 0x7f6798003c40 2023-10-06T02:23:58.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.391+0000 7f67a27fc700 1 -- 172.21.15.31:0/388150799 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67a800ce60 con 0x7f6798003c40 2023-10-06T02:23:58.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.391+0000 7f67a27fc700 1 -- 172.21.15.31:0/388150799 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a801e400 con 0x7f6798003c40 2023-10-06T02:23:58.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.391+0000 7f67b319d700 1 -- 172.21.15.31:0/388150799 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67ac169c60 con 0x7f6798003c40 2023-10-06T02:23:58.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.391+0000 7f67b319d700 1 -- 172.21.15.31:0/388150799 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67ac16a220 con 0x7f6798003c40 2023-10-06T02:23:58.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.392+0000 7f67b319d700 1 -- 172.21.15.31:0/388150799 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67ac0c7d50 con 0x7f6798003c40 2023-10-06T02:23:58.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.393+0000 7f67a27fc700 1 -- 172.21.15.31:0/388150799 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f67a8019030 con 0x7f6798003c40 2023-10-06T02:23:58.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.393+0000 7f67a27fc700 1 --2- 172.21.15.31:0/388150799 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6790066f30 0x7f67900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:58.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.394+0000 7f67a27fc700 1 -- 172.21.15.31:0/388150799 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f67a8085e80 con 0x7f6798003c40 2023-10-06T02:23:58.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.394+0000 7f67b199a700 1 --2- 172.21.15.31:0/388150799 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6790066f30 0x7f67900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:58.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.394+0000 7f67b199a700 1 --2- 172.21.15.31:0/388150799 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6790066f30 0x7f67900693e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f679c001ed0 tx=0x7f679c007040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:58.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.398+0000 7f67a27fc700 1 -- 172.21.15.31:0/388150799 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f67a8052eb0 con 0x7f6798003c40 2023-10-06T02:23:58.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.642+0000 7f67b319d700 1 -- 172.21.15.31:0/388150799 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f67ac04c3c0 con 0x7f6798003c40 2023-10-06T02:23:58.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.643+0000 7f67a27fc700 1 -- 172.21.15.31:0/388150799 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4591 (secure 0 0 0) 0x7f67a8017090 con 0x7f6798003c40 2023-10-06T02:23:58.673 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:23:58.673 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/1808327183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":1808327183},{"type":"v1","addr":"172.21.15.31:6837","nonce":1808327183}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/4263144517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":4263144517},{"type":"v1","addr":"172.21.15.62:6834","nonce":4263144517}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1489208568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":1489208568},{"type":"v1","addr":"172.21.15.62:6835","nonce":1489208568}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:23:09.521236+0000","modified":"2023-10-06T02:23:13.047274+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6836/3809284664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3809284664},{"type":"v1","addr":"172.21.15.31:6836","nonce":3809284664}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T02:23:58.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.645+0000 7f67b319d700 1 -- 172.21.15.31:0/388150799 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6790066f30 msgr2=0x7f67900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:58.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.645+0000 7f67b319d700 1 --2- 172.21.15.31:0/388150799 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6790066f30 0x7f67900693e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f679c001ed0 tx=0x7f679c007040 comp rx=0 tx=0).stop 2023-10-06T02:23:58.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.645+0000 7f67b319d700 1 -- 172.21.15.31:0/388150799 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6798003c40 msgr2=0x7f67ac158690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:58.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.645+0000 7f67b319d700 1 --2- 172.21.15.31:0/388150799 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6798003c40 0x7f67ac158690 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f67a800c3c0 tx=0x7f67a8015000 comp rx=0 tx=0).stop 2023-10-06T02:23:58.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.645+0000 7f67b319d700 1 -- 172.21.15.31:0/388150799 shutdown_connections 2023-10-06T02:23:58.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.645+0000 7f67b319d700 1 --2- 172.21.15.31:0/388150799 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67ac0c7390 0x7f67ac0c7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:58.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.645+0000 7f67b319d700 1 --2- 172.21.15.31:0/388150799 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6798003c40 0x7f67ac158690 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:58.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.645+0000 7f67b319d700 1 --2- 172.21.15.31:0/388150799 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6790066f30 0x7f67900693e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:58.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.645+0000 7f67b319d700 1 --2- 172.21.15.31:0/388150799 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f67ac1695a0 0x7f67ac0b3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:58.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.645+0000 7f67b319d700 1 -- 172.21.15.31:0/388150799 >> 172.21.15.31:0/388150799 conn(0x7f67ac06ce80 msgr2=0x7f67ac1593d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:58.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.645+0000 7f67b319d700 1 -- 172.21.15.31:0/388150799 shutdown_connections 2023-10-06T02:23:58.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.645+0000 7f67b319d700 1 -- 172.21.15.31:0/388150799 wait complete. 2023-10-06T02:23:58.694 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 8 2023-10-06T02:23:58.694 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:23:58.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.879+0000 7f43ff59e700 1 Processor -- start 2023-10-06T02:23:58.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.879+0000 7f43ff59e700 1 -- start start 2023-10-06T02:23:58.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.881+0000 7f43ff59e700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4400159550 0x7f44000ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:58.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.881+0000 7f43ff59e700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44000acc00 con 0x7f440015dfd0 2023-10-06T02:23:58.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.881+0000 7f43ff59e700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44000abff0 con 0x7f4400159920 2023-10-06T02:23:58.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.881+0000 7f43ff59e700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44000ac2d0 con 0x7f4400159550 2023-10-06T02:23:58.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.881+0000 7f43fdd9b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f440015dfd0 0x7f44000aedb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:38618/0 (socket says 172.21.15.31:38618) 2023-10-06T02:23:58.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.881+0000 7f43fdd9b700 1 -- 172.21.15.31:0/439425647 learned_addr learned my addr 172.21.15.31:0/439425647 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:23:58.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.881+0000 7f43fe59c700 1 --2- 172.21.15.31:0/439425647 >> v2:172.21.15.62:3300/0 conn(0x7f4400159550 0x7f44000ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:58.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.882+0000 7f43fd59a700 1 -- 172.21.15.31:0/439425647 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4141671850 0 0) 0x7f44000acc00 con 0x7f440015dfd0 2023-10-06T02:23:58.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.882+0000 7f43fd59a700 1 -- 172.21.15.31:0/439425647 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43e8003570 con 0x7f440015dfd0 2023-10-06T02:23:58.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.882+0000 7f43fe59c700 1 -- 172.21.15.31:0/439425647 >> v1:172.21.15.62:6789/0 conn(0x7f4400159920 legacy=0x7f44000ad090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:58.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.882+0000 7f43fe59c700 1 -- 172.21.15.31:0/439425647 >> v1:172.21.15.31:6789/0 conn(0x7f440015dfd0 legacy=0x7f44000aedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:58.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.882+0000 7f43fe59c700 1 -- 172.21.15.31:0/439425647 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44000ac5b0 con 0x7f4400159550 2023-10-06T02:23:58.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.882+0000 7f43fe59c700 1 --2- 172.21.15.31:0/439425647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4400159550 0x7f44000ceac0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f43f00098e0 tx=0x7f43f00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5dc62a778308295 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:58.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.882+0000 7f43fd59a700 1 -- 172.21.15.31:0/439425647 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f000b640 con 0x7f4400159550 2023-10-06T02:23:58.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.882+0000 7f43fd59a700 1 -- 172.21.15.31:0/439425647 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43f000b7a0 con 0x7f4400159550 2023-10-06T02:23:58.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.882+0000 7f43fd59a700 1 -- 172.21.15.31:0/439425647 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f000bb40 con 0x7f4400159550 2023-10-06T02:23:58.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.883+0000 7f43ff59e700 1 -- 172.21.15.31:0/439425647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4400159550 msgr2=0x7f44000ceac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:58.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.883+0000 7f43ff59e700 1 --2- 172.21.15.31:0/439425647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4400159550 0x7f44000ceac0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f43f00098e0 tx=0x7f43f00091a0 comp rx=0 tx=0).stop 2023-10-06T02:23:58.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.883+0000 7f43ff59e700 1 -- 172.21.15.31:0/439425647 shutdown_connections 2023-10-06T02:23:58.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.883+0000 7f43ff59e700 1 --2- 172.21.15.31:0/439425647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4400159550 0x7f44000ceac0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:58.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.883+0000 7f43ff59e700 1 -- 172.21.15.31:0/439425647 >> 172.21.15.31:0/439425647 conn(0x7f440006ce80 msgr2=0x7f44000ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:58.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.883+0000 7f43ff59e700 1 -- 172.21.15.31:0/439425647 shutdown_connections 2023-10-06T02:23:58.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.883+0000 7f43ff59e700 1 -- 172.21.15.31:0/439425647 wait complete. 2023-10-06T02:23:58.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.884+0000 7f43ff59e700 1 Processor -- start 2023-10-06T02:23:58.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.885+0000 7f43ff59e700 1 -- start start 2023-10-06T02:23:58.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.885+0000 7f43ff59e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4400159550 0x7f4400146ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:58.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.885+0000 7f43ff59e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4400159920 0x7f4400153c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:58.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.885+0000 7f43ff59e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f440015dfd0 0x7f44001541d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:58.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.885+0000 7f43ff59e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4400147400 con 0x7f440015dfd0 2023-10-06T02:23:58.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.885+0000 7f43ff59e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44000cb710 con 0x7f4400159550 2023-10-06T02:23:58.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.885+0000 7f43ff59e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44000cae50 con 0x7f4400159920 2023-10-06T02:23:58.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.886+0000 7f43fed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f440015dfd0 0x7f44001541d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:58.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.886+0000 7f43fed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f440015dfd0 0x7f44001541d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49682/0 (socket says 172.21.15.31:49682) 2023-10-06T02:23:58.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.886+0000 7f43fed9d700 1 -- 172.21.15.31:0/104290412 learned_addr learned my addr 172.21.15.31:0/104290412 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:58.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.886+0000 7f43fe59c700 1 --2- 172.21.15.31:0/104290412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4400159550 0x7f4400146ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:58.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.886+0000 7f43fdd9b700 1 --2- 172.21.15.31:0/104290412 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4400159920 0x7f4400153c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:58.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.886+0000 7f43fed9d700 1 -- 172.21.15.31:0/104290412 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4400159920 msgr2=0x7f4400153c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:58.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.886+0000 7f43fed9d700 1 --2- 172.21.15.31:0/104290412 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4400159920 0x7f4400153c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:58.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.886+0000 7f43fed9d700 1 -- 172.21.15.31:0/104290412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4400159550 msgr2=0x7f4400146ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:58.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.886+0000 7f43fed9d700 1 --2- 172.21.15.31:0/104290412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4400159550 0x7f4400146ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:58.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.886+0000 7f43fed9d700 1 -- 172.21.15.31:0/104290412 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43f0009420 con 0x7f440015dfd0 2023-10-06T02:23:58.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.886+0000 7f43fed9d700 1 --2- 172.21.15.31:0/104290412 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f440015dfd0 0x7f44001541d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f43f800c910 tx=0x7f43f8001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:58.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.887+0000 7f43ee7fc700 1 -- 172.21.15.31:0/104290412 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f8012630 con 0x7f440015dfd0 2023-10-06T02:23:58.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.887+0000 7f43ee7fc700 1 -- 172.21.15.31:0/104290412 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43f8012790 con 0x7f440015dfd0 2023-10-06T02:23:58.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.887+0000 7f43ee7fc700 1 -- 172.21.15.31:0/104290412 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f8012b50 con 0x7f440015dfd0 2023-10-06T02:23:58.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.887+0000 7f43ff59e700 1 -- 172.21.15.31:0/104290412 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44000cb0c0 con 0x7f440015dfd0 2023-10-06T02:23:58.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.887+0000 7f43ff59e700 1 -- 172.21.15.31:0/104290412 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44000cdcd0 con 0x7f440015dfd0 2023-10-06T02:23:58.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.888+0000 7f43ff59e700 1 -- 172.21.15.31:0/104290412 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44000cb880 con 0x7f440015dfd0 2023-10-06T02:23:58.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.889+0000 7f43ee7fc700 1 -- 172.21.15.31:0/104290412 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f43f801a3a0 con 0x7f440015dfd0 2023-10-06T02:23:58.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.890+0000 7f43ee7fc700 1 --2- 172.21.15.31:0/104290412 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f43e0066f30 0x7f43e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:58.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.890+0000 7f43ee7fc700 1 -- 172.21.15.31:0/104290412 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f43f8084960 con 0x7f440015dfd0 2023-10-06T02:23:58.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.890+0000 7f43fe59c700 1 --2- 172.21.15.31:0/104290412 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f43e0066f30 0x7f43e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:58.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.891+0000 7f43fe59c700 1 --2- 172.21.15.31:0/104290412 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f43e0066f30 0x7f43e00693e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f43f00093f0 tx=0x7f43f0017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:58.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:58.895+0000 7f43ee7fc700 1 -- 172.21.15.31:0/104290412 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f43f8051990 con 0x7f440015dfd0 2023-10-06T02:23:59.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.143+0000 7f43ff59e700 1 -- 172.21.15.31:0/104290412 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f440004c3c0 con 0x7f440015dfd0 2023-10-06T02:23:59.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.143+0000 7f43ee7fc700 1 -- 172.21.15.31:0/104290412 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4591 (secure 0 0 0) 0x7f43f8054fb0 con 0x7f440015dfd0 2023-10-06T02:23:59.145 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:23:59.145 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/1808327183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":1808327183},{"type":"v1","addr":"172.21.15.31:6837","nonce":1808327183}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/4263144517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":4263144517},{"type":"v1","addr":"172.21.15.62:6834","nonce":4263144517}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1489208568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":1489208568},{"type":"v1","addr":"172.21.15.62:6835","nonce":1489208568}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:23:09.521236+0000","modified":"2023-10-06T02:23:13.047274+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6836/3809284664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3809284664},{"type":"v1","addr":"172.21.15.31:6836","nonce":3809284664}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T02:23:59.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.145+0000 7f43ff59e700 1 -- 172.21.15.31:0/104290412 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f43e0066f30 msgr2=0x7f43e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:59.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.145+0000 7f43ff59e700 1 --2- 172.21.15.31:0/104290412 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f43e0066f30 0x7f43e00693e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f43f00093f0 tx=0x7f43f0017000 comp rx=0 tx=0).stop 2023-10-06T02:23:59.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.145+0000 7f43ff59e700 1 -- 172.21.15.31:0/104290412 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f440015dfd0 msgr2=0x7f44001541d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:59.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.145+0000 7f43ff59e700 1 --2- 172.21.15.31:0/104290412 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f440015dfd0 0x7f44001541d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f43f800c910 tx=0x7f43f8001710 comp rx=0 tx=0).stop 2023-10-06T02:23:59.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.145+0000 7f43ff59e700 1 -- 172.21.15.31:0/104290412 shutdown_connections 2023-10-06T02:23:59.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.145+0000 7f43ff59e700 1 --2- 172.21.15.31:0/104290412 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f440015dfd0 0x7f44001541d0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:59.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.145+0000 7f43ff59e700 1 --2- 172.21.15.31:0/104290412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4400159550 0x7f4400146ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:59.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.145+0000 7f43ff59e700 1 --2- 172.21.15.31:0/104290412 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f43e0066f30 0x7f43e00693e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:59.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.146+0000 7f43ff59e700 1 --2- 172.21.15.31:0/104290412 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4400159920 0x7f4400153c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:59.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.146+0000 7f43ff59e700 1 -- 172.21.15.31:0/104290412 >> 172.21.15.31:0/104290412 conn(0x7f440006ce80 msgr2=0x7f4400152bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:59.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.146+0000 7f43ff59e700 1 -- 172.21.15.31:0/104290412 shutdown_connections 2023-10-06T02:23:59.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.146+0000 7f43ff59e700 1 -- 172.21.15.31:0/104290412 wait complete. 2023-10-06T02:23:59.148 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 8 2023-10-06T02:23:59.165 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:23:59.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.351+0000 7fa98e5ff700 1 Processor -- start 2023-10-06T02:23:59.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.352+0000 7fa98e5ff700 1 -- start start 2023-10-06T02:23:59.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.353+0000 7fa98e5ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa988159c20 0x7fa9880b0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:59.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.353+0000 7fa98e5ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa9880b0e70 0x7fa9880c93e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:59.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.353+0000 7fa98e5ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9880adb20 con 0x7fa988159850 2023-10-06T02:23:59.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.353+0000 7fa98e5ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9880ccdc0 con 0x7fa988159c20 2023-10-06T02:23:59.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.353+0000 7fa98e5ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9880b12f0 con 0x7fa9880b0e70 2023-10-06T02:23:59.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.353+0000 7fa9877fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa988159c20 0x7fa9880b0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:59.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.353+0000 7fa9877fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa988159c20 0x7fa9880b0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49690/0 (socket says 172.21.15.31:49690) 2023-10-06T02:23:59.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.353+0000 7fa9877fe700 1 -- 172.21.15.31:0/3925874453 learned_addr learned my addr 172.21.15.31:0/3925874453 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:59.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.353+0000 7fa98cb9c700 1 --2- 172.21.15.31:0/3925874453 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa9880b0e70 0x7fa9880c93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:59.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.354+0000 7fa9877fe700 1 -- 172.21.15.31:0/3925874453 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa9880b0e70 msgr2=0x7fa9880c93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:23:59.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.354+0000 7fa9877fe700 1 --2- 172.21.15.31:0/3925874453 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa9880b0e70 0x7fa9880c93e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:59.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.354+0000 7fa9877fe700 1 -- 172.21.15.31:0/3925874453 >> v1:172.21.15.62:6789/0 conn(0x7fa988159850 legacy=0x7fa9880adfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:59.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.354+0000 7fa9877fe700 1 -- 172.21.15.31:0/3925874453 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9880acf10 con 0x7fa988159c20 2023-10-06T02:23:59.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.354+0000 7fa98cb9c700 1 --2- 172.21.15.31:0/3925874453 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa9880b0e70 0x7fa9880c93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:23:59.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.354+0000 7fa9877fe700 1 --2- 172.21.15.31:0/3925874453 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa988159c20 0x7fa9880b0930 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa97c0099e0 tx=0x7fa97c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9adcb429b43dc9d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:59.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.354+0000 7fa986ffd700 1 -- 172.21.15.31:0/3925874453 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa97c00b820 con 0x7fa988159c20 2023-10-06T02:23:59.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.354+0000 7fa986ffd700 1 -- 172.21.15.31:0/3925874453 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa97c00b980 con 0x7fa988159c20 2023-10-06T02:23:59.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.354+0000 7fa986ffd700 1 -- 172.21.15.31:0/3925874453 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa97c00bd20 con 0x7fa988159c20 2023-10-06T02:23:59.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.355+0000 7fa98e5ff700 1 -- 172.21.15.31:0/3925874453 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa988159c20 msgr2=0x7fa9880b0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:59.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.355+0000 7fa98e5ff700 1 --2- 172.21.15.31:0/3925874453 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa988159c20 0x7fa9880b0930 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa97c0099e0 tx=0x7fa97c009250 comp rx=0 tx=0).stop 2023-10-06T02:23:59.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.355+0000 7fa98e5ff700 1 -- 172.21.15.31:0/3925874453 shutdown_connections 2023-10-06T02:23:59.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.355+0000 7fa98e5ff700 1 --2- 172.21.15.31:0/3925874453 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa9880b0e70 0x7fa9880c93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:59.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.355+0000 7fa98e5ff700 1 --2- 172.21.15.31:0/3925874453 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa988159c20 0x7fa9880b0930 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:59.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.355+0000 7fa98e5ff700 1 -- 172.21.15.31:0/3925874453 >> 172.21.15.31:0/3925874453 conn(0x7fa98806d170 msgr2=0x7fa9880cc300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:59.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.355+0000 7fa98e5ff700 1 -- 172.21.15.31:0/3925874453 shutdown_connections 2023-10-06T02:23:59.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.355+0000 7fa98e5ff700 1 -- 172.21.15.31:0/3925874453 wait complete. 2023-10-06T02:23:59.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.356+0000 7fa98e5ff700 1 Processor -- start 2023-10-06T02:23:59.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.357+0000 7fa98e5ff700 1 -- start start 2023-10-06T02:23:59.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.357+0000 7fa98e5ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9880b0e70 0x7fa988146ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:59.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.357+0000 7fa98e5ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa988159850 0x7fa9880af870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:59.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.357+0000 7fa98e5ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa988159c20 0x7fa9880afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:59.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.357+0000 7fa98e5ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9880aa7f0 con 0x7fa9880b0e70 2023-10-06T02:23:59.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.357+0000 7fa98e5ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa988147430 con 0x7fa988159c20 2023-10-06T02:23:59.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.357+0000 7fa98e5ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa98815d3b0 con 0x7fa988159850 2023-10-06T02:23:59.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.358+0000 7fa987fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9880b0e70 0x7fa988146ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:59.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.358+0000 7fa987fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9880b0e70 0x7fa988146ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49696/0 (socket says 172.21.15.31:49696) 2023-10-06T02:23:59.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.358+0000 7fa987fff700 1 -- 172.21.15.31:0/1336239487 learned_addr learned my addr 172.21.15.31:0/1336239487 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:59.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.358+0000 7fa98cb9c700 1 --2- 172.21.15.31:0/1336239487 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa988159c20 0x7fa9880afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:59.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.358+0000 7fa9877fe700 1 --2- 172.21.15.31:0/1336239487 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa988159850 0x7fa9880af870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:59.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.358+0000 7fa987fff700 1 -- 172.21.15.31:0/1336239487 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa988159850 msgr2=0x7fa9880af870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:59.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.358+0000 7fa987fff700 1 --2- 172.21.15.31:0/1336239487 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa988159850 0x7fa9880af870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:59.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.358+0000 7fa987fff700 1 -- 172.21.15.31:0/1336239487 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa988159c20 msgr2=0x7fa9880afdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:59.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.358+0000 7fa987fff700 1 --2- 172.21.15.31:0/1336239487 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa988159c20 0x7fa9880afdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:59.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.358+0000 7fa987fff700 1 -- 172.21.15.31:0/1336239487 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa97c0094f0 con 0x7fa9880b0e70 2023-10-06T02:23:59.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.358+0000 7fa987fff700 1 --2- 172.21.15.31:0/1336239487 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9880b0e70 0x7fa988146ef0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa97800c5c0 tx=0x7fa97800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:59.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.359+0000 7fa96ffff700 1 -- 172.21.15.31:0/1336239487 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa978011590 con 0x7fa9880b0e70 2023-10-06T02:23:59.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.359+0000 7fa96ffff700 1 -- 172.21.15.31:0/1336239487 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa978011710 con 0x7fa9880b0e70 2023-10-06T02:23:59.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.359+0000 7fa96ffff700 1 -- 172.21.15.31:0/1336239487 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa978011ad0 con 0x7fa9880b0e70 2023-10-06T02:23:59.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.359+0000 7fa98e5ff700 1 -- 172.21.15.31:0/1336239487 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa98815def0 con 0x7fa9880b0e70 2023-10-06T02:23:59.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.359+0000 7fa98e5ff700 1 -- 172.21.15.31:0/1336239487 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa98815d8a0 con 0x7fa9880b0e70 2023-10-06T02:23:59.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.360+0000 7fa98e5ff700 1 -- 172.21.15.31:0/1336239487 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa98804c3c0 con 0x7fa9880b0e70 2023-10-06T02:23:59.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.361+0000 7fa96ffff700 1 -- 172.21.15.31:0/1336239487 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa97801b3b0 con 0x7fa9880b0e70 2023-10-06T02:23:59.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.361+0000 7fa96ffff700 1 --2- 172.21.15.31:0/1336239487 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa974066f30 0x7fa9740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:59.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.362+0000 7fa96ffff700 1 -- 172.21.15.31:0/1336239487 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6116+0+0 (secure 0 0 0) 0x7fa978083ca0 con 0x7fa9880b0e70 2023-10-06T02:23:59.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.362+0000 7fa9877fe700 1 --2- 172.21.15.31:0/1336239487 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa974066f30 0x7fa9740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:59.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.362+0000 7fa9877fe700 1 --2- 172.21.15.31:0/1336239487 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa974066f30 0x7fa9740693e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa97c0099e0 tx=0x7fa97c017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:59.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.366+0000 7fa96ffff700 1 -- 172.21.15.31:0/1336239487 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa978050cd0 con 0x7fa9880b0e70 2023-10-06T02:23:59.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.569+0000 7fa98e5ff700 1 -- 172.21.15.31:0/1336239487 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa9880ad1f0 con 0x7fa9880b0e70 2023-10-06T02:23:59.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.570+0000 7fa96ffff700 1 -- 172.21.15.31:0/1336239487 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+16914 (secure 0 0 0) 0x7fa978017020 con 0x7fa9880b0e70 2023-10-06T02:23:59.572 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:23:59.572 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":25,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:23:48.196317+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:23:07.290269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T02:23:08.504679+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.2f","mappings":[{"from":3,"to":2}]},{"pgid":"4.31","mappings":[{"from":1,"to":5}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:23:59.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.573+0000 7fa98e5ff700 1 -- 172.21.15.31:0/1336239487 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa974066f30 msgr2=0x7fa9740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:59.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.573+0000 7fa98e5ff700 1 --2- 172.21.15.31:0/1336239487 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa974066f30 0x7fa9740693e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa97c0099e0 tx=0x7fa97c017000 comp rx=0 tx=0).stop 2023-10-06T02:23:59.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.573+0000 7fa98e5ff700 1 -- 172.21.15.31:0/1336239487 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9880b0e70 msgr2=0x7fa988146ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:59.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.573+0000 7fa98e5ff700 1 --2- 172.21.15.31:0/1336239487 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9880b0e70 0x7fa988146ef0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa97800c5c0 tx=0x7fa97800c8d0 comp rx=0 tx=0).stop 2023-10-06T02:23:59.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.573+0000 7fa98e5ff700 1 -- 172.21.15.31:0/1336239487 shutdown_connections 2023-10-06T02:23:59.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.573+0000 7fa98e5ff700 1 --2- 172.21.15.31:0/1336239487 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa988159c20 0x7fa9880afdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:59.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.573+0000 7fa98e5ff700 1 --2- 172.21.15.31:0/1336239487 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9880b0e70 0x7fa988146ef0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:59.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.573+0000 7fa98e5ff700 1 --2- 172.21.15.31:0/1336239487 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa974066f30 0x7fa9740693e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:59.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.573+0000 7fa98e5ff700 1 --2- 172.21.15.31:0/1336239487 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa988159850 0x7fa9880af870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:59.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.573+0000 7fa98e5ff700 1 -- 172.21.15.31:0/1336239487 >> 172.21.15.31:0/1336239487 conn(0x7fa98806d170 msgr2=0x7fa98806ac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:59.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.573+0000 7fa98e5ff700 1 -- 172.21.15.31:0/1336239487 shutdown_connections 2023-10-06T02:23:59.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.573+0000 7fa98e5ff700 1 -- 172.21.15.31:0/1336239487 wait complete. 2023-10-06T02:23:59.592 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T02:23:59.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.778+0000 7f6da9f9d700 1 Processor -- start 2023-10-06T02:23:59.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.778+0000 7f6da9f9d700 1 -- start start 2023-10-06T02:23:59.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.779+0000 7f6da9f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6da413fb20 0x7f6da415a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:59.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.779+0000 7f6da9f9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6da415ae70 con 0x7f6da413fef0 2023-10-06T02:23:59.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.780+0000 7f6da9f9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6da406af60 con 0x7f6da406aaa0 2023-10-06T02:23:59.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.780+0000 7f6da9f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6da406b240 con 0x7f6da413fb20 2023-10-06T02:23:59.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.780+0000 7f6da3fff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f6da413fef0 0x7f6da40c7990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:38624/0 (socket says 172.21.15.31:38624) 2023-10-06T02:23:59.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.780+0000 7f6da3fff700 1 -- 172.21.15.31:0/1367428626 learned_addr learned my addr 172.21.15.31:0/1367428626 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:23:59.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.780+0000 7f6da8f9b700 1 --2- 172.21.15.31:0/1367428626 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6da413fb20 0x7f6da415a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:59.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.780+0000 7f6da37fe700 1 -- 172.21.15.31:0/1367428626 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4242952013 0 0) 0x7f6d90001290 con 0x7f6da413fef0 2023-10-06T02:23:59.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.780+0000 7f6da37fe700 1 -- 172.21.15.31:0/1367428626 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d94003570 con 0x7f6da413fef0 2023-10-06T02:23:59.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.781+0000 7f6da37fe700 1 -- 172.21.15.31:0/1367428626 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3602867601 0 0) 0x7f6da406af60 con 0x7f6da406aaa0 2023-10-06T02:23:59.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.781+0000 7f6da37fe700 1 -- 172.21.15.31:0/1367428626 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d90001290 con 0x7f6da406aaa0 2023-10-06T02:23:59.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.781+0000 7f6da8f9b700 1 -- 172.21.15.31:0/1367428626 >> v1:172.21.15.62:6789/0 conn(0x7f6da406aaa0 legacy=0x7f6da40c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:59.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.781+0000 7f6da8f9b700 1 -- 172.21.15.31:0/1367428626 >> v1:172.21.15.31:6789/0 conn(0x7f6da413fef0 legacy=0x7f6da40c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:59.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.781+0000 7f6da8f9b700 1 -- 172.21.15.31:0/1367428626 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6da406b520 con 0x7f6da413fb20 2023-10-06T02:23:59.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.781+0000 7f6da8f9b700 1 --2- 172.21.15.31:0/1367428626 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6da413fb20 0x7f6da415a930 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6d98009910 tx=0x7f6d980091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3ef88a9b62f96f7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:59.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.781+0000 7f6da37fe700 1 -- 172.21.15.31:0/1367428626 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d98011630 con 0x7f6da413fb20 2023-10-06T02:23:59.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.781+0000 7f6da37fe700 1 -- 172.21.15.31:0/1367428626 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d98011790 con 0x7f6da413fb20 2023-10-06T02:23:59.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.781+0000 7f6da37fe700 1 -- 172.21.15.31:0/1367428626 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d98011b30 con 0x7f6da413fb20 2023-10-06T02:23:59.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.782+0000 7f6da9f9d700 1 -- 172.21.15.31:0/1367428626 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6da413fb20 msgr2=0x7f6da415a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:59.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.782+0000 7f6da9f9d700 1 --2- 172.21.15.31:0/1367428626 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6da413fb20 0x7f6da415a930 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6d98009910 tx=0x7f6d980091a0 comp rx=0 tx=0).stop 2023-10-06T02:23:59.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.782+0000 7f6da9f9d700 1 -- 172.21.15.31:0/1367428626 shutdown_connections 2023-10-06T02:23:59.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.782+0000 7f6da9f9d700 1 --2- 172.21.15.31:0/1367428626 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6da413fb20 0x7f6da415a930 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:59.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.782+0000 7f6da9f9d700 1 -- 172.21.15.31:0/1367428626 >> 172.21.15.31:0/1367428626 conn(0x7f6da406d170 msgr2=0x7f6da40cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:23:59.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.782+0000 7f6da9f9d700 1 -- 172.21.15.31:0/1367428626 shutdown_connections 2023-10-06T02:23:59.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.782+0000 7f6da9f9d700 1 -- 172.21.15.31:0/1367428626 wait complete. 2023-10-06T02:23:59.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.783+0000 7f6da9f9d700 1 Processor -- start 2023-10-06T02:23:59.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.784+0000 7f6da9f9d700 1 -- start start 2023-10-06T02:23:59.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.784+0000 7f6da9f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6da406aaa0 0x7f6da41561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:59.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.784+0000 7f6da9f9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6da413fb20 0x7f6da4140510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:59.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.784+0000 7f6da9f9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6da413fef0 0x7f6da4156920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:59.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.784+0000 7f6da9f9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da4156780 con 0x7f6da413fef0 2023-10-06T02:23:59.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.784+0000 7f6da9f9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da40b04a0 con 0x7f6da413fb20 2023-10-06T02:23:59.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.784+0000 7f6da9f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6da40afb70 con 0x7f6da406aaa0 2023-10-06T02:23:59.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.785+0000 7f6da979c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6da413fef0 0x7f6da4156920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:59.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.785+0000 7f6da979c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6da413fef0 0x7f6da4156920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49710/0 (socket says 172.21.15.31:49710) 2023-10-06T02:23:59.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.785+0000 7f6da979c700 1 -- 172.21.15.31:0/2562724860 learned_addr learned my addr 172.21.15.31:0/2562724860 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:23:59.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.785+0000 7f6da8f9b700 1 --2- 172.21.15.31:0/2562724860 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6da406aaa0 0x7f6da41561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:59.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.785+0000 7f6da3fff700 1 --2- 172.21.15.31:0/2562724860 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6da413fb20 0x7f6da4140510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:59.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.785+0000 7f6da979c700 1 -- 172.21.15.31:0/2562724860 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6da406aaa0 msgr2=0x7f6da41561a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:59.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.785+0000 7f6da979c700 1 --2- 172.21.15.31:0/2562724860 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6da406aaa0 0x7f6da41561a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:59.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.785+0000 7f6da979c700 1 -- 172.21.15.31:0/2562724860 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6da413fb20 msgr2=0x7f6da4140510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:23:59.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.785+0000 7f6da979c700 1 --2- 172.21.15.31:0/2562724860 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6da413fb20 0x7f6da4140510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:23:59.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.785+0000 7f6da979c700 1 -- 172.21.15.31:0/2562724860 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d98009420 con 0x7f6da413fef0 2023-10-06T02:23:59.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.785+0000 7f6da979c700 1 --2- 172.21.15.31:0/2562724860 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6da413fef0 0x7f6da4156920 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c00c8e0 tx=0x7f6d9c002760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:59.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.786+0000 7f6da0ff9700 1 -- 172.21.15.31:0/2562724860 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d9c012630 con 0x7f6da413fef0 2023-10-06T02:23:59.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.786+0000 7f6da0ff9700 1 -- 172.21.15.31:0/2562724860 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d9c0127b0 con 0x7f6da413fef0 2023-10-06T02:23:59.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.786+0000 7f6da0ff9700 1 -- 172.21.15.31:0/2562724860 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d9c012b70 con 0x7f6da413fef0 2023-10-06T02:23:59.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.786+0000 7f6da9f9d700 1 -- 172.21.15.31:0/2562724860 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6da40afe50 con 0x7f6da413fef0 2023-10-06T02:23:59.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.786+0000 7f6da9f9d700 1 -- 172.21.15.31:0/2562724860 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6da4155de0 con 0x7f6da413fef0 2023-10-06T02:23:59.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.787+0000 7f6da9f9d700 1 -- 172.21.15.31:0/2562724860 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6da404c3c0 con 0x7f6da413fef0 2023-10-06T02:23:59.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.788+0000 7f6da0ff9700 1 -- 172.21.15.31:0/2562724860 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6d9c01b400 con 0x7f6da413fef0 2023-10-06T02:23:59.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.788+0000 7f6da0ff9700 1 --2- 172.21.15.31:0/2562724860 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6d88066f30 0x7f6d880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:23:59.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.789+0000 7f6da0ff9700 1 -- 172.21.15.31:0/2562724860 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f6d9c084ab0 con 0x7f6da413fef0 2023-10-06T02:23:59.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.789+0000 7f6da8f9b700 1 --2- 172.21.15.31:0/2562724860 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6d88066f30 0x7f6d880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:23:59.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.789+0000 7f6da8f9b700 1 --2- 172.21.15.31:0/2562724860 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6d88066f30 0x7f6d880693e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6d98009860 tx=0x7f6d98017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:23:59.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:23:59.792+0000 7f6da0ff9700 1 -- 172.21.15.31:0/2562724860 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6d9c051ae0 con 0x7f6da413fef0 2023-10-06T02:24:00.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:00.043+0000 7f6da9f9d700 1 -- 172.21.15.31:0/2562724860 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f6da4136d50 con 0x7f6da413fef0 2023-10-06T02:24:00.045 INFO:tasks.ceph.mon.a.smithi031.stderr:2023-10-06T02:24:00.044+0000 7f6de01f1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T02:24:00.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:00.049+0000 7f6da0ff9700 1 -- 172.21.15.31:0/2562724860 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7f6d9c055100 con 0x7f6da413fef0 2023-10-06T02:24:00.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:00.052+0000 7f6da9f9d700 1 -- 172.21.15.31:0/2562724860 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6d88066f30 msgr2=0x7f6d880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:00.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:00.052+0000 7f6da9f9d700 1 --2- 172.21.15.31:0/2562724860 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6d88066f30 0x7f6d880693e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6d98009860 tx=0x7f6d98017000 comp rx=0 tx=0).stop 2023-10-06T02:24:00.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:00.052+0000 7f6da9f9d700 1 -- 172.21.15.31:0/2562724860 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6da413fef0 msgr2=0x7f6da4156920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:00.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:00.052+0000 7f6da9f9d700 1 --2- 172.21.15.31:0/2562724860 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6da413fef0 0x7f6da4156920 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c00c8e0 tx=0x7f6d9c002760 comp rx=0 tx=0).stop 2023-10-06T02:24:00.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:00.052+0000 7f6da9f9d700 1 -- 172.21.15.31:0/2562724860 shutdown_connections 2023-10-06T02:24:00.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:00.052+0000 7f6da9f9d700 1 --2- 172.21.15.31:0/2562724860 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6da413fef0 0x7f6da4156920 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:00.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:00.052+0000 7f6da9f9d700 1 --2- 172.21.15.31:0/2562724860 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6d88066f30 0x7f6d880693e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:00.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:00.052+0000 7f6da9f9d700 1 --2- 172.21.15.31:0/2562724860 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6da406aaa0 0x7f6da41561a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:00.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:00.052+0000 7f6da9f9d700 1 --2- 172.21.15.31:0/2562724860 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6da413fb20 0x7f6da4140510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:00.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:00.052+0000 7f6da9f9d700 1 -- 172.21.15.31:0/2562724860 >> 172.21.15.31:0/2562724860 conn(0x7f6da406d170 msgr2=0x7f6da41361a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:00.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:00.052+0000 7f6da9f9d700 1 -- 172.21.15.31:0/2562724860 shutdown_connections 2023-10-06T02:24:00.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:00.052+0000 7f6da9f9d700 1 -- 172.21.15.31:0/2562724860 wait complete. 2023-10-06T02:24:00.055 INFO:teuthology.orchestra.run.smithi031.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T02:24:00.059 INFO:tasks.ceph.mds.a.smithi031.stderr: -2291> 2023-10-06T02:23:12.166+0000 7f8f2fff1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T02:24:00.078 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T02:24:00.106 INFO:tasks.ceph.mds.a.smithi031.stdout:starting mds.a at 2023-10-06T02:24:00.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:00.268+0000 7f520759e700 1 Processor -- start 2023-10-06T02:24:00.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:00.269+0000 7f520759e700 1 -- start start 2023-10-06T02:24:00.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:00.270+0000 7f520759e700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f5208153c90 0x7f520815a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:00.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:00.270+0000 7f520759e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f52080caec0 0x7f520815afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:00.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:00.270+0000 7f520759e700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f520815ada0 con 0x7f520806db20 2023-10-06T02:24:00.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:00.270+0000 7f520759e700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f520806ca80 con 0x7f52080caec0 2023-10-06T02:24:00.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:00.270+0000 7f520759e700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f520806cd60 con 0x7f5208153c90 2023-10-06T02:24:00.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:00.270+0000 7f5206d9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f52080caec0 0x7f520815afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:00.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:00.270+0000 7f5206d9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f52080caec0 0x7f520815afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49730/0 (socket says 172.21.15.31:49730) 2023-10-06T02:24:00.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:00.270+0000 7f5206d9d700 1 -- 172.21.15.31:0/4102436874 learned_addr learned my addr 172.21.15.31:0/4102436874 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:00.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:00.270+0000 7f520659c700 1 --2- 172.21.15.31:0/4102436874 >> v2:172.21.15.62:3300/0 conn(0x7f5208153c90 0x7f520815a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:00.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:00.271+0000 7f5206d9d700 1 -- 172.21.15.31:0/4102436874 >> v2:172.21.15.62:3300/0 conn(0x7f5208153c90 msgr2=0x7f520815a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:00.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:00.271+0000 7f5206d9d700 1 --2- 172.21.15.31:0/4102436874 >> v2:172.21.15.62:3300/0 conn(0x7f5208153c90 0x7f520815a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:00.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:00.271+0000 7f5206d9d700 1 -- 172.21.15.31:0/4102436874 >> v1:172.21.15.62:6789/0 conn(0x7f520806db20 legacy=0x7f52080ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:00.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:00.271+0000 7f5206d9d700 1 -- 172.21.15.31:0/4102436874 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f520806d040 con 0x7f52080caec0 2023-10-06T02:24:00.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:00.271+0000 7f5206d9d700 1 --2- 172.21.15.31:0/4102436874 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52080caec0 0x7f520815afb0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f52080c37f0 tx=0x7f5200009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a236ec537ed1be38 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:00.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:00.271+0000 7f520559a700 1 -- 172.21.15.31:0/4102436874 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f520000e070 con 0x7f52080caec0 2023-10-06T02:24:00.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:00.271+0000 7f520559a700 1 -- 172.21.15.31:0/4102436874 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f520000ba50 con 0x7f52080caec0 2023-10-06T02:24:00.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:00.271+0000 7f520559a700 1 -- 172.21.15.31:0/4102436874 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52000092e0 con 0x7f52080caec0 2023-10-06T02:24:00.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:00.272+0000 7f520759e700 1 -- 172.21.15.31:0/4102436874 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52080caec0 msgr2=0x7f520815afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:00.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:00.272+0000 7f520759e700 1 --2- 172.21.15.31:0/4102436874 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52080caec0 0x7f520815afb0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f52080c37f0 tx=0x7f5200009840 comp rx=0 tx=0).stop 2023-10-06T02:24:00.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:00.272+0000 7f520759e700 1 -- 172.21.15.31:0/4102436874 shutdown_connections 2023-10-06T02:24:00.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:00.272+0000 7f520759e700 1 --2- 172.21.15.31:0/4102436874 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52080caec0 0x7f520815afb0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:00.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:00.272+0000 7f520759e700 1 --2- 172.21.15.31:0/4102436874 >> v2:172.21.15.62:3300/0 conn(0x7f5208153c90 0x7f520815a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:00.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:00.272+0000 7f520759e700 1 -- 172.21.15.31:0/4102436874 >> 172.21.15.31:0/4102436874 conn(0x7f520806e720 msgr2=0x7f520806eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:00.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:00.272+0000 7f520759e700 1 -- 172.21.15.31:0/4102436874 shutdown_connections 2023-10-06T02:24:00.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:00.272+0000 7f520759e700 1 -- 172.21.15.31:0/4102436874 wait complete. 2023-10-06T02:24:00.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:00.273+0000 7f520759e700 1 Processor -- start 2023-10-06T02:24:00.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:00.274+0000 7f520759e700 1 -- start start 2023-10-06T02:24:00.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:00.274+0000 7f520759e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f520806db20 0x7f520814e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:00.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:00.274+0000 7f520759e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52080caec0 0x7f52080c8de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:00.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:00.274+0000 7f520759e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5208153c90 0x7f520814a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:00.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:00.274+0000 7f520759e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52080c9320 con 0x7f52080caec0 2023-10-06T02:24:00.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:00.274+0000 7f520759e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52080b0db0 con 0x7f520806db20 2023-10-06T02:24:00.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:00.275+0000 7f520759e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f520814ecd0 con 0x7f5208153c90 2023-10-06T02:24:00.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:00.275+0000 7f5205d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52080caec0 0x7f52080c8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:00.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:00.275+0000 7f5205d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52080caec0 0x7f52080c8de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49734/0 (socket says 172.21.15.31:49734) 2023-10-06T02:24:00.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:00.275+0000 7f5205d9b700 1 -- 172.21.15.31:0/1343441071 learned_addr learned my addr 172.21.15.31:0/1343441071 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:00.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:00.275+0000 7f520659c700 1 --2- 172.21.15.31:0/1343441071 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f520806db20 0x7f520814e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:00.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:00.275+0000 7f5206d9d700 1 --2- 172.21.15.31:0/1343441071 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5208153c90 0x7f520814a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:00.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:00.275+0000 7f5205d9b700 1 -- 172.21.15.31:0/1343441071 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5208153c90 msgr2=0x7f520814a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:00.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:00.275+0000 7f5205d9b700 1 --2- 172.21.15.31:0/1343441071 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5208153c90 0x7f520814a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:00.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:00.275+0000 7f5205d9b700 1 -- 172.21.15.31:0/1343441071 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f520806db20 msgr2=0x7f520814e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:00.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:00.275+0000 7f5205d9b700 1 --2- 172.21.15.31:0/1343441071 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f520806db20 0x7f520814e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:00.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:00.275+0000 7f5205d9b700 1 -- 172.21.15.31:0/1343441071 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5200009ae0 con 0x7f52080caec0 2023-10-06T02:24:00.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:00.275+0000 7f5206d9d700 1 --2- 172.21.15.31:0/1343441071 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5208153c90 0x7f520814a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:24:00.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:00.275+0000 7f5205d9b700 1 --2- 172.21.15.31:0/1343441071 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52080caec0 0x7f52080c8de0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f51fc00c560 tx=0x7f51fc007f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:00.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:00.276+0000 7f51f67fc700 1 -- 172.21.15.31:0/1343441071 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51fc011590 con 0x7f52080caec0 2023-10-06T02:24:00.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:00.276+0000 7f51f67fc700 1 -- 172.21.15.31:0/1343441071 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51fc011710 con 0x7f52080caec0 2023-10-06T02:24:00.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:00.276+0000 7f51f67fc700 1 -- 172.21.15.31:0/1343441071 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51fc011ad0 con 0x7f52080caec0 2023-10-06T02:24:00.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:00.276+0000 7f520759e700 1 -- 172.21.15.31:0/1343441071 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f520814efb0 con 0x7f52080caec0 2023-10-06T02:24:00.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:00.276+0000 7f520759e700 1 -- 172.21.15.31:0/1343441071 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f520814f570 con 0x7f52080caec0 2023-10-06T02:24:00.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:00.277+0000 7f520759e700 1 -- 172.21.15.31:0/1343441071 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f520804c3c0 con 0x7f52080caec0 2023-10-06T02:24:00.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:00.278+0000 7f51f67fc700 1 -- 172.21.15.31:0/1343441071 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f51fc01a360 con 0x7f52080caec0 2023-10-06T02:24:00.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:00.279+0000 7f51f67fc700 1 --2- 172.21.15.31:0/1343441071 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f51e8066f30 0x7f51e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:00.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:00.279+0000 7f51f67fc700 1 -- 172.21.15.31:0/1343441071 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6202+0+0 (secure 0 0 0) 0x7f51fc083540 con 0x7f52080caec0 2023-10-06T02:24:00.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:00.279+0000 7f520659c700 1 --2- 172.21.15.31:0/1343441071 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f51e8066f30 0x7f51e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:00.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:00.280+0000 7f520659c700 1 --2- 172.21.15.31:0/1343441071 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f51e8066f30 0x7f51e80693e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f51f8006fd0 tx=0x7f51f8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:00.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:00.283+0000 7f51f67fc700 1 -- 172.21.15.31:0/1343441071 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f51fc050520 con 0x7f52080caec0 2023-10-06T02:24:00.338 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T02:24:00.345 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T02:24:00.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:00.539+0000 7f520759e700 1 -- 172.21.15.31:0/1343441071 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f520815ada0 con 0x7f52080caec0 2023-10-06T02:24:01.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:01.050+0000 7f51f67fc700 1 -- 172.21.15.31:0/1343441071 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7f520815ada0 con 0x7f52080caec0 2023-10-06T02:24:01.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:01.053+0000 7f520759e700 1 -- 172.21.15.31:0/1343441071 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f51e8066f30 msgr2=0x7f51e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:01.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:01.053+0000 7f520759e700 1 --2- 172.21.15.31:0/1343441071 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f51e8066f30 0x7f51e80693e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f51f8006fd0 tx=0x7f51f8008040 comp rx=0 tx=0).stop 2023-10-06T02:24:01.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:01.053+0000 7f520759e700 1 -- 172.21.15.31:0/1343441071 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52080caec0 msgr2=0x7f52080c8de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:01.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:01.053+0000 7f520759e700 1 --2- 172.21.15.31:0/1343441071 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52080caec0 0x7f52080c8de0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f51fc00c560 tx=0x7f51fc007f20 comp rx=0 tx=0).stop 2023-10-06T02:24:01.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:01.053+0000 7f520759e700 1 -- 172.21.15.31:0/1343441071 shutdown_connections 2023-10-06T02:24:01.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:01.053+0000 7f520759e700 1 --2- 172.21.15.31:0/1343441071 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f51e8066f30 0x7f51e80693e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:01.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:01.053+0000 7f520759e700 1 --2- 172.21.15.31:0/1343441071 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5208153c90 0x7f520814a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:01.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:01.053+0000 7f520759e700 1 --2- 172.21.15.31:0/1343441071 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f520806db20 0x7f520814e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:01.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:01.053+0000 7f520759e700 1 --2- 172.21.15.31:0/1343441071 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52080caec0 0x7f52080c8de0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:01.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:01.053+0000 7f520759e700 1 -- 172.21.15.31:0/1343441071 >> 172.21.15.31:0/1343441071 conn(0x7f520806e720 msgr2=0x7f5208152150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:01.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:01.053+0000 7f520759e700 1 -- 172.21.15.31:0/1343441071 shutdown_connections 2023-10-06T02:24:01.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:01.053+0000 7f520759e700 1 -- 172.21.15.31:0/1343441071 wait complete. 2023-10-06T02:24:01.073 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T02:24:01.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:01.266+0000 7f5a47fff700 1 Processor -- start 2023-10-06T02:24:01.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:01.266+0000 7f5a47fff700 1 -- start start 2023-10-06T02:24:01.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:01.267+0000 7f5a47fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5a4815aa00 0x7f5a480cb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:01.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:01.267+0000 7f5a47fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5a480cba10 0x7f5a480ce7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:01.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:01.267+0000 7f5a47fff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a480cbe80 con 0x7f5a480ced00 2023-10-06T02:24:01.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:01.267+0000 7f5a47fff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a480cf510 con 0x7f5a480cba10 2023-10-06T02:24:01.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:01.267+0000 7f5a47fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a480cf7f0 con 0x7f5a4815aa00 2023-10-06T02:24:01.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:01.267+0000 7f5a467fc700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5a480cba10 0x7f5a480ce7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:01.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:01.267+0000 7f5a467fc700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5a480cba10 0x7f5a480ce7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49742/0 (socket says 172.21.15.31:49742) 2023-10-06T02:24:01.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:01.267+0000 7f5a467fc700 1 -- 172.21.15.31:0/1130285810 learned_addr learned my addr 172.21.15.31:0/1130285810 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:01.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:01.268+0000 7f5a46ffd700 1 --2- 172.21.15.31:0/1130285810 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5a4815aa00 0x7f5a480cb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:01.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:01.268+0000 7f5a467fc700 1 -- 172.21.15.31:0/1130285810 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5a4815aa00 msgr2=0x7f5a480cb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:01.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:01.268+0000 7f5a467fc700 1 --2- 172.21.15.31:0/1130285810 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5a4815aa00 0x7f5a480cb4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:01.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:01.268+0000 7f5a467fc700 1 -- 172.21.15.31:0/1130285810 >> v1:172.21.15.62:6789/0 conn(0x7f5a480ced00 legacy=0x7f5a480ccef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:01.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:01.268+0000 7f5a467fc700 1 -- 172.21.15.31:0/1130285810 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a480cfad0 con 0x7f5a480cba10 2023-10-06T02:24:01.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:01.268+0000 7f5a46ffd700 1 --2- 172.21.15.31:0/1130285810 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5a4815aa00 0x7f5a480cb4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:24:01.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:01.268+0000 7f5a467fc700 1 --2- 172.21.15.31:0/1130285810 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a480cba10 0x7f5a480ce7c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c0099b0 tx=0x7f5a3c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2c941d44adfdb92 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:01.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:01.268+0000 7f5a45ffb700 1 -- 172.21.15.31:0/1130285810 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c00b860 con 0x7f5a480cba10 2023-10-06T02:24:01.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:01.268+0000 7f5a45ffb700 1 -- 172.21.15.31:0/1130285810 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a3c00b9c0 con 0x7f5a480cba10 2023-10-06T02:24:01.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:01.268+0000 7f5a45ffb700 1 -- 172.21.15.31:0/1130285810 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c00bd60 con 0x7f5a480cba10 2023-10-06T02:24:01.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:01.269+0000 7f5a47fff700 1 -- 172.21.15.31:0/1130285810 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a480cba10 msgr2=0x7f5a480ce7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:01.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:01.269+0000 7f5a47fff700 1 --2- 172.21.15.31:0/1130285810 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a480cba10 0x7f5a480ce7c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c0099b0 tx=0x7f5a3c009250 comp rx=0 tx=0).stop 2023-10-06T02:24:01.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:01.269+0000 7f5a47fff700 1 -- 172.21.15.31:0/1130285810 shutdown_connections 2023-10-06T02:24:01.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:01.269+0000 7f5a47fff700 1 --2- 172.21.15.31:0/1130285810 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5a4815aa00 0x7f5a480cb4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:01.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:01.269+0000 7f5a47fff700 1 --2- 172.21.15.31:0/1130285810 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a480cba10 0x7f5a480ce7c0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:01.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:01.269+0000 7f5a47fff700 1 -- 172.21.15.31:0/1130285810 >> 172.21.15.31:0/1130285810 conn(0x7f5a48099740 msgr2=0x7f5a48070b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:01.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:01.269+0000 7f5a47fff700 1 -- 172.21.15.31:0/1130285810 shutdown_connections 2023-10-06T02:24:01.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:01.269+0000 7f5a47fff700 1 -- 172.21.15.31:0/1130285810 wait complete. 2023-10-06T02:24:01.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:01.271+0000 7f5a47fff700 1 Processor -- start 2023-10-06T02:24:01.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:01.271+0000 7f5a47fff700 1 -- start start 2023-10-06T02:24:01.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:01.272+0000 7f5a47fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a480cba10 0x7f5a480afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:01.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:01.272+0000 7f5a47fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a480ced00 0x7f5a48157100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:01.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:01.272+0000 7f5a47fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5a4815aa00 0x7f5a4815f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:01.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:01.272+0000 7f5a47fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a4815be60 con 0x7f5a480cba10 2023-10-06T02:24:01.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:01.272+0000 7f5a47fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a48157640 con 0x7f5a480ced00 2023-10-06T02:24:01.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:01.272+0000 7f5a47fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a4815b250 con 0x7f5a4815aa00 2023-10-06T02:24:01.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:01.272+0000 7f5a46ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a480cba10 0x7f5a480afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:01.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:01.272+0000 7f5a46ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a480cba10 0x7f5a480afdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49756/0 (socket says 172.21.15.31:49756) 2023-10-06T02:24:01.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:01.272+0000 7f5a46ffd700 1 -- 172.21.15.31:0/271256114 learned_addr learned my addr 172.21.15.31:0/271256114 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:01.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:01.272+0000 7f5a477fe700 1 --2- 172.21.15.31:0/271256114 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5a4815aa00 0x7f5a4815f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:01.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:01.272+0000 7f5a467fc700 1 --2- 172.21.15.31:0/271256114 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a480ced00 0x7f5a48157100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:01.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:01.272+0000 7f5a46ffd700 1 -- 172.21.15.31:0/271256114 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5a4815aa00 msgr2=0x7f5a4815f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:01.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:01.272+0000 7f5a46ffd700 1 --2- 172.21.15.31:0/271256114 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5a4815aa00 0x7f5a4815f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:01.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:01.272+0000 7f5a46ffd700 1 -- 172.21.15.31:0/271256114 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a480ced00 msgr2=0x7f5a48157100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:01.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:01.272+0000 7f5a46ffd700 1 --2- 172.21.15.31:0/271256114 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a480ced00 0x7f5a48157100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:01.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:01.272+0000 7f5a46ffd700 1 -- 172.21.15.31:0/271256114 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a3c0094f0 con 0x7f5a480cba10 2023-10-06T02:24:01.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:01.273+0000 7f5a46ffd700 1 --2- 172.21.15.31:0/271256114 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a480cba10 0x7f5a480afdb0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5a3800d8c0 tx=0x7f5a38008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:01.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:01.273+0000 7f5a2effd700 1 -- 172.21.15.31:0/271256114 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a38005a00 con 0x7f5a480cba10 2023-10-06T02:24:01.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:01.273+0000 7f5a2effd700 1 -- 172.21.15.31:0/271256114 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a38005b60 con 0x7f5a480cba10 2023-10-06T02:24:01.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:01.273+0000 7f5a2effd700 1 -- 172.21.15.31:0/271256114 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a3800b5d0 con 0x7f5a480cba10 2023-10-06T02:24:01.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:01.273+0000 7f5a47fff700 1 -- 172.21.15.31:0/271256114 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a4815b530 con 0x7f5a480cba10 2023-10-06T02:24:01.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:01.273+0000 7f5a47fff700 1 -- 172.21.15.31:0/271256114 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a4815baf0 con 0x7f5a480cba10 2023-10-06T02:24:01.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:01.274+0000 7f5a47fff700 1 -- 172.21.15.31:0/271256114 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a4815c140 con 0x7f5a480cba10 2023-10-06T02:24:01.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:01.275+0000 7f5a2effd700 1 -- 172.21.15.31:0/271256114 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5a3800b980 con 0x7f5a480cba10 2023-10-06T02:24:01.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:01.276+0000 7f5a2effd700 1 --2- 172.21.15.31:0/271256114 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5a34066f30 0x7f5a340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:01.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:01.276+0000 7f5a2effd700 1 -- 172.21.15.31:0/271256114 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6202+0+0 (secure 0 0 0) 0x7f5a38084a30 con 0x7f5a480cba10 2023-10-06T02:24:01.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:01.276+0000 7f5a467fc700 1 --2- 172.21.15.31:0/271256114 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5a34066f30 0x7f5a340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:01.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:01.277+0000 7f5a467fc700 1 --2- 172.21.15.31:0/271256114 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5a34066f30 0x7f5a340693e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c0099b0 tx=0x7f5a3c017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:01.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:01.280+0000 7f5a2effd700 1 -- 172.21.15.31:0/271256114 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5a38051a10 con 0x7f5a480cba10 2023-10-06T02:24:01.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:01.481+0000 7f5a47fff700 1 -- 172.21.15.31:0/271256114 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5a48159ff0 con 0x7f5a480cba10 2023-10-06T02:24:02.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:02.253+0000 7f5a2effd700 1 -- 172.21.15.31:0/271256114 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v27) v1 ==== 181+0+0 (secure 0 0 0) 0x7f5a38055030 con 0x7f5a480cba10 2023-10-06T02:24:02.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:02.257+0000 7f5a47fff700 1 -- 172.21.15.31:0/271256114 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5a34066f30 msgr2=0x7f5a340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:02.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:02.257+0000 7f5a47fff700 1 --2- 172.21.15.31:0/271256114 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5a34066f30 0x7f5a340693e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c0099b0 tx=0x7f5a3c017000 comp rx=0 tx=0).stop 2023-10-06T02:24:02.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:02.257+0000 7f5a47fff700 1 -- 172.21.15.31:0/271256114 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a480cba10 msgr2=0x7f5a480afdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:02.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:02.257+0000 7f5a47fff700 1 --2- 172.21.15.31:0/271256114 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a480cba10 0x7f5a480afdb0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5a3800d8c0 tx=0x7f5a38008040 comp rx=0 tx=0).stop 2023-10-06T02:24:02.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:02.257+0000 7f5a47fff700 1 -- 172.21.15.31:0/271256114 shutdown_connections 2023-10-06T02:24:02.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:02.257+0000 7f5a47fff700 1 --2- 172.21.15.31:0/271256114 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5a34066f30 0x7f5a340693e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:02.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:02.257+0000 7f5a47fff700 1 --2- 172.21.15.31:0/271256114 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5a4815aa00 0x7f5a4815f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:02.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:02.257+0000 7f5a47fff700 1 --2- 172.21.15.31:0/271256114 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a480cba10 0x7f5a480afdb0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:02.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:02.257+0000 7f5a47fff700 1 --2- 172.21.15.31:0/271256114 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a480ced00 0x7f5a48157100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:02.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:02.257+0000 7f5a47fff700 1 -- 172.21.15.31:0/271256114 >> 172.21.15.31:0/271256114 conn(0x7f5a48099740 msgr2=0x7f5a4806e950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:02.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:02.257+0000 7f5a47fff700 1 -- 172.21.15.31:0/271256114 shutdown_connections 2023-10-06T02:24:02.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:02.257+0000 7f5a47fff700 1 -- 172.21.15.31:0/271256114 wait complete. 2023-10-06T02:24:02.260 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' removed 2023-10-06T02:24:02.280 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T02:24:02.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:02.473+0000 7f1812f9d700 1 Processor -- start 2023-10-06T02:24:02.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:02.474+0000 7f1812f9d700 1 -- start start 2023-10-06T02:24:02.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:02.475+0000 7f1812f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f180c153660 0x7f180c0c0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:02.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:02.475+0000 7f1812f9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f180c153a30 0x7f180c156d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:02.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:02.475+0000 7f1812f9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f180c0c8980 0x7f180c0c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:02.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:02.475+0000 7f1812f9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f180c157250 con 0x7f180c153a30 2023-10-06T02:24:02.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:02.475+0000 7f1812f9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f180c0c2200 con 0x7f180c0c8980 2023-10-06T02:24:02.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:02.475+0000 7f1812f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f180c0c15f0 con 0x7f180c153660 2023-10-06T02:24:02.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:02.475+0000 7f181179a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f180c153a30 0x7f180c156d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:02.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:02.475+0000 7f181179a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f180c153a30 0x7f180c156d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49772/0 (socket says 172.21.15.31:49772) 2023-10-06T02:24:02.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:02.475+0000 7f181179a700 1 -- 172.21.15.31:0/1242540665 learned_addr learned my addr 172.21.15.31:0/1242540665 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:02.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:02.476+0000 7f181279c700 1 --2- 172.21.15.31:0/1242540665 >> v2:172.21.15.62:3300/0 conn(0x7f180c0c8980 0x7f180c0c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:02.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:02.476+0000 7f1811f9b700 1 --2- 172.21.15.31:0/1242540665 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f180c153660 0x7f180c0c0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:02.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:02.476+0000 7f181179a700 1 -- 172.21.15.31:0/1242540665 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f180c153660 msgr2=0x7f180c0c0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:02.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:02.476+0000 7f181179a700 1 --2- 172.21.15.31:0/1242540665 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f180c153660 0x7f180c0c0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:02.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:02.476+0000 7f181179a700 1 -- 172.21.15.31:0/1242540665 >> v2:172.21.15.62:3300/0 conn(0x7f180c0c8980 msgr2=0x7f180c0c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:02.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:02.476+0000 7f181179a700 1 --2- 172.21.15.31:0/1242540665 >> v2:172.21.15.62:3300/0 conn(0x7f180c0c8980 0x7f180c0c8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:02.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:02.476+0000 7f181179a700 1 -- 172.21.15.31:0/1242540665 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f180c0c18d0 con 0x7f180c153a30 2023-10-06T02:24:02.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:02.476+0000 7f181179a700 1 --2- 172.21.15.31:0/1242540665 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f180c153a30 0x7f180c156d10 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f180c0c3180 tx=0x7f17fc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=607b76cd117ce978 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:02.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:02.476+0000 7f1810f99700 1 -- 172.21.15.31:0/1242540665 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17fc00b640 con 0x7f180c153a30 2023-10-06T02:24:02.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:02.477+0000 7f1810f99700 1 -- 172.21.15.31:0/1242540665 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17fc00b7a0 con 0x7f180c153a30 2023-10-06T02:24:02.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:02.477+0000 7f1810f99700 1 -- 172.21.15.31:0/1242540665 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17fc00bb60 con 0x7f180c153a30 2023-10-06T02:24:02.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:02.477+0000 7f1812f9d700 1 -- 172.21.15.31:0/1242540665 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f180c153a30 msgr2=0x7f180c156d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:02.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:02.477+0000 7f1812f9d700 1 --2- 172.21.15.31:0/1242540665 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f180c153a30 0x7f180c156d10 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f180c0c3180 tx=0x7f17fc009250 comp rx=0 tx=0).stop 2023-10-06T02:24:02.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:02.477+0000 7f1812f9d700 1 -- 172.21.15.31:0/1242540665 shutdown_connections 2023-10-06T02:24:02.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:02.477+0000 7f1812f9d700 1 --2- 172.21.15.31:0/1242540665 >> v2:172.21.15.62:3300/0 conn(0x7f180c0c8980 0x7f180c0c8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:02.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:02.477+0000 7f1812f9d700 1 --2- 172.21.15.31:0/1242540665 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f180c153660 0x7f180c0c0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:02.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:02.477+0000 7f1812f9d700 1 --2- 172.21.15.31:0/1242540665 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f180c153a30 0x7f180c156d10 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:02.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:02.477+0000 7f1812f9d700 1 -- 172.21.15.31:0/1242540665 >> 172.21.15.31:0/1242540665 conn(0x7f180c06c6a0 msgr2=0x7f180c06caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:02.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:02.477+0000 7f1812f9d700 1 -- 172.21.15.31:0/1242540665 shutdown_connections 2023-10-06T02:24:02.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:02.477+0000 7f1812f9d700 1 -- 172.21.15.31:0/1242540665 wait complete. 2023-10-06T02:24:02.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:02.479+0000 7f1812f9d700 1 Processor -- start 2023-10-06T02:24:02.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:02.479+0000 7f1812f9d700 1 -- start start 2023-10-06T02:24:02.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:02.480+0000 7f1812f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f180c0c8980 0x7f180c16ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:02.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:02.480+0000 7f1812f9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f180c153660 0x7f180c151e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:02.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:02.480+0000 7f1812f9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f180c153a30 0x7f180c0b0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:02.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:02.480+0000 7f1812f9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f180c15a9b0 con 0x7f180c153660 2023-10-06T02:24:02.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:02.480+0000 7f1812f9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f180c15b530 con 0x7f180c153a30 2023-10-06T02:24:02.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:02.480+0000 7f1812f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f180c152390 con 0x7f180c0c8980 2023-10-06T02:24:02.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:02.480+0000 7f181179a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f180c153660 0x7f180c151e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:02.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:02.480+0000 7f181179a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f180c153660 0x7f180c151e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49778/0 (socket says 172.21.15.31:49778) 2023-10-06T02:24:02.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:02.480+0000 7f181179a700 1 -- 172.21.15.31:0/4212724319 learned_addr learned my addr 172.21.15.31:0/4212724319 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:02.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:02.480+0000 7f1811f9b700 1 --2- 172.21.15.31:0/4212724319 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f180c0c8980 0x7f180c16ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:02.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:02.480+0000 7f181279c700 1 --2- 172.21.15.31:0/4212724319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f180c153a30 0x7f180c0b0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:02.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:02.480+0000 7f181179a700 1 -- 172.21.15.31:0/4212724319 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f180c0c8980 msgr2=0x7f180c16ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:02.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:02.480+0000 7f181179a700 1 --2- 172.21.15.31:0/4212724319 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f180c0c8980 0x7f180c16ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:02.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:02.480+0000 7f181179a700 1 -- 172.21.15.31:0/4212724319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f180c153a30 msgr2=0x7f180c0b0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:02.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:02.480+0000 7f181179a700 1 --2- 172.21.15.31:0/4212724319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f180c153a30 0x7f180c0b0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:02.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:02.480+0000 7f181179a700 1 -- 172.21.15.31:0/4212724319 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17fc0094f0 con 0x7f180c153660 2023-10-06T02:24:02.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:02.480+0000 7f1811f9b700 1 --2- 172.21.15.31:0/4212724319 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f180c0c8980 0x7f180c16ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:24:02.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:02.481+0000 7f181179a700 1 --2- 172.21.15.31:0/4212724319 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f180c153660 0x7f180c151e50 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f17fc006fd0 tx=0x7f17fc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:02.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:02.481+0000 7f1801ffb700 1 -- 172.21.15.31:0/4212724319 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17fc00b910 con 0x7f180c153660 2023-10-06T02:24:02.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:02.481+0000 7f1801ffb700 1 -- 172.21.15.31:0/4212724319 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17fc00ba70 con 0x7f180c153660 2023-10-06T02:24:02.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:02.481+0000 7f1801ffb700 1 -- 172.21.15.31:0/4212724319 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17fc01a6a0 con 0x7f180c153660 2023-10-06T02:24:02.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:02.481+0000 7f1812f9d700 1 -- 172.21.15.31:0/4212724319 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f180c15ac00 con 0x7f180c153660 2023-10-06T02:24:02.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:02.481+0000 7f1812f9d700 1 -- 172.21.15.31:0/4212724319 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f180c15b1c0 con 0x7f180c153660 2023-10-06T02:24:02.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:02.482+0000 7f1812f9d700 1 -- 172.21.15.31:0/4212724319 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f180c04c3c0 con 0x7f180c153660 2023-10-06T02:24:02.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:02.483+0000 7f1801ffb700 1 -- 172.21.15.31:0/4212724319 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f17fc01aa90 con 0x7f180c153660 2023-10-06T02:24:02.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:02.484+0000 7f1801ffb700 1 --2- 172.21.15.31:0/4212724319 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f17f0066f30 0x7f17f00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:02.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:02.484+0000 7f1801ffb700 1 -- 172.21.15.31:0/4212724319 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 5764+0+0 (secure 0 0 0) 0x7f17fc085f00 con 0x7f180c153660 2023-10-06T02:24:02.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:02.484+0000 7f1811f9b700 1 --2- 172.21.15.31:0/4212724319 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f17f0066f30 0x7f17f00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:02.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:02.485+0000 7f1811f9b700 1 --2- 172.21.15.31:0/4212724319 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f17f0066f30 0x7f17f00693e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1808009d80 tx=0x7f1808008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:02.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:02.488+0000 7f1801ffb700 1 -- 172.21.15.31:0/4212724319 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f17fc053090 con 0x7f180c153660 2023-10-06T02:24:02.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:02.691+0000 7f1812f9d700 1 -- 172.21.15.31:0/4212724319 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f180c0ccd60 con 0x7f180c153660 2023-10-06T02:24:03.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:03.259+0000 7f1801ffb700 1 -- 172.21.15.31:0/4212724319 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v28) v1 ==== 169+0+0 (secure 0 0 0) 0x7f17fc0566b0 con 0x7f180c153660 2023-10-06T02:24:03.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:03.263+0000 7f1812f9d700 1 -- 172.21.15.31:0/4212724319 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f17f0066f30 msgr2=0x7f17f00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:03.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:03.263+0000 7f1812f9d700 1 --2- 172.21.15.31:0/4212724319 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f17f0066f30 0x7f17f00693e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1808009d80 tx=0x7f1808008040 comp rx=0 tx=0).stop 2023-10-06T02:24:03.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:03.263+0000 7f1812f9d700 1 -- 172.21.15.31:0/4212724319 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f180c153660 msgr2=0x7f180c151e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:03.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:03.263+0000 7f1812f9d700 1 --2- 172.21.15.31:0/4212724319 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f180c153660 0x7f180c151e50 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f17fc006fd0 tx=0x7f17fc017000 comp rx=0 tx=0).stop 2023-10-06T02:24:03.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:03.263+0000 7f1812f9d700 1 -- 172.21.15.31:0/4212724319 shutdown_connections 2023-10-06T02:24:03.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:03.263+0000 7f1812f9d700 1 --2- 172.21.15.31:0/4212724319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f180c153a30 0x7f180c0b0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:03.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:03.263+0000 7f1812f9d700 1 --2- 172.21.15.31:0/4212724319 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f17f0066f30 0x7f17f00693e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:03.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:03.263+0000 7f1812f9d700 1 --2- 172.21.15.31:0/4212724319 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f180c0c8980 0x7f180c16ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:03.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:03.263+0000 7f1812f9d700 1 --2- 172.21.15.31:0/4212724319 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f180c153660 0x7f180c151e50 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:03.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:03.263+0000 7f1812f9d700 1 -- 172.21.15.31:0/4212724319 >> 172.21.15.31:0/4212724319 conn(0x7f180c06c6a0 msgr2=0x7f180c0cb930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:03.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:03.263+0000 7f1812f9d700 1 -- 172.21.15.31:0/4212724319 shutdown_connections 2023-10-06T02:24:03.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:03.263+0000 7f1812f9d700 1 -- 172.21.15.31:0/4212724319 wait complete. 2023-10-06T02:24:03.266 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' removed 2023-10-06T02:24:03.289 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T02:24:03.289 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T02:24:03.289 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T02:24:03.289 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T02:24:03.289 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T02:24:03.289 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T02:24:03.290 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T02:24:03.290 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T02:24:03.290 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T02:24:03.290 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T02:24:03.290 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T02:24:03.290 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T02:24:03.290 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T02:24:03.291 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T02:24:03.291 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T02:24:03.291 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T02:24:03.308 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T02:24:03.308 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T02:24:03.309 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T02:24:03.309 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T02:24:03.310 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T02:24:03.310 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T02:24:03.311 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T02:24:03.311 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T02:24:03.352 INFO:tasks.ceph.mds.b:Started 2023-10-06T02:24:03.352 DEBUG:teuthology.parallel:result is None 2023-10-06T02:24:03.354 INFO:tasks.ceph.mds.a:Started 2023-10-06T02:24:03.354 DEBUG:teuthology.parallel:result is None 2023-10-06T02:24:03.355 INFO:tasks.ceph.mds.d:Started 2023-10-06T02:24:03.355 DEBUG:teuthology.parallel:result is None 2023-10-06T02:24:03.357 INFO:tasks.ceph.mds.c:Started 2023-10-06T02:24:03.357 DEBUG:teuthology.parallel:result is None 2023-10-06T02:24:03.357 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T02:24:03.492 INFO:tasks.ceph.mds.d.smithi062.stdout:starting mds.d at 2023-10-06T02:24:03.493 INFO:tasks.ceph.mds.a.smithi031.stdout:starting mds.a at 2023-10-06T02:24:03.493 INFO:tasks.ceph.mds.b.smithi062.stdout:starting mds.b at 2023-10-06T02:24:03.493 INFO:tasks.ceph.mds.c.smithi031.stdout:starting mds.c at 2023-10-06T02:24:03.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:03.551+0000 7fd39ffff700 1 Processor -- start 2023-10-06T02:24:03.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:03.552+0000 7fd39ffff700 1 -- start start 2023-10-06T02:24:03.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:03.553+0000 7fd39ffff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3a00cf110 0x7fd3a0164600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:03.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:03.553+0000 7fd39ffff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3a015afd0 con 0x7fd3a015de50 2023-10-06T02:24:03.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:03.553+0000 7fd39ffff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3a015b2b0 con 0x7fd3a0153c30 2023-10-06T02:24:03.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:03.553+0000 7fd39ffff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a015b590 con 0x7fd3a00cf110 2023-10-06T02:24:03.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:03.553+0000 7fd39e7fc700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fd3a015de50 0x7fd3a00cea00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:38654/0 (socket says 172.21.15.31:38654) 2023-10-06T02:24:03.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:03.553+0000 7fd39e7fc700 1 -- 172.21.15.31:0/1515918664 learned_addr learned my addr 172.21.15.31:0/1515918664 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:24:03.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:03.553+0000 7fd39f7fe700 1 --2- 172.21.15.31:0/1515918664 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3a00cf110 0x7fd3a0164600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:03.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:03.554+0000 7fd39dffb700 1 -- 172.21.15.31:0/1515918664 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2545404181 0 0) 0x7fd3a015b2b0 con 0x7fd3a0153c30 2023-10-06T02:24:03.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:03.554+0000 7fd39dffb700 1 -- 172.21.15.31:0/1515918664 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd388003570 con 0x7fd3a0153c30 2023-10-06T02:24:03.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:03.554+0000 7fd39dffb700 1 -- 172.21.15.31:0/1515918664 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3918941866 0 0) 0x7fd3a015afd0 con 0x7fd3a015de50 2023-10-06T02:24:03.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:03.554+0000 7fd39f7fe700 1 -- 172.21.15.31:0/1515918664 >> v1:172.21.15.62:6789/0 conn(0x7fd3a0153c30 legacy=0x7fd3a015a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:03.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:03.554+0000 7fd39f7fe700 1 -- 172.21.15.31:0/1515918664 >> v1:172.21.15.31:6789/0 conn(0x7fd3a015de50 legacy=0x7fd3a00cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:03.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:03.554+0000 7fd39f7fe700 1 -- 172.21.15.31:0/1515918664 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3a015b870 con 0x7fd3a00cf110 2023-10-06T02:24:03.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:03.554+0000 7fd39f7fe700 1 --2- 172.21.15.31:0/1515918664 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3a00cf110 0x7fd3a0164600 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd398009e90 tx=0x7fd398009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=863027816e268f1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:03.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:03.554+0000 7fd39dffb700 1 -- 172.21.15.31:0/1515918664 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd39800e070 con 0x7fd3a00cf110 2023-10-06T02:24:03.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:03.554+0000 7fd39dffb700 1 -- 172.21.15.31:0/1515918664 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd39800bc30 con 0x7fd3a00cf110 2023-10-06T02:24:03.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:03.554+0000 7fd39dffb700 1 -- 172.21.15.31:0/1515918664 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd398012430 con 0x7fd3a00cf110 2023-10-06T02:24:03.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:03.555+0000 7fd39ffff700 1 -- 172.21.15.31:0/1515918664 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3a00cf110 msgr2=0x7fd3a0164600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:03.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:03.555+0000 7fd39ffff700 1 --2- 172.21.15.31:0/1515918664 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3a00cf110 0x7fd3a0164600 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd398009e90 tx=0x7fd398009780 comp rx=0 tx=0).stop 2023-10-06T02:24:03.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:03.555+0000 7fd39ffff700 1 -- 172.21.15.31:0/1515918664 shutdown_connections 2023-10-06T02:24:03.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:03.555+0000 7fd39ffff700 1 --2- 172.21.15.31:0/1515918664 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3a00cf110 0x7fd3a0164600 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:03.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:03.555+0000 7fd39ffff700 1 -- 172.21.15.31:0/1515918664 >> 172.21.15.31:0/1515918664 conn(0x7fd3a00703a0 msgr2=0x7fd3a00707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:03.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:03.555+0000 7fd39ffff700 1 -- 172.21.15.31:0/1515918664 shutdown_connections 2023-10-06T02:24:03.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:03.555+0000 7fd39ffff700 1 -- 172.21.15.31:0/1515918664 wait complete. 2023-10-06T02:24:03.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:03.556+0000 7fd39ffff700 1 Processor -- start 2023-10-06T02:24:03.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:03.557+0000 7fd39ffff700 1 -- start start 2023-10-06T02:24:03.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:03.557+0000 7fd39ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3a00cf110 0x7fd3a014e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:03.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:03.557+0000 7fd39ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3a0153c30 0x7fd3a014fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:03.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:03.558+0000 7fd39ffff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3a015de50 0x7fd3a0150100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:03.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:03.558+0000 7fd39ffff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a014e930 con 0x7fd3a0153c30 2023-10-06T02:24:03.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:03.558+0000 7fd39ffff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a00cb480 con 0x7fd3a00cf110 2023-10-06T02:24:03.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:03.558+0000 7fd39ffff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a00c91e0 con 0x7fd3a015de50 2023-10-06T02:24:03.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:03.558+0000 7fd39e7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3a0153c30 0x7fd3a014fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:03.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:03.558+0000 7fd39e7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3a0153c30 0x7fd3a014fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49836/0 (socket says 172.21.15.31:49836) 2023-10-06T02:24:03.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:03.558+0000 7fd39e7fc700 1 -- 172.21.15.31:0/47188113 learned_addr learned my addr 172.21.15.31:0/47188113 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:03.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:03.558+0000 7fd39f7fe700 1 --2- 172.21.15.31:0/47188113 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3a015de50 0x7fd3a0150100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:03.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:03.558+0000 7fd39effd700 1 --2- 172.21.15.31:0/47188113 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3a00cf110 0x7fd3a014e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:03.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:03.558+0000 7fd39e7fc700 1 -- 172.21.15.31:0/47188113 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3a015de50 msgr2=0x7fd3a0150100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:03.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:03.558+0000 7fd39e7fc700 1 --2- 172.21.15.31:0/47188113 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3a015de50 0x7fd3a0150100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:03.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:03.558+0000 7fd39e7fc700 1 -- 172.21.15.31:0/47188113 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3a00cf110 msgr2=0x7fd3a014e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:03.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:03.558+0000 7fd39e7fc700 1 --2- 172.21.15.31:0/47188113 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3a00cf110 0x7fd3a014e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:03.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:03.558+0000 7fd39e7fc700 1 -- 172.21.15.31:0/47188113 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd398009a20 con 0x7fd3a0153c30 2023-10-06T02:24:03.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:03.559+0000 7fd39e7fc700 1 --2- 172.21.15.31:0/47188113 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3a0153c30 0x7fd3a014fbc0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fd39400c5b0 tx=0x7fd394001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:03.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:03.559+0000 7fd386ffd700 1 -- 172.21.15.31:0/47188113 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd394005b30 con 0x7fd3a0153c30 2023-10-06T02:24:03.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:03.559+0000 7fd386ffd700 1 -- 172.21.15.31:0/47188113 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd394005c90 con 0x7fd3a0153c30 2023-10-06T02:24:03.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:03.559+0000 7fd386ffd700 1 -- 172.21.15.31:0/47188113 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd394007300 con 0x7fd3a0153c30 2023-10-06T02:24:03.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:03.559+0000 7fd39ffff700 1 -- 172.21.15.31:0/47188113 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3a00c94c0 con 0x7fd3a0153c30 2023-10-06T02:24:03.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:03.559+0000 7fd39ffff700 1 -- 172.21.15.31:0/47188113 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3a00cb110 con 0x7fd3a0153c30 2023-10-06T02:24:03.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:03.560+0000 7fd39ffff700 1 -- 172.21.15.31:0/47188113 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3a00cb630 con 0x7fd3a0153c30 2023-10-06T02:24:03.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:03.561+0000 7fd386ffd700 1 -- 172.21.15.31:0/47188113 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd3940076b0 con 0x7fd3a0153c30 2023-10-06T02:24:03.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:03.562+0000 7fd386ffd700 1 --2- 172.21.15.31:0/47188113 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fd38c066f30 0x7fd38c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:03.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:03.562+0000 7fd386ffd700 1 -- 172.21.15.31:0/47188113 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5292+0+0 (secure 0 0 0) 0x7fd394083930 con 0x7fd3a0153c30 2023-10-06T02:24:03.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:03.562+0000 7fd39effd700 1 --2- 172.21.15.31:0/47188113 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fd38c066f30 0x7fd38c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:03.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:03.563+0000 7fd39effd700 1 --2- 172.21.15.31:0/47188113 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fd38c066f30 0x7fd38c0693e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd390003870 tx=0x7fd390009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:03.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:03.566+0000 7fd386ffd700 1 -- 172.21.15.31:0/47188113 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd3a00cb630 con 0x7fd3a0153c30 2023-10-06T02:24:03.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:03.783+0000 7fd39ffff700 1 -- 172.21.15.31:0/47188113 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fd3a00cb5f0 con 0x7fd3a0153c30 2023-10-06T02:24:04.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.268+0000 7fd386ffd700 1 -- 172.21.15.31:0/47188113 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v30) v1 ==== 97+0+0 (secure 0 0 0) 0x7fd394017080 con 0x7fd3a0153c30 2023-10-06T02:24:04.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.271+0000 7fd39ffff700 1 -- 172.21.15.31:0/47188113 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fd38c066f30 msgr2=0x7fd38c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:04.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.271+0000 7fd39ffff700 1 --2- 172.21.15.31:0/47188113 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fd38c066f30 0x7fd38c0693e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd390003870 tx=0x7fd390009040 comp rx=0 tx=0).stop 2023-10-06T02:24:04.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.271+0000 7fd39ffff700 1 -- 172.21.15.31:0/47188113 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3a0153c30 msgr2=0x7fd3a014fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:04.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.271+0000 7fd39ffff700 1 --2- 172.21.15.31:0/47188113 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3a0153c30 0x7fd3a014fbc0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fd39400c5b0 tx=0x7fd394001070 comp rx=0 tx=0).stop 2023-10-06T02:24:04.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.271+0000 7fd39ffff700 1 -- 172.21.15.31:0/47188113 shutdown_connections 2023-10-06T02:24:04.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.271+0000 7fd39ffff700 1 --2- 172.21.15.31:0/47188113 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fd38c066f30 0x7fd38c0693e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:04.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.271+0000 7fd39ffff700 1 --2- 172.21.15.31:0/47188113 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3a015de50 0x7fd3a0150100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:04.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.271+0000 7fd39ffff700 1 --2- 172.21.15.31:0/47188113 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3a00cf110 0x7fd3a014e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:04.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.271+0000 7fd39ffff700 1 --2- 172.21.15.31:0/47188113 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3a0153c30 0x7fd3a014fbc0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:04.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.271+0000 7fd39ffff700 1 -- 172.21.15.31:0/47188113 >> 172.21.15.31:0/47188113 conn(0x7fd3a00703a0 msgr2=0x7fd3a0150ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:04.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.271+0000 7fd39ffff700 1 -- 172.21.15.31:0/47188113 shutdown_connections 2023-10-06T02:24:04.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.271+0000 7fd39ffff700 1 -- 172.21.15.31:0/47188113 wait complete. 2023-10-06T02:24:04.274 INFO:teuthology.orchestra.run.smithi031.stderr: removed all blocklist entries 2023-10-06T02:24:04.293 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T02:24:04.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.479+0000 7f561ffff700 1 Processor -- start 2023-10-06T02:24:04.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.480+0000 7f561ffff700 1 -- start start 2023-10-06T02:24:04.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.481+0000 7f561ffff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f562013fef0 0x7f562015a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:04.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.481+0000 7f561ffff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f562006aaa0 0x7f56200c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:04.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.481+0000 7f561ffff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f562015ae70 con 0x7f562013fb20 2023-10-06T02:24:04.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.481+0000 7f561ffff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f562006af60 con 0x7f562013fef0 2023-10-06T02:24:04.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.481+0000 7f561ffff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f562006b240 con 0x7f562006aaa0 2023-10-06T02:24:04.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.482+0000 7f561f7fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f562006aaa0 0x7f56200c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:04.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.482+0000 7f561f7fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f562006aaa0 0x7f56200c9d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:55422/0 (socket says 172.21.15.31:55422) 2023-10-06T02:24:04.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.482+0000 7f561e7fc700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f562013fef0 0x7f562015a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:04.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.482+0000 7f561f7fe700 1 -- 172.21.15.31:0/3967638366 learned_addr learned my addr 172.21.15.31:0/3967638366 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:04.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.482+0000 7f561effd700 1 --1- 172.21.15.31:0/3967638366 >> v1:172.21.15.62:6789/0 conn(0x7f562013fb20 0x7f56200c7990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:54836/0 (socket says 172.21.15.31:54836) 2023-10-06T02:24:04.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.482+0000 7f561f7fe700 1 -- 172.21.15.31:0/3967638366 >> v2:172.21.15.62:3300/0 conn(0x7f562013fef0 msgr2=0x7f562015a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:04.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.482+0000 7f561f7fe700 1 --2- 172.21.15.31:0/3967638366 >> v2:172.21.15.62:3300/0 conn(0x7f562013fef0 0x7f562015a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:04.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.482+0000 7f561f7fe700 1 -- 172.21.15.31:0/3967638366 >> v1:172.21.15.62:6789/0 conn(0x7f562013fb20 legacy=0x7f56200c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:04.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.482+0000 7f561f7fe700 1 -- 172.21.15.31:0/3967638366 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f562006b520 con 0x7f562006aaa0 2023-10-06T02:24:04.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.482+0000 7f561e7fc700 1 --2- 172.21.15.31:0/3967638366 >> v2:172.21.15.62:3300/0 conn(0x7f562013fef0 0x7f562015a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:24:04.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.482+0000 7f561f7fe700 1 --2- 172.21.15.31:0/3967638366 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f562006aaa0 0x7f56200c9d00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f561800d1a0 tx=0x7f561800ba50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4aea39238325870 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:04.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.483+0000 7f561dffb700 1 -- 172.21.15.31:0/3967638366 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5618010070 con 0x7f562006aaa0 2023-10-06T02:24:04.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.483+0000 7f561dffb700 1 -- 172.21.15.31:0/3967638366 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f561800de10 con 0x7f562006aaa0 2023-10-06T02:24:04.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.483+0000 7f561dffb700 1 -- 172.21.15.31:0/3967638366 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f561800b320 con 0x7f562006aaa0 2023-10-06T02:24:04.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.483+0000 7f561ffff700 1 -- 172.21.15.31:0/3967638366 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f562006aaa0 msgr2=0x7f56200c9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:04.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.483+0000 7f561ffff700 1 --2- 172.21.15.31:0/3967638366 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f562006aaa0 0x7f56200c9d00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f561800d1a0 tx=0x7f561800ba50 comp rx=0 tx=0).stop 2023-10-06T02:24:04.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.483+0000 7f561ffff700 1 -- 172.21.15.31:0/3967638366 shutdown_connections 2023-10-06T02:24:04.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.483+0000 7f561ffff700 1 --2- 172.21.15.31:0/3967638366 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f562006aaa0 0x7f56200c9d00 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:04.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.483+0000 7f561ffff700 1 --2- 172.21.15.31:0/3967638366 >> v2:172.21.15.62:3300/0 conn(0x7f562013fef0 0x7f562015a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:04.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.483+0000 7f561ffff700 1 -- 172.21.15.31:0/3967638366 >> 172.21.15.31:0/3967638366 conn(0x7f562006d170 msgr2=0x7f56200cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:04.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.484+0000 7f561ffff700 1 -- 172.21.15.31:0/3967638366 shutdown_connections 2023-10-06T02:24:04.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.484+0000 7f561ffff700 1 -- 172.21.15.31:0/3967638366 wait complete. 2023-10-06T02:24:04.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.485+0000 7f561ffff700 1 Processor -- start 2023-10-06T02:24:04.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.485+0000 7f561ffff700 1 -- start start 2023-10-06T02:24:04.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.486+0000 7f561ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f562006aaa0 0x7f562013cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:04.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.486+0000 7f561ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f562013fb20 0x7f56200ad0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:04.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.486+0000 7f561ffff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f562013fef0 0x7f56200c75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:04.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.486+0000 7f561ffff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f562013d0a0 con 0x7f562006aaa0 2023-10-06T02:24:04.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.486+0000 7f561ffff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5620157320 con 0x7f562013fb20 2023-10-06T02:24:04.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.486+0000 7f561ffff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56201569f0 con 0x7f562013fef0 2023-10-06T02:24:04.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.486+0000 7f561effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f562006aaa0 0x7f562013cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:04.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.486+0000 7f561effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f562006aaa0 0x7f562013cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49848/0 (socket says 172.21.15.31:49848) 2023-10-06T02:24:04.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.486+0000 7f561effd700 1 -- 172.21.15.31:0/3351632147 learned_addr learned my addr 172.21.15.31:0/3351632147 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:04.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.487+0000 7f561f7fe700 1 --2- 172.21.15.31:0/3351632147 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f562013fef0 0x7f56200c75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:04.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.487+0000 7f561e7fc700 1 --2- 172.21.15.31:0/3351632147 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f562013fb20 0x7f56200ad0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:04.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.487+0000 7f561effd700 1 -- 172.21.15.31:0/3351632147 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f562013fef0 msgr2=0x7f56200c75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:04.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.487+0000 7f561effd700 1 --2- 172.21.15.31:0/3351632147 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f562013fef0 0x7f56200c75f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:04.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.487+0000 7f561effd700 1 -- 172.21.15.31:0/3351632147 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f562013fb20 msgr2=0x7f56200ad0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:04.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.487+0000 7f561effd700 1 --2- 172.21.15.31:0/3351632147 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f562013fb20 0x7f56200ad0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:04.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.487+0000 7f561effd700 1 -- 172.21.15.31:0/3351632147 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56140093c0 con 0x7f562006aaa0 2023-10-06T02:24:04.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.487+0000 7f561effd700 1 --2- 172.21.15.31:0/3351632147 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f562006aaa0 0x7f562013cb60 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f561000c7c0 tx=0x7f5610007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:04.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.487+0000 7f5606ffd700 1 -- 172.21.15.31:0/3351632147 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5610005a70 con 0x7f562006aaa0 2023-10-06T02:24:04.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.487+0000 7f5606ffd700 1 -- 172.21.15.31:0/3351632147 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5610005bf0 con 0x7f562006aaa0 2023-10-06T02:24:04.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.487+0000 7f561ffff700 1 -- 172.21.15.31:0/3351632147 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f561800bcf0 con 0x7f562006aaa0 2023-10-06T02:24:04.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.487+0000 7f5606ffd700 1 -- 172.21.15.31:0/3351632147 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56100070f0 con 0x7f562006aaa0 2023-10-06T02:24:04.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.488+0000 7f561ffff700 1 -- 172.21.15.31:0/3351632147 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5620156fb0 con 0x7f562006aaa0 2023-10-06T02:24:04.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.489+0000 7f561ffff700 1 -- 172.21.15.31:0/3351632147 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f562004c3c0 con 0x7f562006aaa0 2023-10-06T02:24:04.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.489+0000 7f5606ffd700 1 -- 172.21.15.31:0/3351632147 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5610007460 con 0x7f562006aaa0 2023-10-06T02:24:04.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.490+0000 7f5606ffd700 1 --2- 172.21.15.31:0/3351632147 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f560c066f30 0x7f560c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:04.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.490+0000 7f5606ffd700 1 -- 172.21.15.31:0/3351632147 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f5610083dc0 con 0x7f562006aaa0 2023-10-06T02:24:04.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.491+0000 7f561e7fc700 1 --2- 172.21.15.31:0/3351632147 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f560c066f30 0x7f560c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:04.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.491+0000 7f561e7fc700 1 --2- 172.21.15.31:0/3351632147 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f560c066f30 0x7f560c0693e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5614009390 tx=0x7f5614008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:04.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.494+0000 7f5606ffd700 1 -- 172.21.15.31:0/3351632147 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5610051180 con 0x7f562006aaa0 2023-10-06T02:24:04.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.774+0000 7f561ffff700 1 -- 172.21.15.31:0/3351632147 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f562015ae70 con 0x7f562006aaa0 2023-10-06T02:24:04.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.775+0000 7f5606ffd700 1 -- 172.21.15.31:0/3351632147 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f562015ae70 con 0x7f562006aaa0 2023-10-06T02:24:04.779 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:24:04.779 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T02:24:04.779 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_dump": [ 2023-10-06T02:24:04.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:24:04.779 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.a", 2023-10-06T02:24:04.779 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBLbx9lbjrOGxAAVoSOoDaZb1temL41F4Nong==", 2023-10-06T02:24:04.779 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:24:04.779 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:24:04.779 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T02:24:04.780 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T02:24:04.780 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:24:04.780 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:24:04.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:24:04.780 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:24:04.780 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.b", 2023-10-06T02:24:04.780 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBLbx9lFgsTMRAATTklzroiyQbTLiZc9j1BHQ==", 2023-10-06T02:24:04.780 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:24:04.780 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:24:04.780 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T02:24:04.781 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T02:24:04.781 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:24:04.781 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:24:04.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:24:04.781 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:24:04.781 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.c", 2023-10-06T02:24:04.781 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBLbx9lezVtJhAA8wK7+dJ1OmIc+iXJkexK0w==", 2023-10-06T02:24:04.781 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:24:04.781 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:24:04.781 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T02:24:04.781 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T02:24:04.782 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:24:04.782 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:24:04.782 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:24:04.782 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:24:04.782 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.d", 2023-10-06T02:24:04.782 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBMbx9l74gWABAApBu2nVv12rGA+OBfBnEc3A==", 2023-10-06T02:24:04.782 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:24:04.782 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:24:04.782 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T02:24:04.782 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T02:24:04.783 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:24:04.783 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:24:04.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:24:04.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:24:04.783 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.0", 2023-10-06T02:24:04.783 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBPbx9lgj6+MBAAXSUD0GCG+rO/9DQBZYt/8w==", 2023-10-06T02:24:04.783 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:24:04.783 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:24:04.783 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:24:04.783 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:24:04.784 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:24:04.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:24:04.784 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:24:04.784 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.1", 2023-10-06T02:24:04.784 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBSbx9lXhMTNBAAeEdPnMP71ZGb6k2Ddg5raQ==", 2023-10-06T02:24:04.784 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:24:04.784 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:24:04.784 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:24:04.784 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:24:04.784 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:24:04.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:24:04.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:24:04.785 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.2", 2023-10-06T02:24:04.785 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBVbx9l4Ly0NhAATYXYDAjYuP53zjqK0yMboA==", 2023-10-06T02:24:04.785 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:24:04.785 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:24:04.785 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:24:04.785 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:24:04.785 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:24:04.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:24:04.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:24:04.786 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.3", 2023-10-06T02:24:04.786 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBZbx9lzsi4ARAAbGnNV0QyL/7itzy827+chg==", 2023-10-06T02:24:04.786 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:24:04.786 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:24:04.786 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:24:04.786 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:24:04.786 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:24:04.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:24:04.786 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:24:04.786 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.4", 2023-10-06T02:24:04.787 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBfbx9lW4OSEhAAj7HnVh/yq12zzCaXd6BHCA==", 2023-10-06T02:24:04.787 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:24:04.787 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:24:04.787 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:24:04.787 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:24:04.787 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:24:04.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:24:04.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:24:04.787 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.5", 2023-10-06T02:24:04.787 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBibx9lBcaIFRAAV8wq8QjsCOkI9Pzf/E+9bA==", 2023-10-06T02:24:04.787 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:24:04.788 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:24:04.788 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:24:04.788 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:24:04.788 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:24:04.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:24:04.788 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:24:04.788 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.6", 2023-10-06T02:24:04.788 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBlbx9lA7S+FxAA38mlSAurT+DpwMzH53zFcw==", 2023-10-06T02:24:04.788 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:24:04.788 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:24:04.789 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:24:04.789 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:24:04.789 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:24:04.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:24:04.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:24:04.789 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.7", 2023-10-06T02:24:04.789 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBobx9l5RkdHhAAxn0Z8IAE/o9oe2LRp3O9gg==", 2023-10-06T02:24:04.789 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:24:04.789 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:24:04.789 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:24:04.789 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:24:04.790 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:24:04.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:24:04.790 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:24:04.790 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.0", 2023-10-06T02:24:04.790 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBMbx9lh44fCRAA7sMISvk1t7dEJg4ZcBiMcA==", 2023-10-06T02:24:04.790 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:24:04.790 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:24:04.790 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-10-06T02:24:04.791 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-10-06T02:24:04.791 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-10-06T02:24:04.791 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:24:04.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:24:04.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:24:04.791 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.1", 2023-10-06T02:24:04.791 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBMbx9lP0zqFhAAlov2y7ipSvS3AkOdQkhg6g==", 2023-10-06T02:24:04.791 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:24:04.791 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:24:04.791 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-10-06T02:24:04.792 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-10-06T02:24:04.792 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-10-06T02:24:04.792 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:24:04.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:24:04.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:24:04.792 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.2", 2023-10-06T02:24:04.792 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBMbx9lpxYAEBAACtiI1XgQXM8yGGcpFyNK1w==", 2023-10-06T02:24:04.792 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:24:04.792 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:24:04.792 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-10-06T02:24:04.792 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-10-06T02:24:04.793 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-10-06T02:24:04.793 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:24:04.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:24:04.793 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:24:04.793 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.3", 2023-10-06T02:24:04.793 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBMbx9l81/PHRAAYFOoaezu0pwE0NdtnlYfVA==", 2023-10-06T02:24:04.793 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:24:04.801 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:24:04.801 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-10-06T02:24:04.802 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-10-06T02:24:04.802 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-10-06T02:24:04.802 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:24:04.802 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:24:04.802 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:24:04.802 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.admin", 2023-10-06T02:24:04.802 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBKbx9liFl1JBAAFyyHYNLVx1/Ex5a2MKm8rA==", 2023-10-06T02:24:04.802 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:24:04.802 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-10-06T02:24:04.802 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T02:24:04.803 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T02:24:04.803 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:24:04.803 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:24:04.803 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:24:04.803 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:24:04.803 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.bootstrap-mds", 2023-10-06T02:24:04.803 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBwbx9l0RDqChAAeDBg1t237oYudBOjfap7wg==", 2023-10-06T02:24:04.803 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:24:04.803 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile bootstrap-mds" 2023-10-06T02:24:04.803 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:24:04.803 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:24:04.804 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:24:04.804 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.bootstrap-mgr", 2023-10-06T02:24:04.804 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBwbx9loxzqChAAEMsxIo3QsF/Zv8t4QcFPFA==", 2023-10-06T02:24:04.804 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:24:04.804 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile bootstrap-mgr" 2023-10-06T02:24:04.804 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:24:04.804 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:24:04.804 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:24:04.804 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.bootstrap-osd", 2023-10-06T02:24:04.804 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBwbx9lxybqChAAqAZFGa+STbtMyCLosnJYpA==", 2023-10-06T02:24:04.805 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:24:04.805 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile bootstrap-osd" 2023-10-06T02:24:04.805 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:24:04.805 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:24:04.805 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:24:04.805 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.bootstrap-rbd", 2023-10-06T02:24:04.805 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBwbx9l5C7qChAAmlQeTSXASm+/SaIXyaS9Cw==", 2023-10-06T02:24:04.805 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:24:04.805 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile bootstrap-rbd" 2023-10-06T02:24:04.805 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:24:04.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:24:04.806 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:24:04.806 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-10-06T02:24:04.806 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBwbx9ldTfqChAAkFl4YsG02/tPxxYtoRko/A==", 2023-10-06T02:24:04.806 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:24:04.806 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-10-06T02:24:04.806 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:24:04.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:24:04.806 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:24:04.806 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.bootstrap-rgw", 2023-10-06T02:24:04.807 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBwbx9luz/qChAAz/7yP3zses61IU9IiF/HzQ==", 2023-10-06T02:24:04.807 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:24:04.807 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile bootstrap-rgw" 2023-10-06T02:24:04.807 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:24:04.807 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:24:04.807 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:24:04.807 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mgr.x", 2023-10-06T02:24:04.807 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBLbx9lTBbIFBAAX2wsioKL4Nx4qz/qwsCsNA==", 2023-10-06T02:24:04.807 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:24:04.807 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-10-06T02:24:04.808 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile mgr", 2023-10-06T02:24:04.808 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:24:04.808 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:24:04.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:24:04.808 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:24:04.808 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mgr.y", 2023-10-06T02:24:04.808 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBLbx9l4+oLDRAA5YVo3q6FI9e9W/rJYROIsg==", 2023-10-06T02:24:04.808 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:24:04.808 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-10-06T02:24:04.808 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile mgr", 2023-10-06T02:24:04.809 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:24:04.809 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:24:04.809 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:24:04.809 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-10-06T02:24:04.809 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T02:24:04.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.777+0000 7f561ffff700 1 -- 172.21.15.31:0/3351632147 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f560c066f30 msgr2=0x7f560c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:04.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.777+0000 7f561ffff700 1 --2- 172.21.15.31:0/3351632147 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f560c066f30 0x7f560c0693e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5614009390 tx=0x7f5614008040 comp rx=0 tx=0).stop 2023-10-06T02:24:04.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.777+0000 7f561ffff700 1 -- 172.21.15.31:0/3351632147 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f562006aaa0 msgr2=0x7f562013cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:04.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.777+0000 7f561ffff700 1 --2- 172.21.15.31:0/3351632147 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f562006aaa0 0x7f562013cb60 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f561000c7c0 tx=0x7f5610007f70 comp rx=0 tx=0).stop 2023-10-06T02:24:04.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.777+0000 7f561ffff700 1 -- 172.21.15.31:0/3351632147 shutdown_connections 2023-10-06T02:24:04.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.777+0000 7f561ffff700 1 --2- 172.21.15.31:0/3351632147 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f560c066f30 0x7f560c0693e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:04.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.777+0000 7f561ffff700 1 --2- 172.21.15.31:0/3351632147 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f562013fef0 0x7f56200c75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:04.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.777+0000 7f561ffff700 1 --2- 172.21.15.31:0/3351632147 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f562006aaa0 0x7f562013cb60 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:04.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.777+0000 7f561ffff700 1 --2- 172.21.15.31:0/3351632147 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f562013fb20 0x7f56200ad0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:04.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.777+0000 7f561ffff700 1 -- 172.21.15.31:0/3351632147 >> 172.21.15.31:0/3351632147 conn(0x7f562006d170 msgr2=0x7f56200c9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:04.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.777+0000 7f561ffff700 1 -- 172.21.15.31:0/3351632147 shutdown_connections 2023-10-06T02:24:04.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.777+0000 7f561ffff700 1 -- 172.21.15.31:0/3351632147 wait complete. 2023-10-06T02:24:04.812 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-10-06T02:24:05.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.999+0000 7f837d99d700 1 Processor -- start 2023-10-06T02:24:05.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:04.999+0000 7f837d99d700 1 -- start start 2023-10-06T02:24:05.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.001+0000 7f837d99d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f83780b2420 0x7f83780b2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:05.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.001+0000 7f837d99d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83780b2dd0 con 0x7f83781574a0 2023-10-06T02:24:05.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.001+0000 7f837d99d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83780cd9d0 con 0x7f83780cb180 2023-10-06T02:24:05.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.001+0000 7f837d99d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8378158bd0 con 0x7f83780b2420 2023-10-06T02:24:05.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.001+0000 7f837c99b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f83781574a0 0x7f83781584c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:38656/0 (socket says 172.21.15.31:38656) 2023-10-06T02:24:05.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.001+0000 7f837c99b700 1 -- 172.21.15.31:0/1079175831 learned_addr learned my addr 172.21.15.31:0/1079175831 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:24:05.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.001+0000 7f8377fff700 1 --2- 172.21.15.31:0/1079175831 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f83780b2420 0x7f83780b2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:05.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.001+0000 7f83777fe700 1 -- 172.21.15.31:0/1079175831 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1565656609 0 0) 0x7f83780b2dd0 con 0x7f83781574a0 2023-10-06T02:24:05.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.002+0000 7f83777fe700 1 -- 172.21.15.31:0/1079175831 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8368003570 con 0x7f83781574a0 2023-10-06T02:24:05.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.002+0000 7f83777fe700 1 -- 172.21.15.31:0/1079175831 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1618403879 0 0) 0x7f83780cd9d0 con 0x7f83780cb180 2023-10-06T02:24:05.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.002+0000 7f83777fe700 1 -- 172.21.15.31:0/1079175831 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83780b2dd0 con 0x7f83780cb180 2023-10-06T02:24:05.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.002+0000 7f8377fff700 1 -- 172.21.15.31:0/1079175831 >> v1:172.21.15.62:6789/0 conn(0x7f83780cb180 legacy=0x7f83780cb600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:05.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.002+0000 7f8377fff700 1 -- 172.21.15.31:0/1079175831 >> v1:172.21.15.31:6789/0 conn(0x7f83781574a0 legacy=0x7f83781584c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:05.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.002+0000 7f8377fff700 1 -- 172.21.15.31:0/1079175831 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83780ccdc0 con 0x7f83780b2420 2023-10-06T02:24:05.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.002+0000 7f8377fff700 1 --2- 172.21.15.31:0/1079175831 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f83780b2420 0x7f83780b2890 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8364009940 tx=0x7f83640091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e45c353631e5bffc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:05.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.003+0000 7f83777fe700 1 -- 172.21.15.31:0/1079175831 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8364011630 con 0x7f83780b2420 2023-10-06T02:24:05.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.003+0000 7f83777fe700 1 -- 172.21.15.31:0/1079175831 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8364011790 con 0x7f83780b2420 2023-10-06T02:24:05.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.003+0000 7f83777fe700 1 -- 172.21.15.31:0/1079175831 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8364011b10 con 0x7f83780b2420 2023-10-06T02:24:05.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.003+0000 7f837d99d700 1 -- 172.21.15.31:0/1079175831 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f83780b2420 msgr2=0x7f83780b2890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:05.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.003+0000 7f837d99d700 1 --2- 172.21.15.31:0/1079175831 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f83780b2420 0x7f83780b2890 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8364009940 tx=0x7f83640091a0 comp rx=0 tx=0).stop 2023-10-06T02:24:05.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.003+0000 7f837d99d700 1 -- 172.21.15.31:0/1079175831 shutdown_connections 2023-10-06T02:24:05.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.003+0000 7f837d99d700 1 --2- 172.21.15.31:0/1079175831 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f83780b2420 0x7f83780b2890 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:05.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.003+0000 7f837d99d700 1 -- 172.21.15.31:0/1079175831 >> 172.21.15.31:0/1079175831 conn(0x7f83780675a0 msgr2=0x7f83780679a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:05.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.003+0000 7f837d99d700 1 -- 172.21.15.31:0/1079175831 shutdown_connections 2023-10-06T02:24:05.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.003+0000 7f837d99d700 1 -- 172.21.15.31:0/1079175831 wait complete. 2023-10-06T02:24:05.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.005+0000 7f837d99d700 1 Processor -- start 2023-10-06T02:24:05.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.005+0000 7f837d99d700 1 -- start start 2023-10-06T02:24:05.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.006+0000 7f837d99d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f83780b2420 0x7f8378146e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:05.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.006+0000 7f837d99d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f83780cb180 0x7f83780ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:05.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.006+0000 7f837d99d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83781574a0 0x7f83780aedf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:05.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.006+0000 7f837d99d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83780afc40 con 0x7f83780b2420 2023-10-06T02:24:05.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.006+0000 7f837d99d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f837815cf80 con 0x7f83781574a0 2023-10-06T02:24:05.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.006+0000 7f837d99d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f837815c370 con 0x7f83780cb180 2023-10-06T02:24:05.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.006+0000 7f837c99b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f83780b2420 0x7f8378146e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:05.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.006+0000 7f837c99b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f83780b2420 0x7f8378146e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49850/0 (socket says 172.21.15.31:49850) 2023-10-06T02:24:05.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.006+0000 7f837c99b700 1 -- 172.21.15.31:0/474662105 learned_addr learned my addr 172.21.15.31:0/474662105 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:05.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.006+0000 7f837c99b700 1 -- 172.21.15.31:0/474662105 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f83780cb180 msgr2=0x7f83780ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:05.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.006+0000 7f837c99b700 1 --2- 172.21.15.31:0/474662105 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f83780cb180 0x7f83780ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:05.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.006+0000 7f837c99b700 1 -- 172.21.15.31:0/474662105 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83781574a0 msgr2=0x7f83780aedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:05.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.006+0000 7f837d19c700 1 --2- 172.21.15.31:0/474662105 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83781574a0 0x7f83780aedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:05.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.007+0000 7f837c99b700 1 --2- 172.21.15.31:0/474662105 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83781574a0 0x7f83780aedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:05.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.007+0000 7f837c99b700 1 -- 172.21.15.31:0/474662105 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8364009420 con 0x7f83780b2420 2023-10-06T02:24:05.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.007+0000 7f837d19c700 1 --2- 172.21.15.31:0/474662105 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83781574a0 0x7f83780aedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:24:05.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.007+0000 7f837c99b700 1 --2- 172.21.15.31:0/474662105 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f83780b2420 0x7f8378146e20 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f836c00c790 tx=0x7f836c007ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:05.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.007+0000 7f8374ff9700 1 -- 172.21.15.31:0/474662105 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f836c0072e0 con 0x7f83780b2420 2023-10-06T02:24:05.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.007+0000 7f8374ff9700 1 -- 172.21.15.31:0/474662105 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f836c007440 con 0x7f83780b2420 2023-10-06T02:24:05.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.007+0000 7f8374ff9700 1 -- 172.21.15.31:0/474662105 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f836c0057f0 con 0x7f83780b2420 2023-10-06T02:24:05.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.007+0000 7f837d99d700 1 -- 172.21.15.31:0/474662105 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f837815c650 con 0x7f83780b2420 2023-10-06T02:24:05.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.007+0000 7f837d99d700 1 -- 172.21.15.31:0/474662105 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f837815cc10 con 0x7f83780b2420 2023-10-06T02:24:05.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.008+0000 7f837d99d700 1 -- 172.21.15.31:0/474662105 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f837804c3c0 con 0x7f83780b2420 2023-10-06T02:24:05.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.009+0000 7f8374ff9700 1 -- 172.21.15.31:0/474662105 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f836c005b60 con 0x7f83780b2420 2023-10-06T02:24:05.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.010+0000 7f8374ff9700 1 --2- 172.21.15.31:0/474662105 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f835c066f30 0x7f835c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:05.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.010+0000 7f8374ff9700 1 -- 172.21.15.31:0/474662105 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f836c083880 con 0x7f83780b2420 2023-10-06T02:24:05.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.010+0000 7f8377fff700 1 --2- 172.21.15.31:0/474662105 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f835c066f30 0x7f835c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:05.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.011+0000 7f8377fff700 1 --2- 172.21.15.31:0/474662105 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f835c066f30 0x7f835c0693e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8378160330 tx=0x7f8364017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:05.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.015+0000 7f8374ff9700 1 -- 172.21.15.31:0/474662105 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f836c04b5a0 con 0x7f83780b2420 2023-10-06T02:24:05.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.303+0000 7f837d99d700 1 -- 172.21.15.31:0/474662105 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f837806b190 con 0x7f83780b2420 2023-10-06T02:24:05.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.309+0000 7f8374ff9700 1 -- 172.21.15.31:0/474662105 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7f836c017020 con 0x7f83780b2420 2023-10-06T02:24:05.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.311+0000 7f837d99d700 1 -- 172.21.15.31:0/474662105 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f835c066f30 msgr2=0x7f835c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:05.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.311+0000 7f837d99d700 1 --2- 172.21.15.31:0/474662105 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f835c066f30 0x7f835c0693e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8378160330 tx=0x7f8364017000 comp rx=0 tx=0).stop 2023-10-06T02:24:05.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.311+0000 7f837d99d700 1 -- 172.21.15.31:0/474662105 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f83780b2420 msgr2=0x7f8378146e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:05.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.311+0000 7f837d99d700 1 --2- 172.21.15.31:0/474662105 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f83780b2420 0x7f8378146e20 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f836c00c790 tx=0x7f836c007ce0 comp rx=0 tx=0).stop 2023-10-06T02:24:05.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.311+0000 7f837d99d700 1 -- 172.21.15.31:0/474662105 shutdown_connections 2023-10-06T02:24:05.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.311+0000 7f837d99d700 1 --2- 172.21.15.31:0/474662105 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83781574a0 0x7f83780aedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:05.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.311+0000 7f837d99d700 1 --2- 172.21.15.31:0/474662105 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f83780b2420 0x7f8378146e20 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:05.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.311+0000 7f837d99d700 1 --2- 172.21.15.31:0/474662105 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f835c066f30 0x7f835c0693e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:05.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.311+0000 7f837d99d700 1 --2- 172.21.15.31:0/474662105 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f83780cb180 0x7f83780ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:05.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.311+0000 7f837d99d700 1 -- 172.21.15.31:0/474662105 >> 172.21.15.31:0/474662105 conn(0x7f83780675a0 msgr2=0x7f8378149c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:05.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.312+0000 7f837d99d700 1 -- 172.21.15.31:0/474662105 shutdown_connections 2023-10-06T02:24:05.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.312+0000 7f837d99d700 1 -- 172.21.15.31:0/474662105 wait complete. 2023-10-06T02:24:05.331 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-10-06T02:24:05.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.519+0000 7fe774f9d700 1 Processor -- start 2023-10-06T02:24:05.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.519+0000 7fe774f9d700 1 -- start start 2023-10-06T02:24:05.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.520+0000 7fe774f9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe770142fd0 0x7fe7700c7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:05.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.520+0000 7fe774f9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe7700c7dc0 0x7fe77006c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:05.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.520+0000 7fe774f9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7700ca480 con 0x7fe770142c00 2023-10-06T02:24:05.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.520+0000 7fe774f9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe77015e240 con 0x7fe7700c7dc0 2023-10-06T02:24:05.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.520+0000 7fe774f9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe77006b350 con 0x7fe770142fd0 2023-10-06T02:24:05.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.520+0000 7fe76ffff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe7700c7dc0 0x7fe77006c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:05.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.520+0000 7fe76ffff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe7700c7dc0 0x7fe77006c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49854/0 (socket says 172.21.15.31:49854) 2023-10-06T02:24:05.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.520+0000 7fe76ffff700 1 -- 172.21.15.31:0/501228783 learned_addr learned my addr 172.21.15.31:0/501228783 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:05.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.521+0000 7fe76effd700 1 --2- 172.21.15.31:0/501228783 >> v2:172.21.15.62:3300/0 conn(0x7fe770142fd0 0x7fe7700c7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:05.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.521+0000 7fe76ffff700 1 -- 172.21.15.31:0/501228783 >> v2:172.21.15.62:3300/0 conn(0x7fe770142fd0 msgr2=0x7fe7700c7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:05.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.521+0000 7fe76ffff700 1 --2- 172.21.15.31:0/501228783 >> v2:172.21.15.62:3300/0 conn(0x7fe770142fd0 0x7fe7700c7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:05.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.521+0000 7fe76ffff700 1 -- 172.21.15.31:0/501228783 >> v1:172.21.15.62:6789/0 conn(0x7fe770142c00 legacy=0x7fe77006b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:05.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.521+0000 7fe76ffff700 1 -- 172.21.15.31:0/501228783 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7700c81d0 con 0x7fe7700c7dc0 2023-10-06T02:24:05.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.521+0000 7fe76ffff700 1 --2- 172.21.15.31:0/501228783 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7700c7dc0 0x7fe77006c6a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe7700ce1b0 tx=0x7fe768009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a825c0b1ec93d24 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:05.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.521+0000 7fe76e7fc700 1 -- 172.21.15.31:0/501228783 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe76800e070 con 0x7fe7700c7dc0 2023-10-06T02:24:05.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.521+0000 7fe76e7fc700 1 -- 172.21.15.31:0/501228783 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe76800ba50 con 0x7fe7700c7dc0 2023-10-06T02:24:05.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.522+0000 7fe76e7fc700 1 -- 172.21.15.31:0/501228783 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7680092e0 con 0x7fe7700c7dc0 2023-10-06T02:24:05.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.522+0000 7fe774f9d700 1 -- 172.21.15.31:0/501228783 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7700c7dc0 msgr2=0x7fe77006c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:05.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.522+0000 7fe774f9d700 1 --2- 172.21.15.31:0/501228783 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7700c7dc0 0x7fe77006c6a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe7700ce1b0 tx=0x7fe768009840 comp rx=0 tx=0).stop 2023-10-06T02:24:05.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.522+0000 7fe774f9d700 1 -- 172.21.15.31:0/501228783 shutdown_connections 2023-10-06T02:24:05.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.522+0000 7fe774f9d700 1 --2- 172.21.15.31:0/501228783 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7700c7dc0 0x7fe77006c6a0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:05.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.522+0000 7fe774f9d700 1 --2- 172.21.15.31:0/501228783 >> v2:172.21.15.62:3300/0 conn(0x7fe770142fd0 0x7fe7700c7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:05.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.522+0000 7fe774f9d700 1 -- 172.21.15.31:0/501228783 >> 172.21.15.31:0/501228783 conn(0x7fe7700675a0 msgr2=0x7fe7700679a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:05.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.522+0000 7fe774f9d700 1 -- 172.21.15.31:0/501228783 shutdown_connections 2023-10-06T02:24:05.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.522+0000 7fe774f9d700 1 -- 172.21.15.31:0/501228783 wait complete. 2023-10-06T02:24:05.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.524+0000 7fe774f9d700 1 Processor -- start 2023-10-06T02:24:05.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.524+0000 7fe774f9d700 1 -- start start 2023-10-06T02:24:05.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.525+0000 7fe774f9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7700c7dc0 0x7fe7700b2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:05.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.525+0000 7fe774f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe770142c00 0x7fe770158f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:05.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.525+0000 7fe774f9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe770142fd0 0x7fe77015bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:05.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.525+0000 7fe774f9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe77015d320 con 0x7fe7700c7dc0 2023-10-06T02:24:05.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.525+0000 7fe774f9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe77015c710 con 0x7fe770142fd0 2023-10-06T02:24:05.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.525+0000 7fe774f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe77015c9f0 con 0x7fe770142c00 2023-10-06T02:24:05.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.525+0000 7fe76f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7700c7dc0 0x7fe7700b2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:05.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.525+0000 7fe76f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7700c7dc0 0x7fe7700b2bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49858/0 (socket says 172.21.15.31:49858) 2023-10-06T02:24:05.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.525+0000 7fe76f7fe700 1 -- 172.21.15.31:0/2371680001 learned_addr learned my addr 172.21.15.31:0/2371680001 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:05.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.525+0000 7fe76f7fe700 1 -- 172.21.15.31:0/2371680001 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe770142c00 msgr2=0x7fe770158f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:05.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.525+0000 7fe76f7fe700 1 --2- 172.21.15.31:0/2371680001 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe770142c00 0x7fe770158f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:05.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.525+0000 7fe76f7fe700 1 -- 172.21.15.31:0/2371680001 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe770142fd0 msgr2=0x7fe77015bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:05.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.525+0000 7fe76f7fe700 1 --2- 172.21.15.31:0/2371680001 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe770142fd0 0x7fe77015bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:05.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.525+0000 7fe76f7fe700 1 -- 172.21.15.31:0/2371680001 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe768009ae0 con 0x7fe7700c7dc0 2023-10-06T02:24:05.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.525+0000 7fe76f7fe700 1 --2- 172.21.15.31:0/2371680001 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7700c7dc0 0x7fe7700b2bc0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe76000c590 tx=0x7fe76000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:05.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.526+0000 7fe7577fe700 1 -- 172.21.15.31:0/2371680001 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe760011590 con 0x7fe7700c7dc0 2023-10-06T02:24:05.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.526+0000 7fe7577fe700 1 -- 172.21.15.31:0/2371680001 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7600116f0 con 0x7fe7700c7dc0 2023-10-06T02:24:05.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.526+0000 7fe7577fe700 1 -- 172.21.15.31:0/2371680001 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe760011ab0 con 0x7fe7700c7dc0 2023-10-06T02:24:05.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.526+0000 7fe774f9d700 1 -- 172.21.15.31:0/2371680001 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe77015ccd0 con 0x7fe7700c7dc0 2023-10-06T02:24:05.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.526+0000 7fe774f9d700 1 -- 172.21.15.31:0/2371680001 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe770159490 con 0x7fe7700c7dc0 2023-10-06T02:24:05.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.527+0000 7fe774f9d700 1 -- 172.21.15.31:0/2371680001 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe770142490 con 0x7fe7700c7dc0 2023-10-06T02:24:05.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.528+0000 7fe7577fe700 1 -- 172.21.15.31:0/2371680001 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe76001b3b0 con 0x7fe7700c7dc0 2023-10-06T02:24:05.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.529+0000 7fe7577fe700 1 --2- 172.21.15.31:0/2371680001 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe75c066f30 0x7fe75c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:05.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.529+0000 7fe7577fe700 1 -- 172.21.15.31:0/2371680001 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fe760083940 con 0x7fe7700c7dc0 2023-10-06T02:24:05.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.529+0000 7fe76effd700 1 --2- 172.21.15.31:0/2371680001 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe75c066f30 0x7fe75c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:05.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.530+0000 7fe76effd700 1 --2- 172.21.15.31:0/2371680001 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe75c066f30 0x7fe75c0693e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe7700b3100 tx=0x7fe764008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:05.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.533+0000 7fe7577fe700 1 -- 172.21.15.31:0/2371680001 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe760050d00 con 0x7fe7700c7dc0 2023-10-06T02:24:05.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.817+0000 7fe774f9d700 1 -- 172.21.15.31:0/2371680001 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7fe77004c3c0 con 0x7fe7700c7dc0 2023-10-06T02:24:05.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.821+0000 7fe7577fe700 1 -- 172.21.15.31:0/2371680001 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7fe760017020 con 0x7fe7700c7dc0 2023-10-06T02:24:05.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.824+0000 7fe774f9d700 1 -- 172.21.15.31:0/2371680001 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe75c066f30 msgr2=0x7fe75c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:05.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.824+0000 7fe774f9d700 1 --2- 172.21.15.31:0/2371680001 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe75c066f30 0x7fe75c0693e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe7700b3100 tx=0x7fe764008040 comp rx=0 tx=0).stop 2023-10-06T02:24:05.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.824+0000 7fe774f9d700 1 -- 172.21.15.31:0/2371680001 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7700c7dc0 msgr2=0x7fe7700b2bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:05.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.824+0000 7fe774f9d700 1 --2- 172.21.15.31:0/2371680001 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7700c7dc0 0x7fe7700b2bc0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe76000c590 tx=0x7fe76000c950 comp rx=0 tx=0).stop 2023-10-06T02:24:05.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.824+0000 7fe774f9d700 1 -- 172.21.15.31:0/2371680001 shutdown_connections 2023-10-06T02:24:05.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.824+0000 7fe774f9d700 1 --2- 172.21.15.31:0/2371680001 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe770142fd0 0x7fe77015bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:05.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.824+0000 7fe774f9d700 1 --2- 172.21.15.31:0/2371680001 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7700c7dc0 0x7fe7700b2bc0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:05.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.824+0000 7fe774f9d700 1 --2- 172.21.15.31:0/2371680001 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe75c066f30 0x7fe75c0693e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:05.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.824+0000 7fe774f9d700 1 --2- 172.21.15.31:0/2371680001 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe770142c00 0x7fe770158f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:05.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.824+0000 7fe774f9d700 1 -- 172.21.15.31:0/2371680001 >> 172.21.15.31:0/2371680001 conn(0x7fe7700675a0 msgr2=0x7fe77006c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:05.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.824+0000 7fe774f9d700 1 -- 172.21.15.31:0/2371680001 shutdown_connections 2023-10-06T02:24:05.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:05.824+0000 7fe774f9d700 1 -- 172.21.15.31:0/2371680001 wait complete. 2023-10-06T02:24:05.843 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-10-06T02:24:06.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.029+0000 7fb12099d700 1 Processor -- start 2023-10-06T02:24:06.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.029+0000 7fb12099d700 1 -- start start 2023-10-06T02:24:06.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.031+0000 7fb12099d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb11c1574a0 0x7fb11c1584c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:06.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.031+0000 7fb12099d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb11c0cd9d0 con 0x7fb11c0b2b30 2023-10-06T02:24:06.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.031+0000 7fb12099d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb11c0ccdc0 con 0x7fb11c158a00 2023-10-06T02:24:06.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.031+0000 7fb12099d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb11c0cd0a0 con 0x7fb11c1574a0 2023-10-06T02:24:06.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.031+0000 7fb11bfff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fb11c0b2b30 0x7fb11c0cb180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:38658/0 (socket says 172.21.15.31:38658) 2023-10-06T02:24:06.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.031+0000 7fb11bfff700 1 -- 172.21.15.31:0/3398653556 learned_addr learned my addr 172.21.15.31:0/3398653556 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:24:06.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.031+0000 7fb11b7fe700 1 --2- 172.21.15.31:0/3398653556 >> v2:172.21.15.62:3300/0 conn(0x7fb11c1574a0 0x7fb11c1584c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:06.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.031+0000 7fb11a7fc700 1 -- 172.21.15.31:0/3398653556 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 953615272 0 0) 0x7fb11c0cd9d0 con 0x7fb11c0b2b30 2023-10-06T02:24:06.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.031+0000 7fb11a7fc700 1 -- 172.21.15.31:0/3398653556 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb104003570 con 0x7fb11c0b2b30 2023-10-06T02:24:06.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.032+0000 7fb11b7fe700 1 -- 172.21.15.31:0/3398653556 >> v1:172.21.15.62:6789/0 conn(0x7fb11c158a00 legacy=0x7fb11c0b2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:06.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.032+0000 7fb11b7fe700 1 -- 172.21.15.31:0/3398653556 >> v1:172.21.15.31:6789/0 conn(0x7fb11c0b2b30 legacy=0x7fb11c0cb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:06.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.032+0000 7fb11b7fe700 1 -- 172.21.15.31:0/3398653556 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb11c0cd380 con 0x7fb11c1574a0 2023-10-06T02:24:06.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.032+0000 7fb11b7fe700 1 --2- 172.21.15.31:0/3398653556 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb11c1574a0 0x7fb11c1584c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb10c009900 tx=0x7fb10c009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49ec57394ef4b3e6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:06.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.032+0000 7fb11a7fc700 1 -- 172.21.15.31:0/3398653556 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb10c005c10 con 0x7fb11c1574a0 2023-10-06T02:24:06.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.032+0000 7fb11a7fc700 1 -- 172.21.15.31:0/3398653556 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb10c005d70 con 0x7fb11c1574a0 2023-10-06T02:24:06.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.032+0000 7fb11a7fc700 1 -- 172.21.15.31:0/3398653556 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb10c006260 con 0x7fb11c1574a0 2023-10-06T02:24:06.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.033+0000 7fb12099d700 1 -- 172.21.15.31:0/3398653556 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb11c1574a0 msgr2=0x7fb11c1584c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:06.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.033+0000 7fb12099d700 1 --2- 172.21.15.31:0/3398653556 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb11c1574a0 0x7fb11c1584c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb10c009900 tx=0x7fb10c009120 comp rx=0 tx=0).stop 2023-10-06T02:24:06.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.033+0000 7fb12099d700 1 -- 172.21.15.31:0/3398653556 shutdown_connections 2023-10-06T02:24:06.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.033+0000 7fb12099d700 1 --2- 172.21.15.31:0/3398653556 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb11c1574a0 0x7fb11c1584c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:06.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.033+0000 7fb12099d700 1 -- 172.21.15.31:0/3398653556 >> 172.21.15.31:0/3398653556 conn(0x7fb11c0675a0 msgr2=0x7fb11c0679a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:06.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.033+0000 7fb12099d700 1 -- 172.21.15.31:0/3398653556 shutdown_connections 2023-10-06T02:24:06.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.033+0000 7fb12099d700 1 -- 172.21.15.31:0/3398653556 wait complete. 2023-10-06T02:24:06.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.034+0000 7fb12099d700 1 Processor -- start 2023-10-06T02:24:06.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.035+0000 7fb12099d700 1 -- start start 2023-10-06T02:24:06.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.035+0000 7fb12099d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb11c0b2b30 0x7fb11c149f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:06.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.035+0000 7fb12099d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb11c1574a0 0x7fb11c14afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:06.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.036+0000 7fb12099d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb11c0b1f60 0x7fb11c0b23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:06.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.036+0000 7fb12099d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb11c0b2920 con 0x7fb11c1574a0 2023-10-06T02:24:06.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.036+0000 7fb12099d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb11c0a98e0 con 0x7fb11c0b2b30 2023-10-06T02:24:06.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.036+0000 7fb12099d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb11c0aa740 con 0x7fb11c0b1f60 2023-10-06T02:24:06.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.036+0000 7fb11affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb11c1574a0 0x7fb11c14afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:06.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.036+0000 7fb11affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb11c1574a0 0x7fb11c14afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49862/0 (socket says 172.21.15.31:49862) 2023-10-06T02:24:06.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.036+0000 7fb11affd700 1 -- 172.21.15.31:0/2623783717 learned_addr learned my addr 172.21.15.31:0/2623783717 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:06.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.036+0000 7fb11b7fe700 1 --2- 172.21.15.31:0/2623783717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb11c0b2b30 0x7fb11c149f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:06.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.036+0000 7fb11bfff700 1 --2- 172.21.15.31:0/2623783717 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb11c0b1f60 0x7fb11c0b23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:06.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.036+0000 7fb11affd700 1 -- 172.21.15.31:0/2623783717 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb11c0b1f60 msgr2=0x7fb11c0b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:06.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.036+0000 7fb11affd700 1 --2- 172.21.15.31:0/2623783717 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb11c0b1f60 0x7fb11c0b23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:06.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.036+0000 7fb11affd700 1 -- 172.21.15.31:0/2623783717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb11c0b2b30 msgr2=0x7fb11c149f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:06.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.036+0000 7fb11affd700 1 --2- 172.21.15.31:0/2623783717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb11c0b2b30 0x7fb11c149f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:06.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.036+0000 7fb11affd700 1 -- 172.21.15.31:0/2623783717 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb10c0093c0 con 0x7fb11c1574a0 2023-10-06T02:24:06.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.036+0000 7fb11affd700 1 --2- 172.21.15.31:0/2623783717 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb11c1574a0 0x7fb11c14afe0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fb11000c510 tx=0x7fb11000c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:06.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.037+0000 7fb1037fe700 1 -- 172.21.15.31:0/2623783717 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb110011590 con 0x7fb11c1574a0 2023-10-06T02:24:06.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.037+0000 7fb1037fe700 1 -- 172.21.15.31:0/2623783717 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb110011710 con 0x7fb11c1574a0 2023-10-06T02:24:06.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.037+0000 7fb12099d700 1 -- 172.21.15.31:0/2623783717 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb11c0a9b30 con 0x7fb11c1574a0 2023-10-06T02:24:06.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.037+0000 7fb1037fe700 1 -- 172.21.15.31:0/2623783717 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb110011ad0 con 0x7fb11c1574a0 2023-10-06T02:24:06.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.037+0000 7fb12099d700 1 -- 172.21.15.31:0/2623783717 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb11c0aa0f0 con 0x7fb11c1574a0 2023-10-06T02:24:06.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.038+0000 7fb12099d700 1 -- 172.21.15.31:0/2623783717 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb11c0aa8d0 con 0x7fb11c1574a0 2023-10-06T02:24:06.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.039+0000 7fb1037fe700 1 -- 172.21.15.31:0/2623783717 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb11001c440 con 0x7fb11c1574a0 2023-10-06T02:24:06.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.040+0000 7fb1037fe700 1 --2- 172.21.15.31:0/2623783717 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fb108066f30 0x7fb1080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:06.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.040+0000 7fb1037fe700 1 -- 172.21.15.31:0/2623783717 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fb110083720 con 0x7fb11c1574a0 2023-10-06T02:24:06.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.040+0000 7fb11b7fe700 1 --2- 172.21.15.31:0/2623783717 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fb108066f30 0x7fb1080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:06.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.041+0000 7fb11b7fe700 1 --2- 172.21.15.31:0/2623783717 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fb108066f30 0x7fb1080693e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb10c009160 tx=0x7fb10c016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:06.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.044+0000 7fb1037fe700 1 -- 172.21.15.31:0/2623783717 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb110050b80 con 0x7fb11c1574a0 2023-10-06T02:24:06.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.327+0000 7fb12099d700 1 -- 172.21.15.31:0/2623783717 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7fb11c04c3c0 con 0x7fb11c1574a0 2023-10-06T02:24:06.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.331+0000 7fb1037fe700 1 -- 172.21.15.31:0/2623783717 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7fb110017020 con 0x7fb11c1574a0 2023-10-06T02:24:06.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.333+0000 7fb12099d700 1 -- 172.21.15.31:0/2623783717 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fb108066f30 msgr2=0x7fb1080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:06.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.333+0000 7fb12099d700 1 --2- 172.21.15.31:0/2623783717 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fb108066f30 0x7fb1080693e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb10c009160 tx=0x7fb10c016000 comp rx=0 tx=0).stop 2023-10-06T02:24:06.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.333+0000 7fb12099d700 1 -- 172.21.15.31:0/2623783717 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb11c1574a0 msgr2=0x7fb11c14afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:06.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.333+0000 7fb12099d700 1 --2- 172.21.15.31:0/2623783717 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb11c1574a0 0x7fb11c14afe0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fb11000c510 tx=0x7fb11000c820 comp rx=0 tx=0).stop 2023-10-06T02:24:06.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.333+0000 7fb12099d700 1 -- 172.21.15.31:0/2623783717 shutdown_connections 2023-10-06T02:24:06.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.333+0000 7fb12099d700 1 --2- 172.21.15.31:0/2623783717 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fb108066f30 0x7fb1080693e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:06.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.333+0000 7fb12099d700 1 --2- 172.21.15.31:0/2623783717 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb11c0b1f60 0x7fb11c0b23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:06.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.333+0000 7fb12099d700 1 --2- 172.21.15.31:0/2623783717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb11c0b2b30 0x7fb11c149f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:06.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.333+0000 7fb12099d700 1 --2- 172.21.15.31:0/2623783717 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb11c1574a0 0x7fb11c14afe0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:06.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.333+0000 7fb12099d700 1 -- 172.21.15.31:0/2623783717 >> 172.21.15.31:0/2623783717 conn(0x7fb11c0675a0 msgr2=0x7fb11c0b0e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:06.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.333+0000 7fb12099d700 1 -- 172.21.15.31:0/2623783717 shutdown_connections 2023-10-06T02:24:06.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.333+0000 7fb12099d700 1 -- 172.21.15.31:0/2623783717 wait complete. 2023-10-06T02:24:06.352 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-10-06T02:24:06.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.538+0000 7f4856b9d700 1 Processor -- start 2023-10-06T02:24:06.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.538+0000 7f4856b9d700 1 -- start start 2023-10-06T02:24:06.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.540+0000 7f4856b9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f485013fb20 0x7f485015a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:06.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.540+0000 7f4856b9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f485013fef0 0x7f48500c7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:06.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.540+0000 7f4856b9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f485006aaa0 0x7f48500c7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:06.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.540+0000 7f4856b9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48500c8410 con 0x7f485013fb20 2023-10-06T02:24:06.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.540+0000 7f4856b9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f485015ae70 con 0x7f485006aaa0 2023-10-06T02:24:06.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.540+0000 7f4856b9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f485006af60 con 0x7f485013fef0 2023-10-06T02:24:06.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.540+0000 7f4855b9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f485013fb20 0x7f485015a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:06.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.540+0000 7f4855b9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f485013fb20 0x7f485015a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49866/0 (socket says 172.21.15.31:49866) 2023-10-06T02:24:06.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.540+0000 7f4855b9b700 1 -- 172.21.15.31:0/2126535187 learned_addr learned my addr 172.21.15.31:0/2126535187 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:06.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.540+0000 7f485639c700 1 --2- 172.21.15.31:0/2126535187 >> v2:172.21.15.62:3300/0 conn(0x7f485006aaa0 0x7f48500c7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:06.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.540+0000 7f485539a700 1 --2- 172.21.15.31:0/2126535187 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f485013fef0 0x7f48500c7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:06.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.541+0000 7f4855b9b700 1 -- 172.21.15.31:0/2126535187 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f485013fef0 msgr2=0x7f48500c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:06.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.541+0000 7f4855b9b700 1 --2- 172.21.15.31:0/2126535187 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f485013fef0 0x7f48500c7990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:06.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.541+0000 7f4855b9b700 1 -- 172.21.15.31:0/2126535187 >> v2:172.21.15.62:3300/0 conn(0x7f485006aaa0 msgr2=0x7f48500c7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:06.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.541+0000 7f4855b9b700 1 --2- 172.21.15.31:0/2126535187 >> v2:172.21.15.62:3300/0 conn(0x7f485006aaa0 0x7f48500c7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:06.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.541+0000 7f4855b9b700 1 -- 172.21.15.31:0/2126535187 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f485006b240 con 0x7f485013fb20 2023-10-06T02:24:06.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.541+0000 7f485639c700 1 --2- 172.21.15.31:0/2126535187 >> v2:172.21.15.62:3300/0 conn(0x7f485006aaa0 0x7f48500c7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:24:06.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.541+0000 7f4855b9b700 1 --2- 172.21.15.31:0/2126535187 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f485013fb20 0x7f485015a930 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4850156920 tx=0x7f484c00b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92f7c361449bc0fb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:06.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.541+0000 7f4854b99700 1 -- 172.21.15.31:0/2126535187 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f484c00dc50 con 0x7f485013fb20 2023-10-06T02:24:06.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.541+0000 7f4854b99700 1 -- 172.21.15.31:0/2126535187 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f484c00ddb0 con 0x7f485013fb20 2023-10-06T02:24:06.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.541+0000 7f4854b99700 1 -- 172.21.15.31:0/2126535187 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f484c0196d0 con 0x7f485013fb20 2023-10-06T02:24:06.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.542+0000 7f4856b9d700 1 -- 172.21.15.31:0/2126535187 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f485013fb20 msgr2=0x7f485015a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:06.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.542+0000 7f4856b9d700 1 --2- 172.21.15.31:0/2126535187 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f485013fb20 0x7f485015a930 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4850156920 tx=0x7f484c00b4f0 comp rx=0 tx=0).stop 2023-10-06T02:24:06.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.542+0000 7f4856b9d700 1 -- 172.21.15.31:0/2126535187 shutdown_connections 2023-10-06T02:24:06.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.542+0000 7f4856b9d700 1 --2- 172.21.15.31:0/2126535187 >> v2:172.21.15.62:3300/0 conn(0x7f485006aaa0 0x7f48500c7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:06.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.542+0000 7f4856b9d700 1 --2- 172.21.15.31:0/2126535187 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f485013fb20 0x7f485015a930 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:06.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.542+0000 7f4856b9d700 1 --2- 172.21.15.31:0/2126535187 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f485013fef0 0x7f48500c7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:06.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.542+0000 7f4856b9d700 1 -- 172.21.15.31:0/2126535187 >> 172.21.15.31:0/2126535187 conn(0x7f485006d170 msgr2=0x7f48500cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:06.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.542+0000 7f4856b9d700 1 -- 172.21.15.31:0/2126535187 shutdown_connections 2023-10-06T02:24:06.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.542+0000 7f4856b9d700 1 -- 172.21.15.31:0/2126535187 wait complete. 2023-10-06T02:24:06.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.543+0000 7f4856b9d700 1 Processor -- start 2023-10-06T02:24:06.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.544+0000 7f4856b9d700 1 -- start start 2023-10-06T02:24:06.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.544+0000 7f4856b9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f485006aaa0 0x7f485013bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:06.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.544+0000 7f4856b9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f485013fb20 0x7f48500ac310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:06.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.544+0000 7f4856b9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f485013fef0 0x7f485016a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:06.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.544+0000 7f4856b9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48500ac850 con 0x7f485013fef0 2023-10-06T02:24:06.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.545+0000 7f4856b9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f485013c3f0 con 0x7f485006aaa0 2023-10-06T02:24:06.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.545+0000 7f4856b9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f485013c6d0 con 0x7f485013fb20 2023-10-06T02:24:06.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.545+0000 7f485639c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f485013fef0 0x7f485016a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:06.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.545+0000 7f485639c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f485013fef0 0x7f485016a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49882/0 (socket says 172.21.15.31:49882) 2023-10-06T02:24:06.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.545+0000 7f485639c700 1 -- 172.21.15.31:0/482629510 learned_addr learned my addr 172.21.15.31:0/482629510 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:06.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.545+0000 7f485539a700 1 --2- 172.21.15.31:0/482629510 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f485013fb20 0x7f48500ac310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:06.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.545+0000 7f4855b9b700 1 --2- 172.21.15.31:0/482629510 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f485006aaa0 0x7f485013bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:06.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.545+0000 7f485639c700 1 -- 172.21.15.31:0/482629510 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f485013fb20 msgr2=0x7f48500ac310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:06.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.545+0000 7f485639c700 1 --2- 172.21.15.31:0/482629510 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f485013fb20 0x7f48500ac310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:06.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.545+0000 7f485639c700 1 -- 172.21.15.31:0/482629510 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f485006aaa0 msgr2=0x7f485013bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:06.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.545+0000 7f485639c700 1 --2- 172.21.15.31:0/482629510 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f485006aaa0 0x7f485013bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:06.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.545+0000 7f485639c700 1 -- 172.21.15.31:0/482629510 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f484c00b790 con 0x7f485013fef0 2023-10-06T02:24:06.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.545+0000 7f485539a700 1 --2- 172.21.15.31:0/482629510 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f485013fb20 0x7f48500ac310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:24:06.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.545+0000 7f4855b9b700 1 --2- 172.21.15.31:0/482629510 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f485006aaa0 0x7f485013bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:24:06.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.545+0000 7f485639c700 1 --2- 172.21.15.31:0/482629510 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f485013fef0 0x7f485016a7f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f484800d150 tx=0x7f4848009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:06.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.545+0000 7f4845ffb700 1 -- 172.21.15.31:0/482629510 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f484800ddf0 con 0x7f485013fef0 2023-10-06T02:24:06.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.546+0000 7f4845ffb700 1 -- 172.21.15.31:0/482629510 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f484800f040 con 0x7f485013fef0 2023-10-06T02:24:06.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.546+0000 7f4845ffb700 1 -- 172.21.15.31:0/482629510 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4848015690 con 0x7f485013fef0 2023-10-06T02:24:06.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.546+0000 7f4856b9d700 1 -- 172.21.15.31:0/482629510 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f485013c9b0 con 0x7f485013fef0 2023-10-06T02:24:06.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.546+0000 7f4856b9d700 1 -- 172.21.15.31:0/482629510 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f485013cf70 con 0x7f485013fef0 2023-10-06T02:24:06.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.547+0000 7f4856b9d700 1 -- 172.21.15.31:0/482629510 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f485013d190 con 0x7f485013fef0 2023-10-06T02:24:06.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.548+0000 7f4845ffb700 1 -- 172.21.15.31:0/482629510 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f484801e3b0 con 0x7f485013fef0 2023-10-06T02:24:06.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.548+0000 7f4845ffb700 1 --2- 172.21.15.31:0/482629510 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f4834066f30 0x7f48340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:06.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.548+0000 7f4845ffb700 1 -- 172.21.15.31:0/482629510 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f48480849e0 con 0x7f485013fef0 2023-10-06T02:24:06.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.549+0000 7f4855b9b700 1 --2- 172.21.15.31:0/482629510 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f4834066f30 0x7f48340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:06.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.549+0000 7f4855b9b700 1 --2- 172.21.15.31:0/482629510 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f4834066f30 0x7f48340693e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4850156920 tx=0x7f484c01b000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:06.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.552+0000 7f4845ffb700 1 -- 172.21.15.31:0/482629510 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4848051da0 con 0x7f485013fef0 2023-10-06T02:24:06.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.832+0000 7f4856b9d700 1 -- 172.21.15.31:0/482629510 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f485004c3c0 con 0x7f485013fef0 2023-10-06T02:24:06.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.838+0000 7f4845ffb700 1 -- 172.21.15.31:0/482629510 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f484801a0a0 con 0x7f485013fef0 2023-10-06T02:24:06.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.840+0000 7f4856b9d700 1 -- 172.21.15.31:0/482629510 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f4834066f30 msgr2=0x7f48340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:06.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.840+0000 7f4856b9d700 1 --2- 172.21.15.31:0/482629510 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f4834066f30 0x7f48340693e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4850156920 tx=0x7f484c01b000 comp rx=0 tx=0).stop 2023-10-06T02:24:06.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.840+0000 7f4856b9d700 1 -- 172.21.15.31:0/482629510 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f485013fef0 msgr2=0x7f485016a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:06.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.840+0000 7f4856b9d700 1 --2- 172.21.15.31:0/482629510 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f485013fef0 0x7f485016a7f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f484800d150 tx=0x7f4848009a50 comp rx=0 tx=0).stop 2023-10-06T02:24:06.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.840+0000 7f4856b9d700 1 -- 172.21.15.31:0/482629510 shutdown_connections 2023-10-06T02:24:06.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.840+0000 7f4856b9d700 1 --2- 172.21.15.31:0/482629510 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f485013fef0 0x7f485016a7f0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:06.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.840+0000 7f4856b9d700 1 --2- 172.21.15.31:0/482629510 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f485006aaa0 0x7f485013bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:06.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.840+0000 7f4856b9d700 1 --2- 172.21.15.31:0/482629510 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f4834066f30 0x7f48340693e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:06.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.840+0000 7f4856b9d700 1 --2- 172.21.15.31:0/482629510 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f485013fb20 0x7f48500ac310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:06.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.840+0000 7f4856b9d700 1 -- 172.21.15.31:0/482629510 >> 172.21.15.31:0/482629510 conn(0x7f485006d170 msgr2=0x7f48500b1530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:06.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.840+0000 7f4856b9d700 1 -- 172.21.15.31:0/482629510 shutdown_connections 2023-10-06T02:24:06.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:06.840+0000 7f4856b9d700 1 -- 172.21.15.31:0/482629510 wait complete. 2023-10-06T02:24:06.859 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-10-06T02:24:07.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.044+0000 7ff6c719d700 1 Processor -- start 2023-10-06T02:24:07.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.044+0000 7ff6c719d700 1 -- start start 2023-10-06T02:24:07.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.045+0000 7ff6c719d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff6c013fb20 0x7ff6c015a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:07.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.045+0000 7ff6c719d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff6c013fef0 0x7ff6c00c7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:07.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.045+0000 7ff6c719d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6c006aaa0 0x7ff6c00c7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:07.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.045+0000 7ff6c719d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6c00c8410 con 0x7ff6c013fef0 2023-10-06T02:24:07.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.045+0000 7ff6c719d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6c015ae70 con 0x7ff6c013fb20 2023-10-06T02:24:07.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.045+0000 7ff6c719d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c006af60 con 0x7ff6c006aaa0 2023-10-06T02:24:07.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.046+0000 7ff6c599a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff6c013fef0 0x7ff6c00c7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:07.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.046+0000 7ff6c599a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff6c013fef0 0x7ff6c00c7990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49892/0 (socket says 172.21.15.31:49892) 2023-10-06T02:24:07.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.046+0000 7ff6c599a700 1 -- 172.21.15.31:0/2095345413 learned_addr learned my addr 172.21.15.31:0/2095345413 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:07.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.046+0000 7ff6c619b700 1 --2- 172.21.15.31:0/2095345413 >> v2:172.21.15.62:3300/0 conn(0x7ff6c013fb20 0x7ff6c015a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:07.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.046+0000 7ff6c699c700 1 --2- 172.21.15.31:0/2095345413 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6c006aaa0 0x7ff6c00c7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:07.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.046+0000 7ff6c599a700 1 -- 172.21.15.31:0/2095345413 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6c006aaa0 msgr2=0x7ff6c00c7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:07.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.046+0000 7ff6c599a700 1 --2- 172.21.15.31:0/2095345413 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6c006aaa0 0x7ff6c00c7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:07.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.046+0000 7ff6c599a700 1 -- 172.21.15.31:0/2095345413 >> v2:172.21.15.62:3300/0 conn(0x7ff6c013fb20 msgr2=0x7ff6c015a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:07.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.046+0000 7ff6c599a700 1 --2- 172.21.15.31:0/2095345413 >> v2:172.21.15.62:3300/0 conn(0x7ff6c013fb20 0x7ff6c015a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:07.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.046+0000 7ff6c599a700 1 -- 172.21.15.31:0/2095345413 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6c006b240 con 0x7ff6c013fef0 2023-10-06T02:24:07.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.046+0000 7ff6c699c700 1 --2- 172.21.15.31:0/2095345413 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6c006aaa0 0x7ff6c00c7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:24:07.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.046+0000 7ff6c619b700 1 --2- 172.21.15.31:0/2095345413 >> v2:172.21.15.62:3300/0 conn(0x7ff6c013fb20 0x7ff6c015a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:24:07.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.047+0000 7ff6c599a700 1 --2- 172.21.15.31:0/2095345413 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6c013fef0 0x7ff6c00c7990 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff6c00695b0 tx=0x7ff6b0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1299c4ff90420516 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:07.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.047+0000 7ff6c5199700 1 -- 172.21.15.31:0/2095345413 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b000b860 con 0x7ff6c013fef0 2023-10-06T02:24:07.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.047+0000 7ff6c5199700 1 -- 172.21.15.31:0/2095345413 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6b000b9c0 con 0x7ff6c013fef0 2023-10-06T02:24:07.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.047+0000 7ff6c5199700 1 -- 172.21.15.31:0/2095345413 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b000bd60 con 0x7ff6c013fef0 2023-10-06T02:24:07.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.047+0000 7ff6c719d700 1 -- 172.21.15.31:0/2095345413 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6c013fef0 msgr2=0x7ff6c00c7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:07.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.047+0000 7ff6c719d700 1 --2- 172.21.15.31:0/2095345413 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6c013fef0 0x7ff6c00c7990 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff6c00695b0 tx=0x7ff6b0009250 comp rx=0 tx=0).stop 2023-10-06T02:24:07.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.047+0000 7ff6c719d700 1 -- 172.21.15.31:0/2095345413 shutdown_connections 2023-10-06T02:24:07.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.047+0000 7ff6c719d700 1 --2- 172.21.15.31:0/2095345413 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6c006aaa0 0x7ff6c00c7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:07.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.047+0000 7ff6c719d700 1 --2- 172.21.15.31:0/2095345413 >> v2:172.21.15.62:3300/0 conn(0x7ff6c013fb20 0x7ff6c015a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:07.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.047+0000 7ff6c719d700 1 --2- 172.21.15.31:0/2095345413 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6c013fef0 0x7ff6c00c7990 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:07.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.047+0000 7ff6c719d700 1 -- 172.21.15.31:0/2095345413 >> 172.21.15.31:0/2095345413 conn(0x7ff6c006d170 msgr2=0x7ff6c00cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:07.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.048+0000 7ff6c719d700 1 -- 172.21.15.31:0/2095345413 shutdown_connections 2023-10-06T02:24:07.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.048+0000 7ff6c719d700 1 -- 172.21.15.31:0/2095345413 wait complete. 2023-10-06T02:24:07.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.049+0000 7ff6c719d700 1 Processor -- start 2023-10-06T02:24:07.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.049+0000 7ff6c719d700 1 -- start start 2023-10-06T02:24:07.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.050+0000 7ff6c719d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6c006aaa0 0x7ff6c013ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:07.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.050+0000 7ff6c719d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6c013fb20 0x7ff6c013b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:07.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.050+0000 7ff6c719d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6c013fef0 0x7ff6c00c75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:07.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.050+0000 7ff6c719d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c00c2040 con 0x7ff6c006aaa0 2023-10-06T02:24:07.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.050+0000 7ff6c719d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c0159a40 con 0x7ff6c013fef0 2023-10-06T02:24:07.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.050+0000 7ff6c719d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c00ab800 con 0x7ff6c013fb20 2023-10-06T02:24:07.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.050+0000 7ff6c619b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6c006aaa0 0x7ff6c013ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:07.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.050+0000 7ff6c619b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6c006aaa0 0x7ff6c013ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49900/0 (socket says 172.21.15.31:49900) 2023-10-06T02:24:07.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.050+0000 7ff6c619b700 1 -- 172.21.15.31:0/3410400360 learned_addr learned my addr 172.21.15.31:0/3410400360 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:07.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.051+0000 7ff6c619b700 1 -- 172.21.15.31:0/3410400360 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6c013fb20 msgr2=0x7ff6c013b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:07.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.051+0000 7ff6c619b700 1 --2- 172.21.15.31:0/3410400360 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6c013fb20 0x7ff6c013b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:07.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.051+0000 7ff6c699c700 1 --2- 172.21.15.31:0/3410400360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6c013fef0 0x7ff6c00c75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:07.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.051+0000 7ff6c619b700 1 -- 172.21.15.31:0/3410400360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6c013fef0 msgr2=0x7ff6c00c75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:07.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.051+0000 7ff6c619b700 1 --2- 172.21.15.31:0/3410400360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6c013fef0 0x7ff6c00c75f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:07.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.051+0000 7ff6c619b700 1 -- 172.21.15.31:0/3410400360 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6b0009490 con 0x7ff6c006aaa0 2023-10-06T02:24:07.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.051+0000 7ff6c699c700 1 --2- 172.21.15.31:0/3410400360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6c013fef0 0x7ff6c00c75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:24:07.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.051+0000 7ff6c619b700 1 --2- 172.21.15.31:0/3410400360 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6c006aaa0 0x7ff6c013ed90 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc00d860 tx=0x7ff6bc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:07.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.051+0000 7ff6b67fc700 1 -- 172.21.15.31:0/3410400360 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6bc0125f0 con 0x7ff6c006aaa0 2023-10-06T02:24:07.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.051+0000 7ff6b67fc700 1 -- 172.21.15.31:0/3410400360 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6bc012750 con 0x7ff6c006aaa0 2023-10-06T02:24:07.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.051+0000 7ff6c719d700 1 -- 172.21.15.31:0/3410400360 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6c016d410 con 0x7ff6c006aaa0 2023-10-06T02:24:07.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.051+0000 7ff6b67fc700 1 -- 172.21.15.31:0/3410400360 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6bc012ad0 con 0x7ff6c006aaa0 2023-10-06T02:24:07.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.051+0000 7ff6c719d700 1 -- 172.21.15.31:0/3410400360 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6c016d9d0 con 0x7ff6c006aaa0 2023-10-06T02:24:07.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.052+0000 7ff6c719d700 1 -- 172.21.15.31:0/3410400360 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6c004c3c0 con 0x7ff6c006aaa0 2023-10-06T02:24:07.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.053+0000 7ff6b67fc700 1 -- 172.21.15.31:0/3410400360 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff6bc014430 con 0x7ff6c006aaa0 2023-10-06T02:24:07.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.054+0000 7ff6b67fc700 1 --2- 172.21.15.31:0/3410400360 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7ff6a4066f30 0x7ff6a40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:07.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.055+0000 7ff6b67fc700 1 -- 172.21.15.31:0/3410400360 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5206+0+0 (secure 0 0 0) 0x7ff6bc084830 con 0x7ff6c006aaa0 2023-10-06T02:24:07.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.055+0000 7ff6c599a700 1 --2- 172.21.15.31:0/3410400360 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7ff6a4066f30 0x7ff6a40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:07.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.055+0000 7ff6c599a700 1 --2- 172.21.15.31:0/3410400360 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7ff6a4066f30 0x7ff6a40693e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff6c0068320 tx=0x7ff6b0017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:07.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.059+0000 7ff6b67fc700 1 -- 172.21.15.31:0/3410400360 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff6bc055680 con 0x7ff6c006aaa0 2023-10-06T02:24:07.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.339+0000 7ff6c719d700 1 -- 172.21.15.31:0/3410400360 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7ff6c006f660 con 0x7ff6c006aaa0 2023-10-06T02:24:07.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.345+0000 7ff6b67fc700 1 -- 172.21.15.31:0/3410400360 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) v1 ==== 97+0+0 (secure 0 0 0) 0x7ff6bc055210 con 0x7ff6c006aaa0 2023-10-06T02:24:07.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.347+0000 7ff6c719d700 1 -- 172.21.15.31:0/3410400360 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7ff6a4066f30 msgr2=0x7ff6a40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:07.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.347+0000 7ff6c719d700 1 --2- 172.21.15.31:0/3410400360 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7ff6a4066f30 0x7ff6a40693e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff6c0068320 tx=0x7ff6b0017000 comp rx=0 tx=0).stop 2023-10-06T02:24:07.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.347+0000 7ff6c719d700 1 -- 172.21.15.31:0/3410400360 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6c006aaa0 msgr2=0x7ff6c013ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:07.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.347+0000 7ff6c719d700 1 --2- 172.21.15.31:0/3410400360 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6c006aaa0 0x7ff6c013ed90 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc00d860 tx=0x7ff6bc008040 comp rx=0 tx=0).stop 2023-10-06T02:24:07.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.347+0000 7ff6c719d700 1 -- 172.21.15.31:0/3410400360 shutdown_connections 2023-10-06T02:24:07.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.347+0000 7ff6c719d700 1 --2- 172.21.15.31:0/3410400360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6c013fef0 0x7ff6c00c75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:07.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.347+0000 7ff6c719d700 1 --2- 172.21.15.31:0/3410400360 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6c006aaa0 0x7ff6c013ed90 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:07.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.347+0000 7ff6c719d700 1 --2- 172.21.15.31:0/3410400360 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7ff6a4066f30 0x7ff6a40693e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:07.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.347+0000 7ff6c719d700 1 --2- 172.21.15.31:0/3410400360 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6c013fb20 0x7ff6c013b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:07.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.347+0000 7ff6c719d700 1 -- 172.21.15.31:0/3410400360 >> 172.21.15.31:0/3410400360 conn(0x7ff6c006d170 msgr2=0x7ff6c00caa60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:07.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.347+0000 7ff6c719d700 1 -- 172.21.15.31:0/3410400360 shutdown_connections 2023-10-06T02:24:07.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.347+0000 7ff6c719d700 1 -- 172.21.15.31:0/3410400360 wait complete. 2023-10-06T02:24:07.366 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-10-06T02:24:07.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.552+0000 7f373cdff700 1 Processor -- start 2023-10-06T02:24:07.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.553+0000 7f373cdff700 1 -- start start 2023-10-06T02:24:07.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.554+0000 7f373cdff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f37380b1040 0x7f37380c93e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:07.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.554+0000 7f373cdff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37380adb20 con 0x7f3738159850 2023-10-06T02:24:07.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.554+0000 7f373cdff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37380acf10 con 0x7f3738159c20 2023-10-06T02:24:07.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.554+0000 7f373cdff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37380ccdc0 con 0x7f37380b1040 2023-10-06T02:24:07.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.554+0000 7f373659c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f3738159850 0x7f37380adfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:38666/0 (socket says 172.21.15.31:38666) 2023-10-06T02:24:07.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.554+0000 7f373659c700 1 -- 172.21.15.31:0/4040667623 learned_addr learned my addr 172.21.15.31:0/4040667623 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:24:07.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.554+0000 7f373559a700 1 -- 172.21.15.31:0/4040667623 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2010474863 0 0) 0x7f37380adb20 con 0x7f3738159850 2023-10-06T02:24:07.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.554+0000 7f3736d9d700 1 --2- 172.21.15.31:0/4040667623 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f37380b1040 0x7f37380c93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:07.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.555+0000 7f373559a700 1 -- 172.21.15.31:0/4040667623 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3720003570 con 0x7f3738159850 2023-10-06T02:24:07.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.555+0000 7f373559a700 1 -- 172.21.15.31:0/4040667623 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4053691530 0 0) 0x7f37380acf10 con 0x7f3738159c20 2023-10-06T02:24:07.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.555+0000 7f373559a700 1 -- 172.21.15.31:0/4040667623 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37380adb20 con 0x7f3738159c20 2023-10-06T02:24:07.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.555+0000 7f373559a700 1 -- 172.21.15.31:0/4040667623 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2982876578 0 0) 0x7f3720003570 con 0x7f3738159850 2023-10-06T02:24:07.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.555+0000 7f373559a700 1 -- 172.21.15.31:0/4040667623 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f37380b1040 msgr2=0x7f37380c93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:07.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.555+0000 7f373559a700 1 --2- 172.21.15.31:0/4040667623 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f37380b1040 0x7f37380c93e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:07.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.555+0000 7f373559a700 1 -- 172.21.15.31:0/4040667623 >> v1:172.21.15.62:6789/0 conn(0x7f3738159c20 legacy=0x7f37380b0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:07.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.555+0000 7f373559a700 1 -- 172.21.15.31:0/4040667623 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37380ad1f0 con 0x7f3738159850 2023-10-06T02:24:07.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.555+0000 7f373559a700 1 -- 172.21.15.31:0/4040667623 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7f37280013f0 con 0x7f3738159850 2023-10-06T02:24:07.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.555+0000 7f373559a700 1 -- 172.21.15.31:0/4040667623 >> v1:172.21.15.31:6789/0 conn(0x7f3738159850 legacy=0x7f37380adfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:07.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.555+0000 7f373559a700 1 --2- 172.21.15.31:0/4040667623 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f37380b1040 0x7f3720003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:07.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.555+0000 7f373559a700 1 --2- 172.21.15.31:0/4040667623 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37200041d0 0x7f3720004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:07.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.555+0000 7f373559a700 1 --2- 172.21.15.31:0/4040667623 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3720004d50 0x7f37200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:07.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.555+0000 7f373559a700 1 -- 172.21.15.31:0/4040667623 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37380ccdc0 con 0x7f37200041d0 2023-10-06T02:24:07.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.555+0000 7f373559a700 1 -- 172.21.15.31:0/4040667623 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f372000bbf0 con 0x7f3720004d50 2023-10-06T02:24:07.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.555+0000 7f373559a700 1 -- 172.21.15.31:0/4040667623 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f372000bdb0 con 0x7f37380b1040 2023-10-06T02:24:07.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.555+0000 7f3735d9b700 1 --2- 172.21.15.31:0/4040667623 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37200041d0 0x7f3720004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:07.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.555+0000 7f3736d9d700 1 --2- 172.21.15.31:0/4040667623 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3720004d50 0x7f37200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:07.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.555+0000 7f373659c700 1 --2- 172.21.15.31:0/4040667623 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f37380b1040 0x7f3720003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:07.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.556+0000 7f3736d9d700 1 -- 172.21.15.31:0/4040667623 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f37380b1040 msgr2=0x7f3720003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:07.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.556+0000 7f3736d9d700 1 --2- 172.21.15.31:0/4040667623 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f37380b1040 0x7f3720003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:07.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.556+0000 7f3736d9d700 1 -- 172.21.15.31:0/4040667623 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37200041d0 msgr2=0x7f3720004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:07.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.556+0000 7f3736d9d700 1 --2- 172.21.15.31:0/4040667623 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37200041d0 0x7f3720004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:07.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.556+0000 7f3736d9d700 1 -- 172.21.15.31:0/4040667623 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3720003570 con 0x7f3720004d50 2023-10-06T02:24:07.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.556+0000 7f3735d9b700 1 --2- 172.21.15.31:0/4040667623 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37200041d0 0x7f3720004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:24:07.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.556+0000 7f3736d9d700 1 --2- 172.21.15.31:0/4040667623 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3720004d50 0x7f37200051c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3738069a10 tx=0x7f373000b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb85392a00c66564 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:07.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.556+0000 7f373559a700 1 -- 172.21.15.31:0/4040667623 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f373000ea60 con 0x7f3720004d50 2023-10-06T02:24:07.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.556+0000 7f373559a700 1 -- 172.21.15.31:0/4040667623 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f373000f040 con 0x7f3720004d50 2023-10-06T02:24:07.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.556+0000 7f373559a700 1 -- 172.21.15.31:0/4040667623 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f373000ee10 con 0x7f3720004d50 2023-10-06T02:24:07.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.557+0000 7f373cdff700 1 -- 172.21.15.31:0/4040667623 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3720004d50 msgr2=0x7f37200051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:07.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.557+0000 7f373cdff700 1 --2- 172.21.15.31:0/4040667623 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3720004d50 0x7f37200051c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3738069a10 tx=0x7f373000b040 comp rx=0 tx=0).stop 2023-10-06T02:24:07.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.557+0000 7f3736d9d700 1 -- 172.21.15.31:0/4040667623 reap_dead start 2023-10-06T02:24:07.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.557+0000 7f373cdff700 1 -- 172.21.15.31:0/4040667623 shutdown_connections 2023-10-06T02:24:07.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.557+0000 7f373cdff700 1 -- 172.21.15.31:0/4040667623 >> 172.21.15.31:0/4040667623 conn(0x7f373806d170 msgr2=0x7f37380cc300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:07.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.557+0000 7f373cdff700 1 -- 172.21.15.31:0/4040667623 shutdown_connections 2023-10-06T02:24:07.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.557+0000 7f373cdff700 1 -- 172.21.15.31:0/4040667623 wait complete. 2023-10-06T02:24:07.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.558+0000 7f373cdff700 1 Processor -- start 2023-10-06T02:24:07.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.559+0000 7f373cdff700 1 -- start start 2023-10-06T02:24:07.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.559+0000 7f373cdff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3720004d50 0x7f373815b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:07.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.559+0000 7f373cdff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f37381559e0 0x7f373815d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:07.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.560+0000 7f373cdff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f373815f420 0x7f373815f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:07.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.560+0000 7f373cdff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f373815c450 con 0x7f373815f420 2023-10-06T02:24:07.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.560+0000 7f373cdff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3738155e50 con 0x7f3720004d50 2023-10-06T02:24:07.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.560+0000 7f373cdff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f373815deb0 con 0x7f37381559e0 2023-10-06T02:24:07.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.560+0000 7f3736d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f373815f420 0x7f373815f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:07.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.560+0000 7f3736d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f373815f420 0x7f373815f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49920/0 (socket says 172.21.15.31:49920) 2023-10-06T02:24:07.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.560+0000 7f3736d9d700 1 -- 172.21.15.31:0/3969184390 learned_addr learned my addr 172.21.15.31:0/3969184390 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:07.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.560+0000 7f373659c700 1 --2- 172.21.15.31:0/3969184390 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3720004d50 0x7f373815b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:07.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.560+0000 7f3735d9b700 1 --2- 172.21.15.31:0/3969184390 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f37381559e0 0x7f373815d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:07.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.560+0000 7f3736d9d700 1 -- 172.21.15.31:0/3969184390 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f37381559e0 msgr2=0x7f373815d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:07.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.560+0000 7f3736d9d700 1 --2- 172.21.15.31:0/3969184390 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f37381559e0 0x7f373815d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:07.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.560+0000 7f3736d9d700 1 -- 172.21.15.31:0/3969184390 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3720004d50 msgr2=0x7f373815b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:07.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.560+0000 7f3736d9d700 1 --2- 172.21.15.31:0/3969184390 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3720004d50 0x7f373815b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:07.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.560+0000 7f3736d9d700 1 -- 172.21.15.31:0/3969184390 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f373815b840 con 0x7f373815f420 2023-10-06T02:24:07.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.560+0000 7f373659c700 1 --2- 172.21.15.31:0/3969184390 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3720004d50 0x7f373815b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:24:07.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.560+0000 7f3736d9d700 1 --2- 172.21.15.31:0/3969184390 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f373815f420 0x7f373815f8a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f37380b1440 tx=0x7f373001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:07.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.561+0000 7f37267fc700 1 -- 172.21.15.31:0/3969184390 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f373000eee0 con 0x7f373815f420 2023-10-06T02:24:07.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.561+0000 7f37267fc700 1 -- 172.21.15.31:0/3969184390 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f373001d630 con 0x7f373815f420 2023-10-06T02:24:07.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.561+0000 7f373cdff700 1 -- 172.21.15.31:0/3969184390 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f373815bb20 con 0x7f373815f420 2023-10-06T02:24:07.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.561+0000 7f37267fc700 1 -- 172.21.15.31:0/3969184390 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f373001d9f0 con 0x7f373815f420 2023-10-06T02:24:07.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.561+0000 7f373cdff700 1 -- 172.21.15.31:0/3969184390 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f373815c0e0 con 0x7f373815f420 2023-10-06T02:24:07.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.562+0000 7f373cdff700 1 -- 172.21.15.31:0/3969184390 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f373804c3c0 con 0x7f373815f420 2023-10-06T02:24:07.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.563+0000 7f37267fc700 1 -- 172.21.15.31:0/3969184390 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f37300133e0 con 0x7f373815f420 2023-10-06T02:24:07.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.564+0000 7f37267fc700 1 --2- 172.21.15.31:0/3969184390 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f3718066f30 0x7f37180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:07.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.564+0000 7f37267fc700 1 -- 172.21.15.31:0/3969184390 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f3730085470 con 0x7f373815f420 2023-10-06T02:24:07.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.564+0000 7f373659c700 1 --2- 172.21.15.31:0/3969184390 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f3718066f30 0x7f37180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:07.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.565+0000 7f373659c700 1 --2- 172.21.15.31:0/3969184390 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f3718066f30 0x7f37180693e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3728001b30 tx=0x7f3728009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:07.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.568+0000 7f37267fc700 1 -- 172.21.15.31:0/3969184390 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f37300528b0 con 0x7f373815f420 2023-10-06T02:24:07.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.851+0000 7f373cdff700 1 -- 172.21.15.31:0/3969184390 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f373815c590 con 0x7f373815f420 2023-10-06T02:24:07.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.856+0000 7f37267fc700 1 -- 172.21.15.31:0/3969184390 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f373815c590 con 0x7f373815f420 2023-10-06T02:24:07.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.858+0000 7f373cdff700 1 -- 172.21.15.31:0/3969184390 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f3718066f30 msgr2=0x7f37180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:07.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.858+0000 7f373cdff700 1 --2- 172.21.15.31:0/3969184390 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f3718066f30 0x7f37180693e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3728001b30 tx=0x7f3728009040 comp rx=0 tx=0).stop 2023-10-06T02:24:07.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.858+0000 7f373cdff700 1 -- 172.21.15.31:0/3969184390 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f373815f420 msgr2=0x7f373815f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:07.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.858+0000 7f373cdff700 1 --2- 172.21.15.31:0/3969184390 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f373815f420 0x7f373815f8a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f37380b1440 tx=0x7f373001a000 comp rx=0 tx=0).stop 2023-10-06T02:24:07.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.859+0000 7f373cdff700 1 -- 172.21.15.31:0/3969184390 shutdown_connections 2023-10-06T02:24:07.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.859+0000 7f373cdff700 1 --2- 172.21.15.31:0/3969184390 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f373815f420 0x7f373815f8a0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:07.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.859+0000 7f373cdff700 1 --2- 172.21.15.31:0/3969184390 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3720004d50 0x7f373815b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:07.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.859+0000 7f373cdff700 1 --2- 172.21.15.31:0/3969184390 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f3718066f30 0x7f37180693e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:07.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.859+0000 7f373cdff700 1 --2- 172.21.15.31:0/3969184390 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f37381559e0 0x7f373815d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:07.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.859+0000 7f373cdff700 1 -- 172.21.15.31:0/3969184390 >> 172.21.15.31:0/3969184390 conn(0x7f373806d170 msgr2=0x7f37380af100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:07.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.859+0000 7f373cdff700 1 -- 172.21.15.31:0/3969184390 shutdown_connections 2023-10-06T02:24:07.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:07.859+0000 7f373cdff700 1 -- 172.21.15.31:0/3969184390 wait complete. 2023-10-06T02:24:07.877 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:24:08.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.062+0000 7f688b39d700 1 Processor -- start 2023-10-06T02:24:08.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.063+0000 7f688b39d700 1 -- start start 2023-10-06T02:24:08.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.064+0000 7f688b39d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6884150380 0x7f6884160ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:08.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.064+0000 7f688b39d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f688406a490 con 0x7f688406e0a0 2023-10-06T02:24:08.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.064+0000 7f688b39d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68840c7230 con 0x7f688406f3a0 2023-10-06T02:24:08.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.064+0000 7f688b39d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f688406b930 con 0x7f6884150380 2023-10-06T02:24:08.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.064+0000 7f688a39b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f688406e0a0 0x7f68840c6b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:33002/0 (socket says 172.21.15.31:33002) 2023-10-06T02:24:08.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.064+0000 7f688a39b700 1 -- 172.21.15.31:0/1037179963 learned_addr learned my addr 172.21.15.31:0/1037179963 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:24:08.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.064+0000 7f688ab9c700 1 --2- 172.21.15.31:0/1037179963 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6884150380 0x7f6884160ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:08.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.065+0000 7f6889399700 1 -- 172.21.15.31:0/1037179963 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3099667279 0 0) 0x7f688406a490 con 0x7f688406e0a0 2023-10-06T02:24:08.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.065+0000 7f6889399700 1 -- 172.21.15.31:0/1037179963 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6870003570 con 0x7f688406e0a0 2023-10-06T02:24:08.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.065+0000 7f6889399700 1 -- 172.21.15.31:0/1037179963 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 540242944 0 0) 0x7f68840c7230 con 0x7f688406f3a0 2023-10-06T02:24:08.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.065+0000 7f6889399700 1 -- 172.21.15.31:0/1037179963 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f688406a490 con 0x7f688406f3a0 2023-10-06T02:24:08.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.065+0000 7f688ab9c700 1 -- 172.21.15.31:0/1037179963 >> v1:172.21.15.62:6789/0 conn(0x7f688406f3a0 legacy=0x7f688414fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:08.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.065+0000 7f688ab9c700 1 -- 172.21.15.31:0/1037179963 >> v1:172.21.15.31:6789/0 conn(0x7f688406e0a0 legacy=0x7f68840c6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:08.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.065+0000 7f6889399700 1 -- 172.21.15.31:0/1037179963 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2149523941 0 0) 0x7f6870003570 con 0x7f688406e0a0 2023-10-06T02:24:08.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.065+0000 7f688ab9c700 1 -- 172.21.15.31:0/1037179963 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68840c7490 con 0x7f6884150380 2023-10-06T02:24:08.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.065+0000 7f688ab9c700 1 --2- 172.21.15.31:0/1037179963 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6884150380 0x7f6884160ae0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f687c00afa0 tx=0x7f687c00a890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d565393a7bb51b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:08.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.066+0000 7f6889399700 1 -- 172.21.15.31:0/1037179963 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f687c00f070 con 0x7f6884150380 2023-10-06T02:24:08.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.066+0000 7f6889399700 1 -- 172.21.15.31:0/1037179963 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f687c00a0d0 con 0x7f6884150380 2023-10-06T02:24:08.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.066+0000 7f6889399700 1 -- 172.21.15.31:0/1037179963 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6880001470 con 0x7f6884150380 2023-10-06T02:24:08.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.066+0000 7f688b39d700 1 -- 172.21.15.31:0/1037179963 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6884150380 msgr2=0x7f6884160ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:08.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.066+0000 7f688b39d700 1 --2- 172.21.15.31:0/1037179963 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6884150380 0x7f6884160ae0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f687c00afa0 tx=0x7f687c00a890 comp rx=0 tx=0).stop 2023-10-06T02:24:08.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.066+0000 7f688b39d700 1 -- 172.21.15.31:0/1037179963 shutdown_connections 2023-10-06T02:24:08.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.066+0000 7f688b39d700 1 --2- 172.21.15.31:0/1037179963 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6884150380 0x7f6884160ae0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:08.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.066+0000 7f688b39d700 1 -- 172.21.15.31:0/1037179963 >> 172.21.15.31:0/1037179963 conn(0x7f688406d170 msgr2=0x7f68840cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:08.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.066+0000 7f688b39d700 1 -- 172.21.15.31:0/1037179963 shutdown_connections 2023-10-06T02:24:08.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.066+0000 7f688b39d700 1 -- 172.21.15.31:0/1037179963 wait complete. 2023-10-06T02:24:08.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.068+0000 7f688b39d700 1 Processor -- start 2023-10-06T02:24:08.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.068+0000 7f688b39d700 1 -- start start 2023-10-06T02:24:08.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.069+0000 7f688b39d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f688406e0a0 0x7f6884143120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:08.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.069+0000 7f688b39d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f688406f3a0 0x7f6884144280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:08.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.069+0000 7f688b39d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6884150380 0x7f68841447c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:08.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.069+0000 7f688b39d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6884143660 con 0x7f6884150380 2023-10-06T02:24:08.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.069+0000 7f688b39d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6884146e60 con 0x7f688406e0a0 2023-10-06T02:24:08.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.069+0000 7f688b39d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68840b04e0 con 0x7f688406f3a0 2023-10-06T02:24:08.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.069+0000 7f688ab9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6884150380 0x7f68841447c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:08.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.069+0000 7f688ab9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6884150380 0x7f68841447c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56514/0 (socket says 172.21.15.31:56514) 2023-10-06T02:24:08.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.069+0000 7f688ab9c700 1 -- 172.21.15.31:0/3972912783 learned_addr learned my addr 172.21.15.31:0/3972912783 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:08.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.069+0000 7f688a39b700 1 --2- 172.21.15.31:0/3972912783 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f688406e0a0 0x7f6884143120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:08.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.069+0000 7f6889b9a700 1 --2- 172.21.15.31:0/3972912783 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f688406f3a0 0x7f6884144280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:08.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.069+0000 7f688ab9c700 1 -- 172.21.15.31:0/3972912783 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f688406f3a0 msgr2=0x7f6884144280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:08.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.069+0000 7f688ab9c700 1 --2- 172.21.15.31:0/3972912783 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f688406f3a0 0x7f6884144280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:08.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.069+0000 7f688ab9c700 1 -- 172.21.15.31:0/3972912783 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f688406e0a0 msgr2=0x7f6884143120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:08.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.069+0000 7f688ab9c700 1 --2- 172.21.15.31:0/3972912783 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f688406e0a0 0x7f6884143120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:08.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.069+0000 7f688ab9c700 1 -- 172.21.15.31:0/3972912783 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f687c00ab30 con 0x7f6884150380 2023-10-06T02:24:08.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.070+0000 7f688ab9c700 1 --2- 172.21.15.31:0/3972912783 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6884150380 0x7f68841447c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f687c003000 tx=0x7f687c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:08.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.070+0000 7f687a7fc700 1 -- 172.21.15.31:0/3972912783 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f687c009840 con 0x7f6884150380 2023-10-06T02:24:08.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.070+0000 7f687a7fc700 1 -- 172.21.15.31:0/3972912783 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f687c0099c0 con 0x7f6884150380 2023-10-06T02:24:08.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.070+0000 7f687a7fc700 1 -- 172.21.15.31:0/3972912783 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f687c009d80 con 0x7f6884150380 2023-10-06T02:24:08.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.070+0000 7f688b39d700 1 -- 172.21.15.31:0/3972912783 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68841465c0 con 0x7f6884150380 2023-10-06T02:24:08.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.070+0000 7f688b39d700 1 -- 172.21.15.31:0/3972912783 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6884146b80 con 0x7f6884150380 2023-10-06T02:24:08.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.072+0000 7f688b39d700 1 -- 172.21.15.31:0/3972912783 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68840b0690 con 0x7f6884150380 2023-10-06T02:24:08.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.072+0000 7f687a7fc700 1 -- 172.21.15.31:0/3972912783 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f687c01e640 con 0x7f6884150380 2023-10-06T02:24:08.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.073+0000 7f687a7fc700 1 --2- 172.21.15.31:0/3972912783 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6868066f30 0x7f68680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:08.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.073+0000 7f687a7fc700 1 -- 172.21.15.31:0/3972912783 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f687c085df0 con 0x7f6884150380 2023-10-06T02:24:08.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.073+0000 7f688a39b700 1 --2- 172.21.15.31:0/3972912783 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6868066f30 0x7f68680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:08.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.074+0000 7f688a39b700 1 --2- 172.21.15.31:0/3972912783 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6868066f30 0x7f68680693e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f68800011e0 tx=0x7f6880001620 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:08.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.077+0000 7f687a7fc700 1 -- 172.21.15.31:0/3972912783 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f68840b0690 con 0x7f6884150380 2023-10-06T02:24:08.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.283+0000 7f688b39d700 1 -- 172.21.15.31:0/3972912783 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f68840b02a0 con 0x7f6884150380 2023-10-06T02:24:08.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.284+0000 7f687a7fc700 1 -- 172.21.15.31:0/3972912783 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+13602 (secure 0 0 0) 0x7f687c0531b0 con 0x7f6884150380 2023-10-06T02:24:08.285 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:24:08.285 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":32,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:24:04.275591+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:24:08.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.286+0000 7f688b39d700 1 -- 172.21.15.31:0/3972912783 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6868066f30 msgr2=0x7f68680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:08.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.286+0000 7f688b39d700 1 --2- 172.21.15.31:0/3972912783 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6868066f30 0x7f68680693e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f68800011e0 tx=0x7f6880001620 comp rx=0 tx=0).stop 2023-10-06T02:24:08.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.286+0000 7f688b39d700 1 -- 172.21.15.31:0/3972912783 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6884150380 msgr2=0x7f68841447c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:08.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.286+0000 7f688b39d700 1 --2- 172.21.15.31:0/3972912783 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6884150380 0x7f68841447c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f687c003000 tx=0x7f687c017000 comp rx=0 tx=0).stop 2023-10-06T02:24:08.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.286+0000 7f688b39d700 1 -- 172.21.15.31:0/3972912783 shutdown_connections 2023-10-06T02:24:08.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.286+0000 7f688b39d700 1 --2- 172.21.15.31:0/3972912783 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6884150380 0x7f68841447c0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:08.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.286+0000 7f688b39d700 1 --2- 172.21.15.31:0/3972912783 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f688406e0a0 0x7f6884143120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:08.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.286+0000 7f688b39d700 1 --2- 172.21.15.31:0/3972912783 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6868066f30 0x7f68680693e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:08.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.286+0000 7f688b39d700 1 --2- 172.21.15.31:0/3972912783 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f688406f3a0 0x7f6884144280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:08.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.286+0000 7f688b39d700 1 -- 172.21.15.31:0/3972912783 >> 172.21.15.31:0/3972912783 conn(0x7f688406d170 msgr2=0x7f68840c8ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:08.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.287+0000 7f688b39d700 1 -- 172.21.15.31:0/3972912783 shutdown_connections 2023-10-06T02:24:08.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.287+0000 7f688b39d700 1 -- 172.21.15.31:0/3972912783 wait complete. 2023-10-06T02:24:08.305 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T02:22:53.183425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T02:23:00.046967+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-10-06T02:24:08.305 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T02:24:08.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.493+0000 7fdcd2f9d700 1 Processor -- start 2023-10-06T02:24:08.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.494+0000 7fdcd2f9d700 1 -- start start 2023-10-06T02:24:08.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.495+0000 7fdcd2f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdccc155890 0x7fdccc0c7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:08.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.495+0000 7fdcd2f9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fdccc155c60 0x7fdccc06a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:08.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.495+0000 7fdcd2f9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdccc0c7e70 con 0x7fdccc06de50 2023-10-06T02:24:08.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.495+0000 7fdcd2f9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdccc06b790 con 0x7fdccc155c60 2023-10-06T02:24:08.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.495+0000 7fdcd2f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdccc06a8e0 con 0x7fdccc155890 2023-10-06T02:24:08.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.495+0000 7fdcd179a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fdccc155c60 0x7fdccc06a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:08.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.495+0000 7fdcd179a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fdccc155c60 0x7fdccc06a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56526/0 (socket says 172.21.15.31:56526) 2023-10-06T02:24:08.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.495+0000 7fdcd179a700 1 -- 172.21.15.31:0/4104678287 learned_addr learned my addr 172.21.15.31:0/4104678287 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:08.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.495+0000 7fdcd279c700 1 --1- 172.21.15.31:0/4104678287 >> v1:172.21.15.31:6789/0 conn(0x7fdccc06de50 0x7fdccc159720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:33014/0 (socket says 172.21.15.31:33014) 2023-10-06T02:24:08.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.496+0000 7fdcd1f9b700 1 --2- 172.21.15.31:0/4104678287 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdccc155890 0x7fdccc0c7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:08.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.496+0000 7fdcd179a700 1 -- 172.21.15.31:0/4104678287 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdccc155890 msgr2=0x7fdccc0c7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:08.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.496+0000 7fdcd179a700 1 --2- 172.21.15.31:0/4104678287 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdccc155890 0x7fdccc0c7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:08.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.496+0000 7fdcd179a700 1 -- 172.21.15.31:0/4104678287 >> v1:172.21.15.31:6789/0 conn(0x7fdccc06de50 legacy=0x7fdccc159720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:08.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.496+0000 7fdcd179a700 1 -- 172.21.15.31:0/4104678287 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdccc06ab80 con 0x7fdccc155c60 2023-10-06T02:24:08.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.496+0000 7fdcd179a700 1 --2- 172.21.15.31:0/4104678287 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdccc155c60 0x7fdccc06a3a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc0099b0 tx=0x7fdcbc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28ef710d5c11020b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:08.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.496+0000 7fdcd0f99700 1 -- 172.21.15.31:0/4104678287 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc00b860 con 0x7fdccc155c60 2023-10-06T02:24:08.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.496+0000 7fdcd0f99700 1 -- 172.21.15.31:0/4104678287 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcbc00b9c0 con 0x7fdccc155c60 2023-10-06T02:24:08.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.496+0000 7fdcd0f99700 1 -- 172.21.15.31:0/4104678287 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc00bd60 con 0x7fdccc155c60 2023-10-06T02:24:08.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.497+0000 7fdcd2f9d700 1 -- 172.21.15.31:0/4104678287 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdccc155c60 msgr2=0x7fdccc06a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:08.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.497+0000 7fdcd2f9d700 1 --2- 172.21.15.31:0/4104678287 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdccc155c60 0x7fdccc06a3a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc0099b0 tx=0x7fdcbc009250 comp rx=0 tx=0).stop 2023-10-06T02:24:08.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.497+0000 7fdcd2f9d700 1 -- 172.21.15.31:0/4104678287 shutdown_connections 2023-10-06T02:24:08.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.497+0000 7fdcd2f9d700 1 --2- 172.21.15.31:0/4104678287 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdccc155890 0x7fdccc0c7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:08.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.497+0000 7fdcd2f9d700 1 --2- 172.21.15.31:0/4104678287 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdccc155c60 0x7fdccc06a3a0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:08.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.497+0000 7fdcd2f9d700 1 -- 172.21.15.31:0/4104678287 >> 172.21.15.31:0/4104678287 conn(0x7fdccc06d110 msgr2=0x7fdccc0cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:08.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.497+0000 7fdcd2f9d700 1 -- 172.21.15.31:0/4104678287 shutdown_connections 2023-10-06T02:24:08.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.497+0000 7fdcd2f9d700 1 -- 172.21.15.31:0/4104678287 wait complete. 2023-10-06T02:24:08.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.498+0000 7fdcd2f9d700 1 Processor -- start 2023-10-06T02:24:08.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.499+0000 7fdcd2f9d700 1 -- start start 2023-10-06T02:24:08.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.499+0000 7fdcd2f9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdccc06de50 0x7fdccc156160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:08.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.499+0000 7fdcd2f9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdccc155890 0x7fdccc149b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:08.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.500+0000 7fdcd2f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdccc155c60 0x7fdccc1541b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:08.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.500+0000 7fdcd2f9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdccc0c3a10 con 0x7fdccc06de50 2023-10-06T02:24:08.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.500+0000 7fdcd2f9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdccc0c3ba0 con 0x7fdccc155890 2023-10-06T02:24:08.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.500+0000 7fdcd2f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdccc1566a0 con 0x7fdccc155c60 2023-10-06T02:24:08.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.500+0000 7fdcd1f9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdccc06de50 0x7fdccc156160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:08.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.500+0000 7fdcd1f9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdccc06de50 0x7fdccc156160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56540/0 (socket says 172.21.15.31:56540) 2023-10-06T02:24:08.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.500+0000 7fdcd1f9b700 1 -- 172.21.15.31:0/2623177561 learned_addr learned my addr 172.21.15.31:0/2623177561 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:08.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.500+0000 7fdcd179a700 1 --2- 172.21.15.31:0/2623177561 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdccc155890 0x7fdccc149b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:08.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.500+0000 7fdcd279c700 1 --2- 172.21.15.31:0/2623177561 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdccc155c60 0x7fdccc1541b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:08.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.500+0000 7fdcd1f9b700 1 -- 172.21.15.31:0/2623177561 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdccc155c60 msgr2=0x7fdccc1541b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:08.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.500+0000 7fdcd1f9b700 1 --2- 172.21.15.31:0/2623177561 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdccc155c60 0x7fdccc1541b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:08.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.500+0000 7fdcd1f9b700 1 -- 172.21.15.31:0/2623177561 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdccc155890 msgr2=0x7fdccc149b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:08.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.500+0000 7fdcd1f9b700 1 --2- 172.21.15.31:0/2623177561 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdccc155890 0x7fdccc149b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:08.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.500+0000 7fdcd1f9b700 1 -- 172.21.15.31:0/2623177561 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcbc0094f0 con 0x7fdccc06de50 2023-10-06T02:24:08.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.500+0000 7fdcd179a700 1 --2- 172.21.15.31:0/2623177561 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdccc155890 0x7fdccc149b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:24:08.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.500+0000 7fdcd279c700 1 --2- 172.21.15.31:0/2623177561 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdccc155c60 0x7fdccc1541b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:24:08.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.500+0000 7fdcd1f9b700 1 --2- 172.21.15.31:0/2623177561 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdccc06de50 0x7fdccc156160 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fdcc800d8c0 tx=0x7fdcc8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:08.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.501+0000 7fdcc1ffb700 1 -- 172.21.15.31:0/2623177561 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc800de60 con 0x7fdccc06de50 2023-10-06T02:24:08.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.501+0000 7fdcc1ffb700 1 -- 172.21.15.31:0/2623177561 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcc800b780 con 0x7fdccc06de50 2023-10-06T02:24:08.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.501+0000 7fdcc1ffb700 1 -- 172.21.15.31:0/2623177561 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc800bb20 con 0x7fdccc06de50 2023-10-06T02:24:08.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.501+0000 7fdcd2f9d700 1 -- 172.21.15.31:0/2623177561 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdccc156920 con 0x7fdccc06de50 2023-10-06T02:24:08.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.501+0000 7fdcd2f9d700 1 -- 172.21.15.31:0/2623177561 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdccc156ee0 con 0x7fdccc06de50 2023-10-06T02:24:08.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.502+0000 7fdcd2f9d700 1 -- 172.21.15.31:0/2623177561 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdccc04c3c0 con 0x7fdccc06de50 2023-10-06T02:24:08.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.503+0000 7fdcc1ffb700 1 -- 172.21.15.31:0/2623177561 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdcc801e460 con 0x7fdccc06de50 2023-10-06T02:24:08.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.504+0000 7fdcc1ffb700 1 --2- 172.21.15.31:0/2623177561 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fdcb0066f30 0x7fdcb00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:08.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.504+0000 7fdcc1ffb700 1 -- 172.21.15.31:0/2623177561 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fdcc80849e0 con 0x7fdccc06de50 2023-10-06T02:24:08.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.504+0000 7fdcd179a700 1 --2- 172.21.15.31:0/2623177561 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fdcb0066f30 0x7fdcb00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:08.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.505+0000 7fdcd179a700 1 --2- 172.21.15.31:0/2623177561 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fdcb0066f30 0x7fdcb00693e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc0099b0 tx=0x7fdcbc017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:08.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.508+0000 7fdcc1ffb700 1 -- 172.21.15.31:0/2623177561 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdcc8051e40 con 0x7fdccc06de50 2023-10-06T02:24:08.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.708+0000 7fdcd2f9d700 1 -- 172.21.15.31:0/2623177561 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdccc0c75b0 con 0x7fdccc06de50 2023-10-06T02:24:08.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.709+0000 7fdcc1ffb700 1 -- 172.21.15.31:0/2623177561 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v32) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdcc8055460 con 0x7fdccc06de50 2023-10-06T02:24:08.711 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-10-06T02:24:08.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.711+0000 7fdcd2f9d700 1 -- 172.21.15.31:0/2623177561 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fdcb0066f30 msgr2=0x7fdcb00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:08.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.711+0000 7fdcd2f9d700 1 --2- 172.21.15.31:0/2623177561 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fdcb0066f30 0x7fdcb00693e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc0099b0 tx=0x7fdcbc017000 comp rx=0 tx=0).stop 2023-10-06T02:24:08.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.711+0000 7fdcd2f9d700 1 -- 172.21.15.31:0/2623177561 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdccc06de50 msgr2=0x7fdccc156160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:08.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.711+0000 7fdcd2f9d700 1 --2- 172.21.15.31:0/2623177561 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdccc06de50 0x7fdccc156160 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fdcc800d8c0 tx=0x7fdcc8008040 comp rx=0 tx=0).stop 2023-10-06T02:24:08.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.711+0000 7fdcd2f9d700 1 -- 172.21.15.31:0/2623177561 shutdown_connections 2023-10-06T02:24:08.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.711+0000 7fdcd2f9d700 1 --2- 172.21.15.31:0/2623177561 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fdcb0066f30 0x7fdcb00693e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:08.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.711+0000 7fdcd2f9d700 1 --2- 172.21.15.31:0/2623177561 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdccc155c60 0x7fdccc1541b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:08.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.711+0000 7fdcd2f9d700 1 --2- 172.21.15.31:0/2623177561 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdccc06de50 0x7fdccc156160 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:08.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.711+0000 7fdcd2f9d700 1 --2- 172.21.15.31:0/2623177561 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdccc155890 0x7fdccc149b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:08.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.711+0000 7fdcd2f9d700 1 -- 172.21.15.31:0/2623177561 >> 172.21.15.31:0/2623177561 conn(0x7fdccc06d110 msgr2=0x7fdccc0c20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:08.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.711+0000 7fdcd2f9d700 1 -- 172.21.15.31:0/2623177561 shutdown_connections 2023-10-06T02:24:08.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.712+0000 7fdcd2f9d700 1 -- 172.21.15.31:0/2623177561 wait complete. 2023-10-06T02:24:08.730 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T02:24:08.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.917+0000 7f34851ff700 1 Processor -- start 2023-10-06T02:24:08.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.917+0000 7f34851ff700 1 -- start start 2023-10-06T02:24:08.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.919+0000 7f34851ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f34801576d0 0x7f34800ae590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:08.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.919+0000 7f34851ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f34800aead0 0x7f34800b14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:08.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.919+0000 7f34851ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34800cc060 con 0x7f34800b1a10 2023-10-06T02:24:08.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.919+0000 7f34851ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34800b1e90 con 0x7f34800aead0 2023-10-06T02:24:08.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.919+0000 7f34851ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34800cb450 con 0x7f34801576d0 2023-10-06T02:24:08.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.919+0000 7f347e59c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f34800aead0 0x7f34800b14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:08.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.919+0000 7f347e59c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f34800aead0 0x7f34800b14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56554/0 (socket says 172.21.15.31:56554) 2023-10-06T02:24:08.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.919+0000 7f347e59c700 1 -- 172.21.15.31:0/3251883190 learned_addr learned my addr 172.21.15.31:0/3251883190 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:08.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.919+0000 7f347ed9d700 1 --2- 172.21.15.31:0/3251883190 >> v2:172.21.15.62:3300/0 conn(0x7f34801576d0 0x7f34800ae590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:08.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.919+0000 7f347e59c700 1 -- 172.21.15.31:0/3251883190 >> v2:172.21.15.62:3300/0 conn(0x7f34801576d0 msgr2=0x7f34800ae590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:08.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.919+0000 7f347e59c700 1 --2- 172.21.15.31:0/3251883190 >> v2:172.21.15.62:3300/0 conn(0x7f34801576d0 0x7f34800ae590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:08.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.919+0000 7f347e59c700 1 -- 172.21.15.31:0/3251883190 >> v1:172.21.15.62:6789/0 conn(0x7f34800b1a10 legacy=0x7f34800abdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:08.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.920+0000 7f347e59c700 1 -- 172.21.15.31:0/3251883190 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34800cb730 con 0x7f34800aead0 2023-10-06T02:24:08.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.920+0000 7f347e59c700 1 --2- 172.21.15.31:0/3251883190 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34800aead0 0x7f34800b14d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f34800633b0 tx=0x7f3474009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=397fcdd585467e7e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:08.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.920+0000 7f347dd9b700 1 -- 172.21.15.31:0/3251883190 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3474009e70 con 0x7f34800aead0 2023-10-06T02:24:08.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.920+0000 7f347dd9b700 1 -- 172.21.15.31:0/3251883190 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f347400b860 con 0x7f34800aead0 2023-10-06T02:24:08.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.920+0000 7f347dd9b700 1 -- 172.21.15.31:0/3251883190 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f347400bc00 con 0x7f34800aead0 2023-10-06T02:24:08.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.921+0000 7f34851ff700 1 -- 172.21.15.31:0/3251883190 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34800aead0 msgr2=0x7f34800b14d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:08.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.921+0000 7f34851ff700 1 --2- 172.21.15.31:0/3251883190 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34800aead0 0x7f34800b14d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f34800633b0 tx=0x7f3474009250 comp rx=0 tx=0).stop 2023-10-06T02:24:08.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.921+0000 7f34851ff700 1 -- 172.21.15.31:0/3251883190 shutdown_connections 2023-10-06T02:24:08.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.921+0000 7f34851ff700 1 --2- 172.21.15.31:0/3251883190 >> v2:172.21.15.62:3300/0 conn(0x7f34801576d0 0x7f34800ae590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:08.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.921+0000 7f34851ff700 1 --2- 172.21.15.31:0/3251883190 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34800aead0 0x7f34800b14d0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:08.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.921+0000 7f34851ff700 1 -- 172.21.15.31:0/3251883190 >> 172.21.15.31:0/3251883190 conn(0x7f348006d170 msgr2=0x7f34800cc400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:08.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.921+0000 7f34851ff700 1 -- 172.21.15.31:0/3251883190 shutdown_connections 2023-10-06T02:24:08.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.921+0000 7f34851ff700 1 -- 172.21.15.31:0/3251883190 wait complete. 2023-10-06T02:24:08.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.922+0000 7f34851ff700 1 Processor -- start 2023-10-06T02:24:08.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.922+0000 7f34851ff700 1 -- start start 2023-10-06T02:24:08.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.923+0000 7f34851ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f34800aead0 0x7f348014af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:08.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.923+0000 7f34851ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34800b1a10 0x7f348014ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:08.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.923+0000 7f34851ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34801576d0 0x7f348014f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:08.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.923+0000 7f34851ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3480151610 con 0x7f34801576d0 2023-10-06T02:24:08.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.923+0000 7f34851ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f348014cc30 con 0x7f34800b1a10 2023-10-06T02:24:08.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.923+0000 7f34851ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f348014c0a0 con 0x7f34800aead0 2023-10-06T02:24:08.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.923+0000 7f347f59e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34801576d0 0x7f348014f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:08.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.923+0000 7f347f59e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34801576d0 0x7f348014f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56568/0 (socket says 172.21.15.31:56568) 2023-10-06T02:24:08.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.923+0000 7f347f59e700 1 -- 172.21.15.31:0/925301957 learned_addr learned my addr 172.21.15.31:0/925301957 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:08.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.924+0000 7f347ed9d700 1 --2- 172.21.15.31:0/925301957 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f34800aead0 0x7f348014af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:08.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.924+0000 7f347e59c700 1 --2- 172.21.15.31:0/925301957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34800b1a10 0x7f348014ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:08.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.924+0000 7f347f59e700 1 -- 172.21.15.31:0/925301957 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f34800aead0 msgr2=0x7f348014af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:08.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.924+0000 7f347f59e700 1 --2- 172.21.15.31:0/925301957 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f34800aead0 0x7f348014af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:08.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.924+0000 7f347f59e700 1 -- 172.21.15.31:0/925301957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34800b1a10 msgr2=0x7f348014ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:08.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.924+0000 7f347f59e700 1 --2- 172.21.15.31:0/925301957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34800b1a10 0x7f348014ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:08.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.924+0000 7f347f59e700 1 -- 172.21.15.31:0/925301957 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34740094f0 con 0x7f34801576d0 2023-10-06T02:24:08.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.924+0000 7f347f59e700 1 --2- 172.21.15.31:0/925301957 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34801576d0 0x7f348014f1b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f347800c8e0 tx=0x7f3478001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:08.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.924+0000 7f3466ffd700 1 -- 172.21.15.31:0/925301957 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f347800ce80 con 0x7f34801576d0 2023-10-06T02:24:08.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.924+0000 7f3466ffd700 1 -- 172.21.15.31:0/925301957 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f347800a7e0 con 0x7f34801576d0 2023-10-06T02:24:08.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.924+0000 7f3466ffd700 1 -- 172.21.15.31:0/925301957 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f347800ab80 con 0x7f34801576d0 2023-10-06T02:24:08.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.924+0000 7f34851ff700 1 -- 172.21.15.31:0/925301957 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f348014b4c0 con 0x7f34801576d0 2023-10-06T02:24:08.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.925+0000 7f34851ff700 1 -- 172.21.15.31:0/925301957 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f348014c5e0 con 0x7f34801576d0 2023-10-06T02:24:08.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.925+0000 7f34851ff700 1 -- 172.21.15.31:0/925301957 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f348014cdc0 con 0x7f34801576d0 2023-10-06T02:24:08.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.926+0000 7f3466ffd700 1 -- 172.21.15.31:0/925301957 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3478012420 con 0x7f34801576d0 2023-10-06T02:24:08.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.928+0000 7f3466ffd700 1 --2- 172.21.15.31:0/925301957 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f346c066f30 0x7f346c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:08.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.928+0000 7f3466ffd700 1 -- 172.21.15.31:0/925301957 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f3478084630 con 0x7f34801576d0 2023-10-06T02:24:08.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.928+0000 7f347ed9d700 1 --2- 172.21.15.31:0/925301957 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f346c066f30 0x7f346c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:08.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.928+0000 7f347ed9d700 1 --2- 172.21.15.31:0/925301957 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f346c066f30 0x7f346c0693e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3470009ca0 tx=0x7f3470008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:08.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:08.931+0000 7f3466ffd700 1 -- 172.21.15.31:0/925301957 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3478051a90 con 0x7f34801576d0 2023-10-06T02:24:09.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.133+0000 7f34851ff700 1 -- 172.21.15.31:0/925301957 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f348004c3c0 con 0x7f34801576d0 2023-10-06T02:24:09.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.134+0000 7f3466ffd700 1 -- 172.21.15.31:0/925301957 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v32) v1 ==== 92+0+10 (secure 0 0 0) 0x7f34780550b0 con 0x7f34801576d0 2023-10-06T02:24:09.135 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 8 2023-10-06T02:24:09.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.136+0000 7f34851ff700 1 -- 172.21.15.31:0/925301957 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f346c066f30 msgr2=0x7f346c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:09.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.136+0000 7f34851ff700 1 --2- 172.21.15.31:0/925301957 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f346c066f30 0x7f346c0693e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3470009ca0 tx=0x7f3470008040 comp rx=0 tx=0).stop 2023-10-06T02:24:09.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.136+0000 7f34851ff700 1 -- 172.21.15.31:0/925301957 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34801576d0 msgr2=0x7f348014f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:09.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.136+0000 7f34851ff700 1 --2- 172.21.15.31:0/925301957 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34801576d0 0x7f348014f1b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f347800c8e0 tx=0x7f3478001710 comp rx=0 tx=0).stop 2023-10-06T02:24:09.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.136+0000 7f34851ff700 1 -- 172.21.15.31:0/925301957 shutdown_connections 2023-10-06T02:24:09.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.136+0000 7f34851ff700 1 --2- 172.21.15.31:0/925301957 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34801576d0 0x7f348014f1b0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:09.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.136+0000 7f34851ff700 1 --2- 172.21.15.31:0/925301957 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f346c066f30 0x7f346c0693e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:09.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.136+0000 7f34851ff700 1 --2- 172.21.15.31:0/925301957 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f34800aead0 0x7f348014af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:09.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.136+0000 7f34851ff700 1 --2- 172.21.15.31:0/925301957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34800b1a10 0x7f348014ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:09.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.136+0000 7f34851ff700 1 -- 172.21.15.31:0/925301957 >> 172.21.15.31:0/925301957 conn(0x7f348006d170 msgr2=0x7f34800afd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:09.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.136+0000 7f34851ff700 1 -- 172.21.15.31:0/925301957 shutdown_connections 2023-10-06T02:24:09.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.136+0000 7f34851ff700 1 -- 172.21.15.31:0/925301957 wait complete. 2023-10-06T02:24:09.155 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T02:24:09.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.341+0000 7f02dbfff700 1 Processor -- start 2023-10-06T02:24:09.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.341+0000 7f02dbfff700 1 -- start start 2023-10-06T02:24:09.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.342+0000 7f02dbfff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f02dc06e0a0 0x7f02dc0c6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:09.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.342+0000 7f02dbfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f02dc06f3a0 0x7f02dc14fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:09.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.342+0000 7f02dbfff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f02dc06e470 0x7f02dc1501b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:09.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.342+0000 7f02dbfff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02dc06a490 con 0x7f02dc06f3a0 2023-10-06T02:24:09.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.342+0000 7f02dbfff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02dc0c7490 con 0x7f02dc06e470 2023-10-06T02:24:09.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.342+0000 7f02dbfff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02dc0c7770 con 0x7f02dc06e0a0 2023-10-06T02:24:09.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.342+0000 7f02da7fc700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f02dc06f3a0 0x7f02dc14fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:09.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.342+0000 7f02da7fc700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f02dc06f3a0 0x7f02dc14fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56580/0 (socket says 172.21.15.31:56580) 2023-10-06T02:24:09.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.342+0000 7f02da7fc700 1 -- 172.21.15.31:0/2293070009 learned_addr learned my addr 172.21.15.31:0/2293070009 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:09.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.343+0000 7f02db7fe700 1 --2- 172.21.15.31:0/2293070009 >> v2:172.21.15.62:3300/0 conn(0x7f02dc06e470 0x7f02dc1501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:09.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.343+0000 7f02daffd700 1 --2- 172.21.15.31:0/2293070009 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f02dc06e0a0 0x7f02dc0c6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:09.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.343+0000 7f02da7fc700 1 -- 172.21.15.31:0/2293070009 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f02dc06e0a0 msgr2=0x7f02dc0c6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:09.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.343+0000 7f02da7fc700 1 --2- 172.21.15.31:0/2293070009 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f02dc06e0a0 0x7f02dc0c6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:09.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.343+0000 7f02da7fc700 1 -- 172.21.15.31:0/2293070009 >> v2:172.21.15.62:3300/0 conn(0x7f02dc06e470 msgr2=0x7f02dc1501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:09.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.343+0000 7f02da7fc700 1 --2- 172.21.15.31:0/2293070009 >> v2:172.21.15.62:3300/0 conn(0x7f02dc06e470 0x7f02dc1501b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:09.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.343+0000 7f02da7fc700 1 -- 172.21.15.31:0/2293070009 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02dc0c7a50 con 0x7f02dc06f3a0 2023-10-06T02:24:09.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.343+0000 7f02db7fe700 1 --2- 172.21.15.31:0/2293070009 >> v2:172.21.15.62:3300/0 conn(0x7f02dc06e470 0x7f02dc1501b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:24:09.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.343+0000 7f02daffd700 1 --2- 172.21.15.31:0/2293070009 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f02dc06e0a0 0x7f02dc0c6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:24:09.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.343+0000 7f02da7fc700 1 --2- 172.21.15.31:0/2293070009 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02dc06f3a0 0x7f02dc14fc70 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f02dc069900 tx=0x7f02d0009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3539e75a045ba31 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:09.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.343+0000 7f02d9ffb700 1 -- 172.21.15.31:0/2293070009 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d0005c10 con 0x7f02dc06f3a0 2023-10-06T02:24:09.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.343+0000 7f02d9ffb700 1 -- 172.21.15.31:0/2293070009 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02d0005d70 con 0x7f02dc06f3a0 2023-10-06T02:24:09.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.344+0000 7f02d9ffb700 1 -- 172.21.15.31:0/2293070009 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d000b880 con 0x7f02dc06f3a0 2023-10-06T02:24:09.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.344+0000 7f02dbfff700 1 -- 172.21.15.31:0/2293070009 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02dc06f3a0 msgr2=0x7f02dc14fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:09.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.344+0000 7f02dbfff700 1 --2- 172.21.15.31:0/2293070009 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02dc06f3a0 0x7f02dc14fc70 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f02dc069900 tx=0x7f02d0009220 comp rx=0 tx=0).stop 2023-10-06T02:24:09.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.344+0000 7f02dbfff700 1 -- 172.21.15.31:0/2293070009 shutdown_connections 2023-10-06T02:24:09.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.344+0000 7f02dbfff700 1 --2- 172.21.15.31:0/2293070009 >> v2:172.21.15.62:3300/0 conn(0x7f02dc06e470 0x7f02dc1501b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:09.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.344+0000 7f02dbfff700 1 --2- 172.21.15.31:0/2293070009 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f02dc06e0a0 0x7f02dc0c6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:09.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.344+0000 7f02dbfff700 1 --2- 172.21.15.31:0/2293070009 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02dc06f3a0 0x7f02dc14fc70 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:09.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.344+0000 7f02dbfff700 1 -- 172.21.15.31:0/2293070009 >> 172.21.15.31:0/2293070009 conn(0x7f02dc06d170 msgr2=0x7f02dc0cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:09.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.344+0000 7f02dbfff700 1 -- 172.21.15.31:0/2293070009 shutdown_connections 2023-10-06T02:24:09.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.344+0000 7f02dbfff700 1 -- 172.21.15.31:0/2293070009 wait complete. 2023-10-06T02:24:09.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.346+0000 7f02dbfff700 1 Processor -- start 2023-10-06T02:24:09.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.346+0000 7f02dbfff700 1 -- start start 2023-10-06T02:24:09.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.347+0000 7f02dbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02dc06e0a0 0x7f02dc1561b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:09.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.347+0000 7f02dbfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02dc06e470 0x7f02dc157590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:09.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.347+0000 7f02dbfff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f02dc06f3a0 0x7f02dc0aef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:09.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.347+0000 7f02dbfff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02dc0af4c0 con 0x7f02dc06e0a0 2023-10-06T02:24:09.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.347+0000 7f02dbfff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02dc156790 con 0x7f02dc06e470 2023-10-06T02:24:09.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.347+0000 7f02dbfff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02dc156a70 con 0x7f02dc06f3a0 2023-10-06T02:24:09.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.347+0000 7f02daffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02dc06e0a0 0x7f02dc1561b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:09.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.347+0000 7f02daffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02dc06e0a0 0x7f02dc1561b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56584/0 (socket says 172.21.15.31:56584) 2023-10-06T02:24:09.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.347+0000 7f02daffd700 1 -- 172.21.15.31:0/3028294516 learned_addr learned my addr 172.21.15.31:0/3028294516 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:09.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.347+0000 7f02daffd700 1 -- 172.21.15.31:0/3028294516 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f02dc06f3a0 msgr2=0x7f02dc0aef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:09.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.347+0000 7f02daffd700 1 --2- 172.21.15.31:0/3028294516 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f02dc06f3a0 0x7f02dc0aef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:09.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.347+0000 7f02da7fc700 1 --2- 172.21.15.31:0/3028294516 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02dc06e470 0x7f02dc157590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:09.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.347+0000 7f02daffd700 1 -- 172.21.15.31:0/3028294516 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02dc06e470 msgr2=0x7f02dc157590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:09.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.347+0000 7f02daffd700 1 --2- 172.21.15.31:0/3028294516 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02dc06e470 0x7f02dc157590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:09.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.347+0000 7f02daffd700 1 -- 172.21.15.31:0/3028294516 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02d0009480 con 0x7f02dc06e0a0 2023-10-06T02:24:09.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.347+0000 7f02da7fc700 1 --2- 172.21.15.31:0/3028294516 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02dc06e470 0x7f02dc157590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:24:09.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.348+0000 7f02daffd700 1 --2- 172.21.15.31:0/3028294516 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02dc06e0a0 0x7f02dc1561b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f02cc00e8d0 tx=0x7f02cc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:09.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.348+0000 7f02c2ffd700 1 -- 172.21.15.31:0/3028294516 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02cc00ee70 con 0x7f02dc06e0a0 2023-10-06T02:24:09.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.348+0000 7f02c2ffd700 1 -- 172.21.15.31:0/3028294516 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02cc00c5e0 con 0x7f02dc06e0a0 2023-10-06T02:24:09.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.348+0000 7f02dbfff700 1 -- 172.21.15.31:0/3028294516 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02dc156d50 con 0x7f02dc06e0a0 2023-10-06T02:24:09.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.348+0000 7f02c2ffd700 1 -- 172.21.15.31:0/3028294516 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02cc00c980 con 0x7f02dc06e0a0 2023-10-06T02:24:09.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.348+0000 7f02dbfff700 1 -- 172.21.15.31:0/3028294516 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02dc16a020 con 0x7f02dc06e0a0 2023-10-06T02:24:09.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.349+0000 7f02dbfff700 1 -- 172.21.15.31:0/3028294516 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02dc16a3d0 con 0x7f02dc06e0a0 2023-10-06T02:24:09.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.350+0000 7f02c2ffd700 1 -- 172.21.15.31:0/3028294516 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f02cc013410 con 0x7f02dc06e0a0 2023-10-06T02:24:09.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.351+0000 7f02c2ffd700 1 --2- 172.21.15.31:0/3028294516 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f02c8066f30 0x7f02c80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:09.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.351+0000 7f02c2ffd700 1 -- 172.21.15.31:0/3028294516 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f02cc0841d0 con 0x7f02dc06e0a0 2023-10-06T02:24:09.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.351+0000 7f02da7fc700 1 --2- 172.21.15.31:0/3028294516 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f02c8066f30 0x7f02c80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:09.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.351+0000 7f02da7fc700 1 --2- 172.21.15.31:0/3028294516 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f02c8066f30 0x7f02c80693e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f02dc06f770 tx=0x7f02d0016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:09.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.355+0000 7f02c2ffd700 1 -- 172.21.15.31:0/3028294516 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f02cc051590 con 0x7f02dc06e0a0 2023-10-06T02:24:09.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.557+0000 7f02dbfff700 1 -- 172.21.15.31:0/3028294516 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f02dc04c3c0 con 0x7f02dc06e0a0 2023-10-06T02:24:09.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.558+0000 7f02c2ffd700 1 -- 172.21.15.31:0/3028294516 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v32) v1 ==== 84+0+126 (secure 0 0 0) 0x7f02cc054bb0 con 0x7f02dc06e0a0 2023-10-06T02:24:09.559 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:24:09.559 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2023-10-06T02:24:09.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:24:09.560 INFO:teuthology.orchestra.run.smithi031.stdout: "poolnum": 1, 2023-10-06T02:24:09.560 INFO:teuthology.orchestra.run.smithi031.stdout: "poolname": ".mgr" 2023-10-06T02:24:09.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:24:09.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:24:09.560 INFO:teuthology.orchestra.run.smithi031.stdout: "poolnum": 2, 2023-10-06T02:24:09.561 INFO:teuthology.orchestra.run.smithi031.stdout: "poolname": "rbd" 2023-10-06T02:24:09.561 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:24:09.561 INFO:teuthology.orchestra.run.smithi031.stdout:] 2023-10-06T02:24:09.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.560+0000 7f02dbfff700 1 -- 172.21.15.31:0/3028294516 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f02c8066f30 msgr2=0x7f02c80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:09.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.560+0000 7f02dbfff700 1 --2- 172.21.15.31:0/3028294516 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f02c8066f30 0x7f02c80693e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f02dc06f770 tx=0x7f02d0016000 comp rx=0 tx=0).stop 2023-10-06T02:24:09.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.560+0000 7f02dbfff700 1 -- 172.21.15.31:0/3028294516 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02dc06e0a0 msgr2=0x7f02dc1561b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:09.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.560+0000 7f02dbfff700 1 --2- 172.21.15.31:0/3028294516 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02dc06e0a0 0x7f02dc1561b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f02cc00e8d0 tx=0x7f02cc008040 comp rx=0 tx=0).stop 2023-10-06T02:24:09.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.560+0000 7f02dbfff700 1 -- 172.21.15.31:0/3028294516 shutdown_connections 2023-10-06T02:24:09.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.560+0000 7f02dbfff700 1 --2- 172.21.15.31:0/3028294516 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f02c8066f30 0x7f02c80693e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:09.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.560+0000 7f02dbfff700 1 --2- 172.21.15.31:0/3028294516 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f02dc06f3a0 0x7f02dc0aef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:09.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.560+0000 7f02dbfff700 1 --2- 172.21.15.31:0/3028294516 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02dc06e0a0 0x7f02dc1561b0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:09.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.560+0000 7f02dbfff700 1 --2- 172.21.15.31:0/3028294516 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02dc06e470 0x7f02dc157590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:09.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.560+0000 7f02dbfff700 1 -- 172.21.15.31:0/3028294516 >> 172.21.15.31:0/3028294516 conn(0x7f02dc06d170 msgr2=0x7f02dc0c9d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:09.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.560+0000 7f02dbfff700 1 -- 172.21.15.31:0/3028294516 shutdown_connections 2023-10-06T02:24:09.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.560+0000 7f02dbfff700 1 -- 172.21.15.31:0/3028294516 wait complete. 2023-10-06T02:24:09.579 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T02:24:09.579 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T02:24:09.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.768+0000 7f682bfff700 1 Processor -- start 2023-10-06T02:24:09.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.769+0000 7f682bfff700 1 -- start start 2023-10-06T02:24:09.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.770+0000 7f682bfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f682c153a30 0x7f682c0c0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:09.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.770+0000 7f682bfff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f682c0c2200 con 0x7f682c153660 2023-10-06T02:24:09.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.770+0000 7f682bfff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f682c0c15f0 con 0x7f682c0c9090 2023-10-06T02:24:09.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.770+0000 7f682bfff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f682c0c18d0 con 0x7f682c153a30 2023-10-06T02:24:09.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.770+0000 7f682a7fc700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f682c153a30 0x7f682c0c0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:09.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.770+0000 7f682affd700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f682c153660 0x7f682c0c8980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:33018/0 (socket says 172.21.15.31:33018) 2023-10-06T02:24:09.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.770+0000 7f682affd700 1 -- 172.21.15.31:0/3177454898 learned_addr learned my addr 172.21.15.31:0/3177454898 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:24:09.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.770+0000 7f682a7fc700 1 --2- 172.21.15.31:0/3177454898 >> v2:172.21.15.31:3300/0 conn(0x7f682c153a30 0x7f682c0c0ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56588/0 (socket says 172.21.15.31:56588) 2023-10-06T02:24:09.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.771+0000 7f682a7fc700 1 -- 172.21.15.31:0/3177454898 >> v1:172.21.15.62:6789/0 conn(0x7f682c0c9090 legacy=0x7f682c06aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:09.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.771+0000 7f682a7fc700 1 -- 172.21.15.31:0/3177454898 >> v1:172.21.15.31:6789/0 conn(0x7f682c153660 legacy=0x7f682c0c8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:09.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.771+0000 7f682a7fc700 1 -- 172.21.15.31:0/3177454898 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f682c0c1bb0 con 0x7f682c153a30 2023-10-06T02:24:09.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.771+0000 7f682a7fc700 1 --2- 172.21.15.31:0/3177454898 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f682c153a30 0x7f682c0c0ef0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f682000bb90 tx=0x7f682000b430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2e1be74b9b9b180 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:09.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.771+0000 7f6829ffb700 1 -- 172.21.15.31:0/3177454898 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f682000dba0 con 0x7f682c153a30 2023-10-06T02:24:09.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.771+0000 7f6829ffb700 1 -- 172.21.15.31:0/3177454898 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f682000dd00 con 0x7f682c153a30 2023-10-06T02:24:09.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.771+0000 7f6829ffb700 1 -- 172.21.15.31:0/3177454898 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68200196b0 con 0x7f682c153a30 2023-10-06T02:24:09.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.772+0000 7f682bfff700 1 -- 172.21.15.31:0/3177454898 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f682c153a30 msgr2=0x7f682c0c0ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:09.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.772+0000 7f682bfff700 1 --2- 172.21.15.31:0/3177454898 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f682c153a30 0x7f682c0c0ef0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f682000bb90 tx=0x7f682000b430 comp rx=0 tx=0).stop 2023-10-06T02:24:09.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.772+0000 7f682bfff700 1 -- 172.21.15.31:0/3177454898 shutdown_connections 2023-10-06T02:24:09.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.772+0000 7f682bfff700 1 --2- 172.21.15.31:0/3177454898 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f682c153a30 0x7f682c0c0ef0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:09.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.772+0000 7f682bfff700 1 -- 172.21.15.31:0/3177454898 >> 172.21.15.31:0/3177454898 conn(0x7f682c06c6a0 msgr2=0x7f682c06caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:09.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.772+0000 7f682bfff700 1 -- 172.21.15.31:0/3177454898 shutdown_connections 2023-10-06T02:24:09.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.772+0000 7f682bfff700 1 -- 172.21.15.31:0/3177454898 wait complete. 2023-10-06T02:24:09.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.773+0000 7f682bfff700 1 Processor -- start 2023-10-06T02:24:09.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.774+0000 7f682bfff700 1 -- start start 2023-10-06T02:24:09.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.774+0000 7f682bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f682c0c9090 0x7f682c14c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:09.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.774+0000 7f682bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f682c153660 0x7f682c14bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:09.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.774+0000 7f682bfff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f682c153a30 0x7f682c154a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:09.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.774+0000 7f682bfff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f682c14c3e0 con 0x7f682c0c9090 2023-10-06T02:24:09.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.774+0000 7f682bfff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f682c154600 con 0x7f682c153660 2023-10-06T02:24:09.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.774+0000 7f682bfff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f682c153ff0 con 0x7f682c153a30 2023-10-06T02:24:09.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.775+0000 7f682affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f682c0c9090 0x7f682c14c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:09.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.775+0000 7f682affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f682c0c9090 0x7f682c14c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56596/0 (socket says 172.21.15.31:56596) 2023-10-06T02:24:09.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.775+0000 7f682affd700 1 -- 172.21.15.31:0/3547251931 learned_addr learned my addr 172.21.15.31:0/3547251931 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:09.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.775+0000 7f682affd700 1 -- 172.21.15.31:0/3547251931 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f682c153a30 msgr2=0x7f682c154a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:09.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.775+0000 7f682affd700 1 --2- 172.21.15.31:0/3547251931 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f682c153a30 0x7f682c154a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:09.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.775+0000 7f682affd700 1 -- 172.21.15.31:0/3547251931 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f682c153660 msgr2=0x7f682c14bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:09.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.775+0000 7f682a7fc700 1 --2- 172.21.15.31:0/3547251931 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f682c153660 0x7f682c14bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:09.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.775+0000 7f682affd700 1 --2- 172.21.15.31:0/3547251931 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f682c153660 0x7f682c14bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:09.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.775+0000 7f682affd700 1 -- 172.21.15.31:0/3547251931 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f682000b6d0 con 0x7f682c0c9090 2023-10-06T02:24:09.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.775+0000 7f682a7fc700 1 --2- 172.21.15.31:0/3547251931 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f682c153660 0x7f682c14bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:24:09.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.775+0000 7f682affd700 1 --2- 172.21.15.31:0/3547251931 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f682c0c9090 0x7f682c14c550 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f681c00c7c0 tx=0x7f681c007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:09.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.776+0000 7f6812ffd700 1 -- 172.21.15.31:0/3547251931 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681c005b30 con 0x7f682c0c9090 2023-10-06T02:24:09.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.776+0000 7f6812ffd700 1 -- 172.21.15.31:0/3547251931 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f681c005cb0 con 0x7f682c0c9090 2023-10-06T02:24:09.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.776+0000 7f6812ffd700 1 -- 172.21.15.31:0/3547251931 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681c004a80 con 0x7f682c0c9090 2023-10-06T02:24:09.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.776+0000 7f682bfff700 1 -- 172.21.15.31:0/3547251931 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f682c158720 con 0x7f682c0c9090 2023-10-06T02:24:09.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.776+0000 7f682bfff700 1 -- 172.21.15.31:0/3547251931 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f682c154290 con 0x7f682c0c9090 2023-10-06T02:24:09.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.777+0000 7f682bfff700 1 -- 172.21.15.31:0/3547251931 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f682c153e50 con 0x7f682c0c9090 2023-10-06T02:24:09.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.778+0000 7f6812ffd700 1 -- 172.21.15.31:0/3547251931 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f681c0070f0 con 0x7f682c0c9090 2023-10-06T02:24:09.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.779+0000 7f6812ffd700 1 --2- 172.21.15.31:0/3547251931 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6818066f30 0x7f68180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:09.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.779+0000 7f6812ffd700 1 -- 172.21.15.31:0/3547251931 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f681c0837c0 con 0x7f682c0c9090 2023-10-06T02:24:09.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.779+0000 7f682a7fc700 1 --2- 172.21.15.31:0/3547251931 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6818066f30 0x7f68180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:09.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.780+0000 7f682a7fc700 1 --2- 172.21.15.31:0/3547251931 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6818066f30 0x7f68180693e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f682c14e450 tx=0x7f682001b000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:09.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.783+0000 7f6812ffd700 1 -- 172.21.15.31:0/3547251931 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f682c153e50 con 0x7f682c0c9090 2023-10-06T02:24:09.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:09.985+0000 7f682bfff700 1 -- 172.21.15.31:0/3547251931 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f682c153e30 con 0x7f682c0c9090 2023-10-06T02:24:10.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:10.986+0000 7f6812ffd700 1 -- 172.21.15.31:0/3547251931 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v33) v1 ==== 138+0+0 (secure 0 0 0) 0x7f681c018020 con 0x7f682c0c9090 2023-10-06T02:24:10.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:10.994+0000 7f682bfff700 1 -- 172.21.15.31:0/3547251931 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6818066f30 msgr2=0x7f68180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:10.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:10.994+0000 7f682bfff700 1 --2- 172.21.15.31:0/3547251931 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6818066f30 0x7f68180693e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f682c14e450 tx=0x7f682001b000 comp rx=0 tx=0).stop 2023-10-06T02:24:10.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:10.994+0000 7f682bfff700 1 -- 172.21.15.31:0/3547251931 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f682c0c9090 msgr2=0x7f682c14c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:10.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:10.994+0000 7f682bfff700 1 --2- 172.21.15.31:0/3547251931 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f682c0c9090 0x7f682c14c550 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f681c00c7c0 tx=0x7f681c007f70 comp rx=0 tx=0).stop 2023-10-06T02:24:10.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:10.994+0000 7f682bfff700 1 -- 172.21.15.31:0/3547251931 shutdown_connections 2023-10-06T02:24:10.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:10.994+0000 7f682bfff700 1 --2- 172.21.15.31:0/3547251931 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6818066f30 0x7f68180693e0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:10.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:10.994+0000 7f682bfff700 1 --2- 172.21.15.31:0/3547251931 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f682c153a30 0x7f682c154a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:10.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:10.994+0000 7f682bfff700 1 --2- 172.21.15.31:0/3547251931 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f682c0c9090 0x7f682c14c550 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:10.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:10.994+0000 7f682bfff700 1 --2- 172.21.15.31:0/3547251931 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f682c153660 0x7f682c14bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:10.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:10.994+0000 7f682bfff700 1 -- 172.21.15.31:0/3547251931 >> 172.21.15.31:0/3547251931 conn(0x7f682c06c6a0 msgr2=0x7f682c0ca780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:10.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:10.994+0000 7f682bfff700 1 -- 172.21.15.31:0/3547251931 shutdown_connections 2023-10-06T02:24:10.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:10.995+0000 7f682bfff700 1 -- 172.21.15.31:0/3547251931 wait complete. 2023-10-06T02:24:10.998 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' created 2023-10-06T02:24:11.018 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T02:24:11.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:11.214+0000 7f962ed9d700 1 Processor -- start 2023-10-06T02:24:11.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:11.215+0000 7f962ed9d700 1 -- start start 2023-10-06T02:24:11.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:11.216+0000 7f962ed9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9628159550 0x7f96280ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:11.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:11.216+0000 7f962ed9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f962815dfd0 0x7f96280aedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:11.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:11.216+0000 7f962ed9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96280acc00 con 0x7f9628159920 2023-10-06T02:24:11.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:11.216+0000 7f962ed9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96280abff0 con 0x7f962815dfd0 2023-10-06T02:24:11.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:11.216+0000 7f962ed9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96280ac2d0 con 0x7f9628159550 2023-10-06T02:24:11.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:11.216+0000 7f962d59a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f962815dfd0 0x7f96280aedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:11.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:11.216+0000 7f962d59a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f962815dfd0 0x7f96280aedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56600/0 (socket says 172.21.15.31:56600) 2023-10-06T02:24:11.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:11.216+0000 7f962d59a700 1 -- 172.21.15.31:0/2670331024 learned_addr learned my addr 172.21.15.31:0/2670331024 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:11.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:11.217+0000 7f962dd9b700 1 --2- 172.21.15.31:0/2670331024 >> v2:172.21.15.62:3300/0 conn(0x7f9628159550 0x7f96280ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:11.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:11.217+0000 7f962d59a700 1 -- 172.21.15.31:0/2670331024 >> v2:172.21.15.62:3300/0 conn(0x7f9628159550 msgr2=0x7f96280ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:11.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:11.217+0000 7f962d59a700 1 --2- 172.21.15.31:0/2670331024 >> v2:172.21.15.62:3300/0 conn(0x7f9628159550 0x7f96280ceac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:11.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:11.217+0000 7f962d59a700 1 -- 172.21.15.31:0/2670331024 >> v1:172.21.15.62:6789/0 conn(0x7f9628159920 legacy=0x7f96280ad090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:11.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:11.217+0000 7f962d59a700 1 -- 172.21.15.31:0/2670331024 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96280ac5b0 con 0x7f962815dfd0 2023-10-06T02:24:11.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:11.217+0000 7f962dd9b700 1 --2- 172.21.15.31:0/2670331024 >> v2:172.21.15.62:3300/0 conn(0x7f9628159550 0x7f96280ceac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:24:11.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:11.217+0000 7f962d59a700 1 --2- 172.21.15.31:0/2670331024 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f962815dfd0 0x7f96280aedb0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f9628064270 tx=0x7f9618009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2ca08f7d100e865 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:11.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:11.217+0000 7f962cd99700 1 -- 172.21.15.31:0/2670331024 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f961800b860 con 0x7f962815dfd0 2023-10-06T02:24:11.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:11.217+0000 7f962cd99700 1 -- 172.21.15.31:0/2670331024 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f961800b9c0 con 0x7f962815dfd0 2023-10-06T02:24:11.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:11.217+0000 7f962cd99700 1 -- 172.21.15.31:0/2670331024 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f961800bd60 con 0x7f962815dfd0 2023-10-06T02:24:11.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:11.218+0000 7f962ed9d700 1 -- 172.21.15.31:0/2670331024 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f962815dfd0 msgr2=0x7f96280aedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:11.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:11.218+0000 7f962ed9d700 1 --2- 172.21.15.31:0/2670331024 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f962815dfd0 0x7f96280aedb0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f9628064270 tx=0x7f9618009250 comp rx=0 tx=0).stop 2023-10-06T02:24:11.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:11.218+0000 7f962ed9d700 1 -- 172.21.15.31:0/2670331024 shutdown_connections 2023-10-06T02:24:11.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:11.218+0000 7f962ed9d700 1 --2- 172.21.15.31:0/2670331024 >> v2:172.21.15.62:3300/0 conn(0x7f9628159550 0x7f96280ceac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:11.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:11.218+0000 7f962ed9d700 1 --2- 172.21.15.31:0/2670331024 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f962815dfd0 0x7f96280aedb0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:11.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:11.218+0000 7f962ed9d700 1 -- 172.21.15.31:0/2670331024 >> 172.21.15.31:0/2670331024 conn(0x7f962806ce80 msgr2=0x7f96280ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:11.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:11.218+0000 7f962ed9d700 1 -- 172.21.15.31:0/2670331024 shutdown_connections 2023-10-06T02:24:11.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:11.218+0000 7f962ed9d700 1 -- 172.21.15.31:0/2670331024 wait complete. 2023-10-06T02:24:11.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:11.220+0000 7f962ed9d700 1 Processor -- start 2023-10-06T02:24:11.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:11.220+0000 7f962ed9d700 1 -- start start 2023-10-06T02:24:11.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:11.221+0000 7f962ed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9628159550 0x7f9628149f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:11.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:11.221+0000 7f962ed9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9628159920 0x7f96280cc600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:11.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:11.221+0000 7f962ed9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f962815dfd0 0x7f9628152380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:11.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:11.221+0000 7f962ed9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96280b2cf0 con 0x7f9628159550 2023-10-06T02:24:11.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:11.221+0000 7f962ed9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96280b3870 con 0x7f9628159920 2023-10-06T02:24:11.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:11.221+0000 7f962ed9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f962814a450 con 0x7f962815dfd0 2023-10-06T02:24:11.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:11.221+0000 7f962dd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9628159550 0x7f9628149f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:11.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:11.221+0000 7f962dd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9628159550 0x7f9628149f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56602/0 (socket says 172.21.15.31:56602) 2023-10-06T02:24:11.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:11.221+0000 7f962dd9b700 1 -- 172.21.15.31:0/1004326030 learned_addr learned my addr 172.21.15.31:0/1004326030 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:11.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:11.221+0000 7f962e59c700 1 --2- 172.21.15.31:0/1004326030 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f962815dfd0 0x7f9628152380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:11.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:11.221+0000 7f962d59a700 1 --2- 172.21.15.31:0/1004326030 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9628159920 0x7f96280cc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:11.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:11.221+0000 7f962dd9b700 1 -- 172.21.15.31:0/1004326030 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f962815dfd0 msgr2=0x7f9628152380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:11.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:11.221+0000 7f962dd9b700 1 --2- 172.21.15.31:0/1004326030 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f962815dfd0 0x7f9628152380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:11.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:11.221+0000 7f962dd9b700 1 -- 172.21.15.31:0/1004326030 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9628159920 msgr2=0x7f96280cc600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:11.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:11.221+0000 7f962dd9b700 1 --2- 172.21.15.31:0/1004326030 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9628159920 0x7f96280cc600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:11.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:11.221+0000 7f962dd9b700 1 -- 172.21.15.31:0/1004326030 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96180094f0 con 0x7f9628159550 2023-10-06T02:24:11.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:11.221+0000 7f962d59a700 1 --2- 172.21.15.31:0/1004326030 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9628159920 0x7f96280cc600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:24:11.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:11.221+0000 7f962e59c700 1 --2- 172.21.15.31:0/1004326030 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f962815dfd0 0x7f9628152380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:24:11.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:11.222+0000 7f962dd9b700 1 --2- 172.21.15.31:0/1004326030 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9628159550 0x7f9628149f10 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f962400e900 tx=0x7f9624008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:11.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:11.222+0000 7f961dffb700 1 -- 172.21.15.31:0/1004326030 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f962400eea0 con 0x7f9628159550 2023-10-06T02:24:11.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:11.222+0000 7f961dffb700 1 -- 172.21.15.31:0/1004326030 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f962400c5e0 con 0x7f9628159550 2023-10-06T02:24:11.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:11.222+0000 7f961dffb700 1 -- 172.21.15.31:0/1004326030 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f962400c980 con 0x7f9628159550 2023-10-06T02:24:11.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:11.222+0000 7f962ed9d700 1 -- 172.21.15.31:0/1004326030 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96280b2f40 con 0x7f9628159550 2023-10-06T02:24:11.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:11.222+0000 7f962ed9d700 1 -- 172.21.15.31:0/1004326030 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96280b3500 con 0x7f9628159550 2023-10-06T02:24:11.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:11.223+0000 7f962ed9d700 1 -- 172.21.15.31:0/1004326030 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96280b3b50 con 0x7f9628159550 2023-10-06T02:24:11.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:11.224+0000 7f961dffb700 1 -- 172.21.15.31:0/1004326030 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f962400cd30 con 0x7f9628159550 2023-10-06T02:24:11.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:11.225+0000 7f961dffb700 1 --2- 172.21.15.31:0/1004326030 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f960c066f30 0x7f960c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:11.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:11.225+0000 7f961dffb700 1 -- 172.21.15.31:0/1004326030 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5576+0+0 (secure 0 0 0) 0x7f96240843a0 con 0x7f9628159550 2023-10-06T02:24:11.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:11.225+0000 7f962d59a700 1 --2- 172.21.15.31:0/1004326030 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f960c066f30 0x7f960c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:11.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:11.226+0000 7f962d59a700 1 --2- 172.21.15.31:0/1004326030 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f960c066f30 0x7f960c0693e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9628064270 tx=0x7f9618017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:11.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:11.230+0000 7f961dffb700 1 -- 172.21.15.31:0/1004326030 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9624055080 con 0x7f9628159550 2023-10-06T02:24:11.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:11.431+0000 7f962ed9d700 1 -- 172.21.15.31:0/1004326030 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f962804c3c0 con 0x7f9628159550 2023-10-06T02:24:11.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:11.994+0000 7f961dffb700 1 -- 172.21.15.31:0/1004326030 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v34) v1 ==== 170+0+0 (secure 0 0 0) 0x7f9624054c10 con 0x7f9628159550 2023-10-06T02:24:12.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:12.000+0000 7f962ed9d700 1 -- 172.21.15.31:0/1004326030 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f960c066f30 msgr2=0x7f960c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:12.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:12.000+0000 7f962ed9d700 1 --2- 172.21.15.31:0/1004326030 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f960c066f30 0x7f960c0693e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9628064270 tx=0x7f9618017000 comp rx=0 tx=0).stop 2023-10-06T02:24:12.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:12.000+0000 7f962ed9d700 1 -- 172.21.15.31:0/1004326030 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9628159550 msgr2=0x7f9628149f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:12.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:12.000+0000 7f962ed9d700 1 --2- 172.21.15.31:0/1004326030 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9628159550 0x7f9628149f10 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f962400e900 tx=0x7f9624008040 comp rx=0 tx=0).stop 2023-10-06T02:24:12.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:12.002+0000 7f962ed9d700 1 -- 172.21.15.31:0/1004326030 shutdown_connections 2023-10-06T02:24:12.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:12.002+0000 7f962ed9d700 1 --2- 172.21.15.31:0/1004326030 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f960c066f30 0x7f960c0693e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:12.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:12.002+0000 7f962ed9d700 1 --2- 172.21.15.31:0/1004326030 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f962815dfd0 0x7f9628152380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:12.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:12.002+0000 7f962ed9d700 1 --2- 172.21.15.31:0/1004326030 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9628159550 0x7f9628149f10 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:12.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:12.002+0000 7f962ed9d700 1 --2- 172.21.15.31:0/1004326030 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9628159920 0x7f96280cc600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:12.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:12.002+0000 7f962ed9d700 1 -- 172.21.15.31:0/1004326030 >> 172.21.15.31:0/1004326030 conn(0x7f962806ce80 msgr2=0x7f96281535c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:12.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:12.007+0000 7f962ed9d700 1 -- 172.21.15.31:0/1004326030 shutdown_connections 2023-10-06T02:24:12.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:12.007+0000 7f962ed9d700 1 -- 172.21.15.31:0/1004326030 wait complete. 2023-10-06T02:24:12.017 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' created 2023-10-06T02:24:12.037 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T02:24:12.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:12.227+0000 7f205679d700 1 Processor -- start 2023-10-06T02:24:12.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:12.228+0000 7f205679d700 1 -- start start 2023-10-06T02:24:12.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:12.229+0000 7f205679d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f205006db20 0x7f205015a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:12.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:12.229+0000 7f205679d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f20500caec0 0x7f205015afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:12.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:12.229+0000 7f205679d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f205015ada0 con 0x7f2050153c90 2023-10-06T02:24:12.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:12.229+0000 7f205679d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f205006ca80 con 0x7f20500caec0 2023-10-06T02:24:12.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:12.229+0000 7f205679d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f205006cd60 con 0x7f205006db20 2023-10-06T02:24:12.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:12.229+0000 7f2055f9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f20500caec0 0x7f205015afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:12.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:12.229+0000 7f2055f9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f20500caec0 0x7f205015afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56616/0 (socket says 172.21.15.31:56616) 2023-10-06T02:24:12.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:12.229+0000 7f2055f9c700 1 -- 172.21.15.31:0/1645532201 learned_addr learned my addr 172.21.15.31:0/1645532201 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:12.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:12.230+0000 7f2054f9a700 1 --2- 172.21.15.31:0/1645532201 >> v2:172.21.15.62:3300/0 conn(0x7f205006db20 0x7f205015a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:12.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:12.230+0000 7f2055f9c700 1 -- 172.21.15.31:0/1645532201 >> v2:172.21.15.62:3300/0 conn(0x7f205006db20 msgr2=0x7f205015a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:12.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:12.230+0000 7f2055f9c700 1 --2- 172.21.15.31:0/1645532201 >> v2:172.21.15.62:3300/0 conn(0x7f205006db20 0x7f205015a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:12.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:12.230+0000 7f2055f9c700 1 -- 172.21.15.31:0/1645532201 >> v1:172.21.15.62:6789/0 conn(0x7f2050153c90 legacy=0x7f20500ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:12.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:12.230+0000 7f2055f9c700 1 -- 172.21.15.31:0/1645532201 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f205006d040 con 0x7f20500caec0 2023-10-06T02:24:12.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:12.230+0000 7f2055f9c700 1 --2- 172.21.15.31:0/1645532201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20500caec0 0x7f205015afb0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f20500cc300 tx=0x7f2048009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64c15f1c9392c355 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:12.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:12.230+0000 7f2047fff700 1 -- 172.21.15.31:0/1645532201 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204800e070 con 0x7f20500caec0 2023-10-06T02:24:12.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:12.230+0000 7f2047fff700 1 -- 172.21.15.31:0/1645532201 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f204800ba50 con 0x7f20500caec0 2023-10-06T02:24:12.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:12.230+0000 7f2047fff700 1 -- 172.21.15.31:0/1645532201 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20480092e0 con 0x7f20500caec0 2023-10-06T02:24:12.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:12.231+0000 7f205679d700 1 -- 172.21.15.31:0/1645532201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20500caec0 msgr2=0x7f205015afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:12.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:12.231+0000 7f205679d700 1 --2- 172.21.15.31:0/1645532201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20500caec0 0x7f205015afb0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f20500cc300 tx=0x7f2048009840 comp rx=0 tx=0).stop 2023-10-06T02:24:12.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:12.231+0000 7f205679d700 1 -- 172.21.15.31:0/1645532201 shutdown_connections 2023-10-06T02:24:12.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:12.231+0000 7f205679d700 1 --2- 172.21.15.31:0/1645532201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20500caec0 0x7f205015afb0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:12.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:12.231+0000 7f205679d700 1 --2- 172.21.15.31:0/1645532201 >> v2:172.21.15.62:3300/0 conn(0x7f205006db20 0x7f205015a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:12.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:12.231+0000 7f205679d700 1 -- 172.21.15.31:0/1645532201 >> 172.21.15.31:0/1645532201 conn(0x7f205006e720 msgr2=0x7f205006eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:12.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:12.231+0000 7f205679d700 1 -- 172.21.15.31:0/1645532201 shutdown_connections 2023-10-06T02:24:12.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:12.231+0000 7f205679d700 1 -- 172.21.15.31:0/1645532201 wait complete. 2023-10-06T02:24:12.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:12.232+0000 7f205679d700 1 Processor -- start 2023-10-06T02:24:12.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:12.233+0000 7f205679d700 1 -- start start 2023-10-06T02:24:12.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:12.233+0000 7f205679d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f205006db20 0x7f20501500e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:12.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:12.233+0000 7f205679d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20500caec0 0x7f205015e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:12.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:12.233+0000 7f205679d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2050153c90 0x7f20500b1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:12.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:12.233+0000 7f205679d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f205015dca0 con 0x7f205006db20 2023-10-06T02:24:12.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:12.233+0000 7f205679d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f205015d090 con 0x7f20500caec0 2023-10-06T02:24:12.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:12.233+0000 7f205679d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f205015d370 con 0x7f2050153c90 2023-10-06T02:24:12.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:12.234+0000 7f205579b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f205006db20 0x7f20501500e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:12.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:12.234+0000 7f205579b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f205006db20 0x7f20501500e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56632/0 (socket says 172.21.15.31:56632) 2023-10-06T02:24:12.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:12.234+0000 7f205579b700 1 -- 172.21.15.31:0/150679706 learned_addr learned my addr 172.21.15.31:0/150679706 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:12.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:12.234+0000 7f2054f9a700 1 --2- 172.21.15.31:0/150679706 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20500caec0 0x7f205015e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:12.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:12.234+0000 7f205579b700 1 -- 172.21.15.31:0/150679706 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2050153c90 msgr2=0x7f20500b1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:12.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:12.234+0000 7f205579b700 1 --2- 172.21.15.31:0/150679706 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2050153c90 0x7f20500b1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:12.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:12.234+0000 7f205579b700 1 -- 172.21.15.31:0/150679706 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20500caec0 msgr2=0x7f205015e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:12.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:12.234+0000 7f205579b700 1 --2- 172.21.15.31:0/150679706 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20500caec0 0x7f205015e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:12.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:12.234+0000 7f205579b700 1 -- 172.21.15.31:0/150679706 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2048009ae0 con 0x7f205006db20 2023-10-06T02:24:12.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:12.234+0000 7f2054f9a700 1 --2- 172.21.15.31:0/150679706 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20500caec0 0x7f205015e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:24:12.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:12.234+0000 7f205579b700 1 --2- 172.21.15.31:0/150679706 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f205006db20 0x7f20501500e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f204c00c590 tx=0x7f204c00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:12.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:12.235+0000 7f20457fa700 1 -- 172.21.15.31:0/150679706 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204c012630 con 0x7f205006db20 2023-10-06T02:24:12.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:12.235+0000 7f20457fa700 1 -- 172.21.15.31:0/150679706 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f204c012790 con 0x7f205006db20 2023-10-06T02:24:12.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:12.235+0000 7f205679d700 1 -- 172.21.15.31:0/150679706 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f205015d650 con 0x7f205006db20 2023-10-06T02:24:12.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:12.235+0000 7f20457fa700 1 -- 172.21.15.31:0/150679706 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204c012b50 con 0x7f205006db20 2023-10-06T02:24:12.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:12.235+0000 7f205679d700 1 -- 172.21.15.31:0/150679706 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20500b1b50 con 0x7f205006db20 2023-10-06T02:24:12.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:12.236+0000 7f205679d700 1 -- 172.21.15.31:0/150679706 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20500b1dd0 con 0x7f205006db20 2023-10-06T02:24:12.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:12.237+0000 7f20457fa700 1 -- 172.21.15.31:0/150679706 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f204c01b400 con 0x7f205006db20 2023-10-06T02:24:12.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:12.238+0000 7f20457fa700 1 --2- 172.21.15.31:0/150679706 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f2034066f30 0x7f20340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:12.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:12.238+0000 7f20457fa700 1 -- 172.21.15.31:0/150679706 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5958+0+0 (secure 0 0 0) 0x7f204c084180 con 0x7f205006db20 2023-10-06T02:24:12.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:12.238+0000 7f2054f9a700 1 --2- 172.21.15.31:0/150679706 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f2034066f30 0x7f20340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:12.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:12.239+0000 7f2054f9a700 1 --2- 172.21.15.31:0/150679706 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f2034066f30 0x7f20340693e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f205006d4e0 tx=0x7f2040008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:12.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:12.242+0000 7f20457fa700 1 -- 172.21.15.31:0/150679706 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f204c051250 con 0x7f205006db20 2023-10-06T02:24:12.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:12.491+0000 7f205679d700 1 -- 172.21.15.31:0/150679706 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f205015ada0 con 0x7f205006db20 2023-10-06T02:24:13.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:12.999+0000 7f20457fa700 1 -- 172.21.15.31:0/150679706 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v13) v1 ==== 172+0+0 (secure 0 0 0) 0x7f205015ada0 con 0x7f205006db20 2023-10-06T02:24:13.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.007+0000 7f205679d700 1 -- 172.21.15.31:0/150679706 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f2034066f30 msgr2=0x7f20340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:13.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.007+0000 7f205679d700 1 --2- 172.21.15.31:0/150679706 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f2034066f30 0x7f20340693e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f205006d4e0 tx=0x7f2040008040 comp rx=0 tx=0).stop 2023-10-06T02:24:13.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.007+0000 7f205679d700 1 -- 172.21.15.31:0/150679706 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f205006db20 msgr2=0x7f20501500e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:13.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.007+0000 7f205679d700 1 --2- 172.21.15.31:0/150679706 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f205006db20 0x7f20501500e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f204c00c590 tx=0x7f204c00c950 comp rx=0 tx=0).stop 2023-10-06T02:24:13.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.013+0000 7f205679d700 1 -- 172.21.15.31:0/150679706 shutdown_connections 2023-10-06T02:24:13.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.013+0000 7f205679d700 1 --2- 172.21.15.31:0/150679706 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f2034066f30 0x7f20340693e0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:13.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.013+0000 7f205679d700 1 --2- 172.21.15.31:0/150679706 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2050153c90 0x7f20500b1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:13.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.013+0000 7f205679d700 1 --2- 172.21.15.31:0/150679706 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f205006db20 0x7f20501500e0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:13.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.013+0000 7f205679d700 1 --2- 172.21.15.31:0/150679706 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20500caec0 0x7f205015e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:13.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.013+0000 7f205679d700 1 -- 172.21.15.31:0/150679706 >> 172.21.15.31:0/150679706 conn(0x7f205006e720 msgr2=0x7f2050159360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:13.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.023+0000 7f205679d700 1 -- 172.21.15.31:0/150679706 shutdown_connections 2023-10-06T02:24:13.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.026+0000 7f205679d700 1 -- 172.21.15.31:0/150679706 wait complete. 2023-10-06T02:24:13.034 INFO:teuthology.orchestra.run.smithi031.stderr:new fs with metadata pool 5 and data pool 6 2023-10-06T02:24:13.052 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:24:13.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.241+0000 7fef78f9d700 1 Processor -- start 2023-10-06T02:24:13.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.241+0000 7fef78f9d700 1 -- start start 2023-10-06T02:24:13.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.243+0000 7fef78f9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fef74142c00 0x7fef7406b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:13.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.243+0000 7fef78f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fef7406bd20 0x7fef740c7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:13.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.243+0000 7fef78f9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef740ca480 con 0x7fef74142fd0 2023-10-06T02:24:13.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.243+0000 7fef78f9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef7415e240 con 0x7fef74142c00 2023-10-06T02:24:13.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.243+0000 7fef78f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef7406b350 con 0x7fef7406bd20 2023-10-06T02:24:13.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.243+0000 7fef73fff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fef74142fd0 0x7fef7406c6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:33020/0 (socket says 172.21.15.31:33020) 2023-10-06T02:24:13.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.243+0000 7fef73fff700 1 -- 172.21.15.31:0/990695363 learned_addr learned my addr 172.21.15.31:0/990695363 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:24:13.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.243+0000 7fef72ffd700 1 --2- 172.21.15.31:0/990695363 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fef7406bd20 0x7fef740c7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:13.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.243+0000 7fef737fe700 1 --2- 172.21.15.31:0/990695363 >> v2:172.21.15.62:3300/0 conn(0x7fef74142c00 0x7fef7406b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:13.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.244+0000 7fef727fc700 1 -- 172.21.15.31:0/990695363 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 313855865 0 0) 0x7fef740ca480 con 0x7fef74142fd0 2023-10-06T02:24:13.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.244+0000 7fef727fc700 1 -- 172.21.15.31:0/990695363 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef5c003570 con 0x7fef74142fd0 2023-10-06T02:24:13.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.244+0000 7fef72ffd700 1 -- 172.21.15.31:0/990695363 >> v2:172.21.15.62:3300/0 conn(0x7fef74142c00 msgr2=0x7fef7406b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:13.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.244+0000 7fef72ffd700 1 --2- 172.21.15.31:0/990695363 >> v2:172.21.15.62:3300/0 conn(0x7fef74142c00 0x7fef7406b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:13.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.244+0000 7fef72ffd700 1 -- 172.21.15.31:0/990695363 >> v1:172.21.15.31:6789/0 conn(0x7fef74142fd0 legacy=0x7fef7406c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:13.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.244+0000 7fef727fc700 1 -- 172.21.15.31:0/990695363 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 336775414 0 0) 0x7fef5c003570 con 0x7fef74142fd0 2023-10-06T02:24:13.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.244+0000 7fef72ffd700 1 -- 172.21.15.31:0/990695363 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef7415df60 con 0x7fef7406bd20 2023-10-06T02:24:13.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.244+0000 7fef737fe700 1 --2- 172.21.15.31:0/990695363 >> v2:172.21.15.62:3300/0 conn(0x7fef74142c00 0x7fef7406b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:24:13.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.244+0000 7fef72ffd700 1 --2- 172.21.15.31:0/990695363 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fef7406bd20 0x7fef740c7880 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fef680098e0 tx=0x7fef680091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a867d28fcec505e9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:13.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.245+0000 7fef727fc700 1 -- 172.21.15.31:0/990695363 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef6c0017f0 con 0x7fef7406bd20 2023-10-06T02:24:13.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.245+0000 7fef727fc700 1 -- 172.21.15.31:0/990695363 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef68011420 con 0x7fef7406bd20 2023-10-06T02:24:13.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.245+0000 7fef727fc700 1 -- 172.21.15.31:0/990695363 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef680117c0 con 0x7fef7406bd20 2023-10-06T02:24:13.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.245+0000 7fef78f9d700 1 -- 172.21.15.31:0/990695363 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fef7406bd20 msgr2=0x7fef740c7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:13.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.245+0000 7fef78f9d700 1 --2- 172.21.15.31:0/990695363 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fef7406bd20 0x7fef740c7880 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fef680098e0 tx=0x7fef680091a0 comp rx=0 tx=0).stop 2023-10-06T02:24:13.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.245+0000 7fef78f9d700 1 -- 172.21.15.31:0/990695363 shutdown_connections 2023-10-06T02:24:13.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.245+0000 7fef78f9d700 1 --2- 172.21.15.31:0/990695363 >> v2:172.21.15.62:3300/0 conn(0x7fef74142c00 0x7fef7406b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:13.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.245+0000 7fef78f9d700 1 --2- 172.21.15.31:0/990695363 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fef7406bd20 0x7fef740c7880 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:13.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.245+0000 7fef78f9d700 1 -- 172.21.15.31:0/990695363 >> 172.21.15.31:0/990695363 conn(0x7fef740675a0 msgr2=0x7fef740679a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:13.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.245+0000 7fef78f9d700 1 -- 172.21.15.31:0/990695363 shutdown_connections 2023-10-06T02:24:13.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.245+0000 7fef78f9d700 1 -- 172.21.15.31:0/990695363 wait complete. 2023-10-06T02:24:13.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.247+0000 7fef78f9d700 1 Processor -- start 2023-10-06T02:24:13.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.247+0000 7fef78f9d700 1 -- start start 2023-10-06T02:24:13.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.248+0000 7fef78f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fef7406bd20 0x7fef740b2890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:13.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.248+0000 7fef78f9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef74142c00 0x7fef74158e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:13.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.248+0000 7fef78f9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef74142fd0 0x7fef740c47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:13.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.248+0000 7fef78f9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef740b09f0 con 0x7fef74142c00 2023-10-06T02:24:13.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.248+0000 7fef78f9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef740b1ca0 con 0x7fef74142fd0 2023-10-06T02:24:13.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.248+0000 7fef78f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef740b1810 con 0x7fef7406bd20 2023-10-06T02:24:13.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.248+0000 7fef72ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef74142c00 0x7fef74158e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:13.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.248+0000 7fef72ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef74142c00 0x7fef74158e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56638/0 (socket says 172.21.15.31:56638) 2023-10-06T02:24:13.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.248+0000 7fef72ffd700 1 -- 172.21.15.31:0/287486238 learned_addr learned my addr 172.21.15.31:0/287486238 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:13.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.248+0000 7fef72ffd700 1 -- 172.21.15.31:0/287486238 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fef7406bd20 msgr2=0x7fef740b2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:13.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.248+0000 7fef72ffd700 1 --2- 172.21.15.31:0/287486238 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fef7406bd20 0x7fef740b2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:13.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.248+0000 7fef72ffd700 1 -- 172.21.15.31:0/287486238 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef74142fd0 msgr2=0x7fef740c47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:13.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.248+0000 7fef72ffd700 1 --2- 172.21.15.31:0/287486238 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef74142fd0 0x7fef740c47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:13.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.248+0000 7fef72ffd700 1 -- 172.21.15.31:0/287486238 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef64009420 con 0x7fef74142c00 2023-10-06T02:24:13.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.249+0000 7fef72ffd700 1 --2- 172.21.15.31:0/287486238 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef74142c00 0x7fef74158e40 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fef680030c0 tx=0x7fef68016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:13.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.249+0000 7fef5b7fe700 1 -- 172.21.15.31:0/287486238 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef68000d10 con 0x7fef74142c00 2023-10-06T02:24:13.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.249+0000 7fef5b7fe700 1 -- 172.21.15.31:0/287486238 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef6801cc90 con 0x7fef74142c00 2023-10-06T02:24:13.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.249+0000 7fef5b7fe700 1 -- 172.21.15.31:0/287486238 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef6801a680 con 0x7fef74142c00 2023-10-06T02:24:13.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.249+0000 7fef78f9d700 1 -- 172.21.15.31:0/287486238 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef68009420 con 0x7fef74142c00 2023-10-06T02:24:13.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.249+0000 7fef78f9d700 1 -- 172.21.15.31:0/287486238 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef740b0ee0 con 0x7fef74142c00 2023-10-06T02:24:13.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.250+0000 7fef78f9d700 1 -- 172.21.15.31:0/287486238 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef7404c3c0 con 0x7fef74142c00 2023-10-06T02:24:13.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.251+0000 7fef5b7fe700 1 -- 172.21.15.31:0/287486238 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fef6801aa70 con 0x7fef74142c00 2023-10-06T02:24:13.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.252+0000 7fef5b7fe700 1 --2- 172.21.15.31:0/287486238 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fef60066f30 0x7fef600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:13.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.253+0000 7fef5b7fe700 1 -- 172.21.15.31:0/287486238 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fef68086070 con 0x7fef74142c00 2023-10-06T02:24:13.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.253+0000 7fef737fe700 1 --2- 172.21.15.31:0/287486238 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fef60066f30 0x7fef600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:13.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.253+0000 7fef737fe700 1 --2- 172.21.15.31:0/287486238 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fef60066f30 0x7fef600693e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fef740c5a70 tx=0x7fef640091a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:13.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.257+0000 7fef5b7fe700 1 -- 172.21.15.31:0/287486238 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fef680b75a0 con 0x7fef74142c00 2023-10-06T02:24:13.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.460+0000 7fef78f9d700 1 -- 172.21.15.31:0/287486238 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fef740b19a0 con 0x7fef74142c00 2023-10-06T02:24:13.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.461+0000 7fef5b7fe700 1 -- 172.21.15.31:0/287486238 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16831 (secure 0 0 0) 0x7fef68056b70 con 0x7fef74142c00 2023-10-06T02:24:13.463 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:24:13.463 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":35,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:24:12.493604+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:24:10.026011+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-06T02:24:11.471970+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:24:13.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.463+0000 7fef78f9d700 1 -- 172.21.15.31:0/287486238 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fef60066f30 msgr2=0x7fef600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:13.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.463+0000 7fef78f9d700 1 --2- 172.21.15.31:0/287486238 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fef60066f30 0x7fef600693e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fef740c5a70 tx=0x7fef640091a0 comp rx=0 tx=0).stop 2023-10-06T02:24:13.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.463+0000 7fef78f9d700 1 -- 172.21.15.31:0/287486238 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef74142c00 msgr2=0x7fef74158e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:13.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.463+0000 7fef78f9d700 1 --2- 172.21.15.31:0/287486238 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef74142c00 0x7fef74158e40 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fef680030c0 tx=0x7fef68016000 comp rx=0 tx=0).stop 2023-10-06T02:24:13.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.463+0000 7fef78f9d700 1 -- 172.21.15.31:0/287486238 shutdown_connections 2023-10-06T02:24:13.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.464+0000 7fef78f9d700 1 --2- 172.21.15.31:0/287486238 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef74142fd0 0x7fef740c47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:13.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.464+0000 7fef78f9d700 1 --2- 172.21.15.31:0/287486238 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fef60066f30 0x7fef600693e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:13.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.464+0000 7fef78f9d700 1 --2- 172.21.15.31:0/287486238 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fef7406bd20 0x7fef740b2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:13.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.464+0000 7fef78f9d700 1 --2- 172.21.15.31:0/287486238 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef74142c00 0x7fef74158e40 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:13.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.464+0000 7fef78f9d700 1 -- 172.21.15.31:0/287486238 >> 172.21.15.31:0/287486238 conn(0x7fef740675a0 msgr2=0x7fef741404e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:13.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.464+0000 7fef78f9d700 1 -- 172.21.15.31:0/287486238 shutdown_connections 2023-10-06T02:24:13.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.464+0000 7fef78f9d700 1 -- 172.21.15.31:0/287486238 wait complete. 2023-10-06T02:24:13.482 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:24:13.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.669+0000 7f933ffff700 1 Processor -- start 2023-10-06T02:24:13.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.669+0000 7f933ffff700 1 -- start start 2023-10-06T02:24:13.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.671+0000 7f933ffff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f934013fb20 0x7f934015a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:13.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.671+0000 7f933ffff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f934013fef0 0x7f93400c7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:13.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.671+0000 7f933ffff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f934015ae70 con 0x7f934006aaa0 2023-10-06T02:24:13.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.671+0000 7f933ffff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f934006af60 con 0x7f934013fb20 2023-10-06T02:24:13.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.671+0000 7f933ffff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f934006b240 con 0x7f934013fef0 2023-10-06T02:24:13.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.671+0000 7f933e7fc700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f934013fef0 0x7f93400c7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:13.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.671+0000 7f933f7fe700 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f934006aaa0 0x7f93400c6770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:42326/0 (socket says 172.21.15.31:42326) 2023-10-06T02:24:13.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.671+0000 7f933f7fe700 1 -- 172.21.15.31:0/1041160704 learned_addr learned my addr 172.21.15.31:0/1041160704 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:24:13.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.671+0000 7f933effd700 1 --2- 172.21.15.31:0/1041160704 >> v2:172.21.15.62:3300/0 conn(0x7f934013fb20 0x7f934015a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:13.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.672+0000 7f933e7fc700 1 -- 172.21.15.31:0/1041160704 >> v2:172.21.15.62:3300/0 conn(0x7f934013fb20 msgr2=0x7f934015a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:13.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.672+0000 7f933e7fc700 1 --2- 172.21.15.31:0/1041160704 >> v2:172.21.15.62:3300/0 conn(0x7f934013fb20 0x7f934015a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:13.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.672+0000 7f933e7fc700 1 -- 172.21.15.31:0/1041160704 >> v1:172.21.15.62:6789/0 conn(0x7f934006aaa0 legacy=0x7f93400c6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:13.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.672+0000 7f933dffb700 1 -- 172.21.15.31:0/1041160704 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1778585032 0 0) 0x7f9338001490 con 0x7f934006aaa0 2023-10-06T02:24:13.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.672+0000 7f933e7fc700 1 -- 172.21.15.31:0/1041160704 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f934006b520 con 0x7f934013fef0 2023-10-06T02:24:13.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.672+0000 7f933e7fc700 1 --2- 172.21.15.31:0/1041160704 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f934013fef0 0x7f93400c7990 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f93340098d0 tx=0x7f9334009120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9fe19c12aa4f6f1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:13.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.672+0000 7f933dffb700 1 -- 172.21.15.31:0/1041160704 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9334005c10 con 0x7f934013fef0 2023-10-06T02:24:13.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.672+0000 7f933dffb700 1 -- 172.21.15.31:0/1041160704 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9334005d70 con 0x7f934013fef0 2023-10-06T02:24:13.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.672+0000 7f933dffb700 1 -- 172.21.15.31:0/1041160704 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9334006280 con 0x7f934013fef0 2023-10-06T02:24:13.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.673+0000 7f933ffff700 1 -- 172.21.15.31:0/1041160704 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f934013fef0 msgr2=0x7f93400c7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:13.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.673+0000 7f933ffff700 1 --2- 172.21.15.31:0/1041160704 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f934013fef0 0x7f93400c7990 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f93340098d0 tx=0x7f9334009120 comp rx=0 tx=0).stop 2023-10-06T02:24:13.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.673+0000 7f933ffff700 1 -- 172.21.15.31:0/1041160704 shutdown_connections 2023-10-06T02:24:13.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.673+0000 7f933ffff700 1 --2- 172.21.15.31:0/1041160704 >> v2:172.21.15.62:3300/0 conn(0x7f934013fb20 0x7f934015a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:13.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.673+0000 7f933ffff700 1 --2- 172.21.15.31:0/1041160704 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f934013fef0 0x7f93400c7990 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:13.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.673+0000 7f933ffff700 1 -- 172.21.15.31:0/1041160704 >> 172.21.15.31:0/1041160704 conn(0x7f934006d170 msgr2=0x7f93400cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:13.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.673+0000 7f933ffff700 1 -- 172.21.15.31:0/1041160704 shutdown_connections 2023-10-06T02:24:13.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.673+0000 7f933ffff700 1 -- 172.21.15.31:0/1041160704 wait complete. 2023-10-06T02:24:13.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.674+0000 7f933ffff700 1 Processor -- start 2023-10-06T02:24:13.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.675+0000 7f933ffff700 1 -- start start 2023-10-06T02:24:13.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.675+0000 7f933ffff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f934006aaa0 0x7f93400c8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:13.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.676+0000 7f933ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f934013fb20 0x7f9340158320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:13.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.676+0000 7f933ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f934013fef0 0x7f9340155bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:13.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.676+0000 7f933ffff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9340158970 con 0x7f934013fb20 2023-10-06T02:24:13.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.676+0000 7f933ffff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f934013e0a0 con 0x7f934013fef0 2023-10-06T02:24:13.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.676+0000 7f933ffff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9340156af0 con 0x7f934006aaa0 2023-10-06T02:24:13.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.676+0000 7f933e7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f934013fb20 0x7f9340158320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:13.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.676+0000 7f933e7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f934013fb20 0x7f9340158320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56646/0 (socket says 172.21.15.31:56646) 2023-10-06T02:24:13.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.676+0000 7f933e7fc700 1 -- 172.21.15.31:0/2809666796 learned_addr learned my addr 172.21.15.31:0/2809666796 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:13.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.676+0000 7f933f7fe700 1 --2- 172.21.15.31:0/2809666796 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f934013fef0 0x7f9340155bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:13.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.676+0000 7f933effd700 1 --2- 172.21.15.31:0/2809666796 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f934006aaa0 0x7f93400c8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:13.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.676+0000 7f933e7fc700 1 -- 172.21.15.31:0/2809666796 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f934006aaa0 msgr2=0x7f93400c8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:13.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.676+0000 7f933e7fc700 1 --2- 172.21.15.31:0/2809666796 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f934006aaa0 0x7f93400c8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:13.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.676+0000 7f933e7fc700 1 -- 172.21.15.31:0/2809666796 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f934013fef0 msgr2=0x7f9340155bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:13.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.676+0000 7f933e7fc700 1 --2- 172.21.15.31:0/2809666796 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f934013fef0 0x7f9340155bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:13.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.676+0000 7f933e7fc700 1 -- 172.21.15.31:0/2809666796 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93340093c0 con 0x7f934013fb20 2023-10-06T02:24:13.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.677+0000 7f933e7fc700 1 --2- 172.21.15.31:0/2809666796 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f934013fb20 0x7f9340158320 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f9334005fd0 tx=0x7f9334016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:13.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.677+0000 7f9326ffd700 1 -- 172.21.15.31:0/2809666796 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f933401d6b0 con 0x7f934013fb20 2023-10-06T02:24:13.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.677+0000 7f9326ffd700 1 -- 172.21.15.31:0/2809666796 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f933401d810 con 0x7f934013fb20 2023-10-06T02:24:13.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.677+0000 7f933ffff700 1 -- 172.21.15.31:0/2809666796 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9340156dd0 con 0x7f934013fb20 2023-10-06T02:24:13.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.677+0000 7f9326ffd700 1 -- 172.21.15.31:0/2809666796 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f933401dbd0 con 0x7f934013fb20 2023-10-06T02:24:13.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.677+0000 7f933ffff700 1 -- 172.21.15.31:0/2809666796 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9340157390 con 0x7f934013fb20 2023-10-06T02:24:13.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.678+0000 7f933ffff700 1 -- 172.21.15.31:0/2809666796 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9340157580 con 0x7f934013fb20 2023-10-06T02:24:13.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.679+0000 7f9326ffd700 1 -- 172.21.15.31:0/2809666796 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f933401b480 con 0x7f934013fb20 2023-10-06T02:24:13.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.680+0000 7f9326ffd700 1 --2- 172.21.15.31:0/2809666796 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f932c066f30 0x7f932c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:13.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.680+0000 7f9326ffd700 1 -- 172.21.15.31:0/2809666796 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9334086340 con 0x7f934013fb20 2023-10-06T02:24:13.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.680+0000 7f933effd700 1 --2- 172.21.15.31:0/2809666796 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f932c066f30 0x7f932c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:13.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.681+0000 7f933effd700 1 --2- 172.21.15.31:0/2809666796 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f932c066f30 0x7f932c0693e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f93400c3190 tx=0x7f93300091a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:13.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.684+0000 7f9326ffd700 1 -- 172.21.15.31:0/2809666796 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f93340533b0 con 0x7f934013fb20 2023-10-06T02:24:13.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.888+0000 7f933ffff700 1 -- 172.21.15.31:0/2809666796 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f934004c3c0 con 0x7f934013fb20 2023-10-06T02:24:13.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.889+0000 7f9326ffd700 1 -- 172.21.15.31:0/2809666796 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16831 (secure 0 0 0) 0x7f93340569d0 con 0x7f934013fb20 2023-10-06T02:24:13.890 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:24:13.890 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":35,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:24:12.493604+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:24:10.026011+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-06T02:24:11.471970+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:24:13.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.891+0000 7f933ffff700 1 -- 172.21.15.31:0/2809666796 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f932c066f30 msgr2=0x7f932c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:13.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.891+0000 7f933ffff700 1 --2- 172.21.15.31:0/2809666796 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f932c066f30 0x7f932c0693e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f93400c3190 tx=0x7f93300091a0 comp rx=0 tx=0).stop 2023-10-06T02:24:13.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.891+0000 7f933ffff700 1 -- 172.21.15.31:0/2809666796 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f934013fb20 msgr2=0x7f9340158320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:13.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.891+0000 7f933ffff700 1 --2- 172.21.15.31:0/2809666796 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f934013fb20 0x7f9340158320 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f9334005fd0 tx=0x7f9334016000 comp rx=0 tx=0).stop 2023-10-06T02:24:13.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.891+0000 7f933ffff700 1 -- 172.21.15.31:0/2809666796 shutdown_connections 2023-10-06T02:24:13.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.891+0000 7f933ffff700 1 --2- 172.21.15.31:0/2809666796 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f934013fef0 0x7f9340155bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:13.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.891+0000 7f933ffff700 1 --2- 172.21.15.31:0/2809666796 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f932c066f30 0x7f932c0693e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:13.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.891+0000 7f933ffff700 1 --2- 172.21.15.31:0/2809666796 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f934006aaa0 0x7f93400c8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:13.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.891+0000 7f933ffff700 1 --2- 172.21.15.31:0/2809666796 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f934013fb20 0x7f9340158320 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:13.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.891+0000 7f933ffff700 1 -- 172.21.15.31:0/2809666796 >> 172.21.15.31:0/2809666796 conn(0x7f934006d170 msgr2=0x7f93400af980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:13.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.891+0000 7f933ffff700 1 -- 172.21.15.31:0/2809666796 shutdown_connections 2023-10-06T02:24:13.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:13.891+0000 7f933ffff700 1 -- 172.21.15.31:0/2809666796 wait complete. 2023-10-06T02:24:13.910 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T02:24:14.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:14.102+0000 7f019f59e700 1 Processor -- start 2023-10-06T02:24:14.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:14.102+0000 7f019f59e700 1 -- start start 2023-10-06T02:24:14.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:14.103+0000 7f019f59e700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f01a00cc5b0 0x7f01a00aaed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:14.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:14.103+0000 7f019f59e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f01a00ca270 0x7f01a006e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:14.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:14.103+0000 7f019f59e700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01a00ad780 con 0x7f01a00accb0 2023-10-06T02:24:14.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:14.103+0000 7f019f59e700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01a00cee60 con 0x7f01a00ca270 2023-10-06T02:24:14.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:14.103+0000 7f019f59e700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01a00cf140 con 0x7f01a00cc5b0 2023-10-06T02:24:14.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:14.103+0000 7f019ed9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f01a00ca270 0x7f01a006e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:14.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:14.103+0000 7f019ed9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f01a00ca270 0x7f01a006e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56652/0 (socket says 172.21.15.31:56652) 2023-10-06T02:24:14.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:14.103+0000 7f019ed9d700 1 -- 172.21.15.31:0/2844682491 learned_addr learned my addr 172.21.15.31:0/2844682491 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:14.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:14.103+0000 7f019dd9b700 1 --2- 172.21.15.31:0/2844682491 >> v2:172.21.15.62:3300/0 conn(0x7f01a00cc5b0 0x7f01a00aaed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:14.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:14.104+0000 7f019ed9d700 1 -- 172.21.15.31:0/2844682491 >> v2:172.21.15.62:3300/0 conn(0x7f01a00cc5b0 msgr2=0x7f01a00aaed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:14.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:14.104+0000 7f019ed9d700 1 --2- 172.21.15.31:0/2844682491 >> v2:172.21.15.62:3300/0 conn(0x7f01a00cc5b0 0x7f01a00aaed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:14.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:14.104+0000 7f019ed9d700 1 -- 172.21.15.31:0/2844682491 >> v1:172.21.15.62:6789/0 conn(0x7f01a00accb0 legacy=0x7f01a00cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:14.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:14.104+0000 7f019ed9d700 1 -- 172.21.15.31:0/2844682491 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01a006d850 con 0x7f01a00ca270 2023-10-06T02:24:14.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:14.104+0000 7f019dd9b700 1 --2- 172.21.15.31:0/2844682491 >> v2:172.21.15.62:3300/0 conn(0x7f01a00cc5b0 0x7f01a00aaed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:24:14.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:14.104+0000 7f019ed9d700 1 --2- 172.21.15.31:0/2844682491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01a00ca270 0x7f01a006e330 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f01a006ac60 tx=0x7f0198009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec47a9d5f8a35e5c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:14.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:14.104+0000 7f019d59a700 1 -- 172.21.15.31:0/2844682491 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f019800e070 con 0x7f01a00ca270 2023-10-06T02:24:14.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:14.104+0000 7f019d59a700 1 -- 172.21.15.31:0/2844682491 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f019800ba50 con 0x7f01a00ca270 2023-10-06T02:24:14.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:14.104+0000 7f019d59a700 1 -- 172.21.15.31:0/2844682491 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01980092e0 con 0x7f01a00ca270 2023-10-06T02:24:14.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:14.105+0000 7f019f59e700 1 -- 172.21.15.31:0/2844682491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01a00ca270 msgr2=0x7f01a006e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:14.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:14.105+0000 7f019f59e700 1 --2- 172.21.15.31:0/2844682491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01a00ca270 0x7f01a006e330 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f01a006ac60 tx=0x7f0198009840 comp rx=0 tx=0).stop 2023-10-06T02:24:14.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:14.105+0000 7f019f59e700 1 -- 172.21.15.31:0/2844682491 shutdown_connections 2023-10-06T02:24:14.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:14.105+0000 7f019f59e700 1 --2- 172.21.15.31:0/2844682491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01a00ca270 0x7f01a006e330 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:14.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:14.105+0000 7f019f59e700 1 --2- 172.21.15.31:0/2844682491 >> v2:172.21.15.62:3300/0 conn(0x7f01a00cc5b0 0x7f01a00aaed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:14.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:14.105+0000 7f019f59e700 1 -- 172.21.15.31:0/2844682491 >> 172.21.15.31:0/2844682491 conn(0x7f01a006ce80 msgr2=0x7f01a00ce370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:14.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:14.105+0000 7f019f59e700 1 -- 172.21.15.31:0/2844682491 shutdown_connections 2023-10-06T02:24:14.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:14.105+0000 7f019f59e700 1 -- 172.21.15.31:0/2844682491 wait complete. 2023-10-06T02:24:14.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:14.106+0000 7f019f59e700 1 Processor -- start 2023-10-06T02:24:14.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:14.107+0000 7f019f59e700 1 -- start start 2023-10-06T02:24:14.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:14.107+0000 7f019f59e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f01a00accb0 0x7f01a01528e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:14.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:14.107+0000 7f019f59e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01a00ca270 0x7f01a00b3120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:14.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:14.107+0000 7f019f59e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01a00cc5b0 0x7f01a00cadc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:14.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:14.107+0000 7f019f59e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a00b2c90 con 0x7f01a00ca270 2023-10-06T02:24:14.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:14.107+0000 7f019f59e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a0152e20 con 0x7f01a00cc5b0 2023-10-06T02:24:14.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:14.107+0000 7f019f59e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01a016da80 con 0x7f01a00accb0 2023-10-06T02:24:14.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:14.108+0000 7f019dd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01a00ca270 0x7f01a00b3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:14.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:14.108+0000 7f019dd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01a00ca270 0x7f01a00b3120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56668/0 (socket says 172.21.15.31:56668) 2023-10-06T02:24:14.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:14.108+0000 7f019dd9b700 1 -- 172.21.15.31:0/1333571844 learned_addr learned my addr 172.21.15.31:0/1333571844 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:14.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:14.108+0000 7f019ed9d700 1 --2- 172.21.15.31:0/1333571844 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01a00cc5b0 0x7f01a00cadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:14.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:14.108+0000 7f019e59c700 1 --2- 172.21.15.31:0/1333571844 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f01a00accb0 0x7f01a01528e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:14.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:14.108+0000 7f019dd9b700 1 -- 172.21.15.31:0/1333571844 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f01a00accb0 msgr2=0x7f01a01528e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:14.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:14.108+0000 7f019dd9b700 1 --2- 172.21.15.31:0/1333571844 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f01a00accb0 0x7f01a01528e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:14.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:14.108+0000 7f019dd9b700 1 -- 172.21.15.31:0/1333571844 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01a00cc5b0 msgr2=0x7f01a00cadc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:14.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:14.108+0000 7f019dd9b700 1 --2- 172.21.15.31:0/1333571844 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01a00cc5b0 0x7f01a00cadc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:14.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:14.108+0000 7f019dd9b700 1 -- 172.21.15.31:0/1333571844 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0198009ae0 con 0x7f01a00ca270 2023-10-06T02:24:14.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:14.108+0000 7f019dd9b700 1 --2- 172.21.15.31:0/1333571844 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01a00ca270 0x7f01a00b3120 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f019400d890 tx=0x7f0194008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:14.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:14.109+0000 7f018e7fc700 1 -- 172.21.15.31:0/1333571844 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01940125f0 con 0x7f01a00ca270 2023-10-06T02:24:14.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:14.109+0000 7f018e7fc700 1 -- 172.21.15.31:0/1333571844 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0194012750 con 0x7f01a00ca270 2023-10-06T02:24:14.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:14.109+0000 7f019f59e700 1 -- 172.21.15.31:0/1333571844 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01a00b2640 con 0x7f01a00ca270 2023-10-06T02:24:14.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:14.109+0000 7f018e7fc700 1 -- 172.21.15.31:0/1333571844 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0194012b10 con 0x7f01a00ca270 2023-10-06T02:24:14.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:14.109+0000 7f019f59e700 1 -- 172.21.15.31:0/1333571844 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01a00b3660 con 0x7f01a00ca270 2023-10-06T02:24:14.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:14.110+0000 7f019f59e700 1 -- 172.21.15.31:0/1333571844 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01a004c3c0 con 0x7f01a00ca270 2023-10-06T02:24:14.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:14.111+0000 7f018e7fc700 1 -- 172.21.15.31:0/1333571844 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0194014430 con 0x7f01a00ca270 2023-10-06T02:24:14.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:14.112+0000 7f018e7fc700 1 --2- 172.21.15.31:0/1333571844 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f0180066f30 0x7f01800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:14.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:14.112+0000 7f018e7fc700 1 -- 172.21.15.31:0/1333571844 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0194084960 con 0x7f01a00ca270 2023-10-06T02:24:14.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:14.112+0000 7f019e59c700 1 --2- 172.21.15.31:0/1333571844 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f0180066f30 0x7f01800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:14.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:14.112+0000 7f019e59c700 1 --2- 172.21.15.31:0/1333571844 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f0180066f30 0x7f01800693e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f01900039e0 tx=0x7f0190007ec0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:14.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:14.116+0000 7f018e7fc700 1 -- 172.21.15.31:0/1333571844 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0194055500 con 0x7f01a00ca270 2023-10-06T02:24:14.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:14.367+0000 7f019f59e700 1 -- 172.21.15.31:0/1333571844 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f01a00b2dd0 con 0x7f01a00ca270 2023-10-06T02:24:15.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:15.026+0000 7f018e7fc700 1 -- 172.21.15.31:0/1333571844 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) v1 ==== 118+0+0 (secure 0 0 0) 0x7f0194055090 con 0x7f01a00ca270 2023-10-06T02:24:15.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:15.029+0000 7f019f59e700 1 -- 172.21.15.31:0/1333571844 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f0180066f30 msgr2=0x7f01800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:15.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:15.029+0000 7f019f59e700 1 --2- 172.21.15.31:0/1333571844 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f0180066f30 0x7f01800693e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f01900039e0 tx=0x7f0190007ec0 comp rx=0 tx=0).stop 2023-10-06T02:24:15.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:15.029+0000 7f019f59e700 1 -- 172.21.15.31:0/1333571844 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01a00ca270 msgr2=0x7f01a00b3120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:15.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:15.029+0000 7f019f59e700 1 --2- 172.21.15.31:0/1333571844 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01a00ca270 0x7f01a00b3120 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f019400d890 tx=0x7f0194008040 comp rx=0 tx=0).stop 2023-10-06T02:24:15.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:15.029+0000 7f019f59e700 1 -- 172.21.15.31:0/1333571844 shutdown_connections 2023-10-06T02:24:15.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:15.029+0000 7f019f59e700 1 --2- 172.21.15.31:0/1333571844 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01a00cc5b0 0x7f01a00cadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:15.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:15.029+0000 7f019f59e700 1 --2- 172.21.15.31:0/1333571844 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f0180066f30 0x7f01800693e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:15.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:15.029+0000 7f019f59e700 1 --2- 172.21.15.31:0/1333571844 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f01a00accb0 0x7f01a01528e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:15.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:15.029+0000 7f019f59e700 1 --2- 172.21.15.31:0/1333571844 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01a00ca270 0x7f01a00b3120 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:15.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:15.029+0000 7f019f59e700 1 -- 172.21.15.31:0/1333571844 >> 172.21.15.31:0/1333571844 conn(0x7f01a006ce80 msgr2=0x7f01a00cbea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:15.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:15.030+0000 7f019f59e700 1 -- 172.21.15.31:0/1333571844 shutdown_connections 2023-10-06T02:24:15.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:15.030+0000 7f019f59e700 1 -- 172.21.15.31:0/1333571844 wait complete. 2023-10-06T02:24:15.051 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T02:24:15.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:15.245+0000 7f1197fff700 1 Processor -- start 2023-10-06T02:24:15.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:15.245+0000 7f1197fff700 1 -- start start 2023-10-06T02:24:15.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:15.246+0000 7f1197fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1198153a00 0x7f1198153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:15.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:15.246+0000 7f1197fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f11980cea00 0x7f11980cee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:15.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:15.246+0000 7f1197fff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f119815ac50 con 0x7f119815de50 2023-10-06T02:24:15.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:15.246+0000 7f1197fff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f119815d010 con 0x7f11980cea00 2023-10-06T02:24:15.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:15.246+0000 7f1197fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f119815af60 con 0x7f1198153a00 2023-10-06T02:24:15.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:15.246+0000 7f11977fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f11980cea00 0x7f11980cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:15.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:15.246+0000 7f11977fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f11980cea00 0x7f11980cee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56684/0 (socket says 172.21.15.31:56684) 2023-10-06T02:24:15.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:15.246+0000 7f11977fe700 1 -- 172.21.15.31:0/169484530 learned_addr learned my addr 172.21.15.31:0/169484530 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:15.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:15.247+0000 7f11977fe700 1 -- 172.21.15.31:0/169484530 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1198153a00 msgr2=0x7f1198153dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:15.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:15.247+0000 7f1196ffd700 1 --2- 172.21.15.31:0/169484530 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1198153a00 0x7f1198153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:15.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:15.247+0000 7f11977fe700 1 --2- 172.21.15.31:0/169484530 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1198153a00 0x7f1198153dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:15.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:15.247+0000 7f11977fe700 1 -- 172.21.15.31:0/169484530 >> v1:172.21.15.62:6789/0 conn(0x7f119815de50 legacy=0x7f119815a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:15.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:15.247+0000 7f11977fe700 1 -- 172.21.15.31:0/169484530 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f119815b240 con 0x7f11980cea00 2023-10-06T02:24:15.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:15.247+0000 7f1196ffd700 1 --2- 172.21.15.31:0/169484530 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1198153a00 0x7f1198153dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:24:15.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:15.247+0000 7f11977fe700 1 --2- 172.21.15.31:0/169484530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11980cea00 0x7f11980cee40 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1190009f20 tx=0x7f1190009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76593fed6184c4cb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:15.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:15.247+0000 7f1195ffb700 1 -- 172.21.15.31:0/169484530 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f119000e070 con 0x7f11980cea00 2023-10-06T02:24:15.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:15.247+0000 7f1195ffb700 1 -- 172.21.15.31:0/169484530 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f119000ba40 con 0x7f11980cea00 2023-10-06T02:24:15.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:15.247+0000 7f1195ffb700 1 -- 172.21.15.31:0/169484530 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11900092e0 con 0x7f11980cea00 2023-10-06T02:24:15.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:15.248+0000 7f1197fff700 1 -- 172.21.15.31:0/169484530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11980cea00 msgr2=0x7f11980cee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:15.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:15.248+0000 7f1197fff700 1 --2- 172.21.15.31:0/169484530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11980cea00 0x7f11980cee40 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1190009f20 tx=0x7f1190009840 comp rx=0 tx=0).stop 2023-10-06T02:24:15.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:15.248+0000 7f1197fff700 1 -- 172.21.15.31:0/169484530 shutdown_connections 2023-10-06T02:24:15.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:15.248+0000 7f1197fff700 1 --2- 172.21.15.31:0/169484530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11980cea00 0x7f11980cee40 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:15.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:15.248+0000 7f1197fff700 1 --2- 172.21.15.31:0/169484530 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1198153a00 0x7f1198153dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:15.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:15.248+0000 7f1197fff700 1 -- 172.21.15.31:0/169484530 >> 172.21.15.31:0/169484530 conn(0x7f11980703a0 msgr2=0x7f11980707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:15.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:15.248+0000 7f1197fff700 1 -- 172.21.15.31:0/169484530 shutdown_connections 2023-10-06T02:24:15.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:15.248+0000 7f1197fff700 1 -- 172.21.15.31:0/169484530 wait complete. 2023-10-06T02:24:15.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:15.249+0000 7f1197fff700 1 Processor -- start 2023-10-06T02:24:15.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:15.250+0000 7f1197fff700 1 -- start start 2023-10-06T02:24:15.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:15.250+0000 7f1197fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11980cea00 0x7f1198150750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:15.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:15.251+0000 7f1197fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1198153a00 0x7f11980c6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:15.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:15.251+0000 7f1197fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f119815de50 0x7f1198152150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:15.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:15.251+0000 7f1197fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1198150de0 con 0x7f119815de50 2023-10-06T02:24:15.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:15.251+0000 7f1197fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f119814ebc0 con 0x7f11980cea00 2023-10-06T02:24:15.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:15.251+0000 7f1197fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11980c7360 con 0x7f1198153a00 2023-10-06T02:24:15.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:15.251+0000 7f11977fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f119815de50 0x7f1198152150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:15.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:15.251+0000 7f11977fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f119815de50 0x7f1198152150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56688/0 (socket says 172.21.15.31:56688) 2023-10-06T02:24:15.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:15.251+0000 7f11977fe700 1 -- 172.21.15.31:0/305197054 learned_addr learned my addr 172.21.15.31:0/305197054 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:15.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:15.251+0000 7f1196ffd700 1 --2- 172.21.15.31:0/305197054 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11980cea00 0x7f1198150750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:15.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:15.251+0000 7f11967fc700 1 --2- 172.21.15.31:0/305197054 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1198153a00 0x7f11980c6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:15.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:15.251+0000 7f11977fe700 1 -- 172.21.15.31:0/305197054 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1198153a00 msgr2=0x7f11980c6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:15.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:15.251+0000 7f11977fe700 1 --2- 172.21.15.31:0/305197054 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1198153a00 0x7f11980c6e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:15.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:15.251+0000 7f11977fe700 1 -- 172.21.15.31:0/305197054 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11980cea00 msgr2=0x7f1198150750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:15.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:15.251+0000 7f11977fe700 1 --2- 172.21.15.31:0/305197054 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11980cea00 0x7f1198150750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:15.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:15.251+0000 7f11977fe700 1 -- 172.21.15.31:0/305197054 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1190009ae0 con 0x7f119815de50 2023-10-06T02:24:15.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:15.251+0000 7f11977fe700 1 --2- 172.21.15.31:0/305197054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f119815de50 0x7f1198152150 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f119806eb00 tx=0x7f1190017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:15.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:15.252+0000 7f117effd700 1 -- 172.21.15.31:0/305197054 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11900032f0 con 0x7f119815de50 2023-10-06T02:24:15.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:15.252+0000 7f117effd700 1 -- 172.21.15.31:0/305197054 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1190012db0 con 0x7f119815de50 2023-10-06T02:24:15.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:15.252+0000 7f1197fff700 1 -- 172.21.15.31:0/305197054 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1198169df0 con 0x7f119815de50 2023-10-06T02:24:15.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:15.252+0000 7f117effd700 1 -- 172.21.15.31:0/305197054 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f119001d670 con 0x7f119815de50 2023-10-06T02:24:15.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:15.252+0000 7f1197fff700 1 -- 172.21.15.31:0/305197054 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11981694c0 con 0x7f119815de50 2023-10-06T02:24:15.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:15.253+0000 7f1197fff700 1 -- 172.21.15.31:0/305197054 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1198169f80 con 0x7f119815de50 2023-10-06T02:24:15.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:15.254+0000 7f117effd700 1 -- 172.21.15.31:0/305197054 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f119001da20 con 0x7f119815de50 2023-10-06T02:24:15.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:15.255+0000 7f117effd700 1 --2- 172.21.15.31:0/305197054 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f1184066f30 0x7f11840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:15.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:15.255+0000 7f117effd700 1 -- 172.21.15.31:0/305197054 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1190085fe0 con 0x7f119815de50 2023-10-06T02:24:15.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:15.255+0000 7f1196ffd700 1 --2- 172.21.15.31:0/305197054 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f1184066f30 0x7f11840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:15.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:15.255+0000 7f1196ffd700 1 --2- 172.21.15.31:0/305197054 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f1184066f30 0x7f11840693e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f119816a6f0 tx=0x7f1188006f90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:15.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:15.259+0000 7f117effd700 1 -- 172.21.15.31:0/305197054 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1190053070 con 0x7f119815de50 2023-10-06T02:24:15.278 INFO:tasks.ceph.mds.c.smithi031.stderr:2023-10-06T02:24:15.277+0000 7f2ef97f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T02:24:15.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:15.508+0000 7f1197fff700 1 -- 172.21.15.31:0/305197054 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f119804c3c0 con 0x7f119815de50 2023-10-06T02:24:16.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.029+0000 7f117effd700 1 -- 172.21.15.31:0/305197054 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1190056690 con 0x7f119815de50 2023-10-06T02:24:16.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.032+0000 7f1197fff700 1 -- 172.21.15.31:0/305197054 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f1184066f30 msgr2=0x7f11840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:16.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.032+0000 7f1197fff700 1 --2- 172.21.15.31:0/305197054 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f1184066f30 0x7f11840693e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f119816a6f0 tx=0x7f1188006f90 comp rx=0 tx=0).stop 2023-10-06T02:24:16.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.032+0000 7f1197fff700 1 -- 172.21.15.31:0/305197054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f119815de50 msgr2=0x7f1198152150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:16.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.032+0000 7f1197fff700 1 --2- 172.21.15.31:0/305197054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f119815de50 0x7f1198152150 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f119806eb00 tx=0x7f1190017000 comp rx=0 tx=0).stop 2023-10-06T02:24:16.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.032+0000 7f1197fff700 1 -- 172.21.15.31:0/305197054 shutdown_connections 2023-10-06T02:24:16.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.032+0000 7f1197fff700 1 --2- 172.21.15.31:0/305197054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f119815de50 0x7f1198152150 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:16.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.032+0000 7f1197fff700 1 --2- 172.21.15.31:0/305197054 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11980cea00 0x7f1198150750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:16.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.032+0000 7f1197fff700 1 --2- 172.21.15.31:0/305197054 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f1184066f30 0x7f11840693e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:16.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.032+0000 7f1197fff700 1 --2- 172.21.15.31:0/305197054 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1198153a00 0x7f11980c6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:16.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.032+0000 7f1197fff700 1 -- 172.21.15.31:0/305197054 >> 172.21.15.31:0/305197054 conn(0x7f11980703a0 msgr2=0x7f11980cedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:16.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.033+0000 7f1197fff700 1 -- 172.21.15.31:0/305197054 shutdown_connections 2023-10-06T02:24:16.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.033+0000 7f1197fff700 1 -- 172.21.15.31:0/305197054 wait complete. 2023-10-06T02:24:16.054 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:24:16.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.245+0000 7f5d3b59e700 1 Processor -- start 2023-10-06T02:24:16.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.245+0000 7f5d3b59e700 1 -- start start 2023-10-06T02:24:16.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.246+0000 7f5d3b59e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5d3c15ce40 0x7f5d3c0c7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:16.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.247+0000 7f5d3b59e700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d3c06d5b0 con 0x7f5d3c15ca70 2023-10-06T02:24:16.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.247+0000 7f5d3b59e700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d3c0b00b0 con 0x7f5d3c06b1e0 2023-10-06T02:24:16.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.247+0000 7f5d3b59e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3c06b6b0 con 0x7f5d3c15ce40 2023-10-06T02:24:16.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.247+0000 7f5d3a59c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f5d3c15ca70 0x7f5d3c06aad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:33032/0 (socket says 172.21.15.31:33032) 2023-10-06T02:24:16.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.247+0000 7f5d3a59c700 1 -- 172.21.15.31:0/928252325 learned_addr learned my addr 172.21.15.31:0/928252325 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:24:16.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.247+0000 7f5d3959a700 1 -- 172.21.15.31:0/928252325 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2023421697 0 0) 0x7f5d3c06d5b0 con 0x7f5d3c15ca70 2023-10-06T02:24:16.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.247+0000 7f5d3959a700 1 -- 172.21.15.31:0/928252325 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d24003570 con 0x7f5d3c15ca70 2023-10-06T02:24:16.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.247+0000 7f5d39d9b700 1 --2- 172.21.15.31:0/928252325 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5d3c15ce40 0x7f5d3c0c7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:16.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.247+0000 7f5d3959a700 1 -- 172.21.15.31:0/928252325 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2434165870 0 0) 0x7f5d24003570 con 0x7f5d3c15ca70 2023-10-06T02:24:16.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.248+0000 7f5d3959a700 1 -- 172.21.15.31:0/928252325 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5d3c15ce40 msgr2=0x7f5d3c0c7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:16.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.248+0000 7f5d3959a700 1 --2- 172.21.15.31:0/928252325 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5d3c15ce40 0x7f5d3c0c7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:16.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.248+0000 7f5d3959a700 1 -- 172.21.15.31:0/928252325 >> v1:172.21.15.62:6789/0 conn(0x7f5d3c06b1e0 legacy=0x7f5d3c0b0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:16.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.248+0000 7f5d3959a700 1 -- 172.21.15.31:0/928252325 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d3c06b990 con 0x7f5d3c15ca70 2023-10-06T02:24:16.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.248+0000 7f5d3959a700 1 -- 172.21.15.31:0/928252325 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7f5d2c001470 con 0x7f5d3c15ca70 2023-10-06T02:24:16.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.248+0000 7f5d3959a700 1 -- 172.21.15.31:0/928252325 >> v1:172.21.15.31:6789/0 conn(0x7f5d3c15ca70 legacy=0x7f5d3c06aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:16.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.248+0000 7f5d3959a700 1 --2- 172.21.15.31:0/928252325 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5d3c15ce40 0x7f5d24003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:16.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.248+0000 7f5d3959a700 1 --2- 172.21.15.31:0/928252325 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d24004110 0x7f5d24004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:16.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.248+0000 7f5d3959a700 1 --2- 172.21.15.31:0/928252325 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d24004e70 0x7f5d240052e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:16.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.248+0000 7f5d3959a700 1 -- 172.21.15.31:0/928252325 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3c06b6b0 con 0x7f5d24004e70 2023-10-06T02:24:16.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.248+0000 7f5d3959a700 1 -- 172.21.15.31:0/928252325 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2400bcc0 con 0x7f5d24004110 2023-10-06T02:24:16.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.248+0000 7f5d3959a700 1 -- 172.21.15.31:0/928252325 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2400be80 con 0x7f5d3c15ce40 2023-10-06T02:24:16.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.248+0000 7f5d3ad9d700 1 --2- 172.21.15.31:0/928252325 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d24004e70 0x7f5d240052e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:16.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.248+0000 7f5d3a59c700 1 --2- 172.21.15.31:0/928252325 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5d3c15ce40 0x7f5d24003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:16.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.248+0000 7f5d39d9b700 1 --2- 172.21.15.31:0/928252325 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d24004110 0x7f5d24004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:16.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.248+0000 7f5d3ad9d700 1 -- 172.21.15.31:0/928252325 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5d3c15ce40 msgr2=0x7f5d24003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:16.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.248+0000 7f5d3ad9d700 1 --2- 172.21.15.31:0/928252325 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5d3c15ce40 0x7f5d24003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:16.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.248+0000 7f5d3ad9d700 1 -- 172.21.15.31:0/928252325 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d24004110 msgr2=0x7f5d24004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:16.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.248+0000 7f5d3ad9d700 1 --2- 172.21.15.31:0/928252325 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d24004110 0x7f5d24004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:16.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.248+0000 7f5d3ad9d700 1 -- 172.21.15.31:0/928252325 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d24003570 con 0x7f5d24004e70 2023-10-06T02:24:16.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.248+0000 7f5d39d9b700 1 --2- 172.21.15.31:0/928252325 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d24004110 0x7f5d24004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:24:16.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.249+0000 7f5d3ad9d700 1 --2- 172.21.15.31:0/928252325 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d24004e70 0x7f5d240052e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5d34001f90 tx=0x7f5d34001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2386d87845f5761 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:16.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.249+0000 7f5d3959a700 1 -- 172.21.15.31:0/928252325 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d34001950 con 0x7f5d24004e70 2023-10-06T02:24:16.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.249+0000 7f5d3959a700 1 -- 172.21.15.31:0/928252325 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d3400c8b0 con 0x7f5d24004e70 2023-10-06T02:24:16.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.249+0000 7f5d3959a700 1 -- 172.21.15.31:0/928252325 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3400cc50 con 0x7f5d24004e70 2023-10-06T02:24:16.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.249+0000 7f5d3b59e700 1 -- 172.21.15.31:0/928252325 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d24004e70 msgr2=0x7f5d240052e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:16.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.249+0000 7f5d3b59e700 1 --2- 172.21.15.31:0/928252325 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d24004e70 0x7f5d240052e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5d34001f90 tx=0x7f5d34001640 comp rx=0 tx=0).stop 2023-10-06T02:24:16.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.249+0000 7f5d3ad9d700 1 -- 172.21.15.31:0/928252325 reap_dead start 2023-10-06T02:24:16.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.250+0000 7f5d3b59e700 1 -- 172.21.15.31:0/928252325 shutdown_connections 2023-10-06T02:24:16.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.250+0000 7f5d3b59e700 1 -- 172.21.15.31:0/928252325 >> 172.21.15.31:0/928252325 conn(0x7f5d3c0675a0 msgr2=0x7f5d3c0679a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:16.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.250+0000 7f5d3b59e700 1 -- 172.21.15.31:0/928252325 shutdown_connections 2023-10-06T02:24:16.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.250+0000 7f5d3b59e700 1 -- 172.21.15.31:0/928252325 wait complete. 2023-10-06T02:24:16.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.251+0000 7f5d3b59e700 1 Processor -- start 2023-10-06T02:24:16.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.251+0000 7f5d3b59e700 1 -- start start 2023-10-06T02:24:16.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.252+0000 7f5d3b59e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d24004e70 0x7f5d3c0cb850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:16.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.252+0000 7f5d3b59e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d3c15d250 0x7f5d3c0c1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:16.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.252+0000 7f5d3b59e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5d3c0c1b40 0x7f5d3c157120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:16.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.252+0000 7f5d3b59e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3c0cbd90 con 0x7f5d24004e70 2023-10-06T02:24:16.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.252+0000 7f5d3b59e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3c0ccb80 con 0x7f5d3c15d250 2023-10-06T02:24:16.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.252+0000 7f5d3b59e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3c0c23a0 con 0x7f5d3c0c1b40 2023-10-06T02:24:16.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.252+0000 7f5d3a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d24004e70 0x7f5d3c0cb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:16.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.252+0000 7f5d3a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d24004e70 0x7f5d3c0cb850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56716/0 (socket says 172.21.15.31:56716) 2023-10-06T02:24:16.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.252+0000 7f5d3a59c700 1 -- 172.21.15.31:0/3861723544 learned_addr learned my addr 172.21.15.31:0/3861723544 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:16.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.252+0000 7f5d3ad9d700 1 --2- 172.21.15.31:0/3861723544 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5d3c0c1b40 0x7f5d3c157120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:16.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.252+0000 7f5d39d9b700 1 --2- 172.21.15.31:0/3861723544 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d3c15d250 0x7f5d3c0c1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:16.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.252+0000 7f5d3a59c700 1 -- 172.21.15.31:0/3861723544 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5d3c0c1b40 msgr2=0x7f5d3c157120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:16.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.253+0000 7f5d3a59c700 1 --2- 172.21.15.31:0/3861723544 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5d3c0c1b40 0x7f5d3c157120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:16.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.253+0000 7f5d3a59c700 1 -- 172.21.15.31:0/3861723544 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d3c15d250 msgr2=0x7f5d3c0c1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:16.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.253+0000 7f5d3a59c700 1 --2- 172.21.15.31:0/3861723544 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d3c15d250 0x7f5d3c0c1600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:16.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.253+0000 7f5d3a59c700 1 -- 172.21.15.31:0/3861723544 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d3c0c20c0 con 0x7f5d24004e70 2023-10-06T02:24:16.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.253+0000 7f5d3ad9d700 1 --2- 172.21.15.31:0/3861723544 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5d3c0c1b40 0x7f5d3c157120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:24:16.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.253+0000 7f5d3a59c700 1 --2- 172.21.15.31:0/3861723544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d24004e70 0x7f5d3c0cb850 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c007390 tx=0x7f5d2c001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:16.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.253+0000 7f5d2a7fc700 1 -- 172.21.15.31:0/3861723544 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d2c00f870 con 0x7f5d24004e70 2023-10-06T02:24:16.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.253+0000 7f5d2a7fc700 1 -- 172.21.15.31:0/3861723544 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d2c00f9d0 con 0x7f5d24004e70 2023-10-06T02:24:16.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.253+0000 7f5d2a7fc700 1 -- 172.21.15.31:0/3861723544 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d2c00fd90 con 0x7f5d24004e70 2023-10-06T02:24:16.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.253+0000 7f5d3b59e700 1 -- 172.21.15.31:0/3861723544 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d3c0cc530 con 0x7f5d24004e70 2023-10-06T02:24:16.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.253+0000 7f5d3b59e700 1 -- 172.21.15.31:0/3861723544 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d3c15d6c0 con 0x7f5d24004e70 2023-10-06T02:24:16.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.254+0000 7f5d3b59e700 1 -- 172.21.15.31:0/3861723544 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d3c0cc9d0 con 0x7f5d24004e70 2023-10-06T02:24:16.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.255+0000 7f5d2a7fc700 1 -- 172.21.15.31:0/3861723544 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5d2c013600 con 0x7f5d24004e70 2023-10-06T02:24:16.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.256+0000 7f5d2a7fc700 1 --2- 172.21.15.31:0/3861723544 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5d1c066f30 0x7f5d1c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:16.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.256+0000 7f5d2a7fc700 1 -- 172.21.15.31:0/3861723544 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f5d2c084860 con 0x7f5d24004e70 2023-10-06T02:24:16.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.256+0000 7f5d39d9b700 1 --2- 172.21.15.31:0/3861723544 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5d1c066f30 0x7f5d1c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:16.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.257+0000 7f5d39d9b700 1 --2- 172.21.15.31:0/3861723544 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5d1c066f30 0x7f5d1c0693e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c0c3540 tx=0x7f5d30008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:16.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.260+0000 7f5d2a7fc700 1 -- 172.21.15.31:0/3861723544 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5d2c051970 con 0x7f5d24004e70 2023-10-06T02:24:16.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.506+0000 7f5d3b59e700 1 -- 172.21.15.31:0/3861723544 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5d3c04c3c0 con 0x7f5d24004e70 2023-10-06T02:24:16.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.506+0000 7f5d2a7fc700 1 -- 172.21.15.31:0/3861723544 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4591 (secure 0 0 0) 0x7f5d2c054f90 con 0x7f5d24004e70 2023-10-06T02:24:16.508 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:24:16.508 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4700,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/384287452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":384287452},{"type":"v1","addr":"172.21.15.62:6834","nonce":384287452}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5037,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/693643908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":693643908},{"type":"v1","addr":"172.21.15.31:6836","nonce":693643908}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5040,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2599441423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2599441423},{"type":"v1","addr":"172.21.15.62:6835","nonce":2599441423}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:24:12.493929+0000","modified":"2023-10-06T02:24:16.026454+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5043},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5043":{"gid":5043,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.31:6837/2608294748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":2608294748},{"type":"v1","addr":"172.21.15.31:6837","nonce":2608294748}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T02:24:16.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.509+0000 7f5d3b59e700 1 -- 172.21.15.31:0/3861723544 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5d1c066f30 msgr2=0x7f5d1c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:16.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.509+0000 7f5d3b59e700 1 --2- 172.21.15.31:0/3861723544 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5d1c066f30 0x7f5d1c0693e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c0c3540 tx=0x7f5d30008040 comp rx=0 tx=0).stop 2023-10-06T02:24:16.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.509+0000 7f5d3b59e700 1 -- 172.21.15.31:0/3861723544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d24004e70 msgr2=0x7f5d3c0cb850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:16.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.509+0000 7f5d3b59e700 1 --2- 172.21.15.31:0/3861723544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d24004e70 0x7f5d3c0cb850 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c007390 tx=0x7f5d2c001d80 comp rx=0 tx=0).stop 2023-10-06T02:24:16.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.509+0000 7f5d3b59e700 1 -- 172.21.15.31:0/3861723544 shutdown_connections 2023-10-06T02:24:16.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.509+0000 7f5d3b59e700 1 --2- 172.21.15.31:0/3861723544 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5d1c066f30 0x7f5d1c0693e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:16.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.509+0000 7f5d3b59e700 1 --2- 172.21.15.31:0/3861723544 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5d3c0c1b40 0x7f5d3c157120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:16.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.509+0000 7f5d3b59e700 1 --2- 172.21.15.31:0/3861723544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d24004e70 0x7f5d3c0cb850 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:16.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.509+0000 7f5d3b59e700 1 --2- 172.21.15.31:0/3861723544 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d3c15d250 0x7f5d3c0c1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:16.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.509+0000 7f5d3b59e700 1 -- 172.21.15.31:0/3861723544 >> 172.21.15.31:0/3861723544 conn(0x7f5d3c0675a0 msgr2=0x7f5d3c06d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:16.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.509+0000 7f5d3b59e700 1 -- 172.21.15.31:0/3861723544 shutdown_connections 2023-10-06T02:24:16.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.509+0000 7f5d3b59e700 1 -- 172.21.15.31:0/3861723544 wait complete. 2023-10-06T02:24:16.512 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 17 2023-10-06T02:24:16.528 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:24:16.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.712+0000 7ff242b9d700 1 Processor -- start 2023-10-06T02:24:16.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.713+0000 7ff242b9d700 1 -- start start 2023-10-06T02:24:16.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.714+0000 7ff242b9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff23c13fb20 0x7ff23c15a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:16.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.714+0000 7ff242b9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff23c15ae70 con 0x7ff23c06aaa0 2023-10-06T02:24:16.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.714+0000 7ff242b9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff23c06af60 con 0x7ff23c13fef0 2023-10-06T02:24:16.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.714+0000 7ff242b9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff23c06b240 con 0x7ff23c13fb20 2023-10-06T02:24:16.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.714+0000 7ff24239c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7ff23c06aaa0 0x7ff23c0c9d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:33048/0 (socket says 172.21.15.31:33048) 2023-10-06T02:24:16.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.714+0000 7ff24239c700 1 -- 172.21.15.31:0/2494407300 learned_addr learned my addr 172.21.15.31:0/2494407300 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:24:16.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.714+0000 7ff241b9b700 1 --2- 172.21.15.31:0/2494407300 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff23c13fb20 0x7ff23c15a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:16.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.715+0000 7ff240b99700 1 -- 172.21.15.31:0/2494407300 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 566735826 0 0) 0x7ff234001490 con 0x7ff23c06aaa0 2023-10-06T02:24:16.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.715+0000 7ff240b99700 1 -- 172.21.15.31:0/2494407300 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff228003570 con 0x7ff23c06aaa0 2023-10-06T02:24:16.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.715+0000 7ff240b99700 1 -- 172.21.15.31:0/2494407300 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 390365702 0 0) 0x7ff23c06af60 con 0x7ff23c13fef0 2023-10-06T02:24:16.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.715+0000 7ff241b9b700 1 -- 172.21.15.31:0/2494407300 >> v1:172.21.15.62:6789/0 conn(0x7ff23c13fef0 legacy=0x7ff23c0c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:16.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.715+0000 7ff241b9b700 1 -- 172.21.15.31:0/2494407300 >> v1:172.21.15.31:6789/0 conn(0x7ff23c06aaa0 legacy=0x7ff23c0c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:16.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.715+0000 7ff241b9b700 1 -- 172.21.15.31:0/2494407300 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff23c06b520 con 0x7ff23c13fb20 2023-10-06T02:24:16.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.715+0000 7ff241b9b700 1 --2- 172.21.15.31:0/2494407300 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff23c13fb20 0x7ff23c15a930 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff238009910 tx=0x7ff2380091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b60cb2471efca61 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:16.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.716+0000 7ff240b99700 1 -- 172.21.15.31:0/2494407300 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff23800b680 con 0x7ff23c13fb20 2023-10-06T02:24:16.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.716+0000 7ff240b99700 1 -- 172.21.15.31:0/2494407300 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff23800b7e0 con 0x7ff23c13fb20 2023-10-06T02:24:16.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.716+0000 7ff240b99700 1 -- 172.21.15.31:0/2494407300 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff23800bba0 con 0x7ff23c13fb20 2023-10-06T02:24:16.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.716+0000 7ff242b9d700 1 -- 172.21.15.31:0/2494407300 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff23c13fb20 msgr2=0x7ff23c15a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:16.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.716+0000 7ff242b9d700 1 --2- 172.21.15.31:0/2494407300 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff23c13fb20 0x7ff23c15a930 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff238009910 tx=0x7ff2380091a0 comp rx=0 tx=0).stop 2023-10-06T02:24:16.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.716+0000 7ff242b9d700 1 -- 172.21.15.31:0/2494407300 shutdown_connections 2023-10-06T02:24:16.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.716+0000 7ff242b9d700 1 --2- 172.21.15.31:0/2494407300 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff23c13fb20 0x7ff23c15a930 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:16.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.716+0000 7ff242b9d700 1 -- 172.21.15.31:0/2494407300 >> 172.21.15.31:0/2494407300 conn(0x7ff23c06d170 msgr2=0x7ff23c0cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:16.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.716+0000 7ff242b9d700 1 -- 172.21.15.31:0/2494407300 shutdown_connections 2023-10-06T02:24:16.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.716+0000 7ff242b9d700 1 -- 172.21.15.31:0/2494407300 wait complete. 2023-10-06T02:24:16.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.718+0000 7ff242b9d700 1 Processor -- start 2023-10-06T02:24:16.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.718+0000 7ff242b9d700 1 -- start start 2023-10-06T02:24:16.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.719+0000 7ff242b9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff23c06aaa0 0x7ff23c158610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:16.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.719+0000 7ff242b9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff23c13fb20 0x7ff23c0b17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:16.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.719+0000 7ff242b9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff23c13fef0 0x7ff23c16d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:16.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.719+0000 7ff242b9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff23c0ae480 con 0x7ff23c13fb20 2023-10-06T02:24:16.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.719+0000 7ff242b9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff23c158b50 con 0x7ff23c13fef0 2023-10-06T02:24:16.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.719+0000 7ff242b9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff23c156310 con 0x7ff23c06aaa0 2023-10-06T02:24:16.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.719+0000 7ff24139a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff23c13fb20 0x7ff23c0b17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:16.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.719+0000 7ff24139a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff23c13fb20 0x7ff23c0b17e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56726/0 (socket says 172.21.15.31:56726) 2023-10-06T02:24:16.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.719+0000 7ff24139a700 1 -- 172.21.15.31:0/1148643997 learned_addr learned my addr 172.21.15.31:0/1148643997 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:16.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.719+0000 7ff24139a700 1 -- 172.21.15.31:0/1148643997 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff23c06aaa0 msgr2=0x7ff23c158610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:16.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.719+0000 7ff24139a700 1 --2- 172.21.15.31:0/1148643997 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff23c06aaa0 0x7ff23c158610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:16.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.719+0000 7ff24139a700 1 -- 172.21.15.31:0/1148643997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff23c13fef0 msgr2=0x7ff23c16d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:16.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.719+0000 7ff24239c700 1 --2- 172.21.15.31:0/1148643997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff23c13fef0 0x7ff23c16d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:16.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.719+0000 7ff24139a700 1 --2- 172.21.15.31:0/1148643997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff23c13fef0 0x7ff23c16d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:16.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.720+0000 7ff24139a700 1 -- 172.21.15.31:0/1148643997 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff238009420 con 0x7ff23c13fb20 2023-10-06T02:24:16.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.720+0000 7ff24239c700 1 --2- 172.21.15.31:0/1148643997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff23c13fef0 0x7ff23c16d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:24:16.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.720+0000 7ff24139a700 1 --2- 172.21.15.31:0/1148643997 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff23c13fb20 0x7ff23c0b17e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ff22c001f00 tx=0x7ff22c001060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:16.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.720+0000 7ff231ffb700 1 -- 172.21.15.31:0/1148643997 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff22c011590 con 0x7ff23c13fb20 2023-10-06T02:24:16.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.720+0000 7ff231ffb700 1 -- 172.21.15.31:0/1148643997 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff22c0116f0 con 0x7ff23c13fb20 2023-10-06T02:24:16.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.720+0000 7ff231ffb700 1 -- 172.21.15.31:0/1148643997 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff22c011ab0 con 0x7ff23c13fb20 2023-10-06T02:24:16.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.720+0000 7ff242b9d700 1 -- 172.21.15.31:0/1148643997 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff23c158180 con 0x7ff23c13fb20 2023-10-06T02:24:16.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.720+0000 7ff242b9d700 1 -- 172.21.15.31:0/1148643997 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff23c157e10 con 0x7ff23c13fb20 2023-10-06T02:24:16.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.721+0000 7ff242b9d700 1 -- 172.21.15.31:0/1148643997 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff23c04c3c0 con 0x7ff23c13fb20 2023-10-06T02:24:16.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.722+0000 7ff231ffb700 1 -- 172.21.15.31:0/1148643997 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff22c01b3b0 con 0x7ff23c13fb20 2023-10-06T02:24:16.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.723+0000 7ff231ffb700 1 --2- 172.21.15.31:0/1148643997 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7ff220066f30 0x7ff2200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:16.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.723+0000 7ff231ffb700 1 -- 172.21.15.31:0/1148643997 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff22c083b30 con 0x7ff23c13fb20 2023-10-06T02:24:16.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.723+0000 7ff241b9b700 1 --2- 172.21.15.31:0/1148643997 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7ff220066f30 0x7ff2200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:16.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.724+0000 7ff241b9b700 1 --2- 172.21.15.31:0/1148643997 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7ff220066f30 0x7ff2200693e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff2380091e0 tx=0x7ff238017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:16.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.727+0000 7ff231ffb700 1 -- 172.21.15.31:0/1148643997 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff22c050cf0 con 0x7ff23c13fb20 2023-10-06T02:24:16.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.930+0000 7ff242b9d700 1 -- 172.21.15.31:0/1148643997 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff23c157ad0 con 0x7ff23c13fb20 2023-10-06T02:24:16.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.931+0000 7ff231ffb700 1 -- 172.21.15.31:0/1148643997 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+16818 (secure 0 0 0) 0x7ff22c017070 con 0x7ff23c13fb20 2023-10-06T02:24:16.933 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:24:16.933 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":36,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:24:14.007126+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:24:10.026011+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-06T02:24:11.471970+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:24:16.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.934+0000 7ff242b9d700 1 -- 172.21.15.31:0/1148643997 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7ff220066f30 msgr2=0x7ff2200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:16.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.934+0000 7ff242b9d700 1 --2- 172.21.15.31:0/1148643997 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7ff220066f30 0x7ff2200693e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff2380091e0 tx=0x7ff238017000 comp rx=0 tx=0).stop 2023-10-06T02:24:16.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.934+0000 7ff242b9d700 1 -- 172.21.15.31:0/1148643997 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff23c13fb20 msgr2=0x7ff23c0b17e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:16.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.934+0000 7ff242b9d700 1 --2- 172.21.15.31:0/1148643997 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff23c13fb20 0x7ff23c0b17e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ff22c001f00 tx=0x7ff22c001060 comp rx=0 tx=0).stop 2023-10-06T02:24:16.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.934+0000 7ff242b9d700 1 -- 172.21.15.31:0/1148643997 shutdown_connections 2023-10-06T02:24:16.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.934+0000 7ff242b9d700 1 --2- 172.21.15.31:0/1148643997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff23c13fef0 0x7ff23c16d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:16.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.934+0000 7ff242b9d700 1 --2- 172.21.15.31:0/1148643997 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7ff220066f30 0x7ff2200693e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:16.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.934+0000 7ff242b9d700 1 --2- 172.21.15.31:0/1148643997 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff23c06aaa0 0x7ff23c158610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:16.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.934+0000 7ff242b9d700 1 --2- 172.21.15.31:0/1148643997 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff23c13fb20 0x7ff23c0b17e0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:16.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.934+0000 7ff242b9d700 1 -- 172.21.15.31:0/1148643997 >> 172.21.15.31:0/1148643997 conn(0x7ff23c06d170 msgr2=0x7ff23c0c1200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:16.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.934+0000 7ff242b9d700 1 -- 172.21.15.31:0/1148643997 shutdown_connections 2023-10-06T02:24:16.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:16.934+0000 7ff242b9d700 1 -- 172.21.15.31:0/1148643997 wait complete. 2023-10-06T02:24:16.952 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T02:24:16.953 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T02:24:17.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.138+0000 7fb5affff700 1 Processor -- start 2023-10-06T02:24:17.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.138+0000 7fb5affff700 1 -- start start 2023-10-06T02:24:17.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.139+0000 7fb5affff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb5b0153c30 0x7fb5b015a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:17.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.139+0000 7fb5affff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb5b015de50 0x7fb5b00cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:17.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.139+0000 7fb5affff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb5b00cef40 0x7fb5b006d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:17.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.139+0000 7fb5affff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5b015afd0 con 0x7fb5b015de50 2023-10-06T02:24:17.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.140+0000 7fb5affff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5b015b2b0 con 0x7fb5b00cef40 2023-10-06T02:24:17.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.140+0000 7fb5affff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b015b590 con 0x7fb5b0153c30 2023-10-06T02:24:17.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.140+0000 7fb5ae7fc700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb5b015de50 0x7fb5b00cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:17.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.140+0000 7fb5ae7fc700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb5b015de50 0x7fb5b00cea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56738/0 (socket says 172.21.15.31:56738) 2023-10-06T02:24:17.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.140+0000 7fb5ae7fc700 1 -- 172.21.15.31:0/2060593040 learned_addr learned my addr 172.21.15.31:0/2060593040 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:17.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.140+0000 7fb5ae7fc700 1 -- 172.21.15.31:0/2060593040 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb5b0153c30 msgr2=0x7fb5b015a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:17.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.140+0000 7fb5ae7fc700 1 --2- 172.21.15.31:0/2060593040 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb5b0153c30 0x7fb5b015a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:17.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.140+0000 7fb5ae7fc700 1 -- 172.21.15.31:0/2060593040 >> v2:172.21.15.62:3300/0 conn(0x7fb5b00cef40 msgr2=0x7fb5b006d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:17.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.140+0000 7fb5ae7fc700 1 --2- 172.21.15.31:0/2060593040 >> v2:172.21.15.62:3300/0 conn(0x7fb5b00cef40 0x7fb5b006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:17.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.140+0000 7fb5ae7fc700 1 -- 172.21.15.31:0/2060593040 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5b015b870 con 0x7fb5b015de50 2023-10-06T02:24:17.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.140+0000 7fb5ae7fc700 1 --2- 172.21.15.31:0/2060593040 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5b015de50 0x7fb5b00cea00 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb5b0071370 tx=0x7fb5a4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc123f1d5395b8d8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:17.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.141+0000 7fb5adffb700 1 -- 172.21.15.31:0/2060593040 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5a400b860 con 0x7fb5b015de50 2023-10-06T02:24:17.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.141+0000 7fb5adffb700 1 -- 172.21.15.31:0/2060593040 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5a400b9c0 con 0x7fb5b015de50 2023-10-06T02:24:17.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.141+0000 7fb5adffb700 1 -- 172.21.15.31:0/2060593040 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5a400bd60 con 0x7fb5b015de50 2023-10-06T02:24:17.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.141+0000 7fb5affff700 1 -- 172.21.15.31:0/2060593040 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5b015de50 msgr2=0x7fb5b00cea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:17.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.141+0000 7fb5affff700 1 --2- 172.21.15.31:0/2060593040 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5b015de50 0x7fb5b00cea00 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb5b0071370 tx=0x7fb5a4009250 comp rx=0 tx=0).stop 2023-10-06T02:24:17.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.141+0000 7fb5affff700 1 -- 172.21.15.31:0/2060593040 shutdown_connections 2023-10-06T02:24:17.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.141+0000 7fb5affff700 1 --2- 172.21.15.31:0/2060593040 >> v2:172.21.15.62:3300/0 conn(0x7fb5b00cef40 0x7fb5b006d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:17.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.141+0000 7fb5affff700 1 --2- 172.21.15.31:0/2060593040 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb5b0153c30 0x7fb5b015a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:17.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.141+0000 7fb5affff700 1 --2- 172.21.15.31:0/2060593040 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5b015de50 0x7fb5b00cea00 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:17.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.141+0000 7fb5affff700 1 -- 172.21.15.31:0/2060593040 >> 172.21.15.31:0/2060593040 conn(0x7fb5b00703a0 msgr2=0x7fb5b00707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:17.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.141+0000 7fb5affff700 1 -- 172.21.15.31:0/2060593040 shutdown_connections 2023-10-06T02:24:17.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.141+0000 7fb5affff700 1 -- 172.21.15.31:0/2060593040 wait complete. 2023-10-06T02:24:17.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.143+0000 7fb5affff700 1 Processor -- start 2023-10-06T02:24:17.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.143+0000 7fb5affff700 1 -- start start 2023-10-06T02:24:17.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.144+0000 7fb5affff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5b00cef40 0x7fb5b0141290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:17.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.144+0000 7fb5affff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5b0153c30 0x7fb5b00c52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:17.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.144+0000 7fb5affff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb5b015de50 0x7fb5b00c5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:17.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.144+0000 7fb5affff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b016cf10 con 0x7fb5b0153c30 2023-10-06T02:24:17.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.144+0000 7fb5affff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b01417d0 con 0x7fb5b00cef40 2023-10-06T02:24:17.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.144+0000 7fb5affff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b014fce0 con 0x7fb5b015de50 2023-10-06T02:24:17.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.144+0000 7fb5ae7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5b0153c30 0x7fb5b00c52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:17.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.144+0000 7fb5ae7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5b0153c30 0x7fb5b00c52d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56752/0 (socket says 172.21.15.31:56752) 2023-10-06T02:24:17.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.144+0000 7fb5ae7fc700 1 -- 172.21.15.31:0/2761683371 learned_addr learned my addr 172.21.15.31:0/2761683371 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:17.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.144+0000 7fb5aeffd700 1 --2- 172.21.15.31:0/2761683371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5b00cef40 0x7fb5b0141290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:17.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.144+0000 7fb5af7fe700 1 --2- 172.21.15.31:0/2761683371 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb5b015de50 0x7fb5b00c5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:17.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.144+0000 7fb5ae7fc700 1 -- 172.21.15.31:0/2761683371 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb5b015de50 msgr2=0x7fb5b00c5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:17.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.144+0000 7fb5ae7fc700 1 --2- 172.21.15.31:0/2761683371 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb5b015de50 0x7fb5b00c5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:17.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.144+0000 7fb5ae7fc700 1 -- 172.21.15.31:0/2761683371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5b00cef40 msgr2=0x7fb5b0141290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:17.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.144+0000 7fb5ae7fc700 1 --2- 172.21.15.31:0/2761683371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5b00cef40 0x7fb5b0141290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:17.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.144+0000 7fb5ae7fc700 1 -- 172.21.15.31:0/2761683371 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5a40094f0 con 0x7fb5b0153c30 2023-10-06T02:24:17.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.145+0000 7fb5af7fe700 1 --2- 172.21.15.31:0/2761683371 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb5b015de50 0x7fb5b00c5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:24:17.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.145+0000 7fb5ae7fc700 1 --2- 172.21.15.31:0/2761683371 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5b0153c30 0x7fb5b00c52d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb5a40090b0 tx=0x7fb5a4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:17.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.145+0000 7fb596ffd700 1 -- 172.21.15.31:0/2761683371 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5a4011e20 con 0x7fb5b0153c30 2023-10-06T02:24:17.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.145+0000 7fb596ffd700 1 -- 172.21.15.31:0/2761683371 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5a401a630 con 0x7fb5b0153c30 2023-10-06T02:24:17.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.145+0000 7fb5affff700 1 -- 172.21.15.31:0/2761683371 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5b00b5ff0 con 0x7fb5b0153c30 2023-10-06T02:24:17.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.145+0000 7fb596ffd700 1 -- 172.21.15.31:0/2761683371 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5a401a9f0 con 0x7fb5b0153c30 2023-10-06T02:24:17.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.145+0000 7fb5affff700 1 -- 172.21.15.31:0/2761683371 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5b016d440 con 0x7fb5b0153c30 2023-10-06T02:24:17.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.146+0000 7fb5affff700 1 -- 172.21.15.31:0/2761683371 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5b004c3c0 con 0x7fb5b0153c30 2023-10-06T02:24:17.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.147+0000 7fb596ffd700 1 -- 172.21.15.31:0/2761683371 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb5a40253f0 con 0x7fb5b0153c30 2023-10-06T02:24:17.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.148+0000 7fb596ffd700 1 --2- 172.21.15.31:0/2761683371 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fb59c066f30 0x7fb59c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:17.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.148+0000 7fb596ffd700 1 -- 172.21.15.31:0/2761683371 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb5a4086160 con 0x7fb5b0153c30 2023-10-06T02:24:17.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.148+0000 7fb5aeffd700 1 --2- 172.21.15.31:0/2761683371 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fb59c066f30 0x7fb59c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:17.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.149+0000 7fb5aeffd700 1 --2- 172.21.15.31:0/2761683371 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fb59c066f30 0x7fb59c0693e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb5b00c7820 tx=0x7fb5a0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:17.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.152+0000 7fb596ffd700 1 -- 172.21.15.31:0/2761683371 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb5a40531d0 con 0x7fb5b0153c30 2023-10-06T02:24:17.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.350+0000 7fb5affff700 1 -- 172.21.15.31:0/2761683371 --> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb5b00c6210 con 0x7fb59c066f30 2023-10-06T02:24:17.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.360+0000 7fb596ffd700 1 -- 172.21.15.31:0/2761683371 <== mgr.4107 v2:172.21.15.31:6832/85292 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311059 (secure 0 0 0) 0x7fb5b00c6210 con 0x7fb59c066f30 2023-10-06T02:24:17.365 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:24:17.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.365+0000 7fb5affff700 1 -- 172.21.15.31:0/2761683371 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fb59c066f30 msgr2=0x7fb59c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:17.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.365+0000 7fb5affff700 1 --2- 172.21.15.31:0/2761683371 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fb59c066f30 0x7fb59c0693e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb5b00c7820 tx=0x7fb5a0008040 comp rx=0 tx=0).stop 2023-10-06T02:24:17.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.365+0000 7fb5affff700 1 -- 172.21.15.31:0/2761683371 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5b0153c30 msgr2=0x7fb5b00c52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:17.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.365+0000 7fb5affff700 1 --2- 172.21.15.31:0/2761683371 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5b0153c30 0x7fb5b00c52d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb5a40090b0 tx=0x7fb5a4017000 comp rx=0 tx=0).stop 2023-10-06T02:24:17.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.365+0000 7fb5affff700 1 -- 172.21.15.31:0/2761683371 shutdown_connections 2023-10-06T02:24:17.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.365+0000 7fb5affff700 1 --2- 172.21.15.31:0/2761683371 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fb59c066f30 0x7fb59c0693e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:17.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.365+0000 7fb5affff700 1 --2- 172.21.15.31:0/2761683371 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb5b015de50 0x7fb5b00c5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:17.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.365+0000 7fb5affff700 1 --2- 172.21.15.31:0/2761683371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5b00cef40 0x7fb5b0141290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:17.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.365+0000 7fb5affff700 1 --2- 172.21.15.31:0/2761683371 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5b0153c30 0x7fb5b00c52d0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:17.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.365+0000 7fb5affff700 1 -- 172.21.15.31:0/2761683371 >> 172.21.15.31:0/2761683371 conn(0x7fb5b00703a0 msgr2=0x7fb5b00c6c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:17.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.365+0000 7fb5affff700 1 -- 172.21.15.31:0/2761683371 shutdown_connections 2023-10-06T02:24:17.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.365+0000 7fb5affff700 1 -- 172.21.15.31:0/2761683371 wait complete. 2023-10-06T02:24:17.369 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-10-06T02:24:17.384 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":71,"stamp":"2023-10-06T02:24:17.130537+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":78,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2382508,"kb_used_data":4204,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752592212,"statfs":{"total":773094113280,"available":770654425088,"internally_reserved":0,"allocated":4304896,"data_stored":2438458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000839"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.031228+0000","last_change":"2023-10-06T02:24:13.031228+0000","last_active":"2023-10-06T02:24:13.031228+0000","last_peered":"2023-10-06T02:24:13.031228+0000","last_clean":"2023-10-06T02:24:13.031228+0000","last_became_active":"2023-10-06T02:24:13.029975+0000","last_became_peered":"2023-10-06T02:24:13.029975+0000","last_unstale":"2023-10-06T02:24:13.031228+0000","last_undegraded":"2023-10-06T02:24:13.031228+0000","last_fullsized":"2023-10-06T02:24:13.031228+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:27:04.362587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.022465+0000","last_change":"2023-10-06T02:24:13.022465+0000","last_active":"2023-10-06T02:24:13.022465+0000","last_peered":"2023-10-06T02:24:13.022465+0000","last_clean":"2023-10-06T02:24:13.022465+0000","last_became_active":"2023-10-06T02:24:13.022287+0000","last_became_peered":"2023-10-06T02:24:13.022287+0000","last_unstale":"2023-10-06T02:24:13.022465+0000","last_undegraded":"2023-10-06T02:24:13.022465+0000","last_fullsized":"2023-10-06T02:24:13.022465+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:33:34.790463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.032908+0000","last_change":"2023-10-06T02:24:13.032908+0000","last_active":"2023-10-06T02:24:13.032908+0000","last_peered":"2023-10-06T02:24:13.032908+0000","last_clean":"2023-10-06T02:24:13.032908+0000","last_became_active":"2023-10-06T02:24:13.032579+0000","last_became_peered":"2023-10-06T02:24:13.032579+0000","last_unstale":"2023-10-06T02:24:13.032908+0000","last_undegraded":"2023-10-06T02:24:13.032908+0000","last_fullsized":"2023-10-06T02:24:13.032908+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:14:08.469730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.024070+0000","last_change":"2023-10-06T02:24:13.024070+0000","last_active":"2023-10-06T02:24:13.024070+0000","last_peered":"2023-10-06T02:24:13.024070+0000","last_clean":"2023-10-06T02:24:13.024070+0000","last_became_active":"2023-10-06T02:24:13.023799+0000","last_became_peered":"2023-10-06T02:24:13.023799+0000","last_unstale":"2023-10-06T02:24:13.024070+0000","last_undegraded":"2023-10-06T02:24:13.024070+0000","last_fullsized":"2023-10-06T02:24:13.024070+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:46:19.025627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.012586+0000","last_change":"2023-10-06T02:24:13.012586+0000","last_active":"2023-10-06T02:24:13.012586+0000","last_peered":"2023-10-06T02:24:13.012586+0000","last_clean":"2023-10-06T02:24:13.012586+0000","last_became_active":"2023-10-06T02:24:13.012292+0000","last_became_peered":"2023-10-06T02:24:13.012292+0000","last_unstale":"2023-10-06T02:24:13.012586+0000","last_undegraded":"2023-10-06T02:24:13.012586+0000","last_fullsized":"2023-10-06T02:24:13.012586+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:40:05.543345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.030546+0000","last_change":"2023-10-06T02:24:13.030546+0000","last_active":"2023-10-06T02:24:13.030546+0000","last_peered":"2023-10-06T02:24:13.030546+0000","last_clean":"2023-10-06T02:24:13.030546+0000","last_became_active":"2023-10-06T02:24:13.030253+0000","last_became_peered":"2023-10-06T02:24:13.030253+0000","last_unstale":"2023-10-06T02:24:13.030546+0000","last_undegraded":"2023-10-06T02:24:13.030546+0000","last_fullsized":"2023-10-06T02:24:13.030546+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:05:41.683141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.025917+0000","last_change":"2023-10-06T02:24:13.025917+0000","last_active":"2023-10-06T02:24:13.025917+0000","last_peered":"2023-10-06T02:24:13.025917+0000","last_clean":"2023-10-06T02:24:13.025917+0000","last_became_active":"2023-10-06T02:24:13.025555+0000","last_became_peered":"2023-10-06T02:24:13.025555+0000","last_unstale":"2023-10-06T02:24:13.025917+0000","last_undegraded":"2023-10-06T02:24:13.025917+0000","last_fullsized":"2023-10-06T02:24:13.025917+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:51:52.519503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.035771+0000","last_change":"2023-10-06T02:24:13.035771+0000","last_active":"2023-10-06T02:24:13.035771+0000","last_peered":"2023-10-06T02:24:13.035771+0000","last_clean":"2023-10-06T02:24:13.035771+0000","last_became_active":"2023-10-06T02:24:13.035467+0000","last_became_peered":"2023-10-06T02:24:13.035467+0000","last_unstale":"2023-10-06T02:24:13.035771+0000","last_undegraded":"2023-10-06T02:24:13.035771+0000","last_fullsized":"2023-10-06T02:24:13.035771+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:14:08.469730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.032399+0000","last_change":"2023-10-06T02:24:13.032399+0000","last_active":"2023-10-06T02:24:13.032399+0000","last_peered":"2023-10-06T02:24:13.032399+0000","last_clean":"2023-10-06T02:24:13.032399+0000","last_became_active":"2023-10-06T02:24:13.031804+0000","last_became_peered":"2023-10-06T02:24:13.031804+0000","last_unstale":"2023-10-06T02:24:13.032399+0000","last_undegraded":"2023-10-06T02:24:13.032399+0000","last_fullsized":"2023-10-06T02:24:13.032399+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:05:07.253674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.027985+0000","last_change":"2023-10-06T02:24:13.027985+0000","last_active":"2023-10-06T02:24:13.027985+0000","last_peered":"2023-10-06T02:24:13.027985+0000","last_clean":"2023-10-06T02:24:13.027985+0000","last_became_active":"2023-10-06T02:24:13.027346+0000","last_became_peered":"2023-10-06T02:24:13.027346+0000","last_unstale":"2023-10-06T02:24:13.027985+0000","last_undegraded":"2023-10-06T02:24:13.027985+0000","last_fullsized":"2023-10-06T02:24:13.027985+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:14:08.469730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.012767+0000","last_change":"2023-10-06T02:24:13.012767+0000","last_active":"2023-10-06T02:24:13.012767+0000","last_peered":"2023-10-06T02:24:13.012767+0000","last_clean":"2023-10-06T02:24:13.012767+0000","last_became_active":"2023-10-06T02:24:13.012492+0000","last_became_peered":"2023-10-06T02:24:13.012492+0000","last_unstale":"2023-10-06T02:24:13.012767+0000","last_undegraded":"2023-10-06T02:24:13.012767+0000","last_fullsized":"2023-10-06T02:24:13.012767+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:06:39.375112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.021260+0000","last_change":"2023-10-06T02:24:13.021260+0000","last_active":"2023-10-06T02:24:13.021260+0000","last_peered":"2023-10-06T02:24:13.021260+0000","last_clean":"2023-10-06T02:24:13.021260+0000","last_became_active":"2023-10-06T02:24:13.021014+0000","last_became_peered":"2023-10-06T02:24:13.021014+0000","last_unstale":"2023-10-06T02:24:13.021260+0000","last_undegraded":"2023-10-06T02:24:13.021260+0000","last_fullsized":"2023-10-06T02:24:13.021260+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:08:46.284322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.026941+0000","last_change":"2023-10-06T02:24:13.026941+0000","last_active":"2023-10-06T02:24:13.026941+0000","last_peered":"2023-10-06T02:24:13.026941+0000","last_clean":"2023-10-06T02:24:13.026941+0000","last_became_active":"2023-10-06T02:24:13.026613+0000","last_became_peered":"2023-10-06T02:24:13.026613+0000","last_unstale":"2023-10-06T02:24:13.026941+0000","last_undegraded":"2023-10-06T02:24:13.026941+0000","last_fullsized":"2023-10-06T02:24:13.026941+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:08:46.284322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.023176+0000","last_change":"2023-10-06T02:24:13.023176+0000","last_active":"2023-10-06T02:24:13.023176+0000","last_peered":"2023-10-06T02:24:13.023176+0000","last_clean":"2023-10-06T02:24:13.023176+0000","last_became_active":"2023-10-06T02:24:13.015662+0000","last_became_peered":"2023-10-06T02:24:13.015662+0000","last_unstale":"2023-10-06T02:24:13.023176+0000","last_undegraded":"2023-10-06T02:24:13.023176+0000","last_fullsized":"2023-10-06T02:24:13.023176+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:55:41.830177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.012835+0000","last_change":"2023-10-06T02:24:13.012835+0000","last_active":"2023-10-06T02:24:13.012835+0000","last_peered":"2023-10-06T02:24:13.012835+0000","last_clean":"2023-10-06T02:24:13.012835+0000","last_became_active":"2023-10-06T02:24:13.012547+0000","last_became_peered":"2023-10-06T02:24:13.012547+0000","last_unstale":"2023-10-06T02:24:13.012835+0000","last_undegraded":"2023-10-06T02:24:13.012835+0000","last_fullsized":"2023-10-06T02:24:13.012835+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:33:34.790463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.023788+0000","last_change":"2023-10-06T02:24:13.023788+0000","last_active":"2023-10-06T02:24:13.023788+0000","last_peered":"2023-10-06T02:24:13.023788+0000","last_clean":"2023-10-06T02:24:13.023788+0000","last_became_active":"2023-10-06T02:24:13.023545+0000","last_became_peered":"2023-10-06T02:24:13.023545+0000","last_unstale":"2023-10-06T02:24:13.023788+0000","last_undegraded":"2023-10-06T02:24:13.023788+0000","last_fullsized":"2023-10-06T02:24:13.023788+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:14:08.469730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.035092+0000","last_change":"2023-10-06T02:24:13.035092+0000","last_active":"2023-10-06T02:24:13.035092+0000","last_peered":"2023-10-06T02:24:13.035092+0000","last_clean":"2023-10-06T02:24:13.035092+0000","last_became_active":"2023-10-06T02:24:13.034809+0000","last_became_peered":"2023-10-06T02:24:13.034809+0000","last_unstale":"2023-10-06T02:24:13.035092+0000","last_undegraded":"2023-10-06T02:24:13.035092+0000","last_fullsized":"2023-10-06T02:24:13.035092+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:41.374491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.011585+0000","last_change":"2023-10-06T02:24:13.011585+0000","last_active":"2023-10-06T02:24:13.011585+0000","last_peered":"2023-10-06T02:24:13.011585+0000","last_clean":"2023-10-06T02:24:13.011585+0000","last_became_active":"2023-10-06T02:24:13.011313+0000","last_became_peered":"2023-10-06T02:24:13.011313+0000","last_unstale":"2023-10-06T02:24:13.011585+0000","last_undegraded":"2023-10-06T02:24:13.011585+0000","last_fullsized":"2023-10-06T02:24:13.011585+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:53:51.425781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.014908+0000","last_change":"2023-10-06T02:24:13.014908+0000","last_active":"2023-10-06T02:24:13.014908+0000","last_peered":"2023-10-06T02:24:13.014908+0000","last_clean":"2023-10-06T02:24:13.014908+0000","last_became_active":"2023-10-06T02:24:13.014638+0000","last_became_peered":"2023-10-06T02:24:13.014638+0000","last_unstale":"2023-10-06T02:24:13.014908+0000","last_undegraded":"2023-10-06T02:24:13.014908+0000","last_fullsized":"2023-10-06T02:24:13.014908+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:21:14.321298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.011999+0000","last_change":"2023-10-06T02:24:13.011999+0000","last_active":"2023-10-06T02:24:13.011999+0000","last_peered":"2023-10-06T02:24:13.011999+0000","last_clean":"2023-10-06T02:24:13.011999+0000","last_became_active":"2023-10-06T02:24:13.011713+0000","last_became_peered":"2023-10-06T02:24:13.011713+0000","last_unstale":"2023-10-06T02:24:13.011999+0000","last_undegraded":"2023-10-06T02:24:13.011999+0000","last_fullsized":"2023-10-06T02:24:13.011999+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:40:05.543345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.017292+0000","last_change":"2023-10-06T02:24:13.017292+0000","last_active":"2023-10-06T02:24:13.017292+0000","last_peered":"2023-10-06T02:24:13.017292+0000","last_clean":"2023-10-06T02:24:13.017292+0000","last_became_active":"2023-10-06T02:24:13.016973+0000","last_became_peered":"2023-10-06T02:24:13.016973+0000","last_unstale":"2023-10-06T02:24:13.017292+0000","last_undegraded":"2023-10-06T02:24:13.017292+0000","last_fullsized":"2023-10-06T02:24:13.017292+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:27:04.362587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.025110+0000","last_change":"2023-10-06T02:24:13.025110+0000","last_active":"2023-10-06T02:24:13.025110+0000","last_peered":"2023-10-06T02:24:13.025110+0000","last_clean":"2023-10-06T02:24:13.025110+0000","last_became_active":"2023-10-06T02:24:13.024853+0000","last_became_peered":"2023-10-06T02:24:13.024853+0000","last_unstale":"2023-10-06T02:24:13.025110+0000","last_undegraded":"2023-10-06T02:24:13.025110+0000","last_fullsized":"2023-10-06T02:24:13.025110+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:27:04.362587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.030566+0000","last_change":"2023-10-06T02:24:13.030566+0000","last_active":"2023-10-06T02:24:13.030566+0000","last_peered":"2023-10-06T02:24:13.030566+0000","last_clean":"2023-10-06T02:24:13.030566+0000","last_became_active":"2023-10-06T02:24:13.030145+0000","last_became_peered":"2023-10-06T02:24:13.030145+0000","last_unstale":"2023-10-06T02:24:13.030566+0000","last_undegraded":"2023-10-06T02:24:13.030566+0000","last_fullsized":"2023-10-06T02:24:13.030566+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:26:17.427362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.024098+0000","last_change":"2023-10-06T02:24:13.024098+0000","last_active":"2023-10-06T02:24:13.024098+0000","last_peered":"2023-10-06T02:24:13.024098+0000","last_clean":"2023-10-06T02:24:13.024098+0000","last_became_active":"2023-10-06T02:24:13.023785+0000","last_became_peered":"2023-10-06T02:24:13.023785+0000","last_unstale":"2023-10-06T02:24:13.024098+0000","last_undegraded":"2023-10-06T02:24:13.024098+0000","last_fullsized":"2023-10-06T02:24:13.024098+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:00:32.796311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.014926+0000","last_change":"2023-10-06T02:24:13.014926+0000","last_active":"2023-10-06T02:24:13.014926+0000","last_peered":"2023-10-06T02:24:13.014926+0000","last_clean":"2023-10-06T02:24:13.014926+0000","last_became_active":"2023-10-06T02:24:13.014584+0000","last_became_peered":"2023-10-06T02:24:13.014584+0000","last_unstale":"2023-10-06T02:24:13.014926+0000","last_undegraded":"2023-10-06T02:24:13.014926+0000","last_fullsized":"2023-10-06T02:24:13.014926+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:21:14.321298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.018209+0000","last_change":"2023-10-06T02:24:13.018209+0000","last_active":"2023-10-06T02:24:13.018209+0000","last_peered":"2023-10-06T02:24:13.018209+0000","last_clean":"2023-10-06T02:24:13.018209+0000","last_became_active":"2023-10-06T02:24:13.017944+0000","last_became_peered":"2023-10-06T02:24:13.017944+0000","last_unstale":"2023-10-06T02:24:13.018209+0000","last_undegraded":"2023-10-06T02:24:13.018209+0000","last_fullsized":"2023-10-06T02:24:13.018209+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:28:47.052857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.009666+0000","last_change":"2023-10-06T02:24:13.009666+0000","last_active":"2023-10-06T02:24:13.009666+0000","last_peered":"2023-10-06T02:24:13.009666+0000","last_clean":"2023-10-06T02:24:13.009666+0000","last_became_active":"2023-10-06T02:24:13.009358+0000","last_became_peered":"2023-10-06T02:24:13.009358+0000","last_unstale":"2023-10-06T02:24:13.009666+0000","last_undegraded":"2023-10-06T02:24:13.009666+0000","last_fullsized":"2023-10-06T02:24:13.009666+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:58:49.176116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.025616+0000","last_change":"2023-10-06T02:24:13.025616+0000","last_active":"2023-10-06T02:24:13.025616+0000","last_peered":"2023-10-06T02:24:13.025616+0000","last_clean":"2023-10-06T02:24:13.025616+0000","last_became_active":"2023-10-06T02:24:13.022921+0000","last_became_peered":"2023-10-06T02:24:13.022921+0000","last_unstale":"2023-10-06T02:24:13.025616+0000","last_undegraded":"2023-10-06T02:24:13.025616+0000","last_fullsized":"2023-10-06T02:24:13.025616+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:41.374491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.015584+0000","last_change":"2023-10-06T02:24:13.015584+0000","last_active":"2023-10-06T02:24:13.015584+0000","last_peered":"2023-10-06T02:24:13.015584+0000","last_clean":"2023-10-06T02:24:13.015584+0000","last_became_active":"2023-10-06T02:24:13.015339+0000","last_became_peered":"2023-10-06T02:24:13.015339+0000","last_unstale":"2023-10-06T02:24:13.015584+0000","last_undegraded":"2023-10-06T02:24:13.015584+0000","last_fullsized":"2023-10-06T02:24:13.015584+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:00:32.796311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.027964+0000","last_change":"2023-10-06T02:24:13.027964+0000","last_active":"2023-10-06T02:24:13.027964+0000","last_peered":"2023-10-06T02:24:13.027964+0000","last_clean":"2023-10-06T02:24:13.027964+0000","last_became_active":"2023-10-06T02:24:13.027676+0000","last_became_peered":"2023-10-06T02:24:13.027676+0000","last_unstale":"2023-10-06T02:24:13.027964+0000","last_undegraded":"2023-10-06T02:24:13.027964+0000","last_fullsized":"2023-10-06T02:24:13.027964+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:46:19.025627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.030914+0000","last_change":"2023-10-06T02:24:13.030914+0000","last_active":"2023-10-06T02:24:13.030914+0000","last_peered":"2023-10-06T02:24:13.030914+0000","last_clean":"2023-10-06T02:24:13.030914+0000","last_became_active":"2023-10-06T02:24:13.030081+0000","last_became_peered":"2023-10-06T02:24:13.030081+0000","last_unstale":"2023-10-06T02:24:13.030914+0000","last_undegraded":"2023-10-06T02:24:13.030914+0000","last_fullsized":"2023-10-06T02:24:13.030914+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:51:52.519503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.024459+0000","last_change":"2023-10-06T02:24:13.024459+0000","last_active":"2023-10-06T02:24:13.024459+0000","last_peered":"2023-10-06T02:24:13.024459+0000","last_clean":"2023-10-06T02:24:13.024459+0000","last_became_active":"2023-10-06T02:24:13.024181+0000","last_became_peered":"2023-10-06T02:24:13.024181+0000","last_unstale":"2023-10-06T02:24:13.024459+0000","last_undegraded":"2023-10-06T02:24:13.024459+0000","last_fullsized":"2023-10-06T02:24:13.024459+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:58:42.494614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.025476+0000","last_change":"2023-10-06T02:24:13.025476+0000","last_active":"2023-10-06T02:24:13.025476+0000","last_peered":"2023-10-06T02:24:13.025476+0000","last_clean":"2023-10-06T02:24:13.025476+0000","last_became_active":"2023-10-06T02:24:13.025202+0000","last_became_peered":"2023-10-06T02:24:13.025202+0000","last_unstale":"2023-10-06T02:24:13.025476+0000","last_undegraded":"2023-10-06T02:24:13.025476+0000","last_fullsized":"2023-10-06T02:24:13.025476+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:08:46.284322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"35'4","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.040669+0000","last_change":"2023-10-06T02:24:12.017253+0000","last_active":"2023-10-06T02:24:13.040669+0000","last_peered":"2023-10-06T02:24:13.040669+0000","last_clean":"2023-10-06T02:24:13.040669+0000","last_became_active":"2023-10-06T02:24:12.016938+0000","last_became_peered":"2023-10-06T02:24:12.016938+0000","last_unstale":"2023-10-06T02:24:13.040669+0000","last_undegraded":"2023-10-06T02:24:13.040669+0000","last_fullsized":"2023-10-06T02:24:13.040669+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_clean_scrub_stamp":"2023-10-06T02:24:10.982473+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:25:27.976647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.018852+0000","last_change":"2023-10-06T02:24:13.018852+0000","last_active":"2023-10-06T02:24:13.018852+0000","last_peered":"2023-10-06T02:24:13.018852+0000","last_clean":"2023-10-06T02:24:13.018852+0000","last_became_active":"2023-10-06T02:24:13.018482+0000","last_became_peered":"2023-10-06T02:24:13.018482+0000","last_unstale":"2023-10-06T02:24:13.018852+0000","last_undegraded":"2023-10-06T02:24:13.018852+0000","last_fullsized":"2023-10-06T02:24:13.018852+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:50:23.084812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"35'1","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.006363+0000","last_change":"2023-10-06T02:24:12.008843+0000","last_active":"2023-10-06T02:24:13.006363+0000","last_peered":"2023-10-06T02:24:13.006363+0000","last_clean":"2023-10-06T02:24:13.006363+0000","last_became_active":"2023-10-06T02:24:12.008524+0000","last_became_peered":"2023-10-06T02:24:12.008524+0000","last_unstale":"2023-10-06T02:24:13.006363+0000","last_undegraded":"2023-10-06T02:24:13.006363+0000","last_fullsized":"2023-10-06T02:24:13.006363+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_clean_scrub_stamp":"2023-10-06T02:24:10.982473+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:14:32.568412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.015592+0000","last_change":"2023-10-06T02:24:13.015592+0000","last_active":"2023-10-06T02:24:13.015592+0000","last_peered":"2023-10-06T02:24:13.015592+0000","last_clean":"2023-10-06T02:24:13.015592+0000","last_became_active":"2023-10-06T02:24:13.015313+0000","last_became_peered":"2023-10-06T02:24:13.015313+0000","last_unstale":"2023-10-06T02:24:13.015592+0000","last_undegraded":"2023-10-06T02:24:13.015592+0000","last_fullsized":"2023-10-06T02:24:13.015592+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:06:39.375112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"35'2","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.030412+0000","last_change":"2023-10-06T02:24:12.017119+0000","last_active":"2023-10-06T02:24:13.030412+0000","last_peered":"2023-10-06T02:24:13.030412+0000","last_clean":"2023-10-06T02:24:13.030412+0000","last_became_active":"2023-10-06T02:24:12.016871+0000","last_became_peered":"2023-10-06T02:24:12.016871+0000","last_unstale":"2023-10-06T02:24:13.030412+0000","last_undegraded":"2023-10-06T02:24:13.030412+0000","last_fullsized":"2023-10-06T02:24:13.030412+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_clean_scrub_stamp":"2023-10-06T02:24:10.982473+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:40:04.539924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.029928+0000","last_change":"2023-10-06T02:24:13.029928+0000","last_active":"2023-10-06T02:24:13.029928+0000","last_peered":"2023-10-06T02:24:13.029928+0000","last_clean":"2023-10-06T02:24:13.029928+0000","last_became_active":"2023-10-06T02:24:13.029599+0000","last_became_peered":"2023-10-06T02:24:13.029599+0000","last_unstale":"2023-10-06T02:24:13.029928+0000","last_undegraded":"2023-10-06T02:24:13.029928+0000","last_fullsized":"2023-10-06T02:24:13.029928+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:06:39.375112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"35'2","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.036267+0000","last_change":"2023-10-06T02:24:12.017523+0000","last_active":"2023-10-06T02:24:13.036267+0000","last_peered":"2023-10-06T02:24:13.036267+0000","last_clean":"2023-10-06T02:24:13.036267+0000","last_became_active":"2023-10-06T02:24:12.017245+0000","last_became_peered":"2023-10-06T02:24:12.017245+0000","last_unstale":"2023-10-06T02:24:13.036267+0000","last_undegraded":"2023-10-06T02:24:13.036267+0000","last_fullsized":"2023-10-06T02:24:13.036267+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_clean_scrub_stamp":"2023-10-06T02:24:10.982473+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:07:18.069742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.015741+0000","last_change":"2023-10-06T02:24:13.015741+0000","last_active":"2023-10-06T02:24:13.015741+0000","last_peered":"2023-10-06T02:24:13.015741+0000","last_clean":"2023-10-06T02:24:13.015741+0000","last_became_active":"2023-10-06T02:24:13.015439+0000","last_became_peered":"2023-10-06T02:24:13.015439+0000","last_unstale":"2023-10-06T02:24:13.015741+0000","last_undegraded":"2023-10-06T02:24:13.015741+0000","last_fullsized":"2023-10-06T02:24:13.015741+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:26:17.427362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"35'1","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:12.999064+0000","last_change":"2023-10-06T02:24:12.006644+0000","last_active":"2023-10-06T02:24:12.999064+0000","last_peered":"2023-10-06T02:24:12.999064+0000","last_clean":"2023-10-06T02:24:12.999064+0000","last_became_active":"2023-10-06T02:24:12.005752+0000","last_became_peered":"2023-10-06T02:24:12.005752+0000","last_unstale":"2023-10-06T02:24:12.999064+0000","last_undegraded":"2023-10-06T02:24:12.999064+0000","last_fullsized":"2023-10-06T02:24:12.999064+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_clean_scrub_stamp":"2023-10-06T02:24:10.982473+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:28:09.110101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.018753+0000","last_change":"2023-10-06T02:24:13.018753+0000","last_active":"2023-10-06T02:24:13.018753+0000","last_peered":"2023-10-06T02:24:13.018753+0000","last_clean":"2023-10-06T02:24:13.018753+0000","last_became_active":"2023-10-06T02:24:13.018293+0000","last_became_peered":"2023-10-06T02:24:13.018293+0000","last_unstale":"2023-10-06T02:24:13.018753+0000","last_undegraded":"2023-10-06T02:24:13.018753+0000","last_fullsized":"2023-10-06T02:24:13.018753+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:55:41.830177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"35'1","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.006915+0000","last_change":"2023-10-06T02:24:12.013892+0000","last_active":"2023-10-06T02:24:13.006915+0000","last_peered":"2023-10-06T02:24:13.006915+0000","last_clean":"2023-10-06T02:24:13.006915+0000","last_became_active":"2023-10-06T02:24:12.013621+0000","last_became_peered":"2023-10-06T02:24:12.013621+0000","last_unstale":"2023-10-06T02:24:13.006915+0000","last_undegraded":"2023-10-06T02:24:13.006915+0000","last_fullsized":"2023-10-06T02:24:13.006915+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_clean_scrub_stamp":"2023-10-06T02:24:10.982473+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:10:10.757086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.021550+0000","last_change":"2023-10-06T02:24:13.021550+0000","last_active":"2023-10-06T02:24:13.021550+0000","last_peered":"2023-10-06T02:24:13.021550+0000","last_clean":"2023-10-06T02:24:13.021550+0000","last_became_active":"2023-10-06T02:24:13.016453+0000","last_became_peered":"2023-10-06T02:24:13.016453+0000","last_unstale":"2023-10-06T02:24:13.021550+0000","last_undegraded":"2023-10-06T02:24:13.021550+0000","last_fullsized":"2023-10-06T02:24:13.021550+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:25:21.270044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T02:24:12.021783+0000","last_change":"2023-10-06T02:24:12.017759+0000","last_active":"2023-10-06T02:24:12.021783+0000","last_peered":"2023-10-06T02:24:12.021783+0000","last_clean":"2023-10-06T02:24:12.021783+0000","last_became_active":"2023-10-06T02:24:12.017474+0000","last_became_peered":"2023-10-06T02:24:12.017474+0000","last_unstale":"2023-10-06T02:24:12.021783+0000","last_undegraded":"2023-10-06T02:24:12.021783+0000","last_fullsized":"2023-10-06T02:24:12.021783+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_clean_scrub_stamp":"2023-10-06T02:24:10.982473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:58:48.172696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.034898+0000","last_change":"2023-10-06T02:24:13.034898+0000","last_active":"2023-10-06T02:24:13.034898+0000","last_peered":"2023-10-06T02:24:13.034898+0000","last_clean":"2023-10-06T02:24:13.034898+0000","last_became_active":"2023-10-06T02:24:13.034639+0000","last_became_peered":"2023-10-06T02:24:13.034639+0000","last_unstale":"2023-10-06T02:24:13.034898+0000","last_undegraded":"2023-10-06T02:24:13.034898+0000","last_fullsized":"2023-10-06T02:24:13.034898+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:30:18.564694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T02:24:12.019432+0000","last_change":"2023-10-06T02:24:12.016124+0000","last_active":"2023-10-06T02:24:12.019432+0000","last_peered":"2023-10-06T02:24:12.019432+0000","last_clean":"2023-10-06T02:24:12.019432+0000","last_became_active":"2023-10-06T02:24:12.015798+0000","last_became_peered":"2023-10-06T02:24:12.015798+0000","last_unstale":"2023-10-06T02:24:12.019432+0000","last_undegraded":"2023-10-06T02:24:12.019432+0000","last_fullsized":"2023-10-06T02:24:12.019432+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_clean_scrub_stamp":"2023-10-06T02:24:10.982473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:51:39.075546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.012410+0000","last_change":"2023-10-06T02:24:13.012410+0000","last_active":"2023-10-06T02:24:13.012410+0000","last_peered":"2023-10-06T02:24:13.012410+0000","last_clean":"2023-10-06T02:24:13.012410+0000","last_became_active":"2023-10-06T02:24:13.012121+0000","last_became_peered":"2023-10-06T02:24:13.012121+0000","last_unstale":"2023-10-06T02:24:13.012410+0000","last_undegraded":"2023-10-06T02:24:13.012410+0000","last_fullsized":"2023-10-06T02:24:13.012410+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:58:49.176116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T02:24:12.019207+0000","last_change":"2023-10-06T02:24:12.016888+0000","last_active":"2023-10-06T02:24:12.019207+0000","last_peered":"2023-10-06T02:24:12.019207+0000","last_clean":"2023-10-06T02:24:12.019207+0000","last_became_active":"2023-10-06T02:24:12.016250+0000","last_became_peered":"2023-10-06T02:24:12.016250+0000","last_unstale":"2023-10-06T02:24:12.019207+0000","last_undegraded":"2023-10-06T02:24:12.019207+0000","last_fullsized":"2023-10-06T02:24:12.019207+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_clean_scrub_stamp":"2023-10-06T02:24:10.982473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:05:40.679720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.018606+0000","last_change":"2023-10-06T02:24:13.018606+0000","last_active":"2023-10-06T02:24:13.018606+0000","last_peered":"2023-10-06T02:24:13.018606+0000","last_clean":"2023-10-06T02:24:13.018606+0000","last_became_active":"2023-10-06T02:24:13.018211+0000","last_became_peered":"2023-10-06T02:24:13.018211+0000","last_unstale":"2023-10-06T02:24:13.018606+0000","last_undegraded":"2023-10-06T02:24:13.018606+0000","last_fullsized":"2023-10-06T02:24:13.018606+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:07:19.073163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T02:24:12.016305+0000","last_change":"2023-10-06T02:24:12.012693+0000","last_active":"2023-10-06T02:24:12.016305+0000","last_peered":"2023-10-06T02:24:12.016305+0000","last_clean":"2023-10-06T02:24:12.016305+0000","last_became_active":"2023-10-06T02:24:12.012397+0000","last_became_peered":"2023-10-06T02:24:12.012397+0000","last_unstale":"2023-10-06T02:24:12.016305+0000","last_undegraded":"2023-10-06T02:24:12.016305+0000","last_fullsized":"2023-10-06T02:24:12.016305+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_clean_scrub_stamp":"2023-10-06T02:24:10.982473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:14:32.568412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.020233+0000","last_change":"2023-10-06T02:24:13.020233+0000","last_active":"2023-10-06T02:24:13.020233+0000","last_peered":"2023-10-06T02:24:13.020233+0000","last_clean":"2023-10-06T02:24:13.020233+0000","last_became_active":"2023-10-06T02:24:13.019926+0000","last_became_peered":"2023-10-06T02:24:13.019926+0000","last_unstale":"2023-10-06T02:24:13.020233+0000","last_undegraded":"2023-10-06T02:24:13.020233+0000","last_fullsized":"2023-10-06T02:24:13.020233+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:41.374491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"35'3","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.030974+0000","last_change":"2023-10-06T02:24:12.017177+0000","last_active":"2023-10-06T02:24:13.030974+0000","last_peered":"2023-10-06T02:24:13.030974+0000","last_clean":"2023-10-06T02:24:13.030974+0000","last_became_active":"2023-10-06T02:24:12.016918+0000","last_became_peered":"2023-10-06T02:24:12.016918+0000","last_unstale":"2023-10-06T02:24:13.030974+0000","last_undegraded":"2023-10-06T02:24:13.030974+0000","last_fullsized":"2023-10-06T02:24:13.030974+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_clean_scrub_stamp":"2023-10-06T02:24:10.982473+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:38:35.957251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.024378+0000","last_change":"2023-10-06T02:24:13.024378+0000","last_active":"2023-10-06T02:24:13.024378+0000","last_peered":"2023-10-06T02:24:13.024378+0000","last_clean":"2023-10-06T02:24:13.024378+0000","last_became_active":"2023-10-06T02:24:13.024090+0000","last_became_peered":"2023-10-06T02:24:13.024090+0000","last_unstale":"2023-10-06T02:24:13.024378+0000","last_undegraded":"2023-10-06T02:24:13.024378+0000","last_fullsized":"2023-10-06T02:24:13.024378+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:30:18.564694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"35'6","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.038511+0000","last_change":"2023-10-06T02:24:12.007344+0000","last_active":"2023-10-06T02:24:13.038511+0000","last_peered":"2023-10-06T02:24:13.038511+0000","last_clean":"2023-10-06T02:24:13.038511+0000","last_became_active":"2023-10-06T02:24:12.006782+0000","last_became_peered":"2023-10-06T02:24:12.006782+0000","last_unstale":"2023-10-06T02:24:13.038511+0000","last_undegraded":"2023-10-06T02:24:13.038511+0000","last_fullsized":"2023-10-06T02:24:13.038511+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_clean_scrub_stamp":"2023-10-06T02:24:10.982473+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:58:41.491194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.021953+0000","last_change":"2023-10-06T02:24:13.021953+0000","last_active":"2023-10-06T02:24:13.021953+0000","last_peered":"2023-10-06T02:24:13.021953+0000","last_clean":"2023-10-06T02:24:13.021953+0000","last_became_active":"2023-10-06T02:24:13.021677+0000","last_became_peered":"2023-10-06T02:24:13.021677+0000","last_unstale":"2023-10-06T02:24:13.021953+0000","last_undegraded":"2023-10-06T02:24:13.021953+0000","last_fullsized":"2023-10-06T02:24:13.021953+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:51:52.519503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"35'2","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.034521+0000","last_change":"2023-10-06T02:24:12.003980+0000","last_active":"2023-10-06T02:24:13.034521+0000","last_peered":"2023-10-06T02:24:13.034521+0000","last_clean":"2023-10-06T02:24:13.034521+0000","last_became_active":"2023-10-06T02:24:12.003709+0000","last_became_peered":"2023-10-06T02:24:12.003709+0000","last_unstale":"2023-10-06T02:24:13.034521+0000","last_undegraded":"2023-10-06T02:24:13.034521+0000","last_fullsized":"2023-10-06T02:24:13.034521+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_clean_scrub_stamp":"2023-10-06T02:24:10.982473+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:19.833704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.019943+0000","last_change":"2023-10-06T02:24:13.019943+0000","last_active":"2023-10-06T02:24:13.019943+0000","last_peered":"2023-10-06T02:24:13.019943+0000","last_clean":"2023-10-06T02:24:13.019943+0000","last_became_active":"2023-10-06T02:24:13.019678+0000","last_became_peered":"2023-10-06T02:24:13.019678+0000","last_unstale":"2023-10-06T02:24:13.019943+0000","last_undegraded":"2023-10-06T02:24:13.019943+0000","last_fullsized":"2023-10-06T02:24:13.019943+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:08:46.284322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T02:24:12.023013+0000","last_change":"2023-10-06T02:24:12.022164+0000","last_active":"2023-10-06T02:24:12.023013+0000","last_peered":"2023-10-06T02:24:12.023013+0000","last_clean":"2023-10-06T02:24:12.023013+0000","last_became_active":"2023-10-06T02:24:12.021878+0000","last_became_peered":"2023-10-06T02:24:12.021878+0000","last_unstale":"2023-10-06T02:24:12.023013+0000","last_undegraded":"2023-10-06T02:24:12.023013+0000","last_fullsized":"2023-10-06T02:24:12.023013+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_clean_scrub_stamp":"2023-10-06T02:24:10.982473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:21:13.317878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.026809+0000","last_change":"2023-10-06T02:24:13.026809+0000","last_active":"2023-10-06T02:24:13.026809+0000","last_peered":"2023-10-06T02:24:13.026809+0000","last_clean":"2023-10-06T02:24:13.026809+0000","last_became_active":"2023-10-06T02:24:13.026530+0000","last_became_peered":"2023-10-06T02:24:13.026530+0000","last_unstale":"2023-10-06T02:24:13.026809+0000","last_undegraded":"2023-10-06T02:24:13.026809+0000","last_fullsized":"2023-10-06T02:24:13.026809+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:05:07.253674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T02:24:12.020420+0000","last_change":"2023-10-06T02:24:12.017885+0000","last_active":"2023-10-06T02:24:12.020420+0000","last_peered":"2023-10-06T02:24:12.020420+0000","last_clean":"2023-10-06T02:24:12.020420+0000","last_became_active":"2023-10-06T02:24:12.017546+0000","last_became_peered":"2023-10-06T02:24:12.017546+0000","last_unstale":"2023-10-06T02:24:12.020420+0000","last_undegraded":"2023-10-06T02:24:12.020420+0000","last_fullsized":"2023-10-06T02:24:12.020420+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_clean_scrub_stamp":"2023-10-06T02:24:10.982473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:43:26.238381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.024894+0000","last_change":"2023-10-06T02:24:13.024894+0000","last_active":"2023-10-06T02:24:13.024894+0000","last_peered":"2023-10-06T02:24:13.024894+0000","last_clean":"2023-10-06T02:24:13.024894+0000","last_became_active":"2023-10-06T02:24:13.019884+0000","last_became_peered":"2023-10-06T02:24:13.019884+0000","last_unstale":"2023-10-06T02:24:13.024894+0000","last_undegraded":"2023-10-06T02:24:13.024894+0000","last_fullsized":"2023-10-06T02:24:13.024894+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:41.374491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"35'8","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.040785+0000","last_change":"2023-10-06T02:24:12.018329+0000","last_active":"2023-10-06T02:24:13.040785+0000","last_peered":"2023-10-06T02:24:13.040785+0000","last_clean":"2023-10-06T02:24:13.040785+0000","last_became_active":"2023-10-06T02:24:12.018017+0000","last_became_peered":"2023-10-06T02:24:12.018017+0000","last_unstale":"2023-10-06T02:24:13.040785+0000","last_undegraded":"2023-10-06T02:24:13.040785+0000","last_fullsized":"2023-10-06T02:24:13.040785+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_clean_scrub_stamp":"2023-10-06T02:24:10.982473+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:40:04.539924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.025438+0000","last_change":"2023-10-06T02:24:13.025438+0000","last_active":"2023-10-06T02:24:13.025438+0000","last_peered":"2023-10-06T02:24:13.025438+0000","last_clean":"2023-10-06T02:24:13.025438+0000","last_became_active":"2023-10-06T02:24:13.025052+0000","last_became_peered":"2023-10-06T02:24:13.025052+0000","last_unstale":"2023-10-06T02:24:13.025438+0000","last_undegraded":"2023-10-06T02:24:13.025438+0000","last_fullsized":"2023-10-06T02:24:13.025438+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:55:41.830177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"35'1","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:12.998778+0000","last_change":"2023-10-06T02:24:12.018252+0000","last_active":"2023-10-06T02:24:12.998778+0000","last_peered":"2023-10-06T02:24:12.998778+0000","last_clean":"2023-10-06T02:24:12.998778+0000","last_became_active":"2023-10-06T02:24:12.017702+0000","last_became_peered":"2023-10-06T02:24:12.017702+0000","last_unstale":"2023-10-06T02:24:12.998778+0000","last_undegraded":"2023-10-06T02:24:12.998778+0000","last_fullsized":"2023-10-06T02:24:12.998778+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_clean_scrub_stamp":"2023-10-06T02:24:10.982473+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:19.833704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.018397+0000","last_change":"2023-10-06T02:24:13.018397+0000","last_active":"2023-10-06T02:24:13.018397+0000","last_peered":"2023-10-06T02:24:13.018397+0000","last_clean":"2023-10-06T02:24:13.018397+0000","last_became_active":"2023-10-06T02:24:13.018084+0000","last_became_peered":"2023-10-06T02:24:13.018084+0000","last_unstale":"2023-10-06T02:24:13.018397+0000","last_undegraded":"2023-10-06T02:24:13.018397+0000","last_fullsized":"2023-10-06T02:24:13.018397+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:30:18.564694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"35'2","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.040512+0000","last_change":"2023-10-06T02:24:12.020590+0000","last_active":"2023-10-06T02:24:13.040512+0000","last_peered":"2023-10-06T02:24:13.040512+0000","last_clean":"2023-10-06T02:24:13.040512+0000","last_became_active":"2023-10-06T02:24:12.020267+0000","last_became_peered":"2023-10-06T02:24:12.020267+0000","last_unstale":"2023-10-06T02:24:13.040512+0000","last_undegraded":"2023-10-06T02:24:13.040512+0000","last_fullsized":"2023-10-06T02:24:13.040512+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_clean_scrub_stamp":"2023-10-06T02:24:10.982473+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:58:52.832569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.023478+0000","last_change":"2023-10-06T02:24:13.023478+0000","last_active":"2023-10-06T02:24:13.023478+0000","last_peered":"2023-10-06T02:24:13.023478+0000","last_clean":"2023-10-06T02:24:13.023478+0000","last_became_active":"2023-10-06T02:24:13.023207+0000","last_became_peered":"2023-10-06T02:24:13.023207+0000","last_unstale":"2023-10-06T02:24:13.023478+0000","last_undegraded":"2023-10-06T02:24:13.023478+0000","last_fullsized":"2023-10-06T02:24:13.023478+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:14:08.469730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"35'1","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:12.999822+0000","last_change":"2023-10-06T02:24:12.016086+0000","last_active":"2023-10-06T02:24:12.999822+0000","last_peered":"2023-10-06T02:24:12.999822+0000","last_clean":"2023-10-06T02:24:12.999822+0000","last_became_active":"2023-10-06T02:24:12.015731+0000","last_became_peered":"2023-10-06T02:24:12.015731+0000","last_unstale":"2023-10-06T02:24:12.999822+0000","last_undegraded":"2023-10-06T02:24:12.999822+0000","last_fullsized":"2023-10-06T02:24:12.999822+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_clean_scrub_stamp":"2023-10-06T02:24:10.982473+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:43:26.238381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.036419+0000","last_change":"2023-10-06T02:24:13.036419+0000","last_active":"2023-10-06T02:24:13.036419+0000","last_peered":"2023-10-06T02:24:13.036419+0000","last_clean":"2023-10-06T02:24:13.036419+0000","last_became_active":"2023-10-06T02:24:13.036125+0000","last_became_peered":"2023-10-06T02:24:13.036125+0000","last_unstale":"2023-10-06T02:24:13.036419+0000","last_undegraded":"2023-10-06T02:24:13.036419+0000","last_fullsized":"2023-10-06T02:24:13.036419+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:49:00.492055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T02:24:12.024538+0000","last_change":"2023-10-06T02:24:12.023429+0000","last_active":"2023-10-06T02:24:12.024538+0000","last_peered":"2023-10-06T02:24:12.024538+0000","last_clean":"2023-10-06T02:24:12.024538+0000","last_became_active":"2023-10-06T02:24:12.023183+0000","last_became_peered":"2023-10-06T02:24:12.023183+0000","last_unstale":"2023-10-06T02:24:12.024538+0000","last_undegraded":"2023-10-06T02:24:12.024538+0000","last_fullsized":"2023-10-06T02:24:12.024538+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_clean_scrub_stamp":"2023-10-06T02:24:10.982473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:46:18.022206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.029069+0000","last_change":"2023-10-06T02:24:13.029069+0000","last_active":"2023-10-06T02:24:13.029069+0000","last_peered":"2023-10-06T02:24:13.029069+0000","last_clean":"2023-10-06T02:24:13.029069+0000","last_became_active":"2023-10-06T02:24:13.028791+0000","last_became_peered":"2023-10-06T02:24:13.028791+0000","last_unstale":"2023-10-06T02:24:13.029069+0000","last_undegraded":"2023-10-06T02:24:13.029069+0000","last_fullsized":"2023-10-06T02:24:13.029069+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:35:03.860739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"35'2","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.030529+0000","last_change":"2023-10-06T02:24:12.012069+0000","last_active":"2023-10-06T02:24:13.030529+0000","last_peered":"2023-10-06T02:24:13.030529+0000","last_clean":"2023-10-06T02:24:13.030529+0000","last_became_active":"2023-10-06T02:24:12.011751+0000","last_became_peered":"2023-10-06T02:24:12.011751+0000","last_unstale":"2023-10-06T02:24:13.030529+0000","last_undegraded":"2023-10-06T02:24:13.030529+0000","last_fullsized":"2023-10-06T02:24:13.030529+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_clean_scrub_stamp":"2023-10-06T02:24:10.982473+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:25:27.976647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.022657+0000","last_change":"2023-10-06T02:24:13.022657+0000","last_active":"2023-10-06T02:24:13.022657+0000","last_peered":"2023-10-06T02:24:13.022657+0000","last_clean":"2023-10-06T02:24:13.022657+0000","last_became_active":"2023-10-06T02:24:13.021717+0000","last_became_peered":"2023-10-06T02:24:13.021717+0000","last_unstale":"2023-10-06T02:24:13.022657+0000","last_undegraded":"2023-10-06T02:24:13.022657+0000","last_fullsized":"2023-10-06T02:24:13.022657+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:35:03.860739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T02:24:12.019598+0000","last_change":"2023-10-06T02:24:12.018179+0000","last_active":"2023-10-06T02:24:12.019598+0000","last_peered":"2023-10-06T02:24:12.019598+0000","last_clean":"2023-10-06T02:24:12.019598+0000","last_became_active":"2023-10-06T02:24:12.017888+0000","last_became_peered":"2023-10-06T02:24:12.017888+0000","last_unstale":"2023-10-06T02:24:12.019598+0000","last_undegraded":"2023-10-06T02:24:12.019598+0000","last_fullsized":"2023-10-06T02:24:12.019598+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_clean_scrub_stamp":"2023-10-06T02:24:10.982473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:54:17.765586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.002910+0000","last_change":"2023-10-06T02:24:13.002910+0000","last_active":"2023-10-06T02:24:13.002910+0000","last_peered":"2023-10-06T02:24:13.002910+0000","last_clean":"2023-10-06T02:24:13.002910+0000","last_became_active":"2023-10-06T02:24:13.002582+0000","last_became_peered":"2023-10-06T02:24:13.002582+0000","last_unstale":"2023-10-06T02:24:13.002910+0000","last_undegraded":"2023-10-06T02:24:13.002910+0000","last_fullsized":"2023-10-06T02:24:13.002910+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:35:03.860739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T02:24:12.015099+0000","last_change":"2023-10-06T02:24:12.009123+0000","last_active":"2023-10-06T02:24:12.015099+0000","last_peered":"2023-10-06T02:24:12.015099+0000","last_clean":"2023-10-06T02:24:12.015099+0000","last_became_active":"2023-10-06T02:24:12.008858+0000","last_became_peered":"2023-10-06T02:24:12.008858+0000","last_unstale":"2023-10-06T02:24:12.015099+0000","last_undegraded":"2023-10-06T02:24:12.015099+0000","last_fullsized":"2023-10-06T02:24:12.015099+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_clean_scrub_stamp":"2023-10-06T02:24:10.982473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:11:31.954572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.035196+0000","last_change":"2023-10-06T02:24:13.035196+0000","last_active":"2023-10-06T02:24:13.035196+0000","last_peered":"2023-10-06T02:24:13.035196+0000","last_clean":"2023-10-06T02:24:13.035196+0000","last_became_active":"2023-10-06T02:24:13.034839+0000","last_became_peered":"2023-10-06T02:24:13.034839+0000","last_unstale":"2023-10-06T02:24:13.035196+0000","last_undegraded":"2023-10-06T02:24:13.035196+0000","last_fullsized":"2023-10-06T02:24:13.035196+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:11:10.805135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T02:24:12.012610+0000","last_change":"2023-10-06T02:24:12.011322+0000","last_active":"2023-10-06T02:24:12.012610+0000","last_peered":"2023-10-06T02:24:12.012610+0000","last_clean":"2023-10-06T02:24:12.012610+0000","last_became_active":"2023-10-06T02:24:12.011030+0000","last_became_peered":"2023-10-06T02:24:12.011030+0000","last_unstale":"2023-10-06T02:24:12.012610+0000","last_undegraded":"2023-10-06T02:24:12.012610+0000","last_fullsized":"2023-10-06T02:24:12.012610+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_clean_scrub_stamp":"2023-10-06T02:24:10.982473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:57:43.512145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.030168+0000","last_change":"2023-10-06T02:24:13.030168+0000","last_active":"2023-10-06T02:24:13.030168+0000","last_peered":"2023-10-06T02:24:13.030168+0000","last_clean":"2023-10-06T02:24:13.030168+0000","last_became_active":"2023-10-06T02:24:13.029891+0000","last_became_peered":"2023-10-06T02:24:13.029891+0000","last_unstale":"2023-10-06T02:24:13.030168+0000","last_undegraded":"2023-10-06T02:24:13.030168+0000","last_fullsized":"2023-10-06T02:24:13.030168+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:33:34.790463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T02:24:12.011160+0000","last_change":"2023-10-06T02:24:12.008986+0000","last_active":"2023-10-06T02:24:12.011160+0000","last_peered":"2023-10-06T02:24:12.011160+0000","last_clean":"2023-10-06T02:24:12.011160+0000","last_became_active":"2023-10-06T02:24:12.008613+0000","last_became_peered":"2023-10-06T02:24:12.008613+0000","last_unstale":"2023-10-06T02:24:12.011160+0000","last_undegraded":"2023-10-06T02:24:12.011160+0000","last_fullsized":"2023-10-06T02:24:12.011160+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_clean_scrub_stamp":"2023-10-06T02:24:10.982473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:58:41.491194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T02:24:12.012316+0000","last_change":"2023-10-06T02:23:02.054039+0000","last_active":"2023-10-06T02:24:12.012316+0000","last_peered":"2023-10-06T02:24:12.012316+0000","last_clean":"2023-10-06T02:24:12.012316+0000","last_became_active":"2023-10-06T02:23:02.053742+0000","last_became_peered":"2023-10-06T02:23:02.053742+0000","last_unstale":"2023-10-06T02:24:12.012316+0000","last_undegraded":"2023-10-06T02:24:12.012316+0000","last_fullsized":"2023-10-06T02:24:12.012316+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:58:18.846475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.027936+0000","last_change":"2023-10-06T02:24:13.027936+0000","last_active":"2023-10-06T02:24:13.027936+0000","last_peered":"2023-10-06T02:24:13.027936+0000","last_clean":"2023-10-06T02:24:13.027936+0000","last_became_active":"2023-10-06T02:24:13.027684+0000","last_became_peered":"2023-10-06T02:24:13.027684+0000","last_unstale":"2023-10-06T02:24:13.027936+0000","last_undegraded":"2023-10-06T02:24:13.027936+0000","last_fullsized":"2023-10-06T02:24:13.027936+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:25:21.270044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T02:24:12.018991+0000","last_change":"2023-10-06T02:24:12.016641+0000","last_active":"2023-10-06T02:24:12.018991+0000","last_peered":"2023-10-06T02:24:12.018991+0000","last_clean":"2023-10-06T02:24:12.018991+0000","last_became_active":"2023-10-06T02:24:12.016315+0000","last_became_peered":"2023-10-06T02:24:12.016315+0000","last_unstale":"2023-10-06T02:24:12.018991+0000","last_undegraded":"2023-10-06T02:24:12.018991+0000","last_fullsized":"2023-10-06T02:24:12.018991+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_clean_scrub_stamp":"2023-10-06T02:24:10.982473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:19.833704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":25,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T02:24:12.000742+0000","last_change":"2023-10-06T02:23:01.988631+0000","last_active":"2023-10-06T02:24:12.000742+0000","last_peered":"2023-10-06T02:24:12.000742+0000","last_clean":"2023-10-06T02:24:12.000742+0000","last_became_active":"2023-10-06T02:23:01.988303+0000","last_became_peered":"2023-10-06T02:23:01.988303+0000","last_unstale":"2023-10-06T02:24:12.000742+0000","last_undegraded":"2023-10-06T02:24:12.000742+0000","last_fullsized":"2023-10-06T02:24:12.000742+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:38.211466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.021692+0000","last_change":"2023-10-06T02:24:13.021692+0000","last_active":"2023-10-06T02:24:13.021692+0000","last_peered":"2023-10-06T02:24:13.021692+0000","last_clean":"2023-10-06T02:24:13.021692+0000","last_became_active":"2023-10-06T02:24:13.021381+0000","last_became_peered":"2023-10-06T02:24:13.021381+0000","last_unstale":"2023-10-06T02:24:13.021692+0000","last_undegraded":"2023-10-06T02:24:13.021692+0000","last_fullsized":"2023-10-06T02:24:13.021692+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:51:40.078966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T02:24:12.019314+0000","last_change":"2023-10-06T02:24:12.017168+0000","last_active":"2023-10-06T02:24:12.019314+0000","last_peered":"2023-10-06T02:24:12.019314+0000","last_clean":"2023-10-06T02:24:12.019314+0000","last_became_active":"2023-10-06T02:24:12.016872+0000","last_became_peered":"2023-10-06T02:24:12.016872+0000","last_unstale":"2023-10-06T02:24:12.019314+0000","last_undegraded":"2023-10-06T02:24:12.019314+0000","last_fullsized":"2023-10-06T02:24:12.019314+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_clean_scrub_stamp":"2023-10-06T02:24:10.982473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:32:36.219666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T02:24:12.010225+0000","last_change":"2023-10-06T02:23:02.054338+0000","last_active":"2023-10-06T02:24:12.010225+0000","last_peered":"2023-10-06T02:24:12.010225+0000","last_clean":"2023-10-06T02:24:12.010225+0000","last_became_active":"2023-10-06T02:23:02.053991+0000","last_became_peered":"2023-10-06T02:23:02.053991+0000","last_unstale":"2023-10-06T02:24:12.010225+0000","last_undegraded":"2023-10-06T02:24:12.010225+0000","last_fullsized":"2023-10-06T02:24:12.010225+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:39:30.192838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.036480+0000","last_change":"2023-10-06T02:24:13.036480+0000","last_active":"2023-10-06T02:24:13.036480+0000","last_peered":"2023-10-06T02:24:13.036480+0000","last_clean":"2023-10-06T02:24:13.036480+0000","last_became_active":"2023-10-06T02:24:13.036199+0000","last_became_peered":"2023-10-06T02:24:13.036199+0000","last_unstale":"2023-10-06T02:24:13.036480+0000","last_undegraded":"2023-10-06T02:24:13.036480+0000","last_fullsized":"2023-10-06T02:24:13.036480+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:11:10.805135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T02:24:12.016078+0000","last_change":"2023-10-06T02:24:12.011693+0000","last_active":"2023-10-06T02:24:12.016078+0000","last_peered":"2023-10-06T02:24:12.016078+0000","last_clean":"2023-10-06T02:24:12.016078+0000","last_became_active":"2023-10-06T02:24:12.011429+0000","last_became_peered":"2023-10-06T02:24:12.011429+0000","last_unstale":"2023-10-06T02:24:12.016078+0000","last_undegraded":"2023-10-06T02:24:12.016078+0000","last_fullsized":"2023-10-06T02:24:12.016078+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_clean_scrub_stamp":"2023-10-06T02:24:10.982473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:31:28.937752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":25,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T02:24:12.000574+0000","last_change":"2023-10-06T02:23:01.988140+0000","last_active":"2023-10-06T02:24:12.000574+0000","last_peered":"2023-10-06T02:24:12.000574+0000","last_clean":"2023-10-06T02:24:12.000574+0000","last_became_active":"2023-10-06T02:23:01.987769+0000","last_became_peered":"2023-10-06T02:23:01.987769+0000","last_unstale":"2023-10-06T02:24:12.000574+0000","last_undegraded":"2023-10-06T02:24:12.000574+0000","last_fullsized":"2023-10-06T02:24:12.000574+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:20:08.391912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":175,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T02:24:11.999923+0000","last_change":"2023-10-06T02:22:55.032519+0000","last_active":"2023-10-06T02:24:11.999923+0000","last_peered":"2023-10-06T02:24:11.999923+0000","last_clean":"2023-10-06T02:24:11.999923+0000","last_became_active":"2023-10-06T02:22:55.031367+0000","last_became_peered":"2023-10-06T02:22:55.031367+0000","last_unstale":"2023-10-06T02:24:11.999923+0000","last_undegraded":"2023-10-06T02:24:11.999923+0000","last_fullsized":"2023-10-06T02:24:11.999923+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:22:53.309097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:22:53.309097+0000","last_clean_scrub_stamp":"2023-10-06T02:22:53.309097+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:53:46.520909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.017936+0000","last_change":"2023-10-06T02:24:13.017936+0000","last_active":"2023-10-06T02:24:13.017936+0000","last_peered":"2023-10-06T02:24:13.017936+0000","last_clean":"2023-10-06T02:24:13.017936+0000","last_became_active":"2023-10-06T02:24:13.017387+0000","last_became_peered":"2023-10-06T02:24:13.017387+0000","last_unstale":"2023-10-06T02:24:13.017936+0000","last_undegraded":"2023-10-06T02:24:13.017936+0000","last_fullsized":"2023-10-06T02:24:13.017936+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:03:03.246853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"35'2","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.029812+0000","last_change":"2023-10-06T02:24:12.016493+0000","last_active":"2023-10-06T02:24:13.029812+0000","last_peered":"2023-10-06T02:24:13.029812+0000","last_clean":"2023-10-06T02:24:13.029812+0000","last_became_active":"2023-10-06T02:24:12.016203+0000","last_became_peered":"2023-10-06T02:24:12.016203+0000","last_unstale":"2023-10-06T02:24:13.029812+0000","last_undegraded":"2023-10-06T02:24:13.029812+0000","last_fullsized":"2023-10-06T02:24:13.029812+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_clean_scrub_stamp":"2023-10-06T02:24:10.982473+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:31:28.937752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T02:24:12.000008+0000","last_change":"2023-10-06T02:23:01.990773+0000","last_active":"2023-10-06T02:24:12.000008+0000","last_peered":"2023-10-06T02:24:12.000008+0000","last_clean":"2023-10-06T02:24:12.000008+0000","last_became_active":"2023-10-06T02:23:01.990511+0000","last_became_peered":"2023-10-06T02:23:01.990511+0000","last_unstale":"2023-10-06T02:24:12.000008+0000","last_undegraded":"2023-10-06T02:24:12.000008+0000","last_fullsized":"2023-10-06T02:24:12.000008+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:42:54.797731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.014755+0000","last_change":"2023-10-06T02:24:13.014755+0000","last_active":"2023-10-06T02:24:13.014755+0000","last_peered":"2023-10-06T02:24:13.014755+0000","last_clean":"2023-10-06T02:24:13.014755+0000","last_became_active":"2023-10-06T02:24:13.014316+0000","last_became_peered":"2023-10-06T02:24:13.014316+0000","last_unstale":"2023-10-06T02:24:13.014755+0000","last_undegraded":"2023-10-06T02:24:13.014755+0000","last_fullsized":"2023-10-06T02:24:13.014755+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:14:33.571832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T02:24:12.015104+0000","last_change":"2023-10-06T02:24:12.011981+0000","last_active":"2023-10-06T02:24:12.015104+0000","last_peered":"2023-10-06T02:24:12.015104+0000","last_clean":"2023-10-06T02:24:12.015104+0000","last_became_active":"2023-10-06T02:24:12.011705+0000","last_became_peered":"2023-10-06T02:24:12.011705+0000","last_unstale":"2023-10-06T02:24:12.015104+0000","last_undegraded":"2023-10-06T02:24:12.015104+0000","last_fullsized":"2023-10-06T02:24:12.015104+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_clean_scrub_stamp":"2023-10-06T02:24:10.982473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:32:36.219666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T02:24:12.000841+0000","last_change":"2023-10-06T02:23:02.054069+0000","last_active":"2023-10-06T02:24:12.000841+0000","last_peered":"2023-10-06T02:24:12.000841+0000","last_clean":"2023-10-06T02:24:12.000841+0000","last_became_active":"2023-10-06T02:23:02.053739+0000","last_became_peered":"2023-10-06T02:23:02.053739+0000","last_unstale":"2023-10-06T02:24:12.000841+0000","last_undegraded":"2023-10-06T02:24:12.000841+0000","last_fullsized":"2023-10-06T02:24:12.000841+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:57:48.822608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.032447+0000","last_change":"2023-10-06T02:24:13.032447+0000","last_active":"2023-10-06T02:24:13.032447+0000","last_peered":"2023-10-06T02:24:13.032447+0000","last_clean":"2023-10-06T02:24:13.032447+0000","last_became_active":"2023-10-06T02:24:13.032116+0000","last_became_peered":"2023-10-06T02:24:13.032116+0000","last_unstale":"2023-10-06T02:24:13.032447+0000","last_undegraded":"2023-10-06T02:24:13.032447+0000","last_fullsized":"2023-10-06T02:24:13.032447+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:11:10.805135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T02:24:12.019553+0000","last_change":"2023-10-06T02:24:12.016143+0000","last_active":"2023-10-06T02:24:12.019553+0000","last_peered":"2023-10-06T02:24:12.019553+0000","last_clean":"2023-10-06T02:24:12.019553+0000","last_became_active":"2023-10-06T02:24:12.015755+0000","last_became_peered":"2023-10-06T02:24:12.015755+0000","last_unstale":"2023-10-06T02:24:12.019553+0000","last_undegraded":"2023-10-06T02:24:12.019553+0000","last_fullsized":"2023-10-06T02:24:12.019553+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_clean_scrub_stamp":"2023-10-06T02:24:10.982473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:36:34.773084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":23,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T02:24:12.007102+0000","last_change":"2023-10-06T02:23:01.989934+0000","last_active":"2023-10-06T02:24:12.007102+0000","last_peered":"2023-10-06T02:24:12.007102+0000","last_clean":"2023-10-06T02:24:12.007102+0000","last_became_active":"2023-10-06T02:23:01.989706+0000","last_became_peered":"2023-10-06T02:23:01.989706+0000","last_unstale":"2023-10-06T02:24:12.007102+0000","last_undegraded":"2023-10-06T02:24:12.007102+0000","last_fullsized":"2023-10-06T02:24:12.007102+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:06.212251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.019293+0000","last_change":"2023-10-06T02:24:13.019293+0000","last_active":"2023-10-06T02:24:13.019293+0000","last_peered":"2023-10-06T02:24:13.019293+0000","last_clean":"2023-10-06T02:24:13.019293+0000","last_became_active":"2023-10-06T02:24:13.018906+0000","last_became_peered":"2023-10-06T02:24:13.018906+0000","last_unstale":"2023-10-06T02:24:13.019293+0000","last_undegraded":"2023-10-06T02:24:13.019293+0000","last_fullsized":"2023-10-06T02:24:13.019293+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:55:51.032433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T02:24:12.022328+0000","last_change":"2023-10-06T02:24:12.020481+0000","last_active":"2023-10-06T02:24:12.022328+0000","last_peered":"2023-10-06T02:24:12.022328+0000","last_clean":"2023-10-06T02:24:12.022328+0000","last_became_active":"2023-10-06T02:24:12.020183+0000","last_became_peered":"2023-10-06T02:24:12.020183+0000","last_unstale":"2023-10-06T02:24:12.022328+0000","last_undegraded":"2023-10-06T02:24:12.022328+0000","last_fullsized":"2023-10-06T02:24:12.022328+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_clean_scrub_stamp":"2023-10-06T02:24:10.982473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:14:32.568412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":24,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T02:24:12.002797+0000","last_change":"2023-10-06T02:23:01.992935+0000","last_active":"2023-10-06T02:24:12.002797+0000","last_peered":"2023-10-06T02:24:12.002797+0000","last_clean":"2023-10-06T02:24:12.002797+0000","last_became_active":"2023-10-06T02:23:01.992568+0000","last_became_peered":"2023-10-06T02:23:01.992568+0000","last_unstale":"2023-10-06T02:24:12.002797+0000","last_undegraded":"2023-10-06T02:24:12.002797+0000","last_fullsized":"2023-10-06T02:24:12.002797+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:06.212251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574868,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298244,"kb_used_data":956,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073596,"statfs":{"total":96636764160,"available":96331362304,"internally_reserved":0,"allocated":978944,"data_stored":745721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.45300000000000001}]},{"osd":1,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.65500000000000003}]},{"osd":2,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.68600000000000005}]},{"osd":3,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.76900000000000002}]},{"osd":4,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.64000000000000001}]},{"osd":5,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.80100000000000005}]},{"osd":6,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.78900000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574868,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297680,"kb_used_data":392,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074160,"statfs":{"total":96636764160,"available":96331939840,"internally_reserved":0,"allocated":401408,"data_stored":163023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.63500000000000001}]},{"osd":1,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.66700000000000004}]},{"osd":2,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.73699999999999999}]},{"osd":3,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.65600000000000003}]},{"osd":4,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.54300000000000004}]},{"osd":5,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.60299999999999998}]},{"osd":7,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.458,"5min":1.458,"15min":1.458},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.755}]}]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297664,"kb_used_data":376,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074176,"statfs":{"total":96636764160,"available":96331956224,"internally_reserved":0,"allocated":385024,"data_stored":155353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.788,"5min":1.788,"15min":1.788},"last":0.54400000000000004}]},{"osd":2,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":2.0840000000000001,"5min":2.0840000000000001,"15min":2.0840000000000001},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.51200000000000001}]},{"osd":3,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.861,"5min":1.861,"15min":1.861},"last":0.48099999999999998}]},{"osd":4,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.9279999999999999,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":2.0179999999999998,"5min":2.0179999999999998,"15min":2.0179999999999998},"last":0.70899999999999996}]},{"osd":5,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.9830000000000001,"5min":1.9830000000000001,"15min":1.9830000000000001},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.78600000000000003}]},{"osd":6,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.829,"5min":1.829,"15min":1.829},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":2.052,"5min":2.052,"15min":2.052},"last":0.75800000000000001}]},{"osd":7,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.895,"5min":1.895,"15min":1.895},"last":0.72599999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574868,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298244,"kb_used_data":956,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073596,"statfs":{"total":96636764160,"available":96331362304,"internally_reserved":0,"allocated":978944,"data_stored":745721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.52500000000000002}]},{"osd":2,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.41099999999999998}]},{"osd":3,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.63700000000000001}]},{"osd":4,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.304,"5min":1.304,"15min":1.304},"last":0.88800000000000001}]},{"osd":5,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.90900000000000003}]},{"osd":6,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.86699999999999999}]},{"osd":7,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.84399999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297664,"kb_used_data":376,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074176,"statfs":{"total":96636764160,"available":96331956224,"internally_reserved":0,"allocated":385024,"data_stored":155353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.44400000000000001}]},{"osd":1,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":0.39700000000000002}]},{"osd":3,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.50600000000000001}]},{"osd":4,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.621,"5min":1.621,"15min":1.621},"last":0.79100000000000004}]},{"osd":5,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.746},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.877}]},{"osd":6,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.86199999999999999}]},{"osd":7,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.77800000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297664,"kb_used_data":376,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074176,"statfs":{"total":96636764160,"available":96331956224,"internally_reserved":0,"allocated":385024,"data_stored":155353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":2.024,"5min":2.024,"15min":2.024},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":2.181,"5min":2.181,"15min":2.181},"last":0.33700000000000002}]},{"osd":1,"last update":"Fri Oct 6 02:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":4.492,"5min":4.492,"15min":4.492},"last":0.312},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":4.569,"5min":4.569,"15min":4.569},"last":0.40100000000000002}]},{"osd":2,"last update":"Fri Oct 6 02:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":2.0870000000000002,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":0.504},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":2.4260000000000002,"5min":2.4260000000000002,"15min":2.4260000000000002},"last":0.442}]},{"osd":4,"last update":"Fri Oct 6 02:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":2.214,"5min":2.214,"15min":2.214},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.66600000000000004}]},{"osd":5,"last update":"Fri Oct 6 02:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":2.242,"5min":2.242,"15min":2.242},"last":0.65400000000000003}]},{"osd":6,"last update":"Fri Oct 6 02:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.357,"5min":1.357,"15min":1.357},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.67900000000000005}]},{"osd":7,"last update":"Fri Oct 6 02:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.27,"5min":2.27,"15min":2.27},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.69499999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297664,"kb_used_data":376,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074176,"statfs":{"total":96636764160,"available":96331956224,"internally_reserved":0,"allocated":385024,"data_stored":155353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:23:55 2023","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":12.337,"5min":12.337,"15min":12.337},"last":0.68600000000000005},{"interface":"front","average":{"1min":1.02,"5min":1.02,"15min":1.02},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":9.6539999999999999,"5min":9.6539999999999999,"15min":9.6539999999999999},"last":0.54300000000000004}]},{"osd":1,"last update":"Fri Oct 6 02:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":3.181,"5min":3.181,"15min":3.181},"last":0.74099999999999999},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":12.946999999999999,"5min":12.946999999999999,"15min":12.946999999999999},"last":0.57899999999999996}]},{"osd":2,"last update":"Fri Oct 6 02:23:55 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":12.377000000000001,"5min":12.377000000000001,"15min":12.377000000000001},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":6.3220000000000001,"5min":6.3220000000000001,"15min":6.3220000000000001},"last":0.63200000000000001}]},{"osd":3,"last update":"Fri Oct 6 02:23:55 2023","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":9.7720000000000002,"5min":9.7720000000000002,"15min":9.7720000000000002},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":3.0840000000000001,"5min":3.0840000000000001,"15min":3.0840000000000001},"last":0.64500000000000002}]},{"osd":5,"last update":"Fri Oct 6 02:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":6.2690000000000001,"5min":6.2690000000000001,"15min":6.2690000000000001},"last":0.83199999999999996},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":10.769,"5min":10.769,"15min":10.769},"last":0.61699999999999999}]},{"osd":6,"last update":"Fri Oct 6 02:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":3.1349999999999998,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":0.80900000000000005},{"interface":"front","average":{"1min":1.077,"5min":1.077,"15min":1.077},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":8.0899999999999999,"5min":8.0899999999999999,"15min":8.0899999999999999},"last":0.77300000000000002}]},{"osd":7,"last update":"Fri Oct 6 02:23:55 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":9.7089999999999996,"5min":9.7089999999999996,"15min":9.7089999999999996},"last":0.67000000000000004},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":7.6719999999999997,"5min":7.6719999999999997,"15min":7.6719999999999997},"last":0.86899999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574868,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297684,"kb_used_data":396,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074156,"statfs":{"total":96636764160,"available":96331935744,"internally_reserved":0,"allocated":405504,"data_stored":162581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.63700000000000001}]},{"osd":1,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.754},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.624}]},{"osd":2,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.59899999999999998}]},{"osd":3,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.82199999999999995}]},{"osd":4,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.77700000000000002}]},{"osd":6,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.86599999999999999}]},{"osd":7,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.90800000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T02:24:17.391 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T02:24:17.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.609+0000 7f5bf6b9d700 1 Processor -- start 2023-10-06T02:24:17.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.609+0000 7f5bf6b9d700 1 -- start start 2023-10-06T02:24:17.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.611+0000 7f5bf6b9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5bf0159550 0x7f5bf00ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:17.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.611+0000 7f5bf6b9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5bf0159920 0x7f5bf00ad090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:17.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.611+0000 7f5bf6b9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bf00acc00 con 0x7f5bf015dfd0 2023-10-06T02:24:17.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.611+0000 7f5bf6b9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bf00abff0 con 0x7f5bf0159920 2023-10-06T02:24:17.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.611+0000 7f5bf6b9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf00ac2d0 con 0x7f5bf0159550 2023-10-06T02:24:17.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.611+0000 7f5bf639c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5bf0159920 0x7f5bf00ad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:17.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.611+0000 7f5bf639c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5bf0159920 0x7f5bf00ad090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56768/0 (socket says 172.21.15.31:56768) 2023-10-06T02:24:17.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.611+0000 7f5bf639c700 1 -- 172.21.15.31:0/1192035213 learned_addr learned my addr 172.21.15.31:0/1192035213 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:17.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.611+0000 7f5bf5b9b700 1 --2- 172.21.15.31:0/1192035213 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5bf0159550 0x7f5bf00ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:17.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.612+0000 7f5bf639c700 1 -- 172.21.15.31:0/1192035213 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5bf0159550 msgr2=0x7f5bf00ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:17.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.612+0000 7f5bf639c700 1 --2- 172.21.15.31:0/1192035213 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5bf0159550 0x7f5bf00ceac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:17.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.612+0000 7f5bf639c700 1 -- 172.21.15.31:0/1192035213 >> v1:172.21.15.62:6789/0 conn(0x7f5bf015dfd0 legacy=0x7f5bf00aedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:17.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.612+0000 7f5bf639c700 1 -- 172.21.15.31:0/1192035213 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bf00ac5b0 con 0x7f5bf0159920 2023-10-06T02:24:17.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.612+0000 7f5bf639c700 1 --2- 172.21.15.31:0/1192035213 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5bf0159920 0x7f5bf00ad090 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f5be8009f20 tx=0x7f5be8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4271ec5f811f366 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:17.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.612+0000 7f5bf4b99700 1 -- 172.21.15.31:0/1192035213 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be800e070 con 0x7f5bf0159920 2023-10-06T02:24:17.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.612+0000 7f5bf4b99700 1 -- 172.21.15.31:0/1192035213 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5be800b9d0 con 0x7f5bf0159920 2023-10-06T02:24:17.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.612+0000 7f5bf4b99700 1 -- 172.21.15.31:0/1192035213 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be80092e0 con 0x7f5bf0159920 2023-10-06T02:24:17.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.613+0000 7f5bf6b9d700 1 -- 172.21.15.31:0/1192035213 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5bf0159920 msgr2=0x7f5bf00ad090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:17.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.613+0000 7f5bf6b9d700 1 --2- 172.21.15.31:0/1192035213 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5bf0159920 0x7f5bf00ad090 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f5be8009f20 tx=0x7f5be8009840 comp rx=0 tx=0).stop 2023-10-06T02:24:17.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.613+0000 7f5bf6b9d700 1 -- 172.21.15.31:0/1192035213 shutdown_connections 2023-10-06T02:24:17.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.613+0000 7f5bf6b9d700 1 --2- 172.21.15.31:0/1192035213 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5bf0159920 0x7f5bf00ad090 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:17.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.613+0000 7f5bf6b9d700 1 --2- 172.21.15.31:0/1192035213 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5bf0159550 0x7f5bf00ceac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:17.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.613+0000 7f5bf6b9d700 1 -- 172.21.15.31:0/1192035213 >> 172.21.15.31:0/1192035213 conn(0x7f5bf006ce80 msgr2=0x7f5bf00ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:17.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.613+0000 7f5bf6b9d700 1 -- 172.21.15.31:0/1192035213 shutdown_connections 2023-10-06T02:24:17.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.613+0000 7f5bf6b9d700 1 -- 172.21.15.31:0/1192035213 wait complete. 2023-10-06T02:24:17.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.614+0000 7f5bf6b9d700 1 Processor -- start 2023-10-06T02:24:17.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.614+0000 7f5bf6b9d700 1 -- start start 2023-10-06T02:24:17.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.615+0000 7f5bf6b9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5bf0159550 0x7f5bf00b32e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:17.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.615+0000 7f5bf6b9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5bf0159920 0x7f5bf014e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:17.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.615+0000 7f5bf6b9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5bf015dfd0 0x7f5bf0150230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:17.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.615+0000 7f5bf6b9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf00c8550 con 0x7f5bf0159550 2023-10-06T02:24:17.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.615+0000 7f5bf6b9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf00c8270 con 0x7f5bf0159920 2023-10-06T02:24:17.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.615+0000 7f5bf6b9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf014dc50 con 0x7f5bf015dfd0 2023-10-06T02:24:17.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.615+0000 7f5bf5b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5bf0159550 0x7f5bf00b32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:17.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.616+0000 7f5bf5b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5bf0159550 0x7f5bf00b32e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56776/0 (socket says 172.21.15.31:56776) 2023-10-06T02:24:17.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.616+0000 7f5bf5b9b700 1 -- 172.21.15.31:0/1450581002 learned_addr learned my addr 172.21.15.31:0/1450581002 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:17.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.616+0000 7f5bf639c700 1 --2- 172.21.15.31:0/1450581002 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5bf015dfd0 0x7f5bf0150230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:17.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.616+0000 7f5bf539a700 1 --2- 172.21.15.31:0/1450581002 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5bf0159920 0x7f5bf014e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:17.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.616+0000 7f5bf5b9b700 1 -- 172.21.15.31:0/1450581002 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5bf015dfd0 msgr2=0x7f5bf0150230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:17.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.616+0000 7f5bf5b9b700 1 --2- 172.21.15.31:0/1450581002 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5bf015dfd0 0x7f5bf0150230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:17.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.616+0000 7f5bf5b9b700 1 -- 172.21.15.31:0/1450581002 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5bf0159920 msgr2=0x7f5bf014e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:17.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.616+0000 7f5bf5b9b700 1 --2- 172.21.15.31:0/1450581002 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5bf0159920 0x7f5bf014e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:17.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.616+0000 7f5bf5b9b700 1 -- 172.21.15.31:0/1450581002 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5be8009ae0 con 0x7f5bf0159550 2023-10-06T02:24:17.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.616+0000 7f5bf5b9b700 1 --2- 172.21.15.31:0/1450581002 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5bf0159550 0x7f5bf00b32e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5bec00e930 tx=0x7f5bec0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:17.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.616+0000 7f5be5ffb700 1 -- 172.21.15.31:0/1450581002 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bec00eed0 con 0x7f5bf0159550 2023-10-06T02:24:17.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.617+0000 7f5be5ffb700 1 -- 172.21.15.31:0/1450581002 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bec00c850 con 0x7f5bf0159550 2023-10-06T02:24:17.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.617+0000 7f5bf6b9d700 1 -- 172.21.15.31:0/1450581002 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bf014df30 con 0x7f5bf0159550 2023-10-06T02:24:17.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.617+0000 7f5be5ffb700 1 -- 172.21.15.31:0/1450581002 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bec00cbf0 con 0x7f5bf0159550 2023-10-06T02:24:17.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.617+0000 7f5bf6b9d700 1 -- 172.21.15.31:0/1450581002 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bf014ec70 con 0x7f5bf0159550 2023-10-06T02:24:17.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.618+0000 7f5bf6b9d700 1 -- 172.21.15.31:0/1450581002 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bf014e580 con 0x7f5bf0159550 2023-10-06T02:24:17.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.619+0000 7f5be5ffb700 1 -- 172.21.15.31:0/1450581002 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5bec013470 con 0x7f5bf0159550 2023-10-06T02:24:17.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.620+0000 7f5be5ffb700 1 --2- 172.21.15.31:0/1450581002 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5bd4066f30 0x7f5bd40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:17.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.620+0000 7f5be5ffb700 1 -- 172.21.15.31:0/1450581002 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f5bec0848f0 con 0x7f5bf0159550 2023-10-06T02:24:17.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.620+0000 7f5bf539a700 1 --2- 172.21.15.31:0/1450581002 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5bd4066f30 0x7f5bd40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:17.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.621+0000 7f5bf539a700 1 --2- 172.21.15.31:0/1450581002 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5bd4066f30 0x7f5bd40693e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5bf0158f30 tx=0x7f5be0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:17.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.624+0000 7f5be5ffb700 1 -- 172.21.15.31:0/1450581002 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5bec051960 con 0x7f5bf0159550 2023-10-06T02:24:17.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.821+0000 7f5bf6b9d700 1 -- 172.21.15.31:0/1450581002 --> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5bf014dab0 con 0x7f5bd4066f30 2023-10-06T02:24:17.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.831+0000 7f5be5ffb700 1 -- 172.21.15.31:0/1450581002 <== mgr.4107 v2:172.21.15.31:6832/85292 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311059 (secure 0 0 0) 0x7f5bf014dab0 con 0x7f5bd4066f30 2023-10-06T02:24:17.835 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:24:17.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.834+0000 7f5bf6b9d700 1 -- 172.21.15.31:0/1450581002 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5bd4066f30 msgr2=0x7f5bd40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:17.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.834+0000 7f5bf6b9d700 1 --2- 172.21.15.31:0/1450581002 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5bd4066f30 0x7f5bd40693e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5bf0158f30 tx=0x7f5be0009040 comp rx=0 tx=0).stop 2023-10-06T02:24:17.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.835+0000 7f5bf6b9d700 1 -- 172.21.15.31:0/1450581002 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5bf0159550 msgr2=0x7f5bf00b32e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:17.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.835+0000 7f5bf6b9d700 1 --2- 172.21.15.31:0/1450581002 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5bf0159550 0x7f5bf00b32e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5bec00e930 tx=0x7f5bec0091a0 comp rx=0 tx=0).stop 2023-10-06T02:24:17.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.835+0000 7f5bf6b9d700 1 -- 172.21.15.31:0/1450581002 shutdown_connections 2023-10-06T02:24:17.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.835+0000 7f5bf6b9d700 1 --2- 172.21.15.31:0/1450581002 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5bd4066f30 0x7f5bd40693e0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:17.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.835+0000 7f5bf6b9d700 1 --2- 172.21.15.31:0/1450581002 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5bf015dfd0 0x7f5bf0150230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:17.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.835+0000 7f5bf6b9d700 1 --2- 172.21.15.31:0/1450581002 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5bf0159550 0x7f5bf00b32e0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:17.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.835+0000 7f5bf6b9d700 1 --2- 172.21.15.31:0/1450581002 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5bf0159920 0x7f5bf014e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:17.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.835+0000 7f5bf6b9d700 1 -- 172.21.15.31:0/1450581002 >> 172.21.15.31:0/1450581002 conn(0x7f5bf006ce80 msgr2=0x7f5bf0069510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:17.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.835+0000 7f5bf6b9d700 1 -- 172.21.15.31:0/1450581002 shutdown_connections 2023-10-06T02:24:17.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:17.835+0000 7f5bf6b9d700 1 -- 172.21.15.31:0/1450581002 wait complete. 2023-10-06T02:24:17.839 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-10-06T02:24:17.853 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":71,"stamp":"2023-10-06T02:24:17.130537+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":78,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2382508,"kb_used_data":4204,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752592212,"statfs":{"total":773094113280,"available":770654425088,"internally_reserved":0,"allocated":4304896,"data_stored":2438458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000839"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.031228+0000","last_change":"2023-10-06T02:24:13.031228+0000","last_active":"2023-10-06T02:24:13.031228+0000","last_peered":"2023-10-06T02:24:13.031228+0000","last_clean":"2023-10-06T02:24:13.031228+0000","last_became_active":"2023-10-06T02:24:13.029975+0000","last_became_peered":"2023-10-06T02:24:13.029975+0000","last_unstale":"2023-10-06T02:24:13.031228+0000","last_undegraded":"2023-10-06T02:24:13.031228+0000","last_fullsized":"2023-10-06T02:24:13.031228+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:27:04.362587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.022465+0000","last_change":"2023-10-06T02:24:13.022465+0000","last_active":"2023-10-06T02:24:13.022465+0000","last_peered":"2023-10-06T02:24:13.022465+0000","last_clean":"2023-10-06T02:24:13.022465+0000","last_became_active":"2023-10-06T02:24:13.022287+0000","last_became_peered":"2023-10-06T02:24:13.022287+0000","last_unstale":"2023-10-06T02:24:13.022465+0000","last_undegraded":"2023-10-06T02:24:13.022465+0000","last_fullsized":"2023-10-06T02:24:13.022465+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:33:34.790463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.032908+0000","last_change":"2023-10-06T02:24:13.032908+0000","last_active":"2023-10-06T02:24:13.032908+0000","last_peered":"2023-10-06T02:24:13.032908+0000","last_clean":"2023-10-06T02:24:13.032908+0000","last_became_active":"2023-10-06T02:24:13.032579+0000","last_became_peered":"2023-10-06T02:24:13.032579+0000","last_unstale":"2023-10-06T02:24:13.032908+0000","last_undegraded":"2023-10-06T02:24:13.032908+0000","last_fullsized":"2023-10-06T02:24:13.032908+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:14:08.469730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.024070+0000","last_change":"2023-10-06T02:24:13.024070+0000","last_active":"2023-10-06T02:24:13.024070+0000","last_peered":"2023-10-06T02:24:13.024070+0000","last_clean":"2023-10-06T02:24:13.024070+0000","last_became_active":"2023-10-06T02:24:13.023799+0000","last_became_peered":"2023-10-06T02:24:13.023799+0000","last_unstale":"2023-10-06T02:24:13.024070+0000","last_undegraded":"2023-10-06T02:24:13.024070+0000","last_fullsized":"2023-10-06T02:24:13.024070+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:46:19.025627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.012586+0000","last_change":"2023-10-06T02:24:13.012586+0000","last_active":"2023-10-06T02:24:13.012586+0000","last_peered":"2023-10-06T02:24:13.012586+0000","last_clean":"2023-10-06T02:24:13.012586+0000","last_became_active":"2023-10-06T02:24:13.012292+0000","last_became_peered":"2023-10-06T02:24:13.012292+0000","last_unstale":"2023-10-06T02:24:13.012586+0000","last_undegraded":"2023-10-06T02:24:13.012586+0000","last_fullsized":"2023-10-06T02:24:13.012586+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:40:05.543345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.030546+0000","last_change":"2023-10-06T02:24:13.030546+0000","last_active":"2023-10-06T02:24:13.030546+0000","last_peered":"2023-10-06T02:24:13.030546+0000","last_clean":"2023-10-06T02:24:13.030546+0000","last_became_active":"2023-10-06T02:24:13.030253+0000","last_became_peered":"2023-10-06T02:24:13.030253+0000","last_unstale":"2023-10-06T02:24:13.030546+0000","last_undegraded":"2023-10-06T02:24:13.030546+0000","last_fullsized":"2023-10-06T02:24:13.030546+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:05:41.683141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.025917+0000","last_change":"2023-10-06T02:24:13.025917+0000","last_active":"2023-10-06T02:24:13.025917+0000","last_peered":"2023-10-06T02:24:13.025917+0000","last_clean":"2023-10-06T02:24:13.025917+0000","last_became_active":"2023-10-06T02:24:13.025555+0000","last_became_peered":"2023-10-06T02:24:13.025555+0000","last_unstale":"2023-10-06T02:24:13.025917+0000","last_undegraded":"2023-10-06T02:24:13.025917+0000","last_fullsized":"2023-10-06T02:24:13.025917+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:51:52.519503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.035771+0000","last_change":"2023-10-06T02:24:13.035771+0000","last_active":"2023-10-06T02:24:13.035771+0000","last_peered":"2023-10-06T02:24:13.035771+0000","last_clean":"2023-10-06T02:24:13.035771+0000","last_became_active":"2023-10-06T02:24:13.035467+0000","last_became_peered":"2023-10-06T02:24:13.035467+0000","last_unstale":"2023-10-06T02:24:13.035771+0000","last_undegraded":"2023-10-06T02:24:13.035771+0000","last_fullsized":"2023-10-06T02:24:13.035771+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:14:08.469730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.032399+0000","last_change":"2023-10-06T02:24:13.032399+0000","last_active":"2023-10-06T02:24:13.032399+0000","last_peered":"2023-10-06T02:24:13.032399+0000","last_clean":"2023-10-06T02:24:13.032399+0000","last_became_active":"2023-10-06T02:24:13.031804+0000","last_became_peered":"2023-10-06T02:24:13.031804+0000","last_unstale":"2023-10-06T02:24:13.032399+0000","last_undegraded":"2023-10-06T02:24:13.032399+0000","last_fullsized":"2023-10-06T02:24:13.032399+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:05:07.253674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.027985+0000","last_change":"2023-10-06T02:24:13.027985+0000","last_active":"2023-10-06T02:24:13.027985+0000","last_peered":"2023-10-06T02:24:13.027985+0000","last_clean":"2023-10-06T02:24:13.027985+0000","last_became_active":"2023-10-06T02:24:13.027346+0000","last_became_peered":"2023-10-06T02:24:13.027346+0000","last_unstale":"2023-10-06T02:24:13.027985+0000","last_undegraded":"2023-10-06T02:24:13.027985+0000","last_fullsized":"2023-10-06T02:24:13.027985+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:14:08.469730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.012767+0000","last_change":"2023-10-06T02:24:13.012767+0000","last_active":"2023-10-06T02:24:13.012767+0000","last_peered":"2023-10-06T02:24:13.012767+0000","last_clean":"2023-10-06T02:24:13.012767+0000","last_became_active":"2023-10-06T02:24:13.012492+0000","last_became_peered":"2023-10-06T02:24:13.012492+0000","last_unstale":"2023-10-06T02:24:13.012767+0000","last_undegraded":"2023-10-06T02:24:13.012767+0000","last_fullsized":"2023-10-06T02:24:13.012767+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:06:39.375112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.021260+0000","last_change":"2023-10-06T02:24:13.021260+0000","last_active":"2023-10-06T02:24:13.021260+0000","last_peered":"2023-10-06T02:24:13.021260+0000","last_clean":"2023-10-06T02:24:13.021260+0000","last_became_active":"2023-10-06T02:24:13.021014+0000","last_became_peered":"2023-10-06T02:24:13.021014+0000","last_unstale":"2023-10-06T02:24:13.021260+0000","last_undegraded":"2023-10-06T02:24:13.021260+0000","last_fullsized":"2023-10-06T02:24:13.021260+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:08:46.284322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.026941+0000","last_change":"2023-10-06T02:24:13.026941+0000","last_active":"2023-10-06T02:24:13.026941+0000","last_peered":"2023-10-06T02:24:13.026941+0000","last_clean":"2023-10-06T02:24:13.026941+0000","last_became_active":"2023-10-06T02:24:13.026613+0000","last_became_peered":"2023-10-06T02:24:13.026613+0000","last_unstale":"2023-10-06T02:24:13.026941+0000","last_undegraded":"2023-10-06T02:24:13.026941+0000","last_fullsized":"2023-10-06T02:24:13.026941+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:08:46.284322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.023176+0000","last_change":"2023-10-06T02:24:13.023176+0000","last_active":"2023-10-06T02:24:13.023176+0000","last_peered":"2023-10-06T02:24:13.023176+0000","last_clean":"2023-10-06T02:24:13.023176+0000","last_became_active":"2023-10-06T02:24:13.015662+0000","last_became_peered":"2023-10-06T02:24:13.015662+0000","last_unstale":"2023-10-06T02:24:13.023176+0000","last_undegraded":"2023-10-06T02:24:13.023176+0000","last_fullsized":"2023-10-06T02:24:13.023176+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:55:41.830177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.012835+0000","last_change":"2023-10-06T02:24:13.012835+0000","last_active":"2023-10-06T02:24:13.012835+0000","last_peered":"2023-10-06T02:24:13.012835+0000","last_clean":"2023-10-06T02:24:13.012835+0000","last_became_active":"2023-10-06T02:24:13.012547+0000","last_became_peered":"2023-10-06T02:24:13.012547+0000","last_unstale":"2023-10-06T02:24:13.012835+0000","last_undegraded":"2023-10-06T02:24:13.012835+0000","last_fullsized":"2023-10-06T02:24:13.012835+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:33:34.790463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.023788+0000","last_change":"2023-10-06T02:24:13.023788+0000","last_active":"2023-10-06T02:24:13.023788+0000","last_peered":"2023-10-06T02:24:13.023788+0000","last_clean":"2023-10-06T02:24:13.023788+0000","last_became_active":"2023-10-06T02:24:13.023545+0000","last_became_peered":"2023-10-06T02:24:13.023545+0000","last_unstale":"2023-10-06T02:24:13.023788+0000","last_undegraded":"2023-10-06T02:24:13.023788+0000","last_fullsized":"2023-10-06T02:24:13.023788+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:14:08.469730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.035092+0000","last_change":"2023-10-06T02:24:13.035092+0000","last_active":"2023-10-06T02:24:13.035092+0000","last_peered":"2023-10-06T02:24:13.035092+0000","last_clean":"2023-10-06T02:24:13.035092+0000","last_became_active":"2023-10-06T02:24:13.034809+0000","last_became_peered":"2023-10-06T02:24:13.034809+0000","last_unstale":"2023-10-06T02:24:13.035092+0000","last_undegraded":"2023-10-06T02:24:13.035092+0000","last_fullsized":"2023-10-06T02:24:13.035092+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:41.374491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.011585+0000","last_change":"2023-10-06T02:24:13.011585+0000","last_active":"2023-10-06T02:24:13.011585+0000","last_peered":"2023-10-06T02:24:13.011585+0000","last_clean":"2023-10-06T02:24:13.011585+0000","last_became_active":"2023-10-06T02:24:13.011313+0000","last_became_peered":"2023-10-06T02:24:13.011313+0000","last_unstale":"2023-10-06T02:24:13.011585+0000","last_undegraded":"2023-10-06T02:24:13.011585+0000","last_fullsized":"2023-10-06T02:24:13.011585+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:53:51.425781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.014908+0000","last_change":"2023-10-06T02:24:13.014908+0000","last_active":"2023-10-06T02:24:13.014908+0000","last_peered":"2023-10-06T02:24:13.014908+0000","last_clean":"2023-10-06T02:24:13.014908+0000","last_became_active":"2023-10-06T02:24:13.014638+0000","last_became_peered":"2023-10-06T02:24:13.014638+0000","last_unstale":"2023-10-06T02:24:13.014908+0000","last_undegraded":"2023-10-06T02:24:13.014908+0000","last_fullsized":"2023-10-06T02:24:13.014908+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:21:14.321298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.011999+0000","last_change":"2023-10-06T02:24:13.011999+0000","last_active":"2023-10-06T02:24:13.011999+0000","last_peered":"2023-10-06T02:24:13.011999+0000","last_clean":"2023-10-06T02:24:13.011999+0000","last_became_active":"2023-10-06T02:24:13.011713+0000","last_became_peered":"2023-10-06T02:24:13.011713+0000","last_unstale":"2023-10-06T02:24:13.011999+0000","last_undegraded":"2023-10-06T02:24:13.011999+0000","last_fullsized":"2023-10-06T02:24:13.011999+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:40:05.543345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.017292+0000","last_change":"2023-10-06T02:24:13.017292+0000","last_active":"2023-10-06T02:24:13.017292+0000","last_peered":"2023-10-06T02:24:13.017292+0000","last_clean":"2023-10-06T02:24:13.017292+0000","last_became_active":"2023-10-06T02:24:13.016973+0000","last_became_peered":"2023-10-06T02:24:13.016973+0000","last_unstale":"2023-10-06T02:24:13.017292+0000","last_undegraded":"2023-10-06T02:24:13.017292+0000","last_fullsized":"2023-10-06T02:24:13.017292+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:27:04.362587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.025110+0000","last_change":"2023-10-06T02:24:13.025110+0000","last_active":"2023-10-06T02:24:13.025110+0000","last_peered":"2023-10-06T02:24:13.025110+0000","last_clean":"2023-10-06T02:24:13.025110+0000","last_became_active":"2023-10-06T02:24:13.024853+0000","last_became_peered":"2023-10-06T02:24:13.024853+0000","last_unstale":"2023-10-06T02:24:13.025110+0000","last_undegraded":"2023-10-06T02:24:13.025110+0000","last_fullsized":"2023-10-06T02:24:13.025110+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:27:04.362587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.030566+0000","last_change":"2023-10-06T02:24:13.030566+0000","last_active":"2023-10-06T02:24:13.030566+0000","last_peered":"2023-10-06T02:24:13.030566+0000","last_clean":"2023-10-06T02:24:13.030566+0000","last_became_active":"2023-10-06T02:24:13.030145+0000","last_became_peered":"2023-10-06T02:24:13.030145+0000","last_unstale":"2023-10-06T02:24:13.030566+0000","last_undegraded":"2023-10-06T02:24:13.030566+0000","last_fullsized":"2023-10-06T02:24:13.030566+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:26:17.427362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.024098+0000","last_change":"2023-10-06T02:24:13.024098+0000","last_active":"2023-10-06T02:24:13.024098+0000","last_peered":"2023-10-06T02:24:13.024098+0000","last_clean":"2023-10-06T02:24:13.024098+0000","last_became_active":"2023-10-06T02:24:13.023785+0000","last_became_peered":"2023-10-06T02:24:13.023785+0000","last_unstale":"2023-10-06T02:24:13.024098+0000","last_undegraded":"2023-10-06T02:24:13.024098+0000","last_fullsized":"2023-10-06T02:24:13.024098+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:00:32.796311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.014926+0000","last_change":"2023-10-06T02:24:13.014926+0000","last_active":"2023-10-06T02:24:13.014926+0000","last_peered":"2023-10-06T02:24:13.014926+0000","last_clean":"2023-10-06T02:24:13.014926+0000","last_became_active":"2023-10-06T02:24:13.014584+0000","last_became_peered":"2023-10-06T02:24:13.014584+0000","last_unstale":"2023-10-06T02:24:13.014926+0000","last_undegraded":"2023-10-06T02:24:13.014926+0000","last_fullsized":"2023-10-06T02:24:13.014926+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:21:14.321298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.018209+0000","last_change":"2023-10-06T02:24:13.018209+0000","last_active":"2023-10-06T02:24:13.018209+0000","last_peered":"2023-10-06T02:24:13.018209+0000","last_clean":"2023-10-06T02:24:13.018209+0000","last_became_active":"2023-10-06T02:24:13.017944+0000","last_became_peered":"2023-10-06T02:24:13.017944+0000","last_unstale":"2023-10-06T02:24:13.018209+0000","last_undegraded":"2023-10-06T02:24:13.018209+0000","last_fullsized":"2023-10-06T02:24:13.018209+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:28:47.052857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.009666+0000","last_change":"2023-10-06T02:24:13.009666+0000","last_active":"2023-10-06T02:24:13.009666+0000","last_peered":"2023-10-06T02:24:13.009666+0000","last_clean":"2023-10-06T02:24:13.009666+0000","last_became_active":"2023-10-06T02:24:13.009358+0000","last_became_peered":"2023-10-06T02:24:13.009358+0000","last_unstale":"2023-10-06T02:24:13.009666+0000","last_undegraded":"2023-10-06T02:24:13.009666+0000","last_fullsized":"2023-10-06T02:24:13.009666+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:58:49.176116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.025616+0000","last_change":"2023-10-06T02:24:13.025616+0000","last_active":"2023-10-06T02:24:13.025616+0000","last_peered":"2023-10-06T02:24:13.025616+0000","last_clean":"2023-10-06T02:24:13.025616+0000","last_became_active":"2023-10-06T02:24:13.022921+0000","last_became_peered":"2023-10-06T02:24:13.022921+0000","last_unstale":"2023-10-06T02:24:13.025616+0000","last_undegraded":"2023-10-06T02:24:13.025616+0000","last_fullsized":"2023-10-06T02:24:13.025616+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:41.374491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.015584+0000","last_change":"2023-10-06T02:24:13.015584+0000","last_active":"2023-10-06T02:24:13.015584+0000","last_peered":"2023-10-06T02:24:13.015584+0000","last_clean":"2023-10-06T02:24:13.015584+0000","last_became_active":"2023-10-06T02:24:13.015339+0000","last_became_peered":"2023-10-06T02:24:13.015339+0000","last_unstale":"2023-10-06T02:24:13.015584+0000","last_undegraded":"2023-10-06T02:24:13.015584+0000","last_fullsized":"2023-10-06T02:24:13.015584+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:00:32.796311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.027964+0000","last_change":"2023-10-06T02:24:13.027964+0000","last_active":"2023-10-06T02:24:13.027964+0000","last_peered":"2023-10-06T02:24:13.027964+0000","last_clean":"2023-10-06T02:24:13.027964+0000","last_became_active":"2023-10-06T02:24:13.027676+0000","last_became_peered":"2023-10-06T02:24:13.027676+0000","last_unstale":"2023-10-06T02:24:13.027964+0000","last_undegraded":"2023-10-06T02:24:13.027964+0000","last_fullsized":"2023-10-06T02:24:13.027964+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:46:19.025627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.030914+0000","last_change":"2023-10-06T02:24:13.030914+0000","last_active":"2023-10-06T02:24:13.030914+0000","last_peered":"2023-10-06T02:24:13.030914+0000","last_clean":"2023-10-06T02:24:13.030914+0000","last_became_active":"2023-10-06T02:24:13.030081+0000","last_became_peered":"2023-10-06T02:24:13.030081+0000","last_unstale":"2023-10-06T02:24:13.030914+0000","last_undegraded":"2023-10-06T02:24:13.030914+0000","last_fullsized":"2023-10-06T02:24:13.030914+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:51:52.519503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.024459+0000","last_change":"2023-10-06T02:24:13.024459+0000","last_active":"2023-10-06T02:24:13.024459+0000","last_peered":"2023-10-06T02:24:13.024459+0000","last_clean":"2023-10-06T02:24:13.024459+0000","last_became_active":"2023-10-06T02:24:13.024181+0000","last_became_peered":"2023-10-06T02:24:13.024181+0000","last_unstale":"2023-10-06T02:24:13.024459+0000","last_undegraded":"2023-10-06T02:24:13.024459+0000","last_fullsized":"2023-10-06T02:24:13.024459+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:58:42.494614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.025476+0000","last_change":"2023-10-06T02:24:13.025476+0000","last_active":"2023-10-06T02:24:13.025476+0000","last_peered":"2023-10-06T02:24:13.025476+0000","last_clean":"2023-10-06T02:24:13.025476+0000","last_became_active":"2023-10-06T02:24:13.025202+0000","last_became_peered":"2023-10-06T02:24:13.025202+0000","last_unstale":"2023-10-06T02:24:13.025476+0000","last_undegraded":"2023-10-06T02:24:13.025476+0000","last_fullsized":"2023-10-06T02:24:13.025476+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:08:46.284322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"35'4","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.040669+0000","last_change":"2023-10-06T02:24:12.017253+0000","last_active":"2023-10-06T02:24:13.040669+0000","last_peered":"2023-10-06T02:24:13.040669+0000","last_clean":"2023-10-06T02:24:13.040669+0000","last_became_active":"2023-10-06T02:24:12.016938+0000","last_became_peered":"2023-10-06T02:24:12.016938+0000","last_unstale":"2023-10-06T02:24:13.040669+0000","last_undegraded":"2023-10-06T02:24:13.040669+0000","last_fullsized":"2023-10-06T02:24:13.040669+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_clean_scrub_stamp":"2023-10-06T02:24:10.982473+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:25:27.976647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.018852+0000","last_change":"2023-10-06T02:24:13.018852+0000","last_active":"2023-10-06T02:24:13.018852+0000","last_peered":"2023-10-06T02:24:13.018852+0000","last_clean":"2023-10-06T02:24:13.018852+0000","last_became_active":"2023-10-06T02:24:13.018482+0000","last_became_peered":"2023-10-06T02:24:13.018482+0000","last_unstale":"2023-10-06T02:24:13.018852+0000","last_undegraded":"2023-10-06T02:24:13.018852+0000","last_fullsized":"2023-10-06T02:24:13.018852+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:50:23.084812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"35'1","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.006363+0000","last_change":"2023-10-06T02:24:12.008843+0000","last_active":"2023-10-06T02:24:13.006363+0000","last_peered":"2023-10-06T02:24:13.006363+0000","last_clean":"2023-10-06T02:24:13.006363+0000","last_became_active":"2023-10-06T02:24:12.008524+0000","last_became_peered":"2023-10-06T02:24:12.008524+0000","last_unstale":"2023-10-06T02:24:13.006363+0000","last_undegraded":"2023-10-06T02:24:13.006363+0000","last_fullsized":"2023-10-06T02:24:13.006363+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_clean_scrub_stamp":"2023-10-06T02:24:10.982473+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:14:32.568412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.015592+0000","last_change":"2023-10-06T02:24:13.015592+0000","last_active":"2023-10-06T02:24:13.015592+0000","last_peered":"2023-10-06T02:24:13.015592+0000","last_clean":"2023-10-06T02:24:13.015592+0000","last_became_active":"2023-10-06T02:24:13.015313+0000","last_became_peered":"2023-10-06T02:24:13.015313+0000","last_unstale":"2023-10-06T02:24:13.015592+0000","last_undegraded":"2023-10-06T02:24:13.015592+0000","last_fullsized":"2023-10-06T02:24:13.015592+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:06:39.375112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"35'2","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.030412+0000","last_change":"2023-10-06T02:24:12.017119+0000","last_active":"2023-10-06T02:24:13.030412+0000","last_peered":"2023-10-06T02:24:13.030412+0000","last_clean":"2023-10-06T02:24:13.030412+0000","last_became_active":"2023-10-06T02:24:12.016871+0000","last_became_peered":"2023-10-06T02:24:12.016871+0000","last_unstale":"2023-10-06T02:24:13.030412+0000","last_undegraded":"2023-10-06T02:24:13.030412+0000","last_fullsized":"2023-10-06T02:24:13.030412+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_clean_scrub_stamp":"2023-10-06T02:24:10.982473+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:40:04.539924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.029928+0000","last_change":"2023-10-06T02:24:13.029928+0000","last_active":"2023-10-06T02:24:13.029928+0000","last_peered":"2023-10-06T02:24:13.029928+0000","last_clean":"2023-10-06T02:24:13.029928+0000","last_became_active":"2023-10-06T02:24:13.029599+0000","last_became_peered":"2023-10-06T02:24:13.029599+0000","last_unstale":"2023-10-06T02:24:13.029928+0000","last_undegraded":"2023-10-06T02:24:13.029928+0000","last_fullsized":"2023-10-06T02:24:13.029928+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:06:39.375112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"35'2","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.036267+0000","last_change":"2023-10-06T02:24:12.017523+0000","last_active":"2023-10-06T02:24:13.036267+0000","last_peered":"2023-10-06T02:24:13.036267+0000","last_clean":"2023-10-06T02:24:13.036267+0000","last_became_active":"2023-10-06T02:24:12.017245+0000","last_became_peered":"2023-10-06T02:24:12.017245+0000","last_unstale":"2023-10-06T02:24:13.036267+0000","last_undegraded":"2023-10-06T02:24:13.036267+0000","last_fullsized":"2023-10-06T02:24:13.036267+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_clean_scrub_stamp":"2023-10-06T02:24:10.982473+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:07:18.069742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.015741+0000","last_change":"2023-10-06T02:24:13.015741+0000","last_active":"2023-10-06T02:24:13.015741+0000","last_peered":"2023-10-06T02:24:13.015741+0000","last_clean":"2023-10-06T02:24:13.015741+0000","last_became_active":"2023-10-06T02:24:13.015439+0000","last_became_peered":"2023-10-06T02:24:13.015439+0000","last_unstale":"2023-10-06T02:24:13.015741+0000","last_undegraded":"2023-10-06T02:24:13.015741+0000","last_fullsized":"2023-10-06T02:24:13.015741+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:26:17.427362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"35'1","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:12.999064+0000","last_change":"2023-10-06T02:24:12.006644+0000","last_active":"2023-10-06T02:24:12.999064+0000","last_peered":"2023-10-06T02:24:12.999064+0000","last_clean":"2023-10-06T02:24:12.999064+0000","last_became_active":"2023-10-06T02:24:12.005752+0000","last_became_peered":"2023-10-06T02:24:12.005752+0000","last_unstale":"2023-10-06T02:24:12.999064+0000","last_undegraded":"2023-10-06T02:24:12.999064+0000","last_fullsized":"2023-10-06T02:24:12.999064+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_clean_scrub_stamp":"2023-10-06T02:24:10.982473+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:28:09.110101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.018753+0000","last_change":"2023-10-06T02:24:13.018753+0000","last_active":"2023-10-06T02:24:13.018753+0000","last_peered":"2023-10-06T02:24:13.018753+0000","last_clean":"2023-10-06T02:24:13.018753+0000","last_became_active":"2023-10-06T02:24:13.018293+0000","last_became_peered":"2023-10-06T02:24:13.018293+0000","last_unstale":"2023-10-06T02:24:13.018753+0000","last_undegraded":"2023-10-06T02:24:13.018753+0000","last_fullsized":"2023-10-06T02:24:13.018753+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:55:41.830177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"35'1","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.006915+0000","last_change":"2023-10-06T02:24:12.013892+0000","last_active":"2023-10-06T02:24:13.006915+0000","last_peered":"2023-10-06T02:24:13.006915+0000","last_clean":"2023-10-06T02:24:13.006915+0000","last_became_active":"2023-10-06T02:24:12.013621+0000","last_became_peered":"2023-10-06T02:24:12.013621+0000","last_unstale":"2023-10-06T02:24:13.006915+0000","last_undegraded":"2023-10-06T02:24:13.006915+0000","last_fullsized":"2023-10-06T02:24:13.006915+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_clean_scrub_stamp":"2023-10-06T02:24:10.982473+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:10:10.757086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.021550+0000","last_change":"2023-10-06T02:24:13.021550+0000","last_active":"2023-10-06T02:24:13.021550+0000","last_peered":"2023-10-06T02:24:13.021550+0000","last_clean":"2023-10-06T02:24:13.021550+0000","last_became_active":"2023-10-06T02:24:13.016453+0000","last_became_peered":"2023-10-06T02:24:13.016453+0000","last_unstale":"2023-10-06T02:24:13.021550+0000","last_undegraded":"2023-10-06T02:24:13.021550+0000","last_fullsized":"2023-10-06T02:24:13.021550+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:25:21.270044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T02:24:12.021783+0000","last_change":"2023-10-06T02:24:12.017759+0000","last_active":"2023-10-06T02:24:12.021783+0000","last_peered":"2023-10-06T02:24:12.021783+0000","last_clean":"2023-10-06T02:24:12.021783+0000","last_became_active":"2023-10-06T02:24:12.017474+0000","last_became_peered":"2023-10-06T02:24:12.017474+0000","last_unstale":"2023-10-06T02:24:12.021783+0000","last_undegraded":"2023-10-06T02:24:12.021783+0000","last_fullsized":"2023-10-06T02:24:12.021783+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_clean_scrub_stamp":"2023-10-06T02:24:10.982473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:58:48.172696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.034898+0000","last_change":"2023-10-06T02:24:13.034898+0000","last_active":"2023-10-06T02:24:13.034898+0000","last_peered":"2023-10-06T02:24:13.034898+0000","last_clean":"2023-10-06T02:24:13.034898+0000","last_became_active":"2023-10-06T02:24:13.034639+0000","last_became_peered":"2023-10-06T02:24:13.034639+0000","last_unstale":"2023-10-06T02:24:13.034898+0000","last_undegraded":"2023-10-06T02:24:13.034898+0000","last_fullsized":"2023-10-06T02:24:13.034898+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:30:18.564694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T02:24:12.019432+0000","last_change":"2023-10-06T02:24:12.016124+0000","last_active":"2023-10-06T02:24:12.019432+0000","last_peered":"2023-10-06T02:24:12.019432+0000","last_clean":"2023-10-06T02:24:12.019432+0000","last_became_active":"2023-10-06T02:24:12.015798+0000","last_became_peered":"2023-10-06T02:24:12.015798+0000","last_unstale":"2023-10-06T02:24:12.019432+0000","last_undegraded":"2023-10-06T02:24:12.019432+0000","last_fullsized":"2023-10-06T02:24:12.019432+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_clean_scrub_stamp":"2023-10-06T02:24:10.982473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:51:39.075546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.012410+0000","last_change":"2023-10-06T02:24:13.012410+0000","last_active":"2023-10-06T02:24:13.012410+0000","last_peered":"2023-10-06T02:24:13.012410+0000","last_clean":"2023-10-06T02:24:13.012410+0000","last_became_active":"2023-10-06T02:24:13.012121+0000","last_became_peered":"2023-10-06T02:24:13.012121+0000","last_unstale":"2023-10-06T02:24:13.012410+0000","last_undegraded":"2023-10-06T02:24:13.012410+0000","last_fullsized":"2023-10-06T02:24:13.012410+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:58:49.176116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T02:24:12.019207+0000","last_change":"2023-10-06T02:24:12.016888+0000","last_active":"2023-10-06T02:24:12.019207+0000","last_peered":"2023-10-06T02:24:12.019207+0000","last_clean":"2023-10-06T02:24:12.019207+0000","last_became_active":"2023-10-06T02:24:12.016250+0000","last_became_peered":"2023-10-06T02:24:12.016250+0000","last_unstale":"2023-10-06T02:24:12.019207+0000","last_undegraded":"2023-10-06T02:24:12.019207+0000","last_fullsized":"2023-10-06T02:24:12.019207+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_clean_scrub_stamp":"2023-10-06T02:24:10.982473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:05:40.679720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.018606+0000","last_change":"2023-10-06T02:24:13.018606+0000","last_active":"2023-10-06T02:24:13.018606+0000","last_peered":"2023-10-06T02:24:13.018606+0000","last_clean":"2023-10-06T02:24:13.018606+0000","last_became_active":"2023-10-06T02:24:13.018211+0000","last_became_peered":"2023-10-06T02:24:13.018211+0000","last_unstale":"2023-10-06T02:24:13.018606+0000","last_undegraded":"2023-10-06T02:24:13.018606+0000","last_fullsized":"2023-10-06T02:24:13.018606+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:07:19.073163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T02:24:12.016305+0000","last_change":"2023-10-06T02:24:12.012693+0000","last_active":"2023-10-06T02:24:12.016305+0000","last_peered":"2023-10-06T02:24:12.016305+0000","last_clean":"2023-10-06T02:24:12.016305+0000","last_became_active":"2023-10-06T02:24:12.012397+0000","last_became_peered":"2023-10-06T02:24:12.012397+0000","last_unstale":"2023-10-06T02:24:12.016305+0000","last_undegraded":"2023-10-06T02:24:12.016305+0000","last_fullsized":"2023-10-06T02:24:12.016305+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_clean_scrub_stamp":"2023-10-06T02:24:10.982473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:14:32.568412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.020233+0000","last_change":"2023-10-06T02:24:13.020233+0000","last_active":"2023-10-06T02:24:13.020233+0000","last_peered":"2023-10-06T02:24:13.020233+0000","last_clean":"2023-10-06T02:24:13.020233+0000","last_became_active":"2023-10-06T02:24:13.019926+0000","last_became_peered":"2023-10-06T02:24:13.019926+0000","last_unstale":"2023-10-06T02:24:13.020233+0000","last_undegraded":"2023-10-06T02:24:13.020233+0000","last_fullsized":"2023-10-06T02:24:13.020233+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:41.374491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"35'3","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.030974+0000","last_change":"2023-10-06T02:24:12.017177+0000","last_active":"2023-10-06T02:24:13.030974+0000","last_peered":"2023-10-06T02:24:13.030974+0000","last_clean":"2023-10-06T02:24:13.030974+0000","last_became_active":"2023-10-06T02:24:12.016918+0000","last_became_peered":"2023-10-06T02:24:12.016918+0000","last_unstale":"2023-10-06T02:24:13.030974+0000","last_undegraded":"2023-10-06T02:24:13.030974+0000","last_fullsized":"2023-10-06T02:24:13.030974+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_clean_scrub_stamp":"2023-10-06T02:24:10.982473+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:38:35.957251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.024378+0000","last_change":"2023-10-06T02:24:13.024378+0000","last_active":"2023-10-06T02:24:13.024378+0000","last_peered":"2023-10-06T02:24:13.024378+0000","last_clean":"2023-10-06T02:24:13.024378+0000","last_became_active":"2023-10-06T02:24:13.024090+0000","last_became_peered":"2023-10-06T02:24:13.024090+0000","last_unstale":"2023-10-06T02:24:13.024378+0000","last_undegraded":"2023-10-06T02:24:13.024378+0000","last_fullsized":"2023-10-06T02:24:13.024378+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:30:18.564694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"35'6","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.038511+0000","last_change":"2023-10-06T02:24:12.007344+0000","last_active":"2023-10-06T02:24:13.038511+0000","last_peered":"2023-10-06T02:24:13.038511+0000","last_clean":"2023-10-06T02:24:13.038511+0000","last_became_active":"2023-10-06T02:24:12.006782+0000","last_became_peered":"2023-10-06T02:24:12.006782+0000","last_unstale":"2023-10-06T02:24:13.038511+0000","last_undegraded":"2023-10-06T02:24:13.038511+0000","last_fullsized":"2023-10-06T02:24:13.038511+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_clean_scrub_stamp":"2023-10-06T02:24:10.982473+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:58:41.491194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.021953+0000","last_change":"2023-10-06T02:24:13.021953+0000","last_active":"2023-10-06T02:24:13.021953+0000","last_peered":"2023-10-06T02:24:13.021953+0000","last_clean":"2023-10-06T02:24:13.021953+0000","last_became_active":"2023-10-06T02:24:13.021677+0000","last_became_peered":"2023-10-06T02:24:13.021677+0000","last_unstale":"2023-10-06T02:24:13.021953+0000","last_undegraded":"2023-10-06T02:24:13.021953+0000","last_fullsized":"2023-10-06T02:24:13.021953+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:51:52.519503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"35'2","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.034521+0000","last_change":"2023-10-06T02:24:12.003980+0000","last_active":"2023-10-06T02:24:13.034521+0000","last_peered":"2023-10-06T02:24:13.034521+0000","last_clean":"2023-10-06T02:24:13.034521+0000","last_became_active":"2023-10-06T02:24:12.003709+0000","last_became_peered":"2023-10-06T02:24:12.003709+0000","last_unstale":"2023-10-06T02:24:13.034521+0000","last_undegraded":"2023-10-06T02:24:13.034521+0000","last_fullsized":"2023-10-06T02:24:13.034521+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_clean_scrub_stamp":"2023-10-06T02:24:10.982473+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:19.833704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.019943+0000","last_change":"2023-10-06T02:24:13.019943+0000","last_active":"2023-10-06T02:24:13.019943+0000","last_peered":"2023-10-06T02:24:13.019943+0000","last_clean":"2023-10-06T02:24:13.019943+0000","last_became_active":"2023-10-06T02:24:13.019678+0000","last_became_peered":"2023-10-06T02:24:13.019678+0000","last_unstale":"2023-10-06T02:24:13.019943+0000","last_undegraded":"2023-10-06T02:24:13.019943+0000","last_fullsized":"2023-10-06T02:24:13.019943+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:08:46.284322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T02:24:12.023013+0000","last_change":"2023-10-06T02:24:12.022164+0000","last_active":"2023-10-06T02:24:12.023013+0000","last_peered":"2023-10-06T02:24:12.023013+0000","last_clean":"2023-10-06T02:24:12.023013+0000","last_became_active":"2023-10-06T02:24:12.021878+0000","last_became_peered":"2023-10-06T02:24:12.021878+0000","last_unstale":"2023-10-06T02:24:12.023013+0000","last_undegraded":"2023-10-06T02:24:12.023013+0000","last_fullsized":"2023-10-06T02:24:12.023013+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_clean_scrub_stamp":"2023-10-06T02:24:10.982473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:21:13.317878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.026809+0000","last_change":"2023-10-06T02:24:13.026809+0000","last_active":"2023-10-06T02:24:13.026809+0000","last_peered":"2023-10-06T02:24:13.026809+0000","last_clean":"2023-10-06T02:24:13.026809+0000","last_became_active":"2023-10-06T02:24:13.026530+0000","last_became_peered":"2023-10-06T02:24:13.026530+0000","last_unstale":"2023-10-06T02:24:13.026809+0000","last_undegraded":"2023-10-06T02:24:13.026809+0000","last_fullsized":"2023-10-06T02:24:13.026809+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:05:07.253674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T02:24:12.020420+0000","last_change":"2023-10-06T02:24:12.017885+0000","last_active":"2023-10-06T02:24:12.020420+0000","last_peered":"2023-10-06T02:24:12.020420+0000","last_clean":"2023-10-06T02:24:12.020420+0000","last_became_active":"2023-10-06T02:24:12.017546+0000","last_became_peered":"2023-10-06T02:24:12.017546+0000","last_unstale":"2023-10-06T02:24:12.020420+0000","last_undegraded":"2023-10-06T02:24:12.020420+0000","last_fullsized":"2023-10-06T02:24:12.020420+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_clean_scrub_stamp":"2023-10-06T02:24:10.982473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:43:26.238381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.024894+0000","last_change":"2023-10-06T02:24:13.024894+0000","last_active":"2023-10-06T02:24:13.024894+0000","last_peered":"2023-10-06T02:24:13.024894+0000","last_clean":"2023-10-06T02:24:13.024894+0000","last_became_active":"2023-10-06T02:24:13.019884+0000","last_became_peered":"2023-10-06T02:24:13.019884+0000","last_unstale":"2023-10-06T02:24:13.024894+0000","last_undegraded":"2023-10-06T02:24:13.024894+0000","last_fullsized":"2023-10-06T02:24:13.024894+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:41.374491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"35'8","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.040785+0000","last_change":"2023-10-06T02:24:12.018329+0000","last_active":"2023-10-06T02:24:13.040785+0000","last_peered":"2023-10-06T02:24:13.040785+0000","last_clean":"2023-10-06T02:24:13.040785+0000","last_became_active":"2023-10-06T02:24:12.018017+0000","last_became_peered":"2023-10-06T02:24:12.018017+0000","last_unstale":"2023-10-06T02:24:13.040785+0000","last_undegraded":"2023-10-06T02:24:13.040785+0000","last_fullsized":"2023-10-06T02:24:13.040785+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_clean_scrub_stamp":"2023-10-06T02:24:10.982473+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:40:04.539924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.025438+0000","last_change":"2023-10-06T02:24:13.025438+0000","last_active":"2023-10-06T02:24:13.025438+0000","last_peered":"2023-10-06T02:24:13.025438+0000","last_clean":"2023-10-06T02:24:13.025438+0000","last_became_active":"2023-10-06T02:24:13.025052+0000","last_became_peered":"2023-10-06T02:24:13.025052+0000","last_unstale":"2023-10-06T02:24:13.025438+0000","last_undegraded":"2023-10-06T02:24:13.025438+0000","last_fullsized":"2023-10-06T02:24:13.025438+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:55:41.830177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"35'1","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:12.998778+0000","last_change":"2023-10-06T02:24:12.018252+0000","last_active":"2023-10-06T02:24:12.998778+0000","last_peered":"2023-10-06T02:24:12.998778+0000","last_clean":"2023-10-06T02:24:12.998778+0000","last_became_active":"2023-10-06T02:24:12.017702+0000","last_became_peered":"2023-10-06T02:24:12.017702+0000","last_unstale":"2023-10-06T02:24:12.998778+0000","last_undegraded":"2023-10-06T02:24:12.998778+0000","last_fullsized":"2023-10-06T02:24:12.998778+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_clean_scrub_stamp":"2023-10-06T02:24:10.982473+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:19.833704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.018397+0000","last_change":"2023-10-06T02:24:13.018397+0000","last_active":"2023-10-06T02:24:13.018397+0000","last_peered":"2023-10-06T02:24:13.018397+0000","last_clean":"2023-10-06T02:24:13.018397+0000","last_became_active":"2023-10-06T02:24:13.018084+0000","last_became_peered":"2023-10-06T02:24:13.018084+0000","last_unstale":"2023-10-06T02:24:13.018397+0000","last_undegraded":"2023-10-06T02:24:13.018397+0000","last_fullsized":"2023-10-06T02:24:13.018397+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:30:18.564694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"35'2","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.040512+0000","last_change":"2023-10-06T02:24:12.020590+0000","last_active":"2023-10-06T02:24:13.040512+0000","last_peered":"2023-10-06T02:24:13.040512+0000","last_clean":"2023-10-06T02:24:13.040512+0000","last_became_active":"2023-10-06T02:24:12.020267+0000","last_became_peered":"2023-10-06T02:24:12.020267+0000","last_unstale":"2023-10-06T02:24:13.040512+0000","last_undegraded":"2023-10-06T02:24:13.040512+0000","last_fullsized":"2023-10-06T02:24:13.040512+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_clean_scrub_stamp":"2023-10-06T02:24:10.982473+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:58:52.832569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.023478+0000","last_change":"2023-10-06T02:24:13.023478+0000","last_active":"2023-10-06T02:24:13.023478+0000","last_peered":"2023-10-06T02:24:13.023478+0000","last_clean":"2023-10-06T02:24:13.023478+0000","last_became_active":"2023-10-06T02:24:13.023207+0000","last_became_peered":"2023-10-06T02:24:13.023207+0000","last_unstale":"2023-10-06T02:24:13.023478+0000","last_undegraded":"2023-10-06T02:24:13.023478+0000","last_fullsized":"2023-10-06T02:24:13.023478+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:14:08.469730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"35'1","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:12.999822+0000","last_change":"2023-10-06T02:24:12.016086+0000","last_active":"2023-10-06T02:24:12.999822+0000","last_peered":"2023-10-06T02:24:12.999822+0000","last_clean":"2023-10-06T02:24:12.999822+0000","last_became_active":"2023-10-06T02:24:12.015731+0000","last_became_peered":"2023-10-06T02:24:12.015731+0000","last_unstale":"2023-10-06T02:24:12.999822+0000","last_undegraded":"2023-10-06T02:24:12.999822+0000","last_fullsized":"2023-10-06T02:24:12.999822+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_clean_scrub_stamp":"2023-10-06T02:24:10.982473+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:43:26.238381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.036419+0000","last_change":"2023-10-06T02:24:13.036419+0000","last_active":"2023-10-06T02:24:13.036419+0000","last_peered":"2023-10-06T02:24:13.036419+0000","last_clean":"2023-10-06T02:24:13.036419+0000","last_became_active":"2023-10-06T02:24:13.036125+0000","last_became_peered":"2023-10-06T02:24:13.036125+0000","last_unstale":"2023-10-06T02:24:13.036419+0000","last_undegraded":"2023-10-06T02:24:13.036419+0000","last_fullsized":"2023-10-06T02:24:13.036419+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:49:00.492055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T02:24:12.024538+0000","last_change":"2023-10-06T02:24:12.023429+0000","last_active":"2023-10-06T02:24:12.024538+0000","last_peered":"2023-10-06T02:24:12.024538+0000","last_clean":"2023-10-06T02:24:12.024538+0000","last_became_active":"2023-10-06T02:24:12.023183+0000","last_became_peered":"2023-10-06T02:24:12.023183+0000","last_unstale":"2023-10-06T02:24:12.024538+0000","last_undegraded":"2023-10-06T02:24:12.024538+0000","last_fullsized":"2023-10-06T02:24:12.024538+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_clean_scrub_stamp":"2023-10-06T02:24:10.982473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:46:18.022206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.029069+0000","last_change":"2023-10-06T02:24:13.029069+0000","last_active":"2023-10-06T02:24:13.029069+0000","last_peered":"2023-10-06T02:24:13.029069+0000","last_clean":"2023-10-06T02:24:13.029069+0000","last_became_active":"2023-10-06T02:24:13.028791+0000","last_became_peered":"2023-10-06T02:24:13.028791+0000","last_unstale":"2023-10-06T02:24:13.029069+0000","last_undegraded":"2023-10-06T02:24:13.029069+0000","last_fullsized":"2023-10-06T02:24:13.029069+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:35:03.860739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"35'2","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.030529+0000","last_change":"2023-10-06T02:24:12.012069+0000","last_active":"2023-10-06T02:24:13.030529+0000","last_peered":"2023-10-06T02:24:13.030529+0000","last_clean":"2023-10-06T02:24:13.030529+0000","last_became_active":"2023-10-06T02:24:12.011751+0000","last_became_peered":"2023-10-06T02:24:12.011751+0000","last_unstale":"2023-10-06T02:24:13.030529+0000","last_undegraded":"2023-10-06T02:24:13.030529+0000","last_fullsized":"2023-10-06T02:24:13.030529+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_clean_scrub_stamp":"2023-10-06T02:24:10.982473+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:25:27.976647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.022657+0000","last_change":"2023-10-06T02:24:13.022657+0000","last_active":"2023-10-06T02:24:13.022657+0000","last_peered":"2023-10-06T02:24:13.022657+0000","last_clean":"2023-10-06T02:24:13.022657+0000","last_became_active":"2023-10-06T02:24:13.021717+0000","last_became_peered":"2023-10-06T02:24:13.021717+0000","last_unstale":"2023-10-06T02:24:13.022657+0000","last_undegraded":"2023-10-06T02:24:13.022657+0000","last_fullsized":"2023-10-06T02:24:13.022657+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:35:03.860739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T02:24:12.019598+0000","last_change":"2023-10-06T02:24:12.018179+0000","last_active":"2023-10-06T02:24:12.019598+0000","last_peered":"2023-10-06T02:24:12.019598+0000","last_clean":"2023-10-06T02:24:12.019598+0000","last_became_active":"2023-10-06T02:24:12.017888+0000","last_became_peered":"2023-10-06T02:24:12.017888+0000","last_unstale":"2023-10-06T02:24:12.019598+0000","last_undegraded":"2023-10-06T02:24:12.019598+0000","last_fullsized":"2023-10-06T02:24:12.019598+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_clean_scrub_stamp":"2023-10-06T02:24:10.982473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:54:17.765586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.002910+0000","last_change":"2023-10-06T02:24:13.002910+0000","last_active":"2023-10-06T02:24:13.002910+0000","last_peered":"2023-10-06T02:24:13.002910+0000","last_clean":"2023-10-06T02:24:13.002910+0000","last_became_active":"2023-10-06T02:24:13.002582+0000","last_became_peered":"2023-10-06T02:24:13.002582+0000","last_unstale":"2023-10-06T02:24:13.002910+0000","last_undegraded":"2023-10-06T02:24:13.002910+0000","last_fullsized":"2023-10-06T02:24:13.002910+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:35:03.860739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T02:24:12.015099+0000","last_change":"2023-10-06T02:24:12.009123+0000","last_active":"2023-10-06T02:24:12.015099+0000","last_peered":"2023-10-06T02:24:12.015099+0000","last_clean":"2023-10-06T02:24:12.015099+0000","last_became_active":"2023-10-06T02:24:12.008858+0000","last_became_peered":"2023-10-06T02:24:12.008858+0000","last_unstale":"2023-10-06T02:24:12.015099+0000","last_undegraded":"2023-10-06T02:24:12.015099+0000","last_fullsized":"2023-10-06T02:24:12.015099+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_clean_scrub_stamp":"2023-10-06T02:24:10.982473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:11:31.954572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.035196+0000","last_change":"2023-10-06T02:24:13.035196+0000","last_active":"2023-10-06T02:24:13.035196+0000","last_peered":"2023-10-06T02:24:13.035196+0000","last_clean":"2023-10-06T02:24:13.035196+0000","last_became_active":"2023-10-06T02:24:13.034839+0000","last_became_peered":"2023-10-06T02:24:13.034839+0000","last_unstale":"2023-10-06T02:24:13.035196+0000","last_undegraded":"2023-10-06T02:24:13.035196+0000","last_fullsized":"2023-10-06T02:24:13.035196+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:11:10.805135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T02:24:12.012610+0000","last_change":"2023-10-06T02:24:12.011322+0000","last_active":"2023-10-06T02:24:12.012610+0000","last_peered":"2023-10-06T02:24:12.012610+0000","last_clean":"2023-10-06T02:24:12.012610+0000","last_became_active":"2023-10-06T02:24:12.011030+0000","last_became_peered":"2023-10-06T02:24:12.011030+0000","last_unstale":"2023-10-06T02:24:12.012610+0000","last_undegraded":"2023-10-06T02:24:12.012610+0000","last_fullsized":"2023-10-06T02:24:12.012610+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_clean_scrub_stamp":"2023-10-06T02:24:10.982473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:57:43.512145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.030168+0000","last_change":"2023-10-06T02:24:13.030168+0000","last_active":"2023-10-06T02:24:13.030168+0000","last_peered":"2023-10-06T02:24:13.030168+0000","last_clean":"2023-10-06T02:24:13.030168+0000","last_became_active":"2023-10-06T02:24:13.029891+0000","last_became_peered":"2023-10-06T02:24:13.029891+0000","last_unstale":"2023-10-06T02:24:13.030168+0000","last_undegraded":"2023-10-06T02:24:13.030168+0000","last_fullsized":"2023-10-06T02:24:13.030168+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:33:34.790463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T02:24:12.011160+0000","last_change":"2023-10-06T02:24:12.008986+0000","last_active":"2023-10-06T02:24:12.011160+0000","last_peered":"2023-10-06T02:24:12.011160+0000","last_clean":"2023-10-06T02:24:12.011160+0000","last_became_active":"2023-10-06T02:24:12.008613+0000","last_became_peered":"2023-10-06T02:24:12.008613+0000","last_unstale":"2023-10-06T02:24:12.011160+0000","last_undegraded":"2023-10-06T02:24:12.011160+0000","last_fullsized":"2023-10-06T02:24:12.011160+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_clean_scrub_stamp":"2023-10-06T02:24:10.982473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:58:41.491194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T02:24:12.012316+0000","last_change":"2023-10-06T02:23:02.054039+0000","last_active":"2023-10-06T02:24:12.012316+0000","last_peered":"2023-10-06T02:24:12.012316+0000","last_clean":"2023-10-06T02:24:12.012316+0000","last_became_active":"2023-10-06T02:23:02.053742+0000","last_became_peered":"2023-10-06T02:23:02.053742+0000","last_unstale":"2023-10-06T02:24:12.012316+0000","last_undegraded":"2023-10-06T02:24:12.012316+0000","last_fullsized":"2023-10-06T02:24:12.012316+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:58:18.846475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.027936+0000","last_change":"2023-10-06T02:24:13.027936+0000","last_active":"2023-10-06T02:24:13.027936+0000","last_peered":"2023-10-06T02:24:13.027936+0000","last_clean":"2023-10-06T02:24:13.027936+0000","last_became_active":"2023-10-06T02:24:13.027684+0000","last_became_peered":"2023-10-06T02:24:13.027684+0000","last_unstale":"2023-10-06T02:24:13.027936+0000","last_undegraded":"2023-10-06T02:24:13.027936+0000","last_fullsized":"2023-10-06T02:24:13.027936+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:25:21.270044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T02:24:12.018991+0000","last_change":"2023-10-06T02:24:12.016641+0000","last_active":"2023-10-06T02:24:12.018991+0000","last_peered":"2023-10-06T02:24:12.018991+0000","last_clean":"2023-10-06T02:24:12.018991+0000","last_became_active":"2023-10-06T02:24:12.016315+0000","last_became_peered":"2023-10-06T02:24:12.016315+0000","last_unstale":"2023-10-06T02:24:12.018991+0000","last_undegraded":"2023-10-06T02:24:12.018991+0000","last_fullsized":"2023-10-06T02:24:12.018991+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_clean_scrub_stamp":"2023-10-06T02:24:10.982473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:19.833704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":25,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T02:24:12.000742+0000","last_change":"2023-10-06T02:23:01.988631+0000","last_active":"2023-10-06T02:24:12.000742+0000","last_peered":"2023-10-06T02:24:12.000742+0000","last_clean":"2023-10-06T02:24:12.000742+0000","last_became_active":"2023-10-06T02:23:01.988303+0000","last_became_peered":"2023-10-06T02:23:01.988303+0000","last_unstale":"2023-10-06T02:24:12.000742+0000","last_undegraded":"2023-10-06T02:24:12.000742+0000","last_fullsized":"2023-10-06T02:24:12.000742+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:38.211466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.021692+0000","last_change":"2023-10-06T02:24:13.021692+0000","last_active":"2023-10-06T02:24:13.021692+0000","last_peered":"2023-10-06T02:24:13.021692+0000","last_clean":"2023-10-06T02:24:13.021692+0000","last_became_active":"2023-10-06T02:24:13.021381+0000","last_became_peered":"2023-10-06T02:24:13.021381+0000","last_unstale":"2023-10-06T02:24:13.021692+0000","last_undegraded":"2023-10-06T02:24:13.021692+0000","last_fullsized":"2023-10-06T02:24:13.021692+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:51:40.078966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T02:24:12.019314+0000","last_change":"2023-10-06T02:24:12.017168+0000","last_active":"2023-10-06T02:24:12.019314+0000","last_peered":"2023-10-06T02:24:12.019314+0000","last_clean":"2023-10-06T02:24:12.019314+0000","last_became_active":"2023-10-06T02:24:12.016872+0000","last_became_peered":"2023-10-06T02:24:12.016872+0000","last_unstale":"2023-10-06T02:24:12.019314+0000","last_undegraded":"2023-10-06T02:24:12.019314+0000","last_fullsized":"2023-10-06T02:24:12.019314+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_clean_scrub_stamp":"2023-10-06T02:24:10.982473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:32:36.219666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T02:24:12.010225+0000","last_change":"2023-10-06T02:23:02.054338+0000","last_active":"2023-10-06T02:24:12.010225+0000","last_peered":"2023-10-06T02:24:12.010225+0000","last_clean":"2023-10-06T02:24:12.010225+0000","last_became_active":"2023-10-06T02:23:02.053991+0000","last_became_peered":"2023-10-06T02:23:02.053991+0000","last_unstale":"2023-10-06T02:24:12.010225+0000","last_undegraded":"2023-10-06T02:24:12.010225+0000","last_fullsized":"2023-10-06T02:24:12.010225+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:39:30.192838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.036480+0000","last_change":"2023-10-06T02:24:13.036480+0000","last_active":"2023-10-06T02:24:13.036480+0000","last_peered":"2023-10-06T02:24:13.036480+0000","last_clean":"2023-10-06T02:24:13.036480+0000","last_became_active":"2023-10-06T02:24:13.036199+0000","last_became_peered":"2023-10-06T02:24:13.036199+0000","last_unstale":"2023-10-06T02:24:13.036480+0000","last_undegraded":"2023-10-06T02:24:13.036480+0000","last_fullsized":"2023-10-06T02:24:13.036480+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:11:10.805135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T02:24:12.016078+0000","last_change":"2023-10-06T02:24:12.011693+0000","last_active":"2023-10-06T02:24:12.016078+0000","last_peered":"2023-10-06T02:24:12.016078+0000","last_clean":"2023-10-06T02:24:12.016078+0000","last_became_active":"2023-10-06T02:24:12.011429+0000","last_became_peered":"2023-10-06T02:24:12.011429+0000","last_unstale":"2023-10-06T02:24:12.016078+0000","last_undegraded":"2023-10-06T02:24:12.016078+0000","last_fullsized":"2023-10-06T02:24:12.016078+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_clean_scrub_stamp":"2023-10-06T02:24:10.982473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:31:28.937752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":25,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T02:24:12.000574+0000","last_change":"2023-10-06T02:23:01.988140+0000","last_active":"2023-10-06T02:24:12.000574+0000","last_peered":"2023-10-06T02:24:12.000574+0000","last_clean":"2023-10-06T02:24:12.000574+0000","last_became_active":"2023-10-06T02:23:01.987769+0000","last_became_peered":"2023-10-06T02:23:01.987769+0000","last_unstale":"2023-10-06T02:24:12.000574+0000","last_undegraded":"2023-10-06T02:24:12.000574+0000","last_fullsized":"2023-10-06T02:24:12.000574+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:20:08.391912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":175,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T02:24:11.999923+0000","last_change":"2023-10-06T02:22:55.032519+0000","last_active":"2023-10-06T02:24:11.999923+0000","last_peered":"2023-10-06T02:24:11.999923+0000","last_clean":"2023-10-06T02:24:11.999923+0000","last_became_active":"2023-10-06T02:22:55.031367+0000","last_became_peered":"2023-10-06T02:22:55.031367+0000","last_unstale":"2023-10-06T02:24:11.999923+0000","last_undegraded":"2023-10-06T02:24:11.999923+0000","last_fullsized":"2023-10-06T02:24:11.999923+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:22:53.309097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:22:53.309097+0000","last_clean_scrub_stamp":"2023-10-06T02:22:53.309097+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:53:46.520909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.017936+0000","last_change":"2023-10-06T02:24:13.017936+0000","last_active":"2023-10-06T02:24:13.017936+0000","last_peered":"2023-10-06T02:24:13.017936+0000","last_clean":"2023-10-06T02:24:13.017936+0000","last_became_active":"2023-10-06T02:24:13.017387+0000","last_became_peered":"2023-10-06T02:24:13.017387+0000","last_unstale":"2023-10-06T02:24:13.017936+0000","last_undegraded":"2023-10-06T02:24:13.017936+0000","last_fullsized":"2023-10-06T02:24:13.017936+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:03:03.246853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"35'2","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.029812+0000","last_change":"2023-10-06T02:24:12.016493+0000","last_active":"2023-10-06T02:24:13.029812+0000","last_peered":"2023-10-06T02:24:13.029812+0000","last_clean":"2023-10-06T02:24:13.029812+0000","last_became_active":"2023-10-06T02:24:12.016203+0000","last_became_peered":"2023-10-06T02:24:12.016203+0000","last_unstale":"2023-10-06T02:24:13.029812+0000","last_undegraded":"2023-10-06T02:24:13.029812+0000","last_fullsized":"2023-10-06T02:24:13.029812+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_clean_scrub_stamp":"2023-10-06T02:24:10.982473+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:31:28.937752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T02:24:12.000008+0000","last_change":"2023-10-06T02:23:01.990773+0000","last_active":"2023-10-06T02:24:12.000008+0000","last_peered":"2023-10-06T02:24:12.000008+0000","last_clean":"2023-10-06T02:24:12.000008+0000","last_became_active":"2023-10-06T02:23:01.990511+0000","last_became_peered":"2023-10-06T02:23:01.990511+0000","last_unstale":"2023-10-06T02:24:12.000008+0000","last_undegraded":"2023-10-06T02:24:12.000008+0000","last_fullsized":"2023-10-06T02:24:12.000008+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:42:54.797731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.014755+0000","last_change":"2023-10-06T02:24:13.014755+0000","last_active":"2023-10-06T02:24:13.014755+0000","last_peered":"2023-10-06T02:24:13.014755+0000","last_clean":"2023-10-06T02:24:13.014755+0000","last_became_active":"2023-10-06T02:24:13.014316+0000","last_became_peered":"2023-10-06T02:24:13.014316+0000","last_unstale":"2023-10-06T02:24:13.014755+0000","last_undegraded":"2023-10-06T02:24:13.014755+0000","last_fullsized":"2023-10-06T02:24:13.014755+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:14:33.571832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T02:24:12.015104+0000","last_change":"2023-10-06T02:24:12.011981+0000","last_active":"2023-10-06T02:24:12.015104+0000","last_peered":"2023-10-06T02:24:12.015104+0000","last_clean":"2023-10-06T02:24:12.015104+0000","last_became_active":"2023-10-06T02:24:12.011705+0000","last_became_peered":"2023-10-06T02:24:12.011705+0000","last_unstale":"2023-10-06T02:24:12.015104+0000","last_undegraded":"2023-10-06T02:24:12.015104+0000","last_fullsized":"2023-10-06T02:24:12.015104+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_clean_scrub_stamp":"2023-10-06T02:24:10.982473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:32:36.219666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T02:24:12.000841+0000","last_change":"2023-10-06T02:23:02.054069+0000","last_active":"2023-10-06T02:24:12.000841+0000","last_peered":"2023-10-06T02:24:12.000841+0000","last_clean":"2023-10-06T02:24:12.000841+0000","last_became_active":"2023-10-06T02:23:02.053739+0000","last_became_peered":"2023-10-06T02:23:02.053739+0000","last_unstale":"2023-10-06T02:24:12.000841+0000","last_undegraded":"2023-10-06T02:24:12.000841+0000","last_fullsized":"2023-10-06T02:24:12.000841+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:57:48.822608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.032447+0000","last_change":"2023-10-06T02:24:13.032447+0000","last_active":"2023-10-06T02:24:13.032447+0000","last_peered":"2023-10-06T02:24:13.032447+0000","last_clean":"2023-10-06T02:24:13.032447+0000","last_became_active":"2023-10-06T02:24:13.032116+0000","last_became_peered":"2023-10-06T02:24:13.032116+0000","last_unstale":"2023-10-06T02:24:13.032447+0000","last_undegraded":"2023-10-06T02:24:13.032447+0000","last_fullsized":"2023-10-06T02:24:13.032447+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:11:10.805135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T02:24:12.019553+0000","last_change":"2023-10-06T02:24:12.016143+0000","last_active":"2023-10-06T02:24:12.019553+0000","last_peered":"2023-10-06T02:24:12.019553+0000","last_clean":"2023-10-06T02:24:12.019553+0000","last_became_active":"2023-10-06T02:24:12.015755+0000","last_became_peered":"2023-10-06T02:24:12.015755+0000","last_unstale":"2023-10-06T02:24:12.019553+0000","last_undegraded":"2023-10-06T02:24:12.019553+0000","last_fullsized":"2023-10-06T02:24:12.019553+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_clean_scrub_stamp":"2023-10-06T02:24:10.982473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:36:34.773084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":23,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T02:24:12.007102+0000","last_change":"2023-10-06T02:23:01.989934+0000","last_active":"2023-10-06T02:24:12.007102+0000","last_peered":"2023-10-06T02:24:12.007102+0000","last_clean":"2023-10-06T02:24:12.007102+0000","last_became_active":"2023-10-06T02:23:01.989706+0000","last_became_peered":"2023-10-06T02:23:01.989706+0000","last_unstale":"2023-10-06T02:24:12.007102+0000","last_undegraded":"2023-10-06T02:24:12.007102+0000","last_fullsized":"2023-10-06T02:24:12.007102+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:06.212251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T02:24:13.019293+0000","last_change":"2023-10-06T02:24:13.019293+0000","last_active":"2023-10-06T02:24:13.019293+0000","last_peered":"2023-10-06T02:24:13.019293+0000","last_clean":"2023-10-06T02:24:13.019293+0000","last_became_active":"2023-10-06T02:24:13.018906+0000","last_became_peered":"2023-10-06T02:24:13.018906+0000","last_unstale":"2023-10-06T02:24:13.019293+0000","last_undegraded":"2023-10-06T02:24:13.019293+0000","last_fullsized":"2023-10-06T02:24:13.019293+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:11.985893+0000","last_clean_scrub_stamp":"2023-10-06T02:24:11.985893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:55:51.032433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T02:24:12.022328+0000","last_change":"2023-10-06T02:24:12.020481+0000","last_active":"2023-10-06T02:24:12.022328+0000","last_peered":"2023-10-06T02:24:12.022328+0000","last_clean":"2023-10-06T02:24:12.022328+0000","last_became_active":"2023-10-06T02:24:12.020183+0000","last_became_peered":"2023-10-06T02:24:12.020183+0000","last_unstale":"2023-10-06T02:24:12.022328+0000","last_undegraded":"2023-10-06T02:24:12.022328+0000","last_fullsized":"2023-10-06T02:24:12.022328+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:10.982473+0000","last_clean_scrub_stamp":"2023-10-06T02:24:10.982473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:14:32.568412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":24,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T02:24:12.002797+0000","last_change":"2023-10-06T02:23:01.992935+0000","last_active":"2023-10-06T02:24:12.002797+0000","last_peered":"2023-10-06T02:24:12.002797+0000","last_clean":"2023-10-06T02:24:12.002797+0000","last_became_active":"2023-10-06T02:23:01.992568+0000","last_became_peered":"2023-10-06T02:23:01.992568+0000","last_unstale":"2023-10-06T02:24:12.002797+0000","last_undegraded":"2023-10-06T02:24:12.002797+0000","last_fullsized":"2023-10-06T02:24:12.002797+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:06.212251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574868,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298244,"kb_used_data":956,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073596,"statfs":{"total":96636764160,"available":96331362304,"internally_reserved":0,"allocated":978944,"data_stored":745721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.45300000000000001}]},{"osd":1,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.65500000000000003}]},{"osd":2,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.68600000000000005}]},{"osd":3,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.76900000000000002}]},{"osd":4,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.64000000000000001}]},{"osd":5,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.80100000000000005}]},{"osd":6,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.78900000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574868,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297680,"kb_used_data":392,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074160,"statfs":{"total":96636764160,"available":96331939840,"internally_reserved":0,"allocated":401408,"data_stored":163023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.63500000000000001}]},{"osd":1,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.66700000000000004}]},{"osd":2,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.73699999999999999}]},{"osd":3,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.65600000000000003}]},{"osd":4,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.54300000000000004}]},{"osd":5,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.60299999999999998}]},{"osd":7,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.458,"5min":1.458,"15min":1.458},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.755}]}]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297664,"kb_used_data":376,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074176,"statfs":{"total":96636764160,"available":96331956224,"internally_reserved":0,"allocated":385024,"data_stored":155353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.788,"5min":1.788,"15min":1.788},"last":0.54400000000000004}]},{"osd":2,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":2.0840000000000001,"5min":2.0840000000000001,"15min":2.0840000000000001},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.51200000000000001}]},{"osd":3,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.861,"5min":1.861,"15min":1.861},"last":0.48099999999999998}]},{"osd":4,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.9279999999999999,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":2.0179999999999998,"5min":2.0179999999999998,"15min":2.0179999999999998},"last":0.70899999999999996}]},{"osd":5,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.9830000000000001,"5min":1.9830000000000001,"15min":1.9830000000000001},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.78600000000000003}]},{"osd":6,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.829,"5min":1.829,"15min":1.829},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":2.052,"5min":2.052,"15min":2.052},"last":0.75800000000000001}]},{"osd":7,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.895,"5min":1.895,"15min":1.895},"last":0.72599999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574868,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298244,"kb_used_data":956,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073596,"statfs":{"total":96636764160,"available":96331362304,"internally_reserved":0,"allocated":978944,"data_stored":745721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.52500000000000002}]},{"osd":2,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.41099999999999998}]},{"osd":3,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.63700000000000001}]},{"osd":4,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.304,"5min":1.304,"15min":1.304},"last":0.88800000000000001}]},{"osd":5,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.90900000000000003}]},{"osd":6,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.86699999999999999}]},{"osd":7,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.84399999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297664,"kb_used_data":376,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074176,"statfs":{"total":96636764160,"available":96331956224,"internally_reserved":0,"allocated":385024,"data_stored":155353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.44400000000000001}]},{"osd":1,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":0.39700000000000002}]},{"osd":3,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.50600000000000001}]},{"osd":4,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.621,"5min":1.621,"15min":1.621},"last":0.79100000000000004}]},{"osd":5,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.746},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.877}]},{"osd":6,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.86199999999999999}]},{"osd":7,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.77800000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297664,"kb_used_data":376,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074176,"statfs":{"total":96636764160,"available":96331956224,"internally_reserved":0,"allocated":385024,"data_stored":155353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":2.024,"5min":2.024,"15min":2.024},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":2.181,"5min":2.181,"15min":2.181},"last":0.33700000000000002}]},{"osd":1,"last update":"Fri Oct 6 02:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":4.492,"5min":4.492,"15min":4.492},"last":0.312},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":4.569,"5min":4.569,"15min":4.569},"last":0.40100000000000002}]},{"osd":2,"last update":"Fri Oct 6 02:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":2.0870000000000002,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":0.504},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":2.4260000000000002,"5min":2.4260000000000002,"15min":2.4260000000000002},"last":0.442}]},{"osd":4,"last update":"Fri Oct 6 02:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":2.214,"5min":2.214,"15min":2.214},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.66600000000000004}]},{"osd":5,"last update":"Fri Oct 6 02:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":2.242,"5min":2.242,"15min":2.242},"last":0.65400000000000003}]},{"osd":6,"last update":"Fri Oct 6 02:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.357,"5min":1.357,"15min":1.357},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.67900000000000005}]},{"osd":7,"last update":"Fri Oct 6 02:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.27,"5min":2.27,"15min":2.27},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.69499999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297664,"kb_used_data":376,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074176,"statfs":{"total":96636764160,"available":96331956224,"internally_reserved":0,"allocated":385024,"data_stored":155353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:23:55 2023","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":12.337,"5min":12.337,"15min":12.337},"last":0.68600000000000005},{"interface":"front","average":{"1min":1.02,"5min":1.02,"15min":1.02},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":9.6539999999999999,"5min":9.6539999999999999,"15min":9.6539999999999999},"last":0.54300000000000004}]},{"osd":1,"last update":"Fri Oct 6 02:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":3.181,"5min":3.181,"15min":3.181},"last":0.74099999999999999},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":12.946999999999999,"5min":12.946999999999999,"15min":12.946999999999999},"last":0.57899999999999996}]},{"osd":2,"last update":"Fri Oct 6 02:23:55 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":12.377000000000001,"5min":12.377000000000001,"15min":12.377000000000001},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":6.3220000000000001,"5min":6.3220000000000001,"15min":6.3220000000000001},"last":0.63200000000000001}]},{"osd":3,"last update":"Fri Oct 6 02:23:55 2023","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":9.7720000000000002,"5min":9.7720000000000002,"15min":9.7720000000000002},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":3.0840000000000001,"5min":3.0840000000000001,"15min":3.0840000000000001},"last":0.64500000000000002}]},{"osd":5,"last update":"Fri Oct 6 02:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":6.2690000000000001,"5min":6.2690000000000001,"15min":6.2690000000000001},"last":0.83199999999999996},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":10.769,"5min":10.769,"15min":10.769},"last":0.61699999999999999}]},{"osd":6,"last update":"Fri Oct 6 02:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":3.1349999999999998,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":0.80900000000000005},{"interface":"front","average":{"1min":1.077,"5min":1.077,"15min":1.077},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":8.0899999999999999,"5min":8.0899999999999999,"15min":8.0899999999999999},"last":0.77300000000000002}]},{"osd":7,"last update":"Fri Oct 6 02:23:55 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":9.7089999999999996,"5min":9.7089999999999996,"15min":9.7089999999999996},"last":0.67000000000000004},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":7.6719999999999997,"5min":7.6719999999999997,"15min":7.6719999999999997},"last":0.86899999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574868,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297684,"kb_used_data":396,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074156,"statfs":{"total":96636764160,"available":96331935744,"internally_reserved":0,"allocated":405504,"data_stored":162581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.63700000000000001}]},{"osd":1,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.754},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.624}]},{"osd":2,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.59899999999999998}]},{"osd":3,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.82199999999999995}]},{"osd":4,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.77700000000000002}]},{"osd":6,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.86599999999999999}]},{"osd":7,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.90800000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T02:24:17.859 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T02:24:17.859 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T02:24:18.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.085+0000 7f9143fff700 1 Processor -- start 2023-10-06T02:24:18.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.086+0000 7f9143fff700 1 -- start start 2023-10-06T02:24:18.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.087+0000 7f9143fff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9144140e40 0x7f91440c6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:18.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.087+0000 7f9143fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f914406de50 0x7f91440aeef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:18.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.087+0000 7f9143fff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91440c6d50 con 0x7f9144140a70 2023-10-06T02:24:18.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.087+0000 7f9143fff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91440c7030 con 0x7f9144140e40 2023-10-06T02:24:18.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.087+0000 7f9143fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91440c7310 con 0x7f914406de50 2023-10-06T02:24:18.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.088+0000 7f91437fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f914406de50 0x7f91440aeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:18.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.088+0000 7f91427fc700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9144140e40 0x7f91440c6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:18.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.088+0000 7f9142ffd700 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f9144140a70 0x7f91440c7df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:38480/0 (socket says 172.21.15.31:38480) 2023-10-06T02:24:18.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.088+0000 7f91437fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f914406de50 0x7f91440aeef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:33384/0 (socket says 172.21.15.31:33384) 2023-10-06T02:24:18.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.088+0000 7f9142ffd700 1 -- 172.21.15.31:0/3450440395 learned_addr learned my addr 172.21.15.31:0/3450440395 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:24:18.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.088+0000 7f9141ffb700 1 -- 172.21.15.31:0/3450440395 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3672837380 0 0) 0x7f91440c6d50 con 0x7f9144140a70 2023-10-06T02:24:18.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.088+0000 7f9141ffb700 1 -- 172.21.15.31:0/3450440395 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f912c003570 con 0x7f9144140a70 2023-10-06T02:24:18.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.088+0000 7f91437fe700 1 -- 172.21.15.31:0/3450440395 >> v2:172.21.15.62:3300/0 conn(0x7f9144140e40 msgr2=0x7f91440c6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:18.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.088+0000 7f91437fe700 1 --2- 172.21.15.31:0/3450440395 >> v2:172.21.15.62:3300/0 conn(0x7f9144140e40 0x7f91440c6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:18.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.088+0000 7f91437fe700 1 -- 172.21.15.31:0/3450440395 >> v1:172.21.15.62:6789/0 conn(0x7f9144140a70 legacy=0x7f91440c7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:18.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.088+0000 7f91437fe700 1 -- 172.21.15.31:0/3450440395 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91440c75f0 con 0x7f914406de50 2023-10-06T02:24:18.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.089+0000 7f91437fe700 1 --2- 172.21.15.31:0/3450440395 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f914406de50 0x7f91440aeef0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f913c009e60 tx=0x7f913c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=357b542591b63d7f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:18.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.089+0000 7f9141ffb700 1 -- 172.21.15.31:0/3450440395 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f913c00e070 con 0x7f914406de50 2023-10-06T02:24:18.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.089+0000 7f9141ffb700 1 -- 172.21.15.31:0/3450440395 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f913c00bab0 con 0x7f914406de50 2023-10-06T02:24:18.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.089+0000 7f9141ffb700 1 -- 172.21.15.31:0/3450440395 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f913c00be70 con 0x7f914406de50 2023-10-06T02:24:18.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.089+0000 7f9143fff700 1 -- 172.21.15.31:0/3450440395 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f914406de50 msgr2=0x7f91440aeef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:18.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.089+0000 7f9143fff700 1 --2- 172.21.15.31:0/3450440395 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f914406de50 0x7f91440aeef0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f913c009e60 tx=0x7f913c009780 comp rx=0 tx=0).stop 2023-10-06T02:24:18.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.089+0000 7f9143fff700 1 -- 172.21.15.31:0/3450440395 shutdown_connections 2023-10-06T02:24:18.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.089+0000 7f9143fff700 1 --2- 172.21.15.31:0/3450440395 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f914406de50 0x7f91440aeef0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:18.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.089+0000 7f9143fff700 1 --2- 172.21.15.31:0/3450440395 >> v2:172.21.15.62:3300/0 conn(0x7f9144140e40 0x7f91440c6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:18.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.090+0000 7f9143fff700 1 -- 172.21.15.31:0/3450440395 >> 172.21.15.31:0/3450440395 conn(0x7f914406d170 msgr2=0x7f91440cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:18.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.090+0000 7f9143fff700 1 -- 172.21.15.31:0/3450440395 shutdown_connections 2023-10-06T02:24:18.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.090+0000 7f9143fff700 1 -- 172.21.15.31:0/3450440395 wait complete. 2023-10-06T02:24:18.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.091+0000 7f9143fff700 1 Processor -- start 2023-10-06T02:24:18.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.091+0000 7f9143fff700 1 -- start start 2023-10-06T02:24:18.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.092+0000 7f9143fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f914406de50 0x7f914413bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:18.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.092+0000 7f9143fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9144140a70 0x7f91440c2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:18.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.092+0000 7f9143fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9144140e40 0x7f91440c28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:18.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.092+0000 7f9143fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91440c9d00 con 0x7f9144140e40 2023-10-06T02:24:18.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.092+0000 7f9143fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91440cab60 con 0x7f9144140a70 2023-10-06T02:24:18.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.092+0000 7f9143fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f914413c190 con 0x7f914406de50 2023-10-06T02:24:18.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.092+0000 7f91437fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9144140e40 0x7f91440c28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:18.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.092+0000 7f91437fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9144140e40 0x7f91440c28d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46618/0 (socket says 172.21.15.31:46618) 2023-10-06T02:24:18.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.092+0000 7f91437fe700 1 -- 172.21.15.31:0/2816357914 learned_addr learned my addr 172.21.15.31:0/2816357914 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:18.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.093+0000 7f9142ffd700 1 --2- 172.21.15.31:0/2816357914 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f914406de50 0x7f914413bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:18.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.093+0000 7f91427fc700 1 --2- 172.21.15.31:0/2816357914 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9144140a70 0x7f91440c2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:18.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.093+0000 7f91437fe700 1 -- 172.21.15.31:0/2816357914 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f914406de50 msgr2=0x7f914413bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:18.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.093+0000 7f91437fe700 1 --2- 172.21.15.31:0/2816357914 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f914406de50 0x7f914413bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:18.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.093+0000 7f91437fe700 1 -- 172.21.15.31:0/2816357914 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9144140a70 msgr2=0x7f91440c2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:18.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.093+0000 7f91437fe700 1 --2- 172.21.15.31:0/2816357914 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9144140a70 0x7f91440c2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:18.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.093+0000 7f91437fe700 1 -- 172.21.15.31:0/2816357914 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f913c009a20 con 0x7f9144140e40 2023-10-06T02:24:18.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.093+0000 7f91427fc700 1 --2- 172.21.15.31:0/2816357914 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9144140a70 0x7f91440c2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:24:18.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.093+0000 7f91437fe700 1 --2- 172.21.15.31:0/2816357914 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9144140e40 0x7f91440c28d0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f91440c7450 tx=0x7f913c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:18.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.093+0000 7f912affd700 1 -- 172.21.15.31:0/2816357914 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f913c027470 con 0x7f9144140e40 2023-10-06T02:24:18.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.093+0000 7f912affd700 1 -- 172.21.15.31:0/2816357914 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f913c0275f0 con 0x7f9144140e40 2023-10-06T02:24:18.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.093+0000 7f912affd700 1 -- 172.21.15.31:0/2816357914 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f913c0279b0 con 0x7f9144140e40 2023-10-06T02:24:18.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.093+0000 7f9143fff700 1 -- 172.21.15.31:0/2816357914 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91440c9f50 con 0x7f9144140e40 2023-10-06T02:24:18.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.094+0000 7f9143fff700 1 -- 172.21.15.31:0/2816357914 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91440ca510 con 0x7f9144140e40 2023-10-06T02:24:18.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.095+0000 7f9143fff700 1 -- 172.21.15.31:0/2816357914 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f914404c3c0 con 0x7f9144140e40 2023-10-06T02:24:18.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.095+0000 7f912affd700 1 -- 172.21.15.31:0/2816357914 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f913c024070 con 0x7f9144140e40 2023-10-06T02:24:18.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.096+0000 7f912affd700 1 --2- 172.21.15.31:0/2816357914 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f9130066f30 0x7f91300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:18.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.097+0000 7f912affd700 1 -- 172.21.15.31:0/2816357914 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f913c087080 con 0x7f9144140e40 2023-10-06T02:24:18.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.097+0000 7f9142ffd700 1 --2- 172.21.15.31:0/2816357914 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f9130066f30 0x7f91300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:18.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.097+0000 7f9142ffd700 1 --2- 172.21.15.31:0/2816357914 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f9130066f30 0x7f91300693e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9134007fd0 tx=0x7f9134009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:18.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.101+0000 7f912affd700 1 -- 172.21.15.31:0/2816357914 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f913c0b85a0 con 0x7f9144140e40 2023-10-06T02:24:18.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.385+0000 7f9143fff700 1 -- 172.21.15.31:0/2816357914 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f91440c78d0 con 0x7f9144140e40 2023-10-06T02:24:18.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.389+0000 7f912affd700 1 -- 172.21.15.31:0/2816357914 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) v1 ==== 190+0+0 (secure 0 0 0) 0x7f913c0540f0 con 0x7f9144140e40 2023-10-06T02:24:18.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.392+0000 7f9143fff700 1 -- 172.21.15.31:0/2816357914 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f9130066f30 msgr2=0x7f91300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:18.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.392+0000 7f9143fff700 1 --2- 172.21.15.31:0/2816357914 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f9130066f30 0x7f91300693e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9134007fd0 tx=0x7f9134009040 comp rx=0 tx=0).stop 2023-10-06T02:24:18.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.392+0000 7f9143fff700 1 -- 172.21.15.31:0/2816357914 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9144140e40 msgr2=0x7f91440c28d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:18.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.392+0000 7f9143fff700 1 --2- 172.21.15.31:0/2816357914 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9144140e40 0x7f91440c28d0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f91440c7450 tx=0x7f913c019000 comp rx=0 tx=0).stop 2023-10-06T02:24:18.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.392+0000 7f9143fff700 1 -- 172.21.15.31:0/2816357914 shutdown_connections 2023-10-06T02:24:18.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.392+0000 7f9143fff700 1 --2- 172.21.15.31:0/2816357914 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9144140e40 0x7f91440c28d0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:18.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.392+0000 7f9143fff700 1 --2- 172.21.15.31:0/2816357914 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f9130066f30 0x7f91300693e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:18.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.392+0000 7f9143fff700 1 --2- 172.21.15.31:0/2816357914 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f914406de50 0x7f914413bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:18.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.392+0000 7f9143fff700 1 --2- 172.21.15.31:0/2816357914 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9144140a70 0x7f91440c2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:18.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.392+0000 7f9143fff700 1 -- 172.21.15.31:0/2816357914 >> 172.21.15.31:0/2816357914 conn(0x7f914406d170 msgr2=0x7f914413ec70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:18.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.392+0000 7f9143fff700 1 -- 172.21.15.31:0/2816357914 shutdown_connections 2023-10-06T02:24:18.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.392+0000 7f9143fff700 1 -- 172.21.15.31:0/2816357914 wait complete. 2023-10-06T02:24:18.395 INFO:teuthology.orchestra.run.smithi031.stderr:updated caps for client.0 2023-10-06T02:24:18.411 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:24:18.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.595+0000 7f850219d700 1 Processor -- start 2023-10-06T02:24:18.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.595+0000 7f850219d700 1 -- start start 2023-10-06T02:24:18.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.597+0000 7f850219d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f84fc142c00 0x7f84fc06b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:18.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.597+0000 7f850219d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f84fc06bd20 0x7f84fc0c7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:18.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.597+0000 7f850219d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f84fc142fd0 0x7f84fc0c7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:18.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.597+0000 7f850219d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84fc0ca480 con 0x7f84fc06bd20 2023-10-06T02:24:18.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.597+0000 7f850219d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84fc15e240 con 0x7f84fc142fd0 2023-10-06T02:24:18.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.597+0000 7f850219d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84fc06b350 con 0x7f84fc142c00 2023-10-06T02:24:18.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.597+0000 7f850099a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f84fc06bd20 0x7f84fc0c7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:18.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.597+0000 7f850099a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f84fc06bd20 0x7f84fc0c7880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46634/0 (socket says 172.21.15.31:46634) 2023-10-06T02:24:18.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.597+0000 7f850099a700 1 -- 172.21.15.31:0/1272299947 learned_addr learned my addr 172.21.15.31:0/1272299947 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:18.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.597+0000 7f850119b700 1 --2- 172.21.15.31:0/1272299947 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f84fc142c00 0x7f84fc06b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:18.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.597+0000 7f850199c700 1 --2- 172.21.15.31:0/1272299947 >> v2:172.21.15.62:3300/0 conn(0x7f84fc142fd0 0x7f84fc0c7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:18.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.597+0000 7f850099a700 1 -- 172.21.15.31:0/1272299947 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f84fc142c00 msgr2=0x7f84fc06b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:18.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.598+0000 7f850099a700 1 --2- 172.21.15.31:0/1272299947 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f84fc142c00 0x7f84fc06b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:18.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.598+0000 7f850099a700 1 -- 172.21.15.31:0/1272299947 >> v2:172.21.15.62:3300/0 conn(0x7f84fc142fd0 msgr2=0x7f84fc0c7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:18.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.598+0000 7f850099a700 1 --2- 172.21.15.31:0/1272299947 >> v2:172.21.15.62:3300/0 conn(0x7f84fc142fd0 0x7f84fc0c7dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:18.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.598+0000 7f850099a700 1 -- 172.21.15.31:0/1272299947 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84fc15df60 con 0x7f84fc06bd20 2023-10-06T02:24:18.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.598+0000 7f850199c700 1 --2- 172.21.15.31:0/1272299947 >> v2:172.21.15.62:3300/0 conn(0x7f84fc142fd0 0x7f84fc0c7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:24:18.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.598+0000 7f850099a700 1 --2- 172.21.15.31:0/1272299947 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84fc06bd20 0x7f84fc0c7880 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f84fc0ca8c0 tx=0x7f84ec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6794e36d21a8fdb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:18.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.598+0000 7f84f3fff700 1 -- 172.21.15.31:0/1272299947 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ec00b860 con 0x7f84fc06bd20 2023-10-06T02:24:18.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.598+0000 7f84f3fff700 1 -- 172.21.15.31:0/1272299947 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84ec00b9c0 con 0x7f84fc06bd20 2023-10-06T02:24:18.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.598+0000 7f84f3fff700 1 -- 172.21.15.31:0/1272299947 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ec00bd80 con 0x7f84fc06bd20 2023-10-06T02:24:18.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.599+0000 7f850219d700 1 -- 172.21.15.31:0/1272299947 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84fc06bd20 msgr2=0x7f84fc0c7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:18.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.599+0000 7f850219d700 1 --2- 172.21.15.31:0/1272299947 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84fc06bd20 0x7f84fc0c7880 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f84fc0ca8c0 tx=0x7f84ec009250 comp rx=0 tx=0).stop 2023-10-06T02:24:18.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.599+0000 7f850219d700 1 -- 172.21.15.31:0/1272299947 shutdown_connections 2023-10-06T02:24:18.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.599+0000 7f850219d700 1 --2- 172.21.15.31:0/1272299947 >> v2:172.21.15.62:3300/0 conn(0x7f84fc142fd0 0x7f84fc0c7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:18.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.599+0000 7f850219d700 1 --2- 172.21.15.31:0/1272299947 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f84fc142c00 0x7f84fc06b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:18.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.599+0000 7f850219d700 1 --2- 172.21.15.31:0/1272299947 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84fc06bd20 0x7f84fc0c7880 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:18.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.599+0000 7f850219d700 1 -- 172.21.15.31:0/1272299947 >> 172.21.15.31:0/1272299947 conn(0x7f84fc0675a0 msgr2=0x7f84fc0679a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:18.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.599+0000 7f850219d700 1 -- 172.21.15.31:0/1272299947 shutdown_connections 2023-10-06T02:24:18.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.599+0000 7f850219d700 1 -- 172.21.15.31:0/1272299947 wait complete. 2023-10-06T02:24:18.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.600+0000 7f850219d700 1 Processor -- start 2023-10-06T02:24:18.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.600+0000 7f850219d700 1 -- start start 2023-10-06T02:24:18.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.601+0000 7f850219d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84fc06bd20 0x7f84fc1309e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:18.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.601+0000 7f850219d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84fc142c00 0x7f84fc13dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:18.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.601+0000 7f850219d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f84fc142fd0 0x7f84fc140fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:18.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.601+0000 7f850219d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84fc0c72a0 con 0x7f84fc06bd20 2023-10-06T02:24:18.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.601+0000 7f850219d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84fc0affe0 con 0x7f84fc142c00 2023-10-06T02:24:18.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.601+0000 7f850219d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84fc0b02c0 con 0x7f84fc142fd0 2023-10-06T02:24:18.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.601+0000 7f850119b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84fc06bd20 0x7f84fc1309e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:18.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.602+0000 7f850119b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84fc06bd20 0x7f84fc1309e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46640/0 (socket says 172.21.15.31:46640) 2023-10-06T02:24:18.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.602+0000 7f850119b700 1 -- 172.21.15.31:0/3920457911 learned_addr learned my addr 172.21.15.31:0/3920457911 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:18.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.602+0000 7f850199c700 1 --2- 172.21.15.31:0/3920457911 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f84fc142fd0 0x7f84fc140fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:18.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.602+0000 7f850099a700 1 --2- 172.21.15.31:0/3920457911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84fc142c00 0x7f84fc13dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:18.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.602+0000 7f850119b700 1 -- 172.21.15.31:0/3920457911 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f84fc142fd0 msgr2=0x7f84fc140fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:18.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.602+0000 7f850119b700 1 --2- 172.21.15.31:0/3920457911 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f84fc142fd0 0x7f84fc140fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:18.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.602+0000 7f850119b700 1 -- 172.21.15.31:0/3920457911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84fc142c00 msgr2=0x7f84fc13dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:18.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.602+0000 7f850119b700 1 --2- 172.21.15.31:0/3920457911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84fc142c00 0x7f84fc13dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:18.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.602+0000 7f850119b700 1 -- 172.21.15.31:0/3920457911 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84ec0094f0 con 0x7f84fc06bd20 2023-10-06T02:24:18.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.602+0000 7f850119b700 1 --2- 172.21.15.31:0/3920457911 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84fc06bd20 0x7f84fc1309e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f84f800d890 tx=0x7f84f8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:18.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.602+0000 7f84f17fa700 1 -- 172.21.15.31:0/3920457911 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f80125f0 con 0x7f84fc06bd20 2023-10-06T02:24:18.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.603+0000 7f84f17fa700 1 -- 172.21.15.31:0/3920457911 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84f8012770 con 0x7f84fc06bd20 2023-10-06T02:24:18.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.603+0000 7f850219d700 1 -- 172.21.15.31:0/3920457911 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84fc0b05a0 con 0x7f84fc06bd20 2023-10-06T02:24:18.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.603+0000 7f84f17fa700 1 -- 172.21.15.31:0/3920457911 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f8012b30 con 0x7f84fc06bd20 2023-10-06T02:24:18.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.603+0000 7f850219d700 1 -- 172.21.15.31:0/3920457911 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84fc0c6f20 con 0x7f84fc06bd20 2023-10-06T02:24:18.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.604+0000 7f850219d700 1 -- 172.21.15.31:0/3920457911 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84fc04c3c0 con 0x7f84fc06bd20 2023-10-06T02:24:18.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.604+0000 7f84f17fa700 1 -- 172.21.15.31:0/3920457911 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f84f801e460 con 0x7f84fc06bd20 2023-10-06T02:24:18.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.606+0000 7f84f17fa700 1 --2- 172.21.15.31:0/3920457911 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f84e0066f30 0x7f84e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:18.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.606+0000 7f84f17fa700 1 -- 172.21.15.31:0/3920457911 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f84f8084d00 con 0x7f84fc06bd20 2023-10-06T02:24:18.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.606+0000 7f850099a700 1 --2- 172.21.15.31:0/3920457911 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f84e0066f30 0x7f84e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:18.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.607+0000 7f850099a700 1 --2- 172.21.15.31:0/3920457911 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f84e0066f30 0x7f84e00693e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f84fc0ca8c0 tx=0x7f84ec017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:18.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.610+0000 7f84f17fa700 1 -- 172.21.15.31:0/3920457911 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f84f8051d70 con 0x7f84fc06bd20 2023-10-06T02:24:18.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.855+0000 7f850219d700 1 -- 172.21.15.31:0/3920457911 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f84fc06ad00 con 0x7f84fc06bd20 2023-10-06T02:24:18.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.856+0000 7f84f17fa700 1 -- 172.21.15.31:0/3920457911 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4591 (secure 0 0 0) 0x7f84f8055390 con 0x7f84fc06bd20 2023-10-06T02:24:18.857 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:24:18.858 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4700,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/384287452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":384287452},{"type":"v1","addr":"172.21.15.62:6834","nonce":384287452}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5037,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/693643908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":693643908},{"type":"v1","addr":"172.21.15.31:6836","nonce":693643908}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5040,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2599441423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2599441423},{"type":"v1","addr":"172.21.15.62:6835","nonce":2599441423}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:24:12.493929+0000","modified":"2023-10-06T02:24:16.026454+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5043},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5043":{"gid":5043,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.31:6837/2608294748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":2608294748},{"type":"v1","addr":"172.21.15.31:6837","nonce":2608294748}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T02:24:18.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.858+0000 7f850219d700 1 -- 172.21.15.31:0/3920457911 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f84e0066f30 msgr2=0x7f84e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:18.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.858+0000 7f850219d700 1 --2- 172.21.15.31:0/3920457911 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f84e0066f30 0x7f84e00693e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f84fc0ca8c0 tx=0x7f84ec017000 comp rx=0 tx=0).stop 2023-10-06T02:24:18.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.858+0000 7f850219d700 1 -- 172.21.15.31:0/3920457911 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84fc06bd20 msgr2=0x7f84fc1309e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:18.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.858+0000 7f850219d700 1 --2- 172.21.15.31:0/3920457911 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84fc06bd20 0x7f84fc1309e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f84f800d890 tx=0x7f84f8008040 comp rx=0 tx=0).stop 2023-10-06T02:24:18.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.858+0000 7f850219d700 1 -- 172.21.15.31:0/3920457911 shutdown_connections 2023-10-06T02:24:18.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.858+0000 7f850219d700 1 --2- 172.21.15.31:0/3920457911 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f84e0066f30 0x7f84e00693e0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:18.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.858+0000 7f850219d700 1 --2- 172.21.15.31:0/3920457911 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f84fc142fd0 0x7f84fc140fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:18.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.858+0000 7f850219d700 1 --2- 172.21.15.31:0/3920457911 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84fc06bd20 0x7f84fc1309e0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:18.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.858+0000 7f850219d700 1 --2- 172.21.15.31:0/3920457911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84fc142c00 0x7f84fc13dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:18.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.858+0000 7f850219d700 1 -- 172.21.15.31:0/3920457911 >> 172.21.15.31:0/3920457911 conn(0x7f84fc0675a0 msgr2=0x7f84fc0c8360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:18.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.859+0000 7f850219d700 1 -- 172.21.15.31:0/3920457911 shutdown_connections 2023-10-06T02:24:18.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:18.859+0000 7f850219d700 1 -- 172.21.15.31:0/3920457911 wait complete. 2023-10-06T02:24:18.861 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 17 2023-10-06T02:24:19.279 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T02:24:12.493929+0000', 'modified': '2023-10-06T02:24:16.026454+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5043}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5043': {'gid': 5043, 'name': 'c', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.31:6837/2608294748', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6835', 'nonce': 2608294748}, {'type': 'v1', 'addr': '172.21.15.31:6837', 'nonce': 2608294748}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T02:24:19.279 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T02:24:19.279 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-06T02:24:19.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.469+0000 7fb4789fe700 1 Processor -- start 2023-10-06T02:24:19.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.469+0000 7fb4789fe700 1 -- start start 2023-10-06T02:24:19.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.471+0000 7fb4789fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb474159550 0x7fb4740ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:19.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.471+0000 7fb4789fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb47415dfd0 0x7fb4740aedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:19.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.471+0000 7fb4789fe700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb474159920 0x7fb4740ad090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:19.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.471+0000 7fb4789fe700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4740acc00 con 0x7fb47415dfd0 2023-10-06T02:24:19.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.471+0000 7fb4789fe700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4740abff0 con 0x7fb474159920 2023-10-06T02:24:19.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.471+0000 7fb4789fe700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4740ac2d0 con 0x7fb474159550 2023-10-06T02:24:19.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.471+0000 7fb47259c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb47415dfd0 0x7fb4740aedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:19.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.471+0000 7fb47259c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb47415dfd0 0x7fb4740aedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46650/0 (socket says 172.21.15.31:46650) 2023-10-06T02:24:19.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.471+0000 7fb47259c700 1 -- 172.21.15.31:0/781057361 learned_addr learned my addr 172.21.15.31:0/781057361 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:19.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.471+0000 7fb472d9d700 1 --2- 172.21.15.31:0/781057361 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb474159550 0x7fb4740ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:19.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.471+0000 7fb47359e700 1 --2- 172.21.15.31:0/781057361 >> v2:172.21.15.62:3300/0 conn(0x7fb474159920 0x7fb4740ad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:19.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.471+0000 7fb47259c700 1 -- 172.21.15.31:0/781057361 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb474159550 msgr2=0x7fb4740ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:19.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.471+0000 7fb47259c700 1 --2- 172.21.15.31:0/781057361 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb474159550 0x7fb4740ceac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:19.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.471+0000 7fb47259c700 1 -- 172.21.15.31:0/781057361 >> v2:172.21.15.62:3300/0 conn(0x7fb474159920 msgr2=0x7fb4740ad090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:19.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.471+0000 7fb47259c700 1 --2- 172.21.15.31:0/781057361 >> v2:172.21.15.62:3300/0 conn(0x7fb474159920 0x7fb4740ad090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:19.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.471+0000 7fb47259c700 1 -- 172.21.15.31:0/781057361 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4740ac5b0 con 0x7fb47415dfd0 2023-10-06T02:24:19.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.472+0000 7fb47259c700 1 --2- 172.21.15.31:0/781057361 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb47415dfd0 0x7fb4740aedb0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb47406c180 tx=0x7fb468009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87cfe318a34f856f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:19.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.472+0000 7fb471d9b700 1 -- 172.21.15.31:0/781057361 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb46800b860 con 0x7fb47415dfd0 2023-10-06T02:24:19.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.472+0000 7fb471d9b700 1 -- 172.21.15.31:0/781057361 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb46800b9c0 con 0x7fb47415dfd0 2023-10-06T02:24:19.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.472+0000 7fb471d9b700 1 -- 172.21.15.31:0/781057361 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb46800bd60 con 0x7fb47415dfd0 2023-10-06T02:24:19.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.472+0000 7fb4789fe700 1 -- 172.21.15.31:0/781057361 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb47415dfd0 msgr2=0x7fb4740aedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:19.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.472+0000 7fb4789fe700 1 --2- 172.21.15.31:0/781057361 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb47415dfd0 0x7fb4740aedb0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb47406c180 tx=0x7fb468009250 comp rx=0 tx=0).stop 2023-10-06T02:24:19.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.473+0000 7fb4789fe700 1 -- 172.21.15.31:0/781057361 shutdown_connections 2023-10-06T02:24:19.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.473+0000 7fb4789fe700 1 --2- 172.21.15.31:0/781057361 >> v2:172.21.15.62:3300/0 conn(0x7fb474159920 0x7fb4740ad090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:19.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.473+0000 7fb4789fe700 1 --2- 172.21.15.31:0/781057361 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb474159550 0x7fb4740ceac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:19.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.473+0000 7fb4789fe700 1 --2- 172.21.15.31:0/781057361 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb47415dfd0 0x7fb4740aedb0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:19.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.473+0000 7fb4789fe700 1 -- 172.21.15.31:0/781057361 >> 172.21.15.31:0/781057361 conn(0x7fb47406ce80 msgr2=0x7fb4740ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:19.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.473+0000 7fb4789fe700 1 -- 172.21.15.31:0/781057361 shutdown_connections 2023-10-06T02:24:19.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.473+0000 7fb4789fe700 1 -- 172.21.15.31:0/781057361 wait complete. 2023-10-06T02:24:19.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.474+0000 7fb4789fe700 1 Processor -- start 2023-10-06T02:24:19.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.474+0000 7fb4789fe700 1 -- start start 2023-10-06T02:24:19.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.475+0000 7fb4789fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb474159550 0x7fb4740c7560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:19.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.475+0000 7fb4789fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb474159920 0x7fb4740c7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:19.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.475+0000 7fb4789fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb47415dfd0 0x7fb4740cb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:19.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.475+0000 7fb4789fe700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4740b1fd0 con 0x7fb47415dfd0 2023-10-06T02:24:19.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.475+0000 7fb4789fe700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4740cce30 con 0x7fb474159550 2023-10-06T02:24:19.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.475+0000 7fb4789fe700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4740cc870 con 0x7fb474159920 2023-10-06T02:24:19.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.476+0000 7fb47359e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb47415dfd0 0x7fb4740cb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:19.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.476+0000 7fb47359e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb47415dfd0 0x7fb4740cb8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46652/0 (socket says 172.21.15.31:46652) 2023-10-06T02:24:19.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.476+0000 7fb47359e700 1 -- 172.21.15.31:0/2862022049 learned_addr learned my addr 172.21.15.31:0/2862022049 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:19.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.476+0000 7fb472d9d700 1 --2- 172.21.15.31:0/2862022049 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb474159550 0x7fb4740c7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:19.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.476+0000 7fb47259c700 1 --2- 172.21.15.31:0/2862022049 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb474159920 0x7fb4740c7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:19.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.476+0000 7fb47359e700 1 -- 172.21.15.31:0/2862022049 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb474159920 msgr2=0x7fb4740c7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:19.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.476+0000 7fb47359e700 1 --2- 172.21.15.31:0/2862022049 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb474159920 0x7fb4740c7aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:19.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.476+0000 7fb47359e700 1 -- 172.21.15.31:0/2862022049 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb474159550 msgr2=0x7fb4740c7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:19.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.476+0000 7fb47359e700 1 --2- 172.21.15.31:0/2862022049 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb474159550 0x7fb4740c7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:19.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.476+0000 7fb47359e700 1 -- 172.21.15.31:0/2862022049 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4680094f0 con 0x7fb47415dfd0 2023-10-06T02:24:19.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.476+0000 7fb472d9d700 1 --2- 172.21.15.31:0/2862022049 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb474159550 0x7fb4740c7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:24:19.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.476+0000 7fb47259c700 1 --2- 172.21.15.31:0/2862022049 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb474159920 0x7fb4740c7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:24:19.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.476+0000 7fb47359e700 1 --2- 172.21.15.31:0/2862022049 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb47415dfd0 0x7fb4740cb8e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb46c009fd0 tx=0x7fb46c009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:19.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.476+0000 7fb45affd700 1 -- 172.21.15.31:0/2862022049 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb46c00ed50 con 0x7fb47415dfd0 2023-10-06T02:24:19.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.476+0000 7fb45affd700 1 -- 172.21.15.31:0/2862022049 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb46c00eeb0 con 0x7fb47415dfd0 2023-10-06T02:24:19.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.476+0000 7fb4789fe700 1 -- 172.21.15.31:0/2862022049 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4740ccb50 con 0x7fb47415dfd0 2023-10-06T02:24:19.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.476+0000 7fb45affd700 1 -- 172.21.15.31:0/2862022049 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb46c014650 con 0x7fb47415dfd0 2023-10-06T02:24:19.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.477+0000 7fb4789fe700 1 -- 172.21.15.31:0/2862022049 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4740ca7f0 con 0x7fb47415dfd0 2023-10-06T02:24:19.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.479+0000 7fb45affd700 1 -- 172.21.15.31:0/2862022049 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb46c0149c0 con 0x7fb47415dfd0 2023-10-06T02:24:19.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.480+0000 7fb45affd700 1 --2- 172.21.15.31:0/2862022049 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fb460066f30 0x7fb4600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:19.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.480+0000 7fb45affd700 1 -- 172.21.15.31:0/2862022049 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb46c084f30 con 0x7fb47415dfd0 2023-10-06T02:24:19.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.480+0000 7fb472d9d700 1 --2- 172.21.15.31:0/2862022049 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fb460066f30 0x7fb4600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:19.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.480+0000 7fb472d9d700 1 --2- 172.21.15.31:0/2862022049 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fb460066f30 0x7fb4600693e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb464006fd0 tx=0x7fb464008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:19.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.481+0000 7fb487a0eb80 1 Processor -- start 2023-10-06T02:24:19.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.481+0000 7fb487a0eb80 1 -- start start 2023-10-06T02:24:19.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.482+0000 7fb487a0eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e3ab085e50 0x55e3ab060cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:19.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.482+0000 7fb487a0eb80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e3ab0662c0 0x55e3ab061200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:19.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.482+0000 7fb487a0eb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e3ab067a10 0x55e3ab061740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:19.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.482+0000 7fb487a0eb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3aafbba70 con 0x55e3ab067a10 2023-10-06T02:24:19.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.482+0000 7fb487a0eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3aafb9270 con 0x55e3ab085e50 2023-10-06T02:24:19.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.482+0000 7fb487a0eb80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e3aafb5060 con 0x55e3ab0662c0 2023-10-06T02:24:19.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.482+0000 7fb47359e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e3ab067a10 0x55e3ab061740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:19.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.483+0000 7fb47359e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e3ab067a10 0x55e3ab061740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46660/0 (socket says 172.21.15.31:46660) 2023-10-06T02:24:19.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.483+0000 7fb47359e700 1 -- 172.21.15.31:0/2089774029 learned_addr learned my addr 172.21.15.31:0/2089774029 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:19.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.483+0000 7fb472d9d700 1 --2- 172.21.15.31:0/2089774029 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e3ab085e50 0x55e3ab060cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:19.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.483+0000 7fb47259c700 1 --2- 172.21.15.31:0/2089774029 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e3ab0662c0 0x55e3ab061200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:19.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.483+0000 7fb47359e700 1 -- 172.21.15.31:0/2089774029 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e3ab0662c0 msgr2=0x55e3ab061200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:19.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.483+0000 7fb47359e700 1 --2- 172.21.15.31:0/2089774029 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e3ab0662c0 0x55e3ab061200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:19.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.483+0000 7fb47359e700 1 -- 172.21.15.31:0/2089774029 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e3ab085e50 msgr2=0x55e3ab060cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:19.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.483+0000 7fb47359e700 1 --2- 172.21.15.31:0/2089774029 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e3ab085e50 0x55e3ab060cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:19.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.483+0000 7fb47359e700 1 -- 172.21.15.31:0/2089774029 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e3aae4f270 con 0x55e3ab067a10 2023-10-06T02:24:19.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.483+0000 7fb472d9d700 1 --2- 172.21.15.31:0/2089774029 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e3ab085e50 0x55e3ab060cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:24:19.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.483+0000 7fb47359e700 1 --2- 172.21.15.31:0/2089774029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e3ab067a10 0x55e3ab061740 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb46c00c040 tx=0x7fb46c0831b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b259e8a3900bade5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:19.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.483+0000 7fb453fff700 1 -- 172.21.15.31:0/2089774029 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb46c058eb0 con 0x55e3ab067a10 2023-10-06T02:24:19.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.483+0000 7fb453fff700 1 -- 172.21.15.31:0/2089774029 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb46c05cc00 con 0x55e3ab067a10 2023-10-06T02:24:19.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.483+0000 7fb453fff700 1 -- 172.21.15.31:0/2089774029 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb46c054000 con 0x55e3ab067a10 2023-10-06T02:24:19.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.484+0000 7fb487a0eb80 1 -- 172.21.15.31:0/2089774029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e3ab067a10 msgr2=0x55e3ab061740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:19.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.484+0000 7fb487a0eb80 1 --2- 172.21.15.31:0/2089774029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e3ab067a10 0x55e3ab061740 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb46c00c040 tx=0x7fb46c0831b0 comp rx=0 tx=0).stop 2023-10-06T02:24:19.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.484+0000 7fb487a0eb80 1 -- 172.21.15.31:0/2089774029 shutdown_connections 2023-10-06T02:24:19.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.484+0000 7fb487a0eb80 1 --2- 172.21.15.31:0/2089774029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e3ab067a10 0x55e3ab061740 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:19.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.484+0000 7fb487a0eb80 1 --2- 172.21.15.31:0/2089774029 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e3ab085e50 0x55e3ab060cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:19.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.484+0000 7fb487a0eb80 1 --2- 172.21.15.31:0/2089774029 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e3ab0662c0 0x55e3ab061200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:19.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.484+0000 7fb487a0eb80 1 -- 172.21.15.31:0/2089774029 >> 172.21.15.31:0/2089774029 conn(0x55e3ab065e80 msgr2=0x55e3ab0666b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:19.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.484+0000 7fb487a0eb80 1 -- 172.21.15.31:0/2089774029 shutdown_connections 2023-10-06T02:24:19.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.484+0000 7fb487a0eb80 1 -- 172.21.15.31:0/2089774029 wait complete. 2023-10-06T02:24:19.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.485+0000 7fb487a0eb80 1 -- start start 2023-10-06T02:24:19.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.485+0000 7fb487a0eb80 1 Processor -- start 2023-10-06T02:24:19.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.486+0000 7fb487a0eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e3ab0662c0 0x55e3ab05b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:19.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.486+0000 7fb487a0eb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e3ab067a10 0x55e3ab0bb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:19.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.486+0000 7fb487a0eb80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e3ab085e50 0x55e3ab08d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:19.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.486+0000 7fb487a0eb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3aaf60890 con 0x55e3ab067a10 2023-10-06T02:24:19.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.486+0000 7fb487a0eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3aaf7bea0 con 0x55e3ab0662c0 2023-10-06T02:24:19.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.486+0000 7fb487a0eb80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e3aae37e30 con 0x55e3ab085e50 2023-10-06T02:24:19.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.486+0000 7fb487a0eb80 10 client.0 _ll_register_callbacks cb 0x55e3ab028020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T02:24:19.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.486+0000 7fb47259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e3ab067a10 0x55e3ab0bb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:19.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.486+0000 7fb47259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e3ab067a10 0x55e3ab0bb8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46664/0 (socket says 172.21.15.31:46664) 2023-10-06T02:24:19.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.486+0000 7fb47259c700 1 -- 172.21.15.31:0/1973286657 learned_addr learned my addr 172.21.15.31:0/1973286657 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:19.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.486+0000 7fb472d9d700 1 --2- 172.21.15.31:0/1973286657 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e3ab0662c0 0x55e3ab05b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:19.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.486+0000 7fb47359e700 1 --2- 172.21.15.31:0/1973286657 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e3ab085e50 0x55e3ab08d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:19.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.487+0000 7fb47259c700 1 -- 172.21.15.31:0/1973286657 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e3ab085e50 msgr2=0x55e3ab08d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:19.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.487+0000 7fb47259c700 1 --2- 172.21.15.31:0/1973286657 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e3ab085e50 0x55e3ab08d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:19.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.487+0000 7fb47259c700 1 -- 172.21.15.31:0/1973286657 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e3ab0662c0 msgr2=0x55e3ab05b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:19.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.487+0000 7fb47259c700 1 --2- 172.21.15.31:0/1973286657 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e3ab0662c0 0x55e3ab05b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:19.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.487+0000 7fb47259c700 1 -- 172.21.15.31:0/1973286657 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4740ca7f0 con 0x55e3ab067a10 2023-10-06T02:24:19.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.487+0000 7fb472d9d700 1 --2- 172.21.15.31:0/1973286657 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e3ab0662c0 0x55e3ab05b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:24:19.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.487+0000 7fb47259c700 1 --2- 172.21.15.31:0/1973286657 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e3ab067a10 0x55e3ab0bb8a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fb468006fd0 tx=0x7fb468017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2234056e01d1822e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:19.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.487+0000 7fb4517fa700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T02:24:19.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.487+0000 7fb4517fa700 1 -- 172.21.15.31:0/1973286657 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb468020070 con 0x55e3ab067a10 2023-10-06T02:24:19.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.487+0000 7fb487a0eb80 1 -- 172.21.15.31:0/1973286657 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e3ab059940 con 0x55e3ab067a10 2023-10-06T02:24:19.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.487+0000 7fb4517fa700 1 -- 172.21.15.31:0/1973286657 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(36..36 src has 1..36) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb468011630 con 0x55e3ab067a10 2023-10-06T02:24:19.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.488+0000 7fb4517fa700 1 -- 172.21.15.31:0/1973286657 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb46800be90 con 0x55e3ab067a10 2023-10-06T02:24:19.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.488+0000 7fb487a0eb80 10 client.5202 fetch_fsmap learned FSMap version 17 2023-10-06T02:24:19.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.488+0000 7fb487a0eb80 1 -- 172.21.15.31:0/1973286657 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x55e3aadf1fe0 con 0x55e3ab067a10 2023-10-06T02:24:19.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.488+0000 7fb4517fa700 1 -- 172.21.15.31:0/1973286657 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e3ab059940 con 0x55e3ab067a10 2023-10-06T02:24:19.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.488+0000 7fb4517fa700 1 -- 172.21.15.31:0/1973286657 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 17) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fb468011a00 con 0x55e3ab067a10 2023-10-06T02:24:19.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.488+0000 7fb487a0eb80 10 client.5202 fetch_fsmap finished waiting for FSMap version 17 2023-10-06T02:24:19.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.488+0000 7fb487a0eb80 10 client.5202 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T02:24:19.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.488+0000 7fb487a0eb80 20 client.5202 populate_metadata read hostname 'smithi031' 2023-10-06T02:24:19.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.488+0000 7fb487a0eb80 1 --2- 172.21.15.31:0/1973286657 >> [v2:172.21.15.31:6835/2608294748,v1:172.21.15.31:6837/2608294748] conn(0x55e3ab0bc3b0 0x55e3ab0bc780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:19.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.488+0000 7fb487a0eb80 4 client.5202 mds_command: new command op to 5043 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T02:24:19.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.488+0000 7fb487a0eb80 1 -- 172.21.15.31:0/1973286657 --> [v2:172.21.15.31:6835/2608294748,v1:172.21.15.31:6837/2608294748] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e3ab036870 con 0x55e3ab0bc3b0 2023-10-06T02:24:19.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.488+0000 7fb472d9d700 1 --2- 172.21.15.31:0/1973286657 >> [v2:172.21.15.31:6835/2608294748,v1:172.21.15.31:6837/2608294748] conn(0x55e3ab0bc3b0 0x55e3ab0bc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:19.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.489+0000 7fb472d9d700 1 --2- 172.21.15.31:0/1973286657 >> [v2:172.21.15.31:6835/2608294748,v1:172.21.15.31:6837/2608294748] conn(0x55e3ab0bc3b0 0x55e3ab0bc780 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7bf3a2aa80ac0c2c server_cookie=865c384b87793db0 in_seq=0 out_seq=0 2023-10-06T02:24:19.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.489+0000 7fb4517fa700 10 client.5202 ms_handle_connect on v2:172.21.15.31:6835/2608294748 2023-10-06T02:24:19.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.489+0000 7fb4517fa700 1 -- 172.21.15.31:0/1973286657 <== mds.0 v2:172.21.15.31:6835/2608294748 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55e3ab036870 con 0x55e3ab0bc3b0 2023-10-06T02:24:19.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.489+0000 7fb4517fa700 10 client.5202 handle_command_reply: tid=0 2023-10-06T02:24:19.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.489+0000 7fb487a0eb80 1 client.5202 shutdown 2023-10-06T02:24:19.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.490+0000 7fb487a0eb80 1 -- 172.21.15.31:0/1973286657 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e3ab067a10 msgr2=0x55e3ab0bb8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:19.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.490+0000 7fb487a0eb80 1 --2- 172.21.15.31:0/1973286657 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e3ab067a10 0x55e3ab0bb8a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fb468006fd0 tx=0x7fb468017000 comp rx=0 tx=0).stop 2023-10-06T02:24:19.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.490+0000 7fb487a0eb80 1 -- 172.21.15.31:0/1973286657 shutdown_connections 2023-10-06T02:24:19.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.490+0000 7fb487a0eb80 1 --2- 172.21.15.31:0/1973286657 >> [v2:172.21.15.31:6835/2608294748,v1:172.21.15.31:6837/2608294748] conn(0x55e3ab0bc3b0 0x55e3ab0bc780 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:19.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.490+0000 7fb487a0eb80 1 --2- 172.21.15.31:0/1973286657 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e3ab085e50 0x55e3ab08d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:19.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.490+0000 7fb487a0eb80 1 --2- 172.21.15.31:0/1973286657 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e3ab0662c0 0x55e3ab05b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:19.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.490+0000 7fb487a0eb80 1 --2- 172.21.15.31:0/1973286657 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e3ab067a10 0x55e3ab0bb8a0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:19.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.490+0000 7fb4517fa700 0 client.5202 ms_handle_reset on v2:172.21.15.31:6835/2608294748 2023-10-06T02:24:19.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.490+0000 7fb487a0eb80 1 -- 172.21.15.31:0/1973286657 >> 172.21.15.31:0/1973286657 conn(0x55e3ab065e80 msgr2=0x55e3ab064db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:19.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.490+0000 7fb487a0eb80 1 -- 172.21.15.31:0/1973286657 shutdown_connections 2023-10-06T02:24:19.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.490+0000 7fb487a0eb80 1 -- 172.21.15.31:0/1973286657 wait complete. 2023-10-06T02:24:19.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.490+0000 7fb487a0eb80 20 client.5202 trim_cache size 0 max 16384 2023-10-06T02:24:19.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.507+0000 7fb487a0eb80 1 Processor -- start 2023-10-06T02:24:19.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.508+0000 7fb487a0eb80 1 -- start start 2023-10-06T02:24:19.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.508+0000 7fb487a0eb80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e3ab0662c0 0x55e3ab0ba3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:19.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.508+0000 7fb487a0eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e3ab067a10 0x55e3ab0863f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:19.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.508+0000 7fb487a0eb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e3ab085e50 0x55e3ab061ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:19.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.508+0000 7fb487a0eb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3ab05c0b0 con 0x55e3ab085e50 2023-10-06T02:24:19.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.508+0000 7fb487a0eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3ab0873d0 con 0x55e3ab067a10 2023-10-06T02:24:19.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.508+0000 7fb487a0eb80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e3ab05d4e0 con 0x55e3ab0662c0 2023-10-06T02:24:19.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.509+0000 7fb47359e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e3ab085e50 0x55e3ab061ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:19.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.509+0000 7fb47359e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e3ab085e50 0x55e3ab061ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46670/0 (socket says 172.21.15.31:46670) 2023-10-06T02:24:19.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.509+0000 7fb47359e700 1 -- 172.21.15.31:0/2005720692 learned_addr learned my addr 172.21.15.31:0/2005720692 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:19.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.509+0000 7fb472d9d700 1 --2- 172.21.15.31:0/2005720692 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e3ab0662c0 0x55e3ab0ba3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:19.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.509+0000 7fb47259c700 1 --2- 172.21.15.31:0/2005720692 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e3ab067a10 0x55e3ab0863f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:19.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.509+0000 7fb47359e700 1 -- 172.21.15.31:0/2005720692 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e3ab0662c0 msgr2=0x55e3ab0ba3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:19.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.509+0000 7fb47359e700 1 --2- 172.21.15.31:0/2005720692 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e3ab0662c0 0x55e3ab0ba3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:19.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.509+0000 7fb47359e700 1 -- 172.21.15.31:0/2005720692 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e3ab067a10 msgr2=0x55e3ab0863f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:19.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.509+0000 7fb47359e700 1 --2- 172.21.15.31:0/2005720692 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e3ab067a10 0x55e3ab0863f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:19.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.509+0000 7fb47359e700 1 -- 172.21.15.31:0/2005720692 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e3aaf83780 con 0x55e3ab085e50 2023-10-06T02:24:19.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.509+0000 7fb47259c700 1 --2- 172.21.15.31:0/2005720692 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e3ab067a10 0x55e3ab0863f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:24:19.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.509+0000 7fb47359e700 1 --2- 172.21.15.31:0/2005720692 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e3ab085e50 0x55e3ab061ca0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fb46c055e20 tx=0x7fb46c06e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba4bc4c31f99b74c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:19.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.510+0000 7fb458ff9700 1 -- 172.21.15.31:0/2005720692 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb46c058eb0 con 0x55e3ab085e50 2023-10-06T02:24:19.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.510+0000 7fb458ff9700 1 -- 172.21.15.31:0/2005720692 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb46c00eb40 con 0x55e3ab085e50 2023-10-06T02:24:19.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.510+0000 7fb458ff9700 1 -- 172.21.15.31:0/2005720692 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb46c054000 con 0x55e3ab085e50 2023-10-06T02:24:19.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.510+0000 7fb487a0eb80 1 -- 172.21.15.31:0/2005720692 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e3ab085e50 msgr2=0x55e3ab061ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:19.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.510+0000 7fb487a0eb80 1 --2- 172.21.15.31:0/2005720692 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e3ab085e50 0x55e3ab061ca0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fb46c055e20 tx=0x7fb46c06e000 comp rx=0 tx=0).stop 2023-10-06T02:24:19.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.510+0000 7fb487a0eb80 1 -- 172.21.15.31:0/2005720692 shutdown_connections 2023-10-06T02:24:19.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.510+0000 7fb487a0eb80 1 --2- 172.21.15.31:0/2005720692 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e3ab085e50 0x55e3ab061ca0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:19.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.510+0000 7fb487a0eb80 1 --2- 172.21.15.31:0/2005720692 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e3ab0662c0 0x55e3ab0ba3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:19.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.510+0000 7fb487a0eb80 1 --2- 172.21.15.31:0/2005720692 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e3ab067a10 0x55e3ab0863f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:19.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.510+0000 7fb487a0eb80 1 -- 172.21.15.31:0/2005720692 >> 172.21.15.31:0/2005720692 conn(0x55e3ab065e80 msgr2=0x55e3ab08d210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:19.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.510+0000 7fb487a0eb80 1 -- 172.21.15.31:0/2005720692 shutdown_connections 2023-10-06T02:24:19.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.510+0000 7fb487a0eb80 1 -- 172.21.15.31:0/2005720692 wait complete. 2023-10-06T02:24:19.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.511+0000 7fb487a0eb80 1 -- start start 2023-10-06T02:24:19.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.511+0000 7fb487a0eb80 1 Processor -- start 2023-10-06T02:24:19.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.512+0000 7fb487a0eb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e3ab0662c0 0x55e3ab0bc780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:19.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.512+0000 7fb487a0eb80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e3ab067a10 0x55e3ab110100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:19.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.512+0000 7fb487a0eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e3ab085e50 0x55e3ab061c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:19.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.512+0000 7fb487a0eb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3ab0acef0 con 0x55e3ab0662c0 2023-10-06T02:24:19.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.512+0000 7fb487a0eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3ab08af30 con 0x55e3ab085e50 2023-10-06T02:24:19.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.512+0000 7fb487a0eb80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e3ab0ac9c0 con 0x55e3ab067a10 2023-10-06T02:24:19.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.512+0000 7fb487a0eb80 10 client.0 _ll_register_callbacks cb 0x55e3ab028020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T02:24:19.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.512+0000 7fb472d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e3ab0662c0 0x55e3ab0bc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:19.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.512+0000 7fb472d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e3ab0662c0 0x55e3ab0bc780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46678/0 (socket says 172.21.15.31:46678) 2023-10-06T02:24:19.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.512+0000 7fb472d9d700 1 -- 172.21.15.31:0/3007006380 learned_addr learned my addr 172.21.15.31:0/3007006380 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:19.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.512+0000 7fb47259c700 1 --2- 172.21.15.31:0/3007006380 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e3ab067a10 0x55e3ab110100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:19.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.512+0000 7fb47359e700 1 --2- 172.21.15.31:0/3007006380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e3ab085e50 0x55e3ab061c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:19.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.513+0000 7fb472d9d700 1 -- 172.21.15.31:0/3007006380 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e3ab067a10 msgr2=0x55e3ab110100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:19.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.513+0000 7fb472d9d700 1 --2- 172.21.15.31:0/3007006380 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e3ab067a10 0x55e3ab110100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:19.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.513+0000 7fb472d9d700 1 -- 172.21.15.31:0/3007006380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e3ab085e50 msgr2=0x55e3ab061c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:19.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.513+0000 7fb472d9d700 1 --2- 172.21.15.31:0/3007006380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e3ab085e50 0x55e3ab061c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:19.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.513+0000 7fb472d9d700 1 -- 172.21.15.31:0/3007006380 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e3aae4f270 con 0x55e3ab0662c0 2023-10-06T02:24:19.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.513+0000 7fb47259c700 1 --2- 172.21.15.31:0/3007006380 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e3ab067a10 0x55e3ab110100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:24:19.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.513+0000 7fb472d9d700 1 --2- 172.21.15.31:0/3007006380 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e3ab0662c0 0x55e3ab0bc780 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb46400e450 tx=0x7fb464027040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc860447b2f13572 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:19.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.513+0000 7fb4517fa700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T02:24:19.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.513+0000 7fb4517fa700 1 -- 172.21.15.31:0/3007006380 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb46402a040 con 0x55e3ab0662c0 2023-10-06T02:24:19.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.513+0000 7fb4517fa700 1 -- 172.21.15.31:0/3007006380 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(36..36 src has 1..36) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb464025910 con 0x55e3ab0662c0 2023-10-06T02:24:19.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.513+0000 7fb487a0eb80 1 -- 172.21.15.31:0/3007006380 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb46c05cc00 con 0x55e3ab0662c0 2023-10-06T02:24:19.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.513+0000 7fb4517fa700 1 -- 172.21.15.31:0/3007006380 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb464025ce0 con 0x55e3ab0662c0 2023-10-06T02:24:19.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.514+0000 7fb487a0eb80 10 client.5208 fetch_fsmap learned FSMap version 17 2023-10-06T02:24:19.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.514+0000 7fb487a0eb80 1 -- 172.21.15.31:0/3007006380 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x55e3aaf6a470 con 0x55e3ab0662c0 2023-10-06T02:24:19.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.514+0000 7fb4517fa700 1 -- 172.21.15.31:0/3007006380 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb46c05cc00 con 0x55e3ab0662c0 2023-10-06T02:24:19.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.514+0000 7fb4517fa700 1 -- 172.21.15.31:0/3007006380 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 17) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fb46402f430 con 0x55e3ab0662c0 2023-10-06T02:24:19.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.514+0000 7fb487a0eb80 10 client.5208 fetch_fsmap finished waiting for FSMap version 17 2023-10-06T02:24:19.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.514+0000 7fb487a0eb80 10 client.5208 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T02:24:19.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.514+0000 7fb487a0eb80 20 client.5208 populate_metadata read hostname 'smithi031' 2023-10-06T02:24:19.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.514+0000 7fb487a0eb80 1 --2- 172.21.15.31:0/3007006380 >> [v2:172.21.15.31:6835/2608294748,v1:172.21.15.31:6837/2608294748] conn(0x55e3ab08a010 0x55e3ab08a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:19.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.514+0000 7fb487a0eb80 4 client.5208 mds_command: new command op to 5043 tid=0[{"prefix": "status"}] 2023-10-06T02:24:19.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.514+0000 7fb487a0eb80 1 -- 172.21.15.31:0/3007006380 --> [v2:172.21.15.31:6835/2608294748,v1:172.21.15.31:6837/2608294748] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e3aaf57e00 con 0x55e3ab08a010 2023-10-06T02:24:19.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.514+0000 7fb47259c700 1 --2- 172.21.15.31:0/3007006380 >> [v2:172.21.15.31:6835/2608294748,v1:172.21.15.31:6837/2608294748] conn(0x55e3ab08a010 0x55e3ab08a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:19.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.515+0000 7fb47259c700 1 --2- 172.21.15.31:0/3007006380 >> [v2:172.21.15.31:6835/2608294748,v1:172.21.15.31:6837/2608294748] conn(0x55e3ab08a010 0x55e3ab08a3e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6c5072f07a3d8d7 server_cookie=71c93ef2df644834 in_seq=0 out_seq=0 2023-10-06T02:24:19.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.515+0000 7fb4517fa700 10 client.5208 ms_handle_connect on v2:172.21.15.31:6835/2608294748 2023-10-06T02:24:19.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.515+0000 7fb4517fa700 1 -- 172.21.15.31:0/3007006380 <== mds.0 v2:172.21.15.31:6835/2608294748 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55e3aaf57e00 con 0x55e3ab08a010 2023-10-06T02:24:19.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.515+0000 7fb4517fa700 10 client.5208 handle_command_reply: tid=0 2023-10-06T02:24:19.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.515+0000 7fb487a0eb80 1 client.5208 shutdown 2023-10-06T02:24:19.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.515+0000 7fb487a0eb80 1 -- 172.21.15.31:0/3007006380 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e3ab0662c0 msgr2=0x55e3ab0bc780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:19.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.515+0000 7fb487a0eb80 1 --2- 172.21.15.31:0/3007006380 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e3ab0662c0 0x55e3ab0bc780 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb46400e450 tx=0x7fb464027040 comp rx=0 tx=0).stop 2023-10-06T02:24:19.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.515+0000 7fb487a0eb80 1 -- 172.21.15.31:0/3007006380 shutdown_connections 2023-10-06T02:24:19.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.515+0000 7fb487a0eb80 1 --2- 172.21.15.31:0/3007006380 >> [v2:172.21.15.31:6835/2608294748,v1:172.21.15.31:6837/2608294748] conn(0x55e3ab08a010 0x55e3ab08a3e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:19.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.515+0000 7fb487a0eb80 1 --2- 172.21.15.31:0/3007006380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e3ab085e50 0x55e3ab061c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:19.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.515+0000 7fb487a0eb80 1 --2- 172.21.15.31:0/3007006380 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e3ab0662c0 0x55e3ab0bc780 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:19.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.515+0000 7fb487a0eb80 1 --2- 172.21.15.31:0/3007006380 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e3ab067a10 0x55e3ab110100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:19.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.515+0000 7fb4517fa700 0 client.5208 ms_handle_reset on v2:172.21.15.31:6835/2608294748 2023-10-06T02:24:19.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.515+0000 7fb487a0eb80 1 -- 172.21.15.31:0/3007006380 >> 172.21.15.31:0/3007006380 conn(0x55e3ab065e80 msgr2=0x55e3ab0afaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:19.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.515+0000 7fb487a0eb80 1 -- 172.21.15.31:0/3007006380 shutdown_connections 2023-10-06T02:24:19.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.515+0000 7fb487a0eb80 1 -- 172.21.15.31:0/3007006380 wait complete. 2023-10-06T02:24:19.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.515+0000 7fb487a0eb80 20 client.5208 trim_cache size 0 max 16384 2023-10-06T02:24:19.524 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T02:24:19.524 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "d6fcf512-874b-4259-b90a-30ae070f00a9", 2023-10-06T02:24:19.525 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 0, 2023-10-06T02:24:19.525 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 5043, 2023-10-06T02:24:19.525 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2023-10-06T02:24:19.525 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2023-10-06T02:24:19.525 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2023-10-06T02:24:19.525 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 6.5215671820000001, 2023-10-06T02:24:19.525 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 17, 2023-10-06T02:24:19.525 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 36, 2023-10-06T02:24:19.525 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 36, 2023-10-06T02:24:19.525 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 16.015514477 2023-10-06T02:24:19.526 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T02:24:19.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.516+0000 7fb4789fe700 1 -- 172.21.15.31:0/2862022049 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fb460066f30 msgr2=0x7fb4600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:19.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.516+0000 7fb4789fe700 1 --2- 172.21.15.31:0/2862022049 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fb460066f30 0x7fb4600693e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb464006fd0 tx=0x7fb464008040 comp rx=0 tx=0).stop 2023-10-06T02:24:19.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.516+0000 7fb4789fe700 1 -- 172.21.15.31:0/2862022049 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb47415dfd0 msgr2=0x7fb4740cb8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:19.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.516+0000 7fb4789fe700 1 --2- 172.21.15.31:0/2862022049 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb47415dfd0 0x7fb4740cb8e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb46c009fd0 tx=0x7fb46c009900 comp rx=0 tx=0).stop 2023-10-06T02:24:19.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.516+0000 7fb4789fe700 1 -- 172.21.15.31:0/2862022049 shutdown_connections 2023-10-06T02:24:19.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.516+0000 7fb4789fe700 1 --2- 172.21.15.31:0/2862022049 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb47415dfd0 0x7fb4740cb8e0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:19.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.516+0000 7fb4789fe700 1 --2- 172.21.15.31:0/2862022049 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb474159550 0x7fb4740c7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:19.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.516+0000 7fb4789fe700 1 --2- 172.21.15.31:0/2862022049 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fb460066f30 0x7fb4600693e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:19.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.516+0000 7fb4789fe700 1 --2- 172.21.15.31:0/2862022049 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb474159920 0x7fb4740c7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:19.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.516+0000 7fb4789fe700 1 -- 172.21.15.31:0/2862022049 >> 172.21.15.31:0/2862022049 conn(0x7fb47406ce80 msgr2=0x7fb47406ef90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:19.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.516+0000 7fb4789fe700 1 -- 172.21.15.31:0/2862022049 shutdown_connections 2023-10-06T02:24:19.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.516+0000 7fb4789fe700 1 -- 172.21.15.31:0/2862022049 wait complete. 2023-10-06T02:24:19.534 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T02:24:19.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.720+0000 7f991ffff700 1 Processor -- start 2023-10-06T02:24:19.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.720+0000 7f991ffff700 1 -- start start 2023-10-06T02:24:19.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.722+0000 7f991ffff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f992013fb20 0x7f992015a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:19.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.722+0000 7f991ffff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f992013fef0 0x7f99200c7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:19.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.722+0000 7f991ffff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f992006aaa0 0x7f99200c7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:19.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.722+0000 7f991ffff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99200c8410 con 0x7f992013fef0 2023-10-06T02:24:19.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.722+0000 7f991ffff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f992015ae70 con 0x7f992006aaa0 2023-10-06T02:24:19.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.722+0000 7f991ffff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f992006af60 con 0x7f992013fb20 2023-10-06T02:24:19.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.722+0000 7f991e7fc700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f992013fef0 0x7f99200c7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:19.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.722+0000 7f991effd700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f992013fb20 0x7f992015a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:19.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.722+0000 7f991e7fc700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f992013fef0 0x7f99200c7990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46690/0 (socket says 172.21.15.31:46690) 2023-10-06T02:24:19.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.722+0000 7f991e7fc700 1 -- 172.21.15.31:0/3001211271 learned_addr learned my addr 172.21.15.31:0/3001211271 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:19.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.722+0000 7f991f7fe700 1 --2- 172.21.15.31:0/3001211271 >> v2:172.21.15.62:3300/0 conn(0x7f992006aaa0 0x7f99200c7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:19.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.722+0000 7f991e7fc700 1 -- 172.21.15.31:0/3001211271 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f992013fb20 msgr2=0x7f992015a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:19.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.722+0000 7f991e7fc700 1 --2- 172.21.15.31:0/3001211271 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f992013fb20 0x7f992015a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:19.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.722+0000 7f991e7fc700 1 -- 172.21.15.31:0/3001211271 >> v2:172.21.15.62:3300/0 conn(0x7f992006aaa0 msgr2=0x7f99200c7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:19.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.722+0000 7f991e7fc700 1 --2- 172.21.15.31:0/3001211271 >> v2:172.21.15.62:3300/0 conn(0x7f992006aaa0 0x7f99200c7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:19.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.723+0000 7f991e7fc700 1 -- 172.21.15.31:0/3001211271 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f992006b240 con 0x7f992013fef0 2023-10-06T02:24:19.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.723+0000 7f991e7fc700 1 --2- 172.21.15.31:0/3001211271 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f992013fef0 0x7f99200c7990 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9920156920 tx=0x7f991400b4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d888925e7a77d86 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:19.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.723+0000 7f991dffb700 1 -- 172.21.15.31:0/3001211271 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9914005c10 con 0x7f992013fef0 2023-10-06T02:24:19.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.723+0000 7f991dffb700 1 -- 172.21.15.31:0/3001211271 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9914005d70 con 0x7f992013fef0 2023-10-06T02:24:19.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.723+0000 7f991dffb700 1 -- 172.21.15.31:0/3001211271 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f991400db60 con 0x7f992013fef0 2023-10-06T02:24:19.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.724+0000 7f991ffff700 1 -- 172.21.15.31:0/3001211271 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f992013fef0 msgr2=0x7f99200c7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:19.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.724+0000 7f991ffff700 1 --2- 172.21.15.31:0/3001211271 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f992013fef0 0x7f99200c7990 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9920156920 tx=0x7f991400b4c0 comp rx=0 tx=0).stop 2023-10-06T02:24:19.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.724+0000 7f991ffff700 1 -- 172.21.15.31:0/3001211271 shutdown_connections 2023-10-06T02:24:19.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.724+0000 7f991ffff700 1 --2- 172.21.15.31:0/3001211271 >> v2:172.21.15.62:3300/0 conn(0x7f992006aaa0 0x7f99200c7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:19.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.724+0000 7f991ffff700 1 --2- 172.21.15.31:0/3001211271 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f992013fb20 0x7f992015a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:19.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.724+0000 7f991ffff700 1 --2- 172.21.15.31:0/3001211271 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f992013fef0 0x7f99200c7990 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:19.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.724+0000 7f991ffff700 1 -- 172.21.15.31:0/3001211271 >> 172.21.15.31:0/3001211271 conn(0x7f992006d170 msgr2=0x7f99200cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:19.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.724+0000 7f991ffff700 1 -- 172.21.15.31:0/3001211271 shutdown_connections 2023-10-06T02:24:19.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.724+0000 7f991ffff700 1 -- 172.21.15.31:0/3001211271 wait complete. 2023-10-06T02:24:19.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.725+0000 7f991ffff700 1 Processor -- start 2023-10-06T02:24:19.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.725+0000 7f991ffff700 1 -- start start 2023-10-06T02:24:19.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.726+0000 7f991ffff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f992006aaa0 0x7f9920156950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:19.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.726+0000 7f991ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f992013fb20 0x7f99200ac5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:19.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.726+0000 7f991ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f992013fef0 0x7f99200ad690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:19.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.726+0000 7f991ffff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99200acae0 con 0x7f992013fef0 2023-10-06T02:24:19.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.726+0000 7f991ffff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9920157c40 con 0x7f992013fb20 2023-10-06T02:24:19.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.726+0000 7f991ffff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9920157030 con 0x7f992006aaa0 2023-10-06T02:24:19.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.727+0000 7f991f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f992013fef0 0x7f99200ad690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:19.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.727+0000 7f991f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f992013fef0 0x7f99200ad690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46696/0 (socket says 172.21.15.31:46696) 2023-10-06T02:24:19.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.727+0000 7f991f7fe700 1 -- 172.21.15.31:0/140102584 learned_addr learned my addr 172.21.15.31:0/140102584 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:19.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.727+0000 7f991effd700 1 --2- 172.21.15.31:0/140102584 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f992006aaa0 0x7f9920156950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:19.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.727+0000 7f991e7fc700 1 --2- 172.21.15.31:0/140102584 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f992013fb20 0x7f99200ac5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:19.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.727+0000 7f991f7fe700 1 -- 172.21.15.31:0/140102584 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f992006aaa0 msgr2=0x7f9920156950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:19.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.727+0000 7f991f7fe700 1 --2- 172.21.15.31:0/140102584 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f992006aaa0 0x7f9920156950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:19.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.727+0000 7f991f7fe700 1 -- 172.21.15.31:0/140102584 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f992013fb20 msgr2=0x7f99200ac5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:19.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.727+0000 7f991f7fe700 1 --2- 172.21.15.31:0/140102584 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f992013fb20 0x7f99200ac5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:19.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.727+0000 7f991f7fe700 1 -- 172.21.15.31:0/140102584 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f991400b720 con 0x7f992013fef0 2023-10-06T02:24:19.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.727+0000 7f991e7fc700 1 --2- 172.21.15.31:0/140102584 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f992013fb20 0x7f99200ac5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:24:19.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.727+0000 7f991f7fe700 1 --2- 172.21.15.31:0/140102584 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f992013fef0 0x7f99200ad690 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f9918009ef0 tx=0x7f991800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:19.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.727+0000 7f9906ffd700 1 -- 172.21.15.31:0/140102584 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f991800cc20 con 0x7f992013fef0 2023-10-06T02:24:19.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.728+0000 7f9906ffd700 1 -- 172.21.15.31:0/140102584 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f991800f040 con 0x7f992013fef0 2023-10-06T02:24:19.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.728+0000 7f991ffff700 1 -- 172.21.15.31:0/140102584 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9920157310 con 0x7f992013fef0 2023-10-06T02:24:19.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.728+0000 7f9906ffd700 1 -- 172.21.15.31:0/140102584 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9918014480 con 0x7f992013fef0 2023-10-06T02:24:19.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.728+0000 7f991ffff700 1 -- 172.21.15.31:0/140102584 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99201578d0 con 0x7f992013fef0 2023-10-06T02:24:19.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.729+0000 7f991ffff700 1 -- 172.21.15.31:0/140102584 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9920157db0 con 0x7f992013fef0 2023-10-06T02:24:19.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.730+0000 7f9906ffd700 1 -- 172.21.15.31:0/140102584 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f99180147f0 con 0x7f992013fef0 2023-10-06T02:24:19.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.730+0000 7f9906ffd700 1 --2- 172.21.15.31:0/140102584 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f990c066f30 0x7f990c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:19.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.731+0000 7f9906ffd700 1 -- 172.21.15.31:0/140102584 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9918085460 con 0x7f992013fef0 2023-10-06T02:24:19.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.731+0000 7f991effd700 1 --2- 172.21.15.31:0/140102584 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f990c066f30 0x7f990c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:19.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.731+0000 7f991effd700 1 --2- 172.21.15.31:0/140102584 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f990c066f30 0x7f990c0693e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9910009d20 tx=0x7f99100091a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:19.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.735+0000 7f9906ffd700 1 -- 172.21.15.31:0/140102584 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f99180524d0 con 0x7f992013fef0 2023-10-06T02:24:19.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.981+0000 7f991ffff700 1 -- 172.21.15.31:0/140102584 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f99200648e0 con 0x7f992013fef0 2023-10-06T02:24:19.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.982+0000 7f9906ffd700 1 -- 172.21.15.31:0/140102584 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+73 (secure 0 0 0) 0x7f99200648e0 con 0x7f992013fef0 2023-10-06T02:24:19.983 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T02:24:19.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.984+0000 7f991ffff700 1 -- 172.21.15.31:0/140102584 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f990c066f30 msgr2=0x7f990c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:19.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.984+0000 7f991ffff700 1 --2- 172.21.15.31:0/140102584 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f990c066f30 0x7f990c0693e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9910009d20 tx=0x7f99100091a0 comp rx=0 tx=0).stop 2023-10-06T02:24:19.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.984+0000 7f991ffff700 1 -- 172.21.15.31:0/140102584 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f992013fef0 msgr2=0x7f99200ad690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:19.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.984+0000 7f991ffff700 1 --2- 172.21.15.31:0/140102584 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f992013fef0 0x7f99200ad690 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f9918009ef0 tx=0x7f991800b040 comp rx=0 tx=0).stop 2023-10-06T02:24:19.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.984+0000 7f991ffff700 1 -- 172.21.15.31:0/140102584 shutdown_connections 2023-10-06T02:24:19.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.984+0000 7f991ffff700 1 --2- 172.21.15.31:0/140102584 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f992013fef0 0x7f99200ad690 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:19.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.984+0000 7f991ffff700 1 --2- 172.21.15.31:0/140102584 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f990c066f30 0x7f990c0693e0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:19.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.984+0000 7f991ffff700 1 --2- 172.21.15.31:0/140102584 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f992006aaa0 0x7f9920156950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:19.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.984+0000 7f991ffff700 1 --2- 172.21.15.31:0/140102584 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f992013fb20 0x7f99200ac5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:19.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.984+0000 7f991ffff700 1 -- 172.21.15.31:0/140102584 >> 172.21.15.31:0/140102584 conn(0x7f992006d170 msgr2=0x7f99200c9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:19.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.984+0000 7f991ffff700 1 -- 172.21.15.31:0/140102584 shutdown_connections 2023-10-06T02:24:19.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:19.984+0000 7f991ffff700 1 -- 172.21.15.31:0/140102584 wait complete. 2023-10-06T02:24:20.003 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T02:24:20.003 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T02:24:20.003 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi031.front.sepia.ceph.com 2023-10-06T02:24:20.003 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T02:24:20.003 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T02:24:20.003 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T02:24:20.003 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T02:24:20.003 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T02:24:20.003 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T02:24:20.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:24:20.004 DEBUG:teuthology.orchestra.run.smithi031:> ip addr 2023-10-06T02:24:20.041 INFO:teuthology.orchestra.run.smithi031.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T02:24:20.041 INFO:teuthology.orchestra.run.smithi031.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T02:24:20.041 INFO:teuthology.orchestra.run.smithi031.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T02:24:20.041 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:24:20.041 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 ::1/128 scope host 2023-10-06T02:24:20.042 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:24:20.042 INFO:teuthology.orchestra.run.smithi031.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T02:24:20.042 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cc brd ff:ff:ff:ff:ff:ff 2023-10-06T02:24:20.042 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f0 2023-10-06T02:24:20.042 INFO:teuthology.orchestra.run.smithi031.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T02:24:20.042 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cd brd ff:ff:ff:ff:ff:ff 2023-10-06T02:24:20.042 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f1 2023-10-06T02:24:20.042 INFO:teuthology.orchestra.run.smithi031.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T02:24:20.042 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:ba brd ff:ff:ff:ff:ff:ff 2023-10-06T02:24:20.042 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f0 2023-10-06T02:24:20.042 INFO:teuthology.orchestra.run.smithi031.stdout: inet 172.21.15.31/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T02:24:20.043 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft 42196sec preferred_lft 42196sec 2023-10-06T02:24:20.043 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::ae1f:6bff:fea5:caba/64 scope link 2023-10-06T02:24:20.043 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:24:20.043 INFO:teuthology.orchestra.run.smithi031.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T02:24:20.043 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:bb brd ff:ff:ff:ff:ff:ff 2023-10-06T02:24:20.043 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f1 2023-10-06T02:24:20.043 INFO:teuthology.orchestra.run.smithi031.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T02:24:20.043 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 16:d7:88:2d:d0:c2 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:24:20.043 INFO:teuthology.orchestra.run.smithi031.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T02:24:20.043 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:24:20.043 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::408:52ff:fe3c:ba8f/64 scope link 2023-10-06T02:24:20.043 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:24:20.044 INFO:teuthology.orchestra.run.smithi031.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T02:24:20.044 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 16:d7:88:2d:d0:c2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T02:24:20.044 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::14d7:88ff:fe2d:d0c2/64 scope link 2023-10-06T02:24:20.044 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:24:20.044 INFO:teuthology.orchestra.run.smithi031.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T02:24:20.044 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 82:16:f1:00:cf:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T02:24:20.044 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::8016:f1ff:fe00:cf2f/64 scope link 2023-10-06T02:24:20.044 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:24:20.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:24:20.045 DEBUG:teuthology.orchestra.run.smithi031:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T02:24:20.144 INFO:teuthology.orchestra.run.smithi031.stdout:1 2023-10-06T02:24:20.147 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:24:20.148 DEBUG:teuthology.orchestra.run.smithi031:> route 2023-10-06T02:24:20.186 INFO:teuthology.orchestra.run.smithi031.stdout:Kernel IP routing table 2023-10-06T02:24:20.186 INFO:teuthology.orchestra.run.smithi031.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T02:24:20.186 INFO:teuthology.orchestra.run.smithi031.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T02:24:20.186 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T02:24:20.186 INFO:teuthology.orchestra.run.smithi031.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T02:24:20.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:24:20.187 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2023-10-06T02:24:20.187 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2023-10-06T02:24:20.188 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T02:24:20.188 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T02:24:20.188 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T02:24:20.188 DEBUG:teuthology.orchestra.run.smithi031:> ') 2023-10-06T02:24:20.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:24:20.385 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2023-10-06T02:24:20.420 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T02:24:20.421 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T02:24:20.421 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T02:24:20.422 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:24:20.422 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set brx.0 up 2023-10-06T02:24:20.524 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T02:24:20.524 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T02:24:20.561 INFO:teuthology.orchestra.run.smithi031.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T02:24:20.561 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T02:24:20.561 DEBUG:teuthology.orchestra.run.smithi031:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T02:24:20.637 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-06T02:24:20.637 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T02:24:20.637 DEBUG:teuthology.orchestra.run.smithi031:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T02:24:20.768 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T02:24:20.768 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T02:24:20.797 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T02:24:20.798 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T02:24:20.798 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.62:3300,172.21.15.62:3301" 2023-10-06T02:24:20.798 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: trying mount with new device syntax: 0@d6fcf512-874b-4259-b90a-30ae070f00a9.cephfs=/ 2023-10-06T02:24:20.798 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.31:3300/172.21.15.62:3300/172.21.15.62:3301" will pass to kernel 2023-10-06T02:24:20.804 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T02:24:20.804 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:24:20.804 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T02:24:20.867 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T02:24:21.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:21.056+0000 7f0c7239d700 1 Processor -- start 2023-10-06T02:24:21.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:21.056+0000 7f0c7239d700 1 -- start start 2023-10-06T02:24:21.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:21.056+0000 7f0c7239d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0c6c159940 0x7f0c6c06d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:21.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:21.056+0000 7f0c7239d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0c6c15dea0 0x7f0c6c06dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:21.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:21.056+0000 7f0c7239d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c6c0b3270 con 0x7f0c6c159570 2023-10-06T02:24:21.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:21.056+0000 7f0c7239d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c6c0b3550 con 0x7f0c6c15dea0 2023-10-06T02:24:21.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:21.056+0000 7f0c7239d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c6c0b3830 con 0x7f0c6c159940 2023-10-06T02:24:21.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:21.057+0000 7f0c71b9c700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0c6c15dea0 0x7f0c6c06dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:21.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:21.057+0000 7f0c70b9a700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0c6c159940 0x7f0c6c06d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:21.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:21.057+0000 7f0c71b9c700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0c6c15dea0 0x7f0c6c06dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:33176/0 (socket says 172.21.15.31:33176) 2023-10-06T02:24:21.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:21.057+0000 7f0c71b9c700 1 -- 172.21.15.31:0/1823577263 learned_addr learned my addr 172.21.15.31:0/1823577263 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:21.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:21.057+0000 7f0c7139b700 1 --1- 172.21.15.31:0/1823577263 >> v1:172.21.15.62:6789/0 conn(0x7f0c6c159570 0x7f0c6c0b23e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:38484/0 (socket says 172.21.15.31:38484) 2023-10-06T02:24:21.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:21.059+0000 7f0c63fff700 1 -- 172.21.15.31:0/1823577263 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 898597353 0 0) 0x7f0c6c0b3270 con 0x7f0c6c159570 2023-10-06T02:24:21.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:21.059+0000 7f0c63fff700 1 -- 172.21.15.31:0/1823577263 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c58003570 con 0x7f0c6c159570 2023-10-06T02:24:21.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:21.059+0000 7f0c71b9c700 1 -- 172.21.15.31:0/1823577263 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0c6c159940 msgr2=0x7f0c6c06d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:21.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:21.059+0000 7f0c71b9c700 1 --2- 172.21.15.31:0/1823577263 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0c6c159940 0x7f0c6c06d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:21.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:21.059+0000 7f0c71b9c700 1 -- 172.21.15.31:0/1823577263 >> v1:172.21.15.62:6789/0 conn(0x7f0c6c159570 legacy=0x7f0c6c0b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:21.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:21.059+0000 7f0c71b9c700 1 -- 172.21.15.31:0/1823577263 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c6c0b3b10 con 0x7f0c6c15dea0 2023-10-06T02:24:21.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:21.059+0000 7f0c70b9a700 1 --2- 172.21.15.31:0/1823577263 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0c6c159940 0x7f0c6c06d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:24:21.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:21.059+0000 7f0c71b9c700 1 --2- 172.21.15.31:0/1823577263 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c6c15dea0 0x7f0c6c06dd00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0c64009f20 tx=0x7f0c64009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a424702c180d585 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:21.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:21.059+0000 7f0c63fff700 1 -- 172.21.15.31:0/1823577263 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c6400e070 con 0x7f0c6c15dea0 2023-10-06T02:24:21.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:21.059+0000 7f0c63fff700 1 -- 172.21.15.31:0/1823577263 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c6400b970 con 0x7f0c6c15dea0 2023-10-06T02:24:21.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:21.059+0000 7f0c63fff700 1 -- 172.21.15.31:0/1823577263 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c640092e0 con 0x7f0c6c15dea0 2023-10-06T02:24:21.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:21.059+0000 7f0c7239d700 1 -- 172.21.15.31:0/1823577263 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c6c15dea0 msgr2=0x7f0c6c06dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:21.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:21.059+0000 7f0c7239d700 1 --2- 172.21.15.31:0/1823577263 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c6c15dea0 0x7f0c6c06dd00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0c64009f20 tx=0x7f0c64009840 comp rx=0 tx=0).stop 2023-10-06T02:24:21.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:21.059+0000 7f0c7239d700 1 -- 172.21.15.31:0/1823577263 shutdown_connections 2023-10-06T02:24:21.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:21.059+0000 7f0c7239d700 1 --2- 172.21.15.31:0/1823577263 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c6c15dea0 0x7f0c6c06dd00 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:21.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:21.059+0000 7f0c7239d700 1 --2- 172.21.15.31:0/1823577263 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0c6c159940 0x7f0c6c06d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:21.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:21.059+0000 7f0c7239d700 1 -- 172.21.15.31:0/1823577263 >> 172.21.15.31:0/1823577263 conn(0x7f0c6c0703a0 msgr2=0x7f0c6c0707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:21.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:21.059+0000 7f0c7239d700 1 -- 172.21.15.31:0/1823577263 shutdown_connections 2023-10-06T02:24:21.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:21.059+0000 7f0c7239d700 1 -- 172.21.15.31:0/1823577263 wait complete. 2023-10-06T02:24:21.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:21.060+0000 7f0c7239d700 1 Processor -- start 2023-10-06T02:24:21.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:21.061+0000 7f0c7239d700 1 -- start start 2023-10-06T02:24:21.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:21.061+0000 7f0c7239d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c6c159570 0x7f0c6c0c4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:21.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:21.062+0000 7f0c7239d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c6c159940 0x7f0c6c157700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:21.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:21.062+0000 7f0c7239d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0c6c15dea0 0x7f0c6c0d04d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:21.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:21.062+0000 7f0c7239d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c6c0c9010 con 0x7f0c6c159940 2023-10-06T02:24:21.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:21.062+0000 7f0c7239d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c6c0c59f0 con 0x7f0c6c159570 2023-10-06T02:24:21.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:21.062+0000 7f0c7239d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c6c0c8d00 con 0x7f0c6c15dea0 2023-10-06T02:24:21.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:21.062+0000 7f0c70b9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c6c159940 0x7f0c6c157700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:21.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:21.062+0000 7f0c70b9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c6c159940 0x7f0c6c157700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46706/0 (socket says 172.21.15.31:46706) 2023-10-06T02:24:21.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:21.062+0000 7f0c70b9a700 1 -- 172.21.15.31:0/919667214 learned_addr learned my addr 172.21.15.31:0/919667214 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:21.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:21.062+0000 7f0c7139b700 1 --2- 172.21.15.31:0/919667214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c6c159570 0x7f0c6c0c4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:21.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:21.062+0000 7f0c71b9c700 1 --2- 172.21.15.31:0/919667214 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0c6c15dea0 0x7f0c6c0d04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:21.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:21.062+0000 7f0c70b9a700 1 -- 172.21.15.31:0/919667214 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0c6c15dea0 msgr2=0x7f0c6c0d04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:21.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:21.062+0000 7f0c70b9a700 1 --2- 172.21.15.31:0/919667214 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0c6c15dea0 0x7f0c6c0d04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:21.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:21.062+0000 7f0c70b9a700 1 -- 172.21.15.31:0/919667214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c6c159570 msgr2=0x7f0c6c0c4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:21.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:21.062+0000 7f0c70b9a700 1 --2- 172.21.15.31:0/919667214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c6c159570 0x7f0c6c0c4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:21.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:21.062+0000 7f0c70b9a700 1 -- 172.21.15.31:0/919667214 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c5c009420 con 0x7f0c6c159940 2023-10-06T02:24:21.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:21.063+0000 7f0c70b9a700 1 --2- 172.21.15.31:0/919667214 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c6c159940 0x7f0c6c157700 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c00ecb0 tx=0x7f0c5c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:21.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:21.063+0000 7f0c617fa700 1 -- 172.21.15.31:0/919667214 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c5c00c920 con 0x7f0c6c159940 2023-10-06T02:24:21.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:21.063+0000 7f0c617fa700 1 -- 172.21.15.31:0/919667214 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c5c00ca80 con 0x7f0c6c159940 2023-10-06T02:24:21.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:21.063+0000 7f0c7239d700 1 -- 172.21.15.31:0/919667214 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c64009ae0 con 0x7f0c6c159940 2023-10-06T02:24:21.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:21.063+0000 7f0c617fa700 1 -- 172.21.15.31:0/919667214 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c5c00ce20 con 0x7f0c6c159940 2023-10-06T02:24:21.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:21.063+0000 7f0c7239d700 1 -- 172.21.15.31:0/919667214 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c6c157cc0 con 0x7f0c6c159940 2023-10-06T02:24:21.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:21.064+0000 7f0c7239d700 1 -- 172.21.15.31:0/919667214 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c6c0c9180 con 0x7f0c6c159940 2023-10-06T02:24:21.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:21.066+0000 7f0c617fa700 1 -- 172.21.15.31:0/919667214 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0c5c013680 con 0x7f0c6c159940 2023-10-06T02:24:21.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:21.067+0000 7f0c617fa700 1 --2- 172.21.15.31:0/919667214 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f0c50066f30 0x7f0c500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:21.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:21.067+0000 7f0c617fa700 1 -- 172.21.15.31:0/919667214 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0c5c0847f0 con 0x7f0c6c159940 2023-10-06T02:24:21.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:21.067+0000 7f0c7139b700 1 --2- 172.21.15.31:0/919667214 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f0c50066f30 0x7f0c500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:21.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:21.067+0000 7f0c7139b700 1 --2- 172.21.15.31:0/919667214 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f0c50066f30 0x7f0c500693e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c0c5e10 tx=0x7f0c68007d40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:21.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:21.070+0000 7f0c617fa700 1 -- 172.21.15.31:0/919667214 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0c6c0c9180 con 0x7f0c6c159940 2023-10-06T02:24:21.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:21.319+0000 7f0c7239d700 1 -- 172.21.15.31:0/919667214 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f0c6c0c9150 con 0x7f0c6c159940 2023-10-06T02:24:21.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:21.320+0000 7f0c617fa700 1 -- 172.21.15.31:0/919667214 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+73 (secure 0 0 0) 0x7f0c5c0552f0 con 0x7f0c6c159940 2023-10-06T02:24:21.321 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T02:24:21.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:21.322+0000 7f0c7239d700 1 -- 172.21.15.31:0/919667214 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f0c50066f30 msgr2=0x7f0c500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:21.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:21.322+0000 7f0c7239d700 1 --2- 172.21.15.31:0/919667214 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f0c50066f30 0x7f0c500693e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c0c5e10 tx=0x7f0c68007d40 comp rx=0 tx=0).stop 2023-10-06T02:24:21.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:21.322+0000 7f0c7239d700 1 -- 172.21.15.31:0/919667214 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c6c159940 msgr2=0x7f0c6c157700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:21.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:21.322+0000 7f0c7239d700 1 --2- 172.21.15.31:0/919667214 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c6c159940 0x7f0c6c157700 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c00ecb0 tx=0x7f0c5c0091a0 comp rx=0 tx=0).stop 2023-10-06T02:24:21.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:21.322+0000 7f0c7239d700 1 -- 172.21.15.31:0/919667214 shutdown_connections 2023-10-06T02:24:21.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:21.322+0000 7f0c7239d700 1 --2- 172.21.15.31:0/919667214 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f0c50066f30 0x7f0c500693e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:21.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:21.322+0000 7f0c7239d700 1 --2- 172.21.15.31:0/919667214 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0c6c15dea0 0x7f0c6c0d04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:21.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:21.322+0000 7f0c7239d700 1 --2- 172.21.15.31:0/919667214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c6c159570 0x7f0c6c0c4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:21.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:21.322+0000 7f0c7239d700 1 --2- 172.21.15.31:0/919667214 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c6c159940 0x7f0c6c157700 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:21.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:21.322+0000 7f0c7239d700 1 -- 172.21.15.31:0/919667214 >> 172.21.15.31:0/919667214 conn(0x7f0c6c0703a0 msgr2=0x7f0c6c159ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:21.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:21.322+0000 7f0c7239d700 1 -- 172.21.15.31:0/919667214 shutdown_connections 2023-10-06T02:24:21.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:21.322+0000 7f0c7239d700 1 -- 172.21.15.31:0/919667214 wait complete. 2023-10-06T02:24:21.340 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T02:24:21.340 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T02:24:21.340 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi062.front.sepia.ceph.com 2023-10-06T02:24:21.340 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-06T02:24:21.341 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-06T02:24:21.341 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T02:24:21.341 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T02:24:21.341 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T02:24:21.341 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T02:24:21.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:24:21.341 DEBUG:teuthology.orchestra.run.smithi062:> ip addr 2023-10-06T02:24:21.382 INFO:teuthology.orchestra.run.smithi062.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T02:24:21.383 INFO:teuthology.orchestra.run.smithi062.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T02:24:21.383 INFO:teuthology.orchestra.run.smithi062.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T02:24:21.383 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:24:21.383 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 ::1/128 scope host 2023-10-06T02:24:21.383 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:24:21.383 INFO:teuthology.orchestra.run.smithi062.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T02:24:21.383 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:14 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:24:21.383 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f0 2023-10-06T02:24:21.383 INFO:teuthology.orchestra.run.smithi062.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T02:24:21.383 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:15 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:24:21.383 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f1 2023-10-06T02:24:21.383 INFO:teuthology.orchestra.run.smithi062.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T02:24:21.384 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c2 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:24:21.384 INFO:teuthology.orchestra.run.smithi062.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T02:24:21.384 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c3 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:24:21.384 INFO:teuthology.orchestra.run.smithi062.stdout: inet 172.21.15.62/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T02:24:21.384 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft 42188sec preferred_lft 42188sec 2023-10-06T02:24:21.384 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::ec4:7aff:febb:d1c3/64 scope link 2023-10-06T02:24:21.384 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:24:21.384 INFO:teuthology.orchestra.run.smithi062.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T02:24:21.384 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether ae:42:a8:dd:f7:9b brd ff:ff:ff:ff:ff:ff 2023-10-06T02:24:21.384 INFO:teuthology.orchestra.run.smithi062.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T02:24:21.384 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:24:21.384 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::1098:a4ff:fe0d:389/64 scope link 2023-10-06T02:24:21.384 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:24:21.385 INFO:teuthology.orchestra.run.smithi062.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T02:24:21.385 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether fe:60:2b:8e:af:28 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T02:24:21.385 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::fc60:2bff:fe8e:af28/64 scope link 2023-10-06T02:24:21.385 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:24:21.385 INFO:teuthology.orchestra.run.smithi062.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T02:24:21.385 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether ae:42:a8:dd:f7:9b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T02:24:21.385 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::ac42:a8ff:fedd:f79b/64 scope link 2023-10-06T02:24:21.385 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:24:21.386 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:24:21.386 DEBUG:teuthology.orchestra.run.smithi062:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T02:24:21.492 INFO:teuthology.orchestra.run.smithi062.stdout:1 2023-10-06T02:24:21.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:24:21.495 DEBUG:teuthology.orchestra.run.smithi062:> route 2023-10-06T02:24:21.534 INFO:teuthology.orchestra.run.smithi062.stdout:Kernel IP routing table 2023-10-06T02:24:21.534 INFO:teuthology.orchestra.run.smithi062.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T02:24:21.534 INFO:teuthology.orchestra.run.smithi062.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T02:24:21.534 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T02:24:21.535 INFO:teuthology.orchestra.run.smithi062.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T02:24:21.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:24:21.536 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2023-10-06T02:24:21.536 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2023-10-06T02:24:21.536 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T02:24:21.536 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T02:24:21.536 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T02:24:21.536 DEBUG:teuthology.orchestra.run.smithi062:> ') 2023-10-06T02:24:21.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:24:21.734 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2023-10-06T02:24:21.770 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-06T02:24:21.770 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T02:24:21.771 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T02:24:21.771 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:24:21.771 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set brx.0 up 2023-10-06T02:24:21.872 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T02:24:21.872 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-06T02:24:21.909 INFO:teuthology.orchestra.run.smithi062.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-06T02:24:21.909 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T02:24:21.909 DEBUG:teuthology.orchestra.run.smithi062:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-06T02:24:21.984 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-06T02:24:21.984 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T02:24:21.984 DEBUG:teuthology.orchestra.run.smithi062:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T02:24:22.115 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T02:24:22.115 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T02:24:22.144 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T02:24:22.144 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T02:24:22.144 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.62:3300,172.21.15.62:3301" 2023-10-06T02:24:22.144 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: trying mount with new device syntax: 1@d6fcf512-874b-4259-b90a-30ae070f00a9.cephfs=/ 2023-10-06T02:24:22.144 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.31:3300/172.21.15.62:3300/172.21.15.62:3301" will pass to kernel 2023-10-06T02:24:22.149 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T02:24:22.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:24:22.149 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-06T02:24:22.213 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-10-06T02:24:22.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:22.416+0000 7f578b59e700 1 Processor -- start 2023-10-06T02:24:22.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:22.416+0000 7f578b59e700 1 -- start start 2023-10-06T02:24:22.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:22.417+0000 7f578b59e700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f578c0c8200 0x7f578c0c1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:22.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:22.417+0000 7f578b59e700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f578c1571b0 con 0x7f578c0c85d0 2023-10-06T02:24:22.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:22.417+0000 7f578b59e700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f578c1579b0 con 0x7f578c0c2a90 2023-10-06T02:24:22.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:22.417+0000 7f578b59e700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f578c0c1390 con 0x7f578c0c8200 2023-10-06T02:24:22.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:22.418+0000 7f578ad9d700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f578c0c85d0 0x7f578c0cc2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60360/0 (socket says 172.21.15.31:60360) 2023-10-06T02:24:22.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:22.418+0000 7f578ad9d700 1 -- 172.21.15.31:0/4204483120 learned_addr learned my addr 172.21.15.31:0/4204483120 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:24:22.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:22.418+0000 7f578a59c700 1 --2- 172.21.15.31:0/4204483120 >> v2:172.21.15.62:3300/0 conn(0x7f578c0c8200 0x7f578c0c1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:22.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:22.418+0000 7f578959a700 1 -- 172.21.15.31:0/4204483120 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4149867323 0 0) 0x7f5784001490 con 0x7f578c0c85d0 2023-10-06T02:24:22.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:22.418+0000 7f578959a700 1 -- 172.21.15.31:0/4204483120 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5774003570 con 0x7f578c0c85d0 2023-10-06T02:24:22.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:22.418+0000 7f578a59c700 1 -- 172.21.15.31:0/4204483120 >> v1:172.21.15.62:6789/0 conn(0x7f578c0c2a90 legacy=0x7f578c156aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:22.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:22.418+0000 7f578a59c700 1 -- 172.21.15.31:0/4204483120 >> v1:172.21.15.31:6789/0 conn(0x7f578c0c85d0 legacy=0x7f578c0cc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:22.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:22.418+0000 7f578a59c700 1 -- 172.21.15.31:0/4204483120 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f578c1573f0 con 0x7f578c0c8200 2023-10-06T02:24:22.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:22.419+0000 7f578a59c700 1 --2- 172.21.15.31:0/4204483120 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f578c0c8200 0x7f578c0c1820 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f577c0098e0 tx=0x7f577c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=370e3cbbb3327e1e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:22.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:22.419+0000 7f578959a700 1 -- 172.21.15.31:0/4204483120 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f577c00b640 con 0x7f578c0c8200 2023-10-06T02:24:22.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:22.419+0000 7f578959a700 1 -- 172.21.15.31:0/4204483120 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f577c00b7a0 con 0x7f578c0c8200 2023-10-06T02:24:22.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:22.419+0000 7f578959a700 1 -- 172.21.15.31:0/4204483120 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f577c00bb40 con 0x7f578c0c8200 2023-10-06T02:24:22.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:22.419+0000 7f578b59e700 1 -- 172.21.15.31:0/4204483120 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f578c0c8200 msgr2=0x7f578c0c1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:22.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:22.419+0000 7f578b59e700 1 --2- 172.21.15.31:0/4204483120 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f578c0c8200 0x7f578c0c1820 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f577c0098e0 tx=0x7f577c0091a0 comp rx=0 tx=0).stop 2023-10-06T02:24:22.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:22.419+0000 7f578b59e700 1 -- 172.21.15.31:0/4204483120 shutdown_connections 2023-10-06T02:24:22.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:22.419+0000 7f578b59e700 1 --2- 172.21.15.31:0/4204483120 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f578c0c8200 0x7f578c0c1820 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:22.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:22.419+0000 7f578b59e700 1 -- 172.21.15.31:0/4204483120 >> 172.21.15.31:0/4204483120 conn(0x7f578c06c6a0 msgr2=0x7f578c06caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:22.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:22.420+0000 7f578b59e700 1 -- 172.21.15.31:0/4204483120 shutdown_connections 2023-10-06T02:24:22.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:22.420+0000 7f578b59e700 1 -- 172.21.15.31:0/4204483120 wait complete. 2023-10-06T02:24:22.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:22.421+0000 7f578b59e700 1 Processor -- start 2023-10-06T02:24:22.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:22.421+0000 7f578b59e700 1 -- start start 2023-10-06T02:24:22.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:22.422+0000 7f578b59e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f578c0c2a90 0x7f578c13db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:22.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:22.422+0000 7f578b59e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f578c0c8200 0x7f578c0cb500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:22.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:22.422+0000 7f578b59e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f578c0c85d0 0x7f578c0cba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:22.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:22.422+0000 7f578b59e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f578c14afe0 con 0x7f578c0c8200 2023-10-06T02:24:22.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:22.422+0000 7f578b59e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f578c14e0d0 con 0x7f578c0c2a90 2023-10-06T02:24:22.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:22.422+0000 7f578b59e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f578c0ab870 con 0x7f578c0c85d0 2023-10-06T02:24:22.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:22.422+0000 7f578a59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f578c0c2a90 0x7f578c13db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:22.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:22.422+0000 7f5789d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f578c0c8200 0x7f578c0cb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:22.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:22.422+0000 7f578a59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f578c0c2a90 0x7f578c13db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:33198/0 (socket says 172.21.15.31:33198) 2023-10-06T02:24:22.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:22.423+0000 7f578a59c700 1 -- 172.21.15.31:0/3138533181 learned_addr learned my addr 172.21.15.31:0/3138533181 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:22.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:22.423+0000 7f578ad9d700 1 --2- 172.21.15.31:0/3138533181 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f578c0c85d0 0x7f578c0cba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:22.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:22.423+0000 7f578a59c700 1 -- 172.21.15.31:0/3138533181 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f578c0c85d0 msgr2=0x7f578c0cba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:22.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:22.423+0000 7f578a59c700 1 --2- 172.21.15.31:0/3138533181 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f578c0c85d0 0x7f578c0cba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:22.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:22.423+0000 7f578a59c700 1 -- 172.21.15.31:0/3138533181 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f578c0c8200 msgr2=0x7f578c0cb500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:22.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:22.423+0000 7f578a59c700 1 --2- 172.21.15.31:0/3138533181 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f578c0c8200 0x7f578c0cb500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:22.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:22.423+0000 7f578a59c700 1 -- 172.21.15.31:0/3138533181 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f577c009420 con 0x7f578c0c2a90 2023-10-06T02:24:22.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:22.423+0000 7f578a59c700 1 --2- 172.21.15.31:0/3138533181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f578c0c2a90 0x7f578c13db20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f577c009f80 tx=0x7f577c017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:22.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:22.423+0000 7f577a7fc700 1 -- 172.21.15.31:0/3138533181 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f577c000d10 con 0x7f578c0c2a90 2023-10-06T02:24:22.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:22.423+0000 7f577a7fc700 1 -- 172.21.15.31:0/3138533181 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f577c00bb20 con 0x7f578c0c2a90 2023-10-06T02:24:22.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:22.424+0000 7f578b59e700 1 -- 172.21.15.31:0/3138533181 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f578c0abb50 con 0x7f578c0c2a90 2023-10-06T02:24:22.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:22.424+0000 7f577a7fc700 1 -- 172.21.15.31:0/3138533181 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f577c011670 con 0x7f578c0c2a90 2023-10-06T02:24:22.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:22.424+0000 7f578b59e700 1 -- 172.21.15.31:0/3138533181 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f578c0ac110 con 0x7f578c0c2a90 2023-10-06T02:24:22.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:22.425+0000 7f578b59e700 1 -- 172.21.15.31:0/3138533181 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f578c04c3c0 con 0x7f578c0c2a90 2023-10-06T02:24:22.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:22.426+0000 7f577a7fc700 1 -- 172.21.15.31:0/3138533181 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f577c011a20 con 0x7f578c0c2a90 2023-10-06T02:24:22.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:22.427+0000 7f577a7fc700 1 --2- 172.21.15.31:0/3138533181 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f576c066f30 0x7f576c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:22.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:22.427+0000 7f577a7fc700 1 -- 172.21.15.31:0/3138533181 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f577c085d10 con 0x7f578c0c2a90 2023-10-06T02:24:22.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:22.427+0000 7f5789d9b700 1 --2- 172.21.15.31:0/3138533181 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f576c066f30 0x7f576c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:22.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:22.427+0000 7f5789d9b700 1 --2- 172.21.15.31:0/3138533181 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f576c066f30 0x7f576c0693e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f578c06d260 tx=0x7f5780007ec0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:22.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:22.431+0000 7f577a7fc700 1 -- 172.21.15.31:0/3138533181 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f577c08b020 con 0x7f578c0c2a90 2023-10-06T02:24:22.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:22.678+0000 7f578b59e700 1 -- 172.21.15.31:0/3138533181 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f578c0c2280 con 0x7f578c0c2a90 2023-10-06T02:24:23.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.194+0000 7f577a7fc700 1 -- 172.21.15.31:0/3138533181 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v18) v1 ==== 136+0+0 (secure 0 0 0) 0x7f577c052d80 con 0x7f578c0c2a90 2023-10-06T02:24:23.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.197+0000 7f578b59e700 1 -- 172.21.15.31:0/3138533181 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f576c066f30 msgr2=0x7f576c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:23.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.197+0000 7f578b59e700 1 --2- 172.21.15.31:0/3138533181 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f576c066f30 0x7f576c0693e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f578c06d260 tx=0x7f5780007ec0 comp rx=0 tx=0).stop 2023-10-06T02:24:23.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.197+0000 7f578b59e700 1 -- 172.21.15.31:0/3138533181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f578c0c2a90 msgr2=0x7f578c13db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:23.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.197+0000 7f578b59e700 1 --2- 172.21.15.31:0/3138533181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f578c0c2a90 0x7f578c13db20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f577c009f80 tx=0x7f577c017000 comp rx=0 tx=0).stop 2023-10-06T02:24:23.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.197+0000 7f578b59e700 1 -- 172.21.15.31:0/3138533181 shutdown_connections 2023-10-06T02:24:23.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.197+0000 7f578b59e700 1 --2- 172.21.15.31:0/3138533181 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f576c066f30 0x7f576c0693e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:23.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.197+0000 7f578b59e700 1 --2- 172.21.15.31:0/3138533181 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f578c0c85d0 0x7f578c0cba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:23.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.197+0000 7f578b59e700 1 --2- 172.21.15.31:0/3138533181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f578c0c2a90 0x7f578c13db20 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:23.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.197+0000 7f578b59e700 1 --2- 172.21.15.31:0/3138533181 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f578c0c8200 0x7f578c0cb500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:23.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.197+0000 7f578b59e700 1 -- 172.21.15.31:0/3138533181 >> 172.21.15.31:0/3138533181 conn(0x7f578c06c6a0 msgr2=0x7f578c0ca630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:23.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.197+0000 7f578b59e700 1 -- 172.21.15.31:0/3138533181 shutdown_connections 2023-10-06T02:24:23.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.197+0000 7f578b59e700 1 -- 172.21.15.31:0/3138533181 wait complete. 2023-10-06T02:24:23.219 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-06T02:24:23.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.411+0000 7f5b65d9d700 1 Processor -- start 2023-10-06T02:24:23.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.411+0000 7f5b65d9d700 1 -- start start 2023-10-06T02:24:23.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.412+0000 7f5b65d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5b60150380 0x7f5b60160ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:23.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.412+0000 7f5b65d9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b6006a490 con 0x7f5b6006f3a0 2023-10-06T02:24:23.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.412+0000 7f5b65d9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b600c7230 con 0x7f5b6006e0a0 2023-10-06T02:24:23.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.412+0000 7f5b65d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b6006b930 con 0x7f5b60150380 2023-10-06T02:24:23.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.412+0000 7f5b5ffff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f5b6006f3a0 0x7f5b6014fc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60368/0 (socket says 172.21.15.31:60368) 2023-10-06T02:24:23.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.412+0000 7f5b5ffff700 1 -- 172.21.15.31:0/3182233379 learned_addr learned my addr 172.21.15.31:0/3182233379 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:24:23.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.412+0000 7f5b6559c700 1 --2- 172.21.15.31:0/3182233379 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5b60150380 0x7f5b60160ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:23.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.413+0000 7f5b5f7fe700 1 -- 172.21.15.31:0/3182233379 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2698165962 0 0) 0x7f5b6006a490 con 0x7f5b6006f3a0 2023-10-06T02:24:23.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.413+0000 7f5b5f7fe700 1 -- 172.21.15.31:0/3182233379 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b50003570 con 0x7f5b6006f3a0 2023-10-06T02:24:23.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.413+0000 7f5b5f7fe700 1 -- 172.21.15.31:0/3182233379 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2644023423 0 0) 0x7f5b600c7230 con 0x7f5b6006e0a0 2023-10-06T02:24:23.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.413+0000 7f5b5f7fe700 1 -- 172.21.15.31:0/3182233379 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b6006a490 con 0x7f5b6006e0a0 2023-10-06T02:24:23.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.413+0000 7f5b6559c700 1 -- 172.21.15.31:0/3182233379 >> v1:172.21.15.62:6789/0 conn(0x7f5b6006e0a0 legacy=0x7f5b600c6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:23.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.413+0000 7f5b6559c700 1 -- 172.21.15.31:0/3182233379 >> v1:172.21.15.31:6789/0 conn(0x7f5b6006f3a0 legacy=0x7f5b6014fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:23.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.413+0000 7f5b5f7fe700 1 -- 172.21.15.31:0/3182233379 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 236790012 0 0) 0x7f5b50003570 con 0x7f5b6006f3a0 2023-10-06T02:24:23.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.413+0000 7f5b6559c700 1 -- 172.21.15.31:0/3182233379 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b600c7490 con 0x7f5b60150380 2023-10-06T02:24:23.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.414+0000 7f5b6559c700 1 --2- 172.21.15.31:0/3182233379 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5b60150380 0x7f5b60160ae0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5b5800afa0 tx=0x7f5b5800a890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2fd4d473bc65fe6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:23.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.414+0000 7f5b5f7fe700 1 -- 172.21.15.31:0/3182233379 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b5800f070 con 0x7f5b60150380 2023-10-06T02:24:23.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.414+0000 7f5b5f7fe700 1 -- 172.21.15.31:0/3182233379 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b5800a0d0 con 0x7f5b60150380 2023-10-06T02:24:23.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.414+0000 7f5b5f7fe700 1 -- 172.21.15.31:0/3182233379 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c0015f0 con 0x7f5b60150380 2023-10-06T02:24:23.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.414+0000 7f5b65d9d700 1 -- 172.21.15.31:0/3182233379 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5b60150380 msgr2=0x7f5b60160ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:23.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.414+0000 7f5b65d9d700 1 --2- 172.21.15.31:0/3182233379 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5b60150380 0x7f5b60160ae0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5b5800afa0 tx=0x7f5b5800a890 comp rx=0 tx=0).stop 2023-10-06T02:24:23.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.414+0000 7f5b65d9d700 1 -- 172.21.15.31:0/3182233379 shutdown_connections 2023-10-06T02:24:23.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.414+0000 7f5b65d9d700 1 --2- 172.21.15.31:0/3182233379 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5b60150380 0x7f5b60160ae0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:23.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.414+0000 7f5b65d9d700 1 -- 172.21.15.31:0/3182233379 >> 172.21.15.31:0/3182233379 conn(0x7f5b6006d170 msgr2=0x7f5b600cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:23.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.415+0000 7f5b65d9d700 1 -- 172.21.15.31:0/3182233379 shutdown_connections 2023-10-06T02:24:23.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.415+0000 7f5b65d9d700 1 -- 172.21.15.31:0/3182233379 wait complete. 2023-10-06T02:24:23.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.416+0000 7f5b65d9d700 1 Processor -- start 2023-10-06T02:24:23.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.416+0000 7f5b65d9d700 1 -- start start 2023-10-06T02:24:23.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.417+0000 7f5b65d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b6006e0a0 0x7f5b60169530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:23.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.417+0000 7f5b65d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b6006f3a0 0x7f5b6016d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:23.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.417+0000 7f5b65d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5b60150380 0x7f5b60156400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:23.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.417+0000 7f5b65d9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b600aef80 con 0x7f5b6006e0a0 2023-10-06T02:24:23.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.417+0000 7f5b65d9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b60169b70 con 0x7f5b6006f3a0 2023-10-06T02:24:23.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.417+0000 7f5b65d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b60169e50 con 0x7f5b60150380 2023-10-06T02:24:23.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.417+0000 7f5b64d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b6006e0a0 0x7f5b60169530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:23.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.417+0000 7f5b64d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b6006e0a0 0x7f5b60169530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46726/0 (socket says 172.21.15.31:46726) 2023-10-06T02:24:23.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.417+0000 7f5b64d9b700 1 -- 172.21.15.31:0/703141426 learned_addr learned my addr 172.21.15.31:0/703141426 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:23.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.418+0000 7f5b5ffff700 1 --2- 172.21.15.31:0/703141426 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b6006f3a0 0x7f5b6016d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:23.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.418+0000 7f5b6559c700 1 --2- 172.21.15.31:0/703141426 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5b60150380 0x7f5b60156400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:23.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.418+0000 7f5b64d9b700 1 -- 172.21.15.31:0/703141426 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5b60150380 msgr2=0x7f5b60156400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:23.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.418+0000 7f5b64d9b700 1 --2- 172.21.15.31:0/703141426 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5b60150380 0x7f5b60156400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:23.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.418+0000 7f5b64d9b700 1 -- 172.21.15.31:0/703141426 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b6006f3a0 msgr2=0x7f5b6016d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:23.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.418+0000 7f5b64d9b700 1 --2- 172.21.15.31:0/703141426 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b6006f3a0 0x7f5b6016d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:23.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.418+0000 7f5b64d9b700 1 -- 172.21.15.31:0/703141426 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b5800ab30 con 0x7f5b6006e0a0 2023-10-06T02:24:23.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.418+0000 7f5b5ffff700 1 --2- 172.21.15.31:0/703141426 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b6006f3a0 0x7f5b6016d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:24:23.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.418+0000 7f5b6559c700 1 --2- 172.21.15.31:0/703141426 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5b60150380 0x7f5b60156400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:24:23.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.418+0000 7f5b64d9b700 1 --2- 172.21.15.31:0/703141426 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b6006e0a0 0x7f5b60169530 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f5b5400c790 tx=0x7f5b54007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:23.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.418+0000 7f5b5cff9700 1 -- 172.21.15.31:0/703141426 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b54010510 con 0x7f5b6006e0a0 2023-10-06T02:24:23.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.418+0000 7f5b5cff9700 1 -- 172.21.15.31:0/703141426 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b54010670 con 0x7f5b6006e0a0 2023-10-06T02:24:23.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.418+0000 7f5b5cff9700 1 -- 172.21.15.31:0/703141426 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b54010a10 con 0x7f5b6006e0a0 2023-10-06T02:24:23.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.418+0000 7f5b65d9d700 1 -- 172.21.15.31:0/703141426 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b6016a130 con 0x7f5b6006e0a0 2023-10-06T02:24:23.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.418+0000 7f5b65d9d700 1 -- 172.21.15.31:0/703141426 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b6016da60 con 0x7f5b6006e0a0 2023-10-06T02:24:23.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.419+0000 7f5b65d9d700 1 -- 172.21.15.31:0/703141426 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b600af0c0 con 0x7f5b6006e0a0 2023-10-06T02:24:23.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.420+0000 7f5b5cff9700 1 -- 172.21.15.31:0/703141426 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5b5401b400 con 0x7f5b6006e0a0 2023-10-06T02:24:23.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.421+0000 7f5b5cff9700 1 --2- 172.21.15.31:0/703141426 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5b44066f30 0x7f5b440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:23.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.421+0000 7f5b5cff9700 1 -- 172.21.15.31:0/703141426 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f5b540844c0 con 0x7f5b6006e0a0 2023-10-06T02:24:23.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.421+0000 7f5b5ffff700 1 --2- 172.21.15.31:0/703141426 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5b44066f30 0x7f5b440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:23.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.422+0000 7f5b5ffff700 1 --2- 172.21.15.31:0/703141426 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5b44066f30 0x7f5b440693e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c000c00 tx=0x7f5b4c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:23.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.425+0000 7f5b5cff9700 1 -- 172.21.15.31:0/703141426 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5b54051530 con 0x7f5b6006e0a0 2023-10-06T02:24:23.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.661+0000 7f5b65d9d700 1 -- 172.21.15.31:0/703141426 --> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f5b6004c3c0 con 0x7f5b44066f30 2023-10-06T02:24:23.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.662+0000 7f5b5cff9700 1 -- 172.21.15.31:0/703141426 <== mgr.4107 v2:172.21.15.31:6832/85292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f5b6004c3c0 con 0x7f5b44066f30 2023-10-06T02:24:23.663 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2023-10-06T02:24:23.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:24:23.663 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephfs" 2023-10-06T02:24:23.663 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:24:23.663 INFO:teuthology.orchestra.run.smithi031.stdout:] 2023-10-06T02:24:23.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.665+0000 7f5b65d9d700 1 -- 172.21.15.31:0/703141426 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5b44066f30 msgr2=0x7f5b440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:23.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.665+0000 7f5b65d9d700 1 --2- 172.21.15.31:0/703141426 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5b44066f30 0x7f5b440693e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c000c00 tx=0x7f5b4c009040 comp rx=0 tx=0).stop 2023-10-06T02:24:23.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.665+0000 7f5b65d9d700 1 -- 172.21.15.31:0/703141426 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b6006e0a0 msgr2=0x7f5b60169530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:23.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.665+0000 7f5b65d9d700 1 --2- 172.21.15.31:0/703141426 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b6006e0a0 0x7f5b60169530 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f5b5400c790 tx=0x7f5b54007e10 comp rx=0 tx=0).stop 2023-10-06T02:24:23.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.665+0000 7f5b65d9d700 1 -- 172.21.15.31:0/703141426 shutdown_connections 2023-10-06T02:24:23.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.665+0000 7f5b65d9d700 1 --2- 172.21.15.31:0/703141426 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5b44066f30 0x7f5b440693e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:23.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.665+0000 7f5b65d9d700 1 --2- 172.21.15.31:0/703141426 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5b60150380 0x7f5b60156400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:23.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.665+0000 7f5b65d9d700 1 --2- 172.21.15.31:0/703141426 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b6006e0a0 0x7f5b60169530 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:23.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.665+0000 7f5b65d9d700 1 --2- 172.21.15.31:0/703141426 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b6006f3a0 0x7f5b6016d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:23.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.665+0000 7f5b65d9d700 1 -- 172.21.15.31:0/703141426 >> 172.21.15.31:0/703141426 conn(0x7f5b6006d170 msgr2=0x7f5b6006a820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:23.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.665+0000 7f5b65d9d700 1 -- 172.21.15.31:0/703141426 shutdown_connections 2023-10-06T02:24:23.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.665+0000 7f5b65d9d700 1 -- 172.21.15.31:0/703141426 wait complete. 2023-10-06T02:24:23.683 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T02:24:23.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.872+0000 7f3878b9d700 1 Processor -- start 2023-10-06T02:24:23.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.872+0000 7f3878b9d700 1 -- start start 2023-10-06T02:24:23.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.874+0000 7f3878b9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3874155890 0x7f38740c7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:23.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.874+0000 7f3878b9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f387406de50 0x7f387406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:23.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.874+0000 7f3878b9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38740c7e70 con 0x7f3874155c60 2023-10-06T02:24:23.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.874+0000 7f3878b9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f387406e830 con 0x7f3874155890 2023-10-06T02:24:23.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.874+0000 7f3878b9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f387406b790 con 0x7f387406de50 2023-10-06T02:24:23.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.874+0000 7f3872ffd700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f3874155c60 0x7f387406a3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60374/0 (socket says 172.21.15.31:60374) 2023-10-06T02:24:23.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.874+0000 7f3872ffd700 1 -- 172.21.15.31:0/2583798755 learned_addr learned my addr 172.21.15.31:0/2583798755 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:24:23.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.874+0000 7f38737fe700 1 --2- 172.21.15.31:0/2583798755 >> v2:172.21.15.62:3300/0 conn(0x7f3874155890 0x7f38740c7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:23.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.874+0000 7f3873fff700 1 --2- 172.21.15.31:0/2583798755 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f387406de50 0x7f387406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:23.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.874+0000 7f38727fc700 1 -- 172.21.15.31:0/2583798755 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2782834441 0 0) 0x7f38740c7e70 con 0x7f3874155c60 2023-10-06T02:24:23.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.874+0000 7f38727fc700 1 -- 172.21.15.31:0/2583798755 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f385c003570 con 0x7f3874155c60 2023-10-06T02:24:23.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.875+0000 7f38737fe700 1 -- 172.21.15.31:0/2583798755 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f387406de50 msgr2=0x7f387406e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:23.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.875+0000 7f38737fe700 1 --2- 172.21.15.31:0/2583798755 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f387406de50 0x7f387406e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:23.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.875+0000 7f38737fe700 1 -- 172.21.15.31:0/2583798755 >> v1:172.21.15.31:6789/0 conn(0x7f3874155c60 legacy=0x7f387406a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:23.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.875+0000 7f38737fe700 1 -- 172.21.15.31:0/2583798755 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f387406ab80 con 0x7f3874155890 2023-10-06T02:24:23.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.875+0000 7f38737fe700 1 --2- 172.21.15.31:0/2583798755 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3874155890 0x7f38740c7930 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3864009910 tx=0x7f38640091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f554f0dbe1d20161 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:23.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.875+0000 7f38727fc700 1 -- 172.21.15.31:0/2583798755 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3864011630 con 0x7f3874155890 2023-10-06T02:24:23.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.875+0000 7f38727fc700 1 -- 172.21.15.31:0/2583798755 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3864011790 con 0x7f3874155890 2023-10-06T02:24:23.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.875+0000 7f38727fc700 1 -- 172.21.15.31:0/2583798755 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3864011b30 con 0x7f3874155890 2023-10-06T02:24:23.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.876+0000 7f3878b9d700 1 -- 172.21.15.31:0/2583798755 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3874155890 msgr2=0x7f38740c7930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:23.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.876+0000 7f3878b9d700 1 --2- 172.21.15.31:0/2583798755 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3874155890 0x7f38740c7930 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3864009910 tx=0x7f38640091a0 comp rx=0 tx=0).stop 2023-10-06T02:24:23.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.876+0000 7f3878b9d700 1 -- 172.21.15.31:0/2583798755 shutdown_connections 2023-10-06T02:24:23.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.876+0000 7f3878b9d700 1 --2- 172.21.15.31:0/2583798755 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3874155890 0x7f38740c7930 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:23.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.876+0000 7f3878b9d700 1 --2- 172.21.15.31:0/2583798755 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f387406de50 0x7f387406e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:23.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.876+0000 7f3878b9d700 1 -- 172.21.15.31:0/2583798755 >> 172.21.15.31:0/2583798755 conn(0x7f387406d110 msgr2=0x7f38740cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:23.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.876+0000 7f3878b9d700 1 -- 172.21.15.31:0/2583798755 shutdown_connections 2023-10-06T02:24:23.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.876+0000 7f3878b9d700 1 -- 172.21.15.31:0/2583798755 wait complete. 2023-10-06T02:24:23.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.877+0000 7f3878b9d700 1 Processor -- start 2023-10-06T02:24:23.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.877+0000 7f3878b9d700 1 -- start start 2023-10-06T02:24:23.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.878+0000 7f3878b9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f387406de50 0x7f38741519f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:23.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.878+0000 7f3878b9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3874155890 0x7f38740c1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:23.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.878+0000 7f3878b9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3874155c60 0x7f38740c2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:23.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.878+0000 7f3878b9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38740c0ed0 con 0x7f3874155c60 2023-10-06T02:24:23.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.878+0000 7f3878b9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38740c1120 con 0x7f387406de50 2023-10-06T02:24:23.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.878+0000 7f3878b9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38740c1400 con 0x7f3874155890 2023-10-06T02:24:23.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.879+0000 7f3873fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3874155c60 0x7f38740c2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:23.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.879+0000 7f3873fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3874155c60 0x7f38740c2420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46736/0 (socket says 172.21.15.31:46736) 2023-10-06T02:24:23.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.879+0000 7f3873fff700 1 -- 172.21.15.31:0/960445264 learned_addr learned my addr 172.21.15.31:0/960445264 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:23.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.879+0000 7f38737fe700 1 --2- 172.21.15.31:0/960445264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f387406de50 0x7f38741519f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:23.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.879+0000 7f3872ffd700 1 --2- 172.21.15.31:0/960445264 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3874155890 0x7f38740c1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:23.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.879+0000 7f3873fff700 1 -- 172.21.15.31:0/960445264 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3874155890 msgr2=0x7f38740c1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:23.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.879+0000 7f3873fff700 1 --2- 172.21.15.31:0/960445264 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3874155890 0x7f38740c1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:23.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.879+0000 7f3873fff700 1 -- 172.21.15.31:0/960445264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f387406de50 msgr2=0x7f38741519f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:23.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.879+0000 7f3873fff700 1 --2- 172.21.15.31:0/960445264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f387406de50 0x7f38741519f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:23.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.879+0000 7f3873fff700 1 -- 172.21.15.31:0/960445264 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3864009420 con 0x7f3874155c60 2023-10-06T02:24:23.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.879+0000 7f3873fff700 1 --2- 172.21.15.31:0/960445264 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3874155c60 0x7f38740c2420 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f386c009df0 tx=0x7f386c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:23.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.879+0000 7f385b7fe700 1 -- 172.21.15.31:0/960445264 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f386c00cb50 con 0x7f3874155c60 2023-10-06T02:24:23.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.880+0000 7f385b7fe700 1 -- 172.21.15.31:0/960445264 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f386c00f040 con 0x7f3874155c60 2023-10-06T02:24:23.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.880+0000 7f385b7fe700 1 -- 172.21.15.31:0/960445264 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f386c014480 con 0x7f3874155c60 2023-10-06T02:24:23.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.880+0000 7f3878b9d700 1 -- 172.21.15.31:0/960445264 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38740c16e0 con 0x7f3874155c60 2023-10-06T02:24:23.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.880+0000 7f3878b9d700 1 -- 172.21.15.31:0/960445264 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f387414c6a0 con 0x7f3874155c60 2023-10-06T02:24:23.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.881+0000 7f3878b9d700 1 -- 172.21.15.31:0/960445264 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38740c1c00 con 0x7f3874155c60 2023-10-06T02:24:23.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.881+0000 7f385b7fe700 1 -- 172.21.15.31:0/960445264 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f386c014830 con 0x7f3874155c60 2023-10-06T02:24:23.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.882+0000 7f385b7fe700 1 --2- 172.21.15.31:0/960445264 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f3860066f30 0x7f38600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:23.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.882+0000 7f385b7fe700 1 -- 172.21.15.31:0/960445264 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f386c084fa0 con 0x7f3874155c60 2023-10-06T02:24:23.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.883+0000 7f38737fe700 1 --2- 172.21.15.31:0/960445264 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f3860066f30 0x7f38600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:23.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.883+0000 7f38737fe700 1 --2- 172.21.15.31:0/960445264 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f3860066f30 0x7f38600693e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3864003080 tx=0x7f3864017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:23.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:23.887+0000 7f385b7fe700 1 -- 172.21.15.31:0/960445264 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f38740c1c00 con 0x7f3874155c60 2023-10-06T02:24:24.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.089+0000 7f3878b9d700 1 -- 172.21.15.31:0/960445264 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f38740c1c00 con 0x7f3874155c60 2023-10-06T02:24:24.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.090+0000 7f385b7fe700 1 -- 172.21.15.31:0/960445264 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f38740c1c00 con 0x7f3874155c60 2023-10-06T02:24:24.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.092+0000 7f3878b9d700 1 -- 172.21.15.31:0/960445264 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f3860066f30 msgr2=0x7f38600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:24.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.092+0000 7f3878b9d700 1 --2- 172.21.15.31:0/960445264 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f3860066f30 0x7f38600693e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3864003080 tx=0x7f3864017000 comp rx=0 tx=0).stop 2023-10-06T02:24:24.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.092+0000 7f3878b9d700 1 -- 172.21.15.31:0/960445264 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3874155c60 msgr2=0x7f38740c2420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:24.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.092+0000 7f3878b9d700 1 --2- 172.21.15.31:0/960445264 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3874155c60 0x7f38740c2420 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f386c009df0 tx=0x7f386c00b040 comp rx=0 tx=0).stop 2023-10-06T02:24:24.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.092+0000 7f3878b9d700 1 -- 172.21.15.31:0/960445264 shutdown_connections 2023-10-06T02:24:24.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.092+0000 7f3878b9d700 1 --2- 172.21.15.31:0/960445264 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3874155c60 0x7f38740c2420 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:24.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.092+0000 7f3878b9d700 1 --2- 172.21.15.31:0/960445264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f387406de50 0x7f38741519f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:24.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.092+0000 7f3878b9d700 1 --2- 172.21.15.31:0/960445264 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f3860066f30 0x7f38600693e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:24.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.092+0000 7f3878b9d700 1 --2- 172.21.15.31:0/960445264 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3874155890 0x7f38740c1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:24.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.092+0000 7f3878b9d700 1 -- 172.21.15.31:0/960445264 >> 172.21.15.31:0/960445264 conn(0x7f387406d110 msgr2=0x7f3874157810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:24.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.092+0000 7f3878b9d700 1 -- 172.21.15.31:0/960445264 shutdown_connections 2023-10-06T02:24:24.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.092+0000 7f3878b9d700 1 -- 172.21.15.31:0/960445264 wait complete. 2023-10-06T02:24:24.110 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:24:24.110 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/group_0000000000615651/subvolume_0000000000932187) 2023-10-06T02:24:24.180 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:24:24.181 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2023-10-06T02:24:24.236 INFO:teuthology.orchestra.run.smithi031.stdout:cephfs_data 2023-10-06T02:24:24.236 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:24:24.237 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data ./volumes/group_0000000000615651/subvolume_0000000000932187) 2023-10-06T02:24:24.304 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T02:24:24.304 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/home/ubuntu/cephtest/mnt.0/./volumes/group_0000000000615651/subvolume_0000000000932187/.meta 2023-10-06T02:24:24.372 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000932187 group_0000000000615651 2023-10-06T02:24:24.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.603+0000 7fc79bfff700 1 Processor -- start 2023-10-06T02:24:24.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.603+0000 7fc79bfff700 1 -- start start 2023-10-06T02:24:24.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.604+0000 7fc79bfff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc7941575a0 0x7fc794069dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:24.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.604+0000 7fc79bfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc7940ae8c0 0x7fc7940c7590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:24.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.604+0000 7fc79bfff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7940af090 con 0x7fc79406eaa0 2023-10-06T02:24:24.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.604+0000 7fc79bfff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc79406a950 con 0x7fc7940ae8c0 2023-10-06T02:24:24.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.604+0000 7fc79bfff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7940c7100 con 0x7fc7941575a0 2023-10-06T02:24:24.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.604+0000 7fc79a59c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc7940ae8c0 0x7fc7940c7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:24.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.604+0000 7fc79a59c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc7940ae8c0 0x7fc7940c7590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46738/0 (socket says 172.21.15.31:46738) 2023-10-06T02:24:24.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.604+0000 7fc79a59c700 1 -- 172.21.15.31:0/3885050955 learned_addr learned my addr 172.21.15.31:0/3885050955 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:24.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.604+0000 7fc799d9b700 1 --2- 172.21.15.31:0/3885050955 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc7941575a0 0x7fc794069dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:24.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.605+0000 7fc79a59c700 1 -- 172.21.15.31:0/3885050955 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc7941575a0 msgr2=0x7fc794069dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:24.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.605+0000 7fc79a59c700 1 --2- 172.21.15.31:0/3885050955 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc7941575a0 0x7fc794069dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:24.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.605+0000 7fc79a59c700 1 -- 172.21.15.31:0/3885050955 >> v1:172.21.15.62:6789/0 conn(0x7fc79406eaa0 legacy=0x7fc7940ae1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:24.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.605+0000 7fc79a59c700 1 -- 172.21.15.31:0/3885050955 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc79406a670 con 0x7fc7940ae8c0 2023-10-06T02:24:24.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.605+0000 7fc79a59c700 1 --2- 172.21.15.31:0/3885050955 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7940ae8c0 0x7fc7940c7590 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fc78c009f20 tx=0x7fc78c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45d225f4b82c6464 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:24.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.605+0000 7fc798d99700 1 -- 172.21.15.31:0/3885050955 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78c00e070 con 0x7fc7940ae8c0 2023-10-06T02:24:24.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.605+0000 7fc798d99700 1 -- 172.21.15.31:0/3885050955 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc78c00b9d0 con 0x7fc7940ae8c0 2023-10-06T02:24:24.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.605+0000 7fc798d99700 1 -- 172.21.15.31:0/3885050955 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78c0092e0 con 0x7fc7940ae8c0 2023-10-06T02:24:24.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.606+0000 7fc79bfff700 1 -- 172.21.15.31:0/3885050955 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7940ae8c0 msgr2=0x7fc7940c7590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:24.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.606+0000 7fc79bfff700 1 --2- 172.21.15.31:0/3885050955 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7940ae8c0 0x7fc7940c7590 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fc78c009f20 tx=0x7fc78c009840 comp rx=0 tx=0).stop 2023-10-06T02:24:24.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.606+0000 7fc79bfff700 1 -- 172.21.15.31:0/3885050955 shutdown_connections 2023-10-06T02:24:24.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.606+0000 7fc79bfff700 1 --2- 172.21.15.31:0/3885050955 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7940ae8c0 0x7fc7940c7590 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:24.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.606+0000 7fc79bfff700 1 --2- 172.21.15.31:0/3885050955 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc7941575a0 0x7fc794069dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:24.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.606+0000 7fc79bfff700 1 -- 172.21.15.31:0/3885050955 >> 172.21.15.31:0/3885050955 conn(0x7fc79406d170 msgr2=0x7fc7940cc2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:24.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.606+0000 7fc79bfff700 1 -- 172.21.15.31:0/3885050955 shutdown_connections 2023-10-06T02:24:24.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.606+0000 7fc79bfff700 1 -- 172.21.15.31:0/3885050955 wait complete. 2023-10-06T02:24:24.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.607+0000 7fc79bfff700 1 Processor -- start 2023-10-06T02:24:24.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.608+0000 7fc79bfff700 1 -- start start 2023-10-06T02:24:24.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.608+0000 7fc79bfff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc79406eaa0 0x7fc7941509e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:24.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.608+0000 7fc79bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7940ae8c0 0x7fc7940c4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:24.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.609+0000 7fc79bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7941575a0 0x7fc7940c3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:24.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.609+0000 7fc79bfff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7940b21a0 con 0x7fc7940ae8c0 2023-10-06T02:24:24.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.609+0000 7fc79bfff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7940c48a0 con 0x7fc7941575a0 2023-10-06T02:24:24.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.609+0000 7fc79bfff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc794150f20 con 0x7fc79406eaa0 2023-10-06T02:24:24.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.609+0000 7fc79959a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7940ae8c0 0x7fc7940c4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:24.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.609+0000 7fc799d9b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc79406eaa0 0x7fc7941509e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:24.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.609+0000 7fc79959a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7940ae8c0 0x7fc7940c4d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46748/0 (socket says 172.21.15.31:46748) 2023-10-06T02:24:24.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.609+0000 7fc79959a700 1 -- 172.21.15.31:0/3333010510 learned_addr learned my addr 172.21.15.31:0/3333010510 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:24.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.609+0000 7fc79a59c700 1 --2- 172.21.15.31:0/3333010510 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7941575a0 0x7fc7940c3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:24.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.609+0000 7fc79959a700 1 -- 172.21.15.31:0/3333010510 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc79406eaa0 msgr2=0x7fc7941509e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:24.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.609+0000 7fc79959a700 1 --2- 172.21.15.31:0/3333010510 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc79406eaa0 0x7fc7941509e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:24.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.609+0000 7fc79959a700 1 -- 172.21.15.31:0/3333010510 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7941575a0 msgr2=0x7fc7940c3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:24.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.609+0000 7fc79959a700 1 --2- 172.21.15.31:0/3333010510 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7941575a0 0x7fc7940c3380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:24.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.609+0000 7fc79959a700 1 -- 172.21.15.31:0/3333010510 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc78c009ae0 con 0x7fc7940ae8c0 2023-10-06T02:24:24.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.610+0000 7fc79959a700 1 --2- 172.21.15.31:0/3333010510 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7940ae8c0 0x7fc7940c4d30 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fc78400c4e0 tx=0x7fc78400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:24.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.610+0000 7fc789ffb700 1 -- 172.21.15.31:0/3333010510 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc784011590 con 0x7fc7940ae8c0 2023-10-06T02:24:24.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.610+0000 7fc789ffb700 1 -- 172.21.15.31:0/3333010510 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc784011710 con 0x7fc7940ae8c0 2023-10-06T02:24:24.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.610+0000 7fc789ffb700 1 -- 172.21.15.31:0/3333010510 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc784011ad0 con 0x7fc7940ae8c0 2023-10-06T02:24:24.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.610+0000 7fc79bfff700 1 -- 172.21.15.31:0/3333010510 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7940c3c90 con 0x7fc7940ae8c0 2023-10-06T02:24:24.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.610+0000 7fc79bfff700 1 -- 172.21.15.31:0/3333010510 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7940c4530 con 0x7fc7940ae8c0 2023-10-06T02:24:24.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.611+0000 7fc79bfff700 1 -- 172.21.15.31:0/3333010510 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7940c4080 con 0x7fc7940ae8c0 2023-10-06T02:24:24.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.612+0000 7fc789ffb700 1 -- 172.21.15.31:0/3333010510 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc78401b3b0 con 0x7fc7940ae8c0 2023-10-06T02:24:24.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.613+0000 7fc789ffb700 1 --2- 172.21.15.31:0/3333010510 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc778066f30 0x7fc7780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:24.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.613+0000 7fc789ffb700 1 -- 172.21.15.31:0/3333010510 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc784083a70 con 0x7fc7940ae8c0 2023-10-06T02:24:24.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.613+0000 7fc799d9b700 1 --2- 172.21.15.31:0/3333010510 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc778066f30 0x7fc7780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:24.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.613+0000 7fc799d9b700 1 --2- 172.21.15.31:0/3333010510 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc778066f30 0x7fc7780693e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc794156d10 tx=0x7fc7900091a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:24.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.617+0000 7fc789ffb700 1 -- 172.21.15.31:0/3333010510 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc784050ae0 con 0x7fc7940ae8c0 2023-10-06T02:24:24.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.850+0000 7fc79bfff700 1 -- 172.21.15.31:0/3333010510 --> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000932187", "group_name": "group_0000000000615651", "target": ["mon-mgr", ""]}) v1 -- 0x7fc7940c4a10 con 0x7fc778066f30 2023-10-06T02:24:24.856 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:24:24.855+0000 7f3e7b47e700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:24:24.856 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:24:24.855+0000 7f3e7b47e700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:24:24.856 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:24:24.855+0000 7f3e7b47e700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:24:24.856 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:24:24.855+0000 7f3e7b47e700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:24:24.856 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:24:24.855+0000 7f3e7b47e700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:24:24.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.900+0000 7fc789ffb700 1 -- 172.21.15.31:0/3333010510 <== mgr.4107 v2:172.21.15.31:6832/85292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+59 (secure 0 0 0) 0x7fc7940c4a10 con 0x7fc778066f30 2023-10-06T02:24:24.901 INFO:teuthology.orchestra.run.smithi031.stdout:/volumes/group_0000000000615651/subvolume_0000000000932187 2023-10-06T02:24:24.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.903+0000 7fc79bfff700 1 -- 172.21.15.31:0/3333010510 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc778066f30 msgr2=0x7fc7780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:24.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.903+0000 7fc79bfff700 1 --2- 172.21.15.31:0/3333010510 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc778066f30 0x7fc7780693e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc794156d10 tx=0x7fc7900091a0 comp rx=0 tx=0).stop 2023-10-06T02:24:24.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.903+0000 7fc79bfff700 1 -- 172.21.15.31:0/3333010510 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7940ae8c0 msgr2=0x7fc7940c4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:24.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.903+0000 7fc79bfff700 1 --2- 172.21.15.31:0/3333010510 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7940ae8c0 0x7fc7940c4d30 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fc78400c4e0 tx=0x7fc78400c7f0 comp rx=0 tx=0).stop 2023-10-06T02:24:24.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.903+0000 7fc79bfff700 1 -- 172.21.15.31:0/3333010510 shutdown_connections 2023-10-06T02:24:24.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.903+0000 7fc79bfff700 1 --2- 172.21.15.31:0/3333010510 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7941575a0 0x7fc7940c3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:24.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.903+0000 7fc79bfff700 1 --2- 172.21.15.31:0/3333010510 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc778066f30 0x7fc7780693e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:24.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.903+0000 7fc79bfff700 1 --2- 172.21.15.31:0/3333010510 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc79406eaa0 0x7fc7941509e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:24.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.903+0000 7fc79bfff700 1 --2- 172.21.15.31:0/3333010510 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7940ae8c0 0x7fc7940c4d30 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:24.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.903+0000 7fc79bfff700 1 -- 172.21.15.31:0/3333010510 >> 172.21.15.31:0/3333010510 conn(0x7fc79406d170 msgr2=0x7fc79415d260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:24.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.903+0000 7fc79bfff700 1 -- 172.21.15.31:0/3333010510 shutdown_connections 2023-10-06T02:24:24.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:24.903+0000 7fc79bfff700 1 -- 172.21.15.31:0/3333010510 wait complete. 2023-10-06T02:24:24.920 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000932187 group_0000000000615651 2023-10-06T02:24:25.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.111+0000 7fa15ad9d700 1 Processor -- start 2023-10-06T02:24:25.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.112+0000 7fa15ad9d700 1 -- start start 2023-10-06T02:24:25.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.113+0000 7fa15ad9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa1540c85d0 0x7fa1540cc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:25.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.113+0000 7fa15ad9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1541571b0 con 0x7fa1540c2a90 2023-10-06T02:24:25.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.113+0000 7fa15ad9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1541579b0 con 0x7fa1540c8200 2023-10-06T02:24:25.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.113+0000 7fa15ad9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1540c1390 con 0x7fa1540c85d0 2023-10-06T02:24:25.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.113+0000 7fa15959a700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fa1540c2a90 0x7fa154156aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60388/0 (socket says 172.21.15.31:60388) 2023-10-06T02:24:25.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.113+0000 7fa15959a700 1 -- 172.21.15.31:0/3351756037 learned_addr learned my addr 172.21.15.31:0/3351756037 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:24:25.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.113+0000 7fa15a59c700 1 --2- 172.21.15.31:0/3351756037 >> v2:172.21.15.62:3300/0 conn(0x7fa1540c85d0 0x7fa1540cc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:25.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.114+0000 7fa158d99700 1 -- 172.21.15.31:0/3351756037 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 639884312 0 0) 0x7fa144001110 con 0x7fa1540c2a90 2023-10-06T02:24:25.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.114+0000 7fa158d99700 1 -- 172.21.15.31:0/3351756037 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa140003570 con 0x7fa1540c2a90 2023-10-06T02:24:25.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.114+0000 7fa158d99700 1 -- 172.21.15.31:0/3351756037 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3020648675 0 0) 0x7fa1541579b0 con 0x7fa1540c8200 2023-10-06T02:24:25.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.114+0000 7fa158d99700 1 -- 172.21.15.31:0/3351756037 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa144001110 con 0x7fa1540c8200 2023-10-06T02:24:25.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.114+0000 7fa15a59c700 1 -- 172.21.15.31:0/3351756037 >> v1:172.21.15.62:6789/0 conn(0x7fa1540c8200 legacy=0x7fa1540c1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:25.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.114+0000 7fa15a59c700 1 -- 172.21.15.31:0/3351756037 >> v1:172.21.15.31:6789/0 conn(0x7fa1540c2a90 legacy=0x7fa154156aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:25.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.114+0000 7fa15a59c700 1 -- 172.21.15.31:0/3351756037 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1541573f0 con 0x7fa1540c85d0 2023-10-06T02:24:25.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.114+0000 7fa15a59c700 1 --2- 172.21.15.31:0/3351756037 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa1540c85d0 0x7fa1540cc2f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa14c009e60 tx=0x7fa14c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=260245aaa64f0d12 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:25.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.115+0000 7fa158d99700 1 -- 172.21.15.31:0/3351756037 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14c00e070 con 0x7fa1540c85d0 2023-10-06T02:24:25.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.115+0000 7fa158d99700 1 -- 172.21.15.31:0/3351756037 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa14c00bab0 con 0x7fa1540c85d0 2023-10-06T02:24:25.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.115+0000 7fa158d99700 1 -- 172.21.15.31:0/3351756037 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14c00be70 con 0x7fa1540c85d0 2023-10-06T02:24:25.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.115+0000 7fa15ad9d700 1 -- 172.21.15.31:0/3351756037 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa1540c85d0 msgr2=0x7fa1540cc2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:25.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.115+0000 7fa15ad9d700 1 --2- 172.21.15.31:0/3351756037 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa1540c85d0 0x7fa1540cc2f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa14c009e60 tx=0x7fa14c009780 comp rx=0 tx=0).stop 2023-10-06T02:24:25.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.115+0000 7fa15ad9d700 1 -- 172.21.15.31:0/3351756037 shutdown_connections 2023-10-06T02:24:25.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.115+0000 7fa15ad9d700 1 --2- 172.21.15.31:0/3351756037 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa1540c85d0 0x7fa1540cc2f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:25.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.115+0000 7fa15ad9d700 1 -- 172.21.15.31:0/3351756037 >> 172.21.15.31:0/3351756037 conn(0x7fa15406c6a0 msgr2=0x7fa15406caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:25.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.115+0000 7fa15ad9d700 1 -- 172.21.15.31:0/3351756037 shutdown_connections 2023-10-06T02:24:25.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.115+0000 7fa15ad9d700 1 -- 172.21.15.31:0/3351756037 wait complete. 2023-10-06T02:24:25.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.117+0000 7fa15ad9d700 1 Processor -- start 2023-10-06T02:24:25.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.117+0000 7fa15ad9d700 1 -- start start 2023-10-06T02:24:25.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.118+0000 7fa15ad9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa1540c2a90 0x7fa1540ae210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:25.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.118+0000 7fa15ad9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa1540c8200 0x7fa1540ac100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:25.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.118+0000 7fa15ad9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1540c85d0 0x7fa154141b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:25.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.118+0000 7fa15ad9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1540ac640 con 0x7fa1540c85d0 2023-10-06T02:24:25.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.118+0000 7fa15ad9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1540ac8c0 con 0x7fa1540c8200 2023-10-06T02:24:25.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.118+0000 7fa15ad9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1540acba0 con 0x7fa1540c2a90 2023-10-06T02:24:25.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.118+0000 7fa15a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1540c85d0 0x7fa154141b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:25.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.118+0000 7fa15a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1540c85d0 0x7fa154141b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46768/0 (socket says 172.21.15.31:46768) 2023-10-06T02:24:25.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.118+0000 7fa15a59c700 1 -- 172.21.15.31:0/4176585483 learned_addr learned my addr 172.21.15.31:0/4176585483 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:25.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.118+0000 7fa15959a700 1 --2- 172.21.15.31:0/4176585483 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa1540c8200 0x7fa1540ac100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:25.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.118+0000 7fa159d9b700 1 --2- 172.21.15.31:0/4176585483 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa1540c2a90 0x7fa1540ae210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:25.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.118+0000 7fa15a59c700 1 -- 172.21.15.31:0/4176585483 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa1540c2a90 msgr2=0x7fa1540ae210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:25.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.119+0000 7fa15a59c700 1 --2- 172.21.15.31:0/4176585483 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa1540c2a90 0x7fa1540ae210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:25.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.119+0000 7fa15a59c700 1 -- 172.21.15.31:0/4176585483 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa1540c8200 msgr2=0x7fa1540ac100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:25.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.119+0000 7fa15a59c700 1 --2- 172.21.15.31:0/4176585483 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa1540c8200 0x7fa1540ac100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:25.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.119+0000 7fa15a59c700 1 -- 172.21.15.31:0/4176585483 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa14c009a20 con 0x7fa1540c85d0 2023-10-06T02:24:25.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.119+0000 7fa159d9b700 1 --2- 172.21.15.31:0/4176585483 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa1540c2a90 0x7fa1540ae210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:24:25.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.119+0000 7fa15a59c700 1 --2- 172.21.15.31:0/4176585483 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1540c85d0 0x7fa154141b40 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fa1540697c0 tx=0x7fa14c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:25.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.119+0000 7fa149ffb700 1 -- 172.21.15.31:0/4176585483 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14c026480 con 0x7fa1540c85d0 2023-10-06T02:24:25.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.119+0000 7fa149ffb700 1 -- 172.21.15.31:0/4176585483 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa14c026600 con 0x7fa1540c85d0 2023-10-06T02:24:25.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.119+0000 7fa149ffb700 1 -- 172.21.15.31:0/4176585483 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14c0269c0 con 0x7fa1540c85d0 2023-10-06T02:24:25.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.119+0000 7fa15ad9d700 1 -- 172.21.15.31:0/4176585483 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1540ace80 con 0x7fa1540c85d0 2023-10-06T02:24:25.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.119+0000 7fa15ad9d700 1 -- 172.21.15.31:0/4176585483 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa154142080 con 0x7fa1540c85d0 2023-10-06T02:24:25.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.120+0000 7fa15ad9d700 1 -- 172.21.15.31:0/4176585483 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa15404c3c0 con 0x7fa1540c85d0 2023-10-06T02:24:25.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.121+0000 7fa149ffb700 1 -- 172.21.15.31:0/4176585483 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa14c01c400 con 0x7fa1540c85d0 2023-10-06T02:24:25.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.122+0000 7fa149ffb700 1 --2- 172.21.15.31:0/4176585483 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa138066f30 0x7fa1380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:25.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.122+0000 7fa149ffb700 1 -- 172.21.15.31:0/4176585483 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa14c086a40 con 0x7fa1540c85d0 2023-10-06T02:24:25.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.122+0000 7fa159d9b700 1 --2- 172.21.15.31:0/4176585483 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa138066f30 0x7fa1380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:25.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.123+0000 7fa159d9b700 1 --2- 172.21.15.31:0/4176585483 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa138066f30 0x7fa1380693e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa150007fd0 tx=0x7fa150009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:25.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.126+0000 7fa149ffb700 1 -- 172.21.15.31:0/4176585483 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa14c053ab0 con 0x7fa1540c85d0 2023-10-06T02:24:25.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.361+0000 7fa15ad9d700 1 -- 172.21.15.31:0/4176585483 --> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000932187", "group_name": "group_0000000000615651", "target": ["mon-mgr", ""]}) v1 -- 0x7fa1540c2280 con 0x7fa138066f30 2023-10-06T02:24:25.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.365+0000 7fa149ffb700 1 -- 172.21.15.31:0/4176585483 <== mgr.4107 v2:172.21.15.31:6832/85292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+59 (secure 0 0 0) 0x7fa1540c2280 con 0x7fa138066f30 2023-10-06T02:24:25.366 INFO:teuthology.orchestra.run.smithi031.stdout:/volumes/group_0000000000615651/subvolume_0000000000932187 2023-10-06T02:24:25.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.368+0000 7fa15ad9d700 1 -- 172.21.15.31:0/4176585483 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa138066f30 msgr2=0x7fa1380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:25.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.368+0000 7fa15ad9d700 1 --2- 172.21.15.31:0/4176585483 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa138066f30 0x7fa1380693e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa150007fd0 tx=0x7fa150009040 comp rx=0 tx=0).stop 2023-10-06T02:24:25.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.368+0000 7fa15ad9d700 1 -- 172.21.15.31:0/4176585483 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1540c85d0 msgr2=0x7fa154141b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:25.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.368+0000 7fa15ad9d700 1 --2- 172.21.15.31:0/4176585483 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1540c85d0 0x7fa154141b40 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fa1540697c0 tx=0x7fa14c019000 comp rx=0 tx=0).stop 2023-10-06T02:24:25.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.368+0000 7fa15ad9d700 1 -- 172.21.15.31:0/4176585483 shutdown_connections 2023-10-06T02:24:25.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.368+0000 7fa15ad9d700 1 --2- 172.21.15.31:0/4176585483 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1540c85d0 0x7fa154141b40 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:25.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.368+0000 7fa15ad9d700 1 --2- 172.21.15.31:0/4176585483 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa138066f30 0x7fa1380693e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:25.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.368+0000 7fa15ad9d700 1 --2- 172.21.15.31:0/4176585483 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa1540c2a90 0x7fa1540ae210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:25.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.368+0000 7fa15ad9d700 1 --2- 172.21.15.31:0/4176585483 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa1540c8200 0x7fa1540ac100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:25.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.368+0000 7fa15ad9d700 1 -- 172.21.15.31:0/4176585483 >> 172.21.15.31:0/4176585483 conn(0x7fa15406c6a0 msgr2=0x7fa15415a8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:25.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.368+0000 7fa15ad9d700 1 -- 172.21.15.31:0/4176585483 shutdown_connections 2023-10-06T02:24:25.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.368+0000 7fa15ad9d700 1 -- 172.21.15.31:0/4176585483 wait complete. 2023-10-06T02:24:25.385 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:24:25.385 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_legacy/408b5aff913c660626b9927478e1b1b7.meta) 2023-10-06T02:24:25.454 INFO:teuthology.orchestra.run.smithi031.stdout:[GLOBAL] 2023-10-06T02:24:25.454 INFO:teuthology.orchestra.run.smithi031.stdout:version = 1 2023-10-06T02:24:25.454 INFO:teuthology.orchestra.run.smithi031.stdout:type = subvolume 2023-10-06T02:24:25.454 INFO:teuthology.orchestra.run.smithi031.stdout:path = /volumes/group_0000000000615651/subvolume_0000000000932187 2023-10-06T02:24:25.454 INFO:teuthology.orchestra.run.smithi031.stdout:state = complete 2023-10-06T02:24:25.454 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:24:25.457 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000932187 group_0000000000615651 2023-10-06T02:24:25.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.647+0000 7f7a6f59e700 1 Processor -- start 2023-10-06T02:24:25.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.647+0000 7f7a6f59e700 1 -- start start 2023-10-06T02:24:25.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.649+0000 7f7a6f59e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7a700c8da0 0x7f7a700c1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:25.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.649+0000 7f7a6f59e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7a700c9170 0x7f7a700c1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:25.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.649+0000 7f7a6f59e700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a70069b00 con 0x7f7a7015c5e0 2023-10-06T02:24:25.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.649+0000 7f7a6f59e700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a700c6d50 con 0x7f7a700c9170 2023-10-06T02:24:25.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.649+0000 7f7a6f59e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a700c6ff0 con 0x7f7a700c8da0 2023-10-06T02:24:25.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.649+0000 7f7a6dd9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7a700c9170 0x7f7a700c1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:25.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.649+0000 7f7a6ed9d700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f7a7015c5e0 0x7f7a7015ca60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60396/0 (socket says 172.21.15.31:60396) 2023-10-06T02:24:25.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.649+0000 7f7a6ed9d700 1 -- 172.21.15.31:0/3311063601 learned_addr learned my addr 172.21.15.31:0/3311063601 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:24:25.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.649+0000 7f7a6e59c700 1 --2- 172.21.15.31:0/3311063601 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7a700c8da0 0x7f7a700c1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:25.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.650+0000 7f7a6dd9b700 1 -- 172.21.15.31:0/3311063601 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7a700c8da0 msgr2=0x7f7a700c1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:25.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.650+0000 7f7a6dd9b700 1 --2- 172.21.15.31:0/3311063601 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7a700c8da0 0x7f7a700c1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:25.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.650+0000 7f7a6dd9b700 1 -- 172.21.15.31:0/3311063601 >> v1:172.21.15.31:6789/0 conn(0x7f7a7015c5e0 legacy=0x7f7a7015ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:25.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.650+0000 7f7a6d59a700 1 -- 172.21.15.31:0/3311063601 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3558782152 0 0) 0x7f7a70069b00 con 0x7f7a7015c5e0 2023-10-06T02:24:25.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.650+0000 7f7a6dd9b700 1 -- 172.21.15.31:0/3311063601 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a7015d170 con 0x7f7a700c9170 2023-10-06T02:24:25.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.650+0000 7f7a6dd9b700 1 --2- 172.21.15.31:0/3311063601 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a700c9170 0x7f7a700c1580 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f7a640098e0 tx=0x7f7a640091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d98bcc116bea8fb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:25.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.650+0000 7f7a6d59a700 1 -- 172.21.15.31:0/3311063601 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a64009e80 con 0x7f7a700c9170 2023-10-06T02:24:25.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.650+0000 7f7a6d59a700 1 -- 172.21.15.31:0/3311063601 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a6400b860 con 0x7f7a700c9170 2023-10-06T02:24:25.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.650+0000 7f7a6d59a700 1 -- 172.21.15.31:0/3311063601 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a6400bbe0 con 0x7f7a700c9170 2023-10-06T02:24:25.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.651+0000 7f7a6f59e700 1 -- 172.21.15.31:0/3311063601 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a700c9170 msgr2=0x7f7a700c1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:25.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.651+0000 7f7a6f59e700 1 --2- 172.21.15.31:0/3311063601 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a700c9170 0x7f7a700c1580 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f7a640098e0 tx=0x7f7a640091a0 comp rx=0 tx=0).stop 2023-10-06T02:24:25.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.651+0000 7f7a6f59e700 1 -- 172.21.15.31:0/3311063601 shutdown_connections 2023-10-06T02:24:25.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.651+0000 7f7a6f59e700 1 --2- 172.21.15.31:0/3311063601 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7a700c8da0 0x7f7a700c1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:25.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.651+0000 7f7a6f59e700 1 --2- 172.21.15.31:0/3311063601 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a700c9170 0x7f7a700c1580 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:25.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.651+0000 7f7a6f59e700 1 -- 172.21.15.31:0/3311063601 >> 172.21.15.31:0/3311063601 conn(0x7f7a7006c6a0 msgr2=0x7f7a7006caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:25.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.651+0000 7f7a6f59e700 1 -- 172.21.15.31:0/3311063601 shutdown_connections 2023-10-06T02:24:25.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.651+0000 7f7a6f59e700 1 -- 172.21.15.31:0/3311063601 wait complete. 2023-10-06T02:24:25.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.652+0000 7f7a6f59e700 1 Processor -- start 2023-10-06T02:24:25.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.652+0000 7f7a6f59e700 1 -- start start 2023-10-06T02:24:25.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.653+0000 7f7a6f59e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7a700c8da0 0x7f7a700caad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:25.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.653+0000 7f7a6f59e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a700c9170 0x7f7a70153fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:25.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.653+0000 7f7a6f59e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a7015c5e0 0x7f7a7015d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:25.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.653+0000 7f7a6f59e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a700b14a0 con 0x7f7a7015c5e0 2023-10-06T02:24:25.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.653+0000 7f7a6f59e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a700b1780 con 0x7f7a700c9170 2023-10-06T02:24:25.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.653+0000 7f7a6f59e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a700b1a60 con 0x7f7a700c8da0 2023-10-06T02:24:25.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.654+0000 7f7a6ed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a7015c5e0 0x7f7a7015d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:25.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.654+0000 7f7a6ed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a7015c5e0 0x7f7a7015d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46794/0 (socket says 172.21.15.31:46794) 2023-10-06T02:24:25.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.654+0000 7f7a6ed9d700 1 -- 172.21.15.31:0/2044762613 learned_addr learned my addr 172.21.15.31:0/2044762613 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:25.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.654+0000 7f7a6e59c700 1 --2- 172.21.15.31:0/2044762613 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7a700c8da0 0x7f7a700caad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:25.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.654+0000 7f7a6dd9b700 1 --2- 172.21.15.31:0/2044762613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a700c9170 0x7f7a70153fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:25.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.654+0000 7f7a6ed9d700 1 -- 172.21.15.31:0/2044762613 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7a700c8da0 msgr2=0x7f7a700caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:25.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.654+0000 7f7a6ed9d700 1 --2- 172.21.15.31:0/2044762613 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7a700c8da0 0x7f7a700caad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:25.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.654+0000 7f7a6ed9d700 1 -- 172.21.15.31:0/2044762613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a700c9170 msgr2=0x7f7a70153fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:25.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.654+0000 7f7a6ed9d700 1 --2- 172.21.15.31:0/2044762613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a700c9170 0x7f7a70153fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:25.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.654+0000 7f7a6ed9d700 1 -- 172.21.15.31:0/2044762613 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a64009420 con 0x7f7a7015c5e0 2023-10-06T02:24:25.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.654+0000 7f7a6ed9d700 1 --2- 172.21.15.31:0/2044762613 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a7015c5e0 0x7f7a7015d530 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f7a6800cbf0 tx=0x7f7a680019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:25.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.654+0000 7f7a5e7fc700 1 -- 172.21.15.31:0/2044762613 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a6800a8b0 con 0x7f7a7015c5e0 2023-10-06T02:24:25.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.655+0000 7f7a5e7fc700 1 -- 172.21.15.31:0/2044762613 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a6800aa10 con 0x7f7a7015c5e0 2023-10-06T02:24:25.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.655+0000 7f7a5e7fc700 1 -- 172.21.15.31:0/2044762613 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a6800add0 con 0x7f7a7015c5e0 2023-10-06T02:24:25.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.655+0000 7f7a6f59e700 1 -- 172.21.15.31:0/2044762613 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a700b1d40 con 0x7f7a7015c5e0 2023-10-06T02:24:25.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.655+0000 7f7a6f59e700 1 -- 172.21.15.31:0/2044762613 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a701544f0 con 0x7f7a7015c5e0 2023-10-06T02:24:25.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.656+0000 7f7a6f59e700 1 -- 172.21.15.31:0/2044762613 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a7004c3c0 con 0x7f7a7015c5e0 2023-10-06T02:24:25.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.657+0000 7f7a5e7fc700 1 -- 172.21.15.31:0/2044762613 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7a68007300 con 0x7f7a7015c5e0 2023-10-06T02:24:25.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.658+0000 7f7a5e7fc700 1 --2- 172.21.15.31:0/2044762613 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f7a50066f30 0x7f7a500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:25.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.659+0000 7f7a5e7fc700 1 -- 172.21.15.31:0/2044762613 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f7a680849d0 con 0x7f7a7015c5e0 2023-10-06T02:24:25.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.659+0000 7f7a6e59c700 1 --2- 172.21.15.31:0/2044762613 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f7a50066f30 0x7f7a500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:25.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.659+0000 7f7a6e59c700 1 --2- 172.21.15.31:0/2044762613 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f7a50066f30 0x7f7a500693e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7a60006fd0 tx=0x7f7a60008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:25.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.662+0000 7f7a5e7fc700 1 -- 172.21.15.31:0/2044762613 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7a68051ac0 con 0x7f7a7015c5e0 2023-10-06T02:24:25.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.901+0000 7f7a6f59e700 1 -- 172.21.15.31:0/2044762613 --> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000932187", "group_name": "group_0000000000615651", "target": ["mon-mgr", ""]}) v1 -- 0x7f7a700a9b20 con 0x7f7a50066f30 2023-10-06T02:24:25.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.920+0000 7f7a5e7fc700 1 -- 172.21.15.31:0/2044762613 <== mgr.4107 v2:172.21.15.31:6832/85292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f7a700a9b20 con 0x7f7a50066f30 2023-10-06T02:24:25.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.923+0000 7f7a6f59e700 1 -- 172.21.15.31:0/2044762613 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f7a50066f30 msgr2=0x7f7a500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:25.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.923+0000 7f7a6f59e700 1 --2- 172.21.15.31:0/2044762613 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f7a50066f30 0x7f7a500693e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7a60006fd0 tx=0x7f7a60008040 comp rx=0 tx=0).stop 2023-10-06T02:24:25.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.923+0000 7f7a6f59e700 1 -- 172.21.15.31:0/2044762613 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a7015c5e0 msgr2=0x7f7a7015d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:25.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.923+0000 7f7a6f59e700 1 --2- 172.21.15.31:0/2044762613 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a7015c5e0 0x7f7a7015d530 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f7a6800cbf0 tx=0x7f7a680019b0 comp rx=0 tx=0).stop 2023-10-06T02:24:25.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.923+0000 7f7a6f59e700 1 -- 172.21.15.31:0/2044762613 shutdown_connections 2023-10-06T02:24:25.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.923+0000 7f7a6f59e700 1 --2- 172.21.15.31:0/2044762613 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a7015c5e0 0x7f7a7015d530 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:25.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.923+0000 7f7a6f59e700 1 --2- 172.21.15.31:0/2044762613 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f7a50066f30 0x7f7a500693e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:25.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.923+0000 7f7a6f59e700 1 --2- 172.21.15.31:0/2044762613 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7a700c8da0 0x7f7a700caad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:25.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.923+0000 7f7a6f59e700 1 --2- 172.21.15.31:0/2044762613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a700c9170 0x7f7a70153fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:25.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.923+0000 7f7a6f59e700 1 -- 172.21.15.31:0/2044762613 >> 172.21.15.31:0/2044762613 conn(0x7f7a7006c6a0 msgr2=0x7f7a7015ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:25.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.923+0000 7f7a6f59e700 1 -- 172.21.15.31:0/2044762613 shutdown_connections 2023-10-06T02:24:25.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:25.923+0000 7f7a6f59e700 1 -- 172.21.15.31:0/2044762613 wait complete. 2023-10-06T02:24:25.925 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:24:25.924+0000 7f3e7cc81700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:24:25.925 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:24:25.924+0000 7f3e7cc81700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:24:25.925 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:24:25.924+0000 7f3e7cc81700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:24:25.925 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:24:25.924+0000 7f3e7cc81700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:24:25.925 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:24:25.925+0000 7f3e7cc81700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:24:25.944 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:24:25.944 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2023-10-06T02:24:25.945 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:24:25.944+0000 7f3e7dc83700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:24:25.945 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:24:25.944+0000 7f3e7dc83700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:24:25.945 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:24:25.944+0000 7f3e7dc83700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:24:25.945 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:24:25.944+0000 7f3e7dc83700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:24:25.945 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:24:25.944+0000 7f3e7dc83700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:24:26.022 INFO:teuthology.orchestra.run.smithi031.stdout:2 2023-10-06T02:24:26.024 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm cephfs group_0000000000615651 2023-10-06T02:24:26.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.215+0000 7f5e4f59d700 1 Processor -- start 2023-10-06T02:24:26.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.215+0000 7f5e4f59d700 1 -- start start 2023-10-06T02:24:26.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.216+0000 7f5e4f59d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5e48153dd0 0x7f5e4815de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:26.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.216+0000 7f5e4f59d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f5e480cea00 0x7f5e480cee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:26.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.216+0000 7f5e4f59d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e4815ac50 con 0x7f5e48153a00 2023-10-06T02:24:26.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.216+0000 7f5e4f59d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e4815d010 con 0x7f5e480cea00 2023-10-06T02:24:26.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.216+0000 7f5e4f59d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4815af60 con 0x7f5e48153dd0 2023-10-06T02:24:26.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.216+0000 7f5e4e59b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f5e48153a00 0x7f5e4815a540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60406/0 (socket says 172.21.15.31:60406) 2023-10-06T02:24:26.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.217+0000 7f5e4e59b700 1 -- 172.21.15.31:0/1213360521 learned_addr learned my addr 172.21.15.31:0/1213360521 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:24:26.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.217+0000 7f5e4dd9a700 1 --2- 172.21.15.31:0/1213360521 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5e48153dd0 0x7f5e4815de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:26.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.217+0000 7f5e4ed9c700 1 --2- 172.21.15.31:0/1213360521 >> v2:172.21.15.62:3300/0 conn(0x7f5e480cea00 0x7f5e480cee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:26.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.217+0000 7f5e4d599700 1 -- 172.21.15.31:0/1213360521 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2946053651 0 0) 0x7f5e4815ac50 con 0x7f5e48153a00 2023-10-06T02:24:26.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.217+0000 7f5e4d599700 1 -- 172.21.15.31:0/1213360521 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e34003570 con 0x7f5e48153a00 2023-10-06T02:24:26.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.217+0000 7f5e4d599700 1 -- 172.21.15.31:0/1213360521 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1792275913 0 0) 0x7f5e34003570 con 0x7f5e48153a00 2023-10-06T02:24:26.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.217+0000 7f5e4d599700 1 -- 172.21.15.31:0/1213360521 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5e48153dd0 msgr2=0x7f5e4815de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:26.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.217+0000 7f5e4d599700 1 --2- 172.21.15.31:0/1213360521 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5e48153dd0 0x7f5e4815de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:26.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.217+0000 7f5e4d599700 1 -- 172.21.15.31:0/1213360521 >> v2:172.21.15.62:3300/0 conn(0x7f5e480cea00 msgr2=0x7f5e480cee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:26.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.217+0000 7f5e4d599700 1 --2- 172.21.15.31:0/1213360521 >> v2:172.21.15.62:3300/0 conn(0x7f5e480cea00 0x7f5e480cee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:26.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.217+0000 7f5e4d599700 1 -- 172.21.15.31:0/1213360521 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e4815b240 con 0x7f5e48153a00 2023-10-06T02:24:26.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.217+0000 7f5e4d599700 1 -- 172.21.15.31:0/1213360521 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7f5e44001470 con 0x7f5e48153a00 2023-10-06T02:24:26.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.217+0000 7f5e4d599700 1 -- 172.21.15.31:0/1213360521 >> v1:172.21.15.31:6789/0 conn(0x7f5e48153a00 legacy=0x7f5e4815a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:26.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.217+0000 7f5e4d599700 1 --2- 172.21.15.31:0/1213360521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e34003ae0 0x7f5e34003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:26.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.217+0000 7f5e4d599700 1 --2- 172.21.15.31:0/1213360521 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5e48153dd0 0x7f5e34004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:26.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.217+0000 7f5e4d599700 1 --2- 172.21.15.31:0/1213360521 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e34004d50 0x7f5e340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:26.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.217+0000 7f5e4d599700 1 -- 172.21.15.31:0/1213360521 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4815d010 con 0x7f5e34004d50 2023-10-06T02:24:26.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.217+0000 7f5e4d599700 1 -- 172.21.15.31:0/1213360521 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4815af60 con 0x7f5e34003ae0 2023-10-06T02:24:26.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.217+0000 7f5e4d599700 1 -- 172.21.15.31:0/1213360521 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3400bb10 con 0x7f5e48153dd0 2023-10-06T02:24:26.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.218+0000 7f5e4ed9c700 1 --2- 172.21.15.31:0/1213360521 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e34004d50 0x7f5e340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:26.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.218+0000 7f5e4e59b700 1 --2- 172.21.15.31:0/1213360521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e34003ae0 0x7f5e34003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:26.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.218+0000 7f5e4dd9a700 1 --2- 172.21.15.31:0/1213360521 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5e48153dd0 0x7f5e34004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:26.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.218+0000 7f5e4ed9c700 1 -- 172.21.15.31:0/1213360521 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5e48153dd0 msgr2=0x7f5e34004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:26.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.218+0000 7f5e4ed9c700 1 --2- 172.21.15.31:0/1213360521 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5e48153dd0 0x7f5e34004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:26.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.218+0000 7f5e4ed9c700 1 -- 172.21.15.31:0/1213360521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e34003ae0 msgr2=0x7f5e34003ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:26.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.218+0000 7f5e4ed9c700 1 --2- 172.21.15.31:0/1213360521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e34003ae0 0x7f5e34003ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:26.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.218+0000 7f5e4ed9c700 1 -- 172.21.15.31:0/1213360521 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e34003570 con 0x7f5e34004d50 2023-10-06T02:24:26.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.218+0000 7f5e4ed9c700 1 --2- 172.21.15.31:0/1213360521 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e34004d50 0x7f5e340051c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f5e4000e240 tx=0x7f5e4000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=808ce560c21cfecc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:26.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.219+0000 7f5e4d599700 1 -- 172.21.15.31:0/1213360521 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e4000eaf0 con 0x7f5e34004d50 2023-10-06T02:24:26.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.219+0000 7f5e4d599700 1 -- 172.21.15.31:0/1213360521 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e4000f040 con 0x7f5e34004d50 2023-10-06T02:24:26.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.219+0000 7f5e4d599700 1 -- 172.21.15.31:0/1213360521 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e4000eea0 con 0x7f5e34004d50 2023-10-06T02:24:26.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.219+0000 7f5e4f59d700 1 -- 172.21.15.31:0/1213360521 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e34004d50 msgr2=0x7f5e340051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:26.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.219+0000 7f5e4f59d700 1 --2- 172.21.15.31:0/1213360521 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e34004d50 0x7f5e340051c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f5e4000e240 tx=0x7f5e4000b040 comp rx=0 tx=0).stop 2023-10-06T02:24:26.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.219+0000 7f5e4ed9c700 1 -- 172.21.15.31:0/1213360521 reap_dead start 2023-10-06T02:24:26.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.219+0000 7f5e4f59d700 1 -- 172.21.15.31:0/1213360521 shutdown_connections 2023-10-06T02:24:26.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.219+0000 7f5e4f59d700 1 -- 172.21.15.31:0/1213360521 >> 172.21.15.31:0/1213360521 conn(0x7f5e480703a0 msgr2=0x7f5e480707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:26.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.219+0000 7f5e4f59d700 1 -- 172.21.15.31:0/1213360521 shutdown_connections 2023-10-06T02:24:26.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.219+0000 7f5e4f59d700 1 -- 172.21.15.31:0/1213360521 wait complete. 2023-10-06T02:24:26.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.221+0000 7f5e4f59d700 1 Processor -- start 2023-10-06T02:24:26.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.221+0000 7f5e4f59d700 1 -- start start 2023-10-06T02:24:26.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.222+0000 7f5e4f59d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5e34004d50 0x7f5e48147030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:26.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.222+0000 7f5e4f59d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e480c9bc0 0x7f5e4816a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:26.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.222+0000 7f5e4f59d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e48156530 0x7f5e4806d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:26.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.222+0000 7f5e4f59d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e48147570 con 0x7f5e48156530 2023-10-06T02:24:26.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.222+0000 7f5e4f59d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e481571b0 con 0x7f5e480c9bc0 2023-10-06T02:24:26.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.222+0000 7f5e4f59d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e481560a0 con 0x7f5e34004d50 2023-10-06T02:24:26.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.222+0000 7f5e4ed9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e48156530 0x7f5e4806d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:26.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.222+0000 7f5e4ed9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e48156530 0x7f5e4806d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46846/0 (socket says 172.21.15.31:46846) 2023-10-06T02:24:26.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.222+0000 7f5e4ed9c700 1 -- 172.21.15.31:0/1879706288 learned_addr learned my addr 172.21.15.31:0/1879706288 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:26.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.222+0000 7f5e4dd9a700 1 --2- 172.21.15.31:0/1879706288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e480c9bc0 0x7f5e4816a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:26.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.222+0000 7f5e4e59b700 1 --2- 172.21.15.31:0/1879706288 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5e34004d50 0x7f5e48147030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:26.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.222+0000 7f5e4ed9c700 1 -- 172.21.15.31:0/1879706288 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5e34004d50 msgr2=0x7f5e48147030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:26.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.223+0000 7f5e4ed9c700 1 --2- 172.21.15.31:0/1879706288 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5e34004d50 0x7f5e48147030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:26.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.223+0000 7f5e4ed9c700 1 -- 172.21.15.31:0/1879706288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e480c9bc0 msgr2=0x7f5e4816a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:26.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.223+0000 7f5e4ed9c700 1 --2- 172.21.15.31:0/1879706288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e480c9bc0 0x7f5e4816a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:26.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.223+0000 7f5e4ed9c700 1 -- 172.21.15.31:0/1879706288 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e4816aa30 con 0x7f5e48156530 2023-10-06T02:24:26.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.223+0000 7f5e4e59b700 1 --2- 172.21.15.31:0/1879706288 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5e34004d50 0x7f5e48147030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:24:26.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.223+0000 7f5e4dd9a700 1 --2- 172.21.15.31:0/1879706288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e480c9bc0 0x7f5e4816a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:24:26.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.223+0000 7f5e4ed9c700 1 --2- 172.21.15.31:0/1879706288 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e48156530 0x7f5e4806d690 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f5e4000e210 tx=0x7f5e4001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:26.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.223+0000 7f5e3e7fc700 1 -- 172.21.15.31:0/1879706288 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e4001d630 con 0x7f5e48156530 2023-10-06T02:24:26.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.223+0000 7f5e3e7fc700 1 -- 172.21.15.31:0/1879706288 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e4001d7b0 con 0x7f5e48156530 2023-10-06T02:24:26.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.223+0000 7f5e3e7fc700 1 -- 172.21.15.31:0/1879706288 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e4001db70 con 0x7f5e48156530 2023-10-06T02:24:26.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.223+0000 7f5e4f59d700 1 -- 172.21.15.31:0/1879706288 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e480ca0c0 con 0x7f5e48156530 2023-10-06T02:24:26.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.223+0000 7f5e4f59d700 1 -- 172.21.15.31:0/1879706288 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e48155a50 con 0x7f5e48156530 2023-10-06T02:24:26.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.224+0000 7f5e4f59d700 1 -- 172.21.15.31:0/1879706288 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e481558b0 con 0x7f5e48156530 2023-10-06T02:24:26.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.225+0000 7f5e3e7fc700 1 -- 172.21.15.31:0/1879706288 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f5e40003040 con 0x7f5e48156530 2023-10-06T02:24:26.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.226+0000 7f5e3e7fc700 1 --2- 172.21.15.31:0/1879706288 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5e2c066fc0 0x7f5e2c069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:26.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.226+0000 7f5e3e7fc700 1 -- 172.21.15.31:0/1879706288 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f5e40085be0 con 0x7f5e48156530 2023-10-06T02:24:26.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.226+0000 7f5e4e59b700 1 --2- 172.21.15.31:0/1879706288 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5e2c066fc0 0x7f5e2c069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:26.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.227+0000 7f5e4e59b700 1 --2- 172.21.15.31:0/1879706288 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5e2c066fc0 0x7f5e2c069470 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5e44001120 tx=0x7f5e44009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:26.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.231+0000 7f5e3e7fc700 1 -- 172.21.15.31:0/1879706288 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5e481558b0 con 0x7f5e48156530 2023-10-06T02:24:26.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.463+0000 7f5e4f59d700 1 -- 172.21.15.31:0/1879706288 --> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "group_name": "group_0000000000615651", "target": ["mon-mgr", ""]}) v1 -- 0x7f5e481558b0 con 0x7f5e2c066fc0 2023-10-06T02:24:26.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.467+0000 7f5e3e7fc700 1 -- 172.21.15.31:0/1879706288 <== mgr.4107 v2:172.21.15.31:6832/85292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f5e481558b0 con 0x7f5e2c066fc0 2023-10-06T02:24:26.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.469+0000 7f5e4f59d700 1 -- 172.21.15.31:0/1879706288 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5e2c066fc0 msgr2=0x7f5e2c069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:26.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.469+0000 7f5e4f59d700 1 --2- 172.21.15.31:0/1879706288 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5e2c066fc0 0x7f5e2c069470 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5e44001120 tx=0x7f5e44009040 comp rx=0 tx=0).stop 2023-10-06T02:24:26.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.469+0000 7f5e4f59d700 1 -- 172.21.15.31:0/1879706288 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e48156530 msgr2=0x7f5e4806d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:26.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.469+0000 7f5e4f59d700 1 --2- 172.21.15.31:0/1879706288 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e48156530 0x7f5e4806d690 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f5e4000e210 tx=0x7f5e4001a000 comp rx=0 tx=0).stop 2023-10-06T02:24:26.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.469+0000 7f5e4f59d700 1 -- 172.21.15.31:0/1879706288 shutdown_connections 2023-10-06T02:24:26.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.469+0000 7f5e4f59d700 1 --2- 172.21.15.31:0/1879706288 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e48156530 0x7f5e4806d690 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:26.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.469+0000 7f5e4f59d700 1 --2- 172.21.15.31:0/1879706288 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5e2c066fc0 0x7f5e2c069470 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:26.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.469+0000 7f5e4f59d700 1 --2- 172.21.15.31:0/1879706288 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5e34004d50 0x7f5e48147030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:26.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.469+0000 7f5e4f59d700 1 --2- 172.21.15.31:0/1879706288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e480c9bc0 0x7f5e4816a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:26.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.469+0000 7f5e4f59d700 1 -- 172.21.15.31:0/1879706288 >> 172.21.15.31:0/1879706288 conn(0x7f5e480703a0 msgr2=0x7f5e48152db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:26.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.469+0000 7f5e4f59d700 1 -- 172.21.15.31:0/1879706288 shutdown_connections 2023-10-06T02:24:26.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:26.469+0000 7f5e4f59d700 1 -- 172.21.15.31:0/1879706288 wait complete. 2023-10-06T02:24:26.487 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T02:24:26.487 DEBUG:teuthology.orchestra.run.smithi031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T02:24:26.565 DEBUG:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T02:24:26.663 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T02:24:26.663 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T02:24:26.663 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:24:26.663 DEBUG:teuthology.orchestra.run.smithi031:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T02:24:26.764 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi031.front.sepia.ceph.com 2023-10-06T02:24:26.765 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:24:26.765 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T02:24:26.810 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-06T02:24:26.810 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:24:26.810 DEBUG:teuthology.orchestra.run.smithi062:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-06T02:24:26.900 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi062.front.sepia.ceph.com 2023-10-06T02:24:26.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:24:26.901 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T02:24:26.937 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:24:27.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.124+0000 7f2ede39d700 1 Processor -- start 2023-10-06T02:24:27.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.124+0000 7f2ede39d700 1 -- start start 2023-10-06T02:24:27.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.125+0000 7f2ede39d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2ed8159640 0x7f2ed806d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:27.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.125+0000 7f2ede39d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ed80c7420 con 0x7f2ed80caec0 2023-10-06T02:24:27.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.125+0000 7f2ede39d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ed806d5a0 con 0x7f2ed8159a10 2023-10-06T02:24:27.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.125+0000 7f2ede39d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ed806d820 con 0x7f2ed8159640 2023-10-06T02:24:27.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.125+0000 7f2eddb9c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f2ed80caec0 0x7f2ed80c78b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60412/0 (socket says 172.21.15.31:60412) 2023-10-06T02:24:27.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.126+0000 7f2eddb9c700 1 -- 172.21.15.31:0/71972697 learned_addr learned my addr 172.21.15.31:0/71972697 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:24:27.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.126+0000 7f2edd39b700 1 --2- 172.21.15.31:0/71972697 >> v2:172.21.15.62:3300/0 conn(0x7f2ed8159640 0x7f2ed806d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:27.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.126+0000 7f2ecffff700 1 -- 172.21.15.31:0/71972697 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3705829705 0 0) 0x7f2ed80c7420 con 0x7f2ed80caec0 2023-10-06T02:24:27.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.126+0000 7f2ecffff700 1 -- 172.21.15.31:0/71972697 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ec4003570 con 0x7f2ed80caec0 2023-10-06T02:24:27.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.126+0000 7f2ecffff700 1 -- 172.21.15.31:0/71972697 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4228797788 0 0) 0x7f2ed806d5a0 con 0x7f2ed8159a10 2023-10-06T02:24:27.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.126+0000 7f2ecffff700 1 -- 172.21.15.31:0/71972697 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ed80c7420 con 0x7f2ed8159a10 2023-10-06T02:24:27.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.126+0000 7f2edd39b700 1 -- 172.21.15.31:0/71972697 >> v1:172.21.15.62:6789/0 conn(0x7f2ed8159a10 legacy=0x7f2ed80ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:27.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.126+0000 7f2edd39b700 1 -- 172.21.15.31:0/71972697 >> v1:172.21.15.31:6789/0 conn(0x7f2ed80caec0 legacy=0x7f2ed80c78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:27.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.126+0000 7f2edd39b700 1 -- 172.21.15.31:0/71972697 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ed806db00 con 0x7f2ed8159640 2023-10-06T02:24:27.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.126+0000 7f2edd39b700 1 --2- 172.21.15.31:0/71972697 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ed8159640 0x7f2ed806d060 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4009910 tx=0x7f2ed40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af3f67039a0d5276 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:27.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.127+0000 7f2ecffff700 1 -- 172.21.15.31:0/71972697 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed4011630 con 0x7f2ed8159640 2023-10-06T02:24:27.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.127+0000 7f2ecffff700 1 -- 172.21.15.31:0/71972697 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ed4011790 con 0x7f2ed8159640 2023-10-06T02:24:27.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.127+0000 7f2ecffff700 1 -- 172.21.15.31:0/71972697 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed4011b30 con 0x7f2ed8159640 2023-10-06T02:24:27.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.127+0000 7f2ede39d700 1 -- 172.21.15.31:0/71972697 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ed8159640 msgr2=0x7f2ed806d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:27.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.127+0000 7f2ede39d700 1 --2- 172.21.15.31:0/71972697 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ed8159640 0x7f2ed806d060 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4009910 tx=0x7f2ed40091a0 comp rx=0 tx=0).stop 2023-10-06T02:24:27.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.127+0000 7f2ede39d700 1 -- 172.21.15.31:0/71972697 shutdown_connections 2023-10-06T02:24:27.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.127+0000 7f2ede39d700 1 --2- 172.21.15.31:0/71972697 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ed8159640 0x7f2ed806d060 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:27.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.127+0000 7f2ede39d700 1 -- 172.21.15.31:0/71972697 >> 172.21.15.31:0/71972697 conn(0x7f2ed806e720 msgr2=0x7f2ed806eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:27.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.127+0000 7f2ede39d700 1 -- 172.21.15.31:0/71972697 shutdown_connections 2023-10-06T02:24:27.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.127+0000 7f2ede39d700 1 -- 172.21.15.31:0/71972697 wait complete. 2023-10-06T02:24:27.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.129+0000 7f2ede39d700 1 Processor -- start 2023-10-06T02:24:27.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.129+0000 7f2ede39d700 1 -- start start 2023-10-06T02:24:27.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.130+0000 7f2ede39d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ed80caec0 0x7f2ed8154510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:27.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.130+0000 7f2ede39d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ed8159640 0x7f2ed80c3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:27.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.130+0000 7f2ede39d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2ed8159a10 0x7f2ed81574b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:27.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.130+0000 7f2ede39d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed80c50e0 con 0x7f2ed8159640 2023-10-06T02:24:27.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.130+0000 7f2ede39d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed80c44d0 con 0x7f2ed80caec0 2023-10-06T02:24:27.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.130+0000 7f2ede39d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed80c47b0 con 0x7f2ed8159a10 2023-10-06T02:24:27.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.130+0000 7f2edcb9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ed8159640 0x7f2ed80c3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:27.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.130+0000 7f2edcb9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ed8159640 0x7f2ed80c3a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46856/0 (socket says 172.21.15.31:46856) 2023-10-06T02:24:27.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.130+0000 7f2edcb9a700 1 -- 172.21.15.31:0/1324338666 learned_addr learned my addr 172.21.15.31:0/1324338666 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:27.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.130+0000 7f2edd39b700 1 --2- 172.21.15.31:0/1324338666 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ed80caec0 0x7f2ed8154510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:27.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.130+0000 7f2eddb9c700 1 --2- 172.21.15.31:0/1324338666 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2ed8159a10 0x7f2ed81574b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:27.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.131+0000 7f2edcb9a700 1 -- 172.21.15.31:0/1324338666 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2ed8159a10 msgr2=0x7f2ed81574b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:27.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.131+0000 7f2edcb9a700 1 --2- 172.21.15.31:0/1324338666 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2ed8159a10 0x7f2ed81574b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:27.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.131+0000 7f2edcb9a700 1 -- 172.21.15.31:0/1324338666 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ed80caec0 msgr2=0x7f2ed8154510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:27.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.131+0000 7f2edcb9a700 1 --2- 172.21.15.31:0/1324338666 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ed80caec0 0x7f2ed8154510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:27.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.131+0000 7f2edcb9a700 1 -- 172.21.15.31:0/1324338666 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ed4009420 con 0x7f2ed8159640 2023-10-06T02:24:27.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.131+0000 7f2edcb9a700 1 --2- 172.21.15.31:0/1324338666 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ed8159640 0x7f2ed80c3a80 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f2ec800c590 tx=0x7f2ec8007de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:27.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.131+0000 7f2ecd7fa700 1 -- 172.21.15.31:0/1324338666 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec8010510 con 0x7f2ed8159640 2023-10-06T02:24:27.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.131+0000 7f2ecd7fa700 1 -- 172.21.15.31:0/1324338666 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ec8010690 con 0x7f2ed8159640 2023-10-06T02:24:27.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.131+0000 7f2ecd7fa700 1 -- 172.21.15.31:0/1324338666 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec8010a50 con 0x7f2ed8159640 2023-10-06T02:24:27.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.131+0000 7f2ede39d700 1 -- 172.21.15.31:0/1324338666 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ed80c4a90 con 0x7f2ed8159640 2023-10-06T02:24:27.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.131+0000 7f2ede39d700 1 -- 172.21.15.31:0/1324338666 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ed815b9d0 con 0x7f2ed8159640 2023-10-06T02:24:27.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.133+0000 7f2ede39d700 1 -- 172.21.15.31:0/1324338666 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ed80c5290 con 0x7f2ed8159640 2023-10-06T02:24:27.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.134+0000 7f2ecd7fa700 1 -- 172.21.15.31:0/1324338666 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f2ec801b3b0 con 0x7f2ed8159640 2023-10-06T02:24:27.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.134+0000 7f2ecd7fa700 1 --2- 172.21.15.31:0/1324338666 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f2ebc066fc0 0x7f2ebc069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:27.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.135+0000 7f2ecd7fa700 1 -- 172.21.15.31:0/1324338666 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2ec8083d20 con 0x7f2ed8159640 2023-10-06T02:24:27.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.135+0000 7f2edd39b700 1 --2- 172.21.15.31:0/1324338666 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f2ebc066fc0 0x7f2ebc069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:27.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.135+0000 7f2edd39b700 1 --2- 172.21.15.31:0/1324338666 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f2ebc066fc0 0x7f2ebc069470 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2ed40093f0 tx=0x7f2ed4017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:27.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.139+0000 7f2ecd7fa700 1 -- 172.21.15.31:0/1324338666 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2ed80c5290 con 0x7f2ed8159640 2023-10-06T02:24:27.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.208+0000 7f2ecd7fa700 1 -- 172.21.15.31:0/1324338666 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f2ec8051210 con 0x7f2ed8159640 2023-10-06T02:24:27.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.387+0000 7f2ede39d700 1 -- 172.21.15.31:0/1324338666 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2ed80c5220 con 0x7f2ed8159640 2023-10-06T02:24:27.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.388+0000 7f2ecd7fa700 1 -- 172.21.15.31:0/1324338666 <== mon.0 v2:172.21.15.31:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4591 (secure 0 0 0) 0x7f2ec8050e40 con 0x7f2ed8159640 2023-10-06T02:24:27.388 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:24:27.388 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4700,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/384287452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":384287452},{"type":"v1","addr":"172.21.15.62:6834","nonce":384287452}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5037,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/693643908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":693643908},{"type":"v1","addr":"172.21.15.31:6836","nonce":693643908}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5040,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2599441423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2599441423},{"type":"v1","addr":"172.21.15.62:6835","nonce":2599441423}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:24:12.493929+0000","modified":"2023-10-06T02:24:16.026454+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5043},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5043":{"gid":5043,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.31:6837/2608294748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":2608294748},{"type":"v1","addr":"172.21.15.31:6837","nonce":2608294748}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T02:24:27.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.390+0000 7f2ede39d700 1 -- 172.21.15.31:0/1324338666 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f2ebc066fc0 msgr2=0x7f2ebc069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:27.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.390+0000 7f2ede39d700 1 --2- 172.21.15.31:0/1324338666 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f2ebc066fc0 0x7f2ebc069470 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2ed40093f0 tx=0x7f2ed4017000 comp rx=0 tx=0).stop 2023-10-06T02:24:27.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.390+0000 7f2ede39d700 1 -- 172.21.15.31:0/1324338666 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ed8159640 msgr2=0x7f2ed80c3a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:27.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.390+0000 7f2ede39d700 1 --2- 172.21.15.31:0/1324338666 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ed8159640 0x7f2ed80c3a80 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f2ec800c590 tx=0x7f2ec8007de0 comp rx=0 tx=0).stop 2023-10-06T02:24:27.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.390+0000 7f2ede39d700 1 -- 172.21.15.31:0/1324338666 shutdown_connections 2023-10-06T02:24:27.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.390+0000 7f2ede39d700 1 --2- 172.21.15.31:0/1324338666 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f2ebc066fc0 0x7f2ebc069470 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:27.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.390+0000 7f2ede39d700 1 --2- 172.21.15.31:0/1324338666 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2ed8159a10 0x7f2ed81574b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:27.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.390+0000 7f2ede39d700 1 --2- 172.21.15.31:0/1324338666 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ed80caec0 0x7f2ed8154510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:27.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.390+0000 7f2ede39d700 1 --2- 172.21.15.31:0/1324338666 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ed8159640 0x7f2ed80c3a80 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:27.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.390+0000 7f2ede39d700 1 -- 172.21.15.31:0/1324338666 >> 172.21.15.31:0/1324338666 conn(0x7f2ed806e720 msgr2=0x7f2ed81588d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:27.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.390+0000 7f2ede39d700 1 -- 172.21.15.31:0/1324338666 shutdown_connections 2023-10-06T02:24:27.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.390+0000 7f2ede39d700 1 -- 172.21.15.31:0/1324338666 wait complete. 2023-10-06T02:24:27.392 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 18 2023-10-06T02:24:27.408 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:24:27.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.595+0000 7f8b3bfff700 1 Processor -- start 2023-10-06T02:24:27.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.595+0000 7f8b3bfff700 1 -- start start 2023-10-06T02:24:27.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.596+0000 7f8b3bfff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8b3c153c30 0x7f8b3c15a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:27.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.596+0000 7f8b3bfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8b3c0cf110 0x7f8b3c15c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:27.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.596+0000 7f8b3bfff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b3c15afd0 con 0x7f8b3c15de50 2023-10-06T02:24:27.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.596+0000 7f8b3bfff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b3c15b2b0 con 0x7f8b3c0cf110 2023-10-06T02:24:27.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.596+0000 7f8b3bfff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b3c15b590 con 0x7f8b3c153c30 2023-10-06T02:24:27.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.597+0000 7f8b3a7fc700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f8b3c15de50 0x7f8b3c0cea00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60426/0 (socket says 172.21.15.31:60426) 2023-10-06T02:24:27.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.597+0000 7f8b3a7fc700 1 -- 172.21.15.31:0/286130298 learned_addr learned my addr 172.21.15.31:0/286130298 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:24:27.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.597+0000 7f8b3b7fe700 1 --2- 172.21.15.31:0/286130298 >> v2:172.21.15.31:3300/0 conn(0x7f8b3c0cf110 0x7f8b3c15c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:27.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.597+0000 7f8b3affd700 1 --2- 172.21.15.31:0/286130298 >> v2:172.21.15.62:3300/0 conn(0x7f8b3c153c30 0x7f8b3c15a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:27.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.597+0000 7f8b39ffb700 1 -- 172.21.15.31:0/286130298 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3093019393 0 0) 0x7f8b3c15afd0 con 0x7f8b3c15de50 2023-10-06T02:24:27.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.597+0000 7f8b39ffb700 1 -- 172.21.15.31:0/286130298 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b24003570 con 0x7f8b3c15de50 2023-10-06T02:24:27.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.597+0000 7f8b3b7fe700 1 -- 172.21.15.31:0/286130298 >> v2:172.21.15.62:3300/0 conn(0x7f8b3c153c30 msgr2=0x7f8b3c15a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:27.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.597+0000 7f8b3b7fe700 1 --2- 172.21.15.31:0/286130298 >> v2:172.21.15.62:3300/0 conn(0x7f8b3c153c30 0x7f8b3c15a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:27.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.597+0000 7f8b3b7fe700 1 -- 172.21.15.31:0/286130298 >> v1:172.21.15.31:6789/0 conn(0x7f8b3c15de50 legacy=0x7f8b3c0cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:27.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.597+0000 7f8b3b7fe700 1 -- 172.21.15.31:0/286130298 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b3c15b870 con 0x7f8b3c0cf110 2023-10-06T02:24:27.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.597+0000 7f8b3b7fe700 1 --2- 172.21.15.31:0/286130298 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b3c0cf110 0x7f8b3c15c070 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c06ead0 tx=0x7f8b34009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fd42599c2910ed1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:27.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.598+0000 7f8b39ffb700 1 -- 172.21.15.31:0/286130298 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b3400e070 con 0x7f8b3c0cf110 2023-10-06T02:24:27.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.598+0000 7f8b39ffb700 1 -- 172.21.15.31:0/286130298 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b3400bab0 con 0x7f8b3c0cf110 2023-10-06T02:24:27.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.598+0000 7f8b39ffb700 1 -- 172.21.15.31:0/286130298 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b3400be70 con 0x7f8b3c0cf110 2023-10-06T02:24:27.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.598+0000 7f8b3bfff700 1 -- 172.21.15.31:0/286130298 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b3c0cf110 msgr2=0x7f8b3c15c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:27.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.598+0000 7f8b3bfff700 1 --2- 172.21.15.31:0/286130298 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b3c0cf110 0x7f8b3c15c070 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c06ead0 tx=0x7f8b34009780 comp rx=0 tx=0).stop 2023-10-06T02:24:27.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.598+0000 7f8b3bfff700 1 -- 172.21.15.31:0/286130298 shutdown_connections 2023-10-06T02:24:27.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.598+0000 7f8b3bfff700 1 --2- 172.21.15.31:0/286130298 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b3c0cf110 0x7f8b3c15c070 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:27.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.598+0000 7f8b3bfff700 1 --2- 172.21.15.31:0/286130298 >> v2:172.21.15.62:3300/0 conn(0x7f8b3c153c30 0x7f8b3c15a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:27.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.598+0000 7f8b3bfff700 1 -- 172.21.15.31:0/286130298 >> 172.21.15.31:0/286130298 conn(0x7f8b3c0703a0 msgr2=0x7f8b3c0707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:27.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.598+0000 7f8b3bfff700 1 -- 172.21.15.31:0/286130298 shutdown_connections 2023-10-06T02:24:27.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.598+0000 7f8b3bfff700 1 -- 172.21.15.31:0/286130298 wait complete. 2023-10-06T02:24:27.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.600+0000 7f8b3bfff700 1 Processor -- start 2023-10-06T02:24:27.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.600+0000 7f8b3bfff700 1 -- start start 2023-10-06T02:24:27.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.601+0000 7f8b3bfff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8b3c0cf110 0x7f8b3c145e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:27.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.601+0000 7f8b3bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b3c153c30 0x7f8b3c16d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:27.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.601+0000 7f8b3bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b3c15de50 0x7f8b3c16d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:27.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.601+0000 7f8b3bfff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3c0b60e0 con 0x7f8b3c153c30 2023-10-06T02:24:27.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.601+0000 7f8b3bfff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3c0c87a0 con 0x7f8b3c15de50 2023-10-06T02:24:27.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.601+0000 7f8b3bfff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3c16dc50 con 0x7f8b3c0cf110 2023-10-06T02:24:27.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.601+0000 7f8b3a7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b3c153c30 0x7f8b3c16d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:27.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.601+0000 7f8b3a7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b3c153c30 0x7f8b3c16d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46880/0 (socket says 172.21.15.31:46880) 2023-10-06T02:24:27.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.601+0000 7f8b3a7fc700 1 -- 172.21.15.31:0/1023101892 learned_addr learned my addr 172.21.15.31:0/1023101892 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:27.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.601+0000 7f8b3a7fc700 1 -- 172.21.15.31:0/1023101892 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8b3c0cf110 msgr2=0x7f8b3c145e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:27.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.601+0000 7f8b3a7fc700 1 --2- 172.21.15.31:0/1023101892 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8b3c0cf110 0x7f8b3c145e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:27.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.601+0000 7f8b3b7fe700 1 --2- 172.21.15.31:0/1023101892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b3c15de50 0x7f8b3c16d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:27.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.601+0000 7f8b3a7fc700 1 -- 172.21.15.31:0/1023101892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b3c15de50 msgr2=0x7f8b3c16d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:27.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.602+0000 7f8b3a7fc700 1 --2- 172.21.15.31:0/1023101892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b3c15de50 0x7f8b3c16d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:27.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.602+0000 7f8b3a7fc700 1 -- 172.21.15.31:0/1023101892 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b34009a20 con 0x7f8b3c153c30 2023-10-06T02:24:27.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.602+0000 7f8b3b7fe700 1 --2- 172.21.15.31:0/1023101892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b3c15de50 0x7f8b3c16d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:24:27.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.602+0000 7f8b3a7fc700 1 --2- 172.21.15.31:0/1023101892 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b3c153c30 0x7f8b3c16d070 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f8b3000c590 tx=0x7f8b30007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:27.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.602+0000 7f8b22ffd700 1 -- 172.21.15.31:0/1023101892 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b30005b30 con 0x7f8b3c153c30 2023-10-06T02:24:27.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.602+0000 7f8b22ffd700 1 -- 172.21.15.31:0/1023101892 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b30005cb0 con 0x7f8b3c153c30 2023-10-06T02:24:27.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.602+0000 7f8b3bfff700 1 -- 172.21.15.31:0/1023101892 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b3c0c8a40 con 0x7f8b3c153c30 2023-10-06T02:24:27.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.602+0000 7f8b22ffd700 1 -- 172.21.15.31:0/1023101892 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b30007320 con 0x7f8b3c153c30 2023-10-06T02:24:27.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.602+0000 7f8b3bfff700 1 -- 172.21.15.31:0/1023101892 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b3c16df30 con 0x7f8b3c153c30 2023-10-06T02:24:27.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.603+0000 7f8b3bfff700 1 -- 172.21.15.31:0/1023101892 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b3c04c3c0 con 0x7f8b3c153c30 2023-10-06T02:24:27.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.604+0000 7f8b22ffd700 1 -- 172.21.15.31:0/1023101892 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f8b3001b3b0 con 0x7f8b3c153c30 2023-10-06T02:24:27.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.605+0000 7f8b22ffd700 1 --2- 172.21.15.31:0/1023101892 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f8b280670e0 0x7f8b28069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:27.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.605+0000 7f8b22ffd700 1 -- 172.21.15.31:0/1023101892 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8b30083fe0 con 0x7f8b3c153c30 2023-10-06T02:24:27.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.605+0000 7f8b3affd700 1 --2- 172.21.15.31:0/1023101892 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f8b280670e0 0x7f8b28069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:27.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.606+0000 7f8b3affd700 1 --2- 172.21.15.31:0/1023101892 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f8b280670e0 0x7f8b28069590 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c009bb0 tx=0x7f8b2c0091a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:27.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.610+0000 7f8b22ffd700 1 -- 172.21.15.31:0/1023101892 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8b30050df0 con 0x7f8b3c153c30 2023-10-06T02:24:27.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.814+0000 7f8b3bfff700 1 -- 172.21.15.31:0/1023101892 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8b3c15bb50 con 0x7f8b3c153c30 2023-10-06T02:24:27.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.815+0000 7f8b22ffd700 1 -- 172.21.15.31:0/1023101892 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+16818 (secure 0 0 0) 0x7f8b30017020 con 0x7f8b3c153c30 2023-10-06T02:24:27.815 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:24:27.815 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":36,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:24:14.007126+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:24:10.026011+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-06T02:24:11.471970+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:24:27.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.817+0000 7f8b3bfff700 1 -- 172.21.15.31:0/1023101892 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f8b280670e0 msgr2=0x7f8b28069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:27.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.817+0000 7f8b3bfff700 1 --2- 172.21.15.31:0/1023101892 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f8b280670e0 0x7f8b28069590 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c009bb0 tx=0x7f8b2c0091a0 comp rx=0 tx=0).stop 2023-10-06T02:24:27.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.817+0000 7f8b3bfff700 1 -- 172.21.15.31:0/1023101892 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b3c153c30 msgr2=0x7f8b3c16d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:27.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.817+0000 7f8b3bfff700 1 --2- 172.21.15.31:0/1023101892 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b3c153c30 0x7f8b3c16d070 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f8b3000c590 tx=0x7f8b30007e00 comp rx=0 tx=0).stop 2023-10-06T02:24:27.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.817+0000 7f8b3bfff700 1 -- 172.21.15.31:0/1023101892 shutdown_connections 2023-10-06T02:24:27.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.817+0000 7f8b3bfff700 1 --2- 172.21.15.31:0/1023101892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b3c15de50 0x7f8b3c16d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:27.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.817+0000 7f8b3bfff700 1 --2- 172.21.15.31:0/1023101892 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f8b280670e0 0x7f8b28069590 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:27.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.817+0000 7f8b3bfff700 1 --2- 172.21.15.31:0/1023101892 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8b3c0cf110 0x7f8b3c145e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:27.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.817+0000 7f8b3bfff700 1 --2- 172.21.15.31:0/1023101892 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b3c153c30 0x7f8b3c16d070 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:27.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.817+0000 7f8b3bfff700 1 -- 172.21.15.31:0/1023101892 >> 172.21.15.31:0/1023101892 conn(0x7f8b3c0703a0 msgr2=0x7f8b3c0c57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:27.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.817+0000 7f8b3bfff700 1 -- 172.21.15.31:0/1023101892 shutdown_connections 2023-10-06T02:24:27.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:27.817+0000 7f8b3bfff700 1 -- 172.21.15.31:0/1023101892 wait complete. 2023-10-06T02:24:27.836 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T02:22:53.183425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T02:23:00.046967+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T02:24:10.026011+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T02:24:11.471970+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T02:24:27.836 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T02:24:28.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.028+0000 7efc33fff700 1 Processor -- start 2023-10-06T02:24:28.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.028+0000 7efc33fff700 1 -- start start 2023-10-06T02:24:28.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.029+0000 7efc33fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7efc3415dfd0 0x7efc340aedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:28.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.029+0000 7efc33fff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc340acc00 con 0x7efc34159550 2023-10-06T02:24:28.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.029+0000 7efc33fff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc340abff0 con 0x7efc34159920 2023-10-06T02:24:28.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.029+0000 7efc33fff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc340ac2d0 con 0x7efc3415dfd0 2023-10-06T02:24:28.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.030+0000 7efc2bfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7efc3415dfd0 0x7efc340aedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:28.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.030+0000 7efc32ffd700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7efc34159550 0x7efc340ceac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58036/0 (socket says 172.21.15.31:58036) 2023-10-06T02:24:28.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.030+0000 7efc32ffd700 1 -- 172.21.15.31:0/2033508254 learned_addr learned my addr 172.21.15.31:0/2033508254 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:24:28.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.030+0000 7efc327fc700 1 -- 172.21.15.31:0/2033508254 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3419222527 0 0) 0x7efc340acc00 con 0x7efc34159550 2023-10-06T02:24:28.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.030+0000 7efc327fc700 1 -- 172.21.15.31:0/2033508254 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc20003570 con 0x7efc34159550 2023-10-06T02:24:28.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.030+0000 7efc2bfff700 1 -- 172.21.15.31:0/2033508254 >> v1:172.21.15.62:6789/0 conn(0x7efc34159920 legacy=0x7efc340ad090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:28.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.030+0000 7efc2bfff700 1 -- 172.21.15.31:0/2033508254 >> v1:172.21.15.31:6789/0 conn(0x7efc34159550 legacy=0x7efc340ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:28.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.030+0000 7efc2bfff700 1 -- 172.21.15.31:0/2033508254 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc340ac5b0 con 0x7efc3415dfd0 2023-10-06T02:24:28.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.031+0000 7efc2bfff700 1 --2- 172.21.15.31:0/2033508254 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc3415dfd0 0x7efc340aedb0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7efc1c0098e0 tx=0x7efc1c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=915bd253812de748 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:28.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.031+0000 7efc327fc700 1 -- 172.21.15.31:0/2033508254 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc1c00b640 con 0x7efc3415dfd0 2023-10-06T02:24:28.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.031+0000 7efc327fc700 1 -- 172.21.15.31:0/2033508254 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc1c00b7a0 con 0x7efc3415dfd0 2023-10-06T02:24:28.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.031+0000 7efc327fc700 1 -- 172.21.15.31:0/2033508254 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc1c00bb40 con 0x7efc3415dfd0 2023-10-06T02:24:28.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.031+0000 7efc33fff700 1 -- 172.21.15.31:0/2033508254 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc3415dfd0 msgr2=0x7efc340aedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:28.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.031+0000 7efc33fff700 1 --2- 172.21.15.31:0/2033508254 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc3415dfd0 0x7efc340aedb0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7efc1c0098e0 tx=0x7efc1c0091a0 comp rx=0 tx=0).stop 2023-10-06T02:24:28.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.031+0000 7efc33fff700 1 -- 172.21.15.31:0/2033508254 shutdown_connections 2023-10-06T02:24:28.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.031+0000 7efc33fff700 1 --2- 172.21.15.31:0/2033508254 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc3415dfd0 0x7efc340aedb0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:28.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.031+0000 7efc33fff700 1 -- 172.21.15.31:0/2033508254 >> 172.21.15.31:0/2033508254 conn(0x7efc3406ce80 msgr2=0x7efc340ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:28.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.032+0000 7efc33fff700 1 -- 172.21.15.31:0/2033508254 shutdown_connections 2023-10-06T02:24:28.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.032+0000 7efc33fff700 1 -- 172.21.15.31:0/2033508254 wait complete. 2023-10-06T02:24:28.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.033+0000 7efc33fff700 1 Processor -- start 2023-10-06T02:24:28.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.033+0000 7efc33fff700 1 -- start start 2023-10-06T02:24:28.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.034+0000 7efc33fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efc34159550 0x7efc3416b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:28.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.034+0000 7efc33fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc34159920 0x7efc34150f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:28.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.034+0000 7efc33fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc3415dfd0 0x7efc340b39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:28.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.034+0000 7efc33fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc34153e10 con 0x7efc3415dfd0 2023-10-06T02:24:28.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.034+0000 7efc33fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc341514c0 con 0x7efc34159920 2023-10-06T02:24:28.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.034+0000 7efc33fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc34153200 con 0x7efc34159550 2023-10-06T02:24:28.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.034+0000 7efc337fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc3415dfd0 0x7efc340b39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:28.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.034+0000 7efc337fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc3415dfd0 0x7efc340b39d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40054/0 (socket says 172.21.15.31:40054) 2023-10-06T02:24:28.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.034+0000 7efc337fe700 1 -- 172.21.15.31:0/2444418195 learned_addr learned my addr 172.21.15.31:0/2444418195 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:28.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.034+0000 7efc2bfff700 1 --2- 172.21.15.31:0/2444418195 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc34159920 0x7efc34150f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:28.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.034+0000 7efc32ffd700 1 --2- 172.21.15.31:0/2444418195 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efc34159550 0x7efc3416b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:28.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.035+0000 7efc337fe700 1 -- 172.21.15.31:0/2444418195 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efc34159550 msgr2=0x7efc3416b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:28.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.035+0000 7efc337fe700 1 --2- 172.21.15.31:0/2444418195 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efc34159550 0x7efc3416b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:28.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.035+0000 7efc337fe700 1 -- 172.21.15.31:0/2444418195 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc34159920 msgr2=0x7efc34150f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:28.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.035+0000 7efc337fe700 1 --2- 172.21.15.31:0/2444418195 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc34159920 0x7efc34150f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:28.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.035+0000 7efc337fe700 1 -- 172.21.15.31:0/2444418195 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc1c009420 con 0x7efc3415dfd0 2023-10-06T02:24:28.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.035+0000 7efc32ffd700 1 --2- 172.21.15.31:0/2444418195 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efc34159550 0x7efc3416b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:24:28.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.035+0000 7efc337fe700 1 --2- 172.21.15.31:0/2444418195 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc3415dfd0 0x7efc340b39d0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7efc2c00c8e0 tx=0x7efc2c001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:28.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.035+0000 7efc2affd700 1 -- 172.21.15.31:0/2444418195 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc2c012630 con 0x7efc3415dfd0 2023-10-06T02:24:28.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.035+0000 7efc2affd700 1 -- 172.21.15.31:0/2444418195 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc2c0127b0 con 0x7efc3415dfd0 2023-10-06T02:24:28.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.035+0000 7efc2affd700 1 -- 172.21.15.31:0/2444418195 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc2c012b70 con 0x7efc3415dfd0 2023-10-06T02:24:28.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.035+0000 7efc33fff700 1 -- 172.21.15.31:0/2444418195 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc341534e0 con 0x7efc3415dfd0 2023-10-06T02:24:28.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.035+0000 7efc33fff700 1 -- 172.21.15.31:0/2444418195 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc34153aa0 con 0x7efc3415dfd0 2023-10-06T02:24:28.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.036+0000 7efc33fff700 1 -- 172.21.15.31:0/2444418195 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc34153f80 con 0x7efc3415dfd0 2023-10-06T02:24:28.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.037+0000 7efc2affd700 1 -- 172.21.15.31:0/2444418195 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7efc2c003030 con 0x7efc3415dfd0 2023-10-06T02:24:28.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.038+0000 7efc2affd700 1 --2- 172.21.15.31:0/2444418195 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7efc140670e0 0x7efc14069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:28.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.038+0000 7efc2affd700 1 -- 172.21.15.31:0/2444418195 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6058+0+0 (secure 0 0 0) 0x7efc2c084a00 con 0x7efc3415dfd0 2023-10-06T02:24:28.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.038+0000 7efc32ffd700 1 --2- 172.21.15.31:0/2444418195 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7efc140670e0 0x7efc14069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:28.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.039+0000 7efc32ffd700 1 --2- 172.21.15.31:0/2444418195 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7efc140670e0 0x7efc14069590 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7efc240058d0 tx=0x7efc24009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:28.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.042+0000 7efc2affd700 1 -- 172.21.15.31:0/2444418195 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efc2c0517f0 con 0x7efc3415dfd0 2023-10-06T02:24:28.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.244+0000 7efc33fff700 1 -- 172.21.15.31:0/2444418195 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efc3404c3c0 con 0x7efc3415dfd0 2023-10-06T02:24:28.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.244+0000 7efc2affd700 1 -- 172.21.15.31:0/2444418195 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) v1 ==== 93+0+10 (secure 0 0 0) 0x7efc2c054e10 con 0x7efc3415dfd0 2023-10-06T02:24:28.245 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-10-06T02:24:28.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.246+0000 7efc33fff700 1 -- 172.21.15.31:0/2444418195 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7efc140670e0 msgr2=0x7efc14069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:28.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.246+0000 7efc33fff700 1 --2- 172.21.15.31:0/2444418195 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7efc140670e0 0x7efc14069590 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7efc240058d0 tx=0x7efc24009040 comp rx=0 tx=0).stop 2023-10-06T02:24:28.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.246+0000 7efc33fff700 1 -- 172.21.15.31:0/2444418195 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc3415dfd0 msgr2=0x7efc340b39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:28.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.247+0000 7efc33fff700 1 --2- 172.21.15.31:0/2444418195 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc3415dfd0 0x7efc340b39d0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7efc2c00c8e0 tx=0x7efc2c001710 comp rx=0 tx=0).stop 2023-10-06T02:24:28.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.247+0000 7efc33fff700 1 -- 172.21.15.31:0/2444418195 shutdown_connections 2023-10-06T02:24:28.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.247+0000 7efc33fff700 1 --2- 172.21.15.31:0/2444418195 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc3415dfd0 0x7efc340b39d0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:28.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.247+0000 7efc33fff700 1 --2- 172.21.15.31:0/2444418195 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7efc140670e0 0x7efc14069590 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:28.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.247+0000 7efc33fff700 1 --2- 172.21.15.31:0/2444418195 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efc34159550 0x7efc3416b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:28.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.247+0000 7efc33fff700 1 --2- 172.21.15.31:0/2444418195 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc34159920 0x7efc34150f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:28.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.247+0000 7efc33fff700 1 -- 172.21.15.31:0/2444418195 >> 172.21.15.31:0/2444418195 conn(0x7efc3406ce80 msgr2=0x7efc340afb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:28.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.247+0000 7efc33fff700 1 -- 172.21.15.31:0/2444418195 shutdown_connections 2023-10-06T02:24:28.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.247+0000 7efc33fff700 1 -- 172.21.15.31:0/2444418195 wait complete. 2023-10-06T02:24:28.264 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T02:24:28.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.453+0000 7f991519d700 1 Processor -- start 2023-10-06T02:24:28.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.453+0000 7f991519d700 1 -- start start 2023-10-06T02:24:28.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.454+0000 7f991519d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f99101595d0 0x7f99100b0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:28.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.454+0000 7f991519d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f99100b0110 0x7f99100b2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:28.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.454+0000 7f991519d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f991006d030 con 0x7f99100b2bc0 2023-10-06T02:24:28.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.455+0000 7f991519d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f991006c740 con 0x7f99100b0110 2023-10-06T02:24:28.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.455+0000 7f991519d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f991006c9e0 con 0x7f99101595d0 2023-10-06T02:24:28.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.455+0000 7f990ffff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f99101595d0 0x7f99100b0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:28.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.455+0000 7f990ffff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f99101595d0 0x7f99100b0d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:36514/0 (socket says 172.21.15.31:36514) 2023-10-06T02:24:28.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.455+0000 7f990ffff700 1 -- 172.21.15.31:0/1531578407 learned_addr learned my addr 172.21.15.31:0/1531578407 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:28.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.455+0000 7f990f7fe700 1 --2- 172.21.15.31:0/1531578407 >> v2:172.21.15.62:3300/0 conn(0x7f99100b0110 0x7f99100b2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:28.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.455+0000 7f991499c700 1 --1- 172.21.15.31:0/1531578407 >> v1:172.21.15.62:6789/0 conn(0x7f99100b2bc0 0x7f991006ac40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:56892/0 (socket says 172.21.15.31:56892) 2023-10-06T02:24:28.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.456+0000 7f990effd700 1 -- 172.21.15.31:0/1531578407 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4070864434 0 0) 0x7f991006d030 con 0x7f99100b2bc0 2023-10-06T02:24:28.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.456+0000 7f990effd700 1 -- 172.21.15.31:0/1531578407 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98f8003570 con 0x7f99100b2bc0 2023-10-06T02:24:28.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.456+0000 7f990ffff700 1 -- 172.21.15.31:0/1531578407 >> v2:172.21.15.62:3300/0 conn(0x7f99100b0110 msgr2=0x7f99100b2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:28.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.456+0000 7f990ffff700 1 --2- 172.21.15.31:0/1531578407 >> v2:172.21.15.62:3300/0 conn(0x7f99100b0110 0x7f99100b2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:28.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.456+0000 7f990ffff700 1 -- 172.21.15.31:0/1531578407 >> v1:172.21.15.62:6789/0 conn(0x7f99100b2bc0 legacy=0x7f991006ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:28.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.456+0000 7f990ffff700 1 -- 172.21.15.31:0/1531578407 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f991006ccc0 con 0x7f99101595d0 2023-10-06T02:24:28.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.456+0000 7f990f7fe700 1 --2- 172.21.15.31:0/1531578407 >> v2:172.21.15.62:3300/0 conn(0x7f99100b0110 0x7f99100b2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:24:28.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.456+0000 7f990ffff700 1 --2- 172.21.15.31:0/1531578407 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f99101595d0 0x7f99100b0d40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f99000099b0 tx=0x7f9900009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=529483035dfa04d1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:28.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.456+0000 7f990effd700 1 -- 172.21.15.31:0/1531578407 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990000b680 con 0x7f99101595d0 2023-10-06T02:24:28.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.457+0000 7f990effd700 1 -- 172.21.15.31:0/1531578407 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f990000b7e0 con 0x7f99101595d0 2023-10-06T02:24:28.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.457+0000 7f990effd700 1 -- 172.21.15.31:0/1531578407 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990000bb80 con 0x7f99101595d0 2023-10-06T02:24:28.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.457+0000 7f991519d700 1 -- 172.21.15.31:0/1531578407 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f99101595d0 msgr2=0x7f99100b0d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:28.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.457+0000 7f991519d700 1 --2- 172.21.15.31:0/1531578407 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f99101595d0 0x7f99100b0d40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f99000099b0 tx=0x7f9900009250 comp rx=0 tx=0).stop 2023-10-06T02:24:28.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.457+0000 7f991519d700 1 -- 172.21.15.31:0/1531578407 shutdown_connections 2023-10-06T02:24:28.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.457+0000 7f991519d700 1 --2- 172.21.15.31:0/1531578407 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f99101595d0 0x7f99100b0d40 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:28.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.457+0000 7f991519d700 1 --2- 172.21.15.31:0/1531578407 >> v2:172.21.15.62:3300/0 conn(0x7f99100b0110 0x7f99100b2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:28.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.457+0000 7f991519d700 1 -- 172.21.15.31:0/1531578407 >> 172.21.15.31:0/1531578407 conn(0x7f99100675a0 msgr2=0x7f99100679a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:28.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.457+0000 7f991519d700 1 -- 172.21.15.31:0/1531578407 shutdown_connections 2023-10-06T02:24:28.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.457+0000 7f991519d700 1 -- 172.21.15.31:0/1531578407 wait complete. 2023-10-06T02:24:28.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.459+0000 7f991519d700 1 Processor -- start 2023-10-06T02:24:28.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.459+0000 7f991519d700 1 -- start start 2023-10-06T02:24:28.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.460+0000 7f991519d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f99100b0110 0x7f9910143b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:28.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.460+0000 7f991519d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99100b2bc0 0x7f991016cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:28.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.460+0000 7f991519d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99101595d0 0x7f99100aa3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:28.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.460+0000 7f991519d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99100cabb0 con 0x7f99100b2bc0 2023-10-06T02:24:28.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.460+0000 7f991519d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99100cb170 con 0x7f99101595d0 2023-10-06T02:24:28.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.460+0000 7f991519d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f991016d980 con 0x7f99100b0110 2023-10-06T02:24:28.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.460+0000 7f990f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99100b2bc0 0x7f991016cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:28.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.460+0000 7f990f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99100b2bc0 0x7f991016cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40062/0 (socket says 172.21.15.31:40062) 2023-10-06T02:24:28.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.460+0000 7f990f7fe700 1 -- 172.21.15.31:0/904895424 learned_addr learned my addr 172.21.15.31:0/904895424 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:28.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.460+0000 7f990ffff700 1 --2- 172.21.15.31:0/904895424 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f99100b0110 0x7f9910143b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:28.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.460+0000 7f990f7fe700 1 -- 172.21.15.31:0/904895424 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f99100b0110 msgr2=0x7f9910143b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:28.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.460+0000 7f990f7fe700 1 --2- 172.21.15.31:0/904895424 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f99100b0110 0x7f9910143b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:28.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.460+0000 7f991499c700 1 --2- 172.21.15.31:0/904895424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99101595d0 0x7f99100aa3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:28.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.460+0000 7f990f7fe700 1 -- 172.21.15.31:0/904895424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99101595d0 msgr2=0x7f99100aa3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:28.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.461+0000 7f990f7fe700 1 --2- 172.21.15.31:0/904895424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99101595d0 0x7f99100aa3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:28.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.461+0000 7f990f7fe700 1 -- 172.21.15.31:0/904895424 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9904009420 con 0x7f99100b2bc0 2023-10-06T02:24:28.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.461+0000 7f990ffff700 1 --2- 172.21.15.31:0/904895424 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f99100b0110 0x7f9910143b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:24:28.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.461+0000 7f991499c700 1 --2- 172.21.15.31:0/904895424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99101595d0 0x7f99100aa3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:24:28.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.461+0000 7f990f7fe700 1 --2- 172.21.15.31:0/904895424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99100b2bc0 0x7f991016cec0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f990400ec80 tx=0x7f99040091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:28.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.461+0000 7f98f7fff700 1 -- 172.21.15.31:0/904895424 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990400c9d0 con 0x7f99100b2bc0 2023-10-06T02:24:28.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.461+0000 7f98f7fff700 1 -- 172.21.15.31:0/904895424 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f990400cb30 con 0x7f99100b2bc0 2023-10-06T02:24:28.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.461+0000 7f98f7fff700 1 -- 172.21.15.31:0/904895424 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990400ced0 con 0x7f99100b2bc0 2023-10-06T02:24:28.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.461+0000 7f991519d700 1 -- 172.21.15.31:0/904895424 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99000094f0 con 0x7f99100b2bc0 2023-10-06T02:24:28.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.461+0000 7f991519d700 1 -- 172.21.15.31:0/904895424 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99100cae00 con 0x7f99100b2bc0 2023-10-06T02:24:28.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.462+0000 7f991519d700 1 -- 172.21.15.31:0/904895424 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99100cb2e0 con 0x7f99100b2bc0 2023-10-06T02:24:28.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.463+0000 7f98f7fff700 1 -- 172.21.15.31:0/904895424 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f9904013680 con 0x7f99100b2bc0 2023-10-06T02:24:28.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.465+0000 7f98f7fff700 1 --2- 172.21.15.31:0/904895424 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f98fc0670e0 0x7f98fc069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:28.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.465+0000 7f98f7fff700 1 -- 172.21.15.31:0/904895424 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9904084de0 con 0x7f99100b2bc0 2023-10-06T02:24:28.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.465+0000 7f990ffff700 1 --2- 172.21.15.31:0/904895424 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f98fc0670e0 0x7f98fc069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:28.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.465+0000 7f990ffff700 1 --2- 172.21.15.31:0/904895424 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f98fc0670e0 0x7f98fc069590 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f99000099b0 tx=0x7f9900017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:28.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.469+0000 7f98f7fff700 1 -- 172.21.15.31:0/904895424 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9904051bd0 con 0x7f99100b2bc0 2023-10-06T02:24:28.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.673+0000 7f991519d700 1 -- 172.21.15.31:0/904895424 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f991004c3c0 con 0x7f99100b2bc0 2023-10-06T02:24:28.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.673+0000 7f98f7fff700 1 -- 172.21.15.31:0/904895424 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v36) v1 ==== 92+0+10 (secure 0 0 0) 0x7f99040551f0 con 0x7f99100b2bc0 2023-10-06T02:24:28.674 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 8 2023-10-06T02:24:28.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.676+0000 7f991519d700 1 -- 172.21.15.31:0/904895424 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f98fc0670e0 msgr2=0x7f98fc069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:28.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.676+0000 7f991519d700 1 --2- 172.21.15.31:0/904895424 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f98fc0670e0 0x7f98fc069590 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f99000099b0 tx=0x7f9900017000 comp rx=0 tx=0).stop 2023-10-06T02:24:28.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.676+0000 7f991519d700 1 -- 172.21.15.31:0/904895424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99100b2bc0 msgr2=0x7f991016cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:28.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.676+0000 7f991519d700 1 --2- 172.21.15.31:0/904895424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99100b2bc0 0x7f991016cec0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f990400ec80 tx=0x7f99040091a0 comp rx=0 tx=0).stop 2023-10-06T02:24:28.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.676+0000 7f991519d700 1 -- 172.21.15.31:0/904895424 shutdown_connections 2023-10-06T02:24:28.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.676+0000 7f991519d700 1 --2- 172.21.15.31:0/904895424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99101595d0 0x7f99100aa3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:28.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.676+0000 7f991519d700 1 --2- 172.21.15.31:0/904895424 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f98fc0670e0 0x7f98fc069590 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:28.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.676+0000 7f991519d700 1 --2- 172.21.15.31:0/904895424 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f99100b0110 0x7f9910143b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:28.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.676+0000 7f991519d700 1 --2- 172.21.15.31:0/904895424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99100b2bc0 0x7f991016cec0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:28.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.676+0000 7f991519d700 1 -- 172.21.15.31:0/904895424 >> 172.21.15.31:0/904895424 conn(0x7f99100675a0 msgr2=0x7f99101599a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:28.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.676+0000 7f991519d700 1 -- 172.21.15.31:0/904895424 shutdown_connections 2023-10-06T02:24:28.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.676+0000 7f991519d700 1 -- 172.21.15.31:0/904895424 wait complete. 2023-10-06T02:24:28.694 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T02:24:28.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.884+0000 7ff45ffff700 1 Processor -- start 2023-10-06T02:24:28.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.884+0000 7ff45ffff700 1 -- start start 2023-10-06T02:24:28.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.886+0000 7ff45ffff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff4600c7be0 0x7ff46006abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:28.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.886+0000 7ff45ffff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff46006b290 con 0x7ff4600c7fb0 2023-10-06T02:24:28.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.886+0000 7ff45ffff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff46006b570 con 0x7ff46013f780 2023-10-06T02:24:28.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.886+0000 7ff45ffff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff46006bf10 con 0x7ff4600c7be0 2023-10-06T02:24:28.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.886+0000 7ff45e7fc700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7ff4600c7fb0 0x7ff46015a990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58046/0 (socket says 172.21.15.31:58046) 2023-10-06T02:24:28.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.886+0000 7ff45e7fc700 1 -- 172.21.15.31:0/3863325949 learned_addr learned my addr 172.21.15.31:0/3863325949 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:24:28.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.886+0000 7ff45effd700 1 --2- 172.21.15.31:0/3863325949 >> v2:172.21.15.62:3300/0 conn(0x7ff4600c7be0 0x7ff46006abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:28.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.887+0000 7ff45dffb700 1 -- 172.21.15.31:0/3863325949 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1392416974 0 0) 0x7ff46006b290 con 0x7ff4600c7fb0 2023-10-06T02:24:28.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.887+0000 7ff45dffb700 1 -- 172.21.15.31:0/3863325949 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff448003570 con 0x7ff4600c7fb0 2023-10-06T02:24:28.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.887+0000 7ff45dffb700 1 -- 172.21.15.31:0/3863325949 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 185069368 0 0) 0x7ff448003570 con 0x7ff4600c7fb0 2023-10-06T02:24:28.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.887+0000 7ff45dffb700 1 -- 172.21.15.31:0/3863325949 >> v2:172.21.15.62:3300/0 conn(0x7ff4600c7be0 msgr2=0x7ff46006abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:28.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.887+0000 7ff45dffb700 1 --2- 172.21.15.31:0/3863325949 >> v2:172.21.15.62:3300/0 conn(0x7ff4600c7be0 0x7ff46006abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:28.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.887+0000 7ff45dffb700 1 -- 172.21.15.31:0/3863325949 >> v1:172.21.15.62:6789/0 conn(0x7ff46013f780 legacy=0x7ff46013fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:28.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.887+0000 7ff45dffb700 1 -- 172.21.15.31:0/3863325949 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff46006b850 con 0x7ff4600c7fb0 2023-10-06T02:24:28.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.887+0000 7ff45dffb700 1 -- 172.21.15.31:0/3863325949 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7ff454001470 con 0x7ff4600c7fb0 2023-10-06T02:24:28.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.887+0000 7ff45dffb700 1 -- 172.21.15.31:0/3863325949 >> v1:172.21.15.31:6789/0 conn(0x7ff4600c7fb0 legacy=0x7ff46015a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:28.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.887+0000 7ff45dffb700 1 --2- 172.21.15.31:0/3863325949 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff448003a60 0x7ff448003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:28.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.887+0000 7ff45dffb700 1 --2- 172.21.15.31:0/3863325949 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff448004810 0x7ff448004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:28.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.887+0000 7ff45dffb700 1 --2- 172.21.15.31:0/3863325949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4480051c0 0x7ff44800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:28.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.887+0000 7ff45dffb700 1 -- 172.21.15.31:0/3863325949 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff46006bf10 con 0x7ff448003a60 2023-10-06T02:24:28.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.887+0000 7ff45dffb700 1 -- 172.21.15.31:0/3863325949 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff44800c0b0 con 0x7ff4480051c0 2023-10-06T02:24:28.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.887+0000 7ff45dffb700 1 -- 172.21.15.31:0/3863325949 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff44800c270 con 0x7ff448004810 2023-10-06T02:24:28.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.887+0000 7ff45effd700 1 --2- 172.21.15.31:0/3863325949 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff448003a60 0x7ff448003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:28.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.888+0000 7ff45e7fc700 1 --2- 172.21.15.31:0/3863325949 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff448004810 0x7ff448004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:28.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.888+0000 7ff45f7fe700 1 --2- 172.21.15.31:0/3863325949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4480051c0 0x7ff44800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:28.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.888+0000 7ff45effd700 1 -- 172.21.15.31:0/3863325949 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff448004810 msgr2=0x7ff448004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:28.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.888+0000 7ff45effd700 1 --2- 172.21.15.31:0/3863325949 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff448004810 0x7ff448004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:28.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.888+0000 7ff45effd700 1 -- 172.21.15.31:0/3863325949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4480051c0 msgr2=0x7ff44800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:28.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.888+0000 7ff45effd700 1 --2- 172.21.15.31:0/3863325949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4480051c0 0x7ff44800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:28.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.888+0000 7ff45effd700 1 -- 172.21.15.31:0/3863325949 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff448003570 con 0x7ff448003a60 2023-10-06T02:24:28.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.888+0000 7ff45f7fe700 1 -- 172.21.15.31:0/3863325949 reap_dead start 2023-10-06T02:24:28.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.888+0000 7ff45effd700 1 --2- 172.21.15.31:0/3863325949 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff448003a60 0x7ff448003e30 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff450009c00 tx=0x7ff4500091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dda6706e3da87f4a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:28.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.888+0000 7ff45dffb700 1 -- 172.21.15.31:0/3863325949 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff45000e710 con 0x7ff448003a60 2023-10-06T02:24:28.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.888+0000 7ff45dffb700 1 -- 172.21.15.31:0/3863325949 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff45000e870 con 0x7ff448003a60 2023-10-06T02:24:28.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.888+0000 7ff45dffb700 1 -- 172.21.15.31:0/3863325949 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff45000ec30 con 0x7ff448003a60 2023-10-06T02:24:28.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.889+0000 7ff45ffff700 1 -- 172.21.15.31:0/3863325949 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff448003a60 msgr2=0x7ff448003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:28.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.889+0000 7ff45ffff700 1 --2- 172.21.15.31:0/3863325949 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff448003a60 0x7ff448003e30 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff450009c00 tx=0x7ff4500091a0 comp rx=0 tx=0).stop 2023-10-06T02:24:28.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.889+0000 7ff45ffff700 1 -- 172.21.15.31:0/3863325949 shutdown_connections 2023-10-06T02:24:28.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.889+0000 7ff45ffff700 1 --2- 172.21.15.31:0/3863325949 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff448003a60 0x7ff448003e30 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:28.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.889+0000 7ff45ffff700 1 -- 172.21.15.31:0/3863325949 >> 172.21.15.31:0/3863325949 conn(0x7ff46006d110 msgr2=0x7ff4600cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:28.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.889+0000 7ff45ffff700 1 -- 172.21.15.31:0/3863325949 shutdown_connections 2023-10-06T02:24:28.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.889+0000 7ff45ffff700 1 -- 172.21.15.31:0/3863325949 wait complete. 2023-10-06T02:24:28.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.890+0000 7ff45ffff700 1 Processor -- start 2023-10-06T02:24:28.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.891+0000 7ff45ffff700 1 -- start start 2023-10-06T02:24:28.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.891+0000 7ff45ffff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff448003a60 0x7ff4600c3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:28.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.891+0000 7ff45ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4601358a0 0x7ff460136e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:28.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.891+0000 7ff45ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4601349b0 0x7ff460137390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:28.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.891+0000 7ff45ffff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4600c70c0 con 0x7ff4601358a0 2023-10-06T02:24:28.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.891+0000 7ff45ffff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4601369c0 con 0x7ff4601349b0 2023-10-06T02:24:28.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.891+0000 7ff45ffff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4601378d0 con 0x7ff448003a60 2023-10-06T02:24:28.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.892+0000 7ff45e7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4601358a0 0x7ff460136e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:28.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.892+0000 7ff45e7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4601358a0 0x7ff460136e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40084/0 (socket says 172.21.15.31:40084) 2023-10-06T02:24:28.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.892+0000 7ff45e7fc700 1 -- 172.21.15.31:0/734667688 learned_addr learned my addr 172.21.15.31:0/734667688 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:28.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.892+0000 7ff45effd700 1 --2- 172.21.15.31:0/734667688 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff448003a60 0x7ff4600c3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:28.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.892+0000 7ff45f7fe700 1 --2- 172.21.15.31:0/734667688 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4601349b0 0x7ff460137390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:28.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.892+0000 7ff45e7fc700 1 -- 172.21.15.31:0/734667688 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff448003a60 msgr2=0x7ff4600c3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:28.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.892+0000 7ff45e7fc700 1 --2- 172.21.15.31:0/734667688 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff448003a60 0x7ff4600c3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:28.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.892+0000 7ff45e7fc700 1 -- 172.21.15.31:0/734667688 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4601349b0 msgr2=0x7ff460137390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:28.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.892+0000 7ff45e7fc700 1 --2- 172.21.15.31:0/734667688 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4601349b0 0x7ff460137390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:28.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.892+0000 7ff45e7fc700 1 -- 172.21.15.31:0/734667688 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff460135db0 con 0x7ff4601358a0 2023-10-06T02:24:28.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.892+0000 7ff45e7fc700 1 --2- 172.21.15.31:0/734667688 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4601358a0 0x7ff460136e50 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ff454001050 tx=0x7ff454001da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:28.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.893+0000 7ff446ffd700 1 -- 172.21.15.31:0/734667688 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4540077d0 con 0x7ff4601358a0 2023-10-06T02:24:28.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.893+0000 7ff446ffd700 1 -- 172.21.15.31:0/734667688 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff45400f8d0 con 0x7ff4601358a0 2023-10-06T02:24:28.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.893+0000 7ff446ffd700 1 -- 172.21.15.31:0/734667688 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff45400fc90 con 0x7ff4601358a0 2023-10-06T02:24:28.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.893+0000 7ff45ffff700 1 -- 172.21.15.31:0/734667688 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff460136090 con 0x7ff4601358a0 2023-10-06T02:24:28.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.893+0000 7ff45ffff700 1 -- 172.21.15.31:0/734667688 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff460136650 con 0x7ff4601358a0 2023-10-06T02:24:28.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.894+0000 7ff45ffff700 1 -- 172.21.15.31:0/734667688 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff460136b50 con 0x7ff4601358a0 2023-10-06T02:24:28.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.895+0000 7ff446ffd700 1 -- 172.21.15.31:0/734667688 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7ff45401d460 con 0x7ff4601358a0 2023-10-06T02:24:28.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.896+0000 7ff446ffd700 1 --2- 172.21.15.31:0/734667688 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7ff44c0670e0 0x7ff44c069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:28.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.896+0000 7ff446ffd700 1 -- 172.21.15.31:0/734667688 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff454084c10 con 0x7ff4601358a0 2023-10-06T02:24:28.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.896+0000 7ff45effd700 1 --2- 172.21.15.31:0/734667688 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7ff44c0670e0 0x7ff44c069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:28.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.897+0000 7ff45effd700 1 --2- 172.21.15.31:0/734667688 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7ff44c0670e0 0x7ff44c069590 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff4500091e0 tx=0x7ff45001a000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:28.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:28.900+0000 7ff446ffd700 1 -- 172.21.15.31:0/734667688 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff454051a00 con 0x7ff4601358a0 2023-10-06T02:24:29.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.104+0000 7ff45ffff700 1 -- 172.21.15.31:0/734667688 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff46004c3c0 con 0x7ff4601358a0 2023-10-06T02:24:29.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.104+0000 7ff446ffd700 1 -- 172.21.15.31:0/734667688 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v36) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff454055020 con 0x7ff4601358a0 2023-10-06T02:24:29.105 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-10-06T02:24:29.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.107+0000 7ff45ffff700 1 -- 172.21.15.31:0/734667688 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7ff44c0670e0 msgr2=0x7ff44c069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:29.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.107+0000 7ff45ffff700 1 --2- 172.21.15.31:0/734667688 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7ff44c0670e0 0x7ff44c069590 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff4500091e0 tx=0x7ff45001a000 comp rx=0 tx=0).stop 2023-10-06T02:24:29.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.107+0000 7ff45ffff700 1 -- 172.21.15.31:0/734667688 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4601358a0 msgr2=0x7ff460136e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:29.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.107+0000 7ff45ffff700 1 --2- 172.21.15.31:0/734667688 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4601358a0 0x7ff460136e50 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ff454001050 tx=0x7ff454001da0 comp rx=0 tx=0).stop 2023-10-06T02:24:29.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.107+0000 7ff45ffff700 1 -- 172.21.15.31:0/734667688 shutdown_connections 2023-10-06T02:24:29.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.107+0000 7ff45ffff700 1 --2- 172.21.15.31:0/734667688 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4601349b0 0x7ff460137390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:29.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.107+0000 7ff45ffff700 1 --2- 172.21.15.31:0/734667688 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7ff44c0670e0 0x7ff44c069590 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:29.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.107+0000 7ff45ffff700 1 --2- 172.21.15.31:0/734667688 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff448003a60 0x7ff4600c3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:29.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.107+0000 7ff45ffff700 1 --2- 172.21.15.31:0/734667688 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4601358a0 0x7ff460136e50 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:29.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.107+0000 7ff45ffff700 1 -- 172.21.15.31:0/734667688 >> 172.21.15.31:0/734667688 conn(0x7ff46006d110 msgr2=0x7ff4600c9d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:29.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.107+0000 7ff45ffff700 1 -- 172.21.15.31:0/734667688 shutdown_connections 2023-10-06T02:24:29.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.107+0000 7ff45ffff700 1 -- 172.21.15.31:0/734667688 wait complete. 2023-10-06T02:24:29.125 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T02:24:29.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.317+0000 7f263299d700 1 Processor -- start 2023-10-06T02:24:29.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.317+0000 7f263299d700 1 -- start start 2023-10-06T02:24:29.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.318+0000 7f263299d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f262c155890 0x7f262c0c7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:29.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.318+0000 7f263299d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f262c155c60 0x7f262c06a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:29.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.318+0000 7f263299d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f262c06de50 0x7f262c06e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:29.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.318+0000 7f263299d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f262c06e810 con 0x7f262c155890 2023-10-06T02:24:29.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.318+0000 7f263299d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f262c0c7e70 con 0x7f262c06de50 2023-10-06T02:24:29.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.318+0000 7f263299d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f262c06b790 con 0x7f262c155c60 2023-10-06T02:24:29.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.318+0000 7f263199b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f262c155890 0x7f262c0c7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:29.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.318+0000 7f263199b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f262c155890 0x7f262c0c7930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40090/0 (socket says 172.21.15.31:40090) 2023-10-06T02:24:29.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.318+0000 7f263199b700 1 -- 172.21.15.31:0/2150750127 learned_addr learned my addr 172.21.15.31:0/2150750127 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:29.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.319+0000 7f263199b700 1 -- 172.21.15.31:0/2150750127 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f262c155c60 msgr2=0x7f262c06a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:29.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.319+0000 7f263119a700 1 --2- 172.21.15.31:0/2150750127 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f262c155c60 0x7f262c06a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:29.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.319+0000 7f263219c700 1 --2- 172.21.15.31:0/2150750127 >> v2:172.21.15.62:3300/0 conn(0x7f262c06de50 0x7f262c06e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:29.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.319+0000 7f263199b700 1 --2- 172.21.15.31:0/2150750127 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f262c155c60 0x7f262c06a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:29.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.319+0000 7f263199b700 1 -- 172.21.15.31:0/2150750127 >> v2:172.21.15.62:3300/0 conn(0x7f262c06de50 msgr2=0x7f262c06e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:29.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.319+0000 7f263199b700 1 --2- 172.21.15.31:0/2150750127 >> v2:172.21.15.62:3300/0 conn(0x7f262c06de50 0x7f262c06e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:29.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.319+0000 7f263199b700 1 -- 172.21.15.31:0/2150750127 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f262c06a8e0 con 0x7f262c155890 2023-10-06T02:24:29.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.319+0000 7f263119a700 1 --2- 172.21.15.31:0/2150750127 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f262c155c60 0x7f262c06a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:24:29.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.319+0000 7f263219c700 1 --2- 172.21.15.31:0/2150750127 >> v2:172.21.15.62:3300/0 conn(0x7f262c06de50 0x7f262c06e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:24:29.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.319+0000 7f263199b700 1 --2- 172.21.15.31:0/2150750127 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f262c155890 0x7f262c0c7930 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f262c0677b0 tx=0x7f2628009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb0cb2977112e988 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:29.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.319+0000 7f2630999700 1 -- 172.21.15.31:0/2150750127 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262800b820 con 0x7f262c155890 2023-10-06T02:24:29.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.319+0000 7f2630999700 1 -- 172.21.15.31:0/2150750127 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f262800b980 con 0x7f262c155890 2023-10-06T02:24:29.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.319+0000 7f2630999700 1 -- 172.21.15.31:0/2150750127 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262800bd20 con 0x7f262c155890 2023-10-06T02:24:29.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.320+0000 7f263299d700 1 -- 172.21.15.31:0/2150750127 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f262c155890 msgr2=0x7f262c0c7930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:29.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.320+0000 7f263299d700 1 --2- 172.21.15.31:0/2150750127 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f262c155890 0x7f262c0c7930 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f262c0677b0 tx=0x7f2628009250 comp rx=0 tx=0).stop 2023-10-06T02:24:29.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.320+0000 7f263299d700 1 -- 172.21.15.31:0/2150750127 shutdown_connections 2023-10-06T02:24:29.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.320+0000 7f263299d700 1 --2- 172.21.15.31:0/2150750127 >> v2:172.21.15.62:3300/0 conn(0x7f262c06de50 0x7f262c06e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:29.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.320+0000 7f263299d700 1 --2- 172.21.15.31:0/2150750127 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f262c155890 0x7f262c0c7930 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:29.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.320+0000 7f263299d700 1 --2- 172.21.15.31:0/2150750127 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f262c155c60 0x7f262c06a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:29.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.320+0000 7f263299d700 1 -- 172.21.15.31:0/2150750127 >> 172.21.15.31:0/2150750127 conn(0x7f262c06d110 msgr2=0x7f262c0cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:29.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.320+0000 7f263299d700 1 -- 172.21.15.31:0/2150750127 shutdown_connections 2023-10-06T02:24:29.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.320+0000 7f263299d700 1 -- 172.21.15.31:0/2150750127 wait complete. 2023-10-06T02:24:29.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.322+0000 7f263299d700 1 Processor -- start 2023-10-06T02:24:29.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.322+0000 7f263299d700 1 -- start start 2023-10-06T02:24:29.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.323+0000 7f263299d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f262c06de50 0x7f262c0a98e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:29.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.323+0000 7f263299d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f262c155890 0x7f262c0c6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:29.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.323+0000 7f263299d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f262c155c60 0x7f262c153ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:29.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.323+0000 7f263299d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f262c0c6540 con 0x7f262c155c60 2023-10-06T02:24:29.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.323+0000 7f263299d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f262c0ab550 con 0x7f262c155890 2023-10-06T02:24:29.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.323+0000 7f263299d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f262c0c4d50 con 0x7f262c06de50 2023-10-06T02:24:29.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.323+0000 7f263219c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f262c155c60 0x7f262c153ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:29.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.323+0000 7f263219c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f262c155c60 0x7f262c153ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40106/0 (socket says 172.21.15.31:40106) 2023-10-06T02:24:29.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.323+0000 7f263219c700 1 -- 172.21.15.31:0/775639084 learned_addr learned my addr 172.21.15.31:0/775639084 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:29.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.323+0000 7f263119a700 1 --2- 172.21.15.31:0/775639084 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f262c155890 0x7f262c0c6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:29.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.323+0000 7f263199b700 1 --2- 172.21.15.31:0/775639084 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f262c06de50 0x7f262c0a98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:29.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.323+0000 7f263219c700 1 -- 172.21.15.31:0/775639084 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f262c06de50 msgr2=0x7f262c0a98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:29.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.323+0000 7f263219c700 1 --2- 172.21.15.31:0/775639084 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f262c06de50 0x7f262c0a98e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:29.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.323+0000 7f263219c700 1 -- 172.21.15.31:0/775639084 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f262c155890 msgr2=0x7f262c0c6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:29.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.323+0000 7f263219c700 1 --2- 172.21.15.31:0/775639084 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f262c155890 0x7f262c0c6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:29.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.323+0000 7f263219c700 1 -- 172.21.15.31:0/775639084 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26280094f0 con 0x7f262c155c60 2023-10-06T02:24:29.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.323+0000 7f263119a700 1 --2- 172.21.15.31:0/775639084 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f262c155890 0x7f262c0c6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:24:29.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.324+0000 7f263219c700 1 --2- 172.21.15.31:0/775639084 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f262c155c60 0x7f262c153ef0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f262400ce40 tx=0x7f26240077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:29.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.324+0000 7f2621ffb700 1 -- 172.21.15.31:0/775639084 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262400adf0 con 0x7f262c155c60 2023-10-06T02:24:29.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.324+0000 7f2621ffb700 1 -- 172.21.15.31:0/775639084 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2624016420 con 0x7f262c155c60 2023-10-06T02:24:29.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.324+0000 7f2621ffb700 1 -- 172.21.15.31:0/775639084 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26240167e0 con 0x7f262c155c60 2023-10-06T02:24:29.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.324+0000 7f263299d700 1 -- 172.21.15.31:0/775639084 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f262c0a9f30 con 0x7f262c155c60 2023-10-06T02:24:29.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.324+0000 7f263299d700 1 -- 172.21.15.31:0/775639084 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f262c0c52f0 con 0x7f262c155c60 2023-10-06T02:24:29.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.325+0000 7f263299d700 1 -- 172.21.15.31:0/775639084 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f262c0ab750 con 0x7f262c155c60 2023-10-06T02:24:29.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.326+0000 7f2621ffb700 1 -- 172.21.15.31:0/775639084 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f2624016b50 con 0x7f262c155c60 2023-10-06T02:24:29.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.327+0000 7f2621ffb700 1 --2- 172.21.15.31:0/775639084 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f26100670e0 0x7f2610069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:29.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.327+0000 7f2621ffb700 1 -- 172.21.15.31:0/775639084 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2624012070 con 0x7f262c155c60 2023-10-06T02:24:29.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.328+0000 7f263199b700 1 --2- 172.21.15.31:0/775639084 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f26100670e0 0x7f2610069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:29.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.328+0000 7f263199b700 1 --2- 172.21.15.31:0/775639084 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f26100670e0 0x7f2610069590 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f262c0677b0 tx=0x7f2628018000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:29.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.331+0000 7f2621ffb700 1 -- 172.21.15.31:0/775639084 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f262404e5a0 con 0x7f262c155c60 2023-10-06T02:24:29.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.534+0000 7f263299d700 1 -- 172.21.15.31:0/775639084 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f262c04c3c0 con 0x7f262c155c60 2023-10-06T02:24:29.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.535+0000 7f2621ffb700 1 -- 172.21.15.31:0/775639084 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v36) v1 ==== 100+0+11 (secure 0 0 0) 0x7f262c0ab750 con 0x7f262c155c60 2023-10-06T02:24:29.535 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-10-06T02:24:29.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.537+0000 7f263299d700 1 -- 172.21.15.31:0/775639084 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f26100670e0 msgr2=0x7f2610069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:29.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.537+0000 7f263299d700 1 --2- 172.21.15.31:0/775639084 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f26100670e0 0x7f2610069590 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f262c0677b0 tx=0x7f2628018000 comp rx=0 tx=0).stop 2023-10-06T02:24:29.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.537+0000 7f263299d700 1 -- 172.21.15.31:0/775639084 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f262c155c60 msgr2=0x7f262c153ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:29.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.537+0000 7f263299d700 1 --2- 172.21.15.31:0/775639084 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f262c155c60 0x7f262c153ef0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f262400ce40 tx=0x7f26240077d0 comp rx=0 tx=0).stop 2023-10-06T02:24:29.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.537+0000 7f263299d700 1 -- 172.21.15.31:0/775639084 shutdown_connections 2023-10-06T02:24:29.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.537+0000 7f263299d700 1 --2- 172.21.15.31:0/775639084 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f262c155c60 0x7f262c153ef0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:29.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.537+0000 7f263299d700 1 --2- 172.21.15.31:0/775639084 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f26100670e0 0x7f2610069590 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:29.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.537+0000 7f263299d700 1 --2- 172.21.15.31:0/775639084 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f262c06de50 0x7f262c0a98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:29.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.537+0000 7f263299d700 1 --2- 172.21.15.31:0/775639084 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f262c155890 0x7f262c0c6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:29.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.537+0000 7f263299d700 1 -- 172.21.15.31:0/775639084 >> 172.21.15.31:0/775639084 conn(0x7f262c06d110 msgr2=0x7f262c1581a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:29.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.537+0000 7f263299d700 1 -- 172.21.15.31:0/775639084 shutdown_connections 2023-10-06T02:24:29.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.537+0000 7f263299d700 1 -- 172.21.15.31:0/775639084 wait complete. 2023-10-06T02:24:29.555 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:24:29.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.740+0000 7f409599d700 1 Processor -- start 2023-10-06T02:24:29.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.740+0000 7f409599d700 1 -- start start 2023-10-06T02:24:29.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.742+0000 7f409599d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f409013fef0 0x7f409015a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:29.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.742+0000 7f409599d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f409006aaa0 0x7f40900c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:29.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.742+0000 7f409599d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f409015ae70 con 0x7f409013fb20 2023-10-06T02:24:29.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.742+0000 7f409599d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f409006af60 con 0x7f409013fef0 2023-10-06T02:24:29.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.742+0000 7f409599d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f409006b240 con 0x7f409006aaa0 2023-10-06T02:24:29.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.742+0000 7f408ffff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f409013fef0 0x7f409015a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:29.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.742+0000 7f409499b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f409013fb20 0x7f40900c7990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58052/0 (socket says 172.21.15.31:58052) 2023-10-06T02:24:29.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.742+0000 7f408ffff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f409013fef0 0x7f409015a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40122/0 (socket says 172.21.15.31:40122) 2023-10-06T02:24:29.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.742+0000 7f409499b700 1 -- 172.21.15.31:0/3137125051 learned_addr learned my addr 172.21.15.31:0/3137125051 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:24:29.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.742+0000 7f409519c700 1 --2- 172.21.15.31:0/3137125051 >> v2:172.21.15.62:3300/0 conn(0x7f409006aaa0 0x7f40900c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:29.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.742+0000 7f408ffff700 1 -- 172.21.15.31:0/3137125051 >> v2:172.21.15.62:3300/0 conn(0x7f409006aaa0 msgr2=0x7f40900c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:29.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.742+0000 7f408ffff700 1 --2- 172.21.15.31:0/3137125051 >> v2:172.21.15.62:3300/0 conn(0x7f409006aaa0 0x7f40900c9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:29.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.742+0000 7f408ffff700 1 -- 172.21.15.31:0/3137125051 >> v1:172.21.15.31:6789/0 conn(0x7f409013fb20 legacy=0x7f40900c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:29.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.742+0000 7f408ffff700 1 -- 172.21.15.31:0/3137125051 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f409006b520 con 0x7f409013fef0 2023-10-06T02:24:29.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.742+0000 7f408f7fe700 1 -- 172.21.15.31:0/3137125051 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1155332639 0 0) 0x7f4084001110 con 0x7f409013fb20 2023-10-06T02:24:29.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.743+0000 7f408ffff700 1 --2- 172.21.15.31:0/3137125051 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f409013fef0 0x7f409015a930 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f4090069d40 tx=0x7f407c009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=887b0966696282ab server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:29.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.743+0000 7f408f7fe700 1 -- 172.21.15.31:0/3137125051 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f407c005c10 con 0x7f409013fef0 2023-10-06T02:24:29.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.743+0000 7f408f7fe700 1 -- 172.21.15.31:0/3137125051 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f407c005d70 con 0x7f409013fef0 2023-10-06T02:24:29.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.743+0000 7f408f7fe700 1 -- 172.21.15.31:0/3137125051 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f407c006260 con 0x7f409013fef0 2023-10-06T02:24:29.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.743+0000 7f409599d700 1 -- 172.21.15.31:0/3137125051 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f409013fef0 msgr2=0x7f409015a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:29.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.743+0000 7f409599d700 1 --2- 172.21.15.31:0/3137125051 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f409013fef0 0x7f409015a930 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f4090069d40 tx=0x7f407c009120 comp rx=0 tx=0).stop 2023-10-06T02:24:29.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.744+0000 7f409599d700 1 -- 172.21.15.31:0/3137125051 shutdown_connections 2023-10-06T02:24:29.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.744+0000 7f409599d700 1 --2- 172.21.15.31:0/3137125051 >> v2:172.21.15.62:3300/0 conn(0x7f409006aaa0 0x7f40900c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:29.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.744+0000 7f409599d700 1 --2- 172.21.15.31:0/3137125051 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f409013fef0 0x7f409015a930 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:29.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.744+0000 7f409599d700 1 -- 172.21.15.31:0/3137125051 >> 172.21.15.31:0/3137125051 conn(0x7f409006d170 msgr2=0x7f40900cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:29.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.744+0000 7f409599d700 1 -- 172.21.15.31:0/3137125051 shutdown_connections 2023-10-06T02:24:29.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.744+0000 7f409599d700 1 -- 172.21.15.31:0/3137125051 wait complete. 2023-10-06T02:24:29.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.745+0000 7f409599d700 1 Processor -- start 2023-10-06T02:24:29.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.745+0000 7f409599d700 1 -- start start 2023-10-06T02:24:29.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.746+0000 7f409599d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f409006aaa0 0x7f4090155890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:29.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.746+0000 7f409599d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f409013fb20 0x7f40900ad610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:29.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.746+0000 7f409599d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f409013fef0 0x7f40900adb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:29.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.746+0000 7f409599d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40900aea70 con 0x7f409006aaa0 2023-10-06T02:24:29.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.746+0000 7f409599d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4090155dd0 con 0x7f409013fb20 2023-10-06T02:24:29.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.746+0000 7f409599d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40901573a0 con 0x7f409013fef0 2023-10-06T02:24:29.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.746+0000 7f409499b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f409006aaa0 0x7f4090155890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:29.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.747+0000 7f409499b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f409006aaa0 0x7f4090155890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40134/0 (socket says 172.21.15.31:40134) 2023-10-06T02:24:29.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.747+0000 7f409499b700 1 -- 172.21.15.31:0/844676430 learned_addr learned my addr 172.21.15.31:0/844676430 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:29.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.747+0000 7f408ffff700 1 --2- 172.21.15.31:0/844676430 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f409013fb20 0x7f40900ad610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:29.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.747+0000 7f409519c700 1 --2- 172.21.15.31:0/844676430 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f409013fef0 0x7f40900adb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:29.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.747+0000 7f409499b700 1 -- 172.21.15.31:0/844676430 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f409013fef0 msgr2=0x7f40900adb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:29.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.747+0000 7f409499b700 1 --2- 172.21.15.31:0/844676430 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f409013fef0 0x7f40900adb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:29.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.747+0000 7f409499b700 1 -- 172.21.15.31:0/844676430 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f409013fb20 msgr2=0x7f40900ad610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:29.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.747+0000 7f409499b700 1 --2- 172.21.15.31:0/844676430 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f409013fb20 0x7f40900ad610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:29.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.747+0000 7f409499b700 1 -- 172.21.15.31:0/844676430 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f407c0093c0 con 0x7f409006aaa0 2023-10-06T02:24:29.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.747+0000 7f409499b700 1 --2- 172.21.15.31:0/844676430 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f409006aaa0 0x7f4090155890 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f408400c810 tx=0x7f4084007c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:29.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.747+0000 7f408cff9700 1 -- 172.21.15.31:0/844676430 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4084005b30 con 0x7f409006aaa0 2023-10-06T02:24:29.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.747+0000 7f408cff9700 1 -- 172.21.15.31:0/844676430 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4084005c90 con 0x7f409006aaa0 2023-10-06T02:24:29.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.747+0000 7f408cff9700 1 -- 172.21.15.31:0/844676430 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4084007300 con 0x7f409006aaa0 2023-10-06T02:24:29.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.747+0000 7f409599d700 1 -- 172.21.15.31:0/844676430 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40900ae1d0 con 0x7f409006aaa0 2023-10-06T02:24:29.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.748+0000 7f409599d700 1 -- 172.21.15.31:0/844676430 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40900ae790 con 0x7f409006aaa0 2023-10-06T02:24:29.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.749+0000 7f409599d700 1 -- 172.21.15.31:0/844676430 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4090157170 con 0x7f409006aaa0 2023-10-06T02:24:29.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.750+0000 7f408cff9700 1 -- 172.21.15.31:0/844676430 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f40840076b0 con 0x7f409006aaa0 2023-10-06T02:24:29.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.750+0000 7f408cff9700 1 --2- 172.21.15.31:0/844676430 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f40740670e0 0x7f4074069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:29.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.750+0000 7f408cff9700 1 -- 172.21.15.31:0/844676430 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f4084084430 con 0x7f409006aaa0 2023-10-06T02:24:29.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.751+0000 7f408ffff700 1 --2- 172.21.15.31:0/844676430 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f40740670e0 0x7f4074069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:29.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.751+0000 7f408ffff700 1 --2- 172.21.15.31:0/844676430 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f40740670e0 0x7f4074069590 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4090069d40 tx=0x7f407c016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:29.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:29.754+0000 7f408cff9700 1 -- 172.21.15.31:0/844676430 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4090157170 con 0x7f409006aaa0 2023-10-06T02:24:30.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.002+0000 7f409599d700 1 -- 172.21.15.31:0/844676430 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f409015ae70 con 0x7f409006aaa0 2023-10-06T02:24:30.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.002+0000 7f408cff9700 1 -- 172.21.15.31:0/844676430 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4591 (secure 0 0 0) 0x7f409015ae70 con 0x7f409006aaa0 2023-10-06T02:24:30.003 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:24:30.003 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4700,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/384287452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":384287452},{"type":"v1","addr":"172.21.15.62:6834","nonce":384287452}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5037,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/693643908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":693643908},{"type":"v1","addr":"172.21.15.31:6836","nonce":693643908}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5040,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2599441423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2599441423},{"type":"v1","addr":"172.21.15.62:6835","nonce":2599441423}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:24:12.493929+0000","modified":"2023-10-06T02:24:16.026454+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5043},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5043":{"gid":5043,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.31:6837/2608294748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":2608294748},{"type":"v1","addr":"172.21.15.31:6837","nonce":2608294748}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T02:24:30.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.005+0000 7f409599d700 1 -- 172.21.15.31:0/844676430 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f40740670e0 msgr2=0x7f4074069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:30.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.005+0000 7f409599d700 1 --2- 172.21.15.31:0/844676430 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f40740670e0 0x7f4074069590 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4090069d40 tx=0x7f407c016000 comp rx=0 tx=0).stop 2023-10-06T02:24:30.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.005+0000 7f409599d700 1 -- 172.21.15.31:0/844676430 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f409006aaa0 msgr2=0x7f4090155890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:30.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.005+0000 7f409599d700 1 --2- 172.21.15.31:0/844676430 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f409006aaa0 0x7f4090155890 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f408400c810 tx=0x7f4084007c50 comp rx=0 tx=0).stop 2023-10-06T02:24:30.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.005+0000 7f409599d700 1 -- 172.21.15.31:0/844676430 shutdown_connections 2023-10-06T02:24:30.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.005+0000 7f409599d700 1 --2- 172.21.15.31:0/844676430 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f40740670e0 0x7f4074069590 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:30.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.005+0000 7f409599d700 1 --2- 172.21.15.31:0/844676430 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f409013fef0 0x7f40900adb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:30.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.005+0000 7f409599d700 1 --2- 172.21.15.31:0/844676430 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f409006aaa0 0x7f4090155890 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:30.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.005+0000 7f409599d700 1 --2- 172.21.15.31:0/844676430 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f409013fb20 0x7f40900ad610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:30.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.005+0000 7f409599d700 1 -- 172.21.15.31:0/844676430 >> 172.21.15.31:0/844676430 conn(0x7f409006d170 msgr2=0x7f40900c16f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:30.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.005+0000 7f409599d700 1 -- 172.21.15.31:0/844676430 shutdown_connections 2023-10-06T02:24:30.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.005+0000 7f409599d700 1 -- 172.21.15.31:0/844676430 wait complete. 2023-10-06T02:24:30.006 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 18 2023-10-06T02:24:30.022 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:24:30.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.208+0000 7f0e1539d700 1 Processor -- start 2023-10-06T02:24:30.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.208+0000 7f0e1539d700 1 -- start start 2023-10-06T02:24:30.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.210+0000 7f0e1539d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0e1013fb20 0x7f0e1015a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:30.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.210+0000 7f0e1539d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e1015ae70 con 0x7f0e1006aaa0 2023-10-06T02:24:30.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.210+0000 7f0e1539d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e1006af60 con 0x7f0e1013fef0 2023-10-06T02:24:30.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.210+0000 7f0e1539d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1006b240 con 0x7f0e1013fb20 2023-10-06T02:24:30.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.210+0000 7f0e14b9c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f0e1006aaa0 0x7f0e100c9d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58066/0 (socket says 172.21.15.31:58066) 2023-10-06T02:24:30.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.210+0000 7f0e14b9c700 1 -- 172.21.15.31:0/3087776419 learned_addr learned my addr 172.21.15.31:0/3087776419 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:24:30.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.210+0000 7f0e0ffff700 1 --2- 172.21.15.31:0/3087776419 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0e1013fb20 0x7f0e1015a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:30.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.210+0000 7f0e0effd700 1 -- 172.21.15.31:0/3087776419 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1423342699 0 0) 0x7f0e08001490 con 0x7f0e1006aaa0 2023-10-06T02:24:30.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.210+0000 7f0e0effd700 1 -- 172.21.15.31:0/3087776419 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0df8003570 con 0x7f0e1006aaa0 2023-10-06T02:24:30.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.211+0000 7f0e0effd700 1 -- 172.21.15.31:0/3087776419 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1130388834 0 0) 0x7f0df8003570 con 0x7f0e1006aaa0 2023-10-06T02:24:30.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.211+0000 7f0e0effd700 1 -- 172.21.15.31:0/3087776419 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0e1013fb20 msgr2=0x7f0e1015a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:30.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.211+0000 7f0e0effd700 1 --2- 172.21.15.31:0/3087776419 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0e1013fb20 0x7f0e1015a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:30.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.211+0000 7f0e0effd700 1 -- 172.21.15.31:0/3087776419 >> v1:172.21.15.62:6789/0 conn(0x7f0e1013fef0 legacy=0x7f0e100c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:30.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.211+0000 7f0e0effd700 1 -- 172.21.15.31:0/3087776419 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e1006b520 con 0x7f0e1006aaa0 2023-10-06T02:24:30.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.211+0000 7f0e0ffff700 1 --2- 172.21.15.31:0/3087776419 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0e1013fb20 0x7f0e1015a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:24:30.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.211+0000 7f0e0effd700 1 -- 172.21.15.31:0/3087776419 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7f0e08001980 con 0x7f0e1006aaa0 2023-10-06T02:24:30.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.211+0000 7f0e0effd700 1 -- 172.21.15.31:0/3087776419 >> v1:172.21.15.31:6789/0 conn(0x7f0e1006aaa0 legacy=0x7f0e100c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:30.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.211+0000 7f0e0effd700 1 --2- 172.21.15.31:0/3087776419 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0e1013fb20 0x7f0df8003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:30.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.211+0000 7f0e0effd700 1 --2- 172.21.15.31:0/3087776419 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0df8004020 0x7f0df8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:30.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.211+0000 7f0e0effd700 1 --2- 172.21.15.31:0/3087776419 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0df8004d50 0x7f0df80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:30.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.211+0000 7f0e0effd700 1 -- 172.21.15.31:0/3087776419 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1006b240 con 0x7f0df8004020 2023-10-06T02:24:30.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.211+0000 7f0e0effd700 1 -- 172.21.15.31:0/3087776419 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0df800bbf0 con 0x7f0df8004d50 2023-10-06T02:24:30.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.211+0000 7f0e0effd700 1 -- 172.21.15.31:0/3087776419 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0df800bdb0 con 0x7f0e1013fb20 2023-10-06T02:24:30.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.211+0000 7f0e0f7fe700 1 --2- 172.21.15.31:0/3087776419 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0df8004020 0x7f0df8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:30.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.211+0000 7f0e0ffff700 1 --2- 172.21.15.31:0/3087776419 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0e1013fb20 0x7f0df8003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:30.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.212+0000 7f0e14b9c700 1 --2- 172.21.15.31:0/3087776419 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0df8004d50 0x7f0df80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:30.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.212+0000 7f0e0ffff700 1 -- 172.21.15.31:0/3087776419 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0df8004d50 msgr2=0x7f0df80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:30.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.212+0000 7f0e0ffff700 1 --2- 172.21.15.31:0/3087776419 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0df8004d50 0x7f0df80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:30.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.212+0000 7f0e0ffff700 1 -- 172.21.15.31:0/3087776419 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0df8004020 msgr2=0x7f0df8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:30.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.212+0000 7f0e0ffff700 1 --2- 172.21.15.31:0/3087776419 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0df8004020 0x7f0df8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:30.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.212+0000 7f0e0ffff700 1 -- 172.21.15.31:0/3087776419 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e00009420 con 0x7f0e1013fb20 2023-10-06T02:24:30.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.212+0000 7f0e0f7fe700 1 --2- 172.21.15.31:0/3087776419 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0df8004020 0x7f0df8004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:24:30.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.212+0000 7f0e0ffff700 1 --2- 172.21.15.31:0/3087776419 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0e1013fb20 0x7f0df8003a90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0e00009df0 tx=0x7f0e000091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e47f6d914211546 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:30.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.216+0000 7f0e0effd700 1 -- 172.21.15.31:0/3087776419 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e0000e920 con 0x7f0e1013fb20 2023-10-06T02:24:30.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.216+0000 7f0e0effd700 1 -- 172.21.15.31:0/3087776419 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e0000ea80 con 0x7f0e1013fb20 2023-10-06T02:24:30.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.216+0000 7f0e0effd700 1 -- 172.21.15.31:0/3087776419 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e0000ee20 con 0x7f0e1013fb20 2023-10-06T02:24:30.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.217+0000 7f0e1539d700 1 -- 172.21.15.31:0/3087776419 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0e1013fb20 msgr2=0x7f0df8003a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:30.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.217+0000 7f0e1539d700 1 --2- 172.21.15.31:0/3087776419 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0e1013fb20 0x7f0df8003a90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0e00009df0 tx=0x7f0e000091a0 comp rx=0 tx=0).stop 2023-10-06T02:24:30.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.217+0000 7f0e14b9c700 1 -- 172.21.15.31:0/3087776419 reap_dead start 2023-10-06T02:24:30.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.217+0000 7f0e1539d700 1 -- 172.21.15.31:0/3087776419 shutdown_connections 2023-10-06T02:24:30.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.217+0000 7f0e1539d700 1 -- 172.21.15.31:0/3087776419 >> 172.21.15.31:0/3087776419 conn(0x7f0e1006d170 msgr2=0x7f0e100cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:30.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.217+0000 7f0e1539d700 1 -- 172.21.15.31:0/3087776419 shutdown_connections 2023-10-06T02:24:30.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.217+0000 7f0e1539d700 1 -- 172.21.15.31:0/3087776419 wait complete. 2023-10-06T02:24:30.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.218+0000 7f0e1539d700 1 Processor -- start 2023-10-06T02:24:30.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.218+0000 7f0e1539d700 1 -- start start 2023-10-06T02:24:30.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.219+0000 7f0e1539d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0e1013fb20 0x7f0e100c9f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:30.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.219+0000 7f0e1539d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e101558e0 0x7f0e10155d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:30.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.219+0000 7f0e1539d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e100b05a0 0x7f0e100b09f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:30.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.219+0000 7f0e1539d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e10156290 con 0x7f0e101558e0 2023-10-06T02:24:30.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.219+0000 7f0e1539d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e10159550 con 0x7f0e100b05a0 2023-10-06T02:24:30.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.219+0000 7f0e1539d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e100ab740 con 0x7f0e1013fb20 2023-10-06T02:24:30.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.220+0000 7f0e0f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e101558e0 0x7f0e10155d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:30.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.220+0000 7f0e0f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e101558e0 0x7f0e10155d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40162/0 (socket says 172.21.15.31:40162) 2023-10-06T02:24:30.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.220+0000 7f0e0f7fe700 1 -- 172.21.15.31:0/1932474035 learned_addr learned my addr 172.21.15.31:0/1932474035 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:30.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.220+0000 7f0e0ffff700 1 --2- 172.21.15.31:0/1932474035 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0e1013fb20 0x7f0e100c9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:30.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.220+0000 7f0e14b9c700 1 --2- 172.21.15.31:0/1932474035 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e100b05a0 0x7f0e100b09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:30.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.220+0000 7f0e0f7fe700 1 -- 172.21.15.31:0/1932474035 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0e1013fb20 msgr2=0x7f0e100c9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:30.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.220+0000 7f0e0f7fe700 1 --2- 172.21.15.31:0/1932474035 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0e1013fb20 0x7f0e100c9f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:30.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.220+0000 7f0e0f7fe700 1 -- 172.21.15.31:0/1932474035 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e100b05a0 msgr2=0x7f0e100b09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:30.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.220+0000 7f0e0f7fe700 1 --2- 172.21.15.31:0/1932474035 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e100b05a0 0x7f0e100b09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:30.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.220+0000 7f0e0f7fe700 1 -- 172.21.15.31:0/1932474035 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e10158c20 con 0x7f0e101558e0 2023-10-06T02:24:30.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.220+0000 7f0e14b9c700 1 --2- 172.21.15.31:0/1932474035 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e100b05a0 0x7f0e100b09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:24:30.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.220+0000 7f0e0f7fe700 1 --2- 172.21.15.31:0/1932474035 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e101558e0 0x7f0e10155d50 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0e040078f0 tx=0x7f0e04009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:30.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.220+0000 7f0df7fff700 1 -- 172.21.15.31:0/1932474035 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e0400d850 con 0x7f0e101558e0 2023-10-06T02:24:30.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.220+0000 7f0df7fff700 1 -- 172.21.15.31:0/1932474035 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e0400d9b0 con 0x7f0e101558e0 2023-10-06T02:24:30.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.221+0000 7f0e1539d700 1 -- 172.21.15.31:0/1932474035 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e10158f00 con 0x7f0e101558e0 2023-10-06T02:24:30.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.221+0000 7f0df7fff700 1 -- 172.21.15.31:0/1932474035 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e0400dd30 con 0x7f0e101558e0 2023-10-06T02:24:30.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.221+0000 7f0e1539d700 1 -- 172.21.15.31:0/1932474035 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e10158910 con 0x7f0e101558e0 2023-10-06T02:24:30.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.222+0000 7f0e1539d700 1 -- 172.21.15.31:0/1932474035 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e10159700 con 0x7f0e101558e0 2023-10-06T02:24:30.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.223+0000 7f0df7fff700 1 -- 172.21.15.31:0/1932474035 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f0e04013600 con 0x7f0e101558e0 2023-10-06T02:24:30.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.223+0000 7f0df7fff700 1 --2- 172.21.15.31:0/1932474035 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f0dfc0670e0 0x7f0dfc069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:30.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.223+0000 7f0df7fff700 1 -- 172.21.15.31:0/1932474035 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0e04084b80 con 0x7f0e101558e0 2023-10-06T02:24:30.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.224+0000 7f0e0ffff700 1 --2- 172.21.15.31:0/1932474035 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f0dfc0670e0 0x7f0dfc069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:30.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.224+0000 7f0e0ffff700 1 --2- 172.21.15.31:0/1932474035 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f0dfc0670e0 0x7f0dfc069590 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0e000091e0 tx=0x7f0e0001a000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:30.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.228+0000 7f0df7fff700 1 -- 172.21.15.31:0/1932474035 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0e10159700 con 0x7f0e101558e0 2023-10-06T02:24:30.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.430+0000 7f0e1539d700 1 -- 172.21.15.31:0/1932474035 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0e101596c0 con 0x7f0e101558e0 2023-10-06T02:24:30.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.431+0000 7f0df7fff700 1 -- 172.21.15.31:0/1932474035 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+16818 (secure 0 0 0) 0x7f0e04051970 con 0x7f0e101558e0 2023-10-06T02:24:30.431 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:24:30.431 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":36,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:24:14.007126+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:24:10.026011+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-06T02:24:11.471970+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:24:30.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.433+0000 7f0e1539d700 1 -- 172.21.15.31:0/1932474035 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f0dfc0670e0 msgr2=0x7f0dfc069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:30.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.433+0000 7f0e1539d700 1 --2- 172.21.15.31:0/1932474035 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f0dfc0670e0 0x7f0dfc069590 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0e000091e0 tx=0x7f0e0001a000 comp rx=0 tx=0).stop 2023-10-06T02:24:30.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.433+0000 7f0e1539d700 1 -- 172.21.15.31:0/1932474035 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e101558e0 msgr2=0x7f0e10155d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:30.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.433+0000 7f0e1539d700 1 --2- 172.21.15.31:0/1932474035 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e101558e0 0x7f0e10155d50 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0e040078f0 tx=0x7f0e04009040 comp rx=0 tx=0).stop 2023-10-06T02:24:30.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.433+0000 7f0e1539d700 1 -- 172.21.15.31:0/1932474035 shutdown_connections 2023-10-06T02:24:30.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.433+0000 7f0e1539d700 1 --2- 172.21.15.31:0/1932474035 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e100b05a0 0x7f0e100b09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:30.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.433+0000 7f0e1539d700 1 --2- 172.21.15.31:0/1932474035 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f0dfc0670e0 0x7f0dfc069590 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:30.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.433+0000 7f0e1539d700 1 --2- 172.21.15.31:0/1932474035 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0e1013fb20 0x7f0e100c9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:30.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.433+0000 7f0e1539d700 1 --2- 172.21.15.31:0/1932474035 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e101558e0 0x7f0e10155d50 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:30.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.433+0000 7f0e1539d700 1 -- 172.21.15.31:0/1932474035 >> 172.21.15.31:0/1932474035 conn(0x7f0e1006d170 msgr2=0x7f0e1006eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:30.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.433+0000 7f0e1539d700 1 -- 172.21.15.31:0/1932474035 shutdown_connections 2023-10-06T02:24:30.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.433+0000 7f0e1539d700 1 -- 172.21.15.31:0/1932474035 wait complete. 2023-10-06T02:24:30.451 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T02:24:30.451 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:24:30.459 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T02:24:30.500 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T02:24:30.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.640+0000 7f772299d700 1 Processor -- start 2023-10-06T02:24:30.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.640+0000 7f772299d700 1 -- start start 2023-10-06T02:24:30.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.641+0000 7f772299d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f771c157420 0x7f771c0b13b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:30.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.641+0000 7f772299d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f771c06d250 0x7f771c0aefa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:30.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.641+0000 7f772299d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f771c0ac840 con 0x7f771c0ceac0 2023-10-06T02:24:30.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.641+0000 7f772299d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f771c0b18f0 con 0x7f771c157420 2023-10-06T02:24:30.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.641+0000 7f772299d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f771c0abf50 con 0x7f771c06d250 2023-10-06T02:24:30.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.641+0000 7f772199b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f771c157420 0x7f771c0b13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:30.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.642+0000 7f772199b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f771c157420 0x7f771c0b13b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40176/0 (socket says 172.21.15.31:40176) 2023-10-06T02:24:30.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.642+0000 7f772199b700 1 -- 172.21.15.31:0/3803511668 learned_addr learned my addr 172.21.15.31:0/3803511668 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:30.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.642+0000 7f772119a700 1 --2- 172.21.15.31:0/3803511668 >> v2:172.21.15.62:3300/0 conn(0x7f771c06d250 0x7f771c0aefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:30.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.642+0000 7f772199b700 1 -- 172.21.15.31:0/3803511668 >> v2:172.21.15.62:3300/0 conn(0x7f771c06d250 msgr2=0x7f771c0aefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:30.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.642+0000 7f772199b700 1 --2- 172.21.15.31:0/3803511668 >> v2:172.21.15.62:3300/0 conn(0x7f771c06d250 0x7f771c0aefa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:30.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.642+0000 7f772199b700 1 -- 172.21.15.31:0/3803511668 >> v1:172.21.15.62:6789/0 conn(0x7f771c0ceac0 legacy=0x7f771c0accd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:30.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.642+0000 7f772199b700 1 -- 172.21.15.31:0/3803511668 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f771c0aea60 con 0x7f771c157420 2023-10-06T02:24:30.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.642+0000 7f772119a700 1 --2- 172.21.15.31:0/3803511668 >> v2:172.21.15.62:3300/0 conn(0x7f771c06d250 0x7f771c0aefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:24:30.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.642+0000 7f772199b700 1 --2- 172.21.15.31:0/3803511668 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f771c157420 0x7f771c0b13b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f771c06aa90 tx=0x7f7718009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2779893c9a89b09 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:30.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.643+0000 7f7720999700 1 -- 172.21.15.31:0/3803511668 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f771800b860 con 0x7f771c157420 2023-10-06T02:24:30.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.643+0000 7f7720999700 1 -- 172.21.15.31:0/3803511668 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f771800b9c0 con 0x7f771c157420 2023-10-06T02:24:30.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.643+0000 7f7720999700 1 -- 172.21.15.31:0/3803511668 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f771800bd60 con 0x7f771c157420 2023-10-06T02:24:30.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.643+0000 7f772299d700 1 -- 172.21.15.31:0/3803511668 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f771c157420 msgr2=0x7f771c0b13b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:30.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.643+0000 7f772299d700 1 --2- 172.21.15.31:0/3803511668 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f771c157420 0x7f771c0b13b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f771c06aa90 tx=0x7f7718009250 comp rx=0 tx=0).stop 2023-10-06T02:24:30.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.643+0000 7f772299d700 1 -- 172.21.15.31:0/3803511668 shutdown_connections 2023-10-06T02:24:30.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.643+0000 7f772299d700 1 --2- 172.21.15.31:0/3803511668 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f771c157420 0x7f771c0b13b0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:30.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.643+0000 7f772299d700 1 --2- 172.21.15.31:0/3803511668 >> v2:172.21.15.62:3300/0 conn(0x7f771c06d250 0x7f771c0aefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:30.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.643+0000 7f772299d700 1 -- 172.21.15.31:0/3803511668 >> 172.21.15.31:0/3803511668 conn(0x7f771c06ce80 msgr2=0x7f771c0ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:30.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.643+0000 7f772299d700 1 -- 172.21.15.31:0/3803511668 shutdown_connections 2023-10-06T02:24:30.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.643+0000 7f772299d700 1 -- 172.21.15.31:0/3803511668 wait complete. 2023-10-06T02:24:30.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.645+0000 7f772299d700 1 Processor -- start 2023-10-06T02:24:30.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.645+0000 7f772299d700 1 -- start start 2023-10-06T02:24:30.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.646+0000 7f772299d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f771c06d250 0x7f771c1577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:30.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.646+0000 7f772299d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f771c0ceac0 0x7f771c15cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:30.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.646+0000 7f772299d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f771c157420 0x7f771c152330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:30.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.646+0000 7f772299d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f771c152870 con 0x7f771c0ceac0 2023-10-06T02:24:30.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.646+0000 7f772299d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f771c158aa0 con 0x7f771c157420 2023-10-06T02:24:30.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.646+0000 7f772299d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f771c157e90 con 0x7f771c06d250 2023-10-06T02:24:30.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.646+0000 7f772119a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f771c0ceac0 0x7f771c15cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:30.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.646+0000 7f772119a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f771c0ceac0 0x7f771c15cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40192/0 (socket says 172.21.15.31:40192) 2023-10-06T02:24:30.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.646+0000 7f772119a700 1 -- 172.21.15.31:0/225254481 learned_addr learned my addr 172.21.15.31:0/225254481 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:30.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.646+0000 7f772199b700 1 --2- 172.21.15.31:0/225254481 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f771c06d250 0x7f771c1577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:30.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.646+0000 7f772219c700 1 --2- 172.21.15.31:0/225254481 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f771c157420 0x7f771c152330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:30.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.646+0000 7f772119a700 1 -- 172.21.15.31:0/225254481 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f771c06d250 msgr2=0x7f771c1577f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:30.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.646+0000 7f772119a700 1 --2- 172.21.15.31:0/225254481 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f771c06d250 0x7f771c1577f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:30.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.646+0000 7f772119a700 1 -- 172.21.15.31:0/225254481 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f771c157420 msgr2=0x7f771c152330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:30.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.646+0000 7f772119a700 1 --2- 172.21.15.31:0/225254481 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f771c157420 0x7f771c152330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:30.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.646+0000 7f772119a700 1 -- 172.21.15.31:0/225254481 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77180094f0 con 0x7f771c0ceac0 2023-10-06T02:24:30.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.647+0000 7f772119a700 1 --2- 172.21.15.31:0/225254481 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f771c0ceac0 0x7f771c15cc70 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f770c00c800 tx=0x7f770c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:30.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.647+0000 7f7711ffb700 1 -- 172.21.15.31:0/225254481 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f770c011600 con 0x7f771c0ceac0 2023-10-06T02:24:30.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.647+0000 7f7711ffb700 1 -- 172.21.15.31:0/225254481 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f770c011760 con 0x7f771c0ceac0 2023-10-06T02:24:30.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.647+0000 7f772299d700 1 -- 172.21.15.31:0/225254481 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f771c158170 con 0x7f771c0ceac0 2023-10-06T02:24:30.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.647+0000 7f7711ffb700 1 -- 172.21.15.31:0/225254481 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f770c011ae0 con 0x7f771c0ceac0 2023-10-06T02:24:30.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.647+0000 7f772299d700 1 -- 172.21.15.31:0/225254481 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f771c158730 con 0x7f771c0ceac0 2023-10-06T02:24:30.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.648+0000 7f772299d700 1 -- 172.21.15.31:0/225254481 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f771c158c50 con 0x7f771c0ceac0 2023-10-06T02:24:30.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.649+0000 7f7711ffb700 1 -- 172.21.15.31:0/225254481 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f770c021350 con 0x7f771c0ceac0 2023-10-06T02:24:30.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.650+0000 7f7711ffb700 1 --2- 172.21.15.31:0/225254481 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f77000670e0 0x7f7700069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:30.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.650+0000 7f7711ffb700 1 -- 172.21.15.31:0/225254481 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f770c0849d0 con 0x7f771c0ceac0 2023-10-06T02:24:30.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.650+0000 7f772199b700 1 --2- 172.21.15.31:0/225254481 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f77000670e0 0x7f7700069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:30.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.651+0000 7f772199b700 1 --2- 172.21.15.31:0/225254481 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f77000670e0 0x7f7700069590 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f771c06aa90 tx=0x7f7718017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:30.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.654+0000 7f7711ffb700 1 -- 172.21.15.31:0/225254481 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f771c158c50 con 0x7f771c0ceac0 2023-10-06T02:24:30.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.901+0000 7f772299d700 1 -- 172.21.15.31:0/225254481 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f771c158c10 con 0x7f771c0ceac0 2023-10-06T02:24:30.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.902+0000 7f7711ffb700 1 -- 172.21.15.31:0/225254481 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4591 (secure 0 0 0) 0x7f770c01e090 con 0x7f771c0ceac0 2023-10-06T02:24:30.902 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:24:30.902 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4700,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/384287452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":384287452},{"type":"v1","addr":"172.21.15.62:6834","nonce":384287452}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5037,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/693643908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":693643908},{"type":"v1","addr":"172.21.15.31:6836","nonce":693643908}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5040,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2599441423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2599441423},{"type":"v1","addr":"172.21.15.62:6835","nonce":2599441423}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:24:12.493929+0000","modified":"2023-10-06T02:24:16.026454+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5043},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5043":{"gid":5043,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.31:6837/2608294748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":2608294748},{"type":"v1","addr":"172.21.15.31:6837","nonce":2608294748}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T02:24:30.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.904+0000 7f772299d700 1 -- 172.21.15.31:0/225254481 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f77000670e0 msgr2=0x7f7700069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:30.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.904+0000 7f772299d700 1 --2- 172.21.15.31:0/225254481 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f77000670e0 0x7f7700069590 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f771c06aa90 tx=0x7f7718017000 comp rx=0 tx=0).stop 2023-10-06T02:24:30.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.904+0000 7f772299d700 1 -- 172.21.15.31:0/225254481 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f771c0ceac0 msgr2=0x7f771c15cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:30.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.904+0000 7f772299d700 1 --2- 172.21.15.31:0/225254481 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f771c0ceac0 0x7f771c15cc70 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f770c00c800 tx=0x7f770c008040 comp rx=0 tx=0).stop 2023-10-06T02:24:30.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.904+0000 7f772299d700 1 -- 172.21.15.31:0/225254481 shutdown_connections 2023-10-06T02:24:30.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.904+0000 7f772299d700 1 --2- 172.21.15.31:0/225254481 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f771c157420 0x7f771c152330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:30.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.904+0000 7f772299d700 1 --2- 172.21.15.31:0/225254481 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f77000670e0 0x7f7700069590 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:30.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.904+0000 7f772299d700 1 --2- 172.21.15.31:0/225254481 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f771c06d250 0x7f771c1577f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:30.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.904+0000 7f772299d700 1 --2- 172.21.15.31:0/225254481 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f771c0ceac0 0x7f771c15cc70 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:30.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.904+0000 7f772299d700 1 -- 172.21.15.31:0/225254481 >> 172.21.15.31:0/225254481 conn(0x7f771c06ce80 msgr2=0x7f771c152d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:30.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.904+0000 7f772299d700 1 -- 172.21.15.31:0/225254481 shutdown_connections 2023-10-06T02:24:30.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:30.904+0000 7f772299d700 1 -- 172.21.15.31:0/225254481 wait complete. 2023-10-06T02:24:30.906 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 18 2023-10-06T02:24:30.922 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:24:31.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.148+0000 7f4fd4b9d700 1 Processor -- start 2023-10-06T02:24:31.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.148+0000 7f4fd4b9d700 1 -- start start 2023-10-06T02:24:31.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.149+0000 7f4fd4b9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4fd013fb20 0x7f4fd015a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:31.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.149+0000 7f4fd4b9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fd015ae70 con 0x7f4fd013fef0 2023-10-06T02:24:31.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.149+0000 7f4fd4b9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fd006af60 con 0x7f4fd006aaa0 2023-10-06T02:24:31.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.149+0000 7f4fd4b9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd006b240 con 0x7f4fd013fb20 2023-10-06T02:24:31.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.149+0000 7f4fceffd700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f4fd013fef0 0x7f4fd00c7990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58078/0 (socket says 172.21.15.31:58078) 2023-10-06T02:24:31.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.150+0000 7f4fceffd700 1 -- 172.21.15.31:0/2692736676 learned_addr learned my addr 172.21.15.31:0/2692736676 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:24:31.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.150+0000 7f4fcf7fe700 1 --2- 172.21.15.31:0/2692736676 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4fd013fb20 0x7f4fd015a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:31.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.150+0000 7f4fce7fc700 1 -- 172.21.15.31:0/2692736676 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 559897545 0 0) 0x7f4fc4001290 con 0x7f4fd013fef0 2023-10-06T02:24:31.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.150+0000 7f4fce7fc700 1 -- 172.21.15.31:0/2692736676 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fb8003570 con 0x7f4fd013fef0 2023-10-06T02:24:31.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.150+0000 7f4fce7fc700 1 -- 172.21.15.31:0/2692736676 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3563312619 0 0) 0x7f4fd006af60 con 0x7f4fd006aaa0 2023-10-06T02:24:31.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.150+0000 7f4fce7fc700 1 -- 172.21.15.31:0/2692736676 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fc4001290 con 0x7f4fd006aaa0 2023-10-06T02:24:31.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.150+0000 7f4fce7fc700 1 -- 172.21.15.31:0/2692736676 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1474464559 0 0) 0x7f4fb8003570 con 0x7f4fd013fef0 2023-10-06T02:24:31.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.150+0000 7f4fce7fc700 1 -- 172.21.15.31:0/2692736676 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4fd013fb20 msgr2=0x7f4fd015a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:31.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.150+0000 7f4fce7fc700 1 --2- 172.21.15.31:0/2692736676 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4fd013fb20 0x7f4fd015a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:31.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.150+0000 7f4fce7fc700 1 -- 172.21.15.31:0/2692736676 >> v1:172.21.15.62:6789/0 conn(0x7f4fd006aaa0 legacy=0x7f4fd00c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:31.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.150+0000 7f4fce7fc700 1 -- 172.21.15.31:0/2692736676 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fd006b520 con 0x7f4fd013fef0 2023-10-06T02:24:31.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.150+0000 7f4fce7fc700 1 -- 172.21.15.31:0/2692736676 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7f4fc4001780 con 0x7f4fd013fef0 2023-10-06T02:24:31.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.150+0000 7f4fce7fc700 1 -- 172.21.15.31:0/2692736676 >> v1:172.21.15.31:6789/0 conn(0x7f4fd013fef0 legacy=0x7f4fd00c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:31.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.150+0000 7f4fce7fc700 1 --2- 172.21.15.31:0/2692736676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fb8003c20 0x7f4fb8004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:31.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.150+0000 7f4fce7fc700 1 --2- 172.21.15.31:0/2692736676 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4fd013fb20 0x7f4fb8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:31.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.150+0000 7f4fce7fc700 1 --2- 172.21.15.31:0/2692736676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fb8004d50 0x7f4fb80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:31.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.150+0000 7f4fce7fc700 1 -- 172.21.15.31:0/2692736676 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd006b240 con 0x7f4fb8003c20 2023-10-06T02:24:31.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.150+0000 7f4fce7fc700 1 -- 172.21.15.31:0/2692736676 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb800bb40 con 0x7f4fb8004d50 2023-10-06T02:24:31.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.150+0000 7f4fce7fc700 1 -- 172.21.15.31:0/2692736676 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb800bd00 con 0x7f4fd013fb20 2023-10-06T02:24:31.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.151+0000 7f4fcf7fe700 1 --2- 172.21.15.31:0/2692736676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fb8003c20 0x7f4fb8004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:31.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.151+0000 7f4fcffff700 1 --2- 172.21.15.31:0/2692736676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fb8004d50 0x7f4fb80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:31.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.151+0000 7f4fceffd700 1 --2- 172.21.15.31:0/2692736676 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4fd013fb20 0x7f4fb8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:31.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.151+0000 7f4fcf7fe700 1 -- 172.21.15.31:0/2692736676 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4fd013fb20 msgr2=0x7f4fb8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:31.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.151+0000 7f4fcf7fe700 1 --2- 172.21.15.31:0/2692736676 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4fd013fb20 0x7f4fb8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:31.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.151+0000 7f4fcf7fe700 1 -- 172.21.15.31:0/2692736676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fb8004d50 msgr2=0x7f4fb80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:31.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.151+0000 7f4fcf7fe700 1 --2- 172.21.15.31:0/2692736676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fb8004d50 0x7f4fb80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:31.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.151+0000 7f4fcf7fe700 1 -- 172.21.15.31:0/2692736676 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fb8003570 con 0x7f4fb8003c20 2023-10-06T02:24:31.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.151+0000 7f4fcffff700 1 --2- 172.21.15.31:0/2692736676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fb8004d50 0x7f4fb80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:24:31.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.151+0000 7f4fcf7fe700 1 --2- 172.21.15.31:0/2692736676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fb8003c20 0x7f4fb8004010 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f4fc0009c00 tx=0x7f4fc00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ff51079e445f62b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:31.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.151+0000 7f4fce7fc700 1 -- 172.21.15.31:0/2692736676 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc000e710 con 0x7f4fb8003c20 2023-10-06T02:24:31.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.151+0000 7f4fce7fc700 1 -- 172.21.15.31:0/2692736676 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fc000e870 con 0x7f4fb8003c20 2023-10-06T02:24:31.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.151+0000 7f4fce7fc700 1 -- 172.21.15.31:0/2692736676 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc000ec30 con 0x7f4fb8003c20 2023-10-06T02:24:31.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.152+0000 7f4fd4b9d700 1 -- 172.21.15.31:0/2692736676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fb8003c20 msgr2=0x7f4fb8004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:31.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.152+0000 7f4fd4b9d700 1 --2- 172.21.15.31:0/2692736676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fb8003c20 0x7f4fb8004010 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f4fc0009c00 tx=0x7f4fc00091a0 comp rx=0 tx=0).stop 2023-10-06T02:24:31.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.152+0000 7f4fcffff700 1 -- 172.21.15.31:0/2692736676 reap_dead start 2023-10-06T02:24:31.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.152+0000 7f4fd4b9d700 1 -- 172.21.15.31:0/2692736676 shutdown_connections 2023-10-06T02:24:31.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.152+0000 7f4fd4b9d700 1 -- 172.21.15.31:0/2692736676 >> 172.21.15.31:0/2692736676 conn(0x7f4fd006d170 msgr2=0x7f4fd00cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:31.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.152+0000 7f4fd4b9d700 1 -- 172.21.15.31:0/2692736676 shutdown_connections 2023-10-06T02:24:31.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.152+0000 7f4fd4b9d700 1 -- 172.21.15.31:0/2692736676 wait complete. 2023-10-06T02:24:31.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.154+0000 7f4fd4b9d700 1 Processor -- start 2023-10-06T02:24:31.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.154+0000 7f4fd4b9d700 1 -- start start 2023-10-06T02:24:31.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.155+0000 7f4fd4b9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4fb8003c20 0x7f4fd013ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:31.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.155+0000 7f4fd4b9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fd0159450 0x7f4fd01598c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:31.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.155+0000 7f4fd4b9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fd00c75f0 0x7f4fd00c7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:31.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.155+0000 7f4fd4b9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd00c7fa0 con 0x7f4fd00c75f0 2023-10-06T02:24:31.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.155+0000 7f4fd4b9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd0131800 con 0x7f4fd0159450 2023-10-06T02:24:31.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.155+0000 7f4fd4b9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd013f360 con 0x7f4fb8003c20 2023-10-06T02:24:31.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.155+0000 7f4fcffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fd00c75f0 0x7f4fd00c7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:31.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.155+0000 7f4fcffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fd00c75f0 0x7f4fd00c7a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40218/0 (socket says 172.21.15.31:40218) 2023-10-06T02:24:31.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.155+0000 7f4fcffff700 1 -- 172.21.15.31:0/2981881612 learned_addr learned my addr 172.21.15.31:0/2981881612 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:31.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.155+0000 7f4fcf7fe700 1 --2- 172.21.15.31:0/2981881612 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4fb8003c20 0x7f4fd013ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:31.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.155+0000 7f4fceffd700 1 --2- 172.21.15.31:0/2981881612 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fd0159450 0x7f4fd01598c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:31.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.155+0000 7f4fcffff700 1 -- 172.21.15.31:0/2981881612 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4fb8003c20 msgr2=0x7f4fd013ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:31.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.155+0000 7f4fcffff700 1 --2- 172.21.15.31:0/2981881612 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4fb8003c20 0x7f4fd013ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:31.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.155+0000 7f4fcffff700 1 -- 172.21.15.31:0/2981881612 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fd0159450 msgr2=0x7f4fd01598c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:31.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.155+0000 7f4fcffff700 1 --2- 172.21.15.31:0/2981881612 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fd0159450 0x7f4fd01598c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:31.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.155+0000 7f4fcffff700 1 -- 172.21.15.31:0/2981881612 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fd00ca260 con 0x7f4fd00c75f0 2023-10-06T02:24:31.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.156+0000 7f4fcffff700 1 --2- 172.21.15.31:0/2981881612 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fd00c75f0 0x7f4fd00c7a60 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8007e00 tx=0x7f4fc8009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:31.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.156+0000 7f4fb77fe700 1 -- 172.21.15.31:0/2981881612 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc8017070 con 0x7f4fd00c75f0 2023-10-06T02:24:31.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.156+0000 7f4fb77fe700 1 -- 172.21.15.31:0/2981881612 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fc8006ba0 con 0x7f4fd00c75f0 2023-10-06T02:24:31.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.156+0000 7f4fb77fe700 1 -- 172.21.15.31:0/2981881612 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc80032e0 con 0x7f4fd00c75f0 2023-10-06T02:24:31.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.156+0000 7f4fd4b9d700 1 -- 172.21.15.31:0/2981881612 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fd0130bf0 con 0x7f4fd00c75f0 2023-10-06T02:24:31.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.156+0000 7f4fd4b9d700 1 -- 172.21.15.31:0/2981881612 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fd01311b0 con 0x7f4fd00c75f0 2023-10-06T02:24:31.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.157+0000 7f4fd4b9d700 1 -- 172.21.15.31:0/2981881612 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fd0131970 con 0x7f4fd00c75f0 2023-10-06T02:24:31.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.158+0000 7f4fb77fe700 1 -- 172.21.15.31:0/2981881612 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f4fc80016c0 con 0x7f4fd00c75f0 2023-10-06T02:24:31.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.159+0000 7f4fb77fe700 1 --2- 172.21.15.31:0/2981881612 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f4fbc0670e0 0x7f4fbc069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:31.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.159+0000 7f4fb77fe700 1 -- 172.21.15.31:0/2981881612 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f4fc80844b0 con 0x7f4fd00c75f0 2023-10-06T02:24:31.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.160+0000 7f4fcf7fe700 1 --2- 172.21.15.31:0/2981881612 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f4fbc0670e0 0x7f4fbc069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:31.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.160+0000 7f4fcf7fe700 1 --2- 172.21.15.31:0/2981881612 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f4fbc0670e0 0x7f4fbc069590 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4fc00091e0 tx=0x7f4fc001a000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:31.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.164+0000 7f4fb77fe700 1 -- 172.21.15.31:0/2981881612 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4fc8054d30 con 0x7f4fd00c75f0 2023-10-06T02:24:31.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.409+0000 7f4fd4b9d700 1 -- 172.21.15.31:0/2981881612 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4fd004c3c0 con 0x7f4fd00c75f0 2023-10-06T02:24:31.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.410+0000 7f4fb77fe700 1 -- 172.21.15.31:0/2981881612 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4591 (secure 0 0 0) 0x7f4fc80548c0 con 0x7f4fd00c75f0 2023-10-06T02:24:31.410 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:24:31.410 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4700,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/384287452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":384287452},{"type":"v1","addr":"172.21.15.62:6834","nonce":384287452}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5037,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/693643908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":693643908},{"type":"v1","addr":"172.21.15.31:6836","nonce":693643908}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5040,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2599441423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2599441423},{"type":"v1","addr":"172.21.15.62:6835","nonce":2599441423}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:24:12.493929+0000","modified":"2023-10-06T02:24:16.026454+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5043},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5043":{"gid":5043,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.31:6837/2608294748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":2608294748},{"type":"v1","addr":"172.21.15.31:6837","nonce":2608294748}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T02:24:31.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.412+0000 7f4fd4b9d700 1 -- 172.21.15.31:0/2981881612 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f4fbc0670e0 msgr2=0x7f4fbc069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:31.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.412+0000 7f4fd4b9d700 1 --2- 172.21.15.31:0/2981881612 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f4fbc0670e0 0x7f4fbc069590 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4fc00091e0 tx=0x7f4fc001a000 comp rx=0 tx=0).stop 2023-10-06T02:24:31.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.412+0000 7f4fd4b9d700 1 -- 172.21.15.31:0/2981881612 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fd00c75f0 msgr2=0x7f4fd00c7a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:31.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.412+0000 7f4fd4b9d700 1 --2- 172.21.15.31:0/2981881612 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fd00c75f0 0x7f4fd00c7a60 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8007e00 tx=0x7f4fc8009040 comp rx=0 tx=0).stop 2023-10-06T02:24:31.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.412+0000 7f4fd4b9d700 1 -- 172.21.15.31:0/2981881612 shutdown_connections 2023-10-06T02:24:31.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.412+0000 7f4fd4b9d700 1 --2- 172.21.15.31:0/2981881612 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fd00c75f0 0x7f4fd00c7a60 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:31.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.412+0000 7f4fd4b9d700 1 --2- 172.21.15.31:0/2981881612 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f4fbc0670e0 0x7f4fbc069590 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:31.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.412+0000 7f4fd4b9d700 1 --2- 172.21.15.31:0/2981881612 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4fb8003c20 0x7f4fd013ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:31.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.412+0000 7f4fd4b9d700 1 --2- 172.21.15.31:0/2981881612 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fd0159450 0x7f4fd01598c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:31.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.412+0000 7f4fd4b9d700 1 -- 172.21.15.31:0/2981881612 >> 172.21.15.31:0/2981881612 conn(0x7f4fd006d170 msgr2=0x7f4fd00b1880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:31.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.412+0000 7f4fd4b9d700 1 -- 172.21.15.31:0/2981881612 shutdown_connections 2023-10-06T02:24:31.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.412+0000 7f4fd4b9d700 1 -- 172.21.15.31:0/2981881612 wait complete. 2023-10-06T02:24:31.414 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 18 2023-10-06T02:24:31.430 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:24:31.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.615+0000 7ff44979d700 1 Processor -- start 2023-10-06T02:24:31.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.615+0000 7ff44979d700 1 -- start start 2023-10-06T02:24:31.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.617+0000 7ff44979d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff44406f3a0 0x7ff44414fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:31.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.617+0000 7ff44979d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff44406a490 con 0x7ff44406e470 2023-10-06T02:24:31.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.617+0000 7ff44979d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4440c7230 con 0x7ff44406e0a0 2023-10-06T02:24:31.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.617+0000 7ff44979d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4440c7490 con 0x7ff44406f3a0 2023-10-06T02:24:31.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.617+0000 7ff448f9c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7ff44406e470 0x7ff44406a920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58080/0 (socket says 172.21.15.31:58080) 2023-10-06T02:24:31.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.617+0000 7ff448f9c700 1 -- 172.21.15.31:0/3073298039 learned_addr learned my addr 172.21.15.31:0/3073298039 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:24:31.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.617+0000 7ff4437fe700 1 --2- 172.21.15.31:0/3073298039 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff44406f3a0 0x7ff44414fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:31.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.618+0000 7ff442ffd700 1 -- 172.21.15.31:0/3073298039 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1277196429 0 0) 0x7ff44406a490 con 0x7ff44406e470 2023-10-06T02:24:31.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.618+0000 7ff442ffd700 1 -- 172.21.15.31:0/3073298039 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff42c003570 con 0x7ff44406e470 2023-10-06T02:24:31.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.618+0000 7ff442ffd700 1 -- 172.21.15.31:0/3073298039 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 5902063 0 0) 0x7ff4440c7230 con 0x7ff44406e0a0 2023-10-06T02:24:31.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.618+0000 7ff442ffd700 1 -- 172.21.15.31:0/3073298039 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff44406a490 con 0x7ff44406e0a0 2023-10-06T02:24:31.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.618+0000 7ff4437fe700 1 -- 172.21.15.31:0/3073298039 >> v1:172.21.15.62:6789/0 conn(0x7ff44406e0a0 legacy=0x7ff4440c6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:31.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.618+0000 7ff4437fe700 1 -- 172.21.15.31:0/3073298039 >> v1:172.21.15.31:6789/0 conn(0x7ff44406e470 legacy=0x7ff44406a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:31.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.618+0000 7ff4437fe700 1 -- 172.21.15.31:0/3073298039 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4440c7770 con 0x7ff44406f3a0 2023-10-06T02:24:31.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.618+0000 7ff4437fe700 1 --2- 172.21.15.31:0/3073298039 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff44406f3a0 0x7ff44414fc70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff43800aaa0 tx=0x7ff43800a2e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19ec875687a0bebf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:31.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.618+0000 7ff442ffd700 1 -- 172.21.15.31:0/3073298039 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff438005c10 con 0x7ff44406f3a0 2023-10-06T02:24:31.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.618+0000 7ff442ffd700 1 -- 172.21.15.31:0/3073298039 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff438005d70 con 0x7ff44406f3a0 2023-10-06T02:24:31.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.618+0000 7ff442ffd700 1 -- 172.21.15.31:0/3073298039 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff43800c970 con 0x7ff44406f3a0 2023-10-06T02:24:31.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.619+0000 7ff44979d700 1 -- 172.21.15.31:0/3073298039 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff44406f3a0 msgr2=0x7ff44414fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:31.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.619+0000 7ff44979d700 1 --2- 172.21.15.31:0/3073298039 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff44406f3a0 0x7ff44414fc70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff43800aaa0 tx=0x7ff43800a2e0 comp rx=0 tx=0).stop 2023-10-06T02:24:31.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.619+0000 7ff44979d700 1 -- 172.21.15.31:0/3073298039 shutdown_connections 2023-10-06T02:24:31.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.619+0000 7ff44979d700 1 --2- 172.21.15.31:0/3073298039 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff44406f3a0 0x7ff44414fc70 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:31.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.619+0000 7ff44979d700 1 -- 172.21.15.31:0/3073298039 >> 172.21.15.31:0/3073298039 conn(0x7ff44406d170 msgr2=0x7ff4440cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:31.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.619+0000 7ff44979d700 1 -- 172.21.15.31:0/3073298039 shutdown_connections 2023-10-06T02:24:31.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.619+0000 7ff44979d700 1 -- 172.21.15.31:0/3073298039 wait complete. 2023-10-06T02:24:31.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.620+0000 7ff44979d700 1 Processor -- start 2023-10-06T02:24:31.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.621+0000 7ff44979d700 1 -- start start 2023-10-06T02:24:31.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.621+0000 7ff44979d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff44406e0a0 0x7ff4440c31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:31.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.621+0000 7ff44979d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff44406e470 0x7ff444158f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:31.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.621+0000 7ff44979d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff44406f3a0 0x7ff4440c4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:31.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.621+0000 7ff44979d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff444157610 con 0x7ff44406e470 2023-10-06T02:24:31.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.621+0000 7ff44979d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4440c5110 con 0x7ff44406e0a0 2023-10-06T02:24:31.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.621+0000 7ff44979d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff444156a00 con 0x7ff44406f3a0 2023-10-06T02:24:31.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.622+0000 7ff4437fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff44406e470 0x7ff444158f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:31.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.622+0000 7ff4437fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff44406e470 0x7ff444158f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40222/0 (socket says 172.21.15.31:40222) 2023-10-06T02:24:31.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.622+0000 7ff4437fe700 1 -- 172.21.15.31:0/2690574615 learned_addr learned my addr 172.21.15.31:0/2690574615 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:31.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.622+0000 7ff443fff700 1 --2- 172.21.15.31:0/2690574615 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff44406e0a0 0x7ff4440c31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:31.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.622+0000 7ff448f9c700 1 --2- 172.21.15.31:0/2690574615 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff44406f3a0 0x7ff4440c4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:31.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.622+0000 7ff4437fe700 1 -- 172.21.15.31:0/2690574615 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff44406f3a0 msgr2=0x7ff4440c4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:31.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.622+0000 7ff4437fe700 1 --2- 172.21.15.31:0/2690574615 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff44406f3a0 0x7ff4440c4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:31.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.622+0000 7ff4437fe700 1 -- 172.21.15.31:0/2690574615 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff44406e0a0 msgr2=0x7ff4440c31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:31.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.622+0000 7ff4437fe700 1 --2- 172.21.15.31:0/2690574615 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff44406e0a0 0x7ff4440c31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:31.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.622+0000 7ff4437fe700 1 -- 172.21.15.31:0/2690574615 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff43800a560 con 0x7ff44406e470 2023-10-06T02:24:31.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.622+0000 7ff4437fe700 1 --2- 172.21.15.31:0/2690574615 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff44406e470 0x7ff444158f30 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ff438013000 tx=0x7ff438019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:31.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.623+0000 7ff42bfff700 1 -- 172.21.15.31:0/2690574615 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff43801f070 con 0x7ff44406e470 2023-10-06T02:24:31.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.623+0000 7ff42bfff700 1 -- 172.21.15.31:0/2690574615 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff438002f30 con 0x7ff44406e470 2023-10-06T02:24:31.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.623+0000 7ff42bfff700 1 -- 172.21.15.31:0/2690574615 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff438005c10 con 0x7ff44406e470 2023-10-06T02:24:31.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.623+0000 7ff44979d700 1 -- 172.21.15.31:0/2690574615 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff444156ce0 con 0x7ff44406e470 2023-10-06T02:24:31.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.623+0000 7ff44979d700 1 -- 172.21.15.31:0/2690574615 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4441572a0 con 0x7ff44406e470 2023-10-06T02:24:31.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.624+0000 7ff44979d700 1 -- 172.21.15.31:0/2690574615 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff44404c3c0 con 0x7ff44406e470 2023-10-06T02:24:31.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.625+0000 7ff42bfff700 1 -- 172.21.15.31:0/2690574615 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7ff438019660 con 0x7ff44406e470 2023-10-06T02:24:31.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.625+0000 7ff42bfff700 1 --2- 172.21.15.31:0/2690574615 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7ff4300670e0 0x7ff430069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:31.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.626+0000 7ff42bfff700 1 -- 172.21.15.31:0/2690574615 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff438085a30 con 0x7ff44406e470 2023-10-06T02:24:31.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.626+0000 7ff443fff700 1 --2- 172.21.15.31:0/2690574615 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7ff4300670e0 0x7ff430069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:31.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.626+0000 7ff443fff700 1 --2- 172.21.15.31:0/2690574615 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7ff4300670e0 0x7ff430069590 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff434007fd0 tx=0x7ff434007d50 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:31.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.630+0000 7ff42bfff700 1 -- 172.21.15.31:0/2690574615 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff4380527c0 con 0x7ff44406e470 2023-10-06T02:24:31.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.835+0000 7ff44979d700 1 -- 172.21.15.31:0/2690574615 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff4440c7a50 con 0x7ff44406e470 2023-10-06T02:24:31.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.836+0000 7ff42bfff700 1 -- 172.21.15.31:0/2690574615 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+16818 (secure 0 0 0) 0x7ff438028090 con 0x7ff44406e470 2023-10-06T02:24:31.837 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:24:31.837 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":36,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:24:14.007126+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:24:10.026011+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-06T02:24:11.471970+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:24:31.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.838+0000 7ff44979d700 1 -- 172.21.15.31:0/2690574615 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7ff4300670e0 msgr2=0x7ff430069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:31.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.838+0000 7ff44979d700 1 --2- 172.21.15.31:0/2690574615 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7ff4300670e0 0x7ff430069590 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff434007fd0 tx=0x7ff434007d50 comp rx=0 tx=0).stop 2023-10-06T02:24:31.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.838+0000 7ff44979d700 1 -- 172.21.15.31:0/2690574615 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff44406e470 msgr2=0x7ff444158f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:31.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.838+0000 7ff44979d700 1 --2- 172.21.15.31:0/2690574615 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff44406e470 0x7ff444158f30 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ff438013000 tx=0x7ff438019000 comp rx=0 tx=0).stop 2023-10-06T02:24:31.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.838+0000 7ff44979d700 1 -- 172.21.15.31:0/2690574615 shutdown_connections 2023-10-06T02:24:31.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.838+0000 7ff44979d700 1 --2- 172.21.15.31:0/2690574615 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7ff4300670e0 0x7ff430069590 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:31.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.838+0000 7ff44979d700 1 --2- 172.21.15.31:0/2690574615 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff44406f3a0 0x7ff4440c4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:31.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.838+0000 7ff44979d700 1 --2- 172.21.15.31:0/2690574615 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff44406e0a0 0x7ff4440c31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:31.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.838+0000 7ff44979d700 1 --2- 172.21.15.31:0/2690574615 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff44406e470 0x7ff444158f30 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:31.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.838+0000 7ff44979d700 1 -- 172.21.15.31:0/2690574615 >> 172.21.15.31:0/2690574615 conn(0x7ff44406d170 msgr2=0x7ff44414d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:31.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.838+0000 7ff44979d700 1 -- 172.21.15.31:0/2690574615 shutdown_connections 2023-10-06T02:24:31.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:31.839+0000 7ff44979d700 1 -- 172.21.15.31:0/2690574615 wait complete. 2023-10-06T02:24:31.856 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T02:24:32.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.044+0000 7fe422d9d700 1 Processor -- start 2023-10-06T02:24:32.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.045+0000 7fe422d9d700 1 -- start start 2023-10-06T02:24:32.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.046+0000 7fe422d9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe41c142c00 0x7fe41c06b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:32.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.046+0000 7fe422d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe41c06bd20 0x7fe41c0c7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:32.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.046+0000 7fe422d9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe41c0ca480 con 0x7fe41c142fd0 2023-10-06T02:24:32.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.046+0000 7fe422d9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe41c15e240 con 0x7fe41c142c00 2023-10-06T02:24:32.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.046+0000 7fe422d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe41c06b350 con 0x7fe41c06bd20 2023-10-06T02:24:32.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.046+0000 7fe421d9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe41c142c00 0x7fe41c06b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:32.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.046+0000 7fe421d9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe41c142c00 0x7fe41c06b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40236/0 (socket says 172.21.15.31:40236) 2023-10-06T02:24:32.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.046+0000 7fe421d9b700 1 -- 172.21.15.31:0/337002867 learned_addr learned my addr 172.21.15.31:0/337002867 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:32.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.047+0000 7fe42159a700 1 --2- 172.21.15.31:0/337002867 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe41c06bd20 0x7fe41c0c7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:32.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.047+0000 7fe421d9b700 1 -- 172.21.15.31:0/337002867 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe41c06bd20 msgr2=0x7fe41c0c7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:32.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.047+0000 7fe421d9b700 1 --2- 172.21.15.31:0/337002867 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe41c06bd20 0x7fe41c0c7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:32.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.047+0000 7fe421d9b700 1 -- 172.21.15.31:0/337002867 >> v1:172.21.15.62:6789/0 conn(0x7fe41c142fd0 legacy=0x7fe41c06c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:32.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.047+0000 7fe421d9b700 1 -- 172.21.15.31:0/337002867 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe41c15df60 con 0x7fe41c142c00 2023-10-06T02:24:32.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.047+0000 7fe42159a700 1 --2- 172.21.15.31:0/337002867 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe41c06bd20 0x7fe41c0c7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:24:32.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.047+0000 7fe421d9b700 1 --2- 172.21.15.31:0/337002867 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe41c142c00 0x7fe41c06b7e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fe4180099b0 tx=0x7fe418009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aead403925fc2d5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:32.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.047+0000 7fe420d99700 1 -- 172.21.15.31:0/337002867 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe41800b820 con 0x7fe41c142c00 2023-10-06T02:24:32.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.047+0000 7fe420d99700 1 -- 172.21.15.31:0/337002867 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe41800b980 con 0x7fe41c142c00 2023-10-06T02:24:32.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.047+0000 7fe420d99700 1 -- 172.21.15.31:0/337002867 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe41800bd40 con 0x7fe41c142c00 2023-10-06T02:24:32.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.048+0000 7fe422d9d700 1 -- 172.21.15.31:0/337002867 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe41c142c00 msgr2=0x7fe41c06b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:32.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.048+0000 7fe422d9d700 1 --2- 172.21.15.31:0/337002867 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe41c142c00 0x7fe41c06b7e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fe4180099b0 tx=0x7fe418009250 comp rx=0 tx=0).stop 2023-10-06T02:24:32.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.048+0000 7fe422d9d700 1 -- 172.21.15.31:0/337002867 shutdown_connections 2023-10-06T02:24:32.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.048+0000 7fe422d9d700 1 --2- 172.21.15.31:0/337002867 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe41c142c00 0x7fe41c06b7e0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:32.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.048+0000 7fe422d9d700 1 --2- 172.21.15.31:0/337002867 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe41c06bd20 0x7fe41c0c7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:32.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.048+0000 7fe422d9d700 1 -- 172.21.15.31:0/337002867 >> 172.21.15.31:0/337002867 conn(0x7fe41c0675a0 msgr2=0x7fe41c0679a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:32.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.048+0000 7fe422d9d700 1 -- 172.21.15.31:0/337002867 shutdown_connections 2023-10-06T02:24:32.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.048+0000 7fe422d9d700 1 -- 172.21.15.31:0/337002867 wait complete. 2023-10-06T02:24:32.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.049+0000 7fe422d9d700 1 Processor -- start 2023-10-06T02:24:32.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.049+0000 7fe422d9d700 1 -- start start 2023-10-06T02:24:32.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.050+0000 7fe422d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe41c06bd20 0x7fe41c13f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:32.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.050+0000 7fe422d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe41c142c00 0x7fe41c0c2470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:32.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.050+0000 7fe422d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe41c142fd0 0x7fe41c0c5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:32.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.050+0000 7fe422d9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe41c13fe90 con 0x7fe41c142c00 2023-10-06T02:24:32.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.050+0000 7fe422d9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe41c13e5f0 con 0x7fe41c06bd20 2023-10-06T02:24:32.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.050+0000 7fe422d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe41c13e8d0 con 0x7fe41c142fd0 2023-10-06T02:24:32.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.051+0000 7fe42159a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe41c142c00 0x7fe41c0c2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:32.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.051+0000 7fe42159a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe41c142c00 0x7fe41c0c2470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40252/0 (socket says 172.21.15.31:40252) 2023-10-06T02:24:32.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.051+0000 7fe42159a700 1 -- 172.21.15.31:0/262332995 learned_addr learned my addr 172.21.15.31:0/262332995 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:32.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.051+0000 7fe421d9b700 1 --2- 172.21.15.31:0/262332995 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe41c06bd20 0x7fe41c13f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:32.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.051+0000 7fe42259c700 1 --2- 172.21.15.31:0/262332995 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe41c142fd0 0x7fe41c0c5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:32.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.051+0000 7fe42159a700 1 -- 172.21.15.31:0/262332995 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe41c142fd0 msgr2=0x7fe41c0c5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:32.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.051+0000 7fe42159a700 1 --2- 172.21.15.31:0/262332995 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe41c142fd0 0x7fe41c0c5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:32.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.051+0000 7fe42159a700 1 -- 172.21.15.31:0/262332995 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe41c06bd20 msgr2=0x7fe41c13f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:32.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.051+0000 7fe42159a700 1 --2- 172.21.15.31:0/262332995 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe41c06bd20 0x7fe41c13f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:32.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.051+0000 7fe42159a700 1 -- 172.21.15.31:0/262332995 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4180094f0 con 0x7fe41c142c00 2023-10-06T02:24:32.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.051+0000 7fe421d9b700 1 --2- 172.21.15.31:0/262332995 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe41c06bd20 0x7fe41c13f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:24:32.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.051+0000 7fe42259c700 1 --2- 172.21.15.31:0/262332995 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe41c142fd0 0x7fe41c0c5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:24:32.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.051+0000 7fe42159a700 1 --2- 172.21.15.31:0/262332995 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe41c142c00 0x7fe41c0c2470 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fe40c00c830 tx=0x7fe40c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:32.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.051+0000 7fe411ffb700 1 -- 172.21.15.31:0/262332995 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe40c011600 con 0x7fe41c142c00 2023-10-06T02:24:32.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.052+0000 7fe411ffb700 1 -- 172.21.15.31:0/262332995 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe40c011760 con 0x7fe41c142c00 2023-10-06T02:24:32.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.052+0000 7fe422d9d700 1 -- 172.21.15.31:0/262332995 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe41c13ebb0 con 0x7fe41c142c00 2023-10-06T02:24:32.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.052+0000 7fe411ffb700 1 -- 172.21.15.31:0/262332995 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe40c011b20 con 0x7fe41c142c00 2023-10-06T02:24:32.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.052+0000 7fe422d9d700 1 -- 172.21.15.31:0/262332995 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe41c13f450 con 0x7fe41c142c00 2023-10-06T02:24:32.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.053+0000 7fe422d9d700 1 -- 172.21.15.31:0/262332995 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe41c04c3c0 con 0x7fe41c142c00 2023-10-06T02:24:32.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.053+0000 7fe411ffb700 1 -- 172.21.15.31:0/262332995 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7fe40c00a3e0 con 0x7fe41c142c00 2023-10-06T02:24:32.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.054+0000 7fe411ffb700 1 --2- 172.21.15.31:0/262332995 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe4000670e0 0x7fe400069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:32.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.054+0000 7fe411ffb700 1 -- 172.21.15.31:0/262332995 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe40c085060 con 0x7fe41c142c00 2023-10-06T02:24:32.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.055+0000 7fe421d9b700 1 --2- 172.21.15.31:0/262332995 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe4000670e0 0x7fe400069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:32.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.055+0000 7fe421d9b700 1 --2- 172.21.15.31:0/262332995 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe4000670e0 0x7fe400069590 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe4180099b0 tx=0x7fe418016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:32.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.058+0000 7fe411ffb700 1 -- 172.21.15.31:0/262332995 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe40c051d40 con 0x7fe41c142c00 2023-10-06T02:24:32.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.308+0000 7fe422d9d700 1 -- 172.21.15.31:0/262332995 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fe41c13ed40 con 0x7fe41c142c00 2023-10-06T02:24:32.308 INFO:tasks.ceph.mon.a.smithi031.stderr:2023-10-06T02:24:32.308+0000 7f6de01f1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T02:24:32.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.315+0000 7fe411ffb700 1 -- 172.21.15.31:0/262332995 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v19) v1 ==== 161+0+0 (secure 0 0 0) 0x7fe40c015090 con 0x7fe41c142c00 2023-10-06T02:24:32.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.317+0000 7fe422d9d700 1 -- 172.21.15.31:0/262332995 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe4000670e0 msgr2=0x7fe400069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:32.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.317+0000 7fe422d9d700 1 --2- 172.21.15.31:0/262332995 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe4000670e0 0x7fe400069590 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe4180099b0 tx=0x7fe418016000 comp rx=0 tx=0).stop 2023-10-06T02:24:32.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.317+0000 7fe422d9d700 1 -- 172.21.15.31:0/262332995 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe41c142c00 msgr2=0x7fe41c0c2470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:32.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.317+0000 7fe422d9d700 1 --2- 172.21.15.31:0/262332995 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe41c142c00 0x7fe41c0c2470 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fe40c00c830 tx=0x7fe40c008040 comp rx=0 tx=0).stop 2023-10-06T02:24:32.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.317+0000 7fe422d9d700 1 -- 172.21.15.31:0/262332995 shutdown_connections 2023-10-06T02:24:32.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.317+0000 7fe422d9d700 1 --2- 172.21.15.31:0/262332995 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe4000670e0 0x7fe400069590 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:32.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.317+0000 7fe422d9d700 1 --2- 172.21.15.31:0/262332995 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe41c142fd0 0x7fe41c0c5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:32.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.317+0000 7fe422d9d700 1 --2- 172.21.15.31:0/262332995 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe41c06bd20 0x7fe41c13f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:32.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.317+0000 7fe422d9d700 1 --2- 172.21.15.31:0/262332995 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe41c142c00 0x7fe41c0c2470 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:32.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.317+0000 7fe422d9d700 1 -- 172.21.15.31:0/262332995 >> 172.21.15.31:0/262332995 conn(0x7fe41c0675a0 msgr2=0x7fe41c0ca910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:32.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.318+0000 7fe422d9d700 1 -- 172.21.15.31:0/262332995 shutdown_connections 2023-10-06T02:24:32.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.318+0000 7fe422d9d700 1 -- 172.21.15.31:0/262332995 wait complete. 2023-10-06T02:24:32.319 INFO:teuthology.orchestra.run.smithi031.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T02:24:32.337 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T02:24:32.379 INFO:tasks.ceph.mds.c.smithi031.stdout:starting mds.c at 2023-10-06T02:24:32.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.530+0000 7f6c32f9d700 1 Processor -- start 2023-10-06T02:24:32.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.530+0000 7f6c32f9d700 1 -- start start 2023-10-06T02:24:32.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.532+0000 7f6c32f9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6c2c0c7fb0 0x7f6c2c06abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:32.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.532+0000 7f6c32f9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c2c06b290 con 0x7f6c2c13f780 2023-10-06T02:24:32.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.532+0000 7f6c32f9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c2c06b570 con 0x7f6c2c0c7be0 2023-10-06T02:24:32.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.532+0000 7f6c32f9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c2c06bf10 con 0x7f6c2c0c7fb0 2023-10-06T02:24:32.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.532+0000 7f6c3179a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6c2c0c7fb0 0x7f6c2c06abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:32.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.532+0000 7f6c3279c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f6c2c13f780 0x7f6c2c13fba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58084/0 (socket says 172.21.15.31:58084) 2023-10-06T02:24:32.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.532+0000 7f6c3279c700 1 -- 172.21.15.31:0/639263306 learned_addr learned my addr 172.21.15.31:0/639263306 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:24:32.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.533+0000 7f6c3179a700 1 -- 172.21.15.31:0/639263306 >> v1:172.21.15.62:6789/0 conn(0x7f6c2c0c7be0 legacy=0x7f6c2c15a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:32.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.533+0000 7f6c3179a700 1 -- 172.21.15.31:0/639263306 >> v1:172.21.15.31:6789/0 conn(0x7f6c2c13f780 legacy=0x7f6c2c13fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:32.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.533+0000 7f6c3179a700 1 -- 172.21.15.31:0/639263306 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c2c06b850 con 0x7f6c2c0c7fb0 2023-10-06T02:24:32.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.533+0000 7f6c30f99700 1 -- 172.21.15.31:0/639263306 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1581766832 0 0) 0x7f6c2c06b290 con 0x7f6c2c13f780 2023-10-06T02:24:32.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.533+0000 7f6c3179a700 1 --2- 172.21.15.31:0/639263306 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c2c0c7fb0 0x7f6c2c06abd0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c0098e0 tx=0x7f6c1c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5a9ad4656b9f169 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:32.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.533+0000 7f6c30f99700 1 -- 172.21.15.31:0/639263306 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c009e80 con 0x7f6c2c0c7fb0 2023-10-06T02:24:32.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.533+0000 7f6c30f99700 1 -- 172.21.15.31:0/639263306 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c1c00b860 con 0x7f6c2c0c7fb0 2023-10-06T02:24:32.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.533+0000 7f6c30f99700 1 -- 172.21.15.31:0/639263306 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c00bbe0 con 0x7f6c2c0c7fb0 2023-10-06T02:24:32.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.534+0000 7f6c32f9d700 1 -- 172.21.15.31:0/639263306 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c2c0c7fb0 msgr2=0x7f6c2c06abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:32.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.534+0000 7f6c32f9d700 1 --2- 172.21.15.31:0/639263306 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c2c0c7fb0 0x7f6c2c06abd0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c0098e0 tx=0x7f6c1c0091a0 comp rx=0 tx=0).stop 2023-10-06T02:24:32.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.534+0000 7f6c32f9d700 1 -- 172.21.15.31:0/639263306 shutdown_connections 2023-10-06T02:24:32.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.534+0000 7f6c32f9d700 1 --2- 172.21.15.31:0/639263306 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c2c0c7fb0 0x7f6c2c06abd0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:32.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.534+0000 7f6c32f9d700 1 -- 172.21.15.31:0/639263306 >> 172.21.15.31:0/639263306 conn(0x7f6c2c06d110 msgr2=0x7f6c2c0cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:32.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.534+0000 7f6c32f9d700 1 -- 172.21.15.31:0/639263306 shutdown_connections 2023-10-06T02:24:32.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.534+0000 7f6c32f9d700 1 -- 172.21.15.31:0/639263306 wait complete. 2023-10-06T02:24:32.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.535+0000 7f6c32f9d700 1 Processor -- start 2023-10-06T02:24:32.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.535+0000 7f6c32f9d700 1 -- start start 2023-10-06T02:24:32.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.536+0000 7f6c32f9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c2c0c7be0 0x7f6c2c0c3430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:32.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.536+0000 7f6c32f9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c2c0c7fb0 0x7f6c2c15c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:32.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.536+0000 7f6c32f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6c2c13f780 0x7f6c2c155410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:32.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.536+0000 7f6c32f9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2c0c51f0 con 0x7f6c2c0c7be0 2023-10-06T02:24:32.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.536+0000 7f6c32f9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2c0c5750 con 0x7f6c2c0c7fb0 2023-10-06T02:24:32.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.536+0000 7f6c32f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2c15d050 con 0x7f6c2c13f780 2023-10-06T02:24:32.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.536+0000 7f6c31f9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c2c0c7be0 0x7f6c2c0c3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:32.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.536+0000 7f6c31f9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c2c0c7be0 0x7f6c2c0c3430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40294/0 (socket says 172.21.15.31:40294) 2023-10-06T02:24:32.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.536+0000 7f6c31f9b700 1 -- 172.21.15.31:0/124229488 learned_addr learned my addr 172.21.15.31:0/124229488 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:32.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.537+0000 7f6c3179a700 1 --2- 172.21.15.31:0/124229488 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c2c0c7fb0 0x7f6c2c15c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:32.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.537+0000 7f6c3279c700 1 --2- 172.21.15.31:0/124229488 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6c2c13f780 0x7f6c2c155410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:32.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.537+0000 7f6c31f9b700 1 -- 172.21.15.31:0/124229488 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6c2c13f780 msgr2=0x7f6c2c155410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:32.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.537+0000 7f6c31f9b700 1 --2- 172.21.15.31:0/124229488 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6c2c13f780 0x7f6c2c155410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:32.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.537+0000 7f6c31f9b700 1 -- 172.21.15.31:0/124229488 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c2c0c7fb0 msgr2=0x7f6c2c15c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:32.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.537+0000 7f6c31f9b700 1 --2- 172.21.15.31:0/124229488 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c2c0c7fb0 0x7f6c2c15c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:32.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.537+0000 7f6c31f9b700 1 -- 172.21.15.31:0/124229488 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c1c009420 con 0x7f6c2c0c7be0 2023-10-06T02:24:32.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.537+0000 7f6c3179a700 1 --2- 172.21.15.31:0/124229488 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c2c0c7fb0 0x7f6c2c15c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:24:32.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.537+0000 7f6c31f9b700 1 --2- 172.21.15.31:0/124229488 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c2c0c7be0 0x7f6c2c0c3430 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f6c2800c590 tx=0x7f6c2800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:32.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.537+0000 7f6c21ffb700 1 -- 172.21.15.31:0/124229488 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c28011590 con 0x7f6c2c0c7be0 2023-10-06T02:24:32.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.537+0000 7f6c21ffb700 1 -- 172.21.15.31:0/124229488 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c280116f0 con 0x7f6c2c0c7be0 2023-10-06T02:24:32.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.537+0000 7f6c21ffb700 1 -- 172.21.15.31:0/124229488 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c28011ab0 con 0x7f6c2c0c7be0 2023-10-06T02:24:32.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.537+0000 7f6c32f9d700 1 -- 172.21.15.31:0/124229488 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c2c0c5420 con 0x7f6c2c0c7be0 2023-10-06T02:24:32.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.538+0000 7f6c32f9d700 1 -- 172.21.15.31:0/124229488 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c2c15cb40 con 0x7f6c2c0c7be0 2023-10-06T02:24:32.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.538+0000 7f6c32f9d700 1 -- 172.21.15.31:0/124229488 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c2c04c3c0 con 0x7f6c2c0c7be0 2023-10-06T02:24:32.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.539+0000 7f6c21ffb700 1 -- 172.21.15.31:0/124229488 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f6c2801b3b0 con 0x7f6c2c0c7be0 2023-10-06T02:24:32.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.540+0000 7f6c21ffb700 1 --2- 172.21.15.31:0/124229488 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6c100670e0 0x7f6c10069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:32.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.540+0000 7f6c21ffb700 1 -- 172.21.15.31:0/124229488 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f6c28084020 con 0x7f6c2c0c7be0 2023-10-06T02:24:32.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.540+0000 7f6c3179a700 1 --2- 172.21.15.31:0/124229488 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6c100670e0 0x7f6c10069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:32.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.541+0000 7f6c3179a700 1 --2- 172.21.15.31:0/124229488 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6c100670e0 0x7f6c10069590 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c0093f0 tx=0x7f6c1c017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:32.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.544+0000 7f6c21ffb700 1 -- 172.21.15.31:0/124229488 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6c28050de0 con 0x7f6c2c0c7be0 2023-10-06T02:24:32.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:32.792+0000 7f6c32f9d700 1 -- 172.21.15.31:0/124229488 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f6c2c13d290 con 0x7f6c2c0c7be0 2023-10-06T02:24:33.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:33.322+0000 7f6c21ffb700 1 -- 172.21.15.31:0/124229488 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v20) v1 ==== 104+0+0 (secure 0 0 0) 0x7f6c280170a0 con 0x7f6c2c0c7be0 2023-10-06T02:24:33.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:33.325+0000 7f6c32f9d700 1 -- 172.21.15.31:0/124229488 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6c100670e0 msgr2=0x7f6c10069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:33.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:33.325+0000 7f6c32f9d700 1 --2- 172.21.15.31:0/124229488 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6c100670e0 0x7f6c10069590 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c0093f0 tx=0x7f6c1c017000 comp rx=0 tx=0).stop 2023-10-06T02:24:33.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:33.325+0000 7f6c32f9d700 1 -- 172.21.15.31:0/124229488 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c2c0c7be0 msgr2=0x7f6c2c0c3430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:33.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:33.325+0000 7f6c32f9d700 1 --2- 172.21.15.31:0/124229488 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c2c0c7be0 0x7f6c2c0c3430 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f6c2800c590 tx=0x7f6c2800c950 comp rx=0 tx=0).stop 2023-10-06T02:24:33.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:33.326+0000 7f6c32f9d700 1 -- 172.21.15.31:0/124229488 shutdown_connections 2023-10-06T02:24:33.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:33.326+0000 7f6c32f9d700 1 --2- 172.21.15.31:0/124229488 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6c100670e0 0x7f6c10069590 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:33.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:33.326+0000 7f6c32f9d700 1 --2- 172.21.15.31:0/124229488 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6c2c13f780 0x7f6c2c155410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:33.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:33.326+0000 7f6c32f9d700 1 --2- 172.21.15.31:0/124229488 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c2c0c7be0 0x7f6c2c0c3430 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:33.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:33.326+0000 7f6c32f9d700 1 --2- 172.21.15.31:0/124229488 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c2c0c7fb0 0x7f6c2c15c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:33.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:33.326+0000 7f6c32f9d700 1 -- 172.21.15.31:0/124229488 >> 172.21.15.31:0/124229488 conn(0x7f6c2c06d110 msgr2=0x7f6c2c0c9db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:33.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:33.326+0000 7f6c32f9d700 1 -- 172.21.15.31:0/124229488 shutdown_connections 2023-10-06T02:24:33.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:33.326+0000 7f6c32f9d700 1 -- 172.21.15.31:0/124229488 wait complete. 2023-10-06T02:24:33.345 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T02:24:33.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:33.578+0000 7f63549fe700 1 Processor -- start 2023-10-06T02:24:33.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:33.579+0000 7f63549fe700 1 -- start start 2023-10-06T02:24:33.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:33.580+0000 7f63549fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f63500c8200 0x7f63500c1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:33.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:33.580+0000 7f63549fe700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f63500c2a90 0x7f6350156aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:33.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:33.580+0000 7f63549fe700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63501579b0 con 0x7f63500c85d0 2023-10-06T02:24:33.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:33.580+0000 7f63549fe700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63500c1390 con 0x7f63500c2a90 2023-10-06T02:24:33.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:33.580+0000 7f63549fe700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63501573f0 con 0x7f63500c8200 2023-10-06T02:24:33.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:33.580+0000 7f634f59e700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f63500c85d0 0x7f63500cc2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58098/0 (socket says 172.21.15.31:58098) 2023-10-06T02:24:33.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:33.580+0000 7f634f59e700 1 -- 172.21.15.31:0/287013848 learned_addr learned my addr 172.21.15.31:0/287013848 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:24:33.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:33.580+0000 7f634e59c700 1 --2- 172.21.15.31:0/287013848 >> v2:172.21.15.62:3300/0 conn(0x7f63500c2a90 0x7f6350156aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:33.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:33.580+0000 7f634ed9d700 1 --2- 172.21.15.31:0/287013848 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f63500c8200 0x7f63500c1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:33.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:33.581+0000 7f634dd9b700 1 -- 172.21.15.31:0/287013848 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2111241545 0 0) 0x7f63501579b0 con 0x7f63500c85d0 2023-10-06T02:24:33.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:33.581+0000 7f634dd9b700 1 -- 172.21.15.31:0/287013848 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6338003570 con 0x7f63500c85d0 2023-10-06T02:24:33.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:33.581+0000 7f634e59c700 1 -- 172.21.15.31:0/287013848 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f63500c8200 msgr2=0x7f63500c1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:33.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:33.581+0000 7f634e59c700 1 --2- 172.21.15.31:0/287013848 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f63500c8200 0x7f63500c1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:33.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:33.581+0000 7f634e59c700 1 -- 172.21.15.31:0/287013848 >> v1:172.21.15.31:6789/0 conn(0x7f63500c85d0 legacy=0x7f63500cc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:33.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:33.581+0000 7f634e59c700 1 -- 172.21.15.31:0/287013848 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63501576d0 con 0x7f63500c2a90 2023-10-06T02:24:33.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:33.581+0000 7f634ed9d700 1 --2- 172.21.15.31:0/287013848 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f63500c8200 0x7f63500c1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:24:33.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:33.581+0000 7f634e59c700 1 --2- 172.21.15.31:0/287013848 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63500c2a90 0x7f6350156aa0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f63440098e0 tx=0x7f63440091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=394f1449083df62f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:33.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:33.581+0000 7f634dd9b700 1 -- 172.21.15.31:0/287013848 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f634400f6b0 con 0x7f63500c2a90 2023-10-06T02:24:33.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:33.581+0000 7f634dd9b700 1 -- 172.21.15.31:0/287013848 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f634400f810 con 0x7f63500c2a90 2023-10-06T02:24:33.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:33.581+0000 7f634dd9b700 1 -- 172.21.15.31:0/287013848 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f634400fbb0 con 0x7f63500c2a90 2023-10-06T02:24:33.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:33.582+0000 7f63549fe700 1 -- 172.21.15.31:0/287013848 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63500c2a90 msgr2=0x7f6350156aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:33.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:33.582+0000 7f63549fe700 1 --2- 172.21.15.31:0/287013848 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63500c2a90 0x7f6350156aa0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f63440098e0 tx=0x7f63440091a0 comp rx=0 tx=0).stop 2023-10-06T02:24:33.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:33.582+0000 7f63549fe700 1 -- 172.21.15.31:0/287013848 shutdown_connections 2023-10-06T02:24:33.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:33.582+0000 7f63549fe700 1 --2- 172.21.15.31:0/287013848 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f63500c8200 0x7f63500c1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:33.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:33.582+0000 7f63549fe700 1 --2- 172.21.15.31:0/287013848 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63500c2a90 0x7f6350156aa0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:33.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:33.582+0000 7f63549fe700 1 -- 172.21.15.31:0/287013848 >> 172.21.15.31:0/287013848 conn(0x7f635006c6a0 msgr2=0x7f635006caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:33.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:33.582+0000 7f63549fe700 1 -- 172.21.15.31:0/287013848 shutdown_connections 2023-10-06T02:24:33.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:33.582+0000 7f63549fe700 1 -- 172.21.15.31:0/287013848 wait complete. 2023-10-06T02:24:33.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:33.584+0000 7f63549fe700 1 Processor -- start 2023-10-06T02:24:33.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:33.584+0000 7f63549fe700 1 -- start start 2023-10-06T02:24:33.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:33.585+0000 7f63549fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63500c2a90 0x7f63500af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:33.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:33.585+0000 7f63549fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63500c8200 0x7f635015e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:33.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:33.585+0000 7f63549fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f63500c85d0 0x7f63500cad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:33.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:33.585+0000 7f63549fe700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63500b1ad0 con 0x7f63500c2a90 2023-10-06T02:24:33.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:33.585+0000 7f63549fe700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63500afe10 con 0x7f63500c8200 2023-10-06T02:24:33.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:33.585+0000 7f63549fe700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63500b0ec0 con 0x7f63500c85d0 2023-10-06T02:24:33.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:33.585+0000 7f634ed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63500c2a90 0x7f63500af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:33.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:33.585+0000 7f634ed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63500c2a90 0x7f63500af8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40308/0 (socket says 172.21.15.31:40308) 2023-10-06T02:24:33.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:33.585+0000 7f634ed9d700 1 -- 172.21.15.31:0/3395742137 learned_addr learned my addr 172.21.15.31:0/3395742137 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:33.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:33.585+0000 7f634e59c700 1 --2- 172.21.15.31:0/3395742137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63500c8200 0x7f635015e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:33.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:33.585+0000 7f634f59e700 1 --2- 172.21.15.31:0/3395742137 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f63500c85d0 0x7f63500cad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:33.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:33.585+0000 7f634ed9d700 1 -- 172.21.15.31:0/3395742137 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f63500c85d0 msgr2=0x7f63500cad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:33.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:33.585+0000 7f634ed9d700 1 --2- 172.21.15.31:0/3395742137 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f63500c85d0 0x7f63500cad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:33.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:33.585+0000 7f634ed9d700 1 -- 172.21.15.31:0/3395742137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63500c8200 msgr2=0x7f635015e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:33.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:33.585+0000 7f634ed9d700 1 --2- 172.21.15.31:0/3395742137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63500c8200 0x7f635015e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:33.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:33.585+0000 7f634ed9d700 1 -- 172.21.15.31:0/3395742137 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6340009420 con 0x7f63500c2a90 2023-10-06T02:24:33.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:33.585+0000 7f634f59e700 1 --2- 172.21.15.31:0/3395742137 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f63500c85d0 0x7f63500cad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:24:33.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:33.585+0000 7f634ed9d700 1 --2- 172.21.15.31:0/3395742137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63500c2a90 0x7f63500af8d0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f634000ecb0 tx=0x7f63400091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:33.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:33.586+0000 7f6336ffd700 1 -- 172.21.15.31:0/3395742137 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f634000ee60 con 0x7f63500c2a90 2023-10-06T02:24:33.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:33.586+0000 7f6336ffd700 1 -- 172.21.15.31:0/3395742137 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f634000cbe0 con 0x7f63500c2a90 2023-10-06T02:24:33.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:33.586+0000 7f6336ffd700 1 -- 172.21.15.31:0/3395742137 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6340013470 con 0x7f63500c2a90 2023-10-06T02:24:33.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:33.586+0000 7f63549fe700 1 -- 172.21.15.31:0/3395742137 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6344009420 con 0x7f63500c2a90 2023-10-06T02:24:33.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:33.586+0000 7f63549fe700 1 -- 172.21.15.31:0/3395742137 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63500b1480 con 0x7f63500c2a90 2023-10-06T02:24:33.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:33.587+0000 7f63549fe700 1 -- 172.21.15.31:0/3395742137 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63500b1db0 con 0x7f63500c2a90 2023-10-06T02:24:33.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:33.588+0000 7f6336ffd700 1 -- 172.21.15.31:0/3395742137 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f6340013820 con 0x7f63500c2a90 2023-10-06T02:24:33.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:33.589+0000 7f6336ffd700 1 --2- 172.21.15.31:0/3395742137 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f633c0670e0 0x7f633c069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:33.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:33.589+0000 7f6336ffd700 1 -- 172.21.15.31:0/3395742137 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f63400851d0 con 0x7f63500c2a90 2023-10-06T02:24:33.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:33.589+0000 7f634e59c700 1 --2- 172.21.15.31:0/3395742137 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f633c0670e0 0x7f633c069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:33.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:33.590+0000 7f634e59c700 1 --2- 172.21.15.31:0/3395742137 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f633c0670e0 0x7f633c069590 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f63440093f0 tx=0x7f6344019000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:33.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:33.592+0000 7f6336ffd700 1 -- 172.21.15.31:0/3395742137 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6340052010 con 0x7f63500c2a90 2023-10-06T02:24:33.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:33.792+0000 7f63549fe700 1 -- 172.21.15.31:0/3395742137 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f635004c3c0 con 0x7f63500c2a90 2023-10-06T02:24:34.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:34.326+0000 7f6336ffd700 1 -- 172.21.15.31:0/3395742137 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v38) v1 ==== 181+0+0 (secure 0 0 0) 0x7f6340055630 con 0x7f63500c2a90 2023-10-06T02:24:34.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:34.330+0000 7f63549fe700 1 -- 172.21.15.31:0/3395742137 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f633c0670e0 msgr2=0x7f633c069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:34.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:34.330+0000 7f63549fe700 1 --2- 172.21.15.31:0/3395742137 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f633c0670e0 0x7f633c069590 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f63440093f0 tx=0x7f6344019000 comp rx=0 tx=0).stop 2023-10-06T02:24:34.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:34.330+0000 7f63549fe700 1 -- 172.21.15.31:0/3395742137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63500c2a90 msgr2=0x7f63500af8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:34.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:34.330+0000 7f63549fe700 1 --2- 172.21.15.31:0/3395742137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63500c2a90 0x7f63500af8d0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f634000ecb0 tx=0x7f63400091a0 comp rx=0 tx=0).stop 2023-10-06T02:24:34.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:34.330+0000 7f63549fe700 1 -- 172.21.15.31:0/3395742137 shutdown_connections 2023-10-06T02:24:34.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:34.330+0000 7f63549fe700 1 --2- 172.21.15.31:0/3395742137 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f633c0670e0 0x7f633c069590 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:34.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:34.330+0000 7f63549fe700 1 --2- 172.21.15.31:0/3395742137 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f63500c85d0 0x7f63500cad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:34.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:34.330+0000 7f63549fe700 1 --2- 172.21.15.31:0/3395742137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63500c2a90 0x7f63500af8d0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:34.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:34.330+0000 7f63549fe700 1 --2- 172.21.15.31:0/3395742137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63500c8200 0x7f635015e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:34.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:34.330+0000 7f63549fe700 1 -- 172.21.15.31:0/3395742137 >> 172.21.15.31:0/3395742137 conn(0x7f635006c6a0 msgr2=0x7f63500c9420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:34.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:34.330+0000 7f63549fe700 1 -- 172.21.15.31:0/3395742137 shutdown_connections 2023-10-06T02:24:34.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:34.330+0000 7f63549fe700 1 -- 172.21.15.31:0/3395742137 wait complete. 2023-10-06T02:24:34.332 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' removed 2023-10-06T02:24:34.351 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T02:24:34.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:34.544+0000 7f763d99d700 1 Processor -- start 2023-10-06T02:24:34.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:34.544+0000 7f763d99d700 1 -- start start 2023-10-06T02:24:34.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:34.546+0000 7f763d99d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7638142ca0 0x7f763815dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:34.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:34.546+0000 7f763d99d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7638143070 0x7f76380cbf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:34.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:34.546+0000 7f763d99d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f763815d8d0 con 0x7f763815a6f0 2023-10-06T02:24:34.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:34.546+0000 7f763d99d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f763815d2f0 con 0x7f7638143070 2023-10-06T02:24:34.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:34.546+0000 7f763d99d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f763815ab90 con 0x7f7638142ca0 2023-10-06T02:24:34.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:34.546+0000 7f763d19c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f763815a6f0 0x7f76380cad40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58108/0 (socket says 172.21.15.31:58108) 2023-10-06T02:24:34.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:34.546+0000 7f763d19c700 1 -- 172.21.15.31:0/2120828028 learned_addr learned my addr 172.21.15.31:0/2120828028 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:24:34.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:34.546+0000 7f763c99b700 1 --2- 172.21.15.31:0/2120828028 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7638142ca0 0x7f763815dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:34.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:34.546+0000 7f7637fff700 1 --2- 172.21.15.31:0/2120828028 >> v2:172.21.15.62:3300/0 conn(0x7f7638143070 0x7f76380cbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:34.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:34.547+0000 7f76377fe700 1 -- 172.21.15.31:0/2120828028 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3429145763 0 0) 0x7f763815d8d0 con 0x7f763815a6f0 2023-10-06T02:24:34.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:34.547+0000 7f76377fe700 1 -- 172.21.15.31:0/2120828028 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7620003570 con 0x7f763815a6f0 2023-10-06T02:24:34.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:34.547+0000 7f7637fff700 1 -- 172.21.15.31:0/2120828028 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7638142ca0 msgr2=0x7f763815dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:34.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:34.547+0000 7f7637fff700 1 --2- 172.21.15.31:0/2120828028 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7638142ca0 0x7f763815dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:34.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:34.547+0000 7f7637fff700 1 -- 172.21.15.31:0/2120828028 >> v1:172.21.15.31:6789/0 conn(0x7f763815a6f0 legacy=0x7f76380cad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:34.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:34.547+0000 7f7637fff700 1 -- 172.21.15.31:0/2120828028 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76380cc630 con 0x7f7638143070 2023-10-06T02:24:34.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:34.547+0000 7f763c99b700 1 --2- 172.21.15.31:0/2120828028 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7638142ca0 0x7f763815dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:24:34.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:34.547+0000 7f7637fff700 1 --2- 172.21.15.31:0/2120828028 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7638143070 0x7f76380cbf30 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f762c0098e0 tx=0x7f762c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10d6c9dac7640c2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:34.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:34.547+0000 7f76377fe700 1 -- 172.21.15.31:0/2120828028 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f762c011630 con 0x7f7638143070 2023-10-06T02:24:34.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:34.548+0000 7f76377fe700 1 -- 172.21.15.31:0/2120828028 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f762c011790 con 0x7f7638143070 2023-10-06T02:24:34.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:34.548+0000 7f76377fe700 1 -- 172.21.15.31:0/2120828028 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f762c011b10 con 0x7f7638143070 2023-10-06T02:24:34.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:34.548+0000 7f763d99d700 1 -- 172.21.15.31:0/2120828028 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7638143070 msgr2=0x7f76380cbf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:34.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:34.548+0000 7f763d99d700 1 --2- 172.21.15.31:0/2120828028 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7638143070 0x7f76380cbf30 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f762c0098e0 tx=0x7f762c0091a0 comp rx=0 tx=0).stop 2023-10-06T02:24:34.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:34.548+0000 7f763d99d700 1 -- 172.21.15.31:0/2120828028 shutdown_connections 2023-10-06T02:24:34.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:34.548+0000 7f763d99d700 1 --2- 172.21.15.31:0/2120828028 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7638142ca0 0x7f763815dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:34.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:34.548+0000 7f763d99d700 1 --2- 172.21.15.31:0/2120828028 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7638143070 0x7f76380cbf30 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:34.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:34.548+0000 7f763d99d700 1 -- 172.21.15.31:0/2120828028 >> 172.21.15.31:0/2120828028 conn(0x7f7638067c80 msgr2=0x7f7638068080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:34.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:34.548+0000 7f763d99d700 1 -- 172.21.15.31:0/2120828028 shutdown_connections 2023-10-06T02:24:34.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:34.548+0000 7f763d99d700 1 -- 172.21.15.31:0/2120828028 wait complete. 2023-10-06T02:24:34.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:34.550+0000 7f763d99d700 1 Processor -- start 2023-10-06T02:24:34.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:34.550+0000 7f763d99d700 1 -- start start 2023-10-06T02:24:34.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:34.551+0000 7f763d99d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7638142ca0 0x7f763816a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:34.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:34.551+0000 7f763d99d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7638143070 0x7f7638159240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:34.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:34.551+0000 7f763d99d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f763815a6f0 0x7f7638159780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:34.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:34.551+0000 7f763d99d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76380af7a0 con 0x7f763815a6f0 2023-10-06T02:24:34.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:34.551+0000 7f763d99d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f763816ad70 con 0x7f7638143070 2023-10-06T02:24:34.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:34.551+0000 7f763d99d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f763816a3a0 con 0x7f7638142ca0 2023-10-06T02:24:34.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:34.551+0000 7f763d19c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f763815a6f0 0x7f7638159780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:34.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:34.551+0000 7f763d19c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f763815a6f0 0x7f7638159780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40316/0 (socket says 172.21.15.31:40316) 2023-10-06T02:24:34.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:34.551+0000 7f763d19c700 1 -- 172.21.15.31:0/322940706 learned_addr learned my addr 172.21.15.31:0/322940706 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:34.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:34.551+0000 7f763c99b700 1 --2- 172.21.15.31:0/322940706 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7638142ca0 0x7f763816a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:34.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:34.551+0000 7f7637fff700 1 --2- 172.21.15.31:0/322940706 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7638143070 0x7f7638159240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:34.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:34.551+0000 7f763d19c700 1 -- 172.21.15.31:0/322940706 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7638142ca0 msgr2=0x7f763816a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:34.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:34.551+0000 7f763d19c700 1 --2- 172.21.15.31:0/322940706 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7638142ca0 0x7f763816a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:34.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:34.551+0000 7f763d19c700 1 -- 172.21.15.31:0/322940706 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7638143070 msgr2=0x7f7638159240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:34.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:34.551+0000 7f763d19c700 1 --2- 172.21.15.31:0/322940706 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7638143070 0x7f7638159240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:34.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:34.551+0000 7f763d19c700 1 -- 172.21.15.31:0/322940706 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7628009420 con 0x7f763815a6f0 2023-10-06T02:24:34.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:34.552+0000 7f763d19c700 1 --2- 172.21.15.31:0/322940706 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f763815a6f0 0x7f7638159780 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f7630001460 tx=0x7f7630001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:34.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:34.552+0000 7f7634ff9700 1 -- 172.21.15.31:0/322940706 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f763000a870 con 0x7f763815a6f0 2023-10-06T02:24:34.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:34.552+0000 7f7634ff9700 1 -- 172.21.15.31:0/322940706 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f763000a9d0 con 0x7f763815a6f0 2023-10-06T02:24:34.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:34.552+0000 7f763d99d700 1 -- 172.21.15.31:0/322940706 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f762c009420 con 0x7f763815a6f0 2023-10-06T02:24:34.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:34.552+0000 7f7634ff9700 1 -- 172.21.15.31:0/322940706 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f763000ad50 con 0x7f763815a6f0 2023-10-06T02:24:34.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:34.552+0000 7f763d99d700 1 -- 172.21.15.31:0/322940706 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7638169d50 con 0x7f763815a6f0 2023-10-06T02:24:34.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:34.553+0000 7f763d99d700 1 -- 172.21.15.31:0/322940706 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f763804c3c0 con 0x7f763815a6f0 2023-10-06T02:24:34.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:34.554+0000 7f7634ff9700 1 -- 172.21.15.31:0/322940706 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f7630007300 con 0x7f763815a6f0 2023-10-06T02:24:34.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:34.555+0000 7f7634ff9700 1 --2- 172.21.15.31:0/322940706 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f76240670e0 0x7f7624069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:34.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:34.555+0000 7f7634ff9700 1 -- 172.21.15.31:0/322940706 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f7630084b80 con 0x7f763815a6f0 2023-10-06T02:24:34.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:34.555+0000 7f763c99b700 1 --2- 172.21.15.31:0/322940706 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f76240670e0 0x7f7624069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:34.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:34.555+0000 7f763c99b700 1 --2- 172.21.15.31:0/322940706 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f76240670e0 0x7f7624069590 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7628009f40 tx=0x7f76280091a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:34.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:34.559+0000 7f7634ff9700 1 -- 172.21.15.31:0/322940706 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7630051b50 con 0x7f763815a6f0 2023-10-06T02:24:34.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:34.761+0000 7f763d99d700 1 -- 172.21.15.31:0/322940706 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f763816a4e0 con 0x7f763815a6f0 2023-10-06T02:24:35.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.332+0000 7f7634ff9700 1 -- 172.21.15.31:0/322940706 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v39) v1 ==== 169+0+0 (secure 0 0 0) 0x7f7630055170 con 0x7f763815a6f0 2023-10-06T02:24:35.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.335+0000 7f763d99d700 1 -- 172.21.15.31:0/322940706 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f76240670e0 msgr2=0x7f7624069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:35.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.335+0000 7f763d99d700 1 --2- 172.21.15.31:0/322940706 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f76240670e0 0x7f7624069590 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7628009f40 tx=0x7f76280091a0 comp rx=0 tx=0).stop 2023-10-06T02:24:35.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.335+0000 7f763d99d700 1 -- 172.21.15.31:0/322940706 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f763815a6f0 msgr2=0x7f7638159780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:35.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.335+0000 7f763d99d700 1 --2- 172.21.15.31:0/322940706 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f763815a6f0 0x7f7638159780 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f7630001460 tx=0x7f7630001a40 comp rx=0 tx=0).stop 2023-10-06T02:24:35.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.337+0000 7f763d99d700 1 -- 172.21.15.31:0/322940706 shutdown_connections 2023-10-06T02:24:35.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.337+0000 7f763d99d700 1 --2- 172.21.15.31:0/322940706 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f763815a6f0 0x7f7638159780 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:35.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.337+0000 7f763d99d700 1 --2- 172.21.15.31:0/322940706 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f76240670e0 0x7f7624069590 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:35.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.337+0000 7f763d99d700 1 --2- 172.21.15.31:0/322940706 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7638142ca0 0x7f763816a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:35.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.337+0000 7f763d99d700 1 --2- 172.21.15.31:0/322940706 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7638143070 0x7f7638159240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:35.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.337+0000 7f763d99d700 1 -- 172.21.15.31:0/322940706 >> 172.21.15.31:0/322940706 conn(0x7f7638067c80 msgr2=0x7f7638141030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:35.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.338+0000 7f763d99d700 1 -- 172.21.15.31:0/322940706 shutdown_connections 2023-10-06T02:24:35.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.338+0000 7f763d99d700 1 -- 172.21.15.31:0/322940706 wait complete. 2023-10-06T02:24:35.340 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' removed 2023-10-06T02:24:35.361 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T02:24:35.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.551+0000 7f17a2d9d700 1 Processor -- start 2023-10-06T02:24:35.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.552+0000 7f17a2d9d700 1 -- start start 2023-10-06T02:24:35.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.553+0000 7f17a2d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f179c153dd0 0x7f179c15de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:35.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.553+0000 7f17a2d9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f179c0cea00 0x7f179c0cee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:35.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.553+0000 7f17a2d9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f179c15ac50 con 0x7f179c153a00 2023-10-06T02:24:35.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.553+0000 7f17a2d9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f179c15d010 con 0x7f179c0cea00 2023-10-06T02:24:35.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.553+0000 7f17a2d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f179c15af60 con 0x7f179c153dd0 2023-10-06T02:24:35.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.553+0000 7f17a1d9b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f179c153a00 0x7f179c15a540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58122/0 (socket says 172.21.15.31:58122) 2023-10-06T02:24:35.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.553+0000 7f17a1d9b700 1 -- 172.21.15.31:0/271456479 learned_addr learned my addr 172.21.15.31:0/271456479 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:24:35.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.554+0000 7f17a159a700 1 --2- 172.21.15.31:0/271456479 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f179c153dd0 0x7f179c15de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:35.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.554+0000 7f17a259c700 1 --2- 172.21.15.31:0/271456479 >> v2:172.21.15.62:3300/0 conn(0x7f179c0cea00 0x7f179c0cee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:35.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.554+0000 7f17a0d99700 1 -- 172.21.15.31:0/271456479 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1271029098 0 0) 0x7f179c15ac50 con 0x7f179c153a00 2023-10-06T02:24:35.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.554+0000 7f17a0d99700 1 -- 172.21.15.31:0/271456479 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1788003570 con 0x7f179c153a00 2023-10-06T02:24:35.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.554+0000 7f17a0d99700 1 -- 172.21.15.31:0/271456479 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1673477259 0 0) 0x7f1788003570 con 0x7f179c153a00 2023-10-06T02:24:35.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.554+0000 7f17a0d99700 1 -- 172.21.15.31:0/271456479 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f179c153dd0 msgr2=0x7f179c15de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:35.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.554+0000 7f17a0d99700 1 --2- 172.21.15.31:0/271456479 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f179c153dd0 0x7f179c15de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:35.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.554+0000 7f17a0d99700 1 -- 172.21.15.31:0/271456479 >> v2:172.21.15.62:3300/0 conn(0x7f179c0cea00 msgr2=0x7f179c0cee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:35.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.554+0000 7f17a0d99700 1 --2- 172.21.15.31:0/271456479 >> v2:172.21.15.62:3300/0 conn(0x7f179c0cea00 0x7f179c0cee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:35.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.554+0000 7f17a0d99700 1 -- 172.21.15.31:0/271456479 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f179c15b240 con 0x7f179c153a00 2023-10-06T02:24:35.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.554+0000 7f17a0d99700 1 -- 172.21.15.31:0/271456479 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7f1798001470 con 0x7f179c153a00 2023-10-06T02:24:35.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.554+0000 7f17a0d99700 1 -- 172.21.15.31:0/271456479 >> v1:172.21.15.31:6789/0 conn(0x7f179c153a00 legacy=0x7f179c15a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:35.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.554+0000 7f17a0d99700 1 --2- 172.21.15.31:0/271456479 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1788003a60 0x7f1788003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:35.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.554+0000 7f17a0d99700 1 --2- 172.21.15.31:0/271456479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1788004810 0x7f1788004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:35.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.554+0000 7f17a0d99700 1 --2- 172.21.15.31:0/271456479 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f179c153dd0 0x7f17880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:35.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.554+0000 7f17a0d99700 1 -- 172.21.15.31:0/271456479 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f179c15d010 con 0x7f1788003a60 2023-10-06T02:24:35.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.554+0000 7f17a0d99700 1 -- 172.21.15.31:0/271456479 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f179c15af60 con 0x7f1788004810 2023-10-06T02:24:35.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.554+0000 7f17a0d99700 1 -- 172.21.15.31:0/271456479 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f178800ba60 con 0x7f179c153dd0 2023-10-06T02:24:35.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.555+0000 7f17a1d9b700 1 --2- 172.21.15.31:0/271456479 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1788003a60 0x7f1788003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:35.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.555+0000 7f17a259c700 1 --2- 172.21.15.31:0/271456479 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f179c153dd0 0x7f17880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:35.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.555+0000 7f17a159a700 1 --2- 172.21.15.31:0/271456479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1788004810 0x7f1788004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:35.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.555+0000 7f17a1d9b700 1 -- 172.21.15.31:0/271456479 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f179c153dd0 msgr2=0x7f17880051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:35.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.555+0000 7f17a1d9b700 1 --2- 172.21.15.31:0/271456479 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f179c153dd0 0x7f17880051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:35.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.555+0000 7f17a1d9b700 1 -- 172.21.15.31:0/271456479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1788004810 msgr2=0x7f1788004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:35.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.555+0000 7f17a1d9b700 1 --2- 172.21.15.31:0/271456479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1788004810 0x7f1788004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:35.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.555+0000 7f17a1d9b700 1 -- 172.21.15.31:0/271456479 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1788003570 con 0x7f1788003a60 2023-10-06T02:24:35.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.555+0000 7f17a1d9b700 1 --2- 172.21.15.31:0/271456479 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1788003a60 0x7f1788003e70 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f179800c3f0 tx=0x7f1798001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f32f8a3f1f7ffecb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:35.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.555+0000 7f17a0d99700 1 -- 172.21.15.31:0/271456479 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f179800cc80 con 0x7f1788003a60 2023-10-06T02:24:35.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.556+0000 7f17a0d99700 1 -- 172.21.15.31:0/271456479 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f179800cde0 con 0x7f1788003a60 2023-10-06T02:24:35.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.556+0000 7f17a0d99700 1 -- 172.21.15.31:0/271456479 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1798005b50 con 0x7f1788003a60 2023-10-06T02:24:35.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.556+0000 7f17a2d9d700 1 -- 172.21.15.31:0/271456479 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1788003a60 msgr2=0x7f1788003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:35.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.556+0000 7f17a2d9d700 1 --2- 172.21.15.31:0/271456479 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1788003a60 0x7f1788003e70 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f179800c3f0 tx=0x7f1798001e50 comp rx=0 tx=0).stop 2023-10-06T02:24:35.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.556+0000 7f17a259c700 1 -- 172.21.15.31:0/271456479 reap_dead start 2023-10-06T02:24:35.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.556+0000 7f17a2d9d700 1 -- 172.21.15.31:0/271456479 shutdown_connections 2023-10-06T02:24:35.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.556+0000 7f17a2d9d700 1 -- 172.21.15.31:0/271456479 >> 172.21.15.31:0/271456479 conn(0x7f179c0703a0 msgr2=0x7f179c0707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:35.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.556+0000 7f17a2d9d700 1 -- 172.21.15.31:0/271456479 shutdown_connections 2023-10-06T02:24:35.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.556+0000 7f17a2d9d700 1 -- 172.21.15.31:0/271456479 wait complete. 2023-10-06T02:24:35.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.558+0000 7f17a2d9d700 1 Processor -- start 2023-10-06T02:24:35.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.558+0000 7f17a2d9d700 1 -- start start 2023-10-06T02:24:35.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.559+0000 7f17a2d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1788003a60 0x7f179c151720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:35.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.559+0000 7f17a2d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f179c0cbca0 0x7f179c150380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:35.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.559+0000 7f17a2d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f179c1508c0 0x7f179c0c7a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:35.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.559+0000 7f17a2d9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f179c0c75e0 con 0x7f179c0cbca0 2023-10-06T02:24:35.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.559+0000 7f17a2d9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f179c152880 con 0x7f179c1508c0 2023-10-06T02:24:35.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.559+0000 7f17a2d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f179c151d00 con 0x7f1788003a60 2023-10-06T02:24:35.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.559+0000 7f17a159a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f179c0cbca0 0x7f179c150380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:35.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.559+0000 7f17a159a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f179c0cbca0 0x7f179c150380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40332/0 (socket says 172.21.15.31:40332) 2023-10-06T02:24:35.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.559+0000 7f17a159a700 1 -- 172.21.15.31:0/3218428729 learned_addr learned my addr 172.21.15.31:0/3218428729 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:35.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.559+0000 7f17a1d9b700 1 --2- 172.21.15.31:0/3218428729 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1788003a60 0x7f179c151720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:35.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.559+0000 7f17a259c700 1 --2- 172.21.15.31:0/3218428729 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f179c1508c0 0x7f179c0c7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:35.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.559+0000 7f17a159a700 1 -- 172.21.15.31:0/3218428729 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1788003a60 msgr2=0x7f179c151720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:35.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.559+0000 7f17a159a700 1 --2- 172.21.15.31:0/3218428729 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1788003a60 0x7f179c151720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:35.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.559+0000 7f17a159a700 1 -- 172.21.15.31:0/3218428729 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f179c1508c0 msgr2=0x7f179c0c7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:35.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.559+0000 7f17a159a700 1 --2- 172.21.15.31:0/3218428729 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f179c1508c0 0x7f179c0c7a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:35.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.559+0000 7f17a159a700 1 -- 172.21.15.31:0/3218428729 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f179c151fe0 con 0x7f179c0cbca0 2023-10-06T02:24:35.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.560+0000 7f17a159a700 1 --2- 172.21.15.31:0/3218428729 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f179c0cbca0 0x7f179c150380 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f178c011b70 tx=0x7f178c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:35.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.560+0000 7f1791ffb700 1 -- 172.21.15.31:0/3218428729 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f178c00f940 con 0x7f179c0cbca0 2023-10-06T02:24:35.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.560+0000 7f1791ffb700 1 -- 172.21.15.31:0/3218428729 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f178c00faa0 con 0x7f179c0cbca0 2023-10-06T02:24:35.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.560+0000 7f1791ffb700 1 -- 172.21.15.31:0/3218428729 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f178c00fe20 con 0x7f179c0cbca0 2023-10-06T02:24:35.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.560+0000 7f17a2d9d700 1 -- 172.21.15.31:0/3218428729 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f179c1522c0 con 0x7f179c0cbca0 2023-10-06T02:24:35.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.560+0000 7f17a2d9d700 1 -- 172.21.15.31:0/3218428729 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f179c0cc110 con 0x7f179c0cbca0 2023-10-06T02:24:35.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.561+0000 7f17a2d9d700 1 -- 172.21.15.31:0/3218428729 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f179c0c7880 con 0x7f179c0cbca0 2023-10-06T02:24:35.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.562+0000 7f1791ffb700 1 -- 172.21.15.31:0/3218428729 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f178c014440 con 0x7f179c0cbca0 2023-10-06T02:24:35.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.563+0000 7f1791ffb700 1 --2- 172.21.15.31:0/3218428729 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f17800670e0 0x7f1780069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:35.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.563+0000 7f1791ffb700 1 -- 172.21.15.31:0/3218428729 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f178c018070 con 0x7f179c0cbca0 2023-10-06T02:24:35.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.563+0000 7f17a1d9b700 1 --2- 172.21.15.31:0/3218428729 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f17800670e0 0x7f1780069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:35.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.564+0000 7f17a1d9b700 1 --2- 172.21.15.31:0/3218428729 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f17800670e0 0x7f1780069590 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1798007cd0 tx=0x7f1798015000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:35.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.567+0000 7f1791ffb700 1 -- 172.21.15.31:0/3218428729 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f178c0525c0 con 0x7f179c0cbca0 2023-10-06T02:24:35.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.771+0000 7f17a2d9d700 1 -- 172.21.15.31:0/3218428729 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f179c04c3c0 con 0x7f179c0cbca0 2023-10-06T02:24:35.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.775+0000 7f1791ffb700 1 -- 172.21.15.31:0/3218428729 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f178c055be0 con 0x7f179c0cbca0 2023-10-06T02:24:35.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.778+0000 7f17a2d9d700 1 -- 172.21.15.31:0/3218428729 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f17800670e0 msgr2=0x7f1780069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:35.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.778+0000 7f17a2d9d700 1 --2- 172.21.15.31:0/3218428729 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f17800670e0 0x7f1780069590 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1798007cd0 tx=0x7f1798015000 comp rx=0 tx=0).stop 2023-10-06T02:24:35.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.778+0000 7f17a2d9d700 1 -- 172.21.15.31:0/3218428729 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f179c0cbca0 msgr2=0x7f179c150380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:35.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.778+0000 7f17a2d9d700 1 --2- 172.21.15.31:0/3218428729 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f179c0cbca0 0x7f179c150380 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f178c011b70 tx=0x7f178c0091a0 comp rx=0 tx=0).stop 2023-10-06T02:24:35.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.778+0000 7f17a2d9d700 1 -- 172.21.15.31:0/3218428729 shutdown_connections 2023-10-06T02:24:35.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.778+0000 7f17a2d9d700 1 --2- 172.21.15.31:0/3218428729 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f179c1508c0 0x7f179c0c7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:35.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.778+0000 7f17a2d9d700 1 --2- 172.21.15.31:0/3218428729 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f17800670e0 0x7f1780069590 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:35.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.778+0000 7f17a2d9d700 1 --2- 172.21.15.31:0/3218428729 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1788003a60 0x7f179c151720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:35.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.778+0000 7f17a2d9d700 1 --2- 172.21.15.31:0/3218428729 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f179c0cbca0 0x7f179c150380 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:35.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.778+0000 7f17a2d9d700 1 -- 172.21.15.31:0/3218428729 >> 172.21.15.31:0/3218428729 conn(0x7f179c0703a0 msgr2=0x7f179c0b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:35.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.778+0000 7f17a2d9d700 1 -- 172.21.15.31:0/3218428729 shutdown_connections 2023-10-06T02:24:35.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.778+0000 7f17a2d9d700 1 -- 172.21.15.31:0/3218428729 wait complete. 2023-10-06T02:24:35.795 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade' 2023-10-06T02:24:35.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.980+0000 7fa0ab19d700 1 Processor -- start 2023-10-06T02:24:35.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.981+0000 7fa0ab19d700 1 -- start start 2023-10-06T02:24:35.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.982+0000 7fa0ab19d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa0a413fb20 0x7fa0a415a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:35.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.982+0000 7fa0ab19d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa0a413fef0 0x7fa0a40c7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:35.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.982+0000 7fa0ab19d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa0a406aaa0 0x7fa0a40c7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:35.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.982+0000 7fa0ab19d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0a40c8410 con 0x7fa0a413fef0 2023-10-06T02:24:35.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.982+0000 7fa0ab19d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0a415ae70 con 0x7fa0a413fb20 2023-10-06T02:24:35.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.982+0000 7fa0ab19d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a406af60 con 0x7fa0a406aaa0 2023-10-06T02:24:35.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.982+0000 7fa0a999a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa0a413fef0 0x7fa0a40c7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:35.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.982+0000 7fa0a999a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa0a413fef0 0x7fa0a40c7990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40346/0 (socket says 172.21.15.31:40346) 2023-10-06T02:24:35.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.982+0000 7fa0a999a700 1 -- 172.21.15.31:0/740636350 learned_addr learned my addr 172.21.15.31:0/740636350 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:35.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.982+0000 7fa0aa99c700 1 --2- 172.21.15.31:0/740636350 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa0a406aaa0 0x7fa0a40c7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:35.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.982+0000 7fa0aa19b700 1 --2- 172.21.15.31:0/740636350 >> v2:172.21.15.62:3300/0 conn(0x7fa0a413fb20 0x7fa0a415a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:35.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.983+0000 7fa0a999a700 1 -- 172.21.15.31:0/740636350 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa0a406aaa0 msgr2=0x7fa0a40c7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:35.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.983+0000 7fa0a999a700 1 --2- 172.21.15.31:0/740636350 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa0a406aaa0 0x7fa0a40c7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:35.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.983+0000 7fa0a999a700 1 -- 172.21.15.31:0/740636350 >> v2:172.21.15.62:3300/0 conn(0x7fa0a413fb20 msgr2=0x7fa0a415a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:35.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.983+0000 7fa0a999a700 1 --2- 172.21.15.31:0/740636350 >> v2:172.21.15.62:3300/0 conn(0x7fa0a413fb20 0x7fa0a415a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:35.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.983+0000 7fa0a999a700 1 -- 172.21.15.31:0/740636350 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0a406b240 con 0x7fa0a413fef0 2023-10-06T02:24:35.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.983+0000 7fa0a999a700 1 --2- 172.21.15.31:0/740636350 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0a413fef0 0x7fa0a40c7990 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fa0a40695b0 tx=0x7fa094009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad5f7f12fba43385 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:35.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.983+0000 7fa0a9199700 1 -- 172.21.15.31:0/740636350 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa094005c10 con 0x7fa0a413fef0 2023-10-06T02:24:35.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.983+0000 7fa0a9199700 1 -- 172.21.15.31:0/740636350 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa094005d70 con 0x7fa0a413fef0 2023-10-06T02:24:35.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.983+0000 7fa0a9199700 1 -- 172.21.15.31:0/740636350 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa09400b880 con 0x7fa0a413fef0 2023-10-06T02:24:35.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.984+0000 7fa0ab19d700 1 -- 172.21.15.31:0/740636350 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0a413fef0 msgr2=0x7fa0a40c7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:35.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.984+0000 7fa0ab19d700 1 --2- 172.21.15.31:0/740636350 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0a413fef0 0x7fa0a40c7990 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fa0a40695b0 tx=0x7fa094009220 comp rx=0 tx=0).stop 2023-10-06T02:24:35.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.984+0000 7fa0ab19d700 1 -- 172.21.15.31:0/740636350 shutdown_connections 2023-10-06T02:24:35.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.984+0000 7fa0ab19d700 1 --2- 172.21.15.31:0/740636350 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa0a406aaa0 0x7fa0a40c7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:35.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.984+0000 7fa0ab19d700 1 --2- 172.21.15.31:0/740636350 >> v2:172.21.15.62:3300/0 conn(0x7fa0a413fb20 0x7fa0a415a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:35.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.984+0000 7fa0ab19d700 1 --2- 172.21.15.31:0/740636350 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0a413fef0 0x7fa0a40c7990 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:35.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.984+0000 7fa0ab19d700 1 -- 172.21.15.31:0/740636350 >> 172.21.15.31:0/740636350 conn(0x7fa0a406d170 msgr2=0x7fa0a40cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:35.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.984+0000 7fa0ab19d700 1 -- 172.21.15.31:0/740636350 shutdown_connections 2023-10-06T02:24:35.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.984+0000 7fa0ab19d700 1 -- 172.21.15.31:0/740636350 wait complete. 2023-10-06T02:24:35.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.985+0000 7fa0ab19d700 1 Processor -- start 2023-10-06T02:24:35.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.985+0000 7fa0ab19d700 1 -- start start 2023-10-06T02:24:35.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.986+0000 7fa0ab19d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0a406aaa0 0x7fa0a40ad590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:35.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.986+0000 7fa0ab19d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0a413fb20 0x7fa0a414f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:35.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.986+0000 7fa0ab19d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa0a413fef0 0x7fa0a40b0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:35.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.986+0000 7fa0ab19d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a4150d10 con 0x7fa0a406aaa0 2023-10-06T02:24:35.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.986+0000 7fa0ab19d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a4150100 con 0x7fa0a413fb20 2023-10-06T02:24:35.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.986+0000 7fa0ab19d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a41503e0 con 0x7fa0a413fef0 2023-10-06T02:24:35.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.987+0000 7fa0aa19b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0a406aaa0 0x7fa0a40ad590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:35.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.987+0000 7fa0aa19b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0a406aaa0 0x7fa0a40ad590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40350/0 (socket says 172.21.15.31:40350) 2023-10-06T02:24:35.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.987+0000 7fa0aa19b700 1 -- 172.21.15.31:0/103582180 learned_addr learned my addr 172.21.15.31:0/103582180 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:35.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.987+0000 7fa0a999a700 1 --2- 172.21.15.31:0/103582180 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0a413fb20 0x7fa0a414f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:35.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.987+0000 7fa0aa99c700 1 --2- 172.21.15.31:0/103582180 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa0a413fef0 0x7fa0a40b0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:35.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.987+0000 7fa0aa19b700 1 -- 172.21.15.31:0/103582180 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa0a413fef0 msgr2=0x7fa0a40b0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:35.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.987+0000 7fa0aa19b700 1 --2- 172.21.15.31:0/103582180 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa0a413fef0 0x7fa0a40b0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:35.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.987+0000 7fa0aa19b700 1 -- 172.21.15.31:0/103582180 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0a413fb20 msgr2=0x7fa0a414f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:35.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.987+0000 7fa0aa19b700 1 --2- 172.21.15.31:0/103582180 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0a413fb20 0x7fa0a414f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:35.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.987+0000 7fa0aa19b700 1 -- 172.21.15.31:0/103582180 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa094009480 con 0x7fa0a406aaa0 2023-10-06T02:24:35.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.987+0000 7fa0a999a700 1 --2- 172.21.15.31:0/103582180 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0a413fb20 0x7fa0a414f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:24:35.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.987+0000 7fa0aa19b700 1 --2- 172.21.15.31:0/103582180 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0a406aaa0 0x7fa0a40ad590 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fa0a000d890 tx=0x7fa0a0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:35.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.988+0000 7fa09a7fc700 1 -- 172.21.15.31:0/103582180 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a000b600 con 0x7fa0a406aaa0 2023-10-06T02:24:35.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.988+0000 7fa09a7fc700 1 -- 172.21.15.31:0/103582180 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0a000b760 con 0x7fa0a406aaa0 2023-10-06T02:24:35.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.988+0000 7fa09a7fc700 1 -- 172.21.15.31:0/103582180 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a000bae0 con 0x7fa0a406aaa0 2023-10-06T02:24:35.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.988+0000 7fa0ab19d700 1 -- 172.21.15.31:0/103582180 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0a41506c0 con 0x7fa0a406aaa0 2023-10-06T02:24:35.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.988+0000 7fa0ab19d700 1 -- 172.21.15.31:0/103582180 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0a40adad0 con 0x7fa0a406aaa0 2023-10-06T02:24:35.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.989+0000 7fa0ab19d700 1 -- 172.21.15.31:0/103582180 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0a4150b60 con 0x7fa0a406aaa0 2023-10-06T02:24:35.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.990+0000 7fa09a7fc700 1 -- 172.21.15.31:0/103582180 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7fa0a001e460 con 0x7fa0a406aaa0 2023-10-06T02:24:35.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.991+0000 7fa09a7fc700 1 --2- 172.21.15.31:0/103582180 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa0880670e0 0x7fa088069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:35.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.991+0000 7fa09a7fc700 1 -- 172.21.15.31:0/103582180 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5292+0+0 (secure 0 0 0) 0x7fa0a0084df0 con 0x7fa0a406aaa0 2023-10-06T02:24:35.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.991+0000 7fa0a999a700 1 --2- 172.21.15.31:0/103582180 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa0880670e0 0x7fa088069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:35.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.991+0000 7fa0a999a700 1 --2- 172.21.15.31:0/103582180 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa0880670e0 0x7fa088069590 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa0a40695b0 tx=0x7fa094016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:35.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:35.995+0000 7fa09a7fc700 1 -- 172.21.15.31:0/103582180 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa0a0051ee0 con 0x7fa0a406aaa0 2023-10-06T02:24:36.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:36.276+0000 7fa0ab19d700 1 -- 172.21.15.31:0/103582180 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade"]} v 0) v1 -- 0x7fa0a4158380 con 0x7fa0a406aaa0 2023-10-06T02:24:36.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:36.334+0000 7fa09a7fc700 1 -- 172.21.15.31:0/103582180 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade"]}]=0 v110) v1 ==== 158+0+0 (secure 0 0 0) 0x7fa0a0055500 con 0x7fa0a406aaa0 2023-10-06T02:24:36.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:36.337+0000 7fa0ab19d700 1 -- 172.21.15.31:0/103582180 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa0880670e0 msgr2=0x7fa088069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:36.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:36.337+0000 7fa0ab19d700 1 --2- 172.21.15.31:0/103582180 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa0880670e0 0x7fa088069590 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa0a40695b0 tx=0x7fa094016000 comp rx=0 tx=0).stop 2023-10-06T02:24:36.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:36.337+0000 7fa0ab19d700 1 -- 172.21.15.31:0/103582180 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0a406aaa0 msgr2=0x7fa0a40ad590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:36.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:36.337+0000 7fa0ab19d700 1 --2- 172.21.15.31:0/103582180 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0a406aaa0 0x7fa0a40ad590 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fa0a000d890 tx=0x7fa0a0008040 comp rx=0 tx=0).stop 2023-10-06T02:24:36.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:36.337+0000 7fa0ab19d700 1 -- 172.21.15.31:0/103582180 shutdown_connections 2023-10-06T02:24:36.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:36.337+0000 7fa0ab19d700 1 --2- 172.21.15.31:0/103582180 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa0880670e0 0x7fa088069590 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:36.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:36.337+0000 7fa0ab19d700 1 --2- 172.21.15.31:0/103582180 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa0a413fef0 0x7fa0a40b0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:36.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:36.337+0000 7fa0ab19d700 1 --2- 172.21.15.31:0/103582180 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0a406aaa0 0x7fa0a40ad590 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:36.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:36.337+0000 7fa0ab19d700 1 --2- 172.21.15.31:0/103582180 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0a413fb20 0x7fa0a414f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:36.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:36.337+0000 7fa0ab19d700 1 -- 172.21.15.31:0/103582180 >> 172.21.15.31:0/103582180 conn(0x7fa0a406d170 msgr2=0x7fa0a40ca7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:36.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:36.337+0000 7fa0ab19d700 1 -- 172.21.15.31:0/103582180 shutdown_connections 2023-10-06T02:24:36.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:36.337+0000 7fa0ab19d700 1 -- 172.21.15.31:0/103582180 wait complete. 2023-10-06T02:24:36.355 INFO:tasks.cephfs_test_runner:Validate binary .meta file on legacy subvol root doesn't break the system ... ok 2023-10-06T02:24:36.355 INFO:tasks.cephfs_test_runner:Starting test: test_connection_expiration (tasks.cephfs.test_volumes.TestMisc) 2023-10-06T02:24:36.356 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration' 2023-10-06T02:24:36.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:36.545+0000 7f53c519d700 1 Processor -- start 2023-10-06T02:24:36.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:36.545+0000 7f53c519d700 1 -- start start 2023-10-06T02:24:36.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:36.547+0000 7f53c519d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f53c0140a70 0x7f53c00c6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:36.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:36.547+0000 7f53c519d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f53c0140e40 0x7f53c00c7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:36.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:36.547+0000 7f53c519d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53c00c6d50 con 0x7f53c00c8330 2023-10-06T02:24:36.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:36.547+0000 7f53c519d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53c00c7030 con 0x7f53c0140a70 2023-10-06T02:24:36.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:36.547+0000 7f53c519d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53c00c7310 con 0x7f53c0140e40 2023-10-06T02:24:36.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:36.547+0000 7f53bffff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f53c0140a70 0x7f53c00c6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:36.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:36.547+0000 7f53c499c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f53c00c8330 0x7f53c00aeef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58136/0 (socket says 172.21.15.31:58136) 2023-10-06T02:24:36.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:36.547+0000 7f53bffff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f53c0140a70 0x7f53c00c6770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40366/0 (socket says 172.21.15.31:40366) 2023-10-06T02:24:36.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:36.547+0000 7f53c499c700 1 -- 172.21.15.31:0/4146702526 learned_addr learned my addr 172.21.15.31:0/4146702526 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:24:36.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:36.547+0000 7f53bf7fe700 1 --2- 172.21.15.31:0/4146702526 >> v2:172.21.15.62:3300/0 conn(0x7f53c0140e40 0x7f53c00c7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:36.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:36.547+0000 7f53bffff700 1 -- 172.21.15.31:0/4146702526 >> v2:172.21.15.62:3300/0 conn(0x7f53c0140e40 msgr2=0x7f53c00c7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:36.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:36.547+0000 7f53bffff700 1 --2- 172.21.15.31:0/4146702526 >> v2:172.21.15.62:3300/0 conn(0x7f53c0140e40 0x7f53c00c7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:36.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:36.547+0000 7f53bffff700 1 -- 172.21.15.31:0/4146702526 >> v1:172.21.15.31:6789/0 conn(0x7f53c00c8330 legacy=0x7f53c00aeef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:36.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:36.547+0000 7f53bffff700 1 -- 172.21.15.31:0/4146702526 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53c00c75f0 con 0x7f53c0140a70 2023-10-06T02:24:36.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:36.547+0000 7f53bf7fe700 1 --2- 172.21.15.31:0/4146702526 >> v2:172.21.15.62:3300/0 conn(0x7f53c0140e40 0x7f53c00c7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:24:36.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:36.548+0000 7f53bffff700 1 --2- 172.21.15.31:0/4146702526 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53c0140a70 0x7f53c00c6770 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f53c00cc0e0 tx=0x7f53b00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d765b3dd4e8e2dc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:36.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:36.548+0000 7f53beffd700 1 -- 172.21.15.31:0/4146702526 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b000b6b0 con 0x7f53c0140a70 2023-10-06T02:24:36.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:36.548+0000 7f53beffd700 1 -- 172.21.15.31:0/4146702526 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53b000b810 con 0x7f53c0140a70 2023-10-06T02:24:36.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:36.548+0000 7f53beffd700 1 -- 172.21.15.31:0/4146702526 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b000bbb0 con 0x7f53c0140a70 2023-10-06T02:24:36.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:36.548+0000 7f53c519d700 1 -- 172.21.15.31:0/4146702526 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53c0140a70 msgr2=0x7f53c00c6770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:36.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:36.548+0000 7f53c519d700 1 --2- 172.21.15.31:0/4146702526 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53c0140a70 0x7f53c00c6770 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f53c00cc0e0 tx=0x7f53b00091a0 comp rx=0 tx=0).stop 2023-10-06T02:24:36.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:36.549+0000 7f53c519d700 1 -- 172.21.15.31:0/4146702526 shutdown_connections 2023-10-06T02:24:36.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:36.549+0000 7f53c519d700 1 --2- 172.21.15.31:0/4146702526 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53c0140a70 0x7f53c00c6770 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:36.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:36.549+0000 7f53c519d700 1 --2- 172.21.15.31:0/4146702526 >> v2:172.21.15.62:3300/0 conn(0x7f53c0140e40 0x7f53c00c7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:36.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:36.549+0000 7f53c519d700 1 -- 172.21.15.31:0/4146702526 >> 172.21.15.31:0/4146702526 conn(0x7f53c006d170 msgr2=0x7f53c00cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:36.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:36.549+0000 7f53c519d700 1 -- 172.21.15.31:0/4146702526 shutdown_connections 2023-10-06T02:24:36.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:36.549+0000 7f53c519d700 1 -- 172.21.15.31:0/4146702526 wait complete. 2023-10-06T02:24:36.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:36.550+0000 7f53c519d700 1 Processor -- start 2023-10-06T02:24:36.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:36.550+0000 7f53c519d700 1 -- start start 2023-10-06T02:24:36.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:36.551+0000 7f53c519d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f53c00c8330 0x7f53c0156e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:36.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:36.551+0000 7f53c519d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53c0140a70 0x7f53c00c37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:36.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:36.551+0000 7f53c519d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53c0140e40 0x7f53c00b0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:36.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:36.551+0000 7f53c519d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c00b08f0 con 0x7f53c0140e40 2023-10-06T02:24:36.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:36.551+0000 7f53c519d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c0157360 con 0x7f53c0140a70 2023-10-06T02:24:36.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:36.551+0000 7f53c519d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53c00afce0 con 0x7f53c00c8330 2023-10-06T02:24:36.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:36.551+0000 7f53c499c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53c0140e40 0x7f53c00b0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:36.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:36.551+0000 7f53c499c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53c0140e40 0x7f53c00b0d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40380/0 (socket says 172.21.15.31:40380) 2023-10-06T02:24:36.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:36.551+0000 7f53c499c700 1 -- 172.21.15.31:0/2181875312 learned_addr learned my addr 172.21.15.31:0/2181875312 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:36.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:36.552+0000 7f53bffff700 1 --2- 172.21.15.31:0/2181875312 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f53c00c8330 0x7f53c0156e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:36.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:36.552+0000 7f53bf7fe700 1 --2- 172.21.15.31:0/2181875312 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53c0140a70 0x7f53c00c37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:36.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:36.552+0000 7f53c499c700 1 -- 172.21.15.31:0/2181875312 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f53c00c8330 msgr2=0x7f53c0156e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:36.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:36.552+0000 7f53c499c700 1 --2- 172.21.15.31:0/2181875312 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f53c00c8330 0x7f53c0156e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:36.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:36.552+0000 7f53c499c700 1 -- 172.21.15.31:0/2181875312 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53c0140a70 msgr2=0x7f53c00c37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:36.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:36.552+0000 7f53c499c700 1 --2- 172.21.15.31:0/2181875312 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53c0140a70 0x7f53c00c37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:36.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:36.552+0000 7f53c499c700 1 -- 172.21.15.31:0/2181875312 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53b0009420 con 0x7f53c0140e40 2023-10-06T02:24:36.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:36.552+0000 7f53bffff700 1 --2- 172.21.15.31:0/2181875312 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f53c00c8330 0x7f53c0156e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:24:36.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:36.552+0000 7f53bf7fe700 1 --2- 172.21.15.31:0/2181875312 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53c0140a70 0x7f53c00c37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:24:36.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:36.552+0000 7f53c499c700 1 --2- 172.21.15.31:0/2181875312 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53c0140e40 0x7f53c00b0d80 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f53b800cc70 tx=0x7f53b8001a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:36.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:36.552+0000 7f53a7fff700 1 -- 172.21.15.31:0/2181875312 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b800ce80 con 0x7f53c0140e40 2023-10-06T02:24:36.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:36.552+0000 7f53a7fff700 1 -- 172.21.15.31:0/2181875312 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53b800ab30 con 0x7f53c0140e40 2023-10-06T02:24:36.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:36.552+0000 7f53c519d700 1 -- 172.21.15.31:0/2181875312 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53c00affc0 con 0x7f53c0140e40 2023-10-06T02:24:36.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:36.552+0000 7f53a7fff700 1 -- 172.21.15.31:0/2181875312 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b80070f0 con 0x7f53c0140e40 2023-10-06T02:24:36.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:36.552+0000 7f53c519d700 1 -- 172.21.15.31:0/2181875312 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53c00b0580 con 0x7f53c0140e40 2023-10-06T02:24:36.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:36.553+0000 7f53c519d700 1 -- 172.21.15.31:0/2181875312 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53c00b0aa0 con 0x7f53c0140e40 2023-10-06T02:24:36.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:36.554+0000 7f53a7fff700 1 -- 172.21.15.31:0/2181875312 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f53b80074e0 con 0x7f53c0140e40 2023-10-06T02:24:36.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:36.555+0000 7f53a7fff700 1 --2- 172.21.15.31:0/2181875312 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f53ac0670e0 0x7f53ac069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:36.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:36.555+0000 7f53a7fff700 1 -- 172.21.15.31:0/2181875312 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f53b8084b20 con 0x7f53c0140e40 2023-10-06T02:24:36.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:36.555+0000 7f53bffff700 1 --2- 172.21.15.31:0/2181875312 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f53ac0670e0 0x7f53ac069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:36.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:36.556+0000 7f53bffff700 1 --2- 172.21.15.31:0/2181875312 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f53ac0670e0 0x7f53ac069590 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f53b00093f0 tx=0x7f53b0017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:36.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:36.559+0000 7f53a7fff700 1 -- 172.21.15.31:0/2181875312 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f53c00b0aa0 con 0x7f53c0140e40 2023-10-06T02:24:36.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:36.870+0000 7f53c519d700 1 -- 172.21.15.31:0/2181875312 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration"]} v 0) v1 -- 0x7f53c00b0aa0 con 0x7f53c0140e40 2023-10-06T02:24:37.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.339+0000 7f53a7fff700 1 -- 172.21.15.31:0/2181875312 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration"]}]=0 v111) v1 ==== 143+0+0 (secure 0 0 0) 0x7f53c00b0aa0 con 0x7f53c0140e40 2023-10-06T02:24:37.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.342+0000 7f53c519d700 1 -- 172.21.15.31:0/2181875312 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f53ac0670e0 msgr2=0x7f53ac069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:37.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.342+0000 7f53c519d700 1 --2- 172.21.15.31:0/2181875312 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f53ac0670e0 0x7f53ac069590 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f53b00093f0 tx=0x7f53b0017000 comp rx=0 tx=0).stop 2023-10-06T02:24:37.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.342+0000 7f53c519d700 1 -- 172.21.15.31:0/2181875312 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53c0140e40 msgr2=0x7f53c00b0d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:37.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.342+0000 7f53c519d700 1 --2- 172.21.15.31:0/2181875312 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53c0140e40 0x7f53c00b0d80 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f53b800cc70 tx=0x7f53b8001a50 comp rx=0 tx=0).stop 2023-10-06T02:24:37.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.342+0000 7f53c519d700 1 -- 172.21.15.31:0/2181875312 shutdown_connections 2023-10-06T02:24:37.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.342+0000 7f53c519d700 1 --2- 172.21.15.31:0/2181875312 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53c0140e40 0x7f53c00b0d80 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:37.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.342+0000 7f53c519d700 1 --2- 172.21.15.31:0/2181875312 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f53ac0670e0 0x7f53ac069590 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:37.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.342+0000 7f53c519d700 1 --2- 172.21.15.31:0/2181875312 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f53c00c8330 0x7f53c0156e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:37.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.342+0000 7f53c519d700 1 --2- 172.21.15.31:0/2181875312 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53c0140a70 0x7f53c00c37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:37.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.342+0000 7f53c519d700 1 -- 172.21.15.31:0/2181875312 >> 172.21.15.31:0/2181875312 conn(0x7f53c006d170 msgr2=0x7f53c00cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:37.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.342+0000 7f53c519d700 1 -- 172.21.15.31:0/2181875312 shutdown_connections 2023-10-06T02:24:37.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.342+0000 7f53c519d700 1 -- 172.21.15.31:0/2181875312 wait complete. 2023-10-06T02:24:37.361 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T02:24:37.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.552+0000 7f608bfff700 1 Processor -- start 2023-10-06T02:24:37.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.552+0000 7f608bfff700 1 -- start start 2023-10-06T02:24:37.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.553+0000 7f608bfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f608c0c7fb0 0x7f608c06abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:37.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.553+0000 7f608bfff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f608c13f780 0x7f608c13fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:37.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.553+0000 7f608bfff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f608c140120 con 0x7f608c0c7be0 2023-10-06T02:24:37.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.554+0000 7f608bfff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f608c06bf10 con 0x7f608c0c7fb0 2023-10-06T02:24:37.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.554+0000 7f608bfff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f608c06b290 con 0x7f608c13f780 2023-10-06T02:24:37.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.554+0000 7f608affd700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f608c0c7be0 0x7f608c15a990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58142/0 (socket says 172.21.15.31:58142) 2023-10-06T02:24:37.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.554+0000 7f608affd700 1 -- 172.21.15.31:0/2605789139 learned_addr learned my addr 172.21.15.31:0/2605789139 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:24:37.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.554+0000 7f608a7fc700 1 --2- 172.21.15.31:0/2605789139 >> v2:172.21.15.31:3300/0 conn(0x7f608c0c7fb0 0x7f608c06abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:37.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.554+0000 7f608b7fe700 1 --2- 172.21.15.31:0/2605789139 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f608c13f780 0x7f608c13fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:37.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.554+0000 7f6089ffb700 1 -- 172.21.15.31:0/2605789139 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3091620023 0 0) 0x7f607c001110 con 0x7f608c0c7be0 2023-10-06T02:24:37.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.554+0000 7f608a7fc700 1 -- 172.21.15.31:0/2605789139 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f608c13f780 msgr2=0x7f608c13fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:37.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.554+0000 7f608a7fc700 1 --2- 172.21.15.31:0/2605789139 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f608c13f780 0x7f608c13fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:37.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.554+0000 7f608a7fc700 1 -- 172.21.15.31:0/2605789139 >> v1:172.21.15.31:6789/0 conn(0x7f608c0c7be0 legacy=0x7f608c15a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:37.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.554+0000 7f608a7fc700 1 -- 172.21.15.31:0/2605789139 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f608c06b570 con 0x7f608c0c7fb0 2023-10-06T02:24:37.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.555+0000 7f608a7fc700 1 --2- 172.21.15.31:0/2605789139 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f608c0c7fb0 0x7f608c06abd0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f60800098e0 tx=0x7f60800091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb0d8b99e05debc2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:37.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.555+0000 7f6089ffb700 1 -- 172.21.15.31:0/2605789139 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6080009e80 con 0x7f608c0c7fb0 2023-10-06T02:24:37.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.555+0000 7f6089ffb700 1 -- 172.21.15.31:0/2605789139 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f608000b860 con 0x7f608c0c7fb0 2023-10-06T02:24:37.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.555+0000 7f6089ffb700 1 -- 172.21.15.31:0/2605789139 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f608000bbe0 con 0x7f608c0c7fb0 2023-10-06T02:24:37.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.555+0000 7f608bfff700 1 -- 172.21.15.31:0/2605789139 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f608c0c7fb0 msgr2=0x7f608c06abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:37.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.555+0000 7f608bfff700 1 --2- 172.21.15.31:0/2605789139 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f608c0c7fb0 0x7f608c06abd0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f60800098e0 tx=0x7f60800091a0 comp rx=0 tx=0).stop 2023-10-06T02:24:37.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.555+0000 7f608bfff700 1 -- 172.21.15.31:0/2605789139 shutdown_connections 2023-10-06T02:24:37.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.555+0000 7f608bfff700 1 --2- 172.21.15.31:0/2605789139 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f608c13f780 0x7f608c13fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:37.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.555+0000 7f608bfff700 1 --2- 172.21.15.31:0/2605789139 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f608c0c7fb0 0x7f608c06abd0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:37.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.555+0000 7f608bfff700 1 -- 172.21.15.31:0/2605789139 >> 172.21.15.31:0/2605789139 conn(0x7f608c06d110 msgr2=0x7f608c0cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:37.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.555+0000 7f608bfff700 1 -- 172.21.15.31:0/2605789139 shutdown_connections 2023-10-06T02:24:37.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.556+0000 7f608bfff700 1 -- 172.21.15.31:0/2605789139 wait complete. 2023-10-06T02:24:37.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.557+0000 7f608bfff700 1 Processor -- start 2023-10-06T02:24:37.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.557+0000 7f608bfff700 1 -- start start 2023-10-06T02:24:37.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.558+0000 7f608bfff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f608c0c7be0 0x7f608c0a9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:37.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.558+0000 7f608bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f608c0c7fb0 0x7f608c0c41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:37.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.558+0000 7f608bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f608c13f780 0x7f608c157040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:37.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.558+0000 7f608bfff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f608c0c46f0 con 0x7f608c0c7fb0 2023-10-06T02:24:37.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.558+0000 7f608bfff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f608c0c5530 con 0x7f608c13f780 2023-10-06T02:24:37.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.558+0000 7f608bfff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f608c0c4920 con 0x7f608c0c7be0 2023-10-06T02:24:37.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.558+0000 7f608a7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f608c0c7fb0 0x7f608c0c41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:37.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.558+0000 7f608a7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f608c0c7fb0 0x7f608c0c41b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40386/0 (socket says 172.21.15.31:40386) 2023-10-06T02:24:37.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.558+0000 7f608a7fc700 1 -- 172.21.15.31:0/1411274655 learned_addr learned my addr 172.21.15.31:0/1411274655 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:37.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.558+0000 7f608affd700 1 --2- 172.21.15.31:0/1411274655 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f608c0c7be0 0x7f608c0a9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:37.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.559+0000 7f608b7fe700 1 --2- 172.21.15.31:0/1411274655 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f608c13f780 0x7f608c157040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:37.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.559+0000 7f608a7fc700 1 -- 172.21.15.31:0/1411274655 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f608c0c7be0 msgr2=0x7f608c0a9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:37.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.559+0000 7f608a7fc700 1 --2- 172.21.15.31:0/1411274655 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f608c0c7be0 0x7f608c0a9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:37.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.559+0000 7f608a7fc700 1 -- 172.21.15.31:0/1411274655 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f608c13f780 msgr2=0x7f608c157040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:37.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.559+0000 7f608a7fc700 1 --2- 172.21.15.31:0/1411274655 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f608c13f780 0x7f608c157040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:37.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.559+0000 7f608a7fc700 1 -- 172.21.15.31:0/1411274655 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6080009420 con 0x7f608c0c7fb0 2023-10-06T02:24:37.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.559+0000 7f608a7fc700 1 --2- 172.21.15.31:0/1411274655 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f608c0c7fb0 0x7f608c0c41b0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f608c0693f0 tx=0x7f6080019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:37.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.559+0000 7f6072ffd700 1 -- 172.21.15.31:0/1411274655 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f608000bbe0 con 0x7f608c0c7fb0 2023-10-06T02:24:37.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.559+0000 7f6072ffd700 1 -- 172.21.15.31:0/1411274655 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6080010470 con 0x7f608c0c7fb0 2023-10-06T02:24:37.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.559+0000 7f6072ffd700 1 -- 172.21.15.31:0/1411274655 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6080010830 con 0x7f608c0c7fb0 2023-10-06T02:24:37.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.559+0000 7f608bfff700 1 -- 172.21.15.31:0/1411274655 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f608c0c4c00 con 0x7f608c0c7fb0 2023-10-06T02:24:37.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.559+0000 7f608bfff700 1 -- 172.21.15.31:0/1411274655 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f608c0c51c0 con 0x7f608c0c7fb0 2023-10-06T02:24:37.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.560+0000 7f608bfff700 1 -- 172.21.15.31:0/1411274655 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f608c04c3c0 con 0x7f608c0c7fb0 2023-10-06T02:24:37.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.561+0000 7f6072ffd700 1 -- 172.21.15.31:0/1411274655 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f60800183e0 con 0x7f608c0c7fb0 2023-10-06T02:24:37.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.562+0000 7f6072ffd700 1 --2- 172.21.15.31:0/1411274655 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f60780670e0 0x7f6078069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:37.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.563+0000 7f6072ffd700 1 -- 172.21.15.31:0/1411274655 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f6080014070 con 0x7f608c0c7fb0 2023-10-06T02:24:37.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.563+0000 7f608affd700 1 --2- 172.21.15.31:0/1411274655 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f60780670e0 0x7f6078069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:37.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.563+0000 7f608affd700 1 --2- 172.21.15.31:0/1411274655 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f60780670e0 0x7f6078069590 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f607c001800 tx=0x7f607c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:37.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.566+0000 7f6072ffd700 1 -- 172.21.15.31:0/1411274655 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f60800534b0 con 0x7f608c0c7fb0 2023-10-06T02:24:37.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.772+0000 7f608bfff700 1 -- 172.21.15.31:0/1411274655 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f608c04c3c0 con 0x7f608c0c7fb0 2023-10-06T02:24:37.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.777+0000 7f6072ffd700 1 -- 172.21.15.31:0/1411274655 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6080056ad0 con 0x7f608c0c7fb0 2023-10-06T02:24:37.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.779+0000 7f608bfff700 1 -- 172.21.15.31:0/1411274655 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f60780670e0 msgr2=0x7f6078069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:37.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.779+0000 7f608bfff700 1 --2- 172.21.15.31:0/1411274655 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f60780670e0 0x7f6078069590 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f607c001800 tx=0x7f607c009040 comp rx=0 tx=0).stop 2023-10-06T02:24:37.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.779+0000 7f608bfff700 1 -- 172.21.15.31:0/1411274655 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f608c0c7fb0 msgr2=0x7f608c0c41b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:37.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.779+0000 7f608bfff700 1 --2- 172.21.15.31:0/1411274655 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f608c0c7fb0 0x7f608c0c41b0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f608c0693f0 tx=0x7f6080019000 comp rx=0 tx=0).stop 2023-10-06T02:24:37.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.779+0000 7f608bfff700 1 -- 172.21.15.31:0/1411274655 shutdown_connections 2023-10-06T02:24:37.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.779+0000 7f608bfff700 1 --2- 172.21.15.31:0/1411274655 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f608c13f780 0x7f608c157040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:37.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.779+0000 7f608bfff700 1 --2- 172.21.15.31:0/1411274655 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f60780670e0 0x7f6078069590 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:37.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.779+0000 7f608bfff700 1 --2- 172.21.15.31:0/1411274655 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f608c0c7be0 0x7f608c0a9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:37.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.779+0000 7f608bfff700 1 --2- 172.21.15.31:0/1411274655 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f608c0c7fb0 0x7f608c0c41b0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:37.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.779+0000 7f608bfff700 1 -- 172.21.15.31:0/1411274655 >> 172.21.15.31:0/1411274655 conn(0x7f608c06d110 msgr2=0x7f608c0c0f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:37.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.779+0000 7f608bfff700 1 -- 172.21.15.31:0/1411274655 shutdown_connections 2023-10-06T02:24:37.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:37.779+0000 7f608bfff700 1 -- 172.21.15.31:0/1411274655 wait complete. 2023-10-06T02:24:37.797 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T02:24:37.797 DEBUG:teuthology.orchestra.run.smithi031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T02:24:37.872 DEBUG:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T02:24:37.957 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T02:24:37.957 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T02:24:37.957 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:24:38.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.143+0000 7feb8099d700 1 Processor -- start 2023-10-06T02:24:38.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.143+0000 7feb8099d700 1 -- start start 2023-10-06T02:24:38.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.144+0000 7feb8099d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7feb7c06e0a0 0x7feb7c0c6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:38.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.145+0000 7feb8099d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb7c06a490 con 0x7feb7c06f3a0 2023-10-06T02:24:38.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.145+0000 7feb8099d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb7c0c7490 con 0x7feb7c150380 2023-10-06T02:24:38.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.145+0000 7feb8099d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb7c0c7770 con 0x7feb7c06e0a0 2023-10-06T02:24:38.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.145+0000 7feb7b7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7feb7c06e0a0 0x7feb7c0c6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:38.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.145+0000 7feb7b7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7feb7c06e0a0 0x7feb7c0c6b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40936/0 (socket says 172.21.15.31:40936) 2023-10-06T02:24:38.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.145+0000 7feb7b7fe700 1 -- 172.21.15.31:0/356011381 learned_addr learned my addr 172.21.15.31:0/356011381 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:38.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.145+0000 7feb7affd700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7feb7c06f3a0 0x7feb7c14fc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58196/0 (socket says 172.21.15.31:58196) 2023-10-06T02:24:38.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.145+0000 7feb7b7fe700 1 -- 172.21.15.31:0/356011381 >> v1:172.21.15.62:6789/0 conn(0x7feb7c150380 legacy=0x7feb7c06a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:38.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.145+0000 7feb7b7fe700 1 -- 172.21.15.31:0/356011381 >> v1:172.21.15.31:6789/0 conn(0x7feb7c06f3a0 legacy=0x7feb7c14fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:38.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.145+0000 7feb7a7fc700 1 -- 172.21.15.31:0/356011381 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2464862591 0 0) 0x7feb7c06a490 con 0x7feb7c06f3a0 2023-10-06T02:24:38.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.145+0000 7feb7b7fe700 1 -- 172.21.15.31:0/356011381 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb7c0c7a50 con 0x7feb7c06e0a0 2023-10-06T02:24:38.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.146+0000 7feb7b7fe700 1 --2- 172.21.15.31:0/356011381 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb7c06e0a0 0x7feb7c0c6b20 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7feb6c0099b0 tx=0x7feb6c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2096e182a12ce56 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:38.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.146+0000 7feb7a7fc700 1 -- 172.21.15.31:0/356011381 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb6c00b860 con 0x7feb7c06e0a0 2023-10-06T02:24:38.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.146+0000 7feb7a7fc700 1 -- 172.21.15.31:0/356011381 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb6c00b9c0 con 0x7feb7c06e0a0 2023-10-06T02:24:38.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.146+0000 7feb7a7fc700 1 -- 172.21.15.31:0/356011381 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb6c00bd60 con 0x7feb7c06e0a0 2023-10-06T02:24:38.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.146+0000 7feb8099d700 1 -- 172.21.15.31:0/356011381 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb7c06e0a0 msgr2=0x7feb7c0c6b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:38.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.146+0000 7feb8099d700 1 --2- 172.21.15.31:0/356011381 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb7c06e0a0 0x7feb7c0c6b20 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7feb6c0099b0 tx=0x7feb6c009250 comp rx=0 tx=0).stop 2023-10-06T02:24:38.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.147+0000 7feb8099d700 1 -- 172.21.15.31:0/356011381 shutdown_connections 2023-10-06T02:24:38.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.147+0000 7feb8099d700 1 --2- 172.21.15.31:0/356011381 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb7c06e0a0 0x7feb7c0c6b20 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:38.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.147+0000 7feb8099d700 1 -- 172.21.15.31:0/356011381 >> 172.21.15.31:0/356011381 conn(0x7feb7c06d170 msgr2=0x7feb7c0cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:38.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.147+0000 7feb8099d700 1 -- 172.21.15.31:0/356011381 shutdown_connections 2023-10-06T02:24:38.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.147+0000 7feb8099d700 1 -- 172.21.15.31:0/356011381 wait complete. 2023-10-06T02:24:38.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.148+0000 7feb8099d700 1 Processor -- start 2023-10-06T02:24:38.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.148+0000 7feb8099d700 1 -- start start 2023-10-06T02:24:38.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.149+0000 7feb8099d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feb7c06e0a0 0x7feb7c148cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:38.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.149+0000 7feb8099d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb7c06f3a0 0x7feb7c156cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:38.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.149+0000 7feb8099d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb7c150380 0x7feb7c0b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:38.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.149+0000 7feb8099d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb7c0b1120 con 0x7feb7c150380 2023-10-06T02:24:38.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.149+0000 7feb8099d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb7c0b0510 con 0x7feb7c06f3a0 2023-10-06T02:24:38.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.149+0000 7feb8099d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb7c0b07f0 con 0x7feb7c06e0a0 2023-10-06T02:24:38.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.149+0000 7feb7bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb7c150380 0x7feb7c0b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:38.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.149+0000 7feb7bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb7c150380 0x7feb7c0b15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40944/0 (socket says 172.21.15.31:40944) 2023-10-06T02:24:38.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.149+0000 7feb7bfff700 1 -- 172.21.15.31:0/3151597869 learned_addr learned my addr 172.21.15.31:0/3151597869 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:38.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.150+0000 7feb7affd700 1 --2- 172.21.15.31:0/3151597869 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb7c06f3a0 0x7feb7c156cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:38.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.150+0000 7feb7b7fe700 1 --2- 172.21.15.31:0/3151597869 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feb7c06e0a0 0x7feb7c148cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:38.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.150+0000 7feb7bfff700 1 -- 172.21.15.31:0/3151597869 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feb7c06e0a0 msgr2=0x7feb7c148cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:38.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.150+0000 7feb7bfff700 1 --2- 172.21.15.31:0/3151597869 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feb7c06e0a0 0x7feb7c148cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:38.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.150+0000 7feb7bfff700 1 -- 172.21.15.31:0/3151597869 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb7c06f3a0 msgr2=0x7feb7c156cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:38.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.150+0000 7feb7bfff700 1 --2- 172.21.15.31:0/3151597869 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb7c06f3a0 0x7feb7c156cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:38.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.150+0000 7feb7bfff700 1 -- 172.21.15.31:0/3151597869 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb6c0094f0 con 0x7feb7c150380 2023-10-06T02:24:38.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.150+0000 7feb7bfff700 1 --2- 172.21.15.31:0/3151597869 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb7c150380 0x7feb7c0b15b0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7feb7400c810 tx=0x7feb74001650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:38.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.150+0000 7feb637fe700 1 -- 172.21.15.31:0/3151597869 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb74012630 con 0x7feb7c150380 2023-10-06T02:24:38.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.150+0000 7feb637fe700 1 -- 172.21.15.31:0/3151597869 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb740127b0 con 0x7feb7c150380 2023-10-06T02:24:38.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.150+0000 7feb637fe700 1 -- 172.21.15.31:0/3151597869 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb74012b70 con 0x7feb7c150380 2023-10-06T02:24:38.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.150+0000 7feb8099d700 1 -- 172.21.15.31:0/3151597869 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb7c0b0ad0 con 0x7feb7c150380 2023-10-06T02:24:38.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.150+0000 7feb8099d700 1 -- 172.21.15.31:0/3151597869 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb7c1491f0 con 0x7feb7c150380 2023-10-06T02:24:38.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.151+0000 7feb8099d700 1 -- 172.21.15.31:0/3151597869 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb7c04c3c0 con 0x7feb7c150380 2023-10-06T02:24:38.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.152+0000 7feb637fe700 1 -- 172.21.15.31:0/3151597869 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7feb7401b400 con 0x7feb7c150380 2023-10-06T02:24:38.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.153+0000 7feb637fe700 1 --2- 172.21.15.31:0/3151597869 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7feb680670e0 0x7feb68069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:38.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.153+0000 7feb637fe700 1 -- 172.21.15.31:0/3151597869 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5292+0+0 (secure 0 0 0) 0x7feb740845d0 con 0x7feb7c150380 2023-10-06T02:24:38.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.153+0000 7feb7b7fe700 1 --2- 172.21.15.31:0/3151597869 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7feb680670e0 0x7feb68069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:38.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.154+0000 7feb7b7fe700 1 --2- 172.21.15.31:0/3151597869 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7feb680670e0 0x7feb68069590 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7feb6c0099b0 tx=0x7feb6c017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:38.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.157+0000 7feb637fe700 1 -- 172.21.15.31:0/3151597869 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7feb74051760 con 0x7feb7c150380 2023-10-06T02:24:38.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.410+0000 7feb8099d700 1 -- 172.21.15.31:0/3151597869 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feb7c0c7d30 con 0x7feb7c150380 2023-10-06T02:24:38.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.411+0000 7feb637fe700 1 -- 172.21.15.31:0/3151597869 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+3523 (secure 0 0 0) 0x7feb74054d80 con 0x7feb7c150380 2023-10-06T02:24:38.411 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:24:38.411 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4700,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/384287452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":384287452},{"type":"v1","addr":"172.21.15.62:6834","nonce":384287452}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5037,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/693643908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":693643908},{"type":"v1","addr":"172.21.15.31:6836","nonce":693643908}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5040,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2599441423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2599441423},{"type":"v1","addr":"172.21.15.62:6835","nonce":2599441423}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5364,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3646820434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":3646820434},{"type":"v1","addr":"172.21.15.31:6837","nonce":3646820434}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[]} 2023-10-06T02:24:38.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.413+0000 7feb8099d700 1 -- 172.21.15.31:0/3151597869 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7feb680670e0 msgr2=0x7feb68069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:38.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.413+0000 7feb8099d700 1 --2- 172.21.15.31:0/3151597869 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7feb680670e0 0x7feb68069590 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7feb6c0099b0 tx=0x7feb6c017000 comp rx=0 tx=0).stop 2023-10-06T02:24:38.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.413+0000 7feb8099d700 1 -- 172.21.15.31:0/3151597869 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb7c150380 msgr2=0x7feb7c0b15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:38.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.413+0000 7feb8099d700 1 --2- 172.21.15.31:0/3151597869 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb7c150380 0x7feb7c0b15b0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7feb7400c810 tx=0x7feb74001650 comp rx=0 tx=0).stop 2023-10-06T02:24:38.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.413+0000 7feb8099d700 1 -- 172.21.15.31:0/3151597869 shutdown_connections 2023-10-06T02:24:38.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.413+0000 7feb8099d700 1 --2- 172.21.15.31:0/3151597869 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb7c150380 0x7feb7c0b15b0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:38.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.413+0000 7feb8099d700 1 --2- 172.21.15.31:0/3151597869 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7feb680670e0 0x7feb68069590 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:38.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.413+0000 7feb8099d700 1 --2- 172.21.15.31:0/3151597869 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feb7c06e0a0 0x7feb7c148cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:38.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.413+0000 7feb8099d700 1 --2- 172.21.15.31:0/3151597869 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb7c06f3a0 0x7feb7c156cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:38.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.413+0000 7feb8099d700 1 -- 172.21.15.31:0/3151597869 >> 172.21.15.31:0/3151597869 conn(0x7feb7c06d170 msgr2=0x7feb7c1596d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:38.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.413+0000 7feb8099d700 1 -- 172.21.15.31:0/3151597869 shutdown_connections 2023-10-06T02:24:38.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.413+0000 7feb8099d700 1 -- 172.21.15.31:0/3151597869 wait complete. 2023-10-06T02:24:38.415 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 20 2023-10-06T02:24:38.431 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T02:24:38.431 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T02:24:38.431 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T02:24:38.432 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T02:24:38.432 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T02:24:38.432 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T02:24:38.432 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T02:24:38.432 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T02:24:38.432 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T02:24:38.432 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T02:24:38.433 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T02:24:38.433 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T02:24:38.433 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T02:24:38.433 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T02:24:38.433 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T02:24:38.433 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T02:24:38.445 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T02:24:38.445 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T02:24:38.446 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T02:24:38.446 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T02:24:38.449 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T02:24:38.449 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T02:24:38.450 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T02:24:38.450 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T02:24:38.490 INFO:tasks.ceph.mds.a:Started 2023-10-06T02:24:38.490 DEBUG:teuthology.parallel:result is None 2023-10-06T02:24:38.492 INFO:tasks.ceph.mds.c:Started 2023-10-06T02:24:38.492 DEBUG:teuthology.parallel:result is None 2023-10-06T02:24:38.494 INFO:tasks.ceph.mds.b:Started 2023-10-06T02:24:38.494 DEBUG:teuthology.parallel:result is None 2023-10-06T02:24:38.497 INFO:tasks.ceph.mds.d:Started 2023-10-06T02:24:38.497 DEBUG:teuthology.parallel:result is None 2023-10-06T02:24:38.498 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T02:24:38.629 INFO:tasks.ceph.mds.a.smithi031.stdout:starting mds.a at 2023-10-06T02:24:38.630 INFO:tasks.ceph.mds.c.smithi031.stdout:starting mds.c at 2023-10-06T02:24:38.638 INFO:tasks.ceph.mds.d.smithi062.stdout:starting mds.d at 2023-10-06T02:24:38.639 INFO:tasks.ceph.mds.b.smithi062.stdout:starting mds.b at 2023-10-06T02:24:38.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.697+0000 7f7bb919d700 1 Processor -- start 2023-10-06T02:24:38.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.697+0000 7f7bb919d700 1 -- start start 2023-10-06T02:24:38.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.698+0000 7f7bb919d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7bb40b2980 0x7f7bb40cce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:38.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.698+0000 7f7bb919d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7bb415a8c0 0x7f7bb406d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:38.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.698+0000 7f7bb919d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7bb406ddd0 0x7f7bb415d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:38.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.698+0000 7f7bb919d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bb406e240 con 0x7f7bb415a8c0 2023-10-06T02:24:38.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.698+0000 7f7bb919d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bb40cec30 con 0x7f7bb40b2980 2023-10-06T02:24:38.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.698+0000 7f7bb919d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb40ce930 con 0x7f7bb406ddd0 2023-10-06T02:24:38.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.699+0000 7f7bb37fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7bb415a8c0 0x7f7bb406d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:38.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.699+0000 7f7bb37fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7bb415a8c0 0x7f7bb406d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40996/0 (socket says 172.21.15.31:40996) 2023-10-06T02:24:38.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.699+0000 7f7bb37fe700 1 -- 172.21.15.31:0/3319854240 learned_addr learned my addr 172.21.15.31:0/3319854240 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:38.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.699+0000 7f7bb899c700 1 --2- 172.21.15.31:0/3319854240 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7bb406ddd0 0x7f7bb415d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:38.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.699+0000 7f7bb3fff700 1 --2- 172.21.15.31:0/3319854240 >> v2:172.21.15.62:3300/0 conn(0x7f7bb40b2980 0x7f7bb40cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:38.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.699+0000 7f7bb37fe700 1 -- 172.21.15.31:0/3319854240 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7bb406ddd0 msgr2=0x7f7bb415d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:38.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.699+0000 7f7bb37fe700 1 --2- 172.21.15.31:0/3319854240 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7bb406ddd0 0x7f7bb415d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:38.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.699+0000 7f7bb37fe700 1 -- 172.21.15.31:0/3319854240 >> v2:172.21.15.62:3300/0 conn(0x7f7bb40b2980 msgr2=0x7f7bb40cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:38.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.699+0000 7f7bb37fe700 1 --2- 172.21.15.31:0/3319854240 >> v2:172.21.15.62:3300/0 conn(0x7f7bb40b2980 0x7f7bb40cce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:38.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.699+0000 7f7bb37fe700 1 -- 172.21.15.31:0/3319854240 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bb40cd7a0 con 0x7f7bb415a8c0 2023-10-06T02:24:38.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.699+0000 7f7bb3fff700 1 --2- 172.21.15.31:0/3319854240 >> v2:172.21.15.62:3300/0 conn(0x7f7bb40b2980 0x7f7bb40cce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:24:38.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.699+0000 7f7bb37fe700 1 --2- 172.21.15.31:0/3319854240 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bb415a8c0 0x7f7bb406d890 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f7bb40d15e0 tx=0x7f7ba8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8165b871f4cbb594 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:38.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.700+0000 7f7bb2ffd700 1 -- 172.21.15.31:0/3319854240 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba800b640 con 0x7f7bb415a8c0 2023-10-06T02:24:38.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.700+0000 7f7bb2ffd700 1 -- 172.21.15.31:0/3319854240 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ba800b7a0 con 0x7f7bb415a8c0 2023-10-06T02:24:38.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.700+0000 7f7bb2ffd700 1 -- 172.21.15.31:0/3319854240 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba800bb60 con 0x7f7bb415a8c0 2023-10-06T02:24:38.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.700+0000 7f7bb919d700 1 -- 172.21.15.31:0/3319854240 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bb415a8c0 msgr2=0x7f7bb406d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:38.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.700+0000 7f7bb919d700 1 --2- 172.21.15.31:0/3319854240 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bb415a8c0 0x7f7bb406d890 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f7bb40d15e0 tx=0x7f7ba8009250 comp rx=0 tx=0).stop 2023-10-06T02:24:38.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.700+0000 7f7bb919d700 1 -- 172.21.15.31:0/3319854240 shutdown_connections 2023-10-06T02:24:38.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.700+0000 7f7bb919d700 1 --2- 172.21.15.31:0/3319854240 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7bb406ddd0 0x7f7bb415d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:38.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.700+0000 7f7bb919d700 1 --2- 172.21.15.31:0/3319854240 >> v2:172.21.15.62:3300/0 conn(0x7f7bb40b2980 0x7f7bb40cce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:38.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.700+0000 7f7bb919d700 1 --2- 172.21.15.31:0/3319854240 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bb415a8c0 0x7f7bb406d890 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:38.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.700+0000 7f7bb919d700 1 -- 172.21.15.31:0/3319854240 >> 172.21.15.31:0/3319854240 conn(0x7f7bb40703a0 msgr2=0x7f7bb40707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:38.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.700+0000 7f7bb919d700 1 -- 172.21.15.31:0/3319854240 shutdown_connections 2023-10-06T02:24:38.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.700+0000 7f7bb919d700 1 -- 172.21.15.31:0/3319854240 wait complete. 2023-10-06T02:24:38.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.702+0000 7f7bb919d700 1 Processor -- start 2023-10-06T02:24:38.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.702+0000 7f7bb919d700 1 -- start start 2023-10-06T02:24:38.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.703+0000 7f7bb919d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7bb406ddd0 0x7f7bb416da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:38.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.703+0000 7f7bb919d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7bb40b2980 0x7f7bb4148140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:38.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.703+0000 7f7bb919d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bb415a8c0 0x7f7bb40b1420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:38.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.703+0000 7f7bb919d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb414b070 con 0x7f7bb415a8c0 2023-10-06T02:24:38.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.703+0000 7f7bb919d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb414abe0 con 0x7f7bb406ddd0 2023-10-06T02:24:38.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.703+0000 7f7bb919d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb4149fd0 con 0x7f7bb40b2980 2023-10-06T02:24:38.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.703+0000 7f7bb899c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bb415a8c0 0x7f7bb40b1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:38.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.703+0000 7f7bb899c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bb415a8c0 0x7f7bb40b1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41000/0 (socket says 172.21.15.31:41000) 2023-10-06T02:24:38.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.703+0000 7f7bb899c700 1 -- 172.21.15.31:0/690775349 learned_addr learned my addr 172.21.15.31:0/690775349 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:38.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.703+0000 7f7bb3fff700 1 --2- 172.21.15.31:0/690775349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7bb406ddd0 0x7f7bb416da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:38.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.703+0000 7f7bb37fe700 1 --2- 172.21.15.31:0/690775349 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7bb40b2980 0x7f7bb4148140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:38.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.703+0000 7f7bb899c700 1 -- 172.21.15.31:0/690775349 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7bb40b2980 msgr2=0x7f7bb4148140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:38.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.703+0000 7f7bb899c700 1 --2- 172.21.15.31:0/690775349 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7bb40b2980 0x7f7bb4148140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:38.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.703+0000 7f7bb899c700 1 -- 172.21.15.31:0/690775349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7bb406ddd0 msgr2=0x7f7bb416da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:38.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.703+0000 7f7bb899c700 1 --2- 172.21.15.31:0/690775349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7bb406ddd0 0x7f7bb416da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:38.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.703+0000 7f7bb899c700 1 -- 172.21.15.31:0/690775349 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ba4009420 con 0x7f7bb415a8c0 2023-10-06T02:24:38.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.703+0000 7f7bb37fe700 1 --2- 172.21.15.31:0/690775349 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7bb40b2980 0x7f7bb4148140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:24:38.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.704+0000 7f7bb899c700 1 --2- 172.21.15.31:0/690775349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bb415a8c0 0x7f7bb40b1420 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f7bac009df0 tx=0x7f7bac00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:38.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.704+0000 7f7b9bfff700 1 -- 172.21.15.31:0/690775349 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bac00cb50 con 0x7f7bb415a8c0 2023-10-06T02:24:38.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.704+0000 7f7b9bfff700 1 -- 172.21.15.31:0/690775349 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bac00f040 con 0x7f7bb415a8c0 2023-10-06T02:24:38.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.704+0000 7f7b9bfff700 1 -- 172.21.15.31:0/690775349 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bac014480 con 0x7f7bb415a8c0 2023-10-06T02:24:38.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.704+0000 7f7bb919d700 1 -- 172.21.15.31:0/690775349 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ba80094f0 con 0x7f7bb415a8c0 2023-10-06T02:24:38.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.704+0000 7f7bb919d700 1 -- 172.21.15.31:0/690775349 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bb414a590 con 0x7f7bb415a8c0 2023-10-06T02:24:38.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.705+0000 7f7bb919d700 1 -- 172.21.15.31:0/690775349 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bb414ad70 con 0x7f7bb415a8c0 2023-10-06T02:24:38.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.706+0000 7f7b9bfff700 1 -- 172.21.15.31:0/690775349 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f7bac014830 con 0x7f7bb415a8c0 2023-10-06T02:24:38.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.707+0000 7f7b9bfff700 1 --2- 172.21.15.31:0/690775349 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f7ba00670e0 0x7f7ba0069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:38.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.707+0000 7f7b9bfff700 1 -- 172.21.15.31:0/690775349 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f7bac085160 con 0x7f7bb415a8c0 2023-10-06T02:24:38.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.707+0000 7f7bb3fff700 1 --2- 172.21.15.31:0/690775349 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f7ba00670e0 0x7f7ba0069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:38.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.708+0000 7f7bb3fff700 1 --2- 172.21.15.31:0/690775349 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f7ba00670e0 0x7f7ba0069590 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7ba4009f40 tx=0x7f7ba40091a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:38.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.711+0000 7f7b9bfff700 1 -- 172.21.15.31:0/690775349 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7bac052140 con 0x7f7bb415a8c0 2023-10-06T02:24:38.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:38.929+0000 7f7bb919d700 1 -- 172.21.15.31:0/690775349 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f7bb404c3c0 con 0x7f7bb415a8c0 2023-10-06T02:24:39.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.353+0000 7f7b9bfff700 1 -- 172.21.15.31:0/690775349 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v41) v1 ==== 97+0+0 (secure 0 0 0) 0x7f7bac055760 con 0x7f7bb415a8c0 2023-10-06T02:24:39.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.356+0000 7f7bb919d700 1 -- 172.21.15.31:0/690775349 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f7ba00670e0 msgr2=0x7f7ba0069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:39.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.356+0000 7f7bb919d700 1 --2- 172.21.15.31:0/690775349 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f7ba00670e0 0x7f7ba0069590 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7ba4009f40 tx=0x7f7ba40091a0 comp rx=0 tx=0).stop 2023-10-06T02:24:39.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.356+0000 7f7bb919d700 1 -- 172.21.15.31:0/690775349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bb415a8c0 msgr2=0x7f7bb40b1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:39.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.356+0000 7f7bb919d700 1 --2- 172.21.15.31:0/690775349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bb415a8c0 0x7f7bb40b1420 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f7bac009df0 tx=0x7f7bac00b040 comp rx=0 tx=0).stop 2023-10-06T02:24:39.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.356+0000 7f7bb919d700 1 -- 172.21.15.31:0/690775349 shutdown_connections 2023-10-06T02:24:39.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.356+0000 7f7bb919d700 1 --2- 172.21.15.31:0/690775349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bb415a8c0 0x7f7bb40b1420 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:39.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.356+0000 7f7bb919d700 1 --2- 172.21.15.31:0/690775349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7bb406ddd0 0x7f7bb416da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:39.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.356+0000 7f7bb919d700 1 --2- 172.21.15.31:0/690775349 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f7ba00670e0 0x7f7ba0069590 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:39.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.356+0000 7f7bb919d700 1 --2- 172.21.15.31:0/690775349 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7bb40b2980 0x7f7bb4148140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:39.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.356+0000 7f7bb919d700 1 -- 172.21.15.31:0/690775349 >> 172.21.15.31:0/690775349 conn(0x7f7bb40703a0 msgr2=0x7f7bb414ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:39.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.357+0000 7f7bb919d700 1 -- 172.21.15.31:0/690775349 shutdown_connections 2023-10-06T02:24:39.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.357+0000 7f7bb919d700 1 -- 172.21.15.31:0/690775349 wait complete. 2023-10-06T02:24:39.359 INFO:teuthology.orchestra.run.smithi031.stderr: removed all blocklist entries 2023-10-06T02:24:39.377 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T02:24:39.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.564+0000 7f62a919d700 1 Processor -- start 2023-10-06T02:24:39.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.564+0000 7f62a919d700 1 -- start start 2023-10-06T02:24:39.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.565+0000 7f62a919d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f62a4153c30 0x7f62a415a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:39.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.565+0000 7f62a919d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f62a415de50 0x7f62a40cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:39.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.565+0000 7f62a919d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f62a40cef40 0x7f62a406d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:39.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.565+0000 7f62a919d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62a415afd0 con 0x7f62a415de50 2023-10-06T02:24:39.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.565+0000 7f62a919d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62a415b2b0 con 0x7f62a40cef40 2023-10-06T02:24:39.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.565+0000 7f62a919d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62a415b590 con 0x7f62a4153c30 2023-10-06T02:24:39.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.566+0000 7f62a37fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f62a415de50 0x7f62a40cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:39.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.566+0000 7f62a37fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f62a415de50 0x7f62a40cea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41012/0 (socket says 172.21.15.31:41012) 2023-10-06T02:24:39.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.566+0000 7f62a37fe700 1 -- 172.21.15.31:0/3542604741 learned_addr learned my addr 172.21.15.31:0/3542604741 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:39.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.566+0000 7f62a899c700 1 --2- 172.21.15.31:0/3542604741 >> v2:172.21.15.62:3300/0 conn(0x7f62a40cef40 0x7f62a406d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:39.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.566+0000 7f62a3fff700 1 --2- 172.21.15.31:0/3542604741 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f62a4153c30 0x7f62a415a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:39.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.566+0000 7f62a37fe700 1 -- 172.21.15.31:0/3542604741 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f62a4153c30 msgr2=0x7f62a415a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:39.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.566+0000 7f62a37fe700 1 --2- 172.21.15.31:0/3542604741 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f62a4153c30 0x7f62a415a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:39.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.566+0000 7f62a37fe700 1 -- 172.21.15.31:0/3542604741 >> v2:172.21.15.62:3300/0 conn(0x7f62a40cef40 msgr2=0x7f62a406d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:39.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.566+0000 7f62a37fe700 1 --2- 172.21.15.31:0/3542604741 >> v2:172.21.15.62:3300/0 conn(0x7f62a40cef40 0x7f62a406d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:39.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.566+0000 7f62a37fe700 1 -- 172.21.15.31:0/3542604741 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62a415b870 con 0x7f62a415de50 2023-10-06T02:24:39.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.566+0000 7f62a899c700 1 --2- 172.21.15.31:0/3542604741 >> v2:172.21.15.62:3300/0 conn(0x7f62a40cef40 0x7f62a406d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:24:39.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.566+0000 7f62a37fe700 1 --2- 172.21.15.31:0/3542604741 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62a415de50 0x7f62a40cea00 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f62a4071370 tx=0x7f6298009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=847dc0b625f92a0e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:39.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.567+0000 7f62a2ffd700 1 -- 172.21.15.31:0/3542604741 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6298009e70 con 0x7f62a415de50 2023-10-06T02:24:39.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.567+0000 7f62a2ffd700 1 -- 172.21.15.31:0/3542604741 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f629800b860 con 0x7f62a415de50 2023-10-06T02:24:39.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.567+0000 7f62a2ffd700 1 -- 172.21.15.31:0/3542604741 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629800bc00 con 0x7f62a415de50 2023-10-06T02:24:39.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.567+0000 7f62a919d700 1 -- 172.21.15.31:0/3542604741 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62a415de50 msgr2=0x7f62a40cea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:39.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.567+0000 7f62a919d700 1 --2- 172.21.15.31:0/3542604741 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62a415de50 0x7f62a40cea00 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f62a4071370 tx=0x7f6298009250 comp rx=0 tx=0).stop 2023-10-06T02:24:39.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.567+0000 7f62a919d700 1 -- 172.21.15.31:0/3542604741 shutdown_connections 2023-10-06T02:24:39.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.567+0000 7f62a919d700 1 --2- 172.21.15.31:0/3542604741 >> v2:172.21.15.62:3300/0 conn(0x7f62a40cef40 0x7f62a406d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:39.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.567+0000 7f62a919d700 1 --2- 172.21.15.31:0/3542604741 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f62a4153c30 0x7f62a415a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:39.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.567+0000 7f62a919d700 1 --2- 172.21.15.31:0/3542604741 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62a415de50 0x7f62a40cea00 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:39.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.567+0000 7f62a919d700 1 -- 172.21.15.31:0/3542604741 >> 172.21.15.31:0/3542604741 conn(0x7f62a40703a0 msgr2=0x7f62a40707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:39.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.567+0000 7f62a919d700 1 -- 172.21.15.31:0/3542604741 shutdown_connections 2023-10-06T02:24:39.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.568+0000 7f62a919d700 1 -- 172.21.15.31:0/3542604741 wait complete. 2023-10-06T02:24:39.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.569+0000 7f62a919d700 1 Processor -- start 2023-10-06T02:24:39.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.569+0000 7f62a919d700 1 -- start start 2023-10-06T02:24:39.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.570+0000 7f62a919d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62a40cef40 0x7f62a413dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:39.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.570+0000 7f62a919d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62a4153c30 0x7f62a40b32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:39.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.570+0000 7f62a919d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f62a415de50 0x7f62a40b3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:39.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.570+0000 7f62a919d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62a40b3d50 con 0x7f62a40cef40 2023-10-06T02:24:39.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.570+0000 7f62a919d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62a40c87c0 con 0x7f62a4153c30 2023-10-06T02:24:39.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.570+0000 7f62a919d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62a40c7bb0 con 0x7f62a415de50 2023-10-06T02:24:39.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.570+0000 7f62a3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62a40cef40 0x7f62a413dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:39.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.570+0000 7f62a3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62a40cef40 0x7f62a413dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41028/0 (socket says 172.21.15.31:41028) 2023-10-06T02:24:39.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.570+0000 7f62a3fff700 1 -- 172.21.15.31:0/3396347297 learned_addr learned my addr 172.21.15.31:0/3396347297 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:39.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.570+0000 7f62a3fff700 1 -- 172.21.15.31:0/3396347297 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f62a415de50 msgr2=0x7f62a40b3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:39.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.570+0000 7f62a37fe700 1 --2- 172.21.15.31:0/3396347297 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62a4153c30 0x7f62a40b32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:39.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.570+0000 7f62a3fff700 1 --2- 172.21.15.31:0/3396347297 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f62a415de50 0x7f62a40b3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:39.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.570+0000 7f62a3fff700 1 -- 172.21.15.31:0/3396347297 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62a4153c30 msgr2=0x7f62a40b32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:39.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.570+0000 7f62a3fff700 1 --2- 172.21.15.31:0/3396347297 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62a4153c30 0x7f62a40b32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:39.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.570+0000 7f62a3fff700 1 -- 172.21.15.31:0/3396347297 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62980094f0 con 0x7f62a40cef40 2023-10-06T02:24:39.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.571+0000 7f62a37fe700 1 --2- 172.21.15.31:0/3396347297 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62a4153c30 0x7f62a40b32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:24:39.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.571+0000 7f62a3fff700 1 --2- 172.21.15.31:0/3396347297 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62a40cef40 0x7f62a413dec0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f629400ec80 tx=0x7f62940091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:39.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.571+0000 7f628bfff700 1 -- 172.21.15.31:0/3396347297 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629400c9d0 con 0x7f62a40cef40 2023-10-06T02:24:39.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.571+0000 7f628bfff700 1 -- 172.21.15.31:0/3396347297 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f629400cb30 con 0x7f62a40cef40 2023-10-06T02:24:39.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.571+0000 7f628bfff700 1 -- 172.21.15.31:0/3396347297 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6294013470 con 0x7f62a40cef40 2023-10-06T02:24:39.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.571+0000 7f62a919d700 1 -- 172.21.15.31:0/3396347297 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62a40c7e90 con 0x7f62a40cef40 2023-10-06T02:24:39.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.571+0000 7f62a919d700 1 -- 172.21.15.31:0/3396347297 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62a40c8450 con 0x7f62a40cef40 2023-10-06T02:24:39.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.572+0000 7f62a919d700 1 -- 172.21.15.31:0/3396347297 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62a404c3c0 con 0x7f62a40cef40 2023-10-06T02:24:39.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.573+0000 7f628bfff700 1 -- 172.21.15.31:0/3396347297 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f62940138a0 con 0x7f62a40cef40 2023-10-06T02:24:39.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.574+0000 7f628bfff700 1 --2- 172.21.15.31:0/3396347297 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f62900670e0 0x7f6290069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:39.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.574+0000 7f628bfff700 1 -- 172.21.15.31:0/3396347297 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f6294084a30 con 0x7f62a40cef40 2023-10-06T02:24:39.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.574+0000 7f62a37fe700 1 --2- 172.21.15.31:0/3396347297 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f62900670e0 0x7f6290069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:39.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.575+0000 7f62a37fe700 1 --2- 172.21.15.31:0/3396347297 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f62900670e0 0x7f6290069590 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f62a4071370 tx=0x7f6298017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:39.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.578+0000 7f628bfff700 1 -- 172.21.15.31:0/3396347297 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6294051b70 con 0x7f62a40cef40 2023-10-06T02:24:39.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.859+0000 7f62a919d700 1 -- 172.21.15.31:0/3396347297 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f62a415bb50 con 0x7f62a40cef40 2023-10-06T02:24:39.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.859+0000 7f628bfff700 1 -- 172.21.15.31:0/3396347297 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f6294055190 con 0x7f62a40cef40 2023-10-06T02:24:39.860 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:24:39.860 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T02:24:39.860 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_dump": [ 2023-10-06T02:24:39.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:24:39.860 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.a", 2023-10-06T02:24:39.860 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBLbx9lbjrOGxAAVoSOoDaZb1temL41F4Nong==", 2023-10-06T02:24:39.860 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:24:39.860 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:24:39.860 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T02:24:39.861 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T02:24:39.861 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:24:39.861 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:24:39.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:24:39.861 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:24:39.861 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.b", 2023-10-06T02:24:39.861 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBLbx9lFgsTMRAATTklzroiyQbTLiZc9j1BHQ==", 2023-10-06T02:24:39.861 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:24:39.861 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:24:39.861 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T02:24:39.861 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T02:24:39.861 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:24:39.862 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:24:39.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:24:39.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:24:39.862 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.c", 2023-10-06T02:24:39.862 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBLbx9lezVtJhAA8wK7+dJ1OmIc+iXJkexK0w==", 2023-10-06T02:24:39.862 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:24:39.862 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:24:39.862 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T02:24:39.862 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T02:24:39.862 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:24:39.862 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:24:39.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:24:39.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:24:39.863 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.d", 2023-10-06T02:24:39.863 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBMbx9l74gWABAApBu2nVv12rGA+OBfBnEc3A==", 2023-10-06T02:24:39.863 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:24:39.863 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:24:39.863 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T02:24:39.863 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T02:24:39.863 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:24:39.863 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:24:39.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:24:39.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:24:39.863 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.0", 2023-10-06T02:24:39.863 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBPbx9lgj6+MBAAXSUD0GCG+rO/9DQBZYt/8w==", 2023-10-06T02:24:39.864 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:24:39.864 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:24:39.864 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:24:39.864 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:24:39.864 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:24:39.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:24:39.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:24:39.864 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.1", 2023-10-06T02:24:39.864 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBSbx9lXhMTNBAAeEdPnMP71ZGb6k2Ddg5raQ==", 2023-10-06T02:24:39.865 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:24:39.865 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:24:39.865 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:24:39.865 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:24:39.865 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:24:39.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:24:39.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:24:39.865 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.2", 2023-10-06T02:24:39.865 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBVbx9l4Ly0NhAATYXYDAjYuP53zjqK0yMboA==", 2023-10-06T02:24:39.865 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:24:39.865 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:24:39.865 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:24:39.866 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:24:39.866 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:24:39.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:24:39.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:24:39.866 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.3", 2023-10-06T02:24:39.866 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBZbx9lzsi4ARAAbGnNV0QyL/7itzy827+chg==", 2023-10-06T02:24:39.866 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:24:39.866 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:24:39.866 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:24:39.866 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:24:39.866 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:24:39.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:24:39.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:24:39.867 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.4", 2023-10-06T02:24:39.867 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBfbx9lW4OSEhAAj7HnVh/yq12zzCaXd6BHCA==", 2023-10-06T02:24:39.867 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:24:39.867 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:24:39.867 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:24:39.867 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:24:39.867 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:24:39.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:24:39.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:24:39.867 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.5", 2023-10-06T02:24:39.867 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBibx9lBcaIFRAAV8wq8QjsCOkI9Pzf/E+9bA==", 2023-10-06T02:24:39.867 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:24:39.868 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:24:39.868 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:24:39.868 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:24:39.868 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:24:39.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:24:39.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:24:39.868 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.6", 2023-10-06T02:24:39.868 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBlbx9lA7S+FxAA38mlSAurT+DpwMzH53zFcw==", 2023-10-06T02:24:39.868 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:24:39.868 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:24:39.868 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:24:39.868 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:24:39.868 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:24:39.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:24:39.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:24:39.869 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.7", 2023-10-06T02:24:39.869 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBobx9l5RkdHhAAxn0Z8IAE/o9oe2LRp3O9gg==", 2023-10-06T02:24:39.869 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:24:39.869 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:24:39.869 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:24:39.869 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:24:39.869 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:24:39.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:24:39.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:24:39.869 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.0", 2023-10-06T02:24:39.870 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBMbx9lh44fCRAA7sMISvk1t7dEJg4ZcBiMcA==", 2023-10-06T02:24:39.870 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:24:39.870 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:24:39.870 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow r", 2023-10-06T02:24:39.870 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T02:24:39.870 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:24:39.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:24:39.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:24:39.870 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.1", 2023-10-06T02:24:39.870 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBMbx9lP0zqFhAAlov2y7ipSvS3AkOdQkhg6g==", 2023-10-06T02:24:39.870 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:24:39.870 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:24:39.870 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-10-06T02:24:39.871 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-10-06T02:24:39.871 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-10-06T02:24:39.871 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:24:39.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:24:39.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:24:39.871 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.2", 2023-10-06T02:24:39.871 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBMbx9lpxYAEBAACtiI1XgQXM8yGGcpFyNK1w==", 2023-10-06T02:24:39.871 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:24:39.871 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:24:39.871 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-10-06T02:24:39.871 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-10-06T02:24:39.871 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-10-06T02:24:39.872 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:24:39.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:24:39.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:24:39.872 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.3", 2023-10-06T02:24:39.872 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBMbx9l81/PHRAAYFOoaezu0pwE0NdtnlYfVA==", 2023-10-06T02:24:39.872 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:24:39.872 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:24:39.872 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-10-06T02:24:39.872 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-10-06T02:24:39.872 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-10-06T02:24:39.872 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:24:39.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:24:39.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:24:39.873 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.admin", 2023-10-06T02:24:39.873 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBKbx9liFl1JBAAFyyHYNLVx1/Ex5a2MKm8rA==", 2023-10-06T02:24:39.873 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:24:39.873 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-10-06T02:24:39.873 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T02:24:39.873 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T02:24:39.873 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:24:39.873 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:24:39.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:24:39.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:24:39.873 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mgr.x", 2023-10-06T02:24:39.873 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBLbx9lTBbIFBAAX2wsioKL4Nx4qz/qwsCsNA==", 2023-10-06T02:24:39.874 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:24:39.874 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-10-06T02:24:39.874 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile mgr", 2023-10-06T02:24:39.874 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:24:39.874 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:24:39.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:24:39.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:24:39.874 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mgr.y", 2023-10-06T02:24:39.874 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBLbx9l4+oLDRAA5YVo3q6FI9e9W/rJYROIsg==", 2023-10-06T02:24:39.874 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:24:39.874 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-10-06T02:24:39.875 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile mgr", 2023-10-06T02:24:39.875 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:24:39.875 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:24:39.875 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:24:39.875 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-10-06T02:24:39.875 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T02:24:39.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.862+0000 7f62a919d700 1 -- 172.21.15.31:0/3396347297 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f62900670e0 msgr2=0x7f6290069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:39.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.862+0000 7f62a919d700 1 --2- 172.21.15.31:0/3396347297 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f62900670e0 0x7f6290069590 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f62a4071370 tx=0x7f6298017000 comp rx=0 tx=0).stop 2023-10-06T02:24:39.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.862+0000 7f62a919d700 1 -- 172.21.15.31:0/3396347297 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62a40cef40 msgr2=0x7f62a413dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:39.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.862+0000 7f62a919d700 1 --2- 172.21.15.31:0/3396347297 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62a40cef40 0x7f62a413dec0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f629400ec80 tx=0x7f62940091a0 comp rx=0 tx=0).stop 2023-10-06T02:24:39.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.862+0000 7f62a919d700 1 -- 172.21.15.31:0/3396347297 shutdown_connections 2023-10-06T02:24:39.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.862+0000 7f62a919d700 1 --2- 172.21.15.31:0/3396347297 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f62900670e0 0x7f6290069590 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:39.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.862+0000 7f62a919d700 1 --2- 172.21.15.31:0/3396347297 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f62a415de50 0x7f62a40b3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:39.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.862+0000 7f62a919d700 1 --2- 172.21.15.31:0/3396347297 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62a40cef40 0x7f62a413dec0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:39.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.862+0000 7f62a919d700 1 --2- 172.21.15.31:0/3396347297 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62a4153c30 0x7f62a40b32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:39.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.862+0000 7f62a919d700 1 -- 172.21.15.31:0/3396347297 >> 172.21.15.31:0/3396347297 conn(0x7f62a40703a0 msgr2=0x7f62a40d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:39.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.862+0000 7f62a919d700 1 -- 172.21.15.31:0/3396347297 shutdown_connections 2023-10-06T02:24:39.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:39.862+0000 7f62a919d700 1 -- 172.21.15.31:0/3396347297 wait complete. 2023-10-06T02:24:39.879 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:24:40.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.066+0000 7f01b0f9d700 1 Processor -- start 2023-10-06T02:24:40.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.066+0000 7f01b0f9d700 1 -- start start 2023-10-06T02:24:40.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.067+0000 7f01b0f9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f01ac140e40 0x7f01ac0c6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:40.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.067+0000 7f01b0f9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f01ac06de50 0x7f01ac0aeef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:40.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.067+0000 7f01b0f9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01ac0c6d50 con 0x7f01ac140a70 2023-10-06T02:24:40.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.067+0000 7f01b0f9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01ac0c7030 con 0x7f01ac06de50 2023-10-06T02:24:40.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.067+0000 7f01b0f9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01ac0c7310 con 0x7f01ac140e40 2023-10-06T02:24:40.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.067+0000 7f01abfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f01ac06de50 0x7f01ac0aeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:40.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.067+0000 7f01abfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f01ac06de50 0x7f01ac0aeef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41032/0 (socket says 172.21.15.31:41032) 2023-10-06T02:24:40.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.067+0000 7f01abfff700 1 -- 172.21.15.31:0/133102806 learned_addr learned my addr 172.21.15.31:0/133102806 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:40.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.068+0000 7f01aaffd700 1 --2- 172.21.15.31:0/133102806 >> v2:172.21.15.62:3300/0 conn(0x7f01ac140e40 0x7f01ac0c6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:40.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.068+0000 7f01abfff700 1 -- 172.21.15.31:0/133102806 >> v2:172.21.15.62:3300/0 conn(0x7f01ac140e40 msgr2=0x7f01ac0c6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:40.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.068+0000 7f01abfff700 1 --2- 172.21.15.31:0/133102806 >> v2:172.21.15.62:3300/0 conn(0x7f01ac140e40 0x7f01ac0c6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:40.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.068+0000 7f01abfff700 1 -- 172.21.15.31:0/133102806 >> v1:172.21.15.62:6789/0 conn(0x7f01ac140a70 legacy=0x7f01ac0c7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:40.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.068+0000 7f01abfff700 1 -- 172.21.15.31:0/133102806 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01ac0c75f0 con 0x7f01ac06de50 2023-10-06T02:24:40.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.068+0000 7f01abfff700 1 --2- 172.21.15.31:0/133102806 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01ac06de50 0x7f01ac0aeef0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f01ac0cc0e0 tx=0x7f01a4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=679e36b5e9bca524 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:40.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.068+0000 7f01aa7fc700 1 -- 172.21.15.31:0/133102806 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a400e070 con 0x7f01ac06de50 2023-10-06T02:24:40.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.068+0000 7f01aa7fc700 1 -- 172.21.15.31:0/133102806 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01a400ba20 con 0x7f01ac06de50 2023-10-06T02:24:40.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.068+0000 7f01aa7fc700 1 -- 172.21.15.31:0/133102806 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a40092e0 con 0x7f01ac06de50 2023-10-06T02:24:40.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.069+0000 7f01b0f9d700 1 -- 172.21.15.31:0/133102806 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01ac06de50 msgr2=0x7f01ac0aeef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:40.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.069+0000 7f01b0f9d700 1 --2- 172.21.15.31:0/133102806 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01ac06de50 0x7f01ac0aeef0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f01ac0cc0e0 tx=0x7f01a4009840 comp rx=0 tx=0).stop 2023-10-06T02:24:40.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.069+0000 7f01b0f9d700 1 -- 172.21.15.31:0/133102806 shutdown_connections 2023-10-06T02:24:40.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.069+0000 7f01b0f9d700 1 --2- 172.21.15.31:0/133102806 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01ac06de50 0x7f01ac0aeef0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:40.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.069+0000 7f01b0f9d700 1 --2- 172.21.15.31:0/133102806 >> v2:172.21.15.62:3300/0 conn(0x7f01ac140e40 0x7f01ac0c6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:40.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.069+0000 7f01b0f9d700 1 -- 172.21.15.31:0/133102806 >> 172.21.15.31:0/133102806 conn(0x7f01ac06d170 msgr2=0x7f01ac0cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:40.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.069+0000 7f01b0f9d700 1 -- 172.21.15.31:0/133102806 shutdown_connections 2023-10-06T02:24:40.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.069+0000 7f01b0f9d700 1 -- 172.21.15.31:0/133102806 wait complete. 2023-10-06T02:24:40.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.070+0000 7f01b0f9d700 1 Processor -- start 2023-10-06T02:24:40.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.071+0000 7f01b0f9d700 1 -- start start 2023-10-06T02:24:40.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.071+0000 7f01b0f9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01ac06de50 0x7f01ac13e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:40.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.071+0000 7f01b0f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f01ac140a70 0x7f01ac16a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:40.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.071+0000 7f01b0f9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01ac140e40 0x7f01ac16d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:40.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.071+0000 7f01b0f9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01ac0c56f0 con 0x7f01ac140e40 2023-10-06T02:24:40.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.072+0000 7f01b0f9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01ac16d4f0 con 0x7f01ac06de50 2023-10-06T02:24:40.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.072+0000 7f01b0f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01ac16a9d0 con 0x7f01ac140a70 2023-10-06T02:24:40.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.072+0000 7f01abfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01ac140e40 0x7f01ac16d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:40.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.072+0000 7f01abfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01ac140e40 0x7f01ac16d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41038/0 (socket says 172.21.15.31:41038) 2023-10-06T02:24:40.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.072+0000 7f01abfff700 1 -- 172.21.15.31:0/169782361 learned_addr learned my addr 172.21.15.31:0/169782361 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:40.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.072+0000 7f01abfff700 1 -- 172.21.15.31:0/169782361 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f01ac140a70 msgr2=0x7f01ac16a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:40.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.072+0000 7f01abfff700 1 --2- 172.21.15.31:0/169782361 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f01ac140a70 0x7f01ac16a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:40.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.072+0000 7f01abfff700 1 -- 172.21.15.31:0/169782361 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01ac06de50 msgr2=0x7f01ac13e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:40.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.072+0000 7f01ab7fe700 1 --2- 172.21.15.31:0/169782361 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01ac06de50 0x7f01ac13e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:40.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.072+0000 7f01abfff700 1 --2- 172.21.15.31:0/169782361 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01ac06de50 0x7f01ac13e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:40.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.072+0000 7f01abfff700 1 -- 172.21.15.31:0/169782361 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01a4009ae0 con 0x7f01ac140e40 2023-10-06T02:24:40.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.072+0000 7f01ab7fe700 1 --2- 172.21.15.31:0/169782361 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01ac06de50 0x7f01ac13e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:24:40.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.072+0000 7f01abfff700 1 --2- 172.21.15.31:0/169782361 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01ac140e40 0x7f01ac16d980 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f01a4009880 tx=0x7f01a4018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:40.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.073+0000 7f01937fe700 1 -- 172.21.15.31:0/169782361 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a4008be0 con 0x7f01ac140e40 2023-10-06T02:24:40.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.073+0000 7f01937fe700 1 -- 172.21.15.31:0/169782361 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01a4008d40 con 0x7f01ac140e40 2023-10-06T02:24:40.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.073+0000 7f01937fe700 1 -- 172.21.15.31:0/169782361 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a40266b0 con 0x7f01ac140e40 2023-10-06T02:24:40.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.073+0000 7f01b0f9d700 1 -- 172.21.15.31:0/169782361 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01ac0c5130 con 0x7f01ac140e40 2023-10-06T02:24:40.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.073+0000 7f01b0f9d700 1 -- 172.21.15.31:0/169782361 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01ac16d180 con 0x7f01ac140e40 2023-10-06T02:24:40.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.074+0000 7f01b0f9d700 1 -- 172.21.15.31:0/169782361 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01ac16d660 con 0x7f01ac140e40 2023-10-06T02:24:40.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.075+0000 7f01937fe700 1 -- 172.21.15.31:0/169782361 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f01a4003040 con 0x7f01ac140e40 2023-10-06T02:24:40.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.076+0000 7f01937fe700 1 --2- 172.21.15.31:0/169782361 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f01980670e0 0x7f0198069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:40.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.076+0000 7f01937fe700 1 -- 172.21.15.31:0/169782361 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f01a4086790 con 0x7f01ac140e40 2023-10-06T02:24:40.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.076+0000 7f01ab7fe700 1 --2- 172.21.15.31:0/169782361 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f01980670e0 0x7f0198069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:40.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.076+0000 7f01ab7fe700 1 --2- 172.21.15.31:0/169782361 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f01980670e0 0x7f0198069590 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f019c001840 tx=0x7f019c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:40.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.080+0000 7f01937fe700 1 -- 172.21.15.31:0/169782361 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f01a4053870 con 0x7f01ac140e40 2023-10-06T02:24:40.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.283+0000 7f01b0f9d700 1 -- 172.21.15.31:0/169782361 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f01ac04c3c0 con 0x7f01ac140e40 2023-10-06T02:24:40.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.283+0000 7f01937fe700 1 -- 172.21.15.31:0/169782361 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+13602 (secure 0 0 0) 0x7f01a4056e90 con 0x7f01ac140e40 2023-10-06T02:24:40.284 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:24:40.284 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":43,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:24:39.358725+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:24:40.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.286+0000 7f01b0f9d700 1 -- 172.21.15.31:0/169782361 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f01980670e0 msgr2=0x7f0198069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:40.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.286+0000 7f01b0f9d700 1 --2- 172.21.15.31:0/169782361 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f01980670e0 0x7f0198069590 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f019c001840 tx=0x7f019c009040 comp rx=0 tx=0).stop 2023-10-06T02:24:40.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.286+0000 7f01b0f9d700 1 -- 172.21.15.31:0/169782361 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01ac140e40 msgr2=0x7f01ac16d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:40.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.286+0000 7f01b0f9d700 1 --2- 172.21.15.31:0/169782361 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01ac140e40 0x7f01ac16d980 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f01a4009880 tx=0x7f01a4018000 comp rx=0 tx=0).stop 2023-10-06T02:24:40.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.286+0000 7f01b0f9d700 1 -- 172.21.15.31:0/169782361 shutdown_connections 2023-10-06T02:24:40.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.286+0000 7f01b0f9d700 1 --2- 172.21.15.31:0/169782361 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01ac140e40 0x7f01ac16d980 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:40.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.286+0000 7f01b0f9d700 1 --2- 172.21.15.31:0/169782361 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01ac06de50 0x7f01ac13e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:40.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.286+0000 7f01b0f9d700 1 --2- 172.21.15.31:0/169782361 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f01980670e0 0x7f0198069590 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:40.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.286+0000 7f01b0f9d700 1 --2- 172.21.15.31:0/169782361 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f01ac140a70 0x7f01ac16a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:40.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.286+0000 7f01b0f9d700 1 -- 172.21.15.31:0/169782361 >> 172.21.15.31:0/169782361 conn(0x7f01ac06d170 msgr2=0x7f01ac0c0ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:40.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.286+0000 7f01b0f9d700 1 -- 172.21.15.31:0/169782361 shutdown_connections 2023-10-06T02:24:40.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.286+0000 7f01b0f9d700 1 -- 172.21.15.31:0/169782361 wait complete. 2023-10-06T02:24:40.303 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T02:22:53.183425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T02:23:00.046967+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-10-06T02:24:40.304 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T02:24:40.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.498+0000 7f1dfe99d700 1 Processor -- start 2023-10-06T02:24:40.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.498+0000 7f1dfe99d700 1 -- start start 2023-10-06T02:24:40.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.499+0000 7f1dfe99d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1df80c7be0 0x7f1df806abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:40.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.499+0000 7f1dfe99d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f1df80c7fb0 0x7f1df815a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:40.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.500+0000 7f1dfe99d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1df815aed0 con 0x7f1df813f780 2023-10-06T02:24:40.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.500+0000 7f1dfe99d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1df806bf10 con 0x7f1df80c7be0 2023-10-06T02:24:40.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.500+0000 7f1dfe99d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1df806b290 con 0x7f1df80c7fb0 2023-10-06T02:24:40.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.500+0000 7f1dfd99b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1df80c7be0 0x7f1df806abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:40.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.500+0000 7f1dfd99b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1df80c7be0 0x7f1df806abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41048/0 (socket says 172.21.15.31:41048) 2023-10-06T02:24:40.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.500+0000 7f1dfd99b700 1 -- 172.21.15.31:0/3676225251 learned_addr learned my addr 172.21.15.31:0/3676225251 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:40.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.500+0000 7f1dfe19c700 1 --1- 172.21.15.31:0/3676225251 >> v1:172.21.15.31:6789/0 conn(0x7f1df813f780 0x7f1df813fb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58216/0 (socket says 172.21.15.31:58216) 2023-10-06T02:24:40.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.500+0000 7f1dfd19a700 1 --2- 172.21.15.31:0/3676225251 >> v2:172.21.15.62:3300/0 conn(0x7f1df80c7fb0 0x7f1df815a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:40.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.500+0000 7f1dfd99b700 1 -- 172.21.15.31:0/3676225251 >> v2:172.21.15.62:3300/0 conn(0x7f1df80c7fb0 msgr2=0x7f1df815a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:40.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.500+0000 7f1dfd99b700 1 --2- 172.21.15.31:0/3676225251 >> v2:172.21.15.62:3300/0 conn(0x7f1df80c7fb0 0x7f1df815a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:40.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.500+0000 7f1dfd99b700 1 -- 172.21.15.31:0/3676225251 >> v1:172.21.15.31:6789/0 conn(0x7f1df813f780 legacy=0x7f1df813fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:40.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.500+0000 7f1dfd99b700 1 -- 172.21.15.31:0/3676225251 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1df806b570 con 0x7f1df80c7be0 2023-10-06T02:24:40.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.500+0000 7f1dfc999700 1 -- 172.21.15.31:0/3676225251 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1809838876 0 0) 0x7f1df815aed0 con 0x7f1df813f780 2023-10-06T02:24:40.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.501+0000 7f1dfd99b700 1 --2- 172.21.15.31:0/3676225251 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1df80c7be0 0x7f1df806abd0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f1df80aec50 tx=0x7f1df4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c6b67c38d27e7f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:40.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.501+0000 7f1dfc999700 1 -- 172.21.15.31:0/3676225251 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df400b860 con 0x7f1df80c7be0 2023-10-06T02:24:40.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.501+0000 7f1dfc999700 1 -- 172.21.15.31:0/3676225251 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1df400b9c0 con 0x7f1df80c7be0 2023-10-06T02:24:40.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.501+0000 7f1dfc999700 1 -- 172.21.15.31:0/3676225251 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df400bd60 con 0x7f1df80c7be0 2023-10-06T02:24:40.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.501+0000 7f1dfe99d700 1 -- 172.21.15.31:0/3676225251 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1df80c7be0 msgr2=0x7f1df806abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:40.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.501+0000 7f1dfe99d700 1 --2- 172.21.15.31:0/3676225251 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1df80c7be0 0x7f1df806abd0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f1df80aec50 tx=0x7f1df4009250 comp rx=0 tx=0).stop 2023-10-06T02:24:40.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.501+0000 7f1dfe99d700 1 -- 172.21.15.31:0/3676225251 shutdown_connections 2023-10-06T02:24:40.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.502+0000 7f1dfe99d700 1 --2- 172.21.15.31:0/3676225251 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1df80c7be0 0x7f1df806abd0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:40.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.502+0000 7f1dfe99d700 1 --2- 172.21.15.31:0/3676225251 >> v2:172.21.15.62:3300/0 conn(0x7f1df80c7fb0 0x7f1df815a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:40.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.502+0000 7f1dfe99d700 1 -- 172.21.15.31:0/3676225251 >> 172.21.15.31:0/3676225251 conn(0x7f1df806d110 msgr2=0x7f1df80cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:40.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.502+0000 7f1dfe99d700 1 -- 172.21.15.31:0/3676225251 shutdown_connections 2023-10-06T02:24:40.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.502+0000 7f1dfe99d700 1 -- 172.21.15.31:0/3676225251 wait complete. 2023-10-06T02:24:40.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.503+0000 7f1dfe99d700 1 Processor -- start 2023-10-06T02:24:40.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.503+0000 7f1dfe99d700 1 -- start start 2023-10-06T02:24:40.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.504+0000 7f1dfe99d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1df80c7be0 0x7f1df815c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:40.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.504+0000 7f1dfe99d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1df80c7fb0 0x7f1df80af8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:40.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.504+0000 7f1dfe99d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1df813f780 0x7f1df80afdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:40.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.504+0000 7f1dfe99d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df815bda0 con 0x7f1df80c7fb0 2023-10-06T02:24:40.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.504+0000 7f1dfe99d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df815ba70 con 0x7f1df80c7be0 2023-10-06T02:24:40.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.504+0000 7f1dfe99d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1df8156b20 con 0x7f1df813f780 2023-10-06T02:24:40.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.504+0000 7f1dfd19a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1df80c7fb0 0x7f1df80af8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:40.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.505+0000 7f1dfd19a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1df80c7fb0 0x7f1df80af8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41052/0 (socket says 172.21.15.31:41052) 2023-10-06T02:24:40.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.505+0000 7f1dfd19a700 1 -- 172.21.15.31:0/2000839199 learned_addr learned my addr 172.21.15.31:0/2000839199 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:40.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.505+0000 7f1dfd99b700 1 --2- 172.21.15.31:0/2000839199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1df80c7be0 0x7f1df815c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:40.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.505+0000 7f1dfe19c700 1 --2- 172.21.15.31:0/2000839199 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1df813f780 0x7f1df80afdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:40.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.505+0000 7f1dfd19a700 1 -- 172.21.15.31:0/2000839199 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1df813f780 msgr2=0x7f1df80afdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:40.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.505+0000 7f1dfd19a700 1 --2- 172.21.15.31:0/2000839199 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1df813f780 0x7f1df80afdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:40.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.505+0000 7f1dfd19a700 1 -- 172.21.15.31:0/2000839199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1df80c7be0 msgr2=0x7f1df815c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:40.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.505+0000 7f1dfd19a700 1 --2- 172.21.15.31:0/2000839199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1df80c7be0 0x7f1df815c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:40.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.505+0000 7f1dfd19a700 1 -- 172.21.15.31:0/2000839199 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1df40094f0 con 0x7f1df80c7fb0 2023-10-06T02:24:40.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.505+0000 7f1dfe19c700 1 --2- 172.21.15.31:0/2000839199 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1df813f780 0x7f1df80afdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:24:40.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.505+0000 7f1dfd19a700 1 --2- 172.21.15.31:0/2000839199 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1df80c7fb0 0x7f1df80af8b0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f1de800d890 tx=0x7f1de8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:40.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.505+0000 7f1dedffb700 1 -- 172.21.15.31:0/2000839199 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de800b600 con 0x7f1df80c7fb0 2023-10-06T02:24:40.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.505+0000 7f1dedffb700 1 -- 172.21.15.31:0/2000839199 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1de800b760 con 0x7f1df80c7fb0 2023-10-06T02:24:40.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.505+0000 7f1dedffb700 1 -- 172.21.15.31:0/2000839199 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de800bae0 con 0x7f1df80c7fb0 2023-10-06T02:24:40.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.506+0000 7f1dfe99d700 1 -- 172.21.15.31:0/2000839199 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1df8156e00 con 0x7f1df80c7fb0 2023-10-06T02:24:40.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.506+0000 7f1dfe99d700 1 -- 172.21.15.31:0/2000839199 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1df80b1d30 con 0x7f1df80c7fb0 2023-10-06T02:24:40.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.507+0000 7f1dfe99d700 1 -- 172.21.15.31:0/2000839199 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1df80b1fb0 con 0x7f1df80c7fb0 2023-10-06T02:24:40.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.508+0000 7f1dedffb700 1 -- 172.21.15.31:0/2000839199 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f1de801e460 con 0x7f1df80c7fb0 2023-10-06T02:24:40.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.508+0000 7f1dedffb700 1 --2- 172.21.15.31:0/2000839199 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f1ddc0670e0 0x7f1ddc069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:40.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.509+0000 7f1dedffb700 1 -- 172.21.15.31:0/2000839199 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f1de8084dd0 con 0x7f1df80c7fb0 2023-10-06T02:24:40.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.509+0000 7f1dfd99b700 1 --2- 172.21.15.31:0/2000839199 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f1ddc0670e0 0x7f1ddc069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:40.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.509+0000 7f1dfd99b700 1 --2- 172.21.15.31:0/2000839199 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f1ddc0670e0 0x7f1ddc069590 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1df80aec50 tx=0x7f1df4017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:40.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.513+0000 7f1dedffb700 1 -- 172.21.15.31:0/2000839199 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1de8051f10 con 0x7f1df80c7fb0 2023-10-06T02:24:40.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.713+0000 7f1dfe99d700 1 -- 172.21.15.31:0/2000839199 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1df804c3c0 con 0x7f1df80c7fb0 2023-10-06T02:24:40.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.713+0000 7f1dedffb700 1 -- 172.21.15.31:0/2000839199 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v43) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1de8055530 con 0x7f1df80c7fb0 2023-10-06T02:24:40.714 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-10-06T02:24:40.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.716+0000 7f1dfe99d700 1 -- 172.21.15.31:0/2000839199 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f1ddc0670e0 msgr2=0x7f1ddc069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:40.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.716+0000 7f1dfe99d700 1 --2- 172.21.15.31:0/2000839199 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f1ddc0670e0 0x7f1ddc069590 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1df80aec50 tx=0x7f1df4017000 comp rx=0 tx=0).stop 2023-10-06T02:24:40.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.716+0000 7f1dfe99d700 1 -- 172.21.15.31:0/2000839199 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1df80c7fb0 msgr2=0x7f1df80af8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:40.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.716+0000 7f1dfe99d700 1 --2- 172.21.15.31:0/2000839199 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1df80c7fb0 0x7f1df80af8b0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f1de800d890 tx=0x7f1de8008040 comp rx=0 tx=0).stop 2023-10-06T02:24:40.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.716+0000 7f1dfe99d700 1 -- 172.21.15.31:0/2000839199 shutdown_connections 2023-10-06T02:24:40.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.716+0000 7f1dfe99d700 1 --2- 172.21.15.31:0/2000839199 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f1ddc0670e0 0x7f1ddc069590 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:40.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.716+0000 7f1dfe99d700 1 --2- 172.21.15.31:0/2000839199 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1df813f780 0x7f1df80afdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:40.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.716+0000 7f1dfe99d700 1 --2- 172.21.15.31:0/2000839199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1df80c7be0 0x7f1df815c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:40.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.716+0000 7f1dfe99d700 1 --2- 172.21.15.31:0/2000839199 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1df80c7fb0 0x7f1df80af8b0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:40.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.716+0000 7f1dfe99d700 1 -- 172.21.15.31:0/2000839199 >> 172.21.15.31:0/2000839199 conn(0x7f1df806d110 msgr2=0x7f1df80c84b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:40.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.716+0000 7f1dfe99d700 1 -- 172.21.15.31:0/2000839199 shutdown_connections 2023-10-06T02:24:40.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.716+0000 7f1dfe99d700 1 -- 172.21.15.31:0/2000839199 wait complete. 2023-10-06T02:24:40.733 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T02:24:40.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.923+0000 7f0938b9d700 1 Processor -- start 2023-10-06T02:24:40.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.923+0000 7f0938b9d700 1 -- start start 2023-10-06T02:24:40.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.924+0000 7f0938b9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0934153c90 0x7f093415a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:40.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.924+0000 7f0938b9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f09340caec0 0x7f093415afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:40.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.924+0000 7f0938b9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f093415ada0 con 0x7f093406db20 2023-10-06T02:24:40.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.924+0000 7f0938b9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f093406ca80 con 0x7f0934153c90 2023-10-06T02:24:40.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.924+0000 7f0938b9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f093406cd60 con 0x7f09340caec0 2023-10-06T02:24:40.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.924+0000 7f09337fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0934153c90 0x7f093415a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:40.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.925+0000 7f09337fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0934153c90 0x7f093415a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41060/0 (socket says 172.21.15.31:41060) 2023-10-06T02:24:40.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.925+0000 7f09337fe700 1 -- 172.21.15.31:0/1963870 learned_addr learned my addr 172.21.15.31:0/1963870 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:40.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.925+0000 7f0932ffd700 1 --1- 172.21.15.31:0/1963870 >> v1:172.21.15.31:6789/0 conn(0x7f093406db20 0x7f09340ca7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58228/0 (socket says 172.21.15.31:58228) 2023-10-06T02:24:40.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.925+0000 7f0933fff700 1 --2- 172.21.15.31:0/1963870 >> v2:172.21.15.62:3300/0 conn(0x7f09340caec0 0x7f093415afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:40.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.925+0000 7f09337fe700 1 -- 172.21.15.31:0/1963870 >> v2:172.21.15.62:3300/0 conn(0x7f09340caec0 msgr2=0x7f093415afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:40.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.925+0000 7f09337fe700 1 --2- 172.21.15.31:0/1963870 >> v2:172.21.15.62:3300/0 conn(0x7f09340caec0 0x7f093415afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:40.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.925+0000 7f09337fe700 1 -- 172.21.15.31:0/1963870 >> v1:172.21.15.31:6789/0 conn(0x7f093406db20 legacy=0x7f09340ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:40.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.925+0000 7f09337fe700 1 -- 172.21.15.31:0/1963870 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f093406d040 con 0x7f0934153c90 2023-10-06T02:24:40.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.925+0000 7f09337fe700 1 --2- 172.21.15.31:0/1963870 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0934153c90 0x7f093415a860 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f09340cba60 tx=0x7f0924009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=181d66f4842f3a90 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:40.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.925+0000 7f09327fc700 1 -- 172.21.15.31:0/1963870 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f092400b860 con 0x7f0934153c90 2023-10-06T02:24:40.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.926+0000 7f09327fc700 1 -- 172.21.15.31:0/1963870 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f092400b9c0 con 0x7f0934153c90 2023-10-06T02:24:40.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.926+0000 7f09327fc700 1 -- 172.21.15.31:0/1963870 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f092400bd60 con 0x7f0934153c90 2023-10-06T02:24:40.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.926+0000 7f0938b9d700 1 -- 172.21.15.31:0/1963870 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0934153c90 msgr2=0x7f093415a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:40.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.926+0000 7f0938b9d700 1 --2- 172.21.15.31:0/1963870 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0934153c90 0x7f093415a860 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f09340cba60 tx=0x7f0924009250 comp rx=0 tx=0).stop 2023-10-06T02:24:40.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.926+0000 7f0938b9d700 1 -- 172.21.15.31:0/1963870 shutdown_connections 2023-10-06T02:24:40.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.926+0000 7f0938b9d700 1 --2- 172.21.15.31:0/1963870 >> v2:172.21.15.62:3300/0 conn(0x7f09340caec0 0x7f093415afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:40.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.926+0000 7f0938b9d700 1 --2- 172.21.15.31:0/1963870 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0934153c90 0x7f093415a860 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:40.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.926+0000 7f0938b9d700 1 -- 172.21.15.31:0/1963870 >> 172.21.15.31:0/1963870 conn(0x7f093406e720 msgr2=0x7f093406eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:40.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.926+0000 7f0938b9d700 1 -- 172.21.15.31:0/1963870 shutdown_connections 2023-10-06T02:24:40.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.926+0000 7f0938b9d700 1 -- 172.21.15.31:0/1963870 wait complete. 2023-10-06T02:24:40.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.928+0000 7f0938b9d700 1 Processor -- start 2023-10-06T02:24:40.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.928+0000 7f0938b9d700 1 -- start start 2023-10-06T02:24:40.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.929+0000 7f0938b9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f093406db20 0x7f093416af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:40.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.929+0000 7f0938b9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f09340caec0 0x7f09340b12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:40.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.929+0000 7f0938b9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0934153c90 0x7f09340c4a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:40.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.929+0000 7f0938b9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09340afce0 con 0x7f0934153c90 2023-10-06T02:24:40.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.929+0000 7f0938b9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f093416a7c0 con 0x7f093406db20 2023-10-06T02:24:40.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.929+0000 7f0938b9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f093416b470 con 0x7f09340caec0 2023-10-06T02:24:40.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.929+0000 7f0933fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0934153c90 0x7f09340c4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:40.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.929+0000 7f0933fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0934153c90 0x7f09340c4a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41072/0 (socket says 172.21.15.31:41072) 2023-10-06T02:24:40.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.929+0000 7f0933fff700 1 -- 172.21.15.31:0/1768777626 learned_addr learned my addr 172.21.15.31:0/1768777626 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:40.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.929+0000 7f0932ffd700 1 --2- 172.21.15.31:0/1768777626 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f09340caec0 0x7f09340b12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:40.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.929+0000 7f09337fe700 1 --2- 172.21.15.31:0/1768777626 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f093406db20 0x7f093416af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:40.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.929+0000 7f0933fff700 1 -- 172.21.15.31:0/1768777626 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f09340caec0 msgr2=0x7f09340b12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:40.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.929+0000 7f0933fff700 1 --2- 172.21.15.31:0/1768777626 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f09340caec0 0x7f09340b12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:40.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.929+0000 7f0933fff700 1 -- 172.21.15.31:0/1768777626 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f093406db20 msgr2=0x7f093416af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:40.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.929+0000 7f0933fff700 1 --2- 172.21.15.31:0/1768777626 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f093406db20 0x7f093416af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:40.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.929+0000 7f0933fff700 1 -- 172.21.15.31:0/1768777626 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09240094f0 con 0x7f0934153c90 2023-10-06T02:24:40.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.929+0000 7f09337fe700 1 --2- 172.21.15.31:0/1768777626 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f093406db20 0x7f093416af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:24:40.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.930+0000 7f0933fff700 1 --2- 172.21.15.31:0/1768777626 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0934153c90 0x7f09340c4a50 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f092c007f60 tx=0x7f092c0098e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:40.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.930+0000 7f091b7fe700 1 -- 172.21.15.31:0/1768777626 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f092c00ec40 con 0x7f0934153c90 2023-10-06T02:24:40.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.930+0000 7f091b7fe700 1 -- 172.21.15.31:0/1768777626 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f092c00eda0 con 0x7f0934153c90 2023-10-06T02:24:40.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.930+0000 7f0938b9d700 1 -- 172.21.15.31:0/1768777626 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09340c55c0 con 0x7f0934153c90 2023-10-06T02:24:40.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.930+0000 7f091b7fe700 1 -- 172.21.15.31:0/1768777626 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f092c014670 con 0x7f0934153c90 2023-10-06T02:24:40.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.930+0000 7f0938b9d700 1 -- 172.21.15.31:0/1768777626 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09340af970 con 0x7f0934153c90 2023-10-06T02:24:40.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.931+0000 7f0938b9d700 1 -- 172.21.15.31:0/1768777626 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09340c9110 con 0x7f0934153c90 2023-10-06T02:24:40.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.932+0000 7f091b7fe700 1 -- 172.21.15.31:0/1768777626 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f092c014a60 con 0x7f0934153c90 2023-10-06T02:24:40.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.933+0000 7f091b7fe700 1 --2- 172.21.15.31:0/1768777626 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f09200670e0 0x7f0920069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:40.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.933+0000 7f091b7fe700 1 -- 172.21.15.31:0/1768777626 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f092c084dd0 con 0x7f0934153c90 2023-10-06T02:24:40.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.934+0000 7f09337fe700 1 --2- 172.21.15.31:0/1768777626 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f09200670e0 0x7f0920069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:40.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.934+0000 7f09337fe700 1 --2- 172.21.15.31:0/1768777626 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f09200670e0 0x7f0920069590 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f09340cba60 tx=0x7f0924017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:40.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:40.937+0000 7f091b7fe700 1 -- 172.21.15.31:0/1768777626 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f092c051f10 con 0x7f0934153c90 2023-10-06T02:24:41.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.138+0000 7f0938b9d700 1 -- 172.21.15.31:0/1768777626 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f093404c3c0 con 0x7f0934153c90 2023-10-06T02:24:41.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.138+0000 7f091b7fe700 1 -- 172.21.15.31:0/1768777626 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v43) v1 ==== 92+0+10 (secure 0 0 0) 0x7f092c009a20 con 0x7f0934153c90 2023-10-06T02:24:41.139 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 8 2023-10-06T02:24:41.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.140+0000 7f0938b9d700 1 -- 172.21.15.31:0/1768777626 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f09200670e0 msgr2=0x7f0920069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:41.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.140+0000 7f0938b9d700 1 --2- 172.21.15.31:0/1768777626 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f09200670e0 0x7f0920069590 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f09340cba60 tx=0x7f0924017000 comp rx=0 tx=0).stop 2023-10-06T02:24:41.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.141+0000 7f0938b9d700 1 -- 172.21.15.31:0/1768777626 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0934153c90 msgr2=0x7f09340c4a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:41.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.141+0000 7f0938b9d700 1 --2- 172.21.15.31:0/1768777626 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0934153c90 0x7f09340c4a50 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f092c007f60 tx=0x7f092c0098e0 comp rx=0 tx=0).stop 2023-10-06T02:24:41.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.141+0000 7f0938b9d700 1 -- 172.21.15.31:0/1768777626 shutdown_connections 2023-10-06T02:24:41.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.141+0000 7f0938b9d700 1 --2- 172.21.15.31:0/1768777626 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0934153c90 0x7f09340c4a50 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:41.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.141+0000 7f0938b9d700 1 --2- 172.21.15.31:0/1768777626 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f093406db20 0x7f093416af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:41.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.141+0000 7f0938b9d700 1 --2- 172.21.15.31:0/1768777626 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f09200670e0 0x7f0920069590 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:41.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.141+0000 7f0938b9d700 1 --2- 172.21.15.31:0/1768777626 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f09340caec0 0x7f09340b12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:41.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.141+0000 7f0938b9d700 1 -- 172.21.15.31:0/1768777626 >> 172.21.15.31:0/1768777626 conn(0x7f093406e720 msgr2=0x7f09340c8a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:41.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.141+0000 7f0938b9d700 1 -- 172.21.15.31:0/1768777626 shutdown_connections 2023-10-06T02:24:41.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.141+0000 7f0938b9d700 1 -- 172.21.15.31:0/1768777626 wait complete. 2023-10-06T02:24:41.159 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T02:24:41.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.347+0000 7f8af0f9d700 1 Processor -- start 2023-10-06T02:24:41.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.347+0000 7f8af0f9d700 1 -- start start 2023-10-06T02:24:41.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.348+0000 7f8af0f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8aec13fb20 0x7f8aec15a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:41.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.348+0000 7f8af0f9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8aec13fef0 0x7f8aec0c7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:41.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.348+0000 7f8af0f9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8aec06aaa0 0x7f8aec0c7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:41.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.348+0000 7f8af0f9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8aec0c8410 con 0x7f8aec13fef0 2023-10-06T02:24:41.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.348+0000 7f8af0f9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8aec15ae70 con 0x7f8aec06aaa0 2023-10-06T02:24:41.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.348+0000 7f8af0f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8aec06af60 con 0x7f8aec13fb20 2023-10-06T02:24:41.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.348+0000 7f8aeaffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8aec13fef0 0x7f8aec0c7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:41.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.348+0000 7f8aeaffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8aec13fef0 0x7f8aec0c7990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41080/0 (socket says 172.21.15.31:41080) 2023-10-06T02:24:41.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.348+0000 7f8aeaffd700 1 -- 172.21.15.31:0/3961072319 learned_addr learned my addr 172.21.15.31:0/3961072319 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:41.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.349+0000 7f8aebfff700 1 --2- 172.21.15.31:0/3961072319 >> v2:172.21.15.62:3300/0 conn(0x7f8aec06aaa0 0x7f8aec0c7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:41.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.349+0000 7f8aeb7fe700 1 --2- 172.21.15.31:0/3961072319 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8aec13fb20 0x7f8aec15a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:41.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.349+0000 7f8aeaffd700 1 -- 172.21.15.31:0/3961072319 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8aec13fb20 msgr2=0x7f8aec15a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:41.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.349+0000 7f8aeaffd700 1 --2- 172.21.15.31:0/3961072319 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8aec13fb20 0x7f8aec15a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:41.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.349+0000 7f8aeaffd700 1 -- 172.21.15.31:0/3961072319 >> v2:172.21.15.62:3300/0 conn(0x7f8aec06aaa0 msgr2=0x7f8aec0c7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:41.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.349+0000 7f8aeaffd700 1 --2- 172.21.15.31:0/3961072319 >> v2:172.21.15.62:3300/0 conn(0x7f8aec06aaa0 0x7f8aec0c7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:41.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.349+0000 7f8aeaffd700 1 -- 172.21.15.31:0/3961072319 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aec06b240 con 0x7f8aec13fef0 2023-10-06T02:24:41.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.349+0000 7f8aeaffd700 1 --2- 172.21.15.31:0/3961072319 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8aec13fef0 0x7f8aec0c7990 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f8aec156920 tx=0x7f8ae0009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7923c69f52fccd1c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:41.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.349+0000 7f8aea7fc700 1 -- 172.21.15.31:0/3961072319 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae0005c10 con 0x7f8aec13fef0 2023-10-06T02:24:41.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.349+0000 7f8aea7fc700 1 -- 172.21.15.31:0/3961072319 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ae0005d70 con 0x7f8aec13fef0 2023-10-06T02:24:41.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.349+0000 7f8aea7fc700 1 -- 172.21.15.31:0/3961072319 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae000b880 con 0x7f8aec13fef0 2023-10-06T02:24:41.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.350+0000 7f8af0f9d700 1 -- 172.21.15.31:0/3961072319 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8aec13fef0 msgr2=0x7f8aec0c7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:41.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.350+0000 7f8af0f9d700 1 --2- 172.21.15.31:0/3961072319 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8aec13fef0 0x7f8aec0c7990 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f8aec156920 tx=0x7f8ae0009220 comp rx=0 tx=0).stop 2023-10-06T02:24:41.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.350+0000 7f8af0f9d700 1 -- 172.21.15.31:0/3961072319 shutdown_connections 2023-10-06T02:24:41.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.350+0000 7f8af0f9d700 1 --2- 172.21.15.31:0/3961072319 >> v2:172.21.15.62:3300/0 conn(0x7f8aec06aaa0 0x7f8aec0c7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:41.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.350+0000 7f8af0f9d700 1 --2- 172.21.15.31:0/3961072319 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8aec13fb20 0x7f8aec15a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:41.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.350+0000 7f8af0f9d700 1 --2- 172.21.15.31:0/3961072319 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8aec13fef0 0x7f8aec0c7990 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:41.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.350+0000 7f8af0f9d700 1 -- 172.21.15.31:0/3961072319 >> 172.21.15.31:0/3961072319 conn(0x7f8aec06d170 msgr2=0x7f8aec0cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:41.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.350+0000 7f8af0f9d700 1 -- 172.21.15.31:0/3961072319 shutdown_connections 2023-10-06T02:24:41.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.350+0000 7f8af0f9d700 1 -- 172.21.15.31:0/3961072319 wait complete. 2023-10-06T02:24:41.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.351+0000 7f8af0f9d700 1 Processor -- start 2023-10-06T02:24:41.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.352+0000 7f8af0f9d700 1 -- start start 2023-10-06T02:24:41.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.352+0000 7f8af0f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8aec06aaa0 0x7f8aec1595c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:41.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.352+0000 7f8af0f9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8aec13fb20 0x7f8aec169f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:41.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.353+0000 7f8af0f9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8aec13fef0 0x7f8aec13b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:41.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.353+0000 7f8af0f9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aec16aee0 con 0x7f8aec13fb20 2023-10-06T02:24:41.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.353+0000 7f8af0f9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aec13b880 con 0x7f8aec13fef0 2023-10-06T02:24:41.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.353+0000 7f8af0f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8aec16a640 con 0x7f8aec06aaa0 2023-10-06T02:24:41.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.353+0000 7f8aeaffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8aec13fb20 0x7f8aec169f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:41.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.353+0000 7f8aeaffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8aec13fb20 0x7f8aec169f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41090/0 (socket says 172.21.15.31:41090) 2023-10-06T02:24:41.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.353+0000 7f8aeaffd700 1 -- 172.21.15.31:0/600799825 learned_addr learned my addr 172.21.15.31:0/600799825 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:41.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.353+0000 7f8aeb7fe700 1 --2- 172.21.15.31:0/600799825 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8aec06aaa0 0x7f8aec1595c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:41.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.353+0000 7f8aebfff700 1 --2- 172.21.15.31:0/600799825 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8aec13fef0 0x7f8aec13b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:41.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.353+0000 7f8aeaffd700 1 -- 172.21.15.31:0/600799825 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8aec06aaa0 msgr2=0x7f8aec1595c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:41.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.353+0000 7f8aeaffd700 1 --2- 172.21.15.31:0/600799825 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8aec06aaa0 0x7f8aec1595c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:41.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.353+0000 7f8aeaffd700 1 -- 172.21.15.31:0/600799825 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8aec13fef0 msgr2=0x7f8aec13b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:41.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.353+0000 7f8aeaffd700 1 --2- 172.21.15.31:0/600799825 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8aec13fef0 0x7f8aec13b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:41.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.353+0000 7f8aeaffd700 1 -- 172.21.15.31:0/600799825 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ae0009480 con 0x7f8aec13fb20 2023-10-06T02:24:41.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.353+0000 7f8aeb7fe700 1 --2- 172.21.15.31:0/600799825 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8aec06aaa0 0x7f8aec1595c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:24:41.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.353+0000 7f8aeaffd700 1 --2- 172.21.15.31:0/600799825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8aec13fb20 0x7f8aec169f40 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f8aec071320 tx=0x7f8ae0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:41.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.354+0000 7f8ad37fe700 1 -- 172.21.15.31:0/600799825 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae0011b30 con 0x7f8aec13fb20 2023-10-06T02:24:41.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.354+0000 7f8ad37fe700 1 -- 172.21.15.31:0/600799825 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ae0011c90 con 0x7f8aec13fb20 2023-10-06T02:24:41.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.354+0000 7f8ad37fe700 1 -- 172.21.15.31:0/600799825 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae001b680 con 0x7f8aec13fb20 2023-10-06T02:24:41.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.354+0000 7f8af0f9d700 1 -- 172.21.15.31:0/600799825 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8aec16a920 con 0x7f8aec13fb20 2023-10-06T02:24:41.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.354+0000 7f8af0f9d700 1 -- 172.21.15.31:0/600799825 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8aec0c1990 con 0x7f8aec13fb20 2023-10-06T02:24:41.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.355+0000 7f8af0f9d700 1 -- 172.21.15.31:0/600799825 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8aec0c1bb0 con 0x7f8aec13fb20 2023-10-06T02:24:41.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.356+0000 7f8ad37fe700 1 -- 172.21.15.31:0/600799825 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f8ae001bab0 con 0x7f8aec13fb20 2023-10-06T02:24:41.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.357+0000 7f8ad37fe700 1 --2- 172.21.15.31:0/600799825 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f8ad80670e0 0x7f8ad8069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:41.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.357+0000 7f8ad37fe700 1 -- 172.21.15.31:0/600799825 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f8ae0086190 con 0x7f8aec13fb20 2023-10-06T02:24:41.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.357+0000 7f8aeb7fe700 1 --2- 172.21.15.31:0/600799825 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f8ad80670e0 0x7f8ad8069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:41.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.358+0000 7f8aeb7fe700 1 --2- 172.21.15.31:0/600799825 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f8ad80670e0 0x7f8ad8069590 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8adc006fd0 tx=0x7f8adc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:41.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.361+0000 7f8ad37fe700 1 -- 172.21.15.31:0/600799825 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8ae00532e0 con 0x7f8aec13fb20 2023-10-06T02:24:41.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.567+0000 7f8af0f9d700 1 -- 172.21.15.31:0/600799825 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f8aec04c3c0 con 0x7f8aec13fb20 2023-10-06T02:24:41.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.568+0000 7f8ad37fe700 1 -- 172.21.15.31:0/600799825 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v43) v1 ==== 84+0+126 (secure 0 0 0) 0x7f8ae002a090 con 0x7f8aec13fb20 2023-10-06T02:24:41.568 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:24:41.568 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2023-10-06T02:24:41.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:24:41.568 INFO:teuthology.orchestra.run.smithi031.stdout: "poolnum": 1, 2023-10-06T02:24:41.569 INFO:teuthology.orchestra.run.smithi031.stdout: "poolname": ".mgr" 2023-10-06T02:24:41.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:24:41.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:24:41.569 INFO:teuthology.orchestra.run.smithi031.stdout: "poolnum": 2, 2023-10-06T02:24:41.569 INFO:teuthology.orchestra.run.smithi031.stdout: "poolname": "rbd" 2023-10-06T02:24:41.569 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:24:41.569 INFO:teuthology.orchestra.run.smithi031.stdout:] 2023-10-06T02:24:41.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.570+0000 7f8af0f9d700 1 -- 172.21.15.31:0/600799825 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f8ad80670e0 msgr2=0x7f8ad8069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:41.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.570+0000 7f8af0f9d700 1 --2- 172.21.15.31:0/600799825 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f8ad80670e0 0x7f8ad8069590 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8adc006fd0 tx=0x7f8adc008040 comp rx=0 tx=0).stop 2023-10-06T02:24:41.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.570+0000 7f8af0f9d700 1 -- 172.21.15.31:0/600799825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8aec13fb20 msgr2=0x7f8aec169f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:41.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.570+0000 7f8af0f9d700 1 --2- 172.21.15.31:0/600799825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8aec13fb20 0x7f8aec169f40 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f8aec071320 tx=0x7f8ae0016000 comp rx=0 tx=0).stop 2023-10-06T02:24:41.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.570+0000 7f8af0f9d700 1 -- 172.21.15.31:0/600799825 shutdown_connections 2023-10-06T02:24:41.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.570+0000 7f8af0f9d700 1 --2- 172.21.15.31:0/600799825 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8aec13fef0 0x7f8aec13b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:41.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.570+0000 7f8af0f9d700 1 --2- 172.21.15.31:0/600799825 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f8ad80670e0 0x7f8ad8069590 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:41.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.570+0000 7f8af0f9d700 1 --2- 172.21.15.31:0/600799825 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8aec06aaa0 0x7f8aec1595c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:41.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.570+0000 7f8af0f9d700 1 --2- 172.21.15.31:0/600799825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8aec13fb20 0x7f8aec169f40 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:41.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.570+0000 7f8af0f9d700 1 -- 172.21.15.31:0/600799825 >> 172.21.15.31:0/600799825 conn(0x7f8aec06d170 msgr2=0x7f8aec0b1260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:41.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.570+0000 7f8af0f9d700 1 -- 172.21.15.31:0/600799825 shutdown_connections 2023-10-06T02:24:41.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.571+0000 7f8af0f9d700 1 -- 172.21.15.31:0/600799825 wait complete. 2023-10-06T02:24:41.588 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T02:24:41.588 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T02:24:41.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.778+0000 7fc3f3fff700 1 Processor -- start 2023-10-06T02:24:41.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.779+0000 7fc3f3fff700 1 -- start start 2023-10-06T02:24:41.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.780+0000 7fc3f3fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc3f4153a30 0x7fc3f40c0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:41.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.780+0000 7fc3f3fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc3f40c9090 0x7fc3f4156d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:41.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.780+0000 7fc3f3fff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3f40c2200 con 0x7fc3f4153660 2023-10-06T02:24:41.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.780+0000 7fc3f3fff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3f4157250 con 0x7fc3f40c9090 2023-10-06T02:24:41.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.780+0000 7fc3f3fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f40c15f0 con 0x7fc3f4153a30 2023-10-06T02:24:41.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.780+0000 7fc3f37fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc3f40c9090 0x7fc3f4156d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:41.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.780+0000 7fc3f37fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc3f40c9090 0x7fc3f4156d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41092/0 (socket says 172.21.15.31:41092) 2023-10-06T02:24:41.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.780+0000 7fc3f37fe700 1 -- 172.21.15.31:0/995248664 learned_addr learned my addr 172.21.15.31:0/995248664 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:41.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.780+0000 7fc3f27fc700 1 --2- 172.21.15.31:0/995248664 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc3f4153a30 0x7fc3f40c0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:41.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.781+0000 7fc3f37fe700 1 -- 172.21.15.31:0/995248664 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc3f4153a30 msgr2=0x7fc3f40c0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:41.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.781+0000 7fc3f37fe700 1 --2- 172.21.15.31:0/995248664 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc3f4153a30 0x7fc3f40c0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:41.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.781+0000 7fc3f37fe700 1 -- 172.21.15.31:0/995248664 >> v1:172.21.15.62:6789/0 conn(0x7fc3f4153660 legacy=0x7fc3f40c8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:41.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.781+0000 7fc3f37fe700 1 -- 172.21.15.31:0/995248664 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3f40c18d0 con 0x7fc3f40c9090 2023-10-06T02:24:41.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.781+0000 7fc3f37fe700 1 --2- 172.21.15.31:0/995248664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3f40c9090 0x7fc3f4156d10 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec009f20 tx=0x7fc3ec009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d7f449f86e9d14a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:41.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.781+0000 7fc3f1ffb700 1 -- 172.21.15.31:0/995248664 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec00e070 con 0x7fc3f40c9090 2023-10-06T02:24:41.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.781+0000 7fc3f1ffb700 1 -- 172.21.15.31:0/995248664 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3ec00b9d0 con 0x7fc3f40c9090 2023-10-06T02:24:41.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.781+0000 7fc3f1ffb700 1 -- 172.21.15.31:0/995248664 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec0092e0 con 0x7fc3f40c9090 2023-10-06T02:24:41.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.782+0000 7fc3f3fff700 1 -- 172.21.15.31:0/995248664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3f40c9090 msgr2=0x7fc3f4156d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:41.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.782+0000 7fc3f3fff700 1 --2- 172.21.15.31:0/995248664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3f40c9090 0x7fc3f4156d10 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec009f20 tx=0x7fc3ec009840 comp rx=0 tx=0).stop 2023-10-06T02:24:41.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.782+0000 7fc3f3fff700 1 -- 172.21.15.31:0/995248664 shutdown_connections 2023-10-06T02:24:41.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.782+0000 7fc3f3fff700 1 --2- 172.21.15.31:0/995248664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3f40c9090 0x7fc3f4156d10 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:41.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.782+0000 7fc3f3fff700 1 --2- 172.21.15.31:0/995248664 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc3f4153a30 0x7fc3f40c0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:41.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.782+0000 7fc3f3fff700 1 -- 172.21.15.31:0/995248664 >> 172.21.15.31:0/995248664 conn(0x7fc3f406c6a0 msgr2=0x7fc3f406caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:41.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.782+0000 7fc3f3fff700 1 -- 172.21.15.31:0/995248664 shutdown_connections 2023-10-06T02:24:41.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.782+0000 7fc3f3fff700 1 -- 172.21.15.31:0/995248664 wait complete. 2023-10-06T02:24:41.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.783+0000 7fc3f3fff700 1 Processor -- start 2023-10-06T02:24:41.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.783+0000 7fc3f3fff700 1 -- start start 2023-10-06T02:24:41.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.784+0000 7fc3f3fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3f40c9090 0x7fc3f41699d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:41.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.784+0000 7fc3f3fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc3f4153660 0x7fc3f41556e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:41.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.784+0000 7fc3f3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3f4153a30 0x7fc3f41508d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:41.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.784+0000 7fc3f3fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f4155250 con 0x7fc3f4153a30 2023-10-06T02:24:41.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.784+0000 7fc3f3fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f416a290 con 0x7fc3f40c9090 2023-10-06T02:24:41.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.784+0000 7fc3f3fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f416a570 con 0x7fc3f4153660 2023-10-06T02:24:41.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.785+0000 7fc3f37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3f4153a30 0x7fc3f41508d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:41.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.785+0000 7fc3f37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3f4153a30 0x7fc3f41508d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41100/0 (socket says 172.21.15.31:41100) 2023-10-06T02:24:41.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.785+0000 7fc3f37fe700 1 -- 172.21.15.31:0/3442474661 learned_addr learned my addr 172.21.15.31:0/3442474661 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:41.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.785+0000 7fc3f27fc700 1 --2- 172.21.15.31:0/3442474661 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc3f4153660 0x7fc3f41556e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:41.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.785+0000 7fc3f2ffd700 1 --2- 172.21.15.31:0/3442474661 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3f40c9090 0x7fc3f41699d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:41.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.785+0000 7fc3f37fe700 1 -- 172.21.15.31:0/3442474661 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc3f4153660 msgr2=0x7fc3f41556e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:41.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.785+0000 7fc3f37fe700 1 --2- 172.21.15.31:0/3442474661 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc3f4153660 0x7fc3f41556e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:41.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.785+0000 7fc3f37fe700 1 -- 172.21.15.31:0/3442474661 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3f40c9090 msgr2=0x7fc3f41699d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:41.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.785+0000 7fc3f37fe700 1 --2- 172.21.15.31:0/3442474661 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3f40c9090 0x7fc3f41699d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:41.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.785+0000 7fc3f37fe700 1 -- 172.21.15.31:0/3442474661 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3ec009ae0 con 0x7fc3f4153a30 2023-10-06T02:24:41.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.785+0000 7fc3f2ffd700 1 --2- 172.21.15.31:0/3442474661 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3f40c9090 0x7fc3f41699d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:24:41.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.785+0000 7fc3f37fe700 1 --2- 172.21.15.31:0/3442474661 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3f4153a30 0x7fc3f41508d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec00b610 tx=0x7fc3ec018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:41.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.786+0000 7fc3daffd700 1 -- 172.21.15.31:0/3442474661 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec008b80 con 0x7fc3f4153a30 2023-10-06T02:24:41.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.786+0000 7fc3daffd700 1 -- 172.21.15.31:0/3442474661 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3ec008ce0 con 0x7fc3f4153a30 2023-10-06T02:24:41.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.786+0000 7fc3f3fff700 1 -- 172.21.15.31:0/3442474661 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3f416a850 con 0x7fc3f4153a30 2023-10-06T02:24:41.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.786+0000 7fc3daffd700 1 -- 172.21.15.31:0/3442474661 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec01c640 con 0x7fc3f4153a30 2023-10-06T02:24:41.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.786+0000 7fc3f3fff700 1 -- 172.21.15.31:0/3442474661 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3f4169f10 con 0x7fc3f4153a30 2023-10-06T02:24:41.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.787+0000 7fc3f3fff700 1 -- 172.21.15.31:0/3442474661 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3f4155400 con 0x7fc3f4153a30 2023-10-06T02:24:41.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.788+0000 7fc3daffd700 1 -- 172.21.15.31:0/3442474661 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7fc3ec003040 con 0x7fc3f4153a30 2023-10-06T02:24:41.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.789+0000 7fc3daffd700 1 --2- 172.21.15.31:0/3442474661 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc3e00670e0 0x7fc3e0069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:41.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.789+0000 7fc3daffd700 1 -- 172.21.15.31:0/3442474661 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fc3ec085f20 con 0x7fc3f4153a30 2023-10-06T02:24:41.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.789+0000 7fc3f2ffd700 1 --2- 172.21.15.31:0/3442474661 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc3e00670e0 0x7fc3e0069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:41.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.789+0000 7fc3f2ffd700 1 --2- 172.21.15.31:0/3442474661 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc3e00670e0 0x7fc3e0069590 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc3e4007fd0 tx=0x7fc3e4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:41.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.793+0000 7fc3daffd700 1 -- 172.21.15.31:0/3442474661 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc3f4155400 con 0x7fc3f4153a30 2023-10-06T02:24:41.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:41.995+0000 7fc3f3fff700 1 -- 172.21.15.31:0/3442474661 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fc3f404c3c0 con 0x7fc3f4153a30 2023-10-06T02:24:42.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:42.370+0000 7fc3daffd700 1 -- 172.21.15.31:0/3442474661 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v44) v1 ==== 138+0+0 (secure 0 0 0) 0x7fc3ec052ff0 con 0x7fc3f4153a30 2023-10-06T02:24:42.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:42.374+0000 7fc3f3fff700 1 -- 172.21.15.31:0/3442474661 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc3e00670e0 msgr2=0x7fc3e0069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:42.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:42.374+0000 7fc3f3fff700 1 --2- 172.21.15.31:0/3442474661 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc3e00670e0 0x7fc3e0069590 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc3e4007fd0 tx=0x7fc3e4009040 comp rx=0 tx=0).stop 2023-10-06T02:24:42.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:42.374+0000 7fc3f3fff700 1 -- 172.21.15.31:0/3442474661 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3f4153a30 msgr2=0x7fc3f41508d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:42.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:42.374+0000 7fc3f3fff700 1 --2- 172.21.15.31:0/3442474661 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3f4153a30 0x7fc3f41508d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec00b610 tx=0x7fc3ec018000 comp rx=0 tx=0).stop 2023-10-06T02:24:42.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:42.374+0000 7fc3f3fff700 1 -- 172.21.15.31:0/3442474661 shutdown_connections 2023-10-06T02:24:42.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:42.374+0000 7fc3f3fff700 1 --2- 172.21.15.31:0/3442474661 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3f4153a30 0x7fc3f41508d0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:42.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:42.374+0000 7fc3f3fff700 1 --2- 172.21.15.31:0/3442474661 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3f40c9090 0x7fc3f41699d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:42.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:42.374+0000 7fc3f3fff700 1 --2- 172.21.15.31:0/3442474661 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc3e00670e0 0x7fc3e0069590 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:42.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:42.374+0000 7fc3f3fff700 1 --2- 172.21.15.31:0/3442474661 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc3f4153660 0x7fc3f41556e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:42.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:42.374+0000 7fc3f3fff700 1 -- 172.21.15.31:0/3442474661 >> 172.21.15.31:0/3442474661 conn(0x7fc3f406c6a0 msgr2=0x7fc3f40b18a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:42.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:42.374+0000 7fc3f3fff700 1 -- 172.21.15.31:0/3442474661 shutdown_connections 2023-10-06T02:24:42.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:42.374+0000 7fc3f3fff700 1 -- 172.21.15.31:0/3442474661 wait complete. 2023-10-06T02:24:42.377 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' created 2023-10-06T02:24:42.397 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T02:24:42.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:42.592+0000 7f8f6b59d700 1 Processor -- start 2023-10-06T02:24:42.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:42.592+0000 7f8f6b59d700 1 -- start start 2023-10-06T02:24:42.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:42.593+0000 7f8f6b59d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8f64142c00 0x7f8f6406b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:42.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:42.593+0000 7f8f6b59d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8f6406bd20 0x7f8f640c7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:42.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:42.593+0000 7f8f6b59d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f640ca480 con 0x7f8f64142fd0 2023-10-06T02:24:42.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:42.593+0000 7f8f6b59d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f6415e240 con 0x7f8f6406bd20 2023-10-06T02:24:42.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:42.593+0000 7f8f6b59d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6406b350 con 0x7f8f64142c00 2023-10-06T02:24:42.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:42.593+0000 7f8f6ad9c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f8f64142fd0 0x7f8f6406c6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58236/0 (socket says 172.21.15.31:58236) 2023-10-06T02:24:42.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:42.593+0000 7f8f6ad9c700 1 -- 172.21.15.31:0/1445899455 learned_addr learned my addr 172.21.15.31:0/1445899455 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:24:42.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:42.594+0000 7f8f6a59b700 1 --2- 172.21.15.31:0/1445899455 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8f64142c00 0x7f8f6406b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:42.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:42.594+0000 7f8f69d9a700 1 --2- 172.21.15.31:0/1445899455 >> v2:172.21.15.62:3300/0 conn(0x7f8f6406bd20 0x7f8f640c7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:42.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:42.594+0000 7f8f69599700 1 -- 172.21.15.31:0/1445899455 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 412626167 0 0) 0x7f8f640ca480 con 0x7f8f64142fd0 2023-10-06T02:24:42.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:42.594+0000 7f8f69599700 1 -- 172.21.15.31:0/1445899455 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f50003570 con 0x7f8f64142fd0 2023-10-06T02:24:42.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:42.594+0000 7f8f6a59b700 1 -- 172.21.15.31:0/1445899455 >> v2:172.21.15.62:3300/0 conn(0x7f8f6406bd20 msgr2=0x7f8f640c7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:42.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:42.594+0000 7f8f6a59b700 1 --2- 172.21.15.31:0/1445899455 >> v2:172.21.15.62:3300/0 conn(0x7f8f6406bd20 0x7f8f640c7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:42.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:42.594+0000 7f8f6a59b700 1 -- 172.21.15.31:0/1445899455 >> v1:172.21.15.31:6789/0 conn(0x7f8f64142fd0 legacy=0x7f8f6406c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:42.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:42.594+0000 7f8f69599700 1 -- 172.21.15.31:0/1445899455 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1507064338 0 0) 0x7f8f50003570 con 0x7f8f64142fd0 2023-10-06T02:24:42.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:42.594+0000 7f8f6a59b700 1 -- 172.21.15.31:0/1445899455 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f6415df60 con 0x7f8f64142c00 2023-10-06T02:24:42.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:42.595+0000 7f8f6a59b700 1 --2- 172.21.15.31:0/1445899455 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8f64142c00 0x7f8f6406b7e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8f600098e0 tx=0x7f8f600091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f107439c63b66a1c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:42.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:42.595+0000 7f8f69599700 1 -- 172.21.15.31:0/1445899455 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f60011630 con 0x7f8f64142c00 2023-10-06T02:24:42.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:42.595+0000 7f8f69599700 1 -- 172.21.15.31:0/1445899455 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f60011790 con 0x7f8f64142c00 2023-10-06T02:24:42.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:42.595+0000 7f8f69599700 1 -- 172.21.15.31:0/1445899455 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f60011b30 con 0x7f8f64142c00 2023-10-06T02:24:42.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:42.596+0000 7f8f6b59d700 1 -- 172.21.15.31:0/1445899455 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8f64142c00 msgr2=0x7f8f6406b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:42.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:42.596+0000 7f8f6b59d700 1 --2- 172.21.15.31:0/1445899455 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8f64142c00 0x7f8f6406b7e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8f600098e0 tx=0x7f8f600091a0 comp rx=0 tx=0).stop 2023-10-06T02:24:42.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:42.596+0000 7f8f6b59d700 1 -- 172.21.15.31:0/1445899455 shutdown_connections 2023-10-06T02:24:42.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:42.596+0000 7f8f6b59d700 1 --2- 172.21.15.31:0/1445899455 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8f64142c00 0x7f8f6406b7e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:42.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:42.596+0000 7f8f6b59d700 1 --2- 172.21.15.31:0/1445899455 >> v2:172.21.15.62:3300/0 conn(0x7f8f6406bd20 0x7f8f640c7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:42.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:42.596+0000 7f8f6b59d700 1 -- 172.21.15.31:0/1445899455 >> 172.21.15.31:0/1445899455 conn(0x7f8f640675a0 msgr2=0x7f8f640679a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:42.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:42.596+0000 7f8f6b59d700 1 -- 172.21.15.31:0/1445899455 shutdown_connections 2023-10-06T02:24:42.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:42.596+0000 7f8f6b59d700 1 -- 172.21.15.31:0/1445899455 wait complete. 2023-10-06T02:24:42.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:42.597+0000 7f8f6b59d700 1 Processor -- start 2023-10-06T02:24:42.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:42.597+0000 7f8f6b59d700 1 -- start start 2023-10-06T02:24:42.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:42.598+0000 7f8f6b59d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f6406bd20 0x7f8f640affe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:42.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:42.598+0000 7f8f6b59d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8f64142c00 0x7f8f6413d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:42.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:42.598+0000 7f8f6b59d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f64142fd0 0x7f8f64158d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:42.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:42.598+0000 7f8f6b59d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6413d660 con 0x7f8f64142fd0 2023-10-06T02:24:42.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:42.598+0000 7f8f6b59d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f640b0670 con 0x7f8f6406bd20 2023-10-06T02:24:42.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:42.598+0000 7f8f6b59d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f640b0950 con 0x7f8f64142c00 2023-10-06T02:24:42.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:42.599+0000 7f8f69d9a700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8f64142c00 0x7f8f6413d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:42.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:42.599+0000 7f8f6a59b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f6406bd20 0x7f8f640affe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:42.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:42.599+0000 7f8f6ad9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f64142fd0 0x7f8f64158d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:42.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:42.599+0000 7f8f69d9a700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8f64142c00 0x7f8f6413d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:56636/0 (socket says 172.21.15.31:56636) 2023-10-06T02:24:42.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:42.599+0000 7f8f69d9a700 1 -- 172.21.15.31:0/1814046 learned_addr learned my addr 172.21.15.31:0/1814046 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:42.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:42.599+0000 7f8f6ad9c700 1 -- 172.21.15.31:0/1814046 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8f64142c00 msgr2=0x7f8f6413d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:42.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:42.599+0000 7f8f6ad9c700 1 --2- 172.21.15.31:0/1814046 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8f64142c00 0x7f8f6413d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:42.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:42.599+0000 7f8f6ad9c700 1 -- 172.21.15.31:0/1814046 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f6406bd20 msgr2=0x7f8f640affe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:42.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:42.599+0000 7f8f6ad9c700 1 --2- 172.21.15.31:0/1814046 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f6406bd20 0x7f8f640affe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:42.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:42.599+0000 7f8f6ad9c700 1 -- 172.21.15.31:0/1814046 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f60009420 con 0x7f8f64142fd0 2023-10-06T02:24:42.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:42.599+0000 7f8f6a59b700 1 --2- 172.21.15.31:0/1814046 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f6406bd20 0x7f8f640affe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:24:42.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:42.599+0000 7f8f6ad9c700 1 --2- 172.21.15.31:0/1814046 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f64142fd0 0x7f8f64158d20 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c00c860 tx=0x7f8f5c001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:42.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:42.600+0000 7f8f5a7fc700 1 -- 172.21.15.31:0/1814046 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f5c004dc0 con 0x7f8f64142fd0 2023-10-06T02:24:42.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:42.600+0000 7f8f5a7fc700 1 -- 172.21.15.31:0/1814046 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f5c007150 con 0x7f8f64142fd0 2023-10-06T02:24:42.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:42.600+0000 7f8f5a7fc700 1 -- 172.21.15.31:0/1814046 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f5c007510 con 0x7f8f64142fd0 2023-10-06T02:24:42.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:42.600+0000 7f8f6b59d700 1 -- 172.21.15.31:0/1814046 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f640b0c30 con 0x7f8f64142fd0 2023-10-06T02:24:42.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:42.600+0000 7f8f6b59d700 1 -- 172.21.15.31:0/1814046 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f640b11f0 con 0x7f8f64142fd0 2023-10-06T02:24:42.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:42.601+0000 7f8f6b59d700 1 -- 172.21.15.31:0/1814046 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f640b1410 con 0x7f8f64142fd0 2023-10-06T02:24:42.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:42.602+0000 7f8f5a7fc700 1 -- 172.21.15.31:0/1814046 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f8f5c01b360 con 0x7f8f64142fd0 2023-10-06T02:24:42.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:42.603+0000 7f8f5a7fc700 1 --2- 172.21.15.31:0/1814046 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f8f48067110 0x7f8f480695c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:42.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:42.603+0000 7f8f5a7fc700 1 -- 172.21.15.31:0/1814046 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5576+0+0 (secure 0 0 0) 0x7f8f5c083ce0 con 0x7f8f64142fd0 2023-10-06T02:24:42.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:42.603+0000 7f8f6a59b700 1 --2- 172.21.15.31:0/1814046 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f8f48067110 0x7f8f480695c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:42.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:42.604+0000 7f8f6a59b700 1 --2- 172.21.15.31:0/1814046 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f8f48067110 0x7f8f480695c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8f60011fd0 tx=0x7f8f60017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:42.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:42.606+0000 7f8f5a7fc700 1 -- 172.21.15.31:0/1814046 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8f5c050db0 con 0x7f8f64142fd0 2023-10-06T02:24:42.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:42.810+0000 7f8f6b59d700 1 -- 172.21.15.31:0/1814046 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f8f6406ad00 con 0x7f8f64142fd0 2023-10-06T02:24:43.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:43.377+0000 7f8f5a7fc700 1 -- 172.21.15.31:0/1814046 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v45) v1 ==== 170+0+0 (secure 0 0 0) 0x7f8f5c018020 con 0x7f8f64142fd0 2023-10-06T02:24:43.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:43.382+0000 7f8f6b59d700 1 -- 172.21.15.31:0/1814046 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f8f48067110 msgr2=0x7f8f480695c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:43.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:43.382+0000 7f8f6b59d700 1 --2- 172.21.15.31:0/1814046 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f8f48067110 0x7f8f480695c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8f60011fd0 tx=0x7f8f60017000 comp rx=0 tx=0).stop 2023-10-06T02:24:43.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:43.382+0000 7f8f6b59d700 1 -- 172.21.15.31:0/1814046 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f64142fd0 msgr2=0x7f8f64158d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:43.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:43.382+0000 7f8f6b59d700 1 --2- 172.21.15.31:0/1814046 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f64142fd0 0x7f8f64158d20 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c00c860 tx=0x7f8f5c001640 comp rx=0 tx=0).stop 2023-10-06T02:24:43.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:43.386+0000 7f8f6b59d700 1 -- 172.21.15.31:0/1814046 shutdown_connections 2023-10-06T02:24:43.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:43.386+0000 7f8f6b59d700 1 --2- 172.21.15.31:0/1814046 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f64142fd0 0x7f8f64158d20 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:43.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:43.386+0000 7f8f6b59d700 1 --2- 172.21.15.31:0/1814046 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f6406bd20 0x7f8f640affe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:43.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:43.386+0000 7f8f6b59d700 1 --2- 172.21.15.31:0/1814046 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f8f48067110 0x7f8f480695c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:43.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:43.386+0000 7f8f6b59d700 1 --2- 172.21.15.31:0/1814046 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8f64142c00 0x7f8f6413d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:43.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:43.386+0000 7f8f6b59d700 1 -- 172.21.15.31:0/1814046 >> 172.21.15.31:0/1814046 conn(0x7f8f640675a0 msgr2=0x7f8f640cd730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:43.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:43.389+0000 7f8f6b59d700 1 -- 172.21.15.31:0/1814046 shutdown_connections 2023-10-06T02:24:43.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:43.389+0000 7f8f6b59d700 1 -- 172.21.15.31:0/1814046 wait complete. 2023-10-06T02:24:43.396 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' created 2023-10-06T02:24:43.418 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T02:24:43.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:43.609+0000 7f79df39d700 1 Processor -- start 2023-10-06T02:24:43.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:43.609+0000 7f79df39d700 1 -- start start 2023-10-06T02:24:43.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:43.610+0000 7f79df39d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f79d8153a00 0x7f79d8153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:43.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:43.610+0000 7f79df39d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f79d80cea00 0x7f79d80cee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:43.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:43.610+0000 7f79df39d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79d815ac50 con 0x7f79d815de50 2023-10-06T02:24:43.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:43.610+0000 7f79df39d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79d815d010 con 0x7f79d80cea00 2023-10-06T02:24:43.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:43.610+0000 7f79df39d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79d815af60 con 0x7f79d8153a00 2023-10-06T02:24:43.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:43.611+0000 7f79deb9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f79d80cea00 0x7f79d80cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:43.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:43.611+0000 7f79deb9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f79d80cea00 0x7f79d80cee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41118/0 (socket says 172.21.15.31:41118) 2023-10-06T02:24:43.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:43.611+0000 7f79deb9c700 1 -- 172.21.15.31:0/2640950896 learned_addr learned my addr 172.21.15.31:0/2640950896 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:43.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:43.611+0000 7f79de39b700 1 --2- 172.21.15.31:0/2640950896 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f79d8153a00 0x7f79d8153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:43.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:43.611+0000 7f79deb9c700 1 -- 172.21.15.31:0/2640950896 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f79d8153a00 msgr2=0x7f79d8153dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:43.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:43.611+0000 7f79deb9c700 1 --2- 172.21.15.31:0/2640950896 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f79d8153a00 0x7f79d8153dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:43.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:43.611+0000 7f79deb9c700 1 -- 172.21.15.31:0/2640950896 >> v1:172.21.15.62:6789/0 conn(0x7f79d815de50 legacy=0x7f79d815a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:43.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:43.611+0000 7f79deb9c700 1 -- 172.21.15.31:0/2640950896 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79d815b240 con 0x7f79d80cea00 2023-10-06T02:24:43.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:43.611+0000 7f79de39b700 1 --2- 172.21.15.31:0/2640950896 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f79d8153a00 0x7f79d8153dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:24:43.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:43.611+0000 7f79deb9c700 1 --2- 172.21.15.31:0/2640950896 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79d80cea00 0x7f79d80cee40 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f79d0009f20 tx=0x7f79d0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=441203c7e5257a96 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:43.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:43.611+0000 7f79dd399700 1 -- 172.21.15.31:0/2640950896 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d000e070 con 0x7f79d80cea00 2023-10-06T02:24:43.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:43.612+0000 7f79dd399700 1 -- 172.21.15.31:0/2640950896 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79d000ba80 con 0x7f79d80cea00 2023-10-06T02:24:43.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:43.612+0000 7f79dd399700 1 -- 172.21.15.31:0/2640950896 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d00092e0 con 0x7f79d80cea00 2023-10-06T02:24:43.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:43.612+0000 7f79df39d700 1 -- 172.21.15.31:0/2640950896 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79d80cea00 msgr2=0x7f79d80cee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:43.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:43.612+0000 7f79df39d700 1 --2- 172.21.15.31:0/2640950896 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79d80cea00 0x7f79d80cee40 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f79d0009f20 tx=0x7f79d0009840 comp rx=0 tx=0).stop 2023-10-06T02:24:43.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:43.612+0000 7f79df39d700 1 -- 172.21.15.31:0/2640950896 shutdown_connections 2023-10-06T02:24:43.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:43.612+0000 7f79df39d700 1 --2- 172.21.15.31:0/2640950896 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79d80cea00 0x7f79d80cee40 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:43.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:43.612+0000 7f79df39d700 1 --2- 172.21.15.31:0/2640950896 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f79d8153a00 0x7f79d8153dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:43.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:43.612+0000 7f79df39d700 1 -- 172.21.15.31:0/2640950896 >> 172.21.15.31:0/2640950896 conn(0x7f79d80703a0 msgr2=0x7f79d80707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:43.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:43.612+0000 7f79df39d700 1 -- 172.21.15.31:0/2640950896 shutdown_connections 2023-10-06T02:24:43.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:43.612+0000 7f79df39d700 1 -- 172.21.15.31:0/2640950896 wait complete. 2023-10-06T02:24:43.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:43.614+0000 7f79df39d700 1 Processor -- start 2023-10-06T02:24:43.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:43.614+0000 7f79df39d700 1 -- start start 2023-10-06T02:24:43.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:43.615+0000 7f79df39d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f79d80cea00 0x7f79d815ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:43.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:43.615+0000 7f79df39d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79d8153a00 0x7f79d8159180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:43.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:43.615+0000 7f79df39d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79d815de50 0x7f79d814def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:43.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:43.615+0000 7f79df39d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d80cb880 con 0x7f79d815de50 2023-10-06T02:24:43.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:43.615+0000 7f79df39d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d8160450 con 0x7f79d8153a00 2023-10-06T02:24:43.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:43.615+0000 7f79df39d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79d815f280 con 0x7f79d80cea00 2023-10-06T02:24:43.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:43.615+0000 7f79deb9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79d815de50 0x7f79d814def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:43.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:43.615+0000 7f79deb9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79d815de50 0x7f79d814def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41122/0 (socket says 172.21.15.31:41122) 2023-10-06T02:24:43.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:43.615+0000 7f79deb9c700 1 -- 172.21.15.31:0/3016809323 learned_addr learned my addr 172.21.15.31:0/3016809323 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:43.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:43.615+0000 7f79de39b700 1 --2- 172.21.15.31:0/3016809323 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f79d80cea00 0x7f79d815ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:43.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:43.615+0000 7f79ddb9a700 1 --2- 172.21.15.31:0/3016809323 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79d8153a00 0x7f79d8159180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:43.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:43.615+0000 7f79deb9c700 1 -- 172.21.15.31:0/3016809323 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f79d80cea00 msgr2=0x7f79d815ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:43.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:43.615+0000 7f79deb9c700 1 --2- 172.21.15.31:0/3016809323 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f79d80cea00 0x7f79d815ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:43.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:43.615+0000 7f79deb9c700 1 -- 172.21.15.31:0/3016809323 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79d8153a00 msgr2=0x7f79d8159180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:43.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:43.615+0000 7f79deb9c700 1 --2- 172.21.15.31:0/3016809323 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79d8153a00 0x7f79d8159180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:43.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:43.615+0000 7f79deb9c700 1 -- 172.21.15.31:0/3016809323 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79d0009ae0 con 0x7f79d815de50 2023-10-06T02:24:43.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:43.615+0000 7f79de39b700 1 --2- 172.21.15.31:0/3016809323 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f79d80cea00 0x7f79d815ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:24:43.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:43.615+0000 7f79ddb9a700 1 --2- 172.21.15.31:0/3016809323 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79d8153a00 0x7f79d8159180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:24:43.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:43.616+0000 7f79deb9c700 1 --2- 172.21.15.31:0/3016809323 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79d815de50 0x7f79d814def0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f79d000b6c0 tx=0x7f79d0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:43.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:43.616+0000 7f79ce7fc700 1 -- 172.21.15.31:0/3016809323 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d00032f0 con 0x7f79d815de50 2023-10-06T02:24:43.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:43.616+0000 7f79ce7fc700 1 -- 172.21.15.31:0/3016809323 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79d0026460 con 0x7f79d815de50 2023-10-06T02:24:43.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:43.616+0000 7f79df39d700 1 -- 172.21.15.31:0/3016809323 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79d80cd100 con 0x7f79d815de50 2023-10-06T02:24:43.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:43.616+0000 7f79ce7fc700 1 -- 172.21.15.31:0/3016809323 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d0026800 con 0x7f79d815de50 2023-10-06T02:24:43.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:43.616+0000 7f79df39d700 1 -- 172.21.15.31:0/3016809323 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79d815fe00 con 0x7f79d815de50 2023-10-06T02:24:43.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:43.617+0000 7f79df39d700 1 -- 172.21.15.31:0/3016809323 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79d8157550 con 0x7f79d815de50 2023-10-06T02:24:43.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:43.618+0000 7f79ce7fc700 1 -- 172.21.15.31:0/3016809323 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f79d0003040 con 0x7f79d815de50 2023-10-06T02:24:43.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:43.619+0000 7f79ce7fc700 1 --2- 172.21.15.31:0/3016809323 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f79bc067110 0x7f79bc0695c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:43.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:43.619+0000 7f79ce7fc700 1 -- 172.21.15.31:0/3016809323 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5958+0+0 (secure 0 0 0) 0x7f79d00869d0 con 0x7f79d815de50 2023-10-06T02:24:43.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:43.619+0000 7f79de39b700 1 --2- 172.21.15.31:0/3016809323 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f79bc067110 0x7f79bc0695c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:43.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:43.620+0000 7f79de39b700 1 --2- 172.21.15.31:0/3016809323 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f79bc067110 0x7f79bc0695c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f79d4006f00 tx=0x7f79d4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:43.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:43.624+0000 7f79ce7fc700 1 -- 172.21.15.31:0/3016809323 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f79d8157550 con 0x7f79d815de50 2023-10-06T02:24:43.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:43.871+0000 7f79df39d700 1 -- 172.21.15.31:0/3016809323 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f79d8157550 con 0x7f79d815de50 2023-10-06T02:24:44.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.382+0000 7f79ce7fc700 1 -- 172.21.15.31:0/3016809323 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 7 and data pool 8 v23) v1 ==== 172+0+0 (secure 0 0 0) 0x7f79d8157550 con 0x7f79d815de50 2023-10-06T02:24:44.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.390+0000 7f79df39d700 1 -- 172.21.15.31:0/3016809323 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f79bc067110 msgr2=0x7f79bc0695c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:44.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.390+0000 7f79df39d700 1 --2- 172.21.15.31:0/3016809323 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f79bc067110 0x7f79bc0695c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f79d4006f00 tx=0x7f79d4008040 comp rx=0 tx=0).stop 2023-10-06T02:24:44.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.390+0000 7f79df39d700 1 -- 172.21.15.31:0/3016809323 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79d815de50 msgr2=0x7f79d814def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:44.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.390+0000 7f79df39d700 1 --2- 172.21.15.31:0/3016809323 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79d815de50 0x7f79d814def0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f79d000b6c0 tx=0x7f79d0018000 comp rx=0 tx=0).stop 2023-10-06T02:24:44.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.394+0000 7f79df39d700 1 -- 172.21.15.31:0/3016809323 shutdown_connections 2023-10-06T02:24:44.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.394+0000 7f79df39d700 1 --2- 172.21.15.31:0/3016809323 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79d815de50 0x7f79d814def0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:44.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.394+0000 7f79df39d700 1 --2- 172.21.15.31:0/3016809323 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f79bc067110 0x7f79bc0695c0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:44.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.394+0000 7f79df39d700 1 --2- 172.21.15.31:0/3016809323 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f79d80cea00 0x7f79d815ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:44.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.394+0000 7f79df39d700 1 --2- 172.21.15.31:0/3016809323 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79d8153a00 0x7f79d8159180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:44.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.394+0000 7f79df39d700 1 -- 172.21.15.31:0/3016809323 >> 172.21.15.31:0/3016809323 conn(0x7f79d80703a0 msgr2=0x7f79d8150fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:44.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.400+0000 7f79df39d700 1 -- 172.21.15.31:0/3016809323 shutdown_connections 2023-10-06T02:24:44.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.401+0000 7f79df39d700 1 -- 172.21.15.31:0/3016809323 wait complete. 2023-10-06T02:24:44.411 INFO:teuthology.orchestra.run.smithi031.stderr:new fs with metadata pool 7 and data pool 8 2023-10-06T02:24:44.434 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:24:44.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.621+0000 7f423cf9d700 1 Processor -- start 2023-10-06T02:24:44.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.621+0000 7f423cf9d700 1 -- start start 2023-10-06T02:24:44.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.623+0000 7f423cf9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f423806e0a0 0x7f42380c6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:44.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.623+0000 7f423cf9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4238150380 0x7f423806a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:44.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.623+0000 7f423cf9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f423806a490 con 0x7f423806f3a0 2023-10-06T02:24:44.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.623+0000 7f423cf9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42380c7490 con 0x7f4238150380 2023-10-06T02:24:44.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.623+0000 7f423cf9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42380c7770 con 0x7f423806e0a0 2023-10-06T02:24:44.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.623+0000 7f4237fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4238150380 0x7f423806a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:44.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.623+0000 7f4237fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4238150380 0x7f423806a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41132/0 (socket says 172.21.15.31:41132) 2023-10-06T02:24:44.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.623+0000 7f4237fff700 1 -- 172.21.15.31:0/522512568 learned_addr learned my addr 172.21.15.31:0/522512568 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:44.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.623+0000 7f4237fff700 1 -- 172.21.15.31:0/522512568 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f423806e0a0 msgr2=0x7f42380c6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:44.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.623+0000 7f4237fff700 1 --2- 172.21.15.31:0/522512568 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f423806e0a0 0x7f42380c6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:44.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.623+0000 7f4237fff700 1 -- 172.21.15.31:0/522512568 >> v1:172.21.15.62:6789/0 conn(0x7f423806f3a0 legacy=0x7f423814fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:44.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.624+0000 7f4237fff700 1 -- 172.21.15.31:0/522512568 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42380c7a50 con 0x7f4238150380 2023-10-06T02:24:44.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.624+0000 7f4237fff700 1 --2- 172.21.15.31:0/522512568 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4238150380 0x7f423806a920 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f423000c0d0 tx=0x7f423000a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8edff6cb6809d76c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:44.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.624+0000 7f42367fc700 1 -- 172.21.15.31:0/522512568 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f423000f070 con 0x7f4238150380 2023-10-06T02:24:44.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.624+0000 7f42367fc700 1 -- 172.21.15.31:0/522512568 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f423000a2e0 con 0x7f4238150380 2023-10-06T02:24:44.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.624+0000 7f42367fc700 1 -- 172.21.15.31:0/522512568 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f423000cdf0 con 0x7f4238150380 2023-10-06T02:24:44.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.625+0000 7f423cf9d700 1 -- 172.21.15.31:0/522512568 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4238150380 msgr2=0x7f423806a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:44.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.625+0000 7f423cf9d700 1 --2- 172.21.15.31:0/522512568 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4238150380 0x7f423806a920 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f423000c0d0 tx=0x7f423000a950 comp rx=0 tx=0).stop 2023-10-06T02:24:44.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.625+0000 7f423cf9d700 1 -- 172.21.15.31:0/522512568 shutdown_connections 2023-10-06T02:24:44.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.625+0000 7f423cf9d700 1 --2- 172.21.15.31:0/522512568 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4238150380 0x7f423806a920 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:44.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.625+0000 7f423cf9d700 1 --2- 172.21.15.31:0/522512568 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f423806e0a0 0x7f42380c6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:44.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.625+0000 7f423cf9d700 1 -- 172.21.15.31:0/522512568 >> 172.21.15.31:0/522512568 conn(0x7f423806d170 msgr2=0x7f42380cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:44.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.625+0000 7f423cf9d700 1 -- 172.21.15.31:0/522512568 shutdown_connections 2023-10-06T02:24:44.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.625+0000 7f423cf9d700 1 -- 172.21.15.31:0/522512568 wait complete. 2023-10-06T02:24:44.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.626+0000 7f423cf9d700 1 Processor -- start 2023-10-06T02:24:44.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.626+0000 7f423cf9d700 1 -- start start 2023-10-06T02:24:44.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.627+0000 7f423cf9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f423806e0a0 0x7f423813c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:44.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.627+0000 7f423cf9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f423806f3a0 0x7f4238156cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:44.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.627+0000 7f423cf9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4238150380 0x7f4238157200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:44.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.627+0000 7f423cf9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f423813cbb0 con 0x7f4238150380 2023-10-06T02:24:44.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.627+0000 7f423cf9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4238157e90 con 0x7f423806f3a0 2023-10-06T02:24:44.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.627+0000 7f423cf9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42381578d0 con 0x7f423806e0a0 2023-10-06T02:24:44.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.627+0000 7f4237fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4238150380 0x7f4238157200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:44.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.627+0000 7f4237fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4238150380 0x7f4238157200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41144/0 (socket says 172.21.15.31:41144) 2023-10-06T02:24:44.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.627+0000 7f4237fff700 1 -- 172.21.15.31:0/59653330 learned_addr learned my addr 172.21.15.31:0/59653330 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:44.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.628+0000 7f42377fe700 1 --2- 172.21.15.31:0/59653330 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f423806e0a0 0x7f423813c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:44.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.628+0000 7f4236ffd700 1 --2- 172.21.15.31:0/59653330 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f423806f3a0 0x7f4238156cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:44.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.628+0000 7f4237fff700 1 -- 172.21.15.31:0/59653330 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f423806e0a0 msgr2=0x7f423813c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:44.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.628+0000 7f4237fff700 1 --2- 172.21.15.31:0/59653330 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f423806e0a0 0x7f423813c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:44.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.628+0000 7f4237fff700 1 -- 172.21.15.31:0/59653330 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f423806f3a0 msgr2=0x7f4238156cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:44.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.628+0000 7f4237fff700 1 --2- 172.21.15.31:0/59653330 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f423806f3a0 0x7f4238156cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:44.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.628+0000 7f4237fff700 1 -- 172.21.15.31:0/59653330 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f423000abf0 con 0x7f4238150380 2023-10-06T02:24:44.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.628+0000 7f4236ffd700 1 --2- 172.21.15.31:0/59653330 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f423806f3a0 0x7f4238156cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:24:44.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.628+0000 7f4237fff700 1 --2- 172.21.15.31:0/59653330 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4238150380 0x7f4238157200 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f423000a990 tx=0x7f4230018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:44.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.628+0000 7f421f7fe700 1 -- 172.21.15.31:0/59653330 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4230008290 con 0x7f4238150380 2023-10-06T02:24:44.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.628+0000 7f421f7fe700 1 -- 172.21.15.31:0/59653330 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4230013430 con 0x7f4238150380 2023-10-06T02:24:44.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.628+0000 7f421f7fe700 1 -- 172.21.15.31:0/59653330 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42300137d0 con 0x7f4238150380 2023-10-06T02:24:44.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.628+0000 7f423cf9d700 1 -- 172.21.15.31:0/59653330 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4238157bb0 con 0x7f4238150380 2023-10-06T02:24:44.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.629+0000 7f423cf9d700 1 -- 172.21.15.31:0/59653330 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4238169350 con 0x7f4238150380 2023-10-06T02:24:44.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.629+0000 7f423cf9d700 1 -- 172.21.15.31:0/59653330 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f423804c3c0 con 0x7f4238150380 2023-10-06T02:24:44.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.630+0000 7f421f7fe700 1 -- 172.21.15.31:0/59653330 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f4230013c00 con 0x7f4238150380 2023-10-06T02:24:44.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.631+0000 7f421f7fe700 1 --2- 172.21.15.31:0/59653330 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f42240670e0 0x7f4224069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:44.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.631+0000 7f421f7fe700 1 -- 172.21.15.31:0/59653330 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f4230086790 con 0x7f4238150380 2023-10-06T02:24:44.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.632+0000 7f42377fe700 1 --2- 172.21.15.31:0/59653330 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f42240670e0 0x7f4224069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:44.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.632+0000 7f42377fe700 1 --2- 172.21.15.31:0/59653330 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f42240670e0 0x7f4224069590 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4228006fd0 tx=0x7f4228008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:44.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.635+0000 7f421f7fe700 1 -- 172.21.15.31:0/59653330 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4230053470 con 0x7f4238150380 2023-10-06T02:24:44.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.838+0000 7f423cf9d700 1 -- 172.21.15.31:0/59653330 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f42380c7d30 con 0x7f4238150380 2023-10-06T02:24:44.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.839+0000 7f421f7fe700 1 -- 172.21.15.31:0/59653330 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+16831 (secure 0 0 0) 0x7f4230056a90 con 0x7f4238150380 2023-10-06T02:24:44.840 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:24:44.840 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":46,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:24:43.871984+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:24:42.035212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-10-06T02:24:42.849819+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:24:44.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.841+0000 7f423cf9d700 1 -- 172.21.15.31:0/59653330 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f42240670e0 msgr2=0x7f4224069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:44.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.841+0000 7f423cf9d700 1 --2- 172.21.15.31:0/59653330 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f42240670e0 0x7f4224069590 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4228006fd0 tx=0x7f4228008040 comp rx=0 tx=0).stop 2023-10-06T02:24:44.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.841+0000 7f423cf9d700 1 -- 172.21.15.31:0/59653330 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4238150380 msgr2=0x7f4238157200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:44.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.841+0000 7f423cf9d700 1 --2- 172.21.15.31:0/59653330 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4238150380 0x7f4238157200 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f423000a990 tx=0x7f4230018000 comp rx=0 tx=0).stop 2023-10-06T02:24:44.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.842+0000 7f423cf9d700 1 -- 172.21.15.31:0/59653330 shutdown_connections 2023-10-06T02:24:44.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.842+0000 7f423cf9d700 1 --2- 172.21.15.31:0/59653330 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4238150380 0x7f4238157200 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:44.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.842+0000 7f423cf9d700 1 --2- 172.21.15.31:0/59653330 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f42240670e0 0x7f4224069590 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:44.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.842+0000 7f423cf9d700 1 --2- 172.21.15.31:0/59653330 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f423806e0a0 0x7f423813c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:44.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.842+0000 7f423cf9d700 1 --2- 172.21.15.31:0/59653330 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f423806f3a0 0x7f4238156cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:44.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.842+0000 7f423cf9d700 1 -- 172.21.15.31:0/59653330 >> 172.21.15.31:0/59653330 conn(0x7f423806d170 msgr2=0x7f42380cae60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:44.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.842+0000 7f423cf9d700 1 -- 172.21.15.31:0/59653330 shutdown_connections 2023-10-06T02:24:44.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:44.842+0000 7f423cf9d700 1 -- 172.21.15.31:0/59653330 wait complete. 2023-10-06T02:24:44.860 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:24:45.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.046+0000 7f9f1d59d700 1 Processor -- start 2023-10-06T02:24:45.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.046+0000 7f9f1d59d700 1 -- start start 2023-10-06T02:24:45.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.047+0000 7f9f1d59d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9f1813fb20 0x7f9f1815a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:45.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.047+0000 7f9f1d59d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9f1813fef0 0x7f9f180c7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:45.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.047+0000 7f9f1d59d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f1815ae70 con 0x7f9f1806aaa0 2023-10-06T02:24:45.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.047+0000 7f9f1d59d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f1806af60 con 0x7f9f1813fef0 2023-10-06T02:24:45.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.047+0000 7f9f1d59d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f1806b240 con 0x7f9f1813fb20 2023-10-06T02:24:45.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.047+0000 7f9f177fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9f1813fef0 0x7f9f180c7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:45.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.048+0000 7f9f177fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9f1813fef0 0x7f9f180c7990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41148/0 (socket says 172.21.15.31:41148) 2023-10-06T02:24:45.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.048+0000 7f9f177fe700 1 -- 172.21.15.31:0/2103256370 learned_addr learned my addr 172.21.15.31:0/2103256370 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:45.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.048+0000 7f9f17fff700 1 --2- 172.21.15.31:0/2103256370 >> v2:172.21.15.62:3300/0 conn(0x7f9f1813fb20 0x7f9f1815a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:45.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.048+0000 7f9f177fe700 1 -- 172.21.15.31:0/2103256370 >> v2:172.21.15.62:3300/0 conn(0x7f9f1813fb20 msgr2=0x7f9f1815a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:45.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.048+0000 7f9f177fe700 1 --2- 172.21.15.31:0/2103256370 >> v2:172.21.15.62:3300/0 conn(0x7f9f1813fb20 0x7f9f1815a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:45.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.048+0000 7f9f177fe700 1 -- 172.21.15.31:0/2103256370 >> v1:172.21.15.62:6789/0 conn(0x7f9f1806aaa0 legacy=0x7f9f180c6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:45.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.048+0000 7f9f177fe700 1 -- 172.21.15.31:0/2103256370 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f1806b520 con 0x7f9f1813fef0 2023-10-06T02:24:45.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.048+0000 7f9f177fe700 1 --2- 172.21.15.31:0/2103256370 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f1813fef0 0x7f9f180c7990 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f9f180695b0 tx=0x7f9f0c009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77694d1e66e8a90c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:45.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.049+0000 7f9f16ffd700 1 -- 172.21.15.31:0/2103256370 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f0c005c10 con 0x7f9f1813fef0 2023-10-06T02:24:45.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.049+0000 7f9f16ffd700 1 -- 172.21.15.31:0/2103256370 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f0c005d70 con 0x7f9f1813fef0 2023-10-06T02:24:45.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.049+0000 7f9f16ffd700 1 -- 172.21.15.31:0/2103256370 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f0c006260 con 0x7f9f1813fef0 2023-10-06T02:24:45.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.049+0000 7f9f1d59d700 1 -- 172.21.15.31:0/2103256370 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f1813fef0 msgr2=0x7f9f180c7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:45.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.049+0000 7f9f1d59d700 1 --2- 172.21.15.31:0/2103256370 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f1813fef0 0x7f9f180c7990 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f9f180695b0 tx=0x7f9f0c009220 comp rx=0 tx=0).stop 2023-10-06T02:24:45.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.049+0000 7f9f1d59d700 1 -- 172.21.15.31:0/2103256370 shutdown_connections 2023-10-06T02:24:45.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.049+0000 7f9f1d59d700 1 --2- 172.21.15.31:0/2103256370 >> v2:172.21.15.62:3300/0 conn(0x7f9f1813fb20 0x7f9f1815a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:45.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.049+0000 7f9f1d59d700 1 --2- 172.21.15.31:0/2103256370 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f1813fef0 0x7f9f180c7990 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:45.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.049+0000 7f9f1d59d700 1 -- 172.21.15.31:0/2103256370 >> 172.21.15.31:0/2103256370 conn(0x7f9f1806d170 msgr2=0x7f9f180cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:45.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.049+0000 7f9f1d59d700 1 -- 172.21.15.31:0/2103256370 shutdown_connections 2023-10-06T02:24:45.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.049+0000 7f9f1d59d700 1 -- 172.21.15.31:0/2103256370 wait complete. 2023-10-06T02:24:45.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.051+0000 7f9f1d59d700 1 Processor -- start 2023-10-06T02:24:45.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.051+0000 7f9f1d59d700 1 -- start start 2023-10-06T02:24:45.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.052+0000 7f9f1d59d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9f1806aaa0 0x7f9f181373f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:45.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.052+0000 7f9f1d59d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f1813fb20 0x7f9f18155410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:45.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.052+0000 7f9f1d59d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f1813fef0 0x7f9f1816a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:45.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.052+0000 7f9f1d59d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f1813a750 con 0x7f9f1813fb20 2023-10-06T02:24:45.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.052+0000 7f9f1d59d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f1813c4b0 con 0x7f9f1813fef0 2023-10-06T02:24:45.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.052+0000 7f9f1d59d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f18155950 con 0x7f9f1806aaa0 2023-10-06T02:24:45.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.052+0000 7f9f177fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f1813fb20 0x7f9f18155410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:45.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.052+0000 7f9f177fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f1813fb20 0x7f9f18155410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41156/0 (socket says 172.21.15.31:41156) 2023-10-06T02:24:45.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.052+0000 7f9f177fe700 1 -- 172.21.15.31:0/3390821040 learned_addr learned my addr 172.21.15.31:0/3390821040 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:45.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.052+0000 7f9f17fff700 1 --2- 172.21.15.31:0/3390821040 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9f1806aaa0 0x7f9f181373f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:45.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.052+0000 7f9f177fe700 1 -- 172.21.15.31:0/3390821040 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9f1806aaa0 msgr2=0x7f9f181373f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:45.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.052+0000 7f9f1cd9c700 1 --2- 172.21.15.31:0/3390821040 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f1813fef0 0x7f9f1816a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:45.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.052+0000 7f9f177fe700 1 --2- 172.21.15.31:0/3390821040 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9f1806aaa0 0x7f9f181373f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:45.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.052+0000 7f9f177fe700 1 -- 172.21.15.31:0/3390821040 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f1813fef0 msgr2=0x7f9f1816a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:45.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.052+0000 7f9f177fe700 1 --2- 172.21.15.31:0/3390821040 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f1813fef0 0x7f9f1816a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:45.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.052+0000 7f9f177fe700 1 -- 172.21.15.31:0/3390821040 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f0c009480 con 0x7f9f1813fb20 2023-10-06T02:24:45.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.052+0000 7f9f17fff700 1 --2- 172.21.15.31:0/3390821040 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9f1806aaa0 0x7f9f181373f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:24:45.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.052+0000 7f9f1cd9c700 1 --2- 172.21.15.31:0/3390821040 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f1813fef0 0x7f9f1816a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:24:45.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.053+0000 7f9f177fe700 1 --2- 172.21.15.31:0/3390821040 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f1813fb20 0x7f9f18155410 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f9f18071320 tx=0x7f9f0c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:45.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.053+0000 7f9efffff700 1 -- 172.21.15.31:0/3390821040 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f0c00b8d0 con 0x7f9f1813fb20 2023-10-06T02:24:45.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.053+0000 7f9efffff700 1 -- 172.21.15.31:0/3390821040 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f0c020070 con 0x7f9f1813fb20 2023-10-06T02:24:45.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.053+0000 7f9f1d59d700 1 -- 172.21.15.31:0/3390821040 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f1816dac0 con 0x7f9f1813fb20 2023-10-06T02:24:45.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.053+0000 7f9efffff700 1 -- 172.21.15.31:0/3390821040 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f0c00bc60 con 0x7f9f1813fb20 2023-10-06T02:24:45.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.053+0000 7f9f1d59d700 1 -- 172.21.15.31:0/3390821040 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f1813bf10 con 0x7f9f1813fb20 2023-10-06T02:24:45.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.054+0000 7f9f1d59d700 1 -- 172.21.15.31:0/3390821040 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f18137930 con 0x7f9f1813fb20 2023-10-06T02:24:45.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.055+0000 7f9efffff700 1 -- 172.21.15.31:0/3390821040 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f9f0c022030 con 0x7f9f1813fb20 2023-10-06T02:24:45.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.056+0000 7f9efffff700 1 --2- 172.21.15.31:0/3390821040 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f9f040670e0 0x7f9f04069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:45.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.056+0000 7f9efffff700 1 -- 172.21.15.31:0/3390821040 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9f0c086600 con 0x7f9f1813fb20 2023-10-06T02:24:45.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.056+0000 7f9f17fff700 1 --2- 172.21.15.31:0/3390821040 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f9f040670e0 0x7f9f04069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:45.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.057+0000 7f9f17fff700 1 --2- 172.21.15.31:0/3390821040 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f9f040670e0 0x7f9f04069590 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9f08006f00 tx=0x7f9f08008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:45.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.061+0000 7f9efffff700 1 -- 172.21.15.31:0/3390821040 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9f0c0533f0 con 0x7f9f1813fb20 2023-10-06T02:24:45.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.264+0000 7f9f1d59d700 1 -- 172.21.15.31:0/3390821040 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9f1804c3c0 con 0x7f9f1813fb20 2023-10-06T02:24:45.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.265+0000 7f9efffff700 1 -- 172.21.15.31:0/3390821040 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+16831 (secure 0 0 0) 0x7f9f0c056a10 con 0x7f9f1813fb20 2023-10-06T02:24:45.265 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:24:45.265 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":46,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:24:43.871984+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:24:42.035212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-10-06T02:24:42.849819+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:24:45.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.267+0000 7f9f1d59d700 1 -- 172.21.15.31:0/3390821040 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f9f040670e0 msgr2=0x7f9f04069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:45.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.267+0000 7f9f1d59d700 1 --2- 172.21.15.31:0/3390821040 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f9f040670e0 0x7f9f04069590 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9f08006f00 tx=0x7f9f08008040 comp rx=0 tx=0).stop 2023-10-06T02:24:45.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.267+0000 7f9f1d59d700 1 -- 172.21.15.31:0/3390821040 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f1813fb20 msgr2=0x7f9f18155410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:45.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.267+0000 7f9f1d59d700 1 --2- 172.21.15.31:0/3390821040 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f1813fb20 0x7f9f18155410 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f9f18071320 tx=0x7f9f0c016000 comp rx=0 tx=0).stop 2023-10-06T02:24:45.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.267+0000 7f9f1d59d700 1 -- 172.21.15.31:0/3390821040 shutdown_connections 2023-10-06T02:24:45.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.267+0000 7f9f1d59d700 1 --2- 172.21.15.31:0/3390821040 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f1813fef0 0x7f9f1816a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:45.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.267+0000 7f9f1d59d700 1 --2- 172.21.15.31:0/3390821040 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f9f040670e0 0x7f9f04069590 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:45.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.267+0000 7f9f1d59d700 1 --2- 172.21.15.31:0/3390821040 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9f1806aaa0 0x7f9f181373f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:45.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.267+0000 7f9f1d59d700 1 --2- 172.21.15.31:0/3390821040 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f1813fb20 0x7f9f18155410 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:45.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.267+0000 7f9f1d59d700 1 -- 172.21.15.31:0/3390821040 >> 172.21.15.31:0/3390821040 conn(0x7f9f1806d170 msgr2=0x7f9f180b0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:45.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.267+0000 7f9f1d59d700 1 -- 172.21.15.31:0/3390821040 shutdown_connections 2023-10-06T02:24:45.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.267+0000 7f9f1d59d700 1 -- 172.21.15.31:0/3390821040 wait complete. 2023-10-06T02:24:45.285 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T02:24:45.361 INFO:tasks.ceph.mds.b.smithi062.stderr:2023-10-06T02:24:45.360+0000 7f631f1f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T02:24:45.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.478+0000 7fe3885ff700 1 Processor -- start 2023-10-06T02:24:45.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.479+0000 7fe3885ff700 1 -- start start 2023-10-06T02:24:45.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.480+0000 7fe3885ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe3800ca7a0 0x7fe38015e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:45.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.480+0000 7fe3885ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe3800cab70 0x7fe38006a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:45.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.480+0000 7fe3885ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe38006b4b0 con 0x7fe380159330 2023-10-06T02:24:45.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.480+0000 7fe3885ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe38006a600 con 0x7fe3800cab70 2023-10-06T02:24:45.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.480+0000 7fe3885ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe38006a8a0 con 0x7fe3800ca7a0 2023-10-06T02:24:45.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.481+0000 7fe38639b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe3800ca7a0 0x7fe38015e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:45.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.481+0000 7fe385b9a700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe3800cab70 0x7fe38006a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:45.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.481+0000 7fe385b9a700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe3800cab70 0x7fe38006a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:53978/0 (socket says 172.21.15.31:53978) 2023-10-06T02:24:45.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.481+0000 7fe385b9a700 1 -- 172.21.15.31:0/1107424589 learned_addr learned my addr 172.21.15.31:0/1107424589 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:45.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.481+0000 7fe386b9c700 1 --1- 172.21.15.31:0/1107424589 >> v1:172.21.15.62:6789/0 conn(0x7fe380159330 0x7fe3801597b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:36712/0 (socket says 172.21.15.31:36712) 2023-10-06T02:24:45.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.481+0000 7fe385b9a700 1 -- 172.21.15.31:0/1107424589 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe3800ca7a0 msgr2=0x7fe38015e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:45.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.481+0000 7fe385b9a700 1 --2- 172.21.15.31:0/1107424589 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe3800ca7a0 0x7fe38015e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:45.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.481+0000 7fe385b9a700 1 -- 172.21.15.31:0/1107424589 >> v1:172.21.15.62:6789/0 conn(0x7fe380159330 legacy=0x7fe3801597b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:45.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.481+0000 7fe385b9a700 1 -- 172.21.15.31:0/1107424589 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe38006ab80 con 0x7fe3800cab70 2023-10-06T02:24:45.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.481+0000 7fe38639b700 1 --2- 172.21.15.31:0/1107424589 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe3800ca7a0 0x7fe38015e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:24:45.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.481+0000 7fe385399700 1 -- 172.21.15.31:0/1107424589 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2390579492 0 0) 0x7fe38006b4b0 con 0x7fe380159330 2023-10-06T02:24:45.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.482+0000 7fe385b9a700 1 --2- 172.21.15.31:0/1107424589 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3800cab70 0x7fe38006a0c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe3700099b0 tx=0x7fe370009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81741019cfcc0433 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:45.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.482+0000 7fe385399700 1 -- 172.21.15.31:0/1107424589 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe370009ea0 con 0x7fe3800cab70 2023-10-06T02:24:45.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.482+0000 7fe385399700 1 -- 172.21.15.31:0/1107424589 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe37000b860 con 0x7fe3800cab70 2023-10-06T02:24:45.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.482+0000 7fe385399700 1 -- 172.21.15.31:0/1107424589 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37000bc00 con 0x7fe3800cab70 2023-10-06T02:24:45.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.482+0000 7fe3885ff700 1 -- 172.21.15.31:0/1107424589 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3800cab70 msgr2=0x7fe38006a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:45.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.482+0000 7fe3885ff700 1 --2- 172.21.15.31:0/1107424589 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3800cab70 0x7fe38006a0c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe3700099b0 tx=0x7fe370009250 comp rx=0 tx=0).stop 2023-10-06T02:24:45.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.482+0000 7fe3885ff700 1 -- 172.21.15.31:0/1107424589 shutdown_connections 2023-10-06T02:24:45.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.482+0000 7fe3885ff700 1 --2- 172.21.15.31:0/1107424589 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe3800ca7a0 0x7fe38015e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:45.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.482+0000 7fe3885ff700 1 --2- 172.21.15.31:0/1107424589 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3800cab70 0x7fe38006a0c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:45.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.482+0000 7fe3885ff700 1 -- 172.21.15.31:0/1107424589 >> 172.21.15.31:0/1107424589 conn(0x7fe38006ea90 msgr2=0x7fe38006ee90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:45.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.483+0000 7fe3885ff700 1 -- 172.21.15.31:0/1107424589 shutdown_connections 2023-10-06T02:24:45.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.483+0000 7fe3885ff700 1 -- 172.21.15.31:0/1107424589 wait complete. 2023-10-06T02:24:45.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.484+0000 7fe3885ff700 1 Processor -- start 2023-10-06T02:24:45.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.484+0000 7fe3885ff700 1 -- start start 2023-10-06T02:24:45.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.485+0000 7fe3885ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3800ca7a0 0x7fe3800c8660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:45.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.485+0000 7fe3885ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe3800cab70 0x7fe3800c9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:45.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.485+0000 7fe3885ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe380159330 0x7fe38016a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:45.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.485+0000 7fe3885ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3800c7120 con 0x7fe380159330 2023-10-06T02:24:45.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.485+0000 7fe3885ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe38016adb0 con 0x7fe3800ca7a0 2023-10-06T02:24:45.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.485+0000 7fe3885ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3800c8c40 con 0x7fe3800cab70 2023-10-06T02:24:45.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.485+0000 7fe386b9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe380159330 0x7fe38016a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:45.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.485+0000 7fe386b9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe380159330 0x7fe38016a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41166/0 (socket says 172.21.15.31:41166) 2023-10-06T02:24:45.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.485+0000 7fe386b9c700 1 -- 172.21.15.31:0/2338201672 learned_addr learned my addr 172.21.15.31:0/2338201672 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:45.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.486+0000 7fe385b9a700 1 --2- 172.21.15.31:0/2338201672 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe3800cab70 0x7fe3800c9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:45.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.486+0000 7fe38639b700 1 --2- 172.21.15.31:0/2338201672 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3800ca7a0 0x7fe3800c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:45.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.486+0000 7fe386b9c700 1 -- 172.21.15.31:0/2338201672 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe3800cab70 msgr2=0x7fe3800c9ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:45.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.486+0000 7fe386b9c700 1 --2- 172.21.15.31:0/2338201672 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe3800cab70 0x7fe3800c9ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:45.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.486+0000 7fe386b9c700 1 -- 172.21.15.31:0/2338201672 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3800ca7a0 msgr2=0x7fe3800c8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:45.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.486+0000 7fe386b9c700 1 --2- 172.21.15.31:0/2338201672 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3800ca7a0 0x7fe3800c8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:45.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.486+0000 7fe386b9c700 1 -- 172.21.15.31:0/2338201672 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe37c009420 con 0x7fe380159330 2023-10-06T02:24:45.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.486+0000 7fe38639b700 1 --2- 172.21.15.31:0/2338201672 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3800ca7a0 0x7fe3800c8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:24:45.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.486+0000 7fe385b9a700 1 --2- 172.21.15.31:0/2338201672 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe3800cab70 0x7fe3800c9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:24:45.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.486+0000 7fe386b9c700 1 --2- 172.21.15.31:0/2338201672 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe380159330 0x7fe38016a870 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fe37800ccc0 tx=0x7fe378001a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:45.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.486+0000 7fe3767fc700 1 -- 172.21.15.31:0/2338201672 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37800ce00 con 0x7fe380159330 2023-10-06T02:24:45.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.486+0000 7fe3767fc700 1 -- 172.21.15.31:0/2338201672 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe378005df0 con 0x7fe380159330 2023-10-06T02:24:45.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.486+0000 7fe3885ff700 1 -- 172.21.15.31:0/2338201672 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3700094f0 con 0x7fe380159330 2023-10-06T02:24:45.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.486+0000 7fe3767fc700 1 -- 172.21.15.31:0/2338201672 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37800abb0 con 0x7fe380159330 2023-10-06T02:24:45.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.486+0000 7fe3885ff700 1 -- 172.21.15.31:0/2338201672 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3800c9560 con 0x7fe380159330 2023-10-06T02:24:45.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.487+0000 7fe3885ff700 1 -- 172.21.15.31:0/2338201672 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3800c9060 con 0x7fe380159330 2023-10-06T02:24:45.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.488+0000 7fe3767fc700 1 -- 172.21.15.31:0/2338201672 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7fe3780070f0 con 0x7fe380159330 2023-10-06T02:24:45.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.489+0000 7fe3767fc700 1 --2- 172.21.15.31:0/2338201672 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe3640670e0 0x7fe364069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:45.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.489+0000 7fe3767fc700 1 -- 172.21.15.31:0/2338201672 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe378084b20 con 0x7fe380159330 2023-10-06T02:24:45.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.489+0000 7fe38639b700 1 --2- 172.21.15.31:0/2338201672 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe3640670e0 0x7fe364069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:45.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.490+0000 7fe38639b700 1 --2- 172.21.15.31:0/2338201672 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe3640670e0 0x7fe364069590 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe37c0093f0 tx=0x7fe37c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:45.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.493+0000 7fe3767fc700 1 -- 172.21.15.31:0/2338201672 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe3800c9060 con 0x7fe380159330 2023-10-06T02:24:45.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:45.742+0000 7fe3885ff700 1 -- 172.21.15.31:0/2338201672 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fe3800c9030 con 0x7fe380159330 2023-10-06T02:24:46.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:46.405+0000 7fe3767fc700 1 -- 172.21.15.31:0/2338201672 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v25) v1 ==== 118+0+0 (secure 0 0 0) 0x7fe378051a30 con 0x7fe380159330 2023-10-06T02:24:46.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:46.408+0000 7fe3885ff700 1 -- 172.21.15.31:0/2338201672 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe3640670e0 msgr2=0x7fe364069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:46.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:46.408+0000 7fe3885ff700 1 --2- 172.21.15.31:0/2338201672 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe3640670e0 0x7fe364069590 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe37c0093f0 tx=0x7fe37c008040 comp rx=0 tx=0).stop 2023-10-06T02:24:46.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:46.408+0000 7fe3885ff700 1 -- 172.21.15.31:0/2338201672 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe380159330 msgr2=0x7fe38016a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:46.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:46.408+0000 7fe3885ff700 1 --2- 172.21.15.31:0/2338201672 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe380159330 0x7fe38016a870 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fe37800ccc0 tx=0x7fe378001a70 comp rx=0 tx=0).stop 2023-10-06T02:24:46.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:46.408+0000 7fe3885ff700 1 -- 172.21.15.31:0/2338201672 shutdown_connections 2023-10-06T02:24:46.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:46.408+0000 7fe3885ff700 1 --2- 172.21.15.31:0/2338201672 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe380159330 0x7fe38016a870 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:46.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:46.408+0000 7fe3885ff700 1 --2- 172.21.15.31:0/2338201672 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3800ca7a0 0x7fe3800c8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:46.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:46.408+0000 7fe3885ff700 1 --2- 172.21.15.31:0/2338201672 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe3640670e0 0x7fe364069590 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:46.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:46.408+0000 7fe3885ff700 1 --2- 172.21.15.31:0/2338201672 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe3800cab70 0x7fe3800c9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:46.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:46.408+0000 7fe3885ff700 1 -- 172.21.15.31:0/2338201672 >> 172.21.15.31:0/2338201672 conn(0x7fe38006ea90 msgr2=0x7fe3800b1a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:46.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:46.408+0000 7fe3885ff700 1 -- 172.21.15.31:0/2338201672 shutdown_connections 2023-10-06T02:24:46.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:46.408+0000 7fe3885ff700 1 -- 172.21.15.31:0/2338201672 wait complete. 2023-10-06T02:24:46.429 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T02:24:46.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:46.623+0000 7f20f6dff700 1 Processor -- start 2023-10-06T02:24:46.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:46.624+0000 7f20f6dff700 1 -- start start 2023-10-06T02:24:46.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:46.625+0000 7f20f6dff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f20f00acdd0 0x7f20f00b14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:46.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:46.625+0000 7f20f6dff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f20f00b1a10 0x7f20f0161350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:46.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:46.625+0000 7f20f6dff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20f00cc060 con 0x7f20f01576d0 2023-10-06T02:24:46.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:46.625+0000 7f20f6dff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20f00b1e90 con 0x7f20f00acdd0 2023-10-06T02:24:46.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:46.625+0000 7f20f6dff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20f00cb450 con 0x7f20f00b1a10 2023-10-06T02:24:46.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:46.625+0000 7f20effff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f20f00acdd0 0x7f20f00b14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:46.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:46.625+0000 7f20effff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f20f00acdd0 0x7f20f00b14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41170/0 (socket says 172.21.15.31:41170) 2023-10-06T02:24:46.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:46.625+0000 7f20effff700 1 -- 172.21.15.31:0/2404337627 learned_addr learned my addr 172.21.15.31:0/2404337627 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:46.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:46.626+0000 7f20f539c700 1 --2- 172.21.15.31:0/2404337627 >> v2:172.21.15.62:3300/0 conn(0x7f20f00b1a10 0x7f20f0161350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:46.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:46.626+0000 7f20effff700 1 -- 172.21.15.31:0/2404337627 >> v2:172.21.15.62:3300/0 conn(0x7f20f00b1a10 msgr2=0x7f20f0161350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:46.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:46.626+0000 7f20effff700 1 --2- 172.21.15.31:0/2404337627 >> v2:172.21.15.62:3300/0 conn(0x7f20f00b1a10 0x7f20f0161350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:46.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:46.626+0000 7f20effff700 1 -- 172.21.15.31:0/2404337627 >> v1:172.21.15.62:6789/0 conn(0x7f20f01576d0 legacy=0x7f20f00ae590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:46.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:46.626+0000 7f20effff700 1 -- 172.21.15.31:0/2404337627 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20f00cb730 con 0x7f20f00acdd0 2023-10-06T02:24:46.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:46.626+0000 7f20f539c700 1 --2- 172.21.15.31:0/2404337627 >> v2:172.21.15.62:3300/0 conn(0x7f20f00b1a10 0x7f20f0161350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:24:46.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:46.626+0000 7f20effff700 1 --2- 172.21.15.31:0/2404337627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20f00acdd0 0x7f20f00b14d0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f20f006d940 tx=0x7f20dc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24b02b0bc0f3fcc1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:46.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:46.626+0000 7f20ef7fe700 1 -- 172.21.15.31:0/2404337627 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20dc00b820 con 0x7f20f00acdd0 2023-10-06T02:24:46.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:46.626+0000 7f20ef7fe700 1 -- 172.21.15.31:0/2404337627 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20dc00b980 con 0x7f20f00acdd0 2023-10-06T02:24:46.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:46.626+0000 7f20ef7fe700 1 -- 172.21.15.31:0/2404337627 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20dc00bd20 con 0x7f20f00acdd0 2023-10-06T02:24:46.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:46.627+0000 7f20f6dff700 1 -- 172.21.15.31:0/2404337627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20f00acdd0 msgr2=0x7f20f00b14d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:46.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:46.627+0000 7f20f6dff700 1 --2- 172.21.15.31:0/2404337627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20f00acdd0 0x7f20f00b14d0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f20f006d940 tx=0x7f20dc009250 comp rx=0 tx=0).stop 2023-10-06T02:24:46.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:46.627+0000 7f20f6dff700 1 -- 172.21.15.31:0/2404337627 shutdown_connections 2023-10-06T02:24:46.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:46.627+0000 7f20f6dff700 1 --2- 172.21.15.31:0/2404337627 >> v2:172.21.15.62:3300/0 conn(0x7f20f00b1a10 0x7f20f0161350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:46.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:46.627+0000 7f20f6dff700 1 --2- 172.21.15.31:0/2404337627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20f00acdd0 0x7f20f00b14d0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:46.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:46.627+0000 7f20f6dff700 1 -- 172.21.15.31:0/2404337627 >> 172.21.15.31:0/2404337627 conn(0x7f20f006d170 msgr2=0x7f20f00cc400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:46.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:46.627+0000 7f20f6dff700 1 -- 172.21.15.31:0/2404337627 shutdown_connections 2023-10-06T02:24:46.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:46.627+0000 7f20f6dff700 1 -- 172.21.15.31:0/2404337627 wait complete. 2023-10-06T02:24:46.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:46.628+0000 7f20f6dff700 1 Processor -- start 2023-10-06T02:24:46.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:46.629+0000 7f20f6dff700 1 -- start start 2023-10-06T02:24:46.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:46.629+0000 7f20f6dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20f00acdd0 0x7f20f0159ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:46.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:46.629+0000 7f20f6dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20f00b1a10 0x7f20f00a5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:46.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:46.629+0000 7f20f6dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f20f01576d0 0x7f20f01530a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:46.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:46.629+0000 7f20f6dff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20f00a63d0 con 0x7f20f00b1a10 2023-10-06T02:24:46.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:46.629+0000 7f20f6dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20f00a71d0 con 0x7f20f00acdd0 2023-10-06T02:24:46.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:46.629+0000 7f20f6dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20f00a65c0 con 0x7f20f01576d0 2023-10-06T02:24:46.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:46.630+0000 7f20effff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20f00b1a10 0x7f20f00a5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:46.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:46.630+0000 7f20effff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20f00b1a10 0x7f20f00a5e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41176/0 (socket says 172.21.15.31:41176) 2023-10-06T02:24:46.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:46.630+0000 7f20effff700 1 -- 172.21.15.31:0/2202758528 learned_addr learned my addr 172.21.15.31:0/2202758528 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:46.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:46.630+0000 7f20f539c700 1 --2- 172.21.15.31:0/2202758528 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f20f01576d0 0x7f20f01530a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:46.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:46.630+0000 7f20f4b9b700 1 --2- 172.21.15.31:0/2202758528 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20f00acdd0 0x7f20f0159ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:46.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:46.630+0000 7f20effff700 1 -- 172.21.15.31:0/2202758528 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f20f01576d0 msgr2=0x7f20f01530a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:46.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:46.630+0000 7f20effff700 1 --2- 172.21.15.31:0/2202758528 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f20f01576d0 0x7f20f01530a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:46.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:46.630+0000 7f20effff700 1 -- 172.21.15.31:0/2202758528 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20f00acdd0 msgr2=0x7f20f0159ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:46.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:46.630+0000 7f20effff700 1 --2- 172.21.15.31:0/2202758528 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20f00acdd0 0x7f20f0159ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:46.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:46.630+0000 7f20effff700 1 -- 172.21.15.31:0/2202758528 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20dc0094f0 con 0x7f20f00b1a10 2023-10-06T02:24:46.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:46.630+0000 7f20effff700 1 --2- 172.21.15.31:0/2202758528 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20f00b1a10 0x7f20f00a5e90 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f20f006e1f0 tx=0x7f20dc016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:46.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:46.631+0000 7f20ecff9700 1 -- 172.21.15.31:0/2202758528 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20dc011de0 con 0x7f20f00b1a10 2023-10-06T02:24:46.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:46.631+0000 7f20ecff9700 1 -- 172.21.15.31:0/2202758528 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20dc020070 con 0x7f20f00b1a10 2023-10-06T02:24:46.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:46.631+0000 7f20ecff9700 1 -- 172.21.15.31:0/2202758528 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20dc01b6a0 con 0x7f20f00b1a10 2023-10-06T02:24:46.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:46.631+0000 7f20f6dff700 1 -- 172.21.15.31:0/2202758528 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20f00a68a0 con 0x7f20f00b1a10 2023-10-06T02:24:46.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:46.631+0000 7f20f6dff700 1 -- 172.21.15.31:0/2202758528 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20f00a6e60 con 0x7f20f00b1a10 2023-10-06T02:24:46.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:46.632+0000 7f20f6dff700 1 -- 172.21.15.31:0/2202758528 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20f00a7360 con 0x7f20f00b1a10 2023-10-06T02:24:46.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:46.633+0000 7f20ecff9700 1 -- 172.21.15.31:0/2202758528 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f20dc022030 con 0x7f20f00b1a10 2023-10-06T02:24:46.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:46.633+0000 7f20ecff9700 1 --2- 172.21.15.31:0/2202758528 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f20d40670e0 0x7f20d4069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:46.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:46.633+0000 7f20ecff9700 1 -- 172.21.15.31:0/2202758528 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f20dc086ac0 con 0x7f20f00b1a10 2023-10-06T02:24:46.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:46.634+0000 7f20f4b9b700 1 --2- 172.21.15.31:0/2202758528 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f20d40670e0 0x7f20d4069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:46.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:46.634+0000 7f20f4b9b700 1 --2- 172.21.15.31:0/2202758528 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f20d40670e0 0x7f20d4069590 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f20e4006fd0 tx=0x7f20e4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:46.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:46.638+0000 7f20ecff9700 1 -- 172.21.15.31:0/2202758528 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f20dc0538b0 con 0x7f20f00b1a10 2023-10-06T02:24:46.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:46.886+0000 7f20f6dff700 1 -- 172.21.15.31:0/2202758528 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f20f00a7360 con 0x7f20f00b1a10 2023-10-06T02:24:47.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.410+0000 7f20ecff9700 1 -- 172.21.15.31:0/2202758528 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v26) v1 ==== 122+0+0 (secure 0 0 0) 0x7f20dc056ed0 con 0x7f20f00b1a10 2023-10-06T02:24:47.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.413+0000 7f20f6dff700 1 -- 172.21.15.31:0/2202758528 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f20d40670e0 msgr2=0x7f20d4069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:47.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.413+0000 7f20f6dff700 1 --2- 172.21.15.31:0/2202758528 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f20d40670e0 0x7f20d4069590 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f20e4006fd0 tx=0x7f20e4008040 comp rx=0 tx=0).stop 2023-10-06T02:24:47.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.413+0000 7f20f6dff700 1 -- 172.21.15.31:0/2202758528 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20f00b1a10 msgr2=0x7f20f00a5e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:47.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.413+0000 7f20f6dff700 1 --2- 172.21.15.31:0/2202758528 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20f00b1a10 0x7f20f00a5e90 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f20f006e1f0 tx=0x7f20dc016000 comp rx=0 tx=0).stop 2023-10-06T02:24:47.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.413+0000 7f20f6dff700 1 -- 172.21.15.31:0/2202758528 shutdown_connections 2023-10-06T02:24:47.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.413+0000 7f20f6dff700 1 --2- 172.21.15.31:0/2202758528 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f20d40670e0 0x7f20d4069590 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:47.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.413+0000 7f20f6dff700 1 --2- 172.21.15.31:0/2202758528 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f20f01576d0 0x7f20f01530a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:47.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.413+0000 7f20f6dff700 1 --2- 172.21.15.31:0/2202758528 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20f00acdd0 0x7f20f0159ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:47.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.413+0000 7f20f6dff700 1 --2- 172.21.15.31:0/2202758528 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20f00b1a10 0x7f20f00a5e90 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:47.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.413+0000 7f20f6dff700 1 -- 172.21.15.31:0/2202758528 >> 172.21.15.31:0/2202758528 conn(0x7f20f006d170 msgr2=0x7f20f006aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:47.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.413+0000 7f20f6dff700 1 -- 172.21.15.31:0/2202758528 shutdown_connections 2023-10-06T02:24:47.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.413+0000 7f20f6dff700 1 -- 172.21.15.31:0/2202758528 wait complete. 2023-10-06T02:24:47.432 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:24:47.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.622+0000 7f89cd59d700 1 Processor -- start 2023-10-06T02:24:47.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.622+0000 7f89cd59d700 1 -- start start 2023-10-06T02:24:47.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.623+0000 7f89cd59d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f89c81596d0 0x7f89c806d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:47.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.623+0000 7f89cd59d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f89c8159aa0 0x7f89c80703a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:47.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.623+0000 7f89cd59d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89c806d3e0 con 0x7f89c80ca7b0 2023-10-06T02:24:47.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.623+0000 7f89cd59d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89c806c7d0 con 0x7f89c8159aa0 2023-10-06T02:24:47.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.623+0000 7f89cd59d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89c806cab0 con 0x7f89c81596d0 2023-10-06T02:24:47.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.624+0000 7f89c7fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f89c81596d0 0x7f89c806d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:47.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.624+0000 7f89c7fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f89c81596d0 0x7f89c806d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:56728/0 (socket says 172.21.15.31:56728) 2023-10-06T02:24:47.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.624+0000 7f89c7fff700 1 -- 172.21.15.31:0/2982442686 learned_addr learned my addr 172.21.15.31:0/2982442686 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:47.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.624+0000 7f89c77fe700 1 --2- 172.21.15.31:0/2982442686 >> v2:172.21.15.62:3300/0 conn(0x7f89c8159aa0 0x7f89c80703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:47.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.624+0000 7f89c7fff700 1 -- 172.21.15.31:0/2982442686 >> v2:172.21.15.62:3300/0 conn(0x7f89c8159aa0 msgr2=0x7f89c80703a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:47.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.624+0000 7f89c7fff700 1 --2- 172.21.15.31:0/2982442686 >> v2:172.21.15.62:3300/0 conn(0x7f89c8159aa0 0x7f89c80703a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:47.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.624+0000 7f89c7fff700 1 -- 172.21.15.31:0/2982442686 >> v1:172.21.15.62:6789/0 conn(0x7f89c80ca7b0 legacy=0x7f89c80cabe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:47.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.624+0000 7f89c7fff700 1 -- 172.21.15.31:0/2982442686 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89c806cd90 con 0x7f89c81596d0 2023-10-06T02:24:47.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.624+0000 7f89c77fe700 1 --2- 172.21.15.31:0/2982442686 >> v2:172.21.15.62:3300/0 conn(0x7f89c8159aa0 0x7f89c80703a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:24:47.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.624+0000 7f89c7fff700 1 --2- 172.21.15.31:0/2982442686 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f89c81596d0 0x7f89c806d870 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f89b80099b0 tx=0x7f89b8009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a17d5551d0e610e5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:47.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.625+0000 7f89c6ffd700 1 -- 172.21.15.31:0/2982442686 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b8009ea0 con 0x7f89c81596d0 2023-10-06T02:24:47.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.625+0000 7f89c6ffd700 1 -- 172.21.15.31:0/2982442686 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89b800b860 con 0x7f89c81596d0 2023-10-06T02:24:47.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.625+0000 7f89c6ffd700 1 -- 172.21.15.31:0/2982442686 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b800bc20 con 0x7f89c81596d0 2023-10-06T02:24:47.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.625+0000 7f89cd59d700 1 -- 172.21.15.31:0/2982442686 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f89c81596d0 msgr2=0x7f89c806d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:47.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.625+0000 7f89cd59d700 1 --2- 172.21.15.31:0/2982442686 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f89c81596d0 0x7f89c806d870 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f89b80099b0 tx=0x7f89b8009250 comp rx=0 tx=0).stop 2023-10-06T02:24:47.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.625+0000 7f89cd59d700 1 -- 172.21.15.31:0/2982442686 shutdown_connections 2023-10-06T02:24:47.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.625+0000 7f89cd59d700 1 --2- 172.21.15.31:0/2982442686 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f89c81596d0 0x7f89c806d870 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:47.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.625+0000 7f89cd59d700 1 --2- 172.21.15.31:0/2982442686 >> v2:172.21.15.62:3300/0 conn(0x7f89c8159aa0 0x7f89c80703a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:47.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.625+0000 7f89cd59d700 1 -- 172.21.15.31:0/2982442686 >> 172.21.15.31:0/2982442686 conn(0x7f89c806e720 msgr2=0x7f89c806eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:47.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.625+0000 7f89cd59d700 1 -- 172.21.15.31:0/2982442686 shutdown_connections 2023-10-06T02:24:47.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.626+0000 7f89cd59d700 1 -- 172.21.15.31:0/2982442686 wait complete. 2023-10-06T02:24:47.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.627+0000 7f89cd59d700 1 Processor -- start 2023-10-06T02:24:47.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.627+0000 7f89cd59d700 1 -- start start 2023-10-06T02:24:47.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.628+0000 7f89cd59d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89c80ca7b0 0x7f89c814b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:47.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.628+0000 7f89cd59d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89c81596d0 0x7f89c80c96e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:47.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.628+0000 7f89cd59d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f89c8159aa0 0x7f89c8157a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:47.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.628+0000 7f89cd59d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89c80c4a30 con 0x7f89c80ca7b0 2023-10-06T02:24:47.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.628+0000 7f89cd59d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89c80af7f0 con 0x7f89c81596d0 2023-10-06T02:24:47.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.628+0000 7f89cd59d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89c80c3e20 con 0x7f89c8159aa0 2023-10-06T02:24:47.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.628+0000 7f89c7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89c80ca7b0 0x7f89c814b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:47.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.628+0000 7f89c7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89c80ca7b0 0x7f89c814b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41182/0 (socket says 172.21.15.31:41182) 2023-10-06T02:24:47.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.628+0000 7f89c7fff700 1 -- 172.21.15.31:0/640404891 learned_addr learned my addr 172.21.15.31:0/640404891 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:47.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.628+0000 7f89ccd9c700 1 --2- 172.21.15.31:0/640404891 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f89c8159aa0 0x7f89c8157a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:47.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.628+0000 7f89c77fe700 1 --2- 172.21.15.31:0/640404891 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89c81596d0 0x7f89c80c96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:47.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.629+0000 7f89c7fff700 1 -- 172.21.15.31:0/640404891 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f89c8159aa0 msgr2=0x7f89c8157a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:47.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.629+0000 7f89c7fff700 1 --2- 172.21.15.31:0/640404891 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f89c8159aa0 0x7f89c8157a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:47.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.629+0000 7f89c7fff700 1 -- 172.21.15.31:0/640404891 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89c81596d0 msgr2=0x7f89c80c96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:47.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.629+0000 7f89c7fff700 1 --2- 172.21.15.31:0/640404891 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89c81596d0 0x7f89c80c96e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:47.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.629+0000 7f89c7fff700 1 -- 172.21.15.31:0/640404891 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89bc009420 con 0x7f89c80ca7b0 2023-10-06T02:24:47.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.629+0000 7f89ccd9c700 1 --2- 172.21.15.31:0/640404891 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f89c8159aa0 0x7f89c8157a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:24:47.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.629+0000 7f89c7fff700 1 --2- 172.21.15.31:0/640404891 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89c80ca7b0 0x7f89c814b690 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f89b8006fd0 tx=0x7f89b8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:47.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.629+0000 7f89affff700 1 -- 172.21.15.31:0/640404891 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b801f070 con 0x7f89c80ca7b0 2023-10-06T02:24:47.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.629+0000 7f89affff700 1 -- 172.21.15.31:0/640404891 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89b8000c90 con 0x7f89c80ca7b0 2023-10-06T02:24:47.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.629+0000 7f89cd59d700 1 -- 172.21.15.31:0/640404891 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89b80094f0 con 0x7f89c80ca7b0 2023-10-06T02:24:47.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.629+0000 7f89affff700 1 -- 172.21.15.31:0/640404891 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b8025420 con 0x7f89c80ca7b0 2023-10-06T02:24:47.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.629+0000 7f89cd59d700 1 -- 172.21.15.31:0/640404891 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89c80c43e0 con 0x7f89c80ca7b0 2023-10-06T02:24:47.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.630+0000 7f89cd59d700 1 -- 172.21.15.31:0/640404891 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89c804c3c0 con 0x7f89c80ca7b0 2023-10-06T02:24:47.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.631+0000 7f89affff700 1 -- 172.21.15.31:0/640404891 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f89b8023070 con 0x7f89c80ca7b0 2023-10-06T02:24:47.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.632+0000 7f89affff700 1 --2- 172.21.15.31:0/640404891 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f89b40670e0 0x7f89b4069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:47.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.632+0000 7f89affff700 1 -- 172.21.15.31:0/640404891 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f89b8085e10 con 0x7f89c80ca7b0 2023-10-06T02:24:47.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.632+0000 7f89c77fe700 1 --2- 172.21.15.31:0/640404891 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f89b40670e0 0x7f89b4069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:47.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.633+0000 7f89c77fe700 1 --2- 172.21.15.31:0/640404891 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f89b40670e0 0x7f89b4069590 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f89bc0093f0 tx=0x7f89bc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:47.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.636+0000 7f89affff700 1 -- 172.21.15.31:0/640404891 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f89b8089030 con 0x7f89c80ca7b0 2023-10-06T02:24:47.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.882+0000 7f89cd59d700 1 -- 172.21.15.31:0/640404891 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f89c80ca0c0 con 0x7f89c80ca7b0 2023-10-06T02:24:47.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.883+0000 7f89affff700 1 -- 172.21.15.31:0/640404891 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4594 (secure 0 0 0) 0x7f89b802a090 con 0x7f89c80ca7b0 2023-10-06T02:24:47.883 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:24:47.883 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4949,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/178547366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":178547366},{"type":"v1","addr":"172.21.15.62:6835","nonce":178547366}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":5278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/2578831517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":2578831517},{"type":"v1","addr":"172.21.15.31:6836","nonce":2578831517}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5287,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3502585737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":3502585737},{"type":"v1","addr":"172.21.15.31:6837","nonce":3502585737}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:24:43.872314+0000","modified":"2023-10-06T02:24:47.404607+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5290":{"gid":5290,"name":"b","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6834/2358776861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2358776861},{"type":"v1","addr":"172.21.15.62:6834","nonce":2358776861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T02:24:47.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.885+0000 7f89cd59d700 1 -- 172.21.15.31:0/640404891 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f89b40670e0 msgr2=0x7f89b4069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:47.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.885+0000 7f89cd59d700 1 --2- 172.21.15.31:0/640404891 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f89b40670e0 0x7f89b4069590 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f89bc0093f0 tx=0x7f89bc008040 comp rx=0 tx=0).stop 2023-10-06T02:24:47.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.885+0000 7f89cd59d700 1 -- 172.21.15.31:0/640404891 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89c80ca7b0 msgr2=0x7f89c814b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:47.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.885+0000 7f89cd59d700 1 --2- 172.21.15.31:0/640404891 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89c80ca7b0 0x7f89c814b690 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f89b8006fd0 tx=0x7f89b8017000 comp rx=0 tx=0).stop 2023-10-06T02:24:47.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.885+0000 7f89cd59d700 1 -- 172.21.15.31:0/640404891 shutdown_connections 2023-10-06T02:24:47.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.885+0000 7f89cd59d700 1 --2- 172.21.15.31:0/640404891 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f89b40670e0 0x7f89b4069590 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:47.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.885+0000 7f89cd59d700 1 --2- 172.21.15.31:0/640404891 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f89c8159aa0 0x7f89c8157a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:47.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.885+0000 7f89cd59d700 1 --2- 172.21.15.31:0/640404891 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89c80ca7b0 0x7f89c814b690 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:47.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.885+0000 7f89cd59d700 1 --2- 172.21.15.31:0/640404891 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89c81596d0 0x7f89c80c96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:47.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.885+0000 7f89cd59d700 1 -- 172.21.15.31:0/640404891 >> 172.21.15.31:0/640404891 conn(0x7f89c806e720 msgr2=0x7f89c8155cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:47.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.885+0000 7f89cd59d700 1 -- 172.21.15.31:0/640404891 shutdown_connections 2023-10-06T02:24:47.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:47.885+0000 7f89cd59d700 1 -- 172.21.15.31:0/640404891 wait complete. 2023-10-06T02:24:47.887 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 26 2023-10-06T02:24:47.903 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:24:48.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.089+0000 7f1400d9d700 1 Processor -- start 2023-10-06T02:24:48.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.089+0000 7f1400d9d700 1 -- start start 2023-10-06T02:24:48.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.090+0000 7f1400d9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f13fc13fb20 0x7f13fc15a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:48.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.090+0000 7f1400d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f13fc06aaa0 0x7f13fc0c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:48.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.090+0000 7f1400d9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13fc15ae70 con 0x7f13fc13fef0 2023-10-06T02:24:48.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.090+0000 7f1400d9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13fc06af60 con 0x7f13fc13fb20 2023-10-06T02:24:48.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.090+0000 7f1400d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13fc06b240 con 0x7f13fc06aaa0 2023-10-06T02:24:48.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.091+0000 7f13fb7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f13fc13fb20 0x7f13fc15a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:48.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.091+0000 7f13fb7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f13fc13fb20 0x7f13fc15a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53244/0 (socket says 172.21.15.31:53244) 2023-10-06T02:24:48.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.091+0000 7f13fb7fe700 1 -- 172.21.15.31:0/1690051785 learned_addr learned my addr 172.21.15.31:0/1690051785 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:48.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.091+0000 7f13fbfff700 1 --2- 172.21.15.31:0/1690051785 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f13fc06aaa0 0x7f13fc0c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:48.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.091+0000 7f13fb7fe700 1 -- 172.21.15.31:0/1690051785 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f13fc06aaa0 msgr2=0x7f13fc0c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:48.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.091+0000 7f13fb7fe700 1 --2- 172.21.15.31:0/1690051785 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f13fc06aaa0 0x7f13fc0c9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:48.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.091+0000 7f13fb7fe700 1 -- 172.21.15.31:0/1690051785 >> v1:172.21.15.62:6789/0 conn(0x7f13fc13fef0 legacy=0x7f13fc0c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:48.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.091+0000 7f13fb7fe700 1 -- 172.21.15.31:0/1690051785 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13fc06b520 con 0x7f13fc13fb20 2023-10-06T02:24:48.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.091+0000 7f13fb7fe700 1 --2- 172.21.15.31:0/1690051785 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13fc13fb20 0x7f13fc15a930 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f13ec0099b0 tx=0x7f13ec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=266dad853b7fe229 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:48.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.092+0000 7f13fa7fc700 1 -- 172.21.15.31:0/1690051785 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13ec009ea0 con 0x7f13fc13fb20 2023-10-06T02:24:48.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.092+0000 7f13fa7fc700 1 -- 172.21.15.31:0/1690051785 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13ec00b860 con 0x7f13fc13fb20 2023-10-06T02:24:48.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.092+0000 7f13fa7fc700 1 -- 172.21.15.31:0/1690051785 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13ec00bc00 con 0x7f13fc13fb20 2023-10-06T02:24:48.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.092+0000 7f1400d9d700 1 -- 172.21.15.31:0/1690051785 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13fc13fb20 msgr2=0x7f13fc15a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:48.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.092+0000 7f1400d9d700 1 --2- 172.21.15.31:0/1690051785 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13fc13fb20 0x7f13fc15a930 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f13ec0099b0 tx=0x7f13ec009250 comp rx=0 tx=0).stop 2023-10-06T02:24:48.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.092+0000 7f1400d9d700 1 -- 172.21.15.31:0/1690051785 shutdown_connections 2023-10-06T02:24:48.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.092+0000 7f1400d9d700 1 --2- 172.21.15.31:0/1690051785 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f13fc06aaa0 0x7f13fc0c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:48.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.092+0000 7f1400d9d700 1 --2- 172.21.15.31:0/1690051785 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13fc13fb20 0x7f13fc15a930 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:48.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.092+0000 7f1400d9d700 1 -- 172.21.15.31:0/1690051785 >> 172.21.15.31:0/1690051785 conn(0x7f13fc06d170 msgr2=0x7f13fc0cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:48.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.092+0000 7f1400d9d700 1 -- 172.21.15.31:0/1690051785 shutdown_connections 2023-10-06T02:24:48.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.092+0000 7f1400d9d700 1 -- 172.21.15.31:0/1690051785 wait complete. 2023-10-06T02:24:48.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.094+0000 7f1400d9d700 1 Processor -- start 2023-10-06T02:24:48.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.094+0000 7f1400d9d700 1 -- start start 2023-10-06T02:24:48.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.095+0000 7f1400d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13fc06aaa0 0x7f13fc156120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:48.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.095+0000 7f1400d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f13fc13fb20 0x7f13fc0ad740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:48.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.095+0000 7f1400d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13fc13fef0 0x7f13fc0adc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:48.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.095+0000 7f1400d9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13fc156d10 con 0x7f13fc13fef0 2023-10-06T02:24:48.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.095+0000 7f1400d9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13fc157030 con 0x7f13fc06aaa0 2023-10-06T02:24:48.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.095+0000 7f1400d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13fc157310 con 0x7f13fc13fb20 2023-10-06T02:24:48.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.095+0000 7f13fbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13fc13fef0 0x7f13fc0adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:48.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.095+0000 7f13fbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13fc13fef0 0x7f13fc0adc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53252/0 (socket says 172.21.15.31:53252) 2023-10-06T02:24:48.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.095+0000 7f13fbfff700 1 -- 172.21.15.31:0/42961227 learned_addr learned my addr 172.21.15.31:0/42961227 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:48.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.095+0000 7f13fb7fe700 1 --2- 172.21.15.31:0/42961227 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13fc06aaa0 0x7f13fc156120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:48.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.095+0000 7f13fbfff700 1 -- 172.21.15.31:0/42961227 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f13fc13fb20 msgr2=0x7f13fc0ad740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:48.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.095+0000 7f13faffd700 1 --2- 172.21.15.31:0/42961227 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f13fc13fb20 0x7f13fc0ad740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:48.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.096+0000 7f13fbfff700 1 --2- 172.21.15.31:0/42961227 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f13fc13fb20 0x7f13fc0ad740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:48.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.096+0000 7f13fbfff700 1 -- 172.21.15.31:0/42961227 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13fc06aaa0 msgr2=0x7f13fc156120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:48.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.096+0000 7f13fbfff700 1 --2- 172.21.15.31:0/42961227 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13fc06aaa0 0x7f13fc156120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:48.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.096+0000 7f13fbfff700 1 -- 172.21.15.31:0/42961227 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13ec0094f0 con 0x7f13fc13fef0 2023-10-06T02:24:48.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.096+0000 7f13fb7fe700 1 --2- 172.21.15.31:0/42961227 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13fc06aaa0 0x7f13fc156120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:24:48.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.096+0000 7f13faffd700 1 --2- 172.21.15.31:0/42961227 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f13fc13fb20 0x7f13fc0ad740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:24:48.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.096+0000 7f13fbfff700 1 --2- 172.21.15.31:0/42961227 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13fc13fef0 0x7f13fc0adc80 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f13f4009d70 tx=0x7f13f400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:48.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.096+0000 7f13e37fe700 1 -- 172.21.15.31:0/42961227 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f400cb50 con 0x7f13fc13fef0 2023-10-06T02:24:48.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.096+0000 7f13e37fe700 1 -- 172.21.15.31:0/42961227 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13f400f040 con 0x7f13fc13fef0 2023-10-06T02:24:48.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.096+0000 7f13e37fe700 1 -- 172.21.15.31:0/42961227 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f4014480 con 0x7f13fc13fef0 2023-10-06T02:24:48.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.096+0000 7f1400d9d700 1 -- 172.21.15.31:0/42961227 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13fc1575f0 con 0x7f13fc13fef0 2023-10-06T02:24:48.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.096+0000 7f1400d9d700 1 -- 172.21.15.31:0/42961227 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13fc157bb0 con 0x7f13fc13fef0 2023-10-06T02:24:48.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.097+0000 7f1400d9d700 1 -- 172.21.15.31:0/42961227 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13fc157dd0 con 0x7f13fc13fef0 2023-10-06T02:24:48.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.098+0000 7f13e37fe700 1 -- 172.21.15.31:0/42961227 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f13f4014830 con 0x7f13fc13fef0 2023-10-06T02:24:48.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.099+0000 7f13e37fe700 1 --2- 172.21.15.31:0/42961227 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f13e80670e0 0x7f13e8069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:48.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.099+0000 7f13e37fe700 1 -- 172.21.15.31:0/42961227 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f13f4085770 con 0x7f13fc13fef0 2023-10-06T02:24:48.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.099+0000 7f13fb7fe700 1 --2- 172.21.15.31:0/42961227 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f13e80670e0 0x7f13e8069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:48.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.100+0000 7f13fb7fe700 1 --2- 172.21.15.31:0/42961227 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f13e80670e0 0x7f13e8069590 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f13ec0099b0 tx=0x7f13ec017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:48.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.103+0000 7f13e37fe700 1 -- 172.21.15.31:0/42961227 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f13f4052450 con 0x7f13fc13fef0 2023-10-06T02:24:48.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.311+0000 7f1400d9d700 1 -- 172.21.15.31:0/42961227 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f13fc04c3c0 con 0x7f13fc13fef0 2023-10-06T02:24:48.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.312+0000 7f13e37fe700 1 -- 172.21.15.31:0/42961227 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+16818 (secure 0 0 0) 0x7f13f4055a70 con 0x7f13fc13fef0 2023-10-06T02:24:48.313 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:24:48.313 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":47,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:24:45.389692+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:24:42.035212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-10-06T02:24:42.849819+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:24:48.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.315+0000 7f1400d9d700 1 -- 172.21.15.31:0/42961227 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f13e80670e0 msgr2=0x7f13e8069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:48.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.315+0000 7f1400d9d700 1 --2- 172.21.15.31:0/42961227 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f13e80670e0 0x7f13e8069590 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f13ec0099b0 tx=0x7f13ec017000 comp rx=0 tx=0).stop 2023-10-06T02:24:48.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.315+0000 7f1400d9d700 1 -- 172.21.15.31:0/42961227 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13fc13fef0 msgr2=0x7f13fc0adc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:48.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.315+0000 7f1400d9d700 1 --2- 172.21.15.31:0/42961227 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13fc13fef0 0x7f13fc0adc80 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f13f4009d70 tx=0x7f13f400b040 comp rx=0 tx=0).stop 2023-10-06T02:24:48.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.315+0000 7f1400d9d700 1 -- 172.21.15.31:0/42961227 shutdown_connections 2023-10-06T02:24:48.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.315+0000 7f1400d9d700 1 --2- 172.21.15.31:0/42961227 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13fc13fef0 0x7f13fc0adc80 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:48.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.315+0000 7f1400d9d700 1 --2- 172.21.15.31:0/42961227 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13fc06aaa0 0x7f13fc156120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:48.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.315+0000 7f1400d9d700 1 --2- 172.21.15.31:0/42961227 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f13e80670e0 0x7f13e8069590 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:48.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.315+0000 7f1400d9d700 1 --2- 172.21.15.31:0/42961227 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f13fc13fb20 0x7f13fc0ad740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:48.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.315+0000 7f1400d9d700 1 -- 172.21.15.31:0/42961227 >> 172.21.15.31:0/42961227 conn(0x7f13fc06d170 msgr2=0x7f13fc0af490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:48.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.315+0000 7f1400d9d700 1 -- 172.21.15.31:0/42961227 shutdown_connections 2023-10-06T02:24:48.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.315+0000 7f1400d9d700 1 -- 172.21.15.31:0/42961227 wait complete. 2023-10-06T02:24:48.333 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T02:24:48.333 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T02:24:48.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.520+0000 7f645b59e700 1 Processor -- start 2023-10-06T02:24:48.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.520+0000 7f645b59e700 1 -- start start 2023-10-06T02:24:48.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.521+0000 7f645b59e700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f645c142c00 0x7f645c06b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:48.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.521+0000 7f645b59e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f645c0c7f90 0x7f645c06c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:48.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.521+0000 7f645b59e700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f645c0ca480 con 0x7f645c06bd20 2023-10-06T02:24:48.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.521+0000 7f645b59e700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f645c15e240 con 0x7f645c0c7f90 2023-10-06T02:24:48.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.521+0000 7f645b59e700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f645c06b350 con 0x7f645c142c00 2023-10-06T02:24:48.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.521+0000 7f645ad9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f645c0c7f90 0x7f645c06c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:48.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.521+0000 7f645ad9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f645c0c7f90 0x7f645c06c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53258/0 (socket says 172.21.15.31:53258) 2023-10-06T02:24:48.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.521+0000 7f645ad9d700 1 -- 172.21.15.31:0/3377298311 learned_addr learned my addr 172.21.15.31:0/3377298311 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:48.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.522+0000 7f645a59c700 1 --2- 172.21.15.31:0/3377298311 >> v2:172.21.15.62:3300/0 conn(0x7f645c142c00 0x7f645c06b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:48.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.522+0000 7f645ad9d700 1 -- 172.21.15.31:0/3377298311 >> v2:172.21.15.62:3300/0 conn(0x7f645c142c00 msgr2=0x7f645c06b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:48.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.522+0000 7f645ad9d700 1 --2- 172.21.15.31:0/3377298311 >> v2:172.21.15.62:3300/0 conn(0x7f645c142c00 0x7f645c06b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:48.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.522+0000 7f645ad9d700 1 -- 172.21.15.31:0/3377298311 >> v1:172.21.15.62:6789/0 conn(0x7f645c06bd20 legacy=0x7f645c0c7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:48.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.522+0000 7f645ad9d700 1 -- 172.21.15.31:0/3377298311 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f645c15df60 con 0x7f645c0c7f90 2023-10-06T02:24:48.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.522+0000 7f645ad9d700 1 --2- 172.21.15.31:0/3377298311 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f645c0c7f90 0x7f645c06c6a0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f645c06f490 tx=0x7f6454009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcef457e4dbf941 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:48.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.522+0000 7f645959a700 1 -- 172.21.15.31:0/3377298311 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f645400e070 con 0x7f645c0c7f90 2023-10-06T02:24:48.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.522+0000 7f645959a700 1 -- 172.21.15.31:0/3377298311 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f645400ba50 con 0x7f645c0c7f90 2023-10-06T02:24:48.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.522+0000 7f645959a700 1 -- 172.21.15.31:0/3377298311 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64540092e0 con 0x7f645c0c7f90 2023-10-06T02:24:48.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.523+0000 7f645b59e700 1 -- 172.21.15.31:0/3377298311 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f645c0c7f90 msgr2=0x7f645c06c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:48.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.523+0000 7f645b59e700 1 --2- 172.21.15.31:0/3377298311 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f645c0c7f90 0x7f645c06c6a0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f645c06f490 tx=0x7f6454009840 comp rx=0 tx=0).stop 2023-10-06T02:24:48.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.523+0000 7f645b59e700 1 -- 172.21.15.31:0/3377298311 shutdown_connections 2023-10-06T02:24:48.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.523+0000 7f645b59e700 1 --2- 172.21.15.31:0/3377298311 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f645c0c7f90 0x7f645c06c6a0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:48.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.523+0000 7f645b59e700 1 --2- 172.21.15.31:0/3377298311 >> v2:172.21.15.62:3300/0 conn(0x7f645c142c00 0x7f645c06b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:48.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.523+0000 7f645b59e700 1 -- 172.21.15.31:0/3377298311 >> 172.21.15.31:0/3377298311 conn(0x7f645c0675a0 msgr2=0x7f645c0679a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:48.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.523+0000 7f645b59e700 1 -- 172.21.15.31:0/3377298311 shutdown_connections 2023-10-06T02:24:48.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.523+0000 7f645b59e700 1 -- 172.21.15.31:0/3377298311 wait complete. 2023-10-06T02:24:48.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.525+0000 7f645b59e700 1 Processor -- start 2023-10-06T02:24:48.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.525+0000 7f645b59e700 1 -- start start 2023-10-06T02:24:48.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.526+0000 7f645b59e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f645c06bd20 0x7f645c158f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:48.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.526+0000 7f645b59e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f645c0c7f90 0x7f645c1698e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:48.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.526+0000 7f645b59e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f645c142c00 0x7f645c0c1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:48.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.526+0000 7f645b59e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f645c0c7260 con 0x7f645c142c00 2023-10-06T02:24:48.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.526+0000 7f645b59e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f645c13f050 con 0x7f645c0c7f90 2023-10-06T02:24:48.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.526+0000 7f645b59e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f645c0c51f0 con 0x7f645c06bd20 2023-10-06T02:24:48.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.526+0000 7f645ad9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f645c142c00 0x7f645c0c1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:48.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.526+0000 7f645ad9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f645c142c00 0x7f645c0c1fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53270/0 (socket says 172.21.15.31:53270) 2023-10-06T02:24:48.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.526+0000 7f645ad9d700 1 -- 172.21.15.31:0/1880859193 learned_addr learned my addr 172.21.15.31:0/1880859193 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:48.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.526+0000 7f6459d9b700 1 --2- 172.21.15.31:0/1880859193 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f645c0c7f90 0x7f645c1698e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:48.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.526+0000 7f645a59c700 1 --2- 172.21.15.31:0/1880859193 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f645c06bd20 0x7f645c158f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:48.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.526+0000 7f645ad9d700 1 -- 172.21.15.31:0/1880859193 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f645c06bd20 msgr2=0x7f645c158f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:48.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.526+0000 7f645ad9d700 1 --2- 172.21.15.31:0/1880859193 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f645c06bd20 0x7f645c158f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:48.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.526+0000 7f645ad9d700 1 -- 172.21.15.31:0/1880859193 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f645c0c7f90 msgr2=0x7f645c1698e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:48.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.526+0000 7f645ad9d700 1 --2- 172.21.15.31:0/1880859193 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f645c0c7f90 0x7f645c1698e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:48.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.526+0000 7f645ad9d700 1 -- 172.21.15.31:0/1880859193 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6454009ae0 con 0x7f645c142c00 2023-10-06T02:24:48.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.527+0000 7f645ad9d700 1 --2- 172.21.15.31:0/1880859193 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f645c142c00 0x7f645c0c1fe0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f6454009fd0 tx=0x7f6454018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:48.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.527+0000 7f644a7fc700 1 -- 172.21.15.31:0/1880859193 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6454007e60 con 0x7f645c142c00 2023-10-06T02:24:48.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.527+0000 7f644a7fc700 1 -- 172.21.15.31:0/1880859193 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6454007000 con 0x7f645c142c00 2023-10-06T02:24:48.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.527+0000 7f644a7fc700 1 -- 172.21.15.31:0/1880859193 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f645401d670 con 0x7f645c142c00 2023-10-06T02:24:48.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.527+0000 7f645b59e700 1 -- 172.21.15.31:0/1880859193 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f645c0c54d0 con 0x7f645c142c00 2023-10-06T02:24:48.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.527+0000 7f645b59e700 1 -- 172.21.15.31:0/1880859193 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f645c0c5a90 con 0x7f645c142c00 2023-10-06T02:24:48.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.528+0000 7f645b59e700 1 -- 172.21.15.31:0/1880859193 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f645c0c5cd0 con 0x7f645c142c00 2023-10-06T02:24:48.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.529+0000 7f644a7fc700 1 -- 172.21.15.31:0/1880859193 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f6454003050 con 0x7f645c142c00 2023-10-06T02:24:48.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.530+0000 7f644a7fc700 1 --2- 172.21.15.31:0/1880859193 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f643c0670e0 0x7f643c069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:48.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.530+0000 7f644a7fc700 1 -- 172.21.15.31:0/1880859193 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f64540868e0 con 0x7f645c142c00 2023-10-06T02:24:48.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.530+0000 7f645a59c700 1 --2- 172.21.15.31:0/1880859193 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f643c0670e0 0x7f643c069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:48.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.531+0000 7f645a59c700 1 --2- 172.21.15.31:0/1880859193 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f643c0670e0 0x7f643c069590 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f644c006fd0 tx=0x7f644c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:48.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.534+0000 7f644a7fc700 1 -- 172.21.15.31:0/1880859193 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f645c0c5cd0 con 0x7f645c142c00 2023-10-06T02:24:48.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.733+0000 7f645b59e700 1 -- 172.21.15.31:0/1880859193 --> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f645c0c5cd0 con 0x7f643c0670e0 2023-10-06T02:24:48.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.743+0000 7f644a7fc700 1 -- 172.21.15.31:0/1880859193 <== mgr.4107 v2:172.21.15.31:6832/85292 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+300385 (secure 0 0 0) 0x7f645c0c5cd0 con 0x7f643c0670e0 2023-10-06T02:24:48.747 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:24:48.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.747+0000 7f645b59e700 1 -- 172.21.15.31:0/1880859193 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f643c0670e0 msgr2=0x7f643c069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:48.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.747+0000 7f645b59e700 1 --2- 172.21.15.31:0/1880859193 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f643c0670e0 0x7f643c069590 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f644c006fd0 tx=0x7f644c008040 comp rx=0 tx=0).stop 2023-10-06T02:24:48.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.747+0000 7f645b59e700 1 -- 172.21.15.31:0/1880859193 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f645c142c00 msgr2=0x7f645c0c1fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:48.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.747+0000 7f645b59e700 1 --2- 172.21.15.31:0/1880859193 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f645c142c00 0x7f645c0c1fe0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f6454009fd0 tx=0x7f6454018000 comp rx=0 tx=0).stop 2023-10-06T02:24:48.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.747+0000 7f645b59e700 1 -- 172.21.15.31:0/1880859193 shutdown_connections 2023-10-06T02:24:48.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.747+0000 7f645b59e700 1 --2- 172.21.15.31:0/1880859193 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f645c142c00 0x7f645c0c1fe0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:48.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.747+0000 7f645b59e700 1 --2- 172.21.15.31:0/1880859193 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f643c0670e0 0x7f643c069590 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:48.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.747+0000 7f645b59e700 1 --2- 172.21.15.31:0/1880859193 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f645c06bd20 0x7f645c158f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:48.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.747+0000 7f645b59e700 1 --2- 172.21.15.31:0/1880859193 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f645c0c7f90 0x7f645c1698e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:48.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.747+0000 7f645b59e700 1 -- 172.21.15.31:0/1880859193 >> 172.21.15.31:0/1880859193 conn(0x7f645c0675a0 msgr2=0x7f645c141d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:48.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.748+0000 7f645b59e700 1 -- 172.21.15.31:0/1880859193 shutdown_connections 2023-10-06T02:24:48.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.748+0000 7f645b59e700 1 -- 172.21.15.31:0/1880859193 wait complete. 2023-10-06T02:24:48.751 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-10-06T02:24:48.765 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":97,"stamp":"2023-10-06T02:24:47.136320+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":28,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2383472,"kb_used_data":5168,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752591248,"statfs":{"total":773094113280,"available":770653437952,"internally_reserved":0,"allocated":5292032,"data_stored":2893924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.766950"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:24:43.369693+0000","last_change":"2023-10-06T02:24:43.369693+0000","last_active":"2023-10-06T02:24:43.369693+0000","last_peered":"2023-10-06T02:24:43.369693+0000","last_clean":"2023-10-06T02:24:43.369693+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:43.369693+0000","last_undegraded":"2023-10-06T02:24:43.369693+0000","last_fullsized":"2023-10-06T02:24:43.369693+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:24:43.369693+0000","last_change":"2023-10-06T02:24:43.369693+0000","last_active":"2023-10-06T02:24:43.369693+0000","last_peered":"2023-10-06T02:24:43.369693+0000","last_clean":"2023-10-06T02:24:43.369693+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:43.369693+0000","last_undegraded":"2023-10-06T02:24:43.369693+0000","last_fullsized":"2023-10-06T02:24:43.369693+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:24:43.369693+0000","last_change":"2023-10-06T02:24:43.369693+0000","last_active":"2023-10-06T02:24:43.369693+0000","last_peered":"2023-10-06T02:24:43.369693+0000","last_clean":"2023-10-06T02:24:43.369693+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:43.369693+0000","last_undegraded":"2023-10-06T02:24:43.369693+0000","last_fullsized":"2023-10-06T02:24:43.369693+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:24:43.369693+0000","last_change":"2023-10-06T02:24:43.369693+0000","last_active":"2023-10-06T02:24:43.369693+0000","last_peered":"2023-10-06T02:24:43.369693+0000","last_clean":"2023-10-06T02:24:43.369693+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:43.369693+0000","last_undegraded":"2023-10-06T02:24:43.369693+0000","last_fullsized":"2023-10-06T02:24:43.369693+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:24:43.369693+0000","last_change":"2023-10-06T02:24:43.369693+0000","last_active":"2023-10-06T02:24:43.369693+0000","last_peered":"2023-10-06T02:24:43.369693+0000","last_clean":"2023-10-06T02:24:43.369693+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:43.369693+0000","last_undegraded":"2023-10-06T02:24:43.369693+0000","last_fullsized":"2023-10-06T02:24:43.369693+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:24:43.369693+0000","last_change":"2023-10-06T02:24:43.369693+0000","last_active":"2023-10-06T02:24:43.369693+0000","last_peered":"2023-10-06T02:24:43.369693+0000","last_clean":"2023-10-06T02:24:43.369693+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:43.369693+0000","last_undegraded":"2023-10-06T02:24:43.369693+0000","last_fullsized":"2023-10-06T02:24:43.369693+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:24:43.369693+0000","last_change":"2023-10-06T02:24:43.369693+0000","last_active":"2023-10-06T02:24:43.369693+0000","last_peered":"2023-10-06T02:24:43.369693+0000","last_clean":"2023-10-06T02:24:43.369693+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:43.369693+0000","last_undegraded":"2023-10-06T02:24:43.369693+0000","last_fullsized":"2023-10-06T02:24:43.369693+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:24:43.369693+0000","last_change":"2023-10-06T02:24:43.369693+0000","last_active":"2023-10-06T02:24:43.369693+0000","last_peered":"2023-10-06T02:24:43.369693+0000","last_clean":"2023-10-06T02:24:43.369693+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:43.369693+0000","last_undegraded":"2023-10-06T02:24:43.369693+0000","last_fullsized":"2023-10-06T02:24:43.369693+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:24:43.369693+0000","last_change":"2023-10-06T02:24:43.369693+0000","last_active":"2023-10-06T02:24:43.369693+0000","last_peered":"2023-10-06T02:24:43.369693+0000","last_clean":"2023-10-06T02:24:43.369693+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:43.369693+0000","last_undegraded":"2023-10-06T02:24:43.369693+0000","last_fullsized":"2023-10-06T02:24:43.369693+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:24:43.369693+0000","last_change":"2023-10-06T02:24:43.369693+0000","last_active":"2023-10-06T02:24:43.369693+0000","last_peered":"2023-10-06T02:24:43.369693+0000","last_clean":"2023-10-06T02:24:43.369693+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:43.369693+0000","last_undegraded":"2023-10-06T02:24:43.369693+0000","last_fullsized":"2023-10-06T02:24:43.369693+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:24:43.369693+0000","last_change":"2023-10-06T02:24:43.369693+0000","last_active":"2023-10-06T02:24:43.369693+0000","last_peered":"2023-10-06T02:24:43.369693+0000","last_clean":"2023-10-06T02:24:43.369693+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:43.369693+0000","last_undegraded":"2023-10-06T02:24:43.369693+0000","last_fullsized":"2023-10-06T02:24:43.369693+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:24:43.369693+0000","last_change":"2023-10-06T02:24:43.369693+0000","last_active":"2023-10-06T02:24:43.369693+0000","last_peered":"2023-10-06T02:24:43.369693+0000","last_clean":"2023-10-06T02:24:43.369693+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:43.369693+0000","last_undegraded":"2023-10-06T02:24:43.369693+0000","last_fullsized":"2023-10-06T02:24:43.369693+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:24:43.369693+0000","last_change":"2023-10-06T02:24:43.369693+0000","last_active":"2023-10-06T02:24:43.369693+0000","last_peered":"2023-10-06T02:24:43.369693+0000","last_clean":"2023-10-06T02:24:43.369693+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:43.369693+0000","last_undegraded":"2023-10-06T02:24:43.369693+0000","last_fullsized":"2023-10-06T02:24:43.369693+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:24:43.369693+0000","last_change":"2023-10-06T02:24:43.369693+0000","last_active":"2023-10-06T02:24:43.369693+0000","last_peered":"2023-10-06T02:24:43.369693+0000","last_clean":"2023-10-06T02:24:43.369693+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:43.369693+0000","last_undegraded":"2023-10-06T02:24:43.369693+0000","last_fullsized":"2023-10-06T02:24:43.369693+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:24:43.369693+0000","last_change":"2023-10-06T02:24:43.369693+0000","last_active":"2023-10-06T02:24:43.369693+0000","last_peered":"2023-10-06T02:24:43.369693+0000","last_clean":"2023-10-06T02:24:43.369693+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:43.369693+0000","last_undegraded":"2023-10-06T02:24:43.369693+0000","last_fullsized":"2023-10-06T02:24:43.369693+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:24:43.369693+0000","last_change":"2023-10-06T02:24:43.369693+0000","last_active":"2023-10-06T02:24:43.369693+0000","last_peered":"2023-10-06T02:24:43.369693+0000","last_clean":"2023-10-06T02:24:43.369693+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:43.369693+0000","last_undegraded":"2023-10-06T02:24:43.369693+0000","last_fullsized":"2023-10-06T02:24:43.369693+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:24:43.369693+0000","last_change":"2023-10-06T02:24:43.369693+0000","last_active":"2023-10-06T02:24:43.369693+0000","last_peered":"2023-10-06T02:24:43.369693+0000","last_clean":"2023-10-06T02:24:43.369693+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:43.369693+0000","last_undegraded":"2023-10-06T02:24:43.369693+0000","last_fullsized":"2023-10-06T02:24:43.369693+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:24:43.369693+0000","last_change":"2023-10-06T02:24:43.369693+0000","last_active":"2023-10-06T02:24:43.369693+0000","last_peered":"2023-10-06T02:24:43.369693+0000","last_clean":"2023-10-06T02:24:43.369693+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:43.369693+0000","last_undegraded":"2023-10-06T02:24:43.369693+0000","last_fullsized":"2023-10-06T02:24:43.369693+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:24:43.369693+0000","last_change":"2023-10-06T02:24:43.369693+0000","last_active":"2023-10-06T02:24:43.369693+0000","last_peered":"2023-10-06T02:24:43.369693+0000","last_clean":"2023-10-06T02:24:43.369693+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:43.369693+0000","last_undegraded":"2023-10-06T02:24:43.369693+0000","last_fullsized":"2023-10-06T02:24:43.369693+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:24:43.369693+0000","last_change":"2023-10-06T02:24:43.369693+0000","last_active":"2023-10-06T02:24:43.369693+0000","last_peered":"2023-10-06T02:24:43.369693+0000","last_clean":"2023-10-06T02:24:43.369693+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:43.369693+0000","last_undegraded":"2023-10-06T02:24:43.369693+0000","last_fullsized":"2023-10-06T02:24:43.369693+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:24:43.369693+0000","last_change":"2023-10-06T02:24:43.369693+0000","last_active":"2023-10-06T02:24:43.369693+0000","last_peered":"2023-10-06T02:24:43.369693+0000","last_clean":"2023-10-06T02:24:43.369693+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:43.369693+0000","last_undegraded":"2023-10-06T02:24:43.369693+0000","last_fullsized":"2023-10-06T02:24:43.369693+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:24:43.369693+0000","last_change":"2023-10-06T02:24:43.369693+0000","last_active":"2023-10-06T02:24:43.369693+0000","last_peered":"2023-10-06T02:24:43.369693+0000","last_clean":"2023-10-06T02:24:43.369693+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:43.369693+0000","last_undegraded":"2023-10-06T02:24:43.369693+0000","last_fullsized":"2023-10-06T02:24:43.369693+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:24:43.369693+0000","last_change":"2023-10-06T02:24:43.369693+0000","last_active":"2023-10-06T02:24:43.369693+0000","last_peered":"2023-10-06T02:24:43.369693+0000","last_clean":"2023-10-06T02:24:43.369693+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:43.369693+0000","last_undegraded":"2023-10-06T02:24:43.369693+0000","last_fullsized":"2023-10-06T02:24:43.369693+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:24:43.369693+0000","last_change":"2023-10-06T02:24:43.369693+0000","last_active":"2023-10-06T02:24:43.369693+0000","last_peered":"2023-10-06T02:24:43.369693+0000","last_clean":"2023-10-06T02:24:43.369693+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:43.369693+0000","last_undegraded":"2023-10-06T02:24:43.369693+0000","last_fullsized":"2023-10-06T02:24:43.369693+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:24:43.369693+0000","last_change":"2023-10-06T02:24:43.369693+0000","last_active":"2023-10-06T02:24:43.369693+0000","last_peered":"2023-10-06T02:24:43.369693+0000","last_clean":"2023-10-06T02:24:43.369693+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:43.369693+0000","last_undegraded":"2023-10-06T02:24:43.369693+0000","last_fullsized":"2023-10-06T02:24:43.369693+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:24:43.369693+0000","last_change":"2023-10-06T02:24:43.369693+0000","last_active":"2023-10-06T02:24:43.369693+0000","last_peered":"2023-10-06T02:24:43.369693+0000","last_clean":"2023-10-06T02:24:43.369693+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:43.369693+0000","last_undegraded":"2023-10-06T02:24:43.369693+0000","last_fullsized":"2023-10-06T02:24:43.369693+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:24:43.369693+0000","last_change":"2023-10-06T02:24:43.369693+0000","last_active":"2023-10-06T02:24:43.369693+0000","last_peered":"2023-10-06T02:24:43.369693+0000","last_clean":"2023-10-06T02:24:43.369693+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:43.369693+0000","last_undegraded":"2023-10-06T02:24:43.369693+0000","last_fullsized":"2023-10-06T02:24:43.369693+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:24:43.369693+0000","last_change":"2023-10-06T02:24:43.369693+0000","last_active":"2023-10-06T02:24:43.369693+0000","last_peered":"2023-10-06T02:24:43.369693+0000","last_clean":"2023-10-06T02:24:43.369693+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:43.369693+0000","last_undegraded":"2023-10-06T02:24:43.369693+0000","last_fullsized":"2023-10-06T02:24:43.369693+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:24:43.369693+0000","last_change":"2023-10-06T02:24:43.369693+0000","last_active":"2023-10-06T02:24:43.369693+0000","last_peered":"2023-10-06T02:24:43.369693+0000","last_clean":"2023-10-06T02:24:43.369693+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:43.369693+0000","last_undegraded":"2023-10-06T02:24:43.369693+0000","last_fullsized":"2023-10-06T02:24:43.369693+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:24:43.369693+0000","last_change":"2023-10-06T02:24:43.369693+0000","last_active":"2023-10-06T02:24:43.369693+0000","last_peered":"2023-10-06T02:24:43.369693+0000","last_clean":"2023-10-06T02:24:43.369693+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:43.369693+0000","last_undegraded":"2023-10-06T02:24:43.369693+0000","last_fullsized":"2023-10-06T02:24:43.369693+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:24:43.369693+0000","last_change":"2023-10-06T02:24:43.369693+0000","last_active":"2023-10-06T02:24:43.369693+0000","last_peered":"2023-10-06T02:24:43.369693+0000","last_clean":"2023-10-06T02:24:43.369693+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:43.369693+0000","last_undegraded":"2023-10-06T02:24:43.369693+0000","last_fullsized":"2023-10-06T02:24:43.369693+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:24:43.369693+0000","last_change":"2023-10-06T02:24:43.369693+0000","last_active":"2023-10-06T02:24:43.369693+0000","last_peered":"2023-10-06T02:24:43.369693+0000","last_clean":"2023-10-06T02:24:43.369693+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:43.369693+0000","last_undegraded":"2023-10-06T02:24:43.369693+0000","last_fullsized":"2023-10-06T02:24:43.369693+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:24:43.369693+0000","last_change":"2023-10-06T02:24:43.369693+0000","last_active":"2023-10-06T02:24:43.369693+0000","last_peered":"2023-10-06T02:24:43.369693+0000","last_clean":"2023-10-06T02:24:43.369693+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:43.369693+0000","last_undegraded":"2023-10-06T02:24:43.369693+0000","last_fullsized":"2023-10-06T02:24:43.369693+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:24:42.362895+0000","last_change":"2023-10-06T02:24:42.373146+0000","last_active":"2023-10-06T02:24:42.362895+0000","last_peered":"2023-10-06T02:24:42.362895+0000","last_clean":"2023-10-06T02:24:42.362895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:42.362895+0000","last_undegraded":"2023-10-06T02:24:42.362895+0000","last_fullsized":"2023-10-06T02:24:42.362895+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_clean_scrub_stamp":"2023-10-06T02:24:42.362895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:20:22.610577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:24:43.369693+0000","last_change":"2023-10-06T02:24:43.369693+0000","last_active":"2023-10-06T02:24:43.369693+0000","last_peered":"2023-10-06T02:24:43.369693+0000","last_clean":"2023-10-06T02:24:43.369693+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:43.369693+0000","last_undegraded":"2023-10-06T02:24:43.369693+0000","last_fullsized":"2023-10-06T02:24:43.369693+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:24:42.362895+0000","last_change":"2023-10-06T02:24:42.378709+0000","last_active":"2023-10-06T02:24:42.362895+0000","last_peered":"2023-10-06T02:24:42.362895+0000","last_clean":"2023-10-06T02:24:42.362895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:42.362895+0000","last_undegraded":"2023-10-06T02:24:42.362895+0000","last_fullsized":"2023-10-06T02:24:42.362895+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_clean_scrub_stamp":"2023-10-06T02:24:42.362895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:13:30.476079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:24:43.369693+0000","last_change":"2023-10-06T02:24:43.369693+0000","last_active":"2023-10-06T02:24:43.369693+0000","last_peered":"2023-10-06T02:24:43.369693+0000","last_clean":"2023-10-06T02:24:43.369693+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:43.369693+0000","last_undegraded":"2023-10-06T02:24:43.369693+0000","last_fullsized":"2023-10-06T02:24:43.369693+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:24:42.362895+0000","last_change":"2023-10-06T02:24:42.379271+0000","last_active":"2023-10-06T02:24:42.362895+0000","last_peered":"2023-10-06T02:24:42.362895+0000","last_clean":"2023-10-06T02:24:42.362895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:42.362895+0000","last_undegraded":"2023-10-06T02:24:42.362895+0000","last_fullsized":"2023-10-06T02:24:42.362895+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_clean_scrub_stamp":"2023-10-06T02:24:42.362895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:02:43.874000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:24:43.369693+0000","last_change":"2023-10-06T02:24:43.369693+0000","last_active":"2023-10-06T02:24:43.369693+0000","last_peered":"2023-10-06T02:24:43.369693+0000","last_clean":"2023-10-06T02:24:43.369693+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:43.369693+0000","last_undegraded":"2023-10-06T02:24:43.369693+0000","last_fullsized":"2023-10-06T02:24:43.369693+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:24:42.362895+0000","last_change":"2023-10-06T02:24:42.372532+0000","last_active":"2023-10-06T02:24:42.362895+0000","last_peered":"2023-10-06T02:24:42.362895+0000","last_clean":"2023-10-06T02:24:42.362895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:42.362895+0000","last_undegraded":"2023-10-06T02:24:42.362895+0000","last_fullsized":"2023-10-06T02:24:42.362895+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_clean_scrub_stamp":"2023-10-06T02:24:42.362895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:50:17.948474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:24:43.369693+0000","last_change":"2023-10-06T02:24:43.369693+0000","last_active":"2023-10-06T02:24:43.369693+0000","last_peered":"2023-10-06T02:24:43.369693+0000","last_clean":"2023-10-06T02:24:43.369693+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:43.369693+0000","last_undegraded":"2023-10-06T02:24:43.369693+0000","last_fullsized":"2023-10-06T02:24:43.369693+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:24:42.362895+0000","last_change":"2023-10-06T02:24:42.381378+0000","last_active":"2023-10-06T02:24:42.362895+0000","last_peered":"2023-10-06T02:24:42.362895+0000","last_clean":"2023-10-06T02:24:42.362895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:42.362895+0000","last_undegraded":"2023-10-06T02:24:42.362895+0000","last_fullsized":"2023-10-06T02:24:42.362895+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_clean_scrub_stamp":"2023-10-06T02:24:42.362895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:05:52.756921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:24:43.369693+0000","last_change":"2023-10-06T02:24:43.369693+0000","last_active":"2023-10-06T02:24:43.369693+0000","last_peered":"2023-10-06T02:24:43.369693+0000","last_clean":"2023-10-06T02:24:43.369693+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:43.369693+0000","last_undegraded":"2023-10-06T02:24:43.369693+0000","last_fullsized":"2023-10-06T02:24:43.369693+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:24:42.362895+0000","last_change":"2023-10-06T02:24:42.377418+0000","last_active":"2023-10-06T02:24:42.362895+0000","last_peered":"2023-10-06T02:24:42.362895+0000","last_clean":"2023-10-06T02:24:42.362895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:42.362895+0000","last_undegraded":"2023-10-06T02:24:42.362895+0000","last_fullsized":"2023-10-06T02:24:42.362895+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_clean_scrub_stamp":"2023-10-06T02:24:42.362895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:50:17.948474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:24:43.369693+0000","last_change":"2023-10-06T02:24:43.369693+0000","last_active":"2023-10-06T02:24:43.369693+0000","last_peered":"2023-10-06T02:24:43.369693+0000","last_clean":"2023-10-06T02:24:43.369693+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:43.369693+0000","last_undegraded":"2023-10-06T02:24:43.369693+0000","last_fullsized":"2023-10-06T02:24:43.369693+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:24:42.362895+0000","last_change":"2023-10-06T02:24:42.372509+0000","last_active":"2023-10-06T02:24:42.362895+0000","last_peered":"2023-10-06T02:24:42.362895+0000","last_clean":"2023-10-06T02:24:42.362895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:42.362895+0000","last_undegraded":"2023-10-06T02:24:42.362895+0000","last_fullsized":"2023-10-06T02:24:42.362895+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_clean_scrub_stamp":"2023-10-06T02:24:42.362895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:24:14.412112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:24:43.369693+0000","last_change":"2023-10-06T02:24:43.369693+0000","last_active":"2023-10-06T02:24:43.369693+0000","last_peered":"2023-10-06T02:24:43.369693+0000","last_clean":"2023-10-06T02:24:43.369693+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:43.369693+0000","last_undegraded":"2023-10-06T02:24:43.369693+0000","last_fullsized":"2023-10-06T02:24:43.369693+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:24:42.362895+0000","last_change":"2023-10-06T02:24:42.378778+0000","last_active":"2023-10-06T02:24:42.362895+0000","last_peered":"2023-10-06T02:24:42.362895+0000","last_clean":"2023-10-06T02:24:42.362895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:42.362895+0000","last_undegraded":"2023-10-06T02:24:42.362895+0000","last_fullsized":"2023-10-06T02:24:42.362895+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_clean_scrub_stamp":"2023-10-06T02:24:42.362895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:17:53.991541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:24:43.369693+0000","last_change":"2023-10-06T02:24:43.369693+0000","last_active":"2023-10-06T02:24:43.369693+0000","last_peered":"2023-10-06T02:24:43.369693+0000","last_clean":"2023-10-06T02:24:43.369693+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:43.369693+0000","last_undegraded":"2023-10-06T02:24:43.369693+0000","last_fullsized":"2023-10-06T02:24:43.369693+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:24:42.362895+0000","last_change":"2023-10-06T02:24:42.375444+0000","last_active":"2023-10-06T02:24:42.362895+0000","last_peered":"2023-10-06T02:24:42.362895+0000","last_clean":"2023-10-06T02:24:42.362895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:42.362895+0000","last_undegraded":"2023-10-06T02:24:42.362895+0000","last_fullsized":"2023-10-06T02:24:42.362895+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_clean_scrub_stamp":"2023-10-06T02:24:42.362895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:57:30.082203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:24:43.369693+0000","last_change":"2023-10-06T02:24:43.369693+0000","last_active":"2023-10-06T02:24:43.369693+0000","last_peered":"2023-10-06T02:24:43.369693+0000","last_clean":"2023-10-06T02:24:43.369693+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:43.369693+0000","last_undegraded":"2023-10-06T02:24:43.369693+0000","last_fullsized":"2023-10-06T02:24:43.369693+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:24:42.362895+0000","last_change":"2023-10-06T02:24:42.376091+0000","last_active":"2023-10-06T02:24:42.362895+0000","last_peered":"2023-10-06T02:24:42.362895+0000","last_clean":"2023-10-06T02:24:42.362895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:42.362895+0000","last_undegraded":"2023-10-06T02:24:42.362895+0000","last_fullsized":"2023-10-06T02:24:42.362895+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_clean_scrub_stamp":"2023-10-06T02:24:42.362895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:18:35.870113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:24:43.369693+0000","last_change":"2023-10-06T02:24:43.369693+0000","last_active":"2023-10-06T02:24:43.369693+0000","last_peered":"2023-10-06T02:24:43.369693+0000","last_clean":"2023-10-06T02:24:43.369693+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:43.369693+0000","last_undegraded":"2023-10-06T02:24:43.369693+0000","last_fullsized":"2023-10-06T02:24:43.369693+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:24:42.362895+0000","last_change":"2023-10-06T02:24:42.376005+0000","last_active":"2023-10-06T02:24:42.362895+0000","last_peered":"2023-10-06T02:24:42.362895+0000","last_clean":"2023-10-06T02:24:42.362895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:42.362895+0000","last_undegraded":"2023-10-06T02:24:42.362895+0000","last_fullsized":"2023-10-06T02:24:42.362895+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_clean_scrub_stamp":"2023-10-06T02:24:42.362895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:18:35.870113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:24:43.369693+0000","last_change":"2023-10-06T02:24:43.369693+0000","last_active":"2023-10-06T02:24:43.369693+0000","last_peered":"2023-10-06T02:24:43.369693+0000","last_clean":"2023-10-06T02:24:43.369693+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:43.369693+0000","last_undegraded":"2023-10-06T02:24:43.369693+0000","last_fullsized":"2023-10-06T02:24:43.369693+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:24:42.362895+0000","last_change":"2023-10-06T02:24:42.375796+0000","last_active":"2023-10-06T02:24:42.362895+0000","last_peered":"2023-10-06T02:24:42.362895+0000","last_clean":"2023-10-06T02:24:42.362895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:42.362895+0000","last_undegraded":"2023-10-06T02:24:42.362895+0000","last_fullsized":"2023-10-06T02:24:42.362895+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_clean_scrub_stamp":"2023-10-06T02:24:42.362895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:05:52.756921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:24:43.369693+0000","last_change":"2023-10-06T02:24:43.369693+0000","last_active":"2023-10-06T02:24:43.369693+0000","last_peered":"2023-10-06T02:24:43.369693+0000","last_clean":"2023-10-06T02:24:43.369693+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:43.369693+0000","last_undegraded":"2023-10-06T02:24:43.369693+0000","last_fullsized":"2023-10-06T02:24:43.369693+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:24:42.362895+0000","last_change":"2023-10-06T02:24:42.379196+0000","last_active":"2023-10-06T02:24:42.362895+0000","last_peered":"2023-10-06T02:24:42.362895+0000","last_clean":"2023-10-06T02:24:42.362895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:42.362895+0000","last_undegraded":"2023-10-06T02:24:42.362895+0000","last_fullsized":"2023-10-06T02:24:42.362895+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_clean_scrub_stamp":"2023-10-06T02:24:42.362895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:03:09.689888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:24:43.369693+0000","last_change":"2023-10-06T02:24:43.369693+0000","last_active":"2023-10-06T02:24:43.369693+0000","last_peered":"2023-10-06T02:24:43.369693+0000","last_clean":"2023-10-06T02:24:43.369693+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:43.369693+0000","last_undegraded":"2023-10-06T02:24:43.369693+0000","last_fullsized":"2023-10-06T02:24:43.369693+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:24:42.362895+0000","last_change":"2023-10-06T02:24:42.373658+0000","last_active":"2023-10-06T02:24:42.362895+0000","last_peered":"2023-10-06T02:24:42.362895+0000","last_clean":"2023-10-06T02:24:42.362895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:42.362895+0000","last_undegraded":"2023-10-06T02:24:42.362895+0000","last_fullsized":"2023-10-06T02:24:42.362895+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_clean_scrub_stamp":"2023-10-06T02:24:42.362895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:42:50.520697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:24:43.369693+0000","last_change":"2023-10-06T02:24:43.369693+0000","last_active":"2023-10-06T02:24:43.369693+0000","last_peered":"2023-10-06T02:24:43.369693+0000","last_clean":"2023-10-06T02:24:43.369693+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:43.369693+0000","last_undegraded":"2023-10-06T02:24:43.369693+0000","last_fullsized":"2023-10-06T02:24:43.369693+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:24:42.362895+0000","last_change":"2023-10-06T02:24:42.375547+0000","last_active":"2023-10-06T02:24:42.362895+0000","last_peered":"2023-10-06T02:24:42.362895+0000","last_clean":"2023-10-06T02:24:42.362895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:42.362895+0000","last_undegraded":"2023-10-06T02:24:42.362895+0000","last_fullsized":"2023-10-06T02:24:42.362895+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_clean_scrub_stamp":"2023-10-06T02:24:42.362895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:03:09.689888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:24:43.369693+0000","last_change":"2023-10-06T02:24:43.369693+0000","last_active":"2023-10-06T02:24:43.369693+0000","last_peered":"2023-10-06T02:24:43.369693+0000","last_clean":"2023-10-06T02:24:43.369693+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:43.369693+0000","last_undegraded":"2023-10-06T02:24:43.369693+0000","last_fullsized":"2023-10-06T02:24:43.369693+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:24:42.362895+0000","last_change":"2023-10-06T02:24:42.375777+0000","last_active":"2023-10-06T02:24:42.362895+0000","last_peered":"2023-10-06T02:24:42.362895+0000","last_clean":"2023-10-06T02:24:42.362895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:42.362895+0000","last_undegraded":"2023-10-06T02:24:42.362895+0000","last_fullsized":"2023-10-06T02:24:42.362895+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_clean_scrub_stamp":"2023-10-06T02:24:42.362895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:47:23.440398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:24:43.369693+0000","last_change":"2023-10-06T02:24:43.369693+0000","last_active":"2023-10-06T02:24:43.369693+0000","last_peered":"2023-10-06T02:24:43.369693+0000","last_clean":"2023-10-06T02:24:43.369693+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:43.369693+0000","last_undegraded":"2023-10-06T02:24:43.369693+0000","last_fullsized":"2023-10-06T02:24:43.369693+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:24:42.362895+0000","last_change":"2023-10-06T02:24:42.378885+0000","last_active":"2023-10-06T02:24:42.362895+0000","last_peered":"2023-10-06T02:24:42.362895+0000","last_clean":"2023-10-06T02:24:42.362895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:42.362895+0000","last_undegraded":"2023-10-06T02:24:42.362895+0000","last_fullsized":"2023-10-06T02:24:42.362895+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_clean_scrub_stamp":"2023-10-06T02:24:42.362895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:54:52.692242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:24:43.369693+0000","last_change":"2023-10-06T02:24:43.369693+0000","last_active":"2023-10-06T02:24:43.369693+0000","last_peered":"2023-10-06T02:24:43.369693+0000","last_clean":"2023-10-06T02:24:43.369693+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:43.369693+0000","last_undegraded":"2023-10-06T02:24:43.369693+0000","last_fullsized":"2023-10-06T02:24:43.369693+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:24:42.362895+0000","last_change":"2023-10-06T02:24:42.372144+0000","last_active":"2023-10-06T02:24:42.362895+0000","last_peered":"2023-10-06T02:24:42.362895+0000","last_clean":"2023-10-06T02:24:42.362895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:42.362895+0000","last_undegraded":"2023-10-06T02:24:42.362895+0000","last_fullsized":"2023-10-06T02:24:42.362895+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_clean_scrub_stamp":"2023-10-06T02:24:42.362895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:11:59.249723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:24:43.369693+0000","last_change":"2023-10-06T02:24:43.369693+0000","last_active":"2023-10-06T02:24:43.369693+0000","last_peered":"2023-10-06T02:24:43.369693+0000","last_clean":"2023-10-06T02:24:43.369693+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:43.369693+0000","last_undegraded":"2023-10-06T02:24:43.369693+0000","last_fullsized":"2023-10-06T02:24:43.369693+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:24:42.362895+0000","last_change":"2023-10-06T02:24:42.372442+0000","last_active":"2023-10-06T02:24:42.362895+0000","last_peered":"2023-10-06T02:24:42.362895+0000","last_clean":"2023-10-06T02:24:42.362895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:42.362895+0000","last_undegraded":"2023-10-06T02:24:42.362895+0000","last_fullsized":"2023-10-06T02:24:42.362895+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_clean_scrub_stamp":"2023-10-06T02:24:42.362895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:39:27.342816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:24:43.369693+0000","last_change":"2023-10-06T02:24:43.369693+0000","last_active":"2023-10-06T02:24:43.369693+0000","last_peered":"2023-10-06T02:24:43.369693+0000","last_clean":"2023-10-06T02:24:43.369693+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:43.369693+0000","last_undegraded":"2023-10-06T02:24:43.369693+0000","last_fullsized":"2023-10-06T02:24:43.369693+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:24:42.362895+0000","last_change":"2023-10-06T02:24:42.373010+0000","last_active":"2023-10-06T02:24:42.362895+0000","last_peered":"2023-10-06T02:24:42.362895+0000","last_clean":"2023-10-06T02:24:42.362895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:42.362895+0000","last_undegraded":"2023-10-06T02:24:42.362895+0000","last_fullsized":"2023-10-06T02:24:42.362895+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_clean_scrub_stamp":"2023-10-06T02:24:42.362895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:53.458652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:24:43.369693+0000","last_change":"2023-10-06T02:24:43.369693+0000","last_active":"2023-10-06T02:24:43.369693+0000","last_peered":"2023-10-06T02:24:43.369693+0000","last_clean":"2023-10-06T02:24:43.369693+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:43.369693+0000","last_undegraded":"2023-10-06T02:24:43.369693+0000","last_fullsized":"2023-10-06T02:24:43.369693+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:24:42.362895+0000","last_change":"2023-10-06T02:24:42.379735+0000","last_active":"2023-10-06T02:24:42.362895+0000","last_peered":"2023-10-06T02:24:42.362895+0000","last_clean":"2023-10-06T02:24:42.362895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:42.362895+0000","last_undegraded":"2023-10-06T02:24:42.362895+0000","last_fullsized":"2023-10-06T02:24:42.362895+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_clean_scrub_stamp":"2023-10-06T02:24:42.362895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:36:47.623656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:24:43.369693+0000","last_change":"2023-10-06T02:24:43.369693+0000","last_active":"2023-10-06T02:24:43.369693+0000","last_peered":"2023-10-06T02:24:43.369693+0000","last_clean":"2023-10-06T02:24:43.369693+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:43.369693+0000","last_undegraded":"2023-10-06T02:24:43.369693+0000","last_fullsized":"2023-10-06T02:24:43.369693+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:24:42.362895+0000","last_change":"2023-10-06T02:24:42.376417+0000","last_active":"2023-10-06T02:24:42.362895+0000","last_peered":"2023-10-06T02:24:42.362895+0000","last_clean":"2023-10-06T02:24:42.362895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:42.362895+0000","last_undegraded":"2023-10-06T02:24:42.362895+0000","last_fullsized":"2023-10-06T02:24:42.362895+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_clean_scrub_stamp":"2023-10-06T02:24:42.362895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:49:49.997672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:24:43.369693+0000","last_change":"2023-10-06T02:24:43.369693+0000","last_active":"2023-10-06T02:24:43.369693+0000","last_peered":"2023-10-06T02:24:43.369693+0000","last_clean":"2023-10-06T02:24:43.369693+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:43.369693+0000","last_undegraded":"2023-10-06T02:24:43.369693+0000","last_fullsized":"2023-10-06T02:24:43.369693+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:24:42.362895+0000","last_change":"2023-10-06T02:24:42.378975+0000","last_active":"2023-10-06T02:24:42.362895+0000","last_peered":"2023-10-06T02:24:42.362895+0000","last_clean":"2023-10-06T02:24:42.362895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:42.362895+0000","last_undegraded":"2023-10-06T02:24:42.362895+0000","last_fullsized":"2023-10-06T02:24:42.362895+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_clean_scrub_stamp":"2023-10-06T02:24:42.362895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:49:09.661411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:24:43.369693+0000","last_change":"2023-10-06T02:24:43.369693+0000","last_active":"2023-10-06T02:24:43.369693+0000","last_peered":"2023-10-06T02:24:43.369693+0000","last_clean":"2023-10-06T02:24:43.369693+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:43.369693+0000","last_undegraded":"2023-10-06T02:24:43.369693+0000","last_fullsized":"2023-10-06T02:24:43.369693+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:24:42.362895+0000","last_change":"2023-10-06T02:24:42.371739+0000","last_active":"2023-10-06T02:24:42.362895+0000","last_peered":"2023-10-06T02:24:42.362895+0000","last_clean":"2023-10-06T02:24:42.362895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:42.362895+0000","last_undegraded":"2023-10-06T02:24:42.362895+0000","last_fullsized":"2023-10-06T02:24:42.362895+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_clean_scrub_stamp":"2023-10-06T02:24:42.362895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:11:50.621481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:24:43.369693+0000","last_change":"2023-10-06T02:24:43.369693+0000","last_active":"2023-10-06T02:24:43.369693+0000","last_peered":"2023-10-06T02:24:43.369693+0000","last_clean":"2023-10-06T02:24:43.369693+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:43.369693+0000","last_undegraded":"2023-10-06T02:24:43.369693+0000","last_fullsized":"2023-10-06T02:24:43.369693+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:24:42.362895+0000","last_change":"2023-10-06T02:24:42.376246+0000","last_active":"2023-10-06T02:24:42.362895+0000","last_peered":"2023-10-06T02:24:42.362895+0000","last_clean":"2023-10-06T02:24:42.362895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:42.362895+0000","last_undegraded":"2023-10-06T02:24:42.362895+0000","last_fullsized":"2023-10-06T02:24:42.362895+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_clean_scrub_stamp":"2023-10-06T02:24:42.362895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:47:23.440398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":30,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:24:39.594389+0000","last_change":"2023-10-06T02:23:02.054039+0000","last_active":"2023-10-06T02:24:39.594389+0000","last_peered":"2023-10-06T02:24:39.594389+0000","last_clean":"2023-10-06T02:24:39.594389+0000","last_became_active":"2023-10-06T02:23:02.053742+0000","last_became_peered":"2023-10-06T02:23:02.053742+0000","last_unstale":"2023-10-06T02:24:39.594389+0000","last_undegraded":"2023-10-06T02:24:39.594389+0000","last_fullsized":"2023-10-06T02:24:39.594389+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:58:18.846475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:24:43.369693+0000","last_change":"2023-10-06T02:24:43.369693+0000","last_active":"2023-10-06T02:24:43.369693+0000","last_peered":"2023-10-06T02:24:43.369693+0000","last_clean":"2023-10-06T02:24:43.369693+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:43.369693+0000","last_undegraded":"2023-10-06T02:24:43.369693+0000","last_fullsized":"2023-10-06T02:24:43.369693+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:24:42.362895+0000","last_change":"2023-10-06T02:24:42.372578+0000","last_active":"2023-10-06T02:24:42.362895+0000","last_peered":"2023-10-06T02:24:42.362895+0000","last_clean":"2023-10-06T02:24:42.362895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:42.362895+0000","last_undegraded":"2023-10-06T02:24:42.362895+0000","last_fullsized":"2023-10-06T02:24:42.362895+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_clean_scrub_stamp":"2023-10-06T02:24:42.362895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:40:45.079228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":34,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:24:39.368010+0000","last_change":"2023-10-06T02:23:01.988631+0000","last_active":"2023-10-06T02:24:39.368010+0000","last_peered":"2023-10-06T02:24:39.368010+0000","last_clean":"2023-10-06T02:24:39.368010+0000","last_became_active":"2023-10-06T02:23:01.988303+0000","last_became_peered":"2023-10-06T02:23:01.988303+0000","last_unstale":"2023-10-06T02:24:39.368010+0000","last_undegraded":"2023-10-06T02:24:39.368010+0000","last_fullsized":"2023-10-06T02:24:39.368010+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:38.211466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:24:43.369693+0000","last_change":"2023-10-06T02:24:43.369693+0000","last_active":"2023-10-06T02:24:43.369693+0000","last_peered":"2023-10-06T02:24:43.369693+0000","last_clean":"2023-10-06T02:24:43.369693+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:43.369693+0000","last_undegraded":"2023-10-06T02:24:43.369693+0000","last_fullsized":"2023-10-06T02:24:43.369693+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:24:42.362895+0000","last_change":"2023-10-06T02:24:42.378755+0000","last_active":"2023-10-06T02:24:42.362895+0000","last_peered":"2023-10-06T02:24:42.362895+0000","last_clean":"2023-10-06T02:24:42.362895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:42.362895+0000","last_undegraded":"2023-10-06T02:24:42.362895+0000","last_fullsized":"2023-10-06T02:24:42.362895+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_clean_scrub_stamp":"2023-10-06T02:24:42.362895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:30.949387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":30,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:24:39.365425+0000","last_change":"2023-10-06T02:23:02.054338+0000","last_active":"2023-10-06T02:24:39.365425+0000","last_peered":"2023-10-06T02:24:39.365425+0000","last_clean":"2023-10-06T02:24:39.365425+0000","last_became_active":"2023-10-06T02:23:02.053991+0000","last_became_peered":"2023-10-06T02:23:02.053991+0000","last_unstale":"2023-10-06T02:24:39.365425+0000","last_undegraded":"2023-10-06T02:24:39.365425+0000","last_fullsized":"2023-10-06T02:24:39.365425+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:39:30.192838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:24:43.369693+0000","last_change":"2023-10-06T02:24:43.369693+0000","last_active":"2023-10-06T02:24:43.369693+0000","last_peered":"2023-10-06T02:24:43.369693+0000","last_clean":"2023-10-06T02:24:43.369693+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:43.369693+0000","last_undegraded":"2023-10-06T02:24:43.369693+0000","last_fullsized":"2023-10-06T02:24:43.369693+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:24:42.362895+0000","last_change":"2023-10-06T02:24:42.378793+0000","last_active":"2023-10-06T02:24:42.362895+0000","last_peered":"2023-10-06T02:24:42.362895+0000","last_clean":"2023-10-06T02:24:42.362895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:42.362895+0000","last_undegraded":"2023-10-06T02:24:42.362895+0000","last_fullsized":"2023-10-06T02:24:42.362895+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_clean_scrub_stamp":"2023-10-06T02:24:42.362895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:57:30.082203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":34,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:24:39.368050+0000","last_change":"2023-10-06T02:23:01.988140+0000","last_active":"2023-10-06T02:24:39.368050+0000","last_peered":"2023-10-06T02:24:39.368050+0000","last_clean":"2023-10-06T02:24:39.368050+0000","last_became_active":"2023-10-06T02:23:01.987769+0000","last_became_peered":"2023-10-06T02:23:01.987769+0000","last_unstale":"2023-10-06T02:24:39.368050+0000","last_undegraded":"2023-10-06T02:24:39.368050+0000","last_fullsized":"2023-10-06T02:24:39.368050+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:20:08.391912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":182,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:24:39.595372+0000","last_change":"2023-10-06T02:22:55.032519+0000","last_active":"2023-10-06T02:24:39.595372+0000","last_peered":"2023-10-06T02:24:39.595372+0000","last_clean":"2023-10-06T02:24:39.595372+0000","last_became_active":"2023-10-06T02:22:55.031367+0000","last_became_peered":"2023-10-06T02:22:55.031367+0000","last_unstale":"2023-10-06T02:24:39.595372+0000","last_undegraded":"2023-10-06T02:24:39.595372+0000","last_fullsized":"2023-10-06T02:24:39.595372+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:22:53.309097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:22:53.309097+0000","last_clean_scrub_stamp":"2023-10-06T02:22:53.309097+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:53:46.520909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:24:43.369693+0000","last_change":"2023-10-06T02:24:43.369693+0000","last_active":"2023-10-06T02:24:43.369693+0000","last_peered":"2023-10-06T02:24:43.369693+0000","last_clean":"2023-10-06T02:24:43.369693+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:43.369693+0000","last_undegraded":"2023-10-06T02:24:43.369693+0000","last_fullsized":"2023-10-06T02:24:43.369693+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:24:42.362895+0000","last_change":"2023-10-06T02:24:42.375435+0000","last_active":"2023-10-06T02:24:42.362895+0000","last_peered":"2023-10-06T02:24:42.362895+0000","last_clean":"2023-10-06T02:24:42.362895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:42.362895+0000","last_undegraded":"2023-10-06T02:24:42.362895+0000","last_fullsized":"2023-10-06T02:24:42.362895+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_clean_scrub_stamp":"2023-10-06T02:24:42.362895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:49:09.661411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":29,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:24:39.595466+0000","last_change":"2023-10-06T02:23:01.990773+0000","last_active":"2023-10-06T02:24:39.595466+0000","last_peered":"2023-10-06T02:24:39.595466+0000","last_clean":"2023-10-06T02:24:39.595466+0000","last_became_active":"2023-10-06T02:23:01.990511+0000","last_became_peered":"2023-10-06T02:23:01.990511+0000","last_unstale":"2023-10-06T02:24:39.595466+0000","last_undegraded":"2023-10-06T02:24:39.595466+0000","last_fullsized":"2023-10-06T02:24:39.595466+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:42:54.797731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:24:43.369693+0000","last_change":"2023-10-06T02:24:43.369693+0000","last_active":"2023-10-06T02:24:43.369693+0000","last_peered":"2023-10-06T02:24:43.369693+0000","last_clean":"2023-10-06T02:24:43.369693+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:43.369693+0000","last_undegraded":"2023-10-06T02:24:43.369693+0000","last_fullsized":"2023-10-06T02:24:43.369693+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:24:42.362895+0000","last_change":"2023-10-06T02:24:42.372439+0000","last_active":"2023-10-06T02:24:42.362895+0000","last_peered":"2023-10-06T02:24:42.362895+0000","last_clean":"2023-10-06T02:24:42.362895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:42.362895+0000","last_undegraded":"2023-10-06T02:24:42.362895+0000","last_fullsized":"2023-10-06T02:24:42.362895+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_clean_scrub_stamp":"2023-10-06T02:24:42.362895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:47:23.440398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":29,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:24:39.595325+0000","last_change":"2023-10-06T02:23:02.054069+0000","last_active":"2023-10-06T02:24:39.595325+0000","last_peered":"2023-10-06T02:24:39.595325+0000","last_clean":"2023-10-06T02:24:39.595325+0000","last_became_active":"2023-10-06T02:23:02.053739+0000","last_became_peered":"2023-10-06T02:23:02.053739+0000","last_unstale":"2023-10-06T02:24:39.595325+0000","last_undegraded":"2023-10-06T02:24:39.595325+0000","last_fullsized":"2023-10-06T02:24:39.595325+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:57:48.822608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:24:43.369693+0000","last_change":"2023-10-06T02:24:43.369693+0000","last_active":"2023-10-06T02:24:43.369693+0000","last_peered":"2023-10-06T02:24:43.369693+0000","last_clean":"2023-10-06T02:24:43.369693+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:43.369693+0000","last_undegraded":"2023-10-06T02:24:43.369693+0000","last_fullsized":"2023-10-06T02:24:43.369693+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:24:42.362895+0000","last_change":"2023-10-06T02:24:42.375841+0000","last_active":"2023-10-06T02:24:42.362895+0000","last_peered":"2023-10-06T02:24:42.362895+0000","last_clean":"2023-10-06T02:24:42.362895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:42.362895+0000","last_undegraded":"2023-10-06T02:24:42.362895+0000","last_fullsized":"2023-10-06T02:24:42.362895+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_clean_scrub_stamp":"2023-10-06T02:24:42.362895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:20:19.069462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":30,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:24:39.594355+0000","last_change":"2023-10-06T02:23:01.989934+0000","last_active":"2023-10-06T02:24:39.594355+0000","last_peered":"2023-10-06T02:24:39.594355+0000","last_clean":"2023-10-06T02:24:39.594355+0000","last_became_active":"2023-10-06T02:23:01.989706+0000","last_became_peered":"2023-10-06T02:23:01.989706+0000","last_unstale":"2023-10-06T02:24:39.594355+0000","last_undegraded":"2023-10-06T02:24:39.594355+0000","last_fullsized":"2023-10-06T02:24:39.594355+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:06.212251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:24:43.369693+0000","last_change":"2023-10-06T02:24:43.369693+0000","last_active":"2023-10-06T02:24:43.369693+0000","last_peered":"2023-10-06T02:24:43.369693+0000","last_clean":"2023-10-06T02:24:43.369693+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:43.369693+0000","last_undegraded":"2023-10-06T02:24:43.369693+0000","last_fullsized":"2023-10-06T02:24:43.369693+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:24:42.362895+0000","last_change":"2023-10-06T02:24:42.375789+0000","last_active":"2023-10-06T02:24:42.362895+0000","last_peered":"2023-10-06T02:24:42.362895+0000","last_clean":"2023-10-06T02:24:42.362895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:24:42.362895+0000","last_undegraded":"2023-10-06T02:24:42.362895+0000","last_fullsized":"2023-10-06T02:24:42.362895+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_clean_scrub_stamp":"2023-10-06T02:24:42.362895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:50:17.948474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":30,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:24:39.595646+0000","last_change":"2023-10-06T02:23:01.992935+0000","last_active":"2023-10-06T02:24:39.595646+0000","last_peered":"2023-10-06T02:24:39.595646+0000","last_clean":"2023-10-06T02:24:39.595646+0000","last_became_active":"2023-10-06T02:23:01.992568+0000","last_became_peered":"2023-10-06T02:23:01.992568+0000","last_unstale":"2023-10-06T02:24:39.595646+0000","last_undegraded":"2023-10-06T02:24:39.595646+0000","last_fullsized":"2023-10-06T02:24:39.595646+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:06.212251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574874,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298372,"kb_used_data":1084,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073468,"statfs":{"total":96636764160,"available":96331231232,"internally_reserved":0,"allocated":1110016,"data_stored":806056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.50800000000000001}]},{"osd":1,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.58499999999999996}]},{"osd":2,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.63500000000000001}]},{"osd":3,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.72799999999999998}]},{"osd":4,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.66500000000000004}]},{"osd":5,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.70799999999999996}]},{"osd":6,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.79200000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574874,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297780,"kb_used_data":492,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074060,"statfs":{"total":96636764160,"available":96331837440,"internally_reserved":0,"allocated":503808,"data_stored":209530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.48299999999999998}]},{"osd":1,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.70399999999999996}]},{"osd":2,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.60499999999999998}]},{"osd":3,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.64100000000000001}]},{"osd":4,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.79700000000000004}]},{"osd":5,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.96599999999999997}]},{"osd":7,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.458,"5min":1.458,"15min":1.458},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.78000000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574873,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297792,"kb_used_data":504,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074048,"statfs":{"total":96636764160,"available":96331825152,"internally_reserved":0,"allocated":516096,"data_stored":215688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.788,"5min":1.788,"15min":1.788},"last":0.46000000000000002}]},{"osd":2,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":2.0840000000000001,"5min":2.0840000000000001,"15min":2.0840000000000001},"last":0.441},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.52800000000000002}]},{"osd":3,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.861,"5min":1.861,"15min":1.861},"last":0.502}]},{"osd":4,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.9279999999999999,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":0.872},{"interface":"front","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":2.0179999999999998,"5min":2.0179999999999998,"15min":2.0179999999999998},"last":0.96799999999999997}]},{"osd":5,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.9830000000000001,"5min":1.9830000000000001,"15min":1.9830000000000001},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.77600000000000002}]},{"osd":6,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.829,"5min":1.829,"15min":1.829},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":2.052,"5min":2.052,"15min":2.052},"last":0.94699999999999995}]},{"osd":7,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.895,"5min":1.895,"15min":1.895},"last":0.749}]}]},{"osd":0,"up_from":13,"seq":55834574874,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298360,"kb_used_data":1072,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073480,"statfs":{"total":96636764160,"available":96331243520,"internally_reserved":0,"allocated":1097728,"data_stored":799898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.48599999999999999}]},{"osd":2,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.375}]},{"osd":3,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.61899999999999999}]},{"osd":4,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.304,"5min":1.304,"15min":1.304},"last":0.92700000000000005}]},{"osd":5,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.66300000000000003}]},{"osd":6,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.78900000000000003}]},{"osd":7,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.872},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.88600000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574875,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297792,"kb_used_data":504,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074048,"statfs":{"total":96636764160,"available":96331825152,"internally_reserved":0,"allocated":516096,"data_stored":215688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.371},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.46700000000000003}]},{"osd":1,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":0.57699999999999996}]},{"osd":3,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.56299999999999994}]},{"osd":4,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.621,"5min":1.621,"15min":1.621},"last":0.94099999999999995}]},{"osd":5,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.88100000000000001}]},{"osd":6,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.95699999999999996}]},{"osd":7,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.64700000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574872,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297792,"kb_used_data":504,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074048,"statfs":{"total":96636764160,"available":96331825152,"internally_reserved":0,"allocated":516096,"data_stored":215688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":2.024,"5min":2.024,"15min":2.024},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":2.181,"5min":2.181,"15min":2.181},"last":0.35999999999999999}]},{"osd":1,"last update":"Fri Oct 6 02:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":4.492,"5min":4.492,"15min":4.492},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":4.569,"5min":4.569,"15min":4.569},"last":0.45900000000000002}]},{"osd":2,"last update":"Fri Oct 6 02:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":2.0870000000000002,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":2.4260000000000002,"5min":2.4260000000000002,"15min":2.4260000000000002},"last":0.47599999999999998}]},{"osd":4,"last update":"Fri Oct 6 02:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":2.214,"5min":2.214,"15min":2.214},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.75900000000000001}]},{"osd":5,"last update":"Fri Oct 6 02:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":2.242,"5min":2.242,"15min":2.242},"last":0.745}]},{"osd":6,"last update":"Fri Oct 6 02:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.357,"5min":1.357,"15min":1.357},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.64500000000000002}]},{"osd":7,"last update":"Fri Oct 6 02:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.27,"5min":2.27,"15min":2.27},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.69399999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574874,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297792,"kb_used_data":504,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074048,"statfs":{"total":96636764160,"available":96331825152,"internally_reserved":0,"allocated":516096,"data_stored":215688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:23:55 2023","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":12.337,"5min":12.337,"15min":12.337},"last":0.58999999999999997},{"interface":"front","average":{"1min":1.02,"5min":1.02,"15min":1.02},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":9.6539999999999999,"5min":9.6539999999999999,"15min":9.6539999999999999},"last":0.65000000000000002}]},{"osd":1,"last update":"Fri Oct 6 02:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":3.181,"5min":3.181,"15min":3.181},"last":0.60899999999999999},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":12.946999999999999,"5min":12.946999999999999,"15min":12.946999999999999},"last":0.68600000000000005}]},{"osd":2,"last update":"Fri Oct 6 02:23:55 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":12.377000000000001,"5min":12.377000000000001,"15min":12.377000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":6.3220000000000001,"5min":6.3220000000000001,"15min":6.3220000000000001},"last":0.627}]},{"osd":3,"last update":"Fri Oct 6 02:23:55 2023","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":9.7720000000000002,"5min":9.7720000000000002,"15min":9.7720000000000002},"last":0.752},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":3.0840000000000001,"5min":3.0840000000000001,"15min":3.0840000000000001},"last":0.72199999999999998}]},{"osd":5,"last update":"Fri Oct 6 02:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":6.2690000000000001,"5min":6.2690000000000001,"15min":6.2690000000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":10.769,"5min":10.769,"15min":10.769},"last":0.78700000000000003}]},{"osd":6,"last update":"Fri Oct 6 02:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":3.1349999999999998,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":0.66900000000000004},{"interface":"front","average":{"1min":1.077,"5min":1.077,"15min":1.077},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":8.0899999999999999,"5min":8.0899999999999999,"15min":8.0899999999999999},"last":0.73299999999999998}]},{"osd":7,"last update":"Fri Oct 6 02:23:55 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":9.7089999999999996,"5min":9.7089999999999996,"15min":9.7089999999999996},"last":0.871},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":7.6719999999999997,"5min":7.6719999999999997,"15min":7.6719999999999997},"last":0.96399999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574874,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297792,"kb_used_data":504,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074048,"statfs":{"total":96636764160,"available":96331825152,"internally_reserved":0,"allocated":516096,"data_stored":215688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.64900000000000002}]},{"osd":1,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.69799999999999995}]},{"osd":2,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.60399999999999998}]},{"osd":3,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.77800000000000002}]},{"osd":4,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.627},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.55300000000000005}]},{"osd":6,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.75900000000000001}]},{"osd":7,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.747},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.72899999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T02:24:48.772 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T02:24:48.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.991+0000 7f2e9719d700 1 Processor -- start 2023-10-06T02:24:48.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.991+0000 7f2e9719d700 1 -- start start 2023-10-06T02:24:48.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.992+0000 7f2e9719d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2e9006e0a0 0x7f2e900c6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:48.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.992+0000 7f2e9719d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2e90150380 0x7f2e9006a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:48.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.993+0000 7f2e9719d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e9006a490 con 0x7f2e9006f3a0 2023-10-06T02:24:48.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.993+0000 7f2e9719d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e900c7490 con 0x7f2e9006e0a0 2023-10-06T02:24:48.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.993+0000 7f2e9719d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e900c7770 con 0x7f2e90150380 2023-10-06T02:24:48.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.993+0000 7f2e9619b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2e9006e0a0 0x7f2e900c6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:48.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.993+0000 7f2e9619b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2e9006e0a0 0x7f2e900c6b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53276/0 (socket says 172.21.15.31:53276) 2023-10-06T02:24:48.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.993+0000 7f2e9619b700 1 -- 172.21.15.31:0/757582029 learned_addr learned my addr 172.21.15.31:0/757582029 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:48.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.993+0000 7f2e9599a700 1 --1- 172.21.15.31:0/757582029 >> v1:172.21.15.31:6789/0 conn(0x7f2e9006f3a0 0x7f2e9014fc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:59420/0 (socket says 172.21.15.31:59420) 2023-10-06T02:24:48.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.993+0000 7f2e9699c700 1 --2- 172.21.15.31:0/757582029 >> v2:172.21.15.62:3300/0 conn(0x7f2e90150380 0x7f2e9006a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:48.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.993+0000 7f2e9619b700 1 -- 172.21.15.31:0/757582029 >> v2:172.21.15.62:3300/0 conn(0x7f2e90150380 msgr2=0x7f2e9006a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:48.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.993+0000 7f2e9619b700 1 --2- 172.21.15.31:0/757582029 >> v2:172.21.15.62:3300/0 conn(0x7f2e90150380 0x7f2e9006a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:48.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.993+0000 7f2e9619b700 1 -- 172.21.15.31:0/757582029 >> v1:172.21.15.31:6789/0 conn(0x7f2e9006f3a0 legacy=0x7f2e9014fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:48.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.993+0000 7f2e9619b700 1 -- 172.21.15.31:0/757582029 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e900c7a50 con 0x7f2e9006e0a0 2023-10-06T02:24:48.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.993+0000 7f2e95199700 1 -- 172.21.15.31:0/757582029 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 482449946 0 0) 0x7f2e9006a490 con 0x7f2e9006f3a0 2023-10-06T02:24:48.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.994+0000 7f2e9619b700 1 --2- 172.21.15.31:0/757582029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e9006e0a0 0x7f2e900c6b20 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f2e9006a400 tx=0x7f2e8c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9b93c35595d8340 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:48.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.994+0000 7f2e95199700 1 -- 172.21.15.31:0/757582029 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e8c00b860 con 0x7f2e9006e0a0 2023-10-06T02:24:48.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.994+0000 7f2e95199700 1 -- 172.21.15.31:0/757582029 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e8c00b9c0 con 0x7f2e9006e0a0 2023-10-06T02:24:48.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.994+0000 7f2e95199700 1 -- 172.21.15.31:0/757582029 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e8c00bd60 con 0x7f2e9006e0a0 2023-10-06T02:24:48.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.994+0000 7f2e9719d700 1 -- 172.21.15.31:0/757582029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e9006e0a0 msgr2=0x7f2e900c6b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:48.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.994+0000 7f2e9719d700 1 --2- 172.21.15.31:0/757582029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e9006e0a0 0x7f2e900c6b20 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f2e9006a400 tx=0x7f2e8c009250 comp rx=0 tx=0).stop 2023-10-06T02:24:48.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.994+0000 7f2e9719d700 1 -- 172.21.15.31:0/757582029 shutdown_connections 2023-10-06T02:24:48.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.994+0000 7f2e9719d700 1 --2- 172.21.15.31:0/757582029 >> v2:172.21.15.62:3300/0 conn(0x7f2e90150380 0x7f2e9006a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:48.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.994+0000 7f2e9719d700 1 --2- 172.21.15.31:0/757582029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e9006e0a0 0x7f2e900c6b20 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:48.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.994+0000 7f2e9719d700 1 -- 172.21.15.31:0/757582029 >> 172.21.15.31:0/757582029 conn(0x7f2e9006d170 msgr2=0x7f2e900cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:48.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.994+0000 7f2e9719d700 1 -- 172.21.15.31:0/757582029 shutdown_connections 2023-10-06T02:24:48.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.995+0000 7f2e9719d700 1 -- 172.21.15.31:0/757582029 wait complete. 2023-10-06T02:24:49.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.996+0000 7f2e9719d700 1 Processor -- start 2023-10-06T02:24:49.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.996+0000 7f2e9719d700 1 -- start start 2023-10-06T02:24:49.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.997+0000 7f2e9719d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e9006e0a0 0x7f2e900c3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:49.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.997+0000 7f2e9719d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2e9006f3a0 0x7f2e9013eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:49.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.997+0000 7f2e9719d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e90150380 0x7f2e900b0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:49.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.997+0000 7f2e9719d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e90156170 con 0x7f2e90150380 2023-10-06T02:24:49.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.997+0000 7f2e9719d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e90156cf0 con 0x7f2e9006e0a0 2023-10-06T02:24:49.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.997+0000 7f2e9719d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e901563c0 con 0x7f2e9006f3a0 2023-10-06T02:24:49.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.997+0000 7f2e9699c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e90150380 0x7f2e900b0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:49.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.997+0000 7f2e9699c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e90150380 0x7f2e900b0db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53284/0 (socket says 172.21.15.31:53284) 2023-10-06T02:24:49.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.997+0000 7f2e9699c700 1 -- 172.21.15.31:0/620285232 learned_addr learned my addr 172.21.15.31:0/620285232 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:49.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.997+0000 7f2e9619b700 1 --2- 172.21.15.31:0/620285232 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e9006e0a0 0x7f2e900c3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:49.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.997+0000 7f2e9599a700 1 --2- 172.21.15.31:0/620285232 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2e9006f3a0 0x7f2e9013eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:49.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.998+0000 7f2e9699c700 1 -- 172.21.15.31:0/620285232 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2e9006f3a0 msgr2=0x7f2e9013eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:49.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.998+0000 7f2e9699c700 1 --2- 172.21.15.31:0/620285232 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2e9006f3a0 0x7f2e9013eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:49.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.998+0000 7f2e9699c700 1 -- 172.21.15.31:0/620285232 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e9006e0a0 msgr2=0x7f2e900c3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:49.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.998+0000 7f2e9699c700 1 --2- 172.21.15.31:0/620285232 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e9006e0a0 0x7f2e900c3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:49.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.998+0000 7f2e9699c700 1 -- 172.21.15.31:0/620285232 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e8c0094f0 con 0x7f2e90150380 2023-10-06T02:24:49.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.998+0000 7f2e9619b700 1 --2- 172.21.15.31:0/620285232 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e9006e0a0 0x7f2e900c3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:24:49.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.998+0000 7f2e9599a700 1 --2- 172.21.15.31:0/620285232 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2e9006f3a0 0x7f2e9013eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:24:49.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.998+0000 7f2e9699c700 1 --2- 172.21.15.31:0/620285232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e90150380 0x7f2e900b0db0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f2e88007ef0 tx=0x7f2e88009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:49.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.998+0000 7f2e867fc700 1 -- 172.21.15.31:0/620285232 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e8800ec10 con 0x7f2e90150380 2023-10-06T02:24:49.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.998+0000 7f2e867fc700 1 -- 172.21.15.31:0/620285232 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e8800ed70 con 0x7f2e90150380 2023-10-06T02:24:49.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.998+0000 7f2e9719d700 1 -- 172.21.15.31:0/620285232 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e901566a0 con 0x7f2e90150380 2023-10-06T02:24:49.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.998+0000 7f2e867fc700 1 -- 172.21.15.31:0/620285232 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e88014670 con 0x7f2e90150380 2023-10-06T02:24:49.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.998+0000 7f2e9719d700 1 -- 172.21.15.31:0/620285232 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e900c4510 con 0x7f2e90150380 2023-10-06T02:24:49.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:48.999+0000 7f2e9719d700 1 -- 172.21.15.31:0/620285232 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e9006e750 con 0x7f2e90150380 2023-10-06T02:24:49.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.000+0000 7f2e867fc700 1 -- 172.21.15.31:0/620285232 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f2e88014a60 con 0x7f2e90150380 2023-10-06T02:24:49.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.001+0000 7f2e867fc700 1 --2- 172.21.15.31:0/620285232 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f2e740670e0 0x7f2e74069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:49.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.002+0000 7f2e867fc700 1 -- 172.21.15.31:0/620285232 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2e88085130 con 0x7f2e90150380 2023-10-06T02:24:49.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.002+0000 7f2e9619b700 1 --2- 172.21.15.31:0/620285232 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f2e740670e0 0x7f2e74069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:49.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.002+0000 7f2e9619b700 1 --2- 172.21.15.31:0/620285232 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f2e740670e0 0x7f2e74069590 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2e9006a400 tx=0x7f2e8c017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:49.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.006+0000 7f2e867fc700 1 -- 172.21.15.31:0/620285232 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2e88051f20 con 0x7f2e90150380 2023-10-06T02:24:49.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.202+0000 7f2e9719d700 1 -- 172.21.15.31:0/620285232 --> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2e900648e0 con 0x7f2e740670e0 2023-10-06T02:24:49.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.211+0000 7f2e867fc700 1 -- 172.21.15.31:0/620285232 <== mgr.4107 v2:172.21.15.31:6832/85292 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311038 (secure 0 0 0) 0x7f2e900648e0 con 0x7f2e740670e0 2023-10-06T02:24:49.215 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:24:49.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.216+0000 7f2e9719d700 1 -- 172.21.15.31:0/620285232 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f2e740670e0 msgr2=0x7f2e74069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:49.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.216+0000 7f2e9719d700 1 --2- 172.21.15.31:0/620285232 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f2e740670e0 0x7f2e74069590 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2e9006a400 tx=0x7f2e8c017000 comp rx=0 tx=0).stop 2023-10-06T02:24:49.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.216+0000 7f2e9719d700 1 -- 172.21.15.31:0/620285232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e90150380 msgr2=0x7f2e900b0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:49.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.216+0000 7f2e9719d700 1 --2- 172.21.15.31:0/620285232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e90150380 0x7f2e900b0db0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f2e88007ef0 tx=0x7f2e88009840 comp rx=0 tx=0).stop 2023-10-06T02:24:49.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.216+0000 7f2e9719d700 1 -- 172.21.15.31:0/620285232 shutdown_connections 2023-10-06T02:24:49.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.216+0000 7f2e9719d700 1 --2- 172.21.15.31:0/620285232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e90150380 0x7f2e900b0db0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:49.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.216+0000 7f2e9719d700 1 --2- 172.21.15.31:0/620285232 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e9006e0a0 0x7f2e900c3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:49.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.216+0000 7f2e9719d700 1 --2- 172.21.15.31:0/620285232 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f2e740670e0 0x7f2e74069590 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:49.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.216+0000 7f2e9719d700 1 --2- 172.21.15.31:0/620285232 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2e9006f3a0 0x7f2e9013eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:49.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.216+0000 7f2e9719d700 1 -- 172.21.15.31:0/620285232 >> 172.21.15.31:0/620285232 conn(0x7f2e9006d170 msgr2=0x7f2e901596d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:49.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.216+0000 7f2e9719d700 1 -- 172.21.15.31:0/620285232 shutdown_connections 2023-10-06T02:24:49.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.216+0000 7f2e9719d700 1 -- 172.21.15.31:0/620285232 wait complete. 2023-10-06T02:24:49.219 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-10-06T02:24:49.234 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":98,"stamp":"2023-10-06T02:24:49.136633+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2383848,"kb_used_data":5544,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752590872,"statfs":{"total":773094113280,"available":770653052928,"internally_reserved":0,"allocated":5677056,"data_stored":3073836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001094"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.404251+0000","last_change":"2023-10-06T02:24:44.399776+0000","last_active":"2023-10-06T02:24:44.404251+0000","last_peered":"2023-10-06T02:24:44.404251+0000","last_clean":"2023-10-06T02:24:44.404251+0000","last_became_active":"2023-10-06T02:24:44.399295+0000","last_became_peered":"2023-10-06T02:24:44.399295+0000","last_unstale":"2023-10-06T02:24:44.404251+0000","last_undegraded":"2023-10-06T02:24:44.404251+0000","last_fullsized":"2023-10-06T02:24:44.404251+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:36:22.792574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.417123+0000","last_change":"2023-10-06T02:24:44.412894+0000","last_active":"2023-10-06T02:24:44.417123+0000","last_peered":"2023-10-06T02:24:44.417123+0000","last_clean":"2023-10-06T02:24:44.417123+0000","last_became_active":"2023-10-06T02:24:44.412631+0000","last_became_peered":"2023-10-06T02:24:44.412631+0000","last_unstale":"2023-10-06T02:24:44.417123+0000","last_undegraded":"2023-10-06T02:24:44.417123+0000","last_fullsized":"2023-10-06T02:24:44.417123+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:52:53.772376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.415196+0000","last_change":"2023-10-06T02:24:44.404559+0000","last_active":"2023-10-06T02:24:44.415196+0000","last_peered":"2023-10-06T02:24:44.415196+0000","last_clean":"2023-10-06T02:24:44.415196+0000","last_became_active":"2023-10-06T02:24:44.404247+0000","last_became_peered":"2023-10-06T02:24:44.404247+0000","last_unstale":"2023-10-06T02:24:44.415196+0000","last_undegraded":"2023-10-06T02:24:44.415196+0000","last_fullsized":"2023-10-06T02:24:44.415196+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:08:36.507186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.417132+0000","last_change":"2023-10-06T02:24:44.415515+0000","last_active":"2023-10-06T02:24:44.417132+0000","last_peered":"2023-10-06T02:24:44.417132+0000","last_clean":"2023-10-06T02:24:44.417132+0000","last_became_active":"2023-10-06T02:24:44.415174+0000","last_became_peered":"2023-10-06T02:24:44.415174+0000","last_unstale":"2023-10-06T02:24:44.417132+0000","last_undegraded":"2023-10-06T02:24:44.417132+0000","last_fullsized":"2023-10-06T02:24:44.417132+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:08:41.842740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.405599+0000","last_change":"2023-10-06T02:24:44.404169+0000","last_active":"2023-10-06T02:24:44.405599+0000","last_peered":"2023-10-06T02:24:44.405599+0000","last_clean":"2023-10-06T02:24:44.405599+0000","last_became_active":"2023-10-06T02:24:44.403729+0000","last_became_peered":"2023-10-06T02:24:44.403729+0000","last_unstale":"2023-10-06T02:24:44.405599+0000","last_undegraded":"2023-10-06T02:24:44.405599+0000","last_fullsized":"2023-10-06T02:24:44.405599+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:45:00.714802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.411397+0000","last_change":"2023-10-06T02:24:44.402453+0000","last_active":"2023-10-06T02:24:44.411397+0000","last_peered":"2023-10-06T02:24:44.411397+0000","last_clean":"2023-10-06T02:24:44.411397+0000","last_became_active":"2023-10-06T02:24:44.402201+0000","last_became_peered":"2023-10-06T02:24:44.402201+0000","last_unstale":"2023-10-06T02:24:44.411397+0000","last_undegraded":"2023-10-06T02:24:44.411397+0000","last_fullsized":"2023-10-06T02:24:44.411397+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:02.100208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.415141+0000","last_change":"2023-10-06T02:24:44.404479+0000","last_active":"2023-10-06T02:24:44.415141+0000","last_peered":"2023-10-06T02:24:44.415141+0000","last_clean":"2023-10-06T02:24:44.415141+0000","last_became_active":"2023-10-06T02:24:44.404159+0000","last_became_peered":"2023-10-06T02:24:44.404159+0000","last_unstale":"2023-10-06T02:24:44.415141+0000","last_undegraded":"2023-10-06T02:24:44.415141+0000","last_fullsized":"2023-10-06T02:24:44.415141+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:38:41.410751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.418091+0000","last_change":"2023-10-06T02:24:44.410537+0000","last_active":"2023-10-06T02:24:44.418091+0000","last_peered":"2023-10-06T02:24:44.418091+0000","last_clean":"2023-10-06T02:24:44.418091+0000","last_became_active":"2023-10-06T02:24:44.410254+0000","last_became_peered":"2023-10-06T02:24:44.410254+0000","last_unstale":"2023-10-06T02:24:44.418091+0000","last_undegraded":"2023-10-06T02:24:44.418091+0000","last_fullsized":"2023-10-06T02:24:44.418091+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:08:36.507186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.418050+0000","last_change":"2023-10-06T02:24:44.412830+0000","last_active":"2023-10-06T02:24:44.418050+0000","last_peered":"2023-10-06T02:24:44.418050+0000","last_clean":"2023-10-06T02:24:44.418050+0000","last_became_active":"2023-10-06T02:24:44.410237+0000","last_became_peered":"2023-10-06T02:24:44.410237+0000","last_unstale":"2023-10-06T02:24:44.418050+0000","last_undegraded":"2023-10-06T02:24:44.418050+0000","last_fullsized":"2023-10-06T02:24:44.418050+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:42:13.276208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.411320+0000","last_change":"2023-10-06T02:24:44.407544+0000","last_active":"2023-10-06T02:24:44.411320+0000","last_peered":"2023-10-06T02:24:44.411320+0000","last_clean":"2023-10-06T02:24:44.411320+0000","last_became_active":"2023-10-06T02:24:44.406366+0000","last_became_peered":"2023-10-06T02:24:44.406366+0000","last_unstale":"2023-10-06T02:24:44.411320+0000","last_undegraded":"2023-10-06T02:24:44.411320+0000","last_fullsized":"2023-10-06T02:24:44.411320+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:05:53.763719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.409606+0000","last_change":"2023-10-06T02:24:44.403960+0000","last_active":"2023-10-06T02:24:44.409606+0000","last_peered":"2023-10-06T02:24:44.409606+0000","last_clean":"2023-10-06T02:24:44.409606+0000","last_became_active":"2023-10-06T02:24:44.403693+0000","last_became_peered":"2023-10-06T02:24:44.403693+0000","last_unstale":"2023-10-06T02:24:44.409606+0000","last_undegraded":"2023-10-06T02:24:44.409606+0000","last_fullsized":"2023-10-06T02:24:44.409606+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:42:51.527496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.407150+0000","last_change":"2023-10-06T02:24:44.403053+0000","last_active":"2023-10-06T02:24:44.407150+0000","last_peered":"2023-10-06T02:24:44.407150+0000","last_clean":"2023-10-06T02:24:44.407150+0000","last_became_active":"2023-10-06T02:24:44.402728+0000","last_became_peered":"2023-10-06T02:24:44.402728+0000","last_unstale":"2023-10-06T02:24:44.407150+0000","last_undegraded":"2023-10-06T02:24:44.407150+0000","last_fullsized":"2023-10-06T02:24:44.407150+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:16:37.690649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.412513+0000","last_change":"2023-10-06T02:24:44.408306+0000","last_active":"2023-10-06T02:24:44.412513+0000","last_peered":"2023-10-06T02:24:44.412513+0000","last_clean":"2023-10-06T02:24:44.412513+0000","last_became_active":"2023-10-06T02:24:44.408011+0000","last_became_peered":"2023-10-06T02:24:44.408011+0000","last_unstale":"2023-10-06T02:24:44.412513+0000","last_undegraded":"2023-10-06T02:24:44.412513+0000","last_fullsized":"2023-10-06T02:24:44.412513+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:20:38.743641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.417233+0000","last_change":"2023-10-06T02:24:44.405527+0000","last_active":"2023-10-06T02:24:44.417233+0000","last_peered":"2023-10-06T02:24:44.417233+0000","last_clean":"2023-10-06T02:24:44.417233+0000","last_became_active":"2023-10-06T02:24:44.405198+0000","last_became_peered":"2023-10-06T02:24:44.405198+0000","last_unstale":"2023-10-06T02:24:44.417233+0000","last_undegraded":"2023-10-06T02:24:44.417233+0000","last_fullsized":"2023-10-06T02:24:44.417233+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:57:31.016724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.407726+0000","last_change":"2023-10-06T02:24:44.404243+0000","last_active":"2023-10-06T02:24:44.407726+0000","last_peered":"2023-10-06T02:24:44.407726+0000","last_clean":"2023-10-06T02:24:44.407726+0000","last_became_active":"2023-10-06T02:24:44.403945+0000","last_became_peered":"2023-10-06T02:24:44.403945+0000","last_unstale":"2023-10-06T02:24:44.407726+0000","last_undegraded":"2023-10-06T02:24:44.407726+0000","last_fullsized":"2023-10-06T02:24:44.407726+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:44:28.775166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.412522+0000","last_change":"2023-10-06T02:24:44.408246+0000","last_active":"2023-10-06T02:24:44.412522+0000","last_peered":"2023-10-06T02:24:44.412522+0000","last_clean":"2023-10-06T02:24:44.412522+0000","last_became_active":"2023-10-06T02:24:44.407929+0000","last_became_peered":"2023-10-06T02:24:44.407929+0000","last_unstale":"2023-10-06T02:24:44.412522+0000","last_undegraded":"2023-10-06T02:24:44.412522+0000","last_fullsized":"2023-10-06T02:24:44.412522+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:20:20.076261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.415151+0000","last_change":"2023-10-06T02:24:44.404461+0000","last_active":"2023-10-06T02:24:44.415151+0000","last_peered":"2023-10-06T02:24:44.415151+0000","last_clean":"2023-10-06T02:24:44.415151+0000","last_became_active":"2023-10-06T02:24:44.404145+0000","last_became_peered":"2023-10-06T02:24:44.404145+0000","last_unstale":"2023-10-06T02:24:44.415151+0000","last_undegraded":"2023-10-06T02:24:44.415151+0000","last_fullsized":"2023-10-06T02:24:44.415151+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:16:37.690649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.417147+0000","last_change":"2023-10-06T02:24:44.413490+0000","last_active":"2023-10-06T02:24:44.417147+0000","last_peered":"2023-10-06T02:24:44.417147+0000","last_clean":"2023-10-06T02:24:44.417147+0000","last_became_active":"2023-10-06T02:24:44.413155+0000","last_became_peered":"2023-10-06T02:24:44.413155+0000","last_unstale":"2023-10-06T02:24:44.417147+0000","last_undegraded":"2023-10-06T02:24:44.417147+0000","last_fullsized":"2023-10-06T02:24:44.417147+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:00:52.624896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.416841+0000","last_change":"2023-10-06T02:24:44.407078+0000","last_active":"2023-10-06T02:24:44.416841+0000","last_peered":"2023-10-06T02:24:44.416841+0000","last_clean":"2023-10-06T02:24:44.416841+0000","last_became_active":"2023-10-06T02:24:44.406801+0000","last_became_peered":"2023-10-06T02:24:44.406801+0000","last_unstale":"2023-10-06T02:24:44.416841+0000","last_undegraded":"2023-10-06T02:24:44.416841+0000","last_fullsized":"2023-10-06T02:24:44.416841+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:07:20.911006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.417074+0000","last_change":"2023-10-06T02:24:44.411390+0000","last_active":"2023-10-06T02:24:44.417074+0000","last_peered":"2023-10-06T02:24:44.417074+0000","last_clean":"2023-10-06T02:24:44.417074+0000","last_became_active":"2023-10-06T02:24:44.408383+0000","last_became_peered":"2023-10-06T02:24:44.408383+0000","last_unstale":"2023-10-06T02:24:44.417074+0000","last_undegraded":"2023-10-06T02:24:44.417074+0000","last_fullsized":"2023-10-06T02:24:44.417074+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:57:31.089002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.402397+0000","last_change":"2023-10-06T02:24:44.400636+0000","last_active":"2023-10-06T02:24:44.402397+0000","last_peered":"2023-10-06T02:24:44.402397+0000","last_clean":"2023-10-06T02:24:44.402397+0000","last_became_active":"2023-10-06T02:24:44.398526+0000","last_became_peered":"2023-10-06T02:24:44.398526+0000","last_unstale":"2023-10-06T02:24:44.402397+0000","last_undegraded":"2023-10-06T02:24:44.402397+0000","last_fullsized":"2023-10-06T02:24:44.402397+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:17:54.998340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.417841+0000","last_change":"2023-10-06T02:24:44.411279+0000","last_active":"2023-10-06T02:24:44.417841+0000","last_peered":"2023-10-06T02:24:44.417841+0000","last_clean":"2023-10-06T02:24:44.417841+0000","last_became_active":"2023-10-06T02:24:44.411004+0000","last_became_peered":"2023-10-06T02:24:44.411004+0000","last_unstale":"2023-10-06T02:24:44.417841+0000","last_undegraded":"2023-10-06T02:24:44.417841+0000","last_fullsized":"2023-10-06T02:24:44.417841+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:57:31.016724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.406711+0000","last_change":"2023-10-06T02:24:44.401430+0000","last_active":"2023-10-06T02:24:44.406711+0000","last_peered":"2023-10-06T02:24:44.406711+0000","last_clean":"2023-10-06T02:24:44.406711+0000","last_became_active":"2023-10-06T02:24:44.401019+0000","last_became_peered":"2023-10-06T02:24:44.401019+0000","last_unstale":"2023-10-06T02:24:44.406711+0000","last_undegraded":"2023-10-06T02:24:44.406711+0000","last_fullsized":"2023-10-06T02:24:44.406711+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:20:23.617375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.409231+0000","last_change":"2023-10-06T02:24:44.396267+0000","last_active":"2023-10-06T02:24:44.409231+0000","last_peered":"2023-10-06T02:24:44.409231+0000","last_clean":"2023-10-06T02:24:44.409231+0000","last_became_active":"2023-10-06T02:24:44.396000+0000","last_became_peered":"2023-10-06T02:24:44.396000+0000","last_unstale":"2023-10-06T02:24:44.409231+0000","last_undegraded":"2023-10-06T02:24:44.409231+0000","last_fullsized":"2023-10-06T02:24:44.409231+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:09:18.667331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.415217+0000","last_change":"2023-10-06T02:24:44.404857+0000","last_active":"2023-10-06T02:24:44.415217+0000","last_peered":"2023-10-06T02:24:44.415217+0000","last_clean":"2023-10-06T02:24:44.415217+0000","last_became_active":"2023-10-06T02:24:44.403648+0000","last_became_peered":"2023-10-06T02:24:44.403648+0000","last_unstale":"2023-10-06T02:24:44.415217+0000","last_undegraded":"2023-10-06T02:24:44.415217+0000","last_fullsized":"2023-10-06T02:24:44.415217+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:42:13.276208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.405606+0000","last_change":"2023-10-06T02:24:44.397371+0000","last_active":"2023-10-06T02:24:44.405606+0000","last_peered":"2023-10-06T02:24:44.405606+0000","last_clean":"2023-10-06T02:24:44.405606+0000","last_became_active":"2023-10-06T02:24:44.397039+0000","last_became_peered":"2023-10-06T02:24:44.397039+0000","last_unstale":"2023-10-06T02:24:44.405606+0000","last_undegraded":"2023-10-06T02:24:44.405606+0000","last_fullsized":"2023-10-06T02:24:44.405606+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:03:37.509527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.415908+0000","last_change":"2023-10-06T02:24:44.408073+0000","last_active":"2023-10-06T02:24:44.415908+0000","last_peered":"2023-10-06T02:24:44.415908+0000","last_clean":"2023-10-06T02:24:44.415908+0000","last_became_active":"2023-10-06T02:24:44.407798+0000","last_became_peered":"2023-10-06T02:24:44.407798+0000","last_unstale":"2023-10-06T02:24:44.415908+0000","last_undegraded":"2023-10-06T02:24:44.415908+0000","last_fullsized":"2023-10-06T02:24:44.415908+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:16:18.991303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.398703+0000","last_change":"2023-10-06T02:24:44.387954+0000","last_active":"2023-10-06T02:24:44.398703+0000","last_peered":"2023-10-06T02:24:44.398703+0000","last_clean":"2023-10-06T02:24:44.398703+0000","last_became_active":"2023-10-06T02:24:44.387621+0000","last_became_peered":"2023-10-06T02:24:44.387621+0000","last_unstale":"2023-10-06T02:24:44.398703+0000","last_undegraded":"2023-10-06T02:24:44.398703+0000","last_fullsized":"2023-10-06T02:24:44.398703+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:52:14.830100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.415377+0000","last_change":"2023-10-06T02:24:44.405197+0000","last_active":"2023-10-06T02:24:44.415377+0000","last_peered":"2023-10-06T02:24:44.415377+0000","last_clean":"2023-10-06T02:24:44.415377+0000","last_became_active":"2023-10-06T02:24:44.404858+0000","last_became_peered":"2023-10-06T02:24:44.404858+0000","last_unstale":"2023-10-06T02:24:44.415377+0000","last_undegraded":"2023-10-06T02:24:44.415377+0000","last_fullsized":"2023-10-06T02:24:44.415377+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:13.213754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.417052+0000","last_change":"2023-10-06T02:24:44.415088+0000","last_active":"2023-10-06T02:24:44.417052+0000","last_peered":"2023-10-06T02:24:44.417052+0000","last_clean":"2023-10-06T02:24:44.417052+0000","last_became_active":"2023-10-06T02:24:44.414623+0000","last_became_peered":"2023-10-06T02:24:44.414623+0000","last_unstale":"2023-10-06T02:24:44.417052+0000","last_undegraded":"2023-10-06T02:24:44.417052+0000","last_fullsized":"2023-10-06T02:24:44.417052+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:23:08.084287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.415341+0000","last_change":"2023-10-06T02:24:44.405038+0000","last_active":"2023-10-06T02:24:44.415341+0000","last_peered":"2023-10-06T02:24:44.415341+0000","last_clean":"2023-10-06T02:24:44.415341+0000","last_became_active":"2023-10-06T02:24:44.404618+0000","last_became_peered":"2023-10-06T02:24:44.404618+0000","last_unstale":"2023-10-06T02:24:44.415341+0000","last_undegraded":"2023-10-06T02:24:44.415341+0000","last_fullsized":"2023-10-06T02:24:44.415341+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:45:00.714802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.418802+0000","last_change":"2023-10-06T02:24:44.414847+0000","last_active":"2023-10-06T02:24:44.418802+0000","last_peered":"2023-10-06T02:24:44.418802+0000","last_clean":"2023-10-06T02:24:44.418802+0000","last_became_active":"2023-10-06T02:24:44.414589+0000","last_became_peered":"2023-10-06T02:24:44.414589+0000","last_unstale":"2023-10-06T02:24:44.418802+0000","last_undegraded":"2023-10-06T02:24:44.418802+0000","last_fullsized":"2023-10-06T02:24:44.418802+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:08:41.842740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.421709+0000","last_change":"2023-10-06T02:24:44.416356+0000","last_active":"2023-10-06T02:24:44.421709+0000","last_peered":"2023-10-06T02:24:44.421709+0000","last_clean":"2023-10-06T02:24:44.421709+0000","last_became_active":"2023-10-06T02:24:44.416063+0000","last_became_peered":"2023-10-06T02:24:44.416063+0000","last_unstale":"2023-10-06T02:24:44.421709+0000","last_undegraded":"2023-10-06T02:24:44.421709+0000","last_fullsized":"2023-10-06T02:24:44.421709+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:09:18.667331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"46'4","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.422963+0000","last_change":"2023-10-06T02:24:43.399938+0000","last_active":"2023-10-06T02:24:44.422963+0000","last_peered":"2023-10-06T02:24:44.422963+0000","last_clean":"2023-10-06T02:24:44.422963+0000","last_became_active":"2023-10-06T02:24:43.399658+0000","last_became_peered":"2023-10-06T02:24:43.399658+0000","last_unstale":"2023-10-06T02:24:44.422963+0000","last_undegraded":"2023-10-06T02:24:44.422963+0000","last_fullsized":"2023-10-06T02:24:44.422963+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_clean_scrub_stamp":"2023-10-06T02:24:42.362895+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:27:24.819106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.417690+0000","last_change":"2023-10-06T02:24:44.402054+0000","last_active":"2023-10-06T02:24:44.417690+0000","last_peered":"2023-10-06T02:24:44.417690+0000","last_clean":"2023-10-06T02:24:44.417690+0000","last_became_active":"2023-10-06T02:24:44.400866+0000","last_became_peered":"2023-10-06T02:24:44.400866+0000","last_unstale":"2023-10-06T02:24:44.417690+0000","last_undegraded":"2023-10-06T02:24:44.417690+0000","last_fullsized":"2023-10-06T02:24:44.417690+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:44:28.775166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"46'1","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.415112+0000","last_change":"2023-10-06T02:24:43.402928+0000","last_active":"2023-10-06T02:24:44.415112+0000","last_peered":"2023-10-06T02:24:44.415112+0000","last_clean":"2023-10-06T02:24:44.415112+0000","last_became_active":"2023-10-06T02:24:43.402631+0000","last_became_peered":"2023-10-06T02:24:43.402631+0000","last_unstale":"2023-10-06T02:24:44.415112+0000","last_undegraded":"2023-10-06T02:24:44.415112+0000","last_fullsized":"2023-10-06T02:24:44.415112+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_clean_scrub_stamp":"2023-10-06T02:24:42.362895+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:08:50.452800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.403758+0000","last_change":"2023-10-06T02:24:44.401653+0000","last_active":"2023-10-06T02:24:44.403758+0000","last_peered":"2023-10-06T02:24:44.403758+0000","last_clean":"2023-10-06T02:24:44.403758+0000","last_became_active":"2023-10-06T02:24:44.401366+0000","last_became_peered":"2023-10-06T02:24:44.401366+0000","last_unstale":"2023-10-06T02:24:44.403758+0000","last_undegraded":"2023-10-06T02:24:44.403758+0000","last_fullsized":"2023-10-06T02:24:44.403758+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:31.956185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"46'2","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.411944+0000","last_change":"2023-10-06T02:24:43.400245+0000","last_active":"2023-10-06T02:24:44.411944+0000","last_peered":"2023-10-06T02:24:44.411944+0000","last_clean":"2023-10-06T02:24:44.411944+0000","last_became_active":"2023-10-06T02:24:43.399997+0000","last_became_peered":"2023-10-06T02:24:43.399997+0000","last_unstale":"2023-10-06T02:24:44.411944+0000","last_undegraded":"2023-10-06T02:24:44.411944+0000","last_fullsized":"2023-10-06T02:24:44.411944+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_clean_scrub_stamp":"2023-10-06T02:24:42.362895+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:00:29.295321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.414234+0000","last_change":"2023-10-06T02:24:44.406850+0000","last_active":"2023-10-06T02:24:44.414234+0000","last_peered":"2023-10-06T02:24:44.414234+0000","last_clean":"2023-10-06T02:24:44.414234+0000","last_became_active":"2023-10-06T02:24:44.406437+0000","last_became_peered":"2023-10-06T02:24:44.406437+0000","last_unstale":"2023-10-06T02:24:44.414234+0000","last_undegraded":"2023-10-06T02:24:44.414234+0000","last_fullsized":"2023-10-06T02:24:44.414234+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:45:55.429682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"46'2","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.415280+0000","last_change":"2023-10-06T02:24:43.394724+0000","last_active":"2023-10-06T02:24:44.415280+0000","last_peered":"2023-10-06T02:24:44.415280+0000","last_clean":"2023-10-06T02:24:44.415280+0000","last_became_active":"2023-10-06T02:24:43.394447+0000","last_became_peered":"2023-10-06T02:24:43.394447+0000","last_unstale":"2023-10-06T02:24:44.415280+0000","last_undegraded":"2023-10-06T02:24:44.415280+0000","last_fullsized":"2023-10-06T02:24:44.415280+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_clean_scrub_stamp":"2023-10-06T02:24:42.362895+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:05:52.756921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.422248+0000","last_change":"2023-10-06T02:24:44.410905+0000","last_active":"2023-10-06T02:24:44.422248+0000","last_peered":"2023-10-06T02:24:44.422248+0000","last_clean":"2023-10-06T02:24:44.422248+0000","last_became_active":"2023-10-06T02:24:44.410408+0000","last_became_peered":"2023-10-06T02:24:44.410408+0000","last_unstale":"2023-10-06T02:24:44.422248+0000","last_undegraded":"2023-10-06T02:24:44.422248+0000","last_fullsized":"2023-10-06T02:24:44.422248+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:44:28.775166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"46'1","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.414664+0000","last_change":"2023-10-06T02:24:43.402545+0000","last_active":"2023-10-06T02:24:44.414664+0000","last_peered":"2023-10-06T02:24:44.414664+0000","last_clean":"2023-10-06T02:24:44.414664+0000","last_became_active":"2023-10-06T02:24:43.402214+0000","last_became_peered":"2023-10-06T02:24:43.402214+0000","last_unstale":"2023-10-06T02:24:44.414664+0000","last_undegraded":"2023-10-06T02:24:44.414664+0000","last_fullsized":"2023-10-06T02:24:44.414664+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_clean_scrub_stamp":"2023-10-06T02:24:42.362895+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:36:21.785776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.417265+0000","last_change":"2023-10-06T02:24:44.415441+0000","last_active":"2023-10-06T02:24:44.417265+0000","last_peered":"2023-10-06T02:24:44.417265+0000","last_clean":"2023-10-06T02:24:44.417265+0000","last_became_active":"2023-10-06T02:24:44.415151+0000","last_became_peered":"2023-10-06T02:24:44.415151+0000","last_unstale":"2023-10-06T02:24:44.417265+0000","last_undegraded":"2023-10-06T02:24:44.417265+0000","last_fullsized":"2023-10-06T02:24:44.417265+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:11:36.193886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"46'1","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.414170+0000","last_change":"2023-10-06T02:24:43.399959+0000","last_active":"2023-10-06T02:24:44.414170+0000","last_peered":"2023-10-06T02:24:44.414170+0000","last_clean":"2023-10-06T02:24:44.414170+0000","last_became_active":"2023-10-06T02:24:43.399690+0000","last_became_peered":"2023-10-06T02:24:43.399690+0000","last_unstale":"2023-10-06T02:24:44.414170+0000","last_undegraded":"2023-10-06T02:24:44.414170+0000","last_fullsized":"2023-10-06T02:24:44.414170+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_clean_scrub_stamp":"2023-10-06T02:24:42.362895+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:05:52.756921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.406156+0000","last_change":"2023-10-06T02:24:44.397459+0000","last_active":"2023-10-06T02:24:44.406156+0000","last_peered":"2023-10-06T02:24:44.406156+0000","last_clean":"2023-10-06T02:24:44.406156+0000","last_became_active":"2023-10-06T02:24:44.397083+0000","last_became_peered":"2023-10-06T02:24:44.397083+0000","last_unstale":"2023-10-06T02:24:44.406156+0000","last_undegraded":"2023-10-06T02:24:44.406156+0000","last_fullsized":"2023-10-06T02:24:44.406156+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:21:28.131186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.382668+0000","last_change":"2023-10-06T02:24:43.397423+0000","last_active":"2023-10-06T02:24:44.382668+0000","last_peered":"2023-10-06T02:24:44.382668+0000","last_clean":"2023-10-06T02:24:44.382668+0000","last_became_active":"2023-10-06T02:24:43.397121+0000","last_became_peered":"2023-10-06T02:24:43.397121+0000","last_unstale":"2023-10-06T02:24:44.382668+0000","last_undegraded":"2023-10-06T02:24:44.382668+0000","last_fullsized":"2023-10-06T02:24:44.382668+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_clean_scrub_stamp":"2023-10-06T02:24:42.362895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:23:33.992242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.422253+0000","last_change":"2023-10-06T02:24:44.418357+0000","last_active":"2023-10-06T02:24:44.422253+0000","last_peered":"2023-10-06T02:24:44.422253+0000","last_clean":"2023-10-06T02:24:44.422253+0000","last_became_active":"2023-10-06T02:24:44.417994+0000","last_became_peered":"2023-10-06T02:24:44.417994+0000","last_unstale":"2023-10-06T02:24:44.422253+0000","last_undegraded":"2023-10-06T02:24:44.422253+0000","last_fullsized":"2023-10-06T02:24:44.422253+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:45:00.714802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.391302+0000","last_change":"2023-10-06T02:24:43.402695+0000","last_active":"2023-10-06T02:24:44.391302+0000","last_peered":"2023-10-06T02:24:44.391302+0000","last_clean":"2023-10-06T02:24:44.391302+0000","last_became_active":"2023-10-06T02:24:43.402400+0000","last_became_peered":"2023-10-06T02:24:43.402400+0000","last_unstale":"2023-10-06T02:24:44.391302+0000","last_undegraded":"2023-10-06T02:24:44.391302+0000","last_fullsized":"2023-10-06T02:24:44.391302+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_clean_scrub_stamp":"2023-10-06T02:24:42.362895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:18:35.870113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.422397+0000","last_change":"2023-10-06T02:24:44.412473+0000","last_active":"2023-10-06T02:24:44.422397+0000","last_peered":"2023-10-06T02:24:44.422397+0000","last_clean":"2023-10-06T02:24:44.422397+0000","last_became_active":"2023-10-06T02:24:44.412121+0000","last_became_peered":"2023-10-06T02:24:44.412121+0000","last_unstale":"2023-10-06T02:24:44.422397+0000","last_undegraded":"2023-10-06T02:24:44.422397+0000","last_fullsized":"2023-10-06T02:24:44.422397+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:08:41.842740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.394424+0000","last_change":"2023-10-06T02:24:43.397626+0000","last_active":"2023-10-06T02:24:44.394424+0000","last_peered":"2023-10-06T02:24:44.394424+0000","last_clean":"2023-10-06T02:24:44.394424+0000","last_became_active":"2023-10-06T02:24:43.397311+0000","last_became_peered":"2023-10-06T02:24:43.397311+0000","last_unstale":"2023-10-06T02:24:44.394424+0000","last_undegraded":"2023-10-06T02:24:44.394424+0000","last_fullsized":"2023-10-06T02:24:44.394424+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_clean_scrub_stamp":"2023-10-06T02:24:42.362895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:47:23.440398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.417710+0000","last_change":"2023-10-06T02:24:44.406263+0000","last_active":"2023-10-06T02:24:44.417710+0000","last_peered":"2023-10-06T02:24:44.417710+0000","last_clean":"2023-10-06T02:24:44.417710+0000","last_became_active":"2023-10-06T02:24:44.405963+0000","last_became_peered":"2023-10-06T02:24:44.405963+0000","last_unstale":"2023-10-06T02:24:44.417710+0000","last_undegraded":"2023-10-06T02:24:44.417710+0000","last_fullsized":"2023-10-06T02:24:44.417710+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:52:53.772376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.396801+0000","last_change":"2023-10-06T02:24:43.404447+0000","last_active":"2023-10-06T02:24:44.396801+0000","last_peered":"2023-10-06T02:24:44.396801+0000","last_clean":"2023-10-06T02:24:44.396801+0000","last_became_active":"2023-10-06T02:24:43.404142+0000","last_became_peered":"2023-10-06T02:24:43.404142+0000","last_unstale":"2023-10-06T02:24:44.396801+0000","last_undegraded":"2023-10-06T02:24:44.396801+0000","last_fullsized":"2023-10-06T02:24:44.396801+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_clean_scrub_stamp":"2023-10-06T02:24:42.362895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:49:49.997672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.417253+0000","last_change":"2023-10-06T02:24:44.406132+0000","last_active":"2023-10-06T02:24:44.417253+0000","last_peered":"2023-10-06T02:24:44.417253+0000","last_clean":"2023-10-06T02:24:44.417253+0000","last_became_active":"2023-10-06T02:24:44.405866+0000","last_became_peered":"2023-10-06T02:24:44.405866+0000","last_unstale":"2023-10-06T02:24:44.417253+0000","last_undegraded":"2023-10-06T02:24:44.417253+0000","last_fullsized":"2023-10-06T02:24:44.417253+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:15:52.636635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"46'3","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.415621+0000","last_change":"2023-10-06T02:24:43.406933+0000","last_active":"2023-10-06T02:24:44.415621+0000","last_peered":"2023-10-06T02:24:44.415621+0000","last_clean":"2023-10-06T02:24:44.415621+0000","last_became_active":"2023-10-06T02:24:43.406623+0000","last_became_peered":"2023-10-06T02:24:43.406623+0000","last_unstale":"2023-10-06T02:24:44.415621+0000","last_undegraded":"2023-10-06T02:24:44.415621+0000","last_fullsized":"2023-10-06T02:24:44.415621+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_clean_scrub_stamp":"2023-10-06T02:24:42.362895+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:20:19.069462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.405582+0000","last_change":"2023-10-06T02:24:44.403494+0000","last_active":"2023-10-06T02:24:44.405582+0000","last_peered":"2023-10-06T02:24:44.405582+0000","last_clean":"2023-10-06T02:24:44.405582+0000","last_became_active":"2023-10-06T02:24:44.403155+0000","last_became_peered":"2023-10-06T02:24:44.403155+0000","last_unstale":"2023-10-06T02:24:44.405582+0000","last_undegraded":"2023-10-06T02:24:44.405582+0000","last_fullsized":"2023-10-06T02:24:44.405582+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:00:52.624896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"46'6","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.431814+0000","last_change":"2023-10-06T02:24:43.400919+0000","last_active":"2023-10-06T02:24:44.431814+0000","last_peered":"2023-10-06T02:24:44.431814+0000","last_clean":"2023-10-06T02:24:44.431814+0000","last_became_active":"2023-10-06T02:24:43.400550+0000","last_became_peered":"2023-10-06T02:24:43.400550+0000","last_unstale":"2023-10-06T02:24:44.431814+0000","last_undegraded":"2023-10-06T02:24:44.431814+0000","last_fullsized":"2023-10-06T02:24:44.431814+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_clean_scrub_stamp":"2023-10-06T02:24:42.362895+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:01.093410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.416776+0000","last_change":"2023-10-06T02:24:44.402080+0000","last_active":"2023-10-06T02:24:44.416776+0000","last_peered":"2023-10-06T02:24:44.416776+0000","last_clean":"2023-10-06T02:24:44.416776+0000","last_became_active":"2023-10-06T02:24:44.401802+0000","last_became_peered":"2023-10-06T02:24:44.401802+0000","last_unstale":"2023-10-06T02:24:44.416776+0000","last_undegraded":"2023-10-06T02:24:44.416776+0000","last_fullsized":"2023-10-06T02:24:44.416776+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:09:18.667331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"46'2","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.409632+0000","last_change":"2023-10-06T02:24:43.394568+0000","last_active":"2023-10-06T02:24:44.409632+0000","last_peered":"2023-10-06T02:24:44.409632+0000","last_clean":"2023-10-06T02:24:44.409632+0000","last_became_active":"2023-10-06T02:24:43.394278+0000","last_became_peered":"2023-10-06T02:24:43.394278+0000","last_unstale":"2023-10-06T02:24:44.409632+0000","last_undegraded":"2023-10-06T02:24:44.409632+0000","last_fullsized":"2023-10-06T02:24:44.409632+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_clean_scrub_stamp":"2023-10-06T02:24:42.362895+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:20:19.069462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.416759+0000","last_change":"2023-10-06T02:24:44.405537+0000","last_active":"2023-10-06T02:24:44.416759+0000","last_peered":"2023-10-06T02:24:44.416759+0000","last_clean":"2023-10-06T02:24:44.416759+0000","last_became_active":"2023-10-06T02:24:44.405241+0000","last_became_peered":"2023-10-06T02:24:44.405241+0000","last_unstale":"2023-10-06T02:24:44.416759+0000","last_undegraded":"2023-10-06T02:24:44.416759+0000","last_fullsized":"2023-10-06T02:24:44.416759+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:03:39.641500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.383425+0000","last_change":"2023-10-06T02:24:43.402945+0000","last_active":"2023-10-06T02:24:44.383425+0000","last_peered":"2023-10-06T02:24:44.383425+0000","last_clean":"2023-10-06T02:24:44.383425+0000","last_became_active":"2023-10-06T02:24:43.402528+0000","last_became_peered":"2023-10-06T02:24:43.402528+0000","last_unstale":"2023-10-06T02:24:44.383425+0000","last_undegraded":"2023-10-06T02:24:44.383425+0000","last_fullsized":"2023-10-06T02:24:44.383425+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_clean_scrub_stamp":"2023-10-06T02:24:42.362895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:13:30.476079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.407102+0000","last_change":"2023-10-06T02:24:44.402029+0000","last_active":"2023-10-06T02:24:44.407102+0000","last_peered":"2023-10-06T02:24:44.407102+0000","last_clean":"2023-10-06T02:24:44.407102+0000","last_became_active":"2023-10-06T02:24:44.401632+0000","last_became_peered":"2023-10-06T02:24:44.401632+0000","last_unstale":"2023-10-06T02:24:44.407102+0000","last_undegraded":"2023-10-06T02:24:44.407102+0000","last_fullsized":"2023-10-06T02:24:44.407102+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:50:39.544674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.395951+0000","last_change":"2023-10-06T02:24:43.400720+0000","last_active":"2023-10-06T02:24:44.395951+0000","last_peered":"2023-10-06T02:24:44.395951+0000","last_clean":"2023-10-06T02:24:44.395951+0000","last_became_active":"2023-10-06T02:24:43.399743+0000","last_became_peered":"2023-10-06T02:24:43.399743+0000","last_unstale":"2023-10-06T02:24:44.395951+0000","last_undegraded":"2023-10-06T02:24:44.395951+0000","last_fullsized":"2023-10-06T02:24:44.395951+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_clean_scrub_stamp":"2023-10-06T02:24:42.362895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:05:52.756921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.417121+0000","last_change":"2023-10-06T02:24:44.405453+0000","last_active":"2023-10-06T02:24:44.417121+0000","last_peered":"2023-10-06T02:24:44.417121+0000","last_clean":"2023-10-06T02:24:44.417121+0000","last_became_active":"2023-10-06T02:24:44.405169+0000","last_became_peered":"2023-10-06T02:24:44.405169+0000","last_unstale":"2023-10-06T02:24:44.417121+0000","last_undegraded":"2023-10-06T02:24:44.417121+0000","last_fullsized":"2023-10-06T02:24:44.417121+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:45:55.429682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"46'8","reported_seq":18,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.430269+0000","last_change":"2023-10-06T02:24:43.402209+0000","last_active":"2023-10-06T02:24:44.430269+0000","last_peered":"2023-10-06T02:24:44.430269+0000","last_clean":"2023-10-06T02:24:44.430269+0000","last_became_active":"2023-10-06T02:24:43.401903+0000","last_became_peered":"2023-10-06T02:24:43.401903+0000","last_unstale":"2023-10-06T02:24:44.430269+0000","last_undegraded":"2023-10-06T02:24:44.430269+0000","last_fullsized":"2023-10-06T02:24:44.430269+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_clean_scrub_stamp":"2023-10-06T02:24:42.362895+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:20:37.736843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.406988+0000","last_change":"2023-10-06T02:24:44.397984+0000","last_active":"2023-10-06T02:24:44.406988+0000","last_peered":"2023-10-06T02:24:44.406988+0000","last_clean":"2023-10-06T02:24:44.406988+0000","last_became_active":"2023-10-06T02:24:44.397662+0000","last_became_peered":"2023-10-06T02:24:44.397662+0000","last_unstale":"2023-10-06T02:24:44.406988+0000","last_undegraded":"2023-10-06T02:24:44.406988+0000","last_fullsized":"2023-10-06T02:24:44.406988+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:30:49.790238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"46'1","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.418328+0000","last_change":"2023-10-06T02:24:43.391052+0000","last_active":"2023-10-06T02:24:44.418328+0000","last_peered":"2023-10-06T02:24:44.418328+0000","last_clean":"2023-10-06T02:24:44.418328+0000","last_became_active":"2023-10-06T02:24:43.390782+0000","last_became_peered":"2023-10-06T02:24:43.390782+0000","last_unstale":"2023-10-06T02:24:44.418328+0000","last_undegraded":"2023-10-06T02:24:44.418328+0000","last_fullsized":"2023-10-06T02:24:44.418328+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_clean_scrub_stamp":"2023-10-06T02:24:42.362895+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:49:49.997672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.411344+0000","last_change":"2023-10-06T02:24:44.406495+0000","last_active":"2023-10-06T02:24:44.411344+0000","last_peered":"2023-10-06T02:24:44.411344+0000","last_clean":"2023-10-06T02:24:44.411344+0000","last_became_active":"2023-10-06T02:24:44.406287+0000","last_became_peered":"2023-10-06T02:24:44.406287+0000","last_unstale":"2023-10-06T02:24:44.411344+0000","last_undegraded":"2023-10-06T02:24:44.411344+0000","last_fullsized":"2023-10-06T02:24:44.411344+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:49:51.004471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"46'2","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.420856+0000","last_change":"2023-10-06T02:24:43.396106+0000","last_active":"2023-10-06T02:24:44.420856+0000","last_peered":"2023-10-06T02:24:44.420856+0000","last_clean":"2023-10-06T02:24:44.420856+0000","last_became_active":"2023-10-06T02:24:43.395864+0000","last_became_peered":"2023-10-06T02:24:43.395864+0000","last_unstale":"2023-10-06T02:24:44.420856+0000","last_undegraded":"2023-10-06T02:24:44.420856+0000","last_fullsized":"2023-10-06T02:24:44.420856+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_clean_scrub_stamp":"2023-10-06T02:24:42.362895+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:57:37.991959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.422276+0000","last_change":"2023-10-06T02:24:44.411445+0000","last_active":"2023-10-06T02:24:44.422276+0000","last_peered":"2023-10-06T02:24:44.422276+0000","last_clean":"2023-10-06T02:24:44.422276+0000","last_became_active":"2023-10-06T02:24:44.410615+0000","last_became_peered":"2023-10-06T02:24:44.410615+0000","last_unstale":"2023-10-06T02:24:44.422276+0000","last_undegraded":"2023-10-06T02:24:44.422276+0000","last_fullsized":"2023-10-06T02:24:44.422276+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:09:18.667331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"46'1","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.418954+0000","last_change":"2023-10-06T02:24:43.403713+0000","last_active":"2023-10-06T02:24:44.418954+0000","last_peered":"2023-10-06T02:24:44.418954+0000","last_clean":"2023-10-06T02:24:44.418954+0000","last_became_active":"2023-10-06T02:24:43.403457+0000","last_became_peered":"2023-10-06T02:24:43.403457+0000","last_unstale":"2023-10-06T02:24:44.418954+0000","last_undegraded":"2023-10-06T02:24:44.418954+0000","last_fullsized":"2023-10-06T02:24:44.418954+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_clean_scrub_stamp":"2023-10-06T02:24:42.362895+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:02:43.874000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.396519+0000","last_change":"2023-10-06T02:24:44.394140+0000","last_active":"2023-10-06T02:24:44.396519+0000","last_peered":"2023-10-06T02:24:44.396519+0000","last_clean":"2023-10-06T02:24:44.396519+0000","last_became_active":"2023-10-06T02:24:44.393852+0000","last_became_peered":"2023-10-06T02:24:44.393852+0000","last_unstale":"2023-10-06T02:24:44.396519+0000","last_undegraded":"2023-10-06T02:24:44.396519+0000","last_fullsized":"2023-10-06T02:24:44.396519+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:03:10.696687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.384728+0000","last_change":"2023-10-06T02:24:43.398488+0000","last_active":"2023-10-06T02:24:44.384728+0000","last_peered":"2023-10-06T02:24:44.384728+0000","last_clean":"2023-10-06T02:24:44.384728+0000","last_became_active":"2023-10-06T02:24:43.398028+0000","last_became_peered":"2023-10-06T02:24:43.398028+0000","last_unstale":"2023-10-06T02:24:44.384728+0000","last_undegraded":"2023-10-06T02:24:44.384728+0000","last_fullsized":"2023-10-06T02:24:44.384728+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_clean_scrub_stamp":"2023-10-06T02:24:42.362895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:31:14.485423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.417127+0000","last_change":"2023-10-06T02:24:44.410189+0000","last_active":"2023-10-06T02:24:44.417127+0000","last_peered":"2023-10-06T02:24:44.417127+0000","last_clean":"2023-10-06T02:24:44.417127+0000","last_became_active":"2023-10-06T02:24:44.409924+0000","last_became_peered":"2023-10-06T02:24:44.409924+0000","last_unstale":"2023-10-06T02:24:44.417127+0000","last_undegraded":"2023-10-06T02:24:44.417127+0000","last_fullsized":"2023-10-06T02:24:44.417127+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:45:00.714802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"46'2","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.417578+0000","last_change":"2023-10-06T02:24:43.400797+0000","last_active":"2023-10-06T02:24:44.417578+0000","last_peered":"2023-10-06T02:24:44.417578+0000","last_clean":"2023-10-06T02:24:44.417578+0000","last_became_active":"2023-10-06T02:24:43.400559+0000","last_became_peered":"2023-10-06T02:24:43.400559+0000","last_unstale":"2023-10-06T02:24:44.417578+0000","last_undegraded":"2023-10-06T02:24:44.417578+0000","last_fullsized":"2023-10-06T02:24:44.417578+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_clean_scrub_stamp":"2023-10-06T02:24:42.362895+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:08:17.467939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.409468+0000","last_change":"2023-10-06T02:24:44.407964+0000","last_active":"2023-10-06T02:24:44.409468+0000","last_peered":"2023-10-06T02:24:44.409468+0000","last_clean":"2023-10-06T02:24:44.409468+0000","last_became_active":"2023-10-06T02:24:44.407694+0000","last_became_peered":"2023-10-06T02:24:44.407694+0000","last_unstale":"2023-10-06T02:24:44.409468+0000","last_undegraded":"2023-10-06T02:24:44.409468+0000","last_fullsized":"2023-10-06T02:24:44.409468+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:49:10.668209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.385928+0000","last_change":"2023-10-06T02:24:43.398315+0000","last_active":"2023-10-06T02:24:44.385928+0000","last_peered":"2023-10-06T02:24:44.385928+0000","last_clean":"2023-10-06T02:24:44.385928+0000","last_became_active":"2023-10-06T02:24:43.398019+0000","last_became_peered":"2023-10-06T02:24:43.398019+0000","last_unstale":"2023-10-06T02:24:44.385928+0000","last_undegraded":"2023-10-06T02:24:44.385928+0000","last_fullsized":"2023-10-06T02:24:44.385928+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_clean_scrub_stamp":"2023-10-06T02:24:42.362895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:16:17.984505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.409322+0000","last_change":"2023-10-06T02:24:44.400889+0000","last_active":"2023-10-06T02:24:44.409322+0000","last_peered":"2023-10-06T02:24:44.409322+0000","last_clean":"2023-10-06T02:24:44.409322+0000","last_became_active":"2023-10-06T02:24:44.400646+0000","last_became_peered":"2023-10-06T02:24:44.400646+0000","last_unstale":"2023-10-06T02:24:44.409322+0000","last_undegraded":"2023-10-06T02:24:44.409322+0000","last_fullsized":"2023-10-06T02:24:44.409322+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:03:37.509527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.392877+0000","last_change":"2023-10-06T02:24:43.397398+0000","last_active":"2023-10-06T02:24:44.392877+0000","last_peered":"2023-10-06T02:24:44.392877+0000","last_clean":"2023-10-06T02:24:44.392877+0000","last_became_active":"2023-10-06T02:24:43.397140+0000","last_became_peered":"2023-10-06T02:24:43.397140+0000","last_unstale":"2023-10-06T02:24:44.392877+0000","last_undegraded":"2023-10-06T02:24:44.392877+0000","last_fullsized":"2023-10-06T02:24:44.392877+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_clean_scrub_stamp":"2023-10-06T02:24:42.362895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:20:37.736843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.411254+0000","last_change":"2023-10-06T02:24:44.403037+0000","last_active":"2023-10-06T02:24:44.411254+0000","last_peered":"2023-10-06T02:24:44.411254+0000","last_clean":"2023-10-06T02:24:44.411254+0000","last_became_active":"2023-10-06T02:24:44.402780+0000","last_became_peered":"2023-10-06T02:24:44.402780+0000","last_unstale":"2023-10-06T02:24:44.411254+0000","last_undegraded":"2023-10-06T02:24:44.411254+0000","last_fullsized":"2023-10-06T02:24:44.411254+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:47:24.447197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.381853+0000","last_change":"2023-10-06T02:24:43.395198+0000","last_active":"2023-10-06T02:24:44.381853+0000","last_peered":"2023-10-06T02:24:44.381853+0000","last_clean":"2023-10-06T02:24:44.381853+0000","last_became_active":"2023-10-06T02:24:43.394863+0000","last_became_peered":"2023-10-06T02:24:43.394863+0000","last_unstale":"2023-10-06T02:24:44.381853+0000","last_undegraded":"2023-10-06T02:24:44.381853+0000","last_fullsized":"2023-10-06T02:24:44.381853+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_clean_scrub_stamp":"2023-10-06T02:24:42.362895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:24:14.412112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.409303+0000","last_change":"2023-10-06T02:24:44.400280+0000","last_active":"2023-10-06T02:24:44.409303+0000","last_peered":"2023-10-06T02:24:44.409303+0000","last_clean":"2023-10-06T02:24:44.409303+0000","last_became_active":"2023-10-06T02:24:44.400011+0000","last_became_peered":"2023-10-06T02:24:44.400011+0000","last_unstale":"2023-10-06T02:24:44.409303+0000","last_undegraded":"2023-10-06T02:24:44.409303+0000","last_fullsized":"2023-10-06T02:24:44.409303+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:08:00.034308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.394143+0000","last_change":"2023-10-06T02:24:43.390877+0000","last_active":"2023-10-06T02:24:44.394143+0000","last_peered":"2023-10-06T02:24:44.394143+0000","last_clean":"2023-10-06T02:24:44.394143+0000","last_became_active":"2023-10-06T02:24:43.390571+0000","last_became_peered":"2023-10-06T02:24:43.390571+0000","last_unstale":"2023-10-06T02:24:44.394143+0000","last_undegraded":"2023-10-06T02:24:44.394143+0000","last_fullsized":"2023-10-06T02:24:44.394143+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_clean_scrub_stamp":"2023-10-06T02:24:42.362895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:20:19.069462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":33,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.390574+0000","last_change":"2023-10-06T02:23:02.054039+0000","last_active":"2023-10-06T02:24:44.390574+0000","last_peered":"2023-10-06T02:24:44.390574+0000","last_clean":"2023-10-06T02:24:44.390574+0000","last_became_active":"2023-10-06T02:23:02.053742+0000","last_became_peered":"2023-10-06T02:23:02.053742+0000","last_unstale":"2023-10-06T02:24:44.390574+0000","last_undegraded":"2023-10-06T02:24:44.390574+0000","last_fullsized":"2023-10-06T02:24:44.390574+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:58:18.846475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.418716+0000","last_change":"2023-10-06T02:24:44.413887+0000","last_active":"2023-10-06T02:24:44.418716+0000","last_peered":"2023-10-06T02:24:44.418716+0000","last_clean":"2023-10-06T02:24:44.418716+0000","last_became_active":"2023-10-06T02:24:44.413603+0000","last_became_peered":"2023-10-06T02:24:44.413603+0000","last_unstale":"2023-10-06T02:24:44.418716+0000","last_undegraded":"2023-10-06T02:24:44.418716+0000","last_fullsized":"2023-10-06T02:24:44.418716+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:57:13.406569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"46'2","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.418291+0000","last_change":"2023-10-06T02:24:43.403075+0000","last_active":"2023-10-06T02:24:44.418291+0000","last_peered":"2023-10-06T02:24:44.418291+0000","last_clean":"2023-10-06T02:24:44.418291+0000","last_became_active":"2023-10-06T02:24:43.402607+0000","last_became_peered":"2023-10-06T02:24:43.402607+0000","last_unstale":"2023-10-06T02:24:44.418291+0000","last_undegraded":"2023-10-06T02:24:44.418291+0000","last_fullsized":"2023-10-06T02:24:44.418291+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_clean_scrub_stamp":"2023-10-06T02:24:42.362895+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:36:47.623656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":37,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.383026+0000","last_change":"2023-10-06T02:23:01.988631+0000","last_active":"2023-10-06T02:24:44.383026+0000","last_peered":"2023-10-06T02:24:44.383026+0000","last_clean":"2023-10-06T02:24:44.383026+0000","last_became_active":"2023-10-06T02:23:01.988303+0000","last_became_peered":"2023-10-06T02:23:01.988303+0000","last_unstale":"2023-10-06T02:24:44.383026+0000","last_undegraded":"2023-10-06T02:24:44.383026+0000","last_fullsized":"2023-10-06T02:24:44.383026+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:38.211466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.417995+0000","last_change":"2023-10-06T02:24:44.414120+0000","last_active":"2023-10-06T02:24:44.417995+0000","last_peered":"2023-10-06T02:24:44.417995+0000","last_clean":"2023-10-06T02:24:44.417995+0000","last_became_active":"2023-10-06T02:24:44.412786+0000","last_became_peered":"2023-10-06T02:24:44.412786+0000","last_unstale":"2023-10-06T02:24:44.417995+0000","last_undegraded":"2023-10-06T02:24:44.417995+0000","last_fullsized":"2023-10-06T02:24:44.417995+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:08:00.034308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.393974+0000","last_change":"2023-10-06T02:24:43.399074+0000","last_active":"2023-10-06T02:24:44.393974+0000","last_peered":"2023-10-06T02:24:44.393974+0000","last_clean":"2023-10-06T02:24:44.393974+0000","last_became_active":"2023-10-06T02:24:43.398802+0000","last_became_peered":"2023-10-06T02:24:43.398802+0000","last_unstale":"2023-10-06T02:24:44.393974+0000","last_undegraded":"2023-10-06T02:24:44.393974+0000","last_fullsized":"2023-10-06T02:24:44.393974+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_clean_scrub_stamp":"2023-10-06T02:24:42.362895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:50:17.948474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":33,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.381850+0000","last_change":"2023-10-06T02:23:02.054338+0000","last_active":"2023-10-06T02:24:44.381850+0000","last_peered":"2023-10-06T02:24:44.381850+0000","last_clean":"2023-10-06T02:24:44.381850+0000","last_became_active":"2023-10-06T02:23:02.053991+0000","last_became_peered":"2023-10-06T02:23:02.053991+0000","last_unstale":"2023-10-06T02:24:44.381850+0000","last_undegraded":"2023-10-06T02:24:44.381850+0000","last_fullsized":"2023-10-06T02:24:44.381850+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:39:30.192838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.398748+0000","last_change":"2023-10-06T02:24:44.388048+0000","last_active":"2023-10-06T02:24:44.398748+0000","last_peered":"2023-10-06T02:24:44.398748+0000","last_clean":"2023-10-06T02:24:44.398748+0000","last_became_active":"2023-10-06T02:24:44.387702+0000","last_became_peered":"2023-10-06T02:24:44.387702+0000","last_unstale":"2023-10-06T02:24:44.398748+0000","last_undegraded":"2023-10-06T02:24:44.398748+0000","last_fullsized":"2023-10-06T02:24:44.398748+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:54:53.699040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.388324+0000","last_change":"2023-10-06T02:24:43.398254+0000","last_active":"2023-10-06T02:24:44.388324+0000","last_peered":"2023-10-06T02:24:44.388324+0000","last_clean":"2023-10-06T02:24:44.388324+0000","last_became_active":"2023-10-06T02:24:43.398001+0000","last_became_peered":"2023-10-06T02:24:43.398001+0000","last_unstale":"2023-10-06T02:24:44.388324+0000","last_undegraded":"2023-10-06T02:24:44.388324+0000","last_fullsized":"2023-10-06T02:24:44.388324+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_clean_scrub_stamp":"2023-10-06T02:24:42.362895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:47:23.440398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":37,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.382187+0000","last_change":"2023-10-06T02:23:01.988140+0000","last_active":"2023-10-06T02:24:44.382187+0000","last_peered":"2023-10-06T02:24:44.382187+0000","last_clean":"2023-10-06T02:24:44.382187+0000","last_became_active":"2023-10-06T02:23:01.987769+0000","last_became_peered":"2023-10-06T02:23:01.987769+0000","last_unstale":"2023-10-06T02:24:44.382187+0000","last_undegraded":"2023-10-06T02:24:44.382187+0000","last_fullsized":"2023-10-06T02:24:44.382187+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:20:08.391912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":185,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.395232+0000","last_change":"2023-10-06T02:22:55.032519+0000","last_active":"2023-10-06T02:24:44.395232+0000","last_peered":"2023-10-06T02:24:44.395232+0000","last_clean":"2023-10-06T02:24:44.395232+0000","last_became_active":"2023-10-06T02:22:55.031367+0000","last_became_peered":"2023-10-06T02:22:55.031367+0000","last_unstale":"2023-10-06T02:24:44.395232+0000","last_undegraded":"2023-10-06T02:24:44.395232+0000","last_fullsized":"2023-10-06T02:24:44.395232+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:22:53.309097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:22:53.309097+0000","last_clean_scrub_stamp":"2023-10-06T02:22:53.309097+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:53:46.520909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.401346+0000","last_change":"2023-10-06T02:24:44.397103+0000","last_active":"2023-10-06T02:24:44.401346+0000","last_peered":"2023-10-06T02:24:44.401346+0000","last_clean":"2023-10-06T02:24:44.401346+0000","last_became_active":"2023-10-06T02:24:44.396829+0000","last_became_peered":"2023-10-06T02:24:44.396829+0000","last_unstale":"2023-10-06T02:24:44.401346+0000","last_undegraded":"2023-10-06T02:24:44.401346+0000","last_fullsized":"2023-10-06T02:24:44.401346+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:18:36.876911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.398258+0000","last_change":"2023-10-06T02:24:43.404277+0000","last_active":"2023-10-06T02:24:44.398258+0000","last_peered":"2023-10-06T02:24:44.398258+0000","last_clean":"2023-10-06T02:24:44.398258+0000","last_became_active":"2023-10-06T02:24:43.403997+0000","last_became_peered":"2023-10-06T02:24:43.403997+0000","last_unstale":"2023-10-06T02:24:44.398258+0000","last_undegraded":"2023-10-06T02:24:44.398258+0000","last_fullsized":"2023-10-06T02:24:44.398258+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_clean_scrub_stamp":"2023-10-06T02:24:42.362895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:50:17.948474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":32,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.395336+0000","last_change":"2023-10-06T02:23:01.990773+0000","last_active":"2023-10-06T02:24:44.395336+0000","last_peered":"2023-10-06T02:24:44.395336+0000","last_clean":"2023-10-06T02:24:44.395336+0000","last_became_active":"2023-10-06T02:23:01.990511+0000","last_became_peered":"2023-10-06T02:23:01.990511+0000","last_unstale":"2023-10-06T02:24:44.395336+0000","last_undegraded":"2023-10-06T02:24:44.395336+0000","last_fullsized":"2023-10-06T02:24:44.395336+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:42:54.797731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.405350+0000","last_change":"2023-10-06T02:24:44.396710+0000","last_active":"2023-10-06T02:24:44.405350+0000","last_peered":"2023-10-06T02:24:44.405350+0000","last_clean":"2023-10-06T02:24:44.405350+0000","last_became_active":"2023-10-06T02:24:44.396400+0000","last_became_peered":"2023-10-06T02:24:44.396400+0000","last_unstale":"2023-10-06T02:24:44.405350+0000","last_undegraded":"2023-10-06T02:24:44.405350+0000","last_fullsized":"2023-10-06T02:24:44.405350+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:38:10.363655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.382564+0000","last_change":"2023-10-06T02:24:43.398658+0000","last_active":"2023-10-06T02:24:44.382564+0000","last_peered":"2023-10-06T02:24:44.382564+0000","last_clean":"2023-10-06T02:24:44.382564+0000","last_became_active":"2023-10-06T02:24:43.398372+0000","last_became_peered":"2023-10-06T02:24:43.398372+0000","last_unstale":"2023-10-06T02:24:44.382564+0000","last_undegraded":"2023-10-06T02:24:44.382564+0000","last_fullsized":"2023-10-06T02:24:44.382564+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_clean_scrub_stamp":"2023-10-06T02:24:42.362895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:20:19.069462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":32,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.398360+0000","last_change":"2023-10-06T02:23:02.054069+0000","last_active":"2023-10-06T02:24:44.398360+0000","last_peered":"2023-10-06T02:24:44.398360+0000","last_clean":"2023-10-06T02:24:44.398360+0000","last_became_active":"2023-10-06T02:23:02.053739+0000","last_became_peered":"2023-10-06T02:23:02.053739+0000","last_unstale":"2023-10-06T02:24:44.398360+0000","last_undegraded":"2023-10-06T02:24:44.398360+0000","last_fullsized":"2023-10-06T02:24:44.398360+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:57:48.822608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.415286+0000","last_change":"2023-10-06T02:24:44.404983+0000","last_active":"2023-10-06T02:24:44.415286+0000","last_peered":"2023-10-06T02:24:44.415286+0000","last_clean":"2023-10-06T02:24:44.415286+0000","last_became_active":"2023-10-06T02:24:44.404711+0000","last_became_peered":"2023-10-06T02:24:44.404711+0000","last_unstale":"2023-10-06T02:24:44.415286+0000","last_undegraded":"2023-10-06T02:24:44.415286+0000","last_fullsized":"2023-10-06T02:24:44.415286+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:34:36.784699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.390176+0000","last_change":"2023-10-06T02:24:43.399159+0000","last_active":"2023-10-06T02:24:44.390176+0000","last_peered":"2023-10-06T02:24:44.390176+0000","last_clean":"2023-10-06T02:24:44.390176+0000","last_became_active":"2023-10-06T02:24:43.398888+0000","last_became_peered":"2023-10-06T02:24:43.398888+0000","last_unstale":"2023-10-06T02:24:44.390176+0000","last_undegraded":"2023-10-06T02:24:44.390176+0000","last_fullsized":"2023-10-06T02:24:44.390176+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_clean_scrub_stamp":"2023-10-06T02:24:42.362895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:36:21.785776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":33,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.393228+0000","last_change":"2023-10-06T02:23:01.989934+0000","last_active":"2023-10-06T02:24:44.393228+0000","last_peered":"2023-10-06T02:24:44.393228+0000","last_clean":"2023-10-06T02:24:44.393228+0000","last_became_active":"2023-10-06T02:23:01.989706+0000","last_became_peered":"2023-10-06T02:23:01.989706+0000","last_unstale":"2023-10-06T02:24:44.393228+0000","last_undegraded":"2023-10-06T02:24:44.393228+0000","last_fullsized":"2023-10-06T02:24:44.393228+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:06.212251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.417096+0000","last_change":"2023-10-06T02:24:44.406497+0000","last_active":"2023-10-06T02:24:44.417096+0000","last_peered":"2023-10-06T02:24:44.417096+0000","last_clean":"2023-10-06T02:24:44.417096+0000","last_became_active":"2023-10-06T02:24:44.406212+0000","last_became_peered":"2023-10-06T02:24:44.406212+0000","last_unstale":"2023-10-06T02:24:44.417096+0000","last_undegraded":"2023-10-06T02:24:44.417096+0000","last_fullsized":"2023-10-06T02:24:44.417096+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:43.369693+0000","last_clean_scrub_stamp":"2023-10-06T02:24:43.369693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:00:52.624896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.385928+0000","last_change":"2023-10-06T02:24:43.399336+0000","last_active":"2023-10-06T02:24:44.385928+0000","last_peered":"2023-10-06T02:24:44.385928+0000","last_clean":"2023-10-06T02:24:44.385928+0000","last_became_active":"2023-10-06T02:24:43.399074+0000","last_became_peered":"2023-10-06T02:24:43.399074+0000","last_unstale":"2023-10-06T02:24:44.385928+0000","last_undegraded":"2023-10-06T02:24:44.385928+0000","last_fullsized":"2023-10-06T02:24:44.385928+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:24:42.362895+0000","last_clean_scrub_stamp":"2023-10-06T02:24:42.362895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:54:29.943579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":33,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:24:44.382656+0000","last_change":"2023-10-06T02:23:01.992935+0000","last_active":"2023-10-06T02:24:44.382656+0000","last_peered":"2023-10-06T02:24:44.382656+0000","last_clean":"2023-10-06T02:24:44.382656+0000","last_became_active":"2023-10-06T02:23:01.992568+0000","last_became_peered":"2023-10-06T02:23:01.992568+0000","last_unstale":"2023-10-06T02:24:44.382656+0000","last_undegraded":"2023-10-06T02:24:44.382656+0000","last_fullsized":"2023-10-06T02:24:44.382656+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:06.212251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574875,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298416,"kb_used_data":1128,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073424,"statfs":{"total":96636764160,"available":96331186176,"internally_reserved":0,"allocated":1155072,"data_stored":826553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.46200000000000002}]},{"osd":1,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.53000000000000003}]},{"osd":2,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.51400000000000001}]},{"osd":3,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.59999999999999998}]},{"osd":4,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.625},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.80600000000000005}]},{"osd":5,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.70599999999999996}]},{"osd":6,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.86199999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574875,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297836,"kb_used_data":548,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074004,"statfs":{"total":96636764160,"available":96331780096,"internally_reserved":0,"allocated":561152,"data_stored":236685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.58599999999999997}]},{"osd":1,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.69499999999999995}]},{"osd":2,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.80500000000000005}]},{"osd":3,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.747}]},{"osd":4,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.877}]},{"osd":5,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.85899999999999999}]},{"osd":7,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.458,"5min":1.458,"15min":1.458},"last":0.999},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.95999999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574874,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297836,"kb_used_data":548,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074004,"statfs":{"total":96636764160,"available":96331780096,"internally_reserved":0,"allocated":561152,"data_stored":236221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.788,"5min":1.788,"15min":1.788},"last":0.33500000000000002}]},{"osd":2,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":2.0840000000000001,"5min":2.0840000000000001,"15min":2.0840000000000001},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.501}]},{"osd":3,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.861,"5min":1.861,"15min":1.861},"last":0.438}]},{"osd":4,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.9279999999999999,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":2.0179999999999998,"5min":2.0179999999999998,"15min":2.0179999999999998},"last":0.77700000000000002}]},{"osd":5,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.9830000000000001,"5min":1.9830000000000001,"15min":1.9830000000000001},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.60899999999999999}]},{"osd":6,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.829,"5min":1.829,"15min":1.829},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":2.052,"5min":2.052,"15min":2.052},"last":0.71599999999999997}]},{"osd":7,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.895,"5min":1.895,"15min":1.895},"last":0.73299999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574875,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298416,"kb_used_data":1128,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073424,"statfs":{"total":96636764160,"available":96331186176,"internally_reserved":0,"allocated":1155072,"data_stored":826485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.49099999999999999}]},{"osd":2,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.44500000000000001}]},{"osd":3,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.54500000000000004}]},{"osd":4,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.626},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.304,"5min":1.304,"15min":1.304},"last":0.81799999999999995}]},{"osd":5,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.63700000000000001}]},{"osd":6,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.76600000000000001}]},{"osd":7,"last update":"Fri Oct 6 02:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.77600000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574876,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297840,"kb_used_data":552,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074000,"statfs":{"total":96636764160,"available":96331776000,"internally_reserved":0,"allocated":565248,"data_stored":237597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.34799999999999998}]},{"osd":1,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":0.39900000000000002}]},{"osd":3,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.53000000000000003}]},{"osd":4,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.621,"5min":1.621,"15min":1.621},"last":1.0129999999999999}]},{"osd":5,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":1.05}]},{"osd":6,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.98599999999999999}]},{"osd":7,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.90400000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574873,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297844,"kb_used_data":556,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073996,"statfs":{"total":96636764160,"available":96331771904,"internally_reserved":0,"allocated":569344,"data_stored":237619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":2.024,"5min":2.024,"15min":2.024},"last":0.38},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":2.181,"5min":2.181,"15min":2.181},"last":0.34899999999999998}]},{"osd":1,"last update":"Fri Oct 6 02:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":4.492,"5min":4.492,"15min":4.492},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":4.569,"5min":4.569,"15min":4.569},"last":0.505}]},{"osd":2,"last update":"Fri Oct 6 02:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":2.0870000000000002,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":2.4260000000000002,"5min":2.4260000000000002,"15min":2.4260000000000002},"last":0.58599999999999997}]},{"osd":4,"last update":"Fri Oct 6 02:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":2.214,"5min":2.214,"15min":2.214},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.91300000000000003}]},{"osd":5,"last update":"Fri Oct 6 02:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":2.242,"5min":2.242,"15min":2.242},"last":0.79500000000000004}]},{"osd":6,"last update":"Fri Oct 6 02:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.357,"5min":1.357,"15min":1.357},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.95499999999999996}]},{"osd":7,"last update":"Fri Oct 6 02:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.27,"5min":2.27,"15min":2.27},"last":0.96599999999999997},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.81899999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574875,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297832,"kb_used_data":544,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074008,"statfs":{"total":96636764160,"available":96331784192,"internally_reserved":0,"allocated":557056,"data_stored":236615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:23:55 2023","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":12.337,"5min":12.337,"15min":12.337},"last":0.77700000000000002},{"interface":"front","average":{"1min":1.02,"5min":1.02,"15min":1.02},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":9.6539999999999999,"5min":9.6539999999999999,"15min":9.6539999999999999},"last":0.81699999999999995}]},{"osd":1,"last update":"Fri Oct 6 02:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":3.181,"5min":3.181,"15min":3.181},"last":0.84399999999999997},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":12.946999999999999,"5min":12.946999999999999,"15min":12.946999999999999},"last":0.86199999999999999}]},{"osd":2,"last update":"Fri Oct 6 02:23:55 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":12.377000000000001,"5min":12.377000000000001,"15min":12.377000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":6.3220000000000001,"5min":6.3220000000000001,"15min":6.3220000000000001},"last":0.82499999999999996}]},{"osd":3,"last update":"Fri Oct 6 02:23:55 2023","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":9.7720000000000002,"5min":9.7720000000000002,"15min":9.7720000000000002},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":3.0840000000000001,"5min":3.0840000000000001,"15min":3.0840000000000001},"last":0.73099999999999998}]},{"osd":5,"last update":"Fri Oct 6 02:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":6.2690000000000001,"5min":6.2690000000000001,"15min":6.2690000000000001},"last":0.66300000000000003},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":10.769,"5min":10.769,"15min":10.769},"last":0.60199999999999998}]},{"osd":6,"last update":"Fri Oct 6 02:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":3.1349999999999998,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":0.57899999999999996},{"interface":"front","average":{"1min":1.077,"5min":1.077,"15min":1.077},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":8.0899999999999999,"5min":8.0899999999999999,"15min":8.0899999999999999},"last":0.74299999999999999}]},{"osd":7,"last update":"Fri Oct 6 02:23:55 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":9.7089999999999996,"5min":9.7089999999999996,"15min":9.7089999999999996},"last":0.79200000000000004},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":7.6719999999999997,"5min":7.6719999999999997,"15min":7.6719999999999997},"last":0.80300000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574875,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297828,"kb_used_data":540,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074012,"statfs":{"total":96636764160,"available":96331788288,"internally_reserved":0,"allocated":552960,"data_stored":236061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.54900000000000004}]},{"osd":1,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.496},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.52700000000000002}]},{"osd":2,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.64100000000000001}]},{"osd":3,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.71699999999999997}]},{"osd":4,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.42499999999999999}]},{"osd":6,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.73299999999999998}]},{"osd":7,"last update":"Fri Oct 6 02:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.76100000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T02:24:49.243 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T02:24:49.243 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T02:24:49.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.469+0000 7f088f59d700 1 Processor -- start 2023-10-06T02:24:49.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.469+0000 7f088f59d700 1 -- start start 2023-10-06T02:24:49.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.470+0000 7f088f59d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f08880cb3f0 0x7f0888159750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:49.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.470+0000 7f088f59d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f08880cea00 0x7f08880cee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:49.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.470+0000 7f088f59d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08880ccfa0 con 0x7f0888159380 2023-10-06T02:24:49.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.470+0000 7f088f59d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08880cf3c0 con 0x7f08880cb3f0 2023-10-06T02:24:49.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.470+0000 7f088f59d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08880cb880 con 0x7f08880cea00 2023-10-06T02:24:49.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.471+0000 7f088dd9a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f08880cb3f0 0x7f0888159750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:49.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.471+0000 7f088dd9a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f08880cb3f0 0x7f0888159750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53290/0 (socket says 172.21.15.31:53290) 2023-10-06T02:24:49.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.471+0000 7f088dd9a700 1 -- 172.21.15.31:0/884257659 learned_addr learned my addr 172.21.15.31:0/884257659 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:49.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.471+0000 7f088ed9c700 1 --2- 172.21.15.31:0/884257659 >> v2:172.21.15.62:3300/0 conn(0x7f08880cea00 0x7f08880cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:49.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.471+0000 7f088dd9a700 1 -- 172.21.15.31:0/884257659 >> v2:172.21.15.62:3300/0 conn(0x7f08880cea00 msgr2=0x7f08880cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:49.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.471+0000 7f088dd9a700 1 --2- 172.21.15.31:0/884257659 >> v2:172.21.15.62:3300/0 conn(0x7f08880cea00 0x7f08880cee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:49.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.471+0000 7f088dd9a700 1 -- 172.21.15.31:0/884257659 >> v1:172.21.15.62:6789/0 conn(0x7f0888159380 legacy=0x7f088815dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:49.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.471+0000 7f088dd9a700 1 -- 172.21.15.31:0/884257659 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08880cbb60 con 0x7f08880cb3f0 2023-10-06T02:24:49.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.471+0000 7f088dd9a700 1 --2- 172.21.15.31:0/884257659 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08880cb3f0 0x7f0888159750 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f08880d0420 tx=0x7f0878009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5c7b7fb3ee3a8da server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:49.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.471+0000 7f088d599700 1 -- 172.21.15.31:0/884257659 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087800b820 con 0x7f08880cb3f0 2023-10-06T02:24:49.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.472+0000 7f088d599700 1 -- 172.21.15.31:0/884257659 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f087800b980 con 0x7f08880cb3f0 2023-10-06T02:24:49.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.472+0000 7f088d599700 1 -- 172.21.15.31:0/884257659 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087800bd20 con 0x7f08880cb3f0 2023-10-06T02:24:49.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.472+0000 7f088f59d700 1 -- 172.21.15.31:0/884257659 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08880cb3f0 msgr2=0x7f0888159750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:49.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.472+0000 7f088f59d700 1 --2- 172.21.15.31:0/884257659 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08880cb3f0 0x7f0888159750 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f08880d0420 tx=0x7f0878009250 comp rx=0 tx=0).stop 2023-10-06T02:24:49.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.472+0000 7f088f59d700 1 -- 172.21.15.31:0/884257659 shutdown_connections 2023-10-06T02:24:49.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.472+0000 7f088f59d700 1 --2- 172.21.15.31:0/884257659 >> v2:172.21.15.62:3300/0 conn(0x7f08880cea00 0x7f08880cee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:49.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.472+0000 7f088f59d700 1 --2- 172.21.15.31:0/884257659 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08880cb3f0 0x7f0888159750 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:49.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.472+0000 7f088f59d700 1 -- 172.21.15.31:0/884257659 >> 172.21.15.31:0/884257659 conn(0x7f08880703a0 msgr2=0x7f08880707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:49.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.472+0000 7f088f59d700 1 -- 172.21.15.31:0/884257659 shutdown_connections 2023-10-06T02:24:49.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.472+0000 7f088f59d700 1 -- 172.21.15.31:0/884257659 wait complete. 2023-10-06T02:24:49.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.474+0000 7f088f59d700 1 Processor -- start 2023-10-06T02:24:49.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.474+0000 7f088f59d700 1 -- start start 2023-10-06T02:24:49.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.475+0000 7f088f59d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08880cb3f0 0x7f0888159cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:49.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.475+0000 7f088f59d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f08880cea00 0x7f088816a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:49.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.475+0000 7f088f59d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0888159380 0x7f08880c9560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:49.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.475+0000 7f088f59d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08880c7ec0 con 0x7f08880cb3f0 2023-10-06T02:24:49.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.475+0000 7f088f59d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f088816aab0 con 0x7f0888159380 2023-10-06T02:24:49.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.475+0000 7f088f59d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f088816d370 con 0x7f08880cea00 2023-10-06T02:24:49.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.475+0000 7f088e59b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08880cb3f0 0x7f0888159cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:49.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.475+0000 7f088e59b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08880cb3f0 0x7f0888159cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53294/0 (socket says 172.21.15.31:53294) 2023-10-06T02:24:49.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.475+0000 7f088e59b700 1 -- 172.21.15.31:0/4290760356 learned_addr learned my addr 172.21.15.31:0/4290760356 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:49.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.475+0000 7f088dd9a700 1 --2- 172.21.15.31:0/4290760356 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f08880cea00 0x7f088816a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:49.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.475+0000 7f088ed9c700 1 --2- 172.21.15.31:0/4290760356 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0888159380 0x7f08880c9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:49.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.475+0000 7f088e59b700 1 -- 172.21.15.31:0/4290760356 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f08880cea00 msgr2=0x7f088816a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:49.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.475+0000 7f088e59b700 1 --2- 172.21.15.31:0/4290760356 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f08880cea00 0x7f088816a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:49.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.475+0000 7f088e59b700 1 -- 172.21.15.31:0/4290760356 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0888159380 msgr2=0x7f08880c9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:49.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.475+0000 7f088e59b700 1 --2- 172.21.15.31:0/4290760356 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0888159380 0x7f08880c9560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:49.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.475+0000 7f088e59b700 1 -- 172.21.15.31:0/4290760356 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08780094f0 con 0x7f08880cb3f0 2023-10-06T02:24:49.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.476+0000 7f088e59b700 1 --2- 172.21.15.31:0/4290760356 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08880cb3f0 0x7f0888159cb0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f088400b5e0 tx=0x7f0884009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:49.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.476+0000 7f087e7fc700 1 -- 172.21.15.31:0/4290760356 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f088400f510 con 0x7f08880cb3f0 2023-10-06T02:24:49.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.476+0000 7f087e7fc700 1 -- 172.21.15.31:0/4290760356 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f088400f690 con 0x7f08880cb3f0 2023-10-06T02:24:49.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.476+0000 7f087e7fc700 1 -- 172.21.15.31:0/4290760356 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f088400fa50 con 0x7f08880cb3f0 2023-10-06T02:24:49.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.476+0000 7f088f59d700 1 -- 172.21.15.31:0/4290760356 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08880c9aa0 con 0x7f08880cb3f0 2023-10-06T02:24:49.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.476+0000 7f088f59d700 1 -- 172.21.15.31:0/4290760356 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08880c7be0 con 0x7f08880cb3f0 2023-10-06T02:24:49.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.477+0000 7f088f59d700 1 -- 172.21.15.31:0/4290760356 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08880cf140 con 0x7f08880cb3f0 2023-10-06T02:24:49.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.478+0000 7f087e7fc700 1 -- 172.21.15.31:0/4290760356 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f088401a400 con 0x7f08880cb3f0 2023-10-06T02:24:49.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.479+0000 7f087e7fc700 1 --2- 172.21.15.31:0/4290760356 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f086c0670e0 0x7f086c069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:49.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.479+0000 7f087e7fc700 1 -- 172.21.15.31:0/4290760356 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0884084530 con 0x7f08880cb3f0 2023-10-06T02:24:49.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.479+0000 7f088dd9a700 1 --2- 172.21.15.31:0/4290760356 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f086c0670e0 0x7f086c069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:49.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.479+0000 7f088dd9a700 1 --2- 172.21.15.31:0/4290760356 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f086c0670e0 0x7f086c069590 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f08880d0420 tx=0x7f0878016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:49.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.483+0000 7f087e7fc700 1 -- 172.21.15.31:0/4290760356 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f08840513a0 con 0x7f08880cb3f0 2023-10-06T02:24:49.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.764+0000 7f088f59d700 1 -- 172.21.15.31:0/4290760356 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f0888159750 con 0x7f08880cb3f0 2023-10-06T02:24:49.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.768+0000 7f087e7fc700 1 -- 172.21.15.31:0/4290760356 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) v1 ==== 190+0+0 (secure 0 0 0) 0x7f08840549c0 con 0x7f08880cb3f0 2023-10-06T02:24:49.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.770+0000 7f088f59d700 1 -- 172.21.15.31:0/4290760356 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f086c0670e0 msgr2=0x7f086c069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:49.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.770+0000 7f088f59d700 1 --2- 172.21.15.31:0/4290760356 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f086c0670e0 0x7f086c069590 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f08880d0420 tx=0x7f0878016000 comp rx=0 tx=0).stop 2023-10-06T02:24:49.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.770+0000 7f088f59d700 1 -- 172.21.15.31:0/4290760356 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08880cb3f0 msgr2=0x7f0888159cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:49.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.770+0000 7f088f59d700 1 --2- 172.21.15.31:0/4290760356 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08880cb3f0 0x7f0888159cb0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f088400b5e0 tx=0x7f0884009120 comp rx=0 tx=0).stop 2023-10-06T02:24:49.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.770+0000 7f088f59d700 1 -- 172.21.15.31:0/4290760356 shutdown_connections 2023-10-06T02:24:49.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.770+0000 7f088f59d700 1 --2- 172.21.15.31:0/4290760356 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0888159380 0x7f08880c9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:49.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.771+0000 7f088f59d700 1 --2- 172.21.15.31:0/4290760356 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08880cb3f0 0x7f0888159cb0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:49.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.771+0000 7f088f59d700 1 --2- 172.21.15.31:0/4290760356 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f086c0670e0 0x7f086c069590 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:49.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.771+0000 7f088f59d700 1 --2- 172.21.15.31:0/4290760356 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f08880cea00 0x7f088816a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:49.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.771+0000 7f088f59d700 1 -- 172.21.15.31:0/4290760356 >> 172.21.15.31:0/4290760356 conn(0x7f08880703a0 msgr2=0x7f08880c56e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:49.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.771+0000 7f088f59d700 1 -- 172.21.15.31:0/4290760356 shutdown_connections 2023-10-06T02:24:49.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.771+0000 7f088f59d700 1 -- 172.21.15.31:0/4290760356 wait complete. 2023-10-06T02:24:49.774 INFO:teuthology.orchestra.run.smithi031.stderr:updated caps for client.0 2023-10-06T02:24:49.790 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:24:49.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.976+0000 7f7b13fff700 1 Processor -- start 2023-10-06T02:24:49.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.976+0000 7f7b13fff700 1 -- start start 2023-10-06T02:24:49.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.978+0000 7f7b13fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7b1413fef0 0x7f7b1415a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:49.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.978+0000 7f7b13fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7b1406aaa0 0x7f7b140c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:49.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.978+0000 7f7b13fff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b1415ae70 con 0x7f7b1413fb20 2023-10-06T02:24:49.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.978+0000 7f7b13fff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b1406af60 con 0x7f7b1413fef0 2023-10-06T02:24:49.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.978+0000 7f7b13fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1406b240 con 0x7f7b1406aaa0 2023-10-06T02:24:49.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.978+0000 7f7b127fc700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7b1413fef0 0x7f7b1415a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:49.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.978+0000 7f7b127fc700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7b1413fef0 0x7f7b1415a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53306/0 (socket says 172.21.15.31:53306) 2023-10-06T02:24:49.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.978+0000 7f7b127fc700 1 -- 172.21.15.31:0/2018571680 learned_addr learned my addr 172.21.15.31:0/2018571680 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:49.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.978+0000 7f7b137fe700 1 --2- 172.21.15.31:0/2018571680 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7b1406aaa0 0x7f7b140c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:49.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.978+0000 7f7b127fc700 1 -- 172.21.15.31:0/2018571680 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7b1406aaa0 msgr2=0x7f7b140c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:49.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.978+0000 7f7b127fc700 1 --2- 172.21.15.31:0/2018571680 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7b1406aaa0 0x7f7b140c9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:49.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.978+0000 7f7b127fc700 1 -- 172.21.15.31:0/2018571680 >> v1:172.21.15.62:6789/0 conn(0x7f7b1413fb20 legacy=0x7f7b140c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:49.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.978+0000 7f7b127fc700 1 -- 172.21.15.31:0/2018571680 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b1406b520 con 0x7f7b1413fef0 2023-10-06T02:24:49.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.979+0000 7f7b127fc700 1 --2- 172.21.15.31:0/2018571680 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b1413fef0 0x7f7b1415a930 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f7b08009990 tx=0x7f7b08009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9832dad204d7613 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:49.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.979+0000 7f7b11ffb700 1 -- 172.21.15.31:0/2018571680 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b08005c10 con 0x7f7b1413fef0 2023-10-06T02:24:49.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.979+0000 7f7b11ffb700 1 -- 172.21.15.31:0/2018571680 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b08005d70 con 0x7f7b1413fef0 2023-10-06T02:24:49.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.979+0000 7f7b11ffb700 1 -- 172.21.15.31:0/2018571680 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b0800b880 con 0x7f7b1413fef0 2023-10-06T02:24:49.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.979+0000 7f7b13fff700 1 -- 172.21.15.31:0/2018571680 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b1413fef0 msgr2=0x7f7b1415a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:49.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.979+0000 7f7b13fff700 1 --2- 172.21.15.31:0/2018571680 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b1413fef0 0x7f7b1415a930 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f7b08009990 tx=0x7f7b08009220 comp rx=0 tx=0).stop 2023-10-06T02:24:49.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.980+0000 7f7b13fff700 1 -- 172.21.15.31:0/2018571680 shutdown_connections 2023-10-06T02:24:49.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.980+0000 7f7b13fff700 1 --2- 172.21.15.31:0/2018571680 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7b1406aaa0 0x7f7b140c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:49.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.980+0000 7f7b13fff700 1 --2- 172.21.15.31:0/2018571680 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b1413fef0 0x7f7b1415a930 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:49.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.980+0000 7f7b13fff700 1 -- 172.21.15.31:0/2018571680 >> 172.21.15.31:0/2018571680 conn(0x7f7b1406d170 msgr2=0x7f7b140cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:49.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.980+0000 7f7b13fff700 1 -- 172.21.15.31:0/2018571680 shutdown_connections 2023-10-06T02:24:49.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.980+0000 7f7b13fff700 1 -- 172.21.15.31:0/2018571680 wait complete. 2023-10-06T02:24:49.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.981+0000 7f7b13fff700 1 Processor -- start 2023-10-06T02:24:49.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.981+0000 7f7b13fff700 1 -- start start 2023-10-06T02:24:49.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.982+0000 7f7b13fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b1406aaa0 0x7f7b140ade30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:49.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.982+0000 7f7b13fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b1413fb20 0x7f7b140ab740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:49.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.982+0000 7f7b13fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7b1413fef0 0x7f7b14155c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:49.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.982+0000 7f7b13fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b14159590 con 0x7f7b1406aaa0 2023-10-06T02:24:49.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.982+0000 7f7b13fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b14156af0 con 0x7f7b1413fb20 2023-10-06T02:24:49.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.982+0000 7f7b13fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b14156dd0 con 0x7f7b1413fef0 2023-10-06T02:24:49.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.982+0000 7f7b12ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b1406aaa0 0x7f7b140ade30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:49.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.983+0000 7f7b12ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b1406aaa0 0x7f7b140ade30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53322/0 (socket says 172.21.15.31:53322) 2023-10-06T02:24:49.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.983+0000 7f7b12ffd700 1 -- 172.21.15.31:0/2376271359 learned_addr learned my addr 172.21.15.31:0/2376271359 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:49.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.983+0000 7f7b127fc700 1 --2- 172.21.15.31:0/2376271359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b1413fb20 0x7f7b140ab740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:49.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.983+0000 7f7b137fe700 1 --2- 172.21.15.31:0/2376271359 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7b1413fef0 0x7f7b14155c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:49.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.983+0000 7f7b12ffd700 1 -- 172.21.15.31:0/2376271359 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7b1413fef0 msgr2=0x7f7b14155c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:49.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.983+0000 7f7b12ffd700 1 --2- 172.21.15.31:0/2376271359 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7b1413fef0 0x7f7b14155c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:49.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.983+0000 7f7b12ffd700 1 -- 172.21.15.31:0/2376271359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b1413fb20 msgr2=0x7f7b140ab740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:49.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.983+0000 7f7b12ffd700 1 --2- 172.21.15.31:0/2376271359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b1413fb20 0x7f7b140ab740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:49.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.983+0000 7f7b12ffd700 1 -- 172.21.15.31:0/2376271359 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b08009480 con 0x7f7b1406aaa0 2023-10-06T02:24:49.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.983+0000 7f7b12ffd700 1 --2- 172.21.15.31:0/2376271359 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b1406aaa0 0x7f7b140ade30 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f7b0400c590 tx=0x7f7b0400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:49.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.983+0000 7f7afaffd700 1 -- 172.21.15.31:0/2376271359 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b04011590 con 0x7f7b1406aaa0 2023-10-06T02:24:49.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.984+0000 7f7afaffd700 1 -- 172.21.15.31:0/2376271359 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b04011710 con 0x7f7b1406aaa0 2023-10-06T02:24:49.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.984+0000 7f7afaffd700 1 -- 172.21.15.31:0/2376271359 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b04011ad0 con 0x7f7b1406aaa0 2023-10-06T02:24:49.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.984+0000 7f7b13fff700 1 -- 172.21.15.31:0/2376271359 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b14158f40 con 0x7f7b1406aaa0 2023-10-06T02:24:49.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.984+0000 7f7b13fff700 1 -- 172.21.15.31:0/2376271359 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b141561c0 con 0x7f7b1406aaa0 2023-10-06T02:24:49.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.985+0000 7f7b13fff700 1 -- 172.21.15.31:0/2376271359 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b1404c3c0 con 0x7f7b1406aaa0 2023-10-06T02:24:49.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.986+0000 7f7afaffd700 1 -- 172.21.15.31:0/2376271359 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f7b0401b3b0 con 0x7f7b1406aaa0 2023-10-06T02:24:49.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.987+0000 7f7afaffd700 1 --2- 172.21.15.31:0/2376271359 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f7b000670e0 0x7f7b00069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:49.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.987+0000 7f7afaffd700 1 -- 172.21.15.31:0/2376271359 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f7b04083d00 con 0x7f7b1406aaa0 2023-10-06T02:24:49.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.987+0000 7f7b127fc700 1 --2- 172.21.15.31:0/2376271359 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f7b000670e0 0x7f7b00069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:49.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.988+0000 7f7b127fc700 1 --2- 172.21.15.31:0/2376271359 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f7b000670e0 0x7f7b00069590 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7b08009990 tx=0x7f7b08016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:49.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:49.991+0000 7f7afaffd700 1 -- 172.21.15.31:0/2376271359 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7b04050af0 con 0x7f7b1406aaa0 2023-10-06T02:24:50.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.234+0000 7f7b13fff700 1 -- 172.21.15.31:0/2376271359 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7b1415ae70 con 0x7f7b1406aaa0 2023-10-06T02:24:50.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.235+0000 7f7afaffd700 1 -- 172.21.15.31:0/2376271359 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4594 (secure 0 0 0) 0x7f7b1415ae70 con 0x7f7b1406aaa0 2023-10-06T02:24:50.235 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:24:50.235 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4949,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/178547366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":178547366},{"type":"v1","addr":"172.21.15.62:6835","nonce":178547366}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":5278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/2578831517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":2578831517},{"type":"v1","addr":"172.21.15.31:6836","nonce":2578831517}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5287,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3502585737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":3502585737},{"type":"v1","addr":"172.21.15.31:6837","nonce":3502585737}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:24:43.872314+0000","modified":"2023-10-06T02:24:47.404607+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5290":{"gid":5290,"name":"b","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6834/2358776861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2358776861},{"type":"v1","addr":"172.21.15.62:6834","nonce":2358776861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T02:24:50.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.237+0000 7f7b13fff700 1 -- 172.21.15.31:0/2376271359 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f7b000670e0 msgr2=0x7f7b00069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:50.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.237+0000 7f7b13fff700 1 --2- 172.21.15.31:0/2376271359 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f7b000670e0 0x7f7b00069590 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7b08009990 tx=0x7f7b08016000 comp rx=0 tx=0).stop 2023-10-06T02:24:50.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.237+0000 7f7b13fff700 1 -- 172.21.15.31:0/2376271359 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b1406aaa0 msgr2=0x7f7b140ade30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:50.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.237+0000 7f7b13fff700 1 --2- 172.21.15.31:0/2376271359 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b1406aaa0 0x7f7b140ade30 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f7b0400c590 tx=0x7f7b0400c8a0 comp rx=0 tx=0).stop 2023-10-06T02:24:50.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.237+0000 7f7b13fff700 1 -- 172.21.15.31:0/2376271359 shutdown_connections 2023-10-06T02:24:50.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.237+0000 7f7b13fff700 1 --2- 172.21.15.31:0/2376271359 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f7b000670e0 0x7f7b00069590 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:50.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.237+0000 7f7b13fff700 1 --2- 172.21.15.31:0/2376271359 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7b1413fef0 0x7f7b14155c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:50.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.237+0000 7f7b13fff700 1 --2- 172.21.15.31:0/2376271359 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b1406aaa0 0x7f7b140ade30 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:50.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.237+0000 7f7b13fff700 1 --2- 172.21.15.31:0/2376271359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b1413fb20 0x7f7b140ab740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:50.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.237+0000 7f7b13fff700 1 -- 172.21.15.31:0/2376271359 >> 172.21.15.31:0/2376271359 conn(0x7f7b1406d170 msgr2=0x7f7b140c7990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:50.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.237+0000 7f7b13fff700 1 -- 172.21.15.31:0/2376271359 shutdown_connections 2023-10-06T02:24:50.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.237+0000 7f7b13fff700 1 -- 172.21.15.31:0/2376271359 wait complete. 2023-10-06T02:24:50.239 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 26 2023-10-06T02:24:50.657 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T02:24:43.872314+0000', 'modified': '2023-10-06T02:24:47.404607+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5290}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5290': {'gid': 5290, 'name': 'b', 'rank': 0, 'incarnation': 23, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.62:6834/2358776861', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6832', 'nonce': 2358776861}, {'type': 'v1', 'addr': '172.21.15.62:6834', 'nonce': 2358776861}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T02:24:50.657 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T02:24:50.657 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-06T02:24:50.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.847+0000 7f6ee619d700 1 Processor -- start 2023-10-06T02:24:50.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.847+0000 7f6ee619d700 1 -- start start 2023-10-06T02:24:50.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.848+0000 7f6ee619d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6ee0153c30 0x7f6ee015a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:50.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.848+0000 7f6ee619d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f6ee00cf110 0x7f6ee015c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:50.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.848+0000 7f6ee619d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ee015afd0 con 0x7f6ee015de50 2023-10-06T02:24:50.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.848+0000 7f6ee619d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ee015b2b0 con 0x7f6ee0153c30 2023-10-06T02:24:50.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.848+0000 7f6ee619d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ee015b590 con 0x7f6ee00cf110 2023-10-06T02:24:50.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.849+0000 7f6ee599c700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f6ee00cf110 0x7f6ee015c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:50.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.849+0000 7f6ee519b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6ee0153c30 0x7f6ee015a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:50.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.849+0000 7f6ee499a700 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f6ee015de50 0x7f6ee00cea00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:52354/0 (socket says 172.21.15.31:52354) 2023-10-06T02:24:50.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.849+0000 7f6ee599c700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f6ee00cf110 0x7f6ee015c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:33448/0 (socket says 172.21.15.31:33448) 2023-10-06T02:24:50.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.849+0000 7f6ee499a700 1 -- 172.21.15.31:0/1209956509 learned_addr learned my addr 172.21.15.31:0/1209956509 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:24:50.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.849+0000 7f6ee519b700 1 --2- 172.21.15.31:0/1209956509 >> v2:172.21.15.31:3300/0 conn(0x7f6ee0153c30 0x7f6ee015a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53336/0 (socket says 172.21.15.31:53336) 2023-10-06T02:24:50.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.849+0000 7f6ee519b700 1 -- 172.21.15.31:0/1209956509 >> v2:172.21.15.62:3300/0 conn(0x7f6ee00cf110 msgr2=0x7f6ee015c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:50.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.849+0000 7f6ee519b700 1 --2- 172.21.15.31:0/1209956509 >> v2:172.21.15.62:3300/0 conn(0x7f6ee00cf110 0x7f6ee015c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:50.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.849+0000 7f6ee519b700 1 -- 172.21.15.31:0/1209956509 >> v1:172.21.15.62:6789/0 conn(0x7f6ee015de50 legacy=0x7f6ee00cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:50.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.849+0000 7f6ee519b700 1 -- 172.21.15.31:0/1209956509 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ee015b870 con 0x7f6ee0153c30 2023-10-06T02:24:50.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.850+0000 7f6ee519b700 1 --2- 172.21.15.31:0/1209956509 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ee0153c30 0x7f6ee015a5e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f6ee00cdab0 tx=0x7f6edc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b579fc5de1730fe server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:50.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.850+0000 7f6ed7fff700 1 -- 172.21.15.31:0/1209956509 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6edc011630 con 0x7f6ee0153c30 2023-10-06T02:24:50.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.850+0000 7f6ed7fff700 1 -- 172.21.15.31:0/1209956509 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6edc011790 con 0x7f6ee0153c30 2023-10-06T02:24:50.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.850+0000 7f6ed7fff700 1 -- 172.21.15.31:0/1209956509 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6edc011b30 con 0x7f6ee0153c30 2023-10-06T02:24:50.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.850+0000 7f6ee619d700 1 -- 172.21.15.31:0/1209956509 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ee0153c30 msgr2=0x7f6ee015a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:50.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.850+0000 7f6ee619d700 1 --2- 172.21.15.31:0/1209956509 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ee0153c30 0x7f6ee015a5e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f6ee00cdab0 tx=0x7f6edc0091a0 comp rx=0 tx=0).stop 2023-10-06T02:24:50.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.850+0000 7f6ee619d700 1 -- 172.21.15.31:0/1209956509 shutdown_connections 2023-10-06T02:24:50.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.850+0000 7f6ee619d700 1 --2- 172.21.15.31:0/1209956509 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ee0153c30 0x7f6ee015a5e0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:50.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.850+0000 7f6ee619d700 1 --2- 172.21.15.31:0/1209956509 >> v2:172.21.15.62:3300/0 conn(0x7f6ee00cf110 0x7f6ee015c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:50.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.850+0000 7f6ee619d700 1 -- 172.21.15.31:0/1209956509 >> 172.21.15.31:0/1209956509 conn(0x7f6ee00703a0 msgr2=0x7f6ee00707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:50.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.851+0000 7f6ee619d700 1 -- 172.21.15.31:0/1209956509 shutdown_connections 2023-10-06T02:24:50.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.851+0000 7f6ee619d700 1 -- 172.21.15.31:0/1209956509 wait complete. 2023-10-06T02:24:50.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.852+0000 7f6ee619d700 1 Processor -- start 2023-10-06T02:24:50.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.852+0000 7f6ee619d700 1 -- start start 2023-10-06T02:24:50.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.853+0000 7f6ee619d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ee00cf110 0x7f6ee00b3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:50.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.853+0000 7f6ee619d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6ee0153c30 0x7f6ee00b5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:50.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.853+0000 7f6ee619d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ee015de50 0x7f6ee0143dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:50.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.853+0000 7f6ee619d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee00b4b40 con 0x7f6ee00cf110 2023-10-06T02:24:50.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.853+0000 7f6ee619d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee00c77a0 con 0x7f6ee015de50 2023-10-06T02:24:50.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.853+0000 7f6ee619d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee00b42a0 con 0x7f6ee0153c30 2023-10-06T02:24:50.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.853+0000 7f6ee519b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ee00cf110 0x7f6ee00b3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:50.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.853+0000 7f6ee519b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ee00cf110 0x7f6ee00b3c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53344/0 (socket says 172.21.15.31:53344) 2023-10-06T02:24:50.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.853+0000 7f6ee519b700 1 -- 172.21.15.31:0/148857766 learned_addr learned my addr 172.21.15.31:0/148857766 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:50.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.854+0000 7f6ee519b700 1 -- 172.21.15.31:0/148857766 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6ee0153c30 msgr2=0x7f6ee00b5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:50.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.854+0000 7f6ee499a700 1 --2- 172.21.15.31:0/148857766 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6ee0153c30 0x7f6ee00b5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:50.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.854+0000 7f6ee599c700 1 --2- 172.21.15.31:0/148857766 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ee015de50 0x7f6ee0143dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:50.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.854+0000 7f6ee519b700 1 --2- 172.21.15.31:0/148857766 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6ee0153c30 0x7f6ee00b5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:50.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.854+0000 7f6ee519b700 1 -- 172.21.15.31:0/148857766 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ee015de50 msgr2=0x7f6ee0143dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:50.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.854+0000 7f6ee519b700 1 --2- 172.21.15.31:0/148857766 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ee015de50 0x7f6ee0143dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:50.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.854+0000 7f6ee519b700 1 -- 172.21.15.31:0/148857766 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6edc009420 con 0x7f6ee00cf110 2023-10-06T02:24:50.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.854+0000 7f6ee599c700 1 --2- 172.21.15.31:0/148857766 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ee015de50 0x7f6ee0143dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:24:50.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.854+0000 7f6ee499a700 1 --2- 172.21.15.31:0/148857766 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6ee0153c30 0x7f6ee00b5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:24:50.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.854+0000 7f6ee519b700 1 --2- 172.21.15.31:0/148857766 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ee00cf110 0x7f6ee00b3c90 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f6edc009f50 tx=0x7f6edc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:50.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.854+0000 7f6ed57fa700 1 -- 172.21.15.31:0/148857766 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6edc011b10 con 0x7f6ee00cf110 2023-10-06T02:24:50.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.854+0000 7f6ed57fa700 1 -- 172.21.15.31:0/148857766 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6edc0253f0 con 0x7f6ee00cf110 2023-10-06T02:24:50.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.854+0000 7f6ed57fa700 1 -- 172.21.15.31:0/148857766 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6edc025790 con 0x7f6ee00cf110 2023-10-06T02:24:50.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.854+0000 7f6ee619d700 1 -- 172.21.15.31:0/148857766 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ee00b4580 con 0x7f6ee00cf110 2023-10-06T02:24:50.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.855+0000 7f6ee619d700 1 -- 172.21.15.31:0/148857766 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ee00b5bc0 con 0x7f6ee00cf110 2023-10-06T02:24:50.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.856+0000 7f6ed57fa700 1 -- 172.21.15.31:0/148857766 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f6edc025b80 con 0x7f6ee00cf110 2023-10-06T02:24:50.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.857+0000 7f6ed57fa700 1 --2- 172.21.15.31:0/148857766 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6ec40670e0 0x7f6ec4069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:50.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.857+0000 7f6ed57fa700 1 -- 172.21.15.31:0/148857766 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6edc0862f0 con 0x7f6ee00cf110 2023-10-06T02:24:50.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.858+0000 7f6ee499a700 1 --2- 172.21.15.31:0/148857766 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6ec40670e0 0x7f6ec4069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:50.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.858+0000 7f6ee499a700 1 --2- 172.21.15.31:0/148857766 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6ec40670e0 0x7f6ec4069590 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6ee01593b0 tx=0x7f6ed0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:50.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.859+0000 7f6ef5b0ab80 1 Processor -- start 2023-10-06T02:24:50.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.860+0000 7f6ef5b0ab80 1 -- start start 2023-10-06T02:24:50.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.860+0000 7f6ef5b0ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56384ba0ce50 0x56384b9e7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:50.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.860+0000 7f6ef5b0ab80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56384b9ed2c0 0x56384b9e8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:50.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.860+0000 7f6ef5b0ab80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56384b9eea10 0x56384b9e8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:50.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.860+0000 7f6ef5b0ab80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56384b942a70 con 0x56384b9eea10 2023-10-06T02:24:50.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.860+0000 7f6ef5b0ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56384b940270 con 0x56384ba0ce50 2023-10-06T02:24:50.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.860+0000 7f6ef5b0ab80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x56384b93c060 con 0x56384b9ed2c0 2023-10-06T02:24:50.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.861+0000 7f6ee599c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56384b9eea10 0x56384b9e8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:50.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.861+0000 7f6ee599c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56384b9eea10 0x56384b9e8740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53346/0 (socket says 172.21.15.31:53346) 2023-10-06T02:24:50.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.861+0000 7f6ee599c700 1 -- 172.21.15.31:0/4113732066 learned_addr learned my addr 172.21.15.31:0/4113732066 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:50.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.861+0000 7f6ee519b700 1 --2- 172.21.15.31:0/4113732066 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56384ba0ce50 0x56384b9e7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:50.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.861+0000 7f6ee599c700 1 -- 172.21.15.31:0/4113732066 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56384b9ed2c0 msgr2=0x56384b9e8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:50.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.861+0000 7f6ee499a700 1 --2- 172.21.15.31:0/4113732066 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56384b9ed2c0 0x56384b9e8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:50.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.861+0000 7f6ee599c700 1 --2- 172.21.15.31:0/4113732066 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56384b9ed2c0 0x56384b9e8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:50.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.861+0000 7f6ee599c700 1 -- 172.21.15.31:0/4113732066 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56384ba0ce50 msgr2=0x56384b9e7cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:50.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.861+0000 7f6ee599c700 1 --2- 172.21.15.31:0/4113732066 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56384ba0ce50 0x56384b9e7cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:50.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.861+0000 7f6ee599c700 1 -- 172.21.15.31:0/4113732066 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56384b7d6270 con 0x56384b9eea10 2023-10-06T02:24:50.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.861+0000 7f6ee519b700 1 --2- 172.21.15.31:0/4113732066 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56384ba0ce50 0x56384b9e7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:24:50.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.861+0000 7f6ee499a700 1 --2- 172.21.15.31:0/4113732066 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56384b9ed2c0 0x56384b9e8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:24:50.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.861+0000 7f6ee599c700 1 --2- 172.21.15.31:0/4113732066 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56384b9eea10 0x56384b9e8740 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f6ed8007610 tx=0x7f6ed800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72684ac07067dabe server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:50.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.862+0000 7f6eca7fc700 1 -- 172.21.15.31:0/4113732066 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed80145e0 con 0x56384b9eea10 2023-10-06T02:24:50.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.862+0000 7f6eca7fc700 1 -- 172.21.15.31:0/4113732066 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ed800f050 con 0x56384b9eea10 2023-10-06T02:24:50.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.862+0000 7f6eca7fc700 1 -- 172.21.15.31:0/4113732066 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed8014950 con 0x56384b9eea10 2023-10-06T02:24:50.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.862+0000 7f6ef5b0ab80 1 -- 172.21.15.31:0/4113732066 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56384b9eea10 msgr2=0x56384b9e8740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:50.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.862+0000 7f6ef5b0ab80 1 --2- 172.21.15.31:0/4113732066 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56384b9eea10 0x56384b9e8740 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f6ed8007610 tx=0x7f6ed800b040 comp rx=0 tx=0).stop 2023-10-06T02:24:50.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.862+0000 7f6ef5b0ab80 1 -- 172.21.15.31:0/4113732066 shutdown_connections 2023-10-06T02:24:50.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.862+0000 7f6ef5b0ab80 1 --2- 172.21.15.31:0/4113732066 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56384b9eea10 0x56384b9e8740 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:50.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.862+0000 7f6ef5b0ab80 1 --2- 172.21.15.31:0/4113732066 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56384ba0ce50 0x56384b9e7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:50.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.862+0000 7f6ef5b0ab80 1 --2- 172.21.15.31:0/4113732066 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56384b9ed2c0 0x56384b9e8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:50.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.862+0000 7f6ef5b0ab80 1 -- 172.21.15.31:0/4113732066 >> 172.21.15.31:0/4113732066 conn(0x56384b9ece80 msgr2=0x56384b9ed6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:50.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.862+0000 7f6ef5b0ab80 1 -- 172.21.15.31:0/4113732066 shutdown_connections 2023-10-06T02:24:50.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.863+0000 7f6ef5b0ab80 1 -- 172.21.15.31:0/4113732066 wait complete. 2023-10-06T02:24:50.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.863+0000 7f6ef5b0ab80 1 -- start start 2023-10-06T02:24:50.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.863+0000 7f6ef5b0ab80 1 Processor -- start 2023-10-06T02:24:50.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.864+0000 7f6ef5b0ab80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56384b9ed2c0 0x56384ba32060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:50.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.864+0000 7f6ef5b0ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56384b9eea10 0x56384ba17a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:50.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.864+0000 7f6ef5b0ab80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56384ba0ce50 0x56384ba03580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:50.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.864+0000 7f6ef5b0ab80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56384b8e7890 con 0x56384b9ed2c0 2023-10-06T02:24:50.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.864+0000 7f6ef5b0ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56384b902ea0 con 0x56384b9eea10 2023-10-06T02:24:50.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.864+0000 7f6ef5b0ab80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x56384b7bee30 con 0x56384ba0ce50 2023-10-06T02:24:50.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.864+0000 7f6ef5b0ab80 10 client.0 _ll_register_callbacks cb 0x56384b9af020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T02:24:50.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.865+0000 7f6ee519b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56384b9ed2c0 0x56384ba32060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:50.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.865+0000 7f6ee499a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56384b9eea10 0x56384ba17a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:50.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.865+0000 7f6ee519b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56384b9ed2c0 0x56384ba32060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53350/0 (socket says 172.21.15.31:53350) 2023-10-06T02:24:50.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.865+0000 7f6ee519b700 1 -- 172.21.15.31:0/2285533654 learned_addr learned my addr 172.21.15.31:0/2285533654 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:50.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.865+0000 7f6ee599c700 1 --2- 172.21.15.31:0/2285533654 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56384ba0ce50 0x56384ba03580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:50.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.865+0000 7f6ee519b700 1 -- 172.21.15.31:0/2285533654 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56384ba0ce50 msgr2=0x56384ba03580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:50.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.865+0000 7f6ee519b700 1 --2- 172.21.15.31:0/2285533654 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56384ba0ce50 0x56384ba03580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:50.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.865+0000 7f6ee519b700 1 -- 172.21.15.31:0/2285533654 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56384b9eea10 msgr2=0x56384ba17a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:50.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.865+0000 7f6ee519b700 1 --2- 172.21.15.31:0/2285533654 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56384b9eea10 0x56384ba17a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:50.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.865+0000 7f6ee519b700 1 -- 172.21.15.31:0/2285533654 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ed8007340 con 0x56384b9ed2c0 2023-10-06T02:24:50.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.865+0000 7f6ee519b700 1 --2- 172.21.15.31:0/2285533654 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56384b9ed2c0 0x56384ba32060 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f6edc0844f0 tx=0x7f6edc003040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e59e3806566b0166 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:50.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.865+0000 7f6ebf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T02:24:50.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.865+0000 7f6ebf7fe700 1 -- 172.21.15.31:0/2285533654 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6edc057520 con 0x56384b9ed2c0 2023-10-06T02:24:50.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.865+0000 7f6ebf7fe700 1 -- 172.21.15.31:0/2285533654 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(47..47 src has 1..47) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6edc0860d0 con 0x56384b9ed2c0 2023-10-06T02:24:50.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.865+0000 7f6ef5b0ab80 1 -- 172.21.15.31:0/2285533654 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56384b9e0940 con 0x56384b9ed2c0 2023-10-06T02:24:50.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.866+0000 7f6ebf7fe700 1 -- 172.21.15.31:0/2285533654 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6edc05c0f0 con 0x56384b9ed2c0 2023-10-06T02:24:50.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.866+0000 7f6ef5b0ab80 10 client.5580 fetch_fsmap learned FSMap version 26 2023-10-06T02:24:50.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.866+0000 7f6ef5b0ab80 1 -- 172.21.15.31:0/2285533654 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x56384b778fe0 con 0x56384b9ed2c0 2023-10-06T02:24:50.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.866+0000 7f6ebf7fe700 1 -- 172.21.15.31:0/2285533654 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56384b9e0940 con 0x56384b9ed2c0 2023-10-06T02:24:50.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.866+0000 7f6ebf7fe700 1 -- 172.21.15.31:0/2285533654 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f6edc056290 con 0x56384b9ed2c0 2023-10-06T02:24:50.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.866+0000 7f6ef5b0ab80 10 client.5580 fetch_fsmap finished waiting for FSMap version 26 2023-10-06T02:24:50.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.866+0000 7f6ef5b0ab80 10 client.5580 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-06T02:24:50.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.866+0000 7f6ef5b0ab80 20 client.5580 populate_metadata read hostname 'smithi031' 2023-10-06T02:24:50.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.866+0000 7f6ef5b0ab80 1 --2- 172.21.15.31:0/2285533654 >> [v2:172.21.15.62:6832/2358776861,v1:172.21.15.62:6834/2358776861] conn(0x56384b9e80c0 0x56384b9e8490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:50.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.866+0000 7f6ef5b0ab80 4 client.5580 mds_command: new command op to 5290 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T02:24:50.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.866+0000 7f6ef5b0ab80 1 -- 172.21.15.31:0/2285533654 --> [v2:172.21.15.62:6832/2358776861,v1:172.21.15.62:6834/2358776861] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56384b9bd870 con 0x56384b9e80c0 2023-10-06T02:24:50.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.867+0000 7f6ee499a700 1 --2- 172.21.15.31:0/2285533654 >> [v2:172.21.15.62:6832/2358776861,v1:172.21.15.62:6834/2358776861] conn(0x56384b9e80c0 0x56384b9e8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:50.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.867+0000 7f6ee499a700 1 --2- 172.21.15.31:0/2285533654 >> [v2:172.21.15.62:6832/2358776861,v1:172.21.15.62:6834/2358776861] conn(0x56384b9e80c0 0x56384b9e8490 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=580368932f811604 server_cookie=db3184e1b354040 in_seq=0 out_seq=0 2023-10-06T02:24:50.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.867+0000 7f6ebf7fe700 10 client.5580 ms_handle_connect on v2:172.21.15.62:6832/2358776861 2023-10-06T02:24:50.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.868+0000 7f6ebf7fe700 1 -- 172.21.15.31:0/2285533654 <== mds.0 v2:172.21.15.62:6832/2358776861 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x56384b9bd870 con 0x56384b9e80c0 2023-10-06T02:24:50.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.868+0000 7f6ebf7fe700 10 client.5580 handle_command_reply: tid=0 2023-10-06T02:24:50.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.868+0000 7f6ef5b0ab80 1 client.5580 shutdown 2023-10-06T02:24:50.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.868+0000 7f6ef5b0ab80 1 -- 172.21.15.31:0/2285533654 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56384b9ed2c0 msgr2=0x56384ba32060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:50.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.868+0000 7f6ef5b0ab80 1 --2- 172.21.15.31:0/2285533654 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56384b9ed2c0 0x56384ba32060 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f6edc0844f0 tx=0x7f6edc003040 comp rx=0 tx=0).stop 2023-10-06T02:24:50.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.868+0000 7f6ef5b0ab80 1 -- 172.21.15.31:0/2285533654 shutdown_connections 2023-10-06T02:24:50.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.868+0000 7f6ef5b0ab80 1 --2- 172.21.15.31:0/2285533654 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56384ba0ce50 0x56384ba03580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:50.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.868+0000 7f6ef5b0ab80 1 --2- 172.21.15.31:0/2285533654 >> [v2:172.21.15.62:6832/2358776861,v1:172.21.15.62:6834/2358776861] conn(0x56384b9e80c0 0x56384b9e8490 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:50.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.868+0000 7f6ef5b0ab80 1 --2- 172.21.15.31:0/2285533654 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56384b9ed2c0 0x56384ba32060 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:50.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.868+0000 7f6ef5b0ab80 1 --2- 172.21.15.31:0/2285533654 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56384b9eea10 0x56384ba17a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:50.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.868+0000 7f6ebf7fe700 0 client.5580 ms_handle_reset on v2:172.21.15.62:6832/2358776861 2023-10-06T02:24:50.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.868+0000 7f6ef5b0ab80 1 -- 172.21.15.31:0/2285533654 >> 172.21.15.31:0/2285533654 conn(0x56384b9ece80 msgr2=0x56384b9ebb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:50.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.869+0000 7f6ef5b0ab80 1 -- 172.21.15.31:0/2285533654 shutdown_connections 2023-10-06T02:24:50.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.869+0000 7f6ef5b0ab80 1 -- 172.21.15.31:0/2285533654 wait complete. 2023-10-06T02:24:50.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.869+0000 7f6ef5b0ab80 20 client.5580 trim_cache size 0 max 16384 2023-10-06T02:24:50.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.886+0000 7f6ef5b0ab80 1 Processor -- start 2023-10-06T02:24:50.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.886+0000 7f6ef5b0ab80 1 -- start start 2023-10-06T02:24:50.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.887+0000 7f6ef5b0ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56384b9ed2c0 0x56384ba04c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:50.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.887+0000 7f6ef5b0ab80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56384b9eea10 0x56384b9e2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:50.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.887+0000 7f6ef5b0ab80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56384ba0ce50 0x56384b9ed860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:50.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.887+0000 7f6ef5b0ab80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56384ba04130 con 0x56384ba0ce50 2023-10-06T02:24:50.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.887+0000 7f6ef5b0ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56384ba03f70 con 0x56384b9ed2c0 2023-10-06T02:24:50.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.887+0000 7f6ef5b0ab80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x56384ba17d90 con 0x56384b9eea10 2023-10-06T02:24:50.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.887+0000 7f6ee599c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56384ba0ce50 0x56384b9ed860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:50.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.887+0000 7f6ee599c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56384ba0ce50 0x56384b9ed860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53352/0 (socket says 172.21.15.31:53352) 2023-10-06T02:24:50.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.887+0000 7f6ee599c700 1 -- 172.21.15.31:0/2390064608 learned_addr learned my addr 172.21.15.31:0/2390064608 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:50.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.888+0000 7f6ee499a700 1 --2- 172.21.15.31:0/2390064608 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56384b9eea10 0x56384b9e2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:50.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.888+0000 7f6ee599c700 1 -- 172.21.15.31:0/2390064608 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56384b9eea10 msgr2=0x56384b9e2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:50.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.888+0000 7f6ee519b700 1 --2- 172.21.15.31:0/2390064608 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56384b9ed2c0 0x56384ba04c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:50.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.888+0000 7f6ee599c700 1 --2- 172.21.15.31:0/2390064608 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56384b9eea10 0x56384b9e2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:50.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.888+0000 7f6ee599c700 1 -- 172.21.15.31:0/2390064608 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56384b9ed2c0 msgr2=0x56384ba04c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:50.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.888+0000 7f6ee599c700 1 --2- 172.21.15.31:0/2390064608 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56384b9ed2c0 0x56384ba04c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:50.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.888+0000 7f6ee599c700 1 -- 172.21.15.31:0/2390064608 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56384b90a780 con 0x56384ba0ce50 2023-10-06T02:24:50.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.888+0000 7f6ee499a700 1 --2- 172.21.15.31:0/2390064608 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56384b9eea10 0x56384b9e2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:24:50.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.888+0000 7f6ee519b700 1 --2- 172.21.15.31:0/2390064608 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56384b9ed2c0 0x56384ba04c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:24:50.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.888+0000 7f6ee599c700 1 --2- 172.21.15.31:0/2390064608 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56384ba0ce50 0x56384b9ed860 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f6ed80075b0 tx=0x7f6ed801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac3326e2f3b22471 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:50.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.888+0000 7f6ecaffd700 1 -- 172.21.15.31:0/2390064608 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed8024070 con 0x56384ba0ce50 2023-10-06T02:24:50.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.888+0000 7f6ecaffd700 1 -- 172.21.15.31:0/2390064608 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ed8009c70 con 0x56384ba0ce50 2023-10-06T02:24:50.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.888+0000 7f6ecaffd700 1 -- 172.21.15.31:0/2390064608 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed80145e0 con 0x56384ba0ce50 2023-10-06T02:24:50.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.889+0000 7f6ef5b0ab80 1 -- 172.21.15.31:0/2390064608 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56384ba0ce50 msgr2=0x56384b9ed860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:50.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.889+0000 7f6ef5b0ab80 1 --2- 172.21.15.31:0/2390064608 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56384ba0ce50 0x56384b9ed860 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f6ed80075b0 tx=0x7f6ed801a000 comp rx=0 tx=0).stop 2023-10-06T02:24:50.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.889+0000 7f6ef5b0ab80 1 -- 172.21.15.31:0/2390064608 shutdown_connections 2023-10-06T02:24:50.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.889+0000 7f6ef5b0ab80 1 --2- 172.21.15.31:0/2390064608 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56384ba0ce50 0x56384b9ed860 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:50.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.889+0000 7f6ef5b0ab80 1 --2- 172.21.15.31:0/2390064608 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56384b9ed2c0 0x56384ba04c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:50.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.889+0000 7f6ef5b0ab80 1 --2- 172.21.15.31:0/2390064608 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56384b9eea10 0x56384b9e2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:50.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.889+0000 7f6ef5b0ab80 1 -- 172.21.15.31:0/2390064608 >> 172.21.15.31:0/2390064608 conn(0x56384b9ece80 msgr2=0x56384b9e8490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:50.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.889+0000 7f6ef5b0ab80 1 -- 172.21.15.31:0/2390064608 shutdown_connections 2023-10-06T02:24:50.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.889+0000 7f6ef5b0ab80 1 -- 172.21.15.31:0/2390064608 wait complete. 2023-10-06T02:24:50.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.890+0000 7f6ef5b0ab80 1 -- start start 2023-10-06T02:24:50.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.890+0000 7f6ef5b0ab80 1 Processor -- start 2023-10-06T02:24:50.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.891+0000 7f6ef5b0ab80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56384b9ed2c0 0x56384ba4b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:50.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.891+0000 7f6ef5b0ab80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56384b9eea10 0x56384b9ed860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:50.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.891+0000 7f6ef5b0ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56384ba0ce50 0x56384b9edda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:50.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.891+0000 7f6ef5b0ab80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56384ba4bbc0 con 0x56384b9ed2c0 2023-10-06T02:24:50.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.891+0000 7f6ef5b0ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56384ba34c20 con 0x56384ba0ce50 2023-10-06T02:24:50.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.891+0000 7f6ef5b0ab80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x56384ba15b80 con 0x56384b9eea10 2023-10-06T02:24:50.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.891+0000 7f6ef5b0ab80 10 client.0 _ll_register_callbacks cb 0x56384b9af020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T02:24:50.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.891+0000 7f6ee519b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56384b9ed2c0 0x56384ba4b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:50.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.891+0000 7f6ee519b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56384b9ed2c0 0x56384ba4b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53360/0 (socket says 172.21.15.31:53360) 2023-10-06T02:24:50.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.891+0000 7f6ee519b700 1 -- 172.21.15.31:0/1323518395 learned_addr learned my addr 172.21.15.31:0/1323518395 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:50.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.891+0000 7f6ee599c700 1 --2- 172.21.15.31:0/1323518395 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56384ba0ce50 0x56384b9edda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:50.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.891+0000 7f6ee499a700 1 --2- 172.21.15.31:0/1323518395 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56384b9eea10 0x56384b9ed860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:50.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.892+0000 7f6ee519b700 1 -- 172.21.15.31:0/1323518395 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56384b9eea10 msgr2=0x56384b9ed860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:50.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.892+0000 7f6ee519b700 1 --2- 172.21.15.31:0/1323518395 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56384b9eea10 0x56384b9ed860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:50.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.892+0000 7f6ee519b700 1 -- 172.21.15.31:0/1323518395 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56384ba0ce50 msgr2=0x56384b9edda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:50.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.892+0000 7f6ee519b700 1 --2- 172.21.15.31:0/1323518395 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56384ba0ce50 0x56384b9edda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:50.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.892+0000 7f6ee519b700 1 -- 172.21.15.31:0/1323518395 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56384b7d6270 con 0x56384b9ed2c0 2023-10-06T02:24:50.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.892+0000 7f6ee599c700 1 --2- 172.21.15.31:0/1323518395 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56384ba0ce50 0x56384b9edda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:24:50.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.892+0000 7f6ee499a700 1 --2- 172.21.15.31:0/1323518395 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56384b9eea10 0x56384b9ed860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:24:50.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.892+0000 7f6ee519b700 1 --2- 172.21.15.31:0/1323518395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56384b9ed2c0 0x56384ba4b130 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f6edc056a30 tx=0x7f6edc02f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe1eaf8728b4ea78 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:50.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.892+0000 7f6ebf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T02:24:50.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.892+0000 7f6ebf7fe700 1 -- 172.21.15.31:0/1323518395 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6edc0530e0 con 0x56384b9ed2c0 2023-10-06T02:24:50.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.892+0000 7f6ebf7fe700 1 -- 172.21.15.31:0/1323518395 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(47..47 src has 1..47) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6edc0860d0 con 0x56384b9ed2c0 2023-10-06T02:24:50.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.892+0000 7f6ef5b0ab80 1 -- 172.21.15.31:0/1323518395 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6ed800f050 con 0x56384b9ed2c0 2023-10-06T02:24:50.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.892+0000 7f6ebf7fe700 1 -- 172.21.15.31:0/1323518395 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6edc06d1a0 con 0x56384b9ed2c0 2023-10-06T02:24:50.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.892+0000 7f6ef5b0ab80 10 client.5586 fetch_fsmap learned FSMap version 26 2023-10-06T02:24:50.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.892+0000 7f6ef5b0ab80 1 -- 172.21.15.31:0/1323518395 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x56384b8f1470 con 0x56384b9ed2c0 2023-10-06T02:24:50.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.893+0000 7f6ebf7fe700 1 -- 172.21.15.31:0/1323518395 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6edc084080 con 0x56384b9ed2c0 2023-10-06T02:24:50.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.893+0000 7f6ebf7fe700 1 -- 172.21.15.31:0/1323518395 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f6edc056290 con 0x56384b9ed2c0 2023-10-06T02:24:50.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.893+0000 7f6ef5b0ab80 10 client.5586 fetch_fsmap finished waiting for FSMap version 26 2023-10-06T02:24:50.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.893+0000 7f6ef5b0ab80 10 client.5586 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-06T02:24:50.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.893+0000 7f6ef5b0ab80 20 client.5586 populate_metadata read hostname 'smithi031' 2023-10-06T02:24:50.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.893+0000 7f6ef5b0ab80 1 --2- 172.21.15.31:0/1323518395 >> [v2:172.21.15.62:6832/2358776861,v1:172.21.15.62:6834/2358776861] conn(0x56384ba4b670 0x56384ba1a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:50.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.893+0000 7f6ef5b0ab80 4 client.5586 mds_command: new command op to 5290 tid=0[{"prefix": "status"}] 2023-10-06T02:24:50.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.893+0000 7f6ef5b0ab80 1 -- 172.21.15.31:0/1323518395 --> [v2:172.21.15.62:6832/2358776861,v1:172.21.15.62:6834/2358776861] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56384b8dee00 con 0x56384ba4b670 2023-10-06T02:24:50.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.893+0000 7f6ee499a700 1 --2- 172.21.15.31:0/1323518395 >> [v2:172.21.15.62:6832/2358776861,v1:172.21.15.62:6834/2358776861] conn(0x56384ba4b670 0x56384ba1a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:50.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.894+0000 7f6ebf7fe700 10 client.5586 ms_handle_connect on v2:172.21.15.62:6832/2358776861 2023-10-06T02:24:50.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.894+0000 7f6ee499a700 1 --2- 172.21.15.31:0/1323518395 >> [v2:172.21.15.62:6832/2358776861,v1:172.21.15.62:6834/2358776861] conn(0x56384ba4b670 0x56384ba1a4d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa8772dbbaf19ab6 server_cookie=83a56c75b453cd0b in_seq=0 out_seq=0 2023-10-06T02:24:50.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.894+0000 7f6ebf7fe700 1 -- 172.21.15.31:0/1323518395 <== mds.0 v2:172.21.15.62:6832/2358776861 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x56384b8dee00 con 0x56384ba4b670 2023-10-06T02:24:50.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.894+0000 7f6ebf7fe700 10 client.5586 handle_command_reply: tid=0 2023-10-06T02:24:50.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.894+0000 7f6ef5b0ab80 1 client.5586 shutdown 2023-10-06T02:24:50.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.894+0000 7f6ef5b0ab80 1 -- 172.21.15.31:0/1323518395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56384b9ed2c0 msgr2=0x56384ba4b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:50.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.894+0000 7f6ef5b0ab80 1 --2- 172.21.15.31:0/1323518395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56384b9ed2c0 0x56384ba4b130 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f6edc056a30 tx=0x7f6edc02f000 comp rx=0 tx=0).stop 2023-10-06T02:24:50.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.894+0000 7f6ef5b0ab80 1 -- 172.21.15.31:0/1323518395 shutdown_connections 2023-10-06T02:24:50.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.894+0000 7f6ef5b0ab80 1 --2- 172.21.15.31:0/1323518395 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56384ba0ce50 0x56384b9edda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:50.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.894+0000 7f6ef5b0ab80 1 --2- 172.21.15.31:0/1323518395 >> [v2:172.21.15.62:6832/2358776861,v1:172.21.15.62:6834/2358776861] conn(0x56384ba4b670 0x56384ba1a4d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:50.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.894+0000 7f6ef5b0ab80 1 --2- 172.21.15.31:0/1323518395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56384b9ed2c0 0x56384ba4b130 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:50.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.894+0000 7f6ef5b0ab80 1 --2- 172.21.15.31:0/1323518395 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56384b9eea10 0x56384b9ed860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:50.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.894+0000 7f6ebf7fe700 0 client.5586 ms_handle_reset on v2:172.21.15.62:6832/2358776861 2023-10-06T02:24:50.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.894+0000 7f6ef5b0ab80 1 -- 172.21.15.31:0/1323518395 >> 172.21.15.31:0/1323518395 conn(0x56384b9ece80 msgr2=0x56384ba0bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:50.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.894+0000 7f6ef5b0ab80 1 -- 172.21.15.31:0/1323518395 shutdown_connections 2023-10-06T02:24:50.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.894+0000 7f6ef5b0ab80 1 -- 172.21.15.31:0/1323518395 wait complete. 2023-10-06T02:24:50.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.895+0000 7f6ef5b0ab80 20 client.5586 trim_cache size 0 max 16384 2023-10-06T02:24:50.901 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T02:24:50.901 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "d6fcf512-874b-4259-b90a-30ae070f00a9", 2023-10-06T02:24:50.901 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 0, 2023-10-06T02:24:50.901 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 5290, 2023-10-06T02:24:50.901 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2023-10-06T02:24:50.902 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2023-10-06T02:24:50.902 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2023-10-06T02:24:50.902 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 6.5157317130000001, 2023-10-06T02:24:50.902 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 26, 2023-10-06T02:24:50.902 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 47, 2023-10-06T02:24:50.902 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 47, 2023-10-06T02:24:50.902 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 12.251815641 2023-10-06T02:24:50.902 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T02:24:50.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.895+0000 7f6ee619d700 1 -- 172.21.15.31:0/148857766 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6ec40670e0 msgr2=0x7f6ec4069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:50.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.895+0000 7f6ee619d700 1 --2- 172.21.15.31:0/148857766 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6ec40670e0 0x7f6ec4069590 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6ee01593b0 tx=0x7f6ed0009040 comp rx=0 tx=0).stop 2023-10-06T02:24:50.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.895+0000 7f6ee619d700 1 -- 172.21.15.31:0/148857766 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ee00cf110 msgr2=0x7f6ee00b3c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:50.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.895+0000 7f6ee619d700 1 --2- 172.21.15.31:0/148857766 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ee00cf110 0x7f6ee00b3c90 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f6edc009f50 tx=0x7f6edc017000 comp rx=0 tx=0).stop 2023-10-06T02:24:50.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.895+0000 7f6ee619d700 1 -- 172.21.15.31:0/148857766 shutdown_connections 2023-10-06T02:24:50.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.895+0000 7f6ee619d700 1 --2- 172.21.15.31:0/148857766 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ee015de50 0x7f6ee0143dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:50.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.895+0000 7f6ee619d700 1 --2- 172.21.15.31:0/148857766 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ee00cf110 0x7f6ee00b3c90 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:50.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.895+0000 7f6ee619d700 1 --2- 172.21.15.31:0/148857766 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6ec40670e0 0x7f6ec4069590 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:50.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.895+0000 7f6ee619d700 1 --2- 172.21.15.31:0/148857766 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6ee0153c30 0x7f6ee00b5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:50.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.895+0000 7f6ee619d700 1 -- 172.21.15.31:0/148857766 >> 172.21.15.31:0/148857766 conn(0x7f6ee00703a0 msgr2=0x7f6ee0153030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:50.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.895+0000 7f6ee619d700 1 -- 172.21.15.31:0/148857766 shutdown_connections 2023-10-06T02:24:50.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:50.895+0000 7f6ee619d700 1 -- 172.21.15.31:0/148857766 wait complete. 2023-10-06T02:24:50.912 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T02:24:51.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:51.099+0000 7f2099d9d700 1 Processor -- start 2023-10-06T02:24:51.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:51.099+0000 7f2099d9d700 1 -- start start 2023-10-06T02:24:51.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:51.100+0000 7f2099d9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2094159550 0x7f20940ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:51.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:51.100+0000 7f2099d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f209415dfd0 0x7f20940aedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:51.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:51.100+0000 7f2099d9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20940acc00 con 0x7f2094159920 2023-10-06T02:24:51.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:51.100+0000 7f2099d9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20940abff0 con 0x7f2094159550 2023-10-06T02:24:51.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:51.100+0000 7f2099d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20940ac2d0 con 0x7f209415dfd0 2023-10-06T02:24:51.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:51.101+0000 7f2098d9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2094159550 0x7f20940ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:51.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:51.101+0000 7f209959c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f2094159920 0x7f20940ad090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:59426/0 (socket says 172.21.15.31:59426) 2023-10-06T02:24:51.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:51.101+0000 7f2098d9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2094159550 0x7f20940ceac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53364/0 (socket says 172.21.15.31:53364) 2023-10-06T02:24:51.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:51.101+0000 7f209959c700 1 -- 172.21.15.31:0/646879588 learned_addr learned my addr 172.21.15.31:0/646879588 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:24:51.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:51.101+0000 7f2093fff700 1 --2- 172.21.15.31:0/646879588 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f209415dfd0 0x7f20940aedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:51.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:51.101+0000 7f2098d9b700 1 -- 172.21.15.31:0/646879588 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f209415dfd0 msgr2=0x7f20940aedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:51.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:51.101+0000 7f2098d9b700 1 --2- 172.21.15.31:0/646879588 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f209415dfd0 0x7f20940aedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:51.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:51.101+0000 7f2098d9b700 1 -- 172.21.15.31:0/646879588 >> v1:172.21.15.31:6789/0 conn(0x7f2094159920 legacy=0x7f20940ad090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:51.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:51.101+0000 7f2098d9b700 1 -- 172.21.15.31:0/646879588 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20940ac5b0 con 0x7f2094159550 2023-10-06T02:24:51.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:51.101+0000 7f20937fe700 1 -- 172.21.15.31:0/646879588 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3952637569 0 0) 0x7f20940acc00 con 0x7f2094159920 2023-10-06T02:24:51.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:51.101+0000 7f2093fff700 1 --2- 172.21.15.31:0/646879588 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f209415dfd0 0x7f20940aedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:24:51.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:51.102+0000 7f2098d9b700 1 --2- 172.21.15.31:0/646879588 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2094159550 0x7f20940ceac0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f20880098e0 tx=0x7f20880091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f51435d82f8b014 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:51.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:51.102+0000 7f20937fe700 1 -- 172.21.15.31:0/646879588 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2088009e80 con 0x7f2094159550 2023-10-06T02:24:51.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:51.102+0000 7f20937fe700 1 -- 172.21.15.31:0/646879588 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f208800b860 con 0x7f2094159550 2023-10-06T02:24:51.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:51.102+0000 7f20937fe700 1 -- 172.21.15.31:0/646879588 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f208800bbe0 con 0x7f2094159550 2023-10-06T02:24:51.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:51.102+0000 7f2099d9d700 1 -- 172.21.15.31:0/646879588 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2094159550 msgr2=0x7f20940ceac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:51.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:51.102+0000 7f2099d9d700 1 --2- 172.21.15.31:0/646879588 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2094159550 0x7f20940ceac0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f20880098e0 tx=0x7f20880091a0 comp rx=0 tx=0).stop 2023-10-06T02:24:51.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:51.102+0000 7f2099d9d700 1 -- 172.21.15.31:0/646879588 shutdown_connections 2023-10-06T02:24:51.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:51.102+0000 7f2099d9d700 1 --2- 172.21.15.31:0/646879588 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2094159550 0x7f20940ceac0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:51.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:51.102+0000 7f2099d9d700 1 --2- 172.21.15.31:0/646879588 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f209415dfd0 0x7f20940aedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:51.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:51.102+0000 7f2099d9d700 1 -- 172.21.15.31:0/646879588 >> 172.21.15.31:0/646879588 conn(0x7f209406ce80 msgr2=0x7f20940ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:51.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:51.103+0000 7f2099d9d700 1 -- 172.21.15.31:0/646879588 shutdown_connections 2023-10-06T02:24:51.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:51.103+0000 7f2099d9d700 1 -- 172.21.15.31:0/646879588 wait complete. 2023-10-06T02:24:51.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:51.104+0000 7f2099d9d700 1 Processor -- start 2023-10-06T02:24:51.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:51.104+0000 7f2099d9d700 1 -- start start 2023-10-06T02:24:51.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:51.105+0000 7f2099d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2094159550 0x7f2094156f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:51.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:51.105+0000 7f2099d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2094159920 0x7f2094140970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:51.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:51.105+0000 7f2099d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f209415dfd0 0x7f2094140eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:51.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:51.105+0000 7f2099d9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20941406d0 con 0x7f2094159920 2023-10-06T02:24:51.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:51.105+0000 7f2099d9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2094156a80 con 0x7f209415dfd0 2023-10-06T02:24:51.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:51.105+0000 7f2099d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2094155e70 con 0x7f2094159550 2023-10-06T02:24:51.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:51.105+0000 7f2093fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2094159920 0x7f2094140970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:51.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:51.105+0000 7f2093fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2094159920 0x7f2094140970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53378/0 (socket says 172.21.15.31:53378) 2023-10-06T02:24:51.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:51.105+0000 7f2093fff700 1 -- 172.21.15.31:0/2053160011 learned_addr learned my addr 172.21.15.31:0/2053160011 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:51.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:51.105+0000 7f209959c700 1 --2- 172.21.15.31:0/2053160011 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f209415dfd0 0x7f2094140eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:51.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:51.106+0000 7f2098d9b700 1 --2- 172.21.15.31:0/2053160011 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2094159550 0x7f2094156f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:51.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:51.106+0000 7f2093fff700 1 -- 172.21.15.31:0/2053160011 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2094159550 msgr2=0x7f2094156f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:51.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:51.106+0000 7f2093fff700 1 --2- 172.21.15.31:0/2053160011 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2094159550 0x7f2094156f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:51.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:51.106+0000 7f2093fff700 1 -- 172.21.15.31:0/2053160011 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f209415dfd0 msgr2=0x7f2094140eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:51.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:51.106+0000 7f2093fff700 1 --2- 172.21.15.31:0/2053160011 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f209415dfd0 0x7f2094140eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:51.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:51.106+0000 7f2093fff700 1 -- 172.21.15.31:0/2053160011 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2088009420 con 0x7f2094159920 2023-10-06T02:24:51.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:51.106+0000 7f209959c700 1 --2- 172.21.15.31:0/2053160011 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f209415dfd0 0x7f2094140eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:24:51.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:51.106+0000 7f2093fff700 1 --2- 172.21.15.31:0/2053160011 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2094159920 0x7f2094140970 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f208000e8d0 tx=0x7f2080008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:51.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:51.106+0000 7f2090ff9700 1 -- 172.21.15.31:0/2053160011 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f208000ee70 con 0x7f2094159920 2023-10-06T02:24:51.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:51.106+0000 7f2090ff9700 1 -- 172.21.15.31:0/2053160011 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f208000c5e0 con 0x7f2094159920 2023-10-06T02:24:51.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:51.106+0000 7f2099d9d700 1 -- 172.21.15.31:0/2053160011 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2094156150 con 0x7f2094159920 2023-10-06T02:24:51.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:51.106+0000 7f2090ff9700 1 -- 172.21.15.31:0/2053160011 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f208000c980 con 0x7f2094159920 2023-10-06T02:24:51.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:51.106+0000 7f2099d9d700 1 -- 172.21.15.31:0/2053160011 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2094156710 con 0x7f2094159920 2023-10-06T02:24:51.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:51.107+0000 7f2099d9d700 1 -- 172.21.15.31:0/2053160011 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2094156c30 con 0x7f2094159920 2023-10-06T02:24:51.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:51.108+0000 7f2090ff9700 1 -- 172.21.15.31:0/2053160011 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f2080013410 con 0x7f2094159920 2023-10-06T02:24:51.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:51.109+0000 7f2090ff9700 1 --2- 172.21.15.31:0/2053160011 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f20780670e0 0x7f2078069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:51.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:51.109+0000 7f2090ff9700 1 -- 172.21.15.31:0/2053160011 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2080084960 con 0x7f2094159920 2023-10-06T02:24:51.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:51.109+0000 7f2098d9b700 1 --2- 172.21.15.31:0/2053160011 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f20780670e0 0x7f2078069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:51.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:51.110+0000 7f2098d9b700 1 --2- 172.21.15.31:0/2053160011 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f20780670e0 0x7f2078069590 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f20880091e0 tx=0x7f2088017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:51.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:51.114+0000 7f2090ff9700 1 -- 172.21.15.31:0/2053160011 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2094156c30 con 0x7f2094159920 2023-10-06T02:24:51.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:51.365+0000 7f2099d9d700 1 -- 172.21.15.31:0/2053160011 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2094156bf0 con 0x7f2094159920 2023-10-06T02:24:51.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:51.365+0000 7f2090ff9700 1 -- 172.21.15.31:0/2053160011 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) v1 ==== 53+0+73 (secure 0 0 0) 0x7f2080051640 con 0x7f2094159920 2023-10-06T02:24:51.366 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T02:24:51.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:51.368+0000 7f2099d9d700 1 -- 172.21.15.31:0/2053160011 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f20780670e0 msgr2=0x7f2078069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:51.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:51.368+0000 7f2099d9d700 1 --2- 172.21.15.31:0/2053160011 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f20780670e0 0x7f2078069590 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f20880091e0 tx=0x7f2088017000 comp rx=0 tx=0).stop 2023-10-06T02:24:51.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:51.368+0000 7f2099d9d700 1 -- 172.21.15.31:0/2053160011 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2094159920 msgr2=0x7f2094140970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:51.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:51.368+0000 7f2099d9d700 1 --2- 172.21.15.31:0/2053160011 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2094159920 0x7f2094140970 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f208000e8d0 tx=0x7f2080008040 comp rx=0 tx=0).stop 2023-10-06T02:24:51.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:51.368+0000 7f2099d9d700 1 -- 172.21.15.31:0/2053160011 shutdown_connections 2023-10-06T02:24:51.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:51.368+0000 7f2099d9d700 1 --2- 172.21.15.31:0/2053160011 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f209415dfd0 0x7f2094140eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:51.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:51.368+0000 7f2099d9d700 1 --2- 172.21.15.31:0/2053160011 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f20780670e0 0x7f2078069590 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:51.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:51.368+0000 7f2099d9d700 1 --2- 172.21.15.31:0/2053160011 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2094159550 0x7f2094156f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:51.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:51.368+0000 7f2099d9d700 1 --2- 172.21.15.31:0/2053160011 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2094159920 0x7f2094140970 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:51.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:51.368+0000 7f2099d9d700 1 -- 172.21.15.31:0/2053160011 >> 172.21.15.31:0/2053160011 conn(0x7f209406ce80 msgr2=0x7f2094068240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:51.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:51.368+0000 7f2099d9d700 1 -- 172.21.15.31:0/2053160011 shutdown_connections 2023-10-06T02:24:51.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:51.368+0000 7f2099d9d700 1 -- 172.21.15.31:0/2053160011 wait complete. 2023-10-06T02:24:51.385 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T02:24:51.385 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T02:24:51.385 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi031.front.sepia.ceph.com 2023-10-06T02:24:51.385 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T02:24:51.386 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T02:24:51.386 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T02:24:51.386 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T02:24:51.386 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T02:24:51.386 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T02:24:51.386 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:24:51.386 DEBUG:teuthology.orchestra.run.smithi031:> ip addr 2023-10-06T02:24:51.426 INFO:teuthology.orchestra.run.smithi031.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T02:24:51.427 INFO:teuthology.orchestra.run.smithi031.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T02:24:51.427 INFO:teuthology.orchestra.run.smithi031.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T02:24:51.427 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:24:51.427 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 ::1/128 scope host 2023-10-06T02:24:51.427 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:24:51.427 INFO:teuthology.orchestra.run.smithi031.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T02:24:51.427 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cc brd ff:ff:ff:ff:ff:ff 2023-10-06T02:24:51.427 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f0 2023-10-06T02:24:51.427 INFO:teuthology.orchestra.run.smithi031.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T02:24:51.427 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cd brd ff:ff:ff:ff:ff:ff 2023-10-06T02:24:51.427 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f1 2023-10-06T02:24:51.428 INFO:teuthology.orchestra.run.smithi031.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T02:24:51.428 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:ba brd ff:ff:ff:ff:ff:ff 2023-10-06T02:24:51.428 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f0 2023-10-06T02:24:51.428 INFO:teuthology.orchestra.run.smithi031.stdout: inet 172.21.15.31/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T02:24:51.428 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft 42165sec preferred_lft 42165sec 2023-10-06T02:24:51.428 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::ae1f:6bff:fea5:caba/64 scope link 2023-10-06T02:24:51.428 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:24:51.428 INFO:teuthology.orchestra.run.smithi031.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T02:24:51.428 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:bb brd ff:ff:ff:ff:ff:ff 2023-10-06T02:24:51.428 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f1 2023-10-06T02:24:51.429 INFO:teuthology.orchestra.run.smithi031.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T02:24:51.429 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 16:d7:88:2d:d0:c2 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:24:51.429 INFO:teuthology.orchestra.run.smithi031.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T02:24:51.429 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:24:51.429 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::408:52ff:fe3c:ba8f/64 scope link 2023-10-06T02:24:51.429 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:24:51.429 INFO:teuthology.orchestra.run.smithi031.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T02:24:51.429 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 16:d7:88:2d:d0:c2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T02:24:51.429 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::14d7:88ff:fe2d:d0c2/64 scope link 2023-10-06T02:24:51.429 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:24:51.429 INFO:teuthology.orchestra.run.smithi031.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T02:24:51.429 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 82:16:f1:00:cf:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T02:24:51.429 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::8016:f1ff:fe00:cf2f/64 scope link 2023-10-06T02:24:51.430 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:24:51.430 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:24:51.430 DEBUG:teuthology.orchestra.run.smithi031:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T02:24:51.527 INFO:teuthology.orchestra.run.smithi031.stdout:1 2023-10-06T02:24:51.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:24:51.530 DEBUG:teuthology.orchestra.run.smithi031:> route 2023-10-06T02:24:51.569 INFO:teuthology.orchestra.run.smithi031.stdout:Kernel IP routing table 2023-10-06T02:24:51.569 INFO:teuthology.orchestra.run.smithi031.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T02:24:51.569 INFO:teuthology.orchestra.run.smithi031.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T02:24:51.569 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T02:24:51.569 INFO:teuthology.orchestra.run.smithi031.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T02:24:51.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:24:51.570 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2023-10-06T02:24:51.570 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2023-10-06T02:24:51.570 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T02:24:51.571 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T02:24:51.571 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T02:24:51.571 DEBUG:teuthology.orchestra.run.smithi031:> ') 2023-10-06T02:24:51.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:24:51.768 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2023-10-06T02:24:51.803 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T02:24:51.804 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T02:24:51.805 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T02:24:51.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:24:51.805 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set brx.0 up 2023-10-06T02:24:51.907 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T02:24:51.907 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T02:24:51.943 INFO:teuthology.orchestra.run.smithi031.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T02:24:51.943 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T02:24:51.943 DEBUG:teuthology.orchestra.run.smithi031:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T02:24:52.018 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-06T02:24:52.019 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T02:24:52.019 DEBUG:teuthology.orchestra.run.smithi031:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T02:24:52.151 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T02:24:52.151 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T02:24:52.179 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T02:24:52.179 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T02:24:52.179 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.62:3300,172.21.15.62:3301" 2023-10-06T02:24:52.179 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: trying mount with new device syntax: 0@d6fcf512-874b-4259-b90a-30ae070f00a9.cephfs=/ 2023-10-06T02:24:52.180 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.31:3300/172.21.15.62:3300/172.21.15.62:3301" will pass to kernel 2023-10-06T02:24:52.185 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T02:24:52.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:24:52.185 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T02:24:52.249 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T02:24:52.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:52.437+0000 7f242219d700 1 Processor -- start 2023-10-06T02:24:52.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:52.437+0000 7f242219d700 1 -- start start 2023-10-06T02:24:52.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:52.438+0000 7f242219d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f241c159540 0x7f241c0c1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:52.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:52.438+0000 7f242219d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f241c0c6be0 0x7f241c0c6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:52.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:52.438+0000 7f242219d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f241c0c7520 con 0x7f241c159910 2023-10-06T02:24:52.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:52.438+0000 7f242219d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f241c068250 con 0x7f241c159540 2023-10-06T02:24:52.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:52.438+0000 7f242219d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f241c068530 con 0x7f241c0c6be0 2023-10-06T02:24:52.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:52.438+0000 7f242119b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f241c159540 0x7f241c0c1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:52.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:52.438+0000 7f242119b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f241c159540 0x7f241c0c1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53384/0 (socket says 172.21.15.31:53384) 2023-10-06T02:24:52.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:52.438+0000 7f242119b700 1 -- 172.21.15.31:0/69725926 learned_addr learned my addr 172.21.15.31:0/69725926 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:52.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:52.439+0000 7f242119b700 1 -- 172.21.15.31:0/69725926 >> v2:172.21.15.62:3300/0 conn(0x7f241c0c6be0 msgr2=0x7f241c0c6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:52.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:52.439+0000 7f242199c700 1 --2- 172.21.15.31:0/69725926 >> v2:172.21.15.62:3300/0 conn(0x7f241c0c6be0 0x7f241c0c6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:52.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:52.439+0000 7f242119b700 1 --2- 172.21.15.31:0/69725926 >> v2:172.21.15.62:3300/0 conn(0x7f241c0c6be0 0x7f241c0c6fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:52.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:52.439+0000 7f242119b700 1 -- 172.21.15.31:0/69725926 >> v1:172.21.15.62:6789/0 conn(0x7f241c159910 legacy=0x7f241c0677b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:52.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:52.439+0000 7f242119b700 1 -- 172.21.15.31:0/69725926 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f241c068810 con 0x7f241c159540 2023-10-06T02:24:52.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:52.439+0000 7f242199c700 1 --2- 172.21.15.31:0/69725926 >> v2:172.21.15.62:3300/0 conn(0x7f241c0c6be0 0x7f241c0c6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:24:52.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:52.439+0000 7f242119b700 1 --2- 172.21.15.31:0/69725926 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f241c159540 0x7f241c0c1340 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f241c069ec0 tx=0x7f2418009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32e6a57bbd5b015b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:52.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:52.439+0000 7f2413fff700 1 -- 172.21.15.31:0/69725926 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241800b820 con 0x7f241c159540 2023-10-06T02:24:52.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:52.439+0000 7f2413fff700 1 -- 172.21.15.31:0/69725926 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f241800b980 con 0x7f241c159540 2023-10-06T02:24:52.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:52.439+0000 7f2413fff700 1 -- 172.21.15.31:0/69725926 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241800bd20 con 0x7f241c159540 2023-10-06T02:24:52.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:52.440+0000 7f242219d700 1 -- 172.21.15.31:0/69725926 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f241c159540 msgr2=0x7f241c0c1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:52.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:52.440+0000 7f242219d700 1 --2- 172.21.15.31:0/69725926 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f241c159540 0x7f241c0c1340 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f241c069ec0 tx=0x7f2418009250 comp rx=0 tx=0).stop 2023-10-06T02:24:52.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:52.440+0000 7f242219d700 1 -- 172.21.15.31:0/69725926 shutdown_connections 2023-10-06T02:24:52.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:52.440+0000 7f242219d700 1 --2- 172.21.15.31:0/69725926 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f241c159540 0x7f241c0c1340 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:52.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:52.440+0000 7f242219d700 1 --2- 172.21.15.31:0/69725926 >> v2:172.21.15.62:3300/0 conn(0x7f241c0c6be0 0x7f241c0c6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:52.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:52.440+0000 7f242219d700 1 -- 172.21.15.31:0/69725926 >> 172.21.15.31:0/69725926 conn(0x7f241c06f310 msgr2=0x7f241c0cc5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:52.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:52.440+0000 7f242219d700 1 -- 172.21.15.31:0/69725926 shutdown_connections 2023-10-06T02:24:52.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:52.440+0000 7f242219d700 1 -- 172.21.15.31:0/69725926 wait complete. 2023-10-06T02:24:52.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:52.441+0000 7f242219d700 1 Processor -- start 2023-10-06T02:24:52.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:52.442+0000 7f242219d700 1 -- start start 2023-10-06T02:24:52.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:52.443+0000 7f242219d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f241c0c6be0 0x7f241c0c3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:52.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:52.443+0000 7f242219d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f241c159540 0x7f241c169ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:52.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:52.443+0000 7f242219d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f241c159910 0x7f241c0b1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:52.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:52.443+0000 7f242219d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f241c0c6660 con 0x7f241c159910 2023-10-06T02:24:52.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:52.443+0000 7f242219d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f241c0c61d0 con 0x7f241c0c6be0 2023-10-06T02:24:52.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:52.443+0000 7f242219d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f241c16aa70 con 0x7f241c159540 2023-10-06T02:24:52.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:52.443+0000 7f242199c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f241c159910 0x7f241c0b1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:52.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:52.443+0000 7f242119b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f241c0c6be0 0x7f241c0c3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:52.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:52.443+0000 7f242199c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f241c159910 0x7f241c0b1320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53398/0 (socket says 172.21.15.31:53398) 2023-10-06T02:24:52.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:52.443+0000 7f242199c700 1 -- 172.21.15.31:0/2629644895 learned_addr learned my addr 172.21.15.31:0/2629644895 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:52.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:52.443+0000 7f242099a700 1 --2- 172.21.15.31:0/2629644895 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f241c159540 0x7f241c169ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:52.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:52.443+0000 7f242199c700 1 -- 172.21.15.31:0/2629644895 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f241c159540 msgr2=0x7f241c169ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:52.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:52.443+0000 7f242199c700 1 --2- 172.21.15.31:0/2629644895 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f241c159540 0x7f241c169ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:52.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:52.443+0000 7f242199c700 1 -- 172.21.15.31:0/2629644895 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f241c0c6be0 msgr2=0x7f241c0c3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:52.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:52.443+0000 7f242199c700 1 --2- 172.21.15.31:0/2629644895 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f241c0c6be0 0x7f241c0c3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:52.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:52.443+0000 7f242199c700 1 -- 172.21.15.31:0/2629644895 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24180094f0 con 0x7f241c159910 2023-10-06T02:24:52.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:52.443+0000 7f242099a700 1 --2- 172.21.15.31:0/2629644895 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f241c159540 0x7f241c169ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:24:52.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:52.444+0000 7f242199c700 1 --2- 172.21.15.31:0/2629644895 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f241c159910 0x7f241c0b1320 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f241400a380 tx=0x7f2414007900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:52.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:52.444+0000 7f24117fa700 1 -- 172.21.15.31:0/2629644895 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2414015440 con 0x7f241c159910 2023-10-06T02:24:52.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:52.444+0000 7f24117fa700 1 -- 172.21.15.31:0/2629644895 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24140155a0 con 0x7f241c159910 2023-10-06T02:24:52.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:52.444+0000 7f24117fa700 1 -- 172.21.15.31:0/2629644895 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2414015960 con 0x7f241c159910 2023-10-06T02:24:52.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:52.444+0000 7f242219d700 1 -- 172.21.15.31:0/2629644895 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f241c0c55c0 con 0x7f241c159910 2023-10-06T02:24:52.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:52.444+0000 7f242219d700 1 -- 172.21.15.31:0/2629644895 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f241c0c5b80 con 0x7f241c159910 2023-10-06T02:24:52.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:52.445+0000 7f242219d700 1 -- 172.21.15.31:0/2629644895 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f241c0c1550 con 0x7f241c159910 2023-10-06T02:24:52.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:52.446+0000 7f24117fa700 1 -- 172.21.15.31:0/2629644895 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f2414015d50 con 0x7f241c159910 2023-10-06T02:24:52.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:52.447+0000 7f24117fa700 1 --2- 172.21.15.31:0/2629644895 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f24000670e0 0x7f2400069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:52.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:52.447+0000 7f24117fa700 1 -- 172.21.15.31:0/2629644895 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2414085380 con 0x7f241c159910 2023-10-06T02:24:52.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:52.447+0000 7f242119b700 1 --2- 172.21.15.31:0/2629644895 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f24000670e0 0x7f2400069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:52.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:52.448+0000 7f242119b700 1 --2- 172.21.15.31:0/2629644895 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f24000670e0 0x7f2400069590 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f241c069ec0 tx=0x7f2418016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:52.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:52.451+0000 7f24117fa700 1 -- 172.21.15.31:0/2629644895 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2414052170 con 0x7f241c159910 2023-10-06T02:24:52.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:52.699+0000 7f242219d700 1 -- 172.21.15.31:0/2629644895 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f241c04c3c0 con 0x7f241c159910 2023-10-06T02:24:52.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:52.699+0000 7f24117fa700 1 -- 172.21.15.31:0/2629644895 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) v1 ==== 53+0+73 (secure 0 0 0) 0x7f2414055790 con 0x7f241c159910 2023-10-06T02:24:52.700 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T02:24:52.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:52.701+0000 7f242219d700 1 -- 172.21.15.31:0/2629644895 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f24000670e0 msgr2=0x7f2400069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:52.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:52.701+0000 7f242219d700 1 --2- 172.21.15.31:0/2629644895 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f24000670e0 0x7f2400069590 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f241c069ec0 tx=0x7f2418016000 comp rx=0 tx=0).stop 2023-10-06T02:24:52.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:52.701+0000 7f242219d700 1 -- 172.21.15.31:0/2629644895 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f241c159910 msgr2=0x7f241c0b1320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:52.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:52.701+0000 7f242219d700 1 --2- 172.21.15.31:0/2629644895 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f241c159910 0x7f241c0b1320 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f241400a380 tx=0x7f2414007900 comp rx=0 tx=0).stop 2023-10-06T02:24:52.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:52.701+0000 7f242219d700 1 -- 172.21.15.31:0/2629644895 shutdown_connections 2023-10-06T02:24:52.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:52.701+0000 7f242219d700 1 --2- 172.21.15.31:0/2629644895 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f241c159910 0x7f241c0b1320 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:52.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:52.701+0000 7f242219d700 1 --2- 172.21.15.31:0/2629644895 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f241c0c6be0 0x7f241c0c3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:52.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:52.701+0000 7f242219d700 1 --2- 172.21.15.31:0/2629644895 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f24000670e0 0x7f2400069590 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:52.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:52.701+0000 7f242219d700 1 --2- 172.21.15.31:0/2629644895 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f241c159540 0x7f241c169ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:52.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:52.701+0000 7f242219d700 1 -- 172.21.15.31:0/2629644895 >> 172.21.15.31:0/2629644895 conn(0x7f241c06f310 msgr2=0x7f241c0ca4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:52.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:52.702+0000 7f242219d700 1 -- 172.21.15.31:0/2629644895 shutdown_connections 2023-10-06T02:24:52.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:52.702+0000 7f242219d700 1 -- 172.21.15.31:0/2629644895 wait complete. 2023-10-06T02:24:52.719 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T02:24:52.719 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T02:24:52.719 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi062.front.sepia.ceph.com 2023-10-06T02:24:52.719 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-06T02:24:52.719 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-06T02:24:52.719 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T02:24:52.719 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T02:24:52.719 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T02:24:52.719 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T02:24:52.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:24:52.720 DEBUG:teuthology.orchestra.run.smithi062:> ip addr 2023-10-06T02:24:52.762 INFO:teuthology.orchestra.run.smithi062.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T02:24:52.762 INFO:teuthology.orchestra.run.smithi062.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T02:24:52.762 INFO:teuthology.orchestra.run.smithi062.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T02:24:52.762 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:24:52.762 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 ::1/128 scope host 2023-10-06T02:24:52.762 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:24:52.762 INFO:teuthology.orchestra.run.smithi062.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T02:24:52.763 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:14 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:24:52.763 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f0 2023-10-06T02:24:52.763 INFO:teuthology.orchestra.run.smithi062.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T02:24:52.763 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:15 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:24:52.763 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f1 2023-10-06T02:24:52.763 INFO:teuthology.orchestra.run.smithi062.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T02:24:52.763 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c2 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:24:52.763 INFO:teuthology.orchestra.run.smithi062.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T02:24:52.763 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c3 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:24:52.763 INFO:teuthology.orchestra.run.smithi062.stdout: inet 172.21.15.62/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T02:24:52.763 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft 42156sec preferred_lft 42156sec 2023-10-06T02:24:52.763 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::ec4:7aff:febb:d1c3/64 scope link 2023-10-06T02:24:52.764 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:24:52.764 INFO:teuthology.orchestra.run.smithi062.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T02:24:52.764 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether ae:42:a8:dd:f7:9b brd ff:ff:ff:ff:ff:ff 2023-10-06T02:24:52.764 INFO:teuthology.orchestra.run.smithi062.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T02:24:52.764 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:24:52.764 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::1098:a4ff:fe0d:389/64 scope link 2023-10-06T02:24:52.764 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:24:52.764 INFO:teuthology.orchestra.run.smithi062.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T02:24:52.764 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether fe:60:2b:8e:af:28 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T02:24:52.764 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::fc60:2bff:fe8e:af28/64 scope link 2023-10-06T02:24:52.764 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:24:52.765 INFO:teuthology.orchestra.run.smithi062.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T02:24:52.765 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether ae:42:a8:dd:f7:9b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T02:24:52.765 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::ac42:a8ff:fedd:f79b/64 scope link 2023-10-06T02:24:52.765 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:24:52.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:24:52.766 DEBUG:teuthology.orchestra.run.smithi062:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T02:24:52.872 INFO:teuthology.orchestra.run.smithi062.stdout:1 2023-10-06T02:24:52.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:24:52.875 DEBUG:teuthology.orchestra.run.smithi062:> route 2023-10-06T02:24:52.915 INFO:teuthology.orchestra.run.smithi062.stdout:Kernel IP routing table 2023-10-06T02:24:52.915 INFO:teuthology.orchestra.run.smithi062.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T02:24:52.915 INFO:teuthology.orchestra.run.smithi062.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T02:24:52.916 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T02:24:52.916 INFO:teuthology.orchestra.run.smithi062.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T02:24:52.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:24:52.917 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2023-10-06T02:24:52.917 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2023-10-06T02:24:52.917 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T02:24:52.917 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T02:24:52.917 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T02:24:52.917 DEBUG:teuthology.orchestra.run.smithi062:> ') 2023-10-06T02:24:53.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:24:53.116 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2023-10-06T02:24:53.152 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-06T02:24:53.152 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T02:24:53.152 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T02:24:53.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:24:53.152 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set brx.0 up 2023-10-06T02:24:53.254 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T02:24:53.254 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-06T02:24:53.291 INFO:teuthology.orchestra.run.smithi062.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-06T02:24:53.291 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T02:24:53.291 DEBUG:teuthology.orchestra.run.smithi062:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-06T02:24:53.366 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-06T02:24:53.366 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T02:24:53.367 DEBUG:teuthology.orchestra.run.smithi062:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T02:24:53.498 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T02:24:53.499 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T02:24:53.527 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T02:24:53.528 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T02:24:53.528 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.62:3300,172.21.15.62:3301" 2023-10-06T02:24:53.528 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: trying mount with new device syntax: 1@d6fcf512-874b-4259-b90a-30ae070f00a9.cephfs=/ 2023-10-06T02:24:53.528 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.31:3300/172.21.15.62:3300/172.21.15.62:3301" will pass to kernel 2023-10-06T02:24:53.532 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T02:24:53.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:24:53.532 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-06T02:24:53.594 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-10-06T02:24:53.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:53.796+0000 7f54f9d9d700 1 Processor -- start 2023-10-06T02:24:53.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:53.797+0000 7f54f9d9d700 1 -- start start 2023-10-06T02:24:53.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:53.798+0000 7f54f9d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f54f415cb90 0x7f54f40c7870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:53.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:53.798+0000 7f54f9d9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f54f415dc10 0x7f54f406c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:53.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:53.798+0000 7f54f9d9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54f406d040 con 0x7f54f406cbe0 2023-10-06T02:24:53.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:53.798+0000 7f54f9d9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54f40ccdf0 con 0x7f54f415dc10 2023-10-06T02:24:53.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:53.798+0000 7f54f9d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54f40c9b20 con 0x7f54f415cb90 2023-10-06T02:24:53.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:53.798+0000 7f54f3fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f54f415dc10 0x7f54f406c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:53.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:53.798+0000 7f54f8d9b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f54f415cb90 0x7f54f40c7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:53.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:53.798+0000 7f54f3fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f54f415dc10 0x7f54f406c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53408/0 (socket says 172.21.15.31:53408) 2023-10-06T02:24:53.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:53.798+0000 7f54f3fff700 1 -- 172.21.15.31:0/3269448858 learned_addr learned my addr 172.21.15.31:0/3269448858 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:53.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:53.798+0000 7f54f959c700 1 --1- 172.21.15.31:0/3269448858 >> v1:172.21.15.62:6789/0 conn(0x7f54f406cbe0 0x7f54f40c3430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:52384/0 (socket says 172.21.15.31:52384) 2023-10-06T02:24:53.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:53.799+0000 7f54f8d9b700 1 -- 172.21.15.31:0/3269448858 >> v2:172.21.15.31:3300/0 conn(0x7f54f415dc10 msgr2=0x7f54f406c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:53.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:53.799+0000 7f54f8d9b700 1 --2- 172.21.15.31:0/3269448858 >> v2:172.21.15.31:3300/0 conn(0x7f54f415dc10 0x7f54f406c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:53.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:53.799+0000 7f54f8d9b700 1 -- 172.21.15.31:0/3269448858 >> v1:172.21.15.62:6789/0 conn(0x7f54f406cbe0 legacy=0x7f54f40c3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:53.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:53.799+0000 7f54f8d9b700 1 -- 172.21.15.31:0/3269448858 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54f40c2fa0 con 0x7f54f415cb90 2023-10-06T02:24:53.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:53.799+0000 7f54f37fe700 1 -- 172.21.15.31:0/3269448858 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1499285562 0 0) 0x7f54ec001490 con 0x7f54f406cbe0 2023-10-06T02:24:53.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:53.799+0000 7f54f8d9b700 1 --2- 172.21.15.31:0/3269448858 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f54f415cb90 0x7f54f40c7870 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f54e80098e0 tx=0x7f54e80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75820289fee31aab server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:53.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:53.799+0000 7f54f37fe700 1 -- 172.21.15.31:0/3269448858 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e800b640 con 0x7f54f415cb90 2023-10-06T02:24:53.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:53.799+0000 7f54f37fe700 1 -- 172.21.15.31:0/3269448858 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54e800b7a0 con 0x7f54f415cb90 2023-10-06T02:24:53.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:53.799+0000 7f54f37fe700 1 -- 172.21.15.31:0/3269448858 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e800bb40 con 0x7f54f415cb90 2023-10-06T02:24:53.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:53.800+0000 7f54f9d9d700 1 -- 172.21.15.31:0/3269448858 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f54f415cb90 msgr2=0x7f54f40c7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:53.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:53.800+0000 7f54f9d9d700 1 --2- 172.21.15.31:0/3269448858 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f54f415cb90 0x7f54f40c7870 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f54e80098e0 tx=0x7f54e80091a0 comp rx=0 tx=0).stop 2023-10-06T02:24:53.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:53.800+0000 7f54f9d9d700 1 -- 172.21.15.31:0/3269448858 shutdown_connections 2023-10-06T02:24:53.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:53.800+0000 7f54f9d9d700 1 --2- 172.21.15.31:0/3269448858 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f54f415cb90 0x7f54f40c7870 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:53.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:53.800+0000 7f54f9d9d700 1 --2- 172.21.15.31:0/3269448858 >> v2:172.21.15.31:3300/0 conn(0x7f54f415dc10 0x7f54f406c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:53.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:53.800+0000 7f54f9d9d700 1 -- 172.21.15.31:0/3269448858 >> 172.21.15.31:0/3269448858 conn(0x7f54f40cd230 msgr2=0x7f54f40cd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:53.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:53.800+0000 7f54f9d9d700 1 -- 172.21.15.31:0/3269448858 shutdown_connections 2023-10-06T02:24:53.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:53.800+0000 7f54f9d9d700 1 -- 172.21.15.31:0/3269448858 wait complete. 2023-10-06T02:24:53.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:53.801+0000 7f54f9d9d700 1 Processor -- start 2023-10-06T02:24:53.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:53.801+0000 7f54f9d9d700 1 -- start start 2023-10-06T02:24:53.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:53.802+0000 7f54f9d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f54f406cbe0 0x7f54f413f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:53.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:53.802+0000 7f54f9d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54f415cb90 0x7f54f40c4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:53.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:53.802+0000 7f54f9d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54f415dc10 0x7f54f40c5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:53.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:53.802+0000 7f54f9d9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54f413fc20 con 0x7f54f415cb90 2023-10-06T02:24:53.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:53.802+0000 7f54f9d9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54f413dce0 con 0x7f54f415dc10 2023-10-06T02:24:53.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:53.802+0000 7f54f9d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54f416de00 con 0x7f54f406cbe0 2023-10-06T02:24:53.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:53.803+0000 7f54f3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54f415cb90 0x7f54f40c4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:53.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:53.803+0000 7f54f3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54f415cb90 0x7f54f40c4c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53414/0 (socket says 172.21.15.31:53414) 2023-10-06T02:24:53.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:53.803+0000 7f54f3fff700 1 -- 172.21.15.31:0/1512753755 learned_addr learned my addr 172.21.15.31:0/1512753755 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:53.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:53.803+0000 7f54f8d9b700 1 --2- 172.21.15.31:0/1512753755 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f54f406cbe0 0x7f54f413f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:53.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:53.803+0000 7f54f959c700 1 --2- 172.21.15.31:0/1512753755 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54f415dc10 0x7f54f40c5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:53.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:53.803+0000 7f54f3fff700 1 -- 172.21.15.31:0/1512753755 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f54f406cbe0 msgr2=0x7f54f413f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:53.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:53.803+0000 7f54f3fff700 1 --2- 172.21.15.31:0/1512753755 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f54f406cbe0 0x7f54f413f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:53.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:53.803+0000 7f54f3fff700 1 -- 172.21.15.31:0/1512753755 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54f415dc10 msgr2=0x7f54f40c5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:53.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:53.803+0000 7f54f3fff700 1 --2- 172.21.15.31:0/1512753755 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54f415dc10 0x7f54f40c5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:53.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:53.803+0000 7f54f3fff700 1 -- 172.21.15.31:0/1512753755 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54e8009420 con 0x7f54f415cb90 2023-10-06T02:24:53.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:53.803+0000 7f54f3fff700 1 --2- 172.21.15.31:0/1512753755 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54f415cb90 0x7f54f40c4c40 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f54e000ea80 tx=0x7f54e0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:53.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:53.803+0000 7f54f0ff9700 1 -- 172.21.15.31:0/1512753755 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e000ee60 con 0x7f54f415cb90 2023-10-06T02:24:53.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:53.803+0000 7f54f0ff9700 1 -- 172.21.15.31:0/1512753755 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54e000c850 con 0x7f54f415cb90 2023-10-06T02:24:53.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:53.803+0000 7f54f0ff9700 1 -- 172.21.15.31:0/1512753755 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e000cbd0 con 0x7f54f415cb90 2023-10-06T02:24:53.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:53.804+0000 7f54f9d9d700 1 -- 172.21.15.31:0/1512753755 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54f416e290 con 0x7f54f415cb90 2023-10-06T02:24:53.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:53.804+0000 7f54f9d9d700 1 -- 172.21.15.31:0/1512753755 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54f416d4d0 con 0x7f54f415cb90 2023-10-06T02:24:53.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:53.805+0000 7f54f9d9d700 1 -- 172.21.15.31:0/1512753755 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54f416df90 con 0x7f54f415cb90 2023-10-06T02:24:53.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:53.805+0000 7f54f0ff9700 1 -- 172.21.15.31:0/1512753755 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f54e0013470 con 0x7f54f415cb90 2023-10-06T02:24:53.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:53.806+0000 7f54f0ff9700 1 --2- 172.21.15.31:0/1512753755 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f54d80670e0 0x7f54d8069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:53.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:53.806+0000 7f54f0ff9700 1 -- 172.21.15.31:0/1512753755 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f54e0084b80 con 0x7f54f415cb90 2023-10-06T02:24:53.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:53.807+0000 7f54f8d9b700 1 --2- 172.21.15.31:0/1512753755 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f54d80670e0 0x7f54d8069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:53.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:53.807+0000 7f54f8d9b700 1 --2- 172.21.15.31:0/1512753755 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f54d80670e0 0x7f54d8069590 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f54e80093f0 tx=0x7f54e8017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:53.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:53.810+0000 7f54f0ff9700 1 -- 172.21.15.31:0/1512753755 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f54e0051970 con 0x7f54f415cb90 2023-10-06T02:24:54.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:54.060+0000 7f54f9d9d700 1 -- 172.21.15.31:0/1512753755 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f54f404c3c0 con 0x7f54f415cb90 2023-10-06T02:24:54.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:54.765+0000 7f54f0ff9700 1 -- 172.21.15.31:0/1512753755 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v27) v1 ==== 136+0+0 (secure 0 0 0) 0x7f54e0054f90 con 0x7f54f415cb90 2023-10-06T02:24:54.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:54.768+0000 7f54f9d9d700 1 -- 172.21.15.31:0/1512753755 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f54d80670e0 msgr2=0x7f54d8069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:54.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:54.768+0000 7f54f9d9d700 1 --2- 172.21.15.31:0/1512753755 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f54d80670e0 0x7f54d8069590 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f54e80093f0 tx=0x7f54e8017000 comp rx=0 tx=0).stop 2023-10-06T02:24:54.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:54.768+0000 7f54f9d9d700 1 -- 172.21.15.31:0/1512753755 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54f415cb90 msgr2=0x7f54f40c4c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:54.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:54.768+0000 7f54f9d9d700 1 --2- 172.21.15.31:0/1512753755 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54f415cb90 0x7f54f40c4c40 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f54e000ea80 tx=0x7f54e0009250 comp rx=0 tx=0).stop 2023-10-06T02:24:54.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:54.768+0000 7f54f9d9d700 1 -- 172.21.15.31:0/1512753755 shutdown_connections 2023-10-06T02:24:54.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:54.768+0000 7f54f9d9d700 1 --2- 172.21.15.31:0/1512753755 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54f415dc10 0x7f54f40c5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:54.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:54.768+0000 7f54f9d9d700 1 --2- 172.21.15.31:0/1512753755 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f54d80670e0 0x7f54d8069590 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:54.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:54.768+0000 7f54f9d9d700 1 --2- 172.21.15.31:0/1512753755 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f54f406cbe0 0x7f54f413f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:54.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:54.768+0000 7f54f9d9d700 1 --2- 172.21.15.31:0/1512753755 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54f415cb90 0x7f54f40c4c40 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:54.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:54.768+0000 7f54f9d9d700 1 -- 172.21.15.31:0/1512753755 >> 172.21.15.31:0/1512753755 conn(0x7f54f40cd230 msgr2=0x7f54f40caf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:54.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:54.768+0000 7f54f9d9d700 1 -- 172.21.15.31:0/1512753755 shutdown_connections 2023-10-06T02:24:54.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:54.768+0000 7f54f9d9d700 1 -- 172.21.15.31:0/1512753755 wait complete. 2023-10-06T02:24:54.790 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-06T02:24:54.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:54.980+0000 7fe694f9d700 1 Processor -- start 2023-10-06T02:24:54.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:54.980+0000 7fe694f9d700 1 -- start start 2023-10-06T02:24:54.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:54.981+0000 7fe694f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe690159550 0x7fe6900ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:54.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:54.981+0000 7fe694f9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe690159920 0x7fe6900ad090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:54.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:54.981+0000 7fe694f9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6900acc00 con 0x7fe69015dfd0 2023-10-06T02:24:54.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:54.981+0000 7fe694f9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6900abff0 con 0x7fe690159920 2023-10-06T02:24:54.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:54.981+0000 7fe694f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6900ac2d0 con 0x7fe690159550 2023-10-06T02:24:54.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:54.981+0000 7fe68ffff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe690159920 0x7fe6900ad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:54.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:54.981+0000 7fe68effd700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fe69015dfd0 0x7fe6900aedb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:59430/0 (socket says 172.21.15.31:59430) 2023-10-06T02:24:54.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:54.982+0000 7fe68effd700 1 -- 172.21.15.31:0/2798438582 learned_addr learned my addr 172.21.15.31:0/2798438582 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:24:54.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:54.982+0000 7fe68f7fe700 1 --2- 172.21.15.31:0/2798438582 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe690159550 0x7fe6900ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:54.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:54.982+0000 7fe68ffff700 1 -- 172.21.15.31:0/2798438582 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe690159550 msgr2=0x7fe6900ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:54.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:54.982+0000 7fe68ffff700 1 --2- 172.21.15.31:0/2798438582 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe690159550 0x7fe6900ceac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:54.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:54.982+0000 7fe68ffff700 1 -- 172.21.15.31:0/2798438582 >> v1:172.21.15.31:6789/0 conn(0x7fe69015dfd0 legacy=0x7fe6900aedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:54.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:54.982+0000 7fe68ffff700 1 -- 172.21.15.31:0/2798438582 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6900ac5b0 con 0x7fe690159920 2023-10-06T02:24:54.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:54.982+0000 7fe68f7fe700 1 --2- 172.21.15.31:0/2798438582 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe690159550 0x7fe6900ceac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:24:54.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:54.982+0000 7fe68ffff700 1 --2- 172.21.15.31:0/2798438582 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe690159920 0x7fe6900ad090 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fe688009e60 tx=0x7fe688009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a91608be26da7b8d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:54.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:54.982+0000 7fe68e7fc700 1 -- 172.21.15.31:0/2798438582 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe68800e070 con 0x7fe690159920 2023-10-06T02:24:54.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:54.982+0000 7fe68e7fc700 1 -- 172.21.15.31:0/2798438582 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe68800bc30 con 0x7fe690159920 2023-10-06T02:24:54.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:54.983+0000 7fe68e7fc700 1 -- 172.21.15.31:0/2798438582 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe688012430 con 0x7fe690159920 2023-10-06T02:24:54.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:54.983+0000 7fe694f9d700 1 -- 172.21.15.31:0/2798438582 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe690159920 msgr2=0x7fe6900ad090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:54.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:54.983+0000 7fe694f9d700 1 --2- 172.21.15.31:0/2798438582 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe690159920 0x7fe6900ad090 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fe688009e60 tx=0x7fe688009780 comp rx=0 tx=0).stop 2023-10-06T02:24:54.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:54.983+0000 7fe694f9d700 1 -- 172.21.15.31:0/2798438582 shutdown_connections 2023-10-06T02:24:54.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:54.983+0000 7fe694f9d700 1 --2- 172.21.15.31:0/2798438582 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe690159920 0x7fe6900ad090 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:54.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:54.983+0000 7fe694f9d700 1 --2- 172.21.15.31:0/2798438582 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe690159550 0x7fe6900ceac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:54.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:54.983+0000 7fe694f9d700 1 -- 172.21.15.31:0/2798438582 >> 172.21.15.31:0/2798438582 conn(0x7fe69006ce80 msgr2=0x7fe6900ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:54.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:54.983+0000 7fe694f9d700 1 -- 172.21.15.31:0/2798438582 shutdown_connections 2023-10-06T02:24:54.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:54.983+0000 7fe694f9d700 1 -- 172.21.15.31:0/2798438582 wait complete. 2023-10-06T02:24:54.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:54.985+0000 7fe694f9d700 1 Processor -- start 2023-10-06T02:24:54.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:54.985+0000 7fe694f9d700 1 -- start start 2023-10-06T02:24:54.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:54.986+0000 7fe694f9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe690159550 0x7fe690158170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:54.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:54.986+0000 7fe694f9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe690159920 0x7fe6901566b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:54.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:54.986+0000 7fe694f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe69015dfd0 0x7fe6900b2690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:54.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:54.986+0000 7fe694f9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe690158e10 con 0x7fe690159920 2023-10-06T02:24:54.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:54.986+0000 7fe694f9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe690158850 con 0x7fe690159550 2023-10-06T02:24:54.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:54.986+0000 7fe694f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe690158b30 con 0x7fe69015dfd0 2023-10-06T02:24:54.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:54.986+0000 7fe68effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe690159920 0x7fe6901566b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:54.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:54.986+0000 7fe68effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe690159920 0x7fe6901566b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53446/0 (socket says 172.21.15.31:53446) 2023-10-06T02:24:54.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:54.986+0000 7fe68effd700 1 -- 172.21.15.31:0/275534922 learned_addr learned my addr 172.21.15.31:0/275534922 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:54.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:54.986+0000 7fe68ffff700 1 --2- 172.21.15.31:0/275534922 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe69015dfd0 0x7fe6900b2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:54.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:54.986+0000 7fe68f7fe700 1 --2- 172.21.15.31:0/275534922 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe690159550 0x7fe690158170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:54.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:54.986+0000 7fe68effd700 1 -- 172.21.15.31:0/275534922 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe69015dfd0 msgr2=0x7fe6900b2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:54.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:54.986+0000 7fe68effd700 1 --2- 172.21.15.31:0/275534922 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe69015dfd0 0x7fe6900b2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:54.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:54.986+0000 7fe68effd700 1 -- 172.21.15.31:0/275534922 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe690159550 msgr2=0x7fe690158170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:54.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:54.986+0000 7fe68effd700 1 --2- 172.21.15.31:0/275534922 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe690159550 0x7fe690158170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:54.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:54.986+0000 7fe68effd700 1 -- 172.21.15.31:0/275534922 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe688009a20 con 0x7fe690159920 2023-10-06T02:24:54.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:54.986+0000 7fe68ffff700 1 --2- 172.21.15.31:0/275534922 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe69015dfd0 0x7fe6900b2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:24:54.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:54.986+0000 7fe68f7fe700 1 --2- 172.21.15.31:0/275534922 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe690159550 0x7fe690158170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:24:54.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:54.986+0000 7fe68effd700 1 --2- 172.21.15.31:0/275534922 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe690159920 0x7fe6901566b0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fe68400c560 tx=0x7fe684001020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:54.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:54.987+0000 7fe6777fe700 1 -- 172.21.15.31:0/275534922 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe684005b30 con 0x7fe690159920 2023-10-06T02:24:54.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:54.987+0000 7fe6777fe700 1 -- 172.21.15.31:0/275534922 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe684005cb0 con 0x7fe690159920 2023-10-06T02:24:54.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:54.987+0000 7fe694f9d700 1 -- 172.21.15.31:0/275534922 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe69014ef80 con 0x7fe690159920 2023-10-06T02:24:54.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:54.987+0000 7fe6777fe700 1 -- 172.21.15.31:0/275534922 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe684007320 con 0x7fe690159920 2023-10-06T02:24:54.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:54.987+0000 7fe694f9d700 1 -- 172.21.15.31:0/275534922 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe690156c70 con 0x7fe690159920 2023-10-06T02:24:54.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:54.988+0000 7fe694f9d700 1 -- 172.21.15.31:0/275534922 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe69004c3c0 con 0x7fe690159920 2023-10-06T02:24:54.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:54.989+0000 7fe6777fe700 1 -- 172.21.15.31:0/275534922 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7fe68401b3b0 con 0x7fe690159920 2023-10-06T02:24:54.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:54.990+0000 7fe6777fe700 1 --2- 172.21.15.31:0/275534922 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe67c0670e0 0x7fe67c069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:54.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:54.990+0000 7fe6777fe700 1 -- 172.21.15.31:0/275534922 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe684083f80 con 0x7fe690159920 2023-10-06T02:24:54.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:54.990+0000 7fe68f7fe700 1 --2- 172.21.15.31:0/275534922 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe67c0670e0 0x7fe67c069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:54.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:54.990+0000 7fe68f7fe700 1 --2- 172.21.15.31:0/275534922 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe67c0670e0 0x7fe67c069590 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe680008b20 tx=0x7fe680008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:54.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:54.994+0000 7fe6777fe700 1 -- 172.21.15.31:0/275534922 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe684050d70 con 0x7fe690159920 2023-10-06T02:24:55.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.223+0000 7fe694f9d700 1 -- 172.21.15.31:0/275534922 --> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fe69006e7c0 con 0x7fe67c0670e0 2023-10-06T02:24:55.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.224+0000 7fe6777fe700 1 -- 172.21.15.31:0/275534922 <== mgr.4107 v2:172.21.15.31:6832/85292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fe69006e7c0 con 0x7fe67c0670e0 2023-10-06T02:24:55.225 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2023-10-06T02:24:55.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:24:55.226 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephfs" 2023-10-06T02:24:55.226 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:24:55.226 INFO:teuthology.orchestra.run.smithi031.stdout:] 2023-10-06T02:24:55.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.226+0000 7fe694f9d700 1 -- 172.21.15.31:0/275534922 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe67c0670e0 msgr2=0x7fe67c069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:55.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.226+0000 7fe694f9d700 1 --2- 172.21.15.31:0/275534922 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe67c0670e0 0x7fe67c069590 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe680008b20 tx=0x7fe680008040 comp rx=0 tx=0).stop 2023-10-06T02:24:55.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.226+0000 7fe694f9d700 1 -- 172.21.15.31:0/275534922 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe690159920 msgr2=0x7fe6901566b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:55.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.226+0000 7fe694f9d700 1 --2- 172.21.15.31:0/275534922 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe690159920 0x7fe6901566b0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fe68400c560 tx=0x7fe684001020 comp rx=0 tx=0).stop 2023-10-06T02:24:55.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.226+0000 7fe694f9d700 1 -- 172.21.15.31:0/275534922 shutdown_connections 2023-10-06T02:24:55.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.226+0000 7fe694f9d700 1 --2- 172.21.15.31:0/275534922 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe67c0670e0 0x7fe67c069590 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:55.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.226+0000 7fe694f9d700 1 --2- 172.21.15.31:0/275534922 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe69015dfd0 0x7fe6900b2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:55.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.226+0000 7fe694f9d700 1 --2- 172.21.15.31:0/275534922 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe690159550 0x7fe690158170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:55.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.226+0000 7fe694f9d700 1 --2- 172.21.15.31:0/275534922 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe690159920 0x7fe6901566b0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:55.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.226+0000 7fe694f9d700 1 -- 172.21.15.31:0/275534922 >> 172.21.15.31:0/275534922 conn(0x7fe69006ce80 msgr2=0x7fe6900b0520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:55.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.226+0000 7fe694f9d700 1 -- 172.21.15.31:0/275534922 shutdown_connections 2023-10-06T02:24:55.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.227+0000 7fe694f9d700 1 -- 172.21.15.31:0/275534922 wait complete. 2023-10-06T02:24:55.245 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T02:24:55.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.433+0000 7fd1e6d9d700 1 Processor -- start 2023-10-06T02:24:55.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.433+0000 7fd1e6d9d700 1 -- start start 2023-10-06T02:24:55.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.435+0000 7fd1e6d9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd1e015dfd0 0x7fd1e00aedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:55.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.435+0000 7fd1e6d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd1e0159920 0x7fd1e00ad090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:55.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.435+0000 7fd1e6d9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1e00acc00 con 0x7fd1e0159550 2023-10-06T02:24:55.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.435+0000 7fd1e6d9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1e00abff0 con 0x7fd1e015dfd0 2023-10-06T02:24:55.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.435+0000 7fd1e6d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e00ac2d0 con 0x7fd1e0159920 2023-10-06T02:24:55.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.435+0000 7fd1e559a700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd1e015dfd0 0x7fd1e00aedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:55.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.435+0000 7fd1e559a700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd1e015dfd0 0x7fd1e00aedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:33546/0 (socket says 172.21.15.31:33546) 2023-10-06T02:24:55.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.435+0000 7fd1e659c700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd1e0159920 0x7fd1e00ad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:55.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.435+0000 7fd1e559a700 1 -- 172.21.15.31:0/1851724492 learned_addr learned my addr 172.21.15.31:0/1851724492 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:55.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.435+0000 7fd1e5d9b700 1 --1- 172.21.15.31:0/1851724492 >> v1:172.21.15.62:6789/0 conn(0x7fd1e0159550 0x7fd1e00ceac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:52400/0 (socket says 172.21.15.31:52400) 2023-10-06T02:24:55.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.436+0000 7fd1e559a700 1 -- 172.21.15.31:0/1851724492 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd1e0159920 msgr2=0x7fd1e00ad090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:55.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.436+0000 7fd1e559a700 1 --2- 172.21.15.31:0/1851724492 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd1e0159920 0x7fd1e00ad090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:55.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.436+0000 7fd1e559a700 1 -- 172.21.15.31:0/1851724492 >> v1:172.21.15.62:6789/0 conn(0x7fd1e0159550 legacy=0x7fd1e00ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:55.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.436+0000 7fd1e559a700 1 -- 172.21.15.31:0/1851724492 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1e00ac5b0 con 0x7fd1e015dfd0 2023-10-06T02:24:55.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.436+0000 7fd1e659c700 1 --2- 172.21.15.31:0/1851724492 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd1e0159920 0x7fd1e00ad090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:24:55.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.436+0000 7fd1e559a700 1 --2- 172.21.15.31:0/1851724492 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1e015dfd0 0x7fd1e00aedb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd1d00099b0 tx=0x7fd1d0009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6d1549c76d25a0c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:55.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.436+0000 7fd1e4d99700 1 -- 172.21.15.31:0/1851724492 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d0009ea0 con 0x7fd1e015dfd0 2023-10-06T02:24:55.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.436+0000 7fd1e4d99700 1 -- 172.21.15.31:0/1851724492 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1d000b860 con 0x7fd1e015dfd0 2023-10-06T02:24:55.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.436+0000 7fd1e4d99700 1 -- 172.21.15.31:0/1851724492 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d000bc00 con 0x7fd1e015dfd0 2023-10-06T02:24:55.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.437+0000 7fd1e6d9d700 1 -- 172.21.15.31:0/1851724492 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1e015dfd0 msgr2=0x7fd1e00aedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:55.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.437+0000 7fd1e6d9d700 1 --2- 172.21.15.31:0/1851724492 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1e015dfd0 0x7fd1e00aedb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd1d00099b0 tx=0x7fd1d0009250 comp rx=0 tx=0).stop 2023-10-06T02:24:55.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.437+0000 7fd1e6d9d700 1 -- 172.21.15.31:0/1851724492 shutdown_connections 2023-10-06T02:24:55.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.437+0000 7fd1e6d9d700 1 --2- 172.21.15.31:0/1851724492 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd1e0159920 0x7fd1e00ad090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:55.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.437+0000 7fd1e6d9d700 1 --2- 172.21.15.31:0/1851724492 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1e015dfd0 0x7fd1e00aedb0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:55.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.437+0000 7fd1e6d9d700 1 -- 172.21.15.31:0/1851724492 >> 172.21.15.31:0/1851724492 conn(0x7fd1e006ce80 msgr2=0x7fd1e00ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:55.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.437+0000 7fd1e6d9d700 1 -- 172.21.15.31:0/1851724492 shutdown_connections 2023-10-06T02:24:55.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.437+0000 7fd1e6d9d700 1 -- 172.21.15.31:0/1851724492 wait complete. 2023-10-06T02:24:55.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.438+0000 7fd1e6d9d700 1 Processor -- start 2023-10-06T02:24:55.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.439+0000 7fd1e6d9d700 1 -- start start 2023-10-06T02:24:55.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.439+0000 7fd1e6d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1e0159550 0x7fd1e0152000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:55.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.440+0000 7fd1e6d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd1e0159920 0x7fd1e01570f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:55.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.440+0000 7fd1e6d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1e015dfd0 0x7fd1e0157630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:55.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.440+0000 7fd1e6d9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e00c85b0 con 0x7fd1e015dfd0 2023-10-06T02:24:55.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.440+0000 7fd1e6d9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e00c8890 con 0x7fd1e0159550 2023-10-06T02:24:55.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.440+0000 7fd1e6d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e00c8b70 con 0x7fd1e0159920 2023-10-06T02:24:55.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.440+0000 7fd1e659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1e015dfd0 0x7fd1e0157630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:55.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.440+0000 7fd1e659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1e015dfd0 0x7fd1e0157630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53448/0 (socket says 172.21.15.31:53448) 2023-10-06T02:24:55.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.440+0000 7fd1e659c700 1 -- 172.21.15.31:0/3947163013 learned_addr learned my addr 172.21.15.31:0/3947163013 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:55.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.440+0000 7fd1e559a700 1 --2- 172.21.15.31:0/3947163013 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd1e0159920 0x7fd1e01570f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:55.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.440+0000 7fd1e5d9b700 1 --2- 172.21.15.31:0/3947163013 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1e0159550 0x7fd1e0152000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:55.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.440+0000 7fd1e659c700 1 -- 172.21.15.31:0/3947163013 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd1e0159920 msgr2=0x7fd1e01570f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:55.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.440+0000 7fd1e659c700 1 --2- 172.21.15.31:0/3947163013 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd1e0159920 0x7fd1e01570f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:55.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.440+0000 7fd1e659c700 1 -- 172.21.15.31:0/3947163013 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1e0159550 msgr2=0x7fd1e0152000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:55.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.440+0000 7fd1e659c700 1 --2- 172.21.15.31:0/3947163013 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1e0159550 0x7fd1e0152000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:55.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.440+0000 7fd1e659c700 1 -- 172.21.15.31:0/3947163013 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1d8009a20 con 0x7fd1e015dfd0 2023-10-06T02:24:55.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.441+0000 7fd1e659c700 1 --2- 172.21.15.31:0/3947163013 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1e015dfd0 0x7fd1e0157630 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fd1d8009fc0 tx=0x7fd1d800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:55.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.441+0000 7fd1d5ffb700 1 -- 172.21.15.31:0/3947163013 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d800cbd0 con 0x7fd1e015dfd0 2023-10-06T02:24:55.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.441+0000 7fd1d5ffb700 1 -- 172.21.15.31:0/3947163013 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1d800f040 con 0x7fd1e015dfd0 2023-10-06T02:24:55.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.441+0000 7fd1d5ffb700 1 -- 172.21.15.31:0/3947163013 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d8014480 con 0x7fd1e015dfd0 2023-10-06T02:24:55.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.441+0000 7fd1e6d9d700 1 -- 172.21.15.31:0/3947163013 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1d00094f0 con 0x7fd1e015dfd0 2023-10-06T02:24:55.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.441+0000 7fd1e6d9d700 1 -- 172.21.15.31:0/3947163013 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1e0155400 con 0x7fd1e015dfd0 2023-10-06T02:24:55.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.442+0000 7fd1e6d9d700 1 -- 172.21.15.31:0/3947163013 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1e00c9370 con 0x7fd1e015dfd0 2023-10-06T02:24:55.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.443+0000 7fd1d5ffb700 1 -- 172.21.15.31:0/3947163013 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7fd1d8003020 con 0x7fd1e015dfd0 2023-10-06T02:24:55.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.444+0000 7fd1d5ffb700 1 --2- 172.21.15.31:0/3947163013 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fd1c40670e0 0x7fd1c4069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:55.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.444+0000 7fd1d5ffb700 1 -- 172.21.15.31:0/3947163013 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd1d8085470 con 0x7fd1e015dfd0 2023-10-06T02:24:55.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.445+0000 7fd1e5d9b700 1 --2- 172.21.15.31:0/3947163013 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fd1c40670e0 0x7fd1c4069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:55.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.445+0000 7fd1e5d9b700 1 --2- 172.21.15.31:0/3947163013 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fd1c40670e0 0x7fd1c4069590 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc004fd0 tx=0x7fd1dc007eb0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:55.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.449+0000 7fd1d5ffb700 1 -- 172.21.15.31:0/3947163013 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd1e00c9370 con 0x7fd1e015dfd0 2023-10-06T02:24:55.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.651+0000 7fd1e6d9d700 1 -- 172.21.15.31:0/3947163013 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fd1e00c9370 con 0x7fd1e015dfd0 2023-10-06T02:24:55.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.651+0000 7fd1d5ffb700 1 -- 172.21.15.31:0/3947163013 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd1e00c9370 con 0x7fd1e015dfd0 2023-10-06T02:24:55.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.653+0000 7fd1e6d9d700 1 -- 172.21.15.31:0/3947163013 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fd1c40670e0 msgr2=0x7fd1c4069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:55.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.654+0000 7fd1e6d9d700 1 --2- 172.21.15.31:0/3947163013 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fd1c40670e0 0x7fd1c4069590 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc004fd0 tx=0x7fd1dc007eb0 comp rx=0 tx=0).stop 2023-10-06T02:24:55.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.654+0000 7fd1e6d9d700 1 -- 172.21.15.31:0/3947163013 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1e015dfd0 msgr2=0x7fd1e0157630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:55.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.654+0000 7fd1e6d9d700 1 --2- 172.21.15.31:0/3947163013 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1e015dfd0 0x7fd1e0157630 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fd1d8009fc0 tx=0x7fd1d800b040 comp rx=0 tx=0).stop 2023-10-06T02:24:55.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.654+0000 7fd1e6d9d700 1 -- 172.21.15.31:0/3947163013 shutdown_connections 2023-10-06T02:24:55.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.654+0000 7fd1e6d9d700 1 --2- 172.21.15.31:0/3947163013 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1e015dfd0 0x7fd1e0157630 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:55.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.654+0000 7fd1e6d9d700 1 --2- 172.21.15.31:0/3947163013 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1e0159550 0x7fd1e0152000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:55.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.654+0000 7fd1e6d9d700 1 --2- 172.21.15.31:0/3947163013 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fd1c40670e0 0x7fd1c4069590 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:55.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.654+0000 7fd1e6d9d700 1 --2- 172.21.15.31:0/3947163013 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd1e0159920 0x7fd1e01570f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:55.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.654+0000 7fd1e6d9d700 1 -- 172.21.15.31:0/3947163013 >> 172.21.15.31:0/3947163013 conn(0x7fd1e006ce80 msgr2=0x7fd1e0152f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:55.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.654+0000 7fd1e6d9d700 1 -- 172.21.15.31:0/3947163013 shutdown_connections 2023-10-06T02:24:55.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:55.654+0000 7fd1e6d9d700 1 -- 172.21.15.31:0/3947163013 wait complete. 2023-10-06T02:24:55.672 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T02:24:55.673 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:24:55.673 DEBUG:teuthology.orchestra.run.smithi031:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T02:24:55.752 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi031.front.sepia.ceph.com 2023-10-06T02:24:55.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:24:55.753 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T02:24:55.790 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-06T02:24:55.791 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:24:55.791 DEBUG:teuthology.orchestra.run.smithi062:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-06T02:24:55.878 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi062.front.sepia.ceph.com 2023-10-06T02:24:55.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:24:55.878 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T02:24:55.917 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:24:56.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.103+0000 7efe1919d700 1 Processor -- start 2023-10-06T02:24:56.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.103+0000 7efe1919d700 1 -- start start 2023-10-06T02:24:56.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.104+0000 7efe1919d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7efe1413fef0 0x7efe1415a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:56.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.104+0000 7efe1919d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe1415ae70 con 0x7efe1413fb20 2023-10-06T02:24:56.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.104+0000 7efe1919d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe1406af60 con 0x7efe1406aaa0 2023-10-06T02:24:56.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.104+0000 7efe1919d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe1406b240 con 0x7efe1413fef0 2023-10-06T02:24:56.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.105+0000 7efe13fff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7efe1413fb20 0x7efe140c7990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:59440/0 (socket says 172.21.15.31:59440) 2023-10-06T02:24:56.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.105+0000 7efe13fff700 1 -- 172.21.15.31:0/3443515126 learned_addr learned my addr 172.21.15.31:0/3443515126 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:24:56.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.105+0000 7efe137fe700 1 --2- 172.21.15.31:0/3443515126 >> v2:172.21.15.62:3300/0 conn(0x7efe1413fef0 0x7efe1415a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:56.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.105+0000 7efe12ffd700 1 -- 172.21.15.31:0/3443515126 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1936265456 0 0) 0x7efe04001110 con 0x7efe1413fb20 2023-10-06T02:24:56.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.105+0000 7efe12ffd700 1 -- 172.21.15.31:0/3443515126 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efdfc003570 con 0x7efe1413fb20 2023-10-06T02:24:56.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.105+0000 7efe137fe700 1 -- 172.21.15.31:0/3443515126 >> v1:172.21.15.62:6789/0 conn(0x7efe1406aaa0 legacy=0x7efe140c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:56.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.105+0000 7efe12ffd700 1 -- 172.21.15.31:0/3443515126 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3311616243 0 0) 0x7efe1406af60 con 0x7efe1406aaa0 2023-10-06T02:24:56.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.105+0000 7efe137fe700 1 -- 172.21.15.31:0/3443515126 >> v1:172.21.15.31:6789/0 conn(0x7efe1413fb20 legacy=0x7efe140c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:56.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.105+0000 7efe137fe700 1 -- 172.21.15.31:0/3443515126 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe1406b520 con 0x7efe1413fef0 2023-10-06T02:24:56.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.106+0000 7efe137fe700 1 --2- 172.21.15.31:0/3443515126 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe1413fef0 0x7efe1415a930 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7efe080098d0 tx=0x7efe08009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c99b8ab4df0793f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:56.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.106+0000 7efe12ffd700 1 -- 172.21.15.31:0/3443515126 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe08005c10 con 0x7efe1413fef0 2023-10-06T02:24:56.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.106+0000 7efe12ffd700 1 -- 172.21.15.31:0/3443515126 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe08005d70 con 0x7efe1413fef0 2023-10-06T02:24:56.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.106+0000 7efe12ffd700 1 -- 172.21.15.31:0/3443515126 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe08006280 con 0x7efe1413fef0 2023-10-06T02:24:56.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.107+0000 7efe1919d700 1 -- 172.21.15.31:0/3443515126 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe1413fef0 msgr2=0x7efe1415a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:56.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.107+0000 7efe1919d700 1 --2- 172.21.15.31:0/3443515126 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe1413fef0 0x7efe1415a930 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7efe080098d0 tx=0x7efe08009120 comp rx=0 tx=0).stop 2023-10-06T02:24:56.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.107+0000 7efe1919d700 1 -- 172.21.15.31:0/3443515126 shutdown_connections 2023-10-06T02:24:56.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.107+0000 7efe1919d700 1 --2- 172.21.15.31:0/3443515126 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe1413fef0 0x7efe1415a930 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:56.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.107+0000 7efe1919d700 1 -- 172.21.15.31:0/3443515126 >> 172.21.15.31:0/3443515126 conn(0x7efe1406d170 msgr2=0x7efe140cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:56.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.107+0000 7efe1919d700 1 -- 172.21.15.31:0/3443515126 shutdown_connections 2023-10-06T02:24:56.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.107+0000 7efe1919d700 1 -- 172.21.15.31:0/3443515126 wait complete. 2023-10-06T02:24:56.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.108+0000 7efe1919d700 1 Processor -- start 2023-10-06T02:24:56.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.108+0000 7efe1919d700 1 -- start start 2023-10-06T02:24:56.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.109+0000 7efe1919d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efe1406aaa0 0x7efe140b14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:56.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.109+0000 7efe1919d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe1413fb20 0x7efe14156a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:56.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.109+0000 7efe1919d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe1413fef0 0x7efe14156f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:56.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.109+0000 7efe1919d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe140b09d0 con 0x7efe1413fb20 2023-10-06T02:24:56.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.109+0000 7efe1919d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe14167550 con 0x7efe1413fef0 2023-10-06T02:24:56.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.109+0000 7efe1919d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe14157e80 con 0x7efe1406aaa0 2023-10-06T02:24:56.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.110+0000 7efe137fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe1413fb20 0x7efe14156a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:56.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.110+0000 7efe13fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efe1406aaa0 0x7efe140b14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:56.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.110+0000 7efe137fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe1413fb20 0x7efe14156a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53454/0 (socket says 172.21.15.31:53454) 2023-10-06T02:24:56.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.110+0000 7efe137fe700 1 -- 172.21.15.31:0/2667774177 learned_addr learned my addr 172.21.15.31:0/2667774177 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:56.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.110+0000 7efe1899c700 1 --2- 172.21.15.31:0/2667774177 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe1413fef0 0x7efe14156f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:56.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.110+0000 7efe137fe700 1 -- 172.21.15.31:0/2667774177 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efe1406aaa0 msgr2=0x7efe140b14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:56.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.110+0000 7efe137fe700 1 --2- 172.21.15.31:0/2667774177 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efe1406aaa0 0x7efe140b14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:56.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.110+0000 7efe137fe700 1 -- 172.21.15.31:0/2667774177 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe1413fef0 msgr2=0x7efe14156f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:56.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.110+0000 7efe137fe700 1 --2- 172.21.15.31:0/2667774177 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe1413fef0 0x7efe14156f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:56.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.110+0000 7efe137fe700 1 -- 172.21.15.31:0/2667774177 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe080093c0 con 0x7efe1413fb20 2023-10-06T02:24:56.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.110+0000 7efe137fe700 1 --2- 172.21.15.31:0/2667774177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe1413fb20 0x7efe14156a00 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7efe08012000 tx=0x7efe08016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:56.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.111+0000 7efdfbfff700 1 -- 172.21.15.31:0/2667774177 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe0801d800 con 0x7efe1413fb20 2023-10-06T02:24:56.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.111+0000 7efdfbfff700 1 -- 172.21.15.31:0/2667774177 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe0801d960 con 0x7efe1413fb20 2023-10-06T02:24:56.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.111+0000 7efdfbfff700 1 -- 172.21.15.31:0/2667774177 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe0801dd20 con 0x7efe1413fb20 2023-10-06T02:24:56.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.111+0000 7efe1919d700 1 -- 172.21.15.31:0/2667774177 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe141575e0 con 0x7efe1413fb20 2023-10-06T02:24:56.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.111+0000 7efe1919d700 1 -- 172.21.15.31:0/2667774177 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe14157ba0 con 0x7efe1413fb20 2023-10-06T02:24:56.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.112+0000 7efe1919d700 1 -- 172.21.15.31:0/2667774177 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe14167700 con 0x7efe1413fb20 2023-10-06T02:24:56.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.113+0000 7efdfbfff700 1 -- 172.21.15.31:0/2667774177 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7efe0801b690 con 0x7efe1413fb20 2023-10-06T02:24:56.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.113+0000 7efdfbfff700 1 --2- 172.21.15.31:0/2667774177 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7efe000670e0 0x7efe00069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:56.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.114+0000 7efdfbfff700 1 -- 172.21.15.31:0/2667774177 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6058+0+0 (secure 0 0 0) 0x7efe080869e0 con 0x7efe1413fb20 2023-10-06T02:24:56.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.114+0000 7efe13fff700 1 --2- 172.21.15.31:0/2667774177 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7efe000670e0 0x7efe00069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:56.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.114+0000 7efe13fff700 1 --2- 172.21.15.31:0/2667774177 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7efe000670e0 0x7efe00069590 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7efe04001270 tx=0x7efe04000bc0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:56.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.118+0000 7efdfbfff700 1 -- 172.21.15.31:0/2667774177 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efe14167700 con 0x7efe1413fb20 2023-10-06T02:24:56.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.364+0000 7efe1919d700 1 -- 172.21.15.31:0/2667774177 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efe141676c0 con 0x7efe1413fb20 2023-10-06T02:24:56.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.364+0000 7efdfbfff700 1 -- 172.21.15.31:0/2667774177 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4594 (secure 0 0 0) 0x7efe080537d0 con 0x7efe1413fb20 2023-10-06T02:24:56.366 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:24:56.366 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4949,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/178547366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":178547366},{"type":"v1","addr":"172.21.15.62:6835","nonce":178547366}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":5278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/2578831517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":2578831517},{"type":"v1","addr":"172.21.15.31:6836","nonce":2578831517}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5287,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3502585737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":3502585737},{"type":"v1","addr":"172.21.15.31:6837","nonce":3502585737}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:24:43.872314+0000","modified":"2023-10-06T02:24:47.404607+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5290":{"gid":5290,"name":"b","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6834/2358776861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2358776861},{"type":"v1","addr":"172.21.15.62:6834","nonce":2358776861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T02:24:56.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.367+0000 7efe1919d700 1 -- 172.21.15.31:0/2667774177 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7efe000670e0 msgr2=0x7efe00069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:56.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.367+0000 7efe1919d700 1 --2- 172.21.15.31:0/2667774177 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7efe000670e0 0x7efe00069590 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7efe04001270 tx=0x7efe04000bc0 comp rx=0 tx=0).stop 2023-10-06T02:24:56.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.367+0000 7efe1919d700 1 -- 172.21.15.31:0/2667774177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe1413fb20 msgr2=0x7efe14156a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:56.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.367+0000 7efe1919d700 1 --2- 172.21.15.31:0/2667774177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe1413fb20 0x7efe14156a00 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7efe08012000 tx=0x7efe08016000 comp rx=0 tx=0).stop 2023-10-06T02:24:56.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.367+0000 7efe1919d700 1 -- 172.21.15.31:0/2667774177 shutdown_connections 2023-10-06T02:24:56.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.367+0000 7efe1919d700 1 --2- 172.21.15.31:0/2667774177 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe1413fef0 0x7efe14156f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:56.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.367+0000 7efe1919d700 1 --2- 172.21.15.31:0/2667774177 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7efe000670e0 0x7efe00069590 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:56.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.367+0000 7efe1919d700 1 --2- 172.21.15.31:0/2667774177 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efe1406aaa0 0x7efe140b14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:56.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.367+0000 7efe1919d700 1 --2- 172.21.15.31:0/2667774177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe1413fb20 0x7efe14156a00 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:56.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.367+0000 7efe1919d700 1 -- 172.21.15.31:0/2667774177 >> 172.21.15.31:0/2667774177 conn(0x7efe1406d170 msgr2=0x7efe140ad970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:56.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.367+0000 7efe1919d700 1 -- 172.21.15.31:0/2667774177 shutdown_connections 2023-10-06T02:24:56.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.367+0000 7efe1919d700 1 -- 172.21.15.31:0/2667774177 wait complete. 2023-10-06T02:24:56.370 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 27 2023-10-06T02:24:56.386 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json session ls 2023-10-06T02:24:56.605 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-10-06T02:24:56.606 DEBUG:tasks.cephfs.filesystem:_json_asok output [] 2023-10-06T02:24:56.606 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000547919 2023-10-06T02:24:56.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.795+0000 7fc022b9d700 1 Processor -- start 2023-10-06T02:24:56.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.795+0000 7fc022b9d700 1 -- start start 2023-10-06T02:24:56.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.796+0000 7fc022b9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc01c06db20 0x7fc01c15a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:56.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.796+0000 7fc022b9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc01c0caec0 0x7fc01c15afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:56.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.796+0000 7fc022b9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc01c15ada0 con 0x7fc01c153c90 2023-10-06T02:24:56.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.796+0000 7fc022b9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc01c06ca80 con 0x7fc01c0caec0 2023-10-06T02:24:56.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.796+0000 7fc022b9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc01c06cd60 con 0x7fc01c06db20 2023-10-06T02:24:56.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.797+0000 7fc02139a700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc01c06db20 0x7fc01c15a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:56.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.797+0000 7fc02239c700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc01c0caec0 0x7fc01c15afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:56.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.797+0000 7fc02139a700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc01c06db20 0x7fc01c15a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:54460/0 (socket says 172.21.15.31:54460) 2023-10-06T02:24:56.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.797+0000 7fc02139a700 1 -- 172.21.15.31:0/2030839723 learned_addr learned my addr 172.21.15.31:0/2030839723 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:56.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.797+0000 7fc02239c700 1 -- 172.21.15.31:0/2030839723 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc01c06db20 msgr2=0x7fc01c15a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:56.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.797+0000 7fc02239c700 1 --2- 172.21.15.31:0/2030839723 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc01c06db20 0x7fc01c15a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:56.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.797+0000 7fc02239c700 1 -- 172.21.15.31:0/2030839723 >> v1:172.21.15.62:6789/0 conn(0x7fc01c153c90 legacy=0x7fc01c0ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:56.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.797+0000 7fc02239c700 1 -- 172.21.15.31:0/2030839723 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc01c06d040 con 0x7fc01c0caec0 2023-10-06T02:24:56.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.797+0000 7fc020b99700 1 -- 172.21.15.31:0/2030839723 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 599958324 0 0) 0x7fc018001110 con 0x7fc01c153c90 2023-10-06T02:24:56.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.797+0000 7fc02139a700 1 --2- 172.21.15.31:0/2030839723 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc01c06db20 0x7fc01c15a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:24:56.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.798+0000 7fc02239c700 1 --2- 172.21.15.31:0/2030839723 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc01c0caec0 0x7fc01c15afb0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc014009e60 tx=0x7fc014009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48fe1c0d21d0143a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:56.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.798+0000 7fc020b99700 1 -- 172.21.15.31:0/2030839723 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc01400e070 con 0x7fc01c0caec0 2023-10-06T02:24:56.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.798+0000 7fc020b99700 1 -- 172.21.15.31:0/2030839723 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc01400bb60 con 0x7fc01c0caec0 2023-10-06T02:24:56.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.798+0000 7fc020b99700 1 -- 172.21.15.31:0/2030839723 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc014012430 con 0x7fc01c0caec0 2023-10-06T02:24:56.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.798+0000 7fc022b9d700 1 -- 172.21.15.31:0/2030839723 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc01c0caec0 msgr2=0x7fc01c15afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:56.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.798+0000 7fc022b9d700 1 --2- 172.21.15.31:0/2030839723 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc01c0caec0 0x7fc01c15afb0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc014009e60 tx=0x7fc014009780 comp rx=0 tx=0).stop 2023-10-06T02:24:56.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.798+0000 7fc022b9d700 1 -- 172.21.15.31:0/2030839723 shutdown_connections 2023-10-06T02:24:56.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.798+0000 7fc022b9d700 1 --2- 172.21.15.31:0/2030839723 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc01c0caec0 0x7fc01c15afb0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:56.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.798+0000 7fc022b9d700 1 --2- 172.21.15.31:0/2030839723 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc01c06db20 0x7fc01c15a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:56.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.798+0000 7fc022b9d700 1 -- 172.21.15.31:0/2030839723 >> 172.21.15.31:0/2030839723 conn(0x7fc01c06e720 msgr2=0x7fc01c06eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:56.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.799+0000 7fc022b9d700 1 -- 172.21.15.31:0/2030839723 shutdown_connections 2023-10-06T02:24:56.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.799+0000 7fc022b9d700 1 -- 172.21.15.31:0/2030839723 wait complete. 2023-10-06T02:24:56.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.800+0000 7fc022b9d700 1 Processor -- start 2023-10-06T02:24:56.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.800+0000 7fc022b9d700 1 -- start start 2023-10-06T02:24:56.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.801+0000 7fc022b9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc01c06db20 0x7fc01c14d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:56.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.801+0000 7fc022b9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc01c0caec0 0x7fc01c14f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:56.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.801+0000 7fc022b9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc01c153c90 0x7fc01c152970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:56.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.801+0000 7fc022b9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc01c0b28c0 con 0x7fc01c0caec0 2023-10-06T02:24:56.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.801+0000 7fc022b9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc01c14fa50 con 0x7fc01c06db20 2023-10-06T02:24:56.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.801+0000 7fc022b9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc01c14d5e0 con 0x7fc01c153c90 2023-10-06T02:24:56.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.801+0000 7fc02139a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc01c0caec0 0x7fc01c14f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:56.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.801+0000 7fc02139a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc01c0caec0 0x7fc01c14f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53462/0 (socket says 172.21.15.31:53462) 2023-10-06T02:24:56.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.801+0000 7fc02139a700 1 -- 172.21.15.31:0/1434162191 learned_addr learned my addr 172.21.15.31:0/1434162191 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:56.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.801+0000 7fc02239c700 1 --2- 172.21.15.31:0/1434162191 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc01c153c90 0x7fc01c152970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:56.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.801+0000 7fc021b9b700 1 --2- 172.21.15.31:0/1434162191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc01c06db20 0x7fc01c14d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:56.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.802+0000 7fc02139a700 1 -- 172.21.15.31:0/1434162191 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc01c153c90 msgr2=0x7fc01c152970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:56.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.802+0000 7fc02139a700 1 --2- 172.21.15.31:0/1434162191 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc01c153c90 0x7fc01c152970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:56.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.802+0000 7fc02139a700 1 -- 172.21.15.31:0/1434162191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc01c06db20 msgr2=0x7fc01c14d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:56.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.802+0000 7fc02139a700 1 --2- 172.21.15.31:0/1434162191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc01c06db20 0x7fc01c14d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:56.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.802+0000 7fc02139a700 1 -- 172.21.15.31:0/1434162191 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc00c0094f0 con 0x7fc01c0caec0 2023-10-06T02:24:56.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.802+0000 7fc021b9b700 1 --2- 172.21.15.31:0/1434162191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc01c06db20 0x7fc01c14d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:24:56.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.802+0000 7fc02139a700 1 --2- 172.21.15.31:0/1434162191 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc01c0caec0 0x7fc01c14f1e0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fc00c00ecb0 tx=0x7fc00c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:56.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.802+0000 7fc011ffb700 1 -- 172.21.15.31:0/1434162191 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc00c00c850 con 0x7fc01c0caec0 2023-10-06T02:24:56.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.802+0000 7fc011ffb700 1 -- 172.21.15.31:0/1434162191 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc00c00c9b0 con 0x7fc01c0caec0 2023-10-06T02:24:56.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.802+0000 7fc022b9d700 1 -- 172.21.15.31:0/1434162191 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc014009a20 con 0x7fc01c0caec0 2023-10-06T02:24:56.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.802+0000 7fc011ffb700 1 -- 172.21.15.31:0/1434162191 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc00c00cd30 con 0x7fc01c0caec0 2023-10-06T02:24:56.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.802+0000 7fc022b9d700 1 -- 172.21.15.31:0/1434162191 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc01c16a070 con 0x7fc01c0caec0 2023-10-06T02:24:56.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.803+0000 7fc022b9d700 1 -- 172.21.15.31:0/1434162191 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc01c04c3c0 con 0x7fc01c0caec0 2023-10-06T02:24:56.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.804+0000 7fc011ffb700 1 -- 172.21.15.31:0/1434162191 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7fc00c013680 con 0x7fc01c0caec0 2023-10-06T02:24:56.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.805+0000 7fc011ffb700 1 --2- 172.21.15.31:0/1434162191 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc0000670e0 0x7fc000069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:56.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.805+0000 7fc011ffb700 1 -- 172.21.15.31:0/1434162191 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc00c084d00 con 0x7fc01c0caec0 2023-10-06T02:24:56.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.805+0000 7fc021b9b700 1 --2- 172.21.15.31:0/1434162191 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc0000670e0 0x7fc000069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:56.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.806+0000 7fc021b9b700 1 --2- 172.21.15.31:0/1434162191 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc0000670e0 0x7fc000069590 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc018001370 tx=0x7fc018009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:56.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:56.810+0000 7fc011ffb700 1 -- 172.21.15.31:0/1434162191 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc00c055580 con 0x7fc01c0caec0 2023-10-06T02:24:57.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.048+0000 7fc022b9d700 1 -- 172.21.15.31:0/1434162191 --> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000547919", "target": ["mon-mgr", ""]}) v1 -- 0x7fc01c15c990 con 0x7fc0000670e0 2023-10-06T02:24:57.055 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:24:57.054+0000 7f3e7b47e700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:24:57.055 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:24:57.054+0000 7f3e7b47e700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:24:57.055 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:24:57.054+0000 7f3e7b47e700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:24:57.055 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:24:57.054+0000 7f3e7b47e700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:24:57.055 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:24:57.054+0000 7f3e7b47e700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:24:57.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.093+0000 7fc011ffb700 1 -- 172.21.15.31:0/1434162191 <== mgr.4107 v2:172.21.15.31:6832/85292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fc01c15c990 con 0x7fc0000670e0 2023-10-06T02:24:57.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.096+0000 7fc022b9d700 1 -- 172.21.15.31:0/1434162191 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc0000670e0 msgr2=0x7fc000069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:57.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.096+0000 7fc022b9d700 1 --2- 172.21.15.31:0/1434162191 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc0000670e0 0x7fc000069590 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc018001370 tx=0x7fc018009040 comp rx=0 tx=0).stop 2023-10-06T02:24:57.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.096+0000 7fc022b9d700 1 -- 172.21.15.31:0/1434162191 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc01c0caec0 msgr2=0x7fc01c14f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:57.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.096+0000 7fc022b9d700 1 --2- 172.21.15.31:0/1434162191 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc01c0caec0 0x7fc01c14f1e0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fc00c00ecb0 tx=0x7fc00c009250 comp rx=0 tx=0).stop 2023-10-06T02:24:57.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.096+0000 7fc022b9d700 1 -- 172.21.15.31:0/1434162191 shutdown_connections 2023-10-06T02:24:57.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.096+0000 7fc022b9d700 1 --2- 172.21.15.31:0/1434162191 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc0000670e0 0x7fc000069590 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:57.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.096+0000 7fc022b9d700 1 --2- 172.21.15.31:0/1434162191 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc01c153c90 0x7fc01c152970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:57.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.096+0000 7fc022b9d700 1 --2- 172.21.15.31:0/1434162191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc01c06db20 0x7fc01c14d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:57.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.096+0000 7fc022b9d700 1 --2- 172.21.15.31:0/1434162191 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc01c0caec0 0x7fc01c14f1e0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:57.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.096+0000 7fc022b9d700 1 -- 172.21.15.31:0/1434162191 >> 172.21.15.31:0/1434162191 conn(0x7fc01c06e720 msgr2=0x7fc01c0c2a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:57.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.096+0000 7fc022b9d700 1 -- 172.21.15.31:0/1434162191 shutdown_connections 2023-10-06T02:24:57.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.096+0000 7fc022b9d700 1 -- 172.21.15.31:0/1434162191 wait complete. 2023-10-06T02:24:57.115 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:24:57.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.301+0000 7f65f939d700 1 Processor -- start 2023-10-06T02:24:57.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.301+0000 7f65f939d700 1 -- start start 2023-10-06T02:24:57.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.302+0000 7f65f939d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f65f4153c30 0x7f65f415a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:57.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.302+0000 7f65f939d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65f415afd0 con 0x7f65f40cf110 2023-10-06T02:24:57.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.302+0000 7f65f939d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65f415b2b0 con 0x7f65f415de50 2023-10-06T02:24:57.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.302+0000 7f65f939d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65f415b590 con 0x7f65f4153c30 2023-10-06T02:24:57.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.303+0000 7f65f3fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f65f4153c30 0x7f65f415a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:57.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.303+0000 7f65f8b9c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f65f40cf110 0x7f65f415c070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:59448/0 (socket says 172.21.15.31:59448) 2023-10-06T02:24:57.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.303+0000 7f65f8b9c700 1 -- 172.21.15.31:0/1583381978 learned_addr learned my addr 172.21.15.31:0/1583381978 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:24:57.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.303+0000 7f65f3fff700 1 -- 172.21.15.31:0/1583381978 >> v1:172.21.15.62:6789/0 conn(0x7f65f415de50 legacy=0x7f65f40cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:57.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.303+0000 7f65f3fff700 1 -- 172.21.15.31:0/1583381978 >> v1:172.21.15.31:6789/0 conn(0x7f65f40cf110 legacy=0x7f65f415c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:57.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.303+0000 7f65f2ffd700 1 -- 172.21.15.31:0/1583381978 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 805270175 0 0) 0x7f65f415afd0 con 0x7f65f40cf110 2023-10-06T02:24:57.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.303+0000 7f65f3fff700 1 -- 172.21.15.31:0/1583381978 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65f415b870 con 0x7f65f4153c30 2023-10-06T02:24:57.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.303+0000 7f65f3fff700 1 --2- 172.21.15.31:0/1583381978 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65f4153c30 0x7f65f415a5e0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f65e4009860 tx=0x7f65e40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0d614f2aa9ce82d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:57.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.304+0000 7f65f2ffd700 1 -- 172.21.15.31:0/1583381978 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e4011630 con 0x7f65f4153c30 2023-10-06T02:24:57.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.304+0000 7f65f2ffd700 1 -- 172.21.15.31:0/1583381978 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65e4011790 con 0x7f65f4153c30 2023-10-06T02:24:57.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.304+0000 7f65f2ffd700 1 -- 172.21.15.31:0/1583381978 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e4011b50 con 0x7f65f4153c30 2023-10-06T02:24:57.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.304+0000 7f65f939d700 1 -- 172.21.15.31:0/1583381978 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65f4153c30 msgr2=0x7f65f415a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:57.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.304+0000 7f65f939d700 1 --2- 172.21.15.31:0/1583381978 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65f4153c30 0x7f65f415a5e0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f65e4009860 tx=0x7f65e40091a0 comp rx=0 tx=0).stop 2023-10-06T02:24:57.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.304+0000 7f65f939d700 1 -- 172.21.15.31:0/1583381978 shutdown_connections 2023-10-06T02:24:57.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.304+0000 7f65f939d700 1 --2- 172.21.15.31:0/1583381978 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65f4153c30 0x7f65f415a5e0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:57.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.304+0000 7f65f939d700 1 -- 172.21.15.31:0/1583381978 >> 172.21.15.31:0/1583381978 conn(0x7f65f40703a0 msgr2=0x7f65f40707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:57.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.304+0000 7f65f939d700 1 -- 172.21.15.31:0/1583381978 shutdown_connections 2023-10-06T02:24:57.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.304+0000 7f65f939d700 1 -- 172.21.15.31:0/1583381978 wait complete. 2023-10-06T02:24:57.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.306+0000 7f65f939d700 1 Processor -- start 2023-10-06T02:24:57.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.306+0000 7f65f939d700 1 -- start start 2023-10-06T02:24:57.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.307+0000 7f65f939d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65f40cf110 0x7f65f40b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:57.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.307+0000 7f65f939d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f65f4153c30 0x7f65f40b32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:57.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.307+0000 7f65f939d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65f415de50 0x7f65f40b3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:57.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.307+0000 7f65f939d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65f40c9530 con 0x7f65f415de50 2023-10-06T02:24:57.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.307+0000 7f65f939d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65f40b3e40 con 0x7f65f40cf110 2023-10-06T02:24:57.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.307+0000 7f65f939d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65f40b4120 con 0x7f65f4153c30 2023-10-06T02:24:57.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.307+0000 7f65f8b9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65f415de50 0x7f65f40b3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:57.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.307+0000 7f65f8b9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65f415de50 0x7f65f40b3810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53498/0 (socket says 172.21.15.31:53498) 2023-10-06T02:24:57.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.307+0000 7f65f8b9c700 1 -- 172.21.15.31:0/714769466 learned_addr learned my addr 172.21.15.31:0/714769466 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:57.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.307+0000 7f65f3fff700 1 --2- 172.21.15.31:0/714769466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65f40cf110 0x7f65f40b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:57.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.307+0000 7f65f37fe700 1 --2- 172.21.15.31:0/714769466 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f65f4153c30 0x7f65f40b32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:57.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.307+0000 7f65f8b9c700 1 -- 172.21.15.31:0/714769466 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f65f4153c30 msgr2=0x7f65f40b32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:57.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.307+0000 7f65f8b9c700 1 --2- 172.21.15.31:0/714769466 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f65f4153c30 0x7f65f40b32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:57.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.307+0000 7f65f8b9c700 1 -- 172.21.15.31:0/714769466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65f40cf110 msgr2=0x7f65f40b4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:57.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.307+0000 7f65f8b9c700 1 --2- 172.21.15.31:0/714769466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65f40cf110 0x7f65f40b4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:57.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.307+0000 7f65f8b9c700 1 -- 172.21.15.31:0/714769466 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65e4009420 con 0x7f65f415de50 2023-10-06T02:24:57.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.308+0000 7f65f8b9c700 1 --2- 172.21.15.31:0/714769466 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65f415de50 0x7f65f40b3810 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f65ec00cd70 tx=0x7f65ec001aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:57.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.308+0000 7f65dbfff700 1 -- 172.21.15.31:0/714769466 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65ec00ce70 con 0x7f65f415de50 2023-10-06T02:24:57.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.308+0000 7f65dbfff700 1 -- 172.21.15.31:0/714769466 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65ec00aac0 con 0x7f65f415de50 2023-10-06T02:24:57.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.308+0000 7f65dbfff700 1 -- 172.21.15.31:0/714769466 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65ec00aea0 con 0x7f65f415de50 2023-10-06T02:24:57.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.308+0000 7f65f939d700 1 -- 172.21.15.31:0/714769466 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65f40b4400 con 0x7f65f415de50 2023-10-06T02:24:57.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.308+0000 7f65f939d700 1 -- 172.21.15.31:0/714769466 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65f40b49c0 con 0x7f65f415de50 2023-10-06T02:24:57.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.309+0000 7f65f939d700 1 -- 172.21.15.31:0/714769466 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65f404c3c0 con 0x7f65f415de50 2023-10-06T02:24:57.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.310+0000 7f65dbfff700 1 -- 172.21.15.31:0/714769466 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f65ec007300 con 0x7f65f415de50 2023-10-06T02:24:57.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.311+0000 7f65dbfff700 1 --2- 172.21.15.31:0/714769466 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f65e00670e0 0x7f65e0069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:57.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.311+0000 7f65dbfff700 1 -- 172.21.15.31:0/714769466 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f65ec084da0 con 0x7f65f415de50 2023-10-06T02:24:57.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.311+0000 7f65f3fff700 1 --2- 172.21.15.31:0/714769466 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f65e00670e0 0x7f65e0069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:57.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.312+0000 7f65f3fff700 1 --2- 172.21.15.31:0/714769466 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f65e00670e0 0x7f65e0069590 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f65e40091e0 tx=0x7f65e4016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:57.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.315+0000 7f65dbfff700 1 -- 172.21.15.31:0/714769466 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f65ec051c10 con 0x7f65f415de50 2023-10-06T02:24:57.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.564+0000 7f65f939d700 1 -- 172.21.15.31:0/714769466 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f65f415bb50 con 0x7f65f415de50 2023-10-06T02:24:57.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.564+0000 7f65dbfff700 1 -- 172.21.15.31:0/714769466 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4594 (secure 0 0 0) 0x7f65ec055230 con 0x7f65f415de50 2023-10-06T02:24:57.566 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:24:57.566 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4949,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/178547366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":178547366},{"type":"v1","addr":"172.21.15.62:6835","nonce":178547366}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":5278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/2578831517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":2578831517},{"type":"v1","addr":"172.21.15.31:6836","nonce":2578831517}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5287,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3502585737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":3502585737},{"type":"v1","addr":"172.21.15.31:6837","nonce":3502585737}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:24:43.872314+0000","modified":"2023-10-06T02:24:47.404607+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5290":{"gid":5290,"name":"b","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6834/2358776861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2358776861},{"type":"v1","addr":"172.21.15.62:6834","nonce":2358776861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T02:24:57.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.567+0000 7f65f939d700 1 -- 172.21.15.31:0/714769466 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f65e00670e0 msgr2=0x7f65e0069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:57.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.567+0000 7f65f939d700 1 --2- 172.21.15.31:0/714769466 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f65e00670e0 0x7f65e0069590 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f65e40091e0 tx=0x7f65e4016000 comp rx=0 tx=0).stop 2023-10-06T02:24:57.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.567+0000 7f65f939d700 1 -- 172.21.15.31:0/714769466 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65f415de50 msgr2=0x7f65f40b3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:57.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.567+0000 7f65f939d700 1 --2- 172.21.15.31:0/714769466 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65f415de50 0x7f65f40b3810 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f65ec00cd70 tx=0x7f65ec001aa0 comp rx=0 tx=0).stop 2023-10-06T02:24:57.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.567+0000 7f65f939d700 1 -- 172.21.15.31:0/714769466 shutdown_connections 2023-10-06T02:24:57.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.567+0000 7f65f939d700 1 --2- 172.21.15.31:0/714769466 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65f415de50 0x7f65f40b3810 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:57.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.567+0000 7f65f939d700 1 --2- 172.21.15.31:0/714769466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65f40cf110 0x7f65f40b4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:57.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.567+0000 7f65f939d700 1 --2- 172.21.15.31:0/714769466 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f65e00670e0 0x7f65e0069590 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:57.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.567+0000 7f65f939d700 1 --2- 172.21.15.31:0/714769466 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f65f4153c30 0x7f65f40b32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:57.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.567+0000 7f65f939d700 1 -- 172.21.15.31:0/714769466 >> 172.21.15.31:0/714769466 conn(0x7f65f40703a0 msgr2=0x7f65f40d0940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:57.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.567+0000 7f65f939d700 1 -- 172.21.15.31:0/714769466 shutdown_connections 2023-10-06T02:24:57.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.567+0000 7f65f939d700 1 -- 172.21.15.31:0/714769466 wait complete. 2023-10-06T02:24:57.571 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 27 2023-10-06T02:24:57.586 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json session ls 2023-10-06T02:24:57.807 INFO:teuthology.orchestra.run.smithi062.stdout:[{"id":5637,"entity":{"name":{"type":"client","num":5637},"addr":{"type":"any","addr":"172.21.15.31:0","nonce":2580495381}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":14,"uptime":0.73498902499999996,"requests_in_flight":2,"num_completed_requests":0,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":19.966667137265986,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5637 172.21.15.31:0/2580495381","completed_requests":[],"prealloc_inos":[{"start":"0x10000000005","length":996}],"client_metadata":{"client_features":{"feature_bits":"0x000000000017ffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"736ac5ba72cb47af7d6f6666b1e100f7e20a20e2","ceph_version":"ceph version 17.2.6-1296-g736ac5ba (736ac5ba72cb47af7d6f6666b1e100f7e20a20e2) quincy (stable)","entity_id":"y","hostname":"smithi031","pid":"85292","root":"/"}}] 2023-10-06T02:24:57.807 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "736ac5ba72cb47af7d6f6666b1e100f7e20a20e2", "ceph_version": "ceph version 17.2.6-1296-g736ac5ba (736ac5ba72cb47af7d6f6666b1e100f7e20a20e2) quincy (stable)", "client_features": { "feature_bits": "0x000000000017ffff" }, "entity_id": "y", "hostname": "smithi031", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "85292", "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.31:0", "nonce": 2580495381, "type": "any" }, "name": { "num": 5637, "type": "client" } }, "id": 5637, "inst": "client.5637 172.21.15.31:0/2580495381", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 0, "num_leases": 1, "prealloc_inos": [ { "length": 996, "start": "0x10000000005" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 14, "requests_in_flight": 2, "session_cache_liveness": { "halflife": 300, "value": 19.966667137265986 }, "state": "open", "uptime": 0.734989025 } ] 2023-10-06T02:24:57.808 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:24:57.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.994+0000 7f2c4359d700 1 Processor -- start 2023-10-06T02:24:57.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.995+0000 7f2c4359d700 1 -- start start 2023-10-06T02:24:57.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.996+0000 7f2c4359d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2c3c140e40 0x7f2c3c0c6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:57.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.996+0000 7f2c4359d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2c3c06de50 0x7f2c3c0aeef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:57.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.996+0000 7f2c4359d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c3c0c6d50 con 0x7f2c3c140a70 2023-10-06T02:24:57.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.996+0000 7f2c4359d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c3c0c7030 con 0x7f2c3c140e40 2023-10-06T02:24:57.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.996+0000 7f2c4359d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c3c0c7310 con 0x7f2c3c06de50 2023-10-06T02:24:57.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.996+0000 7f2c41d9a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2c3c140e40 0x7f2c3c0c6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:57.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.996+0000 7f2c41d9a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2c3c140e40 0x7f2c3c0c6770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43622/0 (socket says 172.21.15.31:43622) 2023-10-06T02:24:57.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.996+0000 7f2c41d9a700 1 -- 172.21.15.31:0/3856635555 learned_addr learned my addr 172.21.15.31:0/3856635555 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:57.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.996+0000 7f2c42d9c700 1 --2- 172.21.15.31:0/3856635555 >> v2:172.21.15.62:3300/0 conn(0x7f2c3c06de50 0x7f2c3c0aeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:57.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.997+0000 7f2c41d9a700 1 -- 172.21.15.31:0/3856635555 >> v2:172.21.15.62:3300/0 conn(0x7f2c3c06de50 msgr2=0x7f2c3c0aeef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:24:57.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.997+0000 7f2c41d9a700 1 --2- 172.21.15.31:0/3856635555 >> v2:172.21.15.62:3300/0 conn(0x7f2c3c06de50 0x7f2c3c0aeef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:57.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.997+0000 7f2c41d9a700 1 -- 172.21.15.31:0/3856635555 >> v1:172.21.15.62:6789/0 conn(0x7f2c3c140a70 legacy=0x7f2c3c0c7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:57.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.997+0000 7f2c41d9a700 1 -- 172.21.15.31:0/3856635555 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c3c0c75f0 con 0x7f2c3c140e40 2023-10-06T02:24:58.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.997+0000 7f2c41d9a700 1 --2- 172.21.15.31:0/3856635555 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c3c140e40 0x7f2c3c0c6770 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c0aec50 tx=0x7f2c2c009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93b6551c00445127 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:58.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.997+0000 7f2c41599700 1 -- 172.21.15.31:0/3856635555 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c005c10 con 0x7f2c3c140e40 2023-10-06T02:24:58.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.997+0000 7f2c41599700 1 -- 172.21.15.31:0/3856635555 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c2c005d70 con 0x7f2c3c140e40 2023-10-06T02:24:58.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.997+0000 7f2c41599700 1 -- 172.21.15.31:0/3856635555 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c006260 con 0x7f2c3c140e40 2023-10-06T02:24:58.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.998+0000 7f2c4359d700 1 -- 172.21.15.31:0/3856635555 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c3c140e40 msgr2=0x7f2c3c0c6770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:58.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.998+0000 7f2c4359d700 1 --2- 172.21.15.31:0/3856635555 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c3c140e40 0x7f2c3c0c6770 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c0aec50 tx=0x7f2c2c009220 comp rx=0 tx=0).stop 2023-10-06T02:24:58.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.998+0000 7f2c4359d700 1 -- 172.21.15.31:0/3856635555 shutdown_connections 2023-10-06T02:24:58.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.998+0000 7f2c4359d700 1 --2- 172.21.15.31:0/3856635555 >> v2:172.21.15.62:3300/0 conn(0x7f2c3c06de50 0x7f2c3c0aeef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:58.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.998+0000 7f2c4359d700 1 --2- 172.21.15.31:0/3856635555 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c3c140e40 0x7f2c3c0c6770 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:58.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.998+0000 7f2c4359d700 1 -- 172.21.15.31:0/3856635555 >> 172.21.15.31:0/3856635555 conn(0x7f2c3c06d170 msgr2=0x7f2c3c0cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:58.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.998+0000 7f2c4359d700 1 -- 172.21.15.31:0/3856635555 shutdown_connections 2023-10-06T02:24:58.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.998+0000 7f2c4359d700 1 -- 172.21.15.31:0/3856635555 wait complete. 2023-10-06T02:24:58.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:57.999+0000 7f2c4359d700 1 Processor -- start 2023-10-06T02:24:58.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:58.000+0000 7f2c4359d700 1 -- start start 2023-10-06T02:24:58.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:58.000+0000 7f2c4359d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c3c06de50 0x7f2c3c0ae060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:58.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:58.000+0000 7f2c4359d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c3c140a70 0x7f2c3c0c4130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:58.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:58.000+0000 7f2c4359d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2c3c140e40 0x7f2c3c13e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:58.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:58.001+0000 7f2c4359d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c3c16db90 con 0x7f2c3c140a70 2023-10-06T02:24:58.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:58.001+0000 7f2c4359d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c3c0ae5a0 con 0x7f2c3c06de50 2023-10-06T02:24:58.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:58.001+0000 7f2c4359d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c3c16d1f0 con 0x7f2c3c140e40 2023-10-06T02:24:58.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:58.001+0000 7f2c41d9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c3c140a70 0x7f2c3c0c4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:58.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:58.001+0000 7f2c41d9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c3c140a70 0x7f2c3c0c4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43634/0 (socket says 172.21.15.31:43634) 2023-10-06T02:24:58.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:58.001+0000 7f2c41d9a700 1 -- 172.21.15.31:0/1599008379 learned_addr learned my addr 172.21.15.31:0/1599008379 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:24:58.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:58.001+0000 7f2c4259b700 1 --2- 172.21.15.31:0/1599008379 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c3c06de50 0x7f2c3c0ae060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:58.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:58.001+0000 7f2c42d9c700 1 --2- 172.21.15.31:0/1599008379 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2c3c140e40 0x7f2c3c13e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:58.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:58.001+0000 7f2c41d9a700 1 -- 172.21.15.31:0/1599008379 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2c3c140e40 msgr2=0x7f2c3c13e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:58.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:58.001+0000 7f2c41d9a700 1 --2- 172.21.15.31:0/1599008379 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2c3c140e40 0x7f2c3c13e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:58.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:58.001+0000 7f2c41d9a700 1 -- 172.21.15.31:0/1599008379 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c3c06de50 msgr2=0x7f2c3c0ae060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:58.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:58.001+0000 7f2c41d9a700 1 --2- 172.21.15.31:0/1599008379 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c3c06de50 0x7f2c3c0ae060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:58.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:58.001+0000 7f2c41d9a700 1 -- 172.21.15.31:0/1599008379 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c2c009480 con 0x7f2c3c140a70 2023-10-06T02:24:58.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:58.001+0000 7f2c41d9a700 1 --2- 172.21.15.31:0/1599008379 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c3c140a70 0x7f2c3c0c4130 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c071320 tx=0x7f2c2c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:58.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:58.002+0000 7f2c327fc700 1 -- 172.21.15.31:0/1599008379 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c00b8d0 con 0x7f2c3c140a70 2023-10-06T02:24:58.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:58.002+0000 7f2c327fc700 1 -- 172.21.15.31:0/1599008379 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c2c00ba30 con 0x7f2c3c140a70 2023-10-06T02:24:58.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:58.003+0000 7f2c327fc700 1 -- 172.21.15.31:0/1599008379 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c00bdd0 con 0x7f2c3c140a70 2023-10-06T02:24:58.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:58.003+0000 7f2c4359d700 1 -- 172.21.15.31:0/1599008379 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c3c16d4d0 con 0x7f2c3c140a70 2023-10-06T02:24:58.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:58.003+0000 7f2c4359d700 1 -- 172.21.15.31:0/1599008379 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c3c0c6490 con 0x7f2c3c140a70 2023-10-06T02:24:58.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:58.004+0000 7f2c4359d700 1 -- 172.21.15.31:0/1599008379 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c3c16dcd0 con 0x7f2c3c140a70 2023-10-06T02:24:58.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:58.005+0000 7f2c327fc700 1 -- 172.21.15.31:0/1599008379 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f2c2c01b640 con 0x7f2c3c140a70 2023-10-06T02:24:58.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:58.006+0000 7f2c327fc700 1 --2- 172.21.15.31:0/1599008379 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f2c200670e0 0x7f2c20069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:24:58.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:58.006+0000 7f2c327fc700 1 -- 172.21.15.31:0/1599008379 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2c2c086230 con 0x7f2c3c140a70 2023-10-06T02:24:58.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:58.007+0000 7f2c4259b700 1 --2- 172.21.15.31:0/1599008379 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f2c200670e0 0x7f2c20069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:24:58.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:58.009+0000 7f2c4259b700 1 --2- 172.21.15.31:0/1599008379 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f2c200670e0 0x7f2c20069590 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c16d0b0 tx=0x7f2c38009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:24:58.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:58.009+0000 7f2c327fc700 1 -- 172.21.15.31:0/1599008379 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mgrmap(e 8) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f2c2c053490 con 0x7f2c3c140a70 2023-10-06T02:24:58.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:58.015+0000 7f2c327fc700 1 -- 172.21.15.31:0/1599008379 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2c2c06d3a0 con 0x7f2c3c140a70 2023-10-06T02:24:58.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:58.264+0000 7f2c4359d700 1 -- 172.21.15.31:0/1599008379 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2c3c16dcd0 con 0x7f2c3c140a70 2023-10-06T02:24:58.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:58.264+0000 7f2c327fc700 1 -- 172.21.15.31:0/1599008379 <== mon.0 v2:172.21.15.31:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4594 (secure 0 0 0) 0x7f2c3c16dcd0 con 0x7f2c3c140a70 2023-10-06T02:24:58.266 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:24:58.266 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4949,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/178547366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":178547366},{"type":"v1","addr":"172.21.15.62:6835","nonce":178547366}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":5278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/2578831517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":2578831517},{"type":"v1","addr":"172.21.15.31:6836","nonce":2578831517}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5287,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3502585737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":3502585737},{"type":"v1","addr":"172.21.15.31:6837","nonce":3502585737}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:24:43.872314+0000","modified":"2023-10-06T02:24:47.404607+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5290":{"gid":5290,"name":"b","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6834/2358776861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2358776861},{"type":"v1","addr":"172.21.15.62:6834","nonce":2358776861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T02:24:58.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:58.266+0000 7f2c4359d700 1 -- 172.21.15.31:0/1599008379 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f2c200670e0 msgr2=0x7f2c20069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:58.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:58.266+0000 7f2c4359d700 1 --2- 172.21.15.31:0/1599008379 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f2c200670e0 0x7f2c20069590 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c16d0b0 tx=0x7f2c38009040 comp rx=0 tx=0).stop 2023-10-06T02:24:58.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:58.266+0000 7f2c4359d700 1 -- 172.21.15.31:0/1599008379 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c3c140a70 msgr2=0x7f2c3c0c4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:24:58.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:58.266+0000 7f2c4359d700 1 --2- 172.21.15.31:0/1599008379 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c3c140a70 0x7f2c3c0c4130 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c071320 tx=0x7f2c2c016000 comp rx=0 tx=0).stop 2023-10-06T02:24:58.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:58.266+0000 7f2c4359d700 1 -- 172.21.15.31:0/1599008379 shutdown_connections 2023-10-06T02:24:58.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:58.266+0000 7f2c4359d700 1 --2- 172.21.15.31:0/1599008379 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f2c200670e0 0x7f2c20069590 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:58.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:58.266+0000 7f2c4359d700 1 --2- 172.21.15.31:0/1599008379 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2c3c140e40 0x7f2c3c13e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:58.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:58.266+0000 7f2c4359d700 1 --2- 172.21.15.31:0/1599008379 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c3c06de50 0x7f2c3c0ae060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:58.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:58.266+0000 7f2c4359d700 1 --2- 172.21.15.31:0/1599008379 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c3c140a70 0x7f2c3c0c4130 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:24:58.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:58.266+0000 7f2c4359d700 1 -- 172.21.15.31:0/1599008379 >> 172.21.15.31:0/1599008379 conn(0x7f2c3c06d170 msgr2=0x7f2c3c0c7c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:24:58.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:58.267+0000 7f2c4359d700 1 -- 172.21.15.31:0/1599008379 shutdown_connections 2023-10-06T02:24:58.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:24:58.267+0000 7f2c4359d700 1 -- 172.21.15.31:0/1599008379 wait complete. 2023-10-06T02:24:58.269 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 27 2023-10-06T02:24:58.285 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json session ls 2023-10-06T02:24:58.495 INFO:teuthology.orchestra.run.smithi062.stdout:[{"id":5637,"entity":{"name":{"type":"client","num":5637},"addr":{"type":"any","addr":"172.21.15.31:0","nonce":2580495381}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":14,"uptime":1.422978753,"requests_in_flight":2,"num_completed_requests":0,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":19.93495345957405,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5637 172.21.15.31:0/2580495381","completed_requests":[],"prealloc_inos":[{"start":"0x10000000005","length":996}],"client_metadata":{"client_features":{"feature_bits":"0x000000000017ffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"736ac5ba72cb47af7d6f6666b1e100f7e20a20e2","ceph_version":"ceph version 17.2.6-1296-g736ac5ba (736ac5ba72cb47af7d6f6666b1e100f7e20a20e2) quincy (stable)","entity_id":"y","hostname":"smithi031","pid":"85292","root":"/"}}] 2023-10-06T02:24:58.496 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "736ac5ba72cb47af7d6f6666b1e100f7e20a20e2", "ceph_version": "ceph version 17.2.6-1296-g736ac5ba (736ac5ba72cb47af7d6f6666b1e100f7e20a20e2) quincy (stable)", "client_features": { "feature_bits": "0x000000000017ffff" }, "entity_id": "y", "hostname": "smithi031", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "85292", "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.31:0", "nonce": 2580495381, "type": "any" }, "name": { "num": 5637, "type": "client" } }, "id": 5637, "inst": "client.5637 172.21.15.31:0/2580495381", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 0, "num_leases": 1, "prealloc_inos": [ { "length": 996, "start": "0x10000000005" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 14, "requests_in_flight": 2, "session_cache_liveness": { "halflife": 300, "value": 19.93495345957405 }, "state": "open", "uptime": 1.422978753 } ] 2023-10-06T02:24:58.496 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-10-06T02:25:00.606 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T02:25:00.611 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T02:25:03.497 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:25:03.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:03.697+0000 7fba333ff700 1 Processor -- start 2023-10-06T02:25:03.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:03.697+0000 7fba333ff700 1 -- start start 2023-10-06T02:25:03.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:03.698+0000 7fba333ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fba2c157600 0x7fba2c0b1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:25:03.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:03.698+0000 7fba333ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fba2c0ccba0 0x7fba2c0b1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:25:03.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:03.698+0000 7fba333ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba2c158210 con 0x7fba2c1579d0 2023-10-06T02:25:03.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:03.698+0000 7fba333ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba2c0cb850 con 0x7fba2c0ccba0 2023-10-06T02:25:03.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:03.698+0000 7fba333ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba2c0cb290 con 0x7fba2c157600 2023-10-06T02:25:03.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:03.699+0000 7fba3119b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fba2c157600 0x7fba2c0b1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:25:03.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:03.699+0000 7fba3099a700 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fba2c1579d0 0x7fba2c1586a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:37660/0 (socket says 172.21.15.31:37660) 2023-10-06T02:25:03.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:03.699+0000 7fba3119b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fba2c157600 0x7fba2c0b1410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:56676/0 (socket says 172.21.15.31:56676) 2023-10-06T02:25:03.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:03.699+0000 7fba3099a700 1 -- 172.21.15.31:0/1524800889 learned_addr learned my addr 172.21.15.31:0/1524800889 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:25:03.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:03.699+0000 7fba3199c700 1 --2- 172.21.15.31:0/1524800889 >> v2:172.21.15.62:3300/0 conn(0x7fba2c0ccba0 0x7fba2c0b1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:25:03.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:03.699+0000 7fba3119b700 1 -- 172.21.15.31:0/1524800889 >> v2:172.21.15.62:3300/0 conn(0x7fba2c0ccba0 msgr2=0x7fba2c0b1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:25:03.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:03.699+0000 7fba3119b700 1 --2- 172.21.15.31:0/1524800889 >> v2:172.21.15.62:3300/0 conn(0x7fba2c0ccba0 0x7fba2c0b1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:03.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:03.699+0000 7fba3119b700 1 -- 172.21.15.31:0/1524800889 >> v1:172.21.15.62:6789/0 conn(0x7fba2c1579d0 legacy=0x7fba2c1586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:25:03.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:03.699+0000 7fba3119b700 1 -- 172.21.15.31:0/1524800889 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba2c0cb570 con 0x7fba2c157600 2023-10-06T02:25:03.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:03.699+0000 7fba3199c700 1 --2- 172.21.15.31:0/1524800889 >> v2:172.21.15.62:3300/0 conn(0x7fba2c0ccba0 0x7fba2c0b1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:25:03.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:03.700+0000 7fba3119b700 1 --2- 172.21.15.31:0/1524800889 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fba2c157600 0x7fba2c0b1410 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fba280098e0 tx=0x7fba280091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60d7b7686136e7a4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:25:03.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:03.700+0000 7fba23fff700 1 -- 172.21.15.31:0/1524800889 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba2800b640 con 0x7fba2c157600 2023-10-06T02:25:03.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:03.700+0000 7fba23fff700 1 -- 172.21.15.31:0/1524800889 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba2800b7a0 con 0x7fba2c157600 2023-10-06T02:25:03.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:03.700+0000 7fba23fff700 1 -- 172.21.15.31:0/1524800889 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba2800bb40 con 0x7fba2c157600 2023-10-06T02:25:03.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:03.700+0000 7fba333ff700 1 -- 172.21.15.31:0/1524800889 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fba2c157600 msgr2=0x7fba2c0b1410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:25:03.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:03.700+0000 7fba333ff700 1 --2- 172.21.15.31:0/1524800889 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fba2c157600 0x7fba2c0b1410 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fba280098e0 tx=0x7fba280091a0 comp rx=0 tx=0).stop 2023-10-06T02:25:03.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:03.701+0000 7fba333ff700 1 -- 172.21.15.31:0/1524800889 shutdown_connections 2023-10-06T02:25:03.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:03.701+0000 7fba333ff700 1 --2- 172.21.15.31:0/1524800889 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fba2c157600 0x7fba2c0b1410 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:03.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:03.701+0000 7fba333ff700 1 --2- 172.21.15.31:0/1524800889 >> v2:172.21.15.62:3300/0 conn(0x7fba2c0ccba0 0x7fba2c0b1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:03.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:03.701+0000 7fba333ff700 1 -- 172.21.15.31:0/1524800889 >> 172.21.15.31:0/1524800889 conn(0x7fba2c06d170 msgr2=0x7fba2c0cc300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:25:03.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:03.701+0000 7fba333ff700 1 -- 172.21.15.31:0/1524800889 shutdown_connections 2023-10-06T02:25:03.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:03.701+0000 7fba333ff700 1 -- 172.21.15.31:0/1524800889 wait complete. 2023-10-06T02:25:03.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:03.702+0000 7fba333ff700 1 Processor -- start 2023-10-06T02:25:03.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:03.702+0000 7fba333ff700 1 -- start start 2023-10-06T02:25:03.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:03.703+0000 7fba333ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fba2c0ccba0 0x7fba2c153c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:25:03.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:03.703+0000 7fba333ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba2c157600 0x7fba2c0c3b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:25:03.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:03.703+0000 7fba333ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba2c1579d0 0x7fba2c159290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:25:03.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:03.703+0000 7fba333ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba2c0c47c0 con 0x7fba2c157600 2023-10-06T02:25:03.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:03.703+0000 7fba333ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba2c0c1890 con 0x7fba2c1579d0 2023-10-06T02:25:03.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:03.703+0000 7fba333ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba2c0c44e0 con 0x7fba2c0ccba0 2023-10-06T02:25:03.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:03.703+0000 7fba3099a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba2c157600 0x7fba2c0c3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:25:03.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:03.704+0000 7fba3099a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba2c157600 0x7fba2c0c3b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43638/0 (socket says 172.21.15.31:43638) 2023-10-06T02:25:03.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:03.704+0000 7fba3099a700 1 -- 172.21.15.31:0/1512815382 learned_addr learned my addr 172.21.15.31:0/1512815382 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:25:03.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:03.704+0000 7fba3119b700 1 --2- 172.21.15.31:0/1512815382 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fba2c0ccba0 0x7fba2c153c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:25:03.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:03.704+0000 7fba3199c700 1 --2- 172.21.15.31:0/1512815382 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba2c1579d0 0x7fba2c159290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:25:03.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:03.704+0000 7fba3099a700 1 -- 172.21.15.31:0/1512815382 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fba2c0ccba0 msgr2=0x7fba2c153c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:25:03.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:03.704+0000 7fba3099a700 1 --2- 172.21.15.31:0/1512815382 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fba2c0ccba0 0x7fba2c153c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:03.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:03.704+0000 7fba3099a700 1 -- 172.21.15.31:0/1512815382 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba2c1579d0 msgr2=0x7fba2c159290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:25:03.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:03.704+0000 7fba3099a700 1 --2- 172.21.15.31:0/1512815382 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba2c1579d0 0x7fba2c159290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:03.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:03.704+0000 7fba3099a700 1 -- 172.21.15.31:0/1512815382 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba24009a20 con 0x7fba2c157600 2023-10-06T02:25:03.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:03.704+0000 7fba3099a700 1 --2- 172.21.15.31:0/1512815382 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba2c157600 0x7fba2c0c3b90 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fba1c00c7c0 tx=0x7fba1c007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:25:03.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:03.704+0000 7fba217fa700 1 -- 172.21.15.31:0/1512815382 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba1c005b30 con 0x7fba2c157600 2023-10-06T02:25:03.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:03.704+0000 7fba217fa700 1 -- 172.21.15.31:0/1512815382 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba1c005cb0 con 0x7fba2c157600 2023-10-06T02:25:03.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:03.704+0000 7fba217fa700 1 -- 172.21.15.31:0/1512815382 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba1c004a80 con 0x7fba2c157600 2023-10-06T02:25:03.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:03.704+0000 7fba333ff700 1 -- 172.21.15.31:0/1512815382 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba28009420 con 0x7fba2c157600 2023-10-06T02:25:03.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:03.705+0000 7fba333ff700 1 -- 172.21.15.31:0/1512815382 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba2c0c1520 con 0x7fba2c157600 2023-10-06T02:25:03.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:03.705+0000 7fba333ff700 1 -- 172.21.15.31:0/1512815382 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba2c0c10a0 con 0x7fba2c157600 2023-10-06T02:25:03.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:03.706+0000 7fba217fa700 1 -- 172.21.15.31:0/1512815382 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 84944+0+0 (secure 0 0 0) 0x7fba1c0070f0 con 0x7fba2c157600 2023-10-06T02:25:03.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:03.707+0000 7fba217fa700 1 --2- 172.21.15.31:0/1512815382 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fba10067170 0x7fba10069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:25:03.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:03.707+0000 7fba217fa700 1 -- 172.21.15.31:0/1512815382 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fba1c083e80 con 0x7fba2c157600 2023-10-06T02:25:03.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:03.707+0000 7fba3119b700 1 --2- 172.21.15.31:0/1512815382 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fba10067170 0x7fba10069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:25:03.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:03.708+0000 7fba3119b700 1 --2- 172.21.15.31:0/1512815382 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fba10067170 0x7fba10069620 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fba280093f0 tx=0x7fba28017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:25:03.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:03.712+0000 7fba217fa700 1 -- 172.21.15.31:0/1512815382 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fba2c0c10a0 con 0x7fba2c157600 2023-10-06T02:25:03.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:03.958+0000 7fba333ff700 1 -- 172.21.15.31:0/1512815382 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fba2c0c10a0 con 0x7fba2c157600 2023-10-06T02:25:03.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:03.959+0000 7fba217fa700 1 -- 172.21.15.31:0/1512815382 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4594 (secure 0 0 0) 0x7fba2c0c10a0 con 0x7fba2c157600 2023-10-06T02:25:03.960 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:25:03.960 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4949,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/178547366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":178547366},{"type":"v1","addr":"172.21.15.62:6835","nonce":178547366}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":5278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/2578831517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":2578831517},{"type":"v1","addr":"172.21.15.31:6836","nonce":2578831517}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5287,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3502585737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":3502585737},{"type":"v1","addr":"172.21.15.31:6837","nonce":3502585737}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:24:43.872314+0000","modified":"2023-10-06T02:24:47.404607+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5290":{"gid":5290,"name":"b","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6834/2358776861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2358776861},{"type":"v1","addr":"172.21.15.62:6834","nonce":2358776861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T02:25:03.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:03.961+0000 7fba333ff700 1 -- 172.21.15.31:0/1512815382 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fba10067170 msgr2=0x7fba10069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:25:03.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:03.961+0000 7fba333ff700 1 --2- 172.21.15.31:0/1512815382 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fba10067170 0x7fba10069620 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fba280093f0 tx=0x7fba28017000 comp rx=0 tx=0).stop 2023-10-06T02:25:03.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:03.961+0000 7fba333ff700 1 -- 172.21.15.31:0/1512815382 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba2c157600 msgr2=0x7fba2c0c3b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:25:03.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:03.961+0000 7fba333ff700 1 --2- 172.21.15.31:0/1512815382 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba2c157600 0x7fba2c0c3b90 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fba1c00c7c0 tx=0x7fba1c007f70 comp rx=0 tx=0).stop 2023-10-06T02:25:03.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:03.961+0000 7fba333ff700 1 -- 172.21.15.31:0/1512815382 shutdown_connections 2023-10-06T02:25:03.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:03.961+0000 7fba333ff700 1 --2- 172.21.15.31:0/1512815382 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba2c1579d0 0x7fba2c159290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:03.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:03.961+0000 7fba333ff700 1 --2- 172.21.15.31:0/1512815382 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fba10067170 0x7fba10069620 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:03.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:03.961+0000 7fba333ff700 1 --2- 172.21.15.31:0/1512815382 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fba2c0ccba0 0x7fba2c153c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:03.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:03.961+0000 7fba333ff700 1 --2- 172.21.15.31:0/1512815382 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba2c157600 0x7fba2c0c3b90 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:03.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:03.961+0000 7fba333ff700 1 -- 172.21.15.31:0/1512815382 >> 172.21.15.31:0/1512815382 conn(0x7fba2c06d170 msgr2=0x7fba2c0ad760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:25:03.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:03.961+0000 7fba333ff700 1 -- 172.21.15.31:0/1512815382 shutdown_connections 2023-10-06T02:25:03.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:03.961+0000 7fba333ff700 1 -- 172.21.15.31:0/1512815382 wait complete. 2023-10-06T02:25:03.964 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 27 2023-10-06T02:25:03.980 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json session ls 2023-10-06T02:25:04.201 INFO:teuthology.orchestra.run.smithi062.stdout:[{"id":5637,"entity":{"name":{"type":"client","num":5637},"addr":{"type":"any","addr":"172.21.15.31:0","nonce":2580495381}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":13,"uptime":7.1278935099999998,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":19.673912002344412,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5637 172.21.15.31:0/2580495381","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x000000000017ffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"736ac5ba72cb47af7d6f6666b1e100f7e20a20e2","ceph_version":"ceph version 17.2.6-1296-g736ac5ba (736ac5ba72cb47af7d6f6666b1e100f7e20a20e2) quincy (stable)","entity_id":"y","hostname":"smithi031","pid":"85292","root":"/"}}] 2023-10-06T02:25:04.201 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "736ac5ba72cb47af7d6f6666b1e100f7e20a20e2", "ceph_version": "ceph version 17.2.6-1296-g736ac5ba (736ac5ba72cb47af7d6f6666b1e100f7e20a20e2) quincy (stable)", "client_features": { "feature_bits": "0x000000000017ffff" }, "entity_id": "y", "hostname": "smithi031", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "85292", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.31:0", "nonce": 2580495381, "type": "any" }, "name": { "num": 5637, "type": "client" } }, "id": 5637, "inst": "client.5637 172.21.15.31:0/2580495381", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 13, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 19.673912002344412 }, "state": "open", "uptime": 7.12789351 } ] 2023-10-06T02:25:04.201 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-10-06T02:25:09.202 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:25:09.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:09.399+0000 7f525cb9d700 1 Processor -- start 2023-10-06T02:25:09.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:09.399+0000 7f525cb9d700 1 -- start start 2023-10-06T02:25:09.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:09.400+0000 7f525cb9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f525806e0a0 0x7f52580c6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:25:09.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:09.400+0000 7f525cb9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5258150380 0x7f525806a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:25:09.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:09.400+0000 7f525cb9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f525806a490 con 0x7f525806f3a0 2023-10-06T02:25:09.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:09.400+0000 7f525cb9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52580c7490 con 0x7f525806e0a0 2023-10-06T02:25:09.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:09.401+0000 7f525cb9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52580c7770 con 0x7f5258150380 2023-10-06T02:25:09.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:09.401+0000 7f52577fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f525806e0a0 0x7f52580c6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:25:09.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:09.401+0000 7f52577fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f525806e0a0 0x7f52580c6b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46090/0 (socket says 172.21.15.31:46090) 2023-10-06T02:25:09.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:09.401+0000 7f52577fe700 1 -- 172.21.15.31:0/3329920215 learned_addr learned my addr 172.21.15.31:0/3329920215 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:25:09.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:09.401+0000 7f5256ffd700 1 --1- 172.21.15.31:0/3329920215 >> v1:172.21.15.31:6789/0 conn(0x7f525806f3a0 0x7f525814fc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:33158/0 (socket says 172.21.15.31:33158) 2023-10-06T02:25:09.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:09.401+0000 7f5257fff700 1 --2- 172.21.15.31:0/3329920215 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5258150380 0x7f525806a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:25:09.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:09.401+0000 7f52577fe700 1 -- 172.21.15.31:0/3329920215 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5258150380 msgr2=0x7f525806a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:25:09.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:09.401+0000 7f52577fe700 1 --2- 172.21.15.31:0/3329920215 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5258150380 0x7f525806a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:09.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:09.401+0000 7f52577fe700 1 -- 172.21.15.31:0/3329920215 >> v1:172.21.15.31:6789/0 conn(0x7f525806f3a0 legacy=0x7f525814fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:25:09.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:09.401+0000 7f52577fe700 1 -- 172.21.15.31:0/3329920215 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52580c7a50 con 0x7f525806e0a0 2023-10-06T02:25:09.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:09.401+0000 7f52567fc700 1 -- 172.21.15.31:0/3329920215 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 868191754 0 0) 0x7f525806a490 con 0x7f525806f3a0 2023-10-06T02:25:09.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:09.402+0000 7f52577fe700 1 --2- 172.21.15.31:0/3329920215 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f525806e0a0 0x7f52580c6b20 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f52480099b0 tx=0x7f5248009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79d6f90839951b2e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:25:09.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:09.402+0000 7f52567fc700 1 -- 172.21.15.31:0/3329920215 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5248009ea0 con 0x7f525806e0a0 2023-10-06T02:25:09.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:09.402+0000 7f52567fc700 1 -- 172.21.15.31:0/3329920215 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f524800b860 con 0x7f525806e0a0 2023-10-06T02:25:09.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:09.402+0000 7f52567fc700 1 -- 172.21.15.31:0/3329920215 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524800bc00 con 0x7f525806e0a0 2023-10-06T02:25:09.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:09.402+0000 7f525cb9d700 1 -- 172.21.15.31:0/3329920215 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f525806e0a0 msgr2=0x7f52580c6b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:25:09.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:09.402+0000 7f525cb9d700 1 --2- 172.21.15.31:0/3329920215 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f525806e0a0 0x7f52580c6b20 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f52480099b0 tx=0x7f5248009250 comp rx=0 tx=0).stop 2023-10-06T02:25:09.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:09.402+0000 7f525cb9d700 1 -- 172.21.15.31:0/3329920215 shutdown_connections 2023-10-06T02:25:09.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:09.402+0000 7f525cb9d700 1 --2- 172.21.15.31:0/3329920215 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f525806e0a0 0x7f52580c6b20 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:09.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:09.402+0000 7f525cb9d700 1 --2- 172.21.15.31:0/3329920215 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5258150380 0x7f525806a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:09.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:09.402+0000 7f525cb9d700 1 -- 172.21.15.31:0/3329920215 >> 172.21.15.31:0/3329920215 conn(0x7f525806d170 msgr2=0x7f52580cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:25:09.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:09.403+0000 7f525cb9d700 1 -- 172.21.15.31:0/3329920215 shutdown_connections 2023-10-06T02:25:09.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:09.403+0000 7f525cb9d700 1 -- 172.21.15.31:0/3329920215 wait complete. 2023-10-06T02:25:09.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:09.404+0000 7f525cb9d700 1 Processor -- start 2023-10-06T02:25:09.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:09.404+0000 7f525cb9d700 1 -- start start 2023-10-06T02:25:09.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:09.405+0000 7f525cb9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f525806e0a0 0x7f525814a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:25:09.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:09.405+0000 7f525cb9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f525806f3a0 0x7f5258165f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:25:09.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:09.405+0000 7f525cb9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5258150380 0x7f52580c31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:25:09.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:09.405+0000 7f525cb9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f525814a5d0 con 0x7f5258150380 2023-10-06T02:25:09.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:09.405+0000 7f525cb9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52580c4870 con 0x7f525806e0a0 2023-10-06T02:25:09.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:09.405+0000 7f525cb9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52580c4b50 con 0x7f525806f3a0 2023-10-06T02:25:09.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:09.405+0000 7f5257fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5258150380 0x7f52580c31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:25:09.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:09.405+0000 7f5257fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5258150380 0x7f52580c31e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46096/0 (socket says 172.21.15.31:46096) 2023-10-06T02:25:09.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:09.405+0000 7f5257fff700 1 -- 172.21.15.31:0/1394988341 learned_addr learned my addr 172.21.15.31:0/1394988341 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:25:09.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:09.405+0000 7f5256ffd700 1 --2- 172.21.15.31:0/1394988341 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f525806f3a0 0x7f5258165f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:25:09.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:09.405+0000 7f52577fe700 1 --2- 172.21.15.31:0/1394988341 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f525806e0a0 0x7f525814a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:25:09.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:09.406+0000 7f5257fff700 1 -- 172.21.15.31:0/1394988341 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f525806f3a0 msgr2=0x7f5258165f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:25:09.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:09.406+0000 7f5257fff700 1 --2- 172.21.15.31:0/1394988341 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f525806f3a0 0x7f5258165f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:09.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:09.406+0000 7f5257fff700 1 -- 172.21.15.31:0/1394988341 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f525806e0a0 msgr2=0x7f525814a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:25:09.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:09.406+0000 7f5257fff700 1 --2- 172.21.15.31:0/1394988341 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f525806e0a0 0x7f525814a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:09.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:09.406+0000 7f5257fff700 1 -- 172.21.15.31:0/1394988341 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52480094f0 con 0x7f5258150380 2023-10-06T02:25:09.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:09.406+0000 7f52577fe700 1 --2- 172.21.15.31:0/1394988341 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f525806e0a0 0x7f525814a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:25:09.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:09.406+0000 7f5257fff700 1 --2- 172.21.15.31:0/1394988341 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5258150380 0x7f52580c31e0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f525000c4c0 tx=0x7f525000a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:25:09.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:09.406+0000 7f523f7fe700 1 -- 172.21.15.31:0/1394988341 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5250019430 con 0x7f5258150380 2023-10-06T02:25:09.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:09.406+0000 7f523f7fe700 1 -- 172.21.15.31:0/1394988341 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5250019590 con 0x7f5258150380 2023-10-06T02:25:09.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:09.406+0000 7f523f7fe700 1 -- 172.21.15.31:0/1394988341 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5250019950 con 0x7f5258150380 2023-10-06T02:25:09.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:09.406+0000 7f525cb9d700 1 -- 172.21.15.31:0/1394988341 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52580c4e30 con 0x7f5258150380 2023-10-06T02:25:09.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:09.406+0000 7f525cb9d700 1 -- 172.21.15.31:0/1394988341 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5258166460 con 0x7f5258150380 2023-10-06T02:25:09.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:09.407+0000 7f525cb9d700 1 -- 172.21.15.31:0/1394988341 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52581484f0 con 0x7f5258150380 2023-10-06T02:25:09.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:09.408+0000 7f523f7fe700 1 -- 172.21.15.31:0/1394988341 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f5250016030 con 0x7f5258150380 2023-10-06T02:25:09.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:09.409+0000 7f523f7fe700 1 --2- 172.21.15.31:0/1394988341 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5244067170 0x7f5244069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:25:09.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:09.410+0000 7f523f7fe700 1 -- 172.21.15.31:0/1394988341 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f5250085ac0 con 0x7f5258150380 2023-10-06T02:25:09.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:09.410+0000 7f52577fe700 1 --2- 172.21.15.31:0/1394988341 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5244067170 0x7f5244069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:25:09.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:09.410+0000 7f52577fe700 1 --2- 172.21.15.31:0/1394988341 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5244067170 0x7f5244069620 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f52480099b0 tx=0x7f5248019000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:25:09.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:09.414+0000 7f523f7fe700 1 -- 172.21.15.31:0/1394988341 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f52581484f0 con 0x7f5258150380 2023-10-06T02:25:09.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:09.661+0000 7f525cb9d700 1 -- 172.21.15.31:0/1394988341 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f52581484f0 con 0x7f5258150380 2023-10-06T02:25:09.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:09.662+0000 7f523f7fe700 1 -- 172.21.15.31:0/1394988341 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4594 (secure 0 0 0) 0x7f52581484f0 con 0x7f5258150380 2023-10-06T02:25:09.663 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:25:09.663 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4949,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/178547366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":178547366},{"type":"v1","addr":"172.21.15.62:6835","nonce":178547366}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":5278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/2578831517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":2578831517},{"type":"v1","addr":"172.21.15.31:6836","nonce":2578831517}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5287,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3502585737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":3502585737},{"type":"v1","addr":"172.21.15.31:6837","nonce":3502585737}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:24:43.872314+0000","modified":"2023-10-06T02:24:47.404607+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5290":{"gid":5290,"name":"b","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6834/2358776861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2358776861},{"type":"v1","addr":"172.21.15.62:6834","nonce":2358776861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T02:25:09.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:09.664+0000 7f525cb9d700 1 -- 172.21.15.31:0/1394988341 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5244067170 msgr2=0x7f5244069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:25:09.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:09.664+0000 7f525cb9d700 1 --2- 172.21.15.31:0/1394988341 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5244067170 0x7f5244069620 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f52480099b0 tx=0x7f5248019000 comp rx=0 tx=0).stop 2023-10-06T02:25:09.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:09.664+0000 7f525cb9d700 1 -- 172.21.15.31:0/1394988341 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5258150380 msgr2=0x7f52580c31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:25:09.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:09.664+0000 7f525cb9d700 1 --2- 172.21.15.31:0/1394988341 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5258150380 0x7f52580c31e0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f525000c4c0 tx=0x7f525000a6a0 comp rx=0 tx=0).stop 2023-10-06T02:25:09.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:09.664+0000 7f525cb9d700 1 -- 172.21.15.31:0/1394988341 shutdown_connections 2023-10-06T02:25:09.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:09.664+0000 7f525cb9d700 1 --2- 172.21.15.31:0/1394988341 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5258150380 0x7f52580c31e0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:09.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:09.664+0000 7f525cb9d700 1 --2- 172.21.15.31:0/1394988341 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f525806e0a0 0x7f525814a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:09.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:09.664+0000 7f525cb9d700 1 --2- 172.21.15.31:0/1394988341 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5244067170 0x7f5244069620 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:09.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:09.664+0000 7f525cb9d700 1 --2- 172.21.15.31:0/1394988341 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f525806f3a0 0x7f5258165f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:09.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:09.664+0000 7f525cb9d700 1 -- 172.21.15.31:0/1394988341 >> 172.21.15.31:0/1394988341 conn(0x7f525806d170 msgr2=0x7f52580c6750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:25:09.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:09.664+0000 7f525cb9d700 1 -- 172.21.15.31:0/1394988341 shutdown_connections 2023-10-06T02:25:09.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:09.664+0000 7f525cb9d700 1 -- 172.21.15.31:0/1394988341 wait complete. 2023-10-06T02:25:09.667 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 27 2023-10-06T02:25:09.683 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json session ls 2023-10-06T02:25:09.903 INFO:teuthology.orchestra.run.smithi062.stdout:[{"id":5637,"entity":{"name":{"type":"client","num":5637},"addr":{"type":"any","addr":"172.21.15.31:0","nonce":2580495381}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":12,"uptime":12.830808276000001,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":19.416378516608368,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5637 172.21.15.31:0/2580495381","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x000000000017ffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"736ac5ba72cb47af7d6f6666b1e100f7e20a20e2","ceph_version":"ceph version 17.2.6-1296-g736ac5ba (736ac5ba72cb47af7d6f6666b1e100f7e20a20e2) quincy (stable)","entity_id":"y","hostname":"smithi031","pid":"85292","root":"/"}}] 2023-10-06T02:25:09.904 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "736ac5ba72cb47af7d6f6666b1e100f7e20a20e2", "ceph_version": "ceph version 17.2.6-1296-g736ac5ba (736ac5ba72cb47af7d6f6666b1e100f7e20a20e2) quincy (stable)", "client_features": { "feature_bits": "0x000000000017ffff" }, "entity_id": "y", "hostname": "smithi031", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "85292", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.31:0", "nonce": 2580495381, "type": "any" }, "name": { "num": 5637, "type": "client" } }, "id": 5637, "inst": "client.5637 172.21.15.31:0/2580495381", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 12, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 19.41637851660837 }, "state": "open", "uptime": 12.830808276 } ] 2023-10-06T02:25:09.904 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-10-06T02:25:14.904 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:25:15.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:15.101+0000 7fe893fff700 1 Processor -- start 2023-10-06T02:25:15.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:15.102+0000 7fe893fff700 1 -- start start 2023-10-06T02:25:15.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:15.103+0000 7fe893fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe89406e0a0 0x7fe8940c6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:25:15.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:15.103+0000 7fe893fff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe89406f3a0 0x7fe89414fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:25:15.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:15.103+0000 7fe893fff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe89406a490 con 0x7fe89406e470 2023-10-06T02:25:15.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:15.103+0000 7fe893fff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe89406b930 con 0x7fe89406f3a0 2023-10-06T02:25:15.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:15.103+0000 7fe893fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8940c7490 con 0x7fe89406e0a0 2023-10-06T02:25:15.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:15.103+0000 7fe8937fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fe89406e470 0x7fe894159740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:33172/0 (socket says 172.21.15.31:33172) 2023-10-06T02:25:15.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:15.103+0000 7fe8937fe700 1 -- 172.21.15.31:0/3047672602 learned_addr learned my addr 172.21.15.31:0/3047672602 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:25:15.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:15.103+0000 7fe8927fc700 1 --2- 172.21.15.31:0/3047672602 >> v2:172.21.15.62:3300/0 conn(0x7fe89406f3a0 0x7fe89414fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:25:15.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:15.103+0000 7fe892ffd700 1 --2- 172.21.15.31:0/3047672602 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe89406e0a0 0x7fe8940c6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:25:15.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:15.104+0000 7fe891ffb700 1 -- 172.21.15.31:0/3047672602 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2499264670 0 0) 0x7fe89406a490 con 0x7fe89406e470 2023-10-06T02:25:15.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:15.104+0000 7fe891ffb700 1 -- 172.21.15.31:0/3047672602 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe87c003570 con 0x7fe89406e470 2023-10-06T02:25:15.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:15.104+0000 7fe8927fc700 1 -- 172.21.15.31:0/3047672602 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe89406e0a0 msgr2=0x7fe8940c6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:25:15.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:15.104+0000 7fe8927fc700 1 --2- 172.21.15.31:0/3047672602 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe89406e0a0 0x7fe8940c6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:15.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:15.104+0000 7fe8927fc700 1 -- 172.21.15.31:0/3047672602 >> v1:172.21.15.31:6789/0 conn(0x7fe89406e470 legacy=0x7fe894159740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:25:15.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:15.104+0000 7fe8927fc700 1 -- 172.21.15.31:0/3047672602 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8940c7770 con 0x7fe89406f3a0 2023-10-06T02:25:15.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:15.104+0000 7fe892ffd700 1 --2- 172.21.15.31:0/3047672602 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe89406e0a0 0x7fe8940c6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:25:15.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:15.104+0000 7fe8927fc700 1 --2- 172.21.15.31:0/3047672602 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe89406f3a0 0x7fe89414fc70 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe8880098d0 tx=0x7fe888009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa8a4ca711365965 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:25:15.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:15.105+0000 7fe891ffb700 1 -- 172.21.15.31:0/3047672602 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe888005c10 con 0x7fe89406f3a0 2023-10-06T02:25:15.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:15.105+0000 7fe891ffb700 1 -- 172.21.15.31:0/3047672602 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe888005d70 con 0x7fe89406f3a0 2023-10-06T02:25:15.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:15.105+0000 7fe891ffb700 1 -- 172.21.15.31:0/3047672602 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe888006260 con 0x7fe89406f3a0 2023-10-06T02:25:15.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:15.105+0000 7fe893fff700 1 -- 172.21.15.31:0/3047672602 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe89406f3a0 msgr2=0x7fe89414fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:25:15.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:15.105+0000 7fe893fff700 1 --2- 172.21.15.31:0/3047672602 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe89406f3a0 0x7fe89414fc70 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe8880098d0 tx=0x7fe888009120 comp rx=0 tx=0).stop 2023-10-06T02:25:15.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:15.105+0000 7fe893fff700 1 -- 172.21.15.31:0/3047672602 shutdown_connections 2023-10-06T02:25:15.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:15.105+0000 7fe893fff700 1 --2- 172.21.15.31:0/3047672602 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe89406e0a0 0x7fe8940c6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:15.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:15.105+0000 7fe893fff700 1 --2- 172.21.15.31:0/3047672602 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe89406f3a0 0x7fe89414fc70 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:15.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:15.105+0000 7fe893fff700 1 -- 172.21.15.31:0/3047672602 >> 172.21.15.31:0/3047672602 conn(0x7fe89406d170 msgr2=0x7fe8940cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:25:15.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:15.105+0000 7fe893fff700 1 -- 172.21.15.31:0/3047672602 shutdown_connections 2023-10-06T02:25:15.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:15.105+0000 7fe893fff700 1 -- 172.21.15.31:0/3047672602 wait complete. 2023-10-06T02:25:15.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:15.107+0000 7fe893fff700 1 Processor -- start 2023-10-06T02:25:15.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:15.107+0000 7fe893fff700 1 -- start start 2023-10-06T02:25:15.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:15.108+0000 7fe893fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe89406e0a0 0x7fe894155390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:25:15.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:15.108+0000 7fe893fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe89406e470 0x7fe8940c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:25:15.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:15.108+0000 7fe893fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe89406f3a0 0x7fe8940c43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:25:15.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:15.108+0000 7fe893fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe894154f00 con 0x7fe89406f3a0 2023-10-06T02:25:15.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:15.108+0000 7fe893fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8940c6080 con 0x7fe89406e470 2023-10-06T02:25:15.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:15.108+0000 7fe893fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe894143860 con 0x7fe89406e0a0 2023-10-06T02:25:15.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:15.108+0000 7fe8937fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe89406f3a0 0x7fe8940c43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:25:15.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:15.108+0000 7fe8937fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe89406f3a0 0x7fe8940c43d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46102/0 (socket says 172.21.15.31:46102) 2023-10-06T02:25:15.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:15.108+0000 7fe8937fe700 1 -- 172.21.15.31:0/2987157366 learned_addr learned my addr 172.21.15.31:0/2987157366 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:25:15.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:15.108+0000 7fe892ffd700 1 --2- 172.21.15.31:0/2987157366 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe89406e0a0 0x7fe894155390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:25:15.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:15.108+0000 7fe8927fc700 1 --2- 172.21.15.31:0/2987157366 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe89406e470 0x7fe8940c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:25:15.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:15.108+0000 7fe8937fe700 1 -- 172.21.15.31:0/2987157366 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe89406e0a0 msgr2=0x7fe894155390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:25:15.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:15.108+0000 7fe8937fe700 1 --2- 172.21.15.31:0/2987157366 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe89406e0a0 0x7fe894155390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:15.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:15.108+0000 7fe8937fe700 1 -- 172.21.15.31:0/2987157366 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe89406e470 msgr2=0x7fe8940c5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:25:15.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:15.108+0000 7fe8937fe700 1 --2- 172.21.15.31:0/2987157366 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe89406e470 0x7fe8940c5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:15.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:15.108+0000 7fe8937fe700 1 -- 172.21.15.31:0/2987157366 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe884009420 con 0x7fe89406f3a0 2023-10-06T02:25:15.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:15.109+0000 7fe8937fe700 1 --2- 172.21.15.31:0/2987157366 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe89406f3a0 0x7fe8940c43d0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fe88c001460 tx=0x7fe88c001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:25:15.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:15.109+0000 7fe87affd700 1 -- 172.21.15.31:0/2987157366 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe88c00a940 con 0x7fe89406f3a0 2023-10-06T02:25:15.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:15.109+0000 7fe87affd700 1 -- 172.21.15.31:0/2987157366 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe88c00aaa0 con 0x7fe89406f3a0 2023-10-06T02:25:15.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:15.109+0000 7fe87affd700 1 -- 172.21.15.31:0/2987157366 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe88c00ae20 con 0x7fe89406f3a0 2023-10-06T02:25:15.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:15.109+0000 7fe893fff700 1 -- 172.21.15.31:0/2987157366 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8880093c0 con 0x7fe89406f3a0 2023-10-06T02:25:15.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:15.109+0000 7fe893fff700 1 -- 172.21.15.31:0/2987157366 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8941545d0 con 0x7fe89406f3a0 2023-10-06T02:25:15.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:15.110+0000 7fe893fff700 1 -- 172.21.15.31:0/2987157366 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe894155070 con 0x7fe89406f3a0 2023-10-06T02:25:15.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:15.111+0000 7fe87affd700 1 -- 172.21.15.31:0/2987157366 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 84944+0+0 (secure 0 0 0) 0x7fe88c007300 con 0x7fe89406f3a0 2023-10-06T02:25:15.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:15.112+0000 7fe87affd700 1 --2- 172.21.15.31:0/2987157366 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe880067170 0x7fe880069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:25:15.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:15.112+0000 7fe87affd700 1 -- 172.21.15.31:0/2987157366 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe88c084e50 con 0x7fe89406f3a0 2023-10-06T02:25:15.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:15.112+0000 7fe892ffd700 1 --2- 172.21.15.31:0/2987157366 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe880067170 0x7fe880069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:25:15.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:15.113+0000 7fe892ffd700 1 --2- 172.21.15.31:0/2987157366 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe880067170 0x7fe880069620 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe884009f40 tx=0x7fe8840091a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:25:15.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:15.116+0000 7fe87affd700 1 -- 172.21.15.31:0/2987157366 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe88c051cb0 con 0x7fe89406f3a0 2023-10-06T02:25:15.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:15.364+0000 7fe893fff700 1 -- 172.21.15.31:0/2987157366 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe89404c3c0 con 0x7fe89406f3a0 2023-10-06T02:25:15.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:15.365+0000 7fe87affd700 1 -- 172.21.15.31:0/2987157366 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4594 (secure 0 0 0) 0x7fe88c0552d0 con 0x7fe89406f3a0 2023-10-06T02:25:15.366 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:25:15.366 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4949,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/178547366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":178547366},{"type":"v1","addr":"172.21.15.62:6835","nonce":178547366}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":5278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/2578831517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":2578831517},{"type":"v1","addr":"172.21.15.31:6836","nonce":2578831517}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5287,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3502585737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":3502585737},{"type":"v1","addr":"172.21.15.31:6837","nonce":3502585737}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:24:43.872314+0000","modified":"2023-10-06T02:24:47.404607+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5290":{"gid":5290,"name":"b","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6834/2358776861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2358776861},{"type":"v1","addr":"172.21.15.62:6834","nonce":2358776861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T02:25:15.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:15.367+0000 7fe893fff700 1 -- 172.21.15.31:0/2987157366 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe880067170 msgr2=0x7fe880069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:25:15.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:15.367+0000 7fe893fff700 1 --2- 172.21.15.31:0/2987157366 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe880067170 0x7fe880069620 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe884009f40 tx=0x7fe8840091a0 comp rx=0 tx=0).stop 2023-10-06T02:25:15.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:15.367+0000 7fe893fff700 1 -- 172.21.15.31:0/2987157366 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe89406f3a0 msgr2=0x7fe8940c43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:25:15.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:15.367+0000 7fe893fff700 1 --2- 172.21.15.31:0/2987157366 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe89406f3a0 0x7fe8940c43d0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fe88c001460 tx=0x7fe88c001ae0 comp rx=0 tx=0).stop 2023-10-06T02:25:15.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:15.367+0000 7fe893fff700 1 -- 172.21.15.31:0/2987157366 shutdown_connections 2023-10-06T02:25:15.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:15.367+0000 7fe893fff700 1 --2- 172.21.15.31:0/2987157366 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe89406f3a0 0x7fe8940c43d0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:15.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:15.367+0000 7fe893fff700 1 --2- 172.21.15.31:0/2987157366 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe880067170 0x7fe880069620 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:15.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:15.367+0000 7fe893fff700 1 --2- 172.21.15.31:0/2987157366 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe89406e0a0 0x7fe894155390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:15.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:15.367+0000 7fe893fff700 1 --2- 172.21.15.31:0/2987157366 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe89406e470 0x7fe8940c5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:15.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:15.367+0000 7fe893fff700 1 -- 172.21.15.31:0/2987157366 >> 172.21.15.31:0/2987157366 conn(0x7fe89406d170 msgr2=0x7fe894142db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:25:15.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:15.367+0000 7fe893fff700 1 -- 172.21.15.31:0/2987157366 shutdown_connections 2023-10-06T02:25:15.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:15.367+0000 7fe893fff700 1 -- 172.21.15.31:0/2987157366 wait complete. 2023-10-06T02:25:15.370 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 27 2023-10-06T02:25:15.386 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json session ls 2023-10-06T02:25:15.607 INFO:teuthology.orchestra.run.smithi062.stdout:[{"id":5637,"entity":{"name":{"type":"client","num":5637},"addr":{"type":"any","addr":"172.21.15.31:0","nonce":2580495381}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":12,"uptime":18.534723026999998,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":19.162171896682807,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5637 172.21.15.31:0/2580495381","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x000000000017ffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"736ac5ba72cb47af7d6f6666b1e100f7e20a20e2","ceph_version":"ceph version 17.2.6-1296-g736ac5ba (736ac5ba72cb47af7d6f6666b1e100f7e20a20e2) quincy (stable)","entity_id":"y","hostname":"smithi031","pid":"85292","root":"/"}}] 2023-10-06T02:25:15.607 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "736ac5ba72cb47af7d6f6666b1e100f7e20a20e2", "ceph_version": "ceph version 17.2.6-1296-g736ac5ba (736ac5ba72cb47af7d6f6666b1e100f7e20a20e2) quincy (stable)", "client_features": { "feature_bits": "0x000000000017ffff" }, "entity_id": "y", "hostname": "smithi031", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "85292", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.31:0", "nonce": 2580495381, "type": "any" }, "name": { "num": 5637, "type": "client" } }, "id": 5637, "inst": "client.5637 172.21.15.31:0/2580495381", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 12, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 19.162171896682807 }, "state": "open", "uptime": 18.534723027 } ] 2023-10-06T02:25:15.608 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-10-06T02:25:20.609 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:25:20.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:20.805+0000 7f786359e700 1 Processor -- start 2023-10-06T02:25:20.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:20.805+0000 7f786359e700 1 -- start start 2023-10-06T02:25:20.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:20.807+0000 7f786359e700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7864140a70 0x7f78640c6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:25:20.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:20.807+0000 7f786359e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7864140e40 0x7f78640c7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:25:20.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:20.807+0000 7f786359e700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78640c6d50 con 0x7f78640c8330 2023-10-06T02:25:20.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:20.807+0000 7f786359e700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78640c7030 con 0x7f7864140a70 2023-10-06T02:25:20.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:20.807+0000 7f786359e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78640c7310 con 0x7f7864140e40 2023-10-06T02:25:20.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:20.807+0000 7f7861d9b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7864140e40 0x7f78640c7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:25:20.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:20.807+0000 7f786259c700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7864140a70 0x7f78640c6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:25:20.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:20.807+0000 7f7861d9b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7864140e40 0x7f78640c7df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:55832/0 (socket says 172.21.15.31:55832) 2023-10-06T02:25:20.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:20.807+0000 7f786259c700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7864140a70 0x7f78640c6770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:51830/0 (socket says 172.21.15.31:51830) 2023-10-06T02:25:20.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:20.807+0000 7f7861d9b700 1 -- 172.21.15.31:0/1266662308 learned_addr learned my addr 172.21.15.31:0/1266662308 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:25:20.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:20.807+0000 7f7862d9d700 1 --1- 172.21.15.31:0/1266662308 >> v1:172.21.15.62:6789/0 conn(0x7f78640c8330 0x7f78640aeef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:36602/0 (socket says 172.21.15.31:36602) 2023-10-06T02:25:20.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:20.808+0000 7f7861d9b700 1 -- 172.21.15.31:0/1266662308 >> v2:172.21.15.62:3300/0 conn(0x7f7864140a70 msgr2=0x7f78640c6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:25:20.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:20.808+0000 7f7861d9b700 1 --2- 172.21.15.31:0/1266662308 >> v2:172.21.15.62:3300/0 conn(0x7f7864140a70 0x7f78640c6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:20.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:20.808+0000 7f7861d9b700 1 -- 172.21.15.31:0/1266662308 >> v1:172.21.15.62:6789/0 conn(0x7f78640c8330 legacy=0x7f78640aeef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:25:20.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:20.808+0000 7f786159a700 1 -- 172.21.15.31:0/1266662308 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1065146792 0 0) 0x7f78640c6d50 con 0x7f78640c8330 2023-10-06T02:25:20.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:20.808+0000 7f7861d9b700 1 -- 172.21.15.31:0/1266662308 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78640c75f0 con 0x7f7864140e40 2023-10-06T02:25:20.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:20.808+0000 7f7861d9b700 1 --2- 172.21.15.31:0/1266662308 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7864140e40 0x7f78640c7df0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7858009990 tx=0x7f7858009220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6728d0bc349e3c67 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:25:20.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:20.808+0000 7f786159a700 1 -- 172.21.15.31:0/1266662308 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7858009e80 con 0x7f7864140e40 2023-10-06T02:25:20.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:20.809+0000 7f786159a700 1 -- 172.21.15.31:0/1266662308 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7858005c10 con 0x7f7864140e40 2023-10-06T02:25:20.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:20.809+0000 7f786159a700 1 -- 172.21.15.31:0/1266662308 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785800b650 con 0x7f7864140e40 2023-10-06T02:25:20.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:20.809+0000 7f786359e700 1 -- 172.21.15.31:0/1266662308 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7864140e40 msgr2=0x7f78640c7df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:25:20.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:20.809+0000 7f786359e700 1 --2- 172.21.15.31:0/1266662308 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7864140e40 0x7f78640c7df0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7858009990 tx=0x7f7858009220 comp rx=0 tx=0).stop 2023-10-06T02:25:20.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:20.809+0000 7f786359e700 1 -- 172.21.15.31:0/1266662308 shutdown_connections 2023-10-06T02:25:20.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:20.809+0000 7f786359e700 1 --2- 172.21.15.31:0/1266662308 >> v2:172.21.15.62:3300/0 conn(0x7f7864140a70 0x7f78640c6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:20.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:20.809+0000 7f786359e700 1 --2- 172.21.15.31:0/1266662308 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7864140e40 0x7f78640c7df0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:20.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:20.809+0000 7f786359e700 1 -- 172.21.15.31:0/1266662308 >> 172.21.15.31:0/1266662308 conn(0x7f786406d170 msgr2=0x7f78640cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:25:20.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:20.809+0000 7f786359e700 1 -- 172.21.15.31:0/1266662308 shutdown_connections 2023-10-06T02:25:20.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:20.809+0000 7f786359e700 1 -- 172.21.15.31:0/1266662308 wait complete. 2023-10-06T02:25:20.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:20.811+0000 7f786359e700 1 Processor -- start 2023-10-06T02:25:20.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:20.811+0000 7f786359e700 1 -- start start 2023-10-06T02:25:20.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:20.812+0000 7f786359e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f78640c8330 0x7f78640c3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:25:20.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:20.812+0000 7f786359e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7864140a70 0x7f7864156560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:25:20.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:20.812+0000 7f786359e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7864140e40 0x7f78640aeed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:25:20.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:20.812+0000 7f786359e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7864169940 con 0x7f7864140a70 2023-10-06T02:25:20.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:20.812+0000 7f786359e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78640c4130 con 0x7f7864140e40 2023-10-06T02:25:20.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:20.812+0000 7f786359e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78640b0300 con 0x7f78640c8330 2023-10-06T02:25:20.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:20.812+0000 7f7861d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7864140a70 0x7f7864156560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:25:20.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:20.812+0000 7f7861d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7864140a70 0x7f7864156560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36394/0 (socket says 172.21.15.31:36394) 2023-10-06T02:25:20.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:20.812+0000 7f7861d9b700 1 -- 172.21.15.31:0/2323208909 learned_addr learned my addr 172.21.15.31:0/2323208909 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:25:20.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:20.812+0000 7f7862d9d700 1 --2- 172.21.15.31:0/2323208909 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7864140e40 0x7f78640aeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:25:20.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:20.812+0000 7f786259c700 1 --2- 172.21.15.31:0/2323208909 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f78640c8330 0x7f78640c3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:25:20.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:20.812+0000 7f7861d9b700 1 -- 172.21.15.31:0/2323208909 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f78640c8330 msgr2=0x7f78640c3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:25:20.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:20.812+0000 7f7861d9b700 1 --2- 172.21.15.31:0/2323208909 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f78640c8330 0x7f78640c3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:20.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:20.812+0000 7f7861d9b700 1 -- 172.21.15.31:0/2323208909 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7864140e40 msgr2=0x7f78640aeed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:25:20.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:20.812+0000 7f7861d9b700 1 --2- 172.21.15.31:0/2323208909 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7864140e40 0x7f78640aeed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:20.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:20.812+0000 7f7861d9b700 1 -- 172.21.15.31:0/2323208909 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7858009480 con 0x7f7864140a70 2023-10-06T02:25:20.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:20.813+0000 7f7861d9b700 1 --2- 172.21.15.31:0/2323208909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7864140a70 0x7f7864156560 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f78640c7450 tx=0x7f7858016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:25:20.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:20.813+0000 7f78527fc700 1 -- 172.21.15.31:0/2323208909 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7858006240 con 0x7f7864140a70 2023-10-06T02:25:20.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:20.813+0000 7f78527fc700 1 -- 172.21.15.31:0/2323208909 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78580063c0 con 0x7f7864140a70 2023-10-06T02:25:20.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:20.813+0000 7f78527fc700 1 -- 172.21.15.31:0/2323208909 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7858006780 con 0x7f7864140a70 2023-10-06T02:25:20.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:20.813+0000 7f786359e700 1 -- 172.21.15.31:0/2323208909 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7864169b20 con 0x7f7864140a70 2023-10-06T02:25:20.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:20.813+0000 7f786359e700 1 -- 172.21.15.31:0/2323208909 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78640c1ac0 con 0x7f7864140a70 2023-10-06T02:25:20.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:20.814+0000 7f786359e700 1 -- 172.21.15.31:0/2323208909 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f786404c3c0 con 0x7f7864140a70 2023-10-06T02:25:20.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:20.815+0000 7f78527fc700 1 -- 172.21.15.31:0/2323208909 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f785801b690 con 0x7f7864140a70 2023-10-06T02:25:20.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:20.816+0000 7f78527fc700 1 --2- 172.21.15.31:0/2323208909 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f7844067170 0x7f7844069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:25:20.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:20.816+0000 7f78527fc700 1 -- 172.21.15.31:0/2323208909 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f7858086ca0 con 0x7f7864140a70 2023-10-06T02:25:20.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:20.816+0000 7f786259c700 1 --2- 172.21.15.31:0/2323208909 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f7844067170 0x7f7844069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:25:20.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:20.817+0000 7f786259c700 1 --2- 172.21.15.31:0/2323208909 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f7844067170 0x7f7844069620 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7854009d20 tx=0x7f78540091a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:25:20.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:20.821+0000 7f78527fc700 1 -- 172.21.15.31:0/2323208909 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f78580539b0 con 0x7f7864140a70 2023-10-06T02:25:21.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:21.067+0000 7f786359e700 1 -- 172.21.15.31:0/2323208909 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f78640681e0 con 0x7f7864140a70 2023-10-06T02:25:21.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:21.068+0000 7f78527fc700 1 -- 172.21.15.31:0/2323208909 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4594 (secure 0 0 0) 0x7f7858056fd0 con 0x7f7864140a70 2023-10-06T02:25:21.069 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:25:21.069 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4949,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/178547366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":178547366},{"type":"v1","addr":"172.21.15.62:6835","nonce":178547366}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":5278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/2578831517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":2578831517},{"type":"v1","addr":"172.21.15.31:6836","nonce":2578831517}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5287,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3502585737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":3502585737},{"type":"v1","addr":"172.21.15.31:6837","nonce":3502585737}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:24:43.872314+0000","modified":"2023-10-06T02:24:47.404607+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5290":{"gid":5290,"name":"b","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6834/2358776861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2358776861},{"type":"v1","addr":"172.21.15.62:6834","nonce":2358776861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T02:25:21.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:21.070+0000 7f786359e700 1 -- 172.21.15.31:0/2323208909 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f7844067170 msgr2=0x7f7844069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:25:21.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:21.070+0000 7f786359e700 1 --2- 172.21.15.31:0/2323208909 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f7844067170 0x7f7844069620 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7854009d20 tx=0x7f78540091a0 comp rx=0 tx=0).stop 2023-10-06T02:25:21.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:21.070+0000 7f786359e700 1 -- 172.21.15.31:0/2323208909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7864140a70 msgr2=0x7f7864156560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:25:21.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:21.070+0000 7f786359e700 1 --2- 172.21.15.31:0/2323208909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7864140a70 0x7f7864156560 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f78640c7450 tx=0x7f7858016000 comp rx=0 tx=0).stop 2023-10-06T02:25:21.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:21.070+0000 7f786359e700 1 -- 172.21.15.31:0/2323208909 shutdown_connections 2023-10-06T02:25:21.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:21.070+0000 7f786359e700 1 --2- 172.21.15.31:0/2323208909 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7864140e40 0x7f78640aeed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:21.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:21.070+0000 7f786359e700 1 --2- 172.21.15.31:0/2323208909 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f7844067170 0x7f7844069620 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:21.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:21.070+0000 7f786359e700 1 --2- 172.21.15.31:0/2323208909 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f78640c8330 0x7f78640c3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:21.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:21.070+0000 7f786359e700 1 --2- 172.21.15.31:0/2323208909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7864140a70 0x7f7864156560 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:21.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:21.070+0000 7f786359e700 1 -- 172.21.15.31:0/2323208909 >> 172.21.15.31:0/2323208909 conn(0x7f786406d170 msgr2=0x7f78640cb6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:25:21.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:21.070+0000 7f786359e700 1 -- 172.21.15.31:0/2323208909 shutdown_connections 2023-10-06T02:25:21.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:21.071+0000 7f786359e700 1 -- 172.21.15.31:0/2323208909 wait complete. 2023-10-06T02:25:21.073 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 27 2023-10-06T02:25:21.089 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json session ls 2023-10-06T02:25:21.313 INFO:teuthology.orchestra.run.smithi062.stdout:[{"id":5637,"entity":{"name":{"type":"client","num":5637},"addr":{"type":"any","addr":"172.21.15.31:0","nonce":2580495381}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":11,"uptime":24.241637731000001,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":18.911162366019823,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5637 172.21.15.31:0/2580495381","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x000000000017ffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"736ac5ba72cb47af7d6f6666b1e100f7e20a20e2","ceph_version":"ceph version 17.2.6-1296-g736ac5ba (736ac5ba72cb47af7d6f6666b1e100f7e20a20e2) quincy (stable)","entity_id":"y","hostname":"smithi031","pid":"85292","root":"/"}}] 2023-10-06T02:25:21.313 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "736ac5ba72cb47af7d6f6666b1e100f7e20a20e2", "ceph_version": "ceph version 17.2.6-1296-g736ac5ba (736ac5ba72cb47af7d6f6666b1e100f7e20a20e2) quincy (stable)", "client_features": { "feature_bits": "0x000000000017ffff" }, "entity_id": "y", "hostname": "smithi031", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "85292", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.31:0", "nonce": 2580495381, "type": "any" }, "name": { "num": 5637, "type": "client" } }, "id": 5637, "inst": "client.5637 172.21.15.31:0/2580495381", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 11, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 18.911162366019823 }, "state": "open", "uptime": 24.241637731 } ] 2023-10-06T02:25:21.313 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-10-06T02:25:26.315 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:25:26.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:26.511+0000 7f191579d700 1 Processor -- start 2023-10-06T02:25:26.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:26.511+0000 7f191579d700 1 -- start start 2023-10-06T02:25:26.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:26.512+0000 7f191579d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f191006f3a0 0x7f191014fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:25:26.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:26.512+0000 7f191579d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f191006e470 0x7f19101501b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:25:26.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:26.512+0000 7f191579d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f191006a490 con 0x7f191006e0a0 2023-10-06T02:25:26.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:26.512+0000 7f191579d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19100c7230 con 0x7f191006f3a0 2023-10-06T02:25:26.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:26.512+0000 7f191579d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f191006b930 con 0x7f191006e470 2023-10-06T02:25:26.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:26.513+0000 7f190f7fe700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f191006f3a0 0x7f191014fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:25:26.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:26.513+0000 7f1914f9c700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f191006e470 0x7f19101501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:25:26.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:26.513+0000 7f190f7fe700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f191006f3a0 0x7f191014fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:51852/0 (socket says 172.21.15.31:51852) 2023-10-06T02:25:26.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:26.513+0000 7f190f7fe700 1 -- 172.21.15.31:0/2149720303 learned_addr learned my addr 172.21.15.31:0/2149720303 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:25:26.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:26.513+0000 7f190ffff700 1 --1- 172.21.15.31:0/2149720303 >> v1:172.21.15.62:6789/0 conn(0x7f191006e0a0 0x7f19100c6b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:36612/0 (socket says 172.21.15.31:36612) 2023-10-06T02:25:26.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:26.513+0000 7f190f7fe700 1 -- 172.21.15.31:0/2149720303 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f191006e470 msgr2=0x7f19101501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:25:26.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:26.513+0000 7f190f7fe700 1 --2- 172.21.15.31:0/2149720303 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f191006e470 0x7f19101501b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:26.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:26.513+0000 7f190f7fe700 1 -- 172.21.15.31:0/2149720303 >> v1:172.21.15.62:6789/0 conn(0x7f191006e0a0 legacy=0x7f19100c6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:25:26.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:26.513+0000 7f190f7fe700 1 -- 172.21.15.31:0/2149720303 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19100c7490 con 0x7f191006f3a0 2023-10-06T02:25:26.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:26.513+0000 7f1914f9c700 1 --2- 172.21.15.31:0/2149720303 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f191006e470 0x7f19101501b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:25:26.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:26.513+0000 7f190effd700 1 -- 172.21.15.31:0/2149720303 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2303634671 0 0) 0x7f191006a490 con 0x7f191006e0a0 2023-10-06T02:25:26.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:26.514+0000 7f190f7fe700 1 --2- 172.21.15.31:0/2149720303 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f191006f3a0 0x7f191014fc70 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1904009990 tx=0x7f1904009220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b553b17e3e1b0cf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:25:26.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:26.514+0000 7f190effd700 1 -- 172.21.15.31:0/2149720303 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1904009e80 con 0x7f191006f3a0 2023-10-06T02:25:26.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:26.514+0000 7f190effd700 1 -- 172.21.15.31:0/2149720303 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1904005c10 con 0x7f191006f3a0 2023-10-06T02:25:26.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:26.514+0000 7f190effd700 1 -- 172.21.15.31:0/2149720303 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f190400b650 con 0x7f191006f3a0 2023-10-06T02:25:26.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:26.514+0000 7f191579d700 1 -- 172.21.15.31:0/2149720303 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f191006f3a0 msgr2=0x7f191014fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:25:26.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:26.514+0000 7f191579d700 1 --2- 172.21.15.31:0/2149720303 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f191006f3a0 0x7f191014fc70 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1904009990 tx=0x7f1904009220 comp rx=0 tx=0).stop 2023-10-06T02:25:26.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:26.514+0000 7f191579d700 1 -- 172.21.15.31:0/2149720303 shutdown_connections 2023-10-06T02:25:26.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:26.514+0000 7f191579d700 1 --2- 172.21.15.31:0/2149720303 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f191006e470 0x7f19101501b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:26.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:26.514+0000 7f191579d700 1 --2- 172.21.15.31:0/2149720303 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f191006f3a0 0x7f191014fc70 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:26.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:26.514+0000 7f191579d700 1 -- 172.21.15.31:0/2149720303 >> 172.21.15.31:0/2149720303 conn(0x7f191006d170 msgr2=0x7f19100cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:25:26.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:26.514+0000 7f191579d700 1 -- 172.21.15.31:0/2149720303 shutdown_connections 2023-10-06T02:25:26.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:26.514+0000 7f191579d700 1 -- 172.21.15.31:0/2149720303 wait complete. 2023-10-06T02:25:26.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:26.516+0000 7f191579d700 1 Processor -- start 2023-10-06T02:25:26.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:26.516+0000 7f191579d700 1 -- start start 2023-10-06T02:25:26.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:26.517+0000 7f191579d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f191006e0a0 0x7f1910147f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:25:26.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:26.517+0000 7f191579d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f191006e470 0x7f19100cc960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:25:26.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:26.517+0000 7f191579d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f191006f3a0 0x7f1910154b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:25:26.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:26.517+0000 7f191579d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19101493d0 con 0x7f191006e470 2023-10-06T02:25:26.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:26.517+0000 7f191579d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19101484e0 con 0x7f191006e0a0 2023-10-06T02:25:26.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:26.517+0000 7f191579d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19101487c0 con 0x7f191006f3a0 2023-10-06T02:25:26.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:26.517+0000 7f190f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f191006e470 0x7f19100cc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:25:26.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:26.517+0000 7f190f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f191006e470 0x7f19100cc960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36398/0 (socket says 172.21.15.31:36398) 2023-10-06T02:25:26.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:26.517+0000 7f190f7fe700 1 -- 172.21.15.31:0/4236183977 learned_addr learned my addr 172.21.15.31:0/4236183977 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:25:26.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:26.517+0000 7f190ffff700 1 --2- 172.21.15.31:0/4236183977 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f191006e0a0 0x7f1910147f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:25:26.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:26.518+0000 7f1914f9c700 1 --2- 172.21.15.31:0/4236183977 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f191006f3a0 0x7f1910154b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:25:26.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:26.518+0000 7f190f7fe700 1 -- 172.21.15.31:0/4236183977 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f191006f3a0 msgr2=0x7f1910154b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:25:26.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:26.518+0000 7f190f7fe700 1 --2- 172.21.15.31:0/4236183977 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f191006f3a0 0x7f1910154b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:26.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:26.518+0000 7f190f7fe700 1 -- 172.21.15.31:0/4236183977 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f191006e0a0 msgr2=0x7f1910147f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:25:26.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:26.518+0000 7f190f7fe700 1 --2- 172.21.15.31:0/4236183977 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f191006e0a0 0x7f1910147f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:26.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:26.518+0000 7f190f7fe700 1 -- 172.21.15.31:0/4236183977 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f190800bc30 con 0x7f191006e470 2023-10-06T02:25:26.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:26.518+0000 7f190ffff700 1 --2- 172.21.15.31:0/4236183977 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f191006e0a0 0x7f1910147f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:25:26.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:26.518+0000 7f190f7fe700 1 --2- 172.21.15.31:0/4236183977 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f191006e470 0x7f19100cc960 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f190400b3c0 tx=0x7f1904016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:25:26.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:26.518+0000 7f18f7fff700 1 -- 172.21.15.31:0/4236183977 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1904011cf0 con 0x7f191006e470 2023-10-06T02:25:26.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:26.518+0000 7f18f7fff700 1 -- 172.21.15.31:0/4236183977 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1904011e50 con 0x7f191006e470 2023-10-06T02:25:26.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:26.518+0000 7f18f7fff700 1 -- 172.21.15.31:0/4236183977 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f190401b640 con 0x7f191006e470 2023-10-06T02:25:26.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:26.518+0000 7f191579d700 1 -- 172.21.15.31:0/4236183977 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1904009480 con 0x7f191006e470 2023-10-06T02:25:26.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:26.518+0000 7f191579d700 1 -- 172.21.15.31:0/4236183977 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1910148d80 con 0x7f191006e470 2023-10-06T02:25:26.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:26.519+0000 7f191579d700 1 -- 172.21.15.31:0/4236183977 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f191004c3c0 con 0x7f191006e470 2023-10-06T02:25:26.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:26.520+0000 7f18f7fff700 1 -- 172.21.15.31:0/4236183977 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f190401ba30 con 0x7f191006e470 2023-10-06T02:25:26.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:26.521+0000 7f18f7fff700 1 --2- 172.21.15.31:0/4236183977 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f18fc067170 0x7f18fc069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:25:26.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:26.521+0000 7f18f7fff700 1 -- 172.21.15.31:0/4236183977 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f19040864d0 con 0x7f191006e470 2023-10-06T02:25:26.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:26.521+0000 7f190ffff700 1 --2- 172.21.15.31:0/4236183977 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f18fc067170 0x7f18fc069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:25:26.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:26.522+0000 7f190ffff700 1 --2- 172.21.15.31:0/4236183977 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f18fc067170 0x7f18fc069620 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f19000011e0 tx=0x7f1900007c90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:25:26.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:26.525+0000 7f18f7fff700 1 -- 172.21.15.31:0/4236183977 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f19040532b0 con 0x7f191006e470 2023-10-06T02:25:26.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:26.770+0000 7f191579d700 1 -- 172.21.15.31:0/4236183977 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f19100c7770 con 0x7f191006e470 2023-10-06T02:25:26.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:26.771+0000 7f18f7fff700 1 -- 172.21.15.31:0/4236183977 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4594 (secure 0 0 0) 0x7f19040568d0 con 0x7f191006e470 2023-10-06T02:25:26.773 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:25:26.773 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4949,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/178547366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":178547366},{"type":"v1","addr":"172.21.15.62:6835","nonce":178547366}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":5278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/2578831517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":2578831517},{"type":"v1","addr":"172.21.15.31:6836","nonce":2578831517}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5287,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3502585737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":3502585737},{"type":"v1","addr":"172.21.15.31:6837","nonce":3502585737}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:24:43.872314+0000","modified":"2023-10-06T02:24:47.404607+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5290":{"gid":5290,"name":"b","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6834/2358776861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2358776861},{"type":"v1","addr":"172.21.15.62:6834","nonce":2358776861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T02:25:26.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:26.774+0000 7f191579d700 1 -- 172.21.15.31:0/4236183977 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f18fc067170 msgr2=0x7f18fc069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:25:26.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:26.774+0000 7f191579d700 1 --2- 172.21.15.31:0/4236183977 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f18fc067170 0x7f18fc069620 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f19000011e0 tx=0x7f1900007c90 comp rx=0 tx=0).stop 2023-10-06T02:25:26.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:26.774+0000 7f191579d700 1 -- 172.21.15.31:0/4236183977 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f191006e470 msgr2=0x7f19100cc960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:25:26.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:26.774+0000 7f191579d700 1 --2- 172.21.15.31:0/4236183977 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f191006e470 0x7f19100cc960 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f190400b3c0 tx=0x7f1904016000 comp rx=0 tx=0).stop 2023-10-06T02:25:26.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:26.774+0000 7f191579d700 1 -- 172.21.15.31:0/4236183977 shutdown_connections 2023-10-06T02:25:26.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:26.774+0000 7f191579d700 1 --2- 172.21.15.31:0/4236183977 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f18fc067170 0x7f18fc069620 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:26.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:26.774+0000 7f191579d700 1 --2- 172.21.15.31:0/4236183977 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f191006f3a0 0x7f1910154b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:26.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:26.774+0000 7f191579d700 1 --2- 172.21.15.31:0/4236183977 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f191006e0a0 0x7f1910147f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:26.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:26.774+0000 7f191579d700 1 --2- 172.21.15.31:0/4236183977 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f191006e470 0x7f19100cc960 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:26.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:26.774+0000 7f191579d700 1 -- 172.21.15.31:0/4236183977 >> 172.21.15.31:0/4236183977 conn(0x7f191006d170 msgr2=0x7f19100ca300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:25:26.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:26.774+0000 7f191579d700 1 -- 172.21.15.31:0/4236183977 shutdown_connections 2023-10-06T02:25:26.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:26.774+0000 7f191579d700 1 -- 172.21.15.31:0/4236183977 wait complete. 2023-10-06T02:25:26.778 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 27 2023-10-06T02:25:26.792 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json session ls 2023-10-06T02:25:27.013 INFO:teuthology.orchestra.run.smithi062.stdout:[{"id":5637,"entity":{"name":{"type":"client","num":5637},"addr":{"type":"any","addr":"172.21.15.31:0","nonce":2580495381}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":10,"uptime":29.941552543,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":18.663742714469446,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5637 172.21.15.31:0/2580495381","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x000000000017ffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"736ac5ba72cb47af7d6f6666b1e100f7e20a20e2","ceph_version":"ceph version 17.2.6-1296-g736ac5ba (736ac5ba72cb47af7d6f6666b1e100f7e20a20e2) quincy (stable)","entity_id":"y","hostname":"smithi031","pid":"85292","root":"/"}}] 2023-10-06T02:25:27.014 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "736ac5ba72cb47af7d6f6666b1e100f7e20a20e2", "ceph_version": "ceph version 17.2.6-1296-g736ac5ba (736ac5ba72cb47af7d6f6666b1e100f7e20a20e2) quincy (stable)", "client_features": { "feature_bits": "0x000000000017ffff" }, "entity_id": "y", "hostname": "smithi031", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "85292", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.31:0", "nonce": 2580495381, "type": "any" }, "name": { "num": 5637, "type": "client" } }, "id": 5637, "inst": "client.5637 172.21.15.31:0/2580495381", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 10, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 18.663742714469446 }, "state": "open", "uptime": 29.941552543 } ] 2023-10-06T02:25:27.014 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-10-06T02:25:30.758 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T02:25:30.763 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T02:25:32.015 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:25:32.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:32.213+0000 7fc698f9d700 1 Processor -- start 2023-10-06T02:25:32.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:32.213+0000 7fc698f9d700 1 -- start start 2023-10-06T02:25:32.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:32.214+0000 7fc698f9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc694140a70 0x7fc6940c6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:25:32.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:32.214+0000 7fc698f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc69406de50 0x7fc6940aeef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:25:32.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:32.214+0000 7fc698f9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6940c6d50 con 0x7fc694140e40 2023-10-06T02:25:32.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:32.214+0000 7fc698f9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6940c7030 con 0x7fc694140a70 2023-10-06T02:25:32.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:32.214+0000 7fc698f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6940c7310 con 0x7fc69406de50 2023-10-06T02:25:32.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:32.214+0000 7fc6937fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc694140a70 0x7fc6940c6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:25:32.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:32.214+0000 7fc6937fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc694140a70 0x7fc6940c6770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47492/0 (socket says 172.21.15.31:47492) 2023-10-06T02:25:32.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:32.214+0000 7fc6937fe700 1 -- 172.21.15.31:0/2876561417 learned_addr learned my addr 172.21.15.31:0/2876561417 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:25:32.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:32.214+0000 7fc693fff700 1 --2- 172.21.15.31:0/2876561417 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc69406de50 0x7fc6940aeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:25:32.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:32.215+0000 7fc6937fe700 1 -- 172.21.15.31:0/2876561417 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc69406de50 msgr2=0x7fc6940aeef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:25:32.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:32.215+0000 7fc6937fe700 1 --2- 172.21.15.31:0/2876561417 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc69406de50 0x7fc6940aeef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:32.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:32.215+0000 7fc6937fe700 1 -- 172.21.15.31:0/2876561417 >> v1:172.21.15.62:6789/0 conn(0x7fc694140e40 legacy=0x7fc6940c7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:25:32.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:32.215+0000 7fc6937fe700 1 -- 172.21.15.31:0/2876561417 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6940c75f0 con 0x7fc694140a70 2023-10-06T02:25:32.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:32.215+0000 7fc693fff700 1 --2- 172.21.15.31:0/2876561417 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc69406de50 0x7fc6940aeef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:25:32.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:32.215+0000 7fc6937fe700 1 --2- 172.21.15.31:0/2876561417 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc694140a70 0x7fc6940c6770 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fc6840099b0 tx=0x7fc684009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff5b0df7687b881c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:25:32.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:32.215+0000 7fc6927fc700 1 -- 172.21.15.31:0/2876561417 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc68400b820 con 0x7fc694140a70 2023-10-06T02:25:32.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:32.215+0000 7fc6927fc700 1 -- 172.21.15.31:0/2876561417 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc68400b980 con 0x7fc694140a70 2023-10-06T02:25:32.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:32.215+0000 7fc6927fc700 1 -- 172.21.15.31:0/2876561417 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc68400bd20 con 0x7fc694140a70 2023-10-06T02:25:32.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:32.216+0000 7fc698f9d700 1 -- 172.21.15.31:0/2876561417 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc694140a70 msgr2=0x7fc6940c6770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:25:32.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:32.216+0000 7fc698f9d700 1 --2- 172.21.15.31:0/2876561417 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc694140a70 0x7fc6940c6770 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fc6840099b0 tx=0x7fc684009250 comp rx=0 tx=0).stop 2023-10-06T02:25:32.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:32.216+0000 7fc698f9d700 1 -- 172.21.15.31:0/2876561417 shutdown_connections 2023-10-06T02:25:32.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:32.216+0000 7fc698f9d700 1 --2- 172.21.15.31:0/2876561417 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc69406de50 0x7fc6940aeef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:32.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:32.216+0000 7fc698f9d700 1 --2- 172.21.15.31:0/2876561417 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc694140a70 0x7fc6940c6770 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:32.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:32.216+0000 7fc698f9d700 1 -- 172.21.15.31:0/2876561417 >> 172.21.15.31:0/2876561417 conn(0x7fc69406d170 msgr2=0x7fc6940cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:25:32.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:32.216+0000 7fc698f9d700 1 -- 172.21.15.31:0/2876561417 shutdown_connections 2023-10-06T02:25:32.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:32.216+0000 7fc698f9d700 1 -- 172.21.15.31:0/2876561417 wait complete. 2023-10-06T02:25:32.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:32.217+0000 7fc698f9d700 1 Processor -- start 2023-10-06T02:25:32.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:32.217+0000 7fc698f9d700 1 -- start start 2023-10-06T02:25:32.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:32.218+0000 7fc698f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc69406de50 0x7fc694158aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:25:32.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:32.218+0000 7fc698f9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc694140a70 0x7fc69413e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:25:32.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:32.218+0000 7fc698f9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc694140e40 0x7fc69413ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:25:32.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:32.218+0000 7fc698f9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6941582b0 con 0x7fc694140e40 2023-10-06T02:25:32.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:32.218+0000 7fc698f9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc69416b0d0 con 0x7fc694140a70 2023-10-06T02:25:32.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:32.218+0000 7fc698f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc694159580 con 0x7fc69406de50 2023-10-06T02:25:32.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:32.219+0000 7fc693fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc694140e40 0x7fc69413ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:25:32.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:32.219+0000 7fc6937fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc69406de50 0x7fc694158aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:25:32.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:32.219+0000 7fc6937fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc69406de50 0x7fc694158aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:44962/0 (socket says 172.21.15.31:44962) 2023-10-06T02:25:32.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:32.219+0000 7fc6937fe700 1 -- 172.21.15.31:0/1706982548 learned_addr learned my addr 172.21.15.31:0/1706982548 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:25:32.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:32.219+0000 7fc693fff700 1 --2- 172.21.15.31:0/1706982548 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc694140e40 0x7fc69413ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47502/0 (socket says 172.21.15.31:47502) 2023-10-06T02:25:32.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:32.219+0000 7fc692ffd700 1 --2- 172.21.15.31:0/1706982548 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc694140a70 0x7fc69413e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:25:32.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:32.219+0000 7fc693fff700 1 -- 172.21.15.31:0/1706982548 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc69406de50 msgr2=0x7fc694158aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:25:32.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:32.219+0000 7fc693fff700 1 --2- 172.21.15.31:0/1706982548 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc69406de50 0x7fc694158aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:32.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:32.219+0000 7fc693fff700 1 -- 172.21.15.31:0/1706982548 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc694140a70 msgr2=0x7fc69413e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:25:32.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:32.219+0000 7fc693fff700 1 --2- 172.21.15.31:0/1706982548 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc694140a70 0x7fc69413e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:32.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:32.219+0000 7fc693fff700 1 -- 172.21.15.31:0/1706982548 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6840094f0 con 0x7fc694140e40 2023-10-06T02:25:32.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:32.219+0000 7fc693fff700 1 --2- 172.21.15.31:0/1706982548 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc694140e40 0x7fc69413ef30 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fc68c00cf30 tx=0x7fc68c007840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:25:32.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:32.220+0000 7fc67b7fe700 1 -- 172.21.15.31:0/1706982548 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc68c00ae40 con 0x7fc694140e40 2023-10-06T02:25:32.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:32.220+0000 7fc67b7fe700 1 -- 172.21.15.31:0/1706982548 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc68c015440 con 0x7fc694140e40 2023-10-06T02:25:32.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:32.220+0000 7fc698f9d700 1 -- 172.21.15.31:0/1706982548 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc694157980 con 0x7fc694140e40 2023-10-06T02:25:32.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:32.220+0000 7fc67b7fe700 1 -- 172.21.15.31:0/1706982548 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc68c015800 con 0x7fc694140e40 2023-10-06T02:25:32.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:32.220+0000 7fc698f9d700 1 -- 172.21.15.31:0/1706982548 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc694157f40 con 0x7fc694140e40 2023-10-06T02:25:32.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:32.221+0000 7fc698f9d700 1 -- 172.21.15.31:0/1706982548 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc694158420 con 0x7fc694140e40 2023-10-06T02:25:32.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:32.222+0000 7fc67b7fe700 1 -- 172.21.15.31:0/1706982548 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 84944+0+0 (secure 0 0 0) 0x7fc68c00d440 con 0x7fc694140e40 2023-10-06T02:25:32.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:32.222+0000 7fc67b7fe700 1 --2- 172.21.15.31:0/1706982548 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc680067170 0x7fc680069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:25:32.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:32.223+0000 7fc67b7fe700 1 -- 172.21.15.31:0/1706982548 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc68c0855e0 con 0x7fc694140e40 2023-10-06T02:25:32.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:32.223+0000 7fc6937fe700 1 --2- 172.21.15.31:0/1706982548 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc680067170 0x7fc680069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:25:32.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:32.223+0000 7fc6937fe700 1 --2- 172.21.15.31:0/1706982548 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc680067170 0x7fc680069620 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc684003080 tx=0x7fc684016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:25:32.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:32.227+0000 7fc67b7fe700 1 -- 172.21.15.31:0/1706982548 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc68c0523c0 con 0x7fc694140e40 2023-10-06T02:25:32.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:32.476+0000 7fc698f9d700 1 -- 172.21.15.31:0/1706982548 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc6940648e0 con 0x7fc694140e40 2023-10-06T02:25:32.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:32.477+0000 7fc67b7fe700 1 -- 172.21.15.31:0/1706982548 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4594 (secure 0 0 0) 0x7fc6940648e0 con 0x7fc694140e40 2023-10-06T02:25:32.478 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:25:32.478 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4949,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/178547366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":178547366},{"type":"v1","addr":"172.21.15.62:6835","nonce":178547366}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":5278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/2578831517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":2578831517},{"type":"v1","addr":"172.21.15.31:6836","nonce":2578831517}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5287,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3502585737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":3502585737},{"type":"v1","addr":"172.21.15.31:6837","nonce":3502585737}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:24:43.872314+0000","modified":"2023-10-06T02:24:47.404607+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5290":{"gid":5290,"name":"b","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6834/2358776861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2358776861},{"type":"v1","addr":"172.21.15.62:6834","nonce":2358776861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T02:25:32.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:32.479+0000 7fc698f9d700 1 -- 172.21.15.31:0/1706982548 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc680067170 msgr2=0x7fc680069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:25:32.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:32.479+0000 7fc698f9d700 1 --2- 172.21.15.31:0/1706982548 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc680067170 0x7fc680069620 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc684003080 tx=0x7fc684016000 comp rx=0 tx=0).stop 2023-10-06T02:25:32.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:32.479+0000 7fc698f9d700 1 -- 172.21.15.31:0/1706982548 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc694140e40 msgr2=0x7fc69413ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:25:32.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:32.479+0000 7fc698f9d700 1 --2- 172.21.15.31:0/1706982548 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc694140e40 0x7fc69413ef30 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fc68c00cf30 tx=0x7fc68c007840 comp rx=0 tx=0).stop 2023-10-06T02:25:32.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:32.479+0000 7fc698f9d700 1 -- 172.21.15.31:0/1706982548 shutdown_connections 2023-10-06T02:25:32.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:32.479+0000 7fc698f9d700 1 --2- 172.21.15.31:0/1706982548 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc694140e40 0x7fc69413ef30 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:32.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:32.479+0000 7fc698f9d700 1 --2- 172.21.15.31:0/1706982548 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc680067170 0x7fc680069620 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:32.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:32.479+0000 7fc698f9d700 1 --2- 172.21.15.31:0/1706982548 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc69406de50 0x7fc694158aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:32.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:32.479+0000 7fc698f9d700 1 --2- 172.21.15.31:0/1706982548 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc694140a70 0x7fc69413e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:32.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:32.479+0000 7fc698f9d700 1 -- 172.21.15.31:0/1706982548 >> 172.21.15.31:0/1706982548 conn(0x7fc69406d170 msgr2=0x7fc6940c1ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:25:32.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:32.479+0000 7fc698f9d700 1 -- 172.21.15.31:0/1706982548 shutdown_connections 2023-10-06T02:25:32.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:32.479+0000 7fc698f9d700 1 -- 172.21.15.31:0/1706982548 wait complete. 2023-10-06T02:25:32.481 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 27 2023-10-06T02:25:32.497 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json session ls 2023-10-06T02:25:32.716 INFO:teuthology.orchestra.run.smithi062.stdout:[{"id":5637,"entity":{"name":{"type":"client","num":5637},"addr":{"type":"any","addr":"172.21.15.31:0","nonce":2580495381}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":9,"uptime":35.643467325000003,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":18.41947500354788,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5637 172.21.15.31:0/2580495381","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x000000000017ffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"736ac5ba72cb47af7d6f6666b1e100f7e20a20e2","ceph_version":"ceph version 17.2.6-1296-g736ac5ba (736ac5ba72cb47af7d6f6666b1e100f7e20a20e2) quincy (stable)","entity_id":"y","hostname":"smithi031","pid":"85292","root":"/"}}] 2023-10-06T02:25:32.717 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "736ac5ba72cb47af7d6f6666b1e100f7e20a20e2", "ceph_version": "ceph version 17.2.6-1296-g736ac5ba (736ac5ba72cb47af7d6f6666b1e100f7e20a20e2) quincy (stable)", "client_features": { "feature_bits": "0x000000000017ffff" }, "entity_id": "y", "hostname": "smithi031", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "85292", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.31:0", "nonce": 2580495381, "type": "any" }, "name": { "num": 5637, "type": "client" } }, "id": 5637, "inst": "client.5637 172.21.15.31:0/2580495381", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 9, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 18.41947500354788 }, "state": "open", "uptime": 35.643467325 } ] 2023-10-06T02:25:32.717 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-10-06T02:25:37.718 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:25:37.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:37.916+0000 7f5452f9d700 1 Processor -- start 2023-10-06T02:25:37.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:37.916+0000 7f5452f9d700 1 -- start start 2023-10-06T02:25:37.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:37.918+0000 7f5452f9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f544c153c30 0x7f544c15a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:25:37.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:37.918+0000 7f5452f9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f544c15de50 0x7f544c0cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:25:37.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:37.918+0000 7f5452f9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f544c0cf3c0 con 0x7f544c0cef40 2023-10-06T02:25:37.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:37.918+0000 7f5452f9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f544c15afd0 con 0x7f544c15de50 2023-10-06T02:25:37.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:37.918+0000 7f5452f9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f544c15b2b0 con 0x7f544c153c30 2023-10-06T02:25:37.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:37.918+0000 7f545179a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f544c15de50 0x7f544c0cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:25:37.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:37.918+0000 7f545179a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f544c15de50 0x7f544c0cea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59514/0 (socket says 172.21.15.31:59514) 2023-10-06T02:25:37.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:37.918+0000 7f545179a700 1 -- 172.21.15.31:0/4237359372 learned_addr learned my addr 172.21.15.31:0/4237359372 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:25:37.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:37.918+0000 7f545279c700 1 --1- 172.21.15.31:0/4237359372 >> v1:172.21.15.31:6789/0 conn(0x7f544c0cef40 0x7f544c06d790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:50986/0 (socket says 172.21.15.31:50986) 2023-10-06T02:25:37.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:37.918+0000 7f5451f9b700 1 --2- 172.21.15.31:0/4237359372 >> v2:172.21.15.62:3300/0 conn(0x7f544c153c30 0x7f544c15a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:25:37.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:37.919+0000 7f545179a700 1 -- 172.21.15.31:0/4237359372 >> v2:172.21.15.62:3300/0 conn(0x7f544c153c30 msgr2=0x7f544c15a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:25:37.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:37.919+0000 7f545179a700 1 --2- 172.21.15.31:0/4237359372 >> v2:172.21.15.62:3300/0 conn(0x7f544c153c30 0x7f544c15a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:37.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:37.919+0000 7f545179a700 1 -- 172.21.15.31:0/4237359372 >> v1:172.21.15.31:6789/0 conn(0x7f544c0cef40 legacy=0x7f544c06d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:25:37.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:37.919+0000 7f545179a700 1 -- 172.21.15.31:0/4237359372 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f544c15b590 con 0x7f544c15de50 2023-10-06T02:25:37.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:37.919+0000 7f5451f9b700 1 --2- 172.21.15.31:0/4237359372 >> v2:172.21.15.62:3300/0 conn(0x7f544c153c30 0x7f544c15a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:25:37.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:37.919+0000 7f545179a700 1 --2- 172.21.15.31:0/4237359372 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f544c15de50 0x7f544c0cea00 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f544c06ead0 tx=0x7f543c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3de3cd35f5dcb664 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:25:37.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:37.919+0000 7f5450f99700 1 -- 172.21.15.31:0/4237359372 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f543c00b860 con 0x7f544c15de50 2023-10-06T02:25:37.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:37.919+0000 7f5450f99700 1 -- 172.21.15.31:0/4237359372 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f543c00b9c0 con 0x7f544c15de50 2023-10-06T02:25:37.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:37.919+0000 7f5450f99700 1 -- 172.21.15.31:0/4237359372 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f543c00bd60 con 0x7f544c15de50 2023-10-06T02:25:37.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:37.920+0000 7f5452f9d700 1 -- 172.21.15.31:0/4237359372 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f544c15de50 msgr2=0x7f544c0cea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:25:37.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:37.920+0000 7f5452f9d700 1 --2- 172.21.15.31:0/4237359372 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f544c15de50 0x7f544c0cea00 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f544c06ead0 tx=0x7f543c009250 comp rx=0 tx=0).stop 2023-10-06T02:25:37.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:37.920+0000 7f5452f9d700 1 -- 172.21.15.31:0/4237359372 shutdown_connections 2023-10-06T02:25:37.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:37.920+0000 7f5452f9d700 1 --2- 172.21.15.31:0/4237359372 >> v2:172.21.15.62:3300/0 conn(0x7f544c153c30 0x7f544c15a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:37.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:37.920+0000 7f5452f9d700 1 --2- 172.21.15.31:0/4237359372 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f544c15de50 0x7f544c0cea00 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:37.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:37.920+0000 7f5452f9d700 1 -- 172.21.15.31:0/4237359372 >> 172.21.15.31:0/4237359372 conn(0x7f544c0703a0 msgr2=0x7f544c0707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:25:37.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:37.920+0000 7f5452f9d700 1 -- 172.21.15.31:0/4237359372 shutdown_connections 2023-10-06T02:25:37.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:37.920+0000 7f5452f9d700 1 -- 172.21.15.31:0/4237359372 wait complete. 2023-10-06T02:25:37.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:37.921+0000 7f5452f9d700 1 Processor -- start 2023-10-06T02:25:37.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:37.921+0000 7f5452f9d700 1 -- start start 2023-10-06T02:25:37.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:37.922+0000 7f5452f9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f544c0cef40 0x7f544c0b48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:25:37.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:37.922+0000 7f5452f9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f544c153c30 0x7f544c0c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:25:37.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:37.922+0000 7f5452f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f544c15de50 0x7f544c13f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:25:37.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:37.922+0000 7f5452f9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f544c0b4450 con 0x7f544c0cef40 2023-10-06T02:25:37.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:37.922+0000 7f5452f9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f544c0b4e20 con 0x7f544c153c30 2023-10-06T02:25:37.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:37.922+0000 7f5452f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f544c0b3840 con 0x7f544c15de50 2023-10-06T02:25:37.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:37.922+0000 7f5451f9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f544c0cef40 0x7f544c0b48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:25:37.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:37.923+0000 7f5451f9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f544c0cef40 0x7f544c0b48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59530/0 (socket says 172.21.15.31:59530) 2023-10-06T02:25:37.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:37.923+0000 7f5451f9b700 1 -- 172.21.15.31:0/1007040535 learned_addr learned my addr 172.21.15.31:0/1007040535 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:25:37.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:37.923+0000 7f545179a700 1 --2- 172.21.15.31:0/1007040535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f544c153c30 0x7f544c0c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:25:37.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:37.923+0000 7f5451f9b700 1 -- 172.21.15.31:0/1007040535 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f544c15de50 msgr2=0x7f544c13f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:25:37.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:37.923+0000 7f5451f9b700 1 --2- 172.21.15.31:0/1007040535 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f544c15de50 0x7f544c13f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:37.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:37.923+0000 7f5451f9b700 1 -- 172.21.15.31:0/1007040535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f544c153c30 msgr2=0x7f544c0c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:25:37.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:37.923+0000 7f5451f9b700 1 --2- 172.21.15.31:0/1007040535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f544c153c30 0x7f544c0c9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:37.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:37.923+0000 7f5451f9b700 1 -- 172.21.15.31:0/1007040535 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f543c0094f0 con 0x7f544c0cef40 2023-10-06T02:25:37.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:37.923+0000 7f545179a700 1 --2- 172.21.15.31:0/1007040535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f544c153c30 0x7f544c0c9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:25:37.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:37.923+0000 7f5451f9b700 1 --2- 172.21.15.31:0/1007040535 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f544c0cef40 0x7f544c0b48e0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f544800d860 tx=0x7f5448008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:25:37.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:37.923+0000 7f5441ffb700 1 -- 172.21.15.31:0/1007040535 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54480125f0 con 0x7f544c0cef40 2023-10-06T02:25:37.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:37.923+0000 7f5441ffb700 1 -- 172.21.15.31:0/1007040535 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5448012750 con 0x7f544c0cef40 2023-10-06T02:25:37.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:37.923+0000 7f5452f9d700 1 -- 172.21.15.31:0/1007040535 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f544c0b3b20 con 0x7f544c0cef40 2023-10-06T02:25:37.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:37.923+0000 7f5441ffb700 1 -- 172.21.15.31:0/1007040535 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5448012ad0 con 0x7f544c0cef40 2023-10-06T02:25:37.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:37.924+0000 7f5452f9d700 1 -- 172.21.15.31:0/1007040535 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f544c0b40e0 con 0x7f544c0cef40 2023-10-06T02:25:37.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:37.925+0000 7f5452f9d700 1 -- 172.21.15.31:0/1007040535 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f544c04c3c0 con 0x7f544c0cef40 2023-10-06T02:25:37.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:37.925+0000 7f5441ffb700 1 -- 172.21.15.31:0/1007040535 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f5448014430 con 0x7f544c0cef40 2023-10-06T02:25:37.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:37.926+0000 7f5441ffb700 1 --2- 172.21.15.31:0/1007040535 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5430067170 0x7f5430069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:25:37.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:37.926+0000 7f5441ffb700 1 -- 172.21.15.31:0/1007040535 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f5448084eb0 con 0x7f544c0cef40 2023-10-06T02:25:37.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:37.927+0000 7f545179a700 1 --2- 172.21.15.31:0/1007040535 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5430067170 0x7f5430069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:25:37.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:37.927+0000 7f545179a700 1 --2- 172.21.15.31:0/1007040535 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5430067170 0x7f5430069620 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f544c06ead0 tx=0x7f543c017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:25:37.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:37.931+0000 7f5441ffb700 1 -- 172.21.15.31:0/1007040535 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5448055720 con 0x7f544c0cef40 2023-10-06T02:25:38.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:38.177+0000 7f5452f9d700 1 -- 172.21.15.31:0/1007040535 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f544c0b2920 con 0x7f544c0cef40 2023-10-06T02:25:38.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:38.178+0000 7f5441ffb700 1 -- 172.21.15.31:0/1007040535 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4594 (secure 0 0 0) 0x7f54480552b0 con 0x7f544c0cef40 2023-10-06T02:25:38.178 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:25:38.178 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4949,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/178547366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":178547366},{"type":"v1","addr":"172.21.15.62:6835","nonce":178547366}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":5278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/2578831517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":2578831517},{"type":"v1","addr":"172.21.15.31:6836","nonce":2578831517}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5287,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3502585737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":3502585737},{"type":"v1","addr":"172.21.15.31:6837","nonce":3502585737}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:24:43.872314+0000","modified":"2023-10-06T02:24:47.404607+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5290":{"gid":5290,"name":"b","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6834/2358776861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2358776861},{"type":"v1","addr":"172.21.15.62:6834","nonce":2358776861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T02:25:38.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:38.180+0000 7f5452f9d700 1 -- 172.21.15.31:0/1007040535 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5430067170 msgr2=0x7f5430069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:25:38.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:38.180+0000 7f5452f9d700 1 --2- 172.21.15.31:0/1007040535 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5430067170 0x7f5430069620 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f544c06ead0 tx=0x7f543c017000 comp rx=0 tx=0).stop 2023-10-06T02:25:38.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:38.180+0000 7f5452f9d700 1 -- 172.21.15.31:0/1007040535 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f544c0cef40 msgr2=0x7f544c0b48e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:25:38.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:38.180+0000 7f5452f9d700 1 --2- 172.21.15.31:0/1007040535 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f544c0cef40 0x7f544c0b48e0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f544800d860 tx=0x7f5448008040 comp rx=0 tx=0).stop 2023-10-06T02:25:38.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:38.180+0000 7f5452f9d700 1 -- 172.21.15.31:0/1007040535 shutdown_connections 2023-10-06T02:25:38.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:38.180+0000 7f5452f9d700 1 --2- 172.21.15.31:0/1007040535 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5430067170 0x7f5430069620 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:38.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:38.180+0000 7f5452f9d700 1 --2- 172.21.15.31:0/1007040535 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f544c15de50 0x7f544c13f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:38.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:38.180+0000 7f5452f9d700 1 --2- 172.21.15.31:0/1007040535 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f544c0cef40 0x7f544c0b48e0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:38.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:38.180+0000 7f5452f9d700 1 --2- 172.21.15.31:0/1007040535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f544c153c30 0x7f544c0c9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:38.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:38.180+0000 7f5452f9d700 1 -- 172.21.15.31:0/1007040535 >> 172.21.15.31:0/1007040535 conn(0x7f544c0703a0 msgr2=0x7f544c070770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:25:38.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:38.181+0000 7f5452f9d700 1 -- 172.21.15.31:0/1007040535 shutdown_connections 2023-10-06T02:25:38.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:38.181+0000 7f5452f9d700 1 -- 172.21.15.31:0/1007040535 wait complete. 2023-10-06T02:25:38.182 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 27 2023-10-06T02:25:38.199 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json session ls 2023-10-06T02:25:38.420 INFO:teuthology.orchestra.run.smithi062.stdout:[{"id":5637,"entity":{"name":{"type":"client","num":5637},"addr":{"type":"any","addr":"172.21.15.31:0","nonce":2580495381}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":9,"uptime":41.347382074999999,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":18.178320224007781,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5637 172.21.15.31:0/2580495381","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x000000000017ffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"736ac5ba72cb47af7d6f6666b1e100f7e20a20e2","ceph_version":"ceph version 17.2.6-1296-g736ac5ba (736ac5ba72cb47af7d6f6666b1e100f7e20a20e2) quincy (stable)","entity_id":"y","hostname":"smithi031","pid":"85292","root":"/"}}] 2023-10-06T02:25:38.420 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "736ac5ba72cb47af7d6f6666b1e100f7e20a20e2", "ceph_version": "ceph version 17.2.6-1296-g736ac5ba (736ac5ba72cb47af7d6f6666b1e100f7e20a20e2) quincy (stable)", "client_features": { "feature_bits": "0x000000000017ffff" }, "entity_id": "y", "hostname": "smithi031", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "85292", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.31:0", "nonce": 2580495381, "type": "any" }, "name": { "num": 5637, "type": "client" } }, "id": 5637, "inst": "client.5637 172.21.15.31:0/2580495381", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 9, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 18.17832022400778 }, "state": "open", "uptime": 41.347382075 } ] 2023-10-06T02:25:38.421 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-10-06T02:25:43.422 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:25:43.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:43.620+0000 7f87da99d700 1 Processor -- start 2023-10-06T02:25:43.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:43.620+0000 7f87da99d700 1 -- start start 2023-10-06T02:25:43.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:43.621+0000 7f87da99d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f87d4140a70 0x7f87d40c6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:25:43.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:43.621+0000 7f87da99d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f87d406de50 0x7f87d40aeef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:25:43.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:43.621+0000 7f87da99d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87d40c6d50 con 0x7f87d4140e40 2023-10-06T02:25:43.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:43.621+0000 7f87da99d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87d40c7030 con 0x7f87d4140a70 2023-10-06T02:25:43.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:43.621+0000 7f87da99d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87d40c7310 con 0x7f87d406de50 2023-10-06T02:25:43.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:43.622+0000 7f87d999b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f87d4140a70 0x7f87d40c6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:25:43.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:43.622+0000 7f87d999b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f87d4140a70 0x7f87d40c6770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59536/0 (socket says 172.21.15.31:59536) 2023-10-06T02:25:43.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:43.622+0000 7f87d999b700 1 -- 172.21.15.31:0/2995576573 learned_addr learned my addr 172.21.15.31:0/2995576573 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:25:43.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:43.622+0000 7f87d919a700 1 --1- 172.21.15.31:0/2995576573 >> v1:172.21.15.31:6789/0 conn(0x7f87d4140e40 0x7f87d40c7df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:50990/0 (socket says 172.21.15.31:50990) 2023-10-06T02:25:43.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:43.622+0000 7f87da19c700 1 --2- 172.21.15.31:0/2995576573 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f87d406de50 0x7f87d40aeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:25:43.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:43.622+0000 7f87d999b700 1 -- 172.21.15.31:0/2995576573 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f87d406de50 msgr2=0x7f87d40aeef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:25:43.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:43.622+0000 7f87d999b700 1 --2- 172.21.15.31:0/2995576573 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f87d406de50 0x7f87d40aeef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:43.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:43.622+0000 7f87d999b700 1 -- 172.21.15.31:0/2995576573 >> v1:172.21.15.31:6789/0 conn(0x7f87d4140e40 legacy=0x7f87d40c7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:25:43.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:43.622+0000 7f87d999b700 1 -- 172.21.15.31:0/2995576573 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87d40c75f0 con 0x7f87d4140a70 2023-10-06T02:25:43.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:43.622+0000 7f87d8999700 1 -- 172.21.15.31:0/2995576573 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1209887145 0 0) 0x7f87d40c6d50 con 0x7f87d4140e40 2023-10-06T02:25:43.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:43.622+0000 7f87d999b700 1 --2- 172.21.15.31:0/2995576573 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87d4140a70 0x7f87d40c6770 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f87d00099b0 tx=0x7f87d0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23097443773acd40 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:25:43.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:43.623+0000 7f87d8999700 1 -- 172.21.15.31:0/2995576573 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d000b860 con 0x7f87d4140a70 2023-10-06T02:25:43.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:43.623+0000 7f87d8999700 1 -- 172.21.15.31:0/2995576573 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87d000b9c0 con 0x7f87d4140a70 2023-10-06T02:25:43.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:43.623+0000 7f87d8999700 1 -- 172.21.15.31:0/2995576573 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d000bd60 con 0x7f87d4140a70 2023-10-06T02:25:43.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:43.623+0000 7f87da99d700 1 -- 172.21.15.31:0/2995576573 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87d4140a70 msgr2=0x7f87d40c6770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:25:43.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:43.623+0000 7f87da99d700 1 --2- 172.21.15.31:0/2995576573 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87d4140a70 0x7f87d40c6770 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f87d00099b0 tx=0x7f87d0009250 comp rx=0 tx=0).stop 2023-10-06T02:25:43.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:43.623+0000 7f87da99d700 1 -- 172.21.15.31:0/2995576573 shutdown_connections 2023-10-06T02:25:43.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:43.623+0000 7f87da99d700 1 --2- 172.21.15.31:0/2995576573 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87d4140a70 0x7f87d40c6770 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:43.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:43.623+0000 7f87da99d700 1 --2- 172.21.15.31:0/2995576573 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f87d406de50 0x7f87d40aeef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:43.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:43.623+0000 7f87da99d700 1 -- 172.21.15.31:0/2995576573 >> 172.21.15.31:0/2995576573 conn(0x7f87d406d170 msgr2=0x7f87d40cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:25:43.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:43.623+0000 7f87da99d700 1 -- 172.21.15.31:0/2995576573 shutdown_connections 2023-10-06T02:25:43.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:43.623+0000 7f87da99d700 1 -- 172.21.15.31:0/2995576573 wait complete. 2023-10-06T02:25:43.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:43.625+0000 7f87da99d700 1 Processor -- start 2023-10-06T02:25:43.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:43.625+0000 7f87da99d700 1 -- start start 2023-10-06T02:25:43.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:43.626+0000 7f87da99d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f87d406de50 0x7f87d4159700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:25:43.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:43.626+0000 7f87da99d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87d4140a70 0x7f87d413ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:25:43.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:43.626+0000 7f87da99d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87d4140e40 0x7f87d40c3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:25:43.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:43.626+0000 7f87da99d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87d40c6ad0 con 0x7f87d4140e40 2023-10-06T02:25:43.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:43.626+0000 7f87da99d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87d4159c40 con 0x7f87d4140a70 2023-10-06T02:25:43.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:43.626+0000 7f87da99d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87d4159e70 con 0x7f87d406de50 2023-10-06T02:25:43.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:43.626+0000 7f87da19c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87d4140e40 0x7f87d40c3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:25:43.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:43.626+0000 7f87da19c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87d4140e40 0x7f87d40c3ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59540/0 (socket says 172.21.15.31:59540) 2023-10-06T02:25:43.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:43.626+0000 7f87da19c700 1 -- 172.21.15.31:0/718854059 learned_addr learned my addr 172.21.15.31:0/718854059 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:25:43.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:43.626+0000 7f87d919a700 1 --2- 172.21.15.31:0/718854059 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87d4140a70 0x7f87d413ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:25:43.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:43.626+0000 7f87d999b700 1 --2- 172.21.15.31:0/718854059 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f87d406de50 0x7f87d4159700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:25:43.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:43.627+0000 7f87da19c700 1 -- 172.21.15.31:0/718854059 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f87d406de50 msgr2=0x7f87d4159700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:25:43.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:43.627+0000 7f87da19c700 1 --2- 172.21.15.31:0/718854059 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f87d406de50 0x7f87d4159700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:43.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:43.627+0000 7f87da19c700 1 -- 172.21.15.31:0/718854059 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87d4140a70 msgr2=0x7f87d413ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:25:43.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:43.627+0000 7f87da19c700 1 --2- 172.21.15.31:0/718854059 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87d4140a70 0x7f87d413ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:43.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:43.627+0000 7f87da19c700 1 -- 172.21.15.31:0/718854059 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87d00094f0 con 0x7f87d4140e40 2023-10-06T02:25:43.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:43.627+0000 7f87da19c700 1 --2- 172.21.15.31:0/718854059 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87d4140e40 0x7f87d40c3ae0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f87cc009fa0 tx=0x7f87cc009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:25:43.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:43.627+0000 7f87c9ffb700 1 -- 172.21.15.31:0/718854059 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87cc00ed20 con 0x7f87d4140e40 2023-10-06T02:25:43.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:43.627+0000 7f87c9ffb700 1 -- 172.21.15.31:0/718854059 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87cc00ee80 con 0x7f87d4140e40 2023-10-06T02:25:43.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:43.627+0000 7f87c9ffb700 1 -- 172.21.15.31:0/718854059 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87cc014670 con 0x7f87d4140e40 2023-10-06T02:25:43.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:43.627+0000 7f87da99d700 1 -- 172.21.15.31:0/718854059 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87d415a150 con 0x7f87d4140e40 2023-10-06T02:25:43.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:43.627+0000 7f87da99d700 1 -- 172.21.15.31:0/718854059 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87d415a710 con 0x7f87d4140e40 2023-10-06T02:25:43.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:43.628+0000 7f87da99d700 1 -- 172.21.15.31:0/718854059 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87d404c3c0 con 0x7f87d4140e40 2023-10-06T02:25:43.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:43.629+0000 7f87c9ffb700 1 -- 172.21.15.31:0/718854059 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f87cc0149e0 con 0x7f87d4140e40 2023-10-06T02:25:43.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:43.630+0000 7f87c9ffb700 1 --2- 172.21.15.31:0/718854059 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f87b8067170 0x7f87b8069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:25:43.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:43.630+0000 7f87c9ffb700 1 -- 172.21.15.31:0/718854059 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f87cc085250 con 0x7f87d4140e40 2023-10-06T02:25:43.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:43.630+0000 7f87d999b700 1 --2- 172.21.15.31:0/718854059 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f87b8067170 0x7f87b8069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:25:43.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:43.631+0000 7f87d999b700 1 --2- 172.21.15.31:0/718854059 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f87b8067170 0x7f87b8069620 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f87d00099b0 tx=0x7f87d0017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:25:43.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:43.634+0000 7f87c9ffb700 1 -- 172.21.15.31:0/718854059 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f87cc052030 con 0x7f87d4140e40 2023-10-06T02:25:43.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:43.882+0000 7f87da99d700 1 -- 172.21.15.31:0/718854059 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f87d40c78d0 con 0x7f87d4140e40 2023-10-06T02:25:43.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:43.882+0000 7f87c9ffb700 1 -- 172.21.15.31:0/718854059 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4594 (secure 0 0 0) 0x7f87cc055650 con 0x7f87d4140e40 2023-10-06T02:25:43.883 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:25:43.883 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4949,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/178547366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":178547366},{"type":"v1","addr":"172.21.15.62:6835","nonce":178547366}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":5278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/2578831517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":2578831517},{"type":"v1","addr":"172.21.15.31:6836","nonce":2578831517}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5287,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3502585737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":3502585737},{"type":"v1","addr":"172.21.15.31:6837","nonce":3502585737}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:24:43.872314+0000","modified":"2023-10-06T02:24:47.404607+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5290":{"gid":5290,"name":"b","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6834/2358776861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2358776861},{"type":"v1","addr":"172.21.15.62:6834","nonce":2358776861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T02:25:43.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:43.885+0000 7f87da99d700 1 -- 172.21.15.31:0/718854059 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f87b8067170 msgr2=0x7f87b8069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:25:43.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:43.885+0000 7f87da99d700 1 --2- 172.21.15.31:0/718854059 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f87b8067170 0x7f87b8069620 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f87d00099b0 tx=0x7f87d0017000 comp rx=0 tx=0).stop 2023-10-06T02:25:43.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:43.885+0000 7f87da99d700 1 -- 172.21.15.31:0/718854059 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87d4140e40 msgr2=0x7f87d40c3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:25:43.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:43.885+0000 7f87da99d700 1 --2- 172.21.15.31:0/718854059 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87d4140e40 0x7f87d40c3ae0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f87cc009fa0 tx=0x7f87cc009900 comp rx=0 tx=0).stop 2023-10-06T02:25:43.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:43.885+0000 7f87da99d700 1 -- 172.21.15.31:0/718854059 shutdown_connections 2023-10-06T02:25:43.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:43.885+0000 7f87da99d700 1 --2- 172.21.15.31:0/718854059 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87d4140e40 0x7f87d40c3ae0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:43.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:43.885+0000 7f87da99d700 1 --2- 172.21.15.31:0/718854059 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f87b8067170 0x7f87b8069620 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:43.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:43.885+0000 7f87da99d700 1 --2- 172.21.15.31:0/718854059 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f87d406de50 0x7f87d4159700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:43.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:43.885+0000 7f87da99d700 1 --2- 172.21.15.31:0/718854059 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87d4140a70 0x7f87d413ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:43.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:43.885+0000 7f87da99d700 1 -- 172.21.15.31:0/718854059 >> 172.21.15.31:0/718854059 conn(0x7f87d406d170 msgr2=0x7f87d413fb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:25:43.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:43.885+0000 7f87da99d700 1 -- 172.21.15.31:0/718854059 shutdown_connections 2023-10-06T02:25:43.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:43.885+0000 7f87da99d700 1 -- 172.21.15.31:0/718854059 wait complete. 2023-10-06T02:25:43.887 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 27 2023-10-06T02:25:43.903 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json session ls 2023-10-06T02:25:44.122 INFO:teuthology.orchestra.run.smithi062.stdout:[{"id":5637,"entity":{"name":{"type":"client","num":5637},"addr":{"type":"any","addr":"172.21.15.31:0","nonce":2580495381}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":8,"uptime":47.050296838999998,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":17.940364185070234,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5637 172.21.15.31:0/2580495381","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x000000000017ffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"736ac5ba72cb47af7d6f6666b1e100f7e20a20e2","ceph_version":"ceph version 17.2.6-1296-g736ac5ba (736ac5ba72cb47af7d6f6666b1e100f7e20a20e2) quincy (stable)","entity_id":"y","hostname":"smithi031","pid":"85292","root":"/"}}] 2023-10-06T02:25:44.123 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "736ac5ba72cb47af7d6f6666b1e100f7e20a20e2", "ceph_version": "ceph version 17.2.6-1296-g736ac5ba (736ac5ba72cb47af7d6f6666b1e100f7e20a20e2) quincy (stable)", "client_features": { "feature_bits": "0x000000000017ffff" }, "entity_id": "y", "hostname": "smithi031", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "85292", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.31:0", "nonce": 2580495381, "type": "any" }, "name": { "num": 5637, "type": "client" } }, "id": 5637, "inst": "client.5637 172.21.15.31:0/2580495381", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 8, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 17.940364185070234 }, "state": "open", "uptime": 47.050296839 } ] 2023-10-06T02:25:44.123 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-10-06T02:25:49.124 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:25:49.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:49.324+0000 7f3cedf9d700 1 Processor -- start 2023-10-06T02:25:49.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:49.324+0000 7f3cedf9d700 1 -- start start 2023-10-06T02:25:49.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:49.325+0000 7f3cedf9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3ce8153c30 0x7f3ce815a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:25:49.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:49.325+0000 7f3cedf9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3ce80cf110 0x7f3ce815c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:25:49.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:49.325+0000 7f3cedf9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ce815afd0 con 0x7f3ce815de50 2023-10-06T02:25:49.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:49.325+0000 7f3cedf9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ce815b2b0 con 0x7f3ce8153c30 2023-10-06T02:25:49.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:49.325+0000 7f3cedf9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ce815b590 con 0x7f3ce80cf110 2023-10-06T02:25:49.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:49.326+0000 7f3cecf9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3ce8153c30 0x7f3ce815a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:25:49.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:49.326+0000 7f3ce7fff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f3ce815de50 0x7f3ce80cea00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:32840/0 (socket says 172.21.15.31:32840) 2023-10-06T02:25:49.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:49.326+0000 7f3ce7fff700 1 -- 172.21.15.31:0/660813397 learned_addr learned my addr 172.21.15.31:0/660813397 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:25:49.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:49.326+0000 7f3ced79c700 1 --2- 172.21.15.31:0/660813397 >> v2:172.21.15.62:3300/0 conn(0x7f3ce80cf110 0x7f3ce815c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:25:49.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:49.326+0000 7f3ce77fe700 1 -- 172.21.15.31:0/660813397 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 879155339 0 0) 0x7f3ce815afd0 con 0x7f3ce815de50 2023-10-06T02:25:49.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:49.326+0000 7f3ce77fe700 1 -- 172.21.15.31:0/660813397 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3cd8003570 con 0x7f3ce815de50 2023-10-06T02:25:49.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:49.326+0000 7f3cecf9b700 1 -- 172.21.15.31:0/660813397 >> v2:172.21.15.62:3300/0 conn(0x7f3ce80cf110 msgr2=0x7f3ce815c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:25:49.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:49.326+0000 7f3cecf9b700 1 --2- 172.21.15.31:0/660813397 >> v2:172.21.15.62:3300/0 conn(0x7f3ce80cf110 0x7f3ce815c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:49.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:49.326+0000 7f3cecf9b700 1 -- 172.21.15.31:0/660813397 >> v1:172.21.15.31:6789/0 conn(0x7f3ce815de50 legacy=0x7f3ce80cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:25:49.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:49.326+0000 7f3cecf9b700 1 -- 172.21.15.31:0/660813397 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ce815b870 con 0x7f3ce8153c30 2023-10-06T02:25:49.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:49.326+0000 7f3cecf9b700 1 --2- 172.21.15.31:0/660813397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ce8153c30 0x7f3ce815a5e0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f3ce80cdab0 tx=0x7f3cdc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5966669d3e44e09 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:25:49.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:49.327+0000 7f3ce77fe700 1 -- 172.21.15.31:0/660813397 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cdc011630 con 0x7f3ce8153c30 2023-10-06T02:25:49.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:49.327+0000 7f3ce77fe700 1 -- 172.21.15.31:0/660813397 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cdc011790 con 0x7f3ce8153c30 2023-10-06T02:25:49.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:49.327+0000 7f3ce77fe700 1 -- 172.21.15.31:0/660813397 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cdc011b30 con 0x7f3ce8153c30 2023-10-06T02:25:49.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:49.327+0000 7f3cedf9d700 1 -- 172.21.15.31:0/660813397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ce8153c30 msgr2=0x7f3ce815a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:25:49.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:49.327+0000 7f3cedf9d700 1 --2- 172.21.15.31:0/660813397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ce8153c30 0x7f3ce815a5e0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f3ce80cdab0 tx=0x7f3cdc0091a0 comp rx=0 tx=0).stop 2023-10-06T02:25:49.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:49.327+0000 7f3cedf9d700 1 -- 172.21.15.31:0/660813397 shutdown_connections 2023-10-06T02:25:49.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:49.327+0000 7f3cedf9d700 1 --2- 172.21.15.31:0/660813397 >> v2:172.21.15.62:3300/0 conn(0x7f3ce80cf110 0x7f3ce815c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:49.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:49.327+0000 7f3cedf9d700 1 --2- 172.21.15.31:0/660813397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ce8153c30 0x7f3ce815a5e0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:49.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:49.327+0000 7f3cedf9d700 1 -- 172.21.15.31:0/660813397 >> 172.21.15.31:0/660813397 conn(0x7f3ce80703a0 msgr2=0x7f3ce80707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:25:49.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:49.327+0000 7f3cedf9d700 1 -- 172.21.15.31:0/660813397 shutdown_connections 2023-10-06T02:25:49.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:49.327+0000 7f3cedf9d700 1 -- 172.21.15.31:0/660813397 wait complete. 2023-10-06T02:25:49.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:49.329+0000 7f3cedf9d700 1 Processor -- start 2023-10-06T02:25:49.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:49.329+0000 7f3cedf9d700 1 -- start start 2023-10-06T02:25:49.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:49.330+0000 7f3cedf9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ce80cf110 0x7f3ce8158ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:25:49.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:49.330+0000 7f3cedf9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ce8153c30 0x7f3ce80b4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:25:49.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:49.330+0000 7f3cedf9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3ce815de50 0x7f3ce80b5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:25:49.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:49.330+0000 7f3cedf9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce816d660 con 0x7f3ce8153c30 2023-10-06T02:25:49.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:49.330+0000 7f3cedf9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce816ca70 con 0x7f3ce80cf110 2023-10-06T02:25:49.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:49.330+0000 7f3cedf9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce816cd30 con 0x7f3ce815de50 2023-10-06T02:25:49.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:49.330+0000 7f3ce7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ce8153c30 0x7f3ce80b4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:25:49.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:49.330+0000 7f3ce7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ce8153c30 0x7f3ce80b4cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35376/0 (socket says 172.21.15.31:35376) 2023-10-06T02:25:49.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:49.330+0000 7f3ce7fff700 1 -- 172.21.15.31:0/1039733069 learned_addr learned my addr 172.21.15.31:0/1039733069 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:25:49.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:49.330+0000 7f3cecf9b700 1 --2- 172.21.15.31:0/1039733069 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ce80cf110 0x7f3ce8158ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:25:49.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:49.330+0000 7f3ced79c700 1 --2- 172.21.15.31:0/1039733069 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3ce815de50 0x7f3ce80b5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:25:49.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:49.330+0000 7f3ce7fff700 1 -- 172.21.15.31:0/1039733069 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3ce815de50 msgr2=0x7f3ce80b5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:25:49.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:49.330+0000 7f3ce7fff700 1 --2- 172.21.15.31:0/1039733069 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3ce815de50 0x7f3ce80b5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:49.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:49.330+0000 7f3ce7fff700 1 -- 172.21.15.31:0/1039733069 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ce80cf110 msgr2=0x7f3ce8158ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:25:49.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:49.330+0000 7f3ce7fff700 1 --2- 172.21.15.31:0/1039733069 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ce80cf110 0x7f3ce8158ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:49.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:49.330+0000 7f3ce7fff700 1 -- 172.21.15.31:0/1039733069 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cdc009420 con 0x7f3ce8153c30 2023-10-06T02:25:49.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:49.331+0000 7f3ced79c700 1 --2- 172.21.15.31:0/1039733069 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3ce815de50 0x7f3ce80b5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:25:49.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:49.331+0000 7f3ce7fff700 1 --2- 172.21.15.31:0/1039733069 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ce8153c30 0x7f3ce80b4cc0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f3cd400c590 tx=0x7f3cd4007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:25:49.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:49.331+0000 7f3ce4ff9700 1 -- 172.21.15.31:0/1039733069 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd4005b30 con 0x7f3ce8153c30 2023-10-06T02:25:49.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:49.331+0000 7f3ce4ff9700 1 -- 172.21.15.31:0/1039733069 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cd4005cb0 con 0x7f3ce8153c30 2023-10-06T02:25:49.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:49.331+0000 7f3ce4ff9700 1 -- 172.21.15.31:0/1039733069 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd4007320 con 0x7f3ce8153c30 2023-10-06T02:25:49.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:49.331+0000 7f3cedf9d700 1 -- 172.21.15.31:0/1039733069 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ce816d010 con 0x7f3ce8153c30 2023-10-06T02:25:49.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:49.331+0000 7f3cedf9d700 1 -- 172.21.15.31:0/1039733069 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ce81591e0 con 0x7f3ce8153c30 2023-10-06T02:25:49.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:49.332+0000 7f3cedf9d700 1 -- 172.21.15.31:0/1039733069 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ce816d810 con 0x7f3ce8153c30 2023-10-06T02:25:49.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:49.333+0000 7f3ce4ff9700 1 -- 172.21.15.31:0/1039733069 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f3cd401b3b0 con 0x7f3ce8153c30 2023-10-06T02:25:49.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:49.334+0000 7f3ce4ff9700 1 --2- 172.21.15.31:0/1039733069 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f3ccc067170 0x7f3ccc069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:25:49.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:49.334+0000 7f3ce4ff9700 1 -- 172.21.15.31:0/1039733069 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f3cd4084120 con 0x7f3ce8153c30 2023-10-06T02:25:49.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:49.334+0000 7f3cecf9b700 1 --2- 172.21.15.31:0/1039733069 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f3ccc067170 0x7f3ccc069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:25:49.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:49.335+0000 7f3cecf9b700 1 --2- 172.21.15.31:0/1039733069 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f3ccc067170 0x7f3ccc069620 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc0093f0 tx=0x7f3cdc017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:25:49.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:49.338+0000 7f3ce4ff9700 1 -- 172.21.15.31:0/1039733069 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3ce816d810 con 0x7f3ce8153c30 2023-10-06T02:25:49.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:49.584+0000 7f3cedf9d700 1 -- 172.21.15.31:0/1039733069 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3ce8159400 con 0x7f3ce8153c30 2023-10-06T02:25:49.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:49.585+0000 7f3ce4ff9700 1 -- 172.21.15.31:0/1039733069 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4594 (secure 0 0 0) 0x7f3cd4050df0 con 0x7f3ce8153c30 2023-10-06T02:25:49.585 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:25:49.585 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4949,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/178547366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":178547366},{"type":"v1","addr":"172.21.15.62:6835","nonce":178547366}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":5278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/2578831517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":2578831517},{"type":"v1","addr":"172.21.15.31:6836","nonce":2578831517}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5287,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3502585737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":3502585737},{"type":"v1","addr":"172.21.15.31:6837","nonce":3502585737}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:24:43.872314+0000","modified":"2023-10-06T02:24:47.404607+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5290":{"gid":5290,"name":"b","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6834/2358776861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2358776861},{"type":"v1","addr":"172.21.15.62:6834","nonce":2358776861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T02:25:49.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:49.587+0000 7f3cedf9d700 1 -- 172.21.15.31:0/1039733069 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f3ccc067170 msgr2=0x7f3ccc069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:25:49.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:49.587+0000 7f3cedf9d700 1 --2- 172.21.15.31:0/1039733069 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f3ccc067170 0x7f3ccc069620 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc0093f0 tx=0x7f3cdc017000 comp rx=0 tx=0).stop 2023-10-06T02:25:49.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:49.587+0000 7f3cedf9d700 1 -- 172.21.15.31:0/1039733069 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ce8153c30 msgr2=0x7f3ce80b4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:25:49.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:49.587+0000 7f3cedf9d700 1 --2- 172.21.15.31:0/1039733069 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ce8153c30 0x7f3ce80b4cc0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f3cd400c590 tx=0x7f3cd4007e00 comp rx=0 tx=0).stop 2023-10-06T02:25:49.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:49.587+0000 7f3cedf9d700 1 -- 172.21.15.31:0/1039733069 shutdown_connections 2023-10-06T02:25:49.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:49.587+0000 7f3cedf9d700 1 --2- 172.21.15.31:0/1039733069 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f3ccc067170 0x7f3ccc069620 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:49.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:49.587+0000 7f3cedf9d700 1 --2- 172.21.15.31:0/1039733069 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3ce815de50 0x7f3ce80b5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:49.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:49.587+0000 7f3cedf9d700 1 --2- 172.21.15.31:0/1039733069 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ce80cf110 0x7f3ce8158ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:49.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:49.587+0000 7f3cedf9d700 1 --2- 172.21.15.31:0/1039733069 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ce8153c30 0x7f3ce80b4cc0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:49.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:49.587+0000 7f3cedf9d700 1 -- 172.21.15.31:0/1039733069 >> 172.21.15.31:0/1039733069 conn(0x7f3ce80703a0 msgr2=0x7f3ce8152e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:25:49.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:49.587+0000 7f3cedf9d700 1 -- 172.21.15.31:0/1039733069 shutdown_connections 2023-10-06T02:25:49.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:49.587+0000 7f3cedf9d700 1 -- 172.21.15.31:0/1039733069 wait complete. 2023-10-06T02:25:49.589 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 27 2023-10-06T02:25:49.605 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json session ls 2023-10-06T02:25:49.826 INFO:teuthology.orchestra.run.smithi062.stdout:[{"id":5637,"entity":{"name":{"type":"client","num":5637},"addr":{"type":"any","addr":"172.21.15.31:0","nonce":2580495381}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":8,"uptime":52.754211589999997,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":17.705482106735218,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5637 172.21.15.31:0/2580495381","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x000000000017ffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"736ac5ba72cb47af7d6f6666b1e100f7e20a20e2","ceph_version":"ceph version 17.2.6-1296-g736ac5ba (736ac5ba72cb47af7d6f6666b1e100f7e20a20e2) quincy (stable)","entity_id":"y","hostname":"smithi031","pid":"85292","root":"/"}}] 2023-10-06T02:25:49.827 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "736ac5ba72cb47af7d6f6666b1e100f7e20a20e2", "ceph_version": "ceph version 17.2.6-1296-g736ac5ba (736ac5ba72cb47af7d6f6666b1e100f7e20a20e2) quincy (stable)", "client_features": { "feature_bits": "0x000000000017ffff" }, "entity_id": "y", "hostname": "smithi031", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "85292", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.31:0", "nonce": 2580495381, "type": "any" }, "name": { "num": 5637, "type": "client" } }, "id": 5637, "inst": "client.5637 172.21.15.31:0/2580495381", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 8, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 17.705482106735218 }, "state": "open", "uptime": 52.75421159 } ] 2023-10-06T02:25:49.828 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-10-06T02:25:54.829 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:25:55.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:55.028+0000 7f3a3139d700 1 Processor -- start 2023-10-06T02:25:55.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:55.029+0000 7f3a3139d700 1 -- start start 2023-10-06T02:25:55.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:55.030+0000 7f3a3139d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3a2c159910 0x7f3a2c0677b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:25:55.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:55.030+0000 7f3a3139d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a2c068250 con 0x7f3a2c159540 2023-10-06T02:25:55.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:55.030+0000 7f3a3139d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a2c068530 con 0x7f3a2c067cf0 2023-10-06T02:25:55.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:55.030+0000 7f3a3139d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a2c068810 con 0x7f3a2c159910 2023-10-06T02:25:55.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:55.030+0000 7f3a2bfff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f3a2c159540 0x7f3a2c0c1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:32854/0 (socket says 172.21.15.31:32854) 2023-10-06T02:25:55.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:55.030+0000 7f3a2bfff700 1 -- 172.21.15.31:0/4231861657 learned_addr learned my addr 172.21.15.31:0/4231861657 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:25:55.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:55.030+0000 7f3a2b7fe700 1 --2- 172.21.15.31:0/4231861657 >> v2:172.21.15.31:3300/0 conn(0x7f3a2c159910 0x7f3a2c0677b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:25:55.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:55.031+0000 7f3a2affd700 1 -- 172.21.15.31:0/4231861657 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 110783453 0 0) 0x7f3a2c068250 con 0x7f3a2c159540 2023-10-06T02:25:55.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:55.031+0000 7f3a2affd700 1 -- 172.21.15.31:0/4231861657 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a14003570 con 0x7f3a2c159540 2023-10-06T02:25:55.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:55.031+0000 7f3a2b7fe700 1 -- 172.21.15.31:0/4231861657 >> v1:172.21.15.62:6789/0 conn(0x7f3a2c067cf0 legacy=0x7f3a2c0c6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:25:55.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:55.031+0000 7f3a2b7fe700 1 -- 172.21.15.31:0/4231861657 >> v1:172.21.15.31:6789/0 conn(0x7f3a2c159540 legacy=0x7f3a2c0c1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:25:55.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:55.031+0000 7f3a2b7fe700 1 -- 172.21.15.31:0/4231861657 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a2c0a9ca0 con 0x7f3a2c159910 2023-10-06T02:25:55.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:55.031+0000 7f3a2b7fe700 1 --2- 172.21.15.31:0/4231861657 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a2c159910 0x7f3a2c0677b0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f3a200098e0 tx=0x7f3a200091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec1454cb254827f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:25:55.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:55.031+0000 7f3a2affd700 1 -- 172.21.15.31:0/4231861657 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a2000b640 con 0x7f3a2c159910 2023-10-06T02:25:55.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:55.031+0000 7f3a2affd700 1 -- 172.21.15.31:0/4231861657 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a2000b7a0 con 0x7f3a2c159910 2023-10-06T02:25:55.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:55.031+0000 7f3a2affd700 1 -- 172.21.15.31:0/4231861657 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a2000bb40 con 0x7f3a2c159910 2023-10-06T02:25:55.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:55.032+0000 7f3a3139d700 1 -- 172.21.15.31:0/4231861657 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a2c159910 msgr2=0x7f3a2c0677b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:25:55.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:55.032+0000 7f3a3139d700 1 --2- 172.21.15.31:0/4231861657 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a2c159910 0x7f3a2c0677b0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f3a200098e0 tx=0x7f3a200091a0 comp rx=0 tx=0).stop 2023-10-06T02:25:55.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:55.032+0000 7f3a3139d700 1 -- 172.21.15.31:0/4231861657 shutdown_connections 2023-10-06T02:25:55.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:55.032+0000 7f3a3139d700 1 --2- 172.21.15.31:0/4231861657 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a2c159910 0x7f3a2c0677b0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:55.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:55.032+0000 7f3a3139d700 1 -- 172.21.15.31:0/4231861657 >> 172.21.15.31:0/4231861657 conn(0x7f3a2c06f310 msgr2=0x7f3a2c0cc5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:25:55.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:55.032+0000 7f3a3139d700 1 -- 172.21.15.31:0/4231861657 shutdown_connections 2023-10-06T02:25:55.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:55.032+0000 7f3a3139d700 1 -- 172.21.15.31:0/4231861657 wait complete. 2023-10-06T02:25:55.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:55.033+0000 7f3a3139d700 1 Processor -- start 2023-10-06T02:25:55.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:55.034+0000 7f3a3139d700 1 -- start start 2023-10-06T02:25:55.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:55.034+0000 7f3a3139d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a2c067cf0 0x7f3a2c151f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:25:55.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:55.034+0000 7f3a3139d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3a2c159540 0x7f3a2c14db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:25:55.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:55.034+0000 7f3a3139d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a2c159910 0x7f3a2c0b1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:25:55.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:55.034+0000 7f3a3139d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2c150d00 con 0x7f3a2c159910 2023-10-06T02:25:55.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:55.034+0000 7f3a3139d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2c1524d0 con 0x7f3a2c067cf0 2023-10-06T02:25:55.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:55.034+0000 7f3a3139d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2c14e050 con 0x7f3a2c159540 2023-10-06T02:25:55.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:55.035+0000 7f3a30b9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a2c159910 0x7f3a2c0b1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:25:55.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:55.035+0000 7f3a30b9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a2c159910 0x7f3a2c0b1cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35390/0 (socket says 172.21.15.31:35390) 2023-10-06T02:25:55.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:55.035+0000 7f3a30b9c700 1 -- 172.21.15.31:0/1621410985 learned_addr learned my addr 172.21.15.31:0/1621410985 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:25:55.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:55.035+0000 7f3a2bfff700 1 --2- 172.21.15.31:0/1621410985 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a2c067cf0 0x7f3a2c151f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:25:55.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:55.035+0000 7f3a2b7fe700 1 --2- 172.21.15.31:0/1621410985 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3a2c159540 0x7f3a2c14db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:25:55.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:55.035+0000 7f3a30b9c700 1 -- 172.21.15.31:0/1621410985 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3a2c159540 msgr2=0x7f3a2c14db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:25:55.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:55.035+0000 7f3a30b9c700 1 --2- 172.21.15.31:0/1621410985 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3a2c159540 0x7f3a2c14db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:55.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:55.035+0000 7f3a30b9c700 1 -- 172.21.15.31:0/1621410985 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a2c067cf0 msgr2=0x7f3a2c151f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:25:55.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:55.035+0000 7f3a30b9c700 1 --2- 172.21.15.31:0/1621410985 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a2c067cf0 0x7f3a2c151f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:55.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:55.035+0000 7f3a30b9c700 1 -- 172.21.15.31:0/1621410985 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a20009420 con 0x7f3a2c159910 2023-10-06T02:25:55.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:55.035+0000 7f3a2bfff700 1 --2- 172.21.15.31:0/1621410985 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a2c067cf0 0x7f3a2c151f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:25:55.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:55.035+0000 7f3a30b9c700 1 --2- 172.21.15.31:0/1621410985 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a2c159910 0x7f3a2c0b1cd0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f3a2400c8e0 tx=0x7f3a24001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:25:55.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:55.036+0000 7f3a13fff700 1 -- 172.21.15.31:0/1621410985 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a24012630 con 0x7f3a2c159910 2023-10-06T02:25:55.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:55.036+0000 7f3a13fff700 1 -- 172.21.15.31:0/1621410985 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a240127b0 con 0x7f3a2c159910 2023-10-06T02:25:55.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:55.036+0000 7f3a13fff700 1 -- 172.21.15.31:0/1621410985 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a24012b70 con 0x7f3a2c159910 2023-10-06T02:25:55.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:55.036+0000 7f3a3139d700 1 -- 172.21.15.31:0/1621410985 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a2c152c80 con 0x7f3a2c159910 2023-10-06T02:25:55.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:55.036+0000 7f3a3139d700 1 -- 172.21.15.31:0/1621410985 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a2c1526c0 con 0x7f3a2c159910 2023-10-06T02:25:55.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:55.037+0000 7f3a3139d700 1 -- 172.21.15.31:0/1621410985 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a2c04c3c0 con 0x7f3a2c159910 2023-10-06T02:25:55.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:55.038+0000 7f3a13fff700 1 -- 172.21.15.31:0/1621410985 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f3a24003030 con 0x7f3a2c159910 2023-10-06T02:25:55.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:55.038+0000 7f3a13fff700 1 --2- 172.21.15.31:0/1621410985 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f3a18067170 0x7f3a18069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:25:55.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:55.038+0000 7f3a13fff700 1 -- 172.21.15.31:0/1621410985 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f3a240849d0 con 0x7f3a2c159910 2023-10-06T02:25:55.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:55.039+0000 7f3a2bfff700 1 --2- 172.21.15.31:0/1621410985 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f3a18067170 0x7f3a18069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:25:55.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:55.039+0000 7f3a2bfff700 1 --2- 172.21.15.31:0/1621410985 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f3a18067170 0x7f3a18069620 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c0058d0 tx=0x7f3a1c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:25:55.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:55.043+0000 7f3a13fff700 1 -- 172.21.15.31:0/1621410985 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3a24051770 con 0x7f3a2c159910 2023-10-06T02:25:55.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:55.290+0000 7f3a3139d700 1 -- 172.21.15.31:0/1621410985 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3a2c155750 con 0x7f3a2c159910 2023-10-06T02:25:55.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:55.291+0000 7f3a13fff700 1 -- 172.21.15.31:0/1621410985 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4594 (secure 0 0 0) 0x7f3a24054d90 con 0x7f3a2c159910 2023-10-06T02:25:55.291 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:25:55.291 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4949,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/178547366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":178547366},{"type":"v1","addr":"172.21.15.62:6835","nonce":178547366}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":5278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/2578831517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":2578831517},{"type":"v1","addr":"172.21.15.31:6836","nonce":2578831517}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5287,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3502585737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":3502585737},{"type":"v1","addr":"172.21.15.31:6837","nonce":3502585737}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:24:43.872314+0000","modified":"2023-10-06T02:24:47.404607+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5290":{"gid":5290,"name":"b","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6834/2358776861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2358776861},{"type":"v1","addr":"172.21.15.62:6834","nonce":2358776861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T02:25:55.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:55.293+0000 7f3a3139d700 1 -- 172.21.15.31:0/1621410985 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f3a18067170 msgr2=0x7f3a18069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:25:55.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:55.293+0000 7f3a3139d700 1 --2- 172.21.15.31:0/1621410985 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f3a18067170 0x7f3a18069620 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c0058d0 tx=0x7f3a1c009040 comp rx=0 tx=0).stop 2023-10-06T02:25:55.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:55.293+0000 7f3a3139d700 1 -- 172.21.15.31:0/1621410985 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a2c159910 msgr2=0x7f3a2c0b1cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:25:55.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:55.293+0000 7f3a3139d700 1 --2- 172.21.15.31:0/1621410985 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a2c159910 0x7f3a2c0b1cd0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f3a2400c8e0 tx=0x7f3a24001710 comp rx=0 tx=0).stop 2023-10-06T02:25:55.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:55.293+0000 7f3a3139d700 1 -- 172.21.15.31:0/1621410985 shutdown_connections 2023-10-06T02:25:55.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:55.293+0000 7f3a3139d700 1 --2- 172.21.15.31:0/1621410985 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a2c159910 0x7f3a2c0b1cd0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:55.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:55.293+0000 7f3a3139d700 1 --2- 172.21.15.31:0/1621410985 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a2c067cf0 0x7f3a2c151f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:55.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:55.293+0000 7f3a3139d700 1 --2- 172.21.15.31:0/1621410985 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f3a18067170 0x7f3a18069620 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:55.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:55.293+0000 7f3a3139d700 1 --2- 172.21.15.31:0/1621410985 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3a2c159540 0x7f3a2c14db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:25:55.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:55.293+0000 7f3a3139d700 1 -- 172.21.15.31:0/1621410985 >> 172.21.15.31:0/1621410985 conn(0x7f3a2c06f310 msgr2=0x7f3a2c14c130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:25:55.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:55.293+0000 7f3a3139d700 1 -- 172.21.15.31:0/1621410985 shutdown_connections 2023-10-06T02:25:55.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:25:55.293+0000 7f3a3139d700 1 -- 172.21.15.31:0/1621410985 wait complete. 2023-10-06T02:25:55.295 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 27 2023-10-06T02:25:55.311 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json session ls 2023-10-06T02:25:55.533 INFO:teuthology.orchestra.run.smithi062.stdout:[{"id":5637,"entity":{"name":{"type":"client","num":5637},"addr":{"type":"any","addr":"172.21.15.31:0","nonce":2580495381}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":7,"uptime":58.46012631,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":17.473594449909751,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5637 172.21.15.31:0/2580495381","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x000000000017ffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"736ac5ba72cb47af7d6f6666b1e100f7e20a20e2","ceph_version":"ceph version 17.2.6-1296-g736ac5ba (736ac5ba72cb47af7d6f6666b1e100f7e20a20e2) quincy (stable)","entity_id":"y","hostname":"smithi031","pid":"85292","root":"/"}}] 2023-10-06T02:25:55.533 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "736ac5ba72cb47af7d6f6666b1e100f7e20a20e2", "ceph_version": "ceph version 17.2.6-1296-g736ac5ba (736ac5ba72cb47af7d6f6666b1e100f7e20a20e2) quincy (stable)", "client_features": { "feature_bits": "0x000000000017ffff" }, "entity_id": "y", "hostname": "smithi031", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "85292", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.31:0", "nonce": 2580495381, "type": "any" }, "name": { "num": 5637, "type": "client" } }, "id": 5637, "inst": "client.5637 172.21.15.31:0/2580495381", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 7, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 17.47359444990975 }, "state": "open", "uptime": 58.46012631 } ] 2023-10-06T02:25:55.533 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-10-06T02:26:00.534 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:26:00.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:00.731+0000 7fd505d9d700 1 Processor -- start 2023-10-06T02:26:00.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:00.731+0000 7fd505d9d700 1 -- start start 2023-10-06T02:26:00.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:00.732+0000 7fd505d9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd500159640 0x7fd50006d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:00.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:00.732+0000 7fd505d9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5000c7420 con 0x7fd500159a10 2023-10-06T02:26:00.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:00.732+0000 7fd505d9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd50006d5a0 con 0x7fd5000caec0 2023-10-06T02:26:00.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:00.732+0000 7fd505d9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd50006d820 con 0x7fd500159640 2023-10-06T02:26:00.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:00.732+0000 7fd504d9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd500159640 0x7fd50006d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:00.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:00.732+0000 7fd4f7fff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fd500159a10 0x7fd5000ca7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:36090/0 (socket says 172.21.15.31:36090) 2023-10-06T02:26:00.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:00.733+0000 7fd4f7fff700 1 -- 172.21.15.31:0/3557297766 learned_addr learned my addr 172.21.15.31:0/3557297766 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:26:00.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:00.733+0000 7fd504d9b700 1 -- 172.21.15.31:0/3557297766 >> v1:172.21.15.62:6789/0 conn(0x7fd5000caec0 legacy=0x7fd5000c78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:00.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:00.733+0000 7fd504d9b700 1 -- 172.21.15.31:0/3557297766 >> v1:172.21.15.31:6789/0 conn(0x7fd500159a10 legacy=0x7fd5000ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:00.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:00.733+0000 7fd504d9b700 1 -- 172.21.15.31:0/3557297766 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd50006db00 con 0x7fd500159640 2023-10-06T02:26:00.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:00.733+0000 7fd504d9b700 1 --2- 172.21.15.31:0/3557297766 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd500159640 0x7fd50006d060 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc009910 tx=0x7fd4fc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4132512d98170920 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:00.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:00.733+0000 7fd4f77fe700 1 -- 172.21.15.31:0/3557297766 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4fc00b820 con 0x7fd500159640 2023-10-06T02:26:00.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:00.734+0000 7fd4f77fe700 1 -- 172.21.15.31:0/3557297766 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4fc00b980 con 0x7fd500159640 2023-10-06T02:26:00.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:00.734+0000 7fd4f77fe700 1 -- 172.21.15.31:0/3557297766 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4fc00bd20 con 0x7fd500159640 2023-10-06T02:26:00.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:00.734+0000 7fd505d9d700 1 -- 172.21.15.31:0/3557297766 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd500159640 msgr2=0x7fd50006d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:00.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:00.734+0000 7fd505d9d700 1 --2- 172.21.15.31:0/3557297766 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd500159640 0x7fd50006d060 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc009910 tx=0x7fd4fc0091a0 comp rx=0 tx=0).stop 2023-10-06T02:26:00.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:00.734+0000 7fd505d9d700 1 -- 172.21.15.31:0/3557297766 shutdown_connections 2023-10-06T02:26:00.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:00.734+0000 7fd505d9d700 1 --2- 172.21.15.31:0/3557297766 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd500159640 0x7fd50006d060 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:00.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:00.734+0000 7fd505d9d700 1 -- 172.21.15.31:0/3557297766 >> 172.21.15.31:0/3557297766 conn(0x7fd50006e720 msgr2=0x7fd50006eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:00.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:00.734+0000 7fd505d9d700 1 -- 172.21.15.31:0/3557297766 shutdown_connections 2023-10-06T02:26:00.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:00.734+0000 7fd505d9d700 1 -- 172.21.15.31:0/3557297766 wait complete. 2023-10-06T02:26:00.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:00.736+0000 7fd505d9d700 1 Processor -- start 2023-10-06T02:26:00.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:00.736+0000 7fd505d9d700 1 -- start start 2023-10-06T02:26:00.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:00.736+0000 7fd505d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5000caec0 0x7fd5000afb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:00.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:00.737+0000 7fd505d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd500159640 0x7fd5000b1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:00.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:00.737+0000 7fd505d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd500159a10 0x7fd50016af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:00.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:00.737+0000 7fd505d9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd500143c30 con 0x7fd500159640 2023-10-06T02:26:00.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:00.737+0000 7fd505d9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd50015d220 con 0x7fd5000caec0 2023-10-06T02:26:00.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:00.737+0000 7fd505d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5000b0090 con 0x7fd500159a10 2023-10-06T02:26:00.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:00.737+0000 7fd4f7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd500159640 0x7fd5000b1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:00.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:00.737+0000 7fd4f7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd500159640 0x7fd5000b1f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40362/0 (socket says 172.21.15.31:40362) 2023-10-06T02:26:00.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:00.737+0000 7fd4f7fff700 1 -- 172.21.15.31:0/141838504 learned_addr learned my addr 172.21.15.31:0/141838504 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:26:00.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:00.737+0000 7fd4f7fff700 1 -- 172.21.15.31:0/141838504 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd500159a10 msgr2=0x7fd50016af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:00.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:00.737+0000 7fd504d9b700 1 --2- 172.21.15.31:0/141838504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5000caec0 0x7fd5000afb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:00.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:00.737+0000 7fd50559c700 1 --2- 172.21.15.31:0/141838504 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd500159a10 0x7fd50016af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:00.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:00.737+0000 7fd4f7fff700 1 --2- 172.21.15.31:0/141838504 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd500159a10 0x7fd50016af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:00.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:00.737+0000 7fd4f7fff700 1 -- 172.21.15.31:0/141838504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5000caec0 msgr2=0x7fd5000afb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:00.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:00.737+0000 7fd4f7fff700 1 --2- 172.21.15.31:0/141838504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5000caec0 0x7fd5000afb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:00.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:00.737+0000 7fd4f7fff700 1 -- 172.21.15.31:0/141838504 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4fc009420 con 0x7fd500159640 2023-10-06T02:26:00.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:00.737+0000 7fd504d9b700 1 --2- 172.21.15.31:0/141838504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5000caec0 0x7fd5000afb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:26:00.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:00.738+0000 7fd4f7fff700 1 --2- 172.21.15.31:0/141838504 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd500159640 0x7fd5000b1f90 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fd4f000c590 tx=0x7fd4f0001020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:00.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:00.738+0000 7fd4f4ff9700 1 -- 172.21.15.31:0/141838504 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f0005b30 con 0x7fd500159640 2023-10-06T02:26:00.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:00.738+0000 7fd4f4ff9700 1 -- 172.21.15.31:0/141838504 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4f0005cb0 con 0x7fd500159640 2023-10-06T02:26:00.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:00.738+0000 7fd505d9d700 1 -- 172.21.15.31:0/141838504 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5000b24d0 con 0x7fd500159640 2023-10-06T02:26:00.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:00.738+0000 7fd4f4ff9700 1 -- 172.21.15.31:0/141838504 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f0007320 con 0x7fd500159640 2023-10-06T02:26:00.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:00.738+0000 7fd505d9d700 1 -- 172.21.15.31:0/141838504 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd50015c970 con 0x7fd500159640 2023-10-06T02:26:00.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:00.739+0000 7fd505d9d700 1 -- 172.21.15.31:0/141838504 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd50015d3d0 con 0x7fd500159640 2023-10-06T02:26:00.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:00.740+0000 7fd4f4ff9700 1 -- 172.21.15.31:0/141838504 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 84944+0+0 (secure 0 0 0) 0x7fd4f001b3b0 con 0x7fd500159640 2023-10-06T02:26:00.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:00.741+0000 7fd4f4ff9700 1 --2- 172.21.15.31:0/141838504 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fd4ec067170 0x7fd4ec069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:00.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:00.741+0000 7fd4f4ff9700 1 -- 172.21.15.31:0/141838504 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6116+0+0 (secure 0 0 0) 0x7fd4f0083f80 con 0x7fd500159640 2023-10-06T02:26:00.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:00.741+0000 7fd504d9b700 1 --2- 172.21.15.31:0/141838504 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fd4ec067170 0x7fd4ec069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:00.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:00.741+0000 7fd504d9b700 1 --2- 172.21.15.31:0/141838504 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fd4ec067170 0x7fd4ec069620 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd5000cc060 tx=0x7fd4fc017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:00.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:00.745+0000 7fd4f4ff9700 1 -- 172.21.15.31:0/141838504 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd50015d3d0 con 0x7fd500159640 2023-10-06T02:26:00.878 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T02:26:00.882 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T02:26:00.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:00.994+0000 7fd505d9d700 1 -- 172.21.15.31:0/141838504 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd50015d390 con 0x7fd500159640 2023-10-06T02:26:00.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:00.995+0000 7fd4f4ff9700 1 -- 172.21.15.31:0/141838504 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4594 (secure 0 0 0) 0x7fd4f0017090 con 0x7fd500159640 2023-10-06T02:26:00.996 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:26:00.997 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4949,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/178547366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":178547366},{"type":"v1","addr":"172.21.15.62:6835","nonce":178547366}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":5278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/2578831517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":2578831517},{"type":"v1","addr":"172.21.15.31:6836","nonce":2578831517}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5287,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3502585737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":3502585737},{"type":"v1","addr":"172.21.15.31:6837","nonce":3502585737}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:24:43.872314+0000","modified":"2023-10-06T02:24:47.404607+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5290":{"gid":5290,"name":"b","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6834/2358776861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2358776861},{"type":"v1","addr":"172.21.15.62:6834","nonce":2358776861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T02:26:00.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:00.997+0000 7fd505d9d700 1 -- 172.21.15.31:0/141838504 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fd4ec067170 msgr2=0x7fd4ec069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:00.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:00.997+0000 7fd505d9d700 1 --2- 172.21.15.31:0/141838504 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fd4ec067170 0x7fd4ec069620 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd5000cc060 tx=0x7fd4fc017000 comp rx=0 tx=0).stop 2023-10-06T02:26:00.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:00.997+0000 7fd505d9d700 1 -- 172.21.15.31:0/141838504 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd500159640 msgr2=0x7fd5000b1f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:00.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:00.997+0000 7fd505d9d700 1 --2- 172.21.15.31:0/141838504 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd500159640 0x7fd5000b1f90 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fd4f000c590 tx=0x7fd4f0001020 comp rx=0 tx=0).stop 2023-10-06T02:26:00.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:00.997+0000 7fd505d9d700 1 -- 172.21.15.31:0/141838504 shutdown_connections 2023-10-06T02:26:01.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:00.997+0000 7fd505d9d700 1 --2- 172.21.15.31:0/141838504 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fd4ec067170 0x7fd4ec069620 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:01.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:00.997+0000 7fd505d9d700 1 --2- 172.21.15.31:0/141838504 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd500159a10 0x7fd50016af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:01.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:00.997+0000 7fd505d9d700 1 --2- 172.21.15.31:0/141838504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5000caec0 0x7fd5000afb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:01.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:00.997+0000 7fd505d9d700 1 --2- 172.21.15.31:0/141838504 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd500159640 0x7fd5000b1f90 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:01.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:00.997+0000 7fd505d9d700 1 -- 172.21.15.31:0/141838504 >> 172.21.15.31:0/141838504 conn(0x7fd50006e720 msgr2=0x7fd500157f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:01.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:00.998+0000 7fd505d9d700 1 -- 172.21.15.31:0/141838504 shutdown_connections 2023-10-06T02:26:01.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:00.998+0000 7fd505d9d700 1 -- 172.21.15.31:0/141838504 wait complete. 2023-10-06T02:26:01.001 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 27 2023-10-06T02:26:01.016 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json session ls 2023-10-06T02:26:01.231 INFO:teuthology.orchestra.run.smithi062.stdout:[{"id":5637,"entity":{"name":{"type":"client","num":5637},"addr":{"type":"any","addr":"172.21.15.31:0","nonce":2580495381}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":7,"uptime":64.158041150000003,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":17.245062560787069,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5637 172.21.15.31:0/2580495381","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x000000000017ffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"736ac5ba72cb47af7d6f6666b1e100f7e20a20e2","ceph_version":"ceph version 17.2.6-1296-g736ac5ba (736ac5ba72cb47af7d6f6666b1e100f7e20a20e2) quincy (stable)","entity_id":"y","hostname":"smithi031","pid":"85292","root":"/"}}] 2023-10-06T02:26:01.232 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "736ac5ba72cb47af7d6f6666b1e100f7e20a20e2", "ceph_version": "ceph version 17.2.6-1296-g736ac5ba (736ac5ba72cb47af7d6f6666b1e100f7e20a20e2) quincy (stable)", "client_features": { "feature_bits": "0x000000000017ffff" }, "entity_id": "y", "hostname": "smithi031", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "85292", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.31:0", "nonce": 2580495381, "type": "any" }, "name": { "num": 5637, "type": "client" } }, "id": 5637, "inst": "client.5637 172.21.15.31:0/2580495381", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 7, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 17.24506256078707 }, "state": "open", "uptime": 64.15804115 } ] 2023-10-06T02:26:01.232 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-10-06T02:26:06.233 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:26:06.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:06.429+0000 7f745cd9d700 1 Processor -- start 2023-10-06T02:26:06.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:06.429+0000 7f745cd9d700 1 -- start start 2023-10-06T02:26:06.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:06.431+0000 7f745cd9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f74580c6fa0 0x7f745814fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:06.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:06.431+0000 7f745cd9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f745806a490 con 0x7f74580c6b20 2023-10-06T02:26:06.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:06.431+0000 7f745cd9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f745806e7b0 con 0x7f745806f3a0 2023-10-06T02:26:06.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:06.431+0000 7f745cd9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74580ccf50 con 0x7f74580c6fa0 2023-10-06T02:26:06.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:06.431+0000 7f7456ffd700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f74580c6b20 0x7f745806e0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:36104/0 (socket says 172.21.15.31:36104) 2023-10-06T02:26:06.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:06.431+0000 7f7456ffd700 1 -- 172.21.15.31:0/3513192857 learned_addr learned my addr 172.21.15.31:0/3513192857 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:26:06.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:06.431+0000 7f7457fff700 1 --2- 172.21.15.31:0/3513192857 >> v2:172.21.15.62:3300/0 conn(0x7f74580c6fa0 0x7f745814fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:06.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:06.431+0000 7f74567fc700 1 -- 172.21.15.31:0/3513192857 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3058446455 0 0) 0x7f745806a490 con 0x7f74580c6b20 2023-10-06T02:26:06.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:06.431+0000 7f74567fc700 1 -- 172.21.15.31:0/3513192857 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7440003570 con 0x7f74580c6b20 2023-10-06T02:26:06.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:06.432+0000 7f74567fc700 1 -- 172.21.15.31:0/3513192857 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1247624203 0 0) 0x7f7440003570 con 0x7f74580c6b20 2023-10-06T02:26:06.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:06.432+0000 7f7457fff700 1 -- 172.21.15.31:0/3513192857 >> v1:172.21.15.62:6789/0 conn(0x7f745806f3a0 legacy=0x7f74580cc840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:06.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:06.432+0000 7f7457fff700 1 -- 172.21.15.31:0/3513192857 >> v1:172.21.15.31:6789/0 conn(0x7f74580c6b20 legacy=0x7f745806e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:06.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:06.432+0000 7f7457fff700 1 -- 172.21.15.31:0/3513192857 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74580c7490 con 0x7f74580c6fa0 2023-10-06T02:26:06.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:06.432+0000 7f7457fff700 1 --2- 172.21.15.31:0/3513192857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74580c6fa0 0x7f745814fc70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7450009e60 tx=0x7f7450009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a347954dd41a8593 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:06.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:06.432+0000 7f74567fc700 1 -- 172.21.15.31:0/3513192857 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f745000e070 con 0x7f74580c6fa0 2023-10-06T02:26:06.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:06.432+0000 7f74567fc700 1 -- 172.21.15.31:0/3513192857 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f745000b850 con 0x7f74580c6fa0 2023-10-06T02:26:06.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:06.432+0000 7f74567fc700 1 -- 172.21.15.31:0/3513192857 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f744c0015f0 con 0x7f74580c6fa0 2023-10-06T02:26:06.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:06.433+0000 7f745cd9d700 1 -- 172.21.15.31:0/3513192857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74580c6fa0 msgr2=0x7f745814fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:06.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:06.433+0000 7f745cd9d700 1 --2- 172.21.15.31:0/3513192857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74580c6fa0 0x7f745814fc70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7450009e60 tx=0x7f7450009780 comp rx=0 tx=0).stop 2023-10-06T02:26:06.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:06.433+0000 7f745cd9d700 1 -- 172.21.15.31:0/3513192857 shutdown_connections 2023-10-06T02:26:06.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:06.433+0000 7f745cd9d700 1 --2- 172.21.15.31:0/3513192857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74580c6fa0 0x7f745814fc70 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:06.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:06.433+0000 7f745cd9d700 1 -- 172.21.15.31:0/3513192857 >> 172.21.15.31:0/3513192857 conn(0x7f745806d170 msgr2=0x7f74580cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:06.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:06.433+0000 7f745cd9d700 1 -- 172.21.15.31:0/3513192857 shutdown_connections 2023-10-06T02:26:06.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:06.433+0000 7f745cd9d700 1 -- 172.21.15.31:0/3513192857 wait complete. 2023-10-06T02:26:06.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:06.434+0000 7f745cd9d700 1 Processor -- start 2023-10-06T02:26:06.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:06.435+0000 7f745cd9d700 1 -- start start 2023-10-06T02:26:06.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:06.436+0000 7f745cd9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f745806f3a0 0x7f7458167070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:06.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:06.436+0000 7f745cd9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f74580c6b20 0x7f7458140b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:06.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:06.436+0000 7f745cd9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74580c6fa0 0x7f745813e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:06.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:06.436+0000 7f745cd9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f745813eb30 con 0x7f745806f3a0 2023-10-06T02:26:06.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:06.436+0000 7f745cd9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f745813ee10 con 0x7f74580c6fa0 2023-10-06T02:26:06.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:06.436+0000 7f745cd9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f745813f0f0 con 0x7f74580c6b20 2023-10-06T02:26:06.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:06.436+0000 7f74577fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f745806f3a0 0x7f7458167070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:06.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:06.436+0000 7f74577fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f745806f3a0 0x7f7458167070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40374/0 (socket says 172.21.15.31:40374) 2023-10-06T02:26:06.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:06.436+0000 7f74577fe700 1 -- 172.21.15.31:0/374935359 learned_addr learned my addr 172.21.15.31:0/374935359 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:26:06.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:06.436+0000 7f74577fe700 1 -- 172.21.15.31:0/374935359 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f74580c6b20 msgr2=0x7f7458140b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:06.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:06.436+0000 7f7456ffd700 1 --2- 172.21.15.31:0/374935359 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f74580c6b20 0x7f7458140b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:06.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:06.436+0000 7f7457fff700 1 --2- 172.21.15.31:0/374935359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74580c6fa0 0x7f745813e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:06.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:06.436+0000 7f74577fe700 1 --2- 172.21.15.31:0/374935359 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f74580c6b20 0x7f7458140b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:06.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:06.436+0000 7f74577fe700 1 -- 172.21.15.31:0/374935359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74580c6fa0 msgr2=0x7f745813e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:06.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:06.436+0000 7f74577fe700 1 --2- 172.21.15.31:0/374935359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74580c6fa0 0x7f745813e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:06.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:06.436+0000 7f74577fe700 1 -- 172.21.15.31:0/374935359 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7450009a20 con 0x7f745806f3a0 2023-10-06T02:26:06.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:06.436+0000 7f7456ffd700 1 --2- 172.21.15.31:0/374935359 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f74580c6b20 0x7f7458140b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:26:06.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:06.436+0000 7f7457fff700 1 --2- 172.21.15.31:0/374935359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74580c6fa0 0x7f745813e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:26:06.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:06.437+0000 7f74577fe700 1 --2- 172.21.15.31:0/374935359 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f745806f3a0 0x7f7458167070 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f744800c790 tx=0x7f7448007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:06.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:06.437+0000 7f743f7fe700 1 -- 172.21.15.31:0/374935359 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7448012630 con 0x7f745806f3a0 2023-10-06T02:26:06.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:06.437+0000 7f743f7fe700 1 -- 172.21.15.31:0/374935359 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74480127b0 con 0x7f745806f3a0 2023-10-06T02:26:06.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:06.437+0000 7f745cd9d700 1 -- 172.21.15.31:0/374935359 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f745813f3d0 con 0x7f745806f3a0 2023-10-06T02:26:06.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:06.437+0000 7f743f7fe700 1 -- 172.21.15.31:0/374935359 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7448012b70 con 0x7f745806f3a0 2023-10-06T02:26:06.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:06.437+0000 7f745cd9d700 1 -- 172.21.15.31:0/374935359 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f745814fc70 con 0x7f745806f3a0 2023-10-06T02:26:06.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:06.438+0000 7f745cd9d700 1 -- 172.21.15.31:0/374935359 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f745804c3c0 con 0x7f745806f3a0 2023-10-06T02:26:06.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:06.439+0000 7f743f7fe700 1 -- 172.21.15.31:0/374935359 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f744801b400 con 0x7f745806f3a0 2023-10-06T02:26:06.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:06.440+0000 7f743f7fe700 1 --2- 172.21.15.31:0/374935359 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f7444067170 0x7f7444069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:06.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:06.440+0000 7f743f7fe700 1 -- 172.21.15.31:0/374935359 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f74480846d0 con 0x7f745806f3a0 2023-10-06T02:26:06.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:06.440+0000 7f7456ffd700 1 --2- 172.21.15.31:0/374935359 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f7444067170 0x7f7444069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:06.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:06.441+0000 7f7456ffd700 1 --2- 172.21.15.31:0/374935359 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f7444067170 0x7f7444069620 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f744c001360 tx=0x7f744c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:06.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:06.444+0000 7f743f7fe700 1 -- 172.21.15.31:0/374935359 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7448051470 con 0x7f745806f3a0 2023-10-06T02:26:06.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:06.688+0000 7f745cd9d700 1 -- 172.21.15.31:0/374935359 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f74580c7770 con 0x7f745806f3a0 2023-10-06T02:26:06.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:06.689+0000 7f743f7fe700 1 -- 172.21.15.31:0/374935359 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4594 (secure 0 0 0) 0x7f7448054a90 con 0x7f745806f3a0 2023-10-06T02:26:06.690 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:26:06.691 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4949,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/178547366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":178547366},{"type":"v1","addr":"172.21.15.62:6835","nonce":178547366}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":5278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/2578831517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":2578831517},{"type":"v1","addr":"172.21.15.31:6836","nonce":2578831517}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5287,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3502585737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":3502585737},{"type":"v1","addr":"172.21.15.31:6837","nonce":3502585737}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:24:43.872314+0000","modified":"2023-10-06T02:24:47.404607+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5290":{"gid":5290,"name":"b","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6834/2358776861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2358776861},{"type":"v1","addr":"172.21.15.62:6834","nonce":2358776861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T02:26:06.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:06.692+0000 7f745cd9d700 1 -- 172.21.15.31:0/374935359 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f7444067170 msgr2=0x7f7444069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:06.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:06.692+0000 7f745cd9d700 1 --2- 172.21.15.31:0/374935359 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f7444067170 0x7f7444069620 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f744c001360 tx=0x7f744c009040 comp rx=0 tx=0).stop 2023-10-06T02:26:06.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:06.692+0000 7f745cd9d700 1 -- 172.21.15.31:0/374935359 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f745806f3a0 msgr2=0x7f7458167070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:06.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:06.692+0000 7f745cd9d700 1 --2- 172.21.15.31:0/374935359 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f745806f3a0 0x7f7458167070 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f744800c790 tx=0x7f7448007f60 comp rx=0 tx=0).stop 2023-10-06T02:26:06.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:06.692+0000 7f745cd9d700 1 -- 172.21.15.31:0/374935359 shutdown_connections 2023-10-06T02:26:06.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:06.692+0000 7f745cd9d700 1 --2- 172.21.15.31:0/374935359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74580c6fa0 0x7f745813e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:06.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:06.692+0000 7f745cd9d700 1 --2- 172.21.15.31:0/374935359 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f745806f3a0 0x7f7458167070 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:06.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:06.692+0000 7f745cd9d700 1 --2- 172.21.15.31:0/374935359 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f7444067170 0x7f7444069620 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:06.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:06.692+0000 7f745cd9d700 1 --2- 172.21.15.31:0/374935359 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f74580c6b20 0x7f7458140b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:06.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:06.692+0000 7f745cd9d700 1 -- 172.21.15.31:0/374935359 >> 172.21.15.31:0/374935359 conn(0x7f745806d170 msgr2=0x7f74580b1760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:06.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:06.692+0000 7f745cd9d700 1 -- 172.21.15.31:0/374935359 shutdown_connections 2023-10-06T02:26:06.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:06.692+0000 7f745cd9d700 1 -- 172.21.15.31:0/374935359 wait complete. 2023-10-06T02:26:06.694 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 27 2023-10-06T02:26:06.711 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json session ls 2023-10-06T02:26:06.932 INFO:teuthology.orchestra.run.smithi062.stdout:[{"id":5637,"entity":{"name":{"type":"client","num":5637},"addr":{"type":"any","addr":"172.21.15.31:0","nonce":2580495381}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":6,"uptime":69.858955836000007,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":17.019401607688469,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5637 172.21.15.31:0/2580495381","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x000000000017ffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"736ac5ba72cb47af7d6f6666b1e100f7e20a20e2","ceph_version":"ceph version 17.2.6-1296-g736ac5ba (736ac5ba72cb47af7d6f6666b1e100f7e20a20e2) quincy (stable)","entity_id":"y","hostname":"smithi031","pid":"85292","root":"/"}}] 2023-10-06T02:26:06.932 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "736ac5ba72cb47af7d6f6666b1e100f7e20a20e2", "ceph_version": "ceph version 17.2.6-1296-g736ac5ba (736ac5ba72cb47af7d6f6666b1e100f7e20a20e2) quincy (stable)", "client_features": { "feature_bits": "0x000000000017ffff" }, "entity_id": "y", "hostname": "smithi031", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "85292", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.31:0", "nonce": 2580495381, "type": "any" }, "name": { "num": 5637, "type": "client" } }, "id": 5637, "inst": "client.5637 172.21.15.31:0/2580495381", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 6, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 17.01940160768847 }, "state": "open", "uptime": 69.858955836 } ] 2023-10-06T02:26:06.933 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-10-06T02:26:11.933 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:26:12.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:12.130+0000 7f50a6f9d700 1 Processor -- start 2023-10-06T02:26:12.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:12.130+0000 7f50a6f9d700 1 -- start start 2023-10-06T02:26:12.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:12.131+0000 7f50a6f9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f50a0142fd0 0x7f50a00c7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:12.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:12.131+0000 7f50a6f9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f50a00c7dc0 0x7f50a006c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:12.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:12.131+0000 7f50a6f9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50a00ca480 con 0x7f50a0142c00 2023-10-06T02:26:12.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:12.131+0000 7f50a6f9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50a015e240 con 0x7f50a0142fd0 2023-10-06T02:26:12.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:12.131+0000 7f50a6f9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50a006b350 con 0x7f50a00c7dc0 2023-10-06T02:26:12.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:12.132+0000 7f50a579a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f50a0142fd0 0x7f50a00c7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:12.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:12.132+0000 7f50a579a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f50a0142fd0 0x7f50a00c7880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49656/0 (socket says 172.21.15.31:49656) 2023-10-06T02:26:12.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:12.132+0000 7f50a579a700 1 -- 172.21.15.31:0/2298749582 learned_addr learned my addr 172.21.15.31:0/2298749582 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:26:12.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:12.132+0000 7f50a679c700 1 --2- 172.21.15.31:0/2298749582 >> v2:172.21.15.62:3300/0 conn(0x7f50a00c7dc0 0x7f50a006c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:12.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:12.132+0000 7f50a579a700 1 -- 172.21.15.31:0/2298749582 >> v2:172.21.15.62:3300/0 conn(0x7f50a00c7dc0 msgr2=0x7f50a006c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:26:12.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:12.132+0000 7f50a579a700 1 --2- 172.21.15.31:0/2298749582 >> v2:172.21.15.62:3300/0 conn(0x7f50a00c7dc0 0x7f50a006c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:12.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:12.132+0000 7f50a579a700 1 -- 172.21.15.31:0/2298749582 >> v1:172.21.15.62:6789/0 conn(0x7f50a0142c00 legacy=0x7f50a006b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:26:12.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:12.132+0000 7f50a579a700 1 -- 172.21.15.31:0/2298749582 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50a00c81d0 con 0x7f50a0142fd0 2023-10-06T02:26:12.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:12.132+0000 7f50a579a700 1 --2- 172.21.15.31:0/2298749582 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50a0142fd0 0x7f50a00c7880 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f50a006d750 tx=0x7f5090009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cba6304768c1ec8d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:12.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:12.133+0000 7f50a4f99700 1 -- 172.21.15.31:0/2298749582 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509000b820 con 0x7f50a0142fd0 2023-10-06T02:26:12.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:12.133+0000 7f50a4f99700 1 -- 172.21.15.31:0/2298749582 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f509000b980 con 0x7f50a0142fd0 2023-10-06T02:26:12.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:12.133+0000 7f50a4f99700 1 -- 172.21.15.31:0/2298749582 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509000bd20 con 0x7f50a0142fd0 2023-10-06T02:26:12.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:12.133+0000 7f50a6f9d700 1 -- 172.21.15.31:0/2298749582 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50a0142fd0 msgr2=0x7f50a00c7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:12.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:12.133+0000 7f50a6f9d700 1 --2- 172.21.15.31:0/2298749582 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50a0142fd0 0x7f50a00c7880 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f50a006d750 tx=0x7f5090009250 comp rx=0 tx=0).stop 2023-10-06T02:26:12.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:12.133+0000 7f50a6f9d700 1 -- 172.21.15.31:0/2298749582 shutdown_connections 2023-10-06T02:26:12.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:12.133+0000 7f50a6f9d700 1 --2- 172.21.15.31:0/2298749582 >> v2:172.21.15.62:3300/0 conn(0x7f50a00c7dc0 0x7f50a006c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:12.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:12.133+0000 7f50a6f9d700 1 --2- 172.21.15.31:0/2298749582 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50a0142fd0 0x7f50a00c7880 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:12.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:12.133+0000 7f50a6f9d700 1 -- 172.21.15.31:0/2298749582 >> 172.21.15.31:0/2298749582 conn(0x7f50a00675a0 msgr2=0x7f50a00679a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:12.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:12.133+0000 7f50a6f9d700 1 -- 172.21.15.31:0/2298749582 shutdown_connections 2023-10-06T02:26:12.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:12.133+0000 7f50a6f9d700 1 -- 172.21.15.31:0/2298749582 wait complete. 2023-10-06T02:26:12.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:12.135+0000 7f50a6f9d700 1 Processor -- start 2023-10-06T02:26:12.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:12.135+0000 7f50a6f9d700 1 -- start start 2023-10-06T02:26:12.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:12.136+0000 7f50a6f9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50a00c7dc0 0x7f50a015ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:12.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:12.136+0000 7f50a6f9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50a0142c00 0x7f50a0159e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:12.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:12.136+0000 7f50a6f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f50a0142fd0 0x7f50a015a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:12.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:12.136+0000 7f50a6f9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a0169a30 con 0x7f50a00c7dc0 2023-10-06T02:26:12.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:12.136+0000 7f50a6f9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a015b050 con 0x7f50a0142c00 2023-10-06T02:26:12.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:12.136+0000 7f50a6f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50a015aa90 con 0x7f50a0142fd0 2023-10-06T02:26:12.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:12.136+0000 7f50a5f9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50a00c7dc0 0x7f50a015ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:12.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:12.136+0000 7f50a5f9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50a00c7dc0 0x7f50a015ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49660/0 (socket says 172.21.15.31:49660) 2023-10-06T02:26:12.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:12.136+0000 7f50a5f9b700 1 -- 172.21.15.31:0/2140514501 learned_addr learned my addr 172.21.15.31:0/2140514501 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:26:12.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:12.136+0000 7f50a5f9b700 1 -- 172.21.15.31:0/2140514501 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f50a0142fd0 msgr2=0x7f50a015a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:12.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:12.136+0000 7f50a5f9b700 1 --2- 172.21.15.31:0/2140514501 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f50a0142fd0 0x7f50a015a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:12.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:12.136+0000 7f50a5f9b700 1 -- 172.21.15.31:0/2140514501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50a0142c00 msgr2=0x7f50a0159e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:12.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:12.136+0000 7f50a5f9b700 1 --2- 172.21.15.31:0/2140514501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50a0142c00 0x7f50a0159e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:12.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:12.136+0000 7f50a5f9b700 1 -- 172.21.15.31:0/2140514501 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50900094f0 con 0x7f50a00c7dc0 2023-10-06T02:26:12.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:12.137+0000 7f50a5f9b700 1 --2- 172.21.15.31:0/2140514501 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50a00c7dc0 0x7f50a015ba80 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f509c00c5d0 tx=0x7f509c007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:12.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:12.137+0000 7f5095ffb700 1 -- 172.21.15.31:0/2140514501 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509c011590 con 0x7f50a00c7dc0 2023-10-06T02:26:12.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:12.137+0000 7f5095ffb700 1 -- 172.21.15.31:0/2140514501 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f509c011710 con 0x7f50a00c7dc0 2023-10-06T02:26:12.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:12.137+0000 7f50a6f9d700 1 -- 172.21.15.31:0/2140514501 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50a015ad70 con 0x7f50a00c7dc0 2023-10-06T02:26:12.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:12.137+0000 7f5095ffb700 1 -- 172.21.15.31:0/2140514501 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509c011ad0 con 0x7f50a00c7dc0 2023-10-06T02:26:12.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:12.137+0000 7f50a6f9d700 1 -- 172.21.15.31:0/2140514501 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50a00c4700 con 0x7f50a00c7dc0 2023-10-06T02:26:12.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:12.138+0000 7f50a6f9d700 1 -- 172.21.15.31:0/2140514501 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50a00c4920 con 0x7f50a00c7dc0 2023-10-06T02:26:12.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:12.139+0000 7f5095ffb700 1 -- 172.21.15.31:0/2140514501 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f509c01c460 con 0x7f50a00c7dc0 2023-10-06T02:26:12.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:12.140+0000 7f5095ffb700 1 --2- 172.21.15.31:0/2140514501 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5084067170 0x7f5084069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:12.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:12.140+0000 7f5095ffb700 1 -- 172.21.15.31:0/2140514501 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f509c0848a0 con 0x7f50a00c7dc0 2023-10-06T02:26:12.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:12.140+0000 7f50a579a700 1 --2- 172.21.15.31:0/2140514501 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5084067170 0x7f5084069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:12.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:12.141+0000 7f50a579a700 1 --2- 172.21.15.31:0/2140514501 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5084067170 0x7f5084069620 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f50a006d750 tx=0x7f5090016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:12.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:12.144+0000 7f5095ffb700 1 -- 172.21.15.31:0/2140514501 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f509c051640 con 0x7f50a00c7dc0 2023-10-06T02:26:12.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:12.394+0000 7f50a6f9d700 1 -- 172.21.15.31:0/2140514501 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f50a004c3c0 con 0x7f50a00c7dc0 2023-10-06T02:26:12.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:12.395+0000 7f5095ffb700 1 -- 172.21.15.31:0/2140514501 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4594 (secure 0 0 0) 0x7f509c054c60 con 0x7f50a00c7dc0 2023-10-06T02:26:12.397 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:26:12.397 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4949,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/178547366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":178547366},{"type":"v1","addr":"172.21.15.62:6835","nonce":178547366}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":5278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/2578831517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":2578831517},{"type":"v1","addr":"172.21.15.31:6836","nonce":2578831517}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5287,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3502585737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":3502585737},{"type":"v1","addr":"172.21.15.31:6837","nonce":3502585737}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:24:43.872314+0000","modified":"2023-10-06T02:24:47.404607+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5290":{"gid":5290,"name":"b","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6834/2358776861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2358776861},{"type":"v1","addr":"172.21.15.62:6834","nonce":2358776861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T02:26:12.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:12.398+0000 7f50a6f9d700 1 -- 172.21.15.31:0/2140514501 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5084067170 msgr2=0x7f5084069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:12.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:12.398+0000 7f50a6f9d700 1 --2- 172.21.15.31:0/2140514501 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5084067170 0x7f5084069620 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f50a006d750 tx=0x7f5090016000 comp rx=0 tx=0).stop 2023-10-06T02:26:12.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:12.398+0000 7f50a6f9d700 1 -- 172.21.15.31:0/2140514501 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50a00c7dc0 msgr2=0x7f50a015ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:12.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:12.398+0000 7f50a6f9d700 1 --2- 172.21.15.31:0/2140514501 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50a00c7dc0 0x7f50a015ba80 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f509c00c5d0 tx=0x7f509c007f60 comp rx=0 tx=0).stop 2023-10-06T02:26:12.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:12.398+0000 7f50a6f9d700 1 -- 172.21.15.31:0/2140514501 shutdown_connections 2023-10-06T02:26:12.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:12.398+0000 7f50a6f9d700 1 --2- 172.21.15.31:0/2140514501 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5084067170 0x7f5084069620 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:12.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:12.398+0000 7f50a6f9d700 1 --2- 172.21.15.31:0/2140514501 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f50a0142fd0 0x7f50a015a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:12.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:12.398+0000 7f50a6f9d700 1 --2- 172.21.15.31:0/2140514501 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50a00c7dc0 0x7f50a015ba80 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:12.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:12.398+0000 7f50a6f9d700 1 --2- 172.21.15.31:0/2140514501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50a0142c00 0x7f50a0159e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:12.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:12.398+0000 7f50a6f9d700 1 -- 172.21.15.31:0/2140514501 >> 172.21.15.31:0/2140514501 conn(0x7f50a00675a0 msgr2=0x7f50a006c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:12.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:12.398+0000 7f50a6f9d700 1 -- 172.21.15.31:0/2140514501 shutdown_connections 2023-10-06T02:26:12.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:12.398+0000 7f50a6f9d700 1 -- 172.21.15.31:0/2140514501 wait complete. 2023-10-06T02:26:12.401 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 27 2023-10-06T02:26:12.419 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json session ls 2023-10-06T02:26:12.636 INFO:teuthology.orchestra.run.smithi062.stdout:[{"id":5637,"entity":{"name":{"type":"client","num":5637},"addr":{"type":"any","addr":"172.21.15.31:0","nonce":2580495381}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":6,"uptime":75.56387024,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":16.796538327231765,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5637 172.21.15.31:0/2580495381","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x000000000017ffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"736ac5ba72cb47af7d6f6666b1e100f7e20a20e2","ceph_version":"ceph version 17.2.6-1296-g736ac5ba (736ac5ba72cb47af7d6f6666b1e100f7e20a20e2) quincy (stable)","entity_id":"y","hostname":"smithi031","pid":"85292","root":"/"}}] 2023-10-06T02:26:12.637 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "736ac5ba72cb47af7d6f6666b1e100f7e20a20e2", "ceph_version": "ceph version 17.2.6-1296-g736ac5ba (736ac5ba72cb47af7d6f6666b1e100f7e20a20e2) quincy (stable)", "client_features": { "feature_bits": "0x000000000017ffff" }, "entity_id": "y", "hostname": "smithi031", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "85292", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.31:0", "nonce": 2580495381, "type": "any" }, "name": { "num": 5637, "type": "client" } }, "id": 5637, "inst": "client.5637 172.21.15.31:0/2580495381", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 6, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 16.796538327231765 }, "state": "open", "uptime": 75.56387024 } ] 2023-10-06T02:26:12.637 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-10-06T02:26:17.639 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:26:17.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:17.835+0000 7fc64f59e700 1 Processor -- start 2023-10-06T02:26:17.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:17.835+0000 7fc64f59e700 1 -- start start 2023-10-06T02:26:17.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:17.836+0000 7fc64f59e700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc650142c00 0x7fc65006b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:17.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:17.836+0000 7fc64f59e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc65006bd20 0x7fc6500c7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:17.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:17.836+0000 7fc64f59e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc650142fd0 0x7fc6500c7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:17.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:17.836+0000 7fc64f59e700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6500ca480 con 0x7fc650142fd0 2023-10-06T02:26:17.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:17.836+0000 7fc64f59e700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc65015e240 con 0x7fc650142c00 2023-10-06T02:26:17.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:17.836+0000 7fc64f59e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc65006b350 con 0x7fc65006bd20 2023-10-06T02:26:17.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:17.836+0000 7fc64ed9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc650142fd0 0x7fc6500c7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:17.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:17.836+0000 7fc64ed9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc650142fd0 0x7fc6500c7dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49674/0 (socket says 172.21.15.31:49674) 2023-10-06T02:26:17.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:17.836+0000 7fc64ed9d700 1 -- 172.21.15.31:0/4235305461 learned_addr learned my addr 172.21.15.31:0/4235305461 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:26:17.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:17.837+0000 7fc64e59c700 1 --2- 172.21.15.31:0/4235305461 >> v2:172.21.15.62:3300/0 conn(0x7fc650142c00 0x7fc65006b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:17.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:17.837+0000 7fc64dd9b700 1 --2- 172.21.15.31:0/4235305461 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc65006bd20 0x7fc6500c7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:17.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:17.837+0000 7fc64ed9d700 1 -- 172.21.15.31:0/4235305461 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc65006bd20 msgr2=0x7fc6500c7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:26:17.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:17.837+0000 7fc64ed9d700 1 --2- 172.21.15.31:0/4235305461 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc65006bd20 0x7fc6500c7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:17.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:17.837+0000 7fc64ed9d700 1 -- 172.21.15.31:0/4235305461 >> v2:172.21.15.62:3300/0 conn(0x7fc650142c00 msgr2=0x7fc65006b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:26:17.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:17.837+0000 7fc64ed9d700 1 --2- 172.21.15.31:0/4235305461 >> v2:172.21.15.62:3300/0 conn(0x7fc650142c00 0x7fc65006b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:17.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:17.837+0000 7fc64ed9d700 1 -- 172.21.15.31:0/4235305461 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc65015df60 con 0x7fc650142fd0 2023-10-06T02:26:17.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:17.837+0000 7fc64e59c700 1 --2- 172.21.15.31:0/4235305461 >> v2:172.21.15.62:3300/0 conn(0x7fc650142c00 0x7fc65006b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:26:17.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:17.837+0000 7fc64ed9d700 1 --2- 172.21.15.31:0/4235305461 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc650142fd0 0x7fc6500c7dc0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fc65006f490 tx=0x7fc648009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b09132de5703521f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:17.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:17.837+0000 7fc64d59a700 1 -- 172.21.15.31:0/4235305461 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc64800e070 con 0x7fc650142fd0 2023-10-06T02:26:17.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:17.837+0000 7fc64d59a700 1 -- 172.21.15.31:0/4235305461 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc64800ba50 con 0x7fc650142fd0 2023-10-06T02:26:17.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:17.837+0000 7fc64d59a700 1 -- 172.21.15.31:0/4235305461 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6480092e0 con 0x7fc650142fd0 2023-10-06T02:26:17.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:17.838+0000 7fc64f59e700 1 -- 172.21.15.31:0/4235305461 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc650142fd0 msgr2=0x7fc6500c7dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:17.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:17.838+0000 7fc64f59e700 1 --2- 172.21.15.31:0/4235305461 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc650142fd0 0x7fc6500c7dc0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fc65006f490 tx=0x7fc648009840 comp rx=0 tx=0).stop 2023-10-06T02:26:17.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:17.838+0000 7fc64f59e700 1 -- 172.21.15.31:0/4235305461 shutdown_connections 2023-10-06T02:26:17.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:17.838+0000 7fc64f59e700 1 --2- 172.21.15.31:0/4235305461 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc650142fd0 0x7fc6500c7dc0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:17.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:17.838+0000 7fc64f59e700 1 --2- 172.21.15.31:0/4235305461 >> v2:172.21.15.62:3300/0 conn(0x7fc650142c00 0x7fc65006b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:17.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:17.838+0000 7fc64f59e700 1 --2- 172.21.15.31:0/4235305461 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc65006bd20 0x7fc6500c7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:17.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:17.838+0000 7fc64f59e700 1 -- 172.21.15.31:0/4235305461 >> 172.21.15.31:0/4235305461 conn(0x7fc6500675a0 msgr2=0x7fc6500679a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:17.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:17.838+0000 7fc64f59e700 1 -- 172.21.15.31:0/4235305461 shutdown_connections 2023-10-06T02:26:17.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:17.838+0000 7fc64f59e700 1 -- 172.21.15.31:0/4235305461 wait complete. 2023-10-06T02:26:17.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:17.840+0000 7fc64f59e700 1 Processor -- start 2023-10-06T02:26:17.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:17.840+0000 7fc64f59e700 1 -- start start 2023-10-06T02:26:17.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:17.840+0000 7fc64f59e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc65006bd20 0x7fc6500c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:17.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:17.841+0000 7fc64f59e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc650142c00 0x7fc650140ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:17.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:17.841+0000 7fc64f59e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc650142fd0 0x7fc650141420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:17.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:17.841+0000 7fc64f59e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc65016cca0 con 0x7fc650142c00 2023-10-06T02:26:17.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:17.841+0000 7fc64f59e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6500b1890 con 0x7fc65006bd20 2023-10-06T02:26:17.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:17.841+0000 7fc64f59e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc65016cef0 con 0x7fc650142fd0 2023-10-06T02:26:17.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:17.841+0000 7fc64dd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc650142c00 0x7fc650140ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:17.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:17.841+0000 7fc64dd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc650142c00 0x7fc650140ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49684/0 (socket says 172.21.15.31:49684) 2023-10-06T02:26:17.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:17.841+0000 7fc64dd9b700 1 -- 172.21.15.31:0/2928316397 learned_addr learned my addr 172.21.15.31:0/2928316397 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:26:17.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:17.841+0000 7fc64e59c700 1 --2- 172.21.15.31:0/2928316397 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc65006bd20 0x7fc6500c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:17.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:17.841+0000 7fc64ed9d700 1 --2- 172.21.15.31:0/2928316397 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc650142fd0 0x7fc650141420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:17.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:17.841+0000 7fc64dd9b700 1 -- 172.21.15.31:0/2928316397 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc650142fd0 msgr2=0x7fc650141420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:17.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:17.841+0000 7fc64dd9b700 1 --2- 172.21.15.31:0/2928316397 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc650142fd0 0x7fc650141420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:17.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:17.841+0000 7fc64dd9b700 1 -- 172.21.15.31:0/2928316397 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc65006bd20 msgr2=0x7fc6500c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:17.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:17.841+0000 7fc64dd9b700 1 --2- 172.21.15.31:0/2928316397 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc65006bd20 0x7fc6500c6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:17.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:17.841+0000 7fc64dd9b700 1 -- 172.21.15.31:0/2928316397 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc648009ae0 con 0x7fc650142c00 2023-10-06T02:26:17.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:17.842+0000 7fc64dd9b700 1 --2- 172.21.15.31:0/2928316397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc650142c00 0x7fc650140ee0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fc64400d890 tx=0x7fc644008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:17.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:17.842+0000 7fc63e7fc700 1 -- 172.21.15.31:0/2928316397 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6440125f0 con 0x7fc650142c00 2023-10-06T02:26:17.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:17.842+0000 7fc63e7fc700 1 -- 172.21.15.31:0/2928316397 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc644012770 con 0x7fc650142c00 2023-10-06T02:26:17.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:17.842+0000 7fc64f59e700 1 -- 172.21.15.31:0/2928316397 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc65016d1d0 con 0x7fc650142c00 2023-10-06T02:26:17.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:17.842+0000 7fc63e7fc700 1 -- 172.21.15.31:0/2928316397 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc644012b30 con 0x7fc650142c00 2023-10-06T02:26:17.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:17.842+0000 7fc64f59e700 1 -- 172.21.15.31:0/2928316397 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc65016d790 con 0x7fc650142c00 2023-10-06T02:26:17.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:17.843+0000 7fc64f59e700 1 -- 172.21.15.31:0/2928316397 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc65004c3c0 con 0x7fc650142c00 2023-10-06T02:26:17.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:17.844+0000 7fc63e7fc700 1 -- 172.21.15.31:0/2928316397 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 84944+0+0 (secure 0 0 0) 0x7fc64401e460 con 0x7fc650142c00 2023-10-06T02:26:17.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:17.845+0000 7fc63e7fc700 1 --2- 172.21.15.31:0/2928316397 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc630067170 0x7fc630069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:17.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:17.845+0000 7fc63e7fc700 1 -- 172.21.15.31:0/2928316397 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6116+0+0 (secure 0 0 0) 0x7fc644084fe0 con 0x7fc650142c00 2023-10-06T02:26:17.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:17.845+0000 7fc64e59c700 1 --2- 172.21.15.31:0/2928316397 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc630067170 0x7fc630069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:17.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:17.846+0000 7fc64e59c700 1 --2- 172.21.15.31:0/2928316397 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc630067170 0x7fc630069620 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc640006f00 tx=0x7fc640008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:17.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:17.849+0000 7fc63e7fc700 1 -- 172.21.15.31:0/2928316397 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc644051de0 con 0x7fc650142c00 2023-10-06T02:26:18.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.096+0000 7fc64f59e700 1 -- 172.21.15.31:0/2928316397 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc65006ad00 con 0x7fc650142c00 2023-10-06T02:26:18.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.097+0000 7fc63e7fc700 1 -- 172.21.15.31:0/2928316397 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4594 (secure 0 0 0) 0x7fc644055400 con 0x7fc650142c00 2023-10-06T02:26:18.098 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:26:18.099 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4949,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/178547366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":178547366},{"type":"v1","addr":"172.21.15.62:6835","nonce":178547366}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":5278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/2578831517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":2578831517},{"type":"v1","addr":"172.21.15.31:6836","nonce":2578831517}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5287,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3502585737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":3502585737},{"type":"v1","addr":"172.21.15.31:6837","nonce":3502585737}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:24:43.872314+0000","modified":"2023-10-06T02:24:47.404607+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5290":{"gid":5290,"name":"b","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6834/2358776861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2358776861},{"type":"v1","addr":"172.21.15.62:6834","nonce":2358776861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T02:26:18.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.099+0000 7fc64f59e700 1 -- 172.21.15.31:0/2928316397 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc630067170 msgr2=0x7fc630069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:18.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.099+0000 7fc64f59e700 1 --2- 172.21.15.31:0/2928316397 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc630067170 0x7fc630069620 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc640006f00 tx=0x7fc640008040 comp rx=0 tx=0).stop 2023-10-06T02:26:18.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.099+0000 7fc64f59e700 1 -- 172.21.15.31:0/2928316397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc650142c00 msgr2=0x7fc650140ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:18.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.099+0000 7fc64f59e700 1 --2- 172.21.15.31:0/2928316397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc650142c00 0x7fc650140ee0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fc64400d890 tx=0x7fc644008040 comp rx=0 tx=0).stop 2023-10-06T02:26:18.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.099+0000 7fc64f59e700 1 -- 172.21.15.31:0/2928316397 shutdown_connections 2023-10-06T02:26:18.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.099+0000 7fc64f59e700 1 --2- 172.21.15.31:0/2928316397 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fc630067170 0x7fc630069620 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:18.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.099+0000 7fc64f59e700 1 --2- 172.21.15.31:0/2928316397 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc650142fd0 0x7fc650141420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:18.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.099+0000 7fc64f59e700 1 --2- 172.21.15.31:0/2928316397 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc65006bd20 0x7fc6500c6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:18.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.099+0000 7fc64f59e700 1 --2- 172.21.15.31:0/2928316397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc650142c00 0x7fc650140ee0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:18.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.099+0000 7fc64f59e700 1 -- 172.21.15.31:0/2928316397 >> 172.21.15.31:0/2928316397 conn(0x7fc6500675a0 msgr2=0x7fc6500ca8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:18.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.100+0000 7fc64f59e700 1 -- 172.21.15.31:0/2928316397 shutdown_connections 2023-10-06T02:26:18.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.100+0000 7fc64f59e700 1 -- 172.21.15.31:0/2928316397 wait complete. 2023-10-06T02:26:18.103 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 27 2023-10-06T02:26:18.119 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json session ls 2023-10-06T02:26:18.340 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-10-06T02:26:18.340 DEBUG:tasks.cephfs.filesystem:_json_asok output [] 2023-10-06T02:26:18.340 DEBUG:tasks.ceph_test_case:wait_until_true: success in 70s and 0 retries 2023-10-06T02:26:18.340 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T02:26:18.340 DEBUG:teuthology.orchestra.run.smithi031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T02:26:18.423 DEBUG:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T02:26:18.506 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T02:26:18.506 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:26:18.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.692+0000 7fca7359e700 1 Processor -- start 2023-10-06T02:26:18.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.692+0000 7fca7359e700 1 -- start start 2023-10-06T02:26:18.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.693+0000 7fca7359e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca7415a8c0 0x7fca7406d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:18.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.693+0000 7fca7359e700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fca7406ddd0 0x7fca740cf0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:18.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.693+0000 7fca7359e700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca740cec30 con 0x7fca740b2980 2023-10-06T02:26:18.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.693+0000 7fca7359e700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca7406e270 con 0x7fca7406ddd0 2023-10-06T02:26:18.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.693+0000 7fca7359e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca740ce930 con 0x7fca7415a8c0 2023-10-06T02:26:18.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.693+0000 7fca7259c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fca740b2980 0x7fca740cce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60584/0 (socket says 172.21.15.31:60584) 2023-10-06T02:26:18.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.694+0000 7fca7259c700 1 -- 172.21.15.31:0/2243030555 learned_addr learned my addr 172.21.15.31:0/2243030555 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:26:18.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.694+0000 7fca71d9b700 1 --2- 172.21.15.31:0/2243030555 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca7415a8c0 0x7fca7406d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:18.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.694+0000 7fca72d9d700 1 --2- 172.21.15.31:0/2243030555 >> v2:172.21.15.62:3300/0 conn(0x7fca7406ddd0 0x7fca740cf0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:18.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.694+0000 7fca7159a700 1 -- 172.21.15.31:0/2243030555 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1298552255 0 0) 0x7fca740cec30 con 0x7fca740b2980 2023-10-06T02:26:18.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.694+0000 7fca7159a700 1 -- 172.21.15.31:0/2243030555 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca5c003570 con 0x7fca740b2980 2023-10-06T02:26:18.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.694+0000 7fca71d9b700 1 -- 172.21.15.31:0/2243030555 >> v2:172.21.15.62:3300/0 conn(0x7fca7406ddd0 msgr2=0x7fca740cf0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:26:18.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.694+0000 7fca71d9b700 1 --2- 172.21.15.31:0/2243030555 >> v2:172.21.15.62:3300/0 conn(0x7fca7406ddd0 0x7fca740cf0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:18.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.694+0000 7fca71d9b700 1 -- 172.21.15.31:0/2243030555 >> v1:172.21.15.31:6789/0 conn(0x7fca740b2980 legacy=0x7fca740cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:18.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.694+0000 7fca71d9b700 1 -- 172.21.15.31:0/2243030555 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca740cd510 con 0x7fca7415a8c0 2023-10-06T02:26:18.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.694+0000 7fca72d9d700 1 --2- 172.21.15.31:0/2243030555 >> v2:172.21.15.62:3300/0 conn(0x7fca7406ddd0 0x7fca740cf0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:26:18.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.694+0000 7fca71d9b700 1 --2- 172.21.15.31:0/2243030555 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca7415a8c0 0x7fca7406d890 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fca680098e0 tx=0x7fca680091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1975b54217c786d8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:18.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.695+0000 7fca7159a700 1 -- 172.21.15.31:0/2243030555 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca68011630 con 0x7fca7415a8c0 2023-10-06T02:26:18.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.695+0000 7fca7159a700 1 -- 172.21.15.31:0/2243030555 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca68011790 con 0x7fca7415a8c0 2023-10-06T02:26:18.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.695+0000 7fca7159a700 1 -- 172.21.15.31:0/2243030555 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca68011b30 con 0x7fca7415a8c0 2023-10-06T02:26:18.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.695+0000 7fca7359e700 1 -- 172.21.15.31:0/2243030555 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca7415a8c0 msgr2=0x7fca7406d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:18.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.695+0000 7fca7359e700 1 --2- 172.21.15.31:0/2243030555 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca7415a8c0 0x7fca7406d890 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fca680098e0 tx=0x7fca680091a0 comp rx=0 tx=0).stop 2023-10-06T02:26:18.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.695+0000 7fca7359e700 1 -- 172.21.15.31:0/2243030555 shutdown_connections 2023-10-06T02:26:18.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.695+0000 7fca7359e700 1 --2- 172.21.15.31:0/2243030555 >> v2:172.21.15.62:3300/0 conn(0x7fca7406ddd0 0x7fca740cf0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:18.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.695+0000 7fca7359e700 1 --2- 172.21.15.31:0/2243030555 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca7415a8c0 0x7fca7406d890 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:18.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.695+0000 7fca7359e700 1 -- 172.21.15.31:0/2243030555 >> 172.21.15.31:0/2243030555 conn(0x7fca740703a0 msgr2=0x7fca740707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:18.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.695+0000 7fca7359e700 1 -- 172.21.15.31:0/2243030555 shutdown_connections 2023-10-06T02:26:18.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.695+0000 7fca7359e700 1 -- 172.21.15.31:0/2243030555 wait complete. 2023-10-06T02:26:18.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.697+0000 7fca7359e700 1 Processor -- start 2023-10-06T02:26:18.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.697+0000 7fca7359e700 1 -- start start 2023-10-06T02:26:18.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.698+0000 7fca7359e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca7406ddd0 0x7fca740c8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:18.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.698+0000 7fca7359e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca740b2980 0x7fca741698f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:18.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.698+0000 7fca7359e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca7415a8c0 0x7fca74169e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:18.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.698+0000 7fca7359e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca7414e980 con 0x7fca7406ddd0 2023-10-06T02:26:18.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.698+0000 7fca7359e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca7416d520 con 0x7fca740b2980 2023-10-06T02:26:18.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.698+0000 7fca7359e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca740c9250 con 0x7fca7415a8c0 2023-10-06T02:26:18.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.698+0000 7fca7259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca7406ddd0 0x7fca740c8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:18.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.698+0000 7fca7259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca7406ddd0 0x7fca740c8d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57088/0 (socket says 172.21.15.31:57088) 2023-10-06T02:26:18.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.698+0000 7fca7259c700 1 -- 172.21.15.31:0/3587258199 learned_addr learned my addr 172.21.15.31:0/3587258199 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:26:18.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.698+0000 7fca71d9b700 1 --2- 172.21.15.31:0/3587258199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca740b2980 0x7fca741698f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:18.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.698+0000 7fca72d9d700 1 --2- 172.21.15.31:0/3587258199 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca7415a8c0 0x7fca74169e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:18.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.698+0000 7fca7259c700 1 -- 172.21.15.31:0/3587258199 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca7415a8c0 msgr2=0x7fca74169e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:18.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.698+0000 7fca7259c700 1 --2- 172.21.15.31:0/3587258199 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca7415a8c0 0x7fca74169e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:18.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.698+0000 7fca7259c700 1 -- 172.21.15.31:0/3587258199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca740b2980 msgr2=0x7fca741698f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:18.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.698+0000 7fca7259c700 1 --2- 172.21.15.31:0/3587258199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca740b2980 0x7fca741698f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:18.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.698+0000 7fca7259c700 1 -- 172.21.15.31:0/3587258199 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca6c009a20 con 0x7fca7406ddd0 2023-10-06T02:26:18.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.699+0000 7fca7259c700 1 --2- 172.21.15.31:0/3587258199 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca7406ddd0 0x7fca740c8d10 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fca6400c7f0 tx=0x7fca64007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:18.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.699+0000 7fca627fc700 1 -- 172.21.15.31:0/3587258199 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca64005b30 con 0x7fca7406ddd0 2023-10-06T02:26:18.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.699+0000 7fca627fc700 1 -- 172.21.15.31:0/3587258199 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca64005cb0 con 0x7fca7406ddd0 2023-10-06T02:26:18.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.699+0000 7fca7359e700 1 -- 172.21.15.31:0/3587258199 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca68009420 con 0x7fca7406ddd0 2023-10-06T02:26:18.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.699+0000 7fca627fc700 1 -- 172.21.15.31:0/3587258199 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca64007320 con 0x7fca7406ddd0 2023-10-06T02:26:18.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.699+0000 7fca7359e700 1 -- 172.21.15.31:0/3587258199 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca7416cef0 con 0x7fca7406ddd0 2023-10-06T02:26:18.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.700+0000 7fca7359e700 1 -- 172.21.15.31:0/3587258199 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca7404c3c0 con 0x7fca7406ddd0 2023-10-06T02:26:18.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.701+0000 7fca627fc700 1 -- 172.21.15.31:0/3587258199 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 84944+0+0 (secure 0 0 0) 0x7fca6401b3b0 con 0x7fca7406ddd0 2023-10-06T02:26:18.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.702+0000 7fca627fc700 1 --2- 172.21.15.31:0/3587258199 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fca54067170 0x7fca54069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:18.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.702+0000 7fca627fc700 1 -- 172.21.15.31:0/3587258199 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6116+0+0 (secure 0 0 0) 0x7fca64083f80 con 0x7fca7406ddd0 2023-10-06T02:26:18.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.702+0000 7fca71d9b700 1 --2- 172.21.15.31:0/3587258199 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fca54067170 0x7fca54069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:18.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.703+0000 7fca71d9b700 1 --2- 172.21.15.31:0/3587258199 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fca54067170 0x7fca54069620 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fca680091e0 tx=0x7fca68017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:18.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.706+0000 7fca627fc700 1 -- 172.21.15.31:0/3587258199 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fca64050db0 con 0x7fca7406ddd0 2023-10-06T02:26:18.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.953+0000 7fca7359e700 1 -- 172.21.15.31:0/3587258199 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fca7416d800 con 0x7fca7406ddd0 2023-10-06T02:26:18.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.954+0000 7fca627fc700 1 -- 172.21.15.31:0/3587258199 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4594 (secure 0 0 0) 0x7fca64017020 con 0x7fca7406ddd0 2023-10-06T02:26:18.955 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:26:18.955 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4949,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/178547366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":178547366},{"type":"v1","addr":"172.21.15.62:6835","nonce":178547366}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":5278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/2578831517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":2578831517},{"type":"v1","addr":"172.21.15.31:6836","nonce":2578831517}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5287,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3502585737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":3502585737},{"type":"v1","addr":"172.21.15.31:6837","nonce":3502585737}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:24:43.872314+0000","modified":"2023-10-06T02:24:47.404607+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5290":{"gid":5290,"name":"b","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6834/2358776861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2358776861},{"type":"v1","addr":"172.21.15.62:6834","nonce":2358776861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T02:26:18.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.956+0000 7fca7359e700 1 -- 172.21.15.31:0/3587258199 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fca54067170 msgr2=0x7fca54069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:18.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.956+0000 7fca7359e700 1 --2- 172.21.15.31:0/3587258199 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fca54067170 0x7fca54069620 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fca680091e0 tx=0x7fca68017000 comp rx=0 tx=0).stop 2023-10-06T02:26:18.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.956+0000 7fca7359e700 1 -- 172.21.15.31:0/3587258199 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca7406ddd0 msgr2=0x7fca740c8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:18.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.956+0000 7fca7359e700 1 --2- 172.21.15.31:0/3587258199 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca7406ddd0 0x7fca740c8d10 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fca6400c7f0 tx=0x7fca64007e00 comp rx=0 tx=0).stop 2023-10-06T02:26:18.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.956+0000 7fca7359e700 1 -- 172.21.15.31:0/3587258199 shutdown_connections 2023-10-06T02:26:18.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.956+0000 7fca7359e700 1 --2- 172.21.15.31:0/3587258199 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fca54067170 0x7fca54069620 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:18.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.956+0000 7fca7359e700 1 --2- 172.21.15.31:0/3587258199 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca7415a8c0 0x7fca74169e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:18.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.956+0000 7fca7359e700 1 --2- 172.21.15.31:0/3587258199 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca7406ddd0 0x7fca740c8d10 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:18.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.956+0000 7fca7359e700 1 --2- 172.21.15.31:0/3587258199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca740b2980 0x7fca741698f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:18.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.956+0000 7fca7359e700 1 -- 172.21.15.31:0/3587258199 >> 172.21.15.31:0/3587258199 conn(0x7fca740703a0 msgr2=0x7fca740cfbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:18.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.956+0000 7fca7359e700 1 -- 172.21.15.31:0/3587258199 shutdown_connections 2023-10-06T02:26:18.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:18.956+0000 7fca7359e700 1 -- 172.21.15.31:0/3587258199 wait complete. 2023-10-06T02:26:18.959 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 27 2023-10-06T02:26:18.975 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:26:19.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.161+0000 7fd1d759e700 1 Processor -- start 2023-10-06T02:26:19.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.161+0000 7fd1d759e700 1 -- start start 2023-10-06T02:26:19.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.162+0000 7fd1d759e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd1d813fb20 0x7fd1d815a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:19.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.162+0000 7fd1d759e700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1d815ae70 con 0x7fd1d813fef0 2023-10-06T02:26:19.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.162+0000 7fd1d759e700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1d806af60 con 0x7fd1d806aaa0 2023-10-06T02:26:19.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.162+0000 7fd1d759e700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1d806b240 con 0x7fd1d813fb20 2023-10-06T02:26:19.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.162+0000 7fd1d659c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd1d813fb20 0x7fd1d815a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:19.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.162+0000 7fd1d659c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd1d813fb20 0x7fd1d815a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57094/0 (socket says 172.21.15.31:57094) 2023-10-06T02:26:19.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.162+0000 7fd1d659c700 1 -- 172.21.15.31:0/374658025 learned_addr learned my addr 172.21.15.31:0/374658025 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:26:19.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.163+0000 7fd1d659c700 1 -- 172.21.15.31:0/374658025 >> v1:172.21.15.62:6789/0 conn(0x7fd1d806aaa0 legacy=0x7fd1d80c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:19.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.163+0000 7fd1d659c700 1 -- 172.21.15.31:0/374658025 >> v1:172.21.15.31:6789/0 conn(0x7fd1d813fef0 legacy=0x7fd1d80c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:19.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.163+0000 7fd1d659c700 1 -- 172.21.15.31:0/374658025 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1d806b520 con 0x7fd1d813fb20 2023-10-06T02:26:19.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.163+0000 7fd1d659c700 1 --2- 172.21.15.31:0/374658025 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1d813fb20 0x7fd1d815a930 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fd1c80099b0 tx=0x7fd1c8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92e2eb5194051c45 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:19.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.163+0000 7fd1d559a700 1 -- 172.21.15.31:0/374658025 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c800b820 con 0x7fd1d813fb20 2023-10-06T02:26:19.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.163+0000 7fd1d559a700 1 -- 172.21.15.31:0/374658025 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1c800b980 con 0x7fd1d813fb20 2023-10-06T02:26:19.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.163+0000 7fd1d559a700 1 -- 172.21.15.31:0/374658025 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c800bd20 con 0x7fd1d813fb20 2023-10-06T02:26:19.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.164+0000 7fd1d759e700 1 -- 172.21.15.31:0/374658025 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1d813fb20 msgr2=0x7fd1d815a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:19.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.164+0000 7fd1d759e700 1 --2- 172.21.15.31:0/374658025 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1d813fb20 0x7fd1d815a930 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fd1c80099b0 tx=0x7fd1c8009250 comp rx=0 tx=0).stop 2023-10-06T02:26:19.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.164+0000 7fd1d759e700 1 -- 172.21.15.31:0/374658025 shutdown_connections 2023-10-06T02:26:19.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.164+0000 7fd1d759e700 1 --2- 172.21.15.31:0/374658025 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1d813fb20 0x7fd1d815a930 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:19.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.164+0000 7fd1d759e700 1 -- 172.21.15.31:0/374658025 >> 172.21.15.31:0/374658025 conn(0x7fd1d806d170 msgr2=0x7fd1d80cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:19.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.164+0000 7fd1d759e700 1 -- 172.21.15.31:0/374658025 shutdown_connections 2023-10-06T02:26:19.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.164+0000 7fd1d759e700 1 -- 172.21.15.31:0/374658025 wait complete. 2023-10-06T02:26:19.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.166+0000 7fd1d759e700 1 Processor -- start 2023-10-06T02:26:19.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.166+0000 7fd1d759e700 1 -- start start 2023-10-06T02:26:19.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.167+0000 7fd1d759e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1d806aaa0 0x7fd1d813cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:19.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.167+0000 7fd1d759e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1d813fb20 0x7fd1d80b0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:19.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.167+0000 7fd1d759e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd1d813fef0 0x7fd1d8156950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:19.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.167+0000 7fd1d759e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d8158890 con 0x7fd1d806aaa0 2023-10-06T02:26:19.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.167+0000 7fd1d759e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d81596b0 con 0x7fd1d813fb20 2023-10-06T02:26:19.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.167+0000 7fd1d759e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d813d4d0 con 0x7fd1d813fef0 2023-10-06T02:26:19.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.167+0000 7fd1d659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1d806aaa0 0x7fd1d813cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:19.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.167+0000 7fd1d659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1d806aaa0 0x7fd1d813cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57108/0 (socket says 172.21.15.31:57108) 2023-10-06T02:26:19.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.167+0000 7fd1d659c700 1 -- 172.21.15.31:0/3051106529 learned_addr learned my addr 172.21.15.31:0/3051106529 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:26:19.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.167+0000 7fd1d5d9b700 1 --2- 172.21.15.31:0/3051106529 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1d813fb20 0x7fd1d80b0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:19.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.167+0000 7fd1d6d9d700 1 --2- 172.21.15.31:0/3051106529 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd1d813fef0 0x7fd1d8156950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:19.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.167+0000 7fd1d659c700 1 -- 172.21.15.31:0/3051106529 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd1d813fef0 msgr2=0x7fd1d8156950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:19.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.167+0000 7fd1d659c700 1 --2- 172.21.15.31:0/3051106529 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd1d813fef0 0x7fd1d8156950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:19.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.167+0000 7fd1d659c700 1 -- 172.21.15.31:0/3051106529 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1d813fb20 msgr2=0x7fd1d80b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:19.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.167+0000 7fd1d659c700 1 --2- 172.21.15.31:0/3051106529 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1d813fb20 0x7fd1d80b0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:19.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.168+0000 7fd1d659c700 1 -- 172.21.15.31:0/3051106529 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1c80094f0 con 0x7fd1d806aaa0 2023-10-06T02:26:19.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.168+0000 7fd1d659c700 1 --2- 172.21.15.31:0/3051106529 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1d806aaa0 0x7fd1d813cf90 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fd1c800c010 tx=0x7fd1c8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:19.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.168+0000 7fd1c67fc700 1 -- 172.21.15.31:0/3051106529 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c801b450 con 0x7fd1d806aaa0 2023-10-06T02:26:19.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.168+0000 7fd1c67fc700 1 -- 172.21.15.31:0/3051106529 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1c8020070 con 0x7fd1d806aaa0 2023-10-06T02:26:19.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.168+0000 7fd1c67fc700 1 -- 172.21.15.31:0/3051106529 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c801b7e0 con 0x7fd1d806aaa0 2023-10-06T02:26:19.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.168+0000 7fd1d759e700 1 -- 172.21.15.31:0/3051106529 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1d8159b40 con 0x7fd1d806aaa0 2023-10-06T02:26:19.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.168+0000 7fd1d759e700 1 -- 172.21.15.31:0/3051106529 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1d8158d80 con 0x7fd1d806aaa0 2023-10-06T02:26:19.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.169+0000 7fd1d759e700 1 -- 172.21.15.31:0/3051106529 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1d804c3c0 con 0x7fd1d806aaa0 2023-10-06T02:26:19.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.170+0000 7fd1c67fc700 1 -- 172.21.15.31:0/3051106529 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 84944+0+0 (secure 0 0 0) 0x7fd1c8022030 con 0x7fd1d806aaa0 2023-10-06T02:26:19.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.171+0000 7fd1c67fc700 1 --2- 172.21.15.31:0/3051106529 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fd1b8067170 0x7fd1b8069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:19.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.171+0000 7fd1c67fc700 1 -- 172.21.15.31:0/3051106529 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6116+0+0 (secure 0 0 0) 0x7fd1c8086aa0 con 0x7fd1d806aaa0 2023-10-06T02:26:19.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.171+0000 7fd1d5d9b700 1 --2- 172.21.15.31:0/3051106529 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fd1b8067170 0x7fd1b8069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:19.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.172+0000 7fd1d5d9b700 1 --2- 172.21.15.31:0/3051106529 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fd1b8067170 0x7fd1b8069620 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd1d80c12a0 tx=0x7fd1cc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:19.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.175+0000 7fd1c67fc700 1 -- 172.21.15.31:0/3051106529 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd1c8053880 con 0x7fd1d806aaa0 2023-10-06T02:26:19.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.224+0000 7fd1c67fc700 1 -- 172.21.15.31:0/3051106529 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mgrmap(e 9) v1 ==== 84893+0+0 (secure 0 0 0) 0x7fd1c8053c50 con 0x7fd1d806aaa0 2023-10-06T02:26:19.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.381+0000 7fd1d759e700 1 -- 172.21.15.31:0/3051106529 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd1d804c3c0 con 0x7fd1d806aaa0 2023-10-06T02:26:19.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.382+0000 7fd1c67fc700 1 -- 172.21.15.31:0/3051106529 <== mon.0 v2:172.21.15.31:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+16914 (secure 0 0 0) 0x7fd1c80572d0 con 0x7fd1d806aaa0 2023-10-06T02:26:19.384 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:26:19.384 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":49,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:25:48.260740+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:24:42.035212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-10-06T02:24:42.849819+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.2f","mappings":[{"from":0,"to":6}]},{"pgid":"8.3d","mappings":[{"from":0,"to":6}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:26:19.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.385+0000 7fd1d759e700 1 -- 172.21.15.31:0/3051106529 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fd1b8067170 msgr2=0x7fd1b8069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:19.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.385+0000 7fd1d759e700 1 --2- 172.21.15.31:0/3051106529 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fd1b8067170 0x7fd1b8069620 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd1d80c12a0 tx=0x7fd1cc009040 comp rx=0 tx=0).stop 2023-10-06T02:26:19.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.385+0000 7fd1d759e700 1 -- 172.21.15.31:0/3051106529 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1d806aaa0 msgr2=0x7fd1d813cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:19.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.385+0000 7fd1d759e700 1 --2- 172.21.15.31:0/3051106529 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1d806aaa0 0x7fd1d813cf90 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fd1c800c010 tx=0x7fd1c8016000 comp rx=0 tx=0).stop 2023-10-06T02:26:19.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.385+0000 7fd1d759e700 1 -- 172.21.15.31:0/3051106529 shutdown_connections 2023-10-06T02:26:19.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.385+0000 7fd1d759e700 1 --2- 172.21.15.31:0/3051106529 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fd1b8067170 0x7fd1b8069620 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:19.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.385+0000 7fd1d759e700 1 --2- 172.21.15.31:0/3051106529 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd1d813fef0 0x7fd1d8156950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:19.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.385+0000 7fd1d759e700 1 --2- 172.21.15.31:0/3051106529 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1d806aaa0 0x7fd1d813cf90 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:19.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.385+0000 7fd1d759e700 1 --2- 172.21.15.31:0/3051106529 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1d813fb20 0x7fd1d80b0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:19.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.385+0000 7fd1d759e700 1 -- 172.21.15.31:0/3051106529 >> 172.21.15.31:0/3051106529 conn(0x7fd1d806d170 msgr2=0x7fd1d813d890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:19.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.385+0000 7fd1d759e700 1 -- 172.21.15.31:0/3051106529 shutdown_connections 2023-10-06T02:26:19.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.385+0000 7fd1d759e700 1 -- 172.21.15.31:0/3051106529 wait complete. 2023-10-06T02:26:19.405 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T02:22:53.183425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T02:23:00.046967+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T02:24:42.035212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T02:24:42.849819+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '47', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T02:26:19.406 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T02:26:19.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.594+0000 7fe1ee59d700 1 Processor -- start 2023-10-06T02:26:19.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.594+0000 7fe1ee59d700 1 -- start start 2023-10-06T02:26:19.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.595+0000 7fe1ee59d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe1e80c7be0 0x7fe1e806abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:19.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.595+0000 7fe1ee59d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe1e80c7fb0 0x7fe1e815a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:19.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.595+0000 7fe1ee59d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe1e813f780 0x7fe1e813fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:19.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.595+0000 7fe1ee59d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1e806bf10 con 0x7fe1e80c7be0 2023-10-06T02:26:19.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.595+0000 7fe1ee59d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1e81400e0 con 0x7fe1e80c7fb0 2023-10-06T02:26:19.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.595+0000 7fe1ee59d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e815aed0 con 0x7fe1e813f780 2023-10-06T02:26:19.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.595+0000 7fe1ed59b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe1e80c7be0 0x7fe1e806abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:19.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.596+0000 7fe1ed59b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe1e80c7be0 0x7fe1e806abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57122/0 (socket says 172.21.15.31:57122) 2023-10-06T02:26:19.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.596+0000 7fe1ed59b700 1 -- 172.21.15.31:0/3334721318 learned_addr learned my addr 172.21.15.31:0/3334721318 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:26:19.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.596+0000 7fe1edd9c700 1 --2- 172.21.15.31:0/3334721318 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe1e813f780 0x7fe1e813fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:19.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.596+0000 7fe1ecd9a700 1 --2- 172.21.15.31:0/3334721318 >> v2:172.21.15.62:3300/0 conn(0x7fe1e80c7fb0 0x7fe1e815a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:19.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.596+0000 7fe1ed59b700 1 -- 172.21.15.31:0/3334721318 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe1e813f780 msgr2=0x7fe1e813fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:26:19.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.596+0000 7fe1ed59b700 1 --2- 172.21.15.31:0/3334721318 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe1e813f780 0x7fe1e813fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:19.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.596+0000 7fe1ed59b700 1 -- 172.21.15.31:0/3334721318 >> v2:172.21.15.62:3300/0 conn(0x7fe1e80c7fb0 msgr2=0x7fe1e815a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:26:19.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.596+0000 7fe1ed59b700 1 --2- 172.21.15.31:0/3334721318 >> v2:172.21.15.62:3300/0 conn(0x7fe1e80c7fb0 0x7fe1e815a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:19.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.596+0000 7fe1ed59b700 1 -- 172.21.15.31:0/3334721318 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1e806b290 con 0x7fe1e80c7be0 2023-10-06T02:26:19.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.596+0000 7fe1ecd9a700 1 --2- 172.21.15.31:0/3334721318 >> v2:172.21.15.62:3300/0 conn(0x7fe1e80c7fb0 0x7fe1e815a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:26:19.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.596+0000 7fe1ed59b700 1 --2- 172.21.15.31:0/3334721318 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1e80c7be0 0x7fe1e806abd0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fe1e80aec50 tx=0x7fe1e4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e75811f2f0a848ae server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:19.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.597+0000 7fe1dffff700 1 -- 172.21.15.31:0/3334721318 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e4009e70 con 0x7fe1e80c7be0 2023-10-06T02:26:19.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.597+0000 7fe1dffff700 1 -- 172.21.15.31:0/3334721318 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1e400b860 con 0x7fe1e80c7be0 2023-10-06T02:26:19.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.597+0000 7fe1dffff700 1 -- 172.21.15.31:0/3334721318 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e400bc00 con 0x7fe1e80c7be0 2023-10-06T02:26:19.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.597+0000 7fe1ee59d700 1 -- 172.21.15.31:0/3334721318 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1e80c7be0 msgr2=0x7fe1e806abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:19.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.597+0000 7fe1ee59d700 1 --2- 172.21.15.31:0/3334721318 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1e80c7be0 0x7fe1e806abd0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fe1e80aec50 tx=0x7fe1e4009250 comp rx=0 tx=0).stop 2023-10-06T02:26:19.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.597+0000 7fe1ee59d700 1 -- 172.21.15.31:0/3334721318 shutdown_connections 2023-10-06T02:26:19.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.597+0000 7fe1ee59d700 1 --2- 172.21.15.31:0/3334721318 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe1e813f780 0x7fe1e813fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:19.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.597+0000 7fe1ee59d700 1 --2- 172.21.15.31:0/3334721318 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1e80c7be0 0x7fe1e806abd0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:19.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.597+0000 7fe1ee59d700 1 --2- 172.21.15.31:0/3334721318 >> v2:172.21.15.62:3300/0 conn(0x7fe1e80c7fb0 0x7fe1e815a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:19.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.597+0000 7fe1ee59d700 1 -- 172.21.15.31:0/3334721318 >> 172.21.15.31:0/3334721318 conn(0x7fe1e806d110 msgr2=0x7fe1e80cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:19.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.597+0000 7fe1ee59d700 1 -- 172.21.15.31:0/3334721318 shutdown_connections 2023-10-06T02:26:19.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.597+0000 7fe1ee59d700 1 -- 172.21.15.31:0/3334721318 wait complete. 2023-10-06T02:26:19.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.599+0000 7fe1ee59d700 1 Processor -- start 2023-10-06T02:26:19.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.599+0000 7fe1ee59d700 1 -- start start 2023-10-06T02:26:19.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.600+0000 7fe1ee59d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1e80c7be0 0x7fe1e813a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:19.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.600+0000 7fe1ee59d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe1e80c7fb0 0x7fe1e80c5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:19.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.600+0000 7fe1ee59d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1e813f780 0x7fe1e8158520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:19.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.600+0000 7fe1ee59d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e8157e00 con 0x7fe1e80c7be0 2023-10-06T02:26:19.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.600+0000 7fe1ee59d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e8137f70 con 0x7fe1e813f780 2023-10-06T02:26:19.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.600+0000 7fe1ee59d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e80c4b30 con 0x7fe1e80c7fb0 2023-10-06T02:26:19.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.600+0000 7fe1ed59b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1e80c7be0 0x7fe1e813a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:19.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.600+0000 7fe1ed59b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1e80c7be0 0x7fe1e813a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57128/0 (socket says 172.21.15.31:57128) 2023-10-06T02:26:19.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.600+0000 7fe1ed59b700 1 -- 172.21.15.31:0/1350334715 learned_addr learned my addr 172.21.15.31:0/1350334715 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:26:19.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.600+0000 7fe1edd9c700 1 --2- 172.21.15.31:0/1350334715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1e813f780 0x7fe1e8158520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:19.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.600+0000 7fe1ecd9a700 1 --2- 172.21.15.31:0/1350334715 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe1e80c7fb0 0x7fe1e80c5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:19.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.600+0000 7fe1ed59b700 1 -- 172.21.15.31:0/1350334715 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe1e80c7fb0 msgr2=0x7fe1e80c5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:19.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.600+0000 7fe1ed59b700 1 --2- 172.21.15.31:0/1350334715 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe1e80c7fb0 0x7fe1e80c5b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:19.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.600+0000 7fe1ed59b700 1 -- 172.21.15.31:0/1350334715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1e813f780 msgr2=0x7fe1e8158520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:19.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.600+0000 7fe1ed59b700 1 --2- 172.21.15.31:0/1350334715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1e813f780 0x7fe1e8158520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:19.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.600+0000 7fe1ed59b700 1 -- 172.21.15.31:0/1350334715 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1e40094f0 con 0x7fe1e80c7be0 2023-10-06T02:26:19.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.600+0000 7fe1edd9c700 1 --2- 172.21.15.31:0/1350334715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1e813f780 0x7fe1e8158520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:26:19.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.601+0000 7fe1ed59b700 1 --2- 172.21.15.31:0/1350334715 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1e80c7be0 0x7fe1e813a100 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fe1e40090b0 tx=0x7fe1e4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:19.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.601+0000 7fe1dd7fa700 1 -- 172.21.15.31:0/1350334715 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e400bbe0 con 0x7fe1e80c7be0 2023-10-06T02:26:19.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.601+0000 7fe1dd7fa700 1 -- 172.21.15.31:0/1350334715 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1e401f070 con 0x7fe1e80c7be0 2023-10-06T02:26:19.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.601+0000 7fe1dd7fa700 1 -- 172.21.15.31:0/1350334715 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e40266b0 con 0x7fe1e80c7be0 2023-10-06T02:26:19.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.601+0000 7fe1ee59d700 1 -- 172.21.15.31:0/1350334715 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1e813a640 con 0x7fe1e80c7be0 2023-10-06T02:26:19.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.601+0000 7fe1ee59d700 1 -- 172.21.15.31:0/1350334715 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1e80c4290 con 0x7fe1e80c7be0 2023-10-06T02:26:19.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.602+0000 7fe1ee59d700 1 -- 172.21.15.31:0/1350334715 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1e80c61c0 con 0x7fe1e80c7be0 2023-10-06T02:26:19.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.603+0000 7fe1dd7fa700 1 -- 172.21.15.31:0/1350334715 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 9) v1 ==== 84893+0+0 (secure 0 0 0) 0x7fe1e4026a20 con 0x7fe1e80c7be0 2023-10-06T02:26:19.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.604+0000 7fe1dd7fa700 1 --2- 172.21.15.31:0/1350334715 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe1cc0670e0 0x7fe1cc069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:19.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.604+0000 7fe1dd7fa700 1 -- 172.21.15.31:0/1350334715 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6116+0+0 (secure 0 0 0) 0x7fe1e4021070 con 0x7fe1e80c7be0 2023-10-06T02:26:19.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.604+0000 7fe1ecd9a700 1 --2- 172.21.15.31:0/1350334715 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe1cc0670e0 0x7fe1cc069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:19.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.605+0000 7fe1ecd9a700 1 --2- 172.21.15.31:0/1350334715 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe1cc0670e0 0x7fe1cc069590 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe1d8006f00 tx=0x7fe1d8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:19.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.608+0000 7fe1dd7fa700 1 -- 172.21.15.31:0/1350334715 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe1e80c61c0 con 0x7fe1e80c7be0 2023-10-06T02:26:19.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.811+0000 7fe1ee59d700 1 -- 172.21.15.31:0/1350334715 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe1e80c61c0 con 0x7fe1e80c7be0 2023-10-06T02:26:19.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.812+0000 7fe1dd7fa700 1 -- 172.21.15.31:0/1350334715 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v49) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe1e80c61c0 con 0x7fe1e80c7be0 2023-10-06T02:26:19.813 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-10-06T02:26:19.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.814+0000 7fe1ee59d700 1 -- 172.21.15.31:0/1350334715 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe1cc0670e0 msgr2=0x7fe1cc069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:19.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.814+0000 7fe1ee59d700 1 --2- 172.21.15.31:0/1350334715 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe1cc0670e0 0x7fe1cc069590 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe1d8006f00 tx=0x7fe1d8008040 comp rx=0 tx=0).stop 2023-10-06T02:26:19.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.814+0000 7fe1ee59d700 1 -- 172.21.15.31:0/1350334715 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1e80c7be0 msgr2=0x7fe1e813a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:19.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.814+0000 7fe1ee59d700 1 --2- 172.21.15.31:0/1350334715 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1e80c7be0 0x7fe1e813a100 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fe1e40090b0 tx=0x7fe1e4017000 comp rx=0 tx=0).stop 2023-10-06T02:26:19.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.814+0000 7fe1ee59d700 1 -- 172.21.15.31:0/1350334715 shutdown_connections 2023-10-06T02:26:19.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.814+0000 7fe1ee59d700 1 --2- 172.21.15.31:0/1350334715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1e813f780 0x7fe1e8158520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:19.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.814+0000 7fe1ee59d700 1 --2- 172.21.15.31:0/1350334715 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1e80c7be0 0x7fe1e813a100 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:19.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.814+0000 7fe1ee59d700 1 --2- 172.21.15.31:0/1350334715 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe1cc0670e0 0x7fe1cc069590 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:19.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.814+0000 7fe1ee59d700 1 --2- 172.21.15.31:0/1350334715 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe1e80c7fb0 0x7fe1e80c5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:19.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.814+0000 7fe1ee59d700 1 -- 172.21.15.31:0/1350334715 >> 172.21.15.31:0/1350334715 conn(0x7fe1e806d110 msgr2=0x7fe1e806e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:19.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.814+0000 7fe1ee59d700 1 -- 172.21.15.31:0/1350334715 shutdown_connections 2023-10-06T02:26:19.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:19.814+0000 7fe1ee59d700 1 -- 172.21.15.31:0/1350334715 wait complete. 2023-10-06T02:26:19.832 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T02:26:20.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.020+0000 7f793579d700 1 Processor -- start 2023-10-06T02:26:20.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.020+0000 7f793579d700 1 -- start start 2023-10-06T02:26:20.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.021+0000 7f793579d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7930153a00 0x7f7930153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:20.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.021+0000 7f793579d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f793015ac50 con 0x7f79300cea00 2023-10-06T02:26:20.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.021+0000 7f793579d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f793015d010 con 0x7f793015de50 2023-10-06T02:26:20.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.021+0000 7f793579d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f793015af60 con 0x7f7930153a00 2023-10-06T02:26:20.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.022+0000 7f792ffff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7930153a00 0x7f7930153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:20.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.022+0000 7f7934f9c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f79300cea00 0x7f79300cee80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60600/0 (socket says 172.21.15.31:60600) 2023-10-06T02:26:20.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.022+0000 7f7934f9c700 1 -- 172.21.15.31:0/773236185 learned_addr learned my addr 172.21.15.31:0/773236185 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:26:20.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.022+0000 7f792ffff700 1 -- 172.21.15.31:0/773236185 >> v1:172.21.15.62:6789/0 conn(0x7f793015de50 legacy=0x7f793015a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:20.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.022+0000 7f792ffff700 1 -- 172.21.15.31:0/773236185 >> v1:172.21.15.31:6789/0 conn(0x7f79300cea00 legacy=0x7f79300cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:20.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.022+0000 7f792ffff700 1 -- 172.21.15.31:0/773236185 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f793015b240 con 0x7f7930153a00 2023-10-06T02:26:20.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.022+0000 7f792effd700 1 -- 172.21.15.31:0/773236185 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2118761829 0 0) 0x7f793015d010 con 0x7f793015de50 2023-10-06T02:26:20.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.022+0000 7f792ffff700 1 --2- 172.21.15.31:0/773236185 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7930153a00 0x7f7930153dd0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f79200098e0 tx=0x7f79200091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49378b13163a3e31 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:20.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.023+0000 7f792effd700 1 -- 172.21.15.31:0/773236185 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7920009e80 con 0x7f7930153a00 2023-10-06T02:26:20.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.023+0000 7f792effd700 1 -- 172.21.15.31:0/773236185 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f792000b760 con 0x7f7930153a00 2023-10-06T02:26:20.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.023+0000 7f792effd700 1 -- 172.21.15.31:0/773236185 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f792000bae0 con 0x7f7930153a00 2023-10-06T02:26:20.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.023+0000 7f793579d700 1 -- 172.21.15.31:0/773236185 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7930153a00 msgr2=0x7f7930153dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:20.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.023+0000 7f793579d700 1 --2- 172.21.15.31:0/773236185 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7930153a00 0x7f7930153dd0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f79200098e0 tx=0x7f79200091a0 comp rx=0 tx=0).stop 2023-10-06T02:26:20.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.023+0000 7f793579d700 1 -- 172.21.15.31:0/773236185 shutdown_connections 2023-10-06T02:26:20.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.023+0000 7f793579d700 1 --2- 172.21.15.31:0/773236185 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7930153a00 0x7f7930153dd0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:20.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.023+0000 7f793579d700 1 -- 172.21.15.31:0/773236185 >> 172.21.15.31:0/773236185 conn(0x7f79300703a0 msgr2=0x7f79300707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:20.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.023+0000 7f793579d700 1 -- 172.21.15.31:0/773236185 shutdown_connections 2023-10-06T02:26:20.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.023+0000 7f793579d700 1 -- 172.21.15.31:0/773236185 wait complete. 2023-10-06T02:26:20.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.025+0000 7f793579d700 1 Processor -- start 2023-10-06T02:26:20.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.025+0000 7f793579d700 1 -- start start 2023-10-06T02:26:20.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.026+0000 7f793579d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f79300cea00 0x7f7930151700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:20.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.026+0000 7f793579d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7930153a00 0x7f79300b0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:20.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.026+0000 7f793579d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f793015de50 0x7f7930151e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:20.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.026+0000 7f793579d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f793015fab0 con 0x7f793015de50 2023-10-06T02:26:20.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.026+0000 7f793579d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f793015ffa0 con 0x7f7930153a00 2023-10-06T02:26:20.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.026+0000 7f793579d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7930151c40 con 0x7f79300cea00 2023-10-06T02:26:20.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.026+0000 7f7934f9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f793015de50 0x7f7930151e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:20.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.026+0000 7f7934f9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f793015de50 0x7f7930151e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57144/0 (socket says 172.21.15.31:57144) 2023-10-06T02:26:20.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.026+0000 7f7934f9c700 1 -- 172.21.15.31:0/2456141686 learned_addr learned my addr 172.21.15.31:0/2456141686 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:26:20.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.026+0000 7f792f7fe700 1 --2- 172.21.15.31:0/2456141686 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7930153a00 0x7f79300b0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:20.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.026+0000 7f792ffff700 1 --2- 172.21.15.31:0/2456141686 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f79300cea00 0x7f7930151700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:20.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.027+0000 7f7934f9c700 1 -- 172.21.15.31:0/2456141686 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f79300cea00 msgr2=0x7f7930151700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:20.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.027+0000 7f7934f9c700 1 --2- 172.21.15.31:0/2456141686 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f79300cea00 0x7f7930151700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:20.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.027+0000 7f7934f9c700 1 -- 172.21.15.31:0/2456141686 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7930153a00 msgr2=0x7f79300b0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:20.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.027+0000 7f7934f9c700 1 --2- 172.21.15.31:0/2456141686 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7930153a00 0x7f79300b0ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:20.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.027+0000 7f7934f9c700 1 -- 172.21.15.31:0/2456141686 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7920009420 con 0x7f793015de50 2023-10-06T02:26:20.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.027+0000 7f792ffff700 1 --2- 172.21.15.31:0/2456141686 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f79300cea00 0x7f7930151700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:26:20.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.027+0000 7f7934f9c700 1 --2- 172.21.15.31:0/2456141686 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f793015de50 0x7f7930151e80 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f792800ccc0 tx=0x7f79280019f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:20.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.027+0000 7f7917fff700 1 -- 172.21.15.31:0/2456141686 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f792800ce70 con 0x7f793015de50 2023-10-06T02:26:20.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.027+0000 7f7917fff700 1 -- 172.21.15.31:0/2456141686 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f792800ab70 con 0x7f793015de50 2023-10-06T02:26:20.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.027+0000 7f7917fff700 1 -- 172.21.15.31:0/2456141686 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79280070f0 con 0x7f793015de50 2023-10-06T02:26:20.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.027+0000 7f793579d700 1 -- 172.21.15.31:0/2456141686 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f793015fcc0 con 0x7f793015de50 2023-10-06T02:26:20.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.027+0000 7f793579d700 1 -- 172.21.15.31:0/2456141686 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79300b06c0 con 0x7f793015de50 2023-10-06T02:26:20.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.028+0000 7f793579d700 1 -- 172.21.15.31:0/2456141686 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f793004c3c0 con 0x7f793015de50 2023-10-06T02:26:20.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.029+0000 7f7917fff700 1 -- 172.21.15.31:0/2456141686 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 9) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f79280074e0 con 0x7f793015de50 2023-10-06T02:26:20.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.030+0000 7f7917fff700 1 --2- 172.21.15.31:0/2456141686 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f791c0670e0 0x7f791c069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:20.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.030+0000 7f7917fff700 1 -- 172.21.15.31:0/2456141686 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f7928084de0 con 0x7f793015de50 2023-10-06T02:26:20.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.030+0000 7f792ffff700 1 --2- 172.21.15.31:0/2456141686 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f791c0670e0 0x7f791c069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:20.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.031+0000 7f792ffff700 1 --2- 172.21.15.31:0/2456141686 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f791c0670e0 0x7f791c069590 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f79200030c0 tx=0x7f7920016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:20.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.035+0000 7f7917fff700 1 -- 172.21.15.31:0/2456141686 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7928051cb0 con 0x7f793015de50 2023-10-06T02:26:20.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.234+0000 7f793579d700 1 -- 172.21.15.31:0/2456141686 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f79300ce4f0 con 0x7f793015de50 2023-10-06T02:26:20.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.235+0000 7f7917fff700 1 -- 172.21.15.31:0/2456141686 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v49) v1 ==== 92+0+10 (secure 0 0 0) 0x7f79280552d0 con 0x7f793015de50 2023-10-06T02:26:20.237 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 8 2023-10-06T02:26:20.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.237+0000 7f793579d700 1 -- 172.21.15.31:0/2456141686 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f791c0670e0 msgr2=0x7f791c069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:20.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.237+0000 7f793579d700 1 --2- 172.21.15.31:0/2456141686 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f791c0670e0 0x7f791c069590 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f79200030c0 tx=0x7f7920016000 comp rx=0 tx=0).stop 2023-10-06T02:26:20.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.237+0000 7f793579d700 1 -- 172.21.15.31:0/2456141686 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f793015de50 msgr2=0x7f7930151e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:20.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.237+0000 7f793579d700 1 --2- 172.21.15.31:0/2456141686 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f793015de50 0x7f7930151e80 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f792800ccc0 tx=0x7f79280019f0 comp rx=0 tx=0).stop 2023-10-06T02:26:20.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.237+0000 7f793579d700 1 -- 172.21.15.31:0/2456141686 shutdown_connections 2023-10-06T02:26:20.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.237+0000 7f793579d700 1 --2- 172.21.15.31:0/2456141686 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f793015de50 0x7f7930151e80 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:20.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.237+0000 7f793579d700 1 --2- 172.21.15.31:0/2456141686 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f791c0670e0 0x7f791c069590 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:20.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.237+0000 7f793579d700 1 --2- 172.21.15.31:0/2456141686 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f79300cea00 0x7f7930151700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:20.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.237+0000 7f793579d700 1 --2- 172.21.15.31:0/2456141686 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7930153a00 0x7f79300b0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:20.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.237+0000 7f793579d700 1 -- 172.21.15.31:0/2456141686 >> 172.21.15.31:0/2456141686 conn(0x7f79300703a0 msgr2=0x7f79300c52d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:20.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.238+0000 7f793579d700 1 -- 172.21.15.31:0/2456141686 shutdown_connections 2023-10-06T02:26:20.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.238+0000 7f793579d700 1 -- 172.21.15.31:0/2456141686 wait complete. 2023-10-06T02:26:20.257 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T02:26:20.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.448+0000 7f5bb9f9d700 1 Processor -- start 2023-10-06T02:26:20.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.448+0000 7f5bb9f9d700 1 -- start start 2023-10-06T02:26:20.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.449+0000 7f5bb9f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5bb4153a00 0x7f5bb4153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:20.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.449+0000 7f5bb9f9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5bb40cea00 0x7f5bb40cee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:20.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.449+0000 7f5bb9f9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bb415ac50 con 0x7f5bb415de50 2023-10-06T02:26:20.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.450+0000 7f5bb9f9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bb415d010 con 0x7f5bb40cea00 2023-10-06T02:26:20.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.450+0000 7f5bb9f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb415af60 con 0x7f5bb4153a00 2023-10-06T02:26:20.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.450+0000 7f5bb3fff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f5bb415de50 0x7f5bb415a540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60610/0 (socket says 172.21.15.31:60610) 2023-10-06T02:26:20.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.450+0000 7f5bb3fff700 1 -- 172.21.15.31:0/429333563 learned_addr learned my addr 172.21.15.31:0/429333563 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:26:20.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.450+0000 7f5bb979c700 1 --2- 172.21.15.31:0/429333563 >> v2:172.21.15.31:3300/0 conn(0x7f5bb40cea00 0x7f5bb40cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:20.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.450+0000 7f5bb8f9b700 1 --2- 172.21.15.31:0/429333563 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5bb4153a00 0x7f5bb4153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:20.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.450+0000 7f5bb37fe700 1 -- 172.21.15.31:0/429333563 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4205437526 0 0) 0x7f5bb415ac50 con 0x7f5bb415de50 2023-10-06T02:26:20.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.450+0000 7f5bb37fe700 1 -- 172.21.15.31:0/429333563 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ba4003570 con 0x7f5bb415de50 2023-10-06T02:26:20.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.450+0000 7f5bb979c700 1 -- 172.21.15.31:0/429333563 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5bb4153a00 msgr2=0x7f5bb4153dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:26:20.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.450+0000 7f5bb979c700 1 --2- 172.21.15.31:0/429333563 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5bb4153a00 0x7f5bb4153dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:20.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.450+0000 7f5bb979c700 1 -- 172.21.15.31:0/429333563 >> v1:172.21.15.31:6789/0 conn(0x7f5bb415de50 legacy=0x7f5bb415a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:20.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.450+0000 7f5bb979c700 1 -- 172.21.15.31:0/429333563 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bb415b240 con 0x7f5bb40cea00 2023-10-06T02:26:20.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.450+0000 7f5bb8f9b700 1 --2- 172.21.15.31:0/429333563 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5bb4153a00 0x7f5bb4153dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:26:20.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.451+0000 7f5bb979c700 1 --2- 172.21.15.31:0/429333563 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5bb40cea00 0x7f5bb40cee40 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f5bac009e60 tx=0x7f5bac009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9cfdb12133254fd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:20.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.451+0000 7f5bb37fe700 1 -- 172.21.15.31:0/429333563 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bac00e070 con 0x7f5bb40cea00 2023-10-06T02:26:20.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.451+0000 7f5bb37fe700 1 -- 172.21.15.31:0/429333563 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bac00bb60 con 0x7f5bb40cea00 2023-10-06T02:26:20.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.451+0000 7f5bb37fe700 1 -- 172.21.15.31:0/429333563 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bac012430 con 0x7f5bb40cea00 2023-10-06T02:26:20.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.451+0000 7f5bb9f9d700 1 -- 172.21.15.31:0/429333563 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5bb40cea00 msgr2=0x7f5bb40cee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:20.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.451+0000 7f5bb9f9d700 1 --2- 172.21.15.31:0/429333563 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5bb40cea00 0x7f5bb40cee40 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f5bac009e60 tx=0x7f5bac009780 comp rx=0 tx=0).stop 2023-10-06T02:26:20.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.451+0000 7f5bb9f9d700 1 -- 172.21.15.31:0/429333563 shutdown_connections 2023-10-06T02:26:20.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.451+0000 7f5bb9f9d700 1 --2- 172.21.15.31:0/429333563 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5bb40cea00 0x7f5bb40cee40 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:20.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.451+0000 7f5bb9f9d700 1 --2- 172.21.15.31:0/429333563 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5bb4153a00 0x7f5bb4153dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:20.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.451+0000 7f5bb9f9d700 1 -- 172.21.15.31:0/429333563 >> 172.21.15.31:0/429333563 conn(0x7f5bb40703a0 msgr2=0x7f5bb40707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:20.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.451+0000 7f5bb9f9d700 1 -- 172.21.15.31:0/429333563 shutdown_connections 2023-10-06T02:26:20.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.451+0000 7f5bb9f9d700 1 -- 172.21.15.31:0/429333563 wait complete. 2023-10-06T02:26:20.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.453+0000 7f5bb9f9d700 1 Processor -- start 2023-10-06T02:26:20.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.453+0000 7f5bb9f9d700 1 -- start start 2023-10-06T02:26:20.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.454+0000 7f5bb9f9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5bb40cea00 0x7f5bb40c75e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:20.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.454+0000 7f5bb9f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5bb4153a00 0x7f5bb40cc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:20.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.454+0000 7f5bb9f9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5bb415de50 0x7f5bb4169080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:20.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.454+0000 7f5bb9f9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb40c7b20 con 0x7f5bb40cea00 2023-10-06T02:26:20.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.454+0000 7f5bb9f9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb40c6d00 con 0x7f5bb415de50 2023-10-06T02:26:20.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.454+0000 7f5bb9f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb40c9aa0 con 0x7f5bb4153a00 2023-10-06T02:26:20.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.454+0000 7f5bb8f9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5bb40cea00 0x7f5bb40c75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:20.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.454+0000 7f5bb8f9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5bb40cea00 0x7f5bb40c75e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57158/0 (socket says 172.21.15.31:57158) 2023-10-06T02:26:20.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.454+0000 7f5bb8f9b700 1 -- 172.21.15.31:0/3481676777 learned_addr learned my addr 172.21.15.31:0/3481676777 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:26:20.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.454+0000 7f5bb3fff700 1 --2- 172.21.15.31:0/3481676777 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5bb4153a00 0x7f5bb40cc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:20.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.454+0000 7f5bb979c700 1 --2- 172.21.15.31:0/3481676777 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5bb415de50 0x7f5bb4169080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:20.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.455+0000 7f5bb8f9b700 1 -- 172.21.15.31:0/3481676777 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5bb4153a00 msgr2=0x7f5bb40cc5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:20.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.455+0000 7f5bb8f9b700 1 --2- 172.21.15.31:0/3481676777 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5bb4153a00 0x7f5bb40cc5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:20.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.455+0000 7f5bb8f9b700 1 -- 172.21.15.31:0/3481676777 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5bb415de50 msgr2=0x7f5bb4169080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:20.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.455+0000 7f5bb8f9b700 1 --2- 172.21.15.31:0/3481676777 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5bb415de50 0x7f5bb4169080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:20.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.455+0000 7f5bb8f9b700 1 -- 172.21.15.31:0/3481676777 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bac009a20 con 0x7f5bb40cea00 2023-10-06T02:26:20.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.455+0000 7f5bb8f9b700 1 --2- 172.21.15.31:0/3481676777 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5bb40cea00 0x7f5bb40c75e0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f5ba800e8d0 tx=0x7f5ba8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:20.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.455+0000 7f5bb0ff9700 1 -- 172.21.15.31:0/3481676777 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba800ee70 con 0x7f5bb40cea00 2023-10-06T02:26:20.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.455+0000 7f5bb0ff9700 1 -- 172.21.15.31:0/3481676777 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ba800c5e0 con 0x7f5bb40cea00 2023-10-06T02:26:20.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.455+0000 7f5bb0ff9700 1 -- 172.21.15.31:0/3481676777 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba800c980 con 0x7f5bb40cea00 2023-10-06T02:26:20.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.455+0000 7f5bb9f9d700 1 -- 172.21.15.31:0/3481676777 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bb40c9d80 con 0x7f5bb40cea00 2023-10-06T02:26:20.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.455+0000 7f5bb9f9d700 1 -- 172.21.15.31:0/3481676777 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bb40ca340 con 0x7f5bb40cea00 2023-10-06T02:26:20.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.456+0000 7f5bb9f9d700 1 -- 172.21.15.31:0/3481676777 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bb404c3c0 con 0x7f5bb40cea00 2023-10-06T02:26:20.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.457+0000 7f5bb0ff9700 1 -- 172.21.15.31:0/3481676777 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 9) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f5ba8013410 con 0x7f5bb40cea00 2023-10-06T02:26:20.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.458+0000 7f5bb0ff9700 1 --2- 172.21.15.31:0/3481676777 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5b98067160 0x7f5b98069610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:20.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.458+0000 7f5bb0ff9700 1 -- 172.21.15.31:0/3481676777 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f5ba80849a0 con 0x7f5bb40cea00 2023-10-06T02:26:20.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.458+0000 7f5bb3fff700 1 --2- 172.21.15.31:0/3481676777 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5b98067160 0x7f5b98069610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:20.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.459+0000 7f5bb3fff700 1 --2- 172.21.15.31:0/3481676777 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5b98067160 0x7f5b98069610 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f5bb4159c60 tx=0x7f5ba0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:20.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.462+0000 7f5bb0ff9700 1 -- 172.21.15.31:0/3481676777 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5ba8051640 con 0x7f5bb40cea00 2023-10-06T02:26:20.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.668+0000 7f5bb9f9d700 1 -- 172.21.15.31:0/3481676777 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5bb415b520 con 0x7f5bb40cea00 2023-10-06T02:26:20.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.668+0000 7f5bb0ff9700 1 -- 172.21.15.31:0/3481676777 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v49) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5ba8054c60 con 0x7f5bb40cea00 2023-10-06T02:26:20.670 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-10-06T02:26:20.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.671+0000 7f5bb9f9d700 1 -- 172.21.15.31:0/3481676777 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5b98067160 msgr2=0x7f5b98069610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:20.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.671+0000 7f5bb9f9d700 1 --2- 172.21.15.31:0/3481676777 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5b98067160 0x7f5b98069610 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f5bb4159c60 tx=0x7f5ba0009040 comp rx=0 tx=0).stop 2023-10-06T02:26:20.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.671+0000 7f5bb9f9d700 1 -- 172.21.15.31:0/3481676777 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5bb40cea00 msgr2=0x7f5bb40c75e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:20.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.671+0000 7f5bb9f9d700 1 --2- 172.21.15.31:0/3481676777 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5bb40cea00 0x7f5bb40c75e0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f5ba800e8d0 tx=0x7f5ba8008040 comp rx=0 tx=0).stop 2023-10-06T02:26:20.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.671+0000 7f5bb9f9d700 1 -- 172.21.15.31:0/3481676777 shutdown_connections 2023-10-06T02:26:20.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.671+0000 7f5bb9f9d700 1 --2- 172.21.15.31:0/3481676777 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5bb415de50 0x7f5bb4169080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:20.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.671+0000 7f5bb9f9d700 1 --2- 172.21.15.31:0/3481676777 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5bb40cea00 0x7f5bb40c75e0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:20.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.671+0000 7f5bb9f9d700 1 --2- 172.21.15.31:0/3481676777 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5b98067160 0x7f5b98069610 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:20.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.671+0000 7f5bb9f9d700 1 --2- 172.21.15.31:0/3481676777 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5bb4153a00 0x7f5bb40cc5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:20.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.671+0000 7f5bb9f9d700 1 -- 172.21.15.31:0/3481676777 >> 172.21.15.31:0/3481676777 conn(0x7f5bb40703a0 msgr2=0x7f5bb40b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:20.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.671+0000 7f5bb9f9d700 1 -- 172.21.15.31:0/3481676777 shutdown_connections 2023-10-06T02:26:20.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.671+0000 7f5bb9f9d700 1 -- 172.21.15.31:0/3481676777 wait complete. 2023-10-06T02:26:20.690 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T02:26:20.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.878+0000 7f293bfff700 1 Processor -- start 2023-10-06T02:26:20.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.879+0000 7f293bfff700 1 -- start start 2023-10-06T02:26:20.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.880+0000 7f293bfff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f293c06de50 0x7f293c0c7360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:20.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.880+0000 7f293bfff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f293c15b640 con 0x7f293c06a210 2023-10-06T02:26:20.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.880+0000 7f293bfff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f293c15b920 con 0x7f293c140a70 2023-10-06T02:26:20.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.880+0000 7f293bfff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f293c0c6800 con 0x7f293c06de50 2023-10-06T02:26:20.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.880+0000 7f293a7fc700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f293c06a210 0x7f293c0c6c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60616/0 (socket says 172.21.15.31:60616) 2023-10-06T02:26:20.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.880+0000 7f293a7fc700 1 -- 172.21.15.31:0/4100859835 learned_addr learned my addr 172.21.15.31:0/4100859835 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:26:20.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.880+0000 7f293b7fe700 1 --2- 172.21.15.31:0/4100859835 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f293c06de50 0x7f293c0c7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:20.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.881+0000 7f2939ffb700 1 -- 172.21.15.31:0/4100859835 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1897440724 0 0) 0x7f293c15b640 con 0x7f293c06a210 2023-10-06T02:26:20.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.881+0000 7f2939ffb700 1 -- 172.21.15.31:0/4100859835 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2924003570 con 0x7f293c06a210 2023-10-06T02:26:20.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.881+0000 7f2939ffb700 1 -- 172.21.15.31:0/4100859835 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2449121517 0 0) 0x7f293c15b920 con 0x7f293c140a70 2023-10-06T02:26:20.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.881+0000 7f2939ffb700 1 -- 172.21.15.31:0/4100859835 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f293c15b640 con 0x7f293c140a70 2023-10-06T02:26:20.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.881+0000 7f2939ffb700 1 -- 172.21.15.31:0/4100859835 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 553341689 0 0) 0x7f2924003570 con 0x7f293c06a210 2023-10-06T02:26:20.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.881+0000 7f2939ffb700 1 -- 172.21.15.31:0/4100859835 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f293c06de50 msgr2=0x7f293c0c7360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:26:20.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.881+0000 7f2939ffb700 1 --2- 172.21.15.31:0/4100859835 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f293c06de50 0x7f293c0c7360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:20.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.881+0000 7f2939ffb700 1 -- 172.21.15.31:0/4100859835 >> v1:172.21.15.62:6789/0 conn(0x7f293c140a70 legacy=0x7f293c069b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:20.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.881+0000 7f2939ffb700 1 -- 172.21.15.31:0/4100859835 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f293c15bc00 con 0x7f293c06a210 2023-10-06T02:26:20.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.881+0000 7f2939ffb700 1 -- 172.21.15.31:0/4100859835 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7f2930001470 con 0x7f293c06a210 2023-10-06T02:26:20.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.881+0000 7f2939ffb700 1 -- 172.21.15.31:0/4100859835 >> v1:172.21.15.31:6789/0 conn(0x7f293c06a210 legacy=0x7f293c0c6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:20.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.881+0000 7f2939ffb700 1 --2- 172.21.15.31:0/4100859835 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2924003c20 0x7f2924004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:20.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.881+0000 7f2939ffb700 1 --2- 172.21.15.31:0/4100859835 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f293c06de50 0x7f2924004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:20.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.881+0000 7f2939ffb700 1 --2- 172.21.15.31:0/4100859835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2924004d50 0x7f29240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:20.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.881+0000 7f2939ffb700 1 -- 172.21.15.31:0/4100859835 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f293c0c6800 con 0x7f2924003c20 2023-10-06T02:26:20.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.881+0000 7f2939ffb700 1 -- 172.21.15.31:0/4100859835 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f292400bbf0 con 0x7f2924004d50 2023-10-06T02:26:20.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.881+0000 7f2939ffb700 1 -- 172.21.15.31:0/4100859835 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f292400bd60 con 0x7f293c06de50 2023-10-06T02:26:20.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.881+0000 7f293affd700 1 --2- 172.21.15.31:0/4100859835 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2924003c20 0x7f2924004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:20.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.882+0000 7f293b7fe700 1 --2- 172.21.15.31:0/4100859835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2924004d50 0x7f29240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:20.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.882+0000 7f293a7fc700 1 --2- 172.21.15.31:0/4100859835 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f293c06de50 0x7f2924004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:20.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.882+0000 7f293affd700 1 -- 172.21.15.31:0/4100859835 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f293c06de50 msgr2=0x7f2924004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:26:20.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.882+0000 7f293affd700 1 --2- 172.21.15.31:0/4100859835 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f293c06de50 0x7f2924004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:20.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.882+0000 7f293affd700 1 -- 172.21.15.31:0/4100859835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2924004d50 msgr2=0x7f29240051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:26:20.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.882+0000 7f293affd700 1 --2- 172.21.15.31:0/4100859835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2924004d50 0x7f29240051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:20.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.882+0000 7f293affd700 1 -- 172.21.15.31:0/4100859835 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2924003570 con 0x7f2924003c20 2023-10-06T02:26:20.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.882+0000 7f293affd700 1 --2- 172.21.15.31:0/4100859835 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2924003c20 0x7f2924004010 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f292c0019e0 tx=0x7f292c007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f456812c651ee921 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:20.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.882+0000 7f2939ffb700 1 -- 172.21.15.31:0/4100859835 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f292c00c5d0 con 0x7f2924003c20 2023-10-06T02:26:20.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.882+0000 7f2939ffb700 1 -- 172.21.15.31:0/4100859835 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f292c00c730 con 0x7f2924003c20 2023-10-06T02:26:20.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.882+0000 7f2939ffb700 1 -- 172.21.15.31:0/4100859835 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f292c00cad0 con 0x7f2924003c20 2023-10-06T02:26:20.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.883+0000 7f293bfff700 1 -- 172.21.15.31:0/4100859835 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2924003c20 msgr2=0x7f2924004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:20.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.883+0000 7f293bfff700 1 --2- 172.21.15.31:0/4100859835 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2924003c20 0x7f2924004010 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f292c0019e0 tx=0x7f292c007d30 comp rx=0 tx=0).stop 2023-10-06T02:26:20.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.883+0000 7f293b7fe700 1 -- 172.21.15.31:0/4100859835 reap_dead start 2023-10-06T02:26:20.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.883+0000 7f293bfff700 1 -- 172.21.15.31:0/4100859835 shutdown_connections 2023-10-06T02:26:20.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.883+0000 7f293bfff700 1 -- 172.21.15.31:0/4100859835 >> 172.21.15.31:0/4100859835 conn(0x7f293c06d110 msgr2=0x7f293c0cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:20.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.883+0000 7f293bfff700 1 -- 172.21.15.31:0/4100859835 shutdown_connections 2023-10-06T02:26:20.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.883+0000 7f293bfff700 1 -- 172.21.15.31:0/4100859835 wait complete. 2023-10-06T02:26:20.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.884+0000 7f293bfff700 1 Processor -- start 2023-10-06T02:26:20.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.885+0000 7f293bfff700 1 -- start start 2023-10-06T02:26:20.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.886+0000 7f293bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2924003c20 0x7f293c157910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:20.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.886+0000 7f293bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f293c167b30 0x7f293c15a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:20.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.886+0000 7f293bfff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f293c15a860 0x7f293c0c6c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:20.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.886+0000 7f293bfff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f293c157e50 con 0x7f2924003c20 2023-10-06T02:26:20.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.886+0000 7f293bfff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f293c167fa0 con 0x7f293c167b30 2023-10-06T02:26:20.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.886+0000 7f293bfff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f293c168280 con 0x7f293c15a860 2023-10-06T02:26:20.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.886+0000 7f293affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2924003c20 0x7f293c157910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:20.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.886+0000 7f293affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2924003c20 0x7f293c157910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57180/0 (socket says 172.21.15.31:57180) 2023-10-06T02:26:20.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.886+0000 7f293affd700 1 -- 172.21.15.31:0/1792251053 learned_addr learned my addr 172.21.15.31:0/1792251053 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:26:20.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.886+0000 7f293a7fc700 1 --2- 172.21.15.31:0/1792251053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f293c167b30 0x7f293c15a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:20.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.886+0000 7f293b7fe700 1 --2- 172.21.15.31:0/1792251053 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f293c15a860 0x7f293c0c6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:20.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.886+0000 7f293affd700 1 -- 172.21.15.31:0/1792251053 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f293c15a860 msgr2=0x7f293c0c6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:20.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.886+0000 7f293affd700 1 --2- 172.21.15.31:0/1792251053 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f293c15a860 0x7f293c0c6c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:20.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.886+0000 7f293affd700 1 -- 172.21.15.31:0/1792251053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f293c167b30 msgr2=0x7f293c15a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:20.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.886+0000 7f293affd700 1 --2- 172.21.15.31:0/1792251053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f293c167b30 0x7f293c15a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:20.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.886+0000 7f293affd700 1 -- 172.21.15.31:0/1792251053 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f293c168560 con 0x7f2924003c20 2023-10-06T02:26:20.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.886+0000 7f293b7fe700 1 --2- 172.21.15.31:0/1792251053 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f293c15a860 0x7f293c0c6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:26:20.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.887+0000 7f293affd700 1 --2- 172.21.15.31:0/1792251053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2924003c20 0x7f293c157910 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f292c00c3c0 tx=0x7f292c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:20.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.887+0000 7f2922ffd700 1 -- 172.21.15.31:0/1792251053 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f292c023070 con 0x7f2924003c20 2023-10-06T02:26:20.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.887+0000 7f2922ffd700 1 -- 172.21.15.31:0/1792251053 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f292c000fe0 con 0x7f2924003c20 2023-10-06T02:26:20.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.887+0000 7f2922ffd700 1 -- 172.21.15.31:0/1792251053 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f292c001360 con 0x7f2924003c20 2023-10-06T02:26:20.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.887+0000 7f293bfff700 1 -- 172.21.15.31:0/1792251053 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f293c168840 con 0x7f2924003c20 2023-10-06T02:26:20.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.887+0000 7f293bfff700 1 -- 172.21.15.31:0/1792251053 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f293c0c71b0 con 0x7f2924003c20 2023-10-06T02:26:20.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.888+0000 7f293bfff700 1 -- 172.21.15.31:0/1792251053 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f293c0af090 con 0x7f2924003c20 2023-10-06T02:26:20.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.889+0000 7f2922ffd700 1 -- 172.21.15.31:0/1792251053 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 9) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f292c019070 con 0x7f2924003c20 2023-10-06T02:26:20.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.890+0000 7f2922ffd700 1 --2- 172.21.15.31:0/1792251053 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f29280670e0 0x7f2928069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:20.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.890+0000 7f2922ffd700 1 -- 172.21.15.31:0/1792251053 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f292c085890 con 0x7f2924003c20 2023-10-06T02:26:20.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.890+0000 7f293a7fc700 1 --2- 172.21.15.31:0/1792251053 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f29280670e0 0x7f2928069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:20.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.891+0000 7f293a7fc700 1 --2- 172.21.15.31:0/1792251053 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f29280670e0 0x7f2928069590 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f293c168cf0 tx=0x7f2930001d20 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:20.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:20.894+0000 7f2922ffd700 1 -- 172.21.15.31:0/1792251053 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f292c052700 con 0x7f2924003c20 2023-10-06T02:26:21.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.098+0000 7f293bfff700 1 -- 172.21.15.31:0/1792251053 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f293c0b1d40 con 0x7f2924003c20 2023-10-06T02:26:21.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.098+0000 7f2922ffd700 1 -- 172.21.15.31:0/1792251053 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v49) v1 ==== 100+0+11 (secure 0 0 0) 0x7f293c0b1d40 con 0x7f2924003c20 2023-10-06T02:26:21.100 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-10-06T02:26:21.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.101+0000 7f293bfff700 1 -- 172.21.15.31:0/1792251053 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f29280670e0 msgr2=0x7f2928069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:21.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.101+0000 7f293bfff700 1 --2- 172.21.15.31:0/1792251053 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f29280670e0 0x7f2928069590 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f293c168cf0 tx=0x7f2930001d20 comp rx=0 tx=0).stop 2023-10-06T02:26:21.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.101+0000 7f293bfff700 1 -- 172.21.15.31:0/1792251053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2924003c20 msgr2=0x7f293c157910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:21.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.101+0000 7f293bfff700 1 --2- 172.21.15.31:0/1792251053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2924003c20 0x7f293c157910 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f292c00c3c0 tx=0x7f292c015000 comp rx=0 tx=0).stop 2023-10-06T02:26:21.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.101+0000 7f293bfff700 1 -- 172.21.15.31:0/1792251053 shutdown_connections 2023-10-06T02:26:21.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.101+0000 7f293bfff700 1 --2- 172.21.15.31:0/1792251053 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f29280670e0 0x7f2928069590 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:21.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.101+0000 7f293bfff700 1 --2- 172.21.15.31:0/1792251053 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f293c15a860 0x7f293c0c6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:21.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.101+0000 7f293bfff700 1 --2- 172.21.15.31:0/1792251053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2924003c20 0x7f293c157910 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:21.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.101+0000 7f293bfff700 1 --2- 172.21.15.31:0/1792251053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f293c167b30 0x7f293c15a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:21.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.101+0000 7f293bfff700 1 -- 172.21.15.31:0/1792251053 >> 172.21.15.31:0/1792251053 conn(0x7f293c06d110 msgr2=0x7f293c13ecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:21.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.101+0000 7f293bfff700 1 -- 172.21.15.31:0/1792251053 shutdown_connections 2023-10-06T02:26:21.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.101+0000 7f293bfff700 1 -- 172.21.15.31:0/1792251053 wait complete. 2023-10-06T02:26:21.119 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:26:21.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.307+0000 7fab3979d700 1 Processor -- start 2023-10-06T02:26:21.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.307+0000 7fab3979d700 1 -- start start 2023-10-06T02:26:21.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.308+0000 7fab3979d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fab34159550 0x7fab340ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:21.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.308+0000 7fab3979d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fab3415dfd0 0x7fab340aedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:21.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.308+0000 7fab3979d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab340acc00 con 0x7fab34159920 2023-10-06T02:26:21.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.308+0000 7fab3979d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab340abff0 con 0x7fab3415dfd0 2023-10-06T02:26:21.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.308+0000 7fab3979d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab340ac2d0 con 0x7fab34159550 2023-10-06T02:26:21.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.309+0000 7fab337fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fab3415dfd0 0x7fab340aedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:21.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.309+0000 7fab38f9c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fab34159920 0x7fab340ad090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60626/0 (socket says 172.21.15.31:60626) 2023-10-06T02:26:21.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.309+0000 7fab337fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fab3415dfd0 0x7fab340aedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57194/0 (socket says 172.21.15.31:57194) 2023-10-06T02:26:21.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.309+0000 7fab38f9c700 1 -- 172.21.15.31:0/798511262 learned_addr learned my addr 172.21.15.31:0/798511262 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:26:21.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.309+0000 7fab33fff700 1 --2- 172.21.15.31:0/798511262 >> v2:172.21.15.62:3300/0 conn(0x7fab34159550 0x7fab340ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:21.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.309+0000 7fab337fe700 1 -- 172.21.15.31:0/798511262 >> v2:172.21.15.62:3300/0 conn(0x7fab34159550 msgr2=0x7fab340ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:26:21.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.309+0000 7fab337fe700 1 --2- 172.21.15.31:0/798511262 >> v2:172.21.15.62:3300/0 conn(0x7fab34159550 0x7fab340ceac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:21.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.309+0000 7fab337fe700 1 -- 172.21.15.31:0/798511262 >> v1:172.21.15.31:6789/0 conn(0x7fab34159920 legacy=0x7fab340ad090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:21.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.309+0000 7fab337fe700 1 -- 172.21.15.31:0/798511262 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab340ac5b0 con 0x7fab3415dfd0 2023-10-06T02:26:21.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.309+0000 7fab337fe700 1 --2- 172.21.15.31:0/798511262 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab3415dfd0 0x7fab340aedb0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fab34064270 tx=0x7fab280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1baaec45f6507c44 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:21.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.310+0000 7fab32ffd700 1 -- 172.21.15.31:0/798511262 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab2800b6b0 con 0x7fab3415dfd0 2023-10-06T02:26:21.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.310+0000 7fab32ffd700 1 -- 172.21.15.31:0/798511262 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab2800b810 con 0x7fab3415dfd0 2023-10-06T02:26:21.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.310+0000 7fab32ffd700 1 -- 172.21.15.31:0/798511262 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab2800bbb0 con 0x7fab3415dfd0 2023-10-06T02:26:21.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.310+0000 7fab3979d700 1 -- 172.21.15.31:0/798511262 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab3415dfd0 msgr2=0x7fab340aedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:21.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.310+0000 7fab3979d700 1 --2- 172.21.15.31:0/798511262 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab3415dfd0 0x7fab340aedb0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fab34064270 tx=0x7fab280091a0 comp rx=0 tx=0).stop 2023-10-06T02:26:21.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.310+0000 7fab3979d700 1 -- 172.21.15.31:0/798511262 shutdown_connections 2023-10-06T02:26:21.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.310+0000 7fab3979d700 1 --2- 172.21.15.31:0/798511262 >> v2:172.21.15.62:3300/0 conn(0x7fab34159550 0x7fab340ceac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:21.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.310+0000 7fab3979d700 1 --2- 172.21.15.31:0/798511262 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab3415dfd0 0x7fab340aedb0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:21.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.310+0000 7fab3979d700 1 -- 172.21.15.31:0/798511262 >> 172.21.15.31:0/798511262 conn(0x7fab3406ce80 msgr2=0x7fab340ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:21.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.310+0000 7fab3979d700 1 -- 172.21.15.31:0/798511262 shutdown_connections 2023-10-06T02:26:21.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.310+0000 7fab3979d700 1 -- 172.21.15.31:0/798511262 wait complete. 2023-10-06T02:26:21.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.312+0000 7fab3979d700 1 Processor -- start 2023-10-06T02:26:21.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.312+0000 7fab3979d700 1 -- start start 2023-10-06T02:26:21.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.313+0000 7fab3979d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab34159550 0x7fab34148d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:21.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.313+0000 7fab3979d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab34159920 0x7fab3409a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:21.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.313+0000 7fab3979d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab3415dfd0 0x7fab340c95f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:21.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.313+0000 7fab3979d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab34169cb0 con 0x7fab34159920 2023-10-06T02:26:21.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.313+0000 7fab3979d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab3409bec0 con 0x7fab3415dfd0 2023-10-06T02:26:21.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.313+0000 7fab3979d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab3409b340 con 0x7fab34159550 2023-10-06T02:26:21.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.313+0000 7fab337fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab34159920 0x7fab3409a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:21.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.313+0000 7fab337fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab34159920 0x7fab3409a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57196/0 (socket says 172.21.15.31:57196) 2023-10-06T02:26:21.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.313+0000 7fab337fe700 1 -- 172.21.15.31:0/1763938955 learned_addr learned my addr 172.21.15.31:0/1763938955 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:26:21.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.313+0000 7fab38f9c700 1 --2- 172.21.15.31:0/1763938955 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab3415dfd0 0x7fab340c95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:21.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.313+0000 7fab33fff700 1 --2- 172.21.15.31:0/1763938955 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab34159550 0x7fab34148d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:21.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.313+0000 7fab337fe700 1 -- 172.21.15.31:0/1763938955 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab34159550 msgr2=0x7fab34148d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:21.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.313+0000 7fab337fe700 1 --2- 172.21.15.31:0/1763938955 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab34159550 0x7fab34148d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:21.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.313+0000 7fab337fe700 1 -- 172.21.15.31:0/1763938955 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab3415dfd0 msgr2=0x7fab340c95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:21.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.313+0000 7fab337fe700 1 --2- 172.21.15.31:0/1763938955 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab3415dfd0 0x7fab340c95f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:21.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.313+0000 7fab337fe700 1 -- 172.21.15.31:0/1763938955 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab28009420 con 0x7fab34159920 2023-10-06T02:26:21.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.314+0000 7fab337fe700 1 --2- 172.21.15.31:0/1763938955 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab34159920 0x7fab3409a940 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fab3406ad70 tx=0x7fab28017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:21.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.314+0000 7fab1bfff700 1 -- 172.21.15.31:0/1763938955 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab28011420 con 0x7fab34159920 2023-10-06T02:26:21.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.314+0000 7fab1bfff700 1 -- 172.21.15.31:0/1763938955 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab28011580 con 0x7fab34159920 2023-10-06T02:26:21.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.314+0000 7fab1bfff700 1 -- 172.21.15.31:0/1763938955 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab28011940 con 0x7fab34159920 2023-10-06T02:26:21.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.314+0000 7fab3979d700 1 -- 172.21.15.31:0/1763938955 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab3409b620 con 0x7fab34159920 2023-10-06T02:26:21.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.314+0000 7fab3979d700 1 -- 172.21.15.31:0/1763938955 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab3409bbe0 con 0x7fab34159920 2023-10-06T02:26:21.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.315+0000 7fab3979d700 1 -- 172.21.15.31:0/1763938955 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab34097f40 con 0x7fab34159920 2023-10-06T02:26:21.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.316+0000 7fab1bfff700 1 -- 172.21.15.31:0/1763938955 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 9) v1 ==== 84893+0+0 (secure 0 0 0) 0x7fab28011d30 con 0x7fab34159920 2023-10-06T02:26:21.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.317+0000 7fab1bfff700 1 --2- 172.21.15.31:0/1763938955 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fab200670e0 0x7fab20069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:21.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.317+0000 7fab1bfff700 1 -- 172.21.15.31:0/1763938955 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6116+0+0 (secure 0 0 0) 0x7fab28086430 con 0x7fab34159920 2023-10-06T02:26:21.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.317+0000 7fab33fff700 1 --2- 172.21.15.31:0/1763938955 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fab200670e0 0x7fab20069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:21.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.318+0000 7fab33fff700 1 --2- 172.21.15.31:0/1763938955 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fab200670e0 0x7fab20069590 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fab24006fd0 tx=0x7fab24008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:21.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.321+0000 7fab1bfff700 1 -- 172.21.15.31:0/1763938955 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fab34097f40 con 0x7fab34159920 2023-10-06T02:26:21.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.571+0000 7fab3979d700 1 -- 172.21.15.31:0/1763938955 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fab34097f40 con 0x7fab34159920 2023-10-06T02:26:21.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.571+0000 7fab1bfff700 1 -- 172.21.15.31:0/1763938955 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4594 (secure 0 0 0) 0x7fab34097f40 con 0x7fab34159920 2023-10-06T02:26:21.573 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:26:21.573 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4949,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/178547366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":178547366},{"type":"v1","addr":"172.21.15.62:6835","nonce":178547366}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":5278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/2578831517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":2578831517},{"type":"v1","addr":"172.21.15.31:6836","nonce":2578831517}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5287,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3502585737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":3502585737},{"type":"v1","addr":"172.21.15.31:6837","nonce":3502585737}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:24:43.872314+0000","modified":"2023-10-06T02:24:47.404607+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5290":{"gid":5290,"name":"b","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6834/2358776861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2358776861},{"type":"v1","addr":"172.21.15.62:6834","nonce":2358776861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T02:26:21.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.574+0000 7fab3979d700 1 -- 172.21.15.31:0/1763938955 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fab200670e0 msgr2=0x7fab20069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:21.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.574+0000 7fab3979d700 1 --2- 172.21.15.31:0/1763938955 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fab200670e0 0x7fab20069590 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fab24006fd0 tx=0x7fab24008040 comp rx=0 tx=0).stop 2023-10-06T02:26:21.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.574+0000 7fab3979d700 1 -- 172.21.15.31:0/1763938955 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab34159920 msgr2=0x7fab3409a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:21.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.574+0000 7fab3979d700 1 --2- 172.21.15.31:0/1763938955 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab34159920 0x7fab3409a940 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fab3406ad70 tx=0x7fab28017000 comp rx=0 tx=0).stop 2023-10-06T02:26:21.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.574+0000 7fab3979d700 1 -- 172.21.15.31:0/1763938955 shutdown_connections 2023-10-06T02:26:21.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.574+0000 7fab3979d700 1 --2- 172.21.15.31:0/1763938955 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab3415dfd0 0x7fab340c95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:21.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.574+0000 7fab3979d700 1 --2- 172.21.15.31:0/1763938955 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fab200670e0 0x7fab20069590 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:21.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.574+0000 7fab3979d700 1 --2- 172.21.15.31:0/1763938955 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab34159550 0x7fab34148d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:21.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.574+0000 7fab3979d700 1 --2- 172.21.15.31:0/1763938955 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab34159920 0x7fab3409a940 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:21.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.574+0000 7fab3979d700 1 -- 172.21.15.31:0/1763938955 >> 172.21.15.31:0/1763938955 conn(0x7fab3406ce80 msgr2=0x7fab340aed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:21.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.574+0000 7fab3979d700 1 -- 172.21.15.31:0/1763938955 shutdown_connections 2023-10-06T02:26:21.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.574+0000 7fab3979d700 1 -- 172.21.15.31:0/1763938955 wait complete. 2023-10-06T02:26:21.577 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 27 2023-10-06T02:26:21.592 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:26:21.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.777+0000 7f9c17fff700 1 Processor -- start 2023-10-06T02:26:21.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.777+0000 7f9c17fff700 1 -- start start 2023-10-06T02:26:21.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.778+0000 7f9c17fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9c18159920 0x7f9c180ad090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:21.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.778+0000 7f9c17fff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c180acc00 con 0x7f9c18159550 2023-10-06T02:26:21.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.778+0000 7f9c17fff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c180abff0 con 0x7f9c1815dfd0 2023-10-06T02:26:21.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.778+0000 7f9c17fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c180ac2d0 con 0x7f9c18159920 2023-10-06T02:26:21.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.779+0000 7f9c16ffd700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f9c18159550 0x7f9c180ceac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60636/0 (socket says 172.21.15.31:60636) 2023-10-06T02:26:21.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.779+0000 7f9c16ffd700 1 -- 172.21.15.31:0/3482312254 learned_addr learned my addr 172.21.15.31:0/3482312254 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:26:21.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.779+0000 7f9c177fe700 1 --2- 172.21.15.31:0/3482312254 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9c18159920 0x7f9c180ad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:21.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.779+0000 7f9c15ffb700 1 -- 172.21.15.31:0/3482312254 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4089062976 0 0) 0x7f9c180acc00 con 0x7f9c18159550 2023-10-06T02:26:21.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.779+0000 7f9c15ffb700 1 -- 172.21.15.31:0/3482312254 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c00003570 con 0x7f9c18159550 2023-10-06T02:26:21.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.779+0000 7f9c15ffb700 1 -- 172.21.15.31:0/3482312254 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1131857638 0 0) 0x7f9c180abff0 con 0x7f9c1815dfd0 2023-10-06T02:26:21.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.779+0000 7f9c15ffb700 1 -- 172.21.15.31:0/3482312254 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c180acc00 con 0x7f9c1815dfd0 2023-10-06T02:26:21.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.779+0000 7f9c177fe700 1 -- 172.21.15.31:0/3482312254 >> v1:172.21.15.62:6789/0 conn(0x7f9c1815dfd0 legacy=0x7f9c180aedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:21.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.779+0000 7f9c177fe700 1 -- 172.21.15.31:0/3482312254 >> v1:172.21.15.31:6789/0 conn(0x7f9c18159550 legacy=0x7f9c180ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:21.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.779+0000 7f9c177fe700 1 -- 172.21.15.31:0/3482312254 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c180ac5b0 con 0x7f9c18159920 2023-10-06T02:26:21.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.780+0000 7f9c177fe700 1 --2- 172.21.15.31:0/3482312254 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9c18159920 0x7f9c180ad090 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9c10009e90 tx=0x7f9c10009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95cb41f64f8499f3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:21.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.780+0000 7f9c15ffb700 1 -- 172.21.15.31:0/3482312254 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c1000e070 con 0x7f9c18159920 2023-10-06T02:26:21.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.780+0000 7f9c15ffb700 1 -- 172.21.15.31:0/3482312254 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c1000bb80 con 0x7f9c18159920 2023-10-06T02:26:21.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.780+0000 7f9c15ffb700 1 -- 172.21.15.31:0/3482312254 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c10012430 con 0x7f9c18159920 2023-10-06T02:26:21.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.780+0000 7f9c17fff700 1 -- 172.21.15.31:0/3482312254 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9c18159920 msgr2=0x7f9c180ad090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:21.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.780+0000 7f9c17fff700 1 --2- 172.21.15.31:0/3482312254 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9c18159920 0x7f9c180ad090 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9c10009e90 tx=0x7f9c10009780 comp rx=0 tx=0).stop 2023-10-06T02:26:21.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.780+0000 7f9c17fff700 1 -- 172.21.15.31:0/3482312254 shutdown_connections 2023-10-06T02:26:21.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.780+0000 7f9c17fff700 1 --2- 172.21.15.31:0/3482312254 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9c18159920 0x7f9c180ad090 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:21.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.780+0000 7f9c17fff700 1 -- 172.21.15.31:0/3482312254 >> 172.21.15.31:0/3482312254 conn(0x7f9c1806ce80 msgr2=0x7f9c180ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:21.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.781+0000 7f9c17fff700 1 -- 172.21.15.31:0/3482312254 shutdown_connections 2023-10-06T02:26:21.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.781+0000 7f9c17fff700 1 -- 172.21.15.31:0/3482312254 wait complete. 2023-10-06T02:26:21.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.782+0000 7f9c17fff700 1 Processor -- start 2023-10-06T02:26:21.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.782+0000 7f9c17fff700 1 -- start start 2023-10-06T02:26:21.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.783+0000 7f9c17fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c18159550 0x7f9c18152ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:21.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.783+0000 7f9c17fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c18159920 0x7f9c180c9b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:21.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.783+0000 7f9c17fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9c1815dfd0 0x7f9c181698e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:21.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.783+0000 7f9c17fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c180c7560 con 0x7f9c18159920 2023-10-06T02:26:21.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.783+0000 7f9c17fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c18152ff0 con 0x7f9c18159550 2023-10-06T02:26:21.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.783+0000 7f9c17fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c180c8350 con 0x7f9c1815dfd0 2023-10-06T02:26:21.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.783+0000 7f9c167fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c18159920 0x7f9c180c9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:21.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.783+0000 7f9c167fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c18159920 0x7f9c180c9b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57200/0 (socket says 172.21.15.31:57200) 2023-10-06T02:26:21.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.783+0000 7f9c167fc700 1 -- 172.21.15.31:0/3532527064 learned_addr learned my addr 172.21.15.31:0/3532527064 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:26:21.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.783+0000 7f9c177fe700 1 --2- 172.21.15.31:0/3532527064 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9c1815dfd0 0x7f9c181698e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:21.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.783+0000 7f9c16ffd700 1 --2- 172.21.15.31:0/3532527064 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c18159550 0x7f9c18152ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:21.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.784+0000 7f9c167fc700 1 -- 172.21.15.31:0/3532527064 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9c1815dfd0 msgr2=0x7f9c181698e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:21.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.784+0000 7f9c167fc700 1 --2- 172.21.15.31:0/3532527064 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9c1815dfd0 0x7f9c181698e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:21.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.784+0000 7f9c167fc700 1 -- 172.21.15.31:0/3532527064 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c18159550 msgr2=0x7f9c18152ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:21.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.784+0000 7f9c167fc700 1 --2- 172.21.15.31:0/3532527064 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c18159550 0x7f9c18152ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:21.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.784+0000 7f9c167fc700 1 -- 172.21.15.31:0/3532527064 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c10009a20 con 0x7f9c18159920 2023-10-06T02:26:21.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.784+0000 7f9c167fc700 1 --2- 172.21.15.31:0/3532527064 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c18159920 0x7f9c180c9b30 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c00c580 tx=0x7f9c0c007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:21.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.784+0000 7f9bfeffd700 1 -- 172.21.15.31:0/3532527064 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c0c010510 con 0x7f9c18159920 2023-10-06T02:26:21.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.784+0000 7f9bfeffd700 1 -- 172.21.15.31:0/3532527064 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c0c010670 con 0x7f9c18159920 2023-10-06T02:26:21.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.784+0000 7f9bfeffd700 1 -- 172.21.15.31:0/3532527064 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c0c010a30 con 0x7f9c18159920 2023-10-06T02:26:21.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.784+0000 7f9c17fff700 1 -- 172.21.15.31:0/3532527064 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c180c7740 con 0x7f9c18159920 2023-10-06T02:26:21.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.784+0000 7f9c17fff700 1 -- 172.21.15.31:0/3532527064 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c180c7d00 con 0x7f9c18159920 2023-10-06T02:26:21.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.785+0000 7f9c17fff700 1 -- 172.21.15.31:0/3532527064 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c1806f380 con 0x7f9c18159920 2023-10-06T02:26:21.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.786+0000 7f9bfeffd700 1 -- 172.21.15.31:0/3532527064 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 9) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f9c0c01b400 con 0x7f9c18159920 2023-10-06T02:26:21.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.787+0000 7f9bfeffd700 1 --2- 172.21.15.31:0/3532527064 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f9c040670e0 0x7f9c04069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:21.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.787+0000 7f9bfeffd700 1 -- 172.21.15.31:0/3532527064 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f9c0c084710 con 0x7f9c18159920 2023-10-06T02:26:21.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.788+0000 7f9c16ffd700 1 --2- 172.21.15.31:0/3532527064 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f9c040670e0 0x7f9c04069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:21.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.788+0000 7f9c16ffd700 1 --2- 172.21.15.31:0/3532527064 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f9c040670e0 0x7f9c04069590 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9c080058d0 tx=0x7f9c08009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:21.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.791+0000 7f9bfeffd700 1 -- 172.21.15.31:0/3532527064 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9c0c0514c0 con 0x7f9c18159920 2023-10-06T02:26:21.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.992+0000 7f9c17fff700 1 -- 172.21.15.31:0/3532527064 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9c180c84e0 con 0x7f9c18159920 2023-10-06T02:26:21.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.993+0000 7f9bfeffd700 1 -- 172.21.15.31:0/3532527064 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+16914 (secure 0 0 0) 0x7f9c0c054ae0 con 0x7f9c18159920 2023-10-06T02:26:21.995 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:26:21.996 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":49,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:25:48.260740+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:24:42.035212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-10-06T02:24:42.849819+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.2f","mappings":[{"from":0,"to":6}]},{"pgid":"8.3d","mappings":[{"from":0,"to":6}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:26:21.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.995+0000 7f9c17fff700 1 -- 172.21.15.31:0/3532527064 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f9c040670e0 msgr2=0x7f9c04069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:21.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.995+0000 7f9c17fff700 1 --2- 172.21.15.31:0/3532527064 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f9c040670e0 0x7f9c04069590 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9c080058d0 tx=0x7f9c08009040 comp rx=0 tx=0).stop 2023-10-06T02:26:21.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.995+0000 7f9c17fff700 1 -- 172.21.15.31:0/3532527064 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c18159920 msgr2=0x7f9c180c9b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:21.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.995+0000 7f9c17fff700 1 --2- 172.21.15.31:0/3532527064 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c18159920 0x7f9c180c9b30 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c00c580 tx=0x7f9c0c007e10 comp rx=0 tx=0).stop 2023-10-06T02:26:21.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.995+0000 7f9c17fff700 1 -- 172.21.15.31:0/3532527064 shutdown_connections 2023-10-06T02:26:21.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.995+0000 7f9c17fff700 1 --2- 172.21.15.31:0/3532527064 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f9c040670e0 0x7f9c04069590 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:21.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.995+0000 7f9c17fff700 1 --2- 172.21.15.31:0/3532527064 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9c1815dfd0 0x7f9c181698e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:21.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.995+0000 7f9c17fff700 1 --2- 172.21.15.31:0/3532527064 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c18159550 0x7f9c18152ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:21.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.995+0000 7f9c17fff700 1 --2- 172.21.15.31:0/3532527064 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c18159920 0x7f9c180c9b30 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:21.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.995+0000 7f9c17fff700 1 -- 172.21.15.31:0/3532527064 >> 172.21.15.31:0/3532527064 conn(0x7f9c1806ce80 msgr2=0x7f9c180b0210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:21.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.995+0000 7f9c17fff700 1 -- 172.21.15.31:0/3532527064 shutdown_connections 2023-10-06T02:26:21.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:21.996+0000 7f9c17fff700 1 -- 172.21.15.31:0/3532527064 wait complete. 2023-10-06T02:26:22.014 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T02:26:22.015 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:26:22.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.200+0000 7f712999d700 1 Processor -- start 2023-10-06T02:26:22.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.200+0000 7f712999d700 1 -- start start 2023-10-06T02:26:22.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.201+0000 7f712999d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f712415de50 0x7f71240cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:22.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.201+0000 7f712999d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f71240cef40 0x7f712415c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:22.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.202+0000 7f712999d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f712415afd0 con 0x7f7124153c30 2023-10-06T02:26:22.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.202+0000 7f712999d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f712415b2b0 con 0x7f71240cef40 2023-10-06T02:26:22.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.202+0000 7f712999d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f712415b590 con 0x7f712415de50 2023-10-06T02:26:22.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.202+0000 7f712919c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f71240cef40 0x7f712415c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:22.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.202+0000 7f7123fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f712415de50 0x7f71240cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:22.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.202+0000 7f712919c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f71240cef40 0x7f712415c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57210/0 (socket says 172.21.15.31:57210) 2023-10-06T02:26:22.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.202+0000 7f712899b700 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f7124153c30 0x7f712415a5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:35548/0 (socket says 172.21.15.31:35548) 2023-10-06T02:26:22.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.202+0000 7f712919c700 1 -- 172.21.15.31:0/1228289436 learned_addr learned my addr 172.21.15.31:0/1228289436 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:26:22.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.202+0000 7f712919c700 1 -- 172.21.15.31:0/1228289436 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f712415de50 msgr2=0x7f71240cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:26:22.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.202+0000 7f712919c700 1 --2- 172.21.15.31:0/1228289436 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f712415de50 0x7f71240cea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:22.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.202+0000 7f712919c700 1 -- 172.21.15.31:0/1228289436 >> v1:172.21.15.62:6789/0 conn(0x7f7124153c30 legacy=0x7f712415a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:22.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.202+0000 7f712919c700 1 -- 172.21.15.31:0/1228289436 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f712415b870 con 0x7f71240cef40 2023-10-06T02:26:22.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.202+0000 7f71237fe700 1 -- 172.21.15.31:0/1228289436 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 167467268 0 0) 0x7f712415afd0 con 0x7f7124153c30 2023-10-06T02:26:22.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.203+0000 7f712919c700 1 --2- 172.21.15.31:0/1228289436 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71240cef40 0x7f712415c070 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f711c009f20 tx=0x7f711c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81c5c01b3479a976 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:22.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.203+0000 7f71237fe700 1 -- 172.21.15.31:0/1228289436 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f711c00e070 con 0x7f71240cef40 2023-10-06T02:26:22.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.203+0000 7f71237fe700 1 -- 172.21.15.31:0/1228289436 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f711c00b9d0 con 0x7f71240cef40 2023-10-06T02:26:22.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.203+0000 7f71237fe700 1 -- 172.21.15.31:0/1228289436 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f711c0092e0 con 0x7f71240cef40 2023-10-06T02:26:22.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.203+0000 7f712999d700 1 -- 172.21.15.31:0/1228289436 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71240cef40 msgr2=0x7f712415c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:22.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.203+0000 7f712999d700 1 --2- 172.21.15.31:0/1228289436 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71240cef40 0x7f712415c070 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f711c009f20 tx=0x7f711c009840 comp rx=0 tx=0).stop 2023-10-06T02:26:22.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.203+0000 7f712999d700 1 -- 172.21.15.31:0/1228289436 shutdown_connections 2023-10-06T02:26:22.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.203+0000 7f712999d700 1 --2- 172.21.15.31:0/1228289436 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71240cef40 0x7f712415c070 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:22.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.203+0000 7f712999d700 1 --2- 172.21.15.31:0/1228289436 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f712415de50 0x7f71240cea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:22.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.203+0000 7f712999d700 1 -- 172.21.15.31:0/1228289436 >> 172.21.15.31:0/1228289436 conn(0x7f71240703a0 msgr2=0x7f71240707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:22.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.204+0000 7f712999d700 1 -- 172.21.15.31:0/1228289436 shutdown_connections 2023-10-06T02:26:22.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.204+0000 7f712999d700 1 -- 172.21.15.31:0/1228289436 wait complete. 2023-10-06T02:26:22.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.205+0000 7f712999d700 1 Processor -- start 2023-10-06T02:26:22.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.205+0000 7f712999d700 1 -- start start 2023-10-06T02:26:22.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.206+0000 7f712999d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71240cef40 0x7f71240ccf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:22.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.206+0000 7f712999d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7124153c30 0x7f7124151bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:22.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.206+0000 7f712999d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f712415de50 0x7f71240b17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:22.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.206+0000 7f712999d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71240b60f0 con 0x7f71240cef40 2023-10-06T02:26:22.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.206+0000 7f712999d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71240b5170 con 0x7f7124153c30 2023-10-06T02:26:22.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.206+0000 7f712999d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71240cd4a0 con 0x7f712415de50 2023-10-06T02:26:22.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.206+0000 7f712899b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71240cef40 0x7f71240ccf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:22.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.206+0000 7f712899b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71240cef40 0x7f71240ccf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57218/0 (socket says 172.21.15.31:57218) 2023-10-06T02:26:22.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.206+0000 7f712899b700 1 -- 172.21.15.31:0/1130274341 learned_addr learned my addr 172.21.15.31:0/1130274341 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:26:22.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.207+0000 7f712919c700 1 --2- 172.21.15.31:0/1130274341 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f712415de50 0x7f71240b17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:22.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.207+0000 7f7123fff700 1 --2- 172.21.15.31:0/1130274341 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7124153c30 0x7f7124151bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:22.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.207+0000 7f712899b700 1 -- 172.21.15.31:0/1130274341 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f712415de50 msgr2=0x7f71240b17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:22.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.207+0000 7f712899b700 1 --2- 172.21.15.31:0/1130274341 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f712415de50 0x7f71240b17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:22.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.207+0000 7f712899b700 1 -- 172.21.15.31:0/1130274341 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7124153c30 msgr2=0x7f7124151bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:22.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.207+0000 7f712899b700 1 --2- 172.21.15.31:0/1130274341 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7124153c30 0x7f7124151bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:22.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.207+0000 7f712899b700 1 -- 172.21.15.31:0/1130274341 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f711c009ae0 con 0x7f71240cef40 2023-10-06T02:26:22.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.207+0000 7f712899b700 1 --2- 172.21.15.31:0/1130274341 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71240cef40 0x7f71240ccf60 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f711800c7e0 tx=0x7f7118007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:22.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.207+0000 7f7120ff9700 1 -- 172.21.15.31:0/1130274341 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7118005a70 con 0x7f71240cef40 2023-10-06T02:26:22.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.207+0000 7f7120ff9700 1 -- 172.21.15.31:0/1130274341 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7118005bd0 con 0x7f71240cef40 2023-10-06T02:26:22.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.207+0000 7f7120ff9700 1 -- 172.21.15.31:0/1130274341 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71180070d0 con 0x7f71240cef40 2023-10-06T02:26:22.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.207+0000 7f712999d700 1 -- 172.21.15.31:0/1130274341 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71240b4560 con 0x7f71240cef40 2023-10-06T02:26:22.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.208+0000 7f712999d700 1 -- 172.21.15.31:0/1130274341 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71240b4b20 con 0x7f71240cef40 2023-10-06T02:26:22.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.208+0000 7f712999d700 1 -- 172.21.15.31:0/1130274341 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71240b5450 con 0x7f71240cef40 2023-10-06T02:26:22.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.209+0000 7f7120ff9700 1 -- 172.21.15.31:0/1130274341 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 9) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f7118007440 con 0x7f71240cef40 2023-10-06T02:26:22.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.210+0000 7f7120ff9700 1 --2- 172.21.15.31:0/1130274341 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f71080670e0 0x7f7108069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:22.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.210+0000 7f7120ff9700 1 -- 172.21.15.31:0/1130274341 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f7118083f30 con 0x7f71240cef40 2023-10-06T02:26:22.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.211+0000 7f7123fff700 1 --2- 172.21.15.31:0/1130274341 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f71080670e0 0x7f7108069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:22.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.211+0000 7f7123fff700 1 --2- 172.21.15.31:0/1130274341 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f71080670e0 0x7f7108069590 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f71240cc9b0 tx=0x7f71100091a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:22.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.214+0000 7f7120ff9700 1 -- 172.21.15.31:0/1130274341 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7118050ce0 con 0x7f71240cef40 2023-10-06T02:26:22.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.462+0000 7f712999d700 1 -- 172.21.15.31:0/1130274341 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f71240cd660 con 0x7f71240cef40 2023-10-06T02:26:22.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.463+0000 7f7120ff9700 1 -- 172.21.15.31:0/1130274341 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4594 (secure 0 0 0) 0x7f7118017070 con 0x7f71240cef40 2023-10-06T02:26:22.464 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:26:22.465 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4949,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/178547366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":178547366},{"type":"v1","addr":"172.21.15.62:6835","nonce":178547366}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":5278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/2578831517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":2578831517},{"type":"v1","addr":"172.21.15.31:6836","nonce":2578831517}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5287,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3502585737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":3502585737},{"type":"v1","addr":"172.21.15.31:6837","nonce":3502585737}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:24:43.872314+0000","modified":"2023-10-06T02:24:47.404607+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5290":{"gid":5290,"name":"b","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6834/2358776861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2358776861},{"type":"v1","addr":"172.21.15.62:6834","nonce":2358776861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T02:26:22.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.465+0000 7f712999d700 1 -- 172.21.15.31:0/1130274341 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f71080670e0 msgr2=0x7f7108069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:22.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.465+0000 7f712999d700 1 --2- 172.21.15.31:0/1130274341 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f71080670e0 0x7f7108069590 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f71240cc9b0 tx=0x7f71100091a0 comp rx=0 tx=0).stop 2023-10-06T02:26:22.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.465+0000 7f712999d700 1 -- 172.21.15.31:0/1130274341 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71240cef40 msgr2=0x7f71240ccf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:22.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.465+0000 7f712999d700 1 --2- 172.21.15.31:0/1130274341 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71240cef40 0x7f71240ccf60 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f711800c7e0 tx=0x7f7118007e10 comp rx=0 tx=0).stop 2023-10-06T02:26:22.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.465+0000 7f712999d700 1 -- 172.21.15.31:0/1130274341 shutdown_connections 2023-10-06T02:26:22.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.465+0000 7f712999d700 1 --2- 172.21.15.31:0/1130274341 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f71080670e0 0x7f7108069590 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:22.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.465+0000 7f712999d700 1 --2- 172.21.15.31:0/1130274341 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f712415de50 0x7f71240b17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:22.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.465+0000 7f712999d700 1 --2- 172.21.15.31:0/1130274341 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71240cef40 0x7f71240ccf60 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:22.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.465+0000 7f712999d700 1 --2- 172.21.15.31:0/1130274341 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7124153c30 0x7f7124151bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:22.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.465+0000 7f712999d700 1 -- 172.21.15.31:0/1130274341 >> 172.21.15.31:0/1130274341 conn(0x7f71240703a0 msgr2=0x7f71240c52d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:22.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.465+0000 7f712999d700 1 -- 172.21.15.31:0/1130274341 shutdown_connections 2023-10-06T02:26:22.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.465+0000 7f712999d700 1 -- 172.21.15.31:0/1130274341 wait complete. 2023-10-06T02:26:22.469 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 27 2023-10-06T02:26:22.484 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:26:22.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.670+0000 7f905999d700 1 Processor -- start 2023-10-06T02:26:22.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.670+0000 7f905999d700 1 -- start start 2023-10-06T02:26:22.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.672+0000 7f905999d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f905413fb20 0x7f905415a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:22.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.672+0000 7f905999d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f905415ae70 con 0x7f905406aaa0 2023-10-06T02:26:22.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.672+0000 7f905999d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f905406af60 con 0x7f905413fef0 2023-10-06T02:26:22.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.672+0000 7f905999d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f905406b240 con 0x7f905413fb20 2023-10-06T02:26:22.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.672+0000 7f905919c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f905406aaa0 0x7f90540c9d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60648/0 (socket says 172.21.15.31:60648) 2023-10-06T02:26:22.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.672+0000 7f905919c700 1 -- 172.21.15.31:0/3793450581 learned_addr learned my addr 172.21.15.31:0/3793450581 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:26:22.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.672+0000 7f905899b700 1 --2- 172.21.15.31:0/3793450581 >> v2:172.21.15.62:3300/0 conn(0x7f905413fb20 0x7f905415a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:22.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.672+0000 7f90537fe700 1 -- 172.21.15.31:0/3793450581 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1317776252 0 0) 0x7f904c001490 con 0x7f905406aaa0 2023-10-06T02:26:22.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.672+0000 7f90537fe700 1 -- 172.21.15.31:0/3793450581 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9044003570 con 0x7f905406aaa0 2023-10-06T02:26:22.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.673+0000 7f905899b700 1 -- 172.21.15.31:0/3793450581 >> v1:172.21.15.62:6789/0 conn(0x7f905413fef0 legacy=0x7f90540c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:22.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.673+0000 7f90537fe700 1 -- 172.21.15.31:0/3793450581 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 394145867 0 0) 0x7f905406af60 con 0x7f905413fef0 2023-10-06T02:26:22.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.673+0000 7f905899b700 1 -- 172.21.15.31:0/3793450581 >> v1:172.21.15.31:6789/0 conn(0x7f905406aaa0 legacy=0x7f90540c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:22.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.673+0000 7f905899b700 1 -- 172.21.15.31:0/3793450581 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f905406b520 con 0x7f905413fb20 2023-10-06T02:26:22.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.673+0000 7f905899b700 1 --2- 172.21.15.31:0/3793450581 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f905413fb20 0x7f905415a930 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f90480098e0 tx=0x7f90480091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25deaac363bfcf82 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:22.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.673+0000 7f90537fe700 1 -- 172.21.15.31:0/3793450581 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9048011630 con 0x7f905413fb20 2023-10-06T02:26:22.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.673+0000 7f90537fe700 1 -- 172.21.15.31:0/3793450581 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9048011790 con 0x7f905413fb20 2023-10-06T02:26:22.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.673+0000 7f90537fe700 1 -- 172.21.15.31:0/3793450581 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9048011b30 con 0x7f905413fb20 2023-10-06T02:26:22.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.674+0000 7f905999d700 1 -- 172.21.15.31:0/3793450581 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f905413fb20 msgr2=0x7f905415a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:22.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.674+0000 7f905999d700 1 --2- 172.21.15.31:0/3793450581 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f905413fb20 0x7f905415a930 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f90480098e0 tx=0x7f90480091a0 comp rx=0 tx=0).stop 2023-10-06T02:26:22.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.674+0000 7f905999d700 1 -- 172.21.15.31:0/3793450581 shutdown_connections 2023-10-06T02:26:22.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.674+0000 7f905999d700 1 --2- 172.21.15.31:0/3793450581 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f905413fb20 0x7f905415a930 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:22.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.674+0000 7f905999d700 1 -- 172.21.15.31:0/3793450581 >> 172.21.15.31:0/3793450581 conn(0x7f905406d170 msgr2=0x7f90540cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:22.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.674+0000 7f905999d700 1 -- 172.21.15.31:0/3793450581 shutdown_connections 2023-10-06T02:26:22.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.674+0000 7f905999d700 1 -- 172.21.15.31:0/3793450581 wait complete. 2023-10-06T02:26:22.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.675+0000 7f905999d700 1 Processor -- start 2023-10-06T02:26:22.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.676+0000 7f905999d700 1 -- start start 2023-10-06T02:26:22.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.676+0000 7f905999d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f905406aaa0 0x7f905413c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:22.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.676+0000 7f905999d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f905413fb20 0x7f90540c0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:22.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.676+0000 7f905999d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f905413fef0 0x7f90540c1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:22.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.676+0000 7f905999d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90540ad040 con 0x7f905413fef0 2023-10-06T02:26:22.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.676+0000 7f905999d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f905413ccf0 con 0x7f905413fb20 2023-10-06T02:26:22.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.676+0000 7f905999d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f905416a5d0 con 0x7f905406aaa0 2023-10-06T02:26:22.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.677+0000 7f905919c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f905413fef0 0x7f90540c1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:22.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.677+0000 7f905919c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f905413fef0 0x7f90540c1410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57222/0 (socket says 172.21.15.31:57222) 2023-10-06T02:26:22.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.677+0000 7f905919c700 1 -- 172.21.15.31:0/3097956509 learned_addr learned my addr 172.21.15.31:0/3097956509 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:26:22.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.677+0000 7f905899b700 1 --2- 172.21.15.31:0/3097956509 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f905406aaa0 0x7f905413c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:22.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.677+0000 7f9053fff700 1 --2- 172.21.15.31:0/3097956509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f905413fb20 0x7f90540c0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:22.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.677+0000 7f905919c700 1 -- 172.21.15.31:0/3097956509 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f905406aaa0 msgr2=0x7f905413c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:22.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.677+0000 7f905919c700 1 --2- 172.21.15.31:0/3097956509 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f905406aaa0 0x7f905413c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:22.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.677+0000 7f905919c700 1 -- 172.21.15.31:0/3097956509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f905413fb20 msgr2=0x7f90540c0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:22.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.677+0000 7f905919c700 1 --2- 172.21.15.31:0/3097956509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f905413fb20 0x7f90540c0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:22.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.677+0000 7f905919c700 1 -- 172.21.15.31:0/3097956509 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9048009420 con 0x7f905413fef0 2023-10-06T02:26:22.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.677+0000 7f9053fff700 1 --2- 172.21.15.31:0/3097956509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f905413fb20 0x7f90540c0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:26:22.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.677+0000 7f905919c700 1 --2- 172.21.15.31:0/3097956509 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f905413fef0 0x7f90540c1410 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f904c00cb60 tx=0x7f904c007e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:22.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.678+0000 7f9050ff9700 1 -- 172.21.15.31:0/3097956509 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f904c004dc0 con 0x7f905413fef0 2023-10-06T02:26:22.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.678+0000 7f9050ff9700 1 -- 172.21.15.31:0/3097956509 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f904c00a5d0 con 0x7f905413fef0 2023-10-06T02:26:22.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.678+0000 7f9050ff9700 1 -- 172.21.15.31:0/3097956509 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f904c00a990 con 0x7f905413fef0 2023-10-06T02:26:22.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.678+0000 7f905999d700 1 -- 172.21.15.31:0/3097956509 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90540ac710 con 0x7f905413fef0 2023-10-06T02:26:22.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.678+0000 7f905999d700 1 -- 172.21.15.31:0/3097956509 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90540accd0 con 0x7f905413fef0 2023-10-06T02:26:22.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.679+0000 7f905999d700 1 -- 172.21.15.31:0/3097956509 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f905404c3c0 con 0x7f905413fef0 2023-10-06T02:26:22.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.680+0000 7f9050ff9700 1 -- 172.21.15.31:0/3097956509 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 9) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f904c00ad40 con 0x7f905413fef0 2023-10-06T02:26:22.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.680+0000 7f9050ff9700 1 --2- 172.21.15.31:0/3097956509 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f90380670e0 0x7f9038069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:22.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.680+0000 7f9050ff9700 1 -- 172.21.15.31:0/3097956509 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f904c084230 con 0x7f905413fef0 2023-10-06T02:26:22.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.681+0000 7f905899b700 1 --2- 172.21.15.31:0/3097956509 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f90380670e0 0x7f9038069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:22.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.681+0000 7f905899b700 1 --2- 172.21.15.31:0/3097956509 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f90380670e0 0x7f9038069590 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f90480091e0 tx=0x7f9048016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:22.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.684+0000 7f9050ff9700 1 -- 172.21.15.31:0/3097956509 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f904c051060 con 0x7f905413fef0 2023-10-06T02:26:22.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.929+0000 7f905999d700 1 -- 172.21.15.31:0/3097956509 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f905415ae70 con 0x7f905413fef0 2023-10-06T02:26:22.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.930+0000 7f9050ff9700 1 -- 172.21.15.31:0/3097956509 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4594 (secure 0 0 0) 0x7f905415ae70 con 0x7f905413fef0 2023-10-06T02:26:22.932 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:26:22.932 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4949,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/178547366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":178547366},{"type":"v1","addr":"172.21.15.62:6835","nonce":178547366}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":5278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/2578831517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":2578831517},{"type":"v1","addr":"172.21.15.31:6836","nonce":2578831517}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5287,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3502585737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":3502585737},{"type":"v1","addr":"172.21.15.31:6837","nonce":3502585737}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:24:43.872314+0000","modified":"2023-10-06T02:24:47.404607+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5290":{"gid":5290,"name":"b","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6834/2358776861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2358776861},{"type":"v1","addr":"172.21.15.62:6834","nonce":2358776861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T02:26:22.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.932+0000 7f905999d700 1 -- 172.21.15.31:0/3097956509 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f90380670e0 msgr2=0x7f9038069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:22.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.932+0000 7f905999d700 1 --2- 172.21.15.31:0/3097956509 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f90380670e0 0x7f9038069590 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f90480091e0 tx=0x7f9048016000 comp rx=0 tx=0).stop 2023-10-06T02:26:22.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.932+0000 7f905999d700 1 -- 172.21.15.31:0/3097956509 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f905413fef0 msgr2=0x7f90540c1410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:22.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.932+0000 7f905999d700 1 --2- 172.21.15.31:0/3097956509 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f905413fef0 0x7f90540c1410 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f904c00cb60 tx=0x7f904c007e90 comp rx=0 tx=0).stop 2023-10-06T02:26:22.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.932+0000 7f905999d700 1 -- 172.21.15.31:0/3097956509 shutdown_connections 2023-10-06T02:26:22.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.932+0000 7f905999d700 1 --2- 172.21.15.31:0/3097956509 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f905413fef0 0x7f90540c1410 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:22.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.933+0000 7f905999d700 1 --2- 172.21.15.31:0/3097956509 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f90380670e0 0x7f9038069590 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:22.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.933+0000 7f905999d700 1 --2- 172.21.15.31:0/3097956509 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f905406aaa0 0x7f905413c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:22.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.933+0000 7f905999d700 1 --2- 172.21.15.31:0/3097956509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f905413fb20 0x7f90540c0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:22.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.933+0000 7f905999d700 1 -- 172.21.15.31:0/3097956509 >> 172.21.15.31:0/3097956509 conn(0x7f905406d170 msgr2=0x7f90540b15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:22.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.933+0000 7f905999d700 1 -- 172.21.15.31:0/3097956509 shutdown_connections 2023-10-06T02:26:22.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:22.933+0000 7f905999d700 1 -- 172.21.15.31:0/3097956509 wait complete. 2023-10-06T02:26:22.938 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 27 2023-10-06T02:26:22.951 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:26:23.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.137+0000 7f6bf519d700 1 Processor -- start 2023-10-06T02:26:23.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.137+0000 7f6bf519d700 1 -- start start 2023-10-06T02:26:23.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.138+0000 7f6bf519d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f6bf0153c30 0x7f6bf015a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:23.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.138+0000 7f6bf519d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bf015afd0 con 0x7f6bf015de50 2023-10-06T02:26:23.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.138+0000 7f6bf519d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bf015b2b0 con 0x7f6bf00cf110 2023-10-06T02:26:23.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.138+0000 7f6bf519d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bf015b590 con 0x7f6bf0153c30 2023-10-06T02:26:23.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.139+0000 7f6bef7fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f6bf015de50 0x7f6bf00cea00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60660/0 (socket says 172.21.15.31:60660) 2023-10-06T02:26:23.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.139+0000 7f6bef7fe700 1 -- 172.21.15.31:0/2526249329 learned_addr learned my addr 172.21.15.31:0/2526249329 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:26:23.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.139+0000 7f6beffff700 1 --2- 172.21.15.31:0/2526249329 >> v2:172.21.15.62:3300/0 conn(0x7f6bf0153c30 0x7f6bf015a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:23.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.139+0000 7f6beeffd700 1 -- 172.21.15.31:0/2526249329 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 482308518 0 0) 0x7f6bf015afd0 con 0x7f6bf015de50 2023-10-06T02:26:23.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.139+0000 7f6beeffd700 1 -- 172.21.15.31:0/2526249329 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6bd8003570 con 0x7f6bf015de50 2023-10-06T02:26:23.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.139+0000 7f6beffff700 1 -- 172.21.15.31:0/2526249329 >> v1:172.21.15.62:6789/0 conn(0x7f6bf00cf110 legacy=0x7f6bf015c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:23.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.139+0000 7f6beffff700 1 -- 172.21.15.31:0/2526249329 >> v1:172.21.15.31:6789/0 conn(0x7f6bf015de50 legacy=0x7f6bf00cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:23.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.139+0000 7f6beffff700 1 -- 172.21.15.31:0/2526249329 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bf015b870 con 0x7f6bf0153c30 2023-10-06T02:26:23.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.139+0000 7f6beffff700 1 --2- 172.21.15.31:0/2526249329 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bf0153c30 0x7f6bf015a5e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6be00098e0 tx=0x7f6be00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fface577616a5370 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:23.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.140+0000 7f6beeffd700 1 -- 172.21.15.31:0/2526249329 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6be000b640 con 0x7f6bf0153c30 2023-10-06T02:26:23.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.140+0000 7f6beeffd700 1 -- 172.21.15.31:0/2526249329 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6be000b7a0 con 0x7f6bf0153c30 2023-10-06T02:26:23.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.140+0000 7f6beeffd700 1 -- 172.21.15.31:0/2526249329 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6be000bb40 con 0x7f6bf0153c30 2023-10-06T02:26:23.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.140+0000 7f6bf519d700 1 -- 172.21.15.31:0/2526249329 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bf0153c30 msgr2=0x7f6bf015a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:23.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.140+0000 7f6bf519d700 1 --2- 172.21.15.31:0/2526249329 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bf0153c30 0x7f6bf015a5e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6be00098e0 tx=0x7f6be00091a0 comp rx=0 tx=0).stop 2023-10-06T02:26:23.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.140+0000 7f6bf519d700 1 -- 172.21.15.31:0/2526249329 shutdown_connections 2023-10-06T02:26:23.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.140+0000 7f6bf519d700 1 --2- 172.21.15.31:0/2526249329 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bf0153c30 0x7f6bf015a5e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:23.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.140+0000 7f6bf519d700 1 -- 172.21.15.31:0/2526249329 >> 172.21.15.31:0/2526249329 conn(0x7f6bf00703a0 msgr2=0x7f6bf00707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:23.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.140+0000 7f6bf519d700 1 -- 172.21.15.31:0/2526249329 shutdown_connections 2023-10-06T02:26:23.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.141+0000 7f6bf519d700 1 -- 172.21.15.31:0/2526249329 wait complete. 2023-10-06T02:26:23.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.142+0000 7f6bf519d700 1 Processor -- start 2023-10-06T02:26:23.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.142+0000 7f6bf519d700 1 -- start start 2023-10-06T02:26:23.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.143+0000 7f6bf519d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bf00cf110 0x7f6bf0146320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:23.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.143+0000 7f6bf519d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6bf0153c30 0x7f6bf00c9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:23.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.143+0000 7f6bf519d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bf015de50 0x7f6bf015e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:23.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.143+0000 7f6bf519d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bf01439e0 con 0x7f6bf015de50 2023-10-06T02:26:23.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.143+0000 7f6bf519d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bf00c79a0 con 0x7f6bf00cf110 2023-10-06T02:26:23.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.143+0000 7f6bf519d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bf00c7c80 con 0x7f6bf0153c30 2023-10-06T02:26:23.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.143+0000 7f6bf499c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bf015de50 0x7f6bf015e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:23.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.143+0000 7f6bf499c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bf015de50 0x7f6bf015e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57238/0 (socket says 172.21.15.31:57238) 2023-10-06T02:26:23.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.143+0000 7f6bf499c700 1 -- 172.21.15.31:0/4069039657 learned_addr learned my addr 172.21.15.31:0/4069039657 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:26:23.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.144+0000 7f6beffff700 1 --2- 172.21.15.31:0/4069039657 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bf00cf110 0x7f6bf0146320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:23.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.144+0000 7f6bef7fe700 1 --2- 172.21.15.31:0/4069039657 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6bf0153c30 0x7f6bf00c9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:23.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.144+0000 7f6bf499c700 1 -- 172.21.15.31:0/4069039657 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6bf0153c30 msgr2=0x7f6bf00c9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:23.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.144+0000 7f6bf499c700 1 --2- 172.21.15.31:0/4069039657 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6bf0153c30 0x7f6bf00c9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:23.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.144+0000 7f6bf499c700 1 -- 172.21.15.31:0/4069039657 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bf00cf110 msgr2=0x7f6bf0146320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:23.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.144+0000 7f6bf499c700 1 --2- 172.21.15.31:0/4069039657 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bf00cf110 0x7f6bf0146320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:23.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.144+0000 7f6bf499c700 1 -- 172.21.15.31:0/4069039657 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6be0009420 con 0x7f6bf015de50 2023-10-06T02:26:23.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.144+0000 7f6bf499c700 1 --2- 172.21.15.31:0/4069039657 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bf015de50 0x7f6bf015e3c0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f6be800c8e0 tx=0x7f6be8001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:23.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.144+0000 7f6bd7fff700 1 -- 172.21.15.31:0/4069039657 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6be8012630 con 0x7f6bf015de50 2023-10-06T02:26:23.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.144+0000 7f6bd7fff700 1 -- 172.21.15.31:0/4069039657 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6be8012790 con 0x7f6bf015de50 2023-10-06T02:26:23.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.144+0000 7f6bd7fff700 1 -- 172.21.15.31:0/4069039657 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6be8012b50 con 0x7f6bf015de50 2023-10-06T02:26:23.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.144+0000 7f6bf519d700 1 -- 172.21.15.31:0/4069039657 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bf00c7f60 con 0x7f6bf015de50 2023-10-06T02:26:23.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.145+0000 7f6bf519d700 1 -- 172.21.15.31:0/4069039657 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bf00c8520 con 0x7f6bf015de50 2023-10-06T02:26:23.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.146+0000 7f6bf519d700 1 -- 172.21.15.31:0/4069039657 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bf00b32d0 con 0x7f6bf015de50 2023-10-06T02:26:23.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.147+0000 7f6bd7fff700 1 -- 172.21.15.31:0/4069039657 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 9) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f6be800a3b0 con 0x7f6bf015de50 2023-10-06T02:26:23.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.148+0000 7f6bd7fff700 1 --2- 172.21.15.31:0/4069039657 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6bdc0670e0 0x7f6bdc069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:23.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.148+0000 7f6bd7fff700 1 -- 172.21.15.31:0/4069039657 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f6be8084c70 con 0x7f6bf015de50 2023-10-06T02:26:23.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.148+0000 7f6beffff700 1 --2- 172.21.15.31:0/4069039657 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6bdc0670e0 0x7f6bdc069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:23.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.149+0000 7f6beffff700 1 --2- 172.21.15.31:0/4069039657 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6bdc0670e0 0x7f6bdc069590 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6be00093f0 tx=0x7f6be0017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:23.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.152+0000 7f6bd7fff700 1 -- 172.21.15.31:0/4069039657 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6be8051ac0 con 0x7f6bf015de50 2023-10-06T02:26:23.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.357+0000 7f6bf519d700 1 -- 172.21.15.31:0/4069039657 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6bf004c3c0 con 0x7f6bf015de50 2023-10-06T02:26:23.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.358+0000 7f6bd7fff700 1 -- 172.21.15.31:0/4069039657 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+16914 (secure 0 0 0) 0x7f6be80550e0 con 0x7f6bf015de50 2023-10-06T02:26:23.359 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:26:23.360 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":49,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:25:48.260740+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:24:42.035212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-10-06T02:24:42.849819+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.2f","mappings":[{"from":0,"to":6}]},{"pgid":"8.3d","mappings":[{"from":0,"to":6}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:26:23.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.360+0000 7f6bf519d700 1 -- 172.21.15.31:0/4069039657 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6bdc0670e0 msgr2=0x7f6bdc069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:23.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.360+0000 7f6bf519d700 1 --2- 172.21.15.31:0/4069039657 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6bdc0670e0 0x7f6bdc069590 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6be00093f0 tx=0x7f6be0017000 comp rx=0 tx=0).stop 2023-10-06T02:26:23.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.360+0000 7f6bf519d700 1 -- 172.21.15.31:0/4069039657 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bf015de50 msgr2=0x7f6bf015e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:23.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.360+0000 7f6bf519d700 1 --2- 172.21.15.31:0/4069039657 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bf015de50 0x7f6bf015e3c0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f6be800c8e0 tx=0x7f6be8001710 comp rx=0 tx=0).stop 2023-10-06T02:26:23.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.360+0000 7f6bf519d700 1 -- 172.21.15.31:0/4069039657 shutdown_connections 2023-10-06T02:26:23.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.360+0000 7f6bf519d700 1 --2- 172.21.15.31:0/4069039657 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bf015de50 0x7f6bf015e3c0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:23.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.360+0000 7f6bf519d700 1 --2- 172.21.15.31:0/4069039657 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bf00cf110 0x7f6bf0146320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:23.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.360+0000 7f6bf519d700 1 --2- 172.21.15.31:0/4069039657 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6bdc0670e0 0x7f6bdc069590 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:23.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.360+0000 7f6bf519d700 1 --2- 172.21.15.31:0/4069039657 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6bf0153c30 0x7f6bf00c9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:23.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.360+0000 7f6bf519d700 1 -- 172.21.15.31:0/4069039657 >> 172.21.15.31:0/4069039657 conn(0x7f6bf00703a0 msgr2=0x7f6bf015bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:23.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.360+0000 7f6bf519d700 1 -- 172.21.15.31:0/4069039657 shutdown_connections 2023-10-06T02:26:23.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.360+0000 7f6bf519d700 1 -- 172.21.15.31:0/4069039657 wait complete. 2023-10-06T02:26:23.379 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T02:26:23.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.565+0000 7fbf649fe700 1 Processor -- start 2023-10-06T02:26:23.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.565+0000 7fbf649fe700 1 -- start start 2023-10-06T02:26:23.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.566+0000 7fbf649fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbf6015de50 0x7fbf600cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:23.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.566+0000 7fbf649fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fbf600cef40 0x7fbf6015c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:23.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.566+0000 7fbf649fe700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf6015afd0 con 0x7fbf60153c30 2023-10-06T02:26:23.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.566+0000 7fbf649fe700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf6015b2b0 con 0x7fbf600cef40 2023-10-06T02:26:23.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.566+0000 7fbf649fe700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6015b590 con 0x7fbf6015de50 2023-10-06T02:26:23.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.566+0000 7fbf5f59e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fbf600cef40 0x7fbf6015c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:23.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.566+0000 7fbf5f59e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fbf600cef40 0x7fbf6015c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57254/0 (socket says 172.21.15.31:57254) 2023-10-06T02:26:23.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.566+0000 7fbf5f59e700 1 -- 172.21.15.31:0/3420516413 learned_addr learned my addr 172.21.15.31:0/3420516413 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:26:23.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.566+0000 7fbf5e59c700 1 --2- 172.21.15.31:0/3420516413 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbf6015de50 0x7fbf600cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:23.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.567+0000 7fbf5f59e700 1 -- 172.21.15.31:0/3420516413 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbf6015de50 msgr2=0x7fbf600cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:26:23.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.567+0000 7fbf5f59e700 1 --2- 172.21.15.31:0/3420516413 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbf6015de50 0x7fbf600cea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:23.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.567+0000 7fbf5f59e700 1 -- 172.21.15.31:0/3420516413 >> v1:172.21.15.62:6789/0 conn(0x7fbf60153c30 legacy=0x7fbf6015a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:23.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.567+0000 7fbf5f59e700 1 -- 172.21.15.31:0/3420516413 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf6015b870 con 0x7fbf600cef40 2023-10-06T02:26:23.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.567+0000 7fbf5f59e700 1 --2- 172.21.15.31:0/3420516413 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf600cef40 0x7fbf6015c070 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fbf58009f20 tx=0x7fbf58009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13d9ed91c190b838 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:23.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.567+0000 7fbf5dd9b700 1 -- 172.21.15.31:0/3420516413 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5800e070 con 0x7fbf600cef40 2023-10-06T02:26:23.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.567+0000 7fbf5dd9b700 1 -- 172.21.15.31:0/3420516413 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf5800b9d0 con 0x7fbf600cef40 2023-10-06T02:26:23.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.567+0000 7fbf5dd9b700 1 -- 172.21.15.31:0/3420516413 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf580092e0 con 0x7fbf600cef40 2023-10-06T02:26:23.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.568+0000 7fbf649fe700 1 -- 172.21.15.31:0/3420516413 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf600cef40 msgr2=0x7fbf6015c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:23.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.568+0000 7fbf649fe700 1 --2- 172.21.15.31:0/3420516413 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf600cef40 0x7fbf6015c070 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fbf58009f20 tx=0x7fbf58009840 comp rx=0 tx=0).stop 2023-10-06T02:26:23.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.568+0000 7fbf649fe700 1 -- 172.21.15.31:0/3420516413 shutdown_connections 2023-10-06T02:26:23.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.568+0000 7fbf649fe700 1 --2- 172.21.15.31:0/3420516413 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf600cef40 0x7fbf6015c070 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:23.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.568+0000 7fbf649fe700 1 --2- 172.21.15.31:0/3420516413 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbf6015de50 0x7fbf600cea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:23.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.568+0000 7fbf649fe700 1 -- 172.21.15.31:0/3420516413 >> 172.21.15.31:0/3420516413 conn(0x7fbf600703a0 msgr2=0x7fbf600707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:23.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.568+0000 7fbf649fe700 1 -- 172.21.15.31:0/3420516413 shutdown_connections 2023-10-06T02:26:23.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.568+0000 7fbf649fe700 1 -- 172.21.15.31:0/3420516413 wait complete. 2023-10-06T02:26:23.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.569+0000 7fbf649fe700 1 Processor -- start 2023-10-06T02:26:23.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.570+0000 7fbf649fe700 1 -- start start 2023-10-06T02:26:23.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.570+0000 7fbf649fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf600cef40 0x7fbf600c9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:23.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.570+0000 7fbf649fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbf60153c30 0x7fbf60158b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:23.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.570+0000 7fbf649fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf6015de50 0x7fbf60140d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:23.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.571+0000 7fbf649fe700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf60159050 con 0x7fbf6015de50 2023-10-06T02:26:23.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.571+0000 7fbf649fe700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf600b4300 con 0x7fbf600cef40 2023-10-06T02:26:23.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.571+0000 7fbf649fe700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf600b36f0 con 0x7fbf60153c30 2023-10-06T02:26:23.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.571+0000 7fbf5f59e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf6015de50 0x7fbf60140d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:23.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.571+0000 7fbf5f59e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf6015de50 0x7fbf60140d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57262/0 (socket says 172.21.15.31:57262) 2023-10-06T02:26:23.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.571+0000 7fbf5f59e700 1 -- 172.21.15.31:0/2782801775 learned_addr learned my addr 172.21.15.31:0/2782801775 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:26:23.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.571+0000 7fbf5e59c700 1 --2- 172.21.15.31:0/2782801775 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbf60153c30 0x7fbf60158b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:23.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.571+0000 7fbf5ed9d700 1 --2- 172.21.15.31:0/2782801775 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf600cef40 0x7fbf600c9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:23.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.571+0000 7fbf5f59e700 1 -- 172.21.15.31:0/2782801775 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbf60153c30 msgr2=0x7fbf60158b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:23.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.571+0000 7fbf5f59e700 1 --2- 172.21.15.31:0/2782801775 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbf60153c30 0x7fbf60158b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:23.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.571+0000 7fbf5f59e700 1 -- 172.21.15.31:0/2782801775 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf600cef40 msgr2=0x7fbf600c9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:23.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.571+0000 7fbf5f59e700 1 --2- 172.21.15.31:0/2782801775 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf600cef40 0x7fbf600c9fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:23.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.571+0000 7fbf5f59e700 1 -- 172.21.15.31:0/2782801775 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf58009ae0 con 0x7fbf6015de50 2023-10-06T02:26:23.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.572+0000 7fbf5f59e700 1 --2- 172.21.15.31:0/2782801775 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf6015de50 0x7fbf60140d00 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fbf5800b610 tx=0x7fbf58018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:23.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.572+0000 7fbf46ffd700 1 -- 172.21.15.31:0/2782801775 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf58008ad0 con 0x7fbf6015de50 2023-10-06T02:26:23.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.572+0000 7fbf46ffd700 1 -- 172.21.15.31:0/2782801775 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf58008c30 con 0x7fbf6015de50 2023-10-06T02:26:23.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.572+0000 7fbf46ffd700 1 -- 172.21.15.31:0/2782801775 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf58025460 con 0x7fbf6015de50 2023-10-06T02:26:23.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.572+0000 7fbf649fe700 1 -- 172.21.15.31:0/2782801775 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf600b39d0 con 0x7fbf6015de50 2023-10-06T02:26:23.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.572+0000 7fbf649fe700 1 -- 172.21.15.31:0/2782801775 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf600b3f90 con 0x7fbf6015de50 2023-10-06T02:26:23.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.573+0000 7fbf649fe700 1 -- 172.21.15.31:0/2782801775 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf6004c3c0 con 0x7fbf6015de50 2023-10-06T02:26:23.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.574+0000 7fbf46ffd700 1 -- 172.21.15.31:0/2782801775 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 9) v1 ==== 84893+0+0 (secure 0 0 0) 0x7fbf58003040 con 0x7fbf6015de50 2023-10-06T02:26:23.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.575+0000 7fbf46ffd700 1 --2- 172.21.15.31:0/2782801775 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fbf4c0670e0 0x7fbf4c069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:23.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.575+0000 7fbf46ffd700 1 -- 172.21.15.31:0/2782801775 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6116+0+0 (secure 0 0 0) 0x7fbf580860e0 con 0x7fbf6015de50 2023-10-06T02:26:23.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.575+0000 7fbf5ed9d700 1 --2- 172.21.15.31:0/2782801775 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fbf4c0670e0 0x7fbf4c069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:23.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.576+0000 7fbf5ed9d700 1 --2- 172.21.15.31:0/2782801775 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fbf4c0670e0 0x7fbf4c069590 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fbf50003a80 tx=0x7fbf50007ec0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:23.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.579+0000 7fbf46ffd700 1 -- 172.21.15.31:0/2782801775 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbf58052d80 con 0x7fbf6015de50 2023-10-06T02:26:23.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.829+0000 7fbf649fe700 1 -- 172.21.15.31:0/2782801775 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fbf600b4440 con 0x7fbf6015de50 2023-10-06T02:26:23.831 INFO:tasks.ceph.mon.a.smithi031.stderr:2023-10-06T02:26:23.830+0000 7f6de01f1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T02:26:23.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.835+0000 7fbf46ffd700 1 -- 172.21.15.31:0/2782801775 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v28) v1 ==== 161+0+0 (secure 0 0 0) 0x7fbf580563a0 con 0x7fbf6015de50 2023-10-06T02:26:23.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.837+0000 7fbf649fe700 1 -- 172.21.15.31:0/2782801775 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fbf4c0670e0 msgr2=0x7fbf4c069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:23.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.837+0000 7fbf649fe700 1 --2- 172.21.15.31:0/2782801775 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fbf4c0670e0 0x7fbf4c069590 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fbf50003a80 tx=0x7fbf50007ec0 comp rx=0 tx=0).stop 2023-10-06T02:26:23.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.837+0000 7fbf649fe700 1 -- 172.21.15.31:0/2782801775 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf6015de50 msgr2=0x7fbf60140d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:23.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.837+0000 7fbf649fe700 1 --2- 172.21.15.31:0/2782801775 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf6015de50 0x7fbf60140d00 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fbf5800b610 tx=0x7fbf58018000 comp rx=0 tx=0).stop 2023-10-06T02:26:23.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.837+0000 7fbf649fe700 1 -- 172.21.15.31:0/2782801775 shutdown_connections 2023-10-06T02:26:23.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.837+0000 7fbf649fe700 1 --2- 172.21.15.31:0/2782801775 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf6015de50 0x7fbf60140d00 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:23.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.837+0000 7fbf649fe700 1 --2- 172.21.15.31:0/2782801775 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf600cef40 0x7fbf600c9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:23.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.837+0000 7fbf649fe700 1 --2- 172.21.15.31:0/2782801775 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fbf4c0670e0 0x7fbf4c069590 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:23.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.837+0000 7fbf649fe700 1 --2- 172.21.15.31:0/2782801775 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbf60153c30 0x7fbf60158b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:23.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.837+0000 7fbf649fe700 1 -- 172.21.15.31:0/2782801775 >> 172.21.15.31:0/2782801775 conn(0x7fbf600703a0 msgr2=0x7fbf600c6e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:23.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.837+0000 7fbf649fe700 1 -- 172.21.15.31:0/2782801775 shutdown_connections 2023-10-06T02:26:23.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:23.837+0000 7fbf649fe700 1 -- 172.21.15.31:0/2782801775 wait complete. 2023-10-06T02:26:23.840 INFO:teuthology.orchestra.run.smithi031.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T02:26:23.850 INFO:tasks.ceph.mds.b.smithi062.stderr: -8586> 2023-10-06T02:24:45.360+0000 7f631f1f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T02:26:23.856 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T02:26:23.904 INFO:tasks.ceph.mds.b.smithi062.stdout:starting mds.b at 2023-10-06T02:26:24.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:24.043+0000 7f533cb9d700 1 Processor -- start 2023-10-06T02:26:24.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:24.043+0000 7f533cb9d700 1 -- start start 2023-10-06T02:26:24.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:24.044+0000 7f533cb9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f533806de50 0x7f53380c7360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:24.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:24.045+0000 7f533cb9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f533815b640 con 0x7f533806a210 2023-10-06T02:26:24.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:24.045+0000 7f533cb9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f533815b920 con 0x7f5338140a70 2023-10-06T02:26:24.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:24.045+0000 7f533cb9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53380c6800 con 0x7f533806de50 2023-10-06T02:26:24.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:24.045+0000 7f5336ffd700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f533806a210 0x7f53380c6c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60668/0 (socket says 172.21.15.31:60668) 2023-10-06T02:26:24.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:24.045+0000 7f5336ffd700 1 -- 172.21.15.31:0/2102819714 learned_addr learned my addr 172.21.15.31:0/2102819714 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:26:24.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:24.045+0000 7f5337fff700 1 --2- 172.21.15.31:0/2102819714 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f533806de50 0x7f53380c7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:24.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:24.045+0000 7f53367fc700 1 -- 172.21.15.31:0/2102819714 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2305358729 0 0) 0x7f533815b640 con 0x7f533806a210 2023-10-06T02:26:24.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:24.045+0000 7f53367fc700 1 -- 172.21.15.31:0/2102819714 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5320003570 con 0x7f533806a210 2023-10-06T02:26:24.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:24.045+0000 7f53367fc700 1 -- 172.21.15.31:0/2102819714 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2914176515 0 0) 0x7f533815b920 con 0x7f5338140a70 2023-10-06T02:26:24.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:24.045+0000 7f53367fc700 1 -- 172.21.15.31:0/2102819714 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f533815b640 con 0x7f5338140a70 2023-10-06T02:26:24.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:24.046+0000 7f5337fff700 1 -- 172.21.15.31:0/2102819714 >> v1:172.21.15.62:6789/0 conn(0x7f5338140a70 legacy=0x7f5338069b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:24.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:24.046+0000 7f53367fc700 1 -- 172.21.15.31:0/2102819714 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 891954759 0 0) 0x7f5320003570 con 0x7f533806a210 2023-10-06T02:26:24.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:24.046+0000 7f5337fff700 1 -- 172.21.15.31:0/2102819714 >> v1:172.21.15.31:6789/0 conn(0x7f533806a210 legacy=0x7f53380c6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:24.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:24.046+0000 7f5337fff700 1 -- 172.21.15.31:0/2102819714 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f533815bc00 con 0x7f533806de50 2023-10-06T02:26:24.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:24.046+0000 7f5337fff700 1 --2- 172.21.15.31:0/2102819714 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f533806de50 0x7f53380c7360 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5330009e90 tx=0x7f5330009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b503be633da0add server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:24.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:24.046+0000 7f53367fc700 1 -- 172.21.15.31:0/2102819714 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f533000e070 con 0x7f533806de50 2023-10-06T02:26:24.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:24.046+0000 7f53367fc700 1 -- 172.21.15.31:0/2102819714 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f533000b870 con 0x7f533806de50 2023-10-06T02:26:24.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:24.046+0000 7f53367fc700 1 -- 172.21.15.31:0/2102819714 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f532c001470 con 0x7f533806de50 2023-10-06T02:26:24.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:24.047+0000 7f533cb9d700 1 -- 172.21.15.31:0/2102819714 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f533806de50 msgr2=0x7f53380c7360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:24.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:24.047+0000 7f533cb9d700 1 --2- 172.21.15.31:0/2102819714 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f533806de50 0x7f53380c7360 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5330009e90 tx=0x7f5330009780 comp rx=0 tx=0).stop 2023-10-06T02:26:24.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:24.047+0000 7f533cb9d700 1 -- 172.21.15.31:0/2102819714 shutdown_connections 2023-10-06T02:26:24.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:24.047+0000 7f533cb9d700 1 --2- 172.21.15.31:0/2102819714 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f533806de50 0x7f53380c7360 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:24.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:24.047+0000 7f533cb9d700 1 -- 172.21.15.31:0/2102819714 >> 172.21.15.31:0/2102819714 conn(0x7f533806d110 msgr2=0x7f53380cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:24.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:24.047+0000 7f533cb9d700 1 -- 172.21.15.31:0/2102819714 shutdown_connections 2023-10-06T02:26:24.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:24.047+0000 7f533cb9d700 1 -- 172.21.15.31:0/2102819714 wait complete. 2023-10-06T02:26:24.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:24.048+0000 7f533cb9d700 1 Processor -- start 2023-10-06T02:26:24.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:24.048+0000 7f533cb9d700 1 -- start start 2023-10-06T02:26:24.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:24.049+0000 7f533cb9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f533806a210 0x7f5338157fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:24.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:24.049+0000 7f533cb9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f533806de50 0x7f53380c2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:24.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:24.049+0000 7f533cb9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5338140a70 0x7f53380c2790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:24.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:24.049+0000 7f533cb9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53380aeed0 con 0x7f533806a210 2023-10-06T02:26:24.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:24.049+0000 7f533cb9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53380afa50 con 0x7f5338140a70 2023-10-06T02:26:24.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:24.049+0000 7f533cb9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53380af120 con 0x7f533806de50 2023-10-06T02:26:24.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:24.050+0000 7f53377fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f533806a210 0x7f5338157fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:24.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:24.050+0000 7f53377fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f533806a210 0x7f5338157fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57278/0 (socket says 172.21.15.31:57278) 2023-10-06T02:26:24.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:24.050+0000 7f53377fe700 1 -- 172.21.15.31:0/2155702024 learned_addr learned my addr 172.21.15.31:0/2155702024 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:26:24.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:24.050+0000 7f5336ffd700 1 --2- 172.21.15.31:0/2155702024 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f533806de50 0x7f53380c2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:24.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:24.050+0000 7f5337fff700 1 --2- 172.21.15.31:0/2155702024 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5338140a70 0x7f53380c2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:24.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:24.050+0000 7f53377fe700 1 -- 172.21.15.31:0/2155702024 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f533806de50 msgr2=0x7f53380c2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:24.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:24.050+0000 7f53377fe700 1 --2- 172.21.15.31:0/2155702024 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f533806de50 0x7f53380c2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:24.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:24.050+0000 7f53377fe700 1 -- 172.21.15.31:0/2155702024 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5338140a70 msgr2=0x7f53380c2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:24.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:24.050+0000 7f53377fe700 1 --2- 172.21.15.31:0/2155702024 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5338140a70 0x7f53380c2790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:24.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:24.050+0000 7f53377fe700 1 -- 172.21.15.31:0/2155702024 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5330009a20 con 0x7f533806a210 2023-10-06T02:26:24.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:24.050+0000 7f5337fff700 1 --2- 172.21.15.31:0/2155702024 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5338140a70 0x7f53380c2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:26:24.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:24.050+0000 7f53377fe700 1 --2- 172.21.15.31:0/2155702024 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f533806a210 0x7f5338157fb0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f532800c5b0 tx=0x7f5328007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:24.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:24.050+0000 7f531f7fe700 1 -- 172.21.15.31:0/2155702024 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5328010510 con 0x7f533806a210 2023-10-06T02:26:24.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:24.051+0000 7f531f7fe700 1 -- 172.21.15.31:0/2155702024 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5328010670 con 0x7f533806a210 2023-10-06T02:26:24.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:24.051+0000 7f533cb9d700 1 -- 172.21.15.31:0/2155702024 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53380af400 con 0x7f533806a210 2023-10-06T02:26:24.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:24.051+0000 7f531f7fe700 1 -- 172.21.15.31:0/2155702024 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5328010a30 con 0x7f533806a210 2023-10-06T02:26:24.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:24.051+0000 7f533cb9d700 1 -- 172.21.15.31:0/2155702024 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f533813cbb0 con 0x7f533806a210 2023-10-06T02:26:24.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:24.052+0000 7f533cb9d700 1 -- 172.21.15.31:0/2155702024 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f533804c3c0 con 0x7f533806a210 2023-10-06T02:26:24.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:24.053+0000 7f531f7fe700 1 -- 172.21.15.31:0/2155702024 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 9) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f532801b400 con 0x7f533806a210 2023-10-06T02:26:24.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:24.053+0000 7f531f7fe700 1 --2- 172.21.15.31:0/2155702024 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f53240670e0 0x7f5324069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:24.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:24.054+0000 7f531f7fe700 1 -- 172.21.15.31:0/2155702024 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6202+0+0 (secure 0 0 0) 0x7f5328084690 con 0x7f533806a210 2023-10-06T02:26:24.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:24.054+0000 7f5336ffd700 1 --2- 172.21.15.31:0/2155702024 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f53240670e0 0x7f5324069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:24.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:24.054+0000 7f5336ffd700 1 --2- 172.21.15.31:0/2155702024 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f53240670e0 0x7f5324069590 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f532c001ed0 tx=0x7f532c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:24.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:24.058+0000 7f531f7fe700 1 -- 172.21.15.31:0/2155702024 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5328051490 con 0x7f533806a210 2023-10-06T02:26:24.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:24.305+0000 7f533cb9d700 1 -- 172.21.15.31:0/2155702024 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f533815bee0 con 0x7f533806a210 2023-10-06T02:26:24.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:24.836+0000 7f531f7fe700 1 -- 172.21.15.31:0/2155702024 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v29) v1 ==== 104+0+0 (secure 0 0 0) 0x7f5328054ab0 con 0x7f533806a210 2023-10-06T02:26:24.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:24.839+0000 7f533cb9d700 1 -- 172.21.15.31:0/2155702024 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f53240670e0 msgr2=0x7f5324069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:24.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:24.839+0000 7f533cb9d700 1 --2- 172.21.15.31:0/2155702024 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f53240670e0 0x7f5324069590 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f532c001ed0 tx=0x7f532c009040 comp rx=0 tx=0).stop 2023-10-06T02:26:24.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:24.839+0000 7f533cb9d700 1 -- 172.21.15.31:0/2155702024 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f533806a210 msgr2=0x7f5338157fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:24.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:24.839+0000 7f533cb9d700 1 --2- 172.21.15.31:0/2155702024 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f533806a210 0x7f5338157fb0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f532800c5b0 tx=0x7f5328007e10 comp rx=0 tx=0).stop 2023-10-06T02:26:24.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:24.839+0000 7f533cb9d700 1 -- 172.21.15.31:0/2155702024 shutdown_connections 2023-10-06T02:26:24.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:24.839+0000 7f533cb9d700 1 --2- 172.21.15.31:0/2155702024 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5338140a70 0x7f53380c2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:24.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:24.839+0000 7f533cb9d700 1 --2- 172.21.15.31:0/2155702024 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f533806a210 0x7f5338157fb0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:24.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:24.839+0000 7f533cb9d700 1 --2- 172.21.15.31:0/2155702024 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f53240670e0 0x7f5324069590 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:24.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:24.839+0000 7f533cb9d700 1 --2- 172.21.15.31:0/2155702024 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f533806de50 0x7f53380c2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:24.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:24.839+0000 7f533cb9d700 1 -- 172.21.15.31:0/2155702024 >> 172.21.15.31:0/2155702024 conn(0x7f533806d110 msgr2=0x7f533813d990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:24.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:24.839+0000 7f533cb9d700 1 -- 172.21.15.31:0/2155702024 shutdown_connections 2023-10-06T02:26:24.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:24.839+0000 7f533cb9d700 1 -- 172.21.15.31:0/2155702024 wait complete. 2023-10-06T02:26:24.861 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T02:26:25.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:25.056+0000 7f90a299d700 1 Processor -- start 2023-10-06T02:26:25.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:25.056+0000 7f90a299d700 1 -- start start 2023-10-06T02:26:25.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:25.057+0000 7f90a299d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f909c0c8200 0x7f909c0c1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:25.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:25.057+0000 7f90a299d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f909c0c2a90 0x7f909c156aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:25.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:25.057+0000 7f90a299d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f909c1579b0 con 0x7f909c0c85d0 2023-10-06T02:26:25.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:25.057+0000 7f90a299d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f909c0c1390 con 0x7f909c0c2a90 2023-10-06T02:26:25.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:25.057+0000 7f90a299d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f909c1573f0 con 0x7f909c0c8200 2023-10-06T02:26:25.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:25.058+0000 7f90a119a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f909c0c2a90 0x7f909c156aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:25.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:25.058+0000 7f90a119a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f909c0c2a90 0x7f909c156aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57282/0 (socket says 172.21.15.31:57282) 2023-10-06T02:26:25.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:25.058+0000 7f90a119a700 1 -- 172.21.15.31:0/499374033 learned_addr learned my addr 172.21.15.31:0/499374033 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:26:25.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:25.058+0000 7f90a199b700 1 --2- 172.21.15.31:0/499374033 >> v2:172.21.15.62:3300/0 conn(0x7f909c0c8200 0x7f909c0c1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:25.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:25.058+0000 7f90a119a700 1 -- 172.21.15.31:0/499374033 >> v2:172.21.15.62:3300/0 conn(0x7f909c0c8200 msgr2=0x7f909c0c1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:26:25.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:25.058+0000 7f90a119a700 1 --2- 172.21.15.31:0/499374033 >> v2:172.21.15.62:3300/0 conn(0x7f909c0c8200 0x7f909c0c1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:25.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:25.058+0000 7f90a119a700 1 -- 172.21.15.31:0/499374033 >> v1:172.21.15.62:6789/0 conn(0x7f909c0c85d0 legacy=0x7f909c0cc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:26:25.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:25.058+0000 7f90a119a700 1 -- 172.21.15.31:0/499374033 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f909c1576d0 con 0x7f909c0c2a90 2023-10-06T02:26:25.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:25.058+0000 7f90a199b700 1 --2- 172.21.15.31:0/499374033 >> v2:172.21.15.62:3300/0 conn(0x7f909c0c8200 0x7f909c0c1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:26:25.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:25.058+0000 7f90a119a700 1 --2- 172.21.15.31:0/499374033 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f909c0c2a90 0x7f909c156aa0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f909c0683d0 tx=0x7f908c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=272592ca8afab97d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:25.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:25.059+0000 7f90a0999700 1 -- 172.21.15.31:0/499374033 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f908c00b820 con 0x7f909c0c2a90 2023-10-06T02:26:25.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:25.059+0000 7f90a0999700 1 -- 172.21.15.31:0/499374033 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f908c00b980 con 0x7f909c0c2a90 2023-10-06T02:26:25.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:25.059+0000 7f90a0999700 1 -- 172.21.15.31:0/499374033 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f908c00bd20 con 0x7f909c0c2a90 2023-10-06T02:26:25.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:25.059+0000 7f90a299d700 1 -- 172.21.15.31:0/499374033 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f909c0c2a90 msgr2=0x7f909c156aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:25.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:25.059+0000 7f90a299d700 1 --2- 172.21.15.31:0/499374033 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f909c0c2a90 0x7f909c156aa0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f909c0683d0 tx=0x7f908c009250 comp rx=0 tx=0).stop 2023-10-06T02:26:25.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:25.059+0000 7f90a299d700 1 -- 172.21.15.31:0/499374033 shutdown_connections 2023-10-06T02:26:25.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:25.059+0000 7f90a299d700 1 --2- 172.21.15.31:0/499374033 >> v2:172.21.15.62:3300/0 conn(0x7f909c0c8200 0x7f909c0c1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:25.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:25.059+0000 7f90a299d700 1 --2- 172.21.15.31:0/499374033 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f909c0c2a90 0x7f909c156aa0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:25.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:25.059+0000 7f90a299d700 1 -- 172.21.15.31:0/499374033 >> 172.21.15.31:0/499374033 conn(0x7f909c06c6a0 msgr2=0x7f909c06caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:25.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:25.059+0000 7f90a299d700 1 -- 172.21.15.31:0/499374033 shutdown_connections 2023-10-06T02:26:25.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:25.059+0000 7f90a299d700 1 -- 172.21.15.31:0/499374033 wait complete. 2023-10-06T02:26:25.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:25.061+0000 7f90a299d700 1 Processor -- start 2023-10-06T02:26:25.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:25.061+0000 7f90a299d700 1 -- start start 2023-10-06T02:26:25.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:25.062+0000 7f90a299d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f909c0c2a90 0x7f909c143b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:25.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:25.062+0000 7f90a299d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f909c0c8200 0x7f909c0ae1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:25.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:25.062+0000 7f90a299d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f909c0c85d0 0x7f909c169b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:25.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:25.062+0000 7f90a299d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f909c158ce0 con 0x7f909c0c2a90 2023-10-06T02:26:25.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:25.062+0000 7f90a299d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f909c1580f0 con 0x7f909c0c8200 2023-10-06T02:26:25.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:25.062+0000 7f90a299d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f909c1583b0 con 0x7f909c0c85d0 2023-10-06T02:26:25.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:25.062+0000 7f90a199b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f909c0c2a90 0x7f909c143b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:25.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:25.062+0000 7f90a199b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f909c0c2a90 0x7f909c143b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57288/0 (socket says 172.21.15.31:57288) 2023-10-06T02:26:25.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:25.062+0000 7f90a199b700 1 -- 172.21.15.31:0/596847917 learned_addr learned my addr 172.21.15.31:0/596847917 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:26:25.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:25.063+0000 7f90a219c700 1 --2- 172.21.15.31:0/596847917 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f909c0c85d0 0x7f909c169b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:25.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:25.063+0000 7f90a119a700 1 --2- 172.21.15.31:0/596847917 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f909c0c8200 0x7f909c0ae1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:25.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:25.063+0000 7f90a199b700 1 -- 172.21.15.31:0/596847917 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f909c0c85d0 msgr2=0x7f909c169b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:25.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:25.063+0000 7f90a199b700 1 --2- 172.21.15.31:0/596847917 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f909c0c85d0 0x7f909c169b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:25.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:25.063+0000 7f90a199b700 1 -- 172.21.15.31:0/596847917 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f909c0c8200 msgr2=0x7f909c0ae1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:25.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:25.063+0000 7f90a199b700 1 --2- 172.21.15.31:0/596847917 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f909c0c8200 0x7f909c0ae1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:25.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:25.063+0000 7f90a199b700 1 -- 172.21.15.31:0/596847917 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f908c0094f0 con 0x7f909c0c2a90 2023-10-06T02:26:25.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:25.063+0000 7f90a119a700 1 --2- 172.21.15.31:0/596847917 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f909c0c8200 0x7f909c0ae1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:26:25.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:25.063+0000 7f90a199b700 1 --2- 172.21.15.31:0/596847917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f909c0c2a90 0x7f909c143b80 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f909800c830 tx=0x7f9098008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:25.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:25.063+0000 7f9091ffb700 1 -- 172.21.15.31:0/596847917 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9098011600 con 0x7f909c0c2a90 2023-10-06T02:26:25.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:25.063+0000 7f9091ffb700 1 -- 172.21.15.31:0/596847917 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9098011760 con 0x7f909c0c2a90 2023-10-06T02:26:25.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:25.063+0000 7f9091ffb700 1 -- 172.21.15.31:0/596847917 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9098011b20 con 0x7f909c0c2a90 2023-10-06T02:26:25.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:25.063+0000 7f90a299d700 1 -- 172.21.15.31:0/596847917 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f909c158690 con 0x7f909c0c2a90 2023-10-06T02:26:25.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:25.063+0000 7f90a299d700 1 -- 172.21.15.31:0/596847917 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f909c1440c0 con 0x7f909c0c2a90 2023-10-06T02:26:25.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:25.064+0000 7f90a299d700 1 -- 172.21.15.31:0/596847917 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f909c0ccf70 con 0x7f909c0c2a90 2023-10-06T02:26:25.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:25.065+0000 7f9091ffb700 1 -- 172.21.15.31:0/596847917 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 9) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f909800a3e0 con 0x7f909c0c2a90 2023-10-06T02:26:25.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:25.067+0000 7f9091ffb700 1 --2- 172.21.15.31:0/596847917 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f90800670e0 0x7f9080069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:25.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:25.067+0000 7f9091ffb700 1 -- 172.21.15.31:0/596847917 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6202+0+0 (secure 0 0 0) 0x7f90980850a0 con 0x7f909c0c2a90 2023-10-06T02:26:25.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:25.067+0000 7f90a119a700 1 --2- 172.21.15.31:0/596847917 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f90800670e0 0x7f9080069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:25.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:25.067+0000 7f90a119a700 1 --2- 172.21.15.31:0/596847917 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f90800670e0 0x7f9080069590 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f909c0683d0 tx=0x7f908c016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:25.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:25.071+0000 7f9091ffb700 1 -- 172.21.15.31:0/596847917 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9098051cf0 con 0x7f909c0c2a90 2023-10-06T02:26:25.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:25.275+0000 7f90a299d700 1 -- 172.21.15.31:0/596847917 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f909c04c3c0 con 0x7f909c0c2a90 2023-10-06T02:26:26.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.244+0000 7f9091ffb700 1 -- 172.21.15.31:0/596847917 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v51) v1 ==== 181+0+0 (secure 0 0 0) 0x7f9098055310 con 0x7f909c0c2a90 2023-10-06T02:26:26.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.249+0000 7f90a299d700 1 -- 172.21.15.31:0/596847917 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f90800670e0 msgr2=0x7f9080069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:26.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.249+0000 7f90a299d700 1 --2- 172.21.15.31:0/596847917 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f90800670e0 0x7f9080069590 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f909c0683d0 tx=0x7f908c016000 comp rx=0 tx=0).stop 2023-10-06T02:26:26.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.249+0000 7f90a299d700 1 -- 172.21.15.31:0/596847917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f909c0c2a90 msgr2=0x7f909c143b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:26.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.249+0000 7f90a299d700 1 --2- 172.21.15.31:0/596847917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f909c0c2a90 0x7f909c143b80 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f909800c830 tx=0x7f9098008040 comp rx=0 tx=0).stop 2023-10-06T02:26:26.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.249+0000 7f90a299d700 1 -- 172.21.15.31:0/596847917 shutdown_connections 2023-10-06T02:26:26.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.249+0000 7f90a299d700 1 --2- 172.21.15.31:0/596847917 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f90800670e0 0x7f9080069590 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:26.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.249+0000 7f90a299d700 1 --2- 172.21.15.31:0/596847917 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f909c0c85d0 0x7f909c169b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:26.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.249+0000 7f90a299d700 1 --2- 172.21.15.31:0/596847917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f909c0c2a90 0x7f909c143b80 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:26.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.249+0000 7f90a299d700 1 --2- 172.21.15.31:0/596847917 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f909c0c8200 0x7f909c0ae1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:26.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.249+0000 7f90a299d700 1 -- 172.21.15.31:0/596847917 >> 172.21.15.31:0/596847917 conn(0x7f909c06c6a0 msgr2=0x7f909c0cc7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:26.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.249+0000 7f90a299d700 1 -- 172.21.15.31:0/596847917 shutdown_connections 2023-10-06T02:26:26.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.249+0000 7f90a299d700 1 -- 172.21.15.31:0/596847917 wait complete. 2023-10-06T02:26:26.252 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' removed 2023-10-06T02:26:26.273 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T02:26:26.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.472+0000 7f13eb19d700 1 Processor -- start 2023-10-06T02:26:26.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.473+0000 7f13eb19d700 1 -- start start 2023-10-06T02:26:26.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.474+0000 7f13eb19d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f13e415cb90 0x7f13e40c7870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:26.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.474+0000 7f13eb19d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f13e415dc10 0x7f13e406c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:26.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.474+0000 7f13eb19d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13e406d040 con 0x7f13e406cbe0 2023-10-06T02:26:26.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.474+0000 7f13eb19d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13e40ccdf0 con 0x7f13e415cb90 2023-10-06T02:26:26.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.474+0000 7f13eb19d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13e40c9b20 con 0x7f13e415dc10 2023-10-06T02:26:26.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.474+0000 7f13ea99c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f13e406cbe0 0x7f13e40c3430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60680/0 (socket says 172.21.15.31:60680) 2023-10-06T02:26:26.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.474+0000 7f13ea99c700 1 -- 172.21.15.31:0/522585701 learned_addr learned my addr 172.21.15.31:0/522585701 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:26:26.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.475+0000 7f13e999a700 1 --2- 172.21.15.31:0/522585701 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f13e415dc10 0x7f13e406c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:26.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.475+0000 7f13ea19b700 1 --2- 172.21.15.31:0/522585701 >> v2:172.21.15.62:3300/0 conn(0x7f13e415cb90 0x7f13e40c7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:26.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.475+0000 7f13e9199700 1 -- 172.21.15.31:0/522585701 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 395809794 0 0) 0x7f13dc001490 con 0x7f13e406cbe0 2023-10-06T02:26:26.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.475+0000 7f13e9199700 1 -- 172.21.15.31:0/522585701 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13d0003570 con 0x7f13e406cbe0 2023-10-06T02:26:26.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.475+0000 7f13e9199700 1 -- 172.21.15.31:0/522585701 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1713014503 0 0) 0x7f13d0003570 con 0x7f13e406cbe0 2023-10-06T02:26:26.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.475+0000 7f13e9199700 1 -- 172.21.15.31:0/522585701 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f13e415dc10 msgr2=0x7f13e406c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:26:26.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.475+0000 7f13e9199700 1 --2- 172.21.15.31:0/522585701 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f13e415dc10 0x7f13e406c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:26.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.475+0000 7f13e9199700 1 -- 172.21.15.31:0/522585701 >> v2:172.21.15.62:3300/0 conn(0x7f13e415cb90 msgr2=0x7f13e40c7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:26:26.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.475+0000 7f13e9199700 1 --2- 172.21.15.31:0/522585701 >> v2:172.21.15.62:3300/0 conn(0x7f13e415cb90 0x7f13e40c7870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:26.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.475+0000 7f13e9199700 1 -- 172.21.15.31:0/522585701 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13e40c2fa0 con 0x7f13e406cbe0 2023-10-06T02:26:26.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.475+0000 7f13e9199700 1 -- 172.21.15.31:0/522585701 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7f13dc001980 con 0x7f13e406cbe0 2023-10-06T02:26:26.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.475+0000 7f13e9199700 1 -- 172.21.15.31:0/522585701 >> v1:172.21.15.31:6789/0 conn(0x7f13e406cbe0 legacy=0x7f13e40c3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:26.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.475+0000 7f13e9199700 1 --2- 172.21.15.31:0/522585701 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13d0003a60 0x7f13d0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:26.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.475+0000 7f13e9199700 1 --2- 172.21.15.31:0/522585701 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f13e415dc10 0x7f13d0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:26.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.475+0000 7f13e9199700 1 --2- 172.21.15.31:0/522585701 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13d0004d50 0x7f13d00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:26.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.475+0000 7f13e9199700 1 -- 172.21.15.31:0/522585701 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e40ccdf0 con 0x7f13d0004d50 2023-10-06T02:26:26.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.475+0000 7f13e9199700 1 -- 172.21.15.31:0/522585701 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e40c9b20 con 0x7f13d0003a60 2023-10-06T02:26:26.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.475+0000 7f13e9199700 1 -- 172.21.15.31:0/522585701 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13d000ba60 con 0x7f13e415dc10 2023-10-06T02:26:26.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.476+0000 7f13ea99c700 1 --2- 172.21.15.31:0/522585701 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13d0004d50 0x7f13d00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:26.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.476+0000 7f13e999a700 1 --2- 172.21.15.31:0/522585701 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f13e415dc10 0x7f13d0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:26.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.476+0000 7f13ea19b700 1 --2- 172.21.15.31:0/522585701 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13d0003a60 0x7f13d0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:26.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.476+0000 7f13ea99c700 1 -- 172.21.15.31:0/522585701 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f13e415dc10 msgr2=0x7f13d0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:26:26.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.476+0000 7f13ea99c700 1 --2- 172.21.15.31:0/522585701 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f13e415dc10 0x7f13d0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:26.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.476+0000 7f13ea99c700 1 -- 172.21.15.31:0/522585701 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13d0003a60 msgr2=0x7f13d0003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:26:26.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.476+0000 7f13ea99c700 1 --2- 172.21.15.31:0/522585701 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13d0003a60 0x7f13d0003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:26.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.476+0000 7f13ea99c700 1 -- 172.21.15.31:0/522585701 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d0003570 con 0x7f13d0004d50 2023-10-06T02:26:26.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.476+0000 7f13ea99c700 1 --2- 172.21.15.31:0/522585701 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13d0004d50 0x7f13d00051c0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f13dc00aaf0 tx=0x7f13e40c2f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d885d32691bf2b37 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:26.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.477+0000 7f13e9199700 1 -- 172.21.15.31:0/522585701 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13dc00a480 con 0x7f13d0004d50 2023-10-06T02:26:26.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.477+0000 7f13e9199700 1 -- 172.21.15.31:0/522585701 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13dc00c570 con 0x7f13d0004d50 2023-10-06T02:26:26.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.477+0000 7f13e9199700 1 -- 172.21.15.31:0/522585701 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13dc00c930 con 0x7f13d0004d50 2023-10-06T02:26:26.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.477+0000 7f13eb19d700 1 -- 172.21.15.31:0/522585701 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13d0004d50 msgr2=0x7f13d00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:26.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.477+0000 7f13eb19d700 1 --2- 172.21.15.31:0/522585701 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13d0004d50 0x7f13d00051c0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f13dc00aaf0 tx=0x7f13e40c2f10 comp rx=0 tx=0).stop 2023-10-06T02:26:26.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.477+0000 7f13ea99c700 1 -- 172.21.15.31:0/522585701 reap_dead start 2023-10-06T02:26:26.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.477+0000 7f13eb19d700 1 -- 172.21.15.31:0/522585701 shutdown_connections 2023-10-06T02:26:26.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.477+0000 7f13eb19d700 1 -- 172.21.15.31:0/522585701 >> 172.21.15.31:0/522585701 conn(0x7f13e40cd230 msgr2=0x7f13e40cd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:26.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.477+0000 7f13eb19d700 1 -- 172.21.15.31:0/522585701 shutdown_connections 2023-10-06T02:26:26.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.477+0000 7f13eb19d700 1 -- 172.21.15.31:0/522585701 wait complete. 2023-10-06T02:26:26.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.479+0000 7f13eb19d700 1 Processor -- start 2023-10-06T02:26:26.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.479+0000 7f13eb19d700 1 -- start start 2023-10-06T02:26:26.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.480+0000 7f13eb19d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f13d0004d50 0x7f13e40c7770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:26.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.480+0000 7f13eb19d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13e413f8a0 0x7f13e4169cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:26.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.480+0000 7f13eb19d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13e416a200 0x7f13e40685e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:26.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.480+0000 7f13eb19d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e416ad10 con 0x7f13e416a200 2023-10-06T02:26:26.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.480+0000 7f13eb19d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e416a750 con 0x7f13e413f8a0 2023-10-06T02:26:26.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.480+0000 7f13eb19d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13e416aa30 con 0x7f13d0004d50 2023-10-06T02:26:26.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.480+0000 7f13ea99c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13e416a200 0x7f13e40685e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:26.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.480+0000 7f13ea99c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13e416a200 0x7f13e40685e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57306/0 (socket says 172.21.15.31:57306) 2023-10-06T02:26:26.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.480+0000 7f13ea99c700 1 -- 172.21.15.31:0/334477265 learned_addr learned my addr 172.21.15.31:0/334477265 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:26:26.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.480+0000 7f13ea19b700 1 --2- 172.21.15.31:0/334477265 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f13d0004d50 0x7f13e40c7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:26.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.480+0000 7f13e999a700 1 --2- 172.21.15.31:0/334477265 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13e413f8a0 0x7f13e4169cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:26.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.480+0000 7f13ea99c700 1 -- 172.21.15.31:0/334477265 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f13d0004d50 msgr2=0x7f13e40c7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:26.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.480+0000 7f13ea99c700 1 --2- 172.21.15.31:0/334477265 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f13d0004d50 0x7f13e40c7770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:26.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.480+0000 7f13ea99c700 1 -- 172.21.15.31:0/334477265 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13e413f8a0 msgr2=0x7f13e4169cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:26.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.480+0000 7f13ea99c700 1 --2- 172.21.15.31:0/334477265 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13e413f8a0 0x7f13e4169cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:26.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.480+0000 7f13ea99c700 1 -- 172.21.15.31:0/334477265 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13e416b020 con 0x7f13e416a200 2023-10-06T02:26:26.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.481+0000 7f13ea99c700 1 --2- 172.21.15.31:0/334477265 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13e416a200 0x7f13e40685e0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f13dc012000 tx=0x7f13dc016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:26.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.481+0000 7f13da7fc700 1 -- 172.21.15.31:0/334477265 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13dc001940 con 0x7f13e416a200 2023-10-06T02:26:26.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.481+0000 7f13da7fc700 1 -- 172.21.15.31:0/334477265 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13dc001490 con 0x7f13e416a200 2023-10-06T02:26:26.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.481+0000 7f13eb19d700 1 -- 172.21.15.31:0/334477265 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13e416b300 con 0x7f13e416a200 2023-10-06T02:26:26.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.481+0000 7f13da7fc700 1 -- 172.21.15.31:0/334477265 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13dc00cca0 con 0x7f13e416a200 2023-10-06T02:26:26.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.481+0000 7f13eb19d700 1 -- 172.21.15.31:0/334477265 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13e4068ce0 con 0x7f13e416a200 2023-10-06T02:26:26.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.482+0000 7f13eb19d700 1 -- 172.21.15.31:0/334477265 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13e4068f90 con 0x7f13e416a200 2023-10-06T02:26:26.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.483+0000 7f13da7fc700 1 -- 172.21.15.31:0/334477265 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 9) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f13dc024070 con 0x7f13e416a200 2023-10-06T02:26:26.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.484+0000 7f13da7fc700 1 --2- 172.21.15.31:0/334477265 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f13c80670e0 0x7f13c8069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:26.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.484+0000 7f13da7fc700 1 -- 172.21.15.31:0/334477265 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5764+0+0 (secure 0 0 0) 0x7f13dc086a40 con 0x7f13e416a200 2023-10-06T02:26:26.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.484+0000 7f13ea19b700 1 --2- 172.21.15.31:0/334477265 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f13c80670e0 0x7f13c8069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:26.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.485+0000 7f13ea19b700 1 --2- 172.21.15.31:0/334477265 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f13c80670e0 0x7f13c8069590 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f13e4167910 tx=0x7f13e0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:26.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.488+0000 7f13da7fc700 1 -- 172.21.15.31:0/334477265 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f13dc053950 con 0x7f13e416a200 2023-10-06T02:26:26.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:26.691+0000 7f13eb19d700 1 -- 172.21.15.31:0/334477265 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f13e406d040 con 0x7f13e416a200 2023-10-06T02:26:27.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.262+0000 7f13da7fc700 1 -- 172.21.15.31:0/334477265 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v52) v1 ==== 169+0+0 (secure 0 0 0) 0x7f13e406d040 con 0x7f13e416a200 2023-10-06T02:26:27.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.265+0000 7f13eb19d700 1 -- 172.21.15.31:0/334477265 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f13c80670e0 msgr2=0x7f13c8069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:27.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.265+0000 7f13eb19d700 1 --2- 172.21.15.31:0/334477265 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f13c80670e0 0x7f13c8069590 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f13e4167910 tx=0x7f13e0008040 comp rx=0 tx=0).stop 2023-10-06T02:26:27.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.265+0000 7f13eb19d700 1 -- 172.21.15.31:0/334477265 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13e416a200 msgr2=0x7f13e40685e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:27.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.265+0000 7f13eb19d700 1 --2- 172.21.15.31:0/334477265 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13e416a200 0x7f13e40685e0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f13dc012000 tx=0x7f13dc016000 comp rx=0 tx=0).stop 2023-10-06T02:26:27.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.265+0000 7f13eb19d700 1 -- 172.21.15.31:0/334477265 shutdown_connections 2023-10-06T02:26:27.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.265+0000 7f13eb19d700 1 --2- 172.21.15.31:0/334477265 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13e416a200 0x7f13e40685e0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:27.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.265+0000 7f13eb19d700 1 --2- 172.21.15.31:0/334477265 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f13c80670e0 0x7f13c8069590 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:27.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.265+0000 7f13eb19d700 1 --2- 172.21.15.31:0/334477265 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f13d0004d50 0x7f13e40c7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:27.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.265+0000 7f13eb19d700 1 --2- 172.21.15.31:0/334477265 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13e413f8a0 0x7f13e4169cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:27.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.265+0000 7f13eb19d700 1 -- 172.21.15.31:0/334477265 >> 172.21.15.31:0/334477265 conn(0x7f13e40cd230 msgr2=0x7f13e415c210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:27.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.265+0000 7f13eb19d700 1 -- 172.21.15.31:0/334477265 shutdown_connections 2023-10-06T02:26:27.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.265+0000 7f13eb19d700 1 -- 172.21.15.31:0/334477265 wait complete. 2023-10-06T02:26:27.269 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' removed 2023-10-06T02:26:27.287 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T02:26:27.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.476+0000 7fafb5f9d700 1 Processor -- start 2023-10-06T02:26:27.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.476+0000 7fafb5f9d700 1 -- start start 2023-10-06T02:26:27.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.477+0000 7fafb5f9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fafb0159550 0x7fafb00ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:27.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.477+0000 7fafb5f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fafb015dfd0 0x7fafb00aedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:27.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.477+0000 7fafb5f9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafb00acc00 con 0x7fafb0159920 2023-10-06T02:26:27.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.477+0000 7fafb5f9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafb00abff0 con 0x7fafb0159550 2023-10-06T02:26:27.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.477+0000 7fafb5f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafb00ac2d0 con 0x7fafb015dfd0 2023-10-06T02:26:27.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.477+0000 7fafb579c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fafb0159920 0x7fafb00ad090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60686/0 (socket says 172.21.15.31:60686) 2023-10-06T02:26:27.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.477+0000 7fafb579c700 1 -- 172.21.15.31:0/2168424513 learned_addr learned my addr 172.21.15.31:0/2168424513 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:26:27.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.478+0000 7fafaffff700 1 --2- 172.21.15.31:0/2168424513 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fafb015dfd0 0x7fafb00aedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:27.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.478+0000 7fafb4f9b700 1 --2- 172.21.15.31:0/2168424513 >> v2:172.21.15.62:3300/0 conn(0x7fafb0159550 0x7fafb00ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:27.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.478+0000 7fafaf7fe700 1 -- 172.21.15.31:0/2168424513 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 797968064 0 0) 0x7fafb00acc00 con 0x7fafb0159920 2023-10-06T02:26:27.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.478+0000 7fafaf7fe700 1 -- 172.21.15.31:0/2168424513 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fafa0003570 con 0x7fafb0159920 2023-10-06T02:26:27.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.478+0000 7fafaffff700 1 -- 172.21.15.31:0/2168424513 >> v2:172.21.15.62:3300/0 conn(0x7fafb0159550 msgr2=0x7fafb00ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:26:27.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.478+0000 7fafaffff700 1 --2- 172.21.15.31:0/2168424513 >> v2:172.21.15.62:3300/0 conn(0x7fafb0159550 0x7fafb00ceac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:27.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.478+0000 7fafaffff700 1 -- 172.21.15.31:0/2168424513 >> v1:172.21.15.31:6789/0 conn(0x7fafb0159920 legacy=0x7fafb00ad090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:27.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.478+0000 7fafaffff700 1 -- 172.21.15.31:0/2168424513 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafb00ac5b0 con 0x7fafb015dfd0 2023-10-06T02:26:27.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.479+0000 7fafaffff700 1 --2- 172.21.15.31:0/2168424513 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fafb015dfd0 0x7fafb00aedb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7faf9c0098e0 tx=0x7faf9c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b368e36e88bf086 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:27.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.479+0000 7fafaf7fe700 1 -- 172.21.15.31:0/2168424513 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf9c011630 con 0x7fafb015dfd0 2023-10-06T02:26:27.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.479+0000 7fafaf7fe700 1 -- 172.21.15.31:0/2168424513 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf9c011790 con 0x7fafb015dfd0 2023-10-06T02:26:27.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.479+0000 7fafaf7fe700 1 -- 172.21.15.31:0/2168424513 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf9c011b10 con 0x7fafb015dfd0 2023-10-06T02:26:27.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.479+0000 7fafb5f9d700 1 -- 172.21.15.31:0/2168424513 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fafb015dfd0 msgr2=0x7fafb00aedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:27.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.479+0000 7fafb5f9d700 1 --2- 172.21.15.31:0/2168424513 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fafb015dfd0 0x7fafb00aedb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7faf9c0098e0 tx=0x7faf9c0091a0 comp rx=0 tx=0).stop 2023-10-06T02:26:27.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.479+0000 7fafb5f9d700 1 -- 172.21.15.31:0/2168424513 shutdown_connections 2023-10-06T02:26:27.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.479+0000 7fafb5f9d700 1 --2- 172.21.15.31:0/2168424513 >> v2:172.21.15.62:3300/0 conn(0x7fafb0159550 0x7fafb00ceac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:27.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.479+0000 7fafb5f9d700 1 --2- 172.21.15.31:0/2168424513 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fafb015dfd0 0x7fafb00aedb0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:27.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.479+0000 7fafb5f9d700 1 -- 172.21.15.31:0/2168424513 >> 172.21.15.31:0/2168424513 conn(0x7fafb006ce80 msgr2=0x7fafb00ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:27.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.479+0000 7fafb5f9d700 1 -- 172.21.15.31:0/2168424513 shutdown_connections 2023-10-06T02:26:27.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.479+0000 7fafb5f9d700 1 -- 172.21.15.31:0/2168424513 wait complete. 2023-10-06T02:26:27.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.481+0000 7fafb5f9d700 1 Processor -- start 2023-10-06T02:26:27.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.481+0000 7fafb5f9d700 1 -- start start 2023-10-06T02:26:27.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.482+0000 7fafb5f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fafb0159550 0x7fafb00c7f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:27.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.482+0000 7fafb5f9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fafb0159920 0x7fafb014b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:27.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.482+0000 7fafb5f9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fafb015dfd0 0x7fafb016d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:27.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.482+0000 7fafb5f9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb00c7a70 con 0x7fafb0159920 2023-10-06T02:26:27.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.482+0000 7fafb5f9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb016dea0 con 0x7fafb015dfd0 2023-10-06T02:26:27.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.482+0000 7fafb5f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafb016e180 con 0x7fafb0159550 2023-10-06T02:26:27.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.482+0000 7fafaffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fafb0159920 0x7fafb014b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:27.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.482+0000 7fafaffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fafb0159920 0x7fafb014b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57318/0 (socket says 172.21.15.31:57318) 2023-10-06T02:26:27.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.482+0000 7fafaffff700 1 -- 172.21.15.31:0/2673444903 learned_addr learned my addr 172.21.15.31:0/2673444903 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:26:27.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.482+0000 7fafb4f9b700 1 --2- 172.21.15.31:0/2673444903 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fafb0159550 0x7fafb00c7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:27.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.482+0000 7fafb579c700 1 --2- 172.21.15.31:0/2673444903 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fafb015dfd0 0x7fafb016d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:27.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.482+0000 7fafaffff700 1 -- 172.21.15.31:0/2673444903 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fafb0159550 msgr2=0x7fafb00c7f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:27.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.482+0000 7fafaffff700 1 --2- 172.21.15.31:0/2673444903 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fafb0159550 0x7fafb00c7f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:27.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.482+0000 7fafaffff700 1 -- 172.21.15.31:0/2673444903 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fafb015dfd0 msgr2=0x7fafb016d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:27.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.482+0000 7fafaffff700 1 --2- 172.21.15.31:0/2673444903 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fafb015dfd0 0x7fafb016d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:27.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.482+0000 7fafaffff700 1 -- 172.21.15.31:0/2673444903 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf9c009420 con 0x7fafb0159920 2023-10-06T02:26:27.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.483+0000 7fafb579c700 1 --2- 172.21.15.31:0/2673444903 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fafb015dfd0 0x7fafb016d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:26:27.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.483+0000 7fafaffff700 1 --2- 172.21.15.31:0/2673444903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fafb0159920 0x7fafb014b250 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fafb00ac410 tx=0x7faf9c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:27.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.483+0000 7fafacff9700 1 -- 172.21.15.31:0/2673444903 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf9c011b10 con 0x7fafb0159920 2023-10-06T02:26:27.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.483+0000 7fafacff9700 1 -- 172.21.15.31:0/2673444903 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf9c01a430 con 0x7fafb0159920 2023-10-06T02:26:27.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.483+0000 7fafacff9700 1 -- 172.21.15.31:0/2673444903 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf9c01a7d0 con 0x7fafb0159920 2023-10-06T02:26:27.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.483+0000 7fafb5f9d700 1 -- 172.21.15.31:0/2673444903 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafb00c7700 con 0x7fafb0159920 2023-10-06T02:26:27.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.483+0000 7fafb5f9d700 1 -- 172.21.15.31:0/2673444903 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafb014b790 con 0x7fafb0159920 2023-10-06T02:26:27.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.484+0000 7fafb5f9d700 1 -- 172.21.15.31:0/2673444903 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafb00c7560 con 0x7fafb0159920 2023-10-06T02:26:27.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.485+0000 7fafacff9700 1 -- 172.21.15.31:0/2673444903 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 9) v1 ==== 84893+0+0 (secure 0 0 0) 0x7faf9c01abc0 con 0x7fafb0159920 2023-10-06T02:26:27.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.487+0000 7fafacff9700 1 --2- 172.21.15.31:0/2673444903 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7faf940670e0 0x7faf94069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:27.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.487+0000 7fafacff9700 1 -- 172.21.15.31:0/2673444903 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5292+0+0 (secure 0 0 0) 0x7faf9c086440 con 0x7fafb0159920 2023-10-06T02:26:27.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.487+0000 7fafb4f9b700 1 --2- 172.21.15.31:0/2673444903 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7faf940670e0 0x7faf94069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:27.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.487+0000 7fafb4f9b700 1 --2- 172.21.15.31:0/2673444903 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7faf940670e0 0x7faf94069590 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fafa4009ca0 tx=0x7fafa4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:27.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.491+0000 7fafacff9700 1 -- 172.21.15.31:0/2673444903 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fafb00c7560 con 0x7fafb0159920 2023-10-06T02:26:27.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.692+0000 7fafb5f9d700 1 -- 172.21.15.31:0/2673444903 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fafb00c7560 con 0x7fafb0159920 2023-10-06T02:26:27.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.696+0000 7fafacff9700 1 -- 172.21.15.31:0/2673444903 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7fafb00c7560 con 0x7fafb0159920 2023-10-06T02:26:27.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.699+0000 7fafb5f9d700 1 -- 172.21.15.31:0/2673444903 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7faf940670e0 msgr2=0x7faf94069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:27.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.699+0000 7fafb5f9d700 1 --2- 172.21.15.31:0/2673444903 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7faf940670e0 0x7faf94069590 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fafa4009ca0 tx=0x7fafa4008040 comp rx=0 tx=0).stop 2023-10-06T02:26:27.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.699+0000 7fafb5f9d700 1 -- 172.21.15.31:0/2673444903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fafb0159920 msgr2=0x7fafb014b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:27.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.699+0000 7fafb5f9d700 1 --2- 172.21.15.31:0/2673444903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fafb0159920 0x7fafb014b250 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fafb00ac410 tx=0x7faf9c017000 comp rx=0 tx=0).stop 2023-10-06T02:26:27.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.699+0000 7fafb5f9d700 1 -- 172.21.15.31:0/2673444903 shutdown_connections 2023-10-06T02:26:27.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.699+0000 7fafb5f9d700 1 --2- 172.21.15.31:0/2673444903 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fafb015dfd0 0x7fafb016d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:27.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.699+0000 7fafb5f9d700 1 --2- 172.21.15.31:0/2673444903 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7faf940670e0 0x7faf94069590 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:27.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.699+0000 7fafb5f9d700 1 --2- 172.21.15.31:0/2673444903 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fafb0159550 0x7fafb00c7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:27.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.699+0000 7fafb5f9d700 1 --2- 172.21.15.31:0/2673444903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fafb0159920 0x7fafb014b250 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:27.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.699+0000 7fafb5f9d700 1 -- 172.21.15.31:0/2673444903 >> 172.21.15.31:0/2673444903 conn(0x7fafb006ce80 msgr2=0x7fafb0152b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:27.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.699+0000 7fafb5f9d700 1 -- 172.21.15.31:0/2673444903 shutdown_connections 2023-10-06T02:26:27.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.699+0000 7fafb5f9d700 1 -- 172.21.15.31:0/2673444903 wait complete. 2023-10-06T02:26:27.717 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration' 2023-10-06T02:26:27.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.902+0000 7f054a99d700 1 Processor -- start 2023-10-06T02:26:27.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.902+0000 7f054a99d700 1 -- start start 2023-10-06T02:26:27.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.904+0000 7f054a99d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f054413fb20 0x7f054415a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:27.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.904+0000 7f054a99d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f054413fef0 0x7f05440c7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:27.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.904+0000 7f054a99d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f054415ae70 con 0x7f054406aaa0 2023-10-06T02:26:27.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.904+0000 7f054a99d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f054406af60 con 0x7f054413fef0 2023-10-06T02:26:27.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.904+0000 7f054a99d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f054406b240 con 0x7f054413fb20 2023-10-06T02:26:27.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.904+0000 7f054a19c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f054406aaa0 0x7f05440c6770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:47610/0 (socket says 172.21.15.31:47610) 2023-10-06T02:26:27.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.904+0000 7f054a19c700 1 -- 172.21.15.31:0/733497919 learned_addr learned my addr 172.21.15.31:0/733497919 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:26:27.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.904+0000 7f0548999700 1 -- 172.21.15.31:0/733497919 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3752049428 0 0) 0x7f053c001490 con 0x7f054406aaa0 2023-10-06T02:26:27.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.904+0000 7f0548999700 1 -- 172.21.15.31:0/733497919 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0530003570 con 0x7f054406aaa0 2023-10-06T02:26:27.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.904+0000 7f054999b700 1 --2- 172.21.15.31:0/733497919 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f054413fb20 0x7f054415a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:27.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.904+0000 7f054919a700 1 --2- 172.21.15.31:0/733497919 >> v2:172.21.15.62:3300/0 conn(0x7f054413fef0 0x7f05440c7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:27.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.905+0000 7f0548999700 1 -- 172.21.15.31:0/733497919 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 261968143 0 0) 0x7f0530003570 con 0x7f054406aaa0 2023-10-06T02:26:27.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.905+0000 7f0548999700 1 -- 172.21.15.31:0/733497919 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f054413fb20 msgr2=0x7f054415a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:26:27.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.905+0000 7f0548999700 1 --2- 172.21.15.31:0/733497919 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f054413fb20 0x7f054415a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:27.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.905+0000 7f0548999700 1 -- 172.21.15.31:0/733497919 >> v2:172.21.15.62:3300/0 conn(0x7f054413fef0 msgr2=0x7f05440c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:26:27.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.905+0000 7f0548999700 1 --2- 172.21.15.31:0/733497919 >> v2:172.21.15.62:3300/0 conn(0x7f054413fef0 0x7f05440c7990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:27.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.905+0000 7f0548999700 1 -- 172.21.15.31:0/733497919 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f054406b520 con 0x7f054406aaa0 2023-10-06T02:26:27.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.905+0000 7f0548999700 1 -- 172.21.15.31:0/733497919 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7f053c001980 con 0x7f054406aaa0 2023-10-06T02:26:27.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.905+0000 7f0548999700 1 -- 172.21.15.31:0/733497919 >> v1:172.21.15.31:6789/0 conn(0x7f054406aaa0 legacy=0x7f05440c6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:27.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.905+0000 7f0548999700 1 --2- 172.21.15.31:0/733497919 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f054413fb20 0x7f0530003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:27.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.905+0000 7f0548999700 1 --2- 172.21.15.31:0/733497919 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0530003fe0 0x7f0530004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:27.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.905+0000 7f0548999700 1 --2- 172.21.15.31:0/733497919 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0530004d50 0x7f05300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:27.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.905+0000 7f0548999700 1 -- 172.21.15.31:0/733497919 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f054406af60 con 0x7f0530004d50 2023-10-06T02:26:27.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.905+0000 7f0548999700 1 -- 172.21.15.31:0/733497919 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f054406b240 con 0x7f0530003fe0 2023-10-06T02:26:27.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.905+0000 7f0548999700 1 -- 172.21.15.31:0/733497919 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f053000ba90 con 0x7f054413fb20 2023-10-06T02:26:27.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.905+0000 7f054a19c700 1 --2- 172.21.15.31:0/733497919 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0530004d50 0x7f05300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:27.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.905+0000 7f054999b700 1 --2- 172.21.15.31:0/733497919 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f054413fb20 0x7f0530003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:27.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.905+0000 7f054919a700 1 --2- 172.21.15.31:0/733497919 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0530003fe0 0x7f0530004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:27.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.906+0000 7f054a19c700 1 -- 172.21.15.31:0/733497919 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f054413fb20 msgr2=0x7f0530003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:26:27.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.906+0000 7f054a19c700 1 --2- 172.21.15.31:0/733497919 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f054413fb20 0x7f0530003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:27.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.906+0000 7f054a19c700 1 -- 172.21.15.31:0/733497919 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0530003fe0 msgr2=0x7f0530004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:26:27.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.906+0000 7f054a19c700 1 --2- 172.21.15.31:0/733497919 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0530003fe0 0x7f0530004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:27.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.906+0000 7f054a19c700 1 -- 172.21.15.31:0/733497919 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0530003570 con 0x7f0530004d50 2023-10-06T02:26:27.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.906+0000 7f054a19c700 1 --2- 172.21.15.31:0/733497919 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0530004d50 0x7f05300051c0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f053c00aad0 tx=0x7f053c00a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9b238b4c08c8386 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:27.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.906+0000 7f0548999700 1 -- 172.21.15.31:0/733497919 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053c00a490 con 0x7f0530004d50 2023-10-06T02:26:27.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.906+0000 7f0548999700 1 -- 172.21.15.31:0/733497919 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f053c00c570 con 0x7f0530004d50 2023-10-06T02:26:27.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.906+0000 7f0548999700 1 -- 172.21.15.31:0/733497919 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053c00c910 con 0x7f0530004d50 2023-10-06T02:26:27.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.907+0000 7f054a99d700 1 -- 172.21.15.31:0/733497919 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0530004d50 msgr2=0x7f05300051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:27.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.907+0000 7f054a99d700 1 --2- 172.21.15.31:0/733497919 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0530004d50 0x7f05300051c0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f053c00aad0 tx=0x7f053c00a1a0 comp rx=0 tx=0).stop 2023-10-06T02:26:27.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.907+0000 7f054a19c700 1 -- 172.21.15.31:0/733497919 reap_dead start 2023-10-06T02:26:27.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.907+0000 7f054a99d700 1 -- 172.21.15.31:0/733497919 shutdown_connections 2023-10-06T02:26:27.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.907+0000 7f054a99d700 1 -- 172.21.15.31:0/733497919 >> 172.21.15.31:0/733497919 conn(0x7f054406d170 msgr2=0x7f05440cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:27.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.907+0000 7f054a99d700 1 -- 172.21.15.31:0/733497919 shutdown_connections 2023-10-06T02:26:27.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.907+0000 7f054a99d700 1 -- 172.21.15.31:0/733497919 wait complete. 2023-10-06T02:26:27.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.908+0000 7f054a99d700 1 Processor -- start 2023-10-06T02:26:27.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.909+0000 7f054a99d700 1 -- start start 2023-10-06T02:26:27.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.909+0000 7f054a99d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0530004d50 0x7f0544130870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:27.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.909+0000 7f054a99d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0544158580 0x7f05441589f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:27.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.909+0000 7f054a99d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f054413d760 0x7f054413dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:27.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.910+0000 7f054a99d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f054413e100 con 0x7f054413d760 2023-10-06T02:26:27.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.910+0000 7f054a99d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05440ab740 con 0x7f0530004d50 2023-10-06T02:26:27.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.910+0000 7f054a99d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05440aba20 con 0x7f0544158580 2023-10-06T02:26:27.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.910+0000 7f054a19c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f054413d760 0x7f054413dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:27.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.910+0000 7f054999b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0530004d50 0x7f0544130870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:27.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.910+0000 7f054919a700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0544158580 0x7f05441589f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:27.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.910+0000 7f054a19c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f054413d760 0x7f054413dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58612/0 (socket says 172.21.15.31:58612) 2023-10-06T02:26:27.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.910+0000 7f054999b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0530004d50 0x7f0544130870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:59256/0 (socket says 172.21.15.31:59256) 2023-10-06T02:26:27.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.910+0000 7f054919a700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0544158580 0x7f05441589f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:57494/0 (socket says 172.21.15.31:57494) 2023-10-06T02:26:27.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.910+0000 7f054a19c700 1 -- 172.21.15.31:0/1416858307 learned_addr learned my addr 172.21.15.31:0/1416858307 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:26:27.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.910+0000 7f054999b700 1 -- 172.21.15.31:0/1416858307 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0544158580 msgr2=0x7f05441589f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:27.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.910+0000 7f054999b700 1 --2- 172.21.15.31:0/1416858307 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0544158580 0x7f05441589f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:27.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.910+0000 7f054999b700 1 -- 172.21.15.31:0/1416858307 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f054413d760 msgr2=0x7f054413dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:27.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.910+0000 7f054999b700 1 --2- 172.21.15.31:0/1416858307 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f054413d760 0x7f054413dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:27.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.910+0000 7f054999b700 1 -- 172.21.15.31:0/1416858307 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05440abd00 con 0x7f0530004d50 2023-10-06T02:26:27.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.910+0000 7f054919a700 1 --2- 172.21.15.31:0/1416858307 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0544158580 0x7f05441589f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:26:27.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.910+0000 7f054a19c700 1 --2- 172.21.15.31:0/1416858307 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f054413d760 0x7f054413dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:26:27.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.911+0000 7f054999b700 1 --2- 172.21.15.31:0/1416858307 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0530004d50 0x7f0544130870 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f05400119f0 tx=0x7f05400091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:27.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.911+0000 7f0539ffb700 1 -- 172.21.15.31:0/1416858307 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f054000f790 con 0x7f0530004d50 2023-10-06T02:26:27.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.911+0000 7f0539ffb700 1 -- 172.21.15.31:0/1416858307 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f054000f8f0 con 0x7f0530004d50 2023-10-06T02:26:27.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.911+0000 7f0539ffb700 1 -- 172.21.15.31:0/1416858307 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f054000fc90 con 0x7f0530004d50 2023-10-06T02:26:27.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.911+0000 7f054a99d700 1 -- 172.21.15.31:0/1416858307 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05440abfe0 con 0x7f0530004d50 2023-10-06T02:26:27.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.911+0000 7f054a99d700 1 -- 172.21.15.31:0/1416858307 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0544130db0 con 0x7f0530004d50 2023-10-06T02:26:27.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.912+0000 7f054a99d700 1 -- 172.21.15.31:0/1416858307 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0544169b80 con 0x7f0530004d50 2023-10-06T02:26:27.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.913+0000 7f0539ffb700 1 -- 172.21.15.31:0/1416858307 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 9) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f0540014650 con 0x7f0530004d50 2023-10-06T02:26:27.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.914+0000 7f0539ffb700 1 --2- 172.21.15.31:0/1416858307 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f05280670e0 0x7f0528069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:27.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.914+0000 7f0539ffb700 1 -- 172.21.15.31:0/1416858307 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f0540018030 con 0x7f0530004d50 2023-10-06T02:26:27.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.914+0000 7f054919a700 1 --2- 172.21.15.31:0/1416858307 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f05280670e0 0x7f0528069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:27.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.915+0000 7f054919a700 1 --2- 172.21.15.31:0/1416858307 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f05280670e0 0x7f0528069590 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f05440ac810 tx=0x7f0534009120 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:27.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:27.919+0000 7f0539ffb700 1 -- 172.21.15.31:0/1416858307 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0540052110 con 0x7f0530004d50 2023-10-06T02:26:28.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:28.211+0000 7f054a99d700 1 -- 172.21.15.31:0/1416858307 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration"]} v 0) v1 -- 0x7f054404c3c0 con 0x7f0530004d50 2023-10-06T02:26:28.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:28.259+0000 7f0539ffb700 1 -- 172.21.15.31:0/1416858307 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration"]}]=0 v190) v1 ==== 140+0+0 (secure 0 0 0) 0x7f05400220d0 con 0x7f0530004d50 2023-10-06T02:26:28.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:28.261+0000 7f054a99d700 1 -- 172.21.15.31:0/1416858307 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f05280670e0 msgr2=0x7f0528069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:28.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:28.261+0000 7f054a99d700 1 --2- 172.21.15.31:0/1416858307 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f05280670e0 0x7f0528069590 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f05440ac810 tx=0x7f0534009120 comp rx=0 tx=0).stop 2023-10-06T02:26:28.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:28.261+0000 7f054a99d700 1 -- 172.21.15.31:0/1416858307 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0530004d50 msgr2=0x7f0544130870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:28.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:28.261+0000 7f054a99d700 1 --2- 172.21.15.31:0/1416858307 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0530004d50 0x7f0544130870 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f05400119f0 tx=0x7f05400091a0 comp rx=0 tx=0).stop 2023-10-06T02:26:28.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:28.262+0000 7f054a99d700 1 -- 172.21.15.31:0/1416858307 shutdown_connections 2023-10-06T02:26:28.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:28.262+0000 7f054a99d700 1 --2- 172.21.15.31:0/1416858307 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f054413d760 0x7f054413dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:28.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:28.262+0000 7f054a99d700 1 --2- 172.21.15.31:0/1416858307 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0530004d50 0x7f0544130870 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:28.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:28.262+0000 7f054a99d700 1 --2- 172.21.15.31:0/1416858307 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f05280670e0 0x7f0528069590 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:28.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:28.262+0000 7f054a99d700 1 --2- 172.21.15.31:0/1416858307 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0544158580 0x7f05441589f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:28.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:28.262+0000 7f054a99d700 1 -- 172.21.15.31:0/1416858307 >> 172.21.15.31:0/1416858307 conn(0x7f054406d170 msgr2=0x7f054413e780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:28.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:28.262+0000 7f054a99d700 1 -- 172.21.15.31:0/1416858307 shutdown_connections 2023-10-06T02:26:28.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:28.262+0000 7f054a99d700 1 -- 172.21.15.31:0/1416858307 wait complete. 2023-10-06T02:26:28.280 INFO:tasks.cephfs_test_runner:test_connection_expiration (tasks.cephfs.test_volumes.TestMisc) ... ok 2023-10-06T02:26:28.281 INFO:tasks.cephfs_test_runner:Starting test: test_malicious_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) Validate handcrafted .meta file on legacy subvol root doesn't break the system 2023-10-06T02:26:28.281 INFO:tasks.cephfs_test_runner:test_malicious_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) 2023-10-06T02:26:28.281 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade' 2023-10-06T02:26:28.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:28.467+0000 7fce5759e700 1 Processor -- start 2023-10-06T02:26:28.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:28.467+0000 7fce5759e700 1 -- start start 2023-10-06T02:26:28.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:28.468+0000 7fce5759e700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fce5813fb20 0x7fce5815a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:28.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:28.468+0000 7fce5759e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce5806aaa0 0x7fce580c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:28.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:28.469+0000 7fce5759e700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce5815ae70 con 0x7fce5813fef0 2023-10-06T02:26:28.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:28.469+0000 7fce5759e700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce5806af60 con 0x7fce5813fb20 2023-10-06T02:26:28.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:28.469+0000 7fce5759e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce5806b240 con 0x7fce5806aaa0 2023-10-06T02:26:28.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:28.469+0000 7fce56d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce5806aaa0 0x7fce580c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:28.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:28.469+0000 7fce5659c700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fce5813fb20 0x7fce5815a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:28.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:28.469+0000 7fce56d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce5806aaa0 0x7fce580c9d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:57496/0 (socket says 172.21.15.31:57496) 2023-10-06T02:26:28.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:28.469+0000 7fce56d9d700 1 -- 172.21.15.31:0/3199277536 learned_addr learned my addr 172.21.15.31:0/3199277536 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:26:28.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:28.470+0000 7fce5659c700 1 -- 172.21.15.31:0/3199277536 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce5806aaa0 msgr2=0x7fce580c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:26:28.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:28.470+0000 7fce5659c700 1 --2- 172.21.15.31:0/3199277536 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce5806aaa0 0x7fce580c9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:28.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:28.470+0000 7fce5659c700 1 -- 172.21.15.31:0/3199277536 >> v1:172.21.15.62:6789/0 conn(0x7fce5813fef0 legacy=0x7fce580c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:28.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:28.470+0000 7fce5659c700 1 -- 172.21.15.31:0/3199277536 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce5806b520 con 0x7fce5813fb20 2023-10-06T02:26:28.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:28.470+0000 7fce56d9d700 1 --2- 172.21.15.31:0/3199277536 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce5806aaa0 0x7fce580c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:26:28.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:28.470+0000 7fce5659c700 1 --2- 172.21.15.31:0/3199277536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce5813fb20 0x7fce5815a930 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fce480098e0 tx=0x7fce480091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d564eaf4536a738e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:28.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:28.470+0000 7fce5559a700 1 -- 172.21.15.31:0/3199277536 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4800b640 con 0x7fce5813fb20 2023-10-06T02:26:28.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:28.470+0000 7fce5559a700 1 -- 172.21.15.31:0/3199277536 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce4800b7a0 con 0x7fce5813fb20 2023-10-06T02:26:28.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:28.470+0000 7fce5559a700 1 -- 172.21.15.31:0/3199277536 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4800bb40 con 0x7fce5813fb20 2023-10-06T02:26:28.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:28.471+0000 7fce5759e700 1 -- 172.21.15.31:0/3199277536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce5813fb20 msgr2=0x7fce5815a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:28.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:28.471+0000 7fce5759e700 1 --2- 172.21.15.31:0/3199277536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce5813fb20 0x7fce5815a930 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fce480098e0 tx=0x7fce480091a0 comp rx=0 tx=0).stop 2023-10-06T02:26:28.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:28.471+0000 7fce5759e700 1 -- 172.21.15.31:0/3199277536 shutdown_connections 2023-10-06T02:26:28.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:28.471+0000 7fce5759e700 1 --2- 172.21.15.31:0/3199277536 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce5806aaa0 0x7fce580c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:28.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:28.471+0000 7fce5759e700 1 --2- 172.21.15.31:0/3199277536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce5813fb20 0x7fce5815a930 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:28.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:28.471+0000 7fce5759e700 1 -- 172.21.15.31:0/3199277536 >> 172.21.15.31:0/3199277536 conn(0x7fce5806d170 msgr2=0x7fce580cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:28.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:28.471+0000 7fce5759e700 1 -- 172.21.15.31:0/3199277536 shutdown_connections 2023-10-06T02:26:28.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:28.471+0000 7fce5759e700 1 -- 172.21.15.31:0/3199277536 wait complete. 2023-10-06T02:26:28.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:28.472+0000 7fce5759e700 1 Processor -- start 2023-10-06T02:26:28.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:28.473+0000 7fce5759e700 1 -- start start 2023-10-06T02:26:28.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:28.474+0000 7fce5759e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce5806aaa0 0x7fce5813ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:28.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:28.474+0000 7fce5759e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce5813fb20 0x7fce58132db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:28.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:28.474+0000 7fce5759e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce5813fef0 0x7fce580c7620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:28.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:28.474+0000 7fce5759e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce58134210 con 0x7fce5813fb20 2023-10-06T02:26:28.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:28.474+0000 7fce5759e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce5813f340 con 0x7fce5806aaa0 2023-10-06T02:26:28.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:28.474+0000 7fce5759e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce58133600 con 0x7fce5813fef0 2023-10-06T02:26:28.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:28.474+0000 7fce55d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce5813fb20 0x7fce58132db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:28.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:28.474+0000 7fce55d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce5813fb20 0x7fce58132db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58626/0 (socket says 172.21.15.31:58626) 2023-10-06T02:26:28.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:28.474+0000 7fce55d9b700 1 -- 172.21.15.31:0/2385404301 learned_addr learned my addr 172.21.15.31:0/2385404301 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:26:28.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:28.474+0000 7fce56d9d700 1 --2- 172.21.15.31:0/2385404301 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce5813fef0 0x7fce580c7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:28.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:28.474+0000 7fce5659c700 1 --2- 172.21.15.31:0/2385404301 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce5806aaa0 0x7fce5813ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:28.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:28.474+0000 7fce55d9b700 1 -- 172.21.15.31:0/2385404301 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce5813fef0 msgr2=0x7fce580c7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:28.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:28.474+0000 7fce55d9b700 1 --2- 172.21.15.31:0/2385404301 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce5813fef0 0x7fce580c7620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:28.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:28.474+0000 7fce55d9b700 1 -- 172.21.15.31:0/2385404301 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce5806aaa0 msgr2=0x7fce5813ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:28.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:28.474+0000 7fce55d9b700 1 --2- 172.21.15.31:0/2385404301 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce5806aaa0 0x7fce5813ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:28.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:28.474+0000 7fce55d9b700 1 -- 172.21.15.31:0/2385404301 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce50009ae0 con 0x7fce5813fb20 2023-10-06T02:26:28.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:28.474+0000 7fce56d9d700 1 --2- 172.21.15.31:0/2385404301 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce5813fef0 0x7fce580c7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:26:28.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:28.475+0000 7fce55d9b700 1 --2- 172.21.15.31:0/2385404301 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce5813fb20 0x7fce58132db0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fce4c00c4e0 tx=0x7fce4c007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:28.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:28.475+0000 7fce467fc700 1 -- 172.21.15.31:0/2385404301 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4c011590 con 0x7fce5813fb20 2023-10-06T02:26:28.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:28.475+0000 7fce467fc700 1 -- 172.21.15.31:0/2385404301 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce4c011710 con 0x7fce5813fb20 2023-10-06T02:26:28.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:28.475+0000 7fce5759e700 1 -- 172.21.15.31:0/2385404301 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce48009420 con 0x7fce5813fb20 2023-10-06T02:26:28.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:28.475+0000 7fce467fc700 1 -- 172.21.15.31:0/2385404301 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4c011ad0 con 0x7fce5813fb20 2023-10-06T02:26:28.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:28.475+0000 7fce5759e700 1 -- 172.21.15.31:0/2385404301 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce58133bc0 con 0x7fce5813fb20 2023-10-06T02:26:28.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:28.476+0000 7fce5759e700 1 -- 172.21.15.31:0/2385404301 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce5804c3c0 con 0x7fce5813fb20 2023-10-06T02:26:28.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:28.477+0000 7fce467fc700 1 -- 172.21.15.31:0/2385404301 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 9) v1 ==== 84893+0+0 (secure 0 0 0) 0x7fce4c01b3b0 con 0x7fce5813fb20 2023-10-06T02:26:28.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:28.478+0000 7fce467fc700 1 --2- 172.21.15.31:0/2385404301 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fce380670e0 0x7fce38069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:28.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:28.478+0000 7fce467fc700 1 -- 172.21.15.31:0/2385404301 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5292+0+0 (secure 0 0 0) 0x7fce4c083900 con 0x7fce5813fb20 2023-10-06T02:26:28.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:28.478+0000 7fce5659c700 1 --2- 172.21.15.31:0/2385404301 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fce380670e0 0x7fce38069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:28.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:28.479+0000 7fce5659c700 1 --2- 172.21.15.31:0/2385404301 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fce380670e0 0x7fce38069590 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fce480093f0 tx=0x7fce48017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:28.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:28.482+0000 7fce467fc700 1 -- 172.21.15.31:0/2385404301 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fce4c050aa0 con 0x7fce5813fb20 2023-10-06T02:26:28.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:28.762+0000 7fce5759e700 1 -- 172.21.15.31:0/2385404301 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade"]} v 0) v1 -- 0x7fce5815ae70 con 0x7fce5813fb20 2023-10-06T02:26:29.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.263+0000 7fce467fc700 1 -- 172.21.15.31:0/2385404301 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade"]}]=0 v191) v1 ==== 164+0+0 (secure 0 0 0) 0x7fce5815ae70 con 0x7fce5813fb20 2023-10-06T02:26:29.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.266+0000 7fce5759e700 1 -- 172.21.15.31:0/2385404301 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fce380670e0 msgr2=0x7fce38069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:29.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.266+0000 7fce5759e700 1 --2- 172.21.15.31:0/2385404301 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fce380670e0 0x7fce38069590 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fce480093f0 tx=0x7fce48017000 comp rx=0 tx=0).stop 2023-10-06T02:26:29.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.266+0000 7fce5759e700 1 -- 172.21.15.31:0/2385404301 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce5813fb20 msgr2=0x7fce58132db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:29.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.266+0000 7fce5759e700 1 --2- 172.21.15.31:0/2385404301 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce5813fb20 0x7fce58132db0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fce4c00c4e0 tx=0x7fce4c007f70 comp rx=0 tx=0).stop 2023-10-06T02:26:29.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.266+0000 7fce5759e700 1 -- 172.21.15.31:0/2385404301 shutdown_connections 2023-10-06T02:26:29.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.266+0000 7fce5759e700 1 --2- 172.21.15.31:0/2385404301 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fce380670e0 0x7fce38069590 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:29.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.266+0000 7fce5759e700 1 --2- 172.21.15.31:0/2385404301 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce5813fef0 0x7fce580c7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:29.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.266+0000 7fce5759e700 1 --2- 172.21.15.31:0/2385404301 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce5806aaa0 0x7fce5813ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:29.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.266+0000 7fce5759e700 1 --2- 172.21.15.31:0/2385404301 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce5813fb20 0x7fce58132db0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:29.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.266+0000 7fce5759e700 1 -- 172.21.15.31:0/2385404301 >> 172.21.15.31:0/2385404301 conn(0x7fce5806d170 msgr2=0x7fce5806de50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:29.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.266+0000 7fce5759e700 1 -- 172.21.15.31:0/2385404301 shutdown_connections 2023-10-06T02:26:29.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.266+0000 7fce5759e700 1 -- 172.21.15.31:0/2385404301 wait complete. 2023-10-06T02:26:29.285 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T02:26:29.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.474+0000 7fdba579d700 1 Processor -- start 2023-10-06T02:26:29.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.474+0000 7fdba579d700 1 -- start start 2023-10-06T02:26:29.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.476+0000 7fdba579d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fdba0155890 0x7fdba00c7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:29.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.476+0000 7fdba579d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fdba006de50 0x7fdba006e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:29.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.476+0000 7fdba579d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdba00c7e70 con 0x7fdba0155c60 2023-10-06T02:26:29.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.476+0000 7fdba579d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdba006e7f0 con 0x7fdba006de50 2023-10-06T02:26:29.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.476+0000 7fdba579d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdba006b790 con 0x7fdba0155890 2023-10-06T02:26:29.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.476+0000 7fdba4f9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fdba006de50 0x7fdba006e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:29.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.476+0000 7fdb9f7fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fdba0155c60 0x7fdba006a3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:47622/0 (socket says 172.21.15.31:47622) 2023-10-06T02:26:29.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.476+0000 7fdb9f7fe700 1 -- 172.21.15.31:0/160294875 learned_addr learned my addr 172.21.15.31:0/160294875 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:26:29.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.476+0000 7fdb9ffff700 1 --2- 172.21.15.31:0/160294875 >> v2:172.21.15.62:3300/0 conn(0x7fdba0155890 0x7fdba00c7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:29.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.477+0000 7fdba4f9c700 1 -- 172.21.15.31:0/160294875 >> v2:172.21.15.62:3300/0 conn(0x7fdba0155890 msgr2=0x7fdba00c7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:26:29.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.477+0000 7fdba4f9c700 1 --2- 172.21.15.31:0/160294875 >> v2:172.21.15.62:3300/0 conn(0x7fdba0155890 0x7fdba00c7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:29.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.477+0000 7fdba4f9c700 1 -- 172.21.15.31:0/160294875 >> v1:172.21.15.31:6789/0 conn(0x7fdba0155c60 legacy=0x7fdba006a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:29.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.477+0000 7fdba4f9c700 1 -- 172.21.15.31:0/160294875 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdba006ab80 con 0x7fdba006de50 2023-10-06T02:26:29.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.477+0000 7fdb9effd700 1 -- 172.21.15.31:0/160294875 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2241148300 0 0) 0x7fdba00c7e70 con 0x7fdba0155c60 2023-10-06T02:26:29.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.477+0000 7fdba4f9c700 1 --2- 172.21.15.31:0/160294875 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdba006de50 0x7fdba006e2b0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fdb98009e60 tx=0x7fdb98009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=870955614f65a718 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:29.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.477+0000 7fdb9effd700 1 -- 172.21.15.31:0/160294875 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb9800e070 con 0x7fdba006de50 2023-10-06T02:26:29.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.477+0000 7fdb9effd700 1 -- 172.21.15.31:0/160294875 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb9800bb20 con 0x7fdba006de50 2023-10-06T02:26:29.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.477+0000 7fdb9effd700 1 -- 172.21.15.31:0/160294875 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb98012430 con 0x7fdba006de50 2023-10-06T02:26:29.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.478+0000 7fdba579d700 1 -- 172.21.15.31:0/160294875 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdba006de50 msgr2=0x7fdba006e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:29.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.478+0000 7fdba579d700 1 --2- 172.21.15.31:0/160294875 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdba006de50 0x7fdba006e2b0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fdb98009e60 tx=0x7fdb98009780 comp rx=0 tx=0).stop 2023-10-06T02:26:29.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.478+0000 7fdba579d700 1 -- 172.21.15.31:0/160294875 shutdown_connections 2023-10-06T02:26:29.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.478+0000 7fdba579d700 1 --2- 172.21.15.31:0/160294875 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdba006de50 0x7fdba006e2b0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:29.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.478+0000 7fdba579d700 1 --2- 172.21.15.31:0/160294875 >> v2:172.21.15.62:3300/0 conn(0x7fdba0155890 0x7fdba00c7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:29.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.478+0000 7fdba579d700 1 -- 172.21.15.31:0/160294875 >> 172.21.15.31:0/160294875 conn(0x7fdba006d110 msgr2=0x7fdba00cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:29.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.478+0000 7fdba579d700 1 -- 172.21.15.31:0/160294875 shutdown_connections 2023-10-06T02:26:29.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.478+0000 7fdba579d700 1 -- 172.21.15.31:0/160294875 wait complete. 2023-10-06T02:26:29.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.479+0000 7fdba579d700 1 Processor -- start 2023-10-06T02:26:29.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.480+0000 7fdba579d700 1 -- start start 2023-10-06T02:26:29.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.480+0000 7fdba579d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdba006de50 0x7fdba00aeed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:29.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.480+0000 7fdba579d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdba0155890 0x7fdba00b1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:29.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.480+0000 7fdba579d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdba0155c60 0x7fdba0158540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:29.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.480+0000 7fdba579d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba00b1e80 con 0x7fdba0155c60 2023-10-06T02:26:29.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.480+0000 7fdba579d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba00c3e20 con 0x7fdba006de50 2023-10-06T02:26:29.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.480+0000 7fdba579d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdba00c3510 con 0x7fdba0155890 2023-10-06T02:26:29.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.481+0000 7fdba4f9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdba0155c60 0x7fdba0158540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:29.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.481+0000 7fdba4f9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdba0155c60 0x7fdba0158540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58634/0 (socket says 172.21.15.31:58634) 2023-10-06T02:26:29.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.481+0000 7fdba4f9c700 1 -- 172.21.15.31:0/2489743015 learned_addr learned my addr 172.21.15.31:0/2489743015 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:26:29.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.481+0000 7fdba4f9c700 1 -- 172.21.15.31:0/2489743015 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdba0155890 msgr2=0x7fdba00b1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:29.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.481+0000 7fdba4f9c700 1 --2- 172.21.15.31:0/2489743015 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdba0155890 0x7fdba00b1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:29.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.481+0000 7fdba4f9c700 1 -- 172.21.15.31:0/2489743015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdba006de50 msgr2=0x7fdba00aeed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:29.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.481+0000 7fdb9ffff700 1 --2- 172.21.15.31:0/2489743015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdba006de50 0x7fdba00aeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:29.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.481+0000 7fdba4f9c700 1 --2- 172.21.15.31:0/2489743015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdba006de50 0x7fdba00aeed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:29.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.481+0000 7fdba4f9c700 1 -- 172.21.15.31:0/2489743015 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb98009a20 con 0x7fdba0155c60 2023-10-06T02:26:29.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.481+0000 7fdb9ffff700 1 --2- 172.21.15.31:0/2489743015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdba006de50 0x7fdba00aeed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:26:29.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.481+0000 7fdba4f9c700 1 --2- 172.21.15.31:0/2489743015 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdba0155c60 0x7fdba0158540 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fdb9800b6c0 tx=0x7fdb98019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:29.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.482+0000 7fdb87fff700 1 -- 172.21.15.31:0/2489743015 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb98012e40 con 0x7fdba0155c60 2023-10-06T02:26:29.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.482+0000 7fdb87fff700 1 -- 172.21.15.31:0/2489743015 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb9801d630 con 0x7fdba0155c60 2023-10-06T02:26:29.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.482+0000 7fdb87fff700 1 -- 172.21.15.31:0/2489743015 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb9801d9f0 con 0x7fdba0155c60 2023-10-06T02:26:29.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.482+0000 7fdba579d700 1 -- 172.21.15.31:0/2489743015 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdba0158a80 con 0x7fdba0155c60 2023-10-06T02:26:29.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.482+0000 7fdba579d700 1 -- 172.21.15.31:0/2489743015 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdba00c37d0 con 0x7fdba0155c60 2023-10-06T02:26:29.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.483+0000 7fdba579d700 1 -- 172.21.15.31:0/2489743015 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdba004c3c0 con 0x7fdba0155c60 2023-10-06T02:26:29.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.484+0000 7fdb87fff700 1 -- 172.21.15.31:0/2489743015 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 9) v1 ==== 84893+0+0 (secure 0 0 0) 0x7fdb98024070 con 0x7fdba0155c60 2023-10-06T02:26:29.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.484+0000 7fdb87fff700 1 --2- 172.21.15.31:0/2489743015 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fdb8c0670e0 0x7fdb8c069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:29.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.485+0000 7fdb87fff700 1 -- 172.21.15.31:0/2489743015 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5292+0+0 (secure 0 0 0) 0x7fdb98086f70 con 0x7fdba0155c60 2023-10-06T02:26:29.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.485+0000 7fdb9ffff700 1 --2- 172.21.15.31:0/2489743015 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fdb8c0670e0 0x7fdb8c069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:29.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.485+0000 7fdb9ffff700 1 --2- 172.21.15.31:0/2489743015 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fdb8c0670e0 0x7fdb8c069590 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fdb90005fd0 tx=0x7fdb90009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:29.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.489+0000 7fdb87fff700 1 -- 172.21.15.31:0/2489743015 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdb98054060 con 0x7fdba0155c60 2023-10-06T02:26:29.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.692+0000 7fdba579d700 1 -- 172.21.15.31:0/2489743015 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fdba00c3f60 con 0x7fdba0155c60 2023-10-06T02:26:29.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.696+0000 7fdb87fff700 1 -- 172.21.15.31:0/2489743015 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7fdb98057680 con 0x7fdba0155c60 2023-10-06T02:26:29.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.698+0000 7fdba579d700 1 -- 172.21.15.31:0/2489743015 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fdb8c0670e0 msgr2=0x7fdb8c069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:29.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.698+0000 7fdba579d700 1 --2- 172.21.15.31:0/2489743015 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fdb8c0670e0 0x7fdb8c069590 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fdb90005fd0 tx=0x7fdb90009040 comp rx=0 tx=0).stop 2023-10-06T02:26:29.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.698+0000 7fdba579d700 1 -- 172.21.15.31:0/2489743015 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdba0155c60 msgr2=0x7fdba0158540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:29.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.698+0000 7fdba579d700 1 --2- 172.21.15.31:0/2489743015 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdba0155c60 0x7fdba0158540 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fdb9800b6c0 tx=0x7fdb98019000 comp rx=0 tx=0).stop 2023-10-06T02:26:29.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.698+0000 7fdba579d700 1 -- 172.21.15.31:0/2489743015 shutdown_connections 2023-10-06T02:26:29.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.698+0000 7fdba579d700 1 --2- 172.21.15.31:0/2489743015 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdba0155c60 0x7fdba0158540 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:29.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.698+0000 7fdba579d700 1 --2- 172.21.15.31:0/2489743015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdba006de50 0x7fdba00aeed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:29.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.698+0000 7fdba579d700 1 --2- 172.21.15.31:0/2489743015 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fdb8c0670e0 0x7fdb8c069590 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:29.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.698+0000 7fdba579d700 1 --2- 172.21.15.31:0/2489743015 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdba0155890 0x7fdba00b1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:29.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.698+0000 7fdba579d700 1 -- 172.21.15.31:0/2489743015 >> 172.21.15.31:0/2489743015 conn(0x7fdba006d110 msgr2=0x7fdba0154c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:29.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.698+0000 7fdba579d700 1 -- 172.21.15.31:0/2489743015 shutdown_connections 2023-10-06T02:26:29.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:29.698+0000 7fdba579d700 1 -- 172.21.15.31:0/2489743015 wait complete. 2023-10-06T02:26:29.716 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T02:26:29.717 DEBUG:teuthology.orchestra.run.smithi031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T02:26:29.803 DEBUG:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T02:26:29.889 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T02:26:29.890 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T02:26:29.890 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:26:30.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:30.076+0000 7f86a359e700 1 Processor -- start 2023-10-06T02:26:30.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:30.076+0000 7f86a359e700 1 -- start start 2023-10-06T02:26:30.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:30.077+0000 7f86a359e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f86a4159550 0x7f86a40ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:30.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:30.077+0000 7f86a359e700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f86a415dfd0 0x7f86a40aedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:30.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:30.077+0000 7f86a359e700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86a40acc00 con 0x7f86a4159920 2023-10-06T02:26:30.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:30.077+0000 7f86a359e700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86a40abff0 con 0x7f86a415dfd0 2023-10-06T02:26:30.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:30.077+0000 7f86a359e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86a40ac2d0 con 0x7f86a4159550 2023-10-06T02:26:30.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:30.078+0000 7f86a259c700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f86a4159550 0x7f86a40ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:30.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:30.078+0000 7f86a259c700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f86a4159550 0x7f86a40ceac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:57512/0 (socket says 172.21.15.31:57512) 2023-10-06T02:26:30.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:30.078+0000 7f86a259c700 1 -- 172.21.15.31:0/3804111223 learned_addr learned my addr 172.21.15.31:0/3804111223 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:26:30.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:30.078+0000 7f86a1d9b700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f86a415dfd0 0x7f86a40aedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:30.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:30.078+0000 7f86a1d9b700 1 -- 172.21.15.31:0/3804111223 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f86a4159550 msgr2=0x7f86a40ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:26:30.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:30.078+0000 7f86a1d9b700 1 --2- 172.21.15.31:0/3804111223 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f86a4159550 0x7f86a40ceac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:30.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:30.078+0000 7f86a1d9b700 1 -- 172.21.15.31:0/3804111223 >> v1:172.21.15.62:6789/0 conn(0x7f86a4159920 legacy=0x7f86a40ad090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:30.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:30.078+0000 7f86a1d9b700 1 -- 172.21.15.31:0/3804111223 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86a40ac5b0 con 0x7f86a415dfd0 2023-10-06T02:26:30.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:30.078+0000 7f86a259c700 1 --2- 172.21.15.31:0/3804111223 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f86a4159550 0x7f86a40ceac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:26:30.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:30.079+0000 7f86a1d9b700 1 --2- 172.21.15.31:0/3804111223 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86a415dfd0 0x7f86a40aedb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f86980098e0 tx=0x7f86980091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15dbd8e7518b817e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:30.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:30.079+0000 7f86a159a700 1 -- 172.21.15.31:0/3804111223 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f869800b640 con 0x7f86a415dfd0 2023-10-06T02:26:30.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:30.079+0000 7f86a159a700 1 -- 172.21.15.31:0/3804111223 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f869800b7a0 con 0x7f86a415dfd0 2023-10-06T02:26:30.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:30.079+0000 7f86a159a700 1 -- 172.21.15.31:0/3804111223 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f869800bb40 con 0x7f86a415dfd0 2023-10-06T02:26:30.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:30.079+0000 7f86a359e700 1 -- 172.21.15.31:0/3804111223 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86a415dfd0 msgr2=0x7f86a40aedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:30.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:30.079+0000 7f86a359e700 1 --2- 172.21.15.31:0/3804111223 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86a415dfd0 0x7f86a40aedb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f86980098e0 tx=0x7f86980091a0 comp rx=0 tx=0).stop 2023-10-06T02:26:30.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:30.080+0000 7f86a359e700 1 -- 172.21.15.31:0/3804111223 shutdown_connections 2023-10-06T02:26:30.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:30.080+0000 7f86a359e700 1 --2- 172.21.15.31:0/3804111223 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f86a4159550 0x7f86a40ceac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:30.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:30.080+0000 7f86a359e700 1 --2- 172.21.15.31:0/3804111223 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86a415dfd0 0x7f86a40aedb0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:30.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:30.080+0000 7f86a359e700 1 -- 172.21.15.31:0/3804111223 >> 172.21.15.31:0/3804111223 conn(0x7f86a406ce80 msgr2=0x7f86a40ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:30.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:30.080+0000 7f86a359e700 1 -- 172.21.15.31:0/3804111223 shutdown_connections 2023-10-06T02:26:30.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:30.080+0000 7f86a359e700 1 -- 172.21.15.31:0/3804111223 wait complete. 2023-10-06T02:26:30.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:30.081+0000 7f86a359e700 1 Processor -- start 2023-10-06T02:26:30.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:30.081+0000 7f86a359e700 1 -- start start 2023-10-06T02:26:30.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:30.082+0000 7f86a359e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86a4159550 0x7f86a40b2140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:30.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:30.082+0000 7f86a359e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f86a4159920 0x7f86a40cd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:30.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:30.082+0000 7f86a359e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86a415dfd0 0x7f86a41469d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:30.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:30.082+0000 7f86a359e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86a416ab70 con 0x7f86a415dfd0 2023-10-06T02:26:30.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:30.082+0000 7f86a359e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86a40b1380 con 0x7f86a4159550 2023-10-06T02:26:30.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:30.082+0000 7f86a359e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86a40b1660 con 0x7f86a4159920 2023-10-06T02:26:30.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:30.082+0000 7f86a2d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86a415dfd0 0x7f86a41469d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:30.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:30.082+0000 7f86a2d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86a415dfd0 0x7f86a41469d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58642/0 (socket says 172.21.15.31:58642) 2023-10-06T02:26:30.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:30.082+0000 7f86a2d9d700 1 -- 172.21.15.31:0/2166807468 learned_addr learned my addr 172.21.15.31:0/2166807468 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:26:30.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:30.083+0000 7f86a1d9b700 1 --2- 172.21.15.31:0/2166807468 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f86a4159920 0x7f86a40cd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:30.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:30.083+0000 7f86a259c700 1 --2- 172.21.15.31:0/2166807468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86a4159550 0x7f86a40b2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:30.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:30.083+0000 7f86a2d9d700 1 -- 172.21.15.31:0/2166807468 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f86a4159920 msgr2=0x7f86a40cd5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:30.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:30.083+0000 7f86a2d9d700 1 --2- 172.21.15.31:0/2166807468 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f86a4159920 0x7f86a40cd5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:30.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:30.083+0000 7f86a2d9d700 1 -- 172.21.15.31:0/2166807468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86a4159550 msgr2=0x7f86a40b2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:30.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:30.083+0000 7f86a2d9d700 1 --2- 172.21.15.31:0/2166807468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86a4159550 0x7f86a40b2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:30.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:30.083+0000 7f86a2d9d700 1 -- 172.21.15.31:0/2166807468 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86940094f0 con 0x7f86a415dfd0 2023-10-06T02:26:30.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:30.083+0000 7f86a2d9d700 1 --2- 172.21.15.31:0/2166807468 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86a415dfd0 0x7f86a41469d0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f869c00c8e0 tx=0x7f869c001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:30.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:30.083+0000 7f86927fc700 1 -- 172.21.15.31:0/2166807468 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f869c012630 con 0x7f86a415dfd0 2023-10-06T02:26:30.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:30.083+0000 7f86927fc700 1 -- 172.21.15.31:0/2166807468 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f869c012790 con 0x7f86a415dfd0 2023-10-06T02:26:30.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:30.083+0000 7f86a359e700 1 -- 172.21.15.31:0/2166807468 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8698009420 con 0x7f86a415dfd0 2023-10-06T02:26:30.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:30.083+0000 7f86927fc700 1 -- 172.21.15.31:0/2166807468 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f869c012b50 con 0x7f86a415dfd0 2023-10-06T02:26:30.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:30.084+0000 7f86a359e700 1 -- 172.21.15.31:0/2166807468 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86a40b1c20 con 0x7f86a415dfd0 2023-10-06T02:26:30.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:30.085+0000 7f86a359e700 1 -- 172.21.15.31:0/2166807468 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86a40b1e10 con 0x7f86a415dfd0 2023-10-06T02:26:30.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:30.085+0000 7f86927fc700 1 -- 172.21.15.31:0/2166807468 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 9) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f869c01a3a0 con 0x7f86a415dfd0 2023-10-06T02:26:30.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:30.086+0000 7f86927fc700 1 --2- 172.21.15.31:0/2166807468 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f86840670e0 0x7f8684069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:30.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:30.086+0000 7f86927fc700 1 -- 172.21.15.31:0/2166807468 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f869c084970 con 0x7f86a415dfd0 2023-10-06T02:26:30.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:30.086+0000 7f86a259c700 1 --2- 172.21.15.31:0/2166807468 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f86840670e0 0x7f8684069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:30.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:30.087+0000 7f86a259c700 1 --2- 172.21.15.31:0/2166807468 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f86840670e0 0x7f8684069590 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8694006d20 tx=0x7f8694009250 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:30.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:30.091+0000 7f86927fc700 1 -- 172.21.15.31:0/2166807468 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f869c051a60 con 0x7f86a415dfd0 2023-10-06T02:26:30.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:30.341+0000 7f86a359e700 1 -- 172.21.15.31:0/2166807468 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f86a404c3c0 con 0x7f86a415dfd0 2023-10-06T02:26:30.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:30.341+0000 7f86927fc700 1 -- 172.21.15.31:0/2166807468 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+3526 (secure 0 0 0) 0x7f869c055080 con 0x7f86a415dfd0 2023-10-06T02:26:30.343 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:26:30.343 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":29,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4949,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/178547366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":178547366},{"type":"v1","addr":"172.21.15.62:6835","nonce":178547366}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":5278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/2578831517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":2578831517},{"type":"v1","addr":"172.21.15.31:6836","nonce":2578831517}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5287,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3502585737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":3502585737},{"type":"v1","addr":"172.21.15.31:6837","nonce":3502585737}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":5608,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/2719803672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2719803672},{"type":"v1","addr":"172.21.15.62:6834","nonce":2719803672}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29}],"filesystems":[]} 2023-10-06T02:26:30.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:30.343+0000 7f86a359e700 1 -- 172.21.15.31:0/2166807468 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f86840670e0 msgr2=0x7f8684069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:30.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:30.343+0000 7f86a359e700 1 --2- 172.21.15.31:0/2166807468 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f86840670e0 0x7f8684069590 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8694006d20 tx=0x7f8694009250 comp rx=0 tx=0).stop 2023-10-06T02:26:30.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:30.343+0000 7f86a359e700 1 -- 172.21.15.31:0/2166807468 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86a415dfd0 msgr2=0x7f86a41469d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:30.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:30.343+0000 7f86a359e700 1 --2- 172.21.15.31:0/2166807468 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86a415dfd0 0x7f86a41469d0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f869c00c8e0 tx=0x7f869c001710 comp rx=0 tx=0).stop 2023-10-06T02:26:30.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:30.344+0000 7f86a359e700 1 -- 172.21.15.31:0/2166807468 shutdown_connections 2023-10-06T02:26:30.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:30.344+0000 7f86a359e700 1 --2- 172.21.15.31:0/2166807468 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86a415dfd0 0x7f86a41469d0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:30.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:30.344+0000 7f86a359e700 1 --2- 172.21.15.31:0/2166807468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86a4159550 0x7f86a40b2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:30.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:30.344+0000 7f86a359e700 1 --2- 172.21.15.31:0/2166807468 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f86840670e0 0x7f8684069590 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:30.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:30.344+0000 7f86a359e700 1 --2- 172.21.15.31:0/2166807468 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f86a4159920 0x7f86a40cd5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:30.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:30.344+0000 7f86a359e700 1 -- 172.21.15.31:0/2166807468 >> 172.21.15.31:0/2166807468 conn(0x7f86a406ce80 msgr2=0x7f86a40aef20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:30.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:30.344+0000 7f86a359e700 1 -- 172.21.15.31:0/2166807468 shutdown_connections 2023-10-06T02:26:30.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:30.344+0000 7f86a359e700 1 -- 172.21.15.31:0/2166807468 wait complete. 2023-10-06T02:26:30.346 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 29 2023-10-06T02:26:30.362 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T02:26:30.363 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T02:26:30.363 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T02:26:30.363 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T02:26:30.363 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T02:26:30.363 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T02:26:30.363 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T02:26:30.364 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T02:26:30.364 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T02:26:30.364 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T02:26:30.364 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T02:26:30.364 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T02:26:30.364 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T02:26:30.364 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T02:26:30.365 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T02:26:30.365 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T02:26:30.379 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T02:26:30.379 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T02:26:30.380 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T02:26:30.380 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T02:26:30.384 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T02:26:30.384 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T02:26:30.424 INFO:tasks.ceph.mds.a:Started 2023-10-06T02:26:30.424 DEBUG:teuthology.parallel:result is None 2023-10-06T02:26:30.427 INFO:tasks.ceph.mds.c:Started 2023-10-06T02:26:30.427 DEBUG:teuthology.parallel:result is None 2023-10-06T02:26:30.429 INFO:tasks.ceph.mds.d:Started 2023-10-06T02:26:30.429 DEBUG:teuthology.parallel:result is None 2023-10-06T02:26:30.557 INFO:tasks.ceph.mds.a.smithi031.stdout:starting mds.a at 2023-10-06T02:26:30.557 INFO:tasks.ceph.mds.c.smithi031.stdout:starting mds.c at 2023-10-06T02:26:30.570 INFO:tasks.ceph.mds.d.smithi062.stdout:starting mds.d at 2023-10-06T02:26:30.994 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T02:26:30.999 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T02:26:36.027 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2023-10-06T02:26:36.384 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T02:26:36.409 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T02:26:36.415 INFO:tasks.ceph.mds.b:Started 2023-10-06T02:26:36.415 DEBUG:teuthology.parallel:result is None 2023-10-06T02:26:36.416 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T02:26:36.556 INFO:tasks.ceph.mds.b.smithi062.stdout:starting mds.b at 2023-10-06T02:26:36.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:36.614+0000 7fef10d9d700 1 Processor -- start 2023-10-06T02:26:36.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:36.615+0000 7fef10d9d700 1 -- start start 2023-10-06T02:26:36.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:36.616+0000 7fef10d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fef0c13fef0 0x7fef0c15a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:36.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:36.616+0000 7fef10d9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fef0c06aaa0 0x7fef0c0c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:36.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:36.616+0000 7fef10d9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef0c15ae70 con 0x7fef0c13fb20 2023-10-06T02:26:36.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:36.616+0000 7fef10d9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef0c06af60 con 0x7fef0c06aaa0 2023-10-06T02:26:36.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:36.616+0000 7fef10d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef0c06b240 con 0x7fef0c13fef0 2023-10-06T02:26:36.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:36.616+0000 7fef0bfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fef0c06aaa0 0x7fef0c0c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:36.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:36.616+0000 7fef0bfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fef0c06aaa0 0x7fef0c0c9d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58690/0 (socket says 172.21.15.31:58690) 2023-10-06T02:26:36.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:36.616+0000 7fef0bfff700 1 -- 172.21.15.31:0/2316325087 learned_addr learned my addr 172.21.15.31:0/2316325087 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:26:36.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:36.617+0000 7fef0affd700 1 --2- 172.21.15.31:0/2316325087 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fef0c13fef0 0x7fef0c15a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:36.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:36.617+0000 7fef0bfff700 1 -- 172.21.15.31:0/2316325087 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fef0c13fef0 msgr2=0x7fef0c15a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:26:36.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:36.617+0000 7fef0bfff700 1 --2- 172.21.15.31:0/2316325087 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fef0c13fef0 0x7fef0c15a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:36.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:36.617+0000 7fef0bfff700 1 -- 172.21.15.31:0/2316325087 >> v1:172.21.15.62:6789/0 conn(0x7fef0c13fb20 legacy=0x7fef0c0c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:36.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:36.617+0000 7fef0bfff700 1 -- 172.21.15.31:0/2316325087 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef0c06b520 con 0x7fef0c06aaa0 2023-10-06T02:26:36.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:36.617+0000 7fef0bfff700 1 --2- 172.21.15.31:0/2316325087 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef0c06aaa0 0x7fef0c0c9d00 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fef0400d1a0 tx=0x7fef0400ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e55bdbf04e3f3c1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:36.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:36.617+0000 7fef0a7fc700 1 -- 172.21.15.31:0/2316325087 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef04010070 con 0x7fef0c06aaa0 2023-10-06T02:26:36.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:36.617+0000 7fef0a7fc700 1 -- 172.21.15.31:0/2316325087 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef0400b0d0 con 0x7fef0c06aaa0 2023-10-06T02:26:36.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:36.617+0000 7fef0a7fc700 1 -- 172.21.15.31:0/2316325087 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef0400b490 con 0x7fef0c06aaa0 2023-10-06T02:26:36.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:36.618+0000 7fef10d9d700 1 -- 172.21.15.31:0/2316325087 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef0c06aaa0 msgr2=0x7fef0c0c9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:36.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:36.618+0000 7fef10d9d700 1 --2- 172.21.15.31:0/2316325087 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef0c06aaa0 0x7fef0c0c9d00 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fef0400d1a0 tx=0x7fef0400ba50 comp rx=0 tx=0).stop 2023-10-06T02:26:36.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:36.618+0000 7fef10d9d700 1 -- 172.21.15.31:0/2316325087 shutdown_connections 2023-10-06T02:26:36.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:36.618+0000 7fef10d9d700 1 --2- 172.21.15.31:0/2316325087 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef0c06aaa0 0x7fef0c0c9d00 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:36.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:36.618+0000 7fef10d9d700 1 --2- 172.21.15.31:0/2316325087 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fef0c13fef0 0x7fef0c15a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:36.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:36.618+0000 7fef10d9d700 1 -- 172.21.15.31:0/2316325087 >> 172.21.15.31:0/2316325087 conn(0x7fef0c06d170 msgr2=0x7fef0c0cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:36.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:36.618+0000 7fef10d9d700 1 -- 172.21.15.31:0/2316325087 shutdown_connections 2023-10-06T02:26:36.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:36.618+0000 7fef10d9d700 1 -- 172.21.15.31:0/2316325087 wait complete. 2023-10-06T02:26:36.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:36.619+0000 7fef10d9d700 1 Processor -- start 2023-10-06T02:26:36.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:36.620+0000 7fef10d9d700 1 -- start start 2023-10-06T02:26:36.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:36.620+0000 7fef10d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef0c06aaa0 0x7fef0c156920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:36.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:36.620+0000 7fef10d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fef0c13fb20 0x7fef0c0ae560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:36.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:36.620+0000 7fef10d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef0c13fef0 0x7fef0c155df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:36.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:36.620+0000 7fef10d9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef0c0aeaa0 con 0x7fef0c13fef0 2023-10-06T02:26:36.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:36.620+0000 7fef10d9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef0c156330 con 0x7fef0c06aaa0 2023-10-06T02:26:36.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:36.620+0000 7fef10d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef0c156fb0 con 0x7fef0c13fb20 2023-10-06T02:26:36.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:36.621+0000 7fef0bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef0c13fef0 0x7fef0c155df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:36.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:36.621+0000 7fef0bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef0c13fef0 0x7fef0c155df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58694/0 (socket says 172.21.15.31:58694) 2023-10-06T02:26:36.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:36.621+0000 7fef0affd700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fef0c13fb20 0x7fef0c0ae560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:36.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:36.621+0000 7fef0bfff700 1 -- 172.21.15.31:0/2490786248 learned_addr learned my addr 172.21.15.31:0/2490786248 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:26:36.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:36.621+0000 7fef0b7fe700 1 --2- 172.21.15.31:0/2490786248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef0c06aaa0 0x7fef0c156920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:36.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:36.621+0000 7fef0bfff700 1 -- 172.21.15.31:0/2490786248 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fef0c13fb20 msgr2=0x7fef0c0ae560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:36.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:36.621+0000 7fef0bfff700 1 --2- 172.21.15.31:0/2490786248 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fef0c13fb20 0x7fef0c0ae560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:36.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:36.621+0000 7fef0bfff700 1 -- 172.21.15.31:0/2490786248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef0c06aaa0 msgr2=0x7fef0c156920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:36.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:36.621+0000 7fef0bfff700 1 --2- 172.21.15.31:0/2490786248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef0c06aaa0 0x7fef0c156920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:36.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:36.621+0000 7fef0bfff700 1 -- 172.21.15.31:0/2490786248 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef0400bcf0 con 0x7fef0c13fef0 2023-10-06T02:26:36.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:36.621+0000 7fef0bfff700 1 --2- 172.21.15.31:0/2490786248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef0c13fef0 0x7fef0c155df0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fef04014000 tx=0x7fef0401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:36.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:36.622+0000 7feef37fe700 1 -- 172.21.15.31:0/2490786248 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef0400ab40 con 0x7fef0c13fef0 2023-10-06T02:26:36.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:36.622+0000 7feef37fe700 1 -- 172.21.15.31:0/2490786248 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef0400acc0 con 0x7fef0c13fef0 2023-10-06T02:26:36.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:36.622+0000 7feef37fe700 1 -- 172.21.15.31:0/2490786248 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef0401d680 con 0x7fef0c13fef0 2023-10-06T02:26:36.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:36.622+0000 7fef10d9d700 1 -- 172.21.15.31:0/2490786248 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef0c157290 con 0x7fef0c13fef0 2023-10-06T02:26:36.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:36.622+0000 7fef10d9d700 1 -- 172.21.15.31:0/2490786248 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef0c157850 con 0x7fef0c13fef0 2023-10-06T02:26:36.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:36.623+0000 7fef10d9d700 1 -- 172.21.15.31:0/2490786248 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef0c04c3c0 con 0x7fef0c13fef0 2023-10-06T02:26:36.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:36.624+0000 7feef37fe700 1 -- 172.21.15.31:0/2490786248 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 9) v1 ==== 84893+0+0 (secure 0 0 0) 0x7fef04003040 con 0x7fef0c13fef0 2023-10-06T02:26:36.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:36.625+0000 7feef37fe700 1 --2- 172.21.15.31:0/2490786248 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7feef80670e0 0x7feef8069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:36.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:36.625+0000 7feef37fe700 1 -- 172.21.15.31:0/2490786248 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5292+0+0 (secure 0 0 0) 0x7fef04085f40 con 0x7fef0c13fef0 2023-10-06T02:26:36.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:36.625+0000 7fef0b7fe700 1 --2- 172.21.15.31:0/2490786248 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7feef80670e0 0x7feef8069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:36.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:36.625+0000 7fef0b7fe700 1 --2- 172.21.15.31:0/2490786248 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7feef80670e0 0x7feef8069590 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7feefc003a80 tx=0x7feefc007ec0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:36.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:36.629+0000 7feef37fe700 1 -- 172.21.15.31:0/2490786248 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fef04052f20 con 0x7fef0c13fef0 2023-10-06T02:26:36.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:36.847+0000 7fef10d9d700 1 -- 172.21.15.31:0/2490786248 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fef0c06e260 con 0x7fef0c13fef0 2023-10-06T02:26:38.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.031+0000 7feef37fe700 1 -- 172.21.15.31:0/2490786248 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v57) v1 ==== 97+0+0 (secure 0 0 0) 0x7fef04056540 con 0x7fef0c13fef0 2023-10-06T02:26:38.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.034+0000 7fef10d9d700 1 -- 172.21.15.31:0/2490786248 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7feef80670e0 msgr2=0x7feef8069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:38.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.034+0000 7fef10d9d700 1 --2- 172.21.15.31:0/2490786248 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7feef80670e0 0x7feef8069590 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7feefc003a80 tx=0x7feefc007ec0 comp rx=0 tx=0).stop 2023-10-06T02:26:38.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.034+0000 7fef10d9d700 1 -- 172.21.15.31:0/2490786248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef0c13fef0 msgr2=0x7fef0c155df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:38.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.034+0000 7fef10d9d700 1 --2- 172.21.15.31:0/2490786248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef0c13fef0 0x7fef0c155df0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fef04014000 tx=0x7fef0401a000 comp rx=0 tx=0).stop 2023-10-06T02:26:38.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.035+0000 7fef10d9d700 1 -- 172.21.15.31:0/2490786248 shutdown_connections 2023-10-06T02:26:38.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.035+0000 7fef10d9d700 1 --2- 172.21.15.31:0/2490786248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef0c13fef0 0x7fef0c155df0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:38.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.035+0000 7fef10d9d700 1 --2- 172.21.15.31:0/2490786248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef0c06aaa0 0x7fef0c156920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:38.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.035+0000 7fef10d9d700 1 --2- 172.21.15.31:0/2490786248 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7feef80670e0 0x7feef8069590 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:38.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.035+0000 7fef10d9d700 1 --2- 172.21.15.31:0/2490786248 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fef0c13fb20 0x7fef0c0ae560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:38.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.035+0000 7fef10d9d700 1 -- 172.21.15.31:0/2490786248 >> 172.21.15.31:0/2490786248 conn(0x7fef0c06d170 msgr2=0x7fef0c06ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:38.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.035+0000 7fef10d9d700 1 -- 172.21.15.31:0/2490786248 shutdown_connections 2023-10-06T02:26:38.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.035+0000 7fef10d9d700 1 -- 172.21.15.31:0/2490786248 wait complete. 2023-10-06T02:26:38.037 INFO:teuthology.orchestra.run.smithi031.stderr: removed all blocklist entries 2023-10-06T02:26:38.056 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T02:26:38.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.247+0000 7f6fdf59e700 1 Processor -- start 2023-10-06T02:26:38.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.247+0000 7f6fdf59e700 1 -- start start 2023-10-06T02:26:38.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.249+0000 7f6fdf59e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6fe013fb20 0x7f6fe015a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:38.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.249+0000 7f6fdf59e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6fe013fef0 0x7f6fe00c7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:38.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.249+0000 7f6fdf59e700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fe015ae70 con 0x7f6fe006aaa0 2023-10-06T02:26:38.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.249+0000 7f6fdf59e700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fe006af60 con 0x7f6fe013fb20 2023-10-06T02:26:38.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.249+0000 7f6fdf59e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe006b240 con 0x7f6fe013fef0 2023-10-06T02:26:38.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.249+0000 7f6fde59c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6fe013fb20 0x7f6fe015a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:38.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.249+0000 7f6fde59c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6fe013fb20 0x7f6fe015a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49744/0 (socket says 172.21.15.31:49744) 2023-10-06T02:26:38.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.249+0000 7f6fde59c700 1 -- 172.21.15.31:0/135436436 learned_addr learned my addr 172.21.15.31:0/135436436 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:26:38.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.249+0000 7f6fddd9b700 1 --2- 172.21.15.31:0/135436436 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6fe013fef0 0x7f6fe00c7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:38.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.250+0000 7f6fde59c700 1 -- 172.21.15.31:0/135436436 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6fe013fef0 msgr2=0x7f6fe00c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:26:38.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.250+0000 7f6fde59c700 1 --2- 172.21.15.31:0/135436436 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6fe013fef0 0x7f6fe00c7990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:38.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.250+0000 7f6fde59c700 1 -- 172.21.15.31:0/135436436 >> v1:172.21.15.62:6789/0 conn(0x7f6fe006aaa0 legacy=0x7f6fe00c6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:38.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.250+0000 7f6fde59c700 1 -- 172.21.15.31:0/135436436 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fe006b520 con 0x7f6fe013fb20 2023-10-06T02:26:38.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.250+0000 7f6fde59c700 1 --2- 172.21.15.31:0/135436436 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fe013fb20 0x7f6fe015a930 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f6fd00099b0 tx=0x7f6fd0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=864932d0462b7569 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:38.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.250+0000 7f6fdd59a700 1 -- 172.21.15.31:0/135436436 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd0009ea0 con 0x7f6fe013fb20 2023-10-06T02:26:38.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.250+0000 7f6fdd59a700 1 -- 172.21.15.31:0/135436436 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fd000b860 con 0x7f6fe013fb20 2023-10-06T02:26:38.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.250+0000 7f6fdd59a700 1 -- 172.21.15.31:0/135436436 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd000bc00 con 0x7f6fe013fb20 2023-10-06T02:26:38.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.251+0000 7f6fdf59e700 1 -- 172.21.15.31:0/135436436 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fe013fb20 msgr2=0x7f6fe015a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:38.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.251+0000 7f6fdf59e700 1 --2- 172.21.15.31:0/135436436 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fe013fb20 0x7f6fe015a930 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f6fd00099b0 tx=0x7f6fd0009250 comp rx=0 tx=0).stop 2023-10-06T02:26:38.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.251+0000 7f6fdf59e700 1 -- 172.21.15.31:0/135436436 shutdown_connections 2023-10-06T02:26:38.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.251+0000 7f6fdf59e700 1 --2- 172.21.15.31:0/135436436 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fe013fb20 0x7f6fe015a930 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:38.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.251+0000 7f6fdf59e700 1 --2- 172.21.15.31:0/135436436 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6fe013fef0 0x7f6fe00c7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:38.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.251+0000 7f6fdf59e700 1 -- 172.21.15.31:0/135436436 >> 172.21.15.31:0/135436436 conn(0x7f6fe006d170 msgr2=0x7f6fe00cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:38.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.251+0000 7f6fdf59e700 1 -- 172.21.15.31:0/135436436 shutdown_connections 2023-10-06T02:26:38.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.251+0000 7f6fdf59e700 1 -- 172.21.15.31:0/135436436 wait complete. 2023-10-06T02:26:38.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.252+0000 7f6fdf59e700 1 Processor -- start 2023-10-06T02:26:38.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.252+0000 7f6fdf59e700 1 -- start start 2023-10-06T02:26:38.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.253+0000 7f6fdf59e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fe006aaa0 0x7f6fe015c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:38.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.253+0000 7f6fdf59e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6fe013fb20 0x7f6fe0155110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:38.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.253+0000 7f6fdf59e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6fe013fef0 0x7f6fe0158710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:38.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.253+0000 7f6fdf59e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe00af7e0 con 0x7f6fe006aaa0 2023-10-06T02:26:38.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.253+0000 7f6fdf59e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe0158c50 con 0x7f6fe013fef0 2023-10-06T02:26:38.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.253+0000 7f6fdf59e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe00aeed0 con 0x7f6fe013fb20 2023-10-06T02:26:38.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.254+0000 7f6fde59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fe006aaa0 0x7f6fe015c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:38.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.254+0000 7f6fde59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fe006aaa0 0x7f6fe015c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49752/0 (socket says 172.21.15.31:49752) 2023-10-06T02:26:38.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.254+0000 7f6fde59c700 1 -- 172.21.15.31:0/606132609 learned_addr learned my addr 172.21.15.31:0/606132609 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:26:38.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.254+0000 7f6fded9d700 1 --2- 172.21.15.31:0/606132609 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6fe013fef0 0x7f6fe0158710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:38.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.254+0000 7f6fddd9b700 1 --2- 172.21.15.31:0/606132609 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6fe013fb20 0x7f6fe0155110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:38.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.254+0000 7f6fde59c700 1 -- 172.21.15.31:0/606132609 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6fe013fb20 msgr2=0x7f6fe0155110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:38.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.254+0000 7f6fde59c700 1 --2- 172.21.15.31:0/606132609 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6fe013fb20 0x7f6fe0155110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:38.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.254+0000 7f6fde59c700 1 -- 172.21.15.31:0/606132609 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6fe013fef0 msgr2=0x7f6fe0158710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:38.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.254+0000 7f6fde59c700 1 --2- 172.21.15.31:0/606132609 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6fe013fef0 0x7f6fe0158710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:38.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.254+0000 7f6fde59c700 1 -- 172.21.15.31:0/606132609 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fd00094f0 con 0x7f6fe006aaa0 2023-10-06T02:26:38.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.254+0000 7f6fde59c700 1 --2- 172.21.15.31:0/606132609 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fe006aaa0 0x7f6fe015c370 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f6fd00094c0 tx=0x7f6fd0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:38.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.254+0000 7f6fce7fc700 1 -- 172.21.15.31:0/606132609 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd000bbe0 con 0x7f6fe006aaa0 2023-10-06T02:26:38.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.255+0000 7f6fce7fc700 1 -- 172.21.15.31:0/606132609 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fd0010470 con 0x7f6fe006aaa0 2023-10-06T02:26:38.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.255+0000 7f6fdf59e700 1 -- 172.21.15.31:0/606132609 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fe0158ed0 con 0x7f6fe006aaa0 2023-10-06T02:26:38.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.255+0000 7f6fce7fc700 1 -- 172.21.15.31:0/606132609 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd0010810 con 0x7f6fe006aaa0 2023-10-06T02:26:38.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.255+0000 7f6fdf59e700 1 -- 172.21.15.31:0/606132609 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fe00af470 con 0x7f6fe006aaa0 2023-10-06T02:26:38.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.256+0000 7f6fdf59e700 1 -- 172.21.15.31:0/606132609 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fe004c3c0 con 0x7f6fe006aaa0 2023-10-06T02:26:38.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.256+0000 7f6fce7fc700 1 -- 172.21.15.31:0/606132609 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 9) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f6fd00233f0 con 0x7f6fe006aaa0 2023-10-06T02:26:38.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.258+0000 7f6fce7fc700 1 --2- 172.21.15.31:0/606132609 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6fc0067110 0x7f6fc00695c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:38.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.258+0000 7f6fce7fc700 1 -- 172.21.15.31:0/606132609 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f6fd0014070 con 0x7f6fe006aaa0 2023-10-06T02:26:38.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.258+0000 7f6fddd9b700 1 --2- 172.21.15.31:0/606132609 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6fc0067110 0x7f6fc00695c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:38.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.258+0000 7f6fddd9b700 1 --2- 172.21.15.31:0/606132609 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6fc0067110 0x7f6fc00695c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6fe016d140 tx=0x7f6fd4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:38.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.262+0000 7f6fce7fc700 1 -- 172.21.15.31:0/606132609 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6fd0053900 con 0x7f6fe006aaa0 2023-10-06T02:26:38.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.545+0000 7f6fdf59e700 1 -- 172.21.15.31:0/606132609 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f6fe013ef00 con 0x7f6fe006aaa0 2023-10-06T02:26:38.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.545+0000 7f6fce7fc700 1 -- 172.21.15.31:0/606132609 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f6fd0056f20 con 0x7f6fe006aaa0 2023-10-06T02:26:38.546 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:26:38.546 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T02:26:38.546 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_dump": [ 2023-10-06T02:26:38.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:26:38.546 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.a", 2023-10-06T02:26:38.546 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBLbx9lbjrOGxAAVoSOoDaZb1temL41F4Nong==", 2023-10-06T02:26:38.547 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:26:38.547 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:26:38.547 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T02:26:38.547 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T02:26:38.547 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:26:38.547 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:26:38.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:26:38.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:26:38.547 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.b", 2023-10-06T02:26:38.548 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBLbx9lFgsTMRAATTklzroiyQbTLiZc9j1BHQ==", 2023-10-06T02:26:38.548 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:26:38.548 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:26:38.548 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T02:26:38.548 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T02:26:38.548 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:26:38.548 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:26:38.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:26:38.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:26:38.549 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.c", 2023-10-06T02:26:38.549 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBLbx9lezVtJhAA8wK7+dJ1OmIc+iXJkexK0w==", 2023-10-06T02:26:38.549 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:26:38.549 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:26:38.549 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T02:26:38.549 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T02:26:38.549 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:26:38.549 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:26:38.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:26:38.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:26:38.549 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.d", 2023-10-06T02:26:38.549 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBMbx9l74gWABAApBu2nVv12rGA+OBfBnEc3A==", 2023-10-06T02:26:38.550 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:26:38.550 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:26:38.550 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T02:26:38.550 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T02:26:38.550 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:26:38.550 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:26:38.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:26:38.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:26:38.550 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.0", 2023-10-06T02:26:38.550 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBPbx9lgj6+MBAAXSUD0GCG+rO/9DQBZYt/8w==", 2023-10-06T02:26:38.551 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:26:38.551 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:26:38.551 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:26:38.551 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:26:38.551 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:26:38.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:26:38.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:26:38.551 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.1", 2023-10-06T02:26:38.551 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBSbx9lXhMTNBAAeEdPnMP71ZGb6k2Ddg5raQ==", 2023-10-06T02:26:38.551 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:26:38.551 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:26:38.551 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:26:38.552 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:26:38.552 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:26:38.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:26:38.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:26:38.552 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.2", 2023-10-06T02:26:38.552 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBVbx9l4Ly0NhAATYXYDAjYuP53zjqK0yMboA==", 2023-10-06T02:26:38.552 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:26:38.552 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:26:38.552 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:26:38.552 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:26:38.552 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:26:38.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:26:38.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:26:38.553 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.3", 2023-10-06T02:26:38.553 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBZbx9lzsi4ARAAbGnNV0QyL/7itzy827+chg==", 2023-10-06T02:26:38.553 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:26:38.553 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:26:38.553 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:26:38.553 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:26:38.553 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:26:38.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:26:38.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:26:38.553 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.4", 2023-10-06T02:26:38.553 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBfbx9lW4OSEhAAj7HnVh/yq12zzCaXd6BHCA==", 2023-10-06T02:26:38.554 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:26:38.554 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:26:38.554 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:26:38.554 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:26:38.554 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:26:38.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:26:38.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:26:38.554 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.5", 2023-10-06T02:26:38.554 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBibx9lBcaIFRAAV8wq8QjsCOkI9Pzf/E+9bA==", 2023-10-06T02:26:38.554 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:26:38.554 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:26:38.554 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:26:38.555 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:26:38.555 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:26:38.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:26:38.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:26:38.555 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.6", 2023-10-06T02:26:38.555 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBlbx9lA7S+FxAA38mlSAurT+DpwMzH53zFcw==", 2023-10-06T02:26:38.555 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:26:38.555 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:26:38.555 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:26:38.555 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:26:38.555 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:26:38.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:26:38.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:26:38.556 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.7", 2023-10-06T02:26:38.556 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBobx9l5RkdHhAAxn0Z8IAE/o9oe2LRp3O9gg==", 2023-10-06T02:26:38.556 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:26:38.556 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:26:38.556 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:26:38.556 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:26:38.556 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:26:38.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:26:38.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:26:38.556 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.0", 2023-10-06T02:26:38.557 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBMbx9lh44fCRAA7sMISvk1t7dEJg4ZcBiMcA==", 2023-10-06T02:26:38.557 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:26:38.557 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:26:38.557 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow r", 2023-10-06T02:26:38.557 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T02:26:38.557 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:26:38.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:26:38.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:26:38.557 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.1", 2023-10-06T02:26:38.557 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBMbx9lP0zqFhAAlov2y7ipSvS3AkOdQkhg6g==", 2023-10-06T02:26:38.557 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:26:38.557 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:26:38.558 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-10-06T02:26:38.558 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-10-06T02:26:38.558 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-10-06T02:26:38.558 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:26:38.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:26:38.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:26:38.558 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.2", 2023-10-06T02:26:38.558 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBMbx9lpxYAEBAACtiI1XgQXM8yGGcpFyNK1w==", 2023-10-06T02:26:38.558 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:26:38.558 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:26:38.558 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-10-06T02:26:38.558 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-10-06T02:26:38.559 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-10-06T02:26:38.559 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:26:38.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:26:38.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:26:38.559 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.3", 2023-10-06T02:26:38.559 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBMbx9l81/PHRAAYFOoaezu0pwE0NdtnlYfVA==", 2023-10-06T02:26:38.559 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:26:38.559 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:26:38.559 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-10-06T02:26:38.559 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-10-06T02:26:38.559 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-10-06T02:26:38.559 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:26:38.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:26:38.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:26:38.560 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.admin", 2023-10-06T02:26:38.560 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBKbx9liFl1JBAAFyyHYNLVx1/Ex5a2MKm8rA==", 2023-10-06T02:26:38.560 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:26:38.560 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-10-06T02:26:38.560 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T02:26:38.560 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T02:26:38.560 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:26:38.560 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:26:38.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:26:38.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:26:38.561 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mgr.x", 2023-10-06T02:26:38.561 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBLbx9lTBbIFBAAX2wsioKL4Nx4qz/qwsCsNA==", 2023-10-06T02:26:38.561 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:26:38.561 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-10-06T02:26:38.561 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile mgr", 2023-10-06T02:26:38.561 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:26:38.561 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:26:38.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:26:38.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:26:38.561 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mgr.y", 2023-10-06T02:26:38.561 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBLbx9l4+oLDRAA5YVo3q6FI9e9W/rJYROIsg==", 2023-10-06T02:26:38.562 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:26:38.562 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-10-06T02:26:38.562 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile mgr", 2023-10-06T02:26:38.562 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:26:38.562 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:26:38.562 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:26:38.562 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-10-06T02:26:38.562 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T02:26:38.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.548+0000 7f6fdf59e700 1 -- 172.21.15.31:0/606132609 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6fc0067110 msgr2=0x7f6fc00695c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:38.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.548+0000 7f6fdf59e700 1 --2- 172.21.15.31:0/606132609 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6fc0067110 0x7f6fc00695c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6fe016d140 tx=0x7f6fd4008040 comp rx=0 tx=0).stop 2023-10-06T02:26:38.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.548+0000 7f6fdf59e700 1 -- 172.21.15.31:0/606132609 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fe006aaa0 msgr2=0x7f6fe015c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:38.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.548+0000 7f6fdf59e700 1 --2- 172.21.15.31:0/606132609 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fe006aaa0 0x7f6fe015c370 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f6fd00094c0 tx=0x7f6fd0019000 comp rx=0 tx=0).stop 2023-10-06T02:26:38.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.548+0000 7f6fdf59e700 1 -- 172.21.15.31:0/606132609 shutdown_connections 2023-10-06T02:26:38.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.548+0000 7f6fdf59e700 1 --2- 172.21.15.31:0/606132609 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6fe013fef0 0x7f6fe0158710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:38.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.548+0000 7f6fdf59e700 1 --2- 172.21.15.31:0/606132609 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fe006aaa0 0x7f6fe015c370 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:38.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.548+0000 7f6fdf59e700 1 --2- 172.21.15.31:0/606132609 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6fc0067110 0x7f6fc00695c0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:38.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.548+0000 7f6fdf59e700 1 --2- 172.21.15.31:0/606132609 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6fe013fb20 0x7f6fe0155110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:38.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.548+0000 7f6fdf59e700 1 -- 172.21.15.31:0/606132609 >> 172.21.15.31:0/606132609 conn(0x7f6fe006d170 msgr2=0x7f6fe013de80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:38.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.548+0000 7f6fdf59e700 1 -- 172.21.15.31:0/606132609 shutdown_connections 2023-10-06T02:26:38.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.548+0000 7f6fdf59e700 1 -- 172.21.15.31:0/606132609 wait complete. 2023-10-06T02:26:38.566 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:26:38.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.752+0000 7f95b2b9d700 1 Processor -- start 2023-10-06T02:26:38.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.752+0000 7f95b2b9d700 1 -- start start 2023-10-06T02:26:38.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.753+0000 7f95b2b9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f95ac140a70 0x7f95ac0c6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:38.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.753+0000 7f95b2b9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95ac0c6d50 con 0x7f95ac140e40 2023-10-06T02:26:38.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.753+0000 7f95b2b9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95ac0c7030 con 0x7f95ac06de50 2023-10-06T02:26:38.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.753+0000 7f95b2b9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95ac0c7310 con 0x7f95ac140a70 2023-10-06T02:26:38.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.753+0000 7f95b1b9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f95ac140a70 0x7f95ac0c6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:38.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.753+0000 7f95b1b9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f95ac140a70 0x7f95ac0c6770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49754/0 (socket says 172.21.15.31:49754) 2023-10-06T02:26:38.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.753+0000 7f95b139a700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f95ac140e40 0x7f95ac0c7df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46630/0 (socket says 172.21.15.31:46630) 2023-10-06T02:26:38.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.754+0000 7f95b1b9b700 1 -- 172.21.15.31:0/1495702155 learned_addr learned my addr 172.21.15.31:0/1495702155 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:26:38.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.754+0000 7f95b1b9b700 1 -- 172.21.15.31:0/1495702155 >> v1:172.21.15.62:6789/0 conn(0x7f95ac06de50 legacy=0x7f95ac0aeef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:38.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.754+0000 7f95b1b9b700 1 -- 172.21.15.31:0/1495702155 >> v1:172.21.15.31:6789/0 conn(0x7f95ac140e40 legacy=0x7f95ac0c7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:38.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.754+0000 7f95b1b9b700 1 -- 172.21.15.31:0/1495702155 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95ac0c75f0 con 0x7f95ac140a70 2023-10-06T02:26:38.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.754+0000 7f95b0b99700 1 -- 172.21.15.31:0/1495702155 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2863729570 0 0) 0x7f95ac0c6d50 con 0x7f95ac140e40 2023-10-06T02:26:38.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.754+0000 7f95b1b9b700 1 --2- 172.21.15.31:0/1495702155 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95ac140a70 0x7f95ac0c6770 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f95a80099b0 tx=0x7f95a8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25cbb9a9ace61190 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:38.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.754+0000 7f95b0b99700 1 -- 172.21.15.31:0/1495702155 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a800b860 con 0x7f95ac140a70 2023-10-06T02:26:38.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.754+0000 7f95b0b99700 1 -- 172.21.15.31:0/1495702155 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95a800b9c0 con 0x7f95ac140a70 2023-10-06T02:26:38.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.754+0000 7f95b0b99700 1 -- 172.21.15.31:0/1495702155 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a800bd60 con 0x7f95ac140a70 2023-10-06T02:26:38.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.755+0000 7f95b2b9d700 1 -- 172.21.15.31:0/1495702155 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95ac140a70 msgr2=0x7f95ac0c6770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:38.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.755+0000 7f95b2b9d700 1 --2- 172.21.15.31:0/1495702155 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95ac140a70 0x7f95ac0c6770 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f95a80099b0 tx=0x7f95a8009250 comp rx=0 tx=0).stop 2023-10-06T02:26:38.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.755+0000 7f95b2b9d700 1 -- 172.21.15.31:0/1495702155 shutdown_connections 2023-10-06T02:26:38.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.755+0000 7f95b2b9d700 1 --2- 172.21.15.31:0/1495702155 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95ac140a70 0x7f95ac0c6770 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:38.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.755+0000 7f95b2b9d700 1 -- 172.21.15.31:0/1495702155 >> 172.21.15.31:0/1495702155 conn(0x7f95ac06d170 msgr2=0x7f95ac0cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:38.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.755+0000 7f95b2b9d700 1 -- 172.21.15.31:0/1495702155 shutdown_connections 2023-10-06T02:26:38.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.755+0000 7f95b2b9d700 1 -- 172.21.15.31:0/1495702155 wait complete. 2023-10-06T02:26:38.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.757+0000 7f95b2b9d700 1 Processor -- start 2023-10-06T02:26:38.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.757+0000 7f95b2b9d700 1 -- start start 2023-10-06T02:26:38.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.758+0000 7f95b2b9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95ac06de50 0x7f95ac158930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:38.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.758+0000 7f95b2b9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95ac140a70 0x7f95ac0c4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:38.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.758+0000 7f95b2b9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f95ac140e40 0x7f95ac156810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:38.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.758+0000 7f95b2b9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac0c5af0 con 0x7f95ac06de50 2023-10-06T02:26:38.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.758+0000 7f95b2b9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac0ca700 con 0x7f95ac140a70 2023-10-06T02:26:38.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.758+0000 7f95b2b9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac16a110 con 0x7f95ac140e40 2023-10-06T02:26:38.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.758+0000 7f95b1b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95ac06de50 0x7f95ac158930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:38.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.758+0000 7f95b1b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95ac06de50 0x7f95ac158930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49764/0 (socket says 172.21.15.31:49764) 2023-10-06T02:26:38.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.758+0000 7f95b1b9b700 1 -- 172.21.15.31:0/2583312446 learned_addr learned my addr 172.21.15.31:0/2583312446 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:26:38.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.758+0000 7f95b1b9b700 1 -- 172.21.15.31:0/2583312446 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f95ac140e40 msgr2=0x7f95ac156810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:38.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.758+0000 7f95b1b9b700 1 --2- 172.21.15.31:0/2583312446 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f95ac140e40 0x7f95ac156810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:38.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.758+0000 7f95b1b9b700 1 -- 172.21.15.31:0/2583312446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95ac140a70 msgr2=0x7f95ac0c4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:38.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.758+0000 7f95b1b9b700 1 --2- 172.21.15.31:0/2583312446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95ac140a70 0x7f95ac0c4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:38.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.758+0000 7f95b1b9b700 1 -- 172.21.15.31:0/2583312446 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95a80094f0 con 0x7f95ac06de50 2023-10-06T02:26:38.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.759+0000 7f95b1b9b700 1 --2- 172.21.15.31:0/2583312446 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95ac06de50 0x7f95ac158930 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f95a80094c0 tx=0x7f95a8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:38.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.759+0000 7f95a1ffb700 1 -- 172.21.15.31:0/2583312446 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a801f070 con 0x7f95ac06de50 2023-10-06T02:26:38.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.759+0000 7f95a1ffb700 1 -- 172.21.15.31:0/2583312446 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95a8011d10 con 0x7f95ac06de50 2023-10-06T02:26:38.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.759+0000 7f95b2b9d700 1 -- 172.21.15.31:0/2583312446 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95ac0c5530 con 0x7f95ac06de50 2023-10-06T02:26:38.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.759+0000 7f95a1ffb700 1 -- 172.21.15.31:0/2583312446 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a8025650 con 0x7f95ac06de50 2023-10-06T02:26:38.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.759+0000 7f95b2b9d700 1 -- 172.21.15.31:0/2583312446 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95ac0ca390 con 0x7f95ac06de50 2023-10-06T02:26:38.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.760+0000 7f95b2b9d700 1 -- 172.21.15.31:0/2583312446 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95ac04c3c0 con 0x7f95ac06de50 2023-10-06T02:26:38.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.761+0000 7f95a1ffb700 1 -- 172.21.15.31:0/2583312446 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 9) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f95a8023070 con 0x7f95ac06de50 2023-10-06T02:26:38.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.762+0000 7f95a1ffb700 1 --2- 172.21.15.31:0/2583312446 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f95900670e0 0x7f9590069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:38.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.762+0000 7f95a1ffb700 1 -- 172.21.15.31:0/2583312446 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f95a8085d20 con 0x7f95ac06de50 2023-10-06T02:26:38.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.762+0000 7f95b139a700 1 --2- 172.21.15.31:0/2583312446 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f95900670e0 0x7f9590069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:38.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.763+0000 7f95b139a700 1 --2- 172.21.15.31:0/2583312446 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f95900670e0 0x7f9590069590 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f959c000c00 tx=0x7f959c007ae0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:38.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.766+0000 7f95a1ffb700 1 -- 172.21.15.31:0/2583312446 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f95a8052e60 con 0x7f95ac06de50 2023-10-06T02:26:38.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.968+0000 7f95b2b9d700 1 -- 172.21.15.31:0/2583312446 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f95ac0c78d0 con 0x7f95ac06de50 2023-10-06T02:26:38.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.969+0000 7f95a1ffb700 1 -- 172.21.15.31:0/2583312446 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) v1 ==== 74+0+13602 (secure 0 0 0) 0x7f95a8015090 con 0x7f95ac06de50 2023-10-06T02:26:38.970 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:26:38.970 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":57,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:26:38.026338+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:26:38.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.971+0000 7f95b2b9d700 1 -- 172.21.15.31:0/2583312446 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f95900670e0 msgr2=0x7f9590069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:38.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.971+0000 7f95b2b9d700 1 --2- 172.21.15.31:0/2583312446 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f95900670e0 0x7f9590069590 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f959c000c00 tx=0x7f959c007ae0 comp rx=0 tx=0).stop 2023-10-06T02:26:38.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.971+0000 7f95b2b9d700 1 -- 172.21.15.31:0/2583312446 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95ac06de50 msgr2=0x7f95ac158930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:38.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.971+0000 7f95b2b9d700 1 --2- 172.21.15.31:0/2583312446 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95ac06de50 0x7f95ac158930 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f95a80094c0 tx=0x7f95a8017000 comp rx=0 tx=0).stop 2023-10-06T02:26:38.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.971+0000 7f95b2b9d700 1 -- 172.21.15.31:0/2583312446 shutdown_connections 2023-10-06T02:26:38.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.971+0000 7f95b2b9d700 1 --2- 172.21.15.31:0/2583312446 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f95900670e0 0x7f9590069590 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:38.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.971+0000 7f95b2b9d700 1 --2- 172.21.15.31:0/2583312446 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f95ac140e40 0x7f95ac156810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:38.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.971+0000 7f95b2b9d700 1 --2- 172.21.15.31:0/2583312446 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95ac06de50 0x7f95ac158930 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:38.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.971+0000 7f95b2b9d700 1 --2- 172.21.15.31:0/2583312446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95ac140a70 0x7f95ac0c4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:38.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.971+0000 7f95b2b9d700 1 -- 172.21.15.31:0/2583312446 >> 172.21.15.31:0/2583312446 conn(0x7f95ac06d170 msgr2=0x7f95ac0c2930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:38.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.972+0000 7f95b2b9d700 1 -- 172.21.15.31:0/2583312446 shutdown_connections 2023-10-06T02:26:38.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:38.972+0000 7f95b2b9d700 1 -- 172.21.15.31:0/2583312446 wait complete. 2023-10-06T02:26:38.989 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T02:22:53.183425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T02:23:00.046967+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-10-06T02:26:38.990 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T02:26:39.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.179+0000 7f5a5d39d700 1 Processor -- start 2023-10-06T02:26:39.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.180+0000 7f5a5d39d700 1 -- start start 2023-10-06T02:26:39.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.181+0000 7f5a5d39d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f5a58155c60 0x7f5a580c7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:39.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.181+0000 7f5a5d39d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5a5806de50 0x7f5a5806e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:39.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.181+0000 7f5a5d39d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a580c7e70 con 0x7f5a58155890 2023-10-06T02:26:39.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.181+0000 7f5a5d39d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a5806e7f0 con 0x7f5a5806de50 2023-10-06T02:26:39.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.181+0000 7f5a5d39d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a5806b790 con 0x7f5a58155c60 2023-10-06T02:26:39.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.181+0000 7f5a5cb9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5a5806de50 0x7f5a5806e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:39.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.181+0000 7f5a5cb9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5a5806de50 0x7f5a5806e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49780/0 (socket says 172.21.15.31:49780) 2023-10-06T02:26:39.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.181+0000 7f5a5cb9c700 1 -- 172.21.15.31:0/904300795 learned_addr learned my addr 172.21.15.31:0/904300795 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:26:39.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.182+0000 7f5a5cb9c700 1 -- 172.21.15.31:0/904300795 >> v2:172.21.15.62:3300/0 conn(0x7f5a58155c60 msgr2=0x7f5a580c7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:26:39.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.182+0000 7f5a577fe700 1 --2- 172.21.15.31:0/904300795 >> v2:172.21.15.62:3300/0 conn(0x7f5a58155c60 0x7f5a580c7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:39.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.182+0000 7f5a5cb9c700 1 --2- 172.21.15.31:0/904300795 >> v2:172.21.15.62:3300/0 conn(0x7f5a58155c60 0x7f5a580c7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:39.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.182+0000 7f5a5cb9c700 1 -- 172.21.15.31:0/904300795 >> v1:172.21.15.62:6789/0 conn(0x7f5a58155890 legacy=0x7f5a5806a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:26:39.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.182+0000 7f5a5cb9c700 1 -- 172.21.15.31:0/904300795 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a5806ab80 con 0x7f5a5806de50 2023-10-06T02:26:39.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.182+0000 7f5a577fe700 1 --2- 172.21.15.31:0/904300795 >> v2:172.21.15.62:3300/0 conn(0x7f5a58155c60 0x7f5a580c7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:26:39.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.182+0000 7f5a5cb9c700 1 --2- 172.21.15.31:0/904300795 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a5806de50 0x7f5a5806e2b0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f5a580697f0 tx=0x7f5a50009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bba671646a362a2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:39.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.182+0000 7f5a56ffd700 1 -- 172.21.15.31:0/904300795 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a5000e070 con 0x7f5a5806de50 2023-10-06T02:26:39.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.182+0000 7f5a56ffd700 1 -- 172.21.15.31:0/904300795 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a5000ba10 con 0x7f5a5806de50 2023-10-06T02:26:39.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.182+0000 7f5a56ffd700 1 -- 172.21.15.31:0/904300795 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a500092e0 con 0x7f5a5806de50 2023-10-06T02:26:39.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.183+0000 7f5a5d39d700 1 -- 172.21.15.31:0/904300795 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a5806de50 msgr2=0x7f5a5806e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:39.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.183+0000 7f5a5d39d700 1 --2- 172.21.15.31:0/904300795 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a5806de50 0x7f5a5806e2b0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f5a580697f0 tx=0x7f5a50009840 comp rx=0 tx=0).stop 2023-10-06T02:26:39.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.183+0000 7f5a5d39d700 1 -- 172.21.15.31:0/904300795 shutdown_connections 2023-10-06T02:26:39.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.183+0000 7f5a5d39d700 1 --2- 172.21.15.31:0/904300795 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a5806de50 0x7f5a5806e2b0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:39.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.183+0000 7f5a5d39d700 1 --2- 172.21.15.31:0/904300795 >> v2:172.21.15.62:3300/0 conn(0x7f5a58155c60 0x7f5a580c7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:39.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.183+0000 7f5a5d39d700 1 -- 172.21.15.31:0/904300795 >> 172.21.15.31:0/904300795 conn(0x7f5a5806d110 msgr2=0x7f5a580cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:39.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.183+0000 7f5a5d39d700 1 -- 172.21.15.31:0/904300795 shutdown_connections 2023-10-06T02:26:39.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.183+0000 7f5a5d39d700 1 -- 172.21.15.31:0/904300795 wait complete. 2023-10-06T02:26:39.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.184+0000 7f5a5d39d700 1 Processor -- start 2023-10-06T02:26:39.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.185+0000 7f5a5d39d700 1 -- start start 2023-10-06T02:26:39.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.185+0000 7f5a5d39d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5a5806de50 0x7f5a580c5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:39.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.185+0000 7f5a5d39d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a58155890 0x7f5a58144b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:39.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.185+0000 7f5a5d39d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a58155c60 0x7f5a5814d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:39.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.186+0000 7f5a5d39d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a580c6190 con 0x7f5a58155890 2023-10-06T02:26:39.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.186+0000 7f5a5d39d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a580c6470 con 0x7f5a58155c60 2023-10-06T02:26:39.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.186+0000 7f5a5d39d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a580c6750 con 0x7f5a5806de50 2023-10-06T02:26:39.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.186+0000 7f5a577fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a58155890 0x7f5a58144b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:39.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.186+0000 7f5a577fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a58155890 0x7f5a58144b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49796/0 (socket says 172.21.15.31:49796) 2023-10-06T02:26:39.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.186+0000 7f5a577fe700 1 -- 172.21.15.31:0/2171420512 learned_addr learned my addr 172.21.15.31:0/2171420512 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:26:39.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.186+0000 7f5a577fe700 1 -- 172.21.15.31:0/2171420512 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5a5806de50 msgr2=0x7f5a580c5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:39.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.186+0000 7f5a57fff700 1 --2- 172.21.15.31:0/2171420512 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5a5806de50 0x7f5a580c5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:39.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.186+0000 7f5a577fe700 1 --2- 172.21.15.31:0/2171420512 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5a5806de50 0x7f5a580c5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:39.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.186+0000 7f5a577fe700 1 -- 172.21.15.31:0/2171420512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a58155c60 msgr2=0x7f5a5814d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:39.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.186+0000 7f5a577fe700 1 --2- 172.21.15.31:0/2171420512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a58155c60 0x7f5a5814d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:39.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.186+0000 7f5a577fe700 1 -- 172.21.15.31:0/2171420512 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a50009ae0 con 0x7f5a58155890 2023-10-06T02:26:39.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.186+0000 7f5a57fff700 1 --2- 172.21.15.31:0/2171420512 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5a5806de50 0x7f5a580c5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:26:39.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.186+0000 7f5a577fe700 1 --2- 172.21.15.31:0/2171420512 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a58155890 0x7f5a58144b50 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c00c850 tx=0x7f5a4c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:39.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.187+0000 7f5a3ffff700 1 -- 172.21.15.31:0/2171420512 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a4c011600 con 0x7f5a58155890 2023-10-06T02:26:39.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.187+0000 7f5a3ffff700 1 -- 172.21.15.31:0/2171420512 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a4c011760 con 0x7f5a58155890 2023-10-06T02:26:39.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.187+0000 7f5a5d39d700 1 -- 172.21.15.31:0/2171420512 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a580af310 con 0x7f5a58155890 2023-10-06T02:26:39.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.187+0000 7f5a3ffff700 1 -- 172.21.15.31:0/2171420512 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a4c011b00 con 0x7f5a58155890 2023-10-06T02:26:39.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.187+0000 7f5a5d39d700 1 -- 172.21.15.31:0/2171420512 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a580c5e90 con 0x7f5a58155890 2023-10-06T02:26:39.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.188+0000 7f5a5d39d700 1 -- 172.21.15.31:0/2171420512 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a5804c3c0 con 0x7f5a58155890 2023-10-06T02:26:39.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.189+0000 7f5a3ffff700 1 -- 172.21.15.31:0/2171420512 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 9) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f5a4c0073b0 con 0x7f5a58155890 2023-10-06T02:26:39.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.190+0000 7f5a3ffff700 1 --2- 172.21.15.31:0/2171420512 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5a440670e0 0x7f5a44069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:39.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.190+0000 7f5a3ffff700 1 -- 172.21.15.31:0/2171420512 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f5a4c084780 con 0x7f5a58155890 2023-10-06T02:26:39.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.190+0000 7f5a57fff700 1 --2- 172.21.15.31:0/2171420512 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5a440670e0 0x7f5a44069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:39.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.191+0000 7f5a57fff700 1 --2- 172.21.15.31:0/2171420512 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5a440670e0 0x7f5a44069590 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5a48002880 tx=0x7f5a48009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:39.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.194+0000 7f5a3ffff700 1 -- 172.21.15.31:0/2171420512 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5a4c055240 con 0x7f5a58155890 2023-10-06T02:26:39.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.397+0000 7f5a5d39d700 1 -- 172.21.15.31:0/2171420512 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5a5806f600 con 0x7f5a58155890 2023-10-06T02:26:39.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.397+0000 7f5a3ffff700 1 -- 172.21.15.31:0/2171420512 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v57) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5a4c054dd0 con 0x7f5a58155890 2023-10-06T02:26:39.397 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-10-06T02:26:39.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.399+0000 7f5a5d39d700 1 -- 172.21.15.31:0/2171420512 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5a440670e0 msgr2=0x7f5a44069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:39.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.399+0000 7f5a5d39d700 1 --2- 172.21.15.31:0/2171420512 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5a440670e0 0x7f5a44069590 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5a48002880 tx=0x7f5a48009040 comp rx=0 tx=0).stop 2023-10-06T02:26:39.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.399+0000 7f5a5d39d700 1 -- 172.21.15.31:0/2171420512 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a58155890 msgr2=0x7f5a58144b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:39.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.399+0000 7f5a5d39d700 1 --2- 172.21.15.31:0/2171420512 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a58155890 0x7f5a58144b50 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c00c850 tx=0x7f5a4c008040 comp rx=0 tx=0).stop 2023-10-06T02:26:39.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.399+0000 7f5a5d39d700 1 -- 172.21.15.31:0/2171420512 shutdown_connections 2023-10-06T02:26:39.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.399+0000 7f5a5d39d700 1 --2- 172.21.15.31:0/2171420512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a58155c60 0x7f5a5814d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:39.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.399+0000 7f5a5d39d700 1 --2- 172.21.15.31:0/2171420512 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5a440670e0 0x7f5a44069590 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:39.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.399+0000 7f5a5d39d700 1 --2- 172.21.15.31:0/2171420512 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5a5806de50 0x7f5a580c5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:39.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.399+0000 7f5a5d39d700 1 --2- 172.21.15.31:0/2171420512 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a58155890 0x7f5a58144b50 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:39.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.399+0000 7f5a5d39d700 1 -- 172.21.15.31:0/2171420512 >> 172.21.15.31:0/2171420512 conn(0x7f5a5806d110 msgr2=0x7f5a581588e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:39.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.399+0000 7f5a5d39d700 1 -- 172.21.15.31:0/2171420512 shutdown_connections 2023-10-06T02:26:39.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.399+0000 7f5a5d39d700 1 -- 172.21.15.31:0/2171420512 wait complete. 2023-10-06T02:26:39.417 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T02:26:39.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.610+0000 7f4d23fff700 1 Processor -- start 2023-10-06T02:26:39.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.610+0000 7f4d23fff700 1 -- start start 2023-10-06T02:26:39.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.611+0000 7f4d23fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4d24155c60 0x7f4d240c7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:39.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.611+0000 7f4d23fff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d240c7e70 con 0x7f4d24155890 2023-10-06T02:26:39.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.611+0000 7f4d23fff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d2406b790 con 0x7f4d2406de50 2023-10-06T02:26:39.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.611+0000 7f4d23fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d2406ab80 con 0x7f4d24155c60 2023-10-06T02:26:39.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.612+0000 7f4d22ffd700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f4d24155890 0x7f4d2406a3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46644/0 (socket says 172.21.15.31:46644) 2023-10-06T02:26:39.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.612+0000 7f4d22ffd700 1 -- 172.21.15.31:0/3774384710 learned_addr learned my addr 172.21.15.31:0/3774384710 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:26:39.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.612+0000 7f4d227fc700 1 --2- 172.21.15.31:0/3774384710 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4d24155c60 0x7f4d240c7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:39.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.612+0000 7f4d21ffb700 1 -- 172.21.15.31:0/3774384710 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3095849836 0 0) 0x7f4d240c7e70 con 0x7f4d24155890 2023-10-06T02:26:39.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.612+0000 7f4d21ffb700 1 -- 172.21.15.31:0/3774384710 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d0c004680 con 0x7f4d24155890 2023-10-06T02:26:39.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.612+0000 7f4d21ffb700 1 -- 172.21.15.31:0/3774384710 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 691311687 0 0) 0x7f4d0c004680 con 0x7f4d24155890 2023-10-06T02:26:39.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.612+0000 7f4d21ffb700 1 -- 172.21.15.31:0/3774384710 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4d24155c60 msgr2=0x7f4d240c7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:26:39.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.612+0000 7f4d21ffb700 1 --2- 172.21.15.31:0/3774384710 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4d24155c60 0x7f4d240c7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:39.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.612+0000 7f4d21ffb700 1 -- 172.21.15.31:0/3774384710 >> v1:172.21.15.62:6789/0 conn(0x7f4d2406de50 legacy=0x7f4d240c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:39.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.612+0000 7f4d21ffb700 1 -- 172.21.15.31:0/3774384710 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d2406ae60 con 0x7f4d24155890 2023-10-06T02:26:39.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.612+0000 7f4d21ffb700 1 -- 172.21.15.31:0/3774384710 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7f4d14001470 con 0x7f4d24155890 2023-10-06T02:26:39.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.612+0000 7f4d21ffb700 1 -- 172.21.15.31:0/3774384710 >> v1:172.21.15.31:6789/0 conn(0x7f4d24155890 legacy=0x7f4d2406a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:39.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.612+0000 7f4d21ffb700 1 --2- 172.21.15.31:0/3774384710 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4d24155c60 0x7f4d0c004ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:39.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.612+0000 7f4d21ffb700 1 --2- 172.21.15.31:0/3774384710 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d0c0050e0 0x7f4d0c005920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:39.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.613+0000 7f4d21ffb700 1 --2- 172.21.15.31:0/3774384710 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d0c005e60 0x7f4d0c0062d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:39.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.613+0000 7f4d21ffb700 1 -- 172.21.15.31:0/3774384710 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d2406ab80 con 0x7f4d0c0050e0 2023-10-06T02:26:39.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.613+0000 7f4d21ffb700 1 -- 172.21.15.31:0/3774384710 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0c003680 con 0x7f4d0c005e60 2023-10-06T02:26:39.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.613+0000 7f4d21ffb700 1 -- 172.21.15.31:0/3774384710 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0c003840 con 0x7f4d24155c60 2023-10-06T02:26:39.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.613+0000 7f4d227fc700 1 --2- 172.21.15.31:0/3774384710 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d0c0050e0 0x7f4d0c005920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:39.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.613+0000 7f4d22ffd700 1 --2- 172.21.15.31:0/3774384710 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4d24155c60 0x7f4d0c004ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:39.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.613+0000 7f4d237fe700 1 --2- 172.21.15.31:0/3774384710 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d0c005e60 0x7f4d0c0062d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:39.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.613+0000 7f4d227fc700 1 -- 172.21.15.31:0/3774384710 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4d24155c60 msgr2=0x7f4d0c004ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:26:39.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.613+0000 7f4d227fc700 1 --2- 172.21.15.31:0/3774384710 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4d24155c60 0x7f4d0c004ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:39.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.613+0000 7f4d227fc700 1 -- 172.21.15.31:0/3774384710 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d0c005e60 msgr2=0x7f4d0c0062d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:26:39.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.613+0000 7f4d227fc700 1 --2- 172.21.15.31:0/3774384710 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d0c005e60 0x7f4d0c0062d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:39.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.613+0000 7f4d227fc700 1 -- 172.21.15.31:0/3774384710 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d0c004680 con 0x7f4d0c0050e0 2023-10-06T02:26:39.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.613+0000 7f4d227fc700 1 --2- 172.21.15.31:0/3774384710 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d0c0050e0 0x7f4d0c005920 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f4d18009c00 tx=0x7f4d180091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79098db5e048dd02 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:39.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.614+0000 7f4d21ffb700 1 -- 172.21.15.31:0/3774384710 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d1800e710 con 0x7f4d0c0050e0 2023-10-06T02:26:39.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.614+0000 7f4d21ffb700 1 -- 172.21.15.31:0/3774384710 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d1800e870 con 0x7f4d0c0050e0 2023-10-06T02:26:39.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.614+0000 7f4d21ffb700 1 -- 172.21.15.31:0/3774384710 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d1800ec30 con 0x7f4d0c0050e0 2023-10-06T02:26:39.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.614+0000 7f4d23fff700 1 -- 172.21.15.31:0/3774384710 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d0c0050e0 msgr2=0x7f4d0c005920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:39.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.614+0000 7f4d23fff700 1 --2- 172.21.15.31:0/3774384710 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d0c0050e0 0x7f4d0c005920 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f4d18009c00 tx=0x7f4d180091a0 comp rx=0 tx=0).stop 2023-10-06T02:26:39.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.614+0000 7f4d237fe700 1 -- 172.21.15.31:0/3774384710 reap_dead start 2023-10-06T02:26:39.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.614+0000 7f4d23fff700 1 -- 172.21.15.31:0/3774384710 shutdown_connections 2023-10-06T02:26:39.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.614+0000 7f4d23fff700 1 -- 172.21.15.31:0/3774384710 >> 172.21.15.31:0/3774384710 conn(0x7f4d2406d110 msgr2=0x7f4d240cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:39.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.614+0000 7f4d23fff700 1 -- 172.21.15.31:0/3774384710 shutdown_connections 2023-10-06T02:26:39.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.614+0000 7f4d23fff700 1 -- 172.21.15.31:0/3774384710 wait complete. 2023-10-06T02:26:39.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.616+0000 7f4d23fff700 1 Processor -- start 2023-10-06T02:26:39.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.616+0000 7f4d23fff700 1 -- start start 2023-10-06T02:26:39.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.617+0000 7f4d23fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d0c0050e0 0x7f4d240b1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:39.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.617+0000 7f4d23fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4d240af8e0 0x7f4d241523e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:39.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.617+0000 7f4d23fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d240c5600 0x7f4d240c5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:39.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.617+0000 7f4d23fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d240c5fc0 con 0x7f4d240c5600 2023-10-06T02:26:39.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.617+0000 7f4d23fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d24152920 con 0x7f4d0c0050e0 2023-10-06T02:26:39.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.617+0000 7f4d23fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d241652a0 con 0x7f4d240af8e0 2023-10-06T02:26:39.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.617+0000 7f4d237fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d240c5600 0x7f4d240c5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:39.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.617+0000 7f4d237fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d240c5600 0x7f4d240c5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49826/0 (socket says 172.21.15.31:49826) 2023-10-06T02:26:39.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.617+0000 7f4d237fe700 1 -- 172.21.15.31:0/663749715 learned_addr learned my addr 172.21.15.31:0/663749715 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:26:39.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.617+0000 7f4d227fc700 1 --2- 172.21.15.31:0/663749715 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4d240af8e0 0x7f4d241523e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:39.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.617+0000 7f4d22ffd700 1 --2- 172.21.15.31:0/663749715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d0c0050e0 0x7f4d240b1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:39.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.617+0000 7f4d237fe700 1 -- 172.21.15.31:0/663749715 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4d240af8e0 msgr2=0x7f4d241523e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:39.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.617+0000 7f4d237fe700 1 --2- 172.21.15.31:0/663749715 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4d240af8e0 0x7f4d241523e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:39.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.617+0000 7f4d237fe700 1 -- 172.21.15.31:0/663749715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d0c0050e0 msgr2=0x7f4d240b1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:39.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.617+0000 7f4d237fe700 1 --2- 172.21.15.31:0/663749715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d0c0050e0 0x7f4d240b1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:39.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.617+0000 7f4d237fe700 1 -- 172.21.15.31:0/663749715 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d240afde0 con 0x7f4d240c5600 2023-10-06T02:26:39.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.617+0000 7f4d227fc700 1 --2- 172.21.15.31:0/663749715 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4d240af8e0 0x7f4d241523e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:26:39.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.618+0000 7f4d237fe700 1 --2- 172.21.15.31:0/663749715 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d240c5600 0x7f4d240c5a80 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f4d2406f140 tx=0x7f4d1c009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:39.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.618+0000 7f4d0affd700 1 -- 172.21.15.31:0/663749715 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d1c006db0 con 0x7f4d240c5600 2023-10-06T02:26:39.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.618+0000 7f4d0affd700 1 -- 172.21.15.31:0/663749715 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d1c003070 con 0x7f4d240c5600 2023-10-06T02:26:39.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.618+0000 7f4d0affd700 1 -- 172.21.15.31:0/663749715 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d1c003430 con 0x7f4d240c5600 2023-10-06T02:26:39.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.618+0000 7f4d23fff700 1 -- 172.21.15.31:0/663749715 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d240b00c0 con 0x7f4d240c5600 2023-10-06T02:26:39.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.618+0000 7f4d23fff700 1 -- 172.21.15.31:0/663749715 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d240b0680 con 0x7f4d240c5600 2023-10-06T02:26:39.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.619+0000 7f4d23fff700 1 -- 172.21.15.31:0/663749715 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d2404c3c0 con 0x7f4d240c5600 2023-10-06T02:26:39.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.620+0000 7f4d0affd700 1 -- 172.21.15.31:0/663749715 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 9) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f4d1c001850 con 0x7f4d240c5600 2023-10-06T02:26:39.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.621+0000 7f4d0affd700 1 --2- 172.21.15.31:0/663749715 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f4d100670e0 0x7f4d10069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:39.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.621+0000 7f4d0affd700 1 -- 172.21.15.31:0/663749715 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f4d1c083ef0 con 0x7f4d240c5600 2023-10-06T02:26:39.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.621+0000 7f4d22ffd700 1 --2- 172.21.15.31:0/663749715 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f4d100670e0 0x7f4d10069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:39.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.622+0000 7f4d22ffd700 1 --2- 172.21.15.31:0/663749715 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f4d100670e0 0x7f4d10069590 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f4d14001120 tx=0x7f4d14009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:39.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.625+0000 7f4d0affd700 1 -- 172.21.15.31:0/663749715 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4d1c04c5a0 con 0x7f4d240c5600 2023-10-06T02:26:39.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.830+0000 7f4d23fff700 1 -- 172.21.15.31:0/663749715 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f4d241653e0 con 0x7f4d240c5600 2023-10-06T02:26:39.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.830+0000 7f4d0affd700 1 -- 172.21.15.31:0/663749715 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v57) v1 ==== 92+0+10 (secure 0 0 0) 0x7f4d1c051030 con 0x7f4d240c5600 2023-10-06T02:26:39.831 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 8 2023-10-06T02:26:39.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.832+0000 7f4d23fff700 1 -- 172.21.15.31:0/663749715 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f4d100670e0 msgr2=0x7f4d10069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:39.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.832+0000 7f4d23fff700 1 --2- 172.21.15.31:0/663749715 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f4d100670e0 0x7f4d10069590 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f4d14001120 tx=0x7f4d14009040 comp rx=0 tx=0).stop 2023-10-06T02:26:39.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.832+0000 7f4d23fff700 1 -- 172.21.15.31:0/663749715 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d240c5600 msgr2=0x7f4d240c5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:39.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.832+0000 7f4d23fff700 1 --2- 172.21.15.31:0/663749715 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d240c5600 0x7f4d240c5a80 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f4d2406f140 tx=0x7f4d1c009040 comp rx=0 tx=0).stop 2023-10-06T02:26:39.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.832+0000 7f4d23fff700 1 -- 172.21.15.31:0/663749715 shutdown_connections 2023-10-06T02:26:39.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.832+0000 7f4d23fff700 1 --2- 172.21.15.31:0/663749715 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d240c5600 0x7f4d240c5a80 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:39.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.832+0000 7f4d23fff700 1 --2- 172.21.15.31:0/663749715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d0c0050e0 0x7f4d240b1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:39.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.832+0000 7f4d23fff700 1 --2- 172.21.15.31:0/663749715 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f4d100670e0 0x7f4d10069590 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:39.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.832+0000 7f4d23fff700 1 --2- 172.21.15.31:0/663749715 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4d240af8e0 0x7f4d241523e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:39.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.832+0000 7f4d23fff700 1 -- 172.21.15.31:0/663749715 >> 172.21.15.31:0/663749715 conn(0x7f4d2406d110 msgr2=0x7f4d240c9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:39.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.833+0000 7f4d23fff700 1 -- 172.21.15.31:0/663749715 shutdown_connections 2023-10-06T02:26:39.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:39.833+0000 7f4d23fff700 1 -- 172.21.15.31:0/663749715 wait complete. 2023-10-06T02:26:39.850 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T02:26:40.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.038+0000 7f918e19d700 1 Processor -- start 2023-10-06T02:26:40.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.038+0000 7f918e19d700 1 -- start start 2023-10-06T02:26:40.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.039+0000 7f918e19d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f91880b2940 0x7f91880a98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:40.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.039+0000 7f918e19d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91880b0110 con 0x7f91880cd9d0 2023-10-06T02:26:40.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.040+0000 7f918e19d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91880b0920 con 0x7f91880b2d10 2023-10-06T02:26:40.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.040+0000 7f918e19d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f918806e730 con 0x7f91880b2940 2023-10-06T02:26:40.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.040+0000 7f918d19b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f91880b2940 0x7f91880a98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:40.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.040+0000 7f918c99a700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f91880cd9d0 0x7f918806e020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46652/0 (socket says 172.21.15.31:46652) 2023-10-06T02:26:40.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.040+0000 7f918c99a700 1 -- 172.21.15.31:0/39777975 learned_addr learned my addr 172.21.15.31:0/39777975 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:26:40.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.040+0000 7f918d19b700 1 -- 172.21.15.31:0/39777975 >> v1:172.21.15.62:6789/0 conn(0x7f91880b2d10 legacy=0x7f9188159250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:40.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.040+0000 7f918d19b700 1 -- 172.21.15.31:0/39777975 >> v1:172.21.15.31:6789/0 conn(0x7f91880cd9d0 legacy=0x7f918806e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:40.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.040+0000 7f918d19b700 1 -- 172.21.15.31:0/39777975 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f918806cc70 con 0x7f91880b2940 2023-10-06T02:26:40.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.041+0000 7f918d19b700 1 --2- 172.21.15.31:0/39777975 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91880b2940 0x7f91880a98e0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f91840098d0 tx=0x7f9184009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eba658623d64216 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:40.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.041+0000 7f917ffff700 1 -- 172.21.15.31:0/39777975 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9184009e70 con 0x7f91880b2940 2023-10-06T02:26:40.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.041+0000 7f917ffff700 1 -- 172.21.15.31:0/39777975 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9184005c10 con 0x7f91880b2940 2023-10-06T02:26:40.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.041+0000 7f917ffff700 1 -- 172.21.15.31:0/39777975 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f918400b650 con 0x7f91880b2940 2023-10-06T02:26:40.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.042+0000 7f918e19d700 1 -- 172.21.15.31:0/39777975 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91880b2940 msgr2=0x7f91880a98e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:40.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.042+0000 7f918e19d700 1 --2- 172.21.15.31:0/39777975 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91880b2940 0x7f91880a98e0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f91840098d0 tx=0x7f9184009120 comp rx=0 tx=0).stop 2023-10-06T02:26:40.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.042+0000 7f918e19d700 1 -- 172.21.15.31:0/39777975 shutdown_connections 2023-10-06T02:26:40.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.042+0000 7f918e19d700 1 --2- 172.21.15.31:0/39777975 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91880b2940 0x7f91880a98e0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:40.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.042+0000 7f918e19d700 1 -- 172.21.15.31:0/39777975 >> 172.21.15.31:0/39777975 conn(0x7f91880675a0 msgr2=0x7f91880679a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:40.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.042+0000 7f918e19d700 1 -- 172.21.15.31:0/39777975 shutdown_connections 2023-10-06T02:26:40.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.042+0000 7f918e19d700 1 -- 172.21.15.31:0/39777975 wait complete. 2023-10-06T02:26:40.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.043+0000 7f918e19d700 1 Processor -- start 2023-10-06T02:26:40.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.044+0000 7f918e19d700 1 -- start start 2023-10-06T02:26:40.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.044+0000 7f918e19d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91880b2940 0x7f91881403e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:40.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.044+0000 7f918e19d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f91880b2d10 0x7f9188156880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:40.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.045+0000 7f918e19d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91880cd9d0 0x7f9188156dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:40.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.045+0000 7f918e19d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91881531d0 con 0x7f91880cd9d0 2023-10-06T02:26:40.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.045+0000 7f918e19d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f918816a7b0 con 0x7f91880b2940 2023-10-06T02:26:40.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.045+0000 7f918e19d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f918816aa90 con 0x7f91880b2d10 2023-10-06T02:26:40.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.045+0000 7f918d99c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91880cd9d0 0x7f9188156dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:40.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.045+0000 7f918d99c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91880cd9d0 0x7f9188156dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49838/0 (socket says 172.21.15.31:49838) 2023-10-06T02:26:40.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.045+0000 7f918d99c700 1 -- 172.21.15.31:0/702020402 learned_addr learned my addr 172.21.15.31:0/702020402 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:26:40.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.045+0000 7f918d19b700 1 --2- 172.21.15.31:0/702020402 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91880b2940 0x7f91881403e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:40.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.045+0000 7f918c99a700 1 --2- 172.21.15.31:0/702020402 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f91880b2d10 0x7f9188156880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:40.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.045+0000 7f918d99c700 1 -- 172.21.15.31:0/702020402 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f91880b2d10 msgr2=0x7f9188156880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:40.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.045+0000 7f918d99c700 1 --2- 172.21.15.31:0/702020402 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f91880b2d10 0x7f9188156880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:40.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.045+0000 7f918d99c700 1 -- 172.21.15.31:0/702020402 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91880b2940 msgr2=0x7f91881403e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:40.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.045+0000 7f918d99c700 1 --2- 172.21.15.31:0/702020402 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91880b2940 0x7f91881403e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:40.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.045+0000 7f918d99c700 1 -- 172.21.15.31:0/702020402 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91840093c0 con 0x7f91880cd9d0 2023-10-06T02:26:40.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.046+0000 7f918d99c700 1 --2- 172.21.15.31:0/702020402 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91880cd9d0 0x7f9188156dc0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f918000c8e0 tx=0x7f9180001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:40.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.046+0000 7f917d7fa700 1 -- 172.21.15.31:0/702020402 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9180012630 con 0x7f91880cd9d0 2023-10-06T02:26:40.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.046+0000 7f917d7fa700 1 -- 172.21.15.31:0/702020402 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9180012790 con 0x7f91880cd9d0 2023-10-06T02:26:40.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.046+0000 7f918e19d700 1 -- 172.21.15.31:0/702020402 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f918816adf0 con 0x7f91880cd9d0 2023-10-06T02:26:40.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.046+0000 7f917d7fa700 1 -- 172.21.15.31:0/702020402 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9180012b50 con 0x7f91880cd9d0 2023-10-06T02:26:40.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.046+0000 7f918e19d700 1 -- 172.21.15.31:0/702020402 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f918816b3b0 con 0x7f91880cd9d0 2023-10-06T02:26:40.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.047+0000 7f918e19d700 1 -- 172.21.15.31:0/702020402 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f918804c3c0 con 0x7f91880cd9d0 2023-10-06T02:26:40.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.048+0000 7f917d7fa700 1 -- 172.21.15.31:0/702020402 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 9) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f918001a3a0 con 0x7f91880cd9d0 2023-10-06T02:26:40.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.049+0000 7f917d7fa700 1 --2- 172.21.15.31:0/702020402 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f916c0670e0 0x7f916c069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:40.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.049+0000 7f917d7fa700 1 -- 172.21.15.31:0/702020402 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f9180084860 con 0x7f91880cd9d0 2023-10-06T02:26:40.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.049+0000 7f918d19b700 1 --2- 172.21.15.31:0/702020402 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f916c0670e0 0x7f916c069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:40.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.050+0000 7f918d19b700 1 --2- 172.21.15.31:0/702020402 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f916c0670e0 0x7f916c069590 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9184009160 tx=0x7f9184016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:40.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.053+0000 7f917d7fa700 1 -- 172.21.15.31:0/702020402 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9180055430 con 0x7f91880cd9d0 2023-10-06T02:26:40.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.261+0000 7f918e19d700 1 -- 172.21.15.31:0/702020402 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f91880b0360 con 0x7f91880cd9d0 2023-10-06T02:26:40.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.262+0000 7f917d7fa700 1 -- 172.21.15.31:0/702020402 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v57) v1 ==== 84+0+126 (secure 0 0 0) 0x7f9180054fc0 con 0x7f91880cd9d0 2023-10-06T02:26:40.262 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:26:40.262 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2023-10-06T02:26:40.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:26:40.262 INFO:teuthology.orchestra.run.smithi031.stdout: "poolnum": 1, 2023-10-06T02:26:40.262 INFO:teuthology.orchestra.run.smithi031.stdout: "poolname": ".mgr" 2023-10-06T02:26:40.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:26:40.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:26:40.263 INFO:teuthology.orchestra.run.smithi031.stdout: "poolnum": 2, 2023-10-06T02:26:40.263 INFO:teuthology.orchestra.run.smithi031.stdout: "poolname": "rbd" 2023-10-06T02:26:40.263 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:26:40.263 INFO:teuthology.orchestra.run.smithi031.stdout:] 2023-10-06T02:26:40.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.264+0000 7f918e19d700 1 -- 172.21.15.31:0/702020402 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f916c0670e0 msgr2=0x7f916c069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:40.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.264+0000 7f918e19d700 1 --2- 172.21.15.31:0/702020402 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f916c0670e0 0x7f916c069590 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9184009160 tx=0x7f9184016000 comp rx=0 tx=0).stop 2023-10-06T02:26:40.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.264+0000 7f918e19d700 1 -- 172.21.15.31:0/702020402 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91880cd9d0 msgr2=0x7f9188156dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:40.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.264+0000 7f918e19d700 1 --2- 172.21.15.31:0/702020402 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91880cd9d0 0x7f9188156dc0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f918000c8e0 tx=0x7f9180001710 comp rx=0 tx=0).stop 2023-10-06T02:26:40.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.264+0000 7f918e19d700 1 -- 172.21.15.31:0/702020402 shutdown_connections 2023-10-06T02:26:40.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.264+0000 7f918e19d700 1 --2- 172.21.15.31:0/702020402 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91880cd9d0 0x7f9188156dc0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:40.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.264+0000 7f918e19d700 1 --2- 172.21.15.31:0/702020402 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91880b2940 0x7f91881403e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:40.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.264+0000 7f918e19d700 1 --2- 172.21.15.31:0/702020402 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f916c0670e0 0x7f916c069590 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:40.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.264+0000 7f918e19d700 1 --2- 172.21.15.31:0/702020402 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f91880b2d10 0x7f9188156880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:40.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.264+0000 7f918e19d700 1 -- 172.21.15.31:0/702020402 >> 172.21.15.31:0/702020402 conn(0x7f91880675a0 msgr2=0x7f91881557e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:40.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.264+0000 7f918e19d700 1 -- 172.21.15.31:0/702020402 shutdown_connections 2023-10-06T02:26:40.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.264+0000 7f918e19d700 1 -- 172.21.15.31:0/702020402 wait complete. 2023-10-06T02:26:40.282 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T02:26:40.282 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T02:26:40.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.472+0000 7f1f33fff700 1 Processor -- start 2023-10-06T02:26:40.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.472+0000 7f1f33fff700 1 -- start start 2023-10-06T02:26:40.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.473+0000 7f1f33fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1f3415cb90 0x7f1f340c7870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:40.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.473+0000 7f1f33fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1f3415dc10 0x7f1f3406c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:40.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.473+0000 7f1f33fff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f3406d040 con 0x7f1f3406cbe0 2023-10-06T02:26:40.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.473+0000 7f1f33fff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f340ccdf0 con 0x7f1f3415cb90 2023-10-06T02:26:40.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.473+0000 7f1f33fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f340c9b20 con 0x7f1f3415dc10 2023-10-06T02:26:40.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.473+0000 7f1f32ffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1f3415cb90 0x7f1f340c7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:40.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.474+0000 7f1f32ffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1f3415cb90 0x7f1f340c7870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49848/0 (socket says 172.21.15.31:49848) 2023-10-06T02:26:40.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.474+0000 7f1f32ffd700 1 -- 172.21.15.31:0/1659117786 learned_addr learned my addr 172.21.15.31:0/1659117786 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:26:40.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.474+0000 7f1f327fc700 1 --2- 172.21.15.31:0/1659117786 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1f3415dc10 0x7f1f3406c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:40.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.474+0000 7f1f32ffd700 1 -- 172.21.15.31:0/1659117786 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1f3415dc10 msgr2=0x7f1f3406c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:26:40.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.474+0000 7f1f32ffd700 1 --2- 172.21.15.31:0/1659117786 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1f3415dc10 0x7f1f3406c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:40.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.474+0000 7f1f32ffd700 1 -- 172.21.15.31:0/1659117786 >> v1:172.21.15.62:6789/0 conn(0x7f1f3406cbe0 legacy=0x7f1f340c3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:40.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.474+0000 7f1f32ffd700 1 -- 172.21.15.31:0/1659117786 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f340c2fa0 con 0x7f1f3415cb90 2023-10-06T02:26:40.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.474+0000 7f1f327fc700 1 --2- 172.21.15.31:0/1659117786 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1f3415dc10 0x7f1f3406c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:26:40.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.474+0000 7f1f32ffd700 1 --2- 172.21.15.31:0/1659117786 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f3415cb90 0x7f1f340c7870 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f1f240099b0 tx=0x7f1f24009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=277440b864b9ca9a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:40.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.474+0000 7f1f31ffb700 1 -- 172.21.15.31:0/1659117786 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f2400b860 con 0x7f1f3415cb90 2023-10-06T02:26:40.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.475+0000 7f1f31ffb700 1 -- 172.21.15.31:0/1659117786 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f2400b9c0 con 0x7f1f3415cb90 2023-10-06T02:26:40.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.475+0000 7f1f31ffb700 1 -- 172.21.15.31:0/1659117786 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f2400bd60 con 0x7f1f3415cb90 2023-10-06T02:26:40.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.475+0000 7f1f33fff700 1 -- 172.21.15.31:0/1659117786 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f3415cb90 msgr2=0x7f1f340c7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:40.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.475+0000 7f1f33fff700 1 --2- 172.21.15.31:0/1659117786 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f3415cb90 0x7f1f340c7870 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f1f240099b0 tx=0x7f1f24009250 comp rx=0 tx=0).stop 2023-10-06T02:26:40.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.475+0000 7f1f33fff700 1 -- 172.21.15.31:0/1659117786 shutdown_connections 2023-10-06T02:26:40.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.475+0000 7f1f33fff700 1 --2- 172.21.15.31:0/1659117786 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f3415cb90 0x7f1f340c7870 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:40.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.475+0000 7f1f33fff700 1 --2- 172.21.15.31:0/1659117786 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1f3415dc10 0x7f1f3406c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:40.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.475+0000 7f1f33fff700 1 -- 172.21.15.31:0/1659117786 >> 172.21.15.31:0/1659117786 conn(0x7f1f340cd230 msgr2=0x7f1f340cd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:40.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.475+0000 7f1f33fff700 1 -- 172.21.15.31:0/1659117786 shutdown_connections 2023-10-06T02:26:40.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.475+0000 7f1f33fff700 1 -- 172.21.15.31:0/1659117786 wait complete. 2023-10-06T02:26:40.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.477+0000 7f1f33fff700 1 Processor -- start 2023-10-06T02:26:40.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.477+0000 7f1f33fff700 1 -- start start 2023-10-06T02:26:40.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.478+0000 7f1f33fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f3406cbe0 0x7f1f3415b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:40.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.478+0000 7f1f33fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1f3415cb90 0x7f1f340c7e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:40.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.478+0000 7f1f33fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f3415dc10 0x7f1f340c83d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:40.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.478+0000 7f1f33fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f340c6970 con 0x7f1f3406cbe0 2023-10-06T02:26:40.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.478+0000 7f1f33fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f340c64e0 con 0x7f1f3415dc10 2023-10-06T02:26:40.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.478+0000 7f1f33fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f340c58d0 con 0x7f1f3415cb90 2023-10-06T02:26:40.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.478+0000 7f1f32ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f3406cbe0 0x7f1f3415b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:40.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.478+0000 7f1f32ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f3406cbe0 0x7f1f3415b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49862/0 (socket says 172.21.15.31:49862) 2023-10-06T02:26:40.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.478+0000 7f1f32ffd700 1 -- 172.21.15.31:0/4210792248 learned_addr learned my addr 172.21.15.31:0/4210792248 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:26:40.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.478+0000 7f1f327fc700 1 --2- 172.21.15.31:0/4210792248 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1f3415cb90 0x7f1f340c7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:40.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.478+0000 7f1f337fe700 1 --2- 172.21.15.31:0/4210792248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f3415dc10 0x7f1f340c83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:40.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.478+0000 7f1f32ffd700 1 -- 172.21.15.31:0/4210792248 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1f3415cb90 msgr2=0x7f1f340c7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:40.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.478+0000 7f1f32ffd700 1 --2- 172.21.15.31:0/4210792248 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1f3415cb90 0x7f1f340c7e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:40.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.478+0000 7f1f32ffd700 1 -- 172.21.15.31:0/4210792248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f3415dc10 msgr2=0x7f1f340c83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:40.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.478+0000 7f1f32ffd700 1 --2- 172.21.15.31:0/4210792248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f3415dc10 0x7f1f340c83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:40.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.478+0000 7f1f32ffd700 1 -- 172.21.15.31:0/4210792248 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f240094f0 con 0x7f1f3406cbe0 2023-10-06T02:26:40.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.479+0000 7f1f32ffd700 1 --2- 172.21.15.31:0/4210792248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f3406cbe0 0x7f1f3415b3c0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f1f24009290 tx=0x7f1f24017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:40.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.479+0000 7f1f1affd700 1 -- 172.21.15.31:0/4210792248 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f24011e00 con 0x7f1f3406cbe0 2023-10-06T02:26:40.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.479+0000 7f1f1affd700 1 -- 172.21.15.31:0/4210792248 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f2401f070 con 0x7f1f3406cbe0 2023-10-06T02:26:40.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.479+0000 7f1f1affd700 1 -- 172.21.15.31:0/4210792248 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f240266d0 con 0x7f1f3406cbe0 2023-10-06T02:26:40.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.479+0000 7f1f33fff700 1 -- 172.21.15.31:0/4210792248 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f340c5bb0 con 0x7f1f3406cbe0 2023-10-06T02:26:40.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.479+0000 7f1f33fff700 1 -- 172.21.15.31:0/4210792248 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f340c6170 con 0x7f1f3406cbe0 2023-10-06T02:26:40.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.480+0000 7f1f33fff700 1 -- 172.21.15.31:0/4210792248 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f340c67c0 con 0x7f1f3406cbe0 2023-10-06T02:26:40.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.481+0000 7f1f1affd700 1 -- 172.21.15.31:0/4210792248 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 9) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f1f24026a40 con 0x7f1f3406cbe0 2023-10-06T02:26:40.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.482+0000 7f1f1affd700 1 --2- 172.21.15.31:0/4210792248 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f1f200670e0 0x7f1f20069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:40.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.482+0000 7f1f1affd700 1 -- 172.21.15.31:0/4210792248 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f1f24021070 con 0x7f1f3406cbe0 2023-10-06T02:26:40.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.483+0000 7f1f327fc700 1 --2- 172.21.15.31:0/4210792248 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f1f200670e0 0x7f1f20069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:40.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.483+0000 7f1f327fc700 1 --2- 172.21.15.31:0/4210792248 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f1f200670e0 0x7f1f20069590 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f1f3414a400 tx=0x7f1f28008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:40.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.487+0000 7f1f1affd700 1 -- 172.21.15.31:0/4210792248 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1f24053510 con 0x7f1f3406cbe0 2023-10-06T02:26:40.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:40.688+0000 7f1f33fff700 1 -- 172.21.15.31:0/4210792248 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f1f3414b6b0 con 0x7f1f3406cbe0 2023-10-06T02:26:41.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:41.047+0000 7f1f1affd700 1 -- 172.21.15.31:0/4210792248 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v58) v1 ==== 138+0+0 (secure 0 0 0) 0x7f1f24084e60 con 0x7f1f3406cbe0 2023-10-06T02:26:41.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:41.050+0000 7f1f33fff700 1 -- 172.21.15.31:0/4210792248 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f1f200670e0 msgr2=0x7f1f20069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:41.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:41.050+0000 7f1f33fff700 1 --2- 172.21.15.31:0/4210792248 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f1f200670e0 0x7f1f20069590 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f1f3414a400 tx=0x7f1f28008040 comp rx=0 tx=0).stop 2023-10-06T02:26:41.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:41.050+0000 7f1f33fff700 1 -- 172.21.15.31:0/4210792248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f3406cbe0 msgr2=0x7f1f3415b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:41.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:41.050+0000 7f1f33fff700 1 --2- 172.21.15.31:0/4210792248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f3406cbe0 0x7f1f3415b3c0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f1f24009290 tx=0x7f1f24017000 comp rx=0 tx=0).stop 2023-10-06T02:26:41.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:41.050+0000 7f1f33fff700 1 -- 172.21.15.31:0/4210792248 shutdown_connections 2023-10-06T02:26:41.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:41.050+0000 7f1f33fff700 1 --2- 172.21.15.31:0/4210792248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f3415dc10 0x7f1f340c83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:41.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:41.050+0000 7f1f33fff700 1 --2- 172.21.15.31:0/4210792248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f3406cbe0 0x7f1f3415b3c0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:41.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:41.050+0000 7f1f33fff700 1 --2- 172.21.15.31:0/4210792248 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f1f200670e0 0x7f1f20069590 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:41.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:41.050+0000 7f1f33fff700 1 --2- 172.21.15.31:0/4210792248 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1f3415cb90 0x7f1f340c7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:41.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:41.050+0000 7f1f33fff700 1 -- 172.21.15.31:0/4210792248 >> 172.21.15.31:0/4210792248 conn(0x7f1f340cd230 msgr2=0x7f1f340b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:41.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:41.051+0000 7f1f33fff700 1 -- 172.21.15.31:0/4210792248 shutdown_connections 2023-10-06T02:26:41.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:41.051+0000 7f1f33fff700 1 -- 172.21.15.31:0/4210792248 wait complete. 2023-10-06T02:26:41.052 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' created 2023-10-06T02:26:41.073 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T02:26:41.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:41.266+0000 7fd0affff700 1 Processor -- start 2023-10-06T02:26:41.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:41.267+0000 7fd0affff700 1 -- start start 2023-10-06T02:26:41.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:41.268+0000 7fd0affff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd0b013fef0 0x7fd0b015a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:41.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:41.268+0000 7fd0affff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd0b006aaa0 0x7fd0b00c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:41.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:41.268+0000 7fd0affff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0b015ae70 con 0x7fd0b013fb20 2023-10-06T02:26:41.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:41.268+0000 7fd0affff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0b006af60 con 0x7fd0b006aaa0 2023-10-06T02:26:41.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:41.268+0000 7fd0affff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b006b240 con 0x7fd0b013fef0 2023-10-06T02:26:41.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:41.268+0000 7fd0aeffd700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fd0b013fb20 0x7fd0b00c7990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46656/0 (socket says 172.21.15.31:46656) 2023-10-06T02:26:41.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:41.268+0000 7fd0aeffd700 1 -- 172.21.15.31:0/270229569 learned_addr learned my addr 172.21.15.31:0/270229569 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:26:41.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:41.268+0000 7fd0af7fe700 1 --2- 172.21.15.31:0/270229569 >> v2:172.21.15.31:3300/0 conn(0x7fd0b006aaa0 0x7fd0b00c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:41.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:41.268+0000 7fd0ae7fc700 1 --2- 172.21.15.31:0/270229569 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd0b013fef0 0x7fd0b015a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:41.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:41.269+0000 7fd0adffb700 1 -- 172.21.15.31:0/270229569 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1958229990 0 0) 0x7fd0a0001110 con 0x7fd0b013fb20 2023-10-06T02:26:41.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:41.269+0000 7fd0af7fe700 1 -- 172.21.15.31:0/270229569 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd0b013fef0 msgr2=0x7fd0b015a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:26:41.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:41.269+0000 7fd0af7fe700 1 --2- 172.21.15.31:0/270229569 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd0b013fef0 0x7fd0b015a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:41.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:41.269+0000 7fd0af7fe700 1 -- 172.21.15.31:0/270229569 >> v1:172.21.15.31:6789/0 conn(0x7fd0b013fb20 legacy=0x7fd0b00c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:41.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:41.269+0000 7fd0af7fe700 1 -- 172.21.15.31:0/270229569 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0b006b520 con 0x7fd0b006aaa0 2023-10-06T02:26:41.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:41.269+0000 7fd0ae7fc700 1 --2- 172.21.15.31:0/270229569 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd0b013fef0 0x7fd0b015a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:26:41.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:41.269+0000 7fd0af7fe700 1 --2- 172.21.15.31:0/270229569 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0b006aaa0 0x7fd0b00c9d00 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fd0a8009e60 tx=0x7fd0a8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfc78983ab0f7021 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:41.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:41.269+0000 7fd0adffb700 1 -- 172.21.15.31:0/270229569 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a800e070 con 0x7fd0b006aaa0 2023-10-06T02:26:41.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:41.269+0000 7fd0adffb700 1 -- 172.21.15.31:0/270229569 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0a800bc30 con 0x7fd0b006aaa0 2023-10-06T02:26:41.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:41.269+0000 7fd0adffb700 1 -- 172.21.15.31:0/270229569 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a8012430 con 0x7fd0b006aaa0 2023-10-06T02:26:41.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:41.270+0000 7fd0affff700 1 -- 172.21.15.31:0/270229569 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0b006aaa0 msgr2=0x7fd0b00c9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:41.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:41.270+0000 7fd0affff700 1 --2- 172.21.15.31:0/270229569 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0b006aaa0 0x7fd0b00c9d00 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fd0a8009e60 tx=0x7fd0a8009780 comp rx=0 tx=0).stop 2023-10-06T02:26:41.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:41.270+0000 7fd0affff700 1 -- 172.21.15.31:0/270229569 shutdown_connections 2023-10-06T02:26:41.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:41.270+0000 7fd0affff700 1 --2- 172.21.15.31:0/270229569 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0b006aaa0 0x7fd0b00c9d00 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:41.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:41.270+0000 7fd0affff700 1 --2- 172.21.15.31:0/270229569 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd0b013fef0 0x7fd0b015a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:41.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:41.270+0000 7fd0affff700 1 -- 172.21.15.31:0/270229569 >> 172.21.15.31:0/270229569 conn(0x7fd0b006d170 msgr2=0x7fd0b00cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:41.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:41.270+0000 7fd0affff700 1 -- 172.21.15.31:0/270229569 shutdown_connections 2023-10-06T02:26:41.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:41.270+0000 7fd0affff700 1 -- 172.21.15.31:0/270229569 wait complete. 2023-10-06T02:26:41.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:41.271+0000 7fd0affff700 1 Processor -- start 2023-10-06T02:26:41.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:41.272+0000 7fd0affff700 1 -- start start 2023-10-06T02:26:41.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:41.272+0000 7fd0affff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd0b006aaa0 0x7fd0b016b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:41.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:41.272+0000 7fd0affff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0b013fb20 0x7fd0b0155c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:41.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:41.272+0000 7fd0affff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0b013fef0 0x7fd0b00b0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:41.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:41.273+0000 7fd0affff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b00acce0 con 0x7fd0b013fb20 2023-10-06T02:26:41.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:41.273+0000 7fd0affff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b00acf80 con 0x7fd0b013fef0 2023-10-06T02:26:41.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:41.273+0000 7fd0affff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b00ad260 con 0x7fd0b006aaa0 2023-10-06T02:26:41.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:41.273+0000 7fd0ae7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0b013fb20 0x7fd0b0155c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:41.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:41.273+0000 7fd0ae7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0b013fb20 0x7fd0b0155c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49872/0 (socket says 172.21.15.31:49872) 2023-10-06T02:26:41.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:41.273+0000 7fd0ae7fc700 1 -- 172.21.15.31:0/1707817821 learned_addr learned my addr 172.21.15.31:0/1707817821 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:26:41.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:41.273+0000 7fd0aeffd700 1 --2- 172.21.15.31:0/1707817821 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd0b006aaa0 0x7fd0b016b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:41.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:41.273+0000 7fd0af7fe700 1 --2- 172.21.15.31:0/1707817821 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0b013fef0 0x7fd0b00b0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:41.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:41.273+0000 7fd0ae7fc700 1 -- 172.21.15.31:0/1707817821 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd0b006aaa0 msgr2=0x7fd0b016b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:41.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:41.273+0000 7fd0ae7fc700 1 --2- 172.21.15.31:0/1707817821 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd0b006aaa0 0x7fd0b016b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:41.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:41.273+0000 7fd0ae7fc700 1 -- 172.21.15.31:0/1707817821 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0b013fef0 msgr2=0x7fd0b00b0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:41.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:41.273+0000 7fd0ae7fc700 1 --2- 172.21.15.31:0/1707817821 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0b013fef0 0x7fd0b00b0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:41.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:41.273+0000 7fd0ae7fc700 1 -- 172.21.15.31:0/1707817821 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0a8009a20 con 0x7fd0b013fb20 2023-10-06T02:26:41.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:41.274+0000 7fd0ae7fc700 1 --2- 172.21.15.31:0/1707817821 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0b013fb20 0x7fd0b0155c80 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fd0a400e8f0 tx=0x7fd0a4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:41.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:41.274+0000 7fd096ffd700 1 -- 172.21.15.31:0/1707817821 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a4005a00 con 0x7fd0b013fb20 2023-10-06T02:26:41.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:41.274+0000 7fd096ffd700 1 -- 172.21.15.31:0/1707817821 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0a4005b60 con 0x7fd0b013fb20 2023-10-06T02:26:41.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:41.274+0000 7fd096ffd700 1 -- 172.21.15.31:0/1707817821 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a4006020 con 0x7fd0b013fb20 2023-10-06T02:26:41.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:41.274+0000 7fd0affff700 1 -- 172.21.15.31:0/1707817821 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0b00ad540 con 0x7fd0b013fb20 2023-10-06T02:26:41.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:41.274+0000 7fd0affff700 1 -- 172.21.15.31:0/1707817821 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0b01561c0 con 0x7fd0b013fb20 2023-10-06T02:26:41.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:41.275+0000 7fd0affff700 1 -- 172.21.15.31:0/1707817821 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0b00ada60 con 0x7fd0b013fb20 2023-10-06T02:26:41.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:41.276+0000 7fd096ffd700 1 -- 172.21.15.31:0/1707817821 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 9) v1 ==== 84893+0+0 (secure 0 0 0) 0x7fd0a4006410 con 0x7fd0b013fb20 2023-10-06T02:26:41.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:41.277+0000 7fd096ffd700 1 --2- 172.21.15.31:0/1707817821 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fd09c0670e0 0x7fd09c069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:41.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:41.277+0000 7fd096ffd700 1 -- 172.21.15.31:0/1707817821 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5576+0+0 (secure 0 0 0) 0x7fd0a40849a0 con 0x7fd0b013fb20 2023-10-06T02:26:41.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:41.277+0000 7fd0aeffd700 1 --2- 172.21.15.31:0/1707817821 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fd09c0670e0 0x7fd09c069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:41.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:41.278+0000 7fd0aeffd700 1 --2- 172.21.15.31:0/1707817821 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fd09c0670e0 0x7fd09c069590 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd0b00c24d0 tx=0x7fd0a0007b90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:41.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:41.281+0000 7fd096ffd700 1 -- 172.21.15.31:0/1707817821 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd0b00ada60 con 0x7fd0b013fb20 2023-10-06T02:26:41.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:41.484+0000 7fd0affff700 1 -- 172.21.15.31:0/1707817821 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fd0b00ada60 con 0x7fd0b013fb20 2023-10-06T02:26:42.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:42.057+0000 7fd096ffd700 1 -- 172.21.15.31:0/1707817821 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v59) v1 ==== 170+0+0 (secure 0 0 0) 0x7fd0b00ada60 con 0x7fd0b013fb20 2023-10-06T02:26:42.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:42.063+0000 7fd0affff700 1 -- 172.21.15.31:0/1707817821 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fd09c0670e0 msgr2=0x7fd09c069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:42.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:42.063+0000 7fd0affff700 1 --2- 172.21.15.31:0/1707817821 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fd09c0670e0 0x7fd09c069590 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd0b00c24d0 tx=0x7fd0a0007b90 comp rx=0 tx=0).stop 2023-10-06T02:26:42.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:42.063+0000 7fd0affff700 1 -- 172.21.15.31:0/1707817821 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0b013fb20 msgr2=0x7fd0b0155c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:42.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:42.063+0000 7fd0affff700 1 --2- 172.21.15.31:0/1707817821 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0b013fb20 0x7fd0b0155c80 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fd0a400e8f0 tx=0x7fd0a4008040 comp rx=0 tx=0).stop 2023-10-06T02:26:42.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:42.063+0000 7fd0affff700 1 -- 172.21.15.31:0/1707817821 shutdown_connections 2023-10-06T02:26:42.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:42.063+0000 7fd0affff700 1 --2- 172.21.15.31:0/1707817821 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0b013fef0 0x7fd0b00b0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:42.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:42.063+0000 7fd0affff700 1 --2- 172.21.15.31:0/1707817821 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fd09c0670e0 0x7fd09c069590 secure :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd0b00c24d0 tx=0x7fd0a0007b90 comp rx=0 tx=0).stop 2023-10-06T02:26:42.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:42.063+0000 7fd0affff700 1 --2- 172.21.15.31:0/1707817821 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd0b006aaa0 0x7fd0b016b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:42.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:42.063+0000 7fd0affff700 1 --2- 172.21.15.31:0/1707817821 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0b013fb20 0x7fd0b0155c80 secure :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fd0a400e8f0 tx=0x7fd0a4008040 comp rx=0 tx=0).stop 2023-10-06T02:26:42.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:42.063+0000 7fd0affff700 1 -- 172.21.15.31:0/1707817821 >> 172.21.15.31:0/1707817821 conn(0x7fd0b006d170 msgr2=0x7fd0b00c2760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:42.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:42.067+0000 7fd0affff700 1 -- 172.21.15.31:0/1707817821 shutdown_connections 2023-10-06T02:26:42.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:42.070+0000 7fd0affff700 1 -- 172.21.15.31:0/1707817821 wait complete. 2023-10-06T02:26:42.075 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' created 2023-10-06T02:26:42.098 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T02:26:42.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:42.291+0000 7fab0899d700 1 Processor -- start 2023-10-06T02:26:42.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:42.291+0000 7fab0899d700 1 -- start start 2023-10-06T02:26:42.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:42.292+0000 7fab0899d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fab04153c90 0x7fab0415a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:42.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:42.293+0000 7fab0899d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab040caec0 0x7fab0415afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:42.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:42.293+0000 7fab0899d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab0415ada0 con 0x7fab0406db20 2023-10-06T02:26:42.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:42.293+0000 7fab0899d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab0406ca80 con 0x7fab04153c90 2023-10-06T02:26:42.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:42.293+0000 7fab0899d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab0406cd60 con 0x7fab040caec0 2023-10-06T02:26:42.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:42.293+0000 7fab02ffd700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fab0406db20 0x7fab040ca7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46660/0 (socket says 172.21.15.31:46660) 2023-10-06T02:26:42.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:42.293+0000 7fab02ffd700 1 -- 172.21.15.31:0/2246059092 learned_addr learned my addr 172.21.15.31:0/2246059092 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:26:42.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:42.293+0000 7fab03fff700 1 --2- 172.21.15.31:0/2246059092 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab040caec0 0x7fab0415afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:42.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:42.293+0000 7fab037fe700 1 --2- 172.21.15.31:0/2246059092 >> v2:172.21.15.62:3300/0 conn(0x7fab04153c90 0x7fab0415a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:42.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:42.293+0000 7fab027fc700 1 -- 172.21.15.31:0/2246059092 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3637313231 0 0) 0x7faaf8001110 con 0x7fab0406db20 2023-10-06T02:26:42.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:42.293+0000 7fab027fc700 1 -- 172.21.15.31:0/2246059092 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faaec003570 con 0x7fab0406db20 2023-10-06T02:26:42.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:42.294+0000 7fab03fff700 1 -- 172.21.15.31:0/2246059092 >> v2:172.21.15.62:3300/0 conn(0x7fab04153c90 msgr2=0x7fab0415a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:26:42.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:42.294+0000 7fab03fff700 1 --2- 172.21.15.31:0/2246059092 >> v2:172.21.15.62:3300/0 conn(0x7fab04153c90 0x7fab0415a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:42.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:42.294+0000 7fab03fff700 1 -- 172.21.15.31:0/2246059092 >> v1:172.21.15.31:6789/0 conn(0x7fab0406db20 legacy=0x7fab040ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:42.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:42.294+0000 7fab03fff700 1 -- 172.21.15.31:0/2246059092 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab0406d040 con 0x7fab040caec0 2023-10-06T02:26:42.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:42.294+0000 7fab03fff700 1 --2- 172.21.15.31:0/2246059092 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab040caec0 0x7fab0415afb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7faafc009e60 tx=0x7faafc009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e17f7167e5e277db server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:42.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:42.294+0000 7fab027fc700 1 -- 172.21.15.31:0/2246059092 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faafc00e070 con 0x7fab040caec0 2023-10-06T02:26:42.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:42.294+0000 7fab027fc700 1 -- 172.21.15.31:0/2246059092 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faafc00bab0 con 0x7fab040caec0 2023-10-06T02:26:42.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:42.294+0000 7fab027fc700 1 -- 172.21.15.31:0/2246059092 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faafc00be30 con 0x7fab040caec0 2023-10-06T02:26:42.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:42.295+0000 7fab0899d700 1 -- 172.21.15.31:0/2246059092 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab040caec0 msgr2=0x7fab0415afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:42.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:42.295+0000 7fab0899d700 1 --2- 172.21.15.31:0/2246059092 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab040caec0 0x7fab0415afb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7faafc009e60 tx=0x7faafc009780 comp rx=0 tx=0).stop 2023-10-06T02:26:42.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:42.295+0000 7fab0899d700 1 -- 172.21.15.31:0/2246059092 shutdown_connections 2023-10-06T02:26:42.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:42.295+0000 7fab0899d700 1 --2- 172.21.15.31:0/2246059092 >> v2:172.21.15.62:3300/0 conn(0x7fab04153c90 0x7fab0415a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:42.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:42.295+0000 7fab0899d700 1 --2- 172.21.15.31:0/2246059092 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab040caec0 0x7fab0415afb0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:42.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:42.295+0000 7fab0899d700 1 -- 172.21.15.31:0/2246059092 >> 172.21.15.31:0/2246059092 conn(0x7fab0406e720 msgr2=0x7fab0406eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:42.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:42.295+0000 7fab0899d700 1 -- 172.21.15.31:0/2246059092 shutdown_connections 2023-10-06T02:26:42.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:42.295+0000 7fab0899d700 1 -- 172.21.15.31:0/2246059092 wait complete. 2023-10-06T02:26:42.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:42.296+0000 7fab0899d700 1 Processor -- start 2023-10-06T02:26:42.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:42.297+0000 7fab0899d700 1 -- start start 2023-10-06T02:26:42.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:42.297+0000 7fab0899d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab0406db20 0x7fab0413e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:42.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:42.297+0000 7fab0899d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab040caec0 0x7fab0416d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:42.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:42.298+0000 7fab0899d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab04153c90 0x7fab0416d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:42.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:42.298+0000 7fab0899d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab0416e240 con 0x7fab04153c90 2023-10-06T02:26:42.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:42.298+0000 7fab0899d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab040c4600 con 0x7fab0406db20 2023-10-06T02:26:42.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:42.298+0000 7fab0899d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab0413e950 con 0x7fab040caec0 2023-10-06T02:26:42.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:42.298+0000 7fab03fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab04153c90 0x7fab0416d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:42.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:42.298+0000 7fab03fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab04153c90 0x7fab0416d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49888/0 (socket says 172.21.15.31:49888) 2023-10-06T02:26:42.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:42.298+0000 7fab03fff700 1 -- 172.21.15.31:0/2890753180 learned_addr learned my addr 172.21.15.31:0/2890753180 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:26:42.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:42.298+0000 7fab02ffd700 1 --2- 172.21.15.31:0/2890753180 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab040caec0 0x7fab0416d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:42.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:42.298+0000 7fab037fe700 1 --2- 172.21.15.31:0/2890753180 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab0406db20 0x7fab0413e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:42.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:42.298+0000 7fab03fff700 1 -- 172.21.15.31:0/2890753180 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab040caec0 msgr2=0x7fab0416d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:42.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:42.298+0000 7fab03fff700 1 --2- 172.21.15.31:0/2890753180 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab040caec0 0x7fab0416d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:42.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:42.298+0000 7fab03fff700 1 -- 172.21.15.31:0/2890753180 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab0406db20 msgr2=0x7fab0413e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:42.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:42.298+0000 7fab03fff700 1 --2- 172.21.15.31:0/2890753180 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab0406db20 0x7fab0413e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:42.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:42.298+0000 7fab03fff700 1 -- 172.21.15.31:0/2890753180 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faafc009a20 con 0x7fab04153c90 2023-10-06T02:26:42.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:42.299+0000 7fab03fff700 1 --2- 172.21.15.31:0/2890753180 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab04153c90 0x7fab0416d8c0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7faafc013000 tx=0x7faafc019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:42.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:42.299+0000 7faaeb7fe700 1 -- 172.21.15.31:0/2890753180 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faafc027470 con 0x7fab04153c90 2023-10-06T02:26:42.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:42.299+0000 7faaeb7fe700 1 -- 172.21.15.31:0/2890753180 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faafc027610 con 0x7fab04153c90 2023-10-06T02:26:42.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:42.299+0000 7faaeb7fe700 1 -- 172.21.15.31:0/2890753180 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faafc0279d0 con 0x7fab04153c90 2023-10-06T02:26:42.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:42.299+0000 7fab0899d700 1 -- 172.21.15.31:0/2890753180 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab0414ddf0 con 0x7fab04153c90 2023-10-06T02:26:42.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:42.299+0000 7fab0899d700 1 -- 172.21.15.31:0/2890753180 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab040c4810 con 0x7fab04153c90 2023-10-06T02:26:42.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:42.300+0000 7fab0899d700 1 -- 172.21.15.31:0/2890753180 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab0404c3c0 con 0x7fab04153c90 2023-10-06T02:26:42.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:42.301+0000 7faaeb7fe700 1 -- 172.21.15.31:0/2890753180 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 9) v1 ==== 84893+0+0 (secure 0 0 0) 0x7faafc024070 con 0x7fab04153c90 2023-10-06T02:26:42.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:42.302+0000 7faaeb7fe700 1 --2- 172.21.15.31:0/2890753180 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7faaf00670e0 0x7faaf0069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:42.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:42.302+0000 7faaeb7fe700 1 -- 172.21.15.31:0/2890753180 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5958+0+0 (secure 0 0 0) 0x7faafc0873b0 con 0x7fab04153c90 2023-10-06T02:26:42.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:42.302+0000 7fab037fe700 1 --2- 172.21.15.31:0/2890753180 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7faaf00670e0 0x7faaf0069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:42.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:42.303+0000 7fab037fe700 1 --2- 172.21.15.31:0/2890753180 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7faaf00670e0 0x7faaf0069590 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fab041527e0 tx=0x7faaf4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:42.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:42.307+0000 7faaeb7fe700 1 -- 172.21.15.31:0/2890753180 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faafc054200 con 0x7fab04153c90 2023-10-06T02:26:42.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:42.553+0000 7fab0899d700 1 -- 172.21.15.31:0/2890753180 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fab04159bc0 con 0x7fab04153c90 2023-10-06T02:26:42.738 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2023-10-06T02:26:43.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.060+0000 7faaeb7fe700 1 -- 172.21.15.31:0/2890753180 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 9 and data pool 10 v33) v1 ==== 173+0+0 (secure 0 0 0) 0x7faafc057820 con 0x7fab04153c90 2023-10-06T02:26:43.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.069+0000 7fab0899d700 1 -- 172.21.15.31:0/2890753180 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7faaf00670e0 msgr2=0x7faaf0069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:43.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.069+0000 7fab0899d700 1 --2- 172.21.15.31:0/2890753180 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7faaf00670e0 0x7faaf0069590 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fab041527e0 tx=0x7faaf4008040 comp rx=0 tx=0).stop 2023-10-06T02:26:43.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.069+0000 7fab0899d700 1 -- 172.21.15.31:0/2890753180 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab04153c90 msgr2=0x7fab0416d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:43.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.069+0000 7fab0899d700 1 --2- 172.21.15.31:0/2890753180 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab04153c90 0x7fab0416d8c0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7faafc013000 tx=0x7faafc019000 comp rx=0 tx=0).stop 2023-10-06T02:26:43.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.070+0000 7fab0899d700 1 -- 172.21.15.31:0/2890753180 shutdown_connections 2023-10-06T02:26:43.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.070+0000 7fab0899d700 1 --2- 172.21.15.31:0/2890753180 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab04153c90 0x7fab0416d8c0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:43.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.070+0000 7fab0899d700 1 --2- 172.21.15.31:0/2890753180 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab0406db20 0x7fab0413e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:43.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.070+0000 7fab0899d700 1 --2- 172.21.15.31:0/2890753180 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7faaf00670e0 0x7faaf0069590 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:43.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.070+0000 7fab0899d700 1 --2- 172.21.15.31:0/2890753180 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab040caec0 0x7fab0416d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:43.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.070+0000 7fab0899d700 1 -- 172.21.15.31:0/2890753180 >> 172.21.15.31:0/2890753180 conn(0x7fab0406e720 msgr2=0x7fab04159de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:43.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.077+0000 7fab0899d700 1 -- 172.21.15.31:0/2890753180 shutdown_connections 2023-10-06T02:26:43.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.078+0000 7fab0899d700 1 -- 172.21.15.31:0/2890753180 wait complete. 2023-10-06T02:26:43.085 INFO:teuthology.orchestra.run.smithi031.stderr:new fs with metadata pool 9 and data pool 10 2023-10-06T02:26:43.107 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:26:43.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.296+0000 7f71d359e700 1 Processor -- start 2023-10-06T02:26:43.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.296+0000 7f71d359e700 1 -- start start 2023-10-06T02:26:43.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.297+0000 7f71d359e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f71d4159920 0x7f71d40ad090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:43.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.297+0000 7f71d359e700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71d40acc00 con 0x7f71d4159550 2023-10-06T02:26:43.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.297+0000 7f71d359e700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71d40abff0 con 0x7f71d415dfd0 2023-10-06T02:26:43.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.298+0000 7f71d359e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71d40ac2d0 con 0x7f71d4159920 2023-10-06T02:26:43.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.298+0000 7f71d259c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f71d4159550 0x7f71d40ceac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46676/0 (socket says 172.21.15.31:46676) 2023-10-06T02:26:43.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.298+0000 7f71d259c700 1 -- 172.21.15.31:0/1174464614 learned_addr learned my addr 172.21.15.31:0/1174464614 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:26:43.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.298+0000 7f71d2d9d700 1 --2- 172.21.15.31:0/1174464614 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f71d4159920 0x7f71d40ad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:43.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.298+0000 7f71d159a700 1 -- 172.21.15.31:0/1174464614 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2304945195 0 0) 0x7f71d40acc00 con 0x7f71d4159550 2023-10-06T02:26:43.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.298+0000 7f71d159a700 1 -- 172.21.15.31:0/1174464614 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71bc003570 con 0x7f71d4159550 2023-10-06T02:26:43.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.298+0000 7f71d159a700 1 -- 172.21.15.31:0/1174464614 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 420409163 0 0) 0x7f71d40abff0 con 0x7f71d415dfd0 2023-10-06T02:26:43.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.298+0000 7f71d159a700 1 -- 172.21.15.31:0/1174464614 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71d40acc00 con 0x7f71d415dfd0 2023-10-06T02:26:43.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.299+0000 7f71d2d9d700 1 -- 172.21.15.31:0/1174464614 >> v1:172.21.15.62:6789/0 conn(0x7f71d415dfd0 legacy=0x7f71d40aedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:43.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.299+0000 7f71d2d9d700 1 -- 172.21.15.31:0/1174464614 >> v1:172.21.15.31:6789/0 conn(0x7f71d4159550 legacy=0x7f71d40ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:43.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.299+0000 7f71d159a700 1 -- 172.21.15.31:0/1174464614 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3058578780 0 0) 0x7f71bc003570 con 0x7f71d4159550 2023-10-06T02:26:43.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.299+0000 7f71d2d9d700 1 -- 172.21.15.31:0/1174464614 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71d40ac5b0 con 0x7f71d4159920 2023-10-06T02:26:43.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.299+0000 7f71d2d9d700 1 --2- 172.21.15.31:0/1174464614 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f71d4159920 0x7f71d40ad090 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f71cc009e90 tx=0x7f71cc009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15be54d42e6d9ae2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:43.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.299+0000 7f71d159a700 1 -- 172.21.15.31:0/1174464614 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71cc00e070 con 0x7f71d4159920 2023-10-06T02:26:43.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.299+0000 7f71d159a700 1 -- 172.21.15.31:0/1174464614 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71cc00b870 con 0x7f71d4159920 2023-10-06T02:26:43.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.299+0000 7f71d159a700 1 -- 172.21.15.31:0/1174464614 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c4001470 con 0x7f71d4159920 2023-10-06T02:26:43.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.300+0000 7f71d359e700 1 -- 172.21.15.31:0/1174464614 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f71d4159920 msgr2=0x7f71d40ad090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:43.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.300+0000 7f71d359e700 1 --2- 172.21.15.31:0/1174464614 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f71d4159920 0x7f71d40ad090 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f71cc009e90 tx=0x7f71cc009780 comp rx=0 tx=0).stop 2023-10-06T02:26:43.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.300+0000 7f71d359e700 1 -- 172.21.15.31:0/1174464614 shutdown_connections 2023-10-06T02:26:43.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.300+0000 7f71d359e700 1 --2- 172.21.15.31:0/1174464614 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f71d4159920 0x7f71d40ad090 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:43.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.300+0000 7f71d359e700 1 -- 172.21.15.31:0/1174464614 >> 172.21.15.31:0/1174464614 conn(0x7f71d406ce80 msgr2=0x7f71d40ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:43.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.300+0000 7f71d359e700 1 -- 172.21.15.31:0/1174464614 shutdown_connections 2023-10-06T02:26:43.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.300+0000 7f71d359e700 1 -- 172.21.15.31:0/1174464614 wait complete. 2023-10-06T02:26:43.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.301+0000 7f71d359e700 1 Processor -- start 2023-10-06T02:26:43.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.301+0000 7f71d359e700 1 -- start start 2023-10-06T02:26:43.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.302+0000 7f71d359e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71d4159550 0x7f71d40b3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:43.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.302+0000 7f71d359e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f71d4159920 0x7f71d40c9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:43.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.302+0000 7f71d359e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71d415dfd0 0x7f71d4158500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:43.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.302+0000 7f71d359e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71d40b20f0 con 0x7f71d415dfd0 2023-10-06T02:26:43.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.302+0000 7f71d359e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71d40b23d0 con 0x7f71d4159550 2023-10-06T02:26:43.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.302+0000 7f71d359e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71d40b26b0 con 0x7f71d4159920 2023-10-06T02:26:43.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.303+0000 7f71d2d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71d415dfd0 0x7f71d4158500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:43.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.303+0000 7f71d2d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71d415dfd0 0x7f71d4158500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49900/0 (socket says 172.21.15.31:49900) 2023-10-06T02:26:43.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.303+0000 7f71d2d9d700 1 -- 172.21.15.31:0/3199022268 learned_addr learned my addr 172.21.15.31:0/3199022268 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:26:43.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.303+0000 7f71d259c700 1 --2- 172.21.15.31:0/3199022268 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71d4159550 0x7f71d40b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:43.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.303+0000 7f71d1d9b700 1 --2- 172.21.15.31:0/3199022268 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f71d4159920 0x7f71d40c9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:43.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.303+0000 7f71d2d9d700 1 -- 172.21.15.31:0/3199022268 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f71d4159920 msgr2=0x7f71d40c9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:43.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.303+0000 7f71d2d9d700 1 --2- 172.21.15.31:0/3199022268 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f71d4159920 0x7f71d40c9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:43.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.303+0000 7f71d2d9d700 1 -- 172.21.15.31:0/3199022268 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71d4159550 msgr2=0x7f71d40b3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:43.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.303+0000 7f71d2d9d700 1 --2- 172.21.15.31:0/3199022268 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71d4159550 0x7f71d40b3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:43.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.303+0000 7f71d2d9d700 1 -- 172.21.15.31:0/3199022268 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71cc009a20 con 0x7f71d415dfd0 2023-10-06T02:26:43.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.303+0000 7f71d2d9d700 1 --2- 172.21.15.31:0/3199022268 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71d415dfd0 0x7f71d4158500 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f71cc009de0 tx=0x7f71cc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:43.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.304+0000 7f71c27fc700 1 -- 172.21.15.31:0/3199022268 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71cc012cd0 con 0x7f71d415dfd0 2023-10-06T02:26:43.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.304+0000 7f71c27fc700 1 -- 172.21.15.31:0/3199022268 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71cc012e30 con 0x7f71d415dfd0 2023-10-06T02:26:43.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.304+0000 7f71c27fc700 1 -- 172.21.15.31:0/3199022268 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71cc01e6a0 con 0x7f71d415dfd0 2023-10-06T02:26:43.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.304+0000 7f71d359e700 1 -- 172.21.15.31:0/3199022268 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71d40b2990 con 0x7f71d415dfd0 2023-10-06T02:26:43.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.304+0000 7f71d359e700 1 -- 172.21.15.31:0/3199022268 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71d40b36d0 con 0x7f71d415dfd0 2023-10-06T02:26:43.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.305+0000 7f71d359e700 1 -- 172.21.15.31:0/3199022268 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71d404c3c0 con 0x7f71d415dfd0 2023-10-06T02:26:43.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.306+0000 7f71c27fc700 1 -- 172.21.15.31:0/3199022268 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 9) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f71cc01ead0 con 0x7f71d415dfd0 2023-10-06T02:26:43.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.307+0000 7f71c27fc700 1 --2- 172.21.15.31:0/3199022268 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f71b40670e0 0x7f71b4069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:43.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.307+0000 7f71c27fc700 1 -- 172.21.15.31:0/3199022268 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f71cc0869e0 con 0x7f71d415dfd0 2023-10-06T02:26:43.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.307+0000 7f71d259c700 1 --2- 172.21.15.31:0/3199022268 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f71b40670e0 0x7f71b4069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:43.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.308+0000 7f71d259c700 1 --2- 172.21.15.31:0/3199022268 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f71b40670e0 0x7f71b4069590 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f71c40011e0 tx=0x7f71c4001620 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:43.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.311+0000 7f71c27fc700 1 -- 172.21.15.31:0/3199022268 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f71cc0536c0 con 0x7f71d415dfd0 2023-10-06T02:26:43.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.512+0000 7f71d359e700 1 -- 172.21.15.31:0/3199022268 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f71d406b8e0 con 0x7f71d415dfd0 2023-10-06T02:26:43.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.513+0000 7f71c27fc700 1 -- 172.21.15.31:0/3199022268 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) v1 ==== 74+0+16833 (secure 0 0 0) 0x7f71cc056ce0 con 0x7f71d415dfd0 2023-10-06T02:26:43.513 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:26:43.514 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":60,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:26:42.554158+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:26:40.728441+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-10-06T02:26:41.524170+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:26:43.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.515+0000 7f71d359e700 1 -- 172.21.15.31:0/3199022268 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f71b40670e0 msgr2=0x7f71b4069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:43.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.515+0000 7f71d359e700 1 --2- 172.21.15.31:0/3199022268 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f71b40670e0 0x7f71b4069590 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f71c40011e0 tx=0x7f71c4001620 comp rx=0 tx=0).stop 2023-10-06T02:26:43.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.515+0000 7f71d359e700 1 -- 172.21.15.31:0/3199022268 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71d415dfd0 msgr2=0x7f71d4158500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:43.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.515+0000 7f71d359e700 1 --2- 172.21.15.31:0/3199022268 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71d415dfd0 0x7f71d4158500 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f71cc009de0 tx=0x7f71cc017000 comp rx=0 tx=0).stop 2023-10-06T02:26:43.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.515+0000 7f71d359e700 1 -- 172.21.15.31:0/3199022268 shutdown_connections 2023-10-06T02:26:43.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.515+0000 7f71d359e700 1 --2- 172.21.15.31:0/3199022268 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71d415dfd0 0x7f71d4158500 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:43.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.515+0000 7f71d359e700 1 --2- 172.21.15.31:0/3199022268 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71d4159550 0x7f71d40b3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:43.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.515+0000 7f71d359e700 1 --2- 172.21.15.31:0/3199022268 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f71b40670e0 0x7f71b4069590 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:43.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.515+0000 7f71d359e700 1 --2- 172.21.15.31:0/3199022268 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f71d4159920 0x7f71d40c9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:43.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.515+0000 7f71d359e700 1 -- 172.21.15.31:0/3199022268 >> 172.21.15.31:0/3199022268 conn(0x7f71d406ce80 msgr2=0x7f71d41542f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:43.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.515+0000 7f71d359e700 1 -- 172.21.15.31:0/3199022268 shutdown_connections 2023-10-06T02:26:43.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.515+0000 7f71d359e700 1 -- 172.21.15.31:0/3199022268 wait complete. 2023-10-06T02:26:43.534 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:26:43.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.721+0000 7fe40bfff700 1 Processor -- start 2023-10-06T02:26:43.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.721+0000 7fe40bfff700 1 -- start start 2023-10-06T02:26:43.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.723+0000 7fe40bfff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe40c142c00 0x7fe40c06b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:43.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.723+0000 7fe40bfff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe40c0c7f90 0x7fe40c06c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:43.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.723+0000 7fe40bfff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe40c0ca480 con 0x7fe40c06bd20 2023-10-06T02:26:43.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.723+0000 7fe40bfff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe40c15e240 con 0x7fe40c142c00 2023-10-06T02:26:43.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.723+0000 7fe40bfff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe40c06b350 con 0x7fe40c0c7f90 2023-10-06T02:26:43.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.723+0000 7fe40a7fc700 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fe40c06bd20 0x7fe40c0c7880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:45974/0 (socket says 172.21.15.31:45974) 2023-10-06T02:26:43.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.723+0000 7fe40a7fc700 1 -- 172.21.15.31:0/2060208830 learned_addr learned my addr 172.21.15.31:0/2060208830 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:26:43.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.723+0000 7fe40affd700 1 --2- 172.21.15.31:0/2060208830 >> v2:172.21.15.62:3300/0 conn(0x7fe40c142c00 0x7fe40c06b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:43.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.723+0000 7fe40b7fe700 1 --2- 172.21.15.31:0/2060208830 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe40c0c7f90 0x7fe40c06c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:43.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.724+0000 7fe409ffb700 1 -- 172.21.15.31:0/2060208830 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 557394753 0 0) 0x7fe40c0ca480 con 0x7fe40c06bd20 2023-10-06T02:26:43.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.724+0000 7fe409ffb700 1 -- 172.21.15.31:0/2060208830 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3f4003570 con 0x7fe40c06bd20 2023-10-06T02:26:43.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.724+0000 7fe40affd700 1 -- 172.21.15.31:0/2060208830 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe40c0c7f90 msgr2=0x7fe40c06c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:26:43.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.724+0000 7fe40affd700 1 --2- 172.21.15.31:0/2060208830 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe40c0c7f90 0x7fe40c06c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:43.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.724+0000 7fe40affd700 1 -- 172.21.15.31:0/2060208830 >> v1:172.21.15.62:6789/0 conn(0x7fe40c06bd20 legacy=0x7fe40c0c7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:43.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.724+0000 7fe40affd700 1 -- 172.21.15.31:0/2060208830 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe40c15df60 con 0x7fe40c142c00 2023-10-06T02:26:43.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.724+0000 7fe40affd700 1 --2- 172.21.15.31:0/2060208830 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe40c142c00 0x7fe40c06b7e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc0098e0 tx=0x7fe3fc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8cca586c3dbf70e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:43.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.724+0000 7fe409ffb700 1 -- 172.21.15.31:0/2060208830 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3fc011630 con 0x7fe40c142c00 2023-10-06T02:26:43.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.725+0000 7fe409ffb700 1 -- 172.21.15.31:0/2060208830 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3fc011790 con 0x7fe40c142c00 2023-10-06T02:26:43.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.725+0000 7fe409ffb700 1 -- 172.21.15.31:0/2060208830 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3fc011b30 con 0x7fe40c142c00 2023-10-06T02:26:43.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.725+0000 7fe40bfff700 1 -- 172.21.15.31:0/2060208830 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe40c142c00 msgr2=0x7fe40c06b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:43.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.725+0000 7fe40bfff700 1 --2- 172.21.15.31:0/2060208830 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe40c142c00 0x7fe40c06b7e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc0098e0 tx=0x7fe3fc0091a0 comp rx=0 tx=0).stop 2023-10-06T02:26:43.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.725+0000 7fe40bfff700 1 -- 172.21.15.31:0/2060208830 shutdown_connections 2023-10-06T02:26:43.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.725+0000 7fe40bfff700 1 --2- 172.21.15.31:0/2060208830 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe40c0c7f90 0x7fe40c06c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:43.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.725+0000 7fe40bfff700 1 --2- 172.21.15.31:0/2060208830 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe40c142c00 0x7fe40c06b7e0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:43.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.725+0000 7fe40bfff700 1 -- 172.21.15.31:0/2060208830 >> 172.21.15.31:0/2060208830 conn(0x7fe40c0675a0 msgr2=0x7fe40c0679a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:43.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.725+0000 7fe40bfff700 1 -- 172.21.15.31:0/2060208830 shutdown_connections 2023-10-06T02:26:43.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.725+0000 7fe40bfff700 1 -- 172.21.15.31:0/2060208830 wait complete. 2023-10-06T02:26:43.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.727+0000 7fe40bfff700 1 Processor -- start 2023-10-06T02:26:43.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.727+0000 7fe40bfff700 1 -- start start 2023-10-06T02:26:43.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.728+0000 7fe40bfff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe40c06bd20 0x7fe40c13d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:43.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.728+0000 7fe40bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe40c0c7f90 0x7fe40c0b09f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:43.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.728+0000 7fe40bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe40c142c00 0x7fe40c0cd490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:43.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.728+0000 7fe40bfff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe40c159f00 con 0x7fe40c0c7f90 2023-10-06T02:26:43.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.728+0000 7fe40bfff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe40c13de80 con 0x7fe40c142c00 2023-10-06T02:26:43.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.728+0000 7fe40bfff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe40c0b13d0 con 0x7fe40c06bd20 2023-10-06T02:26:43.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.728+0000 7fe40a7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe40c0c7f90 0x7fe40c0b09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:43.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.728+0000 7fe40a7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe40c0c7f90 0x7fe40c0b09f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49902/0 (socket says 172.21.15.31:49902) 2023-10-06T02:26:43.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.728+0000 7fe40a7fc700 1 -- 172.21.15.31:0/261749731 learned_addr learned my addr 172.21.15.31:0/261749731 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:26:43.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.728+0000 7fe40b7fe700 1 --2- 172.21.15.31:0/261749731 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe40c142c00 0x7fe40c0cd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:43.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.728+0000 7fe40affd700 1 --2- 172.21.15.31:0/261749731 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe40c06bd20 0x7fe40c13d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:43.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.728+0000 7fe40a7fc700 1 -- 172.21.15.31:0/261749731 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe40c06bd20 msgr2=0x7fe40c13d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:43.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.728+0000 7fe40a7fc700 1 --2- 172.21.15.31:0/261749731 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe40c06bd20 0x7fe40c13d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:43.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.728+0000 7fe40a7fc700 1 -- 172.21.15.31:0/261749731 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe40c142c00 msgr2=0x7fe40c0cd490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:43.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.728+0000 7fe40a7fc700 1 --2- 172.21.15.31:0/261749731 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe40c142c00 0x7fe40c0cd490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:43.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.728+0000 7fe40a7fc700 1 -- 172.21.15.31:0/261749731 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3fc009420 con 0x7fe40c0c7f90 2023-10-06T02:26:43.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.728+0000 7fe40affd700 1 --2- 172.21.15.31:0/261749731 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe40c06bd20 0x7fe40c13d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:26:43.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.729+0000 7fe40a7fc700 1 --2- 172.21.15.31:0/261749731 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe40c0c7f90 0x7fe40c0b09f0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fe40000c7c0 tx=0x7fe400007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:43.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.729+0000 7fe3f2ffd700 1 -- 172.21.15.31:0/261749731 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe400005c00 con 0x7fe40c0c7f90 2023-10-06T02:26:43.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.729+0000 7fe3f2ffd700 1 -- 172.21.15.31:0/261749731 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe400005d60 con 0x7fe40c0c7f90 2023-10-06T02:26:43.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.729+0000 7fe3f2ffd700 1 -- 172.21.15.31:0/261749731 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe400007300 con 0x7fe40c0c7f90 2023-10-06T02:26:43.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.729+0000 7fe40bfff700 1 -- 172.21.15.31:0/261749731 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe40c0b16b0 con 0x7fe40c0c7f90 2023-10-06T02:26:43.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.729+0000 7fe40bfff700 1 -- 172.21.15.31:0/261749731 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe40c0b1c70 con 0x7fe40c0c7f90 2023-10-06T02:26:43.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.730+0000 7fe40bfff700 1 -- 172.21.15.31:0/261749731 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe40c04c3c0 con 0x7fe40c0c7f90 2023-10-06T02:26:43.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.731+0000 7fe3f2ffd700 1 -- 172.21.15.31:0/261749731 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 9) v1 ==== 84893+0+0 (secure 0 0 0) 0x7fe40001b3b0 con 0x7fe40c0c7f90 2023-10-06T02:26:43.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.732+0000 7fe3f2ffd700 1 --2- 172.21.15.31:0/261749731 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe3f80670e0 0x7fe3f8069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:43.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.732+0000 7fe3f2ffd700 1 -- 172.21.15.31:0/261749731 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe400083fb0 con 0x7fe40c0c7f90 2023-10-06T02:26:43.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.732+0000 7fe40affd700 1 --2- 172.21.15.31:0/261749731 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe3f80670e0 0x7fe3f8069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:43.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.733+0000 7fe40affd700 1 --2- 172.21.15.31:0/261749731 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe3f80670e0 0x7fe3f8069590 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc003080 tx=0x7fe3fc017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:43.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.736+0000 7fe3f2ffd700 1 -- 172.21.15.31:0/261749731 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe400050da0 con 0x7fe40c0c7f90 2023-10-06T02:26:43.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.942+0000 7fe40bfff700 1 -- 172.21.15.31:0/261749731 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe40c06ad00 con 0x7fe40c0c7f90 2023-10-06T02:26:43.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.943+0000 7fe3f2ffd700 1 -- 172.21.15.31:0/261749731 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) v1 ==== 74+0+16833 (secure 0 0 0) 0x7fe400017070 con 0x7fe40c0c7f90 2023-10-06T02:26:43.944 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:26:43.944 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":60,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:26:42.554158+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:26:40.728441+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-10-06T02:26:41.524170+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:26:43.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.945+0000 7fe40bfff700 1 -- 172.21.15.31:0/261749731 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe3f80670e0 msgr2=0x7fe3f8069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:43.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.945+0000 7fe40bfff700 1 --2- 172.21.15.31:0/261749731 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe3f80670e0 0x7fe3f8069590 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc003080 tx=0x7fe3fc017000 comp rx=0 tx=0).stop 2023-10-06T02:26:43.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.945+0000 7fe40bfff700 1 -- 172.21.15.31:0/261749731 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe40c0c7f90 msgr2=0x7fe40c0b09f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:43.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.945+0000 7fe40bfff700 1 --2- 172.21.15.31:0/261749731 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe40c0c7f90 0x7fe40c0b09f0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fe40000c7c0 tx=0x7fe400007dd0 comp rx=0 tx=0).stop 2023-10-06T02:26:43.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.946+0000 7fe40bfff700 1 -- 172.21.15.31:0/261749731 shutdown_connections 2023-10-06T02:26:43.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.946+0000 7fe40bfff700 1 --2- 172.21.15.31:0/261749731 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe40c142c00 0x7fe40c0cd490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:43.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.946+0000 7fe40bfff700 1 --2- 172.21.15.31:0/261749731 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe3f80670e0 0x7fe3f8069590 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:43.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.946+0000 7fe40bfff700 1 --2- 172.21.15.31:0/261749731 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe40c06bd20 0x7fe40c13d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:43.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.946+0000 7fe40bfff700 1 --2- 172.21.15.31:0/261749731 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe40c0c7f90 0x7fe40c0b09f0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:43.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.946+0000 7fe40bfff700 1 -- 172.21.15.31:0/261749731 >> 172.21.15.31:0/261749731 conn(0x7fe40c0675a0 msgr2=0x7fe40c13f720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:43.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.946+0000 7fe40bfff700 1 -- 172.21.15.31:0/261749731 shutdown_connections 2023-10-06T02:26:43.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:43.946+0000 7fe40bfff700 1 -- 172.21.15.31:0/261749731 wait complete. 2023-10-06T02:26:43.964 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T02:26:44.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:44.157+0000 7fd68bfff700 1 Processor -- start 2023-10-06T02:26:44.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:44.157+0000 7fd68bfff700 1 -- start start 2023-10-06T02:26:44.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:44.159+0000 7fd68bfff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd68c0accb0 0x7fd68c0aaed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:44.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:44.159+0000 7fd68bfff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd68c0cbea0 0x7fd68c0cc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:44.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:44.159+0000 7fd68bfff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd68c0cc850 con 0x7fd68c06e330 2023-10-06T02:26:44.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:44.159+0000 7fd68bfff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd68c0ad780 con 0x7fd68c0accb0 2023-10-06T02:26:44.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:44.159+0000 7fd68bfff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd68c0cee60 con 0x7fd68c0cbea0 2023-10-06T02:26:44.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:44.159+0000 7fd68b7fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fd68c06e330 0x7fd68c06e7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46682/0 (socket says 172.21.15.31:46682) 2023-10-06T02:26:44.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:44.159+0000 7fd68b7fe700 1 -- 172.21.15.31:0/2847908987 learned_addr learned my addr 172.21.15.31:0/2847908987 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:26:44.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:44.159+0000 7fd68a7fc700 1 --2- 172.21.15.31:0/2847908987 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd68c0cbea0 0x7fd68c0cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:44.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:44.159+0000 7fd68affd700 1 --2- 172.21.15.31:0/2847908987 >> v2:172.21.15.62:3300/0 conn(0x7fd68c0accb0 0x7fd68c0aaed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:44.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:44.159+0000 7fd689ffb700 1 -- 172.21.15.31:0/2847908987 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3800390173 0 0) 0x7fd68c0cc850 con 0x7fd68c06e330 2023-10-06T02:26:44.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:44.160+0000 7fd689ffb700 1 -- 172.21.15.31:0/2847908987 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd674003570 con 0x7fd68c06e330 2023-10-06T02:26:44.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:44.160+0000 7fd68affd700 1 -- 172.21.15.31:0/2847908987 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd68c0cbea0 msgr2=0x7fd68c0cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:26:44.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:44.160+0000 7fd68affd700 1 --2- 172.21.15.31:0/2847908987 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd68c0cbea0 0x7fd68c0cc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:44.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:44.160+0000 7fd68affd700 1 -- 172.21.15.31:0/2847908987 >> v1:172.21.15.31:6789/0 conn(0x7fd68c06e330 legacy=0x7fd68c06e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:44.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:44.160+0000 7fd68affd700 1 -- 172.21.15.31:0/2847908987 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd68c0cf140 con 0x7fd68c0accb0 2023-10-06T02:26:44.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:44.160+0000 7fd68a7fc700 1 --2- 172.21.15.31:0/2847908987 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd68c0cbea0 0x7fd68c0cc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:26:44.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:44.160+0000 7fd68affd700 1 --2- 172.21.15.31:0/2847908987 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd68c0accb0 0x7fd68c0aaed0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd67c0098e0 tx=0x7fd67c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8501611e271ac0a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:44.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:44.160+0000 7fd689ffb700 1 -- 172.21.15.31:0/2847908987 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd67c011630 con 0x7fd68c0accb0 2023-10-06T02:26:44.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:44.160+0000 7fd689ffb700 1 -- 172.21.15.31:0/2847908987 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd67c011790 con 0x7fd68c0accb0 2023-10-06T02:26:44.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:44.160+0000 7fd689ffb700 1 -- 172.21.15.31:0/2847908987 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd67c011b30 con 0x7fd68c0accb0 2023-10-06T02:26:44.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:44.161+0000 7fd68bfff700 1 -- 172.21.15.31:0/2847908987 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd68c0accb0 msgr2=0x7fd68c0aaed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:44.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:44.161+0000 7fd68bfff700 1 --2- 172.21.15.31:0/2847908987 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd68c0accb0 0x7fd68c0aaed0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd67c0098e0 tx=0x7fd67c0091a0 comp rx=0 tx=0).stop 2023-10-06T02:26:44.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:44.161+0000 7fd68bfff700 1 -- 172.21.15.31:0/2847908987 shutdown_connections 2023-10-06T02:26:44.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:44.161+0000 7fd68bfff700 1 --2- 172.21.15.31:0/2847908987 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd68c0accb0 0x7fd68c0aaed0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:44.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:44.161+0000 7fd68bfff700 1 --2- 172.21.15.31:0/2847908987 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd68c0cbea0 0x7fd68c0cc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:44.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:44.161+0000 7fd68bfff700 1 -- 172.21.15.31:0/2847908987 >> 172.21.15.31:0/2847908987 conn(0x7fd68c06ce80 msgr2=0x7fd68c0ce370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:44.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:44.161+0000 7fd68bfff700 1 -- 172.21.15.31:0/2847908987 shutdown_connections 2023-10-06T02:26:44.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:44.161+0000 7fd68bfff700 1 -- 172.21.15.31:0/2847908987 wait complete. 2023-10-06T02:26:44.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:44.162+0000 7fd68bfff700 1 Processor -- start 2023-10-06T02:26:44.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:44.163+0000 7fd68bfff700 1 -- start start 2023-10-06T02:26:44.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:44.164+0000 7fd68bfff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd68c06e330 0x7fd68c0af560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:44.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:44.164+0000 7fd68bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd68c0accb0 0x7fd68c0ad950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:44.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:44.164+0000 7fd68bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd68c0cbea0 0x7fd68c0ade90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:44.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:44.164+0000 7fd68bfff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd68c0af0d0 con 0x7fd68c0cbea0 2023-10-06T02:26:44.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:44.164+0000 7fd68bfff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd68c0c7830 con 0x7fd68c0accb0 2023-10-06T02:26:44.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:44.164+0000 7fd68bfff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd68c0c6e00 con 0x7fd68c06e330 2023-10-06T02:26:44.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:44.164+0000 7fd68b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd68c0cbea0 0x7fd68c0ade90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:44.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:44.164+0000 7fd68b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd68c0cbea0 0x7fd68c0ade90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49916/0 (socket says 172.21.15.31:49916) 2023-10-06T02:26:44.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:44.164+0000 7fd68b7fe700 1 -- 172.21.15.31:0/2513916866 learned_addr learned my addr 172.21.15.31:0/2513916866 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:26:44.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:44.164+0000 7fd68affd700 1 --2- 172.21.15.31:0/2513916866 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd68c06e330 0x7fd68c0af560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:44.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:44.164+0000 7fd68a7fc700 1 --2- 172.21.15.31:0/2513916866 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd68c0accb0 0x7fd68c0ad950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:44.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:44.164+0000 7fd68b7fe700 1 -- 172.21.15.31:0/2513916866 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd68c06e330 msgr2=0x7fd68c0af560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:44.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:44.164+0000 7fd68b7fe700 1 --2- 172.21.15.31:0/2513916866 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd68c06e330 0x7fd68c0af560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:44.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:44.164+0000 7fd68b7fe700 1 -- 172.21.15.31:0/2513916866 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd68c0accb0 msgr2=0x7fd68c0ad950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:44.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:44.164+0000 7fd68b7fe700 1 --2- 172.21.15.31:0/2513916866 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd68c0accb0 0x7fd68c0ad950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:44.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:44.164+0000 7fd68b7fe700 1 -- 172.21.15.31:0/2513916866 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd680009420 con 0x7fd68c0cbea0 2023-10-06T02:26:44.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:44.165+0000 7fd68b7fe700 1 --2- 172.21.15.31:0/2513916866 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd68c0cbea0 0x7fd68c0ade90 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fd684001460 tx=0x7fd684001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:44.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:44.165+0000 7fd672ffd700 1 -- 172.21.15.31:0/2513916866 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd68400a950 con 0x7fd68c0cbea0 2023-10-06T02:26:44.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:44.165+0000 7fd672ffd700 1 -- 172.21.15.31:0/2513916866 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd68400aab0 con 0x7fd68c0cbea0 2023-10-06T02:26:44.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:44.165+0000 7fd68bfff700 1 -- 172.21.15.31:0/2513916866 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd67c009420 con 0x7fd68c0cbea0 2023-10-06T02:26:44.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:44.165+0000 7fd672ffd700 1 -- 172.21.15.31:0/2513916866 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd68400ae30 con 0x7fd68c0cbea0 2023-10-06T02:26:44.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:44.165+0000 7fd68bfff700 1 -- 172.21.15.31:0/2513916866 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd68c0ae7a0 con 0x7fd68c0cbea0 2023-10-06T02:26:44.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:44.166+0000 7fd68bfff700 1 -- 172.21.15.31:0/2513916866 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd68c0af280 con 0x7fd68c0cbea0 2023-10-06T02:26:44.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:44.167+0000 7fd672ffd700 1 -- 172.21.15.31:0/2513916866 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 9) v1 ==== 84893+0+0 (secure 0 0 0) 0x7fd684007300 con 0x7fd68c0cbea0 2023-10-06T02:26:44.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:44.168+0000 7fd672ffd700 1 --2- 172.21.15.31:0/2513916866 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fd6780670e0 0x7fd678069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:44.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:44.168+0000 7fd672ffd700 1 -- 172.21.15.31:0/2513916866 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd684084ea0 con 0x7fd68c0cbea0 2023-10-06T02:26:44.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:44.168+0000 7fd68affd700 1 --2- 172.21.15.31:0/2513916866 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fd6780670e0 0x7fd678069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:44.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:44.169+0000 7fd68affd700 1 --2- 172.21.15.31:0/2513916866 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fd6780670e0 0x7fd678069590 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fd67c0091e0 tx=0x7fd67c017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:44.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:44.172+0000 7fd672ffd700 1 -- 172.21.15.31:0/2513916866 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd68c0af280 con 0x7fd68c0cbea0 2023-10-06T02:26:44.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:44.421+0000 7fd68bfff700 1 -- 172.21.15.31:0/2513916866 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fd68c04c3c0 con 0x7fd68c0cbea0 2023-10-06T02:26:45.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:45.086+0000 7fd672ffd700 1 -- 172.21.15.31:0/2513916866 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v36) v1 ==== 118+0+0 (secure 0 0 0) 0x7fd684051c80 con 0x7fd68c0cbea0 2023-10-06T02:26:45.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:45.089+0000 7fd68bfff700 1 -- 172.21.15.31:0/2513916866 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fd6780670e0 msgr2=0x7fd678069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:45.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:45.089+0000 7fd68bfff700 1 --2- 172.21.15.31:0/2513916866 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fd6780670e0 0x7fd678069590 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fd67c0091e0 tx=0x7fd67c017000 comp rx=0 tx=0).stop 2023-10-06T02:26:45.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:45.089+0000 7fd68bfff700 1 -- 172.21.15.31:0/2513916866 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd68c0cbea0 msgr2=0x7fd68c0ade90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:45.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:45.089+0000 7fd68bfff700 1 --2- 172.21.15.31:0/2513916866 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd68c0cbea0 0x7fd68c0ade90 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fd684001460 tx=0x7fd684001a40 comp rx=0 tx=0).stop 2023-10-06T02:26:45.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:45.089+0000 7fd68bfff700 1 -- 172.21.15.31:0/2513916866 shutdown_connections 2023-10-06T02:26:45.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:45.089+0000 7fd68bfff700 1 --2- 172.21.15.31:0/2513916866 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd68c0cbea0 0x7fd68c0ade90 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:45.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:45.089+0000 7fd68bfff700 1 --2- 172.21.15.31:0/2513916866 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fd6780670e0 0x7fd678069590 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:45.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:45.089+0000 7fd68bfff700 1 --2- 172.21.15.31:0/2513916866 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd68c06e330 0x7fd68c0af560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:45.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:45.089+0000 7fd68bfff700 1 --2- 172.21.15.31:0/2513916866 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd68c0accb0 0x7fd68c0ad950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:45.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:45.089+0000 7fd68bfff700 1 -- 172.21.15.31:0/2513916866 >> 172.21.15.31:0/2513916866 conn(0x7fd68c06ce80 msgr2=0x7fd68c06f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:45.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:45.090+0000 7fd68bfff700 1 -- 172.21.15.31:0/2513916866 shutdown_connections 2023-10-06T02:26:45.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:45.090+0000 7fd68bfff700 1 -- 172.21.15.31:0/2513916866 wait complete. 2023-10-06T02:26:45.112 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T02:26:45.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:45.309+0000 7f2dd759e700 1 Processor -- start 2023-10-06T02:26:45.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:45.309+0000 7f2dd759e700 1 -- start start 2023-10-06T02:26:45.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:45.311+0000 7f2dd759e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2dd815dfd0 0x7f2dd80aedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:45.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:45.311+0000 7f2dd759e700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2dd8159920 0x7f2dd80ad090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:45.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:45.311+0000 7f2dd759e700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dd80acc00 con 0x7f2dd8159550 2023-10-06T02:26:45.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:45.311+0000 7f2dd759e700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2dd80abff0 con 0x7f2dd8159920 2023-10-06T02:26:45.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:45.311+0000 7f2dd759e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd80ac2d0 con 0x7f2dd815dfd0 2023-10-06T02:26:45.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:45.311+0000 7f2dd5d9b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2dd815dfd0 0x7f2dd80aedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:45.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:45.311+0000 7f2dd5d9b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2dd815dfd0 0x7f2dd80aedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:40134/0 (socket says 172.21.15.31:40134) 2023-10-06T02:26:45.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:45.311+0000 7f2dd5d9b700 1 -- 172.21.15.31:0/2983279989 learned_addr learned my addr 172.21.15.31:0/2983279989 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:26:45.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:45.311+0000 7f2dd6d9d700 1 --2- 172.21.15.31:0/2983279989 >> v2:172.21.15.62:3300/0 conn(0x7f2dd8159920 0x7f2dd80ad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:45.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:45.312+0000 7f2dd5d9b700 1 -- 172.21.15.31:0/2983279989 >> v2:172.21.15.62:3300/0 conn(0x7f2dd8159920 msgr2=0x7f2dd80ad090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:26:45.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:45.312+0000 7f2dd5d9b700 1 --2- 172.21.15.31:0/2983279989 >> v2:172.21.15.62:3300/0 conn(0x7f2dd8159920 0x7f2dd80ad090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:45.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:45.312+0000 7f2dd559a700 1 -- 172.21.15.31:0/2983279989 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 37218171 0 0) 0x7f2dd80acc00 con 0x7f2dd8159550 2023-10-06T02:26:45.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:45.312+0000 7f2dd5d9b700 1 -- 172.21.15.31:0/2983279989 >> v1:172.21.15.62:6789/0 conn(0x7f2dd8159550 legacy=0x7f2dd80ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:45.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:45.312+0000 7f2dd5d9b700 1 -- 172.21.15.31:0/2983279989 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dd80ac5b0 con 0x7f2dd815dfd0 2023-10-06T02:26:45.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:45.312+0000 7f2dd5d9b700 1 --2- 172.21.15.31:0/2983279989 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2dd815dfd0 0x7f2dd80aedb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc0099b0 tx=0x7f2dcc009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=828b902e272188bf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:45.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:45.312+0000 7f2dd559a700 1 -- 172.21.15.31:0/2983279989 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dcc009ea0 con 0x7f2dd815dfd0 2023-10-06T02:26:45.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:45.312+0000 7f2dd559a700 1 -- 172.21.15.31:0/2983279989 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dcc00b860 con 0x7f2dd815dfd0 2023-10-06T02:26:45.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:45.312+0000 7f2dd559a700 1 -- 172.21.15.31:0/2983279989 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dcc00bc00 con 0x7f2dd815dfd0 2023-10-06T02:26:45.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:45.313+0000 7f2dd759e700 1 -- 172.21.15.31:0/2983279989 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2dd815dfd0 msgr2=0x7f2dd80aedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:45.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:45.313+0000 7f2dd759e700 1 --2- 172.21.15.31:0/2983279989 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2dd815dfd0 0x7f2dd80aedb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc0099b0 tx=0x7f2dcc009250 comp rx=0 tx=0).stop 2023-10-06T02:26:45.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:45.313+0000 7f2dd759e700 1 -- 172.21.15.31:0/2983279989 shutdown_connections 2023-10-06T02:26:45.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:45.313+0000 7f2dd759e700 1 --2- 172.21.15.31:0/2983279989 >> v2:172.21.15.62:3300/0 conn(0x7f2dd8159920 0x7f2dd80ad090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:45.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:45.313+0000 7f2dd759e700 1 --2- 172.21.15.31:0/2983279989 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2dd815dfd0 0x7f2dd80aedb0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:45.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:45.313+0000 7f2dd759e700 1 -- 172.21.15.31:0/2983279989 >> 172.21.15.31:0/2983279989 conn(0x7f2dd806ce80 msgr2=0x7f2dd80ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:45.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:45.313+0000 7f2dd759e700 1 -- 172.21.15.31:0/2983279989 shutdown_connections 2023-10-06T02:26:45.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:45.313+0000 7f2dd759e700 1 -- 172.21.15.31:0/2983279989 wait complete. 2023-10-06T02:26:45.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:45.314+0000 7f2dd759e700 1 Processor -- start 2023-10-06T02:26:45.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:45.315+0000 7f2dd759e700 1 -- start start 2023-10-06T02:26:45.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:45.315+0000 7f2dd759e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2dd8159550 0x7f2dd8151f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:45.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:45.315+0000 7f2dd759e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2dd8159920 0x7f2dd816d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:45.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:45.315+0000 7f2dd759e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2dd815dfd0 0x7f2dd816d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:45.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:45.315+0000 7f2dd759e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd80c7690 con 0x7f2dd8159920 2023-10-06T02:26:45.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:45.315+0000 7f2dd759e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd80b1ce0 con 0x7f2dd815dfd0 2023-10-06T02:26:45.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:45.315+0000 7f2dd759e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd80b1fc0 con 0x7f2dd8159550 2023-10-06T02:26:45.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:45.316+0000 7f2dd5d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2dd8159920 0x7f2dd816d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:45.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:45.316+0000 7f2dd5d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2dd8159920 0x7f2dd816d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49930/0 (socket says 172.21.15.31:49930) 2023-10-06T02:26:45.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:45.316+0000 7f2dd5d9b700 1 -- 172.21.15.31:0/2044963530 learned_addr learned my addr 172.21.15.31:0/2044963530 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:26:45.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:45.316+0000 7f2dd659c700 1 --2- 172.21.15.31:0/2044963530 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2dd8159550 0x7f2dd8151f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:45.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:45.316+0000 7f2dd6d9d700 1 --2- 172.21.15.31:0/2044963530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2dd815dfd0 0x7f2dd816d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:45.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:45.316+0000 7f2dd5d9b700 1 -- 172.21.15.31:0/2044963530 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2dd8159550 msgr2=0x7f2dd8151f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:45.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:45.316+0000 7f2dd5d9b700 1 --2- 172.21.15.31:0/2044963530 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2dd8159550 0x7f2dd8151f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:45.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:45.316+0000 7f2dd5d9b700 1 -- 172.21.15.31:0/2044963530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2dd815dfd0 msgr2=0x7f2dd816d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:45.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:45.316+0000 7f2dd5d9b700 1 --2- 172.21.15.31:0/2044963530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2dd815dfd0 0x7f2dd816d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:45.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:45.316+0000 7f2dd5d9b700 1 -- 172.21.15.31:0/2044963530 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dcc0094f0 con 0x7f2dd8159920 2023-10-06T02:26:45.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:45.316+0000 7f2dd5d9b700 1 --2- 172.21.15.31:0/2044963530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2dd8159920 0x7f2dd816d320 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc009290 tx=0x7f2dcc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:45.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:45.317+0000 7f2dc67fc700 1 -- 172.21.15.31:0/2044963530 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dcc00bbe0 con 0x7f2dd8159920 2023-10-06T02:26:45.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:45.317+0000 7f2dc67fc700 1 -- 172.21.15.31:0/2044963530 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dcc01a450 con 0x7f2dd8159920 2023-10-06T02:26:45.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:45.317+0000 7f2dc67fc700 1 -- 172.21.15.31:0/2044963530 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dcc01a7f0 con 0x7f2dd8159920 2023-10-06T02:26:45.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:45.317+0000 7f2dd759e700 1 -- 172.21.15.31:0/2044963530 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dd80b22a0 con 0x7f2dd8159920 2023-10-06T02:26:45.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:45.317+0000 7f2dd759e700 1 -- 172.21.15.31:0/2044963530 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dd81524b0 con 0x7f2dd8159920 2023-10-06T02:26:45.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:45.318+0000 7f2dd759e700 1 -- 172.21.15.31:0/2044963530 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dd80c7800 con 0x7f2dd8159920 2023-10-06T02:26:45.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:45.319+0000 7f2dc67fc700 1 -- 172.21.15.31:0/2044963530 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 9) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f2dcc01aba0 con 0x7f2dd8159920 2023-10-06T02:26:45.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:45.320+0000 7f2dc67fc700 1 --2- 172.21.15.31:0/2044963530 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f2db80670e0 0x7f2db8069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:45.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:45.320+0000 7f2dc67fc700 1 -- 172.21.15.31:0/2044963530 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2dcc086370 con 0x7f2dd8159920 2023-10-06T02:26:45.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:45.320+0000 7f2dd659c700 1 --2- 172.21.15.31:0/2044963530 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f2db80670e0 0x7f2db8069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:45.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:45.321+0000 7f2dd659c700 1 --2- 172.21.15.31:0/2044963530 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f2db80670e0 0x7f2db8069590 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f2dc80011e0 tx=0x7f2dc8007ce0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:45.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:45.324+0000 7f2dc67fc700 1 -- 172.21.15.31:0/2044963530 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2dcc053160 con 0x7f2dd8159920 2023-10-06T02:26:45.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:45.570+0000 7f2dd759e700 1 -- 172.21.15.31:0/2044963530 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f2dd804c3c0 con 0x7f2dd8159920 2023-10-06T02:26:46.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.091+0000 7f2dc67fc700 1 -- 172.21.15.31:0/2044963530 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v37) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2dcc056780 con 0x7f2dd8159920 2023-10-06T02:26:46.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.094+0000 7f2dd759e700 1 -- 172.21.15.31:0/2044963530 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f2db80670e0 msgr2=0x7f2db8069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:46.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.094+0000 7f2dd759e700 1 --2- 172.21.15.31:0/2044963530 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f2db80670e0 0x7f2db8069590 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f2dc80011e0 tx=0x7f2dc8007ce0 comp rx=0 tx=0).stop 2023-10-06T02:26:46.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.094+0000 7f2dd759e700 1 -- 172.21.15.31:0/2044963530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2dd8159920 msgr2=0x7f2dd816d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:46.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.094+0000 7f2dd759e700 1 --2- 172.21.15.31:0/2044963530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2dd8159920 0x7f2dd816d320 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc009290 tx=0x7f2dcc017000 comp rx=0 tx=0).stop 2023-10-06T02:26:46.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.094+0000 7f2dd759e700 1 -- 172.21.15.31:0/2044963530 shutdown_connections 2023-10-06T02:26:46.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.094+0000 7f2dd759e700 1 --2- 172.21.15.31:0/2044963530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2dd815dfd0 0x7f2dd816d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:46.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.094+0000 7f2dd759e700 1 --2- 172.21.15.31:0/2044963530 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f2db80670e0 0x7f2db8069590 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:46.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.094+0000 7f2dd759e700 1 --2- 172.21.15.31:0/2044963530 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2dd8159550 0x7f2dd8151f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:46.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.094+0000 7f2dd759e700 1 --2- 172.21.15.31:0/2044963530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2dd8159920 0x7f2dd816d320 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:46.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.094+0000 7f2dd759e700 1 -- 172.21.15.31:0/2044963530 >> 172.21.15.31:0/2044963530 conn(0x7f2dd806ce80 msgr2=0x7f2dd8152a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:46.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.094+0000 7f2dd759e700 1 -- 172.21.15.31:0/2044963530 shutdown_connections 2023-10-06T02:26:46.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.094+0000 7f2dd759e700 1 -- 172.21.15.31:0/2044963530 wait complete. 2023-10-06T02:26:46.115 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:26:46.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.304+0000 7f5c2219d700 1 Processor -- start 2023-10-06T02:26:46.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.304+0000 7f5c2219d700 1 -- start start 2023-10-06T02:26:46.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.306+0000 7f5c2219d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f5c1c13fb20 0x7f5c1c15a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:46.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.306+0000 7f5c2219d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5c1c06aaa0 0x7f5c1c0c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:46.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.306+0000 7f5c2219d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c1c15ae70 con 0x7f5c1c13fef0 2023-10-06T02:26:46.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.306+0000 7f5c2219d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c1c06af60 con 0x7f5c1c06aaa0 2023-10-06T02:26:46.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.306+0000 7f5c2219d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c1c06b240 con 0x7f5c1c13fb20 2023-10-06T02:26:46.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.306+0000 7f5c2199c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5c1c06aaa0 0x7f5c1c0c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:46.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.306+0000 7f5c2199c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5c1c06aaa0 0x7f5c1c0c9d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49940/0 (socket says 172.21.15.31:49940) 2023-10-06T02:26:46.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.306+0000 7f5c2199c700 1 -- 172.21.15.31:0/3023368303 learned_addr learned my addr 172.21.15.31:0/3023368303 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:26:46.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.306+0000 7f5c2099a700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f5c1c13fef0 0x7f5c1c0c7990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46686/0 (socket says 172.21.15.31:46686) 2023-10-06T02:26:46.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.306+0000 7f5c2119b700 1 --2- 172.21.15.31:0/3023368303 >> v2:172.21.15.62:3300/0 conn(0x7f5c1c13fb20 0x7f5c1c15a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:46.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.307+0000 7f5c2199c700 1 -- 172.21.15.31:0/3023368303 >> v2:172.21.15.62:3300/0 conn(0x7f5c1c13fb20 msgr2=0x7f5c1c15a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:26:46.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.307+0000 7f5c2199c700 1 --2- 172.21.15.31:0/3023368303 >> v2:172.21.15.62:3300/0 conn(0x7f5c1c13fb20 0x7f5c1c15a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:46.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.307+0000 7f5c2199c700 1 -- 172.21.15.31:0/3023368303 >> v1:172.21.15.31:6789/0 conn(0x7f5c1c13fef0 legacy=0x7f5c1c0c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:46.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.307+0000 7f5c2199c700 1 -- 172.21.15.31:0/3023368303 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c1c06b520 con 0x7f5c1c06aaa0 2023-10-06T02:26:46.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.307+0000 7f5c13fff700 1 -- 172.21.15.31:0/3023368303 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1432411328 0 0) 0x7f5c0c0011d0 con 0x7f5c1c13fef0 2023-10-06T02:26:46.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.307+0000 7f5c2199c700 1 --2- 172.21.15.31:0/3023368303 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c1c06aaa0 0x7f5c1c0c9d00 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c0695b0 tx=0x7f5c1400ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d4e40d883cde7cb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:46.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.307+0000 7f5c13fff700 1 -- 172.21.15.31:0/3023368303 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c14010070 con 0x7f5c1c06aaa0 2023-10-06T02:26:46.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.307+0000 7f5c13fff700 1 -- 172.21.15.31:0/3023368303 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c1400dde0 con 0x7f5c1c06aaa0 2023-10-06T02:26:46.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.307+0000 7f5c13fff700 1 -- 172.21.15.31:0/3023368303 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c1400b320 con 0x7f5c1c06aaa0 2023-10-06T02:26:46.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.308+0000 7f5c2219d700 1 -- 172.21.15.31:0/3023368303 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c1c06aaa0 msgr2=0x7f5c1c0c9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:46.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.308+0000 7f5c2219d700 1 --2- 172.21.15.31:0/3023368303 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c1c06aaa0 0x7f5c1c0c9d00 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c0695b0 tx=0x7f5c1400ba50 comp rx=0 tx=0).stop 2023-10-06T02:26:46.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.308+0000 7f5c2219d700 1 -- 172.21.15.31:0/3023368303 shutdown_connections 2023-10-06T02:26:46.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.308+0000 7f5c2219d700 1 --2- 172.21.15.31:0/3023368303 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c1c06aaa0 0x7f5c1c0c9d00 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:46.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.308+0000 7f5c2219d700 1 --2- 172.21.15.31:0/3023368303 >> v2:172.21.15.62:3300/0 conn(0x7f5c1c13fb20 0x7f5c1c15a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:46.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.308+0000 7f5c2219d700 1 -- 172.21.15.31:0/3023368303 >> 172.21.15.31:0/3023368303 conn(0x7f5c1c06d170 msgr2=0x7f5c1c0cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:46.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.308+0000 7f5c2219d700 1 -- 172.21.15.31:0/3023368303 shutdown_connections 2023-10-06T02:26:46.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.308+0000 7f5c2219d700 1 -- 172.21.15.31:0/3023368303 wait complete. 2023-10-06T02:26:46.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.309+0000 7f5c2219d700 1 Processor -- start 2023-10-06T02:26:46.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.309+0000 7f5c2219d700 1 -- start start 2023-10-06T02:26:46.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.310+0000 7f5c2219d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5c1c06aaa0 0x7f5c1c169ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:46.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.310+0000 7f5c2219d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c1c13fb20 0x7f5c1c13cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:46.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.310+0000 7f5c2219d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c1c13fef0 0x7f5c1c16a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:46.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.310+0000 7f5c2219d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1c16a430 con 0x7f5c1c13fef0 2023-10-06T02:26:46.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.310+0000 7f5c2219d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1c13d010 con 0x7f5c1c13fb20 2023-10-06T02:26:46.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.310+0000 7f5c2219d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1c158640 con 0x7f5c1c06aaa0 2023-10-06T02:26:46.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.311+0000 7f5c2199c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c1c13fef0 0x7f5c1c16a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:46.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.311+0000 7f5c2199c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c1c13fef0 0x7f5c1c16a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49952/0 (socket says 172.21.15.31:49952) 2023-10-06T02:26:46.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.311+0000 7f5c2199c700 1 -- 172.21.15.31:0/3083249391 learned_addr learned my addr 172.21.15.31:0/3083249391 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:26:46.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.311+0000 7f5c2099a700 1 --2- 172.21.15.31:0/3083249391 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c1c13fb20 0x7f5c1c13cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:46.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.311+0000 7f5c2119b700 1 --2- 172.21.15.31:0/3083249391 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5c1c06aaa0 0x7f5c1c169ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:46.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.311+0000 7f5c2199c700 1 -- 172.21.15.31:0/3083249391 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5c1c06aaa0 msgr2=0x7f5c1c169ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:46.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.311+0000 7f5c2199c700 1 --2- 172.21.15.31:0/3083249391 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5c1c06aaa0 0x7f5c1c169ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:46.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.311+0000 7f5c2199c700 1 -- 172.21.15.31:0/3083249391 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c1c13fb20 msgr2=0x7f5c1c13cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:46.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.311+0000 7f5c2199c700 1 --2- 172.21.15.31:0/3083249391 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c1c13fb20 0x7f5c1c13cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:46.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.311+0000 7f5c2199c700 1 -- 172.21.15.31:0/3083249391 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c1400bcf0 con 0x7f5c1c13fef0 2023-10-06T02:26:46.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.311+0000 7f5c2119b700 1 --2- 172.21.15.31:0/3083249391 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5c1c06aaa0 0x7f5c1c169ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:26:46.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.311+0000 7f5c2099a700 1 --2- 172.21.15.31:0/3083249391 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c1c13fb20 0x7f5c1c13cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:26:46.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.311+0000 7f5c2199c700 1 --2- 172.21.15.31:0/3083249391 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c1c13fef0 0x7f5c1c16a690 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f5c14014000 tx=0x7f5c1401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:46.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.312+0000 7f5c117fa700 1 -- 172.21.15.31:0/3083249391 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c1400ac90 con 0x7f5c1c13fef0 2023-10-06T02:26:46.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.312+0000 7f5c117fa700 1 -- 172.21.15.31:0/3083249391 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c1400ae10 con 0x7f5c1c13fef0 2023-10-06T02:26:46.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.312+0000 7f5c117fa700 1 -- 172.21.15.31:0/3083249391 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c1401e690 con 0x7f5c1c13fef0 2023-10-06T02:26:46.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.312+0000 7f5c2219d700 1 -- 172.21.15.31:0/3083249391 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c1c0abbd0 con 0x7f5c1c13fef0 2023-10-06T02:26:46.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.312+0000 7f5c2219d700 1 -- 172.21.15.31:0/3083249391 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c1c0ac190 con 0x7f5c1c13fef0 2023-10-06T02:26:46.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.313+0000 7f5c2219d700 1 -- 172.21.15.31:0/3083249391 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c1c04c3c0 con 0x7f5c1c13fef0 2023-10-06T02:26:46.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.314+0000 7f5c117fa700 1 -- 172.21.15.31:0/3083249391 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 9) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f5c14003040 con 0x7f5c1c13fef0 2023-10-06T02:26:46.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.315+0000 7f5c117fa700 1 --2- 172.21.15.31:0/3083249391 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5c000670e0 0x7f5c00069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:46.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.315+0000 7f5c117fa700 1 -- 172.21.15.31:0/3083249391 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f5c140867d0 con 0x7f5c1c13fef0 2023-10-06T02:26:46.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.315+0000 7f5c2119b700 1 --2- 172.21.15.31:0/3083249391 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5c000670e0 0x7f5c00069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:46.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.316+0000 7f5c2119b700 1 --2- 172.21.15.31:0/3083249391 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5c000670e0 0x7f5c00069590 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f5c18009d80 tx=0x7f5c18008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:46.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.320+0000 7f5c117fa700 1 -- 172.21.15.31:0/3083249391 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5c140534b0 con 0x7f5c1c13fef0 2023-10-06T02:26:46.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.566+0000 7f5c2219d700 1 -- 172.21.15.31:0/3083249391 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5c1c06b800 con 0x7f5c1c13fef0 2023-10-06T02:26:46.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.567+0000 7f5c117fa700 1 -- 172.21.15.31:0/3083249391 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+4595 (secure 0 0 0) 0x7f5c14056ad0 con 0x7f5c1c13fef0 2023-10-06T02:26:46.567 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:26:46.567 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":37,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5216,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/2282788921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2282788921},{"type":"v1","addr":"172.21.15.62:6833","nonce":2282788921}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5219,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/289861443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":289861443},{"type":"v1","addr":"172.21.15.62:6835","nonce":289861443}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":5889,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/2954679985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":2954679985},{"type":"v1","addr":"172.21.15.31:6836","nonce":2954679985}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:26:42.554516+0000","modified":"2023-10-06T02:26:46.085562+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5892},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5892":{"gid":5892,"name":"c","rank":0,"incarnation":33,"state":"up:active","state_seq":5,"addr":"172.21.15.31:6837/1969136630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":1969136630},{"type":"v1","addr":"172.21.15.31:6837","nonce":1969136630}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-06T02:26:46.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.569+0000 7f5c2219d700 1 -- 172.21.15.31:0/3083249391 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5c000670e0 msgr2=0x7f5c00069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:46.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.569+0000 7f5c2219d700 1 --2- 172.21.15.31:0/3083249391 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5c000670e0 0x7f5c00069590 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f5c18009d80 tx=0x7f5c18008040 comp rx=0 tx=0).stop 2023-10-06T02:26:46.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.569+0000 7f5c2219d700 1 -- 172.21.15.31:0/3083249391 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c1c13fef0 msgr2=0x7f5c1c16a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:46.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.569+0000 7f5c2219d700 1 --2- 172.21.15.31:0/3083249391 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c1c13fef0 0x7f5c1c16a690 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f5c14014000 tx=0x7f5c1401a000 comp rx=0 tx=0).stop 2023-10-06T02:26:46.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.569+0000 7f5c2219d700 1 -- 172.21.15.31:0/3083249391 shutdown_connections 2023-10-06T02:26:46.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.569+0000 7f5c2219d700 1 --2- 172.21.15.31:0/3083249391 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c1c13fef0 0x7f5c1c16a690 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:46.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.569+0000 7f5c2219d700 1 --2- 172.21.15.31:0/3083249391 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5c000670e0 0x7f5c00069590 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:46.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.569+0000 7f5c2219d700 1 --2- 172.21.15.31:0/3083249391 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5c1c06aaa0 0x7f5c1c169ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:46.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.569+0000 7f5c2219d700 1 --2- 172.21.15.31:0/3083249391 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c1c13fb20 0x7f5c1c13cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:46.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.569+0000 7f5c2219d700 1 -- 172.21.15.31:0/3083249391 >> 172.21.15.31:0/3083249391 conn(0x7f5c1c06d170 msgr2=0x7f5c1c0b11a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:46.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.569+0000 7f5c2219d700 1 -- 172.21.15.31:0/3083249391 shutdown_connections 2023-10-06T02:26:46.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.569+0000 7f5c2219d700 1 -- 172.21.15.31:0/3083249391 wait complete. 2023-10-06T02:26:46.571 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 37 2023-10-06T02:26:46.587 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:26:46.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.773+0000 7f1c52d9d700 1 Processor -- start 2023-10-06T02:26:46.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.773+0000 7f1c52d9d700 1 -- start start 2023-10-06T02:26:46.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.774+0000 7f1c52d9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1c4c13fb20 0x7f1c4c15a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:46.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.774+0000 7f1c52d9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f1c4c13fef0 0x7f1c4c0c7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:46.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.774+0000 7f1c52d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1c4c06aaa0 0x7f1c4c0c7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:46.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.774+0000 7f1c52d9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c4c0c8410 con 0x7f1c4c13fb20 2023-10-06T02:26:46.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.774+0000 7f1c52d9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c4c15ae70 con 0x7f1c4c13fef0 2023-10-06T02:26:46.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.774+0000 7f1c52d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4c06af60 con 0x7f1c4c06aaa0 2023-10-06T02:26:46.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.774+0000 7f1c51d9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1c4c13fb20 0x7f1c4c15a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:46.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.774+0000 7f1c51d9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1c4c13fb20 0x7f1c4c15a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49962/0 (socket says 172.21.15.31:49962) 2023-10-06T02:26:46.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.774+0000 7f1c51d9b700 1 -- 172.21.15.31:0/2989570545 learned_addr learned my addr 172.21.15.31:0/2989570545 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:26:46.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.775+0000 7f1c5259c700 1 --2- 172.21.15.31:0/2989570545 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1c4c06aaa0 0x7f1c4c0c7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:46.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.775+0000 7f1c5159a700 1 --2- 172.21.15.31:0/2989570545 >> v2:172.21.15.62:3300/0 conn(0x7f1c4c13fef0 0x7f1c4c0c7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:46.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.775+0000 7f1c51d9b700 1 -- 172.21.15.31:0/2989570545 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1c4c06aaa0 msgr2=0x7f1c4c0c7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:26:46.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.775+0000 7f1c51d9b700 1 --2- 172.21.15.31:0/2989570545 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1c4c06aaa0 0x7f1c4c0c7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:46.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.775+0000 7f1c51d9b700 1 -- 172.21.15.31:0/2989570545 >> v2:172.21.15.62:3300/0 conn(0x7f1c4c13fef0 msgr2=0x7f1c4c0c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:26:46.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.775+0000 7f1c51d9b700 1 --2- 172.21.15.31:0/2989570545 >> v2:172.21.15.62:3300/0 conn(0x7f1c4c13fef0 0x7f1c4c0c7990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:46.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.775+0000 7f1c51d9b700 1 -- 172.21.15.31:0/2989570545 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c4c06b240 con 0x7f1c4c13fb20 2023-10-06T02:26:46.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.775+0000 7f1c5159a700 1 --2- 172.21.15.31:0/2989570545 >> v2:172.21.15.62:3300/0 conn(0x7f1c4c13fef0 0x7f1c4c0c7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:26:46.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.775+0000 7f1c51d9b700 1 --2- 172.21.15.31:0/2989570545 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c4c13fb20 0x7f1c4c15a930 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c06a5e0 tx=0x7f1c48009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98dfef42c82cdab9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:46.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.775+0000 7f1c50d99700 1 -- 172.21.15.31:0/2989570545 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c4800b860 con 0x7f1c4c13fb20 2023-10-06T02:26:46.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.775+0000 7f1c50d99700 1 -- 172.21.15.31:0/2989570545 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c4800b9c0 con 0x7f1c4c13fb20 2023-10-06T02:26:46.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.775+0000 7f1c50d99700 1 -- 172.21.15.31:0/2989570545 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c4800bd60 con 0x7f1c4c13fb20 2023-10-06T02:26:46.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.776+0000 7f1c52d9d700 1 -- 172.21.15.31:0/2989570545 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c4c13fb20 msgr2=0x7f1c4c15a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:46.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.776+0000 7f1c52d9d700 1 --2- 172.21.15.31:0/2989570545 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c4c13fb20 0x7f1c4c15a930 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c06a5e0 tx=0x7f1c48009250 comp rx=0 tx=0).stop 2023-10-06T02:26:46.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.776+0000 7f1c52d9d700 1 -- 172.21.15.31:0/2989570545 shutdown_connections 2023-10-06T02:26:46.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.776+0000 7f1c52d9d700 1 --2- 172.21.15.31:0/2989570545 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1c4c06aaa0 0x7f1c4c0c7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:46.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.776+0000 7f1c52d9d700 1 --2- 172.21.15.31:0/2989570545 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c4c13fb20 0x7f1c4c15a930 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:46.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.776+0000 7f1c52d9d700 1 --2- 172.21.15.31:0/2989570545 >> v2:172.21.15.62:3300/0 conn(0x7f1c4c13fef0 0x7f1c4c0c7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:46.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.776+0000 7f1c52d9d700 1 -- 172.21.15.31:0/2989570545 >> 172.21.15.31:0/2989570545 conn(0x7f1c4c06d170 msgr2=0x7f1c4c0cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:46.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.776+0000 7f1c52d9d700 1 -- 172.21.15.31:0/2989570545 shutdown_connections 2023-10-06T02:26:46.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.776+0000 7f1c52d9d700 1 -- 172.21.15.31:0/2989570545 wait complete. 2023-10-06T02:26:46.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.778+0000 7f1c52d9d700 1 Processor -- start 2023-10-06T02:26:46.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.778+0000 7f1c52d9d700 1 -- start start 2023-10-06T02:26:46.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.779+0000 7f1c52d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c4c06aaa0 0x7f1c4c15d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:46.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.779+0000 7f1c52d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c4c13fb20 0x7f1c4c0b1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:46.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.779+0000 7f1c52d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1c4c13fef0 0x7f1c4c155870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:46.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.779+0000 7f1c52d9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4c159b30 con 0x7f1c4c13fb20 2023-10-06T02:26:46.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.779+0000 7f1c52d9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4c13bf50 con 0x7f1c4c06aaa0 2023-10-06T02:26:46.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.779+0000 7f1c52d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4c15d7a0 con 0x7f1c4c13fef0 2023-10-06T02:26:46.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.779+0000 7f1c5159a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c4c13fb20 0x7f1c4c0b1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:46.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.779+0000 7f1c5159a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c4c13fb20 0x7f1c4c0b1280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49972/0 (socket says 172.21.15.31:49972) 2023-10-06T02:26:46.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.779+0000 7f1c5159a700 1 -- 172.21.15.31:0/1965454945 learned_addr learned my addr 172.21.15.31:0/1965454945 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:26:46.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.779+0000 7f1c51d9b700 1 --2- 172.21.15.31:0/1965454945 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c4c06aaa0 0x7f1c4c15d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:46.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.779+0000 7f1c5259c700 1 --2- 172.21.15.31:0/1965454945 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1c4c13fef0 0x7f1c4c155870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:46.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.779+0000 7f1c5159a700 1 -- 172.21.15.31:0/1965454945 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1c4c13fef0 msgr2=0x7f1c4c155870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:46.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.779+0000 7f1c5159a700 1 --2- 172.21.15.31:0/1965454945 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1c4c13fef0 0x7f1c4c155870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:46.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.779+0000 7f1c5159a700 1 -- 172.21.15.31:0/1965454945 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c4c06aaa0 msgr2=0x7f1c4c15d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:46.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.779+0000 7f1c5159a700 1 --2- 172.21.15.31:0/1965454945 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c4c06aaa0 0x7f1c4c15d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:46.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.779+0000 7f1c5159a700 1 -- 172.21.15.31:0/1965454945 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c480094f0 con 0x7f1c4c13fb20 2023-10-06T02:26:46.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.779+0000 7f1c51d9b700 1 --2- 172.21.15.31:0/1965454945 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c4c06aaa0 0x7f1c4c15d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:26:46.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.780+0000 7f1c5159a700 1 --2- 172.21.15.31:0/1965454945 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c4c13fb20 0x7f1c4c0b1280 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c00c880 tx=0x7f1c3c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:46.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.780+0000 7f1c41ffb700 1 -- 172.21.15.31:0/1965454945 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c3c005a00 con 0x7f1c4c13fb20 2023-10-06T02:26:46.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.780+0000 7f1c41ffb700 1 -- 172.21.15.31:0/1965454945 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c3c005b60 con 0x7f1c4c13fb20 2023-10-06T02:26:46.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.780+0000 7f1c41ffb700 1 -- 172.21.15.31:0/1965454945 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c3c011600 con 0x7f1c4c13fb20 2023-10-06T02:26:46.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.780+0000 7f1c52d9d700 1 -- 172.21.15.31:0/1965454945 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c4c15da80 con 0x7f1c4c13fb20 2023-10-06T02:26:46.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.780+0000 7f1c52d9d700 1 -- 172.21.15.31:0/1965454945 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c4c15e040 con 0x7f1c4c13fb20 2023-10-06T02:26:46.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.781+0000 7f1c52d9d700 1 -- 172.21.15.31:0/1965454945 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c4c15e260 con 0x7f1c4c13fb20 2023-10-06T02:26:46.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.782+0000 7f1c41ffb700 1 -- 172.21.15.31:0/1965454945 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 9) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f1c3c0119b0 con 0x7f1c4c13fb20 2023-10-06T02:26:46.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.783+0000 7f1c41ffb700 1 --2- 172.21.15.31:0/1965454945 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f1c300670e0 0x7f1c30069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:46.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.783+0000 7f1c41ffb700 1 -- 172.21.15.31:0/1965454945 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1c3c0849f0 con 0x7f1c4c13fb20 2023-10-06T02:26:46.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.783+0000 7f1c51d9b700 1 --2- 172.21.15.31:0/1965454945 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f1c300670e0 0x7f1c30069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:46.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.784+0000 7f1c51d9b700 1 --2- 172.21.15.31:0/1965454945 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f1c300670e0 0x7f1c30069590 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c06a5e0 tx=0x7f1c48017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:46.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.787+0000 7f1c41ffb700 1 -- 172.21.15.31:0/1965454945 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1c3c0516d0 con 0x7f1c4c13fb20 2023-10-06T02:26:46.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.993+0000 7f1c52d9d700 1 -- 172.21.15.31:0/1965454945 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1c4c04c3c0 con 0x7f1c4c13fb20 2023-10-06T02:26:46.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.994+0000 7f1c41ffb700 1 -- 172.21.15.31:0/1965454945 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+16820 (secure 0 0 0) 0x7f1c3c01e090 con 0x7f1c4c13fb20 2023-10-06T02:26:46.995 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:26:46.995 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":61,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:26:44.066430+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:26:40.728441+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-10-06T02:26:41.524170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:26:46.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.997+0000 7f1c52d9d700 1 -- 172.21.15.31:0/1965454945 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f1c300670e0 msgr2=0x7f1c30069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:46.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.997+0000 7f1c52d9d700 1 --2- 172.21.15.31:0/1965454945 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f1c300670e0 0x7f1c30069590 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c06a5e0 tx=0x7f1c48017000 comp rx=0 tx=0).stop 2023-10-06T02:26:46.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.997+0000 7f1c52d9d700 1 -- 172.21.15.31:0/1965454945 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c4c13fb20 msgr2=0x7f1c4c0b1280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:46.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.997+0000 7f1c52d9d700 1 --2- 172.21.15.31:0/1965454945 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c4c13fb20 0x7f1c4c0b1280 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c00c880 tx=0x7f1c3c008040 comp rx=0 tx=0).stop 2023-10-06T02:26:46.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.997+0000 7f1c52d9d700 1 -- 172.21.15.31:0/1965454945 shutdown_connections 2023-10-06T02:26:46.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.997+0000 7f1c52d9d700 1 --2- 172.21.15.31:0/1965454945 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f1c300670e0 0x7f1c30069590 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:46.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.997+0000 7f1c52d9d700 1 --2- 172.21.15.31:0/1965454945 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1c4c13fef0 0x7f1c4c155870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:46.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.997+0000 7f1c52d9d700 1 --2- 172.21.15.31:0/1965454945 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c4c06aaa0 0x7f1c4c15d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:46.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.997+0000 7f1c52d9d700 1 --2- 172.21.15.31:0/1965454945 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c4c13fb20 0x7f1c4c0b1280 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:46.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.997+0000 7f1c52d9d700 1 -- 172.21.15.31:0/1965454945 >> 172.21.15.31:0/1965454945 conn(0x7f1c4c06d170 msgr2=0x7f1c4c0cabb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:46.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.997+0000 7f1c52d9d700 1 -- 172.21.15.31:0/1965454945 shutdown_connections 2023-10-06T02:26:46.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:46.997+0000 7f1c52d9d700 1 -- 172.21.15.31:0/1965454945 wait complete. 2023-10-06T02:26:47.015 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T02:26:47.015 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T02:26:47.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.208+0000 7fce30d9d700 1 Processor -- start 2023-10-06T02:26:47.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.208+0000 7fce30d9d700 1 -- start start 2023-10-06T02:26:47.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.209+0000 7fce30d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce2c159550 0x7fce2c0ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:47.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.209+0000 7fce30d9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fce2c15dfd0 0x7fce2c0aedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:47.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.209+0000 7fce30d9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce2c0acc00 con 0x7fce2c159920 2023-10-06T02:26:47.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.209+0000 7fce30d9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce2c0abff0 con 0x7fce2c15dfd0 2023-10-06T02:26:47.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.209+0000 7fce30d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce2c0ac2d0 con 0x7fce2c159550 2023-10-06T02:26:47.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.209+0000 7fce2affd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fce2c15dfd0 0x7fce2c0aedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:47.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.209+0000 7fce2affd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fce2c15dfd0 0x7fce2c0aedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49980/0 (socket says 172.21.15.31:49980) 2023-10-06T02:26:47.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.209+0000 7fce2affd700 1 -- 172.21.15.31:0/1045699152 learned_addr learned my addr 172.21.15.31:0/1045699152 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:26:47.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.210+0000 7fce2b7fe700 1 --2- 172.21.15.31:0/1045699152 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce2c159550 0x7fce2c0ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:47.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.210+0000 7fce2affd700 1 -- 172.21.15.31:0/1045699152 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce2c159550 msgr2=0x7fce2c0ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:26:47.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.210+0000 7fce2affd700 1 --2- 172.21.15.31:0/1045699152 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce2c159550 0x7fce2c0ceac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:47.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.210+0000 7fce2affd700 1 -- 172.21.15.31:0/1045699152 >> v1:172.21.15.62:6789/0 conn(0x7fce2c159920 legacy=0x7fce2c0ad090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:47.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.210+0000 7fce2affd700 1 -- 172.21.15.31:0/1045699152 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce2c0ac5b0 con 0x7fce2c15dfd0 2023-10-06T02:26:47.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.210+0000 7fce2affd700 1 --2- 172.21.15.31:0/1045699152 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce2c15dfd0 0x7fce2c0aedb0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fce200099b0 tx=0x7fce20009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84ffe64be4d3884a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:47.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.210+0000 7fce2a7fc700 1 -- 172.21.15.31:0/1045699152 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2000b860 con 0x7fce2c15dfd0 2023-10-06T02:26:47.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.210+0000 7fce2a7fc700 1 -- 172.21.15.31:0/1045699152 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce2000b9c0 con 0x7fce2c15dfd0 2023-10-06T02:26:47.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.211+0000 7fce2a7fc700 1 -- 172.21.15.31:0/1045699152 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2000bd60 con 0x7fce2c15dfd0 2023-10-06T02:26:47.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.211+0000 7fce30d9d700 1 -- 172.21.15.31:0/1045699152 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce2c15dfd0 msgr2=0x7fce2c0aedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:47.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.211+0000 7fce30d9d700 1 --2- 172.21.15.31:0/1045699152 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce2c15dfd0 0x7fce2c0aedb0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fce200099b0 tx=0x7fce20009250 comp rx=0 tx=0).stop 2023-10-06T02:26:47.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.211+0000 7fce30d9d700 1 -- 172.21.15.31:0/1045699152 shutdown_connections 2023-10-06T02:26:47.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.211+0000 7fce30d9d700 1 --2- 172.21.15.31:0/1045699152 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce2c159550 0x7fce2c0ceac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:47.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.211+0000 7fce30d9d700 1 --2- 172.21.15.31:0/1045699152 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce2c15dfd0 0x7fce2c0aedb0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:47.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.211+0000 7fce30d9d700 1 -- 172.21.15.31:0/1045699152 >> 172.21.15.31:0/1045699152 conn(0x7fce2c06ce80 msgr2=0x7fce2c0ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:47.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.211+0000 7fce30d9d700 1 -- 172.21.15.31:0/1045699152 shutdown_connections 2023-10-06T02:26:47.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.211+0000 7fce30d9d700 1 -- 172.21.15.31:0/1045699152 wait complete. 2023-10-06T02:26:47.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.212+0000 7fce30d9d700 1 Processor -- start 2023-10-06T02:26:47.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.213+0000 7fce30d9d700 1 -- start start 2023-10-06T02:26:47.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.213+0000 7fce30d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce2c159550 0x7fce2c0c8f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:47.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.213+0000 7fce30d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce2c159920 0x7fce2c0afae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:47.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.214+0000 7fce30d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce2c15dfd0 0x7fce2c0b0020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:47.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.214+0000 7fce30d9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce2c155ab0 con 0x7fce2c15dfd0 2023-10-06T02:26:47.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.214+0000 7fce30d9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce2c0cbf60 con 0x7fce2c159550 2023-10-06T02:26:47.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.214+0000 7fce30d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce2c154ea0 con 0x7fce2c159920 2023-10-06T02:26:47.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.214+0000 7fce2bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce2c15dfd0 0x7fce2c0b0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:47.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.214+0000 7fce2bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce2c15dfd0 0x7fce2c0b0020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49994/0 (socket says 172.21.15.31:49994) 2023-10-06T02:26:47.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.214+0000 7fce2bfff700 1 -- 172.21.15.31:0/2531205922 learned_addr learned my addr 172.21.15.31:0/2531205922 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:26:47.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.214+0000 7fce2affd700 1 --2- 172.21.15.31:0/2531205922 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce2c159920 0x7fce2c0afae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:47.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.214+0000 7fce2b7fe700 1 --2- 172.21.15.31:0/2531205922 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce2c159550 0x7fce2c0c8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:47.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.214+0000 7fce2bfff700 1 -- 172.21.15.31:0/2531205922 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce2c159920 msgr2=0x7fce2c0afae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:47.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.214+0000 7fce2bfff700 1 --2- 172.21.15.31:0/2531205922 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce2c159920 0x7fce2c0afae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:47.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.214+0000 7fce2bfff700 1 -- 172.21.15.31:0/2531205922 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce2c159550 msgr2=0x7fce2c0c8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:47.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.214+0000 7fce2bfff700 1 --2- 172.21.15.31:0/2531205922 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce2c159550 0x7fce2c0c8f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:47.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.214+0000 7fce2bfff700 1 -- 172.21.15.31:0/2531205922 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce200094f0 con 0x7fce2c15dfd0 2023-10-06T02:26:47.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.215+0000 7fce2bfff700 1 --2- 172.21.15.31:0/2531205922 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce2c15dfd0 0x7fce2c0b0020 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fce2400c8e0 tx=0x7fce24001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:47.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.215+0000 7fce137fe700 1 -- 172.21.15.31:0/2531205922 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce24012630 con 0x7fce2c15dfd0 2023-10-06T02:26:47.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.215+0000 7fce137fe700 1 -- 172.21.15.31:0/2531205922 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce24012790 con 0x7fce2c15dfd0 2023-10-06T02:26:47.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.215+0000 7fce137fe700 1 -- 172.21.15.31:0/2531205922 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce24012b50 con 0x7fce2c15dfd0 2023-10-06T02:26:47.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.215+0000 7fce30d9d700 1 -- 172.21.15.31:0/2531205922 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce2c155180 con 0x7fce2c15dfd0 2023-10-06T02:26:47.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.215+0000 7fce30d9d700 1 -- 172.21.15.31:0/2531205922 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce2c155740 con 0x7fce2c15dfd0 2023-10-06T02:26:47.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.216+0000 7fce30d9d700 1 -- 172.21.15.31:0/2531205922 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce2c155c20 con 0x7fce2c15dfd0 2023-10-06T02:26:47.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.217+0000 7fce137fe700 1 -- 172.21.15.31:0/2531205922 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 9) v1 ==== 84893+0+0 (secure 0 0 0) 0x7fce2401a3a0 con 0x7fce2c15dfd0 2023-10-06T02:26:47.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.218+0000 7fce137fe700 1 --2- 172.21.15.31:0/2531205922 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fce180670e0 0x7fce18069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:47.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.218+0000 7fce137fe700 1 -- 172.21.15.31:0/2531205922 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fce24084c50 con 0x7fce2c15dfd0 2023-10-06T02:26:47.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.218+0000 7fce2b7fe700 1 --2- 172.21.15.31:0/2531205922 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fce180670e0 0x7fce18069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:47.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.219+0000 7fce2b7fe700 1 --2- 172.21.15.31:0/2531205922 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fce180670e0 0x7fce18069590 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fce1c006fd0 tx=0x7fce1c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:47.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.222+0000 7fce137fe700 1 -- 172.21.15.31:0/2531205922 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fce24051a40 con 0x7fce2c15dfd0 2023-10-06T02:26:47.285 INFO:tasks.ceph.mds.c.smithi031.stderr:2023-10-06T02:26:47.285+0000 7f31bfff1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T02:26:47.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.420+0000 7fce30d9d700 1 -- 172.21.15.31:0/2531205922 --> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fce2c04c3c0 con 0x7fce180670e0 2023-10-06T02:26:47.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.431+0000 7fce137fe700 1 -- 172.21.15.31:0/2531205922 <== mgr.4107 v2:172.21.15.31:6832/85292 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311109 (secure 0 0 0) 0x7fce2c04c3c0 con 0x7fce180670e0 2023-10-06T02:26:47.435 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:26:47.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.436+0000 7fce30d9d700 1 -- 172.21.15.31:0/2531205922 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fce180670e0 msgr2=0x7fce18069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:47.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.436+0000 7fce30d9d700 1 --2- 172.21.15.31:0/2531205922 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fce180670e0 0x7fce18069590 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fce1c006fd0 tx=0x7fce1c008040 comp rx=0 tx=0).stop 2023-10-06T02:26:47.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.436+0000 7fce30d9d700 1 -- 172.21.15.31:0/2531205922 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce2c15dfd0 msgr2=0x7fce2c0b0020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:47.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.436+0000 7fce30d9d700 1 --2- 172.21.15.31:0/2531205922 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce2c15dfd0 0x7fce2c0b0020 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fce2400c8e0 tx=0x7fce24001710 comp rx=0 tx=0).stop 2023-10-06T02:26:47.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.436+0000 7fce30d9d700 1 -- 172.21.15.31:0/2531205922 shutdown_connections 2023-10-06T02:26:47.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.436+0000 7fce30d9d700 1 --2- 172.21.15.31:0/2531205922 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce2c15dfd0 0x7fce2c0b0020 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:47.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.436+0000 7fce30d9d700 1 --2- 172.21.15.31:0/2531205922 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce2c159550 0x7fce2c0c8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:47.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.436+0000 7fce30d9d700 1 --2- 172.21.15.31:0/2531205922 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fce180670e0 0x7fce18069590 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:47.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.436+0000 7fce30d9d700 1 --2- 172.21.15.31:0/2531205922 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce2c159920 0x7fce2c0afae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:47.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.436+0000 7fce30d9d700 1 -- 172.21.15.31:0/2531205922 >> 172.21.15.31:0/2531205922 conn(0x7fce2c06ce80 msgr2=0x7fce2c0b3310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:47.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.436+0000 7fce30d9d700 1 -- 172.21.15.31:0/2531205922 shutdown_connections 2023-10-06T02:26:47.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.436+0000 7fce30d9d700 1 -- 172.21.15.31:0/2531205922 wait complete. 2023-10-06T02:26:47.439 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-10-06T02:26:47.454 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":171,"stamp":"2023-10-06T02:26:47.165810+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":117,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2397304,"kb_used_data":6712,"kb_used_omap":0,"kb_used_meta":2390528,"kb_avail":752577416,"statfs":{"total":773094113280,"available":770639273984,"internally_reserved":0,"allocated":6873088,"data_stored":3649766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2447900672},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001164"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.091655+0000","last_change":"2023-10-06T02:26:43.091655+0000","last_active":"2023-10-06T02:26:43.091655+0000","last_peered":"2023-10-06T02:26:43.091655+0000","last_clean":"2023-10-06T02:26:43.091655+0000","last_became_active":"2023-10-06T02:26:43.091300+0000","last_became_peered":"2023-10-06T02:26:43.091300+0000","last_unstale":"2023-10-06T02:26:43.091655+0000","last_undegraded":"2023-10-06T02:26:43.091655+0000","last_fullsized":"2023-10-06T02:26:43.091655+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:57:39.226225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.083676+0000","last_change":"2023-10-06T02:26:43.083676+0000","last_active":"2023-10-06T02:26:43.083676+0000","last_peered":"2023-10-06T02:26:43.083676+0000","last_clean":"2023-10-06T02:26:43.083676+0000","last_became_active":"2023-10-06T02:26:43.083314+0000","last_became_peered":"2023-10-06T02:26:43.083314+0000","last_unstale":"2023-10-06T02:26:43.083676+0000","last_undegraded":"2023-10-06T02:26:43.083676+0000","last_fullsized":"2023-10-06T02:26:43.083676+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:42:27.666997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.086723+0000","last_change":"2023-10-06T02:26:43.086723+0000","last_active":"2023-10-06T02:26:43.086723+0000","last_peered":"2023-10-06T02:26:43.086723+0000","last_clean":"2023-10-06T02:26:43.086723+0000","last_became_active":"2023-10-06T02:26:43.086430+0000","last_became_peered":"2023-10-06T02:26:43.086430+0000","last_unstale":"2023-10-06T02:26:43.086723+0000","last_undegraded":"2023-10-06T02:26:43.086723+0000","last_fullsized":"2023-10-06T02:26:43.086723+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:50.341025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.067259+0000","last_change":"2023-10-06T02:26:43.067259+0000","last_active":"2023-10-06T02:26:43.067259+0000","last_peered":"2023-10-06T02:26:43.067259+0000","last_clean":"2023-10-06T02:26:43.067259+0000","last_became_active":"2023-10-06T02:26:43.066969+0000","last_became_peered":"2023-10-06T02:26:43.066969+0000","last_unstale":"2023-10-06T02:26:43.067259+0000","last_undegraded":"2023-10-06T02:26:43.067259+0000","last_fullsized":"2023-10-06T02:26:43.067259+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:31:56.002363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.089664+0000","last_change":"2023-10-06T02:26:43.089664+0000","last_active":"2023-10-06T02:26:43.089664+0000","last_peered":"2023-10-06T02:26:43.089664+0000","last_clean":"2023-10-06T02:26:43.089664+0000","last_became_active":"2023-10-06T02:26:43.088984+0000","last_became_peered":"2023-10-06T02:26:43.088984+0000","last_unstale":"2023-10-06T02:26:43.089664+0000","last_undegraded":"2023-10-06T02:26:43.089664+0000","last_fullsized":"2023-10-06T02:26:43.089664+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:42:27.666997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.083507+0000","last_change":"2023-10-06T02:26:43.083507+0000","last_active":"2023-10-06T02:26:43.083507+0000","last_peered":"2023-10-06T02:26:43.083507+0000","last_clean":"2023-10-06T02:26:43.083507+0000","last_became_active":"2023-10-06T02:26:43.083248+0000","last_became_peered":"2023-10-06T02:26:43.083248+0000","last_unstale":"2023-10-06T02:26:43.083507+0000","last_undegraded":"2023-10-06T02:26:43.083507+0000","last_fullsized":"2023-10-06T02:26:43.083507+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:04:15.890422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.085012+0000","last_change":"2023-10-06T02:26:43.085012+0000","last_active":"2023-10-06T02:26:43.085012+0000","last_peered":"2023-10-06T02:26:43.085012+0000","last_clean":"2023-10-06T02:26:43.085012+0000","last_became_active":"2023-10-06T02:26:43.084737+0000","last_became_peered":"2023-10-06T02:26:43.084737+0000","last_unstale":"2023-10-06T02:26:43.085012+0000","last_undegraded":"2023-10-06T02:26:43.085012+0000","last_fullsized":"2023-10-06T02:26:43.085012+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:48:13.211733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.093672+0000","last_change":"2023-10-06T02:26:43.093672+0000","last_active":"2023-10-06T02:26:43.093672+0000","last_peered":"2023-10-06T02:26:43.093672+0000","last_clean":"2023-10-06T02:26:43.093672+0000","last_became_active":"2023-10-06T02:26:43.093410+0000","last_became_peered":"2023-10-06T02:26:43.093410+0000","last_unstale":"2023-10-06T02:26:43.093672+0000","last_undegraded":"2023-10-06T02:26:43.093672+0000","last_fullsized":"2023-10-06T02:26:43.093672+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:35:02.657273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.095037+0000","last_change":"2023-10-06T02:26:43.095037+0000","last_active":"2023-10-06T02:26:43.095037+0000","last_peered":"2023-10-06T02:26:43.095037+0000","last_clean":"2023-10-06T02:26:43.095037+0000","last_became_active":"2023-10-06T02:26:43.093613+0000","last_became_peered":"2023-10-06T02:26:43.093613+0000","last_unstale":"2023-10-06T02:26:43.095037+0000","last_undegraded":"2023-10-06T02:26:43.095037+0000","last_fullsized":"2023-10-06T02:26:43.095037+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:53:29.330021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.094347+0000","last_change":"2023-10-06T02:26:43.094347+0000","last_active":"2023-10-06T02:26:43.094347+0000","last_peered":"2023-10-06T02:26:43.094347+0000","last_clean":"2023-10-06T02:26:43.094347+0000","last_became_active":"2023-10-06T02:26:43.094000+0000","last_became_peered":"2023-10-06T02:26:43.094000+0000","last_unstale":"2023-10-06T02:26:43.094347+0000","last_undegraded":"2023-10-06T02:26:43.094347+0000","last_fullsized":"2023-10-06T02:26:43.094347+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:53:54.198169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.087392+0000","last_change":"2023-10-06T02:26:43.087392+0000","last_active":"2023-10-06T02:26:43.087392+0000","last_peered":"2023-10-06T02:26:43.087392+0000","last_clean":"2023-10-06T02:26:43.087392+0000","last_became_active":"2023-10-06T02:26:43.087054+0000","last_became_peered":"2023-10-06T02:26:43.087054+0000","last_unstale":"2023-10-06T02:26:43.087392+0000","last_undegraded":"2023-10-06T02:26:43.087392+0000","last_fullsized":"2023-10-06T02:26:43.087392+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:52:16.520095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.080654+0000","last_change":"2023-10-06T02:26:43.080654+0000","last_active":"2023-10-06T02:26:43.080654+0000","last_peered":"2023-10-06T02:26:43.080654+0000","last_clean":"2023-10-06T02:26:43.080654+0000","last_became_active":"2023-10-06T02:26:43.080305+0000","last_became_peered":"2023-10-06T02:26:43.080305+0000","last_unstale":"2023-10-06T02:26:43.080654+0000","last_undegraded":"2023-10-06T02:26:43.080654+0000","last_fullsized":"2023-10-06T02:26:43.080654+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:44:18.266240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.079644+0000","last_change":"2023-10-06T02:26:43.079644+0000","last_active":"2023-10-06T02:26:43.079644+0000","last_peered":"2023-10-06T02:26:43.079644+0000","last_clean":"2023-10-06T02:26:43.079644+0000","last_became_active":"2023-10-06T02:26:43.079345+0000","last_became_peered":"2023-10-06T02:26:43.079345+0000","last_unstale":"2023-10-06T02:26:43.079644+0000","last_undegraded":"2023-10-06T02:26:43.079644+0000","last_fullsized":"2023-10-06T02:26:43.079644+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:17:56.058037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.093806+0000","last_change":"2023-10-06T02:26:43.093806+0000","last_active":"2023-10-06T02:26:43.093806+0000","last_peered":"2023-10-06T02:26:43.093806+0000","last_clean":"2023-10-06T02:26:43.093806+0000","last_became_active":"2023-10-06T02:26:43.092556+0000","last_became_peered":"2023-10-06T02:26:43.092556+0000","last_unstale":"2023-10-06T02:26:43.093806+0000","last_undegraded":"2023-10-06T02:26:43.093806+0000","last_fullsized":"2023-10-06T02:26:43.093806+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:48:32.579296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.095209+0000","last_change":"2023-10-06T02:26:43.095209+0000","last_active":"2023-10-06T02:26:43.095209+0000","last_peered":"2023-10-06T02:26:43.095209+0000","last_clean":"2023-10-06T02:26:43.095209+0000","last_became_active":"2023-10-06T02:26:43.094895+0000","last_became_peered":"2023-10-06T02:26:43.094895+0000","last_unstale":"2023-10-06T02:26:43.095209+0000","last_undegraded":"2023-10-06T02:26:43.095209+0000","last_fullsized":"2023-10-06T02:26:43.095209+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:08:29.480329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.080237+0000","last_change":"2023-10-06T02:26:43.080237+0000","last_active":"2023-10-06T02:26:43.080237+0000","last_peered":"2023-10-06T02:26:43.080237+0000","last_clean":"2023-10-06T02:26:43.080237+0000","last_became_active":"2023-10-06T02:26:43.079977+0000","last_became_peered":"2023-10-06T02:26:43.079977+0000","last_unstale":"2023-10-06T02:26:43.080237+0000","last_undegraded":"2023-10-06T02:26:43.080237+0000","last_fullsized":"2023-10-06T02:26:43.080237+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:35:29.322161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.082908+0000","last_change":"2023-10-06T02:26:43.082908+0000","last_active":"2023-10-06T02:26:43.082908+0000","last_peered":"2023-10-06T02:26:43.082908+0000","last_clean":"2023-10-06T02:26:43.082908+0000","last_became_active":"2023-10-06T02:26:43.082623+0000","last_became_peered":"2023-10-06T02:26:43.082623+0000","last_unstale":"2023-10-06T02:26:43.082908+0000","last_undegraded":"2023-10-06T02:26:43.082908+0000","last_fullsized":"2023-10-06T02:26:43.082908+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:44:24.572462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.083330+0000","last_change":"2023-10-06T02:26:43.083330+0000","last_active":"2023-10-06T02:26:43.083330+0000","last_peered":"2023-10-06T02:26:43.083330+0000","last_clean":"2023-10-06T02:26:43.083330+0000","last_became_active":"2023-10-06T02:26:43.083034+0000","last_became_peered":"2023-10-06T02:26:43.083034+0000","last_unstale":"2023-10-06T02:26:43.083330+0000","last_undegraded":"2023-10-06T02:26:43.083330+0000","last_fullsized":"2023-10-06T02:26:43.083330+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:44:24.572462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.084693+0000","last_change":"2023-10-06T02:26:43.084693+0000","last_active":"2023-10-06T02:26:43.084693+0000","last_peered":"2023-10-06T02:26:43.084693+0000","last_clean":"2023-10-06T02:26:43.084693+0000","last_became_active":"2023-10-06T02:26:43.084126+0000","last_became_peered":"2023-10-06T02:26:43.084126+0000","last_unstale":"2023-10-06T02:26:43.084693+0000","last_undegraded":"2023-10-06T02:26:43.084693+0000","last_fullsized":"2023-10-06T02:26:43.084693+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:52.427053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.084201+0000","last_change":"2023-10-06T02:26:43.084201+0000","last_active":"2023-10-06T02:26:43.084201+0000","last_peered":"2023-10-06T02:26:43.084201+0000","last_clean":"2023-10-06T02:26:43.084201+0000","last_became_active":"2023-10-06T02:26:43.083885+0000","last_became_peered":"2023-10-06T02:26:43.083885+0000","last_unstale":"2023-10-06T02:26:43.084201+0000","last_undegraded":"2023-10-06T02:26:43.084201+0000","last_fullsized":"2023-10-06T02:26:43.084201+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:40.671226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.080394+0000","last_change":"2023-10-06T02:26:43.080394+0000","last_active":"2023-10-06T02:26:43.080394+0000","last_peered":"2023-10-06T02:26:43.080394+0000","last_clean":"2023-10-06T02:26:43.080394+0000","last_became_active":"2023-10-06T02:26:43.079980+0000","last_became_peered":"2023-10-06T02:26:43.079980+0000","last_unstale":"2023-10-06T02:26:43.080394+0000","last_undegraded":"2023-10-06T02:26:43.080394+0000","last_fullsized":"2023-10-06T02:26:43.080394+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:57:20.411577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.083271+0000","last_change":"2023-10-06T02:26:43.083271+0000","last_active":"2023-10-06T02:26:43.083271+0000","last_peered":"2023-10-06T02:26:43.083271+0000","last_clean":"2023-10-06T02:26:43.083271+0000","last_became_active":"2023-10-06T02:26:43.082943+0000","last_became_peered":"2023-10-06T02:26:43.082943+0000","last_unstale":"2023-10-06T02:26:43.083271+0000","last_undegraded":"2023-10-06T02:26:43.083271+0000","last_fullsized":"2023-10-06T02:26:43.083271+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:04:15.890422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.083445+0000","last_change":"2023-10-06T02:26:43.083445+0000","last_active":"2023-10-06T02:26:43.083445+0000","last_peered":"2023-10-06T02:26:43.083445+0000","last_clean":"2023-10-06T02:26:43.083445+0000","last_became_active":"2023-10-06T02:26:43.083062+0000","last_became_peered":"2023-10-06T02:26:43.083062+0000","last_unstale":"2023-10-06T02:26:43.083445+0000","last_undegraded":"2023-10-06T02:26:43.083445+0000","last_fullsized":"2023-10-06T02:26:43.083445+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:49:13.194304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.086831+0000","last_change":"2023-10-06T02:26:43.086831+0000","last_active":"2023-10-06T02:26:43.086831+0000","last_peered":"2023-10-06T02:26:43.086831+0000","last_clean":"2023-10-06T02:26:43.086831+0000","last_became_active":"2023-10-06T02:26:43.086533+0000","last_became_peered":"2023-10-06T02:26:43.086533+0000","last_unstale":"2023-10-06T02:26:43.086831+0000","last_undegraded":"2023-10-06T02:26:43.086831+0000","last_fullsized":"2023-10-06T02:26:43.086831+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:03:03.126517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.079956+0000","last_change":"2023-10-06T02:26:43.079956+0000","last_active":"2023-10-06T02:26:43.079956+0000","last_peered":"2023-10-06T02:26:43.079956+0000","last_clean":"2023-10-06T02:26:43.079956+0000","last_became_active":"2023-10-06T02:26:43.076751+0000","last_became_peered":"2023-10-06T02:26:43.076751+0000","last_unstale":"2023-10-06T02:26:43.079956+0000","last_undegraded":"2023-10-06T02:26:43.079956+0000","last_fullsized":"2023-10-06T02:26:43.079956+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:13:19.533378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.078144+0000","last_change":"2023-10-06T02:26:43.078144+0000","last_active":"2023-10-06T02:26:43.078144+0000","last_peered":"2023-10-06T02:26:43.078144+0000","last_clean":"2023-10-06T02:26:43.078144+0000","last_became_active":"2023-10-06T02:26:43.077296+0000","last_became_peered":"2023-10-06T02:26:43.077296+0000","last_unstale":"2023-10-06T02:26:43.078144+0000","last_undegraded":"2023-10-06T02:26:43.078144+0000","last_fullsized":"2023-10-06T02:26:43.078144+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:31:07.380497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.074301+0000","last_change":"2023-10-06T02:26:43.074301+0000","last_active":"2023-10-06T02:26:43.074301+0000","last_peered":"2023-10-06T02:26:43.074301+0000","last_clean":"2023-10-06T02:26:43.074301+0000","last_became_active":"2023-10-06T02:26:43.073992+0000","last_became_peered":"2023-10-06T02:26:43.073992+0000","last_unstale":"2023-10-06T02:26:43.074301+0000","last_undegraded":"2023-10-06T02:26:43.074301+0000","last_fullsized":"2023-10-06T02:26:43.074301+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:28:44.491766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.086761+0000","last_change":"2023-10-06T02:26:43.086761+0000","last_active":"2023-10-06T02:26:43.086761+0000","last_peered":"2023-10-06T02:26:43.086761+0000","last_clean":"2023-10-06T02:26:43.086761+0000","last_became_active":"2023-10-06T02:26:43.086495+0000","last_became_peered":"2023-10-06T02:26:43.086495+0000","last_unstale":"2023-10-06T02:26:43.086761+0000","last_undegraded":"2023-10-06T02:26:43.086761+0000","last_fullsized":"2023-10-06T02:26:43.086761+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:19:55.281699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.091349+0000","last_change":"2023-10-06T02:26:43.091349+0000","last_active":"2023-10-06T02:26:43.091349+0000","last_peered":"2023-10-06T02:26:43.091349+0000","last_clean":"2023-10-06T02:26:43.091349+0000","last_became_active":"2023-10-06T02:26:43.091042+0000","last_became_peered":"2023-10-06T02:26:43.091042+0000","last_unstale":"2023-10-06T02:26:43.091349+0000","last_undegraded":"2023-10-06T02:26:43.091349+0000","last_fullsized":"2023-10-06T02:26:43.091349+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:13:13.506543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.083042+0000","last_change":"2023-10-06T02:26:43.083042+0000","last_active":"2023-10-06T02:26:43.083042+0000","last_peered":"2023-10-06T02:26:43.083042+0000","last_clean":"2023-10-06T02:26:43.083042+0000","last_became_active":"2023-10-06T02:26:43.082752+0000","last_became_peered":"2023-10-06T02:26:43.082752+0000","last_unstale":"2023-10-06T02:26:43.083042+0000","last_undegraded":"2023-10-06T02:26:43.083042+0000","last_fullsized":"2023-10-06T02:26:43.083042+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:08:03.092157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.073639+0000","last_change":"2023-10-06T02:26:43.073639+0000","last_active":"2023-10-06T02:26:43.073639+0000","last_peered":"2023-10-06T02:26:43.073639+0000","last_clean":"2023-10-06T02:26:43.073639+0000","last_became_active":"2023-10-06T02:26:43.073182+0000","last_became_peered":"2023-10-06T02:26:43.073182+0000","last_unstale":"2023-10-06T02:26:43.073639+0000","last_undegraded":"2023-10-06T02:26:43.073639+0000","last_fullsized":"2023-10-06T02:26:43.073639+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:47:21.495991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.074819+0000","last_change":"2023-10-06T02:26:43.074819+0000","last_active":"2023-10-06T02:26:43.074819+0000","last_peered":"2023-10-06T02:26:43.074819+0000","last_clean":"2023-10-06T02:26:43.074819+0000","last_became_active":"2023-10-06T02:26:43.074142+0000","last_became_peered":"2023-10-06T02:26:43.074142+0000","last_unstale":"2023-10-06T02:26:43.074819+0000","last_undegraded":"2023-10-06T02:26:43.074819+0000","last_fullsized":"2023-10-06T02:26:43.074819+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:40.671226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.086161+0000","last_change":"2023-10-06T02:26:43.086161+0000","last_active":"2023-10-06T02:26:43.086161+0000","last_peered":"2023-10-06T02:26:43.086161+0000","last_clean":"2023-10-06T02:26:43.086161+0000","last_became_active":"2023-10-06T02:26:43.085818+0000","last_became_peered":"2023-10-06T02:26:43.085818+0000","last_unstale":"2023-10-06T02:26:43.086161+0000","last_undegraded":"2023-10-06T02:26:43.086161+0000","last_fullsized":"2023-10-06T02:26:43.086161+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:49:19.811251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"60'4","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.099335+0000","last_change":"2023-10-06T02:26:42.079022+0000","last_active":"2023-10-06T02:26:43.099335+0000","last_peered":"2023-10-06T02:26:43.099335+0000","last_clean":"2023-10-06T02:26:43.099335+0000","last_became_active":"2023-10-06T02:26:42.078636+0000","last_became_peered":"2023-10-06T02:26:42.078636+0000","last_unstale":"2023-10-06T02:26:43.099335+0000","last_undegraded":"2023-10-06T02:26:43.099335+0000","last_fullsized":"2023-10-06T02:26:43.099335+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_clean_scrub_stamp":"2023-10-06T02:26:41.040467+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:37:50.109220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.088152+0000","last_change":"2023-10-06T02:26:43.088152+0000","last_active":"2023-10-06T02:26:43.088152+0000","last_peered":"2023-10-06T02:26:43.088152+0000","last_clean":"2023-10-06T02:26:43.088152+0000","last_became_active":"2023-10-06T02:26:43.087824+0000","last_became_peered":"2023-10-06T02:26:43.087824+0000","last_unstale":"2023-10-06T02:26:43.088152+0000","last_undegraded":"2023-10-06T02:26:43.088152+0000","last_fullsized":"2023-10-06T02:26:43.088152+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:49:19.811251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"60'1","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.063032+0000","last_change":"2023-10-06T02:26:42.078432+0000","last_active":"2023-10-06T02:26:43.063032+0000","last_peered":"2023-10-06T02:26:43.063032+0000","last_clean":"2023-10-06T02:26:43.063032+0000","last_became_active":"2023-10-06T02:26:42.078127+0000","last_became_peered":"2023-10-06T02:26:42.078127+0000","last_unstale":"2023-10-06T02:26:43.063032+0000","last_undegraded":"2023-10-06T02:26:43.063032+0000","last_fullsized":"2023-10-06T02:26:43.063032+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_clean_scrub_stamp":"2023-10-06T02:26:41.040467+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:08:28.473770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.073635+0000","last_change":"2023-10-06T02:26:43.073635+0000","last_active":"2023-10-06T02:26:43.073635+0000","last_peered":"2023-10-06T02:26:43.073635+0000","last_clean":"2023-10-06T02:26:43.073635+0000","last_became_active":"2023-10-06T02:26:43.073327+0000","last_became_peered":"2023-10-06T02:26:43.073327+0000","last_unstale":"2023-10-06T02:26:43.073635+0000","last_undegraded":"2023-10-06T02:26:43.073635+0000","last_fullsized":"2023-10-06T02:26:43.073635+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:33:18.711244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"60'2","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.088926+0000","last_change":"2023-10-06T02:26:42.071061+0000","last_active":"2023-10-06T02:26:43.088926+0000","last_peered":"2023-10-06T02:26:43.088926+0000","last_clean":"2023-10-06T02:26:43.088926+0000","last_became_active":"2023-10-06T02:26:42.070781+0000","last_became_peered":"2023-10-06T02:26:42.070781+0000","last_unstale":"2023-10-06T02:26:43.088926+0000","last_undegraded":"2023-10-06T02:26:43.088926+0000","last_fullsized":"2023-10-06T02:26:43.088926+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_clean_scrub_stamp":"2023-10-06T02:26:41.040467+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:53:53.191609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.094686+0000","last_change":"2023-10-06T02:26:43.094686+0000","last_active":"2023-10-06T02:26:43.094686+0000","last_peered":"2023-10-06T02:26:43.094686+0000","last_clean":"2023-10-06T02:26:43.094686+0000","last_became_active":"2023-10-06T02:26:43.094383+0000","last_became_peered":"2023-10-06T02:26:43.094383+0000","last_unstale":"2023-10-06T02:26:43.094686+0000","last_undegraded":"2023-10-06T02:26:43.094686+0000","last_fullsized":"2023-10-06T02:26:43.094686+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:27:50.683731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"60'2","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.090412+0000","last_change":"2023-10-06T02:26:42.079812+0000","last_active":"2023-10-06T02:26:43.090412+0000","last_peered":"2023-10-06T02:26:43.090412+0000","last_clean":"2023-10-06T02:26:43.090412+0000","last_became_active":"2023-10-06T02:26:42.079040+0000","last_became_peered":"2023-10-06T02:26:42.079040+0000","last_unstale":"2023-10-06T02:26:43.090412+0000","last_undegraded":"2023-10-06T02:26:43.090412+0000","last_fullsized":"2023-10-06T02:26:43.090412+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_clean_scrub_stamp":"2023-10-06T02:26:41.040467+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:44:17.259680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.085642+0000","last_change":"2023-10-06T02:26:43.085642+0000","last_active":"2023-10-06T02:26:43.085642+0000","last_peered":"2023-10-06T02:26:43.085642+0000","last_clean":"2023-10-06T02:26:43.085642+0000","last_became_active":"2023-10-06T02:26:43.085329+0000","last_became_peered":"2023-10-06T02:26:43.085329+0000","last_unstale":"2023-10-06T02:26:43.085642+0000","last_undegraded":"2023-10-06T02:26:43.085642+0000","last_fullsized":"2023-10-06T02:26:43.085642+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:29:12.676385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"60'1","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.059038+0000","last_change":"2023-10-06T02:26:42.072152+0000","last_active":"2023-10-06T02:26:43.059038+0000","last_peered":"2023-10-06T02:26:43.059038+0000","last_clean":"2023-10-06T02:26:43.059038+0000","last_became_active":"2023-10-06T02:26:42.071939+0000","last_became_peered":"2023-10-06T02:26:42.071939+0000","last_unstale":"2023-10-06T02:26:43.059038+0000","last_undegraded":"2023-10-06T02:26:43.059038+0000","last_fullsized":"2023-10-06T02:26:43.059038+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_clean_scrub_stamp":"2023-10-06T02:26:41.040467+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:53:53.191609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.081201+0000","last_change":"2023-10-06T02:26:43.081201+0000","last_active":"2023-10-06T02:26:43.081201+0000","last_peered":"2023-10-06T02:26:43.081201+0000","last_clean":"2023-10-06T02:26:43.081201+0000","last_became_active":"2023-10-06T02:26:43.080894+0000","last_became_peered":"2023-10-06T02:26:43.080894+0000","last_unstale":"2023-10-06T02:26:43.081201+0000","last_undegraded":"2023-10-06T02:26:43.081201+0000","last_fullsized":"2023-10-06T02:26:43.081201+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:31:56.002363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"60'1","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.058803+0000","last_change":"2023-10-06T02:26:42.069515+0000","last_active":"2023-10-06T02:26:43.058803+0000","last_peered":"2023-10-06T02:26:43.058803+0000","last_clean":"2023-10-06T02:26:43.058803+0000","last_became_active":"2023-10-06T02:26:42.069215+0000","last_became_peered":"2023-10-06T02:26:42.069215+0000","last_unstale":"2023-10-06T02:26:43.058803+0000","last_undegraded":"2023-10-06T02:26:43.058803+0000","last_fullsized":"2023-10-06T02:26:43.058803+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_clean_scrub_stamp":"2023-10-06T02:26:41.040467+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:08:28.473770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.083466+0000","last_change":"2023-10-06T02:26:43.083466+0000","last_active":"2023-10-06T02:26:43.083466+0000","last_peered":"2023-10-06T02:26:43.083466+0000","last_clean":"2023-10-06T02:26:43.083466+0000","last_became_active":"2023-10-06T02:26:43.083213+0000","last_became_peered":"2023-10-06T02:26:43.083213+0000","last_unstale":"2023-10-06T02:26:43.083466+0000","last_undegraded":"2023-10-06T02:26:43.083466+0000","last_fullsized":"2023-10-06T02:26:43.083466+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:50:38.588741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-06T02:26:42.080803+0000","last_change":"2023-10-06T02:26:42.075580+0000","last_active":"2023-10-06T02:26:42.080803+0000","last_peered":"2023-10-06T02:26:42.080803+0000","last_clean":"2023-10-06T02:26:42.080803+0000","last_became_active":"2023-10-06T02:26:42.075321+0000","last_became_peered":"2023-10-06T02:26:42.075321+0000","last_unstale":"2023-10-06T02:26:42.080803+0000","last_undegraded":"2023-10-06T02:26:42.080803+0000","last_fullsized":"2023-10-06T02:26:42.080803+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_clean_scrub_stamp":"2023-10-06T02:26:41.040467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:08:02.085597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.067766+0000","last_change":"2023-10-06T02:26:43.067766+0000","last_active":"2023-10-06T02:26:43.067766+0000","last_peered":"2023-10-06T02:26:43.067766+0000","last_clean":"2023-10-06T02:26:43.067766+0000","last_became_active":"2023-10-06T02:26:43.067496+0000","last_became_peered":"2023-10-06T02:26:43.067496+0000","last_unstale":"2023-10-06T02:26:43.067766+0000","last_undegraded":"2023-10-06T02:26:43.067766+0000","last_fullsized":"2023-10-06T02:26:43.067766+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:55:59.959651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-06T02:26:42.079239+0000","last_change":"2023-10-06T02:26:42.074166+0000","last_active":"2023-10-06T02:26:42.079239+0000","last_peered":"2023-10-06T02:26:42.079239+0000","last_clean":"2023-10-06T02:26:42.079239+0000","last_became_active":"2023-10-06T02:26:42.073908+0000","last_became_peered":"2023-10-06T02:26:42.073908+0000","last_unstale":"2023-10-06T02:26:42.079239+0000","last_undegraded":"2023-10-06T02:26:42.079239+0000","last_fullsized":"2023-10-06T02:26:42.079239+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_clean_scrub_stamp":"2023-10-06T02:26:41.040467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:11:16.082079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.082770+0000","last_change":"2023-10-06T02:26:43.082770+0000","last_active":"2023-10-06T02:26:43.082770+0000","last_peered":"2023-10-06T02:26:43.082770+0000","last_clean":"2023-10-06T02:26:43.082770+0000","last_became_active":"2023-10-06T02:26:43.082428+0000","last_became_peered":"2023-10-06T02:26:43.082428+0000","last_unstale":"2023-10-06T02:26:43.082770+0000","last_undegraded":"2023-10-06T02:26:43.082770+0000","last_fullsized":"2023-10-06T02:26:43.082770+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:06:31.428249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-06T02:26:42.090191+0000","last_change":"2023-10-06T02:26:42.083623+0000","last_active":"2023-10-06T02:26:42.090191+0000","last_peered":"2023-10-06T02:26:42.090191+0000","last_clean":"2023-10-06T02:26:42.090191+0000","last_became_active":"2023-10-06T02:26:42.083280+0000","last_became_peered":"2023-10-06T02:26:42.083280+0000","last_unstale":"2023-10-06T02:26:42.090191+0000","last_undegraded":"2023-10-06T02:26:42.090191+0000","last_fullsized":"2023-10-06T02:26:42.090191+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_clean_scrub_stamp":"2023-10-06T02:26:41.040467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:44:23.565902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.069687+0000","last_change":"2023-10-06T02:26:43.069687+0000","last_active":"2023-10-06T02:26:43.069687+0000","last_peered":"2023-10-06T02:26:43.069687+0000","last_clean":"2023-10-06T02:26:43.069687+0000","last_became_active":"2023-10-06T02:26:43.069403+0000","last_became_peered":"2023-10-06T02:26:43.069403+0000","last_unstale":"2023-10-06T02:26:43.069687+0000","last_undegraded":"2023-10-06T02:26:43.069687+0000","last_fullsized":"2023-10-06T02:26:43.069687+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:33:18.711244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-06T02:26:42.086500+0000","last_change":"2023-10-06T02:26:42.081692+0000","last_active":"2023-10-06T02:26:42.086500+0000","last_peered":"2023-10-06T02:26:42.086500+0000","last_clean":"2023-10-06T02:26:42.086500+0000","last_became_active":"2023-10-06T02:26:42.080705+0000","last_became_peered":"2023-10-06T02:26:42.080705+0000","last_unstale":"2023-10-06T02:26:42.086500+0000","last_undegraded":"2023-10-06T02:26:42.086500+0000","last_fullsized":"2023-10-06T02:26:42.086500+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_clean_scrub_stamp":"2023-10-06T02:26:41.040467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:12:32.146499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.088010+0000","last_change":"2023-10-06T02:26:43.088010+0000","last_active":"2023-10-06T02:26:43.088010+0000","last_peered":"2023-10-06T02:26:43.088010+0000","last_clean":"2023-10-06T02:26:43.088010+0000","last_became_active":"2023-10-06T02:26:43.087558+0000","last_became_peered":"2023-10-06T02:26:43.087558+0000","last_unstale":"2023-10-06T02:26:43.088010+0000","last_undegraded":"2023-10-06T02:26:43.088010+0000","last_fullsized":"2023-10-06T02:26:43.088010+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:37:00.416857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"60'3","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.101409+0000","last_change":"2023-10-06T02:26:42.083841+0000","last_active":"2023-10-06T02:26:43.101409+0000","last_peered":"2023-10-06T02:26:43.101409+0000","last_clean":"2023-10-06T02:26:43.101409+0000","last_became_active":"2023-10-06T02:26:42.083541+0000","last_became_peered":"2023-10-06T02:26:42.083541+0000","last_unstale":"2023-10-06T02:26:43.101409+0000","last_undegraded":"2023-10-06T02:26:43.101409+0000","last_fullsized":"2023-10-06T02:26:43.101409+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_clean_scrub_stamp":"2023-10-06T02:26:41.040467+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:19:54.275140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.073370+0000","last_change":"2023-10-06T02:26:43.073370+0000","last_active":"2023-10-06T02:26:43.073370+0000","last_peered":"2023-10-06T02:26:43.073370+0000","last_clean":"2023-10-06T02:26:43.073370+0000","last_became_active":"2023-10-06T02:26:43.072988+0000","last_became_peered":"2023-10-06T02:26:43.072988+0000","last_unstale":"2023-10-06T02:26:43.073370+0000","last_undegraded":"2023-10-06T02:26:43.073370+0000","last_fullsized":"2023-10-06T02:26:43.073370+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:50:38.588741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"60'6","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.098602+0000","last_change":"2023-10-06T02:26:42.082345+0000","last_active":"2023-10-06T02:26:43.098602+0000","last_peered":"2023-10-06T02:26:43.098602+0000","last_clean":"2023-10-06T02:26:43.098602+0000","last_became_active":"2023-10-06T02:26:42.082026+0000","last_became_peered":"2023-10-06T02:26:42.082026+0000","last_unstale":"2023-10-06T02:26:43.098602+0000","last_undegraded":"2023-10-06T02:26:43.098602+0000","last_fullsized":"2023-10-06T02:26:43.098602+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_clean_scrub_stamp":"2023-10-06T02:26:41.040467+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:16:48.074587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.084183+0000","last_change":"2023-10-06T02:26:43.084183+0000","last_active":"2023-10-06T02:26:43.084183+0000","last_peered":"2023-10-06T02:26:43.084183+0000","last_clean":"2023-10-06T02:26:43.084183+0000","last_became_active":"2023-10-06T02:26:43.083926+0000","last_became_peered":"2023-10-06T02:26:43.083926+0000","last_unstale":"2023-10-06T02:26:43.084183+0000","last_undegraded":"2023-10-06T02:26:43.084183+0000","last_fullsized":"2023-10-06T02:26:43.084183+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:55.811855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"60'2","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.101481+0000","last_change":"2023-10-06T02:26:42.079842+0000","last_active":"2023-10-06T02:26:43.101481+0000","last_peered":"2023-10-06T02:26:43.101481+0000","last_clean":"2023-10-06T02:26:43.101481+0000","last_became_active":"2023-10-06T02:26:42.079583+0000","last_became_peered":"2023-10-06T02:26:42.079583+0000","last_unstale":"2023-10-06T02:26:43.101481+0000","last_undegraded":"2023-10-06T02:26:43.101481+0000","last_fullsized":"2023-10-06T02:26:43.101481+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_clean_scrub_stamp":"2023-10-06T02:26:41.040467+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:48:34.539906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.075255+0000","last_change":"2023-10-06T02:26:43.075255+0000","last_active":"2023-10-06T02:26:43.075255+0000","last_peered":"2023-10-06T02:26:43.075255+0000","last_clean":"2023-10-06T02:26:43.075255+0000","last_became_active":"2023-10-06T02:26:43.074954+0000","last_became_peered":"2023-10-06T02:26:43.074954+0000","last_unstale":"2023-10-06T02:26:43.075255+0000","last_undegraded":"2023-10-06T02:26:43.075255+0000","last_fullsized":"2023-10-06T02:26:43.075255+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:47:21.495991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-06T02:26:42.086683+0000","last_change":"2023-10-06T02:26:42.080502+0000","last_active":"2023-10-06T02:26:42.086683+0000","last_peered":"2023-10-06T02:26:42.086683+0000","last_clean":"2023-10-06T02:26:42.086683+0000","last_became_active":"2023-10-06T02:26:42.080159+0000","last_became_peered":"2023-10-06T02:26:42.080159+0000","last_unstale":"2023-10-06T02:26:42.086683+0000","last_undegraded":"2023-10-06T02:26:42.086683+0000","last_fullsized":"2023-10-06T02:26:42.086683+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_clean_scrub_stamp":"2023-10-06T02:26:41.040467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:53:20.029463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.079057+0000","last_change":"2023-10-06T02:26:43.079057+0000","last_active":"2023-10-06T02:26:43.079057+0000","last_peered":"2023-10-06T02:26:43.079057+0000","last_clean":"2023-10-06T02:26:43.079057+0000","last_became_active":"2023-10-06T02:26:43.078779+0000","last_became_peered":"2023-10-06T02:26:43.078779+0000","last_unstale":"2023-10-06T02:26:43.079057+0000","last_undegraded":"2023-10-06T02:26:43.079057+0000","last_fullsized":"2023-10-06T02:26:43.079057+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:19:55.281699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-06T02:26:42.086808+0000","last_change":"2023-10-06T02:26:42.081994+0000","last_active":"2023-10-06T02:26:42.086808+0000","last_peered":"2023-10-06T02:26:42.086808+0000","last_clean":"2023-10-06T02:26:42.086808+0000","last_became_active":"2023-10-06T02:26:42.081473+0000","last_became_peered":"2023-10-06T02:26:42.081473+0000","last_unstale":"2023-10-06T02:26:42.086808+0000","last_undegraded":"2023-10-06T02:26:42.086808+0000","last_fullsized":"2023-10-06T02:26:42.086808+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_clean_scrub_stamp":"2023-10-06T02:26:41.040467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:49:18.804692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.086111+0000","last_change":"2023-10-06T02:26:43.086111+0000","last_active":"2023-10-06T02:26:43.086111+0000","last_peered":"2023-10-06T02:26:43.086111+0000","last_clean":"2023-10-06T02:26:43.086111+0000","last_became_active":"2023-10-06T02:26:43.085875+0000","last_became_peered":"2023-10-06T02:26:43.085875+0000","last_unstale":"2023-10-06T02:26:43.086111+0000","last_undegraded":"2023-10-06T02:26:43.086111+0000","last_fullsized":"2023-10-06T02:26:43.086111+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:37:00.416857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"60'8","reported_seq":17,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.097762+0000","last_change":"2023-10-06T02:26:42.071853+0000","last_active":"2023-10-06T02:26:43.097762+0000","last_peered":"2023-10-06T02:26:43.097762+0000","last_clean":"2023-10-06T02:26:43.097762+0000","last_became_active":"2023-10-06T02:26:42.071477+0000","last_became_peered":"2023-10-06T02:26:42.071477+0000","last_unstale":"2023-10-06T02:26:43.097762+0000","last_undegraded":"2023-10-06T02:26:43.097762+0000","last_fullsized":"2023-10-06T02:26:43.097762+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_clean_scrub_stamp":"2023-10-06T02:26:41.040467+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:21:14.686655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.073917+0000","last_change":"2023-10-06T02:26:43.073917+0000","last_active":"2023-10-06T02:26:43.073917+0000","last_peered":"2023-10-06T02:26:43.073917+0000","last_clean":"2023-10-06T02:26:43.073917+0000","last_became_active":"2023-10-06T02:26:43.073659+0000","last_became_peered":"2023-10-06T02:26:43.073659+0000","last_unstale":"2023-10-06T02:26:43.073917+0000","last_undegraded":"2023-10-06T02:26:43.073917+0000","last_fullsized":"2023-10-06T02:26:43.073917+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:44:55.051256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"60'2","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.094126+0000","last_change":"2023-10-06T02:26:42.083006+0000","last_active":"2023-10-06T02:26:43.094126+0000","last_peered":"2023-10-06T02:26:43.094126+0000","last_clean":"2023-10-06T02:26:43.094126+0000","last_became_active":"2023-10-06T02:26:42.082511+0000","last_became_peered":"2023-10-06T02:26:42.082511+0000","last_unstale":"2023-10-06T02:26:43.094126+0000","last_undegraded":"2023-10-06T02:26:43.094126+0000","last_fullsized":"2023-10-06T02:26:43.094126+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_clean_scrub_stamp":"2023-10-06T02:26:41.040467+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:58:22.585200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.076946+0000","last_change":"2023-10-06T02:26:43.076946+0000","last_active":"2023-10-06T02:26:43.076946+0000","last_peered":"2023-10-06T02:26:43.076946+0000","last_clean":"2023-10-06T02:26:43.076946+0000","last_became_active":"2023-10-06T02:26:43.076661+0000","last_became_peered":"2023-10-06T02:26:43.076661+0000","last_unstale":"2023-10-06T02:26:43.076946+0000","last_undegraded":"2023-10-06T02:26:43.076946+0000","last_fullsized":"2023-10-06T02:26:43.076946+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:42:18.780865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-06T02:26:42.080285+0000","last_change":"2023-10-06T02:26:42.073476+0000","last_active":"2023-10-06T02:26:42.080285+0000","last_peered":"2023-10-06T02:26:42.080285+0000","last_clean":"2023-10-06T02:26:42.080285+0000","last_became_active":"2023-10-06T02:26:42.073146+0000","last_became_peered":"2023-10-06T02:26:42.073146+0000","last_unstale":"2023-10-06T02:26:42.080285+0000","last_undegraded":"2023-10-06T02:26:42.080285+0000","last_fullsized":"2023-10-06T02:26:42.080285+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_clean_scrub_stamp":"2023-10-06T02:26:41.040467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:21:19.421613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.077808+0000","last_change":"2023-10-06T02:26:43.077808+0000","last_active":"2023-10-06T02:26:43.077808+0000","last_peered":"2023-10-06T02:26:43.077808+0000","last_clean":"2023-10-06T02:26:43.077808+0000","last_became_active":"2023-10-06T02:26:43.076710+0000","last_became_peered":"2023-10-06T02:26:43.076710+0000","last_unstale":"2023-10-06T02:26:43.077808+0000","last_undegraded":"2023-10-06T02:26:43.077808+0000","last_fullsized":"2023-10-06T02:26:43.077808+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:52:16.520095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-06T02:26:42.073173+0000","last_change":"2023-10-06T02:26:42.066985+0000","last_active":"2023-10-06T02:26:42.073173+0000","last_peered":"2023-10-06T02:26:42.073173+0000","last_clean":"2023-10-06T02:26:42.073173+0000","last_became_active":"2023-10-06T02:26:42.066706+0000","last_became_peered":"2023-10-06T02:26:42.066706+0000","last_unstale":"2023-10-06T02:26:42.073173+0000","last_undegraded":"2023-10-06T02:26:42.073173+0000","last_fullsized":"2023-10-06T02:26:42.073173+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_clean_scrub_stamp":"2023-10-06T02:26:41.040467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:27:49.677171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.078383+0000","last_change":"2023-10-06T02:26:43.078383+0000","last_active":"2023-10-06T02:26:43.078383+0000","last_peered":"2023-10-06T02:26:43.078383+0000","last_clean":"2023-10-06T02:26:43.078383+0000","last_became_active":"2023-10-06T02:26:43.077873+0000","last_became_peered":"2023-10-06T02:26:43.077873+0000","last_unstale":"2023-10-06T02:26:43.078383+0000","last_undegraded":"2023-10-06T02:26:43.078383+0000","last_fullsized":"2023-10-06T02:26:43.078383+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:44:24.572462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-06T02:26:42.083090+0000","last_change":"2023-10-06T02:26:42.078465+0000","last_active":"2023-10-06T02:26:42.083090+0000","last_peered":"2023-10-06T02:26:42.083090+0000","last_clean":"2023-10-06T02:26:42.083090+0000","last_became_active":"2023-10-06T02:26:42.078124+0000","last_became_peered":"2023-10-06T02:26:42.078124+0000","last_unstale":"2023-10-06T02:26:42.083090+0000","last_undegraded":"2023-10-06T02:26:42.083090+0000","last_fullsized":"2023-10-06T02:26:42.083090+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_clean_scrub_stamp":"2023-10-06T02:26:41.040467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:39:51.926230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.090536+0000","last_change":"2023-10-06T02:26:43.090536+0000","last_active":"2023-10-06T02:26:43.090536+0000","last_peered":"2023-10-06T02:26:43.090536+0000","last_clean":"2023-10-06T02:26:43.090536+0000","last_became_active":"2023-10-06T02:26:43.090242+0000","last_became_peered":"2023-10-06T02:26:43.090242+0000","last_unstale":"2023-10-06T02:26:43.090536+0000","last_undegraded":"2023-10-06T02:26:43.090536+0000","last_fullsized":"2023-10-06T02:26:43.090536+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:23:51.057531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-06T02:26:42.090288+0000","last_change":"2023-10-06T02:26:42.084346+0000","last_active":"2023-10-06T02:26:42.090288+0000","last_peered":"2023-10-06T02:26:42.090288+0000","last_clean":"2023-10-06T02:26:42.090288+0000","last_became_active":"2023-10-06T02:26:42.084050+0000","last_became_peered":"2023-10-06T02:26:42.084050+0000","last_unstale":"2023-10-06T02:26:42.090288+0000","last_undegraded":"2023-10-06T02:26:42.090288+0000","last_fullsized":"2023-10-06T02:26:42.090288+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_clean_scrub_stamp":"2023-10-06T02:26:41.040467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:47:20.489431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.084882+0000","last_change":"2023-10-06T02:26:43.084882+0000","last_active":"2023-10-06T02:26:43.084882+0000","last_peered":"2023-10-06T02:26:43.084882+0000","last_clean":"2023-10-06T02:26:43.084882+0000","last_became_active":"2023-10-06T02:26:43.084622+0000","last_became_peered":"2023-10-06T02:26:43.084622+0000","last_unstale":"2023-10-06T02:26:43.084882+0000","last_undegraded":"2023-10-06T02:26:43.084882+0000","last_fullsized":"2023-10-06T02:26:43.084882+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:13:19.533378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-06T02:26:42.082710+0000","last_change":"2023-10-06T02:26:42.078455+0000","last_active":"2023-10-06T02:26:42.082710+0000","last_peered":"2023-10-06T02:26:42.082710+0000","last_clean":"2023-10-06T02:26:42.082710+0000","last_became_active":"2023-10-06T02:26:42.078140+0000","last_became_peered":"2023-10-06T02:26:42.078140+0000","last_unstale":"2023-10-06T02:26:42.082710+0000","last_undegraded":"2023-10-06T02:26:42.082710+0000","last_fullsized":"2023-10-06T02:26:42.082710+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_clean_scrub_stamp":"2023-10-06T02:26:41.040467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:17:55.051477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.088400+0000","last_change":"2023-10-06T02:26:43.088400+0000","last_active":"2023-10-06T02:26:43.088400+0000","last_peered":"2023-10-06T02:26:43.088400+0000","last_clean":"2023-10-06T02:26:43.088400+0000","last_became_active":"2023-10-06T02:26:43.087755+0000","last_became_peered":"2023-10-06T02:26:43.087755+0000","last_unstale":"2023-10-06T02:26:43.088400+0000","last_undegraded":"2023-10-06T02:26:43.088400+0000","last_fullsized":"2023-10-06T02:26:43.088400+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:50:38.588741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-06T02:26:42.080829+0000","last_change":"2023-10-06T02:26:42.075720+0000","last_active":"2023-10-06T02:26:42.080829+0000","last_peered":"2023-10-06T02:26:42.080829+0000","last_clean":"2023-10-06T02:26:42.080829+0000","last_became_active":"2023-10-06T02:26:42.075424+0000","last_became_peered":"2023-10-06T02:26:42.075424+0000","last_unstale":"2023-10-06T02:26:42.080829+0000","last_undegraded":"2023-10-06T02:26:42.080829+0000","last_fullsized":"2023-10-06T02:26:42.080829+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_clean_scrub_stamp":"2023-10-06T02:26:41.040467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:57:19.405017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.080847+0000","last_change":"2023-10-06T02:26:43.080847+0000","last_active":"2023-10-06T02:26:43.080847+0000","last_peered":"2023-10-06T02:26:43.080847+0000","last_clean":"2023-10-06T02:26:43.080847+0000","last_became_active":"2023-10-06T02:26:43.080597+0000","last_became_peered":"2023-10-06T02:26:43.080597+0000","last_unstale":"2023-10-06T02:26:43.080847+0000","last_undegraded":"2023-10-06T02:26:43.080847+0000","last_fullsized":"2023-10-06T02:26:43.080847+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:54:21.378728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-06T02:26:42.077995+0000","last_change":"2023-10-06T02:26:42.073779+0000","last_active":"2023-10-06T02:26:42.077995+0000","last_peered":"2023-10-06T02:26:42.077995+0000","last_clean":"2023-10-06T02:26:42.077995+0000","last_became_active":"2023-10-06T02:26:42.073436+0000","last_became_peered":"2023-10-06T02:26:42.073436+0000","last_unstale":"2023-10-06T02:26:42.077995+0000","last_undegraded":"2023-10-06T02:26:42.077995+0000","last_fullsized":"2023-10-06T02:26:42.077995+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_clean_scrub_stamp":"2023-10-06T02:26:41.040467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:48:58.794790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.092914+0000","last_change":"2023-10-06T02:26:43.092914+0000","last_active":"2023-10-06T02:26:43.092914+0000","last_peered":"2023-10-06T02:26:43.092914+0000","last_clean":"2023-10-06T02:26:43.092914+0000","last_became_active":"2023-10-06T02:26:43.091596+0000","last_became_peered":"2023-10-06T02:26:43.091596+0000","last_unstale":"2023-10-06T02:26:43.092914+0000","last_undegraded":"2023-10-06T02:26:43.092914+0000","last_fullsized":"2023-10-06T02:26:43.092914+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:41.589820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"60'1","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.064019+0000","last_change":"2023-10-06T02:26:42.080936+0000","last_active":"2023-10-06T02:26:43.064019+0000","last_peered":"2023-10-06T02:26:43.064019+0000","last_clean":"2023-10-06T02:26:43.064019+0000","last_became_active":"2023-10-06T02:26:42.080651+0000","last_became_peered":"2023-10-06T02:26:42.080651+0000","last_unstale":"2023-10-06T02:26:43.064019+0000","last_undegraded":"2023-10-06T02:26:43.064019+0000","last_fullsized":"2023-10-06T02:26:43.064019+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_clean_scrub_stamp":"2023-10-06T02:26:41.040467+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:23:50.050971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":44,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-06T02:26:42.072436+0000","last_change":"2023-10-06T02:23:02.054039+0000","last_active":"2023-10-06T02:26:42.072436+0000","last_peered":"2023-10-06T02:26:42.072436+0000","last_clean":"2023-10-06T02:26:42.072436+0000","last_became_active":"2023-10-06T02:23:02.053742+0000","last_became_peered":"2023-10-06T02:23:02.053742+0000","last_unstale":"2023-10-06T02:26:42.072436+0000","last_undegraded":"2023-10-06T02:26:42.072436+0000","last_fullsized":"2023-10-06T02:26:42.072436+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:58:18.846475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.081544+0000","last_change":"2023-10-06T02:26:43.081544+0000","last_active":"2023-10-06T02:26:43.081544+0000","last_peered":"2023-10-06T02:26:43.081544+0000","last_clean":"2023-10-06T02:26:43.081544+0000","last_became_active":"2023-10-06T02:26:43.081242+0000","last_became_peered":"2023-10-06T02:26:43.081242+0000","last_unstale":"2023-10-06T02:26:43.081544+0000","last_undegraded":"2023-10-06T02:26:43.081544+0000","last_fullsized":"2023-10-06T02:26:43.081544+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:31:56.002363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-06T02:26:42.082780+0000","last_change":"2023-10-06T02:26:42.078866+0000","last_active":"2023-10-06T02:26:42.082780+0000","last_peered":"2023-10-06T02:26:42.082780+0000","last_clean":"2023-10-06T02:26:42.082780+0000","last_became_active":"2023-10-06T02:26:42.078571+0000","last_became_peered":"2023-10-06T02:26:42.078571+0000","last_unstale":"2023-10-06T02:26:42.082780+0000","last_undegraded":"2023-10-06T02:26:42.082780+0000","last_fullsized":"2023-10-06T02:26:42.082780+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_clean_scrub_stamp":"2023-10-06T02:26:41.040467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:08:02.085597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":50,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-06T02:26:42.065623+0000","last_change":"2023-10-06T02:23:01.988631+0000","last_active":"2023-10-06T02:26:42.065623+0000","last_peered":"2023-10-06T02:26:42.065623+0000","last_clean":"2023-10-06T02:26:42.065623+0000","last_became_active":"2023-10-06T02:23:01.988303+0000","last_became_peered":"2023-10-06T02:23:01.988303+0000","last_unstale":"2023-10-06T02:26:42.065623+0000","last_undegraded":"2023-10-06T02:26:42.065623+0000","last_fullsized":"2023-10-06T02:26:42.065623+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:38.211466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.076113+0000","last_change":"2023-10-06T02:26:43.076113+0000","last_active":"2023-10-06T02:26:43.076113+0000","last_peered":"2023-10-06T02:26:43.076113+0000","last_clean":"2023-10-06T02:26:43.076113+0000","last_became_active":"2023-10-06T02:26:43.075805+0000","last_became_peered":"2023-10-06T02:26:43.075805+0000","last_unstale":"2023-10-06T02:26:43.076113+0000","last_undegraded":"2023-10-06T02:26:43.076113+0000","last_fullsized":"2023-10-06T02:26:43.076113+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:55.811855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-06T02:26:42.083299+0000","last_change":"2023-10-06T02:26:42.078753+0000","last_active":"2023-10-06T02:26:42.083299+0000","last_peered":"2023-10-06T02:26:42.083299+0000","last_clean":"2023-10-06T02:26:42.083299+0000","last_became_active":"2023-10-06T02:26:42.078443+0000","last_became_peered":"2023-10-06T02:26:42.078443+0000","last_unstale":"2023-10-06T02:26:42.083299+0000","last_undegraded":"2023-10-06T02:26:42.083299+0000","last_fullsized":"2023-10-06T02:26:42.083299+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_clean_scrub_stamp":"2023-10-06T02:26:41.040467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:51.420493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":46,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-06T02:26:42.065420+0000","last_change":"2023-10-06T02:23:02.054338+0000","last_active":"2023-10-06T02:26:42.065420+0000","last_peered":"2023-10-06T02:26:42.065420+0000","last_clean":"2023-10-06T02:26:42.065420+0000","last_became_active":"2023-10-06T02:23:02.053991+0000","last_became_peered":"2023-10-06T02:23:02.053991+0000","last_unstale":"2023-10-06T02:26:42.065420+0000","last_undegraded":"2023-10-06T02:26:42.065420+0000","last_fullsized":"2023-10-06T02:26:42.065420+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:39:30.192838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.090804+0000","last_change":"2023-10-06T02:26:43.090804+0000","last_active":"2023-10-06T02:26:43.090804+0000","last_peered":"2023-10-06T02:26:43.090804+0000","last_clean":"2023-10-06T02:26:43.090804+0000","last_became_active":"2023-10-06T02:26:43.090520+0000","last_became_peered":"2023-10-06T02:26:43.090520+0000","last_unstale":"2023-10-06T02:26:43.090804+0000","last_undegraded":"2023-10-06T02:26:43.090804+0000","last_fullsized":"2023-10-06T02:26:43.090804+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:58:23.591759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-06T02:26:42.080659+0000","last_change":"2023-10-06T02:26:42.071564+0000","last_active":"2023-10-06T02:26:42.080659+0000","last_peered":"2023-10-06T02:26:42.080659+0000","last_clean":"2023-10-06T02:26:42.080659+0000","last_became_active":"2023-10-06T02:26:42.071295+0000","last_became_peered":"2023-10-06T02:26:42.071295+0000","last_unstale":"2023-10-06T02:26:42.080659+0000","last_undegraded":"2023-10-06T02:26:42.080659+0000","last_fullsized":"2023-10-06T02:26:42.080659+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_clean_scrub_stamp":"2023-10-06T02:26:41.040467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:27:49.677171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":50,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-06T02:26:42.065796+0000","last_change":"2023-10-06T02:23:01.988140+0000","last_active":"2023-10-06T02:26:42.065796+0000","last_peered":"2023-10-06T02:26:42.065796+0000","last_clean":"2023-10-06T02:26:42.065796+0000","last_became_active":"2023-10-06T02:23:01.987769+0000","last_became_peered":"2023-10-06T02:23:01.987769+0000","last_unstale":"2023-10-06T02:26:42.065796+0000","last_undegraded":"2023-10-06T02:26:42.065796+0000","last_fullsized":"2023-10-06T02:26:42.065796+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:20:08.391912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":196,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-06T02:26:42.067688+0000","last_change":"2023-10-06T02:22:55.032519+0000","last_active":"2023-10-06T02:26:42.067688+0000","last_peered":"2023-10-06T02:26:42.067688+0000","last_clean":"2023-10-06T02:26:42.067688+0000","last_became_active":"2023-10-06T02:22:55.031367+0000","last_became_peered":"2023-10-06T02:22:55.031367+0000","last_unstale":"2023-10-06T02:26:42.067688+0000","last_undegraded":"2023-10-06T02:26:42.067688+0000","last_fullsized":"2023-10-06T02:26:42.067688+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:22:53.309097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:22:53.309097+0000","last_clean_scrub_stamp":"2023-10-06T02:22:53.309097+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:53:46.520909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.075021+0000","last_change":"2023-10-06T02:26:43.075021+0000","last_active":"2023-10-06T02:26:43.075021+0000","last_peered":"2023-10-06T02:26:43.075021+0000","last_clean":"2023-10-06T02:26:43.075021+0000","last_became_active":"2023-10-06T02:26:43.074541+0000","last_became_peered":"2023-10-06T02:26:43.074541+0000","last_unstale":"2023-10-06T02:26:43.075021+0000","last_undegraded":"2023-10-06T02:26:43.075021+0000","last_fullsized":"2023-10-06T02:26:43.075021+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:55:59.959651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"60'2","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.089600+0000","last_change":"2023-10-06T02:26:42.068544+0000","last_active":"2023-10-06T02:26:43.089600+0000","last_peered":"2023-10-06T02:26:43.089600+0000","last_clean":"2023-10-06T02:26:43.089600+0000","last_became_active":"2023-10-06T02:26:42.068138+0000","last_became_peered":"2023-10-06T02:26:42.068138+0000","last_unstale":"2023-10-06T02:26:43.089600+0000","last_undegraded":"2023-10-06T02:26:43.089600+0000","last_fullsized":"2023-10-06T02:26:43.089600+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_clean_scrub_stamp":"2023-10-06T02:26:41.040467+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:57:19.405017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":43,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-06T02:26:42.067783+0000","last_change":"2023-10-06T02:23:01.990773+0000","last_active":"2023-10-06T02:26:42.067783+0000","last_peered":"2023-10-06T02:26:42.067783+0000","last_clean":"2023-10-06T02:26:42.067783+0000","last_became_active":"2023-10-06T02:23:01.990511+0000","last_became_peered":"2023-10-06T02:23:01.990511+0000","last_unstale":"2023-10-06T02:26:42.067783+0000","last_undegraded":"2023-10-06T02:26:42.067783+0000","last_fullsized":"2023-10-06T02:26:42.067783+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:42:54.797731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.088244+0000","last_change":"2023-10-06T02:26:43.088244+0000","last_active":"2023-10-06T02:26:43.088244+0000","last_peered":"2023-10-06T02:26:43.088244+0000","last_clean":"2023-10-06T02:26:43.088244+0000","last_became_active":"2023-10-06T02:26:43.087810+0000","last_became_peered":"2023-10-06T02:26:43.087810+0000","last_unstale":"2023-10-06T02:26:43.088244+0000","last_undegraded":"2023-10-06T02:26:43.088244+0000","last_fullsized":"2023-10-06T02:26:43.088244+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:06:31.428249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"60'2","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.101281+0000","last_change":"2023-10-06T02:26:42.084296+0000","last_active":"2023-10-06T02:26:43.101281+0000","last_peered":"2023-10-06T02:26:43.101281+0000","last_clean":"2023-10-06T02:26:43.101281+0000","last_became_active":"2023-10-06T02:26:42.083994+0000","last_became_peered":"2023-10-06T02:26:42.083994+0000","last_unstale":"2023-10-06T02:26:43.101281+0000","last_undegraded":"2023-10-06T02:26:43.101281+0000","last_fullsized":"2023-10-06T02:26:43.101281+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_clean_scrub_stamp":"2023-10-06T02:26:41.040467+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:31:54.995803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":43,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-06T02:26:42.075880+0000","last_change":"2023-10-06T02:23:02.054069+0000","last_active":"2023-10-06T02:26:42.075880+0000","last_peered":"2023-10-06T02:26:42.075880+0000","last_clean":"2023-10-06T02:26:42.075880+0000","last_became_active":"2023-10-06T02:23:02.053739+0000","last_became_peered":"2023-10-06T02:23:02.053739+0000","last_unstale":"2023-10-06T02:26:42.075880+0000","last_undegraded":"2023-10-06T02:26:42.075880+0000","last_fullsized":"2023-10-06T02:26:42.075880+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:57:48.822608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.088111+0000","last_change":"2023-10-06T02:26:43.088111+0000","last_active":"2023-10-06T02:26:43.088111+0000","last_peered":"2023-10-06T02:26:43.088111+0000","last_clean":"2023-10-06T02:26:43.088111+0000","last_became_active":"2023-10-06T02:26:43.087811+0000","last_became_peered":"2023-10-06T02:26:43.087811+0000","last_unstale":"2023-10-06T02:26:43.088111+0000","last_undegraded":"2023-10-06T02:26:43.088111+0000","last_fullsized":"2023-10-06T02:26:43.088111+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:48:35.546466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"60'1","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.070651+0000","last_change":"2023-10-06T02:26:42.082060+0000","last_active":"2023-10-06T02:26:43.070651+0000","last_peered":"2023-10-06T02:26:43.070651+0000","last_clean":"2023-10-06T02:26:43.070651+0000","last_became_active":"2023-10-06T02:26:42.081634+0000","last_became_peered":"2023-10-06T02:26:42.081634+0000","last_unstale":"2023-10-06T02:26:43.070651+0000","last_undegraded":"2023-10-06T02:26:43.070651+0000","last_fullsized":"2023-10-06T02:26:43.070651+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_clean_scrub_stamp":"2023-10-06T02:26:41.040467+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:50:37.582181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":44,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-06T02:26:42.072413+0000","last_change":"2023-10-06T02:23:01.989934+0000","last_active":"2023-10-06T02:26:42.072413+0000","last_peered":"2023-10-06T02:26:42.072413+0000","last_clean":"2023-10-06T02:26:42.072413+0000","last_became_active":"2023-10-06T02:23:01.989706+0000","last_became_peered":"2023-10-06T02:23:01.989706+0000","last_unstale":"2023-10-06T02:26:42.072413+0000","last_undegraded":"2023-10-06T02:26:42.072413+0000","last_fullsized":"2023-10-06T02:26:42.072413+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:06.212251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.078135+0000","last_change":"2023-10-06T02:26:43.078135+0000","last_active":"2023-10-06T02:26:43.078135+0000","last_peered":"2023-10-06T02:26:43.078135+0000","last_clean":"2023-10-06T02:26:43.078135+0000","last_became_active":"2023-10-06T02:26:43.077573+0000","last_became_peered":"2023-10-06T02:26:43.077573+0000","last_unstale":"2023-10-06T02:26:43.078135+0000","last_undegraded":"2023-10-06T02:26:43.078135+0000","last_fullsized":"2023-10-06T02:26:43.078135+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:57:39.226225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-06T02:26:42.080726+0000","last_change":"2023-10-06T02:26:42.074940+0000","last_active":"2023-10-06T02:26:42.080726+0000","last_peered":"2023-10-06T02:26:42.080726+0000","last_clean":"2023-10-06T02:26:42.080726+0000","last_became_active":"2023-10-06T02:26:42.074672+0000","last_became_peered":"2023-10-06T02:26:42.074672+0000","last_unstale":"2023-10-06T02:26:42.080726+0000","last_undegraded":"2023-10-06T02:26:42.080726+0000","last_fullsized":"2023-10-06T02:26:42.080726+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_clean_scrub_stamp":"2023-10-06T02:26:41.040467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:44:17.259680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":46,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-06T02:26:42.068912+0000","last_change":"2023-10-06T02:23:01.992935+0000","last_active":"2023-10-06T02:26:42.068912+0000","last_peered":"2023-10-06T02:26:42.068912+0000","last_clean":"2023-10-06T02:26:42.068912+0000","last_became_active":"2023-10-06T02:23:01.992568+0000","last_became_peered":"2023-10-06T02:23:01.992568+0000","last_unstale":"2023-10-06T02:26:42.068912+0000","last_undegraded":"2023-10-06T02:26:42.068912+0000","last_fullsized":"2023-10-06T02:26:42.068912+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:06.212251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574898,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302656,"kb_used_data":1272,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069184,"statfs":{"total":96636764160,"available":96326844416,"internally_reserved":0,"allocated":1302528,"data_stored":898103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.64200000000000002,"15min":0.63800000000000001},"min":{"1min":0.53100000000000003,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.878,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.61599999999999999,"15min":0.60399999999999998},"min":{"1min":0.45400000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.95399999999999996,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.52500000000000002}]},{"osd":1,"last update":"Fri Oct 6 02:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.70399999999999996,"15min":0.68300000000000005},"min":{"1min":0.59499999999999997,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.95199999999999996,"5min":1.147,"15min":1.147},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.68200000000000005,"15min":0.67500000000000004},"min":{"1min":0.504,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.91200000000000003,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.63700000000000001}]},{"osd":2,"last update":"Fri Oct 6 02:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.69799999999999995,"15min":0.69599999999999995},"min":{"1min":0.51100000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.88200000000000001,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.69599999999999995,"15min":0.69299999999999995},"min":{"1min":0.54200000000000004,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.98399999999999999,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.58699999999999997}]},{"osd":3,"last update":"Fri Oct 6 02:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.73899999999999999,"15min":0.72999999999999998},"min":{"1min":0.60599999999999998,"5min":0.497,"15min":0.497},"max":{"1min":0.98999999999999999,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.78100000000000003,"15min":0.77900000000000003},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.98099999999999998,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.745}]},{"osd":4,"last update":"Fri Oct 6 02:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.68000000000000005,"15min":0.70599999999999996},"min":{"1min":0.42999999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.81200000000000006,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.503},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.752,"15min":0.77800000000000002},"min":{"1min":0.41699999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.89600000000000002,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.60899999999999999}]},{"osd":5,"last update":"Fri Oct 6 02:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.72899999999999998,"15min":0.73999999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.73299999999999998,"15min":0.74099999999999999},"min":{"1min":0.52900000000000003,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.91900000000000004,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.69399999999999995}]},{"osd":6,"last update":"Fri Oct 6 02:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.79000000000000004,"15min":0.79200000000000004},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":0.96799999999999997,"5min":1.056,"15min":1.056},"last":0.753},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.81699999999999995,"15min":0.81899999999999995},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.0109999999999999,"5min":1.276,"15min":1.276},"last":0.67700000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574898,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297996,"kb_used_data":708,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073844,"statfs":{"total":96636764160,"available":96331616256,"internally_reserved":0,"allocated":724992,"data_stored":314885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:25:58 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.66600000000000004,"15min":0.66200000000000003},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.747},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.70399999999999996,"15min":0.69399999999999995},"min":{"1min":0.51800000000000002,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.92900000000000005,"5min":1.119,"15min":1.119},"last":0.81100000000000005}]},{"osd":1,"last update":"Fri Oct 6 02:25:58 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.63700000000000001,"15min":0.64300000000000002},"min":{"1min":0.40999999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.91200000000000003,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.67200000000000004,"15min":0.66600000000000004},"min":{"1min":0.41899999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.91000000000000003,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.621}]},{"osd":2,"last update":"Fri Oct 6 02:25:58 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.70599999999999996,"15min":0.69999999999999996},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.98499999999999999,"5min":1.137,"15min":1.137},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.72199999999999998,"15min":0.70899999999999996},"min":{"1min":0.47699999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.91700000000000004,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.77000000000000002}]},{"osd":3,"last update":"Fri Oct 6 02:25:58 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.66500000000000004,"15min":0.65400000000000003},"min":{"1min":0.53600000000000003,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.96399999999999997,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.70999999999999996,"15min":0.70199999999999996},"min":{"1min":0.49299999999999999,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.94899999999999995,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.70899999999999996}]},{"osd":4,"last update":"Fri Oct 6 02:25:58 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.64000000000000001,"15min":0.63400000000000001},"min":{"1min":0.39900000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.871,"5min":1.095,"15min":1.095},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.66700000000000004,"15min":0.65300000000000002},"min":{"1min":0.46100000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.89000000000000001,"5min":1.29,"15min":1.29},"last":0.58899999999999997}]},{"osd":5,"last update":"Fri Oct 6 02:25:58 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.69499999999999995,"15min":0.70199999999999996},"min":{"1min":0.50600000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.83999999999999997,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.70899999999999996,"15min":0.70299999999999996},"min":{"1min":0.46899999999999997,"5min":0.436,"15min":0.436},"max":{"1min":0.89700000000000002,"5min":1.02,"15min":1.02},"last":0.79200000000000004}]},{"osd":7,"last update":"Fri Oct 6 02:25:58 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.75,"15min":0.749},"min":{"1min":0.52600000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.97799999999999998,"5min":1.458,"15min":1.458},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.53500000000000003,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.88500000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.86899999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574897,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297980,"kb_used_data":692,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073860,"statfs":{"total":96636764160,"available":96331632640,"internally_reserved":0,"allocated":708608,"data_stored":307735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:25:55 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.46300000000000002,"15min":0.46700000000000003},"min":{"1min":0.316,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.65500000000000003,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.496},{"interface":"front","average":{"1min":0.503,"5min":0.51600000000000001,"15min":0.53600000000000003},"min":{"1min":0.373,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.71099999999999997,"5min":1.788,"15min":1.788},"last":0.32600000000000001}]},{"osd":2,"last update":"Fri Oct 6 02:25:55 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.51100000000000001,"15min":0.53800000000000003},"min":{"1min":0.34200000000000003,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.60899999999999999,"5min":2.0840000000000001,"15min":2.0840000000000001},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.55200000000000005,"15min":0.56799999999999995},"min":{"1min":0.39500000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.68000000000000005,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.438}]},{"osd":3,"last update":"Fri Oct 6 02:25:55 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.61699999999999999,"15min":0.62},"min":{"1min":0.47599999999999998,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.80100000000000005,"5min":1.266,"15min":1.266},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.61899999999999999,"15min":0.65700000000000003},"min":{"1min":0.46200000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.66800000000000004,"5min":1.861,"15min":1.861},"last":0.372}]},{"osd":4,"last update":"Fri Oct 6 02:25:55 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.85699999999999998,"15min":0.873},"min":{"1min":0.68799999999999994,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.1160000000000001,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.876,"5min":0.85899999999999999,"15min":0.873},"min":{"1min":0.56999999999999995,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0820000000000001,"5min":2.0179999999999998,"15min":2.0179999999999998},"last":0.52900000000000003}]},{"osd":5,"last update":"Fri Oct 6 02:25:55 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.85099999999999998,"15min":0.88300000000000001},"min":{"1min":0.53900000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.91900000000000004,"5min":1.9830000000000001,"15min":1.9830000000000001},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.82199999999999995,"15min":0.83799999999999997},"min":{"1min":0.68500000000000005,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.026,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.64500000000000002}]},{"osd":6,"last update":"Fri Oct 6 02:25:55 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.86499999999999999,"15min":0.88300000000000001},"min":{"1min":0.65800000000000003,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0369999999999999,"5min":1.829,"15min":1.829},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.90500000000000003,"15min":0.93300000000000005},"min":{"1min":0.629,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.103,"5min":2.052,"15min":2.052},"last":0.56399999999999995}]},{"osd":7,"last update":"Fri Oct 6 02:25:55 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.83899999999999997,"15min":0.84999999999999998},"min":{"1min":0.60499999999999998,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.0680000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.87,"15min":0.89700000000000002},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.048,"5min":1.895,"15min":1.895},"last":0.47799999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574898,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302656,"kb_used_data":1272,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069184,"statfs":{"total":96636764160,"available":96326844416,"internally_reserved":0,"allocated":1302528,"data_stored":898103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 02:26:02 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.44700000000000001,"15min":0.45900000000000002},"min":{"1min":0.32800000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.57699999999999996,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.505,"5min":0.48299999999999998,"15min":0.48499999999999999},"min":{"1min":0.37,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.38500000000000001}]},{"osd":2,"last update":"Fri Oct 6 02:26:02 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.504,"15min":0.51600000000000001},"min":{"1min":0.39600000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.63500000000000001,"5min":1.069,"15min":1.069},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.501,"5min":0.51000000000000001,"15min":0.52200000000000002},"min":{"1min":0.379,"5min":0.372,"15min":0.372},"max":{"1min":0.74299999999999999,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.44800000000000001}]},{"osd":3,"last update":"Fri Oct 6 02:26:02 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.50900000000000001},"min":{"1min":0.39400000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.60599999999999998,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.59399999999999997,"15min":0.59699999999999998},"min":{"1min":0.51500000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.70799999999999996,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.50900000000000001}]},{"osd":4,"last update":"Fri Oct 6 02:26:02 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.79300000000000004,"15min":0.81100000000000005},"min":{"1min":0.56299999999999994,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.056,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79500000000000004,"15min":0.80900000000000005},"min":{"1min":0.57099999999999995,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.056,"5min":1.304,"15min":1.304},"last":0.71999999999999997}]},{"osd":5,"last update":"Fri Oct 6 02:26:02 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.78500000000000003,"15min":0.79400000000000004},"min":{"1min":0.63700000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.98999999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.79800000000000004,"15min":0.80500000000000005},"min":{"1min":0.59799999999999998,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.0760000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.752}]},{"osd":6,"last update":"Fri Oct 6 02:26:02 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.81999999999999995,"15min":0.82799999999999996},"min":{"1min":0.65200000000000002,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.079,"5min":1.167,"15min":1.167},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.81999999999999995,"15min":0.83999999999999997},"min":{"1min":0.58899999999999997,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.032,"5min":1.069,"15min":1.069},"last":0.82599999999999996}]},{"osd":7,"last update":"Fri Oct 6 02:26:02 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.81799999999999995,"15min":0.83299999999999996},"min":{"1min":0.63300000000000001,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.81699999999999995,"15min":0.83599999999999997},"min":{"1min":0.59099999999999997,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.04,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.81399999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574899,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297980,"kb_used_data":692,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073860,"statfs":{"total":96636764160,"available":96331632640,"internally_reserved":0,"allocated":708608,"data_stored":307735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.44,"15min":0.443},"min":{"1min":0.28699999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.48299999999999998,"15min":0.48799999999999999},"min":{"1min":0.33400000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.84499999999999997,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.39300000000000002}]},{"osd":1,"last update":"Fri Oct 6 02:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.502,"15min":0.50800000000000001},"min":{"1min":0.33600000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.67100000000000004,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.58899999999999997,"15min":0.621},"min":{"1min":0.38800000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.88500000000000001,"5min":1.345,"15min":1.345},"last":0.52200000000000002}]},{"osd":3,"last update":"Fri Oct 6 02:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53300000000000003,"15min":0.54000000000000004},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.76500000000000001,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.61599999999999999,"15min":0.627},"min":{"1min":0.48099999999999998,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.80700000000000005,"5min":1.028,"15min":1.028},"last":0.65400000000000003}]},{"osd":4,"last update":"Fri Oct 6 02:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.77300000000000002,"15min":0.78400000000000003},"min":{"1min":0.54600000000000004,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.109,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.84199999999999997,"15min":0.86099999999999999},"min":{"1min":0.64000000000000001,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.1240000000000001,"5min":1.621,"15min":1.621},"last":0.71799999999999997}]},{"osd":5,"last update":"Fri Oct 6 02:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.78800000000000003,"15min":0.81200000000000006},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.0580000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.81599999999999995,"15min":0.83399999999999996},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.153,"5min":1.258,"15min":1.258},"last":0.78500000000000003}]},{"osd":6,"last update":"Fri Oct 6 02:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.83699999999999997,"15min":0.85899999999999999},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.1870000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.79700000000000004,"15min":0.80200000000000005},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.82599999999999996}]},{"osd":7,"last update":"Fri Oct 6 02:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.877,"15min":0.90000000000000002},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.099,"5min":1.585,"15min":1.585},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.82399999999999995,"15min":0.84199999999999997},"min":{"1min":0.55500000000000005,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.0369999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.90800000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574896,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297980,"kb_used_data":692,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073860,"statfs":{"total":96636764160,"available":96331632640,"internally_reserved":0,"allocated":708608,"data_stored":307735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:25:59 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.499,"15min":0.52000000000000002},"min":{"1min":0.36199999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.63,"5min":2.024,"15min":2.024},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.52200000000000002,"15min":0.53200000000000003},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.84699999999999998,"5min":2.181,"15min":2.181},"last":0.40699999999999997}]},{"osd":1,"last update":"Fri Oct 6 02:25:59 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.64100000000000001,"15min":0.72099999999999997},"min":{"1min":0.36699999999999999,"5min":0.312,"15min":0.312},"max":{"1min":0.61899999999999999,"5min":4.492,"15min":4.492},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.68899999999999995,"15min":0.76200000000000001},"min":{"1min":0.41999999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.65900000000000003,"5min":4.569,"15min":4.569},"last":0.51900000000000002}]},{"osd":2,"last update":"Fri Oct 6 02:25:59 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.66700000000000004,"15min":0.67500000000000004},"min":{"1min":0.58699999999999997,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.88400000000000001,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.63800000000000001,"15min":0.66400000000000003},"min":{"1min":0.48599999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.90600000000000003,"5min":2.4260000000000002,"15min":2.4260000000000002},"last":0.50800000000000001}]},{"osd":4,"last update":"Fri Oct 6 02:25:59 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.80800000000000005,"15min":0.84599999999999997},"min":{"1min":0.502,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.97899999999999998,"5min":2.214,"15min":2.214},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.80400000000000005,"15min":0.82299999999999995},"min":{"1min":0.60399999999999998,"5min":0.5,"15min":0.5},"max":{"1min":1.079,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.64000000000000001}]},{"osd":5,"last update":"Fri Oct 6 02:25:59 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.82099999999999995,"15min":0.83399999999999996},"min":{"1min":0.56999999999999995,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.0640000000000001,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.82999999999999996,"15min":0.872},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.93300000000000005,"5min":2.242,"15min":2.242},"last":0.72499999999999998}]},{"osd":6,"last update":"Fri Oct 6 02:25:59 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.80300000000000005,"15min":0.82799999999999996},"min":{"1min":0.52400000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.877,"5min":1.357,"15min":1.357},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.82999999999999996,"15min":0.84099999999999997},"min":{"1min":0.68700000000000006,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.0529999999999999,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.66100000000000003}]},{"osd":7,"last update":"Fri Oct 6 02:25:59 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.84599999999999997,"15min":0.874},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.0169999999999999,"5min":2.27,"15min":2.27},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.877,"5min":0.85199999999999998,"15min":0.871},"min":{"1min":0.67700000000000005,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.028,"5min":1.325,"15min":1.325},"last":0.55600000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574898,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302076,"kb_used_data":692,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069764,"statfs":{"total":96636764160,"available":96327438336,"internally_reserved":0,"allocated":708608,"data_stored":307735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:25:59 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.96299999999999997,"15min":1.087},"min":{"1min":0.44700000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.93100000000000005,"5min":12.337,"15min":12.337},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.879,"15min":0.97299999999999998},"min":{"1min":0.497,"5min":0.438,"15min":0.438},"max":{"1min":0.92200000000000004,"5min":9.6539999999999999,"15min":9.6539999999999999},"last":0.73099999999999998}]},{"osd":1,"last update":"Fri Oct 6 02:25:59 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.69399999999999995,"15min":0.70399999999999996},"min":{"1min":0.51800000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.82299999999999995,"5min":3.181,"15min":3.181},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.97499999999999998,"15min":1.107},"min":{"1min":0.53400000000000003,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.85199999999999998,"5min":12.946999999999999,"15min":12.946999999999999},"last":0.47999999999999998}]},{"osd":2,"last update":"Fri Oct 6 02:25:59 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.998,"15min":1.137},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.94899999999999995,"5min":12.377000000000001,"15min":12.377000000000001},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.754,"5min":0.84899999999999998,"15min":0.91100000000000003},"min":{"1min":0.58499999999999996,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.98299999999999998,"5min":6.3220000000000001,"15min":6.3220000000000001},"last":0.60799999999999998}]},{"osd":3,"last update":"Fri Oct 6 02:25:59 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.95599999999999996,"15min":1.0580000000000001},"min":{"1min":0.56299999999999994,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.97499999999999998,"5min":9.7720000000000002,"15min":9.7720000000000002},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.76500000000000001,"15min":0.79200000000000004},"min":{"1min":0.56000000000000005,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.90000000000000002,"5min":3.0840000000000001,"15min":3.0840000000000001},"last":0.70799999999999996}]},{"osd":5,"last update":"Fri Oct 6 02:25:59 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.81599999999999995,"15min":0.88700000000000001},"min":{"1min":0.50600000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.83299999999999996,"5min":6.2690000000000001,"15min":6.2690000000000001},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.97499999999999998,"15min":1.117},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.80900000000000005,"5min":10.769,"15min":10.769},"last":0.86099999999999999}]},{"osd":6,"last update":"Fri Oct 6 02:25:59 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.73799999999999999,"15min":0.77800000000000002},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.82799999999999996,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.92500000000000004,"15min":1.026},"min":{"1min":0.49099999999999999,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.89400000000000002,"5min":8.0899999999999999,"15min":8.0899999999999999},"last":0.75700000000000001}]},{"osd":7,"last update":"Fri Oct 6 02:25:59 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":1.016,"15min":1.143},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.88300000000000001,"5min":9.7089999999999996,"15min":9.7089999999999996},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.95799999999999996,"15min":1.0580000000000001},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.89300000000000002,"5min":7.6719999999999997,"15min":7.6719999999999997},"last":0.83699999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574898,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297980,"kb_used_data":692,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073860,"statfs":{"total":96636764160,"available":96331632640,"internally_reserved":0,"allocated":708608,"data_stored":307735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:25:59 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.64000000000000001,"15min":0.625},"min":{"1min":0.49099999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.79000000000000004,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.63600000000000001,"15min":0.60399999999999998},"min":{"1min":0.623,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.41699999999999998}]},{"osd":1,"last update":"Fri Oct 6 02:25:59 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.66700000000000004,"15min":0.64900000000000002},"min":{"1min":0.51900000000000002,"5min":0.434,"15min":0.434},"max":{"1min":1.002,"5min":1.089,"15min":1.089},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.75,"5min":0.68400000000000005,"15min":0.67300000000000004},"min":{"1min":0.63900000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.93600000000000005,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.56699999999999995}]},{"osd":2,"last update":"Fri Oct 6 02:25:59 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.69299999999999995,"15min":0.66700000000000004},"min":{"1min":0.66200000000000003,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.69999999999999996,"15min":0.68200000000000005},"min":{"1min":0.61099999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.995,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.44600000000000001}]},{"osd":3,"last update":"Fri Oct 6 02:25:59 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.74399999999999999,"15min":0.72499999999999998},"min":{"1min":0.59899999999999998,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.92100000000000004,"5min":1.042,"15min":1.042},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.78100000000000003,"15min":0.76400000000000001},"min":{"1min":0.63500000000000001,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.0760000000000001,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.59499999999999997}]},{"osd":4,"last update":"Fri Oct 6 02:25:59 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.67600000000000005,"15min":0.67800000000000005},"min":{"1min":0.40300000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.80800000000000005,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.68300000000000005,"15min":0.68600000000000005},"min":{"1min":0.42899999999999999,"5min":0.373,"15min":0.373},"max":{"1min":0.82099999999999995,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.68899999999999995}]},{"osd":6,"last update":"Fri Oct 6 02:25:59 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.749,"15min":0.747},"min":{"1min":0.47899999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.92900000000000005,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.82599999999999996,"15min":0.82399999999999995},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.95799999999999996,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.76400000000000001}]},{"osd":7,"last update":"Fri Oct 6 02:25:59 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.77400000000000002,"15min":0.76800000000000002},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.94399999999999995,"5min":1.089,"15min":1.089},"last":0.621},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.83599999999999997,"15min":0.84199999999999997},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.0469999999999999,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.77100000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T02:26:47.462 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T02:26:47.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.679+0000 7f9202d9d700 1 Processor -- start 2023-10-06T02:26:47.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.680+0000 7f9202d9d700 1 -- start start 2023-10-06T02:26:47.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.681+0000 7f9202d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f91fc1574a0 0x7f91fc1584c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:47.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.681+0000 7f9202d9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f91fc158a00 0x7f91fc0b2420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:47.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.681+0000 7f9202d9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91fc0cd9d0 con 0x7f91fc0b2960 2023-10-06T02:26:47.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.681+0000 7f9202d9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91fc0b2de0 con 0x7f91fc158a00 2023-10-06T02:26:47.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.681+0000 7f9202d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91fc0cc190 con 0x7f91fc1574a0 2023-10-06T02:26:47.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.681+0000 7f920259c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f91fc0b2960 0x7f91fc15b090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46688/0 (socket says 172.21.15.31:46688) 2023-10-06T02:26:47.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.681+0000 7f920259c700 1 -- 172.21.15.31:0/2614355546 learned_addr learned my addr 172.21.15.31:0/2614355546 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:26:47.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.681+0000 7f920159a700 1 --2- 172.21.15.31:0/2614355546 >> v2:172.21.15.62:3300/0 conn(0x7f91fc158a00 0x7f91fc0b2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:47.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.681+0000 7f9201d9b700 1 --2- 172.21.15.31:0/2614355546 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f91fc1574a0 0x7f91fc1584c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:47.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.682+0000 7f9200d99700 1 -- 172.21.15.31:0/2614355546 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4209200408 0 0) 0x7f91fc0cd9d0 con 0x7f91fc0b2960 2023-10-06T02:26:47.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.682+0000 7f9200d99700 1 -- 172.21.15.31:0/2614355546 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91e8003570 con 0x7f91fc0b2960 2023-10-06T02:26:47.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.682+0000 7f920159a700 1 -- 172.21.15.31:0/2614355546 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f91fc1574a0 msgr2=0x7f91fc1584c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:26:47.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.682+0000 7f920159a700 1 --2- 172.21.15.31:0/2614355546 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f91fc1574a0 0x7f91fc1584c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:47.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.682+0000 7f920159a700 1 -- 172.21.15.31:0/2614355546 >> v1:172.21.15.31:6789/0 conn(0x7f91fc0b2960 legacy=0x7f91fc15b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:47.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.682+0000 7f920159a700 1 -- 172.21.15.31:0/2614355546 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91fc0ccdc0 con 0x7f91fc158a00 2023-10-06T02:26:47.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.682+0000 7f920159a700 1 --2- 172.21.15.31:0/2614355546 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91fc158a00 0x7f91fc0b2420 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f91ec0098e0 tx=0x7f91ec0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16a40e29feabef04 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:47.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.682+0000 7f9200d99700 1 -- 172.21.15.31:0/2614355546 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91ec011630 con 0x7f91fc158a00 2023-10-06T02:26:47.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.683+0000 7f9200d99700 1 -- 172.21.15.31:0/2614355546 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91ec011790 con 0x7f91fc158a00 2023-10-06T02:26:47.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.683+0000 7f9200d99700 1 -- 172.21.15.31:0/2614355546 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91ec011b30 con 0x7f91fc158a00 2023-10-06T02:26:47.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.683+0000 7f9202d9d700 1 -- 172.21.15.31:0/2614355546 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91fc158a00 msgr2=0x7f91fc0b2420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:47.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.683+0000 7f9202d9d700 1 --2- 172.21.15.31:0/2614355546 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91fc158a00 0x7f91fc0b2420 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f91ec0098e0 tx=0x7f91ec0091a0 comp rx=0 tx=0).stop 2023-10-06T02:26:47.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.683+0000 7f9202d9d700 1 -- 172.21.15.31:0/2614355546 shutdown_connections 2023-10-06T02:26:47.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.683+0000 7f9202d9d700 1 --2- 172.21.15.31:0/2614355546 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f91fc1574a0 0x7f91fc1584c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:47.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.683+0000 7f9202d9d700 1 --2- 172.21.15.31:0/2614355546 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91fc158a00 0x7f91fc0b2420 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:47.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.683+0000 7f9202d9d700 1 -- 172.21.15.31:0/2614355546 >> 172.21.15.31:0/2614355546 conn(0x7f91fc0675a0 msgr2=0x7f91fc0679a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:47.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.683+0000 7f9202d9d700 1 -- 172.21.15.31:0/2614355546 shutdown_connections 2023-10-06T02:26:47.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.683+0000 7f9202d9d700 1 -- 172.21.15.31:0/2614355546 wait complete. 2023-10-06T02:26:47.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.685+0000 7f9202d9d700 1 Processor -- start 2023-10-06T02:26:47.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.685+0000 7f9202d9d700 1 -- start start 2023-10-06T02:26:47.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.686+0000 7f9202d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91fc0b2960 0x7f91fc1502a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:47.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.686+0000 7f9202d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91fc1574a0 0x7f91fc16ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:47.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.686+0000 7f9202d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f91fc16d3b0 0x7f91fc067970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:47.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.686+0000 7f9202d9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91fc0aaf30 con 0x7f91fc1574a0 2023-10-06T02:26:47.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.686+0000 7f9202d9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91fc0aa950 con 0x7f91fc0b2960 2023-10-06T02:26:47.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.686+0000 7f9202d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91fc16d9a0 con 0x7f91fc16d3b0 2023-10-06T02:26:47.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.686+0000 7f920159a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91fc1574a0 0x7f91fc16ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:47.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.686+0000 7f920159a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91fc1574a0 0x7f91fc16ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50002/0 (socket says 172.21.15.31:50002) 2023-10-06T02:26:47.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.686+0000 7f920159a700 1 -- 172.21.15.31:0/2435396957 learned_addr learned my addr 172.21.15.31:0/2435396957 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:26:47.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.686+0000 7f9201d9b700 1 --2- 172.21.15.31:0/2435396957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91fc0b2960 0x7f91fc1502a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:47.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.686+0000 7f920259c700 1 --2- 172.21.15.31:0/2435396957 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f91fc16d3b0 0x7f91fc067970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:47.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.686+0000 7f920159a700 1 -- 172.21.15.31:0/2435396957 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f91fc16d3b0 msgr2=0x7f91fc067970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:47.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.686+0000 7f920159a700 1 --2- 172.21.15.31:0/2435396957 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f91fc16d3b0 0x7f91fc067970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:47.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.686+0000 7f920159a700 1 -- 172.21.15.31:0/2435396957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91fc0b2960 msgr2=0x7f91fc1502a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:47.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.686+0000 7f920159a700 1 --2- 172.21.15.31:0/2435396957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91fc0b2960 0x7f91fc1502a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:47.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.686+0000 7f920159a700 1 -- 172.21.15.31:0/2435396957 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91ec009420 con 0x7f91fc1574a0 2023-10-06T02:26:47.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.687+0000 7f920159a700 1 --2- 172.21.15.31:0/2435396957 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91fc1574a0 0x7f91fc16ce70 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f91fc06ee30 tx=0x7f91ec017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:47.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.687+0000 7f91f1ffb700 1 -- 172.21.15.31:0/2435396957 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91ec011900 con 0x7f91fc1574a0 2023-10-06T02:26:47.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.687+0000 7f91f1ffb700 1 -- 172.21.15.31:0/2435396957 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91ec011a80 con 0x7f91fc1574a0 2023-10-06T02:26:47.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.687+0000 7f9202d9d700 1 -- 172.21.15.31:0/2435396957 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91fc0aabc0 con 0x7f91fc1574a0 2023-10-06T02:26:47.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.687+0000 7f91f1ffb700 1 -- 172.21.15.31:0/2435396957 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91ec025670 con 0x7f91fc1574a0 2023-10-06T02:26:47.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.687+0000 7f9202d9d700 1 -- 172.21.15.31:0/2435396957 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91fc16dc80 con 0x7f91fc1574a0 2023-10-06T02:26:47.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.688+0000 7f9202d9d700 1 -- 172.21.15.31:0/2435396957 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91fc0ab0a0 con 0x7f91fc1574a0 2023-10-06T02:26:47.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.689+0000 7f91f1ffb700 1 -- 172.21.15.31:0/2435396957 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 9) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f91ec023070 con 0x7f91fc1574a0 2023-10-06T02:26:47.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.690+0000 7f91f1ffb700 1 --2- 172.21.15.31:0/2435396957 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f91e00670e0 0x7f91e0069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:47.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.690+0000 7f91f1ffb700 1 -- 172.21.15.31:0/2435396957 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f91ec085d70 con 0x7f91fc1574a0 2023-10-06T02:26:47.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.690+0000 7f9201d9b700 1 --2- 172.21.15.31:0/2435396957 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f91e00670e0 0x7f91e0069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:47.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.691+0000 7f9201d9b700 1 --2- 172.21.15.31:0/2435396957 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f91e00670e0 0x7f91e0069590 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f91f8009da0 tx=0x7f91f8009120 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:47.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.694+0000 7f91f1ffb700 1 -- 172.21.15.31:0/2435396957 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f91ec052c00 con 0x7f91fc1574a0 2023-10-06T02:26:47.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.891+0000 7f9202d9d700 1 -- 172.21.15.31:0/2435396957 --> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f91fc16d780 con 0x7f91e00670e0 2023-10-06T02:26:47.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.900+0000 7f91f1ffb700 1 -- 172.21.15.31:0/2435396957 <== mgr.4107 v2:172.21.15.31:6832/85292 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311109 (secure 0 0 0) 0x7f91f8011a50 con 0x7f91e00670e0 2023-10-06T02:26:47.905 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:26:47.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.905+0000 7f9202d9d700 1 -- 172.21.15.31:0/2435396957 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f91e00670e0 msgr2=0x7f91e0069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:47.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.905+0000 7f9202d9d700 1 --2- 172.21.15.31:0/2435396957 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f91e00670e0 0x7f91e0069590 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f91f8009da0 tx=0x7f91f8009120 comp rx=0 tx=0).stop 2023-10-06T02:26:47.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.905+0000 7f9202d9d700 1 -- 172.21.15.31:0/2435396957 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91fc1574a0 msgr2=0x7f91fc16ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:47.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.905+0000 7f9202d9d700 1 --2- 172.21.15.31:0/2435396957 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91fc1574a0 0x7f91fc16ce70 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f91fc06ee30 tx=0x7f91ec017000 comp rx=0 tx=0).stop 2023-10-06T02:26:47.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.905+0000 7f9202d9d700 1 -- 172.21.15.31:0/2435396957 shutdown_connections 2023-10-06T02:26:47.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.905+0000 7f9202d9d700 1 --2- 172.21.15.31:0/2435396957 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f91e00670e0 0x7f91e0069590 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:47.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.905+0000 7f9202d9d700 1 --2- 172.21.15.31:0/2435396957 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f91fc16d3b0 0x7f91fc067970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:47.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.905+0000 7f9202d9d700 1 --2- 172.21.15.31:0/2435396957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91fc0b2960 0x7f91fc1502a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:47.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.905+0000 7f9202d9d700 1 --2- 172.21.15.31:0/2435396957 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91fc1574a0 0x7f91fc16ce70 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:47.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.905+0000 7f9202d9d700 1 -- 172.21.15.31:0/2435396957 >> 172.21.15.31:0/2435396957 conn(0x7f91fc0675a0 msgr2=0x7f91fc0b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:47.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.905+0000 7f9202d9d700 1 -- 172.21.15.31:0/2435396957 shutdown_connections 2023-10-06T02:26:47.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:47.905+0000 7f9202d9d700 1 -- 172.21.15.31:0/2435396957 wait complete. 2023-10-06T02:26:47.909 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-10-06T02:26:47.923 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":171,"stamp":"2023-10-06T02:26:47.165810+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":117,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2397304,"kb_used_data":6712,"kb_used_omap":0,"kb_used_meta":2390528,"kb_avail":752577416,"statfs":{"total":773094113280,"available":770639273984,"internally_reserved":0,"allocated":6873088,"data_stored":3649766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2447900672},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001164"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.091655+0000","last_change":"2023-10-06T02:26:43.091655+0000","last_active":"2023-10-06T02:26:43.091655+0000","last_peered":"2023-10-06T02:26:43.091655+0000","last_clean":"2023-10-06T02:26:43.091655+0000","last_became_active":"2023-10-06T02:26:43.091300+0000","last_became_peered":"2023-10-06T02:26:43.091300+0000","last_unstale":"2023-10-06T02:26:43.091655+0000","last_undegraded":"2023-10-06T02:26:43.091655+0000","last_fullsized":"2023-10-06T02:26:43.091655+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:57:39.226225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.083676+0000","last_change":"2023-10-06T02:26:43.083676+0000","last_active":"2023-10-06T02:26:43.083676+0000","last_peered":"2023-10-06T02:26:43.083676+0000","last_clean":"2023-10-06T02:26:43.083676+0000","last_became_active":"2023-10-06T02:26:43.083314+0000","last_became_peered":"2023-10-06T02:26:43.083314+0000","last_unstale":"2023-10-06T02:26:43.083676+0000","last_undegraded":"2023-10-06T02:26:43.083676+0000","last_fullsized":"2023-10-06T02:26:43.083676+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:42:27.666997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.086723+0000","last_change":"2023-10-06T02:26:43.086723+0000","last_active":"2023-10-06T02:26:43.086723+0000","last_peered":"2023-10-06T02:26:43.086723+0000","last_clean":"2023-10-06T02:26:43.086723+0000","last_became_active":"2023-10-06T02:26:43.086430+0000","last_became_peered":"2023-10-06T02:26:43.086430+0000","last_unstale":"2023-10-06T02:26:43.086723+0000","last_undegraded":"2023-10-06T02:26:43.086723+0000","last_fullsized":"2023-10-06T02:26:43.086723+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:50.341025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.067259+0000","last_change":"2023-10-06T02:26:43.067259+0000","last_active":"2023-10-06T02:26:43.067259+0000","last_peered":"2023-10-06T02:26:43.067259+0000","last_clean":"2023-10-06T02:26:43.067259+0000","last_became_active":"2023-10-06T02:26:43.066969+0000","last_became_peered":"2023-10-06T02:26:43.066969+0000","last_unstale":"2023-10-06T02:26:43.067259+0000","last_undegraded":"2023-10-06T02:26:43.067259+0000","last_fullsized":"2023-10-06T02:26:43.067259+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:31:56.002363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.089664+0000","last_change":"2023-10-06T02:26:43.089664+0000","last_active":"2023-10-06T02:26:43.089664+0000","last_peered":"2023-10-06T02:26:43.089664+0000","last_clean":"2023-10-06T02:26:43.089664+0000","last_became_active":"2023-10-06T02:26:43.088984+0000","last_became_peered":"2023-10-06T02:26:43.088984+0000","last_unstale":"2023-10-06T02:26:43.089664+0000","last_undegraded":"2023-10-06T02:26:43.089664+0000","last_fullsized":"2023-10-06T02:26:43.089664+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:42:27.666997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.083507+0000","last_change":"2023-10-06T02:26:43.083507+0000","last_active":"2023-10-06T02:26:43.083507+0000","last_peered":"2023-10-06T02:26:43.083507+0000","last_clean":"2023-10-06T02:26:43.083507+0000","last_became_active":"2023-10-06T02:26:43.083248+0000","last_became_peered":"2023-10-06T02:26:43.083248+0000","last_unstale":"2023-10-06T02:26:43.083507+0000","last_undegraded":"2023-10-06T02:26:43.083507+0000","last_fullsized":"2023-10-06T02:26:43.083507+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:04:15.890422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.085012+0000","last_change":"2023-10-06T02:26:43.085012+0000","last_active":"2023-10-06T02:26:43.085012+0000","last_peered":"2023-10-06T02:26:43.085012+0000","last_clean":"2023-10-06T02:26:43.085012+0000","last_became_active":"2023-10-06T02:26:43.084737+0000","last_became_peered":"2023-10-06T02:26:43.084737+0000","last_unstale":"2023-10-06T02:26:43.085012+0000","last_undegraded":"2023-10-06T02:26:43.085012+0000","last_fullsized":"2023-10-06T02:26:43.085012+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:48:13.211733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.093672+0000","last_change":"2023-10-06T02:26:43.093672+0000","last_active":"2023-10-06T02:26:43.093672+0000","last_peered":"2023-10-06T02:26:43.093672+0000","last_clean":"2023-10-06T02:26:43.093672+0000","last_became_active":"2023-10-06T02:26:43.093410+0000","last_became_peered":"2023-10-06T02:26:43.093410+0000","last_unstale":"2023-10-06T02:26:43.093672+0000","last_undegraded":"2023-10-06T02:26:43.093672+0000","last_fullsized":"2023-10-06T02:26:43.093672+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:35:02.657273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.095037+0000","last_change":"2023-10-06T02:26:43.095037+0000","last_active":"2023-10-06T02:26:43.095037+0000","last_peered":"2023-10-06T02:26:43.095037+0000","last_clean":"2023-10-06T02:26:43.095037+0000","last_became_active":"2023-10-06T02:26:43.093613+0000","last_became_peered":"2023-10-06T02:26:43.093613+0000","last_unstale":"2023-10-06T02:26:43.095037+0000","last_undegraded":"2023-10-06T02:26:43.095037+0000","last_fullsized":"2023-10-06T02:26:43.095037+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:53:29.330021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.094347+0000","last_change":"2023-10-06T02:26:43.094347+0000","last_active":"2023-10-06T02:26:43.094347+0000","last_peered":"2023-10-06T02:26:43.094347+0000","last_clean":"2023-10-06T02:26:43.094347+0000","last_became_active":"2023-10-06T02:26:43.094000+0000","last_became_peered":"2023-10-06T02:26:43.094000+0000","last_unstale":"2023-10-06T02:26:43.094347+0000","last_undegraded":"2023-10-06T02:26:43.094347+0000","last_fullsized":"2023-10-06T02:26:43.094347+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:53:54.198169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.087392+0000","last_change":"2023-10-06T02:26:43.087392+0000","last_active":"2023-10-06T02:26:43.087392+0000","last_peered":"2023-10-06T02:26:43.087392+0000","last_clean":"2023-10-06T02:26:43.087392+0000","last_became_active":"2023-10-06T02:26:43.087054+0000","last_became_peered":"2023-10-06T02:26:43.087054+0000","last_unstale":"2023-10-06T02:26:43.087392+0000","last_undegraded":"2023-10-06T02:26:43.087392+0000","last_fullsized":"2023-10-06T02:26:43.087392+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:52:16.520095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.080654+0000","last_change":"2023-10-06T02:26:43.080654+0000","last_active":"2023-10-06T02:26:43.080654+0000","last_peered":"2023-10-06T02:26:43.080654+0000","last_clean":"2023-10-06T02:26:43.080654+0000","last_became_active":"2023-10-06T02:26:43.080305+0000","last_became_peered":"2023-10-06T02:26:43.080305+0000","last_unstale":"2023-10-06T02:26:43.080654+0000","last_undegraded":"2023-10-06T02:26:43.080654+0000","last_fullsized":"2023-10-06T02:26:43.080654+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:44:18.266240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.079644+0000","last_change":"2023-10-06T02:26:43.079644+0000","last_active":"2023-10-06T02:26:43.079644+0000","last_peered":"2023-10-06T02:26:43.079644+0000","last_clean":"2023-10-06T02:26:43.079644+0000","last_became_active":"2023-10-06T02:26:43.079345+0000","last_became_peered":"2023-10-06T02:26:43.079345+0000","last_unstale":"2023-10-06T02:26:43.079644+0000","last_undegraded":"2023-10-06T02:26:43.079644+0000","last_fullsized":"2023-10-06T02:26:43.079644+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:17:56.058037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.093806+0000","last_change":"2023-10-06T02:26:43.093806+0000","last_active":"2023-10-06T02:26:43.093806+0000","last_peered":"2023-10-06T02:26:43.093806+0000","last_clean":"2023-10-06T02:26:43.093806+0000","last_became_active":"2023-10-06T02:26:43.092556+0000","last_became_peered":"2023-10-06T02:26:43.092556+0000","last_unstale":"2023-10-06T02:26:43.093806+0000","last_undegraded":"2023-10-06T02:26:43.093806+0000","last_fullsized":"2023-10-06T02:26:43.093806+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:48:32.579296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.095209+0000","last_change":"2023-10-06T02:26:43.095209+0000","last_active":"2023-10-06T02:26:43.095209+0000","last_peered":"2023-10-06T02:26:43.095209+0000","last_clean":"2023-10-06T02:26:43.095209+0000","last_became_active":"2023-10-06T02:26:43.094895+0000","last_became_peered":"2023-10-06T02:26:43.094895+0000","last_unstale":"2023-10-06T02:26:43.095209+0000","last_undegraded":"2023-10-06T02:26:43.095209+0000","last_fullsized":"2023-10-06T02:26:43.095209+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:08:29.480329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.080237+0000","last_change":"2023-10-06T02:26:43.080237+0000","last_active":"2023-10-06T02:26:43.080237+0000","last_peered":"2023-10-06T02:26:43.080237+0000","last_clean":"2023-10-06T02:26:43.080237+0000","last_became_active":"2023-10-06T02:26:43.079977+0000","last_became_peered":"2023-10-06T02:26:43.079977+0000","last_unstale":"2023-10-06T02:26:43.080237+0000","last_undegraded":"2023-10-06T02:26:43.080237+0000","last_fullsized":"2023-10-06T02:26:43.080237+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:35:29.322161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.082908+0000","last_change":"2023-10-06T02:26:43.082908+0000","last_active":"2023-10-06T02:26:43.082908+0000","last_peered":"2023-10-06T02:26:43.082908+0000","last_clean":"2023-10-06T02:26:43.082908+0000","last_became_active":"2023-10-06T02:26:43.082623+0000","last_became_peered":"2023-10-06T02:26:43.082623+0000","last_unstale":"2023-10-06T02:26:43.082908+0000","last_undegraded":"2023-10-06T02:26:43.082908+0000","last_fullsized":"2023-10-06T02:26:43.082908+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:44:24.572462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.083330+0000","last_change":"2023-10-06T02:26:43.083330+0000","last_active":"2023-10-06T02:26:43.083330+0000","last_peered":"2023-10-06T02:26:43.083330+0000","last_clean":"2023-10-06T02:26:43.083330+0000","last_became_active":"2023-10-06T02:26:43.083034+0000","last_became_peered":"2023-10-06T02:26:43.083034+0000","last_unstale":"2023-10-06T02:26:43.083330+0000","last_undegraded":"2023-10-06T02:26:43.083330+0000","last_fullsized":"2023-10-06T02:26:43.083330+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:44:24.572462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.084693+0000","last_change":"2023-10-06T02:26:43.084693+0000","last_active":"2023-10-06T02:26:43.084693+0000","last_peered":"2023-10-06T02:26:43.084693+0000","last_clean":"2023-10-06T02:26:43.084693+0000","last_became_active":"2023-10-06T02:26:43.084126+0000","last_became_peered":"2023-10-06T02:26:43.084126+0000","last_unstale":"2023-10-06T02:26:43.084693+0000","last_undegraded":"2023-10-06T02:26:43.084693+0000","last_fullsized":"2023-10-06T02:26:43.084693+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:52.427053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.084201+0000","last_change":"2023-10-06T02:26:43.084201+0000","last_active":"2023-10-06T02:26:43.084201+0000","last_peered":"2023-10-06T02:26:43.084201+0000","last_clean":"2023-10-06T02:26:43.084201+0000","last_became_active":"2023-10-06T02:26:43.083885+0000","last_became_peered":"2023-10-06T02:26:43.083885+0000","last_unstale":"2023-10-06T02:26:43.084201+0000","last_undegraded":"2023-10-06T02:26:43.084201+0000","last_fullsized":"2023-10-06T02:26:43.084201+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:40.671226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.080394+0000","last_change":"2023-10-06T02:26:43.080394+0000","last_active":"2023-10-06T02:26:43.080394+0000","last_peered":"2023-10-06T02:26:43.080394+0000","last_clean":"2023-10-06T02:26:43.080394+0000","last_became_active":"2023-10-06T02:26:43.079980+0000","last_became_peered":"2023-10-06T02:26:43.079980+0000","last_unstale":"2023-10-06T02:26:43.080394+0000","last_undegraded":"2023-10-06T02:26:43.080394+0000","last_fullsized":"2023-10-06T02:26:43.080394+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:57:20.411577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.083271+0000","last_change":"2023-10-06T02:26:43.083271+0000","last_active":"2023-10-06T02:26:43.083271+0000","last_peered":"2023-10-06T02:26:43.083271+0000","last_clean":"2023-10-06T02:26:43.083271+0000","last_became_active":"2023-10-06T02:26:43.082943+0000","last_became_peered":"2023-10-06T02:26:43.082943+0000","last_unstale":"2023-10-06T02:26:43.083271+0000","last_undegraded":"2023-10-06T02:26:43.083271+0000","last_fullsized":"2023-10-06T02:26:43.083271+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:04:15.890422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.083445+0000","last_change":"2023-10-06T02:26:43.083445+0000","last_active":"2023-10-06T02:26:43.083445+0000","last_peered":"2023-10-06T02:26:43.083445+0000","last_clean":"2023-10-06T02:26:43.083445+0000","last_became_active":"2023-10-06T02:26:43.083062+0000","last_became_peered":"2023-10-06T02:26:43.083062+0000","last_unstale":"2023-10-06T02:26:43.083445+0000","last_undegraded":"2023-10-06T02:26:43.083445+0000","last_fullsized":"2023-10-06T02:26:43.083445+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:49:13.194304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.086831+0000","last_change":"2023-10-06T02:26:43.086831+0000","last_active":"2023-10-06T02:26:43.086831+0000","last_peered":"2023-10-06T02:26:43.086831+0000","last_clean":"2023-10-06T02:26:43.086831+0000","last_became_active":"2023-10-06T02:26:43.086533+0000","last_became_peered":"2023-10-06T02:26:43.086533+0000","last_unstale":"2023-10-06T02:26:43.086831+0000","last_undegraded":"2023-10-06T02:26:43.086831+0000","last_fullsized":"2023-10-06T02:26:43.086831+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:03:03.126517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.079956+0000","last_change":"2023-10-06T02:26:43.079956+0000","last_active":"2023-10-06T02:26:43.079956+0000","last_peered":"2023-10-06T02:26:43.079956+0000","last_clean":"2023-10-06T02:26:43.079956+0000","last_became_active":"2023-10-06T02:26:43.076751+0000","last_became_peered":"2023-10-06T02:26:43.076751+0000","last_unstale":"2023-10-06T02:26:43.079956+0000","last_undegraded":"2023-10-06T02:26:43.079956+0000","last_fullsized":"2023-10-06T02:26:43.079956+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:13:19.533378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.078144+0000","last_change":"2023-10-06T02:26:43.078144+0000","last_active":"2023-10-06T02:26:43.078144+0000","last_peered":"2023-10-06T02:26:43.078144+0000","last_clean":"2023-10-06T02:26:43.078144+0000","last_became_active":"2023-10-06T02:26:43.077296+0000","last_became_peered":"2023-10-06T02:26:43.077296+0000","last_unstale":"2023-10-06T02:26:43.078144+0000","last_undegraded":"2023-10-06T02:26:43.078144+0000","last_fullsized":"2023-10-06T02:26:43.078144+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:31:07.380497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.074301+0000","last_change":"2023-10-06T02:26:43.074301+0000","last_active":"2023-10-06T02:26:43.074301+0000","last_peered":"2023-10-06T02:26:43.074301+0000","last_clean":"2023-10-06T02:26:43.074301+0000","last_became_active":"2023-10-06T02:26:43.073992+0000","last_became_peered":"2023-10-06T02:26:43.073992+0000","last_unstale":"2023-10-06T02:26:43.074301+0000","last_undegraded":"2023-10-06T02:26:43.074301+0000","last_fullsized":"2023-10-06T02:26:43.074301+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:28:44.491766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.086761+0000","last_change":"2023-10-06T02:26:43.086761+0000","last_active":"2023-10-06T02:26:43.086761+0000","last_peered":"2023-10-06T02:26:43.086761+0000","last_clean":"2023-10-06T02:26:43.086761+0000","last_became_active":"2023-10-06T02:26:43.086495+0000","last_became_peered":"2023-10-06T02:26:43.086495+0000","last_unstale":"2023-10-06T02:26:43.086761+0000","last_undegraded":"2023-10-06T02:26:43.086761+0000","last_fullsized":"2023-10-06T02:26:43.086761+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:19:55.281699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.091349+0000","last_change":"2023-10-06T02:26:43.091349+0000","last_active":"2023-10-06T02:26:43.091349+0000","last_peered":"2023-10-06T02:26:43.091349+0000","last_clean":"2023-10-06T02:26:43.091349+0000","last_became_active":"2023-10-06T02:26:43.091042+0000","last_became_peered":"2023-10-06T02:26:43.091042+0000","last_unstale":"2023-10-06T02:26:43.091349+0000","last_undegraded":"2023-10-06T02:26:43.091349+0000","last_fullsized":"2023-10-06T02:26:43.091349+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:13:13.506543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.083042+0000","last_change":"2023-10-06T02:26:43.083042+0000","last_active":"2023-10-06T02:26:43.083042+0000","last_peered":"2023-10-06T02:26:43.083042+0000","last_clean":"2023-10-06T02:26:43.083042+0000","last_became_active":"2023-10-06T02:26:43.082752+0000","last_became_peered":"2023-10-06T02:26:43.082752+0000","last_unstale":"2023-10-06T02:26:43.083042+0000","last_undegraded":"2023-10-06T02:26:43.083042+0000","last_fullsized":"2023-10-06T02:26:43.083042+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:08:03.092157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.073639+0000","last_change":"2023-10-06T02:26:43.073639+0000","last_active":"2023-10-06T02:26:43.073639+0000","last_peered":"2023-10-06T02:26:43.073639+0000","last_clean":"2023-10-06T02:26:43.073639+0000","last_became_active":"2023-10-06T02:26:43.073182+0000","last_became_peered":"2023-10-06T02:26:43.073182+0000","last_unstale":"2023-10-06T02:26:43.073639+0000","last_undegraded":"2023-10-06T02:26:43.073639+0000","last_fullsized":"2023-10-06T02:26:43.073639+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:47:21.495991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.074819+0000","last_change":"2023-10-06T02:26:43.074819+0000","last_active":"2023-10-06T02:26:43.074819+0000","last_peered":"2023-10-06T02:26:43.074819+0000","last_clean":"2023-10-06T02:26:43.074819+0000","last_became_active":"2023-10-06T02:26:43.074142+0000","last_became_peered":"2023-10-06T02:26:43.074142+0000","last_unstale":"2023-10-06T02:26:43.074819+0000","last_undegraded":"2023-10-06T02:26:43.074819+0000","last_fullsized":"2023-10-06T02:26:43.074819+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:40.671226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.086161+0000","last_change":"2023-10-06T02:26:43.086161+0000","last_active":"2023-10-06T02:26:43.086161+0000","last_peered":"2023-10-06T02:26:43.086161+0000","last_clean":"2023-10-06T02:26:43.086161+0000","last_became_active":"2023-10-06T02:26:43.085818+0000","last_became_peered":"2023-10-06T02:26:43.085818+0000","last_unstale":"2023-10-06T02:26:43.086161+0000","last_undegraded":"2023-10-06T02:26:43.086161+0000","last_fullsized":"2023-10-06T02:26:43.086161+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:49:19.811251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"60'4","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.099335+0000","last_change":"2023-10-06T02:26:42.079022+0000","last_active":"2023-10-06T02:26:43.099335+0000","last_peered":"2023-10-06T02:26:43.099335+0000","last_clean":"2023-10-06T02:26:43.099335+0000","last_became_active":"2023-10-06T02:26:42.078636+0000","last_became_peered":"2023-10-06T02:26:42.078636+0000","last_unstale":"2023-10-06T02:26:43.099335+0000","last_undegraded":"2023-10-06T02:26:43.099335+0000","last_fullsized":"2023-10-06T02:26:43.099335+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_clean_scrub_stamp":"2023-10-06T02:26:41.040467+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:37:50.109220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.088152+0000","last_change":"2023-10-06T02:26:43.088152+0000","last_active":"2023-10-06T02:26:43.088152+0000","last_peered":"2023-10-06T02:26:43.088152+0000","last_clean":"2023-10-06T02:26:43.088152+0000","last_became_active":"2023-10-06T02:26:43.087824+0000","last_became_peered":"2023-10-06T02:26:43.087824+0000","last_unstale":"2023-10-06T02:26:43.088152+0000","last_undegraded":"2023-10-06T02:26:43.088152+0000","last_fullsized":"2023-10-06T02:26:43.088152+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:49:19.811251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"60'1","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.063032+0000","last_change":"2023-10-06T02:26:42.078432+0000","last_active":"2023-10-06T02:26:43.063032+0000","last_peered":"2023-10-06T02:26:43.063032+0000","last_clean":"2023-10-06T02:26:43.063032+0000","last_became_active":"2023-10-06T02:26:42.078127+0000","last_became_peered":"2023-10-06T02:26:42.078127+0000","last_unstale":"2023-10-06T02:26:43.063032+0000","last_undegraded":"2023-10-06T02:26:43.063032+0000","last_fullsized":"2023-10-06T02:26:43.063032+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_clean_scrub_stamp":"2023-10-06T02:26:41.040467+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:08:28.473770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.073635+0000","last_change":"2023-10-06T02:26:43.073635+0000","last_active":"2023-10-06T02:26:43.073635+0000","last_peered":"2023-10-06T02:26:43.073635+0000","last_clean":"2023-10-06T02:26:43.073635+0000","last_became_active":"2023-10-06T02:26:43.073327+0000","last_became_peered":"2023-10-06T02:26:43.073327+0000","last_unstale":"2023-10-06T02:26:43.073635+0000","last_undegraded":"2023-10-06T02:26:43.073635+0000","last_fullsized":"2023-10-06T02:26:43.073635+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:33:18.711244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"60'2","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.088926+0000","last_change":"2023-10-06T02:26:42.071061+0000","last_active":"2023-10-06T02:26:43.088926+0000","last_peered":"2023-10-06T02:26:43.088926+0000","last_clean":"2023-10-06T02:26:43.088926+0000","last_became_active":"2023-10-06T02:26:42.070781+0000","last_became_peered":"2023-10-06T02:26:42.070781+0000","last_unstale":"2023-10-06T02:26:43.088926+0000","last_undegraded":"2023-10-06T02:26:43.088926+0000","last_fullsized":"2023-10-06T02:26:43.088926+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_clean_scrub_stamp":"2023-10-06T02:26:41.040467+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:53:53.191609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.094686+0000","last_change":"2023-10-06T02:26:43.094686+0000","last_active":"2023-10-06T02:26:43.094686+0000","last_peered":"2023-10-06T02:26:43.094686+0000","last_clean":"2023-10-06T02:26:43.094686+0000","last_became_active":"2023-10-06T02:26:43.094383+0000","last_became_peered":"2023-10-06T02:26:43.094383+0000","last_unstale":"2023-10-06T02:26:43.094686+0000","last_undegraded":"2023-10-06T02:26:43.094686+0000","last_fullsized":"2023-10-06T02:26:43.094686+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:27:50.683731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"60'2","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.090412+0000","last_change":"2023-10-06T02:26:42.079812+0000","last_active":"2023-10-06T02:26:43.090412+0000","last_peered":"2023-10-06T02:26:43.090412+0000","last_clean":"2023-10-06T02:26:43.090412+0000","last_became_active":"2023-10-06T02:26:42.079040+0000","last_became_peered":"2023-10-06T02:26:42.079040+0000","last_unstale":"2023-10-06T02:26:43.090412+0000","last_undegraded":"2023-10-06T02:26:43.090412+0000","last_fullsized":"2023-10-06T02:26:43.090412+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_clean_scrub_stamp":"2023-10-06T02:26:41.040467+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:44:17.259680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.085642+0000","last_change":"2023-10-06T02:26:43.085642+0000","last_active":"2023-10-06T02:26:43.085642+0000","last_peered":"2023-10-06T02:26:43.085642+0000","last_clean":"2023-10-06T02:26:43.085642+0000","last_became_active":"2023-10-06T02:26:43.085329+0000","last_became_peered":"2023-10-06T02:26:43.085329+0000","last_unstale":"2023-10-06T02:26:43.085642+0000","last_undegraded":"2023-10-06T02:26:43.085642+0000","last_fullsized":"2023-10-06T02:26:43.085642+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:29:12.676385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"60'1","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.059038+0000","last_change":"2023-10-06T02:26:42.072152+0000","last_active":"2023-10-06T02:26:43.059038+0000","last_peered":"2023-10-06T02:26:43.059038+0000","last_clean":"2023-10-06T02:26:43.059038+0000","last_became_active":"2023-10-06T02:26:42.071939+0000","last_became_peered":"2023-10-06T02:26:42.071939+0000","last_unstale":"2023-10-06T02:26:43.059038+0000","last_undegraded":"2023-10-06T02:26:43.059038+0000","last_fullsized":"2023-10-06T02:26:43.059038+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_clean_scrub_stamp":"2023-10-06T02:26:41.040467+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:53:53.191609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.081201+0000","last_change":"2023-10-06T02:26:43.081201+0000","last_active":"2023-10-06T02:26:43.081201+0000","last_peered":"2023-10-06T02:26:43.081201+0000","last_clean":"2023-10-06T02:26:43.081201+0000","last_became_active":"2023-10-06T02:26:43.080894+0000","last_became_peered":"2023-10-06T02:26:43.080894+0000","last_unstale":"2023-10-06T02:26:43.081201+0000","last_undegraded":"2023-10-06T02:26:43.081201+0000","last_fullsized":"2023-10-06T02:26:43.081201+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:31:56.002363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"60'1","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.058803+0000","last_change":"2023-10-06T02:26:42.069515+0000","last_active":"2023-10-06T02:26:43.058803+0000","last_peered":"2023-10-06T02:26:43.058803+0000","last_clean":"2023-10-06T02:26:43.058803+0000","last_became_active":"2023-10-06T02:26:42.069215+0000","last_became_peered":"2023-10-06T02:26:42.069215+0000","last_unstale":"2023-10-06T02:26:43.058803+0000","last_undegraded":"2023-10-06T02:26:43.058803+0000","last_fullsized":"2023-10-06T02:26:43.058803+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_clean_scrub_stamp":"2023-10-06T02:26:41.040467+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:08:28.473770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.083466+0000","last_change":"2023-10-06T02:26:43.083466+0000","last_active":"2023-10-06T02:26:43.083466+0000","last_peered":"2023-10-06T02:26:43.083466+0000","last_clean":"2023-10-06T02:26:43.083466+0000","last_became_active":"2023-10-06T02:26:43.083213+0000","last_became_peered":"2023-10-06T02:26:43.083213+0000","last_unstale":"2023-10-06T02:26:43.083466+0000","last_undegraded":"2023-10-06T02:26:43.083466+0000","last_fullsized":"2023-10-06T02:26:43.083466+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:50:38.588741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-06T02:26:42.080803+0000","last_change":"2023-10-06T02:26:42.075580+0000","last_active":"2023-10-06T02:26:42.080803+0000","last_peered":"2023-10-06T02:26:42.080803+0000","last_clean":"2023-10-06T02:26:42.080803+0000","last_became_active":"2023-10-06T02:26:42.075321+0000","last_became_peered":"2023-10-06T02:26:42.075321+0000","last_unstale":"2023-10-06T02:26:42.080803+0000","last_undegraded":"2023-10-06T02:26:42.080803+0000","last_fullsized":"2023-10-06T02:26:42.080803+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_clean_scrub_stamp":"2023-10-06T02:26:41.040467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:08:02.085597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.067766+0000","last_change":"2023-10-06T02:26:43.067766+0000","last_active":"2023-10-06T02:26:43.067766+0000","last_peered":"2023-10-06T02:26:43.067766+0000","last_clean":"2023-10-06T02:26:43.067766+0000","last_became_active":"2023-10-06T02:26:43.067496+0000","last_became_peered":"2023-10-06T02:26:43.067496+0000","last_unstale":"2023-10-06T02:26:43.067766+0000","last_undegraded":"2023-10-06T02:26:43.067766+0000","last_fullsized":"2023-10-06T02:26:43.067766+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:55:59.959651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-06T02:26:42.079239+0000","last_change":"2023-10-06T02:26:42.074166+0000","last_active":"2023-10-06T02:26:42.079239+0000","last_peered":"2023-10-06T02:26:42.079239+0000","last_clean":"2023-10-06T02:26:42.079239+0000","last_became_active":"2023-10-06T02:26:42.073908+0000","last_became_peered":"2023-10-06T02:26:42.073908+0000","last_unstale":"2023-10-06T02:26:42.079239+0000","last_undegraded":"2023-10-06T02:26:42.079239+0000","last_fullsized":"2023-10-06T02:26:42.079239+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_clean_scrub_stamp":"2023-10-06T02:26:41.040467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:11:16.082079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.082770+0000","last_change":"2023-10-06T02:26:43.082770+0000","last_active":"2023-10-06T02:26:43.082770+0000","last_peered":"2023-10-06T02:26:43.082770+0000","last_clean":"2023-10-06T02:26:43.082770+0000","last_became_active":"2023-10-06T02:26:43.082428+0000","last_became_peered":"2023-10-06T02:26:43.082428+0000","last_unstale":"2023-10-06T02:26:43.082770+0000","last_undegraded":"2023-10-06T02:26:43.082770+0000","last_fullsized":"2023-10-06T02:26:43.082770+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:06:31.428249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-06T02:26:42.090191+0000","last_change":"2023-10-06T02:26:42.083623+0000","last_active":"2023-10-06T02:26:42.090191+0000","last_peered":"2023-10-06T02:26:42.090191+0000","last_clean":"2023-10-06T02:26:42.090191+0000","last_became_active":"2023-10-06T02:26:42.083280+0000","last_became_peered":"2023-10-06T02:26:42.083280+0000","last_unstale":"2023-10-06T02:26:42.090191+0000","last_undegraded":"2023-10-06T02:26:42.090191+0000","last_fullsized":"2023-10-06T02:26:42.090191+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_clean_scrub_stamp":"2023-10-06T02:26:41.040467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:44:23.565902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.069687+0000","last_change":"2023-10-06T02:26:43.069687+0000","last_active":"2023-10-06T02:26:43.069687+0000","last_peered":"2023-10-06T02:26:43.069687+0000","last_clean":"2023-10-06T02:26:43.069687+0000","last_became_active":"2023-10-06T02:26:43.069403+0000","last_became_peered":"2023-10-06T02:26:43.069403+0000","last_unstale":"2023-10-06T02:26:43.069687+0000","last_undegraded":"2023-10-06T02:26:43.069687+0000","last_fullsized":"2023-10-06T02:26:43.069687+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:33:18.711244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-06T02:26:42.086500+0000","last_change":"2023-10-06T02:26:42.081692+0000","last_active":"2023-10-06T02:26:42.086500+0000","last_peered":"2023-10-06T02:26:42.086500+0000","last_clean":"2023-10-06T02:26:42.086500+0000","last_became_active":"2023-10-06T02:26:42.080705+0000","last_became_peered":"2023-10-06T02:26:42.080705+0000","last_unstale":"2023-10-06T02:26:42.086500+0000","last_undegraded":"2023-10-06T02:26:42.086500+0000","last_fullsized":"2023-10-06T02:26:42.086500+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_clean_scrub_stamp":"2023-10-06T02:26:41.040467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:12:32.146499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.088010+0000","last_change":"2023-10-06T02:26:43.088010+0000","last_active":"2023-10-06T02:26:43.088010+0000","last_peered":"2023-10-06T02:26:43.088010+0000","last_clean":"2023-10-06T02:26:43.088010+0000","last_became_active":"2023-10-06T02:26:43.087558+0000","last_became_peered":"2023-10-06T02:26:43.087558+0000","last_unstale":"2023-10-06T02:26:43.088010+0000","last_undegraded":"2023-10-06T02:26:43.088010+0000","last_fullsized":"2023-10-06T02:26:43.088010+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:37:00.416857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"60'3","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.101409+0000","last_change":"2023-10-06T02:26:42.083841+0000","last_active":"2023-10-06T02:26:43.101409+0000","last_peered":"2023-10-06T02:26:43.101409+0000","last_clean":"2023-10-06T02:26:43.101409+0000","last_became_active":"2023-10-06T02:26:42.083541+0000","last_became_peered":"2023-10-06T02:26:42.083541+0000","last_unstale":"2023-10-06T02:26:43.101409+0000","last_undegraded":"2023-10-06T02:26:43.101409+0000","last_fullsized":"2023-10-06T02:26:43.101409+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_clean_scrub_stamp":"2023-10-06T02:26:41.040467+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:19:54.275140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.073370+0000","last_change":"2023-10-06T02:26:43.073370+0000","last_active":"2023-10-06T02:26:43.073370+0000","last_peered":"2023-10-06T02:26:43.073370+0000","last_clean":"2023-10-06T02:26:43.073370+0000","last_became_active":"2023-10-06T02:26:43.072988+0000","last_became_peered":"2023-10-06T02:26:43.072988+0000","last_unstale":"2023-10-06T02:26:43.073370+0000","last_undegraded":"2023-10-06T02:26:43.073370+0000","last_fullsized":"2023-10-06T02:26:43.073370+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:50:38.588741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"60'6","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.098602+0000","last_change":"2023-10-06T02:26:42.082345+0000","last_active":"2023-10-06T02:26:43.098602+0000","last_peered":"2023-10-06T02:26:43.098602+0000","last_clean":"2023-10-06T02:26:43.098602+0000","last_became_active":"2023-10-06T02:26:42.082026+0000","last_became_peered":"2023-10-06T02:26:42.082026+0000","last_unstale":"2023-10-06T02:26:43.098602+0000","last_undegraded":"2023-10-06T02:26:43.098602+0000","last_fullsized":"2023-10-06T02:26:43.098602+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_clean_scrub_stamp":"2023-10-06T02:26:41.040467+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:16:48.074587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.084183+0000","last_change":"2023-10-06T02:26:43.084183+0000","last_active":"2023-10-06T02:26:43.084183+0000","last_peered":"2023-10-06T02:26:43.084183+0000","last_clean":"2023-10-06T02:26:43.084183+0000","last_became_active":"2023-10-06T02:26:43.083926+0000","last_became_peered":"2023-10-06T02:26:43.083926+0000","last_unstale":"2023-10-06T02:26:43.084183+0000","last_undegraded":"2023-10-06T02:26:43.084183+0000","last_fullsized":"2023-10-06T02:26:43.084183+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:55.811855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"60'2","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.101481+0000","last_change":"2023-10-06T02:26:42.079842+0000","last_active":"2023-10-06T02:26:43.101481+0000","last_peered":"2023-10-06T02:26:43.101481+0000","last_clean":"2023-10-06T02:26:43.101481+0000","last_became_active":"2023-10-06T02:26:42.079583+0000","last_became_peered":"2023-10-06T02:26:42.079583+0000","last_unstale":"2023-10-06T02:26:43.101481+0000","last_undegraded":"2023-10-06T02:26:43.101481+0000","last_fullsized":"2023-10-06T02:26:43.101481+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_clean_scrub_stamp":"2023-10-06T02:26:41.040467+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:48:34.539906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.075255+0000","last_change":"2023-10-06T02:26:43.075255+0000","last_active":"2023-10-06T02:26:43.075255+0000","last_peered":"2023-10-06T02:26:43.075255+0000","last_clean":"2023-10-06T02:26:43.075255+0000","last_became_active":"2023-10-06T02:26:43.074954+0000","last_became_peered":"2023-10-06T02:26:43.074954+0000","last_unstale":"2023-10-06T02:26:43.075255+0000","last_undegraded":"2023-10-06T02:26:43.075255+0000","last_fullsized":"2023-10-06T02:26:43.075255+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:47:21.495991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-06T02:26:42.086683+0000","last_change":"2023-10-06T02:26:42.080502+0000","last_active":"2023-10-06T02:26:42.086683+0000","last_peered":"2023-10-06T02:26:42.086683+0000","last_clean":"2023-10-06T02:26:42.086683+0000","last_became_active":"2023-10-06T02:26:42.080159+0000","last_became_peered":"2023-10-06T02:26:42.080159+0000","last_unstale":"2023-10-06T02:26:42.086683+0000","last_undegraded":"2023-10-06T02:26:42.086683+0000","last_fullsized":"2023-10-06T02:26:42.086683+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_clean_scrub_stamp":"2023-10-06T02:26:41.040467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:53:20.029463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.079057+0000","last_change":"2023-10-06T02:26:43.079057+0000","last_active":"2023-10-06T02:26:43.079057+0000","last_peered":"2023-10-06T02:26:43.079057+0000","last_clean":"2023-10-06T02:26:43.079057+0000","last_became_active":"2023-10-06T02:26:43.078779+0000","last_became_peered":"2023-10-06T02:26:43.078779+0000","last_unstale":"2023-10-06T02:26:43.079057+0000","last_undegraded":"2023-10-06T02:26:43.079057+0000","last_fullsized":"2023-10-06T02:26:43.079057+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:19:55.281699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-06T02:26:42.086808+0000","last_change":"2023-10-06T02:26:42.081994+0000","last_active":"2023-10-06T02:26:42.086808+0000","last_peered":"2023-10-06T02:26:42.086808+0000","last_clean":"2023-10-06T02:26:42.086808+0000","last_became_active":"2023-10-06T02:26:42.081473+0000","last_became_peered":"2023-10-06T02:26:42.081473+0000","last_unstale":"2023-10-06T02:26:42.086808+0000","last_undegraded":"2023-10-06T02:26:42.086808+0000","last_fullsized":"2023-10-06T02:26:42.086808+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_clean_scrub_stamp":"2023-10-06T02:26:41.040467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:49:18.804692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.086111+0000","last_change":"2023-10-06T02:26:43.086111+0000","last_active":"2023-10-06T02:26:43.086111+0000","last_peered":"2023-10-06T02:26:43.086111+0000","last_clean":"2023-10-06T02:26:43.086111+0000","last_became_active":"2023-10-06T02:26:43.085875+0000","last_became_peered":"2023-10-06T02:26:43.085875+0000","last_unstale":"2023-10-06T02:26:43.086111+0000","last_undegraded":"2023-10-06T02:26:43.086111+0000","last_fullsized":"2023-10-06T02:26:43.086111+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:37:00.416857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"60'8","reported_seq":17,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.097762+0000","last_change":"2023-10-06T02:26:42.071853+0000","last_active":"2023-10-06T02:26:43.097762+0000","last_peered":"2023-10-06T02:26:43.097762+0000","last_clean":"2023-10-06T02:26:43.097762+0000","last_became_active":"2023-10-06T02:26:42.071477+0000","last_became_peered":"2023-10-06T02:26:42.071477+0000","last_unstale":"2023-10-06T02:26:43.097762+0000","last_undegraded":"2023-10-06T02:26:43.097762+0000","last_fullsized":"2023-10-06T02:26:43.097762+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_clean_scrub_stamp":"2023-10-06T02:26:41.040467+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:21:14.686655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.073917+0000","last_change":"2023-10-06T02:26:43.073917+0000","last_active":"2023-10-06T02:26:43.073917+0000","last_peered":"2023-10-06T02:26:43.073917+0000","last_clean":"2023-10-06T02:26:43.073917+0000","last_became_active":"2023-10-06T02:26:43.073659+0000","last_became_peered":"2023-10-06T02:26:43.073659+0000","last_unstale":"2023-10-06T02:26:43.073917+0000","last_undegraded":"2023-10-06T02:26:43.073917+0000","last_fullsized":"2023-10-06T02:26:43.073917+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:44:55.051256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"60'2","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.094126+0000","last_change":"2023-10-06T02:26:42.083006+0000","last_active":"2023-10-06T02:26:43.094126+0000","last_peered":"2023-10-06T02:26:43.094126+0000","last_clean":"2023-10-06T02:26:43.094126+0000","last_became_active":"2023-10-06T02:26:42.082511+0000","last_became_peered":"2023-10-06T02:26:42.082511+0000","last_unstale":"2023-10-06T02:26:43.094126+0000","last_undegraded":"2023-10-06T02:26:43.094126+0000","last_fullsized":"2023-10-06T02:26:43.094126+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_clean_scrub_stamp":"2023-10-06T02:26:41.040467+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:58:22.585200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.076946+0000","last_change":"2023-10-06T02:26:43.076946+0000","last_active":"2023-10-06T02:26:43.076946+0000","last_peered":"2023-10-06T02:26:43.076946+0000","last_clean":"2023-10-06T02:26:43.076946+0000","last_became_active":"2023-10-06T02:26:43.076661+0000","last_became_peered":"2023-10-06T02:26:43.076661+0000","last_unstale":"2023-10-06T02:26:43.076946+0000","last_undegraded":"2023-10-06T02:26:43.076946+0000","last_fullsized":"2023-10-06T02:26:43.076946+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:42:18.780865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-06T02:26:42.080285+0000","last_change":"2023-10-06T02:26:42.073476+0000","last_active":"2023-10-06T02:26:42.080285+0000","last_peered":"2023-10-06T02:26:42.080285+0000","last_clean":"2023-10-06T02:26:42.080285+0000","last_became_active":"2023-10-06T02:26:42.073146+0000","last_became_peered":"2023-10-06T02:26:42.073146+0000","last_unstale":"2023-10-06T02:26:42.080285+0000","last_undegraded":"2023-10-06T02:26:42.080285+0000","last_fullsized":"2023-10-06T02:26:42.080285+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_clean_scrub_stamp":"2023-10-06T02:26:41.040467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:21:19.421613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.077808+0000","last_change":"2023-10-06T02:26:43.077808+0000","last_active":"2023-10-06T02:26:43.077808+0000","last_peered":"2023-10-06T02:26:43.077808+0000","last_clean":"2023-10-06T02:26:43.077808+0000","last_became_active":"2023-10-06T02:26:43.076710+0000","last_became_peered":"2023-10-06T02:26:43.076710+0000","last_unstale":"2023-10-06T02:26:43.077808+0000","last_undegraded":"2023-10-06T02:26:43.077808+0000","last_fullsized":"2023-10-06T02:26:43.077808+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:52:16.520095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-06T02:26:42.073173+0000","last_change":"2023-10-06T02:26:42.066985+0000","last_active":"2023-10-06T02:26:42.073173+0000","last_peered":"2023-10-06T02:26:42.073173+0000","last_clean":"2023-10-06T02:26:42.073173+0000","last_became_active":"2023-10-06T02:26:42.066706+0000","last_became_peered":"2023-10-06T02:26:42.066706+0000","last_unstale":"2023-10-06T02:26:42.073173+0000","last_undegraded":"2023-10-06T02:26:42.073173+0000","last_fullsized":"2023-10-06T02:26:42.073173+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_clean_scrub_stamp":"2023-10-06T02:26:41.040467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:27:49.677171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.078383+0000","last_change":"2023-10-06T02:26:43.078383+0000","last_active":"2023-10-06T02:26:43.078383+0000","last_peered":"2023-10-06T02:26:43.078383+0000","last_clean":"2023-10-06T02:26:43.078383+0000","last_became_active":"2023-10-06T02:26:43.077873+0000","last_became_peered":"2023-10-06T02:26:43.077873+0000","last_unstale":"2023-10-06T02:26:43.078383+0000","last_undegraded":"2023-10-06T02:26:43.078383+0000","last_fullsized":"2023-10-06T02:26:43.078383+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:44:24.572462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-06T02:26:42.083090+0000","last_change":"2023-10-06T02:26:42.078465+0000","last_active":"2023-10-06T02:26:42.083090+0000","last_peered":"2023-10-06T02:26:42.083090+0000","last_clean":"2023-10-06T02:26:42.083090+0000","last_became_active":"2023-10-06T02:26:42.078124+0000","last_became_peered":"2023-10-06T02:26:42.078124+0000","last_unstale":"2023-10-06T02:26:42.083090+0000","last_undegraded":"2023-10-06T02:26:42.083090+0000","last_fullsized":"2023-10-06T02:26:42.083090+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_clean_scrub_stamp":"2023-10-06T02:26:41.040467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:39:51.926230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.090536+0000","last_change":"2023-10-06T02:26:43.090536+0000","last_active":"2023-10-06T02:26:43.090536+0000","last_peered":"2023-10-06T02:26:43.090536+0000","last_clean":"2023-10-06T02:26:43.090536+0000","last_became_active":"2023-10-06T02:26:43.090242+0000","last_became_peered":"2023-10-06T02:26:43.090242+0000","last_unstale":"2023-10-06T02:26:43.090536+0000","last_undegraded":"2023-10-06T02:26:43.090536+0000","last_fullsized":"2023-10-06T02:26:43.090536+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:23:51.057531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-06T02:26:42.090288+0000","last_change":"2023-10-06T02:26:42.084346+0000","last_active":"2023-10-06T02:26:42.090288+0000","last_peered":"2023-10-06T02:26:42.090288+0000","last_clean":"2023-10-06T02:26:42.090288+0000","last_became_active":"2023-10-06T02:26:42.084050+0000","last_became_peered":"2023-10-06T02:26:42.084050+0000","last_unstale":"2023-10-06T02:26:42.090288+0000","last_undegraded":"2023-10-06T02:26:42.090288+0000","last_fullsized":"2023-10-06T02:26:42.090288+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_clean_scrub_stamp":"2023-10-06T02:26:41.040467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:47:20.489431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.084882+0000","last_change":"2023-10-06T02:26:43.084882+0000","last_active":"2023-10-06T02:26:43.084882+0000","last_peered":"2023-10-06T02:26:43.084882+0000","last_clean":"2023-10-06T02:26:43.084882+0000","last_became_active":"2023-10-06T02:26:43.084622+0000","last_became_peered":"2023-10-06T02:26:43.084622+0000","last_unstale":"2023-10-06T02:26:43.084882+0000","last_undegraded":"2023-10-06T02:26:43.084882+0000","last_fullsized":"2023-10-06T02:26:43.084882+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:13:19.533378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-06T02:26:42.082710+0000","last_change":"2023-10-06T02:26:42.078455+0000","last_active":"2023-10-06T02:26:42.082710+0000","last_peered":"2023-10-06T02:26:42.082710+0000","last_clean":"2023-10-06T02:26:42.082710+0000","last_became_active":"2023-10-06T02:26:42.078140+0000","last_became_peered":"2023-10-06T02:26:42.078140+0000","last_unstale":"2023-10-06T02:26:42.082710+0000","last_undegraded":"2023-10-06T02:26:42.082710+0000","last_fullsized":"2023-10-06T02:26:42.082710+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_clean_scrub_stamp":"2023-10-06T02:26:41.040467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:17:55.051477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.088400+0000","last_change":"2023-10-06T02:26:43.088400+0000","last_active":"2023-10-06T02:26:43.088400+0000","last_peered":"2023-10-06T02:26:43.088400+0000","last_clean":"2023-10-06T02:26:43.088400+0000","last_became_active":"2023-10-06T02:26:43.087755+0000","last_became_peered":"2023-10-06T02:26:43.087755+0000","last_unstale":"2023-10-06T02:26:43.088400+0000","last_undegraded":"2023-10-06T02:26:43.088400+0000","last_fullsized":"2023-10-06T02:26:43.088400+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:50:38.588741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-06T02:26:42.080829+0000","last_change":"2023-10-06T02:26:42.075720+0000","last_active":"2023-10-06T02:26:42.080829+0000","last_peered":"2023-10-06T02:26:42.080829+0000","last_clean":"2023-10-06T02:26:42.080829+0000","last_became_active":"2023-10-06T02:26:42.075424+0000","last_became_peered":"2023-10-06T02:26:42.075424+0000","last_unstale":"2023-10-06T02:26:42.080829+0000","last_undegraded":"2023-10-06T02:26:42.080829+0000","last_fullsized":"2023-10-06T02:26:42.080829+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_clean_scrub_stamp":"2023-10-06T02:26:41.040467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:57:19.405017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.080847+0000","last_change":"2023-10-06T02:26:43.080847+0000","last_active":"2023-10-06T02:26:43.080847+0000","last_peered":"2023-10-06T02:26:43.080847+0000","last_clean":"2023-10-06T02:26:43.080847+0000","last_became_active":"2023-10-06T02:26:43.080597+0000","last_became_peered":"2023-10-06T02:26:43.080597+0000","last_unstale":"2023-10-06T02:26:43.080847+0000","last_undegraded":"2023-10-06T02:26:43.080847+0000","last_fullsized":"2023-10-06T02:26:43.080847+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:54:21.378728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-06T02:26:42.077995+0000","last_change":"2023-10-06T02:26:42.073779+0000","last_active":"2023-10-06T02:26:42.077995+0000","last_peered":"2023-10-06T02:26:42.077995+0000","last_clean":"2023-10-06T02:26:42.077995+0000","last_became_active":"2023-10-06T02:26:42.073436+0000","last_became_peered":"2023-10-06T02:26:42.073436+0000","last_unstale":"2023-10-06T02:26:42.077995+0000","last_undegraded":"2023-10-06T02:26:42.077995+0000","last_fullsized":"2023-10-06T02:26:42.077995+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_clean_scrub_stamp":"2023-10-06T02:26:41.040467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:48:58.794790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.092914+0000","last_change":"2023-10-06T02:26:43.092914+0000","last_active":"2023-10-06T02:26:43.092914+0000","last_peered":"2023-10-06T02:26:43.092914+0000","last_clean":"2023-10-06T02:26:43.092914+0000","last_became_active":"2023-10-06T02:26:43.091596+0000","last_became_peered":"2023-10-06T02:26:43.091596+0000","last_unstale":"2023-10-06T02:26:43.092914+0000","last_undegraded":"2023-10-06T02:26:43.092914+0000","last_fullsized":"2023-10-06T02:26:43.092914+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:41.589820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"60'1","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.064019+0000","last_change":"2023-10-06T02:26:42.080936+0000","last_active":"2023-10-06T02:26:43.064019+0000","last_peered":"2023-10-06T02:26:43.064019+0000","last_clean":"2023-10-06T02:26:43.064019+0000","last_became_active":"2023-10-06T02:26:42.080651+0000","last_became_peered":"2023-10-06T02:26:42.080651+0000","last_unstale":"2023-10-06T02:26:43.064019+0000","last_undegraded":"2023-10-06T02:26:43.064019+0000","last_fullsized":"2023-10-06T02:26:43.064019+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_clean_scrub_stamp":"2023-10-06T02:26:41.040467+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:23:50.050971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":44,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-06T02:26:42.072436+0000","last_change":"2023-10-06T02:23:02.054039+0000","last_active":"2023-10-06T02:26:42.072436+0000","last_peered":"2023-10-06T02:26:42.072436+0000","last_clean":"2023-10-06T02:26:42.072436+0000","last_became_active":"2023-10-06T02:23:02.053742+0000","last_became_peered":"2023-10-06T02:23:02.053742+0000","last_unstale":"2023-10-06T02:26:42.072436+0000","last_undegraded":"2023-10-06T02:26:42.072436+0000","last_fullsized":"2023-10-06T02:26:42.072436+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:58:18.846475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.081544+0000","last_change":"2023-10-06T02:26:43.081544+0000","last_active":"2023-10-06T02:26:43.081544+0000","last_peered":"2023-10-06T02:26:43.081544+0000","last_clean":"2023-10-06T02:26:43.081544+0000","last_became_active":"2023-10-06T02:26:43.081242+0000","last_became_peered":"2023-10-06T02:26:43.081242+0000","last_unstale":"2023-10-06T02:26:43.081544+0000","last_undegraded":"2023-10-06T02:26:43.081544+0000","last_fullsized":"2023-10-06T02:26:43.081544+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:31:56.002363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-06T02:26:42.082780+0000","last_change":"2023-10-06T02:26:42.078866+0000","last_active":"2023-10-06T02:26:42.082780+0000","last_peered":"2023-10-06T02:26:42.082780+0000","last_clean":"2023-10-06T02:26:42.082780+0000","last_became_active":"2023-10-06T02:26:42.078571+0000","last_became_peered":"2023-10-06T02:26:42.078571+0000","last_unstale":"2023-10-06T02:26:42.082780+0000","last_undegraded":"2023-10-06T02:26:42.082780+0000","last_fullsized":"2023-10-06T02:26:42.082780+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_clean_scrub_stamp":"2023-10-06T02:26:41.040467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:08:02.085597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":50,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-06T02:26:42.065623+0000","last_change":"2023-10-06T02:23:01.988631+0000","last_active":"2023-10-06T02:26:42.065623+0000","last_peered":"2023-10-06T02:26:42.065623+0000","last_clean":"2023-10-06T02:26:42.065623+0000","last_became_active":"2023-10-06T02:23:01.988303+0000","last_became_peered":"2023-10-06T02:23:01.988303+0000","last_unstale":"2023-10-06T02:26:42.065623+0000","last_undegraded":"2023-10-06T02:26:42.065623+0000","last_fullsized":"2023-10-06T02:26:42.065623+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:38.211466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.076113+0000","last_change":"2023-10-06T02:26:43.076113+0000","last_active":"2023-10-06T02:26:43.076113+0000","last_peered":"2023-10-06T02:26:43.076113+0000","last_clean":"2023-10-06T02:26:43.076113+0000","last_became_active":"2023-10-06T02:26:43.075805+0000","last_became_peered":"2023-10-06T02:26:43.075805+0000","last_unstale":"2023-10-06T02:26:43.076113+0000","last_undegraded":"2023-10-06T02:26:43.076113+0000","last_fullsized":"2023-10-06T02:26:43.076113+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:55.811855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-06T02:26:42.083299+0000","last_change":"2023-10-06T02:26:42.078753+0000","last_active":"2023-10-06T02:26:42.083299+0000","last_peered":"2023-10-06T02:26:42.083299+0000","last_clean":"2023-10-06T02:26:42.083299+0000","last_became_active":"2023-10-06T02:26:42.078443+0000","last_became_peered":"2023-10-06T02:26:42.078443+0000","last_unstale":"2023-10-06T02:26:42.083299+0000","last_undegraded":"2023-10-06T02:26:42.083299+0000","last_fullsized":"2023-10-06T02:26:42.083299+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_clean_scrub_stamp":"2023-10-06T02:26:41.040467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:51.420493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":46,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-06T02:26:42.065420+0000","last_change":"2023-10-06T02:23:02.054338+0000","last_active":"2023-10-06T02:26:42.065420+0000","last_peered":"2023-10-06T02:26:42.065420+0000","last_clean":"2023-10-06T02:26:42.065420+0000","last_became_active":"2023-10-06T02:23:02.053991+0000","last_became_peered":"2023-10-06T02:23:02.053991+0000","last_unstale":"2023-10-06T02:26:42.065420+0000","last_undegraded":"2023-10-06T02:26:42.065420+0000","last_fullsized":"2023-10-06T02:26:42.065420+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:39:30.192838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.090804+0000","last_change":"2023-10-06T02:26:43.090804+0000","last_active":"2023-10-06T02:26:43.090804+0000","last_peered":"2023-10-06T02:26:43.090804+0000","last_clean":"2023-10-06T02:26:43.090804+0000","last_became_active":"2023-10-06T02:26:43.090520+0000","last_became_peered":"2023-10-06T02:26:43.090520+0000","last_unstale":"2023-10-06T02:26:43.090804+0000","last_undegraded":"2023-10-06T02:26:43.090804+0000","last_fullsized":"2023-10-06T02:26:43.090804+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:58:23.591759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-06T02:26:42.080659+0000","last_change":"2023-10-06T02:26:42.071564+0000","last_active":"2023-10-06T02:26:42.080659+0000","last_peered":"2023-10-06T02:26:42.080659+0000","last_clean":"2023-10-06T02:26:42.080659+0000","last_became_active":"2023-10-06T02:26:42.071295+0000","last_became_peered":"2023-10-06T02:26:42.071295+0000","last_unstale":"2023-10-06T02:26:42.080659+0000","last_undegraded":"2023-10-06T02:26:42.080659+0000","last_fullsized":"2023-10-06T02:26:42.080659+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_clean_scrub_stamp":"2023-10-06T02:26:41.040467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:27:49.677171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":50,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-06T02:26:42.065796+0000","last_change":"2023-10-06T02:23:01.988140+0000","last_active":"2023-10-06T02:26:42.065796+0000","last_peered":"2023-10-06T02:26:42.065796+0000","last_clean":"2023-10-06T02:26:42.065796+0000","last_became_active":"2023-10-06T02:23:01.987769+0000","last_became_peered":"2023-10-06T02:23:01.987769+0000","last_unstale":"2023-10-06T02:26:42.065796+0000","last_undegraded":"2023-10-06T02:26:42.065796+0000","last_fullsized":"2023-10-06T02:26:42.065796+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:20:08.391912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":196,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-06T02:26:42.067688+0000","last_change":"2023-10-06T02:22:55.032519+0000","last_active":"2023-10-06T02:26:42.067688+0000","last_peered":"2023-10-06T02:26:42.067688+0000","last_clean":"2023-10-06T02:26:42.067688+0000","last_became_active":"2023-10-06T02:22:55.031367+0000","last_became_peered":"2023-10-06T02:22:55.031367+0000","last_unstale":"2023-10-06T02:26:42.067688+0000","last_undegraded":"2023-10-06T02:26:42.067688+0000","last_fullsized":"2023-10-06T02:26:42.067688+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:22:53.309097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:22:53.309097+0000","last_clean_scrub_stamp":"2023-10-06T02:22:53.309097+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:53:46.520909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.075021+0000","last_change":"2023-10-06T02:26:43.075021+0000","last_active":"2023-10-06T02:26:43.075021+0000","last_peered":"2023-10-06T02:26:43.075021+0000","last_clean":"2023-10-06T02:26:43.075021+0000","last_became_active":"2023-10-06T02:26:43.074541+0000","last_became_peered":"2023-10-06T02:26:43.074541+0000","last_unstale":"2023-10-06T02:26:43.075021+0000","last_undegraded":"2023-10-06T02:26:43.075021+0000","last_fullsized":"2023-10-06T02:26:43.075021+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:55:59.959651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"60'2","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.089600+0000","last_change":"2023-10-06T02:26:42.068544+0000","last_active":"2023-10-06T02:26:43.089600+0000","last_peered":"2023-10-06T02:26:43.089600+0000","last_clean":"2023-10-06T02:26:43.089600+0000","last_became_active":"2023-10-06T02:26:42.068138+0000","last_became_peered":"2023-10-06T02:26:42.068138+0000","last_unstale":"2023-10-06T02:26:43.089600+0000","last_undegraded":"2023-10-06T02:26:43.089600+0000","last_fullsized":"2023-10-06T02:26:43.089600+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_clean_scrub_stamp":"2023-10-06T02:26:41.040467+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:57:19.405017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":43,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-06T02:26:42.067783+0000","last_change":"2023-10-06T02:23:01.990773+0000","last_active":"2023-10-06T02:26:42.067783+0000","last_peered":"2023-10-06T02:26:42.067783+0000","last_clean":"2023-10-06T02:26:42.067783+0000","last_became_active":"2023-10-06T02:23:01.990511+0000","last_became_peered":"2023-10-06T02:23:01.990511+0000","last_unstale":"2023-10-06T02:26:42.067783+0000","last_undegraded":"2023-10-06T02:26:42.067783+0000","last_fullsized":"2023-10-06T02:26:42.067783+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:42:54.797731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.088244+0000","last_change":"2023-10-06T02:26:43.088244+0000","last_active":"2023-10-06T02:26:43.088244+0000","last_peered":"2023-10-06T02:26:43.088244+0000","last_clean":"2023-10-06T02:26:43.088244+0000","last_became_active":"2023-10-06T02:26:43.087810+0000","last_became_peered":"2023-10-06T02:26:43.087810+0000","last_unstale":"2023-10-06T02:26:43.088244+0000","last_undegraded":"2023-10-06T02:26:43.088244+0000","last_fullsized":"2023-10-06T02:26:43.088244+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:06:31.428249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"60'2","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.101281+0000","last_change":"2023-10-06T02:26:42.084296+0000","last_active":"2023-10-06T02:26:43.101281+0000","last_peered":"2023-10-06T02:26:43.101281+0000","last_clean":"2023-10-06T02:26:43.101281+0000","last_became_active":"2023-10-06T02:26:42.083994+0000","last_became_peered":"2023-10-06T02:26:42.083994+0000","last_unstale":"2023-10-06T02:26:43.101281+0000","last_undegraded":"2023-10-06T02:26:43.101281+0000","last_fullsized":"2023-10-06T02:26:43.101281+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_clean_scrub_stamp":"2023-10-06T02:26:41.040467+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:31:54.995803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":43,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-06T02:26:42.075880+0000","last_change":"2023-10-06T02:23:02.054069+0000","last_active":"2023-10-06T02:26:42.075880+0000","last_peered":"2023-10-06T02:26:42.075880+0000","last_clean":"2023-10-06T02:26:42.075880+0000","last_became_active":"2023-10-06T02:23:02.053739+0000","last_became_peered":"2023-10-06T02:23:02.053739+0000","last_unstale":"2023-10-06T02:26:42.075880+0000","last_undegraded":"2023-10-06T02:26:42.075880+0000","last_fullsized":"2023-10-06T02:26:42.075880+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:57:48.822608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.088111+0000","last_change":"2023-10-06T02:26:43.088111+0000","last_active":"2023-10-06T02:26:43.088111+0000","last_peered":"2023-10-06T02:26:43.088111+0000","last_clean":"2023-10-06T02:26:43.088111+0000","last_became_active":"2023-10-06T02:26:43.087811+0000","last_became_peered":"2023-10-06T02:26:43.087811+0000","last_unstale":"2023-10-06T02:26:43.088111+0000","last_undegraded":"2023-10-06T02:26:43.088111+0000","last_fullsized":"2023-10-06T02:26:43.088111+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:48:35.546466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"60'1","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.070651+0000","last_change":"2023-10-06T02:26:42.082060+0000","last_active":"2023-10-06T02:26:43.070651+0000","last_peered":"2023-10-06T02:26:43.070651+0000","last_clean":"2023-10-06T02:26:43.070651+0000","last_became_active":"2023-10-06T02:26:42.081634+0000","last_became_peered":"2023-10-06T02:26:42.081634+0000","last_unstale":"2023-10-06T02:26:43.070651+0000","last_undegraded":"2023-10-06T02:26:43.070651+0000","last_fullsized":"2023-10-06T02:26:43.070651+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_clean_scrub_stamp":"2023-10-06T02:26:41.040467+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:50:37.582181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":44,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-06T02:26:42.072413+0000","last_change":"2023-10-06T02:23:01.989934+0000","last_active":"2023-10-06T02:26:42.072413+0000","last_peered":"2023-10-06T02:26:42.072413+0000","last_clean":"2023-10-06T02:26:42.072413+0000","last_became_active":"2023-10-06T02:23:01.989706+0000","last_became_peered":"2023-10-06T02:23:01.989706+0000","last_unstale":"2023-10-06T02:26:42.072413+0000","last_undegraded":"2023-10-06T02:26:42.072413+0000","last_fullsized":"2023-10-06T02:26:42.072413+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:06.212251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T02:26:43.078135+0000","last_change":"2023-10-06T02:26:43.078135+0000","last_active":"2023-10-06T02:26:43.078135+0000","last_peered":"2023-10-06T02:26:43.078135+0000","last_clean":"2023-10-06T02:26:43.078135+0000","last_became_active":"2023-10-06T02:26:43.077573+0000","last_became_peered":"2023-10-06T02:26:43.077573+0000","last_unstale":"2023-10-06T02:26:43.078135+0000","last_undegraded":"2023-10-06T02:26:43.078135+0000","last_fullsized":"2023-10-06T02:26:43.078135+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:42.047027+0000","last_clean_scrub_stamp":"2023-10-06T02:26:42.047027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:57:39.226225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-06T02:26:42.080726+0000","last_change":"2023-10-06T02:26:42.074940+0000","last_active":"2023-10-06T02:26:42.080726+0000","last_peered":"2023-10-06T02:26:42.080726+0000","last_clean":"2023-10-06T02:26:42.080726+0000","last_became_active":"2023-10-06T02:26:42.074672+0000","last_became_peered":"2023-10-06T02:26:42.074672+0000","last_unstale":"2023-10-06T02:26:42.080726+0000","last_undegraded":"2023-10-06T02:26:42.080726+0000","last_fullsized":"2023-10-06T02:26:42.080726+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:26:41.040467+0000","last_clean_scrub_stamp":"2023-10-06T02:26:41.040467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:44:17.259680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":46,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-06T02:26:42.068912+0000","last_change":"2023-10-06T02:23:01.992935+0000","last_active":"2023-10-06T02:26:42.068912+0000","last_peered":"2023-10-06T02:26:42.068912+0000","last_clean":"2023-10-06T02:26:42.068912+0000","last_became_active":"2023-10-06T02:23:01.992568+0000","last_became_peered":"2023-10-06T02:23:01.992568+0000","last_unstale":"2023-10-06T02:26:42.068912+0000","last_undegraded":"2023-10-06T02:26:42.068912+0000","last_fullsized":"2023-10-06T02:26:42.068912+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:06.212251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574898,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302656,"kb_used_data":1272,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069184,"statfs":{"total":96636764160,"available":96326844416,"internally_reserved":0,"allocated":1302528,"data_stored":898103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.64200000000000002,"15min":0.63800000000000001},"min":{"1min":0.53100000000000003,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.878,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.61599999999999999,"15min":0.60399999999999998},"min":{"1min":0.45400000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.95399999999999996,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.52500000000000002}]},{"osd":1,"last update":"Fri Oct 6 02:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.70399999999999996,"15min":0.68300000000000005},"min":{"1min":0.59499999999999997,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.95199999999999996,"5min":1.147,"15min":1.147},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.68200000000000005,"15min":0.67500000000000004},"min":{"1min":0.504,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.91200000000000003,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.63700000000000001}]},{"osd":2,"last update":"Fri Oct 6 02:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.69799999999999995,"15min":0.69599999999999995},"min":{"1min":0.51100000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.88200000000000001,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.69599999999999995,"15min":0.69299999999999995},"min":{"1min":0.54200000000000004,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.98399999999999999,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.58699999999999997}]},{"osd":3,"last update":"Fri Oct 6 02:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.73899999999999999,"15min":0.72999999999999998},"min":{"1min":0.60599999999999998,"5min":0.497,"15min":0.497},"max":{"1min":0.98999999999999999,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.78100000000000003,"15min":0.77900000000000003},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.98099999999999998,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.745}]},{"osd":4,"last update":"Fri Oct 6 02:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.68000000000000005,"15min":0.70599999999999996},"min":{"1min":0.42999999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.81200000000000006,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.503},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.752,"15min":0.77800000000000002},"min":{"1min":0.41699999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.89600000000000002,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.60899999999999999}]},{"osd":5,"last update":"Fri Oct 6 02:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.72899999999999998,"15min":0.73999999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.73299999999999998,"15min":0.74099999999999999},"min":{"1min":0.52900000000000003,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.91900000000000004,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.69399999999999995}]},{"osd":6,"last update":"Fri Oct 6 02:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.79000000000000004,"15min":0.79200000000000004},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":0.96799999999999997,"5min":1.056,"15min":1.056},"last":0.753},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.81699999999999995,"15min":0.81899999999999995},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.0109999999999999,"5min":1.276,"15min":1.276},"last":0.67700000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574898,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297996,"kb_used_data":708,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073844,"statfs":{"total":96636764160,"available":96331616256,"internally_reserved":0,"allocated":724992,"data_stored":314885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:25:58 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.66600000000000004,"15min":0.66200000000000003},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.747},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.70399999999999996,"15min":0.69399999999999995},"min":{"1min":0.51800000000000002,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.92900000000000005,"5min":1.119,"15min":1.119},"last":0.81100000000000005}]},{"osd":1,"last update":"Fri Oct 6 02:25:58 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.63700000000000001,"15min":0.64300000000000002},"min":{"1min":0.40999999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.91200000000000003,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.67200000000000004,"15min":0.66600000000000004},"min":{"1min":0.41899999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.91000000000000003,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.621}]},{"osd":2,"last update":"Fri Oct 6 02:25:58 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.70599999999999996,"15min":0.69999999999999996},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.98499999999999999,"5min":1.137,"15min":1.137},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.72199999999999998,"15min":0.70899999999999996},"min":{"1min":0.47699999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.91700000000000004,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.77000000000000002}]},{"osd":3,"last update":"Fri Oct 6 02:25:58 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.66500000000000004,"15min":0.65400000000000003},"min":{"1min":0.53600000000000003,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.96399999999999997,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.70999999999999996,"15min":0.70199999999999996},"min":{"1min":0.49299999999999999,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.94899999999999995,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.70899999999999996}]},{"osd":4,"last update":"Fri Oct 6 02:25:58 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.64000000000000001,"15min":0.63400000000000001},"min":{"1min":0.39900000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.871,"5min":1.095,"15min":1.095},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.66700000000000004,"15min":0.65300000000000002},"min":{"1min":0.46100000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.89000000000000001,"5min":1.29,"15min":1.29},"last":0.58899999999999997}]},{"osd":5,"last update":"Fri Oct 6 02:25:58 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.69499999999999995,"15min":0.70199999999999996},"min":{"1min":0.50600000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.83999999999999997,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.70899999999999996,"15min":0.70299999999999996},"min":{"1min":0.46899999999999997,"5min":0.436,"15min":0.436},"max":{"1min":0.89700000000000002,"5min":1.02,"15min":1.02},"last":0.79200000000000004}]},{"osd":7,"last update":"Fri Oct 6 02:25:58 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.75,"15min":0.749},"min":{"1min":0.52600000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.97799999999999998,"5min":1.458,"15min":1.458},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.53500000000000003,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.88500000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.86899999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574897,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297980,"kb_used_data":692,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073860,"statfs":{"total":96636764160,"available":96331632640,"internally_reserved":0,"allocated":708608,"data_stored":307735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:25:55 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.46300000000000002,"15min":0.46700000000000003},"min":{"1min":0.316,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.65500000000000003,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.496},{"interface":"front","average":{"1min":0.503,"5min":0.51600000000000001,"15min":0.53600000000000003},"min":{"1min":0.373,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.71099999999999997,"5min":1.788,"15min":1.788},"last":0.32600000000000001}]},{"osd":2,"last update":"Fri Oct 6 02:25:55 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.51100000000000001,"15min":0.53800000000000003},"min":{"1min":0.34200000000000003,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.60899999999999999,"5min":2.0840000000000001,"15min":2.0840000000000001},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.55200000000000005,"15min":0.56799999999999995},"min":{"1min":0.39500000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.68000000000000005,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.438}]},{"osd":3,"last update":"Fri Oct 6 02:25:55 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.61699999999999999,"15min":0.62},"min":{"1min":0.47599999999999998,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.80100000000000005,"5min":1.266,"15min":1.266},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.61899999999999999,"15min":0.65700000000000003},"min":{"1min":0.46200000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.66800000000000004,"5min":1.861,"15min":1.861},"last":0.372}]},{"osd":4,"last update":"Fri Oct 6 02:25:55 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.85699999999999998,"15min":0.873},"min":{"1min":0.68799999999999994,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.1160000000000001,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.876,"5min":0.85899999999999999,"15min":0.873},"min":{"1min":0.56999999999999995,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0820000000000001,"5min":2.0179999999999998,"15min":2.0179999999999998},"last":0.52900000000000003}]},{"osd":5,"last update":"Fri Oct 6 02:25:55 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.85099999999999998,"15min":0.88300000000000001},"min":{"1min":0.53900000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.91900000000000004,"5min":1.9830000000000001,"15min":1.9830000000000001},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.82199999999999995,"15min":0.83799999999999997},"min":{"1min":0.68500000000000005,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.026,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.64500000000000002}]},{"osd":6,"last update":"Fri Oct 6 02:25:55 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.86499999999999999,"15min":0.88300000000000001},"min":{"1min":0.65800000000000003,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0369999999999999,"5min":1.829,"15min":1.829},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.90500000000000003,"15min":0.93300000000000005},"min":{"1min":0.629,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.103,"5min":2.052,"15min":2.052},"last":0.56399999999999995}]},{"osd":7,"last update":"Fri Oct 6 02:25:55 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.83899999999999997,"15min":0.84999999999999998},"min":{"1min":0.60499999999999998,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.0680000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.87,"15min":0.89700000000000002},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.048,"5min":1.895,"15min":1.895},"last":0.47799999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574898,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302656,"kb_used_data":1272,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069184,"statfs":{"total":96636764160,"available":96326844416,"internally_reserved":0,"allocated":1302528,"data_stored":898103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 02:26:02 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.44700000000000001,"15min":0.45900000000000002},"min":{"1min":0.32800000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.57699999999999996,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.505,"5min":0.48299999999999998,"15min":0.48499999999999999},"min":{"1min":0.37,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.38500000000000001}]},{"osd":2,"last update":"Fri Oct 6 02:26:02 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.504,"15min":0.51600000000000001},"min":{"1min":0.39600000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.63500000000000001,"5min":1.069,"15min":1.069},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.501,"5min":0.51000000000000001,"15min":0.52200000000000002},"min":{"1min":0.379,"5min":0.372,"15min":0.372},"max":{"1min":0.74299999999999999,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.44800000000000001}]},{"osd":3,"last update":"Fri Oct 6 02:26:02 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.50900000000000001},"min":{"1min":0.39400000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.60599999999999998,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.59399999999999997,"15min":0.59699999999999998},"min":{"1min":0.51500000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.70799999999999996,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.50900000000000001}]},{"osd":4,"last update":"Fri Oct 6 02:26:02 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.79300000000000004,"15min":0.81100000000000005},"min":{"1min":0.56299999999999994,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.056,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79500000000000004,"15min":0.80900000000000005},"min":{"1min":0.57099999999999995,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.056,"5min":1.304,"15min":1.304},"last":0.71999999999999997}]},{"osd":5,"last update":"Fri Oct 6 02:26:02 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.78500000000000003,"15min":0.79400000000000004},"min":{"1min":0.63700000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.98999999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.79800000000000004,"15min":0.80500000000000005},"min":{"1min":0.59799999999999998,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.0760000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.752}]},{"osd":6,"last update":"Fri Oct 6 02:26:02 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.81999999999999995,"15min":0.82799999999999996},"min":{"1min":0.65200000000000002,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.079,"5min":1.167,"15min":1.167},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.81999999999999995,"15min":0.83999999999999997},"min":{"1min":0.58899999999999997,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.032,"5min":1.069,"15min":1.069},"last":0.82599999999999996}]},{"osd":7,"last update":"Fri Oct 6 02:26:02 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.81799999999999995,"15min":0.83299999999999996},"min":{"1min":0.63300000000000001,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.81699999999999995,"15min":0.83599999999999997},"min":{"1min":0.59099999999999997,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.04,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.81399999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574899,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297980,"kb_used_data":692,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073860,"statfs":{"total":96636764160,"available":96331632640,"internally_reserved":0,"allocated":708608,"data_stored":307735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.44,"15min":0.443},"min":{"1min":0.28699999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.48299999999999998,"15min":0.48799999999999999},"min":{"1min":0.33400000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.84499999999999997,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.39300000000000002}]},{"osd":1,"last update":"Fri Oct 6 02:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.502,"15min":0.50800000000000001},"min":{"1min":0.33600000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.67100000000000004,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.58899999999999997,"15min":0.621},"min":{"1min":0.38800000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.88500000000000001,"5min":1.345,"15min":1.345},"last":0.52200000000000002}]},{"osd":3,"last update":"Fri Oct 6 02:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53300000000000003,"15min":0.54000000000000004},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.76500000000000001,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.61599999999999999,"15min":0.627},"min":{"1min":0.48099999999999998,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.80700000000000005,"5min":1.028,"15min":1.028},"last":0.65400000000000003}]},{"osd":4,"last update":"Fri Oct 6 02:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.77300000000000002,"15min":0.78400000000000003},"min":{"1min":0.54600000000000004,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.109,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.84199999999999997,"15min":0.86099999999999999},"min":{"1min":0.64000000000000001,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.1240000000000001,"5min":1.621,"15min":1.621},"last":0.71799999999999997}]},{"osd":5,"last update":"Fri Oct 6 02:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.78800000000000003,"15min":0.81200000000000006},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.0580000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.81599999999999995,"15min":0.83399999999999996},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.153,"5min":1.258,"15min":1.258},"last":0.78500000000000003}]},{"osd":6,"last update":"Fri Oct 6 02:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.83699999999999997,"15min":0.85899999999999999},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.1870000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.79700000000000004,"15min":0.80200000000000005},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.82599999999999996}]},{"osd":7,"last update":"Fri Oct 6 02:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.877,"15min":0.90000000000000002},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.099,"5min":1.585,"15min":1.585},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.82399999999999995,"15min":0.84199999999999997},"min":{"1min":0.55500000000000005,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.0369999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.90800000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574896,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297980,"kb_used_data":692,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073860,"statfs":{"total":96636764160,"available":96331632640,"internally_reserved":0,"allocated":708608,"data_stored":307735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:25:59 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.499,"15min":0.52000000000000002},"min":{"1min":0.36199999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.63,"5min":2.024,"15min":2.024},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.52200000000000002,"15min":0.53200000000000003},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.84699999999999998,"5min":2.181,"15min":2.181},"last":0.40699999999999997}]},{"osd":1,"last update":"Fri Oct 6 02:25:59 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.64100000000000001,"15min":0.72099999999999997},"min":{"1min":0.36699999999999999,"5min":0.312,"15min":0.312},"max":{"1min":0.61899999999999999,"5min":4.492,"15min":4.492},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.68899999999999995,"15min":0.76200000000000001},"min":{"1min":0.41999999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.65900000000000003,"5min":4.569,"15min":4.569},"last":0.51900000000000002}]},{"osd":2,"last update":"Fri Oct 6 02:25:59 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.66700000000000004,"15min":0.67500000000000004},"min":{"1min":0.58699999999999997,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.88400000000000001,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.63800000000000001,"15min":0.66400000000000003},"min":{"1min":0.48599999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.90600000000000003,"5min":2.4260000000000002,"15min":2.4260000000000002},"last":0.50800000000000001}]},{"osd":4,"last update":"Fri Oct 6 02:25:59 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.80800000000000005,"15min":0.84599999999999997},"min":{"1min":0.502,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.97899999999999998,"5min":2.214,"15min":2.214},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.80400000000000005,"15min":0.82299999999999995},"min":{"1min":0.60399999999999998,"5min":0.5,"15min":0.5},"max":{"1min":1.079,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.64000000000000001}]},{"osd":5,"last update":"Fri Oct 6 02:25:59 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.82099999999999995,"15min":0.83399999999999996},"min":{"1min":0.56999999999999995,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.0640000000000001,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.82999999999999996,"15min":0.872},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.93300000000000005,"5min":2.242,"15min":2.242},"last":0.72499999999999998}]},{"osd":6,"last update":"Fri Oct 6 02:25:59 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.80300000000000005,"15min":0.82799999999999996},"min":{"1min":0.52400000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.877,"5min":1.357,"15min":1.357},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.82999999999999996,"15min":0.84099999999999997},"min":{"1min":0.68700000000000006,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.0529999999999999,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.66100000000000003}]},{"osd":7,"last update":"Fri Oct 6 02:25:59 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.84599999999999997,"15min":0.874},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.0169999999999999,"5min":2.27,"15min":2.27},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.877,"5min":0.85199999999999998,"15min":0.871},"min":{"1min":0.67700000000000005,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.028,"5min":1.325,"15min":1.325},"last":0.55600000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574898,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302076,"kb_used_data":692,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069764,"statfs":{"total":96636764160,"available":96327438336,"internally_reserved":0,"allocated":708608,"data_stored":307735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:25:59 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.96299999999999997,"15min":1.087},"min":{"1min":0.44700000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.93100000000000005,"5min":12.337,"15min":12.337},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.879,"15min":0.97299999999999998},"min":{"1min":0.497,"5min":0.438,"15min":0.438},"max":{"1min":0.92200000000000004,"5min":9.6539999999999999,"15min":9.6539999999999999},"last":0.73099999999999998}]},{"osd":1,"last update":"Fri Oct 6 02:25:59 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.69399999999999995,"15min":0.70399999999999996},"min":{"1min":0.51800000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.82299999999999995,"5min":3.181,"15min":3.181},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.97499999999999998,"15min":1.107},"min":{"1min":0.53400000000000003,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.85199999999999998,"5min":12.946999999999999,"15min":12.946999999999999},"last":0.47999999999999998}]},{"osd":2,"last update":"Fri Oct 6 02:25:59 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.998,"15min":1.137},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.94899999999999995,"5min":12.377000000000001,"15min":12.377000000000001},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.754,"5min":0.84899999999999998,"15min":0.91100000000000003},"min":{"1min":0.58499999999999996,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.98299999999999998,"5min":6.3220000000000001,"15min":6.3220000000000001},"last":0.60799999999999998}]},{"osd":3,"last update":"Fri Oct 6 02:25:59 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.95599999999999996,"15min":1.0580000000000001},"min":{"1min":0.56299999999999994,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.97499999999999998,"5min":9.7720000000000002,"15min":9.7720000000000002},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.76500000000000001,"15min":0.79200000000000004},"min":{"1min":0.56000000000000005,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.90000000000000002,"5min":3.0840000000000001,"15min":3.0840000000000001},"last":0.70799999999999996}]},{"osd":5,"last update":"Fri Oct 6 02:25:59 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.81599999999999995,"15min":0.88700000000000001},"min":{"1min":0.50600000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.83299999999999996,"5min":6.2690000000000001,"15min":6.2690000000000001},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.97499999999999998,"15min":1.117},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.80900000000000005,"5min":10.769,"15min":10.769},"last":0.86099999999999999}]},{"osd":6,"last update":"Fri Oct 6 02:25:59 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.73799999999999999,"15min":0.77800000000000002},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.82799999999999996,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.92500000000000004,"15min":1.026},"min":{"1min":0.49099999999999999,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.89400000000000002,"5min":8.0899999999999999,"15min":8.0899999999999999},"last":0.75700000000000001}]},{"osd":7,"last update":"Fri Oct 6 02:25:59 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":1.016,"15min":1.143},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.88300000000000001,"5min":9.7089999999999996,"15min":9.7089999999999996},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.95799999999999996,"15min":1.0580000000000001},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.89300000000000002,"5min":7.6719999999999997,"15min":7.6719999999999997},"last":0.83699999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574898,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297980,"kb_used_data":692,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073860,"statfs":{"total":96636764160,"available":96331632640,"internally_reserved":0,"allocated":708608,"data_stored":307735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:25:59 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.64000000000000001,"15min":0.625},"min":{"1min":0.49099999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.79000000000000004,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.63600000000000001,"15min":0.60399999999999998},"min":{"1min":0.623,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.41699999999999998}]},{"osd":1,"last update":"Fri Oct 6 02:25:59 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.66700000000000004,"15min":0.64900000000000002},"min":{"1min":0.51900000000000002,"5min":0.434,"15min":0.434},"max":{"1min":1.002,"5min":1.089,"15min":1.089},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.75,"5min":0.68400000000000005,"15min":0.67300000000000004},"min":{"1min":0.63900000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.93600000000000005,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.56699999999999995}]},{"osd":2,"last update":"Fri Oct 6 02:25:59 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.69299999999999995,"15min":0.66700000000000004},"min":{"1min":0.66200000000000003,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.69999999999999996,"15min":0.68200000000000005},"min":{"1min":0.61099999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.995,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.44600000000000001}]},{"osd":3,"last update":"Fri Oct 6 02:25:59 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.74399999999999999,"15min":0.72499999999999998},"min":{"1min":0.59899999999999998,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.92100000000000004,"5min":1.042,"15min":1.042},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.78100000000000003,"15min":0.76400000000000001},"min":{"1min":0.63500000000000001,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.0760000000000001,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.59499999999999997}]},{"osd":4,"last update":"Fri Oct 6 02:25:59 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.67600000000000005,"15min":0.67800000000000005},"min":{"1min":0.40300000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.80800000000000005,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.68300000000000005,"15min":0.68600000000000005},"min":{"1min":0.42899999999999999,"5min":0.373,"15min":0.373},"max":{"1min":0.82099999999999995,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.68899999999999995}]},{"osd":6,"last update":"Fri Oct 6 02:25:59 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.749,"15min":0.747},"min":{"1min":0.47899999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.92900000000000005,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.82599999999999996,"15min":0.82399999999999995},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.95799999999999996,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.76400000000000001}]},{"osd":7,"last update":"Fri Oct 6 02:25:59 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.77400000000000002,"15min":0.76800000000000002},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.94399999999999995,"5min":1.089,"15min":1.089},"last":0.621},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.83599999999999997,"15min":0.84199999999999997},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.0469999999999999,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.77100000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T02:26:47.929 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T02:26:47.930 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T02:26:48.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.163+0000 7f17c1d9d700 1 Processor -- start 2023-10-06T02:26:48.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.163+0000 7f17c1d9d700 1 -- start start 2023-10-06T02:26:48.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.164+0000 7f17c1d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f17bc159380 0x7f17bc159750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:48.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.164+0000 7f17c1d9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17bc0cc350 con 0x7f17bc0cf110 2023-10-06T02:26:48.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.164+0000 7f17c1d9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17bc0cb740 con 0x7f17bc15dc30 2023-10-06T02:26:48.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.164+0000 7f17c1d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17bc0cba20 con 0x7f17bc159380 2023-10-06T02:26:48.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.165+0000 7f17c159c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f17bc0cf110 0x7f17bc0b0310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46788/0 (socket says 172.21.15.31:46788) 2023-10-06T02:26:48.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.165+0000 7f17c159c700 1 -- 172.21.15.31:0/545906720 learned_addr learned my addr 172.21.15.31:0/545906720 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:26:48.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.165+0000 7f17c0d9b700 1 --2- 172.21.15.31:0/545906720 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f17bc159380 0x7f17bc159750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:48.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.165+0000 7f17bb7fe700 1 -- 172.21.15.31:0/545906720 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2212303151 0 0) 0x7f17bc0cc350 con 0x7f17bc0cf110 2023-10-06T02:26:48.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.165+0000 7f17bb7fe700 1 -- 172.21.15.31:0/545906720 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f17ac003570 con 0x7f17bc0cf110 2023-10-06T02:26:48.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.165+0000 7f17bb7fe700 1 -- 172.21.15.31:0/545906720 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 762623365 0 0) 0x7f17bc0cb740 con 0x7f17bc15dc30 2023-10-06T02:26:48.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.165+0000 7f17bb7fe700 1 -- 172.21.15.31:0/545906720 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f17bc0cc350 con 0x7f17bc15dc30 2023-10-06T02:26:48.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.165+0000 7f17c0d9b700 1 -- 172.21.15.31:0/545906720 >> v1:172.21.15.62:6789/0 conn(0x7f17bc15dc30 legacy=0x7f17bc0cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:48.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.165+0000 7f17c0d9b700 1 -- 172.21.15.31:0/545906720 >> v1:172.21.15.31:6789/0 conn(0x7f17bc0cf110 legacy=0x7f17bc0b0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:48.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.165+0000 7f17c0d9b700 1 -- 172.21.15.31:0/545906720 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17bc0cbd00 con 0x7f17bc159380 2023-10-06T02:26:48.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.166+0000 7f17c0d9b700 1 --2- 172.21.15.31:0/545906720 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f17bc159380 0x7f17bc159750 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f17b0009910 tx=0x7f17b00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65d75646a166b496 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:48.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.166+0000 7f17bb7fe700 1 -- 172.21.15.31:0/545906720 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b0011630 con 0x7f17bc159380 2023-10-06T02:26:48.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.166+0000 7f17bb7fe700 1 -- 172.21.15.31:0/545906720 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17b0011790 con 0x7f17bc159380 2023-10-06T02:26:48.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.166+0000 7f17bb7fe700 1 -- 172.21.15.31:0/545906720 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b0011b30 con 0x7f17bc159380 2023-10-06T02:26:48.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.167+0000 7f17c1d9d700 1 -- 172.21.15.31:0/545906720 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f17bc159380 msgr2=0x7f17bc159750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:48.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.167+0000 7f17c1d9d700 1 --2- 172.21.15.31:0/545906720 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f17bc159380 0x7f17bc159750 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f17b0009910 tx=0x7f17b00091a0 comp rx=0 tx=0).stop 2023-10-06T02:26:48.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.167+0000 7f17c1d9d700 1 -- 172.21.15.31:0/545906720 shutdown_connections 2023-10-06T02:26:48.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.167+0000 7f17c1d9d700 1 --2- 172.21.15.31:0/545906720 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f17bc159380 0x7f17bc159750 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:48.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.167+0000 7f17c1d9d700 1 -- 172.21.15.31:0/545906720 >> 172.21.15.31:0/545906720 conn(0x7f17bc0703a0 msgr2=0x7f17bc0707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:48.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.167+0000 7f17c1d9d700 1 -- 172.21.15.31:0/545906720 shutdown_connections 2023-10-06T02:26:48.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.167+0000 7f17c1d9d700 1 -- 172.21.15.31:0/545906720 wait complete. 2023-10-06T02:26:48.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.168+0000 7f17c1d9d700 1 Processor -- start 2023-10-06T02:26:48.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.168+0000 7f17c1d9d700 1 -- start start 2023-10-06T02:26:48.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.169+0000 7f17c1d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17bc0cf110 0x7f17bc0b3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:48.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.169+0000 7f17c1d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17bc159380 0x7f17bc1536e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:48.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.169+0000 7f17c1d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f17bc15dc30 0x7f17bc159c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:48.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.169+0000 7f17c1d9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17bc0b4570 con 0x7f17bc159380 2023-10-06T02:26:48.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.169+0000 7f17c1d9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17bc0b4850 con 0x7f17bc0cf110 2023-10-06T02:26:48.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.169+0000 7f17c1d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17bc0b4b30 con 0x7f17bc15dc30 2023-10-06T02:26:48.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.170+0000 7f17bbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17bc159380 0x7f17bc1536e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:48.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.170+0000 7f17bbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17bc159380 0x7f17bc1536e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34530/0 (socket says 172.21.15.31:34530) 2023-10-06T02:26:48.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.170+0000 7f17bbfff700 1 -- 172.21.15.31:0/496422206 learned_addr learned my addr 172.21.15.31:0/496422206 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:26:48.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.170+0000 7f17c0d9b700 1 --2- 172.21.15.31:0/496422206 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17bc0cf110 0x7f17bc0b3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:48.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.170+0000 7f17c159c700 1 --2- 172.21.15.31:0/496422206 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f17bc15dc30 0x7f17bc159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:48.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.170+0000 7f17bbfff700 1 -- 172.21.15.31:0/496422206 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f17bc15dc30 msgr2=0x7f17bc159c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:48.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.170+0000 7f17bbfff700 1 --2- 172.21.15.31:0/496422206 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f17bc15dc30 0x7f17bc159c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:48.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.170+0000 7f17bbfff700 1 -- 172.21.15.31:0/496422206 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17bc0cf110 msgr2=0x7f17bc0b3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:48.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.170+0000 7f17bbfff700 1 --2- 172.21.15.31:0/496422206 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17bc0cf110 0x7f17bc0b3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:48.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.170+0000 7f17bbfff700 1 -- 172.21.15.31:0/496422206 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17b0009420 con 0x7f17bc159380 2023-10-06T02:26:48.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.170+0000 7f17bbfff700 1 --2- 172.21.15.31:0/496422206 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17bc159380 0x7f17bc1536e0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f17a800c580 tx=0x7f17a8007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:48.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.170+0000 7f17b8ff9700 1 -- 172.21.15.31:0/496422206 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a8010510 con 0x7f17bc159380 2023-10-06T02:26:48.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.171+0000 7f17b8ff9700 1 -- 172.21.15.31:0/496422206 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17a8010670 con 0x7f17bc159380 2023-10-06T02:26:48.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.171+0000 7f17b8ff9700 1 -- 172.21.15.31:0/496422206 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a8010a30 con 0x7f17bc159380 2023-10-06T02:26:48.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.171+0000 7f17c1d9d700 1 -- 172.21.15.31:0/496422206 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17bc155de0 con 0x7f17bc159380 2023-10-06T02:26:48.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.171+0000 7f17c1d9d700 1 -- 172.21.15.31:0/496422206 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17bc0b41d0 con 0x7f17bc159380 2023-10-06T02:26:48.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.172+0000 7f17c1d9d700 1 -- 172.21.15.31:0/496422206 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17bc155c50 con 0x7f17bc159380 2023-10-06T02:26:48.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.173+0000 7f17b8ff9700 1 -- 172.21.15.31:0/496422206 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 9) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f17a801b400 con 0x7f17bc159380 2023-10-06T02:26:48.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.173+0000 7f17b8ff9700 1 --2- 172.21.15.31:0/496422206 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f17a00670e0 0x7f17a0069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:48.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.173+0000 7f17b8ff9700 1 -- 172.21.15.31:0/496422206 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6203+0+0 (secure 0 0 0) 0x7f17a8084600 con 0x7f17bc159380 2023-10-06T02:26:48.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.174+0000 7f17c0d9b700 1 --2- 172.21.15.31:0/496422206 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f17a00670e0 0x7f17a0069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:48.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.174+0000 7f17c0d9b700 1 --2- 172.21.15.31:0/496422206 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f17a00670e0 0x7f17a0069590 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f17b0009860 tx=0x7f17b0017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:48.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.177+0000 7f17b8ff9700 1 -- 172.21.15.31:0/496422206 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f17a8051360 con 0x7f17bc159380 2023-10-06T02:26:48.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.464+0000 7f17c1d9d700 1 -- 172.21.15.31:0/496422206 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f17bc04c3c0 con 0x7f17bc159380 2023-10-06T02:26:48.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.467+0000 7f17b8ff9700 1 -- 172.21.15.31:0/496422206 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+0 (secure 0 0 0) 0x7f17a8054980 con 0x7f17bc159380 2023-10-06T02:26:48.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.470+0000 7f17c1d9d700 1 -- 172.21.15.31:0/496422206 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f17a00670e0 msgr2=0x7f17a0069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:48.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.470+0000 7f17c1d9d700 1 --2- 172.21.15.31:0/496422206 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f17a00670e0 0x7f17a0069590 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f17b0009860 tx=0x7f17b0017000 comp rx=0 tx=0).stop 2023-10-06T02:26:48.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.470+0000 7f17c1d9d700 1 -- 172.21.15.31:0/496422206 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17bc159380 msgr2=0x7f17bc1536e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:48.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.470+0000 7f17c1d9d700 1 --2- 172.21.15.31:0/496422206 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17bc159380 0x7f17bc1536e0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f17a800c580 tx=0x7f17a8007e10 comp rx=0 tx=0).stop 2023-10-06T02:26:48.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.470+0000 7f17c1d9d700 1 -- 172.21.15.31:0/496422206 shutdown_connections 2023-10-06T02:26:48.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.470+0000 7f17c1d9d700 1 --2- 172.21.15.31:0/496422206 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f17a00670e0 0x7f17a0069590 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:48.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.470+0000 7f17c1d9d700 1 --2- 172.21.15.31:0/496422206 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f17bc15dc30 0x7f17bc159c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:48.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.470+0000 7f17c1d9d700 1 --2- 172.21.15.31:0/496422206 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17bc0cf110 0x7f17bc0b3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:48.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.470+0000 7f17c1d9d700 1 --2- 172.21.15.31:0/496422206 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17bc159380 0x7f17bc1536e0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:48.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.470+0000 7f17c1d9d700 1 -- 172.21.15.31:0/496422206 >> 172.21.15.31:0/496422206 conn(0x7f17bc0703a0 msgr2=0x7f17bc0cd0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:48.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.470+0000 7f17c1d9d700 1 -- 172.21.15.31:0/496422206 shutdown_connections 2023-10-06T02:26:48.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.470+0000 7f17c1d9d700 1 -- 172.21.15.31:0/496422206 wait complete. 2023-10-06T02:26:48.472 INFO:teuthology.orchestra.run.smithi031.stderr:updated caps for client.0 2023-10-06T02:26:48.487 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:26:48.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.675+0000 7f74b77ff700 1 Processor -- start 2023-10-06T02:26:48.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.675+0000 7f74b77ff700 1 -- start start 2023-10-06T02:26:48.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.676+0000 7f74b77ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f74b0159c20 0x7f74b00b0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:48.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.676+0000 7f74b77ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f74b00b0e70 0x7f74b00c93e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:48.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.676+0000 7f74b77ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74b00adb20 con 0x7f74b0159850 2023-10-06T02:26:48.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.676+0000 7f74b77ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74b00ccdc0 con 0x7f74b0159c20 2023-10-06T02:26:48.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.676+0000 7f74b77ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74b00b12b0 con 0x7f74b00b0e70 2023-10-06T02:26:48.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.677+0000 7f74b4d9a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f74b0159c20 0x7f74b00b0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:48.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.677+0000 7f74b4d9a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f74b0159c20 0x7f74b00b0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34536/0 (socket says 172.21.15.31:34536) 2023-10-06T02:26:48.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.677+0000 7f74b4d9a700 1 -- 172.21.15.31:0/1871854711 learned_addr learned my addr 172.21.15.31:0/1871854711 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:26:48.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.677+0000 7f74b4d9a700 1 -- 172.21.15.31:0/1871854711 >> v2:172.21.15.62:3300/0 conn(0x7f74b00b0e70 msgr2=0x7f74b00c93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:26:48.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.677+0000 7f74b5d9c700 1 --2- 172.21.15.31:0/1871854711 >> v2:172.21.15.62:3300/0 conn(0x7f74b00b0e70 0x7f74b00c93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:48.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.677+0000 7f74b4d9a700 1 --2- 172.21.15.31:0/1871854711 >> v2:172.21.15.62:3300/0 conn(0x7f74b00b0e70 0x7f74b00c93e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:48.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.677+0000 7f74b4d9a700 1 -- 172.21.15.31:0/1871854711 >> v1:172.21.15.62:6789/0 conn(0x7f74b0159850 legacy=0x7f74b00adfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:26:48.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.677+0000 7f74b4d9a700 1 -- 172.21.15.31:0/1871854711 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74b00acf10 con 0x7f74b0159c20 2023-10-06T02:26:48.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.677+0000 7f74b5d9c700 1 --2- 172.21.15.31:0/1871854711 >> v2:172.21.15.62:3300/0 conn(0x7f74b00b0e70 0x7f74b00c93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:26:48.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.677+0000 7f74b4d9a700 1 --2- 172.21.15.31:0/1871854711 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74b0159c20 0x7f74b00b0930 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f74b00aeda0 tx=0x7f74a000b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d68d9784797f690 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:48.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.678+0000 7f74a7fff700 1 -- 172.21.15.31:0/1871854711 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a000dc10 con 0x7f74b0159c20 2023-10-06T02:26:48.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.678+0000 7f74a7fff700 1 -- 172.21.15.31:0/1871854711 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74a000dd70 con 0x7f74b0159c20 2023-10-06T02:26:48.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.678+0000 7f74a7fff700 1 -- 172.21.15.31:0/1871854711 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a00106c0 con 0x7f74b0159c20 2023-10-06T02:26:48.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.678+0000 7f74b77ff700 1 -- 172.21.15.31:0/1871854711 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74b0159c20 msgr2=0x7f74b00b0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:48.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.678+0000 7f74b77ff700 1 --2- 172.21.15.31:0/1871854711 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74b0159c20 0x7f74b00b0930 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f74b00aeda0 tx=0x7f74a000b4f0 comp rx=0 tx=0).stop 2023-10-06T02:26:48.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.678+0000 7f74b77ff700 1 -- 172.21.15.31:0/1871854711 shutdown_connections 2023-10-06T02:26:48.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.678+0000 7f74b77ff700 1 --2- 172.21.15.31:0/1871854711 >> v2:172.21.15.62:3300/0 conn(0x7f74b00b0e70 0x7f74b00c93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:48.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.678+0000 7f74b77ff700 1 --2- 172.21.15.31:0/1871854711 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74b0159c20 0x7f74b00b0930 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:48.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.678+0000 7f74b77ff700 1 -- 172.21.15.31:0/1871854711 >> 172.21.15.31:0/1871854711 conn(0x7f74b006d170 msgr2=0x7f74b00cc300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:48.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.678+0000 7f74b77ff700 1 -- 172.21.15.31:0/1871854711 shutdown_connections 2023-10-06T02:26:48.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.678+0000 7f74b77ff700 1 -- 172.21.15.31:0/1871854711 wait complete. 2023-10-06T02:26:48.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.680+0000 7f74b77ff700 1 Processor -- start 2023-10-06T02:26:48.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.680+0000 7f74b77ff700 1 -- start start 2023-10-06T02:26:48.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.681+0000 7f74b77ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f74b00b0e70 0x7f74b0150660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:48.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.681+0000 7f74b77ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74b0159850 0x7f74b006a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:48.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.681+0000 7f74b77ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74b0159c20 0x7f74b006aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:48.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.681+0000 7f74b77ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74b0168e50 con 0x7f74b0159c20 2023-10-06T02:26:48.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.681+0000 7f74b77ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74b0169130 con 0x7f74b0159850 2023-10-06T02:26:48.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.681+0000 7f74b77ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74b0169410 con 0x7f74b00b0e70 2023-10-06T02:26:48.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.681+0000 7f74b5d9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74b0159c20 0x7f74b006aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:48.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.681+0000 7f74b5d9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74b0159c20 0x7f74b006aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34546/0 (socket says 172.21.15.31:34546) 2023-10-06T02:26:48.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.681+0000 7f74b5d9c700 1 -- 172.21.15.31:0/1494191776 learned_addr learned my addr 172.21.15.31:0/1494191776 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:26:48.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.681+0000 7f74b4d9a700 1 --2- 172.21.15.31:0/1494191776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74b0159850 0x7f74b006a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:48.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.681+0000 7f74b559b700 1 --2- 172.21.15.31:0/1494191776 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f74b00b0e70 0x7f74b0150660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:48.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.682+0000 7f74b5d9c700 1 -- 172.21.15.31:0/1494191776 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f74b00b0e70 msgr2=0x7f74b0150660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:48.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.682+0000 7f74b5d9c700 1 --2- 172.21.15.31:0/1494191776 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f74b00b0e70 0x7f74b0150660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:48.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.682+0000 7f74b5d9c700 1 -- 172.21.15.31:0/1494191776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74b0159850 msgr2=0x7f74b006a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:48.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.682+0000 7f74b5d9c700 1 --2- 172.21.15.31:0/1494191776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74b0159850 0x7f74b006a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:48.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.682+0000 7f74b5d9c700 1 -- 172.21.15.31:0/1494191776 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74a000b790 con 0x7f74b0159c20 2023-10-06T02:26:48.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.682+0000 7f74b5d9c700 1 --2- 172.21.15.31:0/1494191776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74b0159c20 0x7f74b006aba0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f74a800a380 tx=0x7f74a8007900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:48.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.682+0000 7f74a57fa700 1 -- 172.21.15.31:0/1494191776 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a8015440 con 0x7f74b0159c20 2023-10-06T02:26:48.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.682+0000 7f74a57fa700 1 -- 172.21.15.31:0/1494191776 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74a80155a0 con 0x7f74b0159c20 2023-10-06T02:26:48.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.682+0000 7f74a57fa700 1 -- 172.21.15.31:0/1494191776 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a8015960 con 0x7f74b0159c20 2023-10-06T02:26:48.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.682+0000 7f74b77ff700 1 -- 172.21.15.31:0/1494191776 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74b01696f0 con 0x7f74b0159c20 2023-10-06T02:26:48.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.682+0000 7f74b77ff700 1 -- 172.21.15.31:0/1494191776 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74b0150c20 con 0x7f74b0159c20 2023-10-06T02:26:48.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.683+0000 7f74b77ff700 1 -- 172.21.15.31:0/1494191776 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74b0150f00 con 0x7f74b0159c20 2023-10-06T02:26:48.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.684+0000 7f74a57fa700 1 -- 172.21.15.31:0/1494191776 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 9) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f74a8015d50 con 0x7f74b0159c20 2023-10-06T02:26:48.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.685+0000 7f74a57fa700 1 --2- 172.21.15.31:0/1494191776 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f74940670e0 0x7f7494069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:48.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.685+0000 7f74a57fa700 1 -- 172.21.15.31:0/1494191776 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6203+0+0 (secure 0 0 0) 0x7f74a8085440 con 0x7f74b0159c20 2023-10-06T02:26:48.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.685+0000 7f74b559b700 1 --2- 172.21.15.31:0/1494191776 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f74940670e0 0x7f7494069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:48.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.686+0000 7f74b559b700 1 --2- 172.21.15.31:0/1494191776 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f74940670e0 0x7f7494069590 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f74ac003870 tx=0x7f74ac009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:48.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.689+0000 7f74a57fa700 1 -- 172.21.15.31:0/1494191776 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f74a8052240 con 0x7f74b0159c20 2023-10-06T02:26:48.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.935+0000 7f74b77ff700 1 -- 172.21.15.31:0/1494191776 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f74b004c3c0 con 0x7f74b0159c20 2023-10-06T02:26:48.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.936+0000 7f74a57fa700 1 -- 172.21.15.31:0/1494191776 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+4595 (secure 0 0 0) 0x7f74a8055860 con 0x7f74b0159c20 2023-10-06T02:26:48.936 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:26:48.936 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":37,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5216,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/2282788921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2282788921},{"type":"v1","addr":"172.21.15.62:6833","nonce":2282788921}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5219,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/289861443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":289861443},{"type":"v1","addr":"172.21.15.62:6835","nonce":289861443}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":5889,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/2954679985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":2954679985},{"type":"v1","addr":"172.21.15.31:6836","nonce":2954679985}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:26:42.554516+0000","modified":"2023-10-06T02:26:46.085562+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5892},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5892":{"gid":5892,"name":"c","rank":0,"incarnation":33,"state":"up:active","state_seq":5,"addr":"172.21.15.31:6837/1969136630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":1969136630},{"type":"v1","addr":"172.21.15.31:6837","nonce":1969136630}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-06T02:26:48.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.938+0000 7f74b77ff700 1 -- 172.21.15.31:0/1494191776 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f74940670e0 msgr2=0x7f7494069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:48.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.938+0000 7f74b77ff700 1 --2- 172.21.15.31:0/1494191776 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f74940670e0 0x7f7494069590 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f74ac003870 tx=0x7f74ac009040 comp rx=0 tx=0).stop 2023-10-06T02:26:48.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.938+0000 7f74b77ff700 1 -- 172.21.15.31:0/1494191776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74b0159c20 msgr2=0x7f74b006aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:48.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.938+0000 7f74b77ff700 1 --2- 172.21.15.31:0/1494191776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74b0159c20 0x7f74b006aba0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f74a800a380 tx=0x7f74a8007900 comp rx=0 tx=0).stop 2023-10-06T02:26:48.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.939+0000 7f74b77ff700 1 -- 172.21.15.31:0/1494191776 shutdown_connections 2023-10-06T02:26:48.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.939+0000 7f74b77ff700 1 --2- 172.21.15.31:0/1494191776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74b0159c20 0x7f74b006aba0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:48.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.939+0000 7f74b77ff700 1 --2- 172.21.15.31:0/1494191776 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f74940670e0 0x7f7494069590 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:48.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.939+0000 7f74b77ff700 1 --2- 172.21.15.31:0/1494191776 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f74b00b0e70 0x7f74b0150660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:48.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.939+0000 7f74b77ff700 1 --2- 172.21.15.31:0/1494191776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74b0159850 0x7f74b006a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:48.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.939+0000 7f74b77ff700 1 -- 172.21.15.31:0/1494191776 >> 172.21.15.31:0/1494191776 conn(0x7f74b006d170 msgr2=0x7f74b0069c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:48.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.939+0000 7f74b77ff700 1 -- 172.21.15.31:0/1494191776 shutdown_connections 2023-10-06T02:26:48.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:48.939+0000 7f74b77ff700 1 -- 172.21.15.31:0/1494191776 wait complete. 2023-10-06T02:26:48.940 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 37 2023-10-06T02:26:49.360 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 37, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T02:26:42.554516+0000', 'modified': '2023-10-06T02:26:46.085562+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5892}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5892': {'gid': 5892, 'name': 'c', 'rank': 0, 'incarnation': 33, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.31:6837/1969136630', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6835', 'nonce': 1969136630}, {'type': 'v1', 'addr': '172.21.15.31:6837', 'nonce': 1969136630}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T02:26:49.384 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T02:26:49.385 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-06T02:26:49.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.576+0000 7f4130f9d700 1 Processor -- start 2023-10-06T02:26:49.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.577+0000 7f4130f9d700 1 -- start start 2023-10-06T02:26:49.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.578+0000 7f4130f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f412c13fb20 0x7f412c15a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:49.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.578+0000 7f4130f9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f412c13fef0 0x7f412c0c7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:49.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.578+0000 7f4130f9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f412c15ae70 con 0x7f412c06aaa0 2023-10-06T02:26:49.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.578+0000 7f4130f9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f412c06af60 con 0x7f412c13fef0 2023-10-06T02:26:49.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.578+0000 7f4130f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f412c06b240 con 0x7f412c13fb20 2023-10-06T02:26:49.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.579+0000 7f412b7fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f412c13fb20 0x7f412c15a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:49.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.579+0000 7f412bfff700 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f412c06aaa0 0x7f412c0c6770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:49694/0 (socket says 172.21.15.31:49694) 2023-10-06T02:26:49.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.579+0000 7f412bfff700 1 -- 172.21.15.31:0/1746498570 learned_addr learned my addr 172.21.15.31:0/1746498570 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:26:49.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.579+0000 7f412affd700 1 --2- 172.21.15.31:0/1746498570 >> v2:172.21.15.62:3300/0 conn(0x7f412c13fef0 0x7f412c0c7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:49.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.579+0000 7f412b7fe700 1 -- 172.21.15.31:0/1746498570 >> v2:172.21.15.62:3300/0 conn(0x7f412c13fef0 msgr2=0x7f412c0c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:26:49.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.579+0000 7f412b7fe700 1 --2- 172.21.15.31:0/1746498570 >> v2:172.21.15.62:3300/0 conn(0x7f412c13fef0 0x7f412c0c7990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:49.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.579+0000 7f412b7fe700 1 -- 172.21.15.31:0/1746498570 >> v1:172.21.15.62:6789/0 conn(0x7f412c06aaa0 legacy=0x7f412c0c6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:49.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.579+0000 7f412b7fe700 1 -- 172.21.15.31:0/1746498570 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f412c06b520 con 0x7f412c13fb20 2023-10-06T02:26:49.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.579+0000 7f412affd700 1 --2- 172.21.15.31:0/1746498570 >> v2:172.21.15.62:3300/0 conn(0x7f412c13fef0 0x7f412c0c7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:26:49.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.580+0000 7f412b7fe700 1 --2- 172.21.15.31:0/1746498570 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f412c13fb20 0x7f412c15a930 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f411c0098e0 tx=0x7f411c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5148a0b7a9410b0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:49.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.580+0000 7f412a7fc700 1 -- 172.21.15.31:0/1746498570 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f411c00b640 con 0x7f412c13fb20 2023-10-06T02:26:49.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.580+0000 7f412a7fc700 1 -- 172.21.15.31:0/1746498570 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f411c00b7a0 con 0x7f412c13fb20 2023-10-06T02:26:49.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.580+0000 7f412a7fc700 1 -- 172.21.15.31:0/1746498570 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f411c00bb40 con 0x7f412c13fb20 2023-10-06T02:26:49.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.580+0000 7f4130f9d700 1 -- 172.21.15.31:0/1746498570 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f412c13fb20 msgr2=0x7f412c15a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:49.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.580+0000 7f4130f9d700 1 --2- 172.21.15.31:0/1746498570 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f412c13fb20 0x7f412c15a930 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f411c0098e0 tx=0x7f411c0091a0 comp rx=0 tx=0).stop 2023-10-06T02:26:49.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.580+0000 7f4130f9d700 1 -- 172.21.15.31:0/1746498570 shutdown_connections 2023-10-06T02:26:49.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.580+0000 7f4130f9d700 1 --2- 172.21.15.31:0/1746498570 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f412c13fb20 0x7f412c15a930 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:49.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.580+0000 7f4130f9d700 1 --2- 172.21.15.31:0/1746498570 >> v2:172.21.15.62:3300/0 conn(0x7f412c13fef0 0x7f412c0c7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:49.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.580+0000 7f4130f9d700 1 -- 172.21.15.31:0/1746498570 >> 172.21.15.31:0/1746498570 conn(0x7f412c06d170 msgr2=0x7f412c0cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:49.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.580+0000 7f4130f9d700 1 -- 172.21.15.31:0/1746498570 shutdown_connections 2023-10-06T02:26:49.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.581+0000 7f4130f9d700 1 -- 172.21.15.31:0/1746498570 wait complete. 2023-10-06T02:26:49.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.582+0000 7f4130f9d700 1 Processor -- start 2023-10-06T02:26:49.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.582+0000 7f4130f9d700 1 -- start start 2023-10-06T02:26:49.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.583+0000 7f4130f9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f412c06aaa0 0x7f412c0ae600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:49.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.583+0000 7f4130f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f412c13fb20 0x7f412c0ab740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:49.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.583+0000 7f4130f9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f412c13fef0 0x7f412c0ad3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:49.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.583+0000 7f4130f9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f412c139aa0 con 0x7f412c06aaa0 2023-10-06T02:26:49.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.583+0000 7f4130f9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f412c0c27e0 con 0x7f412c13fef0 2023-10-06T02:26:49.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.583+0000 7f4130f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f412c0c2ac0 con 0x7f412c13fb20 2023-10-06T02:26:49.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.583+0000 7f412b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f412c06aaa0 0x7f412c0ae600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:49.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.583+0000 7f412b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f412c06aaa0 0x7f412c0ae600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34556/0 (socket says 172.21.15.31:34556) 2023-10-06T02:26:49.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.583+0000 7f412b7fe700 1 -- 172.21.15.31:0/3570258429 learned_addr learned my addr 172.21.15.31:0/3570258429 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:26:49.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.584+0000 7f412affd700 1 --2- 172.21.15.31:0/3570258429 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f412c13fb20 0x7f412c0ab740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:49.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.584+0000 7f412b7fe700 1 -- 172.21.15.31:0/3570258429 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f412c13fb20 msgr2=0x7f412c0ab740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:49.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.584+0000 7f412bfff700 1 --2- 172.21.15.31:0/3570258429 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f412c13fef0 0x7f412c0ad3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:49.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.584+0000 7f412b7fe700 1 --2- 172.21.15.31:0/3570258429 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f412c13fb20 0x7f412c0ab740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:49.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.584+0000 7f412b7fe700 1 -- 172.21.15.31:0/3570258429 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f412c13fef0 msgr2=0x7f412c0ad3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:49.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.584+0000 7f412b7fe700 1 --2- 172.21.15.31:0/3570258429 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f412c13fef0 0x7f412c0ad3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:49.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.584+0000 7f412b7fe700 1 -- 172.21.15.31:0/3570258429 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41200093c0 con 0x7f412c06aaa0 2023-10-06T02:26:49.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.584+0000 7f412affd700 1 --2- 172.21.15.31:0/3570258429 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f412c13fb20 0x7f412c0ab740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:26:49.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.584+0000 7f412bfff700 1 --2- 172.21.15.31:0/3570258429 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f412c13fef0 0x7f412c0ad3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:26:49.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.584+0000 7f412b7fe700 1 --2- 172.21.15.31:0/3570258429 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f412c06aaa0 0x7f412c0ae600 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f412c06b380 tx=0x7f411c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:49.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.584+0000 7f41137fe700 1 -- 172.21.15.31:0/3570258429 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f411c01bda0 con 0x7f412c06aaa0 2023-10-06T02:26:49.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.584+0000 7f41137fe700 1 -- 172.21.15.31:0/3570258429 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f411c011420 con 0x7f412c06aaa0 2023-10-06T02:26:49.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.584+0000 7f4130f9d700 1 -- 172.21.15.31:0/3570258429 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f411c009420 con 0x7f412c06aaa0 2023-10-06T02:26:49.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.584+0000 7f41137fe700 1 -- 172.21.15.31:0/3570258429 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f411c0117e0 con 0x7f412c06aaa0 2023-10-06T02:26:49.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.584+0000 7f4130f9d700 1 -- 172.21.15.31:0/3570258429 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f412c139cf0 con 0x7f412c06aaa0 2023-10-06T02:26:49.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.586+0000 7f41137fe700 1 -- 172.21.15.31:0/3570258429 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 9) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f411c011b50 con 0x7f412c06aaa0 2023-10-06T02:26:49.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.587+0000 7f41137fe700 1 --2- 172.21.15.31:0/3570258429 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f41180670e0 0x7f4118069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:49.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.587+0000 7f41137fe700 1 -- 172.21.15.31:0/3570258429 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6203+0+0 (secure 0 0 0) 0x7f411c086250 con 0x7f412c06aaa0 2023-10-06T02:26:49.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.588+0000 7f412affd700 1 --2- 172.21.15.31:0/3570258429 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f41180670e0 0x7f4118069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:49.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.588+0000 7f412affd700 1 --2- 172.21.15.31:0/3570258429 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f41180670e0 0x7f4118069590 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f4120009390 tx=0x7f4120008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:49.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.589+0000 7f414089db80 1 Processor -- start 2023-10-06T02:26:49.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.589+0000 7f414089db80 1 -- start start 2023-10-06T02:26:49.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.590+0000 7f414089db80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d085fcfe50 0x55d085faacc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:49.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.590+0000 7f414089db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d085fb02c0 0x55d085fab200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:49.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.590+0000 7f414089db80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d085fb1a10 0x55d085fab740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:49.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.590+0000 7f414089db80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d085f05a70 con 0x55d085fcfe50 2023-10-06T02:26:49.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.590+0000 7f414089db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d085f03270 con 0x55d085fb02c0 2023-10-06T02:26:49.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.590+0000 7f414089db80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d085eff060 con 0x55d085fb1a10 2023-10-06T02:26:49.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.590+0000 7f412b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d085fcfe50 0x55d085faacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:49.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.590+0000 7f412b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d085fcfe50 0x55d085faacc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34566/0 (socket says 172.21.15.31:34566) 2023-10-06T02:26:49.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.590+0000 7f412b7fe700 1 -- 172.21.15.31:0/3792929596 learned_addr learned my addr 172.21.15.31:0/3792929596 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:26:49.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.590+0000 7f412b7fe700 1 -- 172.21.15.31:0/3792929596 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d085fb1a10 msgr2=0x55d085fab740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:26:49.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.590+0000 7f412b7fe700 1 --2- 172.21.15.31:0/3792929596 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d085fb1a10 0x55d085fab740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:49.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.590+0000 7f412b7fe700 1 -- 172.21.15.31:0/3792929596 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d085fb02c0 msgr2=0x55d085fab200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:26:49.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.590+0000 7f412b7fe700 1 --2- 172.21.15.31:0/3792929596 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d085fb02c0 0x55d085fab200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:49.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.590+0000 7f412b7fe700 1 -- 172.21.15.31:0/3792929596 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d085d99270 con 0x55d085fcfe50 2023-10-06T02:26:49.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.591+0000 7f412b7fe700 1 --2- 172.21.15.31:0/3792929596 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d085fcfe50 0x55d085faacc0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f411c053170 tx=0x7f411c05c050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e8fc8c1094b7dd9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:49.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.591+0000 7f4110ff9700 1 -- 172.21.15.31:0/3792929596 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f411c057370 con 0x55d085fcfe50 2023-10-06T02:26:49.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.591+0000 7f4110ff9700 1 -- 172.21.15.31:0/3792929596 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f411c056c70 con 0x55d085fcfe50 2023-10-06T02:26:49.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.591+0000 7f4110ff9700 1 -- 172.21.15.31:0/3792929596 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f411c059ec0 con 0x55d085fcfe50 2023-10-06T02:26:49.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.591+0000 7f414089db80 1 -- 172.21.15.31:0/3792929596 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d085fcfe50 msgr2=0x55d085faacc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:49.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.591+0000 7f414089db80 1 --2- 172.21.15.31:0/3792929596 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d085fcfe50 0x55d085faacc0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f411c053170 tx=0x7f411c05c050 comp rx=0 tx=0).stop 2023-10-06T02:26:49.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.591+0000 7f414089db80 1 -- 172.21.15.31:0/3792929596 shutdown_connections 2023-10-06T02:26:49.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.591+0000 7f414089db80 1 --2- 172.21.15.31:0/3792929596 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d085fb1a10 0x55d085fab740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:49.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.591+0000 7f414089db80 1 --2- 172.21.15.31:0/3792929596 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d085fcfe50 0x55d085faacc0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:49.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.592+0000 7f414089db80 1 --2- 172.21.15.31:0/3792929596 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d085fb02c0 0x55d085fab200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:49.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.592+0000 7f414089db80 1 -- 172.21.15.31:0/3792929596 >> 172.21.15.31:0/3792929596 conn(0x55d085fafe80 msgr2=0x55d085fb06b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:49.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.592+0000 7f414089db80 1 -- 172.21.15.31:0/3792929596 shutdown_connections 2023-10-06T02:26:49.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.592+0000 7f414089db80 1 -- 172.21.15.31:0/3792929596 wait complete. 2023-10-06T02:26:49.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.592+0000 7f414089db80 1 -- start start 2023-10-06T02:26:49.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.592+0000 7f414089db80 1 Processor -- start 2023-10-06T02:26:49.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.593+0000 7f414089db80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d085fb02c0 0x55d085fd78c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:49.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.593+0000 7f414089db80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d085fb1a10 0x55d085fd02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:49.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.593+0000 7f414089db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d085fcfe50 0x55d085fdd520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:49.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.593+0000 7f414089db80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d085eaa890 con 0x55d085fb02c0 2023-10-06T02:26:49.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.593+0000 7f414089db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d085ec5ea0 con 0x55d085fcfe50 2023-10-06T02:26:49.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.593+0000 7f414089db80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d085d81e30 con 0x55d085fb1a10 2023-10-06T02:26:49.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.594+0000 7f414089db80 10 client.0 _ll_register_callbacks cb 0x55d085f72020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T02:26:49.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.594+0000 7f412b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d085fb02c0 0x55d085fd78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:49.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.594+0000 7f412b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d085fb02c0 0x55d085fd78c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34572/0 (socket says 172.21.15.31:34572) 2023-10-06T02:26:49.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.594+0000 7f412b7fe700 1 -- 172.21.15.31:0/2441686052 learned_addr learned my addr 172.21.15.31:0/2441686052 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:26:49.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.594+0000 7f412b7fe700 1 -- 172.21.15.31:0/2441686052 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d085fb1a10 msgr2=0x55d085fd02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:26:49.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.594+0000 7f412b7fe700 1 --2- 172.21.15.31:0/2441686052 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d085fb1a10 0x55d085fd02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:49.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.594+0000 7f412b7fe700 1 -- 172.21.15.31:0/2441686052 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d085fcfe50 msgr2=0x55d085fdd520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:26:49.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.594+0000 7f412b7fe700 1 --2- 172.21.15.31:0/2441686052 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d085fcfe50 0x55d085fdd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:49.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.594+0000 7f412b7fe700 1 -- 172.21.15.31:0/2441686052 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f412c139cf0 con 0x55d085fb02c0 2023-10-06T02:26:49.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.594+0000 7f412b7fe700 1 --2- 172.21.15.31:0/2441686052 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d085fb02c0 0x55d085fd78c0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f411c0842d0 tx=0x7f411c06a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21ddee3d5b10baab server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:49.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.594+0000 7f4109ffb700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T02:26:49.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.594+0000 7f4109ffb700 1 -- 172.21.15.31:0/2441686052 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f411c05dc10 con 0x55d085fb02c0 2023-10-06T02:26:49.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.594+0000 7f4109ffb700 1 -- 172.21.15.31:0/2441686052 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(63..63 src has 1..63) v4 ==== 6203+0+0 (secure 0 0 0) 0x7f411c059ec0 con 0x55d085fb02c0 2023-10-06T02:26:49.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.594+0000 7f414089db80 1 -- 172.21.15.31:0/2441686052 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d085fa3940 con 0x55d085fb02c0 2023-10-06T02:26:49.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.595+0000 7f4109ffb700 1 -- 172.21.15.31:0/2441686052 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f411c056160 con 0x55d085fb02c0 2023-10-06T02:26:49.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.595+0000 7f414089db80 10 client.6027 fetch_fsmap learned FSMap version 37 2023-10-06T02:26:49.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.595+0000 7f414089db80 1 -- 172.21.15.31:0/2441686052 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) v3 -- 0x55d085d3bfe0 con 0x55d085fb02c0 2023-10-06T02:26:49.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.595+0000 7f4109ffb700 1 -- 172.21.15.31:0/2441686052 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d085fa3940 con 0x55d085fb02c0 2023-10-06T02:26:49.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.595+0000 7f4109ffb700 1 -- 172.21.15.31:0/2441686052 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 37) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f411c05d780 con 0x55d085fb02c0 2023-10-06T02:26:49.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.595+0000 7f414089db80 10 client.6027 fetch_fsmap finished waiting for FSMap version 37 2023-10-06T02:26:49.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.595+0000 7f414089db80 10 client.6027 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T02:26:49.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.595+0000 7f414089db80 20 client.6027 populate_metadata read hostname 'smithi031' 2023-10-06T02:26:49.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.595+0000 7f414089db80 1 --2- 172.21.15.31:0/2441686052 >> [v2:172.21.15.31:6835/1969136630,v1:172.21.15.31:6837/1969136630] conn(0x55d0860060d0 0x55d0860064a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:49.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.595+0000 7f414089db80 4 client.6027 mds_command: new command op to 5892 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T02:26:49.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.595+0000 7f414089db80 1 -- 172.21.15.31:0/2441686052 --> [v2:172.21.15.31:6835/1969136630,v1:172.21.15.31:6837/1969136630] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d085f80870 con 0x55d0860060d0 2023-10-06T02:26:49.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.595+0000 7f412affd700 1 --2- 172.21.15.31:0/2441686052 >> [v2:172.21.15.31:6835/1969136630,v1:172.21.15.31:6837/1969136630] conn(0x55d0860060d0 0x55d0860064a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:49.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.596+0000 7f412affd700 1 --2- 172.21.15.31:0/2441686052 >> [v2:172.21.15.31:6835/1969136630,v1:172.21.15.31:6837/1969136630] conn(0x55d0860060d0 0x55d0860064a0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eedbed72961801be server_cookie=c58b73fa76ac1108 in_seq=0 out_seq=0 2023-10-06T02:26:49.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.596+0000 7f4109ffb700 10 client.6027 ms_handle_connect on v2:172.21.15.31:6835/1969136630 2023-10-06T02:26:49.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.596+0000 7f4109ffb700 1 -- 172.21.15.31:0/2441686052 <== mds.0 v2:172.21.15.31:6835/1969136630 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55d085f80870 con 0x55d0860060d0 2023-10-06T02:26:49.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.597+0000 7f4109ffb700 10 client.6027 handle_command_reply: tid=0 2023-10-06T02:26:49.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.597+0000 7f414089db80 1 client.6027 shutdown 2023-10-06T02:26:49.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.597+0000 7f414089db80 1 -- 172.21.15.31:0/2441686052 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d085fb02c0 msgr2=0x55d085fd78c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:49.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.597+0000 7f414089db80 1 --2- 172.21.15.31:0/2441686052 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d085fb02c0 0x55d085fd78c0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f411c0842d0 tx=0x7f411c06a000 comp rx=0 tx=0).stop 2023-10-06T02:26:49.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.597+0000 7f414089db80 1 -- 172.21.15.31:0/2441686052 shutdown_connections 2023-10-06T02:26:49.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.597+0000 7f414089db80 1 --2- 172.21.15.31:0/2441686052 >> [v2:172.21.15.31:6835/1969136630,v1:172.21.15.31:6837/1969136630] conn(0x55d0860060d0 0x55d0860064a0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:49.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.597+0000 7f414089db80 1 --2- 172.21.15.31:0/2441686052 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d085fcfe50 0x55d085fdd520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:49.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.597+0000 7f414089db80 1 --2- 172.21.15.31:0/2441686052 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d085fb02c0 0x55d085fd78c0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:49.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.597+0000 7f414089db80 1 --2- 172.21.15.31:0/2441686052 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d085fb1a10 0x55d085fd02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:49.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.597+0000 7f4109ffb700 0 client.6027 ms_handle_reset on v2:172.21.15.31:6835/1969136630 2023-10-06T02:26:49.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.597+0000 7f414089db80 1 -- 172.21.15.31:0/2441686052 >> 172.21.15.31:0/2441686052 conn(0x55d085fafe80 msgr2=0x55d085fa5400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:49.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.597+0000 7f414089db80 1 -- 172.21.15.31:0/2441686052 shutdown_connections 2023-10-06T02:26:49.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.597+0000 7f414089db80 1 -- 172.21.15.31:0/2441686052 wait complete. 2023-10-06T02:26:49.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.597+0000 7f414089db80 20 client.6027 trim_cache size 0 max 16384 2023-10-06T02:26:49.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.615+0000 7f414089db80 1 Processor -- start 2023-10-06T02:26:49.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.615+0000 7f414089db80 1 -- start start 2023-10-06T02:26:49.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.616+0000 7f414089db80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d085fb02c0 0x55d085fb0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:49.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.616+0000 7f414089db80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d085fb1a10 0x55d086054fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:49.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.616+0000 7f414089db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d085fcfe50 0x55d0860554f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:49.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.616+0000 7f414089db80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d085fcd090 con 0x55d085fb1a10 2023-10-06T02:26:49.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.616+0000 7f414089db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d085fb2600 con 0x55d085fcfe50 2023-10-06T02:26:49.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.616+0000 7f414089db80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d086007080 con 0x55d085fb02c0 2023-10-06T02:26:49.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.616+0000 7f412affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d085fb1a10 0x55d086054fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:49.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.616+0000 7f412affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d085fb1a10 0x55d086054fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34586/0 (socket says 172.21.15.31:34586) 2023-10-06T02:26:49.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.616+0000 7f412affd700 1 -- 172.21.15.31:0/3718027349 learned_addr learned my addr 172.21.15.31:0/3718027349 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:26:49.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.616+0000 7f412affd700 1 -- 172.21.15.31:0/3718027349 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d085fb02c0 msgr2=0x55d085fb0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:26:49.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.616+0000 7f412affd700 1 --2- 172.21.15.31:0/3718027349 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d085fb02c0 0x55d085fb0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:49.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.616+0000 7f412affd700 1 -- 172.21.15.31:0/3718027349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d085fcfe50 msgr2=0x55d0860554f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:26:49.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.616+0000 7f412affd700 1 --2- 172.21.15.31:0/3718027349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d085fcfe50 0x55d0860554f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:49.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.616+0000 7f412affd700 1 -- 172.21.15.31:0/3718027349 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d085ecd780 con 0x55d085fb1a10 2023-10-06T02:26:49.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.617+0000 7f412affd700 1 --2- 172.21.15.31:0/3718027349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d085fb1a10 0x55d086054fb0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f4120010c10 tx=0x7f4120015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49269c42e720c600 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:49.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.617+0000 7f41117fa700 1 -- 172.21.15.31:0/3718027349 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f412002a070 con 0x55d085fb1a10 2023-10-06T02:26:49.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.617+0000 7f41117fa700 1 -- 172.21.15.31:0/3718027349 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4120025b00 con 0x55d085fb1a10 2023-10-06T02:26:49.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.617+0000 7f41117fa700 1 -- 172.21.15.31:0/3718027349 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4120025ea0 con 0x55d085fb1a10 2023-10-06T02:26:49.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.617+0000 7f414089db80 1 -- 172.21.15.31:0/3718027349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d085fb1a10 msgr2=0x55d086054fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:49.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.617+0000 7f414089db80 1 --2- 172.21.15.31:0/3718027349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d085fb1a10 0x55d086054fb0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f4120010c10 tx=0x7f4120015000 comp rx=0 tx=0).stop 2023-10-06T02:26:49.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.617+0000 7f414089db80 1 -- 172.21.15.31:0/3718027349 shutdown_connections 2023-10-06T02:26:49.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.617+0000 7f414089db80 1 --2- 172.21.15.31:0/3718027349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d085fcfe50 0x55d0860554f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:49.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.617+0000 7f414089db80 1 --2- 172.21.15.31:0/3718027349 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d085fb02c0 0x55d085fb0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:49.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.617+0000 7f414089db80 1 --2- 172.21.15.31:0/3718027349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d085fb1a10 0x55d086054fb0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:49.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.617+0000 7f414089db80 1 -- 172.21.15.31:0/3718027349 >> 172.21.15.31:0/3718027349 conn(0x55d085fafe80 msgr2=0x55d0860052a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:49.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.617+0000 7f414089db80 1 -- 172.21.15.31:0/3718027349 shutdown_connections 2023-10-06T02:26:49.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.618+0000 7f414089db80 1 -- 172.21.15.31:0/3718027349 wait complete. 2023-10-06T02:26:49.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.618+0000 7f414089db80 1 -- start start 2023-10-06T02:26:49.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.618+0000 7f414089db80 1 Processor -- start 2023-10-06T02:26:49.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.619+0000 7f414089db80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d085fb02c0 0x55d085fcc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:49.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.619+0000 7f414089db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d085fb1a10 0x55d085fd1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:49.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.619+0000 7f414089db80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d085fcfe50 0x55d085fcdc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:49.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.619+0000 7f414089db80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d085fd2420 con 0x55d085fb02c0 2023-10-06T02:26:49.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.619+0000 7f414089db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d085fcc780 con 0x55d085fb1a10 2023-10-06T02:26:49.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.619+0000 7f414089db80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d085fd2180 con 0x55d085fcfe50 2023-10-06T02:26:49.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.619+0000 7f414089db80 10 client.0 _ll_register_callbacks cb 0x55d085f72020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T02:26:49.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.620+0000 7f412b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d085fb02c0 0x55d085fcc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:49.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.620+0000 7f412b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d085fb02c0 0x55d085fcc240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34596/0 (socket says 172.21.15.31:34596) 2023-10-06T02:26:49.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.620+0000 7f412b7fe700 1 -- 172.21.15.31:0/3479461426 learned_addr learned my addr 172.21.15.31:0/3479461426 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:26:49.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.620+0000 7f412bfff700 1 --2- 172.21.15.31:0/3479461426 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d085fcfe50 0x55d085fcdc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:49.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.620+0000 7f412affd700 1 --2- 172.21.15.31:0/3479461426 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d085fb1a10 0x55d085fd1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:49.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.620+0000 7f412b7fe700 1 -- 172.21.15.31:0/3479461426 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d085fcfe50 msgr2=0x55d085fcdc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:26:49.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.620+0000 7f412b7fe700 1 --2- 172.21.15.31:0/3479461426 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d085fcfe50 0x55d085fcdc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:49.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.620+0000 7f412b7fe700 1 -- 172.21.15.31:0/3479461426 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d085fb1a10 msgr2=0x55d085fd1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:26:49.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.620+0000 7f412b7fe700 1 --2- 172.21.15.31:0/3479461426 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d085fb1a10 0x55d085fd1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:49.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.620+0000 7f412b7fe700 1 -- 172.21.15.31:0/3479461426 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f412000e050 con 0x55d085fb02c0 2023-10-06T02:26:49.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.620+0000 7f412b7fe700 1 --2- 172.21.15.31:0/3479461426 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d085fb02c0 0x55d085fcc240 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f411c023000 tx=0x7f411c01e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3006454597434b26 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:49.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.620+0000 7f4109ffb700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T02:26:49.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.620+0000 7f4109ffb700 1 -- 172.21.15.31:0/3479461426 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f411c056160 con 0x55d085fb02c0 2023-10-06T02:26:49.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.620+0000 7f4109ffb700 1 -- 172.21.15.31:0/3479461426 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(63..63 src has 1..63) v4 ==== 6203+0+0 (secure 0 0 0) 0x7f411c009210 con 0x55d085fb02c0 2023-10-06T02:26:49.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.620+0000 7f414089db80 1 -- 172.21.15.31:0/3479461426 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f411c056c70 con 0x55d085fb02c0 2023-10-06T02:26:49.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.621+0000 7f4109ffb700 1 -- 172.21.15.31:0/3479461426 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f411c05dc10 con 0x55d085fb02c0 2023-10-06T02:26:49.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.621+0000 7f414089db80 10 client.6033 fetch_fsmap learned FSMap version 37 2023-10-06T02:26:49.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.621+0000 7f414089db80 1 -- 172.21.15.31:0/3479461426 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) v3 -- 0x55d085eb4470 con 0x55d085fb02c0 2023-10-06T02:26:49.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.621+0000 7f4109ffb700 1 -- 172.21.15.31:0/3479461426 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f411c056c70 con 0x55d085fb02c0 2023-10-06T02:26:49.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.621+0000 7f4109ffb700 1 -- 172.21.15.31:0/3479461426 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 37) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f411c05d780 con 0x55d085fb02c0 2023-10-06T02:26:49.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.621+0000 7f414089db80 10 client.6033 fetch_fsmap finished waiting for FSMap version 37 2023-10-06T02:26:49.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.621+0000 7f414089db80 10 client.6033 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T02:26:49.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.621+0000 7f414089db80 20 client.6033 populate_metadata read hostname 'smithi031' 2023-10-06T02:26:49.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.621+0000 7f414089db80 1 --2- 172.21.15.31:0/3479461426 >> [v2:172.21.15.31:6835/1969136630,v1:172.21.15.31:6837/1969136630] conn(0x55d08605a0c0 0x55d08605a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:49.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.621+0000 7f414089db80 4 client.6033 mds_command: new command op to 5892 tid=0[{"prefix": "status"}] 2023-10-06T02:26:49.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.621+0000 7f414089db80 1 -- 172.21.15.31:0/3479461426 --> [v2:172.21.15.31:6835/1969136630,v1:172.21.15.31:6837/1969136630] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d085ea1e00 con 0x55d08605a0c0 2023-10-06T02:26:49.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.621+0000 7f412affd700 1 --2- 172.21.15.31:0/3479461426 >> [v2:172.21.15.31:6835/1969136630,v1:172.21.15.31:6837/1969136630] conn(0x55d08605a0c0 0x55d08605a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:49.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.622+0000 7f4109ffb700 10 client.6033 ms_handle_connect on v2:172.21.15.31:6835/1969136630 2023-10-06T02:26:49.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.622+0000 7f412affd700 1 --2- 172.21.15.31:0/3479461426 >> [v2:172.21.15.31:6835/1969136630,v1:172.21.15.31:6837/1969136630] conn(0x55d08605a0c0 0x55d08605a490 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f51bfa8a2b6acb2 server_cookie=b60d8cb10eec0618 in_seq=0 out_seq=0 2023-10-06T02:26:49.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.622+0000 7f4109ffb700 1 -- 172.21.15.31:0/3479461426 <== mds.0 v2:172.21.15.31:6835/1969136630 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55d085ea1e00 con 0x55d08605a0c0 2023-10-06T02:26:49.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.622+0000 7f4109ffb700 10 client.6033 handle_command_reply: tid=0 2023-10-06T02:26:49.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.622+0000 7f414089db80 1 client.6033 shutdown 2023-10-06T02:26:49.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.622+0000 7f414089db80 1 -- 172.21.15.31:0/3479461426 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d085fb02c0 msgr2=0x55d085fcc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:49.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.622+0000 7f414089db80 1 --2- 172.21.15.31:0/3479461426 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d085fb02c0 0x55d085fcc240 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f411c023000 tx=0x7f411c01e070 comp rx=0 tx=0).stop 2023-10-06T02:26:49.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.622+0000 7f414089db80 1 -- 172.21.15.31:0/3479461426 shutdown_connections 2023-10-06T02:26:49.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.622+0000 7f414089db80 1 --2- 172.21.15.31:0/3479461426 >> [v2:172.21.15.31:6835/1969136630,v1:172.21.15.31:6837/1969136630] conn(0x55d08605a0c0 0x55d08605a490 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:49.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.622+0000 7f414089db80 1 --2- 172.21.15.31:0/3479461426 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d085fcfe50 0x55d085fcdc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:49.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.622+0000 7f414089db80 1 --2- 172.21.15.31:0/3479461426 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d085fb02c0 0x55d085fcc240 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:49.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.622+0000 7f414089db80 1 --2- 172.21.15.31:0/3479461426 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d085fb1a10 0x55d085fd1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:49.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.622+0000 7f414089db80 1 -- 172.21.15.31:0/3479461426 >> 172.21.15.31:0/3479461426 conn(0x55d085fafe80 msgr2=0x55d0860052a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:49.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.622+0000 7f4109ffb700 0 client.6033 ms_handle_reset on v2:172.21.15.31:6835/1969136630 2023-10-06T02:26:49.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.622+0000 7f414089db80 1 -- 172.21.15.31:0/3479461426 shutdown_connections 2023-10-06T02:26:49.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.622+0000 7f414089db80 1 -- 172.21.15.31:0/3479461426 wait complete. 2023-10-06T02:26:49.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.623+0000 7f414089db80 20 client.6033 trim_cache size 0 max 16384 2023-10-06T02:26:49.895 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T02:26:49.897 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "d6fcf512-874b-4259-b90a-30ae070f00a9", 2023-10-06T02:26:49.897 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 0, 2023-10-06T02:26:49.897 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 5892, 2023-10-06T02:26:49.897 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2023-10-06T02:26:49.897 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2023-10-06T02:26:49.897 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2023-10-06T02:26:49.897 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 6.5662746519999997, 2023-10-06T02:26:49.898 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 37, 2023-10-06T02:26:49.898 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 63, 2023-10-06T02:26:49.898 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 61, 2023-10-06T02:26:49.898 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 19.061516946000001 2023-10-06T02:26:49.898 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T02:26:49.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.623+0000 7f4130f9d700 1 -- 172.21.15.31:0/3570258429 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f41180670e0 msgr2=0x7f4118069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:49.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.623+0000 7f4130f9d700 1 --2- 172.21.15.31:0/3570258429 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f41180670e0 0x7f4118069590 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f4120009390 tx=0x7f4120008040 comp rx=0 tx=0).stop 2023-10-06T02:26:49.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.623+0000 7f4130f9d700 1 -- 172.21.15.31:0/3570258429 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f412c06aaa0 msgr2=0x7f412c0ae600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:49.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.623+0000 7f4130f9d700 1 --2- 172.21.15.31:0/3570258429 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f412c06aaa0 0x7f412c0ae600 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f412c06b380 tx=0x7f411c017000 comp rx=0 tx=0).stop 2023-10-06T02:26:49.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.623+0000 7f4130f9d700 1 -- 172.21.15.31:0/3570258429 shutdown_connections 2023-10-06T02:26:49.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.623+0000 7f4130f9d700 1 --2- 172.21.15.31:0/3570258429 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f412c13fef0 0x7f412c0ad3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:49.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.623+0000 7f4130f9d700 1 --2- 172.21.15.31:0/3570258429 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f412c06aaa0 0x7f412c0ae600 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:49.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.623+0000 7f4130f9d700 1 --2- 172.21.15.31:0/3570258429 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f41180670e0 0x7f4118069590 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:49.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.623+0000 7f4130f9d700 1 --2- 172.21.15.31:0/3570258429 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f412c13fb20 0x7f412c0ab740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:49.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.623+0000 7f4130f9d700 1 -- 172.21.15.31:0/3570258429 >> 172.21.15.31:0/3570258429 conn(0x7f412c06d170 msgr2=0x7f412c13d650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:49.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.623+0000 7f4130f9d700 1 -- 172.21.15.31:0/3570258429 shutdown_connections 2023-10-06T02:26:49.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:49.623+0000 7f4130f9d700 1 -- 172.21.15.31:0/3570258429 wait complete. 2023-10-06T02:26:49.901 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T02:26:50.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.090+0000 7f2d3b59e700 1 Processor -- start 2023-10-06T02:26:50.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.091+0000 7f2d3b59e700 1 -- start start 2023-10-06T02:26:50.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.092+0000 7f2d3b59e700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2d3c06ed40 0x7f2d3c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:50.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.092+0000 7f2d3b59e700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d3c0c8ce0 con 0x7f2d3c153d70 2023-10-06T02:26:50.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.092+0000 7f2d3b59e700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d3c06f6e0 con 0x7f2d3c156860 2023-10-06T02:26:50.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.092+0000 7f2d3b59e700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d3c0aea10 con 0x7f2d3c06ed40 2023-10-06T02:26:50.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.092+0000 7f2d3a59c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f2d3c153d70 0x7f2d3c0acba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46794/0 (socket says 172.21.15.31:46794) 2023-10-06T02:26:50.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.092+0000 7f2d3a59c700 1 -- 172.21.15.31:0/929086075 learned_addr learned my addr 172.21.15.31:0/929086075 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:26:50.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.093+0000 7f2d3959a700 1 -- 172.21.15.31:0/929086075 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 245331773 0 0) 0x7f2d3c0c8ce0 con 0x7f2d3c153d70 2023-10-06T02:26:50.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.093+0000 7f2d3959a700 1 -- 172.21.15.31:0/929086075 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d24003570 con 0x7f2d3c153d70 2023-10-06T02:26:50.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.093+0000 7f2d3ad9d700 1 --2- 172.21.15.31:0/929086075 >> v2:172.21.15.62:3300/0 conn(0x7f2d3c06ed40 0x7f2d3c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:50.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.093+0000 7f2d3959a700 1 -- 172.21.15.31:0/929086075 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1213409291 0 0) 0x7f2d24003570 con 0x7f2d3c153d70 2023-10-06T02:26:50.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.093+0000 7f2d3959a700 1 -- 172.21.15.31:0/929086075 >> v2:172.21.15.62:3300/0 conn(0x7f2d3c06ed40 msgr2=0x7f2d3c06f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:26:50.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.093+0000 7f2d3959a700 1 --2- 172.21.15.31:0/929086075 >> v2:172.21.15.62:3300/0 conn(0x7f2d3c06ed40 0x7f2d3c06f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:50.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.093+0000 7f2d3959a700 1 -- 172.21.15.31:0/929086075 >> v1:172.21.15.62:6789/0 conn(0x7f2d3c156860 legacy=0x7f2d3c0ae300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:50.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.093+0000 7f2d3959a700 1 -- 172.21.15.31:0/929086075 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d3c156d90 con 0x7f2d3c153d70 2023-10-06T02:26:50.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.093+0000 7f2d3959a700 1 -- 172.21.15.31:0/929086075 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7f2d2c001470 con 0x7f2d3c153d70 2023-10-06T02:26:50.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.093+0000 7f2d3959a700 1 -- 172.21.15.31:0/929086075 >> v1:172.21.15.31:6789/0 conn(0x7f2d3c153d70 legacy=0x7f2d3c0acba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:50.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.093+0000 7f2d3959a700 1 --2- 172.21.15.31:0/929086075 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2d24003ab0 0x7f2d24003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:50.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.093+0000 7f2d3959a700 1 --2- 172.21.15.31:0/929086075 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d24004810 0x7f2d24004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:50.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.093+0000 7f2d3959a700 1 --2- 172.21.15.31:0/929086075 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d24005210 0x7f2d24010070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:50.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.093+0000 7f2d3959a700 1 -- 172.21.15.31:0/929086075 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d3c0aea10 con 0x7f2d24005210 2023-10-06T02:26:50.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.093+0000 7f2d3959a700 1 -- 172.21.15.31:0/929086075 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d240105b0 con 0x7f2d24004810 2023-10-06T02:26:50.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.093+0000 7f2d3959a700 1 -- 172.21.15.31:0/929086075 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d24010720 con 0x7f2d24003ab0 2023-10-06T02:26:50.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.093+0000 7f2d3ad9d700 1 --2- 172.21.15.31:0/929086075 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d24005210 0x7f2d24010070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:50.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.094+0000 7f2d3a59c700 1 --2- 172.21.15.31:0/929086075 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2d24003ab0 0x7f2d24003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:50.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.094+0000 7f2d39d9b700 1 --2- 172.21.15.31:0/929086075 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d24004810 0x7f2d24004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:50.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.094+0000 7f2d3ad9d700 1 -- 172.21.15.31:0/929086075 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2d24003ab0 msgr2=0x7f2d24003ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:26:50.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.094+0000 7f2d3ad9d700 1 --2- 172.21.15.31:0/929086075 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2d24003ab0 0x7f2d24003ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:50.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.094+0000 7f2d3ad9d700 1 -- 172.21.15.31:0/929086075 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d24004810 msgr2=0x7f2d24004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:26:50.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.094+0000 7f2d3ad9d700 1 --2- 172.21.15.31:0/929086075 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d24004810 0x7f2d24004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:50.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.094+0000 7f2d3ad9d700 1 -- 172.21.15.31:0/929086075 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d24003570 con 0x7f2d24005210 2023-10-06T02:26:50.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.094+0000 7f2d3ad9d700 1 -- 172.21.15.31:0/929086075 reap_dead start 2023-10-06T02:26:50.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.094+0000 7f2d3ad9d700 1 --2- 172.21.15.31:0/929086075 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d24005210 0x7f2d24010070 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f2d3400e210 tx=0x7f2d3400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc3bccfec333311b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:50.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.094+0000 7f2d3959a700 1 -- 172.21.15.31:0/929086075 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d3400eac0 con 0x7f2d24005210 2023-10-06T02:26:50.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.094+0000 7f2d3959a700 1 -- 172.21.15.31:0/929086075 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d3400f040 con 0x7f2d24005210 2023-10-06T02:26:50.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.094+0000 7f2d3959a700 1 -- 172.21.15.31:0/929086075 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d3400ee70 con 0x7f2d24005210 2023-10-06T02:26:50.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.095+0000 7f2d3b59e700 1 -- 172.21.15.31:0/929086075 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d24005210 msgr2=0x7f2d24010070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:50.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.095+0000 7f2d3b59e700 1 --2- 172.21.15.31:0/929086075 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d24005210 0x7f2d24010070 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f2d3400e210 tx=0x7f2d3400b040 comp rx=0 tx=0).stop 2023-10-06T02:26:50.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.095+0000 7f2d3b59e700 1 -- 172.21.15.31:0/929086075 shutdown_connections 2023-10-06T02:26:50.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.095+0000 7f2d3b59e700 1 --2- 172.21.15.31:0/929086075 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d24005210 0x7f2d24010070 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:50.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.095+0000 7f2d3b59e700 1 -- 172.21.15.31:0/929086075 >> 172.21.15.31:0/929086075 conn(0x7f2d3c06d170 msgr2=0x7f2d3c0cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:50.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.095+0000 7f2d3b59e700 1 -- 172.21.15.31:0/929086075 shutdown_connections 2023-10-06T02:26:50.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.095+0000 7f2d3b59e700 1 -- 172.21.15.31:0/929086075 wait complete. 2023-10-06T02:26:50.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.096+0000 7f2d3b59e700 1 Processor -- start 2023-10-06T02:26:50.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.097+0000 7f2d3b59e700 1 -- start start 2023-10-06T02:26:50.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.097+0000 7f2d3b59e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d24005210 0x7f2d3c13b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:50.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.097+0000 7f2d3b59e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d3c0c8f20 0x7f2d3c1579f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:50.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.097+0000 7f2d3b59e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2d3c0c9390 0x7f2d3c157f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:50.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.097+0000 7f2d3b59e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d3c13bc70 con 0x7f2d24005210 2023-10-06T02:26:50.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.097+0000 7f2d3b59e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d3c1658a0 con 0x7f2d3c0c8f20 2023-10-06T02:26:50.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.097+0000 7f2d3b59e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d3c166700 con 0x7f2d3c0c9390 2023-10-06T02:26:50.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.098+0000 7f2d3a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d24005210 0x7f2d3c13b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:50.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.098+0000 7f2d3a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d24005210 0x7f2d3c13b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34624/0 (socket says 172.21.15.31:34624) 2023-10-06T02:26:50.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.098+0000 7f2d3a59c700 1 -- 172.21.15.31:0/3250564124 learned_addr learned my addr 172.21.15.31:0/3250564124 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:26:50.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.098+0000 7f2d3ad9d700 1 --2- 172.21.15.31:0/3250564124 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2d3c0c9390 0x7f2d3c157f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:50.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.098+0000 7f2d39d9b700 1 --2- 172.21.15.31:0/3250564124 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d3c0c8f20 0x7f2d3c1579f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:50.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.098+0000 7f2d3a59c700 1 -- 172.21.15.31:0/3250564124 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2d3c0c9390 msgr2=0x7f2d3c157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:50.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.098+0000 7f2d3a59c700 1 --2- 172.21.15.31:0/3250564124 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2d3c0c9390 0x7f2d3c157f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:50.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.098+0000 7f2d3a59c700 1 -- 172.21.15.31:0/3250564124 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d3c0c8f20 msgr2=0x7f2d3c1579f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:50.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.098+0000 7f2d3a59c700 1 --2- 172.21.15.31:0/3250564124 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d3c0c8f20 0x7f2d3c1579f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:50.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.098+0000 7f2d3a59c700 1 -- 172.21.15.31:0/3250564124 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d3c165af0 con 0x7f2d24005210 2023-10-06T02:26:50.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.098+0000 7f2d39d9b700 1 --2- 172.21.15.31:0/3250564124 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d3c0c8f20 0x7f2d3c1579f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:26:50.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.098+0000 7f2d3a59c700 1 --2- 172.21.15.31:0/3250564124 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d24005210 0x7f2d3c13b730 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c0073c0 tx=0x7f2d3c157910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:50.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.099+0000 7f2d2a7fc700 1 -- 172.21.15.31:0/3250564124 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d2c00f8d0 con 0x7f2d24005210 2023-10-06T02:26:50.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.099+0000 7f2d2a7fc700 1 -- 172.21.15.31:0/3250564124 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d2c00fa70 con 0x7f2d24005210 2023-10-06T02:26:50.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.099+0000 7f2d2a7fc700 1 -- 172.21.15.31:0/3250564124 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d2c00fe30 con 0x7f2d24005210 2023-10-06T02:26:50.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.099+0000 7f2d3b59e700 1 -- 172.21.15.31:0/3250564124 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d3c165dd0 con 0x7f2d24005210 2023-10-06T02:26:50.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.099+0000 7f2d3b59e700 1 -- 172.21.15.31:0/3250564124 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d3c166390 con 0x7f2d24005210 2023-10-06T02:26:50.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.100+0000 7f2d3b59e700 1 -- 172.21.15.31:0/3250564124 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d3c04c3c0 con 0x7f2d24005210 2023-10-06T02:26:50.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.101+0000 7f2d2a7fc700 1 -- 172.21.15.31:0/3250564124 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 9) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f2d2c013600 con 0x7f2d24005210 2023-10-06T02:26:50.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.102+0000 7f2d2a7fc700 1 --2- 172.21.15.31:0/3250564124 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f2d1c0670e0 0x7f2d1c069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:50.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.102+0000 7f2d2a7fc700 1 -- 172.21.15.31:0/3250564124 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6203+0+0 (secure 0 0 0) 0x7f2d2c084c50 con 0x7f2d24005210 2023-10-06T02:26:50.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.102+0000 7f2d39d9b700 1 --2- 172.21.15.31:0/3250564124 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f2d1c0670e0 0x7f2d1c069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:50.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.103+0000 7f2d39d9b700 1 --2- 172.21.15.31:0/3250564124 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f2d1c0670e0 0x7f2d1c069590 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c151b60 tx=0x7f2d30009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:50.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.106+0000 7f2d2a7fc700 1 -- 172.21.15.31:0/3250564124 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2d2c0519b0 con 0x7f2d24005210 2023-10-06T02:26:50.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.354+0000 7f2d3b59e700 1 -- 172.21.15.31:0/3250564124 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2d3c157070 con 0x7f2d24005210 2023-10-06T02:26:50.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.354+0000 7f2d2a7fc700 1 -- 172.21.15.31:0/3250564124 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v37) v1 ==== 53+0+73 (secure 0 0 0) 0x7f2d2c054fd0 con 0x7f2d24005210 2023-10-06T02:26:50.355 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T02:26:50.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.356+0000 7f2d3b59e700 1 -- 172.21.15.31:0/3250564124 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f2d1c0670e0 msgr2=0x7f2d1c069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:50.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.356+0000 7f2d3b59e700 1 --2- 172.21.15.31:0/3250564124 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f2d1c0670e0 0x7f2d1c069590 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c151b60 tx=0x7f2d30009040 comp rx=0 tx=0).stop 2023-10-06T02:26:50.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.356+0000 7f2d3b59e700 1 -- 172.21.15.31:0/3250564124 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d24005210 msgr2=0x7f2d3c13b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:50.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.356+0000 7f2d3b59e700 1 --2- 172.21.15.31:0/3250564124 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d24005210 0x7f2d3c13b730 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c0073c0 tx=0x7f2d3c157910 comp rx=0 tx=0).stop 2023-10-06T02:26:50.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.356+0000 7f2d3b59e700 1 -- 172.21.15.31:0/3250564124 shutdown_connections 2023-10-06T02:26:50.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.356+0000 7f2d3b59e700 1 --2- 172.21.15.31:0/3250564124 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f2d1c0670e0 0x7f2d1c069590 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:50.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.356+0000 7f2d3b59e700 1 --2- 172.21.15.31:0/3250564124 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2d3c0c9390 0x7f2d3c157f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:50.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.356+0000 7f2d3b59e700 1 --2- 172.21.15.31:0/3250564124 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d24005210 0x7f2d3c13b730 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:50.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.356+0000 7f2d3b59e700 1 --2- 172.21.15.31:0/3250564124 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d3c0c8f20 0x7f2d3c1579f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:50.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.356+0000 7f2d3b59e700 1 -- 172.21.15.31:0/3250564124 >> 172.21.15.31:0/3250564124 conn(0x7f2d3c06d170 msgr2=0x7f2d3c159610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:50.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.356+0000 7f2d3b59e700 1 -- 172.21.15.31:0/3250564124 shutdown_connections 2023-10-06T02:26:50.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:50.356+0000 7f2d3b59e700 1 -- 172.21.15.31:0/3250564124 wait complete. 2023-10-06T02:26:50.374 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T02:26:50.374 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T02:26:50.374 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi031.front.sepia.ceph.com 2023-10-06T02:26:50.374 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T02:26:50.374 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T02:26:50.374 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T02:26:50.374 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T02:26:50.375 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T02:26:50.375 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T02:26:50.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:26:50.375 DEBUG:teuthology.orchestra.run.smithi031:> ip addr 2023-10-06T02:26:50.412 INFO:teuthology.orchestra.run.smithi031.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T02:26:50.464 INFO:teuthology.orchestra.run.smithi031.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T02:26:50.464 INFO:teuthology.orchestra.run.smithi031.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T02:26:50.464 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:26:50.465 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 ::1/128 scope host 2023-10-06T02:26:50.465 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:26:50.465 INFO:teuthology.orchestra.run.smithi031.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T02:26:50.465 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cc brd ff:ff:ff:ff:ff:ff 2023-10-06T02:26:50.465 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f0 2023-10-06T02:26:50.466 INFO:teuthology.orchestra.run.smithi031.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T02:26:50.466 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cd brd ff:ff:ff:ff:ff:ff 2023-10-06T02:26:50.466 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f1 2023-10-06T02:26:50.466 INFO:teuthology.orchestra.run.smithi031.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T02:26:50.466 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:ba brd ff:ff:ff:ff:ff:ff 2023-10-06T02:26:50.466 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f0 2023-10-06T02:26:50.467 INFO:teuthology.orchestra.run.smithi031.stdout: inet 172.21.15.31/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T02:26:50.467 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft 42046sec preferred_lft 42046sec 2023-10-06T02:26:50.467 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::ae1f:6bff:fea5:caba/64 scope link 2023-10-06T02:26:50.467 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:26:50.467 INFO:teuthology.orchestra.run.smithi031.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T02:26:50.467 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:bb brd ff:ff:ff:ff:ff:ff 2023-10-06T02:26:50.468 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f1 2023-10-06T02:26:50.468 INFO:teuthology.orchestra.run.smithi031.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T02:26:50.468 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 16:d7:88:2d:d0:c2 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:26:50.468 INFO:teuthology.orchestra.run.smithi031.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T02:26:50.468 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:26:50.468 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::408:52ff:fe3c:ba8f/64 scope link 2023-10-06T02:26:50.468 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:26:50.469 INFO:teuthology.orchestra.run.smithi031.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T02:26:50.469 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 16:d7:88:2d:d0:c2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T02:26:50.469 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::14d7:88ff:fe2d:d0c2/64 scope link 2023-10-06T02:26:50.469 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:26:50.469 INFO:teuthology.orchestra.run.smithi031.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T02:26:50.469 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 82:16:f1:00:cf:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T02:26:50.470 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::8016:f1ff:fe00:cf2f/64 scope link 2023-10-06T02:26:50.470 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:26:50.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:26:50.489 DEBUG:teuthology.orchestra.run.smithi031:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T02:26:50.550 INFO:teuthology.orchestra.run.smithi031.stdout:1 2023-10-06T02:26:50.553 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:26:50.553 DEBUG:teuthology.orchestra.run.smithi031:> route 2023-10-06T02:26:50.591 INFO:teuthology.orchestra.run.smithi031.stdout:Kernel IP routing table 2023-10-06T02:26:50.632 INFO:teuthology.orchestra.run.smithi031.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T02:26:50.632 INFO:teuthology.orchestra.run.smithi031.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T02:26:50.632 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T02:26:50.632 INFO:teuthology.orchestra.run.smithi031.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T02:26:50.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:26:50.633 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2023-10-06T02:26:50.633 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2023-10-06T02:26:50.633 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T02:26:50.633 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T02:26:50.634 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T02:26:50.634 DEBUG:teuthology.orchestra.run.smithi031:> ') 2023-10-06T02:26:50.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:26:50.846 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2023-10-06T02:26:50.883 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T02:26:50.930 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T02:26:50.931 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T02:26:50.931 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:26:50.931 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set brx.0 up 2023-10-06T02:26:50.994 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T02:26:51.279 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T02:26:51.317 INFO:teuthology.orchestra.run.smithi031.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T02:26:51.319 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T02:26:51.319 DEBUG:teuthology.orchestra.run.smithi031:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T02:26:51.393 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-06T02:26:51.397 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T02:26:51.397 DEBUG:teuthology.orchestra.run.smithi031:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T02:26:51.525 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T02:26:51.590 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T02:26:51.591 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T02:26:51.591 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T02:26:51.591 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.62:3300,172.21.15.62:3301" 2023-10-06T02:26:51.591 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: trying mount with new device syntax: 0@d6fcf512-874b-4259-b90a-30ae070f00a9.cephfs=/ 2023-10-06T02:26:51.591 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.31:3300/172.21.15.62:3300/172.21.15.62:3301" will pass to kernel 2023-10-06T02:26:51.592 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T02:26:51.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:26:51.592 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T02:26:51.654 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T02:26:51.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:51.896+0000 7f284ab9d700 1 Processor -- start 2023-10-06T02:26:51.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:51.896+0000 7f284ab9d700 1 -- start start 2023-10-06T02:26:51.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:51.897+0000 7f284ab9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f284415de50 0x7f28440cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:51.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:51.897+0000 7f284ab9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28440cf3c0 con 0x7f28440cef40 2023-10-06T02:26:51.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:51.897+0000 7f284ab9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f284415afd0 con 0x7f2844153c30 2023-10-06T02:26:51.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:51.897+0000 7f284ab9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f284415b2b0 con 0x7f284415de50 2023-10-06T02:26:51.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:51.897+0000 7f284939a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f284415de50 0x7f28440cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:51.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:51.898+0000 7f284939a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f284415de50 0x7f28440cea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34630/0 (socket says 172.21.15.31:34630) 2023-10-06T02:26:51.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:51.898+0000 7f284939a700 1 -- 172.21.15.31:0/2937748354 learned_addr learned my addr 172.21.15.31:0/2937748354 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:26:51.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:51.898+0000 7f284939a700 1 -- 172.21.15.31:0/2937748354 >> v1:172.21.15.62:6789/0 conn(0x7f2844153c30 legacy=0x7f284415a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:51.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:51.898+0000 7f284939a700 1 -- 172.21.15.31:0/2937748354 >> v1:172.21.15.31:6789/0 conn(0x7f28440cef40 legacy=0x7f284415c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:51.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:51.898+0000 7f284939a700 1 -- 172.21.15.31:0/2937748354 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f284415b590 con 0x7f284415de50 2023-10-06T02:26:51.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:51.898+0000 7f284939a700 1 --2- 172.21.15.31:0/2937748354 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f284415de50 0x7f28440cea00 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f28340099b0 tx=0x7f2834009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12e0e9fc0cbfebdc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:51.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:51.899+0000 7f2848b99700 1 -- 172.21.15.31:0/2937748354 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f283400b860 con 0x7f284415de50 2023-10-06T02:26:51.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:51.899+0000 7f2848b99700 1 -- 172.21.15.31:0/2937748354 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f283400b9c0 con 0x7f284415de50 2023-10-06T02:26:51.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:51.899+0000 7f2848b99700 1 -- 172.21.15.31:0/2937748354 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f283400bd60 con 0x7f284415de50 2023-10-06T02:26:51.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:51.899+0000 7f284ab9d700 1 -- 172.21.15.31:0/2937748354 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f284415de50 msgr2=0x7f28440cea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:51.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:51.899+0000 7f284ab9d700 1 --2- 172.21.15.31:0/2937748354 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f284415de50 0x7f28440cea00 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f28340099b0 tx=0x7f2834009250 comp rx=0 tx=0).stop 2023-10-06T02:26:51.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:51.899+0000 7f284ab9d700 1 -- 172.21.15.31:0/2937748354 shutdown_connections 2023-10-06T02:26:51.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:51.899+0000 7f284ab9d700 1 --2- 172.21.15.31:0/2937748354 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f284415de50 0x7f28440cea00 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:51.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:51.899+0000 7f284ab9d700 1 -- 172.21.15.31:0/2937748354 >> 172.21.15.31:0/2937748354 conn(0x7f28440703a0 msgr2=0x7f28440707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:51.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:51.899+0000 7f284ab9d700 1 -- 172.21.15.31:0/2937748354 shutdown_connections 2023-10-06T02:26:51.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:51.899+0000 7f284ab9d700 1 -- 172.21.15.31:0/2937748354 wait complete. 2023-10-06T02:26:51.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:51.901+0000 7f284ab9d700 1 Processor -- start 2023-10-06T02:26:51.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:51.901+0000 7f284ab9d700 1 -- start start 2023-10-06T02:26:51.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:51.902+0000 7f284ab9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28440cef40 0x7f28440b3500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:51.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:51.902+0000 7f284ab9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2844153c30 0x7f28440ccac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:51.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:51.902+0000 7f284ab9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f284415de50 0x7f28440d06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:51.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:51.902+0000 7f284ab9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28440cd000 con 0x7f28440cef40 2023-10-06T02:26:51.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:51.902+0000 7f284ab9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28440b5120 con 0x7f2844153c30 2023-10-06T02:26:51.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:51.902+0000 7f284ab9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28440b5400 con 0x7f284415de50 2023-10-06T02:26:51.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:51.902+0000 7f2849b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28440cef40 0x7f28440b3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:51.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:51.902+0000 7f2849b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28440cef40 0x7f28440b3500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34632/0 (socket says 172.21.15.31:34632) 2023-10-06T02:26:51.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:51.902+0000 7f2849b9b700 1 -- 172.21.15.31:0/2555599394 learned_addr learned my addr 172.21.15.31:0/2555599394 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:26:51.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:51.902+0000 7f284939a700 1 --2- 172.21.15.31:0/2555599394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2844153c30 0x7f28440ccac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:51.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:51.902+0000 7f284a39c700 1 --2- 172.21.15.31:0/2555599394 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f284415de50 0x7f28440d06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:51.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:51.903+0000 7f2849b9b700 1 -- 172.21.15.31:0/2555599394 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f284415de50 msgr2=0x7f28440d06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:51.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:51.903+0000 7f2849b9b700 1 --2- 172.21.15.31:0/2555599394 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f284415de50 0x7f28440d06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:51.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:51.903+0000 7f2849b9b700 1 -- 172.21.15.31:0/2555599394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2844153c30 msgr2=0x7f28440ccac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:51.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:51.903+0000 7f2849b9b700 1 --2- 172.21.15.31:0/2555599394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2844153c30 0x7f28440ccac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:51.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:51.903+0000 7f2849b9b700 1 -- 172.21.15.31:0/2555599394 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28340094f0 con 0x7f28440cef40 2023-10-06T02:26:51.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:51.903+0000 7f2849b9b700 1 --2- 172.21.15.31:0/2555599394 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28440cef40 0x7f28440b3500 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f284000c590 tx=0x7f284000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:51.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:51.903+0000 7f2839ffb700 1 -- 172.21.15.31:0/2555599394 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2840011590 con 0x7f28440cef40 2023-10-06T02:26:51.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:51.903+0000 7f2839ffb700 1 -- 172.21.15.31:0/2555599394 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2840011710 con 0x7f28440cef40 2023-10-06T02:26:51.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:51.903+0000 7f284ab9d700 1 -- 172.21.15.31:0/2555599394 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28440b56e0 con 0x7f28440cef40 2023-10-06T02:26:51.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:51.903+0000 7f2839ffb700 1 -- 172.21.15.31:0/2555599394 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2840011ad0 con 0x7f28440cef40 2023-10-06T02:26:51.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:51.903+0000 7f284ab9d700 1 -- 172.21.15.31:0/2555599394 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28440b4df0 con 0x7f28440cef40 2023-10-06T02:26:51.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:51.904+0000 7f284ab9d700 1 -- 172.21.15.31:0/2555599394 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28440b5c00 con 0x7f28440cef40 2023-10-06T02:26:51.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:51.905+0000 7f2839ffb700 1 -- 172.21.15.31:0/2555599394 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 9) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f284001b3b0 con 0x7f28440cef40 2023-10-06T02:26:51.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:51.906+0000 7f2839ffb700 1 --2- 172.21.15.31:0/2555599394 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f28280670e0 0x7f2828069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:51.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:51.907+0000 7f2839ffb700 1 -- 172.21.15.31:0/2555599394 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6203+0+0 (secure 0 0 0) 0x7f2840083f40 con 0x7f28440cef40 2023-10-06T02:26:51.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:51.907+0000 7f284939a700 1 --2- 172.21.15.31:0/2555599394 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f28280670e0 0x7f2828069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:51.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:51.907+0000 7f284939a700 1 --2- 172.21.15.31:0/2555599394 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f28280670e0 0x7f2828069590 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f28340099b0 tx=0x7f2834017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:51.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:51.911+0000 7f2839ffb700 1 -- 172.21.15.31:0/2555599394 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f28440b5c00 con 0x7f28440cef40 2023-10-06T02:26:52.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:52.160+0000 7f284ab9d700 1 -- 172.21.15.31:0/2555599394 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f284404c3c0 con 0x7f28440cef40 2023-10-06T02:26:52.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:52.161+0000 7f2839ffb700 1 -- 172.21.15.31:0/2555599394 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v37) v1 ==== 53+0+73 (secure 0 0 0) 0x7f2840017070 con 0x7f28440cef40 2023-10-06T02:26:52.161 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T02:26:52.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:52.163+0000 7f284ab9d700 1 -- 172.21.15.31:0/2555599394 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f28280670e0 msgr2=0x7f2828069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:52.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:52.163+0000 7f284ab9d700 1 --2- 172.21.15.31:0/2555599394 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f28280670e0 0x7f2828069590 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f28340099b0 tx=0x7f2834017000 comp rx=0 tx=0).stop 2023-10-06T02:26:52.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:52.163+0000 7f284ab9d700 1 -- 172.21.15.31:0/2555599394 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28440cef40 msgr2=0x7f28440b3500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:52.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:52.163+0000 7f284ab9d700 1 --2- 172.21.15.31:0/2555599394 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28440cef40 0x7f28440b3500 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f284000c590 tx=0x7f284000c8a0 comp rx=0 tx=0).stop 2023-10-06T02:26:52.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:52.163+0000 7f284ab9d700 1 -- 172.21.15.31:0/2555599394 shutdown_connections 2023-10-06T02:26:52.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:52.163+0000 7f284ab9d700 1 --2- 172.21.15.31:0/2555599394 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f28280670e0 0x7f2828069590 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:52.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:52.163+0000 7f284ab9d700 1 --2- 172.21.15.31:0/2555599394 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f284415de50 0x7f28440d06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:52.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:52.163+0000 7f284ab9d700 1 --2- 172.21.15.31:0/2555599394 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28440cef40 0x7f28440b3500 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:52.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:52.163+0000 7f284ab9d700 1 --2- 172.21.15.31:0/2555599394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2844153c30 0x7f28440ccac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:52.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:52.163+0000 7f284ab9d700 1 -- 172.21.15.31:0/2555599394 >> 172.21.15.31:0/2555599394 conn(0x7f28440703a0 msgr2=0x7f28441511b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:52.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:52.164+0000 7f284ab9d700 1 -- 172.21.15.31:0/2555599394 shutdown_connections 2023-10-06T02:26:52.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:52.164+0000 7f284ab9d700 1 -- 172.21.15.31:0/2555599394 wait complete. 2023-10-06T02:26:52.181 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T02:26:52.181 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T02:26:52.181 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi062.front.sepia.ceph.com 2023-10-06T02:26:52.181 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-06T02:26:52.181 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-06T02:26:52.182 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T02:26:52.182 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T02:26:52.182 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T02:26:52.182 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T02:26:52.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:26:52.182 DEBUG:teuthology.orchestra.run.smithi062:> ip addr 2023-10-06T02:26:52.225 INFO:teuthology.orchestra.run.smithi062.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T02:26:52.225 INFO:teuthology.orchestra.run.smithi062.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T02:26:52.225 INFO:teuthology.orchestra.run.smithi062.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T02:26:52.225 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:26:52.225 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 ::1/128 scope host 2023-10-06T02:26:52.226 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:26:52.226 INFO:teuthology.orchestra.run.smithi062.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T02:26:52.226 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:14 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:26:52.226 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f0 2023-10-06T02:26:52.226 INFO:teuthology.orchestra.run.smithi062.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T02:26:52.226 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:15 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:26:52.227 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f1 2023-10-06T02:26:52.227 INFO:teuthology.orchestra.run.smithi062.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T02:26:52.227 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c2 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:26:52.227 INFO:teuthology.orchestra.run.smithi062.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T02:26:52.227 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c3 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:26:52.227 INFO:teuthology.orchestra.run.smithi062.stdout: inet 172.21.15.62/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T02:26:52.227 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft 42037sec preferred_lft 42037sec 2023-10-06T02:26:52.228 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::ec4:7aff:febb:d1c3/64 scope link 2023-10-06T02:26:52.228 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:26:52.228 INFO:teuthology.orchestra.run.smithi062.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T02:26:52.228 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether ae:42:a8:dd:f7:9b brd ff:ff:ff:ff:ff:ff 2023-10-06T02:26:52.228 INFO:teuthology.orchestra.run.smithi062.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T02:26:52.228 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:26:52.229 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::1098:a4ff:fe0d:389/64 scope link 2023-10-06T02:26:52.229 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:26:52.229 INFO:teuthology.orchestra.run.smithi062.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T02:26:52.229 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether fe:60:2b:8e:af:28 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T02:26:52.229 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::fc60:2bff:fe8e:af28/64 scope link 2023-10-06T02:26:52.229 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:26:52.230 INFO:teuthology.orchestra.run.smithi062.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T02:26:52.230 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether ae:42:a8:dd:f7:9b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T02:26:52.230 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::ac42:a8ff:fedd:f79b/64 scope link 2023-10-06T02:26:52.230 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:26:52.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:26:52.231 DEBUG:teuthology.orchestra.run.smithi062:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T02:26:52.328 INFO:teuthology.orchestra.run.smithi062.stdout:1 2023-10-06T02:26:52.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:26:52.331 DEBUG:teuthology.orchestra.run.smithi062:> route 2023-10-06T02:26:52.372 INFO:teuthology.orchestra.run.smithi062.stdout:Kernel IP routing table 2023-10-06T02:26:52.372 INFO:teuthology.orchestra.run.smithi062.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T02:26:52.372 INFO:teuthology.orchestra.run.smithi062.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T02:26:52.372 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T02:26:52.372 INFO:teuthology.orchestra.run.smithi062.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T02:26:52.372 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:26:52.373 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2023-10-06T02:26:52.373 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2023-10-06T02:26:52.373 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T02:26:52.373 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T02:26:52.373 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T02:26:52.373 DEBUG:teuthology.orchestra.run.smithi062:> ') 2023-10-06T02:26:52.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:26:52.571 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2023-10-06T02:26:52.606 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-06T02:26:52.607 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T02:26:52.607 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T02:26:52.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:26:52.608 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set brx.0 up 2023-10-06T02:26:52.710 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T02:26:52.710 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-06T02:26:52.746 INFO:teuthology.orchestra.run.smithi062.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-06T02:26:52.746 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T02:26:52.746 DEBUG:teuthology.orchestra.run.smithi062:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-06T02:26:52.821 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-06T02:26:52.821 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T02:26:52.822 DEBUG:teuthology.orchestra.run.smithi062:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T02:26:52.952 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T02:26:52.953 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T02:26:52.981 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T02:26:52.982 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T02:26:52.982 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.62:3300,172.21.15.62:3301" 2023-10-06T02:26:52.982 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: trying mount with new device syntax: 1@d6fcf512-874b-4259-b90a-30ae070f00a9.cephfs=/ 2023-10-06T02:26:52.982 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.31:3300/172.21.15.62:3300/172.21.15.62:3301" will pass to kernel 2023-10-06T02:26:52.986 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T02:26:52.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:26:52.986 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-06T02:26:53.047 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-10-06T02:26:53.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:53.251+0000 7f23e519d700 1 Processor -- start 2023-10-06T02:26:53.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:53.252+0000 7f23e519d700 1 -- start start 2023-10-06T02:26:53.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:53.253+0000 7f23e519d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f23e0157380 0x7f23e01585f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:53.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:53.253+0000 7f23e519d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f23e0157750 0x7f23e00ae080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:53.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:53.253+0000 7f23e519d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23e00c8680 con 0x7f23e015b1a0 2023-10-06T02:26:53.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:53.253+0000 7f23e519d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23e006a7f0 con 0x7f23e0157750 2023-10-06T02:26:53.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:53.253+0000 7f23e519d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23e00c7270 con 0x7f23e0157380 2023-10-06T02:26:53.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:53.253+0000 7f23e499c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f23e0157750 0x7f23e00ae080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:53.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:53.253+0000 7f23e499c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f23e0157750 0x7f23e00ae080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34636/0 (socket says 172.21.15.31:34636) 2023-10-06T02:26:53.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:53.253+0000 7f23e499c700 1 -- 172.21.15.31:0/2029220139 learned_addr learned my addr 172.21.15.31:0/2029220139 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:26:53.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:53.253+0000 7f23dffff700 1 --2- 172.21.15.31:0/2029220139 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f23e0157380 0x7f23e01585f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:53.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:53.254+0000 7f23e499c700 1 -- 172.21.15.31:0/2029220139 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f23e0157380 msgr2=0x7f23e01585f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:26:53.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:53.254+0000 7f23e499c700 1 --2- 172.21.15.31:0/2029220139 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f23e0157380 0x7f23e01585f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:53.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:53.254+0000 7f23e499c700 1 -- 172.21.15.31:0/2029220139 >> v1:172.21.15.31:6789/0 conn(0x7f23e015b1a0 legacy=0x7f23e00c6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:53.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:53.254+0000 7f23e499c700 1 -- 172.21.15.31:0/2029220139 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23e0158b30 con 0x7f23e0157750 2023-10-06T02:26:53.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:53.254+0000 7f23e499c700 1 --2- 172.21.15.31:0/2029220139 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23e0157750 0x7f23e00ae080 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f23d8009f20 tx=0x7f23d8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12df9ec062e0ba28 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:53.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:53.254+0000 7f23deffd700 1 -- 172.21.15.31:0/2029220139 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d800e070 con 0x7f23e0157750 2023-10-06T02:26:53.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:53.254+0000 7f23deffd700 1 -- 172.21.15.31:0/2029220139 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23d800b9d0 con 0x7f23e0157750 2023-10-06T02:26:53.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:53.254+0000 7f23deffd700 1 -- 172.21.15.31:0/2029220139 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d80092e0 con 0x7f23e0157750 2023-10-06T02:26:53.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:53.255+0000 7f23e519d700 1 -- 172.21.15.31:0/2029220139 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23e0157750 msgr2=0x7f23e00ae080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:53.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:53.255+0000 7f23e519d700 1 --2- 172.21.15.31:0/2029220139 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23e0157750 0x7f23e00ae080 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f23d8009f20 tx=0x7f23d8009840 comp rx=0 tx=0).stop 2023-10-06T02:26:53.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:53.255+0000 7f23e519d700 1 -- 172.21.15.31:0/2029220139 shutdown_connections 2023-10-06T02:26:53.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:53.255+0000 7f23e519d700 1 --2- 172.21.15.31:0/2029220139 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23e0157750 0x7f23e00ae080 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:53.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:53.255+0000 7f23e519d700 1 --2- 172.21.15.31:0/2029220139 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f23e0157380 0x7f23e01585f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:53.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:53.255+0000 7f23e519d700 1 -- 172.21.15.31:0/2029220139 >> 172.21.15.31:0/2029220139 conn(0x7f23e006c6a0 msgr2=0x7f23e006caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:53.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:53.255+0000 7f23e519d700 1 -- 172.21.15.31:0/2029220139 shutdown_connections 2023-10-06T02:26:53.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:53.255+0000 7f23e519d700 1 -- 172.21.15.31:0/2029220139 wait complete. 2023-10-06T02:26:53.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:53.256+0000 7f23e519d700 1 Processor -- start 2023-10-06T02:26:53.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:53.257+0000 7f23e519d700 1 -- start start 2023-10-06T02:26:53.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:53.257+0000 7f23e519d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23e0157380 0x7f23e00b0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:53.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:53.257+0000 7f23e519d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f23e0157750 0x7f23e0159f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:53.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:53.257+0000 7f23e519d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23e015b1a0 0x7f23e00ca170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:53.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:53.257+0000 7f23e519d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e00b12a0 con 0x7f23e0157380 2023-10-06T02:26:53.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:53.257+0000 7f23e519d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e00b08d0 con 0x7f23e015b1a0 2023-10-06T02:26:53.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:53.257+0000 7f23e519d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23e00cae30 con 0x7f23e0157750 2023-10-06T02:26:53.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:53.258+0000 7f23dffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23e0157380 0x7f23e00b0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:53.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:53.258+0000 7f23dffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23e0157380 0x7f23e00b0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34652/0 (socket says 172.21.15.31:34652) 2023-10-06T02:26:53.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:53.258+0000 7f23dffff700 1 -- 172.21.15.31:0/3006720163 learned_addr learned my addr 172.21.15.31:0/3006720163 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:26:53.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:53.258+0000 7f23df7fe700 1 --2- 172.21.15.31:0/3006720163 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f23e0157750 0x7f23e0159f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:53.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:53.258+0000 7f23e499c700 1 --2- 172.21.15.31:0/3006720163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23e015b1a0 0x7f23e00ca170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:53.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:53.258+0000 7f23dffff700 1 -- 172.21.15.31:0/3006720163 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f23e0157750 msgr2=0x7f23e0159f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:53.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:53.258+0000 7f23dffff700 1 --2- 172.21.15.31:0/3006720163 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f23e0157750 0x7f23e0159f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:53.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:53.258+0000 7f23dffff700 1 -- 172.21.15.31:0/3006720163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23e015b1a0 msgr2=0x7f23e00ca170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:53.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:53.258+0000 7f23dffff700 1 --2- 172.21.15.31:0/3006720163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23e015b1a0 0x7f23e00ca170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:53.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:53.258+0000 7f23dffff700 1 -- 172.21.15.31:0/3006720163 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23d8009ae0 con 0x7f23e0157380 2023-10-06T02:26:53.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:53.258+0000 7f23dffff700 1 --2- 172.21.15.31:0/3006720163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23e0157380 0x7f23e00b0d60 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f23d000ec80 tx=0x7f23d00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:53.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:53.259+0000 7f23c7fff700 1 -- 172.21.15.31:0/3006720163 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d000c9d0 con 0x7f23e0157380 2023-10-06T02:26:53.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:53.259+0000 7f23c7fff700 1 -- 172.21.15.31:0/3006720163 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23d000cb30 con 0x7f23e0157380 2023-10-06T02:26:53.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:53.259+0000 7f23e519d700 1 -- 172.21.15.31:0/3006720163 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23e00affa0 con 0x7f23e0157380 2023-10-06T02:26:53.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:53.259+0000 7f23c7fff700 1 -- 172.21.15.31:0/3006720163 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d0013470 con 0x7f23e0157380 2023-10-06T02:26:53.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:53.259+0000 7f23e519d700 1 -- 172.21.15.31:0/3006720163 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23e00b0560 con 0x7f23e0157380 2023-10-06T02:26:53.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:53.260+0000 7f23e519d700 1 -- 172.21.15.31:0/3006720163 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23e00b0bb0 con 0x7f23e0157380 2023-10-06T02:26:53.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:53.260+0000 7f23c7fff700 1 -- 172.21.15.31:0/3006720163 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 9) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f23d00137e0 con 0x7f23e0157380 2023-10-06T02:26:53.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:53.261+0000 7f23c7fff700 1 --2- 172.21.15.31:0/3006720163 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f23cc0670e0 0x7f23cc069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:53.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:53.261+0000 7f23c7fff700 1 -- 172.21.15.31:0/3006720163 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6203+0+0 (secure 0 0 0) 0x7f23d0084ea0 con 0x7f23e0157380 2023-10-06T02:26:53.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:53.262+0000 7f23df7fe700 1 --2- 172.21.15.31:0/3006720163 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f23cc0670e0 0x7f23cc069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:53.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:53.262+0000 7f23df7fe700 1 --2- 172.21.15.31:0/3006720163 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f23cc0670e0 0x7f23cc069590 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f23e006d500 tx=0x7f23d4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:53.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:53.265+0000 7f23c7fff700 1 -- 172.21.15.31:0/3006720163 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f23d0051c00 con 0x7f23e0157380 2023-10-06T02:26:53.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:53.515+0000 7f23e519d700 1 -- 172.21.15.31:0/3006720163 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f23e004c3c0 con 0x7f23e0157380 2023-10-06T02:26:54.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.129+0000 7f23c7fff700 1 -- 172.21.15.31:0/3006720163 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v38) v1 ==== 136+0+0 (secure 0 0 0) 0x7f23d0055220 con 0x7f23e0157380 2023-10-06T02:26:54.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.131+0000 7f23e519d700 1 -- 172.21.15.31:0/3006720163 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f23cc0670e0 msgr2=0x7f23cc069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:54.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.131+0000 7f23e519d700 1 --2- 172.21.15.31:0/3006720163 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f23cc0670e0 0x7f23cc069590 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f23e006d500 tx=0x7f23d4009040 comp rx=0 tx=0).stop 2023-10-06T02:26:54.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.132+0000 7f23e519d700 1 -- 172.21.15.31:0/3006720163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23e0157380 msgr2=0x7f23e00b0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:54.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.132+0000 7f23e519d700 1 --2- 172.21.15.31:0/3006720163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23e0157380 0x7f23e00b0d60 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f23d000ec80 tx=0x7f23d00091a0 comp rx=0 tx=0).stop 2023-10-06T02:26:54.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.132+0000 7f23e519d700 1 -- 172.21.15.31:0/3006720163 shutdown_connections 2023-10-06T02:26:54.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.132+0000 7f23e519d700 1 --2- 172.21.15.31:0/3006720163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23e015b1a0 0x7f23e00ca170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:54.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.132+0000 7f23e519d700 1 --2- 172.21.15.31:0/3006720163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23e0157380 0x7f23e00b0d60 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:54.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.132+0000 7f23e519d700 1 --2- 172.21.15.31:0/3006720163 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f23cc0670e0 0x7f23cc069590 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:54.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.132+0000 7f23e519d700 1 --2- 172.21.15.31:0/3006720163 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f23e0157750 0x7f23e0159f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:54.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.132+0000 7f23e519d700 1 -- 172.21.15.31:0/3006720163 >> 172.21.15.31:0/3006720163 conn(0x7f23e006c6a0 msgr2=0x7f23e0152d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:54.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.132+0000 7f23e519d700 1 -- 172.21.15.31:0/3006720163 shutdown_connections 2023-10-06T02:26:54.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.132+0000 7f23e519d700 1 -- 172.21.15.31:0/3006720163 wait complete. 2023-10-06T02:26:54.152 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-06T02:26:54.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.343+0000 7f475ffff700 1 Processor -- start 2023-10-06T02:26:54.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.343+0000 7f475ffff700 1 -- start start 2023-10-06T02:26:54.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.345+0000 7f475ffff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4760159640 0x7f476006d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:54.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.345+0000 7f475ffff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4760159a10 0x7f47600ca7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:54.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.345+0000 7f475ffff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4760070410 0x7f47600cacf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:54.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.345+0000 7f475ffff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47600c7420 con 0x7f4760159640 2023-10-06T02:26:54.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.345+0000 7f475ffff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f476006d5a0 con 0x7f4760159a10 2023-10-06T02:26:54.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.345+0000 7f475ffff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f476006d820 con 0x7f4760070410 2023-10-06T02:26:54.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.345+0000 7f475effd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4760159640 0x7f476006d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:54.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.345+0000 7f475effd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4760159640 0x7f476006d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34660/0 (socket says 172.21.15.31:34660) 2023-10-06T02:26:54.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.345+0000 7f475effd700 1 -- 172.21.15.31:0/1860334356 learned_addr learned my addr 172.21.15.31:0/1860334356 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:26:54.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.345+0000 7f475f7fe700 1 --2- 172.21.15.31:0/1860334356 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4760070410 0x7f47600cacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:54.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.345+0000 7f475e7fc700 1 --2- 172.21.15.31:0/1860334356 >> v2:172.21.15.62:3300/0 conn(0x7f4760159a10 0x7f47600ca7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:54.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.346+0000 7f475effd700 1 -- 172.21.15.31:0/1860334356 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4760070410 msgr2=0x7f47600cacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:26:54.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.346+0000 7f475effd700 1 --2- 172.21.15.31:0/1860334356 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4760070410 0x7f47600cacf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:54.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.346+0000 7f475effd700 1 -- 172.21.15.31:0/1860334356 >> v2:172.21.15.62:3300/0 conn(0x7f4760159a10 msgr2=0x7f47600ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:26:54.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.346+0000 7f475effd700 1 --2- 172.21.15.31:0/1860334356 >> v2:172.21.15.62:3300/0 conn(0x7f4760159a10 0x7f47600ca7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:54.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.346+0000 7f475effd700 1 -- 172.21.15.31:0/1860334356 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f476006db00 con 0x7f4760159640 2023-10-06T02:26:54.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.346+0000 7f475effd700 1 --2- 172.21.15.31:0/1860334356 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4760159640 0x7f476006d060 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f476006a270 tx=0x7f4750009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=710d6e8e8d769ce6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:54.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.346+0000 7f475dffb700 1 -- 172.21.15.31:0/1860334356 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f475000b860 con 0x7f4760159640 2023-10-06T02:26:54.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.346+0000 7f475dffb700 1 -- 172.21.15.31:0/1860334356 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f475000b9c0 con 0x7f4760159640 2023-10-06T02:26:54.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.346+0000 7f475dffb700 1 -- 172.21.15.31:0/1860334356 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f475000bd60 con 0x7f4760159640 2023-10-06T02:26:54.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.347+0000 7f475ffff700 1 -- 172.21.15.31:0/1860334356 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4760159640 msgr2=0x7f476006d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:54.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.347+0000 7f475ffff700 1 --2- 172.21.15.31:0/1860334356 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4760159640 0x7f476006d060 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f476006a270 tx=0x7f4750009250 comp rx=0 tx=0).stop 2023-10-06T02:26:54.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.347+0000 7f475ffff700 1 -- 172.21.15.31:0/1860334356 shutdown_connections 2023-10-06T02:26:54.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.347+0000 7f475ffff700 1 --2- 172.21.15.31:0/1860334356 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4760070410 0x7f47600cacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:54.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.347+0000 7f475ffff700 1 --2- 172.21.15.31:0/1860334356 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4760159640 0x7f476006d060 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:54.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.347+0000 7f475ffff700 1 --2- 172.21.15.31:0/1860334356 >> v2:172.21.15.62:3300/0 conn(0x7f4760159a10 0x7f47600ca7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:54.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.347+0000 7f475ffff700 1 -- 172.21.15.31:0/1860334356 >> 172.21.15.31:0/1860334356 conn(0x7f476006e720 msgr2=0x7f476006eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:54.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.347+0000 7f475ffff700 1 -- 172.21.15.31:0/1860334356 shutdown_connections 2023-10-06T02:26:54.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.347+0000 7f475ffff700 1 -- 172.21.15.31:0/1860334356 wait complete. 2023-10-06T02:26:54.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.348+0000 7f475ffff700 1 Processor -- start 2023-10-06T02:26:54.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.348+0000 7f475ffff700 1 -- start start 2023-10-06T02:26:54.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.349+0000 7f475ffff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4760070410 0x7f4760150a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:54.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.349+0000 7f475ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4760159640 0x7f4760151a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:54.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.349+0000 7f475ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4760159a10 0x7f47600c6640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:54.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.349+0000 7f475ffff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4760151f80 con 0x7f4760159640 2023-10-06T02:26:54.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.349+0000 7f475ffff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47600c45c0 con 0x7f4760159a10 2023-10-06T02:26:54.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.349+0000 7f475ffff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47600c6b80 con 0x7f4760070410 2023-10-06T02:26:54.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.349+0000 7f475e7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4760159640 0x7f4760151a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:54.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.350+0000 7f475e7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4760159640 0x7f4760151a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34668/0 (socket says 172.21.15.31:34668) 2023-10-06T02:26:54.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.350+0000 7f475e7fc700 1 -- 172.21.15.31:0/2028639771 learned_addr learned my addr 172.21.15.31:0/2028639771 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:26:54.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.350+0000 7f475effd700 1 --2- 172.21.15.31:0/2028639771 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4760070410 0x7f4760150a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:54.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.350+0000 7f475f7fe700 1 --2- 172.21.15.31:0/2028639771 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4760159a10 0x7f47600c6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:54.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.350+0000 7f475e7fc700 1 -- 172.21.15.31:0/2028639771 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4760070410 msgr2=0x7f4760150a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:54.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.350+0000 7f475e7fc700 1 --2- 172.21.15.31:0/2028639771 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4760070410 0x7f4760150a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:54.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.350+0000 7f475e7fc700 1 -- 172.21.15.31:0/2028639771 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4760159a10 msgr2=0x7f47600c6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:54.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.350+0000 7f475e7fc700 1 --2- 172.21.15.31:0/2028639771 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4760159a10 0x7f47600c6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:54.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.350+0000 7f475e7fc700 1 -- 172.21.15.31:0/2028639771 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47500094f0 con 0x7f4760159640 2023-10-06T02:26:54.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.350+0000 7f475effd700 1 --2- 172.21.15.31:0/2028639771 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4760070410 0x7f4760150a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:26:54.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.350+0000 7f475f7fe700 1 --2- 172.21.15.31:0/2028639771 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4760159a10 0x7f47600c6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:26:54.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.350+0000 7f475e7fc700 1 --2- 172.21.15.31:0/2028639771 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4760159640 0x7f4760151a40 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f4754006b00 tx=0x7f4754008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:54.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.350+0000 7f4746ffd700 1 -- 172.21.15.31:0/2028639771 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47540125f0 con 0x7f4760159640 2023-10-06T02:26:54.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.351+0000 7f4746ffd700 1 -- 172.21.15.31:0/2028639771 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4754012770 con 0x7f4760159640 2023-10-06T02:26:54.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.351+0000 7f4746ffd700 1 -- 172.21.15.31:0/2028639771 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4754012b30 con 0x7f4760159640 2023-10-06T02:26:54.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.351+0000 7f475ffff700 1 -- 172.21.15.31:0/2028639771 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47601521b0 con 0x7f4760159640 2023-10-06T02:26:54.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.351+0000 7f475ffff700 1 -- 172.21.15.31:0/2028639771 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47600c3f70 con 0x7f4760159640 2023-10-06T02:26:54.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.352+0000 7f475ffff700 1 -- 172.21.15.31:0/2028639771 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f476004c3c0 con 0x7f4760159640 2023-10-06T02:26:54.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.353+0000 7f4746ffd700 1 -- 172.21.15.31:0/2028639771 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 9) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f475401e460 con 0x7f4760159640 2023-10-06T02:26:54.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.354+0000 7f4746ffd700 1 --2- 172.21.15.31:0/2028639771 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f474c0670e0 0x7f474c069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:54.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.355+0000 7f4746ffd700 1 -- 172.21.15.31:0/2028639771 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6203+0+0 (secure 0 0 0) 0x7f4754085030 con 0x7f4760159640 2023-10-06T02:26:54.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.355+0000 7f475effd700 1 --2- 172.21.15.31:0/2028639771 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f474c0670e0 0x7f474c069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:54.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.355+0000 7f475effd700 1 --2- 172.21.15.31:0/2028639771 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f474c0670e0 0x7f474c069590 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f476006a270 tx=0x7f4750017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:54.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.358+0000 7f4746ffd700 1 -- 172.21.15.31:0/2028639771 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4754051e30 con 0x7f4760159640 2023-10-06T02:26:54.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.588+0000 7f475ffff700 1 -- 172.21.15.31:0/2028639771 --> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f47600cb2c0 con 0x7f474c0670e0 2023-10-06T02:26:54.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.590+0000 7f4746ffd700 1 -- 172.21.15.31:0/2028639771 <== mgr.4107 v2:172.21.15.31:6832/85292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f47600cb2c0 con 0x7f474c0670e0 2023-10-06T02:26:54.590 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2023-10-06T02:26:54.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:26:54.590 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephfs" 2023-10-06T02:26:54.590 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:26:54.591 INFO:teuthology.orchestra.run.smithi031.stdout:] 2023-10-06T02:26:54.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.592+0000 7f475ffff700 1 -- 172.21.15.31:0/2028639771 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f474c0670e0 msgr2=0x7f474c069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:54.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.592+0000 7f475ffff700 1 --2- 172.21.15.31:0/2028639771 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f474c0670e0 0x7f474c069590 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f476006a270 tx=0x7f4750017000 comp rx=0 tx=0).stop 2023-10-06T02:26:54.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.592+0000 7f475ffff700 1 -- 172.21.15.31:0/2028639771 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4760159640 msgr2=0x7f4760151a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:54.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.592+0000 7f475ffff700 1 --2- 172.21.15.31:0/2028639771 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4760159640 0x7f4760151a40 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f4754006b00 tx=0x7f4754008040 comp rx=0 tx=0).stop 2023-10-06T02:26:54.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.592+0000 7f475ffff700 1 -- 172.21.15.31:0/2028639771 shutdown_connections 2023-10-06T02:26:54.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.592+0000 7f475ffff700 1 --2- 172.21.15.31:0/2028639771 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4760159a10 0x7f47600c6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:54.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.592+0000 7f475ffff700 1 --2- 172.21.15.31:0/2028639771 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f474c0670e0 0x7f474c069590 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:54.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.592+0000 7f475ffff700 1 --2- 172.21.15.31:0/2028639771 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4760070410 0x7f4760150a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:54.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.592+0000 7f475ffff700 1 --2- 172.21.15.31:0/2028639771 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4760159640 0x7f4760151a40 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:54.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.592+0000 7f475ffff700 1 -- 172.21.15.31:0/2028639771 >> 172.21.15.31:0/2028639771 conn(0x7f476006e720 msgr2=0x7f476006a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:54.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.592+0000 7f475ffff700 1 -- 172.21.15.31:0/2028639771 shutdown_connections 2023-10-06T02:26:54.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.592+0000 7f475ffff700 1 -- 172.21.15.31:0/2028639771 wait complete. 2023-10-06T02:26:54.610 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T02:26:54.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.801+0000 7fcfa5d9d700 1 Processor -- start 2023-10-06T02:26:54.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.801+0000 7fcfa5d9d700 1 -- start start 2023-10-06T02:26:54.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.802+0000 7fcfa5d9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fcfa006e560 0x7fcfa00c7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:54.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.802+0000 7fcfa5d9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfa00c7e70 con 0x7fcfa0155c60 2023-10-06T02:26:54.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.802+0000 7fcfa5d9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfa006b790 con 0x7fcfa0155890 2023-10-06T02:26:54.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.802+0000 7fcfa5d9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfa006ab80 con 0x7fcfa006e560 2023-10-06T02:26:54.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.803+0000 7fcf9ffff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fcfa0155c60 0x7fcfa006de50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46804/0 (socket says 172.21.15.31:46804) 2023-10-06T02:26:54.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.803+0000 7fcf9ffff700 1 -- 172.21.15.31:0/3610754561 learned_addr learned my addr 172.21.15.31:0/3610754561 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:26:54.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.803+0000 7fcfa559c700 1 --2- 172.21.15.31:0/3610754561 >> v2:172.21.15.62:3300/0 conn(0x7fcfa006e560 0x7fcfa00c7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:54.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.803+0000 7fcf9f7fe700 1 -- 172.21.15.31:0/3610754561 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 991747211 0 0) 0x7fcfa00c7e70 con 0x7fcfa0155c60 2023-10-06T02:26:54.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.803+0000 7fcf9f7fe700 1 -- 172.21.15.31:0/3610754561 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf90003570 con 0x7fcfa0155c60 2023-10-06T02:26:54.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.803+0000 7fcf9f7fe700 1 -- 172.21.15.31:0/3610754561 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1068577641 0 0) 0x7fcf90003570 con 0x7fcfa0155c60 2023-10-06T02:26:54.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.803+0000 7fcfa559c700 1 -- 172.21.15.31:0/3610754561 >> v1:172.21.15.62:6789/0 conn(0x7fcfa0155890 legacy=0x7fcfa006a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:54.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.803+0000 7fcfa559c700 1 -- 172.21.15.31:0/3610754561 >> v1:172.21.15.31:6789/0 conn(0x7fcfa0155c60 legacy=0x7fcfa006de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:54.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.803+0000 7fcfa559c700 1 -- 172.21.15.31:0/3610754561 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfa006ae60 con 0x7fcfa006e560 2023-10-06T02:26:54.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.804+0000 7fcfa559c700 1 --2- 172.21.15.31:0/3610754561 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfa006e560 0x7fcfa00c7930 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fcf98009e60 tx=0x7fcf98009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26cf43a647882b17 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:54.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.804+0000 7fcf9f7fe700 1 -- 172.21.15.31:0/3610754561 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf9800e070 con 0x7fcfa006e560 2023-10-06T02:26:54.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.804+0000 7fcf9f7fe700 1 -- 172.21.15.31:0/3610754561 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf9800b850 con 0x7fcfa006e560 2023-10-06T02:26:54.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.804+0000 7fcf9f7fe700 1 -- 172.21.15.31:0/3610754561 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf8c001470 con 0x7fcfa006e560 2023-10-06T02:26:54.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.804+0000 7fcfa5d9d700 1 -- 172.21.15.31:0/3610754561 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfa006e560 msgr2=0x7fcfa00c7930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:54.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.804+0000 7fcfa5d9d700 1 --2- 172.21.15.31:0/3610754561 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfa006e560 0x7fcfa00c7930 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fcf98009e60 tx=0x7fcf98009780 comp rx=0 tx=0).stop 2023-10-06T02:26:54.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.805+0000 7fcfa5d9d700 1 -- 172.21.15.31:0/3610754561 shutdown_connections 2023-10-06T02:26:54.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.805+0000 7fcfa5d9d700 1 --2- 172.21.15.31:0/3610754561 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfa006e560 0x7fcfa00c7930 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:54.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.805+0000 7fcfa5d9d700 1 -- 172.21.15.31:0/3610754561 >> 172.21.15.31:0/3610754561 conn(0x7fcfa006d110 msgr2=0x7fcfa00cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:54.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.805+0000 7fcfa5d9d700 1 -- 172.21.15.31:0/3610754561 shutdown_connections 2023-10-06T02:26:54.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.805+0000 7fcfa5d9d700 1 -- 172.21.15.31:0/3610754561 wait complete. 2023-10-06T02:26:54.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.806+0000 7fcfa5d9d700 1 Processor -- start 2023-10-06T02:26:54.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.806+0000 7fcfa5d9d700 1 -- start start 2023-10-06T02:26:54.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.807+0000 7fcfa5d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfa006e560 0x7fcfa00c41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:54.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.807+0000 7fcfa5d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcfa0155890 0x7fcfa0153c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:54.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.807+0000 7fcfa5d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcfa0155c60 0x7fcfa0167250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:54.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.807+0000 7fcfa5d9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa0168230 con 0x7fcfa0155890 2023-10-06T02:26:54.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.807+0000 7fcfa5d9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa0167c70 con 0x7fcfa006e560 2023-10-06T02:26:54.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.807+0000 7fcfa5d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa0167f50 con 0x7fcfa0155c60 2023-10-06T02:26:54.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.807+0000 7fcf9ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcfa0155890 0x7fcfa0153c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:54.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.807+0000 7fcf9ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcfa0155890 0x7fcfa0153c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34678/0 (socket says 172.21.15.31:34678) 2023-10-06T02:26:54.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.807+0000 7fcf9ffff700 1 -- 172.21.15.31:0/305701829 learned_addr learned my addr 172.21.15.31:0/305701829 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:26:54.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.808+0000 7fcfa4d9b700 1 --2- 172.21.15.31:0/305701829 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfa006e560 0x7fcfa00c41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:54.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.808+0000 7fcfa559c700 1 --2- 172.21.15.31:0/305701829 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcfa0155c60 0x7fcfa0167250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:54.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.808+0000 7fcf9ffff700 1 -- 172.21.15.31:0/305701829 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcfa0155c60 msgr2=0x7fcfa0167250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:54.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.808+0000 7fcf9ffff700 1 --2- 172.21.15.31:0/305701829 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcfa0155c60 0x7fcfa0167250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:54.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.808+0000 7fcf9ffff700 1 -- 172.21.15.31:0/305701829 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfa006e560 msgr2=0x7fcfa00c41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:54.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.808+0000 7fcf9ffff700 1 --2- 172.21.15.31:0/305701829 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfa006e560 0x7fcfa00c41b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:54.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.808+0000 7fcf9ffff700 1 -- 172.21.15.31:0/305701829 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf98009a20 con 0x7fcfa0155890 2023-10-06T02:26:54.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.808+0000 7fcfa559c700 1 --2- 172.21.15.31:0/305701829 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcfa0155c60 0x7fcfa0167250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:26:54.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.808+0000 7fcf9ffff700 1 --2- 172.21.15.31:0/305701829 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcfa0155890 0x7fcfa0153c40 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c00cb60 tx=0x7fcf8c001ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:54.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.808+0000 7fcf9cff9700 1 -- 172.21.15.31:0/305701829 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf8c001d60 con 0x7fcfa0155890 2023-10-06T02:26:54.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.808+0000 7fcf9cff9700 1 -- 172.21.15.31:0/305701829 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf8c007360 con 0x7fcfa0155890 2023-10-06T02:26:54.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.808+0000 7fcfa5d9d700 1 -- 172.21.15.31:0/305701829 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfa0159500 con 0x7fcfa0155890 2023-10-06T02:26:54.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.808+0000 7fcf9cff9700 1 -- 172.21.15.31:0/305701829 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf8c004830 con 0x7fcfa0155890 2023-10-06T02:26:54.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.809+0000 7fcfa5d9d700 1 -- 172.21.15.31:0/305701829 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfa0167790 con 0x7fcfa0155890 2023-10-06T02:26:54.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.810+0000 7fcfa5d9d700 1 -- 172.21.15.31:0/305701829 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfa0159a20 con 0x7fcfa0155890 2023-10-06T02:26:54.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.810+0000 7fcf9cff9700 1 -- 172.21.15.31:0/305701829 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 9) v1 ==== 84893+0+0 (secure 0 0 0) 0x7fcf8c004ba0 con 0x7fcfa0155890 2023-10-06T02:26:54.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.811+0000 7fcf9cff9700 1 --2- 172.21.15.31:0/305701829 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fcf840670e0 0x7fcf84069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:54.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.811+0000 7fcf9cff9700 1 -- 172.21.15.31:0/305701829 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6203+0+0 (secure 0 0 0) 0x7fcf8c0843a0 con 0x7fcfa0155890 2023-10-06T02:26:54.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.811+0000 7fcfa4d9b700 1 --2- 172.21.15.31:0/305701829 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fcf840670e0 0x7fcf84069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:54.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.812+0000 7fcfa4d9b700 1 --2- 172.21.15.31:0/305701829 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fcf840670e0 0x7fcf84069590 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fcf94000f60 tx=0x7fcf94009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:54.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:54.815+0000 7fcf9cff9700 1 -- 172.21.15.31:0/305701829 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcfa0159a20 con 0x7fcfa0155890 2023-10-06T02:26:55.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.018+0000 7fcfa5d9d700 1 -- 172.21.15.31:0/305701829 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fcfa0159a20 con 0x7fcfa0155890 2023-10-06T02:26:55.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.019+0000 7fcf9cff9700 1 -- 172.21.15.31:0/305701829 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7fcfa0159a20 con 0x7fcfa0155890 2023-10-06T02:26:55.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.021+0000 7fcfa5d9d700 1 -- 172.21.15.31:0/305701829 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fcf840670e0 msgr2=0x7fcf84069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:55.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.021+0000 7fcfa5d9d700 1 --2- 172.21.15.31:0/305701829 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fcf840670e0 0x7fcf84069590 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fcf94000f60 tx=0x7fcf94009040 comp rx=0 tx=0).stop 2023-10-06T02:26:55.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.021+0000 7fcfa5d9d700 1 -- 172.21.15.31:0/305701829 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcfa0155890 msgr2=0x7fcfa0153c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:55.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.021+0000 7fcfa5d9d700 1 --2- 172.21.15.31:0/305701829 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcfa0155890 0x7fcfa0153c40 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c00cb60 tx=0x7fcf8c001ad0 comp rx=0 tx=0).stop 2023-10-06T02:26:55.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.021+0000 7fcfa5d9d700 1 -- 172.21.15.31:0/305701829 shutdown_connections 2023-10-06T02:26:55.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.021+0000 7fcfa5d9d700 1 --2- 172.21.15.31:0/305701829 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fcf840670e0 0x7fcf84069590 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:55.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.021+0000 7fcfa5d9d700 1 --2- 172.21.15.31:0/305701829 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcfa0155c60 0x7fcfa0167250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:55.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.021+0000 7fcfa5d9d700 1 --2- 172.21.15.31:0/305701829 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfa006e560 0x7fcfa00c41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:55.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.021+0000 7fcfa5d9d700 1 --2- 172.21.15.31:0/305701829 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcfa0155890 0x7fcfa0153c40 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:55.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.021+0000 7fcfa5d9d700 1 -- 172.21.15.31:0/305701829 >> 172.21.15.31:0/305701829 conn(0x7fcfa006d110 msgr2=0x7fcfa0156b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:55.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.021+0000 7fcfa5d9d700 1 -- 172.21.15.31:0/305701829 shutdown_connections 2023-10-06T02:26:55.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.021+0000 7fcfa5d9d700 1 -- 172.21.15.31:0/305701829 wait complete. 2023-10-06T02:26:55.040 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:26:55.040 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/_nogroup/subvolume_0000000000626507) 2023-10-06T02:26:55.110 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:26:55.110 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2023-10-06T02:26:55.148 INFO:teuthology.orchestra.run.smithi031.stdout:cephfs_data 2023-10-06T02:26:55.149 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:26:55.149 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data ./volumes/_nogroup/subvolume_0000000000626507) 2023-10-06T02:26:55.256 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000626508 2023-10-06T02:26:55.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.445+0000 7ff87f59e700 1 Processor -- start 2023-10-06T02:26:55.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.445+0000 7ff87f59e700 1 -- start start 2023-10-06T02:26:55.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.447+0000 7ff87f59e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff88013fe90 0x7ff88006abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:55.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.447+0000 7ff87f59e700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff88006b290 con 0x7ff8800c7fb0 2023-10-06T02:26:55.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.447+0000 7ff87f59e700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff88006b570 con 0x7ff8800c7be0 2023-10-06T02:26:55.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.447+0000 7ff87f59e700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff88006bf10 con 0x7ff88013fe90 2023-10-06T02:26:55.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.447+0000 7ff87dd9b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7ff8800c7fb0 0x7ff88013f780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46806/0 (socket says 172.21.15.31:46806) 2023-10-06T02:26:55.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.447+0000 7ff87dd9b700 1 -- 172.21.15.31:0/2359645603 learned_addr learned my addr 172.21.15.31:0/2359645603 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:26:55.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.447+0000 7ff87ed9d700 1 --2- 172.21.15.31:0/2359645603 >> v2:172.21.15.31:3300/0 conn(0x7ff88013fe90 0x7ff88006abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:55.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.447+0000 7ff87ed9d700 1 -- 172.21.15.31:0/2359645603 >> v1:172.21.15.62:6789/0 conn(0x7ff8800c7be0 legacy=0x7ff88015a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:26:55.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.447+0000 7ff87ed9d700 1 -- 172.21.15.31:0/2359645603 >> v1:172.21.15.31:6789/0 conn(0x7ff8800c7fb0 legacy=0x7ff88013f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:55.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.448+0000 7ff87ed9d700 1 -- 172.21.15.31:0/2359645603 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff88006b850 con 0x7ff88013fe90 2023-10-06T02:26:55.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.448+0000 7ff87d59a700 1 -- 172.21.15.31:0/2359645603 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3901574678 0 0) 0x7ff88006b290 con 0x7ff8800c7fb0 2023-10-06T02:26:55.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.448+0000 7ff87ed9d700 1 --2- 172.21.15.31:0/2359645603 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff88013fe90 0x7ff88006abd0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7ff878009e60 tx=0x7ff878009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90e0fd6c02b9e55f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:55.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.448+0000 7ff87d59a700 1 -- 172.21.15.31:0/2359645603 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff87800e070 con 0x7ff88013fe90 2023-10-06T02:26:55.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.448+0000 7ff87d59a700 1 -- 172.21.15.31:0/2359645603 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff87800bbf0 con 0x7ff88013fe90 2023-10-06T02:26:55.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.448+0000 7ff87d59a700 1 -- 172.21.15.31:0/2359645603 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff878012430 con 0x7ff88013fe90 2023-10-06T02:26:55.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.448+0000 7ff87f59e700 1 -- 172.21.15.31:0/2359645603 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff88013fe90 msgr2=0x7ff88006abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:55.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.448+0000 7ff87f59e700 1 --2- 172.21.15.31:0/2359645603 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff88013fe90 0x7ff88006abd0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7ff878009e60 tx=0x7ff878009780 comp rx=0 tx=0).stop 2023-10-06T02:26:55.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.449+0000 7ff87f59e700 1 -- 172.21.15.31:0/2359645603 shutdown_connections 2023-10-06T02:26:55.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.449+0000 7ff87f59e700 1 --2- 172.21.15.31:0/2359645603 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff88013fe90 0x7ff88006abd0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:55.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.449+0000 7ff87f59e700 1 -- 172.21.15.31:0/2359645603 >> 172.21.15.31:0/2359645603 conn(0x7ff88006d110 msgr2=0x7ff8800cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:55.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.449+0000 7ff87f59e700 1 -- 172.21.15.31:0/2359645603 shutdown_connections 2023-10-06T02:26:55.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.449+0000 7ff87f59e700 1 -- 172.21.15.31:0/2359645603 wait complete. 2023-10-06T02:26:55.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.450+0000 7ff87f59e700 1 Processor -- start 2023-10-06T02:26:55.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.450+0000 7ff87f59e700 1 -- start start 2023-10-06T02:26:55.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.451+0000 7ff87f59e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8800c7be0 0x7ff8800aeed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:55.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.451+0000 7ff87f59e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff8800c7fb0 0x7ff880132bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:55.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.451+0000 7ff87f59e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff88013fe90 0x7ff8801383d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:55.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.451+0000 7ff87f59e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff88013f010 con 0x7ff8800c7be0 2023-10-06T02:26:55.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.451+0000 7ff87f59e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff880136b70 con 0x7ff88013fe90 2023-10-06T02:26:55.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.451+0000 7ff87f59e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff88013e770 con 0x7ff8800c7fb0 2023-10-06T02:26:55.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.452+0000 7ff87e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8800c7be0 0x7ff8800aeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:55.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.452+0000 7ff87e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8800c7be0 0x7ff8800aeed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34708/0 (socket says 172.21.15.31:34708) 2023-10-06T02:26:55.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.452+0000 7ff87e59c700 1 -- 172.21.15.31:0/1309316097 learned_addr learned my addr 172.21.15.31:0/1309316097 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:26:55.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.452+0000 7ff87dd9b700 1 --2- 172.21.15.31:0/1309316097 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff8800c7fb0 0x7ff880132bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:55.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.452+0000 7ff87ed9d700 1 --2- 172.21.15.31:0/1309316097 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff88013fe90 0x7ff8801383d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:55.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.452+0000 7ff87e59c700 1 -- 172.21.15.31:0/1309316097 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff8800c7fb0 msgr2=0x7ff880132bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:55.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.452+0000 7ff87e59c700 1 --2- 172.21.15.31:0/1309316097 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff8800c7fb0 0x7ff880132bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:55.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.452+0000 7ff87e59c700 1 -- 172.21.15.31:0/1309316097 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff88013fe90 msgr2=0x7ff8801383d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:55.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.452+0000 7ff87e59c700 1 --2- 172.21.15.31:0/1309316097 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff88013fe90 0x7ff8801383d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:55.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.452+0000 7ff87e59c700 1 -- 172.21.15.31:0/1309316097 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff878009a20 con 0x7ff8800c7be0 2023-10-06T02:26:55.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.452+0000 7ff87ed9d700 1 --2- 172.21.15.31:0/1309316097 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff88013fe90 0x7ff8801383d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:26:55.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.452+0000 7ff87e59c700 1 --2- 172.21.15.31:0/1309316097 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8800c7be0 0x7ff8800aeed0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7ff87000c5d0 tx=0x7ff870007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:55.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.452+0000 7ff86e7fc700 1 -- 172.21.15.31:0/1309316097 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff870011590 con 0x7ff8800c7be0 2023-10-06T02:26:55.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.453+0000 7ff86e7fc700 1 -- 172.21.15.31:0/1309316097 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff870011710 con 0x7ff8800c7be0 2023-10-06T02:26:55.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.453+0000 7ff86e7fc700 1 -- 172.21.15.31:0/1309316097 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff870011ad0 con 0x7ff8800c7be0 2023-10-06T02:26:55.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.453+0000 7ff87f59e700 1 -- 172.21.15.31:0/1309316097 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff88013ea50 con 0x7ff8800c7be0 2023-10-06T02:26:55.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.453+0000 7ff87f59e700 1 -- 172.21.15.31:0/1309316097 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff880133110 con 0x7ff8800c7be0 2023-10-06T02:26:55.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.454+0000 7ff87f59e700 1 -- 172.21.15.31:0/1309316097 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff880136940 con 0x7ff8800c7be0 2023-10-06T02:26:55.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.455+0000 7ff86e7fc700 1 -- 172.21.15.31:0/1309316097 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 9) v1 ==== 84893+0+0 (secure 0 0 0) 0x7ff87001c460 con 0x7ff8800c7be0 2023-10-06T02:26:55.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.456+0000 7ff86e7fc700 1 --2- 172.21.15.31:0/1309316097 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7ff8600670e0 0x7ff860069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:26:55.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.456+0000 7ff86e7fc700 1 -- 172.21.15.31:0/1309316097 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6203+0+0 (secure 0 0 0) 0x7ff870084720 con 0x7ff8800c7be0 2023-10-06T02:26:55.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.456+0000 7ff87dd9b700 1 --2- 172.21.15.31:0/1309316097 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7ff8600670e0 0x7ff860069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:26:55.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.457+0000 7ff87dd9b700 1 --2- 172.21.15.31:0/1309316097 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7ff8600670e0 0x7ff860069590 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff880156610 tx=0x7ff874007d30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:26:55.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.460+0000 7ff86e7fc700 1 -- 172.21.15.31:0/1309316097 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff880136940 con 0x7ff8800c7be0 2023-10-06T02:26:55.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.694+0000 7ff87f59e700 1 -- 172.21.15.31:0/1309316097 --> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000626508", "target": ["mon-mgr", ""]}) v1 -- 0x7ff88004c3c0 con 0x7ff8600670e0 2023-10-06T02:26:55.699 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:26:55.699+0000 7f3e7b47e700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:26:55.700 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:26:55.699+0000 7f3e7b47e700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:26:55.700 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:26:55.699+0000 7f3e7b47e700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:26:55.700 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:26:55.699+0000 7f3e7b47e700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:26:55.700 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:26:55.699+0000 7f3e7b47e700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:26:55.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.747+0000 7ff86e7fc700 1 -- 172.21.15.31:0/1309316097 <== mgr.4107 v2:172.21.15.31:6832/85292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7ff88004c3c0 con 0x7ff8600670e0 2023-10-06T02:26:55.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.750+0000 7ff87f59e700 1 -- 172.21.15.31:0/1309316097 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7ff8600670e0 msgr2=0x7ff860069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:55.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.750+0000 7ff87f59e700 1 --2- 172.21.15.31:0/1309316097 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7ff8600670e0 0x7ff860069590 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff880156610 tx=0x7ff874007d30 comp rx=0 tx=0).stop 2023-10-06T02:26:55.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.750+0000 7ff87f59e700 1 -- 172.21.15.31:0/1309316097 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8800c7be0 msgr2=0x7ff8800aeed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:26:55.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.750+0000 7ff87f59e700 1 --2- 172.21.15.31:0/1309316097 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8800c7be0 0x7ff8800aeed0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7ff87000c5d0 tx=0x7ff870007f60 comp rx=0 tx=0).stop 2023-10-06T02:26:55.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.750+0000 7ff87f59e700 1 -- 172.21.15.31:0/1309316097 shutdown_connections 2023-10-06T02:26:55.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.750+0000 7ff87f59e700 1 --2- 172.21.15.31:0/1309316097 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff88013fe90 0x7ff8801383d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:55.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.750+0000 7ff87f59e700 1 --2- 172.21.15.31:0/1309316097 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8800c7be0 0x7ff8800aeed0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:55.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.750+0000 7ff87f59e700 1 --2- 172.21.15.31:0/1309316097 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7ff8600670e0 0x7ff860069590 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:55.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.750+0000 7ff87f59e700 1 --2- 172.21.15.31:0/1309316097 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff8800c7fb0 0x7ff880132bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:26:55.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.750+0000 7ff87f59e700 1 -- 172.21.15.31:0/1309316097 >> 172.21.15.31:0/1309316097 conn(0x7ff88006d110 msgr2=0x7ff8800c1060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:26:55.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.750+0000 7ff87f59e700 1 -- 172.21.15.31:0/1309316097 shutdown_connections 2023-10-06T02:26:55.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:26:55.750+0000 7ff87f59e700 1 -- 172.21.15.31:0/1309316097 wait complete. 2023-10-06T02:26:55.768 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:26:55.768 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cp ./volumes/_nogroup/subvolume_0000000000626508/.meta ./volumes/_nogroup/subvolume_0000000000626507) 2023-10-06T02:27:00.581 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000626507 2023-10-06T02:27:00.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:00.808+0000 7f1285d9d700 1 Processor -- start 2023-10-06T02:27:00.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:00.808+0000 7f1285d9d700 1 -- start start 2023-10-06T02:27:00.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:00.809+0000 7f1285d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1280155890 0x7f12800c7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:00.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:00.809+0000 7f1285d9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1280155c60 0x7f128006a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:00.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:00.809+0000 7f1285d9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12800c7e70 con 0x7f128006de50 2023-10-06T02:27:00.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:00.809+0000 7f1285d9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f128006b790 con 0x7f1280155c60 2023-10-06T02:27:00.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:00.809+0000 7f1285d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f128006a8e0 con 0x7f1280155890 2023-10-06T02:27:00.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:00.809+0000 7f127ffff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1280155c60 0x7f128006a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:00.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:00.809+0000 7f128559c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f128006de50 0x7f1280159720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:38206/0 (socket says 172.21.15.31:38206) 2023-10-06T02:27:00.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:00.809+0000 7f127ffff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1280155c60 0x7f128006a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50976/0 (socket says 172.21.15.31:50976) 2023-10-06T02:27:00.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:00.809+0000 7f128559c700 1 -- 172.21.15.31:0/785613710 learned_addr learned my addr 172.21.15.31:0/785613710 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:27:00.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:00.810+0000 7f1284d9b700 1 --2- 172.21.15.31:0/785613710 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1280155890 0x7f12800c7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:00.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:00.810+0000 7f127ffff700 1 -- 172.21.15.31:0/785613710 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1280155890 msgr2=0x7f12800c7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:00.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:00.810+0000 7f127ffff700 1 --2- 172.21.15.31:0/785613710 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1280155890 0x7f12800c7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:00.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:00.810+0000 7f127ffff700 1 -- 172.21.15.31:0/785613710 >> v1:172.21.15.31:6789/0 conn(0x7f128006de50 legacy=0x7f1280159720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:00.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:00.810+0000 7f127ffff700 1 -- 172.21.15.31:0/785613710 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f128006ab80 con 0x7f1280155c60 2023-10-06T02:27:00.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:00.810+0000 7f127f7fe700 1 -- 172.21.15.31:0/785613710 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2730064505 0 0) 0x7f12800c7e70 con 0x7f128006de50 2023-10-06T02:27:00.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:00.810+0000 7f127ffff700 1 --2- 172.21.15.31:0/785613710 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1280155c60 0x7f128006a3a0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f126c0098e0 tx=0x7f126c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5718375291509548 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:00.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:00.810+0000 7f127f7fe700 1 -- 172.21.15.31:0/785613710 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f126c009e80 con 0x7f1280155c60 2023-10-06T02:27:00.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:00.810+0000 7f127f7fe700 1 -- 172.21.15.31:0/785613710 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f126c00b860 con 0x7f1280155c60 2023-10-06T02:27:00.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:00.810+0000 7f127f7fe700 1 -- 172.21.15.31:0/785613710 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f126c00bbe0 con 0x7f1280155c60 2023-10-06T02:27:00.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:00.811+0000 7f1285d9d700 1 -- 172.21.15.31:0/785613710 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1280155c60 msgr2=0x7f128006a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:00.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:00.811+0000 7f1285d9d700 1 --2- 172.21.15.31:0/785613710 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1280155c60 0x7f128006a3a0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f126c0098e0 tx=0x7f126c0091a0 comp rx=0 tx=0).stop 2023-10-06T02:27:00.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:00.811+0000 7f1285d9d700 1 -- 172.21.15.31:0/785613710 shutdown_connections 2023-10-06T02:27:00.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:00.811+0000 7f1285d9d700 1 --2- 172.21.15.31:0/785613710 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1280155890 0x7f12800c7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:00.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:00.811+0000 7f1285d9d700 1 --2- 172.21.15.31:0/785613710 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1280155c60 0x7f128006a3a0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:00.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:00.811+0000 7f1285d9d700 1 -- 172.21.15.31:0/785613710 >> 172.21.15.31:0/785613710 conn(0x7f128006d110 msgr2=0x7f12800cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:00.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:00.811+0000 7f1285d9d700 1 -- 172.21.15.31:0/785613710 shutdown_connections 2023-10-06T02:27:00.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:00.811+0000 7f1285d9d700 1 -- 172.21.15.31:0/785613710 wait complete. 2023-10-06T02:27:00.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:00.812+0000 7f1285d9d700 1 Processor -- start 2023-10-06T02:27:00.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:00.813+0000 7f1285d9d700 1 -- start start 2023-10-06T02:27:00.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:00.813+0000 7f1285d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f128006de50 0x7f1280143d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:00.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:00.813+0000 7f1285d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1280155890 0x7f1280153e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:00.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:00.813+0000 7f1285d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1280155c60 0x7f12800b0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:00.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:00.814+0000 7f1285d9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12801539d0 con 0x7f1280155c60 2023-10-06T02:27:00.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:00.814+0000 7f1285d9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1280152dc0 con 0x7f1280155890 2023-10-06T02:27:00.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:00.814+0000 7f1285d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12801530a0 con 0x7f128006de50 2023-10-06T02:27:00.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:00.814+0000 7f128559c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1280155c60 0x7f12800b0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:00.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:00.814+0000 7f128559c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1280155c60 0x7f12800b0070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50978/0 (socket says 172.21.15.31:50978) 2023-10-06T02:27:00.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:00.814+0000 7f128559c700 1 -- 172.21.15.31:0/2429460213 learned_addr learned my addr 172.21.15.31:0/2429460213 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:00.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:00.814+0000 7f128559c700 1 -- 172.21.15.31:0/2429460213 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f128006de50 msgr2=0x7f1280143d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:00.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:00.814+0000 7f128559c700 1 --2- 172.21.15.31:0/2429460213 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f128006de50 0x7f1280143d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:00.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:00.814+0000 7f128559c700 1 -- 172.21.15.31:0/2429460213 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1280155890 msgr2=0x7f1280153e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:00.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:00.814+0000 7f127ffff700 1 --2- 172.21.15.31:0/2429460213 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1280155890 0x7f1280153e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:00.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:00.814+0000 7f128559c700 1 --2- 172.21.15.31:0/2429460213 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1280155890 0x7f1280153e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:00.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:00.814+0000 7f128559c700 1 -- 172.21.15.31:0/2429460213 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f126c009420 con 0x7f1280155c60 2023-10-06T02:27:00.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:00.814+0000 7f128559c700 1 --2- 172.21.15.31:0/2429460213 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1280155c60 0x7f12800b0070 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f127800cc90 tx=0x7f1278001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:00.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:00.815+0000 7f127cff9700 1 -- 172.21.15.31:0/2429460213 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f127800cea0 con 0x7f1280155c60 2023-10-06T02:27:00.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:00.815+0000 7f127cff9700 1 -- 172.21.15.31:0/2429460213 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f127800ab70 con 0x7f1280155c60 2023-10-06T02:27:00.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:00.815+0000 7f127cff9700 1 -- 172.21.15.31:0/2429460213 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12780070f0 con 0x7f1280155c60 2023-10-06T02:27:00.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:00.815+0000 7f1285d9d700 1 -- 172.21.15.31:0/2429460213 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1280153380 con 0x7f1280155c60 2023-10-06T02:27:00.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:00.815+0000 7f1285d9d700 1 -- 172.21.15.31:0/2429460213 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f128014d3f0 con 0x7f1280155c60 2023-10-06T02:27:00.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:00.816+0000 7f1285d9d700 1 -- 172.21.15.31:0/2429460213 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f128004c3c0 con 0x7f1280155c60 2023-10-06T02:27:00.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:00.817+0000 7f127cff9700 1 -- 172.21.15.31:0/2429460213 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f12780074a0 con 0x7f1280155c60 2023-10-06T02:27:00.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:00.817+0000 7f127cff9700 1 --2- 172.21.15.31:0/2429460213 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f1264067170 0x7f1264069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:00.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:00.818+0000 7f127cff9700 1 -- 172.21.15.31:0/2429460213 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6203+0+0 (secure 0 0 0) 0x7f1278084f80 con 0x7f1280155c60 2023-10-06T02:27:00.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:00.818+0000 7f1284d9b700 1 --2- 172.21.15.31:0/2429460213 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f1264067170 0x7f1264069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:00.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:00.818+0000 7f1284d9b700 1 --2- 172.21.15.31:0/2429460213 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f1264067170 0x7f1264069620 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f1280147bb0 tx=0x7f1274008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:00.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:00.822+0000 7f127cff9700 1 -- 172.21.15.31:0/2429460213 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1278051d50 con 0x7f1280155c60 2023-10-06T02:27:01.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.059+0000 7f1285d9d700 1 -- 172.21.15.31:0/2429460213 --> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000626507", "target": ["mon-mgr", ""]}) v1 -- 0x7f1280153b10 con 0x7f1264067170 2023-10-06T02:27:01.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.097+0000 7f127cff9700 1 -- 172.21.15.31:0/2429460213 <== mgr.4107 v2:172.21.15.31:6832/85292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+45 (secure 0 0 0) 0x7f1280153b10 con 0x7f1264067170 2023-10-06T02:27:01.098 INFO:teuthology.orchestra.run.smithi031.stdout:/volumes/_nogroup/subvolume_0000000000626507 2023-10-06T02:27:01.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.099+0000 7f1285d9d700 1 -- 172.21.15.31:0/2429460213 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f1264067170 msgr2=0x7f1264069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:01.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.099+0000 7f1285d9d700 1 --2- 172.21.15.31:0/2429460213 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f1264067170 0x7f1264069620 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f1280147bb0 tx=0x7f1274008040 comp rx=0 tx=0).stop 2023-10-06T02:27:01.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.099+0000 7f1285d9d700 1 -- 172.21.15.31:0/2429460213 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1280155c60 msgr2=0x7f12800b0070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:01.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.099+0000 7f1285d9d700 1 --2- 172.21.15.31:0/2429460213 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1280155c60 0x7f12800b0070 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f127800cc90 tx=0x7f1278001a80 comp rx=0 tx=0).stop 2023-10-06T02:27:01.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.100+0000 7f1285d9d700 1 -- 172.21.15.31:0/2429460213 shutdown_connections 2023-10-06T02:27:01.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.100+0000 7f1285d9d700 1 --2- 172.21.15.31:0/2429460213 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1280155c60 0x7f12800b0070 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:01.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.100+0000 7f1285d9d700 1 --2- 172.21.15.31:0/2429460213 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f1264067170 0x7f1264069620 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:01.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.100+0000 7f1285d9d700 1 --2- 172.21.15.31:0/2429460213 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f128006de50 0x7f1280143d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:01.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.100+0000 7f1285d9d700 1 --2- 172.21.15.31:0/2429460213 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1280155890 0x7f1280153e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:01.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.100+0000 7f1285d9d700 1 -- 172.21.15.31:0/2429460213 >> 172.21.15.31:0/2429460213 conn(0x7f128006d110 msgr2=0x7f128014a240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:01.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.100+0000 7f1285d9d700 1 -- 172.21.15.31:0/2429460213 shutdown_connections 2023-10-06T02:27:01.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.100+0000 7f1285d9d700 1 -- 172.21.15.31:0/2429460213 wait complete. 2023-10-06T02:27:01.116 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T02:27:01.118 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000626507 2023-10-06T02:27:01.162 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T02:27:01.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.354+0000 7f9a3d39d700 1 Processor -- start 2023-10-06T02:27:01.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.354+0000 7f9a3d39d700 1 -- start start 2023-10-06T02:27:01.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.355+0000 7f9a3d39d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9a38153a00 0x7f9a38153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:01.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.355+0000 7f9a3d39d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9a3815de50 0x7f9a3815a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:01.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.355+0000 7f9a3d39d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a3815af60 con 0x7f9a380cea00 2023-10-06T02:27:01.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.355+0000 7f9a3d39d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a3815d010 con 0x7f9a38153a00 2023-10-06T02:27:01.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.355+0000 7f9a3d39d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a3815b240 con 0x7f9a3815de50 2023-10-06T02:27:01.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.356+0000 7f9a377fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9a3815de50 0x7f9a3815a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:01.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.356+0000 7f9a37fff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9a38153a00 0x7f9a38153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:01.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.356+0000 7f9a377fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9a3815de50 0x7f9a3815a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:54070/0 (socket says 172.21.15.31:54070) 2023-10-06T02:27:01.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.356+0000 7f9a377fe700 1 -- 172.21.15.31:0/1765928152 learned_addr learned my addr 172.21.15.31:0/1765928152 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:01.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.356+0000 7f9a37fff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9a38153a00 0x7f9a38153dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:57716/0 (socket says 172.21.15.31:57716) 2023-10-06T02:27:01.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.356+0000 7f9a3cb9c700 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f9a380cea00 0x7f9a380cee80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:45842/0 (socket says 172.21.15.31:45842) 2023-10-06T02:27:01.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.356+0000 7f9a377fe700 1 -- 172.21.15.31:0/1765928152 >> v2:172.21.15.62:3300/0 conn(0x7f9a38153a00 msgr2=0x7f9a38153dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:01.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.356+0000 7f9a377fe700 1 --2- 172.21.15.31:0/1765928152 >> v2:172.21.15.62:3300/0 conn(0x7f9a38153a00 0x7f9a38153dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:01.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.356+0000 7f9a377fe700 1 -- 172.21.15.31:0/1765928152 >> v1:172.21.15.62:6789/0 conn(0x7f9a380cea00 legacy=0x7f9a380cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:01.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.356+0000 7f9a377fe700 1 -- 172.21.15.31:0/1765928152 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a3815b520 con 0x7f9a3815de50 2023-10-06T02:27:01.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.356+0000 7f9a37fff700 1 --2- 172.21.15.31:0/1765928152 >> v2:172.21.15.62:3300/0 conn(0x7f9a38153a00 0x7f9a38153dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:27:01.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.357+0000 7f9a377fe700 1 --2- 172.21.15.31:0/1765928152 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9a3815de50 0x7f9a3815a540 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c0099b0 tx=0x7f9a2c009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f046d853b535a8f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:01.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.357+0000 7f9a36ffd700 1 -- 172.21.15.31:0/1765928152 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c009ea0 con 0x7f9a3815de50 2023-10-06T02:27:01.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.357+0000 7f9a36ffd700 1 -- 172.21.15.31:0/1765928152 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a2c00b860 con 0x7f9a3815de50 2023-10-06T02:27:01.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.357+0000 7f9a36ffd700 1 -- 172.21.15.31:0/1765928152 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c00bc00 con 0x7f9a3815de50 2023-10-06T02:27:01.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.357+0000 7f9a3d39d700 1 -- 172.21.15.31:0/1765928152 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9a3815de50 msgr2=0x7f9a3815a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:01.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.357+0000 7f9a3d39d700 1 --2- 172.21.15.31:0/1765928152 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9a3815de50 0x7f9a3815a540 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c0099b0 tx=0x7f9a2c009250 comp rx=0 tx=0).stop 2023-10-06T02:27:01.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.357+0000 7f9a3d39d700 1 -- 172.21.15.31:0/1765928152 shutdown_connections 2023-10-06T02:27:01.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.358+0000 7f9a3d39d700 1 --2- 172.21.15.31:0/1765928152 >> v2:172.21.15.62:3300/0 conn(0x7f9a38153a00 0x7f9a38153dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:01.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.358+0000 7f9a3d39d700 1 --2- 172.21.15.31:0/1765928152 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9a3815de50 0x7f9a3815a540 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:01.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.358+0000 7f9a3d39d700 1 -- 172.21.15.31:0/1765928152 >> 172.21.15.31:0/1765928152 conn(0x7f9a380703a0 msgr2=0x7f9a380707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:01.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.358+0000 7f9a3d39d700 1 -- 172.21.15.31:0/1765928152 shutdown_connections 2023-10-06T02:27:01.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.358+0000 7f9a3d39d700 1 -- 172.21.15.31:0/1765928152 wait complete. 2023-10-06T02:27:01.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.359+0000 7f9a3d39d700 1 Processor -- start 2023-10-06T02:27:01.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.359+0000 7f9a3d39d700 1 -- start start 2023-10-06T02:27:01.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.360+0000 7f9a3d39d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a380cea00 0x7f9a38158be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:01.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.360+0000 7f9a3d39d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a38153a00 0x7f9a380c9810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:01.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.360+0000 7f9a3d39d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9a3815de50 0x7f9a380c8310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:01.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.360+0000 7f9a3d39d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a380c75e0 con 0x7f9a38153a00 2023-10-06T02:27:01.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.360+0000 7f9a3d39d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a380ca220 con 0x7f9a380cea00 2023-10-06T02:27:01.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.360+0000 7f9a3d39d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a380c7830 con 0x7f9a3815de50 2023-10-06T02:27:01.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.360+0000 7f9a377fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a38153a00 0x7f9a380c9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:01.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.360+0000 7f9a377fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a38153a00 0x7f9a380c9810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50992/0 (socket says 172.21.15.31:50992) 2023-10-06T02:27:01.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.360+0000 7f9a377fe700 1 -- 172.21.15.31:0/4243432760 learned_addr learned my addr 172.21.15.31:0/4243432760 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:01.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.361+0000 7f9a3cb9c700 1 --2- 172.21.15.31:0/4243432760 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9a3815de50 0x7f9a380c8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:01.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.361+0000 7f9a37fff700 1 --2- 172.21.15.31:0/4243432760 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a380cea00 0x7f9a38158be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:01.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.361+0000 7f9a377fe700 1 -- 172.21.15.31:0/4243432760 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9a3815de50 msgr2=0x7f9a380c8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:01.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.361+0000 7f9a377fe700 1 --2- 172.21.15.31:0/4243432760 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9a3815de50 0x7f9a380c8310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:01.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.361+0000 7f9a377fe700 1 -- 172.21.15.31:0/4243432760 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a380cea00 msgr2=0x7f9a38158be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:01.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.361+0000 7f9a377fe700 1 --2- 172.21.15.31:0/4243432760 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a380cea00 0x7f9a38158be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:01.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.361+0000 7f9a377fe700 1 -- 172.21.15.31:0/4243432760 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a28009420 con 0x7f9a38153a00 2023-10-06T02:27:01.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.361+0000 7f9a377fe700 1 --2- 172.21.15.31:0/4243432760 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a38153a00 0x7f9a380c9810 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f9a3815b380 tx=0x7f9a2c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:01.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.361+0000 7f9a1ffff700 1 -- 172.21.15.31:0/4243432760 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c000d10 con 0x7f9a38153a00 2023-10-06T02:27:01.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.361+0000 7f9a1ffff700 1 -- 172.21.15.31:0/4243432760 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a2c00bbe0 con 0x7f9a38153a00 2023-10-06T02:27:01.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.362+0000 7f9a1ffff700 1 -- 172.21.15.31:0/4243432760 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c025670 con 0x7f9a38153a00 2023-10-06T02:27:01.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.362+0000 7f9a3d39d700 1 -- 172.21.15.31:0/4243432760 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a2c0094f0 con 0x7f9a38153a00 2023-10-06T02:27:01.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.362+0000 7f9a3d39d700 1 -- 172.21.15.31:0/4243432760 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a380c7e70 con 0x7f9a38153a00 2023-10-06T02:27:01.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.363+0000 7f9a3d39d700 1 -- 172.21.15.31:0/4243432760 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a380c8000 con 0x7f9a38153a00 2023-10-06T02:27:01.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.364+0000 7f9a1ffff700 1 -- 172.21.15.31:0/4243432760 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f9a2c02e440 con 0x7f9a38153a00 2023-10-06T02:27:01.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.364+0000 7f9a1ffff700 1 --2- 172.21.15.31:0/4243432760 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f9a24067170 0x7f9a24069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:01.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.364+0000 7f9a1ffff700 1 -- 172.21.15.31:0/4243432760 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6203+0+0 (secure 0 0 0) 0x7f9a2c0866d0 con 0x7f9a38153a00 2023-10-06T02:27:01.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.365+0000 7f9a37fff700 1 --2- 172.21.15.31:0/4243432760 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f9a24067170 0x7f9a24069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:01.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.365+0000 7f9a37fff700 1 --2- 172.21.15.31:0/4243432760 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f9a24067170 0x7f9a24069620 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f9a28009f40 tx=0x7f9a280091a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:01.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.369+0000 7f9a1ffff700 1 -- 172.21.15.31:0/4243432760 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9a2c0534a0 con 0x7f9a38153a00 2023-10-06T02:27:01.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.603+0000 7f9a3d39d700 1 -- 172.21.15.31:0/4243432760 --> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000626507", "target": ["mon-mgr", ""]}) v1 -- 0x7f9a3804c3c0 con 0x7f9a24067170 2023-10-06T02:27:01.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.607+0000 7f9a1ffff700 1 -- 172.21.15.31:0/4243432760 <== mgr.4107 v2:172.21.15.31:6832/85292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+45 (secure 0 0 0) 0x7f9a3804c3c0 con 0x7f9a24067170 2023-10-06T02:27:01.608 INFO:teuthology.orchestra.run.smithi031.stdout:/volumes/_nogroup/subvolume_0000000000626507 2023-10-06T02:27:01.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.610+0000 7f9a3d39d700 1 -- 172.21.15.31:0/4243432760 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f9a24067170 msgr2=0x7f9a24069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:01.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.610+0000 7f9a3d39d700 1 --2- 172.21.15.31:0/4243432760 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f9a24067170 0x7f9a24069620 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f9a28009f40 tx=0x7f9a280091a0 comp rx=0 tx=0).stop 2023-10-06T02:27:01.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.610+0000 7f9a3d39d700 1 -- 172.21.15.31:0/4243432760 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a38153a00 msgr2=0x7f9a380c9810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:01.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.610+0000 7f9a3d39d700 1 --2- 172.21.15.31:0/4243432760 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a38153a00 0x7f9a380c9810 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f9a3815b380 tx=0x7f9a2c017000 comp rx=0 tx=0).stop 2023-10-06T02:27:01.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.610+0000 7f9a3d39d700 1 -- 172.21.15.31:0/4243432760 shutdown_connections 2023-10-06T02:27:01.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.610+0000 7f9a3d39d700 1 --2- 172.21.15.31:0/4243432760 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f9a24067170 0x7f9a24069620 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:01.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.610+0000 7f9a3d39d700 1 --2- 172.21.15.31:0/4243432760 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9a3815de50 0x7f9a380c8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:01.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.610+0000 7f9a3d39d700 1 --2- 172.21.15.31:0/4243432760 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a380cea00 0x7f9a38158be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:01.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.610+0000 7f9a3d39d700 1 --2- 172.21.15.31:0/4243432760 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a38153a00 0x7f9a380c9810 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:01.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.610+0000 7f9a3d39d700 1 -- 172.21.15.31:0/4243432760 >> 172.21.15.31:0/4243432760 conn(0x7f9a380703a0 msgr2=0x7f9a380d0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:01.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.610+0000 7f9a3d39d700 1 -- 172.21.15.31:0/4243432760 shutdown_connections 2023-10-06T02:27:01.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.610+0000 7f9a3d39d700 1 -- 172.21.15.31:0/4243432760 wait complete. 2023-10-06T02:27:01.627 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:27:01.628 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_legacy/418e0c85c330fb76e916f2ccb01c1dcd.meta) 2023-10-06T02:27:01.697 INFO:teuthology.orchestra.run.smithi031.stdout:[GLOBAL] 2023-10-06T02:27:01.697 INFO:teuthology.orchestra.run.smithi031.stdout:version = 1 2023-10-06T02:27:01.697 INFO:teuthology.orchestra.run.smithi031.stdout:type = subvolume 2023-10-06T02:27:01.697 INFO:teuthology.orchestra.run.smithi031.stdout:path = /volumes/_nogroup/subvolume_0000000000626507 2023-10-06T02:27:01.697 INFO:teuthology.orchestra.run.smithi031.stdout:state = complete 2023-10-06T02:27:01.697 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:27:01.702 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume authorize cephfs subvolume_0000000000626507 alice 2023-10-06T02:27:01.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.893+0000 7f8cd09fe700 1 Processor -- start 2023-10-06T02:27:01.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.893+0000 7f8cd09fe700 1 -- start start 2023-10-06T02:27:01.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.894+0000 7f8cd09fe700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8ccc0c8200 0x7f8ccc0c1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:01.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.894+0000 7f8cd09fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8ccc0c85d0 0x7f8ccc0cc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:01.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.894+0000 7f8cd09fe700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ccc1571b0 con 0x7f8ccc0c2a90 2023-10-06T02:27:01.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.894+0000 7f8cd09fe700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ccc1579b0 con 0x7f8ccc0c85d0 2023-10-06T02:27:01.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.894+0000 7f8cd09fe700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ccc0c1390 con 0x7f8ccc0c8200 2023-10-06T02:27:01.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.894+0000 7f8ccb59e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8ccc0c85d0 0x7f8ccc0cc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:01.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.894+0000 7f8ccb59e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8ccc0c85d0 0x7f8ccc0cc2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50994/0 (socket says 172.21.15.31:50994) 2023-10-06T02:27:01.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.894+0000 7f8ccb59e700 1 -- 172.21.15.31:0/611287645 learned_addr learned my addr 172.21.15.31:0/611287645 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:01.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.894+0000 7f8ccad9d700 1 --2- 172.21.15.31:0/611287645 >> v2:172.21.15.62:3300/0 conn(0x7f8ccc0c8200 0x7f8ccc0c1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:01.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.895+0000 7f8ccb59e700 1 -- 172.21.15.31:0/611287645 >> v2:172.21.15.62:3300/0 conn(0x7f8ccc0c8200 msgr2=0x7f8ccc0c1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:01.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.895+0000 7f8ccb59e700 1 --2- 172.21.15.31:0/611287645 >> v2:172.21.15.62:3300/0 conn(0x7f8ccc0c8200 0x7f8ccc0c1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:01.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.895+0000 7f8ccb59e700 1 -- 172.21.15.31:0/611287645 >> v1:172.21.15.62:6789/0 conn(0x7f8ccc0c2a90 legacy=0x7f8ccc156aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:01.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.895+0000 7f8ccb59e700 1 -- 172.21.15.31:0/611287645 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ccc1573f0 con 0x7f8ccc0c85d0 2023-10-06T02:27:01.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.895+0000 7f8ccb59e700 1 --2- 172.21.15.31:0/611287645 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ccc0c85d0 0x7f8ccc0cc2f0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc0683d0 tx=0x7f8cc4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81e824de57fa92f7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:01.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.895+0000 7f8cc9d9b700 1 -- 172.21.15.31:0/611287645 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc400e070 con 0x7f8ccc0c85d0 2023-10-06T02:27:01.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.895+0000 7f8cc9d9b700 1 -- 172.21.15.31:0/611287645 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cc400ba20 con 0x7f8ccc0c85d0 2023-10-06T02:27:01.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.895+0000 7f8cc9d9b700 1 -- 172.21.15.31:0/611287645 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc40092e0 con 0x7f8ccc0c85d0 2023-10-06T02:27:01.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.896+0000 7f8cd09fe700 1 -- 172.21.15.31:0/611287645 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ccc0c85d0 msgr2=0x7f8ccc0cc2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:01.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.896+0000 7f8cd09fe700 1 --2- 172.21.15.31:0/611287645 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ccc0c85d0 0x7f8ccc0cc2f0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc0683d0 tx=0x7f8cc4009840 comp rx=0 tx=0).stop 2023-10-06T02:27:01.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.896+0000 7f8cd09fe700 1 -- 172.21.15.31:0/611287645 shutdown_connections 2023-10-06T02:27:01.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.896+0000 7f8cd09fe700 1 --2- 172.21.15.31:0/611287645 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ccc0c85d0 0x7f8ccc0cc2f0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:01.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.896+0000 7f8cd09fe700 1 --2- 172.21.15.31:0/611287645 >> v2:172.21.15.62:3300/0 conn(0x7f8ccc0c8200 0x7f8ccc0c1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:01.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.896+0000 7f8cd09fe700 1 -- 172.21.15.31:0/611287645 >> 172.21.15.31:0/611287645 conn(0x7f8ccc06c6a0 msgr2=0x7f8ccc06caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:01.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.896+0000 7f8cd09fe700 1 -- 172.21.15.31:0/611287645 shutdown_connections 2023-10-06T02:27:01.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.896+0000 7f8cd09fe700 1 -- 172.21.15.31:0/611287645 wait complete. 2023-10-06T02:27:01.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.897+0000 7f8cd09fe700 1 Processor -- start 2023-10-06T02:27:01.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.898+0000 7f8cd09fe700 1 -- start start 2023-10-06T02:27:01.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.898+0000 7f8cd09fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ccc0c2a90 0x7f8ccc0cc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:01.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.898+0000 7f8cd09fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ccc0c8200 0x7f8ccc0ca8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:01.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.898+0000 7f8cd09fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8ccc0c85d0 0x7f8ccc13c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:01.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.898+0000 7f8cd09fe700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ccc0ccf00 con 0x7f8ccc0c8200 2023-10-06T02:27:01.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.898+0000 7f8cd09fe700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ccc0ca460 con 0x7f8ccc0c2a90 2023-10-06T02:27:01.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.898+0000 7f8cd09fe700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ccc0cc940 con 0x7f8ccc0c85d0 2023-10-06T02:27:01.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.899+0000 7f8cca59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ccc0c8200 0x7f8ccc0ca8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:01.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.899+0000 7f8cca59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ccc0c8200 0x7f8ccc0ca8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51008/0 (socket says 172.21.15.31:51008) 2023-10-06T02:27:01.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.899+0000 7f8cca59c700 1 -- 172.21.15.31:0/2573679211 learned_addr learned my addr 172.21.15.31:0/2573679211 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:01.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.899+0000 7f8ccad9d700 1 --2- 172.21.15.31:0/2573679211 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ccc0c2a90 0x7f8ccc0cc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:01.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.899+0000 7f8ccb59e700 1 --2- 172.21.15.31:0/2573679211 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8ccc0c85d0 0x7f8ccc13c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:01.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.899+0000 7f8cca59c700 1 -- 172.21.15.31:0/2573679211 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8ccc0c85d0 msgr2=0x7f8ccc13c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:01.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.899+0000 7f8cca59c700 1 --2- 172.21.15.31:0/2573679211 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8ccc0c85d0 0x7f8ccc13c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:01.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.899+0000 7f8cca59c700 1 -- 172.21.15.31:0/2573679211 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ccc0c2a90 msgr2=0x7f8ccc0cc0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:01.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.899+0000 7f8cca59c700 1 --2- 172.21.15.31:0/2573679211 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ccc0c2a90 0x7f8ccc0cc0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:01.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.899+0000 7f8cca59c700 1 -- 172.21.15.31:0/2573679211 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cc4009ae0 con 0x7f8ccc0c8200 2023-10-06T02:27:01.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.899+0000 7f8cca59c700 1 --2- 172.21.15.31:0/2573679211 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ccc0c8200 0x7f8ccc0ca8f0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f8cc000c7c0 tx=0x7f8cc0007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:01.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.900+0000 7f8cb2ffd700 1 -- 172.21.15.31:0/2573679211 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc0011590 con 0x7f8ccc0c8200 2023-10-06T02:27:01.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.900+0000 7f8cb2ffd700 1 -- 172.21.15.31:0/2573679211 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cc00116f0 con 0x7f8ccc0c8200 2023-10-06T02:27:01.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.900+0000 7f8cb2ffd700 1 -- 172.21.15.31:0/2573679211 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc0011ab0 con 0x7f8ccc0c8200 2023-10-06T02:27:01.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.900+0000 7f8cd09fe700 1 -- 172.21.15.31:0/2573679211 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ccc0ccc20 con 0x7f8ccc0c8200 2023-10-06T02:27:01.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.900+0000 7f8cd09fe700 1 -- 172.21.15.31:0/2573679211 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ccc0cc630 con 0x7f8ccc0c8200 2023-10-06T02:27:01.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.901+0000 7f8cd09fe700 1 -- 172.21.15.31:0/2573679211 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ccc04c3c0 con 0x7f8ccc0c8200 2023-10-06T02:27:01.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.902+0000 7f8cb2ffd700 1 -- 172.21.15.31:0/2573679211 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f8cc001b3b0 con 0x7f8ccc0c8200 2023-10-06T02:27:01.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.903+0000 7f8cb2ffd700 1 --2- 172.21.15.31:0/2573679211 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f8cb8067170 0x7f8cb8069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:01.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.903+0000 7f8cb2ffd700 1 -- 172.21.15.31:0/2573679211 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6203+0+0 (secure 0 0 0) 0x7f8cc0083e00 con 0x7f8ccc0c8200 2023-10-06T02:27:01.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.903+0000 7f8ccad9d700 1 --2- 172.21.15.31:0/2573679211 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f8cb8067170 0x7f8cb8069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:01.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.904+0000 7f8ccad9d700 1 --2- 172.21.15.31:0/2573679211 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f8cb8067170 0x7f8cb8069620 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc009d40 tx=0x7f8cbc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:01.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:01.907+0000 7f8cb2ffd700 1 -- 172.21.15.31:0/2573679211 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8cc0050bf0 con 0x7f8ccc0c8200 2023-10-06T02:27:02.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.142+0000 7f8cd09fe700 1 -- 172.21.15.31:0/2573679211 --> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] -- mgr_command(tid 0: {"prefix": "fs subvolume authorize", "vol_name": "cephfs", "sub_name": "subvolume_0000000000626507", "auth_id": "alice", "target": ["mon-mgr", ""]}) v1 -- 0x7f8ccc1571b0 con 0x7f8cb8067170 2023-10-06T02:27:02.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.187+0000 7f8cb2ffd700 1 -- 172.21.15.31:0/2573679211 <== mgr.4107 v2:172.21.15.31:6832/85292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f8ccc1571b0 con 0x7f8cb8067170 2023-10-06T02:27:02.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.189+0000 7f8cd09fe700 1 -- 172.21.15.31:0/2573679211 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f8cb8067170 msgr2=0x7f8cb8069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:02.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.189+0000 7f8cd09fe700 1 --2- 172.21.15.31:0/2573679211 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f8cb8067170 0x7f8cb8069620 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc009d40 tx=0x7f8cbc008040 comp rx=0 tx=0).stop 2023-10-06T02:27:02.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.189+0000 7f8cd09fe700 1 -- 172.21.15.31:0/2573679211 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ccc0c8200 msgr2=0x7f8ccc0ca8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:02.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.189+0000 7f8cd09fe700 1 --2- 172.21.15.31:0/2573679211 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ccc0c8200 0x7f8ccc0ca8f0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f8cc000c7c0 tx=0x7f8cc0007f50 comp rx=0 tx=0).stop 2023-10-06T02:27:02.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.189+0000 7f8cd09fe700 1 -- 172.21.15.31:0/2573679211 shutdown_connections 2023-10-06T02:27:02.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.189+0000 7f8cd09fe700 1 --2- 172.21.15.31:0/2573679211 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f8cb8067170 0x7f8cb8069620 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:02.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.190+0000 7f8cd09fe700 1 --2- 172.21.15.31:0/2573679211 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8ccc0c85d0 0x7f8ccc13c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:02.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.190+0000 7f8cd09fe700 1 --2- 172.21.15.31:0/2573679211 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ccc0c2a90 0x7f8ccc0cc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:02.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.190+0000 7f8cd09fe700 1 --2- 172.21.15.31:0/2573679211 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ccc0c8200 0x7f8ccc0ca8f0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:02.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.190+0000 7f8cd09fe700 1 -- 172.21.15.31:0/2573679211 >> 172.21.15.31:0/2573679211 conn(0x7f8ccc06c6a0 msgr2=0x7f8ccc0c9180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:02.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.190+0000 7f8cd09fe700 1 -- 172.21.15.31:0/2573679211 shutdown_connections 2023-10-06T02:27:02.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.190+0000 7f8cd09fe700 1 -- 172.21.15.31:0/2573679211 wait complete. 2023-10-06T02:27:02.208 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth get client.alice --format=json-pretty 2023-10-06T02:27:02.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.396+0000 7f68d219d700 1 Processor -- start 2023-10-06T02:27:02.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.396+0000 7f68d219d700 1 -- start start 2023-10-06T02:27:02.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.398+0000 7f68d219d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f68cc159640 0x7f68cc06d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:02.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.398+0000 7f68d219d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f68cc0caec0 0x7f68cc0c78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:02.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.398+0000 7f68d219d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68cc0c7420 con 0x7f68cc159a10 2023-10-06T02:27:02.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.398+0000 7f68d219d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68cc06d5a0 con 0x7f68cc0caec0 2023-10-06T02:27:02.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.398+0000 7f68d219d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68cc06d820 con 0x7f68cc159640 2023-10-06T02:27:02.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.398+0000 7f68d199c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f68cc0caec0 0x7f68cc0c78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:02.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.398+0000 7f68d099a700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f68cc159a10 0x7f68cc0ca7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:38214/0 (socket says 172.21.15.31:38214) 2023-10-06T02:27:02.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.398+0000 7f68d099a700 1 -- 172.21.15.31:0/3915663009 learned_addr learned my addr 172.21.15.31:0/3915663009 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:27:02.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.398+0000 7f68d119b700 1 --2- 172.21.15.31:0/3915663009 >> v2:172.21.15.62:3300/0 conn(0x7f68cc159640 0x7f68cc06d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:02.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.398+0000 7f68d199c700 1 -- 172.21.15.31:0/3915663009 >> v2:172.21.15.62:3300/0 conn(0x7f68cc159640 msgr2=0x7f68cc06d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:02.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.398+0000 7f68d199c700 1 --2- 172.21.15.31:0/3915663009 >> v2:172.21.15.62:3300/0 conn(0x7f68cc159640 0x7f68cc06d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:02.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.398+0000 7f68d199c700 1 -- 172.21.15.31:0/3915663009 >> v1:172.21.15.31:6789/0 conn(0x7f68cc159a10 legacy=0x7f68cc0ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:02.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.398+0000 7f68d199c700 1 -- 172.21.15.31:0/3915663009 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68cc06db00 con 0x7f68cc0caec0 2023-10-06T02:27:02.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.399+0000 7f68d199c700 1 --2- 172.21.15.31:0/3915663009 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68cc0caec0 0x7f68cc0c78b0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f68c4009e60 tx=0x7f68c4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d2b4e57179bcb5e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:02.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.399+0000 7f68c3fff700 1 -- 172.21.15.31:0/3915663009 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c400e070 con 0x7f68cc0caec0 2023-10-06T02:27:02.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.399+0000 7f68c3fff700 1 -- 172.21.15.31:0/3915663009 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68c400bbf0 con 0x7f68cc0caec0 2023-10-06T02:27:02.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.399+0000 7f68c3fff700 1 -- 172.21.15.31:0/3915663009 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c4012430 con 0x7f68cc0caec0 2023-10-06T02:27:02.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.399+0000 7f68d219d700 1 -- 172.21.15.31:0/3915663009 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68cc0caec0 msgr2=0x7f68cc0c78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:02.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.399+0000 7f68d219d700 1 --2- 172.21.15.31:0/3915663009 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68cc0caec0 0x7f68cc0c78b0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f68c4009e60 tx=0x7f68c4009780 comp rx=0 tx=0).stop 2023-10-06T02:27:02.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.400+0000 7f68d219d700 1 -- 172.21.15.31:0/3915663009 shutdown_connections 2023-10-06T02:27:02.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.400+0000 7f68d219d700 1 --2- 172.21.15.31:0/3915663009 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68cc0caec0 0x7f68cc0c78b0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:02.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.400+0000 7f68d219d700 1 --2- 172.21.15.31:0/3915663009 >> v2:172.21.15.62:3300/0 conn(0x7f68cc159640 0x7f68cc06d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:02.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.400+0000 7f68d219d700 1 -- 172.21.15.31:0/3915663009 >> 172.21.15.31:0/3915663009 conn(0x7f68cc06e720 msgr2=0x7f68cc06eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:02.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.400+0000 7f68d219d700 1 -- 172.21.15.31:0/3915663009 shutdown_connections 2023-10-06T02:27:02.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.400+0000 7f68d219d700 1 -- 172.21.15.31:0/3915663009 wait complete. 2023-10-06T02:27:02.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.401+0000 7f68d219d700 1 Processor -- start 2023-10-06T02:27:02.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.401+0000 7f68d219d700 1 -- start start 2023-10-06T02:27:02.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.402+0000 7f68d219d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68cc0caec0 0x7f68cc13bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:02.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.402+0000 7f68d219d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f68cc159640 0x7f68cc1518b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:02.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.402+0000 7f68d219d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68cc159a10 0x7f68cc151df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:02.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.402+0000 7f68d219d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68cc1504f0 con 0x7f68cc159a10 2023-10-06T02:27:02.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.402+0000 7f68d219d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68cc152d10 con 0x7f68cc0caec0 2023-10-06T02:27:02.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.402+0000 7f68d219d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68cc0c5010 con 0x7f68cc159640 2023-10-06T02:27:02.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.402+0000 7f68d199c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68cc159a10 0x7f68cc151df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:02.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.402+0000 7f68d199c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68cc159a10 0x7f68cc151df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51020/0 (socket says 172.21.15.31:51020) 2023-10-06T02:27:02.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.402+0000 7f68d199c700 1 -- 172.21.15.31:0/977866186 learned_addr learned my addr 172.21.15.31:0/977866186 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:02.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.403+0000 7f68d119b700 1 --2- 172.21.15.31:0/977866186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68cc0caec0 0x7f68cc13bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:02.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.403+0000 7f68d099a700 1 --2- 172.21.15.31:0/977866186 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f68cc159640 0x7f68cc1518b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:02.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.403+0000 7f68d199c700 1 -- 172.21.15.31:0/977866186 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f68cc159640 msgr2=0x7f68cc1518b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:02.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.403+0000 7f68d199c700 1 --2- 172.21.15.31:0/977866186 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f68cc159640 0x7f68cc1518b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:02.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.403+0000 7f68d199c700 1 -- 172.21.15.31:0/977866186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68cc0caec0 msgr2=0x7f68cc13bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:02.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.403+0000 7f68d199c700 1 --2- 172.21.15.31:0/977866186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68cc0caec0 0x7f68cc13bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:02.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.403+0000 7f68d199c700 1 -- 172.21.15.31:0/977866186 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68c4009a20 con 0x7f68cc159a10 2023-10-06T02:27:02.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.403+0000 7f68d119b700 1 --2- 172.21.15.31:0/977866186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68cc0caec0 0x7f68cc13bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:27:02.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.403+0000 7f68d199c700 1 --2- 172.21.15.31:0/977866186 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68cc159a10 0x7f68cc151df0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f68cc0c3820 tx=0x7f68c4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:02.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.403+0000 7f68c17fa700 1 -- 172.21.15.31:0/977866186 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c4012e30 con 0x7f68cc159a10 2023-10-06T02:27:02.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.403+0000 7f68c17fa700 1 -- 172.21.15.31:0/977866186 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68c401e630 con 0x7f68cc159a10 2023-10-06T02:27:02.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.403+0000 7f68d219d700 1 -- 172.21.15.31:0/977866186 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68cc152470 con 0x7f68cc159a10 2023-10-06T02:27:02.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.403+0000 7f68c17fa700 1 -- 172.21.15.31:0/977866186 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c401e9f0 con 0x7f68cc159a10 2023-10-06T02:27:02.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.403+0000 7f68d219d700 1 -- 172.21.15.31:0/977866186 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68cc152a30 con 0x7f68cc159a10 2023-10-06T02:27:02.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.404+0000 7f68d219d700 1 -- 172.21.15.31:0/977866186 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68cc04c3c0 con 0x7f68cc159a10 2023-10-06T02:27:02.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.405+0000 7f68c17fa700 1 -- 172.21.15.31:0/977866186 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f68c4024070 con 0x7f68cc159a10 2023-10-06T02:27:02.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.406+0000 7f68c17fa700 1 --2- 172.21.15.31:0/977866186 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f68b00671a0 0x7f68b0069650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:02.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.406+0000 7f68c17fa700 1 -- 172.21.15.31:0/977866186 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6203+0+0 (secure 0 0 0) 0x7f68c4087200 con 0x7f68cc159a10 2023-10-06T02:27:02.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.406+0000 7f68d119b700 1 --2- 172.21.15.31:0/977866186 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f68b00671a0 0x7f68b0069650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:02.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.407+0000 7f68d119b700 1 --2- 172.21.15.31:0/977866186 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f68b00671a0 0x7f68b0069650 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f68c8006fd0 tx=0x7f68c8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:02.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.410+0000 7f68c17fa700 1 -- 172.21.15.31:0/977866186 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f68c4053f50 con 0x7f68cc159a10 2023-10-06T02:27:02.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.691+0000 7f68d219d700 1 -- 172.21.15.31:0/977866186 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth get", "entity": "client.alice", "format": "json-pretty"} v 0) v1 -- 0x7f68cc0c5150 con 0x7f68cc159a10 2023-10-06T02:27:02.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.692+0000 7f68c17fa700 1 -- 172.21.15.31:0/977866186 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth get", "entity": "client.alice", "format": "json-pretty"}]=0 v14) v1 ==== 107+0+295 (secure 0 0 0) 0x7f68c4009880 con 0x7f68cc159a10 2023-10-06T02:27:02.693 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:27:02.693 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2023-10-06T02:27:02.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:02.693 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.alice", 2023-10-06T02:27:02.693 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQB2cB9l3OHmCRAAmc2GdO1sB1j4/XfS9DAw4g==", 2023-10-06T02:27:02.693 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:27:02.693 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow rw path=/volumes/_nogroup/subvolume_0000000000626507", 2023-10-06T02:27:02.693 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow r", 2023-10-06T02:27:02.693 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rw pool=cephfs_data" 2023-10-06T02:27:02.694 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:02.694 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:02.694 INFO:teuthology.orchestra.run.smithi031.stdout:] 2023-10-06T02:27:02.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.694+0000 7f68d219d700 1 -- 172.21.15.31:0/977866186 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f68b00671a0 msgr2=0x7f68b0069650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:02.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.694+0000 7f68d219d700 1 --2- 172.21.15.31:0/977866186 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f68b00671a0 0x7f68b0069650 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f68c8006fd0 tx=0x7f68c8008040 comp rx=0 tx=0).stop 2023-10-06T02:27:02.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.694+0000 7f68d219d700 1 -- 172.21.15.31:0/977866186 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68cc159a10 msgr2=0x7f68cc151df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:02.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.694+0000 7f68d219d700 1 --2- 172.21.15.31:0/977866186 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68cc159a10 0x7f68cc151df0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f68cc0c3820 tx=0x7f68c4019000 comp rx=0 tx=0).stop 2023-10-06T02:27:02.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.694+0000 7f68d219d700 1 -- 172.21.15.31:0/977866186 shutdown_connections 2023-10-06T02:27:02.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.694+0000 7f68d219d700 1 --2- 172.21.15.31:0/977866186 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68cc159a10 0x7f68cc151df0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:02.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.694+0000 7f68d219d700 1 --2- 172.21.15.31:0/977866186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68cc0caec0 0x7f68cc13bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:02.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.694+0000 7f68d219d700 1 --2- 172.21.15.31:0/977866186 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f68b00671a0 0x7f68b0069650 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:02.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.694+0000 7f68d219d700 1 --2- 172.21.15.31:0/977866186 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f68cc159640 0x7f68cc1518b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:02.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.694+0000 7f68d219d700 1 -- 172.21.15.31:0/977866186 >> 172.21.15.31:0/977866186 conn(0x7f68cc06e720 msgr2=0x7f68cc157990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:02.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.694+0000 7f68d219d700 1 -- 172.21.15.31:0/977866186 shutdown_connections 2023-10-06T02:27:02.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.694+0000 7f68d219d700 1 -- 172.21.15.31:0/977866186 wait complete. 2023-10-06T02:27:02.712 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000626507 2023-10-06T02:27:02.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.902+0000 7f52bd19d700 1 Processor -- start 2023-10-06T02:27:02.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.903+0000 7f52bd19d700 1 -- start start 2023-10-06T02:27:02.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.904+0000 7f52bd19d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f52b80c7be0 0x7f52b806abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:02.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.904+0000 7f52bd19d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f52b813f780 0x7f52b813fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:02.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.904+0000 7f52bd19d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52b806b290 con 0x7f52b80c7fb0 2023-10-06T02:27:02.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.904+0000 7f52bd19d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52b806bf10 con 0x7f52b813f780 2023-10-06T02:27:02.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.904+0000 7f52bd19d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52b8140140 con 0x7f52b80c7be0 2023-10-06T02:27:02.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.904+0000 7f52bc99c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f52b813f780 0x7f52b813fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:02.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.904+0000 7f52bc99c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f52b813f780 0x7f52b813fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51034/0 (socket says 172.21.15.31:51034) 2023-10-06T02:27:02.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.904+0000 7f52bc99c700 1 -- 172.21.15.31:0/4239920623 learned_addr learned my addr 172.21.15.31:0/4239920623 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:02.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.904+0000 7f52b7fff700 1 --2- 172.21.15.31:0/4239920623 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f52b80c7be0 0x7f52b806abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:02.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.904+0000 7f52bc99c700 1 -- 172.21.15.31:0/4239920623 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f52b80c7be0 msgr2=0x7f52b806abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:02.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.904+0000 7f52bc99c700 1 --2- 172.21.15.31:0/4239920623 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f52b80c7be0 0x7f52b806abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:02.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.905+0000 7f52bc99c700 1 -- 172.21.15.31:0/4239920623 >> v1:172.21.15.62:6789/0 conn(0x7f52b80c7fb0 legacy=0x7f52b815a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:02.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.905+0000 7f52bc99c700 1 -- 172.21.15.31:0/4239920623 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52b806b570 con 0x7f52b813f780 2023-10-06T02:27:02.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.905+0000 7f52b7fff700 1 --2- 172.21.15.31:0/4239920623 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f52b80c7be0 0x7f52b806abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:27:02.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.905+0000 7f52bc99c700 1 --2- 172.21.15.31:0/4239920623 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52b813f780 0x7f52b813fc00 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f52b0009f50 tx=0x7f52b0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41a3cdb94ba06c67 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:02.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.905+0000 7f52b6ffd700 1 -- 172.21.15.31:0/4239920623 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b000e070 con 0x7f52b813f780 2023-10-06T02:27:02.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.905+0000 7f52b6ffd700 1 -- 172.21.15.31:0/4239920623 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52b000ba30 con 0x7f52b813f780 2023-10-06T02:27:02.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.905+0000 7f52b6ffd700 1 -- 172.21.15.31:0/4239920623 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b00092e0 con 0x7f52b813f780 2023-10-06T02:27:02.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.906+0000 7f52bd19d700 1 -- 172.21.15.31:0/4239920623 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52b813f780 msgr2=0x7f52b813fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:02.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.906+0000 7f52bd19d700 1 --2- 172.21.15.31:0/4239920623 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52b813f780 0x7f52b813fc00 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f52b0009f50 tx=0x7f52b0009840 comp rx=0 tx=0).stop 2023-10-06T02:27:02.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.906+0000 7f52bd19d700 1 -- 172.21.15.31:0/4239920623 shutdown_connections 2023-10-06T02:27:02.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.906+0000 7f52bd19d700 1 --2- 172.21.15.31:0/4239920623 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52b813f780 0x7f52b813fc00 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:02.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.906+0000 7f52bd19d700 1 --2- 172.21.15.31:0/4239920623 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f52b80c7be0 0x7f52b806abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:02.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.906+0000 7f52bd19d700 1 -- 172.21.15.31:0/4239920623 >> 172.21.15.31:0/4239920623 conn(0x7f52b806d110 msgr2=0x7f52b80cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:02.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.906+0000 7f52bd19d700 1 -- 172.21.15.31:0/4239920623 shutdown_connections 2023-10-06T02:27:02.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.906+0000 7f52bd19d700 1 -- 172.21.15.31:0/4239920623 wait complete. 2023-10-06T02:27:02.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.907+0000 7f52bd19d700 1 Processor -- start 2023-10-06T02:27:02.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.907+0000 7f52bd19d700 1 -- start start 2023-10-06T02:27:02.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.908+0000 7f52bd19d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52b80c7be0 0x7f52b8167400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:02.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.908+0000 7f52bd19d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f52b80c7fb0 0x7f52b80add60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:02.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.908+0000 7f52bd19d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52b813f780 0x7f52b80b09e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:02.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.908+0000 7f52bd19d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52b81568c0 con 0x7f52b80c7be0 2023-10-06T02:27:02.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.908+0000 7f52bd19d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52b8156b80 con 0x7f52b813f780 2023-10-06T02:27:02.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.908+0000 7f52bd19d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52b8156e60 con 0x7f52b80c7fb0 2023-10-06T02:27:02.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.909+0000 7f52b7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52b80c7be0 0x7f52b8167400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:02.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.909+0000 7f52b7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52b80c7be0 0x7f52b8167400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51036/0 (socket says 172.21.15.31:51036) 2023-10-06T02:27:02.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.909+0000 7f52b7fff700 1 -- 172.21.15.31:0/2960173999 learned_addr learned my addr 172.21.15.31:0/2960173999 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:02.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.909+0000 7f52bc99c700 1 --2- 172.21.15.31:0/2960173999 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52b813f780 0x7f52b80b09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:02.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.909+0000 7f52b77fe700 1 --2- 172.21.15.31:0/2960173999 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f52b80c7fb0 0x7f52b80add60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:02.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.909+0000 7f52b7fff700 1 -- 172.21.15.31:0/2960173999 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f52b80c7fb0 msgr2=0x7f52b80add60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:02.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.909+0000 7f52b7fff700 1 --2- 172.21.15.31:0/2960173999 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f52b80c7fb0 0x7f52b80add60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:02.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.909+0000 7f52b7fff700 1 -- 172.21.15.31:0/2960173999 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52b813f780 msgr2=0x7f52b80b09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:02.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.909+0000 7f52b7fff700 1 --2- 172.21.15.31:0/2960173999 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52b813f780 0x7f52b80b09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:02.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.909+0000 7f52b7fff700 1 -- 172.21.15.31:0/2960173999 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52b0009ae0 con 0x7f52b80c7be0 2023-10-06T02:27:02.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.909+0000 7f52bc99c700 1 --2- 172.21.15.31:0/2960173999 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52b813f780 0x7f52b80b09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:27:02.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.909+0000 7f52b77fe700 1 --2- 172.21.15.31:0/2960173999 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f52b80c7fb0 0x7f52b80add60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:27:02.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.909+0000 7f52b7fff700 1 --2- 172.21.15.31:0/2960173999 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52b80c7be0 0x7f52b8167400 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f52a800c830 tx=0x7f52a8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:02.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.909+0000 7f529ffff700 1 -- 172.21.15.31:0/2960173999 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a8011600 con 0x7f52b80c7be0 2023-10-06T02:27:02.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.910+0000 7f529ffff700 1 -- 172.21.15.31:0/2960173999 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52a8011760 con 0x7f52b80c7be0 2023-10-06T02:27:02.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.910+0000 7f529ffff700 1 -- 172.21.15.31:0/2960173999 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a8011b20 con 0x7f52b80c7be0 2023-10-06T02:27:02.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.910+0000 7f52bd19d700 1 -- 172.21.15.31:0/2960173999 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52b8157140 con 0x7f52b80c7be0 2023-10-06T02:27:02.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.910+0000 7f52bd19d700 1 -- 172.21.15.31:0/2960173999 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52b80ae2a0 con 0x7f52b80c7be0 2023-10-06T02:27:02.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.911+0000 7f52bd19d700 1 -- 172.21.15.31:0/2960173999 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52b804c3c0 con 0x7f52b80c7be0 2023-10-06T02:27:02.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.912+0000 7f529ffff700 1 -- 172.21.15.31:0/2960173999 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f52a800a3e0 con 0x7f52b80c7be0 2023-10-06T02:27:02.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.912+0000 7f529ffff700 1 --2- 172.21.15.31:0/2960173999 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f52a40671a0 0x7f52a4069650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:02.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.912+0000 7f529ffff700 1 -- 172.21.15.31:0/2960173999 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6203+0+0 (secure 0 0 0) 0x7f52a8084c20 con 0x7f52b80c7be0 2023-10-06T02:27:02.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.913+0000 7f52b77fe700 1 --2- 172.21.15.31:0/2960173999 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f52a40671a0 0x7f52a4069650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:02.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.913+0000 7f52b77fe700 1 --2- 172.21.15.31:0/2960173999 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f52a40671a0 0x7f52a4069650 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f52b816a8e0 tx=0x7f52ac009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:02.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:02.917+0000 7f529ffff700 1 -- 172.21.15.31:0/2960173999 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f52a8051860 con 0x7f52b80c7be0 2023-10-06T02:27:03.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.154+0000 7f52bd19d700 1 -- 172.21.15.31:0/2960173999 --> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000626507", "target": ["mon-mgr", ""]}) v1 -- 0x7f52b8156cf0 con 0x7f52a40671a0 2023-10-06T02:27:03.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.167+0000 7f529ffff700 1 -- 172.21.15.31:0/2960173999 <== mgr.4107 v2:172.21.15.31:6832/85292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f52b8156cf0 con 0x7f52a40671a0 2023-10-06T02:27:03.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.170+0000 7f52bd19d700 1 -- 172.21.15.31:0/2960173999 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f52a40671a0 msgr2=0x7f52a4069650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:03.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.170+0000 7f52bd19d700 1 --2- 172.21.15.31:0/2960173999 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f52a40671a0 0x7f52a4069650 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f52b816a8e0 tx=0x7f52ac009040 comp rx=0 tx=0).stop 2023-10-06T02:27:03.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.170+0000 7f52bd19d700 1 -- 172.21.15.31:0/2960173999 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52b80c7be0 msgr2=0x7f52b8167400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:03.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.170+0000 7f52bd19d700 1 --2- 172.21.15.31:0/2960173999 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52b80c7be0 0x7f52b8167400 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f52a800c830 tx=0x7f52a8008040 comp rx=0 tx=0).stop 2023-10-06T02:27:03.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.170+0000 7f52bd19d700 1 -- 172.21.15.31:0/2960173999 shutdown_connections 2023-10-06T02:27:03.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.170+0000 7f52bd19d700 1 --2- 172.21.15.31:0/2960173999 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52b813f780 0x7f52b80b09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:03.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.170+0000 7f52bd19d700 1 --2- 172.21.15.31:0/2960173999 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52b80c7be0 0x7f52b8167400 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:03.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.170+0000 7f52bd19d700 1 --2- 172.21.15.31:0/2960173999 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f52a40671a0 0x7f52a4069650 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:03.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.170+0000 7f52bd19d700 1 --2- 172.21.15.31:0/2960173999 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f52b80c7fb0 0x7f52b80add60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:03.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.170+0000 7f52bd19d700 1 -- 172.21.15.31:0/2960173999 >> 172.21.15.31:0/2960173999 conn(0x7f52b806d110 msgr2=0x7f52b806de50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:03.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.170+0000 7f52bd19d700 1 -- 172.21.15.31:0/2960173999 shutdown_connections 2023-10-06T02:27:03.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.170+0000 7f52bd19d700 1 -- 172.21.15.31:0/2960173999 wait complete. 2023-10-06T02:27:03.172 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:27:03.172+0000 7f3e7dc83700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:27:03.172 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:27:03.172+0000 7f3e7dc83700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:27:03.173 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:27:03.172+0000 7f3e7dc83700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:27:03.173 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:27:03.172+0000 7f3e7dc83700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:27:03.173 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:27:03.172+0000 7f3e7dc83700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:27:03.191 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000626508 2023-10-06T02:27:03.192 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:27:03.191+0000 7f3e7e484700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:27:03.192 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:27:03.191+0000 7f3e7e484700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:27:03.192 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:27:03.191+0000 7f3e7e484700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:27:03.192 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:27:03.191+0000 7f3e7e484700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:27:03.192 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:27:03.191+0000 7f3e7e484700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:27:03.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.386+0000 7fde6adff700 1 Processor -- start 2023-10-06T02:27:03.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.386+0000 7fde6adff700 1 -- start start 2023-10-06T02:27:03.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.387+0000 7fde6adff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fde640c8800 0x7fde640c94e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:03.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.387+0000 7fde6adff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fde640b1150 0x7fde640ae050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:03.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.387+0000 7fde6adff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde640adbc0 con 0x7fde640c8bd0 2023-10-06T02:27:03.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.387+0000 7fde6adff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde640c9d30 con 0x7fde640b1150 2023-10-06T02:27:03.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.387+0000 7fde6adff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde640acfb0 con 0x7fde640c8800 2023-10-06T02:27:03.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.388+0000 7fde6939c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fde640b1150 0x7fde640ae050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:03.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.388+0000 7fde63fff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fde640c8bd0 0x7fde640b0a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:38216/0 (socket says 172.21.15.31:38216) 2023-10-06T02:27:03.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.388+0000 7fde63fff700 1 -- 172.21.15.31:0/692499652 learned_addr learned my addr 172.21.15.31:0/692499652 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:27:03.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.388+0000 7fde68b9b700 1 --2- 172.21.15.31:0/692499652 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fde640c8800 0x7fde640c94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:03.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.388+0000 7fde6939c700 1 -- 172.21.15.31:0/692499652 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fde640c8800 msgr2=0x7fde640c94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:03.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.388+0000 7fde6939c700 1 --2- 172.21.15.31:0/692499652 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fde640c8800 0x7fde640c94e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:03.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.388+0000 7fde6939c700 1 -- 172.21.15.31:0/692499652 >> v1:172.21.15.31:6789/0 conn(0x7fde640c8bd0 legacy=0x7fde640b0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:03.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.388+0000 7fde6939c700 1 -- 172.21.15.31:0/692499652 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde640ad290 con 0x7fde640b1150 2023-10-06T02:27:03.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.388+0000 7fde637fe700 1 -- 172.21.15.31:0/692499652 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1369652070 0 0) 0x7fde640adbc0 con 0x7fde640c8bd0 2023-10-06T02:27:03.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.389+0000 7fde6939c700 1 --2- 172.21.15.31:0/692499652 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde640b1150 0x7fde640ae050 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fde5c009e60 tx=0x7fde5c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9a214e7d99032ca server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:03.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.389+0000 7fde637fe700 1 -- 172.21.15.31:0/692499652 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde5c00e070 con 0x7fde640b1150 2023-10-06T02:27:03.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.389+0000 7fde637fe700 1 -- 172.21.15.31:0/692499652 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde5c00bb60 con 0x7fde640b1150 2023-10-06T02:27:03.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.389+0000 7fde637fe700 1 -- 172.21.15.31:0/692499652 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde5c012430 con 0x7fde640b1150 2023-10-06T02:27:03.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.389+0000 7fde6adff700 1 -- 172.21.15.31:0/692499652 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde640b1150 msgr2=0x7fde640ae050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:03.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.389+0000 7fde6adff700 1 --2- 172.21.15.31:0/692499652 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde640b1150 0x7fde640ae050 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fde5c009e60 tx=0x7fde5c009780 comp rx=0 tx=0).stop 2023-10-06T02:27:03.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.389+0000 7fde6adff700 1 -- 172.21.15.31:0/692499652 shutdown_connections 2023-10-06T02:27:03.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.389+0000 7fde6adff700 1 --2- 172.21.15.31:0/692499652 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde640b1150 0x7fde640ae050 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:03.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.389+0000 7fde6adff700 1 --2- 172.21.15.31:0/692499652 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fde640c8800 0x7fde640c94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:03.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.389+0000 7fde6adff700 1 -- 172.21.15.31:0/692499652 >> 172.21.15.31:0/692499652 conn(0x7fde6406d170 msgr2=0x7fde640cc400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:03.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.390+0000 7fde6adff700 1 -- 172.21.15.31:0/692499652 shutdown_connections 2023-10-06T02:27:03.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.390+0000 7fde6adff700 1 -- 172.21.15.31:0/692499652 wait complete. 2023-10-06T02:27:03.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.391+0000 7fde6adff700 1 Processor -- start 2023-10-06T02:27:03.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.391+0000 7fde6adff700 1 -- start start 2023-10-06T02:27:03.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.392+0000 7fde6adff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde640b1150 0x7fde64157f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:03.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.392+0000 7fde6adff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fde640c8800 0x7fde6406a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:03.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.392+0000 7fde6adff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde640c8bd0 0x7fde6406acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:03.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.392+0000 7fde6adff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde64158f50 con 0x7fde640c8bd0 2023-10-06T02:27:03.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.392+0000 7fde6adff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde6406b3c0 con 0x7fde640b1150 2023-10-06T02:27:03.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.392+0000 7fde6adff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde6406b6a0 con 0x7fde640c8800 2023-10-06T02:27:03.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.392+0000 7fde6939c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde640c8bd0 0x7fde6406acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:03.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.392+0000 7fde6939c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde640c8bd0 0x7fde6406acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51090/0 (socket says 172.21.15.31:51090) 2023-10-06T02:27:03.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.392+0000 7fde6939c700 1 -- 172.21.15.31:0/2528138435 learned_addr learned my addr 172.21.15.31:0/2528138435 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:03.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.393+0000 7fde68b9b700 1 --2- 172.21.15.31:0/2528138435 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde640b1150 0x7fde64157f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:03.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.392+0000 7fde63fff700 1 --2- 172.21.15.31:0/2528138435 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fde640c8800 0x7fde6406a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:03.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.393+0000 7fde6939c700 1 -- 172.21.15.31:0/2528138435 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fde640c8800 msgr2=0x7fde6406a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:03.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.393+0000 7fde6939c700 1 --2- 172.21.15.31:0/2528138435 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fde640c8800 0x7fde6406a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:03.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.393+0000 7fde6939c700 1 -- 172.21.15.31:0/2528138435 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde640b1150 msgr2=0x7fde64157f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:03.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.393+0000 7fde6939c700 1 --2- 172.21.15.31:0/2528138435 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde640b1150 0x7fde64157f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:03.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.393+0000 7fde6939c700 1 -- 172.21.15.31:0/2528138435 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde5c009a20 con 0x7fde640c8bd0 2023-10-06T02:27:03.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.393+0000 7fde6939c700 1 --2- 172.21.15.31:0/2528138435 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde640c8bd0 0x7fde6406acf0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fde5c013000 tx=0x7fde5c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:03.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.393+0000 7fde60ff9700 1 -- 172.21.15.31:0/2528138435 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde5c027470 con 0x7fde640c8bd0 2023-10-06T02:27:03.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.393+0000 7fde60ff9700 1 -- 172.21.15.31:0/2528138435 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde5c027610 con 0x7fde640c8bd0 2023-10-06T02:27:03.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.393+0000 7fde6adff700 1 -- 172.21.15.31:0/2528138435 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde6406b980 con 0x7fde640c8bd0 2023-10-06T02:27:03.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.393+0000 7fde60ff9700 1 -- 172.21.15.31:0/2528138435 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde5c0279d0 con 0x7fde640c8bd0 2023-10-06T02:27:03.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.393+0000 7fde6adff700 1 -- 172.21.15.31:0/2528138435 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde6415b6b0 con 0x7fde640c8bd0 2023-10-06T02:27:03.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.394+0000 7fde6adff700 1 -- 172.21.15.31:0/2528138435 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde6404c3c0 con 0x7fde640c8bd0 2023-10-06T02:27:03.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.395+0000 7fde60ff9700 1 -- 172.21.15.31:0/2528138435 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 11) v1 ==== 84995+0+0 (secure 0 0 0) 0x7fde5c024070 con 0x7fde640c8bd0 2023-10-06T02:27:03.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.397+0000 7fde60ff9700 1 --2- 172.21.15.31:0/2528138435 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fde48067200 0x7fde480696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:03.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.397+0000 7fde60ff9700 1 -- 172.21.15.31:0/2528138435 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6203+0+0 (secure 0 0 0) 0x7fde5c087700 con 0x7fde640c8bd0 2023-10-06T02:27:03.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.397+0000 7fde68b9b700 1 --2- 172.21.15.31:0/2528138435 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fde48067200 0x7fde480696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:03.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.397+0000 7fde68b9b700 1 --2- 172.21.15.31:0/2528138435 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fde48067200 0x7fde480696b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fde64158b30 tx=0x7fde58008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:03.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.401+0000 7fde60ff9700 1 -- 172.21.15.31:0/2528138435 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fde5c0543f0 con 0x7fde640c8bd0 2023-10-06T02:27:03.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.637+0000 7fde6adff700 1 -- 172.21.15.31:0/2528138435 --> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000626508", "target": ["mon-mgr", ""]}) v1 -- 0x7fde6415a410 con 0x7fde48067200 2023-10-06T02:27:03.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.654+0000 7fde60ff9700 1 -- 172.21.15.31:0/2528138435 <== mgr.4107 v2:172.21.15.31:6832/85292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fde6415a410 con 0x7fde48067200 2023-10-06T02:27:03.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.656+0000 7fde6adff700 1 -- 172.21.15.31:0/2528138435 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fde48067200 msgr2=0x7fde480696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:03.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.656+0000 7fde6adff700 1 --2- 172.21.15.31:0/2528138435 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fde48067200 0x7fde480696b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fde64158b30 tx=0x7fde58008040 comp rx=0 tx=0).stop 2023-10-06T02:27:03.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.656+0000 7fde6adff700 1 -- 172.21.15.31:0/2528138435 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde640c8bd0 msgr2=0x7fde6406acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:03.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.656+0000 7fde6adff700 1 --2- 172.21.15.31:0/2528138435 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde640c8bd0 0x7fde6406acf0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fde5c013000 tx=0x7fde5c019000 comp rx=0 tx=0).stop 2023-10-06T02:27:03.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.656+0000 7fde6adff700 1 -- 172.21.15.31:0/2528138435 shutdown_connections 2023-10-06T02:27:03.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.656+0000 7fde6adff700 1 --2- 172.21.15.31:0/2528138435 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde640c8bd0 0x7fde6406acf0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:03.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.656+0000 7fde6adff700 1 --2- 172.21.15.31:0/2528138435 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde640b1150 0x7fde64157f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:03.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.656+0000 7fde6adff700 1 --2- 172.21.15.31:0/2528138435 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fde48067200 0x7fde480696b0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:03.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.656+0000 7fde6adff700 1 --2- 172.21.15.31:0/2528138435 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fde640c8800 0x7fde6406a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:03.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.656+0000 7fde6adff700 1 -- 172.21.15.31:0/2528138435 >> 172.21.15.31:0/2528138435 conn(0x7fde6406d170 msgr2=0x7fde640ae050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:03.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.657+0000 7fde6adff700 1 -- 172.21.15.31:0/2528138435 shutdown_connections 2023-10-06T02:27:03.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:03.657+0000 7fde6adff700 1 -- 172.21.15.31:0/2528138435 wait complete. 2023-10-06T02:27:03.679 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:27:03.679 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2023-10-06T02:27:03.754 INFO:teuthology.orchestra.run.smithi031.stdout:2 2023-10-06T02:27:03.755 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T02:27:03.756 DEBUG:teuthology.orchestra.run.smithi031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T02:27:03.836 DEBUG:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T02:27:03.931 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T02:27:03.931 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T02:27:03.931 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:27:03.931 DEBUG:teuthology.orchestra.run.smithi031:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T02:27:04.037 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi031.front.sepia.ceph.com 2023-10-06T02:27:04.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:27:04.037 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T02:27:04.078 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-06T02:27:04.078 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:27:04.078 DEBUG:teuthology.orchestra.run.smithi062:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-06T02:27:04.159 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi062.front.sepia.ceph.com 2023-10-06T02:27:04.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:27:04.159 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T02:27:04.197 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:27:04.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.385+0000 7f4e8af9d700 1 Processor -- start 2023-10-06T02:27:04.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.385+0000 7f4e8af9d700 1 -- start start 2023-10-06T02:27:04.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.387+0000 7f4e8af9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4e840b2940 0x7f4e840a98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:04.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.387+0000 7f4e8af9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e840b0110 con 0x7f4e840cd9d0 2023-10-06T02:27:04.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.387+0000 7f4e8af9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e840b0920 con 0x7f4e840b2d10 2023-10-06T02:27:04.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.387+0000 7f4e8af9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e8406e730 con 0x7f4e840b2940 2023-10-06T02:27:04.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.387+0000 7f4e89f9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4e840b2940 0x7f4e840a98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:04.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.387+0000 7f4e8979a700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f4e840cd9d0 0x7f4e8406e020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:38232/0 (socket says 172.21.15.31:38232) 2023-10-06T02:27:04.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.387+0000 7f4e8979a700 1 -- 172.21.15.31:0/378591041 learned_addr learned my addr 172.21.15.31:0/378591041 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:27:04.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.387+0000 7f4e89f9b700 1 -- 172.21.15.31:0/378591041 >> v1:172.21.15.62:6789/0 conn(0x7f4e840b2d10 legacy=0x7f4e84159250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:04.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.387+0000 7f4e89f9b700 1 -- 172.21.15.31:0/378591041 >> v1:172.21.15.31:6789/0 conn(0x7f4e840cd9d0 legacy=0x7f4e8406e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:04.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.388+0000 7f4e89f9b700 1 -- 172.21.15.31:0/378591041 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e8406cc70 con 0x7f4e840b2940 2023-10-06T02:27:04.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.387+0000 7f4e88f99700 1 -- 172.21.15.31:0/378591041 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1004344791 0 0) 0x7f4e840b0110 con 0x7f4e840cd9d0 2023-10-06T02:27:04.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.388+0000 7f4e89f9b700 1 --2- 172.21.15.31:0/378591041 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e840b2940 0x7f4e840a98e0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f4e800098d0 tx=0x7f4e80009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ba9f88f3954087b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:04.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.388+0000 7f4e88f99700 1 -- 172.21.15.31:0/378591041 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e80009e70 con 0x7f4e840b2940 2023-10-06T02:27:04.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.388+0000 7f4e88f99700 1 -- 172.21.15.31:0/378591041 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e80005c10 con 0x7f4e840b2940 2023-10-06T02:27:04.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.388+0000 7f4e88f99700 1 -- 172.21.15.31:0/378591041 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e8000b650 con 0x7f4e840b2940 2023-10-06T02:27:04.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.389+0000 7f4e8af9d700 1 -- 172.21.15.31:0/378591041 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e840b2940 msgr2=0x7f4e840a98e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:04.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.389+0000 7f4e8af9d700 1 --2- 172.21.15.31:0/378591041 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e840b2940 0x7f4e840a98e0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f4e800098d0 tx=0x7f4e80009120 comp rx=0 tx=0).stop 2023-10-06T02:27:04.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.389+0000 7f4e8af9d700 1 -- 172.21.15.31:0/378591041 shutdown_connections 2023-10-06T02:27:04.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.389+0000 7f4e8af9d700 1 --2- 172.21.15.31:0/378591041 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e840b2940 0x7f4e840a98e0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:04.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.389+0000 7f4e8af9d700 1 -- 172.21.15.31:0/378591041 >> 172.21.15.31:0/378591041 conn(0x7f4e840675a0 msgr2=0x7f4e840679a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:04.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.389+0000 7f4e8af9d700 1 -- 172.21.15.31:0/378591041 shutdown_connections 2023-10-06T02:27:04.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.389+0000 7f4e8af9d700 1 -- 172.21.15.31:0/378591041 wait complete. 2023-10-06T02:27:04.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.390+0000 7f4e8af9d700 1 Processor -- start 2023-10-06T02:27:04.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.390+0000 7f4e8af9d700 1 -- start start 2023-10-06T02:27:04.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.391+0000 7f4e8af9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e840b2940 0x7f4e841417e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:04.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.391+0000 7f4e8af9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4e840b2d10 0x7f4e8415aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:04.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.391+0000 7f4e8af9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e840cd9d0 0x7f4e840adf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:04.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.391+0000 7f4e8af9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e8415c150 con 0x7f4e840b2940 2023-10-06T02:27:04.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.391+0000 7f4e8af9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e8415b540 con 0x7f4e840cd9d0 2023-10-06T02:27:04.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.391+0000 7f4e8af9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e8415b820 con 0x7f4e840b2d10 2023-10-06T02:27:04.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.391+0000 7f4e89f9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e840b2940 0x7f4e841417e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:04.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.391+0000 7f4e89f9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e840b2940 0x7f4e841417e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51122/0 (socket says 172.21.15.31:51122) 2023-10-06T02:27:04.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.391+0000 7f4e89f9b700 1 -- 172.21.15.31:0/3921227555 learned_addr learned my addr 172.21.15.31:0/3921227555 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:04.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.392+0000 7f4e8a79c700 1 --2- 172.21.15.31:0/3921227555 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e840cd9d0 0x7f4e840adf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:04.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.392+0000 7f4e8979a700 1 --2- 172.21.15.31:0/3921227555 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4e840b2d10 0x7f4e8415aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:04.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.392+0000 7f4e89f9b700 1 -- 172.21.15.31:0/3921227555 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4e840b2d10 msgr2=0x7f4e8415aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:04.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.392+0000 7f4e89f9b700 1 --2- 172.21.15.31:0/3921227555 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4e840b2d10 0x7f4e8415aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:04.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.392+0000 7f4e89f9b700 1 -- 172.21.15.31:0/3921227555 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e840cd9d0 msgr2=0x7f4e840adf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:04.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.392+0000 7f4e89f9b700 1 --2- 172.21.15.31:0/3921227555 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e840cd9d0 0x7f4e840adf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:04.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.392+0000 7f4e89f9b700 1 -- 172.21.15.31:0/3921227555 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e800093c0 con 0x7f4e840b2940 2023-10-06T02:27:04.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.392+0000 7f4e89f9b700 1 --2- 172.21.15.31:0/3921227555 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e840b2940 0x7f4e841417e0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f4e8406ee30 tx=0x7f4e80016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:04.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.392+0000 7f4e79ffb700 1 -- 172.21.15.31:0/3921227555 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e80006310 con 0x7f4e840b2940 2023-10-06T02:27:04.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.392+0000 7f4e79ffb700 1 -- 172.21.15.31:0/3921227555 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e80006470 con 0x7f4e840b2940 2023-10-06T02:27:04.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.392+0000 7f4e8af9d700 1 -- 172.21.15.31:0/3921227555 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e8415bb00 con 0x7f4e840b2940 2023-10-06T02:27:04.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.392+0000 7f4e79ffb700 1 -- 172.21.15.31:0/3921227555 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e80006830 con 0x7f4e840b2940 2023-10-06T02:27:04.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.393+0000 7f4e8af9d700 1 -- 172.21.15.31:0/3921227555 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e84141d20 con 0x7f4e840b2940 2023-10-06T02:27:04.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.394+0000 7f4e8af9d700 1 -- 172.21.15.31:0/3921227555 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e8404c3c0 con 0x7f4e840b2940 2023-10-06T02:27:04.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.394+0000 7f4e79ffb700 1 -- 172.21.15.31:0/3921227555 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 11) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f4e8001a630 con 0x7f4e840b2940 2023-10-06T02:27:04.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.396+0000 7f4e79ffb700 1 --2- 172.21.15.31:0/3921227555 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f4e68067200 0x7f4e680696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:04.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.396+0000 7f4e79ffb700 1 -- 172.21.15.31:0/3921227555 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6203+0+0 (secure 0 0 0) 0x7f4e80086330 con 0x7f4e840b2940 2023-10-06T02:27:04.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.396+0000 7f4e8979a700 1 --2- 172.21.15.31:0/3921227555 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f4e68067200 0x7f4e680696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:04.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.397+0000 7f4e8979a700 1 --2- 172.21.15.31:0/3921227555 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f4e68067200 0x7f4e680696b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f4e74000c00 tx=0x7f4e74009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:04.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.400+0000 7f4e79ffb700 1 -- 172.21.15.31:0/3921227555 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4e80053020 con 0x7f4e840b2940 2023-10-06T02:27:04.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.646+0000 7f4e8af9d700 1 -- 172.21.15.31:0/3921227555 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4e840b0360 con 0x7f4e840b2940 2023-10-06T02:27:04.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.647+0000 7f4e79ffb700 1 -- 172.21.15.31:0/3921227555 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4595 (secure 0 0 0) 0x7f4e80056640 con 0x7f4e840b2940 2023-10-06T02:27:04.648 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:27:04.648 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":38,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5216,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/2282788921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2282788921},{"type":"v1","addr":"172.21.15.62:6833","nonce":2282788921}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5219,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/289861443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":289861443},{"type":"v1","addr":"172.21.15.62:6835","nonce":289861443}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":5889,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/2954679985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":2954679985},{"type":"v1","addr":"172.21.15.31:6836","nonce":2954679985}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:26:42.554516+0000","modified":"2023-10-06T02:26:46.085562+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5892},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5892":{"gid":5892,"name":"c","rank":0,"incarnation":33,"state":"up:active","state_seq":5,"addr":"172.21.15.31:6837/1969136630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":1969136630},{"type":"v1","addr":"172.21.15.31:6837","nonce":1969136630}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-06T02:27:04.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.649+0000 7f4e8af9d700 1 -- 172.21.15.31:0/3921227555 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f4e68067200 msgr2=0x7f4e680696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:04.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.649+0000 7f4e8af9d700 1 --2- 172.21.15.31:0/3921227555 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f4e68067200 0x7f4e680696b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f4e74000c00 tx=0x7f4e74009040 comp rx=0 tx=0).stop 2023-10-06T02:27:04.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.649+0000 7f4e8af9d700 1 -- 172.21.15.31:0/3921227555 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e840b2940 msgr2=0x7f4e841417e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:04.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.649+0000 7f4e8af9d700 1 --2- 172.21.15.31:0/3921227555 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e840b2940 0x7f4e841417e0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f4e8406ee30 tx=0x7f4e80016000 comp rx=0 tx=0).stop 2023-10-06T02:27:04.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.650+0000 7f4e8af9d700 1 -- 172.21.15.31:0/3921227555 shutdown_connections 2023-10-06T02:27:04.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.650+0000 7f4e8af9d700 1 --2- 172.21.15.31:0/3921227555 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e840cd9d0 0x7f4e840adf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:04.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.650+0000 7f4e8af9d700 1 --2- 172.21.15.31:0/3921227555 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e840b2940 0x7f4e841417e0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:04.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.650+0000 7f4e8af9d700 1 --2- 172.21.15.31:0/3921227555 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f4e68067200 0x7f4e680696b0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:04.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.650+0000 7f4e8af9d700 1 --2- 172.21.15.31:0/3921227555 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4e840b2d10 0x7f4e8415aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:04.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.650+0000 7f4e8af9d700 1 -- 172.21.15.31:0/3921227555 >> 172.21.15.31:0/3921227555 conn(0x7f4e840675a0 msgr2=0x7f4e84157530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:04.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.650+0000 7f4e8af9d700 1 -- 172.21.15.31:0/3921227555 shutdown_connections 2023-10-06T02:27:04.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.650+0000 7f4e8af9d700 1 -- 172.21.15.31:0/3921227555 wait complete. 2023-10-06T02:27:04.651 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 38 2023-10-06T02:27:04.667 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:27:04.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.853+0000 7fa12d79d700 1 Processor -- start 2023-10-06T02:27:04.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.853+0000 7fa12d79d700 1 -- start start 2023-10-06T02:27:04.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.854+0000 7fa12d79d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa128153c30 0x7fa12815a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:04.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.854+0000 7fa12d79d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa12815de50 0x7fa1280cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:04.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.854+0000 7fa12d79d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa1280cef40 0x7fa12806d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:04.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.854+0000 7fa12d79d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1280cf3c0 con 0x7fa12815de50 2023-10-06T02:27:04.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.854+0000 7fa12d79d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa12815afd0 con 0x7fa128153c30 2023-10-06T02:27:04.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.854+0000 7fa12d79d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa12815b2b0 con 0x7fa1280cef40 2023-10-06T02:27:04.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.855+0000 7fa1277fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa12815de50 0x7fa1280cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:04.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.855+0000 7fa1277fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa12815de50 0x7fa1280cea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51134/0 (socket says 172.21.15.31:51134) 2023-10-06T02:27:04.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.855+0000 7fa1277fe700 1 -- 172.21.15.31:0/654661666 learned_addr learned my addr 172.21.15.31:0/654661666 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:04.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.855+0000 7fa127fff700 1 --2- 172.21.15.31:0/654661666 >> v2:172.21.15.62:3300/0 conn(0x7fa128153c30 0x7fa12815a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:04.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.855+0000 7fa12cf9c700 1 --2- 172.21.15.31:0/654661666 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa1280cef40 0x7fa12806d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:04.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.855+0000 7fa1277fe700 1 -- 172.21.15.31:0/654661666 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa1280cef40 msgr2=0x7fa12806d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:04.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.855+0000 7fa1277fe700 1 --2- 172.21.15.31:0/654661666 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa1280cef40 0x7fa12806d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:04.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.855+0000 7fa1277fe700 1 -- 172.21.15.31:0/654661666 >> v2:172.21.15.62:3300/0 conn(0x7fa128153c30 msgr2=0x7fa12815a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:04.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.855+0000 7fa1277fe700 1 --2- 172.21.15.31:0/654661666 >> v2:172.21.15.62:3300/0 conn(0x7fa128153c30 0x7fa12815a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:04.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.855+0000 7fa1277fe700 1 -- 172.21.15.31:0/654661666 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa12815b590 con 0x7fa12815de50 2023-10-06T02:27:04.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.856+0000 7fa1277fe700 1 --2- 172.21.15.31:0/654661666 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa12815de50 0x7fa1280cea00 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fa12806ead0 tx=0x7fa11c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e607599740fbe8bc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:04.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.856+0000 7fa126ffd700 1 -- 172.21.15.31:0/654661666 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa11c00b640 con 0x7fa12815de50 2023-10-06T02:27:04.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.856+0000 7fa126ffd700 1 -- 172.21.15.31:0/654661666 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa11c00b7a0 con 0x7fa12815de50 2023-10-06T02:27:04.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.856+0000 7fa126ffd700 1 -- 172.21.15.31:0/654661666 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa11c00bb60 con 0x7fa12815de50 2023-10-06T02:27:04.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.856+0000 7fa12d79d700 1 -- 172.21.15.31:0/654661666 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa12815de50 msgr2=0x7fa1280cea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:04.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.856+0000 7fa12d79d700 1 --2- 172.21.15.31:0/654661666 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa12815de50 0x7fa1280cea00 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fa12806ead0 tx=0x7fa11c009250 comp rx=0 tx=0).stop 2023-10-06T02:27:04.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.856+0000 7fa12d79d700 1 -- 172.21.15.31:0/654661666 shutdown_connections 2023-10-06T02:27:04.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.856+0000 7fa12d79d700 1 --2- 172.21.15.31:0/654661666 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa1280cef40 0x7fa12806d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:04.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.856+0000 7fa12d79d700 1 --2- 172.21.15.31:0/654661666 >> v2:172.21.15.62:3300/0 conn(0x7fa128153c30 0x7fa12815a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:04.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.856+0000 7fa12d79d700 1 --2- 172.21.15.31:0/654661666 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa12815de50 0x7fa1280cea00 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:04.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.856+0000 7fa12d79d700 1 -- 172.21.15.31:0/654661666 >> 172.21.15.31:0/654661666 conn(0x7fa1280703a0 msgr2=0x7fa1280707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:04.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.856+0000 7fa12d79d700 1 -- 172.21.15.31:0/654661666 shutdown_connections 2023-10-06T02:27:04.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.857+0000 7fa12d79d700 1 -- 172.21.15.31:0/654661666 wait complete. 2023-10-06T02:27:04.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.858+0000 7fa12d79d700 1 Processor -- start 2023-10-06T02:27:04.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.858+0000 7fa12d79d700 1 -- start start 2023-10-06T02:27:04.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.859+0000 7fa12d79d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa1280cef40 0x7fa1280b3520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:04.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.859+0000 7fa12d79d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa128153c30 0x7fa128140be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:04.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.859+0000 7fa12d79d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa12815de50 0x7fa128141120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:04.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.859+0000 7fa12d79d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1280cce20 con 0x7fa128153c30 2023-10-06T02:27:04.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.859+0000 7fa12d79d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1280c85c0 con 0x7fa12815de50 2023-10-06T02:27:04.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.859+0000 7fa12d79d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1280c7c90 con 0x7fa1280cef40 2023-10-06T02:27:04.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.859+0000 7fa1277fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa128153c30 0x7fa128140be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:04.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.859+0000 7fa1277fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa128153c30 0x7fa128140be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51142/0 (socket says 172.21.15.31:51142) 2023-10-06T02:27:04.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.859+0000 7fa1277fe700 1 -- 172.21.15.31:0/2228797710 learned_addr learned my addr 172.21.15.31:0/2228797710 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:04.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.859+0000 7fa12cf9c700 1 --2- 172.21.15.31:0/2228797710 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa12815de50 0x7fa128141120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:04.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.859+0000 7fa127fff700 1 --2- 172.21.15.31:0/2228797710 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa1280cef40 0x7fa1280b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:04.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.860+0000 7fa1277fe700 1 -- 172.21.15.31:0/2228797710 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa1280cef40 msgr2=0x7fa1280b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:04.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.860+0000 7fa1277fe700 1 --2- 172.21.15.31:0/2228797710 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa1280cef40 0x7fa1280b3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:04.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.860+0000 7fa1277fe700 1 -- 172.21.15.31:0/2228797710 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa12815de50 msgr2=0x7fa128141120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:04.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.860+0000 7fa1277fe700 1 --2- 172.21.15.31:0/2228797710 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa12815de50 0x7fa128141120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:04.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.860+0000 7fa1277fe700 1 -- 172.21.15.31:0/2228797710 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa11c0094f0 con 0x7fa128153c30 2023-10-06T02:27:04.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.860+0000 7fa1277fe700 1 --2- 172.21.15.31:0/2228797710 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa128153c30 0x7fa128140be0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fa11c006fd0 tx=0x7fa11c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:04.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.860+0000 7fa10ffff700 1 -- 172.21.15.31:0/2228797710 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa11c00b910 con 0x7fa128153c30 2023-10-06T02:27:04.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.860+0000 7fa10ffff700 1 -- 172.21.15.31:0/2228797710 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa11c00ba70 con 0x7fa128153c30 2023-10-06T02:27:04.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.860+0000 7fa12d79d700 1 -- 172.21.15.31:0/2228797710 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1280c7f70 con 0x7fa128153c30 2023-10-06T02:27:04.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.860+0000 7fa10ffff700 1 -- 172.21.15.31:0/2228797710 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa11c01a6a0 con 0x7fa128153c30 2023-10-06T02:27:04.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.860+0000 7fa12d79d700 1 -- 172.21.15.31:0/2228797710 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1280b3a60 con 0x7fa128153c30 2023-10-06T02:27:04.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.861+0000 7fa12d79d700 1 -- 172.21.15.31:0/2228797710 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1280c8750 con 0x7fa128153c30 2023-10-06T02:27:04.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.862+0000 7fa10ffff700 1 -- 172.21.15.31:0/2228797710 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 11) v1 ==== 84995+0+0 (secure 0 0 0) 0x7fa11c01aa90 con 0x7fa128153c30 2023-10-06T02:27:04.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.863+0000 7fa10ffff700 1 --2- 172.21.15.31:0/2228797710 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa114067200 0x7fa1140696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:04.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.863+0000 7fa10ffff700 1 -- 172.21.15.31:0/2228797710 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6203+0+0 (secure 0 0 0) 0x7fa11c086210 con 0x7fa128153c30 2023-10-06T02:27:04.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.863+0000 7fa127fff700 1 --2- 172.21.15.31:0/2228797710 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa114067200 0x7fa1140696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:04.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.864+0000 7fa127fff700 1 --2- 172.21.15.31:0/2228797710 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa114067200 0x7fa1140696b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fa118009db0 tx=0x7fa1180091a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:04.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:04.868+0000 7fa10ffff700 1 -- 172.21.15.31:0/2228797710 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa11c052f00 con 0x7fa128153c30 2023-10-06T02:27:05.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.073+0000 7fa12d79d700 1 -- 172.21.15.31:0/2228797710 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa12804c3c0 con 0x7fa128153c30 2023-10-06T02:27:05.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.074+0000 7fa10ffff700 1 -- 172.21.15.31:0/2228797710 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+17060 (secure 0 0 0) 0x7fa11c056520 con 0x7fa128153c30 2023-10-06T02:27:05.076 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:27:05.076 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":63,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:26:49.097251+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:26:40.728441+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-10-06T02:26:41.524170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.13","mappings":[{"from":7,"to":2}]},{"pgid":"10.c","mappings":[{"from":4,"to":5}]},{"pgid":"10.21","mappings":[{"from":4,"to":3}]},{"pgid":"10.35","mappings":[{"from":4,"to":3}]},{"pgid":"10.39","mappings":[{"from":7,"to":3}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:27:05.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.077+0000 7fa12d79d700 1 -- 172.21.15.31:0/2228797710 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa114067200 msgr2=0x7fa1140696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:05.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.077+0000 7fa12d79d700 1 --2- 172.21.15.31:0/2228797710 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa114067200 0x7fa1140696b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fa118009db0 tx=0x7fa1180091a0 comp rx=0 tx=0).stop 2023-10-06T02:27:05.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.077+0000 7fa12d79d700 1 -- 172.21.15.31:0/2228797710 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa128153c30 msgr2=0x7fa128140be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:05.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.077+0000 7fa12d79d700 1 --2- 172.21.15.31:0/2228797710 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa128153c30 0x7fa128140be0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fa11c006fd0 tx=0x7fa11c017000 comp rx=0 tx=0).stop 2023-10-06T02:27:05.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.077+0000 7fa12d79d700 1 -- 172.21.15.31:0/2228797710 shutdown_connections 2023-10-06T02:27:05.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.077+0000 7fa12d79d700 1 --2- 172.21.15.31:0/2228797710 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa12815de50 0x7fa128141120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:05.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.077+0000 7fa12d79d700 1 --2- 172.21.15.31:0/2228797710 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa114067200 0x7fa1140696b0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:05.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.077+0000 7fa12d79d700 1 --2- 172.21.15.31:0/2228797710 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa1280cef40 0x7fa1280b3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:05.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.077+0000 7fa12d79d700 1 --2- 172.21.15.31:0/2228797710 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa128153c30 0x7fa128140be0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:05.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.077+0000 7fa12d79d700 1 -- 172.21.15.31:0/2228797710 >> 172.21.15.31:0/2228797710 conn(0x7fa1280703a0 msgr2=0x7fa1280d07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:05.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.077+0000 7fa12d79d700 1 -- 172.21.15.31:0/2228797710 shutdown_connections 2023-10-06T02:27:05.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.077+0000 7fa12d79d700 1 -- 172.21.15.31:0/2228797710 wait complete. 2023-10-06T02:27:05.097 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T02:22:53.183425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T02:23:00.046967+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T02:26:40.728441+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T02:26:41.524170+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T02:27:05.097 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T02:27:05.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.285+0000 7fdbd719d700 1 Processor -- start 2023-10-06T02:27:05.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.285+0000 7fdbd719d700 1 -- start start 2023-10-06T02:27:05.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.287+0000 7fdbd719d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdbd0153a00 0x7fdbd0153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:05.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.287+0000 7fdbd719d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fdbd015de50 0x7fdbd015a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:05.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.287+0000 7fdbd719d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbd015af60 con 0x7fdbd00cea00 2023-10-06T02:27:05.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.287+0000 7fdbd719d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbd015d010 con 0x7fdbd015de50 2023-10-06T02:27:05.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.287+0000 7fdbd719d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd015b240 con 0x7fdbd0153a00 2023-10-06T02:27:05.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.287+0000 7fdbd699c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fdbd00cea00 0x7fdbd00cee40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:38240/0 (socket says 172.21.15.31:38240) 2023-10-06T02:27:05.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.287+0000 7fdbd699c700 1 -- 172.21.15.31:0/1693896853 learned_addr learned my addr 172.21.15.31:0/1693896853 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:27:05.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.287+0000 7fdbd599a700 1 --2- 172.21.15.31:0/1693896853 >> v2:172.21.15.62:3300/0 conn(0x7fdbd015de50 0x7fdbd015a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:05.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.287+0000 7fdbd619b700 1 --2- 172.21.15.31:0/1693896853 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdbd0153a00 0x7fdbd0153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:05.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.287+0000 7fdbd5199700 1 -- 172.21.15.31:0/1693896853 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1613346591 0 0) 0x7fdbd015af60 con 0x7fdbd00cea00 2023-10-06T02:27:05.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.287+0000 7fdbd5199700 1 -- 172.21.15.31:0/1693896853 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdbbc003570 con 0x7fdbd00cea00 2023-10-06T02:27:05.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.288+0000 7fdbd5199700 1 -- 172.21.15.31:0/1693896853 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2046001099 0 0) 0x7fdbbc003570 con 0x7fdbd00cea00 2023-10-06T02:27:05.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.288+0000 7fdbd5199700 1 -- 172.21.15.31:0/1693896853 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdbd0153a00 msgr2=0x7fdbd0153dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:05.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.288+0000 7fdbd5199700 1 --2- 172.21.15.31:0/1693896853 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdbd0153a00 0x7fdbd0153dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:05.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.288+0000 7fdbd5199700 1 -- 172.21.15.31:0/1693896853 >> v2:172.21.15.62:3300/0 conn(0x7fdbd015de50 msgr2=0x7fdbd015a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:05.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.288+0000 7fdbd5199700 1 --2- 172.21.15.31:0/1693896853 >> v2:172.21.15.62:3300/0 conn(0x7fdbd015de50 0x7fdbd015a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:05.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.288+0000 7fdbd5199700 1 -- 172.21.15.31:0/1693896853 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbd015b520 con 0x7fdbd00cea00 2023-10-06T02:27:05.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.288+0000 7fdbd5199700 1 -- 172.21.15.31:0/1693896853 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7fdbc80017f0 con 0x7fdbd00cea00 2023-10-06T02:27:05.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.288+0000 7fdbd5199700 1 -- 172.21.15.31:0/1693896853 >> v1:172.21.15.31:6789/0 conn(0x7fdbd00cea00 legacy=0x7fdbd00cee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:05.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.288+0000 7fdbd5199700 1 --2- 172.21.15.31:0/1693896853 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdbbc003a60 0x7fdbbc003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:05.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.288+0000 7fdbd5199700 1 --2- 172.21.15.31:0/1693896853 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdbd0153a00 0x7fdbbc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:05.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.288+0000 7fdbd5199700 1 --2- 172.21.15.31:0/1693896853 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdbbc004d50 0x7fdbbc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:05.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.288+0000 7fdbd5199700 1 -- 172.21.15.31:0/1693896853 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd015d010 con 0x7fdbbc003a60 2023-10-06T02:27:05.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.288+0000 7fdbd5199700 1 -- 172.21.15.31:0/1693896853 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd015b240 con 0x7fdbbc004d50 2023-10-06T02:27:05.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.288+0000 7fdbd5199700 1 -- 172.21.15.31:0/1693896853 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbbc00ba60 con 0x7fdbd0153a00 2023-10-06T02:27:05.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.288+0000 7fdbd619b700 1 --2- 172.21.15.31:0/1693896853 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdbbc003a60 0x7fdbbc003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:05.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.288+0000 7fdbd599a700 1 --2- 172.21.15.31:0/1693896853 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdbd0153a00 0x7fdbbc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:05.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.288+0000 7fdbd699c700 1 --2- 172.21.15.31:0/1693896853 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdbbc004d50 0x7fdbbc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:05.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.289+0000 7fdbd619b700 1 -- 172.21.15.31:0/1693896853 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdbd0153a00 msgr2=0x7fdbbc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:05.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.289+0000 7fdbd619b700 1 --2- 172.21.15.31:0/1693896853 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdbd0153a00 0x7fdbbc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:05.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.289+0000 7fdbd619b700 1 -- 172.21.15.31:0/1693896853 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdbbc004d50 msgr2=0x7fdbbc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:05.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.289+0000 7fdbd619b700 1 --2- 172.21.15.31:0/1693896853 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdbbc004d50 0x7fdbbc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:05.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.289+0000 7fdbd619b700 1 -- 172.21.15.31:0/1693896853 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbbc003570 con 0x7fdbbc003a60 2023-10-06T02:27:05.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.289+0000 7fdbd699c700 1 --2- 172.21.15.31:0/1693896853 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdbbc004d50 0x7fdbbc0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:27:05.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.289+0000 7fdbd619b700 1 --2- 172.21.15.31:0/1693896853 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdbbc003a60 0x7fdbbc003e70 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc009c00 tx=0x7fdbcc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eda12103a47c1b65 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:05.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.289+0000 7fdbd5199700 1 -- 172.21.15.31:0/1693896853 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc00e6b0 con 0x7fdbbc003a60 2023-10-06T02:27:05.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.289+0000 7fdbd5199700 1 -- 172.21.15.31:0/1693896853 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbcc00e810 con 0x7fdbbc003a60 2023-10-06T02:27:05.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.289+0000 7fdbd5199700 1 -- 172.21.15.31:0/1693896853 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc00ebd0 con 0x7fdbbc003a60 2023-10-06T02:27:05.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.290+0000 7fdbd719d700 1 -- 172.21.15.31:0/1693896853 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdbbc003a60 msgr2=0x7fdbbc003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:05.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.290+0000 7fdbd719d700 1 --2- 172.21.15.31:0/1693896853 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdbbc003a60 0x7fdbbc003e70 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc009c00 tx=0x7fdbcc0091a0 comp rx=0 tx=0).stop 2023-10-06T02:27:05.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.290+0000 7fdbd699c700 1 -- 172.21.15.31:0/1693896853 reap_dead start 2023-10-06T02:27:05.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.290+0000 7fdbd719d700 1 -- 172.21.15.31:0/1693896853 shutdown_connections 2023-10-06T02:27:05.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.290+0000 7fdbd719d700 1 -- 172.21.15.31:0/1693896853 >> 172.21.15.31:0/1693896853 conn(0x7fdbd00703a0 msgr2=0x7fdbd00707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:05.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.290+0000 7fdbd719d700 1 -- 172.21.15.31:0/1693896853 shutdown_connections 2023-10-06T02:27:05.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.290+0000 7fdbd719d700 1 -- 172.21.15.31:0/1693896853 wait complete. 2023-10-06T02:27:05.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.291+0000 7fdbd719d700 1 Processor -- start 2023-10-06T02:27:05.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.291+0000 7fdbd719d700 1 -- start start 2023-10-06T02:27:05.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.292+0000 7fdbd719d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdbbc003a60 0x7fdbd00c75e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:05.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.292+0000 7fdbd719d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdbd0158d50 0x7fdbd0151160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:05.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.292+0000 7fdbd719d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdbd00c98b0 0x7fdbd00c9d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:05.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.292+0000 7fdbd719d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd00cdc60 con 0x7fdbd0158d50 2023-10-06T02:27:05.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.292+0000 7fdbd719d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd00cb480 con 0x7fdbbc003a60 2023-10-06T02:27:05.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.292+0000 7fdbd719d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd01516a0 con 0x7fdbd00c98b0 2023-10-06T02:27:05.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.293+0000 7fdbd599a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdbd0158d50 0x7fdbd0151160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:05.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.293+0000 7fdbd599a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdbd0158d50 0x7fdbd0151160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51162/0 (socket says 172.21.15.31:51162) 2023-10-06T02:27:05.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.293+0000 7fdbd599a700 1 -- 172.21.15.31:0/3956671512 learned_addr learned my addr 172.21.15.31:0/3956671512 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:05.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.293+0000 7fdbd619b700 1 --2- 172.21.15.31:0/3956671512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdbbc003a60 0x7fdbd00c75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:05.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.293+0000 7fdbd699c700 1 --2- 172.21.15.31:0/3956671512 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdbd00c98b0 0x7fdbd00c9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:05.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.293+0000 7fdbd599a700 1 -- 172.21.15.31:0/3956671512 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdbd00c98b0 msgr2=0x7fdbd00c9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:05.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.293+0000 7fdbd599a700 1 --2- 172.21.15.31:0/3956671512 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdbd00c98b0 0x7fdbd00c9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:05.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.293+0000 7fdbd599a700 1 -- 172.21.15.31:0/3956671512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdbbc003a60 msgr2=0x7fdbd00c75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:05.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.293+0000 7fdbd599a700 1 --2- 172.21.15.31:0/3956671512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdbbc003a60 0x7fdbd00c75e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:05.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.293+0000 7fdbd599a700 1 -- 172.21.15.31:0/3956671512 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbd00c7ce0 con 0x7fdbd0158d50 2023-10-06T02:27:05.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.293+0000 7fdbd599a700 1 --2- 172.21.15.31:0/3956671512 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdbd0158d50 0x7fdbd0151160 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fdbc0011d40 tx=0x7fdbc00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:05.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.293+0000 7fdbc67fc700 1 -- 172.21.15.31:0/3956671512 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc000fac0 con 0x7fdbd0158d50 2023-10-06T02:27:05.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.293+0000 7fdbc67fc700 1 -- 172.21.15.31:0/3956671512 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbc000fc20 con 0x7fdbd0158d50 2023-10-06T02:27:05.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.293+0000 7fdbc67fc700 1 -- 172.21.15.31:0/3956671512 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc0014440 con 0x7fdbd0158d50 2023-10-06T02:27:05.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.294+0000 7fdbd719d700 1 -- 172.21.15.31:0/3956671512 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbd00c7fc0 con 0x7fdbd0158d50 2023-10-06T02:27:05.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.294+0000 7fdbd719d700 1 -- 172.21.15.31:0/3956671512 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbd00cb110 con 0x7fdbd0158d50 2023-10-06T02:27:05.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.295+0000 7fdbd719d700 1 -- 172.21.15.31:0/3956671512 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbd00cb5f0 con 0x7fdbd0158d50 2023-10-06T02:27:05.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.295+0000 7fdbc67fc700 1 -- 172.21.15.31:0/3956671512 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 11) v1 ==== 84995+0+0 (secure 0 0 0) 0x7fdbc0014a80 con 0x7fdbd0158d50 2023-10-06T02:27:05.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.296+0000 7fdbc67fc700 1 --2- 172.21.15.31:0/3956671512 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fdbb4067200 0x7fdbb40696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:05.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.296+0000 7fdbc67fc700 1 -- 172.21.15.31:0/3956671512 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6203+0+0 (secure 0 0 0) 0x7fdbc0018030 con 0x7fdbd0158d50 2023-10-06T02:27:05.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.296+0000 7fdbd619b700 1 --2- 172.21.15.31:0/3956671512 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fdbb4067200 0x7fdbb40696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:05.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.297+0000 7fdbd619b700 1 --2- 172.21.15.31:0/3956671512 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fdbb4067200 0x7fdbb40696b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc0091e0 tx=0x7fdbcc01a000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:05.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.301+0000 7fdbc67fc700 1 -- 172.21.15.31:0/3956671512 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdbc0052550 con 0x7fdbd0158d50 2023-10-06T02:27:05.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.505+0000 7fdbd719d700 1 -- 172.21.15.31:0/3956671512 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdbd004c3c0 con 0x7fdbd0158d50 2023-10-06T02:27:05.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.505+0000 7fdbc67fc700 1 -- 172.21.15.31:0/3956671512 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v63) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdbc0055b70 con 0x7fdbd0158d50 2023-10-06T02:27:05.507 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-10-06T02:27:05.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.508+0000 7fdbd719d700 1 -- 172.21.15.31:0/3956671512 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fdbb4067200 msgr2=0x7fdbb40696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:05.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.508+0000 7fdbd719d700 1 --2- 172.21.15.31:0/3956671512 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fdbb4067200 0x7fdbb40696b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc0091e0 tx=0x7fdbcc01a000 comp rx=0 tx=0).stop 2023-10-06T02:27:05.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.508+0000 7fdbd719d700 1 -- 172.21.15.31:0/3956671512 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdbd0158d50 msgr2=0x7fdbd0151160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:05.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.508+0000 7fdbd719d700 1 --2- 172.21.15.31:0/3956671512 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdbd0158d50 0x7fdbd0151160 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fdbc0011d40 tx=0x7fdbc00091a0 comp rx=0 tx=0).stop 2023-10-06T02:27:05.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.508+0000 7fdbd719d700 1 -- 172.21.15.31:0/3956671512 shutdown_connections 2023-10-06T02:27:05.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.508+0000 7fdbd719d700 1 --2- 172.21.15.31:0/3956671512 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fdbb4067200 0x7fdbb40696b0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:05.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.508+0000 7fdbd719d700 1 --2- 172.21.15.31:0/3956671512 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdbd00c98b0 0x7fdbd00c9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:05.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.508+0000 7fdbd719d700 1 --2- 172.21.15.31:0/3956671512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdbbc003a60 0x7fdbd00c75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:05.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.508+0000 7fdbd719d700 1 --2- 172.21.15.31:0/3956671512 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdbd0158d50 0x7fdbd0151160 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:05.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.508+0000 7fdbd719d700 1 -- 172.21.15.31:0/3956671512 >> 172.21.15.31:0/3956671512 conn(0x7fdbd00703a0 msgr2=0x7fdbd0152f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:05.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.508+0000 7fdbd719d700 1 -- 172.21.15.31:0/3956671512 shutdown_connections 2023-10-06T02:27:05.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.508+0000 7fdbd719d700 1 -- 172.21.15.31:0/3956671512 wait complete. 2023-10-06T02:27:05.526 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T02:27:05.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.716+0000 7fa96659d700 1 Processor -- start 2023-10-06T02:27:05.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.716+0000 7fa96659d700 1 -- start start 2023-10-06T02:27:05.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.717+0000 7fa96659d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa9600c7be0 0x7fa96006abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:05.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.717+0000 7fa96659d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa96013f780 0x7fa96013fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:05.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.717+0000 7fa96659d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9601400e0 con 0x7fa9600c7fb0 2023-10-06T02:27:05.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.717+0000 7fa96659d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa96006bf10 con 0x7fa96013f780 2023-10-06T02:27:05.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.717+0000 7fa96659d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa96006b290 con 0x7fa9600c7be0 2023-10-06T02:27:05.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.717+0000 7fa965d9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa96013f780 0x7fa96013fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:05.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.717+0000 7fa965d9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa96013f780 0x7fa96013fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51174/0 (socket says 172.21.15.31:51174) 2023-10-06T02:27:05.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.717+0000 7fa965d9c700 1 -- 172.21.15.31:0/777333028 learned_addr learned my addr 172.21.15.31:0/777333028 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:05.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.718+0000 7fa96559b700 1 --2- 172.21.15.31:0/777333028 >> v2:172.21.15.62:3300/0 conn(0x7fa9600c7be0 0x7fa96006abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:05.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.718+0000 7fa965d9c700 1 -- 172.21.15.31:0/777333028 >> v2:172.21.15.62:3300/0 conn(0x7fa9600c7be0 msgr2=0x7fa96006abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:05.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.718+0000 7fa965d9c700 1 --2- 172.21.15.31:0/777333028 >> v2:172.21.15.62:3300/0 conn(0x7fa9600c7be0 0x7fa96006abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:05.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.718+0000 7fa965d9c700 1 -- 172.21.15.31:0/777333028 >> v1:172.21.15.62:6789/0 conn(0x7fa9600c7fb0 legacy=0x7fa96015a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:05.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.718+0000 7fa965d9c700 1 -- 172.21.15.31:0/777333028 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa96006b570 con 0x7fa96013f780 2023-10-06T02:27:05.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.718+0000 7fa96559b700 1 --2- 172.21.15.31:0/777333028 >> v2:172.21.15.62:3300/0 conn(0x7fa9600c7be0 0x7fa96006abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:27:05.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.718+0000 7fa965d9c700 1 --2- 172.21.15.31:0/777333028 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa96013f780 0x7fa96013fba0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fa9600c8b00 tx=0x7fa958009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=521d11111c23fcbd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:05.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.718+0000 7fa957fff700 1 -- 172.21.15.31:0/777333028 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95800e070 con 0x7fa96013f780 2023-10-06T02:27:05.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.719+0000 7fa957fff700 1 -- 172.21.15.31:0/777333028 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa95800ba50 con 0x7fa96013f780 2023-10-06T02:27:05.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.719+0000 7fa957fff700 1 -- 172.21.15.31:0/777333028 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9580092e0 con 0x7fa96013f780 2023-10-06T02:27:05.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.719+0000 7fa96659d700 1 -- 172.21.15.31:0/777333028 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa96013f780 msgr2=0x7fa96013fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:05.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.719+0000 7fa96659d700 1 --2- 172.21.15.31:0/777333028 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa96013f780 0x7fa96013fba0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fa9600c8b00 tx=0x7fa958009840 comp rx=0 tx=0).stop 2023-10-06T02:27:05.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.719+0000 7fa96659d700 1 -- 172.21.15.31:0/777333028 shutdown_connections 2023-10-06T02:27:05.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.719+0000 7fa96659d700 1 --2- 172.21.15.31:0/777333028 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa96013f780 0x7fa96013fba0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:05.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.719+0000 7fa96659d700 1 --2- 172.21.15.31:0/777333028 >> v2:172.21.15.62:3300/0 conn(0x7fa9600c7be0 0x7fa96006abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:05.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.719+0000 7fa96659d700 1 -- 172.21.15.31:0/777333028 >> 172.21.15.31:0/777333028 conn(0x7fa96006d110 msgr2=0x7fa9600cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:05.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.719+0000 7fa96659d700 1 -- 172.21.15.31:0/777333028 shutdown_connections 2023-10-06T02:27:05.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.719+0000 7fa96659d700 1 -- 172.21.15.31:0/777333028 wait complete. 2023-10-06T02:27:05.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.721+0000 7fa96659d700 1 Processor -- start 2023-10-06T02:27:05.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.721+0000 7fa96659d700 1 -- start start 2023-10-06T02:27:05.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.722+0000 7fa96659d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9600c7be0 0x7fa96016a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:05.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.722+0000 7fa96659d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa9600c7fb0 0x7fa960153d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:05.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.722+0000 7fa96659d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa96013f780 0x7fa960154610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:05.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.722+0000 7fa96659d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9600c1600 con 0x7fa9600c7be0 2023-10-06T02:27:05.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.722+0000 7fa96659d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9600c18e0 con 0x7fa96013f780 2023-10-06T02:27:05.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.722+0000 7fa96659d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9600c1bc0 con 0x7fa9600c7fb0 2023-10-06T02:27:05.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.722+0000 7fa96559b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9600c7be0 0x7fa96016a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:05.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.722+0000 7fa96559b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9600c7be0 0x7fa96016a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51176/0 (socket says 172.21.15.31:51176) 2023-10-06T02:27:05.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.722+0000 7fa96559b700 1 -- 172.21.15.31:0/1210257662 learned_addr learned my addr 172.21.15.31:0/1210257662 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:05.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.722+0000 7fa965d9c700 1 --2- 172.21.15.31:0/1210257662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa96013f780 0x7fa960154610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:05.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.722+0000 7fa964d9a700 1 --2- 172.21.15.31:0/1210257662 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa9600c7fb0 0x7fa960153d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:05.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.722+0000 7fa96559b700 1 -- 172.21.15.31:0/1210257662 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa9600c7fb0 msgr2=0x7fa960153d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:05.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.722+0000 7fa96559b700 1 --2- 172.21.15.31:0/1210257662 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa9600c7fb0 0x7fa960153d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:05.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.722+0000 7fa96559b700 1 -- 172.21.15.31:0/1210257662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa96013f780 msgr2=0x7fa960154610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:05.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.722+0000 7fa96559b700 1 --2- 172.21.15.31:0/1210257662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa96013f780 0x7fa960154610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:05.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.722+0000 7fa96559b700 1 -- 172.21.15.31:0/1210257662 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa958009ae0 con 0x7fa9600c7be0 2023-10-06T02:27:05.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.723+0000 7fa96559b700 1 --2- 172.21.15.31:0/1210257662 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9600c7be0 0x7fa96016a410 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fa95c00e8d0 tx=0x7fa95c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:05.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.723+0000 7fa9557fa700 1 -- 172.21.15.31:0/1210257662 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95c013410 con 0x7fa9600c7be0 2023-10-06T02:27:05.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.723+0000 7fa9557fa700 1 -- 172.21.15.31:0/1210257662 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa95c013570 con 0x7fa9600c7be0 2023-10-06T02:27:05.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.723+0000 7fa9557fa700 1 -- 172.21.15.31:0/1210257662 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95c013910 con 0x7fa9600c7be0 2023-10-06T02:27:05.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.723+0000 7fa96659d700 1 -- 172.21.15.31:0/1210257662 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9600c1ea0 con 0x7fa9600c7be0 2023-10-06T02:27:05.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.723+0000 7fa96659d700 1 -- 172.21.15.31:0/1210257662 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9601542d0 con 0x7fa9600c7be0 2023-10-06T02:27:05.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.724+0000 7fa96659d700 1 -- 172.21.15.31:0/1210257662 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa960158410 con 0x7fa9600c7be0 2023-10-06T02:27:05.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.725+0000 7fa9557fa700 1 -- 172.21.15.31:0/1210257662 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 11) v1 ==== 84995+0+0 (secure 0 0 0) 0x7fa95c013d00 con 0x7fa9600c7be0 2023-10-06T02:27:05.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.726+0000 7fa9557fa700 1 --2- 172.21.15.31:0/1210257662 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa944067200 0x7fa9440696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:05.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.726+0000 7fa9557fa700 1 -- 172.21.15.31:0/1210257662 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6203+0+0 (secure 0 0 0) 0x7fa95c084a30 con 0x7fa9600c7be0 2023-10-06T02:27:05.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.727+0000 7fa964d9a700 1 --2- 172.21.15.31:0/1210257662 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa944067200 0x7fa9440696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:05.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.727+0000 7fa964d9a700 1 --2- 172.21.15.31:0/1210257662 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa944067200 0x7fa9440696b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa960155bc0 tx=0x7fa950009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:05.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.731+0000 7fa9557fa700 1 -- 172.21.15.31:0/1210257662 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa95c051610 con 0x7fa9600c7be0 2023-10-06T02:27:05.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.931+0000 7fa96659d700 1 -- 172.21.15.31:0/1210257662 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa96004c3c0 con 0x7fa9600c7be0 2023-10-06T02:27:05.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.931+0000 7fa9557fa700 1 -- 172.21.15.31:0/1210257662 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v63) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa95c054c30 con 0x7fa9600c7be0 2023-10-06T02:27:05.933 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 8 2023-10-06T02:27:05.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.934+0000 7fa96659d700 1 -- 172.21.15.31:0/1210257662 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa944067200 msgr2=0x7fa9440696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:05.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.934+0000 7fa96659d700 1 --2- 172.21.15.31:0/1210257662 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa944067200 0x7fa9440696b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa960155bc0 tx=0x7fa950009040 comp rx=0 tx=0).stop 2023-10-06T02:27:05.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.934+0000 7fa96659d700 1 -- 172.21.15.31:0/1210257662 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9600c7be0 msgr2=0x7fa96016a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:05.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.934+0000 7fa96659d700 1 --2- 172.21.15.31:0/1210257662 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9600c7be0 0x7fa96016a410 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fa95c00e8d0 tx=0x7fa95c008040 comp rx=0 tx=0).stop 2023-10-06T02:27:05.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.934+0000 7fa96659d700 1 -- 172.21.15.31:0/1210257662 shutdown_connections 2023-10-06T02:27:05.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.934+0000 7fa96659d700 1 --2- 172.21.15.31:0/1210257662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa96013f780 0x7fa960154610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:05.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.934+0000 7fa96659d700 1 --2- 172.21.15.31:0/1210257662 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9600c7be0 0x7fa96016a410 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:05.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.934+0000 7fa96659d700 1 --2- 172.21.15.31:0/1210257662 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa944067200 0x7fa9440696b0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:05.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.934+0000 7fa96659d700 1 --2- 172.21.15.31:0/1210257662 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa9600c7fb0 0x7fa960153d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:05.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.934+0000 7fa96659d700 1 -- 172.21.15.31:0/1210257662 >> 172.21.15.31:0/1210257662 conn(0x7fa96006d110 msgr2=0x7fa960155d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:05.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.934+0000 7fa96659d700 1 -- 172.21.15.31:0/1210257662 shutdown_connections 2023-10-06T02:27:05.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:05.934+0000 7fa96659d700 1 -- 172.21.15.31:0/1210257662 wait complete. 2023-10-06T02:27:05.953 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T02:27:06.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.147+0000 7fe76c9fe700 1 Processor -- start 2023-10-06T02:27:06.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.147+0000 7fe76c9fe700 1 -- start start 2023-10-06T02:27:06.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.148+0000 7fe76c9fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe768140a70 0x7fe768069b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:06.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.148+0000 7fe76c9fe700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe76815b640 con 0x7fe7680c7c60 2023-10-06T02:27:06.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.149+0000 7fe76c9fe700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe76815b920 con 0x7fe76806de50 2023-10-06T02:27:06.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.149+0000 7fe76c9fe700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7680c6800 con 0x7fe768140a70 2023-10-06T02:27:06.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.149+0000 7fe766d9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe768140a70 0x7fe768069b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:06.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.149+0000 7fe76759e700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fe7680c7c60 0x7fe768158190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:38252/0 (socket says 172.21.15.31:38252) 2023-10-06T02:27:06.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.149+0000 7fe76759e700 1 -- 172.21.15.31:0/4140285185 learned_addr learned my addr 172.21.15.31:0/4140285185 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:27:06.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.149+0000 7fe765d9b700 1 -- 172.21.15.31:0/4140285185 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 795549984 0 0) 0x7fe76815b640 con 0x7fe7680c7c60 2023-10-06T02:27:06.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.149+0000 7fe765d9b700 1 -- 172.21.15.31:0/4140285185 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe750003570 con 0x7fe7680c7c60 2023-10-06T02:27:06.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.149+0000 7fe766d9d700 1 -- 172.21.15.31:0/4140285185 >> v1:172.21.15.62:6789/0 conn(0x7fe76806de50 legacy=0x7fe768157a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:06.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.149+0000 7fe766d9d700 1 -- 172.21.15.31:0/4140285185 >> v1:172.21.15.31:6789/0 conn(0x7fe7680c7c60 legacy=0x7fe768158190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:06.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.149+0000 7fe766d9d700 1 -- 172.21.15.31:0/4140285185 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe76815bc00 con 0x7fe768140a70 2023-10-06T02:27:06.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.150+0000 7fe766d9d700 1 --2- 172.21.15.31:0/4140285185 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe768140a70 0x7fe768069b00 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fe7580098e0 tx=0x7fe7580091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e78b6199c13f8283 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:06.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.150+0000 7fe765d9b700 1 -- 172.21.15.31:0/4140285185 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75800b640 con 0x7fe768140a70 2023-10-06T02:27:06.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.150+0000 7fe765d9b700 1 -- 172.21.15.31:0/4140285185 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe75800b7a0 con 0x7fe768140a70 2023-10-06T02:27:06.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.150+0000 7fe765d9b700 1 -- 172.21.15.31:0/4140285185 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75800bb40 con 0x7fe768140a70 2023-10-06T02:27:06.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.150+0000 7fe76c9fe700 1 -- 172.21.15.31:0/4140285185 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe768140a70 msgr2=0x7fe768069b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:06.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.150+0000 7fe76c9fe700 1 --2- 172.21.15.31:0/4140285185 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe768140a70 0x7fe768069b00 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fe7580098e0 tx=0x7fe7580091a0 comp rx=0 tx=0).stop 2023-10-06T02:27:06.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.150+0000 7fe76c9fe700 1 -- 172.21.15.31:0/4140285185 shutdown_connections 2023-10-06T02:27:06.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.150+0000 7fe76c9fe700 1 --2- 172.21.15.31:0/4140285185 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe768140a70 0x7fe768069b00 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:06.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.150+0000 7fe76c9fe700 1 -- 172.21.15.31:0/4140285185 >> 172.21.15.31:0/4140285185 conn(0x7fe76806d110 msgr2=0x7fe7680cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:06.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.150+0000 7fe76c9fe700 1 -- 172.21.15.31:0/4140285185 shutdown_connections 2023-10-06T02:27:06.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.151+0000 7fe76c9fe700 1 -- 172.21.15.31:0/4140285185 wait complete. 2023-10-06T02:27:06.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.152+0000 7fe76c9fe700 1 Processor -- start 2023-10-06T02:27:06.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.152+0000 7fe76c9fe700 1 -- start start 2023-10-06T02:27:06.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.153+0000 7fe76c9fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe76806de50 0x7fe7680b1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:06.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.153+0000 7fe76c9fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7680c7c60 0x7fe768157de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:06.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.153+0000 7fe76c9fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe768140a70 0x7fe7680c43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:06.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.153+0000 7fe76c9fe700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7680c3690 con 0x7fe7680c7c60 2023-10-06T02:27:06.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.153+0000 7fe76c9fe700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7680c3f30 con 0x7fe76806de50 2023-10-06T02:27:06.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.153+0000 7fe76c9fe700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7680b1bd0 con 0x7fe768140a70 2023-10-06T02:27:06.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.153+0000 7fe76659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7680c7c60 0x7fe768157de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:06.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.153+0000 7fe766d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe76806de50 0x7fe7680b1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:06.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.153+0000 7fe766d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe76806de50 0x7fe7680b1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:57914/0 (socket says 172.21.15.31:57914) 2023-10-06T02:27:06.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.153+0000 7fe76659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7680c7c60 0x7fe768157de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51192/0 (socket says 172.21.15.31:51192) 2023-10-06T02:27:06.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.153+0000 7fe766d9d700 1 -- 172.21.15.31:0/538035616 learned_addr learned my addr 172.21.15.31:0/538035616 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:06.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.153+0000 7fe76759e700 1 --2- 172.21.15.31:0/538035616 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe768140a70 0x7fe7680c43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:06.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.154+0000 7fe76659c700 1 -- 172.21.15.31:0/538035616 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe768140a70 msgr2=0x7fe7680c43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:06.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.154+0000 7fe76659c700 1 --2- 172.21.15.31:0/538035616 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe768140a70 0x7fe7680c43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:06.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.154+0000 7fe76659c700 1 -- 172.21.15.31:0/538035616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe76806de50 msgr2=0x7fe7680b1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:06.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.154+0000 7fe76659c700 1 --2- 172.21.15.31:0/538035616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe76806de50 0x7fe7680b1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:06.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.154+0000 7fe76659c700 1 -- 172.21.15.31:0/538035616 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe758009420 con 0x7fe7680c7c60 2023-10-06T02:27:06.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.154+0000 7fe76659c700 1 --2- 172.21.15.31:0/538035616 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7680c7c60 0x7fe768157de0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fe75c00c490 tx=0x7fe75c007ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:06.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.154+0000 7fe74effd700 1 -- 172.21.15.31:0/538035616 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75c011590 con 0x7fe7680c7c60 2023-10-06T02:27:06.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.154+0000 7fe74effd700 1 -- 172.21.15.31:0/538035616 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe75c011710 con 0x7fe7680c7c60 2023-10-06T02:27:06.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.154+0000 7fe74effd700 1 -- 172.21.15.31:0/538035616 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75c011ad0 con 0x7fe7680c7c60 2023-10-06T02:27:06.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.154+0000 7fe76c9fe700 1 -- 172.21.15.31:0/538035616 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7680c38e0 con 0x7fe7680c7c60 2023-10-06T02:27:06.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.154+0000 7fe76c9fe700 1 -- 172.21.15.31:0/538035616 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7680c3420 con 0x7fe7680c7c60 2023-10-06T02:27:06.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.155+0000 7fe76c9fe700 1 -- 172.21.15.31:0/538035616 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7680c40a0 con 0x7fe7680c7c60 2023-10-06T02:27:06.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.156+0000 7fe74effd700 1 -- 172.21.15.31:0/538035616 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fe75c01b3b0 con 0x7fe7680c7c60 2023-10-06T02:27:06.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.158+0000 7fe74effd700 1 --2- 172.21.15.31:0/538035616 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe754067290 0x7fe754069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:06.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.158+0000 7fe74effd700 1 -- 172.21.15.31:0/538035616 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6203+0+0 (secure 0 0 0) 0x7fe75c083fe0 con 0x7fe7680c7c60 2023-10-06T02:27:06.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.158+0000 7fe766d9d700 1 --2- 172.21.15.31:0/538035616 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe754067290 0x7fe754069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:06.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.159+0000 7fe766d9d700 1 --2- 172.21.15.31:0/538035616 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe754067290 0x7fe754069740 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe758003080 tx=0x7fe758017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:06.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.162+0000 7fe74effd700 1 -- 172.21.15.31:0/538035616 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe75c050c10 con 0x7fe7680c7c60 2023-10-06T02:27:06.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.364+0000 7fe76c9fe700 1 -- 172.21.15.31:0/538035616 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe76804c3c0 con 0x7fe7680c7c60 2023-10-06T02:27:06.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.364+0000 7fe74effd700 1 -- 172.21.15.31:0/538035616 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v63) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe75c017020 con 0x7fe7680c7c60 2023-10-06T02:27:06.366 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-10-06T02:27:06.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.367+0000 7fe76c9fe700 1 -- 172.21.15.31:0/538035616 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe754067290 msgr2=0x7fe754069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:06.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.367+0000 7fe76c9fe700 1 --2- 172.21.15.31:0/538035616 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe754067290 0x7fe754069740 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe758003080 tx=0x7fe758017000 comp rx=0 tx=0).stop 2023-10-06T02:27:06.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.367+0000 7fe76c9fe700 1 -- 172.21.15.31:0/538035616 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7680c7c60 msgr2=0x7fe768157de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:06.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.367+0000 7fe76c9fe700 1 --2- 172.21.15.31:0/538035616 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7680c7c60 0x7fe768157de0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fe75c00c490 tx=0x7fe75c007ec0 comp rx=0 tx=0).stop 2023-10-06T02:27:06.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.367+0000 7fe76c9fe700 1 -- 172.21.15.31:0/538035616 shutdown_connections 2023-10-06T02:27:06.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.367+0000 7fe76c9fe700 1 --2- 172.21.15.31:0/538035616 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe754067290 0x7fe754069740 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:06.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.367+0000 7fe76c9fe700 1 --2- 172.21.15.31:0/538035616 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe768140a70 0x7fe7680c43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:06.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.367+0000 7fe76c9fe700 1 --2- 172.21.15.31:0/538035616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe76806de50 0x7fe7680b1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:06.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.367+0000 7fe76c9fe700 1 --2- 172.21.15.31:0/538035616 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7680c7c60 0x7fe768157de0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:06.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.367+0000 7fe76c9fe700 1 -- 172.21.15.31:0/538035616 >> 172.21.15.31:0/538035616 conn(0x7fe76806d110 msgr2=0x7fe7680c0ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:06.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.367+0000 7fe76c9fe700 1 -- 172.21.15.31:0/538035616 shutdown_connections 2023-10-06T02:27:06.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.367+0000 7fe76c9fe700 1 -- 172.21.15.31:0/538035616 wait complete. 2023-10-06T02:27:06.385 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T02:27:06.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.573+0000 7f4b2cb9d700 1 Processor -- start 2023-10-06T02:27:06.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.573+0000 7f4b2cb9d700 1 -- start start 2023-10-06T02:27:06.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.575+0000 7f4b2cb9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4b280c7be0 0x7f4b2806abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:06.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.575+0000 7f4b2cb9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4b280c7fb0 0x7f4b2815a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:06.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.575+0000 7f4b2cb9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b2815aed0 con 0x7f4b2813f780 2023-10-06T02:27:06.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.575+0000 7f4b2cb9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b2806bf10 con 0x7f4b280c7fb0 2023-10-06T02:27:06.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.575+0000 7f4b2cb9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b2806b290 con 0x7f4b280c7be0 2023-10-06T02:27:06.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.575+0000 7f4b26ffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4b280c7fb0 0x7f4b2815a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:06.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.575+0000 7f4b26ffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4b280c7fb0 0x7f4b2815a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51206/0 (socket says 172.21.15.31:51206) 2023-10-06T02:27:06.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.575+0000 7f4b26ffd700 1 -- 172.21.15.31:0/646498761 learned_addr learned my addr 172.21.15.31:0/646498761 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:06.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.575+0000 7f4b277fe700 1 --2- 172.21.15.31:0/646498761 >> v2:172.21.15.62:3300/0 conn(0x7f4b280c7be0 0x7f4b2806abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:06.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.575+0000 7f4b26ffd700 1 -- 172.21.15.31:0/646498761 >> v2:172.21.15.62:3300/0 conn(0x7f4b280c7be0 msgr2=0x7f4b2806abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:06.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.575+0000 7f4b26ffd700 1 --2- 172.21.15.31:0/646498761 >> v2:172.21.15.62:3300/0 conn(0x7f4b280c7be0 0x7f4b2806abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:06.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.575+0000 7f4b26ffd700 1 -- 172.21.15.31:0/646498761 >> v1:172.21.15.62:6789/0 conn(0x7f4b2813f780 legacy=0x7f4b2813fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:06.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.575+0000 7f4b26ffd700 1 -- 172.21.15.31:0/646498761 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b2806b570 con 0x7f4b280c7fb0 2023-10-06T02:27:06.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.575+0000 7f4b277fe700 1 --2- 172.21.15.31:0/646498761 >> v2:172.21.15.62:3300/0 conn(0x7f4b280c7be0 0x7f4b2806abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:27:06.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.576+0000 7f4b26ffd700 1 --2- 172.21.15.31:0/646498761 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b280c7fb0 0x7f4b2815a990 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f4b280c8b00 tx=0x7f4b1c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3be6b5785f91349 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:06.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.576+0000 7f4b267fc700 1 -- 172.21.15.31:0/646498761 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b1c00b820 con 0x7f4b280c7fb0 2023-10-06T02:27:06.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.576+0000 7f4b267fc700 1 -- 172.21.15.31:0/646498761 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b1c00b980 con 0x7f4b280c7fb0 2023-10-06T02:27:06.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.576+0000 7f4b267fc700 1 -- 172.21.15.31:0/646498761 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b1c00bd20 con 0x7f4b280c7fb0 2023-10-06T02:27:06.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.576+0000 7f4b2cb9d700 1 -- 172.21.15.31:0/646498761 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b280c7fb0 msgr2=0x7f4b2815a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:06.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.576+0000 7f4b2cb9d700 1 --2- 172.21.15.31:0/646498761 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b280c7fb0 0x7f4b2815a990 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f4b280c8b00 tx=0x7f4b1c009250 comp rx=0 tx=0).stop 2023-10-06T02:27:06.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.577+0000 7f4b2cb9d700 1 -- 172.21.15.31:0/646498761 shutdown_connections 2023-10-06T02:27:06.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.577+0000 7f4b2cb9d700 1 --2- 172.21.15.31:0/646498761 >> v2:172.21.15.62:3300/0 conn(0x7f4b280c7be0 0x7f4b2806abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:06.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.577+0000 7f4b2cb9d700 1 --2- 172.21.15.31:0/646498761 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b280c7fb0 0x7f4b2815a990 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:06.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.577+0000 7f4b2cb9d700 1 -- 172.21.15.31:0/646498761 >> 172.21.15.31:0/646498761 conn(0x7f4b2806d110 msgr2=0x7f4b280cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:06.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.577+0000 7f4b2cb9d700 1 -- 172.21.15.31:0/646498761 shutdown_connections 2023-10-06T02:27:06.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.577+0000 7f4b2cb9d700 1 -- 172.21.15.31:0/646498761 wait complete. 2023-10-06T02:27:06.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.578+0000 7f4b2cb9d700 1 Processor -- start 2023-10-06T02:27:06.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.578+0000 7f4b2cb9d700 1 -- start start 2023-10-06T02:27:06.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.579+0000 7f4b2cb9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b280c7be0 0x7f4b28154f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:06.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.579+0000 7f4b2cb9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4b280c7fb0 0x7f4b281589b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:06.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.579+0000 7f4b2cb9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b2813f780 0x7f4b280c5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:06.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.579+0000 7f4b2cb9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b28158650 con 0x7f4b280c7be0 2023-10-06T02:27:06.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.579+0000 7f4b2cb9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b28155490 con 0x7f4b2813f780 2023-10-06T02:27:06.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.579+0000 7f4b2cb9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b280c4860 con 0x7f4b280c7fb0 2023-10-06T02:27:06.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.579+0000 7f4b277fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b280c7be0 0x7f4b28154f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:06.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.580+0000 7f4b277fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b280c7be0 0x7f4b28154f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51220/0 (socket says 172.21.15.31:51220) 2023-10-06T02:27:06.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.580+0000 7f4b277fe700 1 -- 172.21.15.31:0/151317829 learned_addr learned my addr 172.21.15.31:0/151317829 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:06.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.580+0000 7f4b26ffd700 1 --2- 172.21.15.31:0/151317829 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4b280c7fb0 0x7f4b281589b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:06.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.580+0000 7f4b27fff700 1 --2- 172.21.15.31:0/151317829 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b2813f780 0x7f4b280c5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:06.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.580+0000 7f4b277fe700 1 -- 172.21.15.31:0/151317829 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4b280c7fb0 msgr2=0x7f4b281589b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:06.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.580+0000 7f4b277fe700 1 --2- 172.21.15.31:0/151317829 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4b280c7fb0 0x7f4b281589b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:06.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.580+0000 7f4b277fe700 1 -- 172.21.15.31:0/151317829 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b2813f780 msgr2=0x7f4b280c5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:06.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.580+0000 7f4b277fe700 1 --2- 172.21.15.31:0/151317829 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b2813f780 0x7f4b280c5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:06.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.580+0000 7f4b277fe700 1 -- 172.21.15.31:0/151317829 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b1c0094f0 con 0x7f4b280c7be0 2023-10-06T02:27:06.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.580+0000 7f4b277fe700 1 --2- 172.21.15.31:0/151317829 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b280c7be0 0x7f4b28154f50 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f4b1800c830 tx=0x7f4b18008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:06.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.580+0000 7f4b0f7fe700 1 -- 172.21.15.31:0/151317829 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b18011600 con 0x7f4b280c7be0 2023-10-06T02:27:06.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.581+0000 7f4b0f7fe700 1 -- 172.21.15.31:0/151317829 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b18011760 con 0x7f4b280c7be0 2023-10-06T02:27:06.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.581+0000 7f4b0f7fe700 1 -- 172.21.15.31:0/151317829 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b18011b20 con 0x7f4b280c7be0 2023-10-06T02:27:06.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.581+0000 7f4b2cb9d700 1 -- 172.21.15.31:0/151317829 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b280c4b40 con 0x7f4b280c7be0 2023-10-06T02:27:06.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.581+0000 7f4b2cb9d700 1 -- 172.21.15.31:0/151317829 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b280c5100 con 0x7f4b280c7be0 2023-10-06T02:27:06.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.582+0000 7f4b2cb9d700 1 -- 172.21.15.31:0/151317829 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b2804c3c0 con 0x7f4b280c7be0 2023-10-06T02:27:06.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.583+0000 7f4b0f7fe700 1 -- 172.21.15.31:0/151317829 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f4b18018360 con 0x7f4b280c7be0 2023-10-06T02:27:06.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.583+0000 7f4b0f7fe700 1 --2- 172.21.15.31:0/151317829 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f4b14067290 0x7f4b14069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:06.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.584+0000 7f4b0f7fe700 1 -- 172.21.15.31:0/151317829 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6203+0+0 (secure 0 0 0) 0x7f4b18084c80 con 0x7f4b280c7be0 2023-10-06T02:27:06.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.584+0000 7f4b26ffd700 1 --2- 172.21.15.31:0/151317829 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f4b14067290 0x7f4b14069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:06.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.584+0000 7f4b26ffd700 1 --2- 172.21.15.31:0/151317829 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f4b14067290 0x7f4b14069740 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f4b280c8b00 tx=0x7f4b1c016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:06.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.588+0000 7f4b0f7fe700 1 -- 172.21.15.31:0/151317829 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4b1808b080 con 0x7f4b280c7be0 2023-10-06T02:27:06.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.788+0000 7f4b2cb9d700 1 -- 172.21.15.31:0/151317829 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4b28156a80 con 0x7f4b280c7be0 2023-10-06T02:27:06.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.788+0000 7f4b0f7fe700 1 -- 172.21.15.31:0/151317829 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v63) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4b180150a0 con 0x7f4b280c7be0 2023-10-06T02:27:06.790 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-10-06T02:27:06.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.790+0000 7f4b2cb9d700 1 -- 172.21.15.31:0/151317829 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f4b14067290 msgr2=0x7f4b14069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:06.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.790+0000 7f4b2cb9d700 1 --2- 172.21.15.31:0/151317829 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f4b14067290 0x7f4b14069740 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f4b280c8b00 tx=0x7f4b1c016000 comp rx=0 tx=0).stop 2023-10-06T02:27:06.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.790+0000 7f4b2cb9d700 1 -- 172.21.15.31:0/151317829 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b280c7be0 msgr2=0x7f4b28154f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:06.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.790+0000 7f4b2cb9d700 1 --2- 172.21.15.31:0/151317829 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b280c7be0 0x7f4b28154f50 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f4b1800c830 tx=0x7f4b18008040 comp rx=0 tx=0).stop 2023-10-06T02:27:06.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.790+0000 7f4b2cb9d700 1 -- 172.21.15.31:0/151317829 shutdown_connections 2023-10-06T02:27:06.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.790+0000 7f4b2cb9d700 1 --2- 172.21.15.31:0/151317829 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b2813f780 0x7f4b280c5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:06.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.790+0000 7f4b2cb9d700 1 --2- 172.21.15.31:0/151317829 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b280c7be0 0x7f4b28154f50 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:06.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.790+0000 7f4b2cb9d700 1 --2- 172.21.15.31:0/151317829 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f4b14067290 0x7f4b14069740 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:06.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.790+0000 7f4b2cb9d700 1 --2- 172.21.15.31:0/151317829 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4b280c7fb0 0x7f4b281589b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:06.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.790+0000 7f4b2cb9d700 1 -- 172.21.15.31:0/151317829 >> 172.21.15.31:0/151317829 conn(0x7f4b2806d110 msgr2=0x7f4b280c8b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:06.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.790+0000 7f4b2cb9d700 1 -- 172.21.15.31:0/151317829 shutdown_connections 2023-10-06T02:27:06.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.791+0000 7f4b2cb9d700 1 -- 172.21.15.31:0/151317829 wait complete. 2023-10-06T02:27:06.809 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:27:06.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.994+0000 7f010759e700 1 Processor -- start 2023-10-06T02:27:06.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.994+0000 7f010759e700 1 -- start start 2023-10-06T02:27:06.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.995+0000 7f010759e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f01080cf110 0x7f0108164600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:06.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.995+0000 7f010759e700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f010815afd0 con 0x7f0108153c30 2023-10-06T02:27:06.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.995+0000 7f010759e700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f010815b2b0 con 0x7f010815de50 2023-10-06T02:27:06.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.995+0000 7f010759e700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f010815b590 con 0x7f01080cf110 2023-10-06T02:27:06.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.995+0000 7f010659c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f0108153c30 0x7f010815a5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:38264/0 (socket says 172.21.15.31:38264) 2023-10-06T02:27:06.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.996+0000 7f010659c700 1 -- 172.21.15.31:0/953703588 learned_addr learned my addr 172.21.15.31:0/953703588 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:27:06.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.996+0000 7f0106d9d700 1 --2- 172.21.15.31:0/953703588 >> v2:172.21.15.31:3300/0 conn(0x7f01080cf110 0x7f0108164600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:06.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.996+0000 7f010559a700 1 -- 172.21.15.31:0/953703588 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2322503522 0 0) 0x7f010815afd0 con 0x7f0108153c30 2023-10-06T02:27:06.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.996+0000 7f010559a700 1 -- 172.21.15.31:0/953703588 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00f0003570 con 0x7f0108153c30 2023-10-06T02:27:06.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.996+0000 7f0106d9d700 1 -- 172.21.15.31:0/953703588 >> v1:172.21.15.62:6789/0 conn(0x7f010815de50 legacy=0x7f01080cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:06.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.996+0000 7f0106d9d700 1 -- 172.21.15.31:0/953703588 >> v1:172.21.15.31:6789/0 conn(0x7f0108153c30 legacy=0x7f010815a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:07.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.996+0000 7f0106d9d700 1 -- 172.21.15.31:0/953703588 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f010815b870 con 0x7f01080cf110 2023-10-06T02:27:07.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.996+0000 7f0106d9d700 1 --2- 172.21.15.31:0/953703588 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01080cf110 0x7f0108164600 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f0100009e60 tx=0x7f0100009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=254375ee4ad98a51 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:07.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.997+0000 7f010559a700 1 -- 172.21.15.31:0/953703588 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f010000e070 con 0x7f01080cf110 2023-10-06T02:27:07.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.997+0000 7f010559a700 1 -- 172.21.15.31:0/953703588 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f010000bb60 con 0x7f01080cf110 2023-10-06T02:27:07.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.997+0000 7f010559a700 1 -- 172.21.15.31:0/953703588 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0100012430 con 0x7f01080cf110 2023-10-06T02:27:07.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.997+0000 7f010759e700 1 -- 172.21.15.31:0/953703588 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01080cf110 msgr2=0x7f0108164600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:07.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.997+0000 7f010759e700 1 --2- 172.21.15.31:0/953703588 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01080cf110 0x7f0108164600 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f0100009e60 tx=0x7f0100009780 comp rx=0 tx=0).stop 2023-10-06T02:27:07.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.997+0000 7f010759e700 1 -- 172.21.15.31:0/953703588 shutdown_connections 2023-10-06T02:27:07.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.997+0000 7f010759e700 1 --2- 172.21.15.31:0/953703588 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01080cf110 0x7f0108164600 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:07.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.997+0000 7f010759e700 1 -- 172.21.15.31:0/953703588 >> 172.21.15.31:0/953703588 conn(0x7f01080703a0 msgr2=0x7f01080707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:07.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.997+0000 7f010759e700 1 -- 172.21.15.31:0/953703588 shutdown_connections 2023-10-06T02:27:07.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.997+0000 7f010759e700 1 -- 172.21.15.31:0/953703588 wait complete. 2023-10-06T02:27:07.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.999+0000 7f010759e700 1 Processor -- start 2023-10-06T02:27:07.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:06.999+0000 7f010759e700 1 -- start start 2023-10-06T02:27:07.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.000+0000 7f010759e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f01080cf110 0x7f0108140ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:07.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.000+0000 7f010759e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0108153c30 0x7f01080cbac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:07.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.000+0000 7f010759e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f010815de50 0x7f01080cc000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:07.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.000+0000 7f010759e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01080b3d10 con 0x7f0108153c30 2023-10-06T02:27:07.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.000+0000 7f010759e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01080cc5e0 con 0x7f010815de50 2023-10-06T02:27:07.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.000+0000 7f010759e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01080cc8c0 con 0x7f01080cf110 2023-10-06T02:27:07.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.000+0000 7f0105d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0108153c30 0x7f01080cbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:07.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.000+0000 7f0105d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0108153c30 0x7f01080cbac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51242/0 (socket says 172.21.15.31:51242) 2023-10-06T02:27:07.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.000+0000 7f0105d9b700 1 -- 172.21.15.31:0/2814974641 learned_addr learned my addr 172.21.15.31:0/2814974641 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:07.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.000+0000 7f0106d9d700 1 --2- 172.21.15.31:0/2814974641 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f010815de50 0x7f01080cc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:07.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.000+0000 7f010659c700 1 --2- 172.21.15.31:0/2814974641 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f01080cf110 0x7f0108140ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:07.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.000+0000 7f0105d9b700 1 -- 172.21.15.31:0/2814974641 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f01080cf110 msgr2=0x7f0108140ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:07.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.000+0000 7f0105d9b700 1 --2- 172.21.15.31:0/2814974641 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f01080cf110 0x7f0108140ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:07.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.000+0000 7f0105d9b700 1 -- 172.21.15.31:0/2814974641 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f010815de50 msgr2=0x7f01080cc000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:07.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.000+0000 7f0105d9b700 1 --2- 172.21.15.31:0/2814974641 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f010815de50 0x7f01080cc000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:07.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.000+0000 7f0105d9b700 1 -- 172.21.15.31:0/2814974641 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0100009a20 con 0x7f0108153c30 2023-10-06T02:27:07.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.000+0000 7f010659c700 1 --2- 172.21.15.31:0/2814974641 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f01080cf110 0x7f0108140ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:27:07.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.001+0000 7f0105d9b700 1 --2- 172.21.15.31:0/2814974641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0108153c30 0x7f01080cbac0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f00fc00c4e0 tx=0x7f00fc00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:07.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.001+0000 7f00f67fc700 1 -- 172.21.15.31:0/2814974641 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00fc011590 con 0x7f0108153c30 2023-10-06T02:27:07.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.001+0000 7f00f67fc700 1 -- 172.21.15.31:0/2814974641 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00fc011710 con 0x7f0108153c30 2023-10-06T02:27:07.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.001+0000 7f010759e700 1 -- 172.21.15.31:0/2814974641 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01080ccba0 con 0x7f0108153c30 2023-10-06T02:27:07.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.001+0000 7f00f67fc700 1 -- 172.21.15.31:0/2814974641 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00fc011ad0 con 0x7f0108153c30 2023-10-06T02:27:07.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.001+0000 7f010759e700 1 -- 172.21.15.31:0/2814974641 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0108141410 con 0x7f0108153c30 2023-10-06T02:27:07.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.002+0000 7f010759e700 1 -- 172.21.15.31:0/2814974641 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01080b3f10 con 0x7f0108153c30 2023-10-06T02:27:07.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.003+0000 7f00f67fc700 1 -- 172.21.15.31:0/2814974641 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f00fc01b3b0 con 0x7f0108153c30 2023-10-06T02:27:07.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.004+0000 7f00f67fc700 1 --2- 172.21.15.31:0/2814974641 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f00e80672c0 0x7f00e8069770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:07.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.004+0000 7f00f67fc700 1 -- 172.21.15.31:0/2814974641 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6203+0+0 (secure 0 0 0) 0x7f00fc084230 con 0x7f0108153c30 2023-10-06T02:27:07.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.004+0000 7f010659c700 1 --2- 172.21.15.31:0/2814974641 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f00e80672c0 0x7f00e8069770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:07.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.005+0000 7f010659c700 1 --2- 172.21.15.31:0/2814974641 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f00e80672c0 0x7f00e8069770 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f00f8007fd0 tx=0x7f00f8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:07.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.008+0000 7f00f67fc700 1 -- 172.21.15.31:0/2814974641 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f01080b3f10 con 0x7f0108153c30 2023-10-06T02:27:07.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.259+0000 7f010759e700 1 -- 172.21.15.31:0/2814974641 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f01080b3f10 con 0x7f0108153c30 2023-10-06T02:27:07.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.260+0000 7f00f67fc700 1 -- 172.21.15.31:0/2814974641 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4595 (secure 0 0 0) 0x7f01080b3f10 con 0x7f0108153c30 2023-10-06T02:27:07.261 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:27:07.261 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":38,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5216,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/2282788921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2282788921},{"type":"v1","addr":"172.21.15.62:6833","nonce":2282788921}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5219,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/289861443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":289861443},{"type":"v1","addr":"172.21.15.62:6835","nonce":289861443}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":5889,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/2954679985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":2954679985},{"type":"v1","addr":"172.21.15.31:6836","nonce":2954679985}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:26:42.554516+0000","modified":"2023-10-06T02:26:46.085562+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5892},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5892":{"gid":5892,"name":"c","rank":0,"incarnation":33,"state":"up:active","state_seq":5,"addr":"172.21.15.31:6837/1969136630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":1969136630},{"type":"v1","addr":"172.21.15.31:6837","nonce":1969136630}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-06T02:27:07.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.262+0000 7f010759e700 1 -- 172.21.15.31:0/2814974641 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f00e80672c0 msgr2=0x7f00e8069770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:07.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.262+0000 7f010759e700 1 --2- 172.21.15.31:0/2814974641 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f00e80672c0 0x7f00e8069770 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f00f8007fd0 tx=0x7f00f8009040 comp rx=0 tx=0).stop 2023-10-06T02:27:07.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.262+0000 7f010759e700 1 -- 172.21.15.31:0/2814974641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0108153c30 msgr2=0x7f01080cbac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:07.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.262+0000 7f010759e700 1 --2- 172.21.15.31:0/2814974641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0108153c30 0x7f01080cbac0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f00fc00c4e0 tx=0x7f00fc00c8a0 comp rx=0 tx=0).stop 2023-10-06T02:27:07.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.262+0000 7f010759e700 1 -- 172.21.15.31:0/2814974641 shutdown_connections 2023-10-06T02:27:07.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.262+0000 7f010759e700 1 --2- 172.21.15.31:0/2814974641 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f010815de50 0x7f01080cc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:07.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.262+0000 7f010759e700 1 --2- 172.21.15.31:0/2814974641 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f00e80672c0 0x7f00e8069770 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:07.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.262+0000 7f010759e700 1 --2- 172.21.15.31:0/2814974641 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f01080cf110 0x7f0108140ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:07.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.262+0000 7f010759e700 1 --2- 172.21.15.31:0/2814974641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0108153c30 0x7f01080cbac0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:07.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.262+0000 7f010759e700 1 -- 172.21.15.31:0/2814974641 >> 172.21.15.31:0/2814974641 conn(0x7f01080703a0 msgr2=0x7f0108070770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:07.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.262+0000 7f010759e700 1 -- 172.21.15.31:0/2814974641 shutdown_connections 2023-10-06T02:27:07.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.262+0000 7f010759e700 1 -- 172.21.15.31:0/2814974641 wait complete. 2023-10-06T02:27:07.265 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 38 2023-10-06T02:27:07.280 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:27:07.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.466+0000 7f6b82b9d700 1 Processor -- start 2023-10-06T02:27:07.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.466+0000 7f6b82b9d700 1 -- start start 2023-10-06T02:27:07.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.468+0000 7f6b82b9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f6b7c13fef0 0x7f6b7c15a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:07.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.468+0000 7f6b82b9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6b7c06aaa0 0x7f6b7c0c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:07.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.468+0000 7f6b82b9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b7c15ae70 con 0x7f6b7c13fb20 2023-10-06T02:27:07.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.468+0000 7f6b82b9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b7c06af60 con 0x7f6b7c13fef0 2023-10-06T02:27:07.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.468+0000 7f6b82b9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b7c06b240 con 0x7f6b7c06aaa0 2023-10-06T02:27:07.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.468+0000 7f6b8239c700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6b7c06aaa0 0x7f6b7c0c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:07.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.468+0000 7f6b8139a700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f6b7c13fef0 0x7f6b7c15a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:07.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.468+0000 7f6b81b9b700 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f6b7c13fb20 0x7f6b7c0c7990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:45910/0 (socket says 172.21.15.31:45910) 2023-10-06T02:27:07.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.468+0000 7f6b81b9b700 1 -- 172.21.15.31:0/3967111757 learned_addr learned my addr 172.21.15.31:0/3967111757 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:27:07.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.469+0000 7f6b80b99700 1 -- 172.21.15.31:0/3967111757 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3300046894 0 0) 0x7f6b78001110 con 0x7f6b7c13fb20 2023-10-06T02:27:07.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.469+0000 7f6b80b99700 1 -- 172.21.15.31:0/3967111757 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b68003570 con 0x7f6b7c13fb20 2023-10-06T02:27:07.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.469+0000 7f6b8139a700 1 -- 172.21.15.31:0/3967111757 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6b7c06aaa0 msgr2=0x7f6b7c0c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:07.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.469+0000 7f6b8139a700 1 --2- 172.21.15.31:0/3967111757 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6b7c06aaa0 0x7f6b7c0c9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:07.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.469+0000 7f6b8139a700 1 -- 172.21.15.31:0/3967111757 >> v1:172.21.15.62:6789/0 conn(0x7f6b7c13fb20 legacy=0x7f6b7c0c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:07.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.469+0000 7f6b8139a700 1 -- 172.21.15.31:0/3967111757 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b7c06b520 con 0x7f6b7c13fef0 2023-10-06T02:27:07.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.469+0000 7f6b8239c700 1 --2- 172.21.15.31:0/3967111757 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6b7c06aaa0 0x7f6b7c0c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:27:07.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.469+0000 7f6b8139a700 1 --2- 172.21.15.31:0/3967111757 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b7c13fef0 0x7f6b7c15a930 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c00bb70 tx=0x7f6b6c00b3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8ef54a32d8190c6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:07.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.469+0000 7f6b80b99700 1 -- 172.21.15.31:0/3967111757 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b6c005c10 con 0x7f6b7c13fef0 2023-10-06T02:27:07.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.470+0000 7f6b80b99700 1 -- 172.21.15.31:0/3967111757 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b6c005d70 con 0x7f6b7c13fef0 2023-10-06T02:27:07.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.470+0000 7f6b80b99700 1 -- 172.21.15.31:0/3967111757 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b6c00da90 con 0x7f6b7c13fef0 2023-10-06T02:27:07.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.470+0000 7f6b82b9d700 1 -- 172.21.15.31:0/3967111757 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b7c13fef0 msgr2=0x7f6b7c15a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:07.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.470+0000 7f6b82b9d700 1 --2- 172.21.15.31:0/3967111757 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b7c13fef0 0x7f6b7c15a930 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c00bb70 tx=0x7f6b6c00b3e0 comp rx=0 tx=0).stop 2023-10-06T02:27:07.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.470+0000 7f6b82b9d700 1 -- 172.21.15.31:0/3967111757 shutdown_connections 2023-10-06T02:27:07.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.470+0000 7f6b82b9d700 1 --2- 172.21.15.31:0/3967111757 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6b7c06aaa0 0x7f6b7c0c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:07.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.470+0000 7f6b82b9d700 1 --2- 172.21.15.31:0/3967111757 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b7c13fef0 0x7f6b7c15a930 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:07.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.470+0000 7f6b82b9d700 1 -- 172.21.15.31:0/3967111757 >> 172.21.15.31:0/3967111757 conn(0x7f6b7c06d170 msgr2=0x7f6b7c0cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:07.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.470+0000 7f6b82b9d700 1 -- 172.21.15.31:0/3967111757 shutdown_connections 2023-10-06T02:27:07.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.470+0000 7f6b82b9d700 1 -- 172.21.15.31:0/3967111757 wait complete. 2023-10-06T02:27:07.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.471+0000 7f6b82b9d700 1 Processor -- start 2023-10-06T02:27:07.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.472+0000 7f6b82b9d700 1 -- start start 2023-10-06T02:27:07.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.472+0000 7f6b82b9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6b7c06aaa0 0x7f6b7c155d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:07.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.472+0000 7f6b82b9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b7c13fb20 0x7f6b7c0c0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:07.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.473+0000 7f6b82b9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b7c13fef0 0x7f6b7c0c1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:07.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.473+0000 7f6b82b9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b7c0ae020 con 0x7f6b7c13fb20 2023-10-06T02:27:07.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.473+0000 7f6b82b9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b7c1562b0 con 0x7f6b7c13fef0 2023-10-06T02:27:07.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.473+0000 7f6b82b9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b7c0ad410 con 0x7f6b7c06aaa0 2023-10-06T02:27:07.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.473+0000 7f6b8139a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b7c13fb20 0x7f6b7c0c0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:07.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.473+0000 7f6b8139a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b7c13fb20 0x7f6b7c0c0ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51258/0 (socket says 172.21.15.31:51258) 2023-10-06T02:27:07.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.473+0000 7f6b8139a700 1 -- 172.21.15.31:0/2811631789 learned_addr learned my addr 172.21.15.31:0/2811631789 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:07.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.473+0000 7f6b81b9b700 1 --2- 172.21.15.31:0/2811631789 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6b7c06aaa0 0x7f6b7c155d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:07.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.473+0000 7f6b8239c700 1 --2- 172.21.15.31:0/2811631789 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b7c13fef0 0x7f6b7c0c1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:07.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.473+0000 7f6b8139a700 1 -- 172.21.15.31:0/2811631789 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6b7c06aaa0 msgr2=0x7f6b7c155d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:07.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.473+0000 7f6b8139a700 1 --2- 172.21.15.31:0/2811631789 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6b7c06aaa0 0x7f6b7c155d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:07.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.473+0000 7f6b8139a700 1 -- 172.21.15.31:0/2811631789 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b7c13fef0 msgr2=0x7f6b7c0c1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:07.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.473+0000 7f6b8139a700 1 --2- 172.21.15.31:0/2811631789 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b7c13fef0 0x7f6b7c0c1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:07.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.473+0000 7f6b8139a700 1 -- 172.21.15.31:0/2811631789 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b74009a20 con 0x7f6b7c13fb20 2023-10-06T02:27:07.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.473+0000 7f6b8239c700 1 --2- 172.21.15.31:0/2811631789 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b7c13fef0 0x7f6b7c0c1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:27:07.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.473+0000 7f6b81b9b700 1 --2- 172.21.15.31:0/2811631789 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6b7c06aaa0 0x7f6b7c155d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:27:07.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.474+0000 7f6b8139a700 1 --2- 172.21.15.31:0/2811631789 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b7c13fb20 0x7f6b7c0c0ed0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c014000 tx=0x7f6b6c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:07.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.474+0000 7f6b71ffb700 1 -- 172.21.15.31:0/2811631789 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b6c010d30 con 0x7f6b7c13fb20 2023-10-06T02:27:07.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.474+0000 7f6b71ffb700 1 -- 172.21.15.31:0/2811631789 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b6c01d070 con 0x7f6b7c13fb20 2023-10-06T02:27:07.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.474+0000 7f6b82b9d700 1 -- 172.21.15.31:0/2811631789 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b6c00b660 con 0x7f6b7c13fb20 2023-10-06T02:27:07.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.474+0000 7f6b71ffb700 1 -- 172.21.15.31:0/2811631789 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b6c019660 con 0x7f6b7c13fb20 2023-10-06T02:27:07.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.474+0000 7f6b82b9d700 1 -- 172.21.15.31:0/2811631789 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b7c0ad9d0 con 0x7f6b7c13fb20 2023-10-06T02:27:07.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.475+0000 7f6b82b9d700 1 -- 172.21.15.31:0/2811631789 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b7c0ae1d0 con 0x7f6b7c13fb20 2023-10-06T02:27:07.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.476+0000 7f6b71ffb700 1 -- 172.21.15.31:0/2811631789 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f6b6c019a10 con 0x7f6b7c13fb20 2023-10-06T02:27:07.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.477+0000 7f6b71ffb700 1 --2- 172.21.15.31:0/2811631789 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6b60067290 0x7f6b60069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:07.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.477+0000 7f6b71ffb700 1 -- 172.21.15.31:0/2811631789 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6203+0+0 (secure 0 0 0) 0x7f6b6c014030 con 0x7f6b7c13fb20 2023-10-06T02:27:07.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.477+0000 7f6b81b9b700 1 --2- 172.21.15.31:0/2811631789 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6b60067290 0x7f6b60069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:07.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.478+0000 7f6b81b9b700 1 --2- 172.21.15.31:0/2811631789 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6b60067290 0x7f6b60069740 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f6b780058d0 tx=0x7f6b78009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:07.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.481+0000 7f6b71ffb700 1 -- 172.21.15.31:0/2811631789 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6b7c0ae1d0 con 0x7f6b7c13fb20 2023-10-06T02:27:07.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.684+0000 7f6b82b9d700 1 -- 172.21.15.31:0/2811631789 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6b7c0ae390 con 0x7f6b7c13fb20 2023-10-06T02:27:07.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.685+0000 7f6b71ffb700 1 -- 172.21.15.31:0/2811631789 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+17060 (secure 0 0 0) 0x7f6b6c029020 con 0x7f6b7c13fb20 2023-10-06T02:27:07.687 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:27:07.687 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":63,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:26:49.097251+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:26:40.728441+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-10-06T02:26:41.524170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.13","mappings":[{"from":7,"to":2}]},{"pgid":"10.c","mappings":[{"from":4,"to":5}]},{"pgid":"10.21","mappings":[{"from":4,"to":3}]},{"pgid":"10.35","mappings":[{"from":4,"to":3}]},{"pgid":"10.39","mappings":[{"from":7,"to":3}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:27:07.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.687+0000 7f6b82b9d700 1 -- 172.21.15.31:0/2811631789 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6b60067290 msgr2=0x7f6b60069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:07.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.687+0000 7f6b82b9d700 1 --2- 172.21.15.31:0/2811631789 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6b60067290 0x7f6b60069740 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f6b780058d0 tx=0x7f6b78009040 comp rx=0 tx=0).stop 2023-10-06T02:27:07.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.687+0000 7f6b82b9d700 1 -- 172.21.15.31:0/2811631789 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b7c13fb20 msgr2=0x7f6b7c0c0ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:07.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.687+0000 7f6b82b9d700 1 --2- 172.21.15.31:0/2811631789 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b7c13fb20 0x7f6b7c0c0ed0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c014000 tx=0x7f6b6c01a000 comp rx=0 tx=0).stop 2023-10-06T02:27:07.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.688+0000 7f6b82b9d700 1 -- 172.21.15.31:0/2811631789 shutdown_connections 2023-10-06T02:27:07.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.688+0000 7f6b82b9d700 1 --2- 172.21.15.31:0/2811631789 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b7c13fef0 0x7f6b7c0c1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:07.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.688+0000 7f6b82b9d700 1 --2- 172.21.15.31:0/2811631789 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6b60067290 0x7f6b60069740 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:07.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.688+0000 7f6b82b9d700 1 --2- 172.21.15.31:0/2811631789 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6b7c06aaa0 0x7f6b7c155d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:07.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.688+0000 7f6b82b9d700 1 --2- 172.21.15.31:0/2811631789 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b7c13fb20 0x7f6b7c0c0ed0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:07.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.688+0000 7f6b82b9d700 1 -- 172.21.15.31:0/2811631789 >> 172.21.15.31:0/2811631789 conn(0x7f6b7c06d170 msgr2=0x7f6b7c13b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:07.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.688+0000 7f6b82b9d700 1 -- 172.21.15.31:0/2811631789 shutdown_connections 2023-10-06T02:27:07.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.688+0000 7f6b82b9d700 1 -- 172.21.15.31:0/2811631789 wait complete. 2023-10-06T02:27:07.707 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T02:27:07.707 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:27:07.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.892+0000 7fe59d39d700 1 Processor -- start 2023-10-06T02:27:07.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.893+0000 7fe59d39d700 1 -- start start 2023-10-06T02:27:07.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.894+0000 7fe59d39d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe59813fef0 0x7fe59815a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:07.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.894+0000 7fe59d39d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe59806aaa0 0x7fe5980c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:07.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.894+0000 7fe59d39d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe59815ae70 con 0x7fe59813fb20 2023-10-06T02:27:07.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.894+0000 7fe59d39d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe59806af60 con 0x7fe59806aaa0 2023-10-06T02:27:07.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.894+0000 7fe59d39d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe59806b240 con 0x7fe59813fef0 2023-10-06T02:27:07.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.894+0000 7fe5977fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe59813fef0 0x7fe59815a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:07.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.894+0000 7fe59cb9c700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe59806aaa0 0x7fe5980c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:07.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.894+0000 7fe597fff700 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fe59813fb20 0x7fe5980c7990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:56126/0 (socket says 172.21.15.31:56126) 2023-10-06T02:27:07.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.894+0000 7fe597fff700 1 -- 172.21.15.31:0/200215754 learned_addr learned my addr 172.21.15.31:0/200215754 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:27:07.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.895+0000 7fe59cb9c700 1 -- 172.21.15.31:0/200215754 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe59813fef0 msgr2=0x7fe59815a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:07.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.895+0000 7fe59cb9c700 1 --2- 172.21.15.31:0/200215754 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe59813fef0 0x7fe59815a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:07.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.895+0000 7fe59cb9c700 1 -- 172.21.15.31:0/200215754 >> v1:172.21.15.62:6789/0 conn(0x7fe59813fb20 legacy=0x7fe5980c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:07.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.895+0000 7fe59cb9c700 1 -- 172.21.15.31:0/200215754 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe59806b520 con 0x7fe59806aaa0 2023-10-06T02:27:07.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.895+0000 7fe596ffd700 1 -- 172.21.15.31:0/200215754 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 907316699 0 0) 0x7fe588001110 con 0x7fe59813fb20 2023-10-06T02:27:07.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.895+0000 7fe59cb9c700 1 --2- 172.21.15.31:0/200215754 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe59806aaa0 0x7fe5980c9d00 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe590009e60 tx=0x7fe590009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8451abb28a7a4c5e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:07.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.895+0000 7fe596ffd700 1 -- 172.21.15.31:0/200215754 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe59000e070 con 0x7fe59806aaa0 2023-10-06T02:27:07.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.895+0000 7fe596ffd700 1 -- 172.21.15.31:0/200215754 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe59000bb60 con 0x7fe59806aaa0 2023-10-06T02:27:07.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.895+0000 7fe596ffd700 1 -- 172.21.15.31:0/200215754 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe590012430 con 0x7fe59806aaa0 2023-10-06T02:27:07.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.896+0000 7fe59d39d700 1 -- 172.21.15.31:0/200215754 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe59806aaa0 msgr2=0x7fe5980c9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:07.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.896+0000 7fe59d39d700 1 --2- 172.21.15.31:0/200215754 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe59806aaa0 0x7fe5980c9d00 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe590009e60 tx=0x7fe590009780 comp rx=0 tx=0).stop 2023-10-06T02:27:07.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.896+0000 7fe59d39d700 1 -- 172.21.15.31:0/200215754 shutdown_connections 2023-10-06T02:27:07.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.896+0000 7fe59d39d700 1 --2- 172.21.15.31:0/200215754 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe59806aaa0 0x7fe5980c9d00 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:07.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.896+0000 7fe59d39d700 1 --2- 172.21.15.31:0/200215754 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe59813fef0 0x7fe59815a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:07.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.896+0000 7fe59d39d700 1 -- 172.21.15.31:0/200215754 >> 172.21.15.31:0/200215754 conn(0x7fe59806d170 msgr2=0x7fe5980cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:07.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.896+0000 7fe59d39d700 1 -- 172.21.15.31:0/200215754 shutdown_connections 2023-10-06T02:27:07.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.896+0000 7fe59d39d700 1 -- 172.21.15.31:0/200215754 wait complete. 2023-10-06T02:27:07.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.898+0000 7fe59d39d700 1 Processor -- start 2023-10-06T02:27:07.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.898+0000 7fe59d39d700 1 -- start start 2023-10-06T02:27:07.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.899+0000 7fe59d39d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe59806aaa0 0x7fe5980ad530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:07.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.899+0000 7fe59d39d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe59813fb20 0x7fe59813af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:07.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.899+0000 7fe59d39d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe59813fef0 0x7fe59815c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:07.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.899+0000 7fe59d39d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5980ada70 con 0x7fe59813fef0 2023-10-06T02:27:07.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.899+0000 7fe59d39d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5980ac6f0 con 0x7fe59813fb20 2023-10-06T02:27:07.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.899+0000 7fe59d39d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5980ac9d0 con 0x7fe59806aaa0 2023-10-06T02:27:07.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.899+0000 7fe59cb9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe59813fef0 0x7fe59815c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:07.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.899+0000 7fe59cb9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe59813fef0 0x7fe59815c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50426/0 (socket says 172.21.15.31:50426) 2023-10-06T02:27:07.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.899+0000 7fe59cb9c700 1 -- 172.21.15.31:0/3462188678 learned_addr learned my addr 172.21.15.31:0/3462188678 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:07.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.899+0000 7fe597fff700 1 --2- 172.21.15.31:0/3462188678 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe59806aaa0 0x7fe5980ad530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:07.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.899+0000 7fe5977fe700 1 --2- 172.21.15.31:0/3462188678 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe59813fb20 0x7fe59813af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:07.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.899+0000 7fe59cb9c700 1 -- 172.21.15.31:0/3462188678 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe59806aaa0 msgr2=0x7fe5980ad530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:07.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.899+0000 7fe59cb9c700 1 --2- 172.21.15.31:0/3462188678 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe59806aaa0 0x7fe5980ad530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:07.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.900+0000 7fe59cb9c700 1 -- 172.21.15.31:0/3462188678 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe59813fb20 msgr2=0x7fe59813af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:07.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.900+0000 7fe59cb9c700 1 --2- 172.21.15.31:0/3462188678 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe59813fb20 0x7fe59813af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:07.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.900+0000 7fe59cb9c700 1 -- 172.21.15.31:0/3462188678 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe590009a20 con 0x7fe59813fef0 2023-10-06T02:27:07.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.900+0000 7fe59cb9c700 1 --2- 172.21.15.31:0/3462188678 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe59813fef0 0x7fe59815c560 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fe590013000 tx=0x7fe590019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:07.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.900+0000 7fe57ffff700 1 -- 172.21.15.31:0/3462188678 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe590012ea0 con 0x7fe59813fef0 2023-10-06T02:27:07.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.900+0000 7fe57ffff700 1 -- 172.21.15.31:0/3462188678 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe590027470 con 0x7fe59813fef0 2023-10-06T02:27:07.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.900+0000 7fe57ffff700 1 -- 172.21.15.31:0/3462188678 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe590027830 con 0x7fe59813fef0 2023-10-06T02:27:07.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.900+0000 7fe59d39d700 1 -- 172.21.15.31:0/3462188678 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5980accb0 con 0x7fe59813fef0 2023-10-06T02:27:07.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.900+0000 7fe59d39d700 1 -- 172.21.15.31:0/3462188678 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe59813b4b0 con 0x7fe59813fef0 2023-10-06T02:27:07.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.901+0000 7fe59d39d700 1 -- 172.21.15.31:0/3462188678 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe59804c3c0 con 0x7fe59813fef0 2023-10-06T02:27:07.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.902+0000 7fe57ffff700 1 -- 172.21.15.31:0/3462188678 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fe590024070 con 0x7fe59813fef0 2023-10-06T02:27:07.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.903+0000 7fe57ffff700 1 --2- 172.21.15.31:0/3462188678 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe584067290 0x7fe584069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:07.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.904+0000 7fe57ffff700 1 -- 172.21.15.31:0/3462188678 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6203+0+0 (secure 0 0 0) 0x7fe590087370 con 0x7fe59813fef0 2023-10-06T02:27:07.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.904+0000 7fe597fff700 1 --2- 172.21.15.31:0/3462188678 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe584067290 0x7fe584069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:07.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.904+0000 7fe597fff700 1 --2- 172.21.15.31:0/3462188678 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe584067290 0x7fe584069740 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fe588006fc0 tx=0x7fe588007b90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:07.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:07.908+0000 7fe57ffff700 1 -- 172.21.15.31:0/3462188678 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe590053fe0 con 0x7fe59813fef0 2023-10-06T02:27:08.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.153+0000 7fe59d39d700 1 -- 172.21.15.31:0/3462188678 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe5980c9840 con 0x7fe59813fef0 2023-10-06T02:27:08.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.154+0000 7fe57ffff700 1 -- 172.21.15.31:0/3462188678 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4595 (secure 0 0 0) 0x7fe590057600 con 0x7fe59813fef0 2023-10-06T02:27:08.156 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:27:08.156 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":38,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5216,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/2282788921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2282788921},{"type":"v1","addr":"172.21.15.62:6833","nonce":2282788921}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5219,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/289861443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":289861443},{"type":"v1","addr":"172.21.15.62:6835","nonce":289861443}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":5889,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/2954679985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":2954679985},{"type":"v1","addr":"172.21.15.31:6836","nonce":2954679985}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:26:42.554516+0000","modified":"2023-10-06T02:26:46.085562+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5892},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5892":{"gid":5892,"name":"c","rank":0,"incarnation":33,"state":"up:active","state_seq":5,"addr":"172.21.15.31:6837/1969136630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":1969136630},{"type":"v1","addr":"172.21.15.31:6837","nonce":1969136630}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-06T02:27:08.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.156+0000 7fe59d39d700 1 -- 172.21.15.31:0/3462188678 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe584067290 msgr2=0x7fe584069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:08.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.156+0000 7fe59d39d700 1 --2- 172.21.15.31:0/3462188678 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe584067290 0x7fe584069740 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fe588006fc0 tx=0x7fe588007b90 comp rx=0 tx=0).stop 2023-10-06T02:27:08.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.156+0000 7fe59d39d700 1 -- 172.21.15.31:0/3462188678 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe59813fef0 msgr2=0x7fe59815c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:08.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.156+0000 7fe59d39d700 1 --2- 172.21.15.31:0/3462188678 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe59813fef0 0x7fe59815c560 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fe590013000 tx=0x7fe590019000 comp rx=0 tx=0).stop 2023-10-06T02:27:08.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.156+0000 7fe59d39d700 1 -- 172.21.15.31:0/3462188678 shutdown_connections 2023-10-06T02:27:08.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.156+0000 7fe59d39d700 1 --2- 172.21.15.31:0/3462188678 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe59813fef0 0x7fe59815c560 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:08.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.156+0000 7fe59d39d700 1 --2- 172.21.15.31:0/3462188678 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe584067290 0x7fe584069740 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:08.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.156+0000 7fe59d39d700 1 --2- 172.21.15.31:0/3462188678 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe59806aaa0 0x7fe5980ad530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:08.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.156+0000 7fe59d39d700 1 --2- 172.21.15.31:0/3462188678 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe59813fb20 0x7fe59813af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:08.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.156+0000 7fe59d39d700 1 -- 172.21.15.31:0/3462188678 >> 172.21.15.31:0/3462188678 conn(0x7fe59806d170 msgr2=0x7fe5980ca570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:08.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.156+0000 7fe59d39d700 1 -- 172.21.15.31:0/3462188678 shutdown_connections 2023-10-06T02:27:08.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.156+0000 7fe59d39d700 1 -- 172.21.15.31:0/3462188678 wait complete. 2023-10-06T02:27:08.159 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 38 2023-10-06T02:27:08.175 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:27:08.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.362+0000 7faeabfff700 1 Processor -- start 2023-10-06T02:27:08.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.362+0000 7faeabfff700 1 -- start start 2023-10-06T02:27:08.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.364+0000 7faeabfff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faeac150380 0x7faeac160ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:08.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.364+0000 7faeabfff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faeac06a490 con 0x7faeac06f3a0 2023-10-06T02:27:08.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.364+0000 7faeabfff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faeac0c7230 con 0x7faeac06e0a0 2023-10-06T02:27:08.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.364+0000 7faeabfff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faeac06b930 con 0x7faeac150380 2023-10-06T02:27:08.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.364+0000 7faeaa7fc700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7faeac06f3a0 0x7faeac14fc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:47708/0 (socket says 172.21.15.31:47708) 2023-10-06T02:27:08.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.364+0000 7faeaa7fc700 1 -- 172.21.15.31:0/2017995502 learned_addr learned my addr 172.21.15.31:0/2017995502 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:27:08.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.364+0000 7faeab7fe700 1 --2- 172.21.15.31:0/2017995502 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faeac150380 0x7faeac160ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:08.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.364+0000 7faea9ffb700 1 -- 172.21.15.31:0/2017995502 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1623391980 0 0) 0x7faeac06a490 con 0x7faeac06f3a0 2023-10-06T02:27:08.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.364+0000 7faea9ffb700 1 -- 172.21.15.31:0/2017995502 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae94003570 con 0x7faeac06f3a0 2023-10-06T02:27:08.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.365+0000 7faea9ffb700 1 -- 172.21.15.31:0/2017995502 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3959003126 0 0) 0x7fae94003570 con 0x7faeac06f3a0 2023-10-06T02:27:08.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.365+0000 7faea9ffb700 1 -- 172.21.15.31:0/2017995502 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faeac150380 msgr2=0x7faeac160ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:08.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.365+0000 7faea9ffb700 1 --2- 172.21.15.31:0/2017995502 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faeac150380 0x7faeac160ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:08.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.365+0000 7faea9ffb700 1 -- 172.21.15.31:0/2017995502 >> v1:172.21.15.62:6789/0 conn(0x7faeac06e0a0 legacy=0x7faeac0c6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:08.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.365+0000 7faea9ffb700 1 -- 172.21.15.31:0/2017995502 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeac0c7490 con 0x7faeac06f3a0 2023-10-06T02:27:08.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.365+0000 7faea9ffb700 1 -- 172.21.15.31:0/2017995502 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7faea00015f0 con 0x7faeac06f3a0 2023-10-06T02:27:08.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.365+0000 7faea9ffb700 1 -- 172.21.15.31:0/2017995502 >> v1:172.21.15.31:6789/0 conn(0x7faeac06f3a0 legacy=0x7faeac14fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:08.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.365+0000 7faea9ffb700 1 --2- 172.21.15.31:0/2017995502 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faeac150380 0x7fae94003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:08.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.365+0000 7faea9ffb700 1 --2- 172.21.15.31:0/2017995502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae94003fd0 0x7fae94004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:08.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.365+0000 7faea9ffb700 1 --2- 172.21.15.31:0/2017995502 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae94004d50 0x7fae940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:08.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.365+0000 7faea9ffb700 1 -- 172.21.15.31:0/2017995502 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeac06b930 con 0x7fae94003fd0 2023-10-06T02:27:08.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.365+0000 7faea9ffb700 1 -- 172.21.15.31:0/2017995502 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae94005b40 con 0x7fae94004d50 2023-10-06T02:27:08.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.365+0000 7faea9ffb700 1 -- 172.21.15.31:0/2017995502 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae94005d00 con 0x7faeac150380 2023-10-06T02:27:08.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.365+0000 7faeaa7fc700 1 --2- 172.21.15.31:0/2017995502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae94003fd0 0x7fae94004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:08.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.365+0000 7faeaaffd700 1 --2- 172.21.15.31:0/2017995502 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faeac150380 0x7fae94003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:08.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.365+0000 7faeab7fe700 1 --2- 172.21.15.31:0/2017995502 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae94004d50 0x7fae940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:08.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.366+0000 7faeaa7fc700 1 -- 172.21.15.31:0/2017995502 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faeac150380 msgr2=0x7fae94003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:08.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.366+0000 7faeaa7fc700 1 --2- 172.21.15.31:0/2017995502 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faeac150380 0x7fae94003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:08.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.366+0000 7faeaa7fc700 1 -- 172.21.15.31:0/2017995502 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae94004d50 msgr2=0x7fae940051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:08.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.366+0000 7faeaa7fc700 1 --2- 172.21.15.31:0/2017995502 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae94004d50 0x7fae940051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:08.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.366+0000 7faeaa7fc700 1 -- 172.21.15.31:0/2017995502 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae94003570 con 0x7fae94003fd0 2023-10-06T02:27:08.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.366+0000 7faeaa7fc700 1 --2- 172.21.15.31:0/2017995502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae94003fd0 0x7fae94004810 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7faea000c2d0 tx=0x7faea00078a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3ea249a526d707e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:08.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.366+0000 7faea9ffb700 1 -- 172.21.15.31:0/2017995502 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea000cc00 con 0x7fae94003fd0 2023-10-06T02:27:08.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.366+0000 7faea9ffb700 1 -- 172.21.15.31:0/2017995502 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faea000cd60 con 0x7fae94003fd0 2023-10-06T02:27:08.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.366+0000 7faea9ffb700 1 -- 172.21.15.31:0/2017995502 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea0007320 con 0x7fae94003fd0 2023-10-06T02:27:08.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.367+0000 7faeabfff700 1 -- 172.21.15.31:0/2017995502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae94003fd0 msgr2=0x7fae94004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:08.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.367+0000 7faeabfff700 1 --2- 172.21.15.31:0/2017995502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae94003fd0 0x7fae94004810 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7faea000c2d0 tx=0x7faea00078a0 comp rx=0 tx=0).stop 2023-10-06T02:27:08.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.367+0000 7faeab7fe700 1 -- 172.21.15.31:0/2017995502 reap_dead start 2023-10-06T02:27:08.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.367+0000 7faeabfff700 1 -- 172.21.15.31:0/2017995502 shutdown_connections 2023-10-06T02:27:08.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.367+0000 7faeabfff700 1 -- 172.21.15.31:0/2017995502 >> 172.21.15.31:0/2017995502 conn(0x7faeac06d170 msgr2=0x7faeac0cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:08.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.367+0000 7faeabfff700 1 -- 172.21.15.31:0/2017995502 shutdown_connections 2023-10-06T02:27:08.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.367+0000 7faeabfff700 1 -- 172.21.15.31:0/2017995502 wait complete. 2023-10-06T02:27:08.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.368+0000 7faeabfff700 1 Processor -- start 2023-10-06T02:27:08.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.368+0000 7faeabfff700 1 -- start start 2023-10-06T02:27:08.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.369+0000 7faeabfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae94003fd0 0x7faeac0b17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:08.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.369+0000 7faeabfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faeac0b0980 0x7faeac147c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:08.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.369+0000 7faeabfff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faeac14bec0 0x7faeac14c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:08.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.369+0000 7faeabfff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeac14c880 con 0x7faeac0b0980 2023-10-06T02:27:08.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.369+0000 7faeabfff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeac0afec0 con 0x7fae94003fd0 2023-10-06T02:27:08.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.369+0000 7faeabfff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faeac0af2b0 con 0x7faeac14bec0 2023-10-06T02:27:08.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.370+0000 7faeaaffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae94003fd0 0x7faeac0b17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:08.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.370+0000 7faeaa7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faeac0b0980 0x7faeac147c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:08.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.370+0000 7faeaaffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae94003fd0 0x7faeac0b17c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:34336/0 (socket says 172.21.15.31:34336) 2023-10-06T02:27:08.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.370+0000 7faeaaffd700 1 -- 172.21.15.31:0/306570447 learned_addr learned my addr 172.21.15.31:0/306570447 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:08.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.370+0000 7faeab7fe700 1 --2- 172.21.15.31:0/306570447 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faeac14bec0 0x7faeac14c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:08.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.370+0000 7faeaa7fc700 1 -- 172.21.15.31:0/306570447 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faeac14bec0 msgr2=0x7faeac14c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:08.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.370+0000 7faeaa7fc700 1 --2- 172.21.15.31:0/306570447 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faeac14bec0 0x7faeac14c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:08.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.370+0000 7faeaa7fc700 1 -- 172.21.15.31:0/306570447 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae94003fd0 msgr2=0x7faeac0b17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:08.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.370+0000 7faeaa7fc700 1 --2- 172.21.15.31:0/306570447 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae94003fd0 0x7faeac0b17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:08.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.370+0000 7faeaa7fc700 1 -- 172.21.15.31:0/306570447 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeac0af590 con 0x7faeac0b0980 2023-10-06T02:27:08.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.370+0000 7faeaaffd700 1 --2- 172.21.15.31:0/306570447 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae94003fd0 0x7faeac0b17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:27:08.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.370+0000 7faeaa7fc700 1 --2- 172.21.15.31:0/306570447 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faeac0b0980 0x7faeac147c10 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7faea0014000 tx=0x7faea0015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:08.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.370+0000 7fae92ffd700 1 -- 172.21.15.31:0/306570447 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea000cc00 con 0x7faeac0b0980 2023-10-06T02:27:08.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.371+0000 7fae92ffd700 1 -- 172.21.15.31:0/306570447 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faea000ced0 con 0x7faeac0b0980 2023-10-06T02:27:08.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.371+0000 7faeabfff700 1 -- 172.21.15.31:0/306570447 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faeac0af870 con 0x7faeac0b0980 2023-10-06T02:27:08.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.371+0000 7fae92ffd700 1 -- 172.21.15.31:0/306570447 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea0005c50 con 0x7faeac0b0980 2023-10-06T02:27:08.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.371+0000 7faeabfff700 1 -- 172.21.15.31:0/306570447 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faeac148150 con 0x7faeac0b0980 2023-10-06T02:27:08.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.372+0000 7faeabfff700 1 -- 172.21.15.31:0/306570447 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faeac0b0070 con 0x7faeac0b0980 2023-10-06T02:27:08.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.373+0000 7fae92ffd700 1 -- 172.21.15.31:0/306570447 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7faea0019030 con 0x7faeac0b0980 2023-10-06T02:27:08.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.373+0000 7fae92ffd700 1 --2- 172.21.15.31:0/306570447 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fae98067290 0x7fae98069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:08.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.373+0000 7fae92ffd700 1 -- 172.21.15.31:0/306570447 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6203+0+0 (secure 0 0 0) 0x7faea0086230 con 0x7faeac0b0980 2023-10-06T02:27:08.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.374+0000 7faeaaffd700 1 --2- 172.21.15.31:0/306570447 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fae98067290 0x7fae98069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:08.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.374+0000 7faeaaffd700 1 --2- 172.21.15.31:0/306570447 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fae98067290 0x7fae98069740 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fae9c0010e0 tx=0x7fae9c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:08.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.378+0000 7fae92ffd700 1 -- 172.21.15.31:0/306570447 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faeac0b0070 con 0x7faeac0b0980 2023-10-06T02:27:08.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.623+0000 7faeabfff700 1 -- 172.21.15.31:0/306570447 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faeac0b0030 con 0x7faeac0b0980 2023-10-06T02:27:08.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.624+0000 7fae92ffd700 1 -- 172.21.15.31:0/306570447 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4595 (secure 0 0 0) 0x7faea0052da0 con 0x7faeac0b0980 2023-10-06T02:27:08.625 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:27:08.625 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":38,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5216,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/2282788921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2282788921},{"type":"v1","addr":"172.21.15.62:6833","nonce":2282788921}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5219,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/289861443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":289861443},{"type":"v1","addr":"172.21.15.62:6835","nonce":289861443}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":5889,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/2954679985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":2954679985},{"type":"v1","addr":"172.21.15.31:6836","nonce":2954679985}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:26:42.554516+0000","modified":"2023-10-06T02:26:46.085562+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5892},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5892":{"gid":5892,"name":"c","rank":0,"incarnation":33,"state":"up:active","state_seq":5,"addr":"172.21.15.31:6837/1969136630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":1969136630},{"type":"v1","addr":"172.21.15.31:6837","nonce":1969136630}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-06T02:27:08.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.626+0000 7faeabfff700 1 -- 172.21.15.31:0/306570447 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fae98067290 msgr2=0x7fae98069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:08.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.626+0000 7faeabfff700 1 --2- 172.21.15.31:0/306570447 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fae98067290 0x7fae98069740 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fae9c0010e0 tx=0x7fae9c009040 comp rx=0 tx=0).stop 2023-10-06T02:27:08.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.626+0000 7faeabfff700 1 -- 172.21.15.31:0/306570447 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faeac0b0980 msgr2=0x7faeac147c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:08.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.626+0000 7faeabfff700 1 --2- 172.21.15.31:0/306570447 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faeac0b0980 0x7faeac147c10 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7faea0014000 tx=0x7faea0015000 comp rx=0 tx=0).stop 2023-10-06T02:27:08.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.626+0000 7faeabfff700 1 -- 172.21.15.31:0/306570447 shutdown_connections 2023-10-06T02:27:08.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.626+0000 7faeabfff700 1 --2- 172.21.15.31:0/306570447 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fae98067290 0x7fae98069740 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:08.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.626+0000 7faeabfff700 1 --2- 172.21.15.31:0/306570447 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faeac14bec0 0x7faeac14c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:08.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.626+0000 7faeabfff700 1 --2- 172.21.15.31:0/306570447 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae94003fd0 0x7faeac0b17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:08.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.626+0000 7faeabfff700 1 --2- 172.21.15.31:0/306570447 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faeac0b0980 0x7faeac147c10 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:08.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.626+0000 7faeabfff700 1 -- 172.21.15.31:0/306570447 >> 172.21.15.31:0/306570447 conn(0x7faeac06d170 msgr2=0x7faeac14a830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:08.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.626+0000 7faeabfff700 1 -- 172.21.15.31:0/306570447 shutdown_connections 2023-10-06T02:27:08.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.627+0000 7faeabfff700 1 -- 172.21.15.31:0/306570447 wait complete. 2023-10-06T02:27:08.631 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 38 2023-10-06T02:27:08.646 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:27:08.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.831+0000 7ff4e639d700 1 Processor -- start 2023-10-06T02:27:08.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.831+0000 7ff4e639d700 1 -- start start 2023-10-06T02:27:08.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.833+0000 7ff4e639d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff4e013fb20 0x7ff4e015a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:08.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.833+0000 7ff4e639d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4e015ae70 con 0x7ff4e013fef0 2023-10-06T02:27:08.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.833+0000 7ff4e639d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4e006af60 con 0x7ff4e006aaa0 2023-10-06T02:27:08.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.833+0000 7ff4e639d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4e006b240 con 0x7ff4e013fb20 2023-10-06T02:27:08.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.833+0000 7ff4e539b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff4e013fb20 0x7ff4e015a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:08.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.833+0000 7ff4e4b9a700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7ff4e013fef0 0x7ff4e00c7990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:47716/0 (socket says 172.21.15.31:47716) 2023-10-06T02:27:08.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.833+0000 7ff4e4b9a700 1 -- 172.21.15.31:0/1778159294 learned_addr learned my addr 172.21.15.31:0/1778159294 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:27:08.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.834+0000 7ff4e539b700 1 -- 172.21.15.31:0/1778159294 >> v1:172.21.15.62:6789/0 conn(0x7ff4e006aaa0 legacy=0x7ff4e00c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:08.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.834+0000 7ff4e539b700 1 -- 172.21.15.31:0/1778159294 >> v1:172.21.15.31:6789/0 conn(0x7ff4e013fef0 legacy=0x7ff4e00c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:08.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.834+0000 7ff4e539b700 1 -- 172.21.15.31:0/1778159294 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4e006b520 con 0x7ff4e013fb20 2023-10-06T02:27:08.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.834+0000 7ff4e539b700 1 --2- 172.21.15.31:0/1778159294 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4e013fb20 0x7ff4e015a930 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc0098e0 tx=0x7ff4dc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bacf5fb39ded92d8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:08.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.834+0000 7ff4d7fff700 1 -- 172.21.15.31:0/1778159294 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4dc009e80 con 0x7ff4e013fb20 2023-10-06T02:27:08.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.834+0000 7ff4d7fff700 1 -- 172.21.15.31:0/1778159294 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4dc00b820 con 0x7ff4e013fb20 2023-10-06T02:27:08.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.834+0000 7ff4d7fff700 1 -- 172.21.15.31:0/1778159294 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4dc00bba0 con 0x7ff4e013fb20 2023-10-06T02:27:08.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.835+0000 7ff4e639d700 1 -- 172.21.15.31:0/1778159294 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4e013fb20 msgr2=0x7ff4e015a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:08.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.835+0000 7ff4e639d700 1 --2- 172.21.15.31:0/1778159294 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4e013fb20 0x7ff4e015a930 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc0098e0 tx=0x7ff4dc0091a0 comp rx=0 tx=0).stop 2023-10-06T02:27:08.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.835+0000 7ff4e639d700 1 -- 172.21.15.31:0/1778159294 shutdown_connections 2023-10-06T02:27:08.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.835+0000 7ff4e639d700 1 --2- 172.21.15.31:0/1778159294 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4e013fb20 0x7ff4e015a930 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:08.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.835+0000 7ff4e639d700 1 -- 172.21.15.31:0/1778159294 >> 172.21.15.31:0/1778159294 conn(0x7ff4e006d170 msgr2=0x7ff4e00cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:08.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.835+0000 7ff4e639d700 1 -- 172.21.15.31:0/1778159294 shutdown_connections 2023-10-06T02:27:08.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.835+0000 7ff4e639d700 1 -- 172.21.15.31:0/1778159294 wait complete. 2023-10-06T02:27:08.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.836+0000 7ff4e639d700 1 Processor -- start 2023-10-06T02:27:08.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.837+0000 7ff4e639d700 1 -- start start 2023-10-06T02:27:08.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.837+0000 7ff4e639d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff4e006aaa0 0x7ff4e01562c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:08.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.837+0000 7ff4e639d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4e013fb20 0x7ff4e00ad890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:08.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.838+0000 7ff4e639d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4e013fef0 0x7ff4e00c75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:08.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.838+0000 7ff4e639d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e00addd0 con 0x7ff4e013fef0 2023-10-06T02:27:08.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.838+0000 7ff4e639d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e0158940 con 0x7ff4e013fb20 2023-10-06T02:27:08.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.838+0000 7ff4e639d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e0157a90 con 0x7ff4e006aaa0 2023-10-06T02:27:08.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.838+0000 7ff4e5b9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4e013fef0 0x7ff4e00c75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:08.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.838+0000 7ff4e5b9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4e013fef0 0x7ff4e00c75f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50450/0 (socket says 172.21.15.31:50450) 2023-10-06T02:27:08.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.838+0000 7ff4e5b9c700 1 -- 172.21.15.31:0/3127841967 learned_addr learned my addr 172.21.15.31:0/3127841967 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:08.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.838+0000 7ff4e539b700 1 --2- 172.21.15.31:0/3127841967 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff4e006aaa0 0x7ff4e01562c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:08.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.838+0000 7ff4e4b9a700 1 --2- 172.21.15.31:0/3127841967 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4e013fb20 0x7ff4e00ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:08.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.838+0000 7ff4e5b9c700 1 -- 172.21.15.31:0/3127841967 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff4e006aaa0 msgr2=0x7ff4e01562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:08.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.838+0000 7ff4e5b9c700 1 --2- 172.21.15.31:0/3127841967 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff4e006aaa0 0x7ff4e01562c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:08.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.838+0000 7ff4e5b9c700 1 -- 172.21.15.31:0/3127841967 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4e013fb20 msgr2=0x7ff4e00ad890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:08.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.838+0000 7ff4e5b9c700 1 --2- 172.21.15.31:0/3127841967 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4e013fb20 0x7ff4e00ad890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:08.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.838+0000 7ff4e5b9c700 1 -- 172.21.15.31:0/3127841967 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4dc009420 con 0x7ff4e013fef0 2023-10-06T02:27:08.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.839+0000 7ff4e5b9c700 1 --2- 172.21.15.31:0/3127841967 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4e013fef0 0x7ff4e00c75f0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7ff4d800c890 tx=0x7ff4d80026a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:08.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.839+0000 7ff4d57fa700 1 -- 172.21.15.31:0/3127841967 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d8012630 con 0x7ff4e013fef0 2023-10-06T02:27:08.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.839+0000 7ff4d57fa700 1 -- 172.21.15.31:0/3127841967 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4d8012790 con 0x7ff4e013fef0 2023-10-06T02:27:08.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.839+0000 7ff4d57fa700 1 -- 172.21.15.31:0/3127841967 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d8012b50 con 0x7ff4e013fef0 2023-10-06T02:27:08.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.839+0000 7ff4e639d700 1 -- 172.21.15.31:0/3127841967 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4e0157d30 con 0x7ff4e013fef0 2023-10-06T02:27:08.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.839+0000 7ff4e639d700 1 -- 172.21.15.31:0/3127841967 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4e01582f0 con 0x7ff4e013fef0 2023-10-06T02:27:08.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.840+0000 7ff4e639d700 1 -- 172.21.15.31:0/3127841967 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4e0158ad0 con 0x7ff4e013fef0 2023-10-06T02:27:08.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.841+0000 7ff4d57fa700 1 -- 172.21.15.31:0/3127841967 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7ff4d800a420 con 0x7ff4e013fef0 2023-10-06T02:27:08.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.842+0000 7ff4d57fa700 1 --2- 172.21.15.31:0/3127841967 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7ff4c4067290 0x7ff4c4069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:08.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.842+0000 7ff4d57fa700 1 -- 172.21.15.31:0/3127841967 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6203+0+0 (secure 0 0 0) 0x7ff4d8085060 con 0x7ff4e013fef0 2023-10-06T02:27:08.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.842+0000 7ff4e539b700 1 --2- 172.21.15.31:0/3127841967 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7ff4c4067290 0x7ff4c4069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:08.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.843+0000 7ff4e539b700 1 --2- 172.21.15.31:0/3127841967 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7ff4c4067290 0x7ff4c4069740 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc0030c0 tx=0x7ff4dc016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:08.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:08.846+0000 7ff4d57fa700 1 -- 172.21.15.31:0/3127841967 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff4d8051c90 con 0x7ff4e013fef0 2023-10-06T02:27:09.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.050+0000 7ff4e639d700 1 -- 172.21.15.31:0/3127841967 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff4e004c3c0 con 0x7ff4e013fef0 2023-10-06T02:27:09.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.051+0000 7ff4d57fa700 1 -- 172.21.15.31:0/3127841967 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+17060 (secure 0 0 0) 0x7ff4d80552b0 con 0x7ff4e013fef0 2023-10-06T02:27:09.053 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:27:09.053 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":63,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:26:49.097251+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:26:40.728441+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-10-06T02:26:41.524170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.13","mappings":[{"from":7,"to":2}]},{"pgid":"10.c","mappings":[{"from":4,"to":5}]},{"pgid":"10.21","mappings":[{"from":4,"to":3}]},{"pgid":"10.35","mappings":[{"from":4,"to":3}]},{"pgid":"10.39","mappings":[{"from":7,"to":3}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:27:09.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.053+0000 7ff4e639d700 1 -- 172.21.15.31:0/3127841967 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7ff4c4067290 msgr2=0x7ff4c4069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:09.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.053+0000 7ff4e639d700 1 --2- 172.21.15.31:0/3127841967 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7ff4c4067290 0x7ff4c4069740 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc0030c0 tx=0x7ff4dc016000 comp rx=0 tx=0).stop 2023-10-06T02:27:09.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.053+0000 7ff4e639d700 1 -- 172.21.15.31:0/3127841967 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4e013fef0 msgr2=0x7ff4e00c75f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:09.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.053+0000 7ff4e639d700 1 --2- 172.21.15.31:0/3127841967 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4e013fef0 0x7ff4e00c75f0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7ff4d800c890 tx=0x7ff4d80026a0 comp rx=0 tx=0).stop 2023-10-06T02:27:09.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.053+0000 7ff4e639d700 1 -- 172.21.15.31:0/3127841967 shutdown_connections 2023-10-06T02:27:09.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.053+0000 7ff4e639d700 1 --2- 172.21.15.31:0/3127841967 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4e013fef0 0x7ff4e00c75f0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:09.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.053+0000 7ff4e639d700 1 --2- 172.21.15.31:0/3127841967 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7ff4c4067290 0x7ff4c4069740 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:09.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.053+0000 7ff4e639d700 1 --2- 172.21.15.31:0/3127841967 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff4e006aaa0 0x7ff4e01562c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:09.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.053+0000 7ff4e639d700 1 --2- 172.21.15.31:0/3127841967 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4e013fb20 0x7ff4e00ad890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:09.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.053+0000 7ff4e639d700 1 -- 172.21.15.31:0/3127841967 >> 172.21.15.31:0/3127841967 conn(0x7ff4e006d170 msgr2=0x7ff4e006ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:09.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.053+0000 7ff4e639d700 1 -- 172.21.15.31:0/3127841967 shutdown_connections 2023-10-06T02:27:09.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.054+0000 7ff4e639d700 1 -- 172.21.15.31:0/3127841967 wait complete. 2023-10-06T02:27:09.072 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T02:27:09.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.260+0000 7ff17159d700 1 Processor -- start 2023-10-06T02:27:09.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.260+0000 7ff17159d700 1 -- start start 2023-10-06T02:27:09.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.261+0000 7ff17159d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff16c159570 0x7ff16c0b23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:09.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.261+0000 7ff17159d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff16c159940 0x7ff16c06d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:09.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.261+0000 7ff17159d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff16c0b3270 con 0x7ff16c15dea0 2023-10-06T02:27:09.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.261+0000 7ff17159d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff16c0b3550 con 0x7ff16c159570 2023-10-06T02:27:09.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.261+0000 7ff17159d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff16c0b3830 con 0x7ff16c159940 2023-10-06T02:27:09.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.261+0000 7ff16bfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff16c159570 0x7ff16c0b23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:09.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.262+0000 7ff16bfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff16c159570 0x7ff16c0b23e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50462/0 (socket says 172.21.15.31:50462) 2023-10-06T02:27:09.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.262+0000 7ff16bfff700 1 -- 172.21.15.31:0/3372024275 learned_addr learned my addr 172.21.15.31:0/3372024275 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:09.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.262+0000 7ff16b7fe700 1 --2- 172.21.15.31:0/3372024275 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff16c159940 0x7ff16c06d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:09.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.262+0000 7ff16bfff700 1 -- 172.21.15.31:0/3372024275 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff16c159940 msgr2=0x7ff16c06d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:09.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.262+0000 7ff16bfff700 1 --2- 172.21.15.31:0/3372024275 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff16c159940 0x7ff16c06d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:09.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.262+0000 7ff16bfff700 1 -- 172.21.15.31:0/3372024275 >> v1:172.21.15.62:6789/0 conn(0x7ff16c15dea0 legacy=0x7ff16c0b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:09.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.262+0000 7ff16bfff700 1 -- 172.21.15.31:0/3372024275 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff16c0b3b10 con 0x7ff16c159570 2023-10-06T02:27:09.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.262+0000 7ff16b7fe700 1 --2- 172.21.15.31:0/3372024275 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff16c159940 0x7ff16c06d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:27:09.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.262+0000 7ff16bfff700 1 --2- 172.21.15.31:0/3372024275 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff16c159570 0x7ff16c0b23e0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7ff15c0099b0 tx=0x7ff15c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=955b8dfc06216397 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:09.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.262+0000 7ff16affd700 1 -- 172.21.15.31:0/3372024275 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff15c00b860 con 0x7ff16c159570 2023-10-06T02:27:09.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.263+0000 7ff16affd700 1 -- 172.21.15.31:0/3372024275 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff15c00b9c0 con 0x7ff16c159570 2023-10-06T02:27:09.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.263+0000 7ff16affd700 1 -- 172.21.15.31:0/3372024275 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff15c00bd60 con 0x7ff16c159570 2023-10-06T02:27:09.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.263+0000 7ff17159d700 1 -- 172.21.15.31:0/3372024275 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff16c159570 msgr2=0x7ff16c0b23e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:09.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.263+0000 7ff17159d700 1 --2- 172.21.15.31:0/3372024275 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff16c159570 0x7ff16c0b23e0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7ff15c0099b0 tx=0x7ff15c009250 comp rx=0 tx=0).stop 2023-10-06T02:27:09.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.263+0000 7ff17159d700 1 -- 172.21.15.31:0/3372024275 shutdown_connections 2023-10-06T02:27:09.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.263+0000 7ff17159d700 1 --2- 172.21.15.31:0/3372024275 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff16c159570 0x7ff16c0b23e0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:09.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.263+0000 7ff17159d700 1 --2- 172.21.15.31:0/3372024275 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff16c159940 0x7ff16c06d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:09.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.263+0000 7ff17159d700 1 -- 172.21.15.31:0/3372024275 >> 172.21.15.31:0/3372024275 conn(0x7ff16c0703a0 msgr2=0x7ff16c0707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:09.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.263+0000 7ff17159d700 1 -- 172.21.15.31:0/3372024275 shutdown_connections 2023-10-06T02:27:09.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.263+0000 7ff17159d700 1 -- 172.21.15.31:0/3372024275 wait complete. 2023-10-06T02:27:09.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.265+0000 7ff17159d700 1 Processor -- start 2023-10-06T02:27:09.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.265+0000 7ff17159d700 1 -- start start 2023-10-06T02:27:09.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.266+0000 7ff17159d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff16c159570 0x7ff16c0ce990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:09.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.266+0000 7ff17159d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff16c159940 0x7ff16c157c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:09.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.266+0000 7ff17159d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff16c15dea0 0x7ff16c0d0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:09.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.266+0000 7ff17159d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff16c0c7f90 con 0x7ff16c159570 2023-10-06T02:27:09.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.266+0000 7ff17159d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff16c158190 con 0x7ff16c15dea0 2023-10-06T02:27:09.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.266+0000 7ff17159d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff16c0d0ad0 con 0x7ff16c159940 2023-10-06T02:27:09.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.266+0000 7ff16bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff16c159570 0x7ff16c0ce990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:09.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.266+0000 7ff16bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff16c159570 0x7ff16c0ce990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50478/0 (socket says 172.21.15.31:50478) 2023-10-06T02:27:09.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.266+0000 7ff16bfff700 1 -- 172.21.15.31:0/3208410076 learned_addr learned my addr 172.21.15.31:0/3208410076 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:09.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.266+0000 7ff16b7fe700 1 --2- 172.21.15.31:0/3208410076 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff16c159940 0x7ff16c157c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:09.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.266+0000 7ff170d9c700 1 --2- 172.21.15.31:0/3208410076 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff16c15dea0 0x7ff16c0d0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:09.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.266+0000 7ff16bfff700 1 -- 172.21.15.31:0/3208410076 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff16c159940 msgr2=0x7ff16c157c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:09.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.266+0000 7ff16bfff700 1 --2- 172.21.15.31:0/3208410076 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff16c159940 0x7ff16c157c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:09.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.266+0000 7ff16bfff700 1 -- 172.21.15.31:0/3208410076 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff16c15dea0 msgr2=0x7ff16c0d0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:09.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.266+0000 7ff16bfff700 1 --2- 172.21.15.31:0/3208410076 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff16c15dea0 0x7ff16c0d0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:09.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.266+0000 7ff16bfff700 1 -- 172.21.15.31:0/3208410076 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff15c0094f0 con 0x7ff16c159570 2023-10-06T02:27:09.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.267+0000 7ff16bfff700 1 --2- 172.21.15.31:0/3208410076 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff16c159570 0x7ff16c0ce990 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7ff15c009290 tx=0x7ff15c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:09.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.267+0000 7ff153fff700 1 -- 172.21.15.31:0/3208410076 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff15c026480 con 0x7ff16c159570 2023-10-06T02:27:09.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.267+0000 7ff153fff700 1 -- 172.21.15.31:0/3208410076 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff15c01f070 con 0x7ff16c159570 2023-10-06T02:27:09.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.267+0000 7ff153fff700 1 -- 172.21.15.31:0/3208410076 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff15c026830 con 0x7ff16c159570 2023-10-06T02:27:09.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.267+0000 7ff17159d700 1 -- 172.21.15.31:0/3208410076 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff16c0d0db0 con 0x7ff16c159570 2023-10-06T02:27:09.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.267+0000 7ff17159d700 1 -- 172.21.15.31:0/3208410076 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff16c0d1370 con 0x7ff16c159570 2023-10-06T02:27:09.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.268+0000 7ff17159d700 1 -- 172.21.15.31:0/3208410076 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff16c04c3c0 con 0x7ff16c159570 2023-10-06T02:27:09.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.269+0000 7ff153fff700 1 -- 172.21.15.31:0/3208410076 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7ff15c026ba0 con 0x7ff16c159570 2023-10-06T02:27:09.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.270+0000 7ff153fff700 1 --2- 172.21.15.31:0/3208410076 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7ff158067290 0x7ff158069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:09.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.271+0000 7ff153fff700 1 -- 172.21.15.31:0/3208410076 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6203+0+0 (secure 0 0 0) 0x7ff15c021070 con 0x7ff16c159570 2023-10-06T02:27:09.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.271+0000 7ff16b7fe700 1 --2- 172.21.15.31:0/3208410076 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7ff158067290 0x7ff158069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:09.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.271+0000 7ff16b7fe700 1 --2- 172.21.15.31:0/3208410076 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7ff158067290 0x7ff158069740 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff16c0b0fa0 tx=0x7ff160008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:09.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.275+0000 7ff153fff700 1 -- 172.21.15.31:0/3208410076 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff15c053710 con 0x7ff16c159570 2023-10-06T02:27:09.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.520+0000 7ff17159d700 1 -- 172.21.15.31:0/3208410076 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7ff16c0b3df0 con 0x7ff16c159570 2023-10-06T02:27:09.521 INFO:tasks.ceph.mon.a.smithi031.stderr:2023-10-06T02:27:09.520+0000 7f6de01f1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T02:27:09.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.525+0000 7ff153fff700 1 -- 172.21.15.31:0/3208410076 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v39) v1 ==== 161+0+0 (secure 0 0 0) 0x7ff15c02b090 con 0x7ff16c159570 2023-10-06T02:27:09.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.529+0000 7ff17159d700 1 -- 172.21.15.31:0/3208410076 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7ff158067290 msgr2=0x7ff158069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:09.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.529+0000 7ff17159d700 1 --2- 172.21.15.31:0/3208410076 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7ff158067290 0x7ff158069740 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff16c0b0fa0 tx=0x7ff160008040 comp rx=0 tx=0).stop 2023-10-06T02:27:09.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.529+0000 7ff17159d700 1 -- 172.21.15.31:0/3208410076 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff16c159570 msgr2=0x7ff16c0ce990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:09.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.529+0000 7ff17159d700 1 --2- 172.21.15.31:0/3208410076 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff16c159570 0x7ff16c0ce990 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7ff15c009290 tx=0x7ff15c017000 comp rx=0 tx=0).stop 2023-10-06T02:27:09.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.529+0000 7ff17159d700 1 -- 172.21.15.31:0/3208410076 shutdown_connections 2023-10-06T02:27:09.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.529+0000 7ff17159d700 1 --2- 172.21.15.31:0/3208410076 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff16c15dea0 0x7ff16c0d0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:09.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.529+0000 7ff17159d700 1 --2- 172.21.15.31:0/3208410076 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff16c159570 0x7ff16c0ce990 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:09.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.529+0000 7ff17159d700 1 --2- 172.21.15.31:0/3208410076 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7ff158067290 0x7ff158069740 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:09.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.529+0000 7ff17159d700 1 --2- 172.21.15.31:0/3208410076 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff16c159940 0x7ff16c157c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:09.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.529+0000 7ff17159d700 1 -- 172.21.15.31:0/3208410076 >> 172.21.15.31:0/3208410076 conn(0x7ff16c0703a0 msgr2=0x7ff16c15d180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:09.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.529+0000 7ff17159d700 1 -- 172.21.15.31:0/3208410076 shutdown_connections 2023-10-06T02:27:09.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.529+0000 7ff17159d700 1 -- 172.21.15.31:0/3208410076 wait complete. 2023-10-06T02:27:09.532 INFO:teuthology.orchestra.run.smithi031.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T02:27:09.556 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T02:27:09.601 INFO:tasks.ceph.mds.c.smithi031.stdout:starting mds.c at 2023-10-06T02:27:09.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.746+0000 7f99add9d700 1 Processor -- start 2023-10-06T02:27:09.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.746+0000 7f99add9d700 1 -- start start 2023-10-06T02:27:09.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.747+0000 7f99add9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f99a8140a70 0x7f99a8069b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:09.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.747+0000 7f99add9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f99a80c7c60 0x7f99a806a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:09.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.747+0000 7f99add9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99a815b640 con 0x7f99a806de50 2023-10-06T02:27:09.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.748+0000 7f99add9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99a80c6800 con 0x7f99a80c7c60 2023-10-06T02:27:09.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.748+0000 7f99add9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99a815b920 con 0x7f99a8140a70 2023-10-06T02:27:09.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.748+0000 7f99acd9b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f99a8140a70 0x7f99a8069b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:09.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.748+0000 7f99a7fff700 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f99a806de50 0x7f99a8157a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:56174/0 (socket says 172.21.15.31:56174) 2023-10-06T02:27:09.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.748+0000 7f99acd9b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f99a8140a70 0x7f99a8069b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:47788/0 (socket says 172.21.15.31:47788) 2023-10-06T02:27:09.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.748+0000 7f99a7fff700 1 -- 172.21.15.31:0/1970424171 learned_addr learned my addr 172.21.15.31:0/1970424171 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:27:09.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.748+0000 7f99ad59c700 1 --2- 172.21.15.31:0/1970424171 >> v2:172.21.15.62:3300/0 conn(0x7f99a80c7c60 0x7f99a806a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:09.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.749+0000 7f99acd9b700 1 -- 172.21.15.31:0/1970424171 >> v2:172.21.15.62:3300/0 conn(0x7f99a80c7c60 msgr2=0x7f99a806a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:09.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.749+0000 7f99acd9b700 1 --2- 172.21.15.31:0/1970424171 >> v2:172.21.15.62:3300/0 conn(0x7f99a80c7c60 0x7f99a806a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:09.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.749+0000 7f99acd9b700 1 -- 172.21.15.31:0/1970424171 >> v1:172.21.15.62:6789/0 conn(0x7f99a806de50 legacy=0x7f99a8157a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:09.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.749+0000 7f99acd9b700 1 -- 172.21.15.31:0/1970424171 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99a815bc00 con 0x7f99a8140a70 2023-10-06T02:27:09.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.749+0000 7f99acd9b700 1 --2- 172.21.15.31:0/1970424171 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f99a8140a70 0x7f99a8069b00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f999c0098e0 tx=0x7f999c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6c90270b25e56f1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:09.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.749+0000 7f99a77fe700 1 -- 172.21.15.31:0/1970424171 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999c00b640 con 0x7f99a8140a70 2023-10-06T02:27:09.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.749+0000 7f99a77fe700 1 -- 172.21.15.31:0/1970424171 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f999c00b7a0 con 0x7f99a8140a70 2023-10-06T02:27:09.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.749+0000 7f99a77fe700 1 -- 172.21.15.31:0/1970424171 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999c00bb40 con 0x7f99a8140a70 2023-10-06T02:27:09.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.750+0000 7f99add9d700 1 -- 172.21.15.31:0/1970424171 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f99a8140a70 msgr2=0x7f99a8069b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:09.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.750+0000 7f99add9d700 1 --2- 172.21.15.31:0/1970424171 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f99a8140a70 0x7f99a8069b00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f999c0098e0 tx=0x7f999c0091a0 comp rx=0 tx=0).stop 2023-10-06T02:27:09.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.750+0000 7f99add9d700 1 -- 172.21.15.31:0/1970424171 shutdown_connections 2023-10-06T02:27:09.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.750+0000 7f99add9d700 1 --2- 172.21.15.31:0/1970424171 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f99a8140a70 0x7f99a8069b00 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:09.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.750+0000 7f99add9d700 1 --2- 172.21.15.31:0/1970424171 >> v2:172.21.15.62:3300/0 conn(0x7f99a80c7c60 0x7f99a806a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:09.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.750+0000 7f99add9d700 1 -- 172.21.15.31:0/1970424171 >> 172.21.15.31:0/1970424171 conn(0x7f99a806d110 msgr2=0x7f99a80cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:09.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.750+0000 7f99add9d700 1 -- 172.21.15.31:0/1970424171 shutdown_connections 2023-10-06T02:27:09.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.750+0000 7f99add9d700 1 -- 172.21.15.31:0/1970424171 wait complete. 2023-10-06T02:27:09.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.752+0000 7f99add9d700 1 Processor -- start 2023-10-06T02:27:09.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.752+0000 7f99add9d700 1 -- start start 2023-10-06T02:27:09.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.753+0000 7f99add9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99a806de50 0x7f99a80b1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:09.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.753+0000 7f99add9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99a80c7c60 0x7f99a816c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:09.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.753+0000 7f99add9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f99a8140a70 0x7f99a80af880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:09.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.753+0000 7f99add9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a8169a90 con 0x7f99a806de50 2023-10-06T02:27:09.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.753+0000 7f99add9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a816ce50 con 0x7f99a80c7c60 2023-10-06T02:27:09.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.753+0000 7f99add9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99a80b00c0 con 0x7f99a8140a70 2023-10-06T02:27:09.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.753+0000 7f99acd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99a806de50 0x7f99a80b1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:09.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.753+0000 7f99acd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99a806de50 0x7f99a80b1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50500/0 (socket says 172.21.15.31:50500) 2023-10-06T02:27:09.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.753+0000 7f99acd9b700 1 -- 172.21.15.31:0/1645975477 learned_addr learned my addr 172.21.15.31:0/1645975477 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:09.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.753+0000 7f99ad59c700 1 --2- 172.21.15.31:0/1645975477 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f99a8140a70 0x7f99a80af880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:09.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.753+0000 7f99acd9b700 1 -- 172.21.15.31:0/1645975477 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f99a8140a70 msgr2=0x7f99a80af880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:09.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.753+0000 7f99acd9b700 1 --2- 172.21.15.31:0/1645975477 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f99a8140a70 0x7f99a80af880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:09.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.753+0000 7f99acd9b700 1 -- 172.21.15.31:0/1645975477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99a80c7c60 msgr2=0x7f99a816c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:09.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.753+0000 7f99acd9b700 1 --2- 172.21.15.31:0/1645975477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99a80c7c60 0x7f99a816c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:09.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.753+0000 7f99acd9b700 1 -- 172.21.15.31:0/1645975477 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f999c009420 con 0x7f99a806de50 2023-10-06T02:27:09.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.754+0000 7f99acd9b700 1 --2- 172.21.15.31:0/1645975477 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99a806de50 0x7f99a80b1900 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f999c00a010 tx=0x7f999c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:09.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.754+0000 7f99a4ff9700 1 -- 172.21.15.31:0/1645975477 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999c01be20 con 0x7f99a806de50 2023-10-06T02:27:09.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.754+0000 7f99a4ff9700 1 -- 172.21.15.31:0/1645975477 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f999c011420 con 0x7f99a806de50 2023-10-06T02:27:09.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.754+0000 7f99add9d700 1 -- 172.21.15.31:0/1645975477 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99a80b03a0 con 0x7f99a806de50 2023-10-06T02:27:09.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.754+0000 7f99a4ff9700 1 -- 172.21.15.31:0/1645975477 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999c0117e0 con 0x7f99a806de50 2023-10-06T02:27:09.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.754+0000 7f99add9d700 1 -- 172.21.15.31:0/1645975477 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99a80b0960 con 0x7f99a806de50 2023-10-06T02:27:09.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.755+0000 7f99add9d700 1 -- 172.21.15.31:0/1645975477 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99a804c3c0 con 0x7f99a806de50 2023-10-06T02:27:09.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.756+0000 7f99a4ff9700 1 -- 172.21.15.31:0/1645975477 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f999c011b50 con 0x7f99a806de50 2023-10-06T02:27:09.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.757+0000 7f99a4ff9700 1 --2- 172.21.15.31:0/1645975477 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f998c067290 0x7f998c069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:09.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.757+0000 7f99a4ff9700 1 -- 172.21.15.31:0/1645975477 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6289+0+0 (secure 0 0 0) 0x7f999c0865e0 con 0x7f99a806de50 2023-10-06T02:27:09.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.757+0000 7f99a7fff700 1 --2- 172.21.15.31:0/1645975477 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f998c067290 0x7f998c069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:09.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.758+0000 7f99a7fff700 1 --2- 172.21.15.31:0/1645975477 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f998c067290 0x7f998c069740 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f99a80b1020 tx=0x7f9994007b30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:09.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:09.761+0000 7f99a4ff9700 1 -- 172.21.15.31:0/1645975477 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f999c0531b0 con 0x7f99a806de50 2023-10-06T02:27:10.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:10.007+0000 7f99add9d700 1 -- 172.21.15.31:0/1645975477 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f99a815bee0 con 0x7f99a806de50 2023-10-06T02:27:10.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:10.526+0000 7f99a4ff9700 1 -- 172.21.15.31:0/1645975477 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v40) v1 ==== 104+0+0 (secure 0 0 0) 0x7f999c0567d0 con 0x7f99a806de50 2023-10-06T02:27:10.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:10.529+0000 7f99add9d700 1 -- 172.21.15.31:0/1645975477 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f998c067290 msgr2=0x7f998c069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:10.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:10.529+0000 7f99add9d700 1 --2- 172.21.15.31:0/1645975477 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f998c067290 0x7f998c069740 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f99a80b1020 tx=0x7f9994007b30 comp rx=0 tx=0).stop 2023-10-06T02:27:10.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:10.530+0000 7f99add9d700 1 -- 172.21.15.31:0/1645975477 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99a806de50 msgr2=0x7f99a80b1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:10.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:10.530+0000 7f99add9d700 1 --2- 172.21.15.31:0/1645975477 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99a806de50 0x7f99a80b1900 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f999c00a010 tx=0x7f999c017000 comp rx=0 tx=0).stop 2023-10-06T02:27:10.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:10.530+0000 7f99add9d700 1 -- 172.21.15.31:0/1645975477 shutdown_connections 2023-10-06T02:27:10.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:10.530+0000 7f99add9d700 1 --2- 172.21.15.31:0/1645975477 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f998c067290 0x7f998c069740 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:10.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:10.530+0000 7f99add9d700 1 --2- 172.21.15.31:0/1645975477 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f99a8140a70 0x7f99a80af880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:10.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:10.530+0000 7f99add9d700 1 --2- 172.21.15.31:0/1645975477 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99a806de50 0x7f99a80b1900 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:10.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:10.530+0000 7f99add9d700 1 --2- 172.21.15.31:0/1645975477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99a80c7c60 0x7f99a816c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:10.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:10.530+0000 7f99add9d700 1 -- 172.21.15.31:0/1645975477 >> 172.21.15.31:0/1645975477 conn(0x7f99a806d110 msgr2=0x7f99a815aba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:10.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:10.530+0000 7f99add9d700 1 -- 172.21.15.31:0/1645975477 shutdown_connections 2023-10-06T02:27:10.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:10.530+0000 7f99add9d700 1 -- 172.21.15.31:0/1645975477 wait complete. 2023-10-06T02:27:10.550 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T02:27:10.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:10.780+0000 7f09f979d700 1 Processor -- start 2023-10-06T02:27:10.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:10.780+0000 7f09f979d700 1 -- start start 2023-10-06T02:27:10.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:10.781+0000 7f09f979d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f09f4159fb0 0x7f09f415a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:10.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:10.781+0000 7f09f979d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f09f40cd5a0 0x7f09f40cb440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:10.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:10.781+0000 7f09f979d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09f415d570 con 0x7f09f415a8c0 2023-10-06T02:27:10.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:10.781+0000 7f09f979d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09f415e180 con 0x7f09f40cd5a0 2023-10-06T02:27:10.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:10.781+0000 7f09f979d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09f415d850 con 0x7f09f4159fb0 2023-10-06T02:27:10.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:10.782+0000 7f09f37fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f09f415a8c0 0x7f09f40cce90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:47720/0 (socket says 172.21.15.31:47720) 2023-10-06T02:27:10.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:10.782+0000 7f09f37fe700 1 -- 172.21.15.31:0/2546406287 learned_addr learned my addr 172.21.15.31:0/2546406287 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:27:10.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:10.782+0000 7f09f8f9c700 1 --2- 172.21.15.31:0/2546406287 >> v2:172.21.15.31:3300/0 conn(0x7f09f40cd5a0 0x7f09f40cb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:10.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:10.782+0000 7f09f3fff700 1 --2- 172.21.15.31:0/2546406287 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f09f4159fb0 0x7f09f415a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:10.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:10.782+0000 7f09f2ffd700 1 -- 172.21.15.31:0/2546406287 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1577688656 0 0) 0x7f09f415d570 con 0x7f09f415a8c0 2023-10-06T02:27:10.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:10.782+0000 7f09f2ffd700 1 -- 172.21.15.31:0/2546406287 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09dc003570 con 0x7f09f415a8c0 2023-10-06T02:27:10.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:10.782+0000 7f09f8f9c700 1 -- 172.21.15.31:0/2546406287 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f09f4159fb0 msgr2=0x7f09f415a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:10.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:10.782+0000 7f09f8f9c700 1 --2- 172.21.15.31:0/2546406287 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f09f4159fb0 0x7f09f415a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:10.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:10.782+0000 7f09f8f9c700 1 -- 172.21.15.31:0/2546406287 >> v1:172.21.15.31:6789/0 conn(0x7f09f415a8c0 legacy=0x7f09f40cce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:10.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:10.782+0000 7f09f8f9c700 1 -- 172.21.15.31:0/2546406287 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09f415db30 con 0x7f09f40cd5a0 2023-10-06T02:27:10.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:10.782+0000 7f09f3fff700 1 --2- 172.21.15.31:0/2546406287 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f09f4159fb0 0x7f09f415a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:27:10.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:10.783+0000 7f09f8f9c700 1 --2- 172.21.15.31:0/2546406287 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09f40cd5a0 0x7f09f40cb440 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f09ec009e60 tx=0x7f09ec009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81177d2855d7f585 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:10.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:10.783+0000 7f09f2ffd700 1 -- 172.21.15.31:0/2546406287 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09ec00e070 con 0x7f09f40cd5a0 2023-10-06T02:27:10.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:10.783+0000 7f09f2ffd700 1 -- 172.21.15.31:0/2546406287 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09ec00bab0 con 0x7f09f40cd5a0 2023-10-06T02:27:10.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:10.783+0000 7f09f2ffd700 1 -- 172.21.15.31:0/2546406287 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09ec00be70 con 0x7f09f40cd5a0 2023-10-06T02:27:10.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:10.783+0000 7f09f979d700 1 -- 172.21.15.31:0/2546406287 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09f40cd5a0 msgr2=0x7f09f40cb440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:10.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:10.783+0000 7f09f979d700 1 --2- 172.21.15.31:0/2546406287 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09f40cd5a0 0x7f09f40cb440 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f09ec009e60 tx=0x7f09ec009780 comp rx=0 tx=0).stop 2023-10-06T02:27:10.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:10.783+0000 7f09f979d700 1 -- 172.21.15.31:0/2546406287 shutdown_connections 2023-10-06T02:27:10.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:10.783+0000 7f09f979d700 1 --2- 172.21.15.31:0/2546406287 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09f40cd5a0 0x7f09f40cb440 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:10.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:10.783+0000 7f09f979d700 1 --2- 172.21.15.31:0/2546406287 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f09f4159fb0 0x7f09f415a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:10.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:10.783+0000 7f09f979d700 1 -- 172.21.15.31:0/2546406287 >> 172.21.15.31:0/2546406287 conn(0x7f09f40703a0 msgr2=0x7f09f40707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:10.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:10.784+0000 7f09f979d700 1 -- 172.21.15.31:0/2546406287 shutdown_connections 2023-10-06T02:27:10.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:10.784+0000 7f09f979d700 1 -- 172.21.15.31:0/2546406287 wait complete. 2023-10-06T02:27:10.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:10.785+0000 7f09f979d700 1 Processor -- start 2023-10-06T02:27:10.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:10.785+0000 7f09f979d700 1 -- start start 2023-10-06T02:27:10.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:10.786+0000 7f09f979d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09f40cd5a0 0x7f09f40b0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:10.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:10.786+0000 7f09f979d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f09f4159fb0 0x7f09f414f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:10.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:10.786+0000 7f09f979d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09f415a8c0 0x7f09f414cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:10.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:10.786+0000 7f09f979d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09f40b2080 con 0x7f09f415a8c0 2023-10-06T02:27:10.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:10.786+0000 7f09f979d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09f40c7cf0 con 0x7f09f40cd5a0 2023-10-06T02:27:10.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:10.786+0000 7f09f979d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09f40b1da0 con 0x7f09f4159fb0 2023-10-06T02:27:10.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:10.786+0000 7f09f8f9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09f415a8c0 0x7f09f414cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:10.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:10.786+0000 7f09f8f9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09f415a8c0 0x7f09f414cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50506/0 (socket says 172.21.15.31:50506) 2023-10-06T02:27:10.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:10.786+0000 7f09f8f9c700 1 -- 172.21.15.31:0/3042732235 learned_addr learned my addr 172.21.15.31:0/3042732235 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:10.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:10.787+0000 7f09f37fe700 1 --2- 172.21.15.31:0/3042732235 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f09f4159fb0 0x7f09f414f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:10.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:10.787+0000 7f09f3fff700 1 --2- 172.21.15.31:0/3042732235 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09f40cd5a0 0x7f09f40b0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:10.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:10.787+0000 7f09f8f9c700 1 -- 172.21.15.31:0/3042732235 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f09f4159fb0 msgr2=0x7f09f414f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:10.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:10.787+0000 7f09f8f9c700 1 --2- 172.21.15.31:0/3042732235 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f09f4159fb0 0x7f09f414f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:10.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:10.787+0000 7f09f8f9c700 1 -- 172.21.15.31:0/3042732235 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09f40cd5a0 msgr2=0x7f09f40b0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:10.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:10.787+0000 7f09f8f9c700 1 --2- 172.21.15.31:0/3042732235 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09f40cd5a0 0x7f09f40b0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:10.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:10.787+0000 7f09f8f9c700 1 -- 172.21.15.31:0/3042732235 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09e4009420 con 0x7f09f415a8c0 2023-10-06T02:27:10.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:10.787+0000 7f09f3fff700 1 --2- 172.21.15.31:0/3042732235 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09f40cd5a0 0x7f09f40b0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:27:10.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:10.787+0000 7f09f37fe700 1 --2- 172.21.15.31:0/3042732235 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f09f4159fb0 0x7f09f414f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:27:10.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:10.787+0000 7f09f8f9c700 1 --2- 172.21.15.31:0/3042732235 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09f415a8c0 0x7f09f414cbc0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f09f406d1f0 tx=0x7f09ec019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:10.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:10.787+0000 7f09dbfff700 1 -- 172.21.15.31:0/3042732235 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09ec027470 con 0x7f09f415a8c0 2023-10-06T02:27:10.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:10.787+0000 7f09dbfff700 1 -- 172.21.15.31:0/3042732235 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09ec0275f0 con 0x7f09f415a8c0 2023-10-06T02:27:10.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:10.787+0000 7f09dbfff700 1 -- 172.21.15.31:0/3042732235 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09ec0279b0 con 0x7f09f415a8c0 2023-10-06T02:27:10.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:10.787+0000 7f09f979d700 1 -- 172.21.15.31:0/3042732235 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09ec009a20 con 0x7f09f415a8c0 2023-10-06T02:27:10.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:10.788+0000 7f09f979d700 1 -- 172.21.15.31:0/3042732235 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09f40c7980 con 0x7f09f415a8c0 2023-10-06T02:27:10.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:10.788+0000 7f09f979d700 1 -- 172.21.15.31:0/3042732235 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09f4070f40 con 0x7f09f415a8c0 2023-10-06T02:27:10.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:10.789+0000 7f09dbfff700 1 -- 172.21.15.31:0/3042732235 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f09ec024070 con 0x7f09f415a8c0 2023-10-06T02:27:10.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:10.790+0000 7f09dbfff700 1 --2- 172.21.15.31:0/3042732235 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f09e0067290 0x7f09e0069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:10.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:10.790+0000 7f09dbfff700 1 -- 172.21.15.31:0/3042732235 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6289+0+0 (secure 0 0 0) 0x7f09ec087630 con 0x7f09f415a8c0 2023-10-06T02:27:10.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:10.790+0000 7f09f3fff700 1 --2- 172.21.15.31:0/3042732235 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f09e0067290 0x7f09e0069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:10.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:10.791+0000 7f09f3fff700 1 --2- 172.21.15.31:0/3042732235 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f09e0067290 0x7f09e0069740 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f09e40093f0 tx=0x7f09e4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:10.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:10.795+0000 7f09dbfff700 1 -- 172.21.15.31:0/3042732235 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f09ec054240 con 0x7f09f415a8c0 2023-10-06T02:27:10.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:10.997+0000 7f09f979d700 1 -- 172.21.15.31:0/3042732235 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f09f404c3c0 con 0x7f09f415a8c0 2023-10-06T02:27:11.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.531+0000 7f09dbfff700 1 -- 172.21.15.31:0/3042732235 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v65) v1 ==== 181+0+0 (secure 0 0 0) 0x7f09ec057860 con 0x7f09f415a8c0 2023-10-06T02:27:11.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.535+0000 7f09f979d700 1 -- 172.21.15.31:0/3042732235 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f09e0067290 msgr2=0x7f09e0069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:11.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.535+0000 7f09f979d700 1 --2- 172.21.15.31:0/3042732235 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f09e0067290 0x7f09e0069740 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f09e40093f0 tx=0x7f09e4008040 comp rx=0 tx=0).stop 2023-10-06T02:27:11.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.536+0000 7f09f979d700 1 -- 172.21.15.31:0/3042732235 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09f415a8c0 msgr2=0x7f09f414cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:11.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.536+0000 7f09f979d700 1 --2- 172.21.15.31:0/3042732235 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09f415a8c0 0x7f09f414cbc0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f09f406d1f0 tx=0x7f09ec019000 comp rx=0 tx=0).stop 2023-10-06T02:27:11.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.536+0000 7f09f979d700 1 -- 172.21.15.31:0/3042732235 shutdown_connections 2023-10-06T02:27:11.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.536+0000 7f09f979d700 1 --2- 172.21.15.31:0/3042732235 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09f415a8c0 0x7f09f414cbc0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:11.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.536+0000 7f09f979d700 1 --2- 172.21.15.31:0/3042732235 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09f40cd5a0 0x7f09f40b0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:11.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.536+0000 7f09f979d700 1 --2- 172.21.15.31:0/3042732235 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f09e0067290 0x7f09e0069740 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:11.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.536+0000 7f09f979d700 1 --2- 172.21.15.31:0/3042732235 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f09f4159fb0 0x7f09f414f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:11.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.536+0000 7f09f979d700 1 -- 172.21.15.31:0/3042732235 >> 172.21.15.31:0/3042732235 conn(0x7f09f40703a0 msgr2=0x7f09f4157770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:11.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.536+0000 7f09f979d700 1 -- 172.21.15.31:0/3042732235 shutdown_connections 2023-10-06T02:27:11.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.536+0000 7f09f979d700 1 -- 172.21.15.31:0/3042732235 wait complete. 2023-10-06T02:27:11.539 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' removed 2023-10-06T02:27:11.558 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T02:27:11.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.751+0000 7f153ffff700 1 Processor -- start 2023-10-06T02:27:11.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.751+0000 7f153ffff700 1 -- start start 2023-10-06T02:27:11.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.753+0000 7f153ffff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f15400c2a90 0x7f1540156aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:11.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.753+0000 7f153ffff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15401579b0 con 0x7f15400c8200 2023-10-06T02:27:11.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.753+0000 7f153ffff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15400c1390 con 0x7f15400c85d0 2023-10-06T02:27:11.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.753+0000 7f153ffff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15401573f0 con 0x7f15400c2a90 2023-10-06T02:27:11.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.753+0000 7f153effd700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f15400c8200 0x7f15400c1820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:47730/0 (socket says 172.21.15.31:47730) 2023-10-06T02:27:11.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.753+0000 7f153effd700 1 -- 172.21.15.31:0/1473984298 learned_addr learned my addr 172.21.15.31:0/1473984298 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:27:11.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.753+0000 7f153e7fc700 1 --2- 172.21.15.31:0/1473984298 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f15400c2a90 0x7f1540156aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:11.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.753+0000 7f153dffb700 1 -- 172.21.15.31:0/1473984298 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1632146979 0 0) 0x7f15401579b0 con 0x7f15400c8200 2023-10-06T02:27:11.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.753+0000 7f153dffb700 1 -- 172.21.15.31:0/1473984298 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1528003570 con 0x7f15400c8200 2023-10-06T02:27:11.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.754+0000 7f153dffb700 1 -- 172.21.15.31:0/1473984298 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1857725403 0 0) 0x7f1528003570 con 0x7f15400c8200 2023-10-06T02:27:11.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.754+0000 7f153dffb700 1 -- 172.21.15.31:0/1473984298 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f15400c2a90 msgr2=0x7f1540156aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:11.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.754+0000 7f153dffb700 1 --2- 172.21.15.31:0/1473984298 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f15400c2a90 0x7f1540156aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:11.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.754+0000 7f153dffb700 1 -- 172.21.15.31:0/1473984298 >> v1:172.21.15.62:6789/0 conn(0x7f15400c85d0 legacy=0x7f15400cc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:11.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.754+0000 7f153dffb700 1 -- 172.21.15.31:0/1473984298 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15401576d0 con 0x7f15400c8200 2023-10-06T02:27:11.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.754+0000 7f153dffb700 1 -- 172.21.15.31:0/1473984298 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7f1530001470 con 0x7f15400c8200 2023-10-06T02:27:11.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.754+0000 7f153dffb700 1 -- 172.21.15.31:0/1473984298 >> v1:172.21.15.31:6789/0 conn(0x7f15400c8200 legacy=0x7f15400c1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:11.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.754+0000 7f153dffb700 1 --2- 172.21.15.31:0/1473984298 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f15400c2a90 0x7f1528003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:11.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.754+0000 7f153dffb700 1 --2- 172.21.15.31:0/1473984298 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1528003fd0 0x7f1528004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:11.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.754+0000 7f153dffb700 1 --2- 172.21.15.31:0/1473984298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1528004d50 0x7f15280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:11.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.754+0000 7f153dffb700 1 -- 172.21.15.31:0/1473984298 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15401573f0 con 0x7f1528003fd0 2023-10-06T02:27:11.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.754+0000 7f153dffb700 1 -- 172.21.15.31:0/1473984298 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f152800bba0 con 0x7f1528004d50 2023-10-06T02:27:11.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.754+0000 7f153dffb700 1 -- 172.21.15.31:0/1473984298 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f152800bd60 con 0x7f15400c2a90 2023-10-06T02:27:11.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.754+0000 7f153e7fc700 1 --2- 172.21.15.31:0/1473984298 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1528003fd0 0x7f1528004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:11.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.754+0000 7f153effd700 1 --2- 172.21.15.31:0/1473984298 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f15400c2a90 0x7f1528003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:11.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.754+0000 7f153f7fe700 1 --2- 172.21.15.31:0/1473984298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1528004d50 0x7f15280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:11.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.754+0000 7f153e7fc700 1 -- 172.21.15.31:0/1473984298 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f15400c2a90 msgr2=0x7f1528003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:11.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.754+0000 7f153e7fc700 1 --2- 172.21.15.31:0/1473984298 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f15400c2a90 0x7f1528003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:11.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.754+0000 7f153e7fc700 1 -- 172.21.15.31:0/1473984298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1528004d50 msgr2=0x7f15280051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:11.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.754+0000 7f153e7fc700 1 --2- 172.21.15.31:0/1473984298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1528004d50 0x7f15280051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:11.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.754+0000 7f153e7fc700 1 -- 172.21.15.31:0/1473984298 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1528003570 con 0x7f1528003fd0 2023-10-06T02:27:11.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.754+0000 7f153f7fe700 1 --2- 172.21.15.31:0/1473984298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1528004d50 0x7f15280051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:27:11.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.755+0000 7f153e7fc700 1 --2- 172.21.15.31:0/1473984298 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1528003fd0 0x7f1528004810 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f153400beb0 tx=0x7f153400b430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=955750df59a223d6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:11.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.755+0000 7f153dffb700 1 -- 172.21.15.31:0/1473984298 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1534010c00 con 0x7f1528003fd0 2023-10-06T02:27:11.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.755+0000 7f153dffb700 1 -- 172.21.15.31:0/1473984298 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1534010d60 con 0x7f1528003fd0 2023-10-06T02:27:11.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.755+0000 7f153dffb700 1 -- 172.21.15.31:0/1473984298 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1534011680 con 0x7f1528003fd0 2023-10-06T02:27:11.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.755+0000 7f153ffff700 1 -- 172.21.15.31:0/1473984298 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1528003fd0 msgr2=0x7f1528004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:11.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.755+0000 7f153ffff700 1 --2- 172.21.15.31:0/1473984298 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1528003fd0 0x7f1528004810 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f153400beb0 tx=0x7f153400b430 comp rx=0 tx=0).stop 2023-10-06T02:27:11.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.755+0000 7f153f7fe700 1 -- 172.21.15.31:0/1473984298 reap_dead start 2023-10-06T02:27:11.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.756+0000 7f153ffff700 1 -- 172.21.15.31:0/1473984298 shutdown_connections 2023-10-06T02:27:11.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.756+0000 7f153ffff700 1 -- 172.21.15.31:0/1473984298 >> 172.21.15.31:0/1473984298 conn(0x7f154006c6a0 msgr2=0x7f154006caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:11.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.756+0000 7f153ffff700 1 -- 172.21.15.31:0/1473984298 shutdown_connections 2023-10-06T02:27:11.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.756+0000 7f153ffff700 1 -- 172.21.15.31:0/1473984298 wait complete. 2023-10-06T02:27:11.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.757+0000 7f153ffff700 1 Processor -- start 2023-10-06T02:27:11.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.757+0000 7f153ffff700 1 -- start start 2023-10-06T02:27:11.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.758+0000 7f153ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1528003fd0 0x7f154014d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:11.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.758+0000 7f153ffff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f15400b02c0 0x7f15400af890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:11.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.758+0000 7f153ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1540143a50 0x7f15400c8d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:11.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.758+0000 7f153ffff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15400af400 con 0x7f1528003fd0 2023-10-06T02:27:11.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.758+0000 7f153ffff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f154014e680 con 0x7f1540143a50 2023-10-06T02:27:11.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.758+0000 7f153ffff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15400b0730 con 0x7f15400b02c0 2023-10-06T02:27:11.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.758+0000 7f153effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1528003fd0 0x7f154014d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:11.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.758+0000 7f153effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1528003fd0 0x7f154014d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50524/0 (socket says 172.21.15.31:50524) 2023-10-06T02:27:11.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.758+0000 7f153effd700 1 -- 172.21.15.31:0/2658095476 learned_addr learned my addr 172.21.15.31:0/2658095476 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:11.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.759+0000 7f153effd700 1 -- 172.21.15.31:0/2658095476 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f15400b02c0 msgr2=0x7f15400af890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:11.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.759+0000 7f153effd700 1 --2- 172.21.15.31:0/2658095476 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f15400b02c0 0x7f15400af890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:11.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.759+0000 7f153effd700 1 -- 172.21.15.31:0/2658095476 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1540143a50 msgr2=0x7f15400c8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:11.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.759+0000 7f153effd700 1 --2- 172.21.15.31:0/2658095476 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1540143a50 0x7f15400c8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:11.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.759+0000 7f153effd700 1 -- 172.21.15.31:0/2658095476 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15400afdd0 con 0x7f1528003fd0 2023-10-06T02:27:11.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.759+0000 7f153effd700 1 --2- 172.21.15.31:0/2658095476 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1528003fd0 0x7f154014d9c0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f153000ebc0 tx=0x7f1540069d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:11.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.759+0000 7f1526ffd700 1 -- 172.21.15.31:0/2658095476 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f153000c760 con 0x7f1528003fd0 2023-10-06T02:27:11.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.759+0000 7f1526ffd700 1 -- 172.21.15.31:0/2658095476 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f153000c8c0 con 0x7f1528003fd0 2023-10-06T02:27:11.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.759+0000 7f153ffff700 1 -- 172.21.15.31:0/2658095476 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f154014e0c0 con 0x7f1528003fd0 2023-10-06T02:27:11.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.759+0000 7f1526ffd700 1 -- 172.21.15.31:0/2658095476 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f153000cc80 con 0x7f1528003fd0 2023-10-06T02:27:11.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.759+0000 7f153ffff700 1 -- 172.21.15.31:0/2658095476 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15400af090 con 0x7f1528003fd0 2023-10-06T02:27:11.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.760+0000 7f153ffff700 1 -- 172.21.15.31:0/2658095476 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15400aeef0 con 0x7f1528003fd0 2023-10-06T02:27:11.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.761+0000 7f1526ffd700 1 -- 172.21.15.31:0/2658095476 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f153001d460 con 0x7f1528003fd0 2023-10-06T02:27:11.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.762+0000 7f1526ffd700 1 --2- 172.21.15.31:0/2658095476 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f152c067290 0x7f152c069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:11.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.762+0000 7f1526ffd700 1 -- 172.21.15.31:0/2658095476 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5822+0+0 (secure 0 0 0) 0x7f1530084a20 con 0x7f1528003fd0 2023-10-06T02:27:11.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.762+0000 7f153e7fc700 1 --2- 172.21.15.31:0/2658095476 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f152c067290 0x7f152c069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:11.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.763+0000 7f153e7fc700 1 --2- 172.21.15.31:0/2658095476 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f152c067290 0x7f152c069740 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f1540159410 tx=0x7f153401c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:11.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.766+0000 7f1526ffd700 1 -- 172.21.15.31:0/2658095476 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f15400aeef0 con 0x7f1528003fd0 2023-10-06T02:27:11.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:11.970+0000 7f153ffff700 1 -- 172.21.15.31:0/2658095476 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f15400aeef0 con 0x7f1528003fd0 2023-10-06T02:27:12.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.533+0000 7f1526ffd700 1 -- 172.21.15.31:0/2658095476 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v66) v1 ==== 169+0+0 (secure 0 0 0) 0x7f15400aeef0 con 0x7f1528003fd0 2023-10-06T02:27:12.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.537+0000 7f153ffff700 1 -- 172.21.15.31:0/2658095476 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f152c067290 msgr2=0x7f152c069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:12.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.537+0000 7f153ffff700 1 --2- 172.21.15.31:0/2658095476 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f152c067290 0x7f152c069740 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f1540159410 tx=0x7f153401c000 comp rx=0 tx=0).stop 2023-10-06T02:27:12.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.537+0000 7f153ffff700 1 -- 172.21.15.31:0/2658095476 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1528003fd0 msgr2=0x7f154014d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:12.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.537+0000 7f153ffff700 1 --2- 172.21.15.31:0/2658095476 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1528003fd0 0x7f154014d9c0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f153000ebc0 tx=0x7f1540069d50 comp rx=0 tx=0).stop 2023-10-06T02:27:12.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.537+0000 7f153ffff700 1 -- 172.21.15.31:0/2658095476 shutdown_connections 2023-10-06T02:27:12.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.537+0000 7f153ffff700 1 --2- 172.21.15.31:0/2658095476 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1540143a50 0x7f15400c8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:12.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.537+0000 7f153ffff700 1 --2- 172.21.15.31:0/2658095476 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1528003fd0 0x7f154014d9c0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:12.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.537+0000 7f153ffff700 1 --2- 172.21.15.31:0/2658095476 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f152c067290 0x7f152c069740 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:12.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.537+0000 7f153ffff700 1 --2- 172.21.15.31:0/2658095476 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f15400b02c0 0x7f15400af890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:12.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.537+0000 7f153ffff700 1 -- 172.21.15.31:0/2658095476 >> 172.21.15.31:0/2658095476 conn(0x7f154006c6a0 msgr2=0x7f15400caac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:12.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.537+0000 7f153ffff700 1 -- 172.21.15.31:0/2658095476 shutdown_connections 2023-10-06T02:27:12.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.537+0000 7f153ffff700 1 -- 172.21.15.31:0/2658095476 wait complete. 2023-10-06T02:27:12.541 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' removed 2023-10-06T02:27:12.560 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T02:27:12.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.752+0000 7f16ddd9d700 1 Processor -- start 2023-10-06T02:27:12.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.752+0000 7f16ddd9d700 1 -- start start 2023-10-06T02:27:12.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.753+0000 7f16ddd9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f16d80c7fb0 0x7f16d806abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:12.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.753+0000 7f16ddd9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16d806b290 con 0x7f16d813f780 2023-10-06T02:27:12.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.753+0000 7f16ddd9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16d806b570 con 0x7f16d80c7be0 2023-10-06T02:27:12.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.753+0000 7f16ddd9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16d806bf10 con 0x7f16d80c7fb0 2023-10-06T02:27:12.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.753+0000 7f16dd59c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f16d813f780 0x7f16d80c9eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:47742/0 (socket says 172.21.15.31:47742) 2023-10-06T02:27:12.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.753+0000 7f16dd59c700 1 -- 172.21.15.31:0/903422449 learned_addr learned my addr 172.21.15.31:0/903422449 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:27:12.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.754+0000 7f16d7fff700 1 --2- 172.21.15.31:0/903422449 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f16d80c7fb0 0x7f16d806abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:12.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.754+0000 7f16d77fe700 1 -- 172.21.15.31:0/903422449 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4192341295 0 0) 0x7f16d806b290 con 0x7f16d813f780 2023-10-06T02:27:12.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.754+0000 7f16d77fe700 1 -- 172.21.15.31:0/903422449 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16c8003570 con 0x7f16d813f780 2023-10-06T02:27:12.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.754+0000 7f16d77fe700 1 -- 172.21.15.31:0/903422449 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3371320038 0 0) 0x7f16d806b570 con 0x7f16d80c7be0 2023-10-06T02:27:12.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.754+0000 7f16d77fe700 1 -- 172.21.15.31:0/903422449 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16d806b290 con 0x7f16d80c7be0 2023-10-06T02:27:12.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.754+0000 7f16d77fe700 1 -- 172.21.15.31:0/903422449 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2324107541 0 0) 0x7f16c8003570 con 0x7f16d813f780 2023-10-06T02:27:12.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.754+0000 7f16d77fe700 1 -- 172.21.15.31:0/903422449 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f16d80c7fb0 msgr2=0x7f16d806abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:12.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.754+0000 7f16d77fe700 1 --2- 172.21.15.31:0/903422449 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f16d80c7fb0 0x7f16d806abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:12.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.754+0000 7f16d77fe700 1 -- 172.21.15.31:0/903422449 >> v1:172.21.15.62:6789/0 conn(0x7f16d80c7be0 legacy=0x7f16d815a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:12.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.754+0000 7f16d77fe700 1 -- 172.21.15.31:0/903422449 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16d806b850 con 0x7f16d813f780 2023-10-06T02:27:12.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.754+0000 7f16d77fe700 1 -- 172.21.15.31:0/903422449 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7f16d00017f0 con 0x7f16d813f780 2023-10-06T02:27:12.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.754+0000 7f16d77fe700 1 -- 172.21.15.31:0/903422449 >> v1:172.21.15.31:6789/0 conn(0x7f16d813f780 legacy=0x7f16d80c9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:12.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.754+0000 7f16d77fe700 1 --2- 172.21.15.31:0/903422449 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f16d80c7fb0 0x7f16c8003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:12.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.754+0000 7f16d77fe700 1 --2- 172.21.15.31:0/903422449 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16c80041d0 0x7f16c8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:12.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.754+0000 7f16d77fe700 1 --2- 172.21.15.31:0/903422449 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16c8004d50 0x7f16c80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:12.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.754+0000 7f16d77fe700 1 -- 172.21.15.31:0/903422449 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16d806bf10 con 0x7f16c8004d50 2023-10-06T02:27:12.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.754+0000 7f16d77fe700 1 -- 172.21.15.31:0/903422449 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c800bbf0 con 0x7f16c80041d0 2023-10-06T02:27:12.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.754+0000 7f16d77fe700 1 -- 172.21.15.31:0/903422449 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16c800bdb0 con 0x7f16d80c7fb0 2023-10-06T02:27:12.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.755+0000 7f16dd59c700 1 --2- 172.21.15.31:0/903422449 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16c8004d50 0x7f16c80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:12.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.755+0000 7f16dcd9b700 1 --2- 172.21.15.31:0/903422449 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f16d80c7fb0 0x7f16c8003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:12.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.755+0000 7f16d7fff700 1 --2- 172.21.15.31:0/903422449 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16c80041d0 0x7f16c8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:12.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.755+0000 7f16dd59c700 1 -- 172.21.15.31:0/903422449 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f16d80c7fb0 msgr2=0x7f16c8003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:12.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.755+0000 7f16dd59c700 1 --2- 172.21.15.31:0/903422449 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f16d80c7fb0 0x7f16c8003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:12.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.755+0000 7f16dd59c700 1 -- 172.21.15.31:0/903422449 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16c80041d0 msgr2=0x7f16c8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:12.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.755+0000 7f16dd59c700 1 --2- 172.21.15.31:0/903422449 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16c80041d0 0x7f16c8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:12.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.755+0000 7f16dd59c700 1 -- 172.21.15.31:0/903422449 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16c8003570 con 0x7f16c8004d50 2023-10-06T02:27:12.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.755+0000 7f16d7fff700 1 --2- 172.21.15.31:0/903422449 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16c80041d0 0x7f16c8004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:27:12.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.755+0000 7f16dd59c700 1 --2- 172.21.15.31:0/903422449 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16c8004d50 0x7f16c80051c0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f16d000a800 tx=0x7f16d806be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=203805287ad599e8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:12.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.756+0000 7f16d77fe700 1 -- 172.21.15.31:0/903422449 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d00071a0 con 0x7f16c8004d50 2023-10-06T02:27:12.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.756+0000 7f16d77fe700 1 -- 172.21.15.31:0/903422449 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16d0007300 con 0x7f16c8004d50 2023-10-06T02:27:12.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.756+0000 7f16d77fe700 1 -- 172.21.15.31:0/903422449 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d00076c0 con 0x7f16c8004d50 2023-10-06T02:27:12.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.756+0000 7f16ddd9d700 1 -- 172.21.15.31:0/903422449 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16c8004d50 msgr2=0x7f16c80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:12.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.756+0000 7f16ddd9d700 1 --2- 172.21.15.31:0/903422449 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16c8004d50 0x7f16c80051c0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f16d000a800 tx=0x7f16d806be10 comp rx=0 tx=0).stop 2023-10-06T02:27:12.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.756+0000 7f16dd59c700 1 -- 172.21.15.31:0/903422449 reap_dead start 2023-10-06T02:27:12.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.756+0000 7f16ddd9d700 1 -- 172.21.15.31:0/903422449 shutdown_connections 2023-10-06T02:27:12.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.756+0000 7f16ddd9d700 1 -- 172.21.15.31:0/903422449 >> 172.21.15.31:0/903422449 conn(0x7f16d806d110 msgr2=0x7f16d80cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:12.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.756+0000 7f16ddd9d700 1 -- 172.21.15.31:0/903422449 shutdown_connections 2023-10-06T02:27:12.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.756+0000 7f16ddd9d700 1 -- 172.21.15.31:0/903422449 wait complete. 2023-10-06T02:27:12.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.758+0000 7f16ddd9d700 1 Processor -- start 2023-10-06T02:27:12.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.758+0000 7f16ddd9d700 1 -- start start 2023-10-06T02:27:12.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.759+0000 7f16ddd9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f16c8004d50 0x7f16d80c3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:12.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.759+0000 7f16ddd9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16d8155120 0x7f16d80b17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:12.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.759+0000 7f16ddd9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16d80b1d30 0x7f16d80c9db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:12.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.759+0000 7f16ddd9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16d80c3860 con 0x7f16d80b1d30 2023-10-06T02:27:12.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.759+0000 7f16ddd9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16d80c3280 con 0x7f16d8155120 2023-10-06T02:27:12.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.759+0000 7f16ddd9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16d80c4720 con 0x7f16c8004d50 2023-10-06T02:27:12.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.759+0000 7f16dd59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16d80b1d30 0x7f16d80c9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:12.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.759+0000 7f16dd59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16d80b1d30 0x7f16d80c9db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50546/0 (socket says 172.21.15.31:50546) 2023-10-06T02:27:12.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.759+0000 7f16dd59c700 1 -- 172.21.15.31:0/1931158292 learned_addr learned my addr 172.21.15.31:0/1931158292 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:12.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.759+0000 7f16d7fff700 1 --2- 172.21.15.31:0/1931158292 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16d8155120 0x7f16d80b17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:12.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.759+0000 7f16dcd9b700 1 --2- 172.21.15.31:0/1931158292 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f16c8004d50 0x7f16d80c3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:12.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.759+0000 7f16dd59c700 1 -- 172.21.15.31:0/1931158292 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f16c8004d50 msgr2=0x7f16d80c3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:12.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.759+0000 7f16dd59c700 1 --2- 172.21.15.31:0/1931158292 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f16c8004d50 0x7f16d80c3cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:12.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.759+0000 7f16dd59c700 1 -- 172.21.15.31:0/1931158292 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16d8155120 msgr2=0x7f16d80b17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:12.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.759+0000 7f16dd59c700 1 --2- 172.21.15.31:0/1931158292 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16d8155120 0x7f16d80b17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:12.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.759+0000 7f16dd59c700 1 -- 172.21.15.31:0/1931158292 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16d80c34f0 con 0x7f16d80b1d30 2023-10-06T02:27:12.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.759+0000 7f16dd59c700 1 --2- 172.21.15.31:0/1931158292 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16d80b1d30 0x7f16d80c9db0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f16d000adf0 tx=0x7f16d0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:12.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.760+0000 7f16d4ff9700 1 -- 172.21.15.31:0/1931158292 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d0020410 con 0x7f16d80b1d30 2023-10-06T02:27:12.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.760+0000 7f16d4ff9700 1 -- 172.21.15.31:0/1931158292 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16d0020570 con 0x7f16d80b1d30 2023-10-06T02:27:12.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.760+0000 7f16d4ff9700 1 -- 172.21.15.31:0/1931158292 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d0020930 con 0x7f16d80b1d30 2023-10-06T02:27:12.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.760+0000 7f16ddd9d700 1 -- 172.21.15.31:0/1931158292 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16d80c4a00 con 0x7f16d80b1d30 2023-10-06T02:27:12.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.760+0000 7f16ddd9d700 1 -- 172.21.15.31:0/1931158292 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16d8155590 con 0x7f16d80b1d30 2023-10-06T02:27:12.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.761+0000 7f16ddd9d700 1 -- 172.21.15.31:0/1931158292 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16d80648e0 con 0x7f16d80b1d30 2023-10-06T02:27:12.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.762+0000 7f16d4ff9700 1 -- 172.21.15.31:0/1931158292 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f16d0012400 con 0x7f16d80b1d30 2023-10-06T02:27:12.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.763+0000 7f16d4ff9700 1 --2- 172.21.15.31:0/1931158292 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f16bc067290 0x7f16bc069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:12.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.763+0000 7f16d4ff9700 1 -- 172.21.15.31:0/1931158292 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f16d00862c0 con 0x7f16d80b1d30 2023-10-06T02:27:12.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.763+0000 7f16dcd9b700 1 --2- 172.21.15.31:0/1931158292 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f16bc067290 0x7f16bc069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:12.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.763+0000 7f16dcd9b700 1 --2- 172.21.15.31:0/1931158292 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f16bc067290 0x7f16bc069740 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f16cc003c80 tx=0x7f16cc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:12.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.767+0000 7f16d4ff9700 1 -- 172.21.15.31:0/1931158292 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f16d80648e0 con 0x7f16d80b1d30 2023-10-06T02:27:12.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.968+0000 7f16ddd9d700 1 -- 172.21.15.31:0/1931158292 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f16d80648e0 con 0x7f16d80b1d30 2023-10-06T02:27:12.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.972+0000 7f16d4ff9700 1 -- 172.21.15.31:0/1931158292 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7f16d80648e0 con 0x7f16d80b1d30 2023-10-06T02:27:12.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.975+0000 7f16ddd9d700 1 -- 172.21.15.31:0/1931158292 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f16bc067290 msgr2=0x7f16bc069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:12.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.975+0000 7f16ddd9d700 1 --2- 172.21.15.31:0/1931158292 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f16bc067290 0x7f16bc069740 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f16cc003c80 tx=0x7f16cc009040 comp rx=0 tx=0).stop 2023-10-06T02:27:12.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.975+0000 7f16ddd9d700 1 -- 172.21.15.31:0/1931158292 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16d80b1d30 msgr2=0x7f16d80c9db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:12.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.975+0000 7f16ddd9d700 1 --2- 172.21.15.31:0/1931158292 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16d80b1d30 0x7f16d80c9db0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f16d000adf0 tx=0x7f16d0016000 comp rx=0 tx=0).stop 2023-10-06T02:27:12.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.975+0000 7f16ddd9d700 1 -- 172.21.15.31:0/1931158292 shutdown_connections 2023-10-06T02:27:12.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.975+0000 7f16ddd9d700 1 --2- 172.21.15.31:0/1931158292 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16d80b1d30 0x7f16d80c9db0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:12.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.975+0000 7f16ddd9d700 1 --2- 172.21.15.31:0/1931158292 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f16bc067290 0x7f16bc069740 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:12.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.975+0000 7f16ddd9d700 1 --2- 172.21.15.31:0/1931158292 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f16c8004d50 0x7f16d80c3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:12.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.975+0000 7f16ddd9d700 1 --2- 172.21.15.31:0/1931158292 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16d8155120 0x7f16d80b17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:12.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.975+0000 7f16ddd9d700 1 -- 172.21.15.31:0/1931158292 >> 172.21.15.31:0/1931158292 conn(0x7f16d806d110 msgr2=0x7f16d806de50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:12.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.975+0000 7f16ddd9d700 1 -- 172.21.15.31:0/1931158292 shutdown_connections 2023-10-06T02:27:12.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:12.975+0000 7f16ddd9d700 1 -- 172.21.15.31:0/1931158292 wait complete. 2023-10-06T02:27:12.994 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade' 2023-10-06T02:27:13.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.180+0000 7fb89dd9d700 1 Processor -- start 2023-10-06T02:27:13.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.180+0000 7fb89dd9d700 1 -- start start 2023-10-06T02:27:13.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.181+0000 7fb89dd9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb8980c7f90 0x7fb89806c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:13.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.181+0000 7fb89dd9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8980ca480 con 0x7fb898142c00 2023-10-06T02:27:13.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.181+0000 7fb89dd9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb89815e240 con 0x7fb898142fd0 2023-10-06T02:27:13.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.181+0000 7fb89dd9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb89806b350 con 0x7fb8980c7f90 2023-10-06T02:27:13.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.181+0000 7fb89cd9b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fb898142c00 0x7fb89806b7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:47746/0 (socket says 172.21.15.31:47746) 2023-10-06T02:27:13.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.181+0000 7fb89cd9b700 1 -- 172.21.15.31:0/475087608 learned_addr learned my addr 172.21.15.31:0/475087608 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:27:13.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.182+0000 7fb8977fe700 1 -- 172.21.15.31:0/475087608 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 247556519 0 0) 0x7fb8980ca480 con 0x7fb898142c00 2023-10-06T02:27:13.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.182+0000 7fb8977fe700 1 -- 172.21.15.31:0/475087608 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb888003570 con 0x7fb898142c00 2023-10-06T02:27:13.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.182+0000 7fb89d59c700 1 --2- 172.21.15.31:0/475087608 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb8980c7f90 0x7fb89806c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:13.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.182+0000 7fb8977fe700 1 -- 172.21.15.31:0/475087608 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3970367367 0 0) 0x7fb888003570 con 0x7fb898142c00 2023-10-06T02:27:13.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.182+0000 7fb8977fe700 1 -- 172.21.15.31:0/475087608 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb8980c7f90 msgr2=0x7fb89806c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:13.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.182+0000 7fb8977fe700 1 --2- 172.21.15.31:0/475087608 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb8980c7f90 0x7fb89806c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:13.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.182+0000 7fb8977fe700 1 -- 172.21.15.31:0/475087608 >> v1:172.21.15.62:6789/0 conn(0x7fb898142fd0 legacy=0x7fb8980c7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:13.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.182+0000 7fb8977fe700 1 -- 172.21.15.31:0/475087608 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb89815df60 con 0x7fb898142c00 2023-10-06T02:27:13.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.182+0000 7fb8977fe700 1 -- 172.21.15.31:0/475087608 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7fb88c001470 con 0x7fb898142c00 2023-10-06T02:27:13.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.182+0000 7fb8977fe700 1 -- 172.21.15.31:0/475087608 >> v1:172.21.15.31:6789/0 conn(0x7fb898142c00 legacy=0x7fb89806b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:13.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.182+0000 7fb8977fe700 1 --2- 172.21.15.31:0/475087608 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb888003a90 0x7fb888003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:13.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.182+0000 7fb8977fe700 1 --2- 172.21.15.31:0/475087608 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb8980c7f90 0x7fb888004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:13.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.182+0000 7fb8977fe700 1 --2- 172.21.15.31:0/475087608 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb888004e70 0x7fb8880052e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:13.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.182+0000 7fb8977fe700 1 -- 172.21.15.31:0/475087608 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb89806b350 con 0x7fb888003a90 2023-10-06T02:27:13.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.182+0000 7fb8977fe700 1 -- 172.21.15.31:0/475087608 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb88800bc70 con 0x7fb888004e70 2023-10-06T02:27:13.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.182+0000 7fb8977fe700 1 -- 172.21.15.31:0/475087608 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb88800be30 con 0x7fb8980c7f90 2023-10-06T02:27:13.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.182+0000 7fb89cd9b700 1 --2- 172.21.15.31:0/475087608 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb888003a90 0x7fb888003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:13.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.183+0000 7fb897fff700 1 --2- 172.21.15.31:0/475087608 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb8980c7f90 0x7fb888004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:13.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.183+0000 7fb89d59c700 1 --2- 172.21.15.31:0/475087608 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb888004e70 0x7fb8880052e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:13.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.183+0000 7fb89cd9b700 1 -- 172.21.15.31:0/475087608 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb8980c7f90 msgr2=0x7fb888004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:13.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.183+0000 7fb89cd9b700 1 --2- 172.21.15.31:0/475087608 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb8980c7f90 0x7fb888004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:13.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.183+0000 7fb89cd9b700 1 -- 172.21.15.31:0/475087608 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb888004e70 msgr2=0x7fb8880052e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:13.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.183+0000 7fb89cd9b700 1 --2- 172.21.15.31:0/475087608 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb888004e70 0x7fb8880052e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:13.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.183+0000 7fb89cd9b700 1 -- 172.21.15.31:0/475087608 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb888003570 con 0x7fb888003a90 2023-10-06T02:27:13.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.183+0000 7fb897fff700 1 --2- 172.21.15.31:0/475087608 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb8980c7f90 0x7fb888004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:27:13.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.183+0000 7fb89cd9b700 1 --2- 172.21.15.31:0/475087608 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb888003a90 0x7fb888003ea0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fb88c001ed0 tx=0x7fb898143470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66ed808fc97249cd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:13.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.183+0000 7fb8977fe700 1 -- 172.21.15.31:0/475087608 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb88c00cb40 con 0x7fb888003a90 2023-10-06T02:27:13.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.183+0000 7fb8977fe700 1 -- 172.21.15.31:0/475087608 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb88c00cca0 con 0x7fb888003a90 2023-10-06T02:27:13.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.183+0000 7fb8977fe700 1 -- 172.21.15.31:0/475087608 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb88c005b70 con 0x7fb888003a90 2023-10-06T02:27:13.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.184+0000 7fb89dd9d700 1 -- 172.21.15.31:0/475087608 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb888003a90 msgr2=0x7fb888003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:13.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.184+0000 7fb89dd9d700 1 --2- 172.21.15.31:0/475087608 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb888003a90 0x7fb888003ea0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fb88c001ed0 tx=0x7fb898143470 comp rx=0 tx=0).stop 2023-10-06T02:27:13.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.184+0000 7fb89d59c700 1 -- 172.21.15.31:0/475087608 reap_dead start 2023-10-06T02:27:13.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.184+0000 7fb89dd9d700 1 -- 172.21.15.31:0/475087608 shutdown_connections 2023-10-06T02:27:13.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.184+0000 7fb89dd9d700 1 -- 172.21.15.31:0/475087608 >> 172.21.15.31:0/475087608 conn(0x7fb8980675a0 msgr2=0x7fb8980679a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:13.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.184+0000 7fb89dd9d700 1 -- 172.21.15.31:0/475087608 shutdown_connections 2023-10-06T02:27:13.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.184+0000 7fb89dd9d700 1 -- 172.21.15.31:0/475087608 wait complete. 2023-10-06T02:27:13.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.185+0000 7fb89dd9d700 1 Processor -- start 2023-10-06T02:27:13.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.186+0000 7fb89dd9d700 1 -- start start 2023-10-06T02:27:13.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.186+0000 7fb89dd9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb888003a90 0x7fb89816a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:13.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.186+0000 7fb89dd9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb89813d1c0 0x7fb8980c2d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:13.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.186+0000 7fb89dd9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb8980c32c0 0x7fb8980affe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:13.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.186+0000 7fb89dd9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8980c3730 con 0x7fb89813d1c0 2023-10-06T02:27:13.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.186+0000 7fb89dd9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb89816a890 con 0x7fb888003a90 2023-10-06T02:27:13.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.186+0000 7fb89dd9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8980c4940 con 0x7fb8980c32c0 2023-10-06T02:27:13.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.187+0000 7fb897fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb89813d1c0 0x7fb8980c2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:13.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.187+0000 7fb897fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb89813d1c0 0x7fb8980c2d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50554/0 (socket says 172.21.15.31:50554) 2023-10-06T02:27:13.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.187+0000 7fb897fff700 1 -- 172.21.15.31:0/3786479000 learned_addr learned my addr 172.21.15.31:0/3786479000 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:13.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.187+0000 7fb89cd9b700 1 --2- 172.21.15.31:0/3786479000 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb888003a90 0x7fb89816a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:13.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.187+0000 7fb89d59c700 1 --2- 172.21.15.31:0/3786479000 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb8980c32c0 0x7fb8980affe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:13.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.187+0000 7fb897fff700 1 -- 172.21.15.31:0/3786479000 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb8980c32c0 msgr2=0x7fb8980affe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:13.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.187+0000 7fb897fff700 1 --2- 172.21.15.31:0/3786479000 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb8980c32c0 0x7fb8980affe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:13.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.187+0000 7fb897fff700 1 -- 172.21.15.31:0/3786479000 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb888003a90 msgr2=0x7fb89816a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:13.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.187+0000 7fb897fff700 1 --2- 172.21.15.31:0/3786479000 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb888003a90 0x7fb89816a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:13.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.187+0000 7fb897fff700 1 -- 172.21.15.31:0/3786479000 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8980b1d30 con 0x7fb89813d1c0 2023-10-06T02:27:13.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.187+0000 7fb897fff700 1 --2- 172.21.15.31:0/3786479000 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb89813d1c0 0x7fb8980c2d80 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fb88400e840 tx=0x7fb884009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:13.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.188+0000 7fb894ff9700 1 -- 172.21.15.31:0/3786479000 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb884017070 con 0x7fb89813d1c0 2023-10-06T02:27:13.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.188+0000 7fb894ff9700 1 -- 172.21.15.31:0/3786479000 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb884006820 con 0x7fb89813d1c0 2023-10-06T02:27:13.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.188+0000 7fb89dd9d700 1 -- 172.21.15.31:0/3786479000 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8980b2010 con 0x7fb89813d1c0 2023-10-06T02:27:13.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.188+0000 7fb894ff9700 1 -- 172.21.15.31:0/3786479000 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb884006be0 con 0x7fb89813d1c0 2023-10-06T02:27:13.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.188+0000 7fb89dd9d700 1 -- 172.21.15.31:0/3786479000 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8980c45d0 con 0x7fb89813d1c0 2023-10-06T02:27:13.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.189+0000 7fb89dd9d700 1 -- 172.21.15.31:0/3786479000 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb89804c3c0 con 0x7fb89813d1c0 2023-10-06T02:27:13.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.190+0000 7fb894ff9700 1 -- 172.21.15.31:0/3786479000 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fb884006d40 con 0x7fb89813d1c0 2023-10-06T02:27:13.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.191+0000 7fb894ff9700 1 --2- 172.21.15.31:0/3786479000 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fb87c067290 0x7fb87c069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:13.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.191+0000 7fb894ff9700 1 -- 172.21.15.31:0/3786479000 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5292+0+0 (secure 0 0 0) 0x7fb884084530 con 0x7fb89813d1c0 2023-10-06T02:27:13.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.191+0000 7fb89cd9b700 1 --2- 172.21.15.31:0/3786479000 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fb87c067290 0x7fb87c069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:13.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.192+0000 7fb89cd9b700 1 --2- 172.21.15.31:0/3786479000 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fb87c067290 0x7fb87c069740 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb89806c170 tx=0x7fb88c015000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:13.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.195+0000 7fb894ff9700 1 -- 172.21.15.31:0/3786479000 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb884051510 con 0x7fb89813d1c0 2023-10-06T02:27:13.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.477+0000 7fb89dd9d700 1 -- 172.21.15.31:0/3786479000 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade"]} v 0) v1 -- 0x7fb8980c9cb0 con 0x7fb89813d1c0 2023-10-06T02:27:13.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.535+0000 7fb894ff9700 1 -- 172.21.15.31:0/3786479000 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade"]}]=0 v232) v1 ==== 161+0+0 (secure 0 0 0) 0x7fb884054b30 con 0x7fb89813d1c0 2023-10-06T02:27:13.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.538+0000 7fb89dd9d700 1 -- 172.21.15.31:0/3786479000 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fb87c067290 msgr2=0x7fb87c069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:13.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.538+0000 7fb89dd9d700 1 --2- 172.21.15.31:0/3786479000 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fb87c067290 0x7fb87c069740 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb89806c170 tx=0x7fb88c015000 comp rx=0 tx=0).stop 2023-10-06T02:27:13.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.538+0000 7fb89dd9d700 1 -- 172.21.15.31:0/3786479000 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb89813d1c0 msgr2=0x7fb8980c2d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:13.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.538+0000 7fb89dd9d700 1 --2- 172.21.15.31:0/3786479000 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb89813d1c0 0x7fb8980c2d80 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fb88400e840 tx=0x7fb884009040 comp rx=0 tx=0).stop 2023-10-06T02:27:13.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.538+0000 7fb89dd9d700 1 -- 172.21.15.31:0/3786479000 shutdown_connections 2023-10-06T02:27:13.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.538+0000 7fb89dd9d700 1 --2- 172.21.15.31:0/3786479000 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fb87c067290 0x7fb87c069740 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:13.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.538+0000 7fb89dd9d700 1 --2- 172.21.15.31:0/3786479000 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb8980c32c0 0x7fb8980affe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:13.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.538+0000 7fb89dd9d700 1 --2- 172.21.15.31:0/3786479000 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb888003a90 0x7fb89816a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:13.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.538+0000 7fb89dd9d700 1 --2- 172.21.15.31:0/3786479000 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb89813d1c0 0x7fb8980c2d80 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:13.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.538+0000 7fb89dd9d700 1 -- 172.21.15.31:0/3786479000 >> 172.21.15.31:0/3786479000 conn(0x7fb8980675a0 msgr2=0x7fb89813f560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:13.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.538+0000 7fb89dd9d700 1 -- 172.21.15.31:0/3786479000 shutdown_connections 2023-10-06T02:27:13.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.538+0000 7fb89dd9d700 1 -- 172.21.15.31:0/3786479000 wait complete. 2023-10-06T02:27:13.567 INFO:tasks.cephfs_test_runner:Validate handcrafted .meta file on legacy subvol root doesn't break the system ... ok 2023-10-06T02:27:13.567 INFO:tasks.cephfs_test_runner:Starting test: test_mgr_eviction (tasks.cephfs.test_volumes.TestMisc) 2023-10-06T02:27:13.568 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction' 2023-10-06T02:27:13.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.753+0000 7f7b7d39d700 1 Processor -- start 2023-10-06T02:27:13.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.754+0000 7f7b7d39d700 1 -- start start 2023-10-06T02:27:13.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.755+0000 7f7b7d39d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7b78140a70 0x7f7b780c6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:13.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.755+0000 7f7b7d39d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7b78140e40 0x7f7b780c7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:13.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.755+0000 7f7b7d39d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7b780c8330 0x7f7b780aeef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:13.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.755+0000 7f7b7d39d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b780c6d50 con 0x7f7b78140a70 2023-10-06T02:27:13.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.755+0000 7f7b7d39d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b780c7030 con 0x7f7b780c8330 2023-10-06T02:27:13.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.755+0000 7f7b7d39d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b780c7310 con 0x7f7b78140e40 2023-10-06T02:27:13.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.755+0000 7f7b77fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7b78140a70 0x7f7b780c6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:13.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.755+0000 7f7b77fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7b78140a70 0x7f7b780c6770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50558/0 (socket says 172.21.15.31:50558) 2023-10-06T02:27:13.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.755+0000 7f7b77fff700 1 -- 172.21.15.31:0/3999979735 learned_addr learned my addr 172.21.15.31:0/3999979735 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:13.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.756+0000 7f7b77fff700 1 -- 172.21.15.31:0/3999979735 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7b78140e40 msgr2=0x7f7b780c7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:13.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.756+0000 7f7b77fff700 1 --2- 172.21.15.31:0/3999979735 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7b78140e40 0x7f7b780c7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:13.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.756+0000 7f7b77fff700 1 -- 172.21.15.31:0/3999979735 >> v2:172.21.15.62:3300/0 conn(0x7f7b780c8330 msgr2=0x7f7b780aeef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:13.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.756+0000 7f7b77fff700 1 --2- 172.21.15.31:0/3999979735 >> v2:172.21.15.62:3300/0 conn(0x7f7b780c8330 0x7f7b780aeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:13.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.756+0000 7f7b77fff700 1 -- 172.21.15.31:0/3999979735 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b780c75f0 con 0x7f7b78140a70 2023-10-06T02:27:13.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.756+0000 7f7b77fff700 1 --2- 172.21.15.31:0/3999979735 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b78140a70 0x7f7b780c6770 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f7b78069a90 tx=0x7f7b68009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b85cfd09e0600d31 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:13.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.756+0000 7f7b76ffd700 1 -- 172.21.15.31:0/3999979735 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b6800b860 con 0x7f7b78140a70 2023-10-06T02:27:13.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.756+0000 7f7b76ffd700 1 -- 172.21.15.31:0/3999979735 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b6800b9c0 con 0x7f7b78140a70 2023-10-06T02:27:13.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.756+0000 7f7b76ffd700 1 -- 172.21.15.31:0/3999979735 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b6800bd60 con 0x7f7b78140a70 2023-10-06T02:27:13.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.757+0000 7f7b7d39d700 1 -- 172.21.15.31:0/3999979735 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b78140a70 msgr2=0x7f7b780c6770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:13.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.757+0000 7f7b7d39d700 1 --2- 172.21.15.31:0/3999979735 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b78140a70 0x7f7b780c6770 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f7b78069a90 tx=0x7f7b68009250 comp rx=0 tx=0).stop 2023-10-06T02:27:13.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.757+0000 7f7b7d39d700 1 -- 172.21.15.31:0/3999979735 shutdown_connections 2023-10-06T02:27:13.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.757+0000 7f7b7d39d700 1 --2- 172.21.15.31:0/3999979735 >> v2:172.21.15.62:3300/0 conn(0x7f7b780c8330 0x7f7b780aeef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:13.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.757+0000 7f7b7d39d700 1 --2- 172.21.15.31:0/3999979735 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b78140a70 0x7f7b780c6770 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:13.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.757+0000 7f7b7d39d700 1 --2- 172.21.15.31:0/3999979735 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7b78140e40 0x7f7b780c7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:13.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.757+0000 7f7b7d39d700 1 -- 172.21.15.31:0/3999979735 >> 172.21.15.31:0/3999979735 conn(0x7f7b7806d170 msgr2=0x7f7b780cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:13.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.757+0000 7f7b7d39d700 1 -- 172.21.15.31:0/3999979735 shutdown_connections 2023-10-06T02:27:13.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.757+0000 7f7b7d39d700 1 -- 172.21.15.31:0/3999979735 wait complete. 2023-10-06T02:27:13.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.758+0000 7f7b7d39d700 1 Processor -- start 2023-10-06T02:27:13.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.759+0000 7f7b7d39d700 1 -- start start 2023-10-06T02:27:13.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.759+0000 7f7b7d39d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7b780c8330 0x7f7b7813ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:13.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.759+0000 7f7b7d39d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b78140a70 0x7f7b78159ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:13.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.759+0000 7f7b7d39d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b78140e40 0x7f7b780b0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:13.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.759+0000 7f7b7d39d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b7813b3b0 con 0x7f7b78140e40 2023-10-06T02:27:13.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.759+0000 7f7b7d39d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b78157960 con 0x7f7b78140a70 2023-10-06T02:27:13.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.759+0000 7f7b7d39d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b780afa10 con 0x7f7b780c8330 2023-10-06T02:27:13.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.760+0000 7f7b7cb9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b78140e40 0x7f7b780b0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:13.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.760+0000 7f7b7cb9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b78140e40 0x7f7b780b0eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50560/0 (socket says 172.21.15.31:50560) 2023-10-06T02:27:13.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.760+0000 7f7b7cb9c700 1 -- 172.21.15.31:0/271914566 learned_addr learned my addr 172.21.15.31:0/271914566 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:13.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.760+0000 7f7b77fff700 1 --2- 172.21.15.31:0/271914566 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7b780c8330 0x7f7b7813ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:13.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.760+0000 7f7b777fe700 1 --2- 172.21.15.31:0/271914566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b78140a70 0x7f7b78159ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:13.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.760+0000 7f7b7cb9c700 1 -- 172.21.15.31:0/271914566 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7b780c8330 msgr2=0x7f7b7813ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:13.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.760+0000 7f7b7cb9c700 1 --2- 172.21.15.31:0/271914566 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7b780c8330 0x7f7b7813ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:13.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.760+0000 7f7b7cb9c700 1 -- 172.21.15.31:0/271914566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b78140a70 msgr2=0x7f7b78159ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:13.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.760+0000 7f7b7cb9c700 1 --2- 172.21.15.31:0/271914566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b78140a70 0x7f7b78159ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:13.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.760+0000 7f7b7cb9c700 1 -- 172.21.15.31:0/271914566 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b680094f0 con 0x7f7b78140e40 2023-10-06T02:27:13.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.760+0000 7f7b7cb9c700 1 --2- 172.21.15.31:0/271914566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b78140e40 0x7f7b780b0eb0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f7b7000bb60 tx=0x7f7b70009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:13.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.761+0000 7f7b5ffff700 1 -- 172.21.15.31:0/271914566 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7000c7e0 con 0x7f7b78140e40 2023-10-06T02:27:13.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.761+0000 7f7b5ffff700 1 -- 172.21.15.31:0/271914566 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b7000c940 con 0x7f7b78140e40 2023-10-06T02:27:13.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.761+0000 7f7b5ffff700 1 -- 172.21.15.31:0/271914566 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7000cd00 con 0x7f7b78140e40 2023-10-06T02:27:13.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.761+0000 7f7b7d39d700 1 -- 172.21.15.31:0/271914566 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b780afcf0 con 0x7f7b78140e40 2023-10-06T02:27:13.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.761+0000 7f7b7d39d700 1 -- 172.21.15.31:0/271914566 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b780b02b0 con 0x7f7b78140e40 2023-10-06T02:27:13.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.762+0000 7f7b7d39d700 1 -- 172.21.15.31:0/271914566 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b78157b10 con 0x7f7b78140e40 2023-10-06T02:27:13.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.763+0000 7f7b5ffff700 1 -- 172.21.15.31:0/271914566 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f7b70014620 con 0x7f7b78140e40 2023-10-06T02:27:13.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.763+0000 7f7b5ffff700 1 --2- 172.21.15.31:0/271914566 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f7b64067290 0x7f7b64069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:13.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.764+0000 7f7b5ffff700 1 -- 172.21.15.31:0/271914566 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f7b70084810 con 0x7f7b78140e40 2023-10-06T02:27:13.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.764+0000 7f7b77fff700 1 --2- 172.21.15.31:0/271914566 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f7b64067290 0x7f7b64069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:13.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.764+0000 7f7b77fff700 1 --2- 172.21.15.31:0/271914566 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f7b64067290 0x7f7b64069740 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f7b78069a90 tx=0x7f7b68017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:13.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:13.768+0000 7f7b5ffff700 1 -- 172.21.15.31:0/271914566 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7b7004c5a0 con 0x7f7b78140e40 2023-10-06T02:27:14.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.048+0000 7f7b7d39d700 1 -- 172.21.15.31:0/271914566 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction"]} v 0) v1 -- 0x7f7b7804c3c0 con 0x7f7b78140e40 2023-10-06T02:27:14.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.537+0000 7f7b5ffff700 1 -- 172.21.15.31:0/271914566 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction"]}]=0 v233) v1 ==== 134+0+0 (secure 0 0 0) 0x7f7b78157b10 con 0x7f7b78140e40 2023-10-06T02:27:14.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.540+0000 7f7b7d39d700 1 -- 172.21.15.31:0/271914566 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f7b64067290 msgr2=0x7f7b64069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:14.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.540+0000 7f7b7d39d700 1 --2- 172.21.15.31:0/271914566 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f7b64067290 0x7f7b64069740 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f7b78069a90 tx=0x7f7b68017000 comp rx=0 tx=0).stop 2023-10-06T02:27:14.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.540+0000 7f7b7d39d700 1 -- 172.21.15.31:0/271914566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b78140e40 msgr2=0x7f7b780b0eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:14.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.540+0000 7f7b7d39d700 1 --2- 172.21.15.31:0/271914566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b78140e40 0x7f7b780b0eb0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f7b7000bb60 tx=0x7f7b70009810 comp rx=0 tx=0).stop 2023-10-06T02:27:14.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.540+0000 7f7b7d39d700 1 -- 172.21.15.31:0/271914566 shutdown_connections 2023-10-06T02:27:14.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.540+0000 7f7b7d39d700 1 --2- 172.21.15.31:0/271914566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b78140e40 0x7f7b780b0eb0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:14.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.540+0000 7f7b7d39d700 1 --2- 172.21.15.31:0/271914566 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f7b64067290 0x7f7b64069740 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:14.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.540+0000 7f7b7d39d700 1 --2- 172.21.15.31:0/271914566 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7b780c8330 0x7f7b7813ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:14.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.540+0000 7f7b7d39d700 1 --2- 172.21.15.31:0/271914566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b78140a70 0x7f7b78159ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:14.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.540+0000 7f7b7d39d700 1 -- 172.21.15.31:0/271914566 >> 172.21.15.31:0/271914566 conn(0x7f7b7806d170 msgr2=0x7f7b78069ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:14.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.540+0000 7f7b7d39d700 1 -- 172.21.15.31:0/271914566 shutdown_connections 2023-10-06T02:27:14.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.540+0000 7f7b7d39d700 1 -- 172.21.15.31:0/271914566 wait complete. 2023-10-06T02:27:14.561 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T02:27:14.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.750+0000 7fe78659d700 1 Processor -- start 2023-10-06T02:27:14.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.750+0000 7fe78659d700 1 -- start start 2023-10-06T02:27:14.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.752+0000 7fe78659d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe7800c7be0 0x7fe78006abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:14.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.752+0000 7fe78659d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe7800c7fb0 0x7fe78015a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:14.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.752+0000 7fe78659d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe78015aed0 con 0x7fe78013f780 2023-10-06T02:27:14.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.752+0000 7fe78659d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe78006bf10 con 0x7fe7800c7fb0 2023-10-06T02:27:14.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.752+0000 7fe78659d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe78006b290 con 0x7fe7800c7be0 2023-10-06T02:27:14.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.752+0000 7fe784d9a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe7800c7fb0 0x7fe78015a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:14.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.752+0000 7fe785d9c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fe78013f780 0x7fe78013fb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:47752/0 (socket says 172.21.15.31:47752) 2023-10-06T02:27:14.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.752+0000 7fe785d9c700 1 -- 172.21.15.31:0/3229308271 learned_addr learned my addr 172.21.15.31:0/3229308271 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:27:14.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.752+0000 7fe78559b700 1 --2- 172.21.15.31:0/3229308271 >> v2:172.21.15.62:3300/0 conn(0x7fe7800c7be0 0x7fe78006abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:14.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.752+0000 7fe784d9a700 1 -- 172.21.15.31:0/3229308271 >> v2:172.21.15.62:3300/0 conn(0x7fe7800c7be0 msgr2=0x7fe78006abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:14.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.752+0000 7fe784d9a700 1 --2- 172.21.15.31:0/3229308271 >> v2:172.21.15.62:3300/0 conn(0x7fe7800c7be0 0x7fe78006abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:14.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.752+0000 7fe784d9a700 1 -- 172.21.15.31:0/3229308271 >> v1:172.21.15.31:6789/0 conn(0x7fe78013f780 legacy=0x7fe78013fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:14.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.752+0000 7fe784d9a700 1 -- 172.21.15.31:0/3229308271 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe78006b570 con 0x7fe7800c7fb0 2023-10-06T02:27:14.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.752+0000 7fe777fff700 1 -- 172.21.15.31:0/3229308271 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2997007314 0 0) 0x7fe78015aed0 con 0x7fe78013f780 2023-10-06T02:27:14.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.753+0000 7fe784d9a700 1 --2- 172.21.15.31:0/3229308271 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7800c7fb0 0x7fe78015a990 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fe7800c8b00 tx=0x7fe7700091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dac4175f14a7cf59 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:14.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.753+0000 7fe777fff700 1 -- 172.21.15.31:0/3229308271 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe77000b6b0 con 0x7fe7800c7fb0 2023-10-06T02:27:14.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.753+0000 7fe777fff700 1 -- 172.21.15.31:0/3229308271 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe77000b810 con 0x7fe7800c7fb0 2023-10-06T02:27:14.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.753+0000 7fe777fff700 1 -- 172.21.15.31:0/3229308271 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe77000bbb0 con 0x7fe7800c7fb0 2023-10-06T02:27:14.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.753+0000 7fe78659d700 1 -- 172.21.15.31:0/3229308271 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7800c7fb0 msgr2=0x7fe78015a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:14.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.753+0000 7fe78659d700 1 --2- 172.21.15.31:0/3229308271 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7800c7fb0 0x7fe78015a990 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fe7800c8b00 tx=0x7fe7700091a0 comp rx=0 tx=0).stop 2023-10-06T02:27:14.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.754+0000 7fe78659d700 1 -- 172.21.15.31:0/3229308271 shutdown_connections 2023-10-06T02:27:14.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.754+0000 7fe78659d700 1 --2- 172.21.15.31:0/3229308271 >> v2:172.21.15.62:3300/0 conn(0x7fe7800c7be0 0x7fe78006abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:14.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.754+0000 7fe78659d700 1 --2- 172.21.15.31:0/3229308271 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7800c7fb0 0x7fe78015a990 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:14.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.754+0000 7fe78659d700 1 -- 172.21.15.31:0/3229308271 >> 172.21.15.31:0/3229308271 conn(0x7fe78006d110 msgr2=0x7fe7800cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:14.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.754+0000 7fe78659d700 1 -- 172.21.15.31:0/3229308271 shutdown_connections 2023-10-06T02:27:14.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.754+0000 7fe78659d700 1 -- 172.21.15.31:0/3229308271 wait complete. 2023-10-06T02:27:14.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.755+0000 7fe78659d700 1 Processor -- start 2023-10-06T02:27:14.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.755+0000 7fe78659d700 1 -- start start 2023-10-06T02:27:14.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.756+0000 7fe78659d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7800c7be0 0x7fe7800c91d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:14.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.756+0000 7fe78659d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe7800c7fb0 0x7fe780155bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:14.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.756+0000 7fe78659d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe78013f780 0x7fe7800c4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:14.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.756+0000 7fe78659d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7800c61c0 con 0x7fe78013f780 2023-10-06T02:27:14.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.756+0000 7fe78659d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7800c3670 con 0x7fe7800c7be0 2023-10-06T02:27:14.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.756+0000 7fe78659d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7800c3950 con 0x7fe7800c7fb0 2023-10-06T02:27:14.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.756+0000 7fe785d9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe78013f780 0x7fe7800c4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:14.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.756+0000 7fe785d9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe78013f780 0x7fe7800c4fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50578/0 (socket says 172.21.15.31:50578) 2023-10-06T02:27:14.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.756+0000 7fe785d9c700 1 -- 172.21.15.31:0/2811538660 learned_addr learned my addr 172.21.15.31:0/2811538660 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:14.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.757+0000 7fe78559b700 1 --2- 172.21.15.31:0/2811538660 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7800c7be0 0x7fe7800c91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:14.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.757+0000 7fe784d9a700 1 --2- 172.21.15.31:0/2811538660 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe7800c7fb0 0x7fe780155bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:14.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.757+0000 7fe785d9c700 1 -- 172.21.15.31:0/2811538660 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe7800c7fb0 msgr2=0x7fe780155bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:14.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.757+0000 7fe785d9c700 1 --2- 172.21.15.31:0/2811538660 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe7800c7fb0 0x7fe780155bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:14.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.757+0000 7fe785d9c700 1 -- 172.21.15.31:0/2811538660 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7800c7be0 msgr2=0x7fe7800c91d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:14.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.757+0000 7fe785d9c700 1 --2- 172.21.15.31:0/2811538660 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7800c7be0 0x7fe7800c91d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:14.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.757+0000 7fe785d9c700 1 -- 172.21.15.31:0/2811538660 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe770009420 con 0x7fe78013f780 2023-10-06T02:27:14.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.757+0000 7fe784d9a700 1 --2- 172.21.15.31:0/2811538660 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe7800c7fb0 0x7fe780155bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:27:14.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.757+0000 7fe785d9c700 1 --2- 172.21.15.31:0/2811538660 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe78013f780 0x7fe7800c4fe0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fe77800cbc0 tx=0x7fe778001980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:14.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.757+0000 7fe7757fa700 1 -- 172.21.15.31:0/2811538660 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe77800a890 con 0x7fe78013f780 2023-10-06T02:27:14.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.757+0000 7fe7757fa700 1 -- 172.21.15.31:0/2811538660 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe77800a9f0 con 0x7fe78013f780 2023-10-06T02:27:14.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.757+0000 7fe7757fa700 1 -- 172.21.15.31:0/2811538660 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe77800adb0 con 0x7fe78013f780 2023-10-06T02:27:14.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.757+0000 7fe78659d700 1 -- 172.21.15.31:0/2811538660 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7800c5e50 con 0x7fe78013f780 2023-10-06T02:27:14.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.757+0000 7fe78659d700 1 -- 172.21.15.31:0/2811538660 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe780156100 con 0x7fe78013f780 2023-10-06T02:27:14.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.758+0000 7fe78659d700 1 -- 172.21.15.31:0/2811538660 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7800c6550 con 0x7fe78013f780 2023-10-06T02:27:14.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.759+0000 7fe7757fa700 1 -- 172.21.15.31:0/2811538660 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fe778007300 con 0x7fe78013f780 2023-10-06T02:27:14.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.761+0000 7fe7757fa700 1 --2- 172.21.15.31:0/2811538660 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe764067290 0x7fe764069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:14.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.761+0000 7fe7757fa700 1 -- 172.21.15.31:0/2811538660 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5292+0+0 (secure 0 0 0) 0x7fe778084cc0 con 0x7fe78013f780 2023-10-06T02:27:14.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.761+0000 7fe78559b700 1 --2- 172.21.15.31:0/2811538660 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe764067290 0x7fe764069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:14.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.761+0000 7fe78559b700 1 --2- 172.21.15.31:0/2811538660 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe764067290 0x7fe764069740 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe77c006fd0 tx=0x7fe77c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:14.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.765+0000 7fe7757fa700 1 -- 172.21.15.31:0/2811538660 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe778051c40 con 0x7fe78013f780 2023-10-06T02:27:14.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.967+0000 7fe78659d700 1 -- 172.21.15.31:0/2811538660 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fe78004c3c0 con 0x7fe78013f780 2023-10-06T02:27:14.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.971+0000 7fe7757fa700 1 -- 172.21.15.31:0/2811538660 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe778055260 con 0x7fe78013f780 2023-10-06T02:27:14.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.974+0000 7fe78659d700 1 -- 172.21.15.31:0/2811538660 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe764067290 msgr2=0x7fe764069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:14.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.974+0000 7fe78659d700 1 --2- 172.21.15.31:0/2811538660 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe764067290 0x7fe764069740 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe77c006fd0 tx=0x7fe77c008040 comp rx=0 tx=0).stop 2023-10-06T02:27:14.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.974+0000 7fe78659d700 1 -- 172.21.15.31:0/2811538660 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe78013f780 msgr2=0x7fe7800c4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:14.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.974+0000 7fe78659d700 1 --2- 172.21.15.31:0/2811538660 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe78013f780 0x7fe7800c4fe0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fe77800cbc0 tx=0x7fe778001980 comp rx=0 tx=0).stop 2023-10-06T02:27:14.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.974+0000 7fe78659d700 1 -- 172.21.15.31:0/2811538660 shutdown_connections 2023-10-06T02:27:14.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.974+0000 7fe78659d700 1 --2- 172.21.15.31:0/2811538660 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe78013f780 0x7fe7800c4fe0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:14.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.974+0000 7fe78659d700 1 --2- 172.21.15.31:0/2811538660 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7800c7be0 0x7fe7800c91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:14.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.974+0000 7fe78659d700 1 --2- 172.21.15.31:0/2811538660 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe764067290 0x7fe764069740 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:14.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.974+0000 7fe78659d700 1 --2- 172.21.15.31:0/2811538660 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe7800c7fb0 0x7fe780155bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:14.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.974+0000 7fe78659d700 1 -- 172.21.15.31:0/2811538660 >> 172.21.15.31:0/2811538660 conn(0x7fe78006d110 msgr2=0x7fe78015b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:14.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.974+0000 7fe78659d700 1 -- 172.21.15.31:0/2811538660 shutdown_connections 2023-10-06T02:27:14.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:14.974+0000 7fe78659d700 1 -- 172.21.15.31:0/2811538660 wait complete. 2023-10-06T02:27:14.993 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T02:27:14.994 DEBUG:teuthology.orchestra.run.smithi031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T02:27:15.072 DEBUG:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T02:27:15.161 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T02:27:15.162 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T02:27:15.162 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:27:15.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.348+0000 7fd83f59e700 1 Processor -- start 2023-10-06T02:27:15.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.348+0000 7fd83f59e700 1 -- start start 2023-10-06T02:27:15.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.349+0000 7fd83f59e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd84015de50 0x7fd8400cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:15.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.349+0000 7fd83f59e700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd84015afd0 con 0x7fd8400cef40 2023-10-06T02:27:15.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.349+0000 7fd83f59e700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd84015b2b0 con 0x7fd840153c30 2023-10-06T02:27:15.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.349+0000 7fd83f59e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd84015b590 con 0x7fd84015de50 2023-10-06T02:27:15.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.349+0000 7fd83ed9d700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fd8400cef40 0x7fd84015c070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:47758/0 (socket says 172.21.15.31:47758) 2023-10-06T02:27:15.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.349+0000 7fd83ed9d700 1 -- 172.21.15.31:0/3679080754 learned_addr learned my addr 172.21.15.31:0/3679080754 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:27:15.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.350+0000 7fd83dd9b700 1 --2- 172.21.15.31:0/3679080754 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd84015de50 0x7fd8400cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:15.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.350+0000 7fd83d59a700 1 -- 172.21.15.31:0/3679080754 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 930495584 0 0) 0x7fd84015afd0 con 0x7fd8400cef40 2023-10-06T02:27:15.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.350+0000 7fd83d59a700 1 -- 172.21.15.31:0/3679080754 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd828003570 con 0x7fd8400cef40 2023-10-06T02:27:15.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.350+0000 7fd83d59a700 1 -- 172.21.15.31:0/3679080754 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 598686543 0 0) 0x7fd84015b2b0 con 0x7fd840153c30 2023-10-06T02:27:15.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.350+0000 7fd83d59a700 1 -- 172.21.15.31:0/3679080754 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd84015afd0 con 0x7fd840153c30 2023-10-06T02:27:15.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.350+0000 7fd83d59a700 1 -- 172.21.15.31:0/3679080754 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4019252835 0 0) 0x7fd828003570 con 0x7fd8400cef40 2023-10-06T02:27:15.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.350+0000 7fd83d59a700 1 -- 172.21.15.31:0/3679080754 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd84015de50 msgr2=0x7fd8400cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:15.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.350+0000 7fd83d59a700 1 --2- 172.21.15.31:0/3679080754 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd84015de50 0x7fd8400cea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:15.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.350+0000 7fd83d59a700 1 -- 172.21.15.31:0/3679080754 >> v1:172.21.15.62:6789/0 conn(0x7fd840153c30 legacy=0x7fd84015a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:15.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.350+0000 7fd83d59a700 1 -- 172.21.15.31:0/3679080754 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd84015b870 con 0x7fd8400cef40 2023-10-06T02:27:15.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.350+0000 7fd83d59a700 1 -- 172.21.15.31:0/3679080754 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7fd8380017f0 con 0x7fd8400cef40 2023-10-06T02:27:15.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.350+0000 7fd83dd9b700 1 --2- 172.21.15.31:0/3679080754 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd84015de50 0x7fd8400cea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:27:15.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.350+0000 7fd83d59a700 1 -- 172.21.15.31:0/3679080754 >> v1:172.21.15.31:6789/0 conn(0x7fd8400cef40 legacy=0x7fd84015c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:15.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.350+0000 7fd83d59a700 1 --2- 172.21.15.31:0/3679080754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd828003c20 0x7fd828004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:15.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.350+0000 7fd83d59a700 1 --2- 172.21.15.31:0/3679080754 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd828004810 0x7fd828004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:15.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.350+0000 7fd83d59a700 1 --2- 172.21.15.31:0/3679080754 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd84015de50 0x7fd8280051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:15.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.350+0000 7fd83d59a700 1 -- 172.21.15.31:0/3679080754 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd84015b590 con 0x7fd828003c20 2023-10-06T02:27:15.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.350+0000 7fd83d59a700 1 -- 172.21.15.31:0/3679080754 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd82800bbe0 con 0x7fd828004810 2023-10-06T02:27:15.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.350+0000 7fd83d59a700 1 -- 172.21.15.31:0/3679080754 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd82800bda0 con 0x7fd84015de50 2023-10-06T02:27:15.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.351+0000 7fd83e59c700 1 --2- 172.21.15.31:0/3679080754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd828003c20 0x7fd828004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:15.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.351+0000 7fd83ed9d700 1 --2- 172.21.15.31:0/3679080754 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd84015de50 0x7fd8280051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:15.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.351+0000 7fd83dd9b700 1 --2- 172.21.15.31:0/3679080754 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd828004810 0x7fd828004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:15.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.351+0000 7fd83e59c700 1 -- 172.21.15.31:0/3679080754 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd84015de50 msgr2=0x7fd8280051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:15.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.351+0000 7fd83e59c700 1 --2- 172.21.15.31:0/3679080754 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd84015de50 0x7fd8280051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:15.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.351+0000 7fd83e59c700 1 -- 172.21.15.31:0/3679080754 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd828004810 msgr2=0x7fd828004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:15.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.351+0000 7fd83e59c700 1 --2- 172.21.15.31:0/3679080754 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd828004810 0x7fd828004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:15.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.351+0000 7fd83e59c700 1 -- 172.21.15.31:0/3679080754 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd834009420 con 0x7fd828003c20 2023-10-06T02:27:15.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.351+0000 7fd83dd9b700 1 --2- 172.21.15.31:0/3679080754 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd828004810 0x7fd828004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:27:15.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.351+0000 7fd83e59c700 1 --2- 172.21.15.31:0/3679080754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd828003c20 0x7fd828004010 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fd830001a10 tx=0x7fd830007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=524c94affa3a65e4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:15.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.352+0000 7fd83d59a700 1 -- 172.21.15.31:0/3679080754 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd83000c5d0 con 0x7fd828003c20 2023-10-06T02:27:15.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.352+0000 7fd83d59a700 1 -- 172.21.15.31:0/3679080754 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd83000c730 con 0x7fd828003c20 2023-10-06T02:27:15.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.352+0000 7fd83d59a700 1 -- 172.21.15.31:0/3679080754 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd83000cad0 con 0x7fd828003c20 2023-10-06T02:27:15.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.352+0000 7fd83f59e700 1 -- 172.21.15.31:0/3679080754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd828003c20 msgr2=0x7fd828004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:15.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.352+0000 7fd83f59e700 1 --2- 172.21.15.31:0/3679080754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd828003c20 0x7fd828004010 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fd830001a10 tx=0x7fd830007d30 comp rx=0 tx=0).stop 2023-10-06T02:27:15.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.352+0000 7fd83ed9d700 1 -- 172.21.15.31:0/3679080754 reap_dead start 2023-10-06T02:27:15.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.352+0000 7fd83f59e700 1 -- 172.21.15.31:0/3679080754 shutdown_connections 2023-10-06T02:27:15.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.352+0000 7fd83f59e700 1 -- 172.21.15.31:0/3679080754 >> 172.21.15.31:0/3679080754 conn(0x7fd8400703a0 msgr2=0x7fd8400707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:15.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.352+0000 7fd83f59e700 1 -- 172.21.15.31:0/3679080754 shutdown_connections 2023-10-06T02:27:15.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.352+0000 7fd83f59e700 1 -- 172.21.15.31:0/3679080754 wait complete. 2023-10-06T02:27:15.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.354+0000 7fd83f59e700 1 Processor -- start 2023-10-06T02:27:15.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.354+0000 7fd83f59e700 1 -- start start 2023-10-06T02:27:15.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.355+0000 7fd83f59e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd828003c20 0x7fd840152ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:15.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.355+0000 7fd83f59e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8400c8db0 0x7fd8400c9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:15.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.355+0000 7fd83f59e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd8400b1210 0x7fd8400b1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:15.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.355+0000 7fd83f59e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd840153530 con 0x7fd828003c20 2023-10-06T02:27:15.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.355+0000 7fd83f59e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8400b1bc0 con 0x7fd8400c8db0 2023-10-06T02:27:15.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.355+0000 7fd83f59e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd840169aa0 con 0x7fd8400b1210 2023-10-06T02:27:15.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.355+0000 7fd83e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd828003c20 0x7fd840152ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:15.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.355+0000 7fd83e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd828003c20 0x7fd840152ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50588/0 (socket says 172.21.15.31:50588) 2023-10-06T02:27:15.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.355+0000 7fd83e59c700 1 -- 172.21.15.31:0/4191163445 learned_addr learned my addr 172.21.15.31:0/4191163445 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:15.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.355+0000 7fd83e59c700 1 -- 172.21.15.31:0/4191163445 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd8400b1210 msgr2=0x7fd8400b1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:15.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.355+0000 7fd83e59c700 1 --2- 172.21.15.31:0/4191163445 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd8400b1210 0x7fd8400b1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:15.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.355+0000 7fd83e59c700 1 -- 172.21.15.31:0/4191163445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8400c8db0 msgr2=0x7fd8400c9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:15.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.355+0000 7fd83e59c700 1 --2- 172.21.15.31:0/4191163445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8400c8db0 0x7fd8400c9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:15.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.355+0000 7fd83e59c700 1 -- 172.21.15.31:0/4191163445 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8400c9870 con 0x7fd828003c20 2023-10-06T02:27:15.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.356+0000 7fd83e59c700 1 --2- 172.21.15.31:0/4191163445 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd828003c20 0x7fd840152ff0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fd82800bbb0 tx=0x7fd830015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:15.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.356+0000 7fd82e7fc700 1 -- 172.21.15.31:0/4191163445 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd830001360 con 0x7fd828003c20 2023-10-06T02:27:15.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.356+0000 7fd82e7fc700 1 -- 172.21.15.31:0/4191163445 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd83000ce40 con 0x7fd828003c20 2023-10-06T02:27:15.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.356+0000 7fd83f59e700 1 -- 172.21.15.31:0/4191163445 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8400c9b50 con 0x7fd828003c20 2023-10-06T02:27:15.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.356+0000 7fd82e7fc700 1 -- 172.21.15.31:0/4191163445 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd830006820 con 0x7fd828003c20 2023-10-06T02:27:15.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.356+0000 7fd83f59e700 1 -- 172.21.15.31:0/4191163445 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8400ca110 con 0x7fd828003c20 2023-10-06T02:27:15.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.357+0000 7fd83f59e700 1 -- 172.21.15.31:0/4191163445 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd84004c3c0 con 0x7fd828003c20 2023-10-06T02:27:15.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.358+0000 7fd82e7fc700 1 -- 172.21.15.31:0/4191163445 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fd830019070 con 0x7fd828003c20 2023-10-06T02:27:15.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.359+0000 7fd82e7fc700 1 --2- 172.21.15.31:0/4191163445 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fd8200672c0 0x7fd820069770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:15.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.359+0000 7fd82e7fc700 1 -- 172.21.15.31:0/4191163445 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5292+0+0 (secure 0 0 0) 0x7fd830085dc0 con 0x7fd828003c20 2023-10-06T02:27:15.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.359+0000 7fd83dd9b700 1 --2- 172.21.15.31:0/4191163445 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fd8200672c0 0x7fd820069770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:15.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.360+0000 7fd83dd9b700 1 --2- 172.21.15.31:0/4191163445 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fd8200672c0 0x7fd820069770 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fd834006f00 tx=0x7fd834008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:15.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.364+0000 7fd82e7fc700 1 -- 172.21.15.31:0/4191163445 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd830052dc0 con 0x7fd828003c20 2023-10-06T02:27:15.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.612+0000 7fd83f59e700 1 -- 172.21.15.31:0/4191163445 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd8400caef0 con 0x7fd828003c20 2023-10-06T02:27:15.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.612+0000 7fd82e7fc700 1 -- 172.21.15.31:0/4191163445 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+3523 (secure 0 0 0) 0x7fd83002cde0 con 0x7fd828003c20 2023-10-06T02:27:15.614 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:27:15.614 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":40,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5216,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/2282788921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2282788921},{"type":"v1","addr":"172.21.15.62:6833","nonce":2282788921}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5219,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/289861443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":289861443},{"type":"v1","addr":"172.21.15.62:6835","nonce":289861443}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":5889,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/2954679985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":2954679985},{"type":"v1","addr":"172.21.15.31:6836","nonce":2954679985}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30},{"gid":6219,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/577697658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":577697658},{"type":"v1","addr":"172.21.15.31:6837","nonce":577697658}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40}],"filesystems":[]} 2023-10-06T02:27:15.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.614+0000 7fd83f59e700 1 -- 172.21.15.31:0/4191163445 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fd8200672c0 msgr2=0x7fd820069770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:15.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.614+0000 7fd83f59e700 1 --2- 172.21.15.31:0/4191163445 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fd8200672c0 0x7fd820069770 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fd834006f00 tx=0x7fd834008040 comp rx=0 tx=0).stop 2023-10-06T02:27:15.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.614+0000 7fd83f59e700 1 -- 172.21.15.31:0/4191163445 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd828003c20 msgr2=0x7fd840152ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:15.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.614+0000 7fd83f59e700 1 --2- 172.21.15.31:0/4191163445 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd828003c20 0x7fd840152ff0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fd82800bbb0 tx=0x7fd830015000 comp rx=0 tx=0).stop 2023-10-06T02:27:15.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.615+0000 7fd83f59e700 1 -- 172.21.15.31:0/4191163445 shutdown_connections 2023-10-06T02:27:15.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.615+0000 7fd83f59e700 1 --2- 172.21.15.31:0/4191163445 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fd8200672c0 0x7fd820069770 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:15.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.615+0000 7fd83f59e700 1 --2- 172.21.15.31:0/4191163445 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd8400b1210 0x7fd8400b1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:15.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.615+0000 7fd83f59e700 1 --2- 172.21.15.31:0/4191163445 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd828003c20 0x7fd840152ff0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:15.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.615+0000 7fd83f59e700 1 --2- 172.21.15.31:0/4191163445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8400c8db0 0x7fd8400c9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:15.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.615+0000 7fd83f59e700 1 -- 172.21.15.31:0/4191163445 >> 172.21.15.31:0/4191163445 conn(0x7fd8400703a0 msgr2=0x7fd8400d0960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:15.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.615+0000 7fd83f59e700 1 -- 172.21.15.31:0/4191163445 shutdown_connections 2023-10-06T02:27:15.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.615+0000 7fd83f59e700 1 -- 172.21.15.31:0/4191163445 wait complete. 2023-10-06T02:27:15.617 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 40 2023-10-06T02:27:15.634 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T02:27:15.634 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T02:27:15.634 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T02:27:15.634 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T02:27:15.634 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T02:27:15.634 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T02:27:15.635 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T02:27:15.635 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T02:27:15.635 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T02:27:15.635 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T02:27:15.635 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T02:27:15.635 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T02:27:15.635 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T02:27:15.635 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T02:27:15.636 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T02:27:15.636 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T02:27:15.648 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T02:27:15.648 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T02:27:15.649 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T02:27:15.649 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T02:27:15.654 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T02:27:15.654 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T02:27:15.655 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T02:27:15.655 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T02:27:15.693 INFO:tasks.ceph.mds.a:Started 2023-10-06T02:27:15.693 DEBUG:teuthology.parallel:result is None 2023-10-06T02:27:15.695 INFO:tasks.ceph.mds.c:Started 2023-10-06T02:27:15.696 DEBUG:teuthology.parallel:result is None 2023-10-06T02:27:15.699 INFO:tasks.ceph.mds.b:Started 2023-10-06T02:27:15.699 DEBUG:teuthology.parallel:result is None 2023-10-06T02:27:15.701 INFO:tasks.ceph.mds.d:Started 2023-10-06T02:27:15.702 DEBUG:teuthology.parallel:result is None 2023-10-06T02:27:15.702 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T02:27:15.829 INFO:tasks.ceph.mds.a.smithi031.stdout:starting mds.a at 2023-10-06T02:27:15.830 INFO:tasks.ceph.mds.c.smithi031.stdout:starting mds.c at 2023-10-06T02:27:15.842 INFO:tasks.ceph.mds.b.smithi062.stdout:starting mds.b at 2023-10-06T02:27:15.843 INFO:tasks.ceph.mds.d.smithi062.stdout:starting mds.d at 2023-10-06T02:27:15.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.897+0000 7f472e79d700 1 Processor -- start 2023-10-06T02:27:15.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.898+0000 7f472e79d700 1 -- start start 2023-10-06T02:27:15.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.899+0000 7f472e79d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4728140a70 0x7f47280c6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:15.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.899+0000 7f472e79d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4728140e40 0x7f47280c7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:15.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.899+0000 7f472e79d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47280c6d50 con 0x7f47280c8330 2023-10-06T02:27:15.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.899+0000 7f472e79d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47280c7030 con 0x7f4728140e40 2023-10-06T02:27:15.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.899+0000 7f472e79d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47280c7310 con 0x7f4728140a70 2023-10-06T02:27:15.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.899+0000 7f472cf9a700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4728140e40 0x7f47280c7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:15.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.899+0000 7f472d79b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4728140a70 0x7f47280c6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:15.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.899+0000 7f472cf9a700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4728140e40 0x7f47280c7df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:34506/0 (socket says 172.21.15.31:34506) 2023-10-06T02:27:15.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.899+0000 7f472cf9a700 1 -- 172.21.15.31:0/295342996 learned_addr learned my addr 172.21.15.31:0/295342996 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:15.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.900+0000 7f472cf9a700 1 -- 172.21.15.31:0/295342996 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4728140a70 msgr2=0x7f47280c6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:15.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.900+0000 7f472cf9a700 1 --2- 172.21.15.31:0/295342996 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4728140a70 0x7f47280c6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:15.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.900+0000 7f472cf9a700 1 -- 172.21.15.31:0/295342996 >> v1:172.21.15.62:6789/0 conn(0x7f47280c8330 legacy=0x7f47280aeef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:15.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.900+0000 7f472cf9a700 1 -- 172.21.15.31:0/295342996 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47280c75f0 con 0x7f4728140e40 2023-10-06T02:27:15.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.900+0000 7f472cf9a700 1 --2- 172.21.15.31:0/295342996 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4728140e40 0x7f47280c7df0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4718009990 tx=0x7f4718009220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b79cd6edd554b454 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:15.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.900+0000 7f471ffff700 1 -- 172.21.15.31:0/295342996 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4718009e80 con 0x7f4728140e40 2023-10-06T02:27:15.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.900+0000 7f471ffff700 1 -- 172.21.15.31:0/295342996 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4718005c10 con 0x7f4728140e40 2023-10-06T02:27:15.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.900+0000 7f471ffff700 1 -- 172.21.15.31:0/295342996 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f471800b650 con 0x7f4728140e40 2023-10-06T02:27:15.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.901+0000 7f472e79d700 1 -- 172.21.15.31:0/295342996 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4728140e40 msgr2=0x7f47280c7df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:15.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.901+0000 7f472e79d700 1 --2- 172.21.15.31:0/295342996 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4728140e40 0x7f47280c7df0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4718009990 tx=0x7f4718009220 comp rx=0 tx=0).stop 2023-10-06T02:27:15.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.901+0000 7f472e79d700 1 -- 172.21.15.31:0/295342996 shutdown_connections 2023-10-06T02:27:15.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.901+0000 7f472e79d700 1 --2- 172.21.15.31:0/295342996 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4728140a70 0x7f47280c6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:15.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.901+0000 7f472e79d700 1 --2- 172.21.15.31:0/295342996 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4728140e40 0x7f47280c7df0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:15.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.901+0000 7f472e79d700 1 -- 172.21.15.31:0/295342996 >> 172.21.15.31:0/295342996 conn(0x7f472806d170 msgr2=0x7f47280cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:15.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.901+0000 7f472e79d700 1 -- 172.21.15.31:0/295342996 shutdown_connections 2023-10-06T02:27:15.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.901+0000 7f472e79d700 1 -- 172.21.15.31:0/295342996 wait complete. 2023-10-06T02:27:15.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.903+0000 7f472e79d700 1 Processor -- start 2023-10-06T02:27:15.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.903+0000 7f472e79d700 1 -- start start 2023-10-06T02:27:15.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.904+0000 7f472e79d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47280c8330 0x7f47280addf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:15.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.904+0000 7f472e79d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4728140a70 0x7f47280b0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:15.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.904+0000 7f472e79d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4728140e40 0x7f47280b1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:15.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.904+0000 7f472e79d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f472816c1f0 con 0x7f4728140a70 2023-10-06T02:27:15.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.904+0000 7f472e79d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47280ae330 con 0x7f47280c8330 2023-10-06T02:27:15.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.904+0000 7f472e79d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f472816b5e0 con 0x7f4728140e40 2023-10-06T02:27:15.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.904+0000 7f472cf9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4728140a70 0x7f47280b0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:15.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.904+0000 7f472cf9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4728140a70 0x7f47280b0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50638/0 (socket says 172.21.15.31:50638) 2023-10-06T02:27:15.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.904+0000 7f472cf9a700 1 -- 172.21.15.31:0/1825744462 learned_addr learned my addr 172.21.15.31:0/1825744462 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:15.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.904+0000 7f472d79b700 1 --2- 172.21.15.31:0/1825744462 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47280c8330 0x7f47280addf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:15.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.904+0000 7f472df9c700 1 --2- 172.21.15.31:0/1825744462 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4728140e40 0x7f47280b1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:15.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.904+0000 7f472cf9a700 1 -- 172.21.15.31:0/1825744462 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4728140e40 msgr2=0x7f47280b1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:15.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.904+0000 7f472cf9a700 1 --2- 172.21.15.31:0/1825744462 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4728140e40 0x7f47280b1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:15.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.904+0000 7f472cf9a700 1 -- 172.21.15.31:0/1825744462 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47280c8330 msgr2=0x7f47280addf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:15.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.904+0000 7f472cf9a700 1 --2- 172.21.15.31:0/1825744462 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47280c8330 0x7f47280addf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:15.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.904+0000 7f472cf9a700 1 -- 172.21.15.31:0/1825744462 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4718009480 con 0x7f4728140a70 2023-10-06T02:27:15.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.905+0000 7f472cf9a700 1 --2- 172.21.15.31:0/1825744462 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4728140a70 0x7f47280b0b10 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f471800b3c0 tx=0x7f4718016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:15.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.905+0000 7f471d7fa700 1 -- 172.21.15.31:0/1825744462 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4718006250 con 0x7f4728140a70 2023-10-06T02:27:15.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.905+0000 7f471d7fa700 1 -- 172.21.15.31:0/1825744462 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4718020070 con 0x7f4728140a70 2023-10-06T02:27:15.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.905+0000 7f471d7fa700 1 -- 172.21.15.31:0/1825744462 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4718006620 con 0x7f4728140a70 2023-10-06T02:27:15.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.905+0000 7f472e79d700 1 -- 172.21.15.31:0/1825744462 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f472816b8c0 con 0x7f4728140a70 2023-10-06T02:27:15.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.905+0000 7f472e79d700 1 -- 172.21.15.31:0/1825744462 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f472816be80 con 0x7f4728140a70 2023-10-06T02:27:15.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.906+0000 7f472e79d700 1 -- 172.21.15.31:0/1825744462 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f472816c3a0 con 0x7f4728140a70 2023-10-06T02:27:15.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.907+0000 7f471d7fa700 1 -- 172.21.15.31:0/1825744462 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f4718022030 con 0x7f4728140a70 2023-10-06T02:27:15.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.908+0000 7f471d7fa700 1 --2- 172.21.15.31:0/1825744462 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f470c067290 0x7f470c069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:15.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.908+0000 7f471d7fa700 1 -- 172.21.15.31:0/1825744462 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f4718086400 con 0x7f4728140a70 2023-10-06T02:27:15.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.908+0000 7f472d79b700 1 --2- 172.21.15.31:0/1825744462 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f470c067290 0x7f470c069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:15.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.909+0000 7f472d79b700 1 --2- 172.21.15.31:0/1825744462 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f470c067290 0x7f470c069740 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f4724009d20 tx=0x7f47240091a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:15.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:15.912+0000 7f471d7fa700 1 -- 172.21.15.31:0/1825744462 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f472816c3a0 con 0x7f4728140a70 2023-10-06T02:27:16.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:16.136+0000 7f472e79d700 1 -- 172.21.15.31:0/1825744462 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f472804c3c0 con 0x7f4728140a70 2023-10-06T02:27:17.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.057+0000 7f471d7fa700 1 -- 172.21.15.31:0/1825744462 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v71) v1 ==== 97+0+0 (secure 0 0 0) 0x7f47180533c0 con 0x7f4728140a70 2023-10-06T02:27:17.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.060+0000 7f472e79d700 1 -- 172.21.15.31:0/1825744462 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f470c067290 msgr2=0x7f470c069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:17.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.060+0000 7f472e79d700 1 --2- 172.21.15.31:0/1825744462 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f470c067290 0x7f470c069740 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f4724009d20 tx=0x7f47240091a0 comp rx=0 tx=0).stop 2023-10-06T02:27:17.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.061+0000 7f472e79d700 1 -- 172.21.15.31:0/1825744462 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4728140a70 msgr2=0x7f47280b0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:17.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.061+0000 7f472e79d700 1 --2- 172.21.15.31:0/1825744462 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4728140a70 0x7f47280b0b10 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f471800b3c0 tx=0x7f4718016000 comp rx=0 tx=0).stop 2023-10-06T02:27:17.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.061+0000 7f472e79d700 1 -- 172.21.15.31:0/1825744462 shutdown_connections 2023-10-06T02:27:17.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.061+0000 7f472e79d700 1 --2- 172.21.15.31:0/1825744462 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f470c067290 0x7f470c069740 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:17.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.061+0000 7f472e79d700 1 --2- 172.21.15.31:0/1825744462 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4728140e40 0x7f47280b1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:17.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.061+0000 7f472e79d700 1 --2- 172.21.15.31:0/1825744462 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47280c8330 0x7f47280addf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:17.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.061+0000 7f472e79d700 1 --2- 172.21.15.31:0/1825744462 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4728140a70 0x7f47280b0b10 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:17.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.061+0000 7f472e79d700 1 -- 172.21.15.31:0/1825744462 >> 172.21.15.31:0/1825744462 conn(0x7f472806d170 msgr2=0x7f472813eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:17.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.061+0000 7f472e79d700 1 -- 172.21.15.31:0/1825744462 shutdown_connections 2023-10-06T02:27:17.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.061+0000 7f472e79d700 1 -- 172.21.15.31:0/1825744462 wait complete. 2023-10-06T02:27:17.064 INFO:teuthology.orchestra.run.smithi031.stderr: removed all blocklist entries 2023-10-06T02:27:17.083 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T02:27:17.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.275+0000 7fa871f9d700 1 Processor -- start 2023-10-06T02:27:17.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.275+0000 7fa871f9d700 1 -- start start 2023-10-06T02:27:17.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.276+0000 7fa871f9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa86c06e0a0 0x7fa86c0c6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:17.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.276+0000 7fa871f9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa86c150380 0x7fa86c06a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:17.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.276+0000 7fa871f9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa86c06a490 con 0x7fa86c06f3a0 2023-10-06T02:27:17.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.276+0000 7fa871f9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa86c0c7490 con 0x7fa86c150380 2023-10-06T02:27:17.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.276+0000 7fa871f9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa86c0c7770 con 0x7fa86c06e0a0 2023-10-06T02:27:17.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.277+0000 7fa87179c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa86c150380 0x7fa86c06a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:17.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.277+0000 7fa87179c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa86c150380 0x7fa86c06a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50640/0 (socket says 172.21.15.31:50640) 2023-10-06T02:27:17.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.277+0000 7fa87179c700 1 -- 172.21.15.31:0/3179652815 learned_addr learned my addr 172.21.15.31:0/3179652815 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:17.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.277+0000 7fa870f9b700 1 --2- 172.21.15.31:0/3179652815 >> v2:172.21.15.62:3300/0 conn(0x7fa86c06e0a0 0x7fa86c0c6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:17.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.277+0000 7fa87179c700 1 -- 172.21.15.31:0/3179652815 >> v2:172.21.15.62:3300/0 conn(0x7fa86c06e0a0 msgr2=0x7fa86c0c6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:17.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.277+0000 7fa87179c700 1 --2- 172.21.15.31:0/3179652815 >> v2:172.21.15.62:3300/0 conn(0x7fa86c06e0a0 0x7fa86c0c6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:17.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.277+0000 7fa87179c700 1 -- 172.21.15.31:0/3179652815 >> v1:172.21.15.62:6789/0 conn(0x7fa86c06f3a0 legacy=0x7fa86c14fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:17.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.277+0000 7fa87179c700 1 -- 172.21.15.31:0/3179652815 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa86c0c7a50 con 0x7fa86c150380 2023-10-06T02:27:17.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.277+0000 7fa87179c700 1 --2- 172.21.15.31:0/3179652815 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa86c150380 0x7fa86c06a920 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fa86c069230 tx=0x7fa864009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4af1595f2a80c5ee server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:17.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.278+0000 7fa86b7fe700 1 -- 172.21.15.31:0/3179652815 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86400b950 con 0x7fa86c150380 2023-10-06T02:27:17.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.278+0000 7fa86b7fe700 1 -- 172.21.15.31:0/3179652815 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa86400bab0 con 0x7fa86c150380 2023-10-06T02:27:17.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.278+0000 7fa86b7fe700 1 -- 172.21.15.31:0/3179652815 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86400be70 con 0x7fa86c150380 2023-10-06T02:27:17.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.278+0000 7fa871f9d700 1 -- 172.21.15.31:0/3179652815 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa86c150380 msgr2=0x7fa86c06a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:17.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.278+0000 7fa871f9d700 1 --2- 172.21.15.31:0/3179652815 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa86c150380 0x7fa86c06a920 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fa86c069230 tx=0x7fa864009840 comp rx=0 tx=0).stop 2023-10-06T02:27:17.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.278+0000 7fa871f9d700 1 -- 172.21.15.31:0/3179652815 shutdown_connections 2023-10-06T02:27:17.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.278+0000 7fa871f9d700 1 --2- 172.21.15.31:0/3179652815 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa86c150380 0x7fa86c06a920 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:17.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.278+0000 7fa871f9d700 1 --2- 172.21.15.31:0/3179652815 >> v2:172.21.15.62:3300/0 conn(0x7fa86c06e0a0 0x7fa86c0c6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:17.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.278+0000 7fa871f9d700 1 -- 172.21.15.31:0/3179652815 >> 172.21.15.31:0/3179652815 conn(0x7fa86c06d170 msgr2=0x7fa86c0cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:17.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.278+0000 7fa871f9d700 1 -- 172.21.15.31:0/3179652815 shutdown_connections 2023-10-06T02:27:17.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.278+0000 7fa871f9d700 1 -- 172.21.15.31:0/3179652815 wait complete. 2023-10-06T02:27:17.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.280+0000 7fa871f9d700 1 Processor -- start 2023-10-06T02:27:17.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.280+0000 7fa871f9d700 1 -- start start 2023-10-06T02:27:17.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.281+0000 7fa871f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa86c06e0a0 0x7fa86c158810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:17.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.281+0000 7fa871f9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa86c06f3a0 0x7fa86c0c2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:17.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.281+0000 7fa871f9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa86c150380 0x7fa86c0c2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:17.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.281+0000 7fa871f9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa86c1591d0 con 0x7fa86c150380 2023-10-06T02:27:17.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.281+0000 7fa871f9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa86c16d900 con 0x7fa86c06f3a0 2023-10-06T02:27:17.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.281+0000 7fa871f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa86c16cfd0 con 0x7fa86c06e0a0 2023-10-06T02:27:17.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.281+0000 7fa87179c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa86c150380 0x7fa86c0c2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:17.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.281+0000 7fa87179c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa86c150380 0x7fa86c0c2aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50648/0 (socket says 172.21.15.31:50648) 2023-10-06T02:27:17.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.281+0000 7fa87179c700 1 -- 172.21.15.31:0/2508142517 learned_addr learned my addr 172.21.15.31:0/2508142517 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:17.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.281+0000 7fa870f9b700 1 --2- 172.21.15.31:0/2508142517 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa86c06e0a0 0x7fa86c158810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:17.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.281+0000 7fa86bfff700 1 --2- 172.21.15.31:0/2508142517 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa86c06f3a0 0x7fa86c0c2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:17.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.282+0000 7fa87179c700 1 -- 172.21.15.31:0/2508142517 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa86c06e0a0 msgr2=0x7fa86c158810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:17.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.282+0000 7fa87179c700 1 --2- 172.21.15.31:0/2508142517 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa86c06e0a0 0x7fa86c158810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:17.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.282+0000 7fa87179c700 1 -- 172.21.15.31:0/2508142517 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa86c06f3a0 msgr2=0x7fa86c0c2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:17.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.282+0000 7fa87179c700 1 --2- 172.21.15.31:0/2508142517 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa86c06f3a0 0x7fa86c0c2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:17.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.282+0000 7fa87179c700 1 -- 172.21.15.31:0/2508142517 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa864009ae0 con 0x7fa86c150380 2023-10-06T02:27:17.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.282+0000 7fa87179c700 1 --2- 172.21.15.31:0/2508142517 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa86c150380 0x7fa86c0c2aa0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fa864014000 tx=0x7fa86401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:17.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.282+0000 7fa868ff9700 1 -- 172.21.15.31:0/2508142517 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86400b950 con 0x7fa86c150380 2023-10-06T02:27:17.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.282+0000 7fa868ff9700 1 -- 172.21.15.31:0/2508142517 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa864019640 con 0x7fa86c150380 2023-10-06T02:27:17.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.282+0000 7fa871f9d700 1 -- 172.21.15.31:0/2508142517 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa86c16d2b0 con 0x7fa86c150380 2023-10-06T02:27:17.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.282+0000 7fa868ff9700 1 -- 172.21.15.31:0/2508142517 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa864019a20 con 0x7fa86c150380 2023-10-06T02:27:17.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.282+0000 7fa871f9d700 1 -- 172.21.15.31:0/2508142517 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa86c158d50 con 0x7fa86c150380 2023-10-06T02:27:17.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.283+0000 7fa871f9d700 1 -- 172.21.15.31:0/2508142517 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa86c158f70 con 0x7fa86c150380 2023-10-06T02:27:17.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.284+0000 7fa868ff9700 1 -- 172.21.15.31:0/2508142517 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fa864003040 con 0x7fa86c150380 2023-10-06T02:27:17.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.285+0000 7fa868ff9700 1 --2- 172.21.15.31:0/2508142517 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa850067290 0x7fa850069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:17.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.285+0000 7fa868ff9700 1 -- 172.21.15.31:0/2508142517 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fa86400e070 con 0x7fa86c150380 2023-10-06T02:27:17.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.285+0000 7fa870f9b700 1 --2- 172.21.15.31:0/2508142517 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa850067290 0x7fa850069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:17.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.286+0000 7fa870f9b700 1 --2- 172.21.15.31:0/2508142517 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa850067290 0x7fa850069740 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fa86c0caee0 tx=0x7fa860008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:17.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.289+0000 7fa868ff9700 1 -- 172.21.15.31:0/2508142517 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa864053590 con 0x7fa86c150380 2023-10-06T02:27:17.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.566+0000 7fa871f9d700 1 -- 172.21.15.31:0/2508142517 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fa86c04c3c0 con 0x7fa86c150380 2023-10-06T02:27:17.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.567+0000 7fa868ff9700 1 -- 172.21.15.31:0/2508142517 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) v1 ==== 80+0+5632 (secure 0 0 0) 0x7fa864056bb0 con 0x7fa86c150380 2023-10-06T02:27:17.569 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:27:17.569 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T02:27:17.569 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_dump": [ 2023-10-06T02:27:17.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:17.570 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.a", 2023-10-06T02:27:17.570 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBLbx9lbjrOGxAAVoSOoDaZb1temL41F4Nong==", 2023-10-06T02:27:17.570 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:27:17.570 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:27:17.570 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T02:27:17.570 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T02:27:17.570 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:27:17.571 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:17.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:17.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:17.571 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.b", 2023-10-06T02:27:17.571 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBLbx9lFgsTMRAATTklzroiyQbTLiZc9j1BHQ==", 2023-10-06T02:27:17.571 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:27:17.571 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:27:17.572 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T02:27:17.572 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T02:27:17.572 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:27:17.572 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:17.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:17.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:17.572 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.c", 2023-10-06T02:27:17.572 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBLbx9lezVtJhAA8wK7+dJ1OmIc+iXJkexK0w==", 2023-10-06T02:27:17.573 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:27:17.573 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:27:17.573 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T02:27:17.573 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T02:27:17.573 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:27:17.573 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:17.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:17.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:17.574 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.d", 2023-10-06T02:27:17.574 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBMbx9l74gWABAApBu2nVv12rGA+OBfBnEc3A==", 2023-10-06T02:27:17.574 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:27:17.574 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:27:17.574 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T02:27:17.574 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T02:27:17.575 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:27:17.575 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:17.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:17.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:17.575 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.0", 2023-10-06T02:27:17.575 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBPbx9lgj6+MBAAXSUD0GCG+rO/9DQBZYt/8w==", 2023-10-06T02:27:17.576 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:27:17.576 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:27:17.576 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:27:17.576 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:27:17.576 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:17.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:17.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:17.576 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.1", 2023-10-06T02:27:17.576 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBSbx9lXhMTNBAAeEdPnMP71ZGb6k2Ddg5raQ==", 2023-10-06T02:27:17.577 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:27:17.577 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:27:17.577 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:27:17.577 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:27:17.577 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:17.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:17.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:17.577 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.2", 2023-10-06T02:27:17.577 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBVbx9l4Ly0NhAATYXYDAjYuP53zjqK0yMboA==", 2023-10-06T02:27:17.577 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:27:17.577 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:27:17.577 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:27:17.577 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:27:17.578 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:17.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:17.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:17.578 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.3", 2023-10-06T02:27:17.578 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBZbx9lzsi4ARAAbGnNV0QyL/7itzy827+chg==", 2023-10-06T02:27:17.578 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:27:17.578 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:27:17.578 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:27:17.578 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:27:17.578 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:17.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:17.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:17.579 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.4", 2023-10-06T02:27:17.579 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBfbx9lW4OSEhAAj7HnVh/yq12zzCaXd6BHCA==", 2023-10-06T02:27:17.579 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:27:17.579 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:27:17.579 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:27:17.579 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:27:17.579 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:17.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:17.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:17.579 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.5", 2023-10-06T02:27:17.579 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBibx9lBcaIFRAAV8wq8QjsCOkI9Pzf/E+9bA==", 2023-10-06T02:27:17.579 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:27:17.580 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:27:17.580 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:27:17.580 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:27:17.580 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:17.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:17.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:17.580 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.6", 2023-10-06T02:27:17.580 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBlbx9lA7S+FxAA38mlSAurT+DpwMzH53zFcw==", 2023-10-06T02:27:17.580 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:27:17.580 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:27:17.580 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:27:17.580 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:27:17.580 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:17.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:17.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:17.581 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.7", 2023-10-06T02:27:17.581 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBobx9l5RkdHhAAxn0Z8IAE/o9oe2LRp3O9gg==", 2023-10-06T02:27:17.581 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:27:17.581 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:27:17.581 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:27:17.581 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:27:17.581 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:17.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:17.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:17.581 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.0", 2023-10-06T02:27:17.581 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBMbx9lh44fCRAA7sMISvk1t7dEJg4ZcBiMcA==", 2023-10-06T02:27:17.582 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:27:17.582 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:27:17.582 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow r", 2023-10-06T02:27:17.582 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T02:27:17.582 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:17.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:17.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:17.582 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.1", 2023-10-06T02:27:17.583 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBMbx9lP0zqFhAAlov2y7ipSvS3AkOdQkhg6g==", 2023-10-06T02:27:17.583 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:27:17.583 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:27:17.583 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-10-06T02:27:17.583 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-10-06T02:27:17.583 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-10-06T02:27:17.583 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:17.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:17.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:17.584 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.2", 2023-10-06T02:27:17.584 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBMbx9lpxYAEBAACtiI1XgQXM8yGGcpFyNK1w==", 2023-10-06T02:27:17.584 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:27:17.584 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:27:17.584 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-10-06T02:27:17.584 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-10-06T02:27:17.585 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-10-06T02:27:17.585 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:17.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:17.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:17.585 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.3", 2023-10-06T02:27:17.585 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBMbx9l81/PHRAAYFOoaezu0pwE0NdtnlYfVA==", 2023-10-06T02:27:17.585 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:27:17.586 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:27:17.586 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-10-06T02:27:17.586 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-10-06T02:27:17.586 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-10-06T02:27:17.586 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:17.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:17.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:17.586 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.admin", 2023-10-06T02:27:17.586 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBKbx9liFl1JBAAFyyHYNLVx1/Ex5a2MKm8rA==", 2023-10-06T02:27:17.586 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:27:17.587 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-10-06T02:27:17.587 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T02:27:17.587 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T02:27:17.587 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:27:17.587 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:17.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:17.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:17.587 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.alice", 2023-10-06T02:27:17.587 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQB2cB9l3OHmCRAAmc2GdO1sB1j4/XfS9DAw4g==", 2023-10-06T02:27:17.587 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:27:17.587 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow rw path=/volumes/_nogroup/subvolume_0000000000626507", 2023-10-06T02:27:17.587 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow r", 2023-10-06T02:27:17.588 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rw pool=cephfs_data" 2023-10-06T02:27:17.588 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:17.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:17.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:17.588 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mgr.x", 2023-10-06T02:27:17.588 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBLbx9lTBbIFBAAX2wsioKL4Nx4qz/qwsCsNA==", 2023-10-06T02:27:17.588 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:27:17.588 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-10-06T02:27:17.588 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile mgr", 2023-10-06T02:27:17.588 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:27:17.589 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:17.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:17.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:17.589 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mgr.y", 2023-10-06T02:27:17.589 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBLbx9l4+oLDRAA5YVo3q6FI9e9W/rJYROIsg==", 2023-10-06T02:27:17.589 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:27:17.589 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-10-06T02:27:17.589 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile mgr", 2023-10-06T02:27:17.589 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:27:17.589 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:17.589 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:17.589 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-10-06T02:27:17.590 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T02:27:17.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.569+0000 7fa871f9d700 1 -- 172.21.15.31:0/2508142517 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa850067290 msgr2=0x7fa850069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:17.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.569+0000 7fa871f9d700 1 --2- 172.21.15.31:0/2508142517 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa850067290 0x7fa850069740 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fa86c0caee0 tx=0x7fa860008040 comp rx=0 tx=0).stop 2023-10-06T02:27:17.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.569+0000 7fa871f9d700 1 -- 172.21.15.31:0/2508142517 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa86c150380 msgr2=0x7fa86c0c2aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:17.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.569+0000 7fa871f9d700 1 --2- 172.21.15.31:0/2508142517 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa86c150380 0x7fa86c0c2aa0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fa864014000 tx=0x7fa86401a000 comp rx=0 tx=0).stop 2023-10-06T02:27:17.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.569+0000 7fa871f9d700 1 -- 172.21.15.31:0/2508142517 shutdown_connections 2023-10-06T02:27:17.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.569+0000 7fa871f9d700 1 --2- 172.21.15.31:0/2508142517 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa86c150380 0x7fa86c0c2aa0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:17.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.569+0000 7fa871f9d700 1 --2- 172.21.15.31:0/2508142517 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa850067290 0x7fa850069740 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:17.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.569+0000 7fa871f9d700 1 --2- 172.21.15.31:0/2508142517 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa86c06e0a0 0x7fa86c158810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:17.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.569+0000 7fa871f9d700 1 --2- 172.21.15.31:0/2508142517 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa86c06f3a0 0x7fa86c0c2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:17.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.569+0000 7fa871f9d700 1 -- 172.21.15.31:0/2508142517 >> 172.21.15.31:0/2508142517 conn(0x7fa86c06d170 msgr2=0x7fa86c06a820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:17.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.569+0000 7fa871f9d700 1 -- 172.21.15.31:0/2508142517 shutdown_connections 2023-10-06T02:27:17.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.569+0000 7fa871f9d700 1 -- 172.21.15.31:0/2508142517 wait complete. 2023-10-06T02:27:17.592 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.alice 2023-10-06T02:27:17.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.778+0000 7f258919d700 1 Processor -- start 2023-10-06T02:27:17.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.779+0000 7f258919d700 1 -- start start 2023-10-06T02:27:17.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.780+0000 7f258919d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2584142c00 0x7f258406b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:17.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.780+0000 7f258919d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f25840c7f90 0x7f258406c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:17.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.780+0000 7f258919d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25840ca480 con 0x7f258406bd20 2023-10-06T02:27:17.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.780+0000 7f258919d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f258415e240 con 0x7f2584142c00 2023-10-06T02:27:17.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.780+0000 7f258919d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f258406b350 con 0x7f25840c7f90 2023-10-06T02:27:17.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.780+0000 7f2583fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2584142c00 0x7f258406b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:17.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.780+0000 7f2583fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2584142c00 0x7f258406b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50660/0 (socket says 172.21.15.31:50660) 2023-10-06T02:27:17.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.780+0000 7f25837fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f258406bd20 0x7f25840c7880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:47778/0 (socket says 172.21.15.31:47778) 2023-10-06T02:27:17.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.780+0000 7f2583fff700 1 -- 172.21.15.31:0/2473352192 learned_addr learned my addr 172.21.15.31:0/2473352192 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:17.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.780+0000 7f258899c700 1 --2- 172.21.15.31:0/2473352192 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f25840c7f90 0x7f258406c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:17.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.781+0000 7f2583fff700 1 -- 172.21.15.31:0/2473352192 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f25840c7f90 msgr2=0x7f258406c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:17.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.781+0000 7f2583fff700 1 --2- 172.21.15.31:0/2473352192 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f25840c7f90 0x7f258406c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:17.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.781+0000 7f2583fff700 1 -- 172.21.15.31:0/2473352192 >> v1:172.21.15.31:6789/0 conn(0x7f258406bd20 legacy=0x7f25840c7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:17.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.781+0000 7f2583fff700 1 -- 172.21.15.31:0/2473352192 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f258415df60 con 0x7f2584142c00 2023-10-06T02:27:17.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.781+0000 7f2583fff700 1 --2- 172.21.15.31:0/2473352192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2584142c00 0x7f258406b7e0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f25740099b0 tx=0x7f2574009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=133d265eda5009d9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:17.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.781+0000 7f2582ffd700 1 -- 172.21.15.31:0/2473352192 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2574009ea0 con 0x7f2584142c00 2023-10-06T02:27:17.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.781+0000 7f2582ffd700 1 -- 172.21.15.31:0/2473352192 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f257400b860 con 0x7f2584142c00 2023-10-06T02:27:17.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.781+0000 7f2582ffd700 1 -- 172.21.15.31:0/2473352192 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257400bc00 con 0x7f2584142c00 2023-10-06T02:27:17.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.782+0000 7f258919d700 1 -- 172.21.15.31:0/2473352192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2584142c00 msgr2=0x7f258406b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:17.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.782+0000 7f258919d700 1 --2- 172.21.15.31:0/2473352192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2584142c00 0x7f258406b7e0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f25740099b0 tx=0x7f2574009250 comp rx=0 tx=0).stop 2023-10-06T02:27:17.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.782+0000 7f258919d700 1 -- 172.21.15.31:0/2473352192 shutdown_connections 2023-10-06T02:27:17.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.782+0000 7f258919d700 1 --2- 172.21.15.31:0/2473352192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2584142c00 0x7f258406b7e0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:17.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.782+0000 7f258919d700 1 --2- 172.21.15.31:0/2473352192 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f25840c7f90 0x7f258406c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:17.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.782+0000 7f258919d700 1 -- 172.21.15.31:0/2473352192 >> 172.21.15.31:0/2473352192 conn(0x7f25840675a0 msgr2=0x7f25840679a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:17.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.782+0000 7f258919d700 1 -- 172.21.15.31:0/2473352192 shutdown_connections 2023-10-06T02:27:17.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.782+0000 7f258919d700 1 -- 172.21.15.31:0/2473352192 wait complete. 2023-10-06T02:27:17.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.783+0000 7f258919d700 1 Processor -- start 2023-10-06T02:27:17.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.783+0000 7f258919d700 1 -- start start 2023-10-06T02:27:17.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.784+0000 7f258919d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f258406bd20 0x7f25840c52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:17.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.784+0000 7f258919d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25840c7f90 0x7f258413ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:17.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.784+0000 7f258919d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2584142c00 0x7f2584158fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:17.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.784+0000 7f258919d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25840c7450 con 0x7f25840c7f90 2023-10-06T02:27:17.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.784+0000 7f258919d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f258415c2d0 con 0x7f258406bd20 2023-10-06T02:27:17.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.784+0000 7f258919d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2584159520 con 0x7f2584142c00 2023-10-06T02:27:17.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.785+0000 7f25837fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25840c7f90 0x7f258413ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:17.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.785+0000 7f25837fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25840c7f90 0x7f258413ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50670/0 (socket says 172.21.15.31:50670) 2023-10-06T02:27:17.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.785+0000 7f25837fe700 1 -- 172.21.15.31:0/1296804931 learned_addr learned my addr 172.21.15.31:0/1296804931 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:17.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.785+0000 7f2583fff700 1 --2- 172.21.15.31:0/1296804931 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f258406bd20 0x7f25840c52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:17.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.785+0000 7f258899c700 1 --2- 172.21.15.31:0/1296804931 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2584142c00 0x7f2584158fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:17.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.785+0000 7f25837fe700 1 -- 172.21.15.31:0/1296804931 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2584142c00 msgr2=0x7f2584158fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:17.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.785+0000 7f25837fe700 1 --2- 172.21.15.31:0/1296804931 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2584142c00 0x7f2584158fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:17.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.785+0000 7f25837fe700 1 -- 172.21.15.31:0/1296804931 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f258406bd20 msgr2=0x7f25840c52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:17.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.785+0000 7f25837fe700 1 --2- 172.21.15.31:0/1296804931 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f258406bd20 0x7f25840c52c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:17.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.785+0000 7f25837fe700 1 -- 172.21.15.31:0/1296804931 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25740094f0 con 0x7f25840c7f90 2023-10-06T02:27:17.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.785+0000 7f25837fe700 1 --2- 172.21.15.31:0/1296804931 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25840c7f90 0x7f258413ff20 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f257800c790 tx=0x7f2578007f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:17.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.785+0000 7f256bfff700 1 -- 172.21.15.31:0/1296804931 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2578005a70 con 0x7f25840c7f90 2023-10-06T02:27:17.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.786+0000 7f256bfff700 1 -- 172.21.15.31:0/1296804931 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2578005bf0 con 0x7f25840c7f90 2023-10-06T02:27:17.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.786+0000 7f258919d700 1 -- 172.21.15.31:0/1296804931 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25840c6e90 con 0x7f25840c7f90 2023-10-06T02:27:17.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.786+0000 7f256bfff700 1 -- 172.21.15.31:0/1296804931 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25780070d0 con 0x7f25840c7f90 2023-10-06T02:27:17.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.786+0000 7f258919d700 1 -- 172.21.15.31:0/1296804931 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25840c68e0 con 0x7f25840c7f90 2023-10-06T02:27:17.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.787+0000 7f258919d700 1 -- 172.21.15.31:0/1296804931 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25840c6460 con 0x7f25840c7f90 2023-10-06T02:27:17.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.788+0000 7f256bfff700 1 -- 172.21.15.31:0/1296804931 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f2578007480 con 0x7f25840c7f90 2023-10-06T02:27:17.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.788+0000 7f256bfff700 1 --2- 172.21.15.31:0/1296804931 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f2570067290 0x7f2570069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:17.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.789+0000 7f256bfff700 1 -- 172.21.15.31:0/1296804931 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f2578083de0 con 0x7f25840c7f90 2023-10-06T02:27:17.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.789+0000 7f2583fff700 1 --2- 172.21.15.31:0/1296804931 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f2570067290 0x7f2570069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:17.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.789+0000 7f2583fff700 1 --2- 172.21.15.31:0/1296804931 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f2570067290 0x7f2570069740 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f25740099b0 tx=0x7f2574017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:17.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:17.792+0000 7f256bfff700 1 -- 172.21.15.31:0/1296804931 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f25840c6460 con 0x7f25840c7f90 2023-10-06T02:27:18.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.074+0000 7f258919d700 1 -- 172.21.15.31:0/1296804931 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.alice"} v 0) v1 -- 0x7f25840c6460 con 0x7f25840c7f90 2023-10-06T02:27:18.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.078+0000 7f256bfff700 1 -- 172.21.15.31:0/1296804931 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.alice"}]=0 v15) v1 ==== 82+0+0 (secure 0 0 0) 0x7f25840c6460 con 0x7f25840c7f90 2023-10-06T02:27:18.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.080+0000 7f258919d700 1 -- 172.21.15.31:0/1296804931 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f2570067290 msgr2=0x7f2570069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:18.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.080+0000 7f258919d700 1 --2- 172.21.15.31:0/1296804931 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f2570067290 0x7f2570069740 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f25740099b0 tx=0x7f2574017000 comp rx=0 tx=0).stop 2023-10-06T02:27:18.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.080+0000 7f258919d700 1 -- 172.21.15.31:0/1296804931 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25840c7f90 msgr2=0x7f258413ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:18.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.080+0000 7f258919d700 1 --2- 172.21.15.31:0/1296804931 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25840c7f90 0x7f258413ff20 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f257800c790 tx=0x7f2578007f40 comp rx=0 tx=0).stop 2023-10-06T02:27:18.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.081+0000 7f258919d700 1 -- 172.21.15.31:0/1296804931 shutdown_connections 2023-10-06T02:27:18.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.081+0000 7f258919d700 1 --2- 172.21.15.31:0/1296804931 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f2570067290 0x7f2570069740 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:18.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.081+0000 7f258919d700 1 --2- 172.21.15.31:0/1296804931 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2584142c00 0x7f2584158fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:18.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.081+0000 7f258919d700 1 --2- 172.21.15.31:0/1296804931 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f258406bd20 0x7f25840c52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:18.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.081+0000 7f258919d700 1 --2- 172.21.15.31:0/1296804931 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25840c7f90 0x7f258413ff20 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:18.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.081+0000 7f258919d700 1 -- 172.21.15.31:0/1296804931 >> 172.21.15.31:0/1296804931 conn(0x7f25840675a0 msgr2=0x7f25840c2400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:18.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.081+0000 7f258919d700 1 -- 172.21.15.31:0/1296804931 shutdown_connections 2023-10-06T02:27:18.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.081+0000 7f258919d700 1 -- 172.21.15.31:0/1296804931 wait complete. 2023-10-06T02:27:18.100 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:27:18.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.288+0000 7f2345b9d700 1 Processor -- start 2023-10-06T02:27:18.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.288+0000 7f2345b9d700 1 -- start start 2023-10-06T02:27:18.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.289+0000 7f2345b9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2340159550 0x7f23400ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:18.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.289+0000 7f2345b9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f234015dfd0 0x7f23400aedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:18.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.289+0000 7f2345b9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23400acc00 con 0x7f2340159920 2023-10-06T02:27:18.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.289+0000 7f2345b9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23400abff0 con 0x7f234015dfd0 2023-10-06T02:27:18.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.289+0000 7f2345b9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23400ac2d0 con 0x7f2340159550 2023-10-06T02:27:18.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.290+0000 7f233ffff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f234015dfd0 0x7f23400aedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:18.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.290+0000 7f233ffff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f234015dfd0 0x7f23400aedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38076/0 (socket says 172.21.15.31:38076) 2023-10-06T02:27:18.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.290+0000 7f233ffff700 1 -- 172.21.15.31:0/4258992203 learned_addr learned my addr 172.21.15.31:0/4258992203 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:18.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.290+0000 7f2344b9b700 1 --2- 172.21.15.31:0/4258992203 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2340159550 0x7f23400ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:18.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.290+0000 7f233ffff700 1 -- 172.21.15.31:0/4258992203 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2340159550 msgr2=0x7f23400ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:18.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.290+0000 7f233ffff700 1 --2- 172.21.15.31:0/4258992203 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2340159550 0x7f23400ceac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:18.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.290+0000 7f233ffff700 1 -- 172.21.15.31:0/4258992203 >> v1:172.21.15.31:6789/0 conn(0x7f2340159920 legacy=0x7f23400ad090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:18.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.290+0000 7f233ffff700 1 -- 172.21.15.31:0/4258992203 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23400ac5b0 con 0x7f234015dfd0 2023-10-06T02:27:18.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.290+0000 7f2344b9b700 1 --2- 172.21.15.31:0/4258992203 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2340159550 0x7f23400ceac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:27:18.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.290+0000 7f233ffff700 1 --2- 172.21.15.31:0/4258992203 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f234015dfd0 0x7f23400aedb0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f232c0099b0 tx=0x7f232c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eec7b0bdb52dd657 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:18.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.291+0000 7f233f7fe700 1 -- 172.21.15.31:0/4258992203 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f232c00b860 con 0x7f234015dfd0 2023-10-06T02:27:18.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.291+0000 7f233f7fe700 1 -- 172.21.15.31:0/4258992203 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f232c00b9c0 con 0x7f234015dfd0 2023-10-06T02:27:18.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.291+0000 7f233f7fe700 1 -- 172.21.15.31:0/4258992203 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f232c00bd60 con 0x7f234015dfd0 2023-10-06T02:27:18.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.291+0000 7f2345b9d700 1 -- 172.21.15.31:0/4258992203 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f234015dfd0 msgr2=0x7f23400aedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:18.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.291+0000 7f2345b9d700 1 --2- 172.21.15.31:0/4258992203 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f234015dfd0 0x7f23400aedb0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f232c0099b0 tx=0x7f232c009250 comp rx=0 tx=0).stop 2023-10-06T02:27:18.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.291+0000 7f2345b9d700 1 -- 172.21.15.31:0/4258992203 shutdown_connections 2023-10-06T02:27:18.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.291+0000 7f2345b9d700 1 --2- 172.21.15.31:0/4258992203 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2340159550 0x7f23400ceac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:18.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.291+0000 7f2345b9d700 1 --2- 172.21.15.31:0/4258992203 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f234015dfd0 0x7f23400aedb0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:18.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.291+0000 7f2345b9d700 1 -- 172.21.15.31:0/4258992203 >> 172.21.15.31:0/4258992203 conn(0x7f234006ce80 msgr2=0x7f23400ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:18.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.291+0000 7f2345b9d700 1 -- 172.21.15.31:0/4258992203 shutdown_connections 2023-10-06T02:27:18.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.291+0000 7f2345b9d700 1 -- 172.21.15.31:0/4258992203 wait complete. 2023-10-06T02:27:18.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.293+0000 7f2345b9d700 1 Processor -- start 2023-10-06T02:27:18.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.293+0000 7f2345b9d700 1 -- start start 2023-10-06T02:27:18.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.294+0000 7f2345b9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2340159550 0x7f2340149170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:18.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.294+0000 7f2345b9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2340159920 0x7f23400cdde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:18.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.294+0000 7f2345b9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f234015dfd0 0x7f23400aed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:18.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.294+0000 7f2345b9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23400cbf60 con 0x7f234015dfd0 2023-10-06T02:27:18.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.294+0000 7f2345b9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23401496b0 con 0x7f2340159550 2023-10-06T02:27:18.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.294+0000 7f2345b9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2340156940 con 0x7f2340159920 2023-10-06T02:27:18.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.294+0000 7f234539c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f234015dfd0 0x7f23400aed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:18.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.294+0000 7f234539c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f234015dfd0 0x7f23400aed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38092/0 (socket says 172.21.15.31:38092) 2023-10-06T02:27:18.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.294+0000 7f234539c700 1 -- 172.21.15.31:0/2193031043 learned_addr learned my addr 172.21.15.31:0/2193031043 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:18.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.294+0000 7f2344b9b700 1 --2- 172.21.15.31:0/2193031043 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2340159550 0x7f2340149170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:18.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.294+0000 7f233ffff700 1 --2- 172.21.15.31:0/2193031043 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2340159920 0x7f23400cdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:18.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.295+0000 7f234539c700 1 -- 172.21.15.31:0/2193031043 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2340159920 msgr2=0x7f23400cdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:18.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.295+0000 7f234539c700 1 --2- 172.21.15.31:0/2193031043 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2340159920 0x7f23400cdde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:18.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.295+0000 7f234539c700 1 -- 172.21.15.31:0/2193031043 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2340159550 msgr2=0x7f2340149170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:18.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.295+0000 7f234539c700 1 --2- 172.21.15.31:0/2193031043 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2340159550 0x7f2340149170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:18.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.295+0000 7f234539c700 1 -- 172.21.15.31:0/2193031043 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f232c0094f0 con 0x7f234015dfd0 2023-10-06T02:27:18.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.295+0000 7f233ffff700 1 --2- 172.21.15.31:0/2193031043 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2340159920 0x7f23400cdde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:27:18.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.295+0000 7f2344b9b700 1 --2- 172.21.15.31:0/2193031043 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2340159550 0x7f2340149170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:27:18.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.295+0000 7f234539c700 1 --2- 172.21.15.31:0/2193031043 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f234015dfd0 0x7f23400aed70 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f233800c910 tx=0x7f2338001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:18.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.295+0000 7f233cff9700 1 -- 172.21.15.31:0/2193031043 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f233800a7a0 con 0x7f234015dfd0 2023-10-06T02:27:18.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.295+0000 7f233cff9700 1 -- 172.21.15.31:0/2193031043 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f233800a920 con 0x7f234015dfd0 2023-10-06T02:27:18.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.295+0000 7f2345b9d700 1 -- 172.21.15.31:0/2193031043 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2340156010 con 0x7f234015dfd0 2023-10-06T02:27:18.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.295+0000 7f233cff9700 1 -- 172.21.15.31:0/2193031043 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f233800ace0 con 0x7f234015dfd0 2023-10-06T02:27:18.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.295+0000 7f2345b9d700 1 -- 172.21.15.31:0/2193031043 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23401565d0 con 0x7f234015dfd0 2023-10-06T02:27:18.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.296+0000 7f2345b9d700 1 -- 172.21.15.31:0/2193031043 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f234004c3c0 con 0x7f234015dfd0 2023-10-06T02:27:18.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.297+0000 7f233cff9700 1 -- 172.21.15.31:0/2193031043 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f2338003030 con 0x7f234015dfd0 2023-10-06T02:27:18.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.299+0000 7f233cff9700 1 --2- 172.21.15.31:0/2193031043 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f2324067290 0x7f2324069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:18.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.299+0000 7f233cff9700 1 -- 172.21.15.31:0/2193031043 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f2338084990 con 0x7f234015dfd0 2023-10-06T02:27:18.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.299+0000 7f2344b9b700 1 --2- 172.21.15.31:0/2193031043 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f2324067290 0x7f2324069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:18.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.300+0000 7f2344b9b700 1 --2- 172.21.15.31:0/2193031043 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f2324067290 0x7f2324069740 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f2334006f00 tx=0x7f2334008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:18.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.303+0000 7f233cff9700 1 -- 172.21.15.31:0/2193031043 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f23380518a0 con 0x7f234015dfd0 2023-10-06T02:27:18.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.506+0000 7f2345b9d700 1 -- 172.21.15.31:0/2193031043 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2340156a80 con 0x7f234015dfd0 2023-10-06T02:27:18.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.506+0000 7f233cff9700 1 -- 172.21.15.31:0/2193031043 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+13603 (secure 0 0 0) 0x7f2338054ec0 con 0x7f234015dfd0 2023-10-06T02:27:18.508 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:27:18.509 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":71,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:27:17.053639+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:27:18.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.509+0000 7f2345b9d700 1 -- 172.21.15.31:0/2193031043 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f2324067290 msgr2=0x7f2324069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:18.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.509+0000 7f2345b9d700 1 --2- 172.21.15.31:0/2193031043 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f2324067290 0x7f2324069740 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f2334006f00 tx=0x7f2334008040 comp rx=0 tx=0).stop 2023-10-06T02:27:18.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.509+0000 7f2345b9d700 1 -- 172.21.15.31:0/2193031043 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f234015dfd0 msgr2=0x7f23400aed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:18.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.509+0000 7f2345b9d700 1 --2- 172.21.15.31:0/2193031043 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f234015dfd0 0x7f23400aed70 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f233800c910 tx=0x7f2338001710 comp rx=0 tx=0).stop 2023-10-06T02:27:18.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.509+0000 7f2345b9d700 1 -- 172.21.15.31:0/2193031043 shutdown_connections 2023-10-06T02:27:18.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.509+0000 7f2345b9d700 1 --2- 172.21.15.31:0/2193031043 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f234015dfd0 0x7f23400aed70 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:18.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.509+0000 7f2345b9d700 1 --2- 172.21.15.31:0/2193031043 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2340159550 0x7f2340149170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:18.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.509+0000 7f2345b9d700 1 --2- 172.21.15.31:0/2193031043 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f2324067290 0x7f2324069740 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:18.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.509+0000 7f2345b9d700 1 --2- 172.21.15.31:0/2193031043 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2340159920 0x7f23400cdde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:18.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.509+0000 7f2345b9d700 1 -- 172.21.15.31:0/2193031043 >> 172.21.15.31:0/2193031043 conn(0x7f234006ce80 msgr2=0x7f2340158540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:18.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.509+0000 7f2345b9d700 1 -- 172.21.15.31:0/2193031043 shutdown_connections 2023-10-06T02:27:18.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.509+0000 7f2345b9d700 1 -- 172.21.15.31:0/2193031043 wait complete. 2023-10-06T02:27:18.528 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T02:22:53.183425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T02:23:00.046967+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-10-06T02:27:18.528 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T02:27:18.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.716+0000 7f8ca099d700 1 Processor -- start 2023-10-06T02:27:18.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.717+0000 7f8ca099d700 1 -- start start 2023-10-06T02:27:18.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.718+0000 7f8ca099d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8c9c0c7fb0 0x7f8c9c06abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:18.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.718+0000 7f8ca099d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c9c06b290 con 0x7f8c9c13f780 2023-10-06T02:27:18.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.718+0000 7f8ca099d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c9c06b570 con 0x7f8c9c0c7be0 2023-10-06T02:27:18.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.718+0000 7f8ca099d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c9c06bf10 con 0x7f8c9c0c7fb0 2023-10-06T02:27:18.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.718+0000 7f8c9bfff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f8c9c13f780 0x7f8c9c13fba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:59674/0 (socket says 172.21.15.31:59674) 2023-10-06T02:27:18.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.718+0000 7f8c9bfff700 1 -- 172.21.15.31:0/3527983118 learned_addr learned my addr 172.21.15.31:0/3527983118 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:27:18.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.719+0000 7f8c9affd700 1 --2- 172.21.15.31:0/3527983118 >> v2:172.21.15.62:3300/0 conn(0x7f8c9c0c7fb0 0x7f8c9c06abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:18.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.719+0000 7f8c9a7fc700 1 -- 172.21.15.31:0/3527983118 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4138813026 0 0) 0x7f8c9c06b290 con 0x7f8c9c13f780 2023-10-06T02:27:18.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.719+0000 7f8c9a7fc700 1 -- 172.21.15.31:0/3527983118 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c84003570 con 0x7f8c9c13f780 2023-10-06T02:27:18.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.719+0000 7f8c9a7fc700 1 -- 172.21.15.31:0/3527983118 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1399024435 0 0) 0x7f8c9c06b570 con 0x7f8c9c0c7be0 2023-10-06T02:27:18.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.719+0000 7f8c9a7fc700 1 -- 172.21.15.31:0/3527983118 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c9c06b290 con 0x7f8c9c0c7be0 2023-10-06T02:27:18.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.719+0000 7f8c9a7fc700 1 -- 172.21.15.31:0/3527983118 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2076689073 0 0) 0x7f8c84003570 con 0x7f8c9c13f780 2023-10-06T02:27:18.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.719+0000 7f8c9a7fc700 1 -- 172.21.15.31:0/3527983118 >> v2:172.21.15.62:3300/0 conn(0x7f8c9c0c7fb0 msgr2=0x7f8c9c06abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:18.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.719+0000 7f8c9a7fc700 1 --2- 172.21.15.31:0/3527983118 >> v2:172.21.15.62:3300/0 conn(0x7f8c9c0c7fb0 0x7f8c9c06abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:18.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.719+0000 7f8c9a7fc700 1 -- 172.21.15.31:0/3527983118 >> v1:172.21.15.62:6789/0 conn(0x7f8c9c0c7be0 legacy=0x7f8c9c15a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:18.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.719+0000 7f8c9a7fc700 1 -- 172.21.15.31:0/3527983118 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c9c06b850 con 0x7f8c9c13f780 2023-10-06T02:27:18.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.719+0000 7f8c9a7fc700 1 -- 172.21.15.31:0/3527983118 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7f8c940017f0 con 0x7f8c9c13f780 2023-10-06T02:27:18.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.719+0000 7f8c9a7fc700 1 -- 172.21.15.31:0/3527983118 >> v1:172.21.15.31:6789/0 conn(0x7f8c9c13f780 legacy=0x7f8c9c13fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:18.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.719+0000 7f8c9a7fc700 1 --2- 172.21.15.31:0/3527983118 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c84003bf0 0x7f8c84003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:18.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.719+0000 7f8c9a7fc700 1 --2- 172.21.15.31:0/3527983118 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c84004810 0x7f8c84004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:18.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.719+0000 7f8c9a7fc700 1 --2- 172.21.15.31:0/3527983118 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8c840051c0 0x7f8c8400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:18.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.719+0000 7f8c9a7fc700 1 -- 172.21.15.31:0/3527983118 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9c06bf10 con 0x7f8c84004810 2023-10-06T02:27:18.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.719+0000 7f8c9a7fc700 1 -- 172.21.15.31:0/3527983118 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c8400c0b0 con 0x7f8c84003bf0 2023-10-06T02:27:18.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.719+0000 7f8c9a7fc700 1 -- 172.21.15.31:0/3527983118 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c8400e480 con 0x7f8c840051c0 2023-10-06T02:27:18.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.720+0000 7f8c9affd700 1 --2- 172.21.15.31:0/3527983118 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c84004810 0x7f8c84004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:18.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.720+0000 7f8c9b7fe700 1 --2- 172.21.15.31:0/3527983118 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c84003bf0 0x7f8c84003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:18.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.720+0000 7f8c9bfff700 1 --2- 172.21.15.31:0/3527983118 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8c840051c0 0x7f8c8400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:18.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.720+0000 7f8c9affd700 1 -- 172.21.15.31:0/3527983118 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8c840051c0 msgr2=0x7f8c8400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:18.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.720+0000 7f8c9affd700 1 --2- 172.21.15.31:0/3527983118 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8c840051c0 0x7f8c8400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:18.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.720+0000 7f8c9affd700 1 -- 172.21.15.31:0/3527983118 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c84003bf0 msgr2=0x7f8c84003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:18.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.720+0000 7f8c9affd700 1 --2- 172.21.15.31:0/3527983118 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c84003bf0 0x7f8c84003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:18.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.720+0000 7f8c9affd700 1 -- 172.21.15.31:0/3527983118 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c84003570 con 0x7f8c84004810 2023-10-06T02:27:18.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.720+0000 7f8c9bfff700 1 -- 172.21.15.31:0/3527983118 reap_dead start 2023-10-06T02:27:18.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.720+0000 7f8c9affd700 1 --2- 172.21.15.31:0/3527983118 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c84004810 0x7f8c84004c80 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f8c90009c00 tx=0x7f8c900091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdf03e8d57785404 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:18.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.720+0000 7f8c9a7fc700 1 -- 172.21.15.31:0/3527983118 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c9000e710 con 0x7f8c84004810 2023-10-06T02:27:18.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.720+0000 7f8c9a7fc700 1 -- 172.21.15.31:0/3527983118 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c9000e870 con 0x7f8c84004810 2023-10-06T02:27:18.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.720+0000 7f8c9a7fc700 1 -- 172.21.15.31:0/3527983118 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c9000ec30 con 0x7f8c84004810 2023-10-06T02:27:18.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.721+0000 7f8ca099d700 1 -- 172.21.15.31:0/3527983118 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c84004810 msgr2=0x7f8c84004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:18.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.721+0000 7f8ca099d700 1 --2- 172.21.15.31:0/3527983118 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c84004810 0x7f8c84004c80 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f8c90009c00 tx=0x7f8c900091a0 comp rx=0 tx=0).stop 2023-10-06T02:27:18.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.721+0000 7f8ca099d700 1 -- 172.21.15.31:0/3527983118 shutdown_connections 2023-10-06T02:27:18.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.721+0000 7f8ca099d700 1 --2- 172.21.15.31:0/3527983118 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c84004810 0x7f8c84004c80 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:18.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.721+0000 7f8ca099d700 1 -- 172.21.15.31:0/3527983118 >> 172.21.15.31:0/3527983118 conn(0x7f8c9c06d110 msgr2=0x7f8c9c0cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:18.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.721+0000 7f8ca099d700 1 -- 172.21.15.31:0/3527983118 shutdown_connections 2023-10-06T02:27:18.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.721+0000 7f8ca099d700 1 -- 172.21.15.31:0/3527983118 wait complete. 2023-10-06T02:27:18.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.723+0000 7f8ca099d700 1 Processor -- start 2023-10-06T02:27:18.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.723+0000 7f8ca099d700 1 -- start start 2023-10-06T02:27:18.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.724+0000 7f8ca099d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8c84004810 0x7f8c9c13dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:18.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.724+0000 7f8ca099d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c9c1584a0 0x7f8c9c0b0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:18.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.724+0000 7f8ca099d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c9c0c51f0 0x7f8c9c0c5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:18.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.724+0000 7f8ca099d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9c1691f0 con 0x7f8c9c1584a0 2023-10-06T02:27:18.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.724+0000 7f8ca099d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9c0c5bb0 con 0x7f8c9c0c51f0 2023-10-06T02:27:18.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.724+0000 7f8ca099d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9c159240 con 0x7f8c84004810 2023-10-06T02:27:18.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.724+0000 7f8c9affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c9c1584a0 0x7f8c9c0b0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:18.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.724+0000 7f8c9affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c9c1584a0 0x7f8c9c0b0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38106/0 (socket says 172.21.15.31:38106) 2023-10-06T02:27:18.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.724+0000 7f8c9affd700 1 -- 172.21.15.31:0/2713065100 learned_addr learned my addr 172.21.15.31:0/2713065100 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:18.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.724+0000 7f8c9b7fe700 1 --2- 172.21.15.31:0/2713065100 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8c84004810 0x7f8c9c13dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:18.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.724+0000 7f8c9bfff700 1 --2- 172.21.15.31:0/2713065100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c9c0c51f0 0x7f8c9c0c5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:18.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.724+0000 7f8c9affd700 1 -- 172.21.15.31:0/2713065100 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8c84004810 msgr2=0x7f8c9c13dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:18.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.724+0000 7f8c9affd700 1 --2- 172.21.15.31:0/2713065100 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8c84004810 0x7f8c9c13dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:18.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.724+0000 7f8c9affd700 1 -- 172.21.15.31:0/2713065100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c9c0c51f0 msgr2=0x7f8c9c0c5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:18.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.724+0000 7f8c9affd700 1 --2- 172.21.15.31:0/2713065100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c9c0c51f0 0x7f8c9c0c5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:18.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.724+0000 7f8c9affd700 1 -- 172.21.15.31:0/2713065100 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c9c0b0da0 con 0x7f8c9c1584a0 2023-10-06T02:27:18.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.725+0000 7f8c9affd700 1 --2- 172.21.15.31:0/2713065100 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c9c1584a0 0x7f8c9c0b0860 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f8c9000e4b0 tx=0x7f8c9001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:18.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.725+0000 7f8c837fe700 1 -- 172.21.15.31:0/2713065100 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c90022070 con 0x7f8c9c1584a0 2023-10-06T02:27:18.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.725+0000 7f8c837fe700 1 -- 172.21.15.31:0/2713065100 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c9000e9e0 con 0x7f8c9c1584a0 2023-10-06T02:27:18.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.725+0000 7f8c837fe700 1 -- 172.21.15.31:0/2713065100 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c9000e710 con 0x7f8c9c1584a0 2023-10-06T02:27:18.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.725+0000 7f8ca099d700 1 -- 172.21.15.31:0/2713065100 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c9c0b1750 con 0x7f8c9c1584a0 2023-10-06T02:27:18.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.725+0000 7f8ca099d700 1 -- 172.21.15.31:0/2713065100 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c9c1589a0 con 0x7f8c9c1584a0 2023-10-06T02:27:18.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.726+0000 7f8ca099d700 1 -- 172.21.15.31:0/2713065100 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c9c04c3c0 con 0x7f8c9c1584a0 2023-10-06T02:27:18.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.727+0000 7f8c837fe700 1 -- 172.21.15.31:0/2713065100 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f8c9001c660 con 0x7f8c9c1584a0 2023-10-06T02:27:18.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.728+0000 7f8c837fe700 1 --2- 172.21.15.31:0/2713065100 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f8c88067290 0x7f8c88069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:18.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.728+0000 7f8c837fe700 1 -- 172.21.15.31:0/2713065100 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f8c90085f30 con 0x7f8c9c1584a0 2023-10-06T02:27:18.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.728+0000 7f8c9b7fe700 1 --2- 172.21.15.31:0/2713065100 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f8c88067290 0x7f8c88069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:18.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.729+0000 7f8c9b7fe700 1 --2- 172.21.15.31:0/2713065100 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f8c88067290 0x7f8c88069740 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c04d2d0 tx=0x7f8c8c007d30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:18.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.732+0000 7f8c837fe700 1 -- 172.21.15.31:0/2713065100 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8c90052f40 con 0x7f8c9c1584a0 2023-10-06T02:27:18.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.935+0000 7f8ca099d700 1 -- 172.21.15.31:0/2713065100 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8c9c06bb30 con 0x7f8c9c1584a0 2023-10-06T02:27:18.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.936+0000 7f8c837fe700 1 -- 172.21.15.31:0/2713065100 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v71) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8c900170a0 con 0x7f8c9c1584a0 2023-10-06T02:27:18.937 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-10-06T02:27:18.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.938+0000 7f8ca099d700 1 -- 172.21.15.31:0/2713065100 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f8c88067290 msgr2=0x7f8c88069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:18.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.938+0000 7f8ca099d700 1 --2- 172.21.15.31:0/2713065100 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f8c88067290 0x7f8c88069740 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c04d2d0 tx=0x7f8c8c007d30 comp rx=0 tx=0).stop 2023-10-06T02:27:18.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.938+0000 7f8ca099d700 1 -- 172.21.15.31:0/2713065100 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c9c1584a0 msgr2=0x7f8c9c0b0860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:18.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.938+0000 7f8ca099d700 1 --2- 172.21.15.31:0/2713065100 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c9c1584a0 0x7f8c9c0b0860 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f8c9000e4b0 tx=0x7f8c9001a000 comp rx=0 tx=0).stop 2023-10-06T02:27:18.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.938+0000 7f8ca099d700 1 -- 172.21.15.31:0/2713065100 shutdown_connections 2023-10-06T02:27:18.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.938+0000 7f8ca099d700 1 --2- 172.21.15.31:0/2713065100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c9c0c51f0 0x7f8c9c0c5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:18.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.938+0000 7f8ca099d700 1 --2- 172.21.15.31:0/2713065100 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f8c88067290 0x7f8c88069740 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:18.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.938+0000 7f8ca099d700 1 --2- 172.21.15.31:0/2713065100 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8c84004810 0x7f8c9c13dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:18.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.938+0000 7f8ca099d700 1 --2- 172.21.15.31:0/2713065100 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c9c1584a0 0x7f8c9c0b0860 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:18.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.938+0000 7f8ca099d700 1 -- 172.21.15.31:0/2713065100 >> 172.21.15.31:0/2713065100 conn(0x7f8c9c06d110 msgr2=0x7f8c9c0caf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:18.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.938+0000 7f8ca099d700 1 -- 172.21.15.31:0/2713065100 shutdown_connections 2023-10-06T02:27:18.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:18.938+0000 7f8ca099d700 1 -- 172.21.15.31:0/2713065100 wait complete. 2023-10-06T02:27:18.957 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T02:27:19.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.146+0000 7f048b59e700 1 Processor -- start 2023-10-06T02:27:19.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.146+0000 7f048b59e700 1 -- start start 2023-10-06T02:27:19.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.147+0000 7f048b59e700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f048c06db20 0x7f048c15a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:19.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.147+0000 7f048b59e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f048c0caec0 0x7f048c15afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:19.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.147+0000 7f048b59e700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f048c15ada0 con 0x7f048c153c90 2023-10-06T02:27:19.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.147+0000 7f048b59e700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f048c06ca80 con 0x7f048c0caec0 2023-10-06T02:27:19.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.147+0000 7f048b59e700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f048c06cd60 con 0x7f048c06db20 2023-10-06T02:27:19.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.148+0000 7f048ad9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f048c0caec0 0x7f048c15afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:19.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.148+0000 7f0489d9b700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f048c06db20 0x7f048c15a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:19.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.148+0000 7f048ad9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f048c0caec0 0x7f048c15afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38108/0 (socket says 172.21.15.31:38108) 2023-10-06T02:27:19.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.148+0000 7f048ad9d700 1 -- 172.21.15.31:0/4155051390 learned_addr learned my addr 172.21.15.31:0/4155051390 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:19.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.148+0000 7f048a59c700 1 --1- 172.21.15.31:0/4155051390 >> v1:172.21.15.62:6789/0 conn(0x7f048c153c90 0x7f048c0ca7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:38368/0 (socket says 172.21.15.31:38368) 2023-10-06T02:27:19.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.148+0000 7f048ad9d700 1 -- 172.21.15.31:0/4155051390 >> v2:172.21.15.62:3300/0 conn(0x7f048c06db20 msgr2=0x7f048c15a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:19.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.148+0000 7f048ad9d700 1 --2- 172.21.15.31:0/4155051390 >> v2:172.21.15.62:3300/0 conn(0x7f048c06db20 0x7f048c15a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:19.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.148+0000 7f048ad9d700 1 -- 172.21.15.31:0/4155051390 >> v1:172.21.15.62:6789/0 conn(0x7f048c153c90 legacy=0x7f048c0ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:19.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.148+0000 7f048ad9d700 1 -- 172.21.15.31:0/4155051390 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f048c06d040 con 0x7f048c0caec0 2023-10-06T02:27:19.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.148+0000 7f048ad9d700 1 --2- 172.21.15.31:0/4155051390 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f048c0caec0 0x7f048c15afb0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f048c0cc300 tx=0x7f0484009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ea5260fde38984e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:19.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.149+0000 7f048959a700 1 -- 172.21.15.31:0/4155051390 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f048400e070 con 0x7f048c0caec0 2023-10-06T02:27:19.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.149+0000 7f048959a700 1 -- 172.21.15.31:0/4155051390 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f048400ba20 con 0x7f048c0caec0 2023-10-06T02:27:19.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.149+0000 7f048959a700 1 -- 172.21.15.31:0/4155051390 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04840092e0 con 0x7f048c0caec0 2023-10-06T02:27:19.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.149+0000 7f048b59e700 1 -- 172.21.15.31:0/4155051390 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f048c0caec0 msgr2=0x7f048c15afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:19.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.149+0000 7f048b59e700 1 --2- 172.21.15.31:0/4155051390 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f048c0caec0 0x7f048c15afb0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f048c0cc300 tx=0x7f0484009840 comp rx=0 tx=0).stop 2023-10-06T02:27:19.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.149+0000 7f048b59e700 1 -- 172.21.15.31:0/4155051390 shutdown_connections 2023-10-06T02:27:19.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.149+0000 7f048b59e700 1 --2- 172.21.15.31:0/4155051390 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f048c0caec0 0x7f048c15afb0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:19.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.149+0000 7f048b59e700 1 --2- 172.21.15.31:0/4155051390 >> v2:172.21.15.62:3300/0 conn(0x7f048c06db20 0x7f048c15a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:19.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.149+0000 7f048b59e700 1 -- 172.21.15.31:0/4155051390 >> 172.21.15.31:0/4155051390 conn(0x7f048c06e720 msgr2=0x7f048c06eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:19.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.149+0000 7f048b59e700 1 -- 172.21.15.31:0/4155051390 shutdown_connections 2023-10-06T02:27:19.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.149+0000 7f048b59e700 1 -- 172.21.15.31:0/4155051390 wait complete. 2023-10-06T02:27:19.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.151+0000 7f048b59e700 1 Processor -- start 2023-10-06T02:27:19.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.151+0000 7f048b59e700 1 -- start start 2023-10-06T02:27:19.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.152+0000 7f048b59e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f048c06db20 0x7f048c1500e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:19.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.152+0000 7f048b59e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f048c0caec0 0x7f048c15e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:19.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.152+0000 7f048b59e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f048c153c90 0x7f048c0b1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:19.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.152+0000 7f048b59e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f048c15dca0 con 0x7f048c06db20 2023-10-06T02:27:19.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.152+0000 7f048b59e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f048c15d090 con 0x7f048c153c90 2023-10-06T02:27:19.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.152+0000 7f048b59e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f048c15d370 con 0x7f048c0caec0 2023-10-06T02:27:19.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.152+0000 7f048a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f048c06db20 0x7f048c1500e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:19.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.152+0000 7f048a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f048c06db20 0x7f048c1500e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38118/0 (socket says 172.21.15.31:38118) 2023-10-06T02:27:19.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.152+0000 7f048a59c700 1 -- 172.21.15.31:0/482807533 learned_addr learned my addr 172.21.15.31:0/482807533 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:19.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.152+0000 7f0489d9b700 1 --2- 172.21.15.31:0/482807533 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f048c0caec0 0x7f048c15e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:19.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.152+0000 7f048ad9d700 1 --2- 172.21.15.31:0/482807533 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f048c153c90 0x7f048c0b1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:19.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.153+0000 7f048a59c700 1 -- 172.21.15.31:0/482807533 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f048c0caec0 msgr2=0x7f048c15e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:19.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.153+0000 7f048a59c700 1 --2- 172.21.15.31:0/482807533 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f048c0caec0 0x7f048c15e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:19.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.153+0000 7f048a59c700 1 -- 172.21.15.31:0/482807533 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f048c153c90 msgr2=0x7f048c0b1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:19.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.153+0000 7f048a59c700 1 --2- 172.21.15.31:0/482807533 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f048c153c90 0x7f048c0b1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:19.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.153+0000 7f048a59c700 1 -- 172.21.15.31:0/482807533 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0484009ae0 con 0x7f048c06db20 2023-10-06T02:27:19.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.153+0000 7f0489d9b700 1 --2- 172.21.15.31:0/482807533 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f048c0caec0 0x7f048c15e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:27:19.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.153+0000 7f048a59c700 1 --2- 172.21.15.31:0/482807533 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f048c06db20 0x7f048c1500e0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f047c00c7c0 tx=0x7f047c007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:19.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.153+0000 7f047a7fc700 1 -- 172.21.15.31:0/482807533 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f047c005a70 con 0x7f048c06db20 2023-10-06T02:27:19.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.153+0000 7f047a7fc700 1 -- 172.21.15.31:0/482807533 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f047c005bf0 con 0x7f048c06db20 2023-10-06T02:27:19.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.153+0000 7f047a7fc700 1 -- 172.21.15.31:0/482807533 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f047c0070f0 con 0x7f048c06db20 2023-10-06T02:27:19.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.153+0000 7f048b59e700 1 -- 172.21.15.31:0/482807533 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f048c15d650 con 0x7f048c06db20 2023-10-06T02:27:19.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.153+0000 7f048b59e700 1 -- 172.21.15.31:0/482807533 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f048c0b1b50 con 0x7f048c06db20 2023-10-06T02:27:19.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.154+0000 7f048b59e700 1 -- 172.21.15.31:0/482807533 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f048c04c3c0 con 0x7f048c06db20 2023-10-06T02:27:19.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.155+0000 7f047a7fc700 1 -- 172.21.15.31:0/482807533 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f047c007460 con 0x7f048c06db20 2023-10-06T02:27:19.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.156+0000 7f047a7fc700 1 --2- 172.21.15.31:0/482807533 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f046c067290 0x7f046c069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:19.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.156+0000 7f047a7fc700 1 -- 172.21.15.31:0/482807533 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f047c084360 con 0x7f048c06db20 2023-10-06T02:27:19.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.156+0000 7f0489d9b700 1 --2- 172.21.15.31:0/482807533 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f046c067290 0x7f046c069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:19.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.157+0000 7f0489d9b700 1 --2- 172.21.15.31:0/482807533 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f046c067290 0x7f046c069740 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f048c06d4e0 tx=0x7f0480008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:19.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.160+0000 7f047a7fc700 1 -- 172.21.15.31:0/482807533 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f047c0512f0 con 0x7f048c06db20 2023-10-06T02:27:19.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.362+0000 7f048b59e700 1 -- 172.21.15.31:0/482807533 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f048c15ada0 con 0x7f048c06db20 2023-10-06T02:27:19.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.363+0000 7f047a7fc700 1 -- 172.21.15.31:0/482807533 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v71) v1 ==== 92+0+10 (secure 0 0 0) 0x7f048c15ada0 con 0x7f048c06db20 2023-10-06T02:27:19.364 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 8 2023-10-06T02:27:19.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.365+0000 7f048b59e700 1 -- 172.21.15.31:0/482807533 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f046c067290 msgr2=0x7f046c069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:19.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.365+0000 7f048b59e700 1 --2- 172.21.15.31:0/482807533 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f046c067290 0x7f046c069740 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f048c06d4e0 tx=0x7f0480008040 comp rx=0 tx=0).stop 2023-10-06T02:27:19.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.365+0000 7f048b59e700 1 -- 172.21.15.31:0/482807533 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f048c06db20 msgr2=0x7f048c1500e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:19.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.365+0000 7f048b59e700 1 --2- 172.21.15.31:0/482807533 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f048c06db20 0x7f048c1500e0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f047c00c7c0 tx=0x7f047c007f70 comp rx=0 tx=0).stop 2023-10-06T02:27:19.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.365+0000 7f048b59e700 1 -- 172.21.15.31:0/482807533 shutdown_connections 2023-10-06T02:27:19.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.365+0000 7f048b59e700 1 --2- 172.21.15.31:0/482807533 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f048c153c90 0x7f048c0b1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:19.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.365+0000 7f048b59e700 1 --2- 172.21.15.31:0/482807533 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f048c06db20 0x7f048c1500e0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:19.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.365+0000 7f048b59e700 1 --2- 172.21.15.31:0/482807533 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f046c067290 0x7f046c069740 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:19.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.365+0000 7f048b59e700 1 --2- 172.21.15.31:0/482807533 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f048c0caec0 0x7f048c15e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:19.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.365+0000 7f048b59e700 1 -- 172.21.15.31:0/482807533 >> 172.21.15.31:0/482807533 conn(0x7f048c06e720 msgr2=0x7f048c159360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:19.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.365+0000 7f048b59e700 1 -- 172.21.15.31:0/482807533 shutdown_connections 2023-10-06T02:27:19.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.365+0000 7f048b59e700 1 -- 172.21.15.31:0/482807533 wait complete. 2023-10-06T02:27:19.384 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T02:27:19.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.570+0000 7f6efbfff700 1 Processor -- start 2023-10-06T02:27:19.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.570+0000 7f6efbfff700 1 -- start start 2023-10-06T02:27:19.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.571+0000 7f6efbfff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f6efc142c00 0x7f6efc06b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:19.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.571+0000 7f6efbfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6efc0c7f90 0x7f6efc06c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:19.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.571+0000 7f6efbfff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6efc0ca480 con 0x7f6efc06bd20 2023-10-06T02:27:19.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.571+0000 7f6efbfff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6efc15e240 con 0x7f6efc0c7f90 2023-10-06T02:27:19.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.571+0000 7f6efbfff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6efc06b350 con 0x7f6efc142c00 2023-10-06T02:27:19.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.572+0000 7f6efb7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6efc0c7f90 0x7f6efc06c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:19.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.572+0000 7f6efb7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6efc0c7f90 0x7f6efc06c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38128/0 (socket says 172.21.15.31:38128) 2023-10-06T02:27:19.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.572+0000 7f6efb7fe700 1 -- 172.21.15.31:0/883511703 learned_addr learned my addr 172.21.15.31:0/883511703 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:19.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.572+0000 7f6efaffd700 1 --2- 172.21.15.31:0/883511703 >> v2:172.21.15.62:3300/0 conn(0x7f6efc142c00 0x7f6efc06b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:19.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.572+0000 7f6efb7fe700 1 -- 172.21.15.31:0/883511703 >> v2:172.21.15.62:3300/0 conn(0x7f6efc142c00 msgr2=0x7f6efc06b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:19.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.572+0000 7f6efb7fe700 1 --2- 172.21.15.31:0/883511703 >> v2:172.21.15.62:3300/0 conn(0x7f6efc142c00 0x7f6efc06b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:19.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.572+0000 7f6efb7fe700 1 -- 172.21.15.31:0/883511703 >> v1:172.21.15.31:6789/0 conn(0x7f6efc06bd20 legacy=0x7f6efc0c7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:19.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.572+0000 7f6efb7fe700 1 -- 172.21.15.31:0/883511703 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6efc15df60 con 0x7f6efc0c7f90 2023-10-06T02:27:19.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.572+0000 7f6efaffd700 1 --2- 172.21.15.31:0/883511703 >> v2:172.21.15.62:3300/0 conn(0x7f6efc142c00 0x7f6efc06b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:27:19.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.572+0000 7f6efb7fe700 1 --2- 172.21.15.31:0/883511703 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6efc0c7f90 0x7f6efc06c6a0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f6efc06f490 tx=0x7f6ef4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=225c7baf4206f43 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:19.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.573+0000 7f6ef9ffb700 1 -- 172.21.15.31:0/883511703 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef400e070 con 0x7f6efc0c7f90 2023-10-06T02:27:19.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.573+0000 7f6ef9ffb700 1 -- 172.21.15.31:0/883511703 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ef400ba50 con 0x7f6efc0c7f90 2023-10-06T02:27:19.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.573+0000 7f6ef9ffb700 1 -- 172.21.15.31:0/883511703 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef40092e0 con 0x7f6efc0c7f90 2023-10-06T02:27:19.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.573+0000 7f6efbfff700 1 -- 172.21.15.31:0/883511703 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6efc0c7f90 msgr2=0x7f6efc06c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:19.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.573+0000 7f6efbfff700 1 --2- 172.21.15.31:0/883511703 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6efc0c7f90 0x7f6efc06c6a0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f6efc06f490 tx=0x7f6ef4009840 comp rx=0 tx=0).stop 2023-10-06T02:27:19.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.573+0000 7f6efbfff700 1 -- 172.21.15.31:0/883511703 shutdown_connections 2023-10-06T02:27:19.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.573+0000 7f6efbfff700 1 --2- 172.21.15.31:0/883511703 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6efc0c7f90 0x7f6efc06c6a0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:19.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.573+0000 7f6efbfff700 1 --2- 172.21.15.31:0/883511703 >> v2:172.21.15.62:3300/0 conn(0x7f6efc142c00 0x7f6efc06b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:19.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.573+0000 7f6efbfff700 1 -- 172.21.15.31:0/883511703 >> 172.21.15.31:0/883511703 conn(0x7f6efc0675a0 msgr2=0x7f6efc0679a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:19.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.574+0000 7f6efbfff700 1 -- 172.21.15.31:0/883511703 shutdown_connections 2023-10-06T02:27:19.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.574+0000 7f6efbfff700 1 -- 172.21.15.31:0/883511703 wait complete. 2023-10-06T02:27:19.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.575+0000 7f6efbfff700 1 Processor -- start 2023-10-06T02:27:19.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.575+0000 7f6efbfff700 1 -- start start 2023-10-06T02:27:19.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.576+0000 7f6efbfff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6efc06bd20 0x7f6efc1407c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:19.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.576+0000 7f6efbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6efc0c7f90 0x7f6efc15ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:19.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.576+0000 7f6efbfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6efc142c00 0x7f6efc0c4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:19.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.576+0000 7f6efbfff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6efc0c5600 con 0x7f6efc0c7f90 2023-10-06T02:27:19.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.576+0000 7f6efbfff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6efc0b0a40 con 0x7f6efc142c00 2023-10-06T02:27:19.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.576+0000 7f6efbfff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6efc0b0110 con 0x7f6efc06bd20 2023-10-06T02:27:19.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.576+0000 7f6efa7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6efc0c7f90 0x7f6efc15ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:19.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.576+0000 7f6efa7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6efc0c7f90 0x7f6efc15ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38140/0 (socket says 172.21.15.31:38140) 2023-10-06T02:27:19.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.576+0000 7f6efa7fc700 1 -- 172.21.15.31:0/3360371149 learned_addr learned my addr 172.21.15.31:0/3360371149 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:19.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.576+0000 7f6efb7fe700 1 --2- 172.21.15.31:0/3360371149 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6efc142c00 0x7f6efc0c4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:19.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.576+0000 7f6efaffd700 1 --2- 172.21.15.31:0/3360371149 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6efc06bd20 0x7f6efc1407c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:19.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.577+0000 7f6efa7fc700 1 -- 172.21.15.31:0/3360371149 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6efc06bd20 msgr2=0x7f6efc1407c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:19.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.577+0000 7f6efa7fc700 1 --2- 172.21.15.31:0/3360371149 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6efc06bd20 0x7f6efc1407c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:19.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.577+0000 7f6efa7fc700 1 -- 172.21.15.31:0/3360371149 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6efc142c00 msgr2=0x7f6efc0c4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:19.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.577+0000 7f6efa7fc700 1 --2- 172.21.15.31:0/3360371149 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6efc142c00 0x7f6efc0c4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:19.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.577+0000 7f6efa7fc700 1 -- 172.21.15.31:0/3360371149 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ef4009ae0 con 0x7f6efc0c7f90 2023-10-06T02:27:19.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.577+0000 7f6efa7fc700 1 --2- 172.21.15.31:0/3360371149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6efc0c7f90 0x7f6efc15ac40 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f6ef000c7f0 tx=0x7f6ef0007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:19.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.577+0000 7f6ee2ffd700 1 -- 172.21.15.31:0/3360371149 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef0011590 con 0x7f6efc0c7f90 2023-10-06T02:27:19.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.577+0000 7f6ee2ffd700 1 -- 172.21.15.31:0/3360371149 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ef0011710 con 0x7f6efc0c7f90 2023-10-06T02:27:19.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.577+0000 7f6ee2ffd700 1 -- 172.21.15.31:0/3360371149 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef0011ad0 con 0x7f6efc0c7f90 2023-10-06T02:27:19.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.577+0000 7f6efbfff700 1 -- 172.21.15.31:0/3360371149 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6efc0b03f0 con 0x7f6efc0c7f90 2023-10-06T02:27:19.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.577+0000 7f6efbfff700 1 -- 172.21.15.31:0/3360371149 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6efc140d00 con 0x7f6efc0c7f90 2023-10-06T02:27:19.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.578+0000 7f6efbfff700 1 -- 172.21.15.31:0/3360371149 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6efc04c3c0 con 0x7f6efc0c7f90 2023-10-06T02:27:19.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.579+0000 7f6ee2ffd700 1 -- 172.21.15.31:0/3360371149 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f6ef0012420 con 0x7f6efc0c7f90 2023-10-06T02:27:19.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.580+0000 7f6ee2ffd700 1 --2- 172.21.15.31:0/3360371149 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6ee8067290 0x7f6ee8069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:19.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.580+0000 7f6ee2ffd700 1 -- 172.21.15.31:0/3360371149 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f6ef0083b40 con 0x7f6efc0c7f90 2023-10-06T02:27:19.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.581+0000 7f6efaffd700 1 --2- 172.21.15.31:0/3360371149 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6ee8067290 0x7f6ee8069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:19.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.581+0000 7f6efaffd700 1 --2- 172.21.15.31:0/3360371149 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6ee8067290 0x7f6ee8069740 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f6eec006f00 tx=0x7f6eec008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:19.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.584+0000 7f6ee2ffd700 1 -- 172.21.15.31:0/3360371149 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6ef004b5a0 con 0x7f6efc0c7f90 2023-10-06T02:27:19.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.788+0000 7f6efbfff700 1 -- 172.21.15.31:0/3360371149 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f6efc0b0b80 con 0x7f6efc0c7f90 2023-10-06T02:27:19.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.789+0000 7f6ee2ffd700 1 -- 172.21.15.31:0/3360371149 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v71) v1 ==== 84+0+126 (secure 0 0 0) 0x7f6ef0017020 con 0x7f6efc0c7f90 2023-10-06T02:27:19.791 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:27:19.791 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2023-10-06T02:27:19.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:19.791 INFO:teuthology.orchestra.run.smithi031.stdout: "poolnum": 1, 2023-10-06T02:27:19.791 INFO:teuthology.orchestra.run.smithi031.stdout: "poolname": ".mgr" 2023-10-06T02:27:19.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:19.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:19.791 INFO:teuthology.orchestra.run.smithi031.stdout: "poolnum": 2, 2023-10-06T02:27:19.791 INFO:teuthology.orchestra.run.smithi031.stdout: "poolname": "rbd" 2023-10-06T02:27:19.791 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:19.792 INFO:teuthology.orchestra.run.smithi031.stdout:] 2023-10-06T02:27:19.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.791+0000 7f6efbfff700 1 -- 172.21.15.31:0/3360371149 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6ee8067290 msgr2=0x7f6ee8069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:19.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.791+0000 7f6efbfff700 1 --2- 172.21.15.31:0/3360371149 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6ee8067290 0x7f6ee8069740 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f6eec006f00 tx=0x7f6eec008040 comp rx=0 tx=0).stop 2023-10-06T02:27:19.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.791+0000 7f6efbfff700 1 -- 172.21.15.31:0/3360371149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6efc0c7f90 msgr2=0x7f6efc15ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:19.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.791+0000 7f6efbfff700 1 --2- 172.21.15.31:0/3360371149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6efc0c7f90 0x7f6efc15ac40 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f6ef000c7f0 tx=0x7f6ef0007f50 comp rx=0 tx=0).stop 2023-10-06T02:27:19.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.791+0000 7f6efbfff700 1 -- 172.21.15.31:0/3360371149 shutdown_connections 2023-10-06T02:27:19.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.791+0000 7f6efbfff700 1 --2- 172.21.15.31:0/3360371149 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6efc142c00 0x7f6efc0c4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:19.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.791+0000 7f6efbfff700 1 --2- 172.21.15.31:0/3360371149 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6ee8067290 0x7f6ee8069740 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:19.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.791+0000 7f6efbfff700 1 --2- 172.21.15.31:0/3360371149 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6efc06bd20 0x7f6efc1407c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:19.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.791+0000 7f6efbfff700 1 --2- 172.21.15.31:0/3360371149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6efc0c7f90 0x7f6efc15ac40 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:19.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.792+0000 7f6efbfff700 1 -- 172.21.15.31:0/3360371149 >> 172.21.15.31:0/3360371149 conn(0x7f6efc0675a0 msgr2=0x7f6efc0c94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:19.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.792+0000 7f6efbfff700 1 -- 172.21.15.31:0/3360371149 shutdown_connections 2023-10-06T02:27:19.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:19.792+0000 7f6efbfff700 1 -- 172.21.15.31:0/3360371149 wait complete. 2023-10-06T02:27:19.810 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T02:27:19.811 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T02:27:20.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:20.000+0000 7f55ccf9d700 1 Processor -- start 2023-10-06T02:27:20.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:20.000+0000 7f55ccf9d700 1 -- start start 2023-10-06T02:27:20.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:20.001+0000 7f55ccf9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55c8142ca0 0x7f55c815dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:20.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:20.001+0000 7f55ccf9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f55c80cb450 0x7f55c80cbf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:20.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:20.001+0000 7f55ccf9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55c815d8d0 con 0x7f55c8143070 2023-10-06T02:27:20.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:20.001+0000 7f55ccf9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55c815d2f0 con 0x7f55c80cb450 2023-10-06T02:27:20.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:20.001+0000 7f55ccf9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55c80cc630 con 0x7f55c8142ca0 2023-10-06T02:27:20.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:20.002+0000 7f55c6ffd700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f55c8143070 0x7f55c80cad40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:59680/0 (socket says 172.21.15.31:59680) 2023-10-06T02:27:20.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:20.002+0000 7f55c6ffd700 1 -- 172.21.15.31:0/594136780 learned_addr learned my addr 172.21.15.31:0/594136780 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:27:20.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:20.002+0000 7f55c7fff700 1 --2- 172.21.15.31:0/594136780 >> v2:172.21.15.62:3300/0 conn(0x7f55c80cb450 0x7f55c80cbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:20.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:20.002+0000 7f55c77fe700 1 --2- 172.21.15.31:0/594136780 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55c8142ca0 0x7f55c815dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:20.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:20.002+0000 7f55c67fc700 1 -- 172.21.15.31:0/594136780 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3084349028 0 0) 0x7f55c815d8d0 con 0x7f55c8143070 2023-10-06T02:27:20.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:20.002+0000 7f55c67fc700 1 -- 172.21.15.31:0/594136780 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55b0003570 con 0x7f55c8143070 2023-10-06T02:27:20.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:20.002+0000 7f55c67fc700 1 -- 172.21.15.31:0/594136780 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1477927739 0 0) 0x7f55b0003570 con 0x7f55c8143070 2023-10-06T02:27:20.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:20.002+0000 7f55c7fff700 1 -- 172.21.15.31:0/594136780 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55c8142ca0 msgr2=0x7f55c815dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:20.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:20.002+0000 7f55c7fff700 1 --2- 172.21.15.31:0/594136780 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55c8142ca0 0x7f55c815dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:20.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:20.002+0000 7f55c7fff700 1 -- 172.21.15.31:0/594136780 >> v1:172.21.15.31:6789/0 conn(0x7f55c8143070 legacy=0x7f55c80cad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:20.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:20.002+0000 7f55c7fff700 1 -- 172.21.15.31:0/594136780 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55c815d560 con 0x7f55c80cb450 2023-10-06T02:27:20.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:20.003+0000 7f55c7fff700 1 --2- 172.21.15.31:0/594136780 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55c80cb450 0x7f55c80cbf30 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f55c0009e60 tx=0x7f55c0009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d979e764bea901c3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:20.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:20.003+0000 7f55c67fc700 1 -- 172.21.15.31:0/594136780 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c000e070 con 0x7f55c80cb450 2023-10-06T02:27:20.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:20.003+0000 7f55c67fc700 1 -- 172.21.15.31:0/594136780 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55c000b7a0 con 0x7f55c80cb450 2023-10-06T02:27:20.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:20.003+0000 7f55c67fc700 1 -- 172.21.15.31:0/594136780 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55bc001470 con 0x7f55c80cb450 2023-10-06T02:27:20.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:20.004+0000 7f55ccf9d700 1 -- 172.21.15.31:0/594136780 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55c80cb450 msgr2=0x7f55c80cbf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:20.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:20.004+0000 7f55ccf9d700 1 --2- 172.21.15.31:0/594136780 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55c80cb450 0x7f55c80cbf30 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f55c0009e60 tx=0x7f55c0009780 comp rx=0 tx=0).stop 2023-10-06T02:27:20.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:20.004+0000 7f55ccf9d700 1 -- 172.21.15.31:0/594136780 shutdown_connections 2023-10-06T02:27:20.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:20.004+0000 7f55ccf9d700 1 --2- 172.21.15.31:0/594136780 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55c80cb450 0x7f55c80cbf30 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:20.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:20.004+0000 7f55ccf9d700 1 --2- 172.21.15.31:0/594136780 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55c8142ca0 0x7f55c815dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:20.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:20.004+0000 7f55ccf9d700 1 -- 172.21.15.31:0/594136780 >> 172.21.15.31:0/594136780 conn(0x7f55c8067c80 msgr2=0x7f55c8068080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:20.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:20.004+0000 7f55ccf9d700 1 -- 172.21.15.31:0/594136780 shutdown_connections 2023-10-06T02:27:20.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:20.004+0000 7f55ccf9d700 1 -- 172.21.15.31:0/594136780 wait complete. 2023-10-06T02:27:20.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:20.005+0000 7f55ccf9d700 1 Processor -- start 2023-10-06T02:27:20.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:20.005+0000 7f55ccf9d700 1 -- start start 2023-10-06T02:27:20.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:20.006+0000 7f55ccf9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55c80cb450 0x7f55c80b39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:20.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:20.006+0000 7f55ccf9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55c8142ca0 0x7f55c8141cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:20.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:20.006+0000 7f55ccf9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55c8143070 0x7f55c813eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:20.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:20.006+0000 7f55ccf9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c80b1190 con 0x7f55c8142ca0 2023-10-06T02:27:20.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:20.006+0000 7f55ccf9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c816d060 con 0x7f55c80cb450 2023-10-06T02:27:20.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:20.006+0000 7f55ccf9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55c816d340 con 0x7f55c8143070 2023-10-06T02:27:20.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:20.006+0000 7f55c6ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55c8142ca0 0x7f55c8141cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:20.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:20.006+0000 7f55c6ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55c8142ca0 0x7f55c8141cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38154/0 (socket says 172.21.15.31:38154) 2023-10-06T02:27:20.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:20.006+0000 7f55c6ffd700 1 -- 172.21.15.31:0/1565426094 learned_addr learned my addr 172.21.15.31:0/1565426094 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:20.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:20.007+0000 7f55c77fe700 1 --2- 172.21.15.31:0/1565426094 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55c80cb450 0x7f55c80b39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:20.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:20.007+0000 7f55c7fff700 1 --2- 172.21.15.31:0/1565426094 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55c8143070 0x7f55c813eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:20.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:20.007+0000 7f55c6ffd700 1 -- 172.21.15.31:0/1565426094 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55c8143070 msgr2=0x7f55c813eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:20.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:20.007+0000 7f55c6ffd700 1 --2- 172.21.15.31:0/1565426094 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55c8143070 0x7f55c813eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:20.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:20.007+0000 7f55c6ffd700 1 -- 172.21.15.31:0/1565426094 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55c80cb450 msgr2=0x7f55c80b39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:20.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:20.007+0000 7f55c6ffd700 1 --2- 172.21.15.31:0/1565426094 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55c80cb450 0x7f55c80b39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:20.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:20.007+0000 7f55c6ffd700 1 -- 172.21.15.31:0/1565426094 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55c0009a20 con 0x7f55c8142ca0 2023-10-06T02:27:20.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:20.007+0000 7f55c6ffd700 1 --2- 172.21.15.31:0/1565426094 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55c8142ca0 0x7f55c8141cd0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f55bc001660 tx=0x7f55bc0019e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:20.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:20.007+0000 7f55af7fe700 1 -- 172.21.15.31:0/1565426094 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55bc007360 con 0x7f55c8142ca0 2023-10-06T02:27:20.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:20.007+0000 7f55af7fe700 1 -- 172.21.15.31:0/1565426094 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55bc0074c0 con 0x7f55c8142ca0 2023-10-06T02:27:20.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:20.007+0000 7f55af7fe700 1 -- 172.21.15.31:0/1565426094 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55bc004830 con 0x7f55c8142ca0 2023-10-06T02:27:20.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:20.007+0000 7f55ccf9d700 1 -- 172.21.15.31:0/1565426094 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55c816d620 con 0x7f55c8142ca0 2023-10-06T02:27:20.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:20.008+0000 7f55ccf9d700 1 -- 172.21.15.31:0/1565426094 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55c80b3f00 con 0x7f55c8142ca0 2023-10-06T02:27:20.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:20.008+0000 7f55ccf9d700 1 -- 172.21.15.31:0/1565426094 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55c80b1300 con 0x7f55c8142ca0 2023-10-06T02:27:20.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:20.009+0000 7f55af7fe700 1 -- 172.21.15.31:0/1565426094 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f55bc004be0 con 0x7f55c8142ca0 2023-10-06T02:27:20.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:20.010+0000 7f55af7fe700 1 --2- 172.21.15.31:0/1565426094 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f55b4067290 0x7f55b4069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:20.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:20.010+0000 7f55af7fe700 1 -- 172.21.15.31:0/1565426094 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f55bc084840 con 0x7f55c8142ca0 2023-10-06T02:27:20.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:20.010+0000 7f55c77fe700 1 --2- 172.21.15.31:0/1565426094 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f55b4067290 0x7f55b4069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:20.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:20.011+0000 7f55c77fe700 1 --2- 172.21.15.31:0/1565426094 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f55b4067290 0x7f55b4069740 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f55b8009db0 tx=0x7f55b80091a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:20.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:20.014+0000 7f55af7fe700 1 -- 172.21.15.31:0/1565426094 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f55bc0517d0 con 0x7f55c8142ca0 2023-10-06T02:27:20.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:20.217+0000 7f55ccf9d700 1 -- 172.21.15.31:0/1565426094 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f55c804c3c0 con 0x7f55c8142ca0 2023-10-06T02:27:21.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:21.033+0000 7f55af7fe700 1 -- 172.21.15.31:0/1565426094 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v72) v1 ==== 138+0+0 (secure 0 0 0) 0x7f55bc054df0 con 0x7f55c8142ca0 2023-10-06T02:27:21.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:21.039+0000 7f55ccf9d700 1 -- 172.21.15.31:0/1565426094 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f55b4067290 msgr2=0x7f55b4069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:21.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:21.039+0000 7f55ccf9d700 1 --2- 172.21.15.31:0/1565426094 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f55b4067290 0x7f55b4069740 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f55b8009db0 tx=0x7f55b80091a0 comp rx=0 tx=0).stop 2023-10-06T02:27:21.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:21.039+0000 7f55ccf9d700 1 -- 172.21.15.31:0/1565426094 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55c8142ca0 msgr2=0x7f55c8141cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:21.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:21.039+0000 7f55ccf9d700 1 --2- 172.21.15.31:0/1565426094 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55c8142ca0 0x7f55c8141cd0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f55bc001660 tx=0x7f55bc0019e0 comp rx=0 tx=0).stop 2023-10-06T02:27:21.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:21.040+0000 7f55ccf9d700 1 -- 172.21.15.31:0/1565426094 shutdown_connections 2023-10-06T02:27:21.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:21.040+0000 7f55ccf9d700 1 --2- 172.21.15.31:0/1565426094 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f55b4067290 0x7f55b4069740 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:21.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:21.040+0000 7f55ccf9d700 1 --2- 172.21.15.31:0/1565426094 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55c8143070 0x7f55c813eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:21.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:21.040+0000 7f55ccf9d700 1 --2- 172.21.15.31:0/1565426094 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55c80cb450 0x7f55c80b39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:21.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:21.040+0000 7f55ccf9d700 1 --2- 172.21.15.31:0/1565426094 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55c8142ca0 0x7f55c8141cd0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:21.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:21.040+0000 7f55ccf9d700 1 -- 172.21.15.31:0/1565426094 >> 172.21.15.31:0/1565426094 conn(0x7f55c8067c80 msgr2=0x7f55c815cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:21.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:21.041+0000 7f55ccf9d700 1 -- 172.21.15.31:0/1565426094 shutdown_connections 2023-10-06T02:27:21.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:21.041+0000 7f55ccf9d700 1 -- 172.21.15.31:0/1565426094 wait complete. 2023-10-06T02:27:21.044 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' created 2023-10-06T02:27:21.064 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T02:27:21.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:21.261+0000 7f68709fe700 1 Processor -- start 2023-10-06T02:27:21.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:21.261+0000 7f68709fe700 1 -- start start 2023-10-06T02:27:21.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:21.262+0000 7f68709fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f686c140e40 0x7f686c0c6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:21.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:21.262+0000 7f68709fe700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f686c0c6d50 con 0x7f686c140a70 2023-10-06T02:27:21.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:21.262+0000 7f68709fe700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f686c0c7030 con 0x7f686c06de50 2023-10-06T02:27:21.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:21.262+0000 7f68709fe700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f686c0c7310 con 0x7f686c140e40 2023-10-06T02:27:21.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:21.262+0000 7f686a59c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f686c140e40 0x7f686c0c6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:21.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:21.262+0000 7f686a59c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f686c140e40 0x7f686c0c6770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38168/0 (socket says 172.21.15.31:38168) 2023-10-06T02:27:21.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:21.262+0000 7f686a59c700 1 -- 172.21.15.31:0/899221560 learned_addr learned my addr 172.21.15.31:0/899221560 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:21.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:21.263+0000 7f686a59c700 1 -- 172.21.15.31:0/899221560 >> v1:172.21.15.62:6789/0 conn(0x7f686c06de50 legacy=0x7f686c0aeef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:21.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:21.263+0000 7f686a59c700 1 -- 172.21.15.31:0/899221560 >> v1:172.21.15.31:6789/0 conn(0x7f686c140a70 legacy=0x7f686c0c7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:21.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:21.263+0000 7f686a59c700 1 -- 172.21.15.31:0/899221560 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f686c0c75f0 con 0x7f686c140e40 2023-10-06T02:27:21.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:21.263+0000 7f686a59c700 1 --2- 172.21.15.31:0/899221560 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f686c140e40 0x7f686c0c6770 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f6860009990 tx=0x7f6860009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4026ff1e7fd3a2b8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:21.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:21.263+0000 7f6869d9b700 1 -- 172.21.15.31:0/899221560 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6860005c10 con 0x7f686c140e40 2023-10-06T02:27:21.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:21.264+0000 7f6869d9b700 1 -- 172.21.15.31:0/899221560 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6860005d70 con 0x7f686c140e40 2023-10-06T02:27:21.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:21.264+0000 7f6869d9b700 1 -- 172.21.15.31:0/899221560 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f686000b860 con 0x7f686c140e40 2023-10-06T02:27:21.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:21.264+0000 7f68709fe700 1 -- 172.21.15.31:0/899221560 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f686c140e40 msgr2=0x7f686c0c6770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:21.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:21.264+0000 7f68709fe700 1 --2- 172.21.15.31:0/899221560 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f686c140e40 0x7f686c0c6770 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f6860009990 tx=0x7f6860009220 comp rx=0 tx=0).stop 2023-10-06T02:27:21.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:21.264+0000 7f68709fe700 1 -- 172.21.15.31:0/899221560 shutdown_connections 2023-10-06T02:27:21.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:21.264+0000 7f68709fe700 1 --2- 172.21.15.31:0/899221560 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f686c140e40 0x7f686c0c6770 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:21.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:21.264+0000 7f68709fe700 1 -- 172.21.15.31:0/899221560 >> 172.21.15.31:0/899221560 conn(0x7f686c06d170 msgr2=0x7f686c0cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:21.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:21.264+0000 7f68709fe700 1 -- 172.21.15.31:0/899221560 shutdown_connections 2023-10-06T02:27:21.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:21.264+0000 7f68709fe700 1 -- 172.21.15.31:0/899221560 wait complete. 2023-10-06T02:27:21.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:21.266+0000 7f68709fe700 1 Processor -- start 2023-10-06T02:27:21.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:21.266+0000 7f68709fe700 1 -- start start 2023-10-06T02:27:21.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:21.267+0000 7f68709fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f686c06de50 0x7f686c16b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:21.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:21.267+0000 7f68709fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f686c140a70 0x7f686c16cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:21.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:21.267+0000 7f68709fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f686c140e40 0x7f686c159ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:21.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:21.267+0000 7f68709fe700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f686c157660 con 0x7f686c140a70 2023-10-06T02:27:21.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:21.267+0000 7f68709fe700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f686c0af800 con 0x7f686c140e40 2023-10-06T02:27:21.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:21.267+0000 7f68709fe700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f686c1591f0 con 0x7f686c06de50 2023-10-06T02:27:21.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:21.267+0000 7f686a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f686c140a70 0x7f686c16cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:21.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:21.267+0000 7f686a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f686c140a70 0x7f686c16cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38174/0 (socket says 172.21.15.31:38174) 2023-10-06T02:27:21.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:21.267+0000 7f686a59c700 1 -- 172.21.15.31:0/26773077 learned_addr learned my addr 172.21.15.31:0/26773077 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:21.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:21.267+0000 7f686ad9d700 1 --2- 172.21.15.31:0/26773077 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f686c06de50 0x7f686c16b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:21.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:21.267+0000 7f686b59e700 1 --2- 172.21.15.31:0/26773077 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f686c140e40 0x7f686c159ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:21.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:21.267+0000 7f686a59c700 1 -- 172.21.15.31:0/26773077 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f686c06de50 msgr2=0x7f686c16b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:21.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:21.267+0000 7f686a59c700 1 --2- 172.21.15.31:0/26773077 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f686c06de50 0x7f686c16b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:21.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:21.267+0000 7f686a59c700 1 -- 172.21.15.31:0/26773077 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f686c140e40 msgr2=0x7f686c159ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:21.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:21.267+0000 7f686a59c700 1 --2- 172.21.15.31:0/26773077 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f686c140e40 0x7f686c159ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:21.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:21.267+0000 7f686a59c700 1 -- 172.21.15.31:0/26773077 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6860009480 con 0x7f686c140a70 2023-10-06T02:27:21.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:21.267+0000 7f686b59e700 1 --2- 172.21.15.31:0/26773077 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f686c140e40 0x7f686c159ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:27:21.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:21.267+0000 7f686ad9d700 1 --2- 172.21.15.31:0/26773077 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f686c06de50 0x7f686c16b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:27:21.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:21.268+0000 7f686a59c700 1 --2- 172.21.15.31:0/26773077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f686c140a70 0x7f686c16cdd0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f6860012000 tx=0x7f6860016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:21.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:21.268+0000 7f6852ffd700 1 -- 172.21.15.31:0/26773077 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6860011ae0 con 0x7f686c140a70 2023-10-06T02:27:21.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:21.268+0000 7f6852ffd700 1 -- 172.21.15.31:0/26773077 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6860011c80 con 0x7f686c140a70 2023-10-06T02:27:21.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:21.268+0000 7f68709fe700 1 -- 172.21.15.31:0/26773077 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f686c0aeed0 con 0x7f686c140a70 2023-10-06T02:27:21.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:21.268+0000 7f6852ffd700 1 -- 172.21.15.31:0/26773077 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f686001b430 con 0x7f686c140a70 2023-10-06T02:27:21.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:21.268+0000 7f68709fe700 1 -- 172.21.15.31:0/26773077 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f686c0af490 con 0x7f686c140a70 2023-10-06T02:27:21.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:21.269+0000 7f68709fe700 1 -- 172.21.15.31:0/26773077 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f686c0af9b0 con 0x7f686c140a70 2023-10-06T02:27:21.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:21.271+0000 7f6852ffd700 1 -- 172.21.15.31:0/26773077 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f686001b7e0 con 0x7f686c140a70 2023-10-06T02:27:21.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:21.272+0000 7f6852ffd700 1 --2- 172.21.15.31:0/26773077 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6858067290 0x7f6858069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:21.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:21.272+0000 7f6852ffd700 1 -- 172.21.15.31:0/26773077 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5576+0+0 (secure 0 0 0) 0x7f68600867d0 con 0x7f686c140a70 2023-10-06T02:27:21.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:21.272+0000 7f686ad9d700 1 --2- 172.21.15.31:0/26773077 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6858067290 0x7f6858069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:21.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:21.272+0000 7f686ad9d700 1 --2- 172.21.15.31:0/26773077 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6858067290 0x7f6858069740 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f686c0aec50 tx=0x7f685c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:21.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:21.276+0000 7f6852ffd700 1 -- 172.21.15.31:0/26773077 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f68600536b0 con 0x7f686c140a70 2023-10-06T02:27:21.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:21.478+0000 7f68709fe700 1 -- 172.21.15.31:0/26773077 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f686c0af970 con 0x7f686c140a70 2023-10-06T02:27:22.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:22.041+0000 7f6852ffd700 1 -- 172.21.15.31:0/26773077 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v73) v1 ==== 170+0+0 (secure 0 0 0) 0x7f686002a0d0 con 0x7f686c140a70 2023-10-06T02:27:22.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:22.050+0000 7f68709fe700 1 -- 172.21.15.31:0/26773077 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6858067290 msgr2=0x7f6858069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:22.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:22.050+0000 7f68709fe700 1 --2- 172.21.15.31:0/26773077 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6858067290 0x7f6858069740 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f686c0aec50 tx=0x7f685c009040 comp rx=0 tx=0).stop 2023-10-06T02:27:22.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:22.050+0000 7f68709fe700 1 -- 172.21.15.31:0/26773077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f686c140a70 msgr2=0x7f686c16cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:22.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:22.050+0000 7f68709fe700 1 --2- 172.21.15.31:0/26773077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f686c140a70 0x7f686c16cdd0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f6860012000 tx=0x7f6860016000 comp rx=0 tx=0).stop 2023-10-06T02:27:22.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:22.052+0000 7f68709fe700 1 -- 172.21.15.31:0/26773077 shutdown_connections 2023-10-06T02:27:22.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:22.052+0000 7f68709fe700 1 --2- 172.21.15.31:0/26773077 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f686c140e40 0x7f686c159ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:22.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:22.052+0000 7f68709fe700 1 --2- 172.21.15.31:0/26773077 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f6858067290 0x7f6858069740 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:22.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:22.052+0000 7f68709fe700 1 --2- 172.21.15.31:0/26773077 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f686c06de50 0x7f686c16b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:22.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:22.052+0000 7f68709fe700 1 --2- 172.21.15.31:0/26773077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f686c140a70 0x7f686c16cdd0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:22.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:22.052+0000 7f68709fe700 1 -- 172.21.15.31:0/26773077 >> 172.21.15.31:0/26773077 conn(0x7f686c06d170 msgr2=0x7f686c0c19b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:22.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:22.053+0000 7f68709fe700 1 -- 172.21.15.31:0/26773077 shutdown_connections 2023-10-06T02:27:22.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:22.053+0000 7f68709fe700 1 -- 172.21.15.31:0/26773077 wait complete. 2023-10-06T02:27:22.060 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' created 2023-10-06T02:27:22.081 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T02:27:22.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:22.270+0000 7f67c0b9d700 1 Processor -- start 2023-10-06T02:27:22.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:22.270+0000 7f67c0b9d700 1 -- start start 2023-10-06T02:27:22.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:22.271+0000 7f67c0b9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f67bc0c7be0 0x7f67bc06abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:22.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:22.271+0000 7f67c0b9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f67bc13f780 0x7f67bc13fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:22.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:22.271+0000 7f67c0b9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67bc140120 con 0x7f67bc0c7fb0 2023-10-06T02:27:22.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:22.271+0000 7f67c0b9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67bc06bf10 con 0x7f67bc0c7be0 2023-10-06T02:27:22.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:22.271+0000 7f67c0b9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67bc06b290 con 0x7f67bc13f780 2023-10-06T02:27:22.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:22.272+0000 7f67bb7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f67bc0c7be0 0x7f67bc06abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:22.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:22.272+0000 7f67bb7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f67bc0c7be0 0x7f67bc06abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38180/0 (socket says 172.21.15.31:38180) 2023-10-06T02:27:22.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:22.272+0000 7f67bb7fe700 1 -- 172.21.15.31:0/2311938005 learned_addr learned my addr 172.21.15.31:0/2311938005 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:22.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:22.272+0000 7f67bbfff700 1 --2- 172.21.15.31:0/2311938005 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f67bc13f780 0x7f67bc13fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:22.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:22.272+0000 7f67bb7fe700 1 -- 172.21.15.31:0/2311938005 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f67bc13f780 msgr2=0x7f67bc13fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:22.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:22.272+0000 7f67bb7fe700 1 --2- 172.21.15.31:0/2311938005 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f67bc13f780 0x7f67bc13fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:22.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:22.272+0000 7f67bb7fe700 1 -- 172.21.15.31:0/2311938005 >> v1:172.21.15.62:6789/0 conn(0x7f67bc0c7fb0 legacy=0x7f67bc15a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:22.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:22.272+0000 7f67bb7fe700 1 -- 172.21.15.31:0/2311938005 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67bc06b570 con 0x7f67bc0c7be0 2023-10-06T02:27:22.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:22.272+0000 7f67bb7fe700 1 --2- 172.21.15.31:0/2311938005 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67bc0c7be0 0x7f67bc06abd0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f67ac0099b0 tx=0x7f67ac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f73f0d8d3c2d2327 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:22.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:22.273+0000 7f67ba7fc700 1 -- 172.21.15.31:0/2311938005 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67ac00b860 con 0x7f67bc0c7be0 2023-10-06T02:27:22.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:22.273+0000 7f67ba7fc700 1 -- 172.21.15.31:0/2311938005 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67ac00b9c0 con 0x7f67bc0c7be0 2023-10-06T02:27:22.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:22.273+0000 7f67ba7fc700 1 -- 172.21.15.31:0/2311938005 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67ac00bd60 con 0x7f67bc0c7be0 2023-10-06T02:27:22.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:22.273+0000 7f67c0b9d700 1 -- 172.21.15.31:0/2311938005 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67bc0c7be0 msgr2=0x7f67bc06abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:22.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:22.273+0000 7f67c0b9d700 1 --2- 172.21.15.31:0/2311938005 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67bc0c7be0 0x7f67bc06abd0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f67ac0099b0 tx=0x7f67ac009250 comp rx=0 tx=0).stop 2023-10-06T02:27:22.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:22.273+0000 7f67c0b9d700 1 -- 172.21.15.31:0/2311938005 shutdown_connections 2023-10-06T02:27:22.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:22.273+0000 7f67c0b9d700 1 --2- 172.21.15.31:0/2311938005 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f67bc13f780 0x7f67bc13fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:22.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:22.273+0000 7f67c0b9d700 1 --2- 172.21.15.31:0/2311938005 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67bc0c7be0 0x7f67bc06abd0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:22.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:22.273+0000 7f67c0b9d700 1 -- 172.21.15.31:0/2311938005 >> 172.21.15.31:0/2311938005 conn(0x7f67bc06d110 msgr2=0x7f67bc0cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:22.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:22.273+0000 7f67c0b9d700 1 -- 172.21.15.31:0/2311938005 shutdown_connections 2023-10-06T02:27:22.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:22.273+0000 7f67c0b9d700 1 -- 172.21.15.31:0/2311938005 wait complete. 2023-10-06T02:27:22.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:22.275+0000 7f67c0b9d700 1 Processor -- start 2023-10-06T02:27:22.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:22.275+0000 7f67c0b9d700 1 -- start start 2023-10-06T02:27:22.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:22.276+0000 7f67c0b9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67bc0c7be0 0x7f67bc1568c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:22.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:22.276+0000 7f67c0b9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f67bc0c7fb0 0x7f67bc155bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:22.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:22.276+0000 7f67c0b9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67bc13f780 0x7f67bc0c5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:22.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:22.276+0000 7f67c0b9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67bc13eda0 con 0x7f67bc0c7be0 2023-10-06T02:27:22.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:22.276+0000 7f67c0b9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67bc169f50 con 0x7f67bc13f780 2023-10-06T02:27:22.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:22.276+0000 7f67c0b9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67bc16a230 con 0x7f67bc0c7fb0 2023-10-06T02:27:22.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:22.276+0000 7f67bb7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67bc0c7be0 0x7f67bc1568c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:22.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:22.276+0000 7f67bb7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67bc0c7be0 0x7f67bc1568c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38186/0 (socket says 172.21.15.31:38186) 2023-10-06T02:27:22.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:22.276+0000 7f67bb7fe700 1 -- 172.21.15.31:0/3927506030 learned_addr learned my addr 172.21.15.31:0/3927506030 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:22.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:22.277+0000 7f67bbfff700 1 --2- 172.21.15.31:0/3927506030 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67bc13f780 0x7f67bc0c5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:22.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:22.276+0000 7f67baffd700 1 --2- 172.21.15.31:0/3927506030 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f67bc0c7fb0 0x7f67bc155bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:22.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:22.277+0000 7f67bb7fe700 1 -- 172.21.15.31:0/3927506030 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f67bc0c7fb0 msgr2=0x7f67bc155bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:22.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:22.277+0000 7f67bb7fe700 1 --2- 172.21.15.31:0/3927506030 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f67bc0c7fb0 0x7f67bc155bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:22.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:22.277+0000 7f67bb7fe700 1 -- 172.21.15.31:0/3927506030 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67bc13f780 msgr2=0x7f67bc0c5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:22.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:22.277+0000 7f67bb7fe700 1 --2- 172.21.15.31:0/3927506030 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67bc13f780 0x7f67bc0c5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:22.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:22.277+0000 7f67bb7fe700 1 -- 172.21.15.31:0/3927506030 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67ac0094f0 con 0x7f67bc0c7be0 2023-10-06T02:27:22.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:22.277+0000 7f67bb7fe700 1 --2- 172.21.15.31:0/3927506030 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67bc0c7be0 0x7f67bc1568c0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f67bc0693f0 tx=0x7f67ac017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:22.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:22.277+0000 7f67a37fe700 1 -- 172.21.15.31:0/3927506030 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67ac024440 con 0x7f67bc0c7be0 2023-10-06T02:27:22.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:22.277+0000 7f67a37fe700 1 -- 172.21.15.31:0/3927506030 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67ac0245c0 con 0x7f67bc0c7be0 2023-10-06T02:27:22.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:22.277+0000 7f67a37fe700 1 -- 172.21.15.31:0/3927506030 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67ac024980 con 0x7f67bc0c7be0 2023-10-06T02:27:22.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:22.277+0000 7f67c0b9d700 1 -- 172.21.15.31:0/3927506030 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67bc16a510 con 0x7f67bc0c7be0 2023-10-06T02:27:22.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:22.277+0000 7f67c0b9d700 1 -- 172.21.15.31:0/3927506030 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67bc169c60 con 0x7f67bc0c7be0 2023-10-06T02:27:22.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:22.279+0000 7f67c0b9d700 1 -- 172.21.15.31:0/3927506030 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67bc04c3c0 con 0x7f67bc0c7be0 2023-10-06T02:27:22.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:22.280+0000 7f67a37fe700 1 -- 172.21.15.31:0/3927506030 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f67ac01a420 con 0x7f67bc0c7be0 2023-10-06T02:27:22.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:22.281+0000 7f67a37fe700 1 --2- 172.21.15.31:0/3927506030 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f67a8067290 0x7f67a8069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:22.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:22.281+0000 7f67a37fe700 1 -- 172.21.15.31:0/3927506030 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5958+0+0 (secure 0 0 0) 0x7f67ac086cf0 con 0x7f67bc0c7be0 2023-10-06T02:27:22.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:22.281+0000 7f67baffd700 1 --2- 172.21.15.31:0/3927506030 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f67a8067290 0x7f67a8069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:22.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:22.282+0000 7f67baffd700 1 --2- 172.21.15.31:0/3927506030 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f67a8067290 0x7f67a8069740 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f67bc159ef0 tx=0x7f67b0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:22.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:22.284+0000 7f67a37fe700 1 -- 172.21.15.31:0/3927506030 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f67ac053a90 con 0x7f67bc0c7be0 2023-10-06T02:27:22.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:22.531+0000 7f67c0b9d700 1 -- 172.21.15.31:0/3927506030 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f67bc06b850 con 0x7f67bc0c7be0 2023-10-06T02:27:23.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.055+0000 7f67a37fe700 1 -- 172.21.15.31:0/3927506030 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 11 and data pool 12 v42) v1 ==== 174+0+0 (secure 0 0 0) 0x7f67ac0570b0 con 0x7f67bc0c7be0 2023-10-06T02:27:23.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.060+0000 7f67c0b9d700 1 -- 172.21.15.31:0/3927506030 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f67a8067290 msgr2=0x7f67a8069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:23.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.060+0000 7f67c0b9d700 1 --2- 172.21.15.31:0/3927506030 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f67a8067290 0x7f67a8069740 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f67bc159ef0 tx=0x7f67b0009040 comp rx=0 tx=0).stop 2023-10-06T02:27:23.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.061+0000 7f67c0b9d700 1 -- 172.21.15.31:0/3927506030 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67bc0c7be0 msgr2=0x7f67bc1568c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:23.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.061+0000 7f67c0b9d700 1 --2- 172.21.15.31:0/3927506030 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67bc0c7be0 0x7f67bc1568c0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f67bc0693f0 tx=0x7f67ac017000 comp rx=0 tx=0).stop 2023-10-06T02:27:23.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.063+0000 7f67c0b9d700 1 -- 172.21.15.31:0/3927506030 shutdown_connections 2023-10-06T02:27:23.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.063+0000 7f67c0b9d700 1 --2- 172.21.15.31:0/3927506030 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67bc13f780 0x7f67bc0c5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:23.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.063+0000 7f67c0b9d700 1 --2- 172.21.15.31:0/3927506030 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67bc0c7be0 0x7f67bc1568c0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:23.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.063+0000 7f67c0b9d700 1 --2- 172.21.15.31:0/3927506030 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f67a8067290 0x7f67a8069740 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:23.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.063+0000 7f67c0b9d700 1 --2- 172.21.15.31:0/3927506030 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f67bc0c7fb0 0x7f67bc155bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:23.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.063+0000 7f67c0b9d700 1 -- 172.21.15.31:0/3927506030 >> 172.21.15.31:0/3927506030 conn(0x7f67bc06d110 msgr2=0x7f67bc0c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:23.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.065+0000 7f67c0b9d700 1 -- 172.21.15.31:0/3927506030 shutdown_connections 2023-10-06T02:27:23.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.066+0000 7f67c0b9d700 1 -- 172.21.15.31:0/3927506030 wait complete. 2023-10-06T02:27:23.076 INFO:teuthology.orchestra.run.smithi031.stderr:new fs with metadata pool 11 and data pool 12 2023-10-06T02:27:23.096 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:27:23.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.283+0000 7fe3c5f9d700 1 Processor -- start 2023-10-06T02:27:23.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.283+0000 7fe3c5f9d700 1 -- start start 2023-10-06T02:27:23.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.284+0000 7fe3c5f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe3c015de50 0x7fe3c00cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:23.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.284+0000 7fe3c5f9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3c015afd0 con 0x7fe3c0153c30 2023-10-06T02:27:23.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.284+0000 7fe3c5f9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3c015b2b0 con 0x7fe3c00cef40 2023-10-06T02:27:23.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.284+0000 7fe3c5f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c015b590 con 0x7fe3c015de50 2023-10-06T02:27:23.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.285+0000 7fe3c4f9b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fe3c0153c30 0x7fe3c015a5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:59684/0 (socket says 172.21.15.31:59684) 2023-10-06T02:27:23.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.285+0000 7fe3c4f9b700 1 -- 172.21.15.31:0/1271996891 learned_addr learned my addr 172.21.15.31:0/1271996891 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:27:23.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.285+0000 7fe3bf7fe700 1 -- 172.21.15.31:0/1271996891 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2527022923 0 0) 0x7fe3c015afd0 con 0x7fe3c0153c30 2023-10-06T02:27:23.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.285+0000 7fe3bf7fe700 1 -- 172.21.15.31:0/1271996891 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3b0003570 con 0x7fe3c0153c30 2023-10-06T02:27:23.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.285+0000 7fe3bffff700 1 --2- 172.21.15.31:0/1271996891 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe3c015de50 0x7fe3c00cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:23.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.285+0000 7fe3bf7fe700 1 -- 172.21.15.31:0/1271996891 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2633015330 0 0) 0x7fe3b0003570 con 0x7fe3c0153c30 2023-10-06T02:27:23.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.285+0000 7fe3bf7fe700 1 -- 172.21.15.31:0/1271996891 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe3c015de50 msgr2=0x7fe3c00cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:23.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.285+0000 7fe3bf7fe700 1 --2- 172.21.15.31:0/1271996891 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe3c015de50 0x7fe3c00cea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:23.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.285+0000 7fe3bf7fe700 1 -- 172.21.15.31:0/1271996891 >> v1:172.21.15.62:6789/0 conn(0x7fe3c00cef40 legacy=0x7fe3c015c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:23.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.285+0000 7fe3bf7fe700 1 -- 172.21.15.31:0/1271996891 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3c015b870 con 0x7fe3c0153c30 2023-10-06T02:27:23.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.285+0000 7fe3bf7fe700 1 -- 172.21.15.31:0/1271996891 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7fe3b4001470 con 0x7fe3c0153c30 2023-10-06T02:27:23.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.285+0000 7fe3bf7fe700 1 -- 172.21.15.31:0/1271996891 >> v1:172.21.15.31:6789/0 conn(0x7fe3c0153c30 legacy=0x7fe3c015a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:23.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.286+0000 7fe3bf7fe700 1 --2- 172.21.15.31:0/1271996891 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3b0003a90 0x7fe3b0003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:23.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.286+0000 7fe3bf7fe700 1 --2- 172.21.15.31:0/1271996891 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3b0004810 0x7fe3b0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:23.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.286+0000 7fe3bf7fe700 1 --2- 172.21.15.31:0/1271996891 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe3c015de50 0x7fe3b00051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:23.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.286+0000 7fe3bf7fe700 1 -- 172.21.15.31:0/1271996891 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c015b590 con 0x7fe3b0004810 2023-10-06T02:27:23.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.286+0000 7fe3bf7fe700 1 -- 172.21.15.31:0/1271996891 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b000bb90 con 0x7fe3b0003a90 2023-10-06T02:27:23.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.286+0000 7fe3bf7fe700 1 -- 172.21.15.31:0/1271996891 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b000bd50 con 0x7fe3c015de50 2023-10-06T02:27:23.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.286+0000 7fe3bffff700 1 --2- 172.21.15.31:0/1271996891 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3b0004810 0x7fe3b0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:23.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.286+0000 7fe3c4f9b700 1 --2- 172.21.15.31:0/1271996891 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3b0003a90 0x7fe3b0003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:23.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.286+0000 7fe3c579c700 1 --2- 172.21.15.31:0/1271996891 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe3c015de50 0x7fe3b00051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:23.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.286+0000 7fe3bffff700 1 -- 172.21.15.31:0/1271996891 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe3c015de50 msgr2=0x7fe3b00051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:23.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.286+0000 7fe3bffff700 1 --2- 172.21.15.31:0/1271996891 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe3c015de50 0x7fe3b00051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:23.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.286+0000 7fe3bffff700 1 -- 172.21.15.31:0/1271996891 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3b0003a90 msgr2=0x7fe3b0003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:23.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.286+0000 7fe3bffff700 1 --2- 172.21.15.31:0/1271996891 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3b0003a90 0x7fe3b0003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:23.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.286+0000 7fe3bffff700 1 -- 172.21.15.31:0/1271996891 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3b0003570 con 0x7fe3b0004810 2023-10-06T02:27:23.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.287+0000 7fe3bffff700 1 --2- 172.21.15.31:0/1271996891 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3b0004810 0x7fe3b0004c80 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac009c00 tx=0x7fe3ac0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e82b51c0c75ee36d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:23.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.287+0000 7fe3bf7fe700 1 -- 172.21.15.31:0/1271996891 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac00e710 con 0x7fe3b0004810 2023-10-06T02:27:23.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.287+0000 7fe3bf7fe700 1 -- 172.21.15.31:0/1271996891 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3ac00e870 con 0x7fe3b0004810 2023-10-06T02:27:23.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.287+0000 7fe3bf7fe700 1 -- 172.21.15.31:0/1271996891 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac00ec30 con 0x7fe3b0004810 2023-10-06T02:27:23.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.287+0000 7fe3c5f9d700 1 -- 172.21.15.31:0/1271996891 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3b0004810 msgr2=0x7fe3b0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:23.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.287+0000 7fe3c5f9d700 1 --2- 172.21.15.31:0/1271996891 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3b0004810 0x7fe3b0004c80 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac009c00 tx=0x7fe3ac0091a0 comp rx=0 tx=0).stop 2023-10-06T02:27:23.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.287+0000 7fe3c579c700 1 -- 172.21.15.31:0/1271996891 reap_dead start 2023-10-06T02:27:23.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.288+0000 7fe3c5f9d700 1 -- 172.21.15.31:0/1271996891 shutdown_connections 2023-10-06T02:27:23.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.288+0000 7fe3c5f9d700 1 -- 172.21.15.31:0/1271996891 >> 172.21.15.31:0/1271996891 conn(0x7fe3c00703a0 msgr2=0x7fe3c00707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:23.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.288+0000 7fe3c5f9d700 1 -- 172.21.15.31:0/1271996891 shutdown_connections 2023-10-06T02:27:23.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.288+0000 7fe3c5f9d700 1 -- 172.21.15.31:0/1271996891 wait complete. 2023-10-06T02:27:23.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.289+0000 7fe3c5f9d700 1 Processor -- start 2023-10-06T02:27:23.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.289+0000 7fe3c5f9d700 1 -- start start 2023-10-06T02:27:23.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.290+0000 7fe3c5f9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3b0004810 0x7fe3c013e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:23.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.290+0000 7fe3c5f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe3c00c6220 0x7fe3c00c6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:23.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.290+0000 7fe3c5f9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3c00b3560 0x7fe3c00b39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:23.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.290+0000 7fe3c5f9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c0142400 con 0x7fe3b0004810 2023-10-06T02:27:23.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.290+0000 7fe3c5f9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c00c7ed0 con 0x7fe3c00b3560 2023-10-06T02:27:23.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.290+0000 7fe3c5f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c013ea40 con 0x7fe3c00c6220 2023-10-06T02:27:23.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.290+0000 7fe3c4f9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3b0004810 0x7fe3c013e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:23.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.290+0000 7fe3c4f9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3b0004810 0x7fe3c013e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38196/0 (socket says 172.21.15.31:38196) 2023-10-06T02:27:23.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.290+0000 7fe3c4f9b700 1 -- 172.21.15.31:0/2312112012 learned_addr learned my addr 172.21.15.31:0/2312112012 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:23.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.290+0000 7fe3bffff700 1 --2- 172.21.15.31:0/2312112012 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe3c00c6220 0x7fe3c00c6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:23.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.290+0000 7fe3c579c700 1 --2- 172.21.15.31:0/2312112012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3c00b3560 0x7fe3c00b39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:23.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.291+0000 7fe3c4f9b700 1 -- 172.21.15.31:0/2312112012 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe3c00c6220 msgr2=0x7fe3c00c6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:23.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.291+0000 7fe3c4f9b700 1 --2- 172.21.15.31:0/2312112012 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe3c00c6220 0x7fe3c00c6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:23.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.291+0000 7fe3c4f9b700 1 -- 172.21.15.31:0/2312112012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3c00b3560 msgr2=0x7fe3c00b39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:23.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.291+0000 7fe3c4f9b700 1 --2- 172.21.15.31:0/2312112012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3c00b3560 0x7fe3c00b39e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:23.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.291+0000 7fe3c4f9b700 1 -- 172.21.15.31:0/2312112012 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3c00c6bd0 con 0x7fe3b0004810 2023-10-06T02:27:23.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.291+0000 7fe3c4f9b700 1 --2- 172.21.15.31:0/2312112012 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3b0004810 0x7fe3c013e500 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fe3b4007390 tx=0x7fe3b4001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:23.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.291+0000 7fe3bcff9700 1 -- 172.21.15.31:0/2312112012 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b4018030 con 0x7fe3b0004810 2023-10-06T02:27:23.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.291+0000 7fe3bcff9700 1 -- 172.21.15.31:0/2312112012 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3b400f8d0 con 0x7fe3b0004810 2023-10-06T02:27:23.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.291+0000 7fe3bcff9700 1 -- 172.21.15.31:0/2312112012 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b400fc90 con 0x7fe3b0004810 2023-10-06T02:27:23.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.291+0000 7fe3c5f9d700 1 -- 172.21.15.31:0/2312112012 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3c00c8cf0 con 0x7fe3b0004810 2023-10-06T02:27:23.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.291+0000 7fe3c5f9d700 1 -- 172.21.15.31:0/2312112012 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3c00c83c0 con 0x7fe3b0004810 2023-10-06T02:27:23.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.292+0000 7fe3c5f9d700 1 -- 172.21.15.31:0/2312112012 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3c004c3c0 con 0x7fe3b0004810 2023-10-06T02:27:23.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.293+0000 7fe3bcff9700 1 -- 172.21.15.31:0/2312112012 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fe3b401b070 con 0x7fe3b0004810 2023-10-06T02:27:23.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.294+0000 7fe3bcff9700 1 --2- 172.21.15.31:0/2312112012 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe3a4067290 0x7fe3a4069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:23.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.294+0000 7fe3bcff9700 1 -- 172.21.15.31:0/2312112012 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe3b4019070 con 0x7fe3b0004810 2023-10-06T02:27:23.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.295+0000 7fe3bffff700 1 --2- 172.21.15.31:0/2312112012 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe3a4067290 0x7fe3a4069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:23.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.295+0000 7fe3bffff700 1 --2- 172.21.15.31:0/2312112012 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe3a4067290 0x7fe3a4069740 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac0091e0 tx=0x7fe3ac01a000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:23.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.298+0000 7fe3bcff9700 1 -- 172.21.15.31:0/2312112012 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe3b40518b0 con 0x7fe3b0004810 2023-10-06T02:27:23.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.501+0000 7fe3c5f9d700 1 -- 172.21.15.31:0/2312112012 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe3c00d10c0 con 0x7fe3b0004810 2023-10-06T02:27:23.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.502+0000 7fe3bcff9700 1 -- 172.21.15.31:0/2312112012 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+16834 (secure 0 0 0) 0x7fe3b4054ed0 con 0x7fe3b0004810 2023-10-06T02:27:23.504 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:27:23.504 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":74,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:27:22.533053+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:27:20.257986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-10-06T02:27:21.518685+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:27:23.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.504+0000 7fe3c5f9d700 1 -- 172.21.15.31:0/2312112012 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe3a4067290 msgr2=0x7fe3a4069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:23.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.504+0000 7fe3c5f9d700 1 --2- 172.21.15.31:0/2312112012 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe3a4067290 0x7fe3a4069740 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac0091e0 tx=0x7fe3ac01a000 comp rx=0 tx=0).stop 2023-10-06T02:27:23.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.504+0000 7fe3c5f9d700 1 -- 172.21.15.31:0/2312112012 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3b0004810 msgr2=0x7fe3c013e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:23.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.504+0000 7fe3c5f9d700 1 --2- 172.21.15.31:0/2312112012 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3b0004810 0x7fe3c013e500 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fe3b4007390 tx=0x7fe3b4001e50 comp rx=0 tx=0).stop 2023-10-06T02:27:23.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.504+0000 7fe3c5f9d700 1 -- 172.21.15.31:0/2312112012 shutdown_connections 2023-10-06T02:27:23.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.504+0000 7fe3c5f9d700 1 --2- 172.21.15.31:0/2312112012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3c00b3560 0x7fe3c00b39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:23.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.504+0000 7fe3c5f9d700 1 --2- 172.21.15.31:0/2312112012 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3b0004810 0x7fe3c013e500 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:23.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.504+0000 7fe3c5f9d700 1 --2- 172.21.15.31:0/2312112012 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe3a4067290 0x7fe3a4069740 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:23.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.504+0000 7fe3c5f9d700 1 --2- 172.21.15.31:0/2312112012 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe3c00c6220 0x7fe3c00c6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:23.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.504+0000 7fe3c5f9d700 1 -- 172.21.15.31:0/2312112012 >> 172.21.15.31:0/2312112012 conn(0x7fe3c00703a0 msgr2=0x7fe3c00b23b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:23.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.504+0000 7fe3c5f9d700 1 -- 172.21.15.31:0/2312112012 shutdown_connections 2023-10-06T02:27:23.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.504+0000 7fe3c5f9d700 1 -- 172.21.15.31:0/2312112012 wait complete. 2023-10-06T02:27:23.524 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:27:23.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.709+0000 7fcb0ab9d700 1 Processor -- start 2023-10-06T02:27:23.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.709+0000 7fcb0ab9d700 1 -- start start 2023-10-06T02:27:23.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.711+0000 7fcb0ab9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fcb040c7f90 0x7fcb0406c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:23.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.711+0000 7fcb0ab9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb040ca480 con 0x7fcb04142c00 2023-10-06T02:27:23.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.711+0000 7fcb0ab9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb0415e240 con 0x7fcb04142fd0 2023-10-06T02:27:23.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.711+0000 7fcb0ab9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb0406b350 con 0x7fcb040c7f90 2023-10-06T02:27:23.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.711+0000 7fcb09b9b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fcb04142c00 0x7fcb0406b7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:59690/0 (socket says 172.21.15.31:59690) 2023-10-06T02:27:23.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.711+0000 7fcb09b9b700 1 -- 172.21.15.31:0/2562932906 learned_addr learned my addr 172.21.15.31:0/2562932906 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:27:23.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.711+0000 7fcb0a39c700 1 --2- 172.21.15.31:0/2562932906 >> v2:172.21.15.62:3300/0 conn(0x7fcb040c7f90 0x7fcb0406c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:23.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.711+0000 7fcb08b99700 1 -- 172.21.15.31:0/2562932906 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 444985132 0 0) 0x7fcb040ca480 con 0x7fcb04142c00 2023-10-06T02:27:23.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.711+0000 7fcb08b99700 1 -- 172.21.15.31:0/2562932906 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcaf0003570 con 0x7fcb04142c00 2023-10-06T02:27:23.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.712+0000 7fcb08b99700 1 -- 172.21.15.31:0/2562932906 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1051577865 0 0) 0x7fcb0415e240 con 0x7fcb04142fd0 2023-10-06T02:27:23.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.712+0000 7fcb08b99700 1 -- 172.21.15.31:0/2562932906 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb040ca480 con 0x7fcb04142fd0 2023-10-06T02:27:23.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.712+0000 7fcb0a39c700 1 -- 172.21.15.31:0/2562932906 >> v1:172.21.15.62:6789/0 conn(0x7fcb04142fd0 legacy=0x7fcb040c7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:23.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.712+0000 7fcb0a39c700 1 -- 172.21.15.31:0/2562932906 >> v1:172.21.15.31:6789/0 conn(0x7fcb04142c00 legacy=0x7fcb0406b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:23.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.712+0000 7fcb0a39c700 1 -- 172.21.15.31:0/2562932906 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb0415df60 con 0x7fcb040c7f90 2023-10-06T02:27:23.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.712+0000 7fcb0a39c700 1 --2- 172.21.15.31:0/2562932906 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb040c7f90 0x7fcb0406c6a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fcafc009e60 tx=0x7fcafc009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6835b6559c0d8c7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:23.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.712+0000 7fcb08b99700 1 -- 172.21.15.31:0/2562932906 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcafc00e070 con 0x7fcb040c7f90 2023-10-06T02:27:23.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.712+0000 7fcb08b99700 1 -- 172.21.15.31:0/2562932906 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcafc00bab0 con 0x7fcb040c7f90 2023-10-06T02:27:23.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.712+0000 7fcb08b99700 1 -- 172.21.15.31:0/2562932906 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcafc00be30 con 0x7fcb040c7f90 2023-10-06T02:27:23.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.713+0000 7fcb0ab9d700 1 -- 172.21.15.31:0/2562932906 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb040c7f90 msgr2=0x7fcb0406c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:23.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.713+0000 7fcb0ab9d700 1 --2- 172.21.15.31:0/2562932906 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb040c7f90 0x7fcb0406c6a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fcafc009e60 tx=0x7fcafc009780 comp rx=0 tx=0).stop 2023-10-06T02:27:23.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.713+0000 7fcb0ab9d700 1 -- 172.21.15.31:0/2562932906 shutdown_connections 2023-10-06T02:27:23.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.713+0000 7fcb0ab9d700 1 --2- 172.21.15.31:0/2562932906 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb040c7f90 0x7fcb0406c6a0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:23.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.713+0000 7fcb0ab9d700 1 -- 172.21.15.31:0/2562932906 >> 172.21.15.31:0/2562932906 conn(0x7fcb040675a0 msgr2=0x7fcb040679a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:23.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.713+0000 7fcb0ab9d700 1 -- 172.21.15.31:0/2562932906 shutdown_connections 2023-10-06T02:27:23.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.713+0000 7fcb0ab9d700 1 -- 172.21.15.31:0/2562932906 wait complete. 2023-10-06T02:27:23.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.714+0000 7fcb0ab9d700 1 Processor -- start 2023-10-06T02:27:23.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.715+0000 7fcb0ab9d700 1 -- start start 2023-10-06T02:27:23.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.715+0000 7fcb0ab9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb040c7f90 0x7fcb0415a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:23.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.715+0000 7fcb0ab9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb04142c00 0x7fcb0415b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:23.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.715+0000 7fcb0ab9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcb04142fd0 0x7fcb0415d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:23.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.715+0000 7fcb0ab9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0415b570 con 0x7fcb04142c00 2023-10-06T02:27:23.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.715+0000 7fcb0ab9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0415b7a0 con 0x7fcb040c7f90 2023-10-06T02:27:23.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.715+0000 7fcb0ab9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0415ba80 con 0x7fcb04142fd0 2023-10-06T02:27:23.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.716+0000 7fcb0939a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb04142c00 0x7fcb0415b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:23.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.716+0000 7fcb0939a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb04142c00 0x7fcb0415b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38208/0 (socket says 172.21.15.31:38208) 2023-10-06T02:27:23.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.716+0000 7fcb0939a700 1 -- 172.21.15.31:0/3169259559 learned_addr learned my addr 172.21.15.31:0/3169259559 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:23.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.716+0000 7fcb09b9b700 1 --2- 172.21.15.31:0/3169259559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb040c7f90 0x7fcb0415a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:23.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.716+0000 7fcb0a39c700 1 --2- 172.21.15.31:0/3169259559 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcb04142fd0 0x7fcb0415d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:23.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.716+0000 7fcb0939a700 1 -- 172.21.15.31:0/3169259559 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcb04142fd0 msgr2=0x7fcb0415d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:23.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.716+0000 7fcb0939a700 1 --2- 172.21.15.31:0/3169259559 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcb04142fd0 0x7fcb0415d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:23.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.716+0000 7fcb0939a700 1 -- 172.21.15.31:0/3169259559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb040c7f90 msgr2=0x7fcb0415a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:23.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.716+0000 7fcb0939a700 1 --2- 172.21.15.31:0/3169259559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb040c7f90 0x7fcb0415a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:23.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.716+0000 7fcb0939a700 1 -- 172.21.15.31:0/3169259559 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcafc009a20 con 0x7fcb04142c00 2023-10-06T02:27:23.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.716+0000 7fcb0939a700 1 --2- 172.21.15.31:0/3169259559 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb04142c00 0x7fcb0415b030 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fcaf400c590 tx=0x7fcaf4007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:23.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.717+0000 7fcaf9ffb700 1 -- 172.21.15.31:0/3169259559 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf4010510 con 0x7fcb04142c00 2023-10-06T02:27:23.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.717+0000 7fcaf9ffb700 1 -- 172.21.15.31:0/3169259559 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaf4010690 con 0x7fcb04142c00 2023-10-06T02:27:23.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.717+0000 7fcb0ab9d700 1 -- 172.21.15.31:0/3169259559 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb040c4390 con 0x7fcb04142c00 2023-10-06T02:27:23.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.717+0000 7fcaf9ffb700 1 -- 172.21.15.31:0/3169259559 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf4010a50 con 0x7fcb04142c00 2023-10-06T02:27:23.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.717+0000 7fcb0ab9d700 1 -- 172.21.15.31:0/3169259559 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb040c4950 con 0x7fcb04142c00 2023-10-06T02:27:23.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.718+0000 7fcb0ab9d700 1 -- 172.21.15.31:0/3169259559 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb040c4b70 con 0x7fcb04142c00 2023-10-06T02:27:23.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.719+0000 7fcaf9ffb700 1 -- 172.21.15.31:0/3169259559 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fcaf401b400 con 0x7fcb04142c00 2023-10-06T02:27:23.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.720+0000 7fcaf9ffb700 1 --2- 172.21.15.31:0/3169259559 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fcae8067290 0x7fcae8069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:23.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.720+0000 7fcaf9ffb700 1 -- 172.21.15.31:0/3169259559 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fcaf4084a20 con 0x7fcb04142c00 2023-10-06T02:27:23.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.720+0000 7fcb09b9b700 1 --2- 172.21.15.31:0/3169259559 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fcae8067290 0x7fcae8069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:23.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.721+0000 7fcb09b9b700 1 --2- 172.21.15.31:0/3169259559 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fcae8067290 0x7fcae8069740 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fcb000058d0 tx=0x7fcb00009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:23.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.724+0000 7fcaf9ffb700 1 -- 172.21.15.31:0/3169259559 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcaf40515e0 con 0x7fcb04142c00 2023-10-06T02:27:23.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.926+0000 7fcb0ab9d700 1 -- 172.21.15.31:0/3169259559 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcb0404c3c0 con 0x7fcb04142c00 2023-10-06T02:27:23.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.927+0000 7fcaf9ffb700 1 -- 172.21.15.31:0/3169259559 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+16834 (secure 0 0 0) 0x7fcaf4054c00 con 0x7fcb04142c00 2023-10-06T02:27:23.929 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:27:23.929 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":74,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:27:22.533053+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:27:20.257986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-10-06T02:27:21.518685+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:27:23.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.930+0000 7fcb0ab9d700 1 -- 172.21.15.31:0/3169259559 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fcae8067290 msgr2=0x7fcae8069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:23.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.930+0000 7fcb0ab9d700 1 --2- 172.21.15.31:0/3169259559 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fcae8067290 0x7fcae8069740 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fcb000058d0 tx=0x7fcb00009040 comp rx=0 tx=0).stop 2023-10-06T02:27:23.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.930+0000 7fcb0ab9d700 1 -- 172.21.15.31:0/3169259559 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb04142c00 msgr2=0x7fcb0415b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:23.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.930+0000 7fcb0ab9d700 1 --2- 172.21.15.31:0/3169259559 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb04142c00 0x7fcb0415b030 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fcaf400c590 tx=0x7fcaf4007dd0 comp rx=0 tx=0).stop 2023-10-06T02:27:23.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.930+0000 7fcb0ab9d700 1 -- 172.21.15.31:0/3169259559 shutdown_connections 2023-10-06T02:27:23.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.930+0000 7fcb0ab9d700 1 --2- 172.21.15.31:0/3169259559 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fcae8067290 0x7fcae8069740 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:23.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.930+0000 7fcb0ab9d700 1 --2- 172.21.15.31:0/3169259559 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcb04142fd0 0x7fcb0415d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:23.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.930+0000 7fcb0ab9d700 1 --2- 172.21.15.31:0/3169259559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb040c7f90 0x7fcb0415a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:23.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.930+0000 7fcb0ab9d700 1 --2- 172.21.15.31:0/3169259559 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb04142c00 0x7fcb0415b030 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:23.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.930+0000 7fcb0ab9d700 1 -- 172.21.15.31:0/3169259559 >> 172.21.15.31:0/3169259559 conn(0x7fcb040675a0 msgr2=0x7fcb040c2940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:23.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.930+0000 7fcb0ab9d700 1 -- 172.21.15.31:0/3169259559 shutdown_connections 2023-10-06T02:27:23.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:23.930+0000 7fcb0ab9d700 1 -- 172.21.15.31:0/3169259559 wait complete. 2023-10-06T02:27:23.950 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T02:27:24.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:24.141+0000 7fde2639d700 1 Processor -- start 2023-10-06T02:27:24.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:24.141+0000 7fde2639d700 1 -- start start 2023-10-06T02:27:24.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:24.142+0000 7fde2639d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fde2006a210 0x7fde200c6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:24.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:24.142+0000 7fde2639d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fde2006de50 0x7fde200c7190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:24.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:24.142+0000 7fde2639d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde2015b640 con 0x7fde20140a70 2023-10-06T02:27:24.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:24.142+0000 7fde2639d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde200c6800 con 0x7fde2006de50 2023-10-06T02:27:24.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:24.142+0000 7fde2639d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde2015b920 con 0x7fde2006a210 2023-10-06T02:27:24.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:24.143+0000 7fde25b9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fde2006de50 0x7fde200c7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:24.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:24.143+0000 7fde25b9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fde2006de50 0x7fde200c7190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38210/0 (socket says 172.21.15.31:38210) 2023-10-06T02:27:24.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:24.143+0000 7fde25b9c700 1 -- 172.21.15.31:0/2873040439 learned_addr learned my addr 172.21.15.31:0/2873040439 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:24.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:24.143+0000 7fde24b9a700 1 --2- 172.21.15.31:0/2873040439 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fde2006a210 0x7fde200c6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:24.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:24.143+0000 7fde25b9c700 1 -- 172.21.15.31:0/2873040439 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fde2006a210 msgr2=0x7fde200c6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:24.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:24.143+0000 7fde25b9c700 1 --2- 172.21.15.31:0/2873040439 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fde2006a210 0x7fde200c6c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:24.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:24.143+0000 7fde25b9c700 1 -- 172.21.15.31:0/2873040439 >> v1:172.21.15.62:6789/0 conn(0x7fde20140a70 legacy=0x7fde20069b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:24.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:24.143+0000 7fde25b9c700 1 -- 172.21.15.31:0/2873040439 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde2015bc00 con 0x7fde2006de50 2023-10-06T02:27:24.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:24.143+0000 7fde24b9a700 1 --2- 172.21.15.31:0/2873040439 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fde2006a210 0x7fde200c6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:27:24.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:24.143+0000 7fde25b9c700 1 --2- 172.21.15.31:0/2873040439 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde2006de50 0x7fde200c7190 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fde18009f20 tx=0x7fde18009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb04ecc21497cb58 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:24.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:24.144+0000 7fde17fff700 1 -- 172.21.15.31:0/2873040439 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde1800e070 con 0x7fde2006de50 2023-10-06T02:27:24.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:24.144+0000 7fde17fff700 1 -- 172.21.15.31:0/2873040439 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde1800ba40 con 0x7fde2006de50 2023-10-06T02:27:24.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:24.144+0000 7fde17fff700 1 -- 172.21.15.31:0/2873040439 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde180092e0 con 0x7fde2006de50 2023-10-06T02:27:24.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:24.144+0000 7fde2639d700 1 -- 172.21.15.31:0/2873040439 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde2006de50 msgr2=0x7fde200c7190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:24.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:24.144+0000 7fde2639d700 1 --2- 172.21.15.31:0/2873040439 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde2006de50 0x7fde200c7190 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fde18009f20 tx=0x7fde18009840 comp rx=0 tx=0).stop 2023-10-06T02:27:24.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:24.144+0000 7fde2639d700 1 -- 172.21.15.31:0/2873040439 shutdown_connections 2023-10-06T02:27:24.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:24.144+0000 7fde2639d700 1 --2- 172.21.15.31:0/2873040439 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde2006de50 0x7fde200c7190 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:24.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:24.144+0000 7fde2639d700 1 --2- 172.21.15.31:0/2873040439 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fde2006a210 0x7fde200c6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:24.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:24.144+0000 7fde2639d700 1 -- 172.21.15.31:0/2873040439 >> 172.21.15.31:0/2873040439 conn(0x7fde2006d110 msgr2=0x7fde200cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:24.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:24.144+0000 7fde2639d700 1 -- 172.21.15.31:0/2873040439 shutdown_connections 2023-10-06T02:27:24.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:24.144+0000 7fde2639d700 1 -- 172.21.15.31:0/2873040439 wait complete. 2023-10-06T02:27:24.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:24.146+0000 7fde2639d700 1 Processor -- start 2023-10-06T02:27:24.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:24.146+0000 7fde2639d700 1 -- start start 2023-10-06T02:27:24.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:24.147+0000 7fde2639d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde2006a210 0x7fde2013cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:24.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:24.147+0000 7fde2639d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fde2006de50 0x7fde200ca7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:24.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:24.147+0000 7fde2639d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde20140a70 0x7fde200cad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:24.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:24.147+0000 7fde2639d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde2015d700 con 0x7fde20140a70 2023-10-06T02:27:24.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:24.147+0000 7fde2639d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde2013d1a0 con 0x7fde2006a210 2023-10-06T02:27:24.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:24.147+0000 7fde2639d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde200afea0 con 0x7fde2006de50 2023-10-06T02:27:24.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:24.147+0000 7fde25b9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde20140a70 0x7fde200cad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:24.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:24.147+0000 7fde25b9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde20140a70 0x7fde200cad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38216/0 (socket says 172.21.15.31:38216) 2023-10-06T02:27:24.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:24.147+0000 7fde25b9c700 1 -- 172.21.15.31:0/3103321770 learned_addr learned my addr 172.21.15.31:0/3103321770 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:24.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:24.147+0000 7fde24b9a700 1 --2- 172.21.15.31:0/3103321770 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fde2006de50 0x7fde200ca7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:24.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:24.147+0000 7fde25b9c700 1 -- 172.21.15.31:0/3103321770 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fde2006de50 msgr2=0x7fde200ca7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:24.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:24.147+0000 7fde2539b700 1 --2- 172.21.15.31:0/3103321770 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde2006a210 0x7fde2013cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:24.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:24.147+0000 7fde25b9c700 1 --2- 172.21.15.31:0/3103321770 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fde2006de50 0x7fde200ca7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:24.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:24.147+0000 7fde25b9c700 1 -- 172.21.15.31:0/3103321770 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde2006a210 msgr2=0x7fde2013cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:24.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:24.147+0000 7fde25b9c700 1 --2- 172.21.15.31:0/3103321770 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde2006a210 0x7fde2013cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:24.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:24.147+0000 7fde25b9c700 1 -- 172.21.15.31:0/3103321770 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde18009ae0 con 0x7fde20140a70 2023-10-06T02:27:24.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:24.147+0000 7fde24b9a700 1 --2- 172.21.15.31:0/3103321770 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fde2006de50 0x7fde200ca7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:27:24.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:24.148+0000 7fde2539b700 1 --2- 172.21.15.31:0/3103321770 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde2006a210 0x7fde2013cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:27:24.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:24.148+0000 7fde25b9c700 1 --2- 172.21.15.31:0/3103321770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde20140a70 0x7fde200cad20 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fde200cc000 tx=0x7fde18017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:24.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:24.148+0000 7fde157fa700 1 -- 172.21.15.31:0/3103321770 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde180094e0 con 0x7fde20140a70 2023-10-06T02:27:24.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:24.148+0000 7fde157fa700 1 -- 172.21.15.31:0/3103321770 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde180032f0 con 0x7fde20140a70 2023-10-06T02:27:24.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:24.148+0000 7fde157fa700 1 -- 172.21.15.31:0/3103321770 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde1801d650 con 0x7fde20140a70 2023-10-06T02:27:24.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:24.148+0000 7fde2639d700 1 -- 172.21.15.31:0/3103321770 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde200b0180 con 0x7fde20140a70 2023-10-06T02:27:24.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:24.148+0000 7fde2639d700 1 -- 172.21.15.31:0/3103321770 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde200b0740 con 0x7fde20140a70 2023-10-06T02:27:24.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:24.149+0000 7fde2639d700 1 -- 172.21.15.31:0/3103321770 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde200b1d70 con 0x7fde20140a70 2023-10-06T02:27:24.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:24.150+0000 7fde157fa700 1 -- 172.21.15.31:0/3103321770 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fde1801da80 con 0x7fde20140a70 2023-10-06T02:27:24.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:24.151+0000 7fde157fa700 1 --2- 172.21.15.31:0/3103321770 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fde04067290 0x7fde04069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:24.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:24.151+0000 7fde157fa700 1 -- 172.21.15.31:0/3103321770 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fde180866a0 con 0x7fde20140a70 2023-10-06T02:27:24.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:24.151+0000 7fde2539b700 1 --2- 172.21.15.31:0/3103321770 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fde04067290 0x7fde04069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:24.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:24.152+0000 7fde2539b700 1 --2- 172.21.15.31:0/3103321770 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fde04067290 0x7fde04069740 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fde1c006fd0 tx=0x7fde1c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:24.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:24.156+0000 7fde157fa700 1 -- 172.21.15.31:0/3103321770 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fde18053150 con 0x7fde20140a70 2023-10-06T02:27:24.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:24.407+0000 7fde2639d700 1 -- 172.21.15.31:0/3103321770 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fde2004c3c0 con 0x7fde20140a70 2023-10-06T02:27:25.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:25.061+0000 7fde157fa700 1 -- 172.21.15.31:0/3103321770 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v45) v1 ==== 118+0+0 (secure 0 0 0) 0x7fde18012e60 con 0x7fde20140a70 2023-10-06T02:27:25.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:25.064+0000 7fde2639d700 1 -- 172.21.15.31:0/3103321770 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fde04067290 msgr2=0x7fde04069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:25.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:25.064+0000 7fde2639d700 1 --2- 172.21.15.31:0/3103321770 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fde04067290 0x7fde04069740 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fde1c006fd0 tx=0x7fde1c008040 comp rx=0 tx=0).stop 2023-10-06T02:27:25.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:25.064+0000 7fde2639d700 1 -- 172.21.15.31:0/3103321770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde20140a70 msgr2=0x7fde200cad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:25.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:25.064+0000 7fde2639d700 1 --2- 172.21.15.31:0/3103321770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde20140a70 0x7fde200cad20 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fde200cc000 tx=0x7fde18017000 comp rx=0 tx=0).stop 2023-10-06T02:27:25.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:25.064+0000 7fde2639d700 1 -- 172.21.15.31:0/3103321770 shutdown_connections 2023-10-06T02:27:25.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:25.064+0000 7fde2639d700 1 --2- 172.21.15.31:0/3103321770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde20140a70 0x7fde200cad20 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:25.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:25.064+0000 7fde2639d700 1 --2- 172.21.15.31:0/3103321770 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde2006a210 0x7fde2013cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:25.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:25.064+0000 7fde2639d700 1 --2- 172.21.15.31:0/3103321770 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fde04067290 0x7fde04069740 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:25.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:25.064+0000 7fde2639d700 1 --2- 172.21.15.31:0/3103321770 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fde2006de50 0x7fde200ca7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:25.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:25.064+0000 7fde2639d700 1 -- 172.21.15.31:0/3103321770 >> 172.21.15.31:0/3103321770 conn(0x7fde2006d110 msgr2=0x7fde200c8020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:25.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:25.065+0000 7fde2639d700 1 -- 172.21.15.31:0/3103321770 shutdown_connections 2023-10-06T02:27:25.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:25.065+0000 7fde2639d700 1 -- 172.21.15.31:0/3103321770 wait complete. 2023-10-06T02:27:25.085 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T02:27:25.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:25.277+0000 7f51d159d700 1 Processor -- start 2023-10-06T02:27:25.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:25.277+0000 7f51d159d700 1 -- start start 2023-10-06T02:27:25.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:25.279+0000 7f51d159d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f51cc153c90 0x7f51cc15a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:25.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:25.279+0000 7f51d159d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f51cc06db20 0x7f51cc0ca7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:25.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:25.279+0000 7f51d159d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51cc15ada0 con 0x7f51cc154060 2023-10-06T02:27:25.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:25.279+0000 7f51d159d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51cc06ca80 con 0x7f51cc06db20 2023-10-06T02:27:25.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:25.279+0000 7f51d159d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51cc06cd60 con 0x7f51cc153c90 2023-10-06T02:27:25.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:25.279+0000 7f51cb7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f51cc06db20 0x7f51cc0ca7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:25.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:25.279+0000 7f51cb7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f51cc06db20 0x7f51cc0ca7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38220/0 (socket says 172.21.15.31:38220) 2023-10-06T02:27:25.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:25.279+0000 7f51cb7fe700 1 -- 172.21.15.31:0/3706614449 learned_addr learned my addr 172.21.15.31:0/3706614449 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:25.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:25.279+0000 7f51cbfff700 1 --2- 172.21.15.31:0/3706614449 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f51cc153c90 0x7f51cc15a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:25.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:25.280+0000 7f51cb7fe700 1 -- 172.21.15.31:0/3706614449 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f51cc153c90 msgr2=0x7f51cc15a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:25.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:25.280+0000 7f51cb7fe700 1 --2- 172.21.15.31:0/3706614449 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f51cc153c90 0x7f51cc15a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:25.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:25.280+0000 7f51cb7fe700 1 -- 172.21.15.31:0/3706614449 >> v1:172.21.15.62:6789/0 conn(0x7f51cc154060 legacy=0x7f51cc06a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:25.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:25.280+0000 7f51cb7fe700 1 -- 172.21.15.31:0/3706614449 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51cc06d040 con 0x7f51cc06db20 2023-10-06T02:27:25.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:25.280+0000 7f51cb7fe700 1 --2- 172.21.15.31:0/3706614449 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51cc06db20 0x7f51cc0ca7b0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f51c00099b0 tx=0x7f51c0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13ef5a003e3e9bbf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:25.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:25.280+0000 7f51caffd700 1 -- 172.21.15.31:0/3706614449 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c0009ea0 con 0x7f51cc06db20 2023-10-06T02:27:25.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:25.280+0000 7f51caffd700 1 -- 172.21.15.31:0/3706614449 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51c000b860 con 0x7f51cc06db20 2023-10-06T02:27:25.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:25.280+0000 7f51caffd700 1 -- 172.21.15.31:0/3706614449 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c000bc00 con 0x7f51cc06db20 2023-10-06T02:27:25.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:25.281+0000 7f51d159d700 1 -- 172.21.15.31:0/3706614449 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51cc06db20 msgr2=0x7f51cc0ca7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:25.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:25.281+0000 7f51d159d700 1 --2- 172.21.15.31:0/3706614449 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51cc06db20 0x7f51cc0ca7b0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f51c00099b0 tx=0x7f51c0009250 comp rx=0 tx=0).stop 2023-10-06T02:27:25.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:25.281+0000 7f51d159d700 1 -- 172.21.15.31:0/3706614449 shutdown_connections 2023-10-06T02:27:25.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:25.281+0000 7f51d159d700 1 --2- 172.21.15.31:0/3706614449 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f51cc153c90 0x7f51cc15a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:25.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:25.281+0000 7f51d159d700 1 --2- 172.21.15.31:0/3706614449 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51cc06db20 0x7f51cc0ca7b0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:25.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:25.281+0000 7f51d159d700 1 -- 172.21.15.31:0/3706614449 >> 172.21.15.31:0/3706614449 conn(0x7f51cc06e720 msgr2=0x7f51cc06eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:25.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:25.281+0000 7f51d159d700 1 -- 172.21.15.31:0/3706614449 shutdown_connections 2023-10-06T02:27:25.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:25.281+0000 7f51d159d700 1 -- 172.21.15.31:0/3706614449 wait complete. 2023-10-06T02:27:25.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:25.282+0000 7f51d159d700 1 Processor -- start 2023-10-06T02:27:25.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:25.282+0000 7f51d159d700 1 -- start start 2023-10-06T02:27:25.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:25.283+0000 7f51d159d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51cc06db20 0x7f51cc0b1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:25.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:25.283+0000 7f51d159d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f51cc153c90 0x7f51cc0b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:25.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:25.283+0000 7f51d159d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51cc154060 0x7f51cc0c5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:25.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:25.283+0000 7f51d159d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51cc158040 con 0x7f51cc154060 2023-10-06T02:27:25.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:25.283+0000 7f51d159d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51cc0c6a40 con 0x7f51cc06db20 2023-10-06T02:27:25.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:25.283+0000 7f51d159d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51cc0c6d20 con 0x7f51cc153c90 2023-10-06T02:27:25.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:25.283+0000 7f51d0d9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51cc154060 0x7f51cc0c5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:25.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:25.284+0000 7f51d0d9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51cc154060 0x7f51cc0c5ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38230/0 (socket says 172.21.15.31:38230) 2023-10-06T02:27:25.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:25.284+0000 7f51d0d9c700 1 -- 172.21.15.31:0/4175927942 learned_addr learned my addr 172.21.15.31:0/4175927942 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:25.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:25.284+0000 7f51cb7fe700 1 --2- 172.21.15.31:0/4175927942 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f51cc153c90 0x7f51cc0b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:25.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:25.284+0000 7f51cbfff700 1 --2- 172.21.15.31:0/4175927942 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51cc06db20 0x7f51cc0b1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:25.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:25.284+0000 7f51d0d9c700 1 -- 172.21.15.31:0/4175927942 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f51cc153c90 msgr2=0x7f51cc0b1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:25.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:25.284+0000 7f51d0d9c700 1 --2- 172.21.15.31:0/4175927942 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f51cc153c90 0x7f51cc0b1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:25.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:25.284+0000 7f51d0d9c700 1 -- 172.21.15.31:0/4175927942 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51cc06db20 msgr2=0x7f51cc0b1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:25.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:25.284+0000 7f51d0d9c700 1 --2- 172.21.15.31:0/4175927942 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51cc06db20 0x7f51cc0b1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:25.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:25.284+0000 7f51d0d9c700 1 -- 172.21.15.31:0/4175927942 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51c00094f0 con 0x7f51cc154060 2023-10-06T02:27:25.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:25.284+0000 7f51d0d9c700 1 --2- 172.21.15.31:0/4175927942 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51cc154060 0x7f51cc0c5ce0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f51c400c910 tx=0x7f51c40016f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:25.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:25.284+0000 7f51b3fff700 1 -- 172.21.15.31:0/4175927942 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c4012630 con 0x7f51cc154060 2023-10-06T02:27:25.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:25.284+0000 7f51b3fff700 1 -- 172.21.15.31:0/4175927942 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51c4012790 con 0x7f51cc154060 2023-10-06T02:27:25.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:25.284+0000 7f51d159d700 1 -- 172.21.15.31:0/4175927942 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51cc1579f0 con 0x7f51cc154060 2023-10-06T02:27:25.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:25.284+0000 7f51b3fff700 1 -- 172.21.15.31:0/4175927942 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c4012b50 con 0x7f51cc154060 2023-10-06T02:27:25.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:25.285+0000 7f51d159d700 1 -- 172.21.15.31:0/4175927942 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51cc154490 con 0x7f51cc154060 2023-10-06T02:27:25.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:25.285+0000 7f51d159d700 1 -- 172.21.15.31:0/4175927942 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51cc157850 con 0x7f51cc154060 2023-10-06T02:27:25.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:25.286+0000 7f51b3fff700 1 -- 172.21.15.31:0/4175927942 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f51c401b450 con 0x7f51cc154060 2023-10-06T02:27:25.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:25.288+0000 7f51b3fff700 1 --2- 172.21.15.31:0/4175927942 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f51b8067290 0x7f51b8069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:25.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:25.288+0000 7f51b3fff700 1 -- 172.21.15.31:0/4175927942 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f51c4084c70 con 0x7f51cc154060 2023-10-06T02:27:25.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:25.288+0000 7f51cbfff700 1 --2- 172.21.15.31:0/4175927942 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f51b8067290 0x7f51b8069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:25.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:25.288+0000 7f51cbfff700 1 --2- 172.21.15.31:0/4175927942 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f51b8067290 0x7f51b8069740 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f51cc1530c0 tx=0x7f51bc0091a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:25.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:25.292+0000 7f51b3fff700 1 -- 172.21.15.31:0/4175927942 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f51cc157850 con 0x7f51cc154060 2023-10-06T02:27:25.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:25.537+0000 7f51d159d700 1 -- 172.21.15.31:0/4175927942 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f51cc15ada0 con 0x7f51cc154060 2023-10-06T02:27:26.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.068+0000 7f51b3fff700 1 -- 172.21.15.31:0/4175927942 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v46) v1 ==== 122+0+0 (secure 0 0 0) 0x7f51cc15ada0 con 0x7f51cc154060 2023-10-06T02:27:26.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.070+0000 7f51d159d700 1 -- 172.21.15.31:0/4175927942 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f51b8067290 msgr2=0x7f51b8069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:26.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.070+0000 7f51d159d700 1 --2- 172.21.15.31:0/4175927942 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f51b8067290 0x7f51b8069740 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f51cc1530c0 tx=0x7f51bc0091a0 comp rx=0 tx=0).stop 2023-10-06T02:27:26.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.070+0000 7f51d159d700 1 -- 172.21.15.31:0/4175927942 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51cc154060 msgr2=0x7f51cc0c5ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:26.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.070+0000 7f51d159d700 1 --2- 172.21.15.31:0/4175927942 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51cc154060 0x7f51cc0c5ce0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f51c400c910 tx=0x7f51c40016f0 comp rx=0 tx=0).stop 2023-10-06T02:27:26.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.071+0000 7f51d159d700 1 -- 172.21.15.31:0/4175927942 shutdown_connections 2023-10-06T02:27:26.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.071+0000 7f51d159d700 1 --2- 172.21.15.31:0/4175927942 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51cc154060 0x7f51cc0c5ce0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:26.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.071+0000 7f51d159d700 1 --2- 172.21.15.31:0/4175927942 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51cc06db20 0x7f51cc0b1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:26.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.071+0000 7f51d159d700 1 --2- 172.21.15.31:0/4175927942 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f51b8067290 0x7f51b8069740 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:26.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.071+0000 7f51d159d700 1 --2- 172.21.15.31:0/4175927942 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f51cc153c90 0x7f51cc0b1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:26.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.071+0000 7f51d159d700 1 -- 172.21.15.31:0/4175927942 >> 172.21.15.31:0/4175927942 conn(0x7f51cc06e720 msgr2=0x7f51cc0c1980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:26.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.071+0000 7f51d159d700 1 -- 172.21.15.31:0/4175927942 shutdown_connections 2023-10-06T02:27:26.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.071+0000 7f51d159d700 1 -- 172.21.15.31:0/4175927942 wait complete. 2023-10-06T02:27:26.091 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:27:26.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.278+0000 7fe7fb59e700 1 Processor -- start 2023-10-06T02:27:26.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.278+0000 7fe7fb59e700 1 -- start start 2023-10-06T02:27:26.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.280+0000 7fe7fb59e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe7fc13fef0 0x7fe7fc15a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:26.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.280+0000 7fe7fb59e700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7fc15ae70 con 0x7fe7fc06aaa0 2023-10-06T02:27:26.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.280+0000 7fe7fb59e700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7fc06af60 con 0x7fe7fc13fb20 2023-10-06T02:27:26.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.280+0000 7fe7fb59e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7fc06b240 con 0x7fe7fc13fef0 2023-10-06T02:27:26.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.280+0000 7fe7fad9d700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fe7fc06aaa0 0x7fe7fc0c9d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:59706/0 (socket says 172.21.15.31:59706) 2023-10-06T02:27:26.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.280+0000 7fe7fad9d700 1 -- 172.21.15.31:0/1352406521 learned_addr learned my addr 172.21.15.31:0/1352406521 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:27:26.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.280+0000 7fe7f9d9b700 1 --2- 172.21.15.31:0/1352406521 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe7fc13fef0 0x7fe7fc15a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:26.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.280+0000 7fe7f959a700 1 -- 172.21.15.31:0/1352406521 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1276918124 0 0) 0x7fe7f4001490 con 0x7fe7fc06aaa0 2023-10-06T02:27:26.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.280+0000 7fe7f959a700 1 -- 172.21.15.31:0/1352406521 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7e4003570 con 0x7fe7fc06aaa0 2023-10-06T02:27:26.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.281+0000 7fe7f959a700 1 -- 172.21.15.31:0/1352406521 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2369922927 0 0) 0x7fe7fc06af60 con 0x7fe7fc13fb20 2023-10-06T02:27:26.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.281+0000 7fe7f959a700 1 -- 172.21.15.31:0/1352406521 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7f4001490 con 0x7fe7fc13fb20 2023-10-06T02:27:26.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.281+0000 7fe7f9d9b700 1 -- 172.21.15.31:0/1352406521 >> v1:172.21.15.62:6789/0 conn(0x7fe7fc13fb20 legacy=0x7fe7fc0c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:26.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.281+0000 7fe7f9d9b700 1 -- 172.21.15.31:0/1352406521 >> v1:172.21.15.31:6789/0 conn(0x7fe7fc06aaa0 legacy=0x7fe7fc0c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:26.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.281+0000 7fe7f9d9b700 1 -- 172.21.15.31:0/1352406521 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7fc06b520 con 0x7fe7fc13fef0 2023-10-06T02:27:26.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.281+0000 7fe7f9d9b700 1 --2- 172.21.15.31:0/1352406521 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe7fc13fef0 0x7fe7fc15a930 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0009900 tx=0x7fe7f0009120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85476cac1343c1f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:26.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.281+0000 7fe7f959a700 1 -- 172.21.15.31:0/1352406521 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f0005c10 con 0x7fe7fc13fef0 2023-10-06T02:27:26.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.281+0000 7fe7f959a700 1 -- 172.21.15.31:0/1352406521 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7f0005d70 con 0x7fe7fc13fef0 2023-10-06T02:27:26.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.281+0000 7fe7f959a700 1 -- 172.21.15.31:0/1352406521 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f0006260 con 0x7fe7fc13fef0 2023-10-06T02:27:26.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.282+0000 7fe7fb59e700 1 -- 172.21.15.31:0/1352406521 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe7fc13fef0 msgr2=0x7fe7fc15a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:26.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.282+0000 7fe7fb59e700 1 --2- 172.21.15.31:0/1352406521 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe7fc13fef0 0x7fe7fc15a930 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0009900 tx=0x7fe7f0009120 comp rx=0 tx=0).stop 2023-10-06T02:27:26.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.282+0000 7fe7fb59e700 1 -- 172.21.15.31:0/1352406521 shutdown_connections 2023-10-06T02:27:26.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.282+0000 7fe7fb59e700 1 --2- 172.21.15.31:0/1352406521 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe7fc13fef0 0x7fe7fc15a930 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:26.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.282+0000 7fe7fb59e700 1 -- 172.21.15.31:0/1352406521 >> 172.21.15.31:0/1352406521 conn(0x7fe7fc06d170 msgr2=0x7fe7fc0cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:26.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.282+0000 7fe7fb59e700 1 -- 172.21.15.31:0/1352406521 shutdown_connections 2023-10-06T02:27:26.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.282+0000 7fe7fb59e700 1 -- 172.21.15.31:0/1352406521 wait complete. 2023-10-06T02:27:26.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.283+0000 7fe7fb59e700 1 Processor -- start 2023-10-06T02:27:26.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.284+0000 7fe7fb59e700 1 -- start start 2023-10-06T02:27:26.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.284+0000 7fe7fb59e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7fc06aaa0 0x7fe7fc0ae6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:26.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.285+0000 7fe7fb59e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe7fc13fb20 0x7fe7fc0b0370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:26.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.285+0000 7fe7fb59e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7fc13fef0 0x7fe7fc155c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:26.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.285+0000 7fe7fb59e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7fc0b1400 con 0x7fe7fc13fef0 2023-10-06T02:27:26.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.285+0000 7fe7fb59e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7fc157cc0 con 0x7fe7fc06aaa0 2023-10-06T02:27:26.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.285+0000 7fe7fb59e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7fc0b08b0 con 0x7fe7fc13fb20 2023-10-06T02:27:26.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.285+0000 7fe7fad9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7fc13fef0 0x7fe7fc155c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:26.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.285+0000 7fe7fad9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7fc13fef0 0x7fe7fc155c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38240/0 (socket says 172.21.15.31:38240) 2023-10-06T02:27:26.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.285+0000 7fe7fad9d700 1 -- 172.21.15.31:0/3709156148 learned_addr learned my addr 172.21.15.31:0/3709156148 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:26.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.285+0000 7fe7fa59c700 1 --2- 172.21.15.31:0/3709156148 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7fc06aaa0 0x7fe7fc0ae6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:26.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.285+0000 7fe7f9d9b700 1 --2- 172.21.15.31:0/3709156148 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe7fc13fb20 0x7fe7fc0b0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:26.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.285+0000 7fe7fad9d700 1 -- 172.21.15.31:0/3709156148 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe7fc13fb20 msgr2=0x7fe7fc0b0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:26.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.285+0000 7fe7fad9d700 1 --2- 172.21.15.31:0/3709156148 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe7fc13fb20 0x7fe7fc0b0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:26.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.285+0000 7fe7fad9d700 1 -- 172.21.15.31:0/3709156148 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7fc06aaa0 msgr2=0x7fe7fc0ae6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:26.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.285+0000 7fe7fad9d700 1 --2- 172.21.15.31:0/3709156148 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7fc06aaa0 0x7fe7fc0ae6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:26.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.285+0000 7fe7fad9d700 1 -- 172.21.15.31:0/3709156148 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7f00093c0 con 0x7fe7fc13fef0 2023-10-06T02:27:26.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.285+0000 7fe7fa59c700 1 --2- 172.21.15.31:0/3709156148 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7fc06aaa0 0x7fe7fc0ae6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:27:26.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.286+0000 7fe7fad9d700 1 --2- 172.21.15.31:0/3709156148 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7fc13fef0 0x7fe7fc155c80 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fe7f4001460 tx=0x7fe7f4001bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:26.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.286+0000 7fe7ea7fc700 1 -- 172.21.15.31:0/3709156148 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f400ab40 con 0x7fe7fc13fef0 2023-10-06T02:27:26.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.286+0000 7fe7ea7fc700 1 -- 172.21.15.31:0/3709156148 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7f400aca0 con 0x7fe7fc13fef0 2023-10-06T02:27:26.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.286+0000 7fe7ea7fc700 1 -- 172.21.15.31:0/3709156148 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f4007340 con 0x7fe7fc13fef0 2023-10-06T02:27:26.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.286+0000 7fe7fb59e700 1 -- 172.21.15.31:0/3709156148 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7fc0b0b60 con 0x7fe7fc13fef0 2023-10-06T02:27:26.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.286+0000 7fe7fb59e700 1 -- 172.21.15.31:0/3709156148 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7fc0b1120 con 0x7fe7fc13fef0 2023-10-06T02:27:26.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.287+0000 7fe7fb59e700 1 -- 172.21.15.31:0/3709156148 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7fc157a90 con 0x7fe7fc13fef0 2023-10-06T02:27:26.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.288+0000 7fe7ea7fc700 1 -- 172.21.15.31:0/3709156148 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fe7f4007730 con 0x7fe7fc13fef0 2023-10-06T02:27:26.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.289+0000 7fe7ea7fc700 1 --2- 172.21.15.31:0/3709156148 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe7dc067290 0x7fe7dc069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:26.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.289+0000 7fe7ea7fc700 1 -- 172.21.15.31:0/3709156148 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe7f4085050 con 0x7fe7fc13fef0 2023-10-06T02:27:26.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.289+0000 7fe7fa59c700 1 --2- 172.21.15.31:0/3709156148 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe7dc067290 0x7fe7dc069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:26.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.290+0000 7fe7fa59c700 1 --2- 172.21.15.31:0/3709156148 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe7dc067290 0x7fe7dc069740 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec003990 tx=0x7fe7ec009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:26.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.293+0000 7fe7ea7fc700 1 -- 172.21.15.31:0/3709156148 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe7fc157a90 con 0x7fe7fc13fef0 2023-10-06T02:27:26.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.537+0000 7fe7fb59e700 1 -- 172.21.15.31:0/3709156148 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe7fc15ae70 con 0x7fe7fc13fef0 2023-10-06T02:27:26.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.538+0000 7fe7ea7fc700 1 -- 172.21.15.31:0/3709156148 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+4596 (secure 0 0 0) 0x7fe7fc15ae70 con 0x7fe7fc13fef0 2023-10-06T02:27:26.539 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:27:26.539 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":46,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5474,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/2085637096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2085637096},{"type":"v1","addr":"172.21.15.62:6834","nonce":2085637096}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":5992,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/573080449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":573080449},{"type":"v1","addr":"172.21.15.31:6836","nonce":573080449}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":6007,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2625778079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2625778079},{"type":"v1","addr":"172.21.15.62:6835","nonce":2625778079}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:27:22.533533+0000","modified":"2023-10-06T02:27:26.062249+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6282":{"gid":6282,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6837/3003674295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":3003674295},{"type":"v1","addr":"172.21.15.31:6837","nonce":3003674295}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T02:27:26.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.540+0000 7fe7fb59e700 1 -- 172.21.15.31:0/3709156148 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe7dc067290 msgr2=0x7fe7dc069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:26.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.540+0000 7fe7fb59e700 1 --2- 172.21.15.31:0/3709156148 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe7dc067290 0x7fe7dc069740 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec003990 tx=0x7fe7ec009040 comp rx=0 tx=0).stop 2023-10-06T02:27:26.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.540+0000 7fe7fb59e700 1 -- 172.21.15.31:0/3709156148 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7fc13fef0 msgr2=0x7fe7fc155c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:26.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.540+0000 7fe7fb59e700 1 --2- 172.21.15.31:0/3709156148 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7fc13fef0 0x7fe7fc155c80 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fe7f4001460 tx=0x7fe7f4001bb0 comp rx=0 tx=0).stop 2023-10-06T02:27:26.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.540+0000 7fe7fb59e700 1 -- 172.21.15.31:0/3709156148 shutdown_connections 2023-10-06T02:27:26.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.540+0000 7fe7fb59e700 1 --2- 172.21.15.31:0/3709156148 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7fc13fef0 0x7fe7fc155c80 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:26.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.540+0000 7fe7fb59e700 1 --2- 172.21.15.31:0/3709156148 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7fc06aaa0 0x7fe7fc0ae6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:26.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.540+0000 7fe7fb59e700 1 --2- 172.21.15.31:0/3709156148 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fe7dc067290 0x7fe7dc069740 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:26.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.540+0000 7fe7fb59e700 1 --2- 172.21.15.31:0/3709156148 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe7fc13fb20 0x7fe7fc0b0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:26.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.540+0000 7fe7fb59e700 1 -- 172.21.15.31:0/3709156148 >> 172.21.15.31:0/3709156148 conn(0x7fe7fc06d170 msgr2=0x7fe7fc0c23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:26.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.540+0000 7fe7fb59e700 1 -- 172.21.15.31:0/3709156148 shutdown_connections 2023-10-06T02:27:26.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.540+0000 7fe7fb59e700 1 -- 172.21.15.31:0/3709156148 wait complete. 2023-10-06T02:27:26.543 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 46 2023-10-06T02:27:26.553 INFO:tasks.ceph.mds.c.smithi031.stderr:2023-10-06T02:27:26.552+0000 7f21947f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T02:27:26.559 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:27:26.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.744+0000 7fa6d8d9d700 1 Processor -- start 2023-10-06T02:27:26.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.744+0000 7fa6d8d9d700 1 -- start start 2023-10-06T02:27:26.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.745+0000 7fa6d8d9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa6d4153c30 0x7fa6d415a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:26.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.745+0000 7fa6d8d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa6d40cf110 0x7fa6d415c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:26.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.745+0000 7fa6d8d9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6d415afd0 con 0x7fa6d415de50 2023-10-06T02:27:26.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.745+0000 7fa6d8d9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6d415b2b0 con 0x7fa6d4153c30 2023-10-06T02:27:26.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.745+0000 7fa6d8d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d415b590 con 0x7fa6d40cf110 2023-10-06T02:27:26.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.746+0000 7fa6d37fe700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa6d4153c30 0x7fa6d415a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:26.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.746+0000 7fa6d2ffd700 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fa6d415de50 0x7fa6d40cea00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:38410/0 (socket says 172.21.15.31:38410) 2023-10-06T02:27:26.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.746+0000 7fa6d2ffd700 1 -- 172.21.15.31:0/3086485153 learned_addr learned my addr 172.21.15.31:0/3086485153 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:27:26.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.746+0000 7fa6d3fff700 1 --2- 172.21.15.31:0/3086485153 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa6d40cf110 0x7fa6d415c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:26.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.746+0000 7fa6d27fc700 1 -- 172.21.15.31:0/3086485153 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3992387370 0 0) 0x7fa6d415afd0 con 0x7fa6d415de50 2023-10-06T02:27:26.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.746+0000 7fa6d27fc700 1 -- 172.21.15.31:0/3086485153 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6bc003570 con 0x7fa6d415de50 2023-10-06T02:27:26.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.746+0000 7fa6d3fff700 1 -- 172.21.15.31:0/3086485153 >> v2:172.21.15.62:3300/0 conn(0x7fa6d4153c30 msgr2=0x7fa6d415a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:26.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.746+0000 7fa6d3fff700 1 --2- 172.21.15.31:0/3086485153 >> v2:172.21.15.62:3300/0 conn(0x7fa6d4153c30 0x7fa6d415a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:26.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.746+0000 7fa6d3fff700 1 -- 172.21.15.31:0/3086485153 >> v1:172.21.15.62:6789/0 conn(0x7fa6d415de50 legacy=0x7fa6d40cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:26.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.746+0000 7fa6d3fff700 1 -- 172.21.15.31:0/3086485153 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6d415b870 con 0x7fa6d40cf110 2023-10-06T02:27:26.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.746+0000 7fa6d37fe700 1 --2- 172.21.15.31:0/3086485153 >> v2:172.21.15.62:3300/0 conn(0x7fa6d4153c30 0x7fa6d415a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:27:26.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.747+0000 7fa6d3fff700 1 --2- 172.21.15.31:0/3086485153 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa6d40cf110 0x7fa6d415c070 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc009e60 tx=0x7fa6cc009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=627570500e6d145d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:26.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.747+0000 7fa6d27fc700 1 -- 172.21.15.31:0/3086485153 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc00e070 con 0x7fa6d40cf110 2023-10-06T02:27:26.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.747+0000 7fa6d27fc700 1 -- 172.21.15.31:0/3086485153 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6cc00bab0 con 0x7fa6d40cf110 2023-10-06T02:27:26.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.747+0000 7fa6d27fc700 1 -- 172.21.15.31:0/3086485153 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc00be70 con 0x7fa6d40cf110 2023-10-06T02:27:26.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.748+0000 7fa6d8d9d700 1 -- 172.21.15.31:0/3086485153 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa6d40cf110 msgr2=0x7fa6d415c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:26.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.748+0000 7fa6d8d9d700 1 --2- 172.21.15.31:0/3086485153 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa6d40cf110 0x7fa6d415c070 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc009e60 tx=0x7fa6cc009780 comp rx=0 tx=0).stop 2023-10-06T02:27:26.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.748+0000 7fa6d8d9d700 1 -- 172.21.15.31:0/3086485153 shutdown_connections 2023-10-06T02:27:26.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.748+0000 7fa6d8d9d700 1 --2- 172.21.15.31:0/3086485153 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa6d40cf110 0x7fa6d415c070 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:26.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.748+0000 7fa6d8d9d700 1 --2- 172.21.15.31:0/3086485153 >> v2:172.21.15.62:3300/0 conn(0x7fa6d4153c30 0x7fa6d415a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:26.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.748+0000 7fa6d8d9d700 1 -- 172.21.15.31:0/3086485153 >> 172.21.15.31:0/3086485153 conn(0x7fa6d40703a0 msgr2=0x7fa6d40707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:26.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.748+0000 7fa6d8d9d700 1 -- 172.21.15.31:0/3086485153 shutdown_connections 2023-10-06T02:27:26.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.748+0000 7fa6d8d9d700 1 -- 172.21.15.31:0/3086485153 wait complete. 2023-10-06T02:27:26.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.749+0000 7fa6d8d9d700 1 Processor -- start 2023-10-06T02:27:26.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.750+0000 7fa6d8d9d700 1 -- start start 2023-10-06T02:27:26.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.750+0000 7fa6d8d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa6d40cf110 0x7fa6d40cd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:26.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.750+0000 7fa6d8d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6d4153c30 0x7fa6d4141080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:26.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.751+0000 7fa6d8d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6d415de50 0x7fa6d41508d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:26.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.751+0000 7fa6d8d9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d40ca170 con 0x7fa6d415de50 2023-10-06T02:27:26.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.751+0000 7fa6d8d9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d40c9880 con 0x7fa6d4153c30 2023-10-06T02:27:26.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.751+0000 7fa6d8d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d4150e10 con 0x7fa6d40cf110 2023-10-06T02:27:26.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.751+0000 7fa6d3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6d415de50 0x7fa6d41508d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:26.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.751+0000 7fa6d3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6d415de50 0x7fa6d41508d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38248/0 (socket says 172.21.15.31:38248) 2023-10-06T02:27:26.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.751+0000 7fa6d3fff700 1 -- 172.21.15.31:0/1990069241 learned_addr learned my addr 172.21.15.31:0/1990069241 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:26.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.751+0000 7fa6d2ffd700 1 --2- 172.21.15.31:0/1990069241 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6d4153c30 0x7fa6d4141080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:26.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.751+0000 7fa6d37fe700 1 --2- 172.21.15.31:0/1990069241 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa6d40cf110 0x7fa6d40cd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:26.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.751+0000 7fa6d3fff700 1 -- 172.21.15.31:0/1990069241 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa6d40cf110 msgr2=0x7fa6d40cd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:26.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.751+0000 7fa6d3fff700 1 --2- 172.21.15.31:0/1990069241 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa6d40cf110 0x7fa6d40cd0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:26.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.751+0000 7fa6d3fff700 1 -- 172.21.15.31:0/1990069241 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6d4153c30 msgr2=0x7fa6d4141080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:26.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.751+0000 7fa6d3fff700 1 --2- 172.21.15.31:0/1990069241 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6d4153c30 0x7fa6d4141080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:26.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.751+0000 7fa6d3fff700 1 -- 172.21.15.31:0/1990069241 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6c4009420 con 0x7fa6d415de50 2023-10-06T02:27:26.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.752+0000 7fa6d3fff700 1 --2- 172.21.15.31:0/1990069241 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6d415de50 0x7fa6d41508d0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fa6d406eb00 tx=0x7fa6cc019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:26.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.752+0000 7fa6bb7fe700 1 -- 172.21.15.31:0/1990069241 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc012e50 con 0x7fa6d415de50 2023-10-06T02:27:26.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.752+0000 7fa6bb7fe700 1 -- 172.21.15.31:0/1990069241 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6cc027470 con 0x7fa6d415de50 2023-10-06T02:27:26.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.752+0000 7fa6d8d9d700 1 -- 172.21.15.31:0/1990069241 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6cc009a20 con 0x7fa6d415de50 2023-10-06T02:27:26.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.752+0000 7fa6bb7fe700 1 -- 172.21.15.31:0/1990069241 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc027830 con 0x7fa6d415de50 2023-10-06T02:27:26.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.752+0000 7fa6d8d9d700 1 -- 172.21.15.31:0/1990069241 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6d40c9b20 con 0x7fa6d415de50 2023-10-06T02:27:26.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.753+0000 7fa6d8d9d700 1 -- 172.21.15.31:0/1990069241 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6d404c3c0 con 0x7fa6d415de50 2023-10-06T02:27:26.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.754+0000 7fa6bb7fe700 1 -- 172.21.15.31:0/1990069241 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fa6cc024070 con 0x7fa6d415de50 2023-10-06T02:27:26.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.755+0000 7fa6bb7fe700 1 --2- 172.21.15.31:0/1990069241 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa6c0067290 0x7fa6c0069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:26.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.755+0000 7fa6bb7fe700 1 -- 172.21.15.31:0/1990069241 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa6cc0872d0 con 0x7fa6d415de50 2023-10-06T02:27:26.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.755+0000 7fa6d37fe700 1 --2- 172.21.15.31:0/1990069241 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa6c0067290 0x7fa6c0069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:26.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.756+0000 7fa6d37fe700 1 --2- 172.21.15.31:0/1990069241 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa6c0067290 0x7fa6c0069740 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fa6c4009f40 tx=0x7fa6c40091a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:26.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.759+0000 7fa6bb7fe700 1 -- 172.21.15.31:0/1990069241 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa6cc053f90 con 0x7fa6d415de50 2023-10-06T02:27:26.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.960+0000 7fa6d8d9d700 1 -- 172.21.15.31:0/1990069241 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa6d415bb50 con 0x7fa6d415de50 2023-10-06T02:27:26.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.961+0000 7fa6bb7fe700 1 -- 172.21.15.31:0/1990069241 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+16821 (secure 0 0 0) 0x7fa6cc0575b0 con 0x7fa6d415de50 2023-10-06T02:27:26.963 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:27:26.963 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":75,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:27:24.049898+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:27:20.257986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-10-06T02:27:21.518685+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:27:26.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.963+0000 7fa6d8d9d700 1 -- 172.21.15.31:0/1990069241 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa6c0067290 msgr2=0x7fa6c0069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:26.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.964+0000 7fa6d8d9d700 1 --2- 172.21.15.31:0/1990069241 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa6c0067290 0x7fa6c0069740 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fa6c4009f40 tx=0x7fa6c40091a0 comp rx=0 tx=0).stop 2023-10-06T02:27:26.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.964+0000 7fa6d8d9d700 1 -- 172.21.15.31:0/1990069241 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6d415de50 msgr2=0x7fa6d41508d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:26.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.964+0000 7fa6d8d9d700 1 --2- 172.21.15.31:0/1990069241 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6d415de50 0x7fa6d41508d0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fa6d406eb00 tx=0x7fa6cc019000 comp rx=0 tx=0).stop 2023-10-06T02:27:26.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.964+0000 7fa6d8d9d700 1 -- 172.21.15.31:0/1990069241 shutdown_connections 2023-10-06T02:27:26.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.964+0000 7fa6d8d9d700 1 --2- 172.21.15.31:0/1990069241 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6d415de50 0x7fa6d41508d0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:26.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.964+0000 7fa6d8d9d700 1 --2- 172.21.15.31:0/1990069241 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fa6c0067290 0x7fa6c0069740 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:26.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.964+0000 7fa6d8d9d700 1 --2- 172.21.15.31:0/1990069241 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa6d40cf110 0x7fa6d40cd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:26.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.964+0000 7fa6d8d9d700 1 --2- 172.21.15.31:0/1990069241 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6d4153c30 0x7fa6d4141080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:26.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.964+0000 7fa6d8d9d700 1 -- 172.21.15.31:0/1990069241 >> 172.21.15.31:0/1990069241 conn(0x7fa6d40703a0 msgr2=0x7fa6d4151920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:26.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.964+0000 7fa6d8d9d700 1 -- 172.21.15.31:0/1990069241 shutdown_connections 2023-10-06T02:27:26.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:26.964+0000 7fa6d8d9d700 1 -- 172.21.15.31:0/1990069241 wait complete. 2023-10-06T02:27:26.983 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T02:27:26.983 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T02:27:27.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.169+0000 7f1fbffff700 1 Processor -- start 2023-10-06T02:27:27.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.170+0000 7f1fbffff700 1 -- start start 2023-10-06T02:27:27.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.171+0000 7f1fbffff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f1fc006e0a0 0x7f1fc00c6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:27.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.171+0000 7f1fbffff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1fc006f3a0 0x7f1fc014fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:27.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.171+0000 7f1fbffff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1fc006e470 0x7f1fc01501b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:27.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.171+0000 7f1fbffff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fc006a490 con 0x7f1fc006f3a0 2023-10-06T02:27:27.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.171+0000 7f1fbffff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fc00c7490 con 0x7f1fc006e0a0 2023-10-06T02:27:27.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.171+0000 7f1fbffff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc00c7770 con 0x7f1fc006e470 2023-10-06T02:27:27.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.171+0000 7f1fbe7fc700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1fc006f3a0 0x7f1fc014fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:27.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.171+0000 7f1fbe7fc700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1fc006f3a0 0x7f1fc014fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38264/0 (socket says 172.21.15.31:38264) 2023-10-06T02:27:27.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.171+0000 7f1fbe7fc700 1 -- 172.21.15.31:0/1253340702 learned_addr learned my addr 172.21.15.31:0/1253340702 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:27.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.171+0000 7f1fbeffd700 1 --2- 172.21.15.31:0/1253340702 >> v2:172.21.15.62:3300/0 conn(0x7f1fc006e0a0 0x7f1fc00c6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:27.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.171+0000 7f1fbf7fe700 1 --2- 172.21.15.31:0/1253340702 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1fc006e470 0x7f1fc01501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:27.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.172+0000 7f1fbe7fc700 1 -- 172.21.15.31:0/1253340702 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1fc006e470 msgr2=0x7f1fc01501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:27.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.172+0000 7f1fbe7fc700 1 --2- 172.21.15.31:0/1253340702 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1fc006e470 0x7f1fc01501b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:27.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.172+0000 7f1fbe7fc700 1 -- 172.21.15.31:0/1253340702 >> v2:172.21.15.62:3300/0 conn(0x7f1fc006e0a0 msgr2=0x7f1fc00c6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:27.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.172+0000 7f1fbe7fc700 1 --2- 172.21.15.31:0/1253340702 >> v2:172.21.15.62:3300/0 conn(0x7f1fc006e0a0 0x7f1fc00c6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:27.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.172+0000 7f1fbe7fc700 1 -- 172.21.15.31:0/1253340702 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fc00c7a50 con 0x7f1fc006f3a0 2023-10-06T02:27:27.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.172+0000 7f1fbe7fc700 1 --2- 172.21.15.31:0/1253340702 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1fc006f3a0 0x7f1fc014fc70 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f1fc0069230 tx=0x7f1fb4009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86d4433017bf51b8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:27.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.172+0000 7f1fbdffb700 1 -- 172.21.15.31:0/1253340702 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb4005c10 con 0x7f1fc006f3a0 2023-10-06T02:27:27.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.172+0000 7f1fbdffb700 1 -- 172.21.15.31:0/1253340702 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fb4005d70 con 0x7f1fc006f3a0 2023-10-06T02:27:27.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.172+0000 7f1fbdffb700 1 -- 172.21.15.31:0/1253340702 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb400b880 con 0x7f1fc006f3a0 2023-10-06T02:27:27.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.173+0000 7f1fbffff700 1 -- 172.21.15.31:0/1253340702 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1fc006f3a0 msgr2=0x7f1fc014fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:27.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.173+0000 7f1fbffff700 1 --2- 172.21.15.31:0/1253340702 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1fc006f3a0 0x7f1fc014fc70 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f1fc0069230 tx=0x7f1fb4009220 comp rx=0 tx=0).stop 2023-10-06T02:27:27.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.173+0000 7f1fbffff700 1 -- 172.21.15.31:0/1253340702 shutdown_connections 2023-10-06T02:27:27.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.173+0000 7f1fbffff700 1 --2- 172.21.15.31:0/1253340702 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1fc006e470 0x7f1fc01501b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:27.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.173+0000 7f1fbffff700 1 --2- 172.21.15.31:0/1253340702 >> v2:172.21.15.62:3300/0 conn(0x7f1fc006e0a0 0x7f1fc00c6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:27.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.173+0000 7f1fbffff700 1 --2- 172.21.15.31:0/1253340702 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1fc006f3a0 0x7f1fc014fc70 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:27.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.173+0000 7f1fbffff700 1 -- 172.21.15.31:0/1253340702 >> 172.21.15.31:0/1253340702 conn(0x7f1fc006d170 msgr2=0x7f1fc00cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:27.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.173+0000 7f1fbffff700 1 -- 172.21.15.31:0/1253340702 shutdown_connections 2023-10-06T02:27:27.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.173+0000 7f1fbffff700 1 -- 172.21.15.31:0/1253340702 wait complete. 2023-10-06T02:27:27.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.175+0000 7f1fbffff700 1 Processor -- start 2023-10-06T02:27:27.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.175+0000 7f1fbffff700 1 -- start start 2023-10-06T02:27:27.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.176+0000 7f1fbffff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1fc006e0a0 0x7f1fc0165f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:27.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.176+0000 7f1fbffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1fc006e470 0x7f1fc01589e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:27.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.176+0000 7f1fbffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1fc006f3a0 0x7f1fc0158f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:27.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.176+0000 7f1fbffff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc00b0c70 con 0x7f1fc006e470 2023-10-06T02:27:27.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.176+0000 7f1fbffff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc00b0060 con 0x7f1fc006f3a0 2023-10-06T02:27:27.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.176+0000 7f1fbffff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc00b0340 con 0x7f1fc006e0a0 2023-10-06T02:27:27.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.176+0000 7f1fbe7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1fc006e470 0x7f1fc01589e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:27.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.176+0000 7f1fbe7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1fc006e470 0x7f1fc01589e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38266/0 (socket says 172.21.15.31:38266) 2023-10-06T02:27:27.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.176+0000 7f1fbe7fc700 1 -- 172.21.15.31:0/2479079236 learned_addr learned my addr 172.21.15.31:0/2479079236 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:27.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.176+0000 7f1fbeffd700 1 --2- 172.21.15.31:0/2479079236 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1fc006e0a0 0x7f1fc0165f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:27.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.176+0000 7f1fbf7fe700 1 --2- 172.21.15.31:0/2479079236 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1fc006f3a0 0x7f1fc0158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:27.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.176+0000 7f1fbe7fc700 1 -- 172.21.15.31:0/2479079236 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1fc006e0a0 msgr2=0x7f1fc0165f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:27.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.176+0000 7f1fbe7fc700 1 --2- 172.21.15.31:0/2479079236 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1fc006e0a0 0x7f1fc0165f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:27.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.176+0000 7f1fbe7fc700 1 -- 172.21.15.31:0/2479079236 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1fc006f3a0 msgr2=0x7f1fc0158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:27.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.176+0000 7f1fbe7fc700 1 --2- 172.21.15.31:0/2479079236 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1fc006f3a0 0x7f1fc0158f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:27.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.176+0000 7f1fbe7fc700 1 -- 172.21.15.31:0/2479079236 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fb4009480 con 0x7f1fc006e470 2023-10-06T02:27:27.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.176+0000 7f1fbeffd700 1 --2- 172.21.15.31:0/2479079236 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1fc006e0a0 0x7f1fc0165f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:27:27.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.177+0000 7f1fbe7fc700 1 --2- 172.21.15.31:0/2479079236 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1fc006e470 0x7f1fc01589e0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f1fc0071320 tx=0x7f1fb4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:27.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.177+0000 7f1fa6ffd700 1 -- 172.21.15.31:0/2479079236 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb4010c70 con 0x7f1fc006e470 2023-10-06T02:27:27.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.177+0000 7f1fa6ffd700 1 -- 172.21.15.31:0/2479079236 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fb4010e10 con 0x7f1fc006e470 2023-10-06T02:27:27.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.177+0000 7f1fbffff700 1 -- 172.21.15.31:0/2479079236 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fc00b0620 con 0x7f1fc006e470 2023-10-06T02:27:27.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.177+0000 7f1fa6ffd700 1 -- 172.21.15.31:0/2479079236 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb4006130 con 0x7f1fc006e470 2023-10-06T02:27:27.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.177+0000 7f1fbffff700 1 -- 172.21.15.31:0/2479079236 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fc0166440 con 0x7f1fc006e470 2023-10-06T02:27:27.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.178+0000 7f1fbffff700 1 -- 172.21.15.31:0/2479079236 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fc00b0db0 con 0x7f1fc006e470 2023-10-06T02:27:27.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.179+0000 7f1fa6ffd700 1 -- 172.21.15.31:0/2479079236 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f1fb4006520 con 0x7f1fc006e470 2023-10-06T02:27:27.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.180+0000 7f1fa6ffd700 1 --2- 172.21.15.31:0/2479079236 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f1fac067290 0x7f1fac069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:27.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.180+0000 7f1fa6ffd700 1 -- 172.21.15.31:0/2479079236 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1fb408ad20 con 0x7f1fc006e470 2023-10-06T02:27:27.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.180+0000 7f1fbeffd700 1 --2- 172.21.15.31:0/2479079236 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f1fac067290 0x7f1fac069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:27.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.180+0000 7f1fbeffd700 1 --2- 172.21.15.31:0/2479079236 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f1fac067290 0x7f1fac069740 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1fb0006fd0 tx=0x7f1fb0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:27.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.184+0000 7f1fa6ffd700 1 -- 172.21.15.31:0/2479079236 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1fc00b0db0 con 0x7f1fc006e470 2023-10-06T02:27:27.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.383+0000 7f1fbffff700 1 -- 172.21.15.31:0/2479079236 --> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1fc00b0db0 con 0x7f1fac067290 2023-10-06T02:27:27.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.393+0000 7f1fa6ffd700 1 -- 172.21.15.31:0/2479079236 <== mgr.4107 v2:172.21.15.31:6832/85292 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311278 (secure 0 0 0) 0x7f1fc00b0db0 con 0x7f1fac067290 2023-10-06T02:27:27.399 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:27:27.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.397+0000 7f1fbffff700 1 -- 172.21.15.31:0/2479079236 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f1fac067290 msgr2=0x7f1fac069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:27.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.397+0000 7f1fbffff700 1 --2- 172.21.15.31:0/2479079236 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f1fac067290 0x7f1fac069740 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1fb0006fd0 tx=0x7f1fb0008040 comp rx=0 tx=0).stop 2023-10-06T02:27:27.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.398+0000 7f1fbffff700 1 -- 172.21.15.31:0/2479079236 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1fc006e470 msgr2=0x7f1fc01589e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:27.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.398+0000 7f1fbffff700 1 --2- 172.21.15.31:0/2479079236 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1fc006e470 0x7f1fc01589e0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f1fc0071320 tx=0x7f1fb4017000 comp rx=0 tx=0).stop 2023-10-06T02:27:27.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.398+0000 7f1fbffff700 1 -- 172.21.15.31:0/2479079236 shutdown_connections 2023-10-06T02:27:27.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.398+0000 7f1fbffff700 1 --2- 172.21.15.31:0/2479079236 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1fc006f3a0 0x7f1fc0158f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:27.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.398+0000 7f1fbffff700 1 --2- 172.21.15.31:0/2479079236 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f1fac067290 0x7f1fac069740 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:27.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.398+0000 7f1fbffff700 1 --2- 172.21.15.31:0/2479079236 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1fc006e0a0 0x7f1fc0165f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:27.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.398+0000 7f1fbffff700 1 --2- 172.21.15.31:0/2479079236 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1fc006e470 0x7f1fc01589e0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:27.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.398+0000 7f1fbffff700 1 -- 172.21.15.31:0/2479079236 >> 172.21.15.31:0/2479079236 conn(0x7f1fc006d170 msgr2=0x7f1fc014e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:27.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.398+0000 7f1fbffff700 1 -- 172.21.15.31:0/2479079236 shutdown_connections 2023-10-06T02:27:27.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.398+0000 7f1fbffff700 1 -- 172.21.15.31:0/2479079236 wait complete. 2023-10-06T02:27:27.402 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-10-06T02:27:27.416 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":205,"stamp":"2023-10-06T02:27:27.173505+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2410996,"kb_used_data":8116,"kb_used_omap":0,"kb_used_meta":2402816,"kb_avail":752563724,"statfs":{"total":773094113280,"available":770625253376,"internally_reserved":0,"allocated":8310784,"data_stored":4337028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2460483584},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000980"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.068773+0000","last_change":"2023-10-06T02:27:23.068773+0000","last_active":"2023-10-06T02:27:23.068773+0000","last_peered":"2023-10-06T02:27:23.068773+0000","last_clean":"2023-10-06T02:27:23.068773+0000","last_became_active":"2023-10-06T02:27:23.068472+0000","last_became_peered":"2023-10-06T02:27:23.068472+0000","last_unstale":"2023-10-06T02:27:23.068773+0000","last_undegraded":"2023-10-06T02:27:23.068773+0000","last_fullsized":"2023-10-06T02:27:23.068773+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:51:19.130397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.063973+0000","last_change":"2023-10-06T02:27:23.063973+0000","last_active":"2023-10-06T02:27:23.063973+0000","last_peered":"2023-10-06T02:27:23.063973+0000","last_clean":"2023-10-06T02:27:23.063973+0000","last_became_active":"2023-10-06T02:27:23.063727+0000","last_became_peered":"2023-10-06T02:27:23.063727+0000","last_unstale":"2023-10-06T02:27:23.063973+0000","last_undegraded":"2023-10-06T02:27:23.063973+0000","last_fullsized":"2023-10-06T02:27:23.063973+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:42:18.952592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.078993+0000","last_change":"2023-10-06T02:27:23.078993+0000","last_active":"2023-10-06T02:27:23.078993+0000","last_peered":"2023-10-06T02:27:23.078993+0000","last_clean":"2023-10-06T02:27:23.078993+0000","last_became_active":"2023-10-06T02:27:23.078693+0000","last_became_peered":"2023-10-06T02:27:23.078693+0000","last_unstale":"2023-10-06T02:27:23.078993+0000","last_undegraded":"2023-10-06T02:27:23.078993+0000","last_fullsized":"2023-10-06T02:27:23.078993+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:38:02.227760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.052031+0000","last_change":"2023-10-06T02:27:23.052031+0000","last_active":"2023-10-06T02:27:23.052031+0000","last_peered":"2023-10-06T02:27:23.052031+0000","last_clean":"2023-10-06T02:27:23.052031+0000","last_became_active":"2023-10-06T02:27:23.051722+0000","last_became_peered":"2023-10-06T02:27:23.051722+0000","last_unstale":"2023-10-06T02:27:23.052031+0000","last_undegraded":"2023-10-06T02:27:23.052031+0000","last_fullsized":"2023-10-06T02:27:23.052031+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:33:15.965337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.064970+0000","last_change":"2023-10-06T02:27:23.064970+0000","last_active":"2023-10-06T02:27:23.064970+0000","last_peered":"2023-10-06T02:27:23.064970+0000","last_clean":"2023-10-06T02:27:23.064970+0000","last_became_active":"2023-10-06T02:27:23.064713+0000","last_became_peered":"2023-10-06T02:27:23.064713+0000","last_unstale":"2023-10-06T02:27:23.064970+0000","last_undegraded":"2023-10-06T02:27:23.064970+0000","last_fullsized":"2023-10-06T02:27:23.064970+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:17:39.514674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.068551+0000","last_change":"2023-10-06T02:27:23.068551+0000","last_active":"2023-10-06T02:27:23.068551+0000","last_peered":"2023-10-06T02:27:23.068551+0000","last_clean":"2023-10-06T02:27:23.068551+0000","last_became_active":"2023-10-06T02:27:23.067733+0000","last_became_peered":"2023-10-06T02:27:23.067733+0000","last_unstale":"2023-10-06T02:27:23.068551+0000","last_undegraded":"2023-10-06T02:27:23.068551+0000","last_fullsized":"2023-10-06T02:27:23.068551+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:59:13.916149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.072647+0000","last_change":"2023-10-06T02:27:23.072647+0000","last_active":"2023-10-06T02:27:23.072647+0000","last_peered":"2023-10-06T02:27:23.072647+0000","last_clean":"2023-10-06T02:27:23.072647+0000","last_became_active":"2023-10-06T02:27:23.072179+0000","last_became_peered":"2023-10-06T02:27:23.072179+0000","last_unstale":"2023-10-06T02:27:23.072647+0000","last_undegraded":"2023-10-06T02:27:23.072647+0000","last_fullsized":"2023-10-06T02:27:23.072647+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:19:46.513602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.064311+0000","last_change":"2023-10-06T02:27:23.064311+0000","last_active":"2023-10-06T02:27:23.064311+0000","last_peered":"2023-10-06T02:27:23.064311+0000","last_clean":"2023-10-06T02:27:23.064311+0000","last_became_active":"2023-10-06T02:27:23.064037+0000","last_became_peered":"2023-10-06T02:27:23.064037+0000","last_unstale":"2023-10-06T02:27:23.064311+0000","last_undegraded":"2023-10-06T02:27:23.064311+0000","last_fullsized":"2023-10-06T02:27:23.064311+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:39:30.046943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.059635+0000","last_change":"2023-10-06T02:27:23.059635+0000","last_active":"2023-10-06T02:27:23.059635+0000","last_peered":"2023-10-06T02:27:23.059635+0000","last_clean":"2023-10-06T02:27:23.059635+0000","last_became_active":"2023-10-06T02:27:23.059300+0000","last_became_peered":"2023-10-06T02:27:23.059300+0000","last_unstale":"2023-10-06T02:27:23.059635+0000","last_undegraded":"2023-10-06T02:27:23.059635+0000","last_fullsized":"2023-10-06T02:27:23.059635+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:40:17.090588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.061936+0000","last_change":"2023-10-06T02:27:23.061936+0000","last_active":"2023-10-06T02:27:23.061936+0000","last_peered":"2023-10-06T02:27:23.061936+0000","last_clean":"2023-10-06T02:27:23.061936+0000","last_became_active":"2023-10-06T02:27:23.061674+0000","last_became_peered":"2023-10-06T02:27:23.061674+0000","last_unstale":"2023-10-06T02:27:23.061936+0000","last_undegraded":"2023-10-06T02:27:23.061936+0000","last_fullsized":"2023-10-06T02:27:23.061936+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:32:00.380963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.053395+0000","last_change":"2023-10-06T02:27:23.053395+0000","last_active":"2023-10-06T02:27:23.053395+0000","last_peered":"2023-10-06T02:27:23.053395+0000","last_clean":"2023-10-06T02:27:23.053395+0000","last_became_active":"2023-10-06T02:27:23.052725+0000","last_became_peered":"2023-10-06T02:27:23.052725+0000","last_unstale":"2023-10-06T02:27:23.053395+0000","last_undegraded":"2023-10-06T02:27:23.053395+0000","last_fullsized":"2023-10-06T02:27:23.053395+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:16:23.777928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.065611+0000","last_change":"2023-10-06T02:27:23.065611+0000","last_active":"2023-10-06T02:27:23.065611+0000","last_peered":"2023-10-06T02:27:23.065611+0000","last_clean":"2023-10-06T02:27:23.065611+0000","last_became_active":"2023-10-06T02:27:23.065320+0000","last_became_peered":"2023-10-06T02:27:23.065320+0000","last_unstale":"2023-10-06T02:27:23.065611+0000","last_undegraded":"2023-10-06T02:27:23.065611+0000","last_fullsized":"2023-10-06T02:27:23.065611+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:03:04.467972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.063100+0000","last_change":"2023-10-06T02:27:23.063100+0000","last_active":"2023-10-06T02:27:23.063100+0000","last_peered":"2023-10-06T02:27:23.063100+0000","last_clean":"2023-10-06T02:27:23.063100+0000","last_became_active":"2023-10-06T02:27:23.062823+0000","last_became_peered":"2023-10-06T02:27:23.062823+0000","last_unstale":"2023-10-06T02:27:23.063100+0000","last_undegraded":"2023-10-06T02:27:23.063100+0000","last_fullsized":"2023-10-06T02:27:23.063100+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:50:09.409985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.068744+0000","last_change":"2023-10-06T02:27:23.068744+0000","last_active":"2023-10-06T02:27:23.068744+0000","last_peered":"2023-10-06T02:27:23.068744+0000","last_clean":"2023-10-06T02:27:23.068744+0000","last_became_active":"2023-10-06T02:27:23.068141+0000","last_became_peered":"2023-10-06T02:27:23.068141+0000","last_unstale":"2023-10-06T02:27:23.068744+0000","last_undegraded":"2023-10-06T02:27:23.068744+0000","last_fullsized":"2023-10-06T02:27:23.068744+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:16:23.777928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.074099+0000","last_change":"2023-10-06T02:27:23.074099+0000","last_active":"2023-10-06T02:27:23.074099+0000","last_peered":"2023-10-06T02:27:23.074099+0000","last_clean":"2023-10-06T02:27:23.074099+0000","last_became_active":"2023-10-06T02:27:23.073845+0000","last_became_peered":"2023-10-06T02:27:23.073845+0000","last_unstale":"2023-10-06T02:27:23.074099+0000","last_undegraded":"2023-10-06T02:27:23.074099+0000","last_fullsized":"2023-10-06T02:27:23.074099+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:21:15.789405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.062096+0000","last_change":"2023-10-06T02:27:23.062096+0000","last_active":"2023-10-06T02:27:23.062096+0000","last_peered":"2023-10-06T02:27:23.062096+0000","last_clean":"2023-10-06T02:27:23.062096+0000","last_became_active":"2023-10-06T02:27:23.061783+0000","last_became_peered":"2023-10-06T02:27:23.061783+0000","last_unstale":"2023-10-06T02:27:23.062096+0000","last_undegraded":"2023-10-06T02:27:23.062096+0000","last_fullsized":"2023-10-06T02:27:23.062096+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:16:23.777928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.062414+0000","last_change":"2023-10-06T02:27:23.062414+0000","last_active":"2023-10-06T02:27:23.062414+0000","last_peered":"2023-10-06T02:27:23.062414+0000","last_clean":"2023-10-06T02:27:23.062414+0000","last_became_active":"2023-10-06T02:27:23.062175+0000","last_became_peered":"2023-10-06T02:27:23.062175+0000","last_unstale":"2023-10-06T02:27:23.062414+0000","last_undegraded":"2023-10-06T02:27:23.062414+0000","last_fullsized":"2023-10-06T02:27:23.062414+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:47:32.073798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.077624+0000","last_change":"2023-10-06T02:27:23.077624+0000","last_active":"2023-10-06T02:27:23.077624+0000","last_peered":"2023-10-06T02:27:23.077624+0000","last_clean":"2023-10-06T02:27:23.077624+0000","last_became_active":"2023-10-06T02:27:23.075075+0000","last_became_peered":"2023-10-06T02:27:23.075075+0000","last_unstale":"2023-10-06T02:27:23.077624+0000","last_undegraded":"2023-10-06T02:27:23.077624+0000","last_fullsized":"2023-10-06T02:27:23.077624+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:32:16.638954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.076001+0000","last_change":"2023-10-06T02:27:23.076001+0000","last_active":"2023-10-06T02:27:23.076001+0000","last_peered":"2023-10-06T02:27:23.076001+0000","last_clean":"2023-10-06T02:27:23.076001+0000","last_became_active":"2023-10-06T02:27:23.075735+0000","last_became_peered":"2023-10-06T02:27:23.075735+0000","last_unstale":"2023-10-06T02:27:23.076001+0000","last_undegraded":"2023-10-06T02:27:23.076001+0000","last_fullsized":"2023-10-06T02:27:23.076001+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:17:58.502290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.073503+0000","last_change":"2023-10-06T02:27:23.073503+0000","last_active":"2023-10-06T02:27:23.073503+0000","last_peered":"2023-10-06T02:27:23.073503+0000","last_clean":"2023-10-06T02:27:23.073503+0000","last_became_active":"2023-10-06T02:27:23.073215+0000","last_became_peered":"2023-10-06T02:27:23.073215+0000","last_unstale":"2023-10-06T02:27:23.073503+0000","last_undegraded":"2023-10-06T02:27:23.073503+0000","last_fullsized":"2023-10-06T02:27:23.073503+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:23:22.126990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.065314+0000","last_change":"2023-10-06T02:27:23.065314+0000","last_active":"2023-10-06T02:27:23.065314+0000","last_peered":"2023-10-06T02:27:23.065314+0000","last_clean":"2023-10-06T02:27:23.065314+0000","last_became_active":"2023-10-06T02:27:23.065031+0000","last_became_peered":"2023-10-06T02:27:23.065031+0000","last_unstale":"2023-10-06T02:27:23.065314+0000","last_undegraded":"2023-10-06T02:27:23.065314+0000","last_fullsized":"2023-10-06T02:27:23.065314+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:03:04.467972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.063636+0000","last_change":"2023-10-06T02:27:23.063636+0000","last_active":"2023-10-06T02:27:23.063636+0000","last_peered":"2023-10-06T02:27:23.063636+0000","last_clean":"2023-10-06T02:27:23.063636+0000","last_became_active":"2023-10-06T02:27:23.063362+0000","last_became_peered":"2023-10-06T02:27:23.063362+0000","last_unstale":"2023-10-06T02:27:23.063636+0000","last_undegraded":"2023-10-06T02:27:23.063636+0000","last_fullsized":"2023-10-06T02:27:23.063636+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:24.612765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.052271+0000","last_change":"2023-10-06T02:27:23.052271+0000","last_active":"2023-10-06T02:27:23.052271+0000","last_peered":"2023-10-06T02:27:23.052271+0000","last_clean":"2023-10-06T02:27:23.052271+0000","last_became_active":"2023-10-06T02:27:23.052006+0000","last_became_peered":"2023-10-06T02:27:23.052006+0000","last_unstale":"2023-10-06T02:27:23.052271+0000","last_undegraded":"2023-10-06T02:27:23.052271+0000","last_fullsized":"2023-10-06T02:27:23.052271+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:52:26.680131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.056190+0000","last_change":"2023-10-06T02:27:23.056190+0000","last_active":"2023-10-06T02:27:23.056190+0000","last_peered":"2023-10-06T02:27:23.056190+0000","last_clean":"2023-10-06T02:27:23.056190+0000","last_became_active":"2023-10-06T02:27:23.055933+0000","last_became_peered":"2023-10-06T02:27:23.055933+0000","last_unstale":"2023-10-06T02:27:23.056190+0000","last_undegraded":"2023-10-06T02:27:23.056190+0000","last_fullsized":"2023-10-06T02:27:23.056190+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:19:52.525565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.059536+0000","last_change":"2023-10-06T02:27:23.059536+0000","last_active":"2023-10-06T02:27:23.059536+0000","last_peered":"2023-10-06T02:27:23.059536+0000","last_clean":"2023-10-06T02:27:23.059536+0000","last_became_active":"2023-10-06T02:27:23.059177+0000","last_became_peered":"2023-10-06T02:27:23.059177+0000","last_unstale":"2023-10-06T02:27:23.059536+0000","last_undegraded":"2023-10-06T02:27:23.059536+0000","last_fullsized":"2023-10-06T02:27:23.059536+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:02:28.690358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.065538+0000","last_change":"2023-10-06T02:27:23.065538+0000","last_active":"2023-10-06T02:27:23.065538+0000","last_peered":"2023-10-06T02:27:23.065538+0000","last_clean":"2023-10-06T02:27:23.065538+0000","last_became_active":"2023-10-06T02:27:23.065175+0000","last_became_peered":"2023-10-06T02:27:23.065175+0000","last_unstale":"2023-10-06T02:27:23.065538+0000","last_undegraded":"2023-10-06T02:27:23.065538+0000","last_fullsized":"2023-10-06T02:27:23.065538+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:16:24.898085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.062267+0000","last_change":"2023-10-06T02:27:23.062267+0000","last_active":"2023-10-06T02:27:23.062267+0000","last_peered":"2023-10-06T02:27:23.062267+0000","last_clean":"2023-10-06T02:27:23.062267+0000","last_became_active":"2023-10-06T02:27:23.061943+0000","last_became_peered":"2023-10-06T02:27:23.061943+0000","last_unstale":"2023-10-06T02:27:23.062267+0000","last_undegraded":"2023-10-06T02:27:23.062267+0000","last_fullsized":"2023-10-06T02:27:23.062267+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:17:58.502290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.064287+0000","last_change":"2023-10-06T02:27:23.064287+0000","last_active":"2023-10-06T02:27:23.064287+0000","last_peered":"2023-10-06T02:27:23.064287+0000","last_clean":"2023-10-06T02:27:23.064287+0000","last_became_active":"2023-10-06T02:27:23.064024+0000","last_became_peered":"2023-10-06T02:27:23.064024+0000","last_unstale":"2023-10-06T02:27:23.064287+0000","last_undegraded":"2023-10-06T02:27:23.064287+0000","last_fullsized":"2023-10-06T02:27:23.064287+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:23:13.833011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.062617+0000","last_change":"2023-10-06T02:27:23.062617+0000","last_active":"2023-10-06T02:27:23.062617+0000","last_peered":"2023-10-06T02:27:23.062617+0000","last_clean":"2023-10-06T02:27:23.062617+0000","last_became_active":"2023-10-06T02:27:23.062365+0000","last_became_peered":"2023-10-06T02:27:23.062365+0000","last_unstale":"2023-10-06T02:27:23.062617+0000","last_undegraded":"2023-10-06T02:27:23.062617+0000","last_fullsized":"2023-10-06T02:27:23.062617+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:46:24.524044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.068203+0000","last_change":"2023-10-06T02:27:23.068203+0000","last_active":"2023-10-06T02:27:23.068203+0000","last_peered":"2023-10-06T02:27:23.068203+0000","last_clean":"2023-10-06T02:27:23.068203+0000","last_became_active":"2023-10-06T02:27:23.067857+0000","last_became_peered":"2023-10-06T02:27:23.067857+0000","last_unstale":"2023-10-06T02:27:23.068203+0000","last_undegraded":"2023-10-06T02:27:23.068203+0000","last_fullsized":"2023-10-06T02:27:23.068203+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:46:24.524044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.076720+0000","last_change":"2023-10-06T02:27:23.076720+0000","last_active":"2023-10-06T02:27:23.076720+0000","last_peered":"2023-10-06T02:27:23.076720+0000","last_clean":"2023-10-06T02:27:23.076720+0000","last_became_active":"2023-10-06T02:27:23.075064+0000","last_became_peered":"2023-10-06T02:27:23.075064+0000","last_unstale":"2023-10-06T02:27:23.076720+0000","last_undegraded":"2023-10-06T02:27:23.076720+0000","last_fullsized":"2023-10-06T02:27:23.076720+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:46:09.787754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.065016+0000","last_change":"2023-10-06T02:27:23.065016+0000","last_active":"2023-10-06T02:27:23.065016+0000","last_peered":"2023-10-06T02:27:23.065016+0000","last_clean":"2023-10-06T02:27:23.065016+0000","last_became_active":"2023-10-06T02:27:23.064711+0000","last_became_peered":"2023-10-06T02:27:23.064711+0000","last_unstale":"2023-10-06T02:27:23.065016+0000","last_undegraded":"2023-10-06T02:27:23.065016+0000","last_fullsized":"2023-10-06T02:27:23.065016+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:42:18.952592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.072224+0000","last_change":"2023-10-06T02:27:23.072224+0000","last_active":"2023-10-06T02:27:23.072224+0000","last_peered":"2023-10-06T02:27:23.072224+0000","last_clean":"2023-10-06T02:27:23.072224+0000","last_became_active":"2023-10-06T02:27:23.071964+0000","last_became_peered":"2023-10-06T02:27:23.071964+0000","last_unstale":"2023-10-06T02:27:23.072224+0000","last_undegraded":"2023-10-06T02:27:23.072224+0000","last_fullsized":"2023-10-06T02:27:23.072224+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:19:46.513602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"74'4","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.087918+0000","last_change":"2023-10-06T02:27:22.057080+0000","last_active":"2023-10-06T02:27:23.087918+0000","last_peered":"2023-10-06T02:27:23.087918+0000","last_clean":"2023-10-06T02:27:23.087918+0000","last_became_active":"2023-10-06T02:27:22.056741+0000","last_became_peered":"2023-10-06T02:27:22.056741+0000","last_unstale":"2023-10-06T02:27:23.087918+0000","last_undegraded":"2023-10-06T02:27:23.087918+0000","last_fullsized":"2023-10-06T02:27:23.087918+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_clean_scrub_stamp":"2023-10-06T02:27:21.029649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:49:52.176926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.057905+0000","last_change":"2023-10-06T02:27:23.057905+0000","last_active":"2023-10-06T02:27:23.057905+0000","last_peered":"2023-10-06T02:27:23.057905+0000","last_clean":"2023-10-06T02:27:23.057905+0000","last_became_active":"2023-10-06T02:27:23.057632+0000","last_became_peered":"2023-10-06T02:27:23.057632+0000","last_unstale":"2023-10-06T02:27:23.057905+0000","last_undegraded":"2023-10-06T02:27:23.057905+0000","last_fullsized":"2023-10-06T02:27:23.057905+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:01:04.770431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"74'1","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.056565+0000","last_change":"2023-10-06T02:27:22.064952+0000","last_active":"2023-10-06T02:27:23.056565+0000","last_peered":"2023-10-06T02:27:23.056565+0000","last_clean":"2023-10-06T02:27:23.056565+0000","last_became_active":"2023-10-06T02:27:22.064633+0000","last_became_peered":"2023-10-06T02:27:22.064633+0000","last_unstale":"2023-10-06T02:27:23.056565+0000","last_undegraded":"2023-10-06T02:27:23.056565+0000","last_fullsized":"2023-10-06T02:27:23.056565+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_clean_scrub_stamp":"2023-10-06T02:27:21.029649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:39.645404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.063990+0000","last_change":"2023-10-06T02:27:23.063990+0000","last_active":"2023-10-06T02:27:23.063990+0000","last_peered":"2023-10-06T02:27:23.063990+0000","last_clean":"2023-10-06T02:27:23.063990+0000","last_became_active":"2023-10-06T02:27:23.063676+0000","last_became_peered":"2023-10-06T02:27:23.063676+0000","last_unstale":"2023-10-06T02:27:23.063990+0000","last_undegraded":"2023-10-06T02:27:23.063990+0000","last_fullsized":"2023-10-06T02:27:23.063990+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:48:57.800799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"74'2","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.077322+0000","last_change":"2023-10-06T02:27:22.061693+0000","last_active":"2023-10-06T02:27:23.077322+0000","last_peered":"2023-10-06T02:27:23.077322+0000","last_clean":"2023-10-06T02:27:23.077322+0000","last_became_active":"2023-10-06T02:27:22.061424+0000","last_became_peered":"2023-10-06T02:27:22.061424+0000","last_unstale":"2023-10-06T02:27:23.077322+0000","last_undegraded":"2023-10-06T02:27:23.077322+0000","last_fullsized":"2023-10-06T02:27:23.077322+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_clean_scrub_stamp":"2023-10-06T02:27:21.029649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:22:55.959048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.064775+0000","last_change":"2023-10-06T02:27:23.064775+0000","last_active":"2023-10-06T02:27:23.064775+0000","last_peered":"2023-10-06T02:27:23.064775+0000","last_clean":"2023-10-06T02:27:23.064775+0000","last_became_active":"2023-10-06T02:27:23.064163+0000","last_became_peered":"2023-10-06T02:27:23.064163+0000","last_unstale":"2023-10-06T02:27:23.064775+0000","last_undegraded":"2023-10-06T02:27:23.064775+0000","last_fullsized":"2023-10-06T02:27:23.064775+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:32:00.380963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"74'2","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.080363+0000","last_change":"2023-10-06T02:27:22.056761+0000","last_active":"2023-10-06T02:27:23.080363+0000","last_peered":"2023-10-06T02:27:23.080363+0000","last_clean":"2023-10-06T02:27:23.080363+0000","last_became_active":"2023-10-06T02:27:22.055980+0000","last_became_peered":"2023-10-06T02:27:22.055980+0000","last_unstale":"2023-10-06T02:27:23.080363+0000","last_undegraded":"2023-10-06T02:27:23.080363+0000","last_fullsized":"2023-10-06T02:27:23.080363+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_clean_scrub_stamp":"2023-10-06T02:27:21.029649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:41:47.817758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.063388+0000","last_change":"2023-10-06T02:27:23.063388+0000","last_active":"2023-10-06T02:27:23.063388+0000","last_peered":"2023-10-06T02:27:23.063388+0000","last_clean":"2023-10-06T02:27:23.063388+0000","last_became_active":"2023-10-06T02:27:23.063136+0000","last_became_peered":"2023-10-06T02:27:23.063136+0000","last_unstale":"2023-10-06T02:27:23.063388+0000","last_undegraded":"2023-10-06T02:27:23.063388+0000","last_fullsized":"2023-10-06T02:27:23.063388+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:17:58.502290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"74'1","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.049854+0000","last_change":"2023-10-06T02:27:22.059047+0000","last_active":"2023-10-06T02:27:23.049854+0000","last_peered":"2023-10-06T02:27:23.049854+0000","last_clean":"2023-10-06T02:27:23.049854+0000","last_became_active":"2023-10-06T02:27:22.058668+0000","last_became_peered":"2023-10-06T02:27:22.058668+0000","last_unstale":"2023-10-06T02:27:23.049854+0000","last_undegraded":"2023-10-06T02:27:23.049854+0000","last_fullsized":"2023-10-06T02:27:23.049854+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_clean_scrub_stamp":"2023-10-06T02:27:21.029649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:43:52.473654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.053615+0000","last_change":"2023-10-06T02:27:23.053615+0000","last_active":"2023-10-06T02:27:23.053615+0000","last_peered":"2023-10-06T02:27:23.053615+0000","last_clean":"2023-10-06T02:27:23.053615+0000","last_became_active":"2023-10-06T02:27:23.053094+0000","last_became_peered":"2023-10-06T02:27:23.053094+0000","last_unstale":"2023-10-06T02:27:23.053615+0000","last_undegraded":"2023-10-06T02:27:23.053615+0000","last_fullsized":"2023-10-06T02:27:23.053615+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:59:13.916149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"74'1","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.045669+0000","last_change":"2023-10-06T02:27:22.061701+0000","last_active":"2023-10-06T02:27:23.045669+0000","last_peered":"2023-10-06T02:27:23.045669+0000","last_clean":"2023-10-06T02:27:23.045669+0000","last_became_active":"2023-10-06T02:27:22.060445+0000","last_became_peered":"2023-10-06T02:27:22.060445+0000","last_unstale":"2023-10-06T02:27:23.045669+0000","last_undegraded":"2023-10-06T02:27:23.045669+0000","last_fullsized":"2023-10-06T02:27:23.045669+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_clean_scrub_stamp":"2023-10-06T02:27:21.029649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:32:15.636003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.063099+0000","last_change":"2023-10-06T02:27:23.063099+0000","last_active":"2023-10-06T02:27:23.063099+0000","last_peered":"2023-10-06T02:27:23.063099+0000","last_clean":"2023-10-06T02:27:23.063099+0000","last_became_active":"2023-10-06T02:27:23.062838+0000","last_became_peered":"2023-10-06T02:27:23.062838+0000","last_unstale":"2023-10-06T02:27:23.063099+0000","last_undegraded":"2023-10-06T02:27:23.063099+0000","last_fullsized":"2023-10-06T02:27:23.063099+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:44:16.996178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-10-06T02:27:22.056584+0000","last_change":"2023-10-06T02:27:22.053444+0000","last_active":"2023-10-06T02:27:22.056584+0000","last_peered":"2023-10-06T02:27:22.056584+0000","last_clean":"2023-10-06T02:27:22.056584+0000","last_became_active":"2023-10-06T02:27:22.053134+0000","last_became_peered":"2023-10-06T02:27:22.053134+0000","last_unstale":"2023-10-06T02:27:22.056584+0000","last_undegraded":"2023-10-06T02:27:22.056584+0000","last_fullsized":"2023-10-06T02:27:22.056584+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_clean_scrub_stamp":"2023-10-06T02:27:21.029649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:31:59.378012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.077354+0000","last_change":"2023-10-06T02:27:23.077354+0000","last_active":"2023-10-06T02:27:23.077354+0000","last_peered":"2023-10-06T02:27:23.077354+0000","last_clean":"2023-10-06T02:27:23.077354+0000","last_became_active":"2023-10-06T02:27:23.076610+0000","last_became_peered":"2023-10-06T02:27:23.076610+0000","last_unstale":"2023-10-06T02:27:23.077354+0000","last_undegraded":"2023-10-06T02:27:23.077354+0000","last_fullsized":"2023-10-06T02:27:23.077354+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:32:16.638954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-10-06T02:27:22.065122+0000","last_change":"2023-10-06T02:27:22.060117+0000","last_active":"2023-10-06T02:27:22.065122+0000","last_peered":"2023-10-06T02:27:22.065122+0000","last_clean":"2023-10-06T02:27:22.065122+0000","last_became_active":"2023-10-06T02:27:22.059803+0000","last_became_peered":"2023-10-06T02:27:22.059803+0000","last_unstale":"2023-10-06T02:27:22.065122+0000","last_undegraded":"2023-10-06T02:27:22.065122+0000","last_fullsized":"2023-10-06T02:27:22.065122+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_clean_scrub_stamp":"2023-10-06T02:27:21.029649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:22:37.398480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.061468+0000","last_change":"2023-10-06T02:27:23.061468+0000","last_active":"2023-10-06T02:27:23.061468+0000","last_peered":"2023-10-06T02:27:23.061468+0000","last_clean":"2023-10-06T02:27:23.061468+0000","last_became_active":"2023-10-06T02:27:23.061210+0000","last_became_peered":"2023-10-06T02:27:23.061210+0000","last_unstale":"2023-10-06T02:27:23.061468+0000","last_undegraded":"2023-10-06T02:27:23.061468+0000","last_fullsized":"2023-10-06T02:27:23.061468+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:23:45.796316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-10-06T02:27:22.066569+0000","last_change":"2023-10-06T02:27:22.063730+0000","last_active":"2023-10-06T02:27:22.066569+0000","last_peered":"2023-10-06T02:27:22.066569+0000","last_clean":"2023-10-06T02:27:22.066569+0000","last_became_active":"2023-10-06T02:27:22.063235+0000","last_became_peered":"2023-10-06T02:27:22.063235+0000","last_unstale":"2023-10-06T02:27:22.066569+0000","last_undegraded":"2023-10-06T02:27:22.066569+0000","last_fullsized":"2023-10-06T02:27:22.066569+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_clean_scrub_stamp":"2023-10-06T02:27:21.029649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:39:45.981932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.052031+0000","last_change":"2023-10-06T02:27:23.052031+0000","last_active":"2023-10-06T02:27:23.052031+0000","last_peered":"2023-10-06T02:27:23.052031+0000","last_clean":"2023-10-06T02:27:23.052031+0000","last_became_active":"2023-10-06T02:27:23.051707+0000","last_became_peered":"2023-10-06T02:27:23.051707+0000","last_unstale":"2023-10-06T02:27:23.052031+0000","last_undegraded":"2023-10-06T02:27:23.052031+0000","last_fullsized":"2023-10-06T02:27:23.052031+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:32:28.509573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-10-06T02:27:22.058845+0000","last_change":"2023-10-06T02:27:22.056402+0000","last_active":"2023-10-06T02:27:22.058845+0000","last_peered":"2023-10-06T02:27:22.058845+0000","last_clean":"2023-10-06T02:27:22.058845+0000","last_became_active":"2023-10-06T02:27:22.055356+0000","last_became_peered":"2023-10-06T02:27:22.055356+0000","last_unstale":"2023-10-06T02:27:22.058845+0000","last_undegraded":"2023-10-06T02:27:22.058845+0000","last_fullsized":"2023-10-06T02:27:22.058845+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_clean_scrub_stamp":"2023-10-06T02:27:21.029649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:48:24.192836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.077450+0000","last_change":"2023-10-06T02:27:23.077450+0000","last_active":"2023-10-06T02:27:23.077450+0000","last_peered":"2023-10-06T02:27:23.077450+0000","last_clean":"2023-10-06T02:27:23.077450+0000","last_became_active":"2023-10-06T02:27:23.077195+0000","last_became_peered":"2023-10-06T02:27:23.077195+0000","last_unstale":"2023-10-06T02:27:23.077450+0000","last_undegraded":"2023-10-06T02:27:23.077450+0000","last_fullsized":"2023-10-06T02:27:23.077450+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:02:28.690358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"74'3","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.084514+0000","last_change":"2023-10-06T02:27:22.060557+0000","last_active":"2023-10-06T02:27:23.084514+0000","last_peered":"2023-10-06T02:27:23.084514+0000","last_clean":"2023-10-06T02:27:23.084514+0000","last_became_active":"2023-10-06T02:27:22.059952+0000","last_became_peered":"2023-10-06T02:27:22.059952+0000","last_unstale":"2023-10-06T02:27:23.084514+0000","last_undegraded":"2023-10-06T02:27:23.084514+0000","last_fullsized":"2023-10-06T02:27:23.084514+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_clean_scrub_stamp":"2023-10-06T02:27:21.029649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:04:54.873045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.078293+0000","last_change":"2023-10-06T02:27:23.078293+0000","last_active":"2023-10-06T02:27:23.078293+0000","last_peered":"2023-10-06T02:27:23.078293+0000","last_clean":"2023-10-06T02:27:23.078293+0000","last_became_active":"2023-10-06T02:27:23.078043+0000","last_became_peered":"2023-10-06T02:27:23.078043+0000","last_unstale":"2023-10-06T02:27:23.078293+0000","last_undegraded":"2023-10-06T02:27:23.078293+0000","last_fullsized":"2023-10-06T02:27:23.078293+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:53:54.031079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"74'6","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.081809+0000","last_change":"2023-10-06T02:27:22.063665+0000","last_active":"2023-10-06T02:27:23.081809+0000","last_peered":"2023-10-06T02:27:23.081809+0000","last_clean":"2023-10-06T02:27:23.081809+0000","last_became_active":"2023-10-06T02:27:22.063333+0000","last_became_peered":"2023-10-06T02:27:22.063333+0000","last_unstale":"2023-10-06T02:27:23.081809+0000","last_undegraded":"2023-10-06T02:27:23.081809+0000","last_fullsized":"2023-10-06T02:27:23.081809+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_clean_scrub_stamp":"2023-10-06T02:27:21.029649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:50:17.646687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.059691+0000","last_change":"2023-10-06T02:27:23.059691+0000","last_active":"2023-10-06T02:27:23.059691+0000","last_peered":"2023-10-06T02:27:23.059691+0000","last_clean":"2023-10-06T02:27:23.059691+0000","last_became_active":"2023-10-06T02:27:23.059330+0000","last_became_peered":"2023-10-06T02:27:23.059330+0000","last_unstale":"2023-10-06T02:27:23.059691+0000","last_undegraded":"2023-10-06T02:27:23.059691+0000","last_fullsized":"2023-10-06T02:27:23.059691+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:16:50.367846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"74'2","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.075573+0000","last_change":"2023-10-06T02:27:22.053819+0000","last_active":"2023-10-06T02:27:23.075573+0000","last_peered":"2023-10-06T02:27:23.075573+0000","last_clean":"2023-10-06T02:27:23.075573+0000","last_became_active":"2023-10-06T02:27:22.053552+0000","last_became_peered":"2023-10-06T02:27:22.053552+0000","last_unstale":"2023-10-06T02:27:23.075573+0000","last_undegraded":"2023-10-06T02:27:23.075573+0000","last_fullsized":"2023-10-06T02:27:23.075573+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_clean_scrub_stamp":"2023-10-06T02:27:21.029649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:19:45.510650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.073816+0000","last_change":"2023-10-06T02:27:23.073816+0000","last_active":"2023-10-06T02:27:23.073816+0000","last_peered":"2023-10-06T02:27:23.073816+0000","last_clean":"2023-10-06T02:27:23.073816+0000","last_became_active":"2023-10-06T02:27:23.073518+0000","last_became_peered":"2023-10-06T02:27:23.073518+0000","last_unstale":"2023-10-06T02:27:23.073816+0000","last_undegraded":"2023-10-06T02:27:23.073816+0000","last_fullsized":"2023-10-06T02:27:23.073816+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:42:51.844533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-10-06T02:27:22.067092+0000","last_change":"2023-10-06T02:27:22.065888+0000","last_active":"2023-10-06T02:27:22.067092+0000","last_peered":"2023-10-06T02:27:22.067092+0000","last_clean":"2023-10-06T02:27:22.067092+0000","last_became_active":"2023-10-06T02:27:22.065557+0000","last_became_peered":"2023-10-06T02:27:22.065557+0000","last_unstale":"2023-10-06T02:27:22.067092+0000","last_undegraded":"2023-10-06T02:27:22.067092+0000","last_fullsized":"2023-10-06T02:27:22.067092+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_clean_scrub_stamp":"2023-10-06T02:27:21.029649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:08:29.170939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.068975+0000","last_change":"2023-10-06T02:27:23.068975+0000","last_active":"2023-10-06T02:27:23.068975+0000","last_peered":"2023-10-06T02:27:23.068975+0000","last_clean":"2023-10-06T02:27:23.068975+0000","last_became_active":"2023-10-06T02:27:23.068676+0000","last_became_peered":"2023-10-06T02:27:23.068676+0000","last_unstale":"2023-10-06T02:27:23.068975+0000","last_undegraded":"2023-10-06T02:27:23.068975+0000","last_fullsized":"2023-10-06T02:27:23.068975+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:48:57.800799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-10-06T02:27:22.065091+0000","last_change":"2023-10-06T02:27:22.057079+0000","last_active":"2023-10-06T02:27:22.065091+0000","last_peered":"2023-10-06T02:27:22.065091+0000","last_clean":"2023-10-06T02:27:22.065091+0000","last_became_active":"2023-10-06T02:27:22.056702+0000","last_became_peered":"2023-10-06T02:27:22.056702+0000","last_unstale":"2023-10-06T02:27:22.065091+0000","last_undegraded":"2023-10-06T02:27:22.065091+0000","last_fullsized":"2023-10-06T02:27:22.065091+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_clean_scrub_stamp":"2023-10-06T02:27:21.029649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:29:11.373410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.070443+0000","last_change":"2023-10-06T02:27:23.070443+0000","last_active":"2023-10-06T02:27:23.070443+0000","last_peered":"2023-10-06T02:27:23.070443+0000","last_clean":"2023-10-06T02:27:23.070443+0000","last_became_active":"2023-10-06T02:27:23.068728+0000","last_became_peered":"2023-10-06T02:27:23.068728+0000","last_unstale":"2023-10-06T02:27:23.070443+0000","last_undegraded":"2023-10-06T02:27:23.070443+0000","last_fullsized":"2023-10-06T02:27:23.070443+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:09.326606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"74'8","reported_seq":17,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.082652+0000","last_change":"2023-10-06T02:27:22.052190+0000","last_active":"2023-10-06T02:27:23.082652+0000","last_peered":"2023-10-06T02:27:23.082652+0000","last_clean":"2023-10-06T02:27:23.082652+0000","last_became_active":"2023-10-06T02:27:22.051912+0000","last_became_peered":"2023-10-06T02:27:22.051912+0000","last_unstale":"2023-10-06T02:27:23.082652+0000","last_undegraded":"2023-10-06T02:27:23.082652+0000","last_fullsized":"2023-10-06T02:27:23.082652+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_clean_scrub_stamp":"2023-10-06T02:27:21.029649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:03:03.465021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.060858+0000","last_change":"2023-10-06T02:27:23.060858+0000","last_active":"2023-10-06T02:27:23.060858+0000","last_peered":"2023-10-06T02:27:23.060858+0000","last_clean":"2023-10-06T02:27:23.060858+0000","last_became_active":"2023-10-06T02:27:23.060555+0000","last_became_peered":"2023-10-06T02:27:23.060555+0000","last_unstale":"2023-10-06T02:27:23.060858+0000","last_undegraded":"2023-10-06T02:27:23.060858+0000","last_fullsized":"2023-10-06T02:27:23.060858+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:45:17.340969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"74'2","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.081466+0000","last_change":"2023-10-06T02:27:22.059753+0000","last_active":"2023-10-06T02:27:23.081466+0000","last_peered":"2023-10-06T02:27:23.081466+0000","last_clean":"2023-10-06T02:27:23.081466+0000","last_became_active":"2023-10-06T02:27:22.059459+0000","last_became_peered":"2023-10-06T02:27:22.059459+0000","last_unstale":"2023-10-06T02:27:23.081466+0000","last_undegraded":"2023-10-06T02:27:23.081466+0000","last_fullsized":"2023-10-06T02:27:23.081466+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_clean_scrub_stamp":"2023-10-06T02:27:21.029649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:53:41.029150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.065794+0000","last_change":"2023-10-06T02:27:23.065794+0000","last_active":"2023-10-06T02:27:23.065794+0000","last_peered":"2023-10-06T02:27:23.065794+0000","last_clean":"2023-10-06T02:27:23.065794+0000","last_became_active":"2023-10-06T02:27:23.065543+0000","last_became_peered":"2023-10-06T02:27:23.065543+0000","last_unstale":"2023-10-06T02:27:23.065794+0000","last_undegraded":"2023-10-06T02:27:23.065794+0000","last_fullsized":"2023-10-06T02:27:23.065794+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:44:16.996178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-10-06T02:27:22.064642+0000","last_change":"2023-10-06T02:27:22.062572+0000","last_active":"2023-10-06T02:27:22.064642+0000","last_peered":"2023-10-06T02:27:22.064642+0000","last_clean":"2023-10-06T02:27:22.064642+0000","last_became_active":"2023-10-06T02:27:22.062255+0000","last_became_peered":"2023-10-06T02:27:22.062255+0000","last_unstale":"2023-10-06T02:27:22.064642+0000","last_undegraded":"2023-10-06T02:27:22.064642+0000","last_fullsized":"2023-10-06T02:27:22.064642+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_clean_scrub_stamp":"2023-10-06T02:27:21.029649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:45:16.338018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.059651+0000","last_change":"2023-10-06T02:27:23.059651+0000","last_active":"2023-10-06T02:27:23.059651+0000","last_peered":"2023-10-06T02:27:23.059651+0000","last_clean":"2023-10-06T02:27:23.059651+0000","last_became_active":"2023-10-06T02:27:23.059323+0000","last_became_peered":"2023-10-06T02:27:23.059323+0000","last_unstale":"2023-10-06T02:27:23.059651+0000","last_undegraded":"2023-10-06T02:27:23.059651+0000","last_fullsized":"2023-10-06T02:27:23.059651+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:09.326606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-10-06T02:27:22.063360+0000","last_change":"2023-10-06T02:27:22.059540+0000","last_active":"2023-10-06T02:27:22.063360+0000","last_peered":"2023-10-06T02:27:22.063360+0000","last_clean":"2023-10-06T02:27:22.063360+0000","last_became_active":"2023-10-06T02:27:22.059292+0000","last_became_peered":"2023-10-06T02:27:22.059292+0000","last_unstale":"2023-10-06T02:27:22.063360+0000","last_undegraded":"2023-10-06T02:27:22.063360+0000","last_fullsized":"2023-10-06T02:27:22.063360+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_clean_scrub_stamp":"2023-10-06T02:27:21.029649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:39:25.391598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.065826+0000","last_change":"2023-10-06T02:27:23.065826+0000","last_active":"2023-10-06T02:27:23.065826+0000","last_peered":"2023-10-06T02:27:23.065826+0000","last_clean":"2023-10-06T02:27:23.065826+0000","last_became_active":"2023-10-06T02:27:23.065550+0000","last_became_peered":"2023-10-06T02:27:23.065550+0000","last_unstale":"2023-10-06T02:27:23.065826+0000","last_undegraded":"2023-10-06T02:27:23.065826+0000","last_fullsized":"2023-10-06T02:27:23.065826+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:10:04.209357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-10-06T02:27:22.065628+0000","last_change":"2023-10-06T02:27:22.060206+0000","last_active":"2023-10-06T02:27:22.065628+0000","last_peered":"2023-10-06T02:27:22.065628+0000","last_clean":"2023-10-06T02:27:22.065628+0000","last_became_active":"2023-10-06T02:27:22.059936+0000","last_became_peered":"2023-10-06T02:27:22.059936+0000","last_unstale":"2023-10-06T02:27:22.065628+0000","last_undegraded":"2023-10-06T02:27:22.065628+0000","last_fullsized":"2023-10-06T02:27:22.065628+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_clean_scrub_stamp":"2023-10-06T02:27:21.029649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:49:10.287961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.066888+0000","last_change":"2023-10-06T02:27:23.066888+0000","last_active":"2023-10-06T02:27:23.066888+0000","last_peered":"2023-10-06T02:27:23.066888+0000","last_clean":"2023-10-06T02:27:23.066888+0000","last_became_active":"2023-10-06T02:27:23.066603+0000","last_became_peered":"2023-10-06T02:27:23.066603+0000","last_unstale":"2023-10-06T02:27:23.066888+0000","last_undegraded":"2023-10-06T02:27:23.066888+0000","last_fullsized":"2023-10-06T02:27:23.066888+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:52:26.680131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-10-06T02:27:22.058815+0000","last_change":"2023-10-06T02:27:22.055863+0000","last_active":"2023-10-06T02:27:22.058815+0000","last_peered":"2023-10-06T02:27:22.058815+0000","last_clean":"2023-10-06T02:27:22.058815+0000","last_became_active":"2023-10-06T02:27:22.055522+0000","last_became_peered":"2023-10-06T02:27:22.055522+0000","last_unstale":"2023-10-06T02:27:22.058815+0000","last_undegraded":"2023-10-06T02:27:22.058815+0000","last_fullsized":"2023-10-06T02:27:22.058815+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_clean_scrub_stamp":"2023-10-06T02:27:21.029649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:02:35.238222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.075801+0000","last_change":"2023-10-06T02:27:23.075801+0000","last_active":"2023-10-06T02:27:23.075801+0000","last_peered":"2023-10-06T02:27:23.075801+0000","last_clean":"2023-10-06T02:27:23.075801+0000","last_became_active":"2023-10-06T02:27:23.075506+0000","last_became_peered":"2023-10-06T02:27:23.075506+0000","last_unstale":"2023-10-06T02:27:23.075801+0000","last_undegraded":"2023-10-06T02:27:23.075801+0000","last_fullsized":"2023-10-06T02:27:23.075801+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:53:42.032101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-10-06T02:27:22.064914+0000","last_change":"2023-10-06T02:27:22.056299+0000","last_active":"2023-10-06T02:27:22.064914+0000","last_peered":"2023-10-06T02:27:22.064914+0000","last_clean":"2023-10-06T02:27:22.064914+0000","last_became_active":"2023-10-06T02:27:22.056042+0000","last_became_peered":"2023-10-06T02:27:22.056042+0000","last_unstale":"2023-10-06T02:27:22.064914+0000","last_undegraded":"2023-10-06T02:27:22.064914+0000","last_fullsized":"2023-10-06T02:27:22.064914+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_clean_scrub_stamp":"2023-10-06T02:27:21.029649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:06:33.609185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.077962+0000","last_change":"2023-10-06T02:27:23.077962+0000","last_active":"2023-10-06T02:27:23.077962+0000","last_peered":"2023-10-06T02:27:23.077962+0000","last_clean":"2023-10-06T02:27:23.077962+0000","last_became_active":"2023-10-06T02:27:23.076544+0000","last_became_peered":"2023-10-06T02:27:23.076544+0000","last_unstale":"2023-10-06T02:27:23.077962+0000","last_undegraded":"2023-10-06T02:27:23.077962+0000","last_fullsized":"2023-10-06T02:27:23.077962+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:16:35.087628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-10-06T02:27:22.063934+0000","last_change":"2023-10-06T02:27:22.058785+0000","last_active":"2023-10-06T02:27:22.063934+0000","last_peered":"2023-10-06T02:27:22.063934+0000","last_clean":"2023-10-06T02:27:22.063934+0000","last_became_active":"2023-10-06T02:27:22.058503+0000","last_became_peered":"2023-10-06T02:27:22.058503+0000","last_unstale":"2023-10-06T02:27:22.063934+0000","last_undegraded":"2023-10-06T02:27:22.063934+0000","last_fullsized":"2023-10-06T02:27:22.063934+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_clean_scrub_stamp":"2023-10-06T02:27:21.029649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:34:18.914626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.071649+0000","last_change":"2023-10-06T02:27:23.071649+0000","last_active":"2023-10-06T02:27:23.071649+0000","last_peered":"2023-10-06T02:27:23.071649+0000","last_clean":"2023-10-06T02:27:23.071649+0000","last_became_active":"2023-10-06T02:27:23.071388+0000","last_became_peered":"2023-10-06T02:27:23.071388+0000","last_unstale":"2023-10-06T02:27:23.071649+0000","last_undegraded":"2023-10-06T02:27:23.071649+0000","last_fullsized":"2023-10-06T02:27:23.071649+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:53:42.032101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-10-06T02:27:22.066462+0000","last_change":"2023-10-06T02:27:22.064288+0000","last_active":"2023-10-06T02:27:22.066462+0000","last_peered":"2023-10-06T02:27:22.066462+0000","last_clean":"2023-10-06T02:27:22.066462+0000","last_became_active":"2023-10-06T02:27:22.063986+0000","last_became_peered":"2023-10-06T02:27:22.063986+0000","last_unstale":"2023-10-06T02:27:22.066462+0000","last_undegraded":"2023-10-06T02:27:22.066462+0000","last_fullsized":"2023-10-06T02:27:22.066462+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_clean_scrub_stamp":"2023-10-06T02:27:21.029649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:34:18.914626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.069048+0000","last_change":"2023-10-06T02:27:23.069048+0000","last_active":"2023-10-06T02:27:23.069048+0000","last_peered":"2023-10-06T02:27:23.069048+0000","last_clean":"2023-10-06T02:27:23.069048+0000","last_became_active":"2023-10-06T02:27:23.065975+0000","last_became_peered":"2023-10-06T02:27:23.065975+0000","last_unstale":"2023-10-06T02:27:23.069048+0000","last_undegraded":"2023-10-06T02:27:23.069048+0000","last_fullsized":"2023-10-06T02:27:23.069048+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:23:53.153268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"74'1","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.049948+0000","last_change":"2023-10-06T02:27:22.060727+0000","last_active":"2023-10-06T02:27:23.049948+0000","last_peered":"2023-10-06T02:27:23.049948+0000","last_clean":"2023-10-06T02:27:23.049948+0000","last_became_active":"2023-10-06T02:27:22.060282+0000","last_became_peered":"2023-10-06T02:27:22.060282+0000","last_unstale":"2023-10-06T02:27:23.049948+0000","last_undegraded":"2023-10-06T02:27:23.049948+0000","last_fullsized":"2023-10-06T02:27:23.049948+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_clean_scrub_stamp":"2023-10-06T02:27:21.029649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:13:48.718661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":55,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-10-06T02:27:22.058514+0000","last_change":"2023-10-06T02:23:02.054039+0000","last_active":"2023-10-06T02:27:22.058514+0000","last_peered":"2023-10-06T02:27:22.058514+0000","last_clean":"2023-10-06T02:27:22.058514+0000","last_became_active":"2023-10-06T02:23:02.053742+0000","last_became_peered":"2023-10-06T02:23:02.053742+0000","last_unstale":"2023-10-06T02:27:22.058514+0000","last_undegraded":"2023-10-06T02:27:22.058514+0000","last_fullsized":"2023-10-06T02:27:22.058514+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:58:18.846475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.053194+0000","last_change":"2023-10-06T02:27:23.053194+0000","last_active":"2023-10-06T02:27:23.053194+0000","last_peered":"2023-10-06T02:27:23.053194+0000","last_clean":"2023-10-06T02:27:23.053194+0000","last_became_active":"2023-10-06T02:27:23.052665+0000","last_became_peered":"2023-10-06T02:27:23.052665+0000","last_unstale":"2023-10-06T02:27:23.053194+0000","last_undegraded":"2023-10-06T02:27:23.053194+0000","last_fullsized":"2023-10-06T02:27:23.053194+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:51:19.130397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"74'2","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.081378+0000","last_change":"2023-10-06T02:27:22.058572+0000","last_active":"2023-10-06T02:27:23.081378+0000","last_peered":"2023-10-06T02:27:23.081378+0000","last_clean":"2023-10-06T02:27:23.081378+0000","last_became_active":"2023-10-06T02:27:22.057907+0000","last_became_peered":"2023-10-06T02:27:22.057907+0000","last_unstale":"2023-10-06T02:27:23.081378+0000","last_undegraded":"2023-10-06T02:27:23.081378+0000","last_fullsized":"2023-10-06T02:27:23.081378+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_clean_scrub_stamp":"2023-10-06T02:27:21.029649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:39.645404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":64,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-10-06T02:27:22.046876+0000","last_change":"2023-10-06T02:23:01.988631+0000","last_active":"2023-10-06T02:27:22.046876+0000","last_peered":"2023-10-06T02:27:22.046876+0000","last_clean":"2023-10-06T02:27:22.046876+0000","last_became_active":"2023-10-06T02:23:01.988303+0000","last_became_peered":"2023-10-06T02:23:01.988303+0000","last_unstale":"2023-10-06T02:27:22.046876+0000","last_undegraded":"2023-10-06T02:27:22.046876+0000","last_fullsized":"2023-10-06T02:27:22.046876+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:38.211466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.062179+0000","last_change":"2023-10-06T02:27:23.062179+0000","last_active":"2023-10-06T02:27:23.062179+0000","last_peered":"2023-10-06T02:27:23.062179+0000","last_clean":"2023-10-06T02:27:23.062179+0000","last_became_active":"2023-10-06T02:27:23.061880+0000","last_became_peered":"2023-10-06T02:27:23.061880+0000","last_unstale":"2023-10-06T02:27:23.062179+0000","last_undegraded":"2023-10-06T02:27:23.062179+0000","last_fullsized":"2023-10-06T02:27:23.062179+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:48:25.195788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-10-06T02:27:22.056261+0000","last_change":"2023-10-06T02:27:22.055164+0000","last_active":"2023-10-06T02:27:22.056261+0000","last_peered":"2023-10-06T02:27:22.056261+0000","last_clean":"2023-10-06T02:27:22.056261+0000","last_became_active":"2023-10-06T02:27:22.054830+0000","last_became_peered":"2023-10-06T02:27:22.054830+0000","last_unstale":"2023-10-06T02:27:22.056261+0000","last_undegraded":"2023-10-06T02:27:22.056261+0000","last_fullsized":"2023-10-06T02:27:22.056261+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_clean_scrub_stamp":"2023-10-06T02:27:21.029649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:49:10.287961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":60,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-10-06T02:27:22.040878+0000","last_change":"2023-10-06T02:23:02.054338+0000","last_active":"2023-10-06T02:27:22.040878+0000","last_peered":"2023-10-06T02:27:22.040878+0000","last_clean":"2023-10-06T02:27:22.040878+0000","last_became_active":"2023-10-06T02:23:02.053991+0000","last_became_peered":"2023-10-06T02:23:02.053991+0000","last_unstale":"2023-10-06T02:27:22.040878+0000","last_undegraded":"2023-10-06T02:27:22.040878+0000","last_fullsized":"2023-10-06T02:27:22.040878+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:39:30.192838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.073630+0000","last_change":"2023-10-06T02:27:23.073630+0000","last_active":"2023-10-06T02:27:23.073630+0000","last_peered":"2023-10-06T02:27:23.073630+0000","last_clean":"2023-10-06T02:27:23.073630+0000","last_became_active":"2023-10-06T02:27:23.073346+0000","last_became_peered":"2023-10-06T02:27:23.073346+0000","last_unstale":"2023-10-06T02:27:23.073630+0000","last_undegraded":"2023-10-06T02:27:23.073630+0000","last_fullsized":"2023-10-06T02:27:23.073630+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:02:28.690358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-10-06T02:27:22.056011+0000","last_change":"2023-10-06T02:27:22.054531+0000","last_active":"2023-10-06T02:27:22.056011+0000","last_peered":"2023-10-06T02:27:22.056011+0000","last_clean":"2023-10-06T02:27:22.056011+0000","last_became_active":"2023-10-06T02:27:22.054226+0000","last_became_peered":"2023-10-06T02:27:22.054226+0000","last_unstale":"2023-10-06T02:27:22.056011+0000","last_undegraded":"2023-10-06T02:27:22.056011+0000","last_fullsized":"2023-10-06T02:27:22.056011+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_clean_scrub_stamp":"2023-10-06T02:27:21.029649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:39:13.090897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":64,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-10-06T02:27:22.047004+0000","last_change":"2023-10-06T02:23:01.988140+0000","last_active":"2023-10-06T02:27:22.047004+0000","last_peered":"2023-10-06T02:27:22.047004+0000","last_clean":"2023-10-06T02:27:22.047004+0000","last_became_active":"2023-10-06T02:23:01.987769+0000","last_became_peered":"2023-10-06T02:23:01.987769+0000","last_unstale":"2023-10-06T02:27:22.047004+0000","last_undegraded":"2023-10-06T02:27:22.047004+0000","last_fullsized":"2023-10-06T02:27:22.047004+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:20:08.391912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":207,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-10-06T02:27:22.053229+0000","last_change":"2023-10-06T02:22:55.032519+0000","last_active":"2023-10-06T02:27:22.053229+0000","last_peered":"2023-10-06T02:27:22.053229+0000","last_clean":"2023-10-06T02:27:22.053229+0000","last_became_active":"2023-10-06T02:22:55.031367+0000","last_became_peered":"2023-10-06T02:22:55.031367+0000","last_unstale":"2023-10-06T02:27:22.053229+0000","last_undegraded":"2023-10-06T02:27:22.053229+0000","last_fullsized":"2023-10-06T02:27:22.053229+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:22:53.309097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:22:53.309097+0000","last_clean_scrub_stamp":"2023-10-06T02:22:53.309097+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:53:46.520909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.059939+0000","last_change":"2023-10-06T02:27:23.059939+0000","last_active":"2023-10-06T02:27:23.059939+0000","last_peered":"2023-10-06T02:27:23.059939+0000","last_clean":"2023-10-06T02:27:23.059939+0000","last_became_active":"2023-10-06T02:27:23.059655+0000","last_became_peered":"2023-10-06T02:27:23.059655+0000","last_unstale":"2023-10-06T02:27:23.059939+0000","last_undegraded":"2023-10-06T02:27:23.059939+0000","last_fullsized":"2023-10-06T02:27:23.059939+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:37:59.546106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-10-06T02:27:22.065873+0000","last_change":"2023-10-06T02:27:22.063295+0000","last_active":"2023-10-06T02:27:22.065873+0000","last_peered":"2023-10-06T02:27:22.065873+0000","last_clean":"2023-10-06T02:27:22.065873+0000","last_became_active":"2023-10-06T02:27:22.063033+0000","last_became_peered":"2023-10-06T02:27:22.063033+0000","last_unstale":"2023-10-06T02:27:22.065873+0000","last_undegraded":"2023-10-06T02:27:22.065873+0000","last_fullsized":"2023-10-06T02:27:22.065873+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_clean_scrub_stamp":"2023-10-06T02:27:21.029649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:01:03.767480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":54,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-10-06T02:27:22.053110+0000","last_change":"2023-10-06T02:23:01.990773+0000","last_active":"2023-10-06T02:27:22.053110+0000","last_peered":"2023-10-06T02:27:22.053110+0000","last_clean":"2023-10-06T02:27:22.053110+0000","last_became_active":"2023-10-06T02:23:01.990511+0000","last_became_peered":"2023-10-06T02:23:01.990511+0000","last_unstale":"2023-10-06T02:27:22.053110+0000","last_undegraded":"2023-10-06T02:27:22.053110+0000","last_fullsized":"2023-10-06T02:27:22.053110+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:42:54.797731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.074083+0000","last_change":"2023-10-06T02:27:23.074083+0000","last_active":"2023-10-06T02:27:23.074083+0000","last_peered":"2023-10-06T02:27:23.074083+0000","last_clean":"2023-10-06T02:27:23.074083+0000","last_became_active":"2023-10-06T02:27:23.072575+0000","last_became_peered":"2023-10-06T02:27:23.072575+0000","last_unstale":"2023-10-06T02:27:23.074083+0000","last_undegraded":"2023-10-06T02:27:23.074083+0000","last_fullsized":"2023-10-06T02:27:23.074083+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:51:19.130397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-10-06T02:27:22.066837+0000","last_change":"2023-10-06T02:27:22.065671+0000","last_active":"2023-10-06T02:27:22.066837+0000","last_peered":"2023-10-06T02:27:22.066837+0000","last_clean":"2023-10-06T02:27:22.066837+0000","last_became_active":"2023-10-06T02:27:22.065295+0000","last_became_peered":"2023-10-06T02:27:22.065295+0000","last_unstale":"2023-10-06T02:27:22.066837+0000","last_undegraded":"2023-10-06T02:27:22.066837+0000","last_fullsized":"2023-10-06T02:27:22.066837+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_clean_scrub_stamp":"2023-10-06T02:27:21.029649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:02:35.238222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":54,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-10-06T02:27:22.054290+0000","last_change":"2023-10-06T02:23:02.054069+0000","last_active":"2023-10-06T02:27:22.054290+0000","last_peered":"2023-10-06T02:27:22.054290+0000","last_clean":"2023-10-06T02:27:22.054290+0000","last_became_active":"2023-10-06T02:23:02.053739+0000","last_became_peered":"2023-10-06T02:23:02.053739+0000","last_unstale":"2023-10-06T02:27:22.054290+0000","last_undegraded":"2023-10-06T02:27:22.054290+0000","last_fullsized":"2023-10-06T02:27:22.054290+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:57:48.822608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.073301+0000","last_change":"2023-10-06T02:27:23.073301+0000","last_active":"2023-10-06T02:27:23.073301+0000","last_peered":"2023-10-06T02:27:23.073301+0000","last_clean":"2023-10-06T02:27:23.073301+0000","last_became_active":"2023-10-06T02:27:23.073050+0000","last_became_peered":"2023-10-06T02:27:23.073050+0000","last_unstale":"2023-10-06T02:27:23.073301+0000","last_undegraded":"2023-10-06T02:27:23.073301+0000","last_fullsized":"2023-10-06T02:27:23.073301+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:42:18.952592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"74'1","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.057979+0000","last_change":"2023-10-06T02:27:22.062770+0000","last_active":"2023-10-06T02:27:23.057979+0000","last_peered":"2023-10-06T02:27:23.057979+0000","last_clean":"2023-10-06T02:27:23.057979+0000","last_became_active":"2023-10-06T02:27:22.062427+0000","last_became_peered":"2023-10-06T02:27:22.062427+0000","last_unstale":"2023-10-06T02:27:23.057979+0000","last_undegraded":"2023-10-06T02:27:23.057979+0000","last_fullsized":"2023-10-06T02:27:23.057979+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_clean_scrub_stamp":"2023-10-06T02:27:21.029649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:16:49.364895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":55,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-10-06T02:27:22.058187+0000","last_change":"2023-10-06T02:23:01.989934+0000","last_active":"2023-10-06T02:27:22.058187+0000","last_peered":"2023-10-06T02:27:22.058187+0000","last_clean":"2023-10-06T02:27:22.058187+0000","last_became_active":"2023-10-06T02:23:01.989706+0000","last_became_peered":"2023-10-06T02:23:01.989706+0000","last_unstale":"2023-10-06T02:27:22.058187+0000","last_undegraded":"2023-10-06T02:27:22.058187+0000","last_fullsized":"2023-10-06T02:27:22.058187+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:06.212251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.061826+0000","last_change":"2023-10-06T02:27:23.061826+0000","last_active":"2023-10-06T02:27:23.061826+0000","last_peered":"2023-10-06T02:27:23.061826+0000","last_clean":"2023-10-06T02:27:23.061826+0000","last_became_active":"2023-10-06T02:27:23.061567+0000","last_became_peered":"2023-10-06T02:27:23.061567+0000","last_unstale":"2023-10-06T02:27:23.061826+0000","last_undegraded":"2023-10-06T02:27:23.061826+0000","last_fullsized":"2023-10-06T02:27:23.061826+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:22:05.196287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"74'2","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.079367+0000","last_change":"2023-10-06T02:27:22.066842+0000","last_active":"2023-10-06T02:27:23.079367+0000","last_peered":"2023-10-06T02:27:23.079367+0000","last_clean":"2023-10-06T02:27:23.079367+0000","last_became_active":"2023-10-06T02:27:22.066549+0000","last_became_peered":"2023-10-06T02:27:22.066549+0000","last_unstale":"2023-10-06T02:27:23.079367+0000","last_undegraded":"2023-10-06T02:27:23.079367+0000","last_fullsized":"2023-10-06T02:27:23.079367+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_clean_scrub_stamp":"2023-10-06T02:27:21.029649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:47:30.949998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":57,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-10-06T02:27:22.048084+0000","last_change":"2023-10-06T02:23:01.992935+0000","last_active":"2023-10-06T02:27:22.048084+0000","last_peered":"2023-10-06T02:27:22.048084+0000","last_clean":"2023-10-06T02:27:22.048084+0000","last_became_active":"2023-10-06T02:23:01.992568+0000","last_became_peered":"2023-10-06T02:23:01.992568+0000","last_unstale":"2023-10-06T02:27:22.048084+0000","last_undegraded":"2023-10-06T02:27:22.048084+0000","last_fullsized":"2023-10-06T02:27:22.048084+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:06.212251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574906,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302832,"kb_used_data":1448,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069008,"statfs":{"total":96636764160,"available":96326664192,"internally_reserved":0,"allocated":1482752,"data_stored":984015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:26:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.64100000000000001,"15min":0.63800000000000001},"min":{"1min":0.42399999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.002,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.624,"15min":0.60599999999999998},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.84399999999999997,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.58399999999999996}]},{"osd":1,"last update":"Fri Oct 6 02:26:58 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.71099999999999997,"15min":0.68500000000000005},"min":{"1min":0.46800000000000003,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.95399999999999996,"5min":1.147,"15min":1.147},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.68200000000000005,"15min":0.67500000000000004},"min":{"1min":0.498,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.86899999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.66200000000000003}]},{"osd":2,"last update":"Fri Oct 6 02:26:58 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.69299999999999995,"15min":0.69499999999999995},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.68700000000000006,"15min":0.68999999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.93500000000000005,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.625}]},{"osd":3,"last update":"Fri Oct 6 02:26:58 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.747,"15min":0.73299999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.754,"5min":0.77600000000000002,"15min":0.77800000000000002},"min":{"1min":0.55500000000000005,"5min":0.497,"15min":0.497},"max":{"1min":0.98199999999999998,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.67100000000000004}]},{"osd":4,"last update":"Fri Oct 6 02:26:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.65000000000000002,"15min":0.69599999999999995},"min":{"1min":0.41099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.95199999999999996,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.72599999999999998,"15min":0.77000000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.105,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.68999999999999995}]},{"osd":5,"last update":"Fri Oct 6 02:26:58 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.71499999999999997,"15min":0.73499999999999999},"min":{"1min":0.44700000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.98499999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.71499999999999997,"15min":0.73499999999999999},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.95399999999999996,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.53300000000000003}]},{"osd":6,"last update":"Fri Oct 6 02:26:58 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.77400000000000002,"15min":0.78700000000000003},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0109999999999999,"5min":1.056,"15min":1.056},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.80600000000000005,"15min":0.81599999999999995},"min":{"1min":0.48199999999999998,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.1659999999999999,"5min":1.276,"15min":1.276},"last":0.77800000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574906,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298168,"kb_used_data":880,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073672,"statfs":{"total":96636764160,"available":96331440128,"internally_reserved":0,"allocated":901120,"data_stored":400763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:27:03 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.67100000000000004,"15min":0.66400000000000003},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.91900000000000004,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.71399999999999997,"15min":0.69699999999999995},"min":{"1min":0.53500000000000003,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.95099999999999996,"5min":1.119,"15min":1.119},"last":0.79600000000000004}]},{"osd":1,"last update":"Fri Oct 6 02:27:03 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.64100000000000001,"15min":0.64400000000000002},"min":{"1min":0.433,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67400000000000004,"15min":0.66600000000000004},"min":{"1min":0.48899999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.89500000000000002,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.68400000000000005}]},{"osd":2,"last update":"Fri Oct 6 02:27:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.71199999999999997,"15min":0.70199999999999996},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.96099999999999997,"5min":1.137,"15min":1.137},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72699999999999998,"15min":0.71099999999999997},"min":{"1min":0.55600000000000005,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.89300000000000002,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.752}]},{"osd":3,"last update":"Fri Oct 6 02:27:03 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.68200000000000005,"15min":0.66000000000000003},"min":{"1min":0.52000000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.97099999999999997,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71299999999999997,"15min":0.70299999999999996},"min":{"1min":0.5,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.91600000000000004,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.621}]},{"osd":4,"last update":"Fri Oct 6 02:27:03 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63900000000000001,"15min":0.63400000000000001},"min":{"1min":0.36199999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.90000000000000002,"5min":1.095,"15min":1.095},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.67600000000000005,"15min":0.65600000000000003},"min":{"1min":0.46200000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.999,"5min":1.29,"15min":1.29},"last":0.69799999999999995}]},{"osd":5,"last update":"Fri Oct 6 02:27:03 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.68100000000000005,"15min":0.69699999999999995},"min":{"1min":0.48399999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.94299999999999995,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.70699999999999996,"15min":0.70199999999999996},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.86099999999999999,"5min":1.02,"15min":1.02},"last":0.56499999999999995}]},{"osd":7,"last update":"Fri Oct 6 02:27:03 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.74199999999999999,"15min":0.746},"min":{"1min":0.41499999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.95399999999999996,"5min":1.458,"15min":1.458},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76000000000000001,"15min":0.75900000000000001},"min":{"1min":0.54600000000000004,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.96499999999999997,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.70799999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574905,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302252,"kb_used_data":868,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069588,"statfs":{"total":96636764160,"available":96327258112,"internally_reserved":0,"allocated":888832,"data_stored":393647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.45200000000000001,"15min":0.46300000000000002},"min":{"1min":0.314,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.61299999999999999,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.499,"15min":0.53100000000000003},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.71799999999999997,"5min":1.788,"15min":1.788},"last":0.70099999999999996}]},{"osd":2,"last update":"Fri Oct 6 02:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.48699999999999999,"15min":0.53000000000000003},"min":{"1min":0.35599999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.57499999999999996,"5min":2.0840000000000001,"15min":2.0840000000000001},"last":0.38},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.53300000000000003,"15min":0.56200000000000006},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.69299999999999995,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.436}]},{"osd":3,"last update":"Fri Oct 6 02:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.61099999999999999,"15min":0.61799999999999999},"min":{"1min":0.44600000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.80300000000000005,"5min":1.266,"15min":1.266},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.59199999999999997,"15min":0.64800000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.70499999999999996,"5min":1.861,"15min":1.861},"last":0.503}]},{"osd":4,"last update":"Fri Oct 6 02:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.83299999999999996,"15min":0.86499999999999999},"min":{"1min":0.61399999999999999,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.91000000000000003,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":0.624},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.83899999999999997,"15min":0.86599999999999999},"min":{"1min":0.52900000000000003,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.93100000000000005,"5min":2.0179999999999998,"15min":2.0179999999999998},"last":0.51400000000000001}]},{"osd":5,"last update":"Fri Oct 6 02:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.81200000000000006,"15min":0.87},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.89000000000000001,"5min":1.9830000000000001,"15min":1.9830000000000001},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.79600000000000004,"15min":0.82899999999999996},"min":{"1min":0.53600000000000003,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.94699999999999995,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.64900000000000002}]},{"osd":6,"last update":"Fri Oct 6 02:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.84499999999999997,"15min":0.876},"min":{"1min":0.57799999999999996,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.04,"5min":1.829,"15min":1.829},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.874,"15min":0.92200000000000004},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.97799999999999998,"5min":2.052,"15min":2.052},"last":0.71899999999999997}]},{"osd":7,"last update":"Fri Oct 6 02:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.81899999999999995,"15min":0.84299999999999997},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.91300000000000003,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.84099999999999997,"15min":0.88700000000000001},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.995,"5min":1.895,"15min":1.895},"last":0.55000000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574906,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302832,"kb_used_data":1448,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069008,"statfs":{"total":96636764160,"available":96326664192,"internally_reserved":0,"allocated":1482752,"data_stored":984015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 02:27:05 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.44,"15min":0.45600000000000002},"min":{"1min":0.32000000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.54300000000000004,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.439,"5min":0.47399999999999998,"15min":0.48199999999999998},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.58799999999999997,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.38200000000000001}]},{"osd":2,"last update":"Fri Oct 6 02:27:05 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.498,"15min":0.51400000000000001},"min":{"1min":0.40899999999999997,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.60699999999999998,"5min":1.069,"15min":1.069},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.51100000000000001,"15min":0.52200000000000002},"min":{"1min":0.39600000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.73099999999999998,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.45000000000000001}]},{"osd":3,"last update":"Fri Oct 6 02:27:05 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.49399999999999999,"15min":0.50600000000000001},"min":{"1min":0.34100000000000003,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.59599999999999997,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59499999999999997,"15min":0.59699999999999998},"min":{"1min":0.47299999999999998,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.73099999999999998,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.54900000000000004}]},{"osd":4,"last update":"Fri Oct 6 02:27:05 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.77000000000000002,"15min":0.80300000000000005},"min":{"1min":0.54800000000000004,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.86899999999999999,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.77700000000000002,"15min":0.80300000000000005},"min":{"1min":0.58099999999999996,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0469999999999999,"5min":1.304,"15min":1.304},"last":0.86899999999999999}]},{"osd":5,"last update":"Fri Oct 6 02:27:05 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.76600000000000001,"15min":0.78800000000000003},"min":{"1min":0.52600000000000002,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.94199999999999995,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.98999999999999999},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.79100000000000004,"15min":0.80300000000000005},"min":{"1min":0.59699999999999998,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.018,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.77200000000000002}]},{"osd":6,"last update":"Fri Oct 6 02:27:05 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.81299999999999994,"15min":0.82599999999999996},"min":{"1min":0.59699999999999998,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.96399999999999997,"5min":1.167,"15min":1.167},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.80500000000000005,"15min":0.83499999999999996},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.0349999999999999,"5min":1.069,"15min":1.069},"last":0.80900000000000005}]},{"osd":7,"last update":"Fri Oct 6 02:27:05 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.80500000000000005,"15min":0.82899999999999996},"min":{"1min":0.56899999999999995,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.98499999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.79600000000000004,"15min":0.82899999999999996},"min":{"1min":0.54700000000000004,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.002,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.79300000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574907,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302252,"kb_used_data":868,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069588,"statfs":{"total":96636764160,"available":96327258112,"internally_reserved":0,"allocated":888832,"data_stored":393647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:27:02 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.434,"15min":0.441},"min":{"1min":0.312,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.54100000000000004,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.501},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.48299999999999998,"15min":0.48799999999999999},"min":{"1min":0.32800000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.88700000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.64100000000000001}]},{"osd":1,"last update":"Fri Oct 6 02:27:02 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.5,"15min":0.50700000000000001},"min":{"1min":0.36699999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.63900000000000001,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.56399999999999995,"15min":0.61299999999999999},"min":{"1min":0.39600000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.70899999999999996,"5min":1.345,"15min":1.345},"last":0.57299999999999995}]},{"osd":3,"last update":"Fri Oct 6 02:27:02 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.52200000000000002,"15min":0.53600000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.69699999999999995,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.60699999999999998,"15min":0.624},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.72799999999999998,"5min":1.028,"15min":1.028},"last":0.66800000000000004}]},{"osd":4,"last update":"Fri Oct 6 02:27:02 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.76400000000000001,"15min":0.78100000000000003},"min":{"1min":0.55500000000000005,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.89300000000000002,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.82399999999999995,"15min":0.85499999999999998},"min":{"1min":0.59999999999999998,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.95999999999999996,"5min":1.621,"15min":1.621},"last":1.006}]},{"osd":5,"last update":"Fri Oct 6 02:27:02 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.77500000000000002,"15min":0.80800000000000005},"min":{"1min":0.57099999999999995,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.94899999999999995,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.80000000000000004,"15min":0.82899999999999996},"min":{"1min":0.60399999999999998,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":0.98199999999999998,"5min":1.258,"15min":1.258},"last":0.85299999999999998}]},{"osd":6,"last update":"Fri Oct 6 02:27:02 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.82299999999999995,"15min":0.85399999999999998},"min":{"1min":0.59099999999999997,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.98099999999999998,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":1.042},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.78900000000000003,"15min":0.79900000000000004},"min":{"1min":0.623,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.0109999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.89600000000000002}]},{"osd":7,"last update":"Fri Oct 6 02:27:02 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.85299999999999998,"15min":0.89200000000000002},"min":{"1min":0.66200000000000003,"5min":0.499,"15min":0.499},"max":{"1min":0.95999999999999996,"5min":1.585,"15min":1.585},"last":1.0660000000000001},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.81100000000000005,"15min":0.83699999999999997},"min":{"1min":0.58999999999999997,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.94399999999999995,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.82899999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574904,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302252,"kb_used_data":868,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069588,"statfs":{"total":96636764160,"available":96327258112,"internally_reserved":0,"allocated":888832,"data_stored":393647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:27:02 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48899999999999999,"15min":0.51700000000000002},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.69599999999999995,"5min":2.024,"15min":2.024},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.51200000000000001,"15min":0.52800000000000002},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.67400000000000004,"5min":2.181,"15min":2.181},"last":0.371}]},{"osd":1,"last update":"Fri Oct 6 02:27:02 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.58799999999999997,"15min":0.70299999999999996},"min":{"1min":0.39600000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.59299999999999997,"5min":4.492,"15min":4.492},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.63700000000000001,"15min":0.745},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.80000000000000004,"5min":4.569,"15min":4.569},"last":0.39600000000000002}]},{"osd":2,"last update":"Fri Oct 6 02:27:02 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.65800000000000003,"15min":0.67200000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.91100000000000003,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.625,"15min":0.65900000000000003},"min":{"1min":0.46200000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.94199999999999995,"5min":2.4260000000000002,"15min":2.4260000000000002},"last":0.63}]},{"osd":4,"last update":"Fri Oct 6 02:27:02 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.78600000000000003,"15min":0.83899999999999997},"min":{"1min":0.61599999999999999,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.96699999999999997,"5min":2.214,"15min":2.214},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.78600000000000003,"15min":0.81699999999999995},"min":{"1min":0.50800000000000001,"5min":0.5,"15min":0.5},"max":{"1min":0.94399999999999995,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.91500000000000004}]},{"osd":5,"last update":"Fri Oct 6 02:27:02 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.80500000000000005,"15min":0.82899999999999996},"min":{"1min":0.57899999999999996,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.97699999999999998,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.81100000000000005,"15min":0.86499999999999999},"min":{"1min":0.63800000000000001,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.0289999999999999,"5min":2.242,"15min":2.242},"last":0.83899999999999997}]},{"osd":6,"last update":"Fri Oct 6 02:27:02 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.79200000000000004,"15min":0.82399999999999995},"min":{"1min":0.59399999999999997,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.0049999999999999,"5min":1.357,"15min":1.357},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.81999999999999995,"15min":0.83799999999999997},"min":{"1min":0.54600000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.0529999999999999,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.77900000000000003}]},{"osd":7,"last update":"Fri Oct 6 02:27:02 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.83399999999999996,"15min":0.87},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.077,"5min":2.27,"15min":2.27},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.83099999999999996,"15min":0.86399999999999999},"min":{"1min":0.55600000000000005,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.056,"5min":1.325,"15min":1.325},"last":0.875}]}]},{"osd":4,"up_from":13,"seq":55834574906,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302252,"kb_used_data":868,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069588,"statfs":{"total":96636764160,"available":96327258112,"internally_reserved":0,"allocated":888832,"data_stored":393647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:27:03 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.86599999999999999,"15min":1.0549999999999999},"min":{"1min":0.45700000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.90700000000000003,"5min":12.337,"15min":12.337},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.81399999999999995,"15min":0.95099999999999996},"min":{"1min":0.496,"5min":0.438,"15min":0.438},"max":{"1min":0.91700000000000004,"5min":9.6539999999999999,"15min":9.6539999999999999},"last":0.56100000000000005}]},{"osd":1,"last update":"Fri Oct 6 02:27:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.69999999999999996,"15min":0.70599999999999996},"min":{"1min":0.53800000000000003,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.96999999999999997,"5min":3.181,"15min":3.181},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.876,"15min":1.0740000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.90000000000000002,"5min":12.946999999999999,"15min":12.946999999999999},"last":0.67200000000000004}]},{"osd":2,"last update":"Fri Oct 6 02:27:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.89700000000000002,"15min":1.103},"min":{"1min":0.44400000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.93300000000000005,"5min":12.377000000000001,"15min":12.377000000000001},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.80100000000000005,"15min":0.89500000000000002},"min":{"1min":0.53800000000000003,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.95899999999999996,"5min":6.3220000000000001,"15min":6.3220000000000001},"last":0.73399999999999999}]},{"osd":3,"last update":"Fri Oct 6 02:27:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.873,"15min":1.0309999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.88500000000000001,"5min":9.7720000000000002,"15min":9.7720000000000002},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.745,"15min":0.78500000000000003},"min":{"1min":0.498,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.91600000000000004,"5min":3.0840000000000001,"15min":3.0840000000000001},"last":0.70999999999999996}]},{"osd":5,"last update":"Fri Oct 6 02:27:03 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.76100000000000001,"15min":0.86899999999999999},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.82899999999999996,"5min":6.2690000000000001,"15min":6.2690000000000001},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.874,"15min":1.0840000000000001},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.90900000000000003,"5min":10.769,"15min":10.769},"last":0.747}]},{"osd":6,"last update":"Fri Oct 6 02:27:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.71199999999999997,"15min":0.77000000000000002},"min":{"1min":0.44,"5min":0.432,"15min":0.432},"max":{"1min":0.84299999999999997,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.85599999999999998,"15min":1.0029999999999999},"min":{"1min":0.49199999999999999,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.94399999999999995,"5min":8.0899999999999999,"15min":8.0899999999999999},"last":0.76600000000000001}]},{"osd":7,"last update":"Fri Oct 6 02:27:03 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.92600000000000005,"15min":1.113},"min":{"1min":0.50600000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.93899999999999995,"5min":9.7089999999999996,"15min":9.7089999999999996},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.88800000000000001,"15min":1.034},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.97799999999999998,"5min":7.6719999999999997,"15min":7.6719999999999997},"last":0.86099999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574906,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298156,"kb_used_data":868,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073684,"statfs":{"total":96636764160,"available":96331452416,"internally_reserved":0,"allocated":888832,"data_stored":393647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:27:00 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.64200000000000002,"15min":0.626},"min":{"1min":0.44700000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.84899999999999998,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.64900000000000002,"15min":0.60799999999999998},"min":{"1min":0.41699999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.93999999999999995,"5min":1.026,"15min":1.026},"last":0.58199999999999996}]},{"osd":1,"last update":"Fri Oct 6 02:27:00 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.67000000000000004,"15min":0.65000000000000002},"min":{"1min":0.438,"5min":0.434,"15min":0.434},"max":{"1min":0.95399999999999996,"5min":1.089,"15min":1.089},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.68899999999999995,"15min":0.67500000000000004},"min":{"1min":0.53200000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.88300000000000001,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.52700000000000002}]},{"osd":2,"last update":"Fri Oct 6 02:27:00 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.69599999999999995,"15min":0.66800000000000004},"min":{"1min":0.499,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.98999999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.70099999999999996,"15min":0.68300000000000005},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.96099999999999997,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.497}]},{"osd":3,"last update":"Fri Oct 6 02:27:00 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.748,"15min":0.72599999999999998},"min":{"1min":0.58399999999999996,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.93999999999999995,"5min":1.042,"15min":1.042},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.78200000000000003,"15min":0.76400000000000001},"min":{"1min":0.59499999999999997,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.0489999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.73699999999999999}]},{"osd":4,"last update":"Fri Oct 6 02:27:00 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.66100000000000003,"15min":0.67300000000000004},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.89000000000000001,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.624,"5min":0.67100000000000004,"15min":0.68200000000000005},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.86499999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.47999999999999998}]},{"osd":6,"last update":"Fri Oct 6 02:27:00 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.73899999999999999,"15min":0.74299999999999999},"min":{"1min":0.45700000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.0549999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.81499999999999995,"15min":0.82099999999999995},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.03,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.754}]},{"osd":7,"last update":"Fri Oct 6 02:27:00 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.75900000000000001,"15min":0.76300000000000001},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.0009999999999999,"5min":1.089,"15min":1.089},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.81799999999999995,"15min":0.83599999999999997},"min":{"1min":0.46999999999999997,"5min":0.437,"15min":0.437},"max":{"1min":1.0229999999999999,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.63200000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T02:27:27.425 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T02:27:27.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.642+0000 7fd10579d700 1 Processor -- start 2023-10-06T02:27:27.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.642+0000 7fd10579d700 1 -- start start 2023-10-06T02:27:27.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.644+0000 7fd10579d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd10013fb20 0x7fd10015a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:27.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.644+0000 7fd10579d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd10013fef0 0x7fd1000c7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:27.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.644+0000 7fd10579d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd10015ae70 con 0x7fd10006aaa0 2023-10-06T02:27:27.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.644+0000 7fd10579d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd10006af60 con 0x7fd10013fb20 2023-10-06T02:27:27.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.644+0000 7fd10579d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd10006b240 con 0x7fd10013fef0 2023-10-06T02:27:27.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.644+0000 7fd0fffff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd10013fb20 0x7fd10015a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:27.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.644+0000 7fd0fffff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd10013fb20 0x7fd10015a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38274/0 (socket says 172.21.15.31:38274) 2023-10-06T02:27:27.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.644+0000 7fd0fffff700 1 -- 172.21.15.31:0/1949842533 learned_addr learned my addr 172.21.15.31:0/1949842533 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:27.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.644+0000 7fd104f9c700 1 --1- 172.21.15.31:0/1949842533 >> v1:172.21.15.31:6789/0 conn(0x7fd10006aaa0 0x7fd1000c6770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:59714/0 (socket says 172.21.15.31:59714) 2023-10-06T02:27:27.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.644+0000 7fd0ff7fe700 1 --2- 172.21.15.31:0/1949842533 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd10013fef0 0x7fd1000c7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:27.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.645+0000 7fd0fffff700 1 -- 172.21.15.31:0/1949842533 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd10013fef0 msgr2=0x7fd1000c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:27.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.645+0000 7fd0fffff700 1 --2- 172.21.15.31:0/1949842533 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd10013fef0 0x7fd1000c7990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:27.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.645+0000 7fd0fffff700 1 -- 172.21.15.31:0/1949842533 >> v1:172.21.15.31:6789/0 conn(0x7fd10006aaa0 legacy=0x7fd1000c6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:27.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.645+0000 7fd0fffff700 1 -- 172.21.15.31:0/1949842533 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd10006b520 con 0x7fd10013fb20 2023-10-06T02:27:27.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.645+0000 7fd0fffff700 1 --2- 172.21.15.31:0/1949842533 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd10013fb20 0x7fd10015a930 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fd0f00099b0 tx=0x7fd0f0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c07601b4051f3fb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:27.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.645+0000 7fd0feffd700 1 -- 172.21.15.31:0/1949842533 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f0009ea0 con 0x7fd10013fb20 2023-10-06T02:27:27.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.645+0000 7fd0feffd700 1 -- 172.21.15.31:0/1949842533 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0f000b860 con 0x7fd10013fb20 2023-10-06T02:27:27.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.645+0000 7fd0feffd700 1 -- 172.21.15.31:0/1949842533 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f000bc00 con 0x7fd10013fb20 2023-10-06T02:27:27.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.646+0000 7fd10579d700 1 -- 172.21.15.31:0/1949842533 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd10013fb20 msgr2=0x7fd10015a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:27.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.646+0000 7fd10579d700 1 --2- 172.21.15.31:0/1949842533 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd10013fb20 0x7fd10015a930 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fd0f00099b0 tx=0x7fd0f0009250 comp rx=0 tx=0).stop 2023-10-06T02:27:27.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.646+0000 7fd10579d700 1 -- 172.21.15.31:0/1949842533 shutdown_connections 2023-10-06T02:27:27.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.646+0000 7fd10579d700 1 --2- 172.21.15.31:0/1949842533 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd10013fb20 0x7fd10015a930 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:27.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.646+0000 7fd10579d700 1 --2- 172.21.15.31:0/1949842533 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd10013fef0 0x7fd1000c7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:27.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.646+0000 7fd10579d700 1 -- 172.21.15.31:0/1949842533 >> 172.21.15.31:0/1949842533 conn(0x7fd10006d170 msgr2=0x7fd1000cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:27.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.646+0000 7fd10579d700 1 -- 172.21.15.31:0/1949842533 shutdown_connections 2023-10-06T02:27:27.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.646+0000 7fd10579d700 1 -- 172.21.15.31:0/1949842533 wait complete. 2023-10-06T02:27:27.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.647+0000 7fd10579d700 1 Processor -- start 2023-10-06T02:27:27.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.647+0000 7fd10579d700 1 -- start start 2023-10-06T02:27:27.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.648+0000 7fd10579d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd10006aaa0 0x7fd1000adec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:27.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.648+0000 7fd10579d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd10013fb20 0x7fd100159330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:27.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.648+0000 7fd10579d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd10013fef0 0x7fd1000ac150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:27.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.648+0000 7fd10579d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd10016d150 con 0x7fd10013fef0 2023-10-06T02:27:27.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.648+0000 7fd10579d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd10016d9f0 con 0x7fd10006aaa0 2023-10-06T02:27:27.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.648+0000 7fd10579d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1000c2490 con 0x7fd10013fb20 2023-10-06T02:27:27.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.648+0000 7fd104f9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd10013fef0 0x7fd1000ac150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:27.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.649+0000 7fd104f9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd10013fef0 0x7fd1000ac150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38280/0 (socket says 172.21.15.31:38280) 2023-10-06T02:27:27.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.649+0000 7fd104f9c700 1 -- 172.21.15.31:0/2239750028 learned_addr learned my addr 172.21.15.31:0/2239750028 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:27.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.649+0000 7fd0fffff700 1 --2- 172.21.15.31:0/2239750028 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd10006aaa0 0x7fd1000adec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:27.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.649+0000 7fd0ff7fe700 1 --2- 172.21.15.31:0/2239750028 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd10013fb20 0x7fd100159330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:27.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.649+0000 7fd104f9c700 1 -- 172.21.15.31:0/2239750028 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd10013fb20 msgr2=0x7fd100159330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:27.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.649+0000 7fd104f9c700 1 --2- 172.21.15.31:0/2239750028 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd10013fb20 0x7fd100159330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:27.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.649+0000 7fd104f9c700 1 -- 172.21.15.31:0/2239750028 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd10006aaa0 msgr2=0x7fd1000adec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:27.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.649+0000 7fd104f9c700 1 --2- 172.21.15.31:0/2239750028 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd10006aaa0 0x7fd1000adec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:27.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.649+0000 7fd104f9c700 1 -- 172.21.15.31:0/2239750028 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0f00094f0 con 0x7fd10013fef0 2023-10-06T02:27:27.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.649+0000 7fd104f9c700 1 --2- 172.21.15.31:0/2239750028 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd10013fef0 0x7fd1000ac150 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fd0f800cca0 tx=0x7fd0f8001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:27.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.649+0000 7fd0e7fff700 1 -- 172.21.15.31:0/2239750028 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f8005df0 con 0x7fd10013fef0 2023-10-06T02:27:27.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.649+0000 7fd0e7fff700 1 -- 172.21.15.31:0/2239750028 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0f800a680 con 0x7fd10013fef0 2023-10-06T02:27:27.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.649+0000 7fd0e7fff700 1 -- 172.21.15.31:0/2239750028 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f800aa00 con 0x7fd10013fef0 2023-10-06T02:27:27.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.649+0000 7fd10579d700 1 -- 172.21.15.31:0/2239750028 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1000ac850 con 0x7fd10013fef0 2023-10-06T02:27:27.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.650+0000 7fd10579d700 1 -- 172.21.15.31:0/2239750028 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd10016d680 con 0x7fd10013fef0 2023-10-06T02:27:27.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.651+0000 7fd10579d700 1 -- 172.21.15.31:0/2239750028 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd10016dba0 con 0x7fd10013fef0 2023-10-06T02:27:27.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.652+0000 7fd0e7fff700 1 -- 172.21.15.31:0/2239750028 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fd0f80070f0 con 0x7fd10013fef0 2023-10-06T02:27:27.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.652+0000 7fd0e7fff700 1 --2- 172.21.15.31:0/2239750028 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fd0ec067290 0x7fd0ec069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:27.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.652+0000 7fd0e7fff700 1 -- 172.21.15.31:0/2239750028 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd0f8084a30 con 0x7fd10013fef0 2023-10-06T02:27:27.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.653+0000 7fd0fffff700 1 --2- 172.21.15.31:0/2239750028 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fd0ec067290 0x7fd0ec069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:27.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.653+0000 7fd0fffff700 1 --2- 172.21.15.31:0/2239750028 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fd0ec067290 0x7fd0ec069740 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd0f00099b0 tx=0x7fd0f0017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:27.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.656+0000 7fd0e7fff700 1 -- 172.21.15.31:0/2239750028 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd10016dba0 con 0x7fd10013fef0 2023-10-06T02:27:27.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.852+0000 7fd10579d700 1 -- 172.21.15.31:0/2239750028 --> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd10015ae70 con 0x7fd0ec067290 2023-10-06T02:27:27.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.862+0000 7fd0e7fff700 1 -- 172.21.15.31:0/2239750028 <== mgr.4107 v2:172.21.15.31:6832/85292 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311278 (secure 0 0 0) 0x7fd10015ae70 con 0x7fd0ec067290 2023-10-06T02:27:27.867 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:27:27.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.866+0000 7fd10579d700 1 -- 172.21.15.31:0/2239750028 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fd0ec067290 msgr2=0x7fd0ec069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:27.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.866+0000 7fd10579d700 1 --2- 172.21.15.31:0/2239750028 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fd0ec067290 0x7fd0ec069740 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd0f00099b0 tx=0x7fd0f0017000 comp rx=0 tx=0).stop 2023-10-06T02:27:27.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.866+0000 7fd10579d700 1 -- 172.21.15.31:0/2239750028 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd10013fef0 msgr2=0x7fd1000ac150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:27.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.866+0000 7fd10579d700 1 --2- 172.21.15.31:0/2239750028 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd10013fef0 0x7fd1000ac150 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fd0f800cca0 tx=0x7fd0f8001a40 comp rx=0 tx=0).stop 2023-10-06T02:27:27.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.866+0000 7fd10579d700 1 -- 172.21.15.31:0/2239750028 shutdown_connections 2023-10-06T02:27:27.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.866+0000 7fd10579d700 1 --2- 172.21.15.31:0/2239750028 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd10013fef0 0x7fd1000ac150 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:27.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.866+0000 7fd10579d700 1 --2- 172.21.15.31:0/2239750028 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd10006aaa0 0x7fd1000adec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:27.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.866+0000 7fd10579d700 1 --2- 172.21.15.31:0/2239750028 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fd0ec067290 0x7fd0ec069740 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:27.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.866+0000 7fd10579d700 1 --2- 172.21.15.31:0/2239750028 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd10013fb20 0x7fd100159330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:27.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.866+0000 7fd10579d700 1 -- 172.21.15.31:0/2239750028 >> 172.21.15.31:0/2239750028 conn(0x7fd10006d170 msgr2=0x7fd10013d430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:27.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.867+0000 7fd10579d700 1 -- 172.21.15.31:0/2239750028 shutdown_connections 2023-10-06T02:27:27.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:27.867+0000 7fd10579d700 1 -- 172.21.15.31:0/2239750028 wait complete. 2023-10-06T02:27:27.871 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-10-06T02:27:27.885 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":205,"stamp":"2023-10-06T02:27:27.173505+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2410996,"kb_used_data":8116,"kb_used_omap":0,"kb_used_meta":2402816,"kb_avail":752563724,"statfs":{"total":773094113280,"available":770625253376,"internally_reserved":0,"allocated":8310784,"data_stored":4337028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2460483584},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000980"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.068773+0000","last_change":"2023-10-06T02:27:23.068773+0000","last_active":"2023-10-06T02:27:23.068773+0000","last_peered":"2023-10-06T02:27:23.068773+0000","last_clean":"2023-10-06T02:27:23.068773+0000","last_became_active":"2023-10-06T02:27:23.068472+0000","last_became_peered":"2023-10-06T02:27:23.068472+0000","last_unstale":"2023-10-06T02:27:23.068773+0000","last_undegraded":"2023-10-06T02:27:23.068773+0000","last_fullsized":"2023-10-06T02:27:23.068773+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:51:19.130397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.063973+0000","last_change":"2023-10-06T02:27:23.063973+0000","last_active":"2023-10-06T02:27:23.063973+0000","last_peered":"2023-10-06T02:27:23.063973+0000","last_clean":"2023-10-06T02:27:23.063973+0000","last_became_active":"2023-10-06T02:27:23.063727+0000","last_became_peered":"2023-10-06T02:27:23.063727+0000","last_unstale":"2023-10-06T02:27:23.063973+0000","last_undegraded":"2023-10-06T02:27:23.063973+0000","last_fullsized":"2023-10-06T02:27:23.063973+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:42:18.952592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.078993+0000","last_change":"2023-10-06T02:27:23.078993+0000","last_active":"2023-10-06T02:27:23.078993+0000","last_peered":"2023-10-06T02:27:23.078993+0000","last_clean":"2023-10-06T02:27:23.078993+0000","last_became_active":"2023-10-06T02:27:23.078693+0000","last_became_peered":"2023-10-06T02:27:23.078693+0000","last_unstale":"2023-10-06T02:27:23.078993+0000","last_undegraded":"2023-10-06T02:27:23.078993+0000","last_fullsized":"2023-10-06T02:27:23.078993+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:38:02.227760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.052031+0000","last_change":"2023-10-06T02:27:23.052031+0000","last_active":"2023-10-06T02:27:23.052031+0000","last_peered":"2023-10-06T02:27:23.052031+0000","last_clean":"2023-10-06T02:27:23.052031+0000","last_became_active":"2023-10-06T02:27:23.051722+0000","last_became_peered":"2023-10-06T02:27:23.051722+0000","last_unstale":"2023-10-06T02:27:23.052031+0000","last_undegraded":"2023-10-06T02:27:23.052031+0000","last_fullsized":"2023-10-06T02:27:23.052031+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:33:15.965337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.064970+0000","last_change":"2023-10-06T02:27:23.064970+0000","last_active":"2023-10-06T02:27:23.064970+0000","last_peered":"2023-10-06T02:27:23.064970+0000","last_clean":"2023-10-06T02:27:23.064970+0000","last_became_active":"2023-10-06T02:27:23.064713+0000","last_became_peered":"2023-10-06T02:27:23.064713+0000","last_unstale":"2023-10-06T02:27:23.064970+0000","last_undegraded":"2023-10-06T02:27:23.064970+0000","last_fullsized":"2023-10-06T02:27:23.064970+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:17:39.514674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.068551+0000","last_change":"2023-10-06T02:27:23.068551+0000","last_active":"2023-10-06T02:27:23.068551+0000","last_peered":"2023-10-06T02:27:23.068551+0000","last_clean":"2023-10-06T02:27:23.068551+0000","last_became_active":"2023-10-06T02:27:23.067733+0000","last_became_peered":"2023-10-06T02:27:23.067733+0000","last_unstale":"2023-10-06T02:27:23.068551+0000","last_undegraded":"2023-10-06T02:27:23.068551+0000","last_fullsized":"2023-10-06T02:27:23.068551+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:59:13.916149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.072647+0000","last_change":"2023-10-06T02:27:23.072647+0000","last_active":"2023-10-06T02:27:23.072647+0000","last_peered":"2023-10-06T02:27:23.072647+0000","last_clean":"2023-10-06T02:27:23.072647+0000","last_became_active":"2023-10-06T02:27:23.072179+0000","last_became_peered":"2023-10-06T02:27:23.072179+0000","last_unstale":"2023-10-06T02:27:23.072647+0000","last_undegraded":"2023-10-06T02:27:23.072647+0000","last_fullsized":"2023-10-06T02:27:23.072647+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:19:46.513602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.064311+0000","last_change":"2023-10-06T02:27:23.064311+0000","last_active":"2023-10-06T02:27:23.064311+0000","last_peered":"2023-10-06T02:27:23.064311+0000","last_clean":"2023-10-06T02:27:23.064311+0000","last_became_active":"2023-10-06T02:27:23.064037+0000","last_became_peered":"2023-10-06T02:27:23.064037+0000","last_unstale":"2023-10-06T02:27:23.064311+0000","last_undegraded":"2023-10-06T02:27:23.064311+0000","last_fullsized":"2023-10-06T02:27:23.064311+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:39:30.046943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.059635+0000","last_change":"2023-10-06T02:27:23.059635+0000","last_active":"2023-10-06T02:27:23.059635+0000","last_peered":"2023-10-06T02:27:23.059635+0000","last_clean":"2023-10-06T02:27:23.059635+0000","last_became_active":"2023-10-06T02:27:23.059300+0000","last_became_peered":"2023-10-06T02:27:23.059300+0000","last_unstale":"2023-10-06T02:27:23.059635+0000","last_undegraded":"2023-10-06T02:27:23.059635+0000","last_fullsized":"2023-10-06T02:27:23.059635+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:40:17.090588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.061936+0000","last_change":"2023-10-06T02:27:23.061936+0000","last_active":"2023-10-06T02:27:23.061936+0000","last_peered":"2023-10-06T02:27:23.061936+0000","last_clean":"2023-10-06T02:27:23.061936+0000","last_became_active":"2023-10-06T02:27:23.061674+0000","last_became_peered":"2023-10-06T02:27:23.061674+0000","last_unstale":"2023-10-06T02:27:23.061936+0000","last_undegraded":"2023-10-06T02:27:23.061936+0000","last_fullsized":"2023-10-06T02:27:23.061936+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:32:00.380963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.053395+0000","last_change":"2023-10-06T02:27:23.053395+0000","last_active":"2023-10-06T02:27:23.053395+0000","last_peered":"2023-10-06T02:27:23.053395+0000","last_clean":"2023-10-06T02:27:23.053395+0000","last_became_active":"2023-10-06T02:27:23.052725+0000","last_became_peered":"2023-10-06T02:27:23.052725+0000","last_unstale":"2023-10-06T02:27:23.053395+0000","last_undegraded":"2023-10-06T02:27:23.053395+0000","last_fullsized":"2023-10-06T02:27:23.053395+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:16:23.777928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.065611+0000","last_change":"2023-10-06T02:27:23.065611+0000","last_active":"2023-10-06T02:27:23.065611+0000","last_peered":"2023-10-06T02:27:23.065611+0000","last_clean":"2023-10-06T02:27:23.065611+0000","last_became_active":"2023-10-06T02:27:23.065320+0000","last_became_peered":"2023-10-06T02:27:23.065320+0000","last_unstale":"2023-10-06T02:27:23.065611+0000","last_undegraded":"2023-10-06T02:27:23.065611+0000","last_fullsized":"2023-10-06T02:27:23.065611+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:03:04.467972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.063100+0000","last_change":"2023-10-06T02:27:23.063100+0000","last_active":"2023-10-06T02:27:23.063100+0000","last_peered":"2023-10-06T02:27:23.063100+0000","last_clean":"2023-10-06T02:27:23.063100+0000","last_became_active":"2023-10-06T02:27:23.062823+0000","last_became_peered":"2023-10-06T02:27:23.062823+0000","last_unstale":"2023-10-06T02:27:23.063100+0000","last_undegraded":"2023-10-06T02:27:23.063100+0000","last_fullsized":"2023-10-06T02:27:23.063100+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:50:09.409985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.068744+0000","last_change":"2023-10-06T02:27:23.068744+0000","last_active":"2023-10-06T02:27:23.068744+0000","last_peered":"2023-10-06T02:27:23.068744+0000","last_clean":"2023-10-06T02:27:23.068744+0000","last_became_active":"2023-10-06T02:27:23.068141+0000","last_became_peered":"2023-10-06T02:27:23.068141+0000","last_unstale":"2023-10-06T02:27:23.068744+0000","last_undegraded":"2023-10-06T02:27:23.068744+0000","last_fullsized":"2023-10-06T02:27:23.068744+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:16:23.777928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.074099+0000","last_change":"2023-10-06T02:27:23.074099+0000","last_active":"2023-10-06T02:27:23.074099+0000","last_peered":"2023-10-06T02:27:23.074099+0000","last_clean":"2023-10-06T02:27:23.074099+0000","last_became_active":"2023-10-06T02:27:23.073845+0000","last_became_peered":"2023-10-06T02:27:23.073845+0000","last_unstale":"2023-10-06T02:27:23.074099+0000","last_undegraded":"2023-10-06T02:27:23.074099+0000","last_fullsized":"2023-10-06T02:27:23.074099+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:21:15.789405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.062096+0000","last_change":"2023-10-06T02:27:23.062096+0000","last_active":"2023-10-06T02:27:23.062096+0000","last_peered":"2023-10-06T02:27:23.062096+0000","last_clean":"2023-10-06T02:27:23.062096+0000","last_became_active":"2023-10-06T02:27:23.061783+0000","last_became_peered":"2023-10-06T02:27:23.061783+0000","last_unstale":"2023-10-06T02:27:23.062096+0000","last_undegraded":"2023-10-06T02:27:23.062096+0000","last_fullsized":"2023-10-06T02:27:23.062096+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:16:23.777928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.062414+0000","last_change":"2023-10-06T02:27:23.062414+0000","last_active":"2023-10-06T02:27:23.062414+0000","last_peered":"2023-10-06T02:27:23.062414+0000","last_clean":"2023-10-06T02:27:23.062414+0000","last_became_active":"2023-10-06T02:27:23.062175+0000","last_became_peered":"2023-10-06T02:27:23.062175+0000","last_unstale":"2023-10-06T02:27:23.062414+0000","last_undegraded":"2023-10-06T02:27:23.062414+0000","last_fullsized":"2023-10-06T02:27:23.062414+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:47:32.073798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.077624+0000","last_change":"2023-10-06T02:27:23.077624+0000","last_active":"2023-10-06T02:27:23.077624+0000","last_peered":"2023-10-06T02:27:23.077624+0000","last_clean":"2023-10-06T02:27:23.077624+0000","last_became_active":"2023-10-06T02:27:23.075075+0000","last_became_peered":"2023-10-06T02:27:23.075075+0000","last_unstale":"2023-10-06T02:27:23.077624+0000","last_undegraded":"2023-10-06T02:27:23.077624+0000","last_fullsized":"2023-10-06T02:27:23.077624+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:32:16.638954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.076001+0000","last_change":"2023-10-06T02:27:23.076001+0000","last_active":"2023-10-06T02:27:23.076001+0000","last_peered":"2023-10-06T02:27:23.076001+0000","last_clean":"2023-10-06T02:27:23.076001+0000","last_became_active":"2023-10-06T02:27:23.075735+0000","last_became_peered":"2023-10-06T02:27:23.075735+0000","last_unstale":"2023-10-06T02:27:23.076001+0000","last_undegraded":"2023-10-06T02:27:23.076001+0000","last_fullsized":"2023-10-06T02:27:23.076001+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:17:58.502290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.073503+0000","last_change":"2023-10-06T02:27:23.073503+0000","last_active":"2023-10-06T02:27:23.073503+0000","last_peered":"2023-10-06T02:27:23.073503+0000","last_clean":"2023-10-06T02:27:23.073503+0000","last_became_active":"2023-10-06T02:27:23.073215+0000","last_became_peered":"2023-10-06T02:27:23.073215+0000","last_unstale":"2023-10-06T02:27:23.073503+0000","last_undegraded":"2023-10-06T02:27:23.073503+0000","last_fullsized":"2023-10-06T02:27:23.073503+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:23:22.126990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.065314+0000","last_change":"2023-10-06T02:27:23.065314+0000","last_active":"2023-10-06T02:27:23.065314+0000","last_peered":"2023-10-06T02:27:23.065314+0000","last_clean":"2023-10-06T02:27:23.065314+0000","last_became_active":"2023-10-06T02:27:23.065031+0000","last_became_peered":"2023-10-06T02:27:23.065031+0000","last_unstale":"2023-10-06T02:27:23.065314+0000","last_undegraded":"2023-10-06T02:27:23.065314+0000","last_fullsized":"2023-10-06T02:27:23.065314+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:03:04.467972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.063636+0000","last_change":"2023-10-06T02:27:23.063636+0000","last_active":"2023-10-06T02:27:23.063636+0000","last_peered":"2023-10-06T02:27:23.063636+0000","last_clean":"2023-10-06T02:27:23.063636+0000","last_became_active":"2023-10-06T02:27:23.063362+0000","last_became_peered":"2023-10-06T02:27:23.063362+0000","last_unstale":"2023-10-06T02:27:23.063636+0000","last_undegraded":"2023-10-06T02:27:23.063636+0000","last_fullsized":"2023-10-06T02:27:23.063636+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:24.612765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.052271+0000","last_change":"2023-10-06T02:27:23.052271+0000","last_active":"2023-10-06T02:27:23.052271+0000","last_peered":"2023-10-06T02:27:23.052271+0000","last_clean":"2023-10-06T02:27:23.052271+0000","last_became_active":"2023-10-06T02:27:23.052006+0000","last_became_peered":"2023-10-06T02:27:23.052006+0000","last_unstale":"2023-10-06T02:27:23.052271+0000","last_undegraded":"2023-10-06T02:27:23.052271+0000","last_fullsized":"2023-10-06T02:27:23.052271+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:52:26.680131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.056190+0000","last_change":"2023-10-06T02:27:23.056190+0000","last_active":"2023-10-06T02:27:23.056190+0000","last_peered":"2023-10-06T02:27:23.056190+0000","last_clean":"2023-10-06T02:27:23.056190+0000","last_became_active":"2023-10-06T02:27:23.055933+0000","last_became_peered":"2023-10-06T02:27:23.055933+0000","last_unstale":"2023-10-06T02:27:23.056190+0000","last_undegraded":"2023-10-06T02:27:23.056190+0000","last_fullsized":"2023-10-06T02:27:23.056190+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:19:52.525565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.059536+0000","last_change":"2023-10-06T02:27:23.059536+0000","last_active":"2023-10-06T02:27:23.059536+0000","last_peered":"2023-10-06T02:27:23.059536+0000","last_clean":"2023-10-06T02:27:23.059536+0000","last_became_active":"2023-10-06T02:27:23.059177+0000","last_became_peered":"2023-10-06T02:27:23.059177+0000","last_unstale":"2023-10-06T02:27:23.059536+0000","last_undegraded":"2023-10-06T02:27:23.059536+0000","last_fullsized":"2023-10-06T02:27:23.059536+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:02:28.690358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.065538+0000","last_change":"2023-10-06T02:27:23.065538+0000","last_active":"2023-10-06T02:27:23.065538+0000","last_peered":"2023-10-06T02:27:23.065538+0000","last_clean":"2023-10-06T02:27:23.065538+0000","last_became_active":"2023-10-06T02:27:23.065175+0000","last_became_peered":"2023-10-06T02:27:23.065175+0000","last_unstale":"2023-10-06T02:27:23.065538+0000","last_undegraded":"2023-10-06T02:27:23.065538+0000","last_fullsized":"2023-10-06T02:27:23.065538+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:16:24.898085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.062267+0000","last_change":"2023-10-06T02:27:23.062267+0000","last_active":"2023-10-06T02:27:23.062267+0000","last_peered":"2023-10-06T02:27:23.062267+0000","last_clean":"2023-10-06T02:27:23.062267+0000","last_became_active":"2023-10-06T02:27:23.061943+0000","last_became_peered":"2023-10-06T02:27:23.061943+0000","last_unstale":"2023-10-06T02:27:23.062267+0000","last_undegraded":"2023-10-06T02:27:23.062267+0000","last_fullsized":"2023-10-06T02:27:23.062267+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:17:58.502290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.064287+0000","last_change":"2023-10-06T02:27:23.064287+0000","last_active":"2023-10-06T02:27:23.064287+0000","last_peered":"2023-10-06T02:27:23.064287+0000","last_clean":"2023-10-06T02:27:23.064287+0000","last_became_active":"2023-10-06T02:27:23.064024+0000","last_became_peered":"2023-10-06T02:27:23.064024+0000","last_unstale":"2023-10-06T02:27:23.064287+0000","last_undegraded":"2023-10-06T02:27:23.064287+0000","last_fullsized":"2023-10-06T02:27:23.064287+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:23:13.833011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.062617+0000","last_change":"2023-10-06T02:27:23.062617+0000","last_active":"2023-10-06T02:27:23.062617+0000","last_peered":"2023-10-06T02:27:23.062617+0000","last_clean":"2023-10-06T02:27:23.062617+0000","last_became_active":"2023-10-06T02:27:23.062365+0000","last_became_peered":"2023-10-06T02:27:23.062365+0000","last_unstale":"2023-10-06T02:27:23.062617+0000","last_undegraded":"2023-10-06T02:27:23.062617+0000","last_fullsized":"2023-10-06T02:27:23.062617+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:46:24.524044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.068203+0000","last_change":"2023-10-06T02:27:23.068203+0000","last_active":"2023-10-06T02:27:23.068203+0000","last_peered":"2023-10-06T02:27:23.068203+0000","last_clean":"2023-10-06T02:27:23.068203+0000","last_became_active":"2023-10-06T02:27:23.067857+0000","last_became_peered":"2023-10-06T02:27:23.067857+0000","last_unstale":"2023-10-06T02:27:23.068203+0000","last_undegraded":"2023-10-06T02:27:23.068203+0000","last_fullsized":"2023-10-06T02:27:23.068203+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:46:24.524044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.076720+0000","last_change":"2023-10-06T02:27:23.076720+0000","last_active":"2023-10-06T02:27:23.076720+0000","last_peered":"2023-10-06T02:27:23.076720+0000","last_clean":"2023-10-06T02:27:23.076720+0000","last_became_active":"2023-10-06T02:27:23.075064+0000","last_became_peered":"2023-10-06T02:27:23.075064+0000","last_unstale":"2023-10-06T02:27:23.076720+0000","last_undegraded":"2023-10-06T02:27:23.076720+0000","last_fullsized":"2023-10-06T02:27:23.076720+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:46:09.787754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.065016+0000","last_change":"2023-10-06T02:27:23.065016+0000","last_active":"2023-10-06T02:27:23.065016+0000","last_peered":"2023-10-06T02:27:23.065016+0000","last_clean":"2023-10-06T02:27:23.065016+0000","last_became_active":"2023-10-06T02:27:23.064711+0000","last_became_peered":"2023-10-06T02:27:23.064711+0000","last_unstale":"2023-10-06T02:27:23.065016+0000","last_undegraded":"2023-10-06T02:27:23.065016+0000","last_fullsized":"2023-10-06T02:27:23.065016+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:42:18.952592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.072224+0000","last_change":"2023-10-06T02:27:23.072224+0000","last_active":"2023-10-06T02:27:23.072224+0000","last_peered":"2023-10-06T02:27:23.072224+0000","last_clean":"2023-10-06T02:27:23.072224+0000","last_became_active":"2023-10-06T02:27:23.071964+0000","last_became_peered":"2023-10-06T02:27:23.071964+0000","last_unstale":"2023-10-06T02:27:23.072224+0000","last_undegraded":"2023-10-06T02:27:23.072224+0000","last_fullsized":"2023-10-06T02:27:23.072224+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:19:46.513602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"74'4","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.087918+0000","last_change":"2023-10-06T02:27:22.057080+0000","last_active":"2023-10-06T02:27:23.087918+0000","last_peered":"2023-10-06T02:27:23.087918+0000","last_clean":"2023-10-06T02:27:23.087918+0000","last_became_active":"2023-10-06T02:27:22.056741+0000","last_became_peered":"2023-10-06T02:27:22.056741+0000","last_unstale":"2023-10-06T02:27:23.087918+0000","last_undegraded":"2023-10-06T02:27:23.087918+0000","last_fullsized":"2023-10-06T02:27:23.087918+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_clean_scrub_stamp":"2023-10-06T02:27:21.029649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:49:52.176926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.057905+0000","last_change":"2023-10-06T02:27:23.057905+0000","last_active":"2023-10-06T02:27:23.057905+0000","last_peered":"2023-10-06T02:27:23.057905+0000","last_clean":"2023-10-06T02:27:23.057905+0000","last_became_active":"2023-10-06T02:27:23.057632+0000","last_became_peered":"2023-10-06T02:27:23.057632+0000","last_unstale":"2023-10-06T02:27:23.057905+0000","last_undegraded":"2023-10-06T02:27:23.057905+0000","last_fullsized":"2023-10-06T02:27:23.057905+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:01:04.770431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"74'1","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.056565+0000","last_change":"2023-10-06T02:27:22.064952+0000","last_active":"2023-10-06T02:27:23.056565+0000","last_peered":"2023-10-06T02:27:23.056565+0000","last_clean":"2023-10-06T02:27:23.056565+0000","last_became_active":"2023-10-06T02:27:22.064633+0000","last_became_peered":"2023-10-06T02:27:22.064633+0000","last_unstale":"2023-10-06T02:27:23.056565+0000","last_undegraded":"2023-10-06T02:27:23.056565+0000","last_fullsized":"2023-10-06T02:27:23.056565+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_clean_scrub_stamp":"2023-10-06T02:27:21.029649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:39.645404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.063990+0000","last_change":"2023-10-06T02:27:23.063990+0000","last_active":"2023-10-06T02:27:23.063990+0000","last_peered":"2023-10-06T02:27:23.063990+0000","last_clean":"2023-10-06T02:27:23.063990+0000","last_became_active":"2023-10-06T02:27:23.063676+0000","last_became_peered":"2023-10-06T02:27:23.063676+0000","last_unstale":"2023-10-06T02:27:23.063990+0000","last_undegraded":"2023-10-06T02:27:23.063990+0000","last_fullsized":"2023-10-06T02:27:23.063990+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:48:57.800799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"74'2","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.077322+0000","last_change":"2023-10-06T02:27:22.061693+0000","last_active":"2023-10-06T02:27:23.077322+0000","last_peered":"2023-10-06T02:27:23.077322+0000","last_clean":"2023-10-06T02:27:23.077322+0000","last_became_active":"2023-10-06T02:27:22.061424+0000","last_became_peered":"2023-10-06T02:27:22.061424+0000","last_unstale":"2023-10-06T02:27:23.077322+0000","last_undegraded":"2023-10-06T02:27:23.077322+0000","last_fullsized":"2023-10-06T02:27:23.077322+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_clean_scrub_stamp":"2023-10-06T02:27:21.029649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:22:55.959048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.064775+0000","last_change":"2023-10-06T02:27:23.064775+0000","last_active":"2023-10-06T02:27:23.064775+0000","last_peered":"2023-10-06T02:27:23.064775+0000","last_clean":"2023-10-06T02:27:23.064775+0000","last_became_active":"2023-10-06T02:27:23.064163+0000","last_became_peered":"2023-10-06T02:27:23.064163+0000","last_unstale":"2023-10-06T02:27:23.064775+0000","last_undegraded":"2023-10-06T02:27:23.064775+0000","last_fullsized":"2023-10-06T02:27:23.064775+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:32:00.380963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"74'2","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.080363+0000","last_change":"2023-10-06T02:27:22.056761+0000","last_active":"2023-10-06T02:27:23.080363+0000","last_peered":"2023-10-06T02:27:23.080363+0000","last_clean":"2023-10-06T02:27:23.080363+0000","last_became_active":"2023-10-06T02:27:22.055980+0000","last_became_peered":"2023-10-06T02:27:22.055980+0000","last_unstale":"2023-10-06T02:27:23.080363+0000","last_undegraded":"2023-10-06T02:27:23.080363+0000","last_fullsized":"2023-10-06T02:27:23.080363+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_clean_scrub_stamp":"2023-10-06T02:27:21.029649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:41:47.817758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.063388+0000","last_change":"2023-10-06T02:27:23.063388+0000","last_active":"2023-10-06T02:27:23.063388+0000","last_peered":"2023-10-06T02:27:23.063388+0000","last_clean":"2023-10-06T02:27:23.063388+0000","last_became_active":"2023-10-06T02:27:23.063136+0000","last_became_peered":"2023-10-06T02:27:23.063136+0000","last_unstale":"2023-10-06T02:27:23.063388+0000","last_undegraded":"2023-10-06T02:27:23.063388+0000","last_fullsized":"2023-10-06T02:27:23.063388+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:17:58.502290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"74'1","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.049854+0000","last_change":"2023-10-06T02:27:22.059047+0000","last_active":"2023-10-06T02:27:23.049854+0000","last_peered":"2023-10-06T02:27:23.049854+0000","last_clean":"2023-10-06T02:27:23.049854+0000","last_became_active":"2023-10-06T02:27:22.058668+0000","last_became_peered":"2023-10-06T02:27:22.058668+0000","last_unstale":"2023-10-06T02:27:23.049854+0000","last_undegraded":"2023-10-06T02:27:23.049854+0000","last_fullsized":"2023-10-06T02:27:23.049854+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_clean_scrub_stamp":"2023-10-06T02:27:21.029649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:43:52.473654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.053615+0000","last_change":"2023-10-06T02:27:23.053615+0000","last_active":"2023-10-06T02:27:23.053615+0000","last_peered":"2023-10-06T02:27:23.053615+0000","last_clean":"2023-10-06T02:27:23.053615+0000","last_became_active":"2023-10-06T02:27:23.053094+0000","last_became_peered":"2023-10-06T02:27:23.053094+0000","last_unstale":"2023-10-06T02:27:23.053615+0000","last_undegraded":"2023-10-06T02:27:23.053615+0000","last_fullsized":"2023-10-06T02:27:23.053615+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:59:13.916149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"74'1","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.045669+0000","last_change":"2023-10-06T02:27:22.061701+0000","last_active":"2023-10-06T02:27:23.045669+0000","last_peered":"2023-10-06T02:27:23.045669+0000","last_clean":"2023-10-06T02:27:23.045669+0000","last_became_active":"2023-10-06T02:27:22.060445+0000","last_became_peered":"2023-10-06T02:27:22.060445+0000","last_unstale":"2023-10-06T02:27:23.045669+0000","last_undegraded":"2023-10-06T02:27:23.045669+0000","last_fullsized":"2023-10-06T02:27:23.045669+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_clean_scrub_stamp":"2023-10-06T02:27:21.029649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:32:15.636003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.063099+0000","last_change":"2023-10-06T02:27:23.063099+0000","last_active":"2023-10-06T02:27:23.063099+0000","last_peered":"2023-10-06T02:27:23.063099+0000","last_clean":"2023-10-06T02:27:23.063099+0000","last_became_active":"2023-10-06T02:27:23.062838+0000","last_became_peered":"2023-10-06T02:27:23.062838+0000","last_unstale":"2023-10-06T02:27:23.063099+0000","last_undegraded":"2023-10-06T02:27:23.063099+0000","last_fullsized":"2023-10-06T02:27:23.063099+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:44:16.996178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-10-06T02:27:22.056584+0000","last_change":"2023-10-06T02:27:22.053444+0000","last_active":"2023-10-06T02:27:22.056584+0000","last_peered":"2023-10-06T02:27:22.056584+0000","last_clean":"2023-10-06T02:27:22.056584+0000","last_became_active":"2023-10-06T02:27:22.053134+0000","last_became_peered":"2023-10-06T02:27:22.053134+0000","last_unstale":"2023-10-06T02:27:22.056584+0000","last_undegraded":"2023-10-06T02:27:22.056584+0000","last_fullsized":"2023-10-06T02:27:22.056584+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_clean_scrub_stamp":"2023-10-06T02:27:21.029649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:31:59.378012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.077354+0000","last_change":"2023-10-06T02:27:23.077354+0000","last_active":"2023-10-06T02:27:23.077354+0000","last_peered":"2023-10-06T02:27:23.077354+0000","last_clean":"2023-10-06T02:27:23.077354+0000","last_became_active":"2023-10-06T02:27:23.076610+0000","last_became_peered":"2023-10-06T02:27:23.076610+0000","last_unstale":"2023-10-06T02:27:23.077354+0000","last_undegraded":"2023-10-06T02:27:23.077354+0000","last_fullsized":"2023-10-06T02:27:23.077354+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:32:16.638954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-10-06T02:27:22.065122+0000","last_change":"2023-10-06T02:27:22.060117+0000","last_active":"2023-10-06T02:27:22.065122+0000","last_peered":"2023-10-06T02:27:22.065122+0000","last_clean":"2023-10-06T02:27:22.065122+0000","last_became_active":"2023-10-06T02:27:22.059803+0000","last_became_peered":"2023-10-06T02:27:22.059803+0000","last_unstale":"2023-10-06T02:27:22.065122+0000","last_undegraded":"2023-10-06T02:27:22.065122+0000","last_fullsized":"2023-10-06T02:27:22.065122+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_clean_scrub_stamp":"2023-10-06T02:27:21.029649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:22:37.398480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.061468+0000","last_change":"2023-10-06T02:27:23.061468+0000","last_active":"2023-10-06T02:27:23.061468+0000","last_peered":"2023-10-06T02:27:23.061468+0000","last_clean":"2023-10-06T02:27:23.061468+0000","last_became_active":"2023-10-06T02:27:23.061210+0000","last_became_peered":"2023-10-06T02:27:23.061210+0000","last_unstale":"2023-10-06T02:27:23.061468+0000","last_undegraded":"2023-10-06T02:27:23.061468+0000","last_fullsized":"2023-10-06T02:27:23.061468+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:23:45.796316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-10-06T02:27:22.066569+0000","last_change":"2023-10-06T02:27:22.063730+0000","last_active":"2023-10-06T02:27:22.066569+0000","last_peered":"2023-10-06T02:27:22.066569+0000","last_clean":"2023-10-06T02:27:22.066569+0000","last_became_active":"2023-10-06T02:27:22.063235+0000","last_became_peered":"2023-10-06T02:27:22.063235+0000","last_unstale":"2023-10-06T02:27:22.066569+0000","last_undegraded":"2023-10-06T02:27:22.066569+0000","last_fullsized":"2023-10-06T02:27:22.066569+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_clean_scrub_stamp":"2023-10-06T02:27:21.029649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:39:45.981932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.052031+0000","last_change":"2023-10-06T02:27:23.052031+0000","last_active":"2023-10-06T02:27:23.052031+0000","last_peered":"2023-10-06T02:27:23.052031+0000","last_clean":"2023-10-06T02:27:23.052031+0000","last_became_active":"2023-10-06T02:27:23.051707+0000","last_became_peered":"2023-10-06T02:27:23.051707+0000","last_unstale":"2023-10-06T02:27:23.052031+0000","last_undegraded":"2023-10-06T02:27:23.052031+0000","last_fullsized":"2023-10-06T02:27:23.052031+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:32:28.509573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-10-06T02:27:22.058845+0000","last_change":"2023-10-06T02:27:22.056402+0000","last_active":"2023-10-06T02:27:22.058845+0000","last_peered":"2023-10-06T02:27:22.058845+0000","last_clean":"2023-10-06T02:27:22.058845+0000","last_became_active":"2023-10-06T02:27:22.055356+0000","last_became_peered":"2023-10-06T02:27:22.055356+0000","last_unstale":"2023-10-06T02:27:22.058845+0000","last_undegraded":"2023-10-06T02:27:22.058845+0000","last_fullsized":"2023-10-06T02:27:22.058845+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_clean_scrub_stamp":"2023-10-06T02:27:21.029649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:48:24.192836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.077450+0000","last_change":"2023-10-06T02:27:23.077450+0000","last_active":"2023-10-06T02:27:23.077450+0000","last_peered":"2023-10-06T02:27:23.077450+0000","last_clean":"2023-10-06T02:27:23.077450+0000","last_became_active":"2023-10-06T02:27:23.077195+0000","last_became_peered":"2023-10-06T02:27:23.077195+0000","last_unstale":"2023-10-06T02:27:23.077450+0000","last_undegraded":"2023-10-06T02:27:23.077450+0000","last_fullsized":"2023-10-06T02:27:23.077450+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:02:28.690358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"74'3","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.084514+0000","last_change":"2023-10-06T02:27:22.060557+0000","last_active":"2023-10-06T02:27:23.084514+0000","last_peered":"2023-10-06T02:27:23.084514+0000","last_clean":"2023-10-06T02:27:23.084514+0000","last_became_active":"2023-10-06T02:27:22.059952+0000","last_became_peered":"2023-10-06T02:27:22.059952+0000","last_unstale":"2023-10-06T02:27:23.084514+0000","last_undegraded":"2023-10-06T02:27:23.084514+0000","last_fullsized":"2023-10-06T02:27:23.084514+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_clean_scrub_stamp":"2023-10-06T02:27:21.029649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:04:54.873045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.078293+0000","last_change":"2023-10-06T02:27:23.078293+0000","last_active":"2023-10-06T02:27:23.078293+0000","last_peered":"2023-10-06T02:27:23.078293+0000","last_clean":"2023-10-06T02:27:23.078293+0000","last_became_active":"2023-10-06T02:27:23.078043+0000","last_became_peered":"2023-10-06T02:27:23.078043+0000","last_unstale":"2023-10-06T02:27:23.078293+0000","last_undegraded":"2023-10-06T02:27:23.078293+0000","last_fullsized":"2023-10-06T02:27:23.078293+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:53:54.031079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"74'6","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.081809+0000","last_change":"2023-10-06T02:27:22.063665+0000","last_active":"2023-10-06T02:27:23.081809+0000","last_peered":"2023-10-06T02:27:23.081809+0000","last_clean":"2023-10-06T02:27:23.081809+0000","last_became_active":"2023-10-06T02:27:22.063333+0000","last_became_peered":"2023-10-06T02:27:22.063333+0000","last_unstale":"2023-10-06T02:27:23.081809+0000","last_undegraded":"2023-10-06T02:27:23.081809+0000","last_fullsized":"2023-10-06T02:27:23.081809+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_clean_scrub_stamp":"2023-10-06T02:27:21.029649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:50:17.646687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.059691+0000","last_change":"2023-10-06T02:27:23.059691+0000","last_active":"2023-10-06T02:27:23.059691+0000","last_peered":"2023-10-06T02:27:23.059691+0000","last_clean":"2023-10-06T02:27:23.059691+0000","last_became_active":"2023-10-06T02:27:23.059330+0000","last_became_peered":"2023-10-06T02:27:23.059330+0000","last_unstale":"2023-10-06T02:27:23.059691+0000","last_undegraded":"2023-10-06T02:27:23.059691+0000","last_fullsized":"2023-10-06T02:27:23.059691+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:16:50.367846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"74'2","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.075573+0000","last_change":"2023-10-06T02:27:22.053819+0000","last_active":"2023-10-06T02:27:23.075573+0000","last_peered":"2023-10-06T02:27:23.075573+0000","last_clean":"2023-10-06T02:27:23.075573+0000","last_became_active":"2023-10-06T02:27:22.053552+0000","last_became_peered":"2023-10-06T02:27:22.053552+0000","last_unstale":"2023-10-06T02:27:23.075573+0000","last_undegraded":"2023-10-06T02:27:23.075573+0000","last_fullsized":"2023-10-06T02:27:23.075573+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_clean_scrub_stamp":"2023-10-06T02:27:21.029649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:19:45.510650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.073816+0000","last_change":"2023-10-06T02:27:23.073816+0000","last_active":"2023-10-06T02:27:23.073816+0000","last_peered":"2023-10-06T02:27:23.073816+0000","last_clean":"2023-10-06T02:27:23.073816+0000","last_became_active":"2023-10-06T02:27:23.073518+0000","last_became_peered":"2023-10-06T02:27:23.073518+0000","last_unstale":"2023-10-06T02:27:23.073816+0000","last_undegraded":"2023-10-06T02:27:23.073816+0000","last_fullsized":"2023-10-06T02:27:23.073816+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:42:51.844533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-10-06T02:27:22.067092+0000","last_change":"2023-10-06T02:27:22.065888+0000","last_active":"2023-10-06T02:27:22.067092+0000","last_peered":"2023-10-06T02:27:22.067092+0000","last_clean":"2023-10-06T02:27:22.067092+0000","last_became_active":"2023-10-06T02:27:22.065557+0000","last_became_peered":"2023-10-06T02:27:22.065557+0000","last_unstale":"2023-10-06T02:27:22.067092+0000","last_undegraded":"2023-10-06T02:27:22.067092+0000","last_fullsized":"2023-10-06T02:27:22.067092+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_clean_scrub_stamp":"2023-10-06T02:27:21.029649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:08:29.170939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.068975+0000","last_change":"2023-10-06T02:27:23.068975+0000","last_active":"2023-10-06T02:27:23.068975+0000","last_peered":"2023-10-06T02:27:23.068975+0000","last_clean":"2023-10-06T02:27:23.068975+0000","last_became_active":"2023-10-06T02:27:23.068676+0000","last_became_peered":"2023-10-06T02:27:23.068676+0000","last_unstale":"2023-10-06T02:27:23.068975+0000","last_undegraded":"2023-10-06T02:27:23.068975+0000","last_fullsized":"2023-10-06T02:27:23.068975+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:48:57.800799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-10-06T02:27:22.065091+0000","last_change":"2023-10-06T02:27:22.057079+0000","last_active":"2023-10-06T02:27:22.065091+0000","last_peered":"2023-10-06T02:27:22.065091+0000","last_clean":"2023-10-06T02:27:22.065091+0000","last_became_active":"2023-10-06T02:27:22.056702+0000","last_became_peered":"2023-10-06T02:27:22.056702+0000","last_unstale":"2023-10-06T02:27:22.065091+0000","last_undegraded":"2023-10-06T02:27:22.065091+0000","last_fullsized":"2023-10-06T02:27:22.065091+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_clean_scrub_stamp":"2023-10-06T02:27:21.029649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:29:11.373410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.070443+0000","last_change":"2023-10-06T02:27:23.070443+0000","last_active":"2023-10-06T02:27:23.070443+0000","last_peered":"2023-10-06T02:27:23.070443+0000","last_clean":"2023-10-06T02:27:23.070443+0000","last_became_active":"2023-10-06T02:27:23.068728+0000","last_became_peered":"2023-10-06T02:27:23.068728+0000","last_unstale":"2023-10-06T02:27:23.070443+0000","last_undegraded":"2023-10-06T02:27:23.070443+0000","last_fullsized":"2023-10-06T02:27:23.070443+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:09.326606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"74'8","reported_seq":17,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.082652+0000","last_change":"2023-10-06T02:27:22.052190+0000","last_active":"2023-10-06T02:27:23.082652+0000","last_peered":"2023-10-06T02:27:23.082652+0000","last_clean":"2023-10-06T02:27:23.082652+0000","last_became_active":"2023-10-06T02:27:22.051912+0000","last_became_peered":"2023-10-06T02:27:22.051912+0000","last_unstale":"2023-10-06T02:27:23.082652+0000","last_undegraded":"2023-10-06T02:27:23.082652+0000","last_fullsized":"2023-10-06T02:27:23.082652+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_clean_scrub_stamp":"2023-10-06T02:27:21.029649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:03:03.465021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.060858+0000","last_change":"2023-10-06T02:27:23.060858+0000","last_active":"2023-10-06T02:27:23.060858+0000","last_peered":"2023-10-06T02:27:23.060858+0000","last_clean":"2023-10-06T02:27:23.060858+0000","last_became_active":"2023-10-06T02:27:23.060555+0000","last_became_peered":"2023-10-06T02:27:23.060555+0000","last_unstale":"2023-10-06T02:27:23.060858+0000","last_undegraded":"2023-10-06T02:27:23.060858+0000","last_fullsized":"2023-10-06T02:27:23.060858+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:45:17.340969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"74'2","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.081466+0000","last_change":"2023-10-06T02:27:22.059753+0000","last_active":"2023-10-06T02:27:23.081466+0000","last_peered":"2023-10-06T02:27:23.081466+0000","last_clean":"2023-10-06T02:27:23.081466+0000","last_became_active":"2023-10-06T02:27:22.059459+0000","last_became_peered":"2023-10-06T02:27:22.059459+0000","last_unstale":"2023-10-06T02:27:23.081466+0000","last_undegraded":"2023-10-06T02:27:23.081466+0000","last_fullsized":"2023-10-06T02:27:23.081466+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_clean_scrub_stamp":"2023-10-06T02:27:21.029649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:53:41.029150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.065794+0000","last_change":"2023-10-06T02:27:23.065794+0000","last_active":"2023-10-06T02:27:23.065794+0000","last_peered":"2023-10-06T02:27:23.065794+0000","last_clean":"2023-10-06T02:27:23.065794+0000","last_became_active":"2023-10-06T02:27:23.065543+0000","last_became_peered":"2023-10-06T02:27:23.065543+0000","last_unstale":"2023-10-06T02:27:23.065794+0000","last_undegraded":"2023-10-06T02:27:23.065794+0000","last_fullsized":"2023-10-06T02:27:23.065794+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:44:16.996178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-10-06T02:27:22.064642+0000","last_change":"2023-10-06T02:27:22.062572+0000","last_active":"2023-10-06T02:27:22.064642+0000","last_peered":"2023-10-06T02:27:22.064642+0000","last_clean":"2023-10-06T02:27:22.064642+0000","last_became_active":"2023-10-06T02:27:22.062255+0000","last_became_peered":"2023-10-06T02:27:22.062255+0000","last_unstale":"2023-10-06T02:27:22.064642+0000","last_undegraded":"2023-10-06T02:27:22.064642+0000","last_fullsized":"2023-10-06T02:27:22.064642+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_clean_scrub_stamp":"2023-10-06T02:27:21.029649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:45:16.338018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.059651+0000","last_change":"2023-10-06T02:27:23.059651+0000","last_active":"2023-10-06T02:27:23.059651+0000","last_peered":"2023-10-06T02:27:23.059651+0000","last_clean":"2023-10-06T02:27:23.059651+0000","last_became_active":"2023-10-06T02:27:23.059323+0000","last_became_peered":"2023-10-06T02:27:23.059323+0000","last_unstale":"2023-10-06T02:27:23.059651+0000","last_undegraded":"2023-10-06T02:27:23.059651+0000","last_fullsized":"2023-10-06T02:27:23.059651+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:09.326606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-10-06T02:27:22.063360+0000","last_change":"2023-10-06T02:27:22.059540+0000","last_active":"2023-10-06T02:27:22.063360+0000","last_peered":"2023-10-06T02:27:22.063360+0000","last_clean":"2023-10-06T02:27:22.063360+0000","last_became_active":"2023-10-06T02:27:22.059292+0000","last_became_peered":"2023-10-06T02:27:22.059292+0000","last_unstale":"2023-10-06T02:27:22.063360+0000","last_undegraded":"2023-10-06T02:27:22.063360+0000","last_fullsized":"2023-10-06T02:27:22.063360+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_clean_scrub_stamp":"2023-10-06T02:27:21.029649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:39:25.391598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.065826+0000","last_change":"2023-10-06T02:27:23.065826+0000","last_active":"2023-10-06T02:27:23.065826+0000","last_peered":"2023-10-06T02:27:23.065826+0000","last_clean":"2023-10-06T02:27:23.065826+0000","last_became_active":"2023-10-06T02:27:23.065550+0000","last_became_peered":"2023-10-06T02:27:23.065550+0000","last_unstale":"2023-10-06T02:27:23.065826+0000","last_undegraded":"2023-10-06T02:27:23.065826+0000","last_fullsized":"2023-10-06T02:27:23.065826+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:10:04.209357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-10-06T02:27:22.065628+0000","last_change":"2023-10-06T02:27:22.060206+0000","last_active":"2023-10-06T02:27:22.065628+0000","last_peered":"2023-10-06T02:27:22.065628+0000","last_clean":"2023-10-06T02:27:22.065628+0000","last_became_active":"2023-10-06T02:27:22.059936+0000","last_became_peered":"2023-10-06T02:27:22.059936+0000","last_unstale":"2023-10-06T02:27:22.065628+0000","last_undegraded":"2023-10-06T02:27:22.065628+0000","last_fullsized":"2023-10-06T02:27:22.065628+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_clean_scrub_stamp":"2023-10-06T02:27:21.029649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:49:10.287961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.066888+0000","last_change":"2023-10-06T02:27:23.066888+0000","last_active":"2023-10-06T02:27:23.066888+0000","last_peered":"2023-10-06T02:27:23.066888+0000","last_clean":"2023-10-06T02:27:23.066888+0000","last_became_active":"2023-10-06T02:27:23.066603+0000","last_became_peered":"2023-10-06T02:27:23.066603+0000","last_unstale":"2023-10-06T02:27:23.066888+0000","last_undegraded":"2023-10-06T02:27:23.066888+0000","last_fullsized":"2023-10-06T02:27:23.066888+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:52:26.680131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-10-06T02:27:22.058815+0000","last_change":"2023-10-06T02:27:22.055863+0000","last_active":"2023-10-06T02:27:22.058815+0000","last_peered":"2023-10-06T02:27:22.058815+0000","last_clean":"2023-10-06T02:27:22.058815+0000","last_became_active":"2023-10-06T02:27:22.055522+0000","last_became_peered":"2023-10-06T02:27:22.055522+0000","last_unstale":"2023-10-06T02:27:22.058815+0000","last_undegraded":"2023-10-06T02:27:22.058815+0000","last_fullsized":"2023-10-06T02:27:22.058815+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_clean_scrub_stamp":"2023-10-06T02:27:21.029649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:02:35.238222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.075801+0000","last_change":"2023-10-06T02:27:23.075801+0000","last_active":"2023-10-06T02:27:23.075801+0000","last_peered":"2023-10-06T02:27:23.075801+0000","last_clean":"2023-10-06T02:27:23.075801+0000","last_became_active":"2023-10-06T02:27:23.075506+0000","last_became_peered":"2023-10-06T02:27:23.075506+0000","last_unstale":"2023-10-06T02:27:23.075801+0000","last_undegraded":"2023-10-06T02:27:23.075801+0000","last_fullsized":"2023-10-06T02:27:23.075801+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:53:42.032101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-10-06T02:27:22.064914+0000","last_change":"2023-10-06T02:27:22.056299+0000","last_active":"2023-10-06T02:27:22.064914+0000","last_peered":"2023-10-06T02:27:22.064914+0000","last_clean":"2023-10-06T02:27:22.064914+0000","last_became_active":"2023-10-06T02:27:22.056042+0000","last_became_peered":"2023-10-06T02:27:22.056042+0000","last_unstale":"2023-10-06T02:27:22.064914+0000","last_undegraded":"2023-10-06T02:27:22.064914+0000","last_fullsized":"2023-10-06T02:27:22.064914+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_clean_scrub_stamp":"2023-10-06T02:27:21.029649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:06:33.609185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.077962+0000","last_change":"2023-10-06T02:27:23.077962+0000","last_active":"2023-10-06T02:27:23.077962+0000","last_peered":"2023-10-06T02:27:23.077962+0000","last_clean":"2023-10-06T02:27:23.077962+0000","last_became_active":"2023-10-06T02:27:23.076544+0000","last_became_peered":"2023-10-06T02:27:23.076544+0000","last_unstale":"2023-10-06T02:27:23.077962+0000","last_undegraded":"2023-10-06T02:27:23.077962+0000","last_fullsized":"2023-10-06T02:27:23.077962+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:16:35.087628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-10-06T02:27:22.063934+0000","last_change":"2023-10-06T02:27:22.058785+0000","last_active":"2023-10-06T02:27:22.063934+0000","last_peered":"2023-10-06T02:27:22.063934+0000","last_clean":"2023-10-06T02:27:22.063934+0000","last_became_active":"2023-10-06T02:27:22.058503+0000","last_became_peered":"2023-10-06T02:27:22.058503+0000","last_unstale":"2023-10-06T02:27:22.063934+0000","last_undegraded":"2023-10-06T02:27:22.063934+0000","last_fullsized":"2023-10-06T02:27:22.063934+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_clean_scrub_stamp":"2023-10-06T02:27:21.029649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:34:18.914626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.071649+0000","last_change":"2023-10-06T02:27:23.071649+0000","last_active":"2023-10-06T02:27:23.071649+0000","last_peered":"2023-10-06T02:27:23.071649+0000","last_clean":"2023-10-06T02:27:23.071649+0000","last_became_active":"2023-10-06T02:27:23.071388+0000","last_became_peered":"2023-10-06T02:27:23.071388+0000","last_unstale":"2023-10-06T02:27:23.071649+0000","last_undegraded":"2023-10-06T02:27:23.071649+0000","last_fullsized":"2023-10-06T02:27:23.071649+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:53:42.032101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-10-06T02:27:22.066462+0000","last_change":"2023-10-06T02:27:22.064288+0000","last_active":"2023-10-06T02:27:22.066462+0000","last_peered":"2023-10-06T02:27:22.066462+0000","last_clean":"2023-10-06T02:27:22.066462+0000","last_became_active":"2023-10-06T02:27:22.063986+0000","last_became_peered":"2023-10-06T02:27:22.063986+0000","last_unstale":"2023-10-06T02:27:22.066462+0000","last_undegraded":"2023-10-06T02:27:22.066462+0000","last_fullsized":"2023-10-06T02:27:22.066462+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_clean_scrub_stamp":"2023-10-06T02:27:21.029649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:34:18.914626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.069048+0000","last_change":"2023-10-06T02:27:23.069048+0000","last_active":"2023-10-06T02:27:23.069048+0000","last_peered":"2023-10-06T02:27:23.069048+0000","last_clean":"2023-10-06T02:27:23.069048+0000","last_became_active":"2023-10-06T02:27:23.065975+0000","last_became_peered":"2023-10-06T02:27:23.065975+0000","last_unstale":"2023-10-06T02:27:23.069048+0000","last_undegraded":"2023-10-06T02:27:23.069048+0000","last_fullsized":"2023-10-06T02:27:23.069048+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:23:53.153268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"74'1","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.049948+0000","last_change":"2023-10-06T02:27:22.060727+0000","last_active":"2023-10-06T02:27:23.049948+0000","last_peered":"2023-10-06T02:27:23.049948+0000","last_clean":"2023-10-06T02:27:23.049948+0000","last_became_active":"2023-10-06T02:27:22.060282+0000","last_became_peered":"2023-10-06T02:27:22.060282+0000","last_unstale":"2023-10-06T02:27:23.049948+0000","last_undegraded":"2023-10-06T02:27:23.049948+0000","last_fullsized":"2023-10-06T02:27:23.049948+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_clean_scrub_stamp":"2023-10-06T02:27:21.029649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:13:48.718661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":55,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-10-06T02:27:22.058514+0000","last_change":"2023-10-06T02:23:02.054039+0000","last_active":"2023-10-06T02:27:22.058514+0000","last_peered":"2023-10-06T02:27:22.058514+0000","last_clean":"2023-10-06T02:27:22.058514+0000","last_became_active":"2023-10-06T02:23:02.053742+0000","last_became_peered":"2023-10-06T02:23:02.053742+0000","last_unstale":"2023-10-06T02:27:22.058514+0000","last_undegraded":"2023-10-06T02:27:22.058514+0000","last_fullsized":"2023-10-06T02:27:22.058514+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:58:18.846475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.053194+0000","last_change":"2023-10-06T02:27:23.053194+0000","last_active":"2023-10-06T02:27:23.053194+0000","last_peered":"2023-10-06T02:27:23.053194+0000","last_clean":"2023-10-06T02:27:23.053194+0000","last_became_active":"2023-10-06T02:27:23.052665+0000","last_became_peered":"2023-10-06T02:27:23.052665+0000","last_unstale":"2023-10-06T02:27:23.053194+0000","last_undegraded":"2023-10-06T02:27:23.053194+0000","last_fullsized":"2023-10-06T02:27:23.053194+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:51:19.130397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"74'2","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.081378+0000","last_change":"2023-10-06T02:27:22.058572+0000","last_active":"2023-10-06T02:27:23.081378+0000","last_peered":"2023-10-06T02:27:23.081378+0000","last_clean":"2023-10-06T02:27:23.081378+0000","last_became_active":"2023-10-06T02:27:22.057907+0000","last_became_peered":"2023-10-06T02:27:22.057907+0000","last_unstale":"2023-10-06T02:27:23.081378+0000","last_undegraded":"2023-10-06T02:27:23.081378+0000","last_fullsized":"2023-10-06T02:27:23.081378+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_clean_scrub_stamp":"2023-10-06T02:27:21.029649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:39.645404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":64,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-10-06T02:27:22.046876+0000","last_change":"2023-10-06T02:23:01.988631+0000","last_active":"2023-10-06T02:27:22.046876+0000","last_peered":"2023-10-06T02:27:22.046876+0000","last_clean":"2023-10-06T02:27:22.046876+0000","last_became_active":"2023-10-06T02:23:01.988303+0000","last_became_peered":"2023-10-06T02:23:01.988303+0000","last_unstale":"2023-10-06T02:27:22.046876+0000","last_undegraded":"2023-10-06T02:27:22.046876+0000","last_fullsized":"2023-10-06T02:27:22.046876+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:38.211466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.062179+0000","last_change":"2023-10-06T02:27:23.062179+0000","last_active":"2023-10-06T02:27:23.062179+0000","last_peered":"2023-10-06T02:27:23.062179+0000","last_clean":"2023-10-06T02:27:23.062179+0000","last_became_active":"2023-10-06T02:27:23.061880+0000","last_became_peered":"2023-10-06T02:27:23.061880+0000","last_unstale":"2023-10-06T02:27:23.062179+0000","last_undegraded":"2023-10-06T02:27:23.062179+0000","last_fullsized":"2023-10-06T02:27:23.062179+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:48:25.195788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-10-06T02:27:22.056261+0000","last_change":"2023-10-06T02:27:22.055164+0000","last_active":"2023-10-06T02:27:22.056261+0000","last_peered":"2023-10-06T02:27:22.056261+0000","last_clean":"2023-10-06T02:27:22.056261+0000","last_became_active":"2023-10-06T02:27:22.054830+0000","last_became_peered":"2023-10-06T02:27:22.054830+0000","last_unstale":"2023-10-06T02:27:22.056261+0000","last_undegraded":"2023-10-06T02:27:22.056261+0000","last_fullsized":"2023-10-06T02:27:22.056261+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_clean_scrub_stamp":"2023-10-06T02:27:21.029649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:49:10.287961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":60,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-10-06T02:27:22.040878+0000","last_change":"2023-10-06T02:23:02.054338+0000","last_active":"2023-10-06T02:27:22.040878+0000","last_peered":"2023-10-06T02:27:22.040878+0000","last_clean":"2023-10-06T02:27:22.040878+0000","last_became_active":"2023-10-06T02:23:02.053991+0000","last_became_peered":"2023-10-06T02:23:02.053991+0000","last_unstale":"2023-10-06T02:27:22.040878+0000","last_undegraded":"2023-10-06T02:27:22.040878+0000","last_fullsized":"2023-10-06T02:27:22.040878+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:39:30.192838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.073630+0000","last_change":"2023-10-06T02:27:23.073630+0000","last_active":"2023-10-06T02:27:23.073630+0000","last_peered":"2023-10-06T02:27:23.073630+0000","last_clean":"2023-10-06T02:27:23.073630+0000","last_became_active":"2023-10-06T02:27:23.073346+0000","last_became_peered":"2023-10-06T02:27:23.073346+0000","last_unstale":"2023-10-06T02:27:23.073630+0000","last_undegraded":"2023-10-06T02:27:23.073630+0000","last_fullsized":"2023-10-06T02:27:23.073630+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:02:28.690358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-10-06T02:27:22.056011+0000","last_change":"2023-10-06T02:27:22.054531+0000","last_active":"2023-10-06T02:27:22.056011+0000","last_peered":"2023-10-06T02:27:22.056011+0000","last_clean":"2023-10-06T02:27:22.056011+0000","last_became_active":"2023-10-06T02:27:22.054226+0000","last_became_peered":"2023-10-06T02:27:22.054226+0000","last_unstale":"2023-10-06T02:27:22.056011+0000","last_undegraded":"2023-10-06T02:27:22.056011+0000","last_fullsized":"2023-10-06T02:27:22.056011+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_clean_scrub_stamp":"2023-10-06T02:27:21.029649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:39:13.090897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":64,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-10-06T02:27:22.047004+0000","last_change":"2023-10-06T02:23:01.988140+0000","last_active":"2023-10-06T02:27:22.047004+0000","last_peered":"2023-10-06T02:27:22.047004+0000","last_clean":"2023-10-06T02:27:22.047004+0000","last_became_active":"2023-10-06T02:23:01.987769+0000","last_became_peered":"2023-10-06T02:23:01.987769+0000","last_unstale":"2023-10-06T02:27:22.047004+0000","last_undegraded":"2023-10-06T02:27:22.047004+0000","last_fullsized":"2023-10-06T02:27:22.047004+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:20:08.391912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":207,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-10-06T02:27:22.053229+0000","last_change":"2023-10-06T02:22:55.032519+0000","last_active":"2023-10-06T02:27:22.053229+0000","last_peered":"2023-10-06T02:27:22.053229+0000","last_clean":"2023-10-06T02:27:22.053229+0000","last_became_active":"2023-10-06T02:22:55.031367+0000","last_became_peered":"2023-10-06T02:22:55.031367+0000","last_unstale":"2023-10-06T02:27:22.053229+0000","last_undegraded":"2023-10-06T02:27:22.053229+0000","last_fullsized":"2023-10-06T02:27:22.053229+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:22:53.309097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:22:53.309097+0000","last_clean_scrub_stamp":"2023-10-06T02:22:53.309097+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:53:46.520909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.059939+0000","last_change":"2023-10-06T02:27:23.059939+0000","last_active":"2023-10-06T02:27:23.059939+0000","last_peered":"2023-10-06T02:27:23.059939+0000","last_clean":"2023-10-06T02:27:23.059939+0000","last_became_active":"2023-10-06T02:27:23.059655+0000","last_became_peered":"2023-10-06T02:27:23.059655+0000","last_unstale":"2023-10-06T02:27:23.059939+0000","last_undegraded":"2023-10-06T02:27:23.059939+0000","last_fullsized":"2023-10-06T02:27:23.059939+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:37:59.546106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-10-06T02:27:22.065873+0000","last_change":"2023-10-06T02:27:22.063295+0000","last_active":"2023-10-06T02:27:22.065873+0000","last_peered":"2023-10-06T02:27:22.065873+0000","last_clean":"2023-10-06T02:27:22.065873+0000","last_became_active":"2023-10-06T02:27:22.063033+0000","last_became_peered":"2023-10-06T02:27:22.063033+0000","last_unstale":"2023-10-06T02:27:22.065873+0000","last_undegraded":"2023-10-06T02:27:22.065873+0000","last_fullsized":"2023-10-06T02:27:22.065873+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_clean_scrub_stamp":"2023-10-06T02:27:21.029649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:01:03.767480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":54,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-10-06T02:27:22.053110+0000","last_change":"2023-10-06T02:23:01.990773+0000","last_active":"2023-10-06T02:27:22.053110+0000","last_peered":"2023-10-06T02:27:22.053110+0000","last_clean":"2023-10-06T02:27:22.053110+0000","last_became_active":"2023-10-06T02:23:01.990511+0000","last_became_peered":"2023-10-06T02:23:01.990511+0000","last_unstale":"2023-10-06T02:27:22.053110+0000","last_undegraded":"2023-10-06T02:27:22.053110+0000","last_fullsized":"2023-10-06T02:27:22.053110+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:42:54.797731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.074083+0000","last_change":"2023-10-06T02:27:23.074083+0000","last_active":"2023-10-06T02:27:23.074083+0000","last_peered":"2023-10-06T02:27:23.074083+0000","last_clean":"2023-10-06T02:27:23.074083+0000","last_became_active":"2023-10-06T02:27:23.072575+0000","last_became_peered":"2023-10-06T02:27:23.072575+0000","last_unstale":"2023-10-06T02:27:23.074083+0000","last_undegraded":"2023-10-06T02:27:23.074083+0000","last_fullsized":"2023-10-06T02:27:23.074083+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:51:19.130397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-10-06T02:27:22.066837+0000","last_change":"2023-10-06T02:27:22.065671+0000","last_active":"2023-10-06T02:27:22.066837+0000","last_peered":"2023-10-06T02:27:22.066837+0000","last_clean":"2023-10-06T02:27:22.066837+0000","last_became_active":"2023-10-06T02:27:22.065295+0000","last_became_peered":"2023-10-06T02:27:22.065295+0000","last_unstale":"2023-10-06T02:27:22.066837+0000","last_undegraded":"2023-10-06T02:27:22.066837+0000","last_fullsized":"2023-10-06T02:27:22.066837+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_clean_scrub_stamp":"2023-10-06T02:27:21.029649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:02:35.238222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":54,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-10-06T02:27:22.054290+0000","last_change":"2023-10-06T02:23:02.054069+0000","last_active":"2023-10-06T02:27:22.054290+0000","last_peered":"2023-10-06T02:27:22.054290+0000","last_clean":"2023-10-06T02:27:22.054290+0000","last_became_active":"2023-10-06T02:23:02.053739+0000","last_became_peered":"2023-10-06T02:23:02.053739+0000","last_unstale":"2023-10-06T02:27:22.054290+0000","last_undegraded":"2023-10-06T02:27:22.054290+0000","last_fullsized":"2023-10-06T02:27:22.054290+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:57:48.822608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.073301+0000","last_change":"2023-10-06T02:27:23.073301+0000","last_active":"2023-10-06T02:27:23.073301+0000","last_peered":"2023-10-06T02:27:23.073301+0000","last_clean":"2023-10-06T02:27:23.073301+0000","last_became_active":"2023-10-06T02:27:23.073050+0000","last_became_peered":"2023-10-06T02:27:23.073050+0000","last_unstale":"2023-10-06T02:27:23.073301+0000","last_undegraded":"2023-10-06T02:27:23.073301+0000","last_fullsized":"2023-10-06T02:27:23.073301+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:42:18.952592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"74'1","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.057979+0000","last_change":"2023-10-06T02:27:22.062770+0000","last_active":"2023-10-06T02:27:23.057979+0000","last_peered":"2023-10-06T02:27:23.057979+0000","last_clean":"2023-10-06T02:27:23.057979+0000","last_became_active":"2023-10-06T02:27:22.062427+0000","last_became_peered":"2023-10-06T02:27:22.062427+0000","last_unstale":"2023-10-06T02:27:23.057979+0000","last_undegraded":"2023-10-06T02:27:23.057979+0000","last_fullsized":"2023-10-06T02:27:23.057979+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_clean_scrub_stamp":"2023-10-06T02:27:21.029649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:16:49.364895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":55,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-10-06T02:27:22.058187+0000","last_change":"2023-10-06T02:23:01.989934+0000","last_active":"2023-10-06T02:27:22.058187+0000","last_peered":"2023-10-06T02:27:22.058187+0000","last_clean":"2023-10-06T02:27:22.058187+0000","last_became_active":"2023-10-06T02:23:01.989706+0000","last_became_peered":"2023-10-06T02:23:01.989706+0000","last_unstale":"2023-10-06T02:27:22.058187+0000","last_undegraded":"2023-10-06T02:27:22.058187+0000","last_fullsized":"2023-10-06T02:27:22.058187+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:06.212251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.061826+0000","last_change":"2023-10-06T02:27:23.061826+0000","last_active":"2023-10-06T02:27:23.061826+0000","last_peered":"2023-10-06T02:27:23.061826+0000","last_clean":"2023-10-06T02:27:23.061826+0000","last_became_active":"2023-10-06T02:27:23.061567+0000","last_became_peered":"2023-10-06T02:27:23.061567+0000","last_unstale":"2023-10-06T02:27:23.061826+0000","last_undegraded":"2023-10-06T02:27:23.061826+0000","last_fullsized":"2023-10-06T02:27:23.061826+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:22.032601+0000","last_clean_scrub_stamp":"2023-10-06T02:27:22.032601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:22:05.196287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"74'2","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T02:27:23.079367+0000","last_change":"2023-10-06T02:27:22.066842+0000","last_active":"2023-10-06T02:27:23.079367+0000","last_peered":"2023-10-06T02:27:23.079367+0000","last_clean":"2023-10-06T02:27:23.079367+0000","last_became_active":"2023-10-06T02:27:22.066549+0000","last_became_peered":"2023-10-06T02:27:22.066549+0000","last_unstale":"2023-10-06T02:27:23.079367+0000","last_undegraded":"2023-10-06T02:27:23.079367+0000","last_fullsized":"2023-10-06T02:27:23.079367+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:21.029649+0000","last_clean_scrub_stamp":"2023-10-06T02:27:21.029649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:47:30.949998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":57,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-10-06T02:27:22.048084+0000","last_change":"2023-10-06T02:23:01.992935+0000","last_active":"2023-10-06T02:27:22.048084+0000","last_peered":"2023-10-06T02:27:22.048084+0000","last_clean":"2023-10-06T02:27:22.048084+0000","last_became_active":"2023-10-06T02:23:01.992568+0000","last_became_peered":"2023-10-06T02:23:01.992568+0000","last_unstale":"2023-10-06T02:27:22.048084+0000","last_undegraded":"2023-10-06T02:27:22.048084+0000","last_fullsized":"2023-10-06T02:27:22.048084+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:06.212251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574906,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302832,"kb_used_data":1448,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069008,"statfs":{"total":96636764160,"available":96326664192,"internally_reserved":0,"allocated":1482752,"data_stored":984015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:26:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.64100000000000001,"15min":0.63800000000000001},"min":{"1min":0.42399999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.002,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.624,"15min":0.60599999999999998},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.84399999999999997,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.58399999999999996}]},{"osd":1,"last update":"Fri Oct 6 02:26:58 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.71099999999999997,"15min":0.68500000000000005},"min":{"1min":0.46800000000000003,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.95399999999999996,"5min":1.147,"15min":1.147},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.68200000000000005,"15min":0.67500000000000004},"min":{"1min":0.498,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.86899999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.66200000000000003}]},{"osd":2,"last update":"Fri Oct 6 02:26:58 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.69299999999999995,"15min":0.69499999999999995},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.68700000000000006,"15min":0.68999999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.93500000000000005,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.625}]},{"osd":3,"last update":"Fri Oct 6 02:26:58 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.747,"15min":0.73299999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.754,"5min":0.77600000000000002,"15min":0.77800000000000002},"min":{"1min":0.55500000000000005,"5min":0.497,"15min":0.497},"max":{"1min":0.98199999999999998,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.67100000000000004}]},{"osd":4,"last update":"Fri Oct 6 02:26:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.65000000000000002,"15min":0.69599999999999995},"min":{"1min":0.41099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.95199999999999996,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.72599999999999998,"15min":0.77000000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.105,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.68999999999999995}]},{"osd":5,"last update":"Fri Oct 6 02:26:58 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.71499999999999997,"15min":0.73499999999999999},"min":{"1min":0.44700000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.98499999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.71499999999999997,"15min":0.73499999999999999},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.95399999999999996,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.53300000000000003}]},{"osd":6,"last update":"Fri Oct 6 02:26:58 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.77400000000000002,"15min":0.78700000000000003},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0109999999999999,"5min":1.056,"15min":1.056},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.80600000000000005,"15min":0.81599999999999995},"min":{"1min":0.48199999999999998,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.1659999999999999,"5min":1.276,"15min":1.276},"last":0.77800000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574906,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298168,"kb_used_data":880,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073672,"statfs":{"total":96636764160,"available":96331440128,"internally_reserved":0,"allocated":901120,"data_stored":400763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:27:03 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.67100000000000004,"15min":0.66400000000000003},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.91900000000000004,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.71399999999999997,"15min":0.69699999999999995},"min":{"1min":0.53500000000000003,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.95099999999999996,"5min":1.119,"15min":1.119},"last":0.79600000000000004}]},{"osd":1,"last update":"Fri Oct 6 02:27:03 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.64100000000000001,"15min":0.64400000000000002},"min":{"1min":0.433,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67400000000000004,"15min":0.66600000000000004},"min":{"1min":0.48899999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.89500000000000002,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.68400000000000005}]},{"osd":2,"last update":"Fri Oct 6 02:27:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.71199999999999997,"15min":0.70199999999999996},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.96099999999999997,"5min":1.137,"15min":1.137},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72699999999999998,"15min":0.71099999999999997},"min":{"1min":0.55600000000000005,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.89300000000000002,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.752}]},{"osd":3,"last update":"Fri Oct 6 02:27:03 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.68200000000000005,"15min":0.66000000000000003},"min":{"1min":0.52000000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.97099999999999997,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71299999999999997,"15min":0.70299999999999996},"min":{"1min":0.5,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.91600000000000004,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.621}]},{"osd":4,"last update":"Fri Oct 6 02:27:03 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63900000000000001,"15min":0.63400000000000001},"min":{"1min":0.36199999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.90000000000000002,"5min":1.095,"15min":1.095},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.67600000000000005,"15min":0.65600000000000003},"min":{"1min":0.46200000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.999,"5min":1.29,"15min":1.29},"last":0.69799999999999995}]},{"osd":5,"last update":"Fri Oct 6 02:27:03 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.68100000000000005,"15min":0.69699999999999995},"min":{"1min":0.48399999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.94299999999999995,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.70699999999999996,"15min":0.70199999999999996},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.86099999999999999,"5min":1.02,"15min":1.02},"last":0.56499999999999995}]},{"osd":7,"last update":"Fri Oct 6 02:27:03 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.74199999999999999,"15min":0.746},"min":{"1min":0.41499999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.95399999999999996,"5min":1.458,"15min":1.458},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76000000000000001,"15min":0.75900000000000001},"min":{"1min":0.54600000000000004,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.96499999999999997,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.70799999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574905,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302252,"kb_used_data":868,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069588,"statfs":{"total":96636764160,"available":96327258112,"internally_reserved":0,"allocated":888832,"data_stored":393647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.45200000000000001,"15min":0.46300000000000002},"min":{"1min":0.314,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.61299999999999999,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.499,"15min":0.53100000000000003},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.71799999999999997,"5min":1.788,"15min":1.788},"last":0.70099999999999996}]},{"osd":2,"last update":"Fri Oct 6 02:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.48699999999999999,"15min":0.53000000000000003},"min":{"1min":0.35599999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.57499999999999996,"5min":2.0840000000000001,"15min":2.0840000000000001},"last":0.38},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.53300000000000003,"15min":0.56200000000000006},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.69299999999999995,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.436}]},{"osd":3,"last update":"Fri Oct 6 02:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.61099999999999999,"15min":0.61799999999999999},"min":{"1min":0.44600000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.80300000000000005,"5min":1.266,"15min":1.266},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.59199999999999997,"15min":0.64800000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.70499999999999996,"5min":1.861,"15min":1.861},"last":0.503}]},{"osd":4,"last update":"Fri Oct 6 02:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.83299999999999996,"15min":0.86499999999999999},"min":{"1min":0.61399999999999999,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.91000000000000003,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":0.624},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.83899999999999997,"15min":0.86599999999999999},"min":{"1min":0.52900000000000003,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.93100000000000005,"5min":2.0179999999999998,"15min":2.0179999999999998},"last":0.51400000000000001}]},{"osd":5,"last update":"Fri Oct 6 02:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.81200000000000006,"15min":0.87},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.89000000000000001,"5min":1.9830000000000001,"15min":1.9830000000000001},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.79600000000000004,"15min":0.82899999999999996},"min":{"1min":0.53600000000000003,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.94699999999999995,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.64900000000000002}]},{"osd":6,"last update":"Fri Oct 6 02:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.84499999999999997,"15min":0.876},"min":{"1min":0.57799999999999996,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.04,"5min":1.829,"15min":1.829},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.874,"15min":0.92200000000000004},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.97799999999999998,"5min":2.052,"15min":2.052},"last":0.71899999999999997}]},{"osd":7,"last update":"Fri Oct 6 02:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.81899999999999995,"15min":0.84299999999999997},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.91300000000000003,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.84099999999999997,"15min":0.88700000000000001},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.995,"5min":1.895,"15min":1.895},"last":0.55000000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574906,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302832,"kb_used_data":1448,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069008,"statfs":{"total":96636764160,"available":96326664192,"internally_reserved":0,"allocated":1482752,"data_stored":984015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 02:27:05 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.44,"15min":0.45600000000000002},"min":{"1min":0.32000000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.54300000000000004,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.439,"5min":0.47399999999999998,"15min":0.48199999999999998},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.58799999999999997,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.38200000000000001}]},{"osd":2,"last update":"Fri Oct 6 02:27:05 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.498,"15min":0.51400000000000001},"min":{"1min":0.40899999999999997,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.60699999999999998,"5min":1.069,"15min":1.069},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.51100000000000001,"15min":0.52200000000000002},"min":{"1min":0.39600000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.73099999999999998,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.45000000000000001}]},{"osd":3,"last update":"Fri Oct 6 02:27:05 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.49399999999999999,"15min":0.50600000000000001},"min":{"1min":0.34100000000000003,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.59599999999999997,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59499999999999997,"15min":0.59699999999999998},"min":{"1min":0.47299999999999998,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.73099999999999998,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.54900000000000004}]},{"osd":4,"last update":"Fri Oct 6 02:27:05 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.77000000000000002,"15min":0.80300000000000005},"min":{"1min":0.54800000000000004,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.86899999999999999,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.77700000000000002,"15min":0.80300000000000005},"min":{"1min":0.58099999999999996,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0469999999999999,"5min":1.304,"15min":1.304},"last":0.86899999999999999}]},{"osd":5,"last update":"Fri Oct 6 02:27:05 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.76600000000000001,"15min":0.78800000000000003},"min":{"1min":0.52600000000000002,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.94199999999999995,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.98999999999999999},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.79100000000000004,"15min":0.80300000000000005},"min":{"1min":0.59699999999999998,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.018,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.77200000000000002}]},{"osd":6,"last update":"Fri Oct 6 02:27:05 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.81299999999999994,"15min":0.82599999999999996},"min":{"1min":0.59699999999999998,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.96399999999999997,"5min":1.167,"15min":1.167},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.80500000000000005,"15min":0.83499999999999996},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.0349999999999999,"5min":1.069,"15min":1.069},"last":0.80900000000000005}]},{"osd":7,"last update":"Fri Oct 6 02:27:05 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.80500000000000005,"15min":0.82899999999999996},"min":{"1min":0.56899999999999995,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.98499999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.79600000000000004,"15min":0.82899999999999996},"min":{"1min":0.54700000000000004,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.002,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.79300000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574907,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302252,"kb_used_data":868,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069588,"statfs":{"total":96636764160,"available":96327258112,"internally_reserved":0,"allocated":888832,"data_stored":393647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:27:02 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.434,"15min":0.441},"min":{"1min":0.312,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.54100000000000004,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.501},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.48299999999999998,"15min":0.48799999999999999},"min":{"1min":0.32800000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.88700000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.64100000000000001}]},{"osd":1,"last update":"Fri Oct 6 02:27:02 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.5,"15min":0.50700000000000001},"min":{"1min":0.36699999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.63900000000000001,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.56399999999999995,"15min":0.61299999999999999},"min":{"1min":0.39600000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.70899999999999996,"5min":1.345,"15min":1.345},"last":0.57299999999999995}]},{"osd":3,"last update":"Fri Oct 6 02:27:02 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.52200000000000002,"15min":0.53600000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.69699999999999995,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.60699999999999998,"15min":0.624},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.72799999999999998,"5min":1.028,"15min":1.028},"last":0.66800000000000004}]},{"osd":4,"last update":"Fri Oct 6 02:27:02 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.76400000000000001,"15min":0.78100000000000003},"min":{"1min":0.55500000000000005,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.89300000000000002,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.82399999999999995,"15min":0.85499999999999998},"min":{"1min":0.59999999999999998,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.95999999999999996,"5min":1.621,"15min":1.621},"last":1.006}]},{"osd":5,"last update":"Fri Oct 6 02:27:02 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.77500000000000002,"15min":0.80800000000000005},"min":{"1min":0.57099999999999995,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.94899999999999995,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.80000000000000004,"15min":0.82899999999999996},"min":{"1min":0.60399999999999998,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":0.98199999999999998,"5min":1.258,"15min":1.258},"last":0.85299999999999998}]},{"osd":6,"last update":"Fri Oct 6 02:27:02 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.82299999999999995,"15min":0.85399999999999998},"min":{"1min":0.59099999999999997,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.98099999999999998,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":1.042},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.78900000000000003,"15min":0.79900000000000004},"min":{"1min":0.623,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.0109999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.89600000000000002}]},{"osd":7,"last update":"Fri Oct 6 02:27:02 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.85299999999999998,"15min":0.89200000000000002},"min":{"1min":0.66200000000000003,"5min":0.499,"15min":0.499},"max":{"1min":0.95999999999999996,"5min":1.585,"15min":1.585},"last":1.0660000000000001},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.81100000000000005,"15min":0.83699999999999997},"min":{"1min":0.58999999999999997,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.94399999999999995,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.82899999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574904,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302252,"kb_used_data":868,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069588,"statfs":{"total":96636764160,"available":96327258112,"internally_reserved":0,"allocated":888832,"data_stored":393647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:27:02 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48899999999999999,"15min":0.51700000000000002},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.69599999999999995,"5min":2.024,"15min":2.024},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.51200000000000001,"15min":0.52800000000000002},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.67400000000000004,"5min":2.181,"15min":2.181},"last":0.371}]},{"osd":1,"last update":"Fri Oct 6 02:27:02 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.58799999999999997,"15min":0.70299999999999996},"min":{"1min":0.39600000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.59299999999999997,"5min":4.492,"15min":4.492},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.63700000000000001,"15min":0.745},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.80000000000000004,"5min":4.569,"15min":4.569},"last":0.39600000000000002}]},{"osd":2,"last update":"Fri Oct 6 02:27:02 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.65800000000000003,"15min":0.67200000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.91100000000000003,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.625,"15min":0.65900000000000003},"min":{"1min":0.46200000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.94199999999999995,"5min":2.4260000000000002,"15min":2.4260000000000002},"last":0.63}]},{"osd":4,"last update":"Fri Oct 6 02:27:02 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.78600000000000003,"15min":0.83899999999999997},"min":{"1min":0.61599999999999999,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.96699999999999997,"5min":2.214,"15min":2.214},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.78600000000000003,"15min":0.81699999999999995},"min":{"1min":0.50800000000000001,"5min":0.5,"15min":0.5},"max":{"1min":0.94399999999999995,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.91500000000000004}]},{"osd":5,"last update":"Fri Oct 6 02:27:02 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.80500000000000005,"15min":0.82899999999999996},"min":{"1min":0.57899999999999996,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.97699999999999998,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.81100000000000005,"15min":0.86499999999999999},"min":{"1min":0.63800000000000001,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.0289999999999999,"5min":2.242,"15min":2.242},"last":0.83899999999999997}]},{"osd":6,"last update":"Fri Oct 6 02:27:02 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.79200000000000004,"15min":0.82399999999999995},"min":{"1min":0.59399999999999997,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.0049999999999999,"5min":1.357,"15min":1.357},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.81999999999999995,"15min":0.83799999999999997},"min":{"1min":0.54600000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.0529999999999999,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.77900000000000003}]},{"osd":7,"last update":"Fri Oct 6 02:27:02 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.83399999999999996,"15min":0.87},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.077,"5min":2.27,"15min":2.27},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.83099999999999996,"15min":0.86399999999999999},"min":{"1min":0.55600000000000005,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.056,"5min":1.325,"15min":1.325},"last":0.875}]}]},{"osd":4,"up_from":13,"seq":55834574906,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302252,"kb_used_data":868,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069588,"statfs":{"total":96636764160,"available":96327258112,"internally_reserved":0,"allocated":888832,"data_stored":393647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:27:03 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.86599999999999999,"15min":1.0549999999999999},"min":{"1min":0.45700000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.90700000000000003,"5min":12.337,"15min":12.337},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.81399999999999995,"15min":0.95099999999999996},"min":{"1min":0.496,"5min":0.438,"15min":0.438},"max":{"1min":0.91700000000000004,"5min":9.6539999999999999,"15min":9.6539999999999999},"last":0.56100000000000005}]},{"osd":1,"last update":"Fri Oct 6 02:27:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.69999999999999996,"15min":0.70599999999999996},"min":{"1min":0.53800000000000003,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.96999999999999997,"5min":3.181,"15min":3.181},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.876,"15min":1.0740000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.90000000000000002,"5min":12.946999999999999,"15min":12.946999999999999},"last":0.67200000000000004}]},{"osd":2,"last update":"Fri Oct 6 02:27:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.89700000000000002,"15min":1.103},"min":{"1min":0.44400000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.93300000000000005,"5min":12.377000000000001,"15min":12.377000000000001},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.80100000000000005,"15min":0.89500000000000002},"min":{"1min":0.53800000000000003,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.95899999999999996,"5min":6.3220000000000001,"15min":6.3220000000000001},"last":0.73399999999999999}]},{"osd":3,"last update":"Fri Oct 6 02:27:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.873,"15min":1.0309999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.88500000000000001,"5min":9.7720000000000002,"15min":9.7720000000000002},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.745,"15min":0.78500000000000003},"min":{"1min":0.498,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.91600000000000004,"5min":3.0840000000000001,"15min":3.0840000000000001},"last":0.70999999999999996}]},{"osd":5,"last update":"Fri Oct 6 02:27:03 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.76100000000000001,"15min":0.86899999999999999},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.82899999999999996,"5min":6.2690000000000001,"15min":6.2690000000000001},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.874,"15min":1.0840000000000001},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.90900000000000003,"5min":10.769,"15min":10.769},"last":0.747}]},{"osd":6,"last update":"Fri Oct 6 02:27:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.71199999999999997,"15min":0.77000000000000002},"min":{"1min":0.44,"5min":0.432,"15min":0.432},"max":{"1min":0.84299999999999997,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.85599999999999998,"15min":1.0029999999999999},"min":{"1min":0.49199999999999999,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.94399999999999995,"5min":8.0899999999999999,"15min":8.0899999999999999},"last":0.76600000000000001}]},{"osd":7,"last update":"Fri Oct 6 02:27:03 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.92600000000000005,"15min":1.113},"min":{"1min":0.50600000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.93899999999999995,"5min":9.7089999999999996,"15min":9.7089999999999996},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.88800000000000001,"15min":1.034},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.97799999999999998,"5min":7.6719999999999997,"15min":7.6719999999999997},"last":0.86099999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574906,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298156,"kb_used_data":868,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073684,"statfs":{"total":96636764160,"available":96331452416,"internally_reserved":0,"allocated":888832,"data_stored":393647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:27:00 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.64200000000000002,"15min":0.626},"min":{"1min":0.44700000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.84899999999999998,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.64900000000000002,"15min":0.60799999999999998},"min":{"1min":0.41699999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.93999999999999995,"5min":1.026,"15min":1.026},"last":0.58199999999999996}]},{"osd":1,"last update":"Fri Oct 6 02:27:00 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.67000000000000004,"15min":0.65000000000000002},"min":{"1min":0.438,"5min":0.434,"15min":0.434},"max":{"1min":0.95399999999999996,"5min":1.089,"15min":1.089},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.68899999999999995,"15min":0.67500000000000004},"min":{"1min":0.53200000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.88300000000000001,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.52700000000000002}]},{"osd":2,"last update":"Fri Oct 6 02:27:00 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.69599999999999995,"15min":0.66800000000000004},"min":{"1min":0.499,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.98999999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.70099999999999996,"15min":0.68300000000000005},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.96099999999999997,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.497}]},{"osd":3,"last update":"Fri Oct 6 02:27:00 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.748,"15min":0.72599999999999998},"min":{"1min":0.58399999999999996,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.93999999999999995,"5min":1.042,"15min":1.042},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.78200000000000003,"15min":0.76400000000000001},"min":{"1min":0.59499999999999997,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.0489999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.73699999999999999}]},{"osd":4,"last update":"Fri Oct 6 02:27:00 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.66100000000000003,"15min":0.67300000000000004},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.89000000000000001,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.624,"5min":0.67100000000000004,"15min":0.68200000000000005},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.86499999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.47999999999999998}]},{"osd":6,"last update":"Fri Oct 6 02:27:00 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.73899999999999999,"15min":0.74299999999999999},"min":{"1min":0.45700000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.0549999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.81499999999999995,"15min":0.82099999999999995},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.03,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.754}]},{"osd":7,"last update":"Fri Oct 6 02:27:00 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.75900000000000001,"15min":0.76300000000000001},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.0009999999999999,"5min":1.089,"15min":1.089},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.81799999999999995,"15min":0.83599999999999997},"min":{"1min":0.46999999999999997,"5min":0.437,"15min":0.437},"max":{"1min":1.0229999999999999,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.63200000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T02:27:27.892 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T02:27:27.893 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T02:27:28.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.121+0000 7fcc2d19d700 1 Processor -- start 2023-10-06T02:27:28.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.121+0000 7fcc2d19d700 1 -- start start 2023-10-06T02:27:28.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.122+0000 7fcc2d19d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcc280accb0 0x7fcc280aaed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:28.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.122+0000 7fcc2d19d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fcc280cc310 0x7fcc2806ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:28.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.122+0000 7fcc2d19d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc280ad780 con 0x7fcc280cbea0 2023-10-06T02:27:28.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.122+0000 7fcc2d19d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc280cee60 con 0x7fcc280cc310 2023-10-06T02:27:28.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.122+0000 7fcc2d19d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc280cf140 con 0x7fcc280accb0 2023-10-06T02:27:28.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.122+0000 7fcc277fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fcc280cbea0 0x7fcc2806e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:42598/0 (socket says 172.21.15.31:42598) 2023-10-06T02:27:28.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.122+0000 7fcc277fe700 1 -- 172.21.15.31:0/873081807 learned_addr learned my addr 172.21.15.31:0/873081807 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:27:28.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.122+0000 7fcc27fff700 1 --2- 172.21.15.31:0/873081807 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcc280accb0 0x7fcc280aaed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:28.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.122+0000 7fcc2c99c700 1 --2- 172.21.15.31:0/873081807 >> v2:172.21.15.62:3300/0 conn(0x7fcc280cc310 0x7fcc2806ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:28.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.123+0000 7fcc26ffd700 1 -- 172.21.15.31:0/873081807 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2021104584 0 0) 0x7fcc280ad780 con 0x7fcc280cbea0 2023-10-06T02:27:28.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.123+0000 7fcc26ffd700 1 -- 172.21.15.31:0/873081807 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc10003570 con 0x7fcc280cbea0 2023-10-06T02:27:28.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.123+0000 7fcc26ffd700 1 -- 172.21.15.31:0/873081807 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2990124161 0 0) 0x7fcc10003570 con 0x7fcc280cbea0 2023-10-06T02:27:28.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.123+0000 7fcc26ffd700 1 -- 172.21.15.31:0/873081807 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcc280accb0 msgr2=0x7fcc280aaed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:28.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.123+0000 7fcc26ffd700 1 --2- 172.21.15.31:0/873081807 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcc280accb0 0x7fcc280aaed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:28.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.123+0000 7fcc26ffd700 1 -- 172.21.15.31:0/873081807 >> v2:172.21.15.62:3300/0 conn(0x7fcc280cc310 msgr2=0x7fcc2806ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:28.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.123+0000 7fcc26ffd700 1 --2- 172.21.15.31:0/873081807 >> v2:172.21.15.62:3300/0 conn(0x7fcc280cc310 0x7fcc2806ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:28.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.123+0000 7fcc26ffd700 1 -- 172.21.15.31:0/873081807 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc2806d850 con 0x7fcc280cbea0 2023-10-06T02:27:28.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.123+0000 7fcc26ffd700 1 -- 172.21.15.31:0/873081807 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7fcc1c001470 con 0x7fcc280cbea0 2023-10-06T02:27:28.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.123+0000 7fcc2c99c700 1 --2- 172.21.15.31:0/873081807 >> v2:172.21.15.62:3300/0 conn(0x7fcc280cc310 0x7fcc2806ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:27:28.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.123+0000 7fcc26ffd700 1 -- 172.21.15.31:0/873081807 >> v1:172.21.15.31:6789/0 conn(0x7fcc280cbea0 legacy=0x7fcc2806e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:28.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.123+0000 7fcc26ffd700 1 --2- 172.21.15.31:0/873081807 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcc10003a60 0x7fcc10003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:28.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.123+0000 7fcc26ffd700 1 --2- 172.21.15.31:0/873081807 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc10004810 0x7fcc10004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:28.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.123+0000 7fcc26ffd700 1 --2- 172.21.15.31:0/873081807 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc100051c0 0x7fcc1000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:28.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.123+0000 7fcc26ffd700 1 -- 172.21.15.31:0/873081807 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc280cee60 con 0x7fcc100051c0 2023-10-06T02:27:28.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.123+0000 7fcc26ffd700 1 -- 172.21.15.31:0/873081807 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc280cf140 con 0x7fcc10004810 2023-10-06T02:27:28.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.123+0000 7fcc26ffd700 1 -- 172.21.15.31:0/873081807 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1000bfa0 con 0x7fcc10003a60 2023-10-06T02:27:28.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.123+0000 7fcc27fff700 1 --2- 172.21.15.31:0/873081807 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcc280accb0 0x7fcc280aaed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:27:28.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.123+0000 7fcc2c99c700 1 --2- 172.21.15.31:0/873081807 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc100051c0 0x7fcc1000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:28.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.123+0000 7fcc277fe700 1 --2- 172.21.15.31:0/873081807 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc10004810 0x7fcc10004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:28.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.124+0000 7fcc27fff700 1 --2- 172.21.15.31:0/873081807 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcc10003a60 0x7fcc10003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:28.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.124+0000 7fcc2c99c700 1 -- 172.21.15.31:0/873081807 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcc10003a60 msgr2=0x7fcc10003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:28.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.124+0000 7fcc2c99c700 1 --2- 172.21.15.31:0/873081807 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcc10003a60 0x7fcc10003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:28.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.124+0000 7fcc2c99c700 1 -- 172.21.15.31:0/873081807 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc10004810 msgr2=0x7fcc10004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:28.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.124+0000 7fcc2c99c700 1 --2- 172.21.15.31:0/873081807 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc10004810 0x7fcc10004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:28.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.124+0000 7fcc2c99c700 1 -- 172.21.15.31:0/873081807 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc10003570 con 0x7fcc100051c0 2023-10-06T02:27:28.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.124+0000 7fcc2c99c700 1 --2- 172.21.15.31:0/873081807 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc100051c0 0x7fcc1000ba60 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fcc2000e3c0 tx=0x7fcc2000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ca156d7d6140175 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:28.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.124+0000 7fcc26ffd700 1 -- 172.21.15.31:0/873081807 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc2000ec70 con 0x7fcc100051c0 2023-10-06T02:27:28.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.124+0000 7fcc26ffd700 1 -- 172.21.15.31:0/873081807 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc2000f040 con 0x7fcc100051c0 2023-10-06T02:27:28.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.124+0000 7fcc26ffd700 1 -- 172.21.15.31:0/873081807 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc20014480 con 0x7fcc100051c0 2023-10-06T02:27:28.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.125+0000 7fcc2d19d700 1 -- 172.21.15.31:0/873081807 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc100051c0 msgr2=0x7fcc1000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:28.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.125+0000 7fcc2d19d700 1 --2- 172.21.15.31:0/873081807 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc100051c0 0x7fcc1000ba60 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fcc2000e3c0 tx=0x7fcc2000b040 comp rx=0 tx=0).stop 2023-10-06T02:27:28.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.125+0000 7fcc2c99c700 1 -- 172.21.15.31:0/873081807 reap_dead start 2023-10-06T02:27:28.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.125+0000 7fcc2d19d700 1 -- 172.21.15.31:0/873081807 shutdown_connections 2023-10-06T02:27:28.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.125+0000 7fcc2d19d700 1 -- 172.21.15.31:0/873081807 >> 172.21.15.31:0/873081807 conn(0x7fcc2806ce80 msgr2=0x7fcc280ce370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:28.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.125+0000 7fcc2d19d700 1 -- 172.21.15.31:0/873081807 shutdown_connections 2023-10-06T02:27:28.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.125+0000 7fcc2d19d700 1 -- 172.21.15.31:0/873081807 wait complete. 2023-10-06T02:27:28.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.126+0000 7fcc2d19d700 1 Processor -- start 2023-10-06T02:27:28.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.126+0000 7fcc2d19d700 1 -- start start 2023-10-06T02:27:28.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.127+0000 7fcc2d19d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcc100051c0 0x7fcc281517b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:28.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.127+0000 7fcc2d19d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc280aff80 0x7fcc280b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:28.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.127+0000 7fcc2d19d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc280c5c60 0x7fcc280c60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:28.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.127+0000 7fcc2d19d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc280ae640 con 0x7fcc280aff80 2023-10-06T02:27:28.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.127+0000 7fcc2d19d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc280ada30 con 0x7fcc280c5c60 2023-10-06T02:27:28.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.127+0000 7fcc2d19d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc280add10 con 0x7fcc100051c0 2023-10-06T02:27:28.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.128+0000 7fcc277fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc280aff80 0x7fcc280b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:28.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.128+0000 7fcc277fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc280aff80 0x7fcc280b03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55106/0 (socket says 172.21.15.31:55106) 2023-10-06T02:27:28.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.128+0000 7fcc277fe700 1 -- 172.21.15.31:0/3322640143 learned_addr learned my addr 172.21.15.31:0/3322640143 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:28.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.128+0000 7fcc2c99c700 1 --2- 172.21.15.31:0/3322640143 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc280c5c60 0x7fcc280c60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:28.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.128+0000 7fcc27fff700 1 --2- 172.21.15.31:0/3322640143 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcc100051c0 0x7fcc281517b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:28.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.128+0000 7fcc277fe700 1 -- 172.21.15.31:0/3322640143 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcc100051c0 msgr2=0x7fcc281517b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:28.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.128+0000 7fcc277fe700 1 --2- 172.21.15.31:0/3322640143 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcc100051c0 0x7fcc281517b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:28.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.128+0000 7fcc277fe700 1 -- 172.21.15.31:0/3322640143 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc280c5c60 msgr2=0x7fcc280c60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:28.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.128+0000 7fcc277fe700 1 --2- 172.21.15.31:0/3322640143 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc280c5c60 0x7fcc280c60d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:28.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.128+0000 7fcc277fe700 1 -- 172.21.15.31:0/3322640143 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc280adff0 con 0x7fcc280aff80 2023-10-06T02:27:28.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.128+0000 7fcc277fe700 1 --2- 172.21.15.31:0/3322640143 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc280aff80 0x7fcc280b03f0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c001d20 tx=0x7fcc1c001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:28.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.129+0000 7fcc0ffff700 1 -- 172.21.15.31:0/3322640143 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c018030 con 0x7fcc280aff80 2023-10-06T02:27:28.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.129+0000 7fcc0ffff700 1 -- 172.21.15.31:0/3322640143 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc1c00f8d0 con 0x7fcc280aff80 2023-10-06T02:27:28.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.129+0000 7fcc0ffff700 1 -- 172.21.15.31:0/3322640143 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c00fc90 con 0x7fcc280aff80 2023-10-06T02:27:28.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.129+0000 7fcc2d19d700 1 -- 172.21.15.31:0/3322640143 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc280ae2d0 con 0x7fcc280aff80 2023-10-06T02:27:28.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.129+0000 7fcc2d19d700 1 -- 172.21.15.31:0/3322640143 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc280c2cc0 con 0x7fcc280aff80 2023-10-06T02:27:28.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.130+0000 7fcc2d19d700 1 -- 172.21.15.31:0/3322640143 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc280ae7b0 con 0x7fcc280aff80 2023-10-06T02:27:28.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.131+0000 7fcc0ffff700 1 -- 172.21.15.31:0/3322640143 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fcc1c01b070 con 0x7fcc280aff80 2023-10-06T02:27:28.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.132+0000 7fcc0ffff700 1 --2- 172.21.15.31:0/3322640143 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fcc14067290 0x7fcc14069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:28.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.132+0000 7fcc0ffff700 1 -- 172.21.15.31:0/3322640143 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fcc1c019070 con 0x7fcc280aff80 2023-10-06T02:27:28.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.133+0000 7fcc27fff700 1 --2- 172.21.15.31:0/3322640143 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fcc14067290 0x7fcc14069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:28.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.133+0000 7fcc27fff700 1 --2- 172.21.15.31:0/3322640143 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fcc14067290 0x7fcc14069740 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fcc2806a9c0 tx=0x7fcc18008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:28.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.137+0000 7fcc0ffff700 1 -- 172.21.15.31:0/3322640143 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcc1c0517e0 con 0x7fcc280aff80 2023-10-06T02:27:28.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.420+0000 7fcc2d19d700 1 -- 172.21.15.31:0/3322640143 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fcc2804c3c0 con 0x7fcc280aff80 2023-10-06T02:27:28.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.424+0000 7fcc0ffff700 1 -- 172.21.15.31:0/3322640143 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) v1 ==== 190+0+0 (secure 0 0 0) 0x7fcc1c054e00 con 0x7fcc280aff80 2023-10-06T02:27:28.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.427+0000 7fcc2d19d700 1 -- 172.21.15.31:0/3322640143 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fcc14067290 msgr2=0x7fcc14069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:28.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.427+0000 7fcc2d19d700 1 --2- 172.21.15.31:0/3322640143 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fcc14067290 0x7fcc14069740 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fcc2806a9c0 tx=0x7fcc18008040 comp rx=0 tx=0).stop 2023-10-06T02:27:28.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.427+0000 7fcc2d19d700 1 -- 172.21.15.31:0/3322640143 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc280aff80 msgr2=0x7fcc280b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:28.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.427+0000 7fcc2d19d700 1 --2- 172.21.15.31:0/3322640143 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc280aff80 0x7fcc280b03f0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c001d20 tx=0x7fcc1c001e90 comp rx=0 tx=0).stop 2023-10-06T02:27:28.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.427+0000 7fcc2d19d700 1 -- 172.21.15.31:0/3322640143 shutdown_connections 2023-10-06T02:27:28.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.427+0000 7fcc2d19d700 1 --2- 172.21.15.31:0/3322640143 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc280c5c60 0x7fcc280c60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:28.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.427+0000 7fcc2d19d700 1 --2- 172.21.15.31:0/3322640143 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fcc14067290 0x7fcc14069740 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:28.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.427+0000 7fcc2d19d700 1 --2- 172.21.15.31:0/3322640143 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcc100051c0 0x7fcc281517b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:28.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.427+0000 7fcc2d19d700 1 --2- 172.21.15.31:0/3322640143 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc280aff80 0x7fcc280b03f0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:28.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.427+0000 7fcc2d19d700 1 -- 172.21.15.31:0/3322640143 >> 172.21.15.31:0/3322640143 conn(0x7fcc2806ce80 msgr2=0x7fcc280ce210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:28.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.427+0000 7fcc2d19d700 1 -- 172.21.15.31:0/3322640143 shutdown_connections 2023-10-06T02:27:28.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.427+0000 7fcc2d19d700 1 -- 172.21.15.31:0/3322640143 wait complete. 2023-10-06T02:27:28.430 INFO:teuthology.orchestra.run.smithi031.stderr:updated caps for client.0 2023-10-06T02:27:28.445 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:27:28.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.631+0000 7f7367fff700 1 Processor -- start 2023-10-06T02:27:28.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.631+0000 7f7367fff700 1 -- start start 2023-10-06T02:27:28.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.633+0000 7f7367fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f736806e0a0 0x7f73680c6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:28.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.633+0000 7f7367fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7368150380 0x7f736806a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:28.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.633+0000 7f7367fff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f736806a490 con 0x7f736806f3a0 2023-10-06T02:27:28.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.633+0000 7f7367fff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73680c7490 con 0x7f7368150380 2023-10-06T02:27:28.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.633+0000 7f7367fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73680c7770 con 0x7f736806e0a0 2023-10-06T02:27:28.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.633+0000 7f73667fc700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f736806f3a0 0x7f736814fc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:42600/0 (socket says 172.21.15.31:42600) 2023-10-06T02:27:28.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.633+0000 7f73667fc700 1 -- 172.21.15.31:0/2572336155 learned_addr learned my addr 172.21.15.31:0/2572336155 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:27:28.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.633+0000 7f73677fe700 1 --2- 172.21.15.31:0/2572336155 >> v2:172.21.15.31:3300/0 conn(0x7f7368150380 0x7f736806a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:28.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.633+0000 7f7366ffd700 1 --2- 172.21.15.31:0/2572336155 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f736806e0a0 0x7f73680c6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:28.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.634+0000 7f7365ffb700 1 -- 172.21.15.31:0/2572336155 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1337118730 0 0) 0x7f736806a490 con 0x7f736806f3a0 2023-10-06T02:27:28.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.634+0000 7f7365ffb700 1 -- 172.21.15.31:0/2572336155 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7350004680 con 0x7f736806f3a0 2023-10-06T02:27:28.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.634+0000 7f73677fe700 1 -- 172.21.15.31:0/2572336155 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f736806e0a0 msgr2=0x7f73680c6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:28.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.634+0000 7f73677fe700 1 --2- 172.21.15.31:0/2572336155 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f736806e0a0 0x7f73680c6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:28.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.634+0000 7f73677fe700 1 -- 172.21.15.31:0/2572336155 >> v1:172.21.15.31:6789/0 conn(0x7f736806f3a0 legacy=0x7f736814fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:28.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.634+0000 7f73677fe700 1 -- 172.21.15.31:0/2572336155 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73680c7a50 con 0x7f7368150380 2023-10-06T02:27:28.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.634+0000 7f7366ffd700 1 --2- 172.21.15.31:0/2572336155 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f736806e0a0 0x7f73680c6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:27:28.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.634+0000 7f73677fe700 1 --2- 172.21.15.31:0/2572336155 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7368150380 0x7f736806a920 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f7360009e60 tx=0x7f7360009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22b56f77e07d05f1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:28.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.634+0000 7f7365ffb700 1 -- 172.21.15.31:0/2572336155 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f736000e070 con 0x7f7368150380 2023-10-06T02:27:28.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.634+0000 7f7365ffb700 1 -- 172.21.15.31:0/2572336155 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f736000bb60 con 0x7f7368150380 2023-10-06T02:27:28.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.634+0000 7f7365ffb700 1 -- 172.21.15.31:0/2572336155 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7360012430 con 0x7f7368150380 2023-10-06T02:27:28.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.635+0000 7f7367fff700 1 -- 172.21.15.31:0/2572336155 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7368150380 msgr2=0x7f736806a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:28.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.635+0000 7f7367fff700 1 --2- 172.21.15.31:0/2572336155 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7368150380 0x7f736806a920 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f7360009e60 tx=0x7f7360009780 comp rx=0 tx=0).stop 2023-10-06T02:27:28.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.635+0000 7f7367fff700 1 -- 172.21.15.31:0/2572336155 shutdown_connections 2023-10-06T02:27:28.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.635+0000 7f7367fff700 1 --2- 172.21.15.31:0/2572336155 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7368150380 0x7f736806a920 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:28.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.635+0000 7f7367fff700 1 --2- 172.21.15.31:0/2572336155 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f736806e0a0 0x7f73680c6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:28.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.635+0000 7f7367fff700 1 -- 172.21.15.31:0/2572336155 >> 172.21.15.31:0/2572336155 conn(0x7f736806d170 msgr2=0x7f73680cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:28.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.635+0000 7f7367fff700 1 -- 172.21.15.31:0/2572336155 shutdown_connections 2023-10-06T02:27:28.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.635+0000 7f7367fff700 1 -- 172.21.15.31:0/2572336155 wait complete. 2023-10-06T02:27:28.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.636+0000 7f7367fff700 1 Processor -- start 2023-10-06T02:27:28.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.636+0000 7f7367fff700 1 -- start start 2023-10-06T02:27:28.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.637+0000 7f7367fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f736806e0a0 0x7f736816c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:28.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.637+0000 7f7367fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f736806f3a0 0x7f73680b0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:28.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.637+0000 7f7367fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7368150380 0x7f73680af120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:28.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.637+0000 7f7367fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7368157970 con 0x7f736806f3a0 2023-10-06T02:27:28.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.637+0000 7f7367fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f736816d940 con 0x7f7368150380 2023-10-06T02:27:28.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.637+0000 7f7367fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73680b09d0 con 0x7f736806e0a0 2023-10-06T02:27:28.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.638+0000 7f73667fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f736806f3a0 0x7f73680b0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:28.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.638+0000 7f73667fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f736806f3a0 0x7f73680b0490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55120/0 (socket says 172.21.15.31:55120) 2023-10-06T02:27:28.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.638+0000 7f73667fc700 1 -- 172.21.15.31:0/1367205580 learned_addr learned my addr 172.21.15.31:0/1367205580 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:28.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.638+0000 7f73677fe700 1 --2- 172.21.15.31:0/1367205580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7368150380 0x7f73680af120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:28.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.638+0000 7f7366ffd700 1 --2- 172.21.15.31:0/1367205580 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f736806e0a0 0x7f736816c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:28.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.638+0000 7f73667fc700 1 -- 172.21.15.31:0/1367205580 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f736806e0a0 msgr2=0x7f736816c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:28.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.638+0000 7f73667fc700 1 --2- 172.21.15.31:0/1367205580 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f736806e0a0 0x7f736816c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:28.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.638+0000 7f73667fc700 1 -- 172.21.15.31:0/1367205580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7368150380 msgr2=0x7f73680af120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:28.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.638+0000 7f73667fc700 1 --2- 172.21.15.31:0/1367205580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7368150380 0x7f73680af120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:28.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.638+0000 7f73667fc700 1 -- 172.21.15.31:0/1367205580 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7360009a20 con 0x7f736806f3a0 2023-10-06T02:27:28.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.638+0000 7f7366ffd700 1 --2- 172.21.15.31:0/1367205580 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f736806e0a0 0x7f736816c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:27:28.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.638+0000 7f73667fc700 1 --2- 172.21.15.31:0/1367205580 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f736806f3a0 0x7f73680b0490 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f735c00c7f0 tx=0x7f735c007c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:28.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.638+0000 7f734effd700 1 -- 172.21.15.31:0/1367205580 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f735c0072e0 con 0x7f736806f3a0 2023-10-06T02:27:28.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.639+0000 7f734effd700 1 -- 172.21.15.31:0/1367205580 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f735c007440 con 0x7f736806f3a0 2023-10-06T02:27:28.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.639+0000 7f734effd700 1 -- 172.21.15.31:0/1367205580 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f735c0057f0 con 0x7f736806f3a0 2023-10-06T02:27:28.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.639+0000 7f7367fff700 1 -- 172.21.15.31:0/1367205580 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f736816d0a0 con 0x7f736806f3a0 2023-10-06T02:27:28.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.639+0000 7f7367fff700 1 -- 172.21.15.31:0/1367205580 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f736816d660 con 0x7f736806f3a0 2023-10-06T02:27:28.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.640+0000 7f7367fff700 1 -- 172.21.15.31:0/1367205580 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7368157ae0 con 0x7f736806f3a0 2023-10-06T02:27:28.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.640+0000 7f734effd700 1 -- 172.21.15.31:0/1367205580 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f735c005c00 con 0x7f736806f3a0 2023-10-06T02:27:28.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.642+0000 7f734effd700 1 --2- 172.21.15.31:0/1367205580 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f7354067290 0x7f7354069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:28.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.642+0000 7f734effd700 1 -- 172.21.15.31:0/1367205580 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f735c084150 con 0x7f736806f3a0 2023-10-06T02:27:28.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.642+0000 7f7366ffd700 1 --2- 172.21.15.31:0/1367205580 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f7354067290 0x7f7354069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:28.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.642+0000 7f7366ffd700 1 --2- 172.21.15.31:0/1367205580 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f7354067290 0x7f7354069740 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f7358006f00 tx=0x7f7358008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:28.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.645+0000 7f734effd700 1 -- 172.21.15.31:0/1367205580 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7368157ae0 con 0x7f736806f3a0 2023-10-06T02:27:28.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.891+0000 7f7367fff700 1 -- 172.21.15.31:0/1367205580 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7368157ae0 con 0x7f736806f3a0 2023-10-06T02:27:28.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.891+0000 7f734effd700 1 -- 172.21.15.31:0/1367205580 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f7368157ae0 con 0x7f736806f3a0 2023-10-06T02:27:28.893 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:27:28.893 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":46,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5474,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/2085637096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2085637096},{"type":"v1","addr":"172.21.15.62:6834","nonce":2085637096}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":5992,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/573080449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":573080449},{"type":"v1","addr":"172.21.15.31:6836","nonce":573080449}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":6007,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2625778079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2625778079},{"type":"v1","addr":"172.21.15.62:6835","nonce":2625778079}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:27:22.533533+0000","modified":"2023-10-06T02:27:26.062249+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6282":{"gid":6282,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6837/3003674295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":3003674295},{"type":"v1","addr":"172.21.15.31:6837","nonce":3003674295}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T02:27:28.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.894+0000 7f7367fff700 1 -- 172.21.15.31:0/1367205580 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f7354067290 msgr2=0x7f7354069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:28.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.894+0000 7f7367fff700 1 --2- 172.21.15.31:0/1367205580 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f7354067290 0x7f7354069740 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f7358006f00 tx=0x7f7358008040 comp rx=0 tx=0).stop 2023-10-06T02:27:28.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.894+0000 7f7367fff700 1 -- 172.21.15.31:0/1367205580 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f736806f3a0 msgr2=0x7f73680b0490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:28.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.894+0000 7f7367fff700 1 --2- 172.21.15.31:0/1367205580 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f736806f3a0 0x7f73680b0490 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f735c00c7f0 tx=0x7f735c007c40 comp rx=0 tx=0).stop 2023-10-06T02:27:28.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.894+0000 7f7367fff700 1 -- 172.21.15.31:0/1367205580 shutdown_connections 2023-10-06T02:27:28.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.894+0000 7f7367fff700 1 --2- 172.21.15.31:0/1367205580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7368150380 0x7f73680af120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:28.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.894+0000 7f7367fff700 1 --2- 172.21.15.31:0/1367205580 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f7354067290 0x7f7354069740 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:28.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.894+0000 7f7367fff700 1 --2- 172.21.15.31:0/1367205580 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f736806e0a0 0x7f736816c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:28.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.894+0000 7f7367fff700 1 --2- 172.21.15.31:0/1367205580 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f736806f3a0 0x7f73680b0490 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:28.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.894+0000 7f7367fff700 1 -- 172.21.15.31:0/1367205580 >> 172.21.15.31:0/1367205580 conn(0x7f736806d170 msgr2=0x7f73680ca990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:28.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.894+0000 7f7367fff700 1 -- 172.21.15.31:0/1367205580 shutdown_connections 2023-10-06T02:27:28.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:28.894+0000 7f7367fff700 1 -- 172.21.15.31:0/1367205580 wait complete. 2023-10-06T02:27:28.897 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 46 2023-10-06T02:27:29.317 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 46, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T02:27:22.533533+0000', 'modified': '2023-10-06T02:27:26.062249+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6282}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6282': {'gid': 6282, 'name': 'c', 'rank': 0, 'incarnation': 42, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.31:6837/3003674295', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6835', 'nonce': 3003674295}, {'type': 'v1', 'addr': '172.21.15.31:6837', 'nonce': 3003674295}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T02:27:29.317 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T02:27:29.317 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-06T02:27:29.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.506+0000 7fde8e59d700 1 Processor -- start 2023-10-06T02:27:29.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.506+0000 7fde8e59d700 1 -- start start 2023-10-06T02:27:29.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.508+0000 7fde8e59d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fde88140a70 0x7fde880c6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:29.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.508+0000 7fde8e59d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fde88140e40 0x7fde880c7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:29.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.508+0000 7fde8e59d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fde880c8330 0x7fde880aeef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:29.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.508+0000 7fde8e59d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde880c6d50 con 0x7fde88140a70 2023-10-06T02:27:29.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.508+0000 7fde8e59d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde880c7030 con 0x7fde880c8330 2023-10-06T02:27:29.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.508+0000 7fde8e59d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde880c7310 con 0x7fde88140e40 2023-10-06T02:27:29.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.508+0000 7fde8d59b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fde88140a70 0x7fde880c6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:29.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.508+0000 7fde8d59b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fde88140a70 0x7fde880c6770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55126/0 (socket says 172.21.15.31:55126) 2023-10-06T02:27:29.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.508+0000 7fde8d59b700 1 -- 172.21.15.31:0/1513771692 learned_addr learned my addr 172.21.15.31:0/1513771692 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:29.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.508+0000 7fde8cd9a700 1 --2- 172.21.15.31:0/1513771692 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fde88140e40 0x7fde880c7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:29.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.508+0000 7fde8dd9c700 1 --2- 172.21.15.31:0/1513771692 >> v2:172.21.15.62:3300/0 conn(0x7fde880c8330 0x7fde880aeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:29.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.508+0000 7fde8d59b700 1 -- 172.21.15.31:0/1513771692 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fde88140e40 msgr2=0x7fde880c7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:29.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.509+0000 7fde8d59b700 1 --2- 172.21.15.31:0/1513771692 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fde88140e40 0x7fde880c7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:29.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.509+0000 7fde8d59b700 1 -- 172.21.15.31:0/1513771692 >> v2:172.21.15.62:3300/0 conn(0x7fde880c8330 msgr2=0x7fde880aeef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:29.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.509+0000 7fde8d59b700 1 --2- 172.21.15.31:0/1513771692 >> v2:172.21.15.62:3300/0 conn(0x7fde880c8330 0x7fde880aeef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:29.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.509+0000 7fde8d59b700 1 -- 172.21.15.31:0/1513771692 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde880c75f0 con 0x7fde88140a70 2023-10-06T02:27:29.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.509+0000 7fde8d59b700 1 --2- 172.21.15.31:0/1513771692 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde88140a70 0x7fde880c6770 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fde88069a90 tx=0x7fde84009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=111eede446ad04c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:29.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.509+0000 7fde7ffff700 1 -- 172.21.15.31:0/1513771692 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde8400b640 con 0x7fde88140a70 2023-10-06T02:27:29.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.509+0000 7fde7ffff700 1 -- 172.21.15.31:0/1513771692 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde8400b7a0 con 0x7fde88140a70 2023-10-06T02:27:29.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.509+0000 7fde7ffff700 1 -- 172.21.15.31:0/1513771692 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde8400bb60 con 0x7fde88140a70 2023-10-06T02:27:29.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.510+0000 7fde8e59d700 1 -- 172.21.15.31:0/1513771692 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde88140a70 msgr2=0x7fde880c6770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:29.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.510+0000 7fde8e59d700 1 --2- 172.21.15.31:0/1513771692 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde88140a70 0x7fde880c6770 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fde88069a90 tx=0x7fde84009250 comp rx=0 tx=0).stop 2023-10-06T02:27:29.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.510+0000 7fde8e59d700 1 -- 172.21.15.31:0/1513771692 shutdown_connections 2023-10-06T02:27:29.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.510+0000 7fde8e59d700 1 --2- 172.21.15.31:0/1513771692 >> v2:172.21.15.62:3300/0 conn(0x7fde880c8330 0x7fde880aeef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:29.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.510+0000 7fde8e59d700 1 --2- 172.21.15.31:0/1513771692 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde88140a70 0x7fde880c6770 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:29.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.510+0000 7fde8e59d700 1 --2- 172.21.15.31:0/1513771692 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fde88140e40 0x7fde880c7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:29.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.510+0000 7fde8e59d700 1 -- 172.21.15.31:0/1513771692 >> 172.21.15.31:0/1513771692 conn(0x7fde8806d170 msgr2=0x7fde880cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:29.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.510+0000 7fde8e59d700 1 -- 172.21.15.31:0/1513771692 shutdown_connections 2023-10-06T02:27:29.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.510+0000 7fde8e59d700 1 -- 172.21.15.31:0/1513771692 wait complete. 2023-10-06T02:27:29.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.511+0000 7fde8e59d700 1 Processor -- start 2023-10-06T02:27:29.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.511+0000 7fde8e59d700 1 -- start start 2023-10-06T02:27:29.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.512+0000 7fde8e59d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde880c8330 0x7fde880ae0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:29.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.512+0000 7fde8e59d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fde88140a70 0x7fde88167ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:29.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.512+0000 7fde8e59d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde88140e40 0x7fde88168400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:29.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.512+0000 7fde8e59d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde8813d660 con 0x7fde88140e40 2023-10-06T02:27:29.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.512+0000 7fde8e59d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde8816bb50 con 0x7fde880c8330 2023-10-06T02:27:29.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.512+0000 7fde8e59d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde8816b520 con 0x7fde88140a70 2023-10-06T02:27:29.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.513+0000 7fde8dd9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde88140e40 0x7fde88168400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:29.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.513+0000 7fde8dd9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde88140e40 0x7fde88168400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55142/0 (socket says 172.21.15.31:55142) 2023-10-06T02:27:29.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.513+0000 7fde8dd9c700 1 -- 172.21.15.31:0/1710609540 learned_addr learned my addr 172.21.15.31:0/1710609540 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:29.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.513+0000 7fde8cd9a700 1 --2- 172.21.15.31:0/1710609540 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fde88140a70 0x7fde88167ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:29.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.513+0000 7fde8d59b700 1 --2- 172.21.15.31:0/1710609540 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde880c8330 0x7fde880ae0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:29.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.513+0000 7fde8dd9c700 1 -- 172.21.15.31:0/1710609540 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fde88140a70 msgr2=0x7fde88167ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:29.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.513+0000 7fde8dd9c700 1 --2- 172.21.15.31:0/1710609540 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fde88140a70 0x7fde88167ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:29.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.513+0000 7fde8dd9c700 1 -- 172.21.15.31:0/1710609540 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde880c8330 msgr2=0x7fde880ae0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:29.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.513+0000 7fde8dd9c700 1 --2- 172.21.15.31:0/1710609540 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde880c8330 0x7fde880ae0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:29.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.513+0000 7fde8dd9c700 1 -- 172.21.15.31:0/1710609540 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde840094f0 con 0x7fde88140e40 2023-10-06T02:27:29.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.513+0000 7fde8dd9c700 1 --2- 172.21.15.31:0/1710609540 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde88140e40 0x7fde88168400 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fde80009ef0 tx=0x7fde8000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:29.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.513+0000 7fde7d7fa700 1 -- 172.21.15.31:0/1710609540 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde8000cb70 con 0x7fde88140e40 2023-10-06T02:27:29.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.513+0000 7fde7d7fa700 1 -- 172.21.15.31:0/1710609540 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde8000f040 con 0x7fde88140e40 2023-10-06T02:27:29.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.514+0000 7fde7d7fa700 1 -- 172.21.15.31:0/1710609540 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde80014480 con 0x7fde88140e40 2023-10-06T02:27:29.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.514+0000 7fde8e59d700 1 -- 172.21.15.31:0/1710609540 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde88168a50 con 0x7fde88140e40 2023-10-06T02:27:29.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.514+0000 7fde8e59d700 1 -- 172.21.15.31:0/1710609540 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde8816b7e0 con 0x7fde88140e40 2023-10-06T02:27:29.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.517+0000 7fde7d7fa700 1 -- 172.21.15.31:0/1710609540 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fde800147f0 con 0x7fde88140e40 2023-10-06T02:27:29.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.517+0000 7fde7d7fa700 1 --2- 172.21.15.31:0/1710609540 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fde6c067290 0x7fde6c069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:29.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.518+0000 7fde7d7fa700 1 -- 172.21.15.31:0/1710609540 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fde80085920 con 0x7fde88140e40 2023-10-06T02:27:29.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.518+0000 7fde8d59b700 1 --2- 172.21.15.31:0/1710609540 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fde6c067290 0x7fde6c069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:29.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.518+0000 7fde8d59b700 1 --2- 172.21.15.31:0/1710609540 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fde6c067290 0x7fde6c069740 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fde88069a90 tx=0x7fde84017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:29.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.519+0000 7fde9de3fb80 1 Processor -- start 2023-10-06T02:27:29.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.519+0000 7fde9de3fb80 1 -- start start 2023-10-06T02:27:29.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.520+0000 7fde9de3fb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7cfaaee50 0x55b7cfa89cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:29.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.520+0000 7fde9de3fb80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b7cfa8f2c0 0x55b7cfa8a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:29.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.520+0000 7fde9de3fb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b7cfa90a10 0x55b7cfa8a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:29.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.520+0000 7fde9de3fb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7cf9e4a70 con 0x55b7cfaaee50 2023-10-06T02:27:29.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.520+0000 7fde9de3fb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7cf9e2270 con 0x55b7cfa90a10 2023-10-06T02:27:29.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.520+0000 7fde9de3fb80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b7cf9de060 con 0x55b7cfa8f2c0 2023-10-06T02:27:29.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.520+0000 7fde8d59b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7cfaaee50 0x55b7cfa89cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:29.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.520+0000 7fde8d59b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7cfaaee50 0x55b7cfa89cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55154/0 (socket says 172.21.15.31:55154) 2023-10-06T02:27:29.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.520+0000 7fde8d59b700 1 -- 172.21.15.31:0/1037318130 learned_addr learned my addr 172.21.15.31:0/1037318130 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:29.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.521+0000 7fde8cd9a700 1 --2- 172.21.15.31:0/1037318130 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b7cfa8f2c0 0x55b7cfa8a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:29.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.521+0000 7fde8dd9c700 1 --2- 172.21.15.31:0/1037318130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b7cfa90a10 0x55b7cfa8a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:29.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.521+0000 7fde8d59b700 1 -- 172.21.15.31:0/1037318130 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b7cfa8f2c0 msgr2=0x55b7cfa8a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:29.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.521+0000 7fde8d59b700 1 --2- 172.21.15.31:0/1037318130 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b7cfa8f2c0 0x55b7cfa8a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:29.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.521+0000 7fde8d59b700 1 -- 172.21.15.31:0/1037318130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b7cfa90a10 msgr2=0x55b7cfa8a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:29.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.521+0000 7fde8d59b700 1 --2- 172.21.15.31:0/1037318130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b7cfa90a10 0x55b7cfa8a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:29.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.521+0000 7fde8d59b700 1 -- 172.21.15.31:0/1037318130 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b7cf878270 con 0x55b7cfaaee50 2023-10-06T02:27:29.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.521+0000 7fde8dd9c700 1 --2- 172.21.15.31:0/1037318130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b7cfa90a10 0x55b7cfa8a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:27:29.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.521+0000 7fde8cd9a700 1 --2- 172.21.15.31:0/1037318130 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b7cfa8f2c0 0x55b7cfa8a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:27:29.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.521+0000 7fde8d59b700 1 --2- 172.21.15.31:0/1037318130 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7cfaaee50 0x55b7cfa89cc0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fde84006e80 tx=0x7fde84021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14dc14c3263e206a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:29.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.521+0000 7fde727fc700 1 -- 172.21.15.31:0/1037318130 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde8402d070 con 0x55b7cfaaee50 2023-10-06T02:27:29.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.521+0000 7fde727fc700 1 -- 172.21.15.31:0/1037318130 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde8401a880 con 0x55b7cfaaee50 2023-10-06T02:27:29.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.521+0000 7fde727fc700 1 -- 172.21.15.31:0/1037318130 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde84021070 con 0x55b7cfaaee50 2023-10-06T02:27:29.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.522+0000 7fde9de3fb80 1 -- 172.21.15.31:0/1037318130 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7cfaaee50 msgr2=0x55b7cfa89cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:29.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.522+0000 7fde9de3fb80 1 --2- 172.21.15.31:0/1037318130 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7cfaaee50 0x55b7cfa89cc0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fde84006e80 tx=0x7fde84021000 comp rx=0 tx=0).stop 2023-10-06T02:27:29.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.522+0000 7fde9de3fb80 1 -- 172.21.15.31:0/1037318130 shutdown_connections 2023-10-06T02:27:29.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.522+0000 7fde9de3fb80 1 --2- 172.21.15.31:0/1037318130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b7cfa90a10 0x55b7cfa8a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:29.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.522+0000 7fde9de3fb80 1 --2- 172.21.15.31:0/1037318130 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7cfaaee50 0x55b7cfa89cc0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:29.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.522+0000 7fde9de3fb80 1 --2- 172.21.15.31:0/1037318130 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b7cfa8f2c0 0x55b7cfa8a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:29.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.522+0000 7fde9de3fb80 1 -- 172.21.15.31:0/1037318130 >> 172.21.15.31:0/1037318130 conn(0x55b7cfa8ee80 msgr2=0x55b7cfa8f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:29.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.522+0000 7fde9de3fb80 1 -- 172.21.15.31:0/1037318130 shutdown_connections 2023-10-06T02:27:29.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.522+0000 7fde9de3fb80 1 -- 172.21.15.31:0/1037318130 wait complete. 2023-10-06T02:27:29.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.523+0000 7fde9de3fb80 1 -- start start 2023-10-06T02:27:29.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.523+0000 7fde9de3fb80 1 Processor -- start 2023-10-06T02:27:29.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.524+0000 7fde9de3fb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b7cfa8f2c0 0x55b7cfad4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:29.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.524+0000 7fde9de3fb80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b7cfa90a10 0x55b7cfaad2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:29.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.524+0000 7fde9de3fb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7cfaaee50 0x55b7cfabc570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:29.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.524+0000 7fde9de3fb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7cf989890 con 0x55b7cfaaee50 2023-10-06T02:27:29.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.524+0000 7fde9de3fb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7cf9a4ea0 con 0x55b7cfa8f2c0 2023-10-06T02:27:29.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.524+0000 7fde9de3fb80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b7cf860e30 con 0x55b7cfa90a10 2023-10-06T02:27:29.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.524+0000 7fde9de3fb80 10 client.0 _ll_register_callbacks cb 0x55b7cfa51020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T02:27:29.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.524+0000 7fde8dd9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7cfaaee50 0x55b7cfabc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:29.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.524+0000 7fde8dd9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7cfaaee50 0x55b7cfabc570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55166/0 (socket says 172.21.15.31:55166) 2023-10-06T02:27:29.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.524+0000 7fde8dd9c700 1 -- 172.21.15.31:0/79352664 learned_addr learned my addr 172.21.15.31:0/79352664 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:29.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.525+0000 7fde8cd9a700 1 --2- 172.21.15.31:0/79352664 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b7cfa90a10 0x55b7cfaad2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:29.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.525+0000 7fde8d59b700 1 --2- 172.21.15.31:0/79352664 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b7cfa8f2c0 0x55b7cfad4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:29.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.525+0000 7fde8dd9c700 1 -- 172.21.15.31:0/79352664 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b7cfa90a10 msgr2=0x55b7cfaad2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:29.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.525+0000 7fde8dd9c700 1 --2- 172.21.15.31:0/79352664 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b7cfa90a10 0x55b7cfaad2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:29.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.525+0000 7fde8dd9c700 1 -- 172.21.15.31:0/79352664 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b7cfa8f2c0 msgr2=0x55b7cfad4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:29.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.525+0000 7fde8dd9c700 1 --2- 172.21.15.31:0/79352664 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b7cfa8f2c0 0x55b7cfad4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:29.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.525+0000 7fde8dd9c700 1 -- 172.21.15.31:0/79352664 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde880c75f0 con 0x55b7cfaaee50 2023-10-06T02:27:29.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.525+0000 7fde8cd9a700 1 --2- 172.21.15.31:0/79352664 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b7cfa90a10 0x55b7cfaad2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:27:29.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.525+0000 7fde8dd9c700 1 --2- 172.21.15.31:0/79352664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7cfaaee50 0x55b7cfabc570 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fde80055ed0 tx=0x7fde80009e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8e89af97a8d437b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:29.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.525+0000 7fde677fe700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T02:27:29.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.525+0000 7fde677fe700 1 -- 172.21.15.31:0/79352664 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde80059420 con 0x55b7cfaaee50 2023-10-06T02:27:29.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.525+0000 7fde9de3fb80 1 -- 172.21.15.31:0/79352664 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b7cfa82940 con 0x55b7cfaaee50 2023-10-06T02:27:29.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.525+0000 7fde677fe700 1 -- 172.21.15.31:0/79352664 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(75..75 src has 1..75) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fde8005d170 con 0x55b7cfaaee50 2023-10-06T02:27:29.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.526+0000 7fde677fe700 1 -- 172.21.15.31:0/79352664 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fde80056640 con 0x55b7cfaaee50 2023-10-06T02:27:29.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.526+0000 7fde9de3fb80 10 client.6441 fetch_fsmap learned FSMap version 46 2023-10-06T02:27:29.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.526+0000 7fde9de3fb80 1 -- 172.21.15.31:0/79352664 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) v3 -- 0x55b7cf81afe0 con 0x55b7cfaaee50 2023-10-06T02:27:29.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.526+0000 7fde677fe700 1 -- 172.21.15.31:0/79352664 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b7cfa82940 con 0x55b7cfaaee50 2023-10-06T02:27:29.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.526+0000 7fde677fe700 1 -- 172.21.15.31:0/79352664 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 46) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fde800556c0 con 0x55b7cfaaee50 2023-10-06T02:27:29.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.526+0000 7fde9de3fb80 10 client.6441 fetch_fsmap finished waiting for FSMap version 46 2023-10-06T02:27:29.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.526+0000 7fde9de3fb80 10 client.6441 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T02:27:29.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.526+0000 7fde9de3fb80 20 client.6441 populate_metadata read hostname 'smithi031' 2023-10-06T02:27:29.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.526+0000 7fde9de3fb80 1 --2- 172.21.15.31:0/79352664 >> [v2:172.21.15.31:6835/3003674295,v1:172.21.15.31:6837/3003674295] conn(0x55b7cfad65e0 0x55b7cfad69b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:29.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.526+0000 7fde9de3fb80 4 client.6441 mds_command: new command op to 6282 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T02:27:29.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.526+0000 7fde9de3fb80 1 -- 172.21.15.31:0/79352664 --> [v2:172.21.15.31:6835/3003674295,v1:172.21.15.31:6837/3003674295] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b7cfa5f870 con 0x55b7cfad65e0 2023-10-06T02:27:29.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.526+0000 7fde8d59b700 1 --2- 172.21.15.31:0/79352664 >> [v2:172.21.15.31:6835/3003674295,v1:172.21.15.31:6837/3003674295] conn(0x55b7cfad65e0 0x55b7cfad69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:29.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.527+0000 7fde8d59b700 1 --2- 172.21.15.31:0/79352664 >> [v2:172.21.15.31:6835/3003674295,v1:172.21.15.31:6837/3003674295] conn(0x55b7cfad65e0 0x55b7cfad69b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98c4f63a72f184a8 server_cookie=8ba74969f8eed16f in_seq=0 out_seq=0 2023-10-06T02:27:29.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.527+0000 7fde677fe700 10 client.6441 ms_handle_connect on v2:172.21.15.31:6835/3003674295 2023-10-06T02:27:29.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.527+0000 7fde677fe700 1 -- 172.21.15.31:0/79352664 <== mds.0 v2:172.21.15.31:6835/3003674295 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55b7cfa5f870 con 0x55b7cfad65e0 2023-10-06T02:27:29.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.527+0000 7fde677fe700 10 client.6441 handle_command_reply: tid=0 2023-10-06T02:27:29.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.528+0000 7fde9de3fb80 1 client.6441 shutdown 2023-10-06T02:27:29.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.528+0000 7fde9de3fb80 1 -- 172.21.15.31:0/79352664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7cfaaee50 msgr2=0x55b7cfabc570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:29.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.528+0000 7fde9de3fb80 1 --2- 172.21.15.31:0/79352664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7cfaaee50 0x55b7cfabc570 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fde80055ed0 tx=0x7fde80009e80 comp rx=0 tx=0).stop 2023-10-06T02:27:29.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.528+0000 7fde9de3fb80 1 -- 172.21.15.31:0/79352664 shutdown_connections 2023-10-06T02:27:29.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.528+0000 7fde9de3fb80 1 --2- 172.21.15.31:0/79352664 >> [v2:172.21.15.31:6835/3003674295,v1:172.21.15.31:6837/3003674295] conn(0x55b7cfad65e0 0x55b7cfad69b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:29.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.528+0000 7fde9de3fb80 1 --2- 172.21.15.31:0/79352664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7cfaaee50 0x55b7cfabc570 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:29.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.528+0000 7fde9de3fb80 1 --2- 172.21.15.31:0/79352664 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b7cfa8f2c0 0x55b7cfad4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:29.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.528+0000 7fde677fe700 0 client.6441 ms_handle_reset on v2:172.21.15.31:6835/3003674295 2023-10-06T02:27:29.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.528+0000 7fde9de3fb80 1 --2- 172.21.15.31:0/79352664 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b7cfa90a10 0x55b7cfaad2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:29.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.528+0000 7fde9de3fb80 1 -- 172.21.15.31:0/79352664 >> 172.21.15.31:0/79352664 conn(0x55b7cfa8ee80 msgr2=0x55b7cfaae480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:29.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.528+0000 7fde9de3fb80 1 -- 172.21.15.31:0/79352664 shutdown_connections 2023-10-06T02:27:29.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.528+0000 7fde9de3fb80 1 -- 172.21.15.31:0/79352664 wait complete. 2023-10-06T02:27:29.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.528+0000 7fde9de3fb80 20 client.6441 trim_cache size 0 max 16384 2023-10-06T02:27:29.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.545+0000 7fde9de3fb80 1 Processor -- start 2023-10-06T02:27:29.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.546+0000 7fde9de3fb80 1 -- start start 2023-10-06T02:27:29.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.546+0000 7fde9de3fb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7cfa8f2c0 0x55b7cfa84800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:29.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.546+0000 7fde9de3fb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b7cfa90a10 0x55b7cfa8f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:29.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.546+0000 7fde9de3fb80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b7cfaaee50 0x55b7cfa8fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:29.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.546+0000 7fde9de3fb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7cfabb910 con 0x55b7cfa8f2c0 2023-10-06T02:27:29.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.546+0000 7fde9de3fb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7cfaae2a0 con 0x55b7cfa90a10 2023-10-06T02:27:29.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.546+0000 7fde9de3fb80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b7cfa661e0 con 0x55b7cfaaee50 2023-10-06T02:27:29.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.547+0000 7fde8d59b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7cfa8f2c0 0x55b7cfa84800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:29.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.547+0000 7fde8d59b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7cfa8f2c0 0x55b7cfa84800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55182/0 (socket says 172.21.15.31:55182) 2023-10-06T02:27:29.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.547+0000 7fde8d59b700 1 -- 172.21.15.31:0/1583862765 learned_addr learned my addr 172.21.15.31:0/1583862765 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:29.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.547+0000 7fde8cd9a700 1 --2- 172.21.15.31:0/1583862765 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b7cfa90a10 0x55b7cfa8f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:29.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.547+0000 7fde8dd9c700 1 --2- 172.21.15.31:0/1583862765 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b7cfaaee50 0x55b7cfa8fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:29.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.547+0000 7fde8d59b700 1 -- 172.21.15.31:0/1583862765 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b7cfaaee50 msgr2=0x55b7cfa8fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:29.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.547+0000 7fde8d59b700 1 --2- 172.21.15.31:0/1583862765 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b7cfaaee50 0x55b7cfa8fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:29.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.547+0000 7fde8d59b700 1 -- 172.21.15.31:0/1583862765 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b7cfa90a10 msgr2=0x55b7cfa8f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:29.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.547+0000 7fde8d59b700 1 --2- 172.21.15.31:0/1583862765 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b7cfa90a10 0x55b7cfa8f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:29.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.547+0000 7fde8d59b700 1 -- 172.21.15.31:0/1583862765 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b7cf9ac780 con 0x55b7cfa8f2c0 2023-10-06T02:27:29.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.547+0000 7fde8dd9c700 1 --2- 172.21.15.31:0/1583862765 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b7cfaaee50 0x55b7cfa8fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:27:29.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.547+0000 7fde8d59b700 1 --2- 172.21.15.31:0/1583862765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7cfa8f2c0 0x55b7cfa84800 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fde84000e50 tx=0x7fde84014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a799e70d33bb29dd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:29.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.547+0000 7fde72ffd700 1 -- 172.21.15.31:0/1583862765 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde84021040 con 0x55b7cfa8f2c0 2023-10-06T02:27:29.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.548+0000 7fde72ffd700 1 -- 172.21.15.31:0/1583862765 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde8401ae20 con 0x55b7cfa8f2c0 2023-10-06T02:27:29.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.548+0000 7fde72ffd700 1 -- 172.21.15.31:0/1583862765 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde8402d040 con 0x55b7cfa8f2c0 2023-10-06T02:27:29.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.548+0000 7fde9de3fb80 1 -- 172.21.15.31:0/1583862765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7cfa8f2c0 msgr2=0x55b7cfa84800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:29.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.548+0000 7fde9de3fb80 1 --2- 172.21.15.31:0/1583862765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7cfa8f2c0 0x55b7cfa84800 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fde84000e50 tx=0x7fde84014000 comp rx=0 tx=0).stop 2023-10-06T02:27:29.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.548+0000 7fde9de3fb80 1 -- 172.21.15.31:0/1583862765 shutdown_connections 2023-10-06T02:27:29.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.548+0000 7fde9de3fb80 1 --2- 172.21.15.31:0/1583862765 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b7cfaaee50 0x55b7cfa8fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:29.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.548+0000 7fde9de3fb80 1 --2- 172.21.15.31:0/1583862765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7cfa8f2c0 0x55b7cfa84800 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:29.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.548+0000 7fde9de3fb80 1 --2- 172.21.15.31:0/1583862765 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b7cfa90a10 0x55b7cfa8f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:29.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.548+0000 7fde9de3fb80 1 -- 172.21.15.31:0/1583862765 >> 172.21.15.31:0/1583862765 conn(0x55b7cfa8ee80 msgr2=0x55b7cfaae480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:29.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.548+0000 7fde9de3fb80 1 -- 172.21.15.31:0/1583862765 shutdown_connections 2023-10-06T02:27:29.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.548+0000 7fde9de3fb80 1 -- 172.21.15.31:0/1583862765 wait complete. 2023-10-06T02:27:29.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.549+0000 7fde9de3fb80 1 -- start start 2023-10-06T02:27:29.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.549+0000 7fde9de3fb80 1 Processor -- start 2023-10-06T02:27:29.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.550+0000 7fde9de3fb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7cfa8f2c0 0x55b7cfae67b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:29.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.550+0000 7fde9de3fb80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b7cfa90a10 0x55b7cfb39870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:29.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.550+0000 7fde9de3fb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b7cfaaee50 0x55b7cfaaf320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:29.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.550+0000 7fde9de3fb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7cfaec690 con 0x55b7cfa8f2c0 2023-10-06T02:27:29.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.550+0000 7fde9de3fb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7cfa8e9d0 con 0x55b7cfaaee50 2023-10-06T02:27:29.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.550+0000 7fde9de3fb80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b7cfa8ddc0 con 0x55b7cfa90a10 2023-10-06T02:27:29.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.550+0000 7fde9de3fb80 10 client.0 _ll_register_callbacks cb 0x55b7cfa51020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T02:27:29.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.550+0000 7fde8d59b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7cfa8f2c0 0x55b7cfae67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:29.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.550+0000 7fde8d59b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7cfa8f2c0 0x55b7cfae67b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55188/0 (socket says 172.21.15.31:55188) 2023-10-06T02:27:29.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.550+0000 7fde8d59b700 1 -- 172.21.15.31:0/904901112 learned_addr learned my addr 172.21.15.31:0/904901112 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:29.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.551+0000 7fde8dd9c700 1 --2- 172.21.15.31:0/904901112 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b7cfaaee50 0x55b7cfaaf320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:29.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.551+0000 7fde8cd9a700 1 --2- 172.21.15.31:0/904901112 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b7cfa90a10 0x55b7cfb39870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:29.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.551+0000 7fde8d59b700 1 -- 172.21.15.31:0/904901112 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b7cfa90a10 msgr2=0x55b7cfb39870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:29.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.551+0000 7fde8d59b700 1 --2- 172.21.15.31:0/904901112 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b7cfa90a10 0x55b7cfb39870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:29.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.551+0000 7fde8d59b700 1 -- 172.21.15.31:0/904901112 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b7cfaaee50 msgr2=0x55b7cfaaf320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:29.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.551+0000 7fde8d59b700 1 --2- 172.21.15.31:0/904901112 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b7cfaaee50 0x55b7cfaaf320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:29.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.551+0000 7fde8d59b700 1 -- 172.21.15.31:0/904901112 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b7cf878270 con 0x55b7cfa8f2c0 2023-10-06T02:27:29.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.551+0000 7fde8dd9c700 1 --2- 172.21.15.31:0/904901112 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b7cfaaee50 0x55b7cfaaf320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:27:29.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.551+0000 7fde8cd9a700 1 --2- 172.21.15.31:0/904901112 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b7cfa90a10 0x55b7cfb39870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:27:29.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.551+0000 7fde8d59b700 1 --2- 172.21.15.31:0/904901112 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7cfa8f2c0 0x55b7cfae67b0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fde840094c0 tx=0x7fde8403d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b729bbc94d526a1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:29.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.551+0000 7fde677fe700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T02:27:29.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.551+0000 7fde677fe700 1 -- 172.21.15.31:0/904901112 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde84021050 con 0x55b7cfa8f2c0 2023-10-06T02:27:29.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.551+0000 7fde677fe700 1 -- 172.21.15.31:0/904901112 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(75..75 src has 1..75) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fde84041c20 con 0x55b7cfa8f2c0 2023-10-06T02:27:29.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.551+0000 7fde9de3fb80 1 -- 172.21.15.31:0/904901112 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fde8401a880 con 0x55b7cfa8f2c0 2023-10-06T02:27:29.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.552+0000 7fde677fe700 1 -- 172.21.15.31:0/904901112 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fde8402d040 con 0x55b7cfa8f2c0 2023-10-06T02:27:29.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.552+0000 7fde9de3fb80 10 client.6447 fetch_fsmap learned FSMap version 46 2023-10-06T02:27:29.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.552+0000 7fde9de3fb80 1 -- 172.21.15.31:0/904901112 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) v3 -- 0x55b7cf993470 con 0x55b7cfa8f2c0 2023-10-06T02:27:29.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.552+0000 7fde677fe700 1 -- 172.21.15.31:0/904901112 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde8401a880 con 0x55b7cfa8f2c0 2023-10-06T02:27:29.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.552+0000 7fde677fe700 1 -- 172.21.15.31:0/904901112 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 46) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fde8400b910 con 0x55b7cfa8f2c0 2023-10-06T02:27:29.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.552+0000 7fde9de3fb80 10 client.6447 fetch_fsmap finished waiting for FSMap version 46 2023-10-06T02:27:29.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.552+0000 7fde9de3fb80 10 client.6447 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T02:27:29.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.552+0000 7fde9de3fb80 20 client.6447 populate_metadata read hostname 'smithi031' 2023-10-06T02:27:29.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.552+0000 7fde9de3fb80 1 --2- 172.21.15.31:0/904901112 >> [v2:172.21.15.31:6835/3003674295,v1:172.21.15.31:6837/3003674295] conn(0x55b7cfa89ad0 0x55b7cfa89ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:29.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.552+0000 7fde9de3fb80 4 client.6447 mds_command: new command op to 6282 tid=0[{"prefix": "status"}] 2023-10-06T02:27:29.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.552+0000 7fde9de3fb80 1 -- 172.21.15.31:0/904901112 --> [v2:172.21.15.31:6835/3003674295,v1:172.21.15.31:6837/3003674295] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b7cf980e00 con 0x55b7cfa89ad0 2023-10-06T02:27:29.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.552+0000 7fde8cd9a700 1 --2- 172.21.15.31:0/904901112 >> [v2:172.21.15.31:6835/3003674295,v1:172.21.15.31:6837/3003674295] conn(0x55b7cfa89ad0 0x55b7cfa89ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:29.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.553+0000 7fde677fe700 10 client.6447 ms_handle_connect on v2:172.21.15.31:6835/3003674295 2023-10-06T02:27:29.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.553+0000 7fde8cd9a700 1 --2- 172.21.15.31:0/904901112 >> [v2:172.21.15.31:6835/3003674295,v1:172.21.15.31:6837/3003674295] conn(0x55b7cfa89ad0 0x55b7cfa89ea0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12d86b013150c645 server_cookie=fcef2c9d80fb539c in_seq=0 out_seq=0 2023-10-06T02:27:29.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.553+0000 7fde677fe700 1 -- 172.21.15.31:0/904901112 <== mds.0 v2:172.21.15.31:6835/3003674295 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55b7cf980e00 con 0x55b7cfa89ad0 2023-10-06T02:27:29.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.553+0000 7fde677fe700 10 client.6447 handle_command_reply: tid=0 2023-10-06T02:27:29.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.553+0000 7fde9de3fb80 1 client.6447 shutdown 2023-10-06T02:27:29.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.553+0000 7fde9de3fb80 1 -- 172.21.15.31:0/904901112 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7cfa8f2c0 msgr2=0x55b7cfae67b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:29.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.553+0000 7fde9de3fb80 1 --2- 172.21.15.31:0/904901112 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7cfa8f2c0 0x55b7cfae67b0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fde840094c0 tx=0x7fde8403d000 comp rx=0 tx=0).stop 2023-10-06T02:27:29.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.553+0000 7fde9de3fb80 1 -- 172.21.15.31:0/904901112 shutdown_connections 2023-10-06T02:27:29.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.553+0000 7fde9de3fb80 1 --2- 172.21.15.31:0/904901112 >> [v2:172.21.15.31:6835/3003674295,v1:172.21.15.31:6837/3003674295] conn(0x55b7cfa89ad0 0x55b7cfa89ea0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:29.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.553+0000 7fde9de3fb80 1 --2- 172.21.15.31:0/904901112 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b7cfaaee50 0x55b7cfaaf320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:29.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.553+0000 7fde9de3fb80 1 --2- 172.21.15.31:0/904901112 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7cfa8f2c0 0x55b7cfae67b0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:29.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.553+0000 7fde677fe700 0 client.6447 ms_handle_reset on v2:172.21.15.31:6835/3003674295 2023-10-06T02:27:29.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.553+0000 7fde9de3fb80 1 --2- 172.21.15.31:0/904901112 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b7cfa90a10 0x55b7cfb39870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:29.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.553+0000 7fde9de3fb80 1 -- 172.21.15.31:0/904901112 >> 172.21.15.31:0/904901112 conn(0x55b7cfa8ee80 msgr2=0x55b7cfa84800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:29.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.553+0000 7fde9de3fb80 1 -- 172.21.15.31:0/904901112 shutdown_connections 2023-10-06T02:27:29.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.554+0000 7fde9de3fb80 1 -- 172.21.15.31:0/904901112 wait complete. 2023-10-06T02:27:29.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.554+0000 7fde9de3fb80 20 client.6447 trim_cache size 0 max 16384 2023-10-06T02:27:29.561 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T02:27:29.561 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "d6fcf512-874b-4259-b90a-30ae070f00a9", 2023-10-06T02:27:29.561 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 0, 2023-10-06T02:27:29.562 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 6282, 2023-10-06T02:27:29.562 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2023-10-06T02:27:29.562 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2023-10-06T02:27:29.562 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2023-10-06T02:27:29.562 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 6.5042979399999998, 2023-10-06T02:27:29.562 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 46, 2023-10-06T02:27:29.562 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 75, 2023-10-06T02:27:29.562 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 75, 2023-10-06T02:27:29.562 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 13.720441072 2023-10-06T02:27:29.562 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T02:27:29.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.554+0000 7fde8e59d700 1 -- 172.21.15.31:0/1710609540 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fde6c067290 msgr2=0x7fde6c069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:29.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.554+0000 7fde8e59d700 1 --2- 172.21.15.31:0/1710609540 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fde6c067290 0x7fde6c069740 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fde88069a90 tx=0x7fde84017000 comp rx=0 tx=0).stop 2023-10-06T02:27:29.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.554+0000 7fde8e59d700 1 -- 172.21.15.31:0/1710609540 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde88140e40 msgr2=0x7fde88168400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:29.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.554+0000 7fde8e59d700 1 --2- 172.21.15.31:0/1710609540 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde88140e40 0x7fde88168400 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fde80009ef0 tx=0x7fde8000b040 comp rx=0 tx=0).stop 2023-10-06T02:27:29.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.554+0000 7fde8e59d700 1 -- 172.21.15.31:0/1710609540 shutdown_connections 2023-10-06T02:27:29.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.554+0000 7fde8e59d700 1 --2- 172.21.15.31:0/1710609540 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde88140e40 0x7fde88168400 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:29.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.554+0000 7fde8e59d700 1 --2- 172.21.15.31:0/1710609540 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde880c8330 0x7fde880ae0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:29.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.554+0000 7fde8e59d700 1 --2- 172.21.15.31:0/1710609540 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fde6c067290 0x7fde6c069740 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:29.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.554+0000 7fde8e59d700 1 --2- 172.21.15.31:0/1710609540 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fde88140a70 0x7fde88167ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:29.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.554+0000 7fde8e59d700 1 -- 172.21.15.31:0/1710609540 >> 172.21.15.31:0/1710609540 conn(0x7fde8806d170 msgr2=0x7fde8813fab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:29.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.554+0000 7fde8e59d700 1 -- 172.21.15.31:0/1710609540 shutdown_connections 2023-10-06T02:27:29.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.554+0000 7fde8e59d700 1 -- 172.21.15.31:0/1710609540 wait complete. 2023-10-06T02:27:29.572 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T02:27:29.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.758+0000 7f43e599d700 1 Processor -- start 2023-10-06T02:27:29.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.759+0000 7f43e599d700 1 -- start start 2023-10-06T02:27:29.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.760+0000 7f43e599d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f43e0142c00 0x7f43e006b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:29.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.760+0000 7f43e599d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43e00ca480 con 0x7f43e006bd20 2023-10-06T02:27:29.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.760+0000 7f43e599d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43e015e240 con 0x7f43e00c7f90 2023-10-06T02:27:29.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.760+0000 7f43e599d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43e006b350 con 0x7f43e0142c00 2023-10-06T02:27:29.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.760+0000 7f43e499b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f43e0142c00 0x7f43e006b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:29.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.760+0000 7f43e499b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f43e0142c00 0x7f43e006b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55194/0 (socket says 172.21.15.31:55194) 2023-10-06T02:27:29.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.760+0000 7f43e499b700 1 -- 172.21.15.31:0/264879248 learned_addr learned my addr 172.21.15.31:0/264879248 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:29.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.760+0000 7f43dffff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f43e006bd20 0x7f43e00c7880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:42604/0 (socket says 172.21.15.31:42604) 2023-10-06T02:27:29.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.761+0000 7f43e499b700 1 -- 172.21.15.31:0/264879248 >> v1:172.21.15.62:6789/0 conn(0x7f43e00c7f90 legacy=0x7f43e006c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:29.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.761+0000 7f43e499b700 1 -- 172.21.15.31:0/264879248 >> v1:172.21.15.31:6789/0 conn(0x7f43e006bd20 legacy=0x7f43e00c7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:29.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.761+0000 7f43e499b700 1 -- 172.21.15.31:0/264879248 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43e015df60 con 0x7f43e0142c00 2023-10-06T02:27:29.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.761+0000 7f43e499b700 1 --2- 172.21.15.31:0/264879248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43e0142c00 0x7f43e006b7e0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f43d40099b0 tx=0x7f43d4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5a1868dc88ea89c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:29.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.761+0000 7f43df7fe700 1 -- 172.21.15.31:0/264879248 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d400b860 con 0x7f43e0142c00 2023-10-06T02:27:29.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.761+0000 7f43df7fe700 1 -- 172.21.15.31:0/264879248 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43d400b9c0 con 0x7f43e0142c00 2023-10-06T02:27:29.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.761+0000 7f43df7fe700 1 -- 172.21.15.31:0/264879248 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d400bd80 con 0x7f43e0142c00 2023-10-06T02:27:29.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.762+0000 7f43e599d700 1 -- 172.21.15.31:0/264879248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43e0142c00 msgr2=0x7f43e006b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:29.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.762+0000 7f43e599d700 1 --2- 172.21.15.31:0/264879248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43e0142c00 0x7f43e006b7e0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f43d40099b0 tx=0x7f43d4009250 comp rx=0 tx=0).stop 2023-10-06T02:27:29.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.762+0000 7f43e599d700 1 -- 172.21.15.31:0/264879248 shutdown_connections 2023-10-06T02:27:29.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.762+0000 7f43e599d700 1 --2- 172.21.15.31:0/264879248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43e0142c00 0x7f43e006b7e0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:29.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.762+0000 7f43e599d700 1 -- 172.21.15.31:0/264879248 >> 172.21.15.31:0/264879248 conn(0x7f43e00675a0 msgr2=0x7f43e00679a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:29.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.762+0000 7f43e599d700 1 -- 172.21.15.31:0/264879248 shutdown_connections 2023-10-06T02:27:29.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.762+0000 7f43e599d700 1 -- 172.21.15.31:0/264879248 wait complete. 2023-10-06T02:27:29.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.763+0000 7f43e599d700 1 Processor -- start 2023-10-06T02:27:29.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.763+0000 7f43e599d700 1 -- start start 2023-10-06T02:27:29.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.764+0000 7f43e599d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43e006bd20 0x7f43e013ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:29.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.764+0000 7f43e599d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f43e00c7f90 0x7f43e01590e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:29.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.764+0000 7f43e599d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43e0142c00 0x7f43e016cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:29.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.764+0000 7f43e599d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e00c56d0 con 0x7f43e0142c00 2023-10-06T02:27:29.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.764+0000 7f43e599d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e0159620 con 0x7f43e006bd20 2023-10-06T02:27:29.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.764+0000 7f43e599d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43e016d7e0 con 0x7f43e00c7f90 2023-10-06T02:27:29.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.765+0000 7f43e519c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43e0142c00 0x7f43e016cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:29.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.765+0000 7f43e519c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43e0142c00 0x7f43e016cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55196/0 (socket says 172.21.15.31:55196) 2023-10-06T02:27:29.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.765+0000 7f43e519c700 1 -- 172.21.15.31:0/464199140 learned_addr learned my addr 172.21.15.31:0/464199140 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:29.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.765+0000 7f43dffff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f43e00c7f90 0x7f43e01590e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:29.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.765+0000 7f43e499b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43e006bd20 0x7f43e013ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:29.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.765+0000 7f43e519c700 1 -- 172.21.15.31:0/464199140 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f43e00c7f90 msgr2=0x7f43e01590e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:29.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.765+0000 7f43e519c700 1 --2- 172.21.15.31:0/464199140 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f43e00c7f90 0x7f43e01590e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:29.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.765+0000 7f43e519c700 1 -- 172.21.15.31:0/464199140 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43e006bd20 msgr2=0x7f43e013ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:29.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.765+0000 7f43e519c700 1 --2- 172.21.15.31:0/464199140 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43e006bd20 0x7f43e013ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:29.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.765+0000 7f43e519c700 1 -- 172.21.15.31:0/464199140 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43d40094f0 con 0x7f43e0142c00 2023-10-06T02:27:29.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.765+0000 7f43e519c700 1 --2- 172.21.15.31:0/464199140 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43e0142c00 0x7f43e016cfb0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f43d800c8e0 tx=0x7f43d8001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:29.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.766+0000 7f43dcff9700 1 -- 172.21.15.31:0/464199140 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d8012630 con 0x7f43e0142c00 2023-10-06T02:27:29.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.766+0000 7f43dcff9700 1 -- 172.21.15.31:0/464199140 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43d8012790 con 0x7f43e0142c00 2023-10-06T02:27:29.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.766+0000 7f43dcff9700 1 -- 172.21.15.31:0/464199140 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d8012b50 con 0x7f43e0142c00 2023-10-06T02:27:29.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.766+0000 7f43e599d700 1 -- 172.21.15.31:0/464199140 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43e016dac0 con 0x7f43e0142c00 2023-10-06T02:27:29.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.766+0000 7f43e599d700 1 -- 172.21.15.31:0/464199140 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43e016d4f0 con 0x7f43e0142c00 2023-10-06T02:27:29.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.767+0000 7f43e599d700 1 -- 172.21.15.31:0/464199140 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43e00c5840 con 0x7f43e0142c00 2023-10-06T02:27:29.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.768+0000 7f43dcff9700 1 -- 172.21.15.31:0/464199140 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f43d801a3a0 con 0x7f43e0142c00 2023-10-06T02:27:29.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.768+0000 7f43dcff9700 1 --2- 172.21.15.31:0/464199140 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f43c4067290 0x7f43c4069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:29.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.769+0000 7f43dcff9700 1 -- 172.21.15.31:0/464199140 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f43d8084f20 con 0x7f43e0142c00 2023-10-06T02:27:29.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.769+0000 7f43e499b700 1 --2- 172.21.15.31:0/464199140 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f43c4067290 0x7f43c4069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:29.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.769+0000 7f43e499b700 1 --2- 172.21.15.31:0/464199140 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f43c4067290 0x7f43c4069740 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f43d40099b0 tx=0x7f43d4017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:29.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:29.773+0000 7f43dcff9700 1 -- 172.21.15.31:0/464199140 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f43d8051ae0 con 0x7f43e0142c00 2023-10-06T02:27:30.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:30.018+0000 7f43e599d700 1 -- 172.21.15.31:0/464199140 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f43e004c3c0 con 0x7f43e0142c00 2023-10-06T02:27:30.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:30.018+0000 7f43dcff9700 1 -- 172.21.15.31:0/464199140 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v46) v1 ==== 53+0+73 (secure 0 0 0) 0x7f43d8055100 con 0x7f43e0142c00 2023-10-06T02:27:30.020 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T02:27:30.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:30.020+0000 7f43e599d700 1 -- 172.21.15.31:0/464199140 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f43c4067290 msgr2=0x7f43c4069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:30.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:30.020+0000 7f43e599d700 1 --2- 172.21.15.31:0/464199140 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f43c4067290 0x7f43c4069740 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f43d40099b0 tx=0x7f43d4017000 comp rx=0 tx=0).stop 2023-10-06T02:27:30.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:30.020+0000 7f43e599d700 1 -- 172.21.15.31:0/464199140 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43e0142c00 msgr2=0x7f43e016cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:30.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:30.020+0000 7f43e599d700 1 --2- 172.21.15.31:0/464199140 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43e0142c00 0x7f43e016cfb0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f43d800c8e0 tx=0x7f43d8001710 comp rx=0 tx=0).stop 2023-10-06T02:27:30.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:30.021+0000 7f43e599d700 1 -- 172.21.15.31:0/464199140 shutdown_connections 2023-10-06T02:27:30.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:30.021+0000 7f43e599d700 1 --2- 172.21.15.31:0/464199140 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43e0142c00 0x7f43e016cfb0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:30.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:30.021+0000 7f43e599d700 1 --2- 172.21.15.31:0/464199140 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43e006bd20 0x7f43e013ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:30.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:30.021+0000 7f43e599d700 1 --2- 172.21.15.31:0/464199140 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f43c4067290 0x7f43c4069740 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:30.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:30.021+0000 7f43e599d700 1 --2- 172.21.15.31:0/464199140 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f43e00c7f90 0x7f43e01590e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:30.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:30.021+0000 7f43e599d700 1 -- 172.21.15.31:0/464199140 >> 172.21.15.31:0/464199140 conn(0x7f43e00675a0 msgr2=0x7f43e0140f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:30.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:30.021+0000 7f43e599d700 1 -- 172.21.15.31:0/464199140 shutdown_connections 2023-10-06T02:27:30.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:30.021+0000 7f43e599d700 1 -- 172.21.15.31:0/464199140 wait complete. 2023-10-06T02:27:30.039 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T02:27:30.040 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T02:27:30.040 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi031.front.sepia.ceph.com 2023-10-06T02:27:30.040 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T02:27:30.040 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T02:27:30.040 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T02:27:30.040 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T02:27:30.040 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T02:27:30.040 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T02:27:30.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:27:30.041 DEBUG:teuthology.orchestra.run.smithi031:> ip addr 2023-10-06T02:27:30.079 INFO:teuthology.orchestra.run.smithi031.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T02:27:30.079 INFO:teuthology.orchestra.run.smithi031.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T02:27:30.079 INFO:teuthology.orchestra.run.smithi031.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T02:27:30.079 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:27:30.079 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 ::1/128 scope host 2023-10-06T02:27:30.079 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:27:30.079 INFO:teuthology.orchestra.run.smithi031.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T02:27:30.080 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cc brd ff:ff:ff:ff:ff:ff 2023-10-06T02:27:30.080 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f0 2023-10-06T02:27:30.080 INFO:teuthology.orchestra.run.smithi031.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T02:27:30.080 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cd brd ff:ff:ff:ff:ff:ff 2023-10-06T02:27:30.080 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f1 2023-10-06T02:27:30.080 INFO:teuthology.orchestra.run.smithi031.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T02:27:30.080 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:ba brd ff:ff:ff:ff:ff:ff 2023-10-06T02:27:30.080 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f0 2023-10-06T02:27:30.080 INFO:teuthology.orchestra.run.smithi031.stdout: inet 172.21.15.31/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T02:27:30.080 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft 42006sec preferred_lft 42006sec 2023-10-06T02:27:30.080 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::ae1f:6bff:fea5:caba/64 scope link 2023-10-06T02:27:30.080 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:27:30.080 INFO:teuthology.orchestra.run.smithi031.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T02:27:30.081 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:bb brd ff:ff:ff:ff:ff:ff 2023-10-06T02:27:30.081 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f1 2023-10-06T02:27:30.081 INFO:teuthology.orchestra.run.smithi031.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T02:27:30.081 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 16:d7:88:2d:d0:c2 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:27:30.081 INFO:teuthology.orchestra.run.smithi031.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T02:27:30.081 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:27:30.081 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::408:52ff:fe3c:ba8f/64 scope link 2023-10-06T02:27:30.081 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:27:30.081 INFO:teuthology.orchestra.run.smithi031.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T02:27:30.081 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 16:d7:88:2d:d0:c2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T02:27:30.081 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::14d7:88ff:fe2d:d0c2/64 scope link 2023-10-06T02:27:30.081 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:27:30.081 INFO:teuthology.orchestra.run.smithi031.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T02:27:30.082 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 82:16:f1:00:cf:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T02:27:30.082 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::8016:f1ff:fe00:cf2f/64 scope link 2023-10-06T02:27:30.082 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:27:30.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:27:30.083 DEBUG:teuthology.orchestra.run.smithi031:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T02:27:30.183 INFO:teuthology.orchestra.run.smithi031.stdout:1 2023-10-06T02:27:30.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:27:30.186 DEBUG:teuthology.orchestra.run.smithi031:> route 2023-10-06T02:27:30.225 INFO:teuthology.orchestra.run.smithi031.stdout:Kernel IP routing table 2023-10-06T02:27:30.225 INFO:teuthology.orchestra.run.smithi031.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T02:27:30.225 INFO:teuthology.orchestra.run.smithi031.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T02:27:30.225 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T02:27:30.225 INFO:teuthology.orchestra.run.smithi031.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T02:27:30.226 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:27:30.227 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2023-10-06T02:27:30.227 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2023-10-06T02:27:30.227 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T02:27:30.227 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T02:27:30.227 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T02:27:30.227 DEBUG:teuthology.orchestra.run.smithi031:> ') 2023-10-06T02:27:30.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:27:30.425 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2023-10-06T02:27:30.462 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T02:27:30.462 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T02:27:30.462 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T02:27:30.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:27:30.463 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set brx.0 up 2023-10-06T02:27:30.564 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T02:27:30.564 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T02:27:30.601 INFO:teuthology.orchestra.run.smithi031.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T02:27:30.601 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T02:27:30.601 DEBUG:teuthology.orchestra.run.smithi031:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T02:27:30.677 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-06T02:27:30.677 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T02:27:30.677 DEBUG:teuthology.orchestra.run.smithi031:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T02:27:30.809 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T02:27:30.809 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T02:27:30.837 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T02:27:30.837 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T02:27:30.838 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.62:3300,172.21.15.62:3301" 2023-10-06T02:27:30.838 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: trying mount with new device syntax: 0@d6fcf512-874b-4259-b90a-30ae070f00a9.cephfs=/ 2023-10-06T02:27:30.838 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.31:3300/172.21.15.62:3300/172.21.15.62:3301" will pass to kernel 2023-10-06T02:27:30.843 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T02:27:30.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:27:30.843 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T02:27:30.906 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T02:27:31.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:31.094+0000 7fcf96b9d700 1 Processor -- start 2023-10-06T02:27:31.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:31.095+0000 7fcf96b9d700 1 -- start start 2023-10-06T02:27:31.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:31.096+0000 7fcf96b9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fcf9013fb20 0x7fcf9015a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:31.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:31.096+0000 7fcf96b9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcf9006aaa0 0x7fcf900c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:31.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:31.096+0000 7fcf96b9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf9015ae70 con 0x7fcf9013fef0 2023-10-06T02:27:31.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:31.096+0000 7fcf96b9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf9006af60 con 0x7fcf9013fb20 2023-10-06T02:27:31.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:31.096+0000 7fcf96b9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf9006b240 con 0x7fcf9006aaa0 2023-10-06T02:27:31.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:31.096+0000 7fcf95b9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fcf9013fb20 0x7fcf9015a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:31.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:31.096+0000 7fcf95b9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fcf9013fb20 0x7fcf9015a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55200/0 (socket says 172.21.15.31:55200) 2023-10-06T02:27:31.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:31.096+0000 7fcf95b9b700 1 -- 172.21.15.31:0/2704752267 learned_addr learned my addr 172.21.15.31:0/2704752267 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:31.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:31.096+0000 7fcf95b9b700 1 -- 172.21.15.31:0/2704752267 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcf9006aaa0 msgr2=0x7fcf900c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:31.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:31.096+0000 7fcf95b9b700 1 --2- 172.21.15.31:0/2704752267 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcf9006aaa0 0x7fcf900c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:31.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:31.096+0000 7fcf95b9b700 1 -- 172.21.15.31:0/2704752267 >> v1:172.21.15.62:6789/0 conn(0x7fcf9013fef0 legacy=0x7fcf900c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:31.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:31.096+0000 7fcf95b9b700 1 -- 172.21.15.31:0/2704752267 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf9006b520 con 0x7fcf9013fb20 2023-10-06T02:27:31.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:31.097+0000 7fcf95b9b700 1 --2- 172.21.15.31:0/2704752267 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf9013fb20 0x7fcf9015a930 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c0099b0 tx=0x7fcf8c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31991ecdb106c613 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:31.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:31.097+0000 7fcf94b99700 1 -- 172.21.15.31:0/2704752267 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf8c00b820 con 0x7fcf9013fb20 2023-10-06T02:27:31.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:31.097+0000 7fcf94b99700 1 -- 172.21.15.31:0/2704752267 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf8c00b980 con 0x7fcf9013fb20 2023-10-06T02:27:31.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:31.097+0000 7fcf94b99700 1 -- 172.21.15.31:0/2704752267 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf8c00bd20 con 0x7fcf9013fb20 2023-10-06T02:27:31.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:31.097+0000 7fcf96b9d700 1 -- 172.21.15.31:0/2704752267 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf9013fb20 msgr2=0x7fcf9015a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:31.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:31.097+0000 7fcf96b9d700 1 --2- 172.21.15.31:0/2704752267 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf9013fb20 0x7fcf9015a930 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c0099b0 tx=0x7fcf8c009250 comp rx=0 tx=0).stop 2023-10-06T02:27:31.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:31.098+0000 7fcf96b9d700 1 -- 172.21.15.31:0/2704752267 shutdown_connections 2023-10-06T02:27:31.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:31.098+0000 7fcf96b9d700 1 --2- 172.21.15.31:0/2704752267 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcf9006aaa0 0x7fcf900c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:31.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:31.098+0000 7fcf96b9d700 1 --2- 172.21.15.31:0/2704752267 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf9013fb20 0x7fcf9015a930 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:31.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:31.098+0000 7fcf96b9d700 1 -- 172.21.15.31:0/2704752267 >> 172.21.15.31:0/2704752267 conn(0x7fcf9006d170 msgr2=0x7fcf900cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:31.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:31.098+0000 7fcf96b9d700 1 -- 172.21.15.31:0/2704752267 shutdown_connections 2023-10-06T02:27:31.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:31.098+0000 7fcf96b9d700 1 -- 172.21.15.31:0/2704752267 wait complete. 2023-10-06T02:27:31.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:31.099+0000 7fcf96b9d700 1 Processor -- start 2023-10-06T02:27:31.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:31.099+0000 7fcf96b9d700 1 -- start start 2023-10-06T02:27:31.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:31.100+0000 7fcf96b9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcf9006aaa0 0x7fcf9013edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:31.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:31.100+0000 7fcf96b9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcf9013fb20 0x7fcf90158400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:31.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:31.100+0000 7fcf96b9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf9013fef0 0x7fcf90136f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:31.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:31.100+0000 7fcf96b9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf900af970 con 0x7fcf9013fef0 2023-10-06T02:27:31.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:31.100+0000 7fcf96b9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf90158de0 con 0x7fcf9013fb20 2023-10-06T02:27:31.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:31.100+0000 7fcf96b9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf900af040 con 0x7fcf9006aaa0 2023-10-06T02:27:31.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:31.100+0000 7fcf9639c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf9013fef0 0x7fcf90136f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:31.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:31.100+0000 7fcf9639c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf9013fef0 0x7fcf90136f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55212/0 (socket says 172.21.15.31:55212) 2023-10-06T02:27:31.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:31.100+0000 7fcf9639c700 1 -- 172.21.15.31:0/4251844501 learned_addr learned my addr 172.21.15.31:0/4251844501 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:31.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:31.101+0000 7fcf95b9b700 1 --2- 172.21.15.31:0/4251844501 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcf9006aaa0 0x7fcf9013edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:31.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:31.101+0000 7fcf9539a700 1 --2- 172.21.15.31:0/4251844501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcf9013fb20 0x7fcf90158400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:31.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:31.101+0000 7fcf9639c700 1 -- 172.21.15.31:0/4251844501 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcf9006aaa0 msgr2=0x7fcf9013edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:31.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:31.101+0000 7fcf9639c700 1 --2- 172.21.15.31:0/4251844501 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcf9006aaa0 0x7fcf9013edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:31.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:31.101+0000 7fcf9639c700 1 -- 172.21.15.31:0/4251844501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcf9013fb20 msgr2=0x7fcf90158400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:31.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:31.101+0000 7fcf9639c700 1 --2- 172.21.15.31:0/4251844501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcf9013fb20 0x7fcf90158400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:31.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:31.101+0000 7fcf9639c700 1 -- 172.21.15.31:0/4251844501 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf8c0094f0 con 0x7fcf9013fef0 2023-10-06T02:27:31.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:31.101+0000 7fcf9639c700 1 --2- 172.21.15.31:0/4251844501 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf9013fef0 0x7fcf90136f40 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fcf8800bb60 tx=0x7fcf88009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:31.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:31.101+0000 7fcf85ffb700 1 -- 172.21.15.31:0/4251844501 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf8800c940 con 0x7fcf9013fef0 2023-10-06T02:27:31.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:31.101+0000 7fcf85ffb700 1 -- 172.21.15.31:0/4251844501 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf8800cac0 con 0x7fcf9013fef0 2023-10-06T02:27:31.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:31.101+0000 7fcf96b9d700 1 -- 172.21.15.31:0/4251844501 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf900af320 con 0x7fcf9013fef0 2023-10-06T02:27:31.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:31.101+0000 7fcf85ffb700 1 -- 172.21.15.31:0/4251844501 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf8800ce80 con 0x7fcf9013fef0 2023-10-06T02:27:31.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:31.101+0000 7fcf96b9d700 1 -- 172.21.15.31:0/4251844501 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf9013f300 con 0x7fcf9013fef0 2023-10-06T02:27:31.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:31.102+0000 7fcf96b9d700 1 -- 172.21.15.31:0/4251844501 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf900648e0 con 0x7fcf9013fef0 2023-10-06T02:27:31.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:31.103+0000 7fcf85ffb700 1 -- 172.21.15.31:0/4251844501 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fcf88014620 con 0x7fcf9013fef0 2023-10-06T02:27:31.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:31.104+0000 7fcf85ffb700 1 --2- 172.21.15.31:0/4251844501 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fcf74067290 0x7fcf74069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:31.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:31.104+0000 7fcf85ffb700 1 -- 172.21.15.31:0/4251844501 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fcf88084cb0 con 0x7fcf9013fef0 2023-10-06T02:27:31.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:31.104+0000 7fcf95b9b700 1 --2- 172.21.15.31:0/4251844501 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fcf74067290 0x7fcf74069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:31.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:31.105+0000 7fcf95b9b700 1 --2- 172.21.15.31:0/4251844501 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fcf74067290 0x7fcf74069740 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c0099b0 tx=0x7fcf8c016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:31.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:31.108+0000 7fcf85ffb700 1 -- 172.21.15.31:0/4251844501 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcf8804c5a0 con 0x7fcf9013fef0 2023-10-06T02:27:31.300 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T02:27:31.304 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T02:27:31.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:31.360+0000 7fcf96b9d700 1 -- 172.21.15.31:0/4251844501 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fcf9004c3c0 con 0x7fcf9013fef0 2023-10-06T02:27:31.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:31.361+0000 7fcf85ffb700 1 -- 172.21.15.31:0/4251844501 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v46) v1 ==== 53+0+73 (secure 0 0 0) 0x7fcf900648e0 con 0x7fcf9013fef0 2023-10-06T02:27:31.362 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T02:27:31.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:31.363+0000 7fcf96b9d700 1 -- 172.21.15.31:0/4251844501 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fcf74067290 msgr2=0x7fcf74069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:31.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:31.363+0000 7fcf96b9d700 1 --2- 172.21.15.31:0/4251844501 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fcf74067290 0x7fcf74069740 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c0099b0 tx=0x7fcf8c016000 comp rx=0 tx=0).stop 2023-10-06T02:27:31.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:31.363+0000 7fcf96b9d700 1 -- 172.21.15.31:0/4251844501 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf9013fef0 msgr2=0x7fcf90136f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:31.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:31.363+0000 7fcf96b9d700 1 --2- 172.21.15.31:0/4251844501 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf9013fef0 0x7fcf90136f40 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fcf8800bb60 tx=0x7fcf88009810 comp rx=0 tx=0).stop 2023-10-06T02:27:31.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:31.363+0000 7fcf96b9d700 1 -- 172.21.15.31:0/4251844501 shutdown_connections 2023-10-06T02:27:31.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:31.363+0000 7fcf96b9d700 1 --2- 172.21.15.31:0/4251844501 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf9013fef0 0x7fcf90136f40 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:31.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:31.363+0000 7fcf96b9d700 1 --2- 172.21.15.31:0/4251844501 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fcf74067290 0x7fcf74069740 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:31.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:31.363+0000 7fcf96b9d700 1 --2- 172.21.15.31:0/4251844501 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcf9006aaa0 0x7fcf9013edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:31.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:31.363+0000 7fcf96b9d700 1 --2- 172.21.15.31:0/4251844501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcf9013fb20 0x7fcf90158400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:31.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:31.363+0000 7fcf96b9d700 1 -- 172.21.15.31:0/4251844501 >> 172.21.15.31:0/4251844501 conn(0x7fcf9006d170 msgr2=0x7fcf9006de50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:31.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:31.363+0000 7fcf96b9d700 1 -- 172.21.15.31:0/4251844501 shutdown_connections 2023-10-06T02:27:31.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:31.363+0000 7fcf96b9d700 1 -- 172.21.15.31:0/4251844501 wait complete. 2023-10-06T02:27:31.382 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T02:27:31.382 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T02:27:31.382 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi062.front.sepia.ceph.com 2023-10-06T02:27:31.382 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-06T02:27:31.383 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-06T02:27:31.383 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T02:27:31.383 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T02:27:31.383 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T02:27:31.383 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T02:27:31.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:27:31.383 DEBUG:teuthology.orchestra.run.smithi062:> ip addr 2023-10-06T02:27:31.422 INFO:teuthology.orchestra.run.smithi062.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T02:27:31.422 INFO:teuthology.orchestra.run.smithi062.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T02:27:31.422 INFO:teuthology.orchestra.run.smithi062.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T02:27:31.422 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:27:31.422 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 ::1/128 scope host 2023-10-06T02:27:31.423 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:27:31.423 INFO:teuthology.orchestra.run.smithi062.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T02:27:31.423 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:14 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:27:31.423 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f0 2023-10-06T02:27:31.423 INFO:teuthology.orchestra.run.smithi062.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T02:27:31.423 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:15 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:27:31.423 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f1 2023-10-06T02:27:31.423 INFO:teuthology.orchestra.run.smithi062.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T02:27:31.423 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c2 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:27:31.423 INFO:teuthology.orchestra.run.smithi062.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T02:27:31.423 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c3 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:27:31.424 INFO:teuthology.orchestra.run.smithi062.stdout: inet 172.21.15.62/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T02:27:31.424 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft 41998sec preferred_lft 41998sec 2023-10-06T02:27:31.424 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::ec4:7aff:febb:d1c3/64 scope link 2023-10-06T02:27:31.424 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:27:31.424 INFO:teuthology.orchestra.run.smithi062.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T02:27:31.424 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether ae:42:a8:dd:f7:9b brd ff:ff:ff:ff:ff:ff 2023-10-06T02:27:31.424 INFO:teuthology.orchestra.run.smithi062.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T02:27:31.424 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:27:31.424 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::1098:a4ff:fe0d:389/64 scope link 2023-10-06T02:27:31.424 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:27:31.424 INFO:teuthology.orchestra.run.smithi062.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T02:27:31.425 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether fe:60:2b:8e:af:28 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T02:27:31.425 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::fc60:2bff:fe8e:af28/64 scope link 2023-10-06T02:27:31.425 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:27:31.425 INFO:teuthology.orchestra.run.smithi062.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T02:27:31.425 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether ae:42:a8:dd:f7:9b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T02:27:31.425 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::ac42:a8ff:fedd:f79b/64 scope link 2023-10-06T02:27:31.425 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:27:31.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:27:31.425 DEBUG:teuthology.orchestra.run.smithi062:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T02:27:31.548 INFO:teuthology.orchestra.run.smithi062.stdout:1 2023-10-06T02:27:31.551 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:27:31.551 DEBUG:teuthology.orchestra.run.smithi062:> route 2023-10-06T02:27:31.590 INFO:teuthology.orchestra.run.smithi062.stdout:Kernel IP routing table 2023-10-06T02:27:31.590 INFO:teuthology.orchestra.run.smithi062.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T02:27:31.590 INFO:teuthology.orchestra.run.smithi062.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T02:27:31.590 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T02:27:31.590 INFO:teuthology.orchestra.run.smithi062.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T02:27:31.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:27:31.591 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2023-10-06T02:27:31.591 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2023-10-06T02:27:31.591 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T02:27:31.591 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T02:27:31.591 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T02:27:31.591 DEBUG:teuthology.orchestra.run.smithi062:> ') 2023-10-06T02:27:31.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:27:31.789 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2023-10-06T02:27:31.825 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-06T02:27:31.825 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T02:27:31.825 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T02:27:31.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:27:31.826 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set brx.0 up 2023-10-06T02:27:31.928 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T02:27:31.928 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-06T02:27:31.964 INFO:teuthology.orchestra.run.smithi062.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-06T02:27:31.964 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T02:27:31.964 DEBUG:teuthology.orchestra.run.smithi062:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-06T02:27:32.039 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-06T02:27:32.039 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T02:27:32.040 DEBUG:teuthology.orchestra.run.smithi062:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T02:27:32.171 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T02:27:32.171 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T02:27:32.200 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T02:27:32.200 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T02:27:32.200 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.62:3300,172.21.15.62:3301" 2023-10-06T02:27:32.200 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: trying mount with new device syntax: 1@d6fcf512-874b-4259-b90a-30ae070f00a9.cephfs=/ 2023-10-06T02:27:32.200 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.31:3300/172.21.15.62:3300/172.21.15.62:3301" will pass to kernel 2023-10-06T02:27:32.205 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T02:27:32.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:27:32.205 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-06T02:27:32.268 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-10-06T02:27:32.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:32.468+0000 7f8768f9d700 1 Processor -- start 2023-10-06T02:27:32.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:32.468+0000 7f8768f9d700 1 -- start start 2023-10-06T02:27:32.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:32.470+0000 7f8768f9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f87640c85d0 0x7f87640cc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:32.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:32.470+0000 7f8768f9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87641571b0 con 0x7f87640c8200 2023-10-06T02:27:32.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:32.470+0000 7f8768f9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87641579b0 con 0x7f87640c2a90 2023-10-06T02:27:32.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:32.470+0000 7f8768f9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87640c1390 con 0x7f87640c85d0 2023-10-06T02:27:32.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:32.470+0000 7f87637fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f87640c8200 0x7f87640c1820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:42612/0 (socket says 172.21.15.31:42612) 2023-10-06T02:27:32.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:32.470+0000 7f87637fe700 1 -- 172.21.15.31:0/2622379077 learned_addr learned my addr 172.21.15.31:0/2622379077 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:27:32.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:32.470+0000 7f8763fff700 1 --2- 172.21.15.31:0/2622379077 >> v2:172.21.15.62:3300/0 conn(0x7f87640c85d0 0x7f87640cc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:32.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:32.470+0000 7f87627fc700 1 -- 172.21.15.31:0/2622379077 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 84611383 0 0) 0x7f8754001110 con 0x7f87640c8200 2023-10-06T02:27:32.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:32.470+0000 7f87627fc700 1 -- 172.21.15.31:0/2622379077 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f874c003570 con 0x7f87640c8200 2023-10-06T02:27:32.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:32.471+0000 7f87627fc700 1 -- 172.21.15.31:0/2622379077 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3973106061 0 0) 0x7f874c003570 con 0x7f87640c8200 2023-10-06T02:27:32.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:32.471+0000 7f87627fc700 1 -- 172.21.15.31:0/2622379077 >> v2:172.21.15.62:3300/0 conn(0x7f87640c85d0 msgr2=0x7f87640cc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:32.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:32.471+0000 7f87627fc700 1 --2- 172.21.15.31:0/2622379077 >> v2:172.21.15.62:3300/0 conn(0x7f87640c85d0 0x7f87640cc2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:32.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:32.471+0000 7f87627fc700 1 -- 172.21.15.31:0/2622379077 >> v1:172.21.15.62:6789/0 conn(0x7f87640c2a90 legacy=0x7f8764156aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:32.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:32.471+0000 7f87627fc700 1 -- 172.21.15.31:0/2622379077 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87641573f0 con 0x7f87640c8200 2023-10-06T02:27:32.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:32.471+0000 7f87627fc700 1 -- 172.21.15.31:0/2622379077 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7f8754001600 con 0x7f87640c8200 2023-10-06T02:27:32.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:32.471+0000 7f8763fff700 1 --2- 172.21.15.31:0/2622379077 >> v2:172.21.15.62:3300/0 conn(0x7f87640c85d0 0x7f87640cc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:27:32.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:32.471+0000 7f87627fc700 1 -- 172.21.15.31:0/2622379077 >> v1:172.21.15.31:6789/0 conn(0x7f87640c8200 legacy=0x7f87640c1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:32.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:32.471+0000 7f87627fc700 1 --2- 172.21.15.31:0/2622379077 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f874c003ab0 0x7f874c003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:32.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:32.471+0000 7f87627fc700 1 --2- 172.21.15.31:0/2622379077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f874c004810 0x7f874c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:32.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:32.471+0000 7f87627fc700 1 --2- 172.21.15.31:0/2622379077 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f874c005210 0x7f874c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:32.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:32.471+0000 7f87627fc700 1 -- 172.21.15.31:0/2622379077 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87640c1390 con 0x7f874c004810 2023-10-06T02:27:32.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:32.471+0000 7f87627fc700 1 -- 172.21.15.31:0/2622379077 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f874c00c1b0 con 0x7f874c003ab0 2023-10-06T02:27:32.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:32.471+0000 7f87627fc700 1 -- 172.21.15.31:0/2622379077 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f874c00c370 con 0x7f874c005210 2023-10-06T02:27:32.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:32.471+0000 7f8762ffd700 1 --2- 172.21.15.31:0/2622379077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f874c004810 0x7f874c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:32.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:32.471+0000 7f87637fe700 1 --2- 172.21.15.31:0/2622379077 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f874c003ab0 0x7f874c003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:32.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:32.471+0000 7f8763fff700 1 --2- 172.21.15.31:0/2622379077 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f874c005210 0x7f874c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:32.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:32.472+0000 7f8762ffd700 1 -- 172.21.15.31:0/2622379077 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f874c005210 msgr2=0x7f874c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:32.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:32.472+0000 7f8762ffd700 1 --2- 172.21.15.31:0/2622379077 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f874c005210 0x7f874c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:32.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:32.472+0000 7f8762ffd700 1 -- 172.21.15.31:0/2622379077 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f874c003ab0 msgr2=0x7f874c003ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:32.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:32.472+0000 7f8762ffd700 1 --2- 172.21.15.31:0/2622379077 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f874c003ab0 0x7f874c003ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:32.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:32.472+0000 7f8762ffd700 1 -- 172.21.15.31:0/2622379077 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f874c003570 con 0x7f874c004810 2023-10-06T02:27:32.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:32.472+0000 7f8763fff700 1 -- 172.21.15.31:0/2622379077 reap_dead start 2023-10-06T02:27:32.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:32.472+0000 7f8762ffd700 1 --2- 172.21.15.31:0/2622379077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f874c004810 0x7f874c004c80 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f8758001880 tx=0x7f8758007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43cb48ec9f162b0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:32.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:32.472+0000 7f87627fc700 1 -- 172.21.15.31:0/2622379077 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8758012420 con 0x7f874c004810 2023-10-06T02:27:32.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:32.472+0000 7f87627fc700 1 -- 172.21.15.31:0/2622379077 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87580125a0 con 0x7f874c004810 2023-10-06T02:27:32.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:32.472+0000 7f87627fc700 1 -- 172.21.15.31:0/2622379077 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8758012960 con 0x7f874c004810 2023-10-06T02:27:32.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:32.473+0000 7f8768f9d700 1 -- 172.21.15.31:0/2622379077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f874c004810 msgr2=0x7f874c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:32.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:32.473+0000 7f8768f9d700 1 --2- 172.21.15.31:0/2622379077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f874c004810 0x7f874c004c80 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f8758001880 tx=0x7f8758007d30 comp rx=0 tx=0).stop 2023-10-06T02:27:32.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:32.473+0000 7f8768f9d700 1 -- 172.21.15.31:0/2622379077 shutdown_connections 2023-10-06T02:27:32.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:32.473+0000 7f8768f9d700 1 --2- 172.21.15.31:0/2622379077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f874c004810 0x7f874c004c80 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:32.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:32.473+0000 7f8768f9d700 1 -- 172.21.15.31:0/2622379077 >> 172.21.15.31:0/2622379077 conn(0x7f876406c6a0 msgr2=0x7f876406caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:32.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:32.473+0000 7f8768f9d700 1 -- 172.21.15.31:0/2622379077 shutdown_connections 2023-10-06T02:27:32.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:32.473+0000 7f8768f9d700 1 -- 172.21.15.31:0/2622379077 wait complete. 2023-10-06T02:27:32.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:32.474+0000 7f8768f9d700 1 Processor -- start 2023-10-06T02:27:32.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:32.474+0000 7f8768f9d700 1 -- start start 2023-10-06T02:27:32.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:32.475+0000 7f8768f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f874c004810 0x7f876413b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:32.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:32.475+0000 7f8768f9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87640cb060 0x7f87640cb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:32.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:32.475+0000 7f8768f9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87640c5b50 0x7f87640c5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:32.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:32.475+0000 7f8768f9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87640c7240 con 0x7f87640cb060 2023-10-06T02:27:32.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:32.475+0000 7f8768f9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87640cba10 con 0x7f87640c5b50 2023-10-06T02:27:32.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:32.475+0000 7f8768f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87640c69a0 con 0x7f874c004810 2023-10-06T02:27:32.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:32.475+0000 7f8762ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87640cb060 0x7f87640cb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:32.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:32.476+0000 7f8762ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87640cb060 0x7f87640cb4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55230/0 (socket says 172.21.15.31:55230) 2023-10-06T02:27:32.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:32.476+0000 7f8762ffd700 1 -- 172.21.15.31:0/1847838639 learned_addr learned my addr 172.21.15.31:0/1847838639 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:32.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:32.476+0000 7f87637fe700 1 --2- 172.21.15.31:0/1847838639 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f874c004810 0x7f876413b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:32.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:32.476+0000 7f8763fff700 1 --2- 172.21.15.31:0/1847838639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87640c5b50 0x7f87640c5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:32.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:32.476+0000 7f8762ffd700 1 -- 172.21.15.31:0/1847838639 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f874c004810 msgr2=0x7f876413b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:32.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:32.476+0000 7f8762ffd700 1 --2- 172.21.15.31:0/1847838639 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f874c004810 0x7f876413b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:32.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:32.476+0000 7f8762ffd700 1 -- 172.21.15.31:0/1847838639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87640c5b50 msgr2=0x7f87640c5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:32.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:32.476+0000 7f8762ffd700 1 --2- 172.21.15.31:0/1847838639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87640c5b50 0x7f87640c5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:32.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:32.476+0000 7f8762ffd700 1 -- 172.21.15.31:0/1847838639 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87640c6c80 con 0x7f87640cb060 2023-10-06T02:27:32.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:32.476+0000 7f8763fff700 1 --2- 172.21.15.31:0/1847838639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87640c5b50 0x7f87640c5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:27:32.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:32.476+0000 7f8762ffd700 1 --2- 172.21.15.31:0/1847838639 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87640cb060 0x7f87640cb4d0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f874c00c180 tx=0x7f8758015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:32.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:32.476+0000 7f874b7fe700 1 -- 172.21.15.31:0/1847838639 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8758001200 con 0x7f87640cb060 2023-10-06T02:27:32.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:32.476+0000 7f874b7fe700 1 -- 172.21.15.31:0/1847838639 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8758012cd0 con 0x7f87640cb060 2023-10-06T02:27:32.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:32.476+0000 7f874b7fe700 1 -- 172.21.15.31:0/1847838639 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8758012960 con 0x7f87640cb060 2023-10-06T02:27:32.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:32.476+0000 7f8768f9d700 1 -- 172.21.15.31:0/1847838639 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87640c6f60 con 0x7f87640cb060 2023-10-06T02:27:32.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:32.477+0000 7f8768f9d700 1 -- 172.21.15.31:0/1847838639 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f876413bc50 con 0x7f87640cb060 2023-10-06T02:27:32.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:32.478+0000 7f8768f9d700 1 -- 172.21.15.31:0/1847838639 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87640abdf0 con 0x7f87640cb060 2023-10-06T02:27:32.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:32.479+0000 7f874b7fe700 1 -- 172.21.15.31:0/1847838639 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f8758019030 con 0x7f87640cb060 2023-10-06T02:27:32.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:32.480+0000 7f874b7fe700 1 --2- 172.21.15.31:0/1847838639 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f8750067290 0x7f8750069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:32.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:32.480+0000 7f874b7fe700 1 -- 172.21.15.31:0/1847838639 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f87580861e0 con 0x7f87640cb060 2023-10-06T02:27:32.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:32.480+0000 7f87637fe700 1 --2- 172.21.15.31:0/1847838639 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f8750067290 0x7f8750069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:32.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:32.481+0000 7f87637fe700 1 --2- 172.21.15.31:0/1847838639 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f8750067290 0x7f8750069740 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f8764066170 tx=0x7f875400a0f0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:32.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:32.484+0000 7f874b7fe700 1 -- 172.21.15.31:0/1847838639 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f87640abdf0 con 0x7f87640cb060 2023-10-06T02:27:32.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:32.731+0000 7f8768f9d700 1 -- 172.21.15.31:0/1847838639 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f87640abdf0 con 0x7f87640cb060 2023-10-06T02:27:33.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.236+0000 7f874b7fe700 1 -- 172.21.15.31:0/1847838639 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v47) v1 ==== 136+0+0 (secure 0 0 0) 0x7f87640abdf0 con 0x7f87640cb060 2023-10-06T02:27:33.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.239+0000 7f8768f9d700 1 -- 172.21.15.31:0/1847838639 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f8750067290 msgr2=0x7f8750069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:33.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.239+0000 7f8768f9d700 1 --2- 172.21.15.31:0/1847838639 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f8750067290 0x7f8750069740 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f8764066170 tx=0x7f875400a0f0 comp rx=0 tx=0).stop 2023-10-06T02:27:33.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.239+0000 7f8768f9d700 1 -- 172.21.15.31:0/1847838639 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87640cb060 msgr2=0x7f87640cb4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:33.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.239+0000 7f8768f9d700 1 --2- 172.21.15.31:0/1847838639 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87640cb060 0x7f87640cb4d0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f874c00c180 tx=0x7f8758015000 comp rx=0 tx=0).stop 2023-10-06T02:27:33.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.239+0000 7f8768f9d700 1 -- 172.21.15.31:0/1847838639 shutdown_connections 2023-10-06T02:27:33.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.239+0000 7f8768f9d700 1 --2- 172.21.15.31:0/1847838639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87640c5b50 0x7f87640c5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:33.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.239+0000 7f8768f9d700 1 --2- 172.21.15.31:0/1847838639 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f8750067290 0x7f8750069740 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:33.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.239+0000 7f8768f9d700 1 --2- 172.21.15.31:0/1847838639 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f874c004810 0x7f876413b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:33.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.239+0000 7f8768f9d700 1 --2- 172.21.15.31:0/1847838639 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87640cb060 0x7f87640cb4d0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:33.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.239+0000 7f8768f9d700 1 -- 172.21.15.31:0/1847838639 >> 172.21.15.31:0/1847838639 conn(0x7f876406c6a0 msgr2=0x7f876414de70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:33.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.239+0000 7f8768f9d700 1 -- 172.21.15.31:0/1847838639 shutdown_connections 2023-10-06T02:27:33.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.239+0000 7f8768f9d700 1 -- 172.21.15.31:0/1847838639 wait complete. 2023-10-06T02:27:33.259 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-06T02:27:33.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.447+0000 7f5189d9d700 1 Processor -- start 2023-10-06T02:27:33.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.447+0000 7f5189d9d700 1 -- start start 2023-10-06T02:27:33.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.448+0000 7f5189d9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f518415de50 0x7f51840cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:33.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.448+0000 7f5189d9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51840cf3c0 con 0x7f5184153c30 2023-10-06T02:27:33.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.448+0000 7f5189d9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f518415afd0 con 0x7f51840cef40 2023-10-06T02:27:33.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.448+0000 7f5189d9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f518415b2b0 con 0x7f518415de50 2023-10-06T02:27:33.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.448+0000 7f5183fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f518415de50 0x7f51840cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:33.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.448+0000 7f5183fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f518415de50 0x7f51840cea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55236/0 (socket says 172.21.15.31:55236) 2023-10-06T02:27:33.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.448+0000 7f5183fff700 1 -- 172.21.15.31:0/2270530926 learned_addr learned my addr 172.21.15.31:0/2270530926 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:33.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.448+0000 7f5188d9b700 1 --1- 172.21.15.31:0/2270530926 >> v1:172.21.15.31:6789/0 conn(0x7f5184153c30 0x7f518415a5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:42624/0 (socket says 172.21.15.31:42624) 2023-10-06T02:27:33.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.449+0000 7f5183fff700 1 -- 172.21.15.31:0/2270530926 >> v1:172.21.15.62:6789/0 conn(0x7f51840cef40 legacy=0x7f518415c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:33.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.449+0000 7f5183fff700 1 -- 172.21.15.31:0/2270530926 >> v1:172.21.15.31:6789/0 conn(0x7f5184153c30 legacy=0x7f518415a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:33.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.449+0000 7f5183fff700 1 -- 172.21.15.31:0/2270530926 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f518415b590 con 0x7f518415de50 2023-10-06T02:27:33.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.449+0000 7f5183fff700 1 --2- 172.21.15.31:0/2270530926 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f518415de50 0x7f51840cea00 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f51700099b0 tx=0x7f5170009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa8201e57090cfef server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:33.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.449+0000 7f51837fe700 1 -- 172.21.15.31:0/2270530926 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f517000b820 con 0x7f518415de50 2023-10-06T02:27:33.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.449+0000 7f51837fe700 1 -- 172.21.15.31:0/2270530926 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f517000b980 con 0x7f518415de50 2023-10-06T02:27:33.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.449+0000 7f51837fe700 1 -- 172.21.15.31:0/2270530926 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f517000bd20 con 0x7f518415de50 2023-10-06T02:27:33.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.450+0000 7f5189d9d700 1 -- 172.21.15.31:0/2270530926 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f518415de50 msgr2=0x7f51840cea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:33.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.450+0000 7f5189d9d700 1 --2- 172.21.15.31:0/2270530926 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f518415de50 0x7f51840cea00 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f51700099b0 tx=0x7f5170009250 comp rx=0 tx=0).stop 2023-10-06T02:27:33.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.450+0000 7f5189d9d700 1 -- 172.21.15.31:0/2270530926 shutdown_connections 2023-10-06T02:27:33.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.450+0000 7f5189d9d700 1 --2- 172.21.15.31:0/2270530926 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f518415de50 0x7f51840cea00 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:33.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.450+0000 7f5189d9d700 1 -- 172.21.15.31:0/2270530926 >> 172.21.15.31:0/2270530926 conn(0x7f51840703a0 msgr2=0x7f51840707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:33.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.450+0000 7f5189d9d700 1 -- 172.21.15.31:0/2270530926 shutdown_connections 2023-10-06T02:27:33.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.450+0000 7f5189d9d700 1 -- 172.21.15.31:0/2270530926 wait complete. 2023-10-06T02:27:33.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.452+0000 7f5189d9d700 1 Processor -- start 2023-10-06T02:27:33.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.452+0000 7f5189d9d700 1 -- start start 2023-10-06T02:27:33.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.453+0000 7f5189d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51840cef40 0x7f518416a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:33.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.453+0000 7f5189d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5184153c30 0x7f5184150130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:33.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.453+0000 7f5189d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f518415de50 0x7f5184144e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:33.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.453+0000 7f5189d9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51840b4f40 con 0x7f51840cef40 2023-10-06T02:27:33.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.453+0000 7f5189d9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51840b32d0 con 0x7f518415de50 2023-10-06T02:27:33.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.453+0000 7f5189d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51840b3570 con 0x7f5184153c30 2023-10-06T02:27:33.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.453+0000 7f5188d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51840cef40 0x7f518416a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:33.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.453+0000 7f5188d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51840cef40 0x7f518416a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55250/0 (socket says 172.21.15.31:55250) 2023-10-06T02:27:33.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.453+0000 7f5188d9b700 1 -- 172.21.15.31:0/773921358 learned_addr learned my addr 172.21.15.31:0/773921358 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:33.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.453+0000 7f5183fff700 1 --2- 172.21.15.31:0/773921358 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5184153c30 0x7f5184150130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:33.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.453+0000 7f518959c700 1 --2- 172.21.15.31:0/773921358 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f518415de50 0x7f5184144e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:33.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.453+0000 7f5188d9b700 1 -- 172.21.15.31:0/773921358 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5184153c30 msgr2=0x7f5184150130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:33.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.453+0000 7f5188d9b700 1 --2- 172.21.15.31:0/773921358 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5184153c30 0x7f5184150130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:33.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.453+0000 7f5188d9b700 1 -- 172.21.15.31:0/773921358 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f518415de50 msgr2=0x7f5184144e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:33.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.453+0000 7f5188d9b700 1 --2- 172.21.15.31:0/773921358 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f518415de50 0x7f5184144e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:33.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.453+0000 7f5188d9b700 1 -- 172.21.15.31:0/773921358 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51700094f0 con 0x7f51840cef40 2023-10-06T02:27:33.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.453+0000 7f5183fff700 1 --2- 172.21.15.31:0/773921358 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5184153c30 0x7f5184150130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:27:33.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.454+0000 7f5188d9b700 1 --2- 172.21.15.31:0/773921358 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51840cef40 0x7f518416a8d0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f517800c7c0 tx=0x7f5178007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:33.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.454+0000 7f5180ff9700 1 -- 172.21.15.31:0/773921358 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5178005a70 con 0x7f51840cef40 2023-10-06T02:27:33.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.454+0000 7f5180ff9700 1 -- 172.21.15.31:0/773921358 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5178005bf0 con 0x7f51840cef40 2023-10-06T02:27:33.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.454+0000 7f5189d9d700 1 -- 172.21.15.31:0/773921358 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f518416a310 con 0x7f51840cef40 2023-10-06T02:27:33.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.454+0000 7f5180ff9700 1 -- 172.21.15.31:0/773921358 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51780070f0 con 0x7f51840cef40 2023-10-06T02:27:33.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.454+0000 7f5189d9d700 1 -- 172.21.15.31:0/773921358 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51840b4bc0 con 0x7f51840cef40 2023-10-06T02:27:33.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.455+0000 7f5189d9d700 1 -- 172.21.15.31:0/773921358 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f518404c3c0 con 0x7f51840cef40 2023-10-06T02:27:33.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.456+0000 7f5180ff9700 1 -- 172.21.15.31:0/773921358 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f5178007460 con 0x7f51840cef40 2023-10-06T02:27:33.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.457+0000 7f5180ff9700 1 --2- 172.21.15.31:0/773921358 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5168067290 0x7f5168069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:33.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.457+0000 7f5180ff9700 1 -- 172.21.15.31:0/773921358 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f51780847c0 con 0x7f51840cef40 2023-10-06T02:27:33.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.457+0000 7f5183fff700 1 --2- 172.21.15.31:0/773921358 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5168067290 0x7f5168069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:33.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.457+0000 7f5183fff700 1 --2- 172.21.15.31:0/773921358 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5168067290 0x7f5168069740 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f51700099b0 tx=0x7f5170016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:33.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.461+0000 7f5180ff9700 1 -- 172.21.15.31:0/773921358 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5178051400 con 0x7f51840cef40 2023-10-06T02:27:33.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.696+0000 7f5189d9d700 1 -- 172.21.15.31:0/773921358 --> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f51840b5080 con 0x7f5168067290 2023-10-06T02:27:33.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.698+0000 7f5180ff9700 1 -- 172.21.15.31:0/773921358 <== mgr.4107 v2:172.21.15.31:6832/85292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f517000baf0 con 0x7f5168067290 2023-10-06T02:27:33.699 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2023-10-06T02:27:33.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:33.699 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephfs" 2023-10-06T02:27:33.699 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:33.699 INFO:teuthology.orchestra.run.smithi031.stdout:] 2023-10-06T02:27:33.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.700+0000 7f5189d9d700 1 -- 172.21.15.31:0/773921358 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5168067290 msgr2=0x7f5168069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:33.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.700+0000 7f5189d9d700 1 --2- 172.21.15.31:0/773921358 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5168067290 0x7f5168069740 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f51700099b0 tx=0x7f5170016000 comp rx=0 tx=0).stop 2023-10-06T02:27:33.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.700+0000 7f5189d9d700 1 -- 172.21.15.31:0/773921358 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51840cef40 msgr2=0x7f518416a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:33.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.700+0000 7f5189d9d700 1 --2- 172.21.15.31:0/773921358 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51840cef40 0x7f518416a8d0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f517800c7c0 tx=0x7f5178007f70 comp rx=0 tx=0).stop 2023-10-06T02:27:33.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.700+0000 7f5189d9d700 1 -- 172.21.15.31:0/773921358 shutdown_connections 2023-10-06T02:27:33.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.700+0000 7f5189d9d700 1 --2- 172.21.15.31:0/773921358 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f518415de50 0x7f5184144e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:33.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.700+0000 7f5189d9d700 1 --2- 172.21.15.31:0/773921358 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51840cef40 0x7f518416a8d0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:33.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.700+0000 7f5189d9d700 1 --2- 172.21.15.31:0/773921358 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5168067290 0x7f5168069740 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:33.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.700+0000 7f5189d9d700 1 --2- 172.21.15.31:0/773921358 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5184153c30 0x7f5184150130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:33.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.700+0000 7f5189d9d700 1 -- 172.21.15.31:0/773921358 >> 172.21.15.31:0/773921358 conn(0x7f51840703a0 msgr2=0x7f51840c6430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:33.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.700+0000 7f5189d9d700 1 -- 172.21.15.31:0/773921358 shutdown_connections 2023-10-06T02:27:33.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.700+0000 7f5189d9d700 1 -- 172.21.15.31:0/773921358 wait complete. 2023-10-06T02:27:33.719 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T02:27:33.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.907+0000 7fd27759d700 1 Processor -- start 2023-10-06T02:27:34.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.907+0000 7fd27759d700 1 -- start start 2023-10-06T02:27:34.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.909+0000 7fd27759d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd270142c00 0x7fd27006b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:34.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.909+0000 7fd27759d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd2700c7f90 0x7fd27006c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:34.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.909+0000 7fd27759d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2700ca480 con 0x7fd27006bd20 2023-10-06T02:27:34.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.909+0000 7fd27759d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd27015e240 con 0x7fd270142c00 2023-10-06T02:27:34.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.909+0000 7fd27759d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd27006b350 con 0x7fd2700c7f90 2023-10-06T02:27:34.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.909+0000 7fd27659b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd270142c00 0x7fd27006b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:34.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.909+0000 7fd27659b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd270142c00 0x7fd27006b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55264/0 (socket says 172.21.15.31:55264) 2023-10-06T02:27:34.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.909+0000 7fd27659b700 1 -- 172.21.15.31:0/1593374776 learned_addr learned my addr 172.21.15.31:0/1593374776 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:34.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.909+0000 7fd27659b700 1 -- 172.21.15.31:0/1593374776 >> v2:172.21.15.62:3300/0 conn(0x7fd2700c7f90 msgr2=0x7fd27006c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:34.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.909+0000 7fd27659b700 1 --2- 172.21.15.31:0/1593374776 >> v2:172.21.15.62:3300/0 conn(0x7fd2700c7f90 0x7fd27006c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:34.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.910+0000 7fd27659b700 1 -- 172.21.15.31:0/1593374776 >> v1:172.21.15.62:6789/0 conn(0x7fd27006bd20 legacy=0x7fd2700c7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:34.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.910+0000 7fd27659b700 1 -- 172.21.15.31:0/1593374776 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd27015df60 con 0x7fd270142c00 2023-10-06T02:27:34.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.910+0000 7fd27659b700 1 --2- 172.21.15.31:0/1593374776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd270142c00 0x7fd27006b7e0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fd27006d780 tx=0x7fd26c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbb5f4f39ef51c64 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:34.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.910+0000 7fd275599700 1 -- 172.21.15.31:0/1593374776 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd26c00b820 con 0x7fd270142c00 2023-10-06T02:27:34.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.910+0000 7fd275599700 1 -- 172.21.15.31:0/1593374776 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd26c00b980 con 0x7fd270142c00 2023-10-06T02:27:34.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.910+0000 7fd275599700 1 -- 172.21.15.31:0/1593374776 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd26c00bd40 con 0x7fd270142c00 2023-10-06T02:27:34.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.911+0000 7fd27759d700 1 -- 172.21.15.31:0/1593374776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd270142c00 msgr2=0x7fd27006b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:34.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.911+0000 7fd27759d700 1 --2- 172.21.15.31:0/1593374776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd270142c00 0x7fd27006b7e0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fd27006d780 tx=0x7fd26c009250 comp rx=0 tx=0).stop 2023-10-06T02:27:34.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.911+0000 7fd27759d700 1 -- 172.21.15.31:0/1593374776 shutdown_connections 2023-10-06T02:27:34.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.911+0000 7fd27759d700 1 --2- 172.21.15.31:0/1593374776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd270142c00 0x7fd27006b7e0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:34.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.911+0000 7fd27759d700 1 --2- 172.21.15.31:0/1593374776 >> v2:172.21.15.62:3300/0 conn(0x7fd2700c7f90 0x7fd27006c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:34.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.911+0000 7fd27759d700 1 -- 172.21.15.31:0/1593374776 >> 172.21.15.31:0/1593374776 conn(0x7fd2700675a0 msgr2=0x7fd2700679a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:34.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.911+0000 7fd27759d700 1 -- 172.21.15.31:0/1593374776 shutdown_connections 2023-10-06T02:27:34.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.911+0000 7fd27759d700 1 -- 172.21.15.31:0/1593374776 wait complete. 2023-10-06T02:27:34.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.912+0000 7fd27759d700 1 Processor -- start 2023-10-06T02:27:34.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.912+0000 7fd27759d700 1 -- start start 2023-10-06T02:27:34.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.913+0000 7fd27759d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd27006bd20 0x7fd27016e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:34.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.913+0000 7fd27759d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd2700c7f90 0x7fd270159070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:34.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.913+0000 7fd27759d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd270142c00 0x7fd2700ca910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:34.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.913+0000 7fd27759d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd27013c490 con 0x7fd27006bd20 2023-10-06T02:27:34.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.913+0000 7fd27759d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd27016e7d0 con 0x7fd270142c00 2023-10-06T02:27:34.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.913+0000 7fd27759d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2700c60b0 con 0x7fd2700c7f90 2023-10-06T02:27:34.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.913+0000 7fd27659b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd27006bd20 0x7fd27016e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:34.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.913+0000 7fd27659b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd27006bd20 0x7fd27016e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55270/0 (socket says 172.21.15.31:55270) 2023-10-06T02:27:34.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.913+0000 7fd27659b700 1 -- 172.21.15.31:0/3430426047 learned_addr learned my addr 172.21.15.31:0/3430426047 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:34.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.914+0000 7fd275d9a700 1 --2- 172.21.15.31:0/3430426047 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd2700c7f90 0x7fd270159070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:34.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.914+0000 7fd276d9c700 1 --2- 172.21.15.31:0/3430426047 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd270142c00 0x7fd2700ca910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:34.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.914+0000 7fd27659b700 1 -- 172.21.15.31:0/3430426047 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd2700c7f90 msgr2=0x7fd270159070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:34.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.914+0000 7fd27659b700 1 --2- 172.21.15.31:0/3430426047 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd2700c7f90 0x7fd270159070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:34.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.914+0000 7fd27659b700 1 -- 172.21.15.31:0/3430426047 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd270142c00 msgr2=0x7fd2700ca910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:34.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.914+0000 7fd27659b700 1 --2- 172.21.15.31:0/3430426047 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd270142c00 0x7fd2700ca910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:34.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.914+0000 7fd27659b700 1 -- 172.21.15.31:0/3430426047 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd26c0094f0 con 0x7fd27006bd20 2023-10-06T02:27:34.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.914+0000 7fd275d9a700 1 --2- 172.21.15.31:0/3430426047 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd2700c7f90 0x7fd270159070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:27:34.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.914+0000 7fd27659b700 1 --2- 172.21.15.31:0/3430426047 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd27006bd20 0x7fd27016e290 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fd26c0090b0 tx=0x7fd26c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:34.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.914+0000 7fd2667fc700 1 -- 172.21.15.31:0/3430426047 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd26c026480 con 0x7fd27006bd20 2023-10-06T02:27:34.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.914+0000 7fd2667fc700 1 -- 172.21.15.31:0/3430426047 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd26c01f070 con 0x7fd27006bd20 2023-10-06T02:27:34.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.914+0000 7fd2667fc700 1 -- 172.21.15.31:0/3430426047 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd26c026870 con 0x7fd27006bd20 2023-10-06T02:27:34.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.914+0000 7fd27759d700 1 -- 172.21.15.31:0/3430426047 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd27013be40 con 0x7fd27006bd20 2023-10-06T02:27:34.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.914+0000 7fd27759d700 1 -- 172.21.15.31:0/3430426047 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2701595b0 con 0x7fd27006bd20 2023-10-06T02:27:34.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.915+0000 7fd27759d700 1 -- 172.21.15.31:0/3430426047 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd27004c3c0 con 0x7fd27006bd20 2023-10-06T02:27:34.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.916+0000 7fd2667fc700 1 -- 172.21.15.31:0/3430426047 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fd26c026c60 con 0x7fd27006bd20 2023-10-06T02:27:34.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.917+0000 7fd2667fc700 1 --2- 172.21.15.31:0/3430426047 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fd254067290 0x7fd254069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:34.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.917+0000 7fd2667fc700 1 -- 172.21.15.31:0/3430426047 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd26c021070 con 0x7fd27006bd20 2023-10-06T02:27:34.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.917+0000 7fd275d9a700 1 --2- 172.21.15.31:0/3430426047 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fd254067290 0x7fd254069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:34.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.918+0000 7fd275d9a700 1 --2- 172.21.15.31:0/3430426047 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fd254067290 0x7fd254069740 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd27006c180 tx=0x7fd260008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:34.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:33.922+0000 7fd2667fc700 1 -- 172.21.15.31:0/3430426047 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd26c053920 con 0x7fd27006bd20 2023-10-06T02:27:34.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:34.127+0000 7fd27759d700 1 -- 172.21.15.31:0/3430426047 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fd27006ad00 con 0x7fd27006bd20 2023-10-06T02:27:34.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:34.127+0000 7fd2667fc700 1 -- 172.21.15.31:0/3430426047 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd26c02b070 con 0x7fd27006bd20 2023-10-06T02:27:34.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:34.130+0000 7fd27759d700 1 -- 172.21.15.31:0/3430426047 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fd254067290 msgr2=0x7fd254069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:34.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:34.130+0000 7fd27759d700 1 --2- 172.21.15.31:0/3430426047 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fd254067290 0x7fd254069740 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd27006c180 tx=0x7fd260008040 comp rx=0 tx=0).stop 2023-10-06T02:27:34.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:34.130+0000 7fd27759d700 1 -- 172.21.15.31:0/3430426047 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd27006bd20 msgr2=0x7fd27016e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:34.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:34.130+0000 7fd27759d700 1 --2- 172.21.15.31:0/3430426047 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd27006bd20 0x7fd27016e290 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fd26c0090b0 tx=0x7fd26c016000 comp rx=0 tx=0).stop 2023-10-06T02:27:34.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:34.130+0000 7fd27759d700 1 -- 172.21.15.31:0/3430426047 shutdown_connections 2023-10-06T02:27:34.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:34.130+0000 7fd27759d700 1 --2- 172.21.15.31:0/3430426047 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd270142c00 0x7fd2700ca910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:34.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:34.130+0000 7fd27759d700 1 --2- 172.21.15.31:0/3430426047 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd27006bd20 0x7fd27016e290 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:34.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:34.130+0000 7fd27759d700 1 --2- 172.21.15.31:0/3430426047 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fd254067290 0x7fd254069740 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:34.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:34.130+0000 7fd27759d700 1 --2- 172.21.15.31:0/3430426047 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd2700c7f90 0x7fd270159070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:34.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:34.130+0000 7fd27759d700 1 -- 172.21.15.31:0/3430426047 >> 172.21.15.31:0/3430426047 conn(0x7fd2700675a0 msgr2=0x7fd270141d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:34.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:34.130+0000 7fd27759d700 1 -- 172.21.15.31:0/3430426047 shutdown_connections 2023-10-06T02:27:34.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:34.130+0000 7fd27759d700 1 -- 172.21.15.31:0/3430426047 wait complete. 2023-10-06T02:27:34.352 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T02:27:34.352 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:27:34.353 DEBUG:teuthology.orchestra.run.smithi031:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T02:27:34.435 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi031.front.sepia.ceph.com 2023-10-06T02:27:34.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:27:34.435 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T02:27:34.472 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-06T02:27:34.472 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:27:34.473 DEBUG:teuthology.orchestra.run.smithi062:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-06T02:27:34.569 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi062.front.sepia.ceph.com 2023-10-06T02:27:34.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:27:34.569 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T02:27:34.608 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:27:34.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:34.792+0000 7f9ed09fe700 1 Processor -- start 2023-10-06T02:27:34.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:34.793+0000 7f9ed09fe700 1 -- start start 2023-10-06T02:27:34.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:34.794+0000 7f9ed09fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9ecc06e0a0 0x7f9ecc0c6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:34.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:34.794+0000 7f9ed09fe700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9ecc06f3a0 0x7f9ecc14fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:34.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:34.794+0000 7f9ed09fe700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ecc06a490 con 0x7f9ecc06e470 2023-10-06T02:27:34.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:34.794+0000 7f9ed09fe700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ecc06b930 con 0x7f9ecc06e0a0 2023-10-06T02:27:34.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:34.794+0000 7f9ed09fe700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ecc0c7490 con 0x7f9ecc06f3a0 2023-10-06T02:27:34.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:34.794+0000 7f9ecad9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9ecc06e0a0 0x7f9ecc0c6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:34.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:34.794+0000 7f9ecad9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9ecc06e0a0 0x7f9ecc0c6b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55272/0 (socket says 172.21.15.31:55272) 2023-10-06T02:27:34.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:34.794+0000 7f9ecad9d700 1 -- 172.21.15.31:0/3599073445 learned_addr learned my addr 172.21.15.31:0/3599073445 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:34.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:34.795+0000 7f9eca59c700 1 --2- 172.21.15.31:0/3599073445 >> v2:172.21.15.62:3300/0 conn(0x7f9ecc06f3a0 0x7f9ecc14fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:34.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:34.795+0000 7f9ecad9d700 1 -- 172.21.15.31:0/3599073445 >> v2:172.21.15.62:3300/0 conn(0x7f9ecc06f3a0 msgr2=0x7f9ecc14fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:34.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:34.795+0000 7f9ecad9d700 1 --2- 172.21.15.31:0/3599073445 >> v2:172.21.15.62:3300/0 conn(0x7f9ecc06f3a0 0x7f9ecc14fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:34.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:34.795+0000 7f9ecad9d700 1 -- 172.21.15.31:0/3599073445 >> v1:172.21.15.62:6789/0 conn(0x7f9ecc06e470 legacy=0x7f9ecc159740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:34.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:34.795+0000 7f9ecad9d700 1 -- 172.21.15.31:0/3599073445 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ecc0c7770 con 0x7f9ecc06e0a0 2023-10-06T02:27:34.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:34.795+0000 7f9ecad9d700 1 --2- 172.21.15.31:0/3599073445 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ecc06e0a0 0x7f9ecc0c6b20 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc156140 tx=0x7f9ebc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbfcfcbfcc7a1d85 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:34.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:34.795+0000 7f9ec9d9b700 1 -- 172.21.15.31:0/3599073445 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc00b820 con 0x7f9ecc06e0a0 2023-10-06T02:27:34.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:34.795+0000 7f9ec9d9b700 1 -- 172.21.15.31:0/3599073445 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ebc00b980 con 0x7f9ecc06e0a0 2023-10-06T02:27:34.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:34.795+0000 7f9ec9d9b700 1 -- 172.21.15.31:0/3599073445 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc00bd20 con 0x7f9ecc06e0a0 2023-10-06T02:27:34.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:34.796+0000 7f9ed09fe700 1 -- 172.21.15.31:0/3599073445 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ecc06e0a0 msgr2=0x7f9ecc0c6b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:34.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:34.796+0000 7f9ed09fe700 1 --2- 172.21.15.31:0/3599073445 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ecc06e0a0 0x7f9ecc0c6b20 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc156140 tx=0x7f9ebc009250 comp rx=0 tx=0).stop 2023-10-06T02:27:34.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:34.796+0000 7f9ed09fe700 1 -- 172.21.15.31:0/3599073445 shutdown_connections 2023-10-06T02:27:34.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:34.796+0000 7f9ed09fe700 1 --2- 172.21.15.31:0/3599073445 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ecc06e0a0 0x7f9ecc0c6b20 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:34.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:34.796+0000 7f9ed09fe700 1 --2- 172.21.15.31:0/3599073445 >> v2:172.21.15.62:3300/0 conn(0x7f9ecc06f3a0 0x7f9ecc14fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:34.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:34.796+0000 7f9ed09fe700 1 -- 172.21.15.31:0/3599073445 >> 172.21.15.31:0/3599073445 conn(0x7f9ecc06d170 msgr2=0x7f9ecc0cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:34.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:34.796+0000 7f9ed09fe700 1 -- 172.21.15.31:0/3599073445 shutdown_connections 2023-10-06T02:27:34.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:34.796+0000 7f9ed09fe700 1 -- 172.21.15.31:0/3599073445 wait complete. 2023-10-06T02:27:34.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:34.797+0000 7f9ed09fe700 1 Processor -- start 2023-10-06T02:27:34.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:34.798+0000 7f9ed09fe700 1 -- start start 2023-10-06T02:27:34.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:34.798+0000 7f9ed09fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ecc06e0a0 0x7f9ecc158860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:34.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:34.798+0000 7f9ed09fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ecc06e470 0x7f9ecc0c4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:34.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:34.799+0000 7f9ed09fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9ecc06f3a0 0x7f9ecc13bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:34.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:34.799+0000 7f9ed09fe700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ecc0aeaa0 con 0x7f9ecc06e470 2023-10-06T02:27:34.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:34.799+0000 7f9ed09fe700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ecc13c330 con 0x7f9ecc06e0a0 2023-10-06T02:27:34.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:34.799+0000 7f9ed09fe700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ecc0c4f00 con 0x7f9ecc06f3a0 2023-10-06T02:27:34.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:34.799+0000 7f9eca59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ecc06e470 0x7f9ecc0c4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:34.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:34.799+0000 7f9eca59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ecc06e470 0x7f9ecc0c4860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55282/0 (socket says 172.21.15.31:55282) 2023-10-06T02:27:34.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:34.799+0000 7f9eca59c700 1 -- 172.21.15.31:0/3605052459 learned_addr learned my addr 172.21.15.31:0/3605052459 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:34.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:34.799+0000 7f9ecad9d700 1 --2- 172.21.15.31:0/3605052459 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ecc06e0a0 0x7f9ecc158860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:34.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:34.799+0000 7f9ecb59e700 1 --2- 172.21.15.31:0/3605052459 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9ecc06f3a0 0x7f9ecc13bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:34.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:34.799+0000 7f9eca59c700 1 -- 172.21.15.31:0/3605052459 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9ecc06f3a0 msgr2=0x7f9ecc13bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:34.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:34.799+0000 7f9eca59c700 1 --2- 172.21.15.31:0/3605052459 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9ecc06f3a0 0x7f9ecc13bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:34.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:34.799+0000 7f9eca59c700 1 -- 172.21.15.31:0/3605052459 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ecc06e0a0 msgr2=0x7f9ecc158860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:34.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:34.799+0000 7f9eca59c700 1 --2- 172.21.15.31:0/3605052459 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ecc06e0a0 0x7f9ecc158860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:34.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:34.799+0000 7f9eca59c700 1 -- 172.21.15.31:0/3605052459 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ebc0094f0 con 0x7f9ecc06e470 2023-10-06T02:27:34.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:34.799+0000 7f9ecb59e700 1 --2- 172.21.15.31:0/3605052459 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9ecc06f3a0 0x7f9ecc13bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:27:34.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:34.799+0000 7f9eca59c700 1 --2- 172.21.15.31:0/3605052459 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ecc06e470 0x7f9ecc0c4860 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f9ec000d870 tx=0x7f9ec0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:34.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:34.800+0000 7f9eb2ffd700 1 -- 172.21.15.31:0/3605052459 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec0014620 con 0x7f9ecc06e470 2023-10-06T02:27:34.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:34.800+0000 7f9eb2ffd700 1 -- 172.21.15.31:0/3605052459 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ec0014780 con 0x7f9ecc06e470 2023-10-06T02:27:34.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:34.800+0000 7f9eb2ffd700 1 -- 172.21.15.31:0/3605052459 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec0014b40 con 0x7f9ecc06e470 2023-10-06T02:27:34.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:34.800+0000 7f9ed09fe700 1 -- 172.21.15.31:0/3605052459 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ecc0c51e0 con 0x7f9ecc06e470 2023-10-06T02:27:34.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:34.800+0000 7f9ed09fe700 1 -- 172.21.15.31:0/3605052459 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ecc0c57a0 con 0x7f9ecc06e470 2023-10-06T02:27:34.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:34.801+0000 7f9ed09fe700 1 -- 172.21.15.31:0/3605052459 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ecc04c3c0 con 0x7f9ecc06e470 2023-10-06T02:27:34.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:34.802+0000 7f9eb2ffd700 1 -- 172.21.15.31:0/3605052459 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f9ec001d400 con 0x7f9ecc06e470 2023-10-06T02:27:34.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:34.803+0000 7f9eb2ffd700 1 --2- 172.21.15.31:0/3605052459 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f9eb8067290 0x7f9eb8069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:34.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:34.803+0000 7f9eb2ffd700 1 -- 172.21.15.31:0/3605052459 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9ec0085420 con 0x7f9ecc06e470 2023-10-06T02:27:34.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:34.803+0000 7f9ecad9d700 1 --2- 172.21.15.31:0/3605052459 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f9eb8067290 0x7f9eb8069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:34.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:34.804+0000 7f9ecad9d700 1 --2- 172.21.15.31:0/3605052459 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f9eb8067290 0x7f9eb8069740 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc156140 tx=0x7f9ebc016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:34.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:34.807+0000 7f9eb2ffd700 1 -- 172.21.15.31:0/3605052459 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9ec00520e0 con 0x7f9ecc06e470 2023-10-06T02:27:35.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.055+0000 7f9ed09fe700 1 -- 172.21.15.31:0/3605052459 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9ecc0c7a50 con 0x7f9ecc06e470 2023-10-06T02:27:35.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.056+0000 7f9eb2ffd700 1 -- 172.21.15.31:0/3605052459 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f9ec0055700 con 0x7f9ecc06e470 2023-10-06T02:27:35.058 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:27:35.058 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5474,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/2085637096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2085637096},{"type":"v1","addr":"172.21.15.62:6834","nonce":2085637096}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":5992,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/573080449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":573080449},{"type":"v1","addr":"172.21.15.31:6836","nonce":573080449}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":6007,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2625778079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2625778079},{"type":"v1","addr":"172.21.15.62:6835","nonce":2625778079}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:27:22.533533+0000","modified":"2023-10-06T02:27:26.062249+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6282":{"gid":6282,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6837/3003674295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":3003674295},{"type":"v1","addr":"172.21.15.31:6837","nonce":3003674295}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T02:27:35.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.058+0000 7f9ed09fe700 1 -- 172.21.15.31:0/3605052459 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f9eb8067290 msgr2=0x7f9eb8069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:35.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.058+0000 7f9ed09fe700 1 --2- 172.21.15.31:0/3605052459 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f9eb8067290 0x7f9eb8069740 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc156140 tx=0x7f9ebc016000 comp rx=0 tx=0).stop 2023-10-06T02:27:35.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.058+0000 7f9ed09fe700 1 -- 172.21.15.31:0/3605052459 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ecc06e470 msgr2=0x7f9ecc0c4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:35.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.058+0000 7f9ed09fe700 1 --2- 172.21.15.31:0/3605052459 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ecc06e470 0x7f9ecc0c4860 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f9ec000d870 tx=0x7f9ec0008040 comp rx=0 tx=0).stop 2023-10-06T02:27:35.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.058+0000 7f9ed09fe700 1 -- 172.21.15.31:0/3605052459 shutdown_connections 2023-10-06T02:27:35.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.058+0000 7f9ed09fe700 1 --2- 172.21.15.31:0/3605052459 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f9eb8067290 0x7f9eb8069740 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:35.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.059+0000 7f9ed09fe700 1 --2- 172.21.15.31:0/3605052459 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9ecc06f3a0 0x7f9ecc13bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:35.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.059+0000 7f9ed09fe700 1 --2- 172.21.15.31:0/3605052459 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ecc06e0a0 0x7f9ecc158860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:35.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.059+0000 7f9ed09fe700 1 --2- 172.21.15.31:0/3605052459 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ecc06e470 0x7f9ecc0c4860 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:35.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.059+0000 7f9ed09fe700 1 -- 172.21.15.31:0/3605052459 >> 172.21.15.31:0/3605052459 conn(0x7f9ecc06d170 msgr2=0x7f9ecc14eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:35.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.059+0000 7f9ed09fe700 1 -- 172.21.15.31:0/3605052459 shutdown_connections 2023-10-06T02:27:35.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.059+0000 7f9ed09fe700 1 -- 172.21.15.31:0/3605052459 wait complete. 2023-10-06T02:27:35.061 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 47 2023-10-06T02:27:35.077 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-10-06T02:27:35.289 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2023-10-06T02:27:35.290 DEBUG:tasks.cephfs.filesystem:_json_asok output [] 2023-10-06T02:27:35.290 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000011235 2023-10-06T02:27:35.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.478+0000 7fee60b9d700 1 Processor -- start 2023-10-06T02:27:35.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.478+0000 7fee60b9d700 1 -- start start 2023-10-06T02:27:35.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.479+0000 7fee60b9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fee5c140a70 0x7fee5c069b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:35.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.479+0000 7fee60b9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fee5c06de50 0x7fee5c06a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:35.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.479+0000 7fee60b9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee5c15b640 con 0x7fee5c15b0a0 2023-10-06T02:27:35.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.479+0000 7fee60b9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee5c0c6800 con 0x7fee5c06de50 2023-10-06T02:27:35.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.479+0000 7fee60b9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee5c15b920 con 0x7fee5c140a70 2023-10-06T02:27:35.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.479+0000 7fee5affd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fee5c06de50 0x7fee5c06a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:35.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.479+0000 7fee5affd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fee5c06de50 0x7fee5c06a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55290/0 (socket says 172.21.15.31:55290) 2023-10-06T02:27:35.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.479+0000 7fee5affd700 1 -- 172.21.15.31:0/2771761957 learned_addr learned my addr 172.21.15.31:0/2771761957 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:35.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.480+0000 7fee5b7fe700 1 --2- 172.21.15.31:0/2771761957 >> v2:172.21.15.62:3300/0 conn(0x7fee5c140a70 0x7fee5c069b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:35.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.480+0000 7fee5affd700 1 -- 172.21.15.31:0/2771761957 >> v2:172.21.15.62:3300/0 conn(0x7fee5c140a70 msgr2=0x7fee5c069b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:35.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.480+0000 7fee5affd700 1 --2- 172.21.15.31:0/2771761957 >> v2:172.21.15.62:3300/0 conn(0x7fee5c140a70 0x7fee5c069b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:35.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.480+0000 7fee5affd700 1 -- 172.21.15.31:0/2771761957 >> v1:172.21.15.62:6789/0 conn(0x7fee5c15b0a0 legacy=0x7fee5c157a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:35.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.480+0000 7fee5affd700 1 -- 172.21.15.31:0/2771761957 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee5c15bc00 con 0x7fee5c06de50 2023-10-06T02:27:35.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.480+0000 7fee5affd700 1 --2- 172.21.15.31:0/2771761957 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee5c06de50 0x7fee5c06a040 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fee5c06f940 tx=0x7fee50009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=359e489dfeea91f8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:35.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.480+0000 7fee5a7fc700 1 -- 172.21.15.31:0/2771761957 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5000b860 con 0x7fee5c06de50 2023-10-06T02:27:35.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.480+0000 7fee5a7fc700 1 -- 172.21.15.31:0/2771761957 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee5000b9c0 con 0x7fee5c06de50 2023-10-06T02:27:35.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.480+0000 7fee5a7fc700 1 -- 172.21.15.31:0/2771761957 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5000bd60 con 0x7fee5c06de50 2023-10-06T02:27:35.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.481+0000 7fee60b9d700 1 -- 172.21.15.31:0/2771761957 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee5c06de50 msgr2=0x7fee5c06a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:35.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.481+0000 7fee60b9d700 1 --2- 172.21.15.31:0/2771761957 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee5c06de50 0x7fee5c06a040 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fee5c06f940 tx=0x7fee50009250 comp rx=0 tx=0).stop 2023-10-06T02:27:35.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.481+0000 7fee60b9d700 1 -- 172.21.15.31:0/2771761957 shutdown_connections 2023-10-06T02:27:35.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.481+0000 7fee60b9d700 1 --2- 172.21.15.31:0/2771761957 >> v2:172.21.15.62:3300/0 conn(0x7fee5c140a70 0x7fee5c069b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:35.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.481+0000 7fee60b9d700 1 --2- 172.21.15.31:0/2771761957 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee5c06de50 0x7fee5c06a040 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:35.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.481+0000 7fee60b9d700 1 -- 172.21.15.31:0/2771761957 >> 172.21.15.31:0/2771761957 conn(0x7fee5c06d110 msgr2=0x7fee5c0cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:35.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.481+0000 7fee60b9d700 1 -- 172.21.15.31:0/2771761957 shutdown_connections 2023-10-06T02:27:35.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.481+0000 7fee60b9d700 1 -- 172.21.15.31:0/2771761957 wait complete. 2023-10-06T02:27:35.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.482+0000 7fee60b9d700 1 Processor -- start 2023-10-06T02:27:35.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.483+0000 7fee60b9d700 1 -- start start 2023-10-06T02:27:35.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.484+0000 7fee60b9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fee5c06de50 0x7fee5c0aff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:35.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.484+0000 7fee60b9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee5c140a70 0x7fee5c1580a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:35.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.484+0000 7fee60b9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee5c15b0a0 0x7fee5c15ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:35.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.484+0000 7fee60b9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee5c15f170 con 0x7fee5c140a70 2023-10-06T02:27:35.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.484+0000 7fee60b9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee5c0b05a0 con 0x7fee5c15b0a0 2023-10-06T02:27:35.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.484+0000 7fee60b9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee5c0b0880 con 0x7fee5c06de50 2023-10-06T02:27:35.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.484+0000 7fee5affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee5c140a70 0x7fee5c1580a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:35.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.484+0000 7fee5affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee5c140a70 0x7fee5c1580a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55294/0 (socket says 172.21.15.31:55294) 2023-10-06T02:27:35.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.484+0000 7fee5affd700 1 -- 172.21.15.31:0/644470299 learned_addr learned my addr 172.21.15.31:0/644470299 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:35.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.484+0000 7fee5b7fe700 1 --2- 172.21.15.31:0/644470299 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fee5c06de50 0x7fee5c0aff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:35.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.484+0000 7fee5affd700 1 -- 172.21.15.31:0/644470299 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fee5c06de50 msgr2=0x7fee5c0aff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:35.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.484+0000 7fee5affd700 1 --2- 172.21.15.31:0/644470299 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fee5c06de50 0x7fee5c0aff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:35.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.484+0000 7fee5affd700 1 -- 172.21.15.31:0/644470299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee5c15b0a0 msgr2=0x7fee5c15ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:35.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.484+0000 7fee5bfff700 1 --2- 172.21.15.31:0/644470299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee5c15b0a0 0x7fee5c15ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:35.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.484+0000 7fee5affd700 1 --2- 172.21.15.31:0/644470299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee5c15b0a0 0x7fee5c15ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:35.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.484+0000 7fee5affd700 1 -- 172.21.15.31:0/644470299 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee500094f0 con 0x7fee5c140a70 2023-10-06T02:27:35.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.484+0000 7fee5b7fe700 1 --2- 172.21.15.31:0/644470299 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fee5c06de50 0x7fee5c0aff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:27:35.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.484+0000 7fee5bfff700 1 --2- 172.21.15.31:0/644470299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee5c15b0a0 0x7fee5c15ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:27:35.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.485+0000 7fee5affd700 1 --2- 172.21.15.31:0/644470299 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee5c140a70 0x7fee5c1580a0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fee5c0cc000 tx=0x7fee50017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:35.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.485+0000 7fee437fe700 1 -- 172.21.15.31:0/644470299 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee50011e60 con 0x7fee5c140a70 2023-10-06T02:27:35.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.485+0000 7fee437fe700 1 -- 172.21.15.31:0/644470299 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee5001a630 con 0x7fee5c140a70 2023-10-06T02:27:35.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.485+0000 7fee437fe700 1 -- 172.21.15.31:0/644470299 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5001a9f0 con 0x7fee5c140a70 2023-10-06T02:27:35.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.485+0000 7fee60b9d700 1 -- 172.21.15.31:0/644470299 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee5c0b0b60 con 0x7fee5c140a70 2023-10-06T02:27:35.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.485+0000 7fee60b9d700 1 -- 172.21.15.31:0/644470299 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee5c0b1120 con 0x7fee5c140a70 2023-10-06T02:27:35.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.486+0000 7fee60b9d700 1 -- 172.21.15.31:0/644470299 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee5c0b1340 con 0x7fee5c140a70 2023-10-06T02:27:35.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.487+0000 7fee437fe700 1 -- 172.21.15.31:0/644470299 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fee5002d440 con 0x7fee5c140a70 2023-10-06T02:27:35.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.488+0000 7fee437fe700 1 --2- 172.21.15.31:0/644470299 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fee48067280 0x7fee48069730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:35.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.488+0000 7fee437fe700 1 -- 172.21.15.31:0/644470299 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fee50086870 con 0x7fee5c140a70 2023-10-06T02:27:35.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.488+0000 7fee5b7fe700 1 --2- 172.21.15.31:0/644470299 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fee48067280 0x7fee48069730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:35.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.489+0000 7fee5b7fe700 1 --2- 172.21.15.31:0/644470299 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fee48067280 0x7fee48069730 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fee4c006fd0 tx=0x7fee4c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:35.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.492+0000 7fee437fe700 1 -- 172.21.15.31:0/644470299 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fee50053550 con 0x7fee5c140a70 2023-10-06T02:27:35.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.725+0000 7fee60b9d700 1 -- 172.21.15.31:0/644470299 --> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000011235", "target": ["mon-mgr", ""]}) v1 -- 0x7fee5c1585e0 con 0x7fee48067280 2023-10-06T02:27:35.733 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:27:35.732+0000 7f3e7b47e700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:27:35.733 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:27:35.732+0000 7f3e7b47e700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:27:35.733 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:27:35.732+0000 7f3e7b47e700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:27:35.733 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:27:35.732+0000 7f3e7b47e700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:27:35.733 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:27:35.732+0000 7f3e7b47e700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:27:35.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.766+0000 7fee437fe700 1 -- 172.21.15.31:0/644470299 <== mgr.4107 v2:172.21.15.31:6832/85292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fee4c0097d0 con 0x7fee48067280 2023-10-06T02:27:35.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.769+0000 7fee60b9d700 1 -- 172.21.15.31:0/644470299 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fee48067280 msgr2=0x7fee48069730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:35.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.769+0000 7fee60b9d700 1 --2- 172.21.15.31:0/644470299 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fee48067280 0x7fee48069730 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fee4c006fd0 tx=0x7fee4c008040 comp rx=0 tx=0).stop 2023-10-06T02:27:35.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.769+0000 7fee60b9d700 1 -- 172.21.15.31:0/644470299 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee5c140a70 msgr2=0x7fee5c1580a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:35.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.769+0000 7fee60b9d700 1 --2- 172.21.15.31:0/644470299 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee5c140a70 0x7fee5c1580a0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fee5c0cc000 tx=0x7fee50017000 comp rx=0 tx=0).stop 2023-10-06T02:27:35.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.769+0000 7fee60b9d700 1 -- 172.21.15.31:0/644470299 shutdown_connections 2023-10-06T02:27:35.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.769+0000 7fee60b9d700 1 --2- 172.21.15.31:0/644470299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee5c15b0a0 0x7fee5c15ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:35.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.769+0000 7fee60b9d700 1 --2- 172.21.15.31:0/644470299 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7fee48067280 0x7fee48069730 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:35.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.769+0000 7fee60b9d700 1 --2- 172.21.15.31:0/644470299 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fee5c06de50 0x7fee5c0aff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:35.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.769+0000 7fee60b9d700 1 --2- 172.21.15.31:0/644470299 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee5c140a70 0x7fee5c1580a0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:35.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.769+0000 7fee60b9d700 1 -- 172.21.15.31:0/644470299 >> 172.21.15.31:0/644470299 conn(0x7fee5c06d110 msgr2=0x7fee5c13fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:35.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.769+0000 7fee60b9d700 1 -- 172.21.15.31:0/644470299 shutdown_connections 2023-10-06T02:27:35.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.769+0000 7fee60b9d700 1 -- 172.21.15.31:0/644470299 wait complete. 2023-10-06T02:27:35.787 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:27:35.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.974+0000 7ffa9f19d700 1 Processor -- start 2023-10-06T02:27:35.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.974+0000 7ffa9f19d700 1 -- start start 2023-10-06T02:27:35.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.976+0000 7ffa9f19d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ffa9813fb20 0x7ffa9815a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:35.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.976+0000 7ffa9f19d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa9815ae70 con 0x7ffa9806aaa0 2023-10-06T02:27:35.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.976+0000 7ffa9f19d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa9806af60 con 0x7ffa9813fef0 2023-10-06T02:27:35.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.976+0000 7ffa9f19d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa9806b240 con 0x7ffa9813fb20 2023-10-06T02:27:35.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.976+0000 7ffa9e19b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ffa9813fb20 0x7ffa9815a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:35.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.976+0000 7ffa9e19b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ffa9813fb20 0x7ffa9815a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55332/0 (socket says 172.21.15.31:55332) 2023-10-06T02:27:35.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.976+0000 7ffa9e19b700 1 -- 172.21.15.31:0/3142735022 learned_addr learned my addr 172.21.15.31:0/3142735022 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:35.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.976+0000 7ffa9e19b700 1 -- 172.21.15.31:0/3142735022 >> v1:172.21.15.62:6789/0 conn(0x7ffa9813fef0 legacy=0x7ffa980c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:35.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.976+0000 7ffa9e19b700 1 -- 172.21.15.31:0/3142735022 >> v1:172.21.15.31:6789/0 conn(0x7ffa9806aaa0 legacy=0x7ffa980c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:35.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.976+0000 7ffa9e19b700 1 -- 172.21.15.31:0/3142735022 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa9806b520 con 0x7ffa9813fb20 2023-10-06T02:27:35.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.977+0000 7ffa9e19b700 1 --2- 172.21.15.31:0/3142735022 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa9813fb20 0x7ffa9815a930 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7ffa9400bc50 tx=0x7ffa9400b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa0e10bedaa54fd6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:35.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.977+0000 7ffa9d199700 1 -- 172.21.15.31:0/3142735022 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa9400be60 con 0x7ffa9813fb20 2023-10-06T02:27:35.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.977+0000 7ffa9d199700 1 -- 172.21.15.31:0/3142735022 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa9400ddc0 con 0x7ffa9813fb20 2023-10-06T02:27:35.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.977+0000 7ffa9d199700 1 -- 172.21.15.31:0/3142735022 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa940106a0 con 0x7ffa9813fb20 2023-10-06T02:27:35.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.977+0000 7ffa9f19d700 1 -- 172.21.15.31:0/3142735022 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa9813fb20 msgr2=0x7ffa9815a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:35.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.977+0000 7ffa9f19d700 1 --2- 172.21.15.31:0/3142735022 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa9813fb20 0x7ffa9815a930 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7ffa9400bc50 tx=0x7ffa9400b4f0 comp rx=0 tx=0).stop 2023-10-06T02:27:35.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.978+0000 7ffa9f19d700 1 -- 172.21.15.31:0/3142735022 shutdown_connections 2023-10-06T02:27:35.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.978+0000 7ffa9f19d700 1 --2- 172.21.15.31:0/3142735022 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa9813fb20 0x7ffa9815a930 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:35.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.978+0000 7ffa9f19d700 1 -- 172.21.15.31:0/3142735022 >> 172.21.15.31:0/3142735022 conn(0x7ffa9806d170 msgr2=0x7ffa980cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:35.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.978+0000 7ffa9f19d700 1 -- 172.21.15.31:0/3142735022 shutdown_connections 2023-10-06T02:27:35.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.978+0000 7ffa9f19d700 1 -- 172.21.15.31:0/3142735022 wait complete. 2023-10-06T02:27:35.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.979+0000 7ffa9f19d700 1 Processor -- start 2023-10-06T02:27:35.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.979+0000 7ffa9f19d700 1 -- start start 2023-10-06T02:27:35.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.980+0000 7ffa9f19d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa9806aaa0 0x7ffa98155f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:35.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.980+0000 7ffa9f19d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa9813fb20 0x7ffa980ab7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:35.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.980+0000 7ffa9f19d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffa9813fef0 0x7ffa980ca6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:35.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.980+0000 7ffa9f19d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa980ac250 con 0x7ffa9813fb20 2023-10-06T02:27:35.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.980+0000 7ffa9f19d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa980ac530 con 0x7ffa9806aaa0 2023-10-06T02:27:35.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.980+0000 7ffa9f19d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa980ac810 con 0x7ffa9813fef0 2023-10-06T02:27:35.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.980+0000 7ffa9d99a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa9813fb20 0x7ffa980ab7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:35.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.980+0000 7ffa9d99a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa9813fb20 0x7ffa980ab7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55338/0 (socket says 172.21.15.31:55338) 2023-10-06T02:27:35.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.980+0000 7ffa9d99a700 1 -- 172.21.15.31:0/1824294993 learned_addr learned my addr 172.21.15.31:0/1824294993 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:35.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.980+0000 7ffa9e19b700 1 --2- 172.21.15.31:0/1824294993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa9806aaa0 0x7ffa98155f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:35.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.981+0000 7ffa9e99c700 1 --2- 172.21.15.31:0/1824294993 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffa9813fef0 0x7ffa980ca6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:35.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.981+0000 7ffa9d99a700 1 -- 172.21.15.31:0/1824294993 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffa9813fef0 msgr2=0x7ffa980ca6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:35.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.981+0000 7ffa9d99a700 1 --2- 172.21.15.31:0/1824294993 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffa9813fef0 0x7ffa980ca6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:35.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.981+0000 7ffa9d99a700 1 -- 172.21.15.31:0/1824294993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa9806aaa0 msgr2=0x7ffa98155f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:35.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.981+0000 7ffa9d99a700 1 --2- 172.21.15.31:0/1824294993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa9806aaa0 0x7ffa98155f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:35.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.981+0000 7ffa9d99a700 1 -- 172.21.15.31:0/1824294993 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa9400b790 con 0x7ffa9813fb20 2023-10-06T02:27:35.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.981+0000 7ffa9d99a700 1 --2- 172.21.15.31:0/1824294993 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa9813fb20 0x7ffa980ab7d0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7ffa8800b530 tx=0x7ffa880090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:35.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.981+0000 7ffa8e7fc700 1 -- 172.21.15.31:0/1824294993 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa88005b40 con 0x7ffa9813fb20 2023-10-06T02:27:35.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.981+0000 7ffa8e7fc700 1 -- 172.21.15.31:0/1824294993 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa88005ca0 con 0x7ffa9813fb20 2023-10-06T02:27:35.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.981+0000 7ffa9f19d700 1 -- 172.21.15.31:0/1824294993 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa980acaf0 con 0x7ffa9813fb20 2023-10-06T02:27:35.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.981+0000 7ffa8e7fc700 1 -- 172.21.15.31:0/1824294993 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa88006260 con 0x7ffa9813fb20 2023-10-06T02:27:35.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.981+0000 7ffa9f19d700 1 -- 172.21.15.31:0/1824294993 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa980ad890 con 0x7ffa9813fb20 2023-10-06T02:27:35.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.983+0000 7ffa9f19d700 1 -- 172.21.15.31:0/1824294993 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa9804c3c0 con 0x7ffa9813fb20 2023-10-06T02:27:35.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.983+0000 7ffa8e7fc700 1 -- 172.21.15.31:0/1824294993 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7ffa88006650 con 0x7ffa9813fb20 2023-10-06T02:27:35.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.984+0000 7ffa8e7fc700 1 --2- 172.21.15.31:0/1824294993 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7ffa7c067290 0x7ffa7c069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:35.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.984+0000 7ffa8e7fc700 1 -- 172.21.15.31:0/1824294993 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ffa88083e20 con 0x7ffa9813fb20 2023-10-06T02:27:35.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.985+0000 7ffa9e19b700 1 --2- 172.21.15.31:0/1824294993 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7ffa7c067290 0x7ffa7c069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:35.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.985+0000 7ffa9e19b700 1 --2- 172.21.15.31:0/1824294993 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7ffa7c067290 0x7ffa7c069740 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ffa9400bc50 tx=0x7ffa9401a000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:35.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:35.989+0000 7ffa8e7fc700 1 -- 172.21.15.31:0/1824294993 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffa8804b5a0 con 0x7ffa9813fb20 2023-10-06T02:27:36.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.235+0000 7ffa9f19d700 1 -- 172.21.15.31:0/1824294993 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffa9806b800 con 0x7ffa9813fb20 2023-10-06T02:27:36.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.235+0000 7ffa8e7fc700 1 -- 172.21.15.31:0/1824294993 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4596 (secure 0 0 0) 0x7ffa88017020 con 0x7ffa9813fb20 2023-10-06T02:27:36.237 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:27:36.237 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5474,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/2085637096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2085637096},{"type":"v1","addr":"172.21.15.62:6834","nonce":2085637096}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":5992,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/573080449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":573080449},{"type":"v1","addr":"172.21.15.31:6836","nonce":573080449}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":6007,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2625778079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2625778079},{"type":"v1","addr":"172.21.15.62:6835","nonce":2625778079}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:27:22.533533+0000","modified":"2023-10-06T02:27:26.062249+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6282":{"gid":6282,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6837/3003674295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":3003674295},{"type":"v1","addr":"172.21.15.31:6837","nonce":3003674295}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T02:27:36.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.238+0000 7ffa9f19d700 1 -- 172.21.15.31:0/1824294993 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7ffa7c067290 msgr2=0x7ffa7c069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:36.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.238+0000 7ffa9f19d700 1 --2- 172.21.15.31:0/1824294993 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7ffa7c067290 0x7ffa7c069740 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ffa9400bc50 tx=0x7ffa9401a000 comp rx=0 tx=0).stop 2023-10-06T02:27:36.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.238+0000 7ffa9f19d700 1 -- 172.21.15.31:0/1824294993 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa9813fb20 msgr2=0x7ffa980ab7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:36.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.238+0000 7ffa9f19d700 1 --2- 172.21.15.31:0/1824294993 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa9813fb20 0x7ffa980ab7d0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7ffa8800b530 tx=0x7ffa880090f0 comp rx=0 tx=0).stop 2023-10-06T02:27:36.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.238+0000 7ffa9f19d700 1 -- 172.21.15.31:0/1824294993 shutdown_connections 2023-10-06T02:27:36.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.238+0000 7ffa9f19d700 1 --2- 172.21.15.31:0/1824294993 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7ffa7c067290 0x7ffa7c069740 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:36.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.238+0000 7ffa9f19d700 1 --2- 172.21.15.31:0/1824294993 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffa9813fef0 0x7ffa980ca6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:36.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.238+0000 7ffa9f19d700 1 --2- 172.21.15.31:0/1824294993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa9806aaa0 0x7ffa98155f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:36.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.238+0000 7ffa9f19d700 1 --2- 172.21.15.31:0/1824294993 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa9813fb20 0x7ffa980ab7d0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:36.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.238+0000 7ffa9f19d700 1 -- 172.21.15.31:0/1824294993 >> 172.21.15.31:0/1824294993 conn(0x7ffa9806d170 msgr2=0x7ffa980c1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:36.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.238+0000 7ffa9f19d700 1 -- 172.21.15.31:0/1824294993 shutdown_connections 2023-10-06T02:27:36.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.238+0000 7ffa9f19d700 1 -- 172.21.15.31:0/1824294993 wait complete. 2023-10-06T02:27:36.241 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 47 2023-10-06T02:27:36.258 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-10-06T02:27:36.467 INFO:teuthology.orchestra.run.smithi031.stdout:[{"id":6507,"entity":{"name":{"type":"client","num":6507},"addr":{"type":"any","addr":"172.21.15.31:0","nonce":1049264351}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":14,"uptime":0.71798902899999995,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":19.967423727858129,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.6507 172.21.15.31:0/1049264351","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x000000000017ffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"736ac5ba72cb47af7d6f6666b1e100f7e20a20e2","ceph_version":"ceph version 17.2.6-1296-g736ac5ba (736ac5ba72cb47af7d6f6666b1e100f7e20a20e2) quincy (stable)","entity_id":"y","hostname":"smithi031","pid":"85292","root":"/"}}] 2023-10-06T02:27:36.468 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "736ac5ba72cb47af7d6f6666b1e100f7e20a20e2", "ceph_version": "ceph version 17.2.6-1296-g736ac5ba (736ac5ba72cb47af7d6f6666b1e100f7e20a20e2) quincy (stable)", "client_features": { "feature_bits": "0x000000000017ffff" }, "entity_id": "y", "hostname": "smithi031", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "85292", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.31:0", "nonce": 1049264351, "type": "any" }, "name": { "num": 6507, "type": "client" } }, "id": 6507, "inst": "client.6507 172.21.15.31:0/1049264351", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 14, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 19.96742372785813 }, "state": "open", "uptime": 0.717989029 } ] 2023-10-06T02:27:36.468 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2023-10-06T02:27:36.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.655+0000 7f2004f9d700 1 Processor -- start 2023-10-06T02:27:36.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.655+0000 7f2004f9d700 1 -- start start 2023-10-06T02:27:36.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.656+0000 7f2004f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2000150380 0x7f2000160ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:36.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.656+0000 7f2004f9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f200006a490 con 0x7f200006e0a0 2023-10-06T02:27:36.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.656+0000 7f2004f9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20000c7230 con 0x7f200006f3a0 2023-10-06T02:27:36.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.656+0000 7f2004f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f200006b930 con 0x7f2000150380 2023-10-06T02:27:36.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.656+0000 7f1fff7fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f200006e0a0 0x7f20000c6b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:42632/0 (socket says 172.21.15.31:42632) 2023-10-06T02:27:36.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.656+0000 7f1fff7fe700 1 -- 172.21.15.31:0/397672878 learned_addr learned my addr 172.21.15.31:0/397672878 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:27:36.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.657+0000 7f1ffffff700 1 --2- 172.21.15.31:0/397672878 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2000150380 0x7f2000160ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:36.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.657+0000 7f1ffe7fc700 1 -- 172.21.15.31:0/397672878 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1068020702 0 0) 0x7f200006a490 con 0x7f200006e0a0 2023-10-06T02:27:36.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.657+0000 7f1ffe7fc700 1 -- 172.21.15.31:0/397672878 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1fe8004680 con 0x7f200006e0a0 2023-10-06T02:27:36.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.657+0000 7f1ffe7fc700 1 -- 172.21.15.31:0/397672878 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3761923247 0 0) 0x7f20000c7230 con 0x7f200006f3a0 2023-10-06T02:27:36.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.657+0000 7f1ffe7fc700 1 -- 172.21.15.31:0/397672878 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f200006a490 con 0x7f200006f3a0 2023-10-06T02:27:36.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.657+0000 7f1ffe7fc700 1 -- 172.21.15.31:0/397672878 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2365557511 0 0) 0x7f1fe8004680 con 0x7f200006e0a0 2023-10-06T02:27:36.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.657+0000 7f1ffe7fc700 1 -- 172.21.15.31:0/397672878 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2000150380 msgr2=0x7f2000160ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:36.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.657+0000 7f1ffe7fc700 1 --2- 172.21.15.31:0/397672878 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2000150380 0x7f2000160ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:36.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.657+0000 7f1ffe7fc700 1 -- 172.21.15.31:0/397672878 >> v1:172.21.15.62:6789/0 conn(0x7f200006f3a0 legacy=0x7f200014fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:36.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.657+0000 7f1ffe7fc700 1 -- 172.21.15.31:0/397672878 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20000c7490 con 0x7f200006e0a0 2023-10-06T02:27:36.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.657+0000 7f1ffe7fc700 1 -- 172.21.15.31:0/397672878 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7f1ff0001470 con 0x7f200006e0a0 2023-10-06T02:27:36.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.657+0000 7f1ffe7fc700 1 -- 172.21.15.31:0/397672878 >> v1:172.21.15.31:6789/0 conn(0x7f200006e0a0 legacy=0x7f20000c6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:36.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.657+0000 7f1ffe7fc700 1 --2- 172.21.15.31:0/397672878 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2000150380 0x7f1fe8004d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:36.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.657+0000 7f1ffe7fc700 1 --2- 172.21.15.31:0/397672878 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1fe80052e0 0x7f1fe8005920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:36.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.657+0000 7f1ffe7fc700 1 --2- 172.21.15.31:0/397672878 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1fe8005e60 0x7f1fe80062d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:36.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.657+0000 7f1ffe7fc700 1 -- 172.21.15.31:0/397672878 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f200006b930 con 0x7f1fe80052e0 2023-10-06T02:27:36.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.657+0000 7f1ffe7fc700 1 -- 172.21.15.31:0/397672878 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe800cd00 con 0x7f1fe8005e60 2023-10-06T02:27:36.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.657+0000 7f1ffe7fc700 1 -- 172.21.15.31:0/397672878 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe800cec0 con 0x7f2000150380 2023-10-06T02:27:36.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.658+0000 7f1ffeffd700 1 --2- 172.21.15.31:0/397672878 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1fe80052e0 0x7f1fe8005920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:36.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.658+0000 7f1fff7fe700 1 --2- 172.21.15.31:0/397672878 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2000150380 0x7f1fe8004d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:36.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.658+0000 7f1ffffff700 1 --2- 172.21.15.31:0/397672878 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1fe8005e60 0x7f1fe80062d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:36.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.658+0000 7f1ffeffd700 1 -- 172.21.15.31:0/397672878 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2000150380 msgr2=0x7f1fe8004d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:36.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.658+0000 7f1ffeffd700 1 --2- 172.21.15.31:0/397672878 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2000150380 0x7f1fe8004d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:36.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.658+0000 7f1ffeffd700 1 -- 172.21.15.31:0/397672878 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1fe8005e60 msgr2=0x7f1fe80062d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:36.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.658+0000 7f1ffeffd700 1 --2- 172.21.15.31:0/397672878 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1fe8005e60 0x7f1fe80062d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:36.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.658+0000 7f1ffeffd700 1 -- 172.21.15.31:0/397672878 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fe8004680 con 0x7f1fe80052e0 2023-10-06T02:27:36.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.658+0000 7f1ffeffd700 1 --2- 172.21.15.31:0/397672878 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1fe80052e0 0x7f1fe8005920 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f1ff4001a30 tx=0x7f1ff4007d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d8569536d76dea1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:36.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.658+0000 7f1ffe7fc700 1 -- 172.21.15.31:0/397672878 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff40013f0 con 0x7f1fe80052e0 2023-10-06T02:27:36.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.659+0000 7f1ffe7fc700 1 -- 172.21.15.31:0/397672878 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ff400c5d0 con 0x7f1fe80052e0 2023-10-06T02:27:36.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.659+0000 7f1ffe7fc700 1 -- 172.21.15.31:0/397672878 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff400c990 con 0x7f1fe80052e0 2023-10-06T02:27:36.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.659+0000 7f2004f9d700 1 -- 172.21.15.31:0/397672878 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1fe80052e0 msgr2=0x7f1fe8005920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:36.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.659+0000 7f2004f9d700 1 --2- 172.21.15.31:0/397672878 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1fe80052e0 0x7f1fe8005920 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f1ff4001a30 tx=0x7f1ff4007d80 comp rx=0 tx=0).stop 2023-10-06T02:27:36.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.659+0000 7f1ffffff700 1 -- 172.21.15.31:0/397672878 reap_dead start 2023-10-06T02:27:36.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.659+0000 7f2004f9d700 1 -- 172.21.15.31:0/397672878 shutdown_connections 2023-10-06T02:27:36.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.659+0000 7f2004f9d700 1 -- 172.21.15.31:0/397672878 >> 172.21.15.31:0/397672878 conn(0x7f200006d170 msgr2=0x7f20000cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:36.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.659+0000 7f2004f9d700 1 -- 172.21.15.31:0/397672878 shutdown_connections 2023-10-06T02:27:36.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.659+0000 7f2004f9d700 1 -- 172.21.15.31:0/397672878 wait complete. 2023-10-06T02:27:36.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.661+0000 7f2004f9d700 1 Processor -- start 2023-10-06T02:27:36.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.661+0000 7f2004f9d700 1 -- start start 2023-10-06T02:27:36.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.662+0000 7f2004f9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1fe80052e0 0x7f200016ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:36.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.662+0000 7f2004f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2000139900 0x7f200016b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:36.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.662+0000 7f2004f9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20000b08d0 0x7f20000b0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:36.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.662+0000 7f2004f9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2000168760 con 0x7f20000b08d0 2023-10-06T02:27:36.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.662+0000 7f2004f9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2000168d20 con 0x7f1fe80052e0 2023-10-06T02:27:36.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.662+0000 7f2004f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20000b1650 con 0x7f2000139900 2023-10-06T02:27:36.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.662+0000 7f1ffffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20000b08d0 0x7f20000b0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:36.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.662+0000 7f1ffffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20000b08d0 0x7f20000b0d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55364/0 (socket says 172.21.15.31:55364) 2023-10-06T02:27:36.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.662+0000 7f1ffffff700 1 -- 172.21.15.31:0/2304334503 learned_addr learned my addr 172.21.15.31:0/2304334503 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:36.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.662+0000 7f1fff7fe700 1 --2- 172.21.15.31:0/2304334503 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1fe80052e0 0x7f200016ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:36.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.662+0000 7f1ffeffd700 1 --2- 172.21.15.31:0/2304334503 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2000139900 0x7f200016b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:36.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.662+0000 7f1ffffff700 1 -- 172.21.15.31:0/2304334503 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2000139900 msgr2=0x7f200016b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:36.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.662+0000 7f1ffffff700 1 --2- 172.21.15.31:0/2304334503 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2000139900 0x7f200016b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:36.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.662+0000 7f1ffffff700 1 -- 172.21.15.31:0/2304334503 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1fe80052e0 msgr2=0x7f200016ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:36.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.662+0000 7f1ffffff700 1 --2- 172.21.15.31:0/2304334503 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1fe80052e0 0x7f200016ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:36.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.662+0000 7f1ffffff700 1 -- 172.21.15.31:0/2304334503 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20000b1930 con 0x7f20000b08d0 2023-10-06T02:27:36.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.662+0000 7f1ffeffd700 1 --2- 172.21.15.31:0/2304334503 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2000139900 0x7f200016b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:27:36.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.663+0000 7f1ffffff700 1 --2- 172.21.15.31:0/2304334503 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20000b08d0 0x7f20000b0d70 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f20001507c0 tx=0x7f1ff800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:36.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.663+0000 7f1fe77fe700 1 -- 172.21.15.31:0/2304334503 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff800e040 con 0x7f20000b08d0 2023-10-06T02:27:36.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.663+0000 7f1fe77fe700 1 -- 172.21.15.31:0/2304334503 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ff800f040 con 0x7f20000b08d0 2023-10-06T02:27:36.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.663+0000 7f2004f9d700 1 -- 172.21.15.31:0/2304334503 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20001689b0 con 0x7f20000b08d0 2023-10-06T02:27:36.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.663+0000 7f1fe77fe700 1 -- 172.21.15.31:0/2304334503 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff800fe10 con 0x7f20000b08d0 2023-10-06T02:27:36.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.663+0000 7f2004f9d700 1 -- 172.21.15.31:0/2304334503 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20001691b0 con 0x7f20000b08d0 2023-10-06T02:27:36.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.664+0000 7f2004f9d700 1 -- 172.21.15.31:0/2304334503 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2000168eb0 con 0x7f20000b08d0 2023-10-06T02:27:36.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.665+0000 7f1fe77fe700 1 -- 172.21.15.31:0/2304334503 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f1ff8015640 con 0x7f20000b08d0 2023-10-06T02:27:36.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.667+0000 7f1fe77fe700 1 --2- 172.21.15.31:0/2304334503 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f1fec067290 0x7f1fec069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:36.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.667+0000 7f1fe77fe700 1 -- 172.21.15.31:0/2304334503 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1ff80857a0 con 0x7f20000b08d0 2023-10-06T02:27:36.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.667+0000 7f1fff7fe700 1 --2- 172.21.15.31:0/2304334503 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f1fec067290 0x7f1fec069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:36.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.667+0000 7f1fff7fe700 1 --2- 172.21.15.31:0/2304334503 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f1fec067290 0x7f1fec069740 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f1ff0001200 tx=0x7f1ff0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:36.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.671+0000 7f1fe77fe700 1 -- 172.21.15.31:0/2304334503 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1ff8052250 con 0x7f20000b08d0 2023-10-06T02:27:36.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.926+0000 7f2004f9d700 1 -- 172.21.15.31:0/2304334503 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7f2000168b40 con 0x7f20000b08d0 2023-10-06T02:27:36.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.933+0000 7f1fe77fe700 1 -- 172.21.15.31:0/2304334503 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v12) v1 ==== 81+0+450694 (secure 0 0 0) 0x7f1ff8055870 con 0x7f20000b08d0 2023-10-06T02:27:36.941 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:27:36.941 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T02:27:36.942 INFO:teuthology.orchestra.run.smithi031.stdout: "epoch": 12, 2023-10-06T02:27:36.942 INFO:teuthology.orchestra.run.smithi031.stdout: "active_gid": 4107, 2023-10-06T02:27:36.942 INFO:teuthology.orchestra.run.smithi031.stdout: "active_name": "y", 2023-10-06T02:27:36.942 INFO:teuthology.orchestra.run.smithi031.stdout: "active_addrs": { 2023-10-06T02:27:36.942 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2023-10-06T02:27:36.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:36.942 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2023-10-06T02:27:36.942 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6832", 2023-10-06T02:27:36.942 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 85292 2023-10-06T02:27:36.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:36.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:36.943 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2023-10-06T02:27:36.943 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6833", 2023-10-06T02:27:36.943 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 85292 2023-10-06T02:27:36.944 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:36.944 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-10-06T02:27:36.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:36.944 INFO:teuthology.orchestra.run.smithi031.stdout: "active_addr": "172.21.15.31:6833/85292", 2023-10-06T02:27:36.944 INFO:teuthology.orchestra.run.smithi031.stdout: "active_change": "2023-10-06T02:22:47.104009+0000", 2023-10-06T02:27:36.944 INFO:teuthology.orchestra.run.smithi031.stdout: "active_mgr_features": 4540138320759226367, 2023-10-06T02:27:36.944 INFO:teuthology.orchestra.run.smithi031.stdout: "available": true, 2023-10-06T02:27:36.945 INFO:teuthology.orchestra.run.smithi031.stdout: "standbys": [ 2023-10-06T02:27:36.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:36.945 INFO:teuthology.orchestra.run.smithi031.stdout: "gid": 4109, 2023-10-06T02:27:36.945 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "x", 2023-10-06T02:27:36.945 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_features": 4540138320759226367, 2023-10-06T02:27:36.945 INFO:teuthology.orchestra.run.smithi031.stdout: "available_modules": [ 2023-10-06T02:27:36.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:36.945 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "alerts", 2023-10-06T02:27:36.945 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T02:27:36.945 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T02:27:36.946 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T02:27:36.946 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2023-10-06T02:27:36.946 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2023-10-06T02:27:36.946 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T02:27:36.946 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:36.946 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:36.946 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2023-10-06T02:27:36.946 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:36.946 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:36.946 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:36.947 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How frequently to reexamine health status", 2023-10-06T02:27:36.947 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:36.947 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:36.947 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:36.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:36.947 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T02:27:36.947 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T02:27:36.947 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:36.947 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:36.947 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:36.948 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:36.948 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:36.948 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:36.948 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:36.948 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:36.948 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:36.948 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:36.948 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:36.948 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:36.948 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:36.948 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:36.949 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:36.949 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:36.949 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:36.949 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:36.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:36.949 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T02:27:36.949 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T02:27:36.949 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:36.949 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:36.949 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:36.950 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:36.950 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:36.950 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:36.950 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:36.950 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:36.950 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:36.950 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:36.950 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:36.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:36.950 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T02:27:36.951 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T02:27:36.951 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:36.951 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:36.951 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:36.951 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T02:27:36.951 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:36.951 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:36.951 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:36.951 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:36.951 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:36.952 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:36.952 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:36.952 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:36.952 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:36.952 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:36.952 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:36.952 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:36.952 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:36.952 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:36.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:36.953 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T02:27:36.953 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T02:27:36.953 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:36.953 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:36.953 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:36.953 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:36.953 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:36.953 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:36.953 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:36.953 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:36.954 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:36.954 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:36.954 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:36.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:36.954 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_destination": { 2023-10-06T02:27:36.954 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_destination", 2023-10-06T02:27:36.954 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:36.954 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:36.954 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:36.954 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:36.955 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:36.955 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:36.955 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:36.955 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Email address to send alerts to", 2023-10-06T02:27:36.955 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:36.955 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:36.955 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:36.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:36.955 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_from_name": { 2023-10-06T02:27:36.955 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_from_name", 2023-10-06T02:27:36.955 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:36.956 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:36.956 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:36.956 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "Ceph", 2023-10-06T02:27:36.956 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:36.956 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:36.956 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:36.956 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Email From: name", 2023-10-06T02:27:36.956 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:36.956 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:36.956 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:36.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:36.957 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_host": { 2023-10-06T02:27:36.957 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_host", 2023-10-06T02:27:36.957 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:36.957 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:36.957 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:36.957 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:36.957 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:36.957 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:36.957 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:36.958 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP server", 2023-10-06T02:27:36.958 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:36.958 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:36.958 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:36.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:36.958 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_password": { 2023-10-06T02:27:36.958 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_password", 2023-10-06T02:27:36.958 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:36.958 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:36.958 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:36.959 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:36.959 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:36.959 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:36.959 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:36.959 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Password to authenticate with", 2023-10-06T02:27:36.959 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:36.959 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:36.959 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:36.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:36.959 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_port": { 2023-10-06T02:27:36.959 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_port", 2023-10-06T02:27:36.960 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T02:27:36.960 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:36.960 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:36.960 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "465", 2023-10-06T02:27:36.960 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:36.960 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:36.960 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:36.960 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP port", 2023-10-06T02:27:36.960 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:36.960 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:36.960 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:36.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:36.961 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_sender": { 2023-10-06T02:27:36.961 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_sender", 2023-10-06T02:27:36.961 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:36.961 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:36.961 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:36.961 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:36.961 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:36.961 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:36.961 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:36.962 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP envelope sender", 2023-10-06T02:27:36.962 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:36.962 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:36.962 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:36.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:36.962 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_ssl": { 2023-10-06T02:27:36.962 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_ssl", 2023-10-06T02:27:36.962 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:36.962 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:36.962 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:36.963 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T02:27:36.963 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:36.963 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:36.963 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:36.963 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use SSL to connect to SMTP server", 2023-10-06T02:27:36.963 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:36.963 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:36.963 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:36.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:36.963 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_user": { 2023-10-06T02:27:36.964 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_user", 2023-10-06T02:27:36.964 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:36.964 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:36.964 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:36.964 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:36.964 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:36.964 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:36.964 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:36.964 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "User to authenticate as", 2023-10-06T02:27:36.964 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:36.964 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:36.965 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:36.965 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:36.965 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:36.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:36.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:36.965 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "balancer", 2023-10-06T02:27:36.965 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T02:27:36.965 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T02:27:36.965 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T02:27:36.965 INFO:teuthology.orchestra.run.smithi031.stdout: "active": { 2023-10-06T02:27:36.966 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "active", 2023-10-06T02:27:36.966 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:36.966 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:36.966 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:36.966 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T02:27:36.966 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:36.966 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:36.966 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:36.966 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "automatically balance PGs across cluster", 2023-10-06T02:27:36.966 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:36.967 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:36.967 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:36.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:36.967 INFO:teuthology.orchestra.run.smithi031.stdout: "begin_time": { 2023-10-06T02:27:36.967 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "begin_time", 2023-10-06T02:27:36.967 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:36.967 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:36.967 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:36.967 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0000", 2023-10-06T02:27:36.967 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:36.968 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:36.968 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:36.968 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "beginning time of day to automatically balance", 2023-10-06T02:27:36.968 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-10-06T02:27:36.968 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:36.968 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:36.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:36.968 INFO:teuthology.orchestra.run.smithi031.stdout: "begin_weekday": { 2023-10-06T02:27:36.968 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "begin_weekday", 2023-10-06T02:27:36.968 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2023-10-06T02:27:36.969 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:36.969 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:36.969 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2023-10-06T02:27:36.969 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0", 2023-10-06T02:27:36.969 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "7", 2023-10-06T02:27:36.969 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:36.969 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-10-06T02:27:36.969 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "0 or 7 = Sunday, 1 = Monday, etc.", 2023-10-06T02:27:36.969 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:36.969 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:36.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:36.970 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_max_iterations": { 2023-10-06T02:27:36.970 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_max_iterations", 2023-10-06T02:27:36.970 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2023-10-06T02:27:36.970 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:36.970 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:36.970 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "25", 2023-10-06T02:27:36.970 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2023-10-06T02:27:36.970 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "250", 2023-10-06T02:27:36.970 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:36.971 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-10-06T02:27:36.971 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:36.971 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:36.971 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:36.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:36.971 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_metrics": { 2023-10-06T02:27:36.971 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_metrics", 2023-10-06T02:27:36.971 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:36.971 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:36.971 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:36.972 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "pgs,objects,bytes", 2023-10-06T02:27:36.972 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:36.972 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:36.972 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:36.972 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-10-06T02:27:36.972 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-10-06T02:27:36.972 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:36.972 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:36.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:36.972 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_step": { 2023-10-06T02:27:36.973 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_step", 2023-10-06T02:27:36.973 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-10-06T02:27:36.973 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:36.973 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:36.973 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0.5", 2023-10-06T02:27:36.973 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0.001", 2023-10-06T02:27:36.973 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "0.999", 2023-10-06T02:27:36.973 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:36.973 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "aggressiveness of optimization", 2023-10-06T02:27:36.973 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-10-06T02:27:36.973 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:36.974 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:36.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:36.974 INFO:teuthology.orchestra.run.smithi031.stdout: "end_time": { 2023-10-06T02:27:36.974 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "end_time", 2023-10-06T02:27:36.974 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:36.974 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:36.974 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:36.974 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "2400", 2023-10-06T02:27:36.974 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:36.975 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:36.975 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:36.975 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "ending time of day to automatically balance", 2023-10-06T02:27:36.975 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-10-06T02:27:36.975 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:36.975 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:36.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:36.975 INFO:teuthology.orchestra.run.smithi031.stdout: "end_weekday": { 2023-10-06T02:27:36.975 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "end_weekday", 2023-10-06T02:27:36.975 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2023-10-06T02:27:36.976 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:36.976 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:36.976 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "7", 2023-10-06T02:27:36.976 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0", 2023-10-06T02:27:36.976 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "7", 2023-10-06T02:27:36.976 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:36.976 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-10-06T02:27:36.976 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "0 or 7 = Sunday, 1 = Monday, etc.", 2023-10-06T02:27:36.976 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:36.976 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:36.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:36.977 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T02:27:36.977 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T02:27:36.977 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:36.977 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:36.977 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:36.977 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:36.977 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:36.977 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:36.977 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:36.978 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:36.978 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:36.978 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:36.978 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:36.978 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:36.978 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:36.978 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:36.978 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:36.978 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:36.978 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:36.979 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:36.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:36.979 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T02:27:36.979 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T02:27:36.979 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:36.979 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:36.979 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:36.979 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:36.979 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:36.979 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:36.980 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:36.980 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:36.980 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:36.980 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:36.980 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:36.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:36.980 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T02:27:36.980 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T02:27:36.980 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:36.980 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:36.980 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:36.981 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T02:27:36.981 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:36.981 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:36.981 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:36.981 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:36.981 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:36.981 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:36.981 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:36.981 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:36.981 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:36.982 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:36.982 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:36.982 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:36.982 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:36.982 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:36.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:36.982 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T02:27:36.982 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T02:27:36.982 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:36.982 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:36.983 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:36.983 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:36.983 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:36.983 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:36.983 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:36.983 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:36.983 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:36.983 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:36.983 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:36.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:36.984 INFO:teuthology.orchestra.run.smithi031.stdout: "min_score": { 2023-10-06T02:27:36.984 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "min_score", 2023-10-06T02:27:36.984 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-10-06T02:27:36.984 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:36.984 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:36.984 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2023-10-06T02:27:36.984 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:36.984 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:36.985 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:36.985 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-10-06T02:27:36.985 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:36.985 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:36.985 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:36.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:36.985 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": { 2023-10-06T02:27:36.985 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mode", 2023-10-06T02:27:36.985 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:36.985 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:36.986 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:36.986 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "upmap", 2023-10-06T02:27:36.986 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:36.986 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:36.986 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:36.986 INFO:teuthology.orchestra.run.smithi031.stdout: "crush-compat", 2023-10-06T02:27:36.986 INFO:teuthology.orchestra.run.smithi031.stdout: "none", 2023-10-06T02:27:36.986 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap" 2023-10-06T02:27:36.986 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:36.986 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Balancer mode", 2023-10-06T02:27:36.987 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:36.987 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:36.987 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:36.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:36.987 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_ids": { 2023-10-06T02:27:36.987 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pool_ids", 2023-10-06T02:27:36.987 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:36.987 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:36.987 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:36.987 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:36.988 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:36.988 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:36.988 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:36.988 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-10-06T02:27:36.988 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:36.988 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:36.988 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:36.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:36.988 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2023-10-06T02:27:36.989 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2023-10-06T02:27:36.989 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T02:27:36.989 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:36.989 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:36.989 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2023-10-06T02:27:36.989 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:36.989 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:36.989 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:36.989 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-10-06T02:27:36.989 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:36.990 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:36.990 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:36.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:36.990 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap_max_deviation": { 2023-10-06T02:27:36.990 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "upmap_max_deviation", 2023-10-06T02:27:36.990 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T02:27:36.990 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:36.990 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:36.990 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2023-10-06T02:27:36.990 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2023-10-06T02:27:36.991 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:36.991 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:36.991 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "deviation below which no optimization is attempted", 2023-10-06T02:27:36.991 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-10-06T02:27:36.991 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:36.991 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:36.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:36.991 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap_max_optimizations": { 2023-10-06T02:27:36.991 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "upmap_max_optimizations", 2023-10-06T02:27:36.991 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2023-10-06T02:27:36.992 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:36.992 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:36.992 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-10-06T02:27:36.992 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:36.992 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:36.992 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:36.992 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-10-06T02:27:36.992 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:36.992 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:36.992 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:36.992 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:36.993 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:36.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:36.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:36.993 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephadm", 2023-10-06T02:27:36.993 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T02:27:36.993 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T02:27:36.993 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T02:27:36.993 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_down_multiplier": { 2023-10-06T02:27:36.993 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_down_multiplier", 2023-10-06T02:27:36.993 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-10-06T02:27:36.994 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:36.994 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:36.994 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3.0", 2023-10-06T02:27:36.994 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:36.994 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:36.994 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:36.994 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-10-06T02:27:36.994 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:36.994 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:36.994 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:36.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:36.995 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_refresh_rate": { 2023-10-06T02:27:36.995 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_refresh_rate", 2023-10-06T02:27:36.995 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T02:27:36.995 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:36.995 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:36.995 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "20", 2023-10-06T02:27:36.995 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:36.995 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:36.995 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:36.996 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-10-06T02:27:36.996 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:36.996 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:36.996 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:36.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:36.996 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_starting_port": { 2023-10-06T02:27:36.996 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_starting_port", 2023-10-06T02:27:36.996 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T02:27:36.996 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:36.996 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:36.996 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "4721", 2023-10-06T02:27:36.997 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:36.997 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:36.997 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:36.997 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-10-06T02:27:36.997 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:36.997 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:36.997 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:36.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:36.997 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_ptrace": { 2023-10-06T02:27:36.998 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_ptrace", 2023-10-06T02:27:36.998 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:36.998 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:36.998 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:36.998 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:36.998 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:36.998 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:36.998 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:36.998 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-10-06T02:27:36.998 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-10-06T02:27:36.999 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:36.999 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:36.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:36.999 INFO:teuthology.orchestra.run.smithi031.stdout: "autotune_interval": { 2023-10-06T02:27:36.999 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "autotune_interval", 2023-10-06T02:27:36.999 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T02:27:36.999 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:36.999 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:36.999 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2023-10-06T02:27:36.999 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.000 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.000 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.000 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to autotune daemon memory", 2023-10-06T02:27:37.000 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.000 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.000 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.000 INFO:teuthology.orchestra.run.smithi031.stdout: "autotune_memory_target_ratio": { 2023-10-06T02:27:37.000 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "autotune_memory_target_ratio", 2023-10-06T02:27:37.000 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-10-06T02:27:37.000 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.001 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.001 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0.7", 2023-10-06T02:27:37.001 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.001 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.001 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.001 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-10-06T02:27:37.001 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.001 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.001 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.002 INFO:teuthology.orchestra.run.smithi031.stdout: "cgroups_split": { 2023-10-06T02:27:37.002 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cgroups_split", 2023-10-06T02:27:37.002 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.002 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.002 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.002 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T02:27:37.002 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.002 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.002 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.003 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-10-06T02:27:37.003 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.003 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.003 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.003 INFO:teuthology.orchestra.run.smithi031.stdout: "config_checks_enabled": { 2023-10-06T02:27:37.003 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_checks_enabled", 2023-10-06T02:27:37.003 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.003 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.004 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.004 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.004 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.004 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.004 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.004 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-10-06T02:27:37.004 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.004 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.004 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.005 INFO:teuthology.orchestra.run.smithi031.stdout: "config_dashboard": { 2023-10-06T02:27:37.005 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_dashboard", 2023-10-06T02:27:37.005 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.005 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.005 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.005 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T02:27:37.005 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.005 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.005 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.005 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-10-06T02:27:37.005 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.006 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.006 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.006 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_alertmanager": { 2023-10-06T02:27:37.006 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_alertmanager", 2023-10-06T02:27:37.006 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.006 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.006 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.006 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-10-06T02:27:37.006 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.007 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.007 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.007 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2023-10-06T02:27:37.007 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.007 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.007 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.007 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_base": { 2023-10-06T02:27:37.007 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_base", 2023-10-06T02:27:37.007 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.008 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.008 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.008 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/ceph", 2023-10-06T02:27:37.008 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.008 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.008 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.008 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Container image name, without the tag", 2023-10-06T02:27:37.008 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.008 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.008 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.009 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_grafana": { 2023-10-06T02:27:37.009 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_grafana", 2023-10-06T02:27:37.009 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.009 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.009 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.009 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-10-06T02:27:37.009 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.009 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.009 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.010 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2023-10-06T02:27:37.010 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.010 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.010 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.010 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_haproxy": { 2023-10-06T02:27:37.010 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_haproxy", 2023-10-06T02:27:37.010 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.010 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.010 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.011 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-10-06T02:27:37.011 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.011 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.011 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.011 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "HAproxy container image", 2023-10-06T02:27:37.011 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.011 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.011 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.012 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_keepalived": { 2023-10-06T02:27:37.012 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_keepalived", 2023-10-06T02:27:37.012 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.012 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.012 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.012 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/keepalived:2.1.5", 2023-10-06T02:27:37.012 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.012 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.012 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.012 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Keepalived container image", 2023-10-06T02:27:37.013 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.013 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.013 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.013 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_loki": { 2023-10-06T02:27:37.013 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_loki", 2023-10-06T02:27:37.013 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.013 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.013 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.013 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-10-06T02:27:37.014 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.014 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.014 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.014 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Loki container image", 2023-10-06T02:27:37.014 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.014 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.014 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.014 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_node_exporter": { 2023-10-06T02:27:37.015 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_node_exporter", 2023-10-06T02:27:37.015 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.015 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.015 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.015 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-10-06T02:27:37.015 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.015 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.015 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.015 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2023-10-06T02:27:37.015 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.016 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.016 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.016 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_prometheus": { 2023-10-06T02:27:37.016 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_prometheus", 2023-10-06T02:27:37.016 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.016 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.016 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.016 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-10-06T02:27:37.016 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.017 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.017 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.017 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2023-10-06T02:27:37.017 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.017 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.017 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.017 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_promtail": { 2023-10-06T02:27:37.018 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_promtail", 2023-10-06T02:27:37.018 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.018 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.018 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.018 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-10-06T02:27:37.018 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.018 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.018 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.018 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Promtail container image", 2023-10-06T02:27:37.018 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.019 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.019 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.019 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_snmp_gateway": { 2023-10-06T02:27:37.019 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_snmp_gateway", 2023-10-06T02:27:37.019 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.019 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.019 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.019 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-10-06T02:27:37.020 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.020 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.020 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.020 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SNMP Gateway container image", 2023-10-06T02:27:37.020 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.020 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.020 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.020 INFO:teuthology.orchestra.run.smithi031.stdout: "container_init": { 2023-10-06T02:27:37.021 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_init", 2023-10-06T02:27:37.021 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.021 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.021 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.021 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T02:27:37.021 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.021 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.021 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.021 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Run podman/docker with `--init`", 2023-10-06T02:27:37.021 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.022 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.022 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.022 INFO:teuthology.orchestra.run.smithi031.stdout: "daemon_cache_timeout": { 2023-10-06T02:27:37.022 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "daemon_cache_timeout", 2023-10-06T02:27:37.022 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T02:27:37.022 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.022 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.022 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2023-10-06T02:27:37.022 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.023 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.023 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.023 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-10-06T02:27:37.023 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.023 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.023 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.023 INFO:teuthology.orchestra.run.smithi031.stdout: "default_cephadm_command_timeout": { 2023-10-06T02:27:37.023 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "default_cephadm_command_timeout", 2023-10-06T02:27:37.023 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T02:27:37.024 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.024 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.024 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "900", 2023-10-06T02:27:37.024 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.024 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.024 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.024 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-10-06T02:27:37.024 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.024 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.024 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.025 INFO:teuthology.orchestra.run.smithi031.stdout: "default_registry": { 2023-10-06T02:27:37.025 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "default_registry", 2023-10-06T02:27:37.025 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.025 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.025 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.025 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io", 2023-10-06T02:27:37.025 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.025 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.025 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.026 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-10-06T02:27:37.026 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.026 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.026 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.026 INFO:teuthology.orchestra.run.smithi031.stdout: "device_cache_timeout": { 2023-10-06T02:27:37.026 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_cache_timeout", 2023-10-06T02:27:37.026 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T02:27:37.026 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.026 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.027 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1800", 2023-10-06T02:27:37.027 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.027 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.027 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.027 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache device inventory", 2023-10-06T02:27:37.027 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.027 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.027 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.028 INFO:teuthology.orchestra.run.smithi031.stdout: "device_enhanced_scan": { 2023-10-06T02:27:37.028 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_enhanced_scan", 2023-10-06T02:27:37.028 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.028 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.028 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.028 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.028 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.028 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.028 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.028 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use libstoragemgmt during device scans", 2023-10-06T02:27:37.029 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.029 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.029 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.029 INFO:teuthology.orchestra.run.smithi031.stdout: "facts_cache_timeout": { 2023-10-06T02:27:37.029 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "facts_cache_timeout", 2023-10-06T02:27:37.029 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T02:27:37.029 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.029 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.029 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2023-10-06T02:27:37.029 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.030 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.030 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.030 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache host facts data", 2023-10-06T02:27:37.030 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.030 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.030 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.030 INFO:teuthology.orchestra.run.smithi031.stdout: "host_check_interval": { 2023-10-06T02:27:37.030 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "host_check_interval", 2023-10-06T02:27:37.030 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T02:27:37.031 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.031 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.031 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2023-10-06T02:27:37.031 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.031 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.031 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.031 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to perform a host check", 2023-10-06T02:27:37.031 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.031 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.031 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.032 INFO:teuthology.orchestra.run.smithi031.stdout: "inventory_list_all": { 2023-10-06T02:27:37.032 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "inventory_list_all", 2023-10-06T02:27:37.032 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.032 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.032 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.032 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.032 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.032 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.032 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.033 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-10-06T02:27:37.033 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.033 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.033 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.033 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T02:27:37.033 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T02:27:37.033 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.033 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.033 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.034 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.034 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.034 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.034 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.034 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.034 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.034 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.034 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.034 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.034 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.034 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.035 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.035 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.035 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.035 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.035 INFO:teuthology.orchestra.run.smithi031.stdout: "log_refresh_metadata": { 2023-10-06T02:27:37.035 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_refresh_metadata", 2023-10-06T02:27:37.035 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.035 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.036 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.036 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.036 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.036 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.036 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.036 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-10-06T02:27:37.036 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.036 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.036 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.037 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T02:27:37.037 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T02:27:37.037 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.037 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.037 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.037 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T02:27:37.037 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.037 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.037 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.037 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-10-06T02:27:37.038 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.038 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.038 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.038 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T02:27:37.038 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T02:27:37.038 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.038 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.038 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.038 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T02:27:37.039 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.039 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.039 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.039 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.039 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.039 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.039 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.039 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.039 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.039 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.039 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.040 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.040 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.040 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.040 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T02:27:37.040 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T02:27:37.040 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.040 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.040 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.040 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.040 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.041 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.041 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.041 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.041 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.041 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.041 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.041 INFO:teuthology.orchestra.run.smithi031.stdout: "manage_etc_ceph_ceph_conf": { 2023-10-06T02:27:37.041 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-10-06T02:27:37.041 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.042 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.042 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.042 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.042 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.042 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.042 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.042 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-10-06T02:27:37.042 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.042 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.042 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.043 INFO:teuthology.orchestra.run.smithi031.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-10-06T02:27:37.043 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-10-06T02:27:37.043 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.043 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.043 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.043 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "*", 2023-10-06T02:27:37.043 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.043 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.043 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.043 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-10-06T02:27:37.044 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.044 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.044 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.044 INFO:teuthology.orchestra.run.smithi031.stdout: "max_count_per_host": { 2023-10-06T02:27:37.044 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_count_per_host", 2023-10-06T02:27:37.044 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T02:27:37.044 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.044 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.044 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-10-06T02:27:37.045 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.045 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.045 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.045 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "max number of daemons per service per host", 2023-10-06T02:27:37.045 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.045 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.045 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.045 INFO:teuthology.orchestra.run.smithi031.stdout: "max_osd_draining_count": { 2023-10-06T02:27:37.045 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_osd_draining_count", 2023-10-06T02:27:37.045 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T02:27:37.046 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.046 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.046 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-10-06T02:27:37.046 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.046 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.046 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.046 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-10-06T02:27:37.046 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.046 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.047 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.047 INFO:teuthology.orchestra.run.smithi031.stdout: "migration_current": { 2023-10-06T02:27:37.047 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "migration_current", 2023-10-06T02:27:37.047 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T02:27:37.047 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.047 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.047 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.047 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.047 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.048 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.048 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "internal - do not modify", 2023-10-06T02:27:37.048 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.048 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.048 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.048 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": { 2023-10-06T02:27:37.048 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mode", 2023-10-06T02:27:37.048 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.048 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.049 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.049 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "root", 2023-10-06T02:27:37.049 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.049 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.049 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.049 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm-package", 2023-10-06T02:27:37.049 INFO:teuthology.orchestra.run.smithi031.stdout: "root" 2023-10-06T02:27:37.049 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.049 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "mode for remote execution of cephadm", 2023-10-06T02:27:37.049 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.050 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.050 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.050 INFO:teuthology.orchestra.run.smithi031.stdout: "prometheus_alerts_path": { 2023-10-06T02:27:37.050 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prometheus_alerts_path", 2023-10-06T02:27:37.050 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.050 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.050 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.050 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-10-06T02:27:37.051 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.051 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.051 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.051 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-10-06T02:27:37.051 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.051 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.051 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.051 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_insecure": { 2023-10-06T02:27:37.052 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_insecure", 2023-10-06T02:27:37.052 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.052 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.052 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.052 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.052 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.052 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.052 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.052 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-10-06T02:27:37.053 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.053 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.053 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.053 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_password": { 2023-10-06T02:27:37.053 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_password", 2023-10-06T02:27:37.053 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.053 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.053 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.053 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.054 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.054 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.054 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.054 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-10-06T02:27:37.054 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.054 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.054 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.054 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_url": { 2023-10-06T02:27:37.054 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_url", 2023-10-06T02:27:37.055 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.055 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.055 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.055 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.055 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.055 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.055 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.055 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-10-06T02:27:37.055 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.055 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.056 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.056 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_username": { 2023-10-06T02:27:37.056 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_username", 2023-10-06T02:27:37.056 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.056 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.056 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.056 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.056 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.056 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.056 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.057 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-10-06T02:27:37.057 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.057 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.057 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.057 INFO:teuthology.orchestra.run.smithi031.stdout: "ssh_config_file": { 2023-10-06T02:27:37.057 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssh_config_file", 2023-10-06T02:27:37.057 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.057 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.057 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.058 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.058 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.058 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.058 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.058 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-10-06T02:27:37.058 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.058 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.058 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.058 INFO:teuthology.orchestra.run.smithi031.stdout: "use_agent": { 2023-10-06T02:27:37.059 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "use_agent", 2023-10-06T02:27:37.059 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.059 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.059 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.059 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.059 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.059 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.059 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.059 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-10-06T02:27:37.059 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.059 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.060 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.060 INFO:teuthology.orchestra.run.smithi031.stdout: "use_repo_digest": { 2023-10-06T02:27:37.060 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "use_repo_digest", 2023-10-06T02:27:37.060 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.060 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.060 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.060 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T02:27:37.060 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.060 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.061 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.061 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-10-06T02:27:37.061 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.061 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.061 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.061 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_failed_host_check": { 2023-10-06T02:27:37.061 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_failed_host_check", 2023-10-06T02:27:37.061 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.062 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.062 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.062 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T02:27:37.062 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.062 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.062 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.062 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if the host check fails", 2023-10-06T02:27:37.062 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.062 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.062 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.063 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_stray_daemons": { 2023-10-06T02:27:37.063 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_stray_daemons", 2023-10-06T02:27:37.063 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.063 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.063 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.063 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T02:27:37.063 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.063 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.063 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.064 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-10-06T02:27:37.064 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.064 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.064 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.064 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_stray_hosts": { 2023-10-06T02:27:37.064 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_stray_hosts", 2023-10-06T02:27:37.064 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.064 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.065 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.065 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T02:27:37.065 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.065 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.065 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.065 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-10-06T02:27:37.065 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.065 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.065 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.065 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.066 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:37.066 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crash", 2023-10-06T02:27:37.066 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T02:27:37.066 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T02:27:37.066 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T02:27:37.066 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T02:27:37.066 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T02:27:37.066 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.066 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.067 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.067 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.067 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.067 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.067 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.067 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.067 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.067 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.067 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.067 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.067 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.067 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.068 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.068 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.068 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.068 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.068 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T02:27:37.068 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T02:27:37.068 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.068 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.068 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.068 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.069 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.069 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.069 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.069 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.069 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.069 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.069 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.069 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T02:27:37.069 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T02:27:37.069 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.069 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.070 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.070 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T02:27:37.070 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.070 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.070 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.070 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.070 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.070 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.070 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.070 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.070 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.071 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.071 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.071 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.071 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.071 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.071 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T02:27:37.071 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T02:27:37.071 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.071 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.071 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.071 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.072 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.072 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.072 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.072 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.072 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.072 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.072 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.072 INFO:teuthology.orchestra.run.smithi031.stdout: "retain_interval": { 2023-10-06T02:27:37.072 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "retain_interval", 2023-10-06T02:27:37.072 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T02:27:37.072 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.073 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.073 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "31536000", 2023-10-06T02:27:37.073 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.073 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.073 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.073 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long to retain crashes before pruning them", 2023-10-06T02:27:37.073 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.073 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.073 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.073 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_recent_interval": { 2023-10-06T02:27:37.073 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_recent_interval", 2023-10-06T02:27:37.074 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T02:27:37.074 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.074 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.074 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1209600", 2023-10-06T02:27:37.074 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.074 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.074 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.074 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "time interval in which to warn about recent crashes", 2023-10-06T02:27:37.074 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.074 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.074 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.074 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.075 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:37.075 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "dashboard", 2023-10-06T02:27:37.075 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T02:27:37.075 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T02:27:37.075 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T02:27:37.075 INFO:teuthology.orchestra.run.smithi031.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-10-06T02:27:37.076 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-10-06T02:27:37.076 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T02:27:37.076 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.076 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.076 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-10-06T02:27:37.077 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.077 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.077 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.077 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.077 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.077 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.077 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.077 INFO:teuthology.orchestra.run.smithi031.stdout: "ALERTMANAGER_API_HOST": { 2023-10-06T02:27:37.077 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ALERTMANAGER_API_HOST", 2023-10-06T02:27:37.077 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.077 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.078 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.078 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.078 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.078 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.078 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.078 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.078 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.078 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.078 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.078 INFO:teuthology.orchestra.run.smithi031.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-10-06T02:27:37.079 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-10-06T02:27:37.079 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.079 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.079 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.079 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T02:27:37.079 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.079 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.079 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.079 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.079 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.079 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.079 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.080 INFO:teuthology.orchestra.run.smithi031.stdout: "AUDIT_API_ENABLED": { 2023-10-06T02:27:37.080 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "AUDIT_API_ENABLED", 2023-10-06T02:27:37.080 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.080 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.080 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.080 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.080 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.080 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.080 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.080 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.081 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.081 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.081 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.081 INFO:teuthology.orchestra.run.smithi031.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-10-06T02:27:37.081 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-10-06T02:27:37.081 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.081 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.081 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.081 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T02:27:37.082 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.082 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.082 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.082 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.082 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.082 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.082 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.082 INFO:teuthology.orchestra.run.smithi031.stdout: "ENABLE_BROWSABLE_API": { 2023-10-06T02:27:37.082 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ENABLE_BROWSABLE_API", 2023-10-06T02:27:37.082 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.082 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.083 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.083 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T02:27:37.083 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.083 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.083 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.083 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.083 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.083 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.083 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.083 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-10-06T02:27:37.083 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-10-06T02:27:37.084 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.084 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.084 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.084 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T02:27:37.084 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.084 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.084 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.084 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.084 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.084 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.084 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.085 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-10-06T02:27:37.085 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-10-06T02:27:37.085 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.085 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.085 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.085 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T02:27:37.085 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.085 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.085 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.085 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.085 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.086 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.086 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.086 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-10-06T02:27:37.086 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-10-06T02:27:37.086 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.086 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.086 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.086 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T02:27:37.086 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.086 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.086 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.087 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.087 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.087 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.087 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.087 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-10-06T02:27:37.087 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-10-06T02:27:37.087 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.087 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.087 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.087 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T02:27:37.087 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.088 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.088 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.088 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.088 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.088 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.088 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.088 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_NFS": { 2023-10-06T02:27:37.088 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-10-06T02:27:37.088 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.088 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.088 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.089 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T02:27:37.089 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.089 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.089 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.089 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.089 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.089 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.089 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.089 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_RBD": { 2023-10-06T02:27:37.089 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-10-06T02:27:37.090 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.090 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.090 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.090 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T02:27:37.090 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.090 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.090 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.090 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.090 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.090 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.090 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.091 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_RGW": { 2023-10-06T02:27:37.091 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-10-06T02:27:37.091 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.091 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.091 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.091 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T02:27:37.091 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.091 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.091 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.091 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.091 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.092 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.092 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.092 INFO:teuthology.orchestra.run.smithi031.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-10-06T02:27:37.092 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-10-06T02:27:37.092 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.092 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.092 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.092 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.092 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.092 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.092 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.093 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.093 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.093 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.093 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.093 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_PASSWORD": { 2023-10-06T02:27:37.093 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_PASSWORD", 2023-10-06T02:27:37.093 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.093 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.093 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.093 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2023-10-06T02:27:37.093 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.093 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.094 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.094 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.094 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.094 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.094 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.094 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-10-06T02:27:37.094 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-10-06T02:27:37.094 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.094 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.094 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.095 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T02:27:37.095 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.095 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.095 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.095 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.095 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.095 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.095 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.095 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_URL": { 2023-10-06T02:27:37.095 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_URL", 2023-10-06T02:27:37.095 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.096 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.096 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.096 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.096 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.096 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.096 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.096 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.096 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.096 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.096 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.097 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_USERNAME": { 2023-10-06T02:27:37.097 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_USERNAME", 2023-10-06T02:27:37.097 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.097 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.097 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.097 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2023-10-06T02:27:37.097 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.097 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.097 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.097 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.097 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.097 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.098 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.098 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-10-06T02:27:37.098 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-10-06T02:27:37.098 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.098 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.098 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.098 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.098 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.098 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.098 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.098 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.099 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.099 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.099 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.099 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-10-06T02:27:37.099 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-10-06T02:27:37.099 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.099 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.099 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.099 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.099 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.100 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.100 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.100 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.100 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.100 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.100 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.100 INFO:teuthology.orchestra.run.smithi031.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-10-06T02:27:37.100 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-10-06T02:27:37.100 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.100 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.100 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.101 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T02:27:37.101 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.101 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.101 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.101 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.101 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.101 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.101 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.101 INFO:teuthology.orchestra.run.smithi031.stdout: "ISSUE_TRACKER_API_KEY": { 2023-10-06T02:27:37.101 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-10-06T02:27:37.101 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.102 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.102 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.102 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.102 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.102 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.102 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.102 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.102 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.102 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.102 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.102 INFO:teuthology.orchestra.run.smithi031.stdout: "PROMETHEUS_API_HOST": { 2023-10-06T02:27:37.103 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PROMETHEUS_API_HOST", 2023-10-06T02:27:37.103 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.103 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.103 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.103 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.103 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.103 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.103 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.103 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.103 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.103 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.103 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.104 INFO:teuthology.orchestra.run.smithi031.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-10-06T02:27:37.104 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-10-06T02:27:37.104 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.104 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.104 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.104 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T02:27:37.104 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.104 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.104 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.104 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.105 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.105 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.105 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.105 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-10-06T02:27:37.105 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-10-06T02:27:37.105 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.105 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.105 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.105 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.105 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.105 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.106 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.106 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.106 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.106 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.106 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.106 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-10-06T02:27:37.106 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-10-06T02:27:37.106 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.106 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.106 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.107 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.107 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.107 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.107 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.107 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.107 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.107 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.107 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.107 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-10-06T02:27:37.107 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-10-06T02:27:37.108 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.108 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.108 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.108 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T02:27:37.108 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.108 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.108 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.108 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.108 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.108 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.108 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.109 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-10-06T02:27:37.109 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-10-06T02:27:37.109 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.109 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.109 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.109 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T02:27:37.109 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.109 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.109 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.109 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.109 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.109 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.109 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.110 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-10-06T02:27:37.110 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-10-06T02:27:37.110 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.110 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.110 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.110 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.110 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.110 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.110 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.110 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.110 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.111 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.111 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.111 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-10-06T02:27:37.111 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-10-06T02:27:37.111 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.111 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.111 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.111 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.111 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.111 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.111 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.112 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.112 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.112 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.112 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.112 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-10-06T02:27:37.112 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-10-06T02:27:37.112 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.112 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.112 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.113 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.113 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.113 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.113 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.113 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.113 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.113 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.113 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.113 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_ENABLED": { 2023-10-06T02:27:37.113 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_ENABLED", 2023-10-06T02:27:37.114 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.114 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.114 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.114 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T02:27:37.114 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.114 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.114 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.114 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.114 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.114 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.114 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.115 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-10-06T02:27:37.115 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-10-06T02:27:37.115 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.115 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.115 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.115 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-10-06T02:27:37.115 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.115 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.115 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.115 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.115 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.116 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.116 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.116 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-10-06T02:27:37.116 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-10-06T02:27:37.116 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T02:27:37.116 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.116 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.116 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-10-06T02:27:37.116 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.116 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.117 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.117 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.117 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.117 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.117 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.117 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-10-06T02:27:37.117 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-10-06T02:27:37.117 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T02:27:37.117 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.117 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.117 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8", 2023-10-06T02:27:37.118 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.118 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.118 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.118 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.118 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.118 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.118 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.118 INFO:teuthology.orchestra.run.smithi031.stdout: "REST_REQUESTS_TIMEOUT": { 2023-10-06T02:27:37.118 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-10-06T02:27:37.119 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T02:27:37.119 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.119 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.119 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "45", 2023-10-06T02:27:37.119 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.119 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.119 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.119 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.119 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.119 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.119 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.120 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_ACCESS_KEY": { 2023-10-06T02:27:37.120 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_ACCESS_KEY", 2023-10-06T02:27:37.120 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.120 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.120 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.120 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.120 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.120 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.120 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.121 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.121 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.121 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.121 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.121 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-10-06T02:27:37.121 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-10-06T02:27:37.121 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.121 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.121 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.121 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2023-10-06T02:27:37.121 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.122 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.122 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.122 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.122 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.122 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.122 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.122 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_SECRET_KEY": { 2023-10-06T02:27:37.122 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_SECRET_KEY", 2023-10-06T02:27:37.122 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.123 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.123 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.123 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.123 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.123 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.123 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.123 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.123 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.123 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.123 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.123 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_SSL_VERIFY": { 2023-10-06T02:27:37.123 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_SSL_VERIFY", 2023-10-06T02:27:37.124 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.124 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.124 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.124 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T02:27:37.124 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.124 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.124 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.124 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.124 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.124 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.124 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.125 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-10-06T02:27:37.125 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-10-06T02:27:37.125 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T02:27:37.125 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.125 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.125 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2023-10-06T02:27:37.125 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.125 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.125 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.125 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.125 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.125 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.126 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.126 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-10-06T02:27:37.126 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-10-06T02:27:37.126 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T02:27:37.126 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.126 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.126 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-10-06T02:27:37.126 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.126 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.126 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.126 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.127 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.127 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.127 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.127 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-10-06T02:27:37.127 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-10-06T02:27:37.127 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T02:27:37.127 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.127 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.127 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2023-10-06T02:27:37.127 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.128 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.128 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.128 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.128 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.128 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.128 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.128 INFO:teuthology.orchestra.run.smithi031.stdout: "cross_origin_url": { 2023-10-06T02:27:37.128 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cross_origin_url", 2023-10-06T02:27:37.128 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.128 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.129 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.129 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.129 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.129 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.129 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.129 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.129 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.129 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.129 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.129 INFO:teuthology.orchestra.run.smithi031.stdout: "crt_file": { 2023-10-06T02:27:37.130 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crt_file", 2023-10-06T02:27:37.130 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.130 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.130 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.130 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.130 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.130 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.130 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.130 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.130 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.131 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.131 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.131 INFO:teuthology.orchestra.run.smithi031.stdout: "debug": { 2023-10-06T02:27:37.131 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "debug", 2023-10-06T02:27:37.131 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.131 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.131 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.131 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.131 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.131 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.132 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.132 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable/disable debug options", 2023-10-06T02:27:37.132 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.132 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.132 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.132 INFO:teuthology.orchestra.run.smithi031.stdout: "jwt_token_ttl": { 2023-10-06T02:27:37.132 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "jwt_token_ttl", 2023-10-06T02:27:37.132 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T02:27:37.132 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.132 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.132 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "28800", 2023-10-06T02:27:37.133 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.133 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.133 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.133 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.133 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.133 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.133 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.133 INFO:teuthology.orchestra.run.smithi031.stdout: "key_file": { 2023-10-06T02:27:37.133 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "key_file", 2023-10-06T02:27:37.133 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.133 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.134 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.134 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.134 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.134 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.134 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.134 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.134 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.134 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.134 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.134 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T02:27:37.134 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T02:27:37.134 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.135 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.135 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.135 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.135 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.135 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.135 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.135 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.135 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.135 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.135 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.135 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.136 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.136 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.136 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.136 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.136 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.136 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.136 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T02:27:37.136 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T02:27:37.136 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.136 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.136 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.137 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.137 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.137 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.137 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.137 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.137 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.137 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.137 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.137 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T02:27:37.137 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T02:27:37.137 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.138 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.138 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.138 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T02:27:37.138 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.138 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.138 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.138 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.138 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.138 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.138 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.138 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.139 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.139 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.139 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.139 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.139 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.139 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.139 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T02:27:37.139 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T02:27:37.139 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.139 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.139 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.140 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.140 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.140 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.140 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.140 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.140 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.140 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.140 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.140 INFO:teuthology.orchestra.run.smithi031.stdout: "motd": { 2023-10-06T02:27:37.140 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "motd", 2023-10-06T02:27:37.140 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.141 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.141 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.141 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.141 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.141 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.141 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.141 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "The message of the day", 2023-10-06T02:27:37.141 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.141 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.141 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.141 INFO:teuthology.orchestra.run.smithi031.stdout: "redirect_resolve_ip_addr": { 2023-10-06T02:27:37.142 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "redirect_resolve_ip_addr", 2023-10-06T02:27:37.142 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.142 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.142 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.142 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.142 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.142 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.142 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.142 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.142 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.142 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.142 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.143 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2023-10-06T02:27:37.143 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2023-10-06T02:27:37.143 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.143 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.143 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.143 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "::", 2023-10-06T02:27:37.143 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.143 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.143 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.143 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.144 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.144 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.144 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.144 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2023-10-06T02:27:37.144 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2023-10-06T02:27:37.144 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T02:27:37.144 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.144 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.144 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8080", 2023-10-06T02:27:37.144 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.144 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.144 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.145 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.145 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.145 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.145 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.145 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl": { 2023-10-06T02:27:37.145 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl", 2023-10-06T02:27:37.145 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.145 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.145 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.145 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T02:27:37.146 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.146 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.146 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.146 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.146 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.146 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.146 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.146 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl_server_port": { 2023-10-06T02:27:37.146 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl_server_port", 2023-10-06T02:27:37.146 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T02:27:37.146 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.147 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.147 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8443", 2023-10-06T02:27:37.147 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.147 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.147 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.147 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.147 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.147 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.147 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.147 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_behaviour": { 2023-10-06T02:27:37.147 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_behaviour", 2023-10-06T02:27:37.147 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.148 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.148 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.148 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "redirect", 2023-10-06T02:27:37.148 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.148 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.148 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.148 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.148 INFO:teuthology.orchestra.run.smithi031.stdout: "redirect" 2023-10-06T02:27:37.148 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.148 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.148 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.148 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.149 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.149 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_error_status_code": { 2023-10-06T02:27:37.149 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_error_status_code", 2023-10-06T02:27:37.149 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T02:27:37.149 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.149 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.149 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "500", 2023-10-06T02:27:37.149 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "400", 2023-10-06T02:27:37.149 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "599", 2023-10-06T02:27:37.149 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.149 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.149 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.150 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.150 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.150 INFO:teuthology.orchestra.run.smithi031.stdout: "url_prefix": { 2023-10-06T02:27:37.150 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "url_prefix", 2023-10-06T02:27:37.150 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.150 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.150 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.150 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.150 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.150 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.150 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.151 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.151 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.151 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.151 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.151 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.151 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:37.151 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "devicehealth", 2023-10-06T02:27:37.151 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T02:27:37.151 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T02:27:37.151 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T02:27:37.152 INFO:teuthology.orchestra.run.smithi031.stdout: "enable_monitoring": { 2023-10-06T02:27:37.152 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enable_monitoring", 2023-10-06T02:27:37.152 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.152 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.152 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.152 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T02:27:37.152 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.152 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.152 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.152 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "monitor device health metrics", 2023-10-06T02:27:37.152 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.152 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.153 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.153 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T02:27:37.153 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T02:27:37.153 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.153 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.153 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.153 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.153 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.153 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.153 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.153 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.153 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.154 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.154 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.154 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.154 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.154 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.154 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.154 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.154 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.154 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.154 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T02:27:37.155 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T02:27:37.155 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.155 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.155 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.155 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.155 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.155 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.155 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.155 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.155 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.155 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.155 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.156 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T02:27:37.156 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T02:27:37.156 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.156 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.156 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.156 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T02:27:37.156 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.156 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.156 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.156 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.156 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.157 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.157 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.157 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.157 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.157 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.157 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.157 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.157 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.157 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.157 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T02:27:37.157 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T02:27:37.157 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.158 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.158 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.158 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.158 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.158 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.158 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.158 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.158 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.158 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.158 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.159 INFO:teuthology.orchestra.run.smithi031.stdout: "mark_out_threshold": { 2023-10-06T02:27:37.159 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mark_out_threshold", 2023-10-06T02:27:37.159 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T02:27:37.159 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.159 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.159 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "2419200", 2023-10-06T02:27:37.159 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.159 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.159 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.159 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-10-06T02:27:37.159 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.159 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.159 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.160 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": { 2023-10-06T02:27:37.160 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pool_name", 2023-10-06T02:27:37.160 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.160 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.160 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.160 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "device_health_metrics", 2023-10-06T02:27:37.160 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.160 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.160 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.161 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "name of pool in which to store device health metrics", 2023-10-06T02:27:37.161 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.161 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.161 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.161 INFO:teuthology.orchestra.run.smithi031.stdout: "retention_period": { 2023-10-06T02:27:37.161 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "retention_period", 2023-10-06T02:27:37.161 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T02:27:37.161 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.161 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.161 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15552000", 2023-10-06T02:27:37.161 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.162 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.162 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.162 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long to retain device health metrics", 2023-10-06T02:27:37.162 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.162 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.162 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.162 INFO:teuthology.orchestra.run.smithi031.stdout: "scrape_frequency": { 2023-10-06T02:27:37.162 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "scrape_frequency", 2023-10-06T02:27:37.162 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T02:27:37.162 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.163 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.163 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "86400", 2023-10-06T02:27:37.163 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.163 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.163 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.163 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to scrape device health metrics", 2023-10-06T02:27:37.163 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.163 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.163 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.163 INFO:teuthology.orchestra.run.smithi031.stdout: "self_heal": { 2023-10-06T02:27:37.164 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "self_heal", 2023-10-06T02:27:37.164 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.164 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.164 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.164 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T02:27:37.164 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.164 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.164 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.164 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-10-06T02:27:37.164 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.164 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.165 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.165 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2023-10-06T02:27:37.165 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2023-10-06T02:27:37.165 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T02:27:37.165 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.165 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.165 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2023-10-06T02:27:37.165 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.165 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.165 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.165 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to wake up and check device health", 2023-10-06T02:27:37.165 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.166 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.166 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.166 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_threshold": { 2023-10-06T02:27:37.166 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_threshold", 2023-10-06T02:27:37.166 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T02:27:37.166 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.166 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.166 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "7257600", 2023-10-06T02:27:37.166 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.166 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.167 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.167 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-10-06T02:27:37.167 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.167 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.167 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.167 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.167 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:37.167 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "diskprediction_local", 2023-10-06T02:27:37.167 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T02:27:37.167 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T02:27:37.168 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T02:27:37.168 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T02:27:37.168 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T02:27:37.168 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.168 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.168 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.168 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.168 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.168 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.168 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.168 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.168 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.169 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.169 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.169 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.169 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.169 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.169 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.169 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.169 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.169 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.169 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T02:27:37.169 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T02:27:37.170 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.170 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.170 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.170 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.170 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.170 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.170 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.170 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.170 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.170 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.170 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.171 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T02:27:37.171 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T02:27:37.171 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.171 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.171 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.171 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T02:27:37.171 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.171 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.171 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.171 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.171 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.171 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.171 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.172 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.172 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.172 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.172 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.172 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.172 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.172 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.172 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T02:27:37.172 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T02:27:37.172 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.172 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.173 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.173 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.173 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.173 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.173 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.173 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.173 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.173 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.173 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.173 INFO:teuthology.orchestra.run.smithi031.stdout: "predict_interval": { 2023-10-06T02:27:37.173 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "predict_interval", 2023-10-06T02:27:37.174 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.174 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.174 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.174 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "86400", 2023-10-06T02:27:37.174 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.174 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.174 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.174 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.174 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.174 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.174 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.174 INFO:teuthology.orchestra.run.smithi031.stdout: "predictor_model": { 2023-10-06T02:27:37.175 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "predictor_model", 2023-10-06T02:27:37.175 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.175 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.175 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.175 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "prophetstor", 2023-10-06T02:27:37.175 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.175 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.175 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.175 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.175 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.175 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.175 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.176 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2023-10-06T02:27:37.176 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2023-10-06T02:27:37.176 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.176 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.176 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.176 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2023-10-06T02:27:37.176 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.176 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.176 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.176 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.176 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.177 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.177 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.177 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.177 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:37.177 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "influx", 2023-10-06T02:27:37.177 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": false, 2023-10-06T02:27:37.177 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "influxdb python module not found", 2023-10-06T02:27:37.177 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T02:27:37.177 INFO:teuthology.orchestra.run.smithi031.stdout: "batch_size": { 2023-10-06T02:27:37.178 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "batch_size", 2023-10-06T02:27:37.178 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T02:27:37.178 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.178 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.178 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5000", 2023-10-06T02:27:37.178 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.178 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.178 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.178 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-10-06T02:27:37.178 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.178 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.178 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.179 INFO:teuthology.orchestra.run.smithi031.stdout: "database": { 2023-10-06T02:27:37.179 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "database", 2023-10-06T02:27:37.179 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.179 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.179 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.179 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "ceph", 2023-10-06T02:27:37.179 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.179 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.179 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.179 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-10-06T02:27:37.179 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.180 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.180 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.180 INFO:teuthology.orchestra.run.smithi031.stdout: "hostname": { 2023-10-06T02:27:37.180 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "hostname", 2023-10-06T02:27:37.180 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.180 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.180 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.180 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.180 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.180 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.180 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.180 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB server hostname", 2023-10-06T02:27:37.181 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.181 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.181 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.181 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2023-10-06T02:27:37.181 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2023-10-06T02:27:37.181 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T02:27:37.181 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.181 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.181 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "30", 2023-10-06T02:27:37.181 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "5", 2023-10-06T02:27:37.182 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.182 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.182 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-10-06T02:27:37.182 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.182 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.182 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.182 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T02:27:37.182 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T02:27:37.182 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.182 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.182 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.182 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.183 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.183 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.183 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.183 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.183 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.183 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.183 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.183 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.183 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.183 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.184 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.184 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.185 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.185 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.185 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T02:27:37.185 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T02:27:37.185 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.185 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.185 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.185 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.185 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.185 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.185 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.186 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.186 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.186 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.186 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.186 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T02:27:37.186 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T02:27:37.186 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.186 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.186 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.186 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T02:27:37.186 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.187 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.187 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.187 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.187 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.187 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.187 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.187 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.187 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.187 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.187 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.187 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.187 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.188 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.188 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T02:27:37.188 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T02:27:37.188 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.188 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.188 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.188 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.188 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.188 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.188 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.188 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.189 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.189 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.189 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.189 INFO:teuthology.orchestra.run.smithi031.stdout: "password": { 2023-10-06T02:27:37.189 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "password", 2023-10-06T02:27:37.189 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.189 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.189 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.189 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.189 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.189 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.190 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.190 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "password of InfluxDB server user", 2023-10-06T02:27:37.190 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.190 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.190 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.190 INFO:teuthology.orchestra.run.smithi031.stdout: "port": { 2023-10-06T02:27:37.190 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "port", 2023-10-06T02:27:37.190 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T02:27:37.190 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.190 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.190 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8086", 2023-10-06T02:27:37.191 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.191 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.191 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.191 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB server port", 2023-10-06T02:27:37.191 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.191 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.191 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.191 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl": { 2023-10-06T02:27:37.191 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl", 2023-10-06T02:27:37.191 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.191 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.192 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.192 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "false", 2023-10-06T02:27:37.192 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.192 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.192 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.192 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-10-06T02:27:37.192 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.192 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.192 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.192 INFO:teuthology.orchestra.run.smithi031.stdout: "threads": { 2023-10-06T02:27:37.192 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "threads", 2023-10-06T02:27:37.192 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T02:27:37.193 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.193 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.193 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2023-10-06T02:27:37.193 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2023-10-06T02:27:37.193 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "32", 2023-10-06T02:27:37.193 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.193 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-10-06T02:27:37.193 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.193 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.193 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.193 INFO:teuthology.orchestra.run.smithi031.stdout: "username": { 2023-10-06T02:27:37.194 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "username", 2023-10-06T02:27:37.194 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.194 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.194 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.194 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.194 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.194 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.194 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.194 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "username of InfluxDB server user", 2023-10-06T02:27:37.194 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.194 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.194 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.195 INFO:teuthology.orchestra.run.smithi031.stdout: "verify_ssl": { 2023-10-06T02:27:37.195 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "verify_ssl", 2023-10-06T02:27:37.195 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.195 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.195 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.195 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "true", 2023-10-06T02:27:37.195 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.195 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.195 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.195 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-10-06T02:27:37.195 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.196 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.196 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.196 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.196 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:37.196 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "insights", 2023-10-06T02:27:37.196 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T02:27:37.196 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T02:27:37.196 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T02:27:37.196 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T02:27:37.196 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T02:27:37.197 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.197 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.197 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.197 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.197 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.197 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.197 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.197 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.197 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.197 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.197 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.198 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.198 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.198 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.198 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.198 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.198 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.198 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.198 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T02:27:37.198 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T02:27:37.198 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.198 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.198 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.199 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.199 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.199 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.199 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.199 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.199 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.199 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.199 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.199 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T02:27:37.199 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T02:27:37.199 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.200 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.200 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.200 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T02:27:37.200 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.200 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.200 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.200 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.200 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.200 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.200 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.200 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.200 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.201 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.201 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.201 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.201 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.201 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.201 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T02:27:37.201 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T02:27:37.201 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.201 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.201 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.201 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.201 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.202 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.202 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.202 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.202 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.202 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.202 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.202 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.202 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:37.203 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "iostat", 2023-10-06T02:27:37.203 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T02:27:37.203 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T02:27:37.203 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T02:27:37.203 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T02:27:37.203 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T02:27:37.203 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.203 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.203 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.203 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.203 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.203 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.204 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.204 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.204 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.204 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.204 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.204 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.204 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.204 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.204 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.204 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.204 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.204 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.205 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T02:27:37.205 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T02:27:37.205 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.205 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.205 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.205 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.205 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.205 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.205 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.205 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.205 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.206 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.206 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.206 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T02:27:37.206 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T02:27:37.206 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.206 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.206 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.206 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T02:27:37.206 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.206 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.206 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.207 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.207 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.207 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.207 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.207 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.207 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.207 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.207 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.207 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.207 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.207 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.208 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T02:27:37.208 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T02:27:37.208 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.208 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.208 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.208 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.208 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.208 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.208 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.208 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.208 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.208 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.209 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.209 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.209 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:37.209 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "localpool", 2023-10-06T02:27:37.209 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T02:27:37.209 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T02:27:37.209 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T02:27:37.209 INFO:teuthology.orchestra.run.smithi031.stdout: "failure_domain": { 2023-10-06T02:27:37.209 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "failure_domain", 2023-10-06T02:27:37.209 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.209 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.210 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.210 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "host", 2023-10-06T02:27:37.210 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.210 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.210 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.210 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "failure domain for any created local pool", 2023-10-06T02:27:37.210 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-10-06T02:27:37.210 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.210 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.210 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T02:27:37.210 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T02:27:37.211 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.211 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.211 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.211 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.211 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.211 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.211 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.211 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.211 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.211 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.211 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.212 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.212 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.212 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.212 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.212 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.212 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.212 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.212 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T02:27:37.212 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T02:27:37.212 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.213 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.213 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.213 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.213 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.213 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.213 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.213 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.213 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.213 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.213 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.213 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T02:27:37.214 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T02:27:37.214 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.214 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.214 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.214 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T02:27:37.214 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.214 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.214 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.214 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.214 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.214 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.215 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.215 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.215 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.215 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.215 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.215 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.215 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.215 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.215 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T02:27:37.215 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T02:27:37.216 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.216 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.216 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.216 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.216 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.216 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.216 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.216 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.216 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.216 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.216 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.216 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": { 2023-10-06T02:27:37.217 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "min_size", 2023-10-06T02:27:37.217 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T02:27:37.217 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.217 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.217 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.217 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.217 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.217 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.217 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default min_size for any created local pool", 2023-10-06T02:27:37.217 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-10-06T02:27:37.217 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.217 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.218 INFO:teuthology.orchestra.run.smithi031.stdout: "num_rep": { 2023-10-06T02:27:37.218 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "num_rep", 2023-10-06T02:27:37.218 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T02:27:37.218 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.218 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.218 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3", 2023-10-06T02:27:37.218 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.218 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.218 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.218 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default replica count for any created local pool", 2023-10-06T02:27:37.218 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.219 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.219 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.219 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": { 2023-10-06T02:27:37.219 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pg_num", 2023-10-06T02:27:37.219 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T02:27:37.219 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.219 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.219 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "128", 2023-10-06T02:27:37.219 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.219 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.219 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.220 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default pg_num for any created local pool", 2023-10-06T02:27:37.220 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.220 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.220 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.220 INFO:teuthology.orchestra.run.smithi031.stdout: "prefix": { 2023-10-06T02:27:37.220 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prefix", 2023-10-06T02:27:37.220 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.220 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.220 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.220 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.220 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.221 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.221 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.221 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "name prefix for any created local pool", 2023-10-06T02:27:37.221 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.221 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.221 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.221 INFO:teuthology.orchestra.run.smithi031.stdout: "subtree": { 2023-10-06T02:27:37.221 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "subtree", 2023-10-06T02:27:37.221 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.221 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.221 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.222 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "rack", 2023-10-06T02:27:37.222 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.222 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.222 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.222 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "CRUSH level for which to create a local pool", 2023-10-06T02:27:37.222 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-10-06T02:27:37.222 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.222 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.222 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.222 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:37.223 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mds_autoscaler", 2023-10-06T02:27:37.223 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T02:27:37.223 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T02:27:37.223 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T02:27:37.223 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T02:27:37.223 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T02:27:37.223 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.223 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.223 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.223 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.223 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.223 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.224 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.224 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.224 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.225 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.225 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.225 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.225 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.225 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.225 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.225 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.225 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.225 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.226 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T02:27:37.226 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T02:27:37.226 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.226 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.226 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.226 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.226 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.226 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.226 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.226 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.226 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.226 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.226 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.227 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T02:27:37.227 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T02:27:37.227 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.227 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.227 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.227 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T02:27:37.227 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.227 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.227 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.227 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.227 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.228 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.228 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.228 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.228 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.228 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.228 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.228 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.228 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.228 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.228 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T02:27:37.228 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T02:27:37.229 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.229 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.229 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.229 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.229 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.229 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.229 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.229 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.229 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.229 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.229 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.229 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.229 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:37.230 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mirroring", 2023-10-06T02:27:37.230 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T02:27:37.230 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T02:27:37.230 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T02:27:37.230 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T02:27:37.230 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T02:27:37.230 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.230 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.230 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.230 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.231 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.231 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.231 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.231 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.231 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.231 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.231 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.231 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.231 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.231 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.231 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.231 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.232 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.232 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.232 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T02:27:37.232 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T02:27:37.232 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.232 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.232 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.232 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.232 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.232 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.232 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.232 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.232 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.233 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.233 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.233 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T02:27:37.233 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T02:27:37.233 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.233 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.233 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.233 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T02:27:37.233 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.233 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.233 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.233 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.234 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.234 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.234 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.234 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.234 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.234 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.234 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.234 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.234 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.234 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.234 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T02:27:37.235 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T02:27:37.235 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.235 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.235 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.235 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.235 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.235 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.235 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.235 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.235 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.235 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.236 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.236 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.236 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:37.236 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "nfs", 2023-10-06T02:27:37.236 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T02:27:37.236 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T02:27:37.236 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T02:27:37.236 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T02:27:37.236 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T02:27:37.236 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.237 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.237 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.237 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.237 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.237 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.237 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.237 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.237 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.237 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.237 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.237 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.237 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.238 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.238 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.238 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.238 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.238 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.238 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T02:27:37.238 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T02:27:37.238 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.238 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.238 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.238 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.239 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.239 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.239 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.239 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.239 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.239 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.239 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.239 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T02:27:37.239 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T02:27:37.239 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.239 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.240 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.240 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T02:27:37.240 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.240 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.240 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.240 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.240 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.240 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.240 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.240 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.240 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.240 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.240 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.241 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.241 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.241 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.241 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T02:27:37.241 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T02:27:37.241 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.241 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.241 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.241 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.241 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.241 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.242 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.242 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.242 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.242 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.242 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.242 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.242 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:37.242 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "orchestrator", 2023-10-06T02:27:37.242 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T02:27:37.242 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T02:27:37.243 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T02:27:37.243 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T02:27:37.243 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T02:27:37.243 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.243 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.243 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.243 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.243 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.243 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.243 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.243 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.243 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.244 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.244 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.244 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.244 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.244 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.244 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.244 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.244 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.244 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.244 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T02:27:37.244 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T02:27:37.244 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.245 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.245 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.245 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.245 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.245 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.245 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.245 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.245 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.245 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.245 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.245 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T02:27:37.246 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T02:27:37.246 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.246 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.246 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.246 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T02:27:37.246 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.246 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.246 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.246 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.246 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.246 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.247 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.247 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.247 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.247 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.247 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.247 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.247 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.247 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.247 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T02:27:37.247 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T02:27:37.247 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.248 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.248 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.248 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.248 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.248 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.248 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.248 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.248 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.248 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.248 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.248 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator": { 2023-10-06T02:27:37.249 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "orchestrator", 2023-10-06T02:27:37.249 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.249 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.249 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.249 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.249 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.249 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.249 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.249 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm", 2023-10-06T02:27:37.249 INFO:teuthology.orchestra.run.smithi031.stdout: "rook", 2023-10-06T02:27:37.250 INFO:teuthology.orchestra.run.smithi031.stdout: "test_orchestrator" 2023-10-06T02:27:37.250 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.250 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Orchestrator backend", 2023-10-06T02:27:37.250 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.250 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.250 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.250 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.250 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:37.250 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "osd_perf_query", 2023-10-06T02:27:37.250 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T02:27:37.251 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T02:27:37.251 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T02:27:37.251 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T02:27:37.251 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T02:27:37.251 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.251 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.251 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.251 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.251 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.251 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.251 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.251 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.251 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.252 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.252 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.252 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.252 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.252 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.252 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.252 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.252 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.252 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.252 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T02:27:37.252 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T02:27:37.253 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.253 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.253 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.253 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.253 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.253 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.253 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.253 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.253 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.253 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.254 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.254 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T02:27:37.254 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T02:27:37.254 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.254 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.254 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.254 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T02:27:37.254 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.254 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.254 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.255 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.255 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.255 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.255 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.255 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.255 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.255 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.255 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.255 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.255 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.255 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.256 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T02:27:37.256 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T02:27:37.256 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.256 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.256 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.256 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.256 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.256 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.256 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.256 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.256 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.256 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.256 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.257 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.257 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:37.257 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "osd_support", 2023-10-06T02:27:37.257 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T02:27:37.257 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T02:27:37.257 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T02:27:37.257 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T02:27:37.257 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T02:27:37.257 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.257 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.257 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.258 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.258 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.258 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.258 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.258 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.258 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.258 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.258 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.258 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.258 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.258 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.258 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.259 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.259 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.259 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.259 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T02:27:37.259 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T02:27:37.259 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.259 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.259 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.259 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.259 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.260 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.260 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.260 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.260 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.260 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.260 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.260 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T02:27:37.260 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T02:27:37.260 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.260 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.260 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.260 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T02:27:37.261 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.261 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.261 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.261 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.261 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.261 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.261 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.261 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.261 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.261 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.261 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.261 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.262 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.262 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.262 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T02:27:37.262 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T02:27:37.262 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.262 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.262 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.262 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.262 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.262 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.262 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.263 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.263 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.263 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.263 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.263 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.263 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:37.263 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pg_autoscaler", 2023-10-06T02:27:37.263 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T02:27:37.263 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T02:27:37.263 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T02:27:37.263 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T02:27:37.264 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T02:27:37.264 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.264 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.264 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.264 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.264 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.264 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.264 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.264 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.264 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.264 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.264 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.265 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.265 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.265 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.265 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.265 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.265 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.265 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.265 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T02:27:37.265 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T02:27:37.265 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.265 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.265 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.266 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.266 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.266 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.266 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.266 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.266 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.266 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.266 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.266 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T02:27:37.266 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T02:27:37.266 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.267 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.267 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.267 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T02:27:37.267 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.267 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.267 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.267 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.267 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.267 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.267 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.267 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.267 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.268 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.268 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.268 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.268 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.268 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.268 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T02:27:37.268 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T02:27:37.268 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.268 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.268 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.269 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.269 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.269 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.269 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.269 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.269 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.269 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.269 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.269 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2023-10-06T02:27:37.269 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2023-10-06T02:27:37.270 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T02:27:37.270 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.270 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.270 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2023-10-06T02:27:37.270 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.270 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.270 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.270 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.270 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.270 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.270 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.270 INFO:teuthology.orchestra.run.smithi031.stdout: "threshold": { 2023-10-06T02:27:37.271 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "threshold", 2023-10-06T02:27:37.271 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-10-06T02:27:37.271 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.271 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.271 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3.0", 2023-10-06T02:27:37.271 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1.0", 2023-10-06T02:27:37.271 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.271 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.271 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "scaling threshold", 2023-10-06T02:27:37.271 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-10-06T02:27:37.271 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.271 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.271 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.272 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:37.272 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "progress", 2023-10-06T02:27:37.272 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T02:27:37.272 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T02:27:37.272 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T02:27:37.272 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_pg_recovery_event": { 2023-10-06T02:27:37.272 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_pg_recovery_event", 2023-10-06T02:27:37.272 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.272 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.272 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.273 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.273 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.273 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.273 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.273 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow the module to show pg recovery progress", 2023-10-06T02:27:37.273 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.273 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.273 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.273 INFO:teuthology.orchestra.run.smithi031.stdout: "enabled": { 2023-10-06T02:27:37.273 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enabled", 2023-10-06T02:27:37.273 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.273 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.273 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.274 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T02:27:37.274 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.274 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.274 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.274 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.274 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.274 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.274 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.274 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T02:27:37.274 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T02:27:37.274 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.275 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.275 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.275 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.275 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.275 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.275 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.275 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.275 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.275 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.275 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.275 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.275 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.275 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.276 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.276 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.276 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.276 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.276 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T02:27:37.276 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T02:27:37.276 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.276 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.276 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.276 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.276 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.277 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.277 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.277 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.277 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.277 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.277 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.277 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T02:27:37.277 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T02:27:37.277 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.277 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.278 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.278 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T02:27:37.278 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.278 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.278 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.278 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.278 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.278 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.278 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.278 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.278 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.278 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.278 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.279 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.279 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.279 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.279 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T02:27:37.279 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T02:27:37.279 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.279 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.279 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.279 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.279 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.279 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.280 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.280 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.280 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.280 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.280 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.280 INFO:teuthology.orchestra.run.smithi031.stdout: "max_completed_events": { 2023-10-06T02:27:37.280 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_completed_events", 2023-10-06T02:27:37.280 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T02:27:37.280 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.280 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.280 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "50", 2023-10-06T02:27:37.280 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.281 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.281 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.281 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "number of past completed events to remember", 2023-10-06T02:27:37.281 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.281 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.281 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.281 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2023-10-06T02:27:37.281 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2023-10-06T02:27:37.281 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T02:27:37.281 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.281 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.281 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2023-10-06T02:27:37.282 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.282 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.282 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.282 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long the module is going to sleep", 2023-10-06T02:27:37.282 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.282 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.282 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.282 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.282 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:37.282 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prometheus", 2023-10-06T02:27:37.283 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T02:27:37.283 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T02:27:37.283 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T02:27:37.283 INFO:teuthology.orchestra.run.smithi031.stdout: "cache": { 2023-10-06T02:27:37.283 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cache", 2023-10-06T02:27:37.283 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.283 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.283 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.283 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T02:27:37.283 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.283 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.283 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.283 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.284 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.284 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.284 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.284 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T02:27:37.284 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T02:27:37.284 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.284 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.284 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.284 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.284 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.284 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.284 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.285 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.285 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.285 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.285 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.285 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.285 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.285 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.285 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.285 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.285 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.285 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.285 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T02:27:37.286 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T02:27:37.286 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.286 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.286 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.286 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.286 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.286 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.286 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.286 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.286 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.286 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.286 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.287 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T02:27:37.287 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T02:27:37.287 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.287 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.287 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.287 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T02:27:37.287 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.287 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.287 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.287 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.287 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.287 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.288 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.288 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.288 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.288 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.288 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.288 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.288 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.288 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.288 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T02:27:37.288 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T02:27:37.288 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.288 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.289 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.289 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.289 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.289 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.289 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.289 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.289 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.289 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.289 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.290 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_stats_pools": { 2023-10-06T02:27:37.290 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_stats_pools", 2023-10-06T02:27:37.290 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.290 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.290 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.290 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.290 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.290 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.290 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.290 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.290 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.290 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.291 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.291 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_stats_pools_refresh_interval": { 2023-10-06T02:27:37.291 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-10-06T02:27:37.291 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T02:27:37.291 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.291 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.291 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "300", 2023-10-06T02:27:37.291 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.291 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.291 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.292 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.292 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.292 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.292 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.292 INFO:teuthology.orchestra.run.smithi031.stdout: "scrape_interval": { 2023-10-06T02:27:37.292 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "scrape_interval", 2023-10-06T02:27:37.292 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-10-06T02:27:37.292 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.292 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.292 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15.0", 2023-10-06T02:27:37.292 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.293 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.293 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.293 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.293 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.293 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.293 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.293 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2023-10-06T02:27:37.293 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2023-10-06T02:27:37.293 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.293 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.293 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.294 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "::", 2023-10-06T02:27:37.294 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.294 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.294 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.294 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-10-06T02:27:37.294 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.294 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.294 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.294 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2023-10-06T02:27:37.294 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2023-10-06T02:27:37.294 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T02:27:37.294 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.295 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.295 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "9283", 2023-10-06T02:27:37.295 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.295 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.295 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.295 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-10-06T02:27:37.295 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.295 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.295 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.295 INFO:teuthology.orchestra.run.smithi031.stdout: "stale_cache_strategy": { 2023-10-06T02:27:37.295 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "stale_cache_strategy", 2023-10-06T02:27:37.296 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.296 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.296 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.296 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "log", 2023-10-06T02:27:37.296 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.296 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.296 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.296 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.296 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.296 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.296 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.297 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_behaviour": { 2023-10-06T02:27:37.297 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_behaviour", 2023-10-06T02:27:37.297 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.297 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.297 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.297 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "default", 2023-10-06T02:27:37.297 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.297 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.297 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.297 INFO:teuthology.orchestra.run.smithi031.stdout: "default", 2023-10-06T02:27:37.297 INFO:teuthology.orchestra.run.smithi031.stdout: "error" 2023-10-06T02:27:37.297 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.298 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.298 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.298 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.298 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.298 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_error_status_code": { 2023-10-06T02:27:37.298 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_error_status_code", 2023-10-06T02:27:37.298 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T02:27:37.298 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.298 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.298 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "500", 2023-10-06T02:27:37.299 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "400", 2023-10-06T02:27:37.299 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "599", 2023-10-06T02:27:37.299 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.299 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.299 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.299 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.299 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.299 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.299 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.299 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:37.299 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_support", 2023-10-06T02:27:37.300 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T02:27:37.300 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T02:27:37.300 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T02:27:37.300 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T02:27:37.300 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T02:27:37.300 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.300 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.300 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.300 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.300 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.300 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.300 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.301 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.301 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.301 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.301 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.301 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.301 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.301 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.301 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.301 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.301 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.301 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.301 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T02:27:37.302 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T02:27:37.302 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.302 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.302 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.302 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.302 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.302 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.302 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.302 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.302 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.302 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.303 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.303 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T02:27:37.303 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T02:27:37.303 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.303 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.303 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.303 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T02:27:37.303 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.303 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.303 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.304 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.304 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.304 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.304 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.304 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.304 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.304 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.304 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.304 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.304 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.304 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.304 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T02:27:37.305 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T02:27:37.305 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.305 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.305 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.305 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.305 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.305 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.305 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.305 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.305 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.305 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.305 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.306 INFO:teuthology.orchestra.run.smithi031.stdout: "max_concurrent_snap_create": { 2023-10-06T02:27:37.306 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_concurrent_snap_create", 2023-10-06T02:27:37.306 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T02:27:37.306 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.306 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.306 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-10-06T02:27:37.306 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.306 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.306 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.306 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.306 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.306 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.307 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.307 INFO:teuthology.orchestra.run.smithi031.stdout: "mirror_snapshot_schedule": { 2023-10-06T02:27:37.307 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mirror_snapshot_schedule", 2023-10-06T02:27:37.307 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.307 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.307 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.307 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.307 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.307 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.307 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.307 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.308 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.308 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.308 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.308 INFO:teuthology.orchestra.run.smithi031.stdout: "trash_purge_schedule": { 2023-10-06T02:27:37.308 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "trash_purge_schedule", 2023-10-06T02:27:37.308 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.308 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.308 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.308 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.308 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.309 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.309 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.309 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.309 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.309 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.309 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.309 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.309 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:37.309 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "restful", 2023-10-06T02:27:37.309 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T02:27:37.310 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T02:27:37.310 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T02:27:37.310 INFO:teuthology.orchestra.run.smithi031.stdout: "enable_auth": { 2023-10-06T02:27:37.310 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enable_auth", 2023-10-06T02:27:37.310 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.310 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.310 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.310 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T02:27:37.310 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.310 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.310 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.310 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.311 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.311 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.311 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.311 INFO:teuthology.orchestra.run.smithi031.stdout: "key_file": { 2023-10-06T02:27:37.311 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "key_file", 2023-10-06T02:27:37.311 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.311 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.311 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.311 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.311 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.311 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.312 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.312 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.312 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.312 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.312 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.312 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T02:27:37.312 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T02:27:37.312 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.312 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.312 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.312 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.313 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.313 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.313 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.313 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.313 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.313 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.313 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.313 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.313 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.313 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.313 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.313 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.313 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.313 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.314 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T02:27:37.314 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T02:27:37.314 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.314 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.314 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.314 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.314 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.314 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.314 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.314 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.315 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.315 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.315 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.315 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T02:27:37.315 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T02:27:37.315 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.315 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.315 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.315 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T02:27:37.315 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.316 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.316 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.316 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.316 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.316 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.316 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.316 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.316 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.316 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.316 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.316 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.316 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.317 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.317 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T02:27:37.317 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T02:27:37.317 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.317 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.317 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.317 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.317 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.317 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.317 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.317 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.317 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.318 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.318 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.318 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2023-10-06T02:27:37.318 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2023-10-06T02:27:37.318 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.318 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.318 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.318 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.318 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.318 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.318 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.319 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.319 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.319 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.319 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.319 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2023-10-06T02:27:37.319 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2023-10-06T02:27:37.319 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.319 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.319 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.319 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.319 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.319 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.320 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.320 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.320 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.320 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.320 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.320 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.320 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:37.320 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rgw", 2023-10-06T02:27:37.320 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T02:27:37.320 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T02:27:37.321 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T02:27:37.321 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T02:27:37.321 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T02:27:37.321 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.321 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.321 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.321 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.321 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.321 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.321 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.321 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.321 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.321 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.322 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.322 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.322 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.322 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.322 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.322 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.322 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.322 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.322 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T02:27:37.322 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T02:27:37.323 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.323 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.323 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.323 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.323 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.323 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.323 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.323 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.323 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.323 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.323 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.324 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T02:27:37.324 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T02:27:37.324 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.324 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.324 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.324 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T02:27:37.324 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.324 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.324 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.325 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.325 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.325 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.325 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.325 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.325 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.325 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.325 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.325 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.325 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.325 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.326 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T02:27:37.326 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T02:27:37.326 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.326 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.326 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.326 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.326 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.326 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.326 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.326 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.326 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.327 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.327 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.327 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.327 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:37.327 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rook", 2023-10-06T02:27:37.327 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T02:27:37.327 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T02:27:37.327 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T02:27:37.327 INFO:teuthology.orchestra.run.smithi031.stdout: "drive_group_interval": { 2023-10-06T02:27:37.327 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "drive_group_interval", 2023-10-06T02:27:37.328 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-10-06T02:27:37.328 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.328 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.328 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "300.0", 2023-10-06T02:27:37.328 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.328 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.328 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.328 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-10-06T02:27:37.328 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.328 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.328 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.329 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T02:27:37.329 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T02:27:37.329 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.329 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.329 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.329 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.329 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.329 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.329 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.329 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.329 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.330 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.330 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.330 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.330 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.330 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.330 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.330 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.330 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.330 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.330 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T02:27:37.331 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T02:27:37.331 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.331 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.331 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.331 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.331 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.331 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.331 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.331 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.331 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.331 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.331 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.332 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T02:27:37.332 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T02:27:37.332 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.332 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.332 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.332 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T02:27:37.332 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.332 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.332 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.332 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.332 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.332 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.333 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.333 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.333 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.333 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.333 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.333 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.333 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.333 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.333 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T02:27:37.333 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T02:27:37.333 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.334 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.334 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.334 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.334 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.334 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.334 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.334 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.334 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.334 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.334 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.334 INFO:teuthology.orchestra.run.smithi031.stdout: "storage_class": { 2023-10-06T02:27:37.335 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "storage_class", 2023-10-06T02:27:37.335 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.335 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.335 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.335 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "local", 2023-10-06T02:27:37.335 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.335 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.335 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.335 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-10-06T02:27:37.335 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.335 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.335 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.336 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.336 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:37.336 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "selftest", 2023-10-06T02:27:37.336 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T02:27:37.336 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T02:27:37.336 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T02:27:37.336 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T02:27:37.336 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T02:27:37.336 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.337 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.337 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.337 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.337 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.337 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.337 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.337 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.337 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.337 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.337 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.337 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.337 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.338 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.338 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.338 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.338 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.338 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.338 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T02:27:37.338 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T02:27:37.338 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.338 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.338 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.339 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.339 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.339 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.339 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.339 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.339 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.339 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.339 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.339 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T02:27:37.339 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T02:27:37.339 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.340 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.340 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.340 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T02:27:37.340 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.340 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.340 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.340 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.340 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.340 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.340 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.340 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.341 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.341 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.341 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.341 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.341 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.341 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.341 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T02:27:37.341 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T02:27:37.341 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.341 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.342 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.342 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.342 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.342 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.342 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.342 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.342 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.342 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.342 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.342 INFO:teuthology.orchestra.run.smithi031.stdout: "roption1": { 2023-10-06T02:27:37.342 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "roption1", 2023-10-06T02:27:37.342 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.343 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.343 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.343 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.343 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.343 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.343 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.343 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.343 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.343 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.343 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.343 INFO:teuthology.orchestra.run.smithi031.stdout: "roption2": { 2023-10-06T02:27:37.344 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "roption2", 2023-10-06T02:27:37.344 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.344 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.344 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.344 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "xyz", 2023-10-06T02:27:37.344 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.344 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.344 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.344 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.344 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.344 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.344 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.345 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption1": { 2023-10-06T02:27:37.345 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption1", 2023-10-06T02:27:37.345 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.345 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.345 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.345 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.345 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.345 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.345 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.345 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.345 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.345 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.345 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.346 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption2": { 2023-10-06T02:27:37.346 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption2", 2023-10-06T02:27:37.346 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T02:27:37.346 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.346 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.346 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.346 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.346 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.346 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.346 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.346 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.346 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.347 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.347 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption3": { 2023-10-06T02:27:37.347 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption3", 2023-10-06T02:27:37.347 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-10-06T02:27:37.347 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.347 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.347 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.347 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.347 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.347 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.347 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.348 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.348 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.348 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.348 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption4": { 2023-10-06T02:27:37.348 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption4", 2023-10-06T02:27:37.348 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.348 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.348 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.348 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.348 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.348 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.348 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.349 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.349 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.349 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.349 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.349 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption5": { 2023-10-06T02:27:37.349 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption5", 2023-10-06T02:27:37.349 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.349 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.349 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.349 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.349 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.349 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.350 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.350 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.350 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.350 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.350 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.350 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption6": { 2023-10-06T02:27:37.350 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption6", 2023-10-06T02:27:37.350 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.350 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.350 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.350 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T02:27:37.351 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.351 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.351 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.351 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.351 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.351 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.351 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.351 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption7": { 2023-10-06T02:27:37.351 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption7", 2023-10-06T02:27:37.351 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T02:27:37.352 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.352 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.352 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.352 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2023-10-06T02:27:37.352 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "42", 2023-10-06T02:27:37.352 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.352 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.352 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.352 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.352 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.352 INFO:teuthology.orchestra.run.smithi031.stdout: "testkey": { 2023-10-06T02:27:37.353 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testkey", 2023-10-06T02:27:37.353 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.353 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.353 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.353 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.353 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.353 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.353 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.353 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.353 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.353 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.354 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.354 INFO:teuthology.orchestra.run.smithi031.stdout: "testlkey": { 2023-10-06T02:27:37.354 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testlkey", 2023-10-06T02:27:37.354 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.354 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.354 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.354 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.354 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.354 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.354 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.354 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.354 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.355 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.355 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.355 INFO:teuthology.orchestra.run.smithi031.stdout: "testnewline": { 2023-10-06T02:27:37.355 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testnewline", 2023-10-06T02:27:37.355 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.355 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.355 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.355 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.355 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.355 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.355 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.356 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.356 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.356 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.356 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.356 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.356 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:37.356 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snap_schedule", 2023-10-06T02:27:37.356 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T02:27:37.356 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T02:27:37.356 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T02:27:37.356 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_m_granularity": { 2023-10-06T02:27:37.357 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_m_granularity", 2023-10-06T02:27:37.357 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.357 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.357 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.357 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.357 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.357 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.357 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.357 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow minute scheduled snapshots", 2023-10-06T02:27:37.357 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.357 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.358 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.358 INFO:teuthology.orchestra.run.smithi031.stdout: "dump_on_update": { 2023-10-06T02:27:37.358 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "dump_on_update", 2023-10-06T02:27:37.358 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.358 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.358 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.358 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.358 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.358 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.358 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.358 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "dump database to debug log on update", 2023-10-06T02:27:37.359 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.359 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.359 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.359 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T02:27:37.359 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T02:27:37.359 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.359 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.359 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.359 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.359 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.360 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.360 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.360 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.360 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.360 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.360 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.360 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.360 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.360 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.360 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.360 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.360 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.360 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.361 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T02:27:37.361 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T02:27:37.361 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.361 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.361 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.361 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.361 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.361 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.361 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.361 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.361 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.362 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.362 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.362 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T02:27:37.362 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T02:27:37.362 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.362 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.362 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.362 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T02:27:37.362 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.363 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.363 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.363 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.363 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.363 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.363 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.363 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.363 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.363 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.363 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.363 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.363 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.363 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.364 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T02:27:37.364 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T02:27:37.364 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.364 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.364 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.364 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.364 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.364 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.364 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.364 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.364 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.364 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.365 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.365 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.365 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:37.365 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "stats", 2023-10-06T02:27:37.365 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T02:27:37.365 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T02:27:37.365 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T02:27:37.365 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T02:27:37.365 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T02:27:37.365 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.366 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.366 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.366 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.366 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.366 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.366 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.366 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.366 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.366 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.366 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.366 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.366 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.367 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.367 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.367 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.367 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.367 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.367 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T02:27:37.367 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T02:27:37.367 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.367 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.367 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.367 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.368 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.368 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.368 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.368 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.368 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.368 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.368 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.368 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T02:27:37.368 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T02:27:37.368 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.369 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.369 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.369 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T02:27:37.369 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.369 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.369 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.369 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.369 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.369 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.369 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.369 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.369 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.370 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.370 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.370 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.370 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.370 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.370 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T02:27:37.370 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T02:27:37.370 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.370 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.370 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.371 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.371 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.371 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.371 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.371 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.371 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.371 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.371 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.371 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.371 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:37.372 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "status", 2023-10-06T02:27:37.372 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T02:27:37.372 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T02:27:37.372 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T02:27:37.372 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T02:27:37.372 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T02:27:37.372 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.372 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.372 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.372 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.372 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.373 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.373 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.373 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.373 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.373 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.373 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.373 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.373 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.373 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.373 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.373 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.373 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.374 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.374 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T02:27:37.374 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T02:27:37.374 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.374 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.374 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.374 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.374 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.374 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.375 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.375 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.375 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.375 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.375 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.375 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T02:27:37.375 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T02:27:37.375 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.375 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.375 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.376 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T02:27:37.376 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.376 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.376 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.376 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.376 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.376 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.376 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.376 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.376 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.376 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.376 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.377 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.377 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.377 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.377 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T02:27:37.377 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T02:27:37.377 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.377 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.377 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.377 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.377 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.377 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.377 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.378 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.378 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.378 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.378 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.378 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.378 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:37.378 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "telegraf", 2023-10-06T02:27:37.378 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T02:27:37.378 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T02:27:37.379 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T02:27:37.379 INFO:teuthology.orchestra.run.smithi031.stdout: "address": { 2023-10-06T02:27:37.379 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "address", 2023-10-06T02:27:37.379 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.379 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.379 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.379 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-10-06T02:27:37.379 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.379 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.379 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.379 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.379 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.379 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.380 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.380 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2023-10-06T02:27:37.380 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2023-10-06T02:27:37.380 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T02:27:37.380 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.380 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.380 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15", 2023-10-06T02:27:37.380 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.380 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.380 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.380 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.381 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.381 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.381 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.381 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T02:27:37.381 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T02:27:37.381 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.381 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.381 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.381 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.381 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.381 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.381 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.382 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.382 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.382 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.382 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.382 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.382 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.382 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.382 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.382 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.382 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.382 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.383 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T02:27:37.383 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T02:27:37.383 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.383 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.383 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.383 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.383 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.383 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.383 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.383 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.383 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.383 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.384 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.384 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T02:27:37.384 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T02:27:37.384 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.384 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.384 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.384 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T02:27:37.384 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.384 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.384 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.384 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.384 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.385 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.385 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.385 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.385 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.385 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.385 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.385 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.385 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.385 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.385 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T02:27:37.386 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T02:27:37.386 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.386 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.386 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.386 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.386 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.386 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.386 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.386 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.386 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.386 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.386 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.387 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.387 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:37.387 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "telemetry", 2023-10-06T02:27:37.387 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T02:27:37.387 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T02:27:37.387 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T02:27:37.387 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_basic": { 2023-10-06T02:27:37.387 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_basic", 2023-10-06T02:27:37.387 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.387 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.387 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.388 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T02:27:37.388 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.388 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.388 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.388 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share basic cluster information (size, version)", 2023-10-06T02:27:37.388 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.388 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.388 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.388 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_crash": { 2023-10-06T02:27:37.388 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_crash", 2023-10-06T02:27:37.389 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.389 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.389 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.389 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T02:27:37.389 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.389 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.389 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.389 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-10-06T02:27:37.389 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.389 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.389 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.390 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_device": { 2023-10-06T02:27:37.390 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_device", 2023-10-06T02:27:37.390 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.390 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.390 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.390 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T02:27:37.390 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.390 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.390 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.390 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-10-06T02:27:37.390 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.391 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.391 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.391 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_ident": { 2023-10-06T02:27:37.391 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_ident", 2023-10-06T02:27:37.391 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.391 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.391 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.391 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.391 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.391 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.391 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.392 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-10-06T02:27:37.392 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.392 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.392 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.392 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_perf": { 2023-10-06T02:27:37.392 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_perf", 2023-10-06T02:27:37.392 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.392 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.392 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.392 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.393 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.393 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.393 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.393 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share various performance metrics of a cluster", 2023-10-06T02:27:37.393 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.393 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.393 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.393 INFO:teuthology.orchestra.run.smithi031.stdout: "contact": { 2023-10-06T02:27:37.393 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "contact", 2023-10-06T02:27:37.393 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.394 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.394 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.394 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.394 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.394 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.394 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.394 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.394 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.394 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.394 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.394 INFO:teuthology.orchestra.run.smithi031.stdout: "description": { 2023-10-06T02:27:37.395 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "description", 2023-10-06T02:27:37.395 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.395 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.395 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.395 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.395 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.395 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.395 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.395 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.395 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.395 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.395 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.396 INFO:teuthology.orchestra.run.smithi031.stdout: "device_url": { 2023-10-06T02:27:37.396 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_url", 2023-10-06T02:27:37.396 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.396 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.396 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.396 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-10-06T02:27:37.396 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.396 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.396 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.396 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.396 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.396 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.397 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.397 INFO:teuthology.orchestra.run.smithi031.stdout: "enabled": { 2023-10-06T02:27:37.397 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enabled", 2023-10-06T02:27:37.397 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.397 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.397 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.397 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.397 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.397 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.397 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.397 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.398 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.398 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.398 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.398 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2023-10-06T02:27:37.398 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2023-10-06T02:27:37.398 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T02:27:37.398 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.398 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.398 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "24", 2023-10-06T02:27:37.398 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "8", 2023-10-06T02:27:37.398 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.399 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.399 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.399 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.399 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.399 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.399 INFO:teuthology.orchestra.run.smithi031.stdout: "last_opt_revision": { 2023-10-06T02:27:37.399 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "last_opt_revision", 2023-10-06T02:27:37.399 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T02:27:37.399 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.399 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.399 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1", 2023-10-06T02:27:37.400 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.400 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.400 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.400 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.400 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.400 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.400 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.400 INFO:teuthology.orchestra.run.smithi031.stdout: "leaderboard": { 2023-10-06T02:27:37.400 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "leaderboard", 2023-10-06T02:27:37.400 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.400 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.400 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.401 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.401 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.401 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.401 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.401 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.401 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.401 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.401 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.401 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T02:27:37.401 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T02:27:37.401 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.402 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.402 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.402 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.402 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.402 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.402 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.402 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.402 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.402 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.402 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.402 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.403 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.403 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.403 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.403 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.403 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.403 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.403 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T02:27:37.403 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T02:27:37.403 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.403 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.403 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.404 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.404 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.404 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.404 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.404 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.404 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.404 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.404 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.404 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T02:27:37.404 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T02:27:37.404 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.404 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.405 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.405 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T02:27:37.405 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.405 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.405 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.405 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.405 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.405 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.405 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.405 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.405 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.405 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.405 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.406 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.406 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.406 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.406 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T02:27:37.406 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T02:27:37.406 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.406 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.406 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.406 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.406 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.406 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.407 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.407 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.407 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.407 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.407 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.407 INFO:teuthology.orchestra.run.smithi031.stdout: "organization": { 2023-10-06T02:27:37.407 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "organization", 2023-10-06T02:27:37.407 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.407 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.407 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.407 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.408 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.408 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.408 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.408 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.408 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.408 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.408 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.408 INFO:teuthology.orchestra.run.smithi031.stdout: "proxy": { 2023-10-06T02:27:37.408 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "proxy", 2023-10-06T02:27:37.408 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.408 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.409 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.409 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.409 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.409 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.409 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.409 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.409 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.409 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.409 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.409 INFO:teuthology.orchestra.run.smithi031.stdout: "url": { 2023-10-06T02:27:37.409 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "url", 2023-10-06T02:27:37.409 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.409 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.410 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.410 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-10-06T02:27:37.410 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.410 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.410 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.410 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.410 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.410 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.410 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.410 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.410 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:37.411 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "test_orchestrator", 2023-10-06T02:27:37.411 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T02:27:37.411 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T02:27:37.411 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T02:27:37.411 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T02:27:37.411 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T02:27:37.411 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.411 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.411 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.411 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.411 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.412 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.412 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.412 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.412 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.412 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.412 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.412 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.412 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.412 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.412 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.412 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.412 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.413 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.413 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T02:27:37.413 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T02:27:37.413 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.413 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.413 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.413 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.413 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.413 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.413 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.413 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.414 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.414 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.414 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.414 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T02:27:37.414 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T02:27:37.414 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.414 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.414 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.414 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T02:27:37.414 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.414 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.415 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.415 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.415 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.415 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.415 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.415 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.415 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.415 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.415 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.415 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.415 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.416 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.416 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T02:27:37.416 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T02:27:37.416 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.416 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.416 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.416 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.416 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.416 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.416 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.416 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.417 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.417 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.417 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.417 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.417 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:37.417 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "volumes", 2023-10-06T02:27:37.417 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T02:27:37.417 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T02:27:37.417 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T02:27:37.417 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T02:27:37.418 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T02:27:37.418 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.418 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.418 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.418 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.418 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.418 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.418 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.418 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.418 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.418 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.418 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.419 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.419 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.419 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.419 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.419 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.419 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.419 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.419 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T02:27:37.419 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T02:27:37.419 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.420 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.420 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.420 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.420 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.420 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.420 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.420 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.420 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.420 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.420 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.421 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T02:27:37.421 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T02:27:37.421 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.421 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.421 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.421 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T02:27:37.421 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.421 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.421 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.421 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.421 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.422 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.422 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.422 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.422 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.422 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.422 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.422 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.422 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.422 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.425 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T02:27:37.426 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T02:27:37.426 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.426 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.426 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.426 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.426 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.426 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.426 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.426 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.426 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.426 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.426 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.427 INFO:teuthology.orchestra.run.smithi031.stdout: "max_concurrent_clones": { 2023-10-06T02:27:37.427 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_concurrent_clones", 2023-10-06T02:27:37.427 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T02:27:37.427 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.427 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.427 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "4", 2023-10-06T02:27:37.427 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.427 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.427 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.427 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Number of asynchronous cloner threads", 2023-10-06T02:27:37.427 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.428 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.428 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.428 INFO:teuthology.orchestra.run.smithi031.stdout: "snapshot_clone_delay": { 2023-10-06T02:27:37.428 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snapshot_clone_delay", 2023-10-06T02:27:37.428 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T02:27:37.428 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.428 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.428 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2023-10-06T02:27:37.428 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.428 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.428 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.429 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-10-06T02:27:37.429 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.429 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.429 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.429 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.429 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:37.429 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix", 2023-10-06T02:27:37.429 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T02:27:37.429 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T02:27:37.429 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T02:27:37.430 INFO:teuthology.orchestra.run.smithi031.stdout: "discovery_interval": { 2023-10-06T02:27:37.430 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "discovery_interval", 2023-10-06T02:27:37.430 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2023-10-06T02:27:37.430 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.430 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.430 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "100", 2023-10-06T02:27:37.430 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.430 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.430 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.430 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.430 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.431 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.431 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.431 INFO:teuthology.orchestra.run.smithi031.stdout: "identifier": { 2023-10-06T02:27:37.431 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "identifier", 2023-10-06T02:27:37.431 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.431 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.431 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.431 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.431 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.431 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.431 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.432 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.432 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.432 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.432 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.432 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2023-10-06T02:27:37.432 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2023-10-06T02:27:37.432 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T02:27:37.432 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.432 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.432 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2023-10-06T02:27:37.432 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.433 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.433 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.433 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.433 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.433 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.433 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.433 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T02:27:37.433 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T02:27:37.433 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.433 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.433 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.433 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.434 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.434 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.434 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.434 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.434 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.434 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.434 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.434 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.434 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.434 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.434 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.434 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.435 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.435 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.435 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T02:27:37.435 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T02:27:37.435 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.435 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.435 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.435 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.435 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.435 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.435 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.436 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.436 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.436 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.436 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.436 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T02:27:37.436 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T02:27:37.436 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.436 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.436 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.436 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T02:27:37.436 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.437 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.437 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.437 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.437 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.437 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.437 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.437 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.437 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.437 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.437 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.437 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.437 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.438 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.438 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T02:27:37.438 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T02:27:37.438 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.438 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.438 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.438 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.438 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.438 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.438 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.438 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.439 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.439 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.439 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.439 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_host": { 2023-10-06T02:27:37.439 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_host", 2023-10-06T02:27:37.439 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.439 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.439 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.439 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.439 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.439 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.440 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.440 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.440 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.440 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.440 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.440 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_port": { 2023-10-06T02:27:37.440 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_port", 2023-10-06T02:27:37.440 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T02:27:37.440 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.440 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.440 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10051", 2023-10-06T02:27:37.440 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.441 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.441 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.441 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.441 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.441 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.441 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.441 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_sender": { 2023-10-06T02:27:37.441 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_sender", 2023-10-06T02:27:37.441 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.441 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.441 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.442 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-10-06T02:27:37.442 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.442 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.442 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.442 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.442 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.442 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.442 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.442 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.442 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.442 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.442 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-10-06T02:27:37.443 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.443 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.443 INFO:teuthology.orchestra.run.smithi031.stdout: "modules": [ 2023-10-06T02:27:37.443 INFO:teuthology.orchestra.run.smithi031.stdout: "iostat", 2023-10-06T02:27:37.443 INFO:teuthology.orchestra.run.smithi031.stdout: "nfs", 2023-10-06T02:27:37.443 INFO:teuthology.orchestra.run.smithi031.stdout: "restful" 2023-10-06T02:27:37.443 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.443 INFO:teuthology.orchestra.run.smithi031.stdout: "available_modules": [ 2023-10-06T02:27:37.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:37.443 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "alerts", 2023-10-06T02:27:37.443 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T02:27:37.443 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T02:27:37.444 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T02:27:37.444 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2023-10-06T02:27:37.444 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2023-10-06T02:27:37.444 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T02:27:37.444 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.444 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.444 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2023-10-06T02:27:37.444 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.444 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.444 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.444 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How frequently to reexamine health status", 2023-10-06T02:27:37.445 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.445 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.445 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.445 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T02:27:37.445 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T02:27:37.445 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.445 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.445 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.445 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.445 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.445 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.446 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.446 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.446 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.446 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.446 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.446 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.446 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.446 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.446 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.446 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.446 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.446 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.447 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T02:27:37.447 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T02:27:37.447 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.447 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.447 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.447 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.447 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.447 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.447 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.447 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.447 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.447 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.448 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.448 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T02:27:37.448 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T02:27:37.448 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.448 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.448 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.448 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T02:27:37.448 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.448 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.448 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.448 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.448 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.449 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.449 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.449 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.449 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.449 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.449 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.449 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.449 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.449 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.449 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T02:27:37.449 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T02:27:37.449 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.450 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.450 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.450 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.450 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.450 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.450 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.450 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.450 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.450 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.450 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.450 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_destination": { 2023-10-06T02:27:37.451 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_destination", 2023-10-06T02:27:37.451 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.451 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.451 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.451 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.451 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.451 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.451 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.451 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Email address to send alerts to", 2023-10-06T02:27:37.451 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.451 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.451 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.452 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_from_name": { 2023-10-06T02:27:37.452 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_from_name", 2023-10-06T02:27:37.452 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.452 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.452 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.452 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "Ceph", 2023-10-06T02:27:37.452 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.452 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.452 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.452 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Email From: name", 2023-10-06T02:27:37.452 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.452 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.453 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.453 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_host": { 2023-10-06T02:27:37.453 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_host", 2023-10-06T02:27:37.453 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.453 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.453 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.453 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.453 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.453 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.453 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.453 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP server", 2023-10-06T02:27:37.454 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.454 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.454 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.454 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_password": { 2023-10-06T02:27:37.454 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_password", 2023-10-06T02:27:37.454 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.454 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.454 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.454 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.454 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.454 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.455 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.455 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Password to authenticate with", 2023-10-06T02:27:37.455 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.455 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.455 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.455 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_port": { 2023-10-06T02:27:37.455 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_port", 2023-10-06T02:27:37.455 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T02:27:37.455 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.455 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.455 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "465", 2023-10-06T02:27:37.456 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.456 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.456 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.456 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP port", 2023-10-06T02:27:37.456 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.456 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.456 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.456 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_sender": { 2023-10-06T02:27:37.456 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_sender", 2023-10-06T02:27:37.456 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.456 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.457 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.457 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.457 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.457 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.457 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.457 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP envelope sender", 2023-10-06T02:27:37.457 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.457 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.457 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.457 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_ssl": { 2023-10-06T02:27:37.457 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_ssl", 2023-10-06T02:27:37.458 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.458 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.458 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.458 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T02:27:37.458 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.458 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.458 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.458 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use SSL to connect to SMTP server", 2023-10-06T02:27:37.458 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.458 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.458 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.459 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_user": { 2023-10-06T02:27:37.459 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_user", 2023-10-06T02:27:37.459 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.459 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.459 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.459 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.459 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.459 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.459 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.459 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "User to authenticate as", 2023-10-06T02:27:37.459 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.459 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.460 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.460 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.460 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:37.460 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "balancer", 2023-10-06T02:27:37.460 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T02:27:37.460 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T02:27:37.460 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T02:27:37.460 INFO:teuthology.orchestra.run.smithi031.stdout: "active": { 2023-10-06T02:27:37.460 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "active", 2023-10-06T02:27:37.460 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.461 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.461 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.461 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T02:27:37.461 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.461 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.461 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.461 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "automatically balance PGs across cluster", 2023-10-06T02:27:37.461 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.461 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.461 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.462 INFO:teuthology.orchestra.run.smithi031.stdout: "begin_time": { 2023-10-06T02:27:37.462 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "begin_time", 2023-10-06T02:27:37.462 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.462 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.462 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.462 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0000", 2023-10-06T02:27:37.462 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.462 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.462 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.462 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "beginning time of day to automatically balance", 2023-10-06T02:27:37.462 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-10-06T02:27:37.462 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.462 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.463 INFO:teuthology.orchestra.run.smithi031.stdout: "begin_weekday": { 2023-10-06T02:27:37.463 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "begin_weekday", 2023-10-06T02:27:37.463 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2023-10-06T02:27:37.463 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.463 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.463 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2023-10-06T02:27:37.463 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0", 2023-10-06T02:27:37.463 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "7", 2023-10-06T02:27:37.463 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.463 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-10-06T02:27:37.464 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "0 or 7 = Sunday, 1 = Monday, etc.", 2023-10-06T02:27:37.464 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.464 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.464 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_max_iterations": { 2023-10-06T02:27:37.464 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_max_iterations", 2023-10-06T02:27:37.464 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2023-10-06T02:27:37.464 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.464 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.464 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "25", 2023-10-06T02:27:37.464 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2023-10-06T02:27:37.465 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "250", 2023-10-06T02:27:37.465 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.465 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-10-06T02:27:37.465 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.465 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.465 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.465 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_metrics": { 2023-10-06T02:27:37.465 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_metrics", 2023-10-06T02:27:37.465 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.465 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.465 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.466 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "pgs,objects,bytes", 2023-10-06T02:27:37.466 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.466 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.466 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.466 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-10-06T02:27:37.466 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-10-06T02:27:37.466 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.466 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.466 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_step": { 2023-10-06T02:27:37.466 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_step", 2023-10-06T02:27:37.466 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-10-06T02:27:37.467 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.467 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.467 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0.5", 2023-10-06T02:27:37.467 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0.001", 2023-10-06T02:27:37.467 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "0.999", 2023-10-06T02:27:37.467 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.467 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "aggressiveness of optimization", 2023-10-06T02:27:37.467 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-10-06T02:27:37.467 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.467 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.467 INFO:teuthology.orchestra.run.smithi031.stdout: "end_time": { 2023-10-06T02:27:37.468 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "end_time", 2023-10-06T02:27:37.468 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.468 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.468 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.468 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "2400", 2023-10-06T02:27:37.468 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.468 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.468 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.468 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "ending time of day to automatically balance", 2023-10-06T02:27:37.468 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-10-06T02:27:37.468 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.468 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.469 INFO:teuthology.orchestra.run.smithi031.stdout: "end_weekday": { 2023-10-06T02:27:37.469 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "end_weekday", 2023-10-06T02:27:37.469 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2023-10-06T02:27:37.469 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.469 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.469 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "7", 2023-10-06T02:27:37.469 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0", 2023-10-06T02:27:37.469 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "7", 2023-10-06T02:27:37.469 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.469 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-10-06T02:27:37.469 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "0 or 7 = Sunday, 1 = Monday, etc.", 2023-10-06T02:27:37.469 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.470 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.470 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T02:27:37.470 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T02:27:37.470 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.470 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.470 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.470 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.470 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.470 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.470 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.470 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.471 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.471 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.471 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.471 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.471 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.471 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.471 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.471 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.471 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.471 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.471 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T02:27:37.471 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T02:27:37.472 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.472 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.472 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.472 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.472 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.472 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.472 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.472 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.472 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.472 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.472 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.473 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T02:27:37.473 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T02:27:37.473 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.473 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.473 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.473 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T02:27:37.473 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.473 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.473 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.473 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.473 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.473 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.473 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.474 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.474 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.474 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.474 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.474 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.474 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.474 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.474 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T02:27:37.474 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T02:27:37.474 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.474 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.475 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.475 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.475 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.475 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.475 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.475 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.475 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.475 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.475 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.475 INFO:teuthology.orchestra.run.smithi031.stdout: "min_score": { 2023-10-06T02:27:37.475 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "min_score", 2023-10-06T02:27:37.476 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-10-06T02:27:37.476 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.476 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.476 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2023-10-06T02:27:37.476 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.476 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.476 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.476 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-10-06T02:27:37.476 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.476 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.476 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.476 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": { 2023-10-06T02:27:37.477 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mode", 2023-10-06T02:27:37.477 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.477 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.477 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.477 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "upmap", 2023-10-06T02:27:37.477 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.477 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.477 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.477 INFO:teuthology.orchestra.run.smithi031.stdout: "crush-compat", 2023-10-06T02:27:37.477 INFO:teuthology.orchestra.run.smithi031.stdout: "none", 2023-10-06T02:27:37.477 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap" 2023-10-06T02:27:37.477 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.478 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Balancer mode", 2023-10-06T02:27:37.478 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.478 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.478 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.478 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_ids": { 2023-10-06T02:27:37.478 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pool_ids", 2023-10-06T02:27:37.478 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.478 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.478 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.478 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.479 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.479 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.479 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.479 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-10-06T02:27:37.479 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.479 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.479 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.479 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2023-10-06T02:27:37.479 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2023-10-06T02:27:37.479 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T02:27:37.479 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.479 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.480 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2023-10-06T02:27:37.480 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.480 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.480 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.480 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-10-06T02:27:37.480 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.480 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.480 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.480 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap_max_deviation": { 2023-10-06T02:27:37.480 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "upmap_max_deviation", 2023-10-06T02:27:37.480 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T02:27:37.480 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.481 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.481 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2023-10-06T02:27:37.481 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2023-10-06T02:27:37.481 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.481 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.481 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "deviation below which no optimization is attempted", 2023-10-06T02:27:37.481 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-10-06T02:27:37.481 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.481 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.481 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap_max_optimizations": { 2023-10-06T02:27:37.481 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "upmap_max_optimizations", 2023-10-06T02:27:37.481 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2023-10-06T02:27:37.482 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.482 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.482 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-10-06T02:27:37.482 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.482 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.482 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.482 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-10-06T02:27:37.482 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.482 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.482 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.482 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.483 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:37.483 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephadm", 2023-10-06T02:27:37.483 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T02:27:37.483 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T02:27:37.483 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T02:27:37.483 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_down_multiplier": { 2023-10-06T02:27:37.483 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_down_multiplier", 2023-10-06T02:27:37.483 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-10-06T02:27:37.483 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.483 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.483 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3.0", 2023-10-06T02:27:37.484 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.484 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.484 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.484 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-10-06T02:27:37.484 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.484 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.484 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.484 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_refresh_rate": { 2023-10-06T02:27:37.484 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_refresh_rate", 2023-10-06T02:27:37.484 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T02:27:37.485 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.485 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.485 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "20", 2023-10-06T02:27:37.485 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.485 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.485 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.485 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-10-06T02:27:37.485 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.485 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.485 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.485 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_starting_port": { 2023-10-06T02:27:37.486 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_starting_port", 2023-10-06T02:27:37.486 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T02:27:37.486 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.486 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.486 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "4721", 2023-10-06T02:27:37.486 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.486 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.486 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.486 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-10-06T02:27:37.486 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.486 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.486 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.487 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_ptrace": { 2023-10-06T02:27:37.487 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_ptrace", 2023-10-06T02:27:37.487 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.487 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.487 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.487 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.487 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.487 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.487 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.487 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-10-06T02:27:37.487 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-10-06T02:27:37.488 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.488 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.488 INFO:teuthology.orchestra.run.smithi031.stdout: "autotune_interval": { 2023-10-06T02:27:37.488 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "autotune_interval", 2023-10-06T02:27:37.488 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T02:27:37.488 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.488 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.488 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2023-10-06T02:27:37.488 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.489 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.489 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.489 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to autotune daemon memory", 2023-10-06T02:27:37.489 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.489 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.489 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.489 INFO:teuthology.orchestra.run.smithi031.stdout: "autotune_memory_target_ratio": { 2023-10-06T02:27:37.489 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "autotune_memory_target_ratio", 2023-10-06T02:27:37.489 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-10-06T02:27:37.489 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.489 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.490 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0.7", 2023-10-06T02:27:37.490 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.490 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.490 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.490 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-10-06T02:27:37.490 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.490 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.490 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.490 INFO:teuthology.orchestra.run.smithi031.stdout: "cgroups_split": { 2023-10-06T02:27:37.490 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cgroups_split", 2023-10-06T02:27:37.490 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.491 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.491 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.491 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T02:27:37.491 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.491 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.491 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.491 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-10-06T02:27:37.491 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.491 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.491 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.492 INFO:teuthology.orchestra.run.smithi031.stdout: "config_checks_enabled": { 2023-10-06T02:27:37.492 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_checks_enabled", 2023-10-06T02:27:37.492 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.492 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.492 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.492 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.492 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.492 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.492 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.492 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-10-06T02:27:37.492 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.492 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.493 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.493 INFO:teuthology.orchestra.run.smithi031.stdout: "config_dashboard": { 2023-10-06T02:27:37.493 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_dashboard", 2023-10-06T02:27:37.493 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.493 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.493 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.493 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T02:27:37.493 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.493 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.493 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.493 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-10-06T02:27:37.493 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.493 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.494 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.494 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_alertmanager": { 2023-10-06T02:27:37.494 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_alertmanager", 2023-10-06T02:27:37.494 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.494 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.494 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.494 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-10-06T02:27:37.494 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.494 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.494 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.494 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2023-10-06T02:27:37.494 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.495 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.495 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.495 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_base": { 2023-10-06T02:27:37.495 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_base", 2023-10-06T02:27:37.495 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.495 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.495 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.495 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/ceph", 2023-10-06T02:27:37.495 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.495 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.495 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.495 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Container image name, without the tag", 2023-10-06T02:27:37.495 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.496 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.496 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.496 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_grafana": { 2023-10-06T02:27:37.496 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_grafana", 2023-10-06T02:27:37.496 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.496 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.496 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.496 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-10-06T02:27:37.496 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.496 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.496 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.497 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2023-10-06T02:27:37.497 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.497 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.497 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.497 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_haproxy": { 2023-10-06T02:27:37.497 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_haproxy", 2023-10-06T02:27:37.497 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.497 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.497 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.497 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-10-06T02:27:37.497 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.498 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.498 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.498 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "HAproxy container image", 2023-10-06T02:27:37.498 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.498 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.498 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.498 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_keepalived": { 2023-10-06T02:27:37.498 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_keepalived", 2023-10-06T02:27:37.498 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.498 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.499 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.499 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/keepalived:2.1.5", 2023-10-06T02:27:37.499 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.499 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.499 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.499 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Keepalived container image", 2023-10-06T02:27:37.499 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.499 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.499 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.499 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_loki": { 2023-10-06T02:27:37.499 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_loki", 2023-10-06T02:27:37.500 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.500 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.500 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.500 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-10-06T02:27:37.500 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.500 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.500 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.500 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Loki container image", 2023-10-06T02:27:37.500 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.500 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.500 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.501 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_node_exporter": { 2023-10-06T02:27:37.501 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_node_exporter", 2023-10-06T02:27:37.501 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.501 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.501 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.501 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-10-06T02:27:37.501 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.501 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.501 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.501 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2023-10-06T02:27:37.501 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.501 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.501 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.502 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_prometheus": { 2023-10-06T02:27:37.502 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_prometheus", 2023-10-06T02:27:37.502 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.502 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.502 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.502 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-10-06T02:27:37.502 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.502 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.502 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.502 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2023-10-06T02:27:37.502 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.503 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.503 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.503 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_promtail": { 2023-10-06T02:27:37.503 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_promtail", 2023-10-06T02:27:37.503 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.503 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.503 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.503 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-10-06T02:27:37.503 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.503 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.503 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.503 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Promtail container image", 2023-10-06T02:27:37.504 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.504 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.504 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.504 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_snmp_gateway": { 2023-10-06T02:27:37.504 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_snmp_gateway", 2023-10-06T02:27:37.504 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.504 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.504 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.504 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-10-06T02:27:37.504 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.504 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.505 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.505 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SNMP Gateway container image", 2023-10-06T02:27:37.505 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.505 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.505 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.505 INFO:teuthology.orchestra.run.smithi031.stdout: "container_init": { 2023-10-06T02:27:37.505 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_init", 2023-10-06T02:27:37.505 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.505 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.505 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.505 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T02:27:37.506 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.506 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.506 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.506 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Run podman/docker with `--init`", 2023-10-06T02:27:37.506 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.506 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.506 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.506 INFO:teuthology.orchestra.run.smithi031.stdout: "daemon_cache_timeout": { 2023-10-06T02:27:37.506 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "daemon_cache_timeout", 2023-10-06T02:27:37.506 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T02:27:37.507 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.507 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.507 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2023-10-06T02:27:37.507 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.507 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.507 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.507 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-10-06T02:27:37.507 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.507 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.507 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.507 INFO:teuthology.orchestra.run.smithi031.stdout: "default_cephadm_command_timeout": { 2023-10-06T02:27:37.507 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "default_cephadm_command_timeout", 2023-10-06T02:27:37.508 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T02:27:37.508 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.508 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.508 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "900", 2023-10-06T02:27:37.508 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.508 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.508 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.508 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-10-06T02:27:37.508 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.508 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.508 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.508 INFO:teuthology.orchestra.run.smithi031.stdout: "default_registry": { 2023-10-06T02:27:37.509 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "default_registry", 2023-10-06T02:27:37.509 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.509 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.509 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.509 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io", 2023-10-06T02:27:37.509 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.509 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.509 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.509 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-10-06T02:27:37.509 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.509 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.509 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.510 INFO:teuthology.orchestra.run.smithi031.stdout: "device_cache_timeout": { 2023-10-06T02:27:37.510 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_cache_timeout", 2023-10-06T02:27:37.510 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T02:27:37.510 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.510 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.510 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1800", 2023-10-06T02:27:37.510 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.510 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.510 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.510 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache device inventory", 2023-10-06T02:27:37.510 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.511 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.511 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.511 INFO:teuthology.orchestra.run.smithi031.stdout: "device_enhanced_scan": { 2023-10-06T02:27:37.511 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_enhanced_scan", 2023-10-06T02:27:37.511 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.511 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.511 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.511 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.511 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.511 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.511 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.511 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use libstoragemgmt during device scans", 2023-10-06T02:27:37.512 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.512 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.512 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.512 INFO:teuthology.orchestra.run.smithi031.stdout: "facts_cache_timeout": { 2023-10-06T02:27:37.512 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "facts_cache_timeout", 2023-10-06T02:27:37.512 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T02:27:37.512 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.512 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.512 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2023-10-06T02:27:37.512 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.513 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.513 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.513 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache host facts data", 2023-10-06T02:27:37.513 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.513 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.513 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.513 INFO:teuthology.orchestra.run.smithi031.stdout: "host_check_interval": { 2023-10-06T02:27:37.513 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "host_check_interval", 2023-10-06T02:27:37.513 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T02:27:37.513 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.513 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.513 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2023-10-06T02:27:37.514 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.514 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.514 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.514 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to perform a host check", 2023-10-06T02:27:37.514 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.514 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.514 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.514 INFO:teuthology.orchestra.run.smithi031.stdout: "inventory_list_all": { 2023-10-06T02:27:37.514 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "inventory_list_all", 2023-10-06T02:27:37.514 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.514 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.515 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.515 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.515 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.515 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.515 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.515 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-10-06T02:27:37.515 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.515 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.515 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.515 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T02:27:37.515 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T02:27:37.516 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.516 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.516 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.516 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.516 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.516 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.516 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.516 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.516 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.516 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.516 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.516 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.516 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.517 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.517 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.517 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.517 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.517 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.517 INFO:teuthology.orchestra.run.smithi031.stdout: "log_refresh_metadata": { 2023-10-06T02:27:37.517 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_refresh_metadata", 2023-10-06T02:27:37.517 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.517 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.517 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.517 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.518 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.518 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.518 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.518 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-10-06T02:27:37.518 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.518 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.518 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.518 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T02:27:37.518 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T02:27:37.518 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.519 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.519 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.519 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T02:27:37.519 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.519 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.519 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.519 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-10-06T02:27:37.519 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.519 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.519 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.519 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T02:27:37.520 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T02:27:37.520 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.520 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.520 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.520 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T02:27:37.520 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.520 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.520 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.520 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.520 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.520 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.521 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.521 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.521 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.521 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.521 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.521 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.521 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.521 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.521 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T02:27:37.521 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T02:27:37.521 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.522 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.522 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.522 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.522 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.522 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.522 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.522 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.522 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.522 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.522 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.523 INFO:teuthology.orchestra.run.smithi031.stdout: "manage_etc_ceph_ceph_conf": { 2023-10-06T02:27:37.523 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-10-06T02:27:37.523 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.523 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.523 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.523 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.523 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.523 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.523 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.523 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-10-06T02:27:37.523 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.524 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.524 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.524 INFO:teuthology.orchestra.run.smithi031.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-10-06T02:27:37.524 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-10-06T02:27:37.524 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.524 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.524 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.524 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "*", 2023-10-06T02:27:37.524 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.524 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.524 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.524 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-10-06T02:27:37.525 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.525 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.525 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.525 INFO:teuthology.orchestra.run.smithi031.stdout: "max_count_per_host": { 2023-10-06T02:27:37.525 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_count_per_host", 2023-10-06T02:27:37.525 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T02:27:37.525 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.525 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.525 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-10-06T02:27:37.525 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.525 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.526 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.526 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "max number of daemons per service per host", 2023-10-06T02:27:37.526 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.526 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.526 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.526 INFO:teuthology.orchestra.run.smithi031.stdout: "max_osd_draining_count": { 2023-10-06T02:27:37.526 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_osd_draining_count", 2023-10-06T02:27:37.526 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T02:27:37.526 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.526 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.526 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-10-06T02:27:37.527 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.527 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.527 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.527 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-10-06T02:27:37.527 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.527 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.527 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.527 INFO:teuthology.orchestra.run.smithi031.stdout: "migration_current": { 2023-10-06T02:27:37.527 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "migration_current", 2023-10-06T02:27:37.527 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T02:27:37.527 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.528 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.528 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.528 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.528 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.528 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.528 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "internal - do not modify", 2023-10-06T02:27:37.528 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.528 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.528 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.528 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": { 2023-10-06T02:27:37.528 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mode", 2023-10-06T02:27:37.528 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.529 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.529 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.529 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "root", 2023-10-06T02:27:37.529 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.529 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.529 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.529 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm-package", 2023-10-06T02:27:37.529 INFO:teuthology.orchestra.run.smithi031.stdout: "root" 2023-10-06T02:27:37.529 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.529 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "mode for remote execution of cephadm", 2023-10-06T02:27:37.529 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.529 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.529 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.530 INFO:teuthology.orchestra.run.smithi031.stdout: "prometheus_alerts_path": { 2023-10-06T02:27:37.530 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prometheus_alerts_path", 2023-10-06T02:27:37.530 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.530 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.530 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.530 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-10-06T02:27:37.530 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.530 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.530 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.530 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-10-06T02:27:37.530 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.531 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.531 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.531 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_insecure": { 2023-10-06T02:27:37.531 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_insecure", 2023-10-06T02:27:37.531 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.531 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.531 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.531 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.531 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.531 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.532 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.532 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-10-06T02:27:37.532 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.532 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.532 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.532 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_password": { 2023-10-06T02:27:37.532 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_password", 2023-10-06T02:27:37.532 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.532 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.532 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.532 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.533 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.533 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.533 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.533 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-10-06T02:27:37.533 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.533 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.533 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.533 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_url": { 2023-10-06T02:27:37.533 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_url", 2023-10-06T02:27:37.533 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.533 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.534 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.534 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.534 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.534 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.534 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.534 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-10-06T02:27:37.534 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.534 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.534 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.534 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_username": { 2023-10-06T02:27:37.534 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_username", 2023-10-06T02:27:37.535 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.535 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.535 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.535 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.535 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.535 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.535 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.535 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-10-06T02:27:37.535 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.535 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.535 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.535 INFO:teuthology.orchestra.run.smithi031.stdout: "ssh_config_file": { 2023-10-06T02:27:37.536 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssh_config_file", 2023-10-06T02:27:37.536 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.536 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.536 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.536 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.536 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.536 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.536 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.536 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-10-06T02:27:37.536 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.536 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.536 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.537 INFO:teuthology.orchestra.run.smithi031.stdout: "use_agent": { 2023-10-06T02:27:37.537 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "use_agent", 2023-10-06T02:27:37.537 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.537 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.537 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.537 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.537 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.537 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.537 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.537 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-10-06T02:27:37.537 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.537 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.537 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.538 INFO:teuthology.orchestra.run.smithi031.stdout: "use_repo_digest": { 2023-10-06T02:27:37.538 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "use_repo_digest", 2023-10-06T02:27:37.538 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.538 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.538 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.538 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T02:27:37.538 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.538 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.538 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.538 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-10-06T02:27:37.539 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.539 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.539 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.539 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_failed_host_check": { 2023-10-06T02:27:37.539 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_failed_host_check", 2023-10-06T02:27:37.539 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.539 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.539 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.539 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T02:27:37.539 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.539 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.540 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.540 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if the host check fails", 2023-10-06T02:27:37.540 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.540 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.540 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.540 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_stray_daemons": { 2023-10-06T02:27:37.540 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_stray_daemons", 2023-10-06T02:27:37.540 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.540 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.540 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.540 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T02:27:37.541 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.541 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.541 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.541 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-10-06T02:27:37.541 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.541 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.541 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.541 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_stray_hosts": { 2023-10-06T02:27:37.541 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_stray_hosts", 2023-10-06T02:27:37.541 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.541 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.542 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.542 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T02:27:37.542 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.542 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.542 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.542 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-10-06T02:27:37.542 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.542 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.542 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.542 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.542 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:37.543 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crash", 2023-10-06T02:27:37.543 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T02:27:37.543 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T02:27:37.543 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T02:27:37.543 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T02:27:37.543 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T02:27:37.543 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.543 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.543 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.543 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.543 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.544 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.544 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.544 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.544 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.544 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.544 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.544 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.544 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.544 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.544 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.544 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.544 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.545 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.545 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T02:27:37.545 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T02:27:37.545 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.545 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.545 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.545 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.545 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.545 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.545 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.545 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.546 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.546 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.546 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.546 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T02:27:37.546 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T02:27:37.546 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.546 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.546 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.546 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T02:27:37.546 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.547 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.547 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.547 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.547 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.547 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.547 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.547 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.547 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.547 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.547 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.547 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.547 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.548 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.548 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T02:27:37.548 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T02:27:37.548 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.548 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.548 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.548 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.548 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.548 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.548 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.549 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.549 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.549 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.549 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.549 INFO:teuthology.orchestra.run.smithi031.stdout: "retain_interval": { 2023-10-06T02:27:37.549 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "retain_interval", 2023-10-06T02:27:37.549 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T02:27:37.549 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.549 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.549 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "31536000", 2023-10-06T02:27:37.549 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.549 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.550 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.550 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long to retain crashes before pruning them", 2023-10-06T02:27:37.550 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.550 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.550 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.550 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_recent_interval": { 2023-10-06T02:27:37.550 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_recent_interval", 2023-10-06T02:27:37.550 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T02:27:37.550 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.550 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.551 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1209600", 2023-10-06T02:27:37.551 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.551 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.551 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.551 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "time interval in which to warn about recent crashes", 2023-10-06T02:27:37.551 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.551 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.551 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.551 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.551 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:37.551 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "dashboard", 2023-10-06T02:27:37.551 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T02:27:37.552 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T02:27:37.552 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T02:27:37.552 INFO:teuthology.orchestra.run.smithi031.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-10-06T02:27:37.552 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-10-06T02:27:37.552 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T02:27:37.552 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.552 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.552 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-10-06T02:27:37.552 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.552 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.552 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.552 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.552 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.553 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.553 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.553 INFO:teuthology.orchestra.run.smithi031.stdout: "ALERTMANAGER_API_HOST": { 2023-10-06T02:27:37.553 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ALERTMANAGER_API_HOST", 2023-10-06T02:27:37.553 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.553 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.553 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.553 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.553 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.553 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.553 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.553 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.553 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.554 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.554 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.554 INFO:teuthology.orchestra.run.smithi031.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-10-06T02:27:37.554 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-10-06T02:27:37.554 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.554 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.554 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.554 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T02:27:37.554 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.554 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.555 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.555 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.555 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.555 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.555 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.555 INFO:teuthology.orchestra.run.smithi031.stdout: "AUDIT_API_ENABLED": { 2023-10-06T02:27:37.555 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "AUDIT_API_ENABLED", 2023-10-06T02:27:37.555 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.555 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.555 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.555 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.556 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.556 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.556 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.556 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.556 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.556 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.556 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.556 INFO:teuthology.orchestra.run.smithi031.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-10-06T02:27:37.556 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-10-06T02:27:37.557 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.557 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.557 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.557 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T02:27:37.557 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.557 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.557 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.557 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.557 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.557 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.557 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.558 INFO:teuthology.orchestra.run.smithi031.stdout: "ENABLE_BROWSABLE_API": { 2023-10-06T02:27:37.558 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ENABLE_BROWSABLE_API", 2023-10-06T02:27:37.558 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.558 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.558 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.558 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T02:27:37.558 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.558 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.558 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.558 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.559 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.559 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.559 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.559 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-10-06T02:27:37.559 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-10-06T02:27:37.559 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.559 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.559 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.559 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T02:27:37.559 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.560 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.560 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.560 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.560 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.560 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.560 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.560 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-10-06T02:27:37.560 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-10-06T02:27:37.561 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.561 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.561 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.561 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T02:27:37.561 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.561 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.561 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.561 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.561 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.561 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.561 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.562 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-10-06T02:27:37.562 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-10-06T02:27:37.562 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.562 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.562 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.562 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T02:27:37.562 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.562 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.562 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.562 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.562 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.563 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.563 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.563 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-10-06T02:27:37.563 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-10-06T02:27:37.563 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.563 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.563 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.563 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T02:27:37.563 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.564 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.564 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.564 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.564 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.564 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.564 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.564 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_NFS": { 2023-10-06T02:27:37.564 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-10-06T02:27:37.564 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.564 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.565 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.565 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T02:27:37.565 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.565 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.565 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.565 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.565 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.565 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.565 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.565 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_RBD": { 2023-10-06T02:27:37.565 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-10-06T02:27:37.565 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.566 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.566 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.566 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T02:27:37.566 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.566 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.566 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.566 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.566 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.566 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.566 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.566 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_RGW": { 2023-10-06T02:27:37.567 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-10-06T02:27:37.567 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.567 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.567 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.567 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T02:27:37.567 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.567 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.567 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.567 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.567 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.567 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.568 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.568 INFO:teuthology.orchestra.run.smithi031.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-10-06T02:27:37.568 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-10-06T02:27:37.568 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.568 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.568 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.568 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.568 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.568 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.568 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.569 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.569 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.569 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.569 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.569 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_PASSWORD": { 2023-10-06T02:27:37.569 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_PASSWORD", 2023-10-06T02:27:37.569 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.569 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.569 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.569 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2023-10-06T02:27:37.570 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.570 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.570 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.570 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.570 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.570 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.570 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.570 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-10-06T02:27:37.570 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-10-06T02:27:37.570 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.571 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.571 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.571 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T02:27:37.571 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.571 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.571 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.571 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.571 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.571 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.571 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.571 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_URL": { 2023-10-06T02:27:37.572 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_URL", 2023-10-06T02:27:37.572 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.572 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.572 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.572 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.572 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.572 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.572 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.572 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.572 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.572 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.572 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.573 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_USERNAME": { 2023-10-06T02:27:37.573 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_USERNAME", 2023-10-06T02:27:37.573 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.573 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.573 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.573 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2023-10-06T02:27:37.573 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.573 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.573 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.573 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.573 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.573 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.574 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.574 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-10-06T02:27:37.574 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-10-06T02:27:37.574 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.574 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.574 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.574 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.574 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.574 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.574 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.575 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.575 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.575 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.575 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.575 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-10-06T02:27:37.575 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-10-06T02:27:37.575 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.575 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.575 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.575 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.575 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.576 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.576 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.576 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.576 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.576 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.576 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.576 INFO:teuthology.orchestra.run.smithi031.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-10-06T02:27:37.576 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-10-06T02:27:37.576 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.576 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.576 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.577 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T02:27:37.577 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.577 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.577 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.577 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.577 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.577 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.577 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.577 INFO:teuthology.orchestra.run.smithi031.stdout: "ISSUE_TRACKER_API_KEY": { 2023-10-06T02:27:37.577 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-10-06T02:27:37.578 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.578 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.578 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.578 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.578 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.578 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.578 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.578 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.578 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.578 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.578 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.579 INFO:teuthology.orchestra.run.smithi031.stdout: "PROMETHEUS_API_HOST": { 2023-10-06T02:27:37.579 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PROMETHEUS_API_HOST", 2023-10-06T02:27:37.579 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.579 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.579 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.579 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.579 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.579 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.579 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.579 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.579 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.579 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.579 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.580 INFO:teuthology.orchestra.run.smithi031.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-10-06T02:27:37.580 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-10-06T02:27:37.580 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.580 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.580 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.580 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T02:27:37.580 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.580 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.580 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.581 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.581 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.581 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.581 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.581 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-10-06T02:27:37.581 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-10-06T02:27:37.581 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.581 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.581 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.582 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.582 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.582 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.582 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.582 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.582 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.582 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.582 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.582 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-10-06T02:27:37.583 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-10-06T02:27:37.583 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.583 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.583 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.583 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.583 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.583 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.583 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.583 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.583 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.583 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.584 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.584 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-10-06T02:27:37.584 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-10-06T02:27:37.584 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.584 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.584 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.584 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T02:27:37.584 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.584 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.585 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.585 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.585 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.585 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.585 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.585 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-10-06T02:27:37.585 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-10-06T02:27:37.585 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.585 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.585 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.586 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T02:27:37.586 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.586 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.586 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.586 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.586 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.586 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.586 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.586 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-10-06T02:27:37.586 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-10-06T02:27:37.587 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.587 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.587 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.587 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.587 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.587 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.587 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.587 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.587 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.587 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.587 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.588 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-10-06T02:27:37.588 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-10-06T02:27:37.588 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.588 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.588 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.588 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.588 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.588 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.588 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.588 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.588 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.588 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.589 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.589 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-10-06T02:27:37.589 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-10-06T02:27:37.589 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.589 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.589 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.589 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.589 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.589 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.589 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.590 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.590 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.590 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.590 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.590 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_ENABLED": { 2023-10-06T02:27:37.590 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_ENABLED", 2023-10-06T02:27:37.590 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.590 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.590 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.590 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T02:27:37.590 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.591 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.591 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.591 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.591 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.591 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.591 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.591 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-10-06T02:27:37.591 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-10-06T02:27:37.591 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.591 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.592 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.592 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-10-06T02:27:37.592 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.592 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.592 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.592 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.592 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.592 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.592 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.592 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-10-06T02:27:37.592 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-10-06T02:27:37.592 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T02:27:37.593 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.593 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.593 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-10-06T02:27:37.593 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.593 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.593 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.593 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.593 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.593 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.593 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.593 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-10-06T02:27:37.594 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-10-06T02:27:37.594 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T02:27:37.594 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.594 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.594 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8", 2023-10-06T02:27:37.594 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.594 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.594 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.594 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.594 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.594 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.594 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.595 INFO:teuthology.orchestra.run.smithi031.stdout: "REST_REQUESTS_TIMEOUT": { 2023-10-06T02:27:37.595 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-10-06T02:27:37.595 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T02:27:37.595 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.595 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.595 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "45", 2023-10-06T02:27:37.595 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.595 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.595 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.595 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.595 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.595 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.596 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.596 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_ACCESS_KEY": { 2023-10-06T02:27:37.596 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_ACCESS_KEY", 2023-10-06T02:27:37.596 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.596 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.596 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.596 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.596 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.596 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.596 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.596 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.596 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.597 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.597 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.597 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-10-06T02:27:37.597 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-10-06T02:27:37.597 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.597 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.597 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.597 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2023-10-06T02:27:37.597 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.597 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.597 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.597 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.598 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.598 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.598 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.598 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_SECRET_KEY": { 2023-10-06T02:27:37.598 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_SECRET_KEY", 2023-10-06T02:27:37.598 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.598 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.598 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.598 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.598 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.598 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.599 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.599 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.599 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.599 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.599 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.599 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_SSL_VERIFY": { 2023-10-06T02:27:37.599 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_SSL_VERIFY", 2023-10-06T02:27:37.599 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.599 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.599 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.599 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T02:27:37.599 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.600 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.600 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.600 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.600 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.600 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.600 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.600 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-10-06T02:27:37.600 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-10-06T02:27:37.600 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T02:27:37.600 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.600 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.600 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2023-10-06T02:27:37.601 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.601 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.601 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.601 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.601 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.601 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.601 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.601 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-10-06T02:27:37.601 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-10-06T02:27:37.601 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T02:27:37.601 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.601 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.602 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-10-06T02:27:37.602 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.602 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.602 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.602 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.602 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.602 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.602 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.602 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-10-06T02:27:37.602 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-10-06T02:27:37.602 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T02:27:37.603 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.603 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.603 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2023-10-06T02:27:37.603 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.603 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.603 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.603 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.603 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.603 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.603 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.603 INFO:teuthology.orchestra.run.smithi031.stdout: "cross_origin_url": { 2023-10-06T02:27:37.604 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cross_origin_url", 2023-10-06T02:27:37.604 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.604 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.604 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.604 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.604 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.604 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.604 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.604 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.604 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.604 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.604 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.605 INFO:teuthology.orchestra.run.smithi031.stdout: "crt_file": { 2023-10-06T02:27:37.605 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crt_file", 2023-10-06T02:27:37.605 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.605 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.605 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.605 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.605 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.605 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.605 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.606 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.606 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.606 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.606 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.606 INFO:teuthology.orchestra.run.smithi031.stdout: "debug": { 2023-10-06T02:27:37.606 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "debug", 2023-10-06T02:27:37.606 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.606 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.606 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.606 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.606 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.607 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.607 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.607 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable/disable debug options", 2023-10-06T02:27:37.607 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.607 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.607 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.607 INFO:teuthology.orchestra.run.smithi031.stdout: "jwt_token_ttl": { 2023-10-06T02:27:37.607 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "jwt_token_ttl", 2023-10-06T02:27:37.607 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T02:27:37.607 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.607 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.608 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "28800", 2023-10-06T02:27:37.608 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.608 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.608 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.608 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.608 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.608 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.608 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.608 INFO:teuthology.orchestra.run.smithi031.stdout: "key_file": { 2023-10-06T02:27:37.608 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "key_file", 2023-10-06T02:27:37.608 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.608 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.609 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.609 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.609 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.609 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.609 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.609 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.609 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.609 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.609 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.609 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T02:27:37.609 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T02:27:37.609 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.609 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.610 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.610 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.610 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.610 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.610 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.610 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.610 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.610 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.610 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.610 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.610 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.610 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.611 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.611 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.611 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.611 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.611 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T02:27:37.611 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T02:27:37.611 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.611 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.611 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.611 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.611 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.611 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.612 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.612 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.612 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.612 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.612 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.612 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T02:27:37.612 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T02:27:37.612 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.612 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.612 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.612 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T02:27:37.613 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.613 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.613 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.613 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.613 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.613 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.613 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.613 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.613 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.613 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.613 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.613 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.613 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.613 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.614 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T02:27:37.614 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T02:27:37.614 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.614 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.614 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.614 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.614 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.614 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.614 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.614 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.615 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.615 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.615 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.615 INFO:teuthology.orchestra.run.smithi031.stdout: "motd": { 2023-10-06T02:27:37.615 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "motd", 2023-10-06T02:27:37.615 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.615 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.615 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.615 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.615 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.615 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.616 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.616 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "The message of the day", 2023-10-06T02:27:37.616 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.616 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.616 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.616 INFO:teuthology.orchestra.run.smithi031.stdout: "redirect_resolve_ip_addr": { 2023-10-06T02:27:37.616 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "redirect_resolve_ip_addr", 2023-10-06T02:27:37.616 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.616 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.616 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.616 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.617 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.617 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.617 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.617 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.617 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.617 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.617 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.617 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2023-10-06T02:27:37.617 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2023-10-06T02:27:37.617 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.617 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.618 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.618 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "::", 2023-10-06T02:27:37.618 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.618 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.618 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.618 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.618 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.618 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.618 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.619 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2023-10-06T02:27:37.619 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2023-10-06T02:27:37.619 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T02:27:37.619 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.619 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.619 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8080", 2023-10-06T02:27:37.619 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.619 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.619 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.619 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.619 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.619 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.620 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.620 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl": { 2023-10-06T02:27:37.620 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl", 2023-10-06T02:27:37.620 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.620 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.620 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.620 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T02:27:37.620 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.620 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.620 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.620 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.620 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.621 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.621 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.621 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl_server_port": { 2023-10-06T02:27:37.621 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl_server_port", 2023-10-06T02:27:37.621 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T02:27:37.621 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.621 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.621 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8443", 2023-10-06T02:27:37.621 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.621 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.621 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.621 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.622 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.622 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.622 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.622 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_behaviour": { 2023-10-06T02:27:37.622 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_behaviour", 2023-10-06T02:27:37.622 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.622 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.622 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.622 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "redirect", 2023-10-06T02:27:37.622 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.623 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.623 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.623 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.623 INFO:teuthology.orchestra.run.smithi031.stdout: "redirect" 2023-10-06T02:27:37.623 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.623 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.623 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.623 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.623 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.623 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_error_status_code": { 2023-10-06T02:27:37.623 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_error_status_code", 2023-10-06T02:27:37.624 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T02:27:37.624 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.624 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.624 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "500", 2023-10-06T02:27:37.624 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "400", 2023-10-06T02:27:37.624 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "599", 2023-10-06T02:27:37.624 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.624 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.624 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.624 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.625 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.625 INFO:teuthology.orchestra.run.smithi031.stdout: "url_prefix": { 2023-10-06T02:27:37.625 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "url_prefix", 2023-10-06T02:27:37.625 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.625 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.625 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.625 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.625 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.625 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.625 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.626 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.626 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.626 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.626 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.626 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.626 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:37.626 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "devicehealth", 2023-10-06T02:27:37.626 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T02:27:37.626 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T02:27:37.626 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T02:27:37.627 INFO:teuthology.orchestra.run.smithi031.stdout: "enable_monitoring": { 2023-10-06T02:27:37.627 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enable_monitoring", 2023-10-06T02:27:37.627 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.627 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.627 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.627 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T02:27:37.627 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.627 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.627 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.627 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "monitor device health metrics", 2023-10-06T02:27:37.627 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.629 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.629 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.629 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T02:27:37.629 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T02:27:37.629 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.630 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.630 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.630 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.630 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.630 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.630 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.630 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.630 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.630 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.630 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.630 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.630 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.630 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.631 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.631 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.631 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.631 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.631 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T02:27:37.631 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T02:27:37.631 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.631 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.631 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.631 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.631 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.631 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.632 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.632 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.632 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.632 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.632 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.632 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T02:27:37.632 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T02:27:37.632 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.632 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.632 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.632 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T02:27:37.632 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.633 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.633 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.633 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.633 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.633 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.633 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.633 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.633 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.633 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.633 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.633 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.633 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.634 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.634 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T02:27:37.634 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T02:27:37.634 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.634 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.634 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.634 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.634 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.634 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.634 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.634 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.635 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.635 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.635 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.635 INFO:teuthology.orchestra.run.smithi031.stdout: "mark_out_threshold": { 2023-10-06T02:27:37.635 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mark_out_threshold", 2023-10-06T02:27:37.635 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T02:27:37.635 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.635 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.635 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "2419200", 2023-10-06T02:27:37.635 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.635 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.635 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.636 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-10-06T02:27:37.636 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.636 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.636 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.636 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": { 2023-10-06T02:27:37.636 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pool_name", 2023-10-06T02:27:37.636 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.636 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.636 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.636 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "device_health_metrics", 2023-10-06T02:27:37.636 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.637 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.637 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.637 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "name of pool in which to store device health metrics", 2023-10-06T02:27:37.637 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.637 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.637 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.637 INFO:teuthology.orchestra.run.smithi031.stdout: "retention_period": { 2023-10-06T02:27:37.637 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "retention_period", 2023-10-06T02:27:37.637 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T02:27:37.637 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.637 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.637 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15552000", 2023-10-06T02:27:37.638 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.638 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.638 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.638 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long to retain device health metrics", 2023-10-06T02:27:37.638 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.638 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.638 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.638 INFO:teuthology.orchestra.run.smithi031.stdout: "scrape_frequency": { 2023-10-06T02:27:37.638 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "scrape_frequency", 2023-10-06T02:27:37.638 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T02:27:37.638 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.639 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.639 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "86400", 2023-10-06T02:27:37.639 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.639 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.639 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.639 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to scrape device health metrics", 2023-10-06T02:27:37.639 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.639 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.639 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.639 INFO:teuthology.orchestra.run.smithi031.stdout: "self_heal": { 2023-10-06T02:27:37.639 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "self_heal", 2023-10-06T02:27:37.639 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.640 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.640 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.640 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T02:27:37.640 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.640 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.640 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.640 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-10-06T02:27:37.640 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.640 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.640 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.641 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2023-10-06T02:27:37.641 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2023-10-06T02:27:37.641 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T02:27:37.641 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.641 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.641 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2023-10-06T02:27:37.641 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.641 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.641 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.641 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to wake up and check device health", 2023-10-06T02:27:37.641 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.641 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.642 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.642 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_threshold": { 2023-10-06T02:27:37.642 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_threshold", 2023-10-06T02:27:37.642 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T02:27:37.642 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.642 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.642 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "7257600", 2023-10-06T02:27:37.642 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.642 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.642 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.642 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-10-06T02:27:37.643 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.643 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.643 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.643 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.643 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:37.643 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "diskprediction_local", 2023-10-06T02:27:37.643 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T02:27:37.643 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T02:27:37.643 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T02:27:37.643 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T02:27:37.643 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T02:27:37.644 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.644 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.644 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.644 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.644 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.644 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.644 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.644 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.644 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.644 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.644 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.645 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.645 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.645 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.645 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.645 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.645 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.645 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.645 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T02:27:37.645 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T02:27:37.645 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.645 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.645 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.646 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.646 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.646 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.646 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.646 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.646 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.646 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.646 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.646 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T02:27:37.646 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T02:27:37.646 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.647 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.647 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.647 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T02:27:37.647 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.647 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.647 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.647 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.647 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.647 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.647 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.647 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.647 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.648 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.648 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.648 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.648 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.648 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.648 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T02:27:37.648 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T02:27:37.648 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.648 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.648 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.648 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.649 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.649 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.649 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.649 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.649 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.649 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.649 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.649 INFO:teuthology.orchestra.run.smithi031.stdout: "predict_interval": { 2023-10-06T02:27:37.649 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "predict_interval", 2023-10-06T02:27:37.649 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.649 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.650 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.650 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "86400", 2023-10-06T02:27:37.650 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.650 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.650 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.650 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.650 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.650 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.650 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.650 INFO:teuthology.orchestra.run.smithi031.stdout: "predictor_model": { 2023-10-06T02:27:37.650 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "predictor_model", 2023-10-06T02:27:37.650 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.651 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.651 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.651 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "prophetstor", 2023-10-06T02:27:37.651 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.651 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.651 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.651 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.651 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.651 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.651 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.651 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2023-10-06T02:27:37.652 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2023-10-06T02:27:37.652 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.652 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.652 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.652 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2023-10-06T02:27:37.652 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.652 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.652 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.652 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.652 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.652 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.652 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.652 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.653 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:37.653 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "influx", 2023-10-06T02:27:37.653 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": false, 2023-10-06T02:27:37.653 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "influxdb python module not found", 2023-10-06T02:27:37.653 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T02:27:37.653 INFO:teuthology.orchestra.run.smithi031.stdout: "batch_size": { 2023-10-06T02:27:37.653 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "batch_size", 2023-10-06T02:27:37.653 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T02:27:37.653 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.653 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.653 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5000", 2023-10-06T02:27:37.654 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.654 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.654 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.654 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-10-06T02:27:37.654 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.654 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.654 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.654 INFO:teuthology.orchestra.run.smithi031.stdout: "database": { 2023-10-06T02:27:37.654 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "database", 2023-10-06T02:27:37.654 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.654 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.654 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.655 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "ceph", 2023-10-06T02:27:37.655 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.655 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.655 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.655 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-10-06T02:27:37.655 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.655 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.655 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.655 INFO:teuthology.orchestra.run.smithi031.stdout: "hostname": { 2023-10-06T02:27:37.655 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "hostname", 2023-10-06T02:27:37.655 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.655 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.656 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.656 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.656 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.656 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.656 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.656 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB server hostname", 2023-10-06T02:27:37.656 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.656 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.656 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.656 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2023-10-06T02:27:37.656 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2023-10-06T02:27:37.657 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T02:27:37.657 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.657 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.657 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "30", 2023-10-06T02:27:37.657 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "5", 2023-10-06T02:27:37.657 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.657 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.657 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-10-06T02:27:37.657 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.657 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.657 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.658 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T02:27:37.658 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T02:27:37.658 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.658 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.658 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.658 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.658 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.658 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.658 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.658 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.658 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.659 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.659 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.659 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.659 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.659 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.659 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.659 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.659 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.659 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.659 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T02:27:37.659 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T02:27:37.660 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.660 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.660 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.660 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.660 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.660 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.660 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.660 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.660 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.660 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.660 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.661 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T02:27:37.661 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T02:27:37.661 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.661 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.661 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.661 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T02:27:37.661 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.661 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.661 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.661 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.661 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.661 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.662 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.662 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.662 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.662 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.662 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.662 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.662 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.662 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.662 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T02:27:37.662 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T02:27:37.663 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.663 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.663 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.663 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.663 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.663 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.663 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.663 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.663 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.663 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.663 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.664 INFO:teuthology.orchestra.run.smithi031.stdout: "password": { 2023-10-06T02:27:37.664 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "password", 2023-10-06T02:27:37.664 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.664 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.664 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.664 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.664 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.664 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.664 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.664 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "password of InfluxDB server user", 2023-10-06T02:27:37.664 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.664 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.664 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.665 INFO:teuthology.orchestra.run.smithi031.stdout: "port": { 2023-10-06T02:27:37.665 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "port", 2023-10-06T02:27:37.665 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T02:27:37.665 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.665 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.665 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8086", 2023-10-06T02:27:37.665 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.665 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.665 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.665 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB server port", 2023-10-06T02:27:37.665 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.666 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.666 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.666 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl": { 2023-10-06T02:27:37.666 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl", 2023-10-06T02:27:37.666 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.666 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.666 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.666 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "false", 2023-10-06T02:27:37.666 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.666 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.666 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.666 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-10-06T02:27:37.667 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.667 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.667 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.667 INFO:teuthology.orchestra.run.smithi031.stdout: "threads": { 2023-10-06T02:27:37.667 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "threads", 2023-10-06T02:27:37.667 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T02:27:37.667 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.667 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.667 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2023-10-06T02:27:37.667 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2023-10-06T02:27:37.667 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "32", 2023-10-06T02:27:37.668 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.668 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-10-06T02:27:37.668 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.668 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.668 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.668 INFO:teuthology.orchestra.run.smithi031.stdout: "username": { 2023-10-06T02:27:37.668 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "username", 2023-10-06T02:27:37.668 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.668 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.668 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.668 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.668 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.669 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.669 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.669 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "username of InfluxDB server user", 2023-10-06T02:27:37.669 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.669 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.669 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.669 INFO:teuthology.orchestra.run.smithi031.stdout: "verify_ssl": { 2023-10-06T02:27:37.669 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "verify_ssl", 2023-10-06T02:27:37.669 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.669 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.669 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.669 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "true", 2023-10-06T02:27:37.670 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.670 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.670 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.670 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-10-06T02:27:37.670 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.670 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.670 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.670 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.670 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:37.670 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "insights", 2023-10-06T02:27:37.671 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T02:27:37.671 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T02:27:37.671 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T02:27:37.671 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T02:27:37.671 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T02:27:37.671 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.671 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.671 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.671 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.671 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.671 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.671 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.671 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.672 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.672 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.672 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.672 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.672 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.672 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.672 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.672 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.672 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.672 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.672 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T02:27:37.672 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T02:27:37.673 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.673 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.673 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.673 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.673 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.673 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.673 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.673 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.673 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.673 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.673 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.674 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T02:27:37.674 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T02:27:37.674 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.674 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.674 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.674 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T02:27:37.674 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.674 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.674 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.674 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.674 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.674 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.674 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.675 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.675 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.675 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.675 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.675 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.675 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.675 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.675 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T02:27:37.675 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T02:27:37.675 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.675 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.675 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.676 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.676 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.676 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.676 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.676 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.676 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.676 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.676 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.676 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.676 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:37.677 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "iostat", 2023-10-06T02:27:37.677 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T02:27:37.677 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T02:27:37.677 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T02:27:37.677 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T02:27:37.677 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T02:27:37.677 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.677 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.677 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.677 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.677 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.677 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.677 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.678 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.678 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.678 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.678 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.678 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.678 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.678 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.678 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.678 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.678 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.678 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.679 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T02:27:37.679 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T02:27:37.679 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.679 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.679 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.679 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.679 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.679 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.679 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.679 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.679 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.680 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.680 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.680 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T02:27:37.680 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T02:27:37.680 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.680 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.680 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.680 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T02:27:37.680 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.680 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.680 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.681 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.681 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.681 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.681 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.681 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.681 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.681 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.681 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.681 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.681 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.681 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.681 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T02:27:37.682 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T02:27:37.682 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.682 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.682 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.682 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.682 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.682 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.682 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.682 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.682 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.682 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.682 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.683 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.683 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:37.683 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "localpool", 2023-10-06T02:27:37.683 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T02:27:37.683 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T02:27:37.683 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T02:27:37.683 INFO:teuthology.orchestra.run.smithi031.stdout: "failure_domain": { 2023-10-06T02:27:37.683 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "failure_domain", 2023-10-06T02:27:37.683 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.683 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.684 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.684 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "host", 2023-10-06T02:27:37.684 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.684 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.684 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.684 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "failure domain for any created local pool", 2023-10-06T02:27:37.684 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-10-06T02:27:37.684 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.684 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.684 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T02:27:37.684 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T02:27:37.684 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.684 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.685 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.685 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.685 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.685 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.685 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.685 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.685 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.685 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.685 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.685 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.685 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.685 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.686 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.686 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.686 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.686 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.686 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T02:27:37.686 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T02:27:37.686 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.686 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.686 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.686 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.686 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.686 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.687 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.687 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.687 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.687 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.687 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.687 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T02:27:37.687 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T02:27:37.687 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.687 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.687 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.687 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T02:27:37.688 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.688 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.688 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.688 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.688 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.688 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.688 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.688 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.688 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.688 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.688 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.688 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.689 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.689 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.689 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T02:27:37.689 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T02:27:37.689 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.689 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.689 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.689 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.689 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.689 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.689 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.690 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.690 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.690 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.690 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.690 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": { 2023-10-06T02:27:37.690 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "min_size", 2023-10-06T02:27:37.690 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T02:27:37.690 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.690 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.690 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.690 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.691 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.691 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.691 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default min_size for any created local pool", 2023-10-06T02:27:37.691 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-10-06T02:27:37.691 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.691 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.691 INFO:teuthology.orchestra.run.smithi031.stdout: "num_rep": { 2023-10-06T02:27:37.691 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "num_rep", 2023-10-06T02:27:37.691 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T02:27:37.691 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.691 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.691 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3", 2023-10-06T02:27:37.692 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.692 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.692 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.692 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default replica count for any created local pool", 2023-10-06T02:27:37.692 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.692 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.692 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.692 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": { 2023-10-06T02:27:37.692 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pg_num", 2023-10-06T02:27:37.692 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T02:27:37.692 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.693 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.693 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "128", 2023-10-06T02:27:37.693 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.693 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.693 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.693 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default pg_num for any created local pool", 2023-10-06T02:27:37.693 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.693 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.693 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.693 INFO:teuthology.orchestra.run.smithi031.stdout: "prefix": { 2023-10-06T02:27:37.693 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prefix", 2023-10-06T02:27:37.693 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.694 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.694 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.694 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.694 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.694 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.694 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.694 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "name prefix for any created local pool", 2023-10-06T02:27:37.694 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.694 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.694 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.695 INFO:teuthology.orchestra.run.smithi031.stdout: "subtree": { 2023-10-06T02:27:37.695 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "subtree", 2023-10-06T02:27:37.695 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.695 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.695 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.695 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "rack", 2023-10-06T02:27:37.695 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.695 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.695 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.695 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "CRUSH level for which to create a local pool", 2023-10-06T02:27:37.695 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-10-06T02:27:37.695 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.696 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.696 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.696 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:37.696 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mds_autoscaler", 2023-10-06T02:27:37.696 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T02:27:37.696 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T02:27:37.696 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T02:27:37.696 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T02:27:37.696 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T02:27:37.696 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.697 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.697 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.697 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.697 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.697 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.697 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.697 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.697 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.697 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.697 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.697 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.697 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.697 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.698 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.698 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.698 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.698 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.698 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T02:27:37.698 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T02:27:37.698 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.698 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.698 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.699 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.699 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.699 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.699 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.699 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.699 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.699 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.699 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.699 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T02:27:37.699 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T02:27:37.699 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.700 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.700 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.700 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T02:27:37.700 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.700 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.700 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.700 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.700 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.700 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.700 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.700 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.700 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.700 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.701 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.701 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.701 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.701 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.701 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T02:27:37.701 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T02:27:37.701 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.701 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.701 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.701 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.701 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.702 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.702 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.702 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.702 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.702 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.702 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.702 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.702 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:37.702 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mirroring", 2023-10-06T02:27:37.702 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T02:27:37.702 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T02:27:37.703 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T02:27:37.703 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T02:27:37.703 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T02:27:37.703 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.703 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.703 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.703 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.703 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.703 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.703 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.703 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.703 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.704 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.704 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.704 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.704 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.704 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.704 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.704 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.704 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.704 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.704 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T02:27:37.704 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T02:27:37.705 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.705 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.705 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.705 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.705 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.705 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.705 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.705 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.705 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.705 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.705 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.705 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T02:27:37.706 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T02:27:37.706 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.706 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.706 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.706 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T02:27:37.706 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.706 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.706 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.706 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.706 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.706 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.706 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.707 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.707 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.707 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.707 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.707 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.707 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.707 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.707 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T02:27:37.707 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T02:27:37.707 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.707 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.707 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.708 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.708 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.708 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.708 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.708 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.708 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.708 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.708 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.708 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.708 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:37.709 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "nfs", 2023-10-06T02:27:37.709 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T02:27:37.709 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T02:27:37.709 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T02:27:37.709 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T02:27:37.709 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T02:27:37.709 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.709 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.709 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.709 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.709 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.709 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.709 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.710 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.710 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.710 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.710 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.710 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.710 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.710 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.710 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.710 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.710 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.710 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.710 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T02:27:37.711 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T02:27:37.711 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.711 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.711 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.711 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.711 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.711 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.711 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.711 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.711 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.711 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.711 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.712 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T02:27:37.712 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T02:27:37.712 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.712 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.712 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.712 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T02:27:37.712 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.712 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.712 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.712 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.713 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.713 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.713 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.713 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.713 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.713 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.713 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.713 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.713 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.713 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.713 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T02:27:37.713 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T02:27:37.714 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.714 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.714 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.714 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.714 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.714 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.714 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.714 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.714 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.714 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.714 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.714 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.715 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:37.715 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "orchestrator", 2023-10-06T02:27:37.715 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T02:27:37.715 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T02:27:37.715 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T02:27:37.715 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T02:27:37.715 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T02:27:37.715 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.715 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.715 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.715 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.716 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.716 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.716 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.716 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.716 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.716 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.716 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.716 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.716 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.716 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.717 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.717 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.717 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.717 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.717 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T02:27:37.717 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T02:27:37.717 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.717 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.717 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.717 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.717 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.718 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.718 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.718 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.718 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.718 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.718 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.718 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T02:27:37.718 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T02:27:37.718 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.718 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.718 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.719 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T02:27:37.719 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.719 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.719 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.719 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.719 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.719 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.719 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.719 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.719 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.719 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.719 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.720 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.720 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.720 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.720 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T02:27:37.720 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T02:27:37.720 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.720 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.720 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.720 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.720 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.720 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.721 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.721 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.721 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.721 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.721 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.721 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator": { 2023-10-06T02:27:37.721 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "orchestrator", 2023-10-06T02:27:37.721 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.721 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.721 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.721 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.722 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.722 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.722 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.722 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm", 2023-10-06T02:27:37.722 INFO:teuthology.orchestra.run.smithi031.stdout: "rook", 2023-10-06T02:27:37.722 INFO:teuthology.orchestra.run.smithi031.stdout: "test_orchestrator" 2023-10-06T02:27:37.722 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.722 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Orchestrator backend", 2023-10-06T02:27:37.722 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.722 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.722 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.722 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.722 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:37.723 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "osd_perf_query", 2023-10-06T02:27:37.723 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T02:27:37.723 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T02:27:37.723 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T02:27:37.723 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T02:27:37.723 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T02:27:37.723 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.723 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.723 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.723 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.723 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.724 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.724 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.724 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.724 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.724 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.724 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.724 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.724 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.724 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.724 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.724 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.724 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.724 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.725 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T02:27:37.725 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T02:27:37.725 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.725 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.725 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.725 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.725 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.725 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.725 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.725 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.725 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.725 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.726 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.726 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T02:27:37.726 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T02:27:37.726 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.726 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.726 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.726 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T02:27:37.726 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.726 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.726 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.727 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.727 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.727 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.727 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.727 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.727 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.727 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.727 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.727 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.727 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.727 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.728 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T02:27:37.728 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T02:27:37.728 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.728 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.728 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.728 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.728 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.728 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.728 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.728 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.728 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.729 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.729 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.729 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.729 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:37.729 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "osd_support", 2023-10-06T02:27:37.729 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T02:27:37.729 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T02:27:37.729 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T02:27:37.729 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T02:27:37.729 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T02:27:37.730 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.730 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.730 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.730 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.730 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.730 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.730 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.730 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.730 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.730 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.730 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.730 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.730 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.731 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.731 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.731 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.731 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.731 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.731 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T02:27:37.731 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T02:27:37.731 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.731 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.731 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.731 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.731 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.732 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.732 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.732 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.732 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.732 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.732 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.732 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T02:27:37.732 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T02:27:37.732 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.732 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.732 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.732 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T02:27:37.733 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.733 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.733 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.733 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.733 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.733 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.733 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.733 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.733 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.733 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.733 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.733 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.734 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.734 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.734 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T02:27:37.734 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T02:27:37.734 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.734 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.734 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.734 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.734 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.734 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.734 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.735 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.735 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.735 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.735 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.735 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.735 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:37.735 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pg_autoscaler", 2023-10-06T02:27:37.735 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T02:27:37.735 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T02:27:37.735 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T02:27:37.735 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T02:27:37.736 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T02:27:37.736 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.736 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.736 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.736 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.736 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.736 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.736 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.736 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.736 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.736 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.737 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.737 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.737 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.737 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.737 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.737 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.737 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.737 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.737 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T02:27:37.737 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T02:27:37.737 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.737 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.738 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.738 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.738 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.738 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.738 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.738 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.738 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.738 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.738 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.738 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T02:27:37.738 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T02:27:37.739 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.739 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.739 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.739 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T02:27:37.739 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.739 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.739 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.739 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.739 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.739 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.739 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.739 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.739 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.740 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.740 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.740 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.740 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.740 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.740 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T02:27:37.740 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T02:27:37.740 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.740 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.740 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.740 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.740 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.741 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.741 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.741 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.741 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.741 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.741 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.741 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2023-10-06T02:27:37.741 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2023-10-06T02:27:37.741 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T02:27:37.741 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.742 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.742 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2023-10-06T02:27:37.742 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.742 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.742 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.742 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.742 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.742 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.742 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.742 INFO:teuthology.orchestra.run.smithi031.stdout: "threshold": { 2023-10-06T02:27:37.742 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "threshold", 2023-10-06T02:27:37.743 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-10-06T02:27:37.743 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.743 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.743 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3.0", 2023-10-06T02:27:37.743 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1.0", 2023-10-06T02:27:37.743 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.743 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.743 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "scaling threshold", 2023-10-06T02:27:37.743 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-10-06T02:27:37.743 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.743 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.743 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.743 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:37.744 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "progress", 2023-10-06T02:27:37.744 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T02:27:37.744 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T02:27:37.744 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T02:27:37.744 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_pg_recovery_event": { 2023-10-06T02:27:37.744 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_pg_recovery_event", 2023-10-06T02:27:37.744 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.744 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.744 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.744 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.745 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.745 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.745 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.745 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow the module to show pg recovery progress", 2023-10-06T02:27:37.745 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.745 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.745 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.745 INFO:teuthology.orchestra.run.smithi031.stdout: "enabled": { 2023-10-06T02:27:37.745 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enabled", 2023-10-06T02:27:37.745 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.745 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.746 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.746 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T02:27:37.746 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.746 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.746 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.746 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.746 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.746 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.746 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.746 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T02:27:37.746 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T02:27:37.747 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.747 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.747 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.747 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.747 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.747 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.747 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.747 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.747 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.747 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.747 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.747 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.748 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.748 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.748 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.748 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.748 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.748 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.748 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T02:27:37.748 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T02:27:37.748 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.748 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.749 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.749 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.749 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.749 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.749 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.749 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.749 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.749 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.749 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.749 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T02:27:37.750 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T02:27:37.750 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.750 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.750 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.750 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T02:27:37.750 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.750 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.750 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.750 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.750 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.750 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.751 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.751 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.751 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.751 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.751 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.751 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.751 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.751 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.751 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T02:27:37.751 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T02:27:37.751 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.752 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.752 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.752 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.752 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.752 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.752 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.752 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.752 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.752 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.752 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.753 INFO:teuthology.orchestra.run.smithi031.stdout: "max_completed_events": { 2023-10-06T02:27:37.753 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_completed_events", 2023-10-06T02:27:37.753 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T02:27:37.753 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.753 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.753 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "50", 2023-10-06T02:27:37.753 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.753 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.753 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.753 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "number of past completed events to remember", 2023-10-06T02:27:37.753 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.753 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.753 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.754 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2023-10-06T02:27:37.754 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2023-10-06T02:27:37.754 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T02:27:37.754 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.754 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.754 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2023-10-06T02:27:37.754 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.754 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.754 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.754 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long the module is going to sleep", 2023-10-06T02:27:37.754 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.755 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.755 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.755 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.755 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:37.755 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prometheus", 2023-10-06T02:27:37.755 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T02:27:37.755 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T02:27:37.755 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T02:27:37.755 INFO:teuthology.orchestra.run.smithi031.stdout: "cache": { 2023-10-06T02:27:37.755 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cache", 2023-10-06T02:27:37.755 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.756 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.756 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.756 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T02:27:37.756 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.756 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.756 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.756 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.756 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.756 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.756 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.756 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T02:27:37.757 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T02:27:37.757 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.757 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.757 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.757 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.757 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.757 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.757 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.757 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.757 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.757 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.757 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.758 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.758 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.758 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.758 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.758 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.758 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.758 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.758 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T02:27:37.758 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T02:27:37.758 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.758 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.759 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.759 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.759 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.759 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.759 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.759 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.759 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.759 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.759 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.759 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T02:27:37.759 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T02:27:37.760 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.760 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.760 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.760 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T02:27:37.760 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.760 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.760 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.760 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.760 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.760 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.760 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.760 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.761 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.761 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.761 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.761 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.761 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.761 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.761 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T02:27:37.761 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T02:27:37.761 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.761 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.761 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.762 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.762 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.762 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.762 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.762 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.762 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.762 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.762 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.762 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_stats_pools": { 2023-10-06T02:27:37.762 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_stats_pools", 2023-10-06T02:27:37.762 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.763 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.763 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.763 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.763 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.763 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.763 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.763 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.763 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.763 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.763 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.763 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_stats_pools_refresh_interval": { 2023-10-06T02:27:37.763 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-10-06T02:27:37.764 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T02:27:37.764 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.764 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.764 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "300", 2023-10-06T02:27:37.764 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.764 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.764 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.764 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.764 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.764 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.764 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.764 INFO:teuthology.orchestra.run.smithi031.stdout: "scrape_interval": { 2023-10-06T02:27:37.765 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "scrape_interval", 2023-10-06T02:27:37.765 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-10-06T02:27:37.765 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.765 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.765 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15.0", 2023-10-06T02:27:37.765 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.765 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.765 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.765 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.765 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.765 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.765 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.766 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2023-10-06T02:27:37.766 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2023-10-06T02:27:37.766 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.766 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.766 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.766 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "::", 2023-10-06T02:27:37.766 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.766 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.766 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.766 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-10-06T02:27:37.766 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.767 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.767 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.767 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2023-10-06T02:27:37.767 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2023-10-06T02:27:37.767 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T02:27:37.767 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.767 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.767 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "9283", 2023-10-06T02:27:37.767 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.767 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.767 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.768 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-10-06T02:27:37.768 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.768 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.768 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.768 INFO:teuthology.orchestra.run.smithi031.stdout: "stale_cache_strategy": { 2023-10-06T02:27:37.768 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "stale_cache_strategy", 2023-10-06T02:27:37.768 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.768 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.768 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.768 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "log", 2023-10-06T02:27:37.769 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.769 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.769 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.769 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.769 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.769 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.769 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.769 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_behaviour": { 2023-10-06T02:27:37.769 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_behaviour", 2023-10-06T02:27:37.769 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.769 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.770 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.770 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "default", 2023-10-06T02:27:37.770 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.770 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.770 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.770 INFO:teuthology.orchestra.run.smithi031.stdout: "default", 2023-10-06T02:27:37.770 INFO:teuthology.orchestra.run.smithi031.stdout: "error" 2023-10-06T02:27:37.770 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.770 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.770 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.770 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.770 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.771 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_error_status_code": { 2023-10-06T02:27:37.771 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_error_status_code", 2023-10-06T02:27:37.771 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T02:27:37.771 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.771 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.771 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "500", 2023-10-06T02:27:37.771 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "400", 2023-10-06T02:27:37.771 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "599", 2023-10-06T02:27:37.771 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.771 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.771 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.771 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.772 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.772 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.772 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.772 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.772 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:37.772 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_support", 2023-10-06T02:27:37.772 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T02:27:37.772 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T02:27:37.772 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T02:27:37.772 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T02:27:37.772 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T02:27:37.772 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.773 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.773 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.773 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.773 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.773 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.773 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.773 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.773 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.773 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.773 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.773 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.773 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.774 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.774 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.774 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.774 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.774 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.774 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T02:27:37.774 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T02:27:37.774 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.774 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.774 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.775 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.775 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.775 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.775 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.775 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.775 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.775 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.775 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.775 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T02:27:37.775 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T02:27:37.775 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.775 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.776 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.776 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T02:27:37.776 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.776 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.776 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.776 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.776 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.776 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.776 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.776 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.776 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.776 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.776 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.777 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.777 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.777 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.777 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T02:27:37.777 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T02:27:37.777 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.777 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.777 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.777 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.777 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.777 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.778 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.778 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.778 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.778 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.778 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.778 INFO:teuthology.orchestra.run.smithi031.stdout: "max_concurrent_snap_create": { 2023-10-06T02:27:37.778 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_concurrent_snap_create", 2023-10-06T02:27:37.778 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T02:27:37.778 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.778 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.778 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-10-06T02:27:37.779 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.779 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.779 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.779 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.779 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.779 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.779 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.779 INFO:teuthology.orchestra.run.smithi031.stdout: "mirror_snapshot_schedule": { 2023-10-06T02:27:37.779 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mirror_snapshot_schedule", 2023-10-06T02:27:37.779 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.779 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.780 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.780 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.780 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.780 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.780 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.780 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.780 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.780 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.780 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.780 INFO:teuthology.orchestra.run.smithi031.stdout: "trash_purge_schedule": { 2023-10-06T02:27:37.780 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "trash_purge_schedule", 2023-10-06T02:27:37.781 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.781 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.781 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.781 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.781 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.781 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.781 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.781 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.781 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.781 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.781 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.782 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.782 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.782 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.782 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:37.782 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "restful", 2023-10-06T02:27:37.782 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T02:27:37.782 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T02:27:37.782 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T02:27:37.782 INFO:teuthology.orchestra.run.smithi031.stdout: "enable_auth": { 2023-10-06T02:27:37.782 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enable_auth", 2023-10-06T02:27:37.782 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.783 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.783 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.783 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T02:27:37.783 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.783 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.783 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.783 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.783 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.783 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.783 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.784 INFO:teuthology.orchestra.run.smithi031.stdout: "key_file": { 2023-10-06T02:27:37.784 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "key_file", 2023-10-06T02:27:37.784 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.784 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.784 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.784 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.784 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.784 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.784 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.784 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.784 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.784 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.785 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.785 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T02:27:37.785 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T02:27:37.785 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.785 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.785 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.785 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.785 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.785 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.785 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.785 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.785 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.786 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.786 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.786 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.786 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.786 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.786 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.786 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.786 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.786 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.786 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T02:27:37.786 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T02:27:37.787 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.787 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.787 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.787 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.787 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.787 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.787 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.787 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.787 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.787 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.787 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.788 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T02:27:37.788 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T02:27:37.788 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.788 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.788 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.788 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T02:27:37.788 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.788 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.788 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.788 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.788 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.788 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.788 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.789 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.789 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.789 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.789 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.789 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.789 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.789 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.789 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T02:27:37.789 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T02:27:37.789 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.789 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.789 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.790 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.790 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.790 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.790 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.790 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.790 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.790 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.790 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.790 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2023-10-06T02:27:37.790 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2023-10-06T02:27:37.790 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.791 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.791 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.791 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.791 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.791 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.791 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.791 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.791 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.791 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.791 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.791 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2023-10-06T02:27:37.791 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2023-10-06T02:27:37.792 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.792 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.792 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.792 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.792 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.792 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.792 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.792 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.792 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.792 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.792 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.792 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.792 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.793 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:37.793 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rgw", 2023-10-06T02:27:37.793 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T02:27:37.793 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T02:27:37.793 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T02:27:37.793 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T02:27:37.793 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T02:27:37.793 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.793 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.793 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.793 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.793 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.794 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.794 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.794 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.794 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.794 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.794 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.794 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.794 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.794 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.794 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.794 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.794 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.794 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.795 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T02:27:37.795 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T02:27:37.795 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.795 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.795 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.795 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.795 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.795 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.795 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.795 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.795 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.796 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.796 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.796 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.796 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T02:27:37.796 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T02:27:37.796 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.796 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.796 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.796 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T02:27:37.796 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.796 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.796 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.796 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.797 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.797 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.797 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.797 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.797 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.797 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.797 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.797 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.797 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.797 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.797 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.797 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T02:27:37.798 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T02:27:37.798 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.798 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.798 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.798 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.798 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.798 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.798 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.798 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.798 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.798 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.799 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.799 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.799 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.799 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.799 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:37.799 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rook", 2023-10-06T02:27:37.799 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T02:27:37.799 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T02:27:37.799 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T02:27:37.799 INFO:teuthology.orchestra.run.smithi031.stdout: "drive_group_interval": { 2023-10-06T02:27:37.799 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "drive_group_interval", 2023-10-06T02:27:37.799 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-10-06T02:27:37.800 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.800 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.800 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "300.0", 2023-10-06T02:27:37.800 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.800 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.800 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.800 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-10-06T02:27:37.800 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.800 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.801 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.801 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.801 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T02:27:37.801 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T02:27:37.801 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.801 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.801 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.801 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.801 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.801 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.801 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.801 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.802 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.802 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.802 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.802 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.802 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.802 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.802 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.802 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.802 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.802 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.802 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.802 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T02:27:37.803 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T02:27:37.803 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.803 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.803 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.803 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.803 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.803 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.803 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.803 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.803 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.803 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.803 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.804 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.804 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T02:27:37.804 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T02:27:37.804 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.804 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.804 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.804 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T02:27:37.804 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.804 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.804 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.804 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.805 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.805 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.805 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.805 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.805 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.805 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.805 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.805 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.805 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.805 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.805 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.805 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T02:27:37.805 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T02:27:37.806 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.806 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.806 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.806 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.806 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.806 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.806 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.806 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.806 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.806 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.806 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.807 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.807 INFO:teuthology.orchestra.run.smithi031.stdout: "storage_class": { 2023-10-06T02:27:37.807 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "storage_class", 2023-10-06T02:27:37.807 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.807 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.807 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.807 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "local", 2023-10-06T02:27:37.807 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.807 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.807 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.807 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-10-06T02:27:37.807 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.808 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.808 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.808 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.808 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.808 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:37.808 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "selftest", 2023-10-06T02:27:37.808 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T02:27:37.808 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T02:27:37.808 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T02:27:37.808 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T02:27:37.808 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T02:27:37.808 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.809 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.809 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.809 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.809 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.809 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.809 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.809 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.809 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.809 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.809 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.809 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.809 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.809 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.810 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.810 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.810 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.810 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.810 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T02:27:37.810 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T02:27:37.810 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.810 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.810 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.810 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.811 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.811 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.811 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.811 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.811 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.811 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.811 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.811 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T02:27:37.811 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T02:27:37.811 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.811 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.811 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.812 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T02:27:37.812 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.812 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.812 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.812 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.812 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.812 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.812 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.812 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.812 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.812 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.812 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.813 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.813 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.813 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.813 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T02:27:37.813 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T02:27:37.813 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.813 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.813 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.813 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.813 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.813 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.814 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.814 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.814 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.814 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.814 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.814 INFO:teuthology.orchestra.run.smithi031.stdout: "roption1": { 2023-10-06T02:27:37.814 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "roption1", 2023-10-06T02:27:37.814 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.814 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.814 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.814 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.815 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.815 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.815 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.815 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.815 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.815 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.815 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.815 INFO:teuthology.orchestra.run.smithi031.stdout: "roption2": { 2023-10-06T02:27:37.815 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "roption2", 2023-10-06T02:27:37.815 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.815 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.815 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.816 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "xyz", 2023-10-06T02:27:37.816 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.816 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.816 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.816 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.816 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.816 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.816 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.816 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption1": { 2023-10-06T02:27:37.816 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption1", 2023-10-06T02:27:37.817 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.817 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.817 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.817 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.817 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.817 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.817 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.817 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.817 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.817 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.817 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.817 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption2": { 2023-10-06T02:27:37.818 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption2", 2023-10-06T02:27:37.818 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T02:27:37.818 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.818 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.818 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.818 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.818 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.818 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.818 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.818 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.818 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.818 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.819 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.819 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption3": { 2023-10-06T02:27:37.819 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption3", 2023-10-06T02:27:37.819 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-10-06T02:27:37.819 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.819 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.819 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.819 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.819 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.819 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.819 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.819 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.820 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.820 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.820 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption4": { 2023-10-06T02:27:37.820 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption4", 2023-10-06T02:27:37.820 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.820 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.820 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.820 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.820 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.820 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.821 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.821 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.821 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.821 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.821 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.821 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption5": { 2023-10-06T02:27:37.821 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption5", 2023-10-06T02:27:37.821 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.821 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.821 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.821 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.821 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.822 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.822 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.822 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.822 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.822 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.822 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.822 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption6": { 2023-10-06T02:27:37.822 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption6", 2023-10-06T02:27:37.822 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.823 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.823 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.823 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T02:27:37.823 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.823 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.823 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.823 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.823 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.823 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.823 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.824 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption7": { 2023-10-06T02:27:37.824 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption7", 2023-10-06T02:27:37.824 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T02:27:37.824 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.824 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.824 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.824 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2023-10-06T02:27:37.824 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "42", 2023-10-06T02:27:37.824 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.824 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.824 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.825 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.825 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.825 INFO:teuthology.orchestra.run.smithi031.stdout: "testkey": { 2023-10-06T02:27:37.825 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testkey", 2023-10-06T02:27:37.825 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.825 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.825 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.825 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.825 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.825 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.826 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.826 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.826 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.826 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.826 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.826 INFO:teuthology.orchestra.run.smithi031.stdout: "testlkey": { 2023-10-06T02:27:37.826 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testlkey", 2023-10-06T02:27:37.826 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.826 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.826 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.827 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.827 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.827 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.827 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.827 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.827 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.827 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.827 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.827 INFO:teuthology.orchestra.run.smithi031.stdout: "testnewline": { 2023-10-06T02:27:37.827 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testnewline", 2023-10-06T02:27:37.827 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.828 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.828 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.828 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.828 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.828 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.828 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.828 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.828 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.828 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.828 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.828 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.828 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:37.829 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snap_schedule", 2023-10-06T02:27:37.829 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T02:27:37.829 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T02:27:37.829 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T02:27:37.829 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_m_granularity": { 2023-10-06T02:27:37.829 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_m_granularity", 2023-10-06T02:27:37.829 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.829 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.829 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.829 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.830 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.830 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.830 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.830 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow minute scheduled snapshots", 2023-10-06T02:27:37.830 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.830 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.830 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.830 INFO:teuthology.orchestra.run.smithi031.stdout: "dump_on_update": { 2023-10-06T02:27:37.830 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "dump_on_update", 2023-10-06T02:27:37.830 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.830 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.831 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.831 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.831 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.831 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.831 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.831 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "dump database to debug log on update", 2023-10-06T02:27:37.831 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.831 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.831 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.831 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T02:27:37.831 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T02:27:37.831 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.831 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.832 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.832 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.832 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.832 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.832 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.832 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.832 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.832 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.832 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.832 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.832 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.832 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.832 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.832 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.833 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.833 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.833 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T02:27:37.833 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T02:27:37.833 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.833 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.833 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.833 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.833 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.833 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.833 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.834 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.834 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.834 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.834 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.834 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T02:27:37.834 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T02:27:37.834 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.834 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.834 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.834 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T02:27:37.834 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.835 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.835 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.835 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.835 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.835 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.835 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.835 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.835 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.835 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.835 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.835 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.835 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.836 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.836 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T02:27:37.836 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T02:27:37.836 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.836 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.836 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.836 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.836 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.836 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.836 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.836 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.836 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.837 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.837 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.837 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.837 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.837 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:37.837 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "stats", 2023-10-06T02:27:37.837 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T02:27:37.837 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T02:27:37.837 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T02:27:37.837 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T02:27:37.837 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T02:27:37.838 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.838 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.838 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.838 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.838 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.838 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.838 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.838 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.838 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.838 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.838 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.839 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.839 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.839 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.839 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.839 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.839 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.839 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.839 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T02:27:37.839 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T02:27:37.839 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.839 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.840 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.840 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.840 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.840 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.840 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.840 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.840 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.840 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.840 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.840 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T02:27:37.840 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T02:27:37.840 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.841 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.841 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.841 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T02:27:37.841 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.841 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.841 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.841 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.841 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.841 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.841 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.841 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.841 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.842 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.842 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.842 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.842 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.842 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.842 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T02:27:37.842 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T02:27:37.842 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.842 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.842 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.842 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.843 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.843 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.843 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.843 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.843 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.843 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.843 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.843 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.843 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:37.844 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "status", 2023-10-06T02:27:37.844 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T02:27:37.844 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T02:27:37.844 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T02:27:37.844 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T02:27:37.844 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T02:27:37.844 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.844 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.844 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.844 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.844 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.844 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.845 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.845 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.845 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.845 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.845 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.845 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.845 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.845 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.845 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.845 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.845 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.846 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.846 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T02:27:37.846 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T02:27:37.846 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.846 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.846 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.846 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.846 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.846 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.846 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.847 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.847 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.847 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.847 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.847 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T02:27:37.847 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T02:27:37.847 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.847 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.847 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.847 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T02:27:37.847 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.847 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.848 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.848 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.848 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.848 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.848 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.848 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.848 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.848 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.848 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.848 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.849 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.849 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.849 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T02:27:37.849 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T02:27:37.849 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.849 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.849 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.849 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.849 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.849 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.849 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.849 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.850 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.850 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.850 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.850 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.850 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:37.850 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "telegraf", 2023-10-06T02:27:37.850 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T02:27:37.850 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T02:27:37.850 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T02:27:37.850 INFO:teuthology.orchestra.run.smithi031.stdout: "address": { 2023-10-06T02:27:37.851 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "address", 2023-10-06T02:27:37.851 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.851 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.851 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.851 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-10-06T02:27:37.851 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.851 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.851 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.851 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.851 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.851 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.851 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.852 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2023-10-06T02:27:37.852 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2023-10-06T02:27:37.852 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T02:27:37.852 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.852 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.852 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15", 2023-10-06T02:27:37.852 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.852 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.852 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.852 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.852 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.853 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.853 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.853 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T02:27:37.853 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T02:27:37.853 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.853 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.853 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.853 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.853 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.853 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.854 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.854 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.854 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.854 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.854 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.854 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.854 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.854 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.854 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.854 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.854 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.854 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.855 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T02:27:37.855 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T02:27:37.855 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.855 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.855 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.855 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.855 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.855 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.855 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.855 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.855 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.856 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.856 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.856 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T02:27:37.856 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T02:27:37.856 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.856 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.856 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.856 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T02:27:37.856 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.856 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.856 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.857 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.857 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.857 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.857 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.857 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.857 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.857 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.857 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.857 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.857 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.857 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.858 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T02:27:37.858 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T02:27:37.858 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.858 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.858 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.858 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.858 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.858 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.858 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.858 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.858 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.859 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.859 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.859 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.859 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:37.859 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "telemetry", 2023-10-06T02:27:37.859 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T02:27:37.859 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T02:27:37.859 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T02:27:37.859 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_basic": { 2023-10-06T02:27:37.859 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_basic", 2023-10-06T02:27:37.860 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.860 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.860 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.860 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T02:27:37.860 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.860 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.860 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.860 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share basic cluster information (size, version)", 2023-10-06T02:27:37.860 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.860 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.861 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.861 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_crash": { 2023-10-06T02:27:37.861 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_crash", 2023-10-06T02:27:37.861 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.861 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.861 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.861 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T02:27:37.861 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.861 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.862 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.862 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-10-06T02:27:37.862 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.862 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.862 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.862 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_device": { 2023-10-06T02:27:37.862 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_device", 2023-10-06T02:27:37.862 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.862 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.863 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.863 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T02:27:37.863 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.863 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.863 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.863 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-10-06T02:27:37.863 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.863 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.863 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.863 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_ident": { 2023-10-06T02:27:37.864 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_ident", 2023-10-06T02:27:37.864 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.864 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.864 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.864 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.864 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.864 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.864 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.864 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-10-06T02:27:37.864 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.865 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.865 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.865 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_perf": { 2023-10-06T02:27:37.865 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_perf", 2023-10-06T02:27:37.865 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.865 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.865 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.865 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.865 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.865 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.866 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.866 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share various performance metrics of a cluster", 2023-10-06T02:27:37.866 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.866 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.866 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.866 INFO:teuthology.orchestra.run.smithi031.stdout: "contact": { 2023-10-06T02:27:37.866 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "contact", 2023-10-06T02:27:37.866 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.866 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.866 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.867 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.867 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.867 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.867 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.867 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.867 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.867 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.867 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.867 INFO:teuthology.orchestra.run.smithi031.stdout: "description": { 2023-10-06T02:27:37.867 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "description", 2023-10-06T02:27:37.868 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.868 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.868 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.868 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.868 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.868 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.868 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.868 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.868 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.868 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.868 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.869 INFO:teuthology.orchestra.run.smithi031.stdout: "device_url": { 2023-10-06T02:27:37.869 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_url", 2023-10-06T02:27:37.869 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.869 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.869 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.869 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-10-06T02:27:37.869 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.869 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.869 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.869 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.870 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.870 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.870 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.870 INFO:teuthology.orchestra.run.smithi031.stdout: "enabled": { 2023-10-06T02:27:37.870 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enabled", 2023-10-06T02:27:37.870 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.870 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.870 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.870 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.870 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.871 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.871 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.871 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.871 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.871 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.871 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.871 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2023-10-06T02:27:37.871 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2023-10-06T02:27:37.871 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T02:27:37.871 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.872 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.872 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "24", 2023-10-06T02:27:37.872 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "8", 2023-10-06T02:27:37.872 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.872 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.872 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.872 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.872 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.872 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.872 INFO:teuthology.orchestra.run.smithi031.stdout: "last_opt_revision": { 2023-10-06T02:27:37.873 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "last_opt_revision", 2023-10-06T02:27:37.873 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T02:27:37.873 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.873 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.873 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1", 2023-10-06T02:27:37.873 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.873 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.873 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.873 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.873 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.873 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.874 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.874 INFO:teuthology.orchestra.run.smithi031.stdout: "leaderboard": { 2023-10-06T02:27:37.874 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "leaderboard", 2023-10-06T02:27:37.874 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.874 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.874 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.874 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.874 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.874 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.874 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.874 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.875 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.875 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.875 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.875 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T02:27:37.875 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T02:27:37.875 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.875 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.875 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.875 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.875 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.875 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.875 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.876 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.876 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.876 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.876 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.876 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.876 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.876 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.876 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.876 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.876 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.876 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.877 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T02:27:37.877 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T02:27:37.877 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.877 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.877 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.877 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.877 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.877 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.877 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.877 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.877 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.877 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.877 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.878 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T02:27:37.878 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T02:27:37.878 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.878 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.878 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.878 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T02:27:37.878 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.878 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.878 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.878 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.879 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.879 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.879 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.879 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.879 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.879 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.879 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.879 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.879 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.879 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.879 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T02:27:37.880 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T02:27:37.880 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.880 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.880 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.880 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.880 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.880 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.880 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.880 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.880 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.880 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.881 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.881 INFO:teuthology.orchestra.run.smithi031.stdout: "organization": { 2023-10-06T02:27:37.881 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "organization", 2023-10-06T02:27:37.881 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.881 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.881 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.881 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.881 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.881 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.881 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.881 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.882 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.882 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.882 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.882 INFO:teuthology.orchestra.run.smithi031.stdout: "proxy": { 2023-10-06T02:27:37.882 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "proxy", 2023-10-06T02:27:37.882 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.882 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.882 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.882 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.882 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.882 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.883 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.883 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.883 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.883 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.883 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.883 INFO:teuthology.orchestra.run.smithi031.stdout: "url": { 2023-10-06T02:27:37.883 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "url", 2023-10-06T02:27:37.883 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.883 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.883 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.884 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-10-06T02:27:37.884 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.884 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.884 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.884 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.884 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.884 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.884 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.884 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.884 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:37.884 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "test_orchestrator", 2023-10-06T02:27:37.885 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T02:27:37.885 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T02:27:37.885 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T02:27:37.885 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T02:27:37.885 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T02:27:37.885 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.885 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.885 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.885 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.885 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.885 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.885 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.885 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.885 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.886 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.886 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.886 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.886 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.886 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.886 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.886 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.886 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.886 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.886 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T02:27:37.886 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T02:27:37.887 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.887 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.887 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.887 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.887 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.887 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.887 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.887 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.887 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.887 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.887 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.888 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T02:27:37.888 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T02:27:37.888 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.888 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.888 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.888 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T02:27:37.888 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.888 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.888 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.888 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.888 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.888 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.888 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.889 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.889 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.889 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.889 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.889 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.889 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.889 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.889 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T02:27:37.889 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T02:27:37.889 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.889 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.890 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.890 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.890 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.890 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.890 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.890 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.890 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.890 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.890 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.890 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.890 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:37.891 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "volumes", 2023-10-06T02:27:37.891 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T02:27:37.891 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T02:27:37.891 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T02:27:37.891 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T02:27:37.891 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T02:27:37.891 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.891 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.891 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.891 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.891 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.891 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.892 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.892 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.892 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.892 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.892 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.892 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.892 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.892 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.892 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.892 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.892 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.892 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.892 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T02:27:37.893 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T02:27:37.893 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.893 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.893 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.893 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.893 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.893 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.893 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.893 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.893 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.893 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.893 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.894 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T02:27:37.894 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T02:27:37.894 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.894 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.894 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.894 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T02:27:37.894 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.894 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.894 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.894 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.894 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.894 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.895 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.895 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.895 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.895 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.895 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.895 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.895 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.895 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.895 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T02:27:37.895 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T02:27:37.895 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.895 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.896 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.896 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.896 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.896 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.896 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.896 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.896 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.896 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.896 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.896 INFO:teuthology.orchestra.run.smithi031.stdout: "max_concurrent_clones": { 2023-10-06T02:27:37.896 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_concurrent_clones", 2023-10-06T02:27:37.896 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T02:27:37.896 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.897 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.897 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "4", 2023-10-06T02:27:37.897 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.897 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.897 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.897 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Number of asynchronous cloner threads", 2023-10-06T02:27:37.897 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.897 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.897 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.897 INFO:teuthology.orchestra.run.smithi031.stdout: "snapshot_clone_delay": { 2023-10-06T02:27:37.897 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snapshot_clone_delay", 2023-10-06T02:27:37.897 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T02:27:37.898 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.898 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.898 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2023-10-06T02:27:37.898 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.898 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.898 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.898 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-10-06T02:27:37.898 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.898 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.898 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.898 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.898 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:37.899 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix", 2023-10-06T02:27:37.899 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T02:27:37.899 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T02:27:37.899 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T02:27:37.899 INFO:teuthology.orchestra.run.smithi031.stdout: "discovery_interval": { 2023-10-06T02:27:37.899 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "discovery_interval", 2023-10-06T02:27:37.899 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2023-10-06T02:27:37.899 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.899 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.899 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "100", 2023-10-06T02:27:37.899 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.899 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.900 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.900 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.900 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.900 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.900 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.900 INFO:teuthology.orchestra.run.smithi031.stdout: "identifier": { 2023-10-06T02:27:37.900 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "identifier", 2023-10-06T02:27:37.900 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.900 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.900 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.900 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.900 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.901 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.901 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.901 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.901 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.901 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.901 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.901 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2023-10-06T02:27:37.901 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2023-10-06T02:27:37.901 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T02:27:37.901 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.901 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.902 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2023-10-06T02:27:37.902 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.902 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.902 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.902 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.902 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.902 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.902 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.902 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T02:27:37.902 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T02:27:37.902 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.903 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.903 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.903 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.903 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.903 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.903 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.903 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.903 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.903 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.903 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.903 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.904 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.904 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.904 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.904 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.904 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.904 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.904 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T02:27:37.904 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T02:27:37.904 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.904 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.904 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.904 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.905 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.905 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.905 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.905 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.905 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.905 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.905 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.905 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T02:27:37.905 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T02:27:37.905 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.905 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.905 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.906 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T02:27:37.906 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.906 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.906 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T02:27:37.906 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T02:27:37.906 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T02:27:37.906 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T02:27:37.906 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T02:27:37.906 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T02:27:37.906 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T02:27:37.906 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.906 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.907 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.907 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.907 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.907 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T02:27:37.907 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T02:27:37.907 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T02:27:37.907 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.907 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T02:27:37.907 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T02:27:37.907 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.907 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.907 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.908 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.908 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.908 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.908 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.908 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_host": { 2023-10-06T02:27:37.908 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_host", 2023-10-06T02:27:37.908 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.908 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.908 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.908 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T02:27:37.908 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.908 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.909 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.909 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.909 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.909 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.909 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.909 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_port": { 2023-10-06T02:27:37.909 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_port", 2023-10-06T02:27:37.909 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T02:27:37.909 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.909 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.909 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10051", 2023-10-06T02:27:37.909 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.910 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.910 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.910 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.910 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.910 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.910 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.910 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_sender": { 2023-10-06T02:27:37.910 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_sender", 2023-10-06T02:27:37.910 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T02:27:37.910 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T02:27:37.910 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T02:27:37.911 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-10-06T02:27:37.911 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T02:27:37.911 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T02:27:37.911 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T02:27:37.911 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T02:27:37.911 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T02:27:37.911 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T02:27:37.911 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T02:27:37.911 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.911 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.911 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.911 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.912 INFO:teuthology.orchestra.run.smithi031.stdout: "services": {}, 2023-10-06T02:27:37.912 INFO:teuthology.orchestra.run.smithi031.stdout: "always_on_modules": { 2023-10-06T02:27:37.912 INFO:teuthology.orchestra.run.smithi031.stdout: "octopus": [ 2023-10-06T02:27:37.912 INFO:teuthology.orchestra.run.smithi031.stdout: "balancer", 2023-10-06T02:27:37.912 INFO:teuthology.orchestra.run.smithi031.stdout: "crash", 2023-10-06T02:27:37.912 INFO:teuthology.orchestra.run.smithi031.stdout: "devicehealth", 2023-10-06T02:27:37.912 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator", 2023-10-06T02:27:37.912 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscaler", 2023-10-06T02:27:37.912 INFO:teuthology.orchestra.run.smithi031.stdout: "progress", 2023-10-06T02:27:37.912 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_support", 2023-10-06T02:27:37.912 INFO:teuthology.orchestra.run.smithi031.stdout: "status", 2023-10-06T02:27:37.913 INFO:teuthology.orchestra.run.smithi031.stdout: "telemetry", 2023-10-06T02:27:37.913 INFO:teuthology.orchestra.run.smithi031.stdout: "volumes" 2023-10-06T02:27:37.913 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.913 INFO:teuthology.orchestra.run.smithi031.stdout: "pacific": [ 2023-10-06T02:27:37.913 INFO:teuthology.orchestra.run.smithi031.stdout: "balancer", 2023-10-06T02:27:37.913 INFO:teuthology.orchestra.run.smithi031.stdout: "crash", 2023-10-06T02:27:37.913 INFO:teuthology.orchestra.run.smithi031.stdout: "devicehealth", 2023-10-06T02:27:37.913 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator", 2023-10-06T02:27:37.913 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscaler", 2023-10-06T02:27:37.913 INFO:teuthology.orchestra.run.smithi031.stdout: "progress", 2023-10-06T02:27:37.913 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_support", 2023-10-06T02:27:37.913 INFO:teuthology.orchestra.run.smithi031.stdout: "status", 2023-10-06T02:27:37.914 INFO:teuthology.orchestra.run.smithi031.stdout: "telemetry", 2023-10-06T02:27:37.914 INFO:teuthology.orchestra.run.smithi031.stdout: "volumes" 2023-10-06T02:27:37.914 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:27:37.914 INFO:teuthology.orchestra.run.smithi031.stdout: "quincy": [ 2023-10-06T02:27:37.914 INFO:teuthology.orchestra.run.smithi031.stdout: "balancer", 2023-10-06T02:27:37.914 INFO:teuthology.orchestra.run.smithi031.stdout: "crash", 2023-10-06T02:27:37.914 INFO:teuthology.orchestra.run.smithi031.stdout: "devicehealth", 2023-10-06T02:27:37.914 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator", 2023-10-06T02:27:37.914 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscaler", 2023-10-06T02:27:37.914 INFO:teuthology.orchestra.run.smithi031.stdout: "progress", 2023-10-06T02:27:37.914 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_support", 2023-10-06T02:27:37.914 INFO:teuthology.orchestra.run.smithi031.stdout: "status", 2023-10-06T02:27:37.915 INFO:teuthology.orchestra.run.smithi031.stdout: "telemetry", 2023-10-06T02:27:37.915 INFO:teuthology.orchestra.run.smithi031.stdout: "volumes" 2023-10-06T02:27:37.915 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-10-06T02:27:37.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.915 INFO:teuthology.orchestra.run.smithi031.stdout: "last_failure_osd_epoch": 0, 2023-10-06T02:27:37.915 INFO:teuthology.orchestra.run.smithi031.stdout: "active_clients": [ 2023-10-06T02:27:37.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:37.915 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "devicehealth", 2023-10-06T02:27:37.915 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2023-10-06T02:27:37.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:37.915 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2023-10-06T02:27:37.915 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:0", 2023-10-06T02:27:37.916 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2607073168 2023-10-06T02:27:37.916 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.916 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-10-06T02:27:37.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:37.916 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "libcephsqlite", 2023-10-06T02:27:37.916 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2023-10-06T02:27:37.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:37.916 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2023-10-06T02:27:37.916 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:0", 2023-10-06T02:27:37.916 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3265960029 2023-10-06T02:27:37.916 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.916 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-10-06T02:27:37.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:37.917 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_support", 2023-10-06T02:27:37.917 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2023-10-06T02:27:37.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:37.917 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2023-10-06T02:27:37.917 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:0", 2023-10-06T02:27:37.917 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2878303473 2023-10-06T02:27:37.917 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.917 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-10-06T02:27:37.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:37.917 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "volumes", 2023-10-06T02:27:37.918 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2023-10-06T02:27:37.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:37.918 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2023-10-06T02:27:37.918 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:0", 2023-10-06T02:27:37.918 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3769130347 2023-10-06T02:27:37.918 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.918 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-10-06T02:27:37.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:37.918 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "volumes", 2023-10-06T02:27:37.918 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2023-10-06T02:27:37.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:37.918 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2023-10-06T02:27:37.919 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:0", 2023-10-06T02:27:37.919 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2430615067 2023-10-06T02:27:37.919 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.919 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-10-06T02:27:37.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:37.919 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "volumes", 2023-10-06T02:27:37.919 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2023-10-06T02:27:37.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:37.919 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2023-10-06T02:27:37.919 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:0", 2023-10-06T02:27:37.919 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2186321038 2023-10-06T02:27:37.919 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.920 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-10-06T02:27:37.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:37.920 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "volumes", 2023-10-06T02:27:37.920 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2023-10-06T02:27:37.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:37.920 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2023-10-06T02:27:37.920 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:0", 2023-10-06T02:27:37.920 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3308984790 2023-10-06T02:27:37.920 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.920 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-10-06T02:27:37.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:37.921 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "volumes", 2023-10-06T02:27:37.921 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2023-10-06T02:27:37.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:37.921 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2023-10-06T02:27:37.921 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:0", 2023-10-06T02:27:37.921 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2627562379 2023-10-06T02:27:37.921 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.921 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-10-06T02:27:37.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:37.922 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "volumes", 2023-10-06T02:27:37.922 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2023-10-06T02:27:37.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:37.922 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2023-10-06T02:27:37.922 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:0", 2023-10-06T02:27:37.922 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1519622771 2023-10-06T02:27:37.922 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.922 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-10-06T02:27:37.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:37.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:37.922 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "volumes", 2023-10-06T02:27:37.923 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2023-10-06T02:27:37.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:37.923 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2023-10-06T02:27:37.923 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:0", 2023-10-06T02:27:37.923 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3343749855 2023-10-06T02:27:37.923 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.923 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-10-06T02:27:37.923 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:37.923 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-10-06T02:27:37.923 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T02:27:37.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.939+0000 7f2004f9d700 1 -- 172.21.15.31:0/2304334503 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f1fec067290 msgr2=0x7f1fec069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:37.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.939+0000 7f2004f9d700 1 --2- 172.21.15.31:0/2304334503 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f1fec067290 0x7f1fec069740 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f1ff0001200 tx=0x7f1ff0009040 comp rx=0 tx=0).stop 2023-10-06T02:27:37.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.939+0000 7f2004f9d700 1 -- 172.21.15.31:0/2304334503 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20000b08d0 msgr2=0x7f20000b0d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:37.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.939+0000 7f2004f9d700 1 --2- 172.21.15.31:0/2304334503 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20000b08d0 0x7f20000b0d70 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f20001507c0 tx=0x7f1ff800b040 comp rx=0 tx=0).stop 2023-10-06T02:27:37.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.939+0000 7f2004f9d700 1 -- 172.21.15.31:0/2304334503 shutdown_connections 2023-10-06T02:27:37.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.939+0000 7f2004f9d700 1 --2- 172.21.15.31:0/2304334503 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20000b08d0 0x7f20000b0d70 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:37.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.939+0000 7f2004f9d700 1 --2- 172.21.15.31:0/2304334503 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1fe80052e0 0x7f200016ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:37.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.939+0000 7f2004f9d700 1 --2- 172.21.15.31:0/2304334503 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f1fec067290 0x7f1fec069740 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:37.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.939+0000 7f2004f9d700 1 --2- 172.21.15.31:0/2304334503 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2000139900 0x7f200016b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:37.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.939+0000 7f2004f9d700 1 -- 172.21.15.31:0/2304334503 >> 172.21.15.31:0/2304334503 conn(0x7f200006d170 msgr2=0x7f200014e9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:37.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.939+0000 7f2004f9d700 1 -- 172.21.15.31:0/2304334503 shutdown_connections 2023-10-06T02:27:37.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:36.939+0000 7f2004f9d700 1 -- 172.21.15.31:0/2304334503 wait complete. 2023-10-06T02:27:37.929 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr fail y 2023-10-06T02:27:38.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.130+0000 7f597939d700 1 Processor -- start 2023-10-06T02:27:38.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.130+0000 7f597939d700 1 -- start start 2023-10-06T02:27:38.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.130+0000 7f597939d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f597413fb20 0x7f597415a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:38.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.130+0000 7f597939d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f597406aaa0 0x7f59740c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:38.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.130+0000 7f597939d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f597415ae70 con 0x7f597413fef0 2023-10-06T02:27:38.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.130+0000 7f597939d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f597406af60 con 0x7f597413fb20 2023-10-06T02:27:38.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.130+0000 7f597939d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f597406b240 con 0x7f597406aaa0 2023-10-06T02:27:38.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.130+0000 7f59737fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f597413fef0 0x7f59740c7990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:37406/0 (socket says 172.21.15.31:37406) 2023-10-06T02:27:38.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.130+0000 7f59737fe700 1 -- 172.21.15.31:0/3786474598 learned_addr learned my addr 172.21.15.31:0/3786474598 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:27:38.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.131+0000 7f5978b9c700 1 --2- 172.21.15.31:0/3786474598 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f597406aaa0 0x7f59740c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:38.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.131+0000 7f5973fff700 1 --2- 172.21.15.31:0/3786474598 >> v2:172.21.15.62:3300/0 conn(0x7f597413fb20 0x7f597415a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:38.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.131+0000 7f5972ffd700 1 -- 172.21.15.31:0/3786474598 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1569129883 0 0) 0x7f5968001290 con 0x7f597413fef0 2023-10-06T02:27:38.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.131+0000 7f5972ffd700 1 -- 172.21.15.31:0/3786474598 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f595c003570 con 0x7f597413fef0 2023-10-06T02:27:38.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.131+0000 7f5972ffd700 1 -- 172.21.15.31:0/3786474598 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2724393744 0 0) 0x7f595c003570 con 0x7f597413fef0 2023-10-06T02:27:38.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.131+0000 7f5972ffd700 1 -- 172.21.15.31:0/3786474598 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f597406aaa0 msgr2=0x7f59740c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:38.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.131+0000 7f5972ffd700 1 --2- 172.21.15.31:0/3786474598 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f597406aaa0 0x7f59740c9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:38.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.131+0000 7f5972ffd700 1 -- 172.21.15.31:0/3786474598 >> v2:172.21.15.62:3300/0 conn(0x7f597413fb20 msgr2=0x7f597415a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:38.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.131+0000 7f5972ffd700 1 --2- 172.21.15.31:0/3786474598 >> v2:172.21.15.62:3300/0 conn(0x7f597413fb20 0x7f597415a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:38.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.131+0000 7f5972ffd700 1 -- 172.21.15.31:0/3786474598 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f597406b520 con 0x7f597413fef0 2023-10-06T02:27:38.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.131+0000 7f5972ffd700 1 -- 172.21.15.31:0/3786474598 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7f5968001780 con 0x7f597413fef0 2023-10-06T02:27:38.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.131+0000 7f5972ffd700 1 -- 172.21.15.31:0/3786474598 >> v1:172.21.15.31:6789/0 conn(0x7f597413fef0 legacy=0x7f59740c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:38.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.131+0000 7f5972ffd700 1 --2- 172.21.15.31:0/3786474598 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f595c003a60 0x7f595c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:38.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.131+0000 7f5972ffd700 1 --2- 172.21.15.31:0/3786474598 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f597406aaa0 0x7f595c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:38.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.131+0000 7f5972ffd700 1 --2- 172.21.15.31:0/3786474598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f595c004d50 0x7f595c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:38.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.131+0000 7f5972ffd700 1 -- 172.21.15.31:0/3786474598 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f597406af60 con 0x7f595c004d50 2023-10-06T02:27:38.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.131+0000 7f5972ffd700 1 -- 172.21.15.31:0/3786474598 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f597406b240 con 0x7f595c003a60 2023-10-06T02:27:38.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.131+0000 7f5972ffd700 1 -- 172.21.15.31:0/3786474598 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f595c00dca0 con 0x7f597406aaa0 2023-10-06T02:27:38.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.132+0000 7f5978b9c700 1 --2- 172.21.15.31:0/3786474598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f595c004d50 0x7f595c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:38.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.132+0000 7f5973fff700 1 --2- 172.21.15.31:0/3786474598 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f595c003a60 0x7f595c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:38.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.132+0000 7f59737fe700 1 --2- 172.21.15.31:0/3786474598 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f597406aaa0 0x7f595c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:38.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.132+0000 7f5978b9c700 1 -- 172.21.15.31:0/3786474598 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f597406aaa0 msgr2=0x7f595c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:38.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.132+0000 7f5978b9c700 1 --2- 172.21.15.31:0/3786474598 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f597406aaa0 0x7f595c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:38.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.132+0000 7f5978b9c700 1 -- 172.21.15.31:0/3786474598 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f595c003a60 msgr2=0x7f595c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:38.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.132+0000 7f5978b9c700 1 --2- 172.21.15.31:0/3786474598 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f595c003a60 0x7f595c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:38.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.132+0000 7f5978b9c700 1 -- 172.21.15.31:0/3786474598 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f595c003570 con 0x7f595c004d50 2023-10-06T02:27:38.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.132+0000 7f5978b9c700 1 --2- 172.21.15.31:0/3786474598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f595c004d50 0x7f595c0051c0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f597406ae90 tx=0x7f596c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94ad30ec1a1b4c44 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:38.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.132+0000 7f5972ffd700 1 -- 172.21.15.31:0/3786474598 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f596c00ea30 con 0x7f595c004d50 2023-10-06T02:27:38.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.132+0000 7f5972ffd700 1 -- 172.21.15.31:0/3786474598 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f596c00f040 con 0x7f595c004d50 2023-10-06T02:27:38.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.132+0000 7f5972ffd700 1 -- 172.21.15.31:0/3786474598 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f596c00ede0 con 0x7f595c004d50 2023-10-06T02:27:38.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.133+0000 7f597939d700 1 -- 172.21.15.31:0/3786474598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f595c004d50 msgr2=0x7f595c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:38.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.133+0000 7f597939d700 1 --2- 172.21.15.31:0/3786474598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f595c004d50 0x7f595c0051c0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f597406ae90 tx=0x7f596c00b040 comp rx=0 tx=0).stop 2023-10-06T02:27:38.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.133+0000 7f5978b9c700 1 -- 172.21.15.31:0/3786474598 reap_dead start 2023-10-06T02:27:38.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.133+0000 7f597939d700 1 -- 172.21.15.31:0/3786474598 shutdown_connections 2023-10-06T02:27:38.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.133+0000 7f597939d700 1 -- 172.21.15.31:0/3786474598 >> 172.21.15.31:0/3786474598 conn(0x7f597406d170 msgr2=0x7f59740cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:38.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.133+0000 7f597939d700 1 -- 172.21.15.31:0/3786474598 shutdown_connections 2023-10-06T02:27:38.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.133+0000 7f597939d700 1 -- 172.21.15.31:0/3786474598 wait complete. 2023-10-06T02:27:38.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.135+0000 7f597939d700 1 Processor -- start 2023-10-06T02:27:38.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.135+0000 7f597939d700 1 -- start start 2023-10-06T02:27:38.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.136+0000 7f597939d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f595c004d50 0x7f59740b1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:38.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.136+0000 7f597939d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59740b1990 0x7f597416a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:38.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.136+0000 7f597939d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f597413a890 0x7f597413ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:38.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.136+0000 7f597939d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f597413b240 con 0x7f595c004d50 2023-10-06T02:27:38.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.136+0000 7f597939d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5974155d80 con 0x7f59740b1990 2023-10-06T02:27:38.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.136+0000 7f597939d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5974138fa0 con 0x7f597413a890 2023-10-06T02:27:38.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.136+0000 7f5973fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f595c004d50 0x7f59740b1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:38.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.136+0000 7f5973fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f595c004d50 0x7f59740b1000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50786/0 (socket says 172.21.15.31:50786) 2023-10-06T02:27:38.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.136+0000 7f5973fff700 1 -- 172.21.15.31:0/2729492711 learned_addr learned my addr 172.21.15.31:0/2729492711 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:38.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.136+0000 7f59737fe700 1 --2- 172.21.15.31:0/2729492711 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59740b1990 0x7f597416a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:38.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.136+0000 7f5978b9c700 1 --2- 172.21.15.31:0/2729492711 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f597413a890 0x7f597413ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:38.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.136+0000 7f5973fff700 1 -- 172.21.15.31:0/2729492711 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f597413a890 msgr2=0x7f597413ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:38.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.136+0000 7f5973fff700 1 --2- 172.21.15.31:0/2729492711 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f597413a890 0x7f597413ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:38.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.136+0000 7f5973fff700 1 -- 172.21.15.31:0/2729492711 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59740b1990 msgr2=0x7f597416a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:38.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.136+0000 7f5973fff700 1 --2- 172.21.15.31:0/2729492711 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59740b1990 0x7f597416a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:38.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.136+0000 7f5973fff700 1 -- 172.21.15.31:0/2729492711 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5974139280 con 0x7f595c004d50 2023-10-06T02:27:38.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.136+0000 7f5973fff700 1 --2- 172.21.15.31:0/2729492711 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f595c004d50 0x7f59740b1000 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f5964006ac0 tx=0x7f59640091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:38.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.137+0000 7f595bfff700 1 -- 172.21.15.31:0/2729492711 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f596400fa10 con 0x7f595c004d50 2023-10-06T02:27:38.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.137+0000 7f595bfff700 1 -- 172.21.15.31:0/2729492711 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f596400fb70 con 0x7f595c004d50 2023-10-06T02:27:38.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.137+0000 7f597939d700 1 -- 172.21.15.31:0/2729492711 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5974139560 con 0x7f595c004d50 2023-10-06T02:27:38.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.137+0000 7f595bfff700 1 -- 172.21.15.31:0/2729492711 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5964014440 con 0x7f595c004d50 2023-10-06T02:27:38.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.137+0000 7f597939d700 1 -- 172.21.15.31:0/2729492711 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59740b1540 con 0x7f595c004d50 2023-10-06T02:27:38.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.138+0000 7f597939d700 1 -- 172.21.15.31:0/2729492711 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f597416a000 con 0x7f595c004d50 2023-10-06T02:27:38.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.139+0000 7f595bfff700 1 -- 172.21.15.31:0/2729492711 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f5964014830 con 0x7f595c004d50 2023-10-06T02:27:38.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.140+0000 7f595bfff700 1 --2- 172.21.15.31:0/2729492711 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5960067350 0x7f5960069800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:38.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.140+0000 7f595bfff700 1 -- 172.21.15.31:0/2729492711 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f5964018030 con 0x7f595c004d50 2023-10-06T02:27:38.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.140+0000 7f59737fe700 1 --2- 172.21.15.31:0/2729492711 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5960067350 0x7f5960069800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:38.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.140+0000 7f59737fe700 1 --2- 172.21.15.31:0/2729492711 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5960067350 0x7f5960069800 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f59680018b0 tx=0x7f5968009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:38.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.144+0000 7f595bfff700 1 -- 172.21.15.31:0/2729492711 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5964052340 con 0x7f595c004d50 2023-10-06T02:27:38.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.404+0000 7f597939d700 1 -- 172.21.15.31:0/2729492711 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mgr fail", "who": "y"} v 0) v1 -- 0x7f597404c3c0 con 0x7f595c004d50 2023-10-06T02:27:38.411 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:27:38.410+0000 7f3ef51f5700 -1 mgr handle_mgr_map I was active but no longer am 2023-10-06T02:27:38.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.413+0000 7f595bfff700 1 -- 172.21.15.31:0/2729492711 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mgrmap(e 14) v1 ==== 42252+0+0 (secure 0 0 0) 0x7f5964052710 con 0x7f595c004d50 2023-10-06T02:27:38.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.413+0000 7f595bfff700 1 -- 172.21.15.31:0/2729492711 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5960067350 msgr2=0x7f5960069800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:38.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.413+0000 7f595bfff700 1 --2- 172.21.15.31:0/2729492711 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5960067350 0x7f5960069800 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f59680018b0 tx=0x7f5968009040 comp rx=0 tx=0).stop 2023-10-06T02:27:38.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.414+0000 7f595bfff700 1 -- 172.21.15.31:0/2729492711 <== mon.0 v2:172.21.15.31:3300/0 8 ==== mon_command_ack([{"prefix": "mgr fail", "who": "y"}]=0 v14) v1 ==== 68+0+0 (secure 0 0 0) 0x7f59640543a0 con 0x7f595c004d50 2023-10-06T02:27:38.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.418+0000 7f597939d700 1 -- 172.21.15.31:0/2729492711 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f595c004d50 msgr2=0x7f59740b1000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:38.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.418+0000 7f597939d700 1 --2- 172.21.15.31:0/2729492711 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f595c004d50 0x7f59740b1000 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f5964006ac0 tx=0x7f59640091a0 comp rx=0 tx=0).stop 2023-10-06T02:27:38.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.418+0000 7f597939d700 1 -- 172.21.15.31:0/2729492711 shutdown_connections 2023-10-06T02:27:38.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.418+0000 7f597939d700 1 --2- 172.21.15.31:0/2729492711 >> [v2:172.21.15.31:6832/85292,v1:172.21.15.31:6833/85292] conn(0x7f5960067350 0x7f5960069800 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:38.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.418+0000 7f597939d700 1 --2- 172.21.15.31:0/2729492711 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f597413a890 0x7f597413ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:38.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.418+0000 7f597939d700 1 --2- 172.21.15.31:0/2729492711 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f595c004d50 0x7f59740b1000 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:38.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.418+0000 7f597939d700 1 --2- 172.21.15.31:0/2729492711 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59740b1990 0x7f597416a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:38.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.418+0000 7f597939d700 1 -- 172.21.15.31:0/2729492711 >> 172.21.15.31:0/2729492711 conn(0x7f597406d170 msgr2=0x7f597413c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:38.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.418+0000 7f597939d700 1 -- 172.21.15.31:0/2729492711 shutdown_connections 2023-10-06T02:27:38.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.418+0000 7f597939d700 1 -- 172.21.15.31:0/2729492711 wait complete. 2023-10-06T02:27:38.446 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:27:38.522 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:27:38.521+0000 7f6567b82700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:27:38.523 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:27:38.521+0000 7f6567b82700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:27:38.523 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:27:38.521+0000 7f6567b82700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:27:38.523 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:27:38.521+0000 7f6567b82700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:27:38.523 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:27:38.521+0000 7f6567b82700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:27:38.523 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:27:38.522+0000 7f656b389700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:27:38.523 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:27:38.522+0000 7f656b389700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:27:38.523 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:27:38.522+0000 7f656b389700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:27:38.523 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:27:38.522+0000 7f656b389700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:27:38.523 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:27:38.522+0000 7f656b389700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:27:38.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.667+0000 7f8cdd79d700 1 Processor -- start 2023-10-06T02:27:38.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.667+0000 7f8cdd79d700 1 -- start start 2023-10-06T02:27:38.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.668+0000 7f8cdd79d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8cd8159550 0x7f8cd80ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:38.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.668+0000 7f8cdd79d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cd80acc00 con 0x7f8cd8159920 2023-10-06T02:27:38.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.668+0000 7f8cdd79d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cd80abff0 con 0x7f8cd815dfd0 2023-10-06T02:27:38.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.668+0000 7f8cdd79d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd80ac2d0 con 0x7f8cd8159550 2023-10-06T02:27:38.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.669+0000 7f8cdcf9c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f8cd8159920 0x7f8cd80ad090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:37408/0 (socket says 172.21.15.31:37408) 2023-10-06T02:27:38.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.669+0000 7f8cdcf9c700 1 -- 172.21.15.31:0/476308806 learned_addr learned my addr 172.21.15.31:0/476308806 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:27:38.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.669+0000 7f8cd7fff700 1 --2- 172.21.15.31:0/476308806 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8cd8159550 0x7f8cd80ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:38.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.669+0000 7f8cd6ffd700 1 -- 172.21.15.31:0/476308806 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4142634685 0 0) 0x7f8cd80acc00 con 0x7f8cd8159920 2023-10-06T02:27:38.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.669+0000 7f8cd6ffd700 1 -- 172.21.15.31:0/476308806 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8cc0003570 con 0x7f8cd8159920 2023-10-06T02:27:38.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.669+0000 7f8cd6ffd700 1 -- 172.21.15.31:0/476308806 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3622580445 0 0) 0x7f8cd80abff0 con 0x7f8cd815dfd0 2023-10-06T02:27:38.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.669+0000 7f8cd7fff700 1 -- 172.21.15.31:0/476308806 >> v1:172.21.15.62:6789/0 conn(0x7f8cd815dfd0 legacy=0x7f8cd80aedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:38.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.669+0000 7f8cd7fff700 1 -- 172.21.15.31:0/476308806 >> v1:172.21.15.31:6789/0 conn(0x7f8cd8159920 legacy=0x7f8cd80ad090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:38.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.669+0000 7f8cd7fff700 1 -- 172.21.15.31:0/476308806 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cd80ac5b0 con 0x7f8cd8159550 2023-10-06T02:27:38.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.670+0000 7f8cd7fff700 1 --2- 172.21.15.31:0/476308806 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8cd8159550 0x7f8cd80ceac0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8cc8009910 tx=0x7f8cc80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2231281309ec453 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:38.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.670+0000 7f8cd6ffd700 1 -- 172.21.15.31:0/476308806 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc800b680 con 0x7f8cd8159550 2023-10-06T02:27:38.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.670+0000 7f8cd6ffd700 1 -- 172.21.15.31:0/476308806 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cc800b7e0 con 0x7f8cd8159550 2023-10-06T02:27:38.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.670+0000 7f8cd6ffd700 1 -- 172.21.15.31:0/476308806 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc800bb60 con 0x7f8cd8159550 2023-10-06T02:27:38.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.670+0000 7f8cdd79d700 1 -- 172.21.15.31:0/476308806 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8cd8159550 msgr2=0x7f8cd80ceac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:38.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.670+0000 7f8cdd79d700 1 --2- 172.21.15.31:0/476308806 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8cd8159550 0x7f8cd80ceac0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8cc8009910 tx=0x7f8cc80091a0 comp rx=0 tx=0).stop 2023-10-06T02:27:38.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.670+0000 7f8cdd79d700 1 -- 172.21.15.31:0/476308806 shutdown_connections 2023-10-06T02:27:38.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.670+0000 7f8cdd79d700 1 --2- 172.21.15.31:0/476308806 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8cd8159550 0x7f8cd80ceac0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:38.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.670+0000 7f8cdd79d700 1 -- 172.21.15.31:0/476308806 >> 172.21.15.31:0/476308806 conn(0x7f8cd806ce80 msgr2=0x7f8cd80ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:38.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.671+0000 7f8cdd79d700 1 -- 172.21.15.31:0/476308806 shutdown_connections 2023-10-06T02:27:38.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.671+0000 7f8cdd79d700 1 -- 172.21.15.31:0/476308806 wait complete. 2023-10-06T02:27:38.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.672+0000 7f8cdd79d700 1 Processor -- start 2023-10-06T02:27:38.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.672+0000 7f8cdd79d700 1 -- start start 2023-10-06T02:27:38.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.673+0000 7f8cdd79d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8cd8159550 0x7f8cd814d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:38.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.673+0000 7f8cdd79d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8cd8159920 0x7f8cd8067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:38.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.673+0000 7f8cdd79d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8cd815dfd0 0x7f8cd80682b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:38.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.673+0000 7f8cdd79d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd80cd9b0 con 0x7f8cd8159550 2023-10-06T02:27:38.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.673+0000 7f8cdd79d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd816a8c0 con 0x7f8cd8159920 2023-10-06T02:27:38.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.673+0000 7f8cdd79d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd80cd080 con 0x7f8cd815dfd0 2023-10-06T02:27:38.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.673+0000 7f8cd7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8cd8159550 0x7f8cd814d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:38.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.673+0000 7f8cd7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8cd8159550 0x7f8cd814d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50818/0 (socket says 172.21.15.31:50818) 2023-10-06T02:27:38.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.673+0000 7f8cd7fff700 1 -- 172.21.15.31:0/2718517504 learned_addr learned my addr 172.21.15.31:0/2718517504 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:38.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.673+0000 7f8cd77fe700 1 --2- 172.21.15.31:0/2718517504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8cd8159920 0x7f8cd8067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:38.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.673+0000 7f8cdcf9c700 1 --2- 172.21.15.31:0/2718517504 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8cd815dfd0 0x7f8cd80682b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:38.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.674+0000 7f8cd7fff700 1 -- 172.21.15.31:0/2718517504 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8cd815dfd0 msgr2=0x7f8cd80682b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:38.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.674+0000 7f8cd7fff700 1 --2- 172.21.15.31:0/2718517504 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8cd815dfd0 0x7f8cd80682b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:38.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.674+0000 7f8cd7fff700 1 -- 172.21.15.31:0/2718517504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8cd8159920 msgr2=0x7f8cd8067d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:38.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.674+0000 7f8cd7fff700 1 --2- 172.21.15.31:0/2718517504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8cd8159920 0x7f8cd8067d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:38.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.674+0000 7f8cd7fff700 1 -- 172.21.15.31:0/2718517504 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cc8009420 con 0x7f8cd8159550 2023-10-06T02:27:38.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.674+0000 7f8cd7fff700 1 --2- 172.21.15.31:0/2718517504 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8cd8159550 0x7f8cd814d050 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f8cd806ad70 tx=0x7f8cc8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:38.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.674+0000 7f8cbffff700 1 -- 172.21.15.31:0/2718517504 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc801f070 con 0x7f8cd8159550 2023-10-06T02:27:38.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.674+0000 7f8cbffff700 1 -- 172.21.15.31:0/2718517504 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cc8000bc0 con 0x7f8cd8159550 2023-10-06T02:27:38.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.674+0000 7f8cdd79d700 1 -- 172.21.15.31:0/2718517504 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cd80cd360 con 0x7f8cd8159550 2023-10-06T02:27:38.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.674+0000 7f8cbffff700 1 -- 172.21.15.31:0/2718517504 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc8025420 con 0x7f8cd8159550 2023-10-06T02:27:38.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.674+0000 7f8cdd79d700 1 -- 172.21.15.31:0/2718517504 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cd814d590 con 0x7f8cd8159550 2023-10-06T02:27:38.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.675+0000 7f8cdd79d700 1 -- 172.21.15.31:0/2718517504 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cd804c3c0 con 0x7f8cd8159550 2023-10-06T02:27:38.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.675+0000 7f8cbffff700 1 -- 172.21.15.31:0/2718517504 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 42252+0+0 (secure 0 0 0) 0x7f8cc8023070 con 0x7f8cd8159550 2023-10-06T02:27:38.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.676+0000 7f8cbffff700 1 -- 172.21.15.31:0/2718517504 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6617+0+0 (secure 0 0 0) 0x7f8cc8048f20 con 0x7f8cd8159550 2023-10-06T02:27:38.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.681+0000 7f8cbffff700 1 -- 172.21.15.31:0/2718517504 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8cc8023cf0 con 0x7f8cd8159550 2023-10-06T02:27:38.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.929+0000 7f8cdd79d700 1 -- 172.21.15.31:0/2718517504 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8cd806e7c0 con 0x7f8cd8159550 2023-10-06T02:27:38.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.930+0000 7f8cbffff700 1 -- 172.21.15.31:0/2718517504 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f8cc802a020 con 0x7f8cd8159550 2023-10-06T02:27:38.931 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:27:38.931 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5474,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/2085637096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2085637096},{"type":"v1","addr":"172.21.15.62:6834","nonce":2085637096}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":5992,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/573080449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":573080449},{"type":"v1","addr":"172.21.15.31:6836","nonce":573080449}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":6007,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2625778079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2625778079},{"type":"v1","addr":"172.21.15.62:6835","nonce":2625778079}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:27:22.533533+0000","modified":"2023-10-06T02:27:26.062249+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6282":{"gid":6282,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6837/3003674295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":3003674295},{"type":"v1","addr":"172.21.15.31:6837","nonce":3003674295}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T02:27:38.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.933+0000 7f8cdd79d700 1 -- 172.21.15.31:0/2718517504 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8cd8159550 msgr2=0x7f8cd814d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:38.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.933+0000 7f8cdd79d700 1 --2- 172.21.15.31:0/2718517504 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8cd8159550 0x7f8cd814d050 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f8cd806ad70 tx=0x7f8cc8017000 comp rx=0 tx=0).stop 2023-10-06T02:27:38.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.933+0000 7f8cdd79d700 1 -- 172.21.15.31:0/2718517504 shutdown_connections 2023-10-06T02:27:38.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.933+0000 7f8cdd79d700 1 --2- 172.21.15.31:0/2718517504 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8cd815dfd0 0x7f8cd80682b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:38.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.933+0000 7f8cdd79d700 1 --2- 172.21.15.31:0/2718517504 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8cd8159550 0x7f8cd814d050 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:38.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.933+0000 7f8cdd79d700 1 --2- 172.21.15.31:0/2718517504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8cd8159920 0x7f8cd8067d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:38.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.933+0000 7f8cdd79d700 1 -- 172.21.15.31:0/2718517504 >> 172.21.15.31:0/2718517504 conn(0x7f8cd806ce80 msgr2=0x7f8cd8155150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:38.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.933+0000 7f8cdd79d700 1 -- 172.21.15.31:0/2718517504 shutdown_connections 2023-10-06T02:27:38.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:38.933+0000 7f8cdd79d700 1 -- 172.21.15.31:0/2718517504 wait complete. 2023-10-06T02:27:38.934 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 47 2023-10-06T02:27:38.951 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-10-06T02:27:39.166 INFO:teuthology.orchestra.run.smithi031.stdout:[{"id":5639,"entity":{"name":{"type":"client","num":5639},"addr":{"type":"any","addr":"172.21.15.62:0","nonce":1754863197}},"state":"open","num_leases":0,"num_caps":2,"request_load_avg":2,"uptime":0.62499044999999998,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":1.9971347793129233,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":2,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5639 172.21.15.62:0/1754863197","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000017ffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"736ac5ba72cb47af7d6f6666b1e100f7e20a20e2","ceph_version":"ceph version 17.2.6-1296-g736ac5ba (736ac5ba72cb47af7d6f6666b1e100f7e20a20e2) quincy (stable)","entity_id":"x","hostname":"smithi062","pid":"85212","root":"/"}},{"id":6546,"entity":{"name":{"type":"client","num":6546},"addr":{"type":"any","addr":"172.21.15.62:0","nonce":2923576947}},"state":"open","num_leases":0,"num_caps":2,"request_load_avg":2,"uptime":0.62499044999999998,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":1.9971347793129233,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":2,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.6546 172.21.15.62:0/2923576947","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000017ffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"736ac5ba72cb47af7d6f6666b1e100f7e20a20e2","ceph_version":"ceph version 17.2.6-1296-g736ac5ba (736ac5ba72cb47af7d6f6666b1e100f7e20a20e2) quincy (stable)","entity_id":"x","hostname":"smithi062","pid":"85212","root":"/"}}] 2023-10-06T02:27:39.167 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "736ac5ba72cb47af7d6f6666b1e100f7e20a20e2", "ceph_version": "ceph version 17.2.6-1296-g736ac5ba (736ac5ba72cb47af7d6f6666b1e100f7e20a20e2) quincy (stable)", "client_features": { "feature_bits": "0x000000000017ffff" }, "entity_id": "x", "hostname": "smithi062", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "85212", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "172.21.15.62:0", "nonce": 1754863197, "type": "any" }, "name": { "num": 5639, "type": "client" } }, "id": 5639, "inst": "client.5639 172.21.15.62:0/1754863197", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 2, "num_caps": 2, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 2, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 1.9971347793129233 }, "state": "open", "uptime": 0.62499045 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "736ac5ba72cb47af7d6f6666b1e100f7e20a20e2", "ceph_version": "ceph version 17.2.6-1296-g736ac5ba (736ac5ba72cb47af7d6f6666b1e100f7e20a20e2) quincy (stable)", "client_features": { "feature_bits": "0x000000000017ffff" }, "entity_id": "x", "hostname": "smithi062", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "85212", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "172.21.15.62:0", "nonce": 2923576947, "type": "any" }, "name": { "num": 6546, "type": "client" } }, "id": 6546, "inst": "client.6546 172.21.15.62:0/2923576947", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 2, "num_caps": 2, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 2, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 1.9971347793129233 }, "state": "open", "uptime": 0.62499045 } ] 2023-10-06T02:27:39.167 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-06T02:27:39.167 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T02:27:39.168 DEBUG:teuthology.orchestra.run.smithi031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T02:27:39.257 DEBUG:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T02:27:39.338 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T02:27:39.338 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:27:39.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:39.530+0000 7f7a7f19d700 1 Processor -- start 2023-10-06T02:27:39.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:39.530+0000 7f7a7f19d700 1 -- start start 2023-10-06T02:27:39.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:39.531+0000 7f7a7f19d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7a7815de50 0x7f7a780cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:39.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:39.531+0000 7f7a7f19d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a7815afd0 con 0x7f7a780cef40 2023-10-06T02:27:39.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:39.531+0000 7f7a7f19d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a7815b2b0 con 0x7f7a78153c30 2023-10-06T02:27:39.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:39.531+0000 7f7a7f19d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a7815b590 con 0x7f7a7815de50 2023-10-06T02:27:39.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:39.531+0000 7f7a7e99c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f7a780cef40 0x7f7a7815c070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:37414/0 (socket says 172.21.15.31:37414) 2023-10-06T02:27:39.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:39.532+0000 7f7a7e99c700 1 -- 172.21.15.31:0/2738909693 learned_addr learned my addr 172.21.15.31:0/2738909693 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:27:39.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:39.532+0000 7f7a7d99a700 1 --2- 172.21.15.31:0/2738909693 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7a7815de50 0x7f7a780cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:39.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:39.532+0000 7f7a7d199700 1 -- 172.21.15.31:0/2738909693 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 782497799 0 0) 0x7f7a7815afd0 con 0x7f7a780cef40 2023-10-06T02:27:39.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:39.532+0000 7f7a7d199700 1 -- 172.21.15.31:0/2738909693 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a64003570 con 0x7f7a780cef40 2023-10-06T02:27:39.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:39.532+0000 7f7a7d199700 1 -- 172.21.15.31:0/2738909693 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1166584648 0 0) 0x7f7a7815b2b0 con 0x7f7a78153c30 2023-10-06T02:27:39.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:39.532+0000 7f7a7d199700 1 -- 172.21.15.31:0/2738909693 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a7815afd0 con 0x7f7a78153c30 2023-10-06T02:27:39.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:39.532+0000 7f7a7d99a700 1 -- 172.21.15.31:0/2738909693 >> v1:172.21.15.62:6789/0 conn(0x7f7a78153c30 legacy=0x7f7a7815a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:39.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:39.532+0000 7f7a7d99a700 1 -- 172.21.15.31:0/2738909693 >> v1:172.21.15.31:6789/0 conn(0x7f7a780cef40 legacy=0x7f7a7815c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:39.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:39.532+0000 7f7a7d99a700 1 -- 172.21.15.31:0/2738909693 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a7815b870 con 0x7f7a7815de50 2023-10-06T02:27:39.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:39.533+0000 7f7a7d99a700 1 --2- 172.21.15.31:0/2738909693 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7a7815de50 0x7f7a780cea00 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7a68009910 tx=0x7f7a680091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52638a95c0b22c78 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:39.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:39.533+0000 7f7a7d199700 1 -- 172.21.15.31:0/2738909693 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a68011630 con 0x7f7a7815de50 2023-10-06T02:27:39.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:39.533+0000 7f7a7d199700 1 -- 172.21.15.31:0/2738909693 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a68011790 con 0x7f7a7815de50 2023-10-06T02:27:39.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:39.533+0000 7f7a7d199700 1 -- 172.21.15.31:0/2738909693 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a68011b10 con 0x7f7a7815de50 2023-10-06T02:27:39.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:39.533+0000 7f7a7f19d700 1 -- 172.21.15.31:0/2738909693 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7a7815de50 msgr2=0x7f7a780cea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:39.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:39.533+0000 7f7a7f19d700 1 --2- 172.21.15.31:0/2738909693 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7a7815de50 0x7f7a780cea00 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7a68009910 tx=0x7f7a680091a0 comp rx=0 tx=0).stop 2023-10-06T02:27:39.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:39.533+0000 7f7a7f19d700 1 -- 172.21.15.31:0/2738909693 shutdown_connections 2023-10-06T02:27:39.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:39.533+0000 7f7a7f19d700 1 --2- 172.21.15.31:0/2738909693 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7a7815de50 0x7f7a780cea00 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:39.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:39.533+0000 7f7a7f19d700 1 -- 172.21.15.31:0/2738909693 >> 172.21.15.31:0/2738909693 conn(0x7f7a780703a0 msgr2=0x7f7a780707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:39.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:39.534+0000 7f7a7f19d700 1 -- 172.21.15.31:0/2738909693 shutdown_connections 2023-10-06T02:27:39.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:39.534+0000 7f7a7f19d700 1 -- 172.21.15.31:0/2738909693 wait complete. 2023-10-06T02:27:39.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:39.535+0000 7f7a7f19d700 1 Processor -- start 2023-10-06T02:27:39.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:39.535+0000 7f7a7f19d700 1 -- start start 2023-10-06T02:27:39.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:39.536+0000 7f7a7f19d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a780cef40 0x7f7a780c96c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:39.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:39.536+0000 7f7a7f19d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7a78153c30 0x7f7a780c7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:39.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:39.536+0000 7f7a7f19d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a7815de50 0x7f7a780b3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:39.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:39.536+0000 7f7a7f19d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a780c8620 con 0x7f7a780cef40 2023-10-06T02:27:39.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:39.536+0000 7f7a7f19d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a780c8900 con 0x7f7a7815de50 2023-10-06T02:27:39.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:39.536+0000 7f7a7f19d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a780c8be0 con 0x7f7a78153c30 2023-10-06T02:27:39.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:39.536+0000 7f7a7e19b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a780cef40 0x7f7a780c96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:39.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:39.536+0000 7f7a7e99c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a7815de50 0x7f7a780b3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:39.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:39.536+0000 7f7a7d99a700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7a78153c30 0x7f7a780c7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:39.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:39.536+0000 7f7a7e19b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a780cef40 0x7f7a780c96c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50826/0 (socket says 172.21.15.31:50826) 2023-10-06T02:27:39.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:39.536+0000 7f7a7e19b700 1 -- 172.21.15.31:0/2863347628 learned_addr learned my addr 172.21.15.31:0/2863347628 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:39.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:39.537+0000 7f7a7e99c700 1 -- 172.21.15.31:0/2863347628 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7a78153c30 msgr2=0x7f7a780c7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:39.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:39.537+0000 7f7a7e99c700 1 --2- 172.21.15.31:0/2863347628 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7a78153c30 0x7f7a780c7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:39.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:39.537+0000 7f7a7e99c700 1 -- 172.21.15.31:0/2863347628 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a780cef40 msgr2=0x7f7a780c96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:39.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:39.537+0000 7f7a7e99c700 1 --2- 172.21.15.31:0/2863347628 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a780cef40 0x7f7a780c96c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:39.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:39.537+0000 7f7a7e99c700 1 -- 172.21.15.31:0/2863347628 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a68009420 con 0x7f7a7815de50 2023-10-06T02:27:39.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:39.537+0000 7f7a7e99c700 1 --2- 172.21.15.31:0/2863347628 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a7815de50 0x7f7a780b3ce0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f7a700011e0 tx=0x7f7a70007d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:39.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:39.538+0000 7f7a6e7fc700 1 -- 172.21.15.31:0/2863347628 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a7000ce50 con 0x7f7a7815de50 2023-10-06T02:27:39.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:39.538+0000 7f7a6e7fc700 1 -- 172.21.15.31:0/2863347628 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a70005b30 con 0x7f7a7815de50 2023-10-06T02:27:39.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:39.538+0000 7f7a6e7fc700 1 -- 172.21.15.31:0/2863347628 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a700070d0 con 0x7f7a7815de50 2023-10-06T02:27:39.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:39.538+0000 7f7a7f19d700 1 -- 172.21.15.31:0/2863347628 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a780c8ec0 con 0x7f7a7815de50 2023-10-06T02:27:39.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:39.538+0000 7f7a7f19d700 1 -- 172.21.15.31:0/2863347628 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a78153080 con 0x7f7a7815de50 2023-10-06T02:27:39.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:39.539+0000 7f7a7f19d700 1 -- 172.21.15.31:0/2863347628 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a780c93e0 con 0x7f7a7815de50 2023-10-06T02:27:39.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:39.539+0000 7f7a6e7fc700 1 -- 172.21.15.31:0/2863347628 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 15) v1 ==== 42379+0+0 (secure 0 0 0) 0x7f7a70007480 con 0x7f7a7815de50 2023-10-06T02:27:39.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:39.539+0000 7f7a6e7fc700 1 --2- 172.21.15.31:0/2863347628 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7a5c035630 0x7f7a5c037ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:39.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:39.539+0000 7f7a6e7fc700 1 -- 172.21.15.31:0/2863347628 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6617+0+0 (secure 0 0 0) 0x7f7a70047240 con 0x7f7a7815de50 2023-10-06T02:27:39.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:39.540+0000 7f7a7e19b700 1 --2- 172.21.15.31:0/2863347628 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7a5c035630 0x7f7a5c037ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:39.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:39.541+0000 7f7a7e19b700 1 --2- 172.21.15.31:0/2863347628 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7a5c035630 0x7f7a5c037ae0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7a740041b0 tx=0x7f7a740027a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:39.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:39.545+0000 7f7a6e7fc700 1 -- 172.21.15.31:0/2863347628 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7a780c93e0 con 0x7f7a7815de50 2023-10-06T02:27:39.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:39.794+0000 7f7a7f19d700 1 -- 172.21.15.31:0/2863347628 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7a780c93e0 con 0x7f7a7815de50 2023-10-06T02:27:39.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:39.794+0000 7f7a6e7fc700 1 -- 172.21.15.31:0/2863347628 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f7a780c93e0 con 0x7f7a7815de50 2023-10-06T02:27:39.795 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:27:39.795 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5474,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/2085637096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2085637096},{"type":"v1","addr":"172.21.15.62:6834","nonce":2085637096}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":5992,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/573080449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":573080449},{"type":"v1","addr":"172.21.15.31:6836","nonce":573080449}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":6007,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2625778079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2625778079},{"type":"v1","addr":"172.21.15.62:6835","nonce":2625778079}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:27:22.533533+0000","modified":"2023-10-06T02:27:26.062249+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6282":{"gid":6282,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6837/3003674295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":3003674295},{"type":"v1","addr":"172.21.15.31:6837","nonce":3003674295}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T02:27:39.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:39.797+0000 7f7a7f19d700 1 -- 172.21.15.31:0/2863347628 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7a5c035630 msgr2=0x7f7a5c037ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:39.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:39.797+0000 7f7a7f19d700 1 --2- 172.21.15.31:0/2863347628 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7a5c035630 0x7f7a5c037ae0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7a740041b0 tx=0x7f7a740027a0 comp rx=0 tx=0).stop 2023-10-06T02:27:39.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:39.797+0000 7f7a7f19d700 1 -- 172.21.15.31:0/2863347628 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a7815de50 msgr2=0x7f7a780b3ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:39.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:39.797+0000 7f7a7f19d700 1 --2- 172.21.15.31:0/2863347628 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a7815de50 0x7f7a780b3ce0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f7a700011e0 tx=0x7f7a70007d30 comp rx=0 tx=0).stop 2023-10-06T02:27:39.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:39.797+0000 7f7a7f19d700 1 -- 172.21.15.31:0/2863347628 shutdown_connections 2023-10-06T02:27:39.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:39.797+0000 7f7a7f19d700 1 --2- 172.21.15.31:0/2863347628 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a7815de50 0x7f7a780b3ce0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:39.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:39.797+0000 7f7a7f19d700 1 --2- 172.21.15.31:0/2863347628 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a780cef40 0x7f7a780c96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:39.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:39.797+0000 7f7a7f19d700 1 --2- 172.21.15.31:0/2863347628 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7a5c035630 0x7f7a5c037ae0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:39.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:39.797+0000 7f7a7f19d700 1 --2- 172.21.15.31:0/2863347628 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7a78153c30 0x7f7a780c7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:39.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:39.797+0000 7f7a7f19d700 1 -- 172.21.15.31:0/2863347628 >> 172.21.15.31:0/2863347628 conn(0x7f7a780703a0 msgr2=0x7f7a780d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:39.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:39.797+0000 7f7a7f19d700 1 -- 172.21.15.31:0/2863347628 shutdown_connections 2023-10-06T02:27:39.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:39.797+0000 7f7a7f19d700 1 -- 172.21.15.31:0/2863347628 wait complete. 2023-10-06T02:27:39.799 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 47 2023-10-06T02:27:39.815 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:27:39.825 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:27:39.825+0000 7f54bc28a180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T02:27:39.999 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:27:39.999+0000 7f54bc28a180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T02:27:40.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.003+0000 7fe9d4f9d700 1 Processor -- start 2023-10-06T02:27:40.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.003+0000 7fe9d4f9d700 1 -- start start 2023-10-06T02:27:40.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.005+0000 7fe9d4f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe9d0153c30 0x7fe9d015a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:40.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.005+0000 7fe9d4f9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe9d00cf110 0x7fe9d015c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:40.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.005+0000 7fe9d4f9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9d015afd0 con 0x7fe9d015de50 2023-10-06T02:27:40.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.005+0000 7fe9d4f9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9d015b2b0 con 0x7fe9d00cf110 2023-10-06T02:27:40.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.005+0000 7fe9d4f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d015b590 con 0x7fe9d0153c30 2023-10-06T02:27:40.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.005+0000 7fe9cffff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe9d00cf110 0x7fe9d015c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:40.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.005+0000 7fe9cffff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe9d00cf110 0x7fe9d015c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50836/0 (socket says 172.21.15.31:50836) 2023-10-06T02:27:40.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.005+0000 7fe9cffff700 1 -- 172.21.15.31:0/60482042 learned_addr learned my addr 172.21.15.31:0/60482042 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:40.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.005+0000 7fe9cf7fe700 1 --2- 172.21.15.31:0/60482042 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe9d0153c30 0x7fe9d015a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:40.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.006+0000 7fe9cffff700 1 -- 172.21.15.31:0/60482042 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe9d0153c30 msgr2=0x7fe9d015a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:40.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.006+0000 7fe9cffff700 1 --2- 172.21.15.31:0/60482042 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe9d0153c30 0x7fe9d015a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:40.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.006+0000 7fe9cffff700 1 -- 172.21.15.31:0/60482042 >> v1:172.21.15.62:6789/0 conn(0x7fe9d015de50 legacy=0x7fe9d00cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:40.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.006+0000 7fe9cffff700 1 -- 172.21.15.31:0/60482042 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9d015b870 con 0x7fe9d00cf110 2023-10-06T02:27:40.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.006+0000 7fe9ce7fc700 1 -- 172.21.15.31:0/60482042 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3430106888 0 0) 0x7fe9d015afd0 con 0x7fe9d015de50 2023-10-06T02:27:40.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.006+0000 7fe9cffff700 1 --2- 172.21.15.31:0/60482042 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9d00cf110 0x7fe9d015c070 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8009f20 tx=0x7fe9c8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff723437c7bf9859 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:40.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.006+0000 7fe9ce7fc700 1 -- 172.21.15.31:0/60482042 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c800e070 con 0x7fe9d00cf110 2023-10-06T02:27:40.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.006+0000 7fe9ce7fc700 1 -- 172.21.15.31:0/60482042 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9c800b9d0 con 0x7fe9d00cf110 2023-10-06T02:27:40.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.006+0000 7fe9ce7fc700 1 -- 172.21.15.31:0/60482042 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c80092e0 con 0x7fe9d00cf110 2023-10-06T02:27:40.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.007+0000 7fe9d4f9d700 1 -- 172.21.15.31:0/60482042 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9d00cf110 msgr2=0x7fe9d015c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:40.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.007+0000 7fe9d4f9d700 1 --2- 172.21.15.31:0/60482042 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9d00cf110 0x7fe9d015c070 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8009f20 tx=0x7fe9c8009840 comp rx=0 tx=0).stop 2023-10-06T02:27:40.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.007+0000 7fe9d4f9d700 1 -- 172.21.15.31:0/60482042 shutdown_connections 2023-10-06T02:27:40.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.007+0000 7fe9d4f9d700 1 --2- 172.21.15.31:0/60482042 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9d00cf110 0x7fe9d015c070 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:40.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.007+0000 7fe9d4f9d700 1 --2- 172.21.15.31:0/60482042 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe9d0153c30 0x7fe9d015a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:40.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.007+0000 7fe9d4f9d700 1 -- 172.21.15.31:0/60482042 >> 172.21.15.31:0/60482042 conn(0x7fe9d00703a0 msgr2=0x7fe9d00707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:40.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.007+0000 7fe9d4f9d700 1 -- 172.21.15.31:0/60482042 shutdown_connections 2023-10-06T02:27:40.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.007+0000 7fe9d4f9d700 1 -- 172.21.15.31:0/60482042 wait complete. 2023-10-06T02:27:40.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.008+0000 7fe9d4f9d700 1 Processor -- start 2023-10-06T02:27:40.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.009+0000 7fe9d4f9d700 1 -- start start 2023-10-06T02:27:40.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.009+0000 7fe9d4f9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9d00cf110 0x7fe9d0150230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:40.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.009+0000 7fe9d4f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe9d0153c30 0x7fe9d014c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:40.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.009+0000 7fe9d4f9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9d015de50 0x7fe9d00cd340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:40.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.009+0000 7fe9d4f9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d00c8e90 con 0x7fe9d015de50 2023-10-06T02:27:40.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.009+0000 7fe9d4f9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d014cab0 con 0x7fe9d00cf110 2023-10-06T02:27:40.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.009+0000 7fe9d4f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d00b32d0 con 0x7fe9d0153c30 2023-10-06T02:27:40.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.010+0000 7fe9cffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9d015de50 0x7fe9d00cd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:40.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.010+0000 7fe9cffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9d015de50 0x7fe9d00cd340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50842/0 (socket says 172.21.15.31:50842) 2023-10-06T02:27:40.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.010+0000 7fe9cffff700 1 -- 172.21.15.31:0/2740984379 learned_addr learned my addr 172.21.15.31:0/2740984379 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:40.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.010+0000 7fe9cf7fe700 1 --2- 172.21.15.31:0/2740984379 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9d00cf110 0x7fe9d0150230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:40.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.010+0000 7fe9ceffd700 1 --2- 172.21.15.31:0/2740984379 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe9d0153c30 0x7fe9d014c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:40.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.010+0000 7fe9cffff700 1 -- 172.21.15.31:0/2740984379 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe9d0153c30 msgr2=0x7fe9d014c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:40.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.010+0000 7fe9cffff700 1 --2- 172.21.15.31:0/2740984379 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe9d0153c30 0x7fe9d014c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:40.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.010+0000 7fe9cffff700 1 -- 172.21.15.31:0/2740984379 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9d00cf110 msgr2=0x7fe9d0150230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:40.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.010+0000 7fe9cffff700 1 --2- 172.21.15.31:0/2740984379 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9d00cf110 0x7fe9d0150230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:40.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.010+0000 7fe9cffff700 1 -- 172.21.15.31:0/2740984379 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9c8009ae0 con 0x7fe9d015de50 2023-10-06T02:27:40.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.011+0000 7fe9cffff700 1 --2- 172.21.15.31:0/2740984379 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9d015de50 0x7fe9d00cd340 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fe9c800b610 tx=0x7fe9c8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:40.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.011+0000 7fe9b77fe700 1 -- 172.21.15.31:0/2740984379 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c8008b80 con 0x7fe9d015de50 2023-10-06T02:27:40.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.011+0000 7fe9b77fe700 1 -- 172.21.15.31:0/2740984379 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9c8008d00 con 0x7fe9d015de50 2023-10-06T02:27:40.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.011+0000 7fe9b77fe700 1 -- 172.21.15.31:0/2740984379 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c80266b0 con 0x7fe9d015de50 2023-10-06T02:27:40.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.011+0000 7fe9d4f9d700 1 -- 172.21.15.31:0/2740984379 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9d00cd880 con 0x7fe9d015de50 2023-10-06T02:27:40.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.011+0000 7fe9d4f9d700 1 -- 172.21.15.31:0/2740984379 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9d0150b80 con 0x7fe9d015de50 2023-10-06T02:27:40.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.012+0000 7fe9b77fe700 1 -- 172.21.15.31:0/2740984379 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 42379+0+0 (secure 0 0 0) 0x7fe9c8003040 con 0x7fe9d015de50 2023-10-06T02:27:40.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.012+0000 7fe9d4f9d700 1 -- 172.21.15.31:0/2740984379 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9d00c9040 con 0x7fe9d015de50 2023-10-06T02:27:40.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.012+0000 7fe9b77fe700 1 --2- 172.21.15.31:0/2740984379 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe9bc035630 0x7fe9bc037ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:40.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.013+0000 7fe9b77fe700 1 -- 172.21.15.31:0/2740984379 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6617+0+0 (secure 0 0 0) 0x7fe9c8049fb0 con 0x7fe9d015de50 2023-10-06T02:27:40.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.013+0000 7fe9cf7fe700 1 --2- 172.21.15.31:0/2740984379 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe9bc035630 0x7fe9bc037ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:40.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.014+0000 7fe9cf7fe700 1 --2- 172.21.15.31:0/2740984379 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe9bc035630 0x7fe9bc037ae0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe9c0009db0 tx=0x7fe9c00091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:40.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.018+0000 7fe9b77fe700 1 -- 172.21.15.31:0/2740984379 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe9d00c9040 con 0x7fe9d015de50 2023-10-06T02:27:40.091 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:27:40.091+0000 7f54bc28a180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T02:27:40.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.224+0000 7fe9d4f9d700 1 -- 172.21.15.31:0/2740984379 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe9d00c9040 con 0x7fe9d015de50 2023-10-06T02:27:40.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.225+0000 7fe9b77fe700 1 -- 172.21.15.31:0/2740984379 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) v1 ==== 74+0+17622 (secure 0 0 0) 0x7fe9d00c9040 con 0x7fe9d015de50 2023-10-06T02:27:40.225 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:27:40.226 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":76,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:27:38.406866+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:27:20.257986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-10-06T02:27:21.518685+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/2878303473":"2023-10-07T02:27:38.406819+0000","172.21.15.31:0/2186321038":"2023-10-07T02:27:38.406819+0000","172.21.15.31:6832/85292":"2023-10-07T02:27:38.406819+0000","172.21.15.31:0/3308984790":"2023-10-07T02:27:38.406819+0000","172.21.15.31:0/3769130347":"2023-10-07T02:27:38.406819+0000","172.21.15.31:0/2627562379":"2023-10-07T02:27:38.406819+0000","172.21.15.31:0/2430615067":"2023-10-07T02:27:38.406819+0000","172.21.15.31:0/1519622771":"2023-10-07T02:27:38.406819+0000","172.21.15.31:0/3343749855":"2023-10-07T02:27:38.406819+0000","172.21.15.31:0/1049264351":"2023-10-07T02:27:38.406819+0000","172.21.15.31:0/2607073168":"2023-10-07T02:27:38.406819+0000","172.21.15.31:6833/85292":"2023-10-07T02:27:38.406819+0000","172.21.15.31:0/3265960029":"2023-10-07T02:27:38.406819+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:27:40.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.227+0000 7fe9d4f9d700 1 -- 172.21.15.31:0/2740984379 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe9bc035630 msgr2=0x7fe9bc037ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:40.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.227+0000 7fe9d4f9d700 1 --2- 172.21.15.31:0/2740984379 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe9bc035630 0x7fe9bc037ae0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe9c0009db0 tx=0x7fe9c00091a0 comp rx=0 tx=0).stop 2023-10-06T02:27:40.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.227+0000 7fe9d4f9d700 1 -- 172.21.15.31:0/2740984379 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9d015de50 msgr2=0x7fe9d00cd340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:40.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.227+0000 7fe9d4f9d700 1 --2- 172.21.15.31:0/2740984379 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9d015de50 0x7fe9d00cd340 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fe9c800b610 tx=0x7fe9c8018000 comp rx=0 tx=0).stop 2023-10-06T02:27:40.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.227+0000 7fe9d4f9d700 1 -- 172.21.15.31:0/2740984379 shutdown_connections 2023-10-06T02:27:40.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.227+0000 7fe9d4f9d700 1 --2- 172.21.15.31:0/2740984379 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9d015de50 0x7fe9d00cd340 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:40.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.227+0000 7fe9d4f9d700 1 --2- 172.21.15.31:0/2740984379 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9d00cf110 0x7fe9d0150230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:40.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.227+0000 7fe9d4f9d700 1 --2- 172.21.15.31:0/2740984379 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe9bc035630 0x7fe9bc037ae0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:40.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.227+0000 7fe9d4f9d700 1 --2- 172.21.15.31:0/2740984379 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe9d0153c30 0x7fe9d014c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:40.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.227+0000 7fe9d4f9d700 1 -- 172.21.15.31:0/2740984379 >> 172.21.15.31:0/2740984379 conn(0x7fe9d00703a0 msgr2=0x7fe9d00c6b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:40.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.228+0000 7fe9d4f9d700 1 -- 172.21.15.31:0/2740984379 shutdown_connections 2023-10-06T02:27:40.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.228+0000 7fe9d4f9d700 1 -- 172.21.15.31:0/2740984379 wait complete. 2023-10-06T02:27:40.254 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T02:22:53.183425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T02:23:00.046967+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T02:27:20.257986+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T02:27:21.518685+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T02:27:40.255 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T02:27:40.333 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:27:40.333+0000 7f54bc28a180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T02:27:40.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.489+0000 7f7a7579d700 1 Processor -- start 2023-10-06T02:27:40.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.489+0000 7f7a7579d700 1 -- start start 2023-10-06T02:27:40.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.490+0000 7f7a7579d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7a70153dd0 0x7f7a7015de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:40.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.490+0000 7f7a7579d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7a700cea00 0x7f7a700ceea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:40.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.490+0000 7f7a7579d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a7015ac50 con 0x7f7a70153a00 2023-10-06T02:27:40.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.490+0000 7f7a7579d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a7015d010 con 0x7f7a70153dd0 2023-10-06T02:27:40.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.490+0000 7f7a7579d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a700cf3e0 con 0x7f7a700cea00 2023-10-06T02:27:40.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.491+0000 7f7a6f7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7a70153dd0 0x7f7a7015de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:40.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.491+0000 7f7a6ffff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f7a70153a00 0x7f7a7015a540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:37430/0 (socket says 172.21.15.31:37430) 2023-10-06T02:27:40.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.491+0000 7f7a6ffff700 1 -- 172.21.15.31:0/326150192 learned_addr learned my addr 172.21.15.31:0/326150192 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:27:40.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.491+0000 7f7a74f9c700 1 --2- 172.21.15.31:0/326150192 >> v2:172.21.15.62:3300/0 conn(0x7f7a700cea00 0x7f7a700ceea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:40.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.491+0000 7f7a6f7fe700 1 -- 172.21.15.31:0/326150192 >> v2:172.21.15.62:3300/0 conn(0x7f7a700cea00 msgr2=0x7f7a700ceea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:40.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.491+0000 7f7a6f7fe700 1 --2- 172.21.15.31:0/326150192 >> v2:172.21.15.62:3300/0 conn(0x7f7a700cea00 0x7f7a700ceea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:40.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.491+0000 7f7a6f7fe700 1 -- 172.21.15.31:0/326150192 >> v1:172.21.15.31:6789/0 conn(0x7f7a70153a00 legacy=0x7f7a7015a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:40.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.491+0000 7f7a6f7fe700 1 -- 172.21.15.31:0/326150192 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a7015af60 con 0x7f7a70153dd0 2023-10-06T02:27:40.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.491+0000 7f7a6f7fe700 1 --2- 172.21.15.31:0/326150192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a70153dd0 0x7f7a7015de50 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f7a700d03b0 tx=0x7f7a640091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e554223057fc73 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:40.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.491+0000 7f7a6effd700 1 -- 172.21.15.31:0/326150192 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a6400b6b0 con 0x7f7a70153dd0 2023-10-06T02:27:40.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.492+0000 7f7a6effd700 1 -- 172.21.15.31:0/326150192 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a6400b810 con 0x7f7a70153dd0 2023-10-06T02:27:40.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.492+0000 7f7a6effd700 1 -- 172.21.15.31:0/326150192 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a6400bbb0 con 0x7f7a70153dd0 2023-10-06T02:27:40.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.492+0000 7f7a7579d700 1 -- 172.21.15.31:0/326150192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a70153dd0 msgr2=0x7f7a7015de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:40.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.492+0000 7f7a7579d700 1 --2- 172.21.15.31:0/326150192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a70153dd0 0x7f7a7015de50 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f7a700d03b0 tx=0x7f7a640091a0 comp rx=0 tx=0).stop 2023-10-06T02:27:40.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.492+0000 7f7a7579d700 1 -- 172.21.15.31:0/326150192 shutdown_connections 2023-10-06T02:27:40.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.492+0000 7f7a7579d700 1 --2- 172.21.15.31:0/326150192 >> v2:172.21.15.62:3300/0 conn(0x7f7a700cea00 0x7f7a700ceea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:40.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.492+0000 7f7a7579d700 1 --2- 172.21.15.31:0/326150192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a70153dd0 0x7f7a7015de50 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:40.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.492+0000 7f7a7579d700 1 -- 172.21.15.31:0/326150192 >> 172.21.15.31:0/326150192 conn(0x7f7a700703a0 msgr2=0x7f7a700707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:40.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.492+0000 7f7a7579d700 1 -- 172.21.15.31:0/326150192 shutdown_connections 2023-10-06T02:27:40.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.492+0000 7f7a7579d700 1 -- 172.21.15.31:0/326150192 wait complete. 2023-10-06T02:27:40.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.494+0000 7f7a7579d700 1 Processor -- start 2023-10-06T02:27:40.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.494+0000 7f7a7579d700 1 -- start start 2023-10-06T02:27:40.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.495+0000 7f7a7579d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7a700cea00 0x7f7a701525d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:40.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.495+0000 7f7a7579d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a70153a00 0x7f7a7006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:40.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.495+0000 7f7a7579d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a70153dd0 0x7f7a7006dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:40.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.495+0000 7f7a7579d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a700c9d30 con 0x7f7a70153a00 2023-10-06T02:27:40.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.495+0000 7f7a7579d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a701576f0 con 0x7f7a70153dd0 2023-10-06T02:27:40.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.495+0000 7f7a7579d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a701579d0 con 0x7f7a700cea00 2023-10-06T02:27:40.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.495+0000 7f7a6f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a70153a00 0x7f7a7006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:40.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.495+0000 7f7a6f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a70153a00 0x7f7a7006d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50856/0 (socket says 172.21.15.31:50856) 2023-10-06T02:27:40.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.495+0000 7f7a6f7fe700 1 -- 172.21.15.31:0/2752233620 learned_addr learned my addr 172.21.15.31:0/2752233620 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:40.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.495+0000 7f7a6f7fe700 1 -- 172.21.15.31:0/2752233620 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7a700cea00 msgr2=0x7f7a701525d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:40.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.495+0000 7f7a6f7fe700 1 --2- 172.21.15.31:0/2752233620 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7a700cea00 0x7f7a701525d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:40.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.495+0000 7f7a74f9c700 1 --2- 172.21.15.31:0/2752233620 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a70153dd0 0x7f7a7006dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:40.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.495+0000 7f7a6f7fe700 1 -- 172.21.15.31:0/2752233620 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a70153dd0 msgr2=0x7f7a7006dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:40.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.495+0000 7f7a6f7fe700 1 --2- 172.21.15.31:0/2752233620 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a70153dd0 0x7f7a7006dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:40.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.495+0000 7f7a6f7fe700 1 -- 172.21.15.31:0/2752233620 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a64009420 con 0x7f7a70153a00 2023-10-06T02:27:40.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.495+0000 7f7a74f9c700 1 --2- 172.21.15.31:0/2752233620 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a70153dd0 0x7f7a7006dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:27:40.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.496+0000 7f7a6f7fe700 1 --2- 172.21.15.31:0/2752233620 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a70153a00 0x7f7a7006d790 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f7a64013000 tx=0x7f7a64017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:40.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.496+0000 7f7a57fff700 1 -- 172.21.15.31:0/2752233620 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a64011420 con 0x7f7a70153a00 2023-10-06T02:27:40.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.496+0000 7f7a57fff700 1 -- 172.21.15.31:0/2752233620 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a64011580 con 0x7f7a70153a00 2023-10-06T02:27:40.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.496+0000 7f7a7579d700 1 -- 172.21.15.31:0/2752233620 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a70157cb0 con 0x7f7a70153a00 2023-10-06T02:27:40.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.496+0000 7f7a57fff700 1 -- 172.21.15.31:0/2752233620 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a64011940 con 0x7f7a70153a00 2023-10-06T02:27:40.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.496+0000 7f7a7579d700 1 -- 172.21.15.31:0/2752233620 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a700c90b0 con 0x7f7a70153a00 2023-10-06T02:27:40.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.497+0000 7f7a7579d700 1 -- 172.21.15.31:0/2752233620 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a7004c3c0 con 0x7f7a70153a00 2023-10-06T02:27:40.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.497+0000 7f7a57fff700 1 -- 172.21.15.31:0/2752233620 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 42643+0+0 (secure 0 0 0) 0x7f7a64011d30 con 0x7f7a70153a00 2023-10-06T02:27:40.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.498+0000 7f7a57fff700 1 --2- 172.21.15.31:0/2752233620 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7a5c035900 0x7f7a5c037db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:40.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.498+0000 7f7a57fff700 1 -- 172.21.15.31:0/2752233620 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6617+0+0 (secure 0 0 0) 0x7f7a64049960 con 0x7f7a70153a00 2023-10-06T02:27:40.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.498+0000 7f7a6ffff700 1 --2- 172.21.15.31:0/2752233620 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7a5c035900 0x7f7a5c037db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:40.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.499+0000 7f7a6ffff700 1 --2- 172.21.15.31:0/2752233620 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7a5c035900 0x7f7a5c037db0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7a7016d120 tx=0x7f7a60007b30 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:40.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.503+0000 7f7a57fff700 1 -- 172.21.15.31:0/2752233620 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7a6402ab90 con 0x7f7a70153a00 2023-10-06T02:27:40.524 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:27:40.524+0000 7f54bc28a180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T02:27:40.621 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:27:40.620+0000 7f54bc28a180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T02:27:40.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.706+0000 7f7a7579d700 1 -- 172.21.15.31:0/2752233620 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7a7015b240 con 0x7f7a70153a00 2023-10-06T02:27:40.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.707+0000 7f7a57fff700 1 -- 172.21.15.31:0/2752233620 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v76) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7a64015070 con 0x7f7a70153a00 2023-10-06T02:27:40.707 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-10-06T02:27:40.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.709+0000 7f7a7579d700 1 -- 172.21.15.31:0/2752233620 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7a5c035900 msgr2=0x7f7a5c037db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:40.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.709+0000 7f7a7579d700 1 --2- 172.21.15.31:0/2752233620 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7a5c035900 0x7f7a5c037db0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7a7016d120 tx=0x7f7a60007b30 comp rx=0 tx=0).stop 2023-10-06T02:27:40.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.709+0000 7f7a7579d700 1 -- 172.21.15.31:0/2752233620 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a70153a00 msgr2=0x7f7a7006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:40.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.709+0000 7f7a7579d700 1 --2- 172.21.15.31:0/2752233620 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a70153a00 0x7f7a7006d790 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f7a64013000 tx=0x7f7a64017000 comp rx=0 tx=0).stop 2023-10-06T02:27:40.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.709+0000 7f7a7579d700 1 -- 172.21.15.31:0/2752233620 shutdown_connections 2023-10-06T02:27:40.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.709+0000 7f7a7579d700 1 --2- 172.21.15.31:0/2752233620 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a70153dd0 0x7f7a7006dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:40.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.709+0000 7f7a7579d700 1 --2- 172.21.15.31:0/2752233620 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7a5c035900 0x7f7a5c037db0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:40.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.709+0000 7f7a7579d700 1 --2- 172.21.15.31:0/2752233620 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7a700cea00 0x7f7a701525d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:40.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.709+0000 7f7a7579d700 1 --2- 172.21.15.31:0/2752233620 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a70153a00 0x7f7a7006d790 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:40.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.709+0000 7f7a7579d700 1 -- 172.21.15.31:0/2752233620 >> 172.21.15.31:0/2752233620 conn(0x7f7a700703a0 msgr2=0x7f7a700b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:40.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.709+0000 7f7a7579d700 1 -- 172.21.15.31:0/2752233620 shutdown_connections 2023-10-06T02:27:40.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.709+0000 7f7a7579d700 1 -- 172.21.15.31:0/2752233620 wait complete. 2023-10-06T02:27:40.728 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T02:27:40.789 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:27:40.789+0000 7f54bc28a180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T02:27:40.912 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:27:40.911+0000 7f54bc28a180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T02:27:40.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.961+0000 7fb1b4b9d700 1 Processor -- start 2023-10-06T02:27:40.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.961+0000 7fb1b4b9d700 1 -- start start 2023-10-06T02:27:40.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.963+0000 7fb1b4b9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb1b015dfd0 0x7fb1b00aedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:40.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.963+0000 7fb1b4b9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb1b0159920 0x7fb1b00ad090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:40.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.963+0000 7fb1b4b9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1b00acc00 con 0x7fb1b0159550 2023-10-06T02:27:40.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.963+0000 7fb1b4b9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1b00abff0 con 0x7fb1b0159920 2023-10-06T02:27:40.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.963+0000 7fb1b4b9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1b00ac2d0 con 0x7fb1b015dfd0 2023-10-06T02:27:40.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.963+0000 7fb1affff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb1b0159920 0x7fb1b00ad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:40.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.963+0000 7fb1affff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb1b0159920 0x7fb1b00ad090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50870/0 (socket says 172.21.15.31:50870) 2023-10-06T02:27:40.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.963+0000 7fb1affff700 1 -- 172.21.15.31:0/3152968553 learned_addr learned my addr 172.21.15.31:0/3152968553 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:40.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.963+0000 7fb1aeffd700 1 --2- 172.21.15.31:0/3152968553 >> v2:172.21.15.62:3300/0 conn(0x7fb1b015dfd0 0x7fb1b00aedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:40.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.963+0000 7fb1affff700 1 -- 172.21.15.31:0/3152968553 >> v2:172.21.15.62:3300/0 conn(0x7fb1b015dfd0 msgr2=0x7fb1b00aedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:40.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.964+0000 7fb1affff700 1 --2- 172.21.15.31:0/3152968553 >> v2:172.21.15.62:3300/0 conn(0x7fb1b015dfd0 0x7fb1b00aedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:40.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.964+0000 7fb1affff700 1 -- 172.21.15.31:0/3152968553 >> v1:172.21.15.62:6789/0 conn(0x7fb1b0159550 legacy=0x7fb1b00ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:40.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.964+0000 7fb1affff700 1 -- 172.21.15.31:0/3152968553 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1b00ac5b0 con 0x7fb1b0159920 2023-10-06T02:27:40.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.964+0000 7fb1affff700 1 --2- 172.21.15.31:0/3152968553 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1b0159920 0x7fb1b00ad090 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fb1b006c180 tx=0x7fb1a8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f466566303e3bde server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:40.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.964+0000 7fb1ae7fc700 1 -- 172.21.15.31:0/3152968553 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a800e070 con 0x7fb1b0159920 2023-10-06T02:27:40.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.964+0000 7fb1ae7fc700 1 -- 172.21.15.31:0/3152968553 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1a800ba10 con 0x7fb1b0159920 2023-10-06T02:27:40.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.964+0000 7fb1ae7fc700 1 -- 172.21.15.31:0/3152968553 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a80092e0 con 0x7fb1b0159920 2023-10-06T02:27:40.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.965+0000 7fb1b4b9d700 1 -- 172.21.15.31:0/3152968553 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1b0159920 msgr2=0x7fb1b00ad090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:40.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.965+0000 7fb1b4b9d700 1 --2- 172.21.15.31:0/3152968553 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1b0159920 0x7fb1b00ad090 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fb1b006c180 tx=0x7fb1a8009840 comp rx=0 tx=0).stop 2023-10-06T02:27:40.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.965+0000 7fb1b4b9d700 1 -- 172.21.15.31:0/3152968553 shutdown_connections 2023-10-06T02:27:40.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.965+0000 7fb1b4b9d700 1 --2- 172.21.15.31:0/3152968553 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1b0159920 0x7fb1b00ad090 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:40.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.965+0000 7fb1b4b9d700 1 --2- 172.21.15.31:0/3152968553 >> v2:172.21.15.62:3300/0 conn(0x7fb1b015dfd0 0x7fb1b00aedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:40.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.965+0000 7fb1b4b9d700 1 -- 172.21.15.31:0/3152968553 >> 172.21.15.31:0/3152968553 conn(0x7fb1b006ce80 msgr2=0x7fb1b00ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:40.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.965+0000 7fb1b4b9d700 1 -- 172.21.15.31:0/3152968553 shutdown_connections 2023-10-06T02:27:40.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.965+0000 7fb1b4b9d700 1 -- 172.21.15.31:0/3152968553 wait complete. 2023-10-06T02:27:40.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.966+0000 7fb1b4b9d700 1 Processor -- start 2023-10-06T02:27:40.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.967+0000 7fb1b4b9d700 1 -- start start 2023-10-06T02:27:40.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.967+0000 7fb1b4b9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1b0159550 0x7fb1b00ca0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:40.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.967+0000 7fb1b4b9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb1b0159920 0x7fb1b0142e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:40.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.967+0000 7fb1b4b9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1b015dfd0 0x7fb1b0156a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:40.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.967+0000 7fb1b4b9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b0143350 con 0x7fb1b015dfd0 2023-10-06T02:27:40.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.967+0000 7fb1b4b9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b0157430 con 0x7fb1b0159550 2023-10-06T02:27:40.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.967+0000 7fb1b4b9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b00c92e0 con 0x7fb1b0159920 2023-10-06T02:27:40.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.968+0000 7fb1affff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1b015dfd0 0x7fb1b0156a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:40.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.968+0000 7fb1affff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1b015dfd0 0x7fb1b0156a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50878/0 (socket says 172.21.15.31:50878) 2023-10-06T02:27:40.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.968+0000 7fb1affff700 1 -- 172.21.15.31:0/629446437 learned_addr learned my addr 172.21.15.31:0/629446437 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:40.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.968+0000 7fb1af7fe700 1 --2- 172.21.15.31:0/629446437 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1b0159550 0x7fb1b00ca0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:40.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.968+0000 7fb1aeffd700 1 --2- 172.21.15.31:0/629446437 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb1b0159920 0x7fb1b0142e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:40.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.968+0000 7fb1affff700 1 -- 172.21.15.31:0/629446437 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb1b0159920 msgr2=0x7fb1b0142e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:40.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.968+0000 7fb1affff700 1 --2- 172.21.15.31:0/629446437 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb1b0159920 0x7fb1b0142e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:40.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.968+0000 7fb1affff700 1 -- 172.21.15.31:0/629446437 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1b0159550 msgr2=0x7fb1b00ca0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:40.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.968+0000 7fb1affff700 1 --2- 172.21.15.31:0/629446437 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1b0159550 0x7fb1b00ca0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:40.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.968+0000 7fb1affff700 1 -- 172.21.15.31:0/629446437 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1a8009ae0 con 0x7fb1b015dfd0 2023-10-06T02:27:40.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.968+0000 7fb1affff700 1 --2- 172.21.15.31:0/629446437 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1b015dfd0 0x7fb1b0156a50 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fb1b006ad70 tx=0x7fb1a8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:40.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.969+0000 7fb1977fe700 1 -- 172.21.15.31:0/629446437 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a800bee0 con 0x7fb1b015dfd0 2023-10-06T02:27:40.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.969+0000 7fb1977fe700 1 -- 172.21.15.31:0/629446437 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1a80032f0 con 0x7fb1b015dfd0 2023-10-06T02:27:40.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.969+0000 7fb1977fe700 1 -- 172.21.15.31:0/629446437 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a801d420 con 0x7fb1b015dfd0 2023-10-06T02:27:40.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.969+0000 7fb1b4b9d700 1 -- 172.21.15.31:0/629446437 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1b00c95c0 con 0x7fb1b015dfd0 2023-10-06T02:27:40.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.969+0000 7fb1b4b9d700 1 -- 172.21.15.31:0/629446437 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1b00c9b80 con 0x7fb1b015dfd0 2023-10-06T02:27:40.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.970+0000 7fb1b4b9d700 1 -- 172.21.15.31:0/629446437 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1b00c9da0 con 0x7fb1b015dfd0 2023-10-06T02:27:40.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.970+0000 7fb1977fe700 1 -- 172.21.15.31:0/629446437 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 42643+0+0 (secure 0 0 0) 0x7fb1a801d850 con 0x7fb1b015dfd0 2023-10-06T02:27:40.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.971+0000 7fb1977fe700 1 --2- 172.21.15.31:0/629446437 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb19c035900 0x7fb19c037db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:40.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.971+0000 7fb1977fe700 1 -- 172.21.15.31:0/629446437 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6617+0+0 (secure 0 0 0) 0x7fb1a8049d30 con 0x7fb1b015dfd0 2023-10-06T02:27:40.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.971+0000 7fb1af7fe700 1 --2- 172.21.15.31:0/629446437 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb19c035900 0x7fb19c037db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:40.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.972+0000 7fb1af7fe700 1 --2- 172.21.15.31:0/629446437 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb19c035900 0x7fb19c037db0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb1a0003a30 tx=0x7fb1a0007ea0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:40.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:40.976+0000 7fb1977fe700 1 -- 172.21.15.31:0/629446437 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb1a80076e0 con 0x7fb1b015dfd0 2023-10-06T02:27:41.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.178+0000 7fb1b4b9d700 1 -- 172.21.15.31:0/629446437 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb1b004c3c0 con 0x7fb1b015dfd0 2023-10-06T02:27:41.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.178+0000 7fb1977fe700 1 -- 172.21.15.31:0/629446437 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v76) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb1a8007320 con 0x7fb1b015dfd0 2023-10-06T02:27:41.179 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 8 2023-10-06T02:27:41.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.181+0000 7fb1b4b9d700 1 -- 172.21.15.31:0/629446437 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb19c035900 msgr2=0x7fb19c037db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:41.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.181+0000 7fb1b4b9d700 1 --2- 172.21.15.31:0/629446437 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb19c035900 0x7fb19c037db0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb1a0003a30 tx=0x7fb1a0007ea0 comp rx=0 tx=0).stop 2023-10-06T02:27:41.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.181+0000 7fb1b4b9d700 1 -- 172.21.15.31:0/629446437 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1b015dfd0 msgr2=0x7fb1b0156a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:41.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.181+0000 7fb1b4b9d700 1 --2- 172.21.15.31:0/629446437 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1b015dfd0 0x7fb1b0156a50 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fb1b006ad70 tx=0x7fb1a8017000 comp rx=0 tx=0).stop 2023-10-06T02:27:41.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.181+0000 7fb1b4b9d700 1 -- 172.21.15.31:0/629446437 shutdown_connections 2023-10-06T02:27:41.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.181+0000 7fb1b4b9d700 1 --2- 172.21.15.31:0/629446437 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1b015dfd0 0x7fb1b0156a50 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:41.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.181+0000 7fb1b4b9d700 1 --2- 172.21.15.31:0/629446437 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1b0159550 0x7fb1b00ca0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:41.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.181+0000 7fb1b4b9d700 1 --2- 172.21.15.31:0/629446437 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb19c035900 0x7fb19c037db0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:41.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.181+0000 7fb1b4b9d700 1 --2- 172.21.15.31:0/629446437 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb1b0159920 0x7fb1b0142e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:41.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.181+0000 7fb1b4b9d700 1 -- 172.21.15.31:0/629446437 >> 172.21.15.31:0/629446437 conn(0x7fb1b006ce80 msgr2=0x7fb1b01527d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:41.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.181+0000 7fb1b4b9d700 1 -- 172.21.15.31:0/629446437 shutdown_connections 2023-10-06T02:27:41.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.181+0000 7fb1b4b9d700 1 -- 172.21.15.31:0/629446437 wait complete. 2023-10-06T02:27:41.200 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T02:27:41.426 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:27:41.425+0000 7f54bc28a180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T02:27:41.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.433+0000 7f25449fe700 1 Processor -- start 2023-10-06T02:27:41.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.433+0000 7f25449fe700 1 -- start start 2023-10-06T02:27:41.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.435+0000 7f25449fe700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2540153a00 0x7f2540153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:41.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.435+0000 7f25449fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f25400cea00 0x7f25400cee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:41.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.435+0000 7f25449fe700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f254015ac50 con 0x7f254015de50 2023-10-06T02:27:41.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.435+0000 7f25449fe700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f254015d010 con 0x7f2540153a00 2023-10-06T02:27:41.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.435+0000 7f25449fe700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25400cf3c0 con 0x7f25400cea00 2023-10-06T02:27:41.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.435+0000 7f253e59c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f254015de50 0x7f254015a540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:37446/0 (socket says 172.21.15.31:37446) 2023-10-06T02:27:41.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.435+0000 7f253e59c700 1 -- 172.21.15.31:0/2414781594 learned_addr learned my addr 172.21.15.31:0/2414781594 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:27:41.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.435+0000 7f253ed9d700 1 --2- 172.21.15.31:0/2414781594 >> v2:172.21.15.62:3300/0 conn(0x7f2540153a00 0x7f2540153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:41.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.435+0000 7f253f59e700 1 --2- 172.21.15.31:0/2414781594 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f25400cea00 0x7f25400cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:41.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.435+0000 7f253dd9b700 1 -- 172.21.15.31:0/2414781594 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 302782658 0 0) 0x7f254015ac50 con 0x7f254015de50 2023-10-06T02:27:41.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.435+0000 7f253dd9b700 1 -- 172.21.15.31:0/2414781594 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2528003570 con 0x7f254015de50 2023-10-06T02:27:41.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.436+0000 7f253ed9d700 1 -- 172.21.15.31:0/2414781594 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f25400cea00 msgr2=0x7f25400cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:41.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.436+0000 7f253ed9d700 1 --2- 172.21.15.31:0/2414781594 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f25400cea00 0x7f25400cee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:41.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.436+0000 7f253ed9d700 1 -- 172.21.15.31:0/2414781594 >> v1:172.21.15.31:6789/0 conn(0x7f254015de50 legacy=0x7f254015a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:41.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.436+0000 7f253ed9d700 1 -- 172.21.15.31:0/2414781594 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f254015af60 con 0x7f2540153a00 2023-10-06T02:27:41.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.436+0000 7f253ed9d700 1 --2- 172.21.15.31:0/2414781594 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2540153a00 0x7f2540153dd0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f25300098e0 tx=0x7f25300091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13417fb7954a26f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:41.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.436+0000 7f253dd9b700 1 -- 172.21.15.31:0/2414781594 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2530011630 con 0x7f2540153a00 2023-10-06T02:27:41.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.436+0000 7f253dd9b700 1 -- 172.21.15.31:0/2414781594 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2530011790 con 0x7f2540153a00 2023-10-06T02:27:41.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.436+0000 7f253dd9b700 1 -- 172.21.15.31:0/2414781594 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2530011b30 con 0x7f2540153a00 2023-10-06T02:27:41.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.437+0000 7f25449fe700 1 -- 172.21.15.31:0/2414781594 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2540153a00 msgr2=0x7f2540153dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:41.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.437+0000 7f25449fe700 1 --2- 172.21.15.31:0/2414781594 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2540153a00 0x7f2540153dd0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f25300098e0 tx=0x7f25300091a0 comp rx=0 tx=0).stop 2023-10-06T02:27:41.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.437+0000 7f25449fe700 1 -- 172.21.15.31:0/2414781594 shutdown_connections 2023-10-06T02:27:41.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.437+0000 7f25449fe700 1 --2- 172.21.15.31:0/2414781594 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2540153a00 0x7f2540153dd0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:41.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.437+0000 7f25449fe700 1 --2- 172.21.15.31:0/2414781594 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f25400cea00 0x7f25400cee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:41.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.437+0000 7f25449fe700 1 -- 172.21.15.31:0/2414781594 >> 172.21.15.31:0/2414781594 conn(0x7f25400703a0 msgr2=0x7f25400707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:41.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.437+0000 7f25449fe700 1 -- 172.21.15.31:0/2414781594 shutdown_connections 2023-10-06T02:27:41.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.437+0000 7f25449fe700 1 -- 172.21.15.31:0/2414781594 wait complete. 2023-10-06T02:27:41.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.438+0000 7f25449fe700 1 Processor -- start 2023-10-06T02:27:41.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.439+0000 7f25449fe700 1 -- start start 2023-10-06T02:27:41.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.439+0000 7f25449fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f25400cea00 0x7f2540166ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:41.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.440+0000 7f25449fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2540153a00 0x7f2540169cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:41.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.440+0000 7f25449fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f254015de50 0x7f2540168b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:41.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.440+0000 7f25449fe700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25400c89c0 con 0x7f2540153a00 2023-10-06T02:27:41.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.440+0000 7f25449fe700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f254016b690 con 0x7f25400cea00 2023-10-06T02:27:41.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.440+0000 7f25449fe700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2540167530 con 0x7f254015de50 2023-10-06T02:27:41.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.440+0000 7f253e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2540153a00 0x7f2540169cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:41.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.440+0000 7f253e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2540153a00 0x7f2540169cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50890/0 (socket says 172.21.15.31:50890) 2023-10-06T02:27:41.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.440+0000 7f253e59c700 1 -- 172.21.15.31:0/3980995045 learned_addr learned my addr 172.21.15.31:0/3980995045 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:41.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.440+0000 7f253f59e700 1 --2- 172.21.15.31:0/3980995045 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f254015de50 0x7f2540168b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:41.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.440+0000 7f253ed9d700 1 --2- 172.21.15.31:0/3980995045 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f25400cea00 0x7f2540166ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:41.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.440+0000 7f253e59c700 1 -- 172.21.15.31:0/3980995045 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f254015de50 msgr2=0x7f2540168b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:41.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.440+0000 7f253e59c700 1 --2- 172.21.15.31:0/3980995045 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f254015de50 0x7f2540168b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:41.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.440+0000 7f253e59c700 1 -- 172.21.15.31:0/3980995045 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f25400cea00 msgr2=0x7f2540166ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:41.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.440+0000 7f253e59c700 1 --2- 172.21.15.31:0/3980995045 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f25400cea00 0x7f2540166ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:41.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.440+0000 7f253e59c700 1 -- 172.21.15.31:0/3980995045 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2530009420 con 0x7f2540153a00 2023-10-06T02:27:41.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.440+0000 7f253ed9d700 1 --2- 172.21.15.31:0/3980995045 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f25400cea00 0x7f2540166ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:27:41.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.441+0000 7f253e59c700 1 --2- 172.21.15.31:0/3980995045 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2540153a00 0x7f2540169cb0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f253400c5c0 tx=0x7f2534007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:41.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.441+0000 7f2526ffd700 1 -- 172.21.15.31:0/3980995045 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2534005b30 con 0x7f2540153a00 2023-10-06T02:27:41.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.441+0000 7f2526ffd700 1 -- 172.21.15.31:0/3980995045 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2534005cb0 con 0x7f2540153a00 2023-10-06T02:27:41.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.441+0000 7f2526ffd700 1 -- 172.21.15.31:0/3980995045 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2534007320 con 0x7f2540153a00 2023-10-06T02:27:41.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.441+0000 7f25449fe700 1 -- 172.21.15.31:0/3980995045 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f254016aae0 con 0x7f2540153a00 2023-10-06T02:27:41.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.441+0000 7f25449fe700 1 -- 172.21.15.31:0/3980995045 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f254016b320 con 0x7f2540153a00 2023-10-06T02:27:41.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.442+0000 7f2526ffd700 1 -- 172.21.15.31:0/3980995045 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 42643+0+0 (secure 0 0 0) 0x7f253401b3b0 con 0x7f2540153a00 2023-10-06T02:27:41.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.442+0000 7f25449fe700 1 -- 172.21.15.31:0/3980995045 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f254016b840 con 0x7f2540153a00 2023-10-06T02:27:41.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.443+0000 7f2526ffd700 1 --2- 172.21.15.31:0/3980995045 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f252c035900 0x7f252c037db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:41.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.443+0000 7f2526ffd700 1 -- 172.21.15.31:0/3980995045 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6617+0+0 (secure 0 0 0) 0x7f2534047bc0 con 0x7f2540153a00 2023-10-06T02:27:41.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.443+0000 7f253ed9d700 1 --2- 172.21.15.31:0/3980995045 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f252c035900 0x7f252c037db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:41.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.444+0000 7f253ed9d700 1 --2- 172.21.15.31:0/3980995045 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f252c035900 0x7f252c037db0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f25300091e0 tx=0x7f2530017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:41.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.451+0000 7f2526ffd700 1 -- 172.21.15.31:0/3980995045 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f254016b840 con 0x7f2540153a00 2023-10-06T02:27:41.529 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:27:41.529+0000 7f54bc28a180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T02:27:41.624 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:27:41.624+0000 7f54bc28a180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T02:27:41.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.655+0000 7f25449fe700 1 -- 172.21.15.31:0/3980995045 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f254016b840 con 0x7f2540153a00 2023-10-06T02:27:41.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.656+0000 7f2526ffd700 1 -- 172.21.15.31:0/3980995045 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v76) v1 ==== 104+0+11 (secure 0 0 0) 0x7f254016b840 con 0x7f2540153a00 2023-10-06T02:27:41.656 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-10-06T02:27:41.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.658+0000 7f25449fe700 1 -- 172.21.15.31:0/3980995045 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f252c035900 msgr2=0x7f252c037db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:41.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.658+0000 7f25449fe700 1 --2- 172.21.15.31:0/3980995045 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f252c035900 0x7f252c037db0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f25300091e0 tx=0x7f2530017000 comp rx=0 tx=0).stop 2023-10-06T02:27:41.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.658+0000 7f25449fe700 1 -- 172.21.15.31:0/3980995045 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2540153a00 msgr2=0x7f2540169cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:41.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.658+0000 7f25449fe700 1 --2- 172.21.15.31:0/3980995045 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2540153a00 0x7f2540169cb0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f253400c5c0 tx=0x7f2534007e00 comp rx=0 tx=0).stop 2023-10-06T02:27:41.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.658+0000 7f25449fe700 1 -- 172.21.15.31:0/3980995045 shutdown_connections 2023-10-06T02:27:41.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.658+0000 7f25449fe700 1 --2- 172.21.15.31:0/3980995045 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f252c035900 0x7f252c037db0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:41.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.658+0000 7f25449fe700 1 --2- 172.21.15.31:0/3980995045 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f254015de50 0x7f2540168b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:41.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.658+0000 7f25449fe700 1 --2- 172.21.15.31:0/3980995045 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f25400cea00 0x7f2540166ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:41.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.658+0000 7f25449fe700 1 --2- 172.21.15.31:0/3980995045 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2540153a00 0x7f2540169cb0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:41.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.658+0000 7f25449fe700 1 -- 172.21.15.31:0/3980995045 >> 172.21.15.31:0/3980995045 conn(0x7f25400703a0 msgr2=0x7f2540152c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:41.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.658+0000 7f25449fe700 1 -- 172.21.15.31:0/3980995045 shutdown_connections 2023-10-06T02:27:41.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.658+0000 7f25449fe700 1 -- 172.21.15.31:0/3980995045 wait complete. 2023-10-06T02:27:41.676 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T02:27:41.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.872+0000 7f6c6bfff700 1 Processor -- start 2023-10-06T02:27:41.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.872+0000 7f6c6bfff700 1 -- start start 2023-10-06T02:27:41.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.873+0000 7f6c6bfff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6c6c06e560 0x7f6c6c0c7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:41.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.873+0000 7f6c6bfff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c6c0c7e70 con 0x7f6c6c155890 2023-10-06T02:27:41.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.873+0000 7f6c6bfff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c6c06b790 con 0x7f6c6c155c60 2023-10-06T02:27:41.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.873+0000 7f6c6bfff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c6c06ab80 con 0x7f6c6c06e560 2023-10-06T02:27:41.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.873+0000 7f6c6affd700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f6c6c155890 0x7f6c6c06a3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:37458/0 (socket says 172.21.15.31:37458) 2023-10-06T02:27:41.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.873+0000 7f6c6affd700 1 -- 172.21.15.31:0/3724972211 learned_addr learned my addr 172.21.15.31:0/3724972211 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:27:41.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.874+0000 7f6c6b7fe700 1 --2- 172.21.15.31:0/3724972211 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6c6c06e560 0x7f6c6c0c7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:41.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.874+0000 7f6c69ffb700 1 -- 172.21.15.31:0/3724972211 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2404263559 0 0) 0x7f6c6c0c7e70 con 0x7f6c6c155890 2023-10-06T02:27:41.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.874+0000 7f6c69ffb700 1 -- 172.21.15.31:0/3724972211 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c54004680 con 0x7f6c6c155890 2023-10-06T02:27:41.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.874+0000 7f6c69ffb700 1 -- 172.21.15.31:0/3724972211 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3148458881 0 0) 0x7f6c6c06b790 con 0x7f6c6c155c60 2023-10-06T02:27:41.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.874+0000 7f6c69ffb700 1 -- 172.21.15.31:0/3724972211 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c6c0c7e70 con 0x7f6c6c155c60 2023-10-06T02:27:41.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.874+0000 7f6c69ffb700 1 -- 172.21.15.31:0/3724972211 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2643580099 0 0) 0x7f6c54004680 con 0x7f6c6c155890 2023-10-06T02:27:41.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.874+0000 7f6c69ffb700 1 -- 172.21.15.31:0/3724972211 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6c6c06e560 msgr2=0x7f6c6c0c7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:41.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.874+0000 7f6c69ffb700 1 --2- 172.21.15.31:0/3724972211 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6c6c06e560 0x7f6c6c0c7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:41.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.874+0000 7f6c69ffb700 1 -- 172.21.15.31:0/3724972211 >> v1:172.21.15.62:6789/0 conn(0x7f6c6c155c60 legacy=0x7f6c6c06de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:41.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.874+0000 7f6c69ffb700 1 -- 172.21.15.31:0/3724972211 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c6c06ae60 con 0x7f6c6c155890 2023-10-06T02:27:41.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.874+0000 7f6c69ffb700 1 -- 172.21.15.31:0/3724972211 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7f6c5c001470 con 0x7f6c6c155890 2023-10-06T02:27:41.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.874+0000 7f6c69ffb700 1 -- 172.21.15.31:0/3724972211 >> v1:172.21.15.31:6789/0 conn(0x7f6c6c155890 legacy=0x7f6c6c06a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:41.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.874+0000 7f6c69ffb700 1 --2- 172.21.15.31:0/3724972211 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6c6c06e560 0x7f6c54004d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:41.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.875+0000 7f6c69ffb700 1 --2- 172.21.15.31:0/3724972211 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c540052e0 0x7f6c54005920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:41.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.875+0000 7f6c69ffb700 1 --2- 172.21.15.31:0/3724972211 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c54005e60 0x7f6c540062d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:41.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.875+0000 7f6c69ffb700 1 -- 172.21.15.31:0/3724972211 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c6c06ab80 con 0x7f6c540052e0 2023-10-06T02:27:41.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.875+0000 7f6c69ffb700 1 -- 172.21.15.31:0/3724972211 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c54003490 con 0x7f6c54005e60 2023-10-06T02:27:41.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.875+0000 7f6c69ffb700 1 -- 172.21.15.31:0/3724972211 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c54003650 con 0x7f6c6c06e560 2023-10-06T02:27:41.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.875+0000 7f6c6affd700 1 --2- 172.21.15.31:0/3724972211 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6c6c06e560 0x7f6c54004d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:41.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.875+0000 7f6c6a7fc700 1 --2- 172.21.15.31:0/3724972211 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c540052e0 0x7f6c54005920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:41.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.875+0000 7f6c6b7fe700 1 --2- 172.21.15.31:0/3724972211 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c54005e60 0x7f6c540062d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:41.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.875+0000 7f6c6a7fc700 1 -- 172.21.15.31:0/3724972211 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6c6c06e560 msgr2=0x7f6c54004d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:41.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.875+0000 7f6c6a7fc700 1 --2- 172.21.15.31:0/3724972211 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6c6c06e560 0x7f6c54004d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:41.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.875+0000 7f6c6a7fc700 1 -- 172.21.15.31:0/3724972211 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c54005e60 msgr2=0x7f6c540062d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:41.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.875+0000 7f6c6a7fc700 1 --2- 172.21.15.31:0/3724972211 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c54005e60 0x7f6c540062d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:41.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.875+0000 7f6c6a7fc700 1 -- 172.21.15.31:0/3724972211 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c54004680 con 0x7f6c540052e0 2023-10-06T02:27:41.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.875+0000 7f6c6affd700 1 --2- 172.21.15.31:0/3724972211 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6c6c06e560 0x7f6c54004d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:27:41.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.876+0000 7f6c6a7fc700 1 --2- 172.21.15.31:0/3724972211 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c540052e0 0x7f6c54005920 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f6c60001a10 tx=0x7f6c60007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce0a307ec5c623e6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:41.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.876+0000 7f6c69ffb700 1 -- 172.21.15.31:0/3724972211 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c6000c5d0 con 0x7f6c540052e0 2023-10-06T02:27:41.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.876+0000 7f6c69ffb700 1 -- 172.21.15.31:0/3724972211 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c6000c730 con 0x7f6c540052e0 2023-10-06T02:27:41.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.876+0000 7f6c69ffb700 1 -- 172.21.15.31:0/3724972211 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c6000cad0 con 0x7f6c540052e0 2023-10-06T02:27:41.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.877+0000 7f6c6bfff700 1 -- 172.21.15.31:0/3724972211 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c540052e0 msgr2=0x7f6c54005920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:41.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.877+0000 7f6c6bfff700 1 --2- 172.21.15.31:0/3724972211 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c540052e0 0x7f6c54005920 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f6c60001a10 tx=0x7f6c60007d30 comp rx=0 tx=0).stop 2023-10-06T02:27:41.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.877+0000 7f6c6b7fe700 1 -- 172.21.15.31:0/3724972211 reap_dead start 2023-10-06T02:27:41.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.877+0000 7f6c6bfff700 1 -- 172.21.15.31:0/3724972211 shutdown_connections 2023-10-06T02:27:41.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.877+0000 7f6c6bfff700 1 -- 172.21.15.31:0/3724972211 >> 172.21.15.31:0/3724972211 conn(0x7f6c6c06d110 msgr2=0x7f6c6c0cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:41.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.877+0000 7f6c6bfff700 1 -- 172.21.15.31:0/3724972211 shutdown_connections 2023-10-06T02:27:41.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.877+0000 7f6c6bfff700 1 -- 172.21.15.31:0/3724972211 wait complete. 2023-10-06T02:27:41.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.878+0000 7f6c6bfff700 1 Processor -- start 2023-10-06T02:27:41.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.878+0000 7f6c6bfff700 1 -- start start 2023-10-06T02:27:41.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.879+0000 7f6c6bfff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6c540052e0 0x7f6c6c141e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:41.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.879+0000 7f6c6bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c6c153830 0x7f6c6c153ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:41.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.879+0000 7f6c6bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c6c0b02a0 0x7f6c6c0b0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:41.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.879+0000 7f6c6bfff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c6c1423b0 con 0x7f6c6c0b02a0 2023-10-06T02:27:41.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.879+0000 7f6c6bfff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c6c1541e0 con 0x7f6c6c153830 2023-10-06T02:27:41.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.879+0000 7f6c6bfff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c6c0afe10 con 0x7f6c540052e0 2023-10-06T02:27:41.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.879+0000 7f6c6b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c6c0b02a0 0x7f6c6c0b0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:41.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.880+0000 7f6c6affd700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6c540052e0 0x7f6c6c141e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:41.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.880+0000 7f6c6b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c6c0b02a0 0x7f6c6c0b0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50908/0 (socket says 172.21.15.31:50908) 2023-10-06T02:27:41.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.880+0000 7f6c6b7fe700 1 -- 172.21.15.31:0/1369619989 learned_addr learned my addr 172.21.15.31:0/1369619989 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:41.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.880+0000 7f6c6a7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c6c153830 0x7f6c6c153ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:41.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.880+0000 7f6c6b7fe700 1 -- 172.21.15.31:0/1369619989 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6c540052e0 msgr2=0x7f6c6c141e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:41.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.880+0000 7f6c6b7fe700 1 --2- 172.21.15.31:0/1369619989 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6c540052e0 0x7f6c6c141e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:41.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.880+0000 7f6c6b7fe700 1 -- 172.21.15.31:0/1369619989 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c6c153830 msgr2=0x7f6c6c153ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:41.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.880+0000 7f6c6b7fe700 1 --2- 172.21.15.31:0/1369619989 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c6c153830 0x7f6c6c153ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:41.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.880+0000 7f6c6b7fe700 1 -- 172.21.15.31:0/1369619989 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c6c0af4e0 con 0x7f6c6c0b02a0 2023-10-06T02:27:41.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.880+0000 7f6c6affd700 1 --2- 172.21.15.31:0/1369619989 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6c540052e0 0x7f6c6c141e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:27:41.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.880+0000 7f6c6b7fe700 1 --2- 172.21.15.31:0/1369619989 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c6c0b02a0 0x7f6c6c0b0720 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c0c6720 tx=0x7f6c6400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:41.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.881+0000 7f6c52ffd700 1 -- 172.21.15.31:0/1369619989 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c6400e040 con 0x7f6c6c0b02a0 2023-10-06T02:27:41.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.881+0000 7f6c52ffd700 1 -- 172.21.15.31:0/1369619989 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c6400f040 con 0x7f6c6c0b02a0 2023-10-06T02:27:41.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.881+0000 7f6c52ffd700 1 -- 172.21.15.31:0/1369619989 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c6400fda0 con 0x7f6c6c0b02a0 2023-10-06T02:27:41.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.881+0000 7f6c6bfff700 1 -- 172.21.15.31:0/1369619989 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c6c0af7c0 con 0x7f6c6c0b02a0 2023-10-06T02:27:41.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.881+0000 7f6c6bfff700 1 -- 172.21.15.31:0/1369619989 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c6c0c51c0 con 0x7f6c6c0b02a0 2023-10-06T02:27:41.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.882+0000 7f6c6bfff700 1 -- 172.21.15.31:0/1369619989 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c6c0affa0 con 0x7f6c6c0b02a0 2023-10-06T02:27:41.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.882+0000 7f6c52ffd700 1 -- 172.21.15.31:0/1369619989 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 42643+0+0 (secure 0 0 0) 0x7f6c64015640 con 0x7f6c6c0b02a0 2023-10-06T02:27:41.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.883+0000 7f6c52ffd700 1 --2- 172.21.15.31:0/1369619989 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6c58035900 0x7f6c58037db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:41.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.883+0000 7f6c52ffd700 1 -- 172.21.15.31:0/1369619989 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6617+0+0 (secure 0 0 0) 0x7f6c64048df0 con 0x7f6c6c0b02a0 2023-10-06T02:27:41.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.883+0000 7f6c6affd700 1 --2- 172.21.15.31:0/1369619989 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6c58035900 0x7f6c58037db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:41.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.884+0000 7f6c6affd700 1 --2- 172.21.15.31:0/1369619989 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6c58035900 0x7f6c58037db0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c001030 tx=0x7f6c5c001d40 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:41.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:41.888+0000 7f6c52ffd700 1 -- 172.21.15.31:0/1369619989 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6c64013d70 con 0x7f6c6c0b02a0 2023-10-06T02:27:42.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.090+0000 7f6c6bfff700 1 -- 172.21.15.31:0/1369619989 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6c6c04c3c0 con 0x7f6c6c0b02a0 2023-10-06T02:27:42.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.091+0000 7f6c52ffd700 1 -- 172.21.15.31:0/1369619989 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v76) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6c64013310 con 0x7f6c6c0b02a0 2023-10-06T02:27:42.091 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-10-06T02:27:42.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.093+0000 7f6c6bfff700 1 -- 172.21.15.31:0/1369619989 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6c58035900 msgr2=0x7f6c58037db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:42.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.093+0000 7f6c6bfff700 1 --2- 172.21.15.31:0/1369619989 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6c58035900 0x7f6c58037db0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c001030 tx=0x7f6c5c001d40 comp rx=0 tx=0).stop 2023-10-06T02:27:42.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.093+0000 7f6c6bfff700 1 -- 172.21.15.31:0/1369619989 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c6c0b02a0 msgr2=0x7f6c6c0b0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:42.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.093+0000 7f6c6bfff700 1 --2- 172.21.15.31:0/1369619989 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c6c0b02a0 0x7f6c6c0b0720 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c0c6720 tx=0x7f6c6400b040 comp rx=0 tx=0).stop 2023-10-06T02:27:42.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.093+0000 7f6c6bfff700 1 -- 172.21.15.31:0/1369619989 shutdown_connections 2023-10-06T02:27:42.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.093+0000 7f6c6bfff700 1 --2- 172.21.15.31:0/1369619989 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c6c0b02a0 0x7f6c6c0b0720 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:42.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.093+0000 7f6c6bfff700 1 --2- 172.21.15.31:0/1369619989 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6c58035900 0x7f6c58037db0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:42.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.093+0000 7f6c6bfff700 1 --2- 172.21.15.31:0/1369619989 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6c540052e0 0x7f6c6c141e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:42.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.093+0000 7f6c6bfff700 1 --2- 172.21.15.31:0/1369619989 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c6c153830 0x7f6c6c153ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:42.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.093+0000 7f6c6bfff700 1 -- 172.21.15.31:0/1369619989 >> 172.21.15.31:0/1369619989 conn(0x7f6c6c06d110 msgr2=0x7f6c6c1581a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:42.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.093+0000 7f6c6bfff700 1 -- 172.21.15.31:0/1369619989 shutdown_connections 2023-10-06T02:27:42.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.093+0000 7f6c6bfff700 1 -- 172.21.15.31:0/1369619989 wait complete. 2023-10-06T02:27:42.111 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:27:42.291 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:27:42.290+0000 7f54bc28a180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T02:27:42.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.331+0000 7fc7a0d9d700 1 Processor -- start 2023-10-06T02:27:42.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.332+0000 7fc7a0d9d700 1 -- start start 2023-10-06T02:27:42.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.333+0000 7fc7a0d9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc79c0ca7b0 0x7fc79c0cac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:42.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.333+0000 7fc7a0d9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc79c06d3e0 con 0x7fc79c1596d0 2023-10-06T02:27:42.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.333+0000 7fc7a0d9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc79c06c7d0 con 0x7fc79c159aa0 2023-10-06T02:27:42.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.333+0000 7fc7a0d9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc79c0cb170 con 0x7fc79c0ca7b0 2023-10-06T02:27:42.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.333+0000 7fc79b7fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fc79c1596d0 0x7fc79c06d870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:37470/0 (socket says 172.21.15.31:37470) 2023-10-06T02:27:42.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.333+0000 7fc79b7fe700 1 -- 172.21.15.31:0/457196649 learned_addr learned my addr 172.21.15.31:0/457196649 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:27:42.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.334+0000 7fc79bfff700 1 --2- 172.21.15.31:0/457196649 >> v2:172.21.15.62:3300/0 conn(0x7fc79c0ca7b0 0x7fc79c0cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:42.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.334+0000 7fc79a7fc700 1 -- 172.21.15.31:0/457196649 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2175265392 0 0) 0x7fc79c06d3e0 con 0x7fc79c1596d0 2023-10-06T02:27:42.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.334+0000 7fc79a7fc700 1 -- 172.21.15.31:0/457196649 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc784003570 con 0x7fc79c1596d0 2023-10-06T02:27:42.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.334+0000 7fc79bfff700 1 -- 172.21.15.31:0/457196649 >> v1:172.21.15.62:6789/0 conn(0x7fc79c159aa0 legacy=0x7fc79c0703a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:42.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.334+0000 7fc79bfff700 1 -- 172.21.15.31:0/457196649 >> v1:172.21.15.31:6789/0 conn(0x7fc79c1596d0 legacy=0x7fc79c06d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:42.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.334+0000 7fc79a7fc700 1 -- 172.21.15.31:0/457196649 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4040557563 0 0) 0x7fc784003570 con 0x7fc79c1596d0 2023-10-06T02:27:42.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.334+0000 7fc79bfff700 1 -- 172.21.15.31:0/457196649 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc79c06cab0 con 0x7fc79c0ca7b0 2023-10-06T02:27:42.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.334+0000 7fc79bfff700 1 --2- 172.21.15.31:0/457196649 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc79c0ca7b0 0x7fc79c0cac30 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc794009e60 tx=0x7fc794009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2b42faabc7dc176 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:42.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.335+0000 7fc79a7fc700 1 -- 172.21.15.31:0/457196649 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc79400e070 con 0x7fc79c0ca7b0 2023-10-06T02:27:42.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.335+0000 7fc79a7fc700 1 -- 172.21.15.31:0/457196649 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc79400b850 con 0x7fc79c0ca7b0 2023-10-06T02:27:42.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.335+0000 7fc79a7fc700 1 -- 172.21.15.31:0/457196649 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78c001470 con 0x7fc79c0ca7b0 2023-10-06T02:27:42.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.335+0000 7fc7a0d9d700 1 -- 172.21.15.31:0/457196649 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc79c0ca7b0 msgr2=0x7fc79c0cac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:42.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.335+0000 7fc7a0d9d700 1 --2- 172.21.15.31:0/457196649 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc79c0ca7b0 0x7fc79c0cac30 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc794009e60 tx=0x7fc794009780 comp rx=0 tx=0).stop 2023-10-06T02:27:42.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.335+0000 7fc7a0d9d700 1 -- 172.21.15.31:0/457196649 shutdown_connections 2023-10-06T02:27:42.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.335+0000 7fc7a0d9d700 1 --2- 172.21.15.31:0/457196649 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc79c0ca7b0 0x7fc79c0cac30 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:42.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.335+0000 7fc7a0d9d700 1 -- 172.21.15.31:0/457196649 >> 172.21.15.31:0/457196649 conn(0x7fc79c06e720 msgr2=0x7fc79c06eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:42.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.335+0000 7fc7a0d9d700 1 -- 172.21.15.31:0/457196649 shutdown_connections 2023-10-06T02:27:42.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.335+0000 7fc7a0d9d700 1 -- 172.21.15.31:0/457196649 wait complete. 2023-10-06T02:27:42.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.337+0000 7fc7a0d9d700 1 Processor -- start 2023-10-06T02:27:42.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.337+0000 7fc7a0d9d700 1 -- start start 2023-10-06T02:27:42.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.338+0000 7fc7a0d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc79c0ca7b0 0x7fc79c0b2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:42.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.338+0000 7fc7a0d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc79c1596d0 0x7fc79c0afb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:42.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.338+0000 7fc7a0d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc79c159aa0 0x7fc79c0c1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:42.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.338+0000 7fc7a0d9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc79c0c6ff0 con 0x7fc79c0ca7b0 2023-10-06T02:27:42.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.338+0000 7fc7a0d9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc79c0c5280 con 0x7fc79c159aa0 2023-10-06T02:27:42.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.338+0000 7fc7a0d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc79c0c4670 con 0x7fc79c1596d0 2023-10-06T02:27:42.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.338+0000 7fc79b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc79c0ca7b0 0x7fc79c0b2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:42.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.338+0000 7fc79b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc79c0ca7b0 0x7fc79c0b2250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50914/0 (socket says 172.21.15.31:50914) 2023-10-06T02:27:42.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.338+0000 7fc79b7fe700 1 -- 172.21.15.31:0/3116568566 learned_addr learned my addr 172.21.15.31:0/3116568566 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:42.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.339+0000 7fc79b7fe700 1 -- 172.21.15.31:0/3116568566 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc79c1596d0 msgr2=0x7fc79c0afb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:42.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.339+0000 7fc79b7fe700 1 --2- 172.21.15.31:0/3116568566 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc79c1596d0 0x7fc79c0afb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:42.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.339+0000 7fc79b7fe700 1 -- 172.21.15.31:0/3116568566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc79c159aa0 msgr2=0x7fc79c0c1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:42.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.339+0000 7fc79b7fe700 1 --2- 172.21.15.31:0/3116568566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc79c159aa0 0x7fc79c0c1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:42.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.339+0000 7fc79b7fe700 1 -- 172.21.15.31:0/3116568566 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc794009a20 con 0x7fc79c0ca7b0 2023-10-06T02:27:42.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.339+0000 7fc79b7fe700 1 --2- 172.21.15.31:0/3116568566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc79c0ca7b0 0x7fc79c0b2250 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fc78c00c7f0 tx=0x7fc78c0019e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:42.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.339+0000 7fc7837fe700 1 -- 172.21.15.31:0/3116568566 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78c007360 con 0x7fc79c0ca7b0 2023-10-06T02:27:42.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.339+0000 7fc7837fe700 1 -- 172.21.15.31:0/3116568566 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc78c0074c0 con 0x7fc79c0ca7b0 2023-10-06T02:27:42.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.339+0000 7fc7a0d9d700 1 -- 172.21.15.31:0/3116568566 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc79c0c4950 con 0x7fc79c0ca7b0 2023-10-06T02:27:42.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.339+0000 7fc7837fe700 1 -- 172.21.15.31:0/3116568566 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78c004830 con 0x7fc79c0ca7b0 2023-10-06T02:27:42.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.339+0000 7fc7a0d9d700 1 -- 172.21.15.31:0/3116568566 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc79c0c4f10 con 0x7fc79c0ca7b0 2023-10-06T02:27:42.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.340+0000 7fc7a0d9d700 1 -- 172.21.15.31:0/3116568566 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc79c04c3c0 con 0x7fc79c0ca7b0 2023-10-06T02:27:42.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.340+0000 7fc7837fe700 1 -- 172.21.15.31:0/3116568566 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 42643+0+0 (secure 0 0 0) 0x7fc78c004c20 con 0x7fc79c0ca7b0 2023-10-06T02:27:42.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.341+0000 7fc7837fe700 1 --2- 172.21.15.31:0/3116568566 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc788035900 0x7fc788037db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:42.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.341+0000 7fc7837fe700 1 -- 172.21.15.31:0/3116568566 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6617+0+0 (secure 0 0 0) 0x7fc78c047e00 con 0x7fc79c0ca7b0 2023-10-06T02:27:42.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.341+0000 7fc79affd700 1 --2- 172.21.15.31:0/3116568566 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc788035900 0x7fc788037db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:42.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.342+0000 7fc79affd700 1 --2- 172.21.15.31:0/3116568566 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc788035900 0x7fc788037db0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc79c158fc0 tx=0x7fc790007b40 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:42.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.346+0000 7fc7837fe700 1 -- 172.21.15.31:0/3116568566 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc78c019260 con 0x7fc79c0ca7b0 2023-10-06T02:27:42.472 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:27:42.472+0000 7f54bc28a180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T02:27:42.572 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:27:42.572+0000 7f54bc28a180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T02:27:42.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.596+0000 7fc7a0d9d700 1 -- 172.21.15.31:0/3116568566 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc79c0cb8d0 con 0x7fc79c0ca7b0 2023-10-06T02:27:42.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.597+0000 7fc7837fe700 1 -- 172.21.15.31:0/3116568566 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4596 (secure 0 0 0) 0x7fc78c018070 con 0x7fc79c0ca7b0 2023-10-06T02:27:42.597 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:27:42.598 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5474,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/2085637096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2085637096},{"type":"v1","addr":"172.21.15.62:6834","nonce":2085637096}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":5992,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/573080449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":573080449},{"type":"v1","addr":"172.21.15.31:6836","nonce":573080449}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":6007,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2625778079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2625778079},{"type":"v1","addr":"172.21.15.62:6835","nonce":2625778079}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:27:22.533533+0000","modified":"2023-10-06T02:27:26.062249+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6282":{"gid":6282,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6837/3003674295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":3003674295},{"type":"v1","addr":"172.21.15.31:6837","nonce":3003674295}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T02:27:42.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.599+0000 7fc7a0d9d700 1 -- 172.21.15.31:0/3116568566 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc788035900 msgr2=0x7fc788037db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:42.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.599+0000 7fc7a0d9d700 1 --2- 172.21.15.31:0/3116568566 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc788035900 0x7fc788037db0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc79c158fc0 tx=0x7fc790007b40 comp rx=0 tx=0).stop 2023-10-06T02:27:42.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.599+0000 7fc7a0d9d700 1 -- 172.21.15.31:0/3116568566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc79c0ca7b0 msgr2=0x7fc79c0b2250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:42.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.599+0000 7fc7a0d9d700 1 --2- 172.21.15.31:0/3116568566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc79c0ca7b0 0x7fc79c0b2250 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fc78c00c7f0 tx=0x7fc78c0019e0 comp rx=0 tx=0).stop 2023-10-06T02:27:42.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.600+0000 7fc7a0d9d700 1 -- 172.21.15.31:0/3116568566 shutdown_connections 2023-10-06T02:27:42.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.600+0000 7fc7a0d9d700 1 --2- 172.21.15.31:0/3116568566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc79c159aa0 0x7fc79c0c1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:42.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.600+0000 7fc7a0d9d700 1 --2- 172.21.15.31:0/3116568566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc79c0ca7b0 0x7fc79c0b2250 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:42.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.600+0000 7fc7a0d9d700 1 --2- 172.21.15.31:0/3116568566 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc788035900 0x7fc788037db0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:42.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.600+0000 7fc7a0d9d700 1 --2- 172.21.15.31:0/3116568566 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc79c1596d0 0x7fc79c0afb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:42.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.600+0000 7fc7a0d9d700 1 -- 172.21.15.31:0/3116568566 >> 172.21.15.31:0/3116568566 conn(0x7fc79c06e720 msgr2=0x7fc79c0cc9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:42.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.600+0000 7fc7a0d9d700 1 -- 172.21.15.31:0/3116568566 shutdown_connections 2023-10-06T02:27:42.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.600+0000 7fc7a0d9d700 1 -- 172.21.15.31:0/3116568566 wait complete. 2023-10-06T02:27:42.602 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 47 2023-10-06T02:27:42.618 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:27:42.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.811+0000 7f950f19d700 1 Processor -- start 2023-10-06T02:27:42.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.811+0000 7f950f19d700 1 -- start start 2023-10-06T02:27:42.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.812+0000 7f950f19d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9508153c30 0x7f950815a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:42.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.812+0000 7f950f19d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f950815de50 0x7f95080cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:42.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.812+0000 7f950f19d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f950815afd0 con 0x7f95080cef40 2023-10-06T02:27:42.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.812+0000 7f950f19d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f950815b2b0 con 0x7f950815de50 2023-10-06T02:27:42.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.812+0000 7f950f19d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f950815b590 con 0x7f9508153c30 2023-10-06T02:27:42.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.813+0000 7f950d99a700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f950815de50 0x7f95080cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:42.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.813+0000 7f950e19b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9508153c30 0x7f950815a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:42.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.813+0000 7f950d99a700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f950815de50 0x7f95080cea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:58616/0 (socket says 172.21.15.31:58616) 2023-10-06T02:27:42.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.813+0000 7f950d99a700 1 -- 172.21.15.31:0/664064765 learned_addr learned my addr 172.21.15.31:0/664064765 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:42.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.813+0000 7f950d99a700 1 -- 172.21.15.31:0/664064765 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9508153c30 msgr2=0x7f950815a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:42.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.813+0000 7f950d99a700 1 --2- 172.21.15.31:0/664064765 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9508153c30 0x7f950815a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:42.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.814+0000 7f950d99a700 1 -- 172.21.15.31:0/664064765 >> v1:172.21.15.62:6789/0 conn(0x7f95080cef40 legacy=0x7f950806d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:42.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.814+0000 7f950d99a700 1 -- 172.21.15.31:0/664064765 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f950815b870 con 0x7f950815de50 2023-10-06T02:27:42.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.814+0000 7f950e19b700 1 --2- 172.21.15.31:0/664064765 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9508153c30 0x7f950815a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:27:42.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.814+0000 7f950d99a700 1 --2- 172.21.15.31:0/664064765 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f950815de50 0x7f95080cea00 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f94f80099b0 tx=0x7f94f8009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7f5e64961eee739 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:42.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.814+0000 7f950d199700 1 -- 172.21.15.31:0/664064765 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f8009ea0 con 0x7f950815de50 2023-10-06T02:27:42.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.814+0000 7f950d199700 1 -- 172.21.15.31:0/664064765 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94f800b860 con 0x7f950815de50 2023-10-06T02:27:42.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.814+0000 7f950d199700 1 -- 172.21.15.31:0/664064765 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f800bc00 con 0x7f950815de50 2023-10-06T02:27:42.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.815+0000 7f950f19d700 1 -- 172.21.15.31:0/664064765 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f950815de50 msgr2=0x7f95080cea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:42.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.815+0000 7f950f19d700 1 --2- 172.21.15.31:0/664064765 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f950815de50 0x7f95080cea00 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f94f80099b0 tx=0x7f94f8009250 comp rx=0 tx=0).stop 2023-10-06T02:27:42.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.815+0000 7f950f19d700 1 -- 172.21.15.31:0/664064765 shutdown_connections 2023-10-06T02:27:42.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.815+0000 7f950f19d700 1 --2- 172.21.15.31:0/664064765 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9508153c30 0x7f950815a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:42.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.815+0000 7f950f19d700 1 --2- 172.21.15.31:0/664064765 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f950815de50 0x7f95080cea00 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:42.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.815+0000 7f950f19d700 1 -- 172.21.15.31:0/664064765 >> 172.21.15.31:0/664064765 conn(0x7f95080703a0 msgr2=0x7f95080707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:42.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.815+0000 7f950f19d700 1 -- 172.21.15.31:0/664064765 shutdown_connections 2023-10-06T02:27:42.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.815+0000 7f950f19d700 1 -- 172.21.15.31:0/664064765 wait complete. 2023-10-06T02:27:42.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.816+0000 7f950f19d700 1 Processor -- start 2023-10-06T02:27:42.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.816+0000 7f950f19d700 1 -- start start 2023-10-06T02:27:42.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.817+0000 7f950f19d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f95080cef40 0x7f950816abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:42.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.817+0000 7f950f19d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9508153c30 0x7f95081522b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:42.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.817+0000 7f950f19d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f950815de50 0x7f95080c85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:42.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.817+0000 7f950f19d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95080b3bb0 con 0x7f9508153c30 2023-10-06T02:27:42.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.817+0000 7f950f19d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95080c8f90 con 0x7f950815de50 2023-10-06T02:27:42.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.817+0000 7f950f19d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95080c9270 con 0x7f95080cef40 2023-10-06T02:27:42.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.818+0000 7f950d99a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9508153c30 0x7f95081522b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:42.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.818+0000 7f950d99a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9508153c30 0x7f95081522b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50928/0 (socket says 172.21.15.31:50928) 2023-10-06T02:27:42.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.818+0000 7f950d99a700 1 -- 172.21.15.31:0/343899763 learned_addr learned my addr 172.21.15.31:0/343899763 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:42.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.818+0000 7f950e99c700 1 --2- 172.21.15.31:0/343899763 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f950815de50 0x7f95080c85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:42.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.818+0000 7f950e19b700 1 --2- 172.21.15.31:0/343899763 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f95080cef40 0x7f950816abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:42.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.818+0000 7f950d99a700 1 -- 172.21.15.31:0/343899763 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f95080cef40 msgr2=0x7f950816abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:42.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.818+0000 7f950d99a700 1 --2- 172.21.15.31:0/343899763 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f95080cef40 0x7f950816abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:42.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.818+0000 7f950d99a700 1 -- 172.21.15.31:0/343899763 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f950815de50 msgr2=0x7f95080c85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:42.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.818+0000 7f950d99a700 1 --2- 172.21.15.31:0/343899763 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f950815de50 0x7f95080c85b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:42.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.818+0000 7f950d99a700 1 -- 172.21.15.31:0/343899763 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9504009420 con 0x7f9508153c30 2023-10-06T02:27:42.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.818+0000 7f950e99c700 1 --2- 172.21.15.31:0/343899763 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f950815de50 0x7f95080c85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:27:42.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.818+0000 7f950d99a700 1 --2- 172.21.15.31:0/343899763 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9508153c30 0x7f95081522b0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f950806eb00 tx=0x7f94f8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:42.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.819+0000 7f94fe7fc700 1 -- 172.21.15.31:0/343899763 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f800bbe0 con 0x7f9508153c30 2023-10-06T02:27:42.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.819+0000 7f94fe7fc700 1 -- 172.21.15.31:0/343899763 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94f801a450 con 0x7f9508153c30 2023-10-06T02:27:42.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.819+0000 7f950f19d700 1 -- 172.21.15.31:0/343899763 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94f80094f0 con 0x7f9508153c30 2023-10-06T02:27:42.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.819+0000 7f94fe7fc700 1 -- 172.21.15.31:0/343899763 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f801a810 con 0x7f9508153c30 2023-10-06T02:27:42.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.819+0000 7f950f19d700 1 -- 172.21.15.31:0/343899763 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95080b3840 con 0x7f9508153c30 2023-10-06T02:27:42.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.820+0000 7f94fe7fc700 1 -- 172.21.15.31:0/343899763 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 42643+0+0 (secure 0 0 0) 0x7f94f801abc0 con 0x7f9508153c30 2023-10-06T02:27:42.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.820+0000 7f950f19d700 1 -- 172.21.15.31:0/343899763 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95080b3d20 con 0x7f9508153c30 2023-10-06T02:27:42.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.821+0000 7f94fe7fc700 1 --2- 172.21.15.31:0/343899763 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f94ec035900 0x7f94ec037db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:42.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.821+0000 7f94fe7fc700 1 -- 172.21.15.31:0/343899763 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6617+0+0 (secure 0 0 0) 0x7f94f8049f30 con 0x7f9508153c30 2023-10-06T02:27:42.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.821+0000 7f950e19b700 1 --2- 172.21.15.31:0/343899763 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f94ec035900 0x7f94ec037db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:42.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.822+0000 7f950e19b700 1 --2- 172.21.15.31:0/343899763 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f94ec035900 0x7f94ec037db0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f95080b2530 tx=0x7f95040091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:42.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:42.825+0000 7f94fe7fc700 1 -- 172.21.15.31:0/343899763 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f94f80199c0 con 0x7f9508153c30 2023-10-06T02:27:42.879 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:27:42.879+0000 7f54bc28a180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T02:27:42.980 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:27:42.980+0000 7f54bc28a180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T02:27:43.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.031+0000 7f950f19d700 1 -- 172.21.15.31:0/343899763 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f950804c3c0 con 0x7f9508153c30 2023-10-06T02:27:43.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.032+0000 7f94fe7fc700 1 -- 172.21.15.31:0/343899763 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) v1 ==== 74+0+17622 (secure 0 0 0) 0x7f94f8029090 con 0x7f9508153c30 2023-10-06T02:27:43.033 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:27:43.033 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":76,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:27:38.406866+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:27:20.257986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-10-06T02:27:21.518685+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/2878303473":"2023-10-07T02:27:38.406819+0000","172.21.15.31:0/2186321038":"2023-10-07T02:27:38.406819+0000","172.21.15.31:6832/85292":"2023-10-07T02:27:38.406819+0000","172.21.15.31:0/3308984790":"2023-10-07T02:27:38.406819+0000","172.21.15.31:0/3769130347":"2023-10-07T02:27:38.406819+0000","172.21.15.31:0/2627562379":"2023-10-07T02:27:38.406819+0000","172.21.15.31:0/2430615067":"2023-10-07T02:27:38.406819+0000","172.21.15.31:0/1519622771":"2023-10-07T02:27:38.406819+0000","172.21.15.31:0/3343749855":"2023-10-07T02:27:38.406819+0000","172.21.15.31:0/1049264351":"2023-10-07T02:27:38.406819+0000","172.21.15.31:0/2607073168":"2023-10-07T02:27:38.406819+0000","172.21.15.31:6833/85292":"2023-10-07T02:27:38.406819+0000","172.21.15.31:0/3265960029":"2023-10-07T02:27:38.406819+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:27:43.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.034+0000 7f950f19d700 1 -- 172.21.15.31:0/343899763 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f94ec035900 msgr2=0x7f94ec037db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:43.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.034+0000 7f950f19d700 1 --2- 172.21.15.31:0/343899763 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f94ec035900 0x7f94ec037db0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f95080b2530 tx=0x7f95040091a0 comp rx=0 tx=0).stop 2023-10-06T02:27:43.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.034+0000 7f950f19d700 1 -- 172.21.15.31:0/343899763 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9508153c30 msgr2=0x7f95081522b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:43.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.034+0000 7f950f19d700 1 --2- 172.21.15.31:0/343899763 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9508153c30 0x7f95081522b0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f950806eb00 tx=0x7f94f8017000 comp rx=0 tx=0).stop 2023-10-06T02:27:43.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.034+0000 7f950f19d700 1 -- 172.21.15.31:0/343899763 shutdown_connections 2023-10-06T02:27:43.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.034+0000 7f950f19d700 1 --2- 172.21.15.31:0/343899763 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f950815de50 0x7f95080c85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:43.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.034+0000 7f950f19d700 1 --2- 172.21.15.31:0/343899763 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f94ec035900 0x7f94ec037db0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:43.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.034+0000 7f950f19d700 1 --2- 172.21.15.31:0/343899763 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f95080cef40 0x7f950816abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:43.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.034+0000 7f950f19d700 1 --2- 172.21.15.31:0/343899763 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9508153c30 0x7f95081522b0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:43.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.034+0000 7f950f19d700 1 -- 172.21.15.31:0/343899763 >> 172.21.15.31:0/343899763 conn(0x7f95080703a0 msgr2=0x7f95081530d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:43.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.034+0000 7f950f19d700 1 -- 172.21.15.31:0/343899763 shutdown_connections 2023-10-06T02:27:43.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.035+0000 7f950f19d700 1 -- 172.21.15.31:0/343899763 wait complete. 2023-10-06T02:27:43.054 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T02:27:43.054 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:27:43.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.242+0000 7fca0bbff700 1 Processor -- start 2023-10-06T02:27:43.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.242+0000 7fca0bbff700 1 -- start start 2023-10-06T02:27:43.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.244+0000 7fca0bbff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fca04157600 0x7fca040b1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:43.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.244+0000 7fca0bbff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fca040ccba0 0x7fca040b1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:43.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.244+0000 7fca0bbff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca04158210 con 0x7fca041579d0 2023-10-06T02:27:43.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.244+0000 7fca0bbff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca040cb850 con 0x7fca040ccba0 2023-10-06T02:27:43.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.244+0000 7fca0bbff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca040cb290 con 0x7fca04157600 2023-10-06T02:27:43.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.244+0000 7fca0a19c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fca040ccba0 0x7fca040b1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:43.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.244+0000 7fca0a19c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fca040ccba0 0x7fca040b1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50942/0 (socket says 172.21.15.31:50942) 2023-10-06T02:27:43.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.244+0000 7fca0a19c700 1 -- 172.21.15.31:0/970809287 learned_addr learned my addr 172.21.15.31:0/970809287 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:43.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.244+0000 7fca0919a700 1 --1- 172.21.15.31:0/970809287 >> v1:172.21.15.31:6789/0 conn(0x7fca041579d0 0x7fca041586a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:37474/0 (socket says 172.21.15.31:37474) 2023-10-06T02:27:43.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.244+0000 7fca0999b700 1 --2- 172.21.15.31:0/970809287 >> v2:172.21.15.62:3300/0 conn(0x7fca04157600 0x7fca040b1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:43.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.245+0000 7fca0a19c700 1 -- 172.21.15.31:0/970809287 >> v2:172.21.15.62:3300/0 conn(0x7fca04157600 msgr2=0x7fca040b1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:43.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.245+0000 7fca0a19c700 1 --2- 172.21.15.31:0/970809287 >> v2:172.21.15.62:3300/0 conn(0x7fca04157600 0x7fca040b1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:43.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.245+0000 7fca0a19c700 1 -- 172.21.15.31:0/970809287 >> v1:172.21.15.31:6789/0 conn(0x7fca041579d0 legacy=0x7fca041586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:43.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.245+0000 7fca0a19c700 1 -- 172.21.15.31:0/970809287 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca040cb570 con 0x7fca040ccba0 2023-10-06T02:27:43.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.245+0000 7fca08999700 1 -- 172.21.15.31:0/970809287 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 685413068 0 0) 0x7fca04158210 con 0x7fca041579d0 2023-10-06T02:27:43.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.245+0000 7fca0999b700 1 --2- 172.21.15.31:0/970809287 >> v2:172.21.15.62:3300/0 conn(0x7fca04157600 0x7fca040b1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:27:43.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.245+0000 7fca0a19c700 1 --2- 172.21.15.31:0/970809287 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca040ccba0 0x7fca040b1950 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fca040650d0 tx=0x7fc9fc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea42a9a3d416f2f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:43.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.245+0000 7fca08999700 1 -- 172.21.15.31:0/970809287 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9fc00e070 con 0x7fca040ccba0 2023-10-06T02:27:43.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.245+0000 7fca08999700 1 -- 172.21.15.31:0/970809287 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9fc00ba50 con 0x7fca040ccba0 2023-10-06T02:27:43.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.245+0000 7fca08999700 1 -- 172.21.15.31:0/970809287 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9fc0092e0 con 0x7fca040ccba0 2023-10-06T02:27:43.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.246+0000 7fca0bbff700 1 -- 172.21.15.31:0/970809287 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca040ccba0 msgr2=0x7fca040b1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:43.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.246+0000 7fca0bbff700 1 --2- 172.21.15.31:0/970809287 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca040ccba0 0x7fca040b1950 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fca040650d0 tx=0x7fc9fc009840 comp rx=0 tx=0).stop 2023-10-06T02:27:43.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.246+0000 7fca0bbff700 1 -- 172.21.15.31:0/970809287 shutdown_connections 2023-10-06T02:27:43.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.246+0000 7fca0bbff700 1 --2- 172.21.15.31:0/970809287 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca040ccba0 0x7fca040b1950 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:43.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.246+0000 7fca0bbff700 1 --2- 172.21.15.31:0/970809287 >> v2:172.21.15.62:3300/0 conn(0x7fca04157600 0x7fca040b1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:43.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.246+0000 7fca0bbff700 1 -- 172.21.15.31:0/970809287 >> 172.21.15.31:0/970809287 conn(0x7fca0406d170 msgr2=0x7fca040cc300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:43.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.246+0000 7fca0bbff700 1 -- 172.21.15.31:0/970809287 shutdown_connections 2023-10-06T02:27:43.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.246+0000 7fca0bbff700 1 -- 172.21.15.31:0/970809287 wait complete. 2023-10-06T02:27:43.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.247+0000 7fca0bbff700 1 Processor -- start 2023-10-06T02:27:43.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.248+0000 7fca0bbff700 1 -- start start 2023-10-06T02:27:43.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.248+0000 7fca0bbff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca040ccba0 0x7fca040b0510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:43.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.248+0000 7fca0bbff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca04157600 0x7fca040c5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:43.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.249+0000 7fca0bbff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca041579d0 0x7fca04147de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:43.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.249+0000 7fca0bbff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca040c7240 con 0x7fca040ccba0 2023-10-06T02:27:43.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.249+0000 7fca0bbff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca040b0a50 con 0x7fca04157600 2023-10-06T02:27:43.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.249+0000 7fca0bbff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca040c63b0 con 0x7fca041579d0 2023-10-06T02:27:43.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.249+0000 7fca0999b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca040ccba0 0x7fca040b0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:43.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.249+0000 7fca0999b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca040ccba0 0x7fca040b0510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50958/0 (socket says 172.21.15.31:50958) 2023-10-06T02:27:43.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.249+0000 7fca0999b700 1 -- 172.21.15.31:0/3398081141 learned_addr learned my addr 172.21.15.31:0/3398081141 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:43.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.249+0000 7fca0919a700 1 --2- 172.21.15.31:0/3398081141 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca04157600 0x7fca040c5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:43.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.249+0000 7fca0999b700 1 -- 172.21.15.31:0/3398081141 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca041579d0 msgr2=0x7fca04147de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:43.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.249+0000 7fca0999b700 1 --2- 172.21.15.31:0/3398081141 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca041579d0 0x7fca04147de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:43.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.249+0000 7fca0999b700 1 -- 172.21.15.31:0/3398081141 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca04157600 msgr2=0x7fca040c5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:43.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.249+0000 7fca0999b700 1 --2- 172.21.15.31:0/3398081141 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca04157600 0x7fca040c5e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:43.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.249+0000 7fca0999b700 1 -- 172.21.15.31:0/3398081141 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9fc009ae0 con 0x7fca040ccba0 2023-10-06T02:27:43.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.250+0000 7fca0999b700 1 --2- 172.21.15.31:0/3398081141 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca040ccba0 0x7fca040b0510 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fca0000d860 tx=0x7fca00008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:43.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.250+0000 7fc9f9ffb700 1 -- 172.21.15.31:0/3398081141 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca000125f0 con 0x7fca040ccba0 2023-10-06T02:27:43.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.250+0000 7fc9f9ffb700 1 -- 172.21.15.31:0/3398081141 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca00012750 con 0x7fca040ccba0 2023-10-06T02:27:43.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.250+0000 7fc9f9ffb700 1 -- 172.21.15.31:0/3398081141 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca00012ad0 con 0x7fca040ccba0 2023-10-06T02:27:43.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.250+0000 7fca0bbff700 1 -- 172.21.15.31:0/3398081141 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca04148320 con 0x7fca040ccba0 2023-10-06T02:27:43.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.250+0000 7fca0bbff700 1 -- 172.21.15.31:0/3398081141 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca040c68a0 con 0x7fca040ccba0 2023-10-06T02:27:43.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.251+0000 7fca0bbff700 1 -- 172.21.15.31:0/3398081141 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca040b2070 con 0x7fca040ccba0 2023-10-06T02:27:43.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.251+0000 7fc9f9ffb700 1 -- 172.21.15.31:0/3398081141 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 42643+0+0 (secure 0 0 0) 0x7fca00014430 con 0x7fca040ccba0 2023-10-06T02:27:43.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.252+0000 7fc9f9ffb700 1 --2- 172.21.15.31:0/3398081141 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc9e8035900 0x7fc9e8037db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:43.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.252+0000 7fc9f9ffb700 1 -- 172.21.15.31:0/3398081141 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6617+0+0 (secure 0 0 0) 0x7fca000488e0 con 0x7fca040ccba0 2023-10-06T02:27:43.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.252+0000 7fca0919a700 1 --2- 172.21.15.31:0/3398081141 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc9e8035900 0x7fc9e8037db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:43.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.253+0000 7fca0919a700 1 --2- 172.21.15.31:0/3398081141 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc9e8035900 0x7fc9e8037db0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc9f4007fd0 tx=0x7fc9f4009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:43.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.256+0000 7fc9f9ffb700 1 -- 172.21.15.31:0/3398081141 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fca0001b6c0 con 0x7fca040ccba0 2023-10-06T02:27:43.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.505+0000 7fca0bbff700 1 -- 172.21.15.31:0/3398081141 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fca0404c3c0 con 0x7fca040ccba0 2023-10-06T02:27:43.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.506+0000 7fc9f9ffb700 1 -- 172.21.15.31:0/3398081141 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4596 (secure 0 0 0) 0x7fca0001b300 con 0x7fca040ccba0 2023-10-06T02:27:43.507 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:27:43.507 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5474,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/2085637096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2085637096},{"type":"v1","addr":"172.21.15.62:6834","nonce":2085637096}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":5992,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/573080449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":573080449},{"type":"v1","addr":"172.21.15.31:6836","nonce":573080449}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":6007,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2625778079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2625778079},{"type":"v1","addr":"172.21.15.62:6835","nonce":2625778079}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:27:22.533533+0000","modified":"2023-10-06T02:27:26.062249+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6282":{"gid":6282,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6837/3003674295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":3003674295},{"type":"v1","addr":"172.21.15.31:6837","nonce":3003674295}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T02:27:43.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.508+0000 7fca0bbff700 1 -- 172.21.15.31:0/3398081141 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc9e8035900 msgr2=0x7fc9e8037db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:43.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.508+0000 7fca0bbff700 1 --2- 172.21.15.31:0/3398081141 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc9e8035900 0x7fc9e8037db0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc9f4007fd0 tx=0x7fc9f4009040 comp rx=0 tx=0).stop 2023-10-06T02:27:43.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.508+0000 7fca0bbff700 1 -- 172.21.15.31:0/3398081141 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca040ccba0 msgr2=0x7fca040b0510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:43.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.508+0000 7fca0bbff700 1 --2- 172.21.15.31:0/3398081141 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca040ccba0 0x7fca040b0510 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fca0000d860 tx=0x7fca00008040 comp rx=0 tx=0).stop 2023-10-06T02:27:43.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.509+0000 7fca0bbff700 1 -- 172.21.15.31:0/3398081141 shutdown_connections 2023-10-06T02:27:43.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.509+0000 7fca0bbff700 1 --2- 172.21.15.31:0/3398081141 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc9e8035900 0x7fc9e8037db0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:43.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.509+0000 7fca0bbff700 1 --2- 172.21.15.31:0/3398081141 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca041579d0 0x7fca04147de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:43.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.509+0000 7fca0bbff700 1 --2- 172.21.15.31:0/3398081141 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca040ccba0 0x7fca040b0510 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:43.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.509+0000 7fca0bbff700 1 --2- 172.21.15.31:0/3398081141 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca04157600 0x7fca040c5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:43.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.509+0000 7fca0bbff700 1 -- 172.21.15.31:0/3398081141 >> 172.21.15.31:0/3398081141 conn(0x7fca0406d170 msgr2=0x7fca04155e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:43.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.509+0000 7fca0bbff700 1 -- 172.21.15.31:0/3398081141 shutdown_connections 2023-10-06T02:27:43.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.509+0000 7fca0bbff700 1 -- 172.21.15.31:0/3398081141 wait complete. 2023-10-06T02:27:43.511 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 47 2023-10-06T02:27:43.527 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:27:43.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.716+0000 7ff04e99d700 1 Processor -- start 2023-10-06T02:27:43.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.717+0000 7ff04e99d700 1 -- start start 2023-10-06T02:27:43.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.718+0000 7ff04e99d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff04815de50 0x7ff0480cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:43.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.718+0000 7ff04e99d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0480cf3c0 con 0x7ff0480cef40 2023-10-06T02:27:43.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.718+0000 7ff04e99d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff04815afd0 con 0x7ff048153c30 2023-10-06T02:27:43.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.718+0000 7ff04e99d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff04815b2b0 con 0x7ff04815de50 2023-10-06T02:27:43.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.718+0000 7ff04e19c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7ff0480cef40 0x7ff04815c070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:37490/0 (socket says 172.21.15.31:37490) 2023-10-06T02:27:43.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.718+0000 7ff04e19c700 1 -- 172.21.15.31:0/4018928972 learned_addr learned my addr 172.21.15.31:0/4018928972 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:27:43.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.718+0000 7ff04d19a700 1 --2- 172.21.15.31:0/4018928972 >> v2:172.21.15.62:3300/0 conn(0x7ff04815de50 0x7ff0480cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:43.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.719+0000 7ff04c999700 1 -- 172.21.15.31:0/4018928972 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1284366951 0 0) 0x7ff0480cf3c0 con 0x7ff0480cef40 2023-10-06T02:27:43.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.719+0000 7ff04c999700 1 -- 172.21.15.31:0/4018928972 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff034003570 con 0x7ff0480cef40 2023-10-06T02:27:43.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.719+0000 7ff04c999700 1 -- 172.21.15.31:0/4018928972 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2396300663 0 0) 0x7ff04815afd0 con 0x7ff048153c30 2023-10-06T02:27:43.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.719+0000 7ff04c999700 1 -- 172.21.15.31:0/4018928972 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0480cf3c0 con 0x7ff048153c30 2023-10-06T02:27:43.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.719+0000 7ff04d19a700 1 -- 172.21.15.31:0/4018928972 >> v1:172.21.15.62:6789/0 conn(0x7ff048153c30 legacy=0x7ff04815a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:43.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.719+0000 7ff04d19a700 1 -- 172.21.15.31:0/4018928972 >> v1:172.21.15.31:6789/0 conn(0x7ff0480cef40 legacy=0x7ff04815c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:43.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.719+0000 7ff04d19a700 1 -- 172.21.15.31:0/4018928972 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff04815b590 con 0x7ff04815de50 2023-10-06T02:27:43.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.719+0000 7ff04d19a700 1 --2- 172.21.15.31:0/4018928972 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff04815de50 0x7ff0480cea00 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff0380098e0 tx=0x7ff0380091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74e9c6c4ec4c041d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:43.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.720+0000 7ff04c999700 1 -- 172.21.15.31:0/4018928972 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff038011630 con 0x7ff04815de50 2023-10-06T02:27:43.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.720+0000 7ff04c999700 1 -- 172.21.15.31:0/4018928972 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff038011790 con 0x7ff04815de50 2023-10-06T02:27:43.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.720+0000 7ff04c999700 1 -- 172.21.15.31:0/4018928972 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff038011b30 con 0x7ff04815de50 2023-10-06T02:27:43.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.720+0000 7ff04e99d700 1 -- 172.21.15.31:0/4018928972 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff04815de50 msgr2=0x7ff0480cea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:43.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.720+0000 7ff04e99d700 1 --2- 172.21.15.31:0/4018928972 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff04815de50 0x7ff0480cea00 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff0380098e0 tx=0x7ff0380091a0 comp rx=0 tx=0).stop 2023-10-06T02:27:43.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.720+0000 7ff04e99d700 1 -- 172.21.15.31:0/4018928972 shutdown_connections 2023-10-06T02:27:43.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.720+0000 7ff04e99d700 1 --2- 172.21.15.31:0/4018928972 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff04815de50 0x7ff0480cea00 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:43.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.720+0000 7ff04e99d700 1 -- 172.21.15.31:0/4018928972 >> 172.21.15.31:0/4018928972 conn(0x7ff0480703a0 msgr2=0x7ff0480707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:43.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.720+0000 7ff04e99d700 1 -- 172.21.15.31:0/4018928972 shutdown_connections 2023-10-06T02:27:43.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.720+0000 7ff04e99d700 1 -- 172.21.15.31:0/4018928972 wait complete. 2023-10-06T02:27:43.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.722+0000 7ff04e99d700 1 Processor -- start 2023-10-06T02:27:43.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.722+0000 7ff04e99d700 1 -- start start 2023-10-06T02:27:43.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.723+0000 7ff04e99d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0480cef40 0x7ff0481522a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:43.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.723+0000 7ff04e99d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff048153c30 0x7ff0480c9e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:43.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.723+0000 7ff04e99d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff04815de50 0x7ff0481696a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:43.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.723+0000 7ff04e99d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0480c9ba0 con 0x7ff04815de50 2023-10-06T02:27:43.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.723+0000 7ff04e99d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0480c93c0 con 0x7ff0480cef40 2023-10-06T02:27:43.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.723+0000 7ff04e99d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff04816ceb0 con 0x7ff048153c30 2023-10-06T02:27:43.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.723+0000 7ff04e19c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff04815de50 0x7ff0481696a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:43.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.723+0000 7ff04e19c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff04815de50 0x7ff0481696a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50974/0 (socket says 172.21.15.31:50974) 2023-10-06T02:27:43.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.723+0000 7ff04e19c700 1 -- 172.21.15.31:0/1550850629 learned_addr learned my addr 172.21.15.31:0/1550850629 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:43.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.723+0000 7ff04d99b700 1 --2- 172.21.15.31:0/1550850629 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0480cef40 0x7ff0481522a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:43.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.723+0000 7ff04d19a700 1 --2- 172.21.15.31:0/1550850629 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff048153c30 0x7ff0480c9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:43.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.724+0000 7ff04e19c700 1 -- 172.21.15.31:0/1550850629 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff048153c30 msgr2=0x7ff0480c9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:43.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.724+0000 7ff04e19c700 1 --2- 172.21.15.31:0/1550850629 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff048153c30 0x7ff0480c9e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:43.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.724+0000 7ff04e19c700 1 -- 172.21.15.31:0/1550850629 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0480cef40 msgr2=0x7ff0481522a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:43.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.724+0000 7ff04e19c700 1 --2- 172.21.15.31:0/1550850629 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0480cef40 0x7ff0481522a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:43.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.724+0000 7ff04e19c700 1 -- 172.21.15.31:0/1550850629 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff038009420 con 0x7ff04815de50 2023-10-06T02:27:43.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.724+0000 7ff04e19c700 1 --2- 172.21.15.31:0/1550850629 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff04815de50 0x7ff0481696a0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7ff04000cc20 tx=0x7ff040001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:43.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.724+0000 7ff03dffb700 1 -- 172.21.15.31:0/1550850629 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04000a960 con 0x7ff04815de50 2023-10-06T02:27:43.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.724+0000 7ff03dffb700 1 -- 172.21.15.31:0/1550850629 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff04000aac0 con 0x7ff04815de50 2023-10-06T02:27:43.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.724+0000 7ff04e99d700 1 -- 172.21.15.31:0/1550850629 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0480ca3b0 con 0x7ff04815de50 2023-10-06T02:27:43.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.724+0000 7ff03dffb700 1 -- 172.21.15.31:0/1550850629 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04000ae80 con 0x7ff04815de50 2023-10-06T02:27:43.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.724+0000 7ff04e99d700 1 -- 172.21.15.31:0/1550850629 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0480c8e00 con 0x7ff04815de50 2023-10-06T02:27:43.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.725+0000 7ff04e99d700 1 -- 172.21.15.31:0/1550850629 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff048159cd0 con 0x7ff04815de50 2023-10-06T02:27:43.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.726+0000 7ff03dffb700 1 -- 172.21.15.31:0/1550850629 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 42643+0+0 (secure 0 0 0) 0x7ff040007300 con 0x7ff04815de50 2023-10-06T02:27:43.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.726+0000 7ff03dffb700 1 --2- 172.21.15.31:0/1550850629 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff02c035900 0x7ff02c037db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:43.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.726+0000 7ff03dffb700 1 -- 172.21.15.31:0/1550850629 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6617+0+0 (secure 0 0 0) 0x7ff040048a70 con 0x7ff04815de50 2023-10-06T02:27:43.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.727+0000 7ff04d99b700 1 --2- 172.21.15.31:0/1550850629 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff02c035900 0x7ff02c037db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:43.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.727+0000 7ff04d99b700 1 --2- 172.21.15.31:0/1550850629 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff02c035900 0x7ff02c037db0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff044003870 tx=0x7ff044009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:43.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.731+0000 7ff03dffb700 1 -- 172.21.15.31:0/1550850629 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff04001b400 con 0x7ff04815de50 2023-10-06T02:27:43.738 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:27:43.738+0000 7f54bc28a180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T02:27:43.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.977+0000 7ff04e99d700 1 -- 172.21.15.31:0/1550850629 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff04804c3c0 con 0x7ff04815de50 2023-10-06T02:27:43.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.978+0000 7ff03dffb700 1 -- 172.21.15.31:0/1550850629 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4596 (secure 0 0 0) 0x7ff040010300 con 0x7ff04815de50 2023-10-06T02:27:43.978 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:27:43.978 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5474,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/2085637096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2085637096},{"type":"v1","addr":"172.21.15.62:6834","nonce":2085637096}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":5992,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/573080449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":573080449},{"type":"v1","addr":"172.21.15.31:6836","nonce":573080449}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":6007,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2625778079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2625778079},{"type":"v1","addr":"172.21.15.62:6835","nonce":2625778079}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:27:22.533533+0000","modified":"2023-10-06T02:27:26.062249+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6282":{"gid":6282,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6837/3003674295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":3003674295},{"type":"v1","addr":"172.21.15.31:6837","nonce":3003674295}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T02:27:43.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.980+0000 7ff04e99d700 1 -- 172.21.15.31:0/1550850629 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff02c035900 msgr2=0x7ff02c037db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:43.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.980+0000 7ff04e99d700 1 --2- 172.21.15.31:0/1550850629 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff02c035900 0x7ff02c037db0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff044003870 tx=0x7ff044009040 comp rx=0 tx=0).stop 2023-10-06T02:27:43.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.980+0000 7ff04e99d700 1 -- 172.21.15.31:0/1550850629 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff04815de50 msgr2=0x7ff0481696a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:43.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.980+0000 7ff04e99d700 1 --2- 172.21.15.31:0/1550850629 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff04815de50 0x7ff0481696a0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7ff04000cc20 tx=0x7ff040001a40 comp rx=0 tx=0).stop 2023-10-06T02:27:43.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.980+0000 7ff04e99d700 1 -- 172.21.15.31:0/1550850629 shutdown_connections 2023-10-06T02:27:43.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.980+0000 7ff04e99d700 1 --2- 172.21.15.31:0/1550850629 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff04815de50 0x7ff0481696a0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:43.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.980+0000 7ff04e99d700 1 --2- 172.21.15.31:0/1550850629 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0480cef40 0x7ff0481522a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:43.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.980+0000 7ff04e99d700 1 --2- 172.21.15.31:0/1550850629 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff02c035900 0x7ff02c037db0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:43.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.980+0000 7ff04e99d700 1 --2- 172.21.15.31:0/1550850629 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff048153c30 0x7ff0480c9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:43.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.980+0000 7ff04e99d700 1 -- 172.21.15.31:0/1550850629 >> 172.21.15.31:0/1550850629 conn(0x7ff0480703a0 msgr2=0x7ff0480c60b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:43.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.980+0000 7ff04e99d700 1 -- 172.21.15.31:0/1550850629 shutdown_connections 2023-10-06T02:27:43.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:43.980+0000 7ff04e99d700 1 -- 172.21.15.31:0/1550850629 wait complete. 2023-10-06T02:27:43.983 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 47 2023-10-06T02:27:43.999 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:27:44.036 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:27:44.036+0000 7f54bc28a180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T02:27:44.136 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:27:44.135+0000 7f54bc28a180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T02:27:44.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.193+0000 7fa18a99d700 1 Processor -- start 2023-10-06T02:27:44.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.193+0000 7fa18a99d700 1 -- start start 2023-10-06T02:27:44.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.194+0000 7fa18a99d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa18415de50 0x7fa1840cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:44.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.194+0000 7fa18a99d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa1840cef40 0x7fa18415c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:44.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.194+0000 7fa18a99d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1840cf3c0 con 0x7fa184153c30 2023-10-06T02:27:44.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.194+0000 7fa18a99d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa18415afd0 con 0x7fa18415de50 2023-10-06T02:27:44.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.194+0000 7fa18a99d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa18415b2b0 con 0x7fa1840cef40 2023-10-06T02:27:44.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.195+0000 7fa18919a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa18415de50 0x7fa1840cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:44.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.195+0000 7fa18919a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa18415de50 0x7fa1840cea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50980/0 (socket says 172.21.15.31:50980) 2023-10-06T02:27:44.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.195+0000 7fa18999b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fa184153c30 0x7fa18415a5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:37492/0 (socket says 172.21.15.31:37492) 2023-10-06T02:27:44.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.195+0000 7fa18919a700 1 -- 172.21.15.31:0/4021376042 learned_addr learned my addr 172.21.15.31:0/4021376042 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:44.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.195+0000 7fa18a19c700 1 --2- 172.21.15.31:0/4021376042 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa1840cef40 0x7fa18415c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:44.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.195+0000 7fa18919a700 1 -- 172.21.15.31:0/4021376042 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa1840cef40 msgr2=0x7fa18415c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:44.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.195+0000 7fa18919a700 1 --2- 172.21.15.31:0/4021376042 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa1840cef40 0x7fa18415c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:44.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.195+0000 7fa18919a700 1 -- 172.21.15.31:0/4021376042 >> v1:172.21.15.31:6789/0 conn(0x7fa184153c30 legacy=0x7fa18415a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:44.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.195+0000 7fa18919a700 1 -- 172.21.15.31:0/4021376042 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa18415b590 con 0x7fa18415de50 2023-10-06T02:27:44.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.195+0000 7fa188999700 1 -- 172.21.15.31:0/4021376042 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4087678975 0 0) 0x7fa1840cf3c0 con 0x7fa184153c30 2023-10-06T02:27:44.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.195+0000 7fa18919a700 1 --2- 172.21.15.31:0/4021376042 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa18415de50 0x7fa1840cea00 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fa1740099b0 tx=0x7fa174009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60eb580a72163c8c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:44.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.196+0000 7fa188999700 1 -- 172.21.15.31:0/4021376042 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17400b860 con 0x7fa18415de50 2023-10-06T02:27:44.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.196+0000 7fa188999700 1 -- 172.21.15.31:0/4021376042 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa17400b9c0 con 0x7fa18415de50 2023-10-06T02:27:44.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.196+0000 7fa188999700 1 -- 172.21.15.31:0/4021376042 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17400bd60 con 0x7fa18415de50 2023-10-06T02:27:44.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.196+0000 7fa18a99d700 1 -- 172.21.15.31:0/4021376042 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa18415de50 msgr2=0x7fa1840cea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:44.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.196+0000 7fa18a99d700 1 --2- 172.21.15.31:0/4021376042 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa18415de50 0x7fa1840cea00 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fa1740099b0 tx=0x7fa174009250 comp rx=0 tx=0).stop 2023-10-06T02:27:44.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.197+0000 7fa18a99d700 1 -- 172.21.15.31:0/4021376042 shutdown_connections 2023-10-06T02:27:44.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.197+0000 7fa18a99d700 1 --2- 172.21.15.31:0/4021376042 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa1840cef40 0x7fa18415c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:44.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.197+0000 7fa18a99d700 1 --2- 172.21.15.31:0/4021376042 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa18415de50 0x7fa1840cea00 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:44.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.197+0000 7fa18a99d700 1 -- 172.21.15.31:0/4021376042 >> 172.21.15.31:0/4021376042 conn(0x7fa1840703a0 msgr2=0x7fa1840707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:44.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.197+0000 7fa18a99d700 1 -- 172.21.15.31:0/4021376042 shutdown_connections 2023-10-06T02:27:44.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.197+0000 7fa18a99d700 1 -- 172.21.15.31:0/4021376042 wait complete. 2023-10-06T02:27:44.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.198+0000 7fa18a99d700 1 Processor -- start 2023-10-06T02:27:44.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.198+0000 7fa18a99d700 1 -- start start 2023-10-06T02:27:44.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.199+0000 7fa18a99d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1840cef40 0x7fa1840b32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:44.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.199+0000 7fa18a99d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa184153c30 0x7fa1840c9590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:44.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.199+0000 7fa18a99d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa18415de50 0x7fa18414c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:44.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.199+0000 7fa18a99d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1840ca4a0 con 0x7fa1840cef40 2023-10-06T02:27:44.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.199+0000 7fa18a99d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1840cb340 con 0x7fa184153c30 2023-10-06T02:27:44.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.199+0000 7fa18a99d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1840c9c00 con 0x7fa18415de50 2023-10-06T02:27:44.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.199+0000 7fa18999b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1840cef40 0x7fa1840b32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:44.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.199+0000 7fa18999b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1840cef40 0x7fa1840b32d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50988/0 (socket says 172.21.15.31:50988) 2023-10-06T02:27:44.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.199+0000 7fa18999b700 1 -- 172.21.15.31:0/3422144519 learned_addr learned my addr 172.21.15.31:0/3422144519 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:44.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.200+0000 7fa18a19c700 1 --2- 172.21.15.31:0/3422144519 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa18415de50 0x7fa18414c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:44.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.200+0000 7fa18919a700 1 --2- 172.21.15.31:0/3422144519 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa184153c30 0x7fa1840c9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:44.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.200+0000 7fa18999b700 1 -- 172.21.15.31:0/3422144519 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa18415de50 msgr2=0x7fa18414c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:44.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.200+0000 7fa18999b700 1 --2- 172.21.15.31:0/3422144519 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa18415de50 0x7fa18414c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:44.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.200+0000 7fa18999b700 1 -- 172.21.15.31:0/3422144519 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa184153c30 msgr2=0x7fa1840c9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:44.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.200+0000 7fa18999b700 1 --2- 172.21.15.31:0/3422144519 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa184153c30 0x7fa1840c9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:44.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.200+0000 7fa18999b700 1 -- 172.21.15.31:0/3422144519 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1740094f0 con 0x7fa1840cef40 2023-10-06T02:27:44.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.200+0000 7fa18a19c700 1 --2- 172.21.15.31:0/3422144519 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa18415de50 0x7fa18414c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:27:44.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.200+0000 7fa18999b700 1 --2- 172.21.15.31:0/3422144519 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1840cef40 0x7fa1840b32d0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fa1800011e0 tx=0x7fa180007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:44.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.200+0000 7fa179ffb700 1 -- 172.21.15.31:0/3422144519 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa180007300 con 0x7fa1840cef40 2023-10-06T02:27:44.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.200+0000 7fa179ffb700 1 -- 172.21.15.31:0/3422144519 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa180007460 con 0x7fa1840cef40 2023-10-06T02:27:44.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.200+0000 7fa179ffb700 1 -- 172.21.15.31:0/3422144519 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa180004830 con 0x7fa1840cef40 2023-10-06T02:27:44.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.200+0000 7fa18a99d700 1 -- 172.21.15.31:0/3422144519 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1840c9ee0 con 0x7fa1840cef40 2023-10-06T02:27:44.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.201+0000 7fa18a99d700 1 -- 172.21.15.31:0/3422144519 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa18414c7f0 con 0x7fa1840cef40 2023-10-06T02:27:44.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.202+0000 7fa18a99d700 1 -- 172.21.15.31:0/3422144519 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa18404c3c0 con 0x7fa1840cef40 2023-10-06T02:27:44.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.202+0000 7fa179ffb700 1 -- 172.21.15.31:0/3422144519 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 42643+0+0 (secure 0 0 0) 0x7fa180004be0 con 0x7fa1840cef40 2023-10-06T02:27:44.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.202+0000 7fa179ffb700 1 --2- 172.21.15.31:0/3422144519 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa168035900 0x7fa168037db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:44.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.202+0000 7fa179ffb700 1 -- 172.21.15.31:0/3422144519 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6617+0+0 (secure 0 0 0) 0x7fa180047b70 con 0x7fa1840cef40 2023-10-06T02:27:44.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.203+0000 7fa18919a700 1 --2- 172.21.15.31:0/3422144519 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa168035900 0x7fa168037db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:44.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.204+0000 7fa18919a700 1 --2- 172.21.15.31:0/3422144519 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa168035900 0x7fa168037db0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa1740099b0 tx=0x7fa174017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:44.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.207+0000 7fa179ffb700 1 -- 172.21.15.31:0/3422144519 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa18001cb10 con 0x7fa1840cef40 2023-10-06T02:27:44.300 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:27:44.300+0000 7f54bc28a180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T02:27:44.409 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:27:44.409+0000 7f54bc28a180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T02:27:44.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.414+0000 7fa18a99d700 1 -- 172.21.15.31:0/3422144519 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa18415ca20 con 0x7fa1840cef40 2023-10-06T02:27:44.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.415+0000 7fa179ffb700 1 -- 172.21.15.31:0/3422144519 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) v1 ==== 74+0+17622 (secure 0 0 0) 0x7fa180018070 con 0x7fa1840cef40 2023-10-06T02:27:44.416 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:27:44.416 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":76,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:27:38.406866+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:27:20.257986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-10-06T02:27:21.518685+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/2878303473":"2023-10-07T02:27:38.406819+0000","172.21.15.31:0/2186321038":"2023-10-07T02:27:38.406819+0000","172.21.15.31:6832/85292":"2023-10-07T02:27:38.406819+0000","172.21.15.31:0/3308984790":"2023-10-07T02:27:38.406819+0000","172.21.15.31:0/3769130347":"2023-10-07T02:27:38.406819+0000","172.21.15.31:0/2627562379":"2023-10-07T02:27:38.406819+0000","172.21.15.31:0/2430615067":"2023-10-07T02:27:38.406819+0000","172.21.15.31:0/1519622771":"2023-10-07T02:27:38.406819+0000","172.21.15.31:0/3343749855":"2023-10-07T02:27:38.406819+0000","172.21.15.31:0/1049264351":"2023-10-07T02:27:38.406819+0000","172.21.15.31:0/2607073168":"2023-10-07T02:27:38.406819+0000","172.21.15.31:6833/85292":"2023-10-07T02:27:38.406819+0000","172.21.15.31:0/3265960029":"2023-10-07T02:27:38.406819+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:27:44.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.417+0000 7fa18a99d700 1 -- 172.21.15.31:0/3422144519 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa168035900 msgr2=0x7fa168037db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:44.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.417+0000 7fa18a99d700 1 --2- 172.21.15.31:0/3422144519 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa168035900 0x7fa168037db0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa1740099b0 tx=0x7fa174017000 comp rx=0 tx=0).stop 2023-10-06T02:27:44.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.417+0000 7fa18a99d700 1 -- 172.21.15.31:0/3422144519 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1840cef40 msgr2=0x7fa1840b32d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:44.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.417+0000 7fa18a99d700 1 --2- 172.21.15.31:0/3422144519 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1840cef40 0x7fa1840b32d0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fa1800011e0 tx=0x7fa180007e10 comp rx=0 tx=0).stop 2023-10-06T02:27:44.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.418+0000 7fa18a99d700 1 -- 172.21.15.31:0/3422144519 shutdown_connections 2023-10-06T02:27:44.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.418+0000 7fa18a99d700 1 --2- 172.21.15.31:0/3422144519 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa168035900 0x7fa168037db0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:44.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.418+0000 7fa18a99d700 1 --2- 172.21.15.31:0/3422144519 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa18415de50 0x7fa18414c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:44.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.418+0000 7fa18a99d700 1 --2- 172.21.15.31:0/3422144519 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1840cef40 0x7fa1840b32d0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:44.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.418+0000 7fa18a99d700 1 --2- 172.21.15.31:0/3422144519 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa184153c30 0x7fa1840c9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:44.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.418+0000 7fa18a99d700 1 -- 172.21.15.31:0/3422144519 >> 172.21.15.31:0/3422144519 conn(0x7fa1840703a0 msgr2=0x7fa18415bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:44.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.418+0000 7fa18a99d700 1 -- 172.21.15.31:0/3422144519 shutdown_connections 2023-10-06T02:27:44.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.418+0000 7fa18a99d700 1 -- 172.21.15.31:0/3422144519 wait complete. 2023-10-06T02:27:44.436 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T02:27:44.611 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:27:44.611+0000 7f54bc28a180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T02:27:44.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.667+0000 7fad61f9d700 1 Processor -- start 2023-10-06T02:27:44.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.667+0000 7fad61f9d700 1 -- start start 2023-10-06T02:27:44.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.668+0000 7fad61f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fad5c15dfd0 0x7fad5c0aedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:44.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.668+0000 7fad61f9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fad5c159920 0x7fad5c0ad090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:44.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.668+0000 7fad61f9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad5c0acc00 con 0x7fad5c159550 2023-10-06T02:27:44.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.668+0000 7fad61f9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad5c0abff0 con 0x7fad5c159920 2023-10-06T02:27:44.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.668+0000 7fad61f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad5c0ac2d0 con 0x7fad5c15dfd0 2023-10-06T02:27:44.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.669+0000 7fad5bfff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fad5c15dfd0 0x7fad5c0aedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:44.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.669+0000 7fad6179c700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fad5c159920 0x7fad5c0ad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:44.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.669+0000 7fad6179c700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fad5c159920 0x7fad5c0ad090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:58680/0 (socket says 172.21.15.31:58680) 2023-10-06T02:27:44.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.669+0000 7fad6179c700 1 -- 172.21.15.31:0/3149526985 learned_addr learned my addr 172.21.15.31:0/3149526985 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:44.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.669+0000 7fad60f9b700 1 --1- 172.21.15.31:0/3149526985 >> v1:172.21.15.62:6789/0 conn(0x7fad5c159550 0x7fad5c0ceac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:40636/0 (socket says 172.21.15.31:40636) 2023-10-06T02:27:44.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.669+0000 7fad6179c700 1 -- 172.21.15.31:0/3149526985 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fad5c15dfd0 msgr2=0x7fad5c0aedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:44.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.669+0000 7fad6179c700 1 --2- 172.21.15.31:0/3149526985 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fad5c15dfd0 0x7fad5c0aedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:44.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.669+0000 7fad6179c700 1 -- 172.21.15.31:0/3149526985 >> v1:172.21.15.62:6789/0 conn(0x7fad5c159550 legacy=0x7fad5c0ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:44.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.669+0000 7fad6179c700 1 -- 172.21.15.31:0/3149526985 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad5c0ac5b0 con 0x7fad5c159920 2023-10-06T02:27:44.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.669+0000 7fad5b7fe700 1 -- 172.21.15.31:0/3149526985 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2837820944 0 0) 0x7fad5c0acc00 con 0x7fad5c159550 2023-10-06T02:27:44.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.669+0000 7fad5bfff700 1 --2- 172.21.15.31:0/3149526985 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fad5c15dfd0 0x7fad5c0aedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:27:44.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.670+0000 7fad6179c700 1 --2- 172.21.15.31:0/3149526985 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad5c159920 0x7fad5c0ad090 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fad54009f20 tx=0x7fad54009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ee2a1e8865d7a19 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:44.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.670+0000 7fad5b7fe700 1 -- 172.21.15.31:0/3149526985 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad5400e070 con 0x7fad5c159920 2023-10-06T02:27:44.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.670+0000 7fad5b7fe700 1 -- 172.21.15.31:0/3149526985 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad5400b9d0 con 0x7fad5c159920 2023-10-06T02:27:44.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.670+0000 7fad5b7fe700 1 -- 172.21.15.31:0/3149526985 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad540092e0 con 0x7fad5c159920 2023-10-06T02:27:44.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.671+0000 7fad61f9d700 1 -- 172.21.15.31:0/3149526985 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad5c159920 msgr2=0x7fad5c0ad090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:44.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.671+0000 7fad61f9d700 1 --2- 172.21.15.31:0/3149526985 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad5c159920 0x7fad5c0ad090 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fad54009f20 tx=0x7fad54009840 comp rx=0 tx=0).stop 2023-10-06T02:27:44.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.671+0000 7fad61f9d700 1 -- 172.21.15.31:0/3149526985 shutdown_connections 2023-10-06T02:27:44.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.671+0000 7fad61f9d700 1 --2- 172.21.15.31:0/3149526985 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad5c159920 0x7fad5c0ad090 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:44.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.671+0000 7fad61f9d700 1 --2- 172.21.15.31:0/3149526985 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fad5c15dfd0 0x7fad5c0aedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:44.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.671+0000 7fad61f9d700 1 -- 172.21.15.31:0/3149526985 >> 172.21.15.31:0/3149526985 conn(0x7fad5c06ce80 msgr2=0x7fad5c0ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:44.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.671+0000 7fad61f9d700 1 -- 172.21.15.31:0/3149526985 shutdown_connections 2023-10-06T02:27:44.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.671+0000 7fad61f9d700 1 -- 172.21.15.31:0/3149526985 wait complete. 2023-10-06T02:27:44.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.672+0000 7fad61f9d700 1 Processor -- start 2023-10-06T02:27:44.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.673+0000 7fad61f9d700 1 -- start start 2023-10-06T02:27:44.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.673+0000 7fad61f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fad5c159550 0x7fad5c0ce920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:44.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.673+0000 7fad61f9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad5c159920 0x7fad5c1695a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:44.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.674+0000 7fad61f9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad5c15dfd0 0x7fad5c151550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:44.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.674+0000 7fad61f9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad5c156ae0 con 0x7fad5c159920 2023-10-06T02:27:44.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.674+0000 7fad61f9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad5c1520f0 con 0x7fad5c15dfd0 2023-10-06T02:27:44.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.674+0000 7fad61f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad5c155ed0 con 0x7fad5c159550 2023-10-06T02:27:44.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.674+0000 7fad5bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad5c159920 0x7fad5c1695a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:44.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.674+0000 7fad60f9b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fad5c159550 0x7fad5c0ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:44.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.674+0000 7fad5bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad5c159920 0x7fad5c1695a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50996/0 (socket says 172.21.15.31:50996) 2023-10-06T02:27:44.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.674+0000 7fad5bfff700 1 -- 172.21.15.31:0/3665375261 learned_addr learned my addr 172.21.15.31:0/3665375261 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:44.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.674+0000 7fad6179c700 1 --2- 172.21.15.31:0/3665375261 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad5c15dfd0 0x7fad5c151550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:44.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.674+0000 7fad5bfff700 1 -- 172.21.15.31:0/3665375261 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fad5c159550 msgr2=0x7fad5c0ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:44.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.674+0000 7fad5bfff700 1 --2- 172.21.15.31:0/3665375261 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fad5c159550 0x7fad5c0ce920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:44.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.674+0000 7fad5bfff700 1 -- 172.21.15.31:0/3665375261 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad5c15dfd0 msgr2=0x7fad5c151550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:44.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.674+0000 7fad5bfff700 1 --2- 172.21.15.31:0/3665375261 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad5c15dfd0 0x7fad5c151550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:44.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.674+0000 7fad5bfff700 1 -- 172.21.15.31:0/3665375261 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad48009420 con 0x7fad5c159920 2023-10-06T02:27:44.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.674+0000 7fad60f9b700 1 --2- 172.21.15.31:0/3665375261 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fad5c159550 0x7fad5c0ce920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:27:44.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.675+0000 7fad5bfff700 1 --2- 172.21.15.31:0/3665375261 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad5c159920 0x7fad5c1695a0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fad4800ec80 tx=0x7fad480091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:44.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.675+0000 7fad58ff9700 1 -- 172.21.15.31:0/3665375261 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad4800c850 con 0x7fad5c159920 2023-10-06T02:27:44.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.675+0000 7fad58ff9700 1 -- 172.21.15.31:0/3665375261 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad4800c9b0 con 0x7fad5c159920 2023-10-06T02:27:44.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.675+0000 7fad58ff9700 1 -- 172.21.15.31:0/3665375261 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad4800cd30 con 0x7fad5c159920 2023-10-06T02:27:44.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.675+0000 7fad61f9d700 1 -- 172.21.15.31:0/3665375261 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad54009ae0 con 0x7fad5c159920 2023-10-06T02:27:44.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.675+0000 7fad61f9d700 1 -- 172.21.15.31:0/3665375261 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad5c156490 con 0x7fad5c159920 2023-10-06T02:27:44.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.676+0000 7fad58ff9700 1 -- 172.21.15.31:0/3665375261 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 42643+0+0 (secure 0 0 0) 0x7fad48013680 con 0x7fad5c159920 2023-10-06T02:27:44.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.676+0000 7fad61f9d700 1 -- 172.21.15.31:0/3665375261 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad5c156c50 con 0x7fad5c159920 2023-10-06T02:27:44.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.677+0000 7fad58ff9700 1 --2- 172.21.15.31:0/3665375261 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fad40035900 0x7fad40037db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:44.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.677+0000 7fad58ff9700 1 -- 172.21.15.31:0/3665375261 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6617+0+0 (secure 0 0 0) 0x7fad48048850 con 0x7fad5c159920 2023-10-06T02:27:44.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.677+0000 7fad60f9b700 1 --2- 172.21.15.31:0/3665375261 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fad40035900 0x7fad40037db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:44.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.678+0000 7fad60f9b700 1 --2- 172.21.15.31:0/3665375261 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fad40035900 0x7fad40037db0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fad50007fd0 tx=0x7fad50007d50 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:44.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.682+0000 7fad58ff9700 1 -- 172.21.15.31:0/3665375261 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fad48013930 con 0x7fad5c159920 2023-10-06T02:27:44.711 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:27:44.710+0000 7f54bc28a180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T02:27:44.806 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:27:44.805+0000 7f54bc28a180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T02:27:44.903 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T02:27:44.903+0000 7f54bc28a180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T02:27:44.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.935+0000 7fad61f9d700 1 -- 172.21.15.31:0/3665375261 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fad5c04c3c0 con 0x7fad5c159920 2023-10-06T02:27:44.935 INFO:tasks.ceph.mon.a.smithi031.stderr:2023-10-06T02:27:44.935+0000 7f6de01f1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T02:27:44.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.940+0000 7fad58ff9700 1 -- 172.21.15.31:0/3665375261 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v48) v1 ==== 161+0+0 (secure 0 0 0) 0x7fad4801bb60 con 0x7fad5c159920 2023-10-06T02:27:44.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.943+0000 7fad61f9d700 1 -- 172.21.15.31:0/3665375261 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fad40035900 msgr2=0x7fad40037db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:44.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.943+0000 7fad61f9d700 1 --2- 172.21.15.31:0/3665375261 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fad40035900 0x7fad40037db0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fad50007fd0 tx=0x7fad50007d50 comp rx=0 tx=0).stop 2023-10-06T02:27:44.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.943+0000 7fad61f9d700 1 -- 172.21.15.31:0/3665375261 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad5c159920 msgr2=0x7fad5c1695a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:44.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.943+0000 7fad61f9d700 1 --2- 172.21.15.31:0/3665375261 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad5c159920 0x7fad5c1695a0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fad4800ec80 tx=0x7fad480091a0 comp rx=0 tx=0).stop 2023-10-06T02:27:44.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.943+0000 7fad61f9d700 1 -- 172.21.15.31:0/3665375261 shutdown_connections 2023-10-06T02:27:44.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.943+0000 7fad61f9d700 1 --2- 172.21.15.31:0/3665375261 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad5c15dfd0 0x7fad5c151550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:44.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.943+0000 7fad61f9d700 1 --2- 172.21.15.31:0/3665375261 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fad40035900 0x7fad40037db0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:44.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.943+0000 7fad61f9d700 1 --2- 172.21.15.31:0/3665375261 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fad5c159550 0x7fad5c0ce920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:44.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.943+0000 7fad61f9d700 1 --2- 172.21.15.31:0/3665375261 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad5c159920 0x7fad5c1695a0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:44.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.943+0000 7fad61f9d700 1 -- 172.21.15.31:0/3665375261 >> 172.21.15.31:0/3665375261 conn(0x7fad5c06ce80 msgr2=0x7fad5c0b3520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:44.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.943+0000 7fad61f9d700 1 -- 172.21.15.31:0/3665375261 shutdown_connections 2023-10-06T02:27:44.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:44.943+0000 7fad61f9d700 1 -- 172.21.15.31:0/3665375261 wait complete. 2023-10-06T02:27:44.945 INFO:teuthology.orchestra.run.smithi031.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T02:27:44.951 INFO:tasks.ceph.mds.c.smithi031.stderr: -6235> 2023-10-06T02:27:26.552+0000 7f21947f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T02:27:44.961 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T02:27:44.998 INFO:tasks.ceph.mds.c.smithi031.stdout:starting mds.c at 2023-10-06T02:27:45.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.151+0000 7f485c99d700 1 Processor -- start 2023-10-06T02:27:45.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.151+0000 7f485c99d700 1 -- start start 2023-10-06T02:27:45.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.152+0000 7f485c99d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4858153a00 0x7f4858153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:45.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.152+0000 7f485c99d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f485815ac50 con 0x7f48580cea00 2023-10-06T02:27:45.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.152+0000 7f485c99d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f485815d010 con 0x7f485815de50 2023-10-06T02:27:45.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.152+0000 7f485c99d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f485815af60 con 0x7f4858153a00 2023-10-06T02:27:45.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.153+0000 7f4857fff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f48580cea00 0x7f48580cee40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:37520/0 (socket says 172.21.15.31:37520) 2023-10-06T02:27:45.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.153+0000 7f4857fff700 1 -- 172.21.15.31:0/1568602573 learned_addr learned my addr 172.21.15.31:0/1568602573 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:27:45.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.153+0000 7f48577fe700 1 --2- 172.21.15.31:0/1568602573 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4858153a00 0x7f4858153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:45.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.153+0000 7f48567fc700 1 -- 172.21.15.31:0/1568602573 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3517232785 0 0) 0x7f485815ac50 con 0x7f48580cea00 2023-10-06T02:27:45.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.153+0000 7f48567fc700 1 -- 172.21.15.31:0/1568602573 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4840003570 con 0x7f48580cea00 2023-10-06T02:27:45.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.153+0000 7f48567fc700 1 -- 172.21.15.31:0/1568602573 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3010235421 0 0) 0x7f485815d010 con 0x7f485815de50 2023-10-06T02:27:45.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.153+0000 7f48567fc700 1 -- 172.21.15.31:0/1568602573 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f485815ac50 con 0x7f485815de50 2023-10-06T02:27:45.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.153+0000 7f48567fc700 1 -- 172.21.15.31:0/1568602573 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1176002263 0 0) 0x7f4840003570 con 0x7f48580cea00 2023-10-06T02:27:45.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.153+0000 7f48567fc700 1 -- 172.21.15.31:0/1568602573 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4858153a00 msgr2=0x7f4858153dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:45.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.153+0000 7f48567fc700 1 --2- 172.21.15.31:0/1568602573 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4858153a00 0x7f4858153dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:45.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.153+0000 7f48567fc700 1 -- 172.21.15.31:0/1568602573 >> v1:172.21.15.62:6789/0 conn(0x7f485815de50 legacy=0x7f485815a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:45.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.153+0000 7f48567fc700 1 -- 172.21.15.31:0/1568602573 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f485815b240 con 0x7f48580cea00 2023-10-06T02:27:45.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.153+0000 7f48567fc700 1 -- 172.21.15.31:0/1568602573 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7f48500017f0 con 0x7f48580cea00 2023-10-06T02:27:45.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.153+0000 7f48577fe700 1 --2- 172.21.15.31:0/1568602573 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4858153a00 0x7f4858153dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:27:45.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.154+0000 7f48567fc700 1 -- 172.21.15.31:0/1568602573 >> v1:172.21.15.31:6789/0 conn(0x7f48580cea00 legacy=0x7f48580cee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:45.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.154+0000 7f48567fc700 1 --2- 172.21.15.31:0/1568602573 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4840003c20 0x7f4840004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:45.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.154+0000 7f48567fc700 1 --2- 172.21.15.31:0/1568602573 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4858153a00 0x7f4840004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:45.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.154+0000 7f48567fc700 1 --2- 172.21.15.31:0/1568602573 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4840004d50 0x7f48400051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:45.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.154+0000 7f48567fc700 1 -- 172.21.15.31:0/1568602573 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f485815af60 con 0x7f4840004d50 2023-10-06T02:27:45.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.154+0000 7f48567fc700 1 -- 172.21.15.31:0/1568602573 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f484000bb70 con 0x7f4840003c20 2023-10-06T02:27:45.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.154+0000 7f48567fc700 1 -- 172.21.15.31:0/1568602573 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f484000bd30 con 0x7f4858153a00 2023-10-06T02:27:45.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.154+0000 7f4857fff700 1 --2- 172.21.15.31:0/1568602573 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4840004d50 0x7f48400051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:45.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.154+0000 7f48577fe700 1 --2- 172.21.15.31:0/1568602573 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4840003c20 0x7f4840004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:45.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.154+0000 7f4856ffd700 1 --2- 172.21.15.31:0/1568602573 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4858153a00 0x7f4840004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:45.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.154+0000 7f4857fff700 1 -- 172.21.15.31:0/1568602573 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4858153a00 msgr2=0x7f4840004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:45.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.154+0000 7f4857fff700 1 --2- 172.21.15.31:0/1568602573 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4858153a00 0x7f4840004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:45.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.154+0000 7f4857fff700 1 -- 172.21.15.31:0/1568602573 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4840003c20 msgr2=0x7f4840004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:45.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.154+0000 7f4857fff700 1 --2- 172.21.15.31:0/1568602573 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4840003c20 0x7f4840004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:45.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.154+0000 7f4857fff700 1 -- 172.21.15.31:0/1568602573 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4848009420 con 0x7f4840004d50 2023-10-06T02:27:45.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.155+0000 7f4857fff700 1 --2- 172.21.15.31:0/1568602573 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4840004d50 0x7f48400051c0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f485000a820 tx=0x7f48500020c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc48007b8512c8a4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:45.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.155+0000 7f48567fc700 1 -- 172.21.15.31:0/1568602573 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48500070f0 con 0x7f4840004d50 2023-10-06T02:27:45.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.155+0000 7f48567fc700 1 -- 172.21.15.31:0/1568602573 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4850007270 con 0x7f4840004d50 2023-10-06T02:27:45.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.155+0000 7f48567fc700 1 -- 172.21.15.31:0/1568602573 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4850007630 con 0x7f4840004d50 2023-10-06T02:27:45.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.155+0000 7f485c99d700 1 -- 172.21.15.31:0/1568602573 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4840004d50 msgr2=0x7f48400051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:45.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.155+0000 7f485c99d700 1 --2- 172.21.15.31:0/1568602573 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4840004d50 0x7f48400051c0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f485000a820 tx=0x7f48500020c0 comp rx=0 tx=0).stop 2023-10-06T02:27:45.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.155+0000 7f4857fff700 1 -- 172.21.15.31:0/1568602573 reap_dead start 2023-10-06T02:27:45.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.155+0000 7f485c99d700 1 -- 172.21.15.31:0/1568602573 shutdown_connections 2023-10-06T02:27:45.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.156+0000 7f485c99d700 1 -- 172.21.15.31:0/1568602573 >> 172.21.15.31:0/1568602573 conn(0x7f48580703a0 msgr2=0x7f48580707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:45.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.156+0000 7f485c99d700 1 -- 172.21.15.31:0/1568602573 shutdown_connections 2023-10-06T02:27:45.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.156+0000 7f485c99d700 1 -- 172.21.15.31:0/1568602573 wait complete. 2023-10-06T02:27:45.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.157+0000 7f485c99d700 1 Processor -- start 2023-10-06T02:27:45.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.157+0000 7f485c99d700 1 -- start start 2023-10-06T02:27:45.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.158+0000 7f485c99d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4840004d50 0x7f4858152aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:45.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.158+0000 7f485c99d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f48580ccaa0 0x7f485815f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:45.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.158+0000 7f485c99d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48580ccf10 0x7f485815fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:45.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.158+0000 7f485c99d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48580c84b0 con 0x7f48580ccf10 2023-10-06T02:27:45.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.158+0000 7f485c99d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48580c75e0 con 0x7f4840004d50 2023-10-06T02:27:45.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.158+0000 7f485c99d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48580c78a0 con 0x7f48580ccaa0 2023-10-06T02:27:45.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.158+0000 7f4857fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48580ccf10 0x7f485815fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:45.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.158+0000 7f4857fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48580ccf10 0x7f485815fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51006/0 (socket says 172.21.15.31:51006) 2023-10-06T02:27:45.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.158+0000 7f4857fff700 1 -- 172.21.15.31:0/3338029028 learned_addr learned my addr 172.21.15.31:0/3338029028 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:45.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.159+0000 7f4857fff700 1 -- 172.21.15.31:0/3338029028 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f48580ccaa0 msgr2=0x7f485815f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:45.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.159+0000 7f4856ffd700 1 --2- 172.21.15.31:0/3338029028 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f48580ccaa0 0x7f485815f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:45.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.159+0000 7f48577fe700 1 --2- 172.21.15.31:0/3338029028 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4840004d50 0x7f4858152aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:45.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.159+0000 7f4857fff700 1 --2- 172.21.15.31:0/3338029028 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f48580ccaa0 0x7f485815f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:45.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.159+0000 7f4857fff700 1 -- 172.21.15.31:0/3338029028 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4840004d50 msgr2=0x7f4858152aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:45.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.159+0000 7f4857fff700 1 --2- 172.21.15.31:0/3338029028 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4840004d50 0x7f4858152aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:45.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.159+0000 7f4857fff700 1 -- 172.21.15.31:0/3338029028 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48580c7b80 con 0x7f48580ccf10 2023-10-06T02:27:45.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.159+0000 7f4856ffd700 1 --2- 172.21.15.31:0/3338029028 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f48580ccaa0 0x7f485815f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:27:45.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.159+0000 7f4857fff700 1 --2- 172.21.15.31:0/3338029028 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48580ccf10 0x7f485815fd70 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f4850012000 tx=0x7f4850016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:45.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.159+0000 7f483f7fe700 1 -- 172.21.15.31:0/3338029028 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4850007070 con 0x7f48580ccf10 2023-10-06T02:27:45.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.159+0000 7f483f7fe700 1 -- 172.21.15.31:0/3338029028 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4850001e00 con 0x7f48580ccf10 2023-10-06T02:27:45.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.159+0000 7f485c99d700 1 -- 172.21.15.31:0/3338029028 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48580c7e60 con 0x7f48580ccf10 2023-10-06T02:27:45.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.159+0000 7f483f7fe700 1 -- 172.21.15.31:0/3338029028 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48500079a0 con 0x7f48580ccf10 2023-10-06T02:27:45.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.159+0000 7f485c99d700 1 -- 172.21.15.31:0/3338029028 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4858152fe0 con 0x7f48580ccf10 2023-10-06T02:27:45.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.160+0000 7f485c99d700 1 -- 172.21.15.31:0/3338029028 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48580c8840 con 0x7f48580ccf10 2023-10-06T02:27:45.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.160+0000 7f483f7fe700 1 -- 172.21.15.31:0/3338029028 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 42643+0+0 (secure 0 0 0) 0x7f4850004bb0 con 0x7f48580ccf10 2023-10-06T02:27:45.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.161+0000 7f483f7fe700 1 --2- 172.21.15.31:0/3338029028 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f4844035900 0x7f4844037db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:45.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.161+0000 7f483f7fe700 1 -- 172.21.15.31:0/3338029028 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6703+0+0 (secure 0 0 0) 0x7f485004ac20 con 0x7f48580ccf10 2023-10-06T02:27:45.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.161+0000 7f48577fe700 1 --2- 172.21.15.31:0/3338029028 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f4844035900 0x7f4844037db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:45.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.162+0000 7f48577fe700 1 --2- 172.21.15.31:0/3338029028 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f4844035900 0x7f4844037db0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f4848006f00 tx=0x7f4848008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:45.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.166+0000 7f483f7fe700 1 -- 172.21.15.31:0/3338029028 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f48580c8840 con 0x7f48580ccf10 2023-10-06T02:27:45.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.413+0000 7f485c99d700 1 -- 172.21.15.31:0/3338029028 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f48580c8840 con 0x7f48580ccf10 2023-10-06T02:27:45.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.442+0000 7f483f7fe700 1 -- 172.21.15.31:0/3338029028 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f485001bcd0 con 0x7f48580ccf10 2023-10-06T02:27:45.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.942+0000 7f483f7fe700 1 -- 172.21.15.31:0/3338029028 <== mon.0 v2:172.21.15.31:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v49) v1 ==== 104+0+0 (secure 0 0 0) 0x7f48500113a0 con 0x7f48580ccf10 2023-10-06T02:27:45.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.945+0000 7f485c99d700 1 -- 172.21.15.31:0/3338029028 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f4844035900 msgr2=0x7f4844037db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:45.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.945+0000 7f485c99d700 1 --2- 172.21.15.31:0/3338029028 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f4844035900 0x7f4844037db0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f4848006f00 tx=0x7f4848008040 comp rx=0 tx=0).stop 2023-10-06T02:27:45.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.945+0000 7f485c99d700 1 -- 172.21.15.31:0/3338029028 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48580ccf10 msgr2=0x7f485815fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:45.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.945+0000 7f485c99d700 1 --2- 172.21.15.31:0/3338029028 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48580ccf10 0x7f485815fd70 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f4850012000 tx=0x7f4850016000 comp rx=0 tx=0).stop 2023-10-06T02:27:45.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.945+0000 7f485c99d700 1 -- 172.21.15.31:0/3338029028 shutdown_connections 2023-10-06T02:27:45.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.945+0000 7f485c99d700 1 --2- 172.21.15.31:0/3338029028 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48580ccf10 0x7f485815fd70 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:45.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.945+0000 7f485c99d700 1 --2- 172.21.15.31:0/3338029028 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4840004d50 0x7f4858152aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:45.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.945+0000 7f485c99d700 1 --2- 172.21.15.31:0/3338029028 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f4844035900 0x7f4844037db0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:45.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.945+0000 7f485c99d700 1 --2- 172.21.15.31:0/3338029028 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f48580ccaa0 0x7f485815f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:45.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.945+0000 7f485c99d700 1 -- 172.21.15.31:0/3338029028 >> 172.21.15.31:0/3338029028 conn(0x7f48580703a0 msgr2=0x7f485815bf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:45.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.945+0000 7f485c99d700 1 -- 172.21.15.31:0/3338029028 shutdown_connections 2023-10-06T02:27:45.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:45.945+0000 7f485c99d700 1 -- 172.21.15.31:0/3338029028 wait complete. 2023-10-06T02:27:45.966 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T02:27:46.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:46.203+0000 7f7ba359e700 1 Processor -- start 2023-10-06T02:27:46.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:46.203+0000 7f7ba359e700 1 -- start start 2023-10-06T02:27:46.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:46.205+0000 7f7ba359e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7ba40c7870 0x7f7ba406c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:46.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:46.205+0000 7f7ba359e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7ba406cbe0 0x7f7ba40c3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:46.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:46.205+0000 7f7ba359e700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ba40c7d10 con 0x7f7ba415cb90 2023-10-06T02:27:46.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:46.205+0000 7f7ba359e700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ba40ccdf0 con 0x7f7ba40c7870 2023-10-06T02:27:46.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:46.205+0000 7f7ba359e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba40c9b20 con 0x7f7ba406cbe0 2023-10-06T02:27:46.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:46.205+0000 7f7ba1d9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7ba40c7870 0x7f7ba406c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:46.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:46.205+0000 7f7ba1d9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7ba40c7870 0x7f7ba406c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51022/0 (socket says 172.21.15.31:51022) 2023-10-06T02:27:46.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:46.205+0000 7f7ba1d9b700 1 -- 172.21.15.31:0/2515909319 learned_addr learned my addr 172.21.15.31:0/2515909319 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:46.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:46.205+0000 7f7ba259c700 1 --1- 172.21.15.31:0/2515909319 >> v1:172.21.15.62:6789/0 conn(0x7f7ba415cb90 0x7f7ba415dc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:40668/0 (socket says 172.21.15.31:40668) 2023-10-06T02:27:46.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:46.205+0000 7f7ba2d9d700 1 --2- 172.21.15.31:0/2515909319 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7ba406cbe0 0x7f7ba40c3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:46.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:46.206+0000 7f7ba1d9b700 1 -- 172.21.15.31:0/2515909319 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7ba406cbe0 msgr2=0x7f7ba40c3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:46.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:46.206+0000 7f7ba1d9b700 1 --2- 172.21.15.31:0/2515909319 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7ba406cbe0 0x7f7ba40c3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:46.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:46.206+0000 7f7ba1d9b700 1 -- 172.21.15.31:0/2515909319 >> v1:172.21.15.62:6789/0 conn(0x7f7ba415cb90 legacy=0x7f7ba415dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:46.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:46.206+0000 7f7ba1d9b700 1 -- 172.21.15.31:0/2515909319 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ba40c2fa0 con 0x7f7ba40c7870 2023-10-06T02:27:46.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:46.206+0000 7f7ba1d9b700 1 --2- 172.21.15.31:0/2515909319 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ba40c7870 0x7f7ba406c6a0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f7b980099b0 tx=0x7f7b98009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5326a4234f2af8a7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:46.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:46.206+0000 7f7ba159a700 1 -- 172.21.15.31:0/2515909319 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b98009ea0 con 0x7f7ba40c7870 2023-10-06T02:27:46.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:46.206+0000 7f7ba159a700 1 -- 172.21.15.31:0/2515909319 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b9800b860 con 0x7f7ba40c7870 2023-10-06T02:27:46.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:46.206+0000 7f7ba159a700 1 -- 172.21.15.31:0/2515909319 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9800bc00 con 0x7f7ba40c7870 2023-10-06T02:27:46.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:46.207+0000 7f7ba359e700 1 -- 172.21.15.31:0/2515909319 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ba40c7870 msgr2=0x7f7ba406c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:46.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:46.207+0000 7f7ba359e700 1 --2- 172.21.15.31:0/2515909319 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ba40c7870 0x7f7ba406c6a0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f7b980099b0 tx=0x7f7b98009250 comp rx=0 tx=0).stop 2023-10-06T02:27:46.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:46.207+0000 7f7ba359e700 1 -- 172.21.15.31:0/2515909319 shutdown_connections 2023-10-06T02:27:46.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:46.207+0000 7f7ba359e700 1 --2- 172.21.15.31:0/2515909319 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7ba406cbe0 0x7f7ba40c3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:46.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:46.207+0000 7f7ba359e700 1 --2- 172.21.15.31:0/2515909319 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ba40c7870 0x7f7ba406c6a0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:46.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:46.207+0000 7f7ba359e700 1 -- 172.21.15.31:0/2515909319 >> 172.21.15.31:0/2515909319 conn(0x7f7ba40cd230 msgr2=0x7f7ba40cd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:46.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:46.207+0000 7f7ba359e700 1 -- 172.21.15.31:0/2515909319 shutdown_connections 2023-10-06T02:27:46.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:46.207+0000 7f7ba359e700 1 -- 172.21.15.31:0/2515909319 wait complete. 2023-10-06T02:27:46.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:46.208+0000 7f7ba359e700 1 Processor -- start 2023-10-06T02:27:46.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:46.208+0000 7f7ba359e700 1 -- start start 2023-10-06T02:27:46.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:46.209+0000 7f7ba359e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7ba406cbe0 0x7f7ba40c42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:46.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:46.209+0000 7f7ba359e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ba40c7870 0x7f7ba413f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:46.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:46.209+0000 7f7ba359e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ba415cb90 0x7f7ba413cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:46.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:46.209+0000 7f7ba359e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba416d7e0 con 0x7f7ba40c7870 2023-10-06T02:27:46.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:46.209+0000 7f7ba359e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba416dac0 con 0x7f7ba415cb90 2023-10-06T02:27:46.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:46.209+0000 7f7ba359e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba40c5fc0 con 0x7f7ba406cbe0 2023-10-06T02:27:46.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:46.209+0000 7f7ba1d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ba40c7870 0x7f7ba413f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:46.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:46.210+0000 7f7ba1d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ba40c7870 0x7f7ba413f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51038/0 (socket says 172.21.15.31:51038) 2023-10-06T02:27:46.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:46.210+0000 7f7ba1d9b700 1 -- 172.21.15.31:0/563503573 learned_addr learned my addr 172.21.15.31:0/563503573 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:46.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:46.210+0000 7f7ba259c700 1 --2- 172.21.15.31:0/563503573 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7ba406cbe0 0x7f7ba40c42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:46.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:46.210+0000 7f7ba1d9b700 1 -- 172.21.15.31:0/563503573 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7ba406cbe0 msgr2=0x7f7ba40c42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:46.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:46.210+0000 7f7ba1d9b700 1 --2- 172.21.15.31:0/563503573 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7ba406cbe0 0x7f7ba40c42e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:46.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:46.210+0000 7f7ba1d9b700 1 -- 172.21.15.31:0/563503573 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ba415cb90 msgr2=0x7f7ba413cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:46.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:46.210+0000 7f7ba1d9b700 1 --2- 172.21.15.31:0/563503573 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ba415cb90 0x7f7ba413cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:46.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:46.210+0000 7f7ba1d9b700 1 -- 172.21.15.31:0/563503573 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b980094f0 con 0x7f7ba40c7870 2023-10-06T02:27:46.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:46.210+0000 7f7ba1d9b700 1 --2- 172.21.15.31:0/563503573 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ba40c7870 0x7f7ba413f650 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f7ba4063720 tx=0x7f7b98017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:46.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:46.210+0000 7f7b927fc700 1 -- 172.21.15.31:0/563503573 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9800bbe0 con 0x7f7ba40c7870 2023-10-06T02:27:46.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:46.210+0000 7f7b927fc700 1 -- 172.21.15.31:0/563503573 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b98011420 con 0x7f7ba40c7870 2023-10-06T02:27:46.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:46.210+0000 7f7ba359e700 1 -- 172.21.15.31:0/563503573 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ba40c62a0 con 0x7f7ba40c7870 2023-10-06T02:27:46.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:46.210+0000 7f7b927fc700 1 -- 172.21.15.31:0/563503573 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b980117c0 con 0x7f7ba40c7870 2023-10-06T02:27:46.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:46.211+0000 7f7ba359e700 1 -- 172.21.15.31:0/563503573 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ba413d1a0 con 0x7f7ba40c7870 2023-10-06T02:27:46.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:46.211+0000 7f7ba359e700 1 -- 172.21.15.31:0/563503573 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ba40c5e30 con 0x7f7ba40c7870 2023-10-06T02:27:46.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:46.212+0000 7f7b927fc700 1 -- 172.21.15.31:0/563503573 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f7b98011bf0 con 0x7f7ba40c7870 2023-10-06T02:27:46.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:46.214+0000 7f7b927fc700 1 --2- 172.21.15.31:0/563503573 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7b84067050 0x7f7b84069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:46.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:46.214+0000 7f7b927fc700 1 -- 172.21.15.31:0/563503573 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6703+0+0 (secure 0 0 0) 0x7f7b98086420 con 0x7f7ba40c7870 2023-10-06T02:27:46.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:46.215+0000 7f7ba259c700 1 --2- 172.21.15.31:0/563503573 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7b84067050 0x7f7b84069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:46.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:46.215+0000 7f7ba259c700 1 --2- 172.21.15.31:0/563503573 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7b84067050 0x7f7b84069500 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7b94001780 tx=0x7f7b94009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:46.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:46.219+0000 7f7b927fc700 1 -- 172.21.15.31:0/563503573 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7b98053100 con 0x7f7ba40c7870 2023-10-06T02:27:46.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:46.421+0000 7f7ba359e700 1 -- 172.21.15.31:0/563503573 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f7ba404c3c0 con 0x7f7ba40c7870 2023-10-06T02:27:46.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:46.944+0000 7f7b927fc700 1 -- 172.21.15.31:0/563503573 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v78) v1 ==== 181+0+0 (secure 0 0 0) 0x7f7b98056720 con 0x7f7ba40c7870 2023-10-06T02:27:46.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:46.948+0000 7f7ba359e700 1 -- 172.21.15.31:0/563503573 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7b84067050 msgr2=0x7f7b84069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:46.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:46.948+0000 7f7ba359e700 1 --2- 172.21.15.31:0/563503573 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7b84067050 0x7f7b84069500 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7b94001780 tx=0x7f7b94009040 comp rx=0 tx=0).stop 2023-10-06T02:27:46.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:46.948+0000 7f7ba359e700 1 -- 172.21.15.31:0/563503573 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ba40c7870 msgr2=0x7f7ba413f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:46.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:46.948+0000 7f7ba359e700 1 --2- 172.21.15.31:0/563503573 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ba40c7870 0x7f7ba413f650 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f7ba4063720 tx=0x7f7b98017000 comp rx=0 tx=0).stop 2023-10-06T02:27:46.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:46.948+0000 7f7ba359e700 1 -- 172.21.15.31:0/563503573 shutdown_connections 2023-10-06T02:27:46.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:46.948+0000 7f7ba359e700 1 --2- 172.21.15.31:0/563503573 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ba415cb90 0x7f7ba413cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:46.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:46.948+0000 7f7ba359e700 1 --2- 172.21.15.31:0/563503573 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7b84067050 0x7f7b84069500 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:46.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:46.948+0000 7f7ba359e700 1 --2- 172.21.15.31:0/563503573 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7ba406cbe0 0x7f7ba40c42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:46.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:46.948+0000 7f7ba359e700 1 --2- 172.21.15.31:0/563503573 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ba40c7870 0x7f7ba413f650 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:46.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:46.948+0000 7f7ba359e700 1 -- 172.21.15.31:0/563503573 >> 172.21.15.31:0/563503573 conn(0x7f7ba40cd230 msgr2=0x7f7ba4159dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:46.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:46.948+0000 7f7ba359e700 1 -- 172.21.15.31:0/563503573 shutdown_connections 2023-10-06T02:27:46.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:46.949+0000 7f7ba359e700 1 -- 172.21.15.31:0/563503573 wait complete. 2023-10-06T02:27:46.950 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' removed 2023-10-06T02:27:46.970 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T02:27:47.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.171+0000 7fd732b9d700 1 Processor -- start 2023-10-06T02:27:47.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.171+0000 7fd732b9d700 1 -- start start 2023-10-06T02:27:47.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.172+0000 7fd732b9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd72c140f40 0x7fd72c0cb8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:47.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.172+0000 7fd732b9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd72c0cbfb0 con 0x7fd72c140b70 2023-10-06T02:27:47.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.172+0000 7fd732b9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd72c0cc290 con 0x7fd72c06cdb0 2023-10-06T02:27:47.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.172+0000 7fd732b9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd72c0b0220 con 0x7fd72c140f40 2023-10-06T02:27:47.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.173+0000 7fd731b9b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fd72c140b70 0x7fd72c06c6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:37534/0 (socket says 172.21.15.31:37534) 2023-10-06T02:27:47.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.173+0000 7fd731b9b700 1 -- 172.21.15.31:0/3793464501 learned_addr learned my addr 172.21.15.31:0/3793464501 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:27:47.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.173+0000 7fd73239c700 1 --2- 172.21.15.31:0/3793464501 >> v2:172.21.15.62:3300/0 conn(0x7fd72c140f40 0x7fd72c0cb8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:47.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.173+0000 7fd730b99700 1 -- 172.21.15.31:0/3793464501 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 813636582 0 0) 0x7fd72c0cbfb0 con 0x7fd72c140b70 2023-10-06T02:27:47.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.173+0000 7fd730b99700 1 -- 172.21.15.31:0/3793464501 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd718003570 con 0x7fd72c140b70 2023-10-06T02:27:47.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.173+0000 7fd730b99700 1 -- 172.21.15.31:0/3793464501 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1260714348 0 0) 0x7fd718003570 con 0x7fd72c140b70 2023-10-06T02:27:47.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.173+0000 7fd730b99700 1 -- 172.21.15.31:0/3793464501 >> v2:172.21.15.62:3300/0 conn(0x7fd72c140f40 msgr2=0x7fd72c0cb8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:47.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.173+0000 7fd730b99700 1 --2- 172.21.15.31:0/3793464501 >> v2:172.21.15.62:3300/0 conn(0x7fd72c140f40 0x7fd72c0cb8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:47.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.173+0000 7fd730b99700 1 -- 172.21.15.31:0/3793464501 >> v1:172.21.15.62:6789/0 conn(0x7fd72c06cdb0 legacy=0x7fd72c0cb1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:47.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.173+0000 7fd730b99700 1 -- 172.21.15.31:0/3793464501 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd72c0cc570 con 0x7fd72c140b70 2023-10-06T02:27:47.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.173+0000 7fd730b99700 1 -- 172.21.15.31:0/3793464501 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7fd728001470 con 0x7fd72c140b70 2023-10-06T02:27:47.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.173+0000 7fd730b99700 1 -- 172.21.15.31:0/3793464501 >> v1:172.21.15.31:6789/0 conn(0x7fd72c140b70 legacy=0x7fd72c06c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:47.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.174+0000 7fd730b99700 1 --2- 172.21.15.31:0/3793464501 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd718003a60 0x7fd718003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:47.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.174+0000 7fd730b99700 1 --2- 172.21.15.31:0/3793464501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd718004810 0x7fd718004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:47.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.174+0000 7fd730b99700 1 --2- 172.21.15.31:0/3793464501 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd7180051c0 0x7fd71800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:47.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.174+0000 7fd730b99700 1 -- 172.21.15.31:0/3793464501 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd72c0b0220 con 0x7fd7180051c0 2023-10-06T02:27:47.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.174+0000 7fd730b99700 1 -- 172.21.15.31:0/3793464501 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd71800c0b0 con 0x7fd718004810 2023-10-06T02:27:47.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.174+0000 7fd730b99700 1 -- 172.21.15.31:0/3793464501 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd71800c270 con 0x7fd718003a60 2023-10-06T02:27:47.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.174+0000 7fd73239c700 1 --2- 172.21.15.31:0/3793464501 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd7180051c0 0x7fd71800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:47.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.174+0000 7fd73139a700 1 --2- 172.21.15.31:0/3793464501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd718004810 0x7fd718004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:47.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.174+0000 7fd731b9b700 1 --2- 172.21.15.31:0/3793464501 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd718003a60 0x7fd718003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:47.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.174+0000 7fd73239c700 1 -- 172.21.15.31:0/3793464501 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd718003a60 msgr2=0x7fd718003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:47.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.174+0000 7fd73239c700 1 --2- 172.21.15.31:0/3793464501 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd718003a60 0x7fd718003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:47.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.174+0000 7fd73239c700 1 -- 172.21.15.31:0/3793464501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd718004810 msgr2=0x7fd718004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:47.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.174+0000 7fd73239c700 1 --2- 172.21.15.31:0/3793464501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd718004810 0x7fd718004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:47.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.174+0000 7fd73239c700 1 -- 172.21.15.31:0/3793464501 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd718003570 con 0x7fd7180051c0 2023-10-06T02:27:47.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.174+0000 7fd73239c700 1 -- 172.21.15.31:0/3793464501 reap_dead start 2023-10-06T02:27:47.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.174+0000 7fd73239c700 1 --2- 172.21.15.31:0/3793464501 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd7180051c0 0x7fd71800ba60 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fd72400e210 tx=0x7fd72400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d28d619b35f4c35 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:47.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.175+0000 7fd730b99700 1 -- 172.21.15.31:0/3793464501 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd72400eac0 con 0x7fd7180051c0 2023-10-06T02:27:47.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.175+0000 7fd730b99700 1 -- 172.21.15.31:0/3793464501 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd72400f040 con 0x7fd7180051c0 2023-10-06T02:27:47.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.175+0000 7fd730b99700 1 -- 172.21.15.31:0/3793464501 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd72400ee70 con 0x7fd7180051c0 2023-10-06T02:27:47.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.175+0000 7fd732b9d700 1 -- 172.21.15.31:0/3793464501 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd7180051c0 msgr2=0x7fd71800ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:47.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.175+0000 7fd732b9d700 1 --2- 172.21.15.31:0/3793464501 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd7180051c0 0x7fd71800ba60 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fd72400e210 tx=0x7fd72400b040 comp rx=0 tx=0).stop 2023-10-06T02:27:47.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.175+0000 7fd732b9d700 1 -- 172.21.15.31:0/3793464501 shutdown_connections 2023-10-06T02:27:47.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.175+0000 7fd732b9d700 1 --2- 172.21.15.31:0/3793464501 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd7180051c0 0x7fd71800ba60 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:47.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.175+0000 7fd732b9d700 1 -- 172.21.15.31:0/3793464501 >> 172.21.15.31:0/3793464501 conn(0x7fd72c0cd230 msgr2=0x7fd72c0cd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:47.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.175+0000 7fd732b9d700 1 -- 172.21.15.31:0/3793464501 shutdown_connections 2023-10-06T02:27:47.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.175+0000 7fd732b9d700 1 -- 172.21.15.31:0/3793464501 wait complete. 2023-10-06T02:27:47.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.177+0000 7fd732b9d700 1 Processor -- start 2023-10-06T02:27:47.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.177+0000 7fd732b9d700 1 -- start start 2023-10-06T02:27:47.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.178+0000 7fd732b9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd7180051c0 0x7fd72c132f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:47.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.178+0000 7fd732b9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd72c0c6dc0 0x7fd72c0aab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:47.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.178+0000 7fd732b9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd72c0c3010 0x7fd72c0c3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:47.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.178+0000 7fd732b9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd72c0c7c60 con 0x7fd72c0c3010 2023-10-06T02:27:47.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.178+0000 7fd732b9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd72c0c39a0 con 0x7fd72c0c6dc0 2023-10-06T02:27:47.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.178+0000 7fd732b9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd72c0c7f40 con 0x7fd7180051c0 2023-10-06T02:27:47.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.178+0000 7fd73239c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd72c0c3010 0x7fd72c0c3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:47.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.178+0000 7fd73239c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd72c0c3010 0x7fd72c0c3460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51048/0 (socket says 172.21.15.31:51048) 2023-10-06T02:27:47.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.178+0000 7fd73239c700 1 -- 172.21.15.31:0/1761169087 learned_addr learned my addr 172.21.15.31:0/1761169087 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:47.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.178+0000 7fd73139a700 1 --2- 172.21.15.31:0/1761169087 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd72c0c6dc0 0x7fd72c0aab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:47.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.178+0000 7fd731b9b700 1 --2- 172.21.15.31:0/1761169087 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd7180051c0 0x7fd72c132f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:47.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.179+0000 7fd73239c700 1 -- 172.21.15.31:0/1761169087 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd7180051c0 msgr2=0x7fd72c132f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:47.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.179+0000 7fd73239c700 1 --2- 172.21.15.31:0/1761169087 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd7180051c0 0x7fd72c132f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:47.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.179+0000 7fd73239c700 1 -- 172.21.15.31:0/1761169087 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd72c0c6dc0 msgr2=0x7fd72c0aab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:47.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.179+0000 7fd73239c700 1 --2- 172.21.15.31:0/1761169087 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd72c0c6dc0 0x7fd72c0aab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:47.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.179+0000 7fd73239c700 1 -- 172.21.15.31:0/1761169087 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd72c0c7330 con 0x7fd72c0c3010 2023-10-06T02:27:47.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.179+0000 7fd73139a700 1 --2- 172.21.15.31:0/1761169087 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd72c0c6dc0 0x7fd72c0aab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:27:47.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.179+0000 7fd73239c700 1 --2- 172.21.15.31:0/1761169087 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd72c0c3010 0x7fd72c0c3460 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fd72c0acaa0 tx=0x7fd724019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:47.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.179+0000 7fd721ffb700 1 -- 172.21.15.31:0/1761169087 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd72401d660 con 0x7fd72c0c3010 2023-10-06T02:27:47.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.179+0000 7fd721ffb700 1 -- 172.21.15.31:0/1761169087 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd724022070 con 0x7fd72c0c3010 2023-10-06T02:27:47.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.179+0000 7fd721ffb700 1 -- 172.21.15.31:0/1761169087 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd72401da30 con 0x7fd72c0c3010 2023-10-06T02:27:47.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.179+0000 7fd732b9d700 1 -- 172.21.15.31:0/1761169087 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd72c0c7610 con 0x7fd72c0c3010 2023-10-06T02:27:47.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.179+0000 7fd732b9d700 1 -- 172.21.15.31:0/1761169087 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd72c1334b0 con 0x7fd72c0c3010 2023-10-06T02:27:47.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.180+0000 7fd732b9d700 1 -- 172.21.15.31:0/1761169087 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd72c04c3c0 con 0x7fd72c0c3010 2023-10-06T02:27:47.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.181+0000 7fd721ffb700 1 -- 172.21.15.31:0/1761169087 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7fd724003020 con 0x7fd72c0c3010 2023-10-06T02:27:47.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.182+0000 7fd721ffb700 1 --2- 172.21.15.31:0/1761169087 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd710067050 0x7fd710069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:47.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.182+0000 7fd721ffb700 1 -- 172.21.15.31:0/1761169087 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6265+0+0 (secure 0 0 0) 0x7fd724024070 con 0x7fd72c0c3010 2023-10-06T02:27:47.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.183+0000 7fd731b9b700 1 --2- 172.21.15.31:0/1761169087 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd710067050 0x7fd710069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:47.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.183+0000 7fd731b9b700 1 --2- 172.21.15.31:0/1761169087 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd710067050 0x7fd710069500 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd728001ed0 tx=0x7fd728009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:47.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.186+0000 7fd721ffb700 1 -- 172.21.15.31:0/1761169087 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd7240539a0 con 0x7fd72c0c3010 2023-10-06T02:27:47.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.387+0000 7fd732b9d700 1 -- 172.21.15.31:0/1761169087 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd72c0c91c0 con 0x7fd72c0c3010 2023-10-06T02:27:47.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.947+0000 7fd721ffb700 1 -- 172.21.15.31:0/1761169087 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v79) v1 ==== 169+0+0 (secure 0 0 0) 0x7fd7240180d0 con 0x7fd72c0c3010 2023-10-06T02:27:47.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.952+0000 7fd732b9d700 1 -- 172.21.15.31:0/1761169087 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd710067050 msgr2=0x7fd710069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:47.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.952+0000 7fd732b9d700 1 --2- 172.21.15.31:0/1761169087 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd710067050 0x7fd710069500 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd728001ed0 tx=0x7fd728009040 comp rx=0 tx=0).stop 2023-10-06T02:27:47.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.952+0000 7fd732b9d700 1 -- 172.21.15.31:0/1761169087 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd72c0c3010 msgr2=0x7fd72c0c3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:47.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.952+0000 7fd732b9d700 1 --2- 172.21.15.31:0/1761169087 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd72c0c3010 0x7fd72c0c3460 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fd72c0acaa0 tx=0x7fd724019000 comp rx=0 tx=0).stop 2023-10-06T02:27:47.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.952+0000 7fd732b9d700 1 -- 172.21.15.31:0/1761169087 shutdown_connections 2023-10-06T02:27:47.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.952+0000 7fd732b9d700 1 --2- 172.21.15.31:0/1761169087 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd72c0c3010 0x7fd72c0c3460 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:47.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.952+0000 7fd732b9d700 1 --2- 172.21.15.31:0/1761169087 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd710067050 0x7fd710069500 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:47.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.952+0000 7fd732b9d700 1 --2- 172.21.15.31:0/1761169087 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd7180051c0 0x7fd72c132f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:47.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.952+0000 7fd732b9d700 1 --2- 172.21.15.31:0/1761169087 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd72c0c6dc0 0x7fd72c0aab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:47.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.952+0000 7fd732b9d700 1 -- 172.21.15.31:0/1761169087 >> 172.21.15.31:0/1761169087 conn(0x7fd72c0cd230 msgr2=0x7fd72c06b850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:47.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.953+0000 7fd732b9d700 1 -- 172.21.15.31:0/1761169087 shutdown_connections 2023-10-06T02:27:47.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:47.953+0000 7fd732b9d700 1 -- 172.21.15.31:0/1761169087 wait complete. 2023-10-06T02:27:47.956 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' removed 2023-10-06T02:27:47.975 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T02:27:48.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.170+0000 7f15d1f9d700 1 Processor -- start 2023-10-06T02:27:48.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.170+0000 7f15d1f9d700 1 -- start start 2023-10-06T02:27:48.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.172+0000 7f15d1f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f15cc155890 0x7f15cc0c7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:48.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.172+0000 7f15d1f9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f15cc06de50 0x7f15cc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:48.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.172+0000 7f15d1f9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15cc0c7e70 con 0x7f15cc155c60 2023-10-06T02:27:48.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.172+0000 7f15d1f9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15cc06e830 con 0x7f15cc06de50 2023-10-06T02:27:48.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.172+0000 7f15d1f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15cc06b790 con 0x7f15cc155890 2023-10-06T02:27:48.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.172+0000 7f15cbfff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f15cc155c60 0x7f15cc06a3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:59448/0 (socket says 172.21.15.31:59448) 2023-10-06T02:27:48.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.172+0000 7f15cbfff700 1 -- 172.21.15.31:0/3930067237 learned_addr learned my addr 172.21.15.31:0/3930067237 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:27:48.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.172+0000 7f15d0f9b700 1 --2- 172.21.15.31:0/3930067237 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f15cc155890 0x7f15cc0c7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:48.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.172+0000 7f15d179c700 1 --2- 172.21.15.31:0/3930067237 >> v2:172.21.15.62:3300/0 conn(0x7f15cc06de50 0x7f15cc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:48.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.172+0000 7f15cb7fe700 1 -- 172.21.15.31:0/3930067237 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 357291251 0 0) 0x7f15cc0c7e70 con 0x7f15cc155c60 2023-10-06T02:27:48.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.173+0000 7f15cb7fe700 1 -- 172.21.15.31:0/3930067237 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15bc004680 con 0x7f15cc155c60 2023-10-06T02:27:48.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.173+0000 7f15cb7fe700 1 -- 172.21.15.31:0/3930067237 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1259850845 0 0) 0x7f15bc004680 con 0x7f15cc155c60 2023-10-06T02:27:48.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.173+0000 7f15cb7fe700 1 -- 172.21.15.31:0/3930067237 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f15cc155890 msgr2=0x7f15cc0c7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:48.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.173+0000 7f15cb7fe700 1 --2- 172.21.15.31:0/3930067237 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f15cc155890 0x7f15cc0c7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:48.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.173+0000 7f15cb7fe700 1 -- 172.21.15.31:0/3930067237 >> v2:172.21.15.62:3300/0 conn(0x7f15cc06de50 msgr2=0x7f15cc06e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:48.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.173+0000 7f15cb7fe700 1 --2- 172.21.15.31:0/3930067237 >> v2:172.21.15.62:3300/0 conn(0x7f15cc06de50 0x7f15cc06e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:48.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.173+0000 7f15cb7fe700 1 -- 172.21.15.31:0/3930067237 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15cc06ab80 con 0x7f15cc155c60 2023-10-06T02:27:48.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.173+0000 7f15cb7fe700 1 -- 172.21.15.31:0/3930067237 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7f15b8001470 con 0x7f15cc155c60 2023-10-06T02:27:48.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.173+0000 7f15cb7fe700 1 -- 172.21.15.31:0/3930067237 >> v1:172.21.15.31:6789/0 conn(0x7f15cc155c60 legacy=0x7f15cc06a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:48.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.173+0000 7f15cb7fe700 1 --2- 172.21.15.31:0/3930067237 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15bc004b70 0x7f15bc004f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:48.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.173+0000 7f15cb7fe700 1 --2- 172.21.15.31:0/3930067237 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15bc005920 0x7f15bc005d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:48.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.173+0000 7f15cb7fe700 1 --2- 172.21.15.31:0/3930067237 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f15cc155890 0x7f15bc0062d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:48.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.173+0000 7f15cb7fe700 1 -- 172.21.15.31:0/3930067237 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15cc06e830 con 0x7f15bc004b70 2023-10-06T02:27:48.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.173+0000 7f15cb7fe700 1 -- 172.21.15.31:0/3930067237 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15cc06b790 con 0x7f15bc005920 2023-10-06T02:27:48.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.173+0000 7f15cb7fe700 1 -- 172.21.15.31:0/3930067237 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15bc00cb70 con 0x7f15cc155890 2023-10-06T02:27:48.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.173+0000 7f15d0f9b700 1 --2- 172.21.15.31:0/3930067237 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15bc004b70 0x7f15bc004f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:48.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.173+0000 7f15cbfff700 1 --2- 172.21.15.31:0/3930067237 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15bc005920 0x7f15bc005d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:48.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.173+0000 7f15d179c700 1 --2- 172.21.15.31:0/3930067237 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f15cc155890 0x7f15bc0062d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:48.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.174+0000 7f15d0f9b700 1 -- 172.21.15.31:0/3930067237 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f15cc155890 msgr2=0x7f15bc0062d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:48.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.174+0000 7f15d0f9b700 1 --2- 172.21.15.31:0/3930067237 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f15cc155890 0x7f15bc0062d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:48.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.174+0000 7f15d0f9b700 1 -- 172.21.15.31:0/3930067237 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15bc005920 msgr2=0x7f15bc005d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:48.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.174+0000 7f15d0f9b700 1 --2- 172.21.15.31:0/3930067237 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15bc005920 0x7f15bc005d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:48.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.174+0000 7f15d0f9b700 1 -- 172.21.15.31:0/3930067237 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15bc004680 con 0x7f15bc004b70 2023-10-06T02:27:48.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.174+0000 7f15cbfff700 1 --2- 172.21.15.31:0/3930067237 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15bc005920 0x7f15bc005d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:27:48.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.174+0000 7f15d0f9b700 1 --2- 172.21.15.31:0/3930067237 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15bc004b70 0x7f15bc004f80 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f15c0009c30 tx=0x7f15c00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=580981aad01b5d55 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:48.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.174+0000 7f15cb7fe700 1 -- 172.21.15.31:0/3930067237 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c000e710 con 0x7f15bc004b70 2023-10-06T02:27:48.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.174+0000 7f15cb7fe700 1 -- 172.21.15.31:0/3930067237 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15c000e870 con 0x7f15bc004b70 2023-10-06T02:27:48.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.174+0000 7f15cb7fe700 1 -- 172.21.15.31:0/3930067237 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c000ec30 con 0x7f15bc004b70 2023-10-06T02:27:48.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.175+0000 7f15d1f9d700 1 -- 172.21.15.31:0/3930067237 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15bc004b70 msgr2=0x7f15bc004f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:48.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.175+0000 7f15d1f9d700 1 --2- 172.21.15.31:0/3930067237 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15bc004b70 0x7f15bc004f80 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f15c0009c30 tx=0x7f15c00091a0 comp rx=0 tx=0).stop 2023-10-06T02:27:48.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.175+0000 7f15d179c700 1 -- 172.21.15.31:0/3930067237 reap_dead start 2023-10-06T02:27:48.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.175+0000 7f15d1f9d700 1 -- 172.21.15.31:0/3930067237 shutdown_connections 2023-10-06T02:27:48.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.175+0000 7f15d1f9d700 1 -- 172.21.15.31:0/3930067237 >> 172.21.15.31:0/3930067237 conn(0x7f15cc06d110 msgr2=0x7f15cc0cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:48.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.175+0000 7f15d1f9d700 1 -- 172.21.15.31:0/3930067237 shutdown_connections 2023-10-06T02:27:48.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.175+0000 7f15d1f9d700 1 -- 172.21.15.31:0/3930067237 wait complete. 2023-10-06T02:27:48.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.176+0000 7f15d1f9d700 1 Processor -- start 2023-10-06T02:27:48.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.176+0000 7f15d1f9d700 1 -- start start 2023-10-06T02:27:48.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.177+0000 7f15d1f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f15bc004b70 0x7f15cc143ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:48.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.177+0000 7f15d1f9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15cc14b630 0x7f15cc14baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:48.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.177+0000 7f15d1f9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15cc158460 0x7f15cc1588d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:48.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.177+0000 7f15d1f9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15cc158e10 con 0x7f15cc14b630 2023-10-06T02:27:48.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.177+0000 7f15d1f9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15cc0af8d0 con 0x7f15cc158460 2023-10-06T02:27:48.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.177+0000 7f15d1f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15cc14c0d0 con 0x7f15bc004b70 2023-10-06T02:27:48.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.177+0000 7f15cbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15cc14b630 0x7f15cc14baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:48.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.178+0000 7f15cbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15cc14b630 0x7f15cc14baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41464/0 (socket says 172.21.15.31:41464) 2023-10-06T02:27:48.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.178+0000 7f15cbfff700 1 -- 172.21.15.31:0/1277136546 learned_addr learned my addr 172.21.15.31:0/1277136546 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:48.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.178+0000 7f15d0f9b700 1 --2- 172.21.15.31:0/1277136546 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f15bc004b70 0x7f15cc143ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:48.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.178+0000 7f15d179c700 1 --2- 172.21.15.31:0/1277136546 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15cc158460 0x7f15cc1588d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:48.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.178+0000 7f15cbfff700 1 -- 172.21.15.31:0/1277136546 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f15bc004b70 msgr2=0x7f15cc143ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:48.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.178+0000 7f15cbfff700 1 --2- 172.21.15.31:0/1277136546 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f15bc004b70 0x7f15cc143ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:48.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.178+0000 7f15cbfff700 1 -- 172.21.15.31:0/1277136546 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15cc158460 msgr2=0x7f15cc1588d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:48.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.178+0000 7f15cbfff700 1 --2- 172.21.15.31:0/1277136546 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15cc158460 0x7f15cc1588d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:48.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.178+0000 7f15cbfff700 1 -- 172.21.15.31:0/1277136546 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15cc0af280 con 0x7f15cc14b630 2023-10-06T02:27:48.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.178+0000 7f15cbfff700 1 --2- 172.21.15.31:0/1277136546 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15cc14b630 0x7f15cc14baa0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f15b800eb90 tx=0x7f15b8001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:48.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.178+0000 7f15c8ff9700 1 -- 172.21.15.31:0/1277136546 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b800eea0 con 0x7f15cc14b630 2023-10-06T02:27:48.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.178+0000 7f15c8ff9700 1 -- 172.21.15.31:0/1277136546 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15b800c760 con 0x7f15cc14b630 2023-10-06T02:27:48.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.178+0000 7f15d1f9d700 1 -- 172.21.15.31:0/1277136546 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15cc0af560 con 0x7f15cc14b630 2023-10-06T02:27:48.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.179+0000 7f15c8ff9700 1 -- 172.21.15.31:0/1277136546 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b800cae0 con 0x7f15cc14b630 2023-10-06T02:27:48.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.179+0000 7f15d1f9d700 1 -- 172.21.15.31:0/1277136546 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15cc1492f0 con 0x7f15cc14b630 2023-10-06T02:27:48.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.180+0000 7f15d1f9d700 1 -- 172.21.15.31:0/1277136546 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15cc0af0e0 con 0x7f15cc14b630 2023-10-06T02:27:48.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.180+0000 7f15c8ff9700 1 -- 172.21.15.31:0/1277136546 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f15b801d460 con 0x7f15cc14b630 2023-10-06T02:27:48.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.181+0000 7f15c8ff9700 1 --2- 172.21.15.31:0/1277136546 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f15b0067050 0x7f15b0069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:48.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.181+0000 7f15c8ff9700 1 -- 172.21.15.31:0/1277136546 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5851+0+0 (secure 0 0 0) 0x7f15b80847a0 con 0x7f15cc14b630 2023-10-06T02:27:48.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.182+0000 7f15d0f9b700 1 --2- 172.21.15.31:0/1277136546 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f15b0067050 0x7f15b0069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:48.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.182+0000 7f15d0f9b700 1 --2- 172.21.15.31:0/1277136546 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f15b0067050 0x7f15b0069500 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f15c00091e0 tx=0x7f15c001a000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:48.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.185+0000 7f15c8ff9700 1 -- 172.21.15.31:0/1277136546 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f15cc0af0e0 con 0x7f15cc14b630 2023-10-06T02:27:48.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.389+0000 7f15d1f9d700 1 -- 172.21.15.31:0/1277136546 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f15cc0af0e0 con 0x7f15cc14b630 2023-10-06T02:27:48.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.393+0000 7f15c8ff9700 1 -- 172.21.15.31:0/1277136546 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7f15cc0af0e0 con 0x7f15cc14b630 2023-10-06T02:27:48.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.395+0000 7f15d1f9d700 1 -- 172.21.15.31:0/1277136546 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f15b0067050 msgr2=0x7f15b0069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:48.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.395+0000 7f15d1f9d700 1 --2- 172.21.15.31:0/1277136546 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f15b0067050 0x7f15b0069500 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f15c00091e0 tx=0x7f15c001a000 comp rx=0 tx=0).stop 2023-10-06T02:27:48.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.395+0000 7f15d1f9d700 1 -- 172.21.15.31:0/1277136546 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15cc14b630 msgr2=0x7f15cc14baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:48.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.395+0000 7f15d1f9d700 1 --2- 172.21.15.31:0/1277136546 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15cc14b630 0x7f15cc14baa0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f15b800eb90 tx=0x7f15b8001e90 comp rx=0 tx=0).stop 2023-10-06T02:27:48.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.395+0000 7f15d1f9d700 1 -- 172.21.15.31:0/1277136546 shutdown_connections 2023-10-06T02:27:48.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.395+0000 7f15d1f9d700 1 --2- 172.21.15.31:0/1277136546 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15cc158460 0x7f15cc1588d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:48.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.395+0000 7f15d1f9d700 1 --2- 172.21.15.31:0/1277136546 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f15b0067050 0x7f15b0069500 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:48.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.395+0000 7f15d1f9d700 1 --2- 172.21.15.31:0/1277136546 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f15bc004b70 0x7f15cc143ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:48.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.395+0000 7f15d1f9d700 1 --2- 172.21.15.31:0/1277136546 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15cc14b630 0x7f15cc14baa0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:48.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.395+0000 7f15d1f9d700 1 -- 172.21.15.31:0/1277136546 >> 172.21.15.31:0/1277136546 conn(0x7f15cc06d110 msgr2=0x7f15cc0c21f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:48.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.395+0000 7f15d1f9d700 1 -- 172.21.15.31:0/1277136546 shutdown_connections 2023-10-06T02:27:48.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.395+0000 7f15d1f9d700 1 -- 172.21.15.31:0/1277136546 wait complete. 2023-10-06T02:27:48.413 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction' 2023-10-06T02:27:48.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.600+0000 7fbd13fff700 1 Processor -- start 2023-10-06T02:27:48.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.601+0000 7fbd13fff700 1 -- start start 2023-10-06T02:27:48.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.602+0000 7fbd13fff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fbd1415dfd0 0x7fbd140aedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:48.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.602+0000 7fbd13fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fbd14159920 0x7fbd140ad090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:48.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.602+0000 7fbd13fff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd140acc00 con 0x7fbd14159550 2023-10-06T02:27:48.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.602+0000 7fbd13fff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd140abff0 con 0x7fbd14159920 2023-10-06T02:27:48.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.602+0000 7fbd13fff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd140ac2d0 con 0x7fbd1415dfd0 2023-10-06T02:27:48.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.602+0000 7fbd12ffd700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fbd14159550 0x7fbd140ceac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:59452/0 (socket says 172.21.15.31:59452) 2023-10-06T02:27:48.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.602+0000 7fbd12ffd700 1 -- 172.21.15.31:0/3770251627 learned_addr learned my addr 172.21.15.31:0/3770251627 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:27:48.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.602+0000 7fbd137fe700 1 --2- 172.21.15.31:0/3770251627 >> v2:172.21.15.31:3300/0 conn(0x7fbd14159920 0x7fbd140ad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:48.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.602+0000 7fbd127fc700 1 --2- 172.21.15.31:0/3770251627 >> v2:172.21.15.62:3300/0 conn(0x7fbd1415dfd0 0x7fbd140aedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:48.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.603+0000 7fbd11ffb700 1 -- 172.21.15.31:0/3770251627 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3480784382 0 0) 0x7fbd140acc00 con 0x7fbd14159550 2023-10-06T02:27:48.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.603+0000 7fbd11ffb700 1 -- 172.21.15.31:0/3770251627 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbcfc003570 con 0x7fbd14159550 2023-10-06T02:27:48.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.603+0000 7fbd137fe700 1 -- 172.21.15.31:0/3770251627 >> v2:172.21.15.62:3300/0 conn(0x7fbd1415dfd0 msgr2=0x7fbd140aedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:48.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.603+0000 7fbd137fe700 1 --2- 172.21.15.31:0/3770251627 >> v2:172.21.15.62:3300/0 conn(0x7fbd1415dfd0 0x7fbd140aedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:48.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.603+0000 7fbd137fe700 1 -- 172.21.15.31:0/3770251627 >> v1:172.21.15.31:6789/0 conn(0x7fbd14159550 legacy=0x7fbd140ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:48.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.603+0000 7fbd137fe700 1 -- 172.21.15.31:0/3770251627 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd140ac5b0 con 0x7fbd14159920 2023-10-06T02:27:48.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.603+0000 7fbd137fe700 1 --2- 172.21.15.31:0/3770251627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd14159920 0x7fbd140ad090 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fbd1406c180 tx=0x7fbd0c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31b732f61e095367 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:48.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.603+0000 7fbd11ffb700 1 -- 172.21.15.31:0/3770251627 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd0c00e070 con 0x7fbd14159920 2023-10-06T02:27:48.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.603+0000 7fbd11ffb700 1 -- 172.21.15.31:0/3770251627 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd0c00bab0 con 0x7fbd14159920 2023-10-06T02:27:48.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.603+0000 7fbd11ffb700 1 -- 172.21.15.31:0/3770251627 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd0c00be70 con 0x7fbd14159920 2023-10-06T02:27:48.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.604+0000 7fbd13fff700 1 -- 172.21.15.31:0/3770251627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd14159920 msgr2=0x7fbd140ad090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:48.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.604+0000 7fbd13fff700 1 --2- 172.21.15.31:0/3770251627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd14159920 0x7fbd140ad090 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fbd1406c180 tx=0x7fbd0c009780 comp rx=0 tx=0).stop 2023-10-06T02:27:48.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.604+0000 7fbd13fff700 1 -- 172.21.15.31:0/3770251627 shutdown_connections 2023-10-06T02:27:48.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.604+0000 7fbd13fff700 1 --2- 172.21.15.31:0/3770251627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd14159920 0x7fbd140ad090 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:48.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.604+0000 7fbd13fff700 1 --2- 172.21.15.31:0/3770251627 >> v2:172.21.15.62:3300/0 conn(0x7fbd1415dfd0 0x7fbd140aedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:48.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.604+0000 7fbd13fff700 1 -- 172.21.15.31:0/3770251627 >> 172.21.15.31:0/3770251627 conn(0x7fbd1406ce80 msgr2=0x7fbd140ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:48.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.604+0000 7fbd13fff700 1 -- 172.21.15.31:0/3770251627 shutdown_connections 2023-10-06T02:27:48.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.604+0000 7fbd13fff700 1 -- 172.21.15.31:0/3770251627 wait complete. 2023-10-06T02:27:48.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.605+0000 7fbd13fff700 1 Processor -- start 2023-10-06T02:27:48.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.606+0000 7fbd13fff700 1 -- start start 2023-10-06T02:27:48.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.606+0000 7fbd13fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd14159550 0x7fbd14154d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:48.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.606+0000 7fbd13fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbd14159920 0x7fbd140c8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:48.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.606+0000 7fbd13fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd1415dfd0 0x7fbd141559a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:48.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.606+0000 7fbd13fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd140c98b0 con 0x7fbd1415dfd0 2023-10-06T02:27:48.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.606+0000 7fbd13fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd140caec0 con 0x7fbd14159550 2023-10-06T02:27:48.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.606+0000 7fbd13fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd14157a00 con 0x7fbd14159920 2023-10-06T02:27:48.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.607+0000 7fbd137fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd1415dfd0 0x7fbd141559a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:48.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.607+0000 7fbd137fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd1415dfd0 0x7fbd141559a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41476/0 (socket says 172.21.15.31:41476) 2023-10-06T02:27:48.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.607+0000 7fbd137fe700 1 -- 172.21.15.31:0/2291847307 learned_addr learned my addr 172.21.15.31:0/2291847307 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:48.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.607+0000 7fbd12ffd700 1 --2- 172.21.15.31:0/2291847307 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd14159550 0x7fbd14154d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:48.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.607+0000 7fbd127fc700 1 --2- 172.21.15.31:0/2291847307 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbd14159920 0x7fbd140c8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:48.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.607+0000 7fbd137fe700 1 -- 172.21.15.31:0/2291847307 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbd14159920 msgr2=0x7fbd140c8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:48.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.607+0000 7fbd137fe700 1 --2- 172.21.15.31:0/2291847307 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbd14159920 0x7fbd140c8ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:48.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.607+0000 7fbd137fe700 1 -- 172.21.15.31:0/2291847307 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd14159550 msgr2=0x7fbd14154d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:48.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.607+0000 7fbd137fe700 1 --2- 172.21.15.31:0/2291847307 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd14159550 0x7fbd14154d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:48.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.607+0000 7fbd137fe700 1 -- 172.21.15.31:0/2291847307 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd0c009a20 con 0x7fbd1415dfd0 2023-10-06T02:27:48.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.607+0000 7fbd137fe700 1 --2- 172.21.15.31:0/2291847307 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd1415dfd0 0x7fbd141559a0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fbd1406ad70 tx=0x7fbd0c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:48.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.608+0000 7fbcfaffd700 1 -- 172.21.15.31:0/2291847307 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd0c012dd0 con 0x7fbd1415dfd0 2023-10-06T02:27:48.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.608+0000 7fbcfaffd700 1 -- 172.21.15.31:0/2291847307 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd0c027470 con 0x7fbd1415dfd0 2023-10-06T02:27:48.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.608+0000 7fbd13fff700 1 -- 172.21.15.31:0/2291847307 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd141570d0 con 0x7fbd1415dfd0 2023-10-06T02:27:48.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.608+0000 7fbcfaffd700 1 -- 172.21.15.31:0/2291847307 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd0c027830 con 0x7fbd1415dfd0 2023-10-06T02:27:48.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.608+0000 7fbd13fff700 1 -- 172.21.15.31:0/2291847307 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd14157690 con 0x7fbd1415dfd0 2023-10-06T02:27:48.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.609+0000 7fbd13fff700 1 -- 172.21.15.31:0/2291847307 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd141697a0 con 0x7fbd1415dfd0 2023-10-06T02:27:48.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.610+0000 7fbcfaffd700 1 -- 172.21.15.31:0/2291847307 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7fbd0c024070 con 0x7fbd1415dfd0 2023-10-06T02:27:48.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.611+0000 7fbcfaffd700 1 --2- 172.21.15.31:0/2291847307 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fbd00067050 0x7fbd00069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:48.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.611+0000 7fbcfaffd700 1 -- 172.21.15.31:0/2291847307 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5851+0+0 (secure 0 0 0) 0x7fbd0c087210 con 0x7fbd1415dfd0 2023-10-06T02:27:48.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.611+0000 7fbd12ffd700 1 --2- 172.21.15.31:0/2291847307 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fbd00067050 0x7fbd00069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:48.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.612+0000 7fbd12ffd700 1 --2- 172.21.15.31:0/2291847307 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fbd00067050 0x7fbd00069500 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fbd04007fd0 tx=0x7fbd04009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:48.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.615+0000 7fbcfaffd700 1 -- 172.21.15.31:0/2291847307 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbd141697a0 con 0x7fbd1415dfd0 2023-10-06T02:27:48.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.897+0000 7fbd13fff700 1 -- 172.21.15.31:0/2291847307 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction"]} v 0) v1 -- 0x7fbd141697a0 con 0x7fbd1415dfd0 2023-10-06T02:27:48.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.948+0000 7fbcfaffd700 1 -- 172.21.15.31:0/2291847307 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction"]}]=0 v265) v1 ==== 131+0+0 (secure 0 0 0) 0x7fbd141697a0 con 0x7fbd1415dfd0 2023-10-06T02:27:48.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.951+0000 7fbd13fff700 1 -- 172.21.15.31:0/2291847307 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fbd00067050 msgr2=0x7fbd00069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:48.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.951+0000 7fbd13fff700 1 --2- 172.21.15.31:0/2291847307 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fbd00067050 0x7fbd00069500 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fbd04007fd0 tx=0x7fbd04009040 comp rx=0 tx=0).stop 2023-10-06T02:27:48.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.951+0000 7fbd13fff700 1 -- 172.21.15.31:0/2291847307 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd1415dfd0 msgr2=0x7fbd141559a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:48.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.951+0000 7fbd13fff700 1 --2- 172.21.15.31:0/2291847307 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd1415dfd0 0x7fbd141559a0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fbd1406ad70 tx=0x7fbd0c019000 comp rx=0 tx=0).stop 2023-10-06T02:27:48.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.951+0000 7fbd13fff700 1 -- 172.21.15.31:0/2291847307 shutdown_connections 2023-10-06T02:27:48.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.951+0000 7fbd13fff700 1 --2- 172.21.15.31:0/2291847307 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd1415dfd0 0x7fbd141559a0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:48.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.951+0000 7fbd13fff700 1 --2- 172.21.15.31:0/2291847307 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd14159550 0x7fbd14154d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:48.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.951+0000 7fbd13fff700 1 --2- 172.21.15.31:0/2291847307 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fbd00067050 0x7fbd00069500 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:48.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.951+0000 7fbd13fff700 1 --2- 172.21.15.31:0/2291847307 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbd14159920 0x7fbd140c8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:48.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.951+0000 7fbd13fff700 1 -- 172.21.15.31:0/2291847307 >> 172.21.15.31:0/2291847307 conn(0x7fbd1406ce80 msgr2=0x7fbd140b35f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:48.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.951+0000 7fbd13fff700 1 -- 172.21.15.31:0/2291847307 shutdown_connections 2023-10-06T02:27:48.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:48.951+0000 7fbd13fff700 1 -- 172.21.15.31:0/2291847307 wait complete. 2023-10-06T02:27:48.969 INFO:tasks.cephfs_test_runner:test_mgr_eviction (tasks.cephfs.test_volumes.TestMisc) ... ok 2023-10-06T02:27:49.028 INFO:tasks.cephfs_test_runner:Starting test: test_names_can_only_be_goodchars (tasks.cephfs.test_volumes.TestMisc) Test the creating vols, subvols subvolgroups fails when their names uses 2023-10-06T02:27:49.029 INFO:tasks.cephfs_test_runner:test_names_can_only_be_goodchars (tasks.cephfs.test_volumes.TestMisc) 2023-10-06T02:27:49.029 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars' 2023-10-06T02:27:49.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.217+0000 7ffbb759d700 1 Processor -- start 2023-10-06T02:27:49.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.217+0000 7ffbb759d700 1 -- start start 2023-10-06T02:27:49.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.218+0000 7ffbb759d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ffbb0153c30 0x7ffbb015a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:49.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.218+0000 7ffbb759d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbb015afd0 con 0x7ffbb015de50 2023-10-06T02:27:49.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.218+0000 7ffbb759d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbb015b2b0 con 0x7ffbb00cf110 2023-10-06T02:27:49.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.218+0000 7ffbb759d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbb015b590 con 0x7ffbb0153c30 2023-10-06T02:27:49.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.218+0000 7ffbb5d9a700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7ffbb015de50 0x7ffbb00cea00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:59460/0 (socket says 172.21.15.31:59460) 2023-10-06T02:27:49.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.218+0000 7ffbb5d9a700 1 -- 172.21.15.31:0/3560475263 learned_addr learned my addr 172.21.15.31:0/3560475263 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:27:49.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.219+0000 7ffbb659b700 1 --2- 172.21.15.31:0/3560475263 >> v2:172.21.15.62:3300/0 conn(0x7ffbb0153c30 0x7ffbb015a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:49.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.219+0000 7ffbb5599700 1 -- 172.21.15.31:0/3560475263 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1460528917 0 0) 0x7ffbb015afd0 con 0x7ffbb015de50 2023-10-06T02:27:49.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.219+0000 7ffbb5599700 1 -- 172.21.15.31:0/3560475263 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb9c003570 con 0x7ffbb015de50 2023-10-06T02:27:49.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.219+0000 7ffbb5599700 1 -- 172.21.15.31:0/3560475263 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2368874551 0 0) 0x7ffb9c003570 con 0x7ffbb015de50 2023-10-06T02:27:49.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.219+0000 7ffbb5599700 1 -- 172.21.15.31:0/3560475263 >> v2:172.21.15.62:3300/0 conn(0x7ffbb0153c30 msgr2=0x7ffbb015a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:49.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.219+0000 7ffbb5599700 1 --2- 172.21.15.31:0/3560475263 >> v2:172.21.15.62:3300/0 conn(0x7ffbb0153c30 0x7ffbb015a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:49.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.219+0000 7ffbb5599700 1 -- 172.21.15.31:0/3560475263 >> v1:172.21.15.62:6789/0 conn(0x7ffbb00cf110 legacy=0x7ffbb015c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:49.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.219+0000 7ffbb5599700 1 -- 172.21.15.31:0/3560475263 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbb015b870 con 0x7ffbb015de50 2023-10-06T02:27:49.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.219+0000 7ffbb5599700 1 -- 172.21.15.31:0/3560475263 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7ffba0001470 con 0x7ffbb015de50 2023-10-06T02:27:49.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.219+0000 7ffbb5599700 1 -- 172.21.15.31:0/3560475263 >> v1:172.21.15.31:6789/0 conn(0x7ffbb015de50 legacy=0x7ffbb00cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:49.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.219+0000 7ffbb5599700 1 --2- 172.21.15.31:0/3560475263 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb9c003a60 0x7ffb9c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:49.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.219+0000 7ffbb5599700 1 --2- 172.21.15.31:0/3560475263 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb9c004810 0x7ffb9c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:49.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.219+0000 7ffbb5599700 1 --2- 172.21.15.31:0/3560475263 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffb9c0051c0 0x7ffb9c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:49.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.219+0000 7ffbb5599700 1 -- 172.21.15.31:0/3560475263 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb015b590 con 0x7ffb9c004810 2023-10-06T02:27:49.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.219+0000 7ffbb5599700 1 -- 172.21.15.31:0/3560475263 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9c00c0d0 con 0x7ffb9c003a60 2023-10-06T02:27:49.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.219+0000 7ffbb5599700 1 -- 172.21.15.31:0/3560475263 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9c00c290 con 0x7ffb9c0051c0 2023-10-06T02:27:49.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.220+0000 7ffbb5d9a700 1 --2- 172.21.15.31:0/3560475263 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb9c004810 0x7ffb9c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:49.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.220+0000 7ffbb659b700 1 --2- 172.21.15.31:0/3560475263 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb9c003a60 0x7ffb9c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:49.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.220+0000 7ffbb6d9c700 1 --2- 172.21.15.31:0/3560475263 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffb9c0051c0 0x7ffb9c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:49.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.220+0000 7ffbb5d9a700 1 -- 172.21.15.31:0/3560475263 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffb9c0051c0 msgr2=0x7ffb9c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:49.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.220+0000 7ffbb5d9a700 1 --2- 172.21.15.31:0/3560475263 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffb9c0051c0 0x7ffb9c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:49.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.220+0000 7ffbb5d9a700 1 -- 172.21.15.31:0/3560475263 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb9c003a60 msgr2=0x7ffb9c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:49.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.220+0000 7ffbb5d9a700 1 --2- 172.21.15.31:0/3560475263 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb9c003a60 0x7ffb9c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:49.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.220+0000 7ffbb5d9a700 1 -- 172.21.15.31:0/3560475263 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb9c003570 con 0x7ffb9c004810 2023-10-06T02:27:49.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.220+0000 7ffbb6d9c700 1 -- 172.21.15.31:0/3560475263 reap_dead start 2023-10-06T02:27:49.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.220+0000 7ffbb5d9a700 1 --2- 172.21.15.31:0/3560475263 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb9c004810 0x7ffb9c004c80 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7ffba000c390 tx=0x7ffbb015e2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3722218ae7088f08 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:49.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.221+0000 7ffbb5599700 1 -- 172.21.15.31:0/3560475263 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba000cc60 con 0x7ffb9c004810 2023-10-06T02:27:49.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.221+0000 7ffbb5599700 1 -- 172.21.15.31:0/3560475263 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffba000cdc0 con 0x7ffb9c004810 2023-10-06T02:27:49.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.221+0000 7ffbb5599700 1 -- 172.21.15.31:0/3560475263 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba0005b70 con 0x7ffb9c004810 2023-10-06T02:27:49.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.221+0000 7ffbb759d700 1 -- 172.21.15.31:0/3560475263 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb9c004810 msgr2=0x7ffb9c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:49.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.221+0000 7ffbb759d700 1 --2- 172.21.15.31:0/3560475263 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb9c004810 0x7ffb9c004c80 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7ffba000c390 tx=0x7ffbb015e2d0 comp rx=0 tx=0).stop 2023-10-06T02:27:49.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.221+0000 7ffbb759d700 1 -- 172.21.15.31:0/3560475263 shutdown_connections 2023-10-06T02:27:49.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.221+0000 7ffbb759d700 1 --2- 172.21.15.31:0/3560475263 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb9c004810 0x7ffb9c004c80 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:49.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.221+0000 7ffbb759d700 1 -- 172.21.15.31:0/3560475263 >> 172.21.15.31:0/3560475263 conn(0x7ffbb00703a0 msgr2=0x7ffbb00707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:49.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.221+0000 7ffbb759d700 1 -- 172.21.15.31:0/3560475263 shutdown_connections 2023-10-06T02:27:49.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.221+0000 7ffbb759d700 1 -- 172.21.15.31:0/3560475263 wait complete. 2023-10-06T02:27:49.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.223+0000 7ffbb759d700 1 Processor -- start 2023-10-06T02:27:49.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.223+0000 7ffbb759d700 1 -- start start 2023-10-06T02:27:49.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.224+0000 7ffbb759d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffb9c004810 0x7ffbb0169b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:49.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.224+0000 7ffbb759d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffbb00b3c10 0x7ffbb00b32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:49.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.224+0000 7ffbb759d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbb00b5b00 0x7ffbb00b15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:49.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.224+0000 7ffbb759d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb016a090 con 0x7ffbb00b5b00 2023-10-06T02:27:49.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.224+0000 7ffbb759d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb00b08a0 con 0x7ffbb00b3c10 2023-10-06T02:27:49.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.224+0000 7ffbb759d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb00b1140 con 0x7ffb9c004810 2023-10-06T02:27:49.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.224+0000 7ffbb6d9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbb00b5b00 0x7ffbb00b15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:49.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.224+0000 7ffbb6d9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbb00b5b00 0x7ffbb00b15d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41486/0 (socket says 172.21.15.31:41486) 2023-10-06T02:27:49.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.224+0000 7ffbb6d9c700 1 -- 172.21.15.31:0/310472317 learned_addr learned my addr 172.21.15.31:0/310472317 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:49.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.224+0000 7ffbb659b700 1 --2- 172.21.15.31:0/310472317 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffb9c004810 0x7ffbb0169b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:49.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.224+0000 7ffbb6d9c700 1 -- 172.21.15.31:0/310472317 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffb9c004810 msgr2=0x7ffbb0169b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:49.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.224+0000 7ffbb5d9a700 1 --2- 172.21.15.31:0/310472317 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffbb00b3c10 0x7ffbb00b32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:49.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.224+0000 7ffbb6d9c700 1 --2- 172.21.15.31:0/310472317 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffb9c004810 0x7ffbb0169b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:49.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.224+0000 7ffbb6d9c700 1 -- 172.21.15.31:0/310472317 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffbb00b3c10 msgr2=0x7ffbb00b32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:49.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.224+0000 7ffbb6d9c700 1 --2- 172.21.15.31:0/310472317 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffbb00b3c10 0x7ffbb00b32d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:49.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.224+0000 7ffbb6d9c700 1 -- 172.21.15.31:0/310472317 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbb00b4110 con 0x7ffbb00b5b00 2023-10-06T02:27:49.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.224+0000 7ffbb659b700 1 --2- 172.21.15.31:0/310472317 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffb9c004810 0x7ffbb0169b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:27:49.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.225+0000 7ffbb6d9c700 1 --2- 172.21.15.31:0/310472317 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbb00b5b00 0x7ffbb00b15d0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7ffbb0071370 tx=0x7ffba8001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:49.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.225+0000 7ffba67fc700 1 -- 172.21.15.31:0/310472317 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba8006db0 con 0x7ffbb00b5b00 2023-10-06T02:27:49.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.225+0000 7ffba67fc700 1 -- 172.21.15.31:0/310472317 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffba800c5a0 con 0x7ffbb00b5b00 2023-10-06T02:27:49.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.225+0000 7ffbb759d700 1 -- 172.21.15.31:0/310472317 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbb00b0af0 con 0x7ffbb00b5b00 2023-10-06T02:27:49.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.225+0000 7ffba67fc700 1 -- 172.21.15.31:0/310472317 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba800c960 con 0x7ffbb00b5b00 2023-10-06T02:27:49.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.225+0000 7ffbb759d700 1 -- 172.21.15.31:0/310472317 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbb00b3810 con 0x7ffbb00b5b00 2023-10-06T02:27:49.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.226+0000 7ffbb759d700 1 -- 172.21.15.31:0/310472317 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbb00b12b0 con 0x7ffbb00b5b00 2023-10-06T02:27:49.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.227+0000 7ffba67fc700 1 -- 172.21.15.31:0/310472317 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7ffba800cd10 con 0x7ffbb00b5b00 2023-10-06T02:27:49.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.228+0000 7ffba67fc700 1 --2- 172.21.15.31:0/310472317 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ffb94067050 0x7ffb94069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:49.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.228+0000 7ffba67fc700 1 -- 172.21.15.31:0/310472317 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5851+0+0 (secure 0 0 0) 0x7ffba8084680 con 0x7ffbb00b5b00 2023-10-06T02:27:49.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.228+0000 7ffbb659b700 1 --2- 172.21.15.31:0/310472317 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ffb94067050 0x7ffb94069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:49.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.229+0000 7ffbb659b700 1 --2- 172.21.15.31:0/310472317 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ffb94067050 0x7ffb94069500 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ffbac0093f0 tx=0x7ffbac008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:49.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.232+0000 7ffba67fc700 1 -- 172.21.15.31:0/310472317 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffba80516b0 con 0x7ffbb00b5b00 2023-10-06T02:27:49.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.519+0000 7ffbb759d700 1 -- 172.21.15.31:0/310472317 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars"]} v 0) v1 -- 0x7ffbb004c3c0 con 0x7ffbb00b5b00 2023-10-06T02:27:50.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.950+0000 7ffba67fc700 1 -- 172.21.15.31:0/310472317 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars"]}]=0 v266) v1 ==== 149+0+0 (secure 0 0 0) 0x7ffba8054cd0 con 0x7ffbb00b5b00 2023-10-06T02:27:50.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.953+0000 7ffbb759d700 1 -- 172.21.15.31:0/310472317 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ffb94067050 msgr2=0x7ffb94069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:50.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.953+0000 7ffbb759d700 1 --2- 172.21.15.31:0/310472317 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ffb94067050 0x7ffb94069500 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ffbac0093f0 tx=0x7ffbac008040 comp rx=0 tx=0).stop 2023-10-06T02:27:50.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.953+0000 7ffbb759d700 1 -- 172.21.15.31:0/310472317 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbb00b5b00 msgr2=0x7ffbb00b15d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:50.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.953+0000 7ffbb759d700 1 --2- 172.21.15.31:0/310472317 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbb00b5b00 0x7ffbb00b15d0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7ffbb0071370 tx=0x7ffba8001640 comp rx=0 tx=0).stop 2023-10-06T02:27:50.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.953+0000 7ffbb759d700 1 -- 172.21.15.31:0/310472317 shutdown_connections 2023-10-06T02:27:50.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.953+0000 7ffbb759d700 1 --2- 172.21.15.31:0/310472317 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbb00b5b00 0x7ffbb00b15d0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:50.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.953+0000 7ffbb759d700 1 --2- 172.21.15.31:0/310472317 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ffb94067050 0x7ffb94069500 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:50.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.953+0000 7ffbb759d700 1 --2- 172.21.15.31:0/310472317 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffb9c004810 0x7ffbb0169b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:50.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.953+0000 7ffbb759d700 1 --2- 172.21.15.31:0/310472317 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffbb00b3c10 0x7ffbb00b32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:50.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.953+0000 7ffbb759d700 1 -- 172.21.15.31:0/310472317 >> 172.21.15.31:0/310472317 conn(0x7ffbb00703a0 msgr2=0x7ffbb00c63a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:50.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.953+0000 7ffbb759d700 1 -- 172.21.15.31:0/310472317 shutdown_connections 2023-10-06T02:27:50.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:49.953+0000 7ffbb759d700 1 -- 172.21.15.31:0/310472317 wait complete. 2023-10-06T02:27:50.090 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T02:27:50.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:50.287+0000 7fb43b59d700 1 Processor -- start 2023-10-06T02:27:50.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:50.288+0000 7fb43b59d700 1 -- start start 2023-10-06T02:27:52.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:50.289+0000 7fb43b59d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb4340c7be0 0x7fb43406abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:52.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:50.289+0000 7fb43b59d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb4340c7fb0 0x7fb43415a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:52.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:50.289+0000 7fb43b59d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb43413f780 0x7fb43413fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:52.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:50.289+0000 7fb43b59d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb43406bf10 con 0x7fb4340c7be0 2023-10-06T02:27:52.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:50.289+0000 7fb43b59d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4341400e0 con 0x7fb4340c7fb0 2023-10-06T02:27:52.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:50.289+0000 7fb43b59d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb43415aed0 con 0x7fb43413f780 2023-10-06T02:27:52.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:50.289+0000 7fb43a59b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb4340c7be0 0x7fb43406abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:52.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:50.289+0000 7fb43a59b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb4340c7be0 0x7fb43406abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41500/0 (socket says 172.21.15.31:41500) 2023-10-06T02:27:52.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:50.289+0000 7fb43a59b700 1 -- 172.21.15.31:0/1560549425 learned_addr learned my addr 172.21.15.31:0/1560549425 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:52.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:50.289+0000 7fb439d9a700 1 --2- 172.21.15.31:0/1560549425 >> v2:172.21.15.62:3300/0 conn(0x7fb4340c7fb0 0x7fb43415a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:52.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:50.289+0000 7fb43ad9c700 1 --2- 172.21.15.31:0/1560549425 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb43413f780 0x7fb43413fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:52.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:50.290+0000 7fb43a59b700 1 -- 172.21.15.31:0/1560549425 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb43413f780 msgr2=0x7fb43413fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:52.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:50.290+0000 7fb43a59b700 1 --2- 172.21.15.31:0/1560549425 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb43413f780 0x7fb43413fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:52.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:50.290+0000 7fb43a59b700 1 -- 172.21.15.31:0/1560549425 >> v2:172.21.15.62:3300/0 conn(0x7fb4340c7fb0 msgr2=0x7fb43415a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:52.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:50.290+0000 7fb43a59b700 1 --2- 172.21.15.31:0/1560549425 >> v2:172.21.15.62:3300/0 conn(0x7fb4340c7fb0 0x7fb43415a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:52.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:50.290+0000 7fb43a59b700 1 -- 172.21.15.31:0/1560549425 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb43406b290 con 0x7fb4340c7be0 2023-10-06T02:27:52.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:50.290+0000 7fb43a59b700 1 --2- 172.21.15.31:0/1560549425 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4340c7be0 0x7fb43406abd0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fb4340aec50 tx=0x7fb43000b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d20f65a2999a052 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:52.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:50.290+0000 7fb439599700 1 -- 172.21.15.31:0/1560549425 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb43000dc50 con 0x7fb4340c7be0 2023-10-06T02:27:52.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:50.290+0000 7fb439599700 1 -- 172.21.15.31:0/1560549425 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb43000ddb0 con 0x7fb4340c7be0 2023-10-06T02:27:52.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:50.290+0000 7fb439599700 1 -- 172.21.15.31:0/1560549425 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4300196d0 con 0x7fb4340c7be0 2023-10-06T02:27:52.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:50.291+0000 7fb43b59d700 1 -- 172.21.15.31:0/1560549425 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4340c7be0 msgr2=0x7fb43406abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:52.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:50.291+0000 7fb43b59d700 1 --2- 172.21.15.31:0/1560549425 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4340c7be0 0x7fb43406abd0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fb4340aec50 tx=0x7fb43000b4f0 comp rx=0 tx=0).stop 2023-10-06T02:27:52.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:50.291+0000 7fb43b59d700 1 -- 172.21.15.31:0/1560549425 shutdown_connections 2023-10-06T02:27:52.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:50.291+0000 7fb43b59d700 1 --2- 172.21.15.31:0/1560549425 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb43413f780 0x7fb43413fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:52.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:50.291+0000 7fb43b59d700 1 --2- 172.21.15.31:0/1560549425 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4340c7be0 0x7fb43406abd0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:52.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:50.291+0000 7fb43b59d700 1 --2- 172.21.15.31:0/1560549425 >> v2:172.21.15.62:3300/0 conn(0x7fb4340c7fb0 0x7fb43415a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:52.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:50.291+0000 7fb43b59d700 1 -- 172.21.15.31:0/1560549425 >> 172.21.15.31:0/1560549425 conn(0x7fb43406d110 msgr2=0x7fb4340cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:52.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:50.291+0000 7fb43b59d700 1 -- 172.21.15.31:0/1560549425 shutdown_connections 2023-10-06T02:27:52.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:50.291+0000 7fb43b59d700 1 -- 172.21.15.31:0/1560549425 wait complete. 2023-10-06T02:27:52.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:50.292+0000 7fb43b59d700 1 Processor -- start 2023-10-06T02:27:52.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:50.292+0000 7fb43b59d700 1 -- start start 2023-10-06T02:27:52.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:50.293+0000 7fb43b59d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4340c7be0 0x7fb434157420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:52.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:50.293+0000 7fb43b59d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb4340c7fb0 0x7fb43413c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:52.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:50.293+0000 7fb43b59d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb43413f780 0x7fb4341590d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:52.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:50.293+0000 7fb43b59d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4340c4160 con 0x7fb4340c7be0 2023-10-06T02:27:52.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:50.293+0000 7fb43b59d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb434157a30 con 0x7fb43413f780 2023-10-06T02:27:52.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:50.293+0000 7fb43b59d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4340c43b0 con 0x7fb4340c7fb0 2023-10-06T02:27:52.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:50.294+0000 7fb43a59b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4340c7be0 0x7fb434157420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:52.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:50.294+0000 7fb43a59b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4340c7be0 0x7fb434157420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41514/0 (socket says 172.21.15.31:41514) 2023-10-06T02:27:52.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:50.294+0000 7fb43a59b700 1 -- 172.21.15.31:0/3842195253 learned_addr learned my addr 172.21.15.31:0/3842195253 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:52.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:50.294+0000 7fb43ad9c700 1 --2- 172.21.15.31:0/3842195253 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb43413f780 0x7fb4341590d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:52.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:50.294+0000 7fb439d9a700 1 --2- 172.21.15.31:0/3842195253 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb4340c7fb0 0x7fb43413c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:52.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:50.294+0000 7fb43a59b700 1 -- 172.21.15.31:0/3842195253 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb4340c7fb0 msgr2=0x7fb43413c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:52.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:50.294+0000 7fb43a59b700 1 --2- 172.21.15.31:0/3842195253 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb4340c7fb0 0x7fb43413c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:52.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:50.294+0000 7fb43a59b700 1 -- 172.21.15.31:0/3842195253 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb43413f780 msgr2=0x7fb4341590d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:52.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:50.294+0000 7fb43a59b700 1 --2- 172.21.15.31:0/3842195253 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb43413f780 0x7fb4341590d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:52.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:50.294+0000 7fb43a59b700 1 -- 172.21.15.31:0/3842195253 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb43000b790 con 0x7fb4340c7be0 2023-10-06T02:27:52.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:50.294+0000 7fb43ad9c700 1 --2- 172.21.15.31:0/3842195253 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb43413f780 0x7fb4341590d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:27:52.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:50.294+0000 7fb43a59b700 1 --2- 172.21.15.31:0/3842195253 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4340c7be0 0x7fb434157420 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fb43000b760 tx=0x7fb43001b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:52.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:50.295+0000 7fb42a7fc700 1 -- 172.21.15.31:0/3842195253 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb430010540 con 0x7fb4340c7be0 2023-10-06T02:27:52.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:50.295+0000 7fb42a7fc700 1 -- 172.21.15.31:0/3842195253 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4300106a0 con 0x7fb4340c7be0 2023-10-06T02:27:52.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:50.295+0000 7fb42a7fc700 1 -- 172.21.15.31:0/3842195253 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb430010a60 con 0x7fb4340c7be0 2023-10-06T02:27:52.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:50.295+0000 7fb43b59d700 1 -- 172.21.15.31:0/3842195253 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4340c4690 con 0x7fb4340c7be0 2023-10-06T02:27:52.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:50.295+0000 7fb43b59d700 1 -- 172.21.15.31:0/3842195253 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb43413c8b0 con 0x7fb4340c7be0 2023-10-06T02:27:52.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:50.296+0000 7fb43b59d700 1 -- 172.21.15.31:0/3842195253 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb43404c3c0 con 0x7fb4340c7be0 2023-10-06T02:27:52.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:50.297+0000 7fb42a7fc700 1 -- 172.21.15.31:0/3842195253 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7fb430022030 con 0x7fb4340c7be0 2023-10-06T02:27:52.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:50.297+0000 7fb42a7fc700 1 --2- 172.21.15.31:0/3842195253 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb418067050 0x7fb418069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:52.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:50.298+0000 7fb42a7fc700 1 -- 172.21.15.31:0/3842195253 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5851+0+0 (secure 0 0 0) 0x7fb430014070 con 0x7fb4340c7be0 2023-10-06T02:27:52.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:50.298+0000 7fb439d9a700 1 --2- 172.21.15.31:0/3842195253 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb418067050 0x7fb418069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:52.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:50.299+0000 7fb439d9a700 1 --2- 172.21.15.31:0/3842195253 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb418067050 0x7fb418069500 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb434157960 tx=0x7fb424008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:52.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:50.302+0000 7fb42a7fc700 1 -- 172.21.15.31:0/3842195253 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb430055a10 con 0x7fb4340c7be0 2023-10-06T02:27:52.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:50.505+0000 7fb43b59d700 1 -- 172.21.15.31:0/3842195253 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fb4340c0fb0 con 0x7fb4340c7be0 2023-10-06T02:27:52.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:50.509+0000 7fb42a7fc700 1 -- 172.21.15.31:0/3842195253 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb430058ff0 con 0x7fb4340c7be0 2023-10-06T02:27:52.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:50.512+0000 7fb43b59d700 1 -- 172.21.15.31:0/3842195253 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb418067050 msgr2=0x7fb418069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:52.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:50.512+0000 7fb43b59d700 1 --2- 172.21.15.31:0/3842195253 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb418067050 0x7fb418069500 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb434157960 tx=0x7fb424008040 comp rx=0 tx=0).stop 2023-10-06T02:27:52.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:50.512+0000 7fb43b59d700 1 -- 172.21.15.31:0/3842195253 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4340c7be0 msgr2=0x7fb434157420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:52.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:50.512+0000 7fb43b59d700 1 --2- 172.21.15.31:0/3842195253 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4340c7be0 0x7fb434157420 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fb43000b760 tx=0x7fb43001b000 comp rx=0 tx=0).stop 2023-10-06T02:27:52.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:50.512+0000 7fb43b59d700 1 -- 172.21.15.31:0/3842195253 shutdown_connections 2023-10-06T02:27:52.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:50.512+0000 7fb43b59d700 1 --2- 172.21.15.31:0/3842195253 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb43413f780 0x7fb4341590d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:52.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:50.512+0000 7fb43b59d700 1 --2- 172.21.15.31:0/3842195253 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4340c7be0 0x7fb434157420 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:52.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:50.512+0000 7fb43b59d700 1 --2- 172.21.15.31:0/3842195253 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb418067050 0x7fb418069500 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:52.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:50.512+0000 7fb43b59d700 1 --2- 172.21.15.31:0/3842195253 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb4340c7fb0 0x7fb43413c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:52.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:50.512+0000 7fb43b59d700 1 -- 172.21.15.31:0/3842195253 >> 172.21.15.31:0/3842195253 conn(0x7fb43406d110 msgr2=0x7fb4340ca6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:52.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:50.512+0000 7fb43b59d700 1 -- 172.21.15.31:0/3842195253 shutdown_connections 2023-10-06T02:27:52.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:50.512+0000 7fb43b59d700 1 -- 172.21.15.31:0/3842195253 wait complete. 2023-10-06T02:27:52.232 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T02:27:52.232 DEBUG:teuthology.orchestra.run.smithi031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T02:27:52.321 DEBUG:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T02:27:52.409 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T02:27:52.409 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T02:27:52.410 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:27:52.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.600+0000 7f7497fff700 1 Processor -- start 2023-10-06T02:27:52.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.600+0000 7f7497fff700 1 -- start start 2023-10-06T02:27:52.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.602+0000 7f7497fff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f74980b2940 0x7f74980a98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:52.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.602+0000 7f7497fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f74980cd9d0 0x7f749806e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:52.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.602+0000 7f7497fff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74980b0110 con 0x7f74980b2d10 2023-10-06T02:27:52.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.602+0000 7f7497fff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74980b0920 con 0x7f74980b2940 2023-10-06T02:27:52.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.602+0000 7f7497fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f749806cc70 con 0x7f74980cd9d0 2023-10-06T02:27:52.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.602+0000 7f74977fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f74980b2d10 0x7f7498159250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:59474/0 (socket says 172.21.15.31:59474) 2023-10-06T02:27:52.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.602+0000 7f74977fe700 1 -- 172.21.15.31:0/2459582192 learned_addr learned my addr 172.21.15.31:0/2459582192 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:27:52.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.602+0000 7f7496ffd700 1 --2- 172.21.15.31:0/2459582192 >> v2:172.21.15.62:3300/0 conn(0x7f74980b2940 0x7f74980a98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:52.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.602+0000 7f74967fc700 1 --2- 172.21.15.31:0/2459582192 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f74980cd9d0 0x7f749806e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:52.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.602+0000 7f7495ffb700 1 -- 172.21.15.31:0/2459582192 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3021308488 0 0) 0x7f74980b0110 con 0x7f74980b2d10 2023-10-06T02:27:52.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.602+0000 7f7495ffb700 1 -- 172.21.15.31:0/2459582192 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7480003570 con 0x7f74980b2d10 2023-10-06T02:27:52.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.603+0000 7f7495ffb700 1 -- 172.21.15.31:0/2459582192 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2275015798 0 0) 0x7f7480003570 con 0x7f74980b2d10 2023-10-06T02:27:52.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.603+0000 7f7495ffb700 1 -- 172.21.15.31:0/2459582192 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f74980cd9d0 msgr2=0x7f749806e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:52.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.603+0000 7f7495ffb700 1 --2- 172.21.15.31:0/2459582192 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f74980cd9d0 0x7f749806e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:52.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.603+0000 7f7495ffb700 1 -- 172.21.15.31:0/2459582192 >> v2:172.21.15.62:3300/0 conn(0x7f74980b2940 msgr2=0x7f74980a98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:52.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.603+0000 7f7495ffb700 1 --2- 172.21.15.31:0/2459582192 >> v2:172.21.15.62:3300/0 conn(0x7f74980b2940 0x7f74980a98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:52.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.603+0000 7f7495ffb700 1 -- 172.21.15.31:0/2459582192 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74980b0360 con 0x7f74980b2d10 2023-10-06T02:27:52.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.603+0000 7f7495ffb700 1 -- 172.21.15.31:0/2459582192 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7f74900017f0 con 0x7f74980b2d10 2023-10-06T02:27:52.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.603+0000 7f7495ffb700 1 -- 172.21.15.31:0/2459582192 >> v1:172.21.15.31:6789/0 conn(0x7f74980b2d10 legacy=0x7f7498159250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:52.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.603+0000 7f7495ffb700 1 --2- 172.21.15.31:0/2459582192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7480003ae0 0x7f7480003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:52.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.603+0000 7f7495ffb700 1 --2- 172.21.15.31:0/2459582192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7480004810 0x7f7480004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:52.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.603+0000 7f7495ffb700 1 --2- 172.21.15.31:0/2459582192 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f74980cd9d0 0x7f74800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:52.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.603+0000 7f7495ffb700 1 -- 172.21.15.31:0/2459582192 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74980b0920 con 0x7f7480004810 2023-10-06T02:27:52.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.603+0000 7f7495ffb700 1 -- 172.21.15.31:0/2459582192 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f749806cc70 con 0x7f7480003ae0 2023-10-06T02:27:52.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.603+0000 7f7495ffb700 1 -- 172.21.15.31:0/2459582192 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f748000bbc0 con 0x7f74980cd9d0 2023-10-06T02:27:52.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.603+0000 7f7496ffd700 1 --2- 172.21.15.31:0/2459582192 >> v2:172.21.15.62:3300/0 conn(0x7f74980b2940 0x7f74980a98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:27:52.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.603+0000 7f74967fc700 1 --2- 172.21.15.31:0/2459582192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7480004810 0x7f7480004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:52.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.603+0000 7f74977fe700 1 --2- 172.21.15.31:0/2459582192 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f74980cd9d0 0x7f74800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:52.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.603+0000 7f7496ffd700 1 --2- 172.21.15.31:0/2459582192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7480003ae0 0x7f7480003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:52.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.604+0000 7f74967fc700 1 -- 172.21.15.31:0/2459582192 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f74980cd9d0 msgr2=0x7f74800051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:52.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.604+0000 7f74967fc700 1 --2- 172.21.15.31:0/2459582192 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f74980cd9d0 0x7f74800051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:52.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.604+0000 7f74967fc700 1 -- 172.21.15.31:0/2459582192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7480003ae0 msgr2=0x7f7480003ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:52.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.604+0000 7f74967fc700 1 --2- 172.21.15.31:0/2459582192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7480003ae0 0x7f7480003ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:52.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.604+0000 7f74967fc700 1 -- 172.21.15.31:0/2459582192 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7480003570 con 0x7f7480004810 2023-10-06T02:27:52.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.604+0000 7f74977fe700 1 --2- 172.21.15.31:0/2459582192 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f74980cd9d0 0x7f74800051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:27:52.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.604+0000 7f74967fc700 1 --2- 172.21.15.31:0/2459582192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7480004810 0x7f7480004c80 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f748c009c00 tx=0x7f748c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29873955b36e011c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:52.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.604+0000 7f7495ffb700 1 -- 172.21.15.31:0/2459582192 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748c00e710 con 0x7f7480004810 2023-10-06T02:27:52.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.604+0000 7f7495ffb700 1 -- 172.21.15.31:0/2459582192 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f748c00e870 con 0x7f7480004810 2023-10-06T02:27:52.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.604+0000 7f7495ffb700 1 -- 172.21.15.31:0/2459582192 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748c00ec30 con 0x7f7480004810 2023-10-06T02:27:52.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.605+0000 7f7497fff700 1 -- 172.21.15.31:0/2459582192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7480004810 msgr2=0x7f7480004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:52.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.605+0000 7f7497fff700 1 --2- 172.21.15.31:0/2459582192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7480004810 0x7f7480004c80 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f748c009c00 tx=0x7f748c0091a0 comp rx=0 tx=0).stop 2023-10-06T02:27:52.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.605+0000 7f74977fe700 1 -- 172.21.15.31:0/2459582192 reap_dead start 2023-10-06T02:27:52.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.605+0000 7f7497fff700 1 -- 172.21.15.31:0/2459582192 shutdown_connections 2023-10-06T02:27:52.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.605+0000 7f7497fff700 1 -- 172.21.15.31:0/2459582192 >> 172.21.15.31:0/2459582192 conn(0x7f74980675a0 msgr2=0x7f74980679a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:52.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.605+0000 7f7497fff700 1 -- 172.21.15.31:0/2459582192 shutdown_connections 2023-10-06T02:27:52.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.605+0000 7f7497fff700 1 -- 172.21.15.31:0/2459582192 wait complete. 2023-10-06T02:27:52.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.606+0000 7f7497fff700 1 Processor -- start 2023-10-06T02:27:52.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.606+0000 7f7497fff700 1 -- start start 2023-10-06T02:27:52.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.607+0000 7f7497fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7480004810 0x7f74980af250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:52.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.607+0000 7f7497fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74980add40 0x7f749816a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:52.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.607+0000 7f7497fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74980af790 0x7f749816aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:52.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.607+0000 7f7497fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74980ae1b0 con 0x7f74980add40 2023-10-06T02:27:52.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.607+0000 7f7497fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74980ae490 con 0x7f74980af790 2023-10-06T02:27:52.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.607+0000 7f7497fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74980ae770 con 0x7f7480004810 2023-10-06T02:27:52.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.608+0000 7f74967fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74980add40 0x7f749816a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:52.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.608+0000 7f74967fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74980add40 0x7f749816a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41520/0 (socket says 172.21.15.31:41520) 2023-10-06T02:27:52.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.608+0000 7f74967fc700 1 -- 172.21.15.31:0/2352080559 learned_addr learned my addr 172.21.15.31:0/2352080559 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:52.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.608+0000 7f74977fe700 1 --2- 172.21.15.31:0/2352080559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74980af790 0x7f749816aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:52.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.608+0000 7f7496ffd700 1 --2- 172.21.15.31:0/2352080559 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7480004810 0x7f74980af250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:52.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.608+0000 7f74967fc700 1 -- 172.21.15.31:0/2352080559 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7480004810 msgr2=0x7f74980af250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:52.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.608+0000 7f74967fc700 1 --2- 172.21.15.31:0/2352080559 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7480004810 0x7f74980af250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:52.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.608+0000 7f74967fc700 1 -- 172.21.15.31:0/2352080559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74980af790 msgr2=0x7f749816aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:52.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.608+0000 7f74967fc700 1 --2- 172.21.15.31:0/2352080559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74980af790 0x7f749816aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:52.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.608+0000 7f74967fc700 1 -- 172.21.15.31:0/2352080559 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74980aea50 con 0x7f74980add40 2023-10-06T02:27:52.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.608+0000 7f74967fc700 1 --2- 172.21.15.31:0/2352080559 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74980add40 0x7f749816a4e0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f748c00e4b0 tx=0x7f748c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:52.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.609+0000 7f747effd700 1 -- 172.21.15.31:0/2352080559 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748c022070 con 0x7f74980add40 2023-10-06T02:27:52.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.609+0000 7f747effd700 1 -- 172.21.15.31:0/2352080559 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f748c00e9e0 con 0x7f74980add40 2023-10-06T02:27:52.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.609+0000 7f747effd700 1 -- 172.21.15.31:0/2352080559 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748c027470 con 0x7f74980add40 2023-10-06T02:27:52.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.609+0000 7f7497fff700 1 -- 172.21.15.31:0/2352080559 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74980aed30 con 0x7f74980add40 2023-10-06T02:27:52.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.609+0000 7f7497fff700 1 -- 172.21.15.31:0/2352080559 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7498151f80 con 0x7f74980add40 2023-10-06T02:27:52.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.610+0000 7f7497fff700 1 -- 172.21.15.31:0/2352080559 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f749804c3c0 con 0x7f74980add40 2023-10-06T02:27:52.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.611+0000 7f747effd700 1 -- 172.21.15.31:0/2352080559 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f748c01c450 con 0x7f74980add40 2023-10-06T02:27:52.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.612+0000 7f747effd700 1 --2- 172.21.15.31:0/2352080559 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7484067050 0x7f7484069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:52.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.613+0000 7f747effd700 1 -- 172.21.15.31:0/2352080559 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5851+0+0 (secure 0 0 0) 0x7f748c086270 con 0x7f74980add40 2023-10-06T02:27:52.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.613+0000 7f7496ffd700 1 --2- 172.21.15.31:0/2352080559 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7484067050 0x7f7484069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:52.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.614+0000 7f7496ffd700 1 --2- 172.21.15.31:0/2352080559 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7484067050 0x7f7484069500 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f749816a490 tx=0x7f7488009120 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:52.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.616+0000 7f747effd700 1 -- 172.21.15.31:0/2352080559 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f748c053360 con 0x7f74980add40 2023-10-06T02:27:52.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.861+0000 7f7497fff700 1 -- 172.21.15.31:0/2352080559 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f74980b0640 con 0x7f74980add40 2023-10-06T02:27:52.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.861+0000 7f747effd700 1 -- 172.21.15.31:0/2352080559 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+3526 (secure 0 0 0) 0x7f748c017de0 con 0x7f74980add40 2023-10-06T02:27:52.862 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:27:52.862 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":49,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5474,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/2085637096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2085637096},{"type":"v1","addr":"172.21.15.62:6834","nonce":2085637096}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":5992,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/573080449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":573080449},{"type":"v1","addr":"172.21.15.31:6836","nonce":573080449}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":6007,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2625778079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2625778079},{"type":"v1","addr":"172.21.15.62:6835","nonce":2625778079}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":6645,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6833/2069375494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":2069375494},{"type":"v1","addr":"172.21.15.31:6833","nonce":2069375494}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49}],"filesystems":[]} 2023-10-06T02:27:52.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.864+0000 7f7497fff700 1 -- 172.21.15.31:0/2352080559 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7484067050 msgr2=0x7f7484069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:52.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.864+0000 7f7497fff700 1 --2- 172.21.15.31:0/2352080559 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7484067050 0x7f7484069500 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f749816a490 tx=0x7f7488009120 comp rx=0 tx=0).stop 2023-10-06T02:27:52.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.864+0000 7f7497fff700 1 -- 172.21.15.31:0/2352080559 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74980add40 msgr2=0x7f749816a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:52.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.864+0000 7f7497fff700 1 --2- 172.21.15.31:0/2352080559 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74980add40 0x7f749816a4e0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f748c00e4b0 tx=0x7f748c01a000 comp rx=0 tx=0).stop 2023-10-06T02:27:52.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.864+0000 7f7497fff700 1 -- 172.21.15.31:0/2352080559 shutdown_connections 2023-10-06T02:27:52.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.864+0000 7f7497fff700 1 --2- 172.21.15.31:0/2352080559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74980af790 0x7f749816aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:52.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.864+0000 7f7497fff700 1 --2- 172.21.15.31:0/2352080559 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7484067050 0x7f7484069500 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:52.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.864+0000 7f7497fff700 1 --2- 172.21.15.31:0/2352080559 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7480004810 0x7f74980af250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:52.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.864+0000 7f7497fff700 1 --2- 172.21.15.31:0/2352080559 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74980add40 0x7f749816a4e0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:52.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.864+0000 7f7497fff700 1 -- 172.21.15.31:0/2352080559 >> 172.21.15.31:0/2352080559 conn(0x7f74980675a0 msgr2=0x7f74980aafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:52.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.864+0000 7f7497fff700 1 -- 172.21.15.31:0/2352080559 shutdown_connections 2023-10-06T02:27:52.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:52.864+0000 7f7497fff700 1 -- 172.21.15.31:0/2352080559 wait complete. 2023-10-06T02:27:52.866 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 49 2023-10-06T02:27:52.882 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T02:27:52.882 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T02:27:52.882 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T02:27:52.882 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T02:27:52.882 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T02:27:52.882 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T02:27:52.883 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T02:27:52.883 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T02:27:52.883 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T02:27:52.883 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T02:27:52.883 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T02:27:52.883 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T02:27:52.883 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T02:27:52.883 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T02:27:52.884 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T02:27:52.884 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T02:27:52.896 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T02:27:52.896 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T02:27:52.897 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T02:27:52.897 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T02:27:52.900 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T02:27:52.900 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T02:27:52.900 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T02:27:52.901 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T02:27:52.941 INFO:tasks.ceph.mds.a:Started 2023-10-06T02:27:52.942 DEBUG:teuthology.parallel:result is None 2023-10-06T02:27:52.944 INFO:tasks.ceph.mds.b:Started 2023-10-06T02:27:52.945 DEBUG:teuthology.parallel:result is None 2023-10-06T02:27:52.945 INFO:tasks.ceph.mds.c:Started 2023-10-06T02:27:52.945 DEBUG:teuthology.parallel:result is None 2023-10-06T02:27:52.947 INFO:tasks.ceph.mds.d:Started 2023-10-06T02:27:52.947 DEBUG:teuthology.parallel:result is None 2023-10-06T02:27:52.948 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T02:27:53.079 INFO:tasks.ceph.mds.a.smithi031.stdout:starting mds.a at 2023-10-06T02:27:53.081 INFO:tasks.ceph.mds.c.smithi031.stdout:starting mds.c at 2023-10-06T02:27:53.086 INFO:tasks.ceph.mds.d.smithi062.stdout:starting mds.d at 2023-10-06T02:27:53.086 INFO:tasks.ceph.mds.b.smithi062.stdout:starting mds.b at 2023-10-06T02:27:53.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.147+0000 7f6361d9d700 1 Processor -- start 2023-10-06T02:27:53.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.147+0000 7f6361d9d700 1 -- start start 2023-10-06T02:27:53.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.148+0000 7f6361d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f635c06aaa0 0x7f635c15a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:53.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.148+0000 7f6361d9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f635c15ae70 con 0x7f635c13fef0 2023-10-06T02:27:53.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.148+0000 7f6361d9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f635c06af60 con 0x7f635c13fb20 2023-10-06T02:27:53.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.148+0000 7f6361d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f635c06b240 con 0x7f635c06aaa0 2023-10-06T02:27:53.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.149+0000 7f6360d9b700 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f635c13fb20 0x7f635c0c7990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:35224/0 (socket says 172.21.15.31:35224) 2023-10-06T02:27:53.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.149+0000 7f6360d9b700 1 -- 172.21.15.31:0/252966369 learned_addr learned my addr 172.21.15.31:0/252966369 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:27:53.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.149+0000 7f635bfff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f635c13fef0 0x7f635c0c9d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:59480/0 (socket says 172.21.15.31:59480) 2023-10-06T02:27:53.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.149+0000 7f636159c700 1 --2- 172.21.15.31:0/252966369 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f635c06aaa0 0x7f635c15a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:53.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.152+0000 7f635b7fe700 1 -- 172.21.15.31:0/252966369 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2294694531 0 0) 0x7f635c06af60 con 0x7f635c13fb20 2023-10-06T02:27:53.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.152+0000 7f635b7fe700 1 -- 172.21.15.31:0/252966369 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f634c003570 con 0x7f635c13fb20 2023-10-06T02:27:53.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.152+0000 7f636159c700 1 -- 172.21.15.31:0/252966369 >> v1:172.21.15.62:6789/0 conn(0x7f635c13fb20 legacy=0x7f635c0c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:53.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.152+0000 7f636159c700 1 -- 172.21.15.31:0/252966369 >> v1:172.21.15.31:6789/0 conn(0x7f635c13fef0 legacy=0x7f635c0c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:53.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.152+0000 7f636159c700 1 -- 172.21.15.31:0/252966369 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f635c06b520 con 0x7f635c06aaa0 2023-10-06T02:27:53.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.152+0000 7f636159c700 1 --2- 172.21.15.31:0/252966369 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f635c06aaa0 0x7f635c15a930 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f635400bf90 tx=0x7f63540092b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5e5f735e20003a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:53.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.153+0000 7f635b7fe700 1 -- 172.21.15.31:0/252966369 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635400b2e0 con 0x7f635c06aaa0 2023-10-06T02:27:53.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.153+0000 7f635b7fe700 1 -- 172.21.15.31:0/252966369 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6354008bc0 con 0x7f635c06aaa0 2023-10-06T02:27:53.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.153+0000 7f635b7fe700 1 -- 172.21.15.31:0/252966369 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6354008f60 con 0x7f635c06aaa0 2023-10-06T02:27:53.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.154+0000 7f6361d9d700 1 -- 172.21.15.31:0/252966369 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f635c06aaa0 msgr2=0x7f635c15a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:53.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.154+0000 7f6361d9d700 1 --2- 172.21.15.31:0/252966369 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f635c06aaa0 0x7f635c15a930 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f635400bf90 tx=0x7f63540092b0 comp rx=0 tx=0).stop 2023-10-06T02:27:53.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.154+0000 7f6361d9d700 1 -- 172.21.15.31:0/252966369 shutdown_connections 2023-10-06T02:27:53.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.154+0000 7f6361d9d700 1 --2- 172.21.15.31:0/252966369 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f635c06aaa0 0x7f635c15a930 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:53.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.154+0000 7f6361d9d700 1 -- 172.21.15.31:0/252966369 >> 172.21.15.31:0/252966369 conn(0x7f635c06d170 msgr2=0x7f635c0cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:53.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.154+0000 7f6361d9d700 1 -- 172.21.15.31:0/252966369 shutdown_connections 2023-10-06T02:27:53.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.154+0000 7f6361d9d700 1 -- 172.21.15.31:0/252966369 wait complete. 2023-10-06T02:27:53.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.157+0000 7f6361d9d700 1 Processor -- start 2023-10-06T02:27:53.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.157+0000 7f6361d9d700 1 -- start start 2023-10-06T02:27:53.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.158+0000 7f6361d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f635c06aaa0 0x7f635c0c7c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:53.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.158+0000 7f6361d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f635c13fb20 0x7f635c13bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:53.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.158+0000 7f6361d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f635c13fef0 0x7f635c0ad8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:53.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.158+0000 7f6361d9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f635c169320 con 0x7f635c13fef0 2023-10-06T02:27:53.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.158+0000 7f6361d9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f635c1594a0 con 0x7f635c06aaa0 2023-10-06T02:27:53.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.158+0000 7f6361d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f635c159780 con 0x7f635c13fb20 2023-10-06T02:27:53.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.158+0000 7f6360d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f635c06aaa0 0x7f635c0c7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:53.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.158+0000 7f635bfff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f635c13fb20 0x7f635c13bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:53.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.158+0000 7f636159c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f635c13fef0 0x7f635c0ad8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:53.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.158+0000 7f635bfff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f635c13fb20 0x7f635c13bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:34722/0 (socket says 172.21.15.31:34722) 2023-10-06T02:27:53.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.158+0000 7f635bfff700 1 -- 172.21.15.31:0/470558194 learned_addr learned my addr 172.21.15.31:0/470558194 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:53.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.159+0000 7f636159c700 1 -- 172.21.15.31:0/470558194 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f635c13fb20 msgr2=0x7f635c13bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:53.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.159+0000 7f636159c700 1 --2- 172.21.15.31:0/470558194 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f635c13fb20 0x7f635c13bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:53.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.159+0000 7f636159c700 1 -- 172.21.15.31:0/470558194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f635c06aaa0 msgr2=0x7f635c0c7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:53.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.159+0000 7f636159c700 1 --2- 172.21.15.31:0/470558194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f635c06aaa0 0x7f635c0c7c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:53.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.159+0000 7f636159c700 1 -- 172.21.15.31:0/470558194 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f635400ba40 con 0x7f635c13fef0 2023-10-06T02:27:53.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.159+0000 7f635bfff700 1 --2- 172.21.15.31:0/470558194 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f635c13fb20 0x7f635c13bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:27:53.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.159+0000 7f636159c700 1 --2- 172.21.15.31:0/470558194 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f635c13fef0 0x7f635c0ad8b0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f635401e000 tx=0x7f635401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:53.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.160+0000 7f6358ff9700 1 -- 172.21.15.31:0/470558194 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6354015470 con 0x7f635c13fef0 2023-10-06T02:27:53.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.160+0000 7f6358ff9700 1 -- 172.21.15.31:0/470558194 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63540155f0 con 0x7f635c13fef0 2023-10-06T02:27:53.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.160+0000 7f6361d9d700 1 -- 172.21.15.31:0/470558194 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f635c13c020 con 0x7f635c13fef0 2023-10-06T02:27:53.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.160+0000 7f6358ff9700 1 -- 172.21.15.31:0/470558194 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63540159b0 con 0x7f635c13fef0 2023-10-06T02:27:53.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.161+0000 7f6361d9d700 1 -- 172.21.15.31:0/470558194 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f635c13c4d0 con 0x7f635c13fef0 2023-10-06T02:27:53.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.162+0000 7f6361d9d700 1 -- 172.21.15.31:0/470558194 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f635c1569e0 con 0x7f635c13fef0 2023-10-06T02:27:53.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.163+0000 7f6358ff9700 1 -- 172.21.15.31:0/470558194 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f6354022400 con 0x7f635c13fef0 2023-10-06T02:27:53.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.164+0000 7f6358ff9700 1 --2- 172.21.15.31:0/470558194 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6340067050 0x7f6340069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:53.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.164+0000 7f6358ff9700 1 -- 172.21.15.31:0/470558194 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5851+0+0 (secure 0 0 0) 0x7f6354085320 con 0x7f635c13fef0 2023-10-06T02:27:53.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.165+0000 7f6360d9b700 1 --2- 172.21.15.31:0/470558194 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6340067050 0x7f6340069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:53.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.165+0000 7f6360d9b700 1 --2- 172.21.15.31:0/470558194 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6340067050 0x7f6340069500 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6350007fd0 tx=0x7f6350009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:53.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.169+0000 7f6358ff9700 1 -- 172.21.15.31:0/470558194 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f635c1569e0 con 0x7f635c13fef0 2023-10-06T02:27:53.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.390+0000 7f6361d9d700 1 -- 172.21.15.31:0/470558194 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f635c15ae70 con 0x7f635c13fef0 2023-10-06T02:27:53.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.507+0000 7f6358ff9700 1 -- 172.21.15.31:0/470558194 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v81) v1 ==== 97+0+0 (secure 0 0 0) 0x7f635c15ae70 con 0x7f635c13fef0 2023-10-06T02:27:53.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.511+0000 7f6361d9d700 1 -- 172.21.15.31:0/470558194 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6340067050 msgr2=0x7f6340069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:53.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.511+0000 7f6361d9d700 1 --2- 172.21.15.31:0/470558194 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6340067050 0x7f6340069500 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6350007fd0 tx=0x7f6350009040 comp rx=0 tx=0).stop 2023-10-06T02:27:53.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.511+0000 7f6361d9d700 1 -- 172.21.15.31:0/470558194 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f635c13fef0 msgr2=0x7f635c0ad8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:53.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.511+0000 7f6361d9d700 1 --2- 172.21.15.31:0/470558194 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f635c13fef0 0x7f635c0ad8b0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f635401e000 tx=0x7f635401f000 comp rx=0 tx=0).stop 2023-10-06T02:27:53.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.511+0000 7f6361d9d700 1 -- 172.21.15.31:0/470558194 shutdown_connections 2023-10-06T02:27:53.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.511+0000 7f6361d9d700 1 --2- 172.21.15.31:0/470558194 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f635c13fef0 0x7f635c0ad8b0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:53.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.511+0000 7f6361d9d700 1 --2- 172.21.15.31:0/470558194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f635c06aaa0 0x7f635c0c7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:53.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.511+0000 7f6361d9d700 1 --2- 172.21.15.31:0/470558194 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6340067050 0x7f6340069500 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:53.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.511+0000 7f6361d9d700 1 --2- 172.21.15.31:0/470558194 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f635c13fb20 0x7f635c13bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:53.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.511+0000 7f6361d9d700 1 -- 172.21.15.31:0/470558194 >> 172.21.15.31:0/470558194 conn(0x7f635c06d170 msgr2=0x7f635c13e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:53.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.511+0000 7f6361d9d700 1 -- 172.21.15.31:0/470558194 shutdown_connections 2023-10-06T02:27:53.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.511+0000 7f6361d9d700 1 -- 172.21.15.31:0/470558194 wait complete. 2023-10-06T02:27:53.513 INFO:teuthology.orchestra.run.smithi031.stderr: removed all blocklist entries 2023-10-06T02:27:53.530 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T02:27:53.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.716+0000 7f1d6d79d700 1 Processor -- start 2023-10-06T02:27:53.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.716+0000 7f1d6d79d700 1 -- start start 2023-10-06T02:27:53.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.717+0000 7f1d6d79d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f1d6813fb20 0x7f1d6815a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:53.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.717+0000 7f1d6d79d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d6815ae70 con 0x7f1d6806aaa0 2023-10-06T02:27:53.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.717+0000 7f1d6d79d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d6806af60 con 0x7f1d6813fef0 2023-10-06T02:27:53.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.717+0000 7f1d6d79d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d6806b240 con 0x7f1d6813fb20 2023-10-06T02:27:53.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.718+0000 7f1d6cf9c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f1d6806aaa0 0x7f1d680c9d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:59484/0 (socket says 172.21.15.31:59484) 2023-10-06T02:27:53.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.718+0000 7f1d6cf9c700 1 -- 172.21.15.31:0/163104556 learned_addr learned my addr 172.21.15.31:0/163104556 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:27:53.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.718+0000 7f1d67fff700 1 --2- 172.21.15.31:0/163104556 >> v2:172.21.15.62:3300/0 conn(0x7f1d6813fb20 0x7f1d6815a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:53.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.718+0000 7f1d66ffd700 1 -- 172.21.15.31:0/163104556 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 66855257 0 0) 0x7f1d60001490 con 0x7f1d6806aaa0 2023-10-06T02:27:53.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.718+0000 7f1d66ffd700 1 -- 172.21.15.31:0/163104556 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d50003570 con 0x7f1d6806aaa0 2023-10-06T02:27:53.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.718+0000 7f1d66ffd700 1 -- 172.21.15.31:0/163104556 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 36210924 0 0) 0x7f1d6806af60 con 0x7f1d6813fef0 2023-10-06T02:27:53.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.718+0000 7f1d66ffd700 1 -- 172.21.15.31:0/163104556 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d60001490 con 0x7f1d6813fef0 2023-10-06T02:27:53.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.718+0000 7f1d66ffd700 1 -- 172.21.15.31:0/163104556 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1710343781 0 0) 0x7f1d50003570 con 0x7f1d6806aaa0 2023-10-06T02:27:53.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.719+0000 7f1d66ffd700 1 -- 172.21.15.31:0/163104556 >> v2:172.21.15.62:3300/0 conn(0x7f1d6813fb20 msgr2=0x7f1d6815a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:53.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.719+0000 7f1d66ffd700 1 --2- 172.21.15.31:0/163104556 >> v2:172.21.15.62:3300/0 conn(0x7f1d6813fb20 0x7f1d6815a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:53.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.719+0000 7f1d66ffd700 1 -- 172.21.15.31:0/163104556 >> v1:172.21.15.62:6789/0 conn(0x7f1d6813fef0 legacy=0x7f1d680c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:53.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.719+0000 7f1d66ffd700 1 -- 172.21.15.31:0/163104556 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d6806b520 con 0x7f1d6806aaa0 2023-10-06T02:27:53.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.719+0000 7f1d66ffd700 1 -- 172.21.15.31:0/163104556 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7f1d60001980 con 0x7f1d6806aaa0 2023-10-06T02:27:53.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.719+0000 7f1d66ffd700 1 -- 172.21.15.31:0/163104556 >> v1:172.21.15.31:6789/0 conn(0x7f1d6806aaa0 legacy=0x7f1d680c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:53.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.719+0000 7f1d66ffd700 1 --2- 172.21.15.31:0/163104556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d50003bf0 0x7f1d50003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:53.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.719+0000 7f1d66ffd700 1 --2- 172.21.15.31:0/163104556 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d50004810 0x7f1d50004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:53.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.719+0000 7f1d66ffd700 1 --2- 172.21.15.31:0/163104556 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1d500051c0 0x7f1d5000dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:53.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.719+0000 7f1d66ffd700 1 -- 172.21.15.31:0/163104556 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6806b240 con 0x7f1d50003bf0 2023-10-06T02:27:53.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.719+0000 7f1d66ffd700 1 -- 172.21.15.31:0/163104556 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d5000e2c0 con 0x7f1d50004810 2023-10-06T02:27:53.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.719+0000 7f1d66ffd700 1 -- 172.21.15.31:0/163104556 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d5000e480 con 0x7f1d500051c0 2023-10-06T02:27:53.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.719+0000 7f1d67fff700 1 --2- 172.21.15.31:0/163104556 >> v2:172.21.15.62:3300/0 conn(0x7f1d6813fb20 0x7f1d6815a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:27:53.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.719+0000 7f1d67fff700 1 --2- 172.21.15.31:0/163104556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d50003bf0 0x7f1d50003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:53.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.719+0000 7f1d6cf9c700 1 --2- 172.21.15.31:0/163104556 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1d500051c0 0x7f1d5000dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:53.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.719+0000 7f1d677fe700 1 --2- 172.21.15.31:0/163104556 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d50004810 0x7f1d50004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:53.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.719+0000 7f1d67fff700 1 -- 172.21.15.31:0/163104556 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1d500051c0 msgr2=0x7f1d5000dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:53.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.719+0000 7f1d67fff700 1 --2- 172.21.15.31:0/163104556 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1d500051c0 0x7f1d5000dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:53.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.719+0000 7f1d67fff700 1 -- 172.21.15.31:0/163104556 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d50004810 msgr2=0x7f1d50004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:53.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.719+0000 7f1d67fff700 1 --2- 172.21.15.31:0/163104556 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d50004810 0x7f1d50004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:53.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.719+0000 7f1d67fff700 1 -- 172.21.15.31:0/163104556 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d50003570 con 0x7f1d50003bf0 2023-10-06T02:27:53.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.719+0000 7f1d677fe700 1 --2- 172.21.15.31:0/163104556 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d50004810 0x7f1d50004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:27:53.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.719+0000 7f1d6cf9c700 1 -- 172.21.15.31:0/163104556 reap_dead start 2023-10-06T02:27:53.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.720+0000 7f1d67fff700 1 --2- 172.21.15.31:0/163104556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d50003bf0 0x7f1d50003fe0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f1d58009db0 tx=0x7f1d580091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df766bb1b15aad29 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:53.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.721+0000 7f1d66ffd700 1 -- 172.21.15.31:0/163104556 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d58009740 con 0x7f1d50003bf0 2023-10-06T02:27:53.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.721+0000 7f1d66ffd700 1 -- 172.21.15.31:0/163104556 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d5800e920 con 0x7f1d50003bf0 2023-10-06T02:27:53.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.721+0000 7f1d66ffd700 1 -- 172.21.15.31:0/163104556 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d5800eca0 con 0x7f1d50003bf0 2023-10-06T02:27:53.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.721+0000 7f1d6d79d700 1 -- 172.21.15.31:0/163104556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d50003bf0 msgr2=0x7f1d50003fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:53.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.721+0000 7f1d6d79d700 1 --2- 172.21.15.31:0/163104556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d50003bf0 0x7f1d50003fe0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f1d58009db0 tx=0x7f1d580091a0 comp rx=0 tx=0).stop 2023-10-06T02:27:53.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.721+0000 7f1d6d79d700 1 -- 172.21.15.31:0/163104556 shutdown_connections 2023-10-06T02:27:53.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.721+0000 7f1d6d79d700 1 --2- 172.21.15.31:0/163104556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d50003bf0 0x7f1d50003fe0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:53.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.721+0000 7f1d6d79d700 1 -- 172.21.15.31:0/163104556 >> 172.21.15.31:0/163104556 conn(0x7f1d6806d170 msgr2=0x7f1d680cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:53.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.721+0000 7f1d6d79d700 1 -- 172.21.15.31:0/163104556 shutdown_connections 2023-10-06T02:27:53.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.721+0000 7f1d6d79d700 1 -- 172.21.15.31:0/163104556 wait complete. 2023-10-06T02:27:53.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.723+0000 7f1d6d79d700 1 Processor -- start 2023-10-06T02:27:53.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.723+0000 7f1d6d79d700 1 -- start start 2023-10-06T02:27:53.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.724+0000 7f1d6d79d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d50003bf0 0x7f1d68155d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:53.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.724+0000 7f1d6d79d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1d680c7620 0x7f1d680c7a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:53.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.724+0000 7f1d6d79d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d680ae060 0x7f1d680ae4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:53.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.724+0000 7f1d6d79d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d680c7fd0 con 0x7f1d50003bf0 2023-10-06T02:27:53.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.724+0000 7f1d6d79d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d680afc30 con 0x7f1d680ae060 2023-10-06T02:27:53.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.724+0000 7f1d6d79d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d680aea10 con 0x7f1d680c7620 2023-10-06T02:27:53.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.724+0000 7f1d67fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d50003bf0 0x7f1d68155d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:53.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.724+0000 7f1d67fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d50003bf0 0x7f1d68155d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41586/0 (socket says 172.21.15.31:41586) 2023-10-06T02:27:53.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.724+0000 7f1d67fff700 1 -- 172.21.15.31:0/2312082248 learned_addr learned my addr 172.21.15.31:0/2312082248 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:53.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.724+0000 7f1d677fe700 1 --2- 172.21.15.31:0/2312082248 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1d680c7620 0x7f1d680c7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:53.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.724+0000 7f1d6cf9c700 1 --2- 172.21.15.31:0/2312082248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d680ae060 0x7f1d680ae4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:53.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.724+0000 7f1d67fff700 1 -- 172.21.15.31:0/2312082248 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1d680c7620 msgr2=0x7f1d680c7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:53.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.725+0000 7f1d67fff700 1 --2- 172.21.15.31:0/2312082248 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1d680c7620 0x7f1d680c7a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:53.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.725+0000 7f1d67fff700 1 -- 172.21.15.31:0/2312082248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d680ae060 msgr2=0x7f1d680ae4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:53.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.725+0000 7f1d67fff700 1 --2- 172.21.15.31:0/2312082248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d680ae060 0x7f1d680ae4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:53.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.725+0000 7f1d67fff700 1 -- 172.21.15.31:0/2312082248 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d6816a440 con 0x7f1d50003bf0 2023-10-06T02:27:53.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.725+0000 7f1d6cf9c700 1 --2- 172.21.15.31:0/2312082248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d680ae060 0x7f1d680ae4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:27:53.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.725+0000 7f1d67fff700 1 --2- 172.21.15.31:0/2312082248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d50003bf0 0x7f1d68155d80 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f1d5800e790 tx=0x7f1d58019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:53.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.725+0000 7f1d4ffff700 1 -- 172.21.15.31:0/2312082248 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d58029070 con 0x7f1d50003bf0 2023-10-06T02:27:53.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.725+0000 7f1d4ffff700 1 -- 172.21.15.31:0/2312082248 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d58021040 con 0x7f1d50003bf0 2023-10-06T02:27:53.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.725+0000 7f1d4ffff700 1 -- 172.21.15.31:0/2312082248 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d5801b850 con 0x7f1d50003bf0 2023-10-06T02:27:53.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.725+0000 7f1d6d79d700 1 -- 172.21.15.31:0/2312082248 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d68169b10 con 0x7f1d50003bf0 2023-10-06T02:27:53.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.725+0000 7f1d6d79d700 1 -- 172.21.15.31:0/2312082248 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d6816a0d0 con 0x7f1d50003bf0 2023-10-06T02:27:53.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.726+0000 7f1d6d79d700 1 -- 172.21.15.31:0/2312082248 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d6804c3c0 con 0x7f1d50003bf0 2023-10-06T02:27:53.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.727+0000 7f1d4ffff700 1 -- 172.21.15.31:0/2312082248 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f1d5801bc00 con 0x7f1d50003bf0 2023-10-06T02:27:53.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.728+0000 7f1d4ffff700 1 --2- 172.21.15.31:0/2312082248 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f1d54067050 0x7f1d54069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:53.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.728+0000 7f1d4ffff700 1 -- 172.21.15.31:0/2312082248 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f1d580863a0 con 0x7f1d50003bf0 2023-10-06T02:27:53.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.728+0000 7f1d677fe700 1 --2- 172.21.15.31:0/2312082248 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f1d54067050 0x7f1d54069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:53.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.729+0000 7f1d677fe700 1 --2- 172.21.15.31:0/2312082248 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f1d54067050 0x7f1d54069500 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1d6813e700 tx=0x7f1d5c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:53.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:53.732+0000 7f1d4ffff700 1 -- 172.21.15.31:0/2312082248 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1d58053710 con 0x7f1d50003bf0 2023-10-06T02:27:54.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.013+0000 7f1d6d79d700 1 -- 172.21.15.31:0/2312082248 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f1d6806b800 con 0x7f1d50003bf0 2023-10-06T02:27:54.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.014+0000 7f1d4ffff700 1 -- 172.21.15.31:0/2312082248 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f1d58056d30 con 0x7f1d50003bf0 2023-10-06T02:27:54.015 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:27:54.015 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T02:27:54.015 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_dump": [ 2023-10-06T02:27:54.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:54.016 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.a", 2023-10-06T02:27:54.016 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBLbx9lbjrOGxAAVoSOoDaZb1temL41F4Nong==", 2023-10-06T02:27:54.016 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:27:54.016 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:27:54.017 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T02:27:54.017 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T02:27:54.017 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:27:54.017 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:54.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:54.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:54.018 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.b", 2023-10-06T02:27:54.018 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBLbx9lFgsTMRAATTklzroiyQbTLiZc9j1BHQ==", 2023-10-06T02:27:54.018 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:27:54.018 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:27:54.018 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T02:27:54.018 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T02:27:54.018 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:27:54.019 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:54.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:54.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:54.019 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.c", 2023-10-06T02:27:54.019 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBLbx9lezVtJhAA8wK7+dJ1OmIc+iXJkexK0w==", 2023-10-06T02:27:54.019 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:27:54.019 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:27:54.020 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T02:27:54.020 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T02:27:54.020 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:27:54.020 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:54.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:54.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:54.020 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.d", 2023-10-06T02:27:54.020 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBMbx9l74gWABAApBu2nVv12rGA+OBfBnEc3A==", 2023-10-06T02:27:54.021 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:27:54.021 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:27:54.021 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T02:27:54.021 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T02:27:54.021 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:27:54.021 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:54.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:54.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:54.022 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.0", 2023-10-06T02:27:54.022 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBPbx9lgj6+MBAAXSUD0GCG+rO/9DQBZYt/8w==", 2023-10-06T02:27:54.022 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:27:54.022 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:27:54.022 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:27:54.022 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:27:54.022 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:54.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:54.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:54.023 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.1", 2023-10-06T02:27:54.023 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBSbx9lXhMTNBAAeEdPnMP71ZGb6k2Ddg5raQ==", 2023-10-06T02:27:54.023 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:27:54.023 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:27:54.023 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:27:54.023 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:27:54.023 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:54.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:54.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:54.023 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.2", 2023-10-06T02:27:54.023 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBVbx9l4Ly0NhAATYXYDAjYuP53zjqK0yMboA==", 2023-10-06T02:27:54.024 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:27:54.024 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:27:54.024 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:27:54.024 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:27:54.024 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:54.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:54.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:54.024 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.3", 2023-10-06T02:27:54.024 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBZbx9lzsi4ARAAbGnNV0QyL/7itzy827+chg==", 2023-10-06T02:27:54.024 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:27:54.024 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:27:54.024 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:27:54.025 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:27:54.025 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:54.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:54.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:54.025 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.4", 2023-10-06T02:27:54.025 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBfbx9lW4OSEhAAj7HnVh/yq12zzCaXd6BHCA==", 2023-10-06T02:27:54.025 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:27:54.025 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:27:54.025 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:27:54.025 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:27:54.025 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:54.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:54.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:54.026 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.5", 2023-10-06T02:27:54.026 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBibx9lBcaIFRAAV8wq8QjsCOkI9Pzf/E+9bA==", 2023-10-06T02:27:54.026 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:27:54.026 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:27:54.026 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:27:54.026 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:27:54.026 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:54.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:54.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:54.026 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.6", 2023-10-06T02:27:54.026 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBlbx9lA7S+FxAA38mlSAurT+DpwMzH53zFcw==", 2023-10-06T02:27:54.026 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:27:54.027 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:27:54.027 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:27:54.027 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:27:54.027 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:54.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:54.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:54.027 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.7", 2023-10-06T02:27:54.027 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBobx9l5RkdHhAAxn0Z8IAE/o9oe2LRp3O9gg==", 2023-10-06T02:27:54.027 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:27:54.027 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:27:54.027 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:27:54.027 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:27:54.028 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:54.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:54.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:54.028 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.0", 2023-10-06T02:27:54.028 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBMbx9lh44fCRAA7sMISvk1t7dEJg4ZcBiMcA==", 2023-10-06T02:27:54.028 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:27:54.028 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:27:54.028 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow r", 2023-10-06T02:27:54.028 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T02:27:54.028 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:54.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:54.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:54.029 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.1", 2023-10-06T02:27:54.029 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBMbx9lP0zqFhAAlov2y7ipSvS3AkOdQkhg6g==", 2023-10-06T02:27:54.029 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:27:54.029 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:27:54.029 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-10-06T02:27:54.029 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-10-06T02:27:54.029 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-10-06T02:27:54.029 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:54.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:54.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:54.029 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.2", 2023-10-06T02:27:54.030 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBMbx9lpxYAEBAACtiI1XgQXM8yGGcpFyNK1w==", 2023-10-06T02:27:54.030 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:27:54.030 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:27:54.030 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-10-06T02:27:54.030 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-10-06T02:27:54.030 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-10-06T02:27:54.030 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:54.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:54.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:54.030 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.3", 2023-10-06T02:27:54.030 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBMbx9l81/PHRAAYFOoaezu0pwE0NdtnlYfVA==", 2023-10-06T02:27:54.030 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:27:54.031 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:27:54.031 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-10-06T02:27:54.031 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-10-06T02:27:54.031 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-10-06T02:27:54.031 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:54.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:54.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:54.031 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.admin", 2023-10-06T02:27:54.031 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBKbx9liFl1JBAAFyyHYNLVx1/Ex5a2MKm8rA==", 2023-10-06T02:27:54.031 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:27:54.031 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-10-06T02:27:54.031 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T02:27:54.031 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T02:27:54.032 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:27:54.032 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:54.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:54.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:54.032 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mgr.x", 2023-10-06T02:27:54.032 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBLbx9lTBbIFBAAX2wsioKL4Nx4qz/qwsCsNA==", 2023-10-06T02:27:54.032 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:27:54.032 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-10-06T02:27:54.032 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile mgr", 2023-10-06T02:27:54.032 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:27:54.032 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:54.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:54.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:54.033 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mgr.y", 2023-10-06T02:27:54.033 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBLbx9l4+oLDRAA5YVo3q6FI9e9W/rJYROIsg==", 2023-10-06T02:27:54.033 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:27:54.033 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-10-06T02:27:54.033 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile mgr", 2023-10-06T02:27:54.033 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:27:54.033 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:54.033 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:54.033 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-10-06T02:27:54.033 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T02:27:54.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.016+0000 7f1d6d79d700 1 -- 172.21.15.31:0/2312082248 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f1d54067050 msgr2=0x7f1d54069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:54.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.016+0000 7f1d6d79d700 1 --2- 172.21.15.31:0/2312082248 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f1d54067050 0x7f1d54069500 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1d6813e700 tx=0x7f1d5c009040 comp rx=0 tx=0).stop 2023-10-06T02:27:54.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.016+0000 7f1d6d79d700 1 -- 172.21.15.31:0/2312082248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d50003bf0 msgr2=0x7f1d68155d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:54.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.016+0000 7f1d6d79d700 1 --2- 172.21.15.31:0/2312082248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d50003bf0 0x7f1d68155d80 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f1d5800e790 tx=0x7f1d58019000 comp rx=0 tx=0).stop 2023-10-06T02:27:54.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.016+0000 7f1d6d79d700 1 -- 172.21.15.31:0/2312082248 shutdown_connections 2023-10-06T02:27:54.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.016+0000 7f1d6d79d700 1 --2- 172.21.15.31:0/2312082248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d680ae060 0x7f1d680ae4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:54.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.016+0000 7f1d6d79d700 1 --2- 172.21.15.31:0/2312082248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d50003bf0 0x7f1d68155d80 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:54.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.016+0000 7f1d6d79d700 1 --2- 172.21.15.31:0/2312082248 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f1d54067050 0x7f1d54069500 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:54.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.016+0000 7f1d6d79d700 1 --2- 172.21.15.31:0/2312082248 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1d680c7620 0x7f1d680c7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:54.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.016+0000 7f1d6d79d700 1 -- 172.21.15.31:0/2312082248 >> 172.21.15.31:0/2312082248 conn(0x7f1d6806d170 msgr2=0x7f1d6813d000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:54.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.017+0000 7f1d6d79d700 1 -- 172.21.15.31:0/2312082248 shutdown_connections 2023-10-06T02:27:54.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.017+0000 7f1d6d79d700 1 -- 172.21.15.31:0/2312082248 wait complete. 2023-10-06T02:27:54.036 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:27:54.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.224+0000 7f17b359d700 1 Processor -- start 2023-10-06T02:27:54.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.224+0000 7f17b359d700 1 -- start start 2023-10-06T02:27:54.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.225+0000 7f17b359d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f17ac142fd0 0x7f17ac0c7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:54.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.225+0000 7f17b359d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f17ac0c7dc0 0x7f17ac06c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:54.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.225+0000 7f17b359d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17ac0ca480 con 0x7f17ac142c00 2023-10-06T02:27:54.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.225+0000 7f17b359d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17ac0c8240 con 0x7f17ac0c7dc0 2023-10-06T02:27:54.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.225+0000 7f17b359d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17ac15e240 con 0x7f17ac142fd0 2023-10-06T02:27:54.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.226+0000 7f17b1d9a700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f17ac142fd0 0x7f17ac0c7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:54.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.226+0000 7f17b2d9c700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f17ac0c7dc0 0x7f17ac06c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:54.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.226+0000 7f17b259b700 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f17ac142c00 0x7f17ac06b7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:35256/0 (socket says 172.21.15.31:35256) 2023-10-06T02:27:54.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.226+0000 7f17b259b700 1 -- 172.21.15.31:0/2996523315 learned_addr learned my addr 172.21.15.31:0/2996523315 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:27:54.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.226+0000 7f17b1599700 1 -- 172.21.15.31:0/2996523315 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2312638411 0 0) 0x7f17ac0ca480 con 0x7f17ac142c00 2023-10-06T02:27:54.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.226+0000 7f17b1599700 1 -- 172.21.15.31:0/2996523315 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1798003570 con 0x7f17ac142c00 2023-10-06T02:27:54.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.227+0000 7f17b1d9a700 1 -- 172.21.15.31:0/2996523315 >> v2:172.21.15.62:3300/0 conn(0x7f17ac0c7dc0 msgr2=0x7f17ac06c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:54.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.227+0000 7f17b1d9a700 1 --2- 172.21.15.31:0/2996523315 >> v2:172.21.15.62:3300/0 conn(0x7f17ac0c7dc0 0x7f17ac06c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:54.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.227+0000 7f17b1d9a700 1 -- 172.21.15.31:0/2996523315 >> v1:172.21.15.62:6789/0 conn(0x7f17ac142c00 legacy=0x7f17ac06b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:54.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.227+0000 7f17b1d9a700 1 -- 172.21.15.31:0/2996523315 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17ac06b350 con 0x7f17ac142fd0 2023-10-06T02:27:54.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.227+0000 7f17b2d9c700 1 --2- 172.21.15.31:0/2996523315 >> v2:172.21.15.62:3300/0 conn(0x7f17ac0c7dc0 0x7f17ac06c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:27:54.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.227+0000 7f17b1d9a700 1 --2- 172.21.15.31:0/2996523315 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f17ac142fd0 0x7f17ac0c7880 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f179c0098e0 tx=0x7f179c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e982b81ae402ca77 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:54.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.227+0000 7f17b1599700 1 -- 172.21.15.31:0/2996523315 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f179c011630 con 0x7f17ac142fd0 2023-10-06T02:27:54.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.227+0000 7f17b1599700 1 -- 172.21.15.31:0/2996523315 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f179c011790 con 0x7f17ac142fd0 2023-10-06T02:27:54.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.227+0000 7f17b1599700 1 -- 172.21.15.31:0/2996523315 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f179c011b30 con 0x7f17ac142fd0 2023-10-06T02:27:54.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.228+0000 7f17b359d700 1 -- 172.21.15.31:0/2996523315 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f17ac142fd0 msgr2=0x7f17ac0c7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:54.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.228+0000 7f17b359d700 1 --2- 172.21.15.31:0/2996523315 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f17ac142fd0 0x7f17ac0c7880 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f179c0098e0 tx=0x7f179c0091a0 comp rx=0 tx=0).stop 2023-10-06T02:27:54.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.228+0000 7f17b359d700 1 -- 172.21.15.31:0/2996523315 shutdown_connections 2023-10-06T02:27:54.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.228+0000 7f17b359d700 1 --2- 172.21.15.31:0/2996523315 >> v2:172.21.15.62:3300/0 conn(0x7f17ac0c7dc0 0x7f17ac06c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:54.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.228+0000 7f17b359d700 1 --2- 172.21.15.31:0/2996523315 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f17ac142fd0 0x7f17ac0c7880 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:54.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.228+0000 7f17b359d700 1 -- 172.21.15.31:0/2996523315 >> 172.21.15.31:0/2996523315 conn(0x7f17ac0675a0 msgr2=0x7f17ac0679a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:54.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.228+0000 7f17b359d700 1 -- 172.21.15.31:0/2996523315 shutdown_connections 2023-10-06T02:27:54.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.228+0000 7f17b359d700 1 -- 172.21.15.31:0/2996523315 wait complete. 2023-10-06T02:27:54.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.229+0000 7f17b359d700 1 Processor -- start 2023-10-06T02:27:54.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.230+0000 7f17b359d700 1 -- start start 2023-10-06T02:27:54.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.230+0000 7f17b359d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f17ac0c7dc0 0x7f17ac15c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:54.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.230+0000 7f17b359d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17ac142c00 0x7f17ac0c6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:54.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.230+0000 7f17b359d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17ac142fd0 0x7f17ac13e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:54.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.230+0000 7f17b359d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17ac0c72e0 con 0x7f17ac142fd0 2023-10-06T02:27:54.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.230+0000 7f17b359d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17ac15d2f0 con 0x7f17ac142c00 2023-10-06T02:27:54.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.230+0000 7f17b359d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17ac13eab0 con 0x7f17ac0c7dc0 2023-10-06T02:27:54.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.231+0000 7f17b2d9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17ac142fd0 0x7f17ac13e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:54.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.231+0000 7f17b2d9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17ac142fd0 0x7f17ac13e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41588/0 (socket says 172.21.15.31:41588) 2023-10-06T02:27:54.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.231+0000 7f17b2d9c700 1 -- 172.21.15.31:0/3497581013 learned_addr learned my addr 172.21.15.31:0/3497581013 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:54.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.231+0000 7f17b1d9a700 1 --2- 172.21.15.31:0/3497581013 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17ac142c00 0x7f17ac0c6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:54.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.231+0000 7f17b259b700 1 --2- 172.21.15.31:0/3497581013 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f17ac0c7dc0 0x7f17ac15c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:54.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.231+0000 7f17b2d9c700 1 -- 172.21.15.31:0/3497581013 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f17ac0c7dc0 msgr2=0x7f17ac15c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:54.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.231+0000 7f17b2d9c700 1 --2- 172.21.15.31:0/3497581013 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f17ac0c7dc0 0x7f17ac15c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:54.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.231+0000 7f17b2d9c700 1 -- 172.21.15.31:0/3497581013 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17ac142c00 msgr2=0x7f17ac0c6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:54.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.231+0000 7f17b2d9c700 1 --2- 172.21.15.31:0/3497581013 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17ac142c00 0x7f17ac0c6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:54.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.231+0000 7f17b2d9c700 1 -- 172.21.15.31:0/3497581013 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17a4009a20 con 0x7f17ac142fd0 2023-10-06T02:27:54.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.232+0000 7f17b2d9c700 1 --2- 172.21.15.31:0/3497581013 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17ac142fd0 0x7f17ac13e570 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f17a4009fc0 tx=0x7f17a400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:54.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.232+0000 7f17a27fc700 1 -- 172.21.15.31:0/3497581013 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a400cd30 con 0x7f17ac142fd0 2023-10-06T02:27:54.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.232+0000 7f17a27fc700 1 -- 172.21.15.31:0/3497581013 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17a400f040 con 0x7f17ac142fd0 2023-10-06T02:27:54.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.232+0000 7f17b359d700 1 -- 172.21.15.31:0/3497581013 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f179c009420 con 0x7f17ac142fd0 2023-10-06T02:27:54.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.232+0000 7f17a27fc700 1 -- 172.21.15.31:0/3497581013 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a40146d0 con 0x7f17ac142fd0 2023-10-06T02:27:54.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.232+0000 7f17b359d700 1 -- 172.21.15.31:0/3497581013 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17ac15c9c0 con 0x7f17ac142fd0 2023-10-06T02:27:54.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.233+0000 7f17b359d700 1 -- 172.21.15.31:0/3497581013 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17ac15d460 con 0x7f17ac142fd0 2023-10-06T02:27:54.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.234+0000 7f17a27fc700 1 -- 172.21.15.31:0/3497581013 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f17a4003020 con 0x7f17ac142fd0 2023-10-06T02:27:54.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.235+0000 7f17a27fc700 1 --2- 172.21.15.31:0/3497581013 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f1790067050 0x7f1790069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:54.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.235+0000 7f17a27fc700 1 -- 172.21.15.31:0/3497581013 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f17a4085160 con 0x7f17ac142fd0 2023-10-06T02:27:54.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.235+0000 7f17b259b700 1 --2- 172.21.15.31:0/3497581013 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f1790067050 0x7f1790069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:54.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.236+0000 7f17b259b700 1 --2- 172.21.15.31:0/3497581013 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f1790067050 0x7f1790069500 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f17a80058d0 tx=0x7f17a8009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:54.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.239+0000 7f17a27fc700 1 -- 172.21.15.31:0/3497581013 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f17a4052300 con 0x7f17ac142fd0 2023-10-06T02:27:54.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.441+0000 7f17b359d700 1 -- 172.21.15.31:0/3497581013 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f17ac04c3c0 con 0x7f17ac142fd0 2023-10-06T02:27:54.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.442+0000 7f17a27fc700 1 -- 172.21.15.31:0/3497581013 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+13603 (secure 0 0 0) 0x7f17a4055920 con 0x7f17ac142fd0 2023-10-06T02:27:54.443 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:27:54.443 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":83,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:27:53.513019+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:27:54.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.444+0000 7f17b359d700 1 -- 172.21.15.31:0/3497581013 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f1790067050 msgr2=0x7f1790069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:54.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.444+0000 7f17b359d700 1 --2- 172.21.15.31:0/3497581013 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f1790067050 0x7f1790069500 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f17a80058d0 tx=0x7f17a8009040 comp rx=0 tx=0).stop 2023-10-06T02:27:54.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.444+0000 7f17b359d700 1 -- 172.21.15.31:0/3497581013 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17ac142fd0 msgr2=0x7f17ac13e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:54.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.444+0000 7f17b359d700 1 --2- 172.21.15.31:0/3497581013 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17ac142fd0 0x7f17ac13e570 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f17a4009fc0 tx=0x7f17a400b040 comp rx=0 tx=0).stop 2023-10-06T02:27:54.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.444+0000 7f17b359d700 1 -- 172.21.15.31:0/3497581013 shutdown_connections 2023-10-06T02:27:54.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.444+0000 7f17b359d700 1 --2- 172.21.15.31:0/3497581013 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17ac142fd0 0x7f17ac13e570 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:54.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.444+0000 7f17b359d700 1 --2- 172.21.15.31:0/3497581013 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f1790067050 0x7f1790069500 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:54.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.444+0000 7f17b359d700 1 --2- 172.21.15.31:0/3497581013 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f17ac0c7dc0 0x7f17ac15c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:54.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.444+0000 7f17b359d700 1 --2- 172.21.15.31:0/3497581013 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17ac142c00 0x7f17ac0c6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:54.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.444+0000 7f17b359d700 1 -- 172.21.15.31:0/3497581013 >> 172.21.15.31:0/3497581013 conn(0x7f17ac0675a0 msgr2=0x7f17ac13f330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:54.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.445+0000 7f17b359d700 1 -- 172.21.15.31:0/3497581013 shutdown_connections 2023-10-06T02:27:54.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.445+0000 7f17b359d700 1 -- 172.21.15.31:0/3497581013 wait complete. 2023-10-06T02:27:54.463 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T02:22:53.183425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T02:23:00.046967+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-10-06T02:27:54.463 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T02:27:54.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.653+0000 7f75e6b9d700 1 Processor -- start 2023-10-06T02:27:54.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.653+0000 7f75e6b9d700 1 -- start start 2023-10-06T02:27:54.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.655+0000 7f75e6b9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f75e006db20 0x7f75e015a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:54.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.655+0000 7f75e6b9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f75e00caec0 0x7f75e015afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:54.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.655+0000 7f75e6b9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75e015ada0 con 0x7f75e0153c90 2023-10-06T02:27:54.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.655+0000 7f75e6b9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75e006ca80 con 0x7f75e00caec0 2023-10-06T02:27:54.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.655+0000 7f75e6b9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75e006cd60 con 0x7f75e006db20 2023-10-06T02:27:54.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.655+0000 7f75e639c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f75e00caec0 0x7f75e015afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:54.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.655+0000 7f75e639c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f75e00caec0 0x7f75e015afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41600/0 (socket says 172.21.15.31:41600) 2023-10-06T02:27:54.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.655+0000 7f75e639c700 1 -- 172.21.15.31:0/1107738095 learned_addr learned my addr 172.21.15.31:0/1107738095 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:54.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.655+0000 7f75e639c700 1 -- 172.21.15.31:0/1107738095 >> v2:172.21.15.62:3300/0 conn(0x7f75e006db20 msgr2=0x7f75e015a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:54.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.655+0000 7f75e539a700 1 --2- 172.21.15.31:0/1107738095 >> v2:172.21.15.62:3300/0 conn(0x7f75e006db20 0x7f75e015a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:54.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.655+0000 7f75e639c700 1 --2- 172.21.15.31:0/1107738095 >> v2:172.21.15.62:3300/0 conn(0x7f75e006db20 0x7f75e015a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:54.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.655+0000 7f75e639c700 1 -- 172.21.15.31:0/1107738095 >> v1:172.21.15.62:6789/0 conn(0x7f75e0153c90 legacy=0x7f75e00ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:54.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.656+0000 7f75e639c700 1 -- 172.21.15.31:0/1107738095 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75e006d040 con 0x7f75e00caec0 2023-10-06T02:27:54.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.656+0000 7f75e539a700 1 --2- 172.21.15.31:0/1107738095 >> v2:172.21.15.62:3300/0 conn(0x7f75e006db20 0x7f75e015a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:27:54.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.656+0000 7f75e639c700 1 --2- 172.21.15.31:0/1107738095 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75e00caec0 0x7f75e015afb0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f75e00cc300 tx=0x7f75d8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eee693b40c3f9538 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:54.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.656+0000 7f75e4b99700 1 -- 172.21.15.31:0/1107738095 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d800e070 con 0x7f75e00caec0 2023-10-06T02:27:54.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.656+0000 7f75e4b99700 1 -- 172.21.15.31:0/1107738095 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75d800ba50 con 0x7f75e00caec0 2023-10-06T02:27:54.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.656+0000 7f75e4b99700 1 -- 172.21.15.31:0/1107738095 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d80092e0 con 0x7f75e00caec0 2023-10-06T02:27:54.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.657+0000 7f75e6b9d700 1 -- 172.21.15.31:0/1107738095 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75e00caec0 msgr2=0x7f75e015afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:54.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.657+0000 7f75e6b9d700 1 --2- 172.21.15.31:0/1107738095 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75e00caec0 0x7f75e015afb0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f75e00cc300 tx=0x7f75d8009840 comp rx=0 tx=0).stop 2023-10-06T02:27:54.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.657+0000 7f75e6b9d700 1 -- 172.21.15.31:0/1107738095 shutdown_connections 2023-10-06T02:27:54.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.657+0000 7f75e6b9d700 1 --2- 172.21.15.31:0/1107738095 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75e00caec0 0x7f75e015afb0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:54.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.657+0000 7f75e6b9d700 1 --2- 172.21.15.31:0/1107738095 >> v2:172.21.15.62:3300/0 conn(0x7f75e006db20 0x7f75e015a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:54.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.657+0000 7f75e6b9d700 1 -- 172.21.15.31:0/1107738095 >> 172.21.15.31:0/1107738095 conn(0x7f75e006e720 msgr2=0x7f75e006eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:54.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.657+0000 7f75e6b9d700 1 -- 172.21.15.31:0/1107738095 shutdown_connections 2023-10-06T02:27:54.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.657+0000 7f75e6b9d700 1 -- 172.21.15.31:0/1107738095 wait complete. 2023-10-06T02:27:54.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.658+0000 7f75e6b9d700 1 Processor -- start 2023-10-06T02:27:54.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.658+0000 7f75e6b9d700 1 -- start start 2023-10-06T02:27:54.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.659+0000 7f75e6b9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f75e006db20 0x7f75e015c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:54.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.659+0000 7f75e6b9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75e00caec0 0x7f75e00b12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:54.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.659+0000 7f75e6b9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75e0153c90 0x7f75e0140300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:54.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.659+0000 7f75e6b9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e015c7b0 con 0x7f75e00caec0 2023-10-06T02:27:54.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.659+0000 7f75e6b9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e015ca90 con 0x7f75e0153c90 2023-10-06T02:27:54.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.659+0000 7f75e6b9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75e015cd70 con 0x7f75e006db20 2023-10-06T02:27:54.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.660+0000 7f75e539a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75e00caec0 0x7f75e00b12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:54.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.660+0000 7f75e539a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75e00caec0 0x7f75e00b12c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41616/0 (socket says 172.21.15.31:41616) 2023-10-06T02:27:54.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.660+0000 7f75e539a700 1 -- 172.21.15.31:0/1436616496 learned_addr learned my addr 172.21.15.31:0/1436616496 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:54.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.660+0000 7f75e5b9b700 1 --2- 172.21.15.31:0/1436616496 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f75e006db20 0x7f75e015c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:54.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.660+0000 7f75e639c700 1 --2- 172.21.15.31:0/1436616496 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75e0153c90 0x7f75e0140300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:54.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.660+0000 7f75e539a700 1 -- 172.21.15.31:0/1436616496 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f75e006db20 msgr2=0x7f75e015c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:54.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.660+0000 7f75e539a700 1 --2- 172.21.15.31:0/1436616496 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f75e006db20 0x7f75e015c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:54.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.660+0000 7f75e539a700 1 -- 172.21.15.31:0/1436616496 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75e0153c90 msgr2=0x7f75e0140300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:54.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.660+0000 7f75e539a700 1 --2- 172.21.15.31:0/1436616496 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75e0153c90 0x7f75e0140300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:54.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.660+0000 7f75e539a700 1 -- 172.21.15.31:0/1436616496 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75d8009ae0 con 0x7f75e00caec0 2023-10-06T02:27:54.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.660+0000 7f75e5b9b700 1 --2- 172.21.15.31:0/1436616496 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f75e006db20 0x7f75e015c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:27:54.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.660+0000 7f75e539a700 1 --2- 172.21.15.31:0/1436616496 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75e00caec0 0x7f75e00b12c0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f75d000c850 tx=0x7f75d0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:54.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.661+0000 7f75d5ffb700 1 -- 172.21.15.31:0/1436616496 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d0011600 con 0x7f75e00caec0 2023-10-06T02:27:54.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.661+0000 7f75d5ffb700 1 -- 172.21.15.31:0/1436616496 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75d0011760 con 0x7f75e00caec0 2023-10-06T02:27:54.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.661+0000 7f75d5ffb700 1 -- 172.21.15.31:0/1436616496 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d0011b00 con 0x7f75e00caec0 2023-10-06T02:27:54.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.661+0000 7f75e6b9d700 1 -- 172.21.15.31:0/1436616496 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75e015d050 con 0x7f75e00caec0 2023-10-06T02:27:54.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.661+0000 7f75e6b9d700 1 -- 172.21.15.31:0/1436616496 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75e0140840 con 0x7f75e00caec0 2023-10-06T02:27:54.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.662+0000 7f75e6b9d700 1 -- 172.21.15.31:0/1436616496 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75e004c3c0 con 0x7f75e00caec0 2023-10-06T02:27:54.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.663+0000 7f75d5ffb700 1 -- 172.21.15.31:0/1436616496 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f75d00223a0 con 0x7f75e00caec0 2023-10-06T02:27:54.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.664+0000 7f75d5ffb700 1 --2- 172.21.15.31:0/1436616496 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f75c4067050 0x7f75c4069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:54.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.664+0000 7f75d5ffb700 1 -- 172.21.15.31:0/1436616496 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f75d0084900 con 0x7f75e00caec0 2023-10-06T02:27:54.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.664+0000 7f75e5b9b700 1 --2- 172.21.15.31:0/1436616496 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f75c4067050 0x7f75c4069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:54.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.665+0000 7f75e5b9b700 1 --2- 172.21.15.31:0/1436616496 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f75c4067050 0x7f75c4069500 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f75dc007fd0 tx=0x7f75dc009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:54.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.668+0000 7f75d5ffb700 1 -- 172.21.15.31:0/1436616496 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f75d0051aa0 con 0x7f75e00caec0 2023-10-06T02:27:54.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.867+0000 7f75e6b9d700 1 -- 172.21.15.31:0/1436616496 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f75e0159bc0 con 0x7f75e00caec0 2023-10-06T02:27:54.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.868+0000 7f75d5ffb700 1 -- 172.21.15.31:0/1436616496 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7f75d001e0a0 con 0x7f75e00caec0 2023-10-06T02:27:54.868 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-10-06T02:27:54.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.870+0000 7f75e6b9d700 1 -- 172.21.15.31:0/1436616496 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f75c4067050 msgr2=0x7f75c4069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:54.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.870+0000 7f75e6b9d700 1 --2- 172.21.15.31:0/1436616496 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f75c4067050 0x7f75c4069500 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f75dc007fd0 tx=0x7f75dc009040 comp rx=0 tx=0).stop 2023-10-06T02:27:54.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.870+0000 7f75e6b9d700 1 -- 172.21.15.31:0/1436616496 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75e00caec0 msgr2=0x7f75e00b12c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:54.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.870+0000 7f75e6b9d700 1 --2- 172.21.15.31:0/1436616496 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75e00caec0 0x7f75e00b12c0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f75d000c850 tx=0x7f75d0008040 comp rx=0 tx=0).stop 2023-10-06T02:27:54.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.870+0000 7f75e6b9d700 1 -- 172.21.15.31:0/1436616496 shutdown_connections 2023-10-06T02:27:54.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.870+0000 7f75e6b9d700 1 --2- 172.21.15.31:0/1436616496 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75e0153c90 0x7f75e0140300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:54.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.870+0000 7f75e6b9d700 1 --2- 172.21.15.31:0/1436616496 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f75c4067050 0x7f75c4069500 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:54.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.870+0000 7f75e6b9d700 1 --2- 172.21.15.31:0/1436616496 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f75e006db20 0x7f75e015c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:54.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.870+0000 7f75e6b9d700 1 --2- 172.21.15.31:0/1436616496 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75e00caec0 0x7f75e00b12c0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:54.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.870+0000 7f75e6b9d700 1 -- 172.21.15.31:0/1436616496 >> 172.21.15.31:0/1436616496 conn(0x7f75e006e720 msgr2=0x7f75e0159360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:54.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.870+0000 7f75e6b9d700 1 -- 172.21.15.31:0/1436616496 shutdown_connections 2023-10-06T02:27:54.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:54.870+0000 7f75e6b9d700 1 -- 172.21.15.31:0/1436616496 wait complete. 2023-10-06T02:27:54.888 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T02:27:55.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.078+0000 7f312c9fe700 1 Processor -- start 2023-10-06T02:27:55.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.078+0000 7f312c9fe700 1 -- start start 2023-10-06T02:27:55.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.079+0000 7f312c9fe700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3128153a00 0x7f3128153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:55.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.079+0000 7f312c9fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f312815de50 0x7f312815a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:55.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.079+0000 7f312c9fe700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f312815af60 con 0x7f31280cea00 2023-10-06T02:27:55.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.079+0000 7f312c9fe700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f312815d010 con 0x7f3128153a00 2023-10-06T02:27:55.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.079+0000 7f312c9fe700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f312815b240 con 0x7f312815de50 2023-10-06T02:27:55.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.080+0000 7f312759e700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f31280cea00 0x7f31280cee80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:59490/0 (socket says 172.21.15.31:59490) 2023-10-06T02:27:55.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.080+0000 7f312759e700 1 -- 172.21.15.31:0/657332479 learned_addr learned my addr 172.21.15.31:0/657332479 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:27:55.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.080+0000 7f3126d9d700 1 --2- 172.21.15.31:0/657332479 >> v2:172.21.15.62:3300/0 conn(0x7f3128153a00 0x7f3128153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:55.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.080+0000 7f312659c700 1 --2- 172.21.15.31:0/657332479 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f312815de50 0x7f312815a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:55.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.080+0000 7f3125d9b700 1 -- 172.21.15.31:0/657332479 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2578675150 0 0) 0x7f312815af60 con 0x7f31280cea00 2023-10-06T02:27:55.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.080+0000 7f3125d9b700 1 -- 172.21.15.31:0/657332479 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3110003570 con 0x7f31280cea00 2023-10-06T02:27:55.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.080+0000 7f3126d9d700 1 -- 172.21.15.31:0/657332479 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f312815de50 msgr2=0x7f312815a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:55.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.080+0000 7f3126d9d700 1 --2- 172.21.15.31:0/657332479 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f312815de50 0x7f312815a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:55.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.080+0000 7f3126d9d700 1 -- 172.21.15.31:0/657332479 >> v1:172.21.15.31:6789/0 conn(0x7f31280cea00 legacy=0x7f31280cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:55.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.080+0000 7f3126d9d700 1 -- 172.21.15.31:0/657332479 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f312815b520 con 0x7f3128153a00 2023-10-06T02:27:55.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.080+0000 7f312659c700 1 --2- 172.21.15.31:0/657332479 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f312815de50 0x7f312815a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:27:55.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.081+0000 7f3126d9d700 1 --2- 172.21.15.31:0/657332479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3128153a00 0x7f3128153dd0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f31180098e0 tx=0x7f31180091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8cb2f0ec3d2c005 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:55.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.081+0000 7f3125d9b700 1 -- 172.21.15.31:0/657332479 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3118011630 con 0x7f3128153a00 2023-10-06T02:27:55.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.081+0000 7f3125d9b700 1 -- 172.21.15.31:0/657332479 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3118011790 con 0x7f3128153a00 2023-10-06T02:27:55.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.081+0000 7f3125d9b700 1 -- 172.21.15.31:0/657332479 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3118011b30 con 0x7f3128153a00 2023-10-06T02:27:55.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.081+0000 7f312c9fe700 1 -- 172.21.15.31:0/657332479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3128153a00 msgr2=0x7f3128153dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:55.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.081+0000 7f312c9fe700 1 --2- 172.21.15.31:0/657332479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3128153a00 0x7f3128153dd0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f31180098e0 tx=0x7f31180091a0 comp rx=0 tx=0).stop 2023-10-06T02:27:55.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.082+0000 7f312c9fe700 1 -- 172.21.15.31:0/657332479 shutdown_connections 2023-10-06T02:27:55.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.082+0000 7f312c9fe700 1 --2- 172.21.15.31:0/657332479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3128153a00 0x7f3128153dd0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:55.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.082+0000 7f312c9fe700 1 --2- 172.21.15.31:0/657332479 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f312815de50 0x7f312815a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:55.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.082+0000 7f312c9fe700 1 -- 172.21.15.31:0/657332479 >> 172.21.15.31:0/657332479 conn(0x7f31280703a0 msgr2=0x7f31280707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:55.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.082+0000 7f312c9fe700 1 -- 172.21.15.31:0/657332479 shutdown_connections 2023-10-06T02:27:55.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.082+0000 7f312c9fe700 1 -- 172.21.15.31:0/657332479 wait complete. 2023-10-06T02:27:55.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.083+0000 7f312c9fe700 1 Processor -- start 2023-10-06T02:27:55.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.083+0000 7f312c9fe700 1 -- start start 2023-10-06T02:27:55.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.084+0000 7f312c9fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f31280cea00 0x7f3128153190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:55.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.084+0000 7f312c9fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3128153a00 0x7f31280c9d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:55.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.084+0000 7f312c9fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f312815de50 0x7f31280b1c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:55.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.084+0000 7f312c9fe700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31280b58b0 con 0x7f3128153a00 2023-10-06T02:27:55.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.084+0000 7f312c9fe700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31280b49c0 con 0x7f312815de50 2023-10-06T02:27:55.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.084+0000 7f312c9fe700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31280b4ca0 con 0x7f31280cea00 2023-10-06T02:27:55.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.084+0000 7f312659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3128153a00 0x7f31280c9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:55.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.084+0000 7f312659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3128153a00 0x7f31280c9d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41632/0 (socket says 172.21.15.31:41632) 2023-10-06T02:27:55.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.084+0000 7f312659c700 1 -- 172.21.15.31:0/2255766182 learned_addr learned my addr 172.21.15.31:0/2255766182 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:55.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.084+0000 7f3126d9d700 1 --2- 172.21.15.31:0/2255766182 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f31280cea00 0x7f3128153190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:55.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.084+0000 7f312759e700 1 --2- 172.21.15.31:0/2255766182 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f312815de50 0x7f31280b1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:55.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.085+0000 7f312659c700 1 -- 172.21.15.31:0/2255766182 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f31280cea00 msgr2=0x7f3128153190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:55.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.085+0000 7f312659c700 1 --2- 172.21.15.31:0/2255766182 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f31280cea00 0x7f3128153190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:55.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.085+0000 7f312659c700 1 -- 172.21.15.31:0/2255766182 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f312815de50 msgr2=0x7f31280b1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:55.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.085+0000 7f312659c700 1 --2- 172.21.15.31:0/2255766182 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f312815de50 0x7f31280b1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:55.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.085+0000 7f312659c700 1 -- 172.21.15.31:0/2255766182 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f311c009420 con 0x7f3128153a00 2023-10-06T02:27:55.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.085+0000 7f312659c700 1 --2- 172.21.15.31:0/2255766182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3128153a00 0x7f31280c9d70 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f311c00ec80 tx=0x7f311c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:55.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.085+0000 7f310effd700 1 -- 172.21.15.31:0/2255766182 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311c00c850 con 0x7f3128153a00 2023-10-06T02:27:55.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.085+0000 7f310effd700 1 -- 172.21.15.31:0/2255766182 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f311c00c9b0 con 0x7f3128153a00 2023-10-06T02:27:55.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.085+0000 7f310effd700 1 -- 172.21.15.31:0/2255766182 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311c00cd30 con 0x7f3128153a00 2023-10-06T02:27:55.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.085+0000 7f312c9fe700 1 -- 172.21.15.31:0/2255766182 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3118009420 con 0x7f3128153a00 2023-10-06T02:27:55.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.086+0000 7f312c9fe700 1 -- 172.21.15.31:0/2255766182 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31280b5260 con 0x7f3128153a00 2023-10-06T02:27:55.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.087+0000 7f312c9fe700 1 -- 172.21.15.31:0/2255766182 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f312804c3c0 con 0x7f3128153a00 2023-10-06T02:27:55.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.087+0000 7f310effd700 1 -- 172.21.15.31:0/2255766182 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f311c013680 con 0x7f3128153a00 2023-10-06T02:27:55.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.089+0000 7f310effd700 1 --2- 172.21.15.31:0/2255766182 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f3114067050 0x7f3114069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:55.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.090+0000 7f310effd700 1 -- 172.21.15.31:0/2255766182 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f311c084a20 con 0x7f3128153a00 2023-10-06T02:27:55.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.090+0000 7f3126d9d700 1 --2- 172.21.15.31:0/2255766182 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f3114067050 0x7f3114069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:55.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.091+0000 7f3126d9d700 1 --2- 172.21.15.31:0/2255766182 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f3114067050 0x7f3114069500 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f31180091e0 tx=0x7f3118017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:55.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.094+0000 7f310effd700 1 -- 172.21.15.31:0/2255766182 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f311c055800 con 0x7f3128153a00 2023-10-06T02:27:55.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.297+0000 7f312c9fe700 1 -- 172.21.15.31:0/2255766182 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f312815b800 con 0x7f3128153a00 2023-10-06T02:27:55.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.297+0000 7f310effd700 1 -- 172.21.15.31:0/2255766182 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v83) v1 ==== 92+0+10 (secure 0 0 0) 0x7f311c0170a0 con 0x7f3128153a00 2023-10-06T02:27:55.298 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 8 2023-10-06T02:27:55.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.300+0000 7f312c9fe700 1 -- 172.21.15.31:0/2255766182 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f3114067050 msgr2=0x7f3114069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:55.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.300+0000 7f312c9fe700 1 --2- 172.21.15.31:0/2255766182 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f3114067050 0x7f3114069500 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f31180091e0 tx=0x7f3118017000 comp rx=0 tx=0).stop 2023-10-06T02:27:55.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.300+0000 7f312c9fe700 1 -- 172.21.15.31:0/2255766182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3128153a00 msgr2=0x7f31280c9d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:55.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.300+0000 7f312c9fe700 1 --2- 172.21.15.31:0/2255766182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3128153a00 0x7f31280c9d70 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f311c00ec80 tx=0x7f311c0091a0 comp rx=0 tx=0).stop 2023-10-06T02:27:55.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.300+0000 7f312c9fe700 1 -- 172.21.15.31:0/2255766182 shutdown_connections 2023-10-06T02:27:55.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.300+0000 7f312c9fe700 1 --2- 172.21.15.31:0/2255766182 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f312815de50 0x7f31280b1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:55.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.300+0000 7f312c9fe700 1 --2- 172.21.15.31:0/2255766182 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f3114067050 0x7f3114069500 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:55.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.300+0000 7f312c9fe700 1 --2- 172.21.15.31:0/2255766182 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f31280cea00 0x7f3128153190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:55.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.300+0000 7f312c9fe700 1 --2- 172.21.15.31:0/2255766182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3128153a00 0x7f31280c9d70 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:55.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.300+0000 7f312c9fe700 1 -- 172.21.15.31:0/2255766182 >> 172.21.15.31:0/2255766182 conn(0x7f31280703a0 msgr2=0x7f31280b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:55.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.300+0000 7f312c9fe700 1 -- 172.21.15.31:0/2255766182 shutdown_connections 2023-10-06T02:27:55.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.300+0000 7f312c9fe700 1 -- 172.21.15.31:0/2255766182 wait complete. 2023-10-06T02:27:55.318 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T02:27:55.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.503+0000 7f7d62d9d700 1 Processor -- start 2023-10-06T02:27:55.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.503+0000 7f7d62d9d700 1 -- start start 2023-10-06T02:27:55.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.504+0000 7f7d62d9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7d5c159a10 0x7f7d5c0ca7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:55.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.504+0000 7f7d62d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7d5c070410 0x7f7d5c0cacf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:55.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.505+0000 7f7d62d9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d5c0c7420 con 0x7f7d5c159640 2023-10-06T02:27:55.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.505+0000 7f7d62d9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d5c06d820 con 0x7f7d5c159a10 2023-10-06T02:27:55.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.505+0000 7f7d62d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5c06db00 con 0x7f7d5c070410 2023-10-06T02:27:55.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.505+0000 7f7d6159a700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7d5c159a10 0x7f7d5c0ca7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:55.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.505+0000 7f7d61d9b700 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f7d5c159640 0x7f7d5c06d060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:35268/0 (socket says 172.21.15.31:35268) 2023-10-06T02:27:55.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.505+0000 7f7d6159a700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7d5c159a10 0x7f7d5c0ca7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:40278/0 (socket says 172.21.15.31:40278) 2023-10-06T02:27:55.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.505+0000 7f7d61d9b700 1 -- 172.21.15.31:0/1528050185 learned_addr learned my addr 172.21.15.31:0/1528050185 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:27:55.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.505+0000 7f7d6259c700 1 --2- 172.21.15.31:0/1528050185 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7d5c070410 0x7f7d5c0cacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:55.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.505+0000 7f7d6159a700 1 -- 172.21.15.31:0/1528050185 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7d5c070410 msgr2=0x7f7d5c0cacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:55.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.505+0000 7f7d6159a700 1 --2- 172.21.15.31:0/1528050185 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7d5c070410 0x7f7d5c0cacf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:55.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.505+0000 7f7d6159a700 1 -- 172.21.15.31:0/1528050185 >> v1:172.21.15.62:6789/0 conn(0x7f7d5c159640 legacy=0x7f7d5c06d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:55.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.505+0000 7f7d6159a700 1 -- 172.21.15.31:0/1528050185 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d5c06dde0 con 0x7f7d5c159a10 2023-10-06T02:27:55.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.506+0000 7f7d6159a700 1 --2- 172.21.15.31:0/1528050185 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d5c159a10 0x7f7d5c0ca7b0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c0098e0 tx=0x7f7d4c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=245e7aa5dc12ccf1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:55.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.506+0000 7f7d60d99700 1 -- 172.21.15.31:0/1528050185 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d4c00b640 con 0x7f7d5c159a10 2023-10-06T02:27:55.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.506+0000 7f7d60d99700 1 -- 172.21.15.31:0/1528050185 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d4c00b7a0 con 0x7f7d5c159a10 2023-10-06T02:27:55.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.506+0000 7f7d60d99700 1 -- 172.21.15.31:0/1528050185 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d4c00bb40 con 0x7f7d5c159a10 2023-10-06T02:27:55.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.507+0000 7f7d62d9d700 1 -- 172.21.15.31:0/1528050185 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d5c159a10 msgr2=0x7f7d5c0ca7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:55.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.507+0000 7f7d62d9d700 1 --2- 172.21.15.31:0/1528050185 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d5c159a10 0x7f7d5c0ca7b0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c0098e0 tx=0x7f7d4c0091a0 comp rx=0 tx=0).stop 2023-10-06T02:27:55.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.507+0000 7f7d62d9d700 1 -- 172.21.15.31:0/1528050185 shutdown_connections 2023-10-06T02:27:55.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.507+0000 7f7d62d9d700 1 --2- 172.21.15.31:0/1528050185 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7d5c070410 0x7f7d5c0cacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:55.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.507+0000 7f7d62d9d700 1 --2- 172.21.15.31:0/1528050185 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d5c159a10 0x7f7d5c0ca7b0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:55.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.507+0000 7f7d62d9d700 1 -- 172.21.15.31:0/1528050185 >> 172.21.15.31:0/1528050185 conn(0x7f7d5c06e720 msgr2=0x7f7d5c06eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:55.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.507+0000 7f7d62d9d700 1 -- 172.21.15.31:0/1528050185 shutdown_connections 2023-10-06T02:27:55.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.507+0000 7f7d62d9d700 1 -- 172.21.15.31:0/1528050185 wait complete. 2023-10-06T02:27:55.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.508+0000 7f7d62d9d700 1 Processor -- start 2023-10-06T02:27:55.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.509+0000 7f7d62d9d700 1 -- start start 2023-10-06T02:27:55.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.510+0000 7f7d62d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d5c070410 0x7f7d5c14f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:55.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.510+0000 7f7d62d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d5c159640 0x7f7d5c159e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:55.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.510+0000 7f7d62d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7d5c159a10 0x7f7d5c06eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:55.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.510+0000 7f7d62d9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5c15a780 con 0x7f7d5c159640 2023-10-06T02:27:55.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.510+0000 7f7d62d9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5c15aa60 con 0x7f7d5c070410 2023-10-06T02:27:55.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.510+0000 7f7d62d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5c15ad40 con 0x7f7d5c159a10 2023-10-06T02:27:55.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.510+0000 7f7d6159a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d5c159640 0x7f7d5c159e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:55.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.510+0000 7f7d6159a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d5c159640 0x7f7d5c159e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41640/0 (socket says 172.21.15.31:41640) 2023-10-06T02:27:55.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.510+0000 7f7d6159a700 1 -- 172.21.15.31:0/4068835041 learned_addr learned my addr 172.21.15.31:0/4068835041 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:55.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.510+0000 7f7d61d9b700 1 --2- 172.21.15.31:0/4068835041 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d5c070410 0x7f7d5c14f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:55.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.510+0000 7f7d6259c700 1 --2- 172.21.15.31:0/4068835041 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7d5c159a10 0x7f7d5c06eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:55.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.510+0000 7f7d6159a700 1 -- 172.21.15.31:0/4068835041 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7d5c159a10 msgr2=0x7f7d5c06eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:55.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.510+0000 7f7d6159a700 1 --2- 172.21.15.31:0/4068835041 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7d5c159a10 0x7f7d5c06eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:55.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.510+0000 7f7d6159a700 1 -- 172.21.15.31:0/4068835041 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d5c070410 msgr2=0x7f7d5c14f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:55.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.510+0000 7f7d6159a700 1 --2- 172.21.15.31:0/4068835041 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d5c070410 0x7f7d5c14f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:55.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.510+0000 7f7d6159a700 1 -- 172.21.15.31:0/4068835041 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d4c009420 con 0x7f7d5c159640 2023-10-06T02:27:55.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.511+0000 7f7d6159a700 1 --2- 172.21.15.31:0/4068835041 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d5c159640 0x7f7d5c159e80 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c013000 tx=0x7f7d4c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:55.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.511+0000 7f7d51ffb700 1 -- 172.21.15.31:0/4068835041 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d4c01bc40 con 0x7f7d5c159640 2023-10-06T02:27:55.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.511+0000 7f7d51ffb700 1 -- 172.21.15.31:0/4068835041 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d4c01bdc0 con 0x7f7d5c159640 2023-10-06T02:27:55.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.511+0000 7f7d51ffb700 1 -- 172.21.15.31:0/4068835041 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d4c011670 con 0x7f7d5c159640 2023-10-06T02:27:55.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.511+0000 7f7d62d9d700 1 -- 172.21.15.31:0/4068835041 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d5c15b020 con 0x7f7d5c159640 2023-10-06T02:27:55.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.511+0000 7f7d62d9d700 1 -- 172.21.15.31:0/4068835041 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d5c15a3c0 con 0x7f7d5c159640 2023-10-06T02:27:55.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.512+0000 7f7d62d9d700 1 -- 172.21.15.31:0/4068835041 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d5c15b540 con 0x7f7d5c159640 2023-10-06T02:27:55.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.513+0000 7f7d51ffb700 1 -- 172.21.15.31:0/4068835041 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f7d4c011a20 con 0x7f7d5c159640 2023-10-06T02:27:55.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.514+0000 7f7d51ffb700 1 --2- 172.21.15.31:0/4068835041 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7d40067050 0x7f7d40069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:55.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.514+0000 7f7d51ffb700 1 -- 172.21.15.31:0/4068835041 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f7d4c085980 con 0x7f7d5c159640 2023-10-06T02:27:55.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.514+0000 7f7d61d9b700 1 --2- 172.21.15.31:0/4068835041 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7d40067050 0x7f7d40069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:55.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.515+0000 7f7d61d9b700 1 --2- 172.21.15.31:0/4068835041 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7d40067050 0x7f7d40069500 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7d58007130 tx=0x7f7d58007eb0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:55.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.518+0000 7f7d51ffb700 1 -- 172.21.15.31:0/4068835041 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7d5c15b540 con 0x7f7d5c159640 2023-10-06T02:27:55.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.722+0000 7f7d62d9d700 1 -- 172.21.15.31:0/4068835041 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f7d5c15b540 con 0x7f7d5c159640 2023-10-06T02:27:55.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.722+0000 7f7d51ffb700 1 -- 172.21.15.31:0/4068835041 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v83) v1 ==== 84+0+126 (secure 0 0 0) 0x7f7d4c015090 con 0x7f7d5c159640 2023-10-06T02:27:55.723 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:27:55.723 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2023-10-06T02:27:55.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:55.723 INFO:teuthology.orchestra.run.smithi031.stdout: "poolnum": 1, 2023-10-06T02:27:55.723 INFO:teuthology.orchestra.run.smithi031.stdout: "poolname": ".mgr" 2023-10-06T02:27:55.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:27:55.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:27:55.723 INFO:teuthology.orchestra.run.smithi031.stdout: "poolnum": 2, 2023-10-06T02:27:55.723 INFO:teuthology.orchestra.run.smithi031.stdout: "poolname": "rbd" 2023-10-06T02:27:55.724 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:27:55.724 INFO:teuthology.orchestra.run.smithi031.stdout:] 2023-10-06T02:27:55.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.724+0000 7f7d62d9d700 1 -- 172.21.15.31:0/4068835041 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7d40067050 msgr2=0x7f7d40069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:55.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.725+0000 7f7d62d9d700 1 --2- 172.21.15.31:0/4068835041 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7d40067050 0x7f7d40069500 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7d58007130 tx=0x7f7d58007eb0 comp rx=0 tx=0).stop 2023-10-06T02:27:55.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.725+0000 7f7d62d9d700 1 -- 172.21.15.31:0/4068835041 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d5c159640 msgr2=0x7f7d5c159e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:55.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.725+0000 7f7d62d9d700 1 --2- 172.21.15.31:0/4068835041 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d5c159640 0x7f7d5c159e80 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c013000 tx=0x7f7d4c017000 comp rx=0 tx=0).stop 2023-10-06T02:27:55.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.725+0000 7f7d62d9d700 1 -- 172.21.15.31:0/4068835041 shutdown_connections 2023-10-06T02:27:55.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.725+0000 7f7d62d9d700 1 --2- 172.21.15.31:0/4068835041 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7d40067050 0x7f7d40069500 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:55.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.725+0000 7f7d62d9d700 1 --2- 172.21.15.31:0/4068835041 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7d5c159a10 0x7f7d5c06eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:55.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.725+0000 7f7d62d9d700 1 --2- 172.21.15.31:0/4068835041 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d5c070410 0x7f7d5c14f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:55.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.725+0000 7f7d62d9d700 1 --2- 172.21.15.31:0/4068835041 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d5c159640 0x7f7d5c159e80 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:55.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.725+0000 7f7d62d9d700 1 -- 172.21.15.31:0/4068835041 >> 172.21.15.31:0/4068835041 conn(0x7f7d5c06e720 msgr2=0x7f7d5c15bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:55.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.725+0000 7f7d62d9d700 1 -- 172.21.15.31:0/4068835041 shutdown_connections 2023-10-06T02:27:55.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.725+0000 7f7d62d9d700 1 -- 172.21.15.31:0/4068835041 wait complete. 2023-10-06T02:27:55.743 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T02:27:55.743 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T02:27:55.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.934+0000 7f16fef9d700 1 Processor -- start 2023-10-06T02:27:55.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.934+0000 7f16fef9d700 1 -- start start 2023-10-06T02:27:55.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.935+0000 7f16fef9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f16f8156d10 0x7f16f80cc150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:55.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.936+0000 7f16fef9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f16f80c8980 0x7f16f80c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:55.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.936+0000 7f16fef9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16f80c2200 con 0x7f16f80c0ef0 2023-10-06T02:27:55.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.936+0000 7f16fef9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16f8157190 con 0x7f16f8156d10 2023-10-06T02:27:55.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.936+0000 7f16fef9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16f80c15f0 con 0x7f16f80c8980 2023-10-06T02:27:55.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.936+0000 7f16fd79a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f16f8156d10 0x7f16f80cc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:55.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.936+0000 7f16fdf9b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f16f80c0ef0 0x7f16f8153660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:59506/0 (socket says 172.21.15.31:59506) 2023-10-06T02:27:55.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.936+0000 7f16fdf9b700 1 -- 172.21.15.31:0/4111776064 learned_addr learned my addr 172.21.15.31:0/4111776064 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:27:55.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.936+0000 7f16fe79c700 1 --2- 172.21.15.31:0/4111776064 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f16f80c8980 0x7f16f80c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:55.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.936+0000 7f16fd79a700 1 -- 172.21.15.31:0/4111776064 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f16f80c8980 msgr2=0x7f16f80c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:55.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.936+0000 7f16fd79a700 1 --2- 172.21.15.31:0/4111776064 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f16f80c8980 0x7f16f80c8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:55.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.936+0000 7f16fd79a700 1 -- 172.21.15.31:0/4111776064 >> v1:172.21.15.31:6789/0 conn(0x7f16f80c0ef0 legacy=0x7f16f8153660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:55.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.936+0000 7f16fd79a700 1 -- 172.21.15.31:0/4111776064 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16f80c18d0 con 0x7f16f8156d10 2023-10-06T02:27:55.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.936+0000 7f16fcf99700 1 -- 172.21.15.31:0/4111776064 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2326598407 0 0) 0x7f16f80c2200 con 0x7f16f80c0ef0 2023-10-06T02:27:55.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.937+0000 7f16fd79a700 1 --2- 172.21.15.31:0/4111776064 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16f8156d10 0x7f16f80cc150 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f16e80098e0 tx=0x7f16e80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c3cd66c4b50e613 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:55.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.937+0000 7f16fcf99700 1 -- 172.21.15.31:0/4111776064 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e800b640 con 0x7f16f8156d10 2023-10-06T02:27:55.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.937+0000 7f16fcf99700 1 -- 172.21.15.31:0/4111776064 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16e800b7a0 con 0x7f16f8156d10 2023-10-06T02:27:55.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.937+0000 7f16fcf99700 1 -- 172.21.15.31:0/4111776064 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e800bb40 con 0x7f16f8156d10 2023-10-06T02:27:55.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.937+0000 7f16fef9d700 1 -- 172.21.15.31:0/4111776064 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16f8156d10 msgr2=0x7f16f80cc150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:55.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.937+0000 7f16fef9d700 1 --2- 172.21.15.31:0/4111776064 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16f8156d10 0x7f16f80cc150 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f16e80098e0 tx=0x7f16e80091a0 comp rx=0 tx=0).stop 2023-10-06T02:27:55.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.938+0000 7f16fef9d700 1 -- 172.21.15.31:0/4111776064 shutdown_connections 2023-10-06T02:27:55.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.938+0000 7f16fef9d700 1 --2- 172.21.15.31:0/4111776064 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f16f80c8980 0x7f16f80c8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:55.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.938+0000 7f16fef9d700 1 --2- 172.21.15.31:0/4111776064 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16f8156d10 0x7f16f80cc150 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:55.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.938+0000 7f16fef9d700 1 -- 172.21.15.31:0/4111776064 >> 172.21.15.31:0/4111776064 conn(0x7f16f806c6a0 msgr2=0x7f16f806caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:55.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.938+0000 7f16fef9d700 1 -- 172.21.15.31:0/4111776064 shutdown_connections 2023-10-06T02:27:55.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.938+0000 7f16fef9d700 1 -- 172.21.15.31:0/4111776064 wait complete. 2023-10-06T02:27:55.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.939+0000 7f16fef9d700 1 Processor -- start 2023-10-06T02:27:55.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.939+0000 7f16fef9d700 1 -- start start 2023-10-06T02:27:55.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.940+0000 7f16fef9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16f80c0ef0 0x7f16f80adb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:55.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.940+0000 7f16fef9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16f80c8980 0x7f16f80aeef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:55.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.940+0000 7f16fef9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f16f8156d10 0x7f16f80ca210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:55.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.940+0000 7f16fef9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16f8153ea0 con 0x7f16f80c8980 2023-10-06T02:27:55.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.940+0000 7f16fef9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16f8154180 con 0x7f16f80c0ef0 2023-10-06T02:27:55.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.940+0000 7f16fef9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16f8154460 con 0x7f16f8156d10 2023-10-06T02:27:55.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.940+0000 7f16fd79a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16f80c8980 0x7f16f80aeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:55.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.940+0000 7f16fd79a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16f80c8980 0x7f16f80aeef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41658/0 (socket says 172.21.15.31:41658) 2023-10-06T02:27:55.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.940+0000 7f16fd79a700 1 -- 172.21.15.31:0/1310231911 learned_addr learned my addr 172.21.15.31:0/1310231911 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:55.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.941+0000 7f16fdf9b700 1 --2- 172.21.15.31:0/1310231911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16f80c0ef0 0x7f16f80adb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:55.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.941+0000 7f16fe79c700 1 --2- 172.21.15.31:0/1310231911 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f16f8156d10 0x7f16f80ca210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:55.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.941+0000 7f16fd79a700 1 -- 172.21.15.31:0/1310231911 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f16f8156d10 msgr2=0x7f16f80ca210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:55.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.941+0000 7f16fd79a700 1 --2- 172.21.15.31:0/1310231911 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f16f8156d10 0x7f16f80ca210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:55.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.941+0000 7f16fd79a700 1 -- 172.21.15.31:0/1310231911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16f80c0ef0 msgr2=0x7f16f80adb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:55.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.941+0000 7f16fd79a700 1 --2- 172.21.15.31:0/1310231911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16f80c0ef0 0x7f16f80adb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:55.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.941+0000 7f16fd79a700 1 -- 172.21.15.31:0/1310231911 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16e8009420 con 0x7f16f80c8980 2023-10-06T02:27:55.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.941+0000 7f16fe79c700 1 --2- 172.21.15.31:0/1310231911 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f16f8156d10 0x7f16f80ca210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:27:55.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.941+0000 7f16fdf9b700 1 --2- 172.21.15.31:0/1310231911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16f80c0ef0 0x7f16f80adb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:27:55.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.941+0000 7f16fd79a700 1 --2- 172.21.15.31:0/1310231911 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16f80c8980 0x7f16f80aeef0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f16f8067f90 tx=0x7f16e8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:55.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.941+0000 7f16edffb700 1 -- 172.21.15.31:0/1310231911 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e800bb20 con 0x7f16f80c8980 2023-10-06T02:27:55.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.941+0000 7f16edffb700 1 -- 172.21.15.31:0/1310231911 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16e8011420 con 0x7f16f80c8980 2023-10-06T02:27:55.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.941+0000 7f16edffb700 1 -- 172.21.15.31:0/1310231911 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e80117c0 con 0x7f16f80c8980 2023-10-06T02:27:55.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.941+0000 7f16fef9d700 1 -- 172.21.15.31:0/1310231911 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16f8154740 con 0x7f16f80c8980 2023-10-06T02:27:55.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.942+0000 7f16fef9d700 1 -- 172.21.15.31:0/1310231911 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16f80ae070 con 0x7f16f80c8980 2023-10-06T02:27:55.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.943+0000 7f16fef9d700 1 -- 172.21.15.31:0/1310231911 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16f804c3c0 con 0x7f16f80c8980 2023-10-06T02:27:55.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.944+0000 7f16edffb700 1 -- 172.21.15.31:0/1310231911 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f16e8011b70 con 0x7f16f80c8980 2023-10-06T02:27:55.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.945+0000 7f16edffb700 1 --2- 172.21.15.31:0/1310231911 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f16dc067050 0x7f16dc069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:55.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.945+0000 7f16edffb700 1 -- 172.21.15.31:0/1310231911 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f16e8085b30 con 0x7f16f80c8980 2023-10-06T02:27:55.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.945+0000 7f16fdf9b700 1 --2- 172.21.15.31:0/1310231911 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f16dc067050 0x7f16dc069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:55.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.946+0000 7f16fdf9b700 1 --2- 172.21.15.31:0/1310231911 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f16dc067050 0x7f16dc069500 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f16f40011e0 tx=0x7f16f4009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:55.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:55.949+0000 7f16edffb700 1 -- 172.21.15.31:0/1310231911 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f16e8052de0 con 0x7f16f80c8980 2023-10-06T02:27:56.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:56.153+0000 7f16fef9d700 1 -- 172.21.15.31:0/1310231911 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f16f8153fe0 con 0x7f16f80c8980 2023-10-06T02:27:56.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:56.516+0000 7f16edffb700 1 -- 172.21.15.31:0/1310231911 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v84) v1 ==== 138+0+0 (secure 0 0 0) 0x7f16f8153fe0 con 0x7f16f80c8980 2023-10-06T02:27:56.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:56.521+0000 7f16fef9d700 1 -- 172.21.15.31:0/1310231911 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f16dc067050 msgr2=0x7f16dc069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:56.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:56.521+0000 7f16fef9d700 1 --2- 172.21.15.31:0/1310231911 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f16dc067050 0x7f16dc069500 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f16f40011e0 tx=0x7f16f4009040 comp rx=0 tx=0).stop 2023-10-06T02:27:56.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:56.521+0000 7f16fef9d700 1 -- 172.21.15.31:0/1310231911 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16f80c8980 msgr2=0x7f16f80aeef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:56.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:56.521+0000 7f16fef9d700 1 --2- 172.21.15.31:0/1310231911 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16f80c8980 0x7f16f80aeef0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f16f8067f90 tx=0x7f16e8017000 comp rx=0 tx=0).stop 2023-10-06T02:27:56.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:56.521+0000 7f16fef9d700 1 -- 172.21.15.31:0/1310231911 shutdown_connections 2023-10-06T02:27:56.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:56.521+0000 7f16fef9d700 1 --2- 172.21.15.31:0/1310231911 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f16dc067050 0x7f16dc069500 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:56.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:56.521+0000 7f16fef9d700 1 --2- 172.21.15.31:0/1310231911 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f16f8156d10 0x7f16f80ca210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:56.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:56.521+0000 7f16fef9d700 1 --2- 172.21.15.31:0/1310231911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16f80c0ef0 0x7f16f80adb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:56.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:56.521+0000 7f16fef9d700 1 --2- 172.21.15.31:0/1310231911 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16f80c8980 0x7f16f80aeef0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:56.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:56.521+0000 7f16fef9d700 1 -- 172.21.15.31:0/1310231911 >> 172.21.15.31:0/1310231911 conn(0x7f16f806c6a0 msgr2=0x7f16f806ca70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:56.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:56.523+0000 7f16fef9d700 1 -- 172.21.15.31:0/1310231911 shutdown_connections 2023-10-06T02:27:56.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:56.523+0000 7f16fef9d700 1 -- 172.21.15.31:0/1310231911 wait complete. 2023-10-06T02:27:56.525 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' created 2023-10-06T02:27:56.543 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T02:27:56.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:56.737+0000 7f75d43ff700 1 Processor -- start 2023-10-06T02:27:56.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:56.737+0000 7f75d43ff700 1 -- start start 2023-10-06T02:27:56.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:56.738+0000 7f75d43ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f75cc0cc320 0x7f75cc13c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:56.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:56.738+0000 7f75d43ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f75cc0cc6f0 0x7f75cc13c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:56.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:56.738+0000 7f75d43ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75cc06d7b0 con 0x7f75cc069a00 2023-10-06T02:27:56.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:56.738+0000 7f75d43ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75cc06a590 con 0x7f75cc0cc320 2023-10-06T02:27:56.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:56.738+0000 7f75d43ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75cc06f7d0 con 0x7f75cc0cc6f0 2023-10-06T02:27:56.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:56.738+0000 7f75d219b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f75cc0cc320 0x7f75cc13c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:56.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:56.738+0000 7f75d299c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f75cc069a00 0x7f75cc069e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:59510/0 (socket says 172.21.15.31:59510) 2023-10-06T02:27:56.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:56.738+0000 7f75d299c700 1 -- 172.21.15.31:0/4270409421 learned_addr learned my addr 172.21.15.31:0/4270409421 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:27:56.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:56.739+0000 7f75d199a700 1 --2- 172.21.15.31:0/4270409421 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f75cc0cc6f0 0x7f75cc13c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:56.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:56.739+0000 7f75d219b700 1 -- 172.21.15.31:0/4270409421 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f75cc0cc6f0 msgr2=0x7f75cc13c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:56.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:56.739+0000 7f75d219b700 1 --2- 172.21.15.31:0/4270409421 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f75cc0cc6f0 0x7f75cc13c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:56.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:56.739+0000 7f75d219b700 1 -- 172.21.15.31:0/4270409421 >> v1:172.21.15.31:6789/0 conn(0x7f75cc069a00 legacy=0x7f75cc069e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:56.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:56.739+0000 7f75d219b700 1 -- 172.21.15.31:0/4270409421 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75cc06e330 con 0x7f75cc0cc320 2023-10-06T02:27:56.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:56.739+0000 7f75d1199700 1 -- 172.21.15.31:0/4270409421 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 755508190 0 0) 0x7f75cc06d7b0 con 0x7f75cc069a00 2023-10-06T02:27:56.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:56.739+0000 7f75d219b700 1 --2- 172.21.15.31:0/4270409421 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75cc0cc320 0x7f75cc13c2a0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f75c80098e0 tx=0x7f75c80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e089628b352755a2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:56.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:56.739+0000 7f75d1199700 1 -- 172.21.15.31:0/4270409421 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c8009e80 con 0x7f75cc0cc320 2023-10-06T02:27:56.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:56.739+0000 7f75d1199700 1 -- 172.21.15.31:0/4270409421 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75c800b860 con 0x7f75cc0cc320 2023-10-06T02:27:56.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:56.739+0000 7f75d1199700 1 -- 172.21.15.31:0/4270409421 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c800bbe0 con 0x7f75cc0cc320 2023-10-06T02:27:56.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:56.740+0000 7f75d43ff700 1 -- 172.21.15.31:0/4270409421 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75cc0cc320 msgr2=0x7f75cc13c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:56.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:56.740+0000 7f75d43ff700 1 --2- 172.21.15.31:0/4270409421 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75cc0cc320 0x7f75cc13c2a0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f75c80098e0 tx=0x7f75c80091a0 comp rx=0 tx=0).stop 2023-10-06T02:27:56.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:56.740+0000 7f75d43ff700 1 -- 172.21.15.31:0/4270409421 shutdown_connections 2023-10-06T02:27:56.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:56.740+0000 7f75d43ff700 1 --2- 172.21.15.31:0/4270409421 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75cc0cc320 0x7f75cc13c2a0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:56.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:56.740+0000 7f75d43ff700 1 --2- 172.21.15.31:0/4270409421 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f75cc0cc6f0 0x7f75cc13c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:56.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:56.740+0000 7f75d43ff700 1 -- 172.21.15.31:0/4270409421 >> 172.21.15.31:0/4270409421 conn(0x7f75cc067f00 msgr2=0x7f75cc068300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:56.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:56.740+0000 7f75d43ff700 1 -- 172.21.15.31:0/4270409421 shutdown_connections 2023-10-06T02:27:56.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:56.740+0000 7f75d43ff700 1 -- 172.21.15.31:0/4270409421 wait complete. 2023-10-06T02:27:56.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:56.742+0000 7f75d43ff700 1 Processor -- start 2023-10-06T02:27:56.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:56.742+0000 7f75d43ff700 1 -- start start 2023-10-06T02:27:56.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:56.743+0000 7f75d43ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f75cc069a00 0x7f75cc0c5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:56.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:56.743+0000 7f75d43ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75cc0cc320 0x7f75cc138c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:56.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:56.743+0000 7f75d43ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75cc0cc6f0 0x7f75cc1391c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:56.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:56.743+0000 7f75d43ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75cc0c9e60 con 0x7f75cc0cc320 2023-10-06T02:27:56.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:56.743+0000 7f75d43ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75cc0ca140 con 0x7f75cc0cc6f0 2023-10-06T02:27:56.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:56.743+0000 7f75d43ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75cc0ca420 con 0x7f75cc069a00 2023-10-06T02:27:56.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:56.743+0000 7f75d199a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75cc0cc320 0x7f75cc138c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:56.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:56.743+0000 7f75d199a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75cc0cc320 0x7f75cc138c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41674/0 (socket says 172.21.15.31:41674) 2023-10-06T02:27:56.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:56.743+0000 7f75d199a700 1 -- 172.21.15.31:0/3037761042 learned_addr learned my addr 172.21.15.31:0/3037761042 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:56.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:56.743+0000 7f75d219b700 1 --2- 172.21.15.31:0/3037761042 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f75cc069a00 0x7f75cc0c5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:56.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:56.743+0000 7f75d299c700 1 --2- 172.21.15.31:0/3037761042 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75cc0cc6f0 0x7f75cc1391c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:56.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:56.743+0000 7f75d199a700 1 -- 172.21.15.31:0/3037761042 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f75cc069a00 msgr2=0x7f75cc0c5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:56.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:56.743+0000 7f75d199a700 1 --2- 172.21.15.31:0/3037761042 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f75cc069a00 0x7f75cc0c5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:56.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:56.743+0000 7f75d199a700 1 -- 172.21.15.31:0/3037761042 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75cc0cc6f0 msgr2=0x7f75cc1391c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:56.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:56.743+0000 7f75d199a700 1 --2- 172.21.15.31:0/3037761042 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75cc0cc6f0 0x7f75cc1391c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:56.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:56.743+0000 7f75d199a700 1 -- 172.21.15.31:0/3037761042 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75c8009420 con 0x7f75cc0cc320 2023-10-06T02:27:56.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:56.744+0000 7f75d199a700 1 --2- 172.21.15.31:0/3037761042 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75cc0cc320 0x7f75cc138c80 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f75bc00d860 tx=0x7f75bc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:56.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:56.744+0000 7f75c27fc700 1 -- 172.21.15.31:0/3037761042 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75bc0125f0 con 0x7f75cc0cc320 2023-10-06T02:27:56.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:56.744+0000 7f75c27fc700 1 -- 172.21.15.31:0/3037761042 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75bc012750 con 0x7f75cc0cc320 2023-10-06T02:27:56.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:56.744+0000 7f75c27fc700 1 -- 172.21.15.31:0/3037761042 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75bc012b10 con 0x7f75cc0cc320 2023-10-06T02:27:56.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:56.744+0000 7f75d43ff700 1 -- 172.21.15.31:0/3037761042 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75cc0ca700 con 0x7f75cc0cc320 2023-10-06T02:27:56.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:56.744+0000 7f75d43ff700 1 -- 172.21.15.31:0/3037761042 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75cc169900 con 0x7f75cc0cc320 2023-10-06T02:27:56.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:56.745+0000 7f75d43ff700 1 -- 172.21.15.31:0/3037761042 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75cc169b20 con 0x7f75cc0cc320 2023-10-06T02:27:56.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:56.746+0000 7f75c27fc700 1 -- 172.21.15.31:0/3037761042 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f75bc028430 con 0x7f75cc0cc320 2023-10-06T02:27:56.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:56.747+0000 7f75c27fc700 1 --2- 172.21.15.31:0/3037761042 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f75b0067050 0x7f75b0069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:56.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:56.747+0000 7f75c27fc700 1 -- 172.21.15.31:0/3037761042 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5576+0+0 (secure 0 0 0) 0x7f75bc085580 con 0x7f75cc0cc320 2023-10-06T02:27:56.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:56.748+0000 7f75d219b700 1 --2- 172.21.15.31:0/3037761042 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f75b0067050 0x7f75b0069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:56.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:56.748+0000 7f75d219b700 1 --2- 172.21.15.31:0/3037761042 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f75b0067050 0x7f75b0069500 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f75c80091e0 tx=0x7f75c8017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:56.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:56.751+0000 7f75c27fc700 1 -- 172.21.15.31:0/3037761042 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f75bc052740 con 0x7f75cc0cc320 2023-10-06T02:27:56.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:56.953+0000 7f75d43ff700 1 -- 172.21.15.31:0/3037761042 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f75cc0c9fa0 con 0x7f75cc0cc320 2023-10-06T02:27:57.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:57.522+0000 7f75c27fc700 1 -- 172.21.15.31:0/3037761042 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v85) v1 ==== 170+0+0 (secure 0 0 0) 0x7f75cc0c9fa0 con 0x7f75cc0cc320 2023-10-06T02:27:57.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:57.530+0000 7f75d43ff700 1 -- 172.21.15.31:0/3037761042 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f75b0067050 msgr2=0x7f75b0069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:57.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:57.530+0000 7f75d43ff700 1 --2- 172.21.15.31:0/3037761042 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f75b0067050 0x7f75b0069500 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f75c80091e0 tx=0x7f75c8017000 comp rx=0 tx=0).stop 2023-10-06T02:27:57.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:57.530+0000 7f75d43ff700 1 -- 172.21.15.31:0/3037761042 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75cc0cc320 msgr2=0x7f75cc138c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:57.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:57.530+0000 7f75d43ff700 1 --2- 172.21.15.31:0/3037761042 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75cc0cc320 0x7f75cc138c80 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f75bc00d860 tx=0x7f75bc008040 comp rx=0 tx=0).stop 2023-10-06T02:27:57.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:57.530+0000 7f75d43ff700 1 -- 172.21.15.31:0/3037761042 shutdown_connections 2023-10-06T02:27:57.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:57.530+0000 7f75d43ff700 1 --2- 172.21.15.31:0/3037761042 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75cc0cc6f0 0x7f75cc1391c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:57.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:57.530+0000 7f75d43ff700 1 --2- 172.21.15.31:0/3037761042 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f75b0067050 0x7f75b0069500 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:57.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:57.530+0000 7f75d43ff700 1 --2- 172.21.15.31:0/3037761042 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f75cc069a00 0x7f75cc0c5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:57.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:57.530+0000 7f75d43ff700 1 --2- 172.21.15.31:0/3037761042 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75cc0cc320 0x7f75cc138c80 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:57.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:57.530+0000 7f75d43ff700 1 -- 172.21.15.31:0/3037761042 >> 172.21.15.31:0/3037761042 conn(0x7f75cc067f00 msgr2=0x7f75cc0c4e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:57.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:57.532+0000 7f75d43ff700 1 -- 172.21.15.31:0/3037761042 shutdown_connections 2023-10-06T02:27:57.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:57.532+0000 7f75d43ff700 1 -- 172.21.15.31:0/3037761042 wait complete. 2023-10-06T02:27:57.536 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' created 2023-10-06T02:27:57.560 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T02:27:57.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:57.751+0000 7fc57e99d700 1 Processor -- start 2023-10-06T02:27:57.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:57.751+0000 7fc57e99d700 1 -- start start 2023-10-06T02:27:57.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:57.753+0000 7fc57e99d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc578159550 0x7fc5780ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:57.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:57.753+0000 7fc57e99d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc57815dfd0 0x7fc5780aedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:57.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:57.753+0000 7fc57e99d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc578159920 0x7fc5780ad090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:57.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:57.753+0000 7fc57e99d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5780acc00 con 0x7fc578159920 2023-10-06T02:27:57.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:57.753+0000 7fc57e99d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5780abff0 con 0x7fc578159550 2023-10-06T02:27:57.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:57.753+0000 7fc57e99d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5780ac2d0 con 0x7fc57815dfd0 2023-10-06T02:27:57.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:57.753+0000 7fc57e19c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc578159920 0x7fc5780ad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:57.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:57.753+0000 7fc57e19c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc578159920 0x7fc5780ad090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41690/0 (socket says 172.21.15.31:41690) 2023-10-06T02:27:57.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:57.753+0000 7fc57e19c700 1 -- 172.21.15.31:0/3495421660 learned_addr learned my addr 172.21.15.31:0/3495421660 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:57.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:57.753+0000 7fc57d99b700 1 --2- 172.21.15.31:0/3495421660 >> v2:172.21.15.62:3300/0 conn(0x7fc578159550 0x7fc5780ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:57.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:57.753+0000 7fc57d19a700 1 --2- 172.21.15.31:0/3495421660 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc57815dfd0 0x7fc5780aedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:57.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:57.754+0000 7fc57e19c700 1 -- 172.21.15.31:0/3495421660 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc57815dfd0 msgr2=0x7fc5780aedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:57.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:57.754+0000 7fc57e19c700 1 --2- 172.21.15.31:0/3495421660 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc57815dfd0 0x7fc5780aedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:57.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:57.754+0000 7fc57e19c700 1 -- 172.21.15.31:0/3495421660 >> v2:172.21.15.62:3300/0 conn(0x7fc578159550 msgr2=0x7fc5780ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:57.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:57.754+0000 7fc57e19c700 1 --2- 172.21.15.31:0/3495421660 >> v2:172.21.15.62:3300/0 conn(0x7fc578159550 0x7fc5780ceac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:57.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:57.754+0000 7fc57e19c700 1 -- 172.21.15.31:0/3495421660 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5780ac5b0 con 0x7fc578159920 2023-10-06T02:27:57.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:57.754+0000 7fc57e19c700 1 --2- 172.21.15.31:0/3495421660 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc578159920 0x7fc5780ad090 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fc578064270 tx=0x7fc570009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d388ece036cec3de server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:57.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:57.754+0000 7fc57c999700 1 -- 172.21.15.31:0/3495421660 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc57000e070 con 0x7fc578159920 2023-10-06T02:27:57.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:57.754+0000 7fc57c999700 1 -- 172.21.15.31:0/3495421660 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc57000b970 con 0x7fc578159920 2023-10-06T02:27:57.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:57.754+0000 7fc57c999700 1 -- 172.21.15.31:0/3495421660 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5700092e0 con 0x7fc578159920 2023-10-06T02:27:57.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:57.755+0000 7fc57e99d700 1 -- 172.21.15.31:0/3495421660 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc578159920 msgr2=0x7fc5780ad090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:57.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:57.755+0000 7fc57e99d700 1 --2- 172.21.15.31:0/3495421660 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc578159920 0x7fc5780ad090 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fc578064270 tx=0x7fc570009840 comp rx=0 tx=0).stop 2023-10-06T02:27:57.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:57.755+0000 7fc57e99d700 1 -- 172.21.15.31:0/3495421660 shutdown_connections 2023-10-06T02:27:57.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:57.755+0000 7fc57e99d700 1 --2- 172.21.15.31:0/3495421660 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc578159920 0x7fc5780ad090 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:57.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:57.755+0000 7fc57e99d700 1 --2- 172.21.15.31:0/3495421660 >> v2:172.21.15.62:3300/0 conn(0x7fc578159550 0x7fc5780ceac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:57.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:57.755+0000 7fc57e99d700 1 --2- 172.21.15.31:0/3495421660 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc57815dfd0 0x7fc5780aedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:57.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:57.755+0000 7fc57e99d700 1 -- 172.21.15.31:0/3495421660 >> 172.21.15.31:0/3495421660 conn(0x7fc57806ce80 msgr2=0x7fc5780ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:57.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:57.755+0000 7fc57e99d700 1 -- 172.21.15.31:0/3495421660 shutdown_connections 2023-10-06T02:27:57.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:57.755+0000 7fc57e99d700 1 -- 172.21.15.31:0/3495421660 wait complete. 2023-10-06T02:27:57.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:57.756+0000 7fc57e99d700 1 Processor -- start 2023-10-06T02:27:57.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:57.757+0000 7fc57e99d700 1 -- start start 2023-10-06T02:27:57.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:57.757+0000 7fc57e99d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc578159550 0x7fc57814c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:57.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:57.757+0000 7fc57e99d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc578159920 0x7fc5780b2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:57.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:57.757+0000 7fc57e99d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc57815dfd0 0x7fc578152400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:57.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:57.757+0000 7fc57e99d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5780b2630 con 0x7fc578159550 2023-10-06T02:27:57.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:57.757+0000 7fc57e99d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5780b1a20 con 0x7fc578159920 2023-10-06T02:27:57.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:57.757+0000 7fc57e99d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5780b1d00 con 0x7fc57815dfd0 2023-10-06T02:27:57.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:57.758+0000 7fc57d99b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc578159550 0x7fc57814c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:57.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:57.758+0000 7fc57d99b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc578159550 0x7fc57814c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41698/0 (socket says 172.21.15.31:41698) 2023-10-06T02:27:57.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:57.758+0000 7fc57d99b700 1 -- 172.21.15.31:0/1812212891 learned_addr learned my addr 172.21.15.31:0/1812212891 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:57.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:57.758+0000 7fc57d19a700 1 --2- 172.21.15.31:0/1812212891 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc578159920 0x7fc5780b2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:57.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:57.758+0000 7fc57e19c700 1 --2- 172.21.15.31:0/1812212891 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc57815dfd0 0x7fc578152400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:57.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:57.758+0000 7fc57d99b700 1 -- 172.21.15.31:0/1812212891 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc57815dfd0 msgr2=0x7fc578152400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:57.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:57.758+0000 7fc57d99b700 1 --2- 172.21.15.31:0/1812212891 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc57815dfd0 0x7fc578152400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:57.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:57.758+0000 7fc57d99b700 1 -- 172.21.15.31:0/1812212891 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc578159920 msgr2=0x7fc5780b2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:57.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:57.758+0000 7fc57d99b700 1 --2- 172.21.15.31:0/1812212891 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc578159920 0x7fc5780b2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:57.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:57.758+0000 7fc57d99b700 1 -- 172.21.15.31:0/1812212891 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc570009ae0 con 0x7fc578159550 2023-10-06T02:27:57.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:57.758+0000 7fc57d19a700 1 --2- 172.21.15.31:0/1812212891 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc578159920 0x7fc5780b2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:27:57.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:57.758+0000 7fc57d99b700 1 --2- 172.21.15.31:0/1812212891 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc578159550 0x7fc57814c040 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fc57400ec80 tx=0x7fc5740091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:57.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:57.759+0000 7fc56dffb700 1 -- 172.21.15.31:0/1812212891 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc57400c9d0 con 0x7fc578159550 2023-10-06T02:27:57.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:57.759+0000 7fc56dffb700 1 -- 172.21.15.31:0/1812212891 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc57400cb30 con 0x7fc578159550 2023-10-06T02:27:57.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:57.759+0000 7fc56dffb700 1 -- 172.21.15.31:0/1812212891 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc574013470 con 0x7fc578159550 2023-10-06T02:27:57.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:57.759+0000 7fc57e99d700 1 -- 172.21.15.31:0/1812212891 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5780b1fe0 con 0x7fc578159550 2023-10-06T02:27:57.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:57.759+0000 7fc57e99d700 1 -- 172.21.15.31:0/1812212891 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5780b3000 con 0x7fc578159550 2023-10-06T02:27:57.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:57.760+0000 7fc57e99d700 1 -- 172.21.15.31:0/1812212891 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5780acd90 con 0x7fc578159550 2023-10-06T02:27:57.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:57.761+0000 7fc56dffb700 1 -- 172.21.15.31:0/1812212891 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7fc574013820 con 0x7fc578159550 2023-10-06T02:27:57.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:57.762+0000 7fc56dffb700 1 --2- 172.21.15.31:0/1812212891 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc55c067050 0x7fc55c069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:57.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:57.762+0000 7fc56dffb700 1 -- 172.21.15.31:0/1812212891 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5958+0+0 (secure 0 0 0) 0x7fc574084c30 con 0x7fc578159550 2023-10-06T02:27:57.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:57.762+0000 7fc57d19a700 1 --2- 172.21.15.31:0/1812212891 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc55c067050 0x7fc55c069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:57.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:57.763+0000 7fc57d19a700 1 --2- 172.21.15.31:0/1812212891 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc55c067050 0x7fc55c069500 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc568006f00 tx=0x7fc568008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:57.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:57.766+0000 7fc56dffb700 1 -- 172.21.15.31:0/1812212891 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc574051bf0 con 0x7fc578159550 2023-10-06T02:27:58.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:58.017+0000 7fc57e99d700 1 -- 172.21.15.31:0/1812212891 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fc5780b2480 con 0x7fc578159550 2023-10-06T02:27:58.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:58.534+0000 7fc56dffb700 1 -- 172.21.15.31:0/1812212891 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 13 and data pool 14 v52) v1 ==== 174+0+0 (secure 0 0 0) 0x7fc574055210 con 0x7fc578159550 2023-10-06T02:27:58.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:58.546+0000 7fc57e99d700 1 -- 172.21.15.31:0/1812212891 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc55c067050 msgr2=0x7fc55c069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:58.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:58.546+0000 7fc57e99d700 1 --2- 172.21.15.31:0/1812212891 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc55c067050 0x7fc55c069500 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc568006f00 tx=0x7fc568008040 comp rx=0 tx=0).stop 2023-10-06T02:27:58.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:58.546+0000 7fc57e99d700 1 -- 172.21.15.31:0/1812212891 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc578159550 msgr2=0x7fc57814c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:58.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:58.546+0000 7fc57e99d700 1 --2- 172.21.15.31:0/1812212891 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc578159550 0x7fc57814c040 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fc57400ec80 tx=0x7fc5740091a0 comp rx=0 tx=0).stop 2023-10-06T02:27:58.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:58.549+0000 7fc57e99d700 1 -- 172.21.15.31:0/1812212891 shutdown_connections 2023-10-06T02:27:58.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:58.549+0000 7fc57e99d700 1 --2- 172.21.15.31:0/1812212891 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc55c067050 0x7fc55c069500 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:58.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:58.549+0000 7fc57e99d700 1 --2- 172.21.15.31:0/1812212891 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc57815dfd0 0x7fc578152400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:58.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:58.549+0000 7fc57e99d700 1 --2- 172.21.15.31:0/1812212891 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc578159550 0x7fc57814c040 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:58.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:58.549+0000 7fc57e99d700 1 --2- 172.21.15.31:0/1812212891 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc578159920 0x7fc5780b2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:58.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:58.549+0000 7fc57e99d700 1 -- 172.21.15.31:0/1812212891 >> 172.21.15.31:0/1812212891 conn(0x7fc57806ce80 msgr2=0x7fc5780b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:58.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:58.553+0000 7fc57e99d700 1 -- 172.21.15.31:0/1812212891 shutdown_connections 2023-10-06T02:27:58.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:58.556+0000 7fc57e99d700 1 -- 172.21.15.31:0/1812212891 wait complete. 2023-10-06T02:27:58.562 INFO:teuthology.orchestra.run.smithi031.stderr:new fs with metadata pool 13 and data pool 14 2023-10-06T02:27:58.589 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:27:58.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:58.777+0000 7fa464dff700 1 Processor -- start 2023-10-06T02:27:58.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:58.777+0000 7fa464dff700 1 -- start start 2023-10-06T02:27:58.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:58.778+0000 7fa464dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa460157600 0x7fa4600b1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:58.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:58.778+0000 7fa464dff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa4600ccba0 0x7fa4600b1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:58.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:58.778+0000 7fa464dff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa460158210 con 0x7fa4601579d0 2023-10-06T02:27:58.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:58.778+0000 7fa464dff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4600cb850 con 0x7fa4600ccba0 2023-10-06T02:27:58.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:58.779+0000 7fa464dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4600cb290 con 0x7fa460157600 2023-10-06T02:27:58.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:58.779+0000 7fa45ed9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa4600ccba0 0x7fa4600b1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:58.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:58.779+0000 7fa45ed9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa4600ccba0 0x7fa4600b1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40384/0 (socket says 172.21.15.31:40384) 2023-10-06T02:27:58.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:58.779+0000 7fa45ed9d700 1 -- 172.21.15.31:0/2244691963 learned_addr learned my addr 172.21.15.31:0/2244691963 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:58.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:58.779+0000 7fa45e59c700 1 --2- 172.21.15.31:0/2244691963 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa460157600 0x7fa4600b1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:58.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:58.779+0000 7fa45ed9d700 1 -- 172.21.15.31:0/2244691963 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa460157600 msgr2=0x7fa4600b1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:58.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:58.779+0000 7fa45ed9d700 1 --2- 172.21.15.31:0/2244691963 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa460157600 0x7fa4600b1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:58.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:58.779+0000 7fa45ed9d700 1 -- 172.21.15.31:0/2244691963 >> v1:172.21.15.62:6789/0 conn(0x7fa4601579d0 legacy=0x7fa4601586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:58.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:58.779+0000 7fa45ed9d700 1 -- 172.21.15.31:0/2244691963 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4600cb570 con 0x7fa4600ccba0 2023-10-06T02:27:58.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:58.779+0000 7fa45e59c700 1 --2- 172.21.15.31:0/2244691963 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa460157600 0x7fa4600b1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:27:58.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:58.779+0000 7fa45ed9d700 1 --2- 172.21.15.31:0/2244691963 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4600ccba0 0x7fa4600b1950 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fa458009f20 tx=0x7fa458009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5774c89c0a6b3166 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:58.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:58.780+0000 7fa45d59a700 1 -- 172.21.15.31:0/2244691963 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa45800e070 con 0x7fa4600ccba0 2023-10-06T02:27:58.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:58.780+0000 7fa45d59a700 1 -- 172.21.15.31:0/2244691963 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa45800ba80 con 0x7fa4600ccba0 2023-10-06T02:27:58.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:58.780+0000 7fa45d59a700 1 -- 172.21.15.31:0/2244691963 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4580092e0 con 0x7fa4600ccba0 2023-10-06T02:27:58.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:58.780+0000 7fa464dff700 1 -- 172.21.15.31:0/2244691963 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4600ccba0 msgr2=0x7fa4600b1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:58.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:58.780+0000 7fa464dff700 1 --2- 172.21.15.31:0/2244691963 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4600ccba0 0x7fa4600b1950 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fa458009f20 tx=0x7fa458009840 comp rx=0 tx=0).stop 2023-10-06T02:27:58.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:58.780+0000 7fa464dff700 1 -- 172.21.15.31:0/2244691963 shutdown_connections 2023-10-06T02:27:58.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:58.780+0000 7fa464dff700 1 --2- 172.21.15.31:0/2244691963 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4600ccba0 0x7fa4600b1950 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:58.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:58.780+0000 7fa464dff700 1 --2- 172.21.15.31:0/2244691963 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa460157600 0x7fa4600b1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:58.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:58.780+0000 7fa464dff700 1 -- 172.21.15.31:0/2244691963 >> 172.21.15.31:0/2244691963 conn(0x7fa46006d170 msgr2=0x7fa4600cc300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:58.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:58.780+0000 7fa464dff700 1 -- 172.21.15.31:0/2244691963 shutdown_connections 2023-10-06T02:27:58.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:58.780+0000 7fa464dff700 1 -- 172.21.15.31:0/2244691963 wait complete. 2023-10-06T02:27:58.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:58.782+0000 7fa464dff700 1 Processor -- start 2023-10-06T02:27:58.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:58.782+0000 7fa464dff700 1 -- start start 2023-10-06T02:27:58.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:58.783+0000 7fa464dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4600ccba0 0x7fa460156300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:58.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:58.783+0000 7fa464dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa460157600 0x7fa46015ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:58.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:58.783+0000 7fa464dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4601579d0 0x7fa46015f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:58.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:58.783+0000 7fa464dff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa46015ff10 con 0x7fa4601579d0 2023-10-06T02:27:58.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:58.783+0000 7fa464dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa460156f90 con 0x7fa4600ccba0 2023-10-06T02:27:58.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:58.783+0000 7fa464dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa46015f950 con 0x7fa460157600 2023-10-06T02:27:58.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:58.783+0000 7fa45ed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4601579d0 0x7fa46015f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:58.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:58.783+0000 7fa45ed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4601579d0 0x7fa46015f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40388/0 (socket says 172.21.15.31:40388) 2023-10-06T02:27:58.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:58.783+0000 7fa45ed9d700 1 -- 172.21.15.31:0/4125648519 learned_addr learned my addr 172.21.15.31:0/4125648519 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:58.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:58.783+0000 7fa45dd9b700 1 --2- 172.21.15.31:0/4125648519 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa460157600 0x7fa46015ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:58.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:58.783+0000 7fa45e59c700 1 --2- 172.21.15.31:0/4125648519 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4600ccba0 0x7fa460156300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:58.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:58.784+0000 7fa45ed9d700 1 -- 172.21.15.31:0/4125648519 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa460157600 msgr2=0x7fa46015ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:58.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:58.784+0000 7fa45ed9d700 1 --2- 172.21.15.31:0/4125648519 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa460157600 0x7fa46015ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:58.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:58.784+0000 7fa45ed9d700 1 -- 172.21.15.31:0/4125648519 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4600ccba0 msgr2=0x7fa460156300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:58.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:58.784+0000 7fa45ed9d700 1 --2- 172.21.15.31:0/4125648519 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4600ccba0 0x7fa460156300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:58.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:58.784+0000 7fa45ed9d700 1 -- 172.21.15.31:0/4125648519 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa458009ae0 con 0x7fa4601579d0 2023-10-06T02:27:58.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:58.784+0000 7fa45ed9d700 1 --2- 172.21.15.31:0/4125648519 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4601579d0 0x7fa46015f2b0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fa45800b6c0 tx=0x7fa458018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:58.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:58.784+0000 7fa44e7fc700 1 -- 172.21.15.31:0/4125648519 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4580032f0 con 0x7fa4601579d0 2023-10-06T02:27:58.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:58.784+0000 7fa44e7fc700 1 -- 172.21.15.31:0/4125648519 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa45801c410 con 0x7fa4601579d0 2023-10-06T02:27:58.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:58.784+0000 7fa44e7fc700 1 -- 172.21.15.31:0/4125648519 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa45801c7b0 con 0x7fa4601579d0 2023-10-06T02:27:58.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:58.784+0000 7fa464dff700 1 -- 172.21.15.31:0/4125648519 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa46015fc30 con 0x7fa4601579d0 2023-10-06T02:27:58.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:58.784+0000 7fa464dff700 1 -- 172.21.15.31:0/4125648519 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa46015cea0 con 0x7fa4601579d0 2023-10-06T02:27:58.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:58.785+0000 7fa464dff700 1 -- 172.21.15.31:0/4125648519 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa46015ca20 con 0x7fa4601579d0 2023-10-06T02:27:58.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:58.786+0000 7fa44e7fc700 1 -- 172.21.15.31:0/4125648519 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7fa458003040 con 0x7fa4601579d0 2023-10-06T02:27:58.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:58.787+0000 7fa44e7fc700 1 --2- 172.21.15.31:0/4125648519 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa440067050 0x7fa440069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:58.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:58.787+0000 7fa44e7fc700 1 -- 172.21.15.31:0/4125648519 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa458086260 con 0x7fa4601579d0 2023-10-06T02:27:58.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:58.788+0000 7fa45e59c700 1 --2- 172.21.15.31:0/4125648519 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa440067050 0x7fa440069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:58.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:58.789+0000 7fa45e59c700 1 --2- 172.21.15.31:0/4125648519 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa440067050 0x7fa440069500 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa450006f00 tx=0x7fa450008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:58.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:58.791+0000 7fa44e7fc700 1 -- 172.21.15.31:0/4125648519 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa46015ca20 con 0x7fa4601579d0 2023-10-06T02:27:59.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.003+0000 7fa464dff700 1 -- 172.21.15.31:0/4125648519 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa46015ca20 con 0x7fa4601579d0 2023-10-06T02:27:59.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.004+0000 7fa44e7fc700 1 -- 172.21.15.31:0/4125648519 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+16834 (secure 0 0 0) 0x7fa46015ca20 con 0x7fa4601579d0 2023-10-06T02:27:59.005 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:27:59.005 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":86,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:27:58.018807+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:27:56.192975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-10-06T02:27:56.993140+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:27:59.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.007+0000 7fa464dff700 1 -- 172.21.15.31:0/4125648519 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa440067050 msgr2=0x7fa440069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:59.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.007+0000 7fa464dff700 1 --2- 172.21.15.31:0/4125648519 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa440067050 0x7fa440069500 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa450006f00 tx=0x7fa450008040 comp rx=0 tx=0).stop 2023-10-06T02:27:59.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.007+0000 7fa464dff700 1 -- 172.21.15.31:0/4125648519 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4601579d0 msgr2=0x7fa46015f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:59.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.007+0000 7fa464dff700 1 --2- 172.21.15.31:0/4125648519 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4601579d0 0x7fa46015f2b0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fa45800b6c0 tx=0x7fa458018000 comp rx=0 tx=0).stop 2023-10-06T02:27:59.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.007+0000 7fa464dff700 1 -- 172.21.15.31:0/4125648519 shutdown_connections 2023-10-06T02:27:59.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.007+0000 7fa464dff700 1 --2- 172.21.15.31:0/4125648519 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4601579d0 0x7fa46015f2b0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:59.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.007+0000 7fa464dff700 1 --2- 172.21.15.31:0/4125648519 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4600ccba0 0x7fa460156300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:59.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.007+0000 7fa464dff700 1 --2- 172.21.15.31:0/4125648519 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa440067050 0x7fa440069500 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:59.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.007+0000 7fa464dff700 1 --2- 172.21.15.31:0/4125648519 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa460157600 0x7fa46015ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:59.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.007+0000 7fa464dff700 1 -- 172.21.15.31:0/4125648519 >> 172.21.15.31:0/4125648519 conn(0x7fa46006d170 msgr2=0x7fa460159ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:59.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.007+0000 7fa464dff700 1 -- 172.21.15.31:0/4125648519 shutdown_connections 2023-10-06T02:27:59.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.007+0000 7fa464dff700 1 -- 172.21.15.31:0/4125648519 wait complete. 2023-10-06T02:27:59.025 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:27:59.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.211+0000 7fc26919d700 1 Processor -- start 2023-10-06T02:27:59.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.211+0000 7fc26919d700 1 -- start start 2023-10-06T02:27:59.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.212+0000 7fc26919d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc26406f3a0 0x7fc26414fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:59.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.212+0000 7fc26919d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc26406a490 con 0x7fc26406e0a0 2023-10-06T02:27:59.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.212+0000 7fc26919d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2640c7230 con 0x7fc26406e470 2023-10-06T02:27:59.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.212+0000 7fc26919d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2640c7490 con 0x7fc26406f3a0 2023-10-06T02:27:59.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.213+0000 7fc263fff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fc26406e0a0 0x7fc2640c6b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58064/0 (socket says 172.21.15.31:58064) 2023-10-06T02:27:59.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.213+0000 7fc263fff700 1 -- 172.21.15.31:0/3277867004 learned_addr learned my addr 172.21.15.31:0/3277867004 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:27:59.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.213+0000 7fc2637fe700 1 --2- 172.21.15.31:0/3277867004 >> v2:172.21.15.62:3300/0 conn(0x7fc26406f3a0 0x7fc26414fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:59.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.213+0000 7fc262ffd700 1 -- 172.21.15.31:0/3277867004 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2038461630 0 0) 0x7fc26406a490 con 0x7fc26406e0a0 2023-10-06T02:27:59.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.213+0000 7fc262ffd700 1 -- 172.21.15.31:0/3277867004 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc24c003570 con 0x7fc26406e0a0 2023-10-06T02:27:59.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.213+0000 7fc262ffd700 1 -- 172.21.15.31:0/3277867004 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 970770207 0 0) 0x7fc2640c7230 con 0x7fc26406e470 2023-10-06T02:27:59.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.213+0000 7fc262ffd700 1 -- 172.21.15.31:0/3277867004 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc26406a490 con 0x7fc26406e470 2023-10-06T02:27:59.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.213+0000 7fc262ffd700 1 -- 172.21.15.31:0/3277867004 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3189283712 0 0) 0x7fc24c003570 con 0x7fc26406e0a0 2023-10-06T02:27:59.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.213+0000 7fc262ffd700 1 -- 172.21.15.31:0/3277867004 >> v2:172.21.15.62:3300/0 conn(0x7fc26406f3a0 msgr2=0x7fc26414fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:59.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.213+0000 7fc262ffd700 1 --2- 172.21.15.31:0/3277867004 >> v2:172.21.15.62:3300/0 conn(0x7fc26406f3a0 0x7fc26414fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:59.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.213+0000 7fc262ffd700 1 -- 172.21.15.31:0/3277867004 >> v1:172.21.15.62:6789/0 conn(0x7fc26406e470 legacy=0x7fc26406a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:59.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.213+0000 7fc262ffd700 1 -- 172.21.15.31:0/3277867004 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2640c7770 con 0x7fc26406e0a0 2023-10-06T02:27:59.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.213+0000 7fc262ffd700 1 -- 172.21.15.31:0/3277867004 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7fc254001470 con 0x7fc26406e0a0 2023-10-06T02:27:59.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.214+0000 7fc262ffd700 1 -- 172.21.15.31:0/3277867004 >> v1:172.21.15.31:6789/0 conn(0x7fc26406e0a0 legacy=0x7fc2640c6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:59.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.214+0000 7fc262ffd700 1 --2- 172.21.15.31:0/3277867004 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc24c003bf0 0x7fc24c003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:59.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.214+0000 7fc262ffd700 1 --2- 172.21.15.31:0/3277867004 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc24c004810 0x7fc24c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:59.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.214+0000 7fc262ffd700 1 --2- 172.21.15.31:0/3277867004 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc24c0051c0 0x7fc24c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:59.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.214+0000 7fc262ffd700 1 -- 172.21.15.31:0/3277867004 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2640c7490 con 0x7fc24c003bf0 2023-10-06T02:27:59.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.214+0000 7fc262ffd700 1 -- 172.21.15.31:0/3277867004 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc24c00c0b0 con 0x7fc24c0051c0 2023-10-06T02:27:59.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.214+0000 7fc262ffd700 1 -- 172.21.15.31:0/3277867004 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc24c00c270 con 0x7fc24c004810 2023-10-06T02:27:59.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.214+0000 7fc263fff700 1 --2- 172.21.15.31:0/3277867004 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc24c003bf0 0x7fc24c003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:59.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.214+0000 7fc2637fe700 1 --2- 172.21.15.31:0/3277867004 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc24c004810 0x7fc24c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:59.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.214+0000 7fc26899c700 1 --2- 172.21.15.31:0/3277867004 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc24c0051c0 0x7fc24c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:59.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.214+0000 7fc263fff700 1 -- 172.21.15.31:0/3277867004 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc24c004810 msgr2=0x7fc24c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:59.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.214+0000 7fc263fff700 1 --2- 172.21.15.31:0/3277867004 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc24c004810 0x7fc24c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:59.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.214+0000 7fc263fff700 1 -- 172.21.15.31:0/3277867004 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc24c0051c0 msgr2=0x7fc24c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:59.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.214+0000 7fc263fff700 1 --2- 172.21.15.31:0/3277867004 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc24c0051c0 0x7fc24c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:59.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.214+0000 7fc263fff700 1 -- 172.21.15.31:0/3277867004 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc24c003570 con 0x7fc24c003bf0 2023-10-06T02:27:59.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.214+0000 7fc26899c700 1 -- 172.21.15.31:0/3277867004 reap_dead start 2023-10-06T02:27:59.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.215+0000 7fc263fff700 1 --2- 172.21.15.31:0/3277867004 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc24c003bf0 0x7fc24c003fe0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fc25400c2d0 tx=0x7fc254001e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1ccb56b0605ebae server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:59.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.215+0000 7fc262ffd700 1 -- 172.21.15.31:0/3277867004 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc25400cbc0 con 0x7fc24c003bf0 2023-10-06T02:27:59.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.215+0000 7fc262ffd700 1 -- 172.21.15.31:0/3277867004 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc25400cd20 con 0x7fc24c003bf0 2023-10-06T02:27:59.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.215+0000 7fc262ffd700 1 -- 172.21.15.31:0/3277867004 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc254005b70 con 0x7fc24c003bf0 2023-10-06T02:27:59.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.215+0000 7fc26919d700 1 -- 172.21.15.31:0/3277867004 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc24c003bf0 msgr2=0x7fc24c003fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:59.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.215+0000 7fc26919d700 1 --2- 172.21.15.31:0/3277867004 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc24c003bf0 0x7fc24c003fe0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fc25400c2d0 tx=0x7fc254001e10 comp rx=0 tx=0).stop 2023-10-06T02:27:59.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.216+0000 7fc26919d700 1 -- 172.21.15.31:0/3277867004 shutdown_connections 2023-10-06T02:27:59.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.216+0000 7fc26919d700 1 --2- 172.21.15.31:0/3277867004 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc24c003bf0 0x7fc24c003fe0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:59.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.216+0000 7fc26919d700 1 -- 172.21.15.31:0/3277867004 >> 172.21.15.31:0/3277867004 conn(0x7fc26406d170 msgr2=0x7fc2640cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:59.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.216+0000 7fc26919d700 1 -- 172.21.15.31:0/3277867004 shutdown_connections 2023-10-06T02:27:59.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.216+0000 7fc26919d700 1 -- 172.21.15.31:0/3277867004 wait complete. 2023-10-06T02:27:59.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.217+0000 7fc26919d700 1 Processor -- start 2023-10-06T02:27:59.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.217+0000 7fc26919d700 1 -- start start 2023-10-06T02:27:59.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.218+0000 7fc26919d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc24c003bf0 0x7fc26414fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:59.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.218+0000 7fc26919d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc26413c280 0x7fc26413cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:59.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.218+0000 7fc26919d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc26414ebd0 0x7fc26414f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:59.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.218+0000 7fc26919d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc26414f580 con 0x7fc26413c280 2023-10-06T02:27:59.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.218+0000 7fc26919d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc264157b20 con 0x7fc26414ebd0 2023-10-06T02:27:59.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.218+0000 7fc26919d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc26414ff80 con 0x7fc24c003bf0 2023-10-06T02:27:59.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.218+0000 7fc2637fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc26413c280 0x7fc26413cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:59.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.218+0000 7fc2637fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc26413c280 0x7fc26413cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40412/0 (socket says 172.21.15.31:40412) 2023-10-06T02:27:59.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.218+0000 7fc2637fe700 1 -- 172.21.15.31:0/721143138 learned_addr learned my addr 172.21.15.31:0/721143138 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:59.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.219+0000 7fc263fff700 1 --2- 172.21.15.31:0/721143138 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc24c003bf0 0x7fc26414fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:59.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.219+0000 7fc26899c700 1 --2- 172.21.15.31:0/721143138 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc26414ebd0 0x7fc26414f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:59.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.219+0000 7fc2637fe700 1 -- 172.21.15.31:0/721143138 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc24c003bf0 msgr2=0x7fc26414fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:59.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.219+0000 7fc2637fe700 1 --2- 172.21.15.31:0/721143138 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc24c003bf0 0x7fc26414fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:59.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.219+0000 7fc2637fe700 1 -- 172.21.15.31:0/721143138 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc26414ebd0 msgr2=0x7fc26414f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:59.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.219+0000 7fc2637fe700 1 --2- 172.21.15.31:0/721143138 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc26414ebd0 0x7fc26414f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:59.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.219+0000 7fc2637fe700 1 -- 172.21.15.31:0/721143138 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc264156c30 con 0x7fc26413c280 2023-10-06T02:27:59.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.219+0000 7fc2637fe700 1 --2- 172.21.15.31:0/721143138 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc26413c280 0x7fc26413cb00 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fc258011a00 tx=0x7fc258009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:59.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.219+0000 7fc24bfff700 1 -- 172.21.15.31:0/721143138 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc258005a00 con 0x7fc26413c280 2023-10-06T02:27:59.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.219+0000 7fc24bfff700 1 -- 172.21.15.31:0/721143138 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc258005b60 con 0x7fc26413c280 2023-10-06T02:27:59.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.219+0000 7fc26919d700 1 -- 172.21.15.31:0/721143138 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc264156f10 con 0x7fc26413c280 2023-10-06T02:27:59.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.219+0000 7fc24bfff700 1 -- 172.21.15.31:0/721143138 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc25800f940 con 0x7fc26413c280 2023-10-06T02:27:59.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.219+0000 7fc26919d700 1 -- 172.21.15.31:0/721143138 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2641574d0 con 0x7fc26413c280 2023-10-06T02:27:59.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.220+0000 7fc26919d700 1 -- 172.21.15.31:0/721143138 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc26404c3c0 con 0x7fc26413c280 2023-10-06T02:27:59.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.221+0000 7fc24bfff700 1 -- 172.21.15.31:0/721143138 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7fc25800fd30 con 0x7fc26413c280 2023-10-06T02:27:59.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.223+0000 7fc24bfff700 1 --2- 172.21.15.31:0/721143138 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc250067050 0x7fc250069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:59.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.223+0000 7fc24bfff700 1 -- 172.21.15.31:0/721143138 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc258018030 con 0x7fc26413c280 2023-10-06T02:27:59.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.223+0000 7fc263fff700 1 --2- 172.21.15.31:0/721143138 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc250067050 0x7fc250069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:59.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.224+0000 7fc263fff700 1 --2- 172.21.15.31:0/721143138 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc250067050 0x7fc250069500 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc254008fd0 tx=0x7fc254015000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:59.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.227+0000 7fc24bfff700 1 -- 172.21.15.31:0/721143138 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc258052260 con 0x7fc26413c280 2023-10-06T02:27:59.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.429+0000 7fc26919d700 1 -- 172.21.15.31:0/721143138 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc2640c7a50 con 0x7fc26413c280 2023-10-06T02:27:59.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.430+0000 7fc24bfff700 1 -- 172.21.15.31:0/721143138 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+16834 (secure 0 0 0) 0x7fc258055880 con 0x7fc26413c280 2023-10-06T02:27:59.431 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:27:59.431 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":86,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:27:58.018807+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:27:56.192975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-10-06T02:27:56.993140+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:27:59.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.432+0000 7fc26919d700 1 -- 172.21.15.31:0/721143138 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc250067050 msgr2=0x7fc250069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:59.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.432+0000 7fc26919d700 1 --2- 172.21.15.31:0/721143138 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc250067050 0x7fc250069500 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc254008fd0 tx=0x7fc254015000 comp rx=0 tx=0).stop 2023-10-06T02:27:59.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.432+0000 7fc26919d700 1 -- 172.21.15.31:0/721143138 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc26413c280 msgr2=0x7fc26413cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:59.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.432+0000 7fc26919d700 1 --2- 172.21.15.31:0/721143138 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc26413c280 0x7fc26413cb00 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fc258011a00 tx=0x7fc258009120 comp rx=0 tx=0).stop 2023-10-06T02:27:59.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.432+0000 7fc26919d700 1 -- 172.21.15.31:0/721143138 shutdown_connections 2023-10-06T02:27:59.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.432+0000 7fc26919d700 1 --2- 172.21.15.31:0/721143138 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc26414ebd0 0x7fc26414f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:59.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.432+0000 7fc26919d700 1 --2- 172.21.15.31:0/721143138 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc250067050 0x7fc250069500 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:59.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.432+0000 7fc26919d700 1 --2- 172.21.15.31:0/721143138 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc24c003bf0 0x7fc26414fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:59.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.432+0000 7fc26919d700 1 --2- 172.21.15.31:0/721143138 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc26413c280 0x7fc26413cb00 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:59.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.432+0000 7fc26919d700 1 -- 172.21.15.31:0/721143138 >> 172.21.15.31:0/721143138 conn(0x7fc26406d170 msgr2=0x7fc2640b0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:59.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.432+0000 7fc26919d700 1 -- 172.21.15.31:0/721143138 shutdown_connections 2023-10-06T02:27:59.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.433+0000 7fc26919d700 1 -- 172.21.15.31:0/721143138 wait complete. 2023-10-06T02:27:59.451 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T02:27:59.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.645+0000 7efd6cf9d700 1 Processor -- start 2023-10-06T02:27:59.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.646+0000 7efd6cf9d700 1 -- start start 2023-10-06T02:27:59.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.647+0000 7efd6cf9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7efd680caa90 0x7efd680c79d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:59.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.647+0000 7efd6cf9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd6806a280 con 0x7efd680ca6c0 2023-10-06T02:27:59.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.647+0000 7efd6cf9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd6815e290 con 0x7efd6806b720 2023-10-06T02:27:59.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.647+0000 7efd6cf9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd680c89e0 con 0x7efd680caa90 2023-10-06T02:27:59.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.647+0000 7efd677fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7efd680ca6c0 0x7efd6815d470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58072/0 (socket says 172.21.15.31:58072) 2023-10-06T02:27:59.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.647+0000 7efd677fe700 1 -- 172.21.15.31:0/2232920152 learned_addr learned my addr 172.21.15.31:0/2232920152 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:27:59.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.647+0000 7efd67fff700 1 --2- 172.21.15.31:0/2232920152 >> v2:172.21.15.62:3300/0 conn(0x7efd680caa90 0x7efd680c79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:59.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.648+0000 7efd667fc700 1 -- 172.21.15.31:0/2232920152 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 418111702 0 0) 0x7efd6806a280 con 0x7efd680ca6c0 2023-10-06T02:27:59.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.648+0000 7efd667fc700 1 -- 172.21.15.31:0/2232920152 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd50003570 con 0x7efd680ca6c0 2023-10-06T02:27:59.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.648+0000 7efd667fc700 1 -- 172.21.15.31:0/2232920152 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1859971747 0 0) 0x7efd50003570 con 0x7efd680ca6c0 2023-10-06T02:27:59.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.648+0000 7efd667fc700 1 -- 172.21.15.31:0/2232920152 >> v2:172.21.15.62:3300/0 conn(0x7efd680caa90 msgr2=0x7efd680c79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:59.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.648+0000 7efd667fc700 1 --2- 172.21.15.31:0/2232920152 >> v2:172.21.15.62:3300/0 conn(0x7efd680caa90 0x7efd680c79d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:59.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.648+0000 7efd667fc700 1 -- 172.21.15.31:0/2232920152 >> v1:172.21.15.62:6789/0 conn(0x7efd6806b720 legacy=0x7efd6815db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:59.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.648+0000 7efd667fc700 1 -- 172.21.15.31:0/2232920152 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd680c8700 con 0x7efd680ca6c0 2023-10-06T02:27:59.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.648+0000 7efd667fc700 1 -- 172.21.15.31:0/2232920152 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7efd58001470 con 0x7efd680ca6c0 2023-10-06T02:27:59.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.648+0000 7efd667fc700 1 -- 172.21.15.31:0/2232920152 >> v1:172.21.15.31:6789/0 conn(0x7efd680ca6c0 legacy=0x7efd6815d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:59.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.648+0000 7efd667fc700 1 --2- 172.21.15.31:0/2232920152 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efd50003a60 0x7efd50003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:59.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.648+0000 7efd667fc700 1 --2- 172.21.15.31:0/2232920152 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd50004810 0x7efd50004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:59.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.648+0000 7efd667fc700 1 --2- 172.21.15.31:0/2232920152 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd500051c0 0x7efd5000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:59.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.648+0000 7efd667fc700 1 -- 172.21.15.31:0/2232920152 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd680c89e0 con 0x7efd50004810 2023-10-06T02:27:59.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.648+0000 7efd667fc700 1 -- 172.21.15.31:0/2232920152 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd5000c130 con 0x7efd500051c0 2023-10-06T02:27:59.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.648+0000 7efd667fc700 1 -- 172.21.15.31:0/2232920152 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd5000c2f0 con 0x7efd50003a60 2023-10-06T02:27:59.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.648+0000 7efd66ffd700 1 --2- 172.21.15.31:0/2232920152 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd50004810 0x7efd50004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:59.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.649+0000 7efd67fff700 1 --2- 172.21.15.31:0/2232920152 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd500051c0 0x7efd5000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:59.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.649+0000 7efd677fe700 1 --2- 172.21.15.31:0/2232920152 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efd50003a60 0x7efd50003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:59.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.649+0000 7efd66ffd700 1 -- 172.21.15.31:0/2232920152 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efd50003a60 msgr2=0x7efd50003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:59.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.649+0000 7efd66ffd700 1 --2- 172.21.15.31:0/2232920152 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efd50003a60 0x7efd50003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:59.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.649+0000 7efd66ffd700 1 -- 172.21.15.31:0/2232920152 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd500051c0 msgr2=0x7efd5000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:27:59.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.649+0000 7efd66ffd700 1 --2- 172.21.15.31:0/2232920152 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd500051c0 0x7efd5000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:59.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.649+0000 7efd67fff700 1 -- 172.21.15.31:0/2232920152 reap_dead start 2023-10-06T02:27:59.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.649+0000 7efd66ffd700 1 -- 172.21.15.31:0/2232920152 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd50003570 con 0x7efd50004810 2023-10-06T02:27:59.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.649+0000 7efd66ffd700 1 --2- 172.21.15.31:0/2232920152 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd50004810 0x7efd50004c80 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7efd5c001970 tx=0x7efd5c007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2394fc7f0646a182 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:59.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.649+0000 7efd667fc700 1 -- 172.21.15.31:0/2232920152 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd5c00c5d0 con 0x7efd50004810 2023-10-06T02:27:59.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.649+0000 7efd667fc700 1 -- 172.21.15.31:0/2232920152 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd5c00c730 con 0x7efd50004810 2023-10-06T02:27:59.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.649+0000 7efd667fc700 1 -- 172.21.15.31:0/2232920152 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd5c00cad0 con 0x7efd50004810 2023-10-06T02:27:59.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.650+0000 7efd6cf9d700 1 -- 172.21.15.31:0/2232920152 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd50004810 msgr2=0x7efd50004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:59.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.650+0000 7efd6cf9d700 1 --2- 172.21.15.31:0/2232920152 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd50004810 0x7efd50004c80 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7efd5c001970 tx=0x7efd5c007d10 comp rx=0 tx=0).stop 2023-10-06T02:27:59.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.650+0000 7efd6cf9d700 1 -- 172.21.15.31:0/2232920152 shutdown_connections 2023-10-06T02:27:59.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.650+0000 7efd6cf9d700 1 --2- 172.21.15.31:0/2232920152 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd50004810 0x7efd50004c80 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:59.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.650+0000 7efd6cf9d700 1 -- 172.21.15.31:0/2232920152 >> 172.21.15.31:0/2232920152 conn(0x7efd6806d110 msgr2=0x7efd680cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:27:59.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.650+0000 7efd6cf9d700 1 -- 172.21.15.31:0/2232920152 shutdown_connections 2023-10-06T02:27:59.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.650+0000 7efd6cf9d700 1 -- 172.21.15.31:0/2232920152 wait complete. 2023-10-06T02:27:59.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.651+0000 7efd6cf9d700 1 Processor -- start 2023-10-06T02:27:59.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.652+0000 7efd6cf9d700 1 -- start start 2023-10-06T02:27:59.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.652+0000 7efd6cf9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efd50004810 0x7efd68156750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:59.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.652+0000 7efd6cf9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd6815acd0 0x7efd680c3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:59.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.652+0000 7efd6cf9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd68154c60 0x7efd68155590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:59.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.652+0000 7efd6cf9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd6815bc90 con 0x7efd68154c60 2023-10-06T02:27:59.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.652+0000 7efd6cf9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd680c56e0 con 0x7efd6815acd0 2023-10-06T02:27:59.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.652+0000 7efd6cf9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd680c4260 con 0x7efd50004810 2023-10-06T02:27:59.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.653+0000 7efd67fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd68154c60 0x7efd68155590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:59.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.653+0000 7efd67fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd68154c60 0x7efd68155590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40432/0 (socket says 172.21.15.31:40432) 2023-10-06T02:27:59.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.653+0000 7efd67fff700 1 -- 172.21.15.31:0/2956079800 learned_addr learned my addr 172.21.15.31:0/2956079800 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:27:59.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.653+0000 7efd677fe700 1 --2- 172.21.15.31:0/2956079800 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efd50004810 0x7efd68156750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:59.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.653+0000 7efd66ffd700 1 --2- 172.21.15.31:0/2956079800 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd6815acd0 0x7efd680c3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:59.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.653+0000 7efd67fff700 1 -- 172.21.15.31:0/2956079800 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efd50004810 msgr2=0x7efd68156750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:59.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.653+0000 7efd67fff700 1 --2- 172.21.15.31:0/2956079800 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efd50004810 0x7efd68156750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:59.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.653+0000 7efd67fff700 1 -- 172.21.15.31:0/2956079800 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd6815acd0 msgr2=0x7efd680c3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:27:59.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.653+0000 7efd67fff700 1 --2- 172.21.15.31:0/2956079800 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd6815acd0 0x7efd680c3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:27:59.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.653+0000 7efd67fff700 1 -- 172.21.15.31:0/2956079800 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd680c5960 con 0x7efd68154c60 2023-10-06T02:27:59.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.653+0000 7efd66ffd700 1 --2- 172.21.15.31:0/2956079800 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd6815acd0 0x7efd680c3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:27:59.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.653+0000 7efd67fff700 1 --2- 172.21.15.31:0/2956079800 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd68154c60 0x7efd68155590 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7efd680697f0 tx=0x7efd6000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:59.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.654+0000 7efd4f7fe700 1 -- 172.21.15.31:0/2956079800 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd6000e070 con 0x7efd68154c60 2023-10-06T02:27:59.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.654+0000 7efd4f7fe700 1 -- 172.21.15.31:0/2956079800 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd60011d00 con 0x7efd68154c60 2023-10-06T02:27:59.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.654+0000 7efd4f7fe700 1 -- 172.21.15.31:0/2956079800 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd600166a0 con 0x7efd68154c60 2023-10-06T02:27:59.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.654+0000 7efd6cf9d700 1 -- 172.21.15.31:0/2956079800 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd680c5c40 con 0x7efd68154c60 2023-10-06T02:27:59.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.654+0000 7efd6cf9d700 1 -- 172.21.15.31:0/2956079800 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd680c5f20 con 0x7efd68154c60 2023-10-06T02:27:59.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.655+0000 7efd6cf9d700 1 -- 172.21.15.31:0/2956079800 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd6815be60 con 0x7efd68154c60 2023-10-06T02:27:59.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.656+0000 7efd4f7fe700 1 -- 172.21.15.31:0/2956079800 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7efd60016ad0 con 0x7efd68154c60 2023-10-06T02:27:59.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.656+0000 7efd4f7fe700 1 --2- 172.21.15.31:0/2956079800 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7efd54067050 0x7efd54069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:27:59.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.657+0000 7efd4f7fe700 1 -- 172.21.15.31:0/2956079800 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6058+0+0 (secure 0 0 0) 0x7efd600856c0 con 0x7efd68154c60 2023-10-06T02:27:59.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.657+0000 7efd677fe700 1 --2- 172.21.15.31:0/2956079800 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7efd54067050 0x7efd54069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:27:59.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.657+0000 7efd677fe700 1 --2- 172.21.15.31:0/2956079800 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7efd54067050 0x7efd54069500 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7efd58001120 tx=0x7efd58009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:27:59.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.660+0000 7efd4f7fe700 1 -- 172.21.15.31:0/2956079800 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efd6815be60 con 0x7efd68154c60 2023-10-06T02:27:59.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:27:59.909+0000 7efd6cf9d700 1 -- 172.21.15.31:0/2956079800 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7efd6815be30 con 0x7efd68154c60 2023-10-06T02:28:00.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:00.556+0000 7efd4f7fe700 1 -- 172.21.15.31:0/2956079800 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v55) v1 ==== 118+0+0 (secure 0 0 0) 0x7efd60052510 con 0x7efd68154c60 2023-10-06T02:28:00.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:00.559+0000 7efd6cf9d700 1 -- 172.21.15.31:0/2956079800 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7efd54067050 msgr2=0x7efd54069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:00.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:00.559+0000 7efd6cf9d700 1 --2- 172.21.15.31:0/2956079800 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7efd54067050 0x7efd54069500 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7efd58001120 tx=0x7efd58009040 comp rx=0 tx=0).stop 2023-10-06T02:28:00.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:00.559+0000 7efd6cf9d700 1 -- 172.21.15.31:0/2956079800 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd68154c60 msgr2=0x7efd68155590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:00.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:00.559+0000 7efd6cf9d700 1 --2- 172.21.15.31:0/2956079800 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd68154c60 0x7efd68155590 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7efd680697f0 tx=0x7efd6000b040 comp rx=0 tx=0).stop 2023-10-06T02:28:00.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:00.559+0000 7efd6cf9d700 1 -- 172.21.15.31:0/2956079800 shutdown_connections 2023-10-06T02:28:00.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:00.559+0000 7efd6cf9d700 1 --2- 172.21.15.31:0/2956079800 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd68154c60 0x7efd68155590 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:00.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:00.559+0000 7efd6cf9d700 1 --2- 172.21.15.31:0/2956079800 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7efd54067050 0x7efd54069500 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:00.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:00.559+0000 7efd6cf9d700 1 --2- 172.21.15.31:0/2956079800 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efd50004810 0x7efd68156750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:00.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:00.559+0000 7efd6cf9d700 1 --2- 172.21.15.31:0/2956079800 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd6815acd0 0x7efd680c3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:00.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:00.559+0000 7efd6cf9d700 1 -- 172.21.15.31:0/2956079800 >> 172.21.15.31:0/2956079800 conn(0x7efd6806d110 msgr2=0x7efd680c69c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:00.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:00.559+0000 7efd6cf9d700 1 -- 172.21.15.31:0/2956079800 shutdown_connections 2023-10-06T02:28:00.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:00.559+0000 7efd6cf9d700 1 -- 172.21.15.31:0/2956079800 wait complete. 2023-10-06T02:28:00.580 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T02:28:00.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:00.774+0000 7ff34359e700 1 Processor -- start 2023-10-06T02:28:00.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:00.775+0000 7ff34359e700 1 -- start start 2023-10-06T02:28:00.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:00.776+0000 7ff34359e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff344153c90 0x7ff34415a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:00.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:00.776+0000 7ff34359e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff34406db20 0x7ff3440ca7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:00.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:00.776+0000 7ff34359e700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff34415ada0 con 0x7ff344154060 2023-10-06T02:28:00.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:00.776+0000 7ff34359e700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff34406ca80 con 0x7ff34406db20 2023-10-06T02:28:00.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:00.776+0000 7ff34359e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff34406cd60 con 0x7ff344153c90 2023-10-06T02:28:00.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:00.776+0000 7ff33bfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff34406db20 0x7ff3440ca7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:00.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:00.776+0000 7ff342d9d700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7ff344154060 0x7ff34406a2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58086/0 (socket says 172.21.15.31:58086) 2023-10-06T02:28:00.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:00.776+0000 7ff342d9d700 1 -- 172.21.15.31:0/2451625007 learned_addr learned my addr 172.21.15.31:0/2451625007 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:28:00.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:00.776+0000 7ff34259c700 1 --2- 172.21.15.31:0/2451625007 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff344153c90 0x7ff34415a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:00.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:00.777+0000 7ff33bfff700 1 -- 172.21.15.31:0/2451625007 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff344153c90 msgr2=0x7ff34415a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:00.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:00.777+0000 7ff33bfff700 1 --2- 172.21.15.31:0/2451625007 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff344153c90 0x7ff34415a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:00.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:00.777+0000 7ff33bfff700 1 -- 172.21.15.31:0/2451625007 >> v1:172.21.15.31:6789/0 conn(0x7ff344154060 legacy=0x7ff34406a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:00.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:00.777+0000 7ff33bfff700 1 -- 172.21.15.31:0/2451625007 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff34406d040 con 0x7ff34406db20 2023-10-06T02:28:00.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:00.777+0000 7ff341d9b700 1 -- 172.21.15.31:0/2451625007 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 279044726 0 0) 0x7ff33c001490 con 0x7ff344154060 2023-10-06T02:28:00.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:00.777+0000 7ff33bfff700 1 --2- 172.21.15.31:0/2451625007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff34406db20 0x7ff3440ca7b0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7ff32c0098e0 tx=0x7ff32c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63985c7c415fde3f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:00.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:00.777+0000 7ff341d9b700 1 -- 172.21.15.31:0/2451625007 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32c009e80 con 0x7ff34406db20 2023-10-06T02:28:00.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:00.777+0000 7ff341d9b700 1 -- 172.21.15.31:0/2451625007 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff32c00b860 con 0x7ff34406db20 2023-10-06T02:28:00.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:00.777+0000 7ff341d9b700 1 -- 172.21.15.31:0/2451625007 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32c00bbe0 con 0x7ff34406db20 2023-10-06T02:28:00.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:00.778+0000 7ff34359e700 1 -- 172.21.15.31:0/2451625007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff34406db20 msgr2=0x7ff3440ca7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:00.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:00.778+0000 7ff34359e700 1 --2- 172.21.15.31:0/2451625007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff34406db20 0x7ff3440ca7b0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7ff32c0098e0 tx=0x7ff32c0091a0 comp rx=0 tx=0).stop 2023-10-06T02:28:00.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:00.778+0000 7ff34359e700 1 -- 172.21.15.31:0/2451625007 shutdown_connections 2023-10-06T02:28:00.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:00.778+0000 7ff34359e700 1 --2- 172.21.15.31:0/2451625007 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff344153c90 0x7ff34415a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:00.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:00.778+0000 7ff34359e700 1 --2- 172.21.15.31:0/2451625007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff34406db20 0x7ff3440ca7b0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:00.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:00.778+0000 7ff34359e700 1 -- 172.21.15.31:0/2451625007 >> 172.21.15.31:0/2451625007 conn(0x7ff34406e720 msgr2=0x7ff34406eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:00.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:00.778+0000 7ff34359e700 1 -- 172.21.15.31:0/2451625007 shutdown_connections 2023-10-06T02:28:00.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:00.778+0000 7ff34359e700 1 -- 172.21.15.31:0/2451625007 wait complete. 2023-10-06T02:28:00.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:00.779+0000 7ff34359e700 1 Processor -- start 2023-10-06T02:28:00.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:00.780+0000 7ff34359e700 1 -- start start 2023-10-06T02:28:00.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:00.780+0000 7ff34359e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff34406db20 0x7ff344152b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:00.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:00.781+0000 7ff34359e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff344153c90 0x7ff344152320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:00.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:00.781+0000 7ff34359e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff344154060 0x7ff3440c5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:00.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:00.781+0000 7ff34359e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3440c7200 con 0x7ff344154060 2023-10-06T02:28:00.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:00.781+0000 7ff34359e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3440c69f0 con 0x7ff344153c90 2023-10-06T02:28:00.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:00.781+0000 7ff34359e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3440c5b20 con 0x7ff34406db20 2023-10-06T02:28:00.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:00.781+0000 7ff342d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff344154060 0x7ff3440c5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:00.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:00.781+0000 7ff342d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff344154060 0x7ff3440c5fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40448/0 (socket says 172.21.15.31:40448) 2023-10-06T02:28:00.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:00.781+0000 7ff342d9d700 1 -- 172.21.15.31:0/903251744 learned_addr learned my addr 172.21.15.31:0/903251744 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:00.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:00.781+0000 7ff34259c700 1 --2- 172.21.15.31:0/903251744 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff34406db20 0x7ff344152b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:00.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:00.781+0000 7ff33bfff700 1 --2- 172.21.15.31:0/903251744 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff344153c90 0x7ff344152320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:00.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:00.781+0000 7ff342d9d700 1 -- 172.21.15.31:0/903251744 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff34406db20 msgr2=0x7ff344152b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:00.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:00.781+0000 7ff342d9d700 1 --2- 172.21.15.31:0/903251744 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff34406db20 0x7ff344152b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:00.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:00.781+0000 7ff342d9d700 1 -- 172.21.15.31:0/903251744 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff344153c90 msgr2=0x7ff344152320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:00.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:00.781+0000 7ff342d9d700 1 --2- 172.21.15.31:0/903251744 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff344153c90 0x7ff344152320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:00.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:00.781+0000 7ff342d9d700 1 -- 172.21.15.31:0/903251744 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff32c009420 con 0x7ff344154060 2023-10-06T02:28:00.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:00.781+0000 7ff34259c700 1 --2- 172.21.15.31:0/903251744 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff34406db20 0x7ff344152b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:28:00.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:00.781+0000 7ff33bfff700 1 --2- 172.21.15.31:0/903251744 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff344153c90 0x7ff344152320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:28:00.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:00.782+0000 7ff342d9d700 1 --2- 172.21.15.31:0/903251744 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff344154060 0x7ff3440c5fb0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7ff33c00cc70 tx=0x7ff33c001a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:00.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:00.782+0000 7ff33a7fc700 1 -- 172.21.15.31:0/903251744 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff33c00ce80 con 0x7ff344154060 2023-10-06T02:28:00.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:00.782+0000 7ff33a7fc700 1 -- 172.21.15.31:0/903251744 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff33c004bb0 con 0x7ff344154060 2023-10-06T02:28:00.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:00.782+0000 7ff33a7fc700 1 -- 172.21.15.31:0/903251744 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff33c00ab70 con 0x7ff344154060 2023-10-06T02:28:00.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:00.782+0000 7ff34359e700 1 -- 172.21.15.31:0/903251744 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3440c6c40 con 0x7ff344154060 2023-10-06T02:28:00.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:00.782+0000 7ff34359e700 1 -- 172.21.15.31:0/903251744 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3440c57b0 con 0x7ff344154060 2023-10-06T02:28:00.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:00.783+0000 7ff34359e700 1 -- 172.21.15.31:0/903251744 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3440c5cb0 con 0x7ff344154060 2023-10-06T02:28:00.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:00.784+0000 7ff33a7fc700 1 -- 172.21.15.31:0/903251744 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7ff33c0070f0 con 0x7ff344154060 2023-10-06T02:28:00.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:00.785+0000 7ff33a7fc700 1 --2- 172.21.15.31:0/903251744 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff324067050 0x7ff324069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:00.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:00.785+0000 7ff33a7fc700 1 -- 172.21.15.31:0/903251744 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff33c084600 con 0x7ff344154060 2023-10-06T02:28:00.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:00.785+0000 7ff34259c700 1 --2- 172.21.15.31:0/903251744 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff324067050 0x7ff324069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:00.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:00.786+0000 7ff34259c700 1 --2- 172.21.15.31:0/903251744 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff324067050 0x7ff324069500 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff334006fd0 tx=0x7ff334008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:00.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:00.789+0000 7ff33a7fc700 1 -- 172.21.15.31:0/903251744 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff33c0515e0 con 0x7ff344154060 2023-10-06T02:28:01.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:01.038+0000 7ff34359e700 1 -- 172.21.15.31:0/903251744 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7ff34415ada0 con 0x7ff344154060 2023-10-06T02:28:01.448 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T02:28:01.453 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T02:28:01.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:01.558+0000 7ff33a7fc700 1 -- 172.21.15.31:0/903251744 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v56) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff34415ada0 con 0x7ff344154060 2023-10-06T02:28:01.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:01.561+0000 7ff34359e700 1 -- 172.21.15.31:0/903251744 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff324067050 msgr2=0x7ff324069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:01.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:01.561+0000 7ff34359e700 1 --2- 172.21.15.31:0/903251744 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff324067050 0x7ff324069500 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff334006fd0 tx=0x7ff334008040 comp rx=0 tx=0).stop 2023-10-06T02:28:01.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:01.561+0000 7ff34359e700 1 -- 172.21.15.31:0/903251744 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff344154060 msgr2=0x7ff3440c5fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:01.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:01.561+0000 7ff34359e700 1 --2- 172.21.15.31:0/903251744 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff344154060 0x7ff3440c5fb0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7ff33c00cc70 tx=0x7ff33c001a30 comp rx=0 tx=0).stop 2023-10-06T02:28:01.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:01.561+0000 7ff34359e700 1 -- 172.21.15.31:0/903251744 shutdown_connections 2023-10-06T02:28:01.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:01.561+0000 7ff34359e700 1 --2- 172.21.15.31:0/903251744 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff344154060 0x7ff3440c5fb0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:01.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:01.561+0000 7ff34359e700 1 --2- 172.21.15.31:0/903251744 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff324067050 0x7ff324069500 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:01.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:01.561+0000 7ff34359e700 1 --2- 172.21.15.31:0/903251744 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff34406db20 0x7ff344152b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:01.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:01.561+0000 7ff34359e700 1 --2- 172.21.15.31:0/903251744 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff344153c90 0x7ff344152320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:01.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:01.561+0000 7ff34359e700 1 -- 172.21.15.31:0/903251744 >> 172.21.15.31:0/903251744 conn(0x7ff34406e720 msgr2=0x7ff34415c020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:01.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:01.561+0000 7ff34359e700 1 -- 172.21.15.31:0/903251744 shutdown_connections 2023-10-06T02:28:01.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:01.561+0000 7ff34359e700 1 -- 172.21.15.31:0/903251744 wait complete. 2023-10-06T02:28:01.579 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:28:01.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:01.806+0000 7fec0f19d700 1 Processor -- start 2023-10-06T02:28:01.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:01.806+0000 7fec0f19d700 1 -- start start 2023-10-06T02:28:01.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:01.807+0000 7fec0f19d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fec0806e0a0 0x7fec080c6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:01.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:01.807+0000 7fec0f19d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fec0806f3a0 0x7fec0814fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:01.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:01.807+0000 7fec0f19d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fec0806e470 0x7fec081501b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:01.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:01.807+0000 7fec0f19d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec0806a490 con 0x7fec0806f3a0 2023-10-06T02:28:01.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:01.807+0000 7fec0f19d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec080c7490 con 0x7fec0806e0a0 2023-10-06T02:28:01.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:01.807+0000 7fec0f19d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec080c7770 con 0x7fec0806e470 2023-10-06T02:28:01.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:01.808+0000 7fec0d99a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fec0806f3a0 0x7fec0814fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:01.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:01.808+0000 7fec0d99a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fec0806f3a0 0x7fec0814fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40456/0 (socket says 172.21.15.31:40456) 2023-10-06T02:28:01.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:01.808+0000 7fec0d99a700 1 -- 172.21.15.31:0/2721167774 learned_addr learned my addr 172.21.15.31:0/2721167774 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:01.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:01.808+0000 7fec0e19b700 1 --2- 172.21.15.31:0/2721167774 >> v2:172.21.15.62:3300/0 conn(0x7fec0806e0a0 0x7fec080c6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:01.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:01.808+0000 7fec0e99c700 1 --2- 172.21.15.31:0/2721167774 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fec0806e470 0x7fec081501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:01.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:01.808+0000 7fec0d99a700 1 -- 172.21.15.31:0/2721167774 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fec0806e470 msgr2=0x7fec081501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:01.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:01.808+0000 7fec0d99a700 1 --2- 172.21.15.31:0/2721167774 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fec0806e470 0x7fec081501b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:01.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:01.808+0000 7fec0d99a700 1 -- 172.21.15.31:0/2721167774 >> v2:172.21.15.62:3300/0 conn(0x7fec0806e0a0 msgr2=0x7fec080c6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:01.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:01.808+0000 7fec0d99a700 1 --2- 172.21.15.31:0/2721167774 >> v2:172.21.15.62:3300/0 conn(0x7fec0806e0a0 0x7fec080c6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:01.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:01.808+0000 7fec0d99a700 1 -- 172.21.15.31:0/2721167774 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec080c7a50 con 0x7fec0806f3a0 2023-10-06T02:28:01.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:01.808+0000 7fec0e19b700 1 --2- 172.21.15.31:0/2721167774 >> v2:172.21.15.62:3300/0 conn(0x7fec0806e0a0 0x7fec080c6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:28:01.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:01.808+0000 7fec0e99c700 1 --2- 172.21.15.31:0/2721167774 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fec0806e470 0x7fec081501b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:28:01.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:01.808+0000 7fec0d99a700 1 --2- 172.21.15.31:0/2721167774 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec0806f3a0 0x7fec0814fc70 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fec08069230 tx=0x7febf8009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56838d9ad37e9f25 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:01.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:01.809+0000 7fec0d199700 1 -- 172.21.15.31:0/2721167774 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febf8005c10 con 0x7fec0806f3a0 2023-10-06T02:28:01.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:01.809+0000 7fec0d199700 1 -- 172.21.15.31:0/2721167774 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febf8005d70 con 0x7fec0806f3a0 2023-10-06T02:28:01.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:01.809+0000 7fec0d199700 1 -- 172.21.15.31:0/2721167774 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febf800b880 con 0x7fec0806f3a0 2023-10-06T02:28:01.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:01.809+0000 7fec0f19d700 1 -- 172.21.15.31:0/2721167774 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec0806f3a0 msgr2=0x7fec0814fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:01.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:01.809+0000 7fec0f19d700 1 --2- 172.21.15.31:0/2721167774 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec0806f3a0 0x7fec0814fc70 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fec08069230 tx=0x7febf8009220 comp rx=0 tx=0).stop 2023-10-06T02:28:01.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:01.809+0000 7fec0f19d700 1 -- 172.21.15.31:0/2721167774 shutdown_connections 2023-10-06T02:28:01.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:01.809+0000 7fec0f19d700 1 --2- 172.21.15.31:0/2721167774 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fec0806e470 0x7fec081501b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:01.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:01.809+0000 7fec0f19d700 1 --2- 172.21.15.31:0/2721167774 >> v2:172.21.15.62:3300/0 conn(0x7fec0806e0a0 0x7fec080c6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:01.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:01.809+0000 7fec0f19d700 1 --2- 172.21.15.31:0/2721167774 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec0806f3a0 0x7fec0814fc70 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:01.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:01.809+0000 7fec0f19d700 1 -- 172.21.15.31:0/2721167774 >> 172.21.15.31:0/2721167774 conn(0x7fec0806d170 msgr2=0x7fec080cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:01.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:01.809+0000 7fec0f19d700 1 -- 172.21.15.31:0/2721167774 shutdown_connections 2023-10-06T02:28:01.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:01.809+0000 7fec0f19d700 1 -- 172.21.15.31:0/2721167774 wait complete. 2023-10-06T02:28:01.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:01.811+0000 7fec0f19d700 1 Processor -- start 2023-10-06T02:28:01.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:01.811+0000 7fec0f19d700 1 -- start start 2023-10-06T02:28:01.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:01.812+0000 7fec0f19d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec0806e0a0 0x7fec0816a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:01.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:01.812+0000 7fec0f19d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec0806e470 0x7fec08146de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:01.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:01.812+0000 7fec0f19d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fec0806f3a0 0x7fec08147320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:01.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:01.812+0000 7fec0f19d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec08147860 con 0x7fec0806e470 2023-10-06T02:28:01.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:01.812+0000 7fec0f19d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec081455d0 con 0x7fec0806e0a0 2023-10-06T02:28:01.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:01.812+0000 7fec0f19d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec081591e0 con 0x7fec0806f3a0 2023-10-06T02:28:01.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:01.812+0000 7fec0d99a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec0806e470 0x7fec08146de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:01.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:01.812+0000 7fec0d99a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec0806e470 0x7fec08146de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40472/0 (socket says 172.21.15.31:40472) 2023-10-06T02:28:01.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:01.812+0000 7fec0d99a700 1 -- 172.21.15.31:0/2236683368 learned_addr learned my addr 172.21.15.31:0/2236683368 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:01.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:01.812+0000 7fec0e99c700 1 --2- 172.21.15.31:0/2236683368 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fec0806f3a0 0x7fec08147320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:01.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:01.812+0000 7fec0e19b700 1 --2- 172.21.15.31:0/2236683368 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec0806e0a0 0x7fec0816a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:01.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:01.813+0000 7fec0d99a700 1 -- 172.21.15.31:0/2236683368 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fec0806f3a0 msgr2=0x7fec08147320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:01.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:01.813+0000 7fec0d99a700 1 --2- 172.21.15.31:0/2236683368 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fec0806f3a0 0x7fec08147320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:01.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:01.813+0000 7fec0d99a700 1 -- 172.21.15.31:0/2236683368 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec0806e0a0 msgr2=0x7fec0816a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:01.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:01.813+0000 7fec0d99a700 1 --2- 172.21.15.31:0/2236683368 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec0806e0a0 0x7fec0816a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:01.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:01.813+0000 7fec0d99a700 1 -- 172.21.15.31:0/2236683368 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febf8009480 con 0x7fec0806e470 2023-10-06T02:28:01.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:01.813+0000 7fec0d99a700 1 --2- 172.21.15.31:0/2236683368 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec0806e470 0x7fec08146de0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fec08071320 tx=0x7febf8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:01.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:01.813+0000 7febfe7fc700 1 -- 172.21.15.31:0/2236683368 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febf8011af0 con 0x7fec0806e470 2023-10-06T02:28:01.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:01.813+0000 7febfe7fc700 1 -- 172.21.15.31:0/2236683368 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febf8011c50 con 0x7fec0806e470 2023-10-06T02:28:01.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:01.813+0000 7febfe7fc700 1 -- 172.21.15.31:0/2236683368 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febf801b430 con 0x7fec0806e470 2023-10-06T02:28:01.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:01.813+0000 7fec0f19d700 1 -- 172.21.15.31:0/2236683368 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec081576c0 con 0x7fec0806e470 2023-10-06T02:28:01.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:01.813+0000 7fec0f19d700 1 -- 172.21.15.31:0/2236683368 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec081573e0 con 0x7fec0806e470 2023-10-06T02:28:01.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:01.814+0000 7fec0f19d700 1 -- 172.21.15.31:0/2236683368 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec08145760 con 0x7fec0806e470 2023-10-06T02:28:01.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:01.815+0000 7febfe7fc700 1 -- 172.21.15.31:0/2236683368 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7febf801b880 con 0x7fec0806e470 2023-10-06T02:28:01.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:01.816+0000 7febfe7fc700 1 --2- 172.21.15.31:0/2236683368 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7febec067050 0x7febec069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:01.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:01.816+0000 7febfe7fc700 1 -- 172.21.15.31:0/2236683368 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6058+0+0 (secure 0 0 0) 0x7febf8086720 con 0x7fec0806e470 2023-10-06T02:28:01.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:01.817+0000 7fec0e19b700 1 --2- 172.21.15.31:0/2236683368 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7febec067050 0x7febec069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:01.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:01.817+0000 7fec0e19b700 1 --2- 172.21.15.31:0/2236683368 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7febec067050 0x7febec069500 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fec0400ab30 tx=0x7fec04008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:01.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:01.820+0000 7febfe7fc700 1 -- 172.21.15.31:0/2236683368 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7febf8053680 con 0x7fec0806e470 2023-10-06T02:28:02.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.066+0000 7fec0f19d700 1 -- 172.21.15.31:0/2236683368 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fec0804c3c0 con 0x7fec0806e470 2023-10-06T02:28:02.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.066+0000 7febfe7fc700 1 -- 172.21.15.31:0/2236683368 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+4596 (secure 0 0 0) 0x7febf8084e60 con 0x7fec0806e470 2023-10-06T02:28:02.067 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:28:02.067 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":56,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5759,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/2470516084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2470516084},{"type":"v1","addr":"172.21.15.62:6834","nonce":2470516084}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":6364,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/790751965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":790751965},{"type":"v1","addr":"172.21.15.31:6834","nonce":790751965}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":6373,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2757737386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2757737386},{"type":"v1","addr":"172.21.15.62:6835","nonce":2757737386}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:27:58.019137+0000","modified":"2023-10-06T02:28:01.555185+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6714},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6714":{"gid":6714,"name":"c","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/1002908930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":1002908930},{"type":"v1","addr":"172.21.15.31:6835","nonce":1002908930}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T02:28:02.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.069+0000 7fec0f19d700 1 -- 172.21.15.31:0/2236683368 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7febec067050 msgr2=0x7febec069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:02.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.069+0000 7fec0f19d700 1 --2- 172.21.15.31:0/2236683368 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7febec067050 0x7febec069500 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fec0400ab30 tx=0x7fec04008040 comp rx=0 tx=0).stop 2023-10-06T02:28:02.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.069+0000 7fec0f19d700 1 -- 172.21.15.31:0/2236683368 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec0806e470 msgr2=0x7fec08146de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:02.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.069+0000 7fec0f19d700 1 --2- 172.21.15.31:0/2236683368 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec0806e470 0x7fec08146de0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fec08071320 tx=0x7febf8016000 comp rx=0 tx=0).stop 2023-10-06T02:28:02.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.069+0000 7fec0f19d700 1 -- 172.21.15.31:0/2236683368 shutdown_connections 2023-10-06T02:28:02.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.069+0000 7fec0f19d700 1 --2- 172.21.15.31:0/2236683368 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7febec067050 0x7febec069500 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:02.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.069+0000 7fec0f19d700 1 --2- 172.21.15.31:0/2236683368 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fec0806f3a0 0x7fec08147320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:02.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.069+0000 7fec0f19d700 1 --2- 172.21.15.31:0/2236683368 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec0806e0a0 0x7fec0816a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:02.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.069+0000 7fec0f19d700 1 --2- 172.21.15.31:0/2236683368 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec0806e470 0x7fec08146de0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:02.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.069+0000 7fec0f19d700 1 -- 172.21.15.31:0/2236683368 >> 172.21.15.31:0/2236683368 conn(0x7fec0806d170 msgr2=0x7fec0814e920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:02.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.069+0000 7fec0f19d700 1 -- 172.21.15.31:0/2236683368 shutdown_connections 2023-10-06T02:28:02.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.069+0000 7fec0f19d700 1 -- 172.21.15.31:0/2236683368 wait complete. 2023-10-06T02:28:02.071 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 56 2023-10-06T02:28:02.087 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:28:02.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.273+0000 7f05adf9d700 1 Processor -- start 2023-10-06T02:28:02.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.273+0000 7f05adf9d700 1 -- start start 2023-10-06T02:28:02.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.274+0000 7f05adf9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f05a80cc5b0 0x7f05a80aaed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:02.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.274+0000 7f05adf9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f05a80ca270 0x7f05a806e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:02.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.275+0000 7f05adf9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05a80ad780 con 0x7f05a80accb0 2023-10-06T02:28:02.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.275+0000 7f05adf9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05a80cee60 con 0x7f05a80cc5b0 2023-10-06T02:28:02.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.275+0000 7f05adf9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05a80cf140 con 0x7f05a80ca270 2023-10-06T02:28:02.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.275+0000 7f05a7fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f05a80cc5b0 0x7f05a80aaed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:02.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.275+0000 7f05a7fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f05a80cc5b0 0x7f05a80aaed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40478/0 (socket says 172.21.15.31:40478) 2023-10-06T02:28:02.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.275+0000 7f05a7fff700 1 -- 172.21.15.31:0/4081273615 learned_addr learned my addr 172.21.15.31:0/4081273615 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:02.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.275+0000 7f05a7fff700 1 -- 172.21.15.31:0/4081273615 >> v2:172.21.15.62:3300/0 conn(0x7f05a80ca270 msgr2=0x7f05a806e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:02.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.275+0000 7f05ad79c700 1 --2- 172.21.15.31:0/4081273615 >> v2:172.21.15.62:3300/0 conn(0x7f05a80ca270 0x7f05a806e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:02.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.275+0000 7f05a7fff700 1 --2- 172.21.15.31:0/4081273615 >> v2:172.21.15.62:3300/0 conn(0x7f05a80ca270 0x7f05a806e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:02.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.275+0000 7f05a7fff700 1 -- 172.21.15.31:0/4081273615 >> v1:172.21.15.62:6789/0 conn(0x7f05a80accb0 legacy=0x7f05a80cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:02.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.275+0000 7f05a7fff700 1 -- 172.21.15.31:0/4081273615 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05a806d850 con 0x7f05a80cc5b0 2023-10-06T02:28:02.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.275+0000 7f05ad79c700 1 --2- 172.21.15.31:0/4081273615 >> v2:172.21.15.62:3300/0 conn(0x7f05a80ca270 0x7f05a806e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:28:02.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.276+0000 7f05a7fff700 1 --2- 172.21.15.31:0/4081273615 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05a80cc5b0 0x7f05a80aaed0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f05a806a5b0 tx=0x7f0594009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d90a1026e3300f64 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:02.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.276+0000 7f05a77fe700 1 -- 172.21.15.31:0/4081273615 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f059400b820 con 0x7f05a80cc5b0 2023-10-06T02:28:02.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.276+0000 7f05a77fe700 1 -- 172.21.15.31:0/4081273615 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f059400b980 con 0x7f05a80cc5b0 2023-10-06T02:28:02.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.276+0000 7f05a77fe700 1 -- 172.21.15.31:0/4081273615 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f059400bd20 con 0x7f05a80cc5b0 2023-10-06T02:28:02.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.276+0000 7f05adf9d700 1 -- 172.21.15.31:0/4081273615 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05a80cc5b0 msgr2=0x7f05a80aaed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:02.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.276+0000 7f05adf9d700 1 --2- 172.21.15.31:0/4081273615 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05a80cc5b0 0x7f05a80aaed0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f05a806a5b0 tx=0x7f0594009250 comp rx=0 tx=0).stop 2023-10-06T02:28:02.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.276+0000 7f05adf9d700 1 -- 172.21.15.31:0/4081273615 shutdown_connections 2023-10-06T02:28:02.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.276+0000 7f05adf9d700 1 --2- 172.21.15.31:0/4081273615 >> v2:172.21.15.62:3300/0 conn(0x7f05a80ca270 0x7f05a806e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:02.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.276+0000 7f05adf9d700 1 --2- 172.21.15.31:0/4081273615 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05a80cc5b0 0x7f05a80aaed0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:02.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.276+0000 7f05adf9d700 1 -- 172.21.15.31:0/4081273615 >> 172.21.15.31:0/4081273615 conn(0x7f05a806ce80 msgr2=0x7f05a80ce370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:02.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.277+0000 7f05adf9d700 1 -- 172.21.15.31:0/4081273615 shutdown_connections 2023-10-06T02:28:02.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.277+0000 7f05adf9d700 1 -- 172.21.15.31:0/4081273615 wait complete. 2023-10-06T02:28:02.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.278+0000 7f05adf9d700 1 Processor -- start 2023-10-06T02:28:02.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.278+0000 7f05adf9d700 1 -- start start 2023-10-06T02:28:02.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.279+0000 7f05adf9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05a80accb0 0x7f05a80b1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:02.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.279+0000 7f05adf9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f05a80ca270 0x7f05a81574e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:02.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.279+0000 7f05adf9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05a80cc5b0 0x7f05a8169b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:02.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.279+0000 7f05adf9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a816d240 con 0x7f05a80cc5b0 2023-10-06T02:28:02.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.279+0000 7f05adf9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a80b1700 con 0x7f05a80accb0 2023-10-06T02:28:02.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.279+0000 7f05adf9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05a80b19e0 con 0x7f05a80ca270 2023-10-06T02:28:02.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.279+0000 7f05ad79c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05a80cc5b0 0x7f05a8169b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:02.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.279+0000 7f05ad79c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05a80cc5b0 0x7f05a8169b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40484/0 (socket says 172.21.15.31:40484) 2023-10-06T02:28:02.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.279+0000 7f05ad79c700 1 -- 172.21.15.31:0/2700152443 learned_addr learned my addr 172.21.15.31:0/2700152443 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:02.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.279+0000 7f05acf9b700 1 --2- 172.21.15.31:0/2700152443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05a80accb0 0x7f05a80b1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:02.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.280+0000 7f05a7fff700 1 --2- 172.21.15.31:0/2700152443 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f05a80ca270 0x7f05a81574e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:02.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.280+0000 7f05ad79c700 1 -- 172.21.15.31:0/2700152443 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f05a80ca270 msgr2=0x7f05a81574e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:02.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.280+0000 7f05ad79c700 1 --2- 172.21.15.31:0/2700152443 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f05a80ca270 0x7f05a81574e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:02.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.280+0000 7f05ad79c700 1 -- 172.21.15.31:0/2700152443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05a80accb0 msgr2=0x7f05a80b1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:02.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.280+0000 7f05ad79c700 1 --2- 172.21.15.31:0/2700152443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05a80accb0 0x7f05a80b1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:02.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.280+0000 7f05ad79c700 1 -- 172.21.15.31:0/2700152443 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05940094f0 con 0x7f05a80cc5b0 2023-10-06T02:28:02.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.280+0000 7f05a7fff700 1 --2- 172.21.15.31:0/2700152443 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f05a80ca270 0x7f05a81574e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:28:02.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.280+0000 7f05ad79c700 1 --2- 172.21.15.31:0/2700152443 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05a80cc5b0 0x7f05a8169b90 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f05a000cef0 tx=0x7f05a0007860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:02.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.280+0000 7f05a4ff9700 1 -- 172.21.15.31:0/2700152443 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a000add0 con 0x7f05a80cc5b0 2023-10-06T02:28:02.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.280+0000 7f05a4ff9700 1 -- 172.21.15.31:0/2700152443 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05a0016420 con 0x7f05a80cc5b0 2023-10-06T02:28:02.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.280+0000 7f05a4ff9700 1 -- 172.21.15.31:0/2700152443 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a00167e0 con 0x7f05a80cc5b0 2023-10-06T02:28:02.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.280+0000 7f05adf9d700 1 -- 172.21.15.31:0/2700152443 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05a80b1cc0 con 0x7f05a80cc5b0 2023-10-06T02:28:02.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.280+0000 7f05adf9d700 1 -- 172.21.15.31:0/2700152443 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05a80b3650 con 0x7f05a80cc5b0 2023-10-06T02:28:02.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.281+0000 7f05adf9d700 1 -- 172.21.15.31:0/2700152443 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05a804c3c0 con 0x7f05a80cc5b0 2023-10-06T02:28:02.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.282+0000 7f05a4ff9700 1 -- 172.21.15.31:0/2700152443 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f05a0016bd0 con 0x7f05a80cc5b0 2023-10-06T02:28:02.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.283+0000 7f05a4ff9700 1 --2- 172.21.15.31:0/2700152443 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f058c067050 0x7f058c069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:02.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.283+0000 7f05a4ff9700 1 -- 172.21.15.31:0/2700152443 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f05a0012070 con 0x7f05a80cc5b0 2023-10-06T02:28:02.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.284+0000 7f05acf9b700 1 --2- 172.21.15.31:0/2700152443 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f058c067050 0x7f058c069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:02.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.284+0000 7f05acf9b700 1 --2- 172.21.15.31:0/2700152443 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f058c067050 0x7f058c069500 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f059c009af0 tx=0x7f059c009060 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:02.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.287+0000 7f05a4ff9700 1 -- 172.21.15.31:0/2700152443 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f05a004e5a0 con 0x7f05a80cc5b0 2023-10-06T02:28:02.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.494+0000 7f05adf9d700 1 -- 172.21.15.31:0/2700152443 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f05a806db30 con 0x7f05a80cc5b0 2023-10-06T02:28:02.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.494+0000 7f05a4ff9700 1 -- 172.21.15.31:0/2700152443 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+16821 (secure 0 0 0) 0x7f05a0052100 con 0x7f05a80cc5b0 2023-10-06T02:28:02.495 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:28:02.495 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":87,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:27:59.529296+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:27:56.192975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-10-06T02:27:56.993140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:28:02.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.497+0000 7f05adf9d700 1 -- 172.21.15.31:0/2700152443 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f058c067050 msgr2=0x7f058c069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:02.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.497+0000 7f05adf9d700 1 --2- 172.21.15.31:0/2700152443 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f058c067050 0x7f058c069500 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f059c009af0 tx=0x7f059c009060 comp rx=0 tx=0).stop 2023-10-06T02:28:02.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.497+0000 7f05adf9d700 1 -- 172.21.15.31:0/2700152443 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05a80cc5b0 msgr2=0x7f05a8169b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:02.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.497+0000 7f05adf9d700 1 --2- 172.21.15.31:0/2700152443 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05a80cc5b0 0x7f05a8169b90 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f05a000cef0 tx=0x7f05a0007860 comp rx=0 tx=0).stop 2023-10-06T02:28:02.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.497+0000 7f05adf9d700 1 -- 172.21.15.31:0/2700152443 shutdown_connections 2023-10-06T02:28:02.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.497+0000 7f05adf9d700 1 --2- 172.21.15.31:0/2700152443 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05a80cc5b0 0x7f05a8169b90 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:02.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.497+0000 7f05adf9d700 1 --2- 172.21.15.31:0/2700152443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05a80accb0 0x7f05a80b1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:02.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.497+0000 7f05adf9d700 1 --2- 172.21.15.31:0/2700152443 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f058c067050 0x7f058c069500 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:02.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.497+0000 7f05adf9d700 1 --2- 172.21.15.31:0/2700152443 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f05a80ca270 0x7f05a81574e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:02.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.497+0000 7f05adf9d700 1 -- 172.21.15.31:0/2700152443 >> 172.21.15.31:0/2700152443 conn(0x7f05a806ce80 msgr2=0x7f05a806f260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:02.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.497+0000 7f05adf9d700 1 -- 172.21.15.31:0/2700152443 shutdown_connections 2023-10-06T02:28:02.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.497+0000 7f05adf9d700 1 -- 172.21.15.31:0/2700152443 wait complete. 2023-10-06T02:28:02.515 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T02:28:02.515 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T02:28:02.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.704+0000 7f19f339d700 1 Processor -- start 2023-10-06T02:28:02.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.704+0000 7f19f339d700 1 -- start start 2023-10-06T02:28:02.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.706+0000 7f19f339d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f19ec13fb20 0x7f19ec15a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:02.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.706+0000 7f19f339d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f19ec13fef0 0x7f19ec0c7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:02.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.706+0000 7f19f339d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19ec15ae70 con 0x7f19ec06aaa0 2023-10-06T02:28:02.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.706+0000 7f19f339d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19ec06af60 con 0x7f19ec13fb20 2023-10-06T02:28:02.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.706+0000 7f19f339d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec06b240 con 0x7f19ec13fef0 2023-10-06T02:28:02.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.706+0000 7f19f2b9c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f19ec06aaa0 0x7f19ec0c6770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58092/0 (socket says 172.21.15.31:58092) 2023-10-06T02:28:02.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.706+0000 7f19f2b9c700 1 -- 172.21.15.31:0/3480950666 learned_addr learned my addr 172.21.15.31:0/3480950666 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:28:02.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.706+0000 7f19f1b9a700 1 --2- 172.21.15.31:0/3480950666 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f19ec13fef0 0x7f19ec0c7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:02.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.706+0000 7f19f239b700 1 --2- 172.21.15.31:0/3480950666 >> v2:172.21.15.62:3300/0 conn(0x7f19ec13fb20 0x7f19ec15a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:02.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.706+0000 7f19f1399700 1 -- 172.21.15.31:0/3480950666 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 407951197 0 0) 0x7f19e4001490 con 0x7f19ec06aaa0 2023-10-06T02:28:02.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.706+0000 7f19f1399700 1 -- 172.21.15.31:0/3480950666 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19d8003570 con 0x7f19ec06aaa0 2023-10-06T02:28:02.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.707+0000 7f19f1399700 1 -- 172.21.15.31:0/3480950666 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 807815990 0 0) 0x7f19d8003570 con 0x7f19ec06aaa0 2023-10-06T02:28:02.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.707+0000 7f19f1399700 1 -- 172.21.15.31:0/3480950666 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f19ec13fef0 msgr2=0x7f19ec0c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:02.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.707+0000 7f19f1399700 1 --2- 172.21.15.31:0/3480950666 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f19ec13fef0 0x7f19ec0c7990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:02.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.707+0000 7f19f1399700 1 -- 172.21.15.31:0/3480950666 >> v2:172.21.15.62:3300/0 conn(0x7f19ec13fb20 msgr2=0x7f19ec15a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:02.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.707+0000 7f19f1399700 1 --2- 172.21.15.31:0/3480950666 >> v2:172.21.15.62:3300/0 conn(0x7f19ec13fb20 0x7f19ec15a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:02.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.707+0000 7f19f1399700 1 -- 172.21.15.31:0/3480950666 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19ec06b520 con 0x7f19ec06aaa0 2023-10-06T02:28:02.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.707+0000 7f19f1399700 1 -- 172.21.15.31:0/3480950666 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7f19e4001980 con 0x7f19ec06aaa0 2023-10-06T02:28:02.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.707+0000 7f19f1399700 1 -- 172.21.15.31:0/3480950666 >> v1:172.21.15.31:6789/0 conn(0x7f19ec06aaa0 legacy=0x7f19ec0c6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:02.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.707+0000 7f19f1399700 1 --2- 172.21.15.31:0/3480950666 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19d8003ae0 0x7f19d8003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:02.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.707+0000 7f19f1399700 1 --2- 172.21.15.31:0/3480950666 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19d8004810 0x7f19d8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:02.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.707+0000 7f19f1399700 1 --2- 172.21.15.31:0/3480950666 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f19ec13fef0 0x7f19d80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:02.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.707+0000 7f19f1399700 1 -- 172.21.15.31:0/3480950666 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec06af60 con 0x7f19d8004810 2023-10-06T02:28:02.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.707+0000 7f19f1399700 1 -- 172.21.15.31:0/3480950666 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec06b240 con 0x7f19d8003ae0 2023-10-06T02:28:02.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.707+0000 7f19f1399700 1 -- 172.21.15.31:0/3480950666 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19d800bb40 con 0x7f19ec13fef0 2023-10-06T02:28:02.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.707+0000 7f19f1b9a700 1 --2- 172.21.15.31:0/3480950666 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19d8004810 0x7f19d8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:02.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.707+0000 7f19f239b700 1 --2- 172.21.15.31:0/3480950666 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19d8003ae0 0x7f19d8003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:02.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.707+0000 7f19f2b9c700 1 --2- 172.21.15.31:0/3480950666 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f19ec13fef0 0x7f19d80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:02.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.707+0000 7f19f1b9a700 1 -- 172.21.15.31:0/3480950666 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f19ec13fef0 msgr2=0x7f19d80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:02.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.708+0000 7f19f1b9a700 1 --2- 172.21.15.31:0/3480950666 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f19ec13fef0 0x7f19d80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:02.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.708+0000 7f19f1b9a700 1 -- 172.21.15.31:0/3480950666 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19d8003ae0 msgr2=0x7f19d8003ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:02.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.708+0000 7f19f1b9a700 1 --2- 172.21.15.31:0/3480950666 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19d8003ae0 0x7f19d8003ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:02.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.708+0000 7f19f1b9a700 1 -- 172.21.15.31:0/3480950666 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19d8003570 con 0x7f19d8004810 2023-10-06T02:28:02.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.708+0000 7f19f2b9c700 1 --2- 172.21.15.31:0/3480950666 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f19ec13fef0 0x7f19d80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:28:02.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.708+0000 7f19f1b9a700 1 --2- 172.21.15.31:0/3480950666 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19d8004810 0x7f19d8004c80 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f19ec069580 tx=0x7f19dc009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dae4df25645c225c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:02.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.708+0000 7f19f1399700 1 -- 172.21.15.31:0/3480950666 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19dc005a00 con 0x7f19d8004810 2023-10-06T02:28:02.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.708+0000 7f19f1399700 1 -- 172.21.15.31:0/3480950666 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19dc005b60 con 0x7f19d8004810 2023-10-06T02:28:02.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.708+0000 7f19f1399700 1 -- 172.21.15.31:0/3480950666 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19dc00e640 con 0x7f19d8004810 2023-10-06T02:28:02.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.709+0000 7f19f339d700 1 -- 172.21.15.31:0/3480950666 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19d8004810 msgr2=0x7f19d8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:02.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.709+0000 7f19f339d700 1 --2- 172.21.15.31:0/3480950666 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19d8004810 0x7f19d8004c80 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f19ec069580 tx=0x7f19dc009120 comp rx=0 tx=0).stop 2023-10-06T02:28:02.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.709+0000 7f19f2b9c700 1 -- 172.21.15.31:0/3480950666 reap_dead start 2023-10-06T02:28:02.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.709+0000 7f19f339d700 1 -- 172.21.15.31:0/3480950666 shutdown_connections 2023-10-06T02:28:02.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.709+0000 7f19f339d700 1 -- 172.21.15.31:0/3480950666 >> 172.21.15.31:0/3480950666 conn(0x7f19ec06d170 msgr2=0x7f19ec0cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:02.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.709+0000 7f19f339d700 1 -- 172.21.15.31:0/3480950666 shutdown_connections 2023-10-06T02:28:02.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.709+0000 7f19f339d700 1 -- 172.21.15.31:0/3480950666 wait complete. 2023-10-06T02:28:02.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.710+0000 7f19f339d700 1 Processor -- start 2023-10-06T02:28:02.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.710+0000 7f19f339d700 1 -- start start 2023-10-06T02:28:02.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.711+0000 7f19f339d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19d8004810 0x7f19ec157e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:02.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.711+0000 7f19f339d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19ec156920 0x7f19ec13e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:02.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.711+0000 7f19f339d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f19ec155b10 0x7f19ec0addc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:02.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.711+0000 7f19f339d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec1583c0 con 0x7f19d8004810 2023-10-06T02:28:02.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.711+0000 7f19f339d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec156de0 con 0x7f19ec156920 2023-10-06T02:28:02.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.711+0000 7f19f339d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec1570c0 con 0x7f19ec155b10 2023-10-06T02:28:02.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.711+0000 7f19f239b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19d8004810 0x7f19ec157e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:02.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.711+0000 7f19f239b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19d8004810 0x7f19ec157e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40496/0 (socket says 172.21.15.31:40496) 2023-10-06T02:28:02.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.711+0000 7f19f239b700 1 -- 172.21.15.31:0/1037455622 learned_addr learned my addr 172.21.15.31:0/1037455622 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:02.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.712+0000 7f19f1b9a700 1 --2- 172.21.15.31:0/1037455622 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19ec156920 0x7f19ec13e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:02.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.712+0000 7f19f2b9c700 1 --2- 172.21.15.31:0/1037455622 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f19ec155b10 0x7f19ec0addc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:02.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.712+0000 7f19f239b700 1 -- 172.21.15.31:0/1037455622 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f19ec155b10 msgr2=0x7f19ec0addc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:02.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.712+0000 7f19f239b700 1 --2- 172.21.15.31:0/1037455622 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f19ec155b10 0x7f19ec0addc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:02.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.712+0000 7f19f239b700 1 -- 172.21.15.31:0/1037455622 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19ec156920 msgr2=0x7f19ec13e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:02.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.712+0000 7f19f239b700 1 --2- 172.21.15.31:0/1037455622 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19ec156920 0x7f19ec13e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:02.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.712+0000 7f19f239b700 1 -- 172.21.15.31:0/1037455622 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19ec1573a0 con 0x7f19d8004810 2023-10-06T02:28:02.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.712+0000 7f19f239b700 1 --2- 172.21.15.31:0/1037455622 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19d8004810 0x7f19ec157e80 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f19e8011d40 tx=0x7f19e80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:02.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.712+0000 7f19e27fc700 1 -- 172.21.15.31:0/1037455622 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e800fac0 con 0x7f19d8004810 2023-10-06T02:28:02.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.712+0000 7f19e27fc700 1 -- 172.21.15.31:0/1037455622 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19e800fc20 con 0x7f19d8004810 2023-10-06T02:28:02.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.712+0000 7f19e27fc700 1 -- 172.21.15.31:0/1037455622 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e8014440 con 0x7f19d8004810 2023-10-06T02:28:02.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.712+0000 7f19f339d700 1 -- 172.21.15.31:0/1037455622 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19ec157680 con 0x7f19d8004810 2023-10-06T02:28:02.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.713+0000 7f19f339d700 1 -- 172.21.15.31:0/1037455622 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19ec13e630 con 0x7f19d8004810 2023-10-06T02:28:02.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.713+0000 7f19f339d700 1 -- 172.21.15.31:0/1037455622 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19ec157b20 con 0x7f19d8004810 2023-10-06T02:28:02.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.714+0000 7f19e27fc700 1 -- 172.21.15.31:0/1037455622 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f19e8014870 con 0x7f19d8004810 2023-10-06T02:28:02.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.715+0000 7f19e27fc700 1 --2- 172.21.15.31:0/1037455622 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f19d0067040 0x7f19d00694f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:02.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.715+0000 7f19e27fc700 1 -- 172.21.15.31:0/1037455622 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f19e8018030 con 0x7f19d8004810 2023-10-06T02:28:02.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.716+0000 7f19f1b9a700 1 --2- 172.21.15.31:0/1037455622 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f19d0067040 0x7f19d00694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:02.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.716+0000 7f19f1b9a700 1 --2- 172.21.15.31:0/1037455622 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f19d0067040 0x7f19d00694f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f19dc009160 tx=0x7f19dc019000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:02.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.719+0000 7f19e27fc700 1 -- 172.21.15.31:0/1037455622 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f19e80522c0 con 0x7f19d8004810 2023-10-06T02:28:02.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.916+0000 7f19f339d700 1 -- 172.21.15.31:0/1037455622 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f19ec04c3c0 con 0x7f19d0067040 2023-10-06T02:28:02.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.929+0000 7f19e27fc700 1 -- 172.21.15.31:0/1037455622 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311244 (secure 0 0 0) 0x7f19ec04c3c0 con 0x7f19d0067040 2023-10-06T02:28:02.933 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:28:02.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.933+0000 7f19f339d700 1 -- 172.21.15.31:0/1037455622 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f19d0067040 msgr2=0x7f19d00694f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:02.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.933+0000 7f19f339d700 1 --2- 172.21.15.31:0/1037455622 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f19d0067040 0x7f19d00694f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f19dc009160 tx=0x7f19dc019000 comp rx=0 tx=0).stop 2023-10-06T02:28:02.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.933+0000 7f19f339d700 1 -- 172.21.15.31:0/1037455622 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19d8004810 msgr2=0x7f19ec157e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:02.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.933+0000 7f19f339d700 1 --2- 172.21.15.31:0/1037455622 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19d8004810 0x7f19ec157e80 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f19e8011d40 tx=0x7f19e80091a0 comp rx=0 tx=0).stop 2023-10-06T02:28:02.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.933+0000 7f19f339d700 1 -- 172.21.15.31:0/1037455622 shutdown_connections 2023-10-06T02:28:02.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.933+0000 7f19f339d700 1 --2- 172.21.15.31:0/1037455622 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f19d0067040 0x7f19d00694f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:02.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.933+0000 7f19f339d700 1 --2- 172.21.15.31:0/1037455622 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f19ec155b10 0x7f19ec0addc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:02.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.933+0000 7f19f339d700 1 --2- 172.21.15.31:0/1037455622 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19d8004810 0x7f19ec157e80 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:02.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.933+0000 7f19f339d700 1 --2- 172.21.15.31:0/1037455622 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19ec156920 0x7f19ec13e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:02.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.933+0000 7f19f339d700 1 -- 172.21.15.31:0/1037455622 >> 172.21.15.31:0/1037455622 conn(0x7f19ec06d170 msgr2=0x7f19ec0b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:02.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.933+0000 7f19f339d700 1 -- 172.21.15.31:0/1037455622 shutdown_connections 2023-10-06T02:28:02.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:02.933+0000 7f19f339d700 1 -- 172.21.15.31:0/1037455622 wait complete. 2023-10-06T02:28:02.938 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-10-06T02:28:02.951 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2023-10-06T02:28:02.446299+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":223,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":166,"ondisk_log_size":166,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":85,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2420532,"kb_used_data":9460,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752554188,"statfs":{"total":773094113280,"available":770615488512,"internally_reserved":0,"allocated":9687040,"data_stored":5006424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.931003"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.551410+0000","last_change":"2023-10-06T02:27:58.551410+0000","last_active":"2023-10-06T02:27:58.551410+0000","last_peered":"2023-10-06T02:27:58.551410+0000","last_clean":"2023-10-06T02:27:58.551410+0000","last_became_active":"2023-10-06T02:27:58.551142+0000","last_became_peered":"2023-10-06T02:27:58.551142+0000","last_unstale":"2023-10-06T02:27:58.551410+0000","last_undegraded":"2023-10-06T02:27:58.551410+0000","last_fullsized":"2023-10-06T02:27:58.551410+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:58:12.432522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.541957+0000","last_change":"2023-10-06T02:27:58.541957+0000","last_active":"2023-10-06T02:27:58.541957+0000","last_peered":"2023-10-06T02:27:58.541957+0000","last_clean":"2023-10-06T02:27:58.541957+0000","last_became_active":"2023-10-06T02:27:58.541684+0000","last_became_peered":"2023-10-06T02:27:58.541684+0000","last_unstale":"2023-10-06T02:27:58.541957+0000","last_undegraded":"2023-10-06T02:27:58.541957+0000","last_fullsized":"2023-10-06T02:27:58.541957+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:57:24.480998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.553499+0000","last_change":"2023-10-06T02:27:58.553499+0000","last_active":"2023-10-06T02:27:58.553499+0000","last_peered":"2023-10-06T02:27:58.553499+0000","last_clean":"2023-10-06T02:27:58.553499+0000","last_became_active":"2023-10-06T02:27:58.553175+0000","last_became_peered":"2023-10-06T02:27:58.553175+0000","last_unstale":"2023-10-06T02:27:58.553499+0000","last_undegraded":"2023-10-06T02:27:58.553499+0000","last_fullsized":"2023-10-06T02:27:58.553499+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:11:13.756297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.573879+0000","last_change":"2023-10-06T02:27:58.573879+0000","last_active":"2023-10-06T02:27:58.573879+0000","last_peered":"2023-10-06T02:27:58.573879+0000","last_clean":"2023-10-06T02:27:58.573879+0000","last_became_active":"2023-10-06T02:27:58.570952+0000","last_became_peered":"2023-10-06T02:27:58.570952+0000","last_unstale":"2023-10-06T02:27:58.573879+0000","last_undegraded":"2023-10-06T02:27:58.573879+0000","last_fullsized":"2023-10-06T02:27:58.573879+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:32:14.499778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.550286+0000","last_change":"2023-10-06T02:27:58.550286+0000","last_active":"2023-10-06T02:27:58.550286+0000","last_peered":"2023-10-06T02:27:58.550286+0000","last_clean":"2023-10-06T02:27:58.550286+0000","last_became_active":"2023-10-06T02:27:58.549972+0000","last_became_peered":"2023-10-06T02:27:58.549972+0000","last_unstale":"2023-10-06T02:27:58.550286+0000","last_undegraded":"2023-10-06T02:27:58.550286+0000","last_fullsized":"2023-10-06T02:27:58.550286+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:35:48.609652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.575473+0000","last_change":"2023-10-06T02:27:58.575473+0000","last_active":"2023-10-06T02:27:58.575473+0000","last_peered":"2023-10-06T02:27:58.575473+0000","last_clean":"2023-10-06T02:27:58.575473+0000","last_became_active":"2023-10-06T02:27:58.575215+0000","last_became_peered":"2023-10-06T02:27:58.575215+0000","last_unstale":"2023-10-06T02:27:58.575473+0000","last_undegraded":"2023-10-06T02:27:58.575473+0000","last_fullsized":"2023-10-06T02:27:58.575473+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:14:39.635904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.544480+0000","last_change":"2023-10-06T02:27:58.544480+0000","last_active":"2023-10-06T02:27:58.544480+0000","last_peered":"2023-10-06T02:27:58.544480+0000","last_clean":"2023-10-06T02:27:58.544480+0000","last_became_active":"2023-10-06T02:27:58.544143+0000","last_became_peered":"2023-10-06T02:27:58.544143+0000","last_unstale":"2023-10-06T02:27:58.544480+0000","last_undegraded":"2023-10-06T02:27:58.544480+0000","last_fullsized":"2023-10-06T02:27:58.544480+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:40.877351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.571026+0000","last_change":"2023-10-06T02:27:58.571026+0000","last_active":"2023-10-06T02:27:58.571026+0000","last_peered":"2023-10-06T02:27:58.571026+0000","last_clean":"2023-10-06T02:27:58.571026+0000","last_became_active":"2023-10-06T02:27:58.568207+0000","last_became_peered":"2023-10-06T02:27:58.568207+0000","last_unstale":"2023-10-06T02:27:58.571026+0000","last_undegraded":"2023-10-06T02:27:58.571026+0000","last_fullsized":"2023-10-06T02:27:58.571026+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:26:25.959100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.550347+0000","last_change":"2023-10-06T02:27:58.550347+0000","last_active":"2023-10-06T02:27:58.550347+0000","last_peered":"2023-10-06T02:27:58.550347+0000","last_clean":"2023-10-06T02:27:58.550347+0000","last_became_active":"2023-10-06T02:27:58.549949+0000","last_became_peered":"2023-10-06T02:27:58.549949+0000","last_unstale":"2023-10-06T02:27:58.550347+0000","last_undegraded":"2023-10-06T02:27:58.550347+0000","last_fullsized":"2023-10-06T02:27:58.550347+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:57:24.480998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.567708+0000","last_change":"2023-10-06T02:27:58.567708+0000","last_active":"2023-10-06T02:27:58.567708+0000","last_peered":"2023-10-06T02:27:58.567708+0000","last_clean":"2023-10-06T02:27:58.567708+0000","last_became_active":"2023-10-06T02:27:58.567434+0000","last_became_peered":"2023-10-06T02:27:58.567434+0000","last_unstale":"2023-10-06T02:27:58.567708+0000","last_undegraded":"2023-10-06T02:27:58.567708+0000","last_fullsized":"2023-10-06T02:27:58.567708+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:31:19.451780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.552709+0000","last_change":"2023-10-06T02:27:58.552709+0000","last_active":"2023-10-06T02:27:58.552709+0000","last_peered":"2023-10-06T02:27:58.552709+0000","last_clean":"2023-10-06T02:27:58.552709+0000","last_became_active":"2023-10-06T02:27:58.552343+0000","last_became_peered":"2023-10-06T02:27:58.552343+0000","last_unstale":"2023-10-06T02:27:58.552709+0000","last_undegraded":"2023-10-06T02:27:58.552709+0000","last_fullsized":"2023-10-06T02:27:58.552709+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:21:25.801489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.542603+0000","last_change":"2023-10-06T02:27:58.542603+0000","last_active":"2023-10-06T02:27:58.542603+0000","last_peered":"2023-10-06T02:27:58.542603+0000","last_clean":"2023-10-06T02:27:58.542603+0000","last_became_active":"2023-10-06T02:27:58.542285+0000","last_became_peered":"2023-10-06T02:27:58.542285+0000","last_unstale":"2023-10-06T02:27:58.542603+0000","last_undegraded":"2023-10-06T02:27:58.542603+0000","last_fullsized":"2023-10-06T02:27:58.542603+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:42:16.586315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.573766+0000","last_change":"2023-10-06T02:27:58.573766+0000","last_active":"2023-10-06T02:27:58.573766+0000","last_peered":"2023-10-06T02:27:58.573766+0000","last_clean":"2023-10-06T02:27:58.573766+0000","last_became_active":"2023-10-06T02:27:58.570955+0000","last_became_peered":"2023-10-06T02:27:58.570955+0000","last_unstale":"2023-10-06T02:27:58.573766+0000","last_undegraded":"2023-10-06T02:27:58.573766+0000","last_fullsized":"2023-10-06T02:27:58.573766+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:44:04.286470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.572808+0000","last_change":"2023-10-06T02:27:58.572808+0000","last_active":"2023-10-06T02:27:58.572808+0000","last_peered":"2023-10-06T02:27:58.572808+0000","last_clean":"2023-10-06T02:27:58.572808+0000","last_became_active":"2023-10-06T02:27:58.572489+0000","last_became_peered":"2023-10-06T02:27:58.572489+0000","last_unstale":"2023-10-06T02:27:58.572808+0000","last_undegraded":"2023-10-06T02:27:58.572808+0000","last_fullsized":"2023-10-06T02:27:58.572808+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:21:01.544058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.566916+0000","last_change":"2023-10-06T02:27:58.566916+0000","last_active":"2023-10-06T02:27:58.566916+0000","last_peered":"2023-10-06T02:27:58.566916+0000","last_clean":"2023-10-06T02:27:58.566916+0000","last_became_active":"2023-10-06T02:27:58.566351+0000","last_became_peered":"2023-10-06T02:27:58.566351+0000","last_unstale":"2023-10-06T02:27:58.566916+0000","last_undegraded":"2023-10-06T02:27:58.566916+0000","last_fullsized":"2023-10-06T02:27:58.566916+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:25:04.772572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.558709+0000","last_change":"2023-10-06T02:27:58.558709+0000","last_active":"2023-10-06T02:27:58.558709+0000","last_peered":"2023-10-06T02:27:58.558709+0000","last_clean":"2023-10-06T02:27:58.558709+0000","last_became_active":"2023-10-06T02:27:58.558435+0000","last_became_peered":"2023-10-06T02:27:58.558435+0000","last_unstale":"2023-10-06T02:27:58.558709+0000","last_undegraded":"2023-10-06T02:27:58.558709+0000","last_fullsized":"2023-10-06T02:27:58.558709+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:22:11.087710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.564983+0000","last_change":"2023-10-06T02:27:58.564983+0000","last_active":"2023-10-06T02:27:58.564983+0000","last_peered":"2023-10-06T02:27:58.564983+0000","last_clean":"2023-10-06T02:27:58.564983+0000","last_became_active":"2023-10-06T02:27:58.564545+0000","last_became_peered":"2023-10-06T02:27:58.564545+0000","last_unstale":"2023-10-06T02:27:58.564983+0000","last_undegraded":"2023-10-06T02:27:58.564983+0000","last_fullsized":"2023-10-06T02:27:58.564983+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:15:39.374928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.550551+0000","last_change":"2023-10-06T02:27:58.550551+0000","last_active":"2023-10-06T02:27:58.550551+0000","last_peered":"2023-10-06T02:27:58.550551+0000","last_clean":"2023-10-06T02:27:58.550551+0000","last_became_active":"2023-10-06T02:27:58.550244+0000","last_became_peered":"2023-10-06T02:27:58.550244+0000","last_unstale":"2023-10-06T02:27:58.550551+0000","last_undegraded":"2023-10-06T02:27:58.550551+0000","last_fullsized":"2023-10-06T02:27:58.550551+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:44:30.929189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.567627+0000","last_change":"2023-10-06T02:27:58.567627+0000","last_active":"2023-10-06T02:27:58.567627+0000","last_peered":"2023-10-06T02:27:58.567627+0000","last_clean":"2023-10-06T02:27:58.567627+0000","last_became_active":"2023-10-06T02:27:58.567277+0000","last_became_peered":"2023-10-06T02:27:58.567277+0000","last_unstale":"2023-10-06T02:27:58.567627+0000","last_undegraded":"2023-10-06T02:27:58.567627+0000","last_fullsized":"2023-10-06T02:27:58.567627+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:04:30.074206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.571310+0000","last_change":"2023-10-06T02:27:58.571310+0000","last_active":"2023-10-06T02:27:58.571310+0000","last_peered":"2023-10-06T02:27:58.571310+0000","last_clean":"2023-10-06T02:27:58.571310+0000","last_became_active":"2023-10-06T02:27:58.570409+0000","last_became_peered":"2023-10-06T02:27:58.570409+0000","last_unstale":"2023-10-06T02:27:58.571310+0000","last_undegraded":"2023-10-06T02:27:58.571310+0000","last_fullsized":"2023-10-06T02:27:58.571310+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:12:31.189846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.549040+0000","last_change":"2023-10-06T02:27:58.549040+0000","last_active":"2023-10-06T02:27:58.549040+0000","last_peered":"2023-10-06T02:27:58.549040+0000","last_clean":"2023-10-06T02:27:58.549040+0000","last_became_active":"2023-10-06T02:27:58.548701+0000","last_became_peered":"2023-10-06T02:27:58.548701+0000","last_unstale":"2023-10-06T02:27:58.549040+0000","last_undegraded":"2023-10-06T02:27:58.549040+0000","last_fullsized":"2023-10-06T02:27:58.549040+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:02:49.666684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.572173+0000","last_change":"2023-10-06T02:27:58.572173+0000","last_active":"2023-10-06T02:27:58.572173+0000","last_peered":"2023-10-06T02:27:58.572173+0000","last_clean":"2023-10-06T02:27:58.572173+0000","last_became_active":"2023-10-06T02:27:58.571683+0000","last_became_peered":"2023-10-06T02:27:58.571683+0000","last_unstale":"2023-10-06T02:27:58.572173+0000","last_undegraded":"2023-10-06T02:27:58.572173+0000","last_fullsized":"2023-10-06T02:27:58.572173+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:18.422648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.574656+0000","last_change":"2023-10-06T02:27:58.574656+0000","last_active":"2023-10-06T02:27:58.574656+0000","last_peered":"2023-10-06T02:27:58.574656+0000","last_clean":"2023-10-06T02:27:58.574656+0000","last_became_active":"2023-10-06T02:27:58.574342+0000","last_became_peered":"2023-10-06T02:27:58.574342+0000","last_unstale":"2023-10-06T02:27:58.574656+0000","last_undegraded":"2023-10-06T02:27:58.574656+0000","last_fullsized":"2023-10-06T02:27:58.574656+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:28:52.562249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.552789+0000","last_change":"2023-10-06T02:27:58.552789+0000","last_active":"2023-10-06T02:27:58.552789+0000","last_peered":"2023-10-06T02:27:58.552789+0000","last_clean":"2023-10-06T02:27:58.552789+0000","last_became_active":"2023-10-06T02:27:58.552598+0000","last_became_peered":"2023-10-06T02:27:58.552598+0000","last_unstale":"2023-10-06T02:27:58.552789+0000","last_undegraded":"2023-10-06T02:27:58.552789+0000","last_fullsized":"2023-10-06T02:27:58.552789+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:04:30.074206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.540391+0000","last_change":"2023-10-06T02:27:58.540391+0000","last_active":"2023-10-06T02:27:58.540391+0000","last_peered":"2023-10-06T02:27:58.540391+0000","last_clean":"2023-10-06T02:27:58.540391+0000","last_became_active":"2023-10-06T02:27:58.540065+0000","last_became_peered":"2023-10-06T02:27:58.540065+0000","last_unstale":"2023-10-06T02:27:58.540391+0000","last_undegraded":"2023-10-06T02:27:58.540391+0000","last_fullsized":"2023-10-06T02:27:58.540391+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:30:19.997629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.567831+0000","last_change":"2023-10-06T02:27:58.567831+0000","last_active":"2023-10-06T02:27:58.567831+0000","last_peered":"2023-10-06T02:27:58.567831+0000","last_clean":"2023-10-06T02:27:58.567831+0000","last_became_active":"2023-10-06T02:27:58.567279+0000","last_became_peered":"2023-10-06T02:27:58.567279+0000","last_unstale":"2023-10-06T02:27:58.567831+0000","last_undegraded":"2023-10-06T02:27:58.567831+0000","last_fullsized":"2023-10-06T02:27:58.567831+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:02:41.309445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.538551+0000","last_change":"2023-10-06T02:27:58.538551+0000","last_active":"2023-10-06T02:27:58.538551+0000","last_peered":"2023-10-06T02:27:58.538551+0000","last_clean":"2023-10-06T02:27:58.538551+0000","last_became_active":"2023-10-06T02:27:58.537888+0000","last_became_peered":"2023-10-06T02:27:58.537888+0000","last_unstale":"2023-10-06T02:27:58.538551+0000","last_undegraded":"2023-10-06T02:27:58.538551+0000","last_fullsized":"2023-10-06T02:27:58.538551+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:38:35.027757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.574430+0000","last_change":"2023-10-06T02:27:58.574430+0000","last_active":"2023-10-06T02:27:58.574430+0000","last_peered":"2023-10-06T02:27:58.574430+0000","last_clean":"2023-10-06T02:27:58.574430+0000","last_became_active":"2023-10-06T02:27:58.574121+0000","last_became_peered":"2023-10-06T02:27:58.574121+0000","last_unstale":"2023-10-06T02:27:58.574430+0000","last_undegraded":"2023-10-06T02:27:58.574430+0000","last_fullsized":"2023-10-06T02:27:58.574430+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:46:55.335987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.553727+0000","last_change":"2023-10-06T02:27:58.553727+0000","last_active":"2023-10-06T02:27:58.553727+0000","last_peered":"2023-10-06T02:27:58.553727+0000","last_clean":"2023-10-06T02:27:58.553727+0000","last_became_active":"2023-10-06T02:27:58.553127+0000","last_became_peered":"2023-10-06T02:27:58.553127+0000","last_unstale":"2023-10-06T02:27:58.553727+0000","last_undegraded":"2023-10-06T02:27:58.553727+0000","last_fullsized":"2023-10-06T02:27:58.553727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:34.381441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.550149+0000","last_change":"2023-10-06T02:27:58.550149+0000","last_active":"2023-10-06T02:27:58.550149+0000","last_peered":"2023-10-06T02:27:58.550149+0000","last_clean":"2023-10-06T02:27:58.550149+0000","last_became_active":"2023-10-06T02:27:58.549830+0000","last_became_peered":"2023-10-06T02:27:58.549830+0000","last_unstale":"2023-10-06T02:27:58.550149+0000","last_undegraded":"2023-10-06T02:27:58.550149+0000","last_fullsized":"2023-10-06T02:27:58.550149+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:42:10.237196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.567724+0000","last_change":"2023-10-06T02:27:58.567724+0000","last_active":"2023-10-06T02:27:58.567724+0000","last_peered":"2023-10-06T02:27:58.567724+0000","last_clean":"2023-10-06T02:27:58.567724+0000","last_became_active":"2023-10-06T02:27:58.567318+0000","last_became_peered":"2023-10-06T02:27:58.567318+0000","last_unstale":"2023-10-06T02:27:58.567724+0000","last_undegraded":"2023-10-06T02:27:58.567724+0000","last_fullsized":"2023-10-06T02:27:58.567724+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:07:32.016889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.558107+0000","last_change":"2023-10-06T02:27:58.558107+0000","last_active":"2023-10-06T02:27:58.558107+0000","last_peered":"2023-10-06T02:27:58.558107+0000","last_clean":"2023-10-06T02:27:58.558107+0000","last_became_active":"2023-10-06T02:27:58.557841+0000","last_became_peered":"2023-10-06T02:27:58.557841+0000","last_unstale":"2023-10-06T02:27:58.558107+0000","last_undegraded":"2023-10-06T02:27:58.558107+0000","last_fullsized":"2023-10-06T02:27:58.558107+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:06:50.626617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.553912+0000","last_change":"2023-10-06T02:27:58.553912+0000","last_active":"2023-10-06T02:27:58.553912+0000","last_peered":"2023-10-06T02:27:58.553912+0000","last_clean":"2023-10-06T02:27:58.553912+0000","last_became_active":"2023-10-06T02:27:58.553648+0000","last_became_peered":"2023-10-06T02:27:58.553648+0000","last_unstale":"2023-10-06T02:27:58.553912+0000","last_undegraded":"2023-10-06T02:27:58.553912+0000","last_fullsized":"2023-10-06T02:27:58.553912+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:53:11.885304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"86'4","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.570566+0000","last_change":"2023-10-06T02:27:57.547688+0000","last_active":"2023-10-06T02:27:58.570566+0000","last_peered":"2023-10-06T02:27:58.570566+0000","last_clean":"2023-10-06T02:27:58.570566+0000","last_became_active":"2023-10-06T02:27:57.547379+0000","last_became_peered":"2023-10-06T02:27:57.547379+0000","last_unstale":"2023-10-06T02:27:58.570566+0000","last_undegraded":"2023-10-06T02:27:58.570566+0000","last_fullsized":"2023-10-06T02:27:58.570566+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_clean_scrub_stamp":"2023-10-06T02:27:56.508109+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:05:11.061891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.554871+0000","last_change":"2023-10-06T02:27:58.554871+0000","last_active":"2023-10-06T02:27:58.554871+0000","last_peered":"2023-10-06T02:27:58.554871+0000","last_clean":"2023-10-06T02:27:58.554871+0000","last_became_active":"2023-10-06T02:27:58.554612+0000","last_became_peered":"2023-10-06T02:27:58.554612+0000","last_unstale":"2023-10-06T02:27:58.554871+0000","last_undegraded":"2023-10-06T02:27:58.554871+0000","last_fullsized":"2023-10-06T02:27:58.554871+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:03:01.765756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"86'1","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.543955+0000","last_change":"2023-10-06T02:27:57.543841+0000","last_active":"2023-10-06T02:27:58.543955+0000","last_peered":"2023-10-06T02:27:58.543955+0000","last_clean":"2023-10-06T02:27:58.543955+0000","last_became_active":"2023-10-06T02:27:57.543501+0000","last_became_peered":"2023-10-06T02:27:57.543501+0000","last_unstale":"2023-10-06T02:27:58.543955+0000","last_undegraded":"2023-10-06T02:27:58.543955+0000","last_fullsized":"2023-10-06T02:27:58.543955+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_clean_scrub_stamp":"2023-10-06T02:27:56.508109+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:41:38.011422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.574352+0000","last_change":"2023-10-06T02:27:58.574352+0000","last_active":"2023-10-06T02:27:58.574352+0000","last_peered":"2023-10-06T02:27:58.574352+0000","last_clean":"2023-10-06T02:27:58.574352+0000","last_became_active":"2023-10-06T02:27:58.573988+0000","last_became_peered":"2023-10-06T02:27:58.573988+0000","last_unstale":"2023-10-06T02:27:58.574352+0000","last_undegraded":"2023-10-06T02:27:58.574352+0000","last_fullsized":"2023-10-06T02:27:58.574352+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:38.699087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"86'2","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.564823+0000","last_change":"2023-10-06T02:27:57.538863+0000","last_active":"2023-10-06T02:27:58.564823+0000","last_peered":"2023-10-06T02:27:58.564823+0000","last_clean":"2023-10-06T02:27:58.564823+0000","last_became_active":"2023-10-06T02:27:57.538560+0000","last_became_peered":"2023-10-06T02:27:57.538560+0000","last_unstale":"2023-10-06T02:27:58.564823+0000","last_undegraded":"2023-10-06T02:27:58.564823+0000","last_fullsized":"2023-10-06T02:27:58.564823+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_clean_scrub_stamp":"2023-10-06T02:27:56.508109+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:26:02.862746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.554262+0000","last_change":"2023-10-06T02:27:58.554262+0000","last_active":"2023-10-06T02:27:58.554262+0000","last_peered":"2023-10-06T02:27:58.554262+0000","last_clean":"2023-10-06T02:27:58.554262+0000","last_became_active":"2023-10-06T02:27:58.554001+0000","last_became_peered":"2023-10-06T02:27:58.554001+0000","last_unstale":"2023-10-06T02:27:58.554262+0000","last_undegraded":"2023-10-06T02:27:58.554262+0000","last_fullsized":"2023-10-06T02:27:58.554262+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:14:41.722039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"86'2","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.574647+0000","last_change":"2023-10-06T02:27:57.539775+0000","last_active":"2023-10-06T02:27:58.574647+0000","last_peered":"2023-10-06T02:27:58.574647+0000","last_clean":"2023-10-06T02:27:58.574647+0000","last_became_active":"2023-10-06T02:27:57.539510+0000","last_became_peered":"2023-10-06T02:27:57.539510+0000","last_unstale":"2023-10-06T02:27:58.574647+0000","last_undegraded":"2023-10-06T02:27:58.574647+0000","last_fullsized":"2023-10-06T02:27:58.574647+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_clean_scrub_stamp":"2023-10-06T02:27:56.508109+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:54:49.207902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.574548+0000","last_change":"2023-10-06T02:27:58.574548+0000","last_active":"2023-10-06T02:27:58.574548+0000","last_peered":"2023-10-06T02:27:58.574548+0000","last_clean":"2023-10-06T02:27:58.574548+0000","last_became_active":"2023-10-06T02:27:58.571079+0000","last_became_peered":"2023-10-06T02:27:58.571079+0000","last_unstale":"2023-10-06T02:27:58.574548+0000","last_undegraded":"2023-10-06T02:27:58.574548+0000","last_fullsized":"2023-10-06T02:27:58.574548+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:05:14.081035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"86'1","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.543703+0000","last_change":"2023-10-06T02:27:57.543564+0000","last_active":"2023-10-06T02:27:58.543703+0000","last_peered":"2023-10-06T02:27:58.543703+0000","last_clean":"2023-10-06T02:27:58.543703+0000","last_became_active":"2023-10-06T02:27:57.543262+0000","last_became_peered":"2023-10-06T02:27:57.543262+0000","last_unstale":"2023-10-06T02:27:58.543703+0000","last_undegraded":"2023-10-06T02:27:58.543703+0000","last_fullsized":"2023-10-06T02:27:58.543703+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_clean_scrub_stamp":"2023-10-06T02:27:56.508109+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:54:49.207902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.542373+0000","last_change":"2023-10-06T02:27:58.542373+0000","last_active":"2023-10-06T02:27:58.542373+0000","last_peered":"2023-10-06T02:27:58.542373+0000","last_clean":"2023-10-06T02:27:58.542373+0000","last_became_active":"2023-10-06T02:27:58.542089+0000","last_became_peered":"2023-10-06T02:27:58.542089+0000","last_unstale":"2023-10-06T02:27:58.542373+0000","last_undegraded":"2023-10-06T02:27:58.542373+0000","last_fullsized":"2023-10-06T02:27:58.542373+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:44:04.286470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"86'1","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.554560+0000","last_change":"2023-10-06T02:27:57.533458+0000","last_active":"2023-10-06T02:27:58.554560+0000","last_peered":"2023-10-06T02:27:58.554560+0000","last_clean":"2023-10-06T02:27:58.554560+0000","last_became_active":"2023-10-06T02:27:57.533167+0000","last_became_peered":"2023-10-06T02:27:57.533167+0000","last_unstale":"2023-10-06T02:27:58.554560+0000","last_undegraded":"2023-10-06T02:27:58.554560+0000","last_fullsized":"2023-10-06T02:27:58.554560+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_clean_scrub_stamp":"2023-10-06T02:27:56.508109+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:37:41.434589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.574834+0000","last_change":"2023-10-06T02:27:58.574834+0000","last_active":"2023-10-06T02:27:58.574834+0000","last_peered":"2023-10-06T02:27:58.574834+0000","last_clean":"2023-10-06T02:27:58.574834+0000","last_became_active":"2023-10-06T02:27:58.574555+0000","last_became_peered":"2023-10-06T02:27:58.574555+0000","last_unstale":"2023-10-06T02:27:58.574834+0000","last_undegraded":"2023-10-06T02:27:58.574834+0000","last_fullsized":"2023-10-06T02:27:58.574834+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:16:33.281287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T02:27:57.542533+0000","last_change":"2023-10-06T02:27:57.535979+0000","last_active":"2023-10-06T02:27:57.542533+0000","last_peered":"2023-10-06T02:27:57.542533+0000","last_clean":"2023-10-06T02:27:57.542533+0000","last_became_active":"2023-10-06T02:27:57.535715+0000","last_became_peered":"2023-10-06T02:27:57.535715+0000","last_unstale":"2023-10-06T02:27:57.542533+0000","last_undegraded":"2023-10-06T02:27:57.542533+0000","last_fullsized":"2023-10-06T02:27:57.542533+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_clean_scrub_stamp":"2023-10-06T02:27:56.508109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:02:48.660542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.570983+0000","last_change":"2023-10-06T02:27:58.570983+0000","last_active":"2023-10-06T02:27:58.570983+0000","last_peered":"2023-10-06T02:27:58.570983+0000","last_clean":"2023-10-06T02:27:58.570983+0000","last_became_active":"2023-10-06T02:27:58.570242+0000","last_became_peered":"2023-10-06T02:27:58.570242+0000","last_unstale":"2023-10-06T02:27:58.570983+0000","last_undegraded":"2023-10-06T02:27:58.570983+0000","last_fullsized":"2023-10-06T02:27:58.570983+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:28:52.562249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T02:27:57.549480+0000","last_change":"2023-10-06T02:27:57.546199+0000","last_active":"2023-10-06T02:27:57.549480+0000","last_peered":"2023-10-06T02:27:57.549480+0000","last_clean":"2023-10-06T02:27:57.549480+0000","last_became_active":"2023-10-06T02:27:57.545896+0000","last_became_peered":"2023-10-06T02:27:57.545896+0000","last_unstale":"2023-10-06T02:27:57.549480+0000","last_undegraded":"2023-10-06T02:27:57.549480+0000","last_fullsized":"2023-10-06T02:27:57.549480+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_clean_scrub_stamp":"2023-10-06T02:27:56.508109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:21:00.537917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.551071+0000","last_change":"2023-10-06T02:27:58.551071+0000","last_active":"2023-10-06T02:27:58.551071+0000","last_peered":"2023-10-06T02:27:58.551071+0000","last_clean":"2023-10-06T02:27:58.551071+0000","last_became_active":"2023-10-06T02:27:58.550648+0000","last_became_peered":"2023-10-06T02:27:58.550648+0000","last_unstale":"2023-10-06T02:27:58.551071+0000","last_undegraded":"2023-10-06T02:27:58.551071+0000","last_fullsized":"2023-10-06T02:27:58.551071+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:56:56.716677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T02:27:57.535984+0000","last_change":"2023-10-06T02:27:57.531200+0000","last_active":"2023-10-06T02:27:57.535984+0000","last_peered":"2023-10-06T02:27:57.535984+0000","last_clean":"2023-10-06T02:27:57.535984+0000","last_became_active":"2023-10-06T02:27:57.530938+0000","last_became_peered":"2023-10-06T02:27:57.530938+0000","last_unstale":"2023-10-06T02:27:57.535984+0000","last_undegraded":"2023-10-06T02:27:57.535984+0000","last_fullsized":"2023-10-06T02:27:57.535984+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_clean_scrub_stamp":"2023-10-06T02:27:56.508109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:18:09.904994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.552268+0000","last_change":"2023-10-06T02:27:58.552268+0000","last_active":"2023-10-06T02:27:58.552268+0000","last_peered":"2023-10-06T02:27:58.552268+0000","last_clean":"2023-10-06T02:27:58.552268+0000","last_became_active":"2023-10-06T02:27:58.551984+0000","last_became_peered":"2023-10-06T02:27:58.551984+0000","last_unstale":"2023-10-06T02:27:58.552268+0000","last_undegraded":"2023-10-06T02:27:58.552268+0000","last_fullsized":"2023-10-06T02:27:58.552268+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:11:41.153641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T02:27:57.546221+0000","last_change":"2023-10-06T02:27:57.545173+0000","last_active":"2023-10-06T02:27:57.546221+0000","last_peered":"2023-10-06T02:27:57.546221+0000","last_clean":"2023-10-06T02:27:57.546221+0000","last_became_active":"2023-10-06T02:27:57.544892+0000","last_became_peered":"2023-10-06T02:27:57.544892+0000","last_unstale":"2023-10-06T02:27:57.546221+0000","last_undegraded":"2023-10-06T02:27:57.546221+0000","last_fullsized":"2023-10-06T02:27:57.546221+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_clean_scrub_stamp":"2023-10-06T02:27:56.508109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:07:50.800408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.554352+0000","last_change":"2023-10-06T02:27:58.554352+0000","last_active":"2023-10-06T02:27:58.554352+0000","last_peered":"2023-10-06T02:27:58.554352+0000","last_clean":"2023-10-06T02:27:58.554352+0000","last_became_active":"2023-10-06T02:27:58.554100+0000","last_became_peered":"2023-10-06T02:27:58.554100+0000","last_unstale":"2023-10-06T02:27:58.554352+0000","last_undegraded":"2023-10-06T02:27:58.554352+0000","last_fullsized":"2023-10-06T02:27:58.554352+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:54:43.471071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"86'3","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.566090+0000","last_change":"2023-10-06T02:27:57.546292+0000","last_active":"2023-10-06T02:27:58.566090+0000","last_peered":"2023-10-06T02:27:58.566090+0000","last_clean":"2023-10-06T02:27:58.566090+0000","last_became_active":"2023-10-06T02:27:57.546002+0000","last_became_peered":"2023-10-06T02:27:57.546002+0000","last_unstale":"2023-10-06T02:27:58.566090+0000","last_undegraded":"2023-10-06T02:27:58.566090+0000","last_fullsized":"2023-10-06T02:27:58.566090+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_clean_scrub_stamp":"2023-10-06T02:27:56.508109+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:14:32.484064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.567826+0000","last_change":"2023-10-06T02:27:58.567826+0000","last_active":"2023-10-06T02:27:58.567826+0000","last_peered":"2023-10-06T02:27:58.567826+0000","last_clean":"2023-10-06T02:27:58.567826+0000","last_became_active":"2023-10-06T02:27:58.567543+0000","last_became_peered":"2023-10-06T02:27:58.567543+0000","last_unstale":"2023-10-06T02:27:58.567826+0000","last_undegraded":"2023-10-06T02:27:58.567826+0000","last_fullsized":"2023-10-06T02:27:58.567826+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:14:33.490205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"86'6","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.570308+0000","last_change":"2023-10-06T02:27:57.547624+0000","last_active":"2023-10-06T02:27:58.570308+0000","last_peered":"2023-10-06T02:27:58.570308+0000","last_clean":"2023-10-06T02:27:58.570308+0000","last_became_active":"2023-10-06T02:27:57.547270+0000","last_became_peered":"2023-10-06T02:27:57.547270+0000","last_unstale":"2023-10-06T02:27:58.570308+0000","last_undegraded":"2023-10-06T02:27:58.570308+0000","last_fullsized":"2023-10-06T02:27:58.570308+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_clean_scrub_stamp":"2023-10-06T02:27:56.508109+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:35:47.603511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.566548+0000","last_change":"2023-10-06T02:27:58.566548+0000","last_active":"2023-10-06T02:27:58.566548+0000","last_peered":"2023-10-06T02:27:58.566548+0000","last_clean":"2023-10-06T02:27:58.566548+0000","last_became_active":"2023-10-06T02:27:58.563290+0000","last_became_peered":"2023-10-06T02:27:58.563290+0000","last_unstale":"2023-10-06T02:27:58.566548+0000","last_undegraded":"2023-10-06T02:27:58.566548+0000","last_fullsized":"2023-10-06T02:27:58.566548+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:28:48.424621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"86'2","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.570550+0000","last_change":"2023-10-06T02:27:57.546517+0000","last_active":"2023-10-06T02:27:58.570550+0000","last_peered":"2023-10-06T02:27:58.570550+0000","last_clean":"2023-10-06T02:27:58.570550+0000","last_became_active":"2023-10-06T02:27:57.545890+0000","last_became_peered":"2023-10-06T02:27:57.545890+0000","last_unstale":"2023-10-06T02:27:58.570550+0000","last_undegraded":"2023-10-06T02:27:58.570550+0000","last_fullsized":"2023-10-06T02:27:58.570550+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_clean_scrub_stamp":"2023-10-06T02:27:56.508109+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:17.416507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.550930+0000","last_change":"2023-10-06T02:27:58.550930+0000","last_active":"2023-10-06T02:27:58.550930+0000","last_peered":"2023-10-06T02:27:58.550930+0000","last_clean":"2023-10-06T02:27:58.550930+0000","last_became_active":"2023-10-06T02:27:58.550573+0000","last_became_peered":"2023-10-06T02:27:58.550573+0000","last_unstale":"2023-10-06T02:27:58.550930+0000","last_undegraded":"2023-10-06T02:27:58.550930+0000","last_fullsized":"2023-10-06T02:27:58.550930+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:26:03.868887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T02:27:57.546037+0000","last_change":"2023-10-06T02:27:57.544404+0000","last_active":"2023-10-06T02:27:57.546037+0000","last_peered":"2023-10-06T02:27:57.546037+0000","last_clean":"2023-10-06T02:27:57.546037+0000","last_became_active":"2023-10-06T02:27:57.543953+0000","last_became_peered":"2023-10-06T02:27:57.543953+0000","last_unstale":"2023-10-06T02:27:57.546037+0000","last_undegraded":"2023-10-06T02:27:57.546037+0000","last_fullsized":"2023-10-06T02:27:57.546037+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_clean_scrub_stamp":"2023-10-06T02:27:56.508109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:21:24.795347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.574633+0000","last_change":"2023-10-06T02:27:58.574633+0000","last_active":"2023-10-06T02:27:58.574633+0000","last_peered":"2023-10-06T02:27:58.574633+0000","last_clean":"2023-10-06T02:27:58.574633+0000","last_became_active":"2023-10-06T02:27:58.574359+0000","last_became_peered":"2023-10-06T02:27:58.574359+0000","last_unstale":"2023-10-06T02:27:58.574633+0000","last_undegraded":"2023-10-06T02:27:58.574633+0000","last_fullsized":"2023-10-06T02:27:58.574633+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:48:26.891158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T02:27:57.544556+0000","last_change":"2023-10-06T02:27:57.540372+0000","last_active":"2023-10-06T02:27:57.544556+0000","last_peered":"2023-10-06T02:27:57.544556+0000","last_clean":"2023-10-06T02:27:57.544556+0000","last_became_active":"2023-10-06T02:27:57.539896+0000","last_became_peered":"2023-10-06T02:27:57.539896+0000","last_unstale":"2023-10-06T02:27:57.544556+0000","last_undegraded":"2023-10-06T02:27:57.544556+0000","last_fullsized":"2023-10-06T02:27:57.544556+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_clean_scrub_stamp":"2023-10-06T02:27:56.508109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:53:38.077483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.551889+0000","last_change":"2023-10-06T02:27:58.551889+0000","last_active":"2023-10-06T02:27:58.551889+0000","last_peered":"2023-10-06T02:27:58.551889+0000","last_clean":"2023-10-06T02:27:58.551889+0000","last_became_active":"2023-10-06T02:27:58.551581+0000","last_became_peered":"2023-10-06T02:27:58.551581+0000","last_unstale":"2023-10-06T02:27:58.551889+0000","last_undegraded":"2023-10-06T02:27:58.551889+0000","last_fullsized":"2023-10-06T02:27:58.551889+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:43:13.376099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"86'8","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.578037+0000","last_change":"2023-10-06T02:27:57.533899+0000","last_active":"2023-10-06T02:27:58.578037+0000","last_peered":"2023-10-06T02:27:58.578037+0000","last_clean":"2023-10-06T02:27:58.578037+0000","last_became_active":"2023-10-06T02:27:57.533566+0000","last_became_peered":"2023-10-06T02:27:57.533566+0000","last_unstale":"2023-10-06T02:27:58.578037+0000","last_undegraded":"2023-10-06T02:27:58.578037+0000","last_fullsized":"2023-10-06T02:27:58.578037+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_clean_scrub_stamp":"2023-10-06T02:27:56.508109+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:33:21.693794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.543558+0000","last_change":"2023-10-06T02:27:58.543558+0000","last_active":"2023-10-06T02:27:58.543558+0000","last_peered":"2023-10-06T02:27:58.543558+0000","last_clean":"2023-10-06T02:27:58.543558+0000","last_became_active":"2023-10-06T02:27:58.543268+0000","last_became_peered":"2023-10-06T02:27:58.543268+0000","last_unstale":"2023-10-06T02:27:58.543558+0000","last_undegraded":"2023-10-06T02:27:58.543558+0000","last_fullsized":"2023-10-06T02:27:58.543558+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:28:48.424621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"86'2","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.565708+0000","last_change":"2023-10-06T02:27:57.543149+0000","last_active":"2023-10-06T02:27:58.565708+0000","last_peered":"2023-10-06T02:27:58.565708+0000","last_clean":"2023-10-06T02:27:58.565708+0000","last_became_active":"2023-10-06T02:27:57.542741+0000","last_became_peered":"2023-10-06T02:27:57.542741+0000","last_unstale":"2023-10-06T02:27:58.565708+0000","last_undegraded":"2023-10-06T02:27:58.565708+0000","last_fullsized":"2023-10-06T02:27:58.565708+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_clean_scrub_stamp":"2023-10-06T02:27:56.508109+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:35:47.603511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.557007+0000","last_change":"2023-10-06T02:27:58.557007+0000","last_active":"2023-10-06T02:27:58.557007+0000","last_peered":"2023-10-06T02:27:58.557007+0000","last_clean":"2023-10-06T02:27:58.557007+0000","last_became_active":"2023-10-06T02:27:58.556451+0000","last_became_peered":"2023-10-06T02:27:58.556451+0000","last_unstale":"2023-10-06T02:27:58.557007+0000","last_undegraded":"2023-10-06T02:27:58.557007+0000","last_fullsized":"2023-10-06T02:27:58.557007+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:05:59.590954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T02:27:57.550749+0000","last_change":"2023-10-06T02:27:57.544358+0000","last_active":"2023-10-06T02:27:57.550749+0000","last_peered":"2023-10-06T02:27:57.550749+0000","last_clean":"2023-10-06T02:27:57.550749+0000","last_became_active":"2023-10-06T02:27:57.543270+0000","last_became_peered":"2023-10-06T02:27:57.543270+0000","last_unstale":"2023-10-06T02:27:57.550749+0000","last_undegraded":"2023-10-06T02:27:57.550749+0000","last_fullsized":"2023-10-06T02:27:57.550749+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_clean_scrub_stamp":"2023-10-06T02:27:56.508109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:09:46.127873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.551391+0000","last_change":"2023-10-06T02:27:58.551391+0000","last_active":"2023-10-06T02:27:58.551391+0000","last_peered":"2023-10-06T02:27:58.551391+0000","last_clean":"2023-10-06T02:27:58.551391+0000","last_became_active":"2023-10-06T02:27:58.551003+0000","last_became_peered":"2023-10-06T02:27:58.551003+0000","last_unstale":"2023-10-06T02:27:58.551391+0000","last_undegraded":"2023-10-06T02:27:58.551391+0000","last_fullsized":"2023-10-06T02:27:58.551391+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:34.381441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T02:27:57.550797+0000","last_change":"2023-10-06T02:27:57.546259+0000","last_active":"2023-10-06T02:27:57.550797+0000","last_peered":"2023-10-06T02:27:57.550797+0000","last_clean":"2023-10-06T02:27:57.550797+0000","last_became_active":"2023-10-06T02:27:57.545938+0000","last_became_peered":"2023-10-06T02:27:57.545938+0000","last_unstale":"2023-10-06T02:27:57.550797+0000","last_undegraded":"2023-10-06T02:27:57.550797+0000","last_fullsized":"2023-10-06T02:27:57.550797+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_clean_scrub_stamp":"2023-10-06T02:27:56.508109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:18:08.720368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.553846+0000","last_change":"2023-10-06T02:27:58.553846+0000","last_active":"2023-10-06T02:27:58.553846+0000","last_peered":"2023-10-06T02:27:58.553846+0000","last_clean":"2023-10-06T02:27:58.553846+0000","last_became_active":"2023-10-06T02:27:58.553587+0000","last_became_peered":"2023-10-06T02:27:58.553587+0000","last_unstale":"2023-10-06T02:27:58.553846+0000","last_undegraded":"2023-10-06T02:27:58.553846+0000","last_fullsized":"2023-10-06T02:27:58.553846+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:42:10.237196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T02:27:57.543414+0000","last_change":"2023-10-06T02:27:57.538063+0000","last_active":"2023-10-06T02:27:57.543414+0000","last_peered":"2023-10-06T02:27:57.543414+0000","last_clean":"2023-10-06T02:27:57.543414+0000","last_became_active":"2023-10-06T02:27:57.537811+0000","last_became_peered":"2023-10-06T02:27:57.537811+0000","last_unstale":"2023-10-06T02:27:57.543414+0000","last_undegraded":"2023-10-06T02:27:57.543414+0000","last_fullsized":"2023-10-06T02:27:57.543414+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_clean_scrub_stamp":"2023-10-06T02:27:56.508109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:55.451098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.544309+0000","last_change":"2023-10-06T02:27:58.544309+0000","last_active":"2023-10-06T02:27:58.544309+0000","last_peered":"2023-10-06T02:27:58.544309+0000","last_clean":"2023-10-06T02:27:58.544309+0000","last_became_active":"2023-10-06T02:27:58.544003+0000","last_became_peered":"2023-10-06T02:27:58.544003+0000","last_unstale":"2023-10-06T02:27:58.544309+0000","last_undegraded":"2023-10-06T02:27:58.544309+0000","last_fullsized":"2023-10-06T02:27:58.544309+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:12:44.455787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T02:27:57.548560+0000","last_change":"2023-10-06T02:27:57.545258+0000","last_active":"2023-10-06T02:27:57.548560+0000","last_peered":"2023-10-06T02:27:57.548560+0000","last_clean":"2023-10-06T02:27:57.548560+0000","last_became_active":"2023-10-06T02:27:57.545000+0000","last_became_peered":"2023-10-06T02:27:57.545000+0000","last_unstale":"2023-10-06T02:27:57.548560+0000","last_undegraded":"2023-10-06T02:27:57.548560+0000","last_fullsized":"2023-10-06T02:27:57.548560+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_clean_scrub_stamp":"2023-10-06T02:27:56.508109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:17.416507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.571739+0000","last_change":"2023-10-06T02:27:58.571739+0000","last_active":"2023-10-06T02:27:58.571739+0000","last_peered":"2023-10-06T02:27:58.571739+0000","last_clean":"2023-10-06T02:27:58.571739+0000","last_became_active":"2023-10-06T02:27:58.571424+0000","last_became_peered":"2023-10-06T02:27:58.571424+0000","last_unstale":"2023-10-06T02:27:58.571739+0000","last_undegraded":"2023-10-06T02:27:58.571739+0000","last_fullsized":"2023-10-06T02:27:58.571739+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:57:49.732231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T02:27:57.541872+0000","last_change":"2023-10-06T02:27:57.539201+0000","last_active":"2023-10-06T02:27:57.541872+0000","last_peered":"2023-10-06T02:27:57.541872+0000","last_clean":"2023-10-06T02:27:57.541872+0000","last_became_active":"2023-10-06T02:27:57.538906+0000","last_became_peered":"2023-10-06T02:27:57.538906+0000","last_unstale":"2023-10-06T02:27:57.541872+0000","last_undegraded":"2023-10-06T02:27:57.541872+0000","last_fullsized":"2023-10-06T02:27:57.541872+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_clean_scrub_stamp":"2023-10-06T02:27:56.508109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:28:47.418480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.574402+0000","last_change":"2023-10-06T02:27:58.574402+0000","last_active":"2023-10-06T02:27:58.574402+0000","last_peered":"2023-10-06T02:27:58.574402+0000","last_clean":"2023-10-06T02:27:58.574402+0000","last_became_active":"2023-10-06T02:27:58.574104+0000","last_became_peered":"2023-10-06T02:27:58.574104+0000","last_unstale":"2023-10-06T02:27:58.574402+0000","last_undegraded":"2023-10-06T02:27:58.574402+0000","last_fullsized":"2023-10-06T02:27:58.574402+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:26:25.959100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T02:27:57.536877+0000","last_change":"2023-10-06T02:27:57.533126+0000","last_active":"2023-10-06T02:27:57.536877+0000","last_peered":"2023-10-06T02:27:57.536877+0000","last_clean":"2023-10-06T02:27:57.536877+0000","last_became_active":"2023-10-06T02:27:57.532738+0000","last_became_peered":"2023-10-06T02:27:57.532738+0000","last_unstale":"2023-10-06T02:27:57.536877+0000","last_undegraded":"2023-10-06T02:27:57.536877+0000","last_fullsized":"2023-10-06T02:27:57.536877+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_clean_scrub_stamp":"2023-10-06T02:27:56.508109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:46:08.570447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.560541+0000","last_change":"2023-10-06T02:27:58.560541+0000","last_active":"2023-10-06T02:27:58.560541+0000","last_peered":"2023-10-06T02:27:58.560541+0000","last_clean":"2023-10-06T02:27:58.560541+0000","last_became_active":"2023-10-06T02:27:58.559183+0000","last_became_peered":"2023-10-06T02:27:58.559183+0000","last_unstale":"2023-10-06T02:27:58.560541+0000","last_undegraded":"2023-10-06T02:27:58.560541+0000","last_fullsized":"2023-10-06T02:27:58.560541+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:02:31.507436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T02:27:57.541451+0000","last_change":"2023-10-06T02:27:57.540489+0000","last_active":"2023-10-06T02:27:57.541451+0000","last_peered":"2023-10-06T02:27:57.541451+0000","last_clean":"2023-10-06T02:27:57.541451+0000","last_became_active":"2023-10-06T02:27:57.540200+0000","last_became_peered":"2023-10-06T02:27:57.540200+0000","last_unstale":"2023-10-06T02:27:57.541451+0000","last_undegraded":"2023-10-06T02:27:57.541451+0000","last_fullsized":"2023-10-06T02:27:57.541451+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_clean_scrub_stamp":"2023-10-06T02:27:56.508109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:02:40.303304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.575695+0000","last_change":"2023-10-06T02:27:58.575695+0000","last_active":"2023-10-06T02:27:58.575695+0000","last_peered":"2023-10-06T02:27:58.575695+0000","last_clean":"2023-10-06T02:27:58.575695+0000","last_became_active":"2023-10-06T02:27:58.575412+0000","last_became_peered":"2023-10-06T02:27:58.575412+0000","last_unstale":"2023-10-06T02:27:58.575695+0000","last_undegraded":"2023-10-06T02:27:58.575695+0000","last_fullsized":"2023-10-06T02:27:58.575695+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:36:38.988969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"86'2","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.561264+0000","last_change":"2023-10-06T02:27:57.534301+0000","last_active":"2023-10-06T02:27:58.561264+0000","last_peered":"2023-10-06T02:27:58.561264+0000","last_clean":"2023-10-06T02:27:58.561264+0000","last_became_active":"2023-10-06T02:27:57.533968+0000","last_became_peered":"2023-10-06T02:27:57.533968+0000","last_unstale":"2023-10-06T02:27:58.561264+0000","last_undegraded":"2023-10-06T02:27:58.561264+0000","last_fullsized":"2023-10-06T02:27:58.561264+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_clean_scrub_stamp":"2023-10-06T02:27:56.508109+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:14:32.484064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":67,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T02:27:57.525569+0000","last_change":"2023-10-06T02:23:02.054039+0000","last_active":"2023-10-06T02:27:57.525569+0000","last_peered":"2023-10-06T02:27:57.525569+0000","last_clean":"2023-10-06T02:27:57.525569+0000","last_became_active":"2023-10-06T02:23:02.053742+0000","last_became_peered":"2023-10-06T02:23:02.053742+0000","last_unstale":"2023-10-06T02:27:57.525569+0000","last_undegraded":"2023-10-06T02:27:57.525569+0000","last_fullsized":"2023-10-06T02:27:57.525569+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:58:18.846475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.554605+0000","last_change":"2023-10-06T02:27:58.554605+0000","last_active":"2023-10-06T02:27:58.554605+0000","last_peered":"2023-10-06T02:27:58.554605+0000","last_clean":"2023-10-06T02:27:58.554605+0000","last_became_active":"2023-10-06T02:27:58.554272+0000","last_became_peered":"2023-10-06T02:27:58.554272+0000","last_unstale":"2023-10-06T02:27:58.554605+0000","last_undegraded":"2023-10-06T02:27:58.554605+0000","last_fullsized":"2023-10-06T02:27:58.554605+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:07:51.806549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"86'1","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.544277+0000","last_change":"2023-10-06T02:27:57.545157+0000","last_active":"2023-10-06T02:27:58.544277+0000","last_peered":"2023-10-06T02:27:58.544277+0000","last_clean":"2023-10-06T02:27:58.544277+0000","last_became_active":"2023-10-06T02:27:57.544943+0000","last_became_peered":"2023-10-06T02:27:57.544943+0000","last_unstale":"2023-10-06T02:27:58.544277+0000","last_undegraded":"2023-10-06T02:27:58.544277+0000","last_fullsized":"2023-10-06T02:27:58.544277+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_clean_scrub_stamp":"2023-10-06T02:27:56.508109+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:18:09.904994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":76,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T02:27:57.524279+0000","last_change":"2023-10-06T02:23:01.988631+0000","last_active":"2023-10-06T02:27:57.524279+0000","last_peered":"2023-10-06T02:27:57.524279+0000","last_clean":"2023-10-06T02:27:57.524279+0000","last_became_active":"2023-10-06T02:23:01.988303+0000","last_became_peered":"2023-10-06T02:23:01.988303+0000","last_unstale":"2023-10-06T02:27:57.524279+0000","last_undegraded":"2023-10-06T02:27:57.524279+0000","last_fullsized":"2023-10-06T02:27:57.524279+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:38.211466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.546378+0000","last_change":"2023-10-06T02:27:58.546378+0000","last_active":"2023-10-06T02:27:58.546378+0000","last_peered":"2023-10-06T02:27:58.546378+0000","last_clean":"2023-10-06T02:27:58.546378+0000","last_became_active":"2023-10-06T02:27:58.546042+0000","last_became_peered":"2023-10-06T02:27:58.546042+0000","last_unstale":"2023-10-06T02:27:58.546378+0000","last_undegraded":"2023-10-06T02:27:58.546378+0000","last_fullsized":"2023-10-06T02:27:58.546378+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:14:33.490205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"86'2","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.564695+0000","last_change":"2023-10-06T02:27:57.536782+0000","last_active":"2023-10-06T02:27:58.564695+0000","last_peered":"2023-10-06T02:27:58.564695+0000","last_clean":"2023-10-06T02:27:58.564695+0000","last_became_active":"2023-10-06T02:27:57.536488+0000","last_became_peered":"2023-10-06T02:27:57.536488+0000","last_unstale":"2023-10-06T02:27:58.564695+0000","last_undegraded":"2023-10-06T02:27:58.564695+0000","last_fullsized":"2023-10-06T02:27:58.564695+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_clean_scrub_stamp":"2023-10-06T02:27:56.508109+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:21:24.795347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":71,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T02:27:57.519904+0000","last_change":"2023-10-06T02:23:02.054338+0000","last_active":"2023-10-06T02:27:57.519904+0000","last_peered":"2023-10-06T02:27:57.519904+0000","last_clean":"2023-10-06T02:27:57.519904+0000","last_became_active":"2023-10-06T02:23:02.053991+0000","last_became_peered":"2023-10-06T02:23:02.053991+0000","last_unstale":"2023-10-06T02:27:57.519904+0000","last_undegraded":"2023-10-06T02:27:57.519904+0000","last_fullsized":"2023-10-06T02:27:57.519904+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:39:30.192838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.550264+0000","last_change":"2023-10-06T02:27:58.550264+0000","last_active":"2023-10-06T02:27:58.550264+0000","last_peered":"2023-10-06T02:27:58.550264+0000","last_clean":"2023-10-06T02:27:58.550264+0000","last_became_active":"2023-10-06T02:27:58.549968+0000","last_became_peered":"2023-10-06T02:27:58.549968+0000","last_unstale":"2023-10-06T02:27:58.550264+0000","last_undegraded":"2023-10-06T02:27:58.550264+0000","last_fullsized":"2023-10-06T02:27:58.550264+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:58:44.780229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T02:27:57.544544+0000","last_change":"2023-10-06T02:27:57.541527+0000","last_active":"2023-10-06T02:27:57.544544+0000","last_peered":"2023-10-06T02:27:57.544544+0000","last_clean":"2023-10-06T02:27:57.544544+0000","last_became_active":"2023-10-06T02:27:57.541166+0000","last_became_peered":"2023-10-06T02:27:57.541166+0000","last_unstale":"2023-10-06T02:27:57.544544+0000","last_undegraded":"2023-10-06T02:27:57.544544+0000","last_fullsized":"2023-10-06T02:27:57.544544+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_clean_scrub_stamp":"2023-10-06T02:27:56.508109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:09:43.416185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":76,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T02:27:57.524276+0000","last_change":"2023-10-06T02:23:01.988140+0000","last_active":"2023-10-06T02:27:57.524276+0000","last_peered":"2023-10-06T02:27:57.524276+0000","last_clean":"2023-10-06T02:27:57.524276+0000","last_became_active":"2023-10-06T02:23:01.987769+0000","last_became_peered":"2023-10-06T02:23:01.987769+0000","last_unstale":"2023-10-06T02:27:57.524276+0000","last_undegraded":"2023-10-06T02:27:57.524276+0000","last_fullsized":"2023-10-06T02:27:57.524276+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:20:08.391912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"76'128","reported_seq":255,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T02:27:57.526574+0000","last_change":"2023-10-06T02:22:55.032519+0000","last_active":"2023-10-06T02:27:57.526574+0000","last_peered":"2023-10-06T02:27:57.526574+0000","last_clean":"2023-10-06T02:27:57.526574+0000","last_became_active":"2023-10-06T02:22:55.031367+0000","last_became_peered":"2023-10-06T02:22:55.031367+0000","last_unstale":"2023-10-06T02:27:57.526574+0000","last_undegraded":"2023-10-06T02:27:57.526574+0000","last_fullsized":"2023-10-06T02:27:57.526574+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:22:53.309097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:22:53.309097+0000","last_clean_scrub_stamp":"2023-10-06T02:22:53.309097+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:53:46.520909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.557038+0000","last_change":"2023-10-06T02:27:58.557038+0000","last_active":"2023-10-06T02:27:58.557038+0000","last_peered":"2023-10-06T02:27:58.557038+0000","last_clean":"2023-10-06T02:27:58.557038+0000","last_became_active":"2023-10-06T02:27:58.556750+0000","last_became_peered":"2023-10-06T02:27:58.556750+0000","last_unstale":"2023-10-06T02:27:58.557038+0000","last_undegraded":"2023-10-06T02:27:58.557038+0000","last_fullsized":"2023-10-06T02:27:58.557038+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:31:58.474184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"86'1","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.534637+0000","last_change":"2023-10-06T02:27:57.535933+0000","last_active":"2023-10-06T02:27:58.534637+0000","last_peered":"2023-10-06T02:27:58.534637+0000","last_clean":"2023-10-06T02:27:58.534637+0000","last_became_active":"2023-10-06T02:27:57.535685+0000","last_became_peered":"2023-10-06T02:27:57.535685+0000","last_unstale":"2023-10-06T02:27:58.534637+0000","last_undegraded":"2023-10-06T02:27:58.534637+0000","last_fullsized":"2023-10-06T02:27:58.534637+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_clean_scrub_stamp":"2023-10-06T02:27:56.508109+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:52:25.592921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":63,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T02:27:57.526478+0000","last_change":"2023-10-06T02:23:01.990773+0000","last_active":"2023-10-06T02:27:57.526478+0000","last_peered":"2023-10-06T02:27:57.526478+0000","last_clean":"2023-10-06T02:27:57.526478+0000","last_became_active":"2023-10-06T02:23:01.990511+0000","last_became_peered":"2023-10-06T02:23:01.990511+0000","last_unstale":"2023-10-06T02:27:57.526478+0000","last_undegraded":"2023-10-06T02:27:57.526478+0000","last_fullsized":"2023-10-06T02:27:57.526478+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:42:54.797731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.541295+0000","last_change":"2023-10-06T02:27:58.541295+0000","last_active":"2023-10-06T02:27:58.541295+0000","last_peered":"2023-10-06T02:27:58.541295+0000","last_clean":"2023-10-06T02:27:58.541295+0000","last_became_active":"2023-10-06T02:27:58.540972+0000","last_became_peered":"2023-10-06T02:27:58.540972+0000","last_unstale":"2023-10-06T02:27:58.541295+0000","last_undegraded":"2023-10-06T02:27:58.541295+0000","last_fullsized":"2023-10-06T02:27:58.541295+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:05:24.882141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T02:27:57.543463+0000","last_change":"2023-10-06T02:27:57.537746+0000","last_active":"2023-10-06T02:27:57.543463+0000","last_peered":"2023-10-06T02:27:57.543463+0000","last_clean":"2023-10-06T02:27:57.543463+0000","last_became_active":"2023-10-06T02:27:57.537140+0000","last_became_peered":"2023-10-06T02:27:57.537140+0000","last_unstale":"2023-10-06T02:27:57.543463+0000","last_undegraded":"2023-10-06T02:27:57.543463+0000","last_fullsized":"2023-10-06T02:27:57.543463+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_clean_scrub_stamp":"2023-10-06T02:27:56.508109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:21:50.264914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":63,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T02:27:57.527295+0000","last_change":"2023-10-06T02:23:02.054069+0000","last_active":"2023-10-06T02:27:57.527295+0000","last_peered":"2023-10-06T02:27:57.527295+0000","last_clean":"2023-10-06T02:27:57.527295+0000","last_became_active":"2023-10-06T02:23:02.053739+0000","last_became_peered":"2023-10-06T02:23:02.053739+0000","last_unstale":"2023-10-06T02:27:57.527295+0000","last_undegraded":"2023-10-06T02:27:57.527295+0000","last_fullsized":"2023-10-06T02:27:57.527295+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:57:48.822608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.572001+0000","last_change":"2023-10-06T02:27:58.572001+0000","last_active":"2023-10-06T02:27:58.572001+0000","last_peered":"2023-10-06T02:27:58.572001+0000","last_clean":"2023-10-06T02:27:58.572001+0000","last_became_active":"2023-10-06T02:27:58.571716+0000","last_became_peered":"2023-10-06T02:27:58.571716+0000","last_unstale":"2023-10-06T02:27:58.572001+0000","last_undegraded":"2023-10-06T02:27:58.572001+0000","last_fullsized":"2023-10-06T02:27:58.572001+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:50:11.405633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T02:27:57.539620+0000","last_change":"2023-10-06T02:27:57.534965+0000","last_active":"2023-10-06T02:27:57.539620+0000","last_peered":"2023-10-06T02:27:57.539620+0000","last_clean":"2023-10-06T02:27:57.539620+0000","last_became_active":"2023-10-06T02:27:57.534690+0000","last_became_peered":"2023-10-06T02:27:57.534690+0000","last_unstale":"2023-10-06T02:27:57.539620+0000","last_undegraded":"2023-10-06T02:27:57.539620+0000","last_fullsized":"2023-10-06T02:27:57.539620+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_clean_scrub_stamp":"2023-10-06T02:27:56.508109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:41:38.011422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":67,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T02:27:57.525521+0000","last_change":"2023-10-06T02:23:01.989934+0000","last_active":"2023-10-06T02:27:57.525521+0000","last_peered":"2023-10-06T02:27:57.525521+0000","last_clean":"2023-10-06T02:27:57.525521+0000","last_became_active":"2023-10-06T02:23:01.989706+0000","last_became_peered":"2023-10-06T02:23:01.989706+0000","last_unstale":"2023-10-06T02:27:57.525521+0000","last_undegraded":"2023-10-06T02:27:57.525521+0000","last_fullsized":"2023-10-06T02:27:57.525521+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:06.212251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.555915+0000","last_change":"2023-10-06T02:27:58.555915+0000","last_active":"2023-10-06T02:27:58.555915+0000","last_peered":"2023-10-06T02:27:58.555915+0000","last_clean":"2023-10-06T02:27:58.555915+0000","last_became_active":"2023-10-06T02:27:58.555559+0000","last_became_peered":"2023-10-06T02:27:58.555559+0000","last_unstale":"2023-10-06T02:27:58.555915+0000","last_undegraded":"2023-10-06T02:27:58.555915+0000","last_fullsized":"2023-10-06T02:27:58.555915+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:38.699087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T02:27:57.542774+0000","last_change":"2023-10-06T02:27:57.532459+0000","last_active":"2023-10-06T02:27:57.542774+0000","last_peered":"2023-10-06T02:27:57.542774+0000","last_clean":"2023-10-06T02:27:57.542774+0000","last_became_active":"2023-10-06T02:27:57.532196+0000","last_became_peered":"2023-10-06T02:27:57.532196+0000","last_unstale":"2023-10-06T02:27:57.542774+0000","last_undegraded":"2023-10-06T02:27:57.542774+0000","last_fullsized":"2023-10-06T02:27:57.542774+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_clean_scrub_stamp":"2023-10-06T02:27:56.508109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:46:58.999553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":68,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T02:27:57.532778+0000","last_change":"2023-10-06T02:23:01.992935+0000","last_active":"2023-10-06T02:27:57.532778+0000","last_peered":"2023-10-06T02:27:57.532778+0000","last_clean":"2023-10-06T02:27:57.532778+0000","last_became_active":"2023-10-06T02:23:01.992568+0000","last_became_peered":"2023-10-06T02:23:01.992568+0000","last_unstale":"2023-10-06T02:27:57.532778+0000","last_undegraded":"2023-10-06T02:27:57.532778+0000","last_fullsized":"2023-10-06T02:27:57.532778+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:06.212251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574914,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302996,"kb_used_data":1612,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068844,"statfs":{"total":96636764160,"available":96326496256,"internally_reserved":0,"allocated":1650688,"data_stored":1068141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:26:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.64100000000000001,"15min":0.63800000000000001},"min":{"1min":0.42399999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.002,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.624,"15min":0.60599999999999998},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.84399999999999997,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.72399999999999998}]},{"osd":1,"last update":"Fri Oct 6 02:26:58 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.71099999999999997,"15min":0.68500000000000005},"min":{"1min":0.46800000000000003,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.95399999999999996,"5min":1.147,"15min":1.147},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.68200000000000005,"15min":0.67500000000000004},"min":{"1min":0.498,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.86899999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.79300000000000004}]},{"osd":2,"last update":"Fri Oct 6 02:26:58 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.69299999999999995,"15min":0.69499999999999995},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.68700000000000006,"15min":0.68999999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.93500000000000005,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.80700000000000005}]},{"osd":3,"last update":"Fri Oct 6 02:26:58 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.747,"15min":0.73299999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.754,"5min":0.77600000000000002,"15min":0.77800000000000002},"min":{"1min":0.55500000000000005,"5min":0.497,"15min":0.497},"max":{"1min":0.98199999999999998,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.83799999999999997}]},{"osd":4,"last update":"Fri Oct 6 02:26:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.65000000000000002,"15min":0.69599999999999995},"min":{"1min":0.41099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.95199999999999996,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.72599999999999998,"15min":0.77000000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.105,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.66500000000000004}]},{"osd":5,"last update":"Fri Oct 6 02:26:58 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.71499999999999997,"15min":0.73499999999999999},"min":{"1min":0.44700000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.98499999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.71499999999999997,"15min":0.73499999999999999},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.95399999999999996,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.64600000000000002}]},{"osd":6,"last update":"Fri Oct 6 02:26:58 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.77400000000000002,"15min":0.78700000000000003},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0109999999999999,"5min":1.056,"15min":1.056},"last":0.748},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.80600000000000005,"15min":0.81599999999999995},"min":{"1min":0.48199999999999998,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.1659999999999999,"5min":1.276,"15min":1.276},"last":0.82199999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574914,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302436,"kb_used_data":1052,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069404,"statfs":{"total":96636764160,"available":96327069696,"internally_reserved":0,"allocated":1077248,"data_stored":479885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:27:03 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.67100000000000004,"15min":0.66400000000000003},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.91900000000000004,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.71399999999999997,"15min":0.69699999999999995},"min":{"1min":0.53500000000000003,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.95099999999999996,"5min":1.119,"15min":1.119},"last":0.628}]},{"osd":1,"last update":"Fri Oct 6 02:27:03 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.64100000000000001,"15min":0.64400000000000002},"min":{"1min":0.433,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67400000000000004,"15min":0.66600000000000004},"min":{"1min":0.48899999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.89500000000000002,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.68500000000000005}]},{"osd":2,"last update":"Fri Oct 6 02:27:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.71199999999999997,"15min":0.70199999999999996},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.96099999999999997,"5min":1.137,"15min":1.137},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72699999999999998,"15min":0.71099999999999997},"min":{"1min":0.55600000000000005,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.89300000000000002,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.55800000000000005}]},{"osd":3,"last update":"Fri Oct 6 02:27:03 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.68200000000000005,"15min":0.66000000000000003},"min":{"1min":0.52000000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.97099999999999997,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71299999999999997,"15min":0.70299999999999996},"min":{"1min":0.5,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.91600000000000004,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.60399999999999998}]},{"osd":4,"last update":"Fri Oct 6 02:27:03 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63900000000000001,"15min":0.63400000000000001},"min":{"1min":0.36199999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.90000000000000002,"5min":1.095,"15min":1.095},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.67600000000000005,"15min":0.65600000000000003},"min":{"1min":0.46200000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.999,"5min":1.29,"15min":1.29},"last":0.80300000000000005}]},{"osd":5,"last update":"Fri Oct 6 02:27:03 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.68100000000000005,"15min":0.69699999999999995},"min":{"1min":0.48399999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.94299999999999995,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.70699999999999996,"15min":0.70199999999999996},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.86099999999999999,"5min":1.02,"15min":1.02},"last":0.82299999999999995}]},{"osd":7,"last update":"Fri Oct 6 02:27:03 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.74199999999999999,"15min":0.746},"min":{"1min":0.41499999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.95399999999999996,"5min":1.458,"15min":1.458},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76000000000000001,"15min":0.75900000000000001},"min":{"1min":0.54600000000000004,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.96499999999999997,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.75}]}]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302420,"kb_used_data":1036,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069420,"statfs":{"total":96636764160,"available":96327086080,"internally_reserved":0,"allocated":1060864,"data_stored":477843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.442,"15min":0.46000000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.58299999999999996,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.315},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.48799999999999999,"15min":0.52700000000000002},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.70099999999999996,"5min":1.788,"15min":1.788},"last":0.40400000000000003}]},{"osd":2,"last update":"Fri Oct 6 02:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.46700000000000003,"15min":0.52300000000000002},"min":{"1min":0.307,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.60899999999999999,"5min":2.0840000000000001,"15min":2.0840000000000001},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.52100000000000002,"15min":0.55800000000000005},"min":{"1min":0.36899999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.66400000000000003,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.376}]},{"osd":3,"last update":"Fri Oct 6 02:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60799999999999998,"15min":0.61699999999999999},"min":{"1min":0.46000000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.84999999999999998,"5min":1.266,"15min":1.266},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.56699999999999995,"15min":0.63900000000000001},"min":{"1min":0.41099999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.79400000000000004,"5min":1.861,"15min":1.861},"last":0.47499999999999998}]},{"osd":4,"last update":"Fri Oct 6 02:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.80700000000000005,"15min":0.85699999999999998},"min":{"1min":0.59099999999999997,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.3580000000000001,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.81499999999999995,"15min":0.85799999999999998},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.4339999999999999,"5min":2.0179999999999998,"15min":2.0179999999999998},"last":0.56200000000000006}]},{"osd":5,"last update":"Fri Oct 6 02:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.77600000000000002,"15min":0.85799999999999998},"min":{"1min":0.52600000000000002,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.1040000000000001,"5min":1.9830000000000001,"15min":1.9830000000000001},"last":0.627},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.77400000000000002,"15min":0.82199999999999995},"min":{"1min":0.58899999999999997,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.161,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.73599999999999999}]},{"osd":6,"last update":"Fri Oct 6 02:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.81799999999999995,"15min":0.86699999999999999},"min":{"1min":0.52700000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0609999999999999,"5min":1.829,"15min":1.829},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.84399999999999997,"15min":0.91200000000000003},"min":{"1min":0.64600000000000002,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.282,"5min":2.052,"15min":2.052},"last":0.68000000000000005}]},{"osd":7,"last update":"Fri Oct 6 02:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.80000000000000004,"15min":0.83699999999999997},"min":{"1min":0.56299999999999994,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.3120000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.81699999999999995,"15min":0.879},"min":{"1min":0.55000000000000004,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.2110000000000001,"5min":1.895,"15min":1.895},"last":0.69499999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574914,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303016,"kb_used_data":1632,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068824,"statfs":{"total":96636764160,"available":96326475776,"internally_reserved":0,"allocated":1671168,"data_stored":1069463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 02:27:05 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.44,"15min":0.45600000000000002},"min":{"1min":0.32000000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.54300000000000004,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.439,"5min":0.47399999999999998,"15min":0.48199999999999998},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.58799999999999997,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.43099999999999999}]},{"osd":2,"last update":"Fri Oct 6 02:27:05 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.498,"15min":0.51400000000000001},"min":{"1min":0.40899999999999997,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.60699999999999998,"5min":1.069,"15min":1.069},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.51100000000000001,"15min":0.52200000000000002},"min":{"1min":0.39600000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.73099999999999998,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.495}]},{"osd":3,"last update":"Fri Oct 6 02:27:05 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.49399999999999999,"15min":0.50600000000000001},"min":{"1min":0.34100000000000003,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.59599999999999997,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59499999999999997,"15min":0.59699999999999998},"min":{"1min":0.47299999999999998,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.73099999999999998,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.56200000000000006}]},{"osd":4,"last update":"Fri Oct 6 02:27:05 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.77000000000000002,"15min":0.80300000000000005},"min":{"1min":0.54800000000000004,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.86899999999999999,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.77700000000000002,"15min":0.80300000000000005},"min":{"1min":0.58099999999999996,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0469999999999999,"5min":1.304,"15min":1.304},"last":0.629}]},{"osd":5,"last update":"Fri Oct 6 02:27:05 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.76600000000000001,"15min":0.78800000000000003},"min":{"1min":0.52600000000000002,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.94199999999999995,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.79100000000000004,"15min":0.80300000000000005},"min":{"1min":0.59699999999999998,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.018,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.70099999999999996}]},{"osd":6,"last update":"Fri Oct 6 02:27:05 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.81299999999999994,"15min":0.82599999999999996},"min":{"1min":0.59699999999999998,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.96399999999999997,"5min":1.167,"15min":1.167},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.80500000000000005,"15min":0.83499999999999996},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.0349999999999999,"5min":1.069,"15min":1.069},"last":0.68000000000000005}]},{"osd":7,"last update":"Fri Oct 6 02:27:05 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.80500000000000005,"15min":0.82899999999999996},"min":{"1min":0.56899999999999995,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.98499999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.79600000000000004,"15min":0.82899999999999996},"min":{"1min":0.54700000000000004,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.002,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.66700000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574915,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302416,"kb_used_data":1032,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069424,"statfs":{"total":96636764160,"available":96327090176,"internally_reserved":0,"allocated":1056768,"data_stored":477773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:27:02 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.434,"15min":0.441},"min":{"1min":0.312,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.54100000000000004,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.48299999999999998,"15min":0.48799999999999999},"min":{"1min":0.32800000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.88700000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.47599999999999998}]},{"osd":1,"last update":"Fri Oct 6 02:27:02 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.5,"15min":0.50700000000000001},"min":{"1min":0.36699999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.63900000000000001,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.56399999999999995,"15min":0.61299999999999999},"min":{"1min":0.39600000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.70899999999999996,"5min":1.345,"15min":1.345},"last":0.51600000000000001}]},{"osd":3,"last update":"Fri Oct 6 02:27:02 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.52200000000000002,"15min":0.53600000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.69699999999999995,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.60699999999999998,"15min":0.624},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.72799999999999998,"5min":1.028,"15min":1.028},"last":0.60099999999999998}]},{"osd":4,"last update":"Fri Oct 6 02:27:02 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.76400000000000001,"15min":0.78100000000000003},"min":{"1min":0.55500000000000005,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.89300000000000002,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.82399999999999995,"15min":0.85499999999999998},"min":{"1min":0.59999999999999998,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.95999999999999996,"5min":1.621,"15min":1.621},"last":0.76200000000000001}]},{"osd":5,"last update":"Fri Oct 6 02:27:02 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.77500000000000002,"15min":0.80800000000000005},"min":{"1min":0.57099999999999995,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.94899999999999995,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.80000000000000004,"15min":0.82899999999999996},"min":{"1min":0.60399999999999998,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":0.98199999999999998,"5min":1.258,"15min":1.258},"last":0.77800000000000002}]},{"osd":6,"last update":"Fri Oct 6 02:27:02 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.82299999999999995,"15min":0.85399999999999998},"min":{"1min":0.59099999999999997,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.98099999999999998,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.78900000000000003,"15min":0.79900000000000004},"min":{"1min":0.623,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.0109999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.78900000000000003}]},{"osd":7,"last update":"Fri Oct 6 02:27:02 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.85299999999999998,"15min":0.89200000000000002},"min":{"1min":0.66200000000000003,"5min":0.499,"15min":0.499},"max":{"1min":0.95999999999999996,"5min":1.585,"15min":1.585},"last":0.628},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.81100000000000005,"15min":0.83699999999999997},"min":{"1min":0.58999999999999997,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.94399999999999995,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.748}]}]},{"osd":3,"up_from":13,"seq":55834574912,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302416,"kb_used_data":1032,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069424,"statfs":{"total":96636764160,"available":96327090176,"internally_reserved":0,"allocated":1056768,"data_stored":477773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:27:02 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48899999999999999,"15min":0.51700000000000002},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.69599999999999995,"5min":2.024,"15min":2.024},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.51200000000000001,"15min":0.52800000000000002},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.67400000000000004,"5min":2.181,"15min":2.181},"last":0.48799999999999999}]},{"osd":1,"last update":"Fri Oct 6 02:27:02 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.58799999999999997,"15min":0.70299999999999996},"min":{"1min":0.39600000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.59299999999999997,"5min":4.492,"15min":4.492},"last":0.438},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.63700000000000001,"15min":0.745},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.80000000000000004,"5min":4.569,"15min":4.569},"last":0.52500000000000002}]},{"osd":2,"last update":"Fri Oct 6 02:27:02 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.65800000000000003,"15min":0.67200000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.91100000000000003,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.625,"15min":0.65900000000000003},"min":{"1min":0.46200000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.94199999999999995,"5min":2.4260000000000002,"15min":2.4260000000000002},"last":0.50700000000000001}]},{"osd":4,"last update":"Fri Oct 6 02:27:02 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.78600000000000003,"15min":0.83899999999999997},"min":{"1min":0.61599999999999999,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.96699999999999997,"5min":2.214,"15min":2.214},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.78600000000000003,"15min":0.81699999999999995},"min":{"1min":0.50800000000000001,"5min":0.5,"15min":0.5},"max":{"1min":0.94399999999999995,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.76900000000000002}]},{"osd":5,"last update":"Fri Oct 6 02:27:02 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.80500000000000005,"15min":0.82899999999999996},"min":{"1min":0.57899999999999996,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.97699999999999998,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.81100000000000005,"15min":0.86499999999999999},"min":{"1min":0.63800000000000001,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.0289999999999999,"5min":2.242,"15min":2.242},"last":0.68799999999999994}]},{"osd":6,"last update":"Fri Oct 6 02:27:02 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.79200000000000004,"15min":0.82399999999999995},"min":{"1min":0.59399999999999997,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.0049999999999999,"5min":1.357,"15min":1.357},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.81999999999999995,"15min":0.83799999999999997},"min":{"1min":0.54600000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.0529999999999999,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.71999999999999997}]},{"osd":7,"last update":"Fri Oct 6 02:27:02 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.83399999999999996,"15min":0.87},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.077,"5min":2.27,"15min":2.27},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.83099999999999996,"15min":0.86399999999999999},"min":{"1min":0.55600000000000005,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.056,"5min":1.325,"15min":1.325},"last":0.80300000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574914,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302416,"kb_used_data":1032,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069424,"statfs":{"total":96636764160,"available":96327090176,"internally_reserved":0,"allocated":1056768,"data_stored":477773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:27:03 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.86599999999999999,"15min":1.0549999999999999},"min":{"1min":0.45700000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.90700000000000003,"5min":12.337,"15min":12.337},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.81399999999999995,"15min":0.95099999999999996},"min":{"1min":0.496,"5min":0.438,"15min":0.438},"max":{"1min":0.91700000000000004,"5min":9.6539999999999999,"15min":9.6539999999999999},"last":0.67100000000000004}]},{"osd":1,"last update":"Fri Oct 6 02:27:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.69999999999999996,"15min":0.70599999999999996},"min":{"1min":0.53800000000000003,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.96999999999999997,"5min":3.181,"15min":3.181},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.876,"15min":1.0740000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.90000000000000002,"5min":12.946999999999999,"15min":12.946999999999999},"last":0.56299999999999994}]},{"osd":2,"last update":"Fri Oct 6 02:27:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.89700000000000002,"15min":1.103},"min":{"1min":0.44400000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.93300000000000005,"5min":12.377000000000001,"15min":12.377000000000001},"last":0.629},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.80100000000000005,"15min":0.89500000000000002},"min":{"1min":0.53800000000000003,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.95899999999999996,"5min":6.3220000000000001,"15min":6.3220000000000001},"last":0.69299999999999995}]},{"osd":3,"last update":"Fri Oct 6 02:27:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.873,"15min":1.0309999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.88500000000000001,"5min":9.7720000000000002,"15min":9.7720000000000002},"last":0.753},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.745,"15min":0.78500000000000003},"min":{"1min":0.498,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.91600000000000004,"5min":3.0840000000000001,"15min":3.0840000000000001},"last":0.52400000000000002}]},{"osd":5,"last update":"Fri Oct 6 02:27:03 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.76100000000000001,"15min":0.86899999999999999},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.82899999999999996,"5min":6.2690000000000001,"15min":6.2690000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.874,"15min":1.0840000000000001},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.90900000000000003,"5min":10.769,"15min":10.769},"last":0.54400000000000004}]},{"osd":6,"last update":"Fri Oct 6 02:27:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.71199999999999997,"15min":0.77000000000000002},"min":{"1min":0.44,"5min":0.432,"15min":0.432},"max":{"1min":0.84299999999999997,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.85599999999999998,"15min":1.0029999999999999},"min":{"1min":0.49199999999999999,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.94399999999999995,"5min":8.0899999999999999,"15min":8.0899999999999999},"last":0.5}]},{"osd":7,"last update":"Fri Oct 6 02:27:03 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.92600000000000005,"15min":1.113},"min":{"1min":0.50600000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.93899999999999995,"5min":9.7089999999999996,"15min":9.7089999999999996},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.88800000000000001,"15min":1.034},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.97799999999999998,"5min":7.6719999999999997,"15min":7.6719999999999997},"last":0.65500000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574914,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302416,"kb_used_data":1032,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069424,"statfs":{"total":96636764160,"available":96327090176,"internally_reserved":0,"allocated":1056768,"data_stored":477773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:27:00 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.64200000000000002,"15min":0.626},"min":{"1min":0.44700000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.84899999999999998,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.64900000000000002,"15min":0.60799999999999998},"min":{"1min":0.41699999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.93999999999999995,"5min":1.026,"15min":1.026},"last":0.58899999999999997}]},{"osd":1,"last update":"Fri Oct 6 02:27:00 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.67000000000000004,"15min":0.65000000000000002},"min":{"1min":0.438,"5min":0.434,"15min":0.434},"max":{"1min":0.95399999999999996,"5min":1.089,"15min":1.089},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.68899999999999995,"15min":0.67500000000000004},"min":{"1min":0.53200000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.88300000000000001,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.622}]},{"osd":2,"last update":"Fri Oct 6 02:27:00 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.69599999999999995,"15min":0.66800000000000004},"min":{"1min":0.499,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.98999999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.70099999999999996,"15min":0.68300000000000005},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.96099999999999997,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.69399999999999995}]},{"osd":3,"last update":"Fri Oct 6 02:27:00 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.748,"15min":0.72599999999999998},"min":{"1min":0.58399999999999996,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.93999999999999995,"5min":1.042,"15min":1.042},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.78200000000000003,"15min":0.76400000000000001},"min":{"1min":0.59499999999999997,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.0489999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.84699999999999998}]},{"osd":4,"last update":"Fri Oct 6 02:27:00 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.66100000000000003,"15min":0.67300000000000004},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.89000000000000001,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.624,"5min":0.67100000000000004,"15min":0.68200000000000005},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.86499999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.80700000000000005}]},{"osd":6,"last update":"Fri Oct 6 02:27:00 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.73899999999999999,"15min":0.74299999999999999},"min":{"1min":0.45700000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.0549999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.81499999999999995,"15min":0.82099999999999995},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.03,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.76500000000000001}]},{"osd":7,"last update":"Fri Oct 6 02:27:00 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.75900000000000001,"15min":0.76300000000000001},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.0009999999999999,"5min":1.089,"15min":1.089},"last":0.745},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.81799999999999995,"15min":0.83599999999999997},"min":{"1min":0.46999999999999997,"5min":0.437,"15min":0.437},"max":{"1min":1.0229999999999999,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.89300000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T02:28:02.957 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T02:28:03.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.177+0000 7fb69ffff700 1 Processor -- start 2023-10-06T02:28:03.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.178+0000 7fb69ffff700 1 -- start start 2023-10-06T02:28:03.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.179+0000 7fb69ffff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb6a0140a70 0x7fb6a00c6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:03.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.179+0000 7fb69ffff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb6a0140e40 0x7fb6a00c7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:03.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.179+0000 7fb69ffff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb6a00c8330 0x7fb6a00aeef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:03.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.179+0000 7fb69ffff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6a00c6d50 con 0x7fb6a0140e40 2023-10-06T02:28:03.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.179+0000 7fb69ffff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6a00c7030 con 0x7fb6a0140a70 2023-10-06T02:28:03.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.179+0000 7fb69ffff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a00c7310 con 0x7fb6a00c8330 2023-10-06T02:28:03.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.179+0000 7fb69e7fc700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb6a0140e40 0x7fb6a00c7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:03.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.179+0000 7fb69e7fc700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb6a0140e40 0x7fb6a00c7df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40500/0 (socket says 172.21.15.31:40500) 2023-10-06T02:28:03.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.179+0000 7fb69e7fc700 1 -- 172.21.15.31:0/3304332501 learned_addr learned my addr 172.21.15.31:0/3304332501 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:03.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.180+0000 7fb69f7fe700 1 --2- 172.21.15.31:0/3304332501 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb6a00c8330 0x7fb6a00aeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:03.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.180+0000 7fb69effd700 1 --2- 172.21.15.31:0/3304332501 >> v2:172.21.15.62:3300/0 conn(0x7fb6a0140a70 0x7fb6a00c6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:03.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.180+0000 7fb69e7fc700 1 -- 172.21.15.31:0/3304332501 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb6a00c8330 msgr2=0x7fb6a00aeef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:03.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.180+0000 7fb69e7fc700 1 --2- 172.21.15.31:0/3304332501 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb6a00c8330 0x7fb6a00aeef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:03.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.180+0000 7fb69e7fc700 1 -- 172.21.15.31:0/3304332501 >> v2:172.21.15.62:3300/0 conn(0x7fb6a0140a70 msgr2=0x7fb6a00c6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:03.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.180+0000 7fb69e7fc700 1 --2- 172.21.15.31:0/3304332501 >> v2:172.21.15.62:3300/0 conn(0x7fb6a0140a70 0x7fb6a00c6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:03.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.180+0000 7fb69e7fc700 1 -- 172.21.15.31:0/3304332501 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6a00c75f0 con 0x7fb6a0140e40 2023-10-06T02:28:03.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.180+0000 7fb69e7fc700 1 --2- 172.21.15.31:0/3304332501 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6a0140e40 0x7fb6a00c7df0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0069230 tx=0x7fb694009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5778096535830829 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:03.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.180+0000 7fb69dffb700 1 -- 172.21.15.31:0/3304332501 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb694005c10 con 0x7fb6a0140e40 2023-10-06T02:28:03.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.180+0000 7fb69dffb700 1 -- 172.21.15.31:0/3304332501 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb694005d70 con 0x7fb6a0140e40 2023-10-06T02:28:03.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.180+0000 7fb69dffb700 1 -- 172.21.15.31:0/3304332501 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb69400b880 con 0x7fb6a0140e40 2023-10-06T02:28:03.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.181+0000 7fb69ffff700 1 -- 172.21.15.31:0/3304332501 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6a0140e40 msgr2=0x7fb6a00c7df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:03.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.181+0000 7fb69ffff700 1 --2- 172.21.15.31:0/3304332501 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6a0140e40 0x7fb6a00c7df0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0069230 tx=0x7fb694009220 comp rx=0 tx=0).stop 2023-10-06T02:28:03.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.181+0000 7fb69ffff700 1 -- 172.21.15.31:0/3304332501 shutdown_connections 2023-10-06T02:28:03.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.181+0000 7fb69ffff700 1 --2- 172.21.15.31:0/3304332501 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb6a00c8330 0x7fb6a00aeef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:03.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.181+0000 7fb69ffff700 1 --2- 172.21.15.31:0/3304332501 >> v2:172.21.15.62:3300/0 conn(0x7fb6a0140a70 0x7fb6a00c6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:03.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.181+0000 7fb69ffff700 1 --2- 172.21.15.31:0/3304332501 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6a0140e40 0x7fb6a00c7df0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:03.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.181+0000 7fb69ffff700 1 -- 172.21.15.31:0/3304332501 >> 172.21.15.31:0/3304332501 conn(0x7fb6a006d170 msgr2=0x7fb6a00cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:03.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.181+0000 7fb69ffff700 1 -- 172.21.15.31:0/3304332501 shutdown_connections 2023-10-06T02:28:03.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.181+0000 7fb69ffff700 1 -- 172.21.15.31:0/3304332501 wait complete. 2023-10-06T02:28:03.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.183+0000 7fb69ffff700 1 Processor -- start 2023-10-06T02:28:03.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.183+0000 7fb69ffff700 1 -- start start 2023-10-06T02:28:03.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.184+0000 7fb69ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6a00c8330 0x7fb6a016df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:03.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.184+0000 7fb69ffff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb6a0140a70 0x7fb6a0169790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:03.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.184+0000 7fb69ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6a0140e40 0x7fb6a00b08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:03.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.184+0000 7fb69ffff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a016e470 con 0x7fb6a0140e40 2023-10-06T02:28:03.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.184+0000 7fb69ffff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a016ac40 con 0x7fb6a00c8330 2023-10-06T02:28:03.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.184+0000 7fb69ffff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a013e830 con 0x7fb6a0140a70 2023-10-06T02:28:03.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.184+0000 7fb69f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6a0140e40 0x7fb6a00b08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:03.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.184+0000 7fb69f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6a0140e40 0x7fb6a00b08e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40514/0 (socket says 172.21.15.31:40514) 2023-10-06T02:28:03.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.184+0000 7fb69f7fe700 1 -- 172.21.15.31:0/2223082955 learned_addr learned my addr 172.21.15.31:0/2223082955 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:03.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.184+0000 7fb69e7fc700 1 --2- 172.21.15.31:0/2223082955 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb6a0140a70 0x7fb6a0169790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:03.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.184+0000 7fb69effd700 1 --2- 172.21.15.31:0/2223082955 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6a00c8330 0x7fb6a016df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:03.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.184+0000 7fb69f7fe700 1 -- 172.21.15.31:0/2223082955 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb6a0140a70 msgr2=0x7fb6a0169790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:03.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.184+0000 7fb69f7fe700 1 --2- 172.21.15.31:0/2223082955 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb6a0140a70 0x7fb6a0169790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:03.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.184+0000 7fb69f7fe700 1 -- 172.21.15.31:0/2223082955 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6a00c8330 msgr2=0x7fb6a016df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:03.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.184+0000 7fb69f7fe700 1 --2- 172.21.15.31:0/2223082955 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6a00c8330 0x7fb6a016df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:03.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.184+0000 7fb69f7fe700 1 -- 172.21.15.31:0/2223082955 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb694009480 con 0x7fb6a0140e40 2023-10-06T02:28:03.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.184+0000 7fb69effd700 1 --2- 172.21.15.31:0/2223082955 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6a00c8330 0x7fb6a016df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:28:03.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.185+0000 7fb69f7fe700 1 --2- 172.21.15.31:0/2223082955 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6a0140e40 0x7fb6a00b08e0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fb698009fd0 tx=0x7fb698009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:03.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.185+0000 7fb686ffd700 1 -- 172.21.15.31:0/2223082955 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb69800ed50 con 0x7fb6a0140e40 2023-10-06T02:28:03.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.185+0000 7fb686ffd700 1 -- 172.21.15.31:0/2223082955 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb69800eeb0 con 0x7fb6a0140e40 2023-10-06T02:28:03.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.185+0000 7fb686ffd700 1 -- 172.21.15.31:0/2223082955 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb698014650 con 0x7fb6a0140e40 2023-10-06T02:28:03.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.185+0000 7fb69ffff700 1 -- 172.21.15.31:0/2223082955 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6a016a030 con 0x7fb6a0140e40 2023-10-06T02:28:03.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.185+0000 7fb69ffff700 1 -- 172.21.15.31:0/2223082955 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6a016a5f0 con 0x7fb6a0140e40 2023-10-06T02:28:03.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.186+0000 7fb69ffff700 1 -- 172.21.15.31:0/2223082955 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6a016add0 con 0x7fb6a0140e40 2023-10-06T02:28:03.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.187+0000 7fb686ffd700 1 -- 172.21.15.31:0/2223082955 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7fb6980149c0 con 0x7fb6a0140e40 2023-10-06T02:28:03.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.189+0000 7fb686ffd700 1 --2- 172.21.15.31:0/2223082955 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb68c067050 0x7fb68c069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:03.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.189+0000 7fb686ffd700 1 -- 172.21.15.31:0/2223082955 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb698085090 con 0x7fb6a0140e40 2023-10-06T02:28:03.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.189+0000 7fb69effd700 1 --2- 172.21.15.31:0/2223082955 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb68c067050 0x7fb68c069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:03.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.190+0000 7fb69effd700 1 --2- 172.21.15.31:0/2223082955 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb68c067050 0x7fb68c069500 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb690006fd0 tx=0x7fb690008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:03.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.193+0000 7fb686ffd700 1 -- 172.21.15.31:0/2223082955 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb698051ff0 con 0x7fb6a0140e40 2023-10-06T02:28:03.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.390+0000 7fb69ffff700 1 -- 172.21.15.31:0/2223082955 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb6a004c3c0 con 0x7fb68c067050 2023-10-06T02:28:03.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.401+0000 7fb686ffd700 1 -- 172.21.15.31:0/2223082955 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311244 (secure 0 0 0) 0x7fb6a004c3c0 con 0x7fb68c067050 2023-10-06T02:28:03.405 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:28:03.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.405+0000 7fb69ffff700 1 -- 172.21.15.31:0/2223082955 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb68c067050 msgr2=0x7fb68c069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:03.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.405+0000 7fb69ffff700 1 --2- 172.21.15.31:0/2223082955 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb68c067050 0x7fb68c069500 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb690006fd0 tx=0x7fb690008040 comp rx=0 tx=0).stop 2023-10-06T02:28:03.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.405+0000 7fb69ffff700 1 -- 172.21.15.31:0/2223082955 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6a0140e40 msgr2=0x7fb6a00b08e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:03.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.405+0000 7fb69ffff700 1 --2- 172.21.15.31:0/2223082955 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6a0140e40 0x7fb6a00b08e0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fb698009fd0 tx=0x7fb698009900 comp rx=0 tx=0).stop 2023-10-06T02:28:03.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.405+0000 7fb69ffff700 1 -- 172.21.15.31:0/2223082955 shutdown_connections 2023-10-06T02:28:03.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.405+0000 7fb69ffff700 1 --2- 172.21.15.31:0/2223082955 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6a0140e40 0x7fb6a00b08e0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:03.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.405+0000 7fb69ffff700 1 --2- 172.21.15.31:0/2223082955 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6a00c8330 0x7fb6a016df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:03.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.405+0000 7fb69ffff700 1 --2- 172.21.15.31:0/2223082955 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb68c067050 0x7fb68c069500 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:03.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.405+0000 7fb69ffff700 1 --2- 172.21.15.31:0/2223082955 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb6a0140a70 0x7fb6a0169790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:03.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.405+0000 7fb69ffff700 1 -- 172.21.15.31:0/2223082955 >> 172.21.15.31:0/2223082955 conn(0x7fb6a006d170 msgr2=0x7fb6a00c1140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:03.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.405+0000 7fb69ffff700 1 -- 172.21.15.31:0/2223082955 shutdown_connections 2023-10-06T02:28:03.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.405+0000 7fb69ffff700 1 -- 172.21.15.31:0/2223082955 wait complete. 2023-10-06T02:28:03.408 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-10-06T02:28:03.423 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2023-10-06T02:28:02.446299+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":223,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":166,"ondisk_log_size":166,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":85,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2420532,"kb_used_data":9460,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752554188,"statfs":{"total":773094113280,"available":770615488512,"internally_reserved":0,"allocated":9687040,"data_stored":5006424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.931003"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.551410+0000","last_change":"2023-10-06T02:27:58.551410+0000","last_active":"2023-10-06T02:27:58.551410+0000","last_peered":"2023-10-06T02:27:58.551410+0000","last_clean":"2023-10-06T02:27:58.551410+0000","last_became_active":"2023-10-06T02:27:58.551142+0000","last_became_peered":"2023-10-06T02:27:58.551142+0000","last_unstale":"2023-10-06T02:27:58.551410+0000","last_undegraded":"2023-10-06T02:27:58.551410+0000","last_fullsized":"2023-10-06T02:27:58.551410+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:58:12.432522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.541957+0000","last_change":"2023-10-06T02:27:58.541957+0000","last_active":"2023-10-06T02:27:58.541957+0000","last_peered":"2023-10-06T02:27:58.541957+0000","last_clean":"2023-10-06T02:27:58.541957+0000","last_became_active":"2023-10-06T02:27:58.541684+0000","last_became_peered":"2023-10-06T02:27:58.541684+0000","last_unstale":"2023-10-06T02:27:58.541957+0000","last_undegraded":"2023-10-06T02:27:58.541957+0000","last_fullsized":"2023-10-06T02:27:58.541957+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:57:24.480998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.553499+0000","last_change":"2023-10-06T02:27:58.553499+0000","last_active":"2023-10-06T02:27:58.553499+0000","last_peered":"2023-10-06T02:27:58.553499+0000","last_clean":"2023-10-06T02:27:58.553499+0000","last_became_active":"2023-10-06T02:27:58.553175+0000","last_became_peered":"2023-10-06T02:27:58.553175+0000","last_unstale":"2023-10-06T02:27:58.553499+0000","last_undegraded":"2023-10-06T02:27:58.553499+0000","last_fullsized":"2023-10-06T02:27:58.553499+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:11:13.756297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.573879+0000","last_change":"2023-10-06T02:27:58.573879+0000","last_active":"2023-10-06T02:27:58.573879+0000","last_peered":"2023-10-06T02:27:58.573879+0000","last_clean":"2023-10-06T02:27:58.573879+0000","last_became_active":"2023-10-06T02:27:58.570952+0000","last_became_peered":"2023-10-06T02:27:58.570952+0000","last_unstale":"2023-10-06T02:27:58.573879+0000","last_undegraded":"2023-10-06T02:27:58.573879+0000","last_fullsized":"2023-10-06T02:27:58.573879+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:32:14.499778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.550286+0000","last_change":"2023-10-06T02:27:58.550286+0000","last_active":"2023-10-06T02:27:58.550286+0000","last_peered":"2023-10-06T02:27:58.550286+0000","last_clean":"2023-10-06T02:27:58.550286+0000","last_became_active":"2023-10-06T02:27:58.549972+0000","last_became_peered":"2023-10-06T02:27:58.549972+0000","last_unstale":"2023-10-06T02:27:58.550286+0000","last_undegraded":"2023-10-06T02:27:58.550286+0000","last_fullsized":"2023-10-06T02:27:58.550286+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:35:48.609652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.575473+0000","last_change":"2023-10-06T02:27:58.575473+0000","last_active":"2023-10-06T02:27:58.575473+0000","last_peered":"2023-10-06T02:27:58.575473+0000","last_clean":"2023-10-06T02:27:58.575473+0000","last_became_active":"2023-10-06T02:27:58.575215+0000","last_became_peered":"2023-10-06T02:27:58.575215+0000","last_unstale":"2023-10-06T02:27:58.575473+0000","last_undegraded":"2023-10-06T02:27:58.575473+0000","last_fullsized":"2023-10-06T02:27:58.575473+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:14:39.635904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.544480+0000","last_change":"2023-10-06T02:27:58.544480+0000","last_active":"2023-10-06T02:27:58.544480+0000","last_peered":"2023-10-06T02:27:58.544480+0000","last_clean":"2023-10-06T02:27:58.544480+0000","last_became_active":"2023-10-06T02:27:58.544143+0000","last_became_peered":"2023-10-06T02:27:58.544143+0000","last_unstale":"2023-10-06T02:27:58.544480+0000","last_undegraded":"2023-10-06T02:27:58.544480+0000","last_fullsized":"2023-10-06T02:27:58.544480+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:40.877351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.571026+0000","last_change":"2023-10-06T02:27:58.571026+0000","last_active":"2023-10-06T02:27:58.571026+0000","last_peered":"2023-10-06T02:27:58.571026+0000","last_clean":"2023-10-06T02:27:58.571026+0000","last_became_active":"2023-10-06T02:27:58.568207+0000","last_became_peered":"2023-10-06T02:27:58.568207+0000","last_unstale":"2023-10-06T02:27:58.571026+0000","last_undegraded":"2023-10-06T02:27:58.571026+0000","last_fullsized":"2023-10-06T02:27:58.571026+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:26:25.959100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.550347+0000","last_change":"2023-10-06T02:27:58.550347+0000","last_active":"2023-10-06T02:27:58.550347+0000","last_peered":"2023-10-06T02:27:58.550347+0000","last_clean":"2023-10-06T02:27:58.550347+0000","last_became_active":"2023-10-06T02:27:58.549949+0000","last_became_peered":"2023-10-06T02:27:58.549949+0000","last_unstale":"2023-10-06T02:27:58.550347+0000","last_undegraded":"2023-10-06T02:27:58.550347+0000","last_fullsized":"2023-10-06T02:27:58.550347+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:57:24.480998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.567708+0000","last_change":"2023-10-06T02:27:58.567708+0000","last_active":"2023-10-06T02:27:58.567708+0000","last_peered":"2023-10-06T02:27:58.567708+0000","last_clean":"2023-10-06T02:27:58.567708+0000","last_became_active":"2023-10-06T02:27:58.567434+0000","last_became_peered":"2023-10-06T02:27:58.567434+0000","last_unstale":"2023-10-06T02:27:58.567708+0000","last_undegraded":"2023-10-06T02:27:58.567708+0000","last_fullsized":"2023-10-06T02:27:58.567708+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:31:19.451780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.552709+0000","last_change":"2023-10-06T02:27:58.552709+0000","last_active":"2023-10-06T02:27:58.552709+0000","last_peered":"2023-10-06T02:27:58.552709+0000","last_clean":"2023-10-06T02:27:58.552709+0000","last_became_active":"2023-10-06T02:27:58.552343+0000","last_became_peered":"2023-10-06T02:27:58.552343+0000","last_unstale":"2023-10-06T02:27:58.552709+0000","last_undegraded":"2023-10-06T02:27:58.552709+0000","last_fullsized":"2023-10-06T02:27:58.552709+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:21:25.801489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.542603+0000","last_change":"2023-10-06T02:27:58.542603+0000","last_active":"2023-10-06T02:27:58.542603+0000","last_peered":"2023-10-06T02:27:58.542603+0000","last_clean":"2023-10-06T02:27:58.542603+0000","last_became_active":"2023-10-06T02:27:58.542285+0000","last_became_peered":"2023-10-06T02:27:58.542285+0000","last_unstale":"2023-10-06T02:27:58.542603+0000","last_undegraded":"2023-10-06T02:27:58.542603+0000","last_fullsized":"2023-10-06T02:27:58.542603+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:42:16.586315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.573766+0000","last_change":"2023-10-06T02:27:58.573766+0000","last_active":"2023-10-06T02:27:58.573766+0000","last_peered":"2023-10-06T02:27:58.573766+0000","last_clean":"2023-10-06T02:27:58.573766+0000","last_became_active":"2023-10-06T02:27:58.570955+0000","last_became_peered":"2023-10-06T02:27:58.570955+0000","last_unstale":"2023-10-06T02:27:58.573766+0000","last_undegraded":"2023-10-06T02:27:58.573766+0000","last_fullsized":"2023-10-06T02:27:58.573766+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:44:04.286470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.572808+0000","last_change":"2023-10-06T02:27:58.572808+0000","last_active":"2023-10-06T02:27:58.572808+0000","last_peered":"2023-10-06T02:27:58.572808+0000","last_clean":"2023-10-06T02:27:58.572808+0000","last_became_active":"2023-10-06T02:27:58.572489+0000","last_became_peered":"2023-10-06T02:27:58.572489+0000","last_unstale":"2023-10-06T02:27:58.572808+0000","last_undegraded":"2023-10-06T02:27:58.572808+0000","last_fullsized":"2023-10-06T02:27:58.572808+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:21:01.544058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.566916+0000","last_change":"2023-10-06T02:27:58.566916+0000","last_active":"2023-10-06T02:27:58.566916+0000","last_peered":"2023-10-06T02:27:58.566916+0000","last_clean":"2023-10-06T02:27:58.566916+0000","last_became_active":"2023-10-06T02:27:58.566351+0000","last_became_peered":"2023-10-06T02:27:58.566351+0000","last_unstale":"2023-10-06T02:27:58.566916+0000","last_undegraded":"2023-10-06T02:27:58.566916+0000","last_fullsized":"2023-10-06T02:27:58.566916+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:25:04.772572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.558709+0000","last_change":"2023-10-06T02:27:58.558709+0000","last_active":"2023-10-06T02:27:58.558709+0000","last_peered":"2023-10-06T02:27:58.558709+0000","last_clean":"2023-10-06T02:27:58.558709+0000","last_became_active":"2023-10-06T02:27:58.558435+0000","last_became_peered":"2023-10-06T02:27:58.558435+0000","last_unstale":"2023-10-06T02:27:58.558709+0000","last_undegraded":"2023-10-06T02:27:58.558709+0000","last_fullsized":"2023-10-06T02:27:58.558709+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:22:11.087710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.564983+0000","last_change":"2023-10-06T02:27:58.564983+0000","last_active":"2023-10-06T02:27:58.564983+0000","last_peered":"2023-10-06T02:27:58.564983+0000","last_clean":"2023-10-06T02:27:58.564983+0000","last_became_active":"2023-10-06T02:27:58.564545+0000","last_became_peered":"2023-10-06T02:27:58.564545+0000","last_unstale":"2023-10-06T02:27:58.564983+0000","last_undegraded":"2023-10-06T02:27:58.564983+0000","last_fullsized":"2023-10-06T02:27:58.564983+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:15:39.374928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.550551+0000","last_change":"2023-10-06T02:27:58.550551+0000","last_active":"2023-10-06T02:27:58.550551+0000","last_peered":"2023-10-06T02:27:58.550551+0000","last_clean":"2023-10-06T02:27:58.550551+0000","last_became_active":"2023-10-06T02:27:58.550244+0000","last_became_peered":"2023-10-06T02:27:58.550244+0000","last_unstale":"2023-10-06T02:27:58.550551+0000","last_undegraded":"2023-10-06T02:27:58.550551+0000","last_fullsized":"2023-10-06T02:27:58.550551+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:44:30.929189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.567627+0000","last_change":"2023-10-06T02:27:58.567627+0000","last_active":"2023-10-06T02:27:58.567627+0000","last_peered":"2023-10-06T02:27:58.567627+0000","last_clean":"2023-10-06T02:27:58.567627+0000","last_became_active":"2023-10-06T02:27:58.567277+0000","last_became_peered":"2023-10-06T02:27:58.567277+0000","last_unstale":"2023-10-06T02:27:58.567627+0000","last_undegraded":"2023-10-06T02:27:58.567627+0000","last_fullsized":"2023-10-06T02:27:58.567627+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:04:30.074206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.571310+0000","last_change":"2023-10-06T02:27:58.571310+0000","last_active":"2023-10-06T02:27:58.571310+0000","last_peered":"2023-10-06T02:27:58.571310+0000","last_clean":"2023-10-06T02:27:58.571310+0000","last_became_active":"2023-10-06T02:27:58.570409+0000","last_became_peered":"2023-10-06T02:27:58.570409+0000","last_unstale":"2023-10-06T02:27:58.571310+0000","last_undegraded":"2023-10-06T02:27:58.571310+0000","last_fullsized":"2023-10-06T02:27:58.571310+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:12:31.189846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.549040+0000","last_change":"2023-10-06T02:27:58.549040+0000","last_active":"2023-10-06T02:27:58.549040+0000","last_peered":"2023-10-06T02:27:58.549040+0000","last_clean":"2023-10-06T02:27:58.549040+0000","last_became_active":"2023-10-06T02:27:58.548701+0000","last_became_peered":"2023-10-06T02:27:58.548701+0000","last_unstale":"2023-10-06T02:27:58.549040+0000","last_undegraded":"2023-10-06T02:27:58.549040+0000","last_fullsized":"2023-10-06T02:27:58.549040+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:02:49.666684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.572173+0000","last_change":"2023-10-06T02:27:58.572173+0000","last_active":"2023-10-06T02:27:58.572173+0000","last_peered":"2023-10-06T02:27:58.572173+0000","last_clean":"2023-10-06T02:27:58.572173+0000","last_became_active":"2023-10-06T02:27:58.571683+0000","last_became_peered":"2023-10-06T02:27:58.571683+0000","last_unstale":"2023-10-06T02:27:58.572173+0000","last_undegraded":"2023-10-06T02:27:58.572173+0000","last_fullsized":"2023-10-06T02:27:58.572173+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:18.422648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.574656+0000","last_change":"2023-10-06T02:27:58.574656+0000","last_active":"2023-10-06T02:27:58.574656+0000","last_peered":"2023-10-06T02:27:58.574656+0000","last_clean":"2023-10-06T02:27:58.574656+0000","last_became_active":"2023-10-06T02:27:58.574342+0000","last_became_peered":"2023-10-06T02:27:58.574342+0000","last_unstale":"2023-10-06T02:27:58.574656+0000","last_undegraded":"2023-10-06T02:27:58.574656+0000","last_fullsized":"2023-10-06T02:27:58.574656+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:28:52.562249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.552789+0000","last_change":"2023-10-06T02:27:58.552789+0000","last_active":"2023-10-06T02:27:58.552789+0000","last_peered":"2023-10-06T02:27:58.552789+0000","last_clean":"2023-10-06T02:27:58.552789+0000","last_became_active":"2023-10-06T02:27:58.552598+0000","last_became_peered":"2023-10-06T02:27:58.552598+0000","last_unstale":"2023-10-06T02:27:58.552789+0000","last_undegraded":"2023-10-06T02:27:58.552789+0000","last_fullsized":"2023-10-06T02:27:58.552789+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:04:30.074206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.540391+0000","last_change":"2023-10-06T02:27:58.540391+0000","last_active":"2023-10-06T02:27:58.540391+0000","last_peered":"2023-10-06T02:27:58.540391+0000","last_clean":"2023-10-06T02:27:58.540391+0000","last_became_active":"2023-10-06T02:27:58.540065+0000","last_became_peered":"2023-10-06T02:27:58.540065+0000","last_unstale":"2023-10-06T02:27:58.540391+0000","last_undegraded":"2023-10-06T02:27:58.540391+0000","last_fullsized":"2023-10-06T02:27:58.540391+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:30:19.997629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.567831+0000","last_change":"2023-10-06T02:27:58.567831+0000","last_active":"2023-10-06T02:27:58.567831+0000","last_peered":"2023-10-06T02:27:58.567831+0000","last_clean":"2023-10-06T02:27:58.567831+0000","last_became_active":"2023-10-06T02:27:58.567279+0000","last_became_peered":"2023-10-06T02:27:58.567279+0000","last_unstale":"2023-10-06T02:27:58.567831+0000","last_undegraded":"2023-10-06T02:27:58.567831+0000","last_fullsized":"2023-10-06T02:27:58.567831+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:02:41.309445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.538551+0000","last_change":"2023-10-06T02:27:58.538551+0000","last_active":"2023-10-06T02:27:58.538551+0000","last_peered":"2023-10-06T02:27:58.538551+0000","last_clean":"2023-10-06T02:27:58.538551+0000","last_became_active":"2023-10-06T02:27:58.537888+0000","last_became_peered":"2023-10-06T02:27:58.537888+0000","last_unstale":"2023-10-06T02:27:58.538551+0000","last_undegraded":"2023-10-06T02:27:58.538551+0000","last_fullsized":"2023-10-06T02:27:58.538551+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:38:35.027757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.574430+0000","last_change":"2023-10-06T02:27:58.574430+0000","last_active":"2023-10-06T02:27:58.574430+0000","last_peered":"2023-10-06T02:27:58.574430+0000","last_clean":"2023-10-06T02:27:58.574430+0000","last_became_active":"2023-10-06T02:27:58.574121+0000","last_became_peered":"2023-10-06T02:27:58.574121+0000","last_unstale":"2023-10-06T02:27:58.574430+0000","last_undegraded":"2023-10-06T02:27:58.574430+0000","last_fullsized":"2023-10-06T02:27:58.574430+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:46:55.335987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.553727+0000","last_change":"2023-10-06T02:27:58.553727+0000","last_active":"2023-10-06T02:27:58.553727+0000","last_peered":"2023-10-06T02:27:58.553727+0000","last_clean":"2023-10-06T02:27:58.553727+0000","last_became_active":"2023-10-06T02:27:58.553127+0000","last_became_peered":"2023-10-06T02:27:58.553127+0000","last_unstale":"2023-10-06T02:27:58.553727+0000","last_undegraded":"2023-10-06T02:27:58.553727+0000","last_fullsized":"2023-10-06T02:27:58.553727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:34.381441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.550149+0000","last_change":"2023-10-06T02:27:58.550149+0000","last_active":"2023-10-06T02:27:58.550149+0000","last_peered":"2023-10-06T02:27:58.550149+0000","last_clean":"2023-10-06T02:27:58.550149+0000","last_became_active":"2023-10-06T02:27:58.549830+0000","last_became_peered":"2023-10-06T02:27:58.549830+0000","last_unstale":"2023-10-06T02:27:58.550149+0000","last_undegraded":"2023-10-06T02:27:58.550149+0000","last_fullsized":"2023-10-06T02:27:58.550149+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:42:10.237196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.567724+0000","last_change":"2023-10-06T02:27:58.567724+0000","last_active":"2023-10-06T02:27:58.567724+0000","last_peered":"2023-10-06T02:27:58.567724+0000","last_clean":"2023-10-06T02:27:58.567724+0000","last_became_active":"2023-10-06T02:27:58.567318+0000","last_became_peered":"2023-10-06T02:27:58.567318+0000","last_unstale":"2023-10-06T02:27:58.567724+0000","last_undegraded":"2023-10-06T02:27:58.567724+0000","last_fullsized":"2023-10-06T02:27:58.567724+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:07:32.016889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.558107+0000","last_change":"2023-10-06T02:27:58.558107+0000","last_active":"2023-10-06T02:27:58.558107+0000","last_peered":"2023-10-06T02:27:58.558107+0000","last_clean":"2023-10-06T02:27:58.558107+0000","last_became_active":"2023-10-06T02:27:58.557841+0000","last_became_peered":"2023-10-06T02:27:58.557841+0000","last_unstale":"2023-10-06T02:27:58.558107+0000","last_undegraded":"2023-10-06T02:27:58.558107+0000","last_fullsized":"2023-10-06T02:27:58.558107+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:06:50.626617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.553912+0000","last_change":"2023-10-06T02:27:58.553912+0000","last_active":"2023-10-06T02:27:58.553912+0000","last_peered":"2023-10-06T02:27:58.553912+0000","last_clean":"2023-10-06T02:27:58.553912+0000","last_became_active":"2023-10-06T02:27:58.553648+0000","last_became_peered":"2023-10-06T02:27:58.553648+0000","last_unstale":"2023-10-06T02:27:58.553912+0000","last_undegraded":"2023-10-06T02:27:58.553912+0000","last_fullsized":"2023-10-06T02:27:58.553912+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:53:11.885304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"86'4","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.570566+0000","last_change":"2023-10-06T02:27:57.547688+0000","last_active":"2023-10-06T02:27:58.570566+0000","last_peered":"2023-10-06T02:27:58.570566+0000","last_clean":"2023-10-06T02:27:58.570566+0000","last_became_active":"2023-10-06T02:27:57.547379+0000","last_became_peered":"2023-10-06T02:27:57.547379+0000","last_unstale":"2023-10-06T02:27:58.570566+0000","last_undegraded":"2023-10-06T02:27:58.570566+0000","last_fullsized":"2023-10-06T02:27:58.570566+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_clean_scrub_stamp":"2023-10-06T02:27:56.508109+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:05:11.061891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.554871+0000","last_change":"2023-10-06T02:27:58.554871+0000","last_active":"2023-10-06T02:27:58.554871+0000","last_peered":"2023-10-06T02:27:58.554871+0000","last_clean":"2023-10-06T02:27:58.554871+0000","last_became_active":"2023-10-06T02:27:58.554612+0000","last_became_peered":"2023-10-06T02:27:58.554612+0000","last_unstale":"2023-10-06T02:27:58.554871+0000","last_undegraded":"2023-10-06T02:27:58.554871+0000","last_fullsized":"2023-10-06T02:27:58.554871+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:03:01.765756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"86'1","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.543955+0000","last_change":"2023-10-06T02:27:57.543841+0000","last_active":"2023-10-06T02:27:58.543955+0000","last_peered":"2023-10-06T02:27:58.543955+0000","last_clean":"2023-10-06T02:27:58.543955+0000","last_became_active":"2023-10-06T02:27:57.543501+0000","last_became_peered":"2023-10-06T02:27:57.543501+0000","last_unstale":"2023-10-06T02:27:58.543955+0000","last_undegraded":"2023-10-06T02:27:58.543955+0000","last_fullsized":"2023-10-06T02:27:58.543955+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_clean_scrub_stamp":"2023-10-06T02:27:56.508109+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:41:38.011422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.574352+0000","last_change":"2023-10-06T02:27:58.574352+0000","last_active":"2023-10-06T02:27:58.574352+0000","last_peered":"2023-10-06T02:27:58.574352+0000","last_clean":"2023-10-06T02:27:58.574352+0000","last_became_active":"2023-10-06T02:27:58.573988+0000","last_became_peered":"2023-10-06T02:27:58.573988+0000","last_unstale":"2023-10-06T02:27:58.574352+0000","last_undegraded":"2023-10-06T02:27:58.574352+0000","last_fullsized":"2023-10-06T02:27:58.574352+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:38.699087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"86'2","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.564823+0000","last_change":"2023-10-06T02:27:57.538863+0000","last_active":"2023-10-06T02:27:58.564823+0000","last_peered":"2023-10-06T02:27:58.564823+0000","last_clean":"2023-10-06T02:27:58.564823+0000","last_became_active":"2023-10-06T02:27:57.538560+0000","last_became_peered":"2023-10-06T02:27:57.538560+0000","last_unstale":"2023-10-06T02:27:58.564823+0000","last_undegraded":"2023-10-06T02:27:58.564823+0000","last_fullsized":"2023-10-06T02:27:58.564823+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_clean_scrub_stamp":"2023-10-06T02:27:56.508109+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:26:02.862746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.554262+0000","last_change":"2023-10-06T02:27:58.554262+0000","last_active":"2023-10-06T02:27:58.554262+0000","last_peered":"2023-10-06T02:27:58.554262+0000","last_clean":"2023-10-06T02:27:58.554262+0000","last_became_active":"2023-10-06T02:27:58.554001+0000","last_became_peered":"2023-10-06T02:27:58.554001+0000","last_unstale":"2023-10-06T02:27:58.554262+0000","last_undegraded":"2023-10-06T02:27:58.554262+0000","last_fullsized":"2023-10-06T02:27:58.554262+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:14:41.722039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"86'2","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.574647+0000","last_change":"2023-10-06T02:27:57.539775+0000","last_active":"2023-10-06T02:27:58.574647+0000","last_peered":"2023-10-06T02:27:58.574647+0000","last_clean":"2023-10-06T02:27:58.574647+0000","last_became_active":"2023-10-06T02:27:57.539510+0000","last_became_peered":"2023-10-06T02:27:57.539510+0000","last_unstale":"2023-10-06T02:27:58.574647+0000","last_undegraded":"2023-10-06T02:27:58.574647+0000","last_fullsized":"2023-10-06T02:27:58.574647+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_clean_scrub_stamp":"2023-10-06T02:27:56.508109+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:54:49.207902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.574548+0000","last_change":"2023-10-06T02:27:58.574548+0000","last_active":"2023-10-06T02:27:58.574548+0000","last_peered":"2023-10-06T02:27:58.574548+0000","last_clean":"2023-10-06T02:27:58.574548+0000","last_became_active":"2023-10-06T02:27:58.571079+0000","last_became_peered":"2023-10-06T02:27:58.571079+0000","last_unstale":"2023-10-06T02:27:58.574548+0000","last_undegraded":"2023-10-06T02:27:58.574548+0000","last_fullsized":"2023-10-06T02:27:58.574548+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:05:14.081035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"86'1","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.543703+0000","last_change":"2023-10-06T02:27:57.543564+0000","last_active":"2023-10-06T02:27:58.543703+0000","last_peered":"2023-10-06T02:27:58.543703+0000","last_clean":"2023-10-06T02:27:58.543703+0000","last_became_active":"2023-10-06T02:27:57.543262+0000","last_became_peered":"2023-10-06T02:27:57.543262+0000","last_unstale":"2023-10-06T02:27:58.543703+0000","last_undegraded":"2023-10-06T02:27:58.543703+0000","last_fullsized":"2023-10-06T02:27:58.543703+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_clean_scrub_stamp":"2023-10-06T02:27:56.508109+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:54:49.207902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.542373+0000","last_change":"2023-10-06T02:27:58.542373+0000","last_active":"2023-10-06T02:27:58.542373+0000","last_peered":"2023-10-06T02:27:58.542373+0000","last_clean":"2023-10-06T02:27:58.542373+0000","last_became_active":"2023-10-06T02:27:58.542089+0000","last_became_peered":"2023-10-06T02:27:58.542089+0000","last_unstale":"2023-10-06T02:27:58.542373+0000","last_undegraded":"2023-10-06T02:27:58.542373+0000","last_fullsized":"2023-10-06T02:27:58.542373+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:44:04.286470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"86'1","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.554560+0000","last_change":"2023-10-06T02:27:57.533458+0000","last_active":"2023-10-06T02:27:58.554560+0000","last_peered":"2023-10-06T02:27:58.554560+0000","last_clean":"2023-10-06T02:27:58.554560+0000","last_became_active":"2023-10-06T02:27:57.533167+0000","last_became_peered":"2023-10-06T02:27:57.533167+0000","last_unstale":"2023-10-06T02:27:58.554560+0000","last_undegraded":"2023-10-06T02:27:58.554560+0000","last_fullsized":"2023-10-06T02:27:58.554560+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_clean_scrub_stamp":"2023-10-06T02:27:56.508109+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:37:41.434589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.574834+0000","last_change":"2023-10-06T02:27:58.574834+0000","last_active":"2023-10-06T02:27:58.574834+0000","last_peered":"2023-10-06T02:27:58.574834+0000","last_clean":"2023-10-06T02:27:58.574834+0000","last_became_active":"2023-10-06T02:27:58.574555+0000","last_became_peered":"2023-10-06T02:27:58.574555+0000","last_unstale":"2023-10-06T02:27:58.574834+0000","last_undegraded":"2023-10-06T02:27:58.574834+0000","last_fullsized":"2023-10-06T02:27:58.574834+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:16:33.281287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T02:27:57.542533+0000","last_change":"2023-10-06T02:27:57.535979+0000","last_active":"2023-10-06T02:27:57.542533+0000","last_peered":"2023-10-06T02:27:57.542533+0000","last_clean":"2023-10-06T02:27:57.542533+0000","last_became_active":"2023-10-06T02:27:57.535715+0000","last_became_peered":"2023-10-06T02:27:57.535715+0000","last_unstale":"2023-10-06T02:27:57.542533+0000","last_undegraded":"2023-10-06T02:27:57.542533+0000","last_fullsized":"2023-10-06T02:27:57.542533+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_clean_scrub_stamp":"2023-10-06T02:27:56.508109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:02:48.660542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.570983+0000","last_change":"2023-10-06T02:27:58.570983+0000","last_active":"2023-10-06T02:27:58.570983+0000","last_peered":"2023-10-06T02:27:58.570983+0000","last_clean":"2023-10-06T02:27:58.570983+0000","last_became_active":"2023-10-06T02:27:58.570242+0000","last_became_peered":"2023-10-06T02:27:58.570242+0000","last_unstale":"2023-10-06T02:27:58.570983+0000","last_undegraded":"2023-10-06T02:27:58.570983+0000","last_fullsized":"2023-10-06T02:27:58.570983+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:28:52.562249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T02:27:57.549480+0000","last_change":"2023-10-06T02:27:57.546199+0000","last_active":"2023-10-06T02:27:57.549480+0000","last_peered":"2023-10-06T02:27:57.549480+0000","last_clean":"2023-10-06T02:27:57.549480+0000","last_became_active":"2023-10-06T02:27:57.545896+0000","last_became_peered":"2023-10-06T02:27:57.545896+0000","last_unstale":"2023-10-06T02:27:57.549480+0000","last_undegraded":"2023-10-06T02:27:57.549480+0000","last_fullsized":"2023-10-06T02:27:57.549480+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_clean_scrub_stamp":"2023-10-06T02:27:56.508109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:21:00.537917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.551071+0000","last_change":"2023-10-06T02:27:58.551071+0000","last_active":"2023-10-06T02:27:58.551071+0000","last_peered":"2023-10-06T02:27:58.551071+0000","last_clean":"2023-10-06T02:27:58.551071+0000","last_became_active":"2023-10-06T02:27:58.550648+0000","last_became_peered":"2023-10-06T02:27:58.550648+0000","last_unstale":"2023-10-06T02:27:58.551071+0000","last_undegraded":"2023-10-06T02:27:58.551071+0000","last_fullsized":"2023-10-06T02:27:58.551071+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:56:56.716677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T02:27:57.535984+0000","last_change":"2023-10-06T02:27:57.531200+0000","last_active":"2023-10-06T02:27:57.535984+0000","last_peered":"2023-10-06T02:27:57.535984+0000","last_clean":"2023-10-06T02:27:57.535984+0000","last_became_active":"2023-10-06T02:27:57.530938+0000","last_became_peered":"2023-10-06T02:27:57.530938+0000","last_unstale":"2023-10-06T02:27:57.535984+0000","last_undegraded":"2023-10-06T02:27:57.535984+0000","last_fullsized":"2023-10-06T02:27:57.535984+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_clean_scrub_stamp":"2023-10-06T02:27:56.508109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:18:09.904994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.552268+0000","last_change":"2023-10-06T02:27:58.552268+0000","last_active":"2023-10-06T02:27:58.552268+0000","last_peered":"2023-10-06T02:27:58.552268+0000","last_clean":"2023-10-06T02:27:58.552268+0000","last_became_active":"2023-10-06T02:27:58.551984+0000","last_became_peered":"2023-10-06T02:27:58.551984+0000","last_unstale":"2023-10-06T02:27:58.552268+0000","last_undegraded":"2023-10-06T02:27:58.552268+0000","last_fullsized":"2023-10-06T02:27:58.552268+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:11:41.153641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T02:27:57.546221+0000","last_change":"2023-10-06T02:27:57.545173+0000","last_active":"2023-10-06T02:27:57.546221+0000","last_peered":"2023-10-06T02:27:57.546221+0000","last_clean":"2023-10-06T02:27:57.546221+0000","last_became_active":"2023-10-06T02:27:57.544892+0000","last_became_peered":"2023-10-06T02:27:57.544892+0000","last_unstale":"2023-10-06T02:27:57.546221+0000","last_undegraded":"2023-10-06T02:27:57.546221+0000","last_fullsized":"2023-10-06T02:27:57.546221+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_clean_scrub_stamp":"2023-10-06T02:27:56.508109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:07:50.800408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.554352+0000","last_change":"2023-10-06T02:27:58.554352+0000","last_active":"2023-10-06T02:27:58.554352+0000","last_peered":"2023-10-06T02:27:58.554352+0000","last_clean":"2023-10-06T02:27:58.554352+0000","last_became_active":"2023-10-06T02:27:58.554100+0000","last_became_peered":"2023-10-06T02:27:58.554100+0000","last_unstale":"2023-10-06T02:27:58.554352+0000","last_undegraded":"2023-10-06T02:27:58.554352+0000","last_fullsized":"2023-10-06T02:27:58.554352+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:54:43.471071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"86'3","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.566090+0000","last_change":"2023-10-06T02:27:57.546292+0000","last_active":"2023-10-06T02:27:58.566090+0000","last_peered":"2023-10-06T02:27:58.566090+0000","last_clean":"2023-10-06T02:27:58.566090+0000","last_became_active":"2023-10-06T02:27:57.546002+0000","last_became_peered":"2023-10-06T02:27:57.546002+0000","last_unstale":"2023-10-06T02:27:58.566090+0000","last_undegraded":"2023-10-06T02:27:58.566090+0000","last_fullsized":"2023-10-06T02:27:58.566090+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_clean_scrub_stamp":"2023-10-06T02:27:56.508109+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:14:32.484064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.567826+0000","last_change":"2023-10-06T02:27:58.567826+0000","last_active":"2023-10-06T02:27:58.567826+0000","last_peered":"2023-10-06T02:27:58.567826+0000","last_clean":"2023-10-06T02:27:58.567826+0000","last_became_active":"2023-10-06T02:27:58.567543+0000","last_became_peered":"2023-10-06T02:27:58.567543+0000","last_unstale":"2023-10-06T02:27:58.567826+0000","last_undegraded":"2023-10-06T02:27:58.567826+0000","last_fullsized":"2023-10-06T02:27:58.567826+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:14:33.490205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"86'6","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.570308+0000","last_change":"2023-10-06T02:27:57.547624+0000","last_active":"2023-10-06T02:27:58.570308+0000","last_peered":"2023-10-06T02:27:58.570308+0000","last_clean":"2023-10-06T02:27:58.570308+0000","last_became_active":"2023-10-06T02:27:57.547270+0000","last_became_peered":"2023-10-06T02:27:57.547270+0000","last_unstale":"2023-10-06T02:27:58.570308+0000","last_undegraded":"2023-10-06T02:27:58.570308+0000","last_fullsized":"2023-10-06T02:27:58.570308+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_clean_scrub_stamp":"2023-10-06T02:27:56.508109+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:35:47.603511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.566548+0000","last_change":"2023-10-06T02:27:58.566548+0000","last_active":"2023-10-06T02:27:58.566548+0000","last_peered":"2023-10-06T02:27:58.566548+0000","last_clean":"2023-10-06T02:27:58.566548+0000","last_became_active":"2023-10-06T02:27:58.563290+0000","last_became_peered":"2023-10-06T02:27:58.563290+0000","last_unstale":"2023-10-06T02:27:58.566548+0000","last_undegraded":"2023-10-06T02:27:58.566548+0000","last_fullsized":"2023-10-06T02:27:58.566548+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:28:48.424621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"86'2","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.570550+0000","last_change":"2023-10-06T02:27:57.546517+0000","last_active":"2023-10-06T02:27:58.570550+0000","last_peered":"2023-10-06T02:27:58.570550+0000","last_clean":"2023-10-06T02:27:58.570550+0000","last_became_active":"2023-10-06T02:27:57.545890+0000","last_became_peered":"2023-10-06T02:27:57.545890+0000","last_unstale":"2023-10-06T02:27:58.570550+0000","last_undegraded":"2023-10-06T02:27:58.570550+0000","last_fullsized":"2023-10-06T02:27:58.570550+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_clean_scrub_stamp":"2023-10-06T02:27:56.508109+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:17.416507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.550930+0000","last_change":"2023-10-06T02:27:58.550930+0000","last_active":"2023-10-06T02:27:58.550930+0000","last_peered":"2023-10-06T02:27:58.550930+0000","last_clean":"2023-10-06T02:27:58.550930+0000","last_became_active":"2023-10-06T02:27:58.550573+0000","last_became_peered":"2023-10-06T02:27:58.550573+0000","last_unstale":"2023-10-06T02:27:58.550930+0000","last_undegraded":"2023-10-06T02:27:58.550930+0000","last_fullsized":"2023-10-06T02:27:58.550930+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:26:03.868887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T02:27:57.546037+0000","last_change":"2023-10-06T02:27:57.544404+0000","last_active":"2023-10-06T02:27:57.546037+0000","last_peered":"2023-10-06T02:27:57.546037+0000","last_clean":"2023-10-06T02:27:57.546037+0000","last_became_active":"2023-10-06T02:27:57.543953+0000","last_became_peered":"2023-10-06T02:27:57.543953+0000","last_unstale":"2023-10-06T02:27:57.546037+0000","last_undegraded":"2023-10-06T02:27:57.546037+0000","last_fullsized":"2023-10-06T02:27:57.546037+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_clean_scrub_stamp":"2023-10-06T02:27:56.508109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:21:24.795347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.574633+0000","last_change":"2023-10-06T02:27:58.574633+0000","last_active":"2023-10-06T02:27:58.574633+0000","last_peered":"2023-10-06T02:27:58.574633+0000","last_clean":"2023-10-06T02:27:58.574633+0000","last_became_active":"2023-10-06T02:27:58.574359+0000","last_became_peered":"2023-10-06T02:27:58.574359+0000","last_unstale":"2023-10-06T02:27:58.574633+0000","last_undegraded":"2023-10-06T02:27:58.574633+0000","last_fullsized":"2023-10-06T02:27:58.574633+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:48:26.891158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T02:27:57.544556+0000","last_change":"2023-10-06T02:27:57.540372+0000","last_active":"2023-10-06T02:27:57.544556+0000","last_peered":"2023-10-06T02:27:57.544556+0000","last_clean":"2023-10-06T02:27:57.544556+0000","last_became_active":"2023-10-06T02:27:57.539896+0000","last_became_peered":"2023-10-06T02:27:57.539896+0000","last_unstale":"2023-10-06T02:27:57.544556+0000","last_undegraded":"2023-10-06T02:27:57.544556+0000","last_fullsized":"2023-10-06T02:27:57.544556+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_clean_scrub_stamp":"2023-10-06T02:27:56.508109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:53:38.077483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.551889+0000","last_change":"2023-10-06T02:27:58.551889+0000","last_active":"2023-10-06T02:27:58.551889+0000","last_peered":"2023-10-06T02:27:58.551889+0000","last_clean":"2023-10-06T02:27:58.551889+0000","last_became_active":"2023-10-06T02:27:58.551581+0000","last_became_peered":"2023-10-06T02:27:58.551581+0000","last_unstale":"2023-10-06T02:27:58.551889+0000","last_undegraded":"2023-10-06T02:27:58.551889+0000","last_fullsized":"2023-10-06T02:27:58.551889+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:43:13.376099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"86'8","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.578037+0000","last_change":"2023-10-06T02:27:57.533899+0000","last_active":"2023-10-06T02:27:58.578037+0000","last_peered":"2023-10-06T02:27:58.578037+0000","last_clean":"2023-10-06T02:27:58.578037+0000","last_became_active":"2023-10-06T02:27:57.533566+0000","last_became_peered":"2023-10-06T02:27:57.533566+0000","last_unstale":"2023-10-06T02:27:58.578037+0000","last_undegraded":"2023-10-06T02:27:58.578037+0000","last_fullsized":"2023-10-06T02:27:58.578037+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_clean_scrub_stamp":"2023-10-06T02:27:56.508109+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:33:21.693794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.543558+0000","last_change":"2023-10-06T02:27:58.543558+0000","last_active":"2023-10-06T02:27:58.543558+0000","last_peered":"2023-10-06T02:27:58.543558+0000","last_clean":"2023-10-06T02:27:58.543558+0000","last_became_active":"2023-10-06T02:27:58.543268+0000","last_became_peered":"2023-10-06T02:27:58.543268+0000","last_unstale":"2023-10-06T02:27:58.543558+0000","last_undegraded":"2023-10-06T02:27:58.543558+0000","last_fullsized":"2023-10-06T02:27:58.543558+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:28:48.424621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"86'2","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.565708+0000","last_change":"2023-10-06T02:27:57.543149+0000","last_active":"2023-10-06T02:27:58.565708+0000","last_peered":"2023-10-06T02:27:58.565708+0000","last_clean":"2023-10-06T02:27:58.565708+0000","last_became_active":"2023-10-06T02:27:57.542741+0000","last_became_peered":"2023-10-06T02:27:57.542741+0000","last_unstale":"2023-10-06T02:27:58.565708+0000","last_undegraded":"2023-10-06T02:27:58.565708+0000","last_fullsized":"2023-10-06T02:27:58.565708+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_clean_scrub_stamp":"2023-10-06T02:27:56.508109+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:35:47.603511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.557007+0000","last_change":"2023-10-06T02:27:58.557007+0000","last_active":"2023-10-06T02:27:58.557007+0000","last_peered":"2023-10-06T02:27:58.557007+0000","last_clean":"2023-10-06T02:27:58.557007+0000","last_became_active":"2023-10-06T02:27:58.556451+0000","last_became_peered":"2023-10-06T02:27:58.556451+0000","last_unstale":"2023-10-06T02:27:58.557007+0000","last_undegraded":"2023-10-06T02:27:58.557007+0000","last_fullsized":"2023-10-06T02:27:58.557007+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:05:59.590954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T02:27:57.550749+0000","last_change":"2023-10-06T02:27:57.544358+0000","last_active":"2023-10-06T02:27:57.550749+0000","last_peered":"2023-10-06T02:27:57.550749+0000","last_clean":"2023-10-06T02:27:57.550749+0000","last_became_active":"2023-10-06T02:27:57.543270+0000","last_became_peered":"2023-10-06T02:27:57.543270+0000","last_unstale":"2023-10-06T02:27:57.550749+0000","last_undegraded":"2023-10-06T02:27:57.550749+0000","last_fullsized":"2023-10-06T02:27:57.550749+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_clean_scrub_stamp":"2023-10-06T02:27:56.508109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:09:46.127873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.551391+0000","last_change":"2023-10-06T02:27:58.551391+0000","last_active":"2023-10-06T02:27:58.551391+0000","last_peered":"2023-10-06T02:27:58.551391+0000","last_clean":"2023-10-06T02:27:58.551391+0000","last_became_active":"2023-10-06T02:27:58.551003+0000","last_became_peered":"2023-10-06T02:27:58.551003+0000","last_unstale":"2023-10-06T02:27:58.551391+0000","last_undegraded":"2023-10-06T02:27:58.551391+0000","last_fullsized":"2023-10-06T02:27:58.551391+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:34.381441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T02:27:57.550797+0000","last_change":"2023-10-06T02:27:57.546259+0000","last_active":"2023-10-06T02:27:57.550797+0000","last_peered":"2023-10-06T02:27:57.550797+0000","last_clean":"2023-10-06T02:27:57.550797+0000","last_became_active":"2023-10-06T02:27:57.545938+0000","last_became_peered":"2023-10-06T02:27:57.545938+0000","last_unstale":"2023-10-06T02:27:57.550797+0000","last_undegraded":"2023-10-06T02:27:57.550797+0000","last_fullsized":"2023-10-06T02:27:57.550797+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_clean_scrub_stamp":"2023-10-06T02:27:56.508109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:18:08.720368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.553846+0000","last_change":"2023-10-06T02:27:58.553846+0000","last_active":"2023-10-06T02:27:58.553846+0000","last_peered":"2023-10-06T02:27:58.553846+0000","last_clean":"2023-10-06T02:27:58.553846+0000","last_became_active":"2023-10-06T02:27:58.553587+0000","last_became_peered":"2023-10-06T02:27:58.553587+0000","last_unstale":"2023-10-06T02:27:58.553846+0000","last_undegraded":"2023-10-06T02:27:58.553846+0000","last_fullsized":"2023-10-06T02:27:58.553846+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:42:10.237196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T02:27:57.543414+0000","last_change":"2023-10-06T02:27:57.538063+0000","last_active":"2023-10-06T02:27:57.543414+0000","last_peered":"2023-10-06T02:27:57.543414+0000","last_clean":"2023-10-06T02:27:57.543414+0000","last_became_active":"2023-10-06T02:27:57.537811+0000","last_became_peered":"2023-10-06T02:27:57.537811+0000","last_unstale":"2023-10-06T02:27:57.543414+0000","last_undegraded":"2023-10-06T02:27:57.543414+0000","last_fullsized":"2023-10-06T02:27:57.543414+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_clean_scrub_stamp":"2023-10-06T02:27:56.508109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:55.451098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.544309+0000","last_change":"2023-10-06T02:27:58.544309+0000","last_active":"2023-10-06T02:27:58.544309+0000","last_peered":"2023-10-06T02:27:58.544309+0000","last_clean":"2023-10-06T02:27:58.544309+0000","last_became_active":"2023-10-06T02:27:58.544003+0000","last_became_peered":"2023-10-06T02:27:58.544003+0000","last_unstale":"2023-10-06T02:27:58.544309+0000","last_undegraded":"2023-10-06T02:27:58.544309+0000","last_fullsized":"2023-10-06T02:27:58.544309+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:12:44.455787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T02:27:57.548560+0000","last_change":"2023-10-06T02:27:57.545258+0000","last_active":"2023-10-06T02:27:57.548560+0000","last_peered":"2023-10-06T02:27:57.548560+0000","last_clean":"2023-10-06T02:27:57.548560+0000","last_became_active":"2023-10-06T02:27:57.545000+0000","last_became_peered":"2023-10-06T02:27:57.545000+0000","last_unstale":"2023-10-06T02:27:57.548560+0000","last_undegraded":"2023-10-06T02:27:57.548560+0000","last_fullsized":"2023-10-06T02:27:57.548560+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_clean_scrub_stamp":"2023-10-06T02:27:56.508109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:17.416507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.571739+0000","last_change":"2023-10-06T02:27:58.571739+0000","last_active":"2023-10-06T02:27:58.571739+0000","last_peered":"2023-10-06T02:27:58.571739+0000","last_clean":"2023-10-06T02:27:58.571739+0000","last_became_active":"2023-10-06T02:27:58.571424+0000","last_became_peered":"2023-10-06T02:27:58.571424+0000","last_unstale":"2023-10-06T02:27:58.571739+0000","last_undegraded":"2023-10-06T02:27:58.571739+0000","last_fullsized":"2023-10-06T02:27:58.571739+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:57:49.732231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T02:27:57.541872+0000","last_change":"2023-10-06T02:27:57.539201+0000","last_active":"2023-10-06T02:27:57.541872+0000","last_peered":"2023-10-06T02:27:57.541872+0000","last_clean":"2023-10-06T02:27:57.541872+0000","last_became_active":"2023-10-06T02:27:57.538906+0000","last_became_peered":"2023-10-06T02:27:57.538906+0000","last_unstale":"2023-10-06T02:27:57.541872+0000","last_undegraded":"2023-10-06T02:27:57.541872+0000","last_fullsized":"2023-10-06T02:27:57.541872+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_clean_scrub_stamp":"2023-10-06T02:27:56.508109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:28:47.418480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.574402+0000","last_change":"2023-10-06T02:27:58.574402+0000","last_active":"2023-10-06T02:27:58.574402+0000","last_peered":"2023-10-06T02:27:58.574402+0000","last_clean":"2023-10-06T02:27:58.574402+0000","last_became_active":"2023-10-06T02:27:58.574104+0000","last_became_peered":"2023-10-06T02:27:58.574104+0000","last_unstale":"2023-10-06T02:27:58.574402+0000","last_undegraded":"2023-10-06T02:27:58.574402+0000","last_fullsized":"2023-10-06T02:27:58.574402+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:26:25.959100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T02:27:57.536877+0000","last_change":"2023-10-06T02:27:57.533126+0000","last_active":"2023-10-06T02:27:57.536877+0000","last_peered":"2023-10-06T02:27:57.536877+0000","last_clean":"2023-10-06T02:27:57.536877+0000","last_became_active":"2023-10-06T02:27:57.532738+0000","last_became_peered":"2023-10-06T02:27:57.532738+0000","last_unstale":"2023-10-06T02:27:57.536877+0000","last_undegraded":"2023-10-06T02:27:57.536877+0000","last_fullsized":"2023-10-06T02:27:57.536877+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_clean_scrub_stamp":"2023-10-06T02:27:56.508109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:46:08.570447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.560541+0000","last_change":"2023-10-06T02:27:58.560541+0000","last_active":"2023-10-06T02:27:58.560541+0000","last_peered":"2023-10-06T02:27:58.560541+0000","last_clean":"2023-10-06T02:27:58.560541+0000","last_became_active":"2023-10-06T02:27:58.559183+0000","last_became_peered":"2023-10-06T02:27:58.559183+0000","last_unstale":"2023-10-06T02:27:58.560541+0000","last_undegraded":"2023-10-06T02:27:58.560541+0000","last_fullsized":"2023-10-06T02:27:58.560541+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:02:31.507436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T02:27:57.541451+0000","last_change":"2023-10-06T02:27:57.540489+0000","last_active":"2023-10-06T02:27:57.541451+0000","last_peered":"2023-10-06T02:27:57.541451+0000","last_clean":"2023-10-06T02:27:57.541451+0000","last_became_active":"2023-10-06T02:27:57.540200+0000","last_became_peered":"2023-10-06T02:27:57.540200+0000","last_unstale":"2023-10-06T02:27:57.541451+0000","last_undegraded":"2023-10-06T02:27:57.541451+0000","last_fullsized":"2023-10-06T02:27:57.541451+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_clean_scrub_stamp":"2023-10-06T02:27:56.508109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:02:40.303304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.575695+0000","last_change":"2023-10-06T02:27:58.575695+0000","last_active":"2023-10-06T02:27:58.575695+0000","last_peered":"2023-10-06T02:27:58.575695+0000","last_clean":"2023-10-06T02:27:58.575695+0000","last_became_active":"2023-10-06T02:27:58.575412+0000","last_became_peered":"2023-10-06T02:27:58.575412+0000","last_unstale":"2023-10-06T02:27:58.575695+0000","last_undegraded":"2023-10-06T02:27:58.575695+0000","last_fullsized":"2023-10-06T02:27:58.575695+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:36:38.988969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"86'2","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.561264+0000","last_change":"2023-10-06T02:27:57.534301+0000","last_active":"2023-10-06T02:27:58.561264+0000","last_peered":"2023-10-06T02:27:58.561264+0000","last_clean":"2023-10-06T02:27:58.561264+0000","last_became_active":"2023-10-06T02:27:57.533968+0000","last_became_peered":"2023-10-06T02:27:57.533968+0000","last_unstale":"2023-10-06T02:27:58.561264+0000","last_undegraded":"2023-10-06T02:27:58.561264+0000","last_fullsized":"2023-10-06T02:27:58.561264+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_clean_scrub_stamp":"2023-10-06T02:27:56.508109+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:14:32.484064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":67,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T02:27:57.525569+0000","last_change":"2023-10-06T02:23:02.054039+0000","last_active":"2023-10-06T02:27:57.525569+0000","last_peered":"2023-10-06T02:27:57.525569+0000","last_clean":"2023-10-06T02:27:57.525569+0000","last_became_active":"2023-10-06T02:23:02.053742+0000","last_became_peered":"2023-10-06T02:23:02.053742+0000","last_unstale":"2023-10-06T02:27:57.525569+0000","last_undegraded":"2023-10-06T02:27:57.525569+0000","last_fullsized":"2023-10-06T02:27:57.525569+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:58:18.846475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.554605+0000","last_change":"2023-10-06T02:27:58.554605+0000","last_active":"2023-10-06T02:27:58.554605+0000","last_peered":"2023-10-06T02:27:58.554605+0000","last_clean":"2023-10-06T02:27:58.554605+0000","last_became_active":"2023-10-06T02:27:58.554272+0000","last_became_peered":"2023-10-06T02:27:58.554272+0000","last_unstale":"2023-10-06T02:27:58.554605+0000","last_undegraded":"2023-10-06T02:27:58.554605+0000","last_fullsized":"2023-10-06T02:27:58.554605+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:07:51.806549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"86'1","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.544277+0000","last_change":"2023-10-06T02:27:57.545157+0000","last_active":"2023-10-06T02:27:58.544277+0000","last_peered":"2023-10-06T02:27:58.544277+0000","last_clean":"2023-10-06T02:27:58.544277+0000","last_became_active":"2023-10-06T02:27:57.544943+0000","last_became_peered":"2023-10-06T02:27:57.544943+0000","last_unstale":"2023-10-06T02:27:58.544277+0000","last_undegraded":"2023-10-06T02:27:58.544277+0000","last_fullsized":"2023-10-06T02:27:58.544277+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_clean_scrub_stamp":"2023-10-06T02:27:56.508109+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:18:09.904994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":76,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T02:27:57.524279+0000","last_change":"2023-10-06T02:23:01.988631+0000","last_active":"2023-10-06T02:27:57.524279+0000","last_peered":"2023-10-06T02:27:57.524279+0000","last_clean":"2023-10-06T02:27:57.524279+0000","last_became_active":"2023-10-06T02:23:01.988303+0000","last_became_peered":"2023-10-06T02:23:01.988303+0000","last_unstale":"2023-10-06T02:27:57.524279+0000","last_undegraded":"2023-10-06T02:27:57.524279+0000","last_fullsized":"2023-10-06T02:27:57.524279+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:38.211466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.546378+0000","last_change":"2023-10-06T02:27:58.546378+0000","last_active":"2023-10-06T02:27:58.546378+0000","last_peered":"2023-10-06T02:27:58.546378+0000","last_clean":"2023-10-06T02:27:58.546378+0000","last_became_active":"2023-10-06T02:27:58.546042+0000","last_became_peered":"2023-10-06T02:27:58.546042+0000","last_unstale":"2023-10-06T02:27:58.546378+0000","last_undegraded":"2023-10-06T02:27:58.546378+0000","last_fullsized":"2023-10-06T02:27:58.546378+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:14:33.490205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"86'2","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.564695+0000","last_change":"2023-10-06T02:27:57.536782+0000","last_active":"2023-10-06T02:27:58.564695+0000","last_peered":"2023-10-06T02:27:58.564695+0000","last_clean":"2023-10-06T02:27:58.564695+0000","last_became_active":"2023-10-06T02:27:57.536488+0000","last_became_peered":"2023-10-06T02:27:57.536488+0000","last_unstale":"2023-10-06T02:27:58.564695+0000","last_undegraded":"2023-10-06T02:27:58.564695+0000","last_fullsized":"2023-10-06T02:27:58.564695+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_clean_scrub_stamp":"2023-10-06T02:27:56.508109+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:21:24.795347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":71,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T02:27:57.519904+0000","last_change":"2023-10-06T02:23:02.054338+0000","last_active":"2023-10-06T02:27:57.519904+0000","last_peered":"2023-10-06T02:27:57.519904+0000","last_clean":"2023-10-06T02:27:57.519904+0000","last_became_active":"2023-10-06T02:23:02.053991+0000","last_became_peered":"2023-10-06T02:23:02.053991+0000","last_unstale":"2023-10-06T02:27:57.519904+0000","last_undegraded":"2023-10-06T02:27:57.519904+0000","last_fullsized":"2023-10-06T02:27:57.519904+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:39:30.192838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.550264+0000","last_change":"2023-10-06T02:27:58.550264+0000","last_active":"2023-10-06T02:27:58.550264+0000","last_peered":"2023-10-06T02:27:58.550264+0000","last_clean":"2023-10-06T02:27:58.550264+0000","last_became_active":"2023-10-06T02:27:58.549968+0000","last_became_peered":"2023-10-06T02:27:58.549968+0000","last_unstale":"2023-10-06T02:27:58.550264+0000","last_undegraded":"2023-10-06T02:27:58.550264+0000","last_fullsized":"2023-10-06T02:27:58.550264+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:58:44.780229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T02:27:57.544544+0000","last_change":"2023-10-06T02:27:57.541527+0000","last_active":"2023-10-06T02:27:57.544544+0000","last_peered":"2023-10-06T02:27:57.544544+0000","last_clean":"2023-10-06T02:27:57.544544+0000","last_became_active":"2023-10-06T02:27:57.541166+0000","last_became_peered":"2023-10-06T02:27:57.541166+0000","last_unstale":"2023-10-06T02:27:57.544544+0000","last_undegraded":"2023-10-06T02:27:57.544544+0000","last_fullsized":"2023-10-06T02:27:57.544544+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_clean_scrub_stamp":"2023-10-06T02:27:56.508109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:09:43.416185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":76,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T02:27:57.524276+0000","last_change":"2023-10-06T02:23:01.988140+0000","last_active":"2023-10-06T02:27:57.524276+0000","last_peered":"2023-10-06T02:27:57.524276+0000","last_clean":"2023-10-06T02:27:57.524276+0000","last_became_active":"2023-10-06T02:23:01.987769+0000","last_became_peered":"2023-10-06T02:23:01.987769+0000","last_unstale":"2023-10-06T02:27:57.524276+0000","last_undegraded":"2023-10-06T02:27:57.524276+0000","last_fullsized":"2023-10-06T02:27:57.524276+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:20:08.391912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"76'128","reported_seq":255,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T02:27:57.526574+0000","last_change":"2023-10-06T02:22:55.032519+0000","last_active":"2023-10-06T02:27:57.526574+0000","last_peered":"2023-10-06T02:27:57.526574+0000","last_clean":"2023-10-06T02:27:57.526574+0000","last_became_active":"2023-10-06T02:22:55.031367+0000","last_became_peered":"2023-10-06T02:22:55.031367+0000","last_unstale":"2023-10-06T02:27:57.526574+0000","last_undegraded":"2023-10-06T02:27:57.526574+0000","last_fullsized":"2023-10-06T02:27:57.526574+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:22:53.309097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:22:53.309097+0000","last_clean_scrub_stamp":"2023-10-06T02:22:53.309097+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:53:46.520909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.557038+0000","last_change":"2023-10-06T02:27:58.557038+0000","last_active":"2023-10-06T02:27:58.557038+0000","last_peered":"2023-10-06T02:27:58.557038+0000","last_clean":"2023-10-06T02:27:58.557038+0000","last_became_active":"2023-10-06T02:27:58.556750+0000","last_became_peered":"2023-10-06T02:27:58.556750+0000","last_unstale":"2023-10-06T02:27:58.557038+0000","last_undegraded":"2023-10-06T02:27:58.557038+0000","last_fullsized":"2023-10-06T02:27:58.557038+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:31:58.474184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"86'1","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.534637+0000","last_change":"2023-10-06T02:27:57.535933+0000","last_active":"2023-10-06T02:27:58.534637+0000","last_peered":"2023-10-06T02:27:58.534637+0000","last_clean":"2023-10-06T02:27:58.534637+0000","last_became_active":"2023-10-06T02:27:57.535685+0000","last_became_peered":"2023-10-06T02:27:57.535685+0000","last_unstale":"2023-10-06T02:27:58.534637+0000","last_undegraded":"2023-10-06T02:27:58.534637+0000","last_fullsized":"2023-10-06T02:27:58.534637+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_clean_scrub_stamp":"2023-10-06T02:27:56.508109+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:52:25.592921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":63,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T02:27:57.526478+0000","last_change":"2023-10-06T02:23:01.990773+0000","last_active":"2023-10-06T02:27:57.526478+0000","last_peered":"2023-10-06T02:27:57.526478+0000","last_clean":"2023-10-06T02:27:57.526478+0000","last_became_active":"2023-10-06T02:23:01.990511+0000","last_became_peered":"2023-10-06T02:23:01.990511+0000","last_unstale":"2023-10-06T02:27:57.526478+0000","last_undegraded":"2023-10-06T02:27:57.526478+0000","last_fullsized":"2023-10-06T02:27:57.526478+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:42:54.797731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.541295+0000","last_change":"2023-10-06T02:27:58.541295+0000","last_active":"2023-10-06T02:27:58.541295+0000","last_peered":"2023-10-06T02:27:58.541295+0000","last_clean":"2023-10-06T02:27:58.541295+0000","last_became_active":"2023-10-06T02:27:58.540972+0000","last_became_peered":"2023-10-06T02:27:58.540972+0000","last_unstale":"2023-10-06T02:27:58.541295+0000","last_undegraded":"2023-10-06T02:27:58.541295+0000","last_fullsized":"2023-10-06T02:27:58.541295+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:05:24.882141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T02:27:57.543463+0000","last_change":"2023-10-06T02:27:57.537746+0000","last_active":"2023-10-06T02:27:57.543463+0000","last_peered":"2023-10-06T02:27:57.543463+0000","last_clean":"2023-10-06T02:27:57.543463+0000","last_became_active":"2023-10-06T02:27:57.537140+0000","last_became_peered":"2023-10-06T02:27:57.537140+0000","last_unstale":"2023-10-06T02:27:57.543463+0000","last_undegraded":"2023-10-06T02:27:57.543463+0000","last_fullsized":"2023-10-06T02:27:57.543463+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_clean_scrub_stamp":"2023-10-06T02:27:56.508109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:21:50.264914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":63,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T02:27:57.527295+0000","last_change":"2023-10-06T02:23:02.054069+0000","last_active":"2023-10-06T02:27:57.527295+0000","last_peered":"2023-10-06T02:27:57.527295+0000","last_clean":"2023-10-06T02:27:57.527295+0000","last_became_active":"2023-10-06T02:23:02.053739+0000","last_became_peered":"2023-10-06T02:23:02.053739+0000","last_unstale":"2023-10-06T02:27:57.527295+0000","last_undegraded":"2023-10-06T02:27:57.527295+0000","last_fullsized":"2023-10-06T02:27:57.527295+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:57:48.822608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.572001+0000","last_change":"2023-10-06T02:27:58.572001+0000","last_active":"2023-10-06T02:27:58.572001+0000","last_peered":"2023-10-06T02:27:58.572001+0000","last_clean":"2023-10-06T02:27:58.572001+0000","last_became_active":"2023-10-06T02:27:58.571716+0000","last_became_peered":"2023-10-06T02:27:58.571716+0000","last_unstale":"2023-10-06T02:27:58.572001+0000","last_undegraded":"2023-10-06T02:27:58.572001+0000","last_fullsized":"2023-10-06T02:27:58.572001+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:50:11.405633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T02:27:57.539620+0000","last_change":"2023-10-06T02:27:57.534965+0000","last_active":"2023-10-06T02:27:57.539620+0000","last_peered":"2023-10-06T02:27:57.539620+0000","last_clean":"2023-10-06T02:27:57.539620+0000","last_became_active":"2023-10-06T02:27:57.534690+0000","last_became_peered":"2023-10-06T02:27:57.534690+0000","last_unstale":"2023-10-06T02:27:57.539620+0000","last_undegraded":"2023-10-06T02:27:57.539620+0000","last_fullsized":"2023-10-06T02:27:57.539620+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_clean_scrub_stamp":"2023-10-06T02:27:56.508109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:41:38.011422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":67,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T02:27:57.525521+0000","last_change":"2023-10-06T02:23:01.989934+0000","last_active":"2023-10-06T02:27:57.525521+0000","last_peered":"2023-10-06T02:27:57.525521+0000","last_clean":"2023-10-06T02:27:57.525521+0000","last_became_active":"2023-10-06T02:23:01.989706+0000","last_became_peered":"2023-10-06T02:23:01.989706+0000","last_unstale":"2023-10-06T02:27:57.525521+0000","last_undegraded":"2023-10-06T02:27:57.525521+0000","last_fullsized":"2023-10-06T02:27:57.525521+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:06.212251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":8,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T02:27:58.555915+0000","last_change":"2023-10-06T02:27:58.555915+0000","last_active":"2023-10-06T02:27:58.555915+0000","last_peered":"2023-10-06T02:27:58.555915+0000","last_clean":"2023-10-06T02:27:58.555915+0000","last_became_active":"2023-10-06T02:27:58.555559+0000","last_became_peered":"2023-10-06T02:27:58.555559+0000","last_unstale":"2023-10-06T02:27:58.555915+0000","last_undegraded":"2023-10-06T02:27:58.555915+0000","last_fullsized":"2023-10-06T02:27:58.555915+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:57.514251+0000","last_clean_scrub_stamp":"2023-10-06T02:27:57.514251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:38.699087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T02:27:57.542774+0000","last_change":"2023-10-06T02:27:57.532459+0000","last_active":"2023-10-06T02:27:57.542774+0000","last_peered":"2023-10-06T02:27:57.542774+0000","last_clean":"2023-10-06T02:27:57.542774+0000","last_became_active":"2023-10-06T02:27:57.532196+0000","last_became_peered":"2023-10-06T02:27:57.532196+0000","last_unstale":"2023-10-06T02:27:57.542774+0000","last_undegraded":"2023-10-06T02:27:57.542774+0000","last_fullsized":"2023-10-06T02:27:57.542774+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:27:56.508109+0000","last_clean_scrub_stamp":"2023-10-06T02:27:56.508109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:46:58.999553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":68,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T02:27:57.532778+0000","last_change":"2023-10-06T02:23:01.992935+0000","last_active":"2023-10-06T02:27:57.532778+0000","last_peered":"2023-10-06T02:27:57.532778+0000","last_clean":"2023-10-06T02:27:57.532778+0000","last_became_active":"2023-10-06T02:23:01.992568+0000","last_became_peered":"2023-10-06T02:23:01.992568+0000","last_unstale":"2023-10-06T02:27:57.532778+0000","last_undegraded":"2023-10-06T02:27:57.532778+0000","last_fullsized":"2023-10-06T02:27:57.532778+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:06.212251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574914,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302996,"kb_used_data":1612,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068844,"statfs":{"total":96636764160,"available":96326496256,"internally_reserved":0,"allocated":1650688,"data_stored":1068141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:26:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.64100000000000001,"15min":0.63800000000000001},"min":{"1min":0.42399999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.002,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.624,"15min":0.60599999999999998},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.84399999999999997,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.72399999999999998}]},{"osd":1,"last update":"Fri Oct 6 02:26:58 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.71099999999999997,"15min":0.68500000000000005},"min":{"1min":0.46800000000000003,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.95399999999999996,"5min":1.147,"15min":1.147},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.68200000000000005,"15min":0.67500000000000004},"min":{"1min":0.498,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.86899999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.79300000000000004}]},{"osd":2,"last update":"Fri Oct 6 02:26:58 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.69299999999999995,"15min":0.69499999999999995},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.68700000000000006,"15min":0.68999999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.93500000000000005,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.80700000000000005}]},{"osd":3,"last update":"Fri Oct 6 02:26:58 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.747,"15min":0.73299999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.754,"5min":0.77600000000000002,"15min":0.77800000000000002},"min":{"1min":0.55500000000000005,"5min":0.497,"15min":0.497},"max":{"1min":0.98199999999999998,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.83799999999999997}]},{"osd":4,"last update":"Fri Oct 6 02:26:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.65000000000000002,"15min":0.69599999999999995},"min":{"1min":0.41099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.95199999999999996,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.72599999999999998,"15min":0.77000000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.105,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.66500000000000004}]},{"osd":5,"last update":"Fri Oct 6 02:26:58 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.71499999999999997,"15min":0.73499999999999999},"min":{"1min":0.44700000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.98499999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.71499999999999997,"15min":0.73499999999999999},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.95399999999999996,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.64600000000000002}]},{"osd":6,"last update":"Fri Oct 6 02:26:58 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.77400000000000002,"15min":0.78700000000000003},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0109999999999999,"5min":1.056,"15min":1.056},"last":0.748},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.80600000000000005,"15min":0.81599999999999995},"min":{"1min":0.48199999999999998,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.1659999999999999,"5min":1.276,"15min":1.276},"last":0.82199999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574914,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302436,"kb_used_data":1052,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069404,"statfs":{"total":96636764160,"available":96327069696,"internally_reserved":0,"allocated":1077248,"data_stored":479885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:27:03 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.67100000000000004,"15min":0.66400000000000003},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.91900000000000004,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.71399999999999997,"15min":0.69699999999999995},"min":{"1min":0.53500000000000003,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.95099999999999996,"5min":1.119,"15min":1.119},"last":0.628}]},{"osd":1,"last update":"Fri Oct 6 02:27:03 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.64100000000000001,"15min":0.64400000000000002},"min":{"1min":0.433,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67400000000000004,"15min":0.66600000000000004},"min":{"1min":0.48899999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.89500000000000002,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.68500000000000005}]},{"osd":2,"last update":"Fri Oct 6 02:27:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.71199999999999997,"15min":0.70199999999999996},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.96099999999999997,"5min":1.137,"15min":1.137},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72699999999999998,"15min":0.71099999999999997},"min":{"1min":0.55600000000000005,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.89300000000000002,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.55800000000000005}]},{"osd":3,"last update":"Fri Oct 6 02:27:03 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.68200000000000005,"15min":0.66000000000000003},"min":{"1min":0.52000000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.97099999999999997,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71299999999999997,"15min":0.70299999999999996},"min":{"1min":0.5,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.91600000000000004,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.60399999999999998}]},{"osd":4,"last update":"Fri Oct 6 02:27:03 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63900000000000001,"15min":0.63400000000000001},"min":{"1min":0.36199999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.90000000000000002,"5min":1.095,"15min":1.095},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.67600000000000005,"15min":0.65600000000000003},"min":{"1min":0.46200000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.999,"5min":1.29,"15min":1.29},"last":0.80300000000000005}]},{"osd":5,"last update":"Fri Oct 6 02:27:03 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.68100000000000005,"15min":0.69699999999999995},"min":{"1min":0.48399999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.94299999999999995,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.70699999999999996,"15min":0.70199999999999996},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.86099999999999999,"5min":1.02,"15min":1.02},"last":0.82299999999999995}]},{"osd":7,"last update":"Fri Oct 6 02:27:03 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.74199999999999999,"15min":0.746},"min":{"1min":0.41499999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.95399999999999996,"5min":1.458,"15min":1.458},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76000000000000001,"15min":0.75900000000000001},"min":{"1min":0.54600000000000004,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.96499999999999997,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.75}]}]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302420,"kb_used_data":1036,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069420,"statfs":{"total":96636764160,"available":96327086080,"internally_reserved":0,"allocated":1060864,"data_stored":477843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.442,"15min":0.46000000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.58299999999999996,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.315},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.48799999999999999,"15min":0.52700000000000002},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.70099999999999996,"5min":1.788,"15min":1.788},"last":0.40400000000000003}]},{"osd":2,"last update":"Fri Oct 6 02:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.46700000000000003,"15min":0.52300000000000002},"min":{"1min":0.307,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.60899999999999999,"5min":2.0840000000000001,"15min":2.0840000000000001},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.52100000000000002,"15min":0.55800000000000005},"min":{"1min":0.36899999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.66400000000000003,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.376}]},{"osd":3,"last update":"Fri Oct 6 02:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60799999999999998,"15min":0.61699999999999999},"min":{"1min":0.46000000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.84999999999999998,"5min":1.266,"15min":1.266},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.56699999999999995,"15min":0.63900000000000001},"min":{"1min":0.41099999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.79400000000000004,"5min":1.861,"15min":1.861},"last":0.47499999999999998}]},{"osd":4,"last update":"Fri Oct 6 02:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.80700000000000005,"15min":0.85699999999999998},"min":{"1min":0.59099999999999997,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.3580000000000001,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.81499999999999995,"15min":0.85799999999999998},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.4339999999999999,"5min":2.0179999999999998,"15min":2.0179999999999998},"last":0.56200000000000006}]},{"osd":5,"last update":"Fri Oct 6 02:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.77600000000000002,"15min":0.85799999999999998},"min":{"1min":0.52600000000000002,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.1040000000000001,"5min":1.9830000000000001,"15min":1.9830000000000001},"last":0.627},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.77400000000000002,"15min":0.82199999999999995},"min":{"1min":0.58899999999999997,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.161,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.73599999999999999}]},{"osd":6,"last update":"Fri Oct 6 02:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.81799999999999995,"15min":0.86699999999999999},"min":{"1min":0.52700000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0609999999999999,"5min":1.829,"15min":1.829},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.84399999999999997,"15min":0.91200000000000003},"min":{"1min":0.64600000000000002,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.282,"5min":2.052,"15min":2.052},"last":0.68000000000000005}]},{"osd":7,"last update":"Fri Oct 6 02:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.80000000000000004,"15min":0.83699999999999997},"min":{"1min":0.56299999999999994,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.3120000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.81699999999999995,"15min":0.879},"min":{"1min":0.55000000000000004,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.2110000000000001,"5min":1.895,"15min":1.895},"last":0.69499999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574914,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303016,"kb_used_data":1632,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068824,"statfs":{"total":96636764160,"available":96326475776,"internally_reserved":0,"allocated":1671168,"data_stored":1069463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 02:27:05 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.44,"15min":0.45600000000000002},"min":{"1min":0.32000000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.54300000000000004,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.439,"5min":0.47399999999999998,"15min":0.48199999999999998},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.58799999999999997,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.43099999999999999}]},{"osd":2,"last update":"Fri Oct 6 02:27:05 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.498,"15min":0.51400000000000001},"min":{"1min":0.40899999999999997,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.60699999999999998,"5min":1.069,"15min":1.069},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.51100000000000001,"15min":0.52200000000000002},"min":{"1min":0.39600000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.73099999999999998,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.495}]},{"osd":3,"last update":"Fri Oct 6 02:27:05 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.49399999999999999,"15min":0.50600000000000001},"min":{"1min":0.34100000000000003,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.59599999999999997,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59499999999999997,"15min":0.59699999999999998},"min":{"1min":0.47299999999999998,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.73099999999999998,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.56200000000000006}]},{"osd":4,"last update":"Fri Oct 6 02:27:05 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.77000000000000002,"15min":0.80300000000000005},"min":{"1min":0.54800000000000004,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.86899999999999999,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.77700000000000002,"15min":0.80300000000000005},"min":{"1min":0.58099999999999996,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0469999999999999,"5min":1.304,"15min":1.304},"last":0.629}]},{"osd":5,"last update":"Fri Oct 6 02:27:05 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.76600000000000001,"15min":0.78800000000000003},"min":{"1min":0.52600000000000002,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.94199999999999995,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.79100000000000004,"15min":0.80300000000000005},"min":{"1min":0.59699999999999998,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.018,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.70099999999999996}]},{"osd":6,"last update":"Fri Oct 6 02:27:05 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.81299999999999994,"15min":0.82599999999999996},"min":{"1min":0.59699999999999998,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.96399999999999997,"5min":1.167,"15min":1.167},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.80500000000000005,"15min":0.83499999999999996},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.0349999999999999,"5min":1.069,"15min":1.069},"last":0.68000000000000005}]},{"osd":7,"last update":"Fri Oct 6 02:27:05 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.80500000000000005,"15min":0.82899999999999996},"min":{"1min":0.56899999999999995,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.98499999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.79600000000000004,"15min":0.82899999999999996},"min":{"1min":0.54700000000000004,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.002,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.66700000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574915,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302416,"kb_used_data":1032,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069424,"statfs":{"total":96636764160,"available":96327090176,"internally_reserved":0,"allocated":1056768,"data_stored":477773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:27:02 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.434,"15min":0.441},"min":{"1min":0.312,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.54100000000000004,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.48299999999999998,"15min":0.48799999999999999},"min":{"1min":0.32800000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.88700000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.47599999999999998}]},{"osd":1,"last update":"Fri Oct 6 02:27:02 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.5,"15min":0.50700000000000001},"min":{"1min":0.36699999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.63900000000000001,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.56399999999999995,"15min":0.61299999999999999},"min":{"1min":0.39600000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.70899999999999996,"5min":1.345,"15min":1.345},"last":0.51600000000000001}]},{"osd":3,"last update":"Fri Oct 6 02:27:02 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.52200000000000002,"15min":0.53600000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.69699999999999995,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.60699999999999998,"15min":0.624},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.72799999999999998,"5min":1.028,"15min":1.028},"last":0.60099999999999998}]},{"osd":4,"last update":"Fri Oct 6 02:27:02 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.76400000000000001,"15min":0.78100000000000003},"min":{"1min":0.55500000000000005,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.89300000000000002,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.82399999999999995,"15min":0.85499999999999998},"min":{"1min":0.59999999999999998,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.95999999999999996,"5min":1.621,"15min":1.621},"last":0.76200000000000001}]},{"osd":5,"last update":"Fri Oct 6 02:27:02 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.77500000000000002,"15min":0.80800000000000005},"min":{"1min":0.57099999999999995,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.94899999999999995,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.80000000000000004,"15min":0.82899999999999996},"min":{"1min":0.60399999999999998,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":0.98199999999999998,"5min":1.258,"15min":1.258},"last":0.77800000000000002}]},{"osd":6,"last update":"Fri Oct 6 02:27:02 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.82299999999999995,"15min":0.85399999999999998},"min":{"1min":0.59099999999999997,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.98099999999999998,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.78900000000000003,"15min":0.79900000000000004},"min":{"1min":0.623,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.0109999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.78900000000000003}]},{"osd":7,"last update":"Fri Oct 6 02:27:02 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.85299999999999998,"15min":0.89200000000000002},"min":{"1min":0.66200000000000003,"5min":0.499,"15min":0.499},"max":{"1min":0.95999999999999996,"5min":1.585,"15min":1.585},"last":0.628},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.81100000000000005,"15min":0.83699999999999997},"min":{"1min":0.58999999999999997,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.94399999999999995,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.748}]}]},{"osd":3,"up_from":13,"seq":55834574912,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302416,"kb_used_data":1032,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069424,"statfs":{"total":96636764160,"available":96327090176,"internally_reserved":0,"allocated":1056768,"data_stored":477773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:27:02 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48899999999999999,"15min":0.51700000000000002},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.69599999999999995,"5min":2.024,"15min":2.024},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.51200000000000001,"15min":0.52800000000000002},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.67400000000000004,"5min":2.181,"15min":2.181},"last":0.48799999999999999}]},{"osd":1,"last update":"Fri Oct 6 02:27:02 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.58799999999999997,"15min":0.70299999999999996},"min":{"1min":0.39600000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.59299999999999997,"5min":4.492,"15min":4.492},"last":0.438},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.63700000000000001,"15min":0.745},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.80000000000000004,"5min":4.569,"15min":4.569},"last":0.52500000000000002}]},{"osd":2,"last update":"Fri Oct 6 02:27:02 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.65800000000000003,"15min":0.67200000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.91100000000000003,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.625,"15min":0.65900000000000003},"min":{"1min":0.46200000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.94199999999999995,"5min":2.4260000000000002,"15min":2.4260000000000002},"last":0.50700000000000001}]},{"osd":4,"last update":"Fri Oct 6 02:27:02 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.78600000000000003,"15min":0.83899999999999997},"min":{"1min":0.61599999999999999,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.96699999999999997,"5min":2.214,"15min":2.214},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.78600000000000003,"15min":0.81699999999999995},"min":{"1min":0.50800000000000001,"5min":0.5,"15min":0.5},"max":{"1min":0.94399999999999995,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.76900000000000002}]},{"osd":5,"last update":"Fri Oct 6 02:27:02 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.80500000000000005,"15min":0.82899999999999996},"min":{"1min":0.57899999999999996,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.97699999999999998,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.81100000000000005,"15min":0.86499999999999999},"min":{"1min":0.63800000000000001,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.0289999999999999,"5min":2.242,"15min":2.242},"last":0.68799999999999994}]},{"osd":6,"last update":"Fri Oct 6 02:27:02 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.79200000000000004,"15min":0.82399999999999995},"min":{"1min":0.59399999999999997,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.0049999999999999,"5min":1.357,"15min":1.357},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.81999999999999995,"15min":0.83799999999999997},"min":{"1min":0.54600000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.0529999999999999,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.71999999999999997}]},{"osd":7,"last update":"Fri Oct 6 02:27:02 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.83399999999999996,"15min":0.87},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.077,"5min":2.27,"15min":2.27},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.83099999999999996,"15min":0.86399999999999999},"min":{"1min":0.55600000000000005,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.056,"5min":1.325,"15min":1.325},"last":0.80300000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574914,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302416,"kb_used_data":1032,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069424,"statfs":{"total":96636764160,"available":96327090176,"internally_reserved":0,"allocated":1056768,"data_stored":477773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:27:03 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.86599999999999999,"15min":1.0549999999999999},"min":{"1min":0.45700000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.90700000000000003,"5min":12.337,"15min":12.337},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.81399999999999995,"15min":0.95099999999999996},"min":{"1min":0.496,"5min":0.438,"15min":0.438},"max":{"1min":0.91700000000000004,"5min":9.6539999999999999,"15min":9.6539999999999999},"last":0.67100000000000004}]},{"osd":1,"last update":"Fri Oct 6 02:27:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.69999999999999996,"15min":0.70599999999999996},"min":{"1min":0.53800000000000003,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.96999999999999997,"5min":3.181,"15min":3.181},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.876,"15min":1.0740000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.90000000000000002,"5min":12.946999999999999,"15min":12.946999999999999},"last":0.56299999999999994}]},{"osd":2,"last update":"Fri Oct 6 02:27:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.89700000000000002,"15min":1.103},"min":{"1min":0.44400000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.93300000000000005,"5min":12.377000000000001,"15min":12.377000000000001},"last":0.629},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.80100000000000005,"15min":0.89500000000000002},"min":{"1min":0.53800000000000003,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.95899999999999996,"5min":6.3220000000000001,"15min":6.3220000000000001},"last":0.69299999999999995}]},{"osd":3,"last update":"Fri Oct 6 02:27:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.873,"15min":1.0309999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.88500000000000001,"5min":9.7720000000000002,"15min":9.7720000000000002},"last":0.753},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.745,"15min":0.78500000000000003},"min":{"1min":0.498,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.91600000000000004,"5min":3.0840000000000001,"15min":3.0840000000000001},"last":0.52400000000000002}]},{"osd":5,"last update":"Fri Oct 6 02:27:03 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.76100000000000001,"15min":0.86899999999999999},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.82899999999999996,"5min":6.2690000000000001,"15min":6.2690000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.874,"15min":1.0840000000000001},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.90900000000000003,"5min":10.769,"15min":10.769},"last":0.54400000000000004}]},{"osd":6,"last update":"Fri Oct 6 02:27:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.71199999999999997,"15min":0.77000000000000002},"min":{"1min":0.44,"5min":0.432,"15min":0.432},"max":{"1min":0.84299999999999997,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.85599999999999998,"15min":1.0029999999999999},"min":{"1min":0.49199999999999999,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.94399999999999995,"5min":8.0899999999999999,"15min":8.0899999999999999},"last":0.5}]},{"osd":7,"last update":"Fri Oct 6 02:27:03 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.92600000000000005,"15min":1.113},"min":{"1min":0.50600000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.93899999999999995,"5min":9.7089999999999996,"15min":9.7089999999999996},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.88800000000000001,"15min":1.034},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.97799999999999998,"5min":7.6719999999999997,"15min":7.6719999999999997},"last":0.65500000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574914,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302416,"kb_used_data":1032,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069424,"statfs":{"total":96636764160,"available":96327090176,"internally_reserved":0,"allocated":1056768,"data_stored":477773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:27:00 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.64200000000000002,"15min":0.626},"min":{"1min":0.44700000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.84899999999999998,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.64900000000000002,"15min":0.60799999999999998},"min":{"1min":0.41699999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.93999999999999995,"5min":1.026,"15min":1.026},"last":0.58899999999999997}]},{"osd":1,"last update":"Fri Oct 6 02:27:00 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.67000000000000004,"15min":0.65000000000000002},"min":{"1min":0.438,"5min":0.434,"15min":0.434},"max":{"1min":0.95399999999999996,"5min":1.089,"15min":1.089},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.68899999999999995,"15min":0.67500000000000004},"min":{"1min":0.53200000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.88300000000000001,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.622}]},{"osd":2,"last update":"Fri Oct 6 02:27:00 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.69599999999999995,"15min":0.66800000000000004},"min":{"1min":0.499,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.98999999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.70099999999999996,"15min":0.68300000000000005},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.96099999999999997,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.69399999999999995}]},{"osd":3,"last update":"Fri Oct 6 02:27:00 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.748,"15min":0.72599999999999998},"min":{"1min":0.58399999999999996,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.93999999999999995,"5min":1.042,"15min":1.042},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.78200000000000003,"15min":0.76400000000000001},"min":{"1min":0.59499999999999997,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.0489999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.84699999999999998}]},{"osd":4,"last update":"Fri Oct 6 02:27:00 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.66100000000000003,"15min":0.67300000000000004},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.89000000000000001,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.624,"5min":0.67100000000000004,"15min":0.68200000000000005},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.86499999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.80700000000000005}]},{"osd":6,"last update":"Fri Oct 6 02:27:00 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.73899999999999999,"15min":0.74299999999999999},"min":{"1min":0.45700000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.0549999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.81499999999999995,"15min":0.82099999999999995},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.03,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.76500000000000001}]},{"osd":7,"last update":"Fri Oct 6 02:27:00 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.75900000000000001,"15min":0.76300000000000001},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.0009999999999999,"5min":1.089,"15min":1.089},"last":0.745},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.81799999999999995,"15min":0.83599999999999997},"min":{"1min":0.46999999999999997,"5min":0.437,"15min":0.437},"max":{"1min":1.0229999999999999,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.89300000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T02:28:03.429 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T02:28:03.429 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T02:28:03.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.657+0000 7f7e1ffff700 1 Processor -- start 2023-10-06T02:28:03.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.658+0000 7f7e1ffff700 1 -- start start 2023-10-06T02:28:03.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.659+0000 7f7e1ffff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7e20142c00 0x7f7e2006b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:03.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.659+0000 7f7e1ffff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7e2006bd20 0x7f7e200c7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:03.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.659+0000 7f7e1ffff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e200ca480 con 0x7f7e20142fd0 2023-10-06T02:28:03.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.659+0000 7f7e1ffff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e2015e240 con 0x7f7e2006bd20 2023-10-06T02:28:03.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.659+0000 7f7e1ffff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e2006b350 con 0x7f7e20142c00 2023-10-06T02:28:03.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.659+0000 7f7e1e7fc700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7e2006bd20 0x7f7e200c7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:03.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.659+0000 7f7e1e7fc700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7e2006bd20 0x7f7e200c7880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40518/0 (socket says 172.21.15.31:40518) 2023-10-06T02:28:03.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.659+0000 7f7e1e7fc700 1 -- 172.21.15.31:0/3111521256 learned_addr learned my addr 172.21.15.31:0/3111521256 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:03.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.659+0000 7f7e1effd700 1 --2- 172.21.15.31:0/3111521256 >> v2:172.21.15.62:3300/0 conn(0x7f7e20142c00 0x7f7e2006b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:03.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.660+0000 7f7e1e7fc700 1 -- 172.21.15.31:0/3111521256 >> v2:172.21.15.62:3300/0 conn(0x7f7e20142c00 msgr2=0x7f7e2006b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:03.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.660+0000 7f7e1e7fc700 1 --2- 172.21.15.31:0/3111521256 >> v2:172.21.15.62:3300/0 conn(0x7f7e20142c00 0x7f7e2006b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:03.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.660+0000 7f7e1e7fc700 1 -- 172.21.15.31:0/3111521256 >> v1:172.21.15.62:6789/0 conn(0x7f7e20142fd0 legacy=0x7f7e2006c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:03.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.660+0000 7f7e1e7fc700 1 -- 172.21.15.31:0/3111521256 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e2015df60 con 0x7f7e2006bd20 2023-10-06T02:28:03.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.660+0000 7f7e1e7fc700 1 --2- 172.21.15.31:0/3111521256 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e2006bd20 0x7f7e200c7880 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f7e2006f490 tx=0x7f7e14009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa148819cb98782c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:03.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.660+0000 7f7e1dffb700 1 -- 172.21.15.31:0/3111521256 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e1400b860 con 0x7f7e2006bd20 2023-10-06T02:28:03.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.660+0000 7f7e1dffb700 1 -- 172.21.15.31:0/3111521256 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e1400b9c0 con 0x7f7e2006bd20 2023-10-06T02:28:03.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.660+0000 7f7e1dffb700 1 -- 172.21.15.31:0/3111521256 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e1400bd60 con 0x7f7e2006bd20 2023-10-06T02:28:03.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.661+0000 7f7e1ffff700 1 -- 172.21.15.31:0/3111521256 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e2006bd20 msgr2=0x7f7e200c7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:03.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.661+0000 7f7e1ffff700 1 --2- 172.21.15.31:0/3111521256 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e2006bd20 0x7f7e200c7880 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f7e2006f490 tx=0x7f7e14009250 comp rx=0 tx=0).stop 2023-10-06T02:28:03.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.661+0000 7f7e1ffff700 1 -- 172.21.15.31:0/3111521256 shutdown_connections 2023-10-06T02:28:03.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.661+0000 7f7e1ffff700 1 --2- 172.21.15.31:0/3111521256 >> v2:172.21.15.62:3300/0 conn(0x7f7e20142c00 0x7f7e2006b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:03.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.661+0000 7f7e1ffff700 1 --2- 172.21.15.31:0/3111521256 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e2006bd20 0x7f7e200c7880 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:03.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.661+0000 7f7e1ffff700 1 -- 172.21.15.31:0/3111521256 >> 172.21.15.31:0/3111521256 conn(0x7f7e200675a0 msgr2=0x7f7e200679a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:03.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.661+0000 7f7e1ffff700 1 -- 172.21.15.31:0/3111521256 shutdown_connections 2023-10-06T02:28:03.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.661+0000 7f7e1ffff700 1 -- 172.21.15.31:0/3111521256 wait complete. 2023-10-06T02:28:03.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.662+0000 7f7e1ffff700 1 Processor -- start 2023-10-06T02:28:03.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.663+0000 7f7e1ffff700 1 -- start start 2023-10-06T02:28:03.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.663+0000 7f7e1ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e2006bd20 0x7f7e200c4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:03.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.663+0000 7f7e1ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e20142c00 0x7f7e200c6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:03.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.664+0000 7f7e1ffff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7e20142fd0 0x7f7e2015e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:03.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.664+0000 7f7e1ffff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e200c7610 con 0x7f7e2006bd20 2023-10-06T02:28:03.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.664+0000 7f7e1ffff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e200b01a0 con 0x7f7e20142c00 2023-10-06T02:28:03.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.664+0000 7f7e1ffff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e200c6d70 con 0x7f7e20142fd0 2023-10-06T02:28:03.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.664+0000 7f7e1effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e2006bd20 0x7f7e200c4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:03.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.664+0000 7f7e1effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e2006bd20 0x7f7e200c4810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40532/0 (socket says 172.21.15.31:40532) 2023-10-06T02:28:03.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.664+0000 7f7e1effd700 1 -- 172.21.15.31:0/1766293784 learned_addr learned my addr 172.21.15.31:0/1766293784 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:03.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.664+0000 7f7e1effd700 1 -- 172.21.15.31:0/1766293784 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7e20142fd0 msgr2=0x7f7e2015e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:03.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.664+0000 7f7e1effd700 1 --2- 172.21.15.31:0/1766293784 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7e20142fd0 0x7f7e2015e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:03.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.664+0000 7f7e1effd700 1 -- 172.21.15.31:0/1766293784 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e20142c00 msgr2=0x7f7e200c6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:03.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.664+0000 7f7e1effd700 1 --2- 172.21.15.31:0/1766293784 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e20142c00 0x7f7e200c6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:03.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.664+0000 7f7e1effd700 1 -- 172.21.15.31:0/1766293784 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e140094f0 con 0x7f7e2006bd20 2023-10-06T02:28:03.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.664+0000 7f7e1effd700 1 --2- 172.21.15.31:0/1766293784 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e2006bd20 0x7f7e200c4810 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f7e1000d890 tx=0x7f7e10008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:03.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.665+0000 7f7e06ffd700 1 -- 172.21.15.31:0/1766293784 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e100125f0 con 0x7f7e2006bd20 2023-10-06T02:28:03.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.665+0000 7f7e06ffd700 1 -- 172.21.15.31:0/1766293784 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e10012770 con 0x7f7e2006bd20 2023-10-06T02:28:03.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.665+0000 7f7e06ffd700 1 -- 172.21.15.31:0/1766293784 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e10012b30 con 0x7f7e2006bd20 2023-10-06T02:28:03.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.665+0000 7f7e1ffff700 1 -- 172.21.15.31:0/1766293784 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e200c7050 con 0x7f7e2006bd20 2023-10-06T02:28:03.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.665+0000 7f7e1ffff700 1 -- 172.21.15.31:0/1766293784 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e200c4d50 con 0x7f7e2006bd20 2023-10-06T02:28:03.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.666+0000 7f7e1ffff700 1 -- 172.21.15.31:0/1766293784 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e200c4f40 con 0x7f7e2006bd20 2023-10-06T02:28:03.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.667+0000 7f7e06ffd700 1 -- 172.21.15.31:0/1766293784 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f7e1001e460 con 0x7f7e2006bd20 2023-10-06T02:28:03.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.669+0000 7f7e06ffd700 1 --2- 172.21.15.31:0/1766293784 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7e0c067080 0x7f7e0c069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:03.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.669+0000 7f7e06ffd700 1 -- 172.21.15.31:0/1766293784 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f7e10084fa0 con 0x7f7e2006bd20 2023-10-06T02:28:03.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.669+0000 7f7e1e7fc700 1 --2- 172.21.15.31:0/1766293784 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7e0c067080 0x7f7e0c069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:03.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.670+0000 7f7e1e7fc700 1 --2- 172.21.15.31:0/1766293784 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7e0c067080 0x7f7e0c069530 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7e2006f490 tx=0x7f7e14017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:03.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.673+0000 7f7e06ffd700 1 -- 172.21.15.31:0/1766293784 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7e10051f00 con 0x7f7e2006bd20 2023-10-06T02:28:03.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.956+0000 7f7e1ffff700 1 -- 172.21.15.31:0/1766293784 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f7e2004c3c0 con 0x7f7e2006bd20 2023-10-06T02:28:03.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.960+0000 7f7e06ffd700 1 -- 172.21.15.31:0/1766293784 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) v1 ==== 190+0+0 (secure 0 0 0) 0x7f7e10055520 con 0x7f7e2006bd20 2023-10-06T02:28:03.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.962+0000 7f7e1ffff700 1 -- 172.21.15.31:0/1766293784 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7e0c067080 msgr2=0x7f7e0c069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:03.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.962+0000 7f7e1ffff700 1 --2- 172.21.15.31:0/1766293784 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7e0c067080 0x7f7e0c069530 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7e2006f490 tx=0x7f7e14017000 comp rx=0 tx=0).stop 2023-10-06T02:28:03.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.962+0000 7f7e1ffff700 1 -- 172.21.15.31:0/1766293784 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e2006bd20 msgr2=0x7f7e200c4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:03.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.962+0000 7f7e1ffff700 1 --2- 172.21.15.31:0/1766293784 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e2006bd20 0x7f7e200c4810 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f7e1000d890 tx=0x7f7e10008040 comp rx=0 tx=0).stop 2023-10-06T02:28:03.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.962+0000 7f7e1ffff700 1 -- 172.21.15.31:0/1766293784 shutdown_connections 2023-10-06T02:28:03.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.962+0000 7f7e1ffff700 1 --2- 172.21.15.31:0/1766293784 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7e0c067080 0x7f7e0c069530 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:03.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.962+0000 7f7e1ffff700 1 --2- 172.21.15.31:0/1766293784 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7e20142fd0 0x7f7e2015e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:03.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.962+0000 7f7e1ffff700 1 --2- 172.21.15.31:0/1766293784 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e2006bd20 0x7f7e200c4810 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:03.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.962+0000 7f7e1ffff700 1 --2- 172.21.15.31:0/1766293784 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e20142c00 0x7f7e200c6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:03.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.962+0000 7f7e1ffff700 1 -- 172.21.15.31:0/1766293784 >> 172.21.15.31:0/1766293784 conn(0x7f7e200675a0 msgr2=0x7f7e20136910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:03.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.963+0000 7f7e1ffff700 1 -- 172.21.15.31:0/1766293784 shutdown_connections 2023-10-06T02:28:03.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:03.963+0000 7f7e1ffff700 1 -- 172.21.15.31:0/1766293784 wait complete. 2023-10-06T02:28:03.964 INFO:teuthology.orchestra.run.smithi031.stderr:updated caps for client.0 2023-10-06T02:28:03.980 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:28:04.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:04.166+0000 7fa9e339d700 1 Processor -- start 2023-10-06T02:28:04.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:04.167+0000 7fa9e339d700 1 -- start start 2023-10-06T02:28:04.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:04.168+0000 7fa9e339d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa9dc13fb20 0x7fa9dc15a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:04.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:04.168+0000 7fa9e339d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa9dc06aaa0 0x7fa9dc0c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:04.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:04.168+0000 7fa9e339d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9dc15ae70 con 0x7fa9dc13fef0 2023-10-06T02:28:04.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:04.168+0000 7fa9e339d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9dc06af60 con 0x7fa9dc06aaa0 2023-10-06T02:28:04.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:04.168+0000 7fa9e339d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9dc06b240 con 0x7fa9dc13fb20 2023-10-06T02:28:04.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:04.168+0000 7fa9e1b9a700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fa9dc13fef0 0x7fa9dc0c7990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58102/0 (socket says 172.21.15.31:58102) 2023-10-06T02:28:04.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:04.168+0000 7fa9e1b9a700 1 -- 172.21.15.31:0/2419181878 learned_addr learned my addr 172.21.15.31:0/2419181878 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:28:04.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:04.169+0000 7fa9e2b9c700 1 --2- 172.21.15.31:0/2419181878 >> v2:172.21.15.62:3300/0 conn(0x7fa9dc06aaa0 0x7fa9dc0c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:04.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:04.169+0000 7fa9e239b700 1 --2- 172.21.15.31:0/2419181878 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa9dc13fb20 0x7fa9dc15a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:04.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:04.169+0000 7fa9e1399700 1 -- 172.21.15.31:0/2419181878 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 254651835 0 0) 0x7fa9cc001290 con 0x7fa9dc13fef0 2023-10-06T02:28:04.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:04.169+0000 7fa9e1399700 1 -- 172.21.15.31:0/2419181878 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9c8003570 con 0x7fa9dc13fef0 2023-10-06T02:28:04.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:04.169+0000 7fa9e1399700 1 -- 172.21.15.31:0/2419181878 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 221957314 0 0) 0x7fa9c8003570 con 0x7fa9dc13fef0 2023-10-06T02:28:04.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:04.169+0000 7fa9e239b700 1 -- 172.21.15.31:0/2419181878 >> v2:172.21.15.62:3300/0 conn(0x7fa9dc06aaa0 msgr2=0x7fa9dc0c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:04.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:04.169+0000 7fa9e239b700 1 --2- 172.21.15.31:0/2419181878 >> v2:172.21.15.62:3300/0 conn(0x7fa9dc06aaa0 0x7fa9dc0c9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:04.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:04.169+0000 7fa9e239b700 1 -- 172.21.15.31:0/2419181878 >> v1:172.21.15.31:6789/0 conn(0x7fa9dc13fef0 legacy=0x7fa9dc0c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:04.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:04.169+0000 7fa9e239b700 1 -- 172.21.15.31:0/2419181878 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9dc06b520 con 0x7fa9dc13fb20 2023-10-06T02:28:04.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:04.169+0000 7fa9e2b9c700 1 --2- 172.21.15.31:0/2419181878 >> v2:172.21.15.62:3300/0 conn(0x7fa9dc06aaa0 0x7fa9dc0c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:28:04.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:04.169+0000 7fa9e239b700 1 --2- 172.21.15.31:0/2419181878 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa9dc13fb20 0x7fa9dc15a930 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa9d80098e0 tx=0x7fa9d80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f67d5b7b6610732c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:04.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:04.170+0000 7fa9e1399700 1 -- 172.21.15.31:0/2419181878 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc001780 con 0x7fa9dc13fb20 2023-10-06T02:28:04.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:04.170+0000 7fa9e1399700 1 -- 172.21.15.31:0/2419181878 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9d8011420 con 0x7fa9dc13fb20 2023-10-06T02:28:04.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:04.170+0000 7fa9e1399700 1 -- 172.21.15.31:0/2419181878 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d80117c0 con 0x7fa9dc13fb20 2023-10-06T02:28:04.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:04.170+0000 7fa9e339d700 1 -- 172.21.15.31:0/2419181878 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa9dc13fb20 msgr2=0x7fa9dc15a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:04.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:04.170+0000 7fa9e339d700 1 --2- 172.21.15.31:0/2419181878 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa9dc13fb20 0x7fa9dc15a930 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa9d80098e0 tx=0x7fa9d80091a0 comp rx=0 tx=0).stop 2023-10-06T02:28:04.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:04.170+0000 7fa9e339d700 1 -- 172.21.15.31:0/2419181878 shutdown_connections 2023-10-06T02:28:04.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:04.170+0000 7fa9e339d700 1 --2- 172.21.15.31:0/2419181878 >> v2:172.21.15.62:3300/0 conn(0x7fa9dc06aaa0 0x7fa9dc0c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:04.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:04.170+0000 7fa9e339d700 1 --2- 172.21.15.31:0/2419181878 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa9dc13fb20 0x7fa9dc15a930 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:04.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:04.170+0000 7fa9e339d700 1 -- 172.21.15.31:0/2419181878 >> 172.21.15.31:0/2419181878 conn(0x7fa9dc06d170 msgr2=0x7fa9dc0cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:04.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:04.170+0000 7fa9e339d700 1 -- 172.21.15.31:0/2419181878 shutdown_connections 2023-10-06T02:28:04.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:04.170+0000 7fa9e339d700 1 -- 172.21.15.31:0/2419181878 wait complete. 2023-10-06T02:28:04.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:04.172+0000 7fa9e339d700 1 Processor -- start 2023-10-06T02:28:04.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:04.172+0000 7fa9e339d700 1 -- start start 2023-10-06T02:28:04.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:04.173+0000 7fa9e339d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9dc06aaa0 0x7fa9dc13a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:04.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:04.173+0000 7fa9e339d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9dc13fb20 0x7fa9dc1372d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:04.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:04.173+0000 7fa9e339d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa9dc13fef0 0x7fa9dc153f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:04.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:04.173+0000 7fa9e339d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9dc1585e0 con 0x7fa9dc06aaa0 2023-10-06T02:28:04.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:04.173+0000 7fa9e339d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9dc0ae6a0 con 0x7fa9dc13fb20 2023-10-06T02:28:04.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:04.173+0000 7fa9e339d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9dc157cf0 con 0x7fa9dc13fef0 2023-10-06T02:28:04.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:04.173+0000 7fa9e239b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9dc06aaa0 0x7fa9dc13a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:04.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:04.173+0000 7fa9e239b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9dc06aaa0 0x7fa9dc13a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40538/0 (socket says 172.21.15.31:40538) 2023-10-06T02:28:04.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:04.173+0000 7fa9e239b700 1 -- 172.21.15.31:0/86963931 learned_addr learned my addr 172.21.15.31:0/86963931 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:04.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:04.173+0000 7fa9e1b9a700 1 --2- 172.21.15.31:0/86963931 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9dc13fb20 0x7fa9dc1372d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:04.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:04.173+0000 7fa9e2b9c700 1 --2- 172.21.15.31:0/86963931 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa9dc13fef0 0x7fa9dc153f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:04.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:04.174+0000 7fa9e239b700 1 -- 172.21.15.31:0/86963931 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa9dc13fef0 msgr2=0x7fa9dc153f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:04.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:04.174+0000 7fa9e239b700 1 --2- 172.21.15.31:0/86963931 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa9dc13fef0 0x7fa9dc153f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:04.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:04.174+0000 7fa9e239b700 1 -- 172.21.15.31:0/86963931 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9dc13fb20 msgr2=0x7fa9dc1372d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:04.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:04.174+0000 7fa9e239b700 1 --2- 172.21.15.31:0/86963931 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9dc13fb20 0x7fa9dc1372d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:04.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:04.174+0000 7fa9e239b700 1 -- 172.21.15.31:0/86963931 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9d4009a20 con 0x7fa9dc06aaa0 2023-10-06T02:28:04.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:04.174+0000 7fa9e239b700 1 --2- 172.21.15.31:0/86963931 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9dc06aaa0 0x7fa9dc13a270 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fa9d80093f0 tx=0x7fa9d8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:04.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:04.174+0000 7fa9d27fc700 1 -- 172.21.15.31:0/86963931 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d801cbe0 con 0x7fa9dc06aaa0 2023-10-06T02:28:04.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:04.174+0000 7fa9d27fc700 1 -- 172.21.15.31:0/86963931 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9d801cd40 con 0x7fa9dc06aaa0 2023-10-06T02:28:04.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:04.174+0000 7fa9d27fc700 1 -- 172.21.15.31:0/86963931 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d801a660 con 0x7fa9dc06aaa0 2023-10-06T02:28:04.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:04.174+0000 7fa9e339d700 1 -- 172.21.15.31:0/86963931 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9d8009420 con 0x7fa9dc06aaa0 2023-10-06T02:28:04.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:04.174+0000 7fa9e339d700 1 -- 172.21.15.31:0/86963931 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9dc158270 con 0x7fa9dc06aaa0 2023-10-06T02:28:04.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:04.175+0000 7fa9e339d700 1 -- 172.21.15.31:0/86963931 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9dc04c3c0 con 0x7fa9dc06aaa0 2023-10-06T02:28:04.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:04.176+0000 7fa9d27fc700 1 -- 172.21.15.31:0/86963931 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7fa9d801aa90 con 0x7fa9dc06aaa0 2023-10-06T02:28:04.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:04.177+0000 7fa9d27fc700 1 --2- 172.21.15.31:0/86963931 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa9c0067050 0x7fa9c0069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:04.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:04.177+0000 7fa9d27fc700 1 -- 172.21.15.31:0/86963931 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa9d8086040 con 0x7fa9dc06aaa0 2023-10-06T02:28:04.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:04.178+0000 7fa9e1b9a700 1 --2- 172.21.15.31:0/86963931 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa9c0067050 0x7fa9c0069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:04.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:04.178+0000 7fa9e1b9a700 1 --2- 172.21.15.31:0/86963931 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa9c0067050 0x7fa9c0069500 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc06d7f0 tx=0x7fa9cc001430 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:04.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:04.181+0000 7fa9d27fc700 1 -- 172.21.15.31:0/86963931 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa9d80b75a0 con 0x7fa9dc06aaa0 2023-10-06T02:28:04.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:04.429+0000 7fa9e339d700 1 -- 172.21.15.31:0/86963931 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa9dc158400 con 0x7fa9dc06aaa0 2023-10-06T02:28:04.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:04.430+0000 7fa9d27fc700 1 -- 172.21.15.31:0/86963931 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+4596 (secure 0 0 0) 0x7fa9dc158400 con 0x7fa9dc06aaa0 2023-10-06T02:28:04.430 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:28:04.430 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":56,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5759,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/2470516084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2470516084},{"type":"v1","addr":"172.21.15.62:6834","nonce":2470516084}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":6364,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/790751965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":790751965},{"type":"v1","addr":"172.21.15.31:6834","nonce":790751965}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":6373,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2757737386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2757737386},{"type":"v1","addr":"172.21.15.62:6835","nonce":2757737386}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:27:58.019137+0000","modified":"2023-10-06T02:28:01.555185+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6714},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6714":{"gid":6714,"name":"c","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/1002908930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":1002908930},{"type":"v1","addr":"172.21.15.31:6835","nonce":1002908930}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T02:28:04.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:04.432+0000 7fa9e339d700 1 -- 172.21.15.31:0/86963931 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa9c0067050 msgr2=0x7fa9c0069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:04.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:04.432+0000 7fa9e339d700 1 --2- 172.21.15.31:0/86963931 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa9c0067050 0x7fa9c0069500 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc06d7f0 tx=0x7fa9cc001430 comp rx=0 tx=0).stop 2023-10-06T02:28:04.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:04.432+0000 7fa9e339d700 1 -- 172.21.15.31:0/86963931 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9dc06aaa0 msgr2=0x7fa9dc13a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:04.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:04.432+0000 7fa9e339d700 1 --2- 172.21.15.31:0/86963931 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9dc06aaa0 0x7fa9dc13a270 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fa9d80093f0 tx=0x7fa9d8016000 comp rx=0 tx=0).stop 2023-10-06T02:28:04.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:04.432+0000 7fa9e339d700 1 -- 172.21.15.31:0/86963931 shutdown_connections 2023-10-06T02:28:04.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:04.432+0000 7fa9e339d700 1 --2- 172.21.15.31:0/86963931 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa9c0067050 0x7fa9c0069500 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:04.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:04.432+0000 7fa9e339d700 1 --2- 172.21.15.31:0/86963931 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa9dc13fef0 0x7fa9dc153f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:04.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:04.432+0000 7fa9e339d700 1 --2- 172.21.15.31:0/86963931 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9dc06aaa0 0x7fa9dc13a270 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:04.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:04.432+0000 7fa9e339d700 1 --2- 172.21.15.31:0/86963931 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9dc13fb20 0x7fa9dc1372d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:04.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:04.432+0000 7fa9e339d700 1 -- 172.21.15.31:0/86963931 >> 172.21.15.31:0/86963931 conn(0x7fa9dc06d170 msgr2=0x7fa9dc155dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:04.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:04.432+0000 7fa9e339d700 1 -- 172.21.15.31:0/86963931 shutdown_connections 2023-10-06T02:28:04.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:04.432+0000 7fa9e339d700 1 -- 172.21.15.31:0/86963931 wait complete. 2023-10-06T02:28:04.434 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 56 2023-10-06T02:28:04.512 INFO:tasks.ceph.mds.c.smithi031.stderr:2023-10-06T02:28:04.512+0000 7f97189f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T02:28:04.853 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 56, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T02:27:58.019137+0000', 'modified': '2023-10-06T02:28:01.555185+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6714}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6714': {'gid': 6714, 'name': 'c', 'rank': 0, 'incarnation': 52, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.31:6835/1002908930', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6833', 'nonce': 1002908930}, {'type': 'v1', 'addr': '172.21.15.31:6835', 'nonce': 1002908930}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T02:28:04.854 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T02:28:04.854 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-06T02:28:05.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.045+0000 7f4576b9d700 1 Processor -- start 2023-10-06T02:28:05.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.046+0000 7f4576b9d700 1 -- start start 2023-10-06T02:28:05.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.047+0000 7f4576b9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4570159550 0x7f45700ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:05.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.047+0000 7f4576b9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45700acc00 con 0x7f4570159920 2023-10-06T02:28:05.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.047+0000 7f4576b9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45700abff0 con 0x7f457015dfd0 2023-10-06T02:28:05.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.047+0000 7f4576b9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45700ac2d0 con 0x7f4570159550 2023-10-06T02:28:05.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.047+0000 7f457639c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f4570159920 0x7f45700ad090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58114/0 (socket says 172.21.15.31:58114) 2023-10-06T02:28:05.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.047+0000 7f457639c700 1 -- 172.21.15.31:0/903927188 learned_addr learned my addr 172.21.15.31:0/903927188 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:28:05.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.047+0000 7f4575b9b700 1 --2- 172.21.15.31:0/903927188 >> v2:172.21.15.62:3300/0 conn(0x7f4570159550 0x7f45700ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:05.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.048+0000 7f4574b99700 1 -- 172.21.15.31:0/903927188 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2285513099 0 0) 0x7f45700acc00 con 0x7f4570159920 2023-10-06T02:28:05.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.048+0000 7f4574b99700 1 -- 172.21.15.31:0/903927188 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f455c003570 con 0x7f4570159920 2023-10-06T02:28:05.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.048+0000 7f4574b99700 1 -- 172.21.15.31:0/903927188 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3823493926 0 0) 0x7f45700abff0 con 0x7f457015dfd0 2023-10-06T02:28:05.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.048+0000 7f4574b99700 1 -- 172.21.15.31:0/903927188 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45700acc00 con 0x7f457015dfd0 2023-10-06T02:28:05.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.048+0000 7f4575b9b700 1 -- 172.21.15.31:0/903927188 >> v1:172.21.15.62:6789/0 conn(0x7f457015dfd0 legacy=0x7f45700aedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:05.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.048+0000 7f4575b9b700 1 -- 172.21.15.31:0/903927188 >> v1:172.21.15.31:6789/0 conn(0x7f4570159920 legacy=0x7f45700ad090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:05.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.048+0000 7f4575b9b700 1 -- 172.21.15.31:0/903927188 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45700ac5b0 con 0x7f4570159550 2023-10-06T02:28:05.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.048+0000 7f4575b9b700 1 --2- 172.21.15.31:0/903927188 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4570159550 0x7f45700ceac0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f456c0098e0 tx=0x7f456c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13378756784eb7de server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:05.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.049+0000 7f4574b99700 1 -- 172.21.15.31:0/903927188 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f456c011630 con 0x7f4570159550 2023-10-06T02:28:05.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.049+0000 7f4574b99700 1 -- 172.21.15.31:0/903927188 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f456c011790 con 0x7f4570159550 2023-10-06T02:28:05.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.049+0000 7f4574b99700 1 -- 172.21.15.31:0/903927188 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f456c011b30 con 0x7f4570159550 2023-10-06T02:28:05.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.049+0000 7f4576b9d700 1 -- 172.21.15.31:0/903927188 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4570159550 msgr2=0x7f45700ceac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:05.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.049+0000 7f4576b9d700 1 --2- 172.21.15.31:0/903927188 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4570159550 0x7f45700ceac0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f456c0098e0 tx=0x7f456c0091a0 comp rx=0 tx=0).stop 2023-10-06T02:28:05.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.050+0000 7f4576b9d700 1 -- 172.21.15.31:0/903927188 shutdown_connections 2023-10-06T02:28:05.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.050+0000 7f4576b9d700 1 --2- 172.21.15.31:0/903927188 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4570159550 0x7f45700ceac0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:05.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.050+0000 7f4576b9d700 1 -- 172.21.15.31:0/903927188 >> 172.21.15.31:0/903927188 conn(0x7f457006ce80 msgr2=0x7f45700ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:05.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.050+0000 7f4576b9d700 1 -- 172.21.15.31:0/903927188 shutdown_connections 2023-10-06T02:28:05.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.050+0000 7f4576b9d700 1 -- 172.21.15.31:0/903927188 wait complete. 2023-10-06T02:28:05.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.051+0000 7f4576b9d700 1 Processor -- start 2023-10-06T02:28:05.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.051+0000 7f4576b9d700 1 -- start start 2023-10-06T02:28:05.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.052+0000 7f4576b9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4570159550 0x7f4570146dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:05.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.052+0000 7f4576b9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4570159920 0x7f45700cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:05.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.052+0000 7f4576b9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f457015dfd0 0x7f45700cd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:05.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.052+0000 7f4576b9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45700cc7c0 con 0x7f4570159920 2023-10-06T02:28:05.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.052+0000 7f4576b9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45700cb720 con 0x7f4570159550 2023-10-06T02:28:05.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.052+0000 7f4576b9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45700cba00 con 0x7f457015dfd0 2023-10-06T02:28:05.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.052+0000 7f457539a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4570159920 0x7f45700cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:05.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.053+0000 7f457539a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4570159920 0x7f45700cadb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40540/0 (socket says 172.21.15.31:40540) 2023-10-06T02:28:05.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.053+0000 7f457539a700 1 -- 172.21.15.31:0/1756495241 learned_addr learned my addr 172.21.15.31:0/1756495241 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:05.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.053+0000 7f4575b9b700 1 --2- 172.21.15.31:0/1756495241 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4570159550 0x7f4570146dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:05.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.053+0000 7f457639c700 1 --2- 172.21.15.31:0/1756495241 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f457015dfd0 0x7f45700cd3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:05.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.053+0000 7f457539a700 1 -- 172.21.15.31:0/1756495241 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f457015dfd0 msgr2=0x7f45700cd3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:05.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.053+0000 7f457539a700 1 --2- 172.21.15.31:0/1756495241 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f457015dfd0 0x7f45700cd3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:05.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.053+0000 7f457539a700 1 -- 172.21.15.31:0/1756495241 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4570159550 msgr2=0x7f4570146dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:05.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.053+0000 7f457539a700 1 --2- 172.21.15.31:0/1756495241 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4570159550 0x7f4570146dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:05.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.053+0000 7f457539a700 1 -- 172.21.15.31:0/1756495241 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f456c009420 con 0x7f4570159920 2023-10-06T02:28:05.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.053+0000 7f457639c700 1 --2- 172.21.15.31:0/1756495241 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f457015dfd0 0x7f45700cd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:28:05.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.053+0000 7f457539a700 1 --2- 172.21.15.31:0/1756495241 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4570159920 0x7f45700cadb0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f456000c790 tx=0x7f4560007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:05.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.053+0000 7f4565ffb700 1 -- 172.21.15.31:0/1756495241 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4560010510 con 0x7f4570159920 2023-10-06T02:28:05.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.053+0000 7f4565ffb700 1 -- 172.21.15.31:0/1756495241 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4560010670 con 0x7f4570159920 2023-10-06T02:28:05.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.053+0000 7f4576b9d700 1 -- 172.21.15.31:0/1756495241 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45700cbce0 con 0x7f4570159920 2023-10-06T02:28:05.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.053+0000 7f4565ffb700 1 -- 172.21.15.31:0/1756495241 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4560010a30 con 0x7f4570159920 2023-10-06T02:28:05.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.054+0000 7f4576b9d700 1 -- 172.21.15.31:0/1756495241 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45700cc2a0 con 0x7f4570159920 2023-10-06T02:28:05.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.057+0000 7f4565ffb700 1 -- 172.21.15.31:0/1756495241 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f456001b3b0 con 0x7f4570159920 2023-10-06T02:28:05.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.057+0000 7f4565ffb700 1 --2- 172.21.15.31:0/1756495241 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f4554067050 0x7f4554069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:05.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.057+0000 7f4565ffb700 1 -- 172.21.15.31:0/1756495241 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f45600840f0 con 0x7f4570159920 2023-10-06T02:28:05.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.058+0000 7f4575b9b700 1 --2- 172.21.15.31:0/1756495241 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f4554067050 0x7f4554069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:05.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.059+0000 7f4575b9b700 1 --2- 172.21.15.31:0/1756495241 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f4554067050 0x7f4554069500 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f456c0093f0 tx=0x7f456c017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:05.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.059+0000 7f45865fbb80 1 Processor -- start 2023-10-06T02:28:05.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.059+0000 7f45865fbb80 1 -- start start 2023-10-06T02:28:05.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.060+0000 7f45865fbb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56293195de50 0x562931938cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:05.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.060+0000 7f45865fbb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56293193e2c0 0x562931939200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:05.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.060+0000 7f45865fbb80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56293193fa10 0x562931939740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:05.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.060+0000 7f45865fbb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x562931893a70 con 0x56293195de50 2023-10-06T02:28:05.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.060+0000 7f45865fbb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x562931891270 con 0x56293193e2c0 2023-10-06T02:28:05.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.060+0000 7f45865fbb80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x56293188d060 con 0x56293193fa10 2023-10-06T02:28:05.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.060+0000 7f4575b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56293195de50 0x562931938cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:05.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.060+0000 7f4575b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56293195de50 0x562931938cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40546/0 (socket says 172.21.15.31:40546) 2023-10-06T02:28:05.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.060+0000 7f4575b9b700 1 -- 172.21.15.31:0/3137239273 learned_addr learned my addr 172.21.15.31:0/3137239273 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:05.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.060+0000 7f457539a700 1 --2- 172.21.15.31:0/3137239273 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56293193e2c0 0x562931939200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:05.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.060+0000 7f457639c700 1 --2- 172.21.15.31:0/3137239273 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56293193fa10 0x562931939740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:05.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.061+0000 7f4575b9b700 1 -- 172.21.15.31:0/3137239273 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56293193fa10 msgr2=0x562931939740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:05.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.061+0000 7f4575b9b700 1 --2- 172.21.15.31:0/3137239273 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56293193fa10 0x562931939740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:05.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.061+0000 7f4575b9b700 1 -- 172.21.15.31:0/3137239273 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56293193e2c0 msgr2=0x562931939200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:05.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.061+0000 7f4575b9b700 1 --2- 172.21.15.31:0/3137239273 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56293193e2c0 0x562931939200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:05.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.061+0000 7f4575b9b700 1 -- 172.21.15.31:0/3137239273 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562931727270 con 0x56293195de50 2023-10-06T02:28:05.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.061+0000 7f4575b9b700 1 --2- 172.21.15.31:0/3137239273 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56293195de50 0x562931938cc0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f457006ad70 tx=0x7f456c022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b804363158f5c8c1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:05.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.061+0000 7f455affd700 1 -- 172.21.15.31:0/3137239273 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f456c02c070 con 0x56293195de50 2023-10-06T02:28:05.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.061+0000 7f455affd700 1 -- 172.21.15.31:0/3137239273 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f456c020040 con 0x56293195de50 2023-10-06T02:28:05.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.061+0000 7f455affd700 1 -- 172.21.15.31:0/3137239273 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f456c01ab00 con 0x56293195de50 2023-10-06T02:28:05.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.062+0000 7f45865fbb80 1 -- 172.21.15.31:0/3137239273 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56293195de50 msgr2=0x562931938cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:05.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.062+0000 7f45865fbb80 1 --2- 172.21.15.31:0/3137239273 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56293195de50 0x562931938cc0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f457006ad70 tx=0x7f456c022000 comp rx=0 tx=0).stop 2023-10-06T02:28:05.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.062+0000 7f45865fbb80 1 -- 172.21.15.31:0/3137239273 shutdown_connections 2023-10-06T02:28:05.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.062+0000 7f45865fbb80 1 --2- 172.21.15.31:0/3137239273 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56293193fa10 0x562931939740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:05.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.062+0000 7f45865fbb80 1 --2- 172.21.15.31:0/3137239273 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56293195de50 0x562931938cc0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:05.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.062+0000 7f45865fbb80 1 --2- 172.21.15.31:0/3137239273 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56293193e2c0 0x562931939200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:05.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.062+0000 7f45865fbb80 1 -- 172.21.15.31:0/3137239273 >> 172.21.15.31:0/3137239273 conn(0x56293193de80 msgr2=0x56293193e6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:05.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.062+0000 7f45865fbb80 1 -- 172.21.15.31:0/3137239273 shutdown_connections 2023-10-06T02:28:05.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.062+0000 7f45865fbb80 1 -- 172.21.15.31:0/3137239273 wait complete. 2023-10-06T02:28:05.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.063+0000 7f45865fbb80 1 -- start start 2023-10-06T02:28:05.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.063+0000 7f45865fbb80 1 Processor -- start 2023-10-06T02:28:05.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.064+0000 7f45865fbb80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56293193e2c0 0x562931965b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:05.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.064+0000 7f45865fbb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56293193fa10 0x562931984520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:05.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.064+0000 7f45865fbb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56293195de50 0x562931939120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:05.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.064+0000 7f45865fbb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x562931838890 con 0x56293193fa10 2023-10-06T02:28:05.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.064+0000 7f45865fbb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x562931853ea0 con 0x56293195de50 2023-10-06T02:28:05.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.064+0000 7f45865fbb80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x56293170fe30 con 0x56293193e2c0 2023-10-06T02:28:05.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.064+0000 7f45865fbb80 10 client.0 _ll_register_callbacks cb 0x562931900020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T02:28:05.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.064+0000 7f457539a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56293193fa10 0x562931984520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:05.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.064+0000 7f457539a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56293193fa10 0x562931984520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40548/0 (socket says 172.21.15.31:40548) 2023-10-06T02:28:05.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.064+0000 7f457539a700 1 -- 172.21.15.31:0/1263561458 learned_addr learned my addr 172.21.15.31:0/1263561458 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:05.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.064+0000 7f4575b9b700 1 --2- 172.21.15.31:0/1263561458 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56293193e2c0 0x562931965b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:05.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.064+0000 7f457639c700 1 --2- 172.21.15.31:0/1263561458 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56293195de50 0x562931939120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:05.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.065+0000 7f457539a700 1 -- 172.21.15.31:0/1263561458 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56293193e2c0 msgr2=0x562931965b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:05.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.065+0000 7f457539a700 1 --2- 172.21.15.31:0/1263561458 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56293193e2c0 0x562931965b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:05.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.065+0000 7f457539a700 1 -- 172.21.15.31:0/1263561458 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56293195de50 msgr2=0x562931939120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:05.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.065+0000 7f457539a700 1 --2- 172.21.15.31:0/1263561458 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56293195de50 0x562931939120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:05.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.065+0000 7f457539a700 1 -- 172.21.15.31:0/1263561458 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45700ac5b0 con 0x56293193fa10 2023-10-06T02:28:05.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.065+0000 7f457539a700 1 --2- 172.21.15.31:0/1263561458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56293193fa10 0x562931984520 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f4560007fd0 tx=0x7f456005a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c66fc09e8be57ad server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:05.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.065+0000 7f454ffff700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T02:28:05.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.065+0000 7f454ffff700 1 -- 172.21.15.31:0/1263561458 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f456005bcb0 con 0x56293193fa10 2023-10-06T02:28:05.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.065+0000 7f454ffff700 1 -- 172.21.15.31:0/1263561458 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f45600708d0 con 0x56293193fa10 2023-10-06T02:28:05.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.065+0000 7f45865fbb80 1 -- 172.21.15.31:0/1263561458 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562931931940 con 0x56293193fa10 2023-10-06T02:28:05.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.065+0000 7f454ffff700 1 -- 172.21.15.31:0/1263561458 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4560054410 con 0x56293193fa10 2023-10-06T02:28:05.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.065+0000 7f45865fbb80 10 client.6846 fetch_fsmap learned FSMap version 56 2023-10-06T02:28:05.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.065+0000 7f45865fbb80 1 -- 172.21.15.31:0/1263561458 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) v3 -- 0x5629316c9fe0 con 0x56293193fa10 2023-10-06T02:28:05.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.066+0000 7f454ffff700 1 -- 172.21.15.31:0/1263561458 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562931931940 con 0x56293193fa10 2023-10-06T02:28:05.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.066+0000 7f454ffff700 1 -- 172.21.15.31:0/1263561458 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 56) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f456005b820 con 0x56293193fa10 2023-10-06T02:28:05.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.066+0000 7f45865fbb80 10 client.6846 fetch_fsmap finished waiting for FSMap version 56 2023-10-06T02:28:05.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.066+0000 7f45865fbb80 10 client.6846 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T02:28:05.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.066+0000 7f45865fbb80 20 client.6846 populate_metadata read hostname 'smithi031' 2023-10-06T02:28:05.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.066+0000 7f45865fbb80 1 --2- 172.21.15.31:0/1263561458 >> [v2:172.21.15.31:6833/1002908930,v1:172.21.15.31:6835/1002908930] conn(0x562931987390 0x562931987760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:05.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.066+0000 7f45865fbb80 4 client.6846 mds_command: new command op to 6714 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T02:28:05.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.066+0000 7f45865fbb80 1 -- 172.21.15.31:0/1263561458 --> [v2:172.21.15.31:6833/1002908930,v1:172.21.15.31:6835/1002908930] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56293190e870 con 0x562931987390 2023-10-06T02:28:05.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.066+0000 7f4575b9b700 1 --2- 172.21.15.31:0/1263561458 >> [v2:172.21.15.31:6833/1002908930,v1:172.21.15.31:6835/1002908930] conn(0x562931987390 0x562931987760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:05.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.066+0000 7f4575b9b700 1 --2- 172.21.15.31:0/1263561458 >> [v2:172.21.15.31:6833/1002908930,v1:172.21.15.31:6835/1002908930] conn(0x562931987390 0x562931987760 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0ffaf5a60a776f1 server_cookie=2b4a8b38a8cad789 in_seq=0 out_seq=0 2023-10-06T02:28:05.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.066+0000 7f454ffff700 10 client.6846 ms_handle_connect on v2:172.21.15.31:6833/1002908930 2023-10-06T02:28:05.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.067+0000 7f454ffff700 1 -- 172.21.15.31:0/1263561458 <== mds.0 v2:172.21.15.31:6833/1002908930 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x56293190e870 con 0x562931987390 2023-10-06T02:28:05.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.067+0000 7f454ffff700 10 client.6846 handle_command_reply: tid=0 2023-10-06T02:28:05.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.067+0000 7f45865fbb80 1 client.6846 shutdown 2023-10-06T02:28:05.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.067+0000 7f45865fbb80 1 -- 172.21.15.31:0/1263561458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56293193fa10 msgr2=0x562931984520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:05.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.067+0000 7f45865fbb80 1 --2- 172.21.15.31:0/1263561458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56293193fa10 0x562931984520 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f4560007fd0 tx=0x7f456005a040 comp rx=0 tx=0).stop 2023-10-06T02:28:05.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.068+0000 7f45865fbb80 1 -- 172.21.15.31:0/1263561458 shutdown_connections 2023-10-06T02:28:05.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.068+0000 7f45865fbb80 1 --2- 172.21.15.31:0/1263561458 >> [v2:172.21.15.31:6833/1002908930,v1:172.21.15.31:6835/1002908930] conn(0x562931987390 0x562931987760 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:05.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.068+0000 7f45865fbb80 1 --2- 172.21.15.31:0/1263561458 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56293195de50 0x562931939120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:05.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.068+0000 7f45865fbb80 1 --2- 172.21.15.31:0/1263561458 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56293193e2c0 0x562931965b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:05.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.068+0000 7f454ffff700 0 client.6846 ms_handle_reset on v2:172.21.15.31:6833/1002908930 2023-10-06T02:28:05.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.068+0000 7f45865fbb80 1 --2- 172.21.15.31:0/1263561458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56293193fa10 0x562931984520 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:05.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.068+0000 7f45865fbb80 1 -- 172.21.15.31:0/1263561458 >> 172.21.15.31:0/1263561458 conn(0x56293193de80 msgr2=0x56293193ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:05.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.068+0000 7f45865fbb80 1 -- 172.21.15.31:0/1263561458 shutdown_connections 2023-10-06T02:28:05.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.068+0000 7f45865fbb80 1 -- 172.21.15.31:0/1263561458 wait complete. 2023-10-06T02:28:05.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.068+0000 7f45865fbb80 20 client.6846 trim_cache size 0 max 16384 2023-10-06T02:28:05.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.085+0000 7f45865fbb80 1 Processor -- start 2023-10-06T02:28:05.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.085+0000 7f45865fbb80 1 -- start start 2023-10-06T02:28:05.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.086+0000 7f45865fbb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56293193e2c0 0x56293199bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:05.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.086+0000 7f45865fbb80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56293193fa10 0x562931965d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:05.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.086+0000 7f45865fbb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56293195de50 0x562931968b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:05.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.086+0000 7f45865fbb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x562931986a90 con 0x56293193e2c0 2023-10-06T02:28:05.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.086+0000 7f45865fbb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56293195fce0 con 0x56293195de50 2023-10-06T02:28:05.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.086+0000 7f45865fbb80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x56293193d8c0 con 0x56293193fa10 2023-10-06T02:28:05.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.086+0000 7f4575b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56293193e2c0 0x56293199bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:05.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.086+0000 7f4575b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56293193e2c0 0x56293199bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40552/0 (socket says 172.21.15.31:40552) 2023-10-06T02:28:05.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.086+0000 7f4575b9b700 1 -- 172.21.15.31:0/2210324881 learned_addr learned my addr 172.21.15.31:0/2210324881 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:05.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.086+0000 7f457639c700 1 --2- 172.21.15.31:0/2210324881 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56293195de50 0x562931968b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:05.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.087+0000 7f457539a700 1 --2- 172.21.15.31:0/2210324881 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56293193fa10 0x562931965d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:05.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.087+0000 7f4575b9b700 1 -- 172.21.15.31:0/2210324881 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56293193fa10 msgr2=0x562931965d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:05.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.087+0000 7f4575b9b700 1 --2- 172.21.15.31:0/2210324881 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56293193fa10 0x562931965d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:05.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.087+0000 7f4575b9b700 1 -- 172.21.15.31:0/2210324881 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56293195de50 msgr2=0x562931968b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:05.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.087+0000 7f4575b9b700 1 --2- 172.21.15.31:0/2210324881 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56293195de50 0x562931968b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:05.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.087+0000 7f4575b9b700 1 -- 172.21.15.31:0/2210324881 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56293185b780 con 0x56293193e2c0 2023-10-06T02:28:05.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.087+0000 7f4575b9b700 1 --2- 172.21.15.31:0/2210324881 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56293193e2c0 0x56293199bef0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f456c034040 tx=0x7f456c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2995082348723ad server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:05.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.087+0000 7f455b7fe700 1 -- 172.21.15.31:0/2210324881 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f456c02c040 con 0x56293193e2c0 2023-10-06T02:28:05.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.087+0000 7f455b7fe700 1 -- 172.21.15.31:0/2210324881 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f456c000c90 con 0x56293193e2c0 2023-10-06T02:28:05.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.087+0000 7f455b7fe700 1 -- 172.21.15.31:0/2210324881 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f456c01a8d0 con 0x56293193e2c0 2023-10-06T02:28:05.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.088+0000 7f45865fbb80 1 -- 172.21.15.31:0/2210324881 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56293193e2c0 msgr2=0x56293199bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:05.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.088+0000 7f45865fbb80 1 --2- 172.21.15.31:0/2210324881 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56293193e2c0 0x56293199bef0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f456c034040 tx=0x7f456c014000 comp rx=0 tx=0).stop 2023-10-06T02:28:05.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.088+0000 7f45865fbb80 1 -- 172.21.15.31:0/2210324881 shutdown_connections 2023-10-06T02:28:05.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.088+0000 7f45865fbb80 1 --2- 172.21.15.31:0/2210324881 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56293195de50 0x562931968b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:05.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.088+0000 7f45865fbb80 1 --2- 172.21.15.31:0/2210324881 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56293193e2c0 0x56293199bef0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:05.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.088+0000 7f45865fbb80 1 --2- 172.21.15.31:0/2210324881 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56293193fa10 0x562931965d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:05.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.088+0000 7f45865fbb80 1 -- 172.21.15.31:0/2210324881 >> 172.21.15.31:0/2210324881 conn(0x56293193de80 msgr2=0x562931940010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:05.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.088+0000 7f45865fbb80 1 -- 172.21.15.31:0/2210324881 shutdown_connections 2023-10-06T02:28:05.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.088+0000 7f45865fbb80 1 -- 172.21.15.31:0/2210324881 wait complete. 2023-10-06T02:28:05.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.088+0000 7f45865fbb80 1 -- start start 2023-10-06T02:28:05.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.089+0000 7f45865fbb80 1 Processor -- start 2023-10-06T02:28:05.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.090+0000 7f45865fbb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56293193e2c0 0x562931968f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:05.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.090+0000 7f45865fbb80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56293193fa10 0x562931962dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:05.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.090+0000 7f45865fbb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56293195de50 0x562931963300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:05.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.090+0000 7f45865fbb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629319849b0 con 0x56293195de50 2023-10-06T02:28:05.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.090+0000 7f45865fbb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x562931964100 con 0x56293193e2c0 2023-10-06T02:28:05.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.090+0000 7f45865fbb80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x56293195ea60 con 0x56293193fa10 2023-10-06T02:28:05.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.090+0000 7f45865fbb80 10 client.0 _ll_register_callbacks cb 0x562931900020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T02:28:05.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.090+0000 7f457639c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56293195de50 0x562931963300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:05.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.090+0000 7f457639c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56293195de50 0x562931963300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40562/0 (socket says 172.21.15.31:40562) 2023-10-06T02:28:05.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.090+0000 7f457639c700 1 -- 172.21.15.31:0/65080511 learned_addr learned my addr 172.21.15.31:0/65080511 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:05.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.090+0000 7f457539a700 1 --2- 172.21.15.31:0/65080511 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56293193fa10 0x562931962dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:05.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.090+0000 7f4575b9b700 1 --2- 172.21.15.31:0/65080511 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56293193e2c0 0x562931968f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:05.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.090+0000 7f457639c700 1 -- 172.21.15.31:0/65080511 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56293193fa10 msgr2=0x562931962dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:05.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.090+0000 7f457639c700 1 --2- 172.21.15.31:0/65080511 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56293193fa10 0x562931962dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:05.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.090+0000 7f457639c700 1 -- 172.21.15.31:0/65080511 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56293193e2c0 msgr2=0x562931968f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:05.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.090+0000 7f457639c700 1 --2- 172.21.15.31:0/65080511 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56293193e2c0 0x562931968f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:05.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.090+0000 7f457639c700 1 -- 172.21.15.31:0/65080511 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562931727270 con 0x56293195de50 2023-10-06T02:28:05.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.091+0000 7f457639c700 1 --2- 172.21.15.31:0/65080511 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56293195de50 0x562931963300 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f4568015ba0 tx=0x7f4568009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d24c2199bf47d99 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:05.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.091+0000 7f454ffff700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T02:28:05.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.091+0000 7f454ffff700 1 -- 172.21.15.31:0/65080511 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4568007760 con 0x56293195de50 2023-10-06T02:28:05.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.091+0000 7f454ffff700 1 -- 172.21.15.31:0/65080511 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f45680078c0 con 0x56293195de50 2023-10-06T02:28:05.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.091+0000 7f45865fbb80 1 -- 172.21.15.31:0/65080511 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f456c020040 con 0x56293195de50 2023-10-06T02:28:05.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.091+0000 7f454ffff700 1 -- 172.21.15.31:0/65080511 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4568017430 con 0x56293195de50 2023-10-06T02:28:05.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.091+0000 7f45865fbb80 10 client.6852 fetch_fsmap learned FSMap version 56 2023-10-06T02:28:05.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.091+0000 7f45865fbb80 1 -- 172.21.15.31:0/65080511 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) v3 -- 0x562931842470 con 0x56293195de50 2023-10-06T02:28:05.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.091+0000 7f454ffff700 1 -- 172.21.15.31:0/65080511 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f456c020040 con 0x56293195de50 2023-10-06T02:28:05.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.091+0000 7f454ffff700 1 -- 172.21.15.31:0/65080511 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 56) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f456801d020 con 0x56293195de50 2023-10-06T02:28:05.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.091+0000 7f45865fbb80 10 client.6852 fetch_fsmap finished waiting for FSMap version 56 2023-10-06T02:28:05.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.091+0000 7f45865fbb80 10 client.6852 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T02:28:05.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.091+0000 7f45865fbb80 20 client.6852 populate_metadata read hostname 'smithi031' 2023-10-06T02:28:05.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.091+0000 7f45865fbb80 1 --2- 172.21.15.31:0/65080511 >> [v2:172.21.15.31:6833/1002908930,v1:172.21.15.31:6835/1002908930] conn(0x5629319e29e0 0x56293196b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:05.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.091+0000 7f45865fbb80 4 client.6852 mds_command: new command op to 6714 tid=0[{"prefix": "status"}] 2023-10-06T02:28:05.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.091+0000 7f45865fbb80 1 -- 172.21.15.31:0/65080511 --> [v2:172.21.15.31:6833/1002908930,v1:172.21.15.31:6835/1002908930] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56293182fe00 con 0x5629319e29e0 2023-10-06T02:28:05.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.092+0000 7f4575b9b700 1 --2- 172.21.15.31:0/65080511 >> [v2:172.21.15.31:6833/1002908930,v1:172.21.15.31:6835/1002908930] conn(0x5629319e29e0 0x56293196b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:05.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.092+0000 7f454ffff700 10 client.6852 ms_handle_connect on v2:172.21.15.31:6833/1002908930 2023-10-06T02:28:05.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.092+0000 7f4575b9b700 1 --2- 172.21.15.31:0/65080511 >> [v2:172.21.15.31:6833/1002908930,v1:172.21.15.31:6835/1002908930] conn(0x5629319e29e0 0x56293196b520 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=399b30cd10d5c0a5 server_cookie=be8b2d2bcc5b635c in_seq=0 out_seq=0 2023-10-06T02:28:05.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.092+0000 7f454ffff700 1 -- 172.21.15.31:0/65080511 <== mds.0 v2:172.21.15.31:6833/1002908930 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x56293182fe00 con 0x5629319e29e0 2023-10-06T02:28:05.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.092+0000 7f454ffff700 10 client.6852 handle_command_reply: tid=0 2023-10-06T02:28:05.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.092+0000 7f45865fbb80 1 client.6852 shutdown 2023-10-06T02:28:05.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.092+0000 7f45865fbb80 1 -- 172.21.15.31:0/65080511 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56293195de50 msgr2=0x562931963300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:05.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.092+0000 7f45865fbb80 1 --2- 172.21.15.31:0/65080511 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56293195de50 0x562931963300 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f4568015ba0 tx=0x7f4568009040 comp rx=0 tx=0).stop 2023-10-06T02:28:05.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.093+0000 7f45865fbb80 1 -- 172.21.15.31:0/65080511 shutdown_connections 2023-10-06T02:28:05.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.093+0000 7f45865fbb80 1 --2- 172.21.15.31:0/65080511 >> [v2:172.21.15.31:6833/1002908930,v1:172.21.15.31:6835/1002908930] conn(0x5629319e29e0 0x56293196b520 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:05.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.093+0000 7f45865fbb80 1 --2- 172.21.15.31:0/65080511 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56293195de50 0x562931963300 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:05.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.093+0000 7f45865fbb80 1 --2- 172.21.15.31:0/65080511 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56293193e2c0 0x562931968f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:05.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.093+0000 7f45865fbb80 1 --2- 172.21.15.31:0/65080511 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56293193fa10 0x562931962dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:05.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.093+0000 7f454ffff700 0 client.6852 ms_handle_reset on v2:172.21.15.31:6833/1002908930 2023-10-06T02:28:05.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.093+0000 7f45865fbb80 1 -- 172.21.15.31:0/65080511 >> 172.21.15.31:0/65080511 conn(0x56293193de80 msgr2=0x56293193e690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:05.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.093+0000 7f45865fbb80 1 -- 172.21.15.31:0/65080511 shutdown_connections 2023-10-06T02:28:05.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.093+0000 7f45865fbb80 1 -- 172.21.15.31:0/65080511 wait complete. 2023-10-06T02:28:05.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.093+0000 7f45865fbb80 20 client.6852 trim_cache size 0 max 16384 2023-10-06T02:28:05.099 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T02:28:05.100 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "d6fcf512-874b-4259-b90a-30ae070f00a9", 2023-10-06T02:28:05.100 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 0, 2023-10-06T02:28:05.100 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 6714, 2023-10-06T02:28:05.100 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2023-10-06T02:28:05.100 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2023-10-06T02:28:05.100 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2023-10-06T02:28:05.100 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 6.5630154940000001, 2023-10-06T02:28:05.100 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 56, 2023-10-06T02:28:05.100 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 87, 2023-10-06T02:28:05.100 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 87, 2023-10-06T02:28:05.100 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 12.007483923000001 2023-10-06T02:28:05.101 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T02:28:05.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.093+0000 7f4576b9d700 1 -- 172.21.15.31:0/1756495241 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f4554067050 msgr2=0x7f4554069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:05.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.093+0000 7f4576b9d700 1 --2- 172.21.15.31:0/1756495241 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f4554067050 0x7f4554069500 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f456c0093f0 tx=0x7f456c017000 comp rx=0 tx=0).stop 2023-10-06T02:28:05.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.093+0000 7f4576b9d700 1 -- 172.21.15.31:0/1756495241 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4570159920 msgr2=0x7f45700cadb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:05.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.093+0000 7f4576b9d700 1 --2- 172.21.15.31:0/1756495241 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4570159920 0x7f45700cadb0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f456000c790 tx=0x7f4560007e10 comp rx=0 tx=0).stop 2023-10-06T02:28:05.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.094+0000 7f4576b9d700 1 -- 172.21.15.31:0/1756495241 shutdown_connections 2023-10-06T02:28:05.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.094+0000 7f4576b9d700 1 --2- 172.21.15.31:0/1756495241 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f4554067050 0x7f4554069500 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:05.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.094+0000 7f4576b9d700 1 --2- 172.21.15.31:0/1756495241 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f457015dfd0 0x7f45700cd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:05.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.094+0000 7f4576b9d700 1 --2- 172.21.15.31:0/1756495241 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4570159550 0x7f4570146dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:05.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.094+0000 7f4576b9d700 1 --2- 172.21.15.31:0/1756495241 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4570159920 0x7f45700cadb0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:05.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.094+0000 7f4576b9d700 1 -- 172.21.15.31:0/1756495241 >> 172.21.15.31:0/1756495241 conn(0x7f457006ce80 msgr2=0x7f457006e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:05.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.094+0000 7f4576b9d700 1 -- 172.21.15.31:0/1756495241 shutdown_connections 2023-10-06T02:28:05.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.094+0000 7f4576b9d700 1 -- 172.21.15.31:0/1756495241 wait complete. 2023-10-06T02:28:05.111 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T02:28:05.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.298+0000 7ff9b7fff700 1 Processor -- start 2023-10-06T02:28:05.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.298+0000 7ff9b7fff700 1 -- start start 2023-10-06T02:28:05.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.299+0000 7ff9b7fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff9b8159540 0x7ff9b80c1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:05.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.299+0000 7ff9b7fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff9b80c6be0 0x7ff9b80c7020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:05.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.299+0000 7ff9b7fff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9b80c7560 con 0x7ff9b8159910 2023-10-06T02:28:05.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.299+0000 7ff9b7fff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9b8068250 con 0x7ff9b8159540 2023-10-06T02:28:05.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.299+0000 7ff9b7fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b8068530 con 0x7ff9b80c6be0 2023-10-06T02:28:05.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.300+0000 7ff9b6ffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff9b8159540 0x7ff9b80c1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:05.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.300+0000 7ff9b6ffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff9b8159540 0x7ff9b80c1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40564/0 (socket says 172.21.15.31:40564) 2023-10-06T02:28:05.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.300+0000 7ff9b6ffd700 1 -- 172.21.15.31:0/2966658102 learned_addr learned my addr 172.21.15.31:0/2966658102 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:05.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.300+0000 7ff9b77fe700 1 --2- 172.21.15.31:0/2966658102 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff9b80c6be0 0x7ff9b80c7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:05.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.300+0000 7ff9b6ffd700 1 -- 172.21.15.31:0/2966658102 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff9b80c6be0 msgr2=0x7ff9b80c7020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:05.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.300+0000 7ff9b6ffd700 1 --2- 172.21.15.31:0/2966658102 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff9b80c6be0 0x7ff9b80c7020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:05.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.300+0000 7ff9b6ffd700 1 -- 172.21.15.31:0/2966658102 >> v1:172.21.15.62:6789/0 conn(0x7ff9b8159910 legacy=0x7ff9b80677b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:05.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.300+0000 7ff9b6ffd700 1 -- 172.21.15.31:0/2966658102 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9b8068810 con 0x7ff9b8159540 2023-10-06T02:28:05.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.301+0000 7ff9b6ffd700 1 --2- 172.21.15.31:0/2966658102 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9b8159540 0x7ff9b80c1340 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7ff9a80099b0 tx=0x7ff9a8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f43ee43ec905f70 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:05.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.301+0000 7ff9b5ffb700 1 -- 172.21.15.31:0/2966658102 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a800b860 con 0x7ff9b8159540 2023-10-06T02:28:05.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.301+0000 7ff9b5ffb700 1 -- 172.21.15.31:0/2966658102 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9a800b9c0 con 0x7ff9b8159540 2023-10-06T02:28:05.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.301+0000 7ff9b5ffb700 1 -- 172.21.15.31:0/2966658102 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a800bd60 con 0x7ff9b8159540 2023-10-06T02:28:05.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.301+0000 7ff9b7fff700 1 -- 172.21.15.31:0/2966658102 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9b8159540 msgr2=0x7ff9b80c1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:05.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.301+0000 7ff9b7fff700 1 --2- 172.21.15.31:0/2966658102 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9b8159540 0x7ff9b80c1340 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7ff9a80099b0 tx=0x7ff9a8009250 comp rx=0 tx=0).stop 2023-10-06T02:28:05.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.301+0000 7ff9b7fff700 1 -- 172.21.15.31:0/2966658102 shutdown_connections 2023-10-06T02:28:05.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.301+0000 7ff9b7fff700 1 --2- 172.21.15.31:0/2966658102 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff9b80c6be0 0x7ff9b80c7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:05.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.301+0000 7ff9b7fff700 1 --2- 172.21.15.31:0/2966658102 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9b8159540 0x7ff9b80c1340 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:05.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.301+0000 7ff9b7fff700 1 -- 172.21.15.31:0/2966658102 >> 172.21.15.31:0/2966658102 conn(0x7ff9b806f310 msgr2=0x7ff9b80cc5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:05.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.301+0000 7ff9b7fff700 1 -- 172.21.15.31:0/2966658102 shutdown_connections 2023-10-06T02:28:05.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.302+0000 7ff9b7fff700 1 -- 172.21.15.31:0/2966658102 wait complete. 2023-10-06T02:28:05.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.303+0000 7ff9b7fff700 1 Processor -- start 2023-10-06T02:28:05.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.303+0000 7ff9b7fff700 1 -- start start 2023-10-06T02:28:05.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.304+0000 7ff9b7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff9b80c6be0 0x7ff9b80b0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:05.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.304+0000 7ff9b7fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff9b8159540 0x7ff9b814d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:05.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.304+0000 7ff9b7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9b8159910 0x7ff9b806b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:05.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.304+0000 7ff9b7fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b80c3ae0 con 0x7ff9b8159910 2023-10-06T02:28:05.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.304+0000 7ff9b7fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b80c3dc0 con 0x7ff9b80c6be0 2023-10-06T02:28:05.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.304+0000 7ff9b7fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b80c40a0 con 0x7ff9b8159540 2023-10-06T02:28:05.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.304+0000 7ff9b77fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9b8159910 0x7ff9b806b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:05.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.304+0000 7ff9b77fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9b8159910 0x7ff9b806b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40580/0 (socket says 172.21.15.31:40580) 2023-10-06T02:28:05.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.304+0000 7ff9b77fe700 1 -- 172.21.15.31:0/3061756001 learned_addr learned my addr 172.21.15.31:0/3061756001 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:05.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.304+0000 7ff9b67fc700 1 --2- 172.21.15.31:0/3061756001 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff9b8159540 0x7ff9b814d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:05.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.304+0000 7ff9b6ffd700 1 --2- 172.21.15.31:0/3061756001 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff9b80c6be0 0x7ff9b80b0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:05.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.305+0000 7ff9b77fe700 1 -- 172.21.15.31:0/3061756001 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff9b8159540 msgr2=0x7ff9b814d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:05.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.305+0000 7ff9b77fe700 1 --2- 172.21.15.31:0/3061756001 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff9b8159540 0x7ff9b814d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:05.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.305+0000 7ff9b77fe700 1 -- 172.21.15.31:0/3061756001 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff9b80c6be0 msgr2=0x7ff9b80b0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:05.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.305+0000 7ff9b77fe700 1 --2- 172.21.15.31:0/3061756001 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff9b80c6be0 0x7ff9b80b0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:05.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.305+0000 7ff9b77fe700 1 -- 172.21.15.31:0/3061756001 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9a80094f0 con 0x7ff9b8159910 2023-10-06T02:28:05.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.305+0000 7ff9b77fe700 1 --2- 172.21.15.31:0/3061756001 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9b8159910 0x7ff9b806b410 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7ff9b0007f20 tx=0x7ff9b0009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:05.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.305+0000 7ff99effd700 1 -- 172.21.15.31:0/3061756001 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b000ed30 con 0x7ff9b8159910 2023-10-06T02:28:05.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.305+0000 7ff99effd700 1 -- 172.21.15.31:0/3061756001 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9b0014420 con 0x7ff9b8159910 2023-10-06T02:28:05.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.305+0000 7ff9b7fff700 1 -- 172.21.15.31:0/3061756001 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9b80c4380 con 0x7ff9b8159910 2023-10-06T02:28:05.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.305+0000 7ff99effd700 1 -- 172.21.15.31:0/3061756001 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b00147e0 con 0x7ff9b8159910 2023-10-06T02:28:05.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.305+0000 7ff9b7fff700 1 -- 172.21.15.31:0/3061756001 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9b814d7b0 con 0x7ff9b8159910 2023-10-06T02:28:05.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.306+0000 7ff9b7fff700 1 -- 172.21.15.31:0/3061756001 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9b80c4570 con 0x7ff9b8159910 2023-10-06T02:28:05.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.307+0000 7ff99effd700 1 -- 172.21.15.31:0/3061756001 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7ff9b0014b50 con 0x7ff9b8159910 2023-10-06T02:28:05.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.308+0000 7ff99effd700 1 --2- 172.21.15.31:0/3061756001 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff9a4067050 0x7ff9a4069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:05.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.308+0000 7ff99effd700 1 -- 172.21.15.31:0/3061756001 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff9b0084fe0 con 0x7ff9b8159910 2023-10-06T02:28:05.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.308+0000 7ff9b6ffd700 1 --2- 172.21.15.31:0/3061756001 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff9a4067050 0x7ff9a4069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:05.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.309+0000 7ff9b6ffd700 1 --2- 172.21.15.31:0/3061756001 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff9a4067050 0x7ff9a4069500 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff9a80099b0 tx=0x7ff9a8017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:05.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.312+0000 7ff99effd700 1 -- 172.21.15.31:0/3061756001 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff9b0051f80 con 0x7ff9b8159910 2023-10-06T02:28:05.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.565+0000 7ff9b7fff700 1 -- 172.21.15.31:0/3061756001 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff9b804c3c0 con 0x7ff9b8159910 2023-10-06T02:28:05.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.566+0000 7ff99effd700 1 -- 172.21.15.31:0/3061756001 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v56) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff9b00555a0 con 0x7ff9b8159910 2023-10-06T02:28:05.566 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T02:28:05.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.568+0000 7ff9b7fff700 1 -- 172.21.15.31:0/3061756001 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff9a4067050 msgr2=0x7ff9a4069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:05.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.568+0000 7ff9b7fff700 1 --2- 172.21.15.31:0/3061756001 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff9a4067050 0x7ff9a4069500 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff9a80099b0 tx=0x7ff9a8017000 comp rx=0 tx=0).stop 2023-10-06T02:28:05.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.568+0000 7ff9b7fff700 1 -- 172.21.15.31:0/3061756001 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9b8159910 msgr2=0x7ff9b806b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:05.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.568+0000 7ff9b7fff700 1 --2- 172.21.15.31:0/3061756001 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9b8159910 0x7ff9b806b410 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7ff9b0007f20 tx=0x7ff9b0009860 comp rx=0 tx=0).stop 2023-10-06T02:28:05.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.568+0000 7ff9b7fff700 1 -- 172.21.15.31:0/3061756001 shutdown_connections 2023-10-06T02:28:05.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.568+0000 7ff9b7fff700 1 --2- 172.21.15.31:0/3061756001 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9b8159910 0x7ff9b806b410 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:05.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.568+0000 7ff9b7fff700 1 --2- 172.21.15.31:0/3061756001 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff9b80c6be0 0x7ff9b80b0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:05.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.568+0000 7ff9b7fff700 1 --2- 172.21.15.31:0/3061756001 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff9a4067050 0x7ff9a4069500 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:05.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.568+0000 7ff9b7fff700 1 --2- 172.21.15.31:0/3061756001 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff9b8159540 0x7ff9b814d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:05.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.568+0000 7ff9b7fff700 1 -- 172.21.15.31:0/3061756001 >> 172.21.15.31:0/3061756001 conn(0x7ff9b806f310 msgr2=0x7ff9b80ca530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:05.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.568+0000 7ff9b7fff700 1 -- 172.21.15.31:0/3061756001 shutdown_connections 2023-10-06T02:28:05.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:05.569+0000 7ff9b7fff700 1 -- 172.21.15.31:0/3061756001 wait complete. 2023-10-06T02:28:05.587 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T02:28:05.587 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T02:28:05.587 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi031.front.sepia.ceph.com 2023-10-06T02:28:05.587 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T02:28:05.587 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T02:28:05.587 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T02:28:05.587 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T02:28:05.587 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T02:28:05.587 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T02:28:05.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:28:05.588 DEBUG:teuthology.orchestra.run.smithi031:> ip addr 2023-10-06T02:28:05.625 INFO:teuthology.orchestra.run.smithi031.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T02:28:05.626 INFO:teuthology.orchestra.run.smithi031.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T02:28:05.626 INFO:teuthology.orchestra.run.smithi031.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T02:28:05.626 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:28:05.626 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 ::1/128 scope host 2023-10-06T02:28:05.626 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:28:05.626 INFO:teuthology.orchestra.run.smithi031.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T02:28:05.626 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cc brd ff:ff:ff:ff:ff:ff 2023-10-06T02:28:05.626 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f0 2023-10-06T02:28:05.626 INFO:teuthology.orchestra.run.smithi031.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T02:28:05.626 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cd brd ff:ff:ff:ff:ff:ff 2023-10-06T02:28:05.626 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f1 2023-10-06T02:28:05.627 INFO:teuthology.orchestra.run.smithi031.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T02:28:05.627 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:ba brd ff:ff:ff:ff:ff:ff 2023-10-06T02:28:05.627 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f0 2023-10-06T02:28:05.627 INFO:teuthology.orchestra.run.smithi031.stdout: inet 172.21.15.31/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T02:28:05.627 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft 41970sec preferred_lft 41970sec 2023-10-06T02:28:05.627 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::ae1f:6bff:fea5:caba/64 scope link 2023-10-06T02:28:05.627 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:28:05.627 INFO:teuthology.orchestra.run.smithi031.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T02:28:05.627 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:bb brd ff:ff:ff:ff:ff:ff 2023-10-06T02:28:05.627 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f1 2023-10-06T02:28:05.627 INFO:teuthology.orchestra.run.smithi031.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T02:28:05.627 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 16:d7:88:2d:d0:c2 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:28:05.627 INFO:teuthology.orchestra.run.smithi031.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T02:28:05.628 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:28:05.628 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::408:52ff:fe3c:ba8f/64 scope link 2023-10-06T02:28:05.628 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:28:05.628 INFO:teuthology.orchestra.run.smithi031.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T02:28:05.628 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 16:d7:88:2d:d0:c2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T02:28:05.628 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::14d7:88ff:fe2d:d0c2/64 scope link 2023-10-06T02:28:05.628 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:28:05.628 INFO:teuthology.orchestra.run.smithi031.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T02:28:05.628 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 82:16:f1:00:cf:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T02:28:05.628 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::8016:f1ff:fe00:cf2f/64 scope link 2023-10-06T02:28:05.628 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:28:05.629 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:28:05.629 DEBUG:teuthology.orchestra.run.smithi031:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T02:28:05.725 INFO:teuthology.orchestra.run.smithi031.stdout:1 2023-10-06T02:28:05.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:28:05.729 DEBUG:teuthology.orchestra.run.smithi031:> route 2023-10-06T02:28:05.767 INFO:teuthology.orchestra.run.smithi031.stdout:Kernel IP routing table 2023-10-06T02:28:05.767 INFO:teuthology.orchestra.run.smithi031.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T02:28:05.767 INFO:teuthology.orchestra.run.smithi031.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T02:28:05.767 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T02:28:05.767 INFO:teuthology.orchestra.run.smithi031.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T02:28:05.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:28:05.768 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2023-10-06T02:28:05.768 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2023-10-06T02:28:05.768 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T02:28:05.769 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T02:28:05.769 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T02:28:05.769 DEBUG:teuthology.orchestra.run.smithi031:> ') 2023-10-06T02:28:05.967 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:28:05.967 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2023-10-06T02:28:06.003 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T02:28:06.004 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T02:28:06.004 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T02:28:06.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:28:06.005 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set brx.0 up 2023-10-06T02:28:06.107 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T02:28:06.108 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T02:28:06.144 INFO:teuthology.orchestra.run.smithi031.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T02:28:06.144 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T02:28:06.145 DEBUG:teuthology.orchestra.run.smithi031:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T02:28:06.220 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-06T02:28:06.220 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T02:28:06.220 DEBUG:teuthology.orchestra.run.smithi031:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T02:28:06.352 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T02:28:06.352 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T02:28:06.380 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T02:28:06.380 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T02:28:06.381 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.62:3300,172.21.15.62:3301" 2023-10-06T02:28:06.381 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: trying mount with new device syntax: 0@d6fcf512-874b-4259-b90a-30ae070f00a9.cephfs=/ 2023-10-06T02:28:06.381 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.31:3300/172.21.15.62:3300/172.21.15.62:3301" will pass to kernel 2023-10-06T02:28:06.385 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T02:28:06.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:28:06.385 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T02:28:06.448 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T02:28:06.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:06.638+0000 7f4fceb9d700 1 Processor -- start 2023-10-06T02:28:06.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:06.638+0000 7f4fceb9d700 1 -- start start 2023-10-06T02:28:06.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:06.639+0000 7f4fceb9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4fc8140e40 0x7f4fc80c6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:06.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:06.639+0000 7f4fceb9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4fc806de50 0x7f4fc80aeef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:06.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:06.639+0000 7f4fceb9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fc80c6d50 con 0x7f4fc8140a70 2023-10-06T02:28:06.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:06.640+0000 7f4fceb9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fc80c7030 con 0x7f4fc806de50 2023-10-06T02:28:06.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:06.640+0000 7f4fceb9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fc80c7310 con 0x7f4fc8140e40 2023-10-06T02:28:06.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:06.640+0000 7f4fce39c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4fc806de50 0x7f4fc80aeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:06.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:06.640+0000 7f4fce39c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4fc806de50 0x7f4fc80aeef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40596/0 (socket says 172.21.15.31:40596) 2023-10-06T02:28:06.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:06.640+0000 7f4fce39c700 1 -- 172.21.15.31:0/4101349042 learned_addr learned my addr 172.21.15.31:0/4101349042 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:06.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:06.640+0000 7f4fcd39a700 1 --2- 172.21.15.31:0/4101349042 >> v2:172.21.15.62:3300/0 conn(0x7f4fc8140e40 0x7f4fc80c6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:06.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:06.640+0000 7f4fce39c700 1 -- 172.21.15.31:0/4101349042 >> v2:172.21.15.62:3300/0 conn(0x7f4fc8140e40 msgr2=0x7f4fc80c6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:06.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:06.640+0000 7f4fce39c700 1 --2- 172.21.15.31:0/4101349042 >> v2:172.21.15.62:3300/0 conn(0x7f4fc8140e40 0x7f4fc80c6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:06.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:06.640+0000 7f4fce39c700 1 -- 172.21.15.31:0/4101349042 >> v1:172.21.15.62:6789/0 conn(0x7f4fc8140a70 legacy=0x7f4fc80c7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:06.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:06.640+0000 7f4fce39c700 1 -- 172.21.15.31:0/4101349042 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fc80c75f0 con 0x7f4fc806de50 2023-10-06T02:28:06.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:06.641+0000 7f4fce39c700 1 --2- 172.21.15.31:0/4101349042 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fc806de50 0x7f4fc80aeef0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f4fc80cc0e0 tx=0x7f4fc0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c42267164cd8cb9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:06.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:06.641+0000 7f4fccb99700 1 -- 172.21.15.31:0/4101349042 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc000e070 con 0x7f4fc806de50 2023-10-06T02:28:06.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:06.641+0000 7f4fccb99700 1 -- 172.21.15.31:0/4101349042 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fc000ba20 con 0x7f4fc806de50 2023-10-06T02:28:06.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:06.641+0000 7f4fccb99700 1 -- 172.21.15.31:0/4101349042 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc00092e0 con 0x7f4fc806de50 2023-10-06T02:28:06.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:06.641+0000 7f4fceb9d700 1 -- 172.21.15.31:0/4101349042 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fc806de50 msgr2=0x7f4fc80aeef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:06.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:06.641+0000 7f4fceb9d700 1 --2- 172.21.15.31:0/4101349042 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fc806de50 0x7f4fc80aeef0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f4fc80cc0e0 tx=0x7f4fc0009840 comp rx=0 tx=0).stop 2023-10-06T02:28:06.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:06.641+0000 7f4fceb9d700 1 -- 172.21.15.31:0/4101349042 shutdown_connections 2023-10-06T02:28:06.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:06.641+0000 7f4fceb9d700 1 --2- 172.21.15.31:0/4101349042 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fc806de50 0x7f4fc80aeef0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:06.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:06.641+0000 7f4fceb9d700 1 --2- 172.21.15.31:0/4101349042 >> v2:172.21.15.62:3300/0 conn(0x7f4fc8140e40 0x7f4fc80c6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:06.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:06.641+0000 7f4fceb9d700 1 -- 172.21.15.31:0/4101349042 >> 172.21.15.31:0/4101349042 conn(0x7f4fc806d170 msgr2=0x7f4fc80cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:06.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:06.641+0000 7f4fceb9d700 1 -- 172.21.15.31:0/4101349042 shutdown_connections 2023-10-06T02:28:06.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:06.641+0000 7f4fceb9d700 1 -- 172.21.15.31:0/4101349042 wait complete. 2023-10-06T02:28:06.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:06.643+0000 7f4fceb9d700 1 Processor -- start 2023-10-06T02:28:06.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:06.643+0000 7f4fceb9d700 1 -- start start 2023-10-06T02:28:06.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:06.644+0000 7f4fceb9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fc806de50 0x7f4fc816a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:06.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:06.644+0000 7f4fceb9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4fc8140a70 0x7f4fc813e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:06.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:06.644+0000 7f4fceb9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fc8140e40 0x7f4fc815dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:06.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:06.644+0000 7f4fceb9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc816a310 con 0x7f4fc806de50 2023-10-06T02:28:06.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:06.644+0000 7f4fceb9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc815e4e0 con 0x7f4fc8140e40 2023-10-06T02:28:06.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:06.644+0000 7f4fceb9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc815e7c0 con 0x7f4fc8140a70 2023-10-06T02:28:06.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:06.644+0000 7f4fcdb9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fc806de50 0x7f4fc816a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:06.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:06.644+0000 7f4fcdb9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fc806de50 0x7f4fc816a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40604/0 (socket says 172.21.15.31:40604) 2023-10-06T02:28:06.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:06.644+0000 7f4fcdb9b700 1 -- 172.21.15.31:0/975734697 learned_addr learned my addr 172.21.15.31:0/975734697 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:06.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:06.645+0000 7f4fcd39a700 1 --2- 172.21.15.31:0/975734697 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4fc8140a70 0x7f4fc813e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:06.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:06.645+0000 7f4fcdb9b700 1 -- 172.21.15.31:0/975734697 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4fc8140a70 msgr2=0x7f4fc813e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:06.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:06.645+0000 7f4fce39c700 1 --2- 172.21.15.31:0/975734697 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fc8140e40 0x7f4fc815dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:06.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:06.645+0000 7f4fcdb9b700 1 --2- 172.21.15.31:0/975734697 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4fc8140a70 0x7f4fc813e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:06.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:06.645+0000 7f4fcdb9b700 1 -- 172.21.15.31:0/975734697 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fc8140e40 msgr2=0x7f4fc815dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:06.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:06.645+0000 7f4fcdb9b700 1 --2- 172.21.15.31:0/975734697 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fc8140e40 0x7f4fc815dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:06.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:06.645+0000 7f4fcdb9b700 1 -- 172.21.15.31:0/975734697 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fc0009ae0 con 0x7f4fc806de50 2023-10-06T02:28:06.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:06.645+0000 7f4fcd39a700 1 --2- 172.21.15.31:0/975734697 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4fc8140a70 0x7f4fc813e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:28:06.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:06.645+0000 7f4fce39c700 1 --2- 172.21.15.31:0/975734697 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fc8140e40 0x7f4fc815dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:28:06.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:06.645+0000 7f4fcdb9b700 1 --2- 172.21.15.31:0/975734697 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fc806de50 0x7f4fc816a7a0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f4fc400c590 tx=0x7f4fc400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:06.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:06.645+0000 7f4fbdffb700 1 -- 172.21.15.31:0/975734697 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc4011590 con 0x7f4fc806de50 2023-10-06T02:28:06.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:06.645+0000 7f4fbdffb700 1 -- 172.21.15.31:0/975734697 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fc40116f0 con 0x7f4fc806de50 2023-10-06T02:28:06.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:06.645+0000 7f4fbdffb700 1 -- 172.21.15.31:0/975734697 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc4011ab0 con 0x7f4fc806de50 2023-10-06T02:28:06.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:06.645+0000 7f4fceb9d700 1 -- 172.21.15.31:0/975734697 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fc8169fa0 con 0x7f4fc806de50 2023-10-06T02:28:06.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:06.645+0000 7f4fceb9d700 1 -- 172.21.15.31:0/975734697 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fc8169c20 con 0x7f4fc806de50 2023-10-06T02:28:06.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:06.646+0000 7f4fceb9d700 1 -- 172.21.15.31:0/975734697 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fc8169e10 con 0x7f4fc806de50 2023-10-06T02:28:06.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:06.647+0000 7f4fbdffb700 1 -- 172.21.15.31:0/975734697 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f4fc401b3b0 con 0x7f4fc806de50 2023-10-06T02:28:06.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:06.648+0000 7f4fbdffb700 1 --2- 172.21.15.31:0/975734697 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f4fac067050 0x7f4fac069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:06.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:06.648+0000 7f4fbdffb700 1 -- 172.21.15.31:0/975734697 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f4fc4083e00 con 0x7f4fc806de50 2023-10-06T02:28:06.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:06.649+0000 7f4fcd39a700 1 --2- 172.21.15.31:0/975734697 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f4fac067050 0x7f4fac069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:06.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:06.649+0000 7f4fcd39a700 1 --2- 172.21.15.31:0/975734697 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f4fac067050 0x7f4fac069500 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8006ea0 tx=0x7f4fb8008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:06.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:06.652+0000 7f4fbdffb700 1 -- 172.21.15.31:0/975734697 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4fc4050d60 con 0x7f4fc806de50 2023-10-06T02:28:06.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:06.899+0000 7f4fceb9d700 1 -- 172.21.15.31:0/975734697 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4fc816a5f0 con 0x7f4fc806de50 2023-10-06T02:28:06.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:06.900+0000 7f4fbdffb700 1 -- 172.21.15.31:0/975734697 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v56) v1 ==== 53+0+73 (secure 0 0 0) 0x7f4fc4017020 con 0x7f4fc806de50 2023-10-06T02:28:06.900 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T02:28:06.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:06.902+0000 7f4fceb9d700 1 -- 172.21.15.31:0/975734697 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f4fac067050 msgr2=0x7f4fac069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:06.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:06.902+0000 7f4fceb9d700 1 --2- 172.21.15.31:0/975734697 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f4fac067050 0x7f4fac069500 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8006ea0 tx=0x7f4fb8008040 comp rx=0 tx=0).stop 2023-10-06T02:28:06.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:06.902+0000 7f4fceb9d700 1 -- 172.21.15.31:0/975734697 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fc806de50 msgr2=0x7f4fc816a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:06.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:06.902+0000 7f4fceb9d700 1 --2- 172.21.15.31:0/975734697 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fc806de50 0x7f4fc816a7a0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f4fc400c590 tx=0x7f4fc400c950 comp rx=0 tx=0).stop 2023-10-06T02:28:06.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:06.902+0000 7f4fceb9d700 1 -- 172.21.15.31:0/975734697 shutdown_connections 2023-10-06T02:28:06.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:06.902+0000 7f4fceb9d700 1 --2- 172.21.15.31:0/975734697 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fc8140e40 0x7f4fc815dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:06.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:06.902+0000 7f4fceb9d700 1 --2- 172.21.15.31:0/975734697 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fc806de50 0x7f4fc816a7a0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:06.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:06.902+0000 7f4fceb9d700 1 --2- 172.21.15.31:0/975734697 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f4fac067050 0x7f4fac069500 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:06.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:06.902+0000 7f4fceb9d700 1 --2- 172.21.15.31:0/975734697 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4fc8140a70 0x7f4fc813e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:06.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:06.902+0000 7f4fceb9d700 1 -- 172.21.15.31:0/975734697 >> 172.21.15.31:0/975734697 conn(0x7f4fc806d170 msgr2=0x7f4fc80cb430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:06.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:06.902+0000 7f4fceb9d700 1 -- 172.21.15.31:0/975734697 shutdown_connections 2023-10-06T02:28:06.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:06.902+0000 7f4fceb9d700 1 -- 172.21.15.31:0/975734697 wait complete. 2023-10-06T02:28:06.920 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T02:28:06.920 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T02:28:06.920 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi062.front.sepia.ceph.com 2023-10-06T02:28:06.920 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-06T02:28:06.920 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-06T02:28:06.920 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T02:28:06.920 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T02:28:06.920 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T02:28:06.920 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T02:28:06.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:28:06.921 DEBUG:teuthology.orchestra.run.smithi062:> ip addr 2023-10-06T02:28:06.963 INFO:teuthology.orchestra.run.smithi062.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T02:28:06.963 INFO:teuthology.orchestra.run.smithi062.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T02:28:06.963 INFO:teuthology.orchestra.run.smithi062.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T02:28:06.963 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:28:06.963 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 ::1/128 scope host 2023-10-06T02:28:06.963 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:28:06.963 INFO:teuthology.orchestra.run.smithi062.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T02:28:06.963 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:14 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:28:06.964 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f0 2023-10-06T02:28:06.964 INFO:teuthology.orchestra.run.smithi062.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T02:28:06.964 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:15 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:28:06.964 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f1 2023-10-06T02:28:06.964 INFO:teuthology.orchestra.run.smithi062.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T02:28:06.964 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c2 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:28:06.964 INFO:teuthology.orchestra.run.smithi062.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T02:28:06.964 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c3 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:28:06.964 INFO:teuthology.orchestra.run.smithi062.stdout: inet 172.21.15.62/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T02:28:06.964 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft 41962sec preferred_lft 41962sec 2023-10-06T02:28:06.964 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::ec4:7aff:febb:d1c3/64 scope link 2023-10-06T02:28:06.964 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:28:06.965 INFO:teuthology.orchestra.run.smithi062.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T02:28:06.965 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether ae:42:a8:dd:f7:9b brd ff:ff:ff:ff:ff:ff 2023-10-06T02:28:06.965 INFO:teuthology.orchestra.run.smithi062.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T02:28:06.965 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:28:06.965 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::1098:a4ff:fe0d:389/64 scope link 2023-10-06T02:28:06.965 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:28:06.965 INFO:teuthology.orchestra.run.smithi062.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T02:28:06.965 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether fe:60:2b:8e:af:28 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T02:28:06.965 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::fc60:2bff:fe8e:af28/64 scope link 2023-10-06T02:28:06.965 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:28:06.965 INFO:teuthology.orchestra.run.smithi062.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T02:28:06.965 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether ae:42:a8:dd:f7:9b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T02:28:06.966 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::ac42:a8ff:fedd:f79b/64 scope link 2023-10-06T02:28:06.966 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:28:06.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:28:06.966 DEBUG:teuthology.orchestra.run.smithi062:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T02:28:07.067 INFO:teuthology.orchestra.run.smithi062.stdout:1 2023-10-06T02:28:07.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:28:07.070 DEBUG:teuthology.orchestra.run.smithi062:> route 2023-10-06T02:28:07.111 INFO:teuthology.orchestra.run.smithi062.stdout:Kernel IP routing table 2023-10-06T02:28:07.111 INFO:teuthology.orchestra.run.smithi062.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T02:28:07.111 INFO:teuthology.orchestra.run.smithi062.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T02:28:07.111 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T02:28:07.111 INFO:teuthology.orchestra.run.smithi062.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T02:28:07.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:28:07.112 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2023-10-06T02:28:07.112 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2023-10-06T02:28:07.112 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T02:28:07.112 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T02:28:07.112 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T02:28:07.112 DEBUG:teuthology.orchestra.run.smithi062:> ') 2023-10-06T02:28:07.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:28:07.310 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2023-10-06T02:28:07.347 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-06T02:28:07.347 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T02:28:07.347 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T02:28:07.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:28:07.348 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set brx.0 up 2023-10-06T02:28:07.449 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T02:28:07.449 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-06T02:28:07.484 INFO:teuthology.orchestra.run.smithi062.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-06T02:28:07.485 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T02:28:07.485 DEBUG:teuthology.orchestra.run.smithi062:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-06T02:28:07.560 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-06T02:28:07.560 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T02:28:07.560 DEBUG:teuthology.orchestra.run.smithi062:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T02:28:07.693 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T02:28:07.694 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T02:28:07.722 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T02:28:07.722 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T02:28:07.722 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.62:3300,172.21.15.62:3301" 2023-10-06T02:28:07.723 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: trying mount with new device syntax: 1@d6fcf512-874b-4259-b90a-30ae070f00a9.cephfs=/ 2023-10-06T02:28:07.723 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.31:3300/172.21.15.62:3300/172.21.15.62:3301" will pass to kernel 2023-10-06T02:28:07.727 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T02:28:07.727 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:28:07.727 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-06T02:28:07.789 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-10-06T02:28:07.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:07.991+0000 7fbaff19d700 1 Processor -- start 2023-10-06T02:28:07.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:07.992+0000 7fbaff19d700 1 -- start start 2023-10-06T02:28:07.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:07.993+0000 7fbaff19d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fbaf8153ac0 0x7fbaf815a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:07.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:07.993+0000 7fbaff19d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbaf8153e90 0x7fbaf80ca5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:07.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:07.993+0000 7fbaff19d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbaf815abf0 con 0x7fbaf80703a0 2023-10-06T02:28:07.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:07.993+0000 7fbaff19d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbaf8070940 con 0x7fbaf8153ac0 2023-10-06T02:28:07.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:07.993+0000 7fbaff19d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf806a330 con 0x7fbaf8153e90 2023-10-06T02:28:07.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:07.993+0000 7fbafe19b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fbaf8153ac0 0x7fbaf815a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:07.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:07.993+0000 7fbafe19b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fbaf8153ac0 0x7fbaf815a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37978/0 (socket says 172.21.15.31:37978) 2023-10-06T02:28:07.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:07.993+0000 7fbafe19b700 1 -- 172.21.15.31:0/100835139 learned_addr learned my addr 172.21.15.31:0/100835139 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:07.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:07.993+0000 7fbafd99a700 1 --2- 172.21.15.31:0/100835139 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbaf8153e90 0x7fbaf80ca5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:07.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:07.994+0000 7fbafe19b700 1 -- 172.21.15.31:0/100835139 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbaf8153e90 msgr2=0x7fbaf80ca5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:07.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:07.994+0000 7fbafe19b700 1 --2- 172.21.15.31:0/100835139 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbaf8153e90 0x7fbaf80ca5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:07.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:07.994+0000 7fbafe19b700 1 -- 172.21.15.31:0/100835139 >> v1:172.21.15.62:6789/0 conn(0x7fbaf80703a0 legacy=0x7fbaf815d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:07.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:07.994+0000 7fbafe19b700 1 -- 172.21.15.31:0/100835139 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaf806a610 con 0x7fbaf8153ac0 2023-10-06T02:28:07.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:07.994+0000 7fbafd99a700 1 --2- 172.21.15.31:0/100835139 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbaf8153e90 0x7fbaf80ca5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:28:07.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:07.994+0000 7fbafe19b700 1 --2- 172.21.15.31:0/100835139 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbaf8153ac0 0x7fbaf815a6b0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fbaf40099b0 tx=0x7fbaf4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8c79eb3afed18d3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:07.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:07.994+0000 7fbafd199700 1 -- 172.21.15.31:0/100835139 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf4009ea0 con 0x7fbaf8153ac0 2023-10-06T02:28:07.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:07.994+0000 7fbafd199700 1 -- 172.21.15.31:0/100835139 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaf400b860 con 0x7fbaf8153ac0 2023-10-06T02:28:07.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:07.994+0000 7fbafd199700 1 -- 172.21.15.31:0/100835139 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf400bc00 con 0x7fbaf8153ac0 2023-10-06T02:28:07.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:07.995+0000 7fbaff19d700 1 -- 172.21.15.31:0/100835139 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbaf8153ac0 msgr2=0x7fbaf815a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:07.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:07.995+0000 7fbaff19d700 1 --2- 172.21.15.31:0/100835139 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbaf8153ac0 0x7fbaf815a6b0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fbaf40099b0 tx=0x7fbaf4009250 comp rx=0 tx=0).stop 2023-10-06T02:28:07.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:07.995+0000 7fbaff19d700 1 -- 172.21.15.31:0/100835139 shutdown_connections 2023-10-06T02:28:07.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:07.995+0000 7fbaff19d700 1 --2- 172.21.15.31:0/100835139 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbaf8153ac0 0x7fbaf815a6b0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:07.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:07.995+0000 7fbaff19d700 1 --2- 172.21.15.31:0/100835139 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbaf8153e90 0x7fbaf80ca5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:07.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:07.995+0000 7fbaff19d700 1 -- 172.21.15.31:0/100835139 >> 172.21.15.31:0/100835139 conn(0x7fbaf806e790 msgr2=0x7fbaf806eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:07.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:07.995+0000 7fbaff19d700 1 -- 172.21.15.31:0/100835139 shutdown_connections 2023-10-06T02:28:07.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:07.995+0000 7fbaff19d700 1 -- 172.21.15.31:0/100835139 wait complete. 2023-10-06T02:28:07.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:07.996+0000 7fbaff19d700 1 Processor -- start 2023-10-06T02:28:07.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:07.997+0000 7fbaff19d700 1 -- start start 2023-10-06T02:28:07.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:07.997+0000 7fbaff19d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbaf80703a0 0x7fbaf80c81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:07.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:07.997+0000 7fbaff19d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbaf8153ac0 0x7fbaf816a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:07.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:07.997+0000 7fbaff19d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbaf8153e90 0x7fbaf814d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:07.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:07.997+0000 7fbaff19d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf816ac70 con 0x7fbaf8153e90 2023-10-06T02:28:07.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:07.997+0000 7fbaff19d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf80c8840 con 0x7fbaf8153ac0 2023-10-06T02:28:08.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:07.997+0000 7fbaff19d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf80c8b20 con 0x7fbaf80703a0 2023-10-06T02:28:08.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:07.998+0000 7fbafe99c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbaf8153e90 0x7fbaf814d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:08.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:07.998+0000 7fbafe99c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbaf8153e90 0x7fbaf814d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37994/0 (socket says 172.21.15.31:37994) 2023-10-06T02:28:08.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:07.998+0000 7fbafe99c700 1 -- 172.21.15.31:0/2392729395 learned_addr learned my addr 172.21.15.31:0/2392729395 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:08.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:07.998+0000 7fbafe19b700 1 --2- 172.21.15.31:0/2392729395 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbaf80703a0 0x7fbaf80c81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:08.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:07.998+0000 7fbafd99a700 1 --2- 172.21.15.31:0/2392729395 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbaf8153ac0 0x7fbaf816a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:08.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:07.998+0000 7fbafe99c700 1 -- 172.21.15.31:0/2392729395 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbaf80703a0 msgr2=0x7fbaf80c81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:08.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:07.998+0000 7fbafe99c700 1 --2- 172.21.15.31:0/2392729395 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbaf80703a0 0x7fbaf80c81a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:08.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:07.998+0000 7fbafe99c700 1 -- 172.21.15.31:0/2392729395 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbaf8153ac0 msgr2=0x7fbaf816a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:08.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:07.998+0000 7fbafe99c700 1 --2- 172.21.15.31:0/2392729395 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbaf8153ac0 0x7fbaf816a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:08.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:07.998+0000 7fbafe99c700 1 -- 172.21.15.31:0/2392729395 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbae8009420 con 0x7fbaf8153e90 2023-10-06T02:28:08.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:07.998+0000 7fbafe99c700 1 --2- 172.21.15.31:0/2392729395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbaf8153e90 0x7fbaf814d1b0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fbaf000c8e0 tx=0x7fbaf0001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:08.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:07.999+0000 7fbaee7fc700 1 -- 172.21.15.31:0/2392729395 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf0012630 con 0x7fbaf8153e90 2023-10-06T02:28:08.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:07.999+0000 7fbaee7fc700 1 -- 172.21.15.31:0/2392729395 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaf0012790 con 0x7fbaf8153e90 2023-10-06T02:28:08.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:07.999+0000 7fbaff19d700 1 -- 172.21.15.31:0/2392729395 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbaf40094f0 con 0x7fbaf8153e90 2023-10-06T02:28:08.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:07.999+0000 7fbaee7fc700 1 -- 172.21.15.31:0/2392729395 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf0012b50 con 0x7fbaf8153e90 2023-10-06T02:28:08.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:07.999+0000 7fbaff19d700 1 -- 172.21.15.31:0/2392729395 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbaf80c90e0 con 0x7fbaf8153e90 2023-10-06T02:28:08.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:08.000+0000 7fbaff19d700 1 -- 172.21.15.31:0/2392729395 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbaf804c3c0 con 0x7fbaf8153e90 2023-10-06T02:28:08.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:08.001+0000 7fbaee7fc700 1 -- 172.21.15.31:0/2392729395 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7fbaf001a3a0 con 0x7fbaf8153e90 2023-10-06T02:28:08.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:08.002+0000 7fbaee7fc700 1 --2- 172.21.15.31:0/2392729395 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fbadc067050 0x7fbadc069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:08.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:08.002+0000 7fbaee7fc700 1 -- 172.21.15.31:0/2392729395 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fbaf0084ad0 con 0x7fbaf8153e90 2023-10-06T02:28:08.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:08.002+0000 7fbafe19b700 1 --2- 172.21.15.31:0/2392729395 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fbadc067050 0x7fbadc069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:08.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:08.003+0000 7fbafe19b700 1 --2- 172.21.15.31:0/2392729395 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fbadc067050 0x7fbadc069500 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbaf40099b0 tx=0x7fbaf4017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:08.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:08.006+0000 7fbaee7fc700 1 -- 172.21.15.31:0/2392729395 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbaf0051a30 con 0x7fbaf8153e90 2023-10-06T02:28:08.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:08.254+0000 7fbaff19d700 1 -- 172.21.15.31:0/2392729395 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fbaf806b770 con 0x7fbaf8153e90 2023-10-06T02:28:08.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:08.583+0000 7fbaee7fc700 1 -- 172.21.15.31:0/2392729395 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v57) v1 ==== 136+0+0 (secure 0 0 0) 0x7fbaf0055050 con 0x7fbaf8153e90 2023-10-06T02:28:08.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:08.586+0000 7fbaff19d700 1 -- 172.21.15.31:0/2392729395 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fbadc067050 msgr2=0x7fbadc069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:08.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:08.586+0000 7fbaff19d700 1 --2- 172.21.15.31:0/2392729395 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fbadc067050 0x7fbadc069500 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbaf40099b0 tx=0x7fbaf4017000 comp rx=0 tx=0).stop 2023-10-06T02:28:08.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:08.586+0000 7fbaff19d700 1 -- 172.21.15.31:0/2392729395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbaf8153e90 msgr2=0x7fbaf814d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:08.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:08.586+0000 7fbaff19d700 1 --2- 172.21.15.31:0/2392729395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbaf8153e90 0x7fbaf814d1b0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fbaf000c8e0 tx=0x7fbaf0001710 comp rx=0 tx=0).stop 2023-10-06T02:28:08.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:08.586+0000 7fbaff19d700 1 -- 172.21.15.31:0/2392729395 shutdown_connections 2023-10-06T02:28:08.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:08.586+0000 7fbaff19d700 1 --2- 172.21.15.31:0/2392729395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbaf8153e90 0x7fbaf814d1b0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:08.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:08.586+0000 7fbaff19d700 1 --2- 172.21.15.31:0/2392729395 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fbadc067050 0x7fbadc069500 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:08.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:08.586+0000 7fbaff19d700 1 --2- 172.21.15.31:0/2392729395 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbaf80703a0 0x7fbaf80c81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:08.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:08.586+0000 7fbaff19d700 1 --2- 172.21.15.31:0/2392729395 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbaf8153ac0 0x7fbaf816a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:08.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:08.586+0000 7fbaff19d700 1 -- 172.21.15.31:0/2392729395 >> 172.21.15.31:0/2392729395 conn(0x7fbaf806e790 msgr2=0x7fbaf80c3a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:08.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:08.586+0000 7fbaff19d700 1 -- 172.21.15.31:0/2392729395 shutdown_connections 2023-10-06T02:28:08.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:08.586+0000 7fbaff19d700 1 -- 172.21.15.31:0/2392729395 wait complete. 2023-10-06T02:28:08.605 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-06T02:28:08.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:08.793+0000 7ff3b8b9d700 1 Processor -- start 2023-10-06T02:28:08.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:08.794+0000 7ff3b8b9d700 1 -- start start 2023-10-06T02:28:08.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:08.795+0000 7ff3b8b9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3b4142c00 0x7ff3b406b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:08.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:08.795+0000 7ff3b8b9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff3b406bd20 0x7ff3b40c7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:08.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:08.795+0000 7ff3b8b9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3b40ca480 con 0x7ff3b4142fd0 2023-10-06T02:28:08.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:08.795+0000 7ff3b8b9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3b415e240 con 0x7ff3b406bd20 2023-10-06T02:28:08.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:08.795+0000 7ff3b8b9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b406b350 con 0x7ff3b4142c00 2023-10-06T02:28:08.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:08.795+0000 7ff3b3fff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7ff3b4142fd0 0x7ff3b406c6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:35250/0 (socket says 172.21.15.31:35250) 2023-10-06T02:28:08.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:08.795+0000 7ff3b3fff700 1 -- 172.21.15.31:0/1822398228 learned_addr learned my addr 172.21.15.31:0/1822398228 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:28:08.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:08.795+0000 7ff3b37fe700 1 --2- 172.21.15.31:0/1822398228 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3b4142c00 0x7ff3b406b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:08.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:08.795+0000 7ff3b2ffd700 1 --2- 172.21.15.31:0/1822398228 >> v2:172.21.15.62:3300/0 conn(0x7ff3b406bd20 0x7ff3b40c7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:08.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:08.796+0000 7ff3b27fc700 1 -- 172.21.15.31:0/1822398228 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2754502801 0 0) 0x7ff3b40ca480 con 0x7ff3b4142fd0 2023-10-06T02:28:08.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:08.796+0000 7ff3b27fc700 1 -- 172.21.15.31:0/1822398228 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff39c003570 con 0x7ff3b4142fd0 2023-10-06T02:28:08.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:08.796+0000 7ff3b37fe700 1 -- 172.21.15.31:0/1822398228 >> v2:172.21.15.62:3300/0 conn(0x7ff3b406bd20 msgr2=0x7ff3b40c7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:08.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:08.796+0000 7ff3b37fe700 1 --2- 172.21.15.31:0/1822398228 >> v2:172.21.15.62:3300/0 conn(0x7ff3b406bd20 0x7ff3b40c7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:08.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:08.796+0000 7ff3b37fe700 1 -- 172.21.15.31:0/1822398228 >> v1:172.21.15.31:6789/0 conn(0x7ff3b4142fd0 legacy=0x7ff3b406c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:08.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:08.796+0000 7ff3b37fe700 1 -- 172.21.15.31:0/1822398228 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3b415df60 con 0x7ff3b4142c00 2023-10-06T02:28:08.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:08.796+0000 7ff3b2ffd700 1 --2- 172.21.15.31:0/1822398228 >> v2:172.21.15.62:3300/0 conn(0x7ff3b406bd20 0x7ff3b40c7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:28:08.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:08.796+0000 7ff3b37fe700 1 --2- 172.21.15.31:0/1822398228 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3b4142c00 0x7ff3b406b7e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff3a40098e0 tx=0x7ff3a40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fcbbf642bd5d4dbf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:08.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:08.797+0000 7ff3b27fc700 1 -- 172.21.15.31:0/1822398228 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a4011630 con 0x7ff3b4142c00 2023-10-06T02:28:08.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:08.797+0000 7ff3b27fc700 1 -- 172.21.15.31:0/1822398228 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3a4011790 con 0x7ff3b4142c00 2023-10-06T02:28:08.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:08.797+0000 7ff3b27fc700 1 -- 172.21.15.31:0/1822398228 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a4011b10 con 0x7ff3b4142c00 2023-10-06T02:28:08.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:08.797+0000 7ff3b8b9d700 1 -- 172.21.15.31:0/1822398228 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3b4142c00 msgr2=0x7ff3b406b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:08.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:08.797+0000 7ff3b8b9d700 1 --2- 172.21.15.31:0/1822398228 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3b4142c00 0x7ff3b406b7e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff3a40098e0 tx=0x7ff3a40091a0 comp rx=0 tx=0).stop 2023-10-06T02:28:08.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:08.797+0000 7ff3b8b9d700 1 -- 172.21.15.31:0/1822398228 shutdown_connections 2023-10-06T02:28:08.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:08.797+0000 7ff3b8b9d700 1 --2- 172.21.15.31:0/1822398228 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3b4142c00 0x7ff3b406b7e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:08.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:08.797+0000 7ff3b8b9d700 1 --2- 172.21.15.31:0/1822398228 >> v2:172.21.15.62:3300/0 conn(0x7ff3b406bd20 0x7ff3b40c7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:08.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:08.797+0000 7ff3b8b9d700 1 -- 172.21.15.31:0/1822398228 >> 172.21.15.31:0/1822398228 conn(0x7ff3b40675a0 msgr2=0x7ff3b40679a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:08.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:08.797+0000 7ff3b8b9d700 1 -- 172.21.15.31:0/1822398228 shutdown_connections 2023-10-06T02:28:08.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:08.797+0000 7ff3b8b9d700 1 -- 172.21.15.31:0/1822398228 wait complete. 2023-10-06T02:28:08.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:08.799+0000 7ff3b8b9d700 1 Processor -- start 2023-10-06T02:28:08.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:08.799+0000 7ff3b8b9d700 1 -- start start 2023-10-06T02:28:08.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:08.800+0000 7ff3b8b9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3b406bd20 0x7ff3b40c4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:08.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:08.800+0000 7ff3b8b9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3b4142c00 0x7ff3b416ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:08.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:08.800+0000 7ff3b8b9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3b4142fd0 0x7ff3b40cd520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:08.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:08.800+0000 7ff3b8b9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b40c5f50 con 0x7ff3b406bd20 2023-10-06T02:28:08.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:08.800+0000 7ff3b8b9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b40c5240 con 0x7ff3b4142c00 2023-10-06T02:28:08.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:08.800+0000 7ff3b8b9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b40b03a0 con 0x7ff3b4142fd0 2023-10-06T02:28:08.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:08.800+0000 7ff3b37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3b406bd20 0x7ff3b40c4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:08.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:08.800+0000 7ff3b37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3b406bd20 0x7ff3b40c4d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38008/0 (socket says 172.21.15.31:38008) 2023-10-06T02:28:08.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:08.800+0000 7ff3b37fe700 1 -- 172.21.15.31:0/3703092851 learned_addr learned my addr 172.21.15.31:0/3703092851 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:08.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:08.800+0000 7ff3b2ffd700 1 --2- 172.21.15.31:0/3703092851 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3b4142c00 0x7ff3b416ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:08.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:08.800+0000 7ff3b3fff700 1 --2- 172.21.15.31:0/3703092851 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3b4142fd0 0x7ff3b40cd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:08.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:08.800+0000 7ff3b37fe700 1 -- 172.21.15.31:0/3703092851 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3b4142fd0 msgr2=0x7ff3b40cd520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:08.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:08.800+0000 7ff3b37fe700 1 --2- 172.21.15.31:0/3703092851 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3b4142fd0 0x7ff3b40cd520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:08.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:08.800+0000 7ff3b37fe700 1 -- 172.21.15.31:0/3703092851 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3b4142c00 msgr2=0x7ff3b416ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:08.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:08.801+0000 7ff3b37fe700 1 --2- 172.21.15.31:0/3703092851 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3b4142c00 0x7ff3b416ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:08.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:08.801+0000 7ff3b37fe700 1 -- 172.21.15.31:0/3703092851 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3a8009420 con 0x7ff3b406bd20 2023-10-06T02:28:08.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:08.801+0000 7ff3b3fff700 1 --2- 172.21.15.31:0/3703092851 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3b4142fd0 0x7ff3b40cd520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:28:08.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:08.801+0000 7ff3b37fe700 1 --2- 172.21.15.31:0/3703092851 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3b406bd20 0x7ff3b40c4d00 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4009ed0 tx=0x7ff3a4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:08.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:08.801+0000 7ff39b7fe700 1 -- 172.21.15.31:0/3703092851 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a4011b10 con 0x7ff3b406bd20 2023-10-06T02:28:08.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:08.801+0000 7ff39b7fe700 1 -- 172.21.15.31:0/3703092851 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3a4000cc0 con 0x7ff3b406bd20 2023-10-06T02:28:08.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:08.801+0000 7ff3b8b9d700 1 -- 172.21.15.31:0/3703092851 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3a4009420 con 0x7ff3b406bd20 2023-10-06T02:28:08.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:08.801+0000 7ff39b7fe700 1 -- 172.21.15.31:0/3703092851 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a401a660 con 0x7ff3b406bd20 2023-10-06T02:28:08.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:08.801+0000 7ff3b8b9d700 1 -- 172.21.15.31:0/3703092851 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3b40b0960 con 0x7ff3b406bd20 2023-10-06T02:28:08.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:08.802+0000 7ff3b8b9d700 1 -- 172.21.15.31:0/3703092851 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3b404c3c0 con 0x7ff3b406bd20 2023-10-06T02:28:08.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:08.803+0000 7ff39b7fe700 1 -- 172.21.15.31:0/3703092851 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7ff3a401aa90 con 0x7ff3b406bd20 2023-10-06T02:28:08.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:08.804+0000 7ff39b7fe700 1 --2- 172.21.15.31:0/3703092851 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff3a0067080 0x7ff3a0069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:08.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:08.804+0000 7ff39b7fe700 1 -- 172.21.15.31:0/3703092851 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff3a40863c0 con 0x7ff3b406bd20 2023-10-06T02:28:08.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:08.804+0000 7ff3b2ffd700 1 --2- 172.21.15.31:0/3703092851 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff3a0067080 0x7ff3a0069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:08.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:08.805+0000 7ff3b2ffd700 1 --2- 172.21.15.31:0/3703092851 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff3a0067080 0x7ff3a0069530 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff3a80093f0 tx=0x7ff3a8008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:08.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:08.808+0000 7ff39b7fe700 1 -- 172.21.15.31:0/3703092851 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff3a40533c0 con 0x7ff3b406bd20 2023-10-06T02:28:09.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.038+0000 7ff3b8b9d700 1 -- 172.21.15.31:0/3703092851 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7ff3b406ad00 con 0x7ff3a0067080 2023-10-06T02:28:09.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.041+0000 7ff39b7fe700 1 -- 172.21.15.31:0/3703092851 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7ff3b406ad00 con 0x7ff3a0067080 2023-10-06T02:28:09.041 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2023-10-06T02:28:09.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:28:09.041 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephfs" 2023-10-06T02:28:09.041 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:28:09.041 INFO:teuthology.orchestra.run.smithi031.stdout:] 2023-10-06T02:28:09.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.043+0000 7ff3b8b9d700 1 -- 172.21.15.31:0/3703092851 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff3a0067080 msgr2=0x7ff3a0069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:09.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.043+0000 7ff3b8b9d700 1 --2- 172.21.15.31:0/3703092851 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff3a0067080 0x7ff3a0069530 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff3a80093f0 tx=0x7ff3a8008040 comp rx=0 tx=0).stop 2023-10-06T02:28:09.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.043+0000 7ff3b8b9d700 1 -- 172.21.15.31:0/3703092851 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3b406bd20 msgr2=0x7ff3b40c4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:09.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.043+0000 7ff3b8b9d700 1 --2- 172.21.15.31:0/3703092851 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3b406bd20 0x7ff3b40c4d00 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4009ed0 tx=0x7ff3a4017000 comp rx=0 tx=0).stop 2023-10-06T02:28:09.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.043+0000 7ff3b8b9d700 1 -- 172.21.15.31:0/3703092851 shutdown_connections 2023-10-06T02:28:09.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.043+0000 7ff3b8b9d700 1 --2- 172.21.15.31:0/3703092851 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff3a0067080 0x7ff3a0069530 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:09.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.043+0000 7ff3b8b9d700 1 --2- 172.21.15.31:0/3703092851 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3b4142fd0 0x7ff3b40cd520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:09.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.043+0000 7ff3b8b9d700 1 --2- 172.21.15.31:0/3703092851 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3b406bd20 0x7ff3b40c4d00 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:09.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.043+0000 7ff3b8b9d700 1 --2- 172.21.15.31:0/3703092851 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3b4142c00 0x7ff3b416ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:09.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.043+0000 7ff3b8b9d700 1 -- 172.21.15.31:0/3703092851 >> 172.21.15.31:0/3703092851 conn(0x7ff3b40675a0 msgr2=0x7ff3b40c78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:09.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.043+0000 7ff3b8b9d700 1 -- 172.21.15.31:0/3703092851 shutdown_connections 2023-10-06T02:28:09.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.043+0000 7ff3b8b9d700 1 -- 172.21.15.31:0/3703092851 wait complete. 2023-10-06T02:28:09.061 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T02:28:09.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.250+0000 7fc5a759e700 1 Processor -- start 2023-10-06T02:28:09.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.250+0000 7fc5a759e700 1 -- start start 2023-10-06T02:28:09.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.252+0000 7fc5a759e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc5a8142fd0 0x7fc5a80c7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:09.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.252+0000 7fc5a759e700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc5a80c7dc0 0x7fc5a806c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:09.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.252+0000 7fc5a759e700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5a80ca480 con 0x7fc5a8142c00 2023-10-06T02:28:09.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.252+0000 7fc5a759e700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5a815e240 con 0x7fc5a8142fd0 2023-10-06T02:28:09.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.252+0000 7fc5a759e700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5a806b350 con 0x7fc5a80c7dc0 2023-10-06T02:28:09.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.252+0000 7fc5a659c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fc5a8142c00 0x7fc5a806b7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:35258/0 (socket says 172.21.15.31:35258) 2023-10-06T02:28:09.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.252+0000 7fc5a659c700 1 -- 172.21.15.31:0/401422938 learned_addr learned my addr 172.21.15.31:0/401422938 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:28:09.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.252+0000 7fc5a5d9b700 1 --2- 172.21.15.31:0/401422938 >> v2:172.21.15.31:3300/0 conn(0x7fc5a8142fd0 0x7fc5a80c7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:09.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.252+0000 7fc5a6d9d700 1 --2- 172.21.15.31:0/401422938 >> v2:172.21.15.62:3300/0 conn(0x7fc5a80c7dc0 0x7fc5a806c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:09.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.252+0000 7fc5a5d9b700 1 -- 172.21.15.31:0/401422938 >> v2:172.21.15.62:3300/0 conn(0x7fc5a80c7dc0 msgr2=0x7fc5a806c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:09.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.252+0000 7fc5a5d9b700 1 --2- 172.21.15.31:0/401422938 >> v2:172.21.15.62:3300/0 conn(0x7fc5a80c7dc0 0x7fc5a806c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:09.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.252+0000 7fc5a5d9b700 1 -- 172.21.15.31:0/401422938 >> v1:172.21.15.31:6789/0 conn(0x7fc5a8142c00 legacy=0x7fc5a806b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:09.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.252+0000 7fc5a5d9b700 1 -- 172.21.15.31:0/401422938 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a80c81d0 con 0x7fc5a8142fd0 2023-10-06T02:28:09.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.252+0000 7fc5a559a700 1 -- 172.21.15.31:0/401422938 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1514381955 0 0) 0x7fc5a80ca480 con 0x7fc5a8142c00 2023-10-06T02:28:09.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.253+0000 7fc5a5d9b700 1 --2- 172.21.15.31:0/401422938 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5a8142fd0 0x7fc5a80c7880 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fc5a806d750 tx=0x7fc59c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82c917e0b55b1a17 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:09.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.253+0000 7fc5a559a700 1 -- 172.21.15.31:0/401422938 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc59c00b6b0 con 0x7fc5a8142fd0 2023-10-06T02:28:09.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.253+0000 7fc5a559a700 1 -- 172.21.15.31:0/401422938 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc59c00b810 con 0x7fc5a8142fd0 2023-10-06T02:28:09.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.253+0000 7fc5a559a700 1 -- 172.21.15.31:0/401422938 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc59c00bbb0 con 0x7fc5a8142fd0 2023-10-06T02:28:09.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.253+0000 7fc5a759e700 1 -- 172.21.15.31:0/401422938 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5a8142fd0 msgr2=0x7fc5a80c7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:09.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.253+0000 7fc5a759e700 1 --2- 172.21.15.31:0/401422938 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5a8142fd0 0x7fc5a80c7880 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fc5a806d750 tx=0x7fc59c0091a0 comp rx=0 tx=0).stop 2023-10-06T02:28:09.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.254+0000 7fc5a759e700 1 -- 172.21.15.31:0/401422938 shutdown_connections 2023-10-06T02:28:09.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.254+0000 7fc5a759e700 1 --2- 172.21.15.31:0/401422938 >> v2:172.21.15.62:3300/0 conn(0x7fc5a80c7dc0 0x7fc5a806c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:09.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.254+0000 7fc5a759e700 1 --2- 172.21.15.31:0/401422938 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5a8142fd0 0x7fc5a80c7880 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:09.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.254+0000 7fc5a759e700 1 -- 172.21.15.31:0/401422938 >> 172.21.15.31:0/401422938 conn(0x7fc5a80675a0 msgr2=0x7fc5a80679a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:09.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.254+0000 7fc5a759e700 1 -- 172.21.15.31:0/401422938 shutdown_connections 2023-10-06T02:28:09.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.254+0000 7fc5a759e700 1 -- 172.21.15.31:0/401422938 wait complete. 2023-10-06T02:28:09.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.255+0000 7fc5a759e700 1 Processor -- start 2023-10-06T02:28:09.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.255+0000 7fc5a759e700 1 -- start start 2023-10-06T02:28:09.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.256+0000 7fc5a759e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc5a80c7dc0 0x7fc5a815a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:09.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.256+0000 7fc5a759e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5a8142c00 0x7fc5a815c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:09.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.256+0000 7fc5a759e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc5a8142fd0 0x7fc5a81590e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:09.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.256+0000 7fc5a759e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a8159620 con 0x7fc5a8142c00 2023-10-06T02:28:09.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.256+0000 7fc5a759e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a815d7f0 con 0x7fc5a8142fd0 2023-10-06T02:28:09.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.256+0000 7fc5a759e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a815cbe0 con 0x7fc5a80c7dc0 2023-10-06T02:28:09.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.256+0000 7fc5a5d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5a8142c00 0x7fc5a815c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:09.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.256+0000 7fc5a5d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5a8142c00 0x7fc5a815c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38026/0 (socket says 172.21.15.31:38026) 2023-10-06T02:28:09.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.256+0000 7fc5a5d9b700 1 -- 172.21.15.31:0/3968364306 learned_addr learned my addr 172.21.15.31:0/3968364306 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:09.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.256+0000 7fc5a6d9d700 1 --2- 172.21.15.31:0/3968364306 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc5a8142fd0 0x7fc5a81590e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:09.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.256+0000 7fc5a659c700 1 --2- 172.21.15.31:0/3968364306 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc5a80c7dc0 0x7fc5a815a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:09.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.257+0000 7fc5a5d9b700 1 -- 172.21.15.31:0/3968364306 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc5a80c7dc0 msgr2=0x7fc5a815a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:09.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.257+0000 7fc5a5d9b700 1 --2- 172.21.15.31:0/3968364306 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc5a80c7dc0 0x7fc5a815a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:09.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.257+0000 7fc5a5d9b700 1 -- 172.21.15.31:0/3968364306 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc5a8142fd0 msgr2=0x7fc5a81590e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:09.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.257+0000 7fc5a5d9b700 1 --2- 172.21.15.31:0/3968364306 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc5a8142fd0 0x7fc5a81590e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:09.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.257+0000 7fc5a5d9b700 1 -- 172.21.15.31:0/3968364306 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc59c009420 con 0x7fc5a8142c00 2023-10-06T02:28:09.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.257+0000 7fc5a5d9b700 1 --2- 172.21.15.31:0/3968364306 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5a8142c00 0x7fc5a815c500 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fc5a806f290 tx=0x7fc59c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:09.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.257+0000 7fc5967fc700 1 -- 172.21.15.31:0/3968364306 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc59c00b980 con 0x7fc5a8142c00 2023-10-06T02:28:09.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.257+0000 7fc5967fc700 1 -- 172.21.15.31:0/3968364306 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc59c00bae0 con 0x7fc5a8142c00 2023-10-06T02:28:09.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.257+0000 7fc5967fc700 1 -- 172.21.15.31:0/3968364306 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc59c011670 con 0x7fc5a8142c00 2023-10-06T02:28:09.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.257+0000 7fc5a759e700 1 -- 172.21.15.31:0/3968364306 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5a815cec0 con 0x7fc5a8142c00 2023-10-06T02:28:09.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.258+0000 7fc5a759e700 1 -- 172.21.15.31:0/3968364306 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5a815d480 con 0x7fc5a8142c00 2023-10-06T02:28:09.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.258+0000 7fc5a759e700 1 -- 172.21.15.31:0/3968364306 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5a815d980 con 0x7fc5a8142c00 2023-10-06T02:28:09.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.259+0000 7fc5967fc700 1 -- 172.21.15.31:0/3968364306 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7fc59c011a60 con 0x7fc5a8142c00 2023-10-06T02:28:09.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.260+0000 7fc5967fc700 1 --2- 172.21.15.31:0/3968364306 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc588067050 0x7fc588069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:09.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.261+0000 7fc5967fc700 1 -- 172.21.15.31:0/3968364306 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc59c085d00 con 0x7fc5a8142c00 2023-10-06T02:28:09.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.261+0000 7fc5a659c700 1 --2- 172.21.15.31:0/3968364306 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc588067050 0x7fc588069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:09.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.261+0000 7fc5a659c700 1 --2- 172.21.15.31:0/3968364306 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc588067050 0x7fc588069500 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc5a80b2880 tx=0x7fc598007d30 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:09.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.264+0000 7fc5967fc700 1 -- 172.21.15.31:0/3968364306 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc59c052ce0 con 0x7fc5a8142c00 2023-10-06T02:28:09.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.465+0000 7fc5a759e700 1 -- 172.21.15.31:0/3968364306 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fc5a804c3c0 con 0x7fc5a8142c00 2023-10-06T02:28:09.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.466+0000 7fc5967fc700 1 -- 172.21.15.31:0/3968364306 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc59c056300 con 0x7fc5a8142c00 2023-10-06T02:28:09.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.468+0000 7fc5a759e700 1 -- 172.21.15.31:0/3968364306 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc588067050 msgr2=0x7fc588069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:09.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.468+0000 7fc5a759e700 1 --2- 172.21.15.31:0/3968364306 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc588067050 0x7fc588069500 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc5a80b2880 tx=0x7fc598007d30 comp rx=0 tx=0).stop 2023-10-06T02:28:09.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.468+0000 7fc5a759e700 1 -- 172.21.15.31:0/3968364306 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5a8142c00 msgr2=0x7fc5a815c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:09.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.468+0000 7fc5a759e700 1 --2- 172.21.15.31:0/3968364306 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5a8142c00 0x7fc5a815c500 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fc5a806f290 tx=0x7fc59c017000 comp rx=0 tx=0).stop 2023-10-06T02:28:09.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.468+0000 7fc5a759e700 1 -- 172.21.15.31:0/3968364306 shutdown_connections 2023-10-06T02:28:09.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.468+0000 7fc5a759e700 1 --2- 172.21.15.31:0/3968364306 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc5a8142fd0 0x7fc5a81590e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:09.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.468+0000 7fc5a759e700 1 --2- 172.21.15.31:0/3968364306 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc588067050 0x7fc588069500 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:09.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.468+0000 7fc5a759e700 1 --2- 172.21.15.31:0/3968364306 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc5a80c7dc0 0x7fc5a815a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:09.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.468+0000 7fc5a759e700 1 --2- 172.21.15.31:0/3968364306 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5a8142c00 0x7fc5a815c500 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:09.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.468+0000 7fc5a759e700 1 -- 172.21.15.31:0/3968364306 >> 172.21.15.31:0/3968364306 conn(0x7fc5a80675a0 msgr2=0x7fc5a80ca910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:09.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.468+0000 7fc5a759e700 1 -- 172.21.15.31:0/3968364306 shutdown_connections 2023-10-06T02:28:09.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.468+0000 7fc5a759e700 1 -- 172.21.15.31:0/3968364306 wait complete. 2023-10-06T02:28:09.486 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume create 'abcd@#' 2023-10-06T02:28:09.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.676+0000 7f35a319d700 1 Processor -- start 2023-10-06T02:28:09.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.676+0000 7f35a319d700 1 -- start start 2023-10-06T02:28:09.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.677+0000 7f35a319d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f359c0c7be0 0x7f359c06abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:09.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.677+0000 7f35a319d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f359c06b290 con 0x7f359c13f780 2023-10-06T02:28:09.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.677+0000 7f35a319d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f359c06b570 con 0x7f359c0c7fb0 2023-10-06T02:28:09.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.677+0000 7f35a319d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f359c06bf10 con 0x7f359c0c7be0 2023-10-06T02:28:09.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.677+0000 7f35a219b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f359c0c7be0 0x7f359c06abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:09.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.677+0000 7f35a299c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f359c13f780 0x7f359c13fba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:35266/0 (socket says 172.21.15.31:35266) 2023-10-06T02:28:09.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.677+0000 7f35a219b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f359c0c7be0 0x7f359c06abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38032/0 (socket says 172.21.15.31:38032) 2023-10-06T02:28:09.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.677+0000 7f35a299c700 1 -- 172.21.15.31:0/540113457 learned_addr learned my addr 172.21.15.31:0/540113457 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:28:09.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.678+0000 7f35a219b700 1 -- 172.21.15.31:0/540113457 >> v1:172.21.15.62:6789/0 conn(0x7f359c0c7fb0 legacy=0x7f359c15a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:09.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.678+0000 7f35a219b700 1 -- 172.21.15.31:0/540113457 >> v1:172.21.15.31:6789/0 conn(0x7f359c13f780 legacy=0x7f359c13fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:09.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.678+0000 7f35a219b700 1 -- 172.21.15.31:0/540113457 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f359c06b850 con 0x7f359c0c7be0 2023-10-06T02:28:09.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.678+0000 7f35a1199700 1 -- 172.21.15.31:0/540113457 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3724683474 0 0) 0x7f359c06b290 con 0x7f359c13f780 2023-10-06T02:28:09.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.678+0000 7f35a219b700 1 --2- 172.21.15.31:0/540113457 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f359c0c7be0 0x7f359c06abd0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f359800bb90 tx=0x7f359800b430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9445c1928e6c92a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:09.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.678+0000 7f35a1199700 1 -- 172.21.15.31:0/540113457 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f359800dba0 con 0x7f359c0c7be0 2023-10-06T02:28:09.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.678+0000 7f35a1199700 1 -- 172.21.15.31:0/540113457 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f359800dd00 con 0x7f359c0c7be0 2023-10-06T02:28:09.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.678+0000 7f35a1199700 1 -- 172.21.15.31:0/540113457 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35980196b0 con 0x7f359c0c7be0 2023-10-06T02:28:09.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.679+0000 7f35a319d700 1 -- 172.21.15.31:0/540113457 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f359c0c7be0 msgr2=0x7f359c06abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:09.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.679+0000 7f35a319d700 1 --2- 172.21.15.31:0/540113457 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f359c0c7be0 0x7f359c06abd0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f359800bb90 tx=0x7f359800b430 comp rx=0 tx=0).stop 2023-10-06T02:28:09.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.679+0000 7f35a319d700 1 -- 172.21.15.31:0/540113457 shutdown_connections 2023-10-06T02:28:09.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.679+0000 7f35a319d700 1 --2- 172.21.15.31:0/540113457 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f359c0c7be0 0x7f359c06abd0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:09.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.679+0000 7f35a319d700 1 -- 172.21.15.31:0/540113457 >> 172.21.15.31:0/540113457 conn(0x7f359c06d110 msgr2=0x7f359c0cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:09.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.679+0000 7f35a319d700 1 -- 172.21.15.31:0/540113457 shutdown_connections 2023-10-06T02:28:09.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.679+0000 7f35a319d700 1 -- 172.21.15.31:0/540113457 wait complete. 2023-10-06T02:28:09.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.680+0000 7f35a319d700 1 Processor -- start 2023-10-06T02:28:09.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.681+0000 7f35a319d700 1 -- start start 2023-10-06T02:28:09.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.681+0000 7f35a319d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f359c0c7be0 0x7f359c13b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:09.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.681+0000 7f35a319d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f359c0c7fb0 0x7f359c158c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:09.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.681+0000 7f35a319d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f359c13f780 0x7f359c159150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:09.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.681+0000 7f35a319d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f359c0c3600 con 0x7f359c0c7fb0 2023-10-06T02:28:09.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.681+0000 7f35a319d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f359c13e880 con 0x7f359c0c7be0 2023-10-06T02:28:09.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.681+0000 7f35a319d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f359c13eb60 con 0x7f359c13f780 2023-10-06T02:28:09.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.682+0000 7f35a199a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f359c0c7fb0 0x7f359c158c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:09.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.682+0000 7f35a199a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f359c0c7fb0 0x7f359c158c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38036/0 (socket says 172.21.15.31:38036) 2023-10-06T02:28:09.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.682+0000 7f35a199a700 1 -- 172.21.15.31:0/4115190990 learned_addr learned my addr 172.21.15.31:0/4115190990 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:09.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.682+0000 7f35a219b700 1 --2- 172.21.15.31:0/4115190990 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f359c0c7be0 0x7f359c13b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:09.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.682+0000 7f35a299c700 1 --2- 172.21.15.31:0/4115190990 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f359c13f780 0x7f359c159150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:09.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.682+0000 7f35a199a700 1 -- 172.21.15.31:0/4115190990 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f359c13f780 msgr2=0x7f359c159150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:09.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.682+0000 7f35a199a700 1 --2- 172.21.15.31:0/4115190990 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f359c13f780 0x7f359c159150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:09.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.682+0000 7f35a199a700 1 -- 172.21.15.31:0/4115190990 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f359c0c7be0 msgr2=0x7f359c13b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:09.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.682+0000 7f35a199a700 1 --2- 172.21.15.31:0/4115190990 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f359c0c7be0 0x7f359c13b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:09.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.682+0000 7f35a199a700 1 -- 172.21.15.31:0/4115190990 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f359800b6d0 con 0x7f359c0c7fb0 2023-10-06T02:28:09.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.682+0000 7f35a199a700 1 --2- 172.21.15.31:0/4115190990 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f359c0c7fb0 0x7f359c158c10 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f358c00c7c0 tx=0x7f358c007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:09.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.683+0000 7f35927fc700 1 -- 172.21.15.31:0/4115190990 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f358c011590 con 0x7f359c0c7fb0 2023-10-06T02:28:09.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.683+0000 7f35927fc700 1 -- 172.21.15.31:0/4115190990 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f358c0116f0 con 0x7f359c0c7fb0 2023-10-06T02:28:09.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.683+0000 7f35927fc700 1 -- 172.21.15.31:0/4115190990 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f358c011ab0 con 0x7f359c0c7fb0 2023-10-06T02:28:09.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.683+0000 7f35a319d700 1 -- 172.21.15.31:0/4115190990 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f359c13ee40 con 0x7f359c0c7fb0 2023-10-06T02:28:09.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.683+0000 7f35a319d700 1 -- 172.21.15.31:0/4115190990 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f359c165f90 con 0x7f359c0c7fb0 2023-10-06T02:28:09.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.684+0000 7f35a319d700 1 -- 172.21.15.31:0/4115190990 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f359c0c33f0 con 0x7f359c0c7fb0 2023-10-06T02:28:09.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.685+0000 7f35927fc700 1 -- 172.21.15.31:0/4115190990 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f358c01b3b0 con 0x7f359c0c7fb0 2023-10-06T02:28:09.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.685+0000 7f35927fc700 1 --2- 172.21.15.31:0/4115190990 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f3580067050 0x7f3580069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:09.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.686+0000 7f35927fc700 1 -- 172.21.15.31:0/4115190990 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f358c083d50 con 0x7f359c0c7fb0 2023-10-06T02:28:09.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.686+0000 7f35a219b700 1 --2- 172.21.15.31:0/4115190990 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f3580067050 0x7f3580069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:09.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.687+0000 7f35a219b700 1 --2- 172.21.15.31:0/4115190990 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f3580067050 0x7f3580069500 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f359800b6a0 tx=0x7f359801b000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:09.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:09.689+0000 7f35927fc700 1 -- 172.21.15.31:0/4115190990 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f358c050cb0 con 0x7f359c0c7fb0 2023-10-06T02:28:09.920 INFO:teuthology.orchestra.run.smithi031.stderr:Invalid command: invalid chars @# in abcd@# 2023-10-06T02:28:09.920 INFO:teuthology.orchestra.run.smithi031.stderr:fs volume create [] : Create a CephFS volume 2023-10-06T02:28:09.920 INFO:teuthology.orchestra.run.smithi031.stderr:Error EINVAL: invalid command 2023-10-06T02:28:09.928 DEBUG:teuthology.orchestra.run:got remote process result: 22 2023-10-06T02:28:09.928 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume create testvol 2023-10-06T02:28:10.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:10.118+0000 7f0cdc99d700 1 Processor -- start 2023-10-06T02:28:10.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:10.118+0000 7f0cdc99d700 1 -- start start 2023-10-06T02:28:10.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:10.119+0000 7f0cdc99d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0cd8153a00 0x7f0cd8153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:10.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:10.119+0000 7f0cdc99d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0cd815de50 0x7f0cd815a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:10.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:10.119+0000 7f0cdc99d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cd815af60 con 0x7f0cd80cea00 2023-10-06T02:28:10.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:10.119+0000 7f0cdc99d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cd815d010 con 0x7f0cd8153a00 2023-10-06T02:28:10.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:10.119+0000 7f0cdc99d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cd815b240 con 0x7f0cd815de50 2023-10-06T02:28:10.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:10.119+0000 7f0cd77fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0cd8153a00 0x7f0cd8153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:10.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:10.120+0000 7f0cd77fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0cd8153a00 0x7f0cd8153dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38038/0 (socket says 172.21.15.31:38038) 2023-10-06T02:28:10.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:10.120+0000 7f0cd77fe700 1 -- 172.21.15.31:0/583071065 learned_addr learned my addr 172.21.15.31:0/583071065 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:10.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:10.120+0000 7f0cd6ffd700 1 --2- 172.21.15.31:0/583071065 >> v2:172.21.15.62:3300/0 conn(0x7f0cd815de50 0x7f0cd815a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:10.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:10.120+0000 7f0cd77fe700 1 -- 172.21.15.31:0/583071065 >> v2:172.21.15.62:3300/0 conn(0x7f0cd815de50 msgr2=0x7f0cd815a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:10.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:10.120+0000 7f0cd77fe700 1 --2- 172.21.15.31:0/583071065 >> v2:172.21.15.62:3300/0 conn(0x7f0cd815de50 0x7f0cd815a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:10.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:10.120+0000 7f0cd77fe700 1 -- 172.21.15.31:0/583071065 >> v1:172.21.15.62:6789/0 conn(0x7f0cd80cea00 legacy=0x7f0cd80cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:10.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:10.120+0000 7f0cd77fe700 1 -- 172.21.15.31:0/583071065 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cd815b520 con 0x7f0cd8153a00 2023-10-06T02:28:10.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:10.120+0000 7f0cd77fe700 1 --2- 172.21.15.31:0/583071065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cd8153a00 0x7f0cd8153dd0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f0cd80b3070 tx=0x7f0cc8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca7052c37ed55727 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:10.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:10.121+0000 7f0cd67fc700 1 -- 172.21.15.31:0/583071065 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc800b860 con 0x7f0cd8153a00 2023-10-06T02:28:10.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:10.121+0000 7f0cd67fc700 1 -- 172.21.15.31:0/583071065 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cc800b9c0 con 0x7f0cd8153a00 2023-10-06T02:28:10.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:10.121+0000 7f0cd67fc700 1 -- 172.21.15.31:0/583071065 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc800bd60 con 0x7f0cd8153a00 2023-10-06T02:28:10.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:10.121+0000 7f0cdc99d700 1 -- 172.21.15.31:0/583071065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cd8153a00 msgr2=0x7f0cd8153dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:10.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:10.121+0000 7f0cdc99d700 1 --2- 172.21.15.31:0/583071065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cd8153a00 0x7f0cd8153dd0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f0cd80b3070 tx=0x7f0cc8009250 comp rx=0 tx=0).stop 2023-10-06T02:28:10.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:10.121+0000 7f0cdc99d700 1 -- 172.21.15.31:0/583071065 shutdown_connections 2023-10-06T02:28:10.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:10.121+0000 7f0cdc99d700 1 --2- 172.21.15.31:0/583071065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cd8153a00 0x7f0cd8153dd0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:10.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:10.121+0000 7f0cdc99d700 1 --2- 172.21.15.31:0/583071065 >> v2:172.21.15.62:3300/0 conn(0x7f0cd815de50 0x7f0cd815a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:10.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:10.121+0000 7f0cdc99d700 1 -- 172.21.15.31:0/583071065 >> 172.21.15.31:0/583071065 conn(0x7f0cd80703a0 msgr2=0x7f0cd80707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:10.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:10.121+0000 7f0cdc99d700 1 -- 172.21.15.31:0/583071065 shutdown_connections 2023-10-06T02:28:10.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:10.121+0000 7f0cdc99d700 1 -- 172.21.15.31:0/583071065 wait complete. 2023-10-06T02:28:10.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:10.123+0000 7f0cdc99d700 1 Processor -- start 2023-10-06T02:28:10.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:10.123+0000 7f0cdc99d700 1 -- start start 2023-10-06T02:28:10.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:10.124+0000 7f0cdc99d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0cd80cea00 0x7f0cd80c7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:10.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:10.124+0000 7f0cdc99d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0cd8153a00 0x7f0cd80c98d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:10.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:10.124+0000 7f0cdc99d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cd815de50 0x7f0cd80c9e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:10.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:10.124+0000 7f0cdc99d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd80cc1b0 con 0x7f0cd815de50 2023-10-06T02:28:10.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:10.124+0000 7f0cdc99d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd80ca350 con 0x7f0cd8153a00 2023-10-06T02:28:10.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:10.124+0000 7f0cdc99d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd80cb5a0 con 0x7f0cd80cea00 2023-10-06T02:28:10.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:10.124+0000 7f0cd7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cd815de50 0x7f0cd80c9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:10.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:10.124+0000 7f0cd7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cd815de50 0x7f0cd80c9e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38042/0 (socket says 172.21.15.31:38042) 2023-10-06T02:28:10.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:10.124+0000 7f0cd7fff700 1 -- 172.21.15.31:0/807180997 learned_addr learned my addr 172.21.15.31:0/807180997 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:10.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:10.124+0000 7f0cd6ffd700 1 --2- 172.21.15.31:0/807180997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0cd8153a00 0x7f0cd80c98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:10.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:10.124+0000 7f0cd77fe700 1 --2- 172.21.15.31:0/807180997 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0cd80cea00 0x7f0cd80c7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:10.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:10.124+0000 7f0cd7fff700 1 -- 172.21.15.31:0/807180997 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0cd80cea00 msgr2=0x7f0cd80c7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:10.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:10.124+0000 7f0cd7fff700 1 --2- 172.21.15.31:0/807180997 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0cd80cea00 0x7f0cd80c7830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:10.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:10.124+0000 7f0cd7fff700 1 -- 172.21.15.31:0/807180997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0cd8153a00 msgr2=0x7f0cd80c98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:10.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:10.124+0000 7f0cd7fff700 1 --2- 172.21.15.31:0/807180997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0cd8153a00 0x7f0cd80c98d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:10.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:10.124+0000 7f0cd7fff700 1 -- 172.21.15.31:0/807180997 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cc80094f0 con 0x7f0cd815de50 2023-10-06T02:28:10.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:10.124+0000 7f0cd6ffd700 1 --2- 172.21.15.31:0/807180997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0cd8153a00 0x7f0cd80c98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:28:10.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:10.124+0000 7f0cd77fe700 1 --2- 172.21.15.31:0/807180997 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0cd80cea00 0x7f0cd80c7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:28:10.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:10.125+0000 7f0cd7fff700 1 --2- 172.21.15.31:0/807180997 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cd815de50 0x7f0cd80c9e10 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f0cd000c840 tx=0x7f0cd0001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:10.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:10.125+0000 7f0cbf7fe700 1 -- 172.21.15.31:0/807180997 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd000a650 con 0x7f0cd815de50 2023-10-06T02:28:10.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:10.125+0000 7f0cbf7fe700 1 -- 172.21.15.31:0/807180997 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cd000a7d0 con 0x7f0cd815de50 2023-10-06T02:28:10.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:10.125+0000 7f0cbf7fe700 1 -- 172.21.15.31:0/807180997 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd000ab90 con 0x7f0cd815de50 2023-10-06T02:28:10.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:10.125+0000 7f0cdc99d700 1 -- 172.21.15.31:0/807180997 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cd80cb880 con 0x7f0cd815de50 2023-10-06T02:28:10.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:10.125+0000 7f0cdc99d700 1 -- 172.21.15.31:0/807180997 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cd80cbe40 con 0x7f0cd815de50 2023-10-06T02:28:10.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:10.126+0000 7f0cdc99d700 1 -- 172.21.15.31:0/807180997 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cd80cc360 con 0x7f0cd815de50 2023-10-06T02:28:10.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:10.127+0000 7f0cbf7fe700 1 -- 172.21.15.31:0/807180997 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f0cd0003030 con 0x7f0cd815de50 2023-10-06T02:28:10.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:10.128+0000 7f0cbf7fe700 1 --2- 172.21.15.31:0/807180997 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0cc4067050 0x7f0cc4069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:10.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:10.128+0000 7f0cbf7fe700 1 -- 172.21.15.31:0/807180997 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0cd0084960 con 0x7f0cd815de50 2023-10-06T02:28:10.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:10.128+0000 7f0cd77fe700 1 --2- 172.21.15.31:0/807180997 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0cc4067050 0x7f0cc4069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:10.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:10.129+0000 7f0cd77fe700 1 --2- 172.21.15.31:0/807180997 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0cc4067050 0x7f0cc4069500 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0cd80b3070 tx=0x7f0cc8017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:10.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:10.132+0000 7f0cbf7fe700 1 -- 172.21.15.31:0/807180997 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0cd80cc360 con 0x7f0cd815de50 2023-10-06T02:28:10.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:10.360+0000 7f0cdc99d700 1 -- 172.21.15.31:0/807180997 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "testvol", "target": ["mon-mgr", ""]}) v1 -- 0x7f0cd80cc360 con 0x7f0cc4067050 2023-10-06T02:28:13.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.075+0000 7f0cbf7fe700 1 -- 172.21.15.31:0/807180997 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: 0 Volume created successfully (no MDS daemons created)) v1 ==== 60+0+0 (secure 0 0 0) 0x7f0cd80cc360 con 0x7f0cc4067050 2023-10-06T02:28:13.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.080+0000 7f0cdc99d700 1 -- 172.21.15.31:0/807180997 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0cc4067050 msgr2=0x7f0cc4069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:13.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.080+0000 7f0cdc99d700 1 --2- 172.21.15.31:0/807180997 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0cc4067050 0x7f0cc4069500 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0cd80b3070 tx=0x7f0cc8017000 comp rx=0 tx=0).stop 2023-10-06T02:28:13.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.080+0000 7f0cdc99d700 1 -- 172.21.15.31:0/807180997 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cd815de50 msgr2=0x7f0cd80c9e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:13.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.080+0000 7f0cdc99d700 1 --2- 172.21.15.31:0/807180997 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cd815de50 0x7f0cd80c9e10 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f0cd000c840 tx=0x7f0cd0001670 comp rx=0 tx=0).stop 2023-10-06T02:28:13.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.081+0000 7f0cdc99d700 1 -- 172.21.15.31:0/807180997 shutdown_connections 2023-10-06T02:28:13.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.081+0000 7f0cdc99d700 1 --2- 172.21.15.31:0/807180997 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cd815de50 0x7f0cd80c9e10 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:13.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.081+0000 7f0cdc99d700 1 --2- 172.21.15.31:0/807180997 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0cc4067050 0x7f0cc4069500 secure :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0cd80b3070 tx=0x7f0cc8017000 comp rx=0 tx=0).stop 2023-10-06T02:28:13.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.081+0000 7f0cdc99d700 1 --2- 172.21.15.31:0/807180997 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0cd80cea00 0x7f0cd80c7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:13.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.081+0000 7f0cdc99d700 1 --2- 172.21.15.31:0/807180997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0cd8153a00 0x7f0cd80c98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:13.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.081+0000 7f0cdc99d700 1 -- 172.21.15.31:0/807180997 >> 172.21.15.31:0/807180997 conn(0x7f0cd80703a0 msgr2=0x7f0cd8153030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:13.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.081+0000 7f0cdc99d700 1 -- 172.21.15.31:0/807180997 shutdown_connections 2023-10-06T02:28:13.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.082+0000 7f0cdc99d700 1 -- 172.21.15.31:0/807180997 wait complete. 2023-10-06T02:28:13.085 INFO:teuthology.orchestra.run.smithi031.stderr:Volume created successfully (no MDS daemons created) 2023-10-06T02:28:13.108 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create testvol 'abcd@#' 2023-10-06T02:28:13.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.300+0000 7ff57cb9d700 1 Processor -- start 2023-10-06T02:28:13.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.300+0000 7ff57cb9d700 1 -- start start 2023-10-06T02:28:13.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.301+0000 7ff57cb9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff578155c60 0x7ff5780c7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:13.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.301+0000 7ff57cb9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5780c7e70 con 0x7ff578155890 2023-10-06T02:28:13.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.301+0000 7ff57cb9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff57806b790 con 0x7ff57806de50 2023-10-06T02:28:13.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.301+0000 7ff57cb9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff57806ab80 con 0x7ff578155c60 2023-10-06T02:28:13.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.301+0000 7ff5777fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7ff578155890 0x7ff57806a3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:35270/0 (socket says 172.21.15.31:35270) 2023-10-06T02:28:13.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.301+0000 7ff5777fe700 1 -- 172.21.15.31:0/3899910372 learned_addr learned my addr 172.21.15.31:0/3899910372 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:28:13.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.301+0000 7ff576ffd700 1 --2- 172.21.15.31:0/3899910372 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff578155c60 0x7ff5780c7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:13.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.302+0000 7ff5767fc700 1 -- 172.21.15.31:0/3899910372 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2839330801 0 0) 0x7ff5780c7e70 con 0x7ff578155890 2023-10-06T02:28:13.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.302+0000 7ff5767fc700 1 -- 172.21.15.31:0/3899910372 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff560003570 con 0x7ff578155890 2023-10-06T02:28:13.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.302+0000 7ff576ffd700 1 -- 172.21.15.31:0/3899910372 >> v1:172.21.15.62:6789/0 conn(0x7ff57806de50 legacy=0x7ff5780c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:13.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.302+0000 7ff5767fc700 1 -- 172.21.15.31:0/3899910372 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2973517048 0 0) 0x7ff57806b790 con 0x7ff57806de50 2023-10-06T02:28:13.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.302+0000 7ff576ffd700 1 -- 172.21.15.31:0/3899910372 >> v1:172.21.15.31:6789/0 conn(0x7ff578155890 legacy=0x7ff57806a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:13.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.302+0000 7ff576ffd700 1 -- 172.21.15.31:0/3899910372 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff57806ae60 con 0x7ff578155c60 2023-10-06T02:28:13.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.302+0000 7ff576ffd700 1 --2- 172.21.15.31:0/3899910372 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff578155c60 0x7ff5780c7930 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff56c009910 tx=0x7ff56c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28c589b7de93a44f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:13.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.303+0000 7ff5767fc700 1 -- 172.21.15.31:0/3899910372 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff56c00b680 con 0x7ff578155c60 2023-10-06T02:28:13.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.303+0000 7ff5767fc700 1 -- 172.21.15.31:0/3899910372 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff56c00b7e0 con 0x7ff578155c60 2023-10-06T02:28:13.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.303+0000 7ff5767fc700 1 -- 172.21.15.31:0/3899910372 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff56c00bba0 con 0x7ff578155c60 2023-10-06T02:28:13.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.303+0000 7ff57cb9d700 1 -- 172.21.15.31:0/3899910372 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff578155c60 msgr2=0x7ff5780c7930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:13.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.303+0000 7ff57cb9d700 1 --2- 172.21.15.31:0/3899910372 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff578155c60 0x7ff5780c7930 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff56c009910 tx=0x7ff56c0091a0 comp rx=0 tx=0).stop 2023-10-06T02:28:13.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.303+0000 7ff57cb9d700 1 -- 172.21.15.31:0/3899910372 shutdown_connections 2023-10-06T02:28:13.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.303+0000 7ff57cb9d700 1 --2- 172.21.15.31:0/3899910372 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff578155c60 0x7ff5780c7930 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:13.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.303+0000 7ff57cb9d700 1 -- 172.21.15.31:0/3899910372 >> 172.21.15.31:0/3899910372 conn(0x7ff57806d110 msgr2=0x7ff5780cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:13.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.303+0000 7ff57cb9d700 1 -- 172.21.15.31:0/3899910372 shutdown_connections 2023-10-06T02:28:13.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.303+0000 7ff57cb9d700 1 -- 172.21.15.31:0/3899910372 wait complete. 2023-10-06T02:28:13.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.305+0000 7ff57cb9d700 1 Processor -- start 2023-10-06T02:28:13.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.305+0000 7ff57cb9d700 1 -- start start 2023-10-06T02:28:13.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.306+0000 7ff57cb9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff57806de50 0x7ff578167240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:13.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.306+0000 7ff57cb9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff578155890 0x7ff57814d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:13.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.306+0000 7ff57cb9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff578155c60 0x7ff578169f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:13.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.306+0000 7ff57cb9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff578150000 con 0x7ff57806de50 2023-10-06T02:28:13.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.306+0000 7ff57cb9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff578150e60 con 0x7ff578155890 2023-10-06T02:28:13.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.306+0000 7ff57cb9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff578150250 con 0x7ff578155c60 2023-10-06T02:28:13.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.306+0000 7ff5777fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff57806de50 0x7ff578167240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:13.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.306+0000 7ff5777fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff57806de50 0x7ff578167240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38056/0 (socket says 172.21.15.31:38056) 2023-10-06T02:28:13.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.306+0000 7ff5777fe700 1 -- 172.21.15.31:0/1836427827 learned_addr learned my addr 172.21.15.31:0/1836427827 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:13.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.306+0000 7ff5777fe700 1 -- 172.21.15.31:0/1836427827 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff578155c60 msgr2=0x7ff578169f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:13.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.306+0000 7ff5777fe700 1 --2- 172.21.15.31:0/1836427827 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff578155c60 0x7ff578169f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:13.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.306+0000 7ff5777fe700 1 -- 172.21.15.31:0/1836427827 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff578155890 msgr2=0x7ff57814d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:13.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.306+0000 7ff5777fe700 1 --2- 172.21.15.31:0/1836427827 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff578155890 0x7ff57814d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:13.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.307+0000 7ff5777fe700 1 -- 172.21.15.31:0/1836427827 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff56c009420 con 0x7ff57806de50 2023-10-06T02:28:13.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.307+0000 7ff5777fe700 1 --2- 172.21.15.31:0/1836427827 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff57806de50 0x7ff578167240 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7ff5680011e0 tx=0x7ff568007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:13.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.307+0000 7ff55f7fe700 1 -- 172.21.15.31:0/1836427827 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff568005b30 con 0x7ff57806de50 2023-10-06T02:28:13.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.307+0000 7ff55f7fe700 1 -- 172.21.15.31:0/1836427827 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff568005c90 con 0x7ff57806de50 2023-10-06T02:28:13.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.307+0000 7ff55f7fe700 1 -- 172.21.15.31:0/1836427827 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5680072e0 con 0x7ff57806de50 2023-10-06T02:28:13.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.307+0000 7ff57cb9d700 1 -- 172.21.15.31:0/1836427827 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff578150530 con 0x7ff57806de50 2023-10-06T02:28:13.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.307+0000 7ff57cb9d700 1 -- 172.21.15.31:0/1836427827 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff578150af0 con 0x7ff57806de50 2023-10-06T02:28:13.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.308+0000 7ff57cb9d700 1 -- 172.21.15.31:0/1836427827 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff57804c3c0 con 0x7ff57806de50 2023-10-06T02:28:13.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.309+0000 7ff55f7fe700 1 -- 172.21.15.31:0/1836427827 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7ff568007650 con 0x7ff57806de50 2023-10-06T02:28:13.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.311+0000 7ff55f7fe700 1 --2- 172.21.15.31:0/1836427827 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff564067050 0x7ff564069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:13.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.311+0000 7ff55f7fe700 1 -- 172.21.15.31:0/1836427827 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6892+0+0 (secure 0 0 0) 0x7ff568083ff0 con 0x7ff57806de50 2023-10-06T02:28:13.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.311+0000 7ff576ffd700 1 --2- 172.21.15.31:0/1836427827 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff564067050 0x7ff564069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:13.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.312+0000 7ff576ffd700 1 --2- 172.21.15.31:0/1836427827 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff564067050 0x7ff564069500 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff57814e4f0 tx=0x7ff56c017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:13.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.316+0000 7ff55f7fe700 1 -- 172.21.15.31:0/1836427827 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff56804b5a0 con 0x7ff57806de50 2023-10-06T02:28:13.551 INFO:teuthology.orchestra.run.smithi031.stderr:Invalid command: invalid chars #@ in abcd@# 2023-10-06T02:28:13.551 INFO:teuthology.orchestra.run.smithi031.stderr:fs subvolumegroup create [] [] [] [] [] : Create a CephFS subvolume group in a volume, and optionally, with a specific data pool layout, and a specific numeric mode 2023-10-06T02:28:13.551 INFO:teuthology.orchestra.run.smithi031.stderr:Error EINVAL: invalid command 2023-10-06T02:28:13.558 DEBUG:teuthology.orchestra.run:got remote process result: 22 2023-10-06T02:28:13.559 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create testvol 'abcd@#' 2023-10-06T02:28:13.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.748+0000 7f18d219d700 1 Processor -- start 2023-10-06T02:28:13.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.748+0000 7f18d219d700 1 -- start start 2023-10-06T02:28:13.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.749+0000 7f18d219d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f18cc153dd0 0x7f18cc15de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:13.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.749+0000 7f18d219d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18cc15ac50 con 0x7f18cc0cea00 2023-10-06T02:28:13.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.749+0000 7f18d219d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18cc15d010 con 0x7f18cc153a00 2023-10-06T02:28:13.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.749+0000 7f18d219d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18cc15af60 con 0x7f18cc153dd0 2023-10-06T02:28:13.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.750+0000 7f18d099a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f18cc153dd0 0x7f18cc15de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:13.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.750+0000 7f18d099a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f18cc153dd0 0x7f18cc15de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38060/0 (socket says 172.21.15.31:38060) 2023-10-06T02:28:13.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.750+0000 7f18d199c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f18cc0cea00 0x7f18cc0ceea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:35282/0 (socket says 172.21.15.31:35282) 2023-10-06T02:28:13.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.750+0000 7f18d099a700 1 -- 172.21.15.31:0/3396394266 learned_addr learned my addr 172.21.15.31:0/3396394266 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:13.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.750+0000 7f18d099a700 1 -- 172.21.15.31:0/3396394266 >> v1:172.21.15.62:6789/0 conn(0x7f18cc153a00 legacy=0x7f18cc15a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:13.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.750+0000 7f18d099a700 1 -- 172.21.15.31:0/3396394266 >> v1:172.21.15.31:6789/0 conn(0x7f18cc0cea00 legacy=0x7f18cc0ceea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:13.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.750+0000 7f18d099a700 1 -- 172.21.15.31:0/3396394266 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18cc15b240 con 0x7f18cc153dd0 2023-10-06T02:28:13.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.750+0000 7f18c3fff700 1 -- 172.21.15.31:0/3396394266 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4180658130 0 0) 0x7f18cc15ac50 con 0x7f18cc0cea00 2023-10-06T02:28:13.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.750+0000 7f18d099a700 1 --2- 172.21.15.31:0/3396394266 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18cc153dd0 0x7f18cc15de50 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f18bc0099b0 tx=0x7f18bc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62a3e0a4df54ca79 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:13.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.751+0000 7f18c3fff700 1 -- 172.21.15.31:0/3396394266 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18bc00b860 con 0x7f18cc153dd0 2023-10-06T02:28:13.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.751+0000 7f18c3fff700 1 -- 172.21.15.31:0/3396394266 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18bc00b9c0 con 0x7f18cc153dd0 2023-10-06T02:28:13.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.751+0000 7f18c3fff700 1 -- 172.21.15.31:0/3396394266 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18bc00bd60 con 0x7f18cc153dd0 2023-10-06T02:28:13.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.751+0000 7f18d219d700 1 -- 172.21.15.31:0/3396394266 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18cc153dd0 msgr2=0x7f18cc15de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:13.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.751+0000 7f18d219d700 1 --2- 172.21.15.31:0/3396394266 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18cc153dd0 0x7f18cc15de50 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f18bc0099b0 tx=0x7f18bc009250 comp rx=0 tx=0).stop 2023-10-06T02:28:13.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.751+0000 7f18d219d700 1 -- 172.21.15.31:0/3396394266 shutdown_connections 2023-10-06T02:28:13.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.751+0000 7f18d219d700 1 --2- 172.21.15.31:0/3396394266 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18cc153dd0 0x7f18cc15de50 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:13.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.751+0000 7f18d219d700 1 -- 172.21.15.31:0/3396394266 >> 172.21.15.31:0/3396394266 conn(0x7f18cc0703a0 msgr2=0x7f18cc0707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:13.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.752+0000 7f18d219d700 1 -- 172.21.15.31:0/3396394266 shutdown_connections 2023-10-06T02:28:13.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.752+0000 7f18d219d700 1 -- 172.21.15.31:0/3396394266 wait complete. 2023-10-06T02:28:13.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.753+0000 7f18d219d700 1 Processor -- start 2023-10-06T02:28:13.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.753+0000 7f18d219d700 1 -- start start 2023-10-06T02:28:13.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.754+0000 7f18d219d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18cc0cea00 0x7f18cc151b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:13.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.754+0000 7f18d219d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18cc153a00 0x7f18cc0c7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:13.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.754+0000 7f18d219d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f18cc0c84e0 0x7f18cc0c9910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:13.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.754+0000 7f18d219d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18cc0c8cb0 con 0x7f18cc0cea00 2023-10-06T02:28:13.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.754+0000 7f18d219d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18cc0b4e50 con 0x7f18cc153a00 2023-10-06T02:28:13.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.754+0000 7f18d219d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18cc0c89d0 con 0x7f18cc0c84e0 2023-10-06T02:28:13.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.754+0000 7f18d119b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18cc0cea00 0x7f18cc151b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:13.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.754+0000 7f18d119b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18cc0cea00 0x7f18cc151b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38074/0 (socket says 172.21.15.31:38074) 2023-10-06T02:28:13.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.754+0000 7f18d119b700 1 -- 172.21.15.31:0/3521598523 learned_addr learned my addr 172.21.15.31:0/3521598523 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:13.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.755+0000 7f18d199c700 1 --2- 172.21.15.31:0/3521598523 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f18cc0c84e0 0x7f18cc0c9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:13.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.755+0000 7f18d099a700 1 --2- 172.21.15.31:0/3521598523 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18cc153a00 0x7f18cc0c7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:13.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.755+0000 7f18d119b700 1 -- 172.21.15.31:0/3521598523 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f18cc0c84e0 msgr2=0x7f18cc0c9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:13.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.755+0000 7f18d119b700 1 --2- 172.21.15.31:0/3521598523 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f18cc0c84e0 0x7f18cc0c9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:13.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.755+0000 7f18d119b700 1 -- 172.21.15.31:0/3521598523 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18cc153a00 msgr2=0x7f18cc0c7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:13.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.755+0000 7f18d119b700 1 --2- 172.21.15.31:0/3521598523 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18cc153a00 0x7f18cc0c7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:13.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.755+0000 7f18d119b700 1 -- 172.21.15.31:0/3521598523 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18bc0094f0 con 0x7f18cc0cea00 2023-10-06T02:28:13.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.755+0000 7f18d119b700 1 --2- 172.21.15.31:0/3521598523 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18cc0cea00 0x7f18cc151b90 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f18c800c4e0 tx=0x7f18c800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:13.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.755+0000 7f18c17fa700 1 -- 172.21.15.31:0/3521598523 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c8011590 con 0x7f18cc0cea00 2023-10-06T02:28:13.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.755+0000 7f18c17fa700 1 -- 172.21.15.31:0/3521598523 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18c8011710 con 0x7f18cc0cea00 2023-10-06T02:28:13.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.755+0000 7f18c17fa700 1 -- 172.21.15.31:0/3521598523 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c8011ad0 con 0x7f18cc0cea00 2023-10-06T02:28:13.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.755+0000 7f18d219d700 1 -- 172.21.15.31:0/3521598523 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18cc0b4800 con 0x7f18cc0cea00 2023-10-06T02:28:13.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.756+0000 7f18d219d700 1 -- 172.21.15.31:0/3521598523 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18cc1520d0 con 0x7f18cc0cea00 2023-10-06T02:28:13.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.756+0000 7f18d219d700 1 -- 172.21.15.31:0/3521598523 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18cc0b4660 con 0x7f18cc0cea00 2023-10-06T02:28:13.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.758+0000 7f18c17fa700 1 -- 172.21.15.31:0/3521598523 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f18c801b3b0 con 0x7f18cc0cea00 2023-10-06T02:28:13.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.758+0000 7f18c17fa700 1 --2- 172.21.15.31:0/3521598523 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f18b0067050 0x7f18b0069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:13.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.759+0000 7f18c17fa700 1 -- 172.21.15.31:0/3521598523 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6892+0+0 (secure 0 0 0) 0x7f18c8083f00 con 0x7f18cc0cea00 2023-10-06T02:28:13.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.759+0000 7f18d099a700 1 --2- 172.21.15.31:0/3521598523 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f18b0067050 0x7f18b0069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:13.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.760+0000 7f18d099a700 1 --2- 172.21.15.31:0/3521598523 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f18b0067050 0x7f18b0069500 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f18cc15e390 tx=0x7f18bc017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:13.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:13.763+0000 7f18c17fa700 1 -- 172.21.15.31:0/3521598523 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f18cc0b4660 con 0x7f18cc0cea00 2023-10-06T02:28:13.999 INFO:teuthology.orchestra.run.smithi031.stderr:Invalid command: invalid chars @# in abcd@# 2023-10-06T02:28:13.999 INFO:teuthology.orchestra.run.smithi031.stderr:fs subvolume create [] [] [] [] [] [] [--namespace-isolated] : Create a CephFS subvolume in a volume, and optionally, with a specific size (in bytes), a specific data pool layout, a specific mode, in a specific subvolume group and in separate RADOS namespace 2023-10-06T02:28:13.999 INFO:teuthology.orchestra.run.smithi031.stderr:Error EINVAL: invalid command 2023-10-06T02:28:14.006 DEBUG:teuthology.orchestra.run:got remote process result: 22 2023-10-06T02:28:14.007 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm testvol --yes-i-really-mean-it 2023-10-06T02:28:14.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:14.202+0000 7fad11f9d700 1 Processor -- start 2023-10-06T02:28:14.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:14.202+0000 7fad11f9d700 1 -- start start 2023-10-06T02:28:14.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:14.203+0000 7fad11f9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fad0c142c00 0x7fad0c06b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:14.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:14.204+0000 7fad11f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fad0c06bd20 0x7fad0c0c7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:14.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:14.204+0000 7fad11f9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fad0c142fd0 0x7fad0c0c7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:14.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:14.204+0000 7fad11f9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad0c0ca480 con 0x7fad0c142fd0 2023-10-06T02:28:14.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:14.204+0000 7fad11f9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad0c15e240 con 0x7fad0c142c00 2023-10-06T02:28:14.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:14.204+0000 7fad11f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad0c06b350 con 0x7fad0c06bd20 2023-10-06T02:28:14.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:14.204+0000 7fad1179c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fad0c142fd0 0x7fad0c0c7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:14.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:14.204+0000 7fad1179c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fad0c142fd0 0x7fad0c0c7dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38078/0 (socket says 172.21.15.31:38078) 2023-10-06T02:28:14.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:14.204+0000 7fad1179c700 1 -- 172.21.15.31:0/1385634342 learned_addr learned my addr 172.21.15.31:0/1385634342 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:14.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:14.204+0000 7fad10f9b700 1 --2- 172.21.15.31:0/1385634342 >> v2:172.21.15.62:3300/0 conn(0x7fad0c142c00 0x7fad0c06b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:14.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:14.204+0000 7fad0bfff700 1 --2- 172.21.15.31:0/1385634342 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fad0c06bd20 0x7fad0c0c7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:14.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:14.204+0000 7fad1179c700 1 -- 172.21.15.31:0/1385634342 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fad0c06bd20 msgr2=0x7fad0c0c7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:14.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:14.204+0000 7fad1179c700 1 --2- 172.21.15.31:0/1385634342 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fad0c06bd20 0x7fad0c0c7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:14.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:14.204+0000 7fad1179c700 1 -- 172.21.15.31:0/1385634342 >> v2:172.21.15.62:3300/0 conn(0x7fad0c142c00 msgr2=0x7fad0c06b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:14.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:14.204+0000 7fad1179c700 1 --2- 172.21.15.31:0/1385634342 >> v2:172.21.15.62:3300/0 conn(0x7fad0c142c00 0x7fad0c06b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:14.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:14.204+0000 7fad1179c700 1 -- 172.21.15.31:0/1385634342 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad0c15df60 con 0x7fad0c142fd0 2023-10-06T02:28:14.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:14.205+0000 7fad1179c700 1 --2- 172.21.15.31:0/1385634342 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad0c142fd0 0x7fad0c0c7dc0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fad0c06f490 tx=0x7fad04009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c27baa04ffc30c6d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:14.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:14.205+0000 7fad0b7fe700 1 -- 172.21.15.31:0/1385634342 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0400e070 con 0x7fad0c142fd0 2023-10-06T02:28:14.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:14.205+0000 7fad0b7fe700 1 -- 172.21.15.31:0/1385634342 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad0400ba50 con 0x7fad0c142fd0 2023-10-06T02:28:14.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:14.205+0000 7fad0b7fe700 1 -- 172.21.15.31:0/1385634342 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad040092e0 con 0x7fad0c142fd0 2023-10-06T02:28:14.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:14.205+0000 7fad11f9d700 1 -- 172.21.15.31:0/1385634342 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad0c142fd0 msgr2=0x7fad0c0c7dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:14.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:14.205+0000 7fad11f9d700 1 --2- 172.21.15.31:0/1385634342 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad0c142fd0 0x7fad0c0c7dc0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fad0c06f490 tx=0x7fad04009840 comp rx=0 tx=0).stop 2023-10-06T02:28:14.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:14.206+0000 7fad11f9d700 1 -- 172.21.15.31:0/1385634342 shutdown_connections 2023-10-06T02:28:14.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:14.206+0000 7fad11f9d700 1 --2- 172.21.15.31:0/1385634342 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad0c142fd0 0x7fad0c0c7dc0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:14.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:14.206+0000 7fad11f9d700 1 --2- 172.21.15.31:0/1385634342 >> v2:172.21.15.62:3300/0 conn(0x7fad0c142c00 0x7fad0c06b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:14.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:14.206+0000 7fad11f9d700 1 --2- 172.21.15.31:0/1385634342 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fad0c06bd20 0x7fad0c0c7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:14.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:14.206+0000 7fad11f9d700 1 -- 172.21.15.31:0/1385634342 >> 172.21.15.31:0/1385634342 conn(0x7fad0c0675a0 msgr2=0x7fad0c0679a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:14.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:14.206+0000 7fad11f9d700 1 -- 172.21.15.31:0/1385634342 shutdown_connections 2023-10-06T02:28:14.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:14.206+0000 7fad11f9d700 1 -- 172.21.15.31:0/1385634342 wait complete. 2023-10-06T02:28:14.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:14.207+0000 7fad11f9d700 1 Processor -- start 2023-10-06T02:28:14.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:14.207+0000 7fad11f9d700 1 -- start start 2023-10-06T02:28:14.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:14.208+0000 7fad11f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fad0c06bd20 0x7fad0c0c42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:14.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:14.208+0000 7fad11f9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad0c142c00 0x7fad0c158f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:14.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:14.208+0000 7fad11f9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad0c142fd0 0x7fad0c0b2910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:14.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:14.208+0000 7fad11f9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad0c1594c0 con 0x7fad0c142c00 2023-10-06T02:28:14.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:14.208+0000 7fad11f9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad0c0b2480 con 0x7fad0c142fd0 2023-10-06T02:28:14.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:14.208+0000 7fad11f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad0c0b1870 con 0x7fad0c06bd20 2023-10-06T02:28:14.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:14.208+0000 7fad0bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad0c142c00 0x7fad0c158f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:14.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:14.208+0000 7fad0bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad0c142c00 0x7fad0c158f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38080/0 (socket says 172.21.15.31:38080) 2023-10-06T02:28:14.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:14.208+0000 7fad0bfff700 1 -- 172.21.15.31:0/1976119228 learned_addr learned my addr 172.21.15.31:0/1976119228 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:14.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:14.208+0000 7fad10f9b700 1 --2- 172.21.15.31:0/1976119228 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fad0c06bd20 0x7fad0c0c42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:14.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:14.209+0000 7fad1179c700 1 --2- 172.21.15.31:0/1976119228 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad0c142fd0 0x7fad0c0b2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:14.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:14.209+0000 7fad0bfff700 1 -- 172.21.15.31:0/1976119228 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fad0c06bd20 msgr2=0x7fad0c0c42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:14.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:14.209+0000 7fad0bfff700 1 --2- 172.21.15.31:0/1976119228 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fad0c06bd20 0x7fad0c0c42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:14.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:14.209+0000 7fad0bfff700 1 -- 172.21.15.31:0/1976119228 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad0c142fd0 msgr2=0x7fad0c0b2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:14.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:14.209+0000 7fad0bfff700 1 --2- 172.21.15.31:0/1976119228 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad0c142fd0 0x7fad0c0b2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:14.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:14.209+0000 7fad0bfff700 1 -- 172.21.15.31:0/1976119228 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad04009ae0 con 0x7fad0c142c00 2023-10-06T02:28:14.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:14.209+0000 7fad0bfff700 1 --2- 172.21.15.31:0/1976119228 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad0c142c00 0x7fad0c158f80 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7facf800d890 tx=0x7facf8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:14.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:14.209+0000 7fad08ff9700 1 -- 172.21.15.31:0/1976119228 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf80125f0 con 0x7fad0c142c00 2023-10-06T02:28:14.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:14.209+0000 7fad08ff9700 1 -- 172.21.15.31:0/1976119228 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facf8012770 con 0x7fad0c142c00 2023-10-06T02:28:14.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:14.209+0000 7fad11f9d700 1 -- 172.21.15.31:0/1976119228 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad0c0b1b50 con 0x7fad0c142c00 2023-10-06T02:28:14.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:14.209+0000 7fad08ff9700 1 -- 172.21.15.31:0/1976119228 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf8012b30 con 0x7fad0c142c00 2023-10-06T02:28:14.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:14.209+0000 7fad11f9d700 1 -- 172.21.15.31:0/1976119228 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad0c0b2110 con 0x7fad0c142c00 2023-10-06T02:28:14.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:14.210+0000 7fad11f9d700 1 -- 172.21.15.31:0/1976119228 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad0c0b25f0 con 0x7fad0c142c00 2023-10-06T02:28:14.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:14.211+0000 7fad08ff9700 1 -- 172.21.15.31:0/1976119228 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7facf801e460 con 0x7fad0c142c00 2023-10-06T02:28:14.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:14.212+0000 7fad08ff9700 1 --2- 172.21.15.31:0/1976119228 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7facf0067050 0x7facf0069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:14.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:14.212+0000 7fad08ff9700 1 -- 172.21.15.31:0/1976119228 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6892+0+0 (secure 0 0 0) 0x7facf8085190 con 0x7fad0c142c00 2023-10-06T02:28:14.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:14.213+0000 7fad10f9b700 1 --2- 172.21.15.31:0/1976119228 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7facf0067050 0x7facf0069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:14.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:14.213+0000 7fad10f9b700 1 --2- 172.21.15.31:0/1976119228 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7facf0067050 0x7facf0069500 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fad00006fd0 tx=0x7fad00008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:14.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:14.216+0000 7fad08ff9700 1 -- 172.21.15.31:0/1976119228 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7facf8051db0 con 0x7fad0c142c00 2023-10-06T02:28:14.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:14.450+0000 7fad11f9d700 1 -- 172.21.15.31:0/1976119228 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "fs volume rm", "vol_name": "testvol", "yes-i-really-mean-it": "--yes-i-really-mean-it", "target": ["mon-mgr", ""]}) v1 -- 0x7fad0c04c3c0 con 0x7facf0067050 2023-10-06T02:28:14.456 INFO:tasks.ceph.mon.a.smithi031.stderr:2023-10-06T02:28:14.455+0000 7f6de01f1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T02:28:14.507 INFO:tasks.ceph.mds.d.smithi062.stdout:starting mds.d at 2023-10-06T02:28:17.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:17.480+0000 7fad08ff9700 1 -- 172.21.15.31:0/1976119228 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+78 (secure 0 0 0) 0x7fad0c04c3c0 con 0x7facf0067050 2023-10-06T02:28:17.482 INFO:teuthology.orchestra.run.smithi031.stdout:metadata pool: cephfs.testvol.meta data pool: ['cephfs.testvol.data'] removed 2023-10-06T02:28:17.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:17.483+0000 7fad11f9d700 1 -- 172.21.15.31:0/1976119228 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7facf0067050 msgr2=0x7facf0069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:17.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:17.483+0000 7fad11f9d700 1 --2- 172.21.15.31:0/1976119228 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7facf0067050 0x7facf0069500 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fad00006fd0 tx=0x7fad00008040 comp rx=0 tx=0).stop 2023-10-06T02:28:17.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:17.483+0000 7fad11f9d700 1 -- 172.21.15.31:0/1976119228 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad0c142c00 msgr2=0x7fad0c158f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:17.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:17.483+0000 7fad11f9d700 1 --2- 172.21.15.31:0/1976119228 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad0c142c00 0x7fad0c158f80 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7facf800d890 tx=0x7facf8008040 comp rx=0 tx=0).stop 2023-10-06T02:28:17.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:17.483+0000 7fad11f9d700 1 -- 172.21.15.31:0/1976119228 shutdown_connections 2023-10-06T02:28:17.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:17.483+0000 7fad11f9d700 1 --2- 172.21.15.31:0/1976119228 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad0c142fd0 0x7fad0c0b2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:17.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:17.483+0000 7fad11f9d700 1 --2- 172.21.15.31:0/1976119228 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7facf0067050 0x7facf0069500 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:17.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:17.483+0000 7fad11f9d700 1 --2- 172.21.15.31:0/1976119228 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fad0c06bd20 0x7fad0c0c42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:17.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:17.483+0000 7fad11f9d700 1 --2- 172.21.15.31:0/1976119228 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad0c142c00 0x7fad0c158f80 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:17.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:17.483+0000 7fad11f9d700 1 -- 172.21.15.31:0/1976119228 >> 172.21.15.31:0/1976119228 conn(0x7fad0c0675a0 msgr2=0x7fad0c0ca8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:17.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:17.483+0000 7fad11f9d700 1 -- 172.21.15.31:0/1976119228 shutdown_connections 2023-10-06T02:28:17.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:17.483+0000 7fad11f9d700 1 -- 172.21.15.31:0/1976119228 wait complete. 2023-10-06T02:28:17.505 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T02:28:17.506 DEBUG:teuthology.orchestra.run.smithi031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T02:28:17.588 DEBUG:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T02:28:17.677 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T02:28:17.678 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T02:28:17.678 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:28:17.678 DEBUG:teuthology.orchestra.run.smithi031:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T02:28:17.765 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi031.front.sepia.ceph.com 2023-10-06T02:28:17.765 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:28:17.765 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T02:28:17.802 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-06T02:28:17.802 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:28:17.802 DEBUG:teuthology.orchestra.run.smithi062:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-06T02:28:17.885 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi062.front.sepia.ceph.com 2023-10-06T02:28:17.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:28:17.885 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T02:28:17.922 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:28:18.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.108+0000 7f7aa959d700 1 Processor -- start 2023-10-06T02:28:18.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.108+0000 7f7aa959d700 1 -- start start 2023-10-06T02:28:18.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.109+0000 7f7aa959d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7aa406aaa0 0x7f7aa415a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:18.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.109+0000 7f7aa959d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7aa415ae70 con 0x7f7aa413fb20 2023-10-06T02:28:18.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.110+0000 7f7aa959d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7aa406af60 con 0x7f7aa413fef0 2023-10-06T02:28:18.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.110+0000 7f7aa959d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7aa406b240 con 0x7f7aa406aaa0 2023-10-06T02:28:18.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.110+0000 7f7aa3fff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f7aa413fb20 0x7f7aa40c7990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:43654/0 (socket says 172.21.15.31:43654) 2023-10-06T02:28:18.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.110+0000 7f7aa3fff700 1 -- 172.21.15.31:0/2201367319 learned_addr learned my addr 172.21.15.31:0/2201367319 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:28:18.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.110+0000 7f7aa8d9c700 1 --2- 172.21.15.31:0/2201367319 >> v2:172.21.15.62:3300/0 conn(0x7f7aa406aaa0 0x7f7aa415a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:18.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.110+0000 7f7aa2ffd700 1 -- 172.21.15.31:0/2201367319 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4165021453 0 0) 0x7f7a94001110 con 0x7f7aa413fb20 2023-10-06T02:28:18.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.110+0000 7f7aa2ffd700 1 -- 172.21.15.31:0/2201367319 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a8c003570 con 0x7f7aa413fb20 2023-10-06T02:28:18.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.111+0000 7f7aa8d9c700 1 -- 172.21.15.31:0/2201367319 >> v1:172.21.15.62:6789/0 conn(0x7f7aa413fef0 legacy=0x7f7aa40c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:18.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.111+0000 7f7aa2ffd700 1 -- 172.21.15.31:0/2201367319 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3566464702 0 0) 0x7f7a8c003570 con 0x7f7aa413fb20 2023-10-06T02:28:18.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.111+0000 7f7aa8d9c700 1 -- 172.21.15.31:0/2201367319 >> v1:172.21.15.31:6789/0 conn(0x7f7aa413fb20 legacy=0x7f7aa40c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:18.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.111+0000 7f7aa8d9c700 1 -- 172.21.15.31:0/2201367319 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aa406b520 con 0x7f7aa406aaa0 2023-10-06T02:28:18.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.111+0000 7f7aa8d9c700 1 --2- 172.21.15.31:0/2201367319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7aa406aaa0 0x7f7aa415a930 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c009e60 tx=0x7f7a9c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d1e176310ac75d9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:18.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.111+0000 7f7aa2ffd700 1 -- 172.21.15.31:0/2201367319 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c00e070 con 0x7f7aa406aaa0 2023-10-06T02:28:18.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.111+0000 7f7aa2ffd700 1 -- 172.21.15.31:0/2201367319 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a9c00b850 con 0x7f7aa406aaa0 2023-10-06T02:28:18.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.111+0000 7f7aa2ffd700 1 -- 172.21.15.31:0/2201367319 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a94001600 con 0x7f7aa406aaa0 2023-10-06T02:28:18.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.112+0000 7f7aa959d700 1 -- 172.21.15.31:0/2201367319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7aa406aaa0 msgr2=0x7f7aa415a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:18.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.112+0000 7f7aa959d700 1 --2- 172.21.15.31:0/2201367319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7aa406aaa0 0x7f7aa415a930 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c009e60 tx=0x7f7a9c009780 comp rx=0 tx=0).stop 2023-10-06T02:28:18.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.112+0000 7f7aa959d700 1 -- 172.21.15.31:0/2201367319 shutdown_connections 2023-10-06T02:28:18.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.112+0000 7f7aa959d700 1 --2- 172.21.15.31:0/2201367319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7aa406aaa0 0x7f7aa415a930 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:18.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.112+0000 7f7aa959d700 1 -- 172.21.15.31:0/2201367319 >> 172.21.15.31:0/2201367319 conn(0x7f7aa406d170 msgr2=0x7f7aa40cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:18.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.112+0000 7f7aa959d700 1 -- 172.21.15.31:0/2201367319 shutdown_connections 2023-10-06T02:28:18.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.112+0000 7f7aa959d700 1 -- 172.21.15.31:0/2201367319 wait complete. 2023-10-06T02:28:18.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.113+0000 7f7aa959d700 1 Processor -- start 2023-10-06T02:28:18.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.113+0000 7f7aa959d700 1 -- start start 2023-10-06T02:28:18.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.114+0000 7f7aa959d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7aa406aaa0 0x7f7aa40c2850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:18.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.114+0000 7f7aa959d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7aa413fb20 0x7f7aa40c7840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:18.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.114+0000 7f7aa959d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7aa413fef0 0x7f7aa40abf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:18.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.114+0000 7f7aa959d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa40c18d0 con 0x7f7aa413fef0 2023-10-06T02:28:18.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.114+0000 7f7aa959d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa40acc70 con 0x7f7aa413fb20 2023-10-06T02:28:18.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.114+0000 7f7aa959d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa40c1ab0 con 0x7f7aa406aaa0 2023-10-06T02:28:18.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.115+0000 7f7aa8d9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7aa413fef0 0x7f7aa40abf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:18.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.115+0000 7f7aa8d9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7aa413fef0 0x7f7aa40abf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:32914/0 (socket says 172.21.15.31:32914) 2023-10-06T02:28:18.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.115+0000 7f7aa8d9c700 1 -- 172.21.15.31:0/3466730385 learned_addr learned my addr 172.21.15.31:0/3466730385 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:18.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.115+0000 7f7aa3fff700 1 --2- 172.21.15.31:0/3466730385 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7aa406aaa0 0x7f7aa40c2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:18.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.115+0000 7f7aa37fe700 1 --2- 172.21.15.31:0/3466730385 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7aa413fb20 0x7f7aa40c7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:18.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.115+0000 7f7aa8d9c700 1 -- 172.21.15.31:0/3466730385 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7aa406aaa0 msgr2=0x7f7aa40c2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:18.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.115+0000 7f7aa8d9c700 1 --2- 172.21.15.31:0/3466730385 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7aa406aaa0 0x7f7aa40c2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:18.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.115+0000 7f7aa8d9c700 1 -- 172.21.15.31:0/3466730385 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7aa413fb20 msgr2=0x7f7aa40c7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:18.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.115+0000 7f7aa8d9c700 1 --2- 172.21.15.31:0/3466730385 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7aa413fb20 0x7f7aa40c7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:18.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.115+0000 7f7aa8d9c700 1 -- 172.21.15.31:0/3466730385 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a9c009a20 con 0x7f7aa413fef0 2023-10-06T02:28:18.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.115+0000 7f7aa8d9c700 1 --2- 172.21.15.31:0/3466730385 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7aa413fef0 0x7f7aa40abf60 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c003000 tx=0x7f7a9c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:18.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.115+0000 7f7a8bfff700 1 -- 172.21.15.31:0/3466730385 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c009420 con 0x7f7aa413fef0 2023-10-06T02:28:18.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.116+0000 7f7a8bfff700 1 -- 172.21.15.31:0/3466730385 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a9c00bec0 con 0x7f7aa413fef0 2023-10-06T02:28:18.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.116+0000 7f7a8bfff700 1 -- 172.21.15.31:0/3466730385 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c01f660 con 0x7f7aa413fef0 2023-10-06T02:28:18.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.116+0000 7f7aa959d700 1 -- 172.21.15.31:0/3466730385 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7aa40c1df0 con 0x7f7aa413fef0 2023-10-06T02:28:18.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.116+0000 7f7aa959d700 1 -- 172.21.15.31:0/3466730385 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7aa40c2120 con 0x7f7aa413fef0 2023-10-06T02:28:18.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.117+0000 7f7aa959d700 1 -- 172.21.15.31:0/3466730385 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7aa404c3c0 con 0x7f7aa413fef0 2023-10-06T02:28:18.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.118+0000 7f7a8bfff700 1 -- 172.21.15.31:0/3466730385 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f7a9c01fa50 con 0x7f7aa413fef0 2023-10-06T02:28:18.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.118+0000 7f7a8bfff700 1 --2- 172.21.15.31:0/3466730385 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7a90067050 0x7f7a90069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:18.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.118+0000 7f7a8bfff700 1 -- 172.21.15.31:0/3466730385 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f7a9c086a10 con 0x7f7aa413fef0 2023-10-06T02:28:18.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.119+0000 7f7aa3fff700 1 --2- 172.21.15.31:0/3466730385 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7a90067050 0x7f7a90069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:18.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.119+0000 7f7aa3fff700 1 --2- 172.21.15.31:0/3466730385 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7a90067050 0x7f7a90069500 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7aa413c570 tx=0x7f7a94000ed0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:18.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.123+0000 7f7a8bfff700 1 -- 172.21.15.31:0/3466730385 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7a9c0538b0 con 0x7f7aa413fef0 2023-10-06T02:28:18.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.371+0000 7f7aa959d700 1 -- 172.21.15.31:0/3466730385 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7aa406b800 con 0x7f7aa413fef0 2023-10-06T02:28:18.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.371+0000 7f7a8bfff700 1 -- 172.21.15.31:0/3466730385 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f7a9c023090 con 0x7f7aa413fef0 2023-10-06T02:28:18.373 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:28:18.373 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":62,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5759,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/2470516084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2470516084},{"type":"v1","addr":"172.21.15.62:6834","nonce":2470516084}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":6364,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/790751965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":790751965},{"type":"v1","addr":"172.21.15.31:6834","nonce":790751965}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":6571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3862349969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":3862349969},{"type":"v1","addr":"172.21.15.62:6835","nonce":3862349969}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:27:58.019137+0000","modified":"2023-10-06T02:28:01.555185+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6714},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6714":{"gid":6714,"name":"c","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/1002908930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":1002908930},{"type":"v1","addr":"172.21.15.31:6835","nonce":1002908930}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T02:28:18.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.373+0000 7f7aa959d700 1 -- 172.21.15.31:0/3466730385 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7a90067050 msgr2=0x7f7a90069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:18.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.373+0000 7f7aa959d700 1 --2- 172.21.15.31:0/3466730385 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7a90067050 0x7f7a90069500 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7aa413c570 tx=0x7f7a94000ed0 comp rx=0 tx=0).stop 2023-10-06T02:28:18.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.373+0000 7f7aa959d700 1 -- 172.21.15.31:0/3466730385 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7aa413fef0 msgr2=0x7f7aa40abf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:18.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.373+0000 7f7aa959d700 1 --2- 172.21.15.31:0/3466730385 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7aa413fef0 0x7f7aa40abf60 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c003000 tx=0x7f7a9c017000 comp rx=0 tx=0).stop 2023-10-06T02:28:18.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.373+0000 7f7aa959d700 1 -- 172.21.15.31:0/3466730385 shutdown_connections 2023-10-06T02:28:18.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.373+0000 7f7aa959d700 1 --2- 172.21.15.31:0/3466730385 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7aa413fef0 0x7f7aa40abf60 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:18.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.373+0000 7f7aa959d700 1 --2- 172.21.15.31:0/3466730385 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7a90067050 0x7f7a90069500 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:18.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.373+0000 7f7aa959d700 1 --2- 172.21.15.31:0/3466730385 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7aa406aaa0 0x7f7aa40c2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:18.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.373+0000 7f7aa959d700 1 --2- 172.21.15.31:0/3466730385 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7aa413fb20 0x7f7aa40c7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:18.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.373+0000 7f7aa959d700 1 -- 172.21.15.31:0/3466730385 >> 172.21.15.31:0/3466730385 conn(0x7f7aa406d170 msgr2=0x7f7aa413c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:18.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.373+0000 7f7aa959d700 1 -- 172.21.15.31:0/3466730385 shutdown_connections 2023-10-06T02:28:18.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.374+0000 7f7aa959d700 1 -- 172.21.15.31:0/3466730385 wait complete. 2023-10-06T02:28:18.376 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 62 2023-10-06T02:28:18.392 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:28:18.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.582+0000 7f789719d700 1 Processor -- start 2023-10-06T02:28:18.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.582+0000 7f789719d700 1 -- start start 2023-10-06T02:28:18.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.584+0000 7f789719d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7890159540 0x7f78900c1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:18.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.584+0000 7f789719d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7890068250 con 0x7f7890159910 2023-10-06T02:28:18.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.584+0000 7f789719d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7890068530 con 0x7f78900c6be0 2023-10-06T02:28:18.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.584+0000 7f789719d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7890068810 con 0x7f7890159540 2023-10-06T02:28:18.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.584+0000 7f789619b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7890159540 0x7f78900c1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:18.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.584+0000 7f789619b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7890159540 0x7f78900c1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:32926/0 (socket says 172.21.15.31:32926) 2023-10-06T02:28:18.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.584+0000 7f789599a700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f7890159910 0x7f78900677b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:43668/0 (socket says 172.21.15.31:43668) 2023-10-06T02:28:18.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.584+0000 7f789619b700 1 -- 172.21.15.31:0/290897528 learned_addr learned my addr 172.21.15.31:0/290897528 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:18.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.585+0000 7f7895199700 1 -- 172.21.15.31:0/290897528 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 45147873 0 0) 0x7f7890068250 con 0x7f7890159910 2023-10-06T02:28:18.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.585+0000 7f789619b700 1 -- 172.21.15.31:0/290897528 >> v1:172.21.15.62:6789/0 conn(0x7f78900c6be0 legacy=0x7f78900c6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:18.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.585+0000 7f789619b700 1 -- 172.21.15.31:0/290897528 >> v1:172.21.15.31:6789/0 conn(0x7f7890159910 legacy=0x7f78900677b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:18.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.585+0000 7f789619b700 1 -- 172.21.15.31:0/290897528 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78900a9ca0 con 0x7f7890159540 2023-10-06T02:28:18.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.585+0000 7f789619b700 1 --2- 172.21.15.31:0/290897528 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7890159540 0x7f78900c1340 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f788c0099b0 tx=0x7f788c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e65f2ba74af00d6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:18.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.585+0000 7f7895199700 1 -- 172.21.15.31:0/290897528 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f788c00b820 con 0x7f7890159540 2023-10-06T02:28:18.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.585+0000 7f7895199700 1 -- 172.21.15.31:0/290897528 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f788c00b980 con 0x7f7890159540 2023-10-06T02:28:18.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.585+0000 7f7895199700 1 -- 172.21.15.31:0/290897528 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f788c00bd20 con 0x7f7890159540 2023-10-06T02:28:18.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.586+0000 7f789719d700 1 -- 172.21.15.31:0/290897528 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7890159540 msgr2=0x7f78900c1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:18.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.586+0000 7f789719d700 1 --2- 172.21.15.31:0/290897528 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7890159540 0x7f78900c1340 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f788c0099b0 tx=0x7f788c009250 comp rx=0 tx=0).stop 2023-10-06T02:28:18.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.586+0000 7f789719d700 1 -- 172.21.15.31:0/290897528 shutdown_connections 2023-10-06T02:28:18.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.586+0000 7f789719d700 1 --2- 172.21.15.31:0/290897528 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7890159540 0x7f78900c1340 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:18.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.586+0000 7f789719d700 1 -- 172.21.15.31:0/290897528 >> 172.21.15.31:0/290897528 conn(0x7f789006f310 msgr2=0x7f78900cc5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:18.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.586+0000 7f789719d700 1 -- 172.21.15.31:0/290897528 shutdown_connections 2023-10-06T02:28:18.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.586+0000 7f789719d700 1 -- 172.21.15.31:0/290897528 wait complete. 2023-10-06T02:28:18.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.587+0000 7f789719d700 1 Processor -- start 2023-10-06T02:28:18.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.587+0000 7f789719d700 1 -- start start 2023-10-06T02:28:18.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.588+0000 7f789719d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f78900c6be0 0x7f789014e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:18.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.588+0000 7f789719d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7890159540 0x7f789015c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:18.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.588+0000 7f789719d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7890159910 0x7f78900b09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:18.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.588+0000 7f789719d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78900c3b60 con 0x7f7890159540 2023-10-06T02:28:18.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.588+0000 7f789719d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f789015a180 con 0x7f7890159910 2023-10-06T02:28:18.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.588+0000 7f789719d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78900afef0 con 0x7f78900c6be0 2023-10-06T02:28:18.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.588+0000 7f789599a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7890159540 0x7f789015c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:18.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.588+0000 7f789599a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7890159540 0x7f789015c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:32928/0 (socket says 172.21.15.31:32928) 2023-10-06T02:28:18.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.589+0000 7f789599a700 1 -- 172.21.15.31:0/1700171812 learned_addr learned my addr 172.21.15.31:0/1700171812 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:18.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.589+0000 7f789699c700 1 --2- 172.21.15.31:0/1700171812 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7890159910 0x7f78900b09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:18.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.589+0000 7f789619b700 1 --2- 172.21.15.31:0/1700171812 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f78900c6be0 0x7f789014e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:18.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.589+0000 7f789599a700 1 -- 172.21.15.31:0/1700171812 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f78900c6be0 msgr2=0x7f789014e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:18.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.589+0000 7f789599a700 1 --2- 172.21.15.31:0/1700171812 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f78900c6be0 0x7f789014e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:18.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.589+0000 7f789599a700 1 -- 172.21.15.31:0/1700171812 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7890159910 msgr2=0x7f78900b09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:18.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.589+0000 7f789599a700 1 --2- 172.21.15.31:0/1700171812 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7890159910 0x7f78900b09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:18.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.589+0000 7f789599a700 1 -- 172.21.15.31:0/1700171812 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f788c0094f0 con 0x7f7890159540 2023-10-06T02:28:18.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.589+0000 7f789599a700 1 --2- 172.21.15.31:0/1700171812 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7890159540 0x7f789015c750 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f788000c5b0 tx=0x7f7880001040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:18.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.589+0000 7f78867fc700 1 -- 172.21.15.31:0/1700171812 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7880005a70 con 0x7f7890159540 2023-10-06T02:28:18.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.589+0000 7f78867fc700 1 -- 172.21.15.31:0/1700171812 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7880005bd0 con 0x7f7890159540 2023-10-06T02:28:18.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.589+0000 7f789719d700 1 -- 172.21.15.31:0/1700171812 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f789015cc90 con 0x7f7890159540 2023-10-06T02:28:18.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.589+0000 7f78867fc700 1 -- 172.21.15.31:0/1700171812 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78800070d0 con 0x7f7890159540 2023-10-06T02:28:18.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.590+0000 7f789719d700 1 -- 172.21.15.31:0/1700171812 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78900b0470 con 0x7f7890159540 2023-10-06T02:28:18.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.591+0000 7f789719d700 1 -- 172.21.15.31:0/1700171812 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f789015a310 con 0x7f7890159540 2023-10-06T02:28:18.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.592+0000 7f78867fc700 1 -- 172.21.15.31:0/1700171812 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f7880007480 con 0x7f7890159540 2023-10-06T02:28:18.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.593+0000 7f78867fc700 1 --2- 172.21.15.31:0/1700171812 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7874067080 0x7f7874069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:18.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.593+0000 7f78867fc700 1 -- 172.21.15.31:0/1700171812 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f7880083dd0 con 0x7f7890159540 2023-10-06T02:28:18.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.593+0000 7f789619b700 1 --2- 172.21.15.31:0/1700171812 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7874067080 0x7f7874069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:18.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.594+0000 7f789619b700 1 --2- 172.21.15.31:0/1700171812 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7874067080 0x7f7874069530 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f788c0099b0 tx=0x7f788c016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:18.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.596+0000 7f78867fc700 1 -- 172.21.15.31:0/1700171812 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7880050d00 con 0x7f7890159540 2023-10-06T02:28:18.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.800+0000 7f789719d700 1 -- 172.21.15.31:0/1700171812 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f789004c3c0 con 0x7f7890159540 2023-10-06T02:28:18.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.801+0000 7f78867fc700 1 -- 172.21.15.31:0/1700171812 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+16950 (secure 0 0 0) 0x7f7880017020 con 0x7f7890159540 2023-10-06T02:28:18.803 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:28:18.803 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":95,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:28:17.469239+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:27:56.192975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-10-06T02:27:56.993140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6835/2757737386":"2023-10-07T02:28:14.456919+0000","172.21.15.62:6833/2757737386":"2023-10-07T02:28:14.456919+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:28:18.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.803+0000 7f789719d700 1 -- 172.21.15.31:0/1700171812 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7874067080 msgr2=0x7f7874069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:18.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.803+0000 7f789719d700 1 --2- 172.21.15.31:0/1700171812 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7874067080 0x7f7874069530 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f788c0099b0 tx=0x7f788c016000 comp rx=0 tx=0).stop 2023-10-06T02:28:18.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.803+0000 7f789719d700 1 -- 172.21.15.31:0/1700171812 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7890159540 msgr2=0x7f789015c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:18.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.803+0000 7f789719d700 1 --2- 172.21.15.31:0/1700171812 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7890159540 0x7f789015c750 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f788000c5b0 tx=0x7f7880001040 comp rx=0 tx=0).stop 2023-10-06T02:28:18.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.804+0000 7f789719d700 1 -- 172.21.15.31:0/1700171812 shutdown_connections 2023-10-06T02:28:18.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.804+0000 7f789719d700 1 --2- 172.21.15.31:0/1700171812 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7890159910 0x7f78900b09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:18.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.804+0000 7f789719d700 1 --2- 172.21.15.31:0/1700171812 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7874067080 0x7f7874069530 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:18.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.804+0000 7f789719d700 1 --2- 172.21.15.31:0/1700171812 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f78900c6be0 0x7f789014e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:18.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.804+0000 7f789719d700 1 --2- 172.21.15.31:0/1700171812 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7890159540 0x7f789015c750 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:18.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.804+0000 7f789719d700 1 -- 172.21.15.31:0/1700171812 >> 172.21.15.31:0/1700171812 conn(0x7f789006f310 msgr2=0x7f78900ca4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:18.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.804+0000 7f789719d700 1 -- 172.21.15.31:0/1700171812 shutdown_connections 2023-10-06T02:28:18.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:18.804+0000 7f789719d700 1 -- 172.21.15.31:0/1700171812 wait complete. 2023-10-06T02:28:18.823 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T02:22:53.183425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T02:23:00.046967+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T02:27:56.192975+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T02:27:56.993140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T02:28:18.823 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T02:28:19.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.012+0000 7f0dc939d700 1 Processor -- start 2023-10-06T02:28:19.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.012+0000 7f0dc939d700 1 -- start start 2023-10-06T02:28:19.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.013+0000 7f0dc939d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0dc40c7fb0 0x7f0dc406abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:19.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.013+0000 7f0dc939d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0dc413f780 0x7f0dc413fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:19.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.013+0000 7f0dc939d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dc4140120 con 0x7f0dc40c7be0 2023-10-06T02:28:19.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.013+0000 7f0dc939d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dc406bf10 con 0x7f0dc413f780 2023-10-06T02:28:19.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.013+0000 7f0dc939d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc406b290 con 0x7f0dc40c7fb0 2023-10-06T02:28:19.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.013+0000 7f0dc3fff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f0dc40c7be0 0x7f0dc415a990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:43682/0 (socket says 172.21.15.31:43682) 2023-10-06T02:28:19.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.013+0000 7f0dc3fff700 1 -- 172.21.15.31:0/1369126424 learned_addr learned my addr 172.21.15.31:0/1369126424 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:28:19.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.013+0000 7f0dc8b9c700 1 --2- 172.21.15.31:0/1369126424 >> v2:172.21.15.31:3300/0 conn(0x7f0dc413f780 0x7f0dc413fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:19.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.014+0000 7f0dc37fe700 1 --2- 172.21.15.31:0/1369126424 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0dc40c7fb0 0x7f0dc406abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:19.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.014+0000 7f0dc8b9c700 1 -- 172.21.15.31:0/1369126424 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0dc40c7fb0 msgr2=0x7f0dc406abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:19.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.014+0000 7f0dc2ffd700 1 -- 172.21.15.31:0/1369126424 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1840134178 0 0) 0x7f0db4001110 con 0x7f0dc40c7be0 2023-10-06T02:28:19.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.014+0000 7f0dc8b9c700 1 --2- 172.21.15.31:0/1369126424 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0dc40c7fb0 0x7f0dc406abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:19.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.014+0000 7f0dc8b9c700 1 -- 172.21.15.31:0/1369126424 >> v1:172.21.15.31:6789/0 conn(0x7f0dc40c7be0 legacy=0x7f0dc415a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:19.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.014+0000 7f0dc8b9c700 1 -- 172.21.15.31:0/1369126424 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dc406b570 con 0x7f0dc413f780 2023-10-06T02:28:19.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.014+0000 7f0dc8b9c700 1 --2- 172.21.15.31:0/1369126424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0dc413f780 0x7f0dc413fbe0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc009e60 tx=0x7f0dbc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e907a7dc474509d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:19.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.014+0000 7f0dc2ffd700 1 -- 172.21.15.31:0/1369126424 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dbc00e070 con 0x7f0dc413f780 2023-10-06T02:28:19.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.015+0000 7f0dc2ffd700 1 -- 172.21.15.31:0/1369126424 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dbc00bc30 con 0x7f0dc413f780 2023-10-06T02:28:19.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.015+0000 7f0dc2ffd700 1 -- 172.21.15.31:0/1369126424 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dbc012430 con 0x7f0dc413f780 2023-10-06T02:28:19.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.015+0000 7f0dc939d700 1 -- 172.21.15.31:0/1369126424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0dc413f780 msgr2=0x7f0dc413fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:19.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.015+0000 7f0dc939d700 1 --2- 172.21.15.31:0/1369126424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0dc413f780 0x7f0dc413fbe0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc009e60 tx=0x7f0dbc009780 comp rx=0 tx=0).stop 2023-10-06T02:28:19.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.015+0000 7f0dc939d700 1 -- 172.21.15.31:0/1369126424 shutdown_connections 2023-10-06T02:28:19.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.015+0000 7f0dc939d700 1 --2- 172.21.15.31:0/1369126424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0dc413f780 0x7f0dc413fbe0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:19.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.015+0000 7f0dc939d700 1 --2- 172.21.15.31:0/1369126424 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0dc40c7fb0 0x7f0dc406abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:19.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.015+0000 7f0dc939d700 1 -- 172.21.15.31:0/1369126424 >> 172.21.15.31:0/1369126424 conn(0x7f0dc406d110 msgr2=0x7f0dc40cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:19.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.015+0000 7f0dc939d700 1 -- 172.21.15.31:0/1369126424 shutdown_connections 2023-10-06T02:28:19.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.015+0000 7f0dc939d700 1 -- 172.21.15.31:0/1369126424 wait complete. 2023-10-06T02:28:19.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.017+0000 7f0dc939d700 1 Processor -- start 2023-10-06T02:28:19.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.017+0000 7f0dc939d700 1 -- start start 2023-10-06T02:28:19.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.018+0000 7f0dc939d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0dc40c7be0 0x7f0dc40b17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:19.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.018+0000 7f0dc939d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0dc40c7fb0 0x7f0dc416d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:19.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.018+0000 7f0dc939d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0dc413f780 0x7f0dc4158780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:19.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.018+0000 7f0dc939d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc416d6c0 con 0x7f0dc40c7fb0 2023-10-06T02:28:19.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.018+0000 7f0dc939d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc415a370 con 0x7f0dc40c7be0 2023-10-06T02:28:19.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.018+0000 7f0dc939d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc4156a80 con 0x7f0dc413f780 2023-10-06T02:28:19.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.018+0000 7f0dc37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0dc40c7fb0 0x7f0dc416d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:19.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.018+0000 7f0dc37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0dc40c7fb0 0x7f0dc416d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:32942/0 (socket says 172.21.15.31:32942) 2023-10-06T02:28:19.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.018+0000 7f0dc37fe700 1 -- 172.21.15.31:0/2097336364 learned_addr learned my addr 172.21.15.31:0/2097336364 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:19.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.018+0000 7f0dc3fff700 1 --2- 172.21.15.31:0/2097336364 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0dc40c7be0 0x7f0dc40b17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:19.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.018+0000 7f0dc8b9c700 1 --2- 172.21.15.31:0/2097336364 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0dc413f780 0x7f0dc4158780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:19.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.018+0000 7f0dc37fe700 1 -- 172.21.15.31:0/2097336364 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0dc413f780 msgr2=0x7f0dc4158780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:19.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.018+0000 7f0dc37fe700 1 --2- 172.21.15.31:0/2097336364 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0dc413f780 0x7f0dc4158780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:19.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.018+0000 7f0dc37fe700 1 -- 172.21.15.31:0/2097336364 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0dc40c7be0 msgr2=0x7f0dc40b17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:19.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.018+0000 7f0dc37fe700 1 --2- 172.21.15.31:0/2097336364 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0dc40c7be0 0x7f0dc40b17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:19.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.018+0000 7f0dc37fe700 1 -- 172.21.15.31:0/2097336364 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dbc009a20 con 0x7f0dc40c7fb0 2023-10-06T02:28:19.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.019+0000 7f0dc37fe700 1 --2- 172.21.15.31:0/2097336364 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0dc40c7fb0 0x7f0dc416d180 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f0db800d8c0 tx=0x7f0db8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:19.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.019+0000 7f0dabfff700 1 -- 172.21.15.31:0/2097336364 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db800de60 con 0x7f0dc40c7fb0 2023-10-06T02:28:19.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.019+0000 7f0dabfff700 1 -- 172.21.15.31:0/2097336364 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0db800b780 con 0x7f0dc40c7fb0 2023-10-06T02:28:19.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.019+0000 7f0dabfff700 1 -- 172.21.15.31:0/2097336364 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db800bb20 con 0x7f0dc40c7fb0 2023-10-06T02:28:19.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.019+0000 7f0dc939d700 1 -- 172.21.15.31:0/2097336364 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dc4156d60 con 0x7f0dc40c7fb0 2023-10-06T02:28:19.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.019+0000 7f0dc939d700 1 -- 172.21.15.31:0/2097336364 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dc4157320 con 0x7f0dc40c7fb0 2023-10-06T02:28:19.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.020+0000 7f0dc939d700 1 -- 172.21.15.31:0/2097336364 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dc404c3c0 con 0x7f0dc40c7fb0 2023-10-06T02:28:19.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.021+0000 7f0dabfff700 1 -- 172.21.15.31:0/2097336364 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f0db801e460 con 0x7f0dc40c7fb0 2023-10-06T02:28:19.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.022+0000 7f0dabfff700 1 --2- 172.21.15.31:0/2097336364 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0db0067050 0x7f0db0069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:19.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.022+0000 7f0dabfff700 1 -- 172.21.15.31:0/2097336364 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f0db8084ef0 con 0x7f0dc40c7fb0 2023-10-06T02:28:19.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.023+0000 7f0dc3fff700 1 --2- 172.21.15.31:0/2097336364 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0db0067050 0x7f0db0069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:19.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.023+0000 7f0dc3fff700 1 --2- 172.21.15.31:0/2097336364 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0db0067050 0x7f0db0069500 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0db40018a0 tx=0x7f0db4009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:19.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.026+0000 7f0dabfff700 1 -- 172.21.15.31:0/2097336364 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0db8051ea0 con 0x7f0dc40c7fb0 2023-10-06T02:28:19.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.227+0000 7f0dc939d700 1 -- 172.21.15.31:0/2097336364 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0dc406b850 con 0x7f0dc40c7fb0 2023-10-06T02:28:19.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.228+0000 7f0dabfff700 1 -- 172.21.15.31:0/2097336364 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0db80554c0 con 0x7f0dc40c7fb0 2023-10-06T02:28:19.229 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-10-06T02:28:19.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.230+0000 7f0dc939d700 1 -- 172.21.15.31:0/2097336364 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0db0067050 msgr2=0x7f0db0069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:19.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.230+0000 7f0dc939d700 1 --2- 172.21.15.31:0/2097336364 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0db0067050 0x7f0db0069500 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0db40018a0 tx=0x7f0db4009040 comp rx=0 tx=0).stop 2023-10-06T02:28:19.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.230+0000 7f0dc939d700 1 -- 172.21.15.31:0/2097336364 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0dc40c7fb0 msgr2=0x7f0dc416d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:19.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.230+0000 7f0dc939d700 1 --2- 172.21.15.31:0/2097336364 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0dc40c7fb0 0x7f0dc416d180 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f0db800d8c0 tx=0x7f0db8008040 comp rx=0 tx=0).stop 2023-10-06T02:28:19.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.230+0000 7f0dc939d700 1 -- 172.21.15.31:0/2097336364 shutdown_connections 2023-10-06T02:28:19.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.230+0000 7f0dc939d700 1 --2- 172.21.15.31:0/2097336364 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0db0067050 0x7f0db0069500 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:19.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.230+0000 7f0dc939d700 1 --2- 172.21.15.31:0/2097336364 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0dc413f780 0x7f0dc4158780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:19.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.230+0000 7f0dc939d700 1 --2- 172.21.15.31:0/2097336364 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0dc40c7be0 0x7f0dc40b17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:19.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.230+0000 7f0dc939d700 1 --2- 172.21.15.31:0/2097336364 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0dc40c7fb0 0x7f0dc416d180 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:19.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.230+0000 7f0dc939d700 1 -- 172.21.15.31:0/2097336364 >> 172.21.15.31:0/2097336364 conn(0x7f0dc406d110 msgr2=0x7f0dc406de50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:19.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.231+0000 7f0dc939d700 1 -- 172.21.15.31:0/2097336364 shutdown_connections 2023-10-06T02:28:19.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.231+0000 7f0dc939d700 1 -- 172.21.15.31:0/2097336364 wait complete. 2023-10-06T02:28:19.249 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T02:28:19.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.439+0000 7fda65f9d700 1 Processor -- start 2023-10-06T02:28:19.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.439+0000 7fda65f9d700 1 -- start start 2023-10-06T02:28:19.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.440+0000 7fda65f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fda60159550 0x7fda600ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:19.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.440+0000 7fda65f9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda600acc00 con 0x7fda60159920 2023-10-06T02:28:19.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.440+0000 7fda65f9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda600abff0 con 0x7fda6015dfd0 2023-10-06T02:28:19.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.440+0000 7fda65f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda600ac2d0 con 0x7fda60159550 2023-10-06T02:28:19.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.440+0000 7fda6579c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fda60159920 0x7fda600ad090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:43692/0 (socket says 172.21.15.31:43692) 2023-10-06T02:28:19.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.440+0000 7fda6579c700 1 -- 172.21.15.31:0/1559811952 learned_addr learned my addr 172.21.15.31:0/1559811952 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:28:19.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.441+0000 7fda64f9b700 1 --2- 172.21.15.31:0/1559811952 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fda60159550 0x7fda600ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:19.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.441+0000 7fda5f7fe700 1 -- 172.21.15.31:0/1559811952 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 521828462 0 0) 0x7fda600acc00 con 0x7fda60159920 2023-10-06T02:28:19.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.441+0000 7fda5f7fe700 1 -- 172.21.15.31:0/1559811952 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda50003570 con 0x7fda60159920 2023-10-06T02:28:19.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.441+0000 7fda5f7fe700 1 -- 172.21.15.31:0/1559811952 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3673602900 0 0) 0x7fda600abff0 con 0x7fda6015dfd0 2023-10-06T02:28:19.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.441+0000 7fda5f7fe700 1 -- 172.21.15.31:0/1559811952 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda600acc00 con 0x7fda6015dfd0 2023-10-06T02:28:19.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.441+0000 7fda64f9b700 1 -- 172.21.15.31:0/1559811952 >> v1:172.21.15.62:6789/0 conn(0x7fda6015dfd0 legacy=0x7fda600aedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:19.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.441+0000 7fda64f9b700 1 -- 172.21.15.31:0/1559811952 >> v1:172.21.15.31:6789/0 conn(0x7fda60159920 legacy=0x7fda600ad090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:19.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.441+0000 7fda5f7fe700 1 -- 172.21.15.31:0/1559811952 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 257084127 0 0) 0x7fda50003570 con 0x7fda60159920 2023-10-06T02:28:19.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.441+0000 7fda64f9b700 1 -- 172.21.15.31:0/1559811952 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda600ac5b0 con 0x7fda60159550 2023-10-06T02:28:19.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.442+0000 7fda64f9b700 1 --2- 172.21.15.31:0/1559811952 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fda60159550 0x7fda600ceac0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fda54009910 tx=0x7fda540091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7110c7e4452571c6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:19.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.442+0000 7fda5f7fe700 1 -- 172.21.15.31:0/1559811952 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda580017f0 con 0x7fda60159550 2023-10-06T02:28:19.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.442+0000 7fda5f7fe700 1 -- 172.21.15.31:0/1559811952 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda54011420 con 0x7fda60159550 2023-10-06T02:28:19.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.442+0000 7fda5f7fe700 1 -- 172.21.15.31:0/1559811952 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda540117c0 con 0x7fda60159550 2023-10-06T02:28:19.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.442+0000 7fda65f9d700 1 -- 172.21.15.31:0/1559811952 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fda60159550 msgr2=0x7fda600ceac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:19.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.442+0000 7fda65f9d700 1 --2- 172.21.15.31:0/1559811952 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fda60159550 0x7fda600ceac0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fda54009910 tx=0x7fda540091a0 comp rx=0 tx=0).stop 2023-10-06T02:28:19.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.443+0000 7fda65f9d700 1 -- 172.21.15.31:0/1559811952 shutdown_connections 2023-10-06T02:28:19.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.443+0000 7fda65f9d700 1 --2- 172.21.15.31:0/1559811952 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fda60159550 0x7fda600ceac0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:19.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.443+0000 7fda65f9d700 1 -- 172.21.15.31:0/1559811952 >> 172.21.15.31:0/1559811952 conn(0x7fda6006ce80 msgr2=0x7fda600ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:19.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.443+0000 7fda65f9d700 1 -- 172.21.15.31:0/1559811952 shutdown_connections 2023-10-06T02:28:19.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.443+0000 7fda65f9d700 1 -- 172.21.15.31:0/1559811952 wait complete. 2023-10-06T02:28:19.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.444+0000 7fda65f9d700 1 Processor -- start 2023-10-06T02:28:19.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.444+0000 7fda65f9d700 1 -- start start 2023-10-06T02:28:19.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.445+0000 7fda65f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fda60159550 0x7fda6014ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:19.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.445+0000 7fda65f9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda60159920 0x7fda600b13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:19.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.445+0000 7fda65f9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda6015dfd0 0x7fda6014c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:19.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.445+0000 7fda65f9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda600cbd80 con 0x7fda6015dfd0 2023-10-06T02:28:19.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.445+0000 7fda65f9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda600cc020 con 0x7fda60159920 2023-10-06T02:28:19.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.445+0000 7fda65f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda600cc300 con 0x7fda60159550 2023-10-06T02:28:19.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.445+0000 7fda6579c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda6015dfd0 0x7fda6014c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:19.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.445+0000 7fda6579c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda6015dfd0 0x7fda6014c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:32956/0 (socket says 172.21.15.31:32956) 2023-10-06T02:28:19.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.445+0000 7fda6579c700 1 -- 172.21.15.31:0/2144413792 learned_addr learned my addr 172.21.15.31:0/2144413792 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:19.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.445+0000 7fda64f9b700 1 --2- 172.21.15.31:0/2144413792 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fda60159550 0x7fda6014ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:19.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.445+0000 7fda5ffff700 1 --2- 172.21.15.31:0/2144413792 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda60159920 0x7fda600b13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:19.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.446+0000 7fda6579c700 1 -- 172.21.15.31:0/2144413792 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fda60159550 msgr2=0x7fda6014ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:19.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.446+0000 7fda6579c700 1 --2- 172.21.15.31:0/2144413792 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fda60159550 0x7fda6014ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:19.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.446+0000 7fda6579c700 1 -- 172.21.15.31:0/2144413792 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda60159920 msgr2=0x7fda600b13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:19.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.446+0000 7fda6579c700 1 --2- 172.21.15.31:0/2144413792 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda60159920 0x7fda600b13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:19.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.446+0000 7fda6579c700 1 -- 172.21.15.31:0/2144413792 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda54009420 con 0x7fda6015dfd0 2023-10-06T02:28:19.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.446+0000 7fda6579c700 1 --2- 172.21.15.31:0/2144413792 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda6015dfd0 0x7fda6014c470 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fda5800cc90 tx=0x7fda58007d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:19.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.446+0000 7fda5cff9700 1 -- 172.21.15.31:0/2144413792 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda58004e90 con 0x7fda6015dfd0 2023-10-06T02:28:19.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.446+0000 7fda5cff9700 1 -- 172.21.15.31:0/2144413792 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda5800a640 con 0x7fda6015dfd0 2023-10-06T02:28:19.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.446+0000 7fda5cff9700 1 -- 172.21.15.31:0/2144413792 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda5800aa00 con 0x7fda6015dfd0 2023-10-06T02:28:19.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.446+0000 7fda65f9d700 1 -- 172.21.15.31:0/2144413792 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda600cc5e0 con 0x7fda6015dfd0 2023-10-06T02:28:19.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.446+0000 7fda65f9d700 1 -- 172.21.15.31:0/2144413792 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda600ccba0 con 0x7fda6015dfd0 2023-10-06T02:28:19.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.448+0000 7fda65f9d700 1 -- 172.21.15.31:0/2144413792 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda600cabe0 con 0x7fda6015dfd0 2023-10-06T02:28:19.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.449+0000 7fda5cff9700 1 -- 172.21.15.31:0/2144413792 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7fda58007150 con 0x7fda6015dfd0 2023-10-06T02:28:19.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.449+0000 7fda5cff9700 1 --2- 172.21.15.31:0/2144413792 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fda44067050 0x7fda44069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:19.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.449+0000 7fda5cff9700 1 -- 172.21.15.31:0/2144413792 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6144+0+0 (secure 0 0 0) 0x7fda58084270 con 0x7fda6015dfd0 2023-10-06T02:28:19.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.450+0000 7fda64f9b700 1 --2- 172.21.15.31:0/2144413792 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fda44067050 0x7fda44069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:19.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.450+0000 7fda64f9b700 1 --2- 172.21.15.31:0/2144413792 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fda44067050 0x7fda44069500 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fda540030c0 tx=0x7fda54016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:19.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.453+0000 7fda5cff9700 1 -- 172.21.15.31:0/2144413792 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fda580512a0 con 0x7fda6015dfd0 2023-10-06T02:28:19.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.658+0000 7fda65f9d700 1 -- 172.21.15.31:0/2144413792 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fda6004c3c0 con 0x7fda6015dfd0 2023-10-06T02:28:19.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.658+0000 7fda5cff9700 1 -- 172.21.15.31:0/2144413792 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v95) v1 ==== 92+0+10 (secure 0 0 0) 0x7fda58018020 con 0x7fda6015dfd0 2023-10-06T02:28:19.660 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 8 2023-10-06T02:28:19.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.660+0000 7fda65f9d700 1 -- 172.21.15.31:0/2144413792 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fda44067050 msgr2=0x7fda44069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:19.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.660+0000 7fda65f9d700 1 --2- 172.21.15.31:0/2144413792 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fda44067050 0x7fda44069500 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fda540030c0 tx=0x7fda54016000 comp rx=0 tx=0).stop 2023-10-06T02:28:19.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.660+0000 7fda65f9d700 1 -- 172.21.15.31:0/2144413792 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda6015dfd0 msgr2=0x7fda6014c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:19.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.660+0000 7fda65f9d700 1 --2- 172.21.15.31:0/2144413792 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda6015dfd0 0x7fda6014c470 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fda5800cc90 tx=0x7fda58007d20 comp rx=0 tx=0).stop 2023-10-06T02:28:19.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.660+0000 7fda65f9d700 1 -- 172.21.15.31:0/2144413792 shutdown_connections 2023-10-06T02:28:19.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.660+0000 7fda65f9d700 1 --2- 172.21.15.31:0/2144413792 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda6015dfd0 0x7fda6014c470 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:19.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.660+0000 7fda65f9d700 1 --2- 172.21.15.31:0/2144413792 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fda44067050 0x7fda44069500 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:19.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.660+0000 7fda65f9d700 1 --2- 172.21.15.31:0/2144413792 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fda60159550 0x7fda6014ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:19.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.660+0000 7fda65f9d700 1 --2- 172.21.15.31:0/2144413792 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda60159920 0x7fda600b13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:19.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.661+0000 7fda65f9d700 1 -- 172.21.15.31:0/2144413792 >> 172.21.15.31:0/2144413792 conn(0x7fda6006ce80 msgr2=0x7fda600aedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:19.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.661+0000 7fda65f9d700 1 -- 172.21.15.31:0/2144413792 shutdown_connections 2023-10-06T02:28:19.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.661+0000 7fda65f9d700 1 -- 172.21.15.31:0/2144413792 wait complete. 2023-10-06T02:28:19.680 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T02:28:19.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.868+0000 7fe517fff700 1 Processor -- start 2023-10-06T02:28:19.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.868+0000 7fe517fff700 1 -- start start 2023-10-06T02:28:19.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.870+0000 7fe517fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe518155890 0x7fe5180c7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:19.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.870+0000 7fe517fff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe518155c60 0x7fe51806a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:19.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.870+0000 7fe517fff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5180c7e70 con 0x7fe51806de50 2023-10-06T02:28:19.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.870+0000 7fe517fff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe51806b790 con 0x7fe518155c60 2023-10-06T02:28:19.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.870+0000 7fe517fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe51806a8e0 con 0x7fe518155890 2023-10-06T02:28:19.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.870+0000 7fe516ffd700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe518155890 0x7fe5180c7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:19.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.870+0000 7fe516ffd700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe518155890 0x7fe5180c7930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:55982/0 (socket says 172.21.15.31:55982) 2023-10-06T02:28:19.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.870+0000 7fe516ffd700 1 -- 172.21.15.31:0/254291421 learned_addr learned my addr 172.21.15.31:0/254291421 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:19.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.870+0000 7fe5177fe700 1 --1- 172.21.15.31:0/254291421 >> v1:172.21.15.62:6789/0 conn(0x7fe51806de50 0x7fe518159720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:42594/0 (socket says 172.21.15.31:42594) 2023-10-06T02:28:19.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.870+0000 7fe5167fc700 1 --2- 172.21.15.31:0/254291421 >> v2:172.21.15.62:3300/0 conn(0x7fe518155c60 0x7fe51806a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:19.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.871+0000 7fe515ffb700 1 -- 172.21.15.31:0/254291421 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3432438684 0 0) 0x7fe5180c7e70 con 0x7fe51806de50 2023-10-06T02:28:19.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.871+0000 7fe515ffb700 1 -- 172.21.15.31:0/254291421 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe500003570 con 0x7fe51806de50 2023-10-06T02:28:19.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.871+0000 7fe5167fc700 1 -- 172.21.15.31:0/254291421 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe518155890 msgr2=0x7fe5180c7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:19.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.871+0000 7fe5167fc700 1 --2- 172.21.15.31:0/254291421 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe518155890 0x7fe5180c7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:19.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.871+0000 7fe5167fc700 1 -- 172.21.15.31:0/254291421 >> v1:172.21.15.62:6789/0 conn(0x7fe51806de50 legacy=0x7fe518159720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:19.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.871+0000 7fe5167fc700 1 -- 172.21.15.31:0/254291421 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe51806ab80 con 0x7fe518155c60 2023-10-06T02:28:19.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.871+0000 7fe516ffd700 1 --2- 172.21.15.31:0/254291421 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe518155890 0x7fe5180c7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:28:19.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.871+0000 7fe5167fc700 1 --2- 172.21.15.31:0/254291421 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe518155c60 0x7fe51806a3a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fe50c0098e0 tx=0x7fe50c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e81557c28217a5eb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:19.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.872+0000 7fe515ffb700 1 -- 172.21.15.31:0/254291421 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe50c010600 con 0x7fe518155c60 2023-10-06T02:28:19.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.872+0000 7fe515ffb700 1 -- 172.21.15.31:0/254291421 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe50c010760 con 0x7fe518155c60 2023-10-06T02:28:19.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.872+0000 7fe515ffb700 1 -- 172.21.15.31:0/254291421 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe50c010b00 con 0x7fe518155c60 2023-10-06T02:28:19.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.872+0000 7fe517fff700 1 -- 172.21.15.31:0/254291421 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe518155c60 msgr2=0x7fe51806a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:19.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.872+0000 7fe517fff700 1 --2- 172.21.15.31:0/254291421 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe518155c60 0x7fe51806a3a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fe50c0098e0 tx=0x7fe50c0091a0 comp rx=0 tx=0).stop 2023-10-06T02:28:19.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.872+0000 7fe517fff700 1 -- 172.21.15.31:0/254291421 shutdown_connections 2023-10-06T02:28:19.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.872+0000 7fe517fff700 1 --2- 172.21.15.31:0/254291421 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe518155890 0x7fe5180c7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:19.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.872+0000 7fe517fff700 1 --2- 172.21.15.31:0/254291421 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe518155c60 0x7fe51806a3a0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:19.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.872+0000 7fe517fff700 1 -- 172.21.15.31:0/254291421 >> 172.21.15.31:0/254291421 conn(0x7fe51806d110 msgr2=0x7fe5180cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:19.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.872+0000 7fe517fff700 1 -- 172.21.15.31:0/254291421 shutdown_connections 2023-10-06T02:28:19.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.872+0000 7fe517fff700 1 -- 172.21.15.31:0/254291421 wait complete. 2023-10-06T02:28:19.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.874+0000 7fe517fff700 1 Processor -- start 2023-10-06T02:28:19.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.874+0000 7fe517fff700 1 -- start start 2023-10-06T02:28:19.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.875+0000 7fe517fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe51806de50 0x7fe518144300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:19.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.875+0000 7fe517fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe518155890 0x7fe5180c4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:19.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.875+0000 7fe517fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe518155c60 0x7fe5180af760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:19.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.875+0000 7fe517fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5180c5cc0 con 0x7fe518155c60 2023-10-06T02:28:19.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.875+0000 7fe517fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5180c6ab0 con 0x7fe51806de50 2023-10-06T02:28:19.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.875+0000 7fe517fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe518144840 con 0x7fe518155890 2023-10-06T02:28:19.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.875+0000 7fe5177fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe518155c60 0x7fe5180af760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:19.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.875+0000 7fe5177fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe518155c60 0x7fe5180af760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:32970/0 (socket says 172.21.15.31:32970) 2023-10-06T02:28:19.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.875+0000 7fe5177fe700 1 -- 172.21.15.31:0/1159822893 learned_addr learned my addr 172.21.15.31:0/1159822893 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:19.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.875+0000 7fe516ffd700 1 --2- 172.21.15.31:0/1159822893 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe51806de50 0x7fe518144300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:19.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.875+0000 7fe5167fc700 1 --2- 172.21.15.31:0/1159822893 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe518155890 0x7fe5180c4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:19.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.876+0000 7fe5177fe700 1 -- 172.21.15.31:0/1159822893 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe518155890 msgr2=0x7fe5180c4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:19.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.876+0000 7fe5177fe700 1 --2- 172.21.15.31:0/1159822893 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe518155890 0x7fe5180c4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:19.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.876+0000 7fe5177fe700 1 -- 172.21.15.31:0/1159822893 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe51806de50 msgr2=0x7fe518144300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:19.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.876+0000 7fe5177fe700 1 --2- 172.21.15.31:0/1159822893 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe51806de50 0x7fe518144300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:19.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.876+0000 7fe5177fe700 1 -- 172.21.15.31:0/1159822893 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5080094f0 con 0x7fe518155c60 2023-10-06T02:28:19.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.876+0000 7fe5167fc700 1 --2- 172.21.15.31:0/1159822893 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe518155890 0x7fe5180c4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:28:19.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.876+0000 7fe5177fe700 1 --2- 172.21.15.31:0/1159822893 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe518155c60 0x7fe5180af760 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fe51000cd30 tx=0x7fe510001b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:19.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.876+0000 7fe4feffd700 1 -- 172.21.15.31:0/1159822893 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe51000ce90 con 0x7fe518155c60 2023-10-06T02:28:19.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.876+0000 7fe4feffd700 1 -- 172.21.15.31:0/1159822893 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe510005df0 con 0x7fe518155c60 2023-10-06T02:28:19.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.876+0000 7fe4feffd700 1 -- 172.21.15.31:0/1159822893 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe51000ab70 con 0x7fe518155c60 2023-10-06T02:28:19.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.876+0000 7fe517fff700 1 -- 172.21.15.31:0/1159822893 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe50c009420 con 0x7fe518155c60 2023-10-06T02:28:19.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.876+0000 7fe517fff700 1 -- 172.21.15.31:0/1159822893 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5180c6180 con 0x7fe518155c60 2023-10-06T02:28:19.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.877+0000 7fe517fff700 1 -- 172.21.15.31:0/1159822893 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5180c6c40 con 0x7fe518155c60 2023-10-06T02:28:19.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.878+0000 7fe4feffd700 1 -- 172.21.15.31:0/1159822893 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7fe5100070f0 con 0x7fe518155c60 2023-10-06T02:28:19.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.879+0000 7fe4feffd700 1 --2- 172.21.15.31:0/1159822893 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe504067050 0x7fe504069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:19.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.879+0000 7fe4feffd700 1 -- 172.21.15.31:0/1159822893 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6144+0+0 (secure 0 0 0) 0x7fe510084a40 con 0x7fe518155c60 2023-10-06T02:28:19.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.880+0000 7fe516ffd700 1 --2- 172.21.15.31:0/1159822893 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe504067050 0x7fe504069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:19.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.880+0000 7fe516ffd700 1 --2- 172.21.15.31:0/1159822893 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe504067050 0x7fe504069500 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe5180c7f50 tx=0x7fe508009250 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:19.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:19.883+0000 7fe4feffd700 1 -- 172.21.15.31:0/1159822893 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe5100519f0 con 0x7fe518155c60 2023-10-06T02:28:20.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.083+0000 7fe517fff700 1 -- 172.21.15.31:0/1159822893 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe51806f170 con 0x7fe518155c60 2023-10-06T02:28:20.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.083+0000 7fe4feffd700 1 -- 172.21.15.31:0/1159822893 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe510055010 con 0x7fe518155c60 2023-10-06T02:28:20.085 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-10-06T02:28:20.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.085+0000 7fe517fff700 1 -- 172.21.15.31:0/1159822893 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe504067050 msgr2=0x7fe504069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:20.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.085+0000 7fe517fff700 1 --2- 172.21.15.31:0/1159822893 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe504067050 0x7fe504069500 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe5180c7f50 tx=0x7fe508009250 comp rx=0 tx=0).stop 2023-10-06T02:28:20.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.085+0000 7fe517fff700 1 -- 172.21.15.31:0/1159822893 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe518155c60 msgr2=0x7fe5180af760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:20.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.085+0000 7fe517fff700 1 --2- 172.21.15.31:0/1159822893 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe518155c60 0x7fe5180af760 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fe51000cd30 tx=0x7fe510001b60 comp rx=0 tx=0).stop 2023-10-06T02:28:20.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.085+0000 7fe517fff700 1 -- 172.21.15.31:0/1159822893 shutdown_connections 2023-10-06T02:28:20.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.085+0000 7fe517fff700 1 --2- 172.21.15.31:0/1159822893 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe518155c60 0x7fe5180af760 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:20.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.085+0000 7fe517fff700 1 --2- 172.21.15.31:0/1159822893 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe51806de50 0x7fe518144300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:20.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.085+0000 7fe517fff700 1 --2- 172.21.15.31:0/1159822893 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe504067050 0x7fe504069500 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:20.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.085+0000 7fe517fff700 1 --2- 172.21.15.31:0/1159822893 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe518155890 0x7fe5180c4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:20.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.085+0000 7fe517fff700 1 -- 172.21.15.31:0/1159822893 >> 172.21.15.31:0/1159822893 conn(0x7fe51806d110 msgr2=0x7fe5180cc680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:20.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.085+0000 7fe517fff700 1 -- 172.21.15.31:0/1159822893 shutdown_connections 2023-10-06T02:28:20.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.086+0000 7fe517fff700 1 -- 172.21.15.31:0/1159822893 wait complete. 2023-10-06T02:28:20.104 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T02:28:20.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.292+0000 7fd83c9fe700 1 Processor -- start 2023-10-06T02:28:20.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.292+0000 7fd83c9fe700 1 -- start start 2023-10-06T02:28:20.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.293+0000 7fd83c9fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd83806a210 0x7fd8380c6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:20.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.293+0000 7fd83c9fe700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd83806de50 0x7fd8380c7190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:20.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.293+0000 7fd83c9fe700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd83815b640 con 0x7fd838140a70 2023-10-06T02:28:20.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.293+0000 7fd83c9fe700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8380c6800 con 0x7fd83806a210 2023-10-06T02:28:20.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.293+0000 7fd83c9fe700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd83815b920 con 0x7fd83806de50 2023-10-06T02:28:20.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.294+0000 7fd836d9d700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fd838140a70 0x7fd838069b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:43706/0 (socket says 172.21.15.31:43706) 2023-10-06T02:28:20.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.294+0000 7fd836d9d700 1 -- 172.21.15.31:0/2594489227 learned_addr learned my addr 172.21.15.31:0/2594489227 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:28:20.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.294+0000 7fd83659c700 1 --2- 172.21.15.31:0/2594489227 >> v2:172.21.15.31:3300/0 conn(0x7fd83806a210 0x7fd8380c6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:20.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.294+0000 7fd835d9b700 1 -- 172.21.15.31:0/2594489227 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 414811269 0 0) 0x7fd83815b640 con 0x7fd838140a70 2023-10-06T02:28:20.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.294+0000 7fd835d9b700 1 -- 172.21.15.31:0/2594489227 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd820003570 con 0x7fd838140a70 2023-10-06T02:28:20.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.294+0000 7fd83759e700 1 --2- 172.21.15.31:0/2594489227 >> v2:172.21.15.62:3300/0 conn(0x7fd83806de50 0x7fd8380c7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:20.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.294+0000 7fd83659c700 1 -- 172.21.15.31:0/2594489227 >> v2:172.21.15.62:3300/0 conn(0x7fd83806de50 msgr2=0x7fd8380c7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:20.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.294+0000 7fd83659c700 1 --2- 172.21.15.31:0/2594489227 >> v2:172.21.15.62:3300/0 conn(0x7fd83806de50 0x7fd8380c7190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:20.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.294+0000 7fd83659c700 1 -- 172.21.15.31:0/2594489227 >> v1:172.21.15.31:6789/0 conn(0x7fd838140a70 legacy=0x7fd838069b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:20.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.294+0000 7fd83659c700 1 -- 172.21.15.31:0/2594489227 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd83815bc00 con 0x7fd83806a210 2023-10-06T02:28:20.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.295+0000 7fd83659c700 1 --2- 172.21.15.31:0/2594489227 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd83806a210 0x7fd8380c6c50 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fd8380c6770 tx=0x7fd82c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78ea2300c3996590 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:20.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.295+0000 7fd835d9b700 1 -- 172.21.15.31:0/2594489227 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82c011630 con 0x7fd83806a210 2023-10-06T02:28:20.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.295+0000 7fd835d9b700 1 -- 172.21.15.31:0/2594489227 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd82c011790 con 0x7fd83806a210 2023-10-06T02:28:20.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.295+0000 7fd835d9b700 1 -- 172.21.15.31:0/2594489227 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82c011b30 con 0x7fd83806a210 2023-10-06T02:28:20.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.295+0000 7fd83c9fe700 1 -- 172.21.15.31:0/2594489227 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd83806a210 msgr2=0x7fd8380c6c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:20.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.295+0000 7fd83c9fe700 1 --2- 172.21.15.31:0/2594489227 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd83806a210 0x7fd8380c6c50 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fd8380c6770 tx=0x7fd82c0091a0 comp rx=0 tx=0).stop 2023-10-06T02:28:20.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.295+0000 7fd83c9fe700 1 -- 172.21.15.31:0/2594489227 shutdown_connections 2023-10-06T02:28:20.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.295+0000 7fd83c9fe700 1 --2- 172.21.15.31:0/2594489227 >> v2:172.21.15.62:3300/0 conn(0x7fd83806de50 0x7fd8380c7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:20.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.295+0000 7fd83c9fe700 1 --2- 172.21.15.31:0/2594489227 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd83806a210 0x7fd8380c6c50 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:20.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.295+0000 7fd83c9fe700 1 -- 172.21.15.31:0/2594489227 >> 172.21.15.31:0/2594489227 conn(0x7fd83806d110 msgr2=0x7fd8380cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:20.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.296+0000 7fd83c9fe700 1 -- 172.21.15.31:0/2594489227 shutdown_connections 2023-10-06T02:28:20.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.296+0000 7fd83c9fe700 1 -- 172.21.15.31:0/2594489227 wait complete. 2023-10-06T02:28:20.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.297+0000 7fd83c9fe700 1 Processor -- start 2023-10-06T02:28:20.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.297+0000 7fd83c9fe700 1 -- start start 2023-10-06T02:28:20.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.298+0000 7fd83c9fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd83806a210 0x7fd8380ae370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:20.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.298+0000 7fd83c9fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd83806de50 0x7fd8380c4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:20.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.298+0000 7fd83c9fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd838140a70 0x7fd83813eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:20.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.298+0000 7fd83c9fe700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8380c3ab0 con 0x7fd83806de50 2023-10-06T02:28:20.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.298+0000 7fd83c9fe700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8380c4630 con 0x7fd83806a210 2023-10-06T02:28:20.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.298+0000 7fd83c9fe700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8380c3d00 con 0x7fd838140a70 2023-10-06T02:28:20.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.298+0000 7fd83659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd83806de50 0x7fd8380c4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:20.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.298+0000 7fd83659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd83806de50 0x7fd8380c4ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:32992/0 (socket says 172.21.15.31:32992) 2023-10-06T02:28:20.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.298+0000 7fd83659c700 1 -- 172.21.15.31:0/20702713 learned_addr learned my addr 172.21.15.31:0/20702713 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:20.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.298+0000 7fd836d9d700 1 --2- 172.21.15.31:0/20702713 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd83806a210 0x7fd8380ae370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:20.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.299+0000 7fd83759e700 1 --2- 172.21.15.31:0/20702713 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd838140a70 0x7fd83813eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:20.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.299+0000 7fd83659c700 1 -- 172.21.15.31:0/20702713 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd838140a70 msgr2=0x7fd83813eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:20.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.299+0000 7fd83659c700 1 --2- 172.21.15.31:0/20702713 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd838140a70 0x7fd83813eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:20.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.299+0000 7fd83659c700 1 -- 172.21.15.31:0/20702713 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd83806a210 msgr2=0x7fd8380ae370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:20.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.299+0000 7fd83659c700 1 --2- 172.21.15.31:0/20702713 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd83806a210 0x7fd8380ae370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:20.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.299+0000 7fd83659c700 1 -- 172.21.15.31:0/20702713 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd82c009420 con 0x7fd83806de50 2023-10-06T02:28:20.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.299+0000 7fd83659c700 1 --2- 172.21.15.31:0/20702713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd83806de50 0x7fd8380c4ac0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fd8380cc000 tx=0x7fd82c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:20.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.299+0000 7fd81effd700 1 -- 172.21.15.31:0/20702713 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82c011b10 con 0x7fd83806de50 2023-10-06T02:28:20.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.299+0000 7fd81effd700 1 -- 172.21.15.31:0/20702713 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd82c0253f0 con 0x7fd83806de50 2023-10-06T02:28:20.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.299+0000 7fd81effd700 1 -- 172.21.15.31:0/20702713 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82c025790 con 0x7fd83806de50 2023-10-06T02:28:20.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.299+0000 7fd83c9fe700 1 -- 172.21.15.31:0/20702713 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8380c3fe0 con 0x7fd83806de50 2023-10-06T02:28:20.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.300+0000 7fd83c9fe700 1 -- 172.21.15.31:0/20702713 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8380c5000 con 0x7fd83806de50 2023-10-06T02:28:20.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.300+0000 7fd83c9fe700 1 -- 172.21.15.31:0/20702713 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8380c4480 con 0x7fd83806de50 2023-10-06T02:28:20.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.302+0000 7fd81effd700 1 -- 172.21.15.31:0/20702713 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7fd82c025b80 con 0x7fd83806de50 2023-10-06T02:28:20.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.302+0000 7fd81effd700 1 --2- 172.21.15.31:0/20702713 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd824067050 0x7fd824069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:20.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.303+0000 7fd81effd700 1 -- 172.21.15.31:0/20702713 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6144+0+0 (secure 0 0 0) 0x7fd82c0861e0 con 0x7fd83806de50 2023-10-06T02:28:20.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.303+0000 7fd836d9d700 1 --2- 172.21.15.31:0/20702713 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd824067050 0x7fd824069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:20.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.304+0000 7fd836d9d700 1 --2- 172.21.15.31:0/20702713 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd824067050 0x7fd824069500 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd828007fd0 tx=0x7fd828009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:20.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.307+0000 7fd81effd700 1 -- 172.21.15.31:0/20702713 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd82c053110 con 0x7fd83806de50 2023-10-06T02:28:20.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.511+0000 7fd83c9fe700 1 -- 172.21.15.31:0/20702713 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd8380c52e0 con 0x7fd83806de50 2023-10-06T02:28:20.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.512+0000 7fd81effd700 1 -- 172.21.15.31:0/20702713 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd82c056730 con 0x7fd83806de50 2023-10-06T02:28:20.513 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-10-06T02:28:20.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.514+0000 7fd83c9fe700 1 -- 172.21.15.31:0/20702713 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd824067050 msgr2=0x7fd824069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:20.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.514+0000 7fd83c9fe700 1 --2- 172.21.15.31:0/20702713 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd824067050 0x7fd824069500 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd828007fd0 tx=0x7fd828009040 comp rx=0 tx=0).stop 2023-10-06T02:28:20.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.514+0000 7fd83c9fe700 1 -- 172.21.15.31:0/20702713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd83806de50 msgr2=0x7fd8380c4ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:20.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.514+0000 7fd83c9fe700 1 --2- 172.21.15.31:0/20702713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd83806de50 0x7fd8380c4ac0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fd8380cc000 tx=0x7fd82c017000 comp rx=0 tx=0).stop 2023-10-06T02:28:20.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.514+0000 7fd83c9fe700 1 -- 172.21.15.31:0/20702713 shutdown_connections 2023-10-06T02:28:20.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.514+0000 7fd83c9fe700 1 --2- 172.21.15.31:0/20702713 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd824067050 0x7fd824069500 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:20.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.514+0000 7fd83c9fe700 1 --2- 172.21.15.31:0/20702713 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd838140a70 0x7fd83813eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:20.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.514+0000 7fd83c9fe700 1 --2- 172.21.15.31:0/20702713 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd83806a210 0x7fd8380ae370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:20.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.514+0000 7fd83c9fe700 1 --2- 172.21.15.31:0/20702713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd83806de50 0x7fd8380c4ac0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:20.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.514+0000 7fd83c9fe700 1 -- 172.21.15.31:0/20702713 >> 172.21.15.31:0/20702713 conn(0x7fd83806d110 msgr2=0x7fd8380c1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:20.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.514+0000 7fd83c9fe700 1 -- 172.21.15.31:0/20702713 shutdown_connections 2023-10-06T02:28:20.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.514+0000 7fd83c9fe700 1 -- 172.21.15.31:0/20702713 wait complete. 2023-10-06T02:28:20.533 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:28:20.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.719+0000 7feda8b9d700 1 Processor -- start 2023-10-06T02:28:20.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.719+0000 7feda8b9d700 1 -- start start 2023-10-06T02:28:20.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.720+0000 7feda8b9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7feda4142c00 0x7feda406b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:20.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.720+0000 7feda8b9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feda40c7f90 0x7feda406c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:20.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.720+0000 7feda8b9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feda40ca480 con 0x7feda406bd20 2023-10-06T02:28:20.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.720+0000 7feda8b9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feda415e240 con 0x7feda4142c00 2023-10-06T02:28:20.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.720+0000 7feda8b9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feda406b350 con 0x7feda40c7f90 2023-10-06T02:28:20.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.721+0000 7feda2ffd700 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7feda406bd20 0x7feda40c7880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:42608/0 (socket says 172.21.15.31:42608) 2023-10-06T02:28:20.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.721+0000 7feda2ffd700 1 -- 172.21.15.31:0/819988237 learned_addr learned my addr 172.21.15.31:0/819988237 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:28:20.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.721+0000 7feda3fff700 1 --2- 172.21.15.31:0/819988237 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feda40c7f90 0x7feda406c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:20.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.721+0000 7feda37fe700 1 --2- 172.21.15.31:0/819988237 >> v2:172.21.15.62:3300/0 conn(0x7feda4142c00 0x7feda406b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:20.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.721+0000 7feda27fc700 1 -- 172.21.15.31:0/819988237 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4070068682 0 0) 0x7feda40ca480 con 0x7feda406bd20 2023-10-06T02:28:20.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.722+0000 7feda27fc700 1 -- 172.21.15.31:0/819988237 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed8c003570 con 0x7feda406bd20 2023-10-06T02:28:20.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.722+0000 7feda37fe700 1 -- 172.21.15.31:0/819988237 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feda40c7f90 msgr2=0x7feda406c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:20.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.722+0000 7feda37fe700 1 --2- 172.21.15.31:0/819988237 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feda40c7f90 0x7feda406c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:20.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.722+0000 7feda37fe700 1 -- 172.21.15.31:0/819988237 >> v1:172.21.15.62:6789/0 conn(0x7feda406bd20 legacy=0x7feda40c7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:20.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.722+0000 7feda37fe700 1 -- 172.21.15.31:0/819988237 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feda415df60 con 0x7feda4142c00 2023-10-06T02:28:20.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.722+0000 7feda3fff700 1 --2- 172.21.15.31:0/819988237 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feda40c7f90 0x7feda406c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:28:20.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.722+0000 7feda37fe700 1 --2- 172.21.15.31:0/819988237 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feda4142c00 0x7feda406b7e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fed940098e0 tx=0x7fed940091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a321157995901624 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:20.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.722+0000 7feda27fc700 1 -- 172.21.15.31:0/819988237 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed94011630 con 0x7feda4142c00 2023-10-06T02:28:20.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.722+0000 7feda27fc700 1 -- 172.21.15.31:0/819988237 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed94011790 con 0x7feda4142c00 2023-10-06T02:28:20.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.722+0000 7feda27fc700 1 -- 172.21.15.31:0/819988237 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed94011b30 con 0x7feda4142c00 2023-10-06T02:28:20.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.723+0000 7feda8b9d700 1 -- 172.21.15.31:0/819988237 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feda4142c00 msgr2=0x7feda406b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:20.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.723+0000 7feda8b9d700 1 --2- 172.21.15.31:0/819988237 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feda4142c00 0x7feda406b7e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fed940098e0 tx=0x7fed940091a0 comp rx=0 tx=0).stop 2023-10-06T02:28:20.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.723+0000 7feda8b9d700 1 -- 172.21.15.31:0/819988237 shutdown_connections 2023-10-06T02:28:20.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.723+0000 7feda8b9d700 1 --2- 172.21.15.31:0/819988237 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feda40c7f90 0x7feda406c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:20.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.723+0000 7feda8b9d700 1 --2- 172.21.15.31:0/819988237 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feda4142c00 0x7feda406b7e0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:20.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.723+0000 7feda8b9d700 1 -- 172.21.15.31:0/819988237 >> 172.21.15.31:0/819988237 conn(0x7feda40675a0 msgr2=0x7feda40679a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:20.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.723+0000 7feda8b9d700 1 -- 172.21.15.31:0/819988237 shutdown_connections 2023-10-06T02:28:20.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.723+0000 7feda8b9d700 1 -- 172.21.15.31:0/819988237 wait complete. 2023-10-06T02:28:20.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.724+0000 7feda8b9d700 1 Processor -- start 2023-10-06T02:28:20.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.725+0000 7feda8b9d700 1 -- start start 2023-10-06T02:28:20.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.725+0000 7feda8b9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feda406bd20 0x7feda406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:20.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.725+0000 7feda8b9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feda40c7f90 0x7feda4158b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:20.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.725+0000 7feda8b9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feda4142c00 0x7feda41590b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:20.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.725+0000 7feda8b9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feda41595f0 con 0x7feda40c7f90 2023-10-06T02:28:20.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.725+0000 7feda8b9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feda40c49b0 con 0x7feda4142c00 2023-10-06T02:28:20.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.725+0000 7feda8b9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feda40c43d0 con 0x7feda406bd20 2023-10-06T02:28:20.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.726+0000 7feda2ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feda40c7f90 0x7feda4158b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:20.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.726+0000 7feda2ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feda40c7f90 0x7feda4158b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33006/0 (socket says 172.21.15.31:33006) 2023-10-06T02:28:20.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.726+0000 7feda2ffd700 1 -- 172.21.15.31:0/936405433 learned_addr learned my addr 172.21.15.31:0/936405433 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:20.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.726+0000 7feda3fff700 1 --2- 172.21.15.31:0/936405433 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feda4142c00 0x7feda41590b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:20.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.726+0000 7feda37fe700 1 --2- 172.21.15.31:0/936405433 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feda406bd20 0x7feda406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:20.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.726+0000 7feda2ffd700 1 -- 172.21.15.31:0/936405433 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feda406bd20 msgr2=0x7feda406c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:20.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.726+0000 7feda2ffd700 1 --2- 172.21.15.31:0/936405433 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feda406bd20 0x7feda406c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:20.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.726+0000 7feda2ffd700 1 -- 172.21.15.31:0/936405433 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feda4142c00 msgr2=0x7feda41590b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:20.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.726+0000 7feda2ffd700 1 --2- 172.21.15.31:0/936405433 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feda4142c00 0x7feda41590b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:20.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.726+0000 7feda2ffd700 1 -- 172.21.15.31:0/936405433 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed9c009a20 con 0x7feda40c7f90 2023-10-06T02:28:20.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.726+0000 7feda2ffd700 1 --2- 172.21.15.31:0/936405433 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feda40c7f90 0x7feda4158b70 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fed9800c820 tx=0x7fed98007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:20.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.727+0000 7fed8b7fe700 1 -- 172.21.15.31:0/936405433 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed98005c00 con 0x7feda40c7f90 2023-10-06T02:28:20.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.727+0000 7fed8b7fe700 1 -- 172.21.15.31:0/936405433 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed98005d80 con 0x7feda40c7f90 2023-10-06T02:28:20.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.727+0000 7feda8b9d700 1 -- 172.21.15.31:0/936405433 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed94009420 con 0x7feda40c7f90 2023-10-06T02:28:20.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.727+0000 7fed8b7fe700 1 -- 172.21.15.31:0/936405433 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed98007320 con 0x7feda40c7f90 2023-10-06T02:28:20.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.727+0000 7feda8b9d700 1 -- 172.21.15.31:0/936405433 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feda40c4640 con 0x7feda40c7f90 2023-10-06T02:28:20.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.728+0000 7feda8b9d700 1 -- 172.21.15.31:0/936405433 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feda404c3c0 con 0x7feda40c7f90 2023-10-06T02:28:20.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.729+0000 7fed8b7fe700 1 -- 172.21.15.31:0/936405433 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7fed9801b3b0 con 0x7feda40c7f90 2023-10-06T02:28:20.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.730+0000 7fed8b7fe700 1 --2- 172.21.15.31:0/936405433 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fed90067050 0x7fed90069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:20.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.730+0000 7fed8b7fe700 1 -- 172.21.15.31:0/936405433 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6144+0+0 (secure 0 0 0) 0x7fed98083dd0 con 0x7feda40c7f90 2023-10-06T02:28:20.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.730+0000 7feda37fe700 1 --2- 172.21.15.31:0/936405433 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fed90067050 0x7fed90069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:20.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.731+0000 7feda37fe700 1 --2- 172.21.15.31:0/936405433 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fed90067050 0x7fed90069500 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fed940091e0 tx=0x7fed94017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:20.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.734+0000 7fed8b7fe700 1 -- 172.21.15.31:0/936405433 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fed98050dc0 con 0x7feda40c7f90 2023-10-06T02:28:20.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.980+0000 7feda8b9d700 1 -- 172.21.15.31:0/936405433 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feda40c4b20 con 0x7feda40c7f90 2023-10-06T02:28:20.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.980+0000 7fed8b7fe700 1 -- 172.21.15.31:0/936405433 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4596 (secure 0 0 0) 0x7fed98017070 con 0x7feda40c7f90 2023-10-06T02:28:20.982 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:28:20.983 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":62,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5759,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/2470516084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2470516084},{"type":"v1","addr":"172.21.15.62:6834","nonce":2470516084}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":6364,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/790751965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":790751965},{"type":"v1","addr":"172.21.15.31:6834","nonce":790751965}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":6571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3862349969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":3862349969},{"type":"v1","addr":"172.21.15.62:6835","nonce":3862349969}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:27:58.019137+0000","modified":"2023-10-06T02:28:01.555185+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6714},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6714":{"gid":6714,"name":"c","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/1002908930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":1002908930},{"type":"v1","addr":"172.21.15.31:6835","nonce":1002908930}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T02:28:20.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.982+0000 7feda8b9d700 1 -- 172.21.15.31:0/936405433 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fed90067050 msgr2=0x7fed90069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:20.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.982+0000 7feda8b9d700 1 --2- 172.21.15.31:0/936405433 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fed90067050 0x7fed90069500 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fed940091e0 tx=0x7fed94017000 comp rx=0 tx=0).stop 2023-10-06T02:28:20.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.982+0000 7feda8b9d700 1 -- 172.21.15.31:0/936405433 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feda40c7f90 msgr2=0x7feda4158b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:20.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.982+0000 7feda8b9d700 1 --2- 172.21.15.31:0/936405433 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feda40c7f90 0x7feda4158b70 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fed9800c820 tx=0x7fed98007dd0 comp rx=0 tx=0).stop 2023-10-06T02:28:20.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.982+0000 7feda8b9d700 1 -- 172.21.15.31:0/936405433 shutdown_connections 2023-10-06T02:28:20.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.982+0000 7feda8b9d700 1 --2- 172.21.15.31:0/936405433 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feda4142c00 0x7feda41590b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:20.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.982+0000 7feda8b9d700 1 --2- 172.21.15.31:0/936405433 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fed90067050 0x7fed90069500 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:20.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.982+0000 7feda8b9d700 1 --2- 172.21.15.31:0/936405433 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feda406bd20 0x7feda406c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:20.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.982+0000 7feda8b9d700 1 --2- 172.21.15.31:0/936405433 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feda40c7f90 0x7feda4158b70 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:20.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.982+0000 7feda8b9d700 1 -- 172.21.15.31:0/936405433 >> 172.21.15.31:0/936405433 conn(0x7feda40675a0 msgr2=0x7feda40c8dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:20.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.983+0000 7feda8b9d700 1 -- 172.21.15.31:0/936405433 shutdown_connections 2023-10-06T02:28:20.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:20.983+0000 7feda8b9d700 1 -- 172.21.15.31:0/936405433 wait complete. 2023-10-06T02:28:20.987 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 62 2023-10-06T02:28:21.001 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:28:21.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.188+0000 7fd87359d700 1 Processor -- start 2023-10-06T02:28:21.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.188+0000 7fd87359d700 1 -- start start 2023-10-06T02:28:21.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.189+0000 7fd87359d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd86c0aaed0 0x7fd86c06e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:21.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.189+0000 7fd87359d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd86c0ca980 con 0x7fd86c0cc5b0 2023-10-06T02:28:21.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.189+0000 7fd87359d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd86c0ad780 con 0x7fd86c0accb0 2023-10-06T02:28:21.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.189+0000 7fd87359d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd86c0ab2d0 con 0x7fd86c0aaed0 2023-10-06T02:28:21.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.189+0000 7fd871d9a700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fd86c0cc5b0 0x7fd86c0ca270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:43722/0 (socket says 172.21.15.31:43722) 2023-10-06T02:28:21.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.189+0000 7fd871d9a700 1 -- 172.21.15.31:0/2348389042 learned_addr learned my addr 172.21.15.31:0/2348389042 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:28:21.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.190+0000 7fd872d9c700 1 --2- 172.21.15.31:0/2348389042 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd86c0aaed0 0x7fd86c06e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:21.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.190+0000 7fd871599700 1 -- 172.21.15.31:0/2348389042 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3450183153 0 0) 0x7fd86c0ca980 con 0x7fd86c0cc5b0 2023-10-06T02:28:21.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.190+0000 7fd871599700 1 -- 172.21.15.31:0/2348389042 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd858003570 con 0x7fd86c0cc5b0 2023-10-06T02:28:21.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.190+0000 7fd871599700 1 -- 172.21.15.31:0/2348389042 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3840734724 0 0) 0x7fd86c0ad780 con 0x7fd86c0accb0 2023-10-06T02:28:21.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.190+0000 7fd871599700 1 -- 172.21.15.31:0/2348389042 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd86c0ca980 con 0x7fd86c0accb0 2023-10-06T02:28:21.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.190+0000 7fd872d9c700 1 -- 172.21.15.31:0/2348389042 >> v1:172.21.15.62:6789/0 conn(0x7fd86c0accb0 legacy=0x7fd86c0cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:21.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.190+0000 7fd872d9c700 1 -- 172.21.15.31:0/2348389042 >> v1:172.21.15.31:6789/0 conn(0x7fd86c0cc5b0 legacy=0x7fd86c0ca270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:21.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.190+0000 7fd872d9c700 1 -- 172.21.15.31:0/2348389042 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd86c0cee60 con 0x7fd86c0aaed0 2023-10-06T02:28:21.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.190+0000 7fd872d9c700 1 --2- 172.21.15.31:0/2348389042 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd86c0aaed0 0x7fd86c06e330 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd864009e90 tx=0x7fd864009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4302cf254ed11b07 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:21.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.191+0000 7fd871599700 1 -- 172.21.15.31:0/2348389042 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd86400e070 con 0x7fd86c0aaed0 2023-10-06T02:28:21.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.191+0000 7fd871599700 1 -- 172.21.15.31:0/2348389042 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd86400bb80 con 0x7fd86c0aaed0 2023-10-06T02:28:21.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.191+0000 7fd871599700 1 -- 172.21.15.31:0/2348389042 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd864012430 con 0x7fd86c0aaed0 2023-10-06T02:28:21.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.191+0000 7fd87359d700 1 -- 172.21.15.31:0/2348389042 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd86c0aaed0 msgr2=0x7fd86c06e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:21.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.191+0000 7fd87359d700 1 --2- 172.21.15.31:0/2348389042 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd86c0aaed0 0x7fd86c06e330 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd864009e90 tx=0x7fd864009780 comp rx=0 tx=0).stop 2023-10-06T02:28:21.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.191+0000 7fd87359d700 1 -- 172.21.15.31:0/2348389042 shutdown_connections 2023-10-06T02:28:21.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.191+0000 7fd87359d700 1 --2- 172.21.15.31:0/2348389042 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd86c0aaed0 0x7fd86c06e330 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:21.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.191+0000 7fd87359d700 1 -- 172.21.15.31:0/2348389042 >> 172.21.15.31:0/2348389042 conn(0x7fd86c06ce80 msgr2=0x7fd86c0ce370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:21.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.191+0000 7fd87359d700 1 -- 172.21.15.31:0/2348389042 shutdown_connections 2023-10-06T02:28:21.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.191+0000 7fd87359d700 1 -- 172.21.15.31:0/2348389042 wait complete. 2023-10-06T02:28:21.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.193+0000 7fd87359d700 1 Processor -- start 2023-10-06T02:28:21.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.193+0000 7fd87359d700 1 -- start start 2023-10-06T02:28:21.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.194+0000 7fd87359d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd86c0aaed0 0x7fd86c16a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:21.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.194+0000 7fd87359d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd86c0accb0 0x7fd86c0b3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:21.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.194+0000 7fd87359d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd86c0cc5b0 0x7fd86c0b38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:21.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.194+0000 7fd87359d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd86c0b11c0 con 0x7fd86c0aaed0 2023-10-06T02:28:21.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.194+0000 7fd87359d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd86c0b14a0 con 0x7fd86c0cc5b0 2023-10-06T02:28:21.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.194+0000 7fd87359d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd86c0b1780 con 0x7fd86c0accb0 2023-10-06T02:28:21.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.194+0000 7fd87259b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd86c0aaed0 0x7fd86c16a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:21.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.194+0000 7fd87259b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd86c0aaed0 0x7fd86c16a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33010/0 (socket says 172.21.15.31:33010) 2023-10-06T02:28:21.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.194+0000 7fd87259b700 1 -- 172.21.15.31:0/891014652 learned_addr learned my addr 172.21.15.31:0/891014652 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:21.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.194+0000 7fd871d9a700 1 --2- 172.21.15.31:0/891014652 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd86c0accb0 0x7fd86c0b3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:21.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.194+0000 7fd872d9c700 1 --2- 172.21.15.31:0/891014652 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd86c0cc5b0 0x7fd86c0b38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:21.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.194+0000 7fd87259b700 1 -- 172.21.15.31:0/891014652 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd86c0accb0 msgr2=0x7fd86c0b3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:21.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.194+0000 7fd87259b700 1 --2- 172.21.15.31:0/891014652 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd86c0accb0 0x7fd86c0b3380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:21.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.195+0000 7fd87259b700 1 -- 172.21.15.31:0/891014652 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd86c0cc5b0 msgr2=0x7fd86c0b38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:21.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.195+0000 7fd87259b700 1 --2- 172.21.15.31:0/891014652 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd86c0cc5b0 0x7fd86c0b38c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:21.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.195+0000 7fd87259b700 1 -- 172.21.15.31:0/891014652 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd864009a20 con 0x7fd86c0aaed0 2023-10-06T02:28:21.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.195+0000 7fd872d9c700 1 --2- 172.21.15.31:0/891014652 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd86c0cc5b0 0x7fd86c0b38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:28:21.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.195+0000 7fd87259b700 1 --2- 172.21.15.31:0/891014652 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd86c0aaed0 0x7fd86c16a440 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fd86800c580 tx=0x7fd868007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:21.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.195+0000 7fd8627fc700 1 -- 172.21.15.31:0/891014652 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd868010510 con 0x7fd86c0aaed0 2023-10-06T02:28:21.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.195+0000 7fd8627fc700 1 -- 172.21.15.31:0/891014652 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd868010670 con 0x7fd86c0aaed0 2023-10-06T02:28:21.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.195+0000 7fd8627fc700 1 -- 172.21.15.31:0/891014652 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd868010a30 con 0x7fd86c0aaed0 2023-10-06T02:28:21.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.195+0000 7fd87359d700 1 -- 172.21.15.31:0/891014652 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd86c0b1a60 con 0x7fd86c0aaed0 2023-10-06T02:28:21.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.195+0000 7fd87359d700 1 -- 172.21.15.31:0/891014652 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd86c16a980 con 0x7fd86c0aaed0 2023-10-06T02:28:21.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.196+0000 7fd87359d700 1 -- 172.21.15.31:0/891014652 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd86c04c3c0 con 0x7fd86c0aaed0 2023-10-06T02:28:21.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.197+0000 7fd8627fc700 1 -- 172.21.15.31:0/891014652 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7fd86801b400 con 0x7fd86c0aaed0 2023-10-06T02:28:21.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.198+0000 7fd8627fc700 1 --2- 172.21.15.31:0/891014652 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd850067080 0x7fd850069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:21.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.198+0000 7fd8627fc700 1 -- 172.21.15.31:0/891014652 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6144+0+0 (secure 0 0 0) 0x7fd868084540 con 0x7fd86c0aaed0 2023-10-06T02:28:21.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.198+0000 7fd871d9a700 1 --2- 172.21.15.31:0/891014652 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd850067080 0x7fd850069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:21.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.199+0000 7fd871d9a700 1 --2- 172.21.15.31:0/891014652 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd850067080 0x7fd850069530 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd85c0058d0 tx=0x7fd85c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:21.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.202+0000 7fd8627fc700 1 -- 172.21.15.31:0/891014652 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd868051470 con 0x7fd86c0aaed0 2023-10-06T02:28:21.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.405+0000 7fd87359d700 1 -- 172.21.15.31:0/891014652 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd86c0cf140 con 0x7fd86c0aaed0 2023-10-06T02:28:21.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.406+0000 7fd8627fc700 1 -- 172.21.15.31:0/891014652 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+16950 (secure 0 0 0) 0x7fd868054a90 con 0x7fd86c0aaed0 2023-10-06T02:28:21.408 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:28:21.408 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":95,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:28:17.469239+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:27:56.192975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-10-06T02:27:56.993140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6835/2757737386":"2023-10-07T02:28:14.456919+0000","172.21.15.62:6833/2757737386":"2023-10-07T02:28:14.456919+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:28:21.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.409+0000 7fd87359d700 1 -- 172.21.15.31:0/891014652 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd850067080 msgr2=0x7fd850069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:21.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.409+0000 7fd87359d700 1 --2- 172.21.15.31:0/891014652 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd850067080 0x7fd850069530 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd85c0058d0 tx=0x7fd85c009040 comp rx=0 tx=0).stop 2023-10-06T02:28:21.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.409+0000 7fd87359d700 1 -- 172.21.15.31:0/891014652 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd86c0aaed0 msgr2=0x7fd86c16a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:21.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.409+0000 7fd87359d700 1 --2- 172.21.15.31:0/891014652 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd86c0aaed0 0x7fd86c16a440 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fd86800c580 tx=0x7fd868007e10 comp rx=0 tx=0).stop 2023-10-06T02:28:21.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.409+0000 7fd87359d700 1 -- 172.21.15.31:0/891014652 shutdown_connections 2023-10-06T02:28:21.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.409+0000 7fd87359d700 1 --2- 172.21.15.31:0/891014652 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd86c0cc5b0 0x7fd86c0b38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:21.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.409+0000 7fd87359d700 1 --2- 172.21.15.31:0/891014652 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd86c0aaed0 0x7fd86c16a440 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:21.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.409+0000 7fd87359d700 1 --2- 172.21.15.31:0/891014652 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd850067080 0x7fd850069530 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:21.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.409+0000 7fd87359d700 1 --2- 172.21.15.31:0/891014652 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd86c0accb0 0x7fd86c0b3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:21.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.409+0000 7fd87359d700 1 -- 172.21.15.31:0/891014652 >> 172.21.15.31:0/891014652 conn(0x7fd86c06ce80 msgr2=0x7fd86c159870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:21.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.409+0000 7fd87359d700 1 -- 172.21.15.31:0/891014652 shutdown_connections 2023-10-06T02:28:21.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.409+0000 7fd87359d700 1 -- 172.21.15.31:0/891014652 wait complete. 2023-10-06T02:28:21.428 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T02:28:21.428 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:28:21.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.615+0000 7f5f39f9d700 1 Processor -- start 2023-10-06T02:28:22.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.615+0000 7f5f39f9d700 1 -- start start 2023-10-06T02:28:22.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.617+0000 7f5f39f9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f5f3413fb20 0x7f5f3415a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:22.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.617+0000 7f5f39f9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5f3413fef0 0x7f5f340c7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:22.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.617+0000 7f5f39f9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f3415ae70 con 0x7f5f3406aaa0 2023-10-06T02:28:22.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.617+0000 7f5f39f9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f3406af60 con 0x7f5f3413fef0 2023-10-06T02:28:22.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.617+0000 7f5f39f9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f3406b240 con 0x7f5f3413fb20 2023-10-06T02:28:22.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.617+0000 7f5f33fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5f3413fef0 0x7f5f340c7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:22.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.617+0000 7f5f33fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5f3413fef0 0x7f5f340c7990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33026/0 (socket says 172.21.15.31:33026) 2023-10-06T02:28:22.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.617+0000 7f5f33fff700 1 -- 172.21.15.31:0/1836508072 learned_addr learned my addr 172.21.15.31:0/1836508072 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:22.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.617+0000 7f5f33fff700 1 -- 172.21.15.31:0/1836508072 >> v2:172.21.15.62:3300/0 conn(0x7f5f3413fb20 msgr2=0x7f5f3415a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:22.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.617+0000 7f5f33fff700 1 --2- 172.21.15.31:0/1836508072 >> v2:172.21.15.62:3300/0 conn(0x7f5f3413fb20 0x7f5f3415a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:22.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.617+0000 7f5f33fff700 1 -- 172.21.15.31:0/1836508072 >> v1:172.21.15.62:6789/0 conn(0x7f5f3406aaa0 legacy=0x7f5f340c6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:22.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.617+0000 7f5f33fff700 1 -- 172.21.15.31:0/1836508072 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f3406b520 con 0x7f5f3413fef0 2023-10-06T02:28:22.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.618+0000 7f5f33fff700 1 --2- 172.21.15.31:0/1836508072 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f3413fef0 0x7f5f340c7990 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f5f340695b0 tx=0x7f5f20009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56b32ff74158e393 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:22.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.618+0000 7f5f337fe700 1 -- 172.21.15.31:0/1836508072 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f20005c10 con 0x7f5f3413fef0 2023-10-06T02:28:22.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.618+0000 7f5f337fe700 1 -- 172.21.15.31:0/1836508072 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f20005d70 con 0x7f5f3413fef0 2023-10-06T02:28:22.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.618+0000 7f5f337fe700 1 -- 172.21.15.31:0/1836508072 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f2000b840 con 0x7f5f3413fef0 2023-10-06T02:28:22.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.618+0000 7f5f39f9d700 1 -- 172.21.15.31:0/1836508072 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f3413fef0 msgr2=0x7f5f340c7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:22.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.618+0000 7f5f39f9d700 1 --2- 172.21.15.31:0/1836508072 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f3413fef0 0x7f5f340c7990 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f5f340695b0 tx=0x7f5f20009220 comp rx=0 tx=0).stop 2023-10-06T02:28:22.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.618+0000 7f5f39f9d700 1 -- 172.21.15.31:0/1836508072 shutdown_connections 2023-10-06T02:28:22.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.618+0000 7f5f39f9d700 1 --2- 172.21.15.31:0/1836508072 >> v2:172.21.15.62:3300/0 conn(0x7f5f3413fb20 0x7f5f3415a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:22.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.618+0000 7f5f39f9d700 1 --2- 172.21.15.31:0/1836508072 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f3413fef0 0x7f5f340c7990 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:22.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.618+0000 7f5f39f9d700 1 -- 172.21.15.31:0/1836508072 >> 172.21.15.31:0/1836508072 conn(0x7f5f3406d170 msgr2=0x7f5f340cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:22.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.619+0000 7f5f39f9d700 1 -- 172.21.15.31:0/1836508072 shutdown_connections 2023-10-06T02:28:22.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.619+0000 7f5f39f9d700 1 -- 172.21.15.31:0/1836508072 wait complete. 2023-10-06T02:28:22.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.620+0000 7f5f39f9d700 1 Processor -- start 2023-10-06T02:28:22.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.620+0000 7f5f39f9d700 1 -- start start 2023-10-06T02:28:22.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.621+0000 7f5f39f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5f3406aaa0 0x7f5f3414f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:22.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.621+0000 7f5f39f9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f3413fb20 0x7f5f340ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:22.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.621+0000 7f5f39f9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5f3413fef0 0x7f5f34137230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:22.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.621+0000 7f5f39f9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f341380c0 con 0x7f5f3413fb20 2023-10-06T02:28:22.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.621+0000 7f5f39f9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f341506e0 con 0x7f5f3413fef0 2023-10-06T02:28:22.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.621+0000 7f5f39f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3414f880 con 0x7f5f3406aaa0 2023-10-06T02:28:22.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.621+0000 7f5f33fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f3413fb20 0x7f5f340ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:22.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.621+0000 7f5f33fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f3413fb20 0x7f5f340ca3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33042/0 (socket says 172.21.15.31:33042) 2023-10-06T02:28:22.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.621+0000 7f5f33fff700 1 -- 172.21.15.31:0/591027236 learned_addr learned my addr 172.21.15.31:0/591027236 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:22.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.621+0000 7f5f38f9b700 1 --2- 172.21.15.31:0/591027236 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5f3406aaa0 0x7f5f3414f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:22.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.621+0000 7f5f3979c700 1 --2- 172.21.15.31:0/591027236 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5f3413fef0 0x7f5f34137230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:22.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.622+0000 7f5f33fff700 1 -- 172.21.15.31:0/591027236 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5f3406aaa0 msgr2=0x7f5f3414f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:22.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.622+0000 7f5f33fff700 1 --2- 172.21.15.31:0/591027236 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5f3406aaa0 0x7f5f3414f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:22.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.622+0000 7f5f33fff700 1 -- 172.21.15.31:0/591027236 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5f3413fef0 msgr2=0x7f5f34137230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:22.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.622+0000 7f5f33fff700 1 --2- 172.21.15.31:0/591027236 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5f3413fef0 0x7f5f34137230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:22.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.622+0000 7f5f33fff700 1 -- 172.21.15.31:0/591027236 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f20009480 con 0x7f5f3413fb20 2023-10-06T02:28:22.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.622+0000 7f5f33fff700 1 --2- 172.21.15.31:0/591027236 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f3413fb20 0x7f5f340ca3c0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f5f20003080 tx=0x7f5f20015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:22.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.622+0000 7f5f30ff9700 1 -- 172.21.15.31:0/591027236 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f20011aa0 con 0x7f5f3413fb20 2023-10-06T02:28:22.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.622+0000 7f5f30ff9700 1 -- 172.21.15.31:0/591027236 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f20011c40 con 0x7f5f3413fb20 2023-10-06T02:28:22.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.622+0000 7f5f30ff9700 1 -- 172.21.15.31:0/591027236 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f2001d430 con 0x7f5f3413fb20 2023-10-06T02:28:22.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.622+0000 7f5f39f9d700 1 -- 172.21.15.31:0/591027236 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f3414fad0 con 0x7f5f3413fb20 2023-10-06T02:28:22.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.622+0000 7f5f39f9d700 1 -- 172.21.15.31:0/591027236 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f34150090 con 0x7f5f3413fb20 2023-10-06T02:28:22.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.623+0000 7f5f39f9d700 1 -- 172.21.15.31:0/591027236 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f34150890 con 0x7f5f3413fb20 2023-10-06T02:28:22.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.624+0000 7f5f30ff9700 1 -- 172.21.15.31:0/591027236 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f5f2001d7e0 con 0x7f5f3413fb20 2023-10-06T02:28:22.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.625+0000 7f5f30ff9700 1 --2- 172.21.15.31:0/591027236 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5f18067050 0x7f5f18069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:22.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.625+0000 7f5f30ff9700 1 -- 172.21.15.31:0/591027236 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f5f200862b0 con 0x7f5f3413fb20 2023-10-06T02:28:22.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.625+0000 7f5f38f9b700 1 --2- 172.21.15.31:0/591027236 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5f18067050 0x7f5f18069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:22.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.626+0000 7f5f38f9b700 1 --2- 172.21.15.31:0/591027236 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5f18067050 0x7f5f18069500 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5f28006fd0 tx=0x7f5f28008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:22.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.629+0000 7f5f30ff9700 1 -- 172.21.15.31:0/591027236 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5f34150890 con 0x7f5f3413fb20 2023-10-06T02:28:22.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.877+0000 7f5f39f9d700 1 -- 172.21.15.31:0/591027236 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5f3415ae70 con 0x7f5f3413fb20 2023-10-06T02:28:22.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.878+0000 7f5f30ff9700 1 -- 172.21.15.31:0/591027236 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f5f3415ae70 con 0x7f5f3413fb20 2023-10-06T02:28:22.284 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:28:22.285 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":62,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5759,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/2470516084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2470516084},{"type":"v1","addr":"172.21.15.62:6834","nonce":2470516084}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":6364,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/790751965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":790751965},{"type":"v1","addr":"172.21.15.31:6834","nonce":790751965}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":6571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3862349969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":3862349969},{"type":"v1","addr":"172.21.15.62:6835","nonce":3862349969}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:27:58.019137+0000","modified":"2023-10-06T02:28:01.555185+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6714},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6714":{"gid":6714,"name":"c","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/1002908930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":1002908930},{"type":"v1","addr":"172.21.15.31:6835","nonce":1002908930}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T02:28:22.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.880+0000 7f5f39f9d700 1 -- 172.21.15.31:0/591027236 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5f18067050 msgr2=0x7f5f18069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:22.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.880+0000 7f5f39f9d700 1 --2- 172.21.15.31:0/591027236 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5f18067050 0x7f5f18069500 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5f28006fd0 tx=0x7f5f28008040 comp rx=0 tx=0).stop 2023-10-06T02:28:22.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.880+0000 7f5f39f9d700 1 -- 172.21.15.31:0/591027236 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f3413fb20 msgr2=0x7f5f340ca3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:22.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.880+0000 7f5f39f9d700 1 --2- 172.21.15.31:0/591027236 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f3413fb20 0x7f5f340ca3c0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f5f20003080 tx=0x7f5f20015000 comp rx=0 tx=0).stop 2023-10-06T02:28:22.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.880+0000 7f5f39f9d700 1 -- 172.21.15.31:0/591027236 shutdown_connections 2023-10-06T02:28:22.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.880+0000 7f5f39f9d700 1 --2- 172.21.15.31:0/591027236 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5f3413fef0 0x7f5f34137230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:22.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.880+0000 7f5f39f9d700 1 --2- 172.21.15.31:0/591027236 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5f18067050 0x7f5f18069500 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:22.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.880+0000 7f5f39f9d700 1 --2- 172.21.15.31:0/591027236 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5f3406aaa0 0x7f5f3414f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:22.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.880+0000 7f5f39f9d700 1 --2- 172.21.15.31:0/591027236 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f3413fb20 0x7f5f340ca3c0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:22.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.880+0000 7f5f39f9d700 1 -- 172.21.15.31:0/591027236 >> 172.21.15.31:0/591027236 conn(0x7f5f3406d170 msgr2=0x7f5f340c2540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:22.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.880+0000 7f5f39f9d700 1 -- 172.21.15.31:0/591027236 shutdown_connections 2023-10-06T02:28:22.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:21.880+0000 7f5f39f9d700 1 -- 172.21.15.31:0/591027236 wait complete. 2023-10-06T02:28:22.289 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 62 2023-10-06T02:28:22.290 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:28:22.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.478+0000 7f47393ff700 1 Processor -- start 2023-10-06T02:28:22.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.478+0000 7f47393ff700 1 -- start start 2023-10-06T02:28:22.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.479+0000 7f47393ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4734159850 0x7f47340adfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:22.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.480+0000 7f47393ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47340adb20 con 0x7f47340b1040 2023-10-06T02:28:22.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.480+0000 7f47393ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47340acf10 con 0x7f4734159c20 2023-10-06T02:28:22.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.480+0000 7f47393ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47340ccdc0 con 0x7f4734159850 2023-10-06T02:28:22.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.480+0000 7f4732ffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4734159850 0x7f47340adfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:22.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.480+0000 7f47337fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f47340b1040 0x7f47340c93e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:43736/0 (socket says 172.21.15.31:43736) 2023-10-06T02:28:22.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.480+0000 7f47337fe700 1 -- 172.21.15.31:0/1045529462 learned_addr learned my addr 172.21.15.31:0/1045529462 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:28:22.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.480+0000 7f4732ffd700 1 -- 172.21.15.31:0/1045529462 >> v1:172.21.15.62:6789/0 conn(0x7f4734159c20 legacy=0x7f47340b0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:22.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.480+0000 7f4732ffd700 1 -- 172.21.15.31:0/1045529462 >> v1:172.21.15.31:6789/0 conn(0x7f47340b1040 legacy=0x7f47340c93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:22.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.480+0000 7f4732ffd700 1 -- 172.21.15.31:0/1045529462 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47340ad1f0 con 0x7f4734159850 2023-10-06T02:28:22.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.481+0000 7f4732ffd700 1 --2- 172.21.15.31:0/1045529462 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4734159850 0x7f47340adfb0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f47240098e0 tx=0x7f47240091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4360579dc6b037bc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:22.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.481+0000 7f4731ffb700 1 -- 172.21.15.31:0/1045529462 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4724009e80 con 0x7f4734159850 2023-10-06T02:28:22.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.481+0000 7f4731ffb700 1 -- 172.21.15.31:0/1045529462 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f472400b860 con 0x7f4734159850 2023-10-06T02:28:22.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.481+0000 7f4731ffb700 1 -- 172.21.15.31:0/1045529462 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f472400bbe0 con 0x7f4734159850 2023-10-06T02:28:22.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.481+0000 7f47393ff700 1 -- 172.21.15.31:0/1045529462 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4734159850 msgr2=0x7f47340adfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:22.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.481+0000 7f47393ff700 1 --2- 172.21.15.31:0/1045529462 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4734159850 0x7f47340adfb0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f47240098e0 tx=0x7f47240091a0 comp rx=0 tx=0).stop 2023-10-06T02:28:22.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.482+0000 7f47393ff700 1 -- 172.21.15.31:0/1045529462 shutdown_connections 2023-10-06T02:28:22.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.482+0000 7f47393ff700 1 --2- 172.21.15.31:0/1045529462 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4734159850 0x7f47340adfb0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:22.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.482+0000 7f47393ff700 1 -- 172.21.15.31:0/1045529462 >> 172.21.15.31:0/1045529462 conn(0x7f473406d170 msgr2=0x7f47340cc300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:22.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.482+0000 7f47393ff700 1 -- 172.21.15.31:0/1045529462 shutdown_connections 2023-10-06T02:28:22.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.482+0000 7f47393ff700 1 -- 172.21.15.31:0/1045529462 wait complete. 2023-10-06T02:28:22.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.483+0000 7f47393ff700 1 Processor -- start 2023-10-06T02:28:22.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.483+0000 7f47393ff700 1 -- start start 2023-10-06T02:28:22.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.484+0000 7f47393ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47340b1040 0x7f4734145810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:22.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.484+0000 7f47393ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4734159850 0x7f47340a8de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:22.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.484+0000 7f47393ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4734159c20 0x7f47340c7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:22.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.484+0000 7f47393ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47340aa5c0 con 0x7f4734159850 2023-10-06T02:28:22.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.484+0000 7f47393ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4734145d50 con 0x7f47340b1040 2023-10-06T02:28:22.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.484+0000 7f47393ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4734159200 con 0x7f4734159c20 2023-10-06T02:28:22.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.484+0000 7f47327fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4734159850 0x7f47340a8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:22.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.484+0000 7f47327fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4734159850 0x7f47340a8de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33056/0 (socket says 172.21.15.31:33056) 2023-10-06T02:28:22.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.484+0000 7f47327fc700 1 -- 172.21.15.31:0/3146660163 learned_addr learned my addr 172.21.15.31:0/3146660163 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:22.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.484+0000 7f4732ffd700 1 --2- 172.21.15.31:0/3146660163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47340b1040 0x7f4734145810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:22.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.484+0000 7f47337fe700 1 --2- 172.21.15.31:0/3146660163 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4734159c20 0x7f47340c7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:22.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.485+0000 7f47327fc700 1 -- 172.21.15.31:0/3146660163 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4734159c20 msgr2=0x7f47340c7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:22.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.485+0000 7f47327fc700 1 --2- 172.21.15.31:0/3146660163 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4734159c20 0x7f47340c7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:22.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.485+0000 7f47327fc700 1 -- 172.21.15.31:0/3146660163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47340b1040 msgr2=0x7f4734145810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:22.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.485+0000 7f47327fc700 1 --2- 172.21.15.31:0/3146660163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47340b1040 0x7f4734145810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:22.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.485+0000 7f47327fc700 1 -- 172.21.15.31:0/3146660163 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4724009420 con 0x7f4734159850 2023-10-06T02:28:22.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.485+0000 7f47327fc700 1 --2- 172.21.15.31:0/3146660163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4734159850 0x7f47340a8de0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f472800c4e0 tx=0x7f472800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:22.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.485+0000 7f471affd700 1 -- 172.21.15.31:0/3146660163 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4728011590 con 0x7f4734159850 2023-10-06T02:28:22.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.485+0000 7f471affd700 1 -- 172.21.15.31:0/3146660163 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4728011710 con 0x7f4734159850 2023-10-06T02:28:22.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.485+0000 7f471affd700 1 -- 172.21.15.31:0/3146660163 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4728011ad0 con 0x7f4734159850 2023-10-06T02:28:22.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.485+0000 7f47393ff700 1 -- 172.21.15.31:0/3146660163 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47340a96f0 con 0x7f4734159850 2023-10-06T02:28:22.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.486+0000 7f47393ff700 1 -- 172.21.15.31:0/3146660163 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47340a9c90 con 0x7f4734159850 2023-10-06T02:28:22.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.487+0000 7f47393ff700 1 -- 172.21.15.31:0/3146660163 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47340aa730 con 0x7f4734159850 2023-10-06T02:28:22.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.487+0000 7f471affd700 1 -- 172.21.15.31:0/3146660163 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f472801b3b0 con 0x7f4734159850 2023-10-06T02:28:22.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.489+0000 7f471affd700 1 --2- 172.21.15.31:0/3146660163 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f4720067050 0x7f4720069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:22.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.489+0000 7f471affd700 1 -- 172.21.15.31:0/3146660163 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f4728083d40 con 0x7f4734159850 2023-10-06T02:28:22.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.489+0000 7f4732ffd700 1 --2- 172.21.15.31:0/3146660163 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f4720067050 0x7f4720069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:22.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.490+0000 7f4732ffd700 1 --2- 172.21.15.31:0/3146660163 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f4720067050 0x7f4720069500 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f47240093f0 tx=0x7f4724017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:22.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.493+0000 7f471affd700 1 -- 172.21.15.31:0/3146660163 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4728050c70 con 0x7f4734159850 2023-10-06T02:28:22.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.745+0000 7f47393ff700 1 -- 172.21.15.31:0/3146660163 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f473404c3c0 con 0x7f4734159850 2023-10-06T02:28:22.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.745+0000 7f471affd700 1 -- 172.21.15.31:0/3146660163 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f4728017070 con 0x7f4734159850 2023-10-06T02:28:22.754 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:28:22.754 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":62,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5759,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/2470516084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2470516084},{"type":"v1","addr":"172.21.15.62:6834","nonce":2470516084}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":6364,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/790751965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":790751965},{"type":"v1","addr":"172.21.15.31:6834","nonce":790751965}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":6571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3862349969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":3862349969},{"type":"v1","addr":"172.21.15.62:6835","nonce":3862349969}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:27:58.019137+0000","modified":"2023-10-06T02:28:01.555185+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6714},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6714":{"gid":6714,"name":"c","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/1002908930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":1002908930},{"type":"v1","addr":"172.21.15.31:6835","nonce":1002908930}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T02:28:22.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.748+0000 7f47393ff700 1 -- 172.21.15.31:0/3146660163 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f4720067050 msgr2=0x7f4720069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:22.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.748+0000 7f47393ff700 1 --2- 172.21.15.31:0/3146660163 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f4720067050 0x7f4720069500 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f47240093f0 tx=0x7f4724017000 comp rx=0 tx=0).stop 2023-10-06T02:28:22.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.748+0000 7f47393ff700 1 -- 172.21.15.31:0/3146660163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4734159850 msgr2=0x7f47340a8de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:22.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.748+0000 7f47393ff700 1 --2- 172.21.15.31:0/3146660163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4734159850 0x7f47340a8de0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f472800c4e0 tx=0x7f472800c7f0 comp rx=0 tx=0).stop 2023-10-06T02:28:22.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.748+0000 7f47393ff700 1 -- 172.21.15.31:0/3146660163 shutdown_connections 2023-10-06T02:28:22.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.748+0000 7f47393ff700 1 --2- 172.21.15.31:0/3146660163 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f4720067050 0x7f4720069500 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:22.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.748+0000 7f47393ff700 1 --2- 172.21.15.31:0/3146660163 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4734159c20 0x7f47340c7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:22.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.748+0000 7f47393ff700 1 --2- 172.21.15.31:0/3146660163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47340b1040 0x7f4734145810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:22.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.748+0000 7f47393ff700 1 --2- 172.21.15.31:0/3146660163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4734159850 0x7f47340a8de0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:22.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.748+0000 7f47393ff700 1 -- 172.21.15.31:0/3146660163 >> 172.21.15.31:0/3146660163 conn(0x7f473406d170 msgr2=0x7f47340adfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:22.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.748+0000 7f47393ff700 1 -- 172.21.15.31:0/3146660163 shutdown_connections 2023-10-06T02:28:22.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.748+0000 7f47393ff700 1 -- 172.21.15.31:0/3146660163 wait complete. 2023-10-06T02:28:22.756 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 62 2023-10-06T02:28:22.768 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:28:22.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.953+0000 7f9372d9d700 1 Processor -- start 2023-10-06T02:28:22.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.954+0000 7f9372d9d700 1 -- start start 2023-10-06T02:28:22.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.955+0000 7f9372d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f936c0b2940 0x7f936c0a98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:22.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.955+0000 7f9372d9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f936c0cd9d0 0x7f936c06e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:22.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.955+0000 7f9372d9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f936c0b0110 con 0x7f936c0b2d10 2023-10-06T02:28:22.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.955+0000 7f9372d9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f936c0b0920 con 0x7f936c0cd9d0 2023-10-06T02:28:22.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.955+0000 7f9372d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f936c06cc70 con 0x7f936c0b2940 2023-10-06T02:28:22.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.955+0000 7f9371d9b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f936c0b2940 0x7f936c0a98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:22.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.956+0000 7f9371d9b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f936c0b2940 0x7f936c0a98e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:56056/0 (socket says 172.21.15.31:56056) 2023-10-06T02:28:22.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.956+0000 7f937259c700 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f936c0b2d10 0x7f936c159250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:42644/0 (socket says 172.21.15.31:42644) 2023-10-06T02:28:22.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.956+0000 7f9371d9b700 1 -- 172.21.15.31:0/2850520503 learned_addr learned my addr 172.21.15.31:0/2850520503 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:22.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.956+0000 7f937159a700 1 --2- 172.21.15.31:0/2850520503 >> v2:172.21.15.62:3300/0 conn(0x7f936c0cd9d0 0x7f936c06e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:22.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.956+0000 7f9371d9b700 1 -- 172.21.15.31:0/2850520503 >> v2:172.21.15.62:3300/0 conn(0x7f936c0cd9d0 msgr2=0x7f936c06e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:22.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.956+0000 7f9371d9b700 1 --2- 172.21.15.31:0/2850520503 >> v2:172.21.15.62:3300/0 conn(0x7f936c0cd9d0 0x7f936c06e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:22.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.956+0000 7f9371d9b700 1 -- 172.21.15.31:0/2850520503 >> v1:172.21.15.62:6789/0 conn(0x7f936c0b2d10 legacy=0x7f936c159250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:22.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.956+0000 7f9371d9b700 1 -- 172.21.15.31:0/2850520503 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f936c0b0360 con 0x7f936c0b2940 2023-10-06T02:28:22.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.956+0000 7f9370d99700 1 -- 172.21.15.31:0/2850520503 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4012222410 0 0) 0x7f936c0b0110 con 0x7f936c0b2d10 2023-10-06T02:28:22.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.956+0000 7f9371d9b700 1 --2- 172.21.15.31:0/2850520503 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f936c0b2940 0x7f936c0a98e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9368009990 tx=0x7f9368009220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b569ac7383f3f0d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:22.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.957+0000 7f9370d99700 1 -- 172.21.15.31:0/2850520503 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9368009e80 con 0x7f936c0b2940 2023-10-06T02:28:22.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.957+0000 7f9370d99700 1 -- 172.21.15.31:0/2850520503 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9368005c10 con 0x7f936c0b2940 2023-10-06T02:28:22.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.957+0000 7f9370d99700 1 -- 172.21.15.31:0/2850520503 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f936800b650 con 0x7f936c0b2940 2023-10-06T02:28:22.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.957+0000 7f9372d9d700 1 -- 172.21.15.31:0/2850520503 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f936c0b2940 msgr2=0x7f936c0a98e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:22.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.957+0000 7f9372d9d700 1 --2- 172.21.15.31:0/2850520503 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f936c0b2940 0x7f936c0a98e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9368009990 tx=0x7f9368009220 comp rx=0 tx=0).stop 2023-10-06T02:28:22.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.957+0000 7f9372d9d700 1 -- 172.21.15.31:0/2850520503 shutdown_connections 2023-10-06T02:28:22.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.957+0000 7f9372d9d700 1 --2- 172.21.15.31:0/2850520503 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f936c0b2940 0x7f936c0a98e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:22.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.957+0000 7f9372d9d700 1 --2- 172.21.15.31:0/2850520503 >> v2:172.21.15.62:3300/0 conn(0x7f936c0cd9d0 0x7f936c06e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:22.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.957+0000 7f9372d9d700 1 -- 172.21.15.31:0/2850520503 >> 172.21.15.31:0/2850520503 conn(0x7f936c0675a0 msgr2=0x7f936c0679a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:22.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.957+0000 7f9372d9d700 1 -- 172.21.15.31:0/2850520503 shutdown_connections 2023-10-06T02:28:22.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.957+0000 7f9372d9d700 1 -- 172.21.15.31:0/2850520503 wait complete. 2023-10-06T02:28:22.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.959+0000 7f9372d9d700 1 Processor -- start 2023-10-06T02:28:22.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.959+0000 7f9372d9d700 1 -- start start 2023-10-06T02:28:22.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.960+0000 7f9372d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f936c0b2940 0x7f936c0adff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:22.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.960+0000 7f9372d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f936c0b2d10 0x7f936c14cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:22.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.960+0000 7f9372d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f936c0cd9d0 0x7f936c14ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:22.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.960+0000 7f9372d9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f936c0ae530 con 0x7f936c0b2d10 2023-10-06T02:28:22.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.960+0000 7f9372d9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f936c0af840 con 0x7f936c0cd9d0 2023-10-06T02:28:22.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.960+0000 7f9372d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f936c14d4e0 con 0x7f936c0b2940 2023-10-06T02:28:22.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.960+0000 7f937159a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f936c0b2d10 0x7f936c14cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:22.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.960+0000 7f937159a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f936c0b2d10 0x7f936c14cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33064/0 (socket says 172.21.15.31:33064) 2023-10-06T02:28:22.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.960+0000 7f937159a700 1 -- 172.21.15.31:0/3776958486 learned_addr learned my addr 172.21.15.31:0/3776958486 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:22.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.960+0000 7f9371d9b700 1 --2- 172.21.15.31:0/3776958486 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f936c0b2940 0x7f936c0adff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:22.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.960+0000 7f937259c700 1 --2- 172.21.15.31:0/3776958486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f936c0cd9d0 0x7f936c14ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:22.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.961+0000 7f937159a700 1 -- 172.21.15.31:0/3776958486 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f936c0b2940 msgr2=0x7f936c0adff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:22.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.961+0000 7f937159a700 1 --2- 172.21.15.31:0/3776958486 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f936c0b2940 0x7f936c0adff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:22.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.961+0000 7f937159a700 1 -- 172.21.15.31:0/3776958486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f936c0cd9d0 msgr2=0x7f936c14ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:22.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.961+0000 7f937159a700 1 --2- 172.21.15.31:0/3776958486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f936c0cd9d0 0x7f936c14ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:22.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.961+0000 7f937159a700 1 -- 172.21.15.31:0/3776958486 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9368009480 con 0x7f936c0b2d10 2023-10-06T02:28:22.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.961+0000 7f9371d9b700 1 --2- 172.21.15.31:0/3776958486 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f936c0b2940 0x7f936c0adff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:28:22.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.961+0000 7f937159a700 1 --2- 172.21.15.31:0/3776958486 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f936c0b2d10 0x7f936c14cfa0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f935c00ecb0 tx=0x7f935c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:22.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.961+0000 7f9361ffb700 1 -- 172.21.15.31:0/3776958486 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f935c00c9d0 con 0x7f936c0b2d10 2023-10-06T02:28:22.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.961+0000 7f9361ffb700 1 -- 172.21.15.31:0/3776958486 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f935c00cb30 con 0x7f936c0b2d10 2023-10-06T02:28:22.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.961+0000 7f9361ffb700 1 -- 172.21.15.31:0/3776958486 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f935c00ced0 con 0x7f936c0b2d10 2023-10-06T02:28:22.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.961+0000 7f9372d9d700 1 -- 172.21.15.31:0/3776958486 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f936c0aec30 con 0x7f936c0b2d10 2023-10-06T02:28:22.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.961+0000 7f9372d9d700 1 -- 172.21.15.31:0/3776958486 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f936c0af1f0 con 0x7f936c0b2d10 2023-10-06T02:28:22.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.962+0000 7f9372d9d700 1 -- 172.21.15.31:0/3776958486 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f936c04c3c0 con 0x7f936c0b2d10 2023-10-06T02:28:22.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.963+0000 7f9361ffb700 1 -- 172.21.15.31:0/3776958486 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f935c013680 con 0x7f936c0b2d10 2023-10-06T02:28:22.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.964+0000 7f9361ffb700 1 --2- 172.21.15.31:0/3776958486 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9350067050 0x7f9350069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:22.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.964+0000 7f9361ffb700 1 -- 172.21.15.31:0/3776958486 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f935c084c30 con 0x7f936c0b2d10 2023-10-06T02:28:22.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.965+0000 7f9371d9b700 1 --2- 172.21.15.31:0/3776958486 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9350067050 0x7f9350069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:22.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.965+0000 7f9371d9b700 1 --2- 172.21.15.31:0/3776958486 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9350067050 0x7f9350069500 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9368009990 tx=0x7f9368016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:22.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:22.968+0000 7f9361ffb700 1 -- 172.21.15.31:0/3776958486 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f935c051b60 con 0x7f936c0b2d10 2023-10-06T02:28:23.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.174+0000 7f9372d9d700 1 -- 172.21.15.31:0/3776958486 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f936c0b0640 con 0x7f936c0b2d10 2023-10-06T02:28:23.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.174+0000 7f9361ffb700 1 -- 172.21.15.31:0/3776958486 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+16950 (secure 0 0 0) 0x7f935c055180 con 0x7f936c0b2d10 2023-10-06T02:28:23.176 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:28:23.176 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":95,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:28:17.469239+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:27:56.192975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-10-06T02:27:56.993140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6835/2757737386":"2023-10-07T02:28:14.456919+0000","172.21.15.62:6833/2757737386":"2023-10-07T02:28:14.456919+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:28:23.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.177+0000 7f9372d9d700 1 -- 172.21.15.31:0/3776958486 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9350067050 msgr2=0x7f9350069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:23.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.177+0000 7f9372d9d700 1 --2- 172.21.15.31:0/3776958486 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9350067050 0x7f9350069500 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9368009990 tx=0x7f9368016000 comp rx=0 tx=0).stop 2023-10-06T02:28:23.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.177+0000 7f9372d9d700 1 -- 172.21.15.31:0/3776958486 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f936c0b2d10 msgr2=0x7f936c14cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:23.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.177+0000 7f9372d9d700 1 --2- 172.21.15.31:0/3776958486 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f936c0b2d10 0x7f936c14cfa0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f935c00ecb0 tx=0x7f935c0091a0 comp rx=0 tx=0).stop 2023-10-06T02:28:23.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.177+0000 7f9372d9d700 1 -- 172.21.15.31:0/3776958486 shutdown_connections 2023-10-06T02:28:23.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.177+0000 7f9372d9d700 1 --2- 172.21.15.31:0/3776958486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f936c0cd9d0 0x7f936c14ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:23.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.177+0000 7f9372d9d700 1 --2- 172.21.15.31:0/3776958486 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9350067050 0x7f9350069500 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:23.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.177+0000 7f9372d9d700 1 --2- 172.21.15.31:0/3776958486 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f936c0b2940 0x7f936c0adff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:23.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.177+0000 7f9372d9d700 1 --2- 172.21.15.31:0/3776958486 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f936c0b2d10 0x7f936c14cfa0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:23.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.177+0000 7f9372d9d700 1 -- 172.21.15.31:0/3776958486 >> 172.21.15.31:0/3776958486 conn(0x7f936c0675a0 msgr2=0x7f936c0abc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:23.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.177+0000 7f9372d9d700 1 -- 172.21.15.31:0/3776958486 shutdown_connections 2023-10-06T02:28:23.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.177+0000 7f9372d9d700 1 -- 172.21.15.31:0/3776958486 wait complete. 2023-10-06T02:28:23.196 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T02:28:23.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.383+0000 7f591c99d700 1 Processor -- start 2023-10-06T02:28:23.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.383+0000 7f591c99d700 1 -- start start 2023-10-06T02:28:23.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.384+0000 7f591c99d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f591806f3a0 0x7f591814fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:23.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.384+0000 7f591c99d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f591806e470 0x7f59181501b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:23.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.384+0000 7f591c99d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f591806a490 con 0x7f591806e0a0 2023-10-06T02:28:23.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.384+0000 7f591c99d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59180c7230 con 0x7f591806e470 2023-10-06T02:28:23.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.384+0000 7f591c99d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f591806b930 con 0x7f591806f3a0 2023-10-06T02:28:23.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.384+0000 7f5917fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f591806e470 0x7f59181501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:23.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.384+0000 7f5917fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f591806e470 0x7f59181501b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33076/0 (socket says 172.21.15.31:33076) 2023-10-06T02:28:23.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.385+0000 7f5917fff700 1 -- 172.21.15.31:0/831600023 learned_addr learned my addr 172.21.15.31:0/831600023 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:23.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.385+0000 7f5916ffd700 1 --2- 172.21.15.31:0/831600023 >> v2:172.21.15.62:3300/0 conn(0x7f591806f3a0 0x7f591814fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:23.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.385+0000 7f5917fff700 1 -- 172.21.15.31:0/831600023 >> v2:172.21.15.62:3300/0 conn(0x7f591806f3a0 msgr2=0x7f591814fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:23.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.385+0000 7f5917fff700 1 --2- 172.21.15.31:0/831600023 >> v2:172.21.15.62:3300/0 conn(0x7f591806f3a0 0x7f591814fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:23.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.385+0000 7f5917fff700 1 -- 172.21.15.31:0/831600023 >> v1:172.21.15.62:6789/0 conn(0x7f591806e0a0 legacy=0x7f59180c6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:23.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.385+0000 7f5917fff700 1 -- 172.21.15.31:0/831600023 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59180c7490 con 0x7f591806e470 2023-10-06T02:28:23.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.385+0000 7f5917fff700 1 --2- 172.21.15.31:0/831600023 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f591806e470 0x7f59181501b0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f5918156140 tx=0x7f5910009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe2d5e6b3072246f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:23.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.386+0000 7f59167fc700 1 -- 172.21.15.31:0/831600023 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f591000e070 con 0x7f591806e470 2023-10-06T02:28:23.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.386+0000 7f59167fc700 1 -- 172.21.15.31:0/831600023 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f591000ba50 con 0x7f591806e470 2023-10-06T02:28:23.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.386+0000 7f59167fc700 1 -- 172.21.15.31:0/831600023 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59100092e0 con 0x7f591806e470 2023-10-06T02:28:23.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.386+0000 7f591c99d700 1 -- 172.21.15.31:0/831600023 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f591806e470 msgr2=0x7f59181501b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:23.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.386+0000 7f591c99d700 1 --2- 172.21.15.31:0/831600023 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f591806e470 0x7f59181501b0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f5918156140 tx=0x7f5910009840 comp rx=0 tx=0).stop 2023-10-06T02:28:23.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.386+0000 7f591c99d700 1 -- 172.21.15.31:0/831600023 shutdown_connections 2023-10-06T02:28:23.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.386+0000 7f591c99d700 1 --2- 172.21.15.31:0/831600023 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f591806e470 0x7f59181501b0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:23.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.386+0000 7f591c99d700 1 --2- 172.21.15.31:0/831600023 >> v2:172.21.15.62:3300/0 conn(0x7f591806f3a0 0x7f591814fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:23.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.386+0000 7f591c99d700 1 -- 172.21.15.31:0/831600023 >> 172.21.15.31:0/831600023 conn(0x7f591806d170 msgr2=0x7f59180cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:23.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.386+0000 7f591c99d700 1 -- 172.21.15.31:0/831600023 shutdown_connections 2023-10-06T02:28:23.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.386+0000 7f591c99d700 1 -- 172.21.15.31:0/831600023 wait complete. 2023-10-06T02:28:23.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.388+0000 7f591c99d700 1 Processor -- start 2023-10-06T02:28:23.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.388+0000 7f591c99d700 1 -- start start 2023-10-06T02:28:23.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.389+0000 7f591c99d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f591806e0a0 0x7f5918157160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:23.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.389+0000 7f591c99d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f591806e470 0x7f591814c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:23.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.389+0000 7f591c99d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f591806f3a0 0x7f59180c4220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:23.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.389+0000 7f591c99d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59180b0c50 con 0x7f591806f3a0 2023-10-06T02:28:23.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.389+0000 7f591c99d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59180b0040 con 0x7f591806e470 2023-10-06T02:28:23.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.389+0000 7f591c99d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59180b0320 con 0x7f591806e0a0 2023-10-06T02:28:23.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.389+0000 7f5917fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f591806f3a0 0x7f59180c4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:23.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.389+0000 7f5917fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f591806f3a0 0x7f59180c4220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33090/0 (socket says 172.21.15.31:33090) 2023-10-06T02:28:23.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.389+0000 7f5917fff700 1 -- 172.21.15.31:0/1059660465 learned_addr learned my addr 172.21.15.31:0/1059660465 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:23.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.389+0000 7f59177fe700 1 --2- 172.21.15.31:0/1059660465 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f591806e0a0 0x7f5918157160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:23.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.389+0000 7f5916ffd700 1 --2- 172.21.15.31:0/1059660465 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f591806e470 0x7f591814c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:23.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.389+0000 7f5917fff700 1 -- 172.21.15.31:0/1059660465 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f591806e0a0 msgr2=0x7f5918157160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:23.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.389+0000 7f5917fff700 1 --2- 172.21.15.31:0/1059660465 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f591806e0a0 0x7f5918157160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:23.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.389+0000 7f5917fff700 1 -- 172.21.15.31:0/1059660465 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f591806e470 msgr2=0x7f591814c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:23.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.389+0000 7f5917fff700 1 --2- 172.21.15.31:0/1059660465 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f591806e470 0x7f591814c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:23.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.389+0000 7f5917fff700 1 -- 172.21.15.31:0/1059660465 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5910009ae0 con 0x7f591806f3a0 2023-10-06T02:28:23.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.390+0000 7f5917fff700 1 --2- 172.21.15.31:0/1059660465 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f591806f3a0 0x7f59180c4220 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f5918071320 tx=0x7f5910018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:23.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.390+0000 7f58ff7fe700 1 -- 172.21.15.31:0/1059660465 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5910008e80 con 0x7f591806f3a0 2023-10-06T02:28:23.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.390+0000 7f58ff7fe700 1 -- 172.21.15.31:0/1059660465 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f591000be60 con 0x7f591806f3a0 2023-10-06T02:28:23.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.390+0000 7f58ff7fe700 1 -- 172.21.15.31:0/1059660465 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5910026690 con 0x7f591806f3a0 2023-10-06T02:28:23.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.390+0000 7f591c99d700 1 -- 172.21.15.31:0/1059660465 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59180b0600 con 0x7f591806f3a0 2023-10-06T02:28:23.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.390+0000 7f591c99d700 1 -- 172.21.15.31:0/1059660465 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59181576a0 con 0x7f591806f3a0 2023-10-06T02:28:23.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.391+0000 7f591c99d700 1 -- 172.21.15.31:0/1059660465 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59180b0e00 con 0x7f591806f3a0 2023-10-06T02:28:23.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.392+0000 7f58ff7fe700 1 -- 172.21.15.31:0/1059660465 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f5910003040 con 0x7f591806f3a0 2023-10-06T02:28:23.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.393+0000 7f58ff7fe700 1 --2- 172.21.15.31:0/1059660465 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5904067050 0x7f5904069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:23.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.393+0000 7f58ff7fe700 1 -- 172.21.15.31:0/1059660465 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f59100869b0 con 0x7f591806f3a0 2023-10-06T02:28:23.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.393+0000 7f59177fe700 1 --2- 172.21.15.31:0/1059660465 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5904067050 0x7f5904069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:23.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.394+0000 7f59177fe700 1 --2- 172.21.15.31:0/1059660465 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5904067050 0x7f5904069500 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5908003860 tx=0x7f5908009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:23.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.397+0000 7f58ff7fe700 1 -- 172.21.15.31:0/1059660465 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f59180b0e00 con 0x7f591806f3a0 2023-10-06T02:28:23.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.645+0000 7f591c99d700 1 -- 172.21.15.31:0/1059660465 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f591804c3c0 con 0x7f591806f3a0 2023-10-06T02:28:23.705 INFO:tasks.ceph.mon.a.smithi031.stderr:2023-10-06T02:28:23.645+0000 7f6de01f1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T02:28:23.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.650+0000 7f58ff7fe700 1 -- 172.21.15.31:0/1059660465 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v63) v1 ==== 161+0+0 (secure 0 0 0) 0x7f59100537d0 con 0x7f591806f3a0 2023-10-06T02:28:23.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.652+0000 7f591c99d700 1 -- 172.21.15.31:0/1059660465 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5904067050 msgr2=0x7f5904069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:23.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.652+0000 7f591c99d700 1 --2- 172.21.15.31:0/1059660465 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5904067050 0x7f5904069500 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5908003860 tx=0x7f5908009040 comp rx=0 tx=0).stop 2023-10-06T02:28:23.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.652+0000 7f591c99d700 1 -- 172.21.15.31:0/1059660465 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f591806f3a0 msgr2=0x7f59180c4220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:23.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.652+0000 7f591c99d700 1 --2- 172.21.15.31:0/1059660465 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f591806f3a0 0x7f59180c4220 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f5918071320 tx=0x7f5910018000 comp rx=0 tx=0).stop 2023-10-06T02:28:23.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.652+0000 7f591c99d700 1 -- 172.21.15.31:0/1059660465 shutdown_connections 2023-10-06T02:28:23.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.652+0000 7f591c99d700 1 --2- 172.21.15.31:0/1059660465 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f591806f3a0 0x7f59180c4220 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:23.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.652+0000 7f591c99d700 1 --2- 172.21.15.31:0/1059660465 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5904067050 0x7f5904069500 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:23.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.652+0000 7f591c99d700 1 --2- 172.21.15.31:0/1059660465 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f591806e0a0 0x7f5918157160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:23.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.652+0000 7f591c99d700 1 --2- 172.21.15.31:0/1059660465 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f591806e470 0x7f591814c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:23.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.652+0000 7f591c99d700 1 -- 172.21.15.31:0/1059660465 >> 172.21.15.31:0/1059660465 conn(0x7f591806d170 msgr2=0x7f5918154b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:23.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.653+0000 7f591c99d700 1 -- 172.21.15.31:0/1059660465 shutdown_connections 2023-10-06T02:28:23.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.653+0000 7f591c99d700 1 -- 172.21.15.31:0/1059660465 wait complete. 2023-10-06T02:28:23.707 INFO:teuthology.orchestra.run.smithi031.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T02:28:23.707 INFO:tasks.ceph.mds.c.smithi031.stderr: -1167> 2023-10-06T02:28:04.512+0000 7f97189f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T02:28:23.708 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T02:28:23.709 INFO:tasks.ceph.mds.c.smithi031.stdout:starting mds.c at 2023-10-06T02:28:23.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.896+0000 7fd8b7fff700 1 Processor -- start 2023-10-06T02:28:23.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.896+0000 7fd8b7fff700 1 -- start start 2023-10-06T02:28:23.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.898+0000 7fd8b7fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd8b8155c60 0x7fd8b80c7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:23.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.898+0000 7fd8b7fff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8b80c7e70 con 0x7fd8b806de50 2023-10-06T02:28:23.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.898+0000 7fd8b7fff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8b806b790 con 0x7fd8b8155890 2023-10-06T02:28:23.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.898+0000 7fd8b7fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b806ab80 con 0x7fd8b8155c60 2023-10-06T02:28:23.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.898+0000 7fd8b77fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fd8b806de50 0x7fd8b80c9d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:43758/0 (socket says 172.21.15.31:43758) 2023-10-06T02:28:23.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.898+0000 7fd8b77fe700 1 -- 172.21.15.31:0/3596182018 learned_addr learned my addr 172.21.15.31:0/3596182018 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:28:23.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.898+0000 7fd8b67fc700 1 --2- 172.21.15.31:0/3596182018 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd8b8155c60 0x7fd8b80c7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:23.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.898+0000 7fd8b5ffb700 1 -- 172.21.15.31:0/3596182018 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3181465832 0 0) 0x7fd8b80c7e70 con 0x7fd8b806de50 2023-10-06T02:28:23.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.898+0000 7fd8b5ffb700 1 -- 172.21.15.31:0/3596182018 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8a0003570 con 0x7fd8b806de50 2023-10-06T02:28:23.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.898+0000 7fd8b5ffb700 1 -- 172.21.15.31:0/3596182018 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1468216899 0 0) 0x7fd8b806b790 con 0x7fd8b8155890 2023-10-06T02:28:23.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.898+0000 7fd8b5ffb700 1 -- 172.21.15.31:0/3596182018 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8b80c7e70 con 0x7fd8b8155890 2023-10-06T02:28:23.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.899+0000 7fd8b67fc700 1 -- 172.21.15.31:0/3596182018 >> v1:172.21.15.62:6789/0 conn(0x7fd8b8155890 legacy=0x7fd8b806a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:23.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.899+0000 7fd8b67fc700 1 -- 172.21.15.31:0/3596182018 >> v1:172.21.15.31:6789/0 conn(0x7fd8b806de50 legacy=0x7fd8b80c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:23.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.899+0000 7fd8b67fc700 1 -- 172.21.15.31:0/3596182018 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8b806ae60 con 0x7fd8b8155c60 2023-10-06T02:28:23.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.899+0000 7fd8b67fc700 1 --2- 172.21.15.31:0/3596182018 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd8b8155c60 0x7fd8b80c7930 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac00aac0 tx=0x7fd8ac00a330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9509c63095407ff0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:23.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.899+0000 7fd8b5ffb700 1 -- 172.21.15.31:0/3596182018 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ac00c970 con 0x7fd8b8155c60 2023-10-06T02:28:23.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.899+0000 7fd8b5ffb700 1 -- 172.21.15.31:0/3596182018 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8ac00cad0 con 0x7fd8b8155c60 2023-10-06T02:28:23.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.899+0000 7fd8b5ffb700 1 -- 172.21.15.31:0/3596182018 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ac00ce70 con 0x7fd8b8155c60 2023-10-06T02:28:23.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.900+0000 7fd8b7fff700 1 -- 172.21.15.31:0/3596182018 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd8b8155c60 msgr2=0x7fd8b80c7930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:23.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.900+0000 7fd8b7fff700 1 --2- 172.21.15.31:0/3596182018 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd8b8155c60 0x7fd8b80c7930 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac00aac0 tx=0x7fd8ac00a330 comp rx=0 tx=0).stop 2023-10-06T02:28:23.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.900+0000 7fd8b7fff700 1 -- 172.21.15.31:0/3596182018 shutdown_connections 2023-10-06T02:28:23.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.900+0000 7fd8b7fff700 1 --2- 172.21.15.31:0/3596182018 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd8b8155c60 0x7fd8b80c7930 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:23.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.900+0000 7fd8b7fff700 1 -- 172.21.15.31:0/3596182018 >> 172.21.15.31:0/3596182018 conn(0x7fd8b806d110 msgr2=0x7fd8b80cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:23.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.900+0000 7fd8b7fff700 1 -- 172.21.15.31:0/3596182018 shutdown_connections 2023-10-06T02:28:23.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.900+0000 7fd8b7fff700 1 -- 172.21.15.31:0/3596182018 wait complete. 2023-10-06T02:28:23.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.901+0000 7fd8b7fff700 1 Processor -- start 2023-10-06T02:28:23.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.901+0000 7fd8b7fff700 1 -- start start 2023-10-06T02:28:23.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.902+0000 7fd8b7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8b806de50 0x7fd8b8149a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:23.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.902+0000 7fd8b7fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd8b8155890 0x7fd8b814b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:23.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.902+0000 7fd8b7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8b8155c60 0x7fd8b814bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:23.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.902+0000 7fd8b7fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b8149f50 con 0x7fd8b806de50 2023-10-06T02:28:23.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.902+0000 7fd8b7fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b814da50 con 0x7fd8b8155c60 2023-10-06T02:28:23.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.902+0000 7fd8b7fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b814ad40 con 0x7fd8b8155890 2023-10-06T02:28:23.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.903+0000 7fd8b6ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8b806de50 0x7fd8b8149a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:23.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.903+0000 7fd8b6ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8b806de50 0x7fd8b8149a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33114/0 (socket says 172.21.15.31:33114) 2023-10-06T02:28:23.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.903+0000 7fd8b6ffd700 1 -- 172.21.15.31:0/201085069 learned_addr learned my addr 172.21.15.31:0/201085069 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:23.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.903+0000 7fd8b77fe700 1 --2- 172.21.15.31:0/201085069 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8b8155c60 0x7fd8b814bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:23.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.903+0000 7fd8b67fc700 1 --2- 172.21.15.31:0/201085069 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd8b8155890 0x7fd8b814b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:23.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.903+0000 7fd8b6ffd700 1 -- 172.21.15.31:0/201085069 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd8b8155890 msgr2=0x7fd8b814b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:23.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.903+0000 7fd8b6ffd700 1 --2- 172.21.15.31:0/201085069 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd8b8155890 0x7fd8b814b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:23.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.903+0000 7fd8b6ffd700 1 -- 172.21.15.31:0/201085069 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8b8155c60 msgr2=0x7fd8b814bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:23.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.903+0000 7fd8b6ffd700 1 --2- 172.21.15.31:0/201085069 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8b8155c60 0x7fd8b814bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:23.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.903+0000 7fd8b6ffd700 1 -- 172.21.15.31:0/201085069 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8ac00a5d0 con 0x7fd8b806de50 2023-10-06T02:28:23.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.903+0000 7fd8b77fe700 1 --2- 172.21.15.31:0/201085069 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8b8155c60 0x7fd8b814bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:28:23.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.903+0000 7fd8b67fc700 1 --2- 172.21.15.31:0/201085069 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd8b8155890 0x7fd8b814b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:28:23.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.903+0000 7fd8b6ffd700 1 --2- 172.21.15.31:0/201085069 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8b806de50 0x7fd8b8149a10 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fd8a800c7c0 tx=0x7fd8a8007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:23.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.903+0000 7fd89effd700 1 -- 172.21.15.31:0/201085069 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a8011590 con 0x7fd8b806de50 2023-10-06T02:28:23.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.903+0000 7fd89effd700 1 -- 172.21.15.31:0/201085069 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8a8011710 con 0x7fd8b806de50 2023-10-06T02:28:23.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.904+0000 7fd89effd700 1 -- 172.21.15.31:0/201085069 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a8011ad0 con 0x7fd8b806de50 2023-10-06T02:28:23.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.904+0000 7fd8b7fff700 1 -- 172.21.15.31:0/201085069 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8b814a130 con 0x7fd8b806de50 2023-10-06T02:28:23.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.904+0000 7fd8b7fff700 1 -- 172.21.15.31:0/201085069 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8b814a6f0 con 0x7fd8b806de50 2023-10-06T02:28:23.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.905+0000 7fd8b7fff700 1 -- 172.21.15.31:0/201085069 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8b80c42e0 con 0x7fd8b806de50 2023-10-06T02:28:23.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.905+0000 7fd89effd700 1 -- 172.21.15.31:0/201085069 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7fd8a801c460 con 0x7fd8b806de50 2023-10-06T02:28:23.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.907+0000 7fd89effd700 1 --2- 172.21.15.31:0/201085069 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd8a4067050 0x7fd8a4069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:23.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.907+0000 7fd89effd700 1 -- 172.21.15.31:0/201085069 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6230+0+0 (secure 0 0 0) 0x7fd8a8084680 con 0x7fd8b806de50 2023-10-06T02:28:23.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.908+0000 7fd8b67fc700 1 --2- 172.21.15.31:0/201085069 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd8a4067050 0x7fd8a4069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:23.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.909+0000 7fd8b67fc700 1 --2- 172.21.15.31:0/201085069 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd8a4067050 0x7fd8a4069500 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac00aa10 tx=0x7fd8ac01a000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:23.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:23.911+0000 7fd89effd700 1 -- 172.21.15.31:0/201085069 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd8a8051530 con 0x7fd8b806de50 2023-10-06T02:28:24.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:24.162+0000 7fd8b7fff700 1 -- 172.21.15.31:0/201085069 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fd8b804c3c0 con 0x7fd8b806de50 2023-10-06T02:28:24.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:24.654+0000 7fd89effd700 1 -- 172.21.15.31:0/201085069 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v64) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd8a8054b50 con 0x7fd8b806de50 2023-10-06T02:28:24.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:24.656+0000 7fd8b7fff700 1 -- 172.21.15.31:0/201085069 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd8a4067050 msgr2=0x7fd8a4069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:24.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:24.656+0000 7fd8b7fff700 1 --2- 172.21.15.31:0/201085069 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd8a4067050 0x7fd8a4069500 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac00aa10 tx=0x7fd8ac01a000 comp rx=0 tx=0).stop 2023-10-06T02:28:24.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:24.657+0000 7fd8b7fff700 1 -- 172.21.15.31:0/201085069 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8b806de50 msgr2=0x7fd8b8149a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:24.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:24.657+0000 7fd8b7fff700 1 --2- 172.21.15.31:0/201085069 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8b806de50 0x7fd8b8149a10 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fd8a800c7c0 tx=0x7fd8a8007ea0 comp rx=0 tx=0).stop 2023-10-06T02:28:24.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:24.657+0000 7fd8b7fff700 1 -- 172.21.15.31:0/201085069 shutdown_connections 2023-10-06T02:28:24.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:24.657+0000 7fd8b7fff700 1 --2- 172.21.15.31:0/201085069 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8b8155c60 0x7fd8b814bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:24.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:24.657+0000 7fd8b7fff700 1 --2- 172.21.15.31:0/201085069 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8b806de50 0x7fd8b8149a10 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:24.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:24.657+0000 7fd8b7fff700 1 --2- 172.21.15.31:0/201085069 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd8a4067050 0x7fd8a4069500 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:24.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:24.657+0000 7fd8b7fff700 1 --2- 172.21.15.31:0/201085069 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd8b8155890 0x7fd8b814b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:24.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:24.657+0000 7fd8b7fff700 1 -- 172.21.15.31:0/201085069 >> 172.21.15.31:0/201085069 conn(0x7fd8b806d110 msgr2=0x7fd8b81581a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:24.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:24.657+0000 7fd8b7fff700 1 -- 172.21.15.31:0/201085069 shutdown_connections 2023-10-06T02:28:24.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:24.657+0000 7fd8b7fff700 1 -- 172.21.15.31:0/201085069 wait complete. 2023-10-06T02:28:24.677 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T02:28:24.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:24.866+0000 7f71b759e700 1 Processor -- start 2023-10-06T02:28:24.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:24.866+0000 7f71b759e700 1 -- start start 2023-10-06T02:28:24.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:24.867+0000 7f71b759e700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f71b8142ca0 0x7f71b815dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:24.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:24.867+0000 7f71b759e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f71b80cb450 0x7f71b80cbf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:24.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:24.867+0000 7f71b759e700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71b815d8d0 con 0x7f71b8143070 2023-10-06T02:28:24.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:24.867+0000 7f71b759e700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71b815d2f0 con 0x7f71b8142ca0 2023-10-06T02:28:24.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:24.867+0000 7f71b759e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71b80cc630 con 0x7f71b80cb450 2023-10-06T02:28:24.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:24.868+0000 7f71b5d9b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f71b8143070 0x7f71b80cad40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:43764/0 (socket says 172.21.15.31:43764) 2023-10-06T02:28:24.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:24.868+0000 7f71b5d9b700 1 -- 172.21.15.31:0/1834377432 learned_addr learned my addr 172.21.15.31:0/1834377432 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:28:24.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:24.868+0000 7f71b6d9d700 1 --2- 172.21.15.31:0/1834377432 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f71b80cb450 0x7f71b80cbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:24.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:24.868+0000 7f71b659c700 1 --2- 172.21.15.31:0/1834377432 >> v2:172.21.15.62:3300/0 conn(0x7f71b8142ca0 0x7f71b815dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:24.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:24.868+0000 7f71b559a700 1 -- 172.21.15.31:0/1834377432 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1041030243 0 0) 0x7f71b815d8d0 con 0x7f71b8143070 2023-10-06T02:28:24.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:24.868+0000 7f71b559a700 1 -- 172.21.15.31:0/1834377432 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71a0003570 con 0x7f71b8143070 2023-10-06T02:28:24.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:24.868+0000 7f71b6d9d700 1 -- 172.21.15.31:0/1834377432 >> v2:172.21.15.62:3300/0 conn(0x7f71b8142ca0 msgr2=0x7f71b815dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:24.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:24.868+0000 7f71b6d9d700 1 --2- 172.21.15.31:0/1834377432 >> v2:172.21.15.62:3300/0 conn(0x7f71b8142ca0 0x7f71b815dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:24.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:24.868+0000 7f71b6d9d700 1 -- 172.21.15.31:0/1834377432 >> v1:172.21.15.31:6789/0 conn(0x7f71b8143070 legacy=0x7f71b80cad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:24.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:24.868+0000 7f71b6d9d700 1 -- 172.21.15.31:0/1834377432 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71b815d560 con 0x7f71b80cb450 2023-10-06T02:28:24.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:24.869+0000 7f71b6d9d700 1 --2- 172.21.15.31:0/1834377432 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f71b80cb450 0x7f71b80cbf30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f71b0009e60 tx=0x7f71b0009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9f31316030a81e0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:24.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:24.869+0000 7f71b559a700 1 -- 172.21.15.31:0/1834377432 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b000e070 con 0x7f71b80cb450 2023-10-06T02:28:24.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:24.869+0000 7f71b559a700 1 -- 172.21.15.31:0/1834377432 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71b000bab0 con 0x7f71b80cb450 2023-10-06T02:28:24.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:24.869+0000 7f71b559a700 1 -- 172.21.15.31:0/1834377432 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b000be70 con 0x7f71b80cb450 2023-10-06T02:28:24.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:24.870+0000 7f71b759e700 1 -- 172.21.15.31:0/1834377432 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f71b80cb450 msgr2=0x7f71b80cbf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:24.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:24.870+0000 7f71b759e700 1 --2- 172.21.15.31:0/1834377432 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f71b80cb450 0x7f71b80cbf30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f71b0009e60 tx=0x7f71b0009780 comp rx=0 tx=0).stop 2023-10-06T02:28:24.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:24.870+0000 7f71b759e700 1 -- 172.21.15.31:0/1834377432 shutdown_connections 2023-10-06T02:28:24.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:24.870+0000 7f71b759e700 1 --2- 172.21.15.31:0/1834377432 >> v2:172.21.15.62:3300/0 conn(0x7f71b8142ca0 0x7f71b815dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:24.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:24.870+0000 7f71b759e700 1 --2- 172.21.15.31:0/1834377432 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f71b80cb450 0x7f71b80cbf30 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:24.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:24.870+0000 7f71b759e700 1 -- 172.21.15.31:0/1834377432 >> 172.21.15.31:0/1834377432 conn(0x7f71b8067c80 msgr2=0x7f71b8068080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:24.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:24.870+0000 7f71b759e700 1 -- 172.21.15.31:0/1834377432 shutdown_connections 2023-10-06T02:28:24.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:24.870+0000 7f71b759e700 1 -- 172.21.15.31:0/1834377432 wait complete. 2023-10-06T02:28:24.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:24.871+0000 7f71b759e700 1 Processor -- start 2023-10-06T02:28:24.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:24.871+0000 7f71b759e700 1 -- start start 2023-10-06T02:28:24.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:24.872+0000 7f71b759e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f71b80cb450 0x7f71b8140140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:24.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:24.872+0000 7f71b759e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71b8142ca0 0x7f71b80ace60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:24.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:24.872+0000 7f71b759e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71b8143070 0x7f71b80ad3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:24.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:24.872+0000 7f71b759e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b80b4180 con 0x7f71b8143070 2023-10-06T02:28:24.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:24.872+0000 7f71b759e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b80a9aa0 con 0x7f71b8142ca0 2023-10-06T02:28:24.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:24.872+0000 7f71b759e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71b80b38e0 con 0x7f71b80cb450 2023-10-06T02:28:24.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:24.872+0000 7f71b6d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71b8143070 0x7f71b80ad3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:24.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:24.873+0000 7f71b6d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71b8143070 0x7f71b80ad3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33130/0 (socket says 172.21.15.31:33130) 2023-10-06T02:28:24.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:24.873+0000 7f71b6d9d700 1 -- 172.21.15.31:0/4191119947 learned_addr learned my addr 172.21.15.31:0/4191119947 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:24.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:24.873+0000 7f71b659c700 1 --2- 172.21.15.31:0/4191119947 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f71b80cb450 0x7f71b8140140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:24.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:24.873+0000 7f71b5d9b700 1 --2- 172.21.15.31:0/4191119947 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71b8142ca0 0x7f71b80ace60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:24.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:24.873+0000 7f71b6d9d700 1 -- 172.21.15.31:0/4191119947 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f71b80cb450 msgr2=0x7f71b8140140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:24.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:24.873+0000 7f71b6d9d700 1 --2- 172.21.15.31:0/4191119947 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f71b80cb450 0x7f71b8140140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:24.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:24.873+0000 7f71b6d9d700 1 -- 172.21.15.31:0/4191119947 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71b8142ca0 msgr2=0x7f71b80ace60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:24.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:24.873+0000 7f71b6d9d700 1 --2- 172.21.15.31:0/4191119947 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71b8142ca0 0x7f71b80ace60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:24.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:24.873+0000 7f71b6d9d700 1 -- 172.21.15.31:0/4191119947 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71b0009a20 con 0x7f71b8143070 2023-10-06T02:28:24.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:24.873+0000 7f71b6d9d700 1 --2- 172.21.15.31:0/4191119947 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71b8143070 0x7f71b80ad3a0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f71b80cc770 tx=0x7f71b0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:24.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:24.874+0000 7f71a67fc700 1 -- 172.21.15.31:0/4191119947 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b0012e10 con 0x7f71b8143070 2023-10-06T02:28:24.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:24.874+0000 7f71a67fc700 1 -- 172.21.15.31:0/4191119947 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71b0027470 con 0x7f71b8143070 2023-10-06T02:28:24.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:24.874+0000 7f71a67fc700 1 -- 172.21.15.31:0/4191119947 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b0027830 con 0x7f71b8143070 2023-10-06T02:28:24.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:24.874+0000 7f71b759e700 1 -- 172.21.15.31:0/4191119947 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71b80b3bc0 con 0x7f71b8143070 2023-10-06T02:28:24.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:24.874+0000 7f71b759e700 1 -- 172.21.15.31:0/4191119947 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71b80b3590 con 0x7f71b8143070 2023-10-06T02:28:24.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:24.875+0000 7f71b759e700 1 -- 172.21.15.31:0/4191119947 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71b804c3c0 con 0x7f71b8143070 2023-10-06T02:28:24.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:24.876+0000 7f71a67fc700 1 -- 172.21.15.31:0/4191119947 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f71b0024070 con 0x7f71b8143070 2023-10-06T02:28:24.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:24.876+0000 7f71a67fc700 1 --2- 172.21.15.31:0/4191119947 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7198067050 0x7f7198069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:24.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:24.877+0000 7f71a67fc700 1 -- 172.21.15.31:0/4191119947 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6230+0+0 (secure 0 0 0) 0x7f71b0086f10 con 0x7f71b8143070 2023-10-06T02:28:24.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:24.877+0000 7f71b659c700 1 --2- 172.21.15.31:0/4191119947 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7198067050 0x7f7198069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:24.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:24.878+0000 7f71b659c700 1 --2- 172.21.15.31:0/4191119947 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7198067050 0x7f7198069500 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f71a8009db0 tx=0x7f71a80091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:24.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:24.881+0000 7f71a67fc700 1 -- 172.21.15.31:0/4191119947 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f71b0053dc0 con 0x7f71b8143070 2023-10-06T02:28:25.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:25.084+0000 7f71b759e700 1 -- 172.21.15.31:0/4191119947 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f71b80cb910 con 0x7f71b8143070 2023-10-06T02:28:26.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:26.048+0000 7f71a67fc700 1 -- 172.21.15.31:0/4191119947 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v97) v1 ==== 181+0+0 (secure 0 0 0) 0x7f71b002c0d0 con 0x7f71b8143070 2023-10-06T02:28:26.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:26.055+0000 7f71b759e700 1 -- 172.21.15.31:0/4191119947 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7198067050 msgr2=0x7f7198069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:26.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:26.055+0000 7f71b759e700 1 --2- 172.21.15.31:0/4191119947 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7198067050 0x7f7198069500 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f71a8009db0 tx=0x7f71a80091a0 comp rx=0 tx=0).stop 2023-10-06T02:28:26.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:26.056+0000 7f71b759e700 1 -- 172.21.15.31:0/4191119947 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71b8143070 msgr2=0x7f71b80ad3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:26.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:26.056+0000 7f71b759e700 1 --2- 172.21.15.31:0/4191119947 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71b8143070 0x7f71b80ad3a0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f71b80cc770 tx=0x7f71b0019000 comp rx=0 tx=0).stop 2023-10-06T02:28:26.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:26.056+0000 7f71b759e700 1 -- 172.21.15.31:0/4191119947 shutdown_connections 2023-10-06T02:28:26.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:26.056+0000 7f71b759e700 1 --2- 172.21.15.31:0/4191119947 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71b8143070 0x7f71b80ad3a0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:26.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:26.056+0000 7f71b759e700 1 --2- 172.21.15.31:0/4191119947 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7198067050 0x7f7198069500 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:26.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:26.056+0000 7f71b759e700 1 --2- 172.21.15.31:0/4191119947 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f71b80cb450 0x7f71b8140140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:26.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:26.056+0000 7f71b759e700 1 --2- 172.21.15.31:0/4191119947 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71b8142ca0 0x7f71b80ace60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:26.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:26.056+0000 7f71b759e700 1 -- 172.21.15.31:0/4191119947 >> 172.21.15.31:0/4191119947 conn(0x7f71b8067c80 msgr2=0x7f71b80cbb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:26.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:26.058+0000 7f71b759e700 1 -- 172.21.15.31:0/4191119947 shutdown_connections 2023-10-06T02:28:26.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:26.059+0000 7f71b759e700 1 -- 172.21.15.31:0/4191119947 wait complete. 2023-10-06T02:28:26.065 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' removed 2023-10-06T02:28:26.084 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T02:28:26.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:26.277+0000 7fe35d79d700 1 Processor -- start 2023-10-06T02:28:26.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:26.277+0000 7fe35d79d700 1 -- start start 2023-10-06T02:28:26.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:26.279+0000 7fe35d79d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe3580c8200 0x7fe3580c1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:26.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:26.279+0000 7fe35d79d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe3580c85d0 0x7fe3580cc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:26.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:26.279+0000 7fe35d79d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3581571b0 con 0x7fe3580c2a90 2023-10-06T02:28:26.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:26.279+0000 7fe35d79d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3581579b0 con 0x7fe3580c8200 2023-10-06T02:28:26.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:26.279+0000 7fe35d79d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3580c1390 con 0x7fe3580c85d0 2023-10-06T02:28:26.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:26.279+0000 7fe35cf9c700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe3580c85d0 0x7fe3580cc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:26.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:26.279+0000 7fe35cf9c700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe3580c85d0 0x7fe3580cc2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:56136/0 (socket says 172.21.15.31:56136) 2023-10-06T02:28:26.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:26.279+0000 7fe35cf9c700 1 -- 172.21.15.31:0/1202734213 learned_addr learned my addr 172.21.15.31:0/1202734213 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:26.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:26.279+0000 7fe3577fe700 1 --1- 172.21.15.31:0/1202734213 >> v1:172.21.15.62:6789/0 conn(0x7fe3580c2a90 0x7fe358156aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:42680/0 (socket says 172.21.15.31:42680) 2023-10-06T02:28:26.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:26.279+0000 7fe357fff700 1 --2- 172.21.15.31:0/1202734213 >> v2:172.21.15.62:3300/0 conn(0x7fe3580c8200 0x7fe3580c1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:26.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:26.280+0000 7fe356ffd700 1 -- 172.21.15.31:0/1202734213 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1856735960 0 0) 0x7fe34c001110 con 0x7fe3580c2a90 2023-10-06T02:28:26.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:26.280+0000 7fe356ffd700 1 -- 172.21.15.31:0/1202734213 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe340003570 con 0x7fe3580c2a90 2023-10-06T02:28:26.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:26.280+0000 7fe357fff700 1 -- 172.21.15.31:0/1202734213 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe3580c85d0 msgr2=0x7fe3580cc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:26.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:26.280+0000 7fe357fff700 1 --2- 172.21.15.31:0/1202734213 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe3580c85d0 0x7fe3580cc2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:26.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:26.280+0000 7fe357fff700 1 -- 172.21.15.31:0/1202734213 >> v1:172.21.15.62:6789/0 conn(0x7fe3580c2a90 legacy=0x7fe358156aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:26.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:26.280+0000 7fe357fff700 1 -- 172.21.15.31:0/1202734213 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3581573f0 con 0x7fe3580c8200 2023-10-06T02:28:26.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:26.280+0000 7fe35cf9c700 1 --2- 172.21.15.31:0/1202734213 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe3580c85d0 0x7fe3580cc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:28:26.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:26.280+0000 7fe357fff700 1 --2- 172.21.15.31:0/1202734213 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3580c8200 0x7fe3580c1820 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fe34800be60 tx=0x7fe34800b780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4cdadf5a37c43d3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:26.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:26.280+0000 7fe356ffd700 1 -- 172.21.15.31:0/1202734213 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe348010070 con 0x7fe3580c8200 2023-10-06T02:28:26.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:26.281+0000 7fe356ffd700 1 -- 172.21.15.31:0/1202734213 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe34800dab0 con 0x7fe3580c8200 2023-10-06T02:28:26.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:26.281+0000 7fe356ffd700 1 -- 172.21.15.31:0/1202734213 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34800de70 con 0x7fe3580c8200 2023-10-06T02:28:26.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:26.281+0000 7fe35d79d700 1 -- 172.21.15.31:0/1202734213 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3580c8200 msgr2=0x7fe3580c1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:26.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:26.281+0000 7fe35d79d700 1 --2- 172.21.15.31:0/1202734213 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3580c8200 0x7fe3580c1820 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fe34800be60 tx=0x7fe34800b780 comp rx=0 tx=0).stop 2023-10-06T02:28:26.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:26.281+0000 7fe35d79d700 1 -- 172.21.15.31:0/1202734213 shutdown_connections 2023-10-06T02:28:26.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:26.281+0000 7fe35d79d700 1 --2- 172.21.15.31:0/1202734213 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe3580c85d0 0x7fe3580cc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:26.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:26.281+0000 7fe35d79d700 1 --2- 172.21.15.31:0/1202734213 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3580c8200 0x7fe3580c1820 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:26.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:26.281+0000 7fe35d79d700 1 -- 172.21.15.31:0/1202734213 >> 172.21.15.31:0/1202734213 conn(0x7fe35806c6a0 msgr2=0x7fe35806caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:26.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:26.281+0000 7fe35d79d700 1 -- 172.21.15.31:0/1202734213 shutdown_connections 2023-10-06T02:28:26.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:26.281+0000 7fe35d79d700 1 -- 172.21.15.31:0/1202734213 wait complete. 2023-10-06T02:28:26.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:26.283+0000 7fe35d79d700 1 Processor -- start 2023-10-06T02:28:26.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:26.283+0000 7fe35d79d700 1 -- start start 2023-10-06T02:28:26.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:26.284+0000 7fe35d79d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3580c2a90 0x7fe3580b01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:26.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:26.284+0000 7fe35d79d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3580c8200 0x7fe3580ca050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:26.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:26.284+0000 7fe35d79d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe3580c85d0 0x7fe35813eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:26.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:26.284+0000 7fe35d79d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3580cb470 con 0x7fe3580c2a90 2023-10-06T02:28:26.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:26.284+0000 7fe35d79d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3580ca590 con 0x7fe3580c8200 2023-10-06T02:28:26.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:26.284+0000 7fe35d79d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3580ca860 con 0x7fe3580c85d0 2023-10-06T02:28:26.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:26.284+0000 7fe357fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3580c2a90 0x7fe3580b01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:26.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:26.284+0000 7fe357fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3580c2a90 0x7fe3580b01a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33134/0 (socket says 172.21.15.31:33134) 2023-10-06T02:28:26.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:26.284+0000 7fe357fff700 1 -- 172.21.15.31:0/4284708317 learned_addr learned my addr 172.21.15.31:0/4284708317 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:26.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:26.284+0000 7fe3577fe700 1 --2- 172.21.15.31:0/4284708317 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3580c8200 0x7fe3580ca050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:26.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:26.284+0000 7fe35cf9c700 1 --2- 172.21.15.31:0/4284708317 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe3580c85d0 0x7fe35813eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:26.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:26.284+0000 7fe357fff700 1 -- 172.21.15.31:0/4284708317 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe3580c85d0 msgr2=0x7fe35813eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:26.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:26.284+0000 7fe357fff700 1 --2- 172.21.15.31:0/4284708317 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe3580c85d0 0x7fe35813eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:26.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:26.284+0000 7fe357fff700 1 -- 172.21.15.31:0/4284708317 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3580c8200 msgr2=0x7fe3580ca050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:26.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:26.284+0000 7fe357fff700 1 --2- 172.21.15.31:0/4284708317 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3580c8200 0x7fe3580ca050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:26.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:26.284+0000 7fe357fff700 1 -- 172.21.15.31:0/4284708317 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe350009ae0 con 0x7fe3580c2a90 2023-10-06T02:28:26.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:26.285+0000 7fe357fff700 1 --2- 172.21.15.31:0/4284708317 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3580c2a90 0x7fe3580b01a0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fe3580697c0 tx=0x7fe34801b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:26.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:26.285+0000 7fe33ffff700 1 -- 172.21.15.31:0/4284708317 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe348014850 con 0x7fe3580c2a90 2023-10-06T02:28:26.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:26.285+0000 7fe33ffff700 1 -- 172.21.15.31:0/4284708317 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3480149b0 con 0x7fe3580c2a90 2023-10-06T02:28:26.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:26.285+0000 7fe33ffff700 1 -- 172.21.15.31:0/4284708317 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe348014d70 con 0x7fe3580c2a90 2023-10-06T02:28:26.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:26.285+0000 7fe35d79d700 1 -- 172.21.15.31:0/4284708317 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe34800ba20 con 0x7fe3580c2a90 2023-10-06T02:28:26.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:26.285+0000 7fe35d79d700 1 -- 172.21.15.31:0/4284708317 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3580cae20 con 0x7fe3580c2a90 2023-10-06T02:28:26.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:26.286+0000 7fe35d79d700 1 -- 172.21.15.31:0/4284708317 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe35804c3c0 con 0x7fe3580c2a90 2023-10-06T02:28:26.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:26.287+0000 7fe33ffff700 1 -- 172.21.15.31:0/4284708317 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7fe34801d630 con 0x7fe3580c2a90 2023-10-06T02:28:26.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:26.288+0000 7fe33ffff700 1 --2- 172.21.15.31:0/4284708317 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe344067050 0x7fe344069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:26.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:26.288+0000 7fe33ffff700 1 -- 172.21.15.31:0/4284708317 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5792+0+0 (secure 0 0 0) 0x7fe348086160 con 0x7fe3580c2a90 2023-10-06T02:28:26.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:26.288+0000 7fe3577fe700 1 --2- 172.21.15.31:0/4284708317 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe344067050 0x7fe344069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:26.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:26.289+0000 7fe3577fe700 1 --2- 172.21.15.31:0/4284708317 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe344067050 0x7fe344069500 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe35806f390 tx=0x7fe34c007a10 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:26.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:26.292+0000 7fe33ffff700 1 -- 172.21.15.31:0/4284708317 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe3480531f0 con 0x7fe3580c2a90 2023-10-06T02:28:26.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:26.497+0000 7fe35d79d700 1 -- 172.21.15.31:0/4284708317 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe358150840 con 0x7fe3580c2a90 2023-10-06T02:28:27.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.055+0000 7fe33ffff700 1 -- 172.21.15.31:0/4284708317 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v98) v1 ==== 169+0+0 (secure 0 0 0) 0x7fe348056810 con 0x7fe3580c2a90 2023-10-06T02:28:27.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.059+0000 7fe35d79d700 1 -- 172.21.15.31:0/4284708317 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe344067050 msgr2=0x7fe344069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:27.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.059+0000 7fe35d79d700 1 --2- 172.21.15.31:0/4284708317 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe344067050 0x7fe344069500 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe35806f390 tx=0x7fe34c007a10 comp rx=0 tx=0).stop 2023-10-06T02:28:27.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.059+0000 7fe35d79d700 1 -- 172.21.15.31:0/4284708317 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3580c2a90 msgr2=0x7fe3580b01a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:27.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.059+0000 7fe35d79d700 1 --2- 172.21.15.31:0/4284708317 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3580c2a90 0x7fe3580b01a0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fe3580697c0 tx=0x7fe34801b000 comp rx=0 tx=0).stop 2023-10-06T02:28:27.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.060+0000 7fe35d79d700 1 -- 172.21.15.31:0/4284708317 shutdown_connections 2023-10-06T02:28:27.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.060+0000 7fe35d79d700 1 --2- 172.21.15.31:0/4284708317 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe344067050 0x7fe344069500 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:27.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.060+0000 7fe35d79d700 1 --2- 172.21.15.31:0/4284708317 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe3580c85d0 0x7fe35813eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:27.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.060+0000 7fe35d79d700 1 --2- 172.21.15.31:0/4284708317 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3580c2a90 0x7fe3580b01a0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:27.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.060+0000 7fe35d79d700 1 --2- 172.21.15.31:0/4284708317 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3580c8200 0x7fe3580ca050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:27.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.060+0000 7fe35d79d700 1 -- 172.21.15.31:0/4284708317 >> 172.21.15.31:0/4284708317 conn(0x7fe35806c6a0 msgr2=0x7fe35814fc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:27.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.060+0000 7fe35d79d700 1 -- 172.21.15.31:0/4284708317 shutdown_connections 2023-10-06T02:28:27.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.060+0000 7fe35d79d700 1 -- 172.21.15.31:0/4284708317 wait complete. 2023-10-06T02:28:27.063 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' removed 2023-10-06T02:28:27.082 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T02:28:27.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.273+0000 7f959359e700 1 Processor -- start 2023-10-06T02:28:27.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.274+0000 7f959359e700 1 -- start start 2023-10-06T02:28:27.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.275+0000 7f959359e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9594155c60 0x7f95940c7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:27.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.275+0000 7f959359e700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95940c7e70 con 0x7f9594155890 2023-10-06T02:28:27.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.275+0000 7f959359e700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f959406b790 con 0x7f959406de50 2023-10-06T02:28:27.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.275+0000 7f959359e700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f959406ab80 con 0x7f9594155c60 2023-10-06T02:28:27.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.275+0000 7f959259c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f9594155890 0x7f959406a3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:43770/0 (socket says 172.21.15.31:43770) 2023-10-06T02:28:27.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.275+0000 7f959259c700 1 -- 172.21.15.31:0/1276413823 learned_addr learned my addr 172.21.15.31:0/1276413823 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:28:27.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.275+0000 7f9591d9b700 1 --2- 172.21.15.31:0/1276413823 >> v2:172.21.15.31:3300/0 conn(0x7f9594155c60 0x7f95940c7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:27.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.276+0000 7f959159a700 1 -- 172.21.15.31:0/1276413823 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3945124719 0 0) 0x7f95940c7e70 con 0x7f9594155890 2023-10-06T02:28:27.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.276+0000 7f959159a700 1 -- 172.21.15.31:0/1276413823 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f957c003570 con 0x7f9594155890 2023-10-06T02:28:27.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.276+0000 7f9591d9b700 1 -- 172.21.15.31:0/1276413823 >> v1:172.21.15.62:6789/0 conn(0x7f959406de50 legacy=0x7f95940c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:27.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.276+0000 7f9591d9b700 1 -- 172.21.15.31:0/1276413823 >> v1:172.21.15.31:6789/0 conn(0x7f9594155890 legacy=0x7f959406a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:27.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.276+0000 7f9591d9b700 1 -- 172.21.15.31:0/1276413823 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f959406ae60 con 0x7f9594155c60 2023-10-06T02:28:27.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.276+0000 7f9591d9b700 1 --2- 172.21.15.31:0/1276413823 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9594155c60 0x7f95940c7930 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f958800aa90 tx=0x7f958800a330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d088cef94978e704 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:27.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.276+0000 7f959159a700 1 -- 172.21.15.31:0/1276413823 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f958800c970 con 0x7f9594155c60 2023-10-06T02:28:27.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.276+0000 7f959159a700 1 -- 172.21.15.31:0/1276413823 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f958800cad0 con 0x7f9594155c60 2023-10-06T02:28:27.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.276+0000 7f959159a700 1 -- 172.21.15.31:0/1276413823 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f958800ce50 con 0x7f9594155c60 2023-10-06T02:28:27.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.277+0000 7f959359e700 1 -- 172.21.15.31:0/1276413823 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9594155c60 msgr2=0x7f95940c7930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:27.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.277+0000 7f959359e700 1 --2- 172.21.15.31:0/1276413823 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9594155c60 0x7f95940c7930 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f958800aa90 tx=0x7f958800a330 comp rx=0 tx=0).stop 2023-10-06T02:28:27.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.277+0000 7f959359e700 1 -- 172.21.15.31:0/1276413823 shutdown_connections 2023-10-06T02:28:27.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.277+0000 7f959359e700 1 --2- 172.21.15.31:0/1276413823 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9594155c60 0x7f95940c7930 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:27.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.277+0000 7f959359e700 1 -- 172.21.15.31:0/1276413823 >> 172.21.15.31:0/1276413823 conn(0x7f959406d110 msgr2=0x7f95940cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:27.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.277+0000 7f959359e700 1 -- 172.21.15.31:0/1276413823 shutdown_connections 2023-10-06T02:28:27.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.277+0000 7f959359e700 1 -- 172.21.15.31:0/1276413823 wait complete. 2023-10-06T02:28:27.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.278+0000 7f959359e700 1 Processor -- start 2023-10-06T02:28:27.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.279+0000 7f959359e700 1 -- start start 2023-10-06T02:28:27.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.279+0000 7f959359e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f959406de50 0x7f95940c5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:27.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.279+0000 7f959359e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9594155890 0x7f95940c3330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:27.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.280+0000 7f959359e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9594155c60 0x7f959413fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:27.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.280+0000 7f959359e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95940c5ec0 con 0x7f9594155c60 2023-10-06T02:28:27.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.280+0000 7f959359e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95940c61a0 con 0x7f959406de50 2023-10-06T02:28:27.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.280+0000 7f959359e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95940c6480 con 0x7f9594155890 2023-10-06T02:28:27.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.280+0000 7f9592d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9594155c60 0x7f959413fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:27.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.280+0000 7f9592d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9594155c60 0x7f959413fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33140/0 (socket says 172.21.15.31:33140) 2023-10-06T02:28:27.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.280+0000 7f9592d9d700 1 -- 172.21.15.31:0/3014282370 learned_addr learned my addr 172.21.15.31:0/3014282370 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:27.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.280+0000 7f959259c700 1 --2- 172.21.15.31:0/3014282370 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f959406de50 0x7f95940c5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:27.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.280+0000 7f9591d9b700 1 --2- 172.21.15.31:0/3014282370 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9594155890 0x7f95940c3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:27.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.280+0000 7f9592d9d700 1 -- 172.21.15.31:0/3014282370 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9594155890 msgr2=0x7f95940c3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:27.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.280+0000 7f9592d9d700 1 --2- 172.21.15.31:0/3014282370 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9594155890 0x7f95940c3330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:27.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.280+0000 7f9592d9d700 1 -- 172.21.15.31:0/3014282370 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f959406de50 msgr2=0x7f95940c5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:27.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.280+0000 7f9592d9d700 1 --2- 172.21.15.31:0/3014282370 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f959406de50 0x7f95940c5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:27.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.280+0000 7f9592d9d700 1 -- 172.21.15.31:0/3014282370 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f958800a5d0 con 0x7f9594155c60 2023-10-06T02:28:27.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.280+0000 7f9591d9b700 1 --2- 172.21.15.31:0/3014282370 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9594155890 0x7f95940c3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:28:27.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.281+0000 7f9592d9d700 1 --2- 172.21.15.31:0/3014282370 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9594155c60 0x7f959413fa60 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f958c00c8e0 tx=0x7f958c001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:27.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.281+0000 7f95827fc700 1 -- 172.21.15.31:0/3014282370 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f958c012630 con 0x7f9594155c60 2023-10-06T02:28:27.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.281+0000 7f95827fc700 1 -- 172.21.15.31:0/3014282370 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f958c012790 con 0x7f9594155c60 2023-10-06T02:28:27.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.281+0000 7f959359e700 1 -- 172.21.15.31:0/3014282370 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95940c6760 con 0x7f9594155c60 2023-10-06T02:28:27.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.281+0000 7f95827fc700 1 -- 172.21.15.31:0/3014282370 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f958c012b50 con 0x7f9594155c60 2023-10-06T02:28:27.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.281+0000 7f959359e700 1 -- 172.21.15.31:0/3014282370 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9594140160 con 0x7f9594155c60 2023-10-06T02:28:27.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.282+0000 7f959359e700 1 -- 172.21.15.31:0/3014282370 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f959404c3c0 con 0x7f9594155c60 2023-10-06T02:28:27.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.283+0000 7f95827fc700 1 -- 172.21.15.31:0/3014282370 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f958c00a3b0 con 0x7f9594155c60 2023-10-06T02:28:27.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.284+0000 7f95827fc700 1 --2- 172.21.15.31:0/3014282370 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9574067050 0x7f9574069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:27.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.284+0000 7f95827fc700 1 -- 172.21.15.31:0/3014282370 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5378+0+0 (secure 0 0 0) 0x7f958c084900 con 0x7f9594155c60 2023-10-06T02:28:27.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.284+0000 7f959259c700 1 --2- 172.21.15.31:0/3014282370 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9574067050 0x7f9574069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:27.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.285+0000 7f959259c700 1 --2- 172.21.15.31:0/3014282370 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9574067050 0x7f9574069500 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9584007fd0 tx=0x7f9584009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:27.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.288+0000 7f95827fc700 1 -- 172.21.15.31:0/3014282370 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f958c051b00 con 0x7f9594155c60 2023-10-06T02:28:27.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.493+0000 7f959359e700 1 -- 172.21.15.31:0/3014282370 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f959406f170 con 0x7f9594155c60 2023-10-06T02:28:27.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.498+0000 7f95827fc700 1 -- 172.21.15.31:0/3014282370 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7f958c055120 con 0x7f9594155c60 2023-10-06T02:28:27.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.500+0000 7f959359e700 1 -- 172.21.15.31:0/3014282370 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9574067050 msgr2=0x7f9574069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:27.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.500+0000 7f959359e700 1 --2- 172.21.15.31:0/3014282370 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9574067050 0x7f9574069500 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9584007fd0 tx=0x7f9584009040 comp rx=0 tx=0).stop 2023-10-06T02:28:27.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.500+0000 7f959359e700 1 -- 172.21.15.31:0/3014282370 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9594155c60 msgr2=0x7f959413fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:27.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.500+0000 7f959359e700 1 --2- 172.21.15.31:0/3014282370 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9594155c60 0x7f959413fa60 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f958c00c8e0 tx=0x7f958c001710 comp rx=0 tx=0).stop 2023-10-06T02:28:27.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.500+0000 7f959359e700 1 -- 172.21.15.31:0/3014282370 shutdown_connections 2023-10-06T02:28:27.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.500+0000 7f959359e700 1 --2- 172.21.15.31:0/3014282370 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9594155c60 0x7f959413fa60 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:27.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.500+0000 7f959359e700 1 --2- 172.21.15.31:0/3014282370 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f959406de50 0x7f95940c5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:27.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.500+0000 7f959359e700 1 --2- 172.21.15.31:0/3014282370 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9574067050 0x7f9574069500 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:27.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.500+0000 7f959359e700 1 --2- 172.21.15.31:0/3014282370 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9594155890 0x7f95940c3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:27.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.500+0000 7f959359e700 1 -- 172.21.15.31:0/3014282370 >> 172.21.15.31:0/3014282370 conn(0x7f959406d110 msgr2=0x7f9594156cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:27.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.500+0000 7f959359e700 1 -- 172.21.15.31:0/3014282370 shutdown_connections 2023-10-06T02:28:27.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.500+0000 7f959359e700 1 -- 172.21.15.31:0/3014282370 wait complete. 2023-10-06T02:28:27.519 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars' 2023-10-06T02:28:27.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.706+0000 7fa96099d700 1 Processor -- start 2023-10-06T02:28:27.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.706+0000 7fa96099d700 1 -- start start 2023-10-06T02:28:27.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.707+0000 7fa96099d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa95c15de50 0x7fa95c0cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:27.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.707+0000 7fa96099d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa95c0cef40 0x7fa95c15c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:27.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.707+0000 7fa96099d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa95c0cf3c0 con 0x7fa95c153c30 2023-10-06T02:28:27.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.707+0000 7fa96099d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa95c15afd0 con 0x7fa95c15de50 2023-10-06T02:28:27.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.707+0000 7fa96099d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa95c15b2b0 con 0x7fa95c0cef40 2023-10-06T02:28:27.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.708+0000 7fa95affd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa95c15de50 0x7fa95c0cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:27.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.708+0000 7fa95affd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa95c15de50 0x7fa95c0cea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33154/0 (socket says 172.21.15.31:33154) 2023-10-06T02:28:27.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.708+0000 7fa95affd700 1 -- 172.21.15.31:0/470241788 learned_addr learned my addr 172.21.15.31:0/470241788 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:27.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.708+0000 7fa95bfff700 1 --2- 172.21.15.31:0/470241788 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa95c0cef40 0x7fa95c15c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:27.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.708+0000 7fa95affd700 1 -- 172.21.15.31:0/470241788 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa95c0cef40 msgr2=0x7fa95c15c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:27.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.708+0000 7fa95affd700 1 --2- 172.21.15.31:0/470241788 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa95c0cef40 0x7fa95c15c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:27.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.708+0000 7fa95affd700 1 -- 172.21.15.31:0/470241788 >> v1:172.21.15.62:6789/0 conn(0x7fa95c153c30 legacy=0x7fa95c15a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:27.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.708+0000 7fa95affd700 1 -- 172.21.15.31:0/470241788 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa95c15b590 con 0x7fa95c15de50 2023-10-06T02:28:27.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.708+0000 7fa95affd700 1 --2- 172.21.15.31:0/470241788 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa95c15de50 0x7fa95c0cea00 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fa9500099b0 tx=0x7fa950009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=651055d912a88ba3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:27.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.709+0000 7fa95a7fc700 1 -- 172.21.15.31:0/470241788 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95000b860 con 0x7fa95c15de50 2023-10-06T02:28:27.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.709+0000 7fa95a7fc700 1 -- 172.21.15.31:0/470241788 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa95000b9c0 con 0x7fa95c15de50 2023-10-06T02:28:27.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.709+0000 7fa95a7fc700 1 -- 172.21.15.31:0/470241788 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95000bd60 con 0x7fa95c15de50 2023-10-06T02:28:27.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.709+0000 7fa96099d700 1 -- 172.21.15.31:0/470241788 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa95c15de50 msgr2=0x7fa95c0cea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:27.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.709+0000 7fa96099d700 1 --2- 172.21.15.31:0/470241788 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa95c15de50 0x7fa95c0cea00 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fa9500099b0 tx=0x7fa950009250 comp rx=0 tx=0).stop 2023-10-06T02:28:27.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.709+0000 7fa96099d700 1 -- 172.21.15.31:0/470241788 shutdown_connections 2023-10-06T02:28:27.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.709+0000 7fa96099d700 1 --2- 172.21.15.31:0/470241788 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa95c0cef40 0x7fa95c15c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:27.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.709+0000 7fa96099d700 1 --2- 172.21.15.31:0/470241788 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa95c15de50 0x7fa95c0cea00 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:27.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.709+0000 7fa96099d700 1 -- 172.21.15.31:0/470241788 >> 172.21.15.31:0/470241788 conn(0x7fa95c0703a0 msgr2=0x7fa95c0707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:27.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.709+0000 7fa96099d700 1 -- 172.21.15.31:0/470241788 shutdown_connections 2023-10-06T02:28:27.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.709+0000 7fa96099d700 1 -- 172.21.15.31:0/470241788 wait complete. 2023-10-06T02:28:27.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.711+0000 7fa96099d700 1 Processor -- start 2023-10-06T02:28:27.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.711+0000 7fa96099d700 1 -- start start 2023-10-06T02:28:27.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.712+0000 7fa96099d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa95c0cef40 0x7fa95c144f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:27.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.712+0000 7fa96099d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa95c153c30 0x7fa95c0b5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:27.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.712+0000 7fa96099d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa95c15de50 0x7fa95c0b0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:27.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.712+0000 7fa96099d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa95c0c75e0 con 0x7fa95c0cef40 2023-10-06T02:28:27.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.712+0000 7fa96099d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa95c0c8400 con 0x7fa95c15de50 2023-10-06T02:28:27.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.712+0000 7fa96099d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa95c0c77f0 con 0x7fa95c153c30 2023-10-06T02:28:27.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.712+0000 7fa95b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa95c0cef40 0x7fa95c144f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:27.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.712+0000 7fa95b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa95c0cef40 0x7fa95c144f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33164/0 (socket says 172.21.15.31:33164) 2023-10-06T02:28:27.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.712+0000 7fa95b7fe700 1 -- 172.21.15.31:0/472656575 learned_addr learned my addr 172.21.15.31:0/472656575 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:27.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.712+0000 7fa95bfff700 1 --2- 172.21.15.31:0/472656575 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa95c15de50 0x7fa95c0b0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:27.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.712+0000 7fa95affd700 1 --2- 172.21.15.31:0/472656575 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa95c153c30 0x7fa95c0b5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:27.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.713+0000 7fa95b7fe700 1 -- 172.21.15.31:0/472656575 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa95c153c30 msgr2=0x7fa95c0b5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:27.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.713+0000 7fa95b7fe700 1 --2- 172.21.15.31:0/472656575 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa95c153c30 0x7fa95c0b5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:27.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.713+0000 7fa95b7fe700 1 -- 172.21.15.31:0/472656575 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa95c15de50 msgr2=0x7fa95c0b0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:27.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.713+0000 7fa95b7fe700 1 --2- 172.21.15.31:0/472656575 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa95c15de50 0x7fa95c0b0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:27.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.713+0000 7fa95b7fe700 1 -- 172.21.15.31:0/472656575 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9500094f0 con 0x7fa95c0cef40 2023-10-06T02:28:27.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.713+0000 7fa95b7fe700 1 --2- 172.21.15.31:0/472656575 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa95c0cef40 0x7fa95c144f10 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fa94c00c590 tx=0x7fa94c00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:27.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.713+0000 7fa9437fe700 1 -- 172.21.15.31:0/472656575 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa94c011590 con 0x7fa95c0cef40 2023-10-06T02:28:27.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.713+0000 7fa9437fe700 1 -- 172.21.15.31:0/472656575 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa94c0116f0 con 0x7fa95c0cef40 2023-10-06T02:28:27.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.713+0000 7fa96099d700 1 -- 172.21.15.31:0/472656575 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa95c0c7ad0 con 0x7fa95c0cef40 2023-10-06T02:28:27.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.713+0000 7fa9437fe700 1 -- 172.21.15.31:0/472656575 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa94c011ab0 con 0x7fa95c0cef40 2023-10-06T02:28:27.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.713+0000 7fa96099d700 1 -- 172.21.15.31:0/472656575 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa95c0c8090 con 0x7fa95c0cef40 2023-10-06T02:28:27.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.714+0000 7fa96099d700 1 -- 172.21.15.31:0/472656575 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa95c0d0170 con 0x7fa95c0cef40 2023-10-06T02:28:27.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.715+0000 7fa9437fe700 1 -- 172.21.15.31:0/472656575 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7fa94c01b3b0 con 0x7fa95c0cef40 2023-10-06T02:28:27.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.716+0000 7fa9437fe700 1 --2- 172.21.15.31:0/472656575 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa948067050 0x7fa948069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:27.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.717+0000 7fa9437fe700 1 -- 172.21.15.31:0/472656575 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5378+0+0 (secure 0 0 0) 0x7fa94c083bb0 con 0x7fa95c0cef40 2023-10-06T02:28:27.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.717+0000 7fa95affd700 1 --2- 172.21.15.31:0/472656575 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa948067050 0x7fa948069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:27.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.718+0000 7fa95affd700 1 --2- 172.21.15.31:0/472656575 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa948067050 0x7fa948069500 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa9500099b0 tx=0x7fa950017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:27.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:27.721+0000 7fa9437fe700 1 -- 172.21.15.31:0/472656575 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa94c050db0 con 0x7fa95c0cef40 2023-10-06T02:28:28.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:28.002+0000 7fa96099d700 1 -- 172.21.15.31:0/472656575 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars"]} v 0) v1 -- 0x7fa95c04c3c0 con 0x7fa95c0cef40 2023-10-06T02:28:28.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:28.056+0000 7fa9437fe700 1 -- 172.21.15.31:0/472656575 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars"]}]=0 v300) v1 ==== 146+0+0 (secure 0 0 0) 0x7fa94c017070 con 0x7fa95c0cef40 2023-10-06T02:28:28.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:28.058+0000 7fa96099d700 1 -- 172.21.15.31:0/472656575 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa948067050 msgr2=0x7fa948069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:28.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:28.058+0000 7fa96099d700 1 --2- 172.21.15.31:0/472656575 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa948067050 0x7fa948069500 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa9500099b0 tx=0x7fa950017000 comp rx=0 tx=0).stop 2023-10-06T02:28:28.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:28.059+0000 7fa96099d700 1 -- 172.21.15.31:0/472656575 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa95c0cef40 msgr2=0x7fa95c144f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:28.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:28.059+0000 7fa96099d700 1 --2- 172.21.15.31:0/472656575 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa95c0cef40 0x7fa95c144f10 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fa94c00c590 tx=0x7fa94c00c950 comp rx=0 tx=0).stop 2023-10-06T02:28:28.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:28.059+0000 7fa96099d700 1 -- 172.21.15.31:0/472656575 shutdown_connections 2023-10-06T02:28:28.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:28.059+0000 7fa96099d700 1 --2- 172.21.15.31:0/472656575 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa95c15de50 0x7fa95c0b0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:28.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:28.059+0000 7fa96099d700 1 --2- 172.21.15.31:0/472656575 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa95c0cef40 0x7fa95c144f10 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:28.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:28.059+0000 7fa96099d700 1 --2- 172.21.15.31:0/472656575 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa948067050 0x7fa948069500 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:28.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:28.059+0000 7fa96099d700 1 --2- 172.21.15.31:0/472656575 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa95c153c30 0x7fa95c0b5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:28.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:28.059+0000 7fa96099d700 1 -- 172.21.15.31:0/472656575 >> 172.21.15.31:0/472656575 conn(0x7fa95c0703a0 msgr2=0x7fa95c151010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:28.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:28.059+0000 7fa96099d700 1 -- 172.21.15.31:0/472656575 shutdown_connections 2023-10-06T02:28:28.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:28.059+0000 7fa96099d700 1 -- 172.21.15.31:0/472656575 wait complete. 2023-10-06T02:28:28.077 INFO:tasks.cephfs_test_runner:Test the creating vols, subvols subvolgroups fails when their names uses ... ok 2023-10-06T02:28:28.078 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_no_upgrade_v1_sanity (tasks.cephfs.test_volumes.TestMisc) poor man's upgrade test -- theme continues... 2023-10-06T02:28:28.078 INFO:tasks.cephfs_test_runner:test_subvolume_no_upgrade_v1_sanity (tasks.cephfs.test_volumes.TestMisc) 2023-10-06T02:28:28.078 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity' 2023-10-06T02:28:28.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:28.271+0000 7f18c679d700 1 Processor -- start 2023-10-06T02:28:28.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:28.271+0000 7f18c679d700 1 -- start start 2023-10-06T02:28:28.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:28.272+0000 7f18c679d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f18c0153c30 0x7f18c015a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:28.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:28.272+0000 7f18c679d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f18c015de50 0x7f18c00cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:28.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:28.272+0000 7f18c679d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f18c00cef40 0x7f18c006d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:28.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:28.272+0000 7f18c679d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18c015afd0 con 0x7f18c015de50 2023-10-06T02:28:28.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:28.272+0000 7f18c679d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18c015b2b0 con 0x7f18c00cef40 2023-10-06T02:28:28.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:28.272+0000 7f18c679d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18c015b590 con 0x7f18c0153c30 2023-10-06T02:28:28.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:28.273+0000 7f18c4f9a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f18c015de50 0x7f18c00cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:28.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:28.273+0000 7f18c4f9a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f18c015de50 0x7f18c00cea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40430/0 (socket says 172.21.15.31:40430) 2023-10-06T02:28:28.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:28.273+0000 7f18c4f9a700 1 -- 172.21.15.31:0/1503158602 learned_addr learned my addr 172.21.15.31:0/1503158602 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:28.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:28.273+0000 7f18c5f9c700 1 --2- 172.21.15.31:0/1503158602 >> v2:172.21.15.62:3300/0 conn(0x7f18c00cef40 0x7f18c006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:28.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:28.273+0000 7f18c579b700 1 --2- 172.21.15.31:0/1503158602 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f18c0153c30 0x7f18c015a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:28.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:28.273+0000 7f18c4f9a700 1 -- 172.21.15.31:0/1503158602 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f18c0153c30 msgr2=0x7f18c015a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:28.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:28.273+0000 7f18c4f9a700 1 --2- 172.21.15.31:0/1503158602 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f18c0153c30 0x7f18c015a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:28.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:28.273+0000 7f18c4f9a700 1 -- 172.21.15.31:0/1503158602 >> v2:172.21.15.62:3300/0 conn(0x7f18c00cef40 msgr2=0x7f18c006d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:28.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:28.273+0000 7f18c4f9a700 1 --2- 172.21.15.31:0/1503158602 >> v2:172.21.15.62:3300/0 conn(0x7f18c00cef40 0x7f18c006d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:28.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:28.273+0000 7f18c4f9a700 1 -- 172.21.15.31:0/1503158602 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18c015b870 con 0x7f18c015de50 2023-10-06T02:28:28.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:28.273+0000 7f18c4f9a700 1 --2- 172.21.15.31:0/1503158602 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18c015de50 0x7f18c00cea00 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f18c0071370 tx=0x7f18b0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c649bfc38e75d4e5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:28.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:28.274+0000 7f18b7fff700 1 -- 172.21.15.31:0/1503158602 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b000b860 con 0x7f18c015de50 2023-10-06T02:28:28.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:28.274+0000 7f18b7fff700 1 -- 172.21.15.31:0/1503158602 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18b000b9c0 con 0x7f18c015de50 2023-10-06T02:28:28.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:28.274+0000 7f18b7fff700 1 -- 172.21.15.31:0/1503158602 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b000bd60 con 0x7f18c015de50 2023-10-06T02:28:28.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:28.274+0000 7f18c679d700 1 -- 172.21.15.31:0/1503158602 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18c015de50 msgr2=0x7f18c00cea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:28.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:28.274+0000 7f18c679d700 1 --2- 172.21.15.31:0/1503158602 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18c015de50 0x7f18c00cea00 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f18c0071370 tx=0x7f18b0009250 comp rx=0 tx=0).stop 2023-10-06T02:28:28.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:28.274+0000 7f18c679d700 1 -- 172.21.15.31:0/1503158602 shutdown_connections 2023-10-06T02:28:28.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:28.274+0000 7f18c679d700 1 --2- 172.21.15.31:0/1503158602 >> v2:172.21.15.62:3300/0 conn(0x7f18c00cef40 0x7f18c006d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:28.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:28.274+0000 7f18c679d700 1 --2- 172.21.15.31:0/1503158602 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f18c0153c30 0x7f18c015a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:28.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:28.274+0000 7f18c679d700 1 --2- 172.21.15.31:0/1503158602 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18c015de50 0x7f18c00cea00 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:28.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:28.274+0000 7f18c679d700 1 -- 172.21.15.31:0/1503158602 >> 172.21.15.31:0/1503158602 conn(0x7f18c00703a0 msgr2=0x7f18c00707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:28.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:28.274+0000 7f18c679d700 1 -- 172.21.15.31:0/1503158602 shutdown_connections 2023-10-06T02:28:28.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:28.274+0000 7f18c679d700 1 -- 172.21.15.31:0/1503158602 wait complete. 2023-10-06T02:28:28.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:28.276+0000 7f18c679d700 1 Processor -- start 2023-10-06T02:28:28.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:28.276+0000 7f18c679d700 1 -- start start 2023-10-06T02:28:28.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:28.277+0000 7f18c679d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18c00cef40 0x7f18c014bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:28.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:28.277+0000 7f18c679d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f18c0153c30 0x7f18c00c9f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:28.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:28.277+0000 7f18c679d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18c015de50 0x7f18c00b5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:28.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:28.277+0000 7f18c679d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18c00b60c0 con 0x7f18c00cef40 2023-10-06T02:28:28.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:28.277+0000 7f18c679d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18c00cd2b0 con 0x7f18c015de50 2023-10-06T02:28:28.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:28.277+0000 7f18c679d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18c00cbd40 con 0x7f18c0153c30 2023-10-06T02:28:28.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:28.277+0000 7f18c579b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18c00cef40 0x7f18c014bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:28.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:28.277+0000 7f18c579b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18c00cef40 0x7f18c014bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40442/0 (socket says 172.21.15.31:40442) 2023-10-06T02:28:28.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:28.277+0000 7f18c579b700 1 -- 172.21.15.31:0/468994314 learned_addr learned my addr 172.21.15.31:0/468994314 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:28.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:28.277+0000 7f18c5f9c700 1 --2- 172.21.15.31:0/468994314 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18c015de50 0x7f18c00b5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:28.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:28.277+0000 7f18c4f9a700 1 --2- 172.21.15.31:0/468994314 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f18c0153c30 0x7f18c00c9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:28.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:28.277+0000 7f18c579b700 1 -- 172.21.15.31:0/468994314 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f18c0153c30 msgr2=0x7f18c00c9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:28.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:28.277+0000 7f18c579b700 1 --2- 172.21.15.31:0/468994314 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f18c0153c30 0x7f18c00c9f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:28.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:28.277+0000 7f18c579b700 1 -- 172.21.15.31:0/468994314 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18c015de50 msgr2=0x7f18c00b5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:28.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:28.277+0000 7f18c579b700 1 --2- 172.21.15.31:0/468994314 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18c015de50 0x7f18c00b5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:28.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:28.277+0000 7f18c579b700 1 -- 172.21.15.31:0/468994314 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18b00094f0 con 0x7f18c00cef40 2023-10-06T02:28:28.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:28.278+0000 7f18c579b700 1 --2- 172.21.15.31:0/468994314 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18c00cef40 0x7f18c014bda0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f18bc00d890 tx=0x7f18bc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:28.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:28.278+0000 7f18b57fa700 1 -- 172.21.15.31:0/468994314 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18bc0125f0 con 0x7f18c00cef40 2023-10-06T02:28:28.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:28.278+0000 7f18b57fa700 1 -- 172.21.15.31:0/468994314 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18bc012770 con 0x7f18c00cef40 2023-10-06T02:28:28.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:28.278+0000 7f18c679d700 1 -- 172.21.15.31:0/468994314 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18c00ccaf0 con 0x7f18c00cef40 2023-10-06T02:28:28.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:28.278+0000 7f18b57fa700 1 -- 172.21.15.31:0/468994314 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18bc012b30 con 0x7f18c00cef40 2023-10-06T02:28:28.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:28.278+0000 7f18c679d700 1 -- 172.21.15.31:0/468994314 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18c00cba60 con 0x7f18c00cef40 2023-10-06T02:28:28.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:28.279+0000 7f18c679d700 1 -- 172.21.15.31:0/468994314 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18c004c3c0 con 0x7f18c00cef40 2023-10-06T02:28:28.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:28.280+0000 7f18b57fa700 1 -- 172.21.15.31:0/468994314 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f18bc01e460 con 0x7f18c00cef40 2023-10-06T02:28:28.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:28.281+0000 7f18b57fa700 1 --2- 172.21.15.31:0/468994314 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f18a4067050 0x7f18a4069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:28.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:28.281+0000 7f18b57fa700 1 -- 172.21.15.31:0/468994314 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5378+0+0 (secure 0 0 0) 0x7f18bc084d20 con 0x7f18c00cef40 2023-10-06T02:28:28.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:28.281+0000 7f18c4f9a700 1 --2- 172.21.15.31:0/468994314 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f18a4067050 0x7f18a4069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:28.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:28.282+0000 7f18c4f9a700 1 --2- 172.21.15.31:0/468994314 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f18a4067050 0x7f18a4069500 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f18c0071370 tx=0x7f18b0017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:28.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:28.285+0000 7f18b57fa700 1 -- 172.21.15.31:0/468994314 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f18bc051f20 con 0x7f18c00cef40 2023-10-06T02:28:28.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:28.574+0000 7f18c679d700 1 -- 172.21.15.31:0/468994314 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity"]} v 0) v1 -- 0x7f18c015bb50 con 0x7f18c00cef40 2023-10-06T02:28:29.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.061+0000 7f18b57fa700 1 -- 172.21.15.31:0/468994314 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity"]}]=0 v301) v1 ==== 152+0+0 (secure 0 0 0) 0x7f18bc055540 con 0x7f18c00cef40 2023-10-06T02:28:29.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.063+0000 7f18c679d700 1 -- 172.21.15.31:0/468994314 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f18a4067050 msgr2=0x7f18a4069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:29.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.063+0000 7f18c679d700 1 --2- 172.21.15.31:0/468994314 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f18a4067050 0x7f18a4069500 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f18c0071370 tx=0x7f18b0017000 comp rx=0 tx=0).stop 2023-10-06T02:28:29.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.063+0000 7f18c679d700 1 -- 172.21.15.31:0/468994314 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18c00cef40 msgr2=0x7f18c014bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:29.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.063+0000 7f18c679d700 1 --2- 172.21.15.31:0/468994314 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18c00cef40 0x7f18c014bda0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f18bc00d890 tx=0x7f18bc008040 comp rx=0 tx=0).stop 2023-10-06T02:28:29.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.064+0000 7f18c679d700 1 -- 172.21.15.31:0/468994314 shutdown_connections 2023-10-06T02:28:29.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.064+0000 7f18c679d700 1 --2- 172.21.15.31:0/468994314 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18c015de50 0x7f18c00b5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:29.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.064+0000 7f18c679d700 1 --2- 172.21.15.31:0/468994314 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18c00cef40 0x7f18c014bda0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:29.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.064+0000 7f18c679d700 1 --2- 172.21.15.31:0/468994314 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f18a4067050 0x7f18a4069500 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:29.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.064+0000 7f18c679d700 1 --2- 172.21.15.31:0/468994314 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f18c0153c30 0x7f18c00c9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:29.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.064+0000 7f18c679d700 1 -- 172.21.15.31:0/468994314 >> 172.21.15.31:0/468994314 conn(0x7f18c00703a0 msgr2=0x7f18c00c5ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:29.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.064+0000 7f18c679d700 1 -- 172.21.15.31:0/468994314 shutdown_connections 2023-10-06T02:28:29.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.064+0000 7f18c679d700 1 -- 172.21.15.31:0/468994314 wait complete. 2023-10-06T02:28:29.083 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T02:28:29.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.274+0000 7f0c1ffff700 1 Processor -- start 2023-10-06T02:28:29.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.275+0000 7f0c1ffff700 1 -- start start 2023-10-06T02:28:29.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.276+0000 7f0c1ffff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0c200c7be0 0x7f0c2006abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:29.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.276+0000 7f0c1ffff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0c200c7fb0 0x7f0c2015a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:29.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.276+0000 7f0c1ffff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c2015aed0 con 0x7f0c2013f780 2023-10-06T02:28:29.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.276+0000 7f0c1ffff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c2006bf10 con 0x7f0c200c7fb0 2023-10-06T02:28:29.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.276+0000 7f0c1ffff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2006b290 con 0x7f0c200c7be0 2023-10-06T02:28:29.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.276+0000 7f0c1e7fc700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0c200c7fb0 0x7f0c2015a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:29.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.276+0000 7f0c1e7fc700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0c200c7fb0 0x7f0c2015a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40446/0 (socket says 172.21.15.31:40446) 2023-10-06T02:28:29.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.276+0000 7f0c1e7fc700 1 -- 172.21.15.31:0/826571818 learned_addr learned my addr 172.21.15.31:0/826571818 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:29.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.277+0000 7f0c1effd700 1 --2- 172.21.15.31:0/826571818 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0c200c7be0 0x7f0c2006abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:29.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.277+0000 7f0c1e7fc700 1 -- 172.21.15.31:0/826571818 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0c200c7be0 msgr2=0x7f0c2006abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:29.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.277+0000 7f0c1e7fc700 1 --2- 172.21.15.31:0/826571818 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0c200c7be0 0x7f0c2006abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:29.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.277+0000 7f0c1e7fc700 1 -- 172.21.15.31:0/826571818 >> v1:172.21.15.62:6789/0 conn(0x7f0c2013f780 legacy=0x7f0c200c6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:29.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.277+0000 7f0c1e7fc700 1 -- 172.21.15.31:0/826571818 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c2006b570 con 0x7f0c200c7fb0 2023-10-06T02:28:29.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.277+0000 7f0c1e7fc700 1 --2- 172.21.15.31:0/826571818 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c200c7fb0 0x7f0c2015a990 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f0c140099b0 tx=0x7f0c14009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60c4ca8aac89170a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:29.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.277+0000 7f0c1dffb700 1 -- 172.21.15.31:0/826571818 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c14009ea0 con 0x7f0c200c7fb0 2023-10-06T02:28:29.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.277+0000 7f0c1dffb700 1 -- 172.21.15.31:0/826571818 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c1400b860 con 0x7f0c200c7fb0 2023-10-06T02:28:29.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.278+0000 7f0c1dffb700 1 -- 172.21.15.31:0/826571818 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1400bc00 con 0x7f0c200c7fb0 2023-10-06T02:28:29.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.278+0000 7f0c1ffff700 1 -- 172.21.15.31:0/826571818 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c200c7fb0 msgr2=0x7f0c2015a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:29.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.278+0000 7f0c1ffff700 1 --2- 172.21.15.31:0/826571818 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c200c7fb0 0x7f0c2015a990 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f0c140099b0 tx=0x7f0c14009250 comp rx=0 tx=0).stop 2023-10-06T02:28:29.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.278+0000 7f0c1ffff700 1 -- 172.21.15.31:0/826571818 shutdown_connections 2023-10-06T02:28:29.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.278+0000 7f0c1ffff700 1 --2- 172.21.15.31:0/826571818 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0c200c7be0 0x7f0c2006abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:29.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.278+0000 7f0c1ffff700 1 --2- 172.21.15.31:0/826571818 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c200c7fb0 0x7f0c2015a990 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:29.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.278+0000 7f0c1ffff700 1 -- 172.21.15.31:0/826571818 >> 172.21.15.31:0/826571818 conn(0x7f0c2006d110 msgr2=0x7f0c200cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:29.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.278+0000 7f0c1ffff700 1 -- 172.21.15.31:0/826571818 shutdown_connections 2023-10-06T02:28:29.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.278+0000 7f0c1ffff700 1 -- 172.21.15.31:0/826571818 wait complete. 2023-10-06T02:28:29.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.280+0000 7f0c1ffff700 1 Processor -- start 2023-10-06T02:28:29.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.280+0000 7f0c1ffff700 1 -- start start 2023-10-06T02:28:29.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.281+0000 7f0c1ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c200c7be0 0x7f0c20155790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:29.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.281+0000 7f0c1ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c200c7fb0 0x7f0c20139c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:29.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.281+0000 7f0c1ffff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0c2013f780 0x7f0c2006e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:29.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.281+0000 7f0c1ffff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c200b0040 con 0x7f0c200c7fb0 2023-10-06T02:28:29.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.281+0000 7f0c1ffff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c20155cd0 con 0x7f0c200c7be0 2023-10-06T02:28:29.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.281+0000 7f0c1ffff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2013a5c0 con 0x7f0c2013f780 2023-10-06T02:28:29.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.281+0000 7f0c1e7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c200c7fb0 0x7f0c20139c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:29.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.281+0000 7f0c1e7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c200c7fb0 0x7f0c20139c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40458/0 (socket says 172.21.15.31:40458) 2023-10-06T02:28:29.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.281+0000 7f0c1e7fc700 1 -- 172.21.15.31:0/730015735 learned_addr learned my addr 172.21.15.31:0/730015735 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:29.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.281+0000 7f0c1effd700 1 --2- 172.21.15.31:0/730015735 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c200c7be0 0x7f0c20155790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:29.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.281+0000 7f0c1f7fe700 1 --2- 172.21.15.31:0/730015735 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0c2013f780 0x7f0c2006e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:29.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.281+0000 7f0c1e7fc700 1 -- 172.21.15.31:0/730015735 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0c2013f780 msgr2=0x7f0c2006e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:29.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.281+0000 7f0c1e7fc700 1 --2- 172.21.15.31:0/730015735 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0c2013f780 0x7f0c2006e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:29.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.281+0000 7f0c1e7fc700 1 -- 172.21.15.31:0/730015735 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c200c7be0 msgr2=0x7f0c20155790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:29.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.281+0000 7f0c1e7fc700 1 --2- 172.21.15.31:0/730015735 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c200c7be0 0x7f0c20155790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:29.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.281+0000 7f0c1e7fc700 1 -- 172.21.15.31:0/730015735 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c140094f0 con 0x7f0c200c7fb0 2023-10-06T02:28:29.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.282+0000 7f0c1e7fc700 1 --2- 172.21.15.31:0/730015735 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c200c7fb0 0x7f0c20139c50 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f0c14006fd0 tx=0x7f0c14017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:29.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.282+0000 7f0c06ffd700 1 -- 172.21.15.31:0/730015735 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1400bbe0 con 0x7f0c200c7fb0 2023-10-06T02:28:29.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.282+0000 7f0c06ffd700 1 -- 172.21.15.31:0/730015735 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c14011420 con 0x7f0c200c7fb0 2023-10-06T02:28:29.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.282+0000 7f0c1ffff700 1 -- 172.21.15.31:0/730015735 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c200af710 con 0x7f0c200c7fb0 2023-10-06T02:28:29.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.282+0000 7f0c06ffd700 1 -- 172.21.15.31:0/730015735 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c140117c0 con 0x7f0c200c7fb0 2023-10-06T02:28:29.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.282+0000 7f0c1ffff700 1 -- 172.21.15.31:0/730015735 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c200afcd0 con 0x7f0c200c7fb0 2023-10-06T02:28:29.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.283+0000 7f0c1ffff700 1 -- 172.21.15.31:0/730015735 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c2004c3c0 con 0x7f0c200c7fb0 2023-10-06T02:28:29.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.284+0000 7f0c06ffd700 1 -- 172.21.15.31:0/730015735 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f0c14011bf0 con 0x7f0c200c7fb0 2023-10-06T02:28:29.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.285+0000 7f0c06ffd700 1 --2- 172.21.15.31:0/730015735 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0c0c067050 0x7f0c0c069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:29.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.285+0000 7f0c06ffd700 1 -- 172.21.15.31:0/730015735 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5378+0+0 (secure 0 0 0) 0x7f0c14085f70 con 0x7f0c200c7fb0 2023-10-06T02:28:29.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.286+0000 7f0c1effd700 1 --2- 172.21.15.31:0/730015735 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0c0c067050 0x7f0c0c069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:29.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.286+0000 7f0c1effd700 1 --2- 172.21.15.31:0/730015735 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0c0c067050 0x7f0c0c069500 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0c10009d40 tx=0x7f0c10008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:29.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.289+0000 7f0c06ffd700 1 -- 172.21.15.31:0/730015735 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0c14053170 con 0x7f0c200c7fb0 2023-10-06T02:28:29.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.492+0000 7f0c1ffff700 1 -- 172.21.15.31:0/730015735 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f0c2006b850 con 0x7f0c200c7fb0 2023-10-06T02:28:29.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.497+0000 7f0c06ffd700 1 -- 172.21.15.31:0/730015735 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0c14056790 con 0x7f0c200c7fb0 2023-10-06T02:28:29.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.499+0000 7f0c1ffff700 1 -- 172.21.15.31:0/730015735 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0c0c067050 msgr2=0x7f0c0c069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:29.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.499+0000 7f0c1ffff700 1 --2- 172.21.15.31:0/730015735 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0c0c067050 0x7f0c0c069500 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0c10009d40 tx=0x7f0c10008040 comp rx=0 tx=0).stop 2023-10-06T02:28:29.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.499+0000 7f0c1ffff700 1 -- 172.21.15.31:0/730015735 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c200c7fb0 msgr2=0x7f0c20139c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:29.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.499+0000 7f0c1ffff700 1 --2- 172.21.15.31:0/730015735 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c200c7fb0 0x7f0c20139c50 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f0c14006fd0 tx=0x7f0c14017000 comp rx=0 tx=0).stop 2023-10-06T02:28:29.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.499+0000 7f0c1ffff700 1 -- 172.21.15.31:0/730015735 shutdown_connections 2023-10-06T02:28:29.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.499+0000 7f0c1ffff700 1 --2- 172.21.15.31:0/730015735 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0c0c067050 0x7f0c0c069500 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:29.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.500+0000 7f0c1ffff700 1 --2- 172.21.15.31:0/730015735 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0c2013f780 0x7f0c2006e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:29.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.500+0000 7f0c1ffff700 1 --2- 172.21.15.31:0/730015735 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c200c7be0 0x7f0c20155790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:29.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.500+0000 7f0c1ffff700 1 --2- 172.21.15.31:0/730015735 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c200c7fb0 0x7f0c20139c50 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:29.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.500+0000 7f0c1ffff700 1 -- 172.21.15.31:0/730015735 >> 172.21.15.31:0/730015735 conn(0x7f0c2006d110 msgr2=0x7f0c2013ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:29.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.500+0000 7f0c1ffff700 1 -- 172.21.15.31:0/730015735 shutdown_connections 2023-10-06T02:28:29.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.500+0000 7f0c1ffff700 1 -- 172.21.15.31:0/730015735 wait complete. 2023-10-06T02:28:29.519 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T02:28:29.519 DEBUG:teuthology.orchestra.run.smithi031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T02:28:29.600 DEBUG:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T02:28:29.687 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T02:28:29.687 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T02:28:29.687 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:28:29.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.872+0000 7f340559d700 1 Processor -- start 2023-10-06T02:28:29.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.872+0000 7f340559d700 1 -- start start 2023-10-06T02:28:29.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.874+0000 7f340559d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f340006e0a0 0x7f34000c6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:29.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.874+0000 7f340559d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f340006f3a0 0x7f340014fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:29.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.874+0000 7f340559d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f340006e470 0x7f34001501b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:29.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.874+0000 7f340559d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f340006a490 con 0x7f340006e0a0 2023-10-06T02:28:29.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.874+0000 7f340559d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34000c7490 con 0x7f340006f3a0 2023-10-06T02:28:29.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.874+0000 7f340559d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34000c7770 con 0x7f340006e470 2023-10-06T02:28:29.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.874+0000 7f33fffff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f340006e0a0 0x7f34000c6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:29.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.874+0000 7f33fffff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f340006e0a0 0x7f34000c6b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40466/0 (socket says 172.21.15.31:40466) 2023-10-06T02:28:29.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.874+0000 7f33fffff700 1 -- 172.21.15.31:0/4045435487 learned_addr learned my addr 172.21.15.31:0/4045435487 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:29.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.874+0000 7f3404d9c700 1 --2- 172.21.15.31:0/4045435487 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f340006e470 0x7f34001501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:29.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.874+0000 7f33ff7fe700 1 --2- 172.21.15.31:0/4045435487 >> v2:172.21.15.62:3300/0 conn(0x7f340006f3a0 0x7f340014fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:29.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.874+0000 7f33fffff700 1 -- 172.21.15.31:0/4045435487 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f340006e470 msgr2=0x7f34001501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:29.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.874+0000 7f33fffff700 1 --2- 172.21.15.31:0/4045435487 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f340006e470 0x7f34001501b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:29.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.874+0000 7f33fffff700 1 -- 172.21.15.31:0/4045435487 >> v2:172.21.15.62:3300/0 conn(0x7f340006f3a0 msgr2=0x7f340014fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:29.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.874+0000 7f33fffff700 1 --2- 172.21.15.31:0/4045435487 >> v2:172.21.15.62:3300/0 conn(0x7f340006f3a0 0x7f340014fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:29.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.875+0000 7f33fffff700 1 -- 172.21.15.31:0/4045435487 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34000c7a50 con 0x7f340006e0a0 2023-10-06T02:28:29.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.875+0000 7f33fffff700 1 --2- 172.21.15.31:0/4045435487 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f340006e0a0 0x7f34000c6b20 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f3400156140 tx=0x7f33f000b4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e71a1a27a8f1851b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:29.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.875+0000 7f33feffd700 1 -- 172.21.15.31:0/4045435487 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f000db10 con 0x7f340006e0a0 2023-10-06T02:28:29.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.875+0000 7f33feffd700 1 -- 172.21.15.31:0/4045435487 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33f000dc70 con 0x7f340006e0a0 2023-10-06T02:28:29.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.875+0000 7f33feffd700 1 -- 172.21.15.31:0/4045435487 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f0017410 con 0x7f340006e0a0 2023-10-06T02:28:29.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.876+0000 7f340559d700 1 -- 172.21.15.31:0/4045435487 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f340006e0a0 msgr2=0x7f34000c6b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:29.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.876+0000 7f340559d700 1 --2- 172.21.15.31:0/4045435487 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f340006e0a0 0x7f34000c6b20 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f3400156140 tx=0x7f33f000b4e0 comp rx=0 tx=0).stop 2023-10-06T02:28:29.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.876+0000 7f340559d700 1 -- 172.21.15.31:0/4045435487 shutdown_connections 2023-10-06T02:28:29.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.876+0000 7f340559d700 1 --2- 172.21.15.31:0/4045435487 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f340006e470 0x7f34001501b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:29.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.876+0000 7f340559d700 1 --2- 172.21.15.31:0/4045435487 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f340006e0a0 0x7f34000c6b20 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:29.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.876+0000 7f340559d700 1 --2- 172.21.15.31:0/4045435487 >> v2:172.21.15.62:3300/0 conn(0x7f340006f3a0 0x7f340014fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:29.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.876+0000 7f340559d700 1 -- 172.21.15.31:0/4045435487 >> 172.21.15.31:0/4045435487 conn(0x7f340006d170 msgr2=0x7f34000cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:29.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.876+0000 7f340559d700 1 -- 172.21.15.31:0/4045435487 shutdown_connections 2023-10-06T02:28:29.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.876+0000 7f340559d700 1 -- 172.21.15.31:0/4045435487 wait complete. 2023-10-06T02:28:29.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.877+0000 7f340559d700 1 Processor -- start 2023-10-06T02:28:29.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.877+0000 7f340559d700 1 -- start start 2023-10-06T02:28:29.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.878+0000 7f340559d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f340006e0a0 0x7f340013ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:29.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.878+0000 7f340559d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f340006e470 0x7f34000c47e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:29.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.878+0000 7f340559d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f340006f3a0 0x7f34000caed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:29.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.878+0000 7f340559d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f340016cb70 con 0x7f340006f3a0 2023-10-06T02:28:29.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.878+0000 7f340559d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f340016c260 con 0x7f340006e0a0 2023-10-06T02:28:29.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.878+0000 7f340559d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34000c5210 con 0x7f340006e470 2023-10-06T02:28:29.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.878+0000 7f3404d9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f340006f3a0 0x7f34000caed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:29.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.879+0000 7f3404d9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f340006f3a0 0x7f34000caed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40474/0 (socket says 172.21.15.31:40474) 2023-10-06T02:28:29.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.879+0000 7f3404d9c700 1 -- 172.21.15.31:0/1039749313 learned_addr learned my addr 172.21.15.31:0/1039749313 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:29.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.879+0000 7f33fffff700 1 --2- 172.21.15.31:0/1039749313 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f340006e0a0 0x7f340013ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:29.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.879+0000 7f33ff7fe700 1 --2- 172.21.15.31:0/1039749313 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f340006e470 0x7f34000c47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:29.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.879+0000 7f3404d9c700 1 -- 172.21.15.31:0/1039749313 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f340006e470 msgr2=0x7f34000c47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:29.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.879+0000 7f3404d9c700 1 --2- 172.21.15.31:0/1039749313 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f340006e470 0x7f34000c47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:29.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.879+0000 7f3404d9c700 1 -- 172.21.15.31:0/1039749313 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f340006e0a0 msgr2=0x7f340013ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:29.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.879+0000 7f3404d9c700 1 --2- 172.21.15.31:0/1039749313 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f340006e0a0 0x7f340013ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:29.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.879+0000 7f3404d9c700 1 -- 172.21.15.31:0/1039749313 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33f000b780 con 0x7f340006f3a0 2023-10-06T02:28:29.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.879+0000 7f3404d9c700 1 --2- 172.21.15.31:0/1039749313 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f340006f3a0 0x7f34000caed0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f33f8009fa0 tx=0x7f33f8009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:29.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.879+0000 7f33e7fff700 1 -- 172.21.15.31:0/1039749313 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f800ebc0 con 0x7f340006f3a0 2023-10-06T02:28:29.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.879+0000 7f33e7fff700 1 -- 172.21.15.31:0/1039749313 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33f800ed20 con 0x7f340006f3a0 2023-10-06T02:28:29.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.879+0000 7f33e7fff700 1 -- 172.21.15.31:0/1039749313 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f8014670 con 0x7f340006f3a0 2023-10-06T02:28:29.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.880+0000 7f340559d700 1 -- 172.21.15.31:0/1039749313 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f340016c520 con 0x7f340006f3a0 2023-10-06T02:28:29.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.880+0000 7f340559d700 1 -- 172.21.15.31:0/1039749313 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f340014c2c0 con 0x7f340006f3a0 2023-10-06T02:28:29.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.881+0000 7f340559d700 1 -- 172.21.15.31:0/1039749313 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f340016cd20 con 0x7f340006f3a0 2023-10-06T02:28:29.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.882+0000 7f33e7fff700 1 -- 172.21.15.31:0/1039749313 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f33f80149e0 con 0x7f340006f3a0 2023-10-06T02:28:29.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.882+0000 7f33e7fff700 1 --2- 172.21.15.31:0/1039749313 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f33ec067050 0x7f33ec069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:29.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.882+0000 7f33e7fff700 1 -- 172.21.15.31:0/1039749313 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5378+0+0 (secure 0 0 0) 0x7f33f8084c60 con 0x7f340006f3a0 2023-10-06T02:28:29.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.883+0000 7f33fffff700 1 --2- 172.21.15.31:0/1039749313 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f33ec067050 0x7f33ec069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:29.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.884+0000 7f33fffff700 1 --2- 172.21.15.31:0/1039749313 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f33ec067050 0x7f33ec069500 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f33f000b520 tx=0x7f33f0019000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:29.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:29.886+0000 7f33e7fff700 1 -- 172.21.15.31:0/1039749313 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f340016cd20 con 0x7f340006f3a0 2023-10-06T02:28:30.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:30.139+0000 7f340559d700 1 -- 172.21.15.31:0/1039749313 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f340004c3c0 con 0x7f340006f3a0 2023-10-06T02:28:30.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:30.140+0000 7f33e7fff700 1 -- 172.21.15.31:0/1039749313 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+3520 (secure 0 0 0) 0x7f33f8051e60 con 0x7f340006f3a0 2023-10-06T02:28:30.141 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:28:30.142 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":64,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5759,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/2470516084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2470516084},{"type":"v1","addr":"172.21.15.62:6834","nonce":2470516084}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":6364,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/790751965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":790751965},{"type":"v1","addr":"172.21.15.31:6834","nonce":790751965}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":6571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3862349969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":3862349969},{"type":"v1","addr":"172.21.15.62:6835","nonce":3862349969}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":7008,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/48886335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":48886335},{"type":"v1","addr":"172.21.15.31:6835","nonce":48886335}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":64}],"filesystems":[]} 2023-10-06T02:28:30.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:30.142+0000 7f340559d700 1 -- 172.21.15.31:0/1039749313 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f33ec067050 msgr2=0x7f33ec069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:30.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:30.142+0000 7f340559d700 1 --2- 172.21.15.31:0/1039749313 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f33ec067050 0x7f33ec069500 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f33f000b520 tx=0x7f33f0019000 comp rx=0 tx=0).stop 2023-10-06T02:28:30.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:30.142+0000 7f340559d700 1 -- 172.21.15.31:0/1039749313 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f340006f3a0 msgr2=0x7f34000caed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:30.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:30.142+0000 7f340559d700 1 --2- 172.21.15.31:0/1039749313 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f340006f3a0 0x7f34000caed0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f33f8009fa0 tx=0x7f33f8009900 comp rx=0 tx=0).stop 2023-10-06T02:28:30.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:30.142+0000 7f340559d700 1 -- 172.21.15.31:0/1039749313 shutdown_connections 2023-10-06T02:28:30.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:30.142+0000 7f340559d700 1 --2- 172.21.15.31:0/1039749313 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f340006f3a0 0x7f34000caed0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:30.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:30.142+0000 7f340559d700 1 --2- 172.21.15.31:0/1039749313 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f340006e0a0 0x7f340013ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:30.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:30.142+0000 7f340559d700 1 --2- 172.21.15.31:0/1039749313 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f33ec067050 0x7f33ec069500 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:30.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:30.142+0000 7f340559d700 1 --2- 172.21.15.31:0/1039749313 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f340006e470 0x7f34000c47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:30.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:30.142+0000 7f340559d700 1 -- 172.21.15.31:0/1039749313 >> 172.21.15.31:0/1039749313 conn(0x7f340006d170 msgr2=0x7f340014d860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:30.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:30.142+0000 7f340559d700 1 -- 172.21.15.31:0/1039749313 shutdown_connections 2023-10-06T02:28:30.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:30.142+0000 7f340559d700 1 -- 172.21.15.31:0/1039749313 wait complete. 2023-10-06T02:28:30.145 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 64 2023-10-06T02:28:30.161 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T02:28:30.162 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T02:28:30.162 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T02:28:30.162 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T02:28:30.162 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T02:28:30.163 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T02:28:30.163 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T02:28:30.163 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T02:28:30.163 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T02:28:30.163 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T02:28:30.163 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T02:28:30.164 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T02:28:30.164 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T02:28:30.164 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T02:28:30.164 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T02:28:30.164 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T02:28:30.175 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T02:28:30.175 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T02:28:30.176 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T02:28:30.176 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T02:28:30.180 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T02:28:30.180 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T02:28:30.181 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T02:28:30.182 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T02:28:30.220 INFO:tasks.ceph.mds.a:Started 2023-10-06T02:28:30.220 DEBUG:teuthology.parallel:result is None 2023-10-06T02:28:30.222 INFO:tasks.ceph.mds.c:Started 2023-10-06T02:28:30.223 DEBUG:teuthology.parallel:result is None 2023-10-06T02:28:30.226 INFO:tasks.ceph.mds.b:Started 2023-10-06T02:28:30.226 DEBUG:teuthology.parallel:result is None 2023-10-06T02:28:30.228 INFO:tasks.ceph.mds.d:Started 2023-10-06T02:28:30.229 DEBUG:teuthology.parallel:result is None 2023-10-06T02:28:30.229 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T02:28:30.357 INFO:tasks.ceph.mds.a.smithi031.stdout:starting mds.a at 2023-10-06T02:28:30.358 INFO:tasks.ceph.mds.c.smithi031.stdout:starting mds.c at 2023-10-06T02:28:30.370 INFO:tasks.ceph.mds.b.smithi062.stdout:starting mds.b at 2023-10-06T02:28:30.371 INFO:tasks.ceph.mds.d.smithi062.stdout:starting mds.d at 2023-10-06T02:28:30.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:30.425+0000 7f55cc99d700 1 Processor -- start 2023-10-06T02:28:30.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:30.426+0000 7f55cc99d700 1 -- start start 2023-10-06T02:28:30.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:30.427+0000 7f55cc99d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f55c8153c30 0x7f55c815a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:30.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:30.427+0000 7f55cc99d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f55c815de50 0x7f55c80cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:30.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:30.427+0000 7f55cc99d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55c80cef40 0x7f55c806d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:30.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:30.427+0000 7f55cc99d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55c80cf3c0 con 0x7f55c8153c30 2023-10-06T02:28:30.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:30.427+0000 7f55cc99d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55c815afd0 con 0x7f55c815de50 2023-10-06T02:28:30.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:30.427+0000 7f55cc99d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55c815b2b0 con 0x7f55c80cef40 2023-10-06T02:28:30.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:30.427+0000 7f55c77fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f55c8153c30 0x7f55c815a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:30.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:30.427+0000 7f55c77fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f55c8153c30 0x7f55c815a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40498/0 (socket says 172.21.15.31:40498) 2023-10-06T02:28:30.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:30.427+0000 7f55c77fe700 1 -- 172.21.15.31:0/1744294517 learned_addr learned my addr 172.21.15.31:0/1744294517 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:30.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:30.427+0000 7f55c7fff700 1 --2- 172.21.15.31:0/1744294517 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55c80cef40 0x7f55c806d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:30.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:30.427+0000 7f55c6ffd700 1 --2- 172.21.15.31:0/1744294517 >> v2:172.21.15.62:3300/0 conn(0x7f55c815de50 0x7f55c80cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:30.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:30.428+0000 7f55c77fe700 1 -- 172.21.15.31:0/1744294517 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55c80cef40 msgr2=0x7f55c806d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:30.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:30.428+0000 7f55c77fe700 1 --2- 172.21.15.31:0/1744294517 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55c80cef40 0x7f55c806d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:30.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:30.428+0000 7f55c77fe700 1 -- 172.21.15.31:0/1744294517 >> v2:172.21.15.62:3300/0 conn(0x7f55c815de50 msgr2=0x7f55c80cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:30.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:30.428+0000 7f55c77fe700 1 --2- 172.21.15.31:0/1744294517 >> v2:172.21.15.62:3300/0 conn(0x7f55c815de50 0x7f55c80cea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:30.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:30.428+0000 7f55c77fe700 1 -- 172.21.15.31:0/1744294517 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55c815b590 con 0x7f55c8153c30 2023-10-06T02:28:30.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:30.428+0000 7f55c6ffd700 1 --2- 172.21.15.31:0/1744294517 >> v2:172.21.15.62:3300/0 conn(0x7f55c815de50 0x7f55c80cea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:28:30.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:30.428+0000 7f55c77fe700 1 --2- 172.21.15.31:0/1744294517 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55c8153c30 0x7f55c815a5e0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f55c80713a0 tx=0x7f55b8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2ad15a2181cdae2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:30.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:30.428+0000 7f55c67fc700 1 -- 172.21.15.31:0/1744294517 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b8009e70 con 0x7f55c8153c30 2023-10-06T02:28:30.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:30.428+0000 7f55c67fc700 1 -- 172.21.15.31:0/1744294517 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55b800b860 con 0x7f55c8153c30 2023-10-06T02:28:30.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:30.428+0000 7f55c67fc700 1 -- 172.21.15.31:0/1744294517 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b800bc00 con 0x7f55c8153c30 2023-10-06T02:28:30.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:30.429+0000 7f55cc99d700 1 -- 172.21.15.31:0/1744294517 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55c8153c30 msgr2=0x7f55c815a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:30.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:30.429+0000 7f55cc99d700 1 --2- 172.21.15.31:0/1744294517 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55c8153c30 0x7f55c815a5e0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f55c80713a0 tx=0x7f55b8009250 comp rx=0 tx=0).stop 2023-10-06T02:28:30.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:30.429+0000 7f55cc99d700 1 -- 172.21.15.31:0/1744294517 shutdown_connections 2023-10-06T02:28:30.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:30.429+0000 7f55cc99d700 1 --2- 172.21.15.31:0/1744294517 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55c80cef40 0x7f55c806d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:30.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:30.429+0000 7f55cc99d700 1 --2- 172.21.15.31:0/1744294517 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55c8153c30 0x7f55c815a5e0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:30.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:30.429+0000 7f55cc99d700 1 --2- 172.21.15.31:0/1744294517 >> v2:172.21.15.62:3300/0 conn(0x7f55c815de50 0x7f55c80cea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:30.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:30.429+0000 7f55cc99d700 1 -- 172.21.15.31:0/1744294517 >> 172.21.15.31:0/1744294517 conn(0x7f55c80703a0 msgr2=0x7f55c80707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:30.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:30.429+0000 7f55cc99d700 1 -- 172.21.15.31:0/1744294517 shutdown_connections 2023-10-06T02:28:30.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:30.429+0000 7f55cc99d700 1 -- 172.21.15.31:0/1744294517 wait complete. 2023-10-06T02:28:30.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:30.430+0000 7f55cc99d700 1 Processor -- start 2023-10-06T02:28:30.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:30.430+0000 7f55cc99d700 1 -- start start 2023-10-06T02:28:30.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:30.431+0000 7f55cc99d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55c80cef40 0x7f55c814c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:30.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:30.431+0000 7f55cc99d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55c8153c30 0x7f55c8150230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:30.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:30.431+0000 7f55cc99d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55c815de50 0x7f55c8150770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:30.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:30.431+0000 7f55cc99d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c80ca0b0 con 0x7f55c815de50 2023-10-06T02:28:30.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:30.431+0000 7f55cc99d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c8150da0 con 0x7f55c8153c30 2023-10-06T02:28:30.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:30.431+0000 7f55cc99d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55c8151080 con 0x7f55c80cef40 2023-10-06T02:28:30.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:30.431+0000 7f55c7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55c815de50 0x7f55c8150770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:30.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:30.432+0000 7f55c6ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55c8153c30 0x7f55c8150230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:30.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:30.432+0000 7f55c77fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55c80cef40 0x7f55c814c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:30.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:30.432+0000 7f55c7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55c815de50 0x7f55c8150770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40506/0 (socket says 172.21.15.31:40506) 2023-10-06T02:28:30.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:30.432+0000 7f55c7fff700 1 -- 172.21.15.31:0/2168108552 learned_addr learned my addr 172.21.15.31:0/2168108552 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:30.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:30.432+0000 7f55c77fe700 1 -- 172.21.15.31:0/2168108552 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55c8153c30 msgr2=0x7f55c8150230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:30.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:30.432+0000 7f55c77fe700 1 --2- 172.21.15.31:0/2168108552 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55c8153c30 0x7f55c8150230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:30.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:30.432+0000 7f55c77fe700 1 -- 172.21.15.31:0/2168108552 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55c815de50 msgr2=0x7f55c8150770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:30.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:30.432+0000 7f55c77fe700 1 --2- 172.21.15.31:0/2168108552 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55c815de50 0x7f55c8150770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:30.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:30.432+0000 7f55c77fe700 1 -- 172.21.15.31:0/2168108552 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55b80094f0 con 0x7f55c80cef40 2023-10-06T02:28:30.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:30.432+0000 7f55c77fe700 1 --2- 172.21.15.31:0/2168108552 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55c80cef40 0x7f55c814c420 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f55c806ead0 tx=0x7f55b8017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:30.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:30.433+0000 7f55af7fe700 1 -- 172.21.15.31:0/2168108552 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b800bbe0 con 0x7f55c80cef40 2023-10-06T02:28:30.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:30.433+0000 7f55af7fe700 1 -- 172.21.15.31:0/2168108552 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55b801f070 con 0x7f55c80cef40 2023-10-06T02:28:30.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:30.433+0000 7f55cc99d700 1 -- 172.21.15.31:0/2168108552 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55c8151360 con 0x7f55c80cef40 2023-10-06T02:28:30.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:30.433+0000 7f55af7fe700 1 -- 172.21.15.31:0/2168108552 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b80266b0 con 0x7f55c80cef40 2023-10-06T02:28:30.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:30.433+0000 7f55cc99d700 1 -- 172.21.15.31:0/2168108552 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55c8151920 con 0x7f55c80cef40 2023-10-06T02:28:30.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:30.434+0000 7f55ad7fa700 1 -- 172.21.15.31:0/2168108552 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55c804c3c0 con 0x7f55c80cef40 2023-10-06T02:28:30.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:30.435+0000 7f55af7fe700 1 -- 172.21.15.31:0/2168108552 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f55b8026aa0 con 0x7f55c80cef40 2023-10-06T02:28:30.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:30.435+0000 7f55af7fe700 1 --2- 172.21.15.31:0/2168108552 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f55b4067050 0x7f55b4069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:30.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:30.435+0000 7f55af7fe700 1 -- 172.21.15.31:0/2168108552 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5378+0+0 (secure 0 0 0) 0x7f55b8021070 con 0x7f55c80cef40 2023-10-06T02:28:30.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:30.436+0000 7f55c6ffd700 1 --2- 172.21.15.31:0/2168108552 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f55b4067050 0x7f55b4069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:30.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:30.437+0000 7f55c6ffd700 1 --2- 172.21.15.31:0/2168108552 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f55b4067050 0x7f55b4069500 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f55c81523c0 tx=0x7f55bc008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:30.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:30.441+0000 7f55af7fe700 1 -- 172.21.15.31:0/2168108552 <== mon.2 v2:172.21.15.62:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f55b80533b0 con 0x7f55c80cef40 2023-10-06T02:28:30.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:30.657+0000 7f55ad7fa700 1 -- 172.21.15.31:0/2168108552 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f55c81523f0 con 0x7f55c80cef40 2023-10-06T02:28:31.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.080+0000 7f55af7fe700 1 -- 172.21.15.31:0/2168108552 <== mon.2 v2:172.21.15.62:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v100) v1 ==== 97+0+0 (secure 0 0 0) 0x7f55b80569d0 con 0x7f55c80cef40 2023-10-06T02:28:31.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.082+0000 7f55ad7fa700 1 -- 172.21.15.31:0/2168108552 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f55b4067050 msgr2=0x7f55b4069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:31.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.082+0000 7f55ad7fa700 1 --2- 172.21.15.31:0/2168108552 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f55b4067050 0x7f55b4069500 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f55c81523c0 tx=0x7f55bc008040 comp rx=0 tx=0).stop 2023-10-06T02:28:31.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.082+0000 7f55ad7fa700 1 -- 172.21.15.31:0/2168108552 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55c80cef40 msgr2=0x7f55c814c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:31.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.082+0000 7f55ad7fa700 1 --2- 172.21.15.31:0/2168108552 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55c80cef40 0x7f55c814c420 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f55c806ead0 tx=0x7f55b8017000 comp rx=0 tx=0).stop 2023-10-06T02:28:31.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.082+0000 7f55ad7fa700 1 -- 172.21.15.31:0/2168108552 shutdown_connections 2023-10-06T02:28:31.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.082+0000 7f55ad7fa700 1 --2- 172.21.15.31:0/2168108552 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55c815de50 0x7f55c8150770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:31.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.082+0000 7f55ad7fa700 1 --2- 172.21.15.31:0/2168108552 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f55b4067050 0x7f55b4069500 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:31.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.082+0000 7f55ad7fa700 1 --2- 172.21.15.31:0/2168108552 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55c80cef40 0x7f55c814c420 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:31.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.082+0000 7f55ad7fa700 1 --2- 172.21.15.31:0/2168108552 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55c8153c30 0x7f55c8150230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:31.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.082+0000 7f55ad7fa700 1 -- 172.21.15.31:0/2168108552 >> 172.21.15.31:0/2168108552 conn(0x7f55c80703a0 msgr2=0x7f55c80d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:31.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.083+0000 7f55ad7fa700 1 -- 172.21.15.31:0/2168108552 shutdown_connections 2023-10-06T02:28:31.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.083+0000 7f55ad7fa700 1 -- 172.21.15.31:0/2168108552 wait complete. 2023-10-06T02:28:31.085 INFO:teuthology.orchestra.run.smithi031.stderr: removed all blocklist entries 2023-10-06T02:28:31.104 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T02:28:31.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.292+0000 7f9c77fff700 1 Processor -- start 2023-10-06T02:28:31.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.292+0000 7f9c77fff700 1 -- start start 2023-10-06T02:28:31.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.293+0000 7f9c77fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9c7806e0a0 0x7f9c780c6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:31.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.293+0000 7f9c77fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9c7806f3a0 0x7f9c7814fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:31.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.293+0000 7f9c77fff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c7806a490 con 0x7f9c7806e470 2023-10-06T02:28:31.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.293+0000 7f9c77fff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c7806b930 con 0x7f9c7806f3a0 2023-10-06T02:28:31.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.293+0000 7f9c77fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c780c7490 con 0x7f9c7806e0a0 2023-10-06T02:28:31.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.293+0000 7f9c767fc700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9c7806f3a0 0x7f9c7814fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:31.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.293+0000 7f9c777fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f9c7806e470 0x7f9c78159740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56718/0 (socket says 172.21.15.31:56718) 2023-10-06T02:28:31.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.293+0000 7f9c777fe700 1 -- 172.21.15.31:0/2042670261 learned_addr learned my addr 172.21.15.31:0/2042670261 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:28:31.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.294+0000 7f9c76ffd700 1 --2- 172.21.15.31:0/2042670261 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9c7806e0a0 0x7f9c780c6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:31.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.294+0000 7f9c767fc700 1 -- 172.21.15.31:0/2042670261 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9c7806e0a0 msgr2=0x7f9c780c6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:31.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.294+0000 7f9c767fc700 1 --2- 172.21.15.31:0/2042670261 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9c7806e0a0 0x7f9c780c6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:31.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.294+0000 7f9c767fc700 1 -- 172.21.15.31:0/2042670261 >> v1:172.21.15.31:6789/0 conn(0x7f9c7806e470 legacy=0x7f9c78159740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:31.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.294+0000 7f9c767fc700 1 -- 172.21.15.31:0/2042670261 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c780c7770 con 0x7f9c7806f3a0 2023-10-06T02:28:31.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.294+0000 7f9c767fc700 1 --2- 172.21.15.31:0/2042670261 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c7806f3a0 0x7f9c7814fc70 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c0098d0 tx=0x7f9c6c009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0b4d07e9af7c283 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:31.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.294+0000 7f9c75ffb700 1 -- 172.21.15.31:0/2042670261 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c6c009e70 con 0x7f9c7806f3a0 2023-10-06T02:28:31.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.294+0000 7f9c75ffb700 1 -- 172.21.15.31:0/2042670261 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c6c005c10 con 0x7f9c7806f3a0 2023-10-06T02:28:31.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.294+0000 7f9c75ffb700 1 -- 172.21.15.31:0/2042670261 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c6c00b650 con 0x7f9c7806f3a0 2023-10-06T02:28:31.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.295+0000 7f9c77fff700 1 -- 172.21.15.31:0/2042670261 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c7806f3a0 msgr2=0x7f9c7814fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:31.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.295+0000 7f9c77fff700 1 --2- 172.21.15.31:0/2042670261 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c7806f3a0 0x7f9c7814fc70 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c0098d0 tx=0x7f9c6c009120 comp rx=0 tx=0).stop 2023-10-06T02:28:31.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.295+0000 7f9c77fff700 1 -- 172.21.15.31:0/2042670261 shutdown_connections 2023-10-06T02:28:31.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.295+0000 7f9c77fff700 1 --2- 172.21.15.31:0/2042670261 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9c7806e0a0 0x7f9c780c6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:31.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.295+0000 7f9c77fff700 1 --2- 172.21.15.31:0/2042670261 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c7806f3a0 0x7f9c7814fc70 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:31.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.295+0000 7f9c77fff700 1 -- 172.21.15.31:0/2042670261 >> 172.21.15.31:0/2042670261 conn(0x7f9c7806d170 msgr2=0x7f9c780cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:31.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.295+0000 7f9c77fff700 1 -- 172.21.15.31:0/2042670261 shutdown_connections 2023-10-06T02:28:31.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.295+0000 7f9c77fff700 1 -- 172.21.15.31:0/2042670261 wait complete. 2023-10-06T02:28:31.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.296+0000 7f9c77fff700 1 Processor -- start 2023-10-06T02:28:31.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.297+0000 7f9c77fff700 1 -- start start 2023-10-06T02:28:31.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.297+0000 7f9c77fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c7806e0a0 0x7f9c780b16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:31.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.297+0000 7f9c77fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c7806e470 0x7f9c780affc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:31.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.297+0000 7f9c77fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9c7806f3a0 0x7f9c781588b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:31.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.297+0000 7f9c77fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c780b1260 con 0x7f9c7806e0a0 2023-10-06T02:28:31.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.297+0000 7f9c77fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c78158df0 con 0x7f9c7806e470 2023-10-06T02:28:31.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.297+0000 7f9c77fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c780b0930 con 0x7f9c7806f3a0 2023-10-06T02:28:31.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.298+0000 7f9c76ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c7806e0a0 0x7f9c780b16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:31.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.298+0000 7f9c76ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c7806e0a0 0x7f9c780b16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40524/0 (socket says 172.21.15.31:40524) 2023-10-06T02:28:31.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.298+0000 7f9c76ffd700 1 -- 172.21.15.31:0/836356991 learned_addr learned my addr 172.21.15.31:0/836356991 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:31.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.298+0000 7f9c76ffd700 1 -- 172.21.15.31:0/836356991 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9c7806f3a0 msgr2=0x7f9c781588b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:31.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.298+0000 7f9c777fe700 1 --2- 172.21.15.31:0/836356991 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9c7806f3a0 0x7f9c781588b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:31.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.298+0000 7f9c767fc700 1 --2- 172.21.15.31:0/836356991 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c7806e470 0x7f9c780affc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:31.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.298+0000 7f9c76ffd700 1 --2- 172.21.15.31:0/836356991 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9c7806f3a0 0x7f9c781588b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:31.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.298+0000 7f9c76ffd700 1 -- 172.21.15.31:0/836356991 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c7806e470 msgr2=0x7f9c780affc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:31.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.298+0000 7f9c76ffd700 1 --2- 172.21.15.31:0/836356991 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c7806e470 0x7f9c780affc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:31.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.298+0000 7f9c76ffd700 1 -- 172.21.15.31:0/836356991 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c6c0093c0 con 0x7f9c7806e0a0 2023-10-06T02:28:31.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.298+0000 7f9c777fe700 1 --2- 172.21.15.31:0/836356991 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9c7806f3a0 0x7f9c781588b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:28:31.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.298+0000 7f9c767fc700 1 --2- 172.21.15.31:0/836356991 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c7806e470 0x7f9c780affc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:28:31.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.298+0000 7f9c76ffd700 1 --2- 172.21.15.31:0/836356991 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c7806e0a0 0x7f9c780b16f0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f9c6800d890 tx=0x7f9c68008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:31.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.299+0000 7f9c5effd700 1 -- 172.21.15.31:0/836356991 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c6800b600 con 0x7f9c7806e0a0 2023-10-06T02:28:31.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.299+0000 7f9c5effd700 1 -- 172.21.15.31:0/836356991 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c6800b760 con 0x7f9c7806e0a0 2023-10-06T02:28:31.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.299+0000 7f9c5effd700 1 -- 172.21.15.31:0/836356991 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c6800bae0 con 0x7f9c7806e0a0 2023-10-06T02:28:31.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.299+0000 7f9c77fff700 1 -- 172.21.15.31:0/836356991 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c780b0c10 con 0x7f9c7806e0a0 2023-10-06T02:28:31.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.299+0000 7f9c77fff700 1 -- 172.21.15.31:0/836356991 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c78148c00 con 0x7f9c7806e0a0 2023-10-06T02:28:31.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.300+0000 7f9c77fff700 1 -- 172.21.15.31:0/836356991 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c780b1540 con 0x7f9c7806e0a0 2023-10-06T02:28:31.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.301+0000 7f9c5effd700 1 -- 172.21.15.31:0/836356991 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f9c6801e460 con 0x7f9c7806e0a0 2023-10-06T02:28:31.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.302+0000 7f9c5effd700 1 --2- 172.21.15.31:0/836356991 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9c64067050 0x7f9c64069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:31.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.302+0000 7f9c5effd700 1 -- 172.21.15.31:0/836356991 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f9c68084c00 con 0x7f9c7806e0a0 2023-10-06T02:28:31.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.302+0000 7f9c767fc700 1 --2- 172.21.15.31:0/836356991 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9c64067050 0x7f9c64069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:31.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.303+0000 7f9c767fc700 1 --2- 172.21.15.31:0/836356991 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9c64067050 0x7f9c64069500 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9c7806f770 tx=0x7f9c6c017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:31.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.306+0000 7f9c5effd700 1 -- 172.21.15.31:0/836356991 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9c68051eb0 con 0x7f9c7806e0a0 2023-10-06T02:28:31.573 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T02:28:31.578 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T02:28:31.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.589+0000 7f9c77fff700 1 -- 172.21.15.31:0/836356991 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f9c7804c3c0 con 0x7f9c7806e0a0 2023-10-06T02:28:31.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.589+0000 7f9c5effd700 1 -- 172.21.15.31:0/836356991 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f9c680554d0 con 0x7f9c7806e0a0 2023-10-06T02:28:31.591 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:28:31.591 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T02:28:31.591 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_dump": [ 2023-10-06T02:28:31.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:28:31.591 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.a", 2023-10-06T02:28:31.591 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBLbx9lbjrOGxAAVoSOoDaZb1temL41F4Nong==", 2023-10-06T02:28:31.591 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:28:31.591 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:28:31.591 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T02:28:31.591 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T02:28:31.591 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:28:31.592 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:28:31.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:28:31.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:28:31.592 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.b", 2023-10-06T02:28:31.592 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBLbx9lFgsTMRAATTklzroiyQbTLiZc9j1BHQ==", 2023-10-06T02:28:31.592 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:28:31.592 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:28:31.592 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T02:28:31.592 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T02:28:31.592 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:28:31.592 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:28:31.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:28:31.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:28:31.593 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.c", 2023-10-06T02:28:31.593 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBLbx9lezVtJhAA8wK7+dJ1OmIc+iXJkexK0w==", 2023-10-06T02:28:31.593 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:28:31.593 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:28:31.593 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T02:28:31.593 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T02:28:31.593 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:28:31.593 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:28:31.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:28:31.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:28:31.593 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.d", 2023-10-06T02:28:31.593 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBMbx9l74gWABAApBu2nVv12rGA+OBfBnEc3A==", 2023-10-06T02:28:31.594 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:28:31.594 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:28:31.594 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T02:28:31.594 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T02:28:31.594 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:28:31.594 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:28:31.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:28:31.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:28:31.594 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.0", 2023-10-06T02:28:31.594 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBPbx9lgj6+MBAAXSUD0GCG+rO/9DQBZYt/8w==", 2023-10-06T02:28:31.594 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:28:31.594 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:28:31.595 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:28:31.595 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:28:31.595 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:28:31.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:28:31.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:28:31.595 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.1", 2023-10-06T02:28:31.595 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBSbx9lXhMTNBAAeEdPnMP71ZGb6k2Ddg5raQ==", 2023-10-06T02:28:31.595 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:28:31.595 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:28:31.595 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:28:31.595 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:28:31.595 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:28:31.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:28:31.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:28:31.596 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.2", 2023-10-06T02:28:31.596 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBVbx9l4Ly0NhAATYXYDAjYuP53zjqK0yMboA==", 2023-10-06T02:28:31.596 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:28:31.596 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:28:31.596 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:28:31.596 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:28:31.596 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:28:31.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:28:31.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:28:31.596 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.3", 2023-10-06T02:28:31.596 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBZbx9lzsi4ARAAbGnNV0QyL/7itzy827+chg==", 2023-10-06T02:28:31.597 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:28:31.597 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:28:31.597 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:28:31.597 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:28:31.597 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:28:31.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:28:31.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:28:31.597 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.4", 2023-10-06T02:28:31.597 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBfbx9lW4OSEhAAj7HnVh/yq12zzCaXd6BHCA==", 2023-10-06T02:28:31.597 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:28:31.597 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:28:31.597 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:28:31.597 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:28:31.598 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:28:31.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:28:31.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:28:31.598 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.5", 2023-10-06T02:28:31.598 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBibx9lBcaIFRAAV8wq8QjsCOkI9Pzf/E+9bA==", 2023-10-06T02:28:31.598 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:28:31.598 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:28:31.598 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:28:31.598 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:28:31.598 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:28:31.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:28:31.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:28:31.599 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.6", 2023-10-06T02:28:31.599 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBlbx9lA7S+FxAA38mlSAurT+DpwMzH53zFcw==", 2023-10-06T02:28:31.599 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:28:31.599 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:28:31.599 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:28:31.599 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:28:31.599 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:28:31.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:28:31.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:28:31.599 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.7", 2023-10-06T02:28:31.599 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBobx9l5RkdHhAAxn0Z8IAE/o9oe2LRp3O9gg==", 2023-10-06T02:28:31.599 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:28:31.600 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:28:31.600 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:28:31.600 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:28:31.600 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:28:31.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:28:31.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:28:31.600 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.0", 2023-10-06T02:28:31.600 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBMbx9lh44fCRAA7sMISvk1t7dEJg4ZcBiMcA==", 2023-10-06T02:28:31.600 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:28:31.600 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:28:31.600 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow r", 2023-10-06T02:28:31.600 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T02:28:31.600 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:28:31.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:28:31.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:28:31.601 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.1", 2023-10-06T02:28:31.601 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBMbx9lP0zqFhAAlov2y7ipSvS3AkOdQkhg6g==", 2023-10-06T02:28:31.601 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:28:31.601 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:28:31.601 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-10-06T02:28:31.601 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-10-06T02:28:31.601 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-10-06T02:28:31.601 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:28:31.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:28:31.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:28:31.601 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.2", 2023-10-06T02:28:31.602 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBMbx9lpxYAEBAACtiI1XgQXM8yGGcpFyNK1w==", 2023-10-06T02:28:31.602 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:28:31.602 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:28:31.602 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-10-06T02:28:31.602 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-10-06T02:28:31.602 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-10-06T02:28:31.602 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:28:31.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:28:31.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:28:31.602 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.3", 2023-10-06T02:28:31.602 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBMbx9l81/PHRAAYFOoaezu0pwE0NdtnlYfVA==", 2023-10-06T02:28:31.602 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:28:31.602 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:28:31.603 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-10-06T02:28:31.603 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-10-06T02:28:31.603 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-10-06T02:28:31.603 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:28:31.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:28:31.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:28:31.603 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.admin", 2023-10-06T02:28:31.603 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBKbx9liFl1JBAAFyyHYNLVx1/Ex5a2MKm8rA==", 2023-10-06T02:28:31.603 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:28:31.603 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-10-06T02:28:31.603 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T02:28:31.603 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T02:28:31.604 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:28:31.604 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:28:31.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:28:31.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:28:31.604 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mgr.x", 2023-10-06T02:28:31.604 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBLbx9lTBbIFBAAX2wsioKL4Nx4qz/qwsCsNA==", 2023-10-06T02:28:31.604 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:28:31.604 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-10-06T02:28:31.604 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile mgr", 2023-10-06T02:28:31.604 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:28:31.604 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:28:31.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:28:31.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:28:31.605 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mgr.y", 2023-10-06T02:28:31.605 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBLbx9l4+oLDRAA5YVo3q6FI9e9W/rJYROIsg==", 2023-10-06T02:28:31.605 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:28:31.605 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-10-06T02:28:31.605 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile mgr", 2023-10-06T02:28:31.605 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:28:31.605 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:28:31.605 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:28:31.605 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-10-06T02:28:31.605 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T02:28:31.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.591+0000 7f9c77fff700 1 -- 172.21.15.31:0/836356991 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9c64067050 msgr2=0x7f9c64069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:31.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.591+0000 7f9c77fff700 1 --2- 172.21.15.31:0/836356991 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9c64067050 0x7f9c64069500 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9c7806f770 tx=0x7f9c6c017000 comp rx=0 tx=0).stop 2023-10-06T02:28:31.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.591+0000 7f9c77fff700 1 -- 172.21.15.31:0/836356991 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c7806e0a0 msgr2=0x7f9c780b16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:31.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.591+0000 7f9c77fff700 1 --2- 172.21.15.31:0/836356991 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c7806e0a0 0x7f9c780b16f0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f9c6800d890 tx=0x7f9c68008040 comp rx=0 tx=0).stop 2023-10-06T02:28:31.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.591+0000 7f9c77fff700 1 -- 172.21.15.31:0/836356991 shutdown_connections 2023-10-06T02:28:31.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.591+0000 7f9c77fff700 1 --2- 172.21.15.31:0/836356991 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9c64067050 0x7f9c64069500 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:31.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.591+0000 7f9c77fff700 1 --2- 172.21.15.31:0/836356991 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9c7806f3a0 0x7f9c781588b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:31.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.591+0000 7f9c77fff700 1 --2- 172.21.15.31:0/836356991 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c7806e0a0 0x7f9c780b16f0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:31.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.591+0000 7f9c77fff700 1 --2- 172.21.15.31:0/836356991 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c7806e470 0x7f9c780affc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:31.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.591+0000 7f9c77fff700 1 -- 172.21.15.31:0/836356991 >> 172.21.15.31:0/836356991 conn(0x7f9c7806d170 msgr2=0x7f9c78159ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:31.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.592+0000 7f9c77fff700 1 -- 172.21.15.31:0/836356991 shutdown_connections 2023-10-06T02:28:31.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.592+0000 7f9c77fff700 1 -- 172.21.15.31:0/836356991 wait complete. 2023-10-06T02:28:31.611 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:28:31.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.837+0000 7f820c9fe700 1 Processor -- start 2023-10-06T02:28:31.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.838+0000 7f820c9fe700 1 -- start start 2023-10-06T02:28:31.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.839+0000 7f820c9fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f820813fb20 0x7f820815a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:31.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.839+0000 7f820c9fe700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f820815ae70 con 0x7f820806aaa0 2023-10-06T02:28:31.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.839+0000 7f820c9fe700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f820806af60 con 0x7f820813fef0 2023-10-06T02:28:31.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.839+0000 7f820c9fe700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f820806b240 con 0x7f820813fb20 2023-10-06T02:28:31.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.839+0000 7f8206d9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f820813fb20 0x7f820815a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:31.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.839+0000 7f820759e700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f820806aaa0 0x7f82080c9d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56724/0 (socket says 172.21.15.31:56724) 2023-10-06T02:28:31.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.839+0000 7f820759e700 1 -- 172.21.15.31:0/2341376206 learned_addr learned my addr 172.21.15.31:0/2341376206 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:28:31.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.840+0000 7f8206d9d700 1 -- 172.21.15.31:0/2341376206 >> v1:172.21.15.62:6789/0 conn(0x7f820813fef0 legacy=0x7f82080c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:31.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.840+0000 7f8205d9b700 1 -- 172.21.15.31:0/2341376206 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 438083269 0 0) 0x7f8200001490 con 0x7f820806aaa0 2023-10-06T02:28:31.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.840+0000 7f8206d9d700 1 -- 172.21.15.31:0/2341376206 >> v1:172.21.15.31:6789/0 conn(0x7f820806aaa0 legacy=0x7f82080c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:31.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.840+0000 7f8206d9d700 1 -- 172.21.15.31:0/2341376206 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f820806b520 con 0x7f820813fb20 2023-10-06T02:28:31.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.840+0000 7f8206d9d700 1 --2- 172.21.15.31:0/2341376206 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f820813fb20 0x7f820815a930 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f81f80098e0 tx=0x7f81f80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1030cb69a66c932 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:31.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.840+0000 7f8205d9b700 1 -- 172.21.15.31:0/2341376206 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f8009e80 con 0x7f820813fb20 2023-10-06T02:28:31.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.840+0000 7f8205d9b700 1 -- 172.21.15.31:0/2341376206 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81f800b860 con 0x7f820813fb20 2023-10-06T02:28:31.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.840+0000 7f8205d9b700 1 -- 172.21.15.31:0/2341376206 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f800bbe0 con 0x7f820813fb20 2023-10-06T02:28:31.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.841+0000 7f820c9fe700 1 -- 172.21.15.31:0/2341376206 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f820813fb20 msgr2=0x7f820815a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:31.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.841+0000 7f820c9fe700 1 --2- 172.21.15.31:0/2341376206 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f820813fb20 0x7f820815a930 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f81f80098e0 tx=0x7f81f80091a0 comp rx=0 tx=0).stop 2023-10-06T02:28:31.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.841+0000 7f820c9fe700 1 -- 172.21.15.31:0/2341376206 shutdown_connections 2023-10-06T02:28:31.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.841+0000 7f820c9fe700 1 --2- 172.21.15.31:0/2341376206 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f820813fb20 0x7f820815a930 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:31.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.841+0000 7f820c9fe700 1 -- 172.21.15.31:0/2341376206 >> 172.21.15.31:0/2341376206 conn(0x7f820806d170 msgr2=0x7f82080cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:31.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.841+0000 7f820c9fe700 1 -- 172.21.15.31:0/2341376206 shutdown_connections 2023-10-06T02:28:31.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.841+0000 7f820c9fe700 1 -- 172.21.15.31:0/2341376206 wait complete. 2023-10-06T02:28:31.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.842+0000 7f820c9fe700 1 Processor -- start 2023-10-06T02:28:31.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.842+0000 7f820c9fe700 1 -- start start 2023-10-06T02:28:31.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.843+0000 7f820c9fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f820806aaa0 0x7f82081373f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:31.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.843+0000 7f820c9fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f820813fb20 0x7f82080ac710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:31.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.843+0000 7f820c9fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f820813fef0 0x7f82080acc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:31.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.843+0000 7f820c9fe700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82081580a0 con 0x7f820813fef0 2023-10-06T02:28:31.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.843+0000 7f820c9fe700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8208157490 con 0x7f820813fb20 2023-10-06T02:28:31.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.843+0000 7f820c9fe700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8208157770 con 0x7f820806aaa0 2023-10-06T02:28:31.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.844+0000 7f820759e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f820813fef0 0x7f82080acc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:31.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.844+0000 7f820759e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f820813fef0 0x7f82080acc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40550/0 (socket says 172.21.15.31:40550) 2023-10-06T02:28:31.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.844+0000 7f820759e700 1 -- 172.21.15.31:0/6880206 learned_addr learned my addr 172.21.15.31:0/6880206 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:31.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.844+0000 7f8206d9d700 1 --2- 172.21.15.31:0/6880206 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f820806aaa0 0x7f82081373f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:31.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.844+0000 7f820659c700 1 --2- 172.21.15.31:0/6880206 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f820813fb20 0x7f82080ac710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:31.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.844+0000 7f820759e700 1 -- 172.21.15.31:0/6880206 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f820806aaa0 msgr2=0x7f82081373f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:31.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.844+0000 7f820759e700 1 --2- 172.21.15.31:0/6880206 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f820806aaa0 0x7f82081373f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:31.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.844+0000 7f820759e700 1 -- 172.21.15.31:0/6880206 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f820813fb20 msgr2=0x7f82080ac710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:31.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.844+0000 7f820759e700 1 --2- 172.21.15.31:0/6880206 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f820813fb20 0x7f82080ac710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:31.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.844+0000 7f820759e700 1 -- 172.21.15.31:0/6880206 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81f8009420 con 0x7f820813fef0 2023-10-06T02:28:31.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.844+0000 7f8206d9d700 1 --2- 172.21.15.31:0/6880206 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f820806aaa0 0x7f82081373f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:28:31.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.844+0000 7f820659c700 1 --2- 172.21.15.31:0/6880206 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f820813fb20 0x7f82080ac710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:28:31.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.844+0000 7f820759e700 1 --2- 172.21.15.31:0/6880206 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f820813fef0 0x7f82080acc50 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f820000cbc0 tx=0x7f8200001980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:31.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.845+0000 7f81eeffd700 1 -- 172.21.15.31:0/6880206 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f820000a940 con 0x7f820813fef0 2023-10-06T02:28:31.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.845+0000 7f81eeffd700 1 -- 172.21.15.31:0/6880206 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f820000aaa0 con 0x7f820813fef0 2023-10-06T02:28:31.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.845+0000 7f81eeffd700 1 -- 172.21.15.31:0/6880206 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f820000ae60 con 0x7f820813fef0 2023-10-06T02:28:31.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.845+0000 7f820c9fe700 1 -- 172.21.15.31:0/6880206 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8208157a50 con 0x7f820813fef0 2023-10-06T02:28:31.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.845+0000 7f820c9fe700 1 -- 172.21.15.31:0/6880206 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82080adf50 con 0x7f820813fef0 2023-10-06T02:28:31.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.846+0000 7f820c9fe700 1 -- 172.21.15.31:0/6880206 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8208158250 con 0x7f820813fef0 2023-10-06T02:28:31.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.847+0000 7f81eeffd700 1 -- 172.21.15.31:0/6880206 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f8200007300 con 0x7f820813fef0 2023-10-06T02:28:31.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.848+0000 7f81eeffd700 1 --2- 172.21.15.31:0/6880206 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f81f4067050 0x7f81f4069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:31.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.848+0000 7f81eeffd700 1 -- 172.21.15.31:0/6880206 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f8200084760 con 0x7f820813fef0 2023-10-06T02:28:31.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.848+0000 7f8206d9d700 1 --2- 172.21.15.31:0/6880206 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f81f4067050 0x7f81f4069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:31.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.849+0000 7f8206d9d700 1 --2- 172.21.15.31:0/6880206 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f81f4067050 0x7f81f4069500 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f81f80098e0 tx=0x7f81f8017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:31.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:31.852+0000 7f81eeffd700 1 -- 172.21.15.31:0/6880206 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8208158250 con 0x7f820813fef0 2023-10-06T02:28:32.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.054+0000 7f820c9fe700 1 -- 172.21.15.31:0/6880206 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f820815ae70 con 0x7f820813fef0 2023-10-06T02:28:32.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.055+0000 7f81eeffd700 1 -- 172.21.15.31:0/6880206 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) v1 ==== 74+0+13604 (secure 0 0 0) 0x7f820815ae70 con 0x7f820813fef0 2023-10-06T02:28:32.057 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:28:32.057 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":102,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:28:31.089272+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:28:32.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.057+0000 7f820c9fe700 1 -- 172.21.15.31:0/6880206 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f81f4067050 msgr2=0x7f81f4069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:32.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.057+0000 7f820c9fe700 1 --2- 172.21.15.31:0/6880206 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f81f4067050 0x7f81f4069500 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f81f80098e0 tx=0x7f81f8017000 comp rx=0 tx=0).stop 2023-10-06T02:28:32.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.057+0000 7f820c9fe700 1 -- 172.21.15.31:0/6880206 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f820813fef0 msgr2=0x7f82080acc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:32.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.057+0000 7f820c9fe700 1 --2- 172.21.15.31:0/6880206 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f820813fef0 0x7f82080acc50 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f820000cbc0 tx=0x7f8200001980 comp rx=0 tx=0).stop 2023-10-06T02:28:32.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.058+0000 7f820c9fe700 1 -- 172.21.15.31:0/6880206 shutdown_connections 2023-10-06T02:28:32.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.058+0000 7f820c9fe700 1 --2- 172.21.15.31:0/6880206 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f820813fef0 0x7f82080acc50 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:32.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.058+0000 7f820c9fe700 1 --2- 172.21.15.31:0/6880206 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f81f4067050 0x7f81f4069500 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:32.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.058+0000 7f820c9fe700 1 --2- 172.21.15.31:0/6880206 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f820806aaa0 0x7f82081373f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:32.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.058+0000 7f820c9fe700 1 --2- 172.21.15.31:0/6880206 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f820813fb20 0x7f82080ac710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:32.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.058+0000 7f820c9fe700 1 -- 172.21.15.31:0/6880206 >> 172.21.15.31:0/6880206 conn(0x7f820806d170 msgr2=0x7f82080cc810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:32.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.058+0000 7f820c9fe700 1 -- 172.21.15.31:0/6880206 shutdown_connections 2023-10-06T02:28:32.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.058+0000 7f820c9fe700 1 -- 172.21.15.31:0/6880206 wait complete. 2023-10-06T02:28:32.078 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T02:22:53.183425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T02:23:00.046967+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-10-06T02:28:32.078 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T02:28:32.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.268+0000 7fd4b179d700 1 Processor -- start 2023-10-06T02:28:32.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.268+0000 7fd4b179d700 1 -- start start 2023-10-06T02:28:32.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.269+0000 7fd4b179d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd4ac157420 0x7fd4ac0b13b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:32.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.269+0000 7fd4b179d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd4ac0af6b0 0x7fd4ac0accd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:32.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.269+0000 7fd4b179d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4ac0ac840 con 0x7fd4ac06d250 2023-10-06T02:28:32.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.269+0000 7fd4b179d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4ac0b18f0 con 0x7fd4ac157420 2023-10-06T02:28:32.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.269+0000 7fd4b179d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ac0abf50 con 0x7fd4ac0af6b0 2023-10-06T02:28:32.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.270+0000 7fd4ab7fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fd4ac06d250 0x7fd4ac0aefa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56732/0 (socket says 172.21.15.31:56732) 2023-10-06T02:28:32.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.270+0000 7fd4ab7fe700 1 -- 172.21.15.31:0/1849938542 learned_addr learned my addr 172.21.15.31:0/1849938542 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:28:32.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.270+0000 7fd4aaffd700 1 -- 172.21.15.31:0/1849938542 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3051559362 0 0) 0x7fd4ac0ac840 con 0x7fd4ac06d250 2023-10-06T02:28:32.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.270+0000 7fd4aaffd700 1 -- 172.21.15.31:0/1849938542 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd494003570 con 0x7fd4ac06d250 2023-10-06T02:28:32.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.270+0000 7fd4b0f9c700 1 --2- 172.21.15.31:0/1849938542 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd4ac0af6b0 0x7fd4ac0accd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:32.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.270+0000 7fd4abfff700 1 --2- 172.21.15.31:0/1849938542 >> v2:172.21.15.62:3300/0 conn(0x7fd4ac157420 0x7fd4ac0b13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:32.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.270+0000 7fd4aaffd700 1 -- 172.21.15.31:0/1849938542 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3407179664 0 0) 0x7fd494003570 con 0x7fd4ac06d250 2023-10-06T02:28:32.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.270+0000 7fd4aaffd700 1 -- 172.21.15.31:0/1849938542 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd4ac0af6b0 msgr2=0x7fd4ac0accd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:32.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.270+0000 7fd4aaffd700 1 --2- 172.21.15.31:0/1849938542 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd4ac0af6b0 0x7fd4ac0accd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:32.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.270+0000 7fd4aaffd700 1 -- 172.21.15.31:0/1849938542 >> v2:172.21.15.62:3300/0 conn(0x7fd4ac157420 msgr2=0x7fd4ac0b13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:32.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.270+0000 7fd4aaffd700 1 --2- 172.21.15.31:0/1849938542 >> v2:172.21.15.62:3300/0 conn(0x7fd4ac157420 0x7fd4ac0b13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:32.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.270+0000 7fd4aaffd700 1 -- 172.21.15.31:0/1849938542 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4ac0aea60 con 0x7fd4ac06d250 2023-10-06T02:28:32.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.270+0000 7fd4aaffd700 1 -- 172.21.15.31:0/1849938542 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7fd4a0001470 con 0x7fd4ac06d250 2023-10-06T02:28:32.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.270+0000 7fd4aaffd700 1 -- 172.21.15.31:0/1849938542 >> v1:172.21.15.31:6789/0 conn(0x7fd4ac06d250 legacy=0x7fd4ac0aefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:32.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.270+0000 7fd4aaffd700 1 --2- 172.21.15.31:0/1849938542 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd4ac0af6b0 0x7fd494003ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:32.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.271+0000 7fd4aaffd700 1 --2- 172.21.15.31:0/1849938542 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd494004160 0x7fd494004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:32.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.271+0000 7fd4aaffd700 1 --2- 172.21.15.31:0/1849938542 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd494004d50 0x7fd4940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:32.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.271+0000 7fd4aaffd700 1 -- 172.21.15.31:0/1849938542 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ac0b18f0 con 0x7fd494004d50 2023-10-06T02:28:32.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.271+0000 7fd4aaffd700 1 -- 172.21.15.31:0/1849938542 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ac0abf50 con 0x7fd494004160 2023-10-06T02:28:32.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.271+0000 7fd4aaffd700 1 -- 172.21.15.31:0/1849938542 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd49400bd20 con 0x7fd4ac0af6b0 2023-10-06T02:28:32.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.271+0000 7fd4b0f9c700 1 --2- 172.21.15.31:0/1849938542 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd494004d50 0x7fd4940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:32.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.271+0000 7fd4abfff700 1 --2- 172.21.15.31:0/1849938542 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd4ac0af6b0 0x7fd494003ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:32.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.271+0000 7fd4ab7fe700 1 --2- 172.21.15.31:0/1849938542 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd494004160 0x7fd494004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:32.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.271+0000 7fd4b0f9c700 1 -- 172.21.15.31:0/1849938542 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd4ac0af6b0 msgr2=0x7fd494003ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:32.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.271+0000 7fd4b0f9c700 1 --2- 172.21.15.31:0/1849938542 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd4ac0af6b0 0x7fd494003ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:32.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.271+0000 7fd4b0f9c700 1 -- 172.21.15.31:0/1849938542 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd494004160 msgr2=0x7fd494004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:32.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.271+0000 7fd4b0f9c700 1 --2- 172.21.15.31:0/1849938542 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd494004160 0x7fd494004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:32.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.271+0000 7fd4b0f9c700 1 -- 172.21.15.31:0/1849938542 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd494003570 con 0x7fd494004d50 2023-10-06T02:28:32.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.272+0000 7fd4b0f9c700 1 --2- 172.21.15.31:0/1849938542 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd494004d50 0x7fd4940051c0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac0afab0 tx=0x7fd4a4009730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd7438075c476be7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:32.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.272+0000 7fd4aaffd700 1 -- 172.21.15.31:0/1849938542 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a4009a40 con 0x7fd494004d50 2023-10-06T02:28:32.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.272+0000 7fd4aaffd700 1 -- 172.21.15.31:0/1849938542 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4a4007c90 con 0x7fd494004d50 2023-10-06T02:28:32.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.272+0000 7fd4aaffd700 1 -- 172.21.15.31:0/1849938542 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a4014690 con 0x7fd494004d50 2023-10-06T02:28:32.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.272+0000 7fd4b179d700 1 -- 172.21.15.31:0/1849938542 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd494004d50 msgr2=0x7fd4940051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:32.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.272+0000 7fd4b179d700 1 --2- 172.21.15.31:0/1849938542 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd494004d50 0x7fd4940051c0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac0afab0 tx=0x7fd4a4009730 comp rx=0 tx=0).stop 2023-10-06T02:28:32.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.272+0000 7fd4b0f9c700 1 -- 172.21.15.31:0/1849938542 reap_dead start 2023-10-06T02:28:32.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.272+0000 7fd4b179d700 1 -- 172.21.15.31:0/1849938542 shutdown_connections 2023-10-06T02:28:32.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.272+0000 7fd4b179d700 1 -- 172.21.15.31:0/1849938542 >> 172.21.15.31:0/1849938542 conn(0x7fd4ac06ce80 msgr2=0x7fd4ac0ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:32.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.272+0000 7fd4b179d700 1 -- 172.21.15.31:0/1849938542 shutdown_connections 2023-10-06T02:28:32.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.272+0000 7fd4b179d700 1 -- 172.21.15.31:0/1849938542 wait complete. 2023-10-06T02:28:32.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.274+0000 7fd4b179d700 1 Processor -- start 2023-10-06T02:28:32.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.274+0000 7fd4b179d700 1 -- start start 2023-10-06T02:28:32.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.275+0000 7fd4b179d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd494004d50 0x7fd4ac152af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:32.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.275+0000 7fd4b179d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd4ac098040 0x7fd4ac156820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:32.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.275+0000 7fd4b179d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4ac1698a0 0x7fd4ac169d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:32.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.275+0000 7fd4b179d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ac09afd0 con 0x7fd494004d50 2023-10-06T02:28:32.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.275+0000 7fd4b179d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ac0985b0 con 0x7fd4ac1698a0 2023-10-06T02:28:32.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.275+0000 7fd4b179d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ac09a980 con 0x7fd4ac098040 2023-10-06T02:28:32.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.275+0000 7fd4abfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd494004d50 0x7fd4ac152af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:32.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.275+0000 7fd4abfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd494004d50 0x7fd4ac152af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40556/0 (socket says 172.21.15.31:40556) 2023-10-06T02:28:32.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.275+0000 7fd4abfff700 1 -- 172.21.15.31:0/4176629913 learned_addr learned my addr 172.21.15.31:0/4176629913 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:32.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.275+0000 7fd4ab7fe700 1 --2- 172.21.15.31:0/4176629913 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd4ac098040 0x7fd4ac156820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:32.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.275+0000 7fd4abfff700 1 -- 172.21.15.31:0/4176629913 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd4ac098040 msgr2=0x7fd4ac156820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:32.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.275+0000 7fd4abfff700 1 --2- 172.21.15.31:0/4176629913 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd4ac098040 0x7fd4ac156820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:32.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.275+0000 7fd4abfff700 1 -- 172.21.15.31:0/4176629913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4ac1698a0 msgr2=0x7fd4ac169d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:32.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.275+0000 7fd4abfff700 1 --2- 172.21.15.31:0/4176629913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4ac1698a0 0x7fd4ac169d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:32.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.275+0000 7fd4abfff700 1 -- 172.21.15.31:0/4176629913 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4ac09ac60 con 0x7fd494004d50 2023-10-06T02:28:32.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.275+0000 7fd4ab7fe700 1 --2- 172.21.15.31:0/4176629913 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd4ac098040 0x7fd4ac156820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:28:32.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.276+0000 7fd4abfff700 1 --2- 172.21.15.31:0/4176629913 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd494004d50 0x7fd4ac152af0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fd49c0109f0 tx=0x7fd49c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:32.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.276+0000 7fd493fff700 1 -- 172.21.15.31:0/4176629913 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd49c00e8d0 con 0x7fd494004d50 2023-10-06T02:28:32.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.276+0000 7fd493fff700 1 -- 172.21.15.31:0/4176629913 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd49c00ea30 con 0x7fd494004d50 2023-10-06T02:28:32.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.276+0000 7fd493fff700 1 -- 172.21.15.31:0/4176629913 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd49c00ee30 con 0x7fd494004d50 2023-10-06T02:28:32.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.276+0000 7fd4b179d700 1 -- 172.21.15.31:0/4176629913 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4ac16a250 con 0x7fd494004d50 2023-10-06T02:28:32.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.276+0000 7fd4b179d700 1 -- 172.21.15.31:0/4176629913 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4ac153270 con 0x7fd494004d50 2023-10-06T02:28:32.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.277+0000 7fd4b179d700 1 -- 172.21.15.31:0/4176629913 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4ac09b360 con 0x7fd494004d50 2023-10-06T02:28:32.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.278+0000 7fd493fff700 1 -- 172.21.15.31:0/4176629913 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7fd49c014640 con 0x7fd494004d50 2023-10-06T02:28:32.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.279+0000 7fd493fff700 1 --2- 172.21.15.31:0/4176629913 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd498067050 0x7fd498069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:32.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.279+0000 7fd493fff700 1 -- 172.21.15.31:0/4176629913 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fd49c084c20 con 0x7fd494004d50 2023-10-06T02:28:32.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.279+0000 7fd4ab7fe700 1 --2- 172.21.15.31:0/4176629913 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd498067050 0x7fd498069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:32.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.280+0000 7fd4ab7fe700 1 --2- 172.21.15.31:0/4176629913 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd498067050 0x7fd498069500 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd4a00010e0 tx=0x7fd4a0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:32.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.283+0000 7fd493fff700 1 -- 172.21.15.31:0/4176629913 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd49c051ed0 con 0x7fd494004d50 2023-10-06T02:28:32.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.484+0000 7fd4b179d700 1 -- 172.21.15.31:0/4176629913 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd4ac04c3c0 con 0x7fd494004d50 2023-10-06T02:28:32.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.484+0000 7fd493fff700 1 -- 172.21.15.31:0/4176629913 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v102) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd49c0180a0 con 0x7fd494004d50 2023-10-06T02:28:32.486 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-10-06T02:28:32.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.486+0000 7fd4b179d700 1 -- 172.21.15.31:0/4176629913 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd498067050 msgr2=0x7fd498069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:32.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.486+0000 7fd4b179d700 1 --2- 172.21.15.31:0/4176629913 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd498067050 0x7fd498069500 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd4a00010e0 tx=0x7fd4a0009040 comp rx=0 tx=0).stop 2023-10-06T02:28:32.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.486+0000 7fd4b179d700 1 -- 172.21.15.31:0/4176629913 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd494004d50 msgr2=0x7fd4ac152af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:32.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.486+0000 7fd4b179d700 1 --2- 172.21.15.31:0/4176629913 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd494004d50 0x7fd4ac152af0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fd49c0109f0 tx=0x7fd49c00b040 comp rx=0 tx=0).stop 2023-10-06T02:28:32.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.486+0000 7fd4b179d700 1 -- 172.21.15.31:0/4176629913 shutdown_connections 2023-10-06T02:28:32.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.486+0000 7fd4b179d700 1 --2- 172.21.15.31:0/4176629913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4ac1698a0 0x7fd4ac169d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:32.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.486+0000 7fd4b179d700 1 --2- 172.21.15.31:0/4176629913 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd494004d50 0x7fd4ac152af0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:32.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.486+0000 7fd4b179d700 1 --2- 172.21.15.31:0/4176629913 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd498067050 0x7fd498069500 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:32.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.486+0000 7fd4b179d700 1 --2- 172.21.15.31:0/4176629913 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd4ac098040 0x7fd4ac156820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:32.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.486+0000 7fd4b179d700 1 -- 172.21.15.31:0/4176629913 >> 172.21.15.31:0/4176629913 conn(0x7fd4ac06ce80 msgr2=0x7fd4ac067d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:32.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.487+0000 7fd4b179d700 1 -- 172.21.15.31:0/4176629913 shutdown_connections 2023-10-06T02:28:32.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.487+0000 7fd4b179d700 1 -- 172.21.15.31:0/4176629913 wait complete. 2023-10-06T02:28:32.505 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T02:28:32.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.694+0000 7f6f8bfff700 1 Processor -- start 2023-10-06T02:28:32.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.695+0000 7f6f8bfff700 1 -- start start 2023-10-06T02:28:32.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.696+0000 7f6f8bfff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6f8c0c7be0 0x7f6f8c06abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:32.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.696+0000 7f6f8bfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6f8c13f780 0x7f6f8c13fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:32.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.696+0000 7f6f8bfff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f8c06b290 con 0x7f6f8c0c7fb0 2023-10-06T02:28:32.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.696+0000 7f6f8bfff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f8c06bf10 con 0x7f6f8c13f780 2023-10-06T02:28:32.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.696+0000 7f6f8bfff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f8c140140 con 0x7f6f8c0c7be0 2023-10-06T02:28:32.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.696+0000 7f6f8a7fc700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f6f8c0c7fb0 0x7f6f8c15a990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56744/0 (socket says 172.21.15.31:56744) 2023-10-06T02:28:32.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.696+0000 7f6f8a7fc700 1 -- 172.21.15.31:0/3728009929 learned_addr learned my addr 172.21.15.31:0/3728009929 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:28:32.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.696+0000 7f6f8b7fe700 1 --2- 172.21.15.31:0/3728009929 >> v2:172.21.15.31:3300/0 conn(0x7f6f8c13f780 0x7f6f8c13fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:32.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.696+0000 7f6f8affd700 1 --2- 172.21.15.31:0/3728009929 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6f8c0c7be0 0x7f6f8c06abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:32.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.697+0000 7f6f89ffb700 1 -- 172.21.15.31:0/3728009929 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 721150701 0 0) 0x7f6f8c06b290 con 0x7f6f8c0c7fb0 2023-10-06T02:28:32.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.697+0000 7f6f89ffb700 1 -- 172.21.15.31:0/3728009929 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f74003570 con 0x7f6f8c0c7fb0 2023-10-06T02:28:32.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.697+0000 7f6f8b7fe700 1 -- 172.21.15.31:0/3728009929 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6f8c0c7be0 msgr2=0x7f6f8c06abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:32.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.697+0000 7f6f8b7fe700 1 --2- 172.21.15.31:0/3728009929 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6f8c0c7be0 0x7f6f8c06abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:32.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.697+0000 7f6f8b7fe700 1 -- 172.21.15.31:0/3728009929 >> v1:172.21.15.31:6789/0 conn(0x7f6f8c0c7fb0 legacy=0x7f6f8c15a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:32.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.697+0000 7f6f8b7fe700 1 -- 172.21.15.31:0/3728009929 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f8c06b570 con 0x7f6f8c13f780 2023-10-06T02:28:32.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.697+0000 7f6f8b7fe700 1 --2- 172.21.15.31:0/3728009929 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f8c13f780 0x7f6f8c13fc00 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f6f84009e90 tx=0x7f6f84009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f31e4ea3c90472ec server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:32.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.697+0000 7f6f89ffb700 1 -- 172.21.15.31:0/3728009929 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f8400e070 con 0x7f6f8c13f780 2023-10-06T02:28:32.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.697+0000 7f6f89ffb700 1 -- 172.21.15.31:0/3728009929 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f8400bb40 con 0x7f6f8c13f780 2023-10-06T02:28:32.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.697+0000 7f6f89ffb700 1 -- 172.21.15.31:0/3728009929 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f84012430 con 0x7f6f8c13f780 2023-10-06T02:28:32.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.698+0000 7f6f8bfff700 1 -- 172.21.15.31:0/3728009929 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f8c13f780 msgr2=0x7f6f8c13fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:32.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.698+0000 7f6f8bfff700 1 --2- 172.21.15.31:0/3728009929 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f8c13f780 0x7f6f8c13fc00 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f6f84009e90 tx=0x7f6f84009780 comp rx=0 tx=0).stop 2023-10-06T02:28:32.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.698+0000 7f6f8bfff700 1 -- 172.21.15.31:0/3728009929 shutdown_connections 2023-10-06T02:28:32.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.698+0000 7f6f8bfff700 1 --2- 172.21.15.31:0/3728009929 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f8c13f780 0x7f6f8c13fc00 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:32.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.698+0000 7f6f8bfff700 1 --2- 172.21.15.31:0/3728009929 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6f8c0c7be0 0x7f6f8c06abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:32.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.698+0000 7f6f8bfff700 1 -- 172.21.15.31:0/3728009929 >> 172.21.15.31:0/3728009929 conn(0x7f6f8c06d110 msgr2=0x7f6f8c0cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:32.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.698+0000 7f6f8bfff700 1 -- 172.21.15.31:0/3728009929 shutdown_connections 2023-10-06T02:28:32.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.698+0000 7f6f8bfff700 1 -- 172.21.15.31:0/3728009929 wait complete. 2023-10-06T02:28:32.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.699+0000 7f6f8bfff700 1 Processor -- start 2023-10-06T02:28:32.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.700+0000 7f6f8bfff700 1 -- start start 2023-10-06T02:28:32.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.701+0000 7f6f8bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f8c0c7be0 0x7f6f8c155d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:32.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.701+0000 7f6f8bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f8c0c7fb0 0x7f6f8c0c5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:32.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.701+0000 7f6f8bfff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6f8c13f780 0x7f6f8c1588a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:32.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.701+0000 7f6f8bfff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f8c157e50 con 0x7f6f8c0c7fb0 2023-10-06T02:28:32.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.701+0000 7f6f8bfff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f8c1572d0 con 0x7f6f8c0c7be0 2023-10-06T02:28:32.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.701+0000 7f6f8bfff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f8c1575b0 con 0x7f6f8c13f780 2023-10-06T02:28:32.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.701+0000 7f6f8a7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f8c0c7fb0 0x7f6f8c0c5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:32.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.701+0000 7f6f8a7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f8c0c7fb0 0x7f6f8c0c5d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40578/0 (socket says 172.21.15.31:40578) 2023-10-06T02:28:32.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.701+0000 7f6f8a7fc700 1 -- 172.21.15.31:0/2655727320 learned_addr learned my addr 172.21.15.31:0/2655727320 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:32.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.701+0000 7f6f8affd700 1 --2- 172.21.15.31:0/2655727320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f8c0c7be0 0x7f6f8c155d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:32.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.701+0000 7f6f8b7fe700 1 --2- 172.21.15.31:0/2655727320 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6f8c13f780 0x7f6f8c1588a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:32.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.701+0000 7f6f8a7fc700 1 -- 172.21.15.31:0/2655727320 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6f8c13f780 msgr2=0x7f6f8c1588a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:32.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.701+0000 7f6f8a7fc700 1 --2- 172.21.15.31:0/2655727320 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6f8c13f780 0x7f6f8c1588a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:32.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.701+0000 7f6f8a7fc700 1 -- 172.21.15.31:0/2655727320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f8c0c7be0 msgr2=0x7f6f8c155d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:32.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.701+0000 7f6f8a7fc700 1 --2- 172.21.15.31:0/2655727320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f8c0c7be0 0x7f6f8c155d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:32.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.701+0000 7f6f8a7fc700 1 -- 172.21.15.31:0/2655727320 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f84009a20 con 0x7f6f8c0c7fb0 2023-10-06T02:28:32.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.701+0000 7f6f8b7fe700 1 --2- 172.21.15.31:0/2655727320 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6f8c13f780 0x7f6f8c1588a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:28:32.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.702+0000 7f6f8a7fc700 1 --2- 172.21.15.31:0/2655727320 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f8c0c7fb0 0x7f6f8c0c5d30 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f6f8000c7c0 tx=0x7f6f80007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:32.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.702+0000 7f6f72ffd700 1 -- 172.21.15.31:0/2655727320 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f80005c00 con 0x7f6f8c0c7fb0 2023-10-06T02:28:32.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.702+0000 7f6f72ffd700 1 -- 172.21.15.31:0/2655727320 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f80005d60 con 0x7f6f8c0c7fb0 2023-10-06T02:28:32.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.702+0000 7f6f8bfff700 1 -- 172.21.15.31:0/2655727320 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f8c157890 con 0x7f6f8c0c7fb0 2023-10-06T02:28:32.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.702+0000 7f6f72ffd700 1 -- 172.21.15.31:0/2655727320 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f80007320 con 0x7f6f8c0c7fb0 2023-10-06T02:28:32.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.702+0000 7f6f8bfff700 1 -- 172.21.15.31:0/2655727320 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f8c0c3310 con 0x7f6f8c0c7fb0 2023-10-06T02:28:32.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.703+0000 7f6f8bfff700 1 -- 172.21.15.31:0/2655727320 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f8c0c3530 con 0x7f6f8c0c7fb0 2023-10-06T02:28:32.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.704+0000 7f6f72ffd700 1 -- 172.21.15.31:0/2655727320 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f6f80007690 con 0x7f6f8c0c7fb0 2023-10-06T02:28:32.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.704+0000 7f6f72ffd700 1 --2- 172.21.15.31:0/2655727320 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6f78067050 0x7f6f78069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:32.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.705+0000 7f6f72ffd700 1 -- 172.21.15.31:0/2655727320 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f6f80083ff0 con 0x7f6f8c0c7fb0 2023-10-06T02:28:32.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.705+0000 7f6f8affd700 1 --2- 172.21.15.31:0/2655727320 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6f78067050 0x7f6f78069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:32.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.706+0000 7f6f8affd700 1 --2- 172.21.15.31:0/2655727320 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6f78067050 0x7f6f78069500 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c009d40 tx=0x7f6f7c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:32.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.709+0000 7f6f72ffd700 1 -- 172.21.15.31:0/2655727320 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6f80051320 con 0x7f6f8c0c7fb0 2023-10-06T02:28:32.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.910+0000 7f6f8bfff700 1 -- 172.21.15.31:0/2655727320 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f6f8c04c3c0 con 0x7f6f8c0c7fb0 2023-10-06T02:28:32.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.910+0000 7f6f72ffd700 1 -- 172.21.15.31:0/2655727320 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v102) v1 ==== 92+0+10 (secure 0 0 0) 0x7f6f80054940 con 0x7f6f8c0c7fb0 2023-10-06T02:28:32.912 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 8 2023-10-06T02:28:32.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.913+0000 7f6f8bfff700 1 -- 172.21.15.31:0/2655727320 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6f78067050 msgr2=0x7f6f78069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:32.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.913+0000 7f6f8bfff700 1 --2- 172.21.15.31:0/2655727320 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6f78067050 0x7f6f78069500 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c009d40 tx=0x7f6f7c008040 comp rx=0 tx=0).stop 2023-10-06T02:28:32.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.913+0000 7f6f8bfff700 1 -- 172.21.15.31:0/2655727320 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f8c0c7fb0 msgr2=0x7f6f8c0c5d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:32.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.913+0000 7f6f8bfff700 1 --2- 172.21.15.31:0/2655727320 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f8c0c7fb0 0x7f6f8c0c5d30 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f6f8000c7c0 tx=0x7f6f80007dd0 comp rx=0 tx=0).stop 2023-10-06T02:28:32.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.913+0000 7f6f8bfff700 1 -- 172.21.15.31:0/2655727320 shutdown_connections 2023-10-06T02:28:32.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.913+0000 7f6f8bfff700 1 --2- 172.21.15.31:0/2655727320 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6f78067050 0x7f6f78069500 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:32.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.913+0000 7f6f8bfff700 1 --2- 172.21.15.31:0/2655727320 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6f8c13f780 0x7f6f8c1588a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:32.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.913+0000 7f6f8bfff700 1 --2- 172.21.15.31:0/2655727320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f8c0c7be0 0x7f6f8c155d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:32.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.913+0000 7f6f8bfff700 1 --2- 172.21.15.31:0/2655727320 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f8c0c7fb0 0x7f6f8c0c5d30 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:32.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.913+0000 7f6f8bfff700 1 -- 172.21.15.31:0/2655727320 >> 172.21.15.31:0/2655727320 conn(0x7f6f8c06d110 msgr2=0x7f6f8c0cad50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:32.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.913+0000 7f6f8bfff700 1 -- 172.21.15.31:0/2655727320 shutdown_connections 2023-10-06T02:28:32.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:32.913+0000 7f6f8bfff700 1 -- 172.21.15.31:0/2655727320 wait complete. 2023-10-06T02:28:32.932 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T02:28:33.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.117+0000 7fe5f139d700 1 Processor -- start 2023-10-06T02:28:33.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.117+0000 7fe5f139d700 1 -- start start 2023-10-06T02:28:33.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.118+0000 7fe5f139d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe5ec159a10 0x7fe5ec0ca7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:33.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.118+0000 7fe5f139d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5ec0c7420 con 0x7fe5ec159640 2023-10-06T02:28:33.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.118+0000 7fe5f139d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5ec06d820 con 0x7fe5ec070410 2023-10-06T02:28:33.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.118+0000 7fe5f139d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5ec06db00 con 0x7fe5ec159a10 2023-10-06T02:28:33.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.119+0000 7fe5ebfff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fe5ec159640 0x7fe5ec06d060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56752/0 (socket says 172.21.15.31:56752) 2023-10-06T02:28:33.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.119+0000 7fe5ebfff700 1 -- 172.21.15.31:0/1426423079 learned_addr learned my addr 172.21.15.31:0/1426423079 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:28:33.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.119+0000 7fe5eb7fe700 1 --2- 172.21.15.31:0/1426423079 >> v2:172.21.15.62:3300/0 conn(0x7fe5ec159a10 0x7fe5ec0ca7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:33.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.119+0000 7fe5eaffd700 1 -- 172.21.15.31:0/1426423079 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3985785391 0 0) 0x7fe5ec0c7420 con 0x7fe5ec159640 2023-10-06T02:28:33.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.119+0000 7fe5eaffd700 1 -- 172.21.15.31:0/1426423079 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5d4003570 con 0x7fe5ec159640 2023-10-06T02:28:33.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.119+0000 7fe5eb7fe700 1 -- 172.21.15.31:0/1426423079 >> v1:172.21.15.62:6789/0 conn(0x7fe5ec070410 legacy=0x7fe5ec0c78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:33.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.119+0000 7fe5eaffd700 1 -- 172.21.15.31:0/1426423079 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1403070955 0 0) 0x7fe5d4003570 con 0x7fe5ec159640 2023-10-06T02:28:33.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.119+0000 7fe5eb7fe700 1 -- 172.21.15.31:0/1426423079 >> v1:172.21.15.31:6789/0 conn(0x7fe5ec159640 legacy=0x7fe5ec06d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:33.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.119+0000 7fe5eb7fe700 1 -- 172.21.15.31:0/1426423079 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5ec06dde0 con 0x7fe5ec159a10 2023-10-06T02:28:33.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.120+0000 7fe5eb7fe700 1 --2- 172.21.15.31:0/1426423079 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5ec159a10 0x7fe5ec0ca7b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe5e00098e0 tx=0x7fe5e00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d99dea8cb6ecf0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:33.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.120+0000 7fe5eaffd700 1 -- 172.21.15.31:0/1426423079 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5dc001470 con 0x7fe5ec159a10 2023-10-06T02:28:33.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.120+0000 7fe5eaffd700 1 -- 172.21.15.31:0/1426423079 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5e0011420 con 0x7fe5ec159a10 2023-10-06T02:28:33.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.120+0000 7fe5eaffd700 1 -- 172.21.15.31:0/1426423079 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e00117c0 con 0x7fe5ec159a10 2023-10-06T02:28:33.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.120+0000 7fe5f139d700 1 -- 172.21.15.31:0/1426423079 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5ec159a10 msgr2=0x7fe5ec0ca7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:33.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.120+0000 7fe5f139d700 1 --2- 172.21.15.31:0/1426423079 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5ec159a10 0x7fe5ec0ca7b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe5e00098e0 tx=0x7fe5e00091a0 comp rx=0 tx=0).stop 2023-10-06T02:28:33.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.120+0000 7fe5f139d700 1 -- 172.21.15.31:0/1426423079 shutdown_connections 2023-10-06T02:28:33.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.120+0000 7fe5f139d700 1 --2- 172.21.15.31:0/1426423079 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5ec159a10 0x7fe5ec0ca7b0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:33.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.120+0000 7fe5f139d700 1 -- 172.21.15.31:0/1426423079 >> 172.21.15.31:0/1426423079 conn(0x7fe5ec06e720 msgr2=0x7fe5ec06eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:33.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.120+0000 7fe5f139d700 1 -- 172.21.15.31:0/1426423079 shutdown_connections 2023-10-06T02:28:33.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.121+0000 7fe5f139d700 1 -- 172.21.15.31:0/1426423079 wait complete. 2023-10-06T02:28:33.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.122+0000 7fe5f139d700 1 Processor -- start 2023-10-06T02:28:33.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.122+0000 7fe5f139d700 1 -- start start 2023-10-06T02:28:33.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.123+0000 7fe5f139d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5ec070410 0x7fe5ec0c3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:33.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.123+0000 7fe5f139d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe5ec159640 0x7fe5ec0c8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:33.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.123+0000 7fe5f139d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5ec159a10 0x7fe5ec154180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:33.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.123+0000 7fe5f139d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ec0b0530 con 0x7fe5ec070410 2023-10-06T02:28:33.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.123+0000 7fe5f139d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ec0af920 con 0x7fe5ec159a10 2023-10-06T02:28:33.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.123+0000 7fe5f139d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ec0afc00 con 0x7fe5ec159640 2023-10-06T02:28:33.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.123+0000 7fe5ebfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5ec070410 0x7fe5ec0c3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:33.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.123+0000 7fe5ebfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5ec070410 0x7fe5ec0c3a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40582/0 (socket says 172.21.15.31:40582) 2023-10-06T02:28:33.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.123+0000 7fe5ebfff700 1 -- 172.21.15.31:0/1322316974 learned_addr learned my addr 172.21.15.31:0/1322316974 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:33.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.123+0000 7fe5ebfff700 1 -- 172.21.15.31:0/1322316974 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe5ec159640 msgr2=0x7fe5ec0c8ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:33.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.123+0000 7fe5eb7fe700 1 --2- 172.21.15.31:0/1322316974 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe5ec159640 0x7fe5ec0c8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:33.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.123+0000 7fe5f0b9c700 1 --2- 172.21.15.31:0/1322316974 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5ec159a10 0x7fe5ec154180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:33.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.124+0000 7fe5ebfff700 1 --2- 172.21.15.31:0/1322316974 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe5ec159640 0x7fe5ec0c8ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:33.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.124+0000 7fe5ebfff700 1 -- 172.21.15.31:0/1322316974 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5ec159a10 msgr2=0x7fe5ec154180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:33.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.124+0000 7fe5ebfff700 1 --2- 172.21.15.31:0/1322316974 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5ec159a10 0x7fe5ec154180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:33.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.124+0000 7fe5ebfff700 1 -- 172.21.15.31:0/1322316974 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5e0009420 con 0x7fe5ec070410 2023-10-06T02:28:33.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.124+0000 7fe5eb7fe700 1 --2- 172.21.15.31:0/1322316974 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe5ec159640 0x7fe5ec0c8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:28:33.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.124+0000 7fe5f0b9c700 1 --2- 172.21.15.31:0/1322316974 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5ec159a10 0x7fe5ec154180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:28:33.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.124+0000 7fe5ebfff700 1 --2- 172.21.15.31:0/1322316974 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5ec070410 0x7fe5ec0c3a80 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc00cb90 tx=0x7fe5dc001ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:33.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.124+0000 7fe5d3fff700 1 -- 172.21.15.31:0/1322316974 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5dc007150 con 0x7fe5ec070410 2023-10-06T02:28:33.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.124+0000 7fe5d3fff700 1 -- 172.21.15.31:0/1322316974 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5dc0072d0 con 0x7fe5ec070410 2023-10-06T02:28:33.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.124+0000 7fe5f139d700 1 -- 172.21.15.31:0/1322316974 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5ec0afee0 con 0x7fe5ec070410 2023-10-06T02:28:33.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.124+0000 7fe5d3fff700 1 -- 172.21.15.31:0/1322316974 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5dc007690 con 0x7fe5ec070410 2023-10-06T02:28:33.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.124+0000 7fe5f139d700 1 -- 172.21.15.31:0/1322316974 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5ec0c3fc0 con 0x7fe5ec070410 2023-10-06T02:28:33.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.125+0000 7fe5f139d700 1 -- 172.21.15.31:0/1322316974 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5ec0cb5b0 con 0x7fe5ec070410 2023-10-06T02:28:33.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.126+0000 7fe5d3fff700 1 -- 172.21.15.31:0/1322316974 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7fe5dc004a40 con 0x7fe5ec070410 2023-10-06T02:28:33.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.127+0000 7fe5d3fff700 1 --2- 172.21.15.31:0/1322316974 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe5d8067050 0x7fe5d8069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:33.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.127+0000 7fe5d3fff700 1 -- 172.21.15.31:0/1322316974 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fe5dc083d00 con 0x7fe5ec070410 2023-10-06T02:28:33.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.128+0000 7fe5eb7fe700 1 --2- 172.21.15.31:0/1322316974 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe5d8067050 0x7fe5d8069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:33.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.128+0000 7fe5eb7fe700 1 --2- 172.21.15.31:0/1322316974 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe5d8067050 0x7fe5d8069500 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec159e50 tx=0x7fe5e0016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:33.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.131+0000 7fe5d3fff700 1 -- 172.21.15.31:0/1322316974 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe5dc051030 con 0x7fe5ec070410 2023-10-06T02:28:33.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.334+0000 7fe5f139d700 1 -- 172.21.15.31:0/1322316974 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fe5ec0ca580 con 0x7fe5ec070410 2023-10-06T02:28:33.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.335+0000 7fe5d3fff700 1 -- 172.21.15.31:0/1322316974 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v102) v1 ==== 84+0+126 (secure 0 0 0) 0x7fe5dc018090 con 0x7fe5ec070410 2023-10-06T02:28:33.336 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:28:33.336 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2023-10-06T02:28:33.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:28:33.337 INFO:teuthology.orchestra.run.smithi031.stdout: "poolnum": 1, 2023-10-06T02:28:33.337 INFO:teuthology.orchestra.run.smithi031.stdout: "poolname": ".mgr" 2023-10-06T02:28:33.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:28:33.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:28:33.337 INFO:teuthology.orchestra.run.smithi031.stdout: "poolnum": 2, 2023-10-06T02:28:33.337 INFO:teuthology.orchestra.run.smithi031.stdout: "poolname": "rbd" 2023-10-06T02:28:33.337 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:28:33.337 INFO:teuthology.orchestra.run.smithi031.stdout:] 2023-10-06T02:28:33.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.337+0000 7fe5f139d700 1 -- 172.21.15.31:0/1322316974 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe5d8067050 msgr2=0x7fe5d8069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:33.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.337+0000 7fe5f139d700 1 --2- 172.21.15.31:0/1322316974 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe5d8067050 0x7fe5d8069500 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec159e50 tx=0x7fe5e0016000 comp rx=0 tx=0).stop 2023-10-06T02:28:33.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.337+0000 7fe5f139d700 1 -- 172.21.15.31:0/1322316974 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5ec070410 msgr2=0x7fe5ec0c3a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:33.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.337+0000 7fe5f139d700 1 --2- 172.21.15.31:0/1322316974 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5ec070410 0x7fe5ec0c3a80 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc00cb90 tx=0x7fe5dc001ad0 comp rx=0 tx=0).stop 2023-10-06T02:28:33.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.337+0000 7fe5f139d700 1 -- 172.21.15.31:0/1322316974 shutdown_connections 2023-10-06T02:28:33.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.337+0000 7fe5f139d700 1 --2- 172.21.15.31:0/1322316974 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5ec159a10 0x7fe5ec154180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:33.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.337+0000 7fe5f139d700 1 --2- 172.21.15.31:0/1322316974 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5ec070410 0x7fe5ec0c3a80 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:33.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.337+0000 7fe5f139d700 1 --2- 172.21.15.31:0/1322316974 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe5d8067050 0x7fe5d8069500 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:33.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.337+0000 7fe5f139d700 1 --2- 172.21.15.31:0/1322316974 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe5ec159640 0x7fe5ec0c8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:33.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.337+0000 7fe5f139d700 1 -- 172.21.15.31:0/1322316974 >> 172.21.15.31:0/1322316974 conn(0x7fe5ec06e720 msgr2=0x7fe5ec0aed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:33.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.337+0000 7fe5f139d700 1 -- 172.21.15.31:0/1322316974 shutdown_connections 2023-10-06T02:28:33.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.337+0000 7fe5f139d700 1 -- 172.21.15.31:0/1322316974 wait complete. 2023-10-06T02:28:33.356 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T02:28:33.356 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T02:28:33.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.546+0000 7fd81f59d700 1 Processor -- start 2023-10-06T02:28:33.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.546+0000 7fd81f59d700 1 -- start start 2023-10-06T02:28:33.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.548+0000 7fd81f59d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd818157380 0x7fd8181585f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:33.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.548+0000 7fd81f59d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd81815b1a0 0x7fd8180c6b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:33.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.548+0000 7fd81f59d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd818157750 0x7fd8180ae080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:33.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.548+0000 7fd81f59d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd81806a7f0 con 0x7fd81815b1a0 2023-10-06T02:28:33.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.548+0000 7fd81f59d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8180c8680 con 0x7fd818157380 2023-10-06T02:28:33.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.548+0000 7fd81f59d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd818158b30 con 0x7fd818157750 2023-10-06T02:28:33.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.548+0000 7fd81dd9a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd81815b1a0 0x7fd8180c6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:33.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.548+0000 7fd81dd9a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd81815b1a0 0x7fd8180c6b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40598/0 (socket says 172.21.15.31:40598) 2023-10-06T02:28:33.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.548+0000 7fd81dd9a700 1 -- 172.21.15.31:0/3415131306 learned_addr learned my addr 172.21.15.31:0/3415131306 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:33.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.548+0000 7fd81ed9c700 1 --2- 172.21.15.31:0/3415131306 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd818157750 0x7fd8180ae080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:33.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.548+0000 7fd81e59b700 1 --2- 172.21.15.31:0/3415131306 >> v2:172.21.15.62:3300/0 conn(0x7fd818157380 0x7fd8181585f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:33.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.549+0000 7fd81dd9a700 1 -- 172.21.15.31:0/3415131306 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd818157750 msgr2=0x7fd8180ae080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:33.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.549+0000 7fd81dd9a700 1 --2- 172.21.15.31:0/3415131306 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd818157750 0x7fd8180ae080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:33.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.549+0000 7fd81dd9a700 1 -- 172.21.15.31:0/3415131306 >> v2:172.21.15.62:3300/0 conn(0x7fd818157380 msgr2=0x7fd8181585f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:33.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.549+0000 7fd81dd9a700 1 --2- 172.21.15.31:0/3415131306 >> v2:172.21.15.62:3300/0 conn(0x7fd818157380 0x7fd8181585f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:33.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.549+0000 7fd81dd9a700 1 -- 172.21.15.31:0/3415131306 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd81815b6a0 con 0x7fd81815b1a0 2023-10-06T02:28:33.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.549+0000 7fd81ed9c700 1 --2- 172.21.15.31:0/3415131306 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd818157750 0x7fd8180ae080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:28:33.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.549+0000 7fd81e59b700 1 --2- 172.21.15.31:0/3415131306 >> v2:172.21.15.62:3300/0 conn(0x7fd818157380 0x7fd8181585f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:28:33.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.549+0000 7fd81dd9a700 1 --2- 172.21.15.31:0/3415131306 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd81815b1a0 0x7fd8180c6b60 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fd8180cd220 tx=0x7fd808009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cc07ca2af6c83cc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:33.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.549+0000 7fd81d599700 1 -- 172.21.15.31:0/3415131306 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd80800b860 con 0x7fd81815b1a0 2023-10-06T02:28:33.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.549+0000 7fd81d599700 1 -- 172.21.15.31:0/3415131306 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd80800b9c0 con 0x7fd81815b1a0 2023-10-06T02:28:33.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.549+0000 7fd81d599700 1 -- 172.21.15.31:0/3415131306 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd80800bd60 con 0x7fd81815b1a0 2023-10-06T02:28:33.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.550+0000 7fd81f59d700 1 -- 172.21.15.31:0/3415131306 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd81815b1a0 msgr2=0x7fd8180c6b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:33.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.550+0000 7fd81f59d700 1 --2- 172.21.15.31:0/3415131306 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd81815b1a0 0x7fd8180c6b60 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fd8180cd220 tx=0x7fd808009250 comp rx=0 tx=0).stop 2023-10-06T02:28:33.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.550+0000 7fd81f59d700 1 -- 172.21.15.31:0/3415131306 shutdown_connections 2023-10-06T02:28:33.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.550+0000 7fd81f59d700 1 --2- 172.21.15.31:0/3415131306 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd818157750 0x7fd8180ae080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:33.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.550+0000 7fd81f59d700 1 --2- 172.21.15.31:0/3415131306 >> v2:172.21.15.62:3300/0 conn(0x7fd818157380 0x7fd8181585f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:33.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.550+0000 7fd81f59d700 1 --2- 172.21.15.31:0/3415131306 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd81815b1a0 0x7fd8180c6b60 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:33.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.550+0000 7fd81f59d700 1 -- 172.21.15.31:0/3415131306 >> 172.21.15.31:0/3415131306 conn(0x7fd81806c6a0 msgr2=0x7fd81806caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:33.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.550+0000 7fd81f59d700 1 -- 172.21.15.31:0/3415131306 shutdown_connections 2023-10-06T02:28:33.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.550+0000 7fd81f59d700 1 -- 172.21.15.31:0/3415131306 wait complete. 2023-10-06T02:28:33.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.551+0000 7fd81f59d700 1 Processor -- start 2023-10-06T02:28:33.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.551+0000 7fd81f59d700 1 -- start start 2023-10-06T02:28:33.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.552+0000 7fd81f59d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd818157380 0x7fd818148c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:33.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.552+0000 7fd81f59d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd818157750 0x7fd818151510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:33.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.552+0000 7fd81f59d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd81815b1a0 0x7fd818151a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:33.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.552+0000 7fd81f59d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8180ac3b0 con 0x7fd818157380 2023-10-06T02:28:33.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.552+0000 7fd81f59d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd818155490 con 0x7fd818157750 2023-10-06T02:28:33.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.552+0000 7fd81f59d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd818158d00 con 0x7fd81815b1a0 2023-10-06T02:28:33.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.553+0000 7fd81e59b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd818157380 0x7fd818148c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:33.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.553+0000 7fd81e59b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd818157380 0x7fd818148c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40600/0 (socket says 172.21.15.31:40600) 2023-10-06T02:28:33.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.553+0000 7fd81e59b700 1 -- 172.21.15.31:0/3807827240 learned_addr learned my addr 172.21.15.31:0/3807827240 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:33.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.553+0000 7fd81ed9c700 1 --2- 172.21.15.31:0/3807827240 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd81815b1a0 0x7fd818151a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:33.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.553+0000 7fd81dd9a700 1 --2- 172.21.15.31:0/3807827240 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd818157750 0x7fd818151510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:33.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.553+0000 7fd81e59b700 1 -- 172.21.15.31:0/3807827240 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd81815b1a0 msgr2=0x7fd818151a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:33.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.553+0000 7fd81e59b700 1 --2- 172.21.15.31:0/3807827240 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd81815b1a0 0x7fd818151a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:33.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.553+0000 7fd81e59b700 1 -- 172.21.15.31:0/3807827240 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd818157750 msgr2=0x7fd818151510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:33.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.553+0000 7fd81e59b700 1 --2- 172.21.15.31:0/3807827240 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd818157750 0x7fd818151510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:33.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.553+0000 7fd81e59b700 1 -- 172.21.15.31:0/3807827240 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8080094f0 con 0x7fd818157380 2023-10-06T02:28:33.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.553+0000 7fd81dd9a700 1 --2- 172.21.15.31:0/3807827240 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd818157750 0x7fd818151510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:28:33.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.553+0000 7fd81e59b700 1 --2- 172.21.15.31:0/3807827240 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd818157380 0x7fd818148c40 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fd81400d860 tx=0x7fd814008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:33.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.553+0000 7fd80e7fc700 1 -- 172.21.15.31:0/3807827240 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8140125f0 con 0x7fd818157380 2023-10-06T02:28:33.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.554+0000 7fd80e7fc700 1 -- 172.21.15.31:0/3807827240 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd814012750 con 0x7fd818157380 2023-10-06T02:28:33.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.554+0000 7fd80e7fc700 1 -- 172.21.15.31:0/3807827240 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd814012af0 con 0x7fd818157380 2023-10-06T02:28:33.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.554+0000 7fd81f59d700 1 -- 172.21.15.31:0/3807827240 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd818155620 con 0x7fd818157380 2023-10-06T02:28:33.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.554+0000 7fd81f59d700 1 -- 172.21.15.31:0/3807827240 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8181523d0 con 0x7fd818157380 2023-10-06T02:28:33.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.555+0000 7fd81f59d700 1 -- 172.21.15.31:0/3807827240 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8181547a0 con 0x7fd818157380 2023-10-06T02:28:33.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.556+0000 7fd80e7fc700 1 -- 172.21.15.31:0/3807827240 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7fd814014430 con 0x7fd818157380 2023-10-06T02:28:33.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.556+0000 7fd80e7fc700 1 --2- 172.21.15.31:0/3807827240 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd7fc067050 0x7fd7fc069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:33.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.556+0000 7fd80e7fc700 1 -- 172.21.15.31:0/3807827240 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fd8140848f0 con 0x7fd818157380 2023-10-06T02:28:33.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.557+0000 7fd81dd9a700 1 --2- 172.21.15.31:0/3807827240 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd7fc067050 0x7fd7fc069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:33.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.558+0000 7fd81dd9a700 1 --2- 172.21.15.31:0/3807827240 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd7fc067050 0x7fd7fc069500 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd8180cd220 tx=0x7fd808017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:33.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.561+0000 7fd80e7fc700 1 -- 172.21.15.31:0/3807827240 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd814051ba0 con 0x7fd818157380 2023-10-06T02:28:33.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:33.762+0000 7fd81f59d700 1 -- 172.21.15.31:0/3807827240 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fd81804c3c0 con 0x7fd818157380 2023-10-06T02:28:34.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:34.092+0000 7fd80e7fc700 1 -- 172.21.15.31:0/3807827240 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v103) v1 ==== 138+0+0 (secure 0 0 0) 0x7fd8140551c0 con 0x7fd818157380 2023-10-06T02:28:34.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:34.096+0000 7fd81f59d700 1 -- 172.21.15.31:0/3807827240 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd7fc067050 msgr2=0x7fd7fc069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:34.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:34.096+0000 7fd81f59d700 1 --2- 172.21.15.31:0/3807827240 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd7fc067050 0x7fd7fc069500 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd8180cd220 tx=0x7fd808017000 comp rx=0 tx=0).stop 2023-10-06T02:28:34.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:34.096+0000 7fd81f59d700 1 -- 172.21.15.31:0/3807827240 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd818157380 msgr2=0x7fd818148c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:34.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:34.096+0000 7fd81f59d700 1 --2- 172.21.15.31:0/3807827240 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd818157380 0x7fd818148c40 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fd81400d860 tx=0x7fd814008040 comp rx=0 tx=0).stop 2023-10-06T02:28:34.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:34.098+0000 7fd81f59d700 1 -- 172.21.15.31:0/3807827240 shutdown_connections 2023-10-06T02:28:34.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:34.098+0000 7fd81f59d700 1 --2- 172.21.15.31:0/3807827240 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd7fc067050 0x7fd7fc069500 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:34.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:34.098+0000 7fd81f59d700 1 --2- 172.21.15.31:0/3807827240 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd81815b1a0 0x7fd818151a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:34.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:34.098+0000 7fd81f59d700 1 --2- 172.21.15.31:0/3807827240 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd818157380 0x7fd818148c40 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:34.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:34.098+0000 7fd81f59d700 1 --2- 172.21.15.31:0/3807827240 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd818157750 0x7fd818151510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:34.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:34.098+0000 7fd81f59d700 1 -- 172.21.15.31:0/3807827240 >> 172.21.15.31:0/3807827240 conn(0x7fd81806c6a0 msgr2=0x7fd8180cc280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:34.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:34.098+0000 7fd81f59d700 1 -- 172.21.15.31:0/3807827240 shutdown_connections 2023-10-06T02:28:34.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:34.098+0000 7fd81f59d700 1 -- 172.21.15.31:0/3807827240 wait complete. 2023-10-06T02:28:34.101 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' created 2023-10-06T02:28:34.119 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T02:28:34.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:34.312+0000 7f745339d700 1 Processor -- start 2023-10-06T02:28:34.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:34.312+0000 7f745339d700 1 -- start start 2023-10-06T02:28:34.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:34.313+0000 7f745339d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f744c06e0a0 0x7f744c0c6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:34.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:34.313+0000 7f745339d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f744c150380 0x7f744c06a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:34.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:34.313+0000 7f745339d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f744c06a490 con 0x7f744c06f3a0 2023-10-06T02:28:34.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:34.313+0000 7f745339d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f744c0c7490 con 0x7f744c150380 2023-10-06T02:28:34.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:34.313+0000 7f745339d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f744c0c7770 con 0x7f744c06e0a0 2023-10-06T02:28:34.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:34.313+0000 7f7452b9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f744c150380 0x7f744c06a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:34.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:34.313+0000 7f7452b9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f744c150380 0x7f744c06a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40606/0 (socket says 172.21.15.31:40606) 2023-10-06T02:28:34.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:34.313+0000 7f7452b9c700 1 -- 172.21.15.31:0/2302093425 learned_addr learned my addr 172.21.15.31:0/2302093425 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:34.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:34.314+0000 7f745239b700 1 --2- 172.21.15.31:0/2302093425 >> v2:172.21.15.62:3300/0 conn(0x7f744c06e0a0 0x7f744c0c6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:34.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:34.314+0000 7f7452b9c700 1 -- 172.21.15.31:0/2302093425 >> v2:172.21.15.62:3300/0 conn(0x7f744c06e0a0 msgr2=0x7f744c0c6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:34.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:34.314+0000 7f7452b9c700 1 --2- 172.21.15.31:0/2302093425 >> v2:172.21.15.62:3300/0 conn(0x7f744c06e0a0 0x7f744c0c6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:34.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:34.314+0000 7f7452b9c700 1 -- 172.21.15.31:0/2302093425 >> v1:172.21.15.62:6789/0 conn(0x7f744c06f3a0 legacy=0x7f744c14fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:34.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:34.314+0000 7f7452b9c700 1 -- 172.21.15.31:0/2302093425 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f744c0c7a50 con 0x7f744c150380 2023-10-06T02:28:34.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:34.314+0000 7f7452b9c700 1 --2- 172.21.15.31:0/2302093425 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f744c150380 0x7f744c06a920 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f744c069230 tx=0x7f7444009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86687f57d360eb34 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:34.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:34.314+0000 7f7451399700 1 -- 172.21.15.31:0/2302093425 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f744400e070 con 0x7f744c150380 2023-10-06T02:28:34.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:34.315+0000 7f7451399700 1 -- 172.21.15.31:0/2302093425 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f744400ba50 con 0x7f744c150380 2023-10-06T02:28:34.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:34.315+0000 7f7451399700 1 -- 172.21.15.31:0/2302093425 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74440092e0 con 0x7f744c150380 2023-10-06T02:28:34.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:34.315+0000 7f745339d700 1 -- 172.21.15.31:0/2302093425 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f744c150380 msgr2=0x7f744c06a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:34.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:34.315+0000 7f745339d700 1 --2- 172.21.15.31:0/2302093425 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f744c150380 0x7f744c06a920 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f744c069230 tx=0x7f7444009840 comp rx=0 tx=0).stop 2023-10-06T02:28:34.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:34.315+0000 7f745339d700 1 -- 172.21.15.31:0/2302093425 shutdown_connections 2023-10-06T02:28:34.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:34.315+0000 7f745339d700 1 --2- 172.21.15.31:0/2302093425 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f744c150380 0x7f744c06a920 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:34.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:34.315+0000 7f745339d700 1 --2- 172.21.15.31:0/2302093425 >> v2:172.21.15.62:3300/0 conn(0x7f744c06e0a0 0x7f744c0c6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:34.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:34.315+0000 7f745339d700 1 -- 172.21.15.31:0/2302093425 >> 172.21.15.31:0/2302093425 conn(0x7f744c06d170 msgr2=0x7f744c0cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:34.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:34.315+0000 7f745339d700 1 -- 172.21.15.31:0/2302093425 shutdown_connections 2023-10-06T02:28:34.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:34.315+0000 7f745339d700 1 -- 172.21.15.31:0/2302093425 wait complete. 2023-10-06T02:28:34.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:34.317+0000 7f745339d700 1 Processor -- start 2023-10-06T02:28:34.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:34.317+0000 7f745339d700 1 -- start start 2023-10-06T02:28:34.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:34.318+0000 7f745339d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f744c06e0a0 0x7f744c158a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:34.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:34.318+0000 7f745339d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f744c06f3a0 0x7f744c16ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:34.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:34.318+0000 7f745339d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f744c150380 0x7f744c14a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:34.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:34.318+0000 7f745339d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f744c158f90 con 0x7f744c06e0a0 2023-10-06T02:28:34.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:34.318+0000 7f745339d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f744c1456e0 con 0x7f744c150380 2023-10-06T02:28:34.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:34.318+0000 7f745339d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f744c1566a0 con 0x7f744c06f3a0 2023-10-06T02:28:34.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:34.318+0000 7f745239b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f744c06e0a0 0x7f744c158a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:34.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:34.318+0000 7f745239b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f744c06e0a0 0x7f744c158a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40614/0 (socket says 172.21.15.31:40614) 2023-10-06T02:28:34.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:34.318+0000 7f745239b700 1 -- 172.21.15.31:0/1832612293 learned_addr learned my addr 172.21.15.31:0/1832612293 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:34.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:34.318+0000 7f7451b9a700 1 --2- 172.21.15.31:0/1832612293 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f744c06f3a0 0x7f744c16ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:34.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:34.318+0000 7f7452b9c700 1 --2- 172.21.15.31:0/1832612293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f744c150380 0x7f744c14a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:34.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:34.318+0000 7f745239b700 1 -- 172.21.15.31:0/1832612293 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f744c06f3a0 msgr2=0x7f744c16ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:34.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:34.318+0000 7f745239b700 1 --2- 172.21.15.31:0/1832612293 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f744c06f3a0 0x7f744c16ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:34.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:34.318+0000 7f745239b700 1 -- 172.21.15.31:0/1832612293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f744c150380 msgr2=0x7f744c14a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:34.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:34.318+0000 7f745239b700 1 --2- 172.21.15.31:0/1832612293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f744c150380 0x7f744c14a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:34.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:34.318+0000 7f745239b700 1 -- 172.21.15.31:0/1832612293 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7444009ae0 con 0x7f744c06e0a0 2023-10-06T02:28:34.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:34.319+0000 7f745239b700 1 --2- 172.21.15.31:0/1832612293 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f744c06e0a0 0x7f744c158a50 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f744800d890 tx=0x7f7448008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:34.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:34.319+0000 7f74427fc700 1 -- 172.21.15.31:0/1832612293 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74480125f0 con 0x7f744c06e0a0 2023-10-06T02:28:34.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:34.319+0000 7f74427fc700 1 -- 172.21.15.31:0/1832612293 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7448012770 con 0x7f744c06e0a0 2023-10-06T02:28:34.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:34.319+0000 7f745339d700 1 -- 172.21.15.31:0/1832612293 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f744c16d720 con 0x7f744c06e0a0 2023-10-06T02:28:34.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:34.319+0000 7f74427fc700 1 -- 172.21.15.31:0/1832612293 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7448012b30 con 0x7f744c06e0a0 2023-10-06T02:28:34.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:34.319+0000 7f745339d700 1 -- 172.21.15.31:0/1832612293 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f744c156330 con 0x7f744c06e0a0 2023-10-06T02:28:34.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:34.320+0000 7f745339d700 1 -- 172.21.15.31:0/1832612293 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f744c16dc40 con 0x7f744c06e0a0 2023-10-06T02:28:34.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:34.321+0000 7f74427fc700 1 -- 172.21.15.31:0/1832612293 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f744801e460 con 0x7f744c06e0a0 2023-10-06T02:28:34.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:34.322+0000 7f74427fc700 1 --2- 172.21.15.31:0/1832612293 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7430067050 0x7f7430069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:34.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:34.322+0000 7f74427fc700 1 -- 172.21.15.31:0/1832612293 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5576+0+0 (secure 0 0 0) 0x7f7448084df0 con 0x7f744c06e0a0 2023-10-06T02:28:34.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:34.322+0000 7f7451b9a700 1 --2- 172.21.15.31:0/1832612293 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7430067050 0x7f7430069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:34.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:34.323+0000 7f7451b9a700 1 --2- 172.21.15.31:0/1832612293 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7430067050 0x7f7430069500 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f744c0cae60 tx=0x7f743c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:34.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:34.326+0000 7f74427fc700 1 -- 172.21.15.31:0/1832612293 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f744c16dc40 con 0x7f744c06e0a0 2023-10-06T02:28:34.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:34.529+0000 7f745339d700 1 -- 172.21.15.31:0/1832612293 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f744c16dc40 con 0x7f744c06e0a0 2023-10-06T02:28:35.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:35.099+0000 7f74427fc700 1 -- 172.21.15.31:0/1832612293 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v104) v1 ==== 170+0+0 (secure 0 0 0) 0x7f744c16dc40 con 0x7f744c06e0a0 2023-10-06T02:28:35.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:35.106+0000 7f745339d700 1 -- 172.21.15.31:0/1832612293 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7430067050 msgr2=0x7f7430069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:35.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:35.106+0000 7f745339d700 1 --2- 172.21.15.31:0/1832612293 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7430067050 0x7f7430069500 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f744c0cae60 tx=0x7f743c009040 comp rx=0 tx=0).stop 2023-10-06T02:28:35.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:35.106+0000 7f745339d700 1 -- 172.21.15.31:0/1832612293 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f744c06e0a0 msgr2=0x7f744c158a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:35.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:35.106+0000 7f745339d700 1 --2- 172.21.15.31:0/1832612293 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f744c06e0a0 0x7f744c158a50 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f744800d890 tx=0x7f7448008040 comp rx=0 tx=0).stop 2023-10-06T02:28:35.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:35.107+0000 7f745339d700 1 -- 172.21.15.31:0/1832612293 shutdown_connections 2023-10-06T02:28:35.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:35.107+0000 7f745339d700 1 --2- 172.21.15.31:0/1832612293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f744c150380 0x7f744c14a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:35.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:35.107+0000 7f745339d700 1 --2- 172.21.15.31:0/1832612293 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f744c06e0a0 0x7f744c158a50 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:35.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:35.107+0000 7f745339d700 1 --2- 172.21.15.31:0/1832612293 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7430067050 0x7f7430069500 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:35.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:35.107+0000 7f745339d700 1 --2- 172.21.15.31:0/1832612293 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f744c06f3a0 0x7f744c16ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:35.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:35.107+0000 7f745339d700 1 -- 172.21.15.31:0/1832612293 >> 172.21.15.31:0/1832612293 conn(0x7f744c06d170 msgr2=0x7f744c1557c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:35.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:35.111+0000 7f745339d700 1 -- 172.21.15.31:0/1832612293 shutdown_connections 2023-10-06T02:28:35.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:35.112+0000 7f745339d700 1 -- 172.21.15.31:0/1832612293 wait complete. 2023-10-06T02:28:35.118 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' created 2023-10-06T02:28:35.142 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T02:28:35.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:35.333+0000 7fd3c2f9d700 1 Processor -- start 2023-10-06T02:28:35.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:35.333+0000 7fd3c2f9d700 1 -- start start 2023-10-06T02:28:35.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:35.334+0000 7fd3c2f9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd3bc0c7be0 0x7fd3bc06abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:35.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:35.334+0000 7fd3c2f9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd3bc13f780 0x7fd3bc13fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:35.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:35.334+0000 7fd3c2f9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3bc1400e0 con 0x7fd3bc0c7fb0 2023-10-06T02:28:35.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:35.334+0000 7fd3c2f9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3bc06bf10 con 0x7fd3bc0c7be0 2023-10-06T02:28:35.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:35.334+0000 7fd3c2f9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3bc06b290 con 0x7fd3bc13f780 2023-10-06T02:28:35.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:35.334+0000 7fd3c1f9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd3bc0c7be0 0x7fd3bc06abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:35.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:35.334+0000 7fd3c1f9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd3bc0c7be0 0x7fd3bc06abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40624/0 (socket says 172.21.15.31:40624) 2023-10-06T02:28:35.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:35.334+0000 7fd3c1f9b700 1 -- 172.21.15.31:0/3442333279 learned_addr learned my addr 172.21.15.31:0/3442333279 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:35.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:35.334+0000 7fd3c179a700 1 --1- 172.21.15.31:0/3442333279 >> v1:172.21.15.31:6789/0 conn(0x7fd3bc0c7fb0 0x7fd3bc15a990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56768/0 (socket says 172.21.15.31:56768) 2023-10-06T02:28:35.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:35.334+0000 7fd3c279c700 1 --2- 172.21.15.31:0/3442333279 >> v2:172.21.15.62:3300/0 conn(0x7fd3bc13f780 0x7fd3bc13fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:35.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:35.335+0000 7fd3c0f99700 1 -- 172.21.15.31:0/3442333279 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 601906980 0 0) 0x7fd3bc1400e0 con 0x7fd3bc0c7fb0 2023-10-06T02:28:35.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:35.335+0000 7fd3c1f9b700 1 -- 172.21.15.31:0/3442333279 >> v2:172.21.15.62:3300/0 conn(0x7fd3bc13f780 msgr2=0x7fd3bc13fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:35.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:35.335+0000 7fd3c1f9b700 1 --2- 172.21.15.31:0/3442333279 >> v2:172.21.15.62:3300/0 conn(0x7fd3bc13f780 0x7fd3bc13fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:35.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:35.335+0000 7fd3c1f9b700 1 -- 172.21.15.31:0/3442333279 >> v1:172.21.15.31:6789/0 conn(0x7fd3bc0c7fb0 legacy=0x7fd3bc15a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:35.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:35.335+0000 7fd3c1f9b700 1 -- 172.21.15.31:0/3442333279 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3bc06b570 con 0x7fd3bc0c7be0 2023-10-06T02:28:35.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:35.335+0000 7fd3c1f9b700 1 --2- 172.21.15.31:0/3442333279 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3bc0c7be0 0x7fd3bc06abd0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc0cba10 tx=0x7fd3b8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c5ac60bc595d29e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:35.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:35.335+0000 7fd3c0f99700 1 -- 172.21.15.31:0/3442333279 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b8009e70 con 0x7fd3bc0c7be0 2023-10-06T02:28:35.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:35.336+0000 7fd3c0f99700 1 -- 172.21.15.31:0/3442333279 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3b800b860 con 0x7fd3bc0c7be0 2023-10-06T02:28:35.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:35.336+0000 7fd3c0f99700 1 -- 172.21.15.31:0/3442333279 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b800bc00 con 0x7fd3bc0c7be0 2023-10-06T02:28:35.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:35.336+0000 7fd3c2f9d700 1 -- 172.21.15.31:0/3442333279 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3bc0c7be0 msgr2=0x7fd3bc06abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:35.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:35.336+0000 7fd3c2f9d700 1 --2- 172.21.15.31:0/3442333279 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3bc0c7be0 0x7fd3bc06abd0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc0cba10 tx=0x7fd3b8009250 comp rx=0 tx=0).stop 2023-10-06T02:28:35.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:35.336+0000 7fd3c2f9d700 1 -- 172.21.15.31:0/3442333279 shutdown_connections 2023-10-06T02:28:35.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:35.336+0000 7fd3c2f9d700 1 --2- 172.21.15.31:0/3442333279 >> v2:172.21.15.62:3300/0 conn(0x7fd3bc13f780 0x7fd3bc13fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:35.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:35.336+0000 7fd3c2f9d700 1 --2- 172.21.15.31:0/3442333279 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3bc0c7be0 0x7fd3bc06abd0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:35.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:35.336+0000 7fd3c2f9d700 1 -- 172.21.15.31:0/3442333279 >> 172.21.15.31:0/3442333279 conn(0x7fd3bc06d110 msgr2=0x7fd3bc0cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:35.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:35.336+0000 7fd3c2f9d700 1 -- 172.21.15.31:0/3442333279 shutdown_connections 2023-10-06T02:28:35.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:35.336+0000 7fd3c2f9d700 1 -- 172.21.15.31:0/3442333279 wait complete. 2023-10-06T02:28:35.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:35.338+0000 7fd3c2f9d700 1 Processor -- start 2023-10-06T02:28:35.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:35.338+0000 7fd3c2f9d700 1 -- start start 2023-10-06T02:28:35.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:35.339+0000 7fd3c2f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3bc0c7be0 0x7fd3bc156ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:35.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:35.339+0000 7fd3c2f9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3bc0c7fb0 0x7fd3bc0ae060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:35.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:35.339+0000 7fd3c2f9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3bc13f780 0x7fd3bc06de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:35.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:35.339+0000 7fd3c2f9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3bc13c160 con 0x7fd3bc13f780 2023-10-06T02:28:35.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:35.339+0000 7fd3c2f9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3bc0c5e50 con 0x7fd3bc0c7fb0 2023-10-06T02:28:35.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:35.339+0000 7fd3c2f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3bc0c6130 con 0x7fd3bc0c7be0 2023-10-06T02:28:35.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:35.339+0000 7fd3c279c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3bc13f780 0x7fd3bc06de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:35.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:35.339+0000 7fd3c279c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3bc13f780 0x7fd3bc06de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40628/0 (socket says 172.21.15.31:40628) 2023-10-06T02:28:35.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:35.339+0000 7fd3c279c700 1 -- 172.21.15.31:0/91624642 learned_addr learned my addr 172.21.15.31:0/91624642 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:35.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:35.339+0000 7fd3c1f9b700 1 --2- 172.21.15.31:0/91624642 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3bc0c7be0 0x7fd3bc156ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:35.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:35.339+0000 7fd3c179a700 1 --2- 172.21.15.31:0/91624642 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3bc0c7fb0 0x7fd3bc0ae060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:35.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:35.340+0000 7fd3c279c700 1 -- 172.21.15.31:0/91624642 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3bc0c7be0 msgr2=0x7fd3bc156ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:35.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:35.340+0000 7fd3c279c700 1 --2- 172.21.15.31:0/91624642 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3bc0c7be0 0x7fd3bc156ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:35.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:35.340+0000 7fd3c279c700 1 -- 172.21.15.31:0/91624642 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3bc0c7fb0 msgr2=0x7fd3bc0ae060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:35.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:35.340+0000 7fd3c279c700 1 --2- 172.21.15.31:0/91624642 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3bc0c7fb0 0x7fd3bc0ae060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:35.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:35.340+0000 7fd3c279c700 1 -- 172.21.15.31:0/91624642 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3b80094f0 con 0x7fd3bc13f780 2023-10-06T02:28:35.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:35.340+0000 7fd3c1f9b700 1 --2- 172.21.15.31:0/91624642 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3bc0c7be0 0x7fd3bc156ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:28:35.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:35.340+0000 7fd3c179a700 1 --2- 172.21.15.31:0/91624642 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3bc0c7fb0 0x7fd3bc0ae060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:28:35.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:35.340+0000 7fd3c279c700 1 --2- 172.21.15.31:0/91624642 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3bc13f780 0x7fd3bc06de50 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fd3b4009df0 tx=0x7fd3b400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:35.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:35.340+0000 7fd3b1ffb700 1 -- 172.21.15.31:0/91624642 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b400cb50 con 0x7fd3bc13f780 2023-10-06T02:28:35.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:35.340+0000 7fd3b1ffb700 1 -- 172.21.15.31:0/91624642 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3b400f040 con 0x7fd3bc13f780 2023-10-06T02:28:35.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:35.340+0000 7fd3c2f9d700 1 -- 172.21.15.31:0/91624642 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3bc13bb10 con 0x7fd3bc13f780 2023-10-06T02:28:35.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:35.340+0000 7fd3b1ffb700 1 -- 172.21.15.31:0/91624642 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b4014480 con 0x7fd3bc13f780 2023-10-06T02:28:35.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:35.340+0000 7fd3c2f9d700 1 -- 172.21.15.31:0/91624642 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3bc0ae5a0 con 0x7fd3bc13f780 2023-10-06T02:28:35.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:35.341+0000 7fd3c2f9d700 1 -- 172.21.15.31:0/91624642 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3bc13c310 con 0x7fd3bc13f780 2023-10-06T02:28:35.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:35.342+0000 7fd3b1ffb700 1 -- 172.21.15.31:0/91624642 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7fd3b4014830 con 0x7fd3bc13f780 2023-10-06T02:28:35.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:35.343+0000 7fd3b1ffb700 1 --2- 172.21.15.31:0/91624642 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd3a0067050 0x7fd3a0069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:35.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:35.343+0000 7fd3b1ffb700 1 -- 172.21.15.31:0/91624642 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5958+0+0 (secure 0 0 0) 0x7fd3b40855c0 con 0x7fd3bc13f780 2023-10-06T02:28:35.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:35.344+0000 7fd3c1f9b700 1 --2- 172.21.15.31:0/91624642 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd3a0067050 0x7fd3a0069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:35.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:35.345+0000 7fd3c1f9b700 1 --2- 172.21.15.31:0/91624642 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd3a0067050 0x7fd3a0069500 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc0cba10 tx=0x7fd3b8017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:35.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:35.348+0000 7fd3b1ffb700 1 -- 172.21.15.31:0/91624642 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd3bc13c310 con 0x7fd3bc13f780 2023-10-06T02:28:35.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:35.596+0000 7fd3c2f9d700 1 -- 172.21.15.31:0/91624642 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fd3bc0648e0 con 0x7fd3bc13f780 2023-10-06T02:28:36.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.103+0000 7fd3b1ffb700 1 -- 172.21.15.31:0/91624642 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 17 and data pool 18 v67) v1 ==== 174+0+0 (secure 0 0 0) 0x7fd3bc0648e0 con 0x7fd3bc13f780 2023-10-06T02:28:36.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.109+0000 7fd3c2f9d700 1 -- 172.21.15.31:0/91624642 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd3a0067050 msgr2=0x7fd3a0069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:36.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.109+0000 7fd3c2f9d700 1 --2- 172.21.15.31:0/91624642 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd3a0067050 0x7fd3a0069500 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc0cba10 tx=0x7fd3b8017000 comp rx=0 tx=0).stop 2023-10-06T02:28:36.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.110+0000 7fd3c2f9d700 1 -- 172.21.15.31:0/91624642 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3bc13f780 msgr2=0x7fd3bc06de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:36.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.110+0000 7fd3c2f9d700 1 --2- 172.21.15.31:0/91624642 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3bc13f780 0x7fd3bc06de50 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fd3b4009df0 tx=0x7fd3b400b040 comp rx=0 tx=0).stop 2023-10-06T02:28:36.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.110+0000 7fd3c2f9d700 1 -- 172.21.15.31:0/91624642 shutdown_connections 2023-10-06T02:28:36.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.110+0000 7fd3c2f9d700 1 --2- 172.21.15.31:0/91624642 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3bc13f780 0x7fd3bc06de50 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:36.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.110+0000 7fd3c2f9d700 1 --2- 172.21.15.31:0/91624642 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd3a0067050 0x7fd3a0069500 secure :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc0cba10 tx=0x7fd3b8017000 comp rx=0 tx=0).stop 2023-10-06T02:28:36.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.110+0000 7fd3c2f9d700 1 --2- 172.21.15.31:0/91624642 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3bc0c7be0 0x7fd3bc156ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:36.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.110+0000 7fd3c2f9d700 1 --2- 172.21.15.31:0/91624642 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3bc0c7fb0 0x7fd3bc0ae060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:36.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.110+0000 7fd3c2f9d700 1 -- 172.21.15.31:0/91624642 >> 172.21.15.31:0/91624642 conn(0x7fd3bc06d110 msgr2=0x7fd3bc0b1120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:36.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.121+0000 7fd3c2f9d700 1 -- 172.21.15.31:0/91624642 shutdown_connections 2023-10-06T02:28:36.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.123+0000 7fd3c2f9d700 1 -- 172.21.15.31:0/91624642 wait complete. 2023-10-06T02:28:36.132 INFO:teuthology.orchestra.run.smithi031.stderr:new fs with metadata pool 17 and data pool 18 2023-10-06T02:28:36.154 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:28:36.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.342+0000 7fdd5d79d700 1 Processor -- start 2023-10-06T02:28:36.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.342+0000 7fdd5d79d700 1 -- start start 2023-10-06T02:28:36.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.343+0000 7fdd5d79d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fdd58140a70 0x7fdd580c6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:36.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.343+0000 7fdd5d79d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd580c6d50 con 0x7fdd58140e40 2023-10-06T02:28:36.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.343+0000 7fdd5d79d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd580c7030 con 0x7fdd5806de50 2023-10-06T02:28:36.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.343+0000 7fdd5d79d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd580c7310 con 0x7fdd58140a70 2023-10-06T02:28:36.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.344+0000 7fdd577fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fdd58140e40 0x7fdd580c7df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56780/0 (socket says 172.21.15.31:56780) 2023-10-06T02:28:36.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.344+0000 7fdd577fe700 1 -- 172.21.15.31:0/3963702562 learned_addr learned my addr 172.21.15.31:0/3963702562 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:28:36.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.344+0000 7fdd57fff700 1 --2- 172.21.15.31:0/3963702562 >> v2:172.21.15.62:3300/0 conn(0x7fdd58140a70 0x7fdd580c6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:36.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.344+0000 7fdd56ffd700 1 -- 172.21.15.31:0/3963702562 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2916695105 0 0) 0x7fdd580c6d50 con 0x7fdd58140e40 2023-10-06T02:28:36.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.344+0000 7fdd56ffd700 1 -- 172.21.15.31:0/3963702562 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd40003570 con 0x7fdd58140e40 2023-10-06T02:28:36.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.344+0000 7fdd56ffd700 1 -- 172.21.15.31:0/3963702562 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2943014736 0 0) 0x7fdd580c7030 con 0x7fdd5806de50 2023-10-06T02:28:36.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.344+0000 7fdd56ffd700 1 -- 172.21.15.31:0/3963702562 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd580c6d50 con 0x7fdd5806de50 2023-10-06T02:28:36.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.344+0000 7fdd57fff700 1 -- 172.21.15.31:0/3963702562 >> v1:172.21.15.62:6789/0 conn(0x7fdd5806de50 legacy=0x7fdd580aeef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:36.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.344+0000 7fdd57fff700 1 -- 172.21.15.31:0/3963702562 >> v1:172.21.15.31:6789/0 conn(0x7fdd58140e40 legacy=0x7fdd580c7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:36.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.344+0000 7fdd57fff700 1 -- 172.21.15.31:0/3963702562 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd580c75f0 con 0x7fdd58140a70 2023-10-06T02:28:36.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.345+0000 7fdd57fff700 1 --2- 172.21.15.31:0/3963702562 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd58140a70 0x7fdd580c6770 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fdd480098e0 tx=0x7fdd480091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d9376d1626fe9d3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:36.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.345+0000 7fdd56ffd700 1 -- 172.21.15.31:0/3963702562 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd48011630 con 0x7fdd58140a70 2023-10-06T02:28:36.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.345+0000 7fdd56ffd700 1 -- 172.21.15.31:0/3963702562 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd48011790 con 0x7fdd58140a70 2023-10-06T02:28:36.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.345+0000 7fdd56ffd700 1 -- 172.21.15.31:0/3963702562 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd48011b30 con 0x7fdd58140a70 2023-10-06T02:28:36.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.346+0000 7fdd5d79d700 1 -- 172.21.15.31:0/3963702562 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd58140a70 msgr2=0x7fdd580c6770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:36.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.346+0000 7fdd5d79d700 1 --2- 172.21.15.31:0/3963702562 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd58140a70 0x7fdd580c6770 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fdd480098e0 tx=0x7fdd480091a0 comp rx=0 tx=0).stop 2023-10-06T02:28:36.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.346+0000 7fdd5d79d700 1 -- 172.21.15.31:0/3963702562 shutdown_connections 2023-10-06T02:28:36.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.346+0000 7fdd5d79d700 1 --2- 172.21.15.31:0/3963702562 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd58140a70 0x7fdd580c6770 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:36.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.346+0000 7fdd5d79d700 1 -- 172.21.15.31:0/3963702562 >> 172.21.15.31:0/3963702562 conn(0x7fdd5806d170 msgr2=0x7fdd580cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:36.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.346+0000 7fdd5d79d700 1 -- 172.21.15.31:0/3963702562 shutdown_connections 2023-10-06T02:28:36.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.346+0000 7fdd5d79d700 1 -- 172.21.15.31:0/3963702562 wait complete. 2023-10-06T02:28:36.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.347+0000 7fdd5d79d700 1 Processor -- start 2023-10-06T02:28:36.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.347+0000 7fdd5d79d700 1 -- start start 2023-10-06T02:28:36.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.348+0000 7fdd5d79d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd5806de50 0x7fdd58130590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:36.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.348+0000 7fdd5d79d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd58140a70 0x7fdd580ada30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:36.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.348+0000 7fdd5d79d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd58140e40 0x7fdd5813f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:36.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.348+0000 7fdd5d79d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd58131aa0 con 0x7fdd58140e40 2023-10-06T02:28:36.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.348+0000 7fdd5d79d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd58130e90 con 0x7fdd58140a70 2023-10-06T02:28:36.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.348+0000 7fdd5d79d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd58131170 con 0x7fdd5806de50 2023-10-06T02:28:36.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.349+0000 7fdd5cf9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd58140e40 0x7fdd5813f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:36.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.349+0000 7fdd577fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd58140a70 0x7fdd580ada30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:36.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.349+0000 7fdd5cf9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd58140e40 0x7fdd5813f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40638/0 (socket says 172.21.15.31:40638) 2023-10-06T02:28:36.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.349+0000 7fdd5cf9c700 1 -- 172.21.15.31:0/859951432 learned_addr learned my addr 172.21.15.31:0/859951432 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:36.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.349+0000 7fdd57fff700 1 --2- 172.21.15.31:0/859951432 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd5806de50 0x7fdd58130590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:36.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.349+0000 7fdd5cf9c700 1 -- 172.21.15.31:0/859951432 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd5806de50 msgr2=0x7fdd58130590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:36.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.349+0000 7fdd5cf9c700 1 --2- 172.21.15.31:0/859951432 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd5806de50 0x7fdd58130590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:36.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.349+0000 7fdd5cf9c700 1 -- 172.21.15.31:0/859951432 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd58140a70 msgr2=0x7fdd580ada30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:36.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.349+0000 7fdd5cf9c700 1 --2- 172.21.15.31:0/859951432 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd58140a70 0x7fdd580ada30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:36.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.349+0000 7fdd5cf9c700 1 -- 172.21.15.31:0/859951432 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd48009420 con 0x7fdd58140e40 2023-10-06T02:28:36.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.349+0000 7fdd5cf9c700 1 --2- 172.21.15.31:0/859951432 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd58140e40 0x7fdd5813f940 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fdd5000c910 tx=0x7fdd500027a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:36.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.350+0000 7fdd3ffff700 1 -- 172.21.15.31:0/859951432 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd50012630 con 0x7fdd58140e40 2023-10-06T02:28:36.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.350+0000 7fdd3ffff700 1 -- 172.21.15.31:0/859951432 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd500127b0 con 0x7fdd58140e40 2023-10-06T02:28:36.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.350+0000 7fdd3ffff700 1 -- 172.21.15.31:0/859951432 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd50012b70 con 0x7fdd58140e40 2023-10-06T02:28:36.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.350+0000 7fdd5d79d700 1 -- 172.21.15.31:0/859951432 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd58131450 con 0x7fdd58140e40 2023-10-06T02:28:36.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.350+0000 7fdd5d79d700 1 -- 172.21.15.31:0/859951432 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd58130ad0 con 0x7fdd58140e40 2023-10-06T02:28:36.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.351+0000 7fdd5d79d700 1 -- 172.21.15.31:0/859951432 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd58069730 con 0x7fdd58140e40 2023-10-06T02:28:36.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.352+0000 7fdd3ffff700 1 -- 172.21.15.31:0/859951432 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7fdd5001b400 con 0x7fdd58140e40 2023-10-06T02:28:36.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.353+0000 7fdd3ffff700 1 --2- 172.21.15.31:0/859951432 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fdd44067050 0x7fdd44069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:36.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.353+0000 7fdd3ffff700 1 -- 172.21.15.31:0/859951432 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fdd50084c00 con 0x7fdd58140e40 2023-10-06T02:28:36.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.353+0000 7fdd57fff700 1 --2- 172.21.15.31:0/859951432 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fdd44067050 0x7fdd44069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:36.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.354+0000 7fdd57fff700 1 --2- 172.21.15.31:0/859951432 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fdd44067050 0x7fdd44069500 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdd480093f0 tx=0x7fdd48017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:36.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.357+0000 7fdd3ffff700 1 -- 172.21.15.31:0/859951432 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdd50051b60 con 0x7fdd58140e40 2023-10-06T02:28:36.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.561+0000 7fdd5d79d700 1 -- 172.21.15.31:0/859951432 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdd5804c3c0 con 0x7fdd58140e40 2023-10-06T02:28:36.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.562+0000 7fdd3ffff700 1 -- 172.21.15.31:0/859951432 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) v1 ==== 74+0+16845 (secure 0 0 0) 0x7fdd50055180 con 0x7fdd58140e40 2023-10-06T02:28:36.563 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:28:36.563 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":105,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:28:35.598071+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:28:33.803742+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-10-06T02:28:34.570331+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:28:36.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.564+0000 7fdd5d79d700 1 -- 172.21.15.31:0/859951432 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fdd44067050 msgr2=0x7fdd44069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:36.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.564+0000 7fdd5d79d700 1 --2- 172.21.15.31:0/859951432 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fdd44067050 0x7fdd44069500 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdd480093f0 tx=0x7fdd48017000 comp rx=0 tx=0).stop 2023-10-06T02:28:36.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.564+0000 7fdd5d79d700 1 -- 172.21.15.31:0/859951432 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd58140e40 msgr2=0x7fdd5813f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:36.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.564+0000 7fdd5d79d700 1 --2- 172.21.15.31:0/859951432 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd58140e40 0x7fdd5813f940 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fdd5000c910 tx=0x7fdd500027a0 comp rx=0 tx=0).stop 2023-10-06T02:28:36.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.564+0000 7fdd5d79d700 1 -- 172.21.15.31:0/859951432 shutdown_connections 2023-10-06T02:28:36.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.564+0000 7fdd5d79d700 1 --2- 172.21.15.31:0/859951432 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd58140e40 0x7fdd5813f940 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:36.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.564+0000 7fdd5d79d700 1 --2- 172.21.15.31:0/859951432 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fdd44067050 0x7fdd44069500 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:36.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.564+0000 7fdd5d79d700 1 --2- 172.21.15.31:0/859951432 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd5806de50 0x7fdd58130590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:36.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.564+0000 7fdd5d79d700 1 --2- 172.21.15.31:0/859951432 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd58140a70 0x7fdd580ada30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:36.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.564+0000 7fdd5d79d700 1 -- 172.21.15.31:0/859951432 >> 172.21.15.31:0/859951432 conn(0x7fdd5806d170 msgr2=0x7fdd580b1780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:36.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.564+0000 7fdd5d79d700 1 -- 172.21.15.31:0/859951432 shutdown_connections 2023-10-06T02:28:36.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.564+0000 7fdd5d79d700 1 -- 172.21.15.31:0/859951432 wait complete. 2023-10-06T02:28:36.584 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:28:36.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.769+0000 7fb33cd9d700 1 Processor -- start 2023-10-06T02:28:36.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.769+0000 7fb33cd9d700 1 -- start start 2023-10-06T02:28:36.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.770+0000 7fb33cd9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb338159640 0x7fb33806d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:36.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.770+0000 7fb33cd9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb3380caec0 0x7fb3380c78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:36.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.770+0000 7fb33cd9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3380c7420 con 0x7fb338159a10 2023-10-06T02:28:36.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.770+0000 7fb33cd9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb33806d5a0 con 0x7fb338159640 2023-10-06T02:28:36.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.770+0000 7fb33cd9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb33806d820 con 0x7fb3380caec0 2023-10-06T02:28:36.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.771+0000 7fb336ffd700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fb338159a10 0x7fb3380ca7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56784/0 (socket says 172.21.15.31:56784) 2023-10-06T02:28:36.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.771+0000 7fb336ffd700 1 -- 172.21.15.31:0/1803653528 learned_addr learned my addr 172.21.15.31:0/1803653528 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:28:36.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.771+0000 7fb3377fe700 1 --2- 172.21.15.31:0/1803653528 >> v2:172.21.15.62:3300/0 conn(0x7fb338159640 0x7fb33806d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:36.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.771+0000 7fb337fff700 1 --2- 172.21.15.31:0/1803653528 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb3380caec0 0x7fb3380c78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:36.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.771+0000 7fb3367fc700 1 -- 172.21.15.31:0/1803653528 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3499606544 0 0) 0x7fb3380c7420 con 0x7fb338159a10 2023-10-06T02:28:36.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.771+0000 7fb3367fc700 1 -- 172.21.15.31:0/1803653528 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb320003570 con 0x7fb338159a10 2023-10-06T02:28:36.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.771+0000 7fb3377fe700 1 -- 172.21.15.31:0/1803653528 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb3380caec0 msgr2=0x7fb3380c78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:36.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.771+0000 7fb3377fe700 1 --2- 172.21.15.31:0/1803653528 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb3380caec0 0x7fb3380c78b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:36.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.771+0000 7fb3377fe700 1 -- 172.21.15.31:0/1803653528 >> v1:172.21.15.31:6789/0 conn(0x7fb338159a10 legacy=0x7fb3380ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:36.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.771+0000 7fb3377fe700 1 -- 172.21.15.31:0/1803653528 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb33806db00 con 0x7fb338159640 2023-10-06T02:28:36.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.772+0000 7fb3377fe700 1 --2- 172.21.15.31:0/1803653528 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb338159640 0x7fb33806d060 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fb3280098e0 tx=0x7fb3280091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c140167b280ac745 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:36.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.772+0000 7fb3367fc700 1 -- 172.21.15.31:0/1803653528 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb328011630 con 0x7fb338159640 2023-10-06T02:28:36.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.772+0000 7fb3367fc700 1 -- 172.21.15.31:0/1803653528 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb328011790 con 0x7fb338159640 2023-10-06T02:28:36.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.772+0000 7fb3367fc700 1 -- 172.21.15.31:0/1803653528 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb328011b30 con 0x7fb338159640 2023-10-06T02:28:36.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.772+0000 7fb33cd9d700 1 -- 172.21.15.31:0/1803653528 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb338159640 msgr2=0x7fb33806d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:36.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.772+0000 7fb33cd9d700 1 --2- 172.21.15.31:0/1803653528 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb338159640 0x7fb33806d060 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fb3280098e0 tx=0x7fb3280091a0 comp rx=0 tx=0).stop 2023-10-06T02:28:36.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.772+0000 7fb33cd9d700 1 -- 172.21.15.31:0/1803653528 shutdown_connections 2023-10-06T02:28:36.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.772+0000 7fb33cd9d700 1 --2- 172.21.15.31:0/1803653528 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb338159640 0x7fb33806d060 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:36.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.772+0000 7fb33cd9d700 1 --2- 172.21.15.31:0/1803653528 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb3380caec0 0x7fb3380c78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:36.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.772+0000 7fb33cd9d700 1 -- 172.21.15.31:0/1803653528 >> 172.21.15.31:0/1803653528 conn(0x7fb33806e720 msgr2=0x7fb33806eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:36.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.772+0000 7fb33cd9d700 1 -- 172.21.15.31:0/1803653528 shutdown_connections 2023-10-06T02:28:36.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.773+0000 7fb33cd9d700 1 -- 172.21.15.31:0/1803653528 wait complete. 2023-10-06T02:28:36.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.774+0000 7fb33cd9d700 1 Processor -- start 2023-10-06T02:28:36.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.774+0000 7fb33cd9d700 1 -- start start 2023-10-06T02:28:36.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.775+0000 7fb33cd9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3380caec0 0x7fb33814e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:36.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.775+0000 7fb33cd9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb338159640 0x7fb3380b1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:36.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.775+0000 7fb33cd9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb338159a10 0x7fb3380c6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:36.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.775+0000 7fb33cd9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3380c3f90 con 0x7fb3380caec0 2023-10-06T02:28:36.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.775+0000 7fb33cd9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3380b1c90 con 0x7fb338159a10 2023-10-06T02:28:36.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.775+0000 7fb33cd9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3380b1f70 con 0x7fb338159640 2023-10-06T02:28:36.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.775+0000 7fb3377fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3380caec0 0x7fb33814e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:36.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.775+0000 7fb3377fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3380caec0 0x7fb33814e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40654/0 (socket says 172.21.15.31:40654) 2023-10-06T02:28:36.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.775+0000 7fb3377fe700 1 -- 172.21.15.31:0/4001644929 learned_addr learned my addr 172.21.15.31:0/4001644929 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:36.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.775+0000 7fb336ffd700 1 --2- 172.21.15.31:0/4001644929 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb338159640 0x7fb3380b1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:36.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.775+0000 7fb337fff700 1 --2- 172.21.15.31:0/4001644929 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb338159a10 0x7fb3380c6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:36.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.776+0000 7fb3377fe700 1 -- 172.21.15.31:0/4001644929 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb338159640 msgr2=0x7fb3380b1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:36.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.776+0000 7fb3377fe700 1 --2- 172.21.15.31:0/4001644929 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb338159640 0x7fb3380b1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:36.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.776+0000 7fb3377fe700 1 -- 172.21.15.31:0/4001644929 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb338159a10 msgr2=0x7fb3380c6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:36.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.776+0000 7fb3377fe700 1 --2- 172.21.15.31:0/4001644929 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb338159a10 0x7fb3380c6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:36.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.776+0000 7fb3377fe700 1 -- 172.21.15.31:0/4001644929 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb328009420 con 0x7fb3380caec0 2023-10-06T02:28:36.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.776+0000 7fb336ffd700 1 --2- 172.21.15.31:0/4001644929 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb338159640 0x7fb3380b1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:28:36.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.776+0000 7fb3377fe700 1 --2- 172.21.15.31:0/4001644929 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3380caec0 0x7fb33814e3f0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fb3380c3820 tx=0x7fb328017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:36.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.776+0000 7fb31f7fe700 1 -- 172.21.15.31:0/4001644929 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb328011b10 con 0x7fb3380caec0 2023-10-06T02:28:36.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.776+0000 7fb31f7fe700 1 -- 172.21.15.31:0/4001644929 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3280253f0 con 0x7fb3380caec0 2023-10-06T02:28:36.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.776+0000 7fb31f7fe700 1 -- 172.21.15.31:0/4001644929 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb328025790 con 0x7fb3380caec0 2023-10-06T02:28:36.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.776+0000 7fb33cd9d700 1 -- 172.21.15.31:0/4001644929 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3380c3c20 con 0x7fb3380caec0 2023-10-06T02:28:36.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.776+0000 7fb33cd9d700 1 -- 172.21.15.31:0/4001644929 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb338152220 con 0x7fb3380caec0 2023-10-06T02:28:36.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.777+0000 7fb33cd9d700 1 -- 172.21.15.31:0/4001644929 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3380c3a60 con 0x7fb3380caec0 2023-10-06T02:28:36.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.778+0000 7fb31f7fe700 1 -- 172.21.15.31:0/4001644929 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7fb328025b80 con 0x7fb3380caec0 2023-10-06T02:28:36.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.779+0000 7fb31f7fe700 1 --2- 172.21.15.31:0/4001644929 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb324067050 0x7fb324069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:36.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.779+0000 7fb31f7fe700 1 -- 172.21.15.31:0/4001644929 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb328086080 con 0x7fb3380caec0 2023-10-06T02:28:36.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.780+0000 7fb336ffd700 1 --2- 172.21.15.31:0/4001644929 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb324067050 0x7fb324069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:36.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.780+0000 7fb336ffd700 1 --2- 172.21.15.31:0/4001644929 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb324067050 0x7fb324069500 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb32c007fd0 tx=0x7fb32c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:36.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.783+0000 7fb31f7fe700 1 -- 172.21.15.31:0/4001644929 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb328053080 con 0x7fb3380caec0 2023-10-06T02:28:36.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.986+0000 7fb33cd9d700 1 -- 172.21.15.31:0/4001644929 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb33804c3c0 con 0x7fb3380caec0 2023-10-06T02:28:36.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.987+0000 7fb31f7fe700 1 -- 172.21.15.31:0/4001644929 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) v1 ==== 74+0+16845 (secure 0 0 0) 0x7fb3280566a0 con 0x7fb3380caec0 2023-10-06T02:28:36.989 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:28:36.989 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":105,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:28:35.598071+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:28:33.803742+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-10-06T02:28:34.570331+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:28:36.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.989+0000 7fb33cd9d700 1 -- 172.21.15.31:0/4001644929 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb324067050 msgr2=0x7fb324069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:36.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.989+0000 7fb33cd9d700 1 --2- 172.21.15.31:0/4001644929 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb324067050 0x7fb324069500 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb32c007fd0 tx=0x7fb32c009040 comp rx=0 tx=0).stop 2023-10-06T02:28:36.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.989+0000 7fb33cd9d700 1 -- 172.21.15.31:0/4001644929 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3380caec0 msgr2=0x7fb33814e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:36.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.989+0000 7fb33cd9d700 1 --2- 172.21.15.31:0/4001644929 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3380caec0 0x7fb33814e3f0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fb3380c3820 tx=0x7fb328017000 comp rx=0 tx=0).stop 2023-10-06T02:28:36.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.989+0000 7fb33cd9d700 1 -- 172.21.15.31:0/4001644929 shutdown_connections 2023-10-06T02:28:36.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.989+0000 7fb33cd9d700 1 --2- 172.21.15.31:0/4001644929 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb338159a10 0x7fb3380c6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:36.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.989+0000 7fb33cd9d700 1 --2- 172.21.15.31:0/4001644929 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3380caec0 0x7fb33814e3f0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:36.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.989+0000 7fb33cd9d700 1 --2- 172.21.15.31:0/4001644929 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb324067050 0x7fb324069500 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:36.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.989+0000 7fb33cd9d700 1 --2- 172.21.15.31:0/4001644929 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb338159640 0x7fb3380b1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:36.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.989+0000 7fb33cd9d700 1 -- 172.21.15.31:0/4001644929 >> 172.21.15.31:0/4001644929 conn(0x7fb33806e720 msgr2=0x7fb33806a810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:36.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.989+0000 7fb33cd9d700 1 -- 172.21.15.31:0/4001644929 shutdown_connections 2023-10-06T02:28:36.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:36.989+0000 7fb33cd9d700 1 -- 172.21.15.31:0/4001644929 wait complete. 2023-10-06T02:28:37.009 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T02:28:37.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:37.201+0000 7fa5749fe700 1 Processor -- start 2023-10-06T02:28:37.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:37.201+0000 7fa5749fe700 1 -- start start 2023-10-06T02:28:37.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:37.202+0000 7fa5749fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa570155890 0x7fa5700c7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:37.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:37.202+0000 7fa5749fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa57006de50 0x7fa57006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:37.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:37.202+0000 7fa5749fe700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5700c7e70 con 0x7fa570155c60 2023-10-06T02:28:37.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:37.202+0000 7fa5749fe700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa57006e830 con 0x7fa57006de50 2023-10-06T02:28:37.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:37.202+0000 7fa5749fe700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa57006b790 con 0x7fa570155890 2023-10-06T02:28:37.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:37.202+0000 7fa56f59e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa57006de50 0x7fa57006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:37.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:37.202+0000 7fa56f59e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa57006de50 0x7fa57006e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40658/0 (socket says 172.21.15.31:40658) 2023-10-06T02:28:37.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:37.202+0000 7fa56f59e700 1 -- 172.21.15.31:0/3437424177 learned_addr learned my addr 172.21.15.31:0/3437424177 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:37.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:37.202+0000 7fa56ed9d700 1 --2- 172.21.15.31:0/3437424177 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa570155890 0x7fa5700c7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:37.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:37.203+0000 7fa56f59e700 1 -- 172.21.15.31:0/3437424177 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa570155890 msgr2=0x7fa5700c7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:37.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:37.203+0000 7fa56f59e700 1 --2- 172.21.15.31:0/3437424177 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa570155890 0x7fa5700c7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:37.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:37.203+0000 7fa56f59e700 1 -- 172.21.15.31:0/3437424177 >> v1:172.21.15.62:6789/0 conn(0x7fa570155c60 legacy=0x7fa57006a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:37.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:37.203+0000 7fa56f59e700 1 -- 172.21.15.31:0/3437424177 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa57006ab80 con 0x7fa57006de50 2023-10-06T02:28:37.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:37.203+0000 7fa56f59e700 1 --2- 172.21.15.31:0/3437424177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa57006de50 0x7fa57006e2f0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fa568009f20 tx=0x7fa568009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81bbbf8d134cd660 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:37.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:37.203+0000 7fa56dd9b700 1 -- 172.21.15.31:0/3437424177 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa56800e070 con 0x7fa57006de50 2023-10-06T02:28:37.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:37.203+0000 7fa56dd9b700 1 -- 172.21.15.31:0/3437424177 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa56800ba80 con 0x7fa57006de50 2023-10-06T02:28:37.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:37.203+0000 7fa56dd9b700 1 -- 172.21.15.31:0/3437424177 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5680092e0 con 0x7fa57006de50 2023-10-06T02:28:37.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:37.204+0000 7fa5749fe700 1 -- 172.21.15.31:0/3437424177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa57006de50 msgr2=0x7fa57006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:37.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:37.204+0000 7fa5749fe700 1 --2- 172.21.15.31:0/3437424177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa57006de50 0x7fa57006e2f0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fa568009f20 tx=0x7fa568009840 comp rx=0 tx=0).stop 2023-10-06T02:28:37.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:37.204+0000 7fa5749fe700 1 -- 172.21.15.31:0/3437424177 shutdown_connections 2023-10-06T02:28:37.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:37.204+0000 7fa5749fe700 1 --2- 172.21.15.31:0/3437424177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa57006de50 0x7fa57006e2f0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:37.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:37.204+0000 7fa5749fe700 1 --2- 172.21.15.31:0/3437424177 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa570155890 0x7fa5700c7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:37.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:37.204+0000 7fa5749fe700 1 -- 172.21.15.31:0/3437424177 >> 172.21.15.31:0/3437424177 conn(0x7fa57006d110 msgr2=0x7fa5700cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:37.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:37.204+0000 7fa5749fe700 1 -- 172.21.15.31:0/3437424177 shutdown_connections 2023-10-06T02:28:37.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:37.204+0000 7fa5749fe700 1 -- 172.21.15.31:0/3437424177 wait complete. 2023-10-06T02:28:37.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:37.205+0000 7fa5749fe700 1 Processor -- start 2023-10-06T02:28:37.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:37.205+0000 7fa5749fe700 1 -- start start 2023-10-06T02:28:37.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:37.206+0000 7fa5749fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa57006de50 0x7fa570153ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:37.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:37.206+0000 7fa5749fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa570155890 0x7fa570154430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:37.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:37.206+0000 7fa5749fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa570155c60 0x7fa5700b1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:37.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:37.206+0000 7fa5749fe700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa570156720 con 0x7fa57006de50 2023-10-06T02:28:37.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:37.206+0000 7fa5749fe700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa570156990 con 0x7fa570155890 2023-10-06T02:28:37.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:37.206+0000 7fa5749fe700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa570156c70 con 0x7fa570155c60 2023-10-06T02:28:37.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:37.207+0000 7fa56ed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa57006de50 0x7fa570153ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:37.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:37.207+0000 7fa56ed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa57006de50 0x7fa570153ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40664/0 (socket says 172.21.15.31:40664) 2023-10-06T02:28:37.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:37.207+0000 7fa56ed9d700 1 -- 172.21.15.31:0/3636541881 learned_addr learned my addr 172.21.15.31:0/3636541881 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:37.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:37.207+0000 7fa56f59e700 1 --2- 172.21.15.31:0/3636541881 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa570155c60 0x7fa5700b1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:37.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:37.207+0000 7fa56e59c700 1 --2- 172.21.15.31:0/3636541881 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa570155890 0x7fa570154430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:37.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:37.207+0000 7fa56ed9d700 1 -- 172.21.15.31:0/3636541881 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa570155c60 msgr2=0x7fa5700b1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:37.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:37.207+0000 7fa56ed9d700 1 --2- 172.21.15.31:0/3636541881 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa570155c60 0x7fa5700b1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:37.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:37.207+0000 7fa56ed9d700 1 -- 172.21.15.31:0/3636541881 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa570155890 msgr2=0x7fa570154430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:37.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:37.207+0000 7fa56ed9d700 1 --2- 172.21.15.31:0/3636541881 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa570155890 0x7fa570154430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:37.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:37.207+0000 7fa56ed9d700 1 -- 172.21.15.31:0/3636541881 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa568009ae0 con 0x7fa57006de50 2023-10-06T02:28:37.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:37.207+0000 7fa56e59c700 1 --2- 172.21.15.31:0/3636541881 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa570155890 0x7fa570154430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:28:37.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:37.207+0000 7fa56ed9d700 1 --2- 172.21.15.31:0/3636541881 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa57006de50 0x7fa570153ef0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fa56000d890 tx=0x7fa560008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:37.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:37.207+0000 7fa556ffd700 1 -- 172.21.15.31:0/3636541881 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5600125f0 con 0x7fa57006de50 2023-10-06T02:28:37.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:37.208+0000 7fa556ffd700 1 -- 172.21.15.31:0/3636541881 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa560012750 con 0x7fa57006de50 2023-10-06T02:28:37.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:37.208+0000 7fa556ffd700 1 -- 172.21.15.31:0/3636541881 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa560012ad0 con 0x7fa57006de50 2023-10-06T02:28:37.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:37.208+0000 7fa5749fe700 1 -- 172.21.15.31:0/3636541881 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa570156f50 con 0x7fa57006de50 2023-10-06T02:28:37.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:37.208+0000 7fa5749fe700 1 -- 172.21.15.31:0/3636541881 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5700b18b0 con 0x7fa57006de50 2023-10-06T02:28:37.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:37.209+0000 7fa5749fe700 1 -- 172.21.15.31:0/3636541881 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa57004c3c0 con 0x7fa57006de50 2023-10-06T02:28:37.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:37.210+0000 7fa556ffd700 1 -- 172.21.15.31:0/3636541881 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7fa56001e460 con 0x7fa57006de50 2023-10-06T02:28:37.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:37.211+0000 7fa556ffd700 1 --2- 172.21.15.31:0/3636541881 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa55c067050 0x7fa55c069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:37.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:37.211+0000 7fa556ffd700 1 -- 172.21.15.31:0/3636541881 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa560084ec0 con 0x7fa57006de50 2023-10-06T02:28:37.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:37.211+0000 7fa56e59c700 1 --2- 172.21.15.31:0/3636541881 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa55c067050 0x7fa55c069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:37.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:37.212+0000 7fa56e59c700 1 --2- 172.21.15.31:0/3636541881 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa55c067050 0x7fa55c069500 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa564007fd0 tx=0x7fa564009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:37.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:37.214+0000 7fa556ffd700 1 -- 172.21.15.31:0/3636541881 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa560051e20 con 0x7fa57006de50 2023-10-06T02:28:37.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:37.466+0000 7fa5749fe700 1 -- 172.21.15.31:0/3636541881 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fa57006f170 con 0x7fa57006de50 2023-10-06T02:28:38.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:38.126+0000 7fa556ffd700 1 -- 172.21.15.31:0/3636541881 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v70) v1 ==== 118+0+0 (secure 0 0 0) 0x7fa560055440 con 0x7fa57006de50 2023-10-06T02:28:38.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:38.129+0000 7fa5749fe700 1 -- 172.21.15.31:0/3636541881 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa55c067050 msgr2=0x7fa55c069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:38.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:38.129+0000 7fa5749fe700 1 --2- 172.21.15.31:0/3636541881 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa55c067050 0x7fa55c069500 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa564007fd0 tx=0x7fa564009040 comp rx=0 tx=0).stop 2023-10-06T02:28:38.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:38.129+0000 7fa5749fe700 1 -- 172.21.15.31:0/3636541881 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa57006de50 msgr2=0x7fa570153ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:38.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:38.129+0000 7fa5749fe700 1 --2- 172.21.15.31:0/3636541881 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa57006de50 0x7fa570153ef0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fa56000d890 tx=0x7fa560008040 comp rx=0 tx=0).stop 2023-10-06T02:28:38.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:38.129+0000 7fa5749fe700 1 -- 172.21.15.31:0/3636541881 shutdown_connections 2023-10-06T02:28:38.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:38.129+0000 7fa5749fe700 1 --2- 172.21.15.31:0/3636541881 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa55c067050 0x7fa55c069500 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:38.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:38.129+0000 7fa5749fe700 1 --2- 172.21.15.31:0/3636541881 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa570155c60 0x7fa5700b1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:38.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:38.129+0000 7fa5749fe700 1 --2- 172.21.15.31:0/3636541881 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa57006de50 0x7fa570153ef0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:38.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:38.129+0000 7fa5749fe700 1 --2- 172.21.15.31:0/3636541881 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa570155890 0x7fa570154430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:38.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:38.129+0000 7fa5749fe700 1 -- 172.21.15.31:0/3636541881 >> 172.21.15.31:0/3636541881 conn(0x7fa57006d110 msgr2=0x7fa5700c1760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:38.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:38.129+0000 7fa5749fe700 1 -- 172.21.15.31:0/3636541881 shutdown_connections 2023-10-06T02:28:38.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:38.129+0000 7fa5749fe700 1 -- 172.21.15.31:0/3636541881 wait complete. 2023-10-06T02:28:38.152 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T02:28:38.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:38.349+0000 7f3f4df9d700 1 Processor -- start 2023-10-06T02:28:38.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:38.349+0000 7f3f4df9d700 1 -- start start 2023-10-06T02:28:38.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:38.350+0000 7f3f4df9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3f48155c60 0x7f3f480c7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:38.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:38.350+0000 7f3f4df9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3f4806de50 0x7f3f4806e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:38.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:38.350+0000 7f3f4df9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f480c7e70 con 0x7f3f48155890 2023-10-06T02:28:38.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:38.350+0000 7f3f4df9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f4806e810 con 0x7f3f48155c60 2023-10-06T02:28:38.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:38.350+0000 7f3f4df9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4806b790 con 0x7f3f4806de50 2023-10-06T02:28:38.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:38.351+0000 7f3f4cf9b700 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f3f48155890 0x7f3f4806a3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:55144/0 (socket says 172.21.15.31:55144) 2023-10-06T02:28:38.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:38.351+0000 7f3f4d79c700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3f4806de50 0x7f3f4806e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:38.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:38.351+0000 7f3f4cf9b700 1 -- 172.21.15.31:0/4294095593 learned_addr learned my addr 172.21.15.31:0/4294095593 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:28:38.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:38.351+0000 7f3f47fff700 1 --2- 172.21.15.31:0/4294095593 >> v2:172.21.15.62:3300/0 conn(0x7f3f48155c60 0x7f3f480c7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:38.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:38.351+0000 7f3f477fe700 1 -- 172.21.15.31:0/4294095593 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1589223779 0 0) 0x7f3f480c7e70 con 0x7f3f48155890 2023-10-06T02:28:38.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:38.351+0000 7f3f477fe700 1 -- 172.21.15.31:0/4294095593 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f38004680 con 0x7f3f48155890 2023-10-06T02:28:38.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:38.351+0000 7f3f4d79c700 1 -- 172.21.15.31:0/4294095593 >> v2:172.21.15.62:3300/0 conn(0x7f3f48155c60 msgr2=0x7f3f480c7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:38.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:38.351+0000 7f3f4d79c700 1 --2- 172.21.15.31:0/4294095593 >> v2:172.21.15.62:3300/0 conn(0x7f3f48155c60 0x7f3f480c7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:38.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:38.351+0000 7f3f4d79c700 1 -- 172.21.15.31:0/4294095593 >> v1:172.21.15.62:6789/0 conn(0x7f3f48155890 legacy=0x7f3f4806a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:38.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:38.351+0000 7f3f4d79c700 1 -- 172.21.15.31:0/4294095593 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f4806ab80 con 0x7f3f4806de50 2023-10-06T02:28:38.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:38.352+0000 7f3f4d79c700 1 --2- 172.21.15.31:0/4294095593 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3f4806de50 0x7f3f4806e2d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3f40009e60 tx=0x7f3f40009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=122b5e8971ea0489 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:38.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:38.352+0000 7f3f477fe700 1 -- 172.21.15.31:0/4294095593 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f4000e070 con 0x7f3f4806de50 2023-10-06T02:28:38.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:38.352+0000 7f3f477fe700 1 -- 172.21.15.31:0/4294095593 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f4000bab0 con 0x7f3f4806de50 2023-10-06T02:28:38.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:38.352+0000 7f3f477fe700 1 -- 172.21.15.31:0/4294095593 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f4000be70 con 0x7f3f4806de50 2023-10-06T02:28:38.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:38.352+0000 7f3f4df9d700 1 -- 172.21.15.31:0/4294095593 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3f4806de50 msgr2=0x7f3f4806e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:38.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:38.352+0000 7f3f4df9d700 1 --2- 172.21.15.31:0/4294095593 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3f4806de50 0x7f3f4806e2d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3f40009e60 tx=0x7f3f40009780 comp rx=0 tx=0).stop 2023-10-06T02:28:38.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:38.353+0000 7f3f4df9d700 1 -- 172.21.15.31:0/4294095593 shutdown_connections 2023-10-06T02:28:38.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:38.353+0000 7f3f4df9d700 1 --2- 172.21.15.31:0/4294095593 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3f4806de50 0x7f3f4806e2d0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:38.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:38.353+0000 7f3f4df9d700 1 --2- 172.21.15.31:0/4294095593 >> v2:172.21.15.62:3300/0 conn(0x7f3f48155c60 0x7f3f480c7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:38.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:38.353+0000 7f3f4df9d700 1 -- 172.21.15.31:0/4294095593 >> 172.21.15.31:0/4294095593 conn(0x7f3f4806d110 msgr2=0x7f3f480cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:38.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:38.353+0000 7f3f4df9d700 1 -- 172.21.15.31:0/4294095593 shutdown_connections 2023-10-06T02:28:38.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:38.353+0000 7f3f4df9d700 1 -- 172.21.15.31:0/4294095593 wait complete. 2023-10-06T02:28:38.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:38.354+0000 7f3f4df9d700 1 Processor -- start 2023-10-06T02:28:38.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:38.354+0000 7f3f4df9d700 1 -- start start 2023-10-06T02:28:38.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:38.355+0000 7f3f4df9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f4806de50 0x7f3f48151cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:38.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:38.355+0000 7f3f4df9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f48155890 0x7f3f4814c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:38.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:38.355+0000 7f3f4df9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3f48155c60 0x7f3f480b1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:38.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:38.355+0000 7f3f4df9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4814ce70 con 0x7f3f4806de50 2023-10-06T02:28:38.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:38.355+0000 7f3f4df9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f48168130 con 0x7f3f48155890 2023-10-06T02:28:38.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:38.355+0000 7f3f4df9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f48167370 con 0x7f3f48155c60 2023-10-06T02:28:38.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:38.355+0000 7f3f4cf9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f4806de50 0x7f3f48151cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:38.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:38.355+0000 7f3f4cf9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f4806de50 0x7f3f48151cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36722/0 (socket says 172.21.15.31:36722) 2023-10-06T02:28:38.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:38.355+0000 7f3f4cf9b700 1 -- 172.21.15.31:0/2281950451 learned_addr learned my addr 172.21.15.31:0/2281950451 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:38.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:38.356+0000 7f3f4d79c700 1 --2- 172.21.15.31:0/2281950451 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3f48155c60 0x7f3f480b1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:38.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:38.356+0000 7f3f47fff700 1 --2- 172.21.15.31:0/2281950451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f48155890 0x7f3f4814c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:38.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:38.356+0000 7f3f4cf9b700 1 -- 172.21.15.31:0/2281950451 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3f48155c60 msgr2=0x7f3f480b1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:38.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:38.356+0000 7f3f4cf9b700 1 --2- 172.21.15.31:0/2281950451 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3f48155c60 0x7f3f480b1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:38.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:38.356+0000 7f3f4cf9b700 1 -- 172.21.15.31:0/2281950451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f48155890 msgr2=0x7f3f4814c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:38.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:38.356+0000 7f3f4cf9b700 1 --2- 172.21.15.31:0/2281950451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f48155890 0x7f3f4814c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:38.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:38.356+0000 7f3f4cf9b700 1 -- 172.21.15.31:0/2281950451 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f40009a20 con 0x7f3f4806de50 2023-10-06T02:28:38.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:38.356+0000 7f3f4d79c700 1 --2- 172.21.15.31:0/2281950451 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3f48155c60 0x7f3f480b1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:28:38.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:38.356+0000 7f3f47fff700 1 --2- 172.21.15.31:0/2281950451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f48155890 0x7f3f4814c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:28:38.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:38.356+0000 7f3f4cf9b700 1 --2- 172.21.15.31:0/2281950451 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f4806de50 0x7f3f48151cf0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c00c7f0 tx=0x7f3f3c007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:38.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:38.356+0000 7f3f44ff9700 1 -- 172.21.15.31:0/2281950451 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f3c005b30 con 0x7f3f4806de50 2023-10-06T02:28:38.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:38.356+0000 7f3f44ff9700 1 -- 172.21.15.31:0/2281950451 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f3c005c90 con 0x7f3f4806de50 2023-10-06T02:28:38.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:38.356+0000 7f3f44ff9700 1 -- 172.21.15.31:0/2281950451 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f3c007320 con 0x7f3f4806de50 2023-10-06T02:28:38.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:38.356+0000 7f3f4df9d700 1 -- 172.21.15.31:0/2281950451 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f48167650 con 0x7f3f4806de50 2023-10-06T02:28:38.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:38.357+0000 7f3f4df9d700 1 -- 172.21.15.31:0/2281950451 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f48167c10 con 0x7f3f4806de50 2023-10-06T02:28:38.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:38.358+0000 7f3f4df9d700 1 -- 172.21.15.31:0/2281950451 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f4804c3c0 con 0x7f3f4806de50 2023-10-06T02:28:38.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:38.358+0000 7f3f44ff9700 1 -- 172.21.15.31:0/2281950451 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f3f3c01b400 con 0x7f3f4806de50 2023-10-06T02:28:38.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:38.359+0000 7f3f44ff9700 1 --2- 172.21.15.31:0/2281950451 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f3f2c067050 0x7f3f2c069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:38.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:38.359+0000 7f3f44ff9700 1 -- 172.21.15.31:0/2281950451 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3f3c084310 con 0x7f3f4806de50 2023-10-06T02:28:38.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:38.360+0000 7f3f47fff700 1 --2- 172.21.15.31:0/2281950451 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f3f2c067050 0x7f3f2c069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:38.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:38.360+0000 7f3f47fff700 1 --2- 172.21.15.31:0/2281950451 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f3f2c067050 0x7f3f2c069500 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3f34006f00 tx=0x7f3f34008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:38.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:38.363+0000 7f3f44ff9700 1 -- 172.21.15.31:0/2281950451 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3f3c0512f0 con 0x7f3f4806de50 2023-10-06T02:28:38.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:38.610+0000 7f3f4df9d700 1 -- 172.21.15.31:0/2281950451 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f3f48158440 con 0x7f3f4806de50 2023-10-06T02:28:39.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.131+0000 7f3f44ff9700 1 -- 172.21.15.31:0/2281950451 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v71) v1 ==== 122+0+0 (secure 0 0 0) 0x7f3f3c054910 con 0x7f3f4806de50 2023-10-06T02:28:39.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.134+0000 7f3f4df9d700 1 -- 172.21.15.31:0/2281950451 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f3f2c067050 msgr2=0x7f3f2c069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:39.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.134+0000 7f3f4df9d700 1 --2- 172.21.15.31:0/2281950451 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f3f2c067050 0x7f3f2c069500 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3f34006f00 tx=0x7f3f34008040 comp rx=0 tx=0).stop 2023-10-06T02:28:39.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.134+0000 7f3f4df9d700 1 -- 172.21.15.31:0/2281950451 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f4806de50 msgr2=0x7f3f48151cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:39.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.134+0000 7f3f4df9d700 1 --2- 172.21.15.31:0/2281950451 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f4806de50 0x7f3f48151cf0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c00c7f0 tx=0x7f3f3c007e00 comp rx=0 tx=0).stop 2023-10-06T02:28:39.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.134+0000 7f3f4df9d700 1 -- 172.21.15.31:0/2281950451 shutdown_connections 2023-10-06T02:28:39.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.134+0000 7f3f4df9d700 1 --2- 172.21.15.31:0/2281950451 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f3f2c067050 0x7f3f2c069500 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:39.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.134+0000 7f3f4df9d700 1 --2- 172.21.15.31:0/2281950451 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3f48155c60 0x7f3f480b1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:39.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.134+0000 7f3f4df9d700 1 --2- 172.21.15.31:0/2281950451 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f4806de50 0x7f3f48151cf0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:39.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.134+0000 7f3f4df9d700 1 --2- 172.21.15.31:0/2281950451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f48155890 0x7f3f4814c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:39.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.134+0000 7f3f4df9d700 1 -- 172.21.15.31:0/2281950451 >> 172.21.15.31:0/2281950451 conn(0x7f3f4806d110 msgr2=0x7f3f48154b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:39.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.135+0000 7f3f4df9d700 1 -- 172.21.15.31:0/2281950451 shutdown_connections 2023-10-06T02:28:39.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.135+0000 7f3f4df9d700 1 -- 172.21.15.31:0/2281950451 wait complete. 2023-10-06T02:28:39.156 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:28:39.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.344+0000 7f317359e700 1 Processor -- start 2023-10-06T02:28:39.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.344+0000 7f317359e700 1 -- start start 2023-10-06T02:28:39.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.345+0000 7f317359e700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3174153c30 0x7f317415a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:39.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.345+0000 7f317359e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f31740cf110 0x7f317415c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:39.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.345+0000 7f317359e700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f317415afd0 con 0x7f317415de50 2023-10-06T02:28:39.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.345+0000 7f317359e700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f317415b2b0 con 0x7f3174153c30 2023-10-06T02:28:39.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.346+0000 7f317359e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f317415b590 con 0x7f31740cf110 2023-10-06T02:28:39.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.346+0000 7f3172d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f31740cf110 0x7f317415c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:39.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.346+0000 7f317259c700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3174153c30 0x7f317415a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:39.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.346+0000 7f3171d9b700 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f317415de50 0x7f31740cea00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:55160/0 (socket says 172.21.15.31:55160) 2023-10-06T02:28:39.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.346+0000 7f3171d9b700 1 -- 172.21.15.31:0/2614315583 learned_addr learned my addr 172.21.15.31:0/2614315583 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:28:39.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.346+0000 7f3172d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f31740cf110 0x7f317415c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:41676/0 (socket says 172.21.15.31:41676) 2023-10-06T02:28:39.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.347+0000 7f3172d9d700 1 -- 172.21.15.31:0/2614315583 >> v2:172.21.15.62:3300/0 conn(0x7f3174153c30 msgr2=0x7f317415a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:39.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.347+0000 7f3172d9d700 1 --2- 172.21.15.31:0/2614315583 >> v2:172.21.15.62:3300/0 conn(0x7f3174153c30 0x7f317415a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:39.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.347+0000 7f3172d9d700 1 -- 172.21.15.31:0/2614315583 >> v1:172.21.15.62:6789/0 conn(0x7f317415de50 legacy=0x7f31740cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:39.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.347+0000 7f3172d9d700 1 -- 172.21.15.31:0/2614315583 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f317415b870 con 0x7f31740cf110 2023-10-06T02:28:39.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.347+0000 7f317259c700 1 --2- 172.21.15.31:0/2614315583 >> v2:172.21.15.62:3300/0 conn(0x7f3174153c30 0x7f317415a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:28:39.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.347+0000 7f3172d9d700 1 --2- 172.21.15.31:0/2614315583 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f31740cf110 0x7f317415c070 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f316c009e60 tx=0x7f316c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd0fdf4045d75944 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:39.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.347+0000 7f317159a700 1 -- 172.21.15.31:0/2614315583 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f316c00e070 con 0x7f31740cf110 2023-10-06T02:28:39.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.347+0000 7f317159a700 1 -- 172.21.15.31:0/2614315583 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f316c00bb60 con 0x7f31740cf110 2023-10-06T02:28:39.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.347+0000 7f317159a700 1 -- 172.21.15.31:0/2614315583 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f316c012430 con 0x7f31740cf110 2023-10-06T02:28:39.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.348+0000 7f317359e700 1 -- 172.21.15.31:0/2614315583 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f31740cf110 msgr2=0x7f317415c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:39.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.348+0000 7f317359e700 1 --2- 172.21.15.31:0/2614315583 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f31740cf110 0x7f317415c070 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f316c009e60 tx=0x7f316c009780 comp rx=0 tx=0).stop 2023-10-06T02:28:39.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.348+0000 7f317359e700 1 -- 172.21.15.31:0/2614315583 shutdown_connections 2023-10-06T02:28:39.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.348+0000 7f317359e700 1 --2- 172.21.15.31:0/2614315583 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f31740cf110 0x7f317415c070 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:39.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.348+0000 7f317359e700 1 --2- 172.21.15.31:0/2614315583 >> v2:172.21.15.62:3300/0 conn(0x7f3174153c30 0x7f317415a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:39.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.348+0000 7f317359e700 1 -- 172.21.15.31:0/2614315583 >> 172.21.15.31:0/2614315583 conn(0x7f31740703a0 msgr2=0x7f31740707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:39.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.348+0000 7f317359e700 1 -- 172.21.15.31:0/2614315583 shutdown_connections 2023-10-06T02:28:39.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.348+0000 7f317359e700 1 -- 172.21.15.31:0/2614315583 wait complete. 2023-10-06T02:28:39.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.350+0000 7f317359e700 1 Processor -- start 2023-10-06T02:28:39.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.350+0000 7f317359e700 1 -- start start 2023-10-06T02:28:39.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.351+0000 7f317359e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31740cf110 0x7f317413fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:39.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.351+0000 7f317359e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3174153c30 0x7f317415f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:39.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.351+0000 7f317359e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f317415de50 0x7f317414f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:39.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.351+0000 7f317359e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31740b0690 con 0x7f31740cf110 2023-10-06T02:28:39.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.351+0000 7f317359e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31740b0970 con 0x7f3174153c30 2023-10-06T02:28:39.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.351+0000 7f317359e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f317415f9b0 con 0x7f317415de50 2023-10-06T02:28:39.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.351+0000 7f317259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31740cf110 0x7f317413fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:39.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.351+0000 7f317259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31740cf110 0x7f317413fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36736/0 (socket says 172.21.15.31:36736) 2023-10-06T02:28:39.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.351+0000 7f317259c700 1 -- 172.21.15.31:0/2411568049 learned_addr learned my addr 172.21.15.31:0/2411568049 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:39.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.351+0000 7f3172d9d700 1 --2- 172.21.15.31:0/2411568049 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f317415de50 0x7f317414f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:39.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.351+0000 7f3171d9b700 1 --2- 172.21.15.31:0/2411568049 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3174153c30 0x7f317415f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:39.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.351+0000 7f317259c700 1 -- 172.21.15.31:0/2411568049 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f317415de50 msgr2=0x7f317414f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:39.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.351+0000 7f317259c700 1 --2- 172.21.15.31:0/2411568049 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f317415de50 0x7f317414f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:39.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.351+0000 7f317259c700 1 -- 172.21.15.31:0/2411568049 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3174153c30 msgr2=0x7f317415f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:39.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.351+0000 7f317259c700 1 --2- 172.21.15.31:0/2411568049 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3174153c30 0x7f317415f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:39.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.351+0000 7f317259c700 1 -- 172.21.15.31:0/2411568049 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3164009420 con 0x7f31740cf110 2023-10-06T02:28:39.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.351+0000 7f3171d9b700 1 --2- 172.21.15.31:0/2411568049 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3174153c30 0x7f317415f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:28:39.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.352+0000 7f317259c700 1 --2- 172.21.15.31:0/2411568049 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31740cf110 0x7f317413fee0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f316400ec80 tx=0x7f31640091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:39.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.352+0000 7f31627fc700 1 -- 172.21.15.31:0/2411568049 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f316400c9d0 con 0x7f31740cf110 2023-10-06T02:28:39.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.352+0000 7f31627fc700 1 -- 172.21.15.31:0/2411568049 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f316400cb30 con 0x7f31740cf110 2023-10-06T02:28:39.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.352+0000 7f31627fc700 1 -- 172.21.15.31:0/2411568049 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f316400ceb0 con 0x7f31740cf110 2023-10-06T02:28:39.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.352+0000 7f317359e700 1 -- 172.21.15.31:0/2411568049 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f316c009a20 con 0x7f31740cf110 2023-10-06T02:28:39.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.352+0000 7f317359e700 1 -- 172.21.15.31:0/2411568049 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f317415ff70 con 0x7f31740cf110 2023-10-06T02:28:39.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.353+0000 7f317359e700 1 -- 172.21.15.31:0/2411568049 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31740b0c50 con 0x7f31740cf110 2023-10-06T02:28:39.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.354+0000 7f31627fc700 1 -- 172.21.15.31:0/2411568049 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f3164013680 con 0x7f31740cf110 2023-10-06T02:28:39.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.355+0000 7f31627fc700 1 --2- 172.21.15.31:0/2411568049 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f3154067050 0x7f3154069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:39.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.355+0000 7f31627fc700 1 -- 172.21.15.31:0/2411568049 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3164084c40 con 0x7f31740cf110 2023-10-06T02:28:39.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.355+0000 7f3171d9b700 1 --2- 172.21.15.31:0/2411568049 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f3154067050 0x7f3154069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:39.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.356+0000 7f3171d9b700 1 --2- 172.21.15.31:0/2411568049 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f3154067050 0x7f3154069500 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3168001780 tx=0x7f3168009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:39.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.359+0000 7f31627fc700 1 -- 172.21.15.31:0/2411568049 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3164051c40 con 0x7f31740cf110 2023-10-06T02:28:39.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.604+0000 7f317359e700 1 -- 172.21.15.31:0/2411568049 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f317404c3c0 con 0x7f31740cf110 2023-10-06T02:28:39.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.605+0000 7f31627fc700 1 -- 172.21.15.31:0/2411568049 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f3164055260 con 0x7f31740cf110 2023-10-06T02:28:39.607 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:28:39.607 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":71,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6026,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/3801334948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3801334948},{"type":"v1","addr":"172.21.15.62:6834","nonce":3801334948}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":6724,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/1804582836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":1804582836},{"type":"v1","addr":"172.21.15.31:6835","nonce":1804582836}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":6736,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1428432734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":1428432734},{"type":"v1","addr":"172.21.15.62:6835","nonce":1428432734}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:28:35.598388+0000","modified":"2023-10-06T02:28:39.126355+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7065},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7065":{"gid":7065,"name":"a","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6834/603135594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":603135594},{"type":"v1","addr":"172.21.15.31:6834","nonce":603135594}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-06T02:28:39.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.607+0000 7f317359e700 1 -- 172.21.15.31:0/2411568049 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f3154067050 msgr2=0x7f3154069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:39.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.607+0000 7f317359e700 1 --2- 172.21.15.31:0/2411568049 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f3154067050 0x7f3154069500 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3168001780 tx=0x7f3168009040 comp rx=0 tx=0).stop 2023-10-06T02:28:39.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.607+0000 7f317359e700 1 -- 172.21.15.31:0/2411568049 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31740cf110 msgr2=0x7f317413fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:39.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.607+0000 7f317359e700 1 --2- 172.21.15.31:0/2411568049 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31740cf110 0x7f317413fee0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f316400ec80 tx=0x7f31640091a0 comp rx=0 tx=0).stop 2023-10-06T02:28:39.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.607+0000 7f317359e700 1 -- 172.21.15.31:0/2411568049 shutdown_connections 2023-10-06T02:28:39.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.607+0000 7f317359e700 1 --2- 172.21.15.31:0/2411568049 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f3154067050 0x7f3154069500 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:39.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.607+0000 7f317359e700 1 --2- 172.21.15.31:0/2411568049 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f317415de50 0x7f317414f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:39.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.607+0000 7f317359e700 1 --2- 172.21.15.31:0/2411568049 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31740cf110 0x7f317413fee0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:39.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.607+0000 7f317359e700 1 --2- 172.21.15.31:0/2411568049 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3174153c30 0x7f317415f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:39.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.608+0000 7f317359e700 1 -- 172.21.15.31:0/2411568049 >> 172.21.15.31:0/2411568049 conn(0x7f31740703a0 msgr2=0x7f31740c5490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:39.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.608+0000 7f317359e700 1 -- 172.21.15.31:0/2411568049 shutdown_connections 2023-10-06T02:28:39.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.608+0000 7f317359e700 1 -- 172.21.15.31:0/2411568049 wait complete. 2023-10-06T02:28:39.610 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 71 2023-10-06T02:28:39.626 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:28:39.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.811+0000 7fcc2759e700 1 Processor -- start 2023-10-06T02:28:39.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.812+0000 7fcc2759e700 1 -- start start 2023-10-06T02:28:39.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.813+0000 7fcc2759e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcc2806d370 0x7fcc280c42f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:39.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.813+0000 7fcc2759e700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc280cd170 con 0x7fcc28140b70 2023-10-06T02:28:39.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.813+0000 7fcc2759e700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc280c9a20 con 0x7fcc28140f40 2023-10-06T02:28:39.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.813+0000 7fcc2759e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc280cd400 con 0x7fcc2806d370 2023-10-06T02:28:39.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.813+0000 7fcc2659c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fcc28140b70 0x7fcc280c7880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:43102/0 (socket says 172.21.15.31:43102) 2023-10-06T02:28:39.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.813+0000 7fcc2659c700 1 -- 172.21.15.31:0/746043608 learned_addr learned my addr 172.21.15.31:0/746043608 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:28:39.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.813+0000 7fcc25d9b700 1 --2- 172.21.15.31:0/746043608 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcc2806d370 0x7fcc280c42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:39.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.814+0000 7fcc2559a700 1 -- 172.21.15.31:0/746043608 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 310856113 0 0) 0x7fcc280cd170 con 0x7fcc28140b70 2023-10-06T02:28:39.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.814+0000 7fcc2559a700 1 -- 172.21.15.31:0/746043608 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc10003570 con 0x7fcc28140b70 2023-10-06T02:28:39.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.814+0000 7fcc2559a700 1 -- 172.21.15.31:0/746043608 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1501085123 0 0) 0x7fcc280c9a20 con 0x7fcc28140f40 2023-10-06T02:28:39.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.814+0000 7fcc2559a700 1 -- 172.21.15.31:0/746043608 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc280cd170 con 0x7fcc28140f40 2023-10-06T02:28:39.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.814+0000 7fcc25d9b700 1 -- 172.21.15.31:0/746043608 >> v1:172.21.15.62:6789/0 conn(0x7fcc28140f40 legacy=0x7fcc2815b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:39.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.814+0000 7fcc25d9b700 1 -- 172.21.15.31:0/746043608 >> v1:172.21.15.31:6789/0 conn(0x7fcc28140b70 legacy=0x7fcc280c7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:39.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.814+0000 7fcc25d9b700 1 -- 172.21.15.31:0/746043608 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc280cd6e0 con 0x7fcc2806d370 2023-10-06T02:28:39.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.814+0000 7fcc25d9b700 1 --2- 172.21.15.31:0/746043608 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcc2806d370 0x7fcc280c42f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c009910 tx=0x7fcc1c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=567dce7ce1090673 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:39.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.815+0000 7fcc2559a700 1 -- 172.21.15.31:0/746043608 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c011630 con 0x7fcc2806d370 2023-10-06T02:28:39.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.815+0000 7fcc2559a700 1 -- 172.21.15.31:0/746043608 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc1c011790 con 0x7fcc2806d370 2023-10-06T02:28:39.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.815+0000 7fcc2559a700 1 -- 172.21.15.31:0/746043608 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c011b30 con 0x7fcc2806d370 2023-10-06T02:28:39.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.815+0000 7fcc2759e700 1 -- 172.21.15.31:0/746043608 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcc2806d370 msgr2=0x7fcc280c42f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:39.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.815+0000 7fcc2759e700 1 --2- 172.21.15.31:0/746043608 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcc2806d370 0x7fcc280c42f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c009910 tx=0x7fcc1c0091a0 comp rx=0 tx=0).stop 2023-10-06T02:28:39.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.815+0000 7fcc2759e700 1 -- 172.21.15.31:0/746043608 shutdown_connections 2023-10-06T02:28:39.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.815+0000 7fcc2759e700 1 --2- 172.21.15.31:0/746043608 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcc2806d370 0x7fcc280c42f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:39.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.815+0000 7fcc2759e700 1 -- 172.21.15.31:0/746043608 >> 172.21.15.31:0/746043608 conn(0x7fcc280675a0 msgr2=0x7fcc280679a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:39.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.815+0000 7fcc2759e700 1 -- 172.21.15.31:0/746043608 shutdown_connections 2023-10-06T02:28:39.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.815+0000 7fcc2759e700 1 -- 172.21.15.31:0/746043608 wait complete. 2023-10-06T02:28:39.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.817+0000 7fcc2759e700 1 Processor -- start 2023-10-06T02:28:39.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.817+0000 7fcc2759e700 1 -- start start 2023-10-06T02:28:39.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.818+0000 7fcc2759e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc2806d370 0x7fcc280c4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:39.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.818+0000 7fcc2759e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc28140b70 0x7fcc2815b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:39.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.818+0000 7fcc2759e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcc28140f40 0x7fcc2815d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:39.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.818+0000 7fcc2759e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc280c6310 con 0x7fcc28140b70 2023-10-06T02:28:39.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.818+0000 7fcc2759e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc280c5550 con 0x7fcc2806d370 2023-10-06T02:28:39.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.818+0000 7fcc2759e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc280c5830 con 0x7fcc28140f40 2023-10-06T02:28:39.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.818+0000 7fcc25d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc28140b70 0x7fcc2815b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:39.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.818+0000 7fcc25d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc28140b70 0x7fcc2815b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36742/0 (socket says 172.21.15.31:36742) 2023-10-06T02:28:39.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.818+0000 7fcc25d9b700 1 -- 172.21.15.31:0/83278885 learned_addr learned my addr 172.21.15.31:0/83278885 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:39.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.818+0000 7fcc26d9d700 1 --2- 172.21.15.31:0/83278885 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcc28140f40 0x7fcc2815d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:39.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.818+0000 7fcc2659c700 1 --2- 172.21.15.31:0/83278885 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc2806d370 0x7fcc280c4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:39.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.818+0000 7fcc25d9b700 1 -- 172.21.15.31:0/83278885 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcc28140f40 msgr2=0x7fcc2815d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:39.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.818+0000 7fcc25d9b700 1 --2- 172.21.15.31:0/83278885 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcc28140f40 0x7fcc2815d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:39.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.818+0000 7fcc25d9b700 1 -- 172.21.15.31:0/83278885 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc2806d370 msgr2=0x7fcc280c4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:39.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.818+0000 7fcc25d9b700 1 --2- 172.21.15.31:0/83278885 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc2806d370 0x7fcc280c4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:39.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.818+0000 7fcc25d9b700 1 -- 172.21.15.31:0/83278885 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc1c009420 con 0x7fcc28140b70 2023-10-06T02:28:39.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.818+0000 7fcc2659c700 1 --2- 172.21.15.31:0/83278885 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc2806d370 0x7fcc280c4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:28:39.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.818+0000 7fcc26d9d700 1 --2- 172.21.15.31:0/83278885 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcc28140f40 0x7fcc2815d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:28:39.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.819+0000 7fcc25d9b700 1 --2- 172.21.15.31:0/83278885 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc28140b70 0x7fcc2815b360 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fcc2806f290 tx=0x7fcc1c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:39.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.819+0000 7fcc167fc700 1 -- 172.21.15.31:0/83278885 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c01f070 con 0x7fcc28140b70 2023-10-06T02:28:39.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.819+0000 7fcc167fc700 1 -- 172.21.15.31:0/83278885 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc1c000bc0 con 0x7fcc28140b70 2023-10-06T02:28:39.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.819+0000 7fcc167fc700 1 -- 172.21.15.31:0/83278885 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c011b30 con 0x7fcc28140b70 2023-10-06T02:28:39.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.819+0000 7fcc2759e700 1 -- 172.21.15.31:0/83278885 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc280c5b10 con 0x7fcc28140b70 2023-10-06T02:28:39.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.819+0000 7fcc2759e700 1 -- 172.21.15.31:0/83278885 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc2815b8a0 con 0x7fcc28140b70 2023-10-06T02:28:39.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.820+0000 7fcc2759e700 1 -- 172.21.15.31:0/83278885 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc280c6030 con 0x7fcc28140b70 2023-10-06T02:28:39.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.821+0000 7fcc167fc700 1 -- 172.21.15.31:0/83278885 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7fcc1c023070 con 0x7fcc28140b70 2023-10-06T02:28:39.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.822+0000 7fcc167fc700 1 --2- 172.21.15.31:0/83278885 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fcc08067050 0x7fcc08069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:39.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.822+0000 7fcc167fc700 1 -- 172.21.15.31:0/83278885 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fcc1c085ba0 con 0x7fcc28140b70 2023-10-06T02:28:39.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.822+0000 7fcc2659c700 1 --2- 172.21.15.31:0/83278885 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fcc08067050 0x7fcc08069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:39.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.823+0000 7fcc2659c700 1 --2- 172.21.15.31:0/83278885 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fcc08067050 0x7fcc08069500 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fcc18007fd0 tx=0x7fcc18009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:39.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:39.826+0000 7fcc167fc700 1 -- 172.21.15.31:0/83278885 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcc1c052b00 con 0x7fcc28140b70 2023-10-06T02:28:40.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.028+0000 7fcc2759e700 1 -- 172.21.15.31:0/83278885 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcc2804c3c0 con 0x7fcc28140b70 2023-10-06T02:28:40.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.029+0000 7fcc167fc700 1 -- 172.21.15.31:0/83278885 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) v1 ==== 74+0+16832 (secure 0 0 0) 0x7fcc1c015020 con 0x7fcc28140b70 2023-10-06T02:28:40.031 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:28:40.031 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":106,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:28:37.111821+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:28:33.803742+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-10-06T02:28:34.570331+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:28:40.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.031+0000 7fcc2759e700 1 -- 172.21.15.31:0/83278885 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fcc08067050 msgr2=0x7fcc08069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:40.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.031+0000 7fcc2759e700 1 --2- 172.21.15.31:0/83278885 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fcc08067050 0x7fcc08069500 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fcc18007fd0 tx=0x7fcc18009040 comp rx=0 tx=0).stop 2023-10-06T02:28:40.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.031+0000 7fcc2759e700 1 -- 172.21.15.31:0/83278885 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc28140b70 msgr2=0x7fcc2815b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:40.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.031+0000 7fcc2759e700 1 --2- 172.21.15.31:0/83278885 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc28140b70 0x7fcc2815b360 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fcc2806f290 tx=0x7fcc1c017000 comp rx=0 tx=0).stop 2023-10-06T02:28:40.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.031+0000 7fcc2759e700 1 -- 172.21.15.31:0/83278885 shutdown_connections 2023-10-06T02:28:40.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.031+0000 7fcc2759e700 1 --2- 172.21.15.31:0/83278885 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fcc08067050 0x7fcc08069500 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:40.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.031+0000 7fcc2759e700 1 --2- 172.21.15.31:0/83278885 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcc28140f40 0x7fcc2815d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:40.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.031+0000 7fcc2759e700 1 --2- 172.21.15.31:0/83278885 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc2806d370 0x7fcc280c4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:40.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.031+0000 7fcc2759e700 1 --2- 172.21.15.31:0/83278885 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc28140b70 0x7fcc2815b360 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:40.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.031+0000 7fcc2759e700 1 -- 172.21.15.31:0/83278885 >> 172.21.15.31:0/83278885 conn(0x7fcc280675a0 msgr2=0x7fcc280c2940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:40.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.031+0000 7fcc2759e700 1 -- 172.21.15.31:0/83278885 shutdown_connections 2023-10-06T02:28:40.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.031+0000 7fcc2759e700 1 -- 172.21.15.31:0/83278885 wait complete. 2023-10-06T02:28:40.050 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T02:28:40.051 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T02:28:40.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.238+0000 7fe83539d700 1 Processor -- start 2023-10-06T02:28:40.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.239+0000 7fe83539d700 1 -- start start 2023-10-06T02:28:40.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.240+0000 7fe83539d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe830142fd0 0x7fe8300c7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:40.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.240+0000 7fe83539d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe8300c7dc0 0x7fe83006c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:40.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.240+0000 7fe83539d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8300ca480 con 0x7fe830142c00 2023-10-06T02:28:40.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.240+0000 7fe83539d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8300c8240 con 0x7fe8300c7dc0 2023-10-06T02:28:40.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.240+0000 7fe83539d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe83015e240 con 0x7fe830142fd0 2023-10-06T02:28:40.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.240+0000 7fe82ffff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fe830142c00 0x7fe83006b7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:43106/0 (socket says 172.21.15.31:43106) 2023-10-06T02:28:40.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.240+0000 7fe82ffff700 1 -- 172.21.15.31:0/3309625662 learned_addr learned my addr 172.21.15.31:0/3309625662 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:28:40.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.241+0000 7fe82f7fe700 1 --2- 172.21.15.31:0/3309625662 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe830142fd0 0x7fe8300c7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:40.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.241+0000 7fe834b9c700 1 --2- 172.21.15.31:0/3309625662 >> v2:172.21.15.62:3300/0 conn(0x7fe8300c7dc0 0x7fe83006c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:40.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.241+0000 7fe82effd700 1 -- 172.21.15.31:0/3309625662 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3627772612 0 0) 0x7fe8300ca480 con 0x7fe830142c00 2023-10-06T02:28:40.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.241+0000 7fe82effd700 1 -- 172.21.15.31:0/3309625662 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe818003570 con 0x7fe830142c00 2023-10-06T02:28:40.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.241+0000 7fe82effd700 1 -- 172.21.15.31:0/3309625662 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1008082502 0 0) 0x7fe818003570 con 0x7fe830142c00 2023-10-06T02:28:40.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.241+0000 7fe82effd700 1 -- 172.21.15.31:0/3309625662 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe830142fd0 msgr2=0x7fe8300c7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:40.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.241+0000 7fe82effd700 1 --2- 172.21.15.31:0/3309625662 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe830142fd0 0x7fe8300c7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:40.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.241+0000 7fe82effd700 1 -- 172.21.15.31:0/3309625662 >> v2:172.21.15.62:3300/0 conn(0x7fe8300c7dc0 msgr2=0x7fe83006c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:40.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.241+0000 7fe82effd700 1 --2- 172.21.15.31:0/3309625662 >> v2:172.21.15.62:3300/0 conn(0x7fe8300c7dc0 0x7fe83006c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:40.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.241+0000 7fe82effd700 1 -- 172.21.15.31:0/3309625662 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe83006b350 con 0x7fe830142c00 2023-10-06T02:28:40.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.241+0000 7fe82effd700 1 -- 172.21.15.31:0/3309625662 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7fe820001470 con 0x7fe830142c00 2023-10-06T02:28:40.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.241+0000 7fe82effd700 1 -- 172.21.15.31:0/3309625662 >> v1:172.21.15.31:6789/0 conn(0x7fe830142c00 legacy=0x7fe83006b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:40.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.241+0000 7fe82effd700 1 --2- 172.21.15.31:0/3309625662 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe830142fd0 0x7fe818003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:40.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.241+0000 7fe82effd700 1 --2- 172.21.15.31:0/3309625662 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe818003fe0 0x7fe818004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:40.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.241+0000 7fe82effd700 1 --2- 172.21.15.31:0/3309625662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe818004d50 0x7fe8180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:40.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.241+0000 7fe82effd700 1 -- 172.21.15.31:0/3309625662 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8300c8240 con 0x7fe818003fe0 2023-10-06T02:28:40.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.241+0000 7fe82effd700 1 -- 172.21.15.31:0/3309625662 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe83015e240 con 0x7fe818004d50 2023-10-06T02:28:40.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.241+0000 7fe82effd700 1 -- 172.21.15.31:0/3309625662 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe81800ba90 con 0x7fe830142fd0 2023-10-06T02:28:40.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.241+0000 7fe82f7fe700 1 --2- 172.21.15.31:0/3309625662 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe818003fe0 0x7fe818004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:40.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.242+0000 7fe82ffff700 1 --2- 172.21.15.31:0/3309625662 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe830142fd0 0x7fe818003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:40.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.242+0000 7fe834b9c700 1 --2- 172.21.15.31:0/3309625662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe818004d50 0x7fe8180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:40.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.242+0000 7fe82f7fe700 1 -- 172.21.15.31:0/3309625662 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe830142fd0 msgr2=0x7fe818003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:40.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.242+0000 7fe82f7fe700 1 --2- 172.21.15.31:0/3309625662 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe830142fd0 0x7fe818003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:40.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.242+0000 7fe82f7fe700 1 -- 172.21.15.31:0/3309625662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe818004d50 msgr2=0x7fe8180051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:40.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.242+0000 7fe82f7fe700 1 --2- 172.21.15.31:0/3309625662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe818004d50 0x7fe8180051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:40.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.242+0000 7fe82f7fe700 1 -- 172.21.15.31:0/3309625662 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe818003570 con 0x7fe818003fe0 2023-10-06T02:28:40.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.242+0000 7fe834b9c700 1 --2- 172.21.15.31:0/3309625662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe818004d50 0x7fe8180051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:28:40.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.242+0000 7fe82ffff700 1 --2- 172.21.15.31:0/3309625662 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe830142fd0 0x7fe818003a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:28:40.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.242+0000 7fe82f7fe700 1 --2- 172.21.15.31:0/3309625662 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe818003fe0 0x7fe818004810 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fe8300ce1b0 tx=0x7fe8240091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=379c92d9b268bd5c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:40.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.242+0000 7fe82effd700 1 -- 172.21.15.31:0/3309625662 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe82400e6b0 con 0x7fe818003fe0 2023-10-06T02:28:40.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.242+0000 7fe82effd700 1 -- 172.21.15.31:0/3309625662 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe82400e810 con 0x7fe818003fe0 2023-10-06T02:28:40.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.242+0000 7fe82effd700 1 -- 172.21.15.31:0/3309625662 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe82400ebd0 con 0x7fe818003fe0 2023-10-06T02:28:40.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.243+0000 7fe83539d700 1 -- 172.21.15.31:0/3309625662 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe818003fe0 msgr2=0x7fe818004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:40.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.243+0000 7fe83539d700 1 --2- 172.21.15.31:0/3309625662 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe818003fe0 0x7fe818004810 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fe8300ce1b0 tx=0x7fe8240091a0 comp rx=0 tx=0).stop 2023-10-06T02:28:40.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.243+0000 7fe834b9c700 1 -- 172.21.15.31:0/3309625662 reap_dead start 2023-10-06T02:28:40.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.243+0000 7fe83539d700 1 -- 172.21.15.31:0/3309625662 shutdown_connections 2023-10-06T02:28:40.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.243+0000 7fe83539d700 1 -- 172.21.15.31:0/3309625662 >> 172.21.15.31:0/3309625662 conn(0x7fe8300675a0 msgr2=0x7fe8300679a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:40.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.243+0000 7fe83539d700 1 -- 172.21.15.31:0/3309625662 shutdown_connections 2023-10-06T02:28:40.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.243+0000 7fe83539d700 1 -- 172.21.15.31:0/3309625662 wait complete. 2023-10-06T02:28:40.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.244+0000 7fe83539d700 1 Processor -- start 2023-10-06T02:28:40.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.245+0000 7fe83539d700 1 -- start start 2023-10-06T02:28:40.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.245+0000 7fe83539d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe818003fe0 0x7fe8300c3170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:40.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.245+0000 7fe83539d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe83013e440 0x7fe830158bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:40.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.246+0000 7fe83539d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe83013e8b0 0x7fe830159130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:40.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.246+0000 7fe83539d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe83013fa30 con 0x7fe83013e440 2023-10-06T02:28:40.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.246+0000 7fe83539d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe83013ee20 con 0x7fe818003fe0 2023-10-06T02:28:40.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.246+0000 7fe83539d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe83013f100 con 0x7fe83013e8b0 2023-10-06T02:28:40.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.246+0000 7fe82f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe83013e440 0x7fe830158bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:40.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.246+0000 7fe82f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe83013e440 0x7fe830158bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36762/0 (socket says 172.21.15.31:36762) 2023-10-06T02:28:40.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.246+0000 7fe82f7fe700 1 -- 172.21.15.31:0/3416163268 learned_addr learned my addr 172.21.15.31:0/3416163268 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:40.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.246+0000 7fe834b9c700 1 --2- 172.21.15.31:0/3416163268 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe83013e8b0 0x7fe830159130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:40.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.246+0000 7fe82ffff700 1 --2- 172.21.15.31:0/3416163268 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe818003fe0 0x7fe8300c3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:40.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.246+0000 7fe82f7fe700 1 -- 172.21.15.31:0/3416163268 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe83013e8b0 msgr2=0x7fe830159130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:40.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.246+0000 7fe82f7fe700 1 --2- 172.21.15.31:0/3416163268 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe83013e8b0 0x7fe830159130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:40.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.246+0000 7fe82f7fe700 1 -- 172.21.15.31:0/3416163268 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe818003fe0 msgr2=0x7fe8300c3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:40.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.246+0000 7fe82f7fe700 1 --2- 172.21.15.31:0/3416163268 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe818003fe0 0x7fe8300c3170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:40.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.246+0000 7fe82f7fe700 1 -- 172.21.15.31:0/3416163268 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe83013f3e0 con 0x7fe83013e440 2023-10-06T02:28:40.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.246+0000 7fe82ffff700 1 --2- 172.21.15.31:0/3416163268 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe818003fe0 0x7fe8300c3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:28:40.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.246+0000 7fe834b9c700 1 --2- 172.21.15.31:0/3416163268 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe83013e8b0 0x7fe830159130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:28:40.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.247+0000 7fe82f7fe700 1 --2- 172.21.15.31:0/3416163268 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe83013e440 0x7fe830158bf0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fe818004730 tx=0x7fe82401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:40.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.247+0000 7fe817fff700 1 -- 172.21.15.31:0/3416163268 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe824023070 con 0x7fe83013e440 2023-10-06T02:28:40.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.247+0000 7fe817fff700 1 -- 172.21.15.31:0/3416163268 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe82400e980 con 0x7fe83013e440 2023-10-06T02:28:40.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.247+0000 7fe817fff700 1 -- 172.21.15.31:0/3416163268 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe82400e6b0 con 0x7fe83013e440 2023-10-06T02:28:40.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.247+0000 7fe83539d700 1 -- 172.21.15.31:0/3416163268 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe83013f6c0 con 0x7fe83013e440 2023-10-06T02:28:40.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.247+0000 7fe83539d700 1 -- 172.21.15.31:0/3416163268 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8300c36b0 con 0x7fe83013e440 2023-10-06T02:28:40.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.248+0000 7fe83539d700 1 -- 172.21.15.31:0/3416163268 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe83004c3c0 con 0x7fe83013e440 2023-10-06T02:28:40.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.249+0000 7fe817fff700 1 -- 172.21.15.31:0/3416163268 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7fe824013470 con 0x7fe83013e440 2023-10-06T02:28:40.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.250+0000 7fe817fff700 1 --2- 172.21.15.31:0/3416163268 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe81c067050 0x7fe81c069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:40.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.250+0000 7fe817fff700 1 -- 172.21.15.31:0/3416163268 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe824085e50 con 0x7fe83013e440 2023-10-06T02:28:40.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.250+0000 7fe82ffff700 1 --2- 172.21.15.31:0/3416163268 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe81c067050 0x7fe81c069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:40.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.251+0000 7fe82ffff700 1 --2- 172.21.15.31:0/3416163268 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe81c067050 0x7fe81c069500 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe8200016e0 tx=0x7fe83006b2c0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:40.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.254+0000 7fe817fff700 1 -- 172.21.15.31:0/3416163268 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe824052db0 con 0x7fe83013e440 2023-10-06T02:28:40.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.454+0000 7fe83539d700 1 -- 172.21.15.31:0/3416163268 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe83013f270 con 0x7fe81c067050 2023-10-06T02:28:40.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.465+0000 7fe817fff700 1 -- 172.21.15.31:0/3416163268 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+300149 (secure 0 0 0) 0x7fe83013f270 con 0x7fe81c067050 2023-10-06T02:28:40.496 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:28:40.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.470+0000 7fe83539d700 1 -- 172.21.15.31:0/3416163268 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe81c067050 msgr2=0x7fe81c069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:40.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.470+0000 7fe83539d700 1 --2- 172.21.15.31:0/3416163268 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe81c067050 0x7fe81c069500 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe8200016e0 tx=0x7fe83006b2c0 comp rx=0 tx=0).stop 2023-10-06T02:28:40.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.470+0000 7fe83539d700 1 -- 172.21.15.31:0/3416163268 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe83013e440 msgr2=0x7fe830158bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:40.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.470+0000 7fe83539d700 1 --2- 172.21.15.31:0/3416163268 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe83013e440 0x7fe830158bf0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fe818004730 tx=0x7fe82401a000 comp rx=0 tx=0).stop 2023-10-06T02:28:40.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.470+0000 7fe83539d700 1 -- 172.21.15.31:0/3416163268 shutdown_connections 2023-10-06T02:28:40.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.470+0000 7fe83539d700 1 --2- 172.21.15.31:0/3416163268 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe81c067050 0x7fe81c069500 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:40.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.470+0000 7fe83539d700 1 --2- 172.21.15.31:0/3416163268 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe83013e8b0 0x7fe830159130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:40.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.470+0000 7fe83539d700 1 --2- 172.21.15.31:0/3416163268 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe818003fe0 0x7fe8300c3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:40.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.470+0000 7fe83539d700 1 --2- 172.21.15.31:0/3416163268 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe83013e440 0x7fe830158bf0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:40.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.470+0000 7fe83539d700 1 -- 172.21.15.31:0/3416163268 >> 172.21.15.31:0/3416163268 conn(0x7fe8300675a0 msgr2=0x7fe830140970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:40.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.470+0000 7fe83539d700 1 -- 172.21.15.31:0/3416163268 shutdown_connections 2023-10-06T02:28:40.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.470+0000 7fe83539d700 1 -- 172.21.15.31:0/3416163268 wait complete. 2023-10-06T02:28:40.499 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-10-06T02:28:40.500 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":64,"stamp":"2023-10-06T02:28:38.456123+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":21,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2422188,"kb_used_data":11116,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752552532,"statfs":{"total":773094113280,"available":770613792768,"internally_reserved":0,"allocated":11382784,"data_stored":5869266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.360085"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:28:35.092173+0000","last_change":"2023-10-06T02:28:35.092173+0000","last_active":"2023-10-06T02:28:35.092173+0000","last_peered":"2023-10-06T02:28:35.092173+0000","last_clean":"2023-10-06T02:28:35.092173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:35.092173+0000","last_undegraded":"2023-10-06T02:28:35.092173+0000","last_fullsized":"2023-10-06T02:28:35.092173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:28:35.092173+0000","last_change":"2023-10-06T02:28:35.092173+0000","last_active":"2023-10-06T02:28:35.092173+0000","last_peered":"2023-10-06T02:28:35.092173+0000","last_clean":"2023-10-06T02:28:35.092173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:35.092173+0000","last_undegraded":"2023-10-06T02:28:35.092173+0000","last_fullsized":"2023-10-06T02:28:35.092173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:28:35.092173+0000","last_change":"2023-10-06T02:28:35.092173+0000","last_active":"2023-10-06T02:28:35.092173+0000","last_peered":"2023-10-06T02:28:35.092173+0000","last_clean":"2023-10-06T02:28:35.092173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:35.092173+0000","last_undegraded":"2023-10-06T02:28:35.092173+0000","last_fullsized":"2023-10-06T02:28:35.092173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:28:35.092173+0000","last_change":"2023-10-06T02:28:35.092173+0000","last_active":"2023-10-06T02:28:35.092173+0000","last_peered":"2023-10-06T02:28:35.092173+0000","last_clean":"2023-10-06T02:28:35.092173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:35.092173+0000","last_undegraded":"2023-10-06T02:28:35.092173+0000","last_fullsized":"2023-10-06T02:28:35.092173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:28:35.092173+0000","last_change":"2023-10-06T02:28:35.092173+0000","last_active":"2023-10-06T02:28:35.092173+0000","last_peered":"2023-10-06T02:28:35.092173+0000","last_clean":"2023-10-06T02:28:35.092173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:35.092173+0000","last_undegraded":"2023-10-06T02:28:35.092173+0000","last_fullsized":"2023-10-06T02:28:35.092173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:28:35.092173+0000","last_change":"2023-10-06T02:28:35.092173+0000","last_active":"2023-10-06T02:28:35.092173+0000","last_peered":"2023-10-06T02:28:35.092173+0000","last_clean":"2023-10-06T02:28:35.092173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:35.092173+0000","last_undegraded":"2023-10-06T02:28:35.092173+0000","last_fullsized":"2023-10-06T02:28:35.092173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:28:35.092173+0000","last_change":"2023-10-06T02:28:35.092173+0000","last_active":"2023-10-06T02:28:35.092173+0000","last_peered":"2023-10-06T02:28:35.092173+0000","last_clean":"2023-10-06T02:28:35.092173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:35.092173+0000","last_undegraded":"2023-10-06T02:28:35.092173+0000","last_fullsized":"2023-10-06T02:28:35.092173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:28:35.092173+0000","last_change":"2023-10-06T02:28:35.092173+0000","last_active":"2023-10-06T02:28:35.092173+0000","last_peered":"2023-10-06T02:28:35.092173+0000","last_clean":"2023-10-06T02:28:35.092173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:35.092173+0000","last_undegraded":"2023-10-06T02:28:35.092173+0000","last_fullsized":"2023-10-06T02:28:35.092173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:28:35.092173+0000","last_change":"2023-10-06T02:28:35.092173+0000","last_active":"2023-10-06T02:28:35.092173+0000","last_peered":"2023-10-06T02:28:35.092173+0000","last_clean":"2023-10-06T02:28:35.092173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:35.092173+0000","last_undegraded":"2023-10-06T02:28:35.092173+0000","last_fullsized":"2023-10-06T02:28:35.092173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:28:35.092173+0000","last_change":"2023-10-06T02:28:35.092173+0000","last_active":"2023-10-06T02:28:35.092173+0000","last_peered":"2023-10-06T02:28:35.092173+0000","last_clean":"2023-10-06T02:28:35.092173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:35.092173+0000","last_undegraded":"2023-10-06T02:28:35.092173+0000","last_fullsized":"2023-10-06T02:28:35.092173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:28:35.092173+0000","last_change":"2023-10-06T02:28:35.092173+0000","last_active":"2023-10-06T02:28:35.092173+0000","last_peered":"2023-10-06T02:28:35.092173+0000","last_clean":"2023-10-06T02:28:35.092173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:35.092173+0000","last_undegraded":"2023-10-06T02:28:35.092173+0000","last_fullsized":"2023-10-06T02:28:35.092173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:28:35.092173+0000","last_change":"2023-10-06T02:28:35.092173+0000","last_active":"2023-10-06T02:28:35.092173+0000","last_peered":"2023-10-06T02:28:35.092173+0000","last_clean":"2023-10-06T02:28:35.092173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:35.092173+0000","last_undegraded":"2023-10-06T02:28:35.092173+0000","last_fullsized":"2023-10-06T02:28:35.092173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:28:35.092173+0000","last_change":"2023-10-06T02:28:35.092173+0000","last_active":"2023-10-06T02:28:35.092173+0000","last_peered":"2023-10-06T02:28:35.092173+0000","last_clean":"2023-10-06T02:28:35.092173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:35.092173+0000","last_undegraded":"2023-10-06T02:28:35.092173+0000","last_fullsized":"2023-10-06T02:28:35.092173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:28:35.092173+0000","last_change":"2023-10-06T02:28:35.092173+0000","last_active":"2023-10-06T02:28:35.092173+0000","last_peered":"2023-10-06T02:28:35.092173+0000","last_clean":"2023-10-06T02:28:35.092173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:35.092173+0000","last_undegraded":"2023-10-06T02:28:35.092173+0000","last_fullsized":"2023-10-06T02:28:35.092173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:28:35.092173+0000","last_change":"2023-10-06T02:28:35.092173+0000","last_active":"2023-10-06T02:28:35.092173+0000","last_peered":"2023-10-06T02:28:35.092173+0000","last_clean":"2023-10-06T02:28:35.092173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:35.092173+0000","last_undegraded":"2023-10-06T02:28:35.092173+0000","last_fullsized":"2023-10-06T02:28:35.092173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:28:35.092173+0000","last_change":"2023-10-06T02:28:35.092173+0000","last_active":"2023-10-06T02:28:35.092173+0000","last_peered":"2023-10-06T02:28:35.092173+0000","last_clean":"2023-10-06T02:28:35.092173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:35.092173+0000","last_undegraded":"2023-10-06T02:28:35.092173+0000","last_fullsized":"2023-10-06T02:28:35.092173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:28:35.092173+0000","last_change":"2023-10-06T02:28:35.092173+0000","last_active":"2023-10-06T02:28:35.092173+0000","last_peered":"2023-10-06T02:28:35.092173+0000","last_clean":"2023-10-06T02:28:35.092173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:35.092173+0000","last_undegraded":"2023-10-06T02:28:35.092173+0000","last_fullsized":"2023-10-06T02:28:35.092173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:28:35.092173+0000","last_change":"2023-10-06T02:28:35.092173+0000","last_active":"2023-10-06T02:28:35.092173+0000","last_peered":"2023-10-06T02:28:35.092173+0000","last_clean":"2023-10-06T02:28:35.092173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:35.092173+0000","last_undegraded":"2023-10-06T02:28:35.092173+0000","last_fullsized":"2023-10-06T02:28:35.092173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:28:35.092173+0000","last_change":"2023-10-06T02:28:35.092173+0000","last_active":"2023-10-06T02:28:35.092173+0000","last_peered":"2023-10-06T02:28:35.092173+0000","last_clean":"2023-10-06T02:28:35.092173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:35.092173+0000","last_undegraded":"2023-10-06T02:28:35.092173+0000","last_fullsized":"2023-10-06T02:28:35.092173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:28:35.092173+0000","last_change":"2023-10-06T02:28:35.092173+0000","last_active":"2023-10-06T02:28:35.092173+0000","last_peered":"2023-10-06T02:28:35.092173+0000","last_clean":"2023-10-06T02:28:35.092173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:35.092173+0000","last_undegraded":"2023-10-06T02:28:35.092173+0000","last_fullsized":"2023-10-06T02:28:35.092173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:28:35.092173+0000","last_change":"2023-10-06T02:28:35.092173+0000","last_active":"2023-10-06T02:28:35.092173+0000","last_peered":"2023-10-06T02:28:35.092173+0000","last_clean":"2023-10-06T02:28:35.092173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:35.092173+0000","last_undegraded":"2023-10-06T02:28:35.092173+0000","last_fullsized":"2023-10-06T02:28:35.092173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:28:35.092173+0000","last_change":"2023-10-06T02:28:35.092173+0000","last_active":"2023-10-06T02:28:35.092173+0000","last_peered":"2023-10-06T02:28:35.092173+0000","last_clean":"2023-10-06T02:28:35.092173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:35.092173+0000","last_undegraded":"2023-10-06T02:28:35.092173+0000","last_fullsized":"2023-10-06T02:28:35.092173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:28:35.092173+0000","last_change":"2023-10-06T02:28:35.092173+0000","last_active":"2023-10-06T02:28:35.092173+0000","last_peered":"2023-10-06T02:28:35.092173+0000","last_clean":"2023-10-06T02:28:35.092173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:35.092173+0000","last_undegraded":"2023-10-06T02:28:35.092173+0000","last_fullsized":"2023-10-06T02:28:35.092173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:28:35.092173+0000","last_change":"2023-10-06T02:28:35.092173+0000","last_active":"2023-10-06T02:28:35.092173+0000","last_peered":"2023-10-06T02:28:35.092173+0000","last_clean":"2023-10-06T02:28:35.092173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:35.092173+0000","last_undegraded":"2023-10-06T02:28:35.092173+0000","last_fullsized":"2023-10-06T02:28:35.092173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:28:35.092173+0000","last_change":"2023-10-06T02:28:35.092173+0000","last_active":"2023-10-06T02:28:35.092173+0000","last_peered":"2023-10-06T02:28:35.092173+0000","last_clean":"2023-10-06T02:28:35.092173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:35.092173+0000","last_undegraded":"2023-10-06T02:28:35.092173+0000","last_fullsized":"2023-10-06T02:28:35.092173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:28:35.092173+0000","last_change":"2023-10-06T02:28:35.092173+0000","last_active":"2023-10-06T02:28:35.092173+0000","last_peered":"2023-10-06T02:28:35.092173+0000","last_clean":"2023-10-06T02:28:35.092173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:35.092173+0000","last_undegraded":"2023-10-06T02:28:35.092173+0000","last_fullsized":"2023-10-06T02:28:35.092173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:28:35.092173+0000","last_change":"2023-10-06T02:28:35.092173+0000","last_active":"2023-10-06T02:28:35.092173+0000","last_peered":"2023-10-06T02:28:35.092173+0000","last_clean":"2023-10-06T02:28:35.092173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:35.092173+0000","last_undegraded":"2023-10-06T02:28:35.092173+0000","last_fullsized":"2023-10-06T02:28:35.092173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:28:35.092173+0000","last_change":"2023-10-06T02:28:35.092173+0000","last_active":"2023-10-06T02:28:35.092173+0000","last_peered":"2023-10-06T02:28:35.092173+0000","last_clean":"2023-10-06T02:28:35.092173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:35.092173+0000","last_undegraded":"2023-10-06T02:28:35.092173+0000","last_fullsized":"2023-10-06T02:28:35.092173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:28:35.092173+0000","last_change":"2023-10-06T02:28:35.092173+0000","last_active":"2023-10-06T02:28:35.092173+0000","last_peered":"2023-10-06T02:28:35.092173+0000","last_clean":"2023-10-06T02:28:35.092173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:35.092173+0000","last_undegraded":"2023-10-06T02:28:35.092173+0000","last_fullsized":"2023-10-06T02:28:35.092173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:28:35.092173+0000","last_change":"2023-10-06T02:28:35.092173+0000","last_active":"2023-10-06T02:28:35.092173+0000","last_peered":"2023-10-06T02:28:35.092173+0000","last_clean":"2023-10-06T02:28:35.092173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:35.092173+0000","last_undegraded":"2023-10-06T02:28:35.092173+0000","last_fullsized":"2023-10-06T02:28:35.092173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:28:35.092173+0000","last_change":"2023-10-06T02:28:35.092173+0000","last_active":"2023-10-06T02:28:35.092173+0000","last_peered":"2023-10-06T02:28:35.092173+0000","last_clean":"2023-10-06T02:28:35.092173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:35.092173+0000","last_undegraded":"2023-10-06T02:28:35.092173+0000","last_fullsized":"2023-10-06T02:28:35.092173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:28:35.092173+0000","last_change":"2023-10-06T02:28:35.092173+0000","last_active":"2023-10-06T02:28:35.092173+0000","last_peered":"2023-10-06T02:28:35.092173+0000","last_clean":"2023-10-06T02:28:35.092173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:35.092173+0000","last_undegraded":"2023-10-06T02:28:35.092173+0000","last_fullsized":"2023-10-06T02:28:35.092173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:28:35.092173+0000","last_change":"2023-10-06T02:28:35.092173+0000","last_active":"2023-10-06T02:28:35.092173+0000","last_peered":"2023-10-06T02:28:35.092173+0000","last_clean":"2023-10-06T02:28:35.092173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:35.092173+0000","last_undegraded":"2023-10-06T02:28:35.092173+0000","last_fullsized":"2023-10-06T02:28:35.092173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:28:34.085717+0000","last_change":"2023-10-06T02:28:34.104135+0000","last_active":"2023-10-06T02:28:34.085717+0000","last_peered":"2023-10-06T02:28:34.085717+0000","last_clean":"2023-10-06T02:28:34.085717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:34.085717+0000","last_undegraded":"2023-10-06T02:28:34.085717+0000","last_fullsized":"2023-10-06T02:28:34.085717+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_clean_scrub_stamp":"2023-10-06T02:28:34.085717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:25:29.830195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:28:35.092173+0000","last_change":"2023-10-06T02:28:35.092173+0000","last_active":"2023-10-06T02:28:35.092173+0000","last_peered":"2023-10-06T02:28:35.092173+0000","last_clean":"2023-10-06T02:28:35.092173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:35.092173+0000","last_undegraded":"2023-10-06T02:28:35.092173+0000","last_fullsized":"2023-10-06T02:28:35.092173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:28:34.085717+0000","last_change":"2023-10-06T02:28:34.098013+0000","last_active":"2023-10-06T02:28:34.085717+0000","last_peered":"2023-10-06T02:28:34.085717+0000","last_clean":"2023-10-06T02:28:34.085717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:34.085717+0000","last_undegraded":"2023-10-06T02:28:34.085717+0000","last_fullsized":"2023-10-06T02:28:34.085717+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_clean_scrub_stamp":"2023-10-06T02:28:34.085717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:35:05.456705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:28:35.092173+0000","last_change":"2023-10-06T02:28:35.092173+0000","last_active":"2023-10-06T02:28:35.092173+0000","last_peered":"2023-10-06T02:28:35.092173+0000","last_clean":"2023-10-06T02:28:35.092173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:35.092173+0000","last_undegraded":"2023-10-06T02:28:35.092173+0000","last_fullsized":"2023-10-06T02:28:35.092173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:28:34.085717+0000","last_change":"2023-10-06T02:28:34.101053+0000","last_active":"2023-10-06T02:28:34.085717+0000","last_peered":"2023-10-06T02:28:34.085717+0000","last_clean":"2023-10-06T02:28:34.085717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:34.085717+0000","last_undegraded":"2023-10-06T02:28:34.085717+0000","last_fullsized":"2023-10-06T02:28:34.085717+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_clean_scrub_stamp":"2023-10-06T02:28:34.085717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:49:29.193059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:28:35.092173+0000","last_change":"2023-10-06T02:28:35.092173+0000","last_active":"2023-10-06T02:28:35.092173+0000","last_peered":"2023-10-06T02:28:35.092173+0000","last_clean":"2023-10-06T02:28:35.092173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:35.092173+0000","last_undegraded":"2023-10-06T02:28:35.092173+0000","last_fullsized":"2023-10-06T02:28:35.092173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:28:34.085717+0000","last_change":"2023-10-06T02:28:34.097972+0000","last_active":"2023-10-06T02:28:34.085717+0000","last_peered":"2023-10-06T02:28:34.085717+0000","last_clean":"2023-10-06T02:28:34.085717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:34.085717+0000","last_undegraded":"2023-10-06T02:28:34.085717+0000","last_fullsized":"2023-10-06T02:28:34.085717+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_clean_scrub_stamp":"2023-10-06T02:28:34.085717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:19:58.581969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:28:35.092173+0000","last_change":"2023-10-06T02:28:35.092173+0000","last_active":"2023-10-06T02:28:35.092173+0000","last_peered":"2023-10-06T02:28:35.092173+0000","last_clean":"2023-10-06T02:28:35.092173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:35.092173+0000","last_undegraded":"2023-10-06T02:28:35.092173+0000","last_fullsized":"2023-10-06T02:28:35.092173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:28:34.085717+0000","last_change":"2023-10-06T02:28:34.100551+0000","last_active":"2023-10-06T02:28:34.085717+0000","last_peered":"2023-10-06T02:28:34.085717+0000","last_clean":"2023-10-06T02:28:34.085717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:34.085717+0000","last_undegraded":"2023-10-06T02:28:34.085717+0000","last_fullsized":"2023-10-06T02:28:34.085717+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_clean_scrub_stamp":"2023-10-06T02:28:34.085717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:12:29.493807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:28:35.092173+0000","last_change":"2023-10-06T02:28:35.092173+0000","last_active":"2023-10-06T02:28:35.092173+0000","last_peered":"2023-10-06T02:28:35.092173+0000","last_clean":"2023-10-06T02:28:35.092173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:35.092173+0000","last_undegraded":"2023-10-06T02:28:35.092173+0000","last_fullsized":"2023-10-06T02:28:35.092173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:28:34.085717+0000","last_change":"2023-10-06T02:28:34.095458+0000","last_active":"2023-10-06T02:28:34.085717+0000","last_peered":"2023-10-06T02:28:34.085717+0000","last_clean":"2023-10-06T02:28:34.085717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:34.085717+0000","last_undegraded":"2023-10-06T02:28:34.085717+0000","last_fullsized":"2023-10-06T02:28:34.085717+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_clean_scrub_stamp":"2023-10-06T02:28:34.085717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:27:56.209255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:28:35.092173+0000","last_change":"2023-10-06T02:28:35.092173+0000","last_active":"2023-10-06T02:28:35.092173+0000","last_peered":"2023-10-06T02:28:35.092173+0000","last_clean":"2023-10-06T02:28:35.092173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:35.092173+0000","last_undegraded":"2023-10-06T02:28:35.092173+0000","last_fullsized":"2023-10-06T02:28:35.092173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:28:34.085717+0000","last_change":"2023-10-06T02:28:34.098716+0000","last_active":"2023-10-06T02:28:34.085717+0000","last_peered":"2023-10-06T02:28:34.085717+0000","last_clean":"2023-10-06T02:28:34.085717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:34.085717+0000","last_undegraded":"2023-10-06T02:28:34.085717+0000","last_fullsized":"2023-10-06T02:28:34.085717+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_clean_scrub_stamp":"2023-10-06T02:28:34.085717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:17:59.529005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:28:35.092173+0000","last_change":"2023-10-06T02:28:35.092173+0000","last_active":"2023-10-06T02:28:35.092173+0000","last_peered":"2023-10-06T02:28:35.092173+0000","last_clean":"2023-10-06T02:28:35.092173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:35.092173+0000","last_undegraded":"2023-10-06T02:28:35.092173+0000","last_fullsized":"2023-10-06T02:28:35.092173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:28:34.085717+0000","last_change":"2023-10-06T02:28:34.095147+0000","last_active":"2023-10-06T02:28:34.085717+0000","last_peered":"2023-10-06T02:28:34.085717+0000","last_clean":"2023-10-06T02:28:34.085717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:34.085717+0000","last_undegraded":"2023-10-06T02:28:34.085717+0000","last_fullsized":"2023-10-06T02:28:34.085717+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_clean_scrub_stamp":"2023-10-06T02:28:34.085717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:55:57.723091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:28:35.092173+0000","last_change":"2023-10-06T02:28:35.092173+0000","last_active":"2023-10-06T02:28:35.092173+0000","last_peered":"2023-10-06T02:28:35.092173+0000","last_clean":"2023-10-06T02:28:35.092173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:35.092173+0000","last_undegraded":"2023-10-06T02:28:35.092173+0000","last_fullsized":"2023-10-06T02:28:35.092173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:28:34.085717+0000","last_change":"2023-10-06T02:28:34.097294+0000","last_active":"2023-10-06T02:28:34.085717+0000","last_peered":"2023-10-06T02:28:34.085717+0000","last_clean":"2023-10-06T02:28:34.085717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:34.085717+0000","last_undegraded":"2023-10-06T02:28:34.085717+0000","last_fullsized":"2023-10-06T02:28:34.085717+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_clean_scrub_stamp":"2023-10-06T02:28:34.085717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:31.463520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:28:35.092173+0000","last_change":"2023-10-06T02:28:35.092173+0000","last_active":"2023-10-06T02:28:35.092173+0000","last_peered":"2023-10-06T02:28:35.092173+0000","last_clean":"2023-10-06T02:28:35.092173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:35.092173+0000","last_undegraded":"2023-10-06T02:28:35.092173+0000","last_fullsized":"2023-10-06T02:28:35.092173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:28:34.085717+0000","last_change":"2023-10-06T02:28:34.104450+0000","last_active":"2023-10-06T02:28:34.085717+0000","last_peered":"2023-10-06T02:28:34.085717+0000","last_clean":"2023-10-06T02:28:34.085717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:34.085717+0000","last_undegraded":"2023-10-06T02:28:34.085717+0000","last_fullsized":"2023-10-06T02:28:34.085717+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_clean_scrub_stamp":"2023-10-06T02:28:34.085717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:50:02.088679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:28:35.092173+0000","last_change":"2023-10-06T02:28:35.092173+0000","last_active":"2023-10-06T02:28:35.092173+0000","last_peered":"2023-10-06T02:28:35.092173+0000","last_clean":"2023-10-06T02:28:35.092173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:35.092173+0000","last_undegraded":"2023-10-06T02:28:35.092173+0000","last_fullsized":"2023-10-06T02:28:35.092173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:28:34.085717+0000","last_change":"2023-10-06T02:28:34.100446+0000","last_active":"2023-10-06T02:28:34.085717+0000","last_peered":"2023-10-06T02:28:34.085717+0000","last_clean":"2023-10-06T02:28:34.085717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:34.085717+0000","last_undegraded":"2023-10-06T02:28:34.085717+0000","last_fullsized":"2023-10-06T02:28:34.085717+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_clean_scrub_stamp":"2023-10-06T02:28:34.085717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:42:56.526314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:28:35.092173+0000","last_change":"2023-10-06T02:28:35.092173+0000","last_active":"2023-10-06T02:28:35.092173+0000","last_peered":"2023-10-06T02:28:35.092173+0000","last_clean":"2023-10-06T02:28:35.092173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:35.092173+0000","last_undegraded":"2023-10-06T02:28:35.092173+0000","last_fullsized":"2023-10-06T02:28:35.092173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:28:34.085717+0000","last_change":"2023-10-06T02:28:34.101228+0000","last_active":"2023-10-06T02:28:34.085717+0000","last_peered":"2023-10-06T02:28:34.085717+0000","last_clean":"2023-10-06T02:28:34.085717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:34.085717+0000","last_undegraded":"2023-10-06T02:28:34.085717+0000","last_fullsized":"2023-10-06T02:28:34.085717+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_clean_scrub_stamp":"2023-10-06T02:28:34.085717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:35:26.925918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:28:35.092173+0000","last_change":"2023-10-06T02:28:35.092173+0000","last_active":"2023-10-06T02:28:35.092173+0000","last_peered":"2023-10-06T02:28:35.092173+0000","last_clean":"2023-10-06T02:28:35.092173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:35.092173+0000","last_undegraded":"2023-10-06T02:28:35.092173+0000","last_fullsized":"2023-10-06T02:28:35.092173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:28:34.085717+0000","last_change":"2023-10-06T02:28:34.095027+0000","last_active":"2023-10-06T02:28:34.085717+0000","last_peered":"2023-10-06T02:28:34.085717+0000","last_clean":"2023-10-06T02:28:34.085717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:34.085717+0000","last_undegraded":"2023-10-06T02:28:34.085717+0000","last_fullsized":"2023-10-06T02:28:34.085717+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_clean_scrub_stamp":"2023-10-06T02:28:34.085717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:27:28.946486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:28:35.092173+0000","last_change":"2023-10-06T02:28:35.092173+0000","last_active":"2023-10-06T02:28:35.092173+0000","last_peered":"2023-10-06T02:28:35.092173+0000","last_clean":"2023-10-06T02:28:35.092173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:35.092173+0000","last_undegraded":"2023-10-06T02:28:35.092173+0000","last_fullsized":"2023-10-06T02:28:35.092173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:28:34.085717+0000","last_change":"2023-10-06T02:28:34.101380+0000","last_active":"2023-10-06T02:28:34.085717+0000","last_peered":"2023-10-06T02:28:34.085717+0000","last_clean":"2023-10-06T02:28:34.085717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:34.085717+0000","last_undegraded":"2023-10-06T02:28:34.085717+0000","last_fullsized":"2023-10-06T02:28:34.085717+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_clean_scrub_stamp":"2023-10-06T02:28:34.085717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:19:38.579720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:28:35.092173+0000","last_change":"2023-10-06T02:28:35.092173+0000","last_active":"2023-10-06T02:28:35.092173+0000","last_peered":"2023-10-06T02:28:35.092173+0000","last_clean":"2023-10-06T02:28:35.092173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:35.092173+0000","last_undegraded":"2023-10-06T02:28:35.092173+0000","last_fullsized":"2023-10-06T02:28:35.092173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:28:34.085717+0000","last_change":"2023-10-06T02:28:34.098411+0000","last_active":"2023-10-06T02:28:34.085717+0000","last_peered":"2023-10-06T02:28:34.085717+0000","last_clean":"2023-10-06T02:28:34.085717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:34.085717+0000","last_undegraded":"2023-10-06T02:28:34.085717+0000","last_fullsized":"2023-10-06T02:28:34.085717+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_clean_scrub_stamp":"2023-10-06T02:28:34.085717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:42:56.526314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:28:35.092173+0000","last_change":"2023-10-06T02:28:35.092173+0000","last_active":"2023-10-06T02:28:35.092173+0000","last_peered":"2023-10-06T02:28:35.092173+0000","last_clean":"2023-10-06T02:28:35.092173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:35.092173+0000","last_undegraded":"2023-10-06T02:28:35.092173+0000","last_fullsized":"2023-10-06T02:28:35.092173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:28:34.085717+0000","last_change":"2023-10-06T02:28:34.102127+0000","last_active":"2023-10-06T02:28:34.085717+0000","last_peered":"2023-10-06T02:28:34.085717+0000","last_clean":"2023-10-06T02:28:34.085717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:34.085717+0000","last_undegraded":"2023-10-06T02:28:34.085717+0000","last_fullsized":"2023-10-06T02:28:34.085717+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_clean_scrub_stamp":"2023-10-06T02:28:34.085717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:35:12.793893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:28:35.092173+0000","last_change":"2023-10-06T02:28:35.092173+0000","last_active":"2023-10-06T02:28:35.092173+0000","last_peered":"2023-10-06T02:28:35.092173+0000","last_clean":"2023-10-06T02:28:35.092173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:35.092173+0000","last_undegraded":"2023-10-06T02:28:35.092173+0000","last_fullsized":"2023-10-06T02:28:35.092173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:28:34.085717+0000","last_change":"2023-10-06T02:28:34.095669+0000","last_active":"2023-10-06T02:28:34.085717+0000","last_peered":"2023-10-06T02:28:34.085717+0000","last_clean":"2023-10-06T02:28:34.085717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:34.085717+0000","last_undegraded":"2023-10-06T02:28:34.085717+0000","last_fullsized":"2023-10-06T02:28:34.085717+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_clean_scrub_stamp":"2023-10-06T02:28:34.085717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:34:21.786686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:28:35.092173+0000","last_change":"2023-10-06T02:28:35.092173+0000","last_active":"2023-10-06T02:28:35.092173+0000","last_peered":"2023-10-06T02:28:35.092173+0000","last_clean":"2023-10-06T02:28:35.092173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:35.092173+0000","last_undegraded":"2023-10-06T02:28:35.092173+0000","last_fullsized":"2023-10-06T02:28:35.092173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:28:34.085717+0000","last_change":"2023-10-06T02:28:34.104059+0000","last_active":"2023-10-06T02:28:34.085717+0000","last_peered":"2023-10-06T02:28:34.085717+0000","last_clean":"2023-10-06T02:28:34.085717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:34.085717+0000","last_undegraded":"2023-10-06T02:28:34.085717+0000","last_fullsized":"2023-10-06T02:28:34.085717+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_clean_scrub_stamp":"2023-10-06T02:28:34.085717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:26:16.345317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:28:35.092173+0000","last_change":"2023-10-06T02:28:35.092173+0000","last_active":"2023-10-06T02:28:35.092173+0000","last_peered":"2023-10-06T02:28:35.092173+0000","last_clean":"2023-10-06T02:28:35.092173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:35.092173+0000","last_undegraded":"2023-10-06T02:28:35.092173+0000","last_fullsized":"2023-10-06T02:28:35.092173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:28:34.085717+0000","last_change":"2023-10-06T02:28:34.103528+0000","last_active":"2023-10-06T02:28:34.085717+0000","last_peered":"2023-10-06T02:28:34.085717+0000","last_clean":"2023-10-06T02:28:34.085717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:34.085717+0000","last_undegraded":"2023-10-06T02:28:34.085717+0000","last_fullsized":"2023-10-06T02:28:34.085717+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_clean_scrub_stamp":"2023-10-06T02:28:34.085717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:53:14.969785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:28:35.092173+0000","last_change":"2023-10-06T02:28:35.092173+0000","last_active":"2023-10-06T02:28:35.092173+0000","last_peered":"2023-10-06T02:28:35.092173+0000","last_clean":"2023-10-06T02:28:35.092173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:35.092173+0000","last_undegraded":"2023-10-06T02:28:35.092173+0000","last_fullsized":"2023-10-06T02:28:35.092173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:28:34.085717+0000","last_change":"2023-10-06T02:28:34.095016+0000","last_active":"2023-10-06T02:28:34.085717+0000","last_peered":"2023-10-06T02:28:34.085717+0000","last_clean":"2023-10-06T02:28:34.085717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:34.085717+0000","last_undegraded":"2023-10-06T02:28:34.085717+0000","last_fullsized":"2023-10-06T02:28:34.085717+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_clean_scrub_stamp":"2023-10-06T02:28:34.085717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:07:01.101933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:28:35.092173+0000","last_change":"2023-10-06T02:28:35.092173+0000","last_active":"2023-10-06T02:28:35.092173+0000","last_peered":"2023-10-06T02:28:35.092173+0000","last_clean":"2023-10-06T02:28:35.092173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:35.092173+0000","last_undegraded":"2023-10-06T02:28:35.092173+0000","last_fullsized":"2023-10-06T02:28:35.092173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:28:34.085717+0000","last_change":"2023-10-06T02:28:34.099439+0000","last_active":"2023-10-06T02:28:34.085717+0000","last_peered":"2023-10-06T02:28:34.085717+0000","last_clean":"2023-10-06T02:28:34.085717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:34.085717+0000","last_undegraded":"2023-10-06T02:28:34.085717+0000","last_fullsized":"2023-10-06T02:28:34.085717+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_clean_scrub_stamp":"2023-10-06T02:28:34.085717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:49:58.112298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:28:35.092173+0000","last_change":"2023-10-06T02:28:35.092173+0000","last_active":"2023-10-06T02:28:35.092173+0000","last_peered":"2023-10-06T02:28:35.092173+0000","last_clean":"2023-10-06T02:28:35.092173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:35.092173+0000","last_undegraded":"2023-10-06T02:28:35.092173+0000","last_fullsized":"2023-10-06T02:28:35.092173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:28:34.085717+0000","last_change":"2023-10-06T02:28:34.100323+0000","last_active":"2023-10-06T02:28:34.085717+0000","last_peered":"2023-10-06T02:28:34.085717+0000","last_clean":"2023-10-06T02:28:34.085717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:34.085717+0000","last_undegraded":"2023-10-06T02:28:34.085717+0000","last_fullsized":"2023-10-06T02:28:34.085717+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_clean_scrub_stamp":"2023-10-06T02:28:34.085717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:16:50.958977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:28:35.092173+0000","last_change":"2023-10-06T02:28:35.092173+0000","last_active":"2023-10-06T02:28:35.092173+0000","last_peered":"2023-10-06T02:28:35.092173+0000","last_clean":"2023-10-06T02:28:35.092173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:35.092173+0000","last_undegraded":"2023-10-06T02:28:35.092173+0000","last_fullsized":"2023-10-06T02:28:35.092173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:28:34.085717+0000","last_change":"2023-10-06T02:28:34.094472+0000","last_active":"2023-10-06T02:28:34.085717+0000","last_peered":"2023-10-06T02:28:34.085717+0000","last_clean":"2023-10-06T02:28:34.085717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:34.085717+0000","last_undegraded":"2023-10-06T02:28:34.085717+0000","last_fullsized":"2023-10-06T02:28:34.085717+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_clean_scrub_stamp":"2023-10-06T02:28:34.085717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:41:35.282495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:28:35.092173+0000","last_change":"2023-10-06T02:28:35.092173+0000","last_active":"2023-10-06T02:28:35.092173+0000","last_peered":"2023-10-06T02:28:35.092173+0000","last_clean":"2023-10-06T02:28:35.092173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:35.092173+0000","last_undegraded":"2023-10-06T02:28:35.092173+0000","last_fullsized":"2023-10-06T02:28:35.092173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:28:34.085717+0000","last_change":"2023-10-06T02:28:34.101604+0000","last_active":"2023-10-06T02:28:34.085717+0000","last_peered":"2023-10-06T02:28:34.085717+0000","last_clean":"2023-10-06T02:28:34.085717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:34.085717+0000","last_undegraded":"2023-10-06T02:28:34.085717+0000","last_fullsized":"2023-10-06T02:28:34.085717+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_clean_scrub_stamp":"2023-10-06T02:28:34.085717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:14:10.077126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:28:35.092173+0000","last_change":"2023-10-06T02:28:35.092173+0000","last_active":"2023-10-06T02:28:35.092173+0000","last_peered":"2023-10-06T02:28:35.092173+0000","last_clean":"2023-10-06T02:28:35.092173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:35.092173+0000","last_undegraded":"2023-10-06T02:28:35.092173+0000","last_fullsized":"2023-10-06T02:28:35.092173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:28:34.085717+0000","last_change":"2023-10-06T02:28:34.098623+0000","last_active":"2023-10-06T02:28:34.085717+0000","last_peered":"2023-10-06T02:28:34.085717+0000","last_clean":"2023-10-06T02:28:34.085717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:34.085717+0000","last_undegraded":"2023-10-06T02:28:34.085717+0000","last_fullsized":"2023-10-06T02:28:34.085717+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_clean_scrub_stamp":"2023-10-06T02:28:34.085717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:49:29.193059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":82,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T02:28:31.259103+0000","last_change":"2023-10-06T02:23:01.992935+0000","last_active":"2023-10-06T02:28:31.259103+0000","last_peered":"2023-10-06T02:28:31.259103+0000","last_clean":"2023-10-06T02:28:31.259103+0000","last_became_active":"2023-10-06T02:23:01.992568+0000","last_became_peered":"2023-10-06T02:23:01.992568+0000","last_unstale":"2023-10-06T02:28:31.259103+0000","last_undegraded":"2023-10-06T02:28:31.259103+0000","last_fullsized":"2023-10-06T02:28:31.259103+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:06.212251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:28:35.092173+0000","last_change":"2023-10-06T02:28:35.092173+0000","last_active":"2023-10-06T02:28:35.092173+0000","last_peered":"2023-10-06T02:28:35.092173+0000","last_clean":"2023-10-06T02:28:35.092173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:35.092173+0000","last_undegraded":"2023-10-06T02:28:35.092173+0000","last_fullsized":"2023-10-06T02:28:35.092173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:28:34.085717+0000","last_change":"2023-10-06T02:28:34.098730+0000","last_active":"2023-10-06T02:28:34.085717+0000","last_peered":"2023-10-06T02:28:34.085717+0000","last_clean":"2023-10-06T02:28:34.085717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:34.085717+0000","last_undegraded":"2023-10-06T02:28:34.085717+0000","last_fullsized":"2023-10-06T02:28:34.085717+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_clean_scrub_stamp":"2023-10-06T02:28:34.085717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:17:15.823723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":81,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T02:28:31.258629+0000","last_change":"2023-10-06T02:23:01.989934+0000","last_active":"2023-10-06T02:28:31.258629+0000","last_peered":"2023-10-06T02:28:31.258629+0000","last_clean":"2023-10-06T02:28:31.258629+0000","last_became_active":"2023-10-06T02:23:01.989706+0000","last_became_peered":"2023-10-06T02:23:01.989706+0000","last_unstale":"2023-10-06T02:28:31.258629+0000","last_undegraded":"2023-10-06T02:28:31.258629+0000","last_fullsized":"2023-10-06T02:28:31.258629+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:06.212251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:28:35.092173+0000","last_change":"2023-10-06T02:28:35.092173+0000","last_active":"2023-10-06T02:28:35.092173+0000","last_peered":"2023-10-06T02:28:35.092173+0000","last_clean":"2023-10-06T02:28:35.092173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:35.092173+0000","last_undegraded":"2023-10-06T02:28:35.092173+0000","last_fullsized":"2023-10-06T02:28:35.092173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:28:34.085717+0000","last_change":"2023-10-06T02:28:34.098651+0000","last_active":"2023-10-06T02:28:34.085717+0000","last_peered":"2023-10-06T02:28:34.085717+0000","last_clean":"2023-10-06T02:28:34.085717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:34.085717+0000","last_undegraded":"2023-10-06T02:28:34.085717+0000","last_fullsized":"2023-10-06T02:28:34.085717+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_clean_scrub_stamp":"2023-10-06T02:28:34.085717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:30:07.501845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":77,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T02:28:31.257937+0000","last_change":"2023-10-06T02:23:02.054069+0000","last_active":"2023-10-06T02:28:31.257937+0000","last_peered":"2023-10-06T02:28:31.257937+0000","last_clean":"2023-10-06T02:28:31.257937+0000","last_became_active":"2023-10-06T02:23:02.053739+0000","last_became_peered":"2023-10-06T02:23:02.053739+0000","last_unstale":"2023-10-06T02:28:31.257937+0000","last_undegraded":"2023-10-06T02:28:31.257937+0000","last_fullsized":"2023-10-06T02:28:31.257937+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:57:48.822608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:28:35.092173+0000","last_change":"2023-10-06T02:28:35.092173+0000","last_active":"2023-10-06T02:28:35.092173+0000","last_peered":"2023-10-06T02:28:35.092173+0000","last_clean":"2023-10-06T02:28:35.092173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:35.092173+0000","last_undegraded":"2023-10-06T02:28:35.092173+0000","last_fullsized":"2023-10-06T02:28:35.092173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:28:34.085717+0000","last_change":"2023-10-06T02:28:34.098354+0000","last_active":"2023-10-06T02:28:34.085717+0000","last_peered":"2023-10-06T02:28:34.085717+0000","last_clean":"2023-10-06T02:28:34.085717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:34.085717+0000","last_undegraded":"2023-10-06T02:28:34.085717+0000","last_fullsized":"2023-10-06T02:28:34.085717+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_clean_scrub_stamp":"2023-10-06T02:28:34.085717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:10:45.309448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":77,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T02:28:31.257931+0000","last_change":"2023-10-06T02:23:01.990773+0000","last_active":"2023-10-06T02:28:31.257931+0000","last_peered":"2023-10-06T02:28:31.257931+0000","last_clean":"2023-10-06T02:28:31.257931+0000","last_became_active":"2023-10-06T02:23:01.990511+0000","last_became_peered":"2023-10-06T02:23:01.990511+0000","last_unstale":"2023-10-06T02:28:31.257931+0000","last_undegraded":"2023-10-06T02:28:31.257931+0000","last_fullsized":"2023-10-06T02:28:31.257931+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:42:54.797731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:28:35.092173+0000","last_change":"2023-10-06T02:28:35.092173+0000","last_active":"2023-10-06T02:28:35.092173+0000","last_peered":"2023-10-06T02:28:35.092173+0000","last_clean":"2023-10-06T02:28:35.092173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:35.092173+0000","last_undegraded":"2023-10-06T02:28:35.092173+0000","last_fullsized":"2023-10-06T02:28:35.092173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:28:34.085717+0000","last_change":"2023-10-06T02:28:34.098862+0000","last_active":"2023-10-06T02:28:34.085717+0000","last_peered":"2023-10-06T02:28:34.085717+0000","last_clean":"2023-10-06T02:28:34.085717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:34.085717+0000","last_undegraded":"2023-10-06T02:28:34.085717+0000","last_fullsized":"2023-10-06T02:28:34.085717+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_clean_scrub_stamp":"2023-10-06T02:28:34.085717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:35:26.925918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"76'128","reported_seq":269,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T02:28:31.257979+0000","last_change":"2023-10-06T02:22:55.032519+0000","last_active":"2023-10-06T02:28:31.257979+0000","last_peered":"2023-10-06T02:28:31.257979+0000","last_clean":"2023-10-06T02:28:31.257979+0000","last_became_active":"2023-10-06T02:22:55.031367+0000","last_became_peered":"2023-10-06T02:22:55.031367+0000","last_unstale":"2023-10-06T02:28:31.257979+0000","last_undegraded":"2023-10-06T02:28:31.257979+0000","last_fullsized":"2023-10-06T02:28:31.257979+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:22:53.309097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:22:53.309097+0000","last_clean_scrub_stamp":"2023-10-06T02:22:53.309097+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:53:46.520909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":93,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T02:28:31.102915+0000","last_change":"2023-10-06T02:23:01.988140+0000","last_active":"2023-10-06T02:28:31.102915+0000","last_peered":"2023-10-06T02:28:31.102915+0000","last_clean":"2023-10-06T02:28:31.102915+0000","last_became_active":"2023-10-06T02:23:01.987769+0000","last_became_peered":"2023-10-06T02:23:01.987769+0000","last_unstale":"2023-10-06T02:28:31.102915+0000","last_undegraded":"2023-10-06T02:28:31.102915+0000","last_fullsized":"2023-10-06T02:28:31.102915+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:20:08.391912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:28:35.092173+0000","last_change":"2023-10-06T02:28:35.092173+0000","last_active":"2023-10-06T02:28:35.092173+0000","last_peered":"2023-10-06T02:28:35.092173+0000","last_clean":"2023-10-06T02:28:35.092173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:35.092173+0000","last_undegraded":"2023-10-06T02:28:35.092173+0000","last_fullsized":"2023-10-06T02:28:35.092173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:28:34.085717+0000","last_change":"2023-10-06T02:28:34.098304+0000","last_active":"2023-10-06T02:28:34.085717+0000","last_peered":"2023-10-06T02:28:34.085717+0000","last_clean":"2023-10-06T02:28:34.085717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:34.085717+0000","last_undegraded":"2023-10-06T02:28:34.085717+0000","last_fullsized":"2023-10-06T02:28:34.085717+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_clean_scrub_stamp":"2023-10-06T02:28:34.085717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:12:29.493807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":86,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T02:28:31.257592+0000","last_change":"2023-10-06T02:23:02.054338+0000","last_active":"2023-10-06T02:28:31.257592+0000","last_peered":"2023-10-06T02:28:31.257592+0000","last_clean":"2023-10-06T02:28:31.257592+0000","last_became_active":"2023-10-06T02:23:02.053991+0000","last_became_peered":"2023-10-06T02:23:02.053991+0000","last_unstale":"2023-10-06T02:28:31.257592+0000","last_undegraded":"2023-10-06T02:28:31.257592+0000","last_fullsized":"2023-10-06T02:28:31.257592+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:39:30.192838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:28:35.092173+0000","last_change":"2023-10-06T02:28:35.092173+0000","last_active":"2023-10-06T02:28:35.092173+0000","last_peered":"2023-10-06T02:28:35.092173+0000","last_clean":"2023-10-06T02:28:35.092173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:35.092173+0000","last_undegraded":"2023-10-06T02:28:35.092173+0000","last_fullsized":"2023-10-06T02:28:35.092173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:28:34.085717+0000","last_change":"2023-10-06T02:28:34.098919+0000","last_active":"2023-10-06T02:28:34.085717+0000","last_peered":"2023-10-06T02:28:34.085717+0000","last_clean":"2023-10-06T02:28:34.085717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:34.085717+0000","last_undegraded":"2023-10-06T02:28:34.085717+0000","last_fullsized":"2023-10-06T02:28:34.085717+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_clean_scrub_stamp":"2023-10-06T02:28:34.085717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:19:38.579720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":93,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T02:28:31.102902+0000","last_change":"2023-10-06T02:23:01.988631+0000","last_active":"2023-10-06T02:28:31.102902+0000","last_peered":"2023-10-06T02:28:31.102902+0000","last_clean":"2023-10-06T02:28:31.102902+0000","last_became_active":"2023-10-06T02:23:01.988303+0000","last_became_peered":"2023-10-06T02:23:01.988303+0000","last_unstale":"2023-10-06T02:28:31.102902+0000","last_undegraded":"2023-10-06T02:28:31.102902+0000","last_fullsized":"2023-10-06T02:28:31.102902+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:38.211466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T02:28:35.092173+0000","last_change":"2023-10-06T02:28:35.092173+0000","last_active":"2023-10-06T02:28:35.092173+0000","last_peered":"2023-10-06T02:28:35.092173+0000","last_clean":"2023-10-06T02:28:35.092173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:35.092173+0000","last_undegraded":"2023-10-06T02:28:35.092173+0000","last_fullsized":"2023-10-06T02:28:35.092173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T02:28:34.085717+0000","last_change":"2023-10-06T02:28:34.103595+0000","last_active":"2023-10-06T02:28:34.085717+0000","last_peered":"2023-10-06T02:28:34.085717+0000","last_clean":"2023-10-06T02:28:34.085717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T02:28:34.085717+0000","last_undegraded":"2023-10-06T02:28:34.085717+0000","last_fullsized":"2023-10-06T02:28:34.085717+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_clean_scrub_stamp":"2023-10-06T02:28:34.085717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:28:19.953693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":81,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T02:28:31.258616+0000","last_change":"2023-10-06T02:23:02.054039+0000","last_active":"2023-10-06T02:28:31.258616+0000","last_peered":"2023-10-06T02:28:31.258616+0000","last_clean":"2023-10-06T02:28:31.258616+0000","last_became_active":"2023-10-06T02:23:02.053742+0000","last_became_peered":"2023-10-06T02:23:02.053742+0000","last_unstale":"2023-10-06T02:28:31.258616+0000","last_undegraded":"2023-10-06T02:28:31.258616+0000","last_fullsized":"2023-10-06T02:28:31.258616+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:58:18.846475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574921,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303204,"kb_used_data":1820,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068636,"statfs":{"total":96636764160,"available":96326283264,"internally_reserved":0,"allocated":1863680,"data_stored":1174125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64200000000000002,"15min":0.63800000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.496},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63200000000000001,"15min":0.60899999999999999},"min":{"1min":0.379,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.52400000000000002}]},{"osd":1,"last update":"Fri Oct 6 02:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.71399999999999997,"15min":0.68600000000000005},"min":{"1min":0.48899999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.052,"5min":1.147,"15min":1.147},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.67700000000000005,"15min":0.67300000000000004},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.79300000000000004,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.83999999999999997}]},{"osd":2,"last update":"Fri Oct 6 02:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.68799999999999994,"15min":0.69299999999999995},"min":{"1min":0.46899999999999997,"5min":0.375,"15min":0.375},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68500000000000005,"15min":0.68899999999999995},"min":{"1min":0.46999999999999997,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.94499999999999995,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.70399999999999996}]},{"osd":3,"last update":"Fri Oct 6 02:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.749,"15min":0.73299999999999998},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.77200000000000002,"15min":0.77600000000000002},"min":{"1min":0.54500000000000004,"5min":0.497,"15min":0.497},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.58499999999999996}]},{"osd":4,"last update":"Fri Oct 6 02:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.63,"15min":0.68899999999999995},"min":{"1min":0.432,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.002,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.71399999999999997,"15min":0.76600000000000001},"min":{"1min":0.47199999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.51,"5min":1.51,"15min":1.51},"last":0.82399999999999995}]},{"osd":5,"last update":"Fri Oct 6 02:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.70799999999999996,"15min":0.73299999999999998},"min":{"1min":0.443,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70599999999999996,"15min":0.73199999999999998},"min":{"1min":0.48699999999999999,"5min":0.442,"15min":0.442},"max":{"1min":0.98099999999999998,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.67500000000000004}]},{"osd":6,"last update":"Fri Oct 6 02:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.77500000000000002,"15min":0.78700000000000003},"min":{"1min":0.60499999999999998,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.81299999999999994,"15min":0.81799999999999995},"min":{"1min":0.64900000000000002,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.6020000000000001,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.879}]}]},{"osd":6,"up_from":13,"seq":55834574921,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302624,"kb_used_data":1240,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069216,"statfs":{"total":96636764160,"available":96326877184,"internally_reserved":0,"allocated":1269760,"data_stored":583757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:28:05 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67300000000000004,"15min":0.66500000000000004},"min":{"1min":0.502,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.84399999999999997,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.71699999999999997,"15min":0.69799999999999995},"min":{"1min":0.51800000000000002,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0169999999999999,"5min":1.119,"15min":1.119},"last":0.82099999999999995}]},{"osd":1,"last update":"Fri Oct 6 02:28:05 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.64300000000000002,"15min":0.64500000000000002},"min":{"1min":0.45100000000000001,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.67800000000000005,"15min":0.66800000000000004},"min":{"1min":0.47399999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.66400000000000003}]},{"osd":2,"last update":"Fri Oct 6 02:28:05 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.71899999999999997,"15min":0.70499999999999996},"min":{"1min":0.54600000000000004,"5min":0.437,"15min":0.437},"max":{"1min":1.0369999999999999,"5min":1.137,"15min":1.137},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.72699999999999998,"15min":0.71099999999999997},"min":{"1min":0.45300000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.91300000000000003,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.88400000000000001}]},{"osd":3,"last update":"Fri Oct 6 02:28:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.69499999999999995,"15min":0.66400000000000003},"min":{"1min":0.55700000000000005,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.72099999999999997,"15min":0.70499999999999996},"min":{"1min":0.60399999999999998,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.92800000000000005,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.79500000000000004}]},{"osd":4,"last update":"Fri Oct 6 02:28:05 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.65900000000000003,"15min":0.64100000000000001},"min":{"1min":0.52300000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.94799999999999995,"5min":1.095,"15min":1.095},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.69799999999999995,"15min":0.66300000000000003},"min":{"1min":0.433,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.002,"5min":1.29,"15min":1.29},"last":0.92100000000000004}]},{"osd":5,"last update":"Fri Oct 6 02:28:05 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.69299999999999995,"15min":0.70099999999999996},"min":{"1min":0.50700000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.94999999999999996,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.754,"5min":0.71699999999999997,"15min":0.70499999999999996},"min":{"1min":0.53900000000000003,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.94299999999999995,"5min":1.02,"15min":1.02},"last":0.84799999999999998}]},{"osd":7,"last update":"Fri Oct 6 02:28:05 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.746,"15min":0.748},"min":{"1min":0.57499999999999996,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.96899999999999997,"5min":1.458,"15min":1.458},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.76400000000000001,"15min":0.76000000000000001},"min":{"1min":0.59099999999999997,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.98299999999999998,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.87}]}]},{"osd":1,"up_from":13,"seq":55834574920,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302636,"kb_used_data":1252,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069204,"statfs":{"total":96636764160,"available":96326864896,"internally_reserved":0,"allocated":1282048,"data_stored":589915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.442,"15min":0.46000000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.58299999999999996,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.48799999999999999,"15min":0.52700000000000002},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.70099999999999996,"5min":1.788,"15min":1.788},"last":0.39700000000000002}]},{"osd":2,"last update":"Fri Oct 6 02:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.46700000000000003,"15min":0.52300000000000002},"min":{"1min":0.307,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.60899999999999999,"5min":2.0840000000000001,"15min":2.0840000000000001},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.52100000000000002,"15min":0.55800000000000005},"min":{"1min":0.36899999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.66400000000000003,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.47599999999999998}]},{"osd":3,"last update":"Fri Oct 6 02:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60799999999999998,"15min":0.61699999999999999},"min":{"1min":0.46000000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.84999999999999998,"5min":1.266,"15min":1.266},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.56699999999999995,"15min":0.63900000000000001},"min":{"1min":0.41099999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.79400000000000004,"5min":1.861,"15min":1.861},"last":0.48199999999999998}]},{"osd":4,"last update":"Fri Oct 6 02:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.80700000000000005,"15min":0.85699999999999998},"min":{"1min":0.59099999999999997,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.3580000000000001,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.81499999999999995,"15min":0.85799999999999998},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.4339999999999999,"5min":2.0179999999999998,"15min":2.0179999999999998},"last":0.63900000000000001}]},{"osd":5,"last update":"Fri Oct 6 02:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.77600000000000002,"15min":0.85799999999999998},"min":{"1min":0.52600000000000002,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.1040000000000001,"5min":1.9830000000000001,"15min":1.9830000000000001},"last":1.3069999999999999},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.77400000000000002,"15min":0.82199999999999995},"min":{"1min":0.58899999999999997,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.161,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":1.327}]},{"osd":6,"last update":"Fri Oct 6 02:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.81799999999999995,"15min":0.86699999999999999},"min":{"1min":0.52700000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0609999999999999,"5min":1.829,"15min":1.829},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.84399999999999997,"15min":0.91200000000000003},"min":{"1min":0.64600000000000002,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.282,"5min":2.052,"15min":2.052},"last":1.351}]},{"osd":7,"last update":"Fri Oct 6 02:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.80000000000000004,"15min":0.83699999999999997},"min":{"1min":0.56299999999999994,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.3120000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.81699999999999995,"15min":0.879},"min":{"1min":0.55000000000000004,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.2110000000000001,"5min":1.895,"15min":1.895},"last":0.57099999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574921,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303204,"kb_used_data":1820,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068636,"statfs":{"total":96636764160,"available":96326283264,"internally_reserved":0,"allocated":1863680,"data_stored":1174125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 02:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.434,"15min":0.45400000000000001},"min":{"1min":0.28899999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.59099999999999997,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.46100000000000002,"15min":0.47699999999999998},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.57299999999999995,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.36699999999999999}]},{"osd":2,"last update":"Fri Oct 6 02:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.49299999999999999,"15min":0.51200000000000001},"min":{"1min":0.35999999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.67400000000000004,"5min":1.069,"15min":1.069},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.501,"15min":0.51900000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.67000000000000004,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.53400000000000003}]},{"osd":3,"last update":"Fri Oct 6 02:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.48599999999999999,"15min":0.503},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.60399999999999998,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.58699999999999997,"15min":0.59499999999999997},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.68899999999999995,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.54400000000000004}]},{"osd":4,"last update":"Fri Oct 6 02:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.751,"15min":0.79700000000000004},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.94199999999999995,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.80000000000000004},"min":{"1min":0.51000000000000001,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.085,"5min":1.304,"15min":1.304},"last":0.753}]},{"osd":5,"last update":"Fri Oct 6 02:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.76100000000000001,"15min":0.78600000000000003},"min":{"1min":0.58499999999999996,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78500000000000003,"15min":0.80100000000000005},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.998,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.76700000000000002}]},{"osd":6,"last update":"Fri Oct 6 02:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.80200000000000005,"15min":0.82199999999999995},"min":{"1min":0.60499999999999998,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.0089999999999999,"5min":1.167,"15min":1.167},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.78900000000000003,"15min":0.82899999999999996},"min":{"1min":0.61399999999999999,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.69699999999999995}]},{"osd":7,"last update":"Fri Oct 6 02:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79600000000000004,"15min":0.82599999999999996},"min":{"1min":0.61199999999999999,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.244,"5min":1.244,"15min":1.244},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.748,"5min":0.77600000000000002,"15min":0.82199999999999995},"min":{"1min":0.57599999999999996,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.016,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.68300000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574922,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302636,"kb_used_data":1252,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069204,"statfs":{"total":96636764160,"available":96326864896,"internally_reserved":0,"allocated":1282048,"data_stored":589915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:28:07 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.42899999999999999,"15min":0.439},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.372},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.47599999999999998,"15min":0.48599999999999999},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.80200000000000005,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.45500000000000002}]},{"osd":1,"last update":"Fri Oct 6 02:28:07 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.48999999999999999,"15min":0.504},"min":{"1min":0.35399999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.59199999999999997,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.55100000000000005,"15min":0.60899999999999999},"min":{"1min":0.40300000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.778,"5min":1.778,"15min":1.778},"last":0.50700000000000001}]},{"osd":3,"last update":"Fri Oct 6 02:28:07 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.51300000000000001,"15min":0.53300000000000003},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.77000000000000002,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.59499999999999997,"15min":0.62},"min":{"1min":0.46500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.81999999999999995,"5min":1.028,"15min":1.028},"last":0.55600000000000005}]},{"osd":4,"last update":"Fri Oct 6 02:28:07 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.752,"15min":0.77700000000000002},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0149999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.629},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.80800000000000005,"15min":0.84999999999999998},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.907,"5min":1.907,"15min":1.907},"last":0.67800000000000005}]},{"osd":5,"last update":"Fri Oct 6 02:28:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.752,"15min":0.80000000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.109,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.77600000000000002,"15min":0.82099999999999995},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":0.90700000000000003,"5min":1.258,"15min":1.258},"last":0.69399999999999995}]},{"osd":6,"last update":"Fri Oct 6 02:28:07 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.80800000000000005,"15min":0.84899999999999998},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.827,"5min":1.827,"15min":1.827},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.77200000000000002,"15min":0.79400000000000004},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.91900000000000004,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.86499999999999999}]},{"osd":7,"last update":"Fri Oct 6 02:28:07 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.82899999999999996,"15min":0.88400000000000001},"min":{"1min":0.5,"5min":0.499,"15min":0.499},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.746,"5min":0.79000000000000004,"15min":0.82999999999999996},"min":{"1min":0.55300000000000005,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.0600000000000001,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.90600000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574919,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302624,"kb_used_data":1240,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069216,"statfs":{"total":96636764160,"available":96326877184,"internally_reserved":0,"allocated":1269760,"data_stored":583757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:28:05 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.47599999999999998,"15min":0.51200000000000001},"min":{"1min":0.32400000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":1.0489999999999999,"5min":2.024,"15min":2.024},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.502,"15min":0.52500000000000002},"min":{"1min":0.371,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.622,"5min":2.181,"15min":2.181},"last":0.40600000000000003}]},{"osd":1,"last update":"Fri Oct 6 02:28:05 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.52800000000000002,"15min":0.68300000000000005},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.66100000000000003,"5min":4.492,"15min":4.492},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.58899999999999997,"15min":0.72899999999999998},"min":{"1min":0.39600000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.079,"5min":4.569,"15min":4.569},"last":0.47299999999999998}]},{"osd":2,"last update":"Fri Oct 6 02:28:05 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.64700000000000002,"15min":0.66900000000000004},"min":{"1min":0.47199999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.024,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":0.438},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61199999999999999,"15min":0.65500000000000003},"min":{"1min":0.40999999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.92100000000000004,"5min":2.4260000000000002,"15min":2.4260000000000002},"last":0.497}]},{"osd":4,"last update":"Fri Oct 6 02:28:05 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.76200000000000001,"15min":0.83099999999999996},"min":{"1min":0.56499999999999995,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.0149999999999999,"5min":2.214,"15min":2.214},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.77200000000000002,"15min":0.81200000000000006},"min":{"1min":0.56899999999999995,"5min":0.5,"15min":0.5},"max":{"1min":1.0389999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.56499999999999995}]},{"osd":5,"last update":"Fri Oct 6 02:28:05 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.79000000000000004,"15min":0.82399999999999995},"min":{"1min":0.60399999999999998,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.99099999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78800000000000003,"15min":0.85799999999999998},"min":{"1min":0.59699999999999998,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.0580000000000001,"5min":2.242,"15min":2.242},"last":0.60099999999999998}]},{"osd":6,"last update":"Fri Oct 6 02:28:05 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78100000000000003,"15min":0.82099999999999995},"min":{"1min":0.58699999999999997,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.0489999999999999,"5min":1.357,"15min":1.357},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.81100000000000005,"15min":0.83499999999999996},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.016,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.68600000000000005}]},{"osd":7,"last update":"Fri Oct 6 02:28:05 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.81399999999999995,"15min":0.86399999999999999},"min":{"1min":0.61599999999999999,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.94699999999999995,"5min":2.27,"15min":2.27},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81899999999999995,"15min":0.85999999999999999},"min":{"1min":0.57999999999999996,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.0760000000000001,"5min":1.325,"15min":1.325},"last":0.629}]}]},{"osd":4,"up_from":13,"seq":55834574921,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302624,"kb_used_data":1240,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069216,"statfs":{"total":96636764160,"available":96326877184,"internally_reserved":0,"allocated":1269760,"data_stored":583757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:28:05 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.76200000000000001,"15min":1.02},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.86699999999999999,"5min":12.337,"15min":12.337},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.625,"5min":0.73499999999999999,"15min":0.92500000000000004},"min":{"1min":0.44400000000000001,"5min":0.438,"15min":0.438},"max":{"1min":1.3520000000000001,"5min":9.6539999999999999,"15min":9.6539999999999999},"last":0.46200000000000002}]},{"osd":1,"last update":"Fri Oct 6 02:28:05 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.68300000000000005,"15min":0.69999999999999996},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.99299999999999999,"5min":3.181,"15min":3.181},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.77500000000000002,"15min":1.0409999999999999},"min":{"1min":0.498,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.91800000000000004,"5min":12.946999999999999,"15min":12.946999999999999},"last":0.67400000000000004}]},{"osd":2,"last update":"Fri Oct 6 02:28:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.79800000000000004,"15min":1.0700000000000001},"min":{"1min":0.505,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.2989999999999999,"5min":12.377000000000001,"15min":12.377000000000001},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.75600000000000001,"15min":0.88},"min":{"1min":0.60599999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.97399999999999998,"5min":6.3220000000000001,"15min":6.3220000000000001},"last":0.56000000000000005}]},{"osd":3,"last update":"Fri Oct 6 02:28:05 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.79300000000000004,"15min":1.004},"min":{"1min":0.46200000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.5229999999999999,"5min":9.7720000000000002,"15min":9.7720000000000002},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.72399999999999998,"15min":0.77800000000000002},"min":{"1min":0.52400000000000002,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.94199999999999995,"5min":3.0840000000000001,"15min":3.0840000000000001},"last":0.76000000000000001}]},{"osd":5,"last update":"Fri Oct 6 02:28:05 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.71699999999999997,"15min":0.85399999999999998},"min":{"1min":0.504,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.02,"5min":6.2690000000000001,"15min":6.2690000000000001},"last":0.626},{"interface":"front","average":{"1min":0.754,"5min":0.78700000000000003,"15min":1.0549999999999999},"min":{"1min":0.54100000000000004,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.4359999999999999,"5min":10.769,"15min":10.769},"last":0.64400000000000002}]},{"osd":6,"last update":"Fri Oct 6 02:28:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.69599999999999995,"15min":0.76400000000000001},"min":{"1min":0.433,"5min":0.432,"15min":0.432},"max":{"1min":1.4590000000000001,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.80100000000000005,"15min":0.98499999999999999},"min":{"1min":0.5,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.3839999999999999,"5min":8.0899999999999999,"15min":8.0899999999999999},"last":0.72899999999999998}]},{"osd":7,"last update":"Fri Oct 6 02:28:05 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.84399999999999997,"15min":1.0860000000000001},"min":{"1min":0.59499999999999997,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.494,"5min":9.7089999999999996,"15min":9.7089999999999996},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.746,"5min":0.81599999999999995,"15min":1.01},"min":{"1min":0.52100000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.995,"5min":7.6719999999999997,"15min":7.6719999999999997},"last":0.871}]}]},{"osd":5,"up_from":13,"seq":55834574921,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302636,"kb_used_data":1252,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069204,"statfs":{"total":96636764160,"available":96326864896,"internally_reserved":0,"allocated":1282048,"data_stored":589915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:28:03 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.64400000000000002,"15min":0.626},"min":{"1min":0.47099999999999997,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.96999999999999997,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.66100000000000003,"15min":0.61199999999999999},"min":{"1min":0.44600000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.55800000000000005}]},{"osd":1,"last update":"Fri Oct 6 02:28:03 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.68200000000000005,"15min":0.65400000000000003},"min":{"1min":0.48599999999999999,"5min":0.434,"15min":0.434},"max":{"1min":1.0660000000000001,"5min":1.089,"15min":1.089},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.68700000000000006,"15min":0.67400000000000004},"min":{"1min":0.41999999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.66900000000000004}]},{"osd":2,"last update":"Fri Oct 6 02:28:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.69899999999999995,"15min":0.66900000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.70099999999999996,"15min":0.68300000000000005},"min":{"1min":0.46100000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.73999999999999999}]},{"osd":3,"last update":"Fri Oct 6 02:28:03 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.75,"15min":0.72699999999999998},"min":{"1min":0.503,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.79300000000000004,"15min":0.76800000000000002},"min":{"1min":0.629,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.194,"5min":1.194,"15min":1.194},"last":0.76900000000000002}]},{"osd":4,"last update":"Fri Oct 6 02:28:03 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.66300000000000003,"15min":0.67300000000000004},"min":{"1min":0.48299999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.0089999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.751},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.67000000000000004,"15min":0.68200000000000005},"min":{"1min":0.40799999999999997,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.63400000000000001}]},{"osd":6,"last update":"Fri Oct 6 02:28:03 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.73499999999999999,"15min":0.74199999999999999},"min":{"1min":0.45900000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.97699999999999998,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81299999999999994,"15min":0.81999999999999995},"min":{"1min":0.56799999999999995,"5min":0.373,"15min":0.373},"max":{"1min":1.1200000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.79400000000000004}]},{"osd":7,"last update":"Fri Oct 6 02:28:03 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.76100000000000001,"15min":0.76400000000000001},"min":{"1min":0.498,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.81100000000000005,"15min":0.83299999999999996},"min":{"1min":0.51700000000000002,"5min":0.437,"15min":0.437},"max":{"1min":1.1040000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.84399999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T02:28:40.506 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T02:28:40.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.693+0000 7f841699d700 1 Processor -- start 2023-10-06T02:28:40.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.693+0000 7f841699d700 1 -- start start 2023-10-06T02:28:40.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.694+0000 7f841699d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8410142c00 0x7f841006b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:40.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.694+0000 7f841699d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f841006bd20 0x7f84100c7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:40.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.694+0000 7f841699d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8410142fd0 0x7f84100c7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:40.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.694+0000 7f841699d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84100ca480 con 0x7f841006bd20 2023-10-06T02:28:40.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.694+0000 7f841699d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f841015e240 con 0x7f8410142c00 2023-10-06T02:28:40.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.694+0000 7f841699d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f841006b350 con 0x7f8410142fd0 2023-10-06T02:28:40.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.694+0000 7f841519a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f841006bd20 0x7f84100c7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:40.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.694+0000 7f841519a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f841006bd20 0x7f84100c7880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36770/0 (socket says 172.21.15.31:36770) 2023-10-06T02:28:40.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.695+0000 7f841519a700 1 -- 172.21.15.31:0/2652922756 learned_addr learned my addr 172.21.15.31:0/2652922756 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:40.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.695+0000 7f841619c700 1 --2- 172.21.15.31:0/2652922756 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8410142fd0 0x7f84100c7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:40.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.695+0000 7f841599b700 1 --2- 172.21.15.31:0/2652922756 >> v2:172.21.15.62:3300/0 conn(0x7f8410142c00 0x7f841006b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:40.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.695+0000 7f841519a700 1 -- 172.21.15.31:0/2652922756 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8410142fd0 msgr2=0x7f84100c7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:40.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.695+0000 7f841519a700 1 --2- 172.21.15.31:0/2652922756 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8410142fd0 0x7f84100c7dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:40.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.695+0000 7f841519a700 1 -- 172.21.15.31:0/2652922756 >> v2:172.21.15.62:3300/0 conn(0x7f8410142c00 msgr2=0x7f841006b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:40.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.695+0000 7f841519a700 1 --2- 172.21.15.31:0/2652922756 >> v2:172.21.15.62:3300/0 conn(0x7f8410142c00 0x7f841006b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:40.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.695+0000 7f841519a700 1 -- 172.21.15.31:0/2652922756 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f841015df60 con 0x7f841006bd20 2023-10-06T02:28:40.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.695+0000 7f841519a700 1 --2- 172.21.15.31:0/2652922756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f841006bd20 0x7f84100c7880 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f841006f490 tx=0x7f8400009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc0ac0907ba8fd0e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:40.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.695+0000 7f8414999700 1 -- 172.21.15.31:0/2652922756 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840000b860 con 0x7f841006bd20 2023-10-06T02:28:40.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.695+0000 7f8414999700 1 -- 172.21.15.31:0/2652922756 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f840000b9c0 con 0x7f841006bd20 2023-10-06T02:28:40.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.695+0000 7f8414999700 1 -- 172.21.15.31:0/2652922756 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840000bd60 con 0x7f841006bd20 2023-10-06T02:28:40.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.696+0000 7f841699d700 1 -- 172.21.15.31:0/2652922756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f841006bd20 msgr2=0x7f84100c7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:40.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.696+0000 7f841699d700 1 --2- 172.21.15.31:0/2652922756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f841006bd20 0x7f84100c7880 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f841006f490 tx=0x7f8400009250 comp rx=0 tx=0).stop 2023-10-06T02:28:40.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.696+0000 7f841699d700 1 -- 172.21.15.31:0/2652922756 shutdown_connections 2023-10-06T02:28:40.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.696+0000 7f841699d700 1 --2- 172.21.15.31:0/2652922756 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8410142fd0 0x7f84100c7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:40.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.696+0000 7f841699d700 1 --2- 172.21.15.31:0/2652922756 >> v2:172.21.15.62:3300/0 conn(0x7f8410142c00 0x7f841006b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:40.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.696+0000 7f841699d700 1 --2- 172.21.15.31:0/2652922756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f841006bd20 0x7f84100c7880 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:40.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.696+0000 7f841699d700 1 -- 172.21.15.31:0/2652922756 >> 172.21.15.31:0/2652922756 conn(0x7f84100675a0 msgr2=0x7f84100679a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:40.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.696+0000 7f841699d700 1 -- 172.21.15.31:0/2652922756 shutdown_connections 2023-10-06T02:28:40.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.696+0000 7f841699d700 1 -- 172.21.15.31:0/2652922756 wait complete. 2023-10-06T02:28:40.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.698+0000 7f841699d700 1 Processor -- start 2023-10-06T02:28:40.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.698+0000 7f841699d700 1 -- start start 2023-10-06T02:28:40.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.699+0000 7f841699d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f841006bd20 0x7f8410158f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:40.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.699+0000 7f841699d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8410142c00 0x7f84100affe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:40.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.699+0000 7f841699d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8410142fd0 0x7f841013f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:40.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.699+0000 7f841699d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84100c54b0 con 0x7f8410142fd0 2023-10-06T02:28:40.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.699+0000 7f841699d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84100b1670 con 0x7f8410142c00 2023-10-06T02:28:40.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.699+0000 7f841699d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84100b0a60 con 0x7f841006bd20 2023-10-06T02:28:40.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.699+0000 7f841619c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8410142fd0 0x7f841013f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:40.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.699+0000 7f841619c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8410142fd0 0x7f841013f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36782/0 (socket says 172.21.15.31:36782) 2023-10-06T02:28:40.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.699+0000 7f841619c700 1 -- 172.21.15.31:0/2684058858 learned_addr learned my addr 172.21.15.31:0/2684058858 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:40.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.699+0000 7f841599b700 1 --2- 172.21.15.31:0/2684058858 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f841006bd20 0x7f8410158f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:40.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.699+0000 7f841519a700 1 --2- 172.21.15.31:0/2684058858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8410142c00 0x7f84100affe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:40.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.699+0000 7f841619c700 1 -- 172.21.15.31:0/2684058858 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f841006bd20 msgr2=0x7f8410158f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:40.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.699+0000 7f841619c700 1 --2- 172.21.15.31:0/2684058858 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f841006bd20 0x7f8410158f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:40.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.699+0000 7f841619c700 1 -- 172.21.15.31:0/2684058858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8410142c00 msgr2=0x7f84100affe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:40.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.699+0000 7f841619c700 1 --2- 172.21.15.31:0/2684058858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8410142c00 0x7f84100affe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:40.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.699+0000 7f841619c700 1 -- 172.21.15.31:0/2684058858 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84000094f0 con 0x7f8410142fd0 2023-10-06T02:28:40.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.699+0000 7f841519a700 1 --2- 172.21.15.31:0/2684058858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8410142c00 0x7f84100affe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:28:40.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.700+0000 7f841619c700 1 --2- 172.21.15.31:0/2684058858 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8410142fd0 0x7f841013f440 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f8408007f20 tx=0x7f8408009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:40.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.700+0000 7f8405ffb700 1 -- 172.21.15.31:0/2684058858 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840800ebf0 con 0x7f8410142fd0 2023-10-06T02:28:40.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.700+0000 7f8405ffb700 1 -- 172.21.15.31:0/2684058858 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f840800ed70 con 0x7f8410142fd0 2023-10-06T02:28:40.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.700+0000 7f8405ffb700 1 -- 172.21.15.31:0/2684058858 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8408014670 con 0x7f8410142fd0 2023-10-06T02:28:40.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.700+0000 7f841699d700 1 -- 172.21.15.31:0/2684058858 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84100b0d40 con 0x7f8410142fd0 2023-10-06T02:28:40.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.700+0000 7f841699d700 1 -- 172.21.15.31:0/2684058858 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84100b1380 con 0x7f8410142fd0 2023-10-06T02:28:40.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.701+0000 7f841699d700 1 -- 172.21.15.31:0/2684058858 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f841004c3c0 con 0x7f8410142fd0 2023-10-06T02:28:40.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.702+0000 7f8405ffb700 1 -- 172.21.15.31:0/2684058858 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f84080149e0 con 0x7f8410142fd0 2023-10-06T02:28:40.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.703+0000 7f8405ffb700 1 --2- 172.21.15.31:0/2684058858 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f83f4067050 0x7f83f4069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:40.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.703+0000 7f8405ffb700 1 -- 172.21.15.31:0/2684058858 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8408084f30 con 0x7f8410142fd0 2023-10-06T02:28:40.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.703+0000 7f841599b700 1 --2- 172.21.15.31:0/2684058858 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f83f4067050 0x7f83f4069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:40.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.704+0000 7f841599b700 1 --2- 172.21.15.31:0/2684058858 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f83f4067050 0x7f83f4069500 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f840c006fd0 tx=0x7f840c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:40.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.707+0000 7f8405ffb700 1 -- 172.21.15.31:0/2684058858 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8408051e90 con 0x7f8410142fd0 2023-10-06T02:28:40.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.910+0000 7f841699d700 1 -- 172.21.15.31:0/2684058858 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f841006ad00 con 0x7f83f4067050 2023-10-06T02:28:40.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.920+0000 7f8405ffb700 1 -- 172.21.15.31:0/2684058858 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311121 (secure 0 0 0) 0x7f841006ad00 con 0x7f83f4067050 2023-10-06T02:28:40.925 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:28:40.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.924+0000 7f841699d700 1 -- 172.21.15.31:0/2684058858 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f83f4067050 msgr2=0x7f83f4069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:40.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.924+0000 7f841699d700 1 --2- 172.21.15.31:0/2684058858 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f83f4067050 0x7f83f4069500 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f840c006fd0 tx=0x7f840c008040 comp rx=0 tx=0).stop 2023-10-06T02:28:40.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.924+0000 7f841699d700 1 -- 172.21.15.31:0/2684058858 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8410142fd0 msgr2=0x7f841013f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:40.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.924+0000 7f841699d700 1 --2- 172.21.15.31:0/2684058858 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8410142fd0 0x7f841013f440 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f8408007f20 tx=0x7f8408009860 comp rx=0 tx=0).stop 2023-10-06T02:28:40.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.924+0000 7f841699d700 1 -- 172.21.15.31:0/2684058858 shutdown_connections 2023-10-06T02:28:40.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.924+0000 7f841699d700 1 --2- 172.21.15.31:0/2684058858 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8410142fd0 0x7f841013f440 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:40.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.924+0000 7f841699d700 1 --2- 172.21.15.31:0/2684058858 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f83f4067050 0x7f83f4069500 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:40.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.924+0000 7f841699d700 1 --2- 172.21.15.31:0/2684058858 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f841006bd20 0x7f8410158f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:40.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.924+0000 7f841699d700 1 --2- 172.21.15.31:0/2684058858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8410142c00 0x7f84100affe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:40.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.924+0000 7f841699d700 1 -- 172.21.15.31:0/2684058858 >> 172.21.15.31:0/2684058858 conn(0x7f84100675a0 msgr2=0x7f84100c7860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:40.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.925+0000 7f841699d700 1 -- 172.21.15.31:0/2684058858 shutdown_connections 2023-10-06T02:28:40.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:40.925+0000 7f841699d700 1 -- 172.21.15.31:0/2684058858 wait complete. 2023-10-06T02:28:40.929 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-10-06T02:28:40.943 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":65,"stamp":"2023-10-06T02:28:40.465933+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":223,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":166,"ondisk_log_size":166,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":188,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2422600,"kb_used_data":11528,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752552120,"statfs":{"total":773094113280,"available":770613370880,"internally_reserved":0,"allocated":11804672,"data_stored":6067652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.010722"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.120863+0000","last_change":"2023-10-06T02:28:36.119634+0000","last_active":"2023-10-06T02:28:36.120863+0000","last_peered":"2023-10-06T02:28:36.120863+0000","last_clean":"2023-10-06T02:28:36.120863+0000","last_became_active":"2023-10-06T02:28:36.119343+0000","last_became_peered":"2023-10-06T02:28:36.119343+0000","last_unstale":"2023-10-06T02:28:36.120863+0000","last_undegraded":"2023-10-06T02:28:36.120863+0000","last_fullsized":"2023-10-06T02:28:36.120863+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:24:14.880041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.142413+0000","last_change":"2023-10-06T02:28:36.135139+0000","last_active":"2023-10-06T02:28:36.142413+0000","last_peered":"2023-10-06T02:28:36.142413+0000","last_clean":"2023-10-06T02:28:36.142413+0000","last_became_active":"2023-10-06T02:28:36.134841+0000","last_became_peered":"2023-10-06T02:28:36.134841+0000","last_unstale":"2023-10-06T02:28:36.142413+0000","last_undegraded":"2023-10-06T02:28:36.142413+0000","last_fullsized":"2023-10-06T02:28:36.142413+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:55:58.729547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.133140+0000","last_change":"2023-10-06T02:28:36.129480+0000","last_active":"2023-10-06T02:28:36.133140+0000","last_peered":"2023-10-06T02:28:36.133140+0000","last_clean":"2023-10-06T02:28:36.133140+0000","last_became_active":"2023-10-06T02:28:36.129218+0000","last_became_peered":"2023-10-06T02:28:36.129218+0000","last_unstale":"2023-10-06T02:28:36.133140+0000","last_undegraded":"2023-10-06T02:28:36.133140+0000","last_fullsized":"2023-10-06T02:28:36.133140+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:23:06.700299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.141573+0000","last_change":"2023-10-06T02:28:36.135873+0000","last_active":"2023-10-06T02:28:36.141573+0000","last_peered":"2023-10-06T02:28:36.141573+0000","last_clean":"2023-10-06T02:28:36.141573+0000","last_became_active":"2023-10-06T02:28:36.135260+0000","last_became_peered":"2023-10-06T02:28:36.135260+0000","last_unstale":"2023-10-06T02:28:36.141573+0000","last_undegraded":"2023-10-06T02:28:36.141573+0000","last_fullsized":"2023-10-06T02:28:36.141573+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:45:18.729211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.131686+0000","last_change":"2023-10-06T02:28:36.127508+0000","last_active":"2023-10-06T02:28:36.131686+0000","last_peered":"2023-10-06T02:28:36.131686+0000","last_clean":"2023-10-06T02:28:36.131686+0000","last_became_active":"2023-10-06T02:28:36.127004+0000","last_became_peered":"2023-10-06T02:28:36.127004+0000","last_unstale":"2023-10-06T02:28:36.131686+0000","last_undegraded":"2023-10-06T02:28:36.131686+0000","last_fullsized":"2023-10-06T02:28:36.131686+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:53:31.275391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.125650+0000","last_change":"2023-10-06T02:28:36.122826+0000","last_active":"2023-10-06T02:28:36.125650+0000","last_peered":"2023-10-06T02:28:36.125650+0000","last_clean":"2023-10-06T02:28:36.125650+0000","last_became_active":"2023-10-06T02:28:36.122452+0000","last_became_peered":"2023-10-06T02:28:36.122452+0000","last_unstale":"2023-10-06T02:28:36.125650+0000","last_undegraded":"2023-10-06T02:28:36.125650+0000","last_fullsized":"2023-10-06T02:28:36.125650+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:20:29.810481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.132174+0000","last_change":"2023-10-06T02:28:36.130651+0000","last_active":"2023-10-06T02:28:36.132174+0000","last_peered":"2023-10-06T02:28:36.132174+0000","last_clean":"2023-10-06T02:28:36.132174+0000","last_became_active":"2023-10-06T02:28:36.130368+0000","last_became_peered":"2023-10-06T02:28:36.130368+0000","last_unstale":"2023-10-06T02:28:36.132174+0000","last_undegraded":"2023-10-06T02:28:36.132174+0000","last_fullsized":"2023-10-06T02:28:36.132174+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:58:36.105967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.133108+0000","last_change":"2023-10-06T02:28:36.131837+0000","last_active":"2023-10-06T02:28:36.133108+0000","last_peered":"2023-10-06T02:28:36.133108+0000","last_clean":"2023-10-06T02:28:36.133108+0000","last_became_active":"2023-10-06T02:28:36.131516+0000","last_became_peered":"2023-10-06T02:28:36.131516+0000","last_unstale":"2023-10-06T02:28:36.133108+0000","last_undegraded":"2023-10-06T02:28:36.133108+0000","last_fullsized":"2023-10-06T02:28:36.133108+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:20:12.262390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.139983+0000","last_change":"2023-10-06T02:28:36.134165+0000","last_active":"2023-10-06T02:28:36.139983+0000","last_peered":"2023-10-06T02:28:36.139983+0000","last_clean":"2023-10-06T02:28:36.139983+0000","last_became_active":"2023-10-06T02:28:36.133900+0000","last_became_peered":"2023-10-06T02:28:36.133900+0000","last_unstale":"2023-10-06T02:28:36.139983+0000","last_undegraded":"2023-10-06T02:28:36.139983+0000","last_fullsized":"2023-10-06T02:28:36.139983+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:23:06.700299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.130083+0000","last_change":"2023-10-06T02:28:36.128461+0000","last_active":"2023-10-06T02:28:36.130083+0000","last_peered":"2023-10-06T02:28:36.130083+0000","last_clean":"2023-10-06T02:28:36.130083+0000","last_became_active":"2023-10-06T02:28:36.128211+0000","last_became_peered":"2023-10-06T02:28:36.128211+0000","last_unstale":"2023-10-06T02:28:36.130083+0000","last_undegraded":"2023-10-06T02:28:36.130083+0000","last_fullsized":"2023-10-06T02:28:36.130083+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:16:51.965432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.132124+0000","last_change":"2023-10-06T02:28:36.128085+0000","last_active":"2023-10-06T02:28:36.132124+0000","last_peered":"2023-10-06T02:28:36.132124+0000","last_clean":"2023-10-06T02:28:36.132124+0000","last_became_active":"2023-10-06T02:28:36.127816+0000","last_became_peered":"2023-10-06T02:28:36.127816+0000","last_unstale":"2023-10-06T02:28:36.132124+0000","last_undegraded":"2023-10-06T02:28:36.132124+0000","last_fullsized":"2023-10-06T02:28:36.132124+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:51:19.033267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.121575+0000","last_change":"2023-10-06T02:28:36.116285+0000","last_active":"2023-10-06T02:28:36.121575+0000","last_peered":"2023-10-06T02:28:36.121575+0000","last_clean":"2023-10-06T02:28:36.121575+0000","last_became_active":"2023-10-06T02:28:36.115798+0000","last_became_peered":"2023-10-06T02:28:36.115798+0000","last_unstale":"2023-10-06T02:28:36.121575+0000","last_undegraded":"2023-10-06T02:28:36.121575+0000","last_fullsized":"2023-10-06T02:28:36.121575+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:34.609090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.130118+0000","last_change":"2023-10-06T02:28:36.126850+0000","last_active":"2023-10-06T02:28:36.130118+0000","last_peered":"2023-10-06T02:28:36.130118+0000","last_clean":"2023-10-06T02:28:36.130118+0000","last_became_active":"2023-10-06T02:28:36.126536+0000","last_became_peered":"2023-10-06T02:28:36.126536+0000","last_unstale":"2023-10-06T02:28:36.130118+0000","last_undegraded":"2023-10-06T02:28:36.130118+0000","last_fullsized":"2023-10-06T02:28:36.130118+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:04:14.164111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.129494+0000","last_change":"2023-10-06T02:28:36.127832+0000","last_active":"2023-10-06T02:28:36.129494+0000","last_peered":"2023-10-06T02:28:36.129494+0000","last_clean":"2023-10-06T02:28:36.129494+0000","last_became_active":"2023-10-06T02:28:36.127599+0000","last_became_peered":"2023-10-06T02:28:36.127599+0000","last_unstale":"2023-10-06T02:28:36.129494+0000","last_undegraded":"2023-10-06T02:28:36.129494+0000","last_fullsized":"2023-10-06T02:28:36.129494+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:41:36.288950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.131706+0000","last_change":"2023-10-06T02:28:36.127500+0000","last_active":"2023-10-06T02:28:36.131706+0000","last_peered":"2023-10-06T02:28:36.131706+0000","last_clean":"2023-10-06T02:28:36.131706+0000","last_became_active":"2023-10-06T02:28:36.127167+0000","last_became_peered":"2023-10-06T02:28:36.127167+0000","last_unstale":"2023-10-06T02:28:36.131706+0000","last_undegraded":"2023-10-06T02:28:36.131706+0000","last_fullsized":"2023-10-06T02:28:36.131706+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:13:40.323328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.141210+0000","last_change":"2023-10-06T02:28:36.138217+0000","last_active":"2023-10-06T02:28:36.141210+0000","last_peered":"2023-10-06T02:28:36.141210+0000","last_clean":"2023-10-06T02:28:36.141210+0000","last_became_active":"2023-10-06T02:28:36.137916+0000","last_became_peered":"2023-10-06T02:28:36.137916+0000","last_unstale":"2023-10-06T02:28:36.141210+0000","last_undegraded":"2023-10-06T02:28:36.141210+0000","last_fullsized":"2023-10-06T02:28:36.141210+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:31:55.822221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.130056+0000","last_change":"2023-10-06T02:28:36.126856+0000","last_active":"2023-10-06T02:28:36.130056+0000","last_peered":"2023-10-06T02:28:36.130056+0000","last_clean":"2023-10-06T02:28:36.130056+0000","last_became_active":"2023-10-06T02:28:36.126532+0000","last_became_peered":"2023-10-06T02:28:36.126532+0000","last_unstale":"2023-10-06T02:28:36.130056+0000","last_undegraded":"2023-10-06T02:28:36.130056+0000","last_fullsized":"2023-10-06T02:28:36.130056+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:24:37.571928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.135394+0000","last_change":"2023-10-06T02:28:36.132201+0000","last_active":"2023-10-06T02:28:36.135394+0000","last_peered":"2023-10-06T02:28:36.135394+0000","last_clean":"2023-10-06T02:28:36.135394+0000","last_became_active":"2023-10-06T02:28:36.131914+0000","last_became_peered":"2023-10-06T02:28:36.131914+0000","last_unstale":"2023-10-06T02:28:36.135394+0000","last_undegraded":"2023-10-06T02:28:36.135394+0000","last_fullsized":"2023-10-06T02:28:36.135394+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:27:29.952942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.139187+0000","last_change":"2023-10-06T02:28:36.135029+0000","last_active":"2023-10-06T02:28:36.139187+0000","last_peered":"2023-10-06T02:28:36.139187+0000","last_clean":"2023-10-06T02:28:36.139187+0000","last_became_active":"2023-10-06T02:28:36.134717+0000","last_became_peered":"2023-10-06T02:28:36.134717+0000","last_unstale":"2023-10-06T02:28:36.139187+0000","last_undegraded":"2023-10-06T02:28:36.139187+0000","last_fullsized":"2023-10-06T02:28:36.139187+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:58.951832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.141433+0000","last_change":"2023-10-06T02:28:36.136110+0000","last_active":"2023-10-06T02:28:36.141433+0000","last_peered":"2023-10-06T02:28:36.141433+0000","last_clean":"2023-10-06T02:28:36.141433+0000","last_became_active":"2023-10-06T02:28:36.135697+0000","last_became_peered":"2023-10-06T02:28:36.135697+0000","last_unstale":"2023-10-06T02:28:36.141433+0000","last_undegraded":"2023-10-06T02:28:36.141433+0000","last_fullsized":"2023-10-06T02:28:36.141433+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:05:34.791425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.143720+0000","last_change":"2023-10-06T02:28:36.142273+0000","last_active":"2023-10-06T02:28:36.143720+0000","last_peered":"2023-10-06T02:28:36.143720+0000","last_clean":"2023-10-06T02:28:36.143720+0000","last_became_active":"2023-10-06T02:28:36.141949+0000","last_became_peered":"2023-10-06T02:28:36.141949+0000","last_unstale":"2023-10-06T02:28:36.143720+0000","last_undegraded":"2023-10-06T02:28:36.143720+0000","last_fullsized":"2023-10-06T02:28:36.143720+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:47:27.628381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.121011+0000","last_change":"2023-10-06T02:28:36.118870+0000","last_active":"2023-10-06T02:28:36.121011+0000","last_peered":"2023-10-06T02:28:36.121011+0000","last_clean":"2023-10-06T02:28:36.121011+0000","last_became_active":"2023-10-06T02:28:36.118555+0000","last_became_peered":"2023-10-06T02:28:36.118555+0000","last_unstale":"2023-10-06T02:28:36.121011+0000","last_undegraded":"2023-10-06T02:28:36.121011+0000","last_fullsized":"2023-10-06T02:28:36.121011+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:51:57.859263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.140001+0000","last_change":"2023-10-06T02:28:36.135548+0000","last_active":"2023-10-06T02:28:36.140001+0000","last_peered":"2023-10-06T02:28:36.140001+0000","last_clean":"2023-10-06T02:28:36.140001+0000","last_became_active":"2023-10-06T02:28:36.134082+0000","last_became_peered":"2023-10-06T02:28:36.134082+0000","last_unstale":"2023-10-06T02:28:36.140001+0000","last_undegraded":"2023-10-06T02:28:36.140001+0000","last_fullsized":"2023-10-06T02:28:36.140001+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:20:12.262390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.125785+0000","last_change":"2023-10-06T02:28:36.123868+0000","last_active":"2023-10-06T02:28:36.125785+0000","last_peered":"2023-10-06T02:28:36.125785+0000","last_clean":"2023-10-06T02:28:36.125785+0000","last_became_active":"2023-10-06T02:28:36.123463+0000","last_became_peered":"2023-10-06T02:28:36.123463+0000","last_unstale":"2023-10-06T02:28:36.125785+0000","last_undegraded":"2023-10-06T02:28:36.125785+0000","last_fullsized":"2023-10-06T02:28:36.125785+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:07:11.178131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.122491+0000","last_change":"2023-10-06T02:28:36.115107+0000","last_active":"2023-10-06T02:28:36.122491+0000","last_peered":"2023-10-06T02:28:36.122491+0000","last_clean":"2023-10-06T02:28:36.122491+0000","last_became_active":"2023-10-06T02:28:36.114821+0000","last_became_peered":"2023-10-06T02:28:36.114821+0000","last_unstale":"2023-10-06T02:28:36.122491+0000","last_undegraded":"2023-10-06T02:28:36.122491+0000","last_fullsized":"2023-10-06T02:28:36.122491+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:13:42.549119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.141977+0000","last_change":"2023-10-06T02:28:36.136213+0000","last_active":"2023-10-06T02:28:36.141977+0000","last_peered":"2023-10-06T02:28:36.141977+0000","last_clean":"2023-10-06T02:28:36.141977+0000","last_became_active":"2023-10-06T02:28:36.135870+0000","last_became_peered":"2023-10-06T02:28:36.135870+0000","last_unstale":"2023-10-06T02:28:36.141977+0000","last_undegraded":"2023-10-06T02:28:36.141977+0000","last_fullsized":"2023-10-06T02:28:36.141977+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:53:15.976240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.141560+0000","last_change":"2023-10-06T02:28:36.137444+0000","last_active":"2023-10-06T02:28:36.141560+0000","last_peered":"2023-10-06T02:28:36.141560+0000","last_clean":"2023-10-06T02:28:36.141560+0000","last_became_active":"2023-10-06T02:28:36.137135+0000","last_became_peered":"2023-10-06T02:28:36.137135+0000","last_unstale":"2023-10-06T02:28:36.141560+0000","last_undegraded":"2023-10-06T02:28:36.141560+0000","last_fullsized":"2023-10-06T02:28:36.141560+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:24:22.081760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.121556+0000","last_change":"2023-10-06T02:28:36.114052+0000","last_active":"2023-10-06T02:28:36.121556+0000","last_peered":"2023-10-06T02:28:36.121556+0000","last_clean":"2023-10-06T02:28:36.121556+0000","last_became_active":"2023-10-06T02:28:36.113254+0000","last_became_peered":"2023-10-06T02:28:36.113254+0000","last_unstale":"2023-10-06T02:28:36.121556+0000","last_undegraded":"2023-10-06T02:28:36.121556+0000","last_fullsized":"2023-10-06T02:28:36.121556+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:25:08.177495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.142277+0000","last_change":"2023-10-06T02:28:36.136665+0000","last_active":"2023-10-06T02:28:36.142277+0000","last_peered":"2023-10-06T02:28:36.142277+0000","last_clean":"2023-10-06T02:28:36.142277+0000","last_became_active":"2023-10-06T02:28:36.136214+0000","last_became_peered":"2023-10-06T02:28:36.136214+0000","last_unstale":"2023-10-06T02:28:36.142277+0000","last_undegraded":"2023-10-06T02:28:36.142277+0000","last_fullsized":"2023-10-06T02:28:36.142277+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:21:33.506189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.127231+0000","last_change":"2023-10-06T02:28:36.119302+0000","last_active":"2023-10-06T02:28:36.127231+0000","last_peered":"2023-10-06T02:28:36.127231+0000","last_clean":"2023-10-06T02:28:36.127231+0000","last_became_active":"2023-10-06T02:28:36.118985+0000","last_became_peered":"2023-10-06T02:28:36.118985+0000","last_unstale":"2023-10-06T02:28:36.127231+0000","last_undegraded":"2023-10-06T02:28:36.127231+0000","last_fullsized":"2023-10-06T02:28:36.127231+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:57:01.855971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.139005+0000","last_change":"2023-10-06T02:28:36.137063+0000","last_active":"2023-10-06T02:28:36.139005+0000","last_peered":"2023-10-06T02:28:36.139005+0000","last_clean":"2023-10-06T02:28:36.139005+0000","last_became_active":"2023-10-06T02:28:36.136793+0000","last_became_peered":"2023-10-06T02:28:36.136793+0000","last_unstale":"2023-10-06T02:28:36.139005+0000","last_undegraded":"2023-10-06T02:28:36.139005+0000","last_fullsized":"2023-10-06T02:28:36.139005+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:19:39.586176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.125809+0000","last_change":"2023-10-06T02:28:36.124318+0000","last_active":"2023-10-06T02:28:36.125809+0000","last_peered":"2023-10-06T02:28:36.125809+0000","last_clean":"2023-10-06T02:28:36.125809+0000","last_became_active":"2023-10-06T02:28:36.124037+0000","last_became_peered":"2023-10-06T02:28:36.124037+0000","last_unstale":"2023-10-06T02:28:36.125809+0000","last_undegraded":"2023-10-06T02:28:36.125809+0000","last_fullsized":"2023-10-06T02:28:36.125809+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:15:38.814168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.129868+0000","last_change":"2023-10-06T02:28:36.127698+0000","last_active":"2023-10-06T02:28:36.129868+0000","last_peered":"2023-10-06T02:28:36.129868+0000","last_clean":"2023-10-06T02:28:36.129868+0000","last_became_active":"2023-10-06T02:28:36.127357+0000","last_became_peered":"2023-10-06T02:28:36.127357+0000","last_unstale":"2023-10-06T02:28:36.129868+0000","last_undegraded":"2023-10-06T02:28:36.129868+0000","last_fullsized":"2023-10-06T02:28:36.129868+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:49:59.118753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"105'4","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.140273+0000","last_change":"2023-10-06T02:28:35.124204+0000","last_active":"2023-10-06T02:28:36.140273+0000","last_peered":"2023-10-06T02:28:36.140273+0000","last_clean":"2023-10-06T02:28:36.140273+0000","last_became_active":"2023-10-06T02:28:35.123954+0000","last_became_peered":"2023-10-06T02:28:35.123954+0000","last_unstale":"2023-10-06T02:28:36.140273+0000","last_undegraded":"2023-10-06T02:28:36.140273+0000","last_fullsized":"2023-10-06T02:28:36.140273+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_clean_scrub_stamp":"2023-10-06T02:28:34.085717+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:35:05.456705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.125547+0000","last_change":"2023-10-06T02:28:36.122593+0000","last_active":"2023-10-06T02:28:36.125547+0000","last_peered":"2023-10-06T02:28:36.125547+0000","last_clean":"2023-10-06T02:28:36.125547+0000","last_became_active":"2023-10-06T02:28:36.122296+0000","last_became_peered":"2023-10-06T02:28:36.122296+0000","last_unstale":"2023-10-06T02:28:36.125547+0000","last_undegraded":"2023-10-06T02:28:36.125547+0000","last_fullsized":"2023-10-06T02:28:36.125547+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:36:07.845291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"105'1","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.136391+0000","last_change":"2023-10-06T02:28:35.121658+0000","last_active":"2023-10-06T02:28:36.136391+0000","last_peered":"2023-10-06T02:28:36.136391+0000","last_clean":"2023-10-06T02:28:36.136391+0000","last_became_active":"2023-10-06T02:28:35.121361+0000","last_became_peered":"2023-10-06T02:28:35.121361+0000","last_unstale":"2023-10-06T02:28:36.136391+0000","last_undegraded":"2023-10-06T02:28:36.136391+0000","last_fullsized":"2023-10-06T02:28:36.136391+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_clean_scrub_stamp":"2023-10-06T02:28:34.085717+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:28:40.056707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.122444+0000","last_change":"2023-10-06T02:28:36.112029+0000","last_active":"2023-10-06T02:28:36.122444+0000","last_peered":"2023-10-06T02:28:36.122444+0000","last_clean":"2023-10-06T02:28:36.122444+0000","last_became_active":"2023-10-06T02:28:36.111708+0000","last_became_peered":"2023-10-06T02:28:36.111708+0000","last_unstale":"2023-10-06T02:28:36.122444+0000","last_undegraded":"2023-10-06T02:28:36.122444+0000","last_fullsized":"2023-10-06T02:28:36.122444+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:11:06.828040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"105'2","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.141354+0000","last_change":"2023-10-06T02:28:35.123152+0000","last_active":"2023-10-06T02:28:36.141354+0000","last_peered":"2023-10-06T02:28:36.141354+0000","last_clean":"2023-10-06T02:28:36.141354+0000","last_became_active":"2023-10-06T02:28:35.122358+0000","last_became_peered":"2023-10-06T02:28:35.122358+0000","last_unstale":"2023-10-06T02:28:36.141354+0000","last_undegraded":"2023-10-06T02:28:36.141354+0000","last_fullsized":"2023-10-06T02:28:36.141354+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_clean_scrub_stamp":"2023-10-06T02:28:34.085717+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:30.317836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.143726+0000","last_change":"2023-10-06T02:28:36.142278+0000","last_active":"2023-10-06T02:28:36.143726+0000","last_peered":"2023-10-06T02:28:36.143726+0000","last_clean":"2023-10-06T02:28:36.143726+0000","last_became_active":"2023-10-06T02:28:36.141553+0000","last_became_peered":"2023-10-06T02:28:36.141553+0000","last_unstale":"2023-10-06T02:28:36.143726+0000","last_undegraded":"2023-10-06T02:28:36.143726+0000","last_fullsized":"2023-10-06T02:28:36.143726+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:57:01.855971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"105'2","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.118591+0000","last_change":"2023-10-06T02:28:35.122890+0000","last_active":"2023-10-06T02:28:36.118591+0000","last_peered":"2023-10-06T02:28:36.118591+0000","last_clean":"2023-10-06T02:28:36.118591+0000","last_became_active":"2023-10-06T02:28:35.122598+0000","last_became_peered":"2023-10-06T02:28:35.122598+0000","last_unstale":"2023-10-06T02:28:36.118591+0000","last_undegraded":"2023-10-06T02:28:36.118591+0000","last_fullsized":"2023-10-06T02:28:36.118591+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_clean_scrub_stamp":"2023-10-06T02:28:34.085717+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:53.130812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.135281+0000","last_change":"2023-10-06T02:28:36.130828+0000","last_active":"2023-10-06T02:28:36.135281+0000","last_peered":"2023-10-06T02:28:36.135281+0000","last_clean":"2023-10-06T02:28:36.135281+0000","last_became_active":"2023-10-06T02:28:36.130503+0000","last_became_peered":"2023-10-06T02:28:36.130503+0000","last_unstale":"2023-10-06T02:28:36.135281+0000","last_undegraded":"2023-10-06T02:28:36.135281+0000","last_fullsized":"2023-10-06T02:28:36.135281+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:49:30.199514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"105'1","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.126224+0000","last_change":"2023-10-06T02:28:35.120347+0000","last_active":"2023-10-06T02:28:36.126224+0000","last_peered":"2023-10-06T02:28:36.126224+0000","last_clean":"2023-10-06T02:28:36.126224+0000","last_became_active":"2023-10-06T02:28:35.119999+0000","last_became_peered":"2023-10-06T02:28:35.119999+0000","last_unstale":"2023-10-06T02:28:36.126224+0000","last_undegraded":"2023-10-06T02:28:36.126224+0000","last_fullsized":"2023-10-06T02:28:36.126224+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_clean_scrub_stamp":"2023-10-06T02:28:34.085717+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:10:45.309448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.132108+0000","last_change":"2023-10-06T02:28:36.130525+0000","last_active":"2023-10-06T02:28:36.132108+0000","last_peered":"2023-10-06T02:28:36.132108+0000","last_clean":"2023-10-06T02:28:36.132108+0000","last_became_active":"2023-10-06T02:28:36.130246+0000","last_became_peered":"2023-10-06T02:28:36.130246+0000","last_unstale":"2023-10-06T02:28:36.132108+0000","last_undegraded":"2023-10-06T02:28:36.132108+0000","last_fullsized":"2023-10-06T02:28:36.132108+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:49:18.264978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"105'1","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.133450+0000","last_change":"2023-10-06T02:28:35.114148+0000","last_active":"2023-10-06T02:28:36.133450+0000","last_peered":"2023-10-06T02:28:36.133450+0000","last_clean":"2023-10-06T02:28:36.133450+0000","last_became_active":"2023-10-06T02:28:35.113897+0000","last_became_peered":"2023-10-06T02:28:35.113897+0000","last_unstale":"2023-10-06T02:28:36.133450+0000","last_undegraded":"2023-10-06T02:28:36.133450+0000","last_fullsized":"2023-10-06T02:28:36.133450+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_clean_scrub_stamp":"2023-10-06T02:28:34.085717+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:35:48.970697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.128968+0000","last_change":"2023-10-06T02:28:36.120785+0000","last_active":"2023-10-06T02:28:36.128968+0000","last_peered":"2023-10-06T02:28:36.128968+0000","last_clean":"2023-10-06T02:28:36.128968+0000","last_became_active":"2023-10-06T02:28:36.120458+0000","last_became_peered":"2023-10-06T02:28:36.120458+0000","last_unstale":"2023-10-06T02:28:36.128968+0000","last_undegraded":"2023-10-06T02:28:36.128968+0000","last_fullsized":"2023-10-06T02:28:36.128968+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:07:02.108389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.106640+0000","last_change":"2023-10-06T02:28:35.120834+0000","last_active":"2023-10-06T02:28:36.106640+0000","last_peered":"2023-10-06T02:28:36.106640+0000","last_clean":"2023-10-06T02:28:36.106640+0000","last_became_active":"2023-10-06T02:28:35.120480+0000","last_became_peered":"2023-10-06T02:28:35.120480+0000","last_unstale":"2023-10-06T02:28:36.106640+0000","last_undegraded":"2023-10-06T02:28:36.106640+0000","last_fullsized":"2023-10-06T02:28:36.106640+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_clean_scrub_stamp":"2023-10-06T02:28:34.085717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:57.945376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.123836+0000","last_change":"2023-10-06T02:28:36.118333+0000","last_active":"2023-10-06T02:28:36.123836+0000","last_peered":"2023-10-06T02:28:36.123836+0000","last_clean":"2023-10-06T02:28:36.123836+0000","last_became_active":"2023-10-06T02:28:36.117977+0000","last_became_peered":"2023-10-06T02:28:36.117977+0000","last_unstale":"2023-10-06T02:28:36.123836+0000","last_undegraded":"2023-10-06T02:28:36.123836+0000","last_fullsized":"2023-10-06T02:28:36.123836+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:52:49.832902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.102870+0000","last_change":"2023-10-06T02:28:35.124806+0000","last_active":"2023-10-06T02:28:36.102870+0000","last_peered":"2023-10-06T02:28:36.102870+0000","last_clean":"2023-10-06T02:28:36.102870+0000","last_became_active":"2023-10-06T02:28:35.124377+0000","last_became_peered":"2023-10-06T02:28:35.124377+0000","last_unstale":"2023-10-06T02:28:36.102870+0000","last_undegraded":"2023-10-06T02:28:36.102870+0000","last_fullsized":"2023-10-06T02:28:36.102870+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_clean_scrub_stamp":"2023-10-06T02:28:34.085717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:47:02.217094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.127348+0000","last_change":"2023-10-06T02:28:36.125167+0000","last_active":"2023-10-06T02:28:36.127348+0000","last_peered":"2023-10-06T02:28:36.127348+0000","last_clean":"2023-10-06T02:28:36.127348+0000","last_became_active":"2023-10-06T02:28:36.124857+0000","last_became_peered":"2023-10-06T02:28:36.124857+0000","last_unstale":"2023-10-06T02:28:36.127348+0000","last_undegraded":"2023-10-06T02:28:36.127348+0000","last_fullsized":"2023-10-06T02:28:36.127348+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:17:21.845776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"105'1","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.128991+0000","last_change":"2023-10-06T02:28:35.123873+0000","last_active":"2023-10-06T02:28:36.128991+0000","last_peered":"2023-10-06T02:28:36.128991+0000","last_clean":"2023-10-06T02:28:36.128991+0000","last_became_active":"2023-10-06T02:28:35.123625+0000","last_became_peered":"2023-10-06T02:28:35.123625+0000","last_unstale":"2023-10-06T02:28:36.128991+0000","last_undegraded":"2023-10-06T02:28:36.128991+0000","last_fullsized":"2023-10-06T02:28:36.128991+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_clean_scrub_stamp":"2023-10-06T02:28:34.085717+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:26:51.054130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.129807+0000","last_change":"2023-10-06T02:28:36.125411+0000","last_active":"2023-10-06T02:28:36.129807+0000","last_peered":"2023-10-06T02:28:36.129807+0000","last_clean":"2023-10-06T02:28:36.129807+0000","last_became_active":"2023-10-06T02:28:36.125087+0000","last_became_peered":"2023-10-06T02:28:36.125087+0000","last_unstale":"2023-10-06T02:28:36.129807+0000","last_undegraded":"2023-10-06T02:28:36.129807+0000","last_fullsized":"2023-10-06T02:28:36.129807+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:02:37.295261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"105'2","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.135786+0000","last_change":"2023-10-06T02:28:35.112239+0000","last_active":"2023-10-06T02:28:36.135786+0000","last_peered":"2023-10-06T02:28:36.135786+0000","last_clean":"2023-10-06T02:28:36.135786+0000","last_became_active":"2023-10-06T02:28:35.111912+0000","last_became_peered":"2023-10-06T02:28:35.111912+0000","last_unstale":"2023-10-06T02:28:36.135786+0000","last_undegraded":"2023-10-06T02:28:36.135786+0000","last_fullsized":"2023-10-06T02:28:36.135786+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_clean_scrub_stamp":"2023-10-06T02:28:34.085717+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:09:19.976182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.127571+0000","last_change":"2023-10-06T02:28:36.124688+0000","last_active":"2023-10-06T02:28:36.127571+0000","last_peered":"2023-10-06T02:28:36.127571+0000","last_clean":"2023-10-06T02:28:36.127571+0000","last_became_active":"2023-10-06T02:28:36.124409+0000","last_became_peered":"2023-10-06T02:28:36.124409+0000","last_unstale":"2023-10-06T02:28:36.127571+0000","last_undegraded":"2023-10-06T02:28:36.127571+0000","last_fullsized":"2023-10-06T02:28:36.127571+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:20:09.089751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"105'1","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.139297+0000","last_change":"2023-10-06T02:28:35.123362+0000","last_active":"2023-10-06T02:28:36.139297+0000","last_peered":"2023-10-06T02:28:36.139297+0000","last_clean":"2023-10-06T02:28:36.139297+0000","last_became_active":"2023-10-06T02:28:35.122777+0000","last_became_peered":"2023-10-06T02:28:35.122777+0000","last_unstale":"2023-10-06T02:28:36.139297+0000","last_undegraded":"2023-10-06T02:28:36.139297+0000","last_fullsized":"2023-10-06T02:28:36.139297+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_clean_scrub_stamp":"2023-10-06T02:28:34.085717+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:17:20.839320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.135141+0000","last_change":"2023-10-06T02:28:36.131419+0000","last_active":"2023-10-06T02:28:36.135141+0000","last_peered":"2023-10-06T02:28:36.135141+0000","last_clean":"2023-10-06T02:28:36.135141+0000","last_became_active":"2023-10-06T02:28:36.130499+0000","last_became_peered":"2023-10-06T02:28:36.130499+0000","last_unstale":"2023-10-06T02:28:36.135141+0000","last_undegraded":"2023-10-06T02:28:36.135141+0000","last_fullsized":"2023-10-06T02:28:36.135141+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:31.324291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.117784+0000","last_change":"2023-10-06T02:28:35.127386+0000","last_active":"2023-10-06T02:28:36.117784+0000","last_peered":"2023-10-06T02:28:36.117784+0000","last_clean":"2023-10-06T02:28:36.117784+0000","last_became_active":"2023-10-06T02:28:35.127087+0000","last_became_peered":"2023-10-06T02:28:35.127087+0000","last_unstale":"2023-10-06T02:28:36.117784+0000","last_undegraded":"2023-10-06T02:28:36.117784+0000","last_fullsized":"2023-10-06T02:28:36.117784+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_clean_scrub_stamp":"2023-10-06T02:28:34.085717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:11:05.821584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.127655+0000","last_change":"2023-10-06T02:28:36.124057+0000","last_active":"2023-10-06T02:28:36.127655+0000","last_peered":"2023-10-06T02:28:36.127655+0000","last_clean":"2023-10-06T02:28:36.127655+0000","last_became_active":"2023-10-06T02:28:36.123778+0000","last_became_peered":"2023-10-06T02:28:36.123778+0000","last_unstale":"2023-10-06T02:28:36.127655+0000","last_undegraded":"2023-10-06T02:28:36.127655+0000","last_fullsized":"2023-10-06T02:28:36.127655+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:21:18.246108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"105'2","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.139778+0000","last_change":"2023-10-06T02:28:35.118025+0000","last_active":"2023-10-06T02:28:36.139778+0000","last_peered":"2023-10-06T02:28:36.139778+0000","last_clean":"2023-10-06T02:28:36.139778+0000","last_became_active":"2023-10-06T02:28:35.117742+0000","last_became_peered":"2023-10-06T02:28:35.117742+0000","last_unstale":"2023-10-06T02:28:36.139778+0000","last_undegraded":"2023-10-06T02:28:36.139778+0000","last_fullsized":"2023-10-06T02:28:36.139778+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_clean_scrub_stamp":"2023-10-06T02:28:34.085717+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:19:36.580800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.133232+0000","last_change":"2023-10-06T02:28:36.131688+0000","last_active":"2023-10-06T02:28:36.133232+0000","last_peered":"2023-10-06T02:28:36.133232+0000","last_clean":"2023-10-06T02:28:36.133232+0000","last_became_active":"2023-10-06T02:28:36.131348+0000","last_became_peered":"2023-10-06T02:28:36.131348+0000","last_unstale":"2023-10-06T02:28:36.133232+0000","last_undegraded":"2023-10-06T02:28:36.133232+0000","last_fullsized":"2023-10-06T02:28:36.133232+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:24:09.782655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.116556+0000","last_change":"2023-10-06T02:28:35.126060+0000","last_active":"2023-10-06T02:28:36.116556+0000","last_peered":"2023-10-06T02:28:36.116556+0000","last_clean":"2023-10-06T02:28:36.116556+0000","last_became_active":"2023-10-06T02:28:35.125709+0000","last_became_peered":"2023-10-06T02:28:35.125709+0000","last_unstale":"2023-10-06T02:28:36.116556+0000","last_undegraded":"2023-10-06T02:28:36.116556+0000","last_fullsized":"2023-10-06T02:28:36.116556+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_clean_scrub_stamp":"2023-10-06T02:28:34.085717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:07:10.171676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.128988+0000","last_change":"2023-10-06T02:28:36.124481+0000","last_active":"2023-10-06T02:28:36.128988+0000","last_peered":"2023-10-06T02:28:36.128988+0000","last_clean":"2023-10-06T02:28:36.128988+0000","last_became_active":"2023-10-06T02:28:36.124203+0000","last_became_peered":"2023-10-06T02:28:36.124203+0000","last_unstale":"2023-10-06T02:28:36.128988+0000","last_undegraded":"2023-10-06T02:28:36.128988+0000","last_fullsized":"2023-10-06T02:28:36.128988+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:45:18.729211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.115314+0000","last_change":"2023-10-06T02:28:35.117250+0000","last_active":"2023-10-06T02:28:36.115314+0000","last_peered":"2023-10-06T02:28:36.115314+0000","last_clean":"2023-10-06T02:28:36.115314+0000","last_became_active":"2023-10-06T02:28:35.116974+0000","last_became_peered":"2023-10-06T02:28:35.116974+0000","last_unstale":"2023-10-06T02:28:36.115314+0000","last_undegraded":"2023-10-06T02:28:36.115314+0000","last_fullsized":"2023-10-06T02:28:36.115314+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_clean_scrub_stamp":"2023-10-06T02:28:34.085717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:14:10.077126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.141832+0000","last_change":"2023-10-06T02:28:36.138189+0000","last_active":"2023-10-06T02:28:36.141832+0000","last_peered":"2023-10-06T02:28:36.141832+0000","last_clean":"2023-10-06T02:28:36.141832+0000","last_became_active":"2023-10-06T02:28:36.137653+0000","last_became_peered":"2023-10-06T02:28:36.137653+0000","last_unstale":"2023-10-06T02:28:36.141832+0000","last_undegraded":"2023-10-06T02:28:36.141832+0000","last_fullsized":"2023-10-06T02:28:36.141832+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:35:13.800349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.116469+0000","last_change":"2023-10-06T02:28:35.124397+0000","last_active":"2023-10-06T02:28:36.116469+0000","last_peered":"2023-10-06T02:28:36.116469+0000","last_clean":"2023-10-06T02:28:36.116469+0000","last_became_active":"2023-10-06T02:28:35.124019+0000","last_became_peered":"2023-10-06T02:28:35.124019+0000","last_unstale":"2023-10-06T02:28:36.116469+0000","last_undegraded":"2023-10-06T02:28:36.116469+0000","last_fullsized":"2023-10-06T02:28:36.116469+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_clean_scrub_stamp":"2023-10-06T02:28:34.085717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:13:41.542664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.122911+0000","last_change":"2023-10-06T02:28:36.121233+0000","last_active":"2023-10-06T02:28:36.122911+0000","last_peered":"2023-10-06T02:28:36.122911+0000","last_clean":"2023-10-06T02:28:36.122911+0000","last_became_active":"2023-10-06T02:28:36.120900+0000","last_became_peered":"2023-10-06T02:28:36.120900+0000","last_unstale":"2023-10-06T02:28:36.122911+0000","last_undegraded":"2023-10-06T02:28:36.122911+0000","last_fullsized":"2023-10-06T02:28:36.122911+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:12:06.980718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"105'2","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.127772+0000","last_change":"2023-10-06T02:28:35.115624+0000","last_active":"2023-10-06T02:28:36.127772+0000","last_peered":"2023-10-06T02:28:36.127772+0000","last_clean":"2023-10-06T02:28:36.127772+0000","last_became_active":"2023-10-06T02:28:35.115314+0000","last_became_peered":"2023-10-06T02:28:35.115314+0000","last_unstale":"2023-10-06T02:28:36.127772+0000","last_undegraded":"2023-10-06T02:28:36.127772+0000","last_fullsized":"2023-10-06T02:28:36.127772+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_clean_scrub_stamp":"2023-10-06T02:28:34.085717+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:56:03.936511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.132910+0000","last_change":"2023-10-06T02:28:36.129912+0000","last_active":"2023-10-06T02:28:36.132910+0000","last_peered":"2023-10-06T02:28:36.132910+0000","last_clean":"2023-10-06T02:28:36.132910+0000","last_became_active":"2023-10-06T02:28:36.129625+0000","last_became_peered":"2023-10-06T02:28:36.129625+0000","last_unstale":"2023-10-06T02:28:36.132910+0000","last_undegraded":"2023-10-06T02:28:36.132910+0000","last_fullsized":"2023-10-06T02:28:36.132910+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:42:57.532770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.114535+0000","last_change":"2023-10-06T02:28:35.123075+0000","last_active":"2023-10-06T02:28:36.114535+0000","last_peered":"2023-10-06T02:28:36.114535+0000","last_clean":"2023-10-06T02:28:36.114535+0000","last_became_active":"2023-10-06T02:28:35.122772+0000","last_became_peered":"2023-10-06T02:28:35.122772+0000","last_unstale":"2023-10-06T02:28:36.114535+0000","last_undegraded":"2023-10-06T02:28:36.114535+0000","last_fullsized":"2023-10-06T02:28:36.114535+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_clean_scrub_stamp":"2023-10-06T02:28:34.085717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:19:58.581969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.141540+0000","last_change":"2023-10-06T02:28:36.135759+0000","last_active":"2023-10-06T02:28:36.141540+0000","last_peered":"2023-10-06T02:28:36.141540+0000","last_clean":"2023-10-06T02:28:36.141540+0000","last_became_active":"2023-10-06T02:28:36.135296+0000","last_became_peered":"2023-10-06T02:28:36.135296+0000","last_unstale":"2023-10-06T02:28:36.141540+0000","last_undegraded":"2023-10-06T02:28:36.141540+0000","last_fullsized":"2023-10-06T02:28:36.141540+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:41:36.288950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.114134+0000","last_change":"2023-10-06T02:28:35.122002+0000","last_active":"2023-10-06T02:28:36.114134+0000","last_peered":"2023-10-06T02:28:36.114134+0000","last_clean":"2023-10-06T02:28:36.114134+0000","last_became_active":"2023-10-06T02:28:35.121697+0000","last_became_peered":"2023-10-06T02:28:35.121697+0000","last_unstale":"2023-10-06T02:28:36.114134+0000","last_undegraded":"2023-10-06T02:28:36.114134+0000","last_fullsized":"2023-10-06T02:28:36.114134+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_clean_scrub_stamp":"2023-10-06T02:28:34.085717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:31.463520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.142333+0000","last_change":"2023-10-06T02:28:36.136285+0000","last_active":"2023-10-06T02:28:36.142333+0000","last_peered":"2023-10-06T02:28:36.142333+0000","last_clean":"2023-10-06T02:28:36.142333+0000","last_became_active":"2023-10-06T02:28:36.135728+0000","last_became_peered":"2023-10-06T02:28:36.135728+0000","last_unstale":"2023-10-06T02:28:36.142333+0000","last_undegraded":"2023-10-06T02:28:36.142333+0000","last_fullsized":"2023-10-06T02:28:36.142333+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:17.145599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.103969+0000","last_change":"2023-10-06T02:28:35.125059+0000","last_active":"2023-10-06T02:28:36.103969+0000","last_peered":"2023-10-06T02:28:36.103969+0000","last_clean":"2023-10-06T02:28:36.103969+0000","last_became_active":"2023-10-06T02:28:35.124710+0000","last_became_peered":"2023-10-06T02:28:35.124710+0000","last_unstale":"2023-10-06T02:28:36.103969+0000","last_undegraded":"2023-10-06T02:28:36.103969+0000","last_fullsized":"2023-10-06T02:28:36.103969+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_clean_scrub_stamp":"2023-10-06T02:28:34.085717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:13:32.200644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.121481+0000","last_change":"2023-10-06T02:28:36.113150+0000","last_active":"2023-10-06T02:28:36.121481+0000","last_peered":"2023-10-06T02:28:36.121481+0000","last_clean":"2023-10-06T02:28:36.121481+0000","last_became_active":"2023-10-06T02:28:36.112700+0000","last_became_peered":"2023-10-06T02:28:36.112700+0000","last_unstale":"2023-10-06T02:28:36.121481+0000","last_undegraded":"2023-10-06T02:28:36.121481+0000","last_fullsized":"2023-10-06T02:28:36.121481+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:25:37.481624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.113540+0000","last_change":"2023-10-06T02:28:35.118883+0000","last_active":"2023-10-06T02:28:36.113540+0000","last_peered":"2023-10-06T02:28:36.113540+0000","last_clean":"2023-10-06T02:28:36.113540+0000","last_became_active":"2023-10-06T02:28:35.117757+0000","last_became_peered":"2023-10-06T02:28:35.117757+0000","last_unstale":"2023-10-06T02:28:36.113540+0000","last_undegraded":"2023-10-06T02:28:36.113540+0000","last_fullsized":"2023-10-06T02:28:36.113540+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_clean_scrub_stamp":"2023-10-06T02:28:34.085717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:49:29.193059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.133217+0000","last_change":"2023-10-06T02:28:36.130612+0000","last_active":"2023-10-06T02:28:36.133217+0000","last_peered":"2023-10-06T02:28:36.133217+0000","last_clean":"2023-10-06T02:28:36.133217+0000","last_became_active":"2023-10-06T02:28:36.130360+0000","last_became_peered":"2023-10-06T02:28:36.130360+0000","last_unstale":"2023-10-06T02:28:36.133217+0000","last_undegraded":"2023-10-06T02:28:36.133217+0000","last_fullsized":"2023-10-06T02:28:36.133217+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:27:57.215710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.113538+0000","last_change":"2023-10-06T02:28:35.119290+0000","last_active":"2023-10-06T02:28:36.113538+0000","last_peered":"2023-10-06T02:28:36.113538+0000","last_clean":"2023-10-06T02:28:36.113538+0000","last_became_active":"2023-10-06T02:28:35.118312+0000","last_became_peered":"2023-10-06T02:28:35.118312+0000","last_unstale":"2023-10-06T02:28:36.113538+0000","last_undegraded":"2023-10-06T02:28:36.113538+0000","last_fullsized":"2023-10-06T02:28:36.113538+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_clean_scrub_stamp":"2023-10-06T02:28:34.085717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:42:56.526314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.139111+0000","last_change":"2023-10-06T02:28:36.132396+0000","last_active":"2023-10-06T02:28:36.139111+0000","last_peered":"2023-10-06T02:28:36.139111+0000","last_clean":"2023-10-06T02:28:36.139111+0000","last_became_active":"2023-10-06T02:28:36.129422+0000","last_became_peered":"2023-10-06T02:28:36.129422+0000","last_unstale":"2023-10-06T02:28:36.139111+0000","last_undegraded":"2023-10-06T02:28:36.139111+0000","last_fullsized":"2023-10-06T02:28:36.139111+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:32:47.115243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.103036+0000","last_change":"2023-10-06T02:28:35.125258+0000","last_active":"2023-10-06T02:28:36.103036+0000","last_peered":"2023-10-06T02:28:36.103036+0000","last_clean":"2023-10-06T02:28:36.103036+0000","last_became_active":"2023-10-06T02:28:35.124958+0000","last_became_peered":"2023-10-06T02:28:35.124958+0000","last_unstale":"2023-10-06T02:28:36.103036+0000","last_undegraded":"2023-10-06T02:28:36.103036+0000","last_fullsized":"2023-10-06T02:28:36.103036+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_clean_scrub_stamp":"2023-10-06T02:28:34.085717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:41:00.494842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.143361+0000","last_change":"2023-10-06T02:28:36.139012+0000","last_active":"2023-10-06T02:28:36.143361+0000","last_peered":"2023-10-06T02:28:36.143361+0000","last_clean":"2023-10-06T02:28:36.143361+0000","last_became_active":"2023-10-06T02:28:36.138764+0000","last_became_peered":"2023-10-06T02:28:36.138764+0000","last_unstale":"2023-10-06T02:28:36.143361+0000","last_undegraded":"2023-10-06T02:28:36.143361+0000","last_fullsized":"2023-10-06T02:28:36.143361+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:10:46.315904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.116596+0000","last_change":"2023-10-06T02:28:35.126265+0000","last_active":"2023-10-06T02:28:36.116596+0000","last_peered":"2023-10-06T02:28:36.116596+0000","last_clean":"2023-10-06T02:28:36.116596+0000","last_became_active":"2023-10-06T02:28:35.125949+0000","last_became_peered":"2023-10-06T02:28:35.125949+0000","last_unstale":"2023-10-06T02:28:36.116596+0000","last_undegraded":"2023-10-06T02:28:36.116596+0000","last_fullsized":"2023-10-06T02:28:36.116596+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_clean_scrub_stamp":"2023-10-06T02:28:34.085717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:15:37.807712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.141863+0000","last_change":"2023-10-06T02:28:36.138386+0000","last_active":"2023-10-06T02:28:36.141863+0000","last_peered":"2023-10-06T02:28:36.141863+0000","last_clean":"2023-10-06T02:28:36.141863+0000","last_became_active":"2023-10-06T02:28:36.137818+0000","last_became_peered":"2023-10-06T02:28:36.137818+0000","last_unstale":"2023-10-06T02:28:36.141863+0000","last_undegraded":"2023-10-06T02:28:36.141863+0000","last_fullsized":"2023-10-06T02:28:36.141863+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:14:11.083582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"105'6","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.142389+0000","last_change":"2023-10-06T02:28:35.115928+0000","last_active":"2023-10-06T02:28:36.142389+0000","last_peered":"2023-10-06T02:28:36.142389+0000","last_clean":"2023-10-06T02:28:36.142389+0000","last_became_active":"2023-10-06T02:28:35.115671+0000","last_became_peered":"2023-10-06T02:28:35.115671+0000","last_unstale":"2023-10-06T02:28:36.142389+0000","last_undegraded":"2023-10-06T02:28:36.142389+0000","last_fullsized":"2023-10-06T02:28:36.142389+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_clean_scrub_stamp":"2023-10-06T02:28:34.085717+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:11:05.821584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":85,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.111320+0000","last_change":"2023-10-06T02:23:01.992935+0000","last_active":"2023-10-06T02:28:36.111320+0000","last_peered":"2023-10-06T02:28:36.111320+0000","last_clean":"2023-10-06T02:28:36.111320+0000","last_became_active":"2023-10-06T02:23:01.992568+0000","last_became_peered":"2023-10-06T02:23:01.992568+0000","last_unstale":"2023-10-06T02:28:36.111320+0000","last_undegraded":"2023-10-06T02:28:36.111320+0000","last_fullsized":"2023-10-06T02:28:36.111320+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:06.212251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.129592+0000","last_change":"2023-10-06T02:28:36.126882+0000","last_active":"2023-10-06T02:28:36.129592+0000","last_peered":"2023-10-06T02:28:36.129592+0000","last_clean":"2023-10-06T02:28:36.129592+0000","last_became_active":"2023-10-06T02:28:36.126421+0000","last_became_peered":"2023-10-06T02:28:36.126421+0000","last_unstale":"2023-10-06T02:28:36.129592+0000","last_undegraded":"2023-10-06T02:28:36.129592+0000","last_fullsized":"2023-10-06T02:28:36.129592+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:47:27.628381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"105'3","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.140033+0000","last_change":"2023-10-06T02:28:35.121119+0000","last_active":"2023-10-06T02:28:36.140033+0000","last_peered":"2023-10-06T02:28:36.140033+0000","last_clean":"2023-10-06T02:28:36.140033+0000","last_became_active":"2023-10-06T02:28:35.120722+0000","last_became_peered":"2023-10-06T02:28:35.120722+0000","last_unstale":"2023-10-06T02:28:36.140033+0000","last_undegraded":"2023-10-06T02:28:36.140033+0000","last_fullsized":"2023-10-06T02:28:36.140033+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_clean_scrub_stamp":"2023-10-06T02:28:34.085717+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:24:21.075304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":84,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.110896+0000","last_change":"2023-10-06T02:23:01.989934+0000","last_active":"2023-10-06T02:28:36.110896+0000","last_peered":"2023-10-06T02:28:36.110896+0000","last_clean":"2023-10-06T02:28:36.110896+0000","last_became_active":"2023-10-06T02:23:01.989706+0000","last_became_peered":"2023-10-06T02:23:01.989706+0000","last_unstale":"2023-10-06T02:28:36.110896+0000","last_undegraded":"2023-10-06T02:28:36.110896+0000","last_fullsized":"2023-10-06T02:28:36.110896+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:06.212251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.122903+0000","last_change":"2023-10-06T02:28:36.119124+0000","last_active":"2023-10-06T02:28:36.122903+0000","last_peered":"2023-10-06T02:28:36.122903+0000","last_clean":"2023-10-06T02:28:36.122903+0000","last_became_active":"2023-10-06T02:28:36.118851+0000","last_became_peered":"2023-10-06T02:28:36.118851+0000","last_unstale":"2023-10-06T02:28:36.122903+0000","last_undegraded":"2023-10-06T02:28:36.122903+0000","last_fullsized":"2023-10-06T02:28:36.122903+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:35:27.932373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.107046+0000","last_change":"2023-10-06T02:28:35.115976+0000","last_active":"2023-10-06T02:28:36.107046+0000","last_peered":"2023-10-06T02:28:36.107046+0000","last_clean":"2023-10-06T02:28:36.107046+0000","last_became_active":"2023-10-06T02:28:35.115702+0000","last_became_peered":"2023-10-06T02:28:35.115702+0000","last_unstale":"2023-10-06T02:28:36.107046+0000","last_undegraded":"2023-10-06T02:28:36.107046+0000","last_fullsized":"2023-10-06T02:28:36.107046+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_clean_scrub_stamp":"2023-10-06T02:28:34.085717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:04:43.736773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":80,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.123666+0000","last_change":"2023-10-06T02:23:02.054069+0000","last_active":"2023-10-06T02:28:36.123666+0000","last_peered":"2023-10-06T02:28:36.123666+0000","last_clean":"2023-10-06T02:28:36.123666+0000","last_became_active":"2023-10-06T02:23:02.053739+0000","last_became_peered":"2023-10-06T02:23:02.053739+0000","last_unstale":"2023-10-06T02:28:36.123666+0000","last_undegraded":"2023-10-06T02:28:36.123666+0000","last_fullsized":"2023-10-06T02:28:36.123666+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:57:48.822608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.136768+0000","last_change":"2023-10-06T02:28:36.132802+0000","last_active":"2023-10-06T02:28:36.136768+0000","last_peered":"2023-10-06T02:28:36.136768+0000","last_clean":"2023-10-06T02:28:36.136768+0000","last_became_active":"2023-10-06T02:28:36.132465+0000","last_became_peered":"2023-10-06T02:28:36.132465+0000","last_unstale":"2023-10-06T02:28:36.136768+0000","last_undegraded":"2023-10-06T02:28:36.136768+0000","last_fullsized":"2023-10-06T02:28:36.136768+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:18:00.535461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"105'2","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.137026+0000","last_change":"2023-10-06T02:28:35.117421+0000","last_active":"2023-10-06T02:28:36.137026+0000","last_peered":"2023-10-06T02:28:36.137026+0000","last_clean":"2023-10-06T02:28:36.137026+0000","last_became_active":"2023-10-06T02:28:35.117166+0000","last_became_peered":"2023-10-06T02:28:35.117166+0000","last_unstale":"2023-10-06T02:28:36.137026+0000","last_undegraded":"2023-10-06T02:28:36.137026+0000","last_fullsized":"2023-10-06T02:28:36.137026+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_clean_scrub_stamp":"2023-10-06T02:28:34.085717+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:17:20.839320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":80,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.116696+0000","last_change":"2023-10-06T02:23:01.990773+0000","last_active":"2023-10-06T02:28:36.116696+0000","last_peered":"2023-10-06T02:28:36.116696+0000","last_clean":"2023-10-06T02:28:36.116696+0000","last_became_active":"2023-10-06T02:23:01.990511+0000","last_became_peered":"2023-10-06T02:23:01.990511+0000","last_unstale":"2023-10-06T02:28:36.116696+0000","last_undegraded":"2023-10-06T02:28:36.116696+0000","last_fullsized":"2023-10-06T02:28:36.116696+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:42:54.797731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.142856+0000","last_change":"2023-10-06T02:28:36.141958+0000","last_active":"2023-10-06T02:28:36.142856+0000","last_peered":"2023-10-06T02:28:36.142856+0000","last_clean":"2023-10-06T02:28:36.142856+0000","last_became_active":"2023-10-06T02:28:36.141685+0000","last_became_peered":"2023-10-06T02:28:36.141685+0000","last_unstale":"2023-10-06T02:28:36.142856+0000","last_undegraded":"2023-10-06T02:28:36.142856+0000","last_fullsized":"2023-10-06T02:28:36.142856+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:24:09.782655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"105'8","reported_seq":18,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.141109+0000","last_change":"2023-10-06T02:28:35.118111+0000","last_active":"2023-10-06T02:28:36.141109+0000","last_peered":"2023-10-06T02:28:36.141109+0000","last_clean":"2023-10-06T02:28:36.141109+0000","last_became_active":"2023-10-06T02:28:35.117854+0000","last_became_peered":"2023-10-06T02:28:35.117854+0000","last_unstale":"2023-10-06T02:28:36.141109+0000","last_undegraded":"2023-10-06T02:28:36.141109+0000","last_fullsized":"2023-10-06T02:28:36.141109+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_clean_scrub_stamp":"2023-10-06T02:28:34.085717+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:15:37.807712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"76'128","reported_seq":272,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.121250+0000","last_change":"2023-10-06T02:22:55.032519+0000","last_active":"2023-10-06T02:28:36.121250+0000","last_peered":"2023-10-06T02:28:36.121250+0000","last_clean":"2023-10-06T02:28:36.121250+0000","last_became_active":"2023-10-06T02:22:55.031367+0000","last_became_peered":"2023-10-06T02:22:55.031367+0000","last_unstale":"2023-10-06T02:28:36.121250+0000","last_undegraded":"2023-10-06T02:28:36.121250+0000","last_fullsized":"2023-10-06T02:28:36.121250+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:22:53.309097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:22:53.309097+0000","last_clean_scrub_stamp":"2023-10-06T02:22:53.309097+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:53:46.520909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":96,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.106684+0000","last_change":"2023-10-06T02:23:01.988140+0000","last_active":"2023-10-06T02:28:36.106684+0000","last_peered":"2023-10-06T02:28:36.106684+0000","last_clean":"2023-10-06T02:28:36.106684+0000","last_became_active":"2023-10-06T02:23:01.987769+0000","last_became_peered":"2023-10-06T02:23:01.987769+0000","last_unstale":"2023-10-06T02:28:36.106684+0000","last_undegraded":"2023-10-06T02:28:36.106684+0000","last_fullsized":"2023-10-06T02:28:36.106684+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:20:08.391912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.143332+0000","last_change":"2023-10-06T02:28:36.138642+0000","last_active":"2023-10-06T02:28:36.143332+0000","last_peered":"2023-10-06T02:28:36.143332+0000","last_clean":"2023-10-06T02:28:36.143332+0000","last_became_active":"2023-10-06T02:28:36.138343+0000","last_became_peered":"2023-10-06T02:28:36.138343+0000","last_unstale":"2023-10-06T02:28:36.143332+0000","last_undegraded":"2023-10-06T02:28:36.143332+0000","last_fullsized":"2023-10-06T02:28:36.143332+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:12:30.500263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.115067+0000","last_change":"2023-10-06T02:28:35.121804+0000","last_active":"2023-10-06T02:28:36.115067+0000","last_peered":"2023-10-06T02:28:36.115067+0000","last_clean":"2023-10-06T02:28:36.115067+0000","last_became_active":"2023-10-06T02:28:35.121523+0000","last_became_peered":"2023-10-06T02:28:35.121523+0000","last_unstale":"2023-10-06T02:28:36.115067+0000","last_undegraded":"2023-10-06T02:28:36.115067+0000","last_fullsized":"2023-10-06T02:28:36.115067+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_clean_scrub_stamp":"2023-10-06T02:28:34.085717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:35:12.793893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":89,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.117612+0000","last_change":"2023-10-06T02:23:02.054338+0000","last_active":"2023-10-06T02:28:36.117612+0000","last_peered":"2023-10-06T02:28:36.117612+0000","last_clean":"2023-10-06T02:28:36.117612+0000","last_became_active":"2023-10-06T02:23:02.053991+0000","last_became_peered":"2023-10-06T02:23:02.053991+0000","last_unstale":"2023-10-06T02:28:36.117612+0000","last_undegraded":"2023-10-06T02:28:36.117612+0000","last_fullsized":"2023-10-06T02:28:36.117612+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:39:30.192838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.142287+0000","last_change":"2023-10-06T02:28:36.139132+0000","last_active":"2023-10-06T02:28:36.142287+0000","last_peered":"2023-10-06T02:28:36.142287+0000","last_clean":"2023-10-06T02:28:36.142287+0000","last_became_active":"2023-10-06T02:28:36.138866+0000","last_became_peered":"2023-10-06T02:28:36.138866+0000","last_unstale":"2023-10-06T02:28:36.142287+0000","last_undegraded":"2023-10-06T02:28:36.142287+0000","last_fullsized":"2023-10-06T02:28:36.142287+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:53:16.708643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.115220+0000","last_change":"2023-10-06T02:28:35.123185+0000","last_active":"2023-10-06T02:28:36.115220+0000","last_peered":"2023-10-06T02:28:36.115220+0000","last_clean":"2023-10-06T02:28:36.115220+0000","last_became_active":"2023-10-06T02:28:35.122832+0000","last_became_peered":"2023-10-06T02:28:35.122832+0000","last_unstale":"2023-10-06T02:28:36.115220+0000","last_undegraded":"2023-10-06T02:28:36.115220+0000","last_fullsized":"2023-10-06T02:28:36.115220+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_clean_scrub_stamp":"2023-10-06T02:28:34.085717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:07:10.171676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":96,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.106795+0000","last_change":"2023-10-06T02:23:01.988631+0000","last_active":"2023-10-06T02:28:36.106795+0000","last_peered":"2023-10-06T02:28:36.106795+0000","last_clean":"2023-10-06T02:28:36.106795+0000","last_became_active":"2023-10-06T02:23:01.988303+0000","last_became_peered":"2023-10-06T02:23:01.988303+0000","last_unstale":"2023-10-06T02:28:36.106795+0000","last_undegraded":"2023-10-06T02:28:36.106795+0000","last_fullsized":"2023-10-06T02:28:36.106795+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:38.211466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.136813+0000","last_change":"2023-10-06T02:28:36.132942+0000","last_active":"2023-10-06T02:28:36.136813+0000","last_peered":"2023-10-06T02:28:36.136813+0000","last_clean":"2023-10-06T02:28:36.136813+0000","last_became_active":"2023-10-06T02:28:36.132134+0000","last_became_peered":"2023-10-06T02:28:36.132134+0000","last_unstale":"2023-10-06T02:28:36.136813+0000","last_undegraded":"2023-10-06T02:28:36.136813+0000","last_fullsized":"2023-10-06T02:28:36.136813+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:35.092173+0000","last_clean_scrub_stamp":"2023-10-06T02:28:35.092173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:04:44.743228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.111231+0000","last_change":"2023-10-06T02:28:35.122267+0000","last_active":"2023-10-06T02:28:36.111231+0000","last_peered":"2023-10-06T02:28:36.111231+0000","last_clean":"2023-10-06T02:28:36.111231+0000","last_became_active":"2023-10-06T02:28:35.121743+0000","last_became_peered":"2023-10-06T02:28:35.121743+0000","last_unstale":"2023-10-06T02:28:36.111231+0000","last_undegraded":"2023-10-06T02:28:36.111231+0000","last_fullsized":"2023-10-06T02:28:36.111231+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:28:34.085717+0000","last_clean_scrub_stamp":"2023-10-06T02:28:34.085717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:26:51.054130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":84,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T02:28:36.107653+0000","last_change":"2023-10-06T02:23:02.054039+0000","last_active":"2023-10-06T02:28:36.107653+0000","last_peered":"2023-10-06T02:28:36.107653+0000","last_clean":"2023-10-06T02:28:36.107653+0000","last_became_active":"2023-10-06T02:23:02.053742+0000","last_became_peered":"2023-10-06T02:23:02.053742+0000","last_unstale":"2023-10-06T02:28:36.107653+0000","last_undegraded":"2023-10-06T02:28:36.107653+0000","last_fullsized":"2023-10-06T02:28:36.107653+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:58:18.846475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574922,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303268,"kb_used_data":1884,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068572,"statfs":{"total":96636764160,"available":96326217728,"internally_reserved":0,"allocated":1929216,"data_stored":1201404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64200000000000002,"15min":0.63800000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63200000000000001,"15min":0.60899999999999999},"min":{"1min":0.379,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.56999999999999995}]},{"osd":1,"last update":"Fri Oct 6 02:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.71399999999999997,"15min":0.68600000000000005},"min":{"1min":0.48899999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.052,"5min":1.147,"15min":1.147},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.67700000000000005,"15min":0.67300000000000004},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.79300000000000004,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.59799999999999998}]},{"osd":2,"last update":"Fri Oct 6 02:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.68799999999999994,"15min":0.69299999999999995},"min":{"1min":0.46899999999999997,"5min":0.375,"15min":0.375},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68500000000000005,"15min":0.68899999999999995},"min":{"1min":0.46999999999999997,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.94499999999999995,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.64200000000000002}]},{"osd":3,"last update":"Fri Oct 6 02:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.749,"15min":0.73299999999999998},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.77200000000000002,"15min":0.77600000000000002},"min":{"1min":0.54500000000000004,"5min":0.497,"15min":0.497},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.74099999999999999}]},{"osd":4,"last update":"Fri Oct 6 02:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.63,"15min":0.68899999999999995},"min":{"1min":0.432,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.002,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.71399999999999997,"15min":0.76600000000000001},"min":{"1min":0.47199999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.51,"5min":1.51,"15min":1.51},"last":0.53200000000000003}]},{"osd":5,"last update":"Fri Oct 6 02:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.70799999999999996,"15min":0.73299999999999998},"min":{"1min":0.443,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70599999999999996,"15min":0.73199999999999998},"min":{"1min":0.48699999999999999,"5min":0.442,"15min":0.442},"max":{"1min":0.98099999999999998,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.55300000000000005}]},{"osd":6,"last update":"Fri Oct 6 02:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.77500000000000002,"15min":0.78700000000000003},"min":{"1min":0.60499999999999998,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.626},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.81299999999999994,"15min":0.81799999999999995},"min":{"1min":0.64900000000000002,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.6020000000000001,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.73199999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574922,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302676,"kb_used_data":1292,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069164,"statfs":{"total":96636764160,"available":96326823936,"internally_reserved":0,"allocated":1323008,"data_stored":610842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:28:05 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67300000000000004,"15min":0.66500000000000004},"min":{"1min":0.502,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.84399999999999997,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.71699999999999997,"15min":0.69799999999999995},"min":{"1min":0.51800000000000002,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0169999999999999,"5min":1.119,"15min":1.119},"last":0.81899999999999995}]},{"osd":1,"last update":"Fri Oct 6 02:28:05 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.64300000000000002,"15min":0.64500000000000002},"min":{"1min":0.45100000000000001,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.67800000000000005,"15min":0.66800000000000004},"min":{"1min":0.47399999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.66600000000000004}]},{"osd":2,"last update":"Fri Oct 6 02:28:05 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.71899999999999997,"15min":0.70499999999999996},"min":{"1min":0.54600000000000004,"5min":0.437,"15min":0.437},"max":{"1min":1.0369999999999999,"5min":1.137,"15min":1.137},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.72699999999999998,"15min":0.71099999999999997},"min":{"1min":0.45300000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.91300000000000003,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.97999999999999998}]},{"osd":3,"last update":"Fri Oct 6 02:28:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.69499999999999995,"15min":0.66400000000000003},"min":{"1min":0.55700000000000005,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.72099999999999997,"15min":0.70499999999999996},"min":{"1min":0.60399999999999998,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.92800000000000005,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.84499999999999997}]},{"osd":4,"last update":"Fri Oct 6 02:28:05 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.65900000000000003,"15min":0.64100000000000001},"min":{"1min":0.52300000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.94799999999999995,"5min":1.095,"15min":1.095},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.69799999999999995,"15min":0.66300000000000003},"min":{"1min":0.433,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.002,"5min":1.29,"15min":1.29},"last":0.92800000000000005}]},{"osd":5,"last update":"Fri Oct 6 02:28:05 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.69299999999999995,"15min":0.70099999999999996},"min":{"1min":0.50700000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.94999999999999996,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.754,"5min":0.71699999999999997,"15min":0.70499999999999996},"min":{"1min":0.53900000000000003,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.94299999999999995,"5min":1.02,"15min":1.02},"last":0.71499999999999997}]},{"osd":7,"last update":"Fri Oct 6 02:28:05 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.746,"15min":0.748},"min":{"1min":0.57499999999999996,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.96899999999999997,"5min":1.458,"15min":1.458},"last":0.88},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.76400000000000001,"15min":0.76000000000000001},"min":{"1min":0.59099999999999997,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.98299999999999998,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.94899999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574921,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302676,"kb_used_data":1292,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069164,"statfs":{"total":96636764160,"available":96326823936,"internally_reserved":0,"allocated":1323008,"data_stored":610310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.442,"15min":0.46000000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.58299999999999996,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.307},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.48799999999999999,"15min":0.52700000000000002},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.70099999999999996,"5min":1.788,"15min":1.788},"last":0.35699999999999998}]},{"osd":2,"last update":"Fri Oct 6 02:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.46700000000000003,"15min":0.52300000000000002},"min":{"1min":0.307,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.60899999999999999,"5min":2.0840000000000001,"15min":2.0840000000000001},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.52100000000000002,"15min":0.55800000000000005},"min":{"1min":0.36899999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.66400000000000003,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.41099999999999998}]},{"osd":3,"last update":"Fri Oct 6 02:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60799999999999998,"15min":0.61699999999999999},"min":{"1min":0.46000000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.84999999999999998,"5min":1.266,"15min":1.266},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.56699999999999995,"15min":0.63900000000000001},"min":{"1min":0.41099999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.79400000000000004,"5min":1.861,"15min":1.861},"last":0.46899999999999997}]},{"osd":4,"last update":"Fri Oct 6 02:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.80700000000000005,"15min":0.85699999999999998},"min":{"1min":0.59099999999999997,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.3580000000000001,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.81499999999999995,"15min":0.85799999999999998},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.4339999999999999,"5min":2.0179999999999998,"15min":2.0179999999999998},"last":0.747}]},{"osd":5,"last update":"Fri Oct 6 02:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.77600000000000002,"15min":0.85799999999999998},"min":{"1min":0.52600000000000002,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.1040000000000001,"5min":1.9830000000000001,"15min":1.9830000000000001},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.77400000000000002,"15min":0.82199999999999995},"min":{"1min":0.58899999999999997,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.161,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.75900000000000001}]},{"osd":6,"last update":"Fri Oct 6 02:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.81799999999999995,"15min":0.86699999999999999},"min":{"1min":0.52700000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0609999999999999,"5min":1.829,"15min":1.829},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.84399999999999997,"15min":0.91200000000000003},"min":{"1min":0.64600000000000002,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.282,"5min":2.052,"15min":2.052},"last":0.70799999999999996}]},{"osd":7,"last update":"Fri Oct 6 02:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.80000000000000004,"15min":0.83699999999999997},"min":{"1min":0.56299999999999994,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.3120000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.81699999999999995,"15min":0.879},"min":{"1min":0.55000000000000004,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.2110000000000001,"5min":1.895,"15min":1.895},"last":0.623}]}]},{"osd":0,"up_from":13,"seq":55834574922,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303256,"kb_used_data":1872,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068584,"statfs":{"total":96636764160,"available":96326230016,"internally_reserved":0,"allocated":1916928,"data_stored":1200678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 02:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.434,"15min":0.45400000000000001},"min":{"1min":0.28899999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.59099999999999997,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.46100000000000002,"15min":0.47699999999999998},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.57299999999999995,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.47199999999999998}]},{"osd":2,"last update":"Fri Oct 6 02:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.49299999999999999,"15min":0.51200000000000001},"min":{"1min":0.35999999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.67400000000000004,"5min":1.069,"15min":1.069},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.501,"15min":0.51900000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.67000000000000004,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.51900000000000002}]},{"osd":3,"last update":"Fri Oct 6 02:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.48599999999999999,"15min":0.503},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.60399999999999998,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.58699999999999997,"15min":0.59499999999999997},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.68899999999999995,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.622}]},{"osd":4,"last update":"Fri Oct 6 02:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.751,"15min":0.79700000000000004},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.94199999999999995,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.873},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.80000000000000004},"min":{"1min":0.51000000000000001,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.085,"5min":1.304,"15min":1.304},"last":0.79200000000000004}]},{"osd":5,"last update":"Fri Oct 6 02:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.76100000000000001,"15min":0.78600000000000003},"min":{"1min":0.58499999999999996,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78500000000000003,"15min":0.80100000000000005},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.998,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.78300000000000003}]},{"osd":6,"last update":"Fri Oct 6 02:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.80200000000000005,"15min":0.82199999999999995},"min":{"1min":0.60499999999999998,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.0089999999999999,"5min":1.167,"15min":1.167},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.78900000000000003,"15min":0.82899999999999996},"min":{"1min":0.61399999999999999,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.84099999999999997}]},{"osd":7,"last update":"Fri Oct 6 02:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79600000000000004,"15min":0.82599999999999996},"min":{"1min":0.61199999999999999,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.244,"5min":1.244,"15min":1.244},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.748,"5min":0.77600000000000002,"15min":0.82199999999999995},"min":{"1min":0.57599999999999996,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.016,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.88800000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574923,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302672,"kb_used_data":1288,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069168,"statfs":{"total":96636764160,"available":96326828032,"internally_reserved":0,"allocated":1318912,"data_stored":610288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:28:07 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.42899999999999999,"15min":0.439},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.47599999999999998,"15min":0.48599999999999999},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.80200000000000005,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.35999999999999999}]},{"osd":1,"last update":"Fri Oct 6 02:28:07 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.48999999999999999,"15min":0.504},"min":{"1min":0.35399999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.59199999999999997,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.499},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.55100000000000005,"15min":0.60899999999999999},"min":{"1min":0.40300000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.778,"5min":1.778,"15min":1.778},"last":0.47299999999999998}]},{"osd":3,"last update":"Fri Oct 6 02:28:07 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.51300000000000001,"15min":0.53300000000000003},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.77000000000000002,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.59499999999999997,"15min":0.62},"min":{"1min":0.46500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.81999999999999995,"5min":1.028,"15min":1.028},"last":0.60099999999999998}]},{"osd":4,"last update":"Fri Oct 6 02:28:07 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.752,"15min":0.77700000000000002},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0149999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.80800000000000005,"15min":0.84999999999999998},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.907,"5min":1.907,"15min":1.907},"last":0.58099999999999996}]},{"osd":5,"last update":"Fri Oct 6 02:28:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.752,"15min":0.80000000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.109,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.77600000000000002,"15min":0.82099999999999995},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":0.90700000000000003,"5min":1.258,"15min":1.258},"last":0.752}]},{"osd":6,"last update":"Fri Oct 6 02:28:07 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.80800000000000005,"15min":0.84899999999999998},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.827,"5min":1.827,"15min":1.827},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.77200000000000002,"15min":0.79400000000000004},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.91900000000000004,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.73299999999999998}]},{"osd":7,"last update":"Fri Oct 6 02:28:07 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.82899999999999996,"15min":0.88400000000000001},"min":{"1min":0.5,"5min":0.499,"15min":0.499},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.746,"5min":0.79000000000000004,"15min":0.82999999999999996},"min":{"1min":0.55300000000000005,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.0600000000000001,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.871}]}]},{"osd":3,"up_from":13,"seq":55834574920,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302688,"kb_used_data":1304,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069152,"statfs":{"total":96636764160,"available":96326811648,"internally_reserved":0,"allocated":1335296,"data_stored":611858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:28:05 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.47599999999999998,"15min":0.51200000000000001},"min":{"1min":0.32400000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":1.0489999999999999,"5min":2.024,"15min":2.024},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.502,"15min":0.52500000000000002},"min":{"1min":0.371,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.622,"5min":2.181,"15min":2.181},"last":0.45500000000000002}]},{"osd":1,"last update":"Fri Oct 6 02:28:05 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.52800000000000002,"15min":0.68300000000000005},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.66100000000000003,"5min":4.492,"15min":4.492},"last":0.316},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.58899999999999997,"15min":0.72899999999999998},"min":{"1min":0.39600000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.079,"5min":4.569,"15min":4.569},"last":0.47599999999999998}]},{"osd":2,"last update":"Fri Oct 6 02:28:05 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.64700000000000002,"15min":0.66900000000000004},"min":{"1min":0.47199999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.024,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61199999999999999,"15min":0.65500000000000003},"min":{"1min":0.40999999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.92100000000000004,"5min":2.4260000000000002,"15min":2.4260000000000002},"last":0.499}]},{"osd":4,"last update":"Fri Oct 6 02:28:05 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.76200000000000001,"15min":0.83099999999999996},"min":{"1min":0.56499999999999995,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.0149999999999999,"5min":2.214,"15min":2.214},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.77200000000000002,"15min":0.81200000000000006},"min":{"1min":0.56899999999999995,"5min":0.5,"15min":0.5},"max":{"1min":1.0389999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.68700000000000006}]},{"osd":5,"last update":"Fri Oct 6 02:28:05 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.79000000000000004,"15min":0.82399999999999995},"min":{"1min":0.60399999999999998,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.99099999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78800000000000003,"15min":0.85799999999999998},"min":{"1min":0.59699999999999998,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.0580000000000001,"5min":2.242,"15min":2.242},"last":0.54800000000000004}]},{"osd":6,"last update":"Fri Oct 6 02:28:05 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78100000000000003,"15min":0.82099999999999995},"min":{"1min":0.58699999999999997,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.0489999999999999,"5min":1.357,"15min":1.357},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.81100000000000005,"15min":0.83499999999999996},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.016,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.751}]},{"osd":7,"last update":"Fri Oct 6 02:28:05 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.81399999999999995,"15min":0.86399999999999999},"min":{"1min":0.61599999999999999,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.94699999999999995,"5min":2.27,"15min":2.27},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81899999999999995,"15min":0.85999999999999999},"min":{"1min":0.57999999999999996,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.0760000000000001,"5min":1.325,"15min":1.325},"last":0.71399999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574922,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302684,"kb_used_data":1300,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069156,"statfs":{"total":96636764160,"available":96326815744,"internally_reserved":0,"allocated":1331200,"data_stored":611824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:28:05 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.76200000000000001,"15min":1.02},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.86699999999999999,"5min":12.337,"15min":12.337},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.625,"5min":0.73499999999999999,"15min":0.92500000000000004},"min":{"1min":0.44400000000000001,"5min":0.438,"15min":0.438},"max":{"1min":1.3520000000000001,"5min":9.6539999999999999,"15min":9.6539999999999999},"last":0.79100000000000004}]},{"osd":1,"last update":"Fri Oct 6 02:28:05 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.68300000000000005,"15min":0.69999999999999996},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.99299999999999999,"5min":3.181,"15min":3.181},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.77500000000000002,"15min":1.0409999999999999},"min":{"1min":0.498,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.91800000000000004,"5min":12.946999999999999,"15min":12.946999999999999},"last":0.80200000000000005}]},{"osd":2,"last update":"Fri Oct 6 02:28:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.79800000000000004,"15min":1.0700000000000001},"min":{"1min":0.505,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.2989999999999999,"5min":12.377000000000001,"15min":12.377000000000001},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.75600000000000001,"15min":0.88},"min":{"1min":0.60599999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.97399999999999998,"5min":6.3220000000000001,"15min":6.3220000000000001},"last":0.75800000000000001}]},{"osd":3,"last update":"Fri Oct 6 02:28:05 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.79300000000000004,"15min":1.004},"min":{"1min":0.46200000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.5229999999999999,"5min":9.7720000000000002,"15min":9.7720000000000002},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.72399999999999998,"15min":0.77800000000000002},"min":{"1min":0.52400000000000002,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.94199999999999995,"5min":3.0840000000000001,"15min":3.0840000000000001},"last":0.72199999999999998}]},{"osd":5,"last update":"Fri Oct 6 02:28:05 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.71699999999999997,"15min":0.85399999999999998},"min":{"1min":0.504,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.02,"5min":6.2690000000000001,"15min":6.2690000000000001},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.754,"5min":0.78700000000000003,"15min":1.0549999999999999},"min":{"1min":0.54100000000000004,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.4359999999999999,"5min":10.769,"15min":10.769},"last":0.67000000000000004}]},{"osd":6,"last update":"Fri Oct 6 02:28:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.69599999999999995,"15min":0.76400000000000001},"min":{"1min":0.433,"5min":0.432,"15min":0.432},"max":{"1min":1.4590000000000001,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.80100000000000005,"15min":0.98499999999999999},"min":{"1min":0.5,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.3839999999999999,"5min":8.0899999999999999,"15min":8.0899999999999999},"last":0.69599999999999995}]},{"osd":7,"last update":"Fri Oct 6 02:28:05 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.84399999999999997,"15min":1.0860000000000001},"min":{"1min":0.59499999999999997,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.494,"5min":9.7089999999999996,"15min":9.7089999999999996},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.746,"5min":0.81599999999999995,"15min":1.01},"min":{"1min":0.52100000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.995,"5min":7.6719999999999997,"15min":7.6719999999999997},"last":0.77300000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574922,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302680,"kb_used_data":1296,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069160,"statfs":{"total":96636764160,"available":96326819840,"internally_reserved":0,"allocated":1327104,"data_stored":610448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:28:03 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.64400000000000002,"15min":0.626},"min":{"1min":0.47099999999999997,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.96999999999999997,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.66100000000000003,"15min":0.61199999999999999},"min":{"1min":0.44600000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.45100000000000001}]},{"osd":1,"last update":"Fri Oct 6 02:28:03 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.68200000000000005,"15min":0.65400000000000003},"min":{"1min":0.48599999999999999,"5min":0.434,"15min":0.434},"max":{"1min":1.0660000000000001,"5min":1.089,"15min":1.089},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.68700000000000006,"15min":0.67400000000000004},"min":{"1min":0.41999999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.504}]},{"osd":2,"last update":"Fri Oct 6 02:28:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.69899999999999995,"15min":0.66900000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.70099999999999996,"15min":0.68300000000000005},"min":{"1min":0.46100000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.54400000000000004}]},{"osd":3,"last update":"Fri Oct 6 02:28:03 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.75,"15min":0.72699999999999998},"min":{"1min":0.503,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.79300000000000004,"15min":0.76800000000000002},"min":{"1min":0.629,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.194,"5min":1.194,"15min":1.194},"last":0.85199999999999998}]},{"osd":4,"last update":"Fri Oct 6 02:28:03 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.66300000000000003,"15min":0.67300000000000004},"min":{"1min":0.48299999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.0089999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.67000000000000004,"15min":0.68200000000000005},"min":{"1min":0.40799999999999997,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.83699999999999997}]},{"osd":6,"last update":"Fri Oct 6 02:28:03 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.73499999999999999,"15min":0.74199999999999999},"min":{"1min":0.45900000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.97699999999999998,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81299999999999994,"15min":0.81999999999999995},"min":{"1min":0.56799999999999995,"5min":0.373,"15min":0.373},"max":{"1min":1.1200000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.98199999999999998}]},{"osd":7,"last update":"Fri Oct 6 02:28:03 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.76100000000000001,"15min":0.76400000000000001},"min":{"1min":0.498,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.81100000000000005,"15min":0.83299999999999996},"min":{"1min":0.51700000000000002,"5min":0.437,"15min":0.437},"max":{"1min":1.1040000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.88300000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T02:28:40.951 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T02:28:40.951 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T02:28:41.077 INFO:tasks.ceph.mds.a.smithi031.stderr:2023-10-06T02:28:41.075+0000 7fc9fc5f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T02:28:41.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.179+0000 7fc87899d700 1 Processor -- start 2023-10-06T02:28:41.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.179+0000 7fc87899d700 1 -- start start 2023-10-06T02:28:41.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.180+0000 7fc87899d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc87415dfd0 0x7fc8740aedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:41.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.180+0000 7fc87899d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc874159920 0x7fc8740ad090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:41.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.180+0000 7fc87899d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8740acc00 con 0x7fc874159550 2023-10-06T02:28:41.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.180+0000 7fc87899d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8740abff0 con 0x7fc87415dfd0 2023-10-06T02:28:41.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.180+0000 7fc87899d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8740ac2d0 con 0x7fc874159920 2023-10-06T02:28:41.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.180+0000 7fc872ffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc87415dfd0 0x7fc8740aedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:41.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.180+0000 7fc872ffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc87415dfd0 0x7fc8740aedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36788/0 (socket says 172.21.15.31:36788) 2023-10-06T02:28:41.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.181+0000 7fc872ffd700 1 -- 172.21.15.31:0/2950594141 learned_addr learned my addr 172.21.15.31:0/2950594141 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:41.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.181+0000 7fc873fff700 1 --2- 172.21.15.31:0/2950594141 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc874159920 0x7fc8740ad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:41.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.181+0000 7fc872ffd700 1 -- 172.21.15.31:0/2950594141 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc874159920 msgr2=0x7fc8740ad090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:41.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.181+0000 7fc872ffd700 1 --2- 172.21.15.31:0/2950594141 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc874159920 0x7fc8740ad090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:41.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.181+0000 7fc872ffd700 1 -- 172.21.15.31:0/2950594141 >> v1:172.21.15.62:6789/0 conn(0x7fc874159550 legacy=0x7fc8740ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:41.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.181+0000 7fc872ffd700 1 -- 172.21.15.31:0/2950594141 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8740ac5b0 con 0x7fc87415dfd0 2023-10-06T02:28:41.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.181+0000 7fc872ffd700 1 --2- 172.21.15.31:0/2950594141 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc87415dfd0 0x7fc8740aedb0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fc8680099b0 tx=0x7fc868009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2f2a7b885fb3d4a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:41.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.181+0000 7fc8727fc700 1 -- 172.21.15.31:0/2950594141 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc868009ea0 con 0x7fc87415dfd0 2023-10-06T02:28:41.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.182+0000 7fc8727fc700 1 -- 172.21.15.31:0/2950594141 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc86800b860 con 0x7fc87415dfd0 2023-10-06T02:28:41.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.182+0000 7fc8727fc700 1 -- 172.21.15.31:0/2950594141 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86800bc00 con 0x7fc87415dfd0 2023-10-06T02:28:41.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.182+0000 7fc87899d700 1 -- 172.21.15.31:0/2950594141 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc87415dfd0 msgr2=0x7fc8740aedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:41.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.182+0000 7fc87899d700 1 --2- 172.21.15.31:0/2950594141 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc87415dfd0 0x7fc8740aedb0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fc8680099b0 tx=0x7fc868009250 comp rx=0 tx=0).stop 2023-10-06T02:28:41.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.182+0000 7fc87899d700 1 -- 172.21.15.31:0/2950594141 shutdown_connections 2023-10-06T02:28:41.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.182+0000 7fc87899d700 1 --2- 172.21.15.31:0/2950594141 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc874159920 0x7fc8740ad090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:41.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.182+0000 7fc87899d700 1 --2- 172.21.15.31:0/2950594141 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc87415dfd0 0x7fc8740aedb0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:41.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.182+0000 7fc87899d700 1 -- 172.21.15.31:0/2950594141 >> 172.21.15.31:0/2950594141 conn(0x7fc87406ce80 msgr2=0x7fc8740ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:41.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.182+0000 7fc87899d700 1 -- 172.21.15.31:0/2950594141 shutdown_connections 2023-10-06T02:28:41.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.182+0000 7fc87899d700 1 -- 172.21.15.31:0/2950594141 wait complete. 2023-10-06T02:28:41.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.184+0000 7fc87899d700 1 Processor -- start 2023-10-06T02:28:41.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.184+0000 7fc87899d700 1 -- start start 2023-10-06T02:28:41.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.185+0000 7fc87899d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc874159550 0x7fc8740cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:41.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.185+0000 7fc87899d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc874159920 0x7fc874169ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:41.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.185+0000 7fc87899d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc87415dfd0 0x7fc87416a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:41.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.185+0000 7fc87899d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8740cc320 con 0x7fc874159550 2023-10-06T02:28:41.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.185+0000 7fc87899d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8740cb710 con 0x7fc874159920 2023-10-06T02:28:41.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.185+0000 7fc87899d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8740cb9f0 con 0x7fc87415dfd0 2023-10-06T02:28:41.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.185+0000 7fc8737fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc874159550 0x7fc8740cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:41.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.185+0000 7fc8737fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc874159550 0x7fc8740cc7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36798/0 (socket says 172.21.15.31:36798) 2023-10-06T02:28:41.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.185+0000 7fc8737fe700 1 -- 172.21.15.31:0/2669327477 learned_addr learned my addr 172.21.15.31:0/2669327477 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:41.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.185+0000 7fc872ffd700 1 --2- 172.21.15.31:0/2669327477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc874159920 0x7fc874169ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:41.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.185+0000 7fc873fff700 1 --2- 172.21.15.31:0/2669327477 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc87415dfd0 0x7fc87416a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:41.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.185+0000 7fc8737fe700 1 -- 172.21.15.31:0/2669327477 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc87415dfd0 msgr2=0x7fc87416a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:41.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.185+0000 7fc8737fe700 1 --2- 172.21.15.31:0/2669327477 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc87415dfd0 0x7fc87416a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:41.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.185+0000 7fc8737fe700 1 -- 172.21.15.31:0/2669327477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc874159920 msgr2=0x7fc874169ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:41.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.185+0000 7fc8737fe700 1 --2- 172.21.15.31:0/2669327477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc874159920 0x7fc874169ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:41.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.185+0000 7fc8737fe700 1 -- 172.21.15.31:0/2669327477 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8680094f0 con 0x7fc874159550 2023-10-06T02:28:41.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.185+0000 7fc872ffd700 1 --2- 172.21.15.31:0/2669327477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc874159920 0x7fc874169ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:28:41.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.185+0000 7fc873fff700 1 --2- 172.21.15.31:0/2669327477 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc87415dfd0 0x7fc87416a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:28:41.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.186+0000 7fc8737fe700 1 --2- 172.21.15.31:0/2669327477 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc874159550 0x7fc8740cc7b0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fc86400c590 tx=0x7fc86400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:41.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.186+0000 7fc85b7fe700 1 -- 172.21.15.31:0/2669327477 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc864011590 con 0x7fc874159550 2023-10-06T02:28:41.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.186+0000 7fc85b7fe700 1 -- 172.21.15.31:0/2669327477 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8640116f0 con 0x7fc874159550 2023-10-06T02:28:41.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.186+0000 7fc85b7fe700 1 -- 172.21.15.31:0/2669327477 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc864011ab0 con 0x7fc874159550 2023-10-06T02:28:41.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.186+0000 7fc87899d700 1 -- 172.21.15.31:0/2669327477 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8740cbcd0 con 0x7fc874159550 2023-10-06T02:28:41.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.186+0000 7fc87899d700 1 -- 172.21.15.31:0/2669327477 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8740cb230 con 0x7fc874159550 2023-10-06T02:28:41.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.187+0000 7fc87899d700 1 -- 172.21.15.31:0/2669327477 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8740cc600 con 0x7fc874159550 2023-10-06T02:28:41.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.188+0000 7fc85b7fe700 1 -- 172.21.15.31:0/2669327477 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7fc86401b3b0 con 0x7fc874159550 2023-10-06T02:28:41.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.189+0000 7fc85b7fe700 1 --2- 172.21.15.31:0/2669327477 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc860067050 0x7fc860069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:41.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.189+0000 7fc85b7fe700 1 -- 172.21.15.31:0/2669327477 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc864083e50 con 0x7fc874159550 2023-10-06T02:28:41.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.190+0000 7fc872ffd700 1 --2- 172.21.15.31:0/2669327477 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc860067050 0x7fc860069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:41.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.190+0000 7fc872ffd700 1 --2- 172.21.15.31:0/2669327477 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc860067050 0x7fc860069500 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc8680099b0 tx=0x7fc868017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:41.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.193+0000 7fc85b7fe700 1 -- 172.21.15.31:0/2669327477 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc864050db0 con 0x7fc874159550 2023-10-06T02:28:41.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.472+0000 7fc87899d700 1 -- 172.21.15.31:0/2669327477 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fc8740b0ce0 con 0x7fc874159550 2023-10-06T02:28:41.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.478+0000 7fc85b7fe700 1 -- 172.21.15.31:0/2669327477 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) v1 ==== 190+0+0 (secure 0 0 0) 0x7fc864017020 con 0x7fc874159550 2023-10-06T02:28:41.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.480+0000 7fc87899d700 1 -- 172.21.15.31:0/2669327477 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc860067050 msgr2=0x7fc860069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:41.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.480+0000 7fc87899d700 1 --2- 172.21.15.31:0/2669327477 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc860067050 0x7fc860069500 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc8680099b0 tx=0x7fc868017000 comp rx=0 tx=0).stop 2023-10-06T02:28:41.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.480+0000 7fc87899d700 1 -- 172.21.15.31:0/2669327477 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc874159550 msgr2=0x7fc8740cc7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:41.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.480+0000 7fc87899d700 1 --2- 172.21.15.31:0/2669327477 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc874159550 0x7fc8740cc7b0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fc86400c590 tx=0x7fc86400c950 comp rx=0 tx=0).stop 2023-10-06T02:28:41.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.480+0000 7fc87899d700 1 -- 172.21.15.31:0/2669327477 shutdown_connections 2023-10-06T02:28:41.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.480+0000 7fc87899d700 1 --2- 172.21.15.31:0/2669327477 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc860067050 0x7fc860069500 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:41.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.480+0000 7fc87899d700 1 --2- 172.21.15.31:0/2669327477 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc87415dfd0 0x7fc87416a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:41.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.480+0000 7fc87899d700 1 --2- 172.21.15.31:0/2669327477 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc874159550 0x7fc8740cc7b0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:41.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.480+0000 7fc87899d700 1 --2- 172.21.15.31:0/2669327477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc874159920 0x7fc874169ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:41.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.480+0000 7fc87899d700 1 -- 172.21.15.31:0/2669327477 >> 172.21.15.31:0/2669327477 conn(0x7fc87406ce80 msgr2=0x7fc8740b0270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:41.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.480+0000 7fc87899d700 1 -- 172.21.15.31:0/2669327477 shutdown_connections 2023-10-06T02:28:41.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.480+0000 7fc87899d700 1 -- 172.21.15.31:0/2669327477 wait complete. 2023-10-06T02:28:41.483 INFO:teuthology.orchestra.run.smithi031.stderr:updated caps for client.0 2023-10-06T02:28:41.499 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:28:41.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.684+0000 7f953ad9d700 1 Processor -- start 2023-10-06T02:28:41.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.684+0000 7f953ad9d700 1 -- start start 2023-10-06T02:28:41.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.686+0000 7f953ad9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f953413fb20 0x7f953415a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:41.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.686+0000 7f953ad9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f953415ae70 con 0x7f953413fef0 2023-10-06T02:28:41.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.686+0000 7f953ad9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f953406af60 con 0x7f953406aaa0 2023-10-06T02:28:41.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.686+0000 7f953ad9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f953406b240 con 0x7f953413fb20 2023-10-06T02:28:41.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.686+0000 7f9539d9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f953413fb20 0x7f953415a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:41.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.686+0000 7f953959a700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f953413fef0 0x7f95340c7990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:43110/0 (socket says 172.21.15.31:43110) 2023-10-06T02:28:41.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.686+0000 7f953959a700 1 -- 172.21.15.31:0/1736004065 learned_addr learned my addr 172.21.15.31:0/1736004065 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:28:41.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.686+0000 7f9538d99700 1 -- 172.21.15.31:0/1736004065 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2515054261 0 0) 0x7f9524001290 con 0x7f953413fef0 2023-10-06T02:28:41.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.686+0000 7f9538d99700 1 -- 172.21.15.31:0/1736004065 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9520003570 con 0x7f953413fef0 2023-10-06T02:28:41.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.686+0000 7f9539d9b700 1 -- 172.21.15.31:0/1736004065 >> v1:172.21.15.62:6789/0 conn(0x7f953406aaa0 legacy=0x7f95340c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:41.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.686+0000 7f9539d9b700 1 -- 172.21.15.31:0/1736004065 >> v1:172.21.15.31:6789/0 conn(0x7f953413fef0 legacy=0x7f95340c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:41.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.686+0000 7f9539d9b700 1 -- 172.21.15.31:0/1736004065 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f953406b520 con 0x7f953413fb20 2023-10-06T02:28:41.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.687+0000 7f9539d9b700 1 --2- 172.21.15.31:0/1736004065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f953413fb20 0x7f953415a930 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f95300098e0 tx=0x7f95300091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beb4dc6ead1755e5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:41.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.687+0000 7f9538d99700 1 -- 172.21.15.31:0/1736004065 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953000b640 con 0x7f953413fb20 2023-10-06T02:28:41.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.687+0000 7f9538d99700 1 -- 172.21.15.31:0/1736004065 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f953000b7a0 con 0x7f953413fb20 2023-10-06T02:28:41.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.687+0000 7f9538d99700 1 -- 172.21.15.31:0/1736004065 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953000bb40 con 0x7f953413fb20 2023-10-06T02:28:41.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.687+0000 7f953ad9d700 1 -- 172.21.15.31:0/1736004065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f953413fb20 msgr2=0x7f953415a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:41.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.687+0000 7f953ad9d700 1 --2- 172.21.15.31:0/1736004065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f953413fb20 0x7f953415a930 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f95300098e0 tx=0x7f95300091a0 comp rx=0 tx=0).stop 2023-10-06T02:28:41.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.688+0000 7f953ad9d700 1 -- 172.21.15.31:0/1736004065 shutdown_connections 2023-10-06T02:28:41.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.688+0000 7f953ad9d700 1 --2- 172.21.15.31:0/1736004065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f953413fb20 0x7f953415a930 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:41.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.688+0000 7f953ad9d700 1 -- 172.21.15.31:0/1736004065 >> 172.21.15.31:0/1736004065 conn(0x7f953406d170 msgr2=0x7f95340cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:41.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.688+0000 7f953ad9d700 1 -- 172.21.15.31:0/1736004065 shutdown_connections 2023-10-06T02:28:41.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.688+0000 7f953ad9d700 1 -- 172.21.15.31:0/1736004065 wait complete. 2023-10-06T02:28:41.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.689+0000 7f953ad9d700 1 Processor -- start 2023-10-06T02:28:41.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.689+0000 7f953ad9d700 1 -- start start 2023-10-06T02:28:41.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.690+0000 7f953ad9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f953406aaa0 0x7f9534155f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:41.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.690+0000 7f953ad9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f953413fb20 0x7f9534139aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:41.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.690+0000 7f953ad9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f953413fef0 0x7f9534159ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:41.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.690+0000 7f953ad9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95341569f0 con 0x7f953413fef0 2023-10-06T02:28:41.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.690+0000 7f953ad9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9534159270 con 0x7f953413fb20 2023-10-06T02:28:41.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.690+0000 7f953ad9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9534159550 con 0x7f953406aaa0 2023-10-06T02:28:41.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.690+0000 7f953a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f953413fef0 0x7f9534159ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:41.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.690+0000 7f9539d9b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f953406aaa0 0x7f9534155f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:41.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.691+0000 7f953959a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f953413fb20 0x7f9534139aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:41.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.691+0000 7f953a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f953413fef0 0x7f9534159ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36816/0 (socket says 172.21.15.31:36816) 2023-10-06T02:28:41.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.691+0000 7f953a59c700 1 -- 172.21.15.31:0/3290229654 learned_addr learned my addr 172.21.15.31:0/3290229654 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:41.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.691+0000 7f953a59c700 1 -- 172.21.15.31:0/3290229654 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f953406aaa0 msgr2=0x7f9534155f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:41.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.691+0000 7f953a59c700 1 --2- 172.21.15.31:0/3290229654 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f953406aaa0 0x7f9534155f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:41.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.691+0000 7f953a59c700 1 -- 172.21.15.31:0/3290229654 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f953413fb20 msgr2=0x7f9534139aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:41.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.691+0000 7f953a59c700 1 --2- 172.21.15.31:0/3290229654 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f953413fb20 0x7f9534139aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:41.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.691+0000 7f953a59c700 1 -- 172.21.15.31:0/3290229654 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9530009420 con 0x7f953413fef0 2023-10-06T02:28:41.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.691+0000 7f9539d9b700 1 --2- 172.21.15.31:0/3290229654 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f953406aaa0 0x7f9534155f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:28:41.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.691+0000 7f953a59c700 1 --2- 172.21.15.31:0/3290229654 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f953413fef0 0x7f9534159ab0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f952c00c810 tx=0x7f952c001650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:41.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.691+0000 7f9529ffb700 1 -- 172.21.15.31:0/3290229654 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f952c012630 con 0x7f953413fef0 2023-10-06T02:28:41.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.692+0000 7f9529ffb700 1 -- 172.21.15.31:0/3290229654 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f952c0127b0 con 0x7f953413fef0 2023-10-06T02:28:41.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.692+0000 7f9529ffb700 1 -- 172.21.15.31:0/3290229654 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f952c012b70 con 0x7f953413fef0 2023-10-06T02:28:41.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.692+0000 7f953ad9d700 1 -- 172.21.15.31:0/3290229654 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9534156c90 con 0x7f953413fef0 2023-10-06T02:28:41.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.692+0000 7f953ad9d700 1 -- 172.21.15.31:0/3290229654 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95341564d0 con 0x7f953413fef0 2023-10-06T02:28:41.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.693+0000 7f953ad9d700 1 -- 172.21.15.31:0/3290229654 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9534156e80 con 0x7f953413fef0 2023-10-06T02:28:41.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.693+0000 7f9529ffb700 1 -- 172.21.15.31:0/3290229654 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f952c003030 con 0x7f953413fef0 2023-10-06T02:28:41.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.694+0000 7f9529ffb700 1 --2- 172.21.15.31:0/3290229654 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9518067050 0x7f9518069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:41.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.694+0000 7f9529ffb700 1 -- 172.21.15.31:0/3290229654 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f952c0846e0 con 0x7f953413fef0 2023-10-06T02:28:41.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.695+0000 7f9539d9b700 1 --2- 172.21.15.31:0/3290229654 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9518067050 0x7f9518069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:41.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.695+0000 7f9539d9b700 1 --2- 172.21.15.31:0/3290229654 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9518067050 0x7f9518069500 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f95300093f0 tx=0x7f9530017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:41.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.698+0000 7f9529ffb700 1 -- 172.21.15.31:0/3290229654 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f952c051640 con 0x7f953413fef0 2023-10-06T02:28:41.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.944+0000 7f953ad9d700 1 -- 172.21.15.31:0/3290229654 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f953404c3c0 con 0x7f953413fef0 2023-10-06T02:28:41.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.945+0000 7f9529ffb700 1 -- 172.21.15.31:0/3290229654 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f952c054c60 con 0x7f953413fef0 2023-10-06T02:28:41.946 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:28:41.946 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":71,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6026,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/3801334948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3801334948},{"type":"v1","addr":"172.21.15.62:6834","nonce":3801334948}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":6724,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/1804582836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":1804582836},{"type":"v1","addr":"172.21.15.31:6835","nonce":1804582836}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":6736,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1428432734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":1428432734},{"type":"v1","addr":"172.21.15.62:6835","nonce":1428432734}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:28:35.598388+0000","modified":"2023-10-06T02:28:39.126355+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7065},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7065":{"gid":7065,"name":"a","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6834/603135594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":603135594},{"type":"v1","addr":"172.21.15.31:6834","nonce":603135594}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-06T02:28:41.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.947+0000 7f953ad9d700 1 -- 172.21.15.31:0/3290229654 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9518067050 msgr2=0x7f9518069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:41.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.947+0000 7f953ad9d700 1 --2- 172.21.15.31:0/3290229654 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9518067050 0x7f9518069500 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f95300093f0 tx=0x7f9530017000 comp rx=0 tx=0).stop 2023-10-06T02:28:41.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.947+0000 7f953ad9d700 1 -- 172.21.15.31:0/3290229654 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f953413fef0 msgr2=0x7f9534159ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:41.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.947+0000 7f953ad9d700 1 --2- 172.21.15.31:0/3290229654 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f953413fef0 0x7f9534159ab0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f952c00c810 tx=0x7f952c001650 comp rx=0 tx=0).stop 2023-10-06T02:28:41.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.947+0000 7f953ad9d700 1 -- 172.21.15.31:0/3290229654 shutdown_connections 2023-10-06T02:28:41.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.947+0000 7f953ad9d700 1 --2- 172.21.15.31:0/3290229654 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f953413fef0 0x7f9534159ab0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:41.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.947+0000 7f953ad9d700 1 --2- 172.21.15.31:0/3290229654 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9518067050 0x7f9518069500 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:41.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.947+0000 7f953ad9d700 1 --2- 172.21.15.31:0/3290229654 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f953406aaa0 0x7f9534155f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:41.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.947+0000 7f953ad9d700 1 --2- 172.21.15.31:0/3290229654 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f953413fb20 0x7f9534139aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:41.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.947+0000 7f953ad9d700 1 -- 172.21.15.31:0/3290229654 >> 172.21.15.31:0/3290229654 conn(0x7f953406d170 msgr2=0x7f95340c2a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:41.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.947+0000 7f953ad9d700 1 -- 172.21.15.31:0/3290229654 shutdown_connections 2023-10-06T02:28:41.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:41.947+0000 7f953ad9d700 1 -- 172.21.15.31:0/3290229654 wait complete. 2023-10-06T02:28:41.949 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 71 2023-10-06T02:28:42.368 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 71, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T02:28:35.598388+0000', 'modified': '2023-10-06T02:28:39.126355+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7065}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7065': {'gid': 7065, 'name': 'a', 'rank': 0, 'incarnation': 67, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.31:6834/603135594', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6832', 'nonce': 603135594}, {'type': 'v1', 'addr': '172.21.15.31:6834', 'nonce': 603135594}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T02:28:42.368 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T02:28:42.368 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-06T02:28:42.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.555+0000 7f5a8e79d700 1 Processor -- start 2023-10-06T02:28:42.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.555+0000 7f5a8e79d700 1 -- start start 2023-10-06T02:28:42.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.556+0000 7f5a8e79d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5a88153c30 0x7f5a8815a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:42.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.556+0000 7f5a8e79d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f5a880cf110 0x7f5a8815c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:42.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.556+0000 7f5a8e79d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a8815afd0 con 0x7f5a8815de50 2023-10-06T02:28:42.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.556+0000 7f5a8e79d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a8815b2b0 con 0x7f5a880cf110 2023-10-06T02:28:42.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.556+0000 7f5a8e79d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a8815b590 con 0x7f5a88153c30 2023-10-06T02:28:42.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.557+0000 7f5a8cf9a700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f5a8815de50 0x7f5a880cea00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:43124/0 (socket says 172.21.15.31:43124) 2023-10-06T02:28:42.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.557+0000 7f5a8cf9a700 1 -- 172.21.15.31:0/298917036 learned_addr learned my addr 172.21.15.31:0/298917036 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:28:42.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.557+0000 7f5a8df9c700 1 --2- 172.21.15.31:0/298917036 >> v2:172.21.15.62:3300/0 conn(0x7f5a880cf110 0x7f5a8815c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:42.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.557+0000 7f5a8d79b700 1 --2- 172.21.15.31:0/298917036 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5a88153c30 0x7f5a8815a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:42.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.557+0000 7f5a7ffff700 1 -- 172.21.15.31:0/298917036 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1045704555 0 0) 0x7f5a8815afd0 con 0x7f5a8815de50 2023-10-06T02:28:42.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.557+0000 7f5a7ffff700 1 -- 172.21.15.31:0/298917036 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a74003570 con 0x7f5a8815de50 2023-10-06T02:28:42.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.557+0000 7f5a8df9c700 1 -- 172.21.15.31:0/298917036 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5a88153c30 msgr2=0x7f5a8815a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:42.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.557+0000 7f5a8df9c700 1 --2- 172.21.15.31:0/298917036 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5a88153c30 0x7f5a8815a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:42.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.557+0000 7f5a7ffff700 1 -- 172.21.15.31:0/298917036 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2237447333 0 0) 0x7f5a74003570 con 0x7f5a8815de50 2023-10-06T02:28:42.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.557+0000 7f5a8df9c700 1 -- 172.21.15.31:0/298917036 >> v1:172.21.15.31:6789/0 conn(0x7f5a8815de50 legacy=0x7f5a880cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:42.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.557+0000 7f5a8df9c700 1 -- 172.21.15.31:0/298917036 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a8815b870 con 0x7f5a880cf110 2023-10-06T02:28:42.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.557+0000 7f5a8d79b700 1 --2- 172.21.15.31:0/298917036 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5a88153c30 0x7f5a8815a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:28:42.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.558+0000 7f5a8df9c700 1 --2- 172.21.15.31:0/298917036 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a880cf110 0x7f5a8815c070 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f5a80009e60 tx=0x7f5a80009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3009a4a4d2421f6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:42.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.558+0000 7f5a7ffff700 1 -- 172.21.15.31:0/298917036 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a8000e070 con 0x7f5a880cf110 2023-10-06T02:28:42.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.558+0000 7f5a7ffff700 1 -- 172.21.15.31:0/298917036 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a8000bab0 con 0x7f5a880cf110 2023-10-06T02:28:42.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.558+0000 7f5a7ffff700 1 -- 172.21.15.31:0/298917036 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a8000be30 con 0x7f5a880cf110 2023-10-06T02:28:42.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.558+0000 7f5a8e79d700 1 -- 172.21.15.31:0/298917036 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a880cf110 msgr2=0x7f5a8815c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:42.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.558+0000 7f5a8e79d700 1 --2- 172.21.15.31:0/298917036 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a880cf110 0x7f5a8815c070 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f5a80009e60 tx=0x7f5a80009780 comp rx=0 tx=0).stop 2023-10-06T02:28:42.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.559+0000 7f5a8e79d700 1 -- 172.21.15.31:0/298917036 shutdown_connections 2023-10-06T02:28:42.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.559+0000 7f5a8e79d700 1 --2- 172.21.15.31:0/298917036 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a880cf110 0x7f5a8815c070 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:42.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.559+0000 7f5a8e79d700 1 --2- 172.21.15.31:0/298917036 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5a88153c30 0x7f5a8815a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:42.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.559+0000 7f5a8e79d700 1 -- 172.21.15.31:0/298917036 >> 172.21.15.31:0/298917036 conn(0x7f5a880703a0 msgr2=0x7f5a880707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:42.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.559+0000 7f5a8e79d700 1 -- 172.21.15.31:0/298917036 shutdown_connections 2023-10-06T02:28:42.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.559+0000 7f5a8e79d700 1 -- 172.21.15.31:0/298917036 wait complete. 2023-10-06T02:28:42.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.560+0000 7f5a8e79d700 1 Processor -- start 2023-10-06T02:28:42.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.560+0000 7f5a8e79d700 1 -- start start 2023-10-06T02:28:42.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.561+0000 7f5a8e79d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a880cf110 0x7f5a88151b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:42.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.561+0000 7f5a8e79d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a88153c30 0x7f5a880c9a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:42.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.561+0000 7f5a8e79d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5a8815de50 0x7f5a880b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:42.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.561+0000 7f5a8e79d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a88152060 con 0x7f5a88153c30 2023-10-06T02:28:42.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.561+0000 7f5a8e79d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a880c75e0 con 0x7f5a880cf110 2023-10-06T02:28:42.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.561+0000 7f5a8e79d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a880b4150 con 0x7f5a8815de50 2023-10-06T02:28:42.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.561+0000 7f5a8cf9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a88153c30 0x7f5a880c9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:42.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.561+0000 7f5a8cf9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a88153c30 0x7f5a880c9a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36830/0 (socket says 172.21.15.31:36830) 2023-10-06T02:28:42.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.561+0000 7f5a8cf9a700 1 -- 172.21.15.31:0/3308754822 learned_addr learned my addr 172.21.15.31:0/3308754822 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:42.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.562+0000 7f5a8d79b700 1 --2- 172.21.15.31:0/3308754822 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a880cf110 0x7f5a88151b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:42.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.562+0000 7f5a8df9c700 1 --2- 172.21.15.31:0/3308754822 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5a8815de50 0x7f5a880b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:42.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.562+0000 7f5a8cf9a700 1 -- 172.21.15.31:0/3308754822 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5a8815de50 msgr2=0x7f5a880b5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:42.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.562+0000 7f5a8cf9a700 1 --2- 172.21.15.31:0/3308754822 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5a8815de50 0x7f5a880b5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:42.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.562+0000 7f5a8cf9a700 1 -- 172.21.15.31:0/3308754822 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a880cf110 msgr2=0x7f5a88151b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:42.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.562+0000 7f5a8cf9a700 1 --2- 172.21.15.31:0/3308754822 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a880cf110 0x7f5a88151b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:42.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.562+0000 7f5a8cf9a700 1 -- 172.21.15.31:0/3308754822 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a84009420 con 0x7f5a88153c30 2023-10-06T02:28:42.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.562+0000 7f5a8df9c700 1 --2- 172.21.15.31:0/3308754822 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5a8815de50 0x7f5a880b5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:28:42.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.562+0000 7f5a8cf9a700 1 --2- 172.21.15.31:0/3308754822 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a88153c30 0x7f5a880c9a50 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f5a7800c5e0 tx=0x7f5a78007da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:42.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.562+0000 7f5a7d7fa700 1 -- 172.21.15.31:0/3308754822 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a78007430 con 0x7f5a88153c30 2023-10-06T02:28:42.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.562+0000 7f5a7d7fa700 1 -- 172.21.15.31:0/3308754822 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a78007590 con 0x7f5a88153c30 2023-10-06T02:28:42.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.562+0000 7f5a8e79d700 1 -- 172.21.15.31:0/3308754822 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a80009a20 con 0x7f5a88153c30 2023-10-06T02:28:42.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.562+0000 7f5a7d7fa700 1 -- 172.21.15.31:0/3308754822 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a78007700 con 0x7f5a88153c30 2023-10-06T02:28:42.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.562+0000 7f5a8e79d700 1 -- 172.21.15.31:0/3308754822 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a880b4710 con 0x7f5a88153c30 2023-10-06T02:28:42.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.565+0000 7f5a7d7fa700 1 -- 172.21.15.31:0/3308754822 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f5a78004cb0 con 0x7f5a88153c30 2023-10-06T02:28:42.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.566+0000 7f5a7d7fa700 1 --2- 172.21.15.31:0/3308754822 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5a6c067050 0x7f5a6c069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:42.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.566+0000 7f5a7d7fa700 1 -- 172.21.15.31:0/3308754822 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f5a780845d0 con 0x7f5a88153c30 2023-10-06T02:28:42.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.567+0000 7f5a8d79b700 1 --2- 172.21.15.31:0/3308754822 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5a6c067050 0x7f5a6c069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:42.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.567+0000 7f5a8d79b700 1 --2- 172.21.15.31:0/3308754822 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5a6c067050 0x7f5a6c069500 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5a84009f40 tx=0x7f5a840091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:42.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.568+0000 7f5a9e11ab80 1 Processor -- start 2023-10-06T02:28:42.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.568+0000 7f5a9e11ab80 1 -- start start 2023-10-06T02:28:42.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.569+0000 7f5a9e11ab80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55754fa62e50 0x55754fa3dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:42.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.569+0000 7f5a9e11ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55754fa432c0 0x55754fa3e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:42.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.569+0000 7f5a9e11ab80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55754fa44a10 0x55754fa3e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:42.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.569+0000 7f5a9e11ab80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55754f998a70 con 0x55754fa62e50 2023-10-06T02:28:42.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.569+0000 7f5a9e11ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55754f996270 con 0x55754fa432c0 2023-10-06T02:28:42.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.569+0000 7f5a9e11ab80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x55754f992060 con 0x55754fa44a10 2023-10-06T02:28:42.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.569+0000 7f5a8d79b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55754fa62e50 0x55754fa3dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:42.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.569+0000 7f5a8d79b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55754fa62e50 0x55754fa3dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36832/0 (socket says 172.21.15.31:36832) 2023-10-06T02:28:42.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.569+0000 7f5a8d79b700 1 -- 172.21.15.31:0/1678967834 learned_addr learned my addr 172.21.15.31:0/1678967834 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:42.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.569+0000 7f5a8d79b700 1 -- 172.21.15.31:0/1678967834 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55754fa44a10 msgr2=0x55754fa3e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:42.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.569+0000 7f5a8d79b700 1 --2- 172.21.15.31:0/1678967834 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55754fa44a10 0x55754fa3e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:42.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.569+0000 7f5a8d79b700 1 -- 172.21.15.31:0/1678967834 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55754fa432c0 msgr2=0x55754fa3e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:42.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.569+0000 7f5a8d79b700 1 --2- 172.21.15.31:0/1678967834 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55754fa432c0 0x55754fa3e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:42.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.569+0000 7f5a8d79b700 1 -- 172.21.15.31:0/1678967834 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55754f82c270 con 0x55754fa62e50 2023-10-06T02:28:42.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.570+0000 7f5a8d79b700 1 --2- 172.21.15.31:0/1678967834 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55754fa62e50 0x55754fa3dcc0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f5a84012010 tx=0x7f5a84016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee56ca4f4ea0f018 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:42.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.570+0000 7f5a727fc700 1 -- 172.21.15.31:0/1678967834 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a84021420 con 0x55754fa62e50 2023-10-06T02:28:42.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.570+0000 7f5a727fc700 1 -- 172.21.15.31:0/1678967834 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a84021580 con 0x55754fa62e50 2023-10-06T02:28:42.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.570+0000 7f5a727fc700 1 -- 172.21.15.31:0/1678967834 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a84021900 con 0x55754fa62e50 2023-10-06T02:28:42.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.571+0000 7f5a9e11ab80 1 -- 172.21.15.31:0/1678967834 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55754fa62e50 msgr2=0x55754fa3dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:42.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.571+0000 7f5a9e11ab80 1 --2- 172.21.15.31:0/1678967834 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55754fa62e50 0x55754fa3dcc0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f5a84012010 tx=0x7f5a84016000 comp rx=0 tx=0).stop 2023-10-06T02:28:42.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.571+0000 7f5a9e11ab80 1 -- 172.21.15.31:0/1678967834 shutdown_connections 2023-10-06T02:28:42.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.571+0000 7f5a9e11ab80 1 --2- 172.21.15.31:0/1678967834 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55754fa44a10 0x55754fa3e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:42.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.571+0000 7f5a9e11ab80 1 --2- 172.21.15.31:0/1678967834 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55754fa62e50 0x55754fa3dcc0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:42.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.571+0000 7f5a9e11ab80 1 --2- 172.21.15.31:0/1678967834 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55754fa432c0 0x55754fa3e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:42.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.571+0000 7f5a9e11ab80 1 -- 172.21.15.31:0/1678967834 >> 172.21.15.31:0/1678967834 conn(0x55754fa42e80 msgr2=0x55754fa436b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:42.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.571+0000 7f5a9e11ab80 1 -- 172.21.15.31:0/1678967834 shutdown_connections 2023-10-06T02:28:42.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.571+0000 7f5a9e11ab80 1 -- 172.21.15.31:0/1678967834 wait complete. 2023-10-06T02:28:42.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.572+0000 7f5a9e11ab80 1 -- start start 2023-10-06T02:28:42.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.572+0000 7f5a9e11ab80 1 Processor -- start 2023-10-06T02:28:42.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.573+0000 7f5a9e11ab80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55754fa432c0 0x55754fa60970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:42.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.573+0000 7f5a9e11ab80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55754fa44a10 0x55754fa97e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:42.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.573+0000 7f5a9e11ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55754fa62e50 0x55754fa707a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:42.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.573+0000 7f5a9e11ab80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55754f93d890 con 0x55754fa432c0 2023-10-06T02:28:42.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.573+0000 7f5a9e11ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55754f958ea0 con 0x55754fa62e50 2023-10-06T02:28:42.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.573+0000 7f5a9e11ab80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x55754f814e30 con 0x55754fa44a10 2023-10-06T02:28:42.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.573+0000 7f5a9e11ab80 10 client.0 _ll_register_callbacks cb 0x55754fa05020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T02:28:42.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.573+0000 7f5a8d79b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55754fa432c0 0x55754fa60970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:42.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.573+0000 7f5a8d79b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55754fa432c0 0x55754fa60970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36834/0 (socket says 172.21.15.31:36834) 2023-10-06T02:28:42.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.573+0000 7f5a8d79b700 1 -- 172.21.15.31:0/3979305499 learned_addr learned my addr 172.21.15.31:0/3979305499 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:42.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.573+0000 7f5a8d79b700 1 -- 172.21.15.31:0/3979305499 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55754fa44a10 msgr2=0x55754fa97e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:42.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.573+0000 7f5a8d79b700 1 --2- 172.21.15.31:0/3979305499 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55754fa44a10 0x55754fa97e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:42.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.573+0000 7f5a8d79b700 1 -- 172.21.15.31:0/3979305499 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55754fa62e50 msgr2=0x55754fa707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:42.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.573+0000 7f5a8d79b700 1 --2- 172.21.15.31:0/3979305499 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55754fa62e50 0x55754fa707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:42.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.573+0000 7f5a8d79b700 1 -- 172.21.15.31:0/3979305499 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a84011cc0 con 0x55754fa432c0 2023-10-06T02:28:42.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.574+0000 7f5a8d79b700 1 --2- 172.21.15.31:0/3979305499 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55754fa432c0 0x55754fa60970 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f5a84024040 tx=0x7f5a84022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a4834180b89c0bc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:42.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.574+0000 7f5a677fe700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T02:28:42.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.574+0000 7f5a677fe700 1 -- 172.21.15.31:0/3979305499 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a84009bb0 con 0x55754fa432c0 2023-10-06T02:28:42.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.574+0000 7f5a677fe700 1 -- 172.21.15.31:0/3979305499 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f5a8402a630 con 0x55754fa432c0 2023-10-06T02:28:42.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.574+0000 7f5a9e11ab80 1 -- 172.21.15.31:0/3979305499 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55754fa36940 con 0x55754fa432c0 2023-10-06T02:28:42.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.574+0000 7f5a677fe700 1 -- 172.21.15.31:0/3979305499 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a8402aa00 con 0x55754fa432c0 2023-10-06T02:28:42.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.574+0000 7f5a9e11ab80 10 client.7215 fetch_fsmap learned FSMap version 71 2023-10-06T02:28:42.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.574+0000 7f5a9e11ab80 1 -- 172.21.15.31:0/3979305499 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x55754f7cefe0 con 0x55754fa432c0 2023-10-06T02:28:42.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.574+0000 7f5a677fe700 1 -- 172.21.15.31:0/3979305499 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55754fa36940 con 0x55754fa432c0 2023-10-06T02:28:42.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.575+0000 7f5a677fe700 1 -- 172.21.15.31:0/3979305499 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 71) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f5a8402ab60 con 0x55754fa432c0 2023-10-06T02:28:42.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.575+0000 7f5a9e11ab80 10 client.7215 fetch_fsmap finished waiting for FSMap version 71 2023-10-06T02:28:42.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.575+0000 7f5a9e11ab80 10 client.7215 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T02:28:42.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.575+0000 7f5a9e11ab80 20 client.7215 populate_metadata read hostname 'smithi031' 2023-10-06T02:28:42.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.575+0000 7f5a9e11ab80 1 --2- 172.21.15.31:0/3979305499 >> [v2:172.21.15.31:6832/603135594,v1:172.21.15.31:6834/603135594] conn(0x55754fa63640 0x55754fa612b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:42.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.575+0000 7f5a9e11ab80 4 client.7215 mds_command: new command op to 7065 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T02:28:42.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.575+0000 7f5a9e11ab80 1 -- 172.21.15.31:0/3979305499 --> [v2:172.21.15.31:6832/603135594,v1:172.21.15.31:6834/603135594] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55754fa13870 con 0x55754fa63640 2023-10-06T02:28:42.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.575+0000 7f5a8cf9a700 1 --2- 172.21.15.31:0/3979305499 >> [v2:172.21.15.31:6832/603135594,v1:172.21.15.31:6834/603135594] conn(0x55754fa63640 0x55754fa612b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:42.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.575+0000 7f5a8cf9a700 1 --2- 172.21.15.31:0/3979305499 >> [v2:172.21.15.31:6832/603135594,v1:172.21.15.31:6834/603135594] conn(0x55754fa63640 0x55754fa612b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6815411ef6a4b99 server_cookie=f98a710d79e26881 in_seq=0 out_seq=0 2023-10-06T02:28:42.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.575+0000 7f5a677fe700 10 client.7215 ms_handle_connect on v2:172.21.15.31:6832/603135594 2023-10-06T02:28:42.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.576+0000 7f5a677fe700 1 -- 172.21.15.31:0/3979305499 <== mds.0 v2:172.21.15.31:6832/603135594 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55754fa13870 con 0x55754fa63640 2023-10-06T02:28:42.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.576+0000 7f5a677fe700 10 client.7215 handle_command_reply: tid=0 2023-10-06T02:28:42.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.576+0000 7f5a9e11ab80 1 client.7215 shutdown 2023-10-06T02:28:42.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.576+0000 7f5a9e11ab80 1 -- 172.21.15.31:0/3979305499 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55754fa432c0 msgr2=0x55754fa60970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:42.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.576+0000 7f5a9e11ab80 1 --2- 172.21.15.31:0/3979305499 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55754fa432c0 0x55754fa60970 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f5a84024040 tx=0x7f5a84022000 comp rx=0 tx=0).stop 2023-10-06T02:28:42.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.576+0000 7f5a9e11ab80 1 -- 172.21.15.31:0/3979305499 shutdown_connections 2023-10-06T02:28:42.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.576+0000 7f5a9e11ab80 1 --2- 172.21.15.31:0/3979305499 >> [v2:172.21.15.31:6832/603135594,v1:172.21.15.31:6834/603135594] conn(0x55754fa63640 0x55754fa612b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:42.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.576+0000 7f5a9e11ab80 1 --2- 172.21.15.31:0/3979305499 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55754fa62e50 0x55754fa707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:42.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.576+0000 7f5a9e11ab80 1 --2- 172.21.15.31:0/3979305499 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55754fa432c0 0x55754fa60970 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:42.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.576+0000 7f5a9e11ab80 1 --2- 172.21.15.31:0/3979305499 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55754fa44a10 0x55754fa97e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:42.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.576+0000 7f5a677fe700 0 client.7215 ms_handle_reset on v2:172.21.15.31:6832/603135594 2023-10-06T02:28:42.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.576+0000 7f5a9e11ab80 1 -- 172.21.15.31:0/3979305499 >> 172.21.15.31:0/3979305499 conn(0x55754fa42e80 msgr2=0x55754fa38400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:42.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.577+0000 7f5a9e11ab80 1 -- 172.21.15.31:0/3979305499 shutdown_connections 2023-10-06T02:28:42.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.577+0000 7f5a9e11ab80 1 -- 172.21.15.31:0/3979305499 wait complete. 2023-10-06T02:28:42.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.577+0000 7f5a9e11ab80 20 client.7215 trim_cache size 0 max 16384 2023-10-06T02:28:42.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.594+0000 7f5a9e11ab80 1 Processor -- start 2023-10-06T02:28:42.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.594+0000 7f5a9e11ab80 1 -- start start 2023-10-06T02:28:42.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.595+0000 7f5a9e11ab80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55754fa432c0 0x55754fa43690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:42.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.595+0000 7f5a9e11ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55754fa44a10 0x55754fae76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:42.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.595+0000 7f5a9e11ab80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55754fa62e50 0x55754fa38400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:42.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.595+0000 7f5a9e11ab80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55754fa64330 con 0x55754fa62e50 2023-10-06T02:28:42.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.595+0000 7f5a9e11ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55754fa89010 con 0x55754fa44a10 2023-10-06T02:28:42.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.595+0000 7f5a9e11ab80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x55754fa6f2f0 con 0x55754fa432c0 2023-10-06T02:28:42.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.595+0000 7f5a8df9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55754fa62e50 0x55754fa38400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:42.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.595+0000 7f5a8df9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55754fa62e50 0x55754fa38400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36848/0 (socket says 172.21.15.31:36848) 2023-10-06T02:28:42.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.595+0000 7f5a8df9c700 1 -- 172.21.15.31:0/2996008594 learned_addr learned my addr 172.21.15.31:0/2996008594 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:42.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.596+0000 7f5a8df9c700 1 -- 172.21.15.31:0/2996008594 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55754fa432c0 msgr2=0x55754fa43690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:42.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.596+0000 7f5a8df9c700 1 --2- 172.21.15.31:0/2996008594 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55754fa432c0 0x55754fa43690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:42.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.596+0000 7f5a8cf9a700 1 --2- 172.21.15.31:0/2996008594 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55754fa44a10 0x55754fae76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:42.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.596+0000 7f5a8df9c700 1 -- 172.21.15.31:0/2996008594 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55754fa44a10 msgr2=0x55754fae76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:42.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.596+0000 7f5a8df9c700 1 --2- 172.21.15.31:0/2996008594 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55754fa44a10 0x55754fae76c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:42.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.596+0000 7f5a8df9c700 1 -- 172.21.15.31:0/2996008594 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55754f960780 con 0x55754fa62e50 2023-10-06T02:28:42.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.596+0000 7f5a8df9c700 1 --2- 172.21.15.31:0/2996008594 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55754fa62e50 0x55754fa38400 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f5a8806eb00 tx=0x7f5a80019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9f0af84a9673b7d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:42.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.596+0000 7f5a72ffd700 1 -- 172.21.15.31:0/2996008594 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a8000bc20 con 0x55754fa62e50 2023-10-06T02:28:42.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.596+0000 7f5a72ffd700 1 -- 172.21.15.31:0/2996008594 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a8000bd80 con 0x55754fa62e50 2023-10-06T02:28:42.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.596+0000 7f5a72ffd700 1 -- 172.21.15.31:0/2996008594 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a80012890 con 0x55754fa62e50 2023-10-06T02:28:42.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.597+0000 7f5a9e11ab80 1 -- 172.21.15.31:0/2996008594 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55754fa62e50 msgr2=0x55754fa38400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:42.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.597+0000 7f5a9e11ab80 1 --2- 172.21.15.31:0/2996008594 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55754fa62e50 0x55754fa38400 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f5a8806eb00 tx=0x7f5a80019000 comp rx=0 tx=0).stop 2023-10-06T02:28:42.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.597+0000 7f5a9e11ab80 1 -- 172.21.15.31:0/2996008594 shutdown_connections 2023-10-06T02:28:42.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.597+0000 7f5a9e11ab80 1 --2- 172.21.15.31:0/2996008594 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55754fa62e50 0x55754fa38400 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:42.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.597+0000 7f5a9e11ab80 1 --2- 172.21.15.31:0/2996008594 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55754fa432c0 0x55754fa43690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:42.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.597+0000 7f5a9e11ab80 1 --2- 172.21.15.31:0/2996008594 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55754fa44a10 0x55754fae76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:42.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.597+0000 7f5a9e11ab80 1 -- 172.21.15.31:0/2996008594 >> 172.21.15.31:0/2996008594 conn(0x55754fa42e80 msgr2=0x55754fa61780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:42.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.597+0000 7f5a9e11ab80 1 -- 172.21.15.31:0/2996008594 shutdown_connections 2023-10-06T02:28:42.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.597+0000 7f5a9e11ab80 1 -- 172.21.15.31:0/2996008594 wait complete. 2023-10-06T02:28:42.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.598+0000 7f5a9e11ab80 1 -- start start 2023-10-06T02:28:42.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.598+0000 7f5a9e11ab80 1 Processor -- start 2023-10-06T02:28:42.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.599+0000 7f5a9e11ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55754fa432c0 0x55754fa387e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:42.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.599+0000 7f5a9e11ab80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55754fa44a10 0x55754faa6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:42.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.599+0000 7f5a9e11ab80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55754fa62e50 0x55754faa4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:42.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.599+0000 7f5a9e11ab80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55754fa42cb0 con 0x55754fa44a10 2023-10-06T02:28:42.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.599+0000 7f5a9e11ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55754fa6e320 con 0x55754fa432c0 2023-10-06T02:28:42.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.599+0000 7f5a9e11ab80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x55754faa4740 con 0x55754fa62e50 2023-10-06T02:28:42.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.599+0000 7f5a9e11ab80 10 client.0 _ll_register_callbacks cb 0x55754fa05020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T02:28:42.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.599+0000 7f5a8cf9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55754fa44a10 0x55754faa6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:42.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.599+0000 7f5a8cf9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55754fa44a10 0x55754faa6870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36856/0 (socket says 172.21.15.31:36856) 2023-10-06T02:28:42.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.599+0000 7f5a8cf9a700 1 -- 172.21.15.31:0/1458616506 learned_addr learned my addr 172.21.15.31:0/1458616506 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:42.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.599+0000 7f5a8d79b700 1 --2- 172.21.15.31:0/1458616506 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55754fa432c0 0x55754fa387e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:42.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.599+0000 7f5a8df9c700 1 --2- 172.21.15.31:0/1458616506 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55754fa62e50 0x55754faa4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:42.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.600+0000 7f5a8cf9a700 1 -- 172.21.15.31:0/1458616506 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55754fa62e50 msgr2=0x55754faa4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:42.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.600+0000 7f5a8cf9a700 1 --2- 172.21.15.31:0/1458616506 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55754fa62e50 0x55754faa4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:42.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.600+0000 7f5a8cf9a700 1 -- 172.21.15.31:0/1458616506 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55754fa432c0 msgr2=0x55754fa387e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:42.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.600+0000 7f5a8cf9a700 1 --2- 172.21.15.31:0/1458616506 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55754fa432c0 0x55754fa387e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:42.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.600+0000 7f5a8cf9a700 1 -- 172.21.15.31:0/1458616506 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a8815b870 con 0x55754fa44a10 2023-10-06T02:28:42.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.600+0000 7f5a8df9c700 1 --2- 172.21.15.31:0/1458616506 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55754fa62e50 0x55754faa4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:28:42.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.600+0000 7f5a8cf9a700 1 --2- 172.21.15.31:0/1458616506 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55754fa44a10 0x55754faa6870 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f5a78051770 tx=0x7f5a78009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff1e7d8ed9145029 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:42.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.600+0000 7f5a677fe700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T02:28:42.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.600+0000 7f5a677fe700 1 -- 172.21.15.31:0/1458616506 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a780584c0 con 0x55754fa44a10 2023-10-06T02:28:42.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.600+0000 7f5a9e11ab80 1 -- 172.21.15.31:0/1458616506 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5a84021580 con 0x55754fa44a10 2023-10-06T02:28:42.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.600+0000 7f5a677fe700 1 -- 172.21.15.31:0/1458616506 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f5a7805c210 con 0x55754fa44a10 2023-10-06T02:28:42.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.600+0000 7f5a677fe700 1 -- 172.21.15.31:0/1458616506 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a78082af0 con 0x55754fa44a10 2023-10-06T02:28:42.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.600+0000 7f5a9e11ab80 10 client.7221 fetch_fsmap learned FSMap version 71 2023-10-06T02:28:42.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.600+0000 7f5a9e11ab80 1 -- 172.21.15.31:0/1458616506 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x55754f947470 con 0x55754fa44a10 2023-10-06T02:28:42.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.600+0000 7f5a677fe700 1 -- 172.21.15.31:0/1458616506 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a84021580 con 0x55754fa44a10 2023-10-06T02:28:42.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.601+0000 7f5a677fe700 1 -- 172.21.15.31:0/1458616506 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 71) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f5a7801a2b0 con 0x55754fa44a10 2023-10-06T02:28:42.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.601+0000 7f5a9e11ab80 10 client.7221 fetch_fsmap finished waiting for FSMap version 71 2023-10-06T02:28:42.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.601+0000 7f5a9e11ab80 10 client.7221 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T02:28:42.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.601+0000 7f5a9e11ab80 20 client.7221 populate_metadata read hostname 'smithi031' 2023-10-06T02:28:42.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.601+0000 7f5a9e11ab80 1 --2- 172.21.15.31:0/1458616506 >> [v2:172.21.15.31:6832/603135594,v1:172.21.15.31:6834/603135594] conn(0x55754fae64f0 0x55754fae68c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:42.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.601+0000 7f5a9e11ab80 4 client.7221 mds_command: new command op to 7065 tid=0[{"prefix": "status"}] 2023-10-06T02:28:42.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.601+0000 7f5a9e11ab80 1 -- 172.21.15.31:0/1458616506 --> [v2:172.21.15.31:6832/603135594,v1:172.21.15.31:6834/603135594] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55754f934e00 con 0x55754fae64f0 2023-10-06T02:28:42.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.601+0000 7f5a8d79b700 1 --2- 172.21.15.31:0/1458616506 >> [v2:172.21.15.31:6832/603135594,v1:172.21.15.31:6834/603135594] conn(0x55754fae64f0 0x55754fae68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:42.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.601+0000 7f5a677fe700 10 client.7221 ms_handle_connect on v2:172.21.15.31:6832/603135594 2023-10-06T02:28:42.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.601+0000 7f5a8d79b700 1 --2- 172.21.15.31:0/1458616506 >> [v2:172.21.15.31:6832/603135594,v1:172.21.15.31:6834/603135594] conn(0x55754fae64f0 0x55754fae68c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfcebefdfd82a57b server_cookie=bb7586cbd3f62872 in_seq=0 out_seq=0 2023-10-06T02:28:42.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.602+0000 7f5a677fe700 1 -- 172.21.15.31:0/1458616506 <== mds.0 v2:172.21.15.31:6832/603135594 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55754f934e00 con 0x55754fae64f0 2023-10-06T02:28:42.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.602+0000 7f5a677fe700 10 client.7221 handle_command_reply: tid=0 2023-10-06T02:28:42.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.602+0000 7f5a9e11ab80 1 client.7221 shutdown 2023-10-06T02:28:42.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.602+0000 7f5a9e11ab80 1 -- 172.21.15.31:0/1458616506 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55754fa44a10 msgr2=0x55754faa6870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:42.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.602+0000 7f5a9e11ab80 1 --2- 172.21.15.31:0/1458616506 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55754fa44a10 0x55754faa6870 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f5a78051770 tx=0x7f5a78009040 comp rx=0 tx=0).stop 2023-10-06T02:28:42.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.602+0000 7f5a9e11ab80 1 -- 172.21.15.31:0/1458616506 shutdown_connections 2023-10-06T02:28:42.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.602+0000 7f5a9e11ab80 1 --2- 172.21.15.31:0/1458616506 >> [v2:172.21.15.31:6832/603135594,v1:172.21.15.31:6834/603135594] conn(0x55754fae64f0 0x55754fae68c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:42.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.602+0000 7f5a9e11ab80 1 --2- 172.21.15.31:0/1458616506 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55754fa62e50 0x55754faa4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:42.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.602+0000 7f5a9e11ab80 1 --2- 172.21.15.31:0/1458616506 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55754fa432c0 0x55754fa387e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:42.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.602+0000 7f5a677fe700 0 client.7221 ms_handle_reset on v2:172.21.15.31:6832/603135594 2023-10-06T02:28:42.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.602+0000 7f5a9e11ab80 1 --2- 172.21.15.31:0/1458616506 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55754fa44a10 0x55754faa6870 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:42.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.602+0000 7f5a9e11ab80 1 -- 172.21.15.31:0/1458616506 >> 172.21.15.31:0/1458616506 conn(0x55754fa42e80 msgr2=0x55754fa6f7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:42.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.602+0000 7f5a9e11ab80 1 -- 172.21.15.31:0/1458616506 shutdown_connections 2023-10-06T02:28:42.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.602+0000 7f5a9e11ab80 1 -- 172.21.15.31:0/1458616506 wait complete. 2023-10-06T02:28:42.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.602+0000 7f5a9e11ab80 20 client.7221 trim_cache size 0 max 16384 2023-10-06T02:28:42.609 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T02:28:42.609 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "d6fcf512-874b-4259-b90a-30ae070f00a9", 2023-10-06T02:28:42.610 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 0, 2023-10-06T02:28:42.610 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 7065, 2023-10-06T02:28:42.610 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2023-10-06T02:28:42.610 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2023-10-06T02:28:42.610 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2023-10-06T02:28:42.610 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 6.50295836, 2023-10-06T02:28:42.610 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 71, 2023-10-06T02:28:42.610 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 106, 2023-10-06T02:28:42.611 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 106, 2023-10-06T02:28:42.611 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 12.241822620000001 2023-10-06T02:28:42.611 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T02:28:42.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.603+0000 7f5a8e79d700 1 -- 172.21.15.31:0/3308754822 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5a6c067050 msgr2=0x7f5a6c069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:42.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.603+0000 7f5a8e79d700 1 --2- 172.21.15.31:0/3308754822 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5a6c067050 0x7f5a6c069500 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5a84009f40 tx=0x7f5a840091a0 comp rx=0 tx=0).stop 2023-10-06T02:28:42.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.603+0000 7f5a8e79d700 1 -- 172.21.15.31:0/3308754822 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a88153c30 msgr2=0x7f5a880c9a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:42.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.603+0000 7f5a8e79d700 1 --2- 172.21.15.31:0/3308754822 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a88153c30 0x7f5a880c9a50 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f5a7800c5e0 tx=0x7f5a78007da0 comp rx=0 tx=0).stop 2023-10-06T02:28:42.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.603+0000 7f5a8e79d700 1 -- 172.21.15.31:0/3308754822 shutdown_connections 2023-10-06T02:28:42.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.603+0000 7f5a8e79d700 1 --2- 172.21.15.31:0/3308754822 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5a6c067050 0x7f5a6c069500 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:42.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.603+0000 7f5a8e79d700 1 --2- 172.21.15.31:0/3308754822 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5a8815de50 0x7f5a880b5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:42.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.603+0000 7f5a8e79d700 1 --2- 172.21.15.31:0/3308754822 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a880cf110 0x7f5a88151b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:42.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.603+0000 7f5a8e79d700 1 --2- 172.21.15.31:0/3308754822 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a88153c30 0x7f5a880c9a50 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:42.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.603+0000 7f5a8e79d700 1 -- 172.21.15.31:0/3308754822 >> 172.21.15.31:0/3308754822 conn(0x7f5a880703a0 msgr2=0x7f5a880d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:42.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.603+0000 7f5a8e79d700 1 -- 172.21.15.31:0/3308754822 shutdown_connections 2023-10-06T02:28:42.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.603+0000 7f5a8e79d700 1 -- 172.21.15.31:0/3308754822 wait complete. 2023-10-06T02:28:42.621 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T02:28:42.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.806+0000 7f888b59d700 1 Processor -- start 2023-10-06T02:28:42.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.806+0000 7f888b59d700 1 -- start start 2023-10-06T02:28:42.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.808+0000 7f888b59d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f888406e0a0 0x7f88840c6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:42.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.808+0000 7f888b59d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f888406a490 con 0x7f888406f3a0 2023-10-06T02:28:42.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.808+0000 7f888b59d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88840c7490 con 0x7f8884150380 2023-10-06T02:28:42.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.808+0000 7f888b59d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88840c7770 con 0x7f888406e0a0 2023-10-06T02:28:42.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.808+0000 7f8889d9a700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f888406f3a0 0x7f888414fc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:43126/0 (socket says 172.21.15.31:43126) 2023-10-06T02:28:42.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.808+0000 7f8889d9a700 1 -- 172.21.15.31:0/1501505743 learned_addr learned my addr 172.21.15.31:0/1501505743 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:28:42.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.808+0000 7f8889599700 1 -- 172.21.15.31:0/1501505743 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 25865022 0 0) 0x7f888406a490 con 0x7f888406f3a0 2023-10-06T02:28:42.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.808+0000 7f888a59b700 1 --2- 172.21.15.31:0/1501505743 >> v2:172.21.15.62:3300/0 conn(0x7f888406e0a0 0x7f88840c6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:42.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.808+0000 7f8889599700 1 -- 172.21.15.31:0/1501505743 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8870003570 con 0x7f888406f3a0 2023-10-06T02:28:42.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.809+0000 7f8889599700 1 -- 172.21.15.31:0/1501505743 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3330371389 0 0) 0x7f8870003570 con 0x7f888406f3a0 2023-10-06T02:28:42.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.809+0000 7f8889599700 1 -- 172.21.15.31:0/1501505743 >> v2:172.21.15.62:3300/0 conn(0x7f888406e0a0 msgr2=0x7f88840c6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:42.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.809+0000 7f8889599700 1 --2- 172.21.15.31:0/1501505743 >> v2:172.21.15.62:3300/0 conn(0x7f888406e0a0 0x7f88840c6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:42.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.809+0000 7f8889599700 1 -- 172.21.15.31:0/1501505743 >> v1:172.21.15.62:6789/0 conn(0x7f8884150380 legacy=0x7f888406a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:42.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.809+0000 7f8889599700 1 -- 172.21.15.31:0/1501505743 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88840c7a50 con 0x7f888406f3a0 2023-10-06T02:28:42.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.809+0000 7f8889599700 1 -- 172.21.15.31:0/1501505743 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7f88740015f0 con 0x7f888406f3a0 2023-10-06T02:28:42.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.809+0000 7f888a59b700 1 --2- 172.21.15.31:0/1501505743 >> v2:172.21.15.62:3300/0 conn(0x7f888406e0a0 0x7f88840c6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:28:42.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.809+0000 7f8889599700 1 -- 172.21.15.31:0/1501505743 >> v1:172.21.15.31:6789/0 conn(0x7f888406f3a0 legacy=0x7f888414fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:42.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.809+0000 7f8889599700 1 --2- 172.21.15.31:0/1501505743 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8870003a60 0x7f8870003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:42.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.809+0000 7f8889599700 1 --2- 172.21.15.31:0/1501505743 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8870004810 0x7f8870004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:42.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.809+0000 7f8889599700 1 --2- 172.21.15.31:0/1501505743 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88700052e0 0x7f887000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:42.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.809+0000 7f8889599700 1 -- 172.21.15.31:0/1501505743 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88840c7770 con 0x7f88700052e0 2023-10-06T02:28:42.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.809+0000 7f8889599700 1 -- 172.21.15.31:0/1501505743 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f887000c190 con 0x7f8870004810 2023-10-06T02:28:42.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.809+0000 7f8889599700 1 -- 172.21.15.31:0/1501505743 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f887000c350 con 0x7f8870003a60 2023-10-06T02:28:42.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.809+0000 7f888ad9c700 1 --2- 172.21.15.31:0/1501505743 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88700052e0 0x7f887000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:42.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.809+0000 7f888a59b700 1 --2- 172.21.15.31:0/1501505743 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8870003a60 0x7f8870003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:42.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.809+0000 7f8889d9a700 1 --2- 172.21.15.31:0/1501505743 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8870004810 0x7f8870004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:42.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.810+0000 7f888ad9c700 1 -- 172.21.15.31:0/1501505743 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8870003a60 msgr2=0x7f8870003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:42.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.810+0000 7f888ad9c700 1 --2- 172.21.15.31:0/1501505743 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8870003a60 0x7f8870003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:42.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.810+0000 7f888ad9c700 1 -- 172.21.15.31:0/1501505743 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8870004810 msgr2=0x7f8870004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:42.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.810+0000 7f888ad9c700 1 --2- 172.21.15.31:0/1501505743 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8870004810 0x7f8870004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:42.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.810+0000 7f888ad9c700 1 -- 172.21.15.31:0/1501505743 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8870003570 con 0x7f88700052e0 2023-10-06T02:28:42.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.810+0000 7f888ad9c700 1 -- 172.21.15.31:0/1501505743 reap_dead start 2023-10-06T02:28:42.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.810+0000 7f888ad9c700 1 --2- 172.21.15.31:0/1501505743 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88700052e0 0x7f887000ba60 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f887c001f90 tx=0x7f887c001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98136b7be59fed61 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:42.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.810+0000 7f8889599700 1 -- 172.21.15.31:0/1501505743 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f887c001950 con 0x7f88700052e0 2023-10-06T02:28:42.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.810+0000 7f8889599700 1 -- 172.21.15.31:0/1501505743 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f887c00c8b0 con 0x7f88700052e0 2023-10-06T02:28:42.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.810+0000 7f8889599700 1 -- 172.21.15.31:0/1501505743 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f887c00cc50 con 0x7f88700052e0 2023-10-06T02:28:42.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.811+0000 7f888b59d700 1 -- 172.21.15.31:0/1501505743 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88700052e0 msgr2=0x7f887000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:42.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.811+0000 7f888b59d700 1 --2- 172.21.15.31:0/1501505743 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88700052e0 0x7f887000ba60 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f887c001f90 tx=0x7f887c001640 comp rx=0 tx=0).stop 2023-10-06T02:28:42.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.811+0000 7f888b59d700 1 -- 172.21.15.31:0/1501505743 shutdown_connections 2023-10-06T02:28:42.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.811+0000 7f888b59d700 1 --2- 172.21.15.31:0/1501505743 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88700052e0 0x7f887000ba60 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:42.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.811+0000 7f888b59d700 1 -- 172.21.15.31:0/1501505743 >> 172.21.15.31:0/1501505743 conn(0x7f888406d170 msgr2=0x7f88840cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:42.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.811+0000 7f888b59d700 1 -- 172.21.15.31:0/1501505743 shutdown_connections 2023-10-06T02:28:42.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.811+0000 7f888b59d700 1 -- 172.21.15.31:0/1501505743 wait complete. 2023-10-06T02:28:42.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.812+0000 7f888b59d700 1 Processor -- start 2023-10-06T02:28:42.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.812+0000 7f888b59d700 1 -- start start 2023-10-06T02:28:42.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.813+0000 7f888b59d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88700052e0 0x7f88840c53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:42.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.813+0000 7f888b59d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8884149b40 0x7f88840c3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:42.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.813+0000 7f888b59d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f888413ca30 0x7f88840c63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:42.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.813+0000 7f888b59d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88840c4f30 con 0x7f8884149b40 2023-10-06T02:28:42.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.813+0000 7f888b59d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88840c4320 con 0x7f88700052e0 2023-10-06T02:28:42.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.813+0000 7f888b59d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88840c4600 con 0x7f888413ca30 2023-10-06T02:28:42.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.814+0000 7f8889d9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8884149b40 0x7f88840c3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:42.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.814+0000 7f8889d9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8884149b40 0x7f88840c3a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36876/0 (socket says 172.21.15.31:36876) 2023-10-06T02:28:42.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.814+0000 7f8889d9a700 1 -- 172.21.15.31:0/2541709737 learned_addr learned my addr 172.21.15.31:0/2541709737 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:42.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.814+0000 7f888a59b700 1 --2- 172.21.15.31:0/2541709737 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88700052e0 0x7f88840c53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:42.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.814+0000 7f888ad9c700 1 --2- 172.21.15.31:0/2541709737 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f888413ca30 0x7f88840c63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:42.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.814+0000 7f8889d9a700 1 -- 172.21.15.31:0/2541709737 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f888413ca30 msgr2=0x7f88840c63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:42.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.814+0000 7f8889d9a700 1 --2- 172.21.15.31:0/2541709737 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f888413ca30 0x7f88840c63b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:42.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.814+0000 7f8889d9a700 1 -- 172.21.15.31:0/2541709737 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88700052e0 msgr2=0x7f88840c53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:42.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.814+0000 7f8889d9a700 1 --2- 172.21.15.31:0/2541709737 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88700052e0 0x7f88840c53c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:42.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.814+0000 7f8889d9a700 1 -- 172.21.15.31:0/2541709737 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88840c48e0 con 0x7f8884149b40 2023-10-06T02:28:42.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.814+0000 7f8889d9a700 1 --2- 172.21.15.31:0/2541709737 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8884149b40 0x7f88840c3a40 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f887400fc00 tx=0x7f8874009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:42.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.815+0000 7f887a7fc700 1 -- 172.21.15.31:0/2541709737 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88740072e0 con 0x7f8884149b40 2023-10-06T02:28:42.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.815+0000 7f887a7fc700 1 -- 172.21.15.31:0/2541709737 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8874007460 con 0x7f8884149b40 2023-10-06T02:28:42.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.815+0000 7f887a7fc700 1 -- 172.21.15.31:0/2541709737 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88740057f0 con 0x7f8884149b40 2023-10-06T02:28:42.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.815+0000 7f888b59d700 1 -- 172.21.15.31:0/2541709737 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88840c4bc0 con 0x7f8884149b40 2023-10-06T02:28:42.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.815+0000 7f888b59d700 1 -- 172.21.15.31:0/2541709737 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f888413ced0 con 0x7f8884149b40 2023-10-06T02:28:42.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.816+0000 7f888b59d700 1 -- 172.21.15.31:0/2541709737 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f888413d0c0 con 0x7f8884149b40 2023-10-06T02:28:42.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.817+0000 7f887a7fc700 1 -- 172.21.15.31:0/2541709737 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f8874005b60 con 0x7f8884149b40 2023-10-06T02:28:42.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.818+0000 7f887a7fc700 1 --2- 172.21.15.31:0/2541709737 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f8868067050 0x7f8868069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:42.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.818+0000 7f887a7fc700 1 -- 172.21.15.31:0/2541709737 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f88740849a0 con 0x7f8884149b40 2023-10-06T02:28:42.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.818+0000 7f888a59b700 1 --2- 172.21.15.31:0/2541709737 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f8868067050 0x7f8868069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:42.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.819+0000 7f888a59b700 1 --2- 172.21.15.31:0/2541709737 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f8868067050 0x7f8868069500 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8880006fd0 tx=0x7f8880008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:42.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:42.822+0000 7f887a7fc700 1 -- 172.21.15.31:0/2541709737 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f88740519a0 con 0x7f8884149b40 2023-10-06T02:28:43.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:43.071+0000 7f888b59d700 1 -- 172.21.15.31:0/2541709737 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f888404c3c0 con 0x7f8884149b40 2023-10-06T02:28:43.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:43.071+0000 7f887a7fc700 1 -- 172.21.15.31:0/2541709737 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v71) v1 ==== 53+0+73 (secure 0 0 0) 0x7f8874054fc0 con 0x7f8884149b40 2023-10-06T02:28:43.073 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T02:28:43.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:43.074+0000 7f888b59d700 1 -- 172.21.15.31:0/2541709737 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f8868067050 msgr2=0x7f8868069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:43.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:43.074+0000 7f888b59d700 1 --2- 172.21.15.31:0/2541709737 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f8868067050 0x7f8868069500 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8880006fd0 tx=0x7f8880008040 comp rx=0 tx=0).stop 2023-10-06T02:28:43.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:43.074+0000 7f888b59d700 1 -- 172.21.15.31:0/2541709737 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8884149b40 msgr2=0x7f88840c3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:43.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:43.074+0000 7f888b59d700 1 --2- 172.21.15.31:0/2541709737 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8884149b40 0x7f88840c3a40 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f887400fc00 tx=0x7f8874009040 comp rx=0 tx=0).stop 2023-10-06T02:28:43.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:43.074+0000 7f888b59d700 1 -- 172.21.15.31:0/2541709737 shutdown_connections 2023-10-06T02:28:43.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:43.074+0000 7f888b59d700 1 --2- 172.21.15.31:0/2541709737 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f8868067050 0x7f8868069500 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:43.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:43.074+0000 7f888b59d700 1 --2- 172.21.15.31:0/2541709737 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f888413ca30 0x7f88840c63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:43.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:43.074+0000 7f888b59d700 1 --2- 172.21.15.31:0/2541709737 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88700052e0 0x7f88840c53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:43.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:43.074+0000 7f888b59d700 1 --2- 172.21.15.31:0/2541709737 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8884149b40 0x7f88840c3a40 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:43.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:43.074+0000 7f888b59d700 1 -- 172.21.15.31:0/2541709737 >> 172.21.15.31:0/2541709737 conn(0x7f888406d170 msgr2=0x7f888414eaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:43.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:43.074+0000 7f888b59d700 1 -- 172.21.15.31:0/2541709737 shutdown_connections 2023-10-06T02:28:43.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:43.074+0000 7f888b59d700 1 -- 172.21.15.31:0/2541709737 wait complete. 2023-10-06T02:28:43.093 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T02:28:43.093 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T02:28:43.093 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi031.front.sepia.ceph.com 2023-10-06T02:28:43.093 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T02:28:43.093 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T02:28:43.093 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T02:28:43.093 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T02:28:43.093 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T02:28:43.093 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T02:28:43.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:28:43.094 DEBUG:teuthology.orchestra.run.smithi031:> ip addr 2023-10-06T02:28:43.133 INFO:teuthology.orchestra.run.smithi031.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T02:28:43.133 INFO:teuthology.orchestra.run.smithi031.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T02:28:43.133 INFO:teuthology.orchestra.run.smithi031.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T02:28:43.133 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:28:43.133 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 ::1/128 scope host 2023-10-06T02:28:43.133 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:28:43.133 INFO:teuthology.orchestra.run.smithi031.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T02:28:43.133 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cc brd ff:ff:ff:ff:ff:ff 2023-10-06T02:28:43.134 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f0 2023-10-06T02:28:43.134 INFO:teuthology.orchestra.run.smithi031.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T02:28:43.134 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cd brd ff:ff:ff:ff:ff:ff 2023-10-06T02:28:43.134 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f1 2023-10-06T02:28:43.134 INFO:teuthology.orchestra.run.smithi031.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T02:28:43.134 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:ba brd ff:ff:ff:ff:ff:ff 2023-10-06T02:28:43.134 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f0 2023-10-06T02:28:43.134 INFO:teuthology.orchestra.run.smithi031.stdout: inet 172.21.15.31/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T02:28:43.134 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft 41933sec preferred_lft 41933sec 2023-10-06T02:28:43.134 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::ae1f:6bff:fea5:caba/64 scope link 2023-10-06T02:28:43.135 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:28:43.135 INFO:teuthology.orchestra.run.smithi031.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T02:28:43.135 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:bb brd ff:ff:ff:ff:ff:ff 2023-10-06T02:28:43.135 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f1 2023-10-06T02:28:43.135 INFO:teuthology.orchestra.run.smithi031.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T02:28:43.135 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 16:d7:88:2d:d0:c2 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:28:43.135 INFO:teuthology.orchestra.run.smithi031.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T02:28:43.135 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:28:43.135 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::408:52ff:fe3c:ba8f/64 scope link 2023-10-06T02:28:43.135 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:28:43.135 INFO:teuthology.orchestra.run.smithi031.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T02:28:43.135 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 16:d7:88:2d:d0:c2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T02:28:43.136 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::14d7:88ff:fe2d:d0c2/64 scope link 2023-10-06T02:28:43.136 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:28:43.136 INFO:teuthology.orchestra.run.smithi031.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T02:28:43.136 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 82:16:f1:00:cf:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T02:28:43.136 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::8016:f1ff:fe00:cf2f/64 scope link 2023-10-06T02:28:43.136 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:28:43.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:28:43.136 DEBUG:teuthology.orchestra.run.smithi031:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T02:28:43.234 INFO:teuthology.orchestra.run.smithi031.stdout:1 2023-10-06T02:28:43.237 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:28:43.237 DEBUG:teuthology.orchestra.run.smithi031:> route 2023-10-06T02:28:43.276 INFO:teuthology.orchestra.run.smithi031.stdout:Kernel IP routing table 2023-10-06T02:28:43.276 INFO:teuthology.orchestra.run.smithi031.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T02:28:43.276 INFO:teuthology.orchestra.run.smithi031.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T02:28:43.276 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T02:28:43.277 INFO:teuthology.orchestra.run.smithi031.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T02:28:43.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:28:43.277 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2023-10-06T02:28:43.277 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2023-10-06T02:28:43.277 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T02:28:43.277 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T02:28:43.277 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T02:28:43.277 DEBUG:teuthology.orchestra.run.smithi031:> ') 2023-10-06T02:28:43.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:28:43.476 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2023-10-06T02:28:43.512 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T02:28:43.513 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T02:28:43.513 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T02:28:43.513 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:28:43.513 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set brx.0 up 2023-10-06T02:28:43.615 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T02:28:43.615 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T02:28:43.652 INFO:teuthology.orchestra.run.smithi031.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T02:28:43.653 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T02:28:43.653 DEBUG:teuthology.orchestra.run.smithi031:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T02:28:43.729 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-06T02:28:43.729 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T02:28:43.730 DEBUG:teuthology.orchestra.run.smithi031:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T02:28:43.862 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T02:28:43.863 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T02:28:43.891 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T02:28:43.891 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T02:28:43.892 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.62:3300,172.21.15.62:3301" 2023-10-06T02:28:43.892 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: trying mount with new device syntax: 0@d6fcf512-874b-4259-b90a-30ae070f00a9.cephfs=/ 2023-10-06T02:28:43.892 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.31:3300/172.21.15.62:3300/172.21.15.62:3301" will pass to kernel 2023-10-06T02:28:43.896 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T02:28:43.896 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:28:43.897 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T02:28:43.959 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T02:28:44.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.149+0000 7f131ad9d700 1 Processor -- start 2023-10-06T02:28:44.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.149+0000 7f131ad9d700 1 -- start start 2023-10-06T02:28:44.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.150+0000 7f131ad9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1314159920 0x7f13140ad090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:44.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.150+0000 7f131ad9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13140acc00 con 0x7f1314159550 2023-10-06T02:28:44.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.150+0000 7f131ad9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13140abff0 con 0x7f131415dfd0 2023-10-06T02:28:44.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.150+0000 7f131ad9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13140ac2d0 con 0x7f1314159920 2023-10-06T02:28:44.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.151+0000 7f1319d9b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f1314159550 0x7f13140ceac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:43134/0 (socket says 172.21.15.31:43134) 2023-10-06T02:28:44.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.151+0000 7f1319d9b700 1 -- 172.21.15.31:0/2414051487 learned_addr learned my addr 172.21.15.31:0/2414051487 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:28:44.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.151+0000 7f131a59c700 1 --2- 172.21.15.31:0/2414051487 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1314159920 0x7f13140ad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:44.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.151+0000 7f1318d99700 1 -- 172.21.15.31:0/2414051487 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 281162706 0 0) 0x7f13140acc00 con 0x7f1314159550 2023-10-06T02:28:44.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.151+0000 7f1318d99700 1 -- 172.21.15.31:0/2414051487 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1300003570 con 0x7f1314159550 2023-10-06T02:28:44.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.151+0000 7f1318d99700 1 -- 172.21.15.31:0/2414051487 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3502651658 0 0) 0x7f1300003570 con 0x7f1314159550 2023-10-06T02:28:44.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.151+0000 7f1318d99700 1 -- 172.21.15.31:0/2414051487 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1314159920 msgr2=0x7f13140ad090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:44.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.151+0000 7f1318d99700 1 --2- 172.21.15.31:0/2414051487 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1314159920 0x7f13140ad090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:44.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.151+0000 7f1318d99700 1 -- 172.21.15.31:0/2414051487 >> v1:172.21.15.62:6789/0 conn(0x7f131415dfd0 legacy=0x7f13140aedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:44.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.151+0000 7f1318d99700 1 -- 172.21.15.31:0/2414051487 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13140ac5b0 con 0x7f1314159550 2023-10-06T02:28:44.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.151+0000 7f1318d99700 1 -- 172.21.15.31:0/2414051487 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7f1310001470 con 0x7f1314159550 2023-10-06T02:28:44.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.151+0000 7f1318d99700 1 -- 172.21.15.31:0/2414051487 >> v1:172.21.15.31:6789/0 conn(0x7f1314159550 legacy=0x7f13140ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:44.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.152+0000 7f1318d99700 1 --2- 172.21.15.31:0/2414051487 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1314159920 0x7f1300003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:44.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.152+0000 7f1318d99700 1 --2- 172.21.15.31:0/2414051487 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1300003fd0 0x7f1300004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:44.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.152+0000 7f1318d99700 1 --2- 172.21.15.31:0/2414051487 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1300004d50 0x7f13000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:44.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.152+0000 7f1318d99700 1 -- 172.21.15.31:0/2414051487 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13140ac2d0 con 0x7f1300003fd0 2023-10-06T02:28:44.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.152+0000 7f1318d99700 1 -- 172.21.15.31:0/2414051487 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f130000bba0 con 0x7f1300004d50 2023-10-06T02:28:44.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.152+0000 7f1318d99700 1 -- 172.21.15.31:0/2414051487 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f130000bd60 con 0x7f1314159920 2023-10-06T02:28:44.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.152+0000 7f131959a700 1 --2- 172.21.15.31:0/2414051487 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1300003fd0 0x7f1300004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:44.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.152+0000 7f1319d9b700 1 --2- 172.21.15.31:0/2414051487 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1314159920 0x7f1300003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:44.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.152+0000 7f131a59c700 1 --2- 172.21.15.31:0/2414051487 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1300004d50 0x7f13000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:44.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.152+0000 7f131959a700 1 -- 172.21.15.31:0/2414051487 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1314159920 msgr2=0x7f1300003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:44.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.152+0000 7f131959a700 1 --2- 172.21.15.31:0/2414051487 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1314159920 0x7f1300003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:44.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.152+0000 7f131959a700 1 -- 172.21.15.31:0/2414051487 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1300004d50 msgr2=0x7f13000051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:44.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.152+0000 7f131959a700 1 --2- 172.21.15.31:0/2414051487 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1300004d50 0x7f13000051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:44.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.152+0000 7f131959a700 1 -- 172.21.15.31:0/2414051487 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1300003570 con 0x7f1300003fd0 2023-10-06T02:28:44.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.153+0000 7f131959a700 1 --2- 172.21.15.31:0/2414051487 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1300003fd0 0x7f1300004810 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f13040019a0 tx=0x7f1304007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f31c324133e1d6f1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:44.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.153+0000 7f1318d99700 1 -- 172.21.15.31:0/2414051487 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f130400c5d0 con 0x7f1300003fd0 2023-10-06T02:28:44.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.153+0000 7f1318d99700 1 -- 172.21.15.31:0/2414051487 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f130400c750 con 0x7f1300003fd0 2023-10-06T02:28:44.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.153+0000 7f1318d99700 1 -- 172.21.15.31:0/2414051487 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f130400cb10 con 0x7f1300003fd0 2023-10-06T02:28:44.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.153+0000 7f131ad9d700 1 -- 172.21.15.31:0/2414051487 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1300003fd0 msgr2=0x7f1300004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:44.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.153+0000 7f131ad9d700 1 --2- 172.21.15.31:0/2414051487 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1300003fd0 0x7f1300004810 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f13040019a0 tx=0x7f1304007d10 comp rx=0 tx=0).stop 2023-10-06T02:28:44.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.153+0000 7f131a59c700 1 -- 172.21.15.31:0/2414051487 reap_dead start 2023-10-06T02:28:44.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.153+0000 7f131ad9d700 1 -- 172.21.15.31:0/2414051487 shutdown_connections 2023-10-06T02:28:44.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.153+0000 7f131ad9d700 1 -- 172.21.15.31:0/2414051487 >> 172.21.15.31:0/2414051487 conn(0x7f131406ce80 msgr2=0x7f13140ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:44.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.153+0000 7f131ad9d700 1 -- 172.21.15.31:0/2414051487 shutdown_connections 2023-10-06T02:28:44.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.153+0000 7f131ad9d700 1 -- 172.21.15.31:0/2414051487 wait complete. 2023-10-06T02:28:44.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.155+0000 7f131ad9d700 1 Processor -- start 2023-10-06T02:28:44.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.155+0000 7f131ad9d700 1 -- start start 2023-10-06T02:28:44.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.156+0000 7f131ad9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1300003fd0 0x7f1314150b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:44.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.156+0000 7f131ad9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f131416d330 0x7f131414ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:44.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.156+0000 7f131ad9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1314149c30 0x7f131414a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:44.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.156+0000 7f131ad9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13140b1950 con 0x7f131416d330 2023-10-06T02:28:44.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.156+0000 7f131ad9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f131416db10 con 0x7f1314149c30 2023-10-06T02:28:44.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.156+0000 7f131ad9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f131416ddf0 con 0x7f1300003fd0 2023-10-06T02:28:44.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.156+0000 7f131959a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f131416d330 0x7f131414ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:44.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.156+0000 7f131959a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f131416d330 0x7f131414ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36880/0 (socket says 172.21.15.31:36880) 2023-10-06T02:28:44.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.156+0000 7f131959a700 1 -- 172.21.15.31:0/1361922679 learned_addr learned my addr 172.21.15.31:0/1361922679 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:44.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.156+0000 7f1319d9b700 1 --2- 172.21.15.31:0/1361922679 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1300003fd0 0x7f1314150b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:44.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.157+0000 7f131a59c700 1 --2- 172.21.15.31:0/1361922679 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1314149c30 0x7f131414a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:44.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.157+0000 7f131959a700 1 -- 172.21.15.31:0/1361922679 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1300003fd0 msgr2=0x7f1314150b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:44.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.157+0000 7f131959a700 1 --2- 172.21.15.31:0/1361922679 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1300003fd0 0x7f1314150b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:44.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.157+0000 7f131959a700 1 -- 172.21.15.31:0/1361922679 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1314149c30 msgr2=0x7f131414a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:44.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.157+0000 7f131959a700 1 --2- 172.21.15.31:0/1361922679 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1314149c30 0x7f131414a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:44.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.157+0000 7f131959a700 1 -- 172.21.15.31:0/1361922679 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f131416e0d0 con 0x7f131416d330 2023-10-06T02:28:44.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.157+0000 7f131a59c700 1 --2- 172.21.15.31:0/1361922679 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1314149c30 0x7f131414a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:28:44.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.157+0000 7f131959a700 1 --2- 172.21.15.31:0/1361922679 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f131416d330 0x7f131414ba70 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f130400c3c0 tx=0x7f1304015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:44.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.157+0000 7f1309ffb700 1 -- 172.21.15.31:0/1361922679 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f130400c5d0 con 0x7f131416d330 2023-10-06T02:28:44.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.157+0000 7f1309ffb700 1 -- 172.21.15.31:0/1361922679 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f130400ce80 con 0x7f131416d330 2023-10-06T02:28:44.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.157+0000 7f131ad9d700 1 -- 172.21.15.31:0/1361922679 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13140b1ba0 con 0x7f131416d330 2023-10-06T02:28:44.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.157+0000 7f1309ffb700 1 -- 172.21.15.31:0/1361922679 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f130400cb10 con 0x7f131416d330 2023-10-06T02:28:44.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.158+0000 7f131ad9d700 1 -- 172.21.15.31:0/1361922679 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f131416d7a0 con 0x7f131416d330 2023-10-06T02:28:44.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.159+0000 7f131ad9d700 1 -- 172.21.15.31:0/1361922679 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13140b1de0 con 0x7f131416d330 2023-10-06T02:28:44.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.160+0000 7f1309ffb700 1 -- 172.21.15.31:0/1361922679 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f1304019030 con 0x7f131416d330 2023-10-06T02:28:44.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.160+0000 7f1309ffb700 1 --2- 172.21.15.31:0/1361922679 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f12f8067080 0x7f12f8069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:44.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.160+0000 7f1309ffb700 1 -- 172.21.15.31:0/1361922679 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1304085fe0 con 0x7f131416d330 2023-10-06T02:28:44.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.161+0000 7f1319d9b700 1 --2- 172.21.15.31:0/1361922679 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f12f8067080 0x7f12f8069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:44.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.162+0000 7f1319d9b700 1 --2- 172.21.15.31:0/1361922679 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f12f8067080 0x7f12f8069530 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1310001120 tx=0x7f1310001da0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:44.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.165+0000 7f1309ffb700 1 -- 172.21.15.31:0/1361922679 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f13140b1de0 con 0x7f131416d330 2023-10-06T02:28:44.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.411+0000 7f131ad9d700 1 -- 172.21.15.31:0/1361922679 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f131404c3c0 con 0x7f131416d330 2023-10-06T02:28:44.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.411+0000 7f1309ffb700 1 -- 172.21.15.31:0/1361922679 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v71) v1 ==== 53+0+73 (secure 0 0 0) 0x7f1304017070 con 0x7f131416d330 2023-10-06T02:28:44.412 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T02:28:44.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.413+0000 7f131ad9d700 1 -- 172.21.15.31:0/1361922679 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f12f8067080 msgr2=0x7f12f8069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:44.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.413+0000 7f131ad9d700 1 --2- 172.21.15.31:0/1361922679 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f12f8067080 0x7f12f8069530 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1310001120 tx=0x7f1310001da0 comp rx=0 tx=0).stop 2023-10-06T02:28:44.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.413+0000 7f131ad9d700 1 -- 172.21.15.31:0/1361922679 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f131416d330 msgr2=0x7f131414ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:44.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.413+0000 7f131ad9d700 1 --2- 172.21.15.31:0/1361922679 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f131416d330 0x7f131414ba70 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f130400c3c0 tx=0x7f1304015000 comp rx=0 tx=0).stop 2023-10-06T02:28:44.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.413+0000 7f131ad9d700 1 -- 172.21.15.31:0/1361922679 shutdown_connections 2023-10-06T02:28:44.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.413+0000 7f131ad9d700 1 --2- 172.21.15.31:0/1361922679 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1314149c30 0x7f131414a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:44.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.413+0000 7f131ad9d700 1 --2- 172.21.15.31:0/1361922679 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f12f8067080 0x7f12f8069530 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:44.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.413+0000 7f131ad9d700 1 --2- 172.21.15.31:0/1361922679 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1300003fd0 0x7f1314150b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:44.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.413+0000 7f131ad9d700 1 --2- 172.21.15.31:0/1361922679 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f131416d330 0x7f131414ba70 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:44.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.413+0000 7f131ad9d700 1 -- 172.21.15.31:0/1361922679 >> 172.21.15.31:0/1361922679 conn(0x7f131406ce80 msgr2=0x7f131414d330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:44.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.413+0000 7f131ad9d700 1 -- 172.21.15.31:0/1361922679 shutdown_connections 2023-10-06T02:28:44.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:44.413+0000 7f131ad9d700 1 -- 172.21.15.31:0/1361922679 wait complete. 2023-10-06T02:28:44.431 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T02:28:44.431 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T02:28:44.431 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi062.front.sepia.ceph.com 2023-10-06T02:28:44.432 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-06T02:28:44.432 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-06T02:28:44.432 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T02:28:44.432 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T02:28:44.432 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T02:28:44.432 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T02:28:44.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:28:44.432 DEBUG:teuthology.orchestra.run.smithi062:> ip addr 2023-10-06T02:28:44.475 INFO:teuthology.orchestra.run.smithi062.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T02:28:44.475 INFO:teuthology.orchestra.run.smithi062.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T02:28:44.475 INFO:teuthology.orchestra.run.smithi062.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T02:28:44.475 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:28:44.475 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 ::1/128 scope host 2023-10-06T02:28:44.475 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:28:44.475 INFO:teuthology.orchestra.run.smithi062.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T02:28:44.475 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:14 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:28:44.476 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f0 2023-10-06T02:28:44.476 INFO:teuthology.orchestra.run.smithi062.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T02:28:44.476 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:15 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:28:44.476 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f1 2023-10-06T02:28:44.476 INFO:teuthology.orchestra.run.smithi062.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T02:28:44.476 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c2 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:28:44.476 INFO:teuthology.orchestra.run.smithi062.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T02:28:44.476 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c3 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:28:44.476 INFO:teuthology.orchestra.run.smithi062.stdout: inet 172.21.15.62/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T02:28:44.476 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft 41925sec preferred_lft 41925sec 2023-10-06T02:28:44.476 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::ec4:7aff:febb:d1c3/64 scope link 2023-10-06T02:28:44.476 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:28:44.477 INFO:teuthology.orchestra.run.smithi062.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T02:28:44.477 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether ae:42:a8:dd:f7:9b brd ff:ff:ff:ff:ff:ff 2023-10-06T02:28:44.477 INFO:teuthology.orchestra.run.smithi062.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T02:28:44.477 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:28:44.477 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::1098:a4ff:fe0d:389/64 scope link 2023-10-06T02:28:44.477 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:28:44.477 INFO:teuthology.orchestra.run.smithi062.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T02:28:44.477 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether fe:60:2b:8e:af:28 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T02:28:44.477 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::fc60:2bff:fe8e:af28/64 scope link 2023-10-06T02:28:44.477 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:28:44.477 INFO:teuthology.orchestra.run.smithi062.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T02:28:44.477 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether ae:42:a8:dd:f7:9b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T02:28:44.478 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::ac42:a8ff:fedd:f79b/64 scope link 2023-10-06T02:28:44.478 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:28:44.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:28:44.478 DEBUG:teuthology.orchestra.run.smithi062:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T02:28:44.578 INFO:teuthology.orchestra.run.smithi062.stdout:1 2023-10-06T02:28:44.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:28:44.581 DEBUG:teuthology.orchestra.run.smithi062:> route 2023-10-06T02:28:44.621 INFO:teuthology.orchestra.run.smithi062.stdout:Kernel IP routing table 2023-10-06T02:28:44.622 INFO:teuthology.orchestra.run.smithi062.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T02:28:44.622 INFO:teuthology.orchestra.run.smithi062.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T02:28:44.622 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T02:28:44.622 INFO:teuthology.orchestra.run.smithi062.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T02:28:44.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:28:44.622 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2023-10-06T02:28:44.622 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2023-10-06T02:28:44.622 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T02:28:44.622 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T02:28:44.623 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T02:28:44.623 DEBUG:teuthology.orchestra.run.smithi062:> ') 2023-10-06T02:28:44.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:28:44.820 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2023-10-06T02:28:44.856 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-06T02:28:44.856 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T02:28:44.857 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T02:28:44.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:28:44.857 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set brx.0 up 2023-10-06T02:28:44.960 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T02:28:44.960 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-06T02:28:44.996 INFO:teuthology.orchestra.run.smithi062.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-06T02:28:44.997 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T02:28:44.997 DEBUG:teuthology.orchestra.run.smithi062:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-06T02:28:45.072 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-06T02:28:45.073 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T02:28:45.073 DEBUG:teuthology.orchestra.run.smithi062:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T02:28:45.204 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T02:28:45.205 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T02:28:45.233 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T02:28:45.233 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T02:28:45.233 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.62:3300,172.21.15.62:3301" 2023-10-06T02:28:45.233 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: trying mount with new device syntax: 1@d6fcf512-874b-4259-b90a-30ae070f00a9.cephfs=/ 2023-10-06T02:28:45.233 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.31:3300/172.21.15.62:3300/172.21.15.62:3301" will pass to kernel 2023-10-06T02:28:45.237 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T02:28:45.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:28:45.238 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-06T02:28:45.299 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-10-06T02:28:45.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:45.502+0000 7f8a5df9d700 1 Processor -- start 2023-10-06T02:28:45.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:45.503+0000 7f8a5df9d700 1 -- start start 2023-10-06T02:28:45.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:45.504+0000 7f8a5df9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8a5815cb90 0x7f8a580c7870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:45.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:45.504+0000 7f8a5df9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8a5815dc10 0x7f8a5806c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:45.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:45.504+0000 7f8a5df9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8a5806cbe0 0x7f8a580c3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:45.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:45.504+0000 7f8a5df9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a580ccdf0 con 0x7f8a5806cbe0 2023-10-06T02:28:45.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:45.504+0000 7f8a5df9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a580c9b20 con 0x7f8a5815cb90 2023-10-06T02:28:45.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:45.504+0000 7f8a5df9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a5806d040 con 0x7f8a5815dc10 2023-10-06T02:28:45.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:45.504+0000 7f8a5d79c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8a5806cbe0 0x7f8a580c3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:45.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:45.504+0000 7f8a5d79c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8a5806cbe0 0x7f8a580c3430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36888/0 (socket says 172.21.15.31:36888) 2023-10-06T02:28:45.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:45.504+0000 7f8a5d79c700 1 -- 172.21.15.31:0/1107947432 learned_addr learned my addr 172.21.15.31:0/1107947432 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:45.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:45.504+0000 7f8a57fff700 1 --2- 172.21.15.31:0/1107947432 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8a5815dc10 0x7f8a5806c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:45.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:45.504+0000 7f8a5cf9b700 1 --2- 172.21.15.31:0/1107947432 >> v2:172.21.15.62:3300/0 conn(0x7f8a5815cb90 0x7f8a580c7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:45.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:45.505+0000 7f8a5d79c700 1 -- 172.21.15.31:0/1107947432 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8a5815dc10 msgr2=0x7f8a5806c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:45.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:45.505+0000 7f8a5d79c700 1 --2- 172.21.15.31:0/1107947432 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8a5815dc10 0x7f8a5806c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:45.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:45.505+0000 7f8a5d79c700 1 -- 172.21.15.31:0/1107947432 >> v2:172.21.15.62:3300/0 conn(0x7f8a5815cb90 msgr2=0x7f8a580c7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:45.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:45.505+0000 7f8a5d79c700 1 --2- 172.21.15.31:0/1107947432 >> v2:172.21.15.62:3300/0 conn(0x7f8a5815cb90 0x7f8a580c7870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:45.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:45.505+0000 7f8a5d79c700 1 -- 172.21.15.31:0/1107947432 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a580c2fa0 con 0x7f8a5806cbe0 2023-10-06T02:28:45.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:45.505+0000 7f8a5cf9b700 1 --2- 172.21.15.31:0/1107947432 >> v2:172.21.15.62:3300/0 conn(0x7f8a5815cb90 0x7f8a580c7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:28:45.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:45.505+0000 7f8a5d79c700 1 --2- 172.21.15.31:0/1107947432 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a5806cbe0 0x7f8a580c3430 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f8a58063580 tx=0x7f8a50009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c1f2687b1ba30d1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:45.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:45.505+0000 7f8a577fe700 1 -- 172.21.15.31:0/1107947432 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a5000e070 con 0x7f8a5806cbe0 2023-10-06T02:28:45.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:45.505+0000 7f8a577fe700 1 -- 172.21.15.31:0/1107947432 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a5000ba50 con 0x7f8a5806cbe0 2023-10-06T02:28:45.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:45.505+0000 7f8a577fe700 1 -- 172.21.15.31:0/1107947432 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a500092e0 con 0x7f8a5806cbe0 2023-10-06T02:28:45.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:45.506+0000 7f8a5df9d700 1 -- 172.21.15.31:0/1107947432 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a5806cbe0 msgr2=0x7f8a580c3430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:45.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:45.506+0000 7f8a5df9d700 1 --2- 172.21.15.31:0/1107947432 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a5806cbe0 0x7f8a580c3430 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f8a58063580 tx=0x7f8a50009840 comp rx=0 tx=0).stop 2023-10-06T02:28:45.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:45.506+0000 7f8a5df9d700 1 -- 172.21.15.31:0/1107947432 shutdown_connections 2023-10-06T02:28:45.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:45.506+0000 7f8a5df9d700 1 --2- 172.21.15.31:0/1107947432 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a5806cbe0 0x7f8a580c3430 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:45.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:45.506+0000 7f8a5df9d700 1 --2- 172.21.15.31:0/1107947432 >> v2:172.21.15.62:3300/0 conn(0x7f8a5815cb90 0x7f8a580c7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:45.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:45.506+0000 7f8a5df9d700 1 --2- 172.21.15.31:0/1107947432 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8a5815dc10 0x7f8a5806c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:45.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:45.506+0000 7f8a5df9d700 1 -- 172.21.15.31:0/1107947432 >> 172.21.15.31:0/1107947432 conn(0x7f8a580cd230 msgr2=0x7f8a580cd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:45.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:45.506+0000 7f8a5df9d700 1 -- 172.21.15.31:0/1107947432 shutdown_connections 2023-10-06T02:28:45.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:45.506+0000 7f8a5df9d700 1 -- 172.21.15.31:0/1107947432 wait complete. 2023-10-06T02:28:45.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:45.507+0000 7f8a5df9d700 1 Processor -- start 2023-10-06T02:28:45.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:45.507+0000 7f8a5df9d700 1 -- start start 2023-10-06T02:28:45.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:45.508+0000 7f8a5df9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8a5806cbe0 0x7f8a5813be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:45.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:45.508+0000 7f8a5df9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a5815cb90 0x7f8a580c42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:45.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:45.508+0000 7f8a5df9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a5815dc10 0x7f8a580c7f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:45.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:45.508+0000 7f8a5df9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a58169370 con 0x7f8a5815dc10 2023-10-06T02:28:45.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:45.508+0000 7f8a5df9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a580c58d0 con 0x7f8a5815cb90 2023-10-06T02:28:45.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:45.508+0000 7f8a5df9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a5813c3a0 con 0x7f8a5806cbe0 2023-10-06T02:28:45.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:45.509+0000 7f8a5d79c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a5815dc10 0x7f8a580c7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:45.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:45.509+0000 7f8a5cf9b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8a5806cbe0 0x7f8a5813be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:45.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:45.509+0000 7f8a5d79c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a5815dc10 0x7f8a580c7f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36902/0 (socket says 172.21.15.31:36902) 2023-10-06T02:28:45.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:45.509+0000 7f8a5d79c700 1 -- 172.21.15.31:0/42651443 learned_addr learned my addr 172.21.15.31:0/42651443 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:45.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:45.509+0000 7f8a57fff700 1 --2- 172.21.15.31:0/42651443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a5815cb90 0x7f8a580c42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:45.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:45.509+0000 7f8a5d79c700 1 -- 172.21.15.31:0/42651443 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8a5806cbe0 msgr2=0x7f8a5813be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:45.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:45.509+0000 7f8a5d79c700 1 --2- 172.21.15.31:0/42651443 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8a5806cbe0 0x7f8a5813be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:45.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:45.509+0000 7f8a5d79c700 1 -- 172.21.15.31:0/42651443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a5815cb90 msgr2=0x7f8a580c42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:45.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:45.509+0000 7f8a5d79c700 1 --2- 172.21.15.31:0/42651443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a5815cb90 0x7f8a580c42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:45.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:45.509+0000 7f8a5d79c700 1 -- 172.21.15.31:0/42651443 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a50009ae0 con 0x7f8a5815dc10 2023-10-06T02:28:45.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:45.509+0000 7f8a5d79c700 1 --2- 172.21.15.31:0/42651443 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a5815dc10 0x7f8a580c7f60 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f8a58063720 tx=0x7f8a50018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:45.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:45.510+0000 7f8a54ff9700 1 -- 172.21.15.31:0/42651443 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a500032f0 con 0x7f8a5815dc10 2023-10-06T02:28:45.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:45.510+0000 7f8a54ff9700 1 -- 172.21.15.31:0/42651443 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a50008d20 con 0x7f8a5815dc10 2023-10-06T02:28:45.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:45.510+0000 7f8a54ff9700 1 -- 172.21.15.31:0/42651443 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a50026690 con 0x7f8a5815dc10 2023-10-06T02:28:45.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:45.510+0000 7f8a5df9d700 1 -- 172.21.15.31:0/42651443 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a580c4cc0 con 0x7f8a5815dc10 2023-10-06T02:28:45.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:45.510+0000 7f8a5df9d700 1 -- 172.21.15.31:0/42651443 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a580c5280 con 0x7f8a5815dc10 2023-10-06T02:28:45.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:45.511+0000 7f8a5df9d700 1 -- 172.21.15.31:0/42651443 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a5804c3c0 con 0x7f8a5815dc10 2023-10-06T02:28:45.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:45.512+0000 7f8a54ff9700 1 -- 172.21.15.31:0/42651443 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f8a50003040 con 0x7f8a5815dc10 2023-10-06T02:28:45.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:45.513+0000 7f8a54ff9700 1 --2- 172.21.15.31:0/42651443 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f8a3c067050 0x7f8a3c069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:45.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:45.513+0000 7f8a54ff9700 1 -- 172.21.15.31:0/42651443 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8a500861e0 con 0x7f8a5815dc10 2023-10-06T02:28:45.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:45.513+0000 7f8a5cf9b700 1 --2- 172.21.15.31:0/42651443 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f8a3c067050 0x7f8a3c069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:45.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:45.514+0000 7f8a5cf9b700 1 --2- 172.21.15.31:0/42651443 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f8a3c067050 0x7f8a3c069500 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c00adc0 tx=0x7f8a4c006f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:45.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:45.518+0000 7f8a54ff9700 1 -- 172.21.15.31:0/42651443 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8a500530d0 con 0x7f8a5815dc10 2023-10-06T02:28:45.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:45.764+0000 7f8a5df9d700 1 -- 172.21.15.31:0/42651443 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f8a580c2400 con 0x7f8a5815dc10 2023-10-06T02:28:46.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:46.477+0000 7f8a54ff9700 1 -- 172.21.15.31:0/42651443 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v72) v1 ==== 136+0+0 (secure 0 0 0) 0x7f8a500566f0 con 0x7f8a5815dc10 2023-10-06T02:28:46.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:46.480+0000 7f8a5df9d700 1 -- 172.21.15.31:0/42651443 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f8a3c067050 msgr2=0x7f8a3c069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:46.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:46.480+0000 7f8a5df9d700 1 --2- 172.21.15.31:0/42651443 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f8a3c067050 0x7f8a3c069500 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c00adc0 tx=0x7f8a4c006f90 comp rx=0 tx=0).stop 2023-10-06T02:28:46.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:46.480+0000 7f8a5df9d700 1 -- 172.21.15.31:0/42651443 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a5815dc10 msgr2=0x7f8a580c7f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:46.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:46.480+0000 7f8a5df9d700 1 --2- 172.21.15.31:0/42651443 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a5815dc10 0x7f8a580c7f60 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f8a58063720 tx=0x7f8a50018000 comp rx=0 tx=0).stop 2023-10-06T02:28:46.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:46.480+0000 7f8a5df9d700 1 -- 172.21.15.31:0/42651443 shutdown_connections 2023-10-06T02:28:46.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:46.480+0000 7f8a5df9d700 1 --2- 172.21.15.31:0/42651443 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a5815dc10 0x7f8a580c7f60 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:46.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:46.480+0000 7f8a5df9d700 1 --2- 172.21.15.31:0/42651443 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f8a3c067050 0x7f8a3c069500 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:46.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:46.480+0000 7f8a5df9d700 1 --2- 172.21.15.31:0/42651443 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8a5806cbe0 0x7f8a5813be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:46.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:46.480+0000 7f8a5df9d700 1 --2- 172.21.15.31:0/42651443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a5815cb90 0x7f8a580c42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:46.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:46.480+0000 7f8a5df9d700 1 -- 172.21.15.31:0/42651443 >> 172.21.15.31:0/42651443 conn(0x7f8a580cd230 msgr2=0x7f8a5815ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:46.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:46.480+0000 7f8a5df9d700 1 -- 172.21.15.31:0/42651443 shutdown_connections 2023-10-06T02:28:46.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:46.480+0000 7f8a5df9d700 1 -- 172.21.15.31:0/42651443 wait complete. 2023-10-06T02:28:46.501 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-06T02:28:46.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:46.692+0000 7f5c5359e700 1 Processor -- start 2023-10-06T02:28:46.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:46.692+0000 7f5c5359e700 1 -- start start 2023-10-06T02:28:46.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:46.693+0000 7f5c5359e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5c5413fef0 0x7f5c5415a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:46.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:46.694+0000 7f5c5359e700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f5c5406aaa0 0x7f5c540c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:46.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:46.694+0000 7f5c5359e700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c5415ae70 con 0x7f5c5413fb20 2023-10-06T02:28:46.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:46.694+0000 7f5c5359e700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c5406af60 con 0x7f5c5413fef0 2023-10-06T02:28:46.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:46.694+0000 7f5c5359e700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c5406b240 con 0x7f5c5406aaa0 2023-10-06T02:28:46.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:46.694+0000 7f5c51d9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5c5413fef0 0x7f5c5415a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:46.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:46.694+0000 7f5c51d9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5c5413fef0 0x7f5c5415a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36904/0 (socket says 172.21.15.31:36904) 2023-10-06T02:28:46.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:46.694+0000 7f5c51d9b700 1 -- 172.21.15.31:0/3147078710 learned_addr learned my addr 172.21.15.31:0/3147078710 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:46.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:46.694+0000 7f5c52d9d700 1 --2- 172.21.15.31:0/3147078710 >> v2:172.21.15.62:3300/0 conn(0x7f5c5406aaa0 0x7f5c540c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:46.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:46.694+0000 7f5c51d9b700 1 -- 172.21.15.31:0/3147078710 >> v2:172.21.15.62:3300/0 conn(0x7f5c5406aaa0 msgr2=0x7f5c540c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:46.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:46.694+0000 7f5c51d9b700 1 --2- 172.21.15.31:0/3147078710 >> v2:172.21.15.62:3300/0 conn(0x7f5c5406aaa0 0x7f5c540c9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:46.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:46.694+0000 7f5c51d9b700 1 -- 172.21.15.31:0/3147078710 >> v1:172.21.15.62:6789/0 conn(0x7f5c5413fb20 legacy=0x7f5c540c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:46.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:46.694+0000 7f5c51d9b700 1 -- 172.21.15.31:0/3147078710 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c5406b520 con 0x7f5c5413fef0 2023-10-06T02:28:46.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:46.695+0000 7f5c51d9b700 1 --2- 172.21.15.31:0/3147078710 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c5413fef0 0x7f5c5415a930 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f5c54069d40 tx=0x7f5c4800b4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dc9d348e377db7f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:46.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:46.695+0000 7f5c5159a700 1 -- 172.21.15.31:0/3147078710 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c48005c10 con 0x7f5c5413fef0 2023-10-06T02:28:46.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:46.695+0000 7f5c5159a700 1 -- 172.21.15.31:0/3147078710 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c48005d70 con 0x7f5c5413fef0 2023-10-06T02:28:46.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:46.695+0000 7f5c5159a700 1 -- 172.21.15.31:0/3147078710 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c4800db80 con 0x7f5c5413fef0 2023-10-06T02:28:46.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:46.695+0000 7f5c5359e700 1 -- 172.21.15.31:0/3147078710 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c5413fef0 msgr2=0x7f5c5415a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:46.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:46.695+0000 7f5c5359e700 1 --2- 172.21.15.31:0/3147078710 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c5413fef0 0x7f5c5415a930 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f5c54069d40 tx=0x7f5c4800b4c0 comp rx=0 tx=0).stop 2023-10-06T02:28:46.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:46.696+0000 7f5c5359e700 1 -- 172.21.15.31:0/3147078710 shutdown_connections 2023-10-06T02:28:46.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:46.696+0000 7f5c5359e700 1 --2- 172.21.15.31:0/3147078710 >> v2:172.21.15.62:3300/0 conn(0x7f5c5406aaa0 0x7f5c540c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:46.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:46.696+0000 7f5c5359e700 1 --2- 172.21.15.31:0/3147078710 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c5413fef0 0x7f5c5415a930 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:46.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:46.696+0000 7f5c5359e700 1 -- 172.21.15.31:0/3147078710 >> 172.21.15.31:0/3147078710 conn(0x7f5c5406d170 msgr2=0x7f5c540cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:46.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:46.696+0000 7f5c5359e700 1 -- 172.21.15.31:0/3147078710 shutdown_connections 2023-10-06T02:28:46.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:46.696+0000 7f5c5359e700 1 -- 172.21.15.31:0/3147078710 wait complete. 2023-10-06T02:28:46.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:46.697+0000 7f5c5359e700 1 Processor -- start 2023-10-06T02:28:46.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:46.697+0000 7f5c5359e700 1 -- start start 2023-10-06T02:28:46.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:46.698+0000 7f5c5359e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c5406aaa0 0x7f5c54158dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:46.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:46.698+0000 7f5c5359e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5c5413fb20 0x7f5c54169950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:46.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:46.698+0000 7f5c5359e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c5413fef0 0x7f5c540ac710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:46.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:46.698+0000 7f5c5359e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c54159310 con 0x7f5c5406aaa0 2023-10-06T02:28:46.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:46.698+0000 7f5c5359e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c540ad670 con 0x7f5c5413fef0 2023-10-06T02:28:46.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:46.698+0000 7f5c5359e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c54156df0 con 0x7f5c5413fb20 2023-10-06T02:28:46.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:46.698+0000 7f5c5259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c5406aaa0 0x7f5c54158dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:46.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:46.698+0000 7f5c5259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c5406aaa0 0x7f5c54158dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36912/0 (socket says 172.21.15.31:36912) 2023-10-06T02:28:46.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:46.698+0000 7f5c5259c700 1 -- 172.21.15.31:0/3383179393 learned_addr learned my addr 172.21.15.31:0/3383179393 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:46.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:46.699+0000 7f5c5259c700 1 -- 172.21.15.31:0/3383179393 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5c5413fb20 msgr2=0x7f5c54169950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:46.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:46.699+0000 7f5c5259c700 1 --2- 172.21.15.31:0/3383179393 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5c5413fb20 0x7f5c54169950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:46.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:46.699+0000 7f5c5259c700 1 -- 172.21.15.31:0/3383179393 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c5413fef0 msgr2=0x7f5c540ac710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:46.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:46.699+0000 7f5c5259c700 1 --2- 172.21.15.31:0/3383179393 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c5413fef0 0x7f5c540ac710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:46.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:46.699+0000 7f5c5259c700 1 -- 172.21.15.31:0/3383179393 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c4800b720 con 0x7f5c5406aaa0 2023-10-06T02:28:46.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:46.699+0000 7f5c5259c700 1 --2- 172.21.15.31:0/3383179393 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c5406aaa0 0x7f5c54158dd0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f5c4400c5c0 tx=0x7f5c4400c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:46.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:46.699+0000 7f5c427fc700 1 -- 172.21.15.31:0/3383179393 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c44011590 con 0x7f5c5406aaa0 2023-10-06T02:28:46.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:46.699+0000 7f5c427fc700 1 -- 172.21.15.31:0/3383179393 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c44011710 con 0x7f5c5406aaa0 2023-10-06T02:28:46.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:46.699+0000 7f5c5359e700 1 -- 172.21.15.31:0/3383179393 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c540ad0b0 con 0x7f5c5406aaa0 2023-10-06T02:28:46.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:46.699+0000 7f5c427fc700 1 -- 172.21.15.31:0/3383179393 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c44011ad0 con 0x7f5c5406aaa0 2023-10-06T02:28:46.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:46.699+0000 7f5c5359e700 1 -- 172.21.15.31:0/3383179393 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c54156a80 con 0x7f5c5406aaa0 2023-10-06T02:28:46.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:46.700+0000 7f5c5359e700 1 -- 172.21.15.31:0/3383179393 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c5404c3c0 con 0x7f5c5406aaa0 2023-10-06T02:28:46.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:46.701+0000 7f5c427fc700 1 -- 172.21.15.31:0/3383179393 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f5c4401b3b0 con 0x7f5c5406aaa0 2023-10-06T02:28:46.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:46.702+0000 7f5c427fc700 1 --2- 172.21.15.31:0/3383179393 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5c34067050 0x7f5c34069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:46.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:46.702+0000 7f5c427fc700 1 -- 172.21.15.31:0/3383179393 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f5c44083ca0 con 0x7f5c5406aaa0 2023-10-06T02:28:46.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:46.703+0000 7f5c51d9b700 1 --2- 172.21.15.31:0/3383179393 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5c34067050 0x7f5c34069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:46.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:46.703+0000 7f5c51d9b700 1 --2- 172.21.15.31:0/3383179393 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5c34067050 0x7f5c34069500 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5c54069d40 tx=0x7f5c4801a000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:46.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:46.706+0000 7f5c427fc700 1 -- 172.21.15.31:0/3383179393 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5c44050c00 con 0x7f5c5406aaa0 2023-10-06T02:28:46.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:46.936+0000 7f5c5359e700 1 -- 172.21.15.31:0/3383179393 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f5c54156f30 con 0x7f5c34067050 2023-10-06T02:28:46.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:46.939+0000 7f5c427fc700 1 -- 172.21.15.31:0/3383179393 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f5c4800d880 con 0x7f5c34067050 2023-10-06T02:28:46.939 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2023-10-06T02:28:46.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:28:46.940 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephfs" 2023-10-06T02:28:46.940 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:28:46.940 INFO:teuthology.orchestra.run.smithi031.stdout:] 2023-10-06T02:28:46.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:46.941+0000 7f5c5359e700 1 -- 172.21.15.31:0/3383179393 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5c34067050 msgr2=0x7f5c34069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:46.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:46.941+0000 7f5c5359e700 1 --2- 172.21.15.31:0/3383179393 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5c34067050 0x7f5c34069500 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5c54069d40 tx=0x7f5c4801a000 comp rx=0 tx=0).stop 2023-10-06T02:28:46.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:46.941+0000 7f5c5359e700 1 -- 172.21.15.31:0/3383179393 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c5406aaa0 msgr2=0x7f5c54158dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:46.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:46.941+0000 7f5c5359e700 1 --2- 172.21.15.31:0/3383179393 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c5406aaa0 0x7f5c54158dd0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f5c4400c5c0 tx=0x7f5c4400c980 comp rx=0 tx=0).stop 2023-10-06T02:28:46.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:46.941+0000 7f5c5359e700 1 -- 172.21.15.31:0/3383179393 shutdown_connections 2023-10-06T02:28:46.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:46.941+0000 7f5c5359e700 1 --2- 172.21.15.31:0/3383179393 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c5413fef0 0x7f5c540ac710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:46.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:46.941+0000 7f5c5359e700 1 --2- 172.21.15.31:0/3383179393 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c5406aaa0 0x7f5c54158dd0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:46.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:46.941+0000 7f5c5359e700 1 --2- 172.21.15.31:0/3383179393 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5c34067050 0x7f5c34069500 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:46.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:46.941+0000 7f5c5359e700 1 --2- 172.21.15.31:0/3383179393 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5c5413fb20 0x7f5c54169950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:46.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:46.941+0000 7f5c5359e700 1 -- 172.21.15.31:0/3383179393 >> 172.21.15.31:0/3383179393 conn(0x7f5c5406d170 msgr2=0x7f5c540cbd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:46.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:46.941+0000 7f5c5359e700 1 -- 172.21.15.31:0/3383179393 shutdown_connections 2023-10-06T02:28:46.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:46.941+0000 7f5c5359e700 1 -- 172.21.15.31:0/3383179393 wait complete. 2023-10-06T02:28:46.959 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T02:28:47.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.148+0000 7f46caf9d700 1 Processor -- start 2023-10-06T02:28:47.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.148+0000 7f46caf9d700 1 -- start start 2023-10-06T02:28:47.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.150+0000 7f46caf9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f46c4142c00 0x7f46c406b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:47.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.150+0000 7f46caf9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f46c40c7f90 0x7f46c406c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:47.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.150+0000 7f46caf9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46c40ca480 con 0x7f46c406bd20 2023-10-06T02:28:47.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.150+0000 7f46caf9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46c415e240 con 0x7f46c4142c00 2023-10-06T02:28:47.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.150+0000 7f46caf9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46c406b350 con 0x7f46c40c7f90 2023-10-06T02:28:47.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.150+0000 7f46c9f9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f46c4142c00 0x7f46c406b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:47.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.150+0000 7f46c9f9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f46c4142c00 0x7f46c406b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36922/0 (socket says 172.21.15.31:36922) 2023-10-06T02:28:47.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.150+0000 7f46c9f9b700 1 -- 172.21.15.31:0/1185787266 learned_addr learned my addr 172.21.15.31:0/1185787266 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:47.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.150+0000 7f46c979a700 1 --1- 172.21.15.31:0/1185787266 >> v1:172.21.15.31:6789/0 conn(0x7f46c406bd20 0x7f46c40c7880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:43146/0 (socket says 172.21.15.31:43146) 2023-10-06T02:28:47.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.150+0000 7f46ca79c700 1 --2- 172.21.15.31:0/1185787266 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f46c40c7f90 0x7f46c406c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:47.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.151+0000 7f46c9f9b700 1 -- 172.21.15.31:0/1185787266 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f46c40c7f90 msgr2=0x7f46c406c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:47.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.151+0000 7f46c9f9b700 1 --2- 172.21.15.31:0/1185787266 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f46c40c7f90 0x7f46c406c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:47.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.151+0000 7f46c9f9b700 1 -- 172.21.15.31:0/1185787266 >> v1:172.21.15.31:6789/0 conn(0x7f46c406bd20 legacy=0x7f46c40c7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:47.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.151+0000 7f46c9f9b700 1 -- 172.21.15.31:0/1185787266 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46c415df60 con 0x7f46c4142c00 2023-10-06T02:28:47.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.151+0000 7f46c8f99700 1 -- 172.21.15.31:0/1185787266 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4184763284 0 0) 0x7f46c40ca480 con 0x7f46c406bd20 2023-10-06T02:28:47.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.151+0000 7f46c9f9b700 1 --2- 172.21.15.31:0/1185787266 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46c4142c00 0x7f46c406b7e0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f46c00099b0 tx=0x7f46c0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c570b1c93d909a1d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:47.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.151+0000 7f46c8f99700 1 -- 172.21.15.31:0/1185787266 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c0009ea0 con 0x7f46c4142c00 2023-10-06T02:28:47.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.151+0000 7f46c8f99700 1 -- 172.21.15.31:0/1185787266 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46c000b860 con 0x7f46c4142c00 2023-10-06T02:28:47.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.151+0000 7f46c8f99700 1 -- 172.21.15.31:0/1185787266 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c000bc00 con 0x7f46c4142c00 2023-10-06T02:28:47.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.152+0000 7f46caf9d700 1 -- 172.21.15.31:0/1185787266 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46c4142c00 msgr2=0x7f46c406b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:47.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.152+0000 7f46caf9d700 1 --2- 172.21.15.31:0/1185787266 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46c4142c00 0x7f46c406b7e0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f46c00099b0 tx=0x7f46c0009250 comp rx=0 tx=0).stop 2023-10-06T02:28:47.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.152+0000 7f46caf9d700 1 -- 172.21.15.31:0/1185787266 shutdown_connections 2023-10-06T02:28:47.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.152+0000 7f46caf9d700 1 --2- 172.21.15.31:0/1185787266 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46c4142c00 0x7f46c406b7e0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:47.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.152+0000 7f46caf9d700 1 --2- 172.21.15.31:0/1185787266 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f46c40c7f90 0x7f46c406c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:47.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.152+0000 7f46caf9d700 1 -- 172.21.15.31:0/1185787266 >> 172.21.15.31:0/1185787266 conn(0x7f46c40675a0 msgr2=0x7f46c40679a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:47.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.152+0000 7f46caf9d700 1 -- 172.21.15.31:0/1185787266 shutdown_connections 2023-10-06T02:28:47.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.152+0000 7f46caf9d700 1 -- 172.21.15.31:0/1185787266 wait complete. 2023-10-06T02:28:47.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.153+0000 7f46caf9d700 1 Processor -- start 2023-10-06T02:28:47.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.153+0000 7f46caf9d700 1 -- start start 2023-10-06T02:28:47.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.154+0000 7f46caf9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46c406bd20 0x7f46c40b29a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:47.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.154+0000 7f46caf9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f46c40c7f90 0x7f46c40c8380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:47.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.154+0000 7f46caf9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46c4142c00 0x7f46c40c88c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:47.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.154+0000 7f46caf9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46c40b0a30 con 0x7f46c4142c00 2023-10-06T02:28:47.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.154+0000 7f46caf9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46c414c4c0 con 0x7f46c406bd20 2023-10-06T02:28:47.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.154+0000 7f46caf9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46c416ae50 con 0x7f46c40c7f90 2023-10-06T02:28:47.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.155+0000 7f46ca79c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46c4142c00 0x7f46c40c88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:47.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.155+0000 7f46ca79c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46c4142c00 0x7f46c40c88c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36938/0 (socket says 172.21.15.31:36938) 2023-10-06T02:28:47.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.155+0000 7f46ca79c700 1 -- 172.21.15.31:0/768430603 learned_addr learned my addr 172.21.15.31:0/768430603 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:47.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.155+0000 7f46c9f9b700 1 --2- 172.21.15.31:0/768430603 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46c406bd20 0x7f46c40b29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:47.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.155+0000 7f46c979a700 1 --2- 172.21.15.31:0/768430603 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f46c40c7f90 0x7f46c40c8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:47.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.155+0000 7f46ca79c700 1 -- 172.21.15.31:0/768430603 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f46c40c7f90 msgr2=0x7f46c40c8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:47.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.155+0000 7f46ca79c700 1 --2- 172.21.15.31:0/768430603 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f46c40c7f90 0x7f46c40c8380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:47.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.155+0000 7f46ca79c700 1 -- 172.21.15.31:0/768430603 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46c406bd20 msgr2=0x7f46c40b29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:47.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.155+0000 7f46ca79c700 1 --2- 172.21.15.31:0/768430603 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46c406bd20 0x7f46c40b29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:47.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.155+0000 7f46ca79c700 1 -- 172.21.15.31:0/768430603 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46c00094f0 con 0x7f46c4142c00 2023-10-06T02:28:47.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.155+0000 7f46ca79c700 1 --2- 172.21.15.31:0/768430603 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46c4142c00 0x7f46c40c88c0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f46bc009ef0 tx=0x7f46bc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:47.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.156+0000 7f46b9ffb700 1 -- 172.21.15.31:0/768430603 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46bc00cb70 con 0x7f46c4142c00 2023-10-06T02:28:47.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.156+0000 7f46b9ffb700 1 -- 172.21.15.31:0/768430603 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46bc00f040 con 0x7f46c4142c00 2023-10-06T02:28:47.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.156+0000 7f46b9ffb700 1 -- 172.21.15.31:0/768430603 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46bc014480 con 0x7f46c4142c00 2023-10-06T02:28:47.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.156+0000 7f46caf9d700 1 -- 172.21.15.31:0/768430603 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46c40b2ee0 con 0x7f46c4142c00 2023-10-06T02:28:47.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.156+0000 7f46caf9d700 1 -- 172.21.15.31:0/768430603 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46c40b0470 con 0x7f46c4142c00 2023-10-06T02:28:47.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.157+0000 7f46caf9d700 1 -- 172.21.15.31:0/768430603 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46c404c3c0 con 0x7f46c4142c00 2023-10-06T02:28:47.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.158+0000 7f46b9ffb700 1 -- 172.21.15.31:0/768430603 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f46bc0147f0 con 0x7f46c4142c00 2023-10-06T02:28:47.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.159+0000 7f46b9ffb700 1 --2- 172.21.15.31:0/768430603 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f46a8067050 0x7f46a8069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:47.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.159+0000 7f46b9ffb700 1 -- 172.21.15.31:0/768430603 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f46bc0855f0 con 0x7f46c4142c00 2023-10-06T02:28:47.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.159+0000 7f46c9f9b700 1 --2- 172.21.15.31:0/768430603 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f46a8067050 0x7f46a8069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:47.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.160+0000 7f46c9f9b700 1 --2- 172.21.15.31:0/768430603 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f46a8067050 0x7f46a8069500 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f46c00099b0 tx=0x7f46c0017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:47.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.163+0000 7f46b9ffb700 1 -- 172.21.15.31:0/768430603 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f46bc052440 con 0x7f46c4142c00 2023-10-06T02:28:47.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.362+0000 7f46caf9d700 1 -- 172.21.15.31:0/768430603 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f46c40c93c0 con 0x7f46c4142c00 2023-10-06T02:28:47.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.363+0000 7f46b9ffb700 1 -- 172.21.15.31:0/768430603 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) v1 ==== 122+0+0 (secure 0 0 0) 0x7f46bc055a60 con 0x7f46c4142c00 2023-10-06T02:28:47.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.365+0000 7f46caf9d700 1 -- 172.21.15.31:0/768430603 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f46a8067050 msgr2=0x7f46a8069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:47.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.365+0000 7f46caf9d700 1 --2- 172.21.15.31:0/768430603 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f46a8067050 0x7f46a8069500 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f46c00099b0 tx=0x7f46c0017000 comp rx=0 tx=0).stop 2023-10-06T02:28:47.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.365+0000 7f46caf9d700 1 -- 172.21.15.31:0/768430603 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46c4142c00 msgr2=0x7f46c40c88c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:47.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.365+0000 7f46caf9d700 1 --2- 172.21.15.31:0/768430603 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46c4142c00 0x7f46c40c88c0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f46bc009ef0 tx=0x7f46bc00b040 comp rx=0 tx=0).stop 2023-10-06T02:28:47.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.365+0000 7f46caf9d700 1 -- 172.21.15.31:0/768430603 shutdown_connections 2023-10-06T02:28:47.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.365+0000 7f46caf9d700 1 --2- 172.21.15.31:0/768430603 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46c4142c00 0x7f46c40c88c0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:47.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.365+0000 7f46caf9d700 1 --2- 172.21.15.31:0/768430603 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46c406bd20 0x7f46c40b29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:47.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.365+0000 7f46caf9d700 1 --2- 172.21.15.31:0/768430603 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f46a8067050 0x7f46a8069500 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:47.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.365+0000 7f46caf9d700 1 --2- 172.21.15.31:0/768430603 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f46c40c7f90 0x7f46c40c8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:47.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.365+0000 7f46caf9d700 1 -- 172.21.15.31:0/768430603 >> 172.21.15.31:0/768430603 conn(0x7f46c40675a0 msgr2=0x7f46c40c21a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:47.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.365+0000 7f46caf9d700 1 -- 172.21.15.31:0/768430603 shutdown_connections 2023-10-06T02:28:47.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.365+0000 7f46caf9d700 1 -- 172.21.15.31:0/768430603 wait complete. 2023-10-06T02:28:47.383 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:28:47.383 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvolume_0000000000011953/5172ced3-24b3-4d86-a22c-dcfea079edc7) 2023-10-06T02:28:47.458 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:28:47.458 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvolume_0000000000011953/5172ced3-24b3-4d86-a22c-dcfea079edc7/.snap/fake) 2023-10-06T02:28:47.530 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:28:47.531 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2023-10-06T02:28:47.568 INFO:teuthology.orchestra.run.smithi031.stdout:cephfs_data 2023-10-06T02:28:47.569 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:28:47.569 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/_nogroup/subvolume_0000000000011953/5172ced3-24b3-4d86-a22c-dcfea079edc7) 2023-10-06T02:28:47.675 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T02:28:47.675 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/_nogroup/subvolume_0000000000011953/.meta 2023-10-06T02:28:47.741 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000011953 2023-10-06T02:28:47.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.971+0000 7f41adb9d700 1 Processor -- start 2023-10-06T02:28:47.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.972+0000 7f41adb9d700 1 -- start start 2023-10-06T02:28:47.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.973+0000 7f41adb9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f41a80c7be0 0x7f41a806abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:47.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.973+0000 7f41adb9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f41a80c7fb0 0x7f41a815a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:47.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.973+0000 7f41adb9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41a815aed0 con 0x7f41a813f780 2023-10-06T02:28:47.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.973+0000 7f41adb9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41a806bf10 con 0x7f41a80c7be0 2023-10-06T02:28:47.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.973+0000 7f41adb9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41a806b290 con 0x7f41a80c7fb0 2023-10-06T02:28:47.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.973+0000 7f41acb9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f41a80c7be0 0x7f41a806abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:47.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.973+0000 7f41ad39c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f41a813f780 0x7f41a813fb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:39336/0 (socket says 172.21.15.31:39336) 2023-10-06T02:28:47.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.973+0000 7f41ad39c700 1 -- 172.21.15.31:0/3330589841 learned_addr learned my addr 172.21.15.31:0/3330589841 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:28:47.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.973+0000 7f41acb9b700 1 -- 172.21.15.31:0/3330589841 >> v2:172.21.15.62:3300/0 conn(0x7f41a80c7fb0 msgr2=0x7f41a815a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:47.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.974+0000 7f41acb9b700 1 --2- 172.21.15.31:0/3330589841 >> v2:172.21.15.62:3300/0 conn(0x7f41a80c7fb0 0x7f41a815a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:47.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.974+0000 7f41acb9b700 1 -- 172.21.15.31:0/3330589841 >> v1:172.21.15.31:6789/0 conn(0x7f41a813f780 legacy=0x7f41a813fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:47.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.974+0000 7f41acb9b700 1 -- 172.21.15.31:0/3330589841 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41a806b570 con 0x7f41a80c7be0 2023-10-06T02:28:47.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.974+0000 7f41acb9b700 1 --2- 172.21.15.31:0/3330589841 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41a80c7be0 0x7f41a806abd0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f41a80aec50 tx=0x7f419c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e71591f4d639e9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:47.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.974+0000 7f41a77fe700 1 -- 172.21.15.31:0/3330589841 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f419c00b6b0 con 0x7f41a80c7be0 2023-10-06T02:28:47.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.974+0000 7f41a77fe700 1 -- 172.21.15.31:0/3330589841 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f419c00b810 con 0x7f41a80c7be0 2023-10-06T02:28:47.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.974+0000 7f41a77fe700 1 -- 172.21.15.31:0/3330589841 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f419c00bbb0 con 0x7f41a80c7be0 2023-10-06T02:28:47.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.975+0000 7f41adb9d700 1 -- 172.21.15.31:0/3330589841 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41a80c7be0 msgr2=0x7f41a806abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:47.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.975+0000 7f41adb9d700 1 --2- 172.21.15.31:0/3330589841 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41a80c7be0 0x7f41a806abd0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f41a80aec50 tx=0x7f419c0091a0 comp rx=0 tx=0).stop 2023-10-06T02:28:47.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.975+0000 7f41adb9d700 1 -- 172.21.15.31:0/3330589841 shutdown_connections 2023-10-06T02:28:47.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.975+0000 7f41adb9d700 1 --2- 172.21.15.31:0/3330589841 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41a80c7be0 0x7f41a806abd0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:47.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.975+0000 7f41adb9d700 1 --2- 172.21.15.31:0/3330589841 >> v2:172.21.15.62:3300/0 conn(0x7f41a80c7fb0 0x7f41a815a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:47.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.975+0000 7f41adb9d700 1 -- 172.21.15.31:0/3330589841 >> 172.21.15.31:0/3330589841 conn(0x7f41a806d110 msgr2=0x7f41a80cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:47.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.975+0000 7f41adb9d700 1 -- 172.21.15.31:0/3330589841 shutdown_connections 2023-10-06T02:28:47.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.975+0000 7f41adb9d700 1 -- 172.21.15.31:0/3330589841 wait complete. 2023-10-06T02:28:47.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.976+0000 7f41adb9d700 1 Processor -- start 2023-10-06T02:28:47.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.976+0000 7f41adb9d700 1 -- start start 2023-10-06T02:28:47.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.977+0000 7f41adb9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41a80c7be0 0x7f41a8159410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:47.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.977+0000 7f41adb9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f41a80c7fb0 0x7f41a80c5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:47.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.977+0000 7f41adb9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41a813f780 0x7f41a80c5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:47.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.977+0000 7f41adb9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41a80c1220 con 0x7f41a813f780 2023-10-06T02:28:47.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.977+0000 7f41adb9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41a80c64d0 con 0x7f41a80c7be0 2023-10-06T02:28:47.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.977+0000 7f41adb9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41a80c5c60 con 0x7f41a80c7fb0 2023-10-06T02:28:47.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.978+0000 7f41ad39c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41a813f780 0x7f41a80c5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:47.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.978+0000 7f41ad39c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41a813f780 0x7f41a80c5590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48808/0 (socket says 172.21.15.31:48808) 2023-10-06T02:28:47.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.978+0000 7f41ad39c700 1 -- 172.21.15.31:0/1806131719 learned_addr learned my addr 172.21.15.31:0/1806131719 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:47.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.978+0000 7f41acb9b700 1 --2- 172.21.15.31:0/1806131719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41a80c7be0 0x7f41a8159410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:47.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.978+0000 7f41a7fff700 1 --2- 172.21.15.31:0/1806131719 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f41a80c7fb0 0x7f41a80c5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:47.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.978+0000 7f41ad39c700 1 -- 172.21.15.31:0/1806131719 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f41a80c7fb0 msgr2=0x7f41a80c5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:47.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.978+0000 7f41ad39c700 1 --2- 172.21.15.31:0/1806131719 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f41a80c7fb0 0x7f41a80c5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:47.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.978+0000 7f41ad39c700 1 -- 172.21.15.31:0/1806131719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41a80c7be0 msgr2=0x7f41a8159410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:47.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.978+0000 7f41ad39c700 1 --2- 172.21.15.31:0/1806131719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41a80c7be0 0x7f41a8159410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:47.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.978+0000 7f41ad39c700 1 -- 172.21.15.31:0/1806131719 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f419c009420 con 0x7f41a813f780 2023-10-06T02:28:47.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.978+0000 7f41ad39c700 1 --2- 172.21.15.31:0/1806131719 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41a813f780 0x7f41a80c5590 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f41a000cbc0 tx=0x7f41a0001990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:47.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.979+0000 7f41a4ff9700 1 -- 172.21.15.31:0/1806131719 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a000a7a0 con 0x7f41a813f780 2023-10-06T02:28:47.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.979+0000 7f41a4ff9700 1 -- 172.21.15.31:0/1806131719 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41a000a900 con 0x7f41a813f780 2023-10-06T02:28:47.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.979+0000 7f41a4ff9700 1 -- 172.21.15.31:0/1806131719 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a000ac80 con 0x7f41a813f780 2023-10-06T02:28:47.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.979+0000 7f41adb9d700 1 -- 172.21.15.31:0/1806131719 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41a80c61f0 con 0x7f41a813f780 2023-10-06T02:28:47.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.979+0000 7f41adb9d700 1 -- 172.21.15.31:0/1806131719 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41a8159950 con 0x7f41a813f780 2023-10-06T02:28:47.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.980+0000 7f41adb9d700 1 -- 172.21.15.31:0/1806131719 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41a804c3c0 con 0x7f41a813f780 2023-10-06T02:28:47.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.981+0000 7f41a4ff9700 1 -- 172.21.15.31:0/1806131719 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f41a0007300 con 0x7f41a813f780 2023-10-06T02:28:47.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.981+0000 7f41a4ff9700 1 --2- 172.21.15.31:0/1806131719 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f418c067050 0x7f418c069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:47.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.982+0000 7f41a4ff9700 1 -- 172.21.15.31:0/1806131719 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f41a0084ac0 con 0x7f41a813f780 2023-10-06T02:28:47.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.982+0000 7f41acb9b700 1 --2- 172.21.15.31:0/1806131719 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f418c067050 0x7f418c069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:47.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.983+0000 7f41acb9b700 1 --2- 172.21.15.31:0/1806131719 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f418c067050 0x7f418c069500 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f419c0093f0 tx=0x7f419c017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:47.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:47.986+0000 7f41a4ff9700 1 -- 172.21.15.31:0/1806131719 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f41a0051aa0 con 0x7f41a813f780 2023-10-06T02:28:48.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.221+0000 7f41adb9d700 1 -- 172.21.15.31:0/1806131719 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000011953", "target": ["mon-mgr", ""]}) v1 -- 0x7f41a806b850 con 0x7f418c067050 2023-10-06T02:28:48.228 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:28:48.228+0000 7f656537d700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:28:48.228 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:28:48.228+0000 7f656537d700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:28:48.228 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:28:48.228+0000 7f656537d700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:28:48.228 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:28:48.228+0000 7f656537d700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:28:48.228 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:28:48.228+0000 7f656537d700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:28:48.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.273+0000 7f41a4ff9700 1 -- 172.21.15.31:0/1806131719 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+82 (secure 0 0 0) 0x7f41a806b850 con 0x7f418c067050 2023-10-06T02:28:48.274 INFO:teuthology.orchestra.run.smithi031.stdout:/volumes/_nogroup/subvolume_0000000000011953/5172ced3-24b3-4d86-a22c-dcfea079edc7 2023-10-06T02:28:48.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.276+0000 7f41adb9d700 1 -- 172.21.15.31:0/1806131719 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f418c067050 msgr2=0x7f418c069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:48.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.276+0000 7f41adb9d700 1 --2- 172.21.15.31:0/1806131719 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f418c067050 0x7f418c069500 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f419c0093f0 tx=0x7f419c017000 comp rx=0 tx=0).stop 2023-10-06T02:28:48.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.276+0000 7f41adb9d700 1 -- 172.21.15.31:0/1806131719 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41a813f780 msgr2=0x7f41a80c5590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:48.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.276+0000 7f41adb9d700 1 --2- 172.21.15.31:0/1806131719 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41a813f780 0x7f41a80c5590 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f41a000cbc0 tx=0x7f41a0001990 comp rx=0 tx=0).stop 2023-10-06T02:28:48.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.276+0000 7f41adb9d700 1 -- 172.21.15.31:0/1806131719 shutdown_connections 2023-10-06T02:28:48.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.276+0000 7f41adb9d700 1 --2- 172.21.15.31:0/1806131719 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41a813f780 0x7f41a80c5590 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:48.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.276+0000 7f41adb9d700 1 --2- 172.21.15.31:0/1806131719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41a80c7be0 0x7f41a8159410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:48.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.276+0000 7f41adb9d700 1 --2- 172.21.15.31:0/1806131719 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f418c067050 0x7f418c069500 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:48.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.276+0000 7f41adb9d700 1 --2- 172.21.15.31:0/1806131719 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f41a80c7fb0 0x7f41a80c5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:48.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.276+0000 7f41adb9d700 1 -- 172.21.15.31:0/1806131719 >> 172.21.15.31:0/1806131719 conn(0x7f41a806d110 msgr2=0x7f41a813dcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:48.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.276+0000 7f41adb9d700 1 -- 172.21.15.31:0/1806131719 shutdown_connections 2023-10-06T02:28:48.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.276+0000 7f41adb9d700 1 -- 172.21.15.31:0/1806131719 wait complete. 2023-10-06T02:28:48.294 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs 2023-10-06T02:28:48.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.483+0000 7eff6ffff700 1 Processor -- start 2023-10-06T02:28:48.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.483+0000 7eff6ffff700 1 -- start start 2023-10-06T02:28:48.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.484+0000 7eff6ffff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7eff70153a00 0x7eff70153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:48.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.485+0000 7eff6ffff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7eff700cea00 0x7eff700cee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:48.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.485+0000 7eff6ffff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff7015ac50 con 0x7eff7015de50 2023-10-06T02:28:48.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.485+0000 7eff6ffff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff7015d010 con 0x7eff700cea00 2023-10-06T02:28:48.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.485+0000 7eff6ffff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff700cf3c0 con 0x7eff70153a00 2023-10-06T02:28:48.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.485+0000 7eff6f7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7eff700cea00 0x7eff700cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:48.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.485+0000 7eff6e7fc700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7eff7015de50 0x7eff7015a540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:39340/0 (socket says 172.21.15.31:39340) 2023-10-06T02:28:48.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.485+0000 7eff6e7fc700 1 -- 172.21.15.31:0/3128455907 learned_addr learned my addr 172.21.15.31:0/3128455907 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:28:48.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.485+0000 7eff6effd700 1 --2- 172.21.15.31:0/3128455907 >> v2:172.21.15.62:3300/0 conn(0x7eff70153a00 0x7eff70153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:48.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.485+0000 7eff6f7fe700 1 -- 172.21.15.31:0/3128455907 >> v2:172.21.15.62:3300/0 conn(0x7eff70153a00 msgr2=0x7eff70153dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:48.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.485+0000 7eff6f7fe700 1 --2- 172.21.15.31:0/3128455907 >> v2:172.21.15.62:3300/0 conn(0x7eff70153a00 0x7eff70153dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:48.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.485+0000 7eff6f7fe700 1 -- 172.21.15.31:0/3128455907 >> v1:172.21.15.31:6789/0 conn(0x7eff7015de50 legacy=0x7eff7015a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:48.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.485+0000 7eff6f7fe700 1 -- 172.21.15.31:0/3128455907 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff7015af60 con 0x7eff700cea00 2023-10-06T02:28:48.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.485+0000 7eff6effd700 1 --2- 172.21.15.31:0/3128455907 >> v2:172.21.15.62:3300/0 conn(0x7eff70153a00 0x7eff70153dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:28:48.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.486+0000 7eff6f7fe700 1 --2- 172.21.15.31:0/3128455907 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff700cea00 0x7eff700cee80 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7eff700cac20 tx=0x7eff68009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=900a1b1e4eced19c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:48.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.487+0000 7eff6dffb700 1 -- 172.21.15.31:0/3128455907 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff6800e070 con 0x7eff700cea00 2023-10-06T02:28:48.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.487+0000 7eff6dffb700 1 -- 172.21.15.31:0/3128455907 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff6800bc30 con 0x7eff700cea00 2023-10-06T02:28:48.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.487+0000 7eff6dffb700 1 -- 172.21.15.31:0/3128455907 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff68012430 con 0x7eff700cea00 2023-10-06T02:28:48.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.487+0000 7eff6ffff700 1 -- 172.21.15.31:0/3128455907 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff700cea00 msgr2=0x7eff700cee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:48.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.487+0000 7eff6ffff700 1 --2- 172.21.15.31:0/3128455907 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff700cea00 0x7eff700cee80 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7eff700cac20 tx=0x7eff68009780 comp rx=0 tx=0).stop 2023-10-06T02:28:48.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.488+0000 7eff6ffff700 1 -- 172.21.15.31:0/3128455907 shutdown_connections 2023-10-06T02:28:48.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.488+0000 7eff6ffff700 1 --2- 172.21.15.31:0/3128455907 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff700cea00 0x7eff700cee80 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:48.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.488+0000 7eff6ffff700 1 --2- 172.21.15.31:0/3128455907 >> v2:172.21.15.62:3300/0 conn(0x7eff70153a00 0x7eff70153dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:48.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.488+0000 7eff6ffff700 1 -- 172.21.15.31:0/3128455907 >> 172.21.15.31:0/3128455907 conn(0x7eff700703a0 msgr2=0x7eff700707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:48.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.488+0000 7eff6ffff700 1 -- 172.21.15.31:0/3128455907 shutdown_connections 2023-10-06T02:28:48.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.488+0000 7eff6ffff700 1 -- 172.21.15.31:0/3128455907 wait complete. 2023-10-06T02:28:48.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.489+0000 7eff6ffff700 1 Processor -- start 2023-10-06T02:28:48.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.489+0000 7eff6ffff700 1 -- start start 2023-10-06T02:28:48.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.490+0000 7eff6ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff700cea00 0x7eff70145fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:48.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.490+0000 7eff6ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff70153a00 0x7eff70158b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:48.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.490+0000 7eff6ffff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7eff7015de50 0x7eff70159090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:48.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.490+0000 7eff6ffff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff70159960 con 0x7eff700cea00 2023-10-06T02:28:48.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.490+0000 7eff6ffff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff701464f0 con 0x7eff70153a00 2023-10-06T02:28:48.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.490+0000 7eff6ffff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff70159680 con 0x7eff7015de50 2023-10-06T02:28:48.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.490+0000 7eff6effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff700cea00 0x7eff70145fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:48.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.491+0000 7eff6effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff700cea00 0x7eff70145fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48824/0 (socket says 172.21.15.31:48824) 2023-10-06T02:28:48.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.491+0000 7eff6effd700 1 -- 172.21.15.31:0/3252205160 learned_addr learned my addr 172.21.15.31:0/3252205160 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:48.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.491+0000 7eff6e7fc700 1 --2- 172.21.15.31:0/3252205160 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff70153a00 0x7eff70158b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:48.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.491+0000 7eff6f7fe700 1 --2- 172.21.15.31:0/3252205160 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7eff7015de50 0x7eff70159090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:48.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.491+0000 7eff6effd700 1 -- 172.21.15.31:0/3252205160 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7eff7015de50 msgr2=0x7eff70159090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:48.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.491+0000 7eff6effd700 1 --2- 172.21.15.31:0/3252205160 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7eff7015de50 0x7eff70159090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:48.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.491+0000 7eff6effd700 1 -- 172.21.15.31:0/3252205160 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff70153a00 msgr2=0x7eff70158b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:48.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.491+0000 7eff6effd700 1 --2- 172.21.15.31:0/3252205160 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff70153a00 0x7eff70158b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:48.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.491+0000 7eff6effd700 1 -- 172.21.15.31:0/3252205160 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff68009a20 con 0x7eff700cea00 2023-10-06T02:28:48.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.491+0000 7eff6e7fc700 1 --2- 172.21.15.31:0/3252205160 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff70153a00 0x7eff70158b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:28:48.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.491+0000 7eff6effd700 1 --2- 172.21.15.31:0/3252205160 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff700cea00 0x7eff70145fb0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7eff6000e8d0 tx=0x7eff60008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:48.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.491+0000 7eff56ffd700 1 -- 172.21.15.31:0/3252205160 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff6000ee70 con 0x7eff700cea00 2023-10-06T02:28:48.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.491+0000 7eff56ffd700 1 -- 172.21.15.31:0/3252205160 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff6000c5e0 con 0x7eff700cea00 2023-10-06T02:28:48.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.491+0000 7eff56ffd700 1 -- 172.21.15.31:0/3252205160 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff6000c980 con 0x7eff700cea00 2023-10-06T02:28:48.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.492+0000 7eff6ffff700 1 -- 172.21.15.31:0/3252205160 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff7016d190 con 0x7eff700cea00 2023-10-06T02:28:48.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.492+0000 7eff6ffff700 1 -- 172.21.15.31:0/3252205160 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff7016d750 con 0x7eff700cea00 2023-10-06T02:28:48.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.493+0000 7eff6ffff700 1 -- 172.21.15.31:0/3252205160 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff7004c3c0 con 0x7eff700cea00 2023-10-06T02:28:48.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.494+0000 7eff56ffd700 1 -- 172.21.15.31:0/3252205160 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7eff60013410 con 0x7eff700cea00 2023-10-06T02:28:48.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.495+0000 7eff56ffd700 1 --2- 172.21.15.31:0/3252205160 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7eff5c067050 0x7eff5c069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:48.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.495+0000 7eff56ffd700 1 -- 172.21.15.31:0/3252205160 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6058+0+0 (secure 0 0 0) 0x7eff600847a0 con 0x7eff700cea00 2023-10-06T02:28:48.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.496+0000 7eff6e7fc700 1 --2- 172.21.15.31:0/3252205160 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7eff5c067050 0x7eff5c069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:48.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.496+0000 7eff6e7fc700 1 --2- 172.21.15.31:0/3252205160 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7eff5c067050 0x7eff5c069500 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7eff700d09c0 tx=0x7eff64009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:48.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.499+0000 7eff56ffd700 1 -- 172.21.15.31:0/3252205160 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7eff600515f0 con 0x7eff700cea00 2023-10-06T02:28:48.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.735+0000 7eff6ffff700 1 -- 172.21.15.31:0/3252205160 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""]}) v1 -- 0x7eff700caef0 con 0x7eff5c067050 2023-10-06T02:28:48.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.740+0000 7eff56ffd700 1 -- 172.21.15.31:0/3252205160 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+61 (secure 0 0 0) 0x7eff700caef0 con 0x7eff5c067050 2023-10-06T02:28:48.740 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2023-10-06T02:28:48.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:28:48.741 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "subvolume_0000000000011953" 2023-10-06T02:28:48.741 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:28:48.741 INFO:teuthology.orchestra.run.smithi031.stdout:] 2023-10-06T02:28:48.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.742+0000 7eff6ffff700 1 -- 172.21.15.31:0/3252205160 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7eff5c067050 msgr2=0x7eff5c069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:48.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.742+0000 7eff6ffff700 1 --2- 172.21.15.31:0/3252205160 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7eff5c067050 0x7eff5c069500 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7eff700d09c0 tx=0x7eff64009040 comp rx=0 tx=0).stop 2023-10-06T02:28:48.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.742+0000 7eff6ffff700 1 -- 172.21.15.31:0/3252205160 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff700cea00 msgr2=0x7eff70145fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:48.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.742+0000 7eff6ffff700 1 --2- 172.21.15.31:0/3252205160 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff700cea00 0x7eff70145fb0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7eff6000e8d0 tx=0x7eff60008040 comp rx=0 tx=0).stop 2023-10-06T02:28:48.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.742+0000 7eff6ffff700 1 -- 172.21.15.31:0/3252205160 shutdown_connections 2023-10-06T02:28:48.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.742+0000 7eff6ffff700 1 --2- 172.21.15.31:0/3252205160 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7eff5c067050 0x7eff5c069500 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:48.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.742+0000 7eff6ffff700 1 --2- 172.21.15.31:0/3252205160 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7eff7015de50 0x7eff70159090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:48.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.742+0000 7eff6ffff700 1 --2- 172.21.15.31:0/3252205160 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff700cea00 0x7eff70145fb0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:48.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.742+0000 7eff6ffff700 1 --2- 172.21.15.31:0/3252205160 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff70153a00 0x7eff70158b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:48.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.742+0000 7eff6ffff700 1 -- 172.21.15.31:0/3252205160 >> 172.21.15.31:0/3252205160 conn(0x7eff700703a0 msgr2=0x7eff7006d790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:48.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.742+0000 7eff6ffff700 1 -- 172.21.15.31:0/3252205160 shutdown_connections 2023-10-06T02:28:48.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.742+0000 7eff6ffff700 1 -- 172.21.15.31:0/3252205160 wait complete. 2023-10-06T02:28:48.760 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume info cephfs subvolume_0000000000011953 2023-10-06T02:28:48.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.951+0000 7fde1759e700 1 Processor -- start 2023-10-06T02:28:48.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.952+0000 7fde1759e700 1 -- start start 2023-10-06T02:28:48.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.953+0000 7fde1759e700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fde180c7be0 0x7fde1806abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:48.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.953+0000 7fde1759e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fde180c7fb0 0x7fde1815a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:48.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.953+0000 7fde1759e700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde1815aed0 con 0x7fde1813f780 2023-10-06T02:28:48.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.953+0000 7fde1759e700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde1806bf10 con 0x7fde180c7fb0 2023-10-06T02:28:48.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.953+0000 7fde1759e700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde1806b290 con 0x7fde180c7be0 2023-10-06T02:28:48.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.953+0000 7fde15d9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fde180c7fb0 0x7fde1815a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:48.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.953+0000 7fde16d9d700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fde1813f780 0x7fde1813fb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:39356/0 (socket says 172.21.15.31:39356) 2023-10-06T02:28:48.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.953+0000 7fde16d9d700 1 -- 172.21.15.31:0/2512727973 learned_addr learned my addr 172.21.15.31:0/2512727973 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:28:48.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.953+0000 7fde1659c700 1 --2- 172.21.15.31:0/2512727973 >> v2:172.21.15.62:3300/0 conn(0x7fde180c7be0 0x7fde1806abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:48.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.954+0000 7fde15d9b700 1 -- 172.21.15.31:0/2512727973 >> v2:172.21.15.62:3300/0 conn(0x7fde180c7be0 msgr2=0x7fde1806abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:48.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.953+0000 7fde1559a700 1 -- 172.21.15.31:0/2512727973 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1073913649 0 0) 0x7fde1815aed0 con 0x7fde1813f780 2023-10-06T02:28:48.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.954+0000 7fde15d9b700 1 --2- 172.21.15.31:0/2512727973 >> v2:172.21.15.62:3300/0 conn(0x7fde180c7be0 0x7fde1806abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:48.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.954+0000 7fde15d9b700 1 -- 172.21.15.31:0/2512727973 >> v1:172.21.15.31:6789/0 conn(0x7fde1813f780 legacy=0x7fde1813fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:48.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.954+0000 7fde15d9b700 1 -- 172.21.15.31:0/2512727973 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde1806b570 con 0x7fde180c7fb0 2023-10-06T02:28:48.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.954+0000 7fde15d9b700 1 --2- 172.21.15.31:0/2512727973 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde180c7fb0 0x7fde1815a990 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fde180c8b00 tx=0x7fde0c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d52f2b52c2d16a28 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:48.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.954+0000 7fde1559a700 1 -- 172.21.15.31:0/2512727973 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde0c011630 con 0x7fde180c7fb0 2023-10-06T02:28:48.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.954+0000 7fde1559a700 1 -- 172.21.15.31:0/2512727973 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde0c011790 con 0x7fde180c7fb0 2023-10-06T02:28:48.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.954+0000 7fde1559a700 1 -- 172.21.15.31:0/2512727973 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde0c011b30 con 0x7fde180c7fb0 2023-10-06T02:28:48.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.955+0000 7fde1759e700 1 -- 172.21.15.31:0/2512727973 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde180c7fb0 msgr2=0x7fde1815a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:48.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.955+0000 7fde1759e700 1 --2- 172.21.15.31:0/2512727973 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde180c7fb0 0x7fde1815a990 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fde180c8b00 tx=0x7fde0c0091a0 comp rx=0 tx=0).stop 2023-10-06T02:28:48.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.955+0000 7fde1759e700 1 -- 172.21.15.31:0/2512727973 shutdown_connections 2023-10-06T02:28:48.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.955+0000 7fde1759e700 1 --2- 172.21.15.31:0/2512727973 >> v2:172.21.15.62:3300/0 conn(0x7fde180c7be0 0x7fde1806abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:48.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.955+0000 7fde1759e700 1 --2- 172.21.15.31:0/2512727973 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde180c7fb0 0x7fde1815a990 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:48.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.955+0000 7fde1759e700 1 -- 172.21.15.31:0/2512727973 >> 172.21.15.31:0/2512727973 conn(0x7fde1806d110 msgr2=0x7fde180cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:48.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.955+0000 7fde1759e700 1 -- 172.21.15.31:0/2512727973 shutdown_connections 2023-10-06T02:28:48.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.955+0000 7fde1759e700 1 -- 172.21.15.31:0/2512727973 wait complete. 2023-10-06T02:28:48.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.956+0000 7fde1759e700 1 Processor -- start 2023-10-06T02:28:48.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.956+0000 7fde1759e700 1 -- start start 2023-10-06T02:28:48.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.957+0000 7fde1759e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde180c7be0 0x7fde1816a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:48.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.957+0000 7fde1759e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fde180c7fb0 0x7fde18155640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:48.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.957+0000 7fde1759e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde1813f780 0x7fde1813d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:48.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.957+0000 7fde1759e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde1816abd0 con 0x7fde1813f780 2023-10-06T02:28:48.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.957+0000 7fde1759e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde1816a200 con 0x7fde180c7be0 2023-10-06T02:28:48.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.957+0000 7fde1759e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde18169920 con 0x7fde180c7fb0 2023-10-06T02:28:48.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.958+0000 7fde16d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde1813f780 0x7fde1813d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:48.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.958+0000 7fde16d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde1813f780 0x7fde1813d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48832/0 (socket says 172.21.15.31:48832) 2023-10-06T02:28:48.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.958+0000 7fde16d9d700 1 -- 172.21.15.31:0/1202320023 learned_addr learned my addr 172.21.15.31:0/1202320023 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:48.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.958+0000 7fde1659c700 1 --2- 172.21.15.31:0/1202320023 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde180c7be0 0x7fde1816a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:48.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.958+0000 7fde15d9b700 1 --2- 172.21.15.31:0/1202320023 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fde180c7fb0 0x7fde18155640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:48.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.958+0000 7fde16d9d700 1 -- 172.21.15.31:0/1202320023 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fde180c7fb0 msgr2=0x7fde18155640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:48.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.958+0000 7fde16d9d700 1 --2- 172.21.15.31:0/1202320023 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fde180c7fb0 0x7fde18155640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:48.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.958+0000 7fde16d9d700 1 -- 172.21.15.31:0/1202320023 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde180c7be0 msgr2=0x7fde1816a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:48.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.958+0000 7fde16d9d700 1 --2- 172.21.15.31:0/1202320023 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde180c7be0 0x7fde1816a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:48.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.958+0000 7fde16d9d700 1 -- 172.21.15.31:0/1202320023 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde0c009420 con 0x7fde1813f780 2023-10-06T02:28:48.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.958+0000 7fde16d9d700 1 --2- 172.21.15.31:0/1202320023 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde1813f780 0x7fde1813d2d0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fde1000cbc0 tx=0x7fde10001980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:48.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.959+0000 7fde067fc700 1 -- 172.21.15.31:0/1202320023 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde1000a7e0 con 0x7fde1813f780 2023-10-06T02:28:48.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.959+0000 7fde067fc700 1 -- 172.21.15.31:0/1202320023 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde1000a940 con 0x7fde1813f780 2023-10-06T02:28:48.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.959+0000 7fde1759e700 1 -- 172.21.15.31:0/1202320023 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde1816dbe0 con 0x7fde1813f780 2023-10-06T02:28:48.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.959+0000 7fde067fc700 1 -- 172.21.15.31:0/1202320023 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde1000ad00 con 0x7fde1813f780 2023-10-06T02:28:48.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.959+0000 7fde1759e700 1 -- 172.21.15.31:0/1202320023 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde18169f10 con 0x7fde1813f780 2023-10-06T02:28:48.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.960+0000 7fde1759e700 1 -- 172.21.15.31:0/1202320023 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde1804c3c0 con 0x7fde1813f780 2023-10-06T02:28:48.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.961+0000 7fde067fc700 1 -- 172.21.15.31:0/1202320023 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7fde10007300 con 0x7fde1813f780 2023-10-06T02:28:48.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.962+0000 7fde067fc700 1 --2- 172.21.15.31:0/1202320023 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fddf8067050 0x7fddf8069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:48.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.962+0000 7fde067fc700 1 -- 172.21.15.31:0/1202320023 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fde10084ac0 con 0x7fde1813f780 2023-10-06T02:28:48.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.962+0000 7fde1659c700 1 --2- 172.21.15.31:0/1202320023 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fddf8067050 0x7fddf8069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:48.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.963+0000 7fde1659c700 1 --2- 172.21.15.31:0/1202320023 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fddf8067050 0x7fddf8069500 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fde08009db0 tx=0x7fde080091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:48.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:48.965+0000 7fde067fc700 1 -- 172.21.15.31:0/1202320023 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fde10051aa0 con 0x7fde1813f780 2023-10-06T02:28:49.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:49.200+0000 7fde1759e700 1 -- 172.21.15.31:0/1202320023 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "fs subvolume info", "vol_name": "cephfs", "sub_name": "subvolume_0000000000011953", "target": ["mon-mgr", ""]}) v1 -- 0x7fde1806b850 con 0x7fddf8067050 2023-10-06T02:28:49.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:49.236+0000 7fde067fc700 1 -- 172.21.15.31:0/1202320023 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+682 (secure 0 0 0) 0x7fde1806b850 con 0x7fddf8067050 2023-10-06T02:28:50.070 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T02:28:50.070 INFO:teuthology.orchestra.run.smithi031.stdout: "atime": "2023-10-06 02:28:47", 2023-10-06T02:28:50.070 INFO:teuthology.orchestra.run.smithi031.stdout: "bytes_pcent": "undefined", 2023-10-06T02:28:50.071 INFO:teuthology.orchestra.run.smithi031.stdout: "bytes_quota": "infinite", 2023-10-06T02:28:50.071 INFO:teuthology.orchestra.run.smithi031.stdout: "bytes_used": 0, 2023-10-06T02:28:50.071 INFO:teuthology.orchestra.run.smithi031.stdout: "created_at": "2023-10-06 02:28:47", 2023-10-06T02:28:50.071 INFO:teuthology.orchestra.run.smithi031.stdout: "ctime": "2023-10-06 02:28:48", 2023-10-06T02:28:50.071 INFO:teuthology.orchestra.run.smithi031.stdout: "data_pool": "cephfs_data", 2023-10-06T02:28:50.071 INFO:teuthology.orchestra.run.smithi031.stdout: "features": [ 2023-10-06T02:28:50.072 INFO:teuthology.orchestra.run.smithi031.stdout: "snapshot-clone", 2023-10-06T02:28:50.072 INFO:teuthology.orchestra.run.smithi031.stdout: "snapshot-autoprotect" 2023-10-06T02:28:50.072 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:28:50.072 INFO:teuthology.orchestra.run.smithi031.stdout: "gid": 0, 2023-10-06T02:28:50.072 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": 16877, 2023-10-06T02:28:50.073 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_addrs": [ 2023-10-06T02:28:50.073 INFO:teuthology.orchestra.run.smithi031.stdout: "172.21.15.31:6789", 2023-10-06T02:28:50.073 INFO:teuthology.orchestra.run.smithi031.stdout: "172.21.15.62:6789", 2023-10-06T02:28:50.073 INFO:teuthology.orchestra.run.smithi031.stdout: "172.21.15.62:6790" 2023-10-06T02:28:50.073 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:28:50.073 INFO:teuthology.orchestra.run.smithi031.stdout: "mtime": "2023-10-06 02:28:47", 2023-10-06T02:28:50.074 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/volumes/_nogroup/subvolume_0000000000011953/5172ced3-24b3-4d86-a22c-dcfea079edc7", 2023-10-06T02:28:50.074 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_namespace": "", 2023-10-06T02:28:50.074 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "complete", 2023-10-06T02:28:50.074 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "subvolume", 2023-10-06T02:28:50.074 INFO:teuthology.orchestra.run.smithi031.stdout: "uid": 0 2023-10-06T02:28:50.074 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T02:28:50.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:49.238+0000 7fde1759e700 1 -- 172.21.15.31:0/1202320023 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fddf8067050 msgr2=0x7fddf8069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:50.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:49.238+0000 7fde1759e700 1 --2- 172.21.15.31:0/1202320023 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fddf8067050 0x7fddf8069500 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fde08009db0 tx=0x7fde080091a0 comp rx=0 tx=0).stop 2023-10-06T02:28:50.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:49.238+0000 7fde1759e700 1 -- 172.21.15.31:0/1202320023 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde1813f780 msgr2=0x7fde1813d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:50.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:49.238+0000 7fde1759e700 1 --2- 172.21.15.31:0/1202320023 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde1813f780 0x7fde1813d2d0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fde1000cbc0 tx=0x7fde10001980 comp rx=0 tx=0).stop 2023-10-06T02:28:50.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:49.238+0000 7fde1759e700 1 -- 172.21.15.31:0/1202320023 shutdown_connections 2023-10-06T02:28:50.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:49.238+0000 7fde1759e700 1 --2- 172.21.15.31:0/1202320023 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde1813f780 0x7fde1813d2d0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:50.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:49.238+0000 7fde1759e700 1 --2- 172.21.15.31:0/1202320023 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde180c7be0 0x7fde1816a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:50.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:49.238+0000 7fde1759e700 1 --2- 172.21.15.31:0/1202320023 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fddf8067050 0x7fddf8069500 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:50.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:49.238+0000 7fde1759e700 1 --2- 172.21.15.31:0/1202320023 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fde180c7fb0 0x7fde18155640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:50.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:49.238+0000 7fde1759e700 1 -- 172.21.15.31:0/1202320023 >> 172.21.15.31:0/1202320023 conn(0x7fde1806d110 msgr2=0x7fde180c13d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:50.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:49.238+0000 7fde1759e700 1 -- 172.21.15.31:0/1202320023 shutdown_connections 2023-10-06T02:28:50.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:49.238+0000 7fde1759e700 1 -- 172.21.15.31:0/1202320023 wait complete. 2023-10-06T02:28:50.077 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume resize cephfs subvolume_0000000000011953 10485760 2023-10-06T02:28:50.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.279+0000 7f138db9d700 1 Processor -- start 2023-10-06T02:28:50.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.279+0000 7f138db9d700 1 -- start start 2023-10-06T02:28:50.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.280+0000 7f138db9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f13880c2a90 0x7f1388156aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:50.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.280+0000 7f138db9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f13880c85d0 0x7f13880cc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:50.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.280+0000 7f138db9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13881579b0 con 0x7f13880c8200 2023-10-06T02:28:50.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.280+0000 7f138db9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13880c1390 con 0x7f13880c2a90 2023-10-06T02:28:50.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.280+0000 7f138db9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13881573f0 con 0x7f13880c85d0 2023-10-06T02:28:50.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.281+0000 7f138cb9b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f13880c8200 0x7f13880c1820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:39372/0 (socket says 172.21.15.31:39372) 2023-10-06T02:28:50.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.281+0000 7f138cb9b700 1 -- 172.21.15.31:0/70902166 learned_addr learned my addr 172.21.15.31:0/70902166 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:28:50.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.281+0000 7f1387fff700 1 --2- 172.21.15.31:0/70902166 >> v2:172.21.15.62:3300/0 conn(0x7f13880c2a90 0x7f1388156aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:50.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.281+0000 7f138d39c700 1 --2- 172.21.15.31:0/70902166 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f13880c85d0 0x7f13880cc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:50.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.281+0000 7f13877fe700 1 -- 172.21.15.31:0/70902166 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3686190604 0 0) 0x7f13881579b0 con 0x7f13880c8200 2023-10-06T02:28:50.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.281+0000 7f13877fe700 1 -- 172.21.15.31:0/70902166 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1378003570 con 0x7f13880c8200 2023-10-06T02:28:50.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.281+0000 7f13877fe700 1 -- 172.21.15.31:0/70902166 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3320252970 0 0) 0x7f1378003570 con 0x7f13880c8200 2023-10-06T02:28:50.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.281+0000 7f13877fe700 1 -- 172.21.15.31:0/70902166 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f13880c85d0 msgr2=0x7f13880cc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:50.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.281+0000 7f13877fe700 1 --2- 172.21.15.31:0/70902166 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f13880c85d0 0x7f13880cc2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:50.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.281+0000 7f13877fe700 1 -- 172.21.15.31:0/70902166 >> v2:172.21.15.62:3300/0 conn(0x7f13880c2a90 msgr2=0x7f1388156aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:50.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.281+0000 7f13877fe700 1 --2- 172.21.15.31:0/70902166 >> v2:172.21.15.62:3300/0 conn(0x7f13880c2a90 0x7f1388156aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:50.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.281+0000 7f13877fe700 1 -- 172.21.15.31:0/70902166 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13881576d0 con 0x7f13880c8200 2023-10-06T02:28:50.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.281+0000 7f13877fe700 1 -- 172.21.15.31:0/70902166 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7f137c001470 con 0x7f13880c8200 2023-10-06T02:28:50.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.281+0000 7f13877fe700 1 -- 172.21.15.31:0/70902166 >> v1:172.21.15.31:6789/0 conn(0x7f13880c8200 legacy=0x7f13880c1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:50.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.281+0000 7f13877fe700 1 --2- 172.21.15.31:0/70902166 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1378003a60 0x7f1378003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:50.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.282+0000 7f13877fe700 1 --2- 172.21.15.31:0/70902166 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1378004810 0x7f1378004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:50.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.282+0000 7f13877fe700 1 --2- 172.21.15.31:0/70902166 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f13880c85d0 0x7f13780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:50.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.282+0000 7f13877fe700 1 -- 172.21.15.31:0/70902166 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13880c1390 con 0x7f1378003a60 2023-10-06T02:28:50.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.282+0000 7f13877fe700 1 -- 172.21.15.31:0/70902166 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13881573f0 con 0x7f1378004810 2023-10-06T02:28:50.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.282+0000 7f13877fe700 1 -- 172.21.15.31:0/70902166 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f137800ba60 con 0x7f13880c85d0 2023-10-06T02:28:50.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.282+0000 7f1387fff700 1 --2- 172.21.15.31:0/70902166 >> v2:172.21.15.62:3300/0 conn(0x7f13880c2a90 0x7f1388156aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:28:50.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.282+0000 7f138cb9b700 1 --2- 172.21.15.31:0/70902166 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1378003a60 0x7f1378003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:50.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.282+0000 7f138d39c700 1 --2- 172.21.15.31:0/70902166 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f13880c85d0 0x7f13780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:50.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.282+0000 7f1387fff700 1 --2- 172.21.15.31:0/70902166 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1378004810 0x7f1378004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:50.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.282+0000 7f138cb9b700 1 -- 172.21.15.31:0/70902166 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f13880c85d0 msgr2=0x7f13780051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:50.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.282+0000 7f138cb9b700 1 --2- 172.21.15.31:0/70902166 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f13880c85d0 0x7f13780051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:50.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.282+0000 7f138cb9b700 1 -- 172.21.15.31:0/70902166 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1378004810 msgr2=0x7f1378004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:50.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.282+0000 7f138cb9b700 1 --2- 172.21.15.31:0/70902166 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1378004810 0x7f1378004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:50.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.282+0000 7f138cb9b700 1 -- 172.21.15.31:0/70902166 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1378003570 con 0x7f1378003a60 2023-10-06T02:28:50.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.282+0000 7f1387fff700 1 --2- 172.21.15.31:0/70902166 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1378004810 0x7f1378004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:28:50.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.283+0000 7f138cb9b700 1 --2- 172.21.15.31:0/70902166 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1378003a60 0x7f1378003e70 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f137c00c3f0 tx=0x7f1388069d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd90285ab0555d01 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:50.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.283+0000 7f13877fe700 1 -- 172.21.15.31:0/70902166 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f137c00cc80 con 0x7f1378003a60 2023-10-06T02:28:50.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.283+0000 7f13877fe700 1 -- 172.21.15.31:0/70902166 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f137c00ce20 con 0x7f1378003a60 2023-10-06T02:28:50.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.283+0000 7f13877fe700 1 -- 172.21.15.31:0/70902166 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f137c005b70 con 0x7f1378003a60 2023-10-06T02:28:50.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.283+0000 7f138db9d700 1 -- 172.21.15.31:0/70902166 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1378003a60 msgr2=0x7f1378003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:50.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.283+0000 7f138db9d700 1 --2- 172.21.15.31:0/70902166 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1378003a60 0x7f1378003e70 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f137c00c3f0 tx=0x7f1388069d50 comp rx=0 tx=0).stop 2023-10-06T02:28:50.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.283+0000 7f138d39c700 1 -- 172.21.15.31:0/70902166 reap_dead start 2023-10-06T02:28:50.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.283+0000 7f138db9d700 1 -- 172.21.15.31:0/70902166 shutdown_connections 2023-10-06T02:28:50.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.283+0000 7f138db9d700 1 -- 172.21.15.31:0/70902166 >> 172.21.15.31:0/70902166 conn(0x7f138806c6a0 msgr2=0x7f138806caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:50.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.283+0000 7f138db9d700 1 -- 172.21.15.31:0/70902166 shutdown_connections 2023-10-06T02:28:50.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.284+0000 7f138db9d700 1 -- 172.21.15.31:0/70902166 wait complete. 2023-10-06T02:28:50.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.285+0000 7f138db9d700 1 Processor -- start 2023-10-06T02:28:50.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.285+0000 7f138db9d700 1 -- start start 2023-10-06T02:28:50.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.286+0000 7f138db9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1378003a60 0x7f138814df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:50.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.286+0000 7f138db9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f13880afbe0 0x7f13880b0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:50.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.286+0000 7f138db9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f138815e3d0 0x7f138815a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:50.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.286+0000 7f138db9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13880b1290 con 0x7f138815e3d0 2023-10-06T02:28:50.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.286+0000 7f138db9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f138815d3b0 con 0x7f1378003a60 2023-10-06T02:28:50.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.286+0000 7f138db9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f138815d650 con 0x7f13880afbe0 2023-10-06T02:28:50.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.286+0000 7f138d39c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f138815e3d0 0x7f138815a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:50.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.286+0000 7f138d39c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f138815e3d0 0x7f138815a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48842/0 (socket says 172.21.15.31:48842) 2023-10-06T02:28:50.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.286+0000 7f138d39c700 1 -- 172.21.15.31:0/2495132351 learned_addr learned my addr 172.21.15.31:0/2495132351 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:50.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.286+0000 7f138cb9b700 1 --2- 172.21.15.31:0/2495132351 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1378003a60 0x7f138814df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:50.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.286+0000 7f1387fff700 1 --2- 172.21.15.31:0/2495132351 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f13880afbe0 0x7f13880b0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:50.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.287+0000 7f138d39c700 1 -- 172.21.15.31:0/2495132351 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f13880afbe0 msgr2=0x7f13880b0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:50.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.287+0000 7f138d39c700 1 --2- 172.21.15.31:0/2495132351 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f13880afbe0 0x7f13880b0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:50.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.287+0000 7f138d39c700 1 -- 172.21.15.31:0/2495132351 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1378003a60 msgr2=0x7f138814df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:50.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.287+0000 7f138d39c700 1 --2- 172.21.15.31:0/2495132351 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1378003a60 0x7f138814df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:50.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.287+0000 7f138d39c700 1 -- 172.21.15.31:0/2495132351 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f138815d8f0 con 0x7f138815e3d0 2023-10-06T02:28:50.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.287+0000 7f138d39c700 1 --2- 172.21.15.31:0/2495132351 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f138815e3d0 0x7f138815a030 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f13880c2840 tx=0x7f138000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:50.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.287+0000 7f1384ff9700 1 -- 172.21.15.31:0/2495132351 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f138000e040 con 0x7f138815e3d0 2023-10-06T02:28:50.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.287+0000 7f1384ff9700 1 -- 172.21.15.31:0/2495132351 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f138000f040 con 0x7f138815e3d0 2023-10-06T02:28:50.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.287+0000 7f1384ff9700 1 -- 172.21.15.31:0/2495132351 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f138000fdc0 con 0x7f138815e3d0 2023-10-06T02:28:50.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.287+0000 7f138db9d700 1 -- 172.21.15.31:0/2495132351 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f138815dbd0 con 0x7f138815e3d0 2023-10-06T02:28:50.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.287+0000 7f138db9d700 1 -- 172.21.15.31:0/2495132351 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13880b0f80 con 0x7f138815e3d0 2023-10-06T02:28:50.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.288+0000 7f138db9d700 1 -- 172.21.15.31:0/2495132351 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f138815e0d0 con 0x7f138815e3d0 2023-10-06T02:28:50.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.289+0000 7f1384ff9700 1 -- 172.21.15.31:0/2495132351 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f1380015640 con 0x7f138815e3d0 2023-10-06T02:28:50.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.290+0000 7f1384ff9700 1 --2- 172.21.15.31:0/2495132351 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f136c067050 0x7f136c069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:50.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.290+0000 7f1384ff9700 1 -- 172.21.15.31:0/2495132351 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1380085210 con 0x7f138815e3d0 2023-10-06T02:28:50.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.291+0000 7f138cb9b700 1 --2- 172.21.15.31:0/2495132351 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f136c067050 0x7f136c069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:50.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.292+0000 7f138cb9b700 1 --2- 172.21.15.31:0/2495132351 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f136c067050 0x7f136c069500 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f137c00c3f0 tx=0x7f137c015000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:50.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.295+0000 7f1384ff9700 1 -- 172.21.15.31:0/2495132351 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1380052060 con 0x7f138815e3d0 2023-10-06T02:28:50.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.498+0000 7f1384ff9700 1 -- 172.21.15.31:0/2495132351 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mgrmap(e 18) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f1380052430 con 0x7f138815e3d0 2023-10-06T02:28:50.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.541+0000 7f138db9d700 1 -- 172.21.15.31:0/2495132351 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "fs subvolume resize", "vol_name": "cephfs", "sub_name": "subvolume_0000000000011953", "new_size": "10485760", "target": ["mon-mgr", ""]}) v1 -- 0x7f138814e490 con 0x7f136c067050 2023-10-06T02:28:50.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.570+0000 7f1384ff9700 1 -- 172.21.15.31:0/2495132351 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+128 (secure 0 0 0) 0x7f138814e490 con 0x7f136c067050 2023-10-06T02:28:50.571 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2023-10-06T02:28:50.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:28:50.572 INFO:teuthology.orchestra.run.smithi031.stdout: "bytes_used": 0 2023-10-06T02:28:50.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:28:50.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:28:50.572 INFO:teuthology.orchestra.run.smithi031.stdout: "bytes_quota": 10485760 2023-10-06T02:28:50.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:28:50.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:28:50.573 INFO:teuthology.orchestra.run.smithi031.stdout: "bytes_pcent": "0.00" 2023-10-06T02:28:50.573 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:28:50.573 INFO:teuthology.orchestra.run.smithi031.stdout:] 2023-10-06T02:28:50.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.573+0000 7f138db9d700 1 -- 172.21.15.31:0/2495132351 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f136c067050 msgr2=0x7f136c069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:50.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.573+0000 7f138db9d700 1 --2- 172.21.15.31:0/2495132351 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f136c067050 0x7f136c069500 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f137c00c3f0 tx=0x7f137c015000 comp rx=0 tx=0).stop 2023-10-06T02:28:50.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.573+0000 7f138db9d700 1 -- 172.21.15.31:0/2495132351 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f138815e3d0 msgr2=0x7f138815a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:50.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.573+0000 7f138db9d700 1 --2- 172.21.15.31:0/2495132351 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f138815e3d0 0x7f138815a030 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f13880c2840 tx=0x7f138000b040 comp rx=0 tx=0).stop 2023-10-06T02:28:50.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.573+0000 7f138db9d700 1 -- 172.21.15.31:0/2495132351 shutdown_connections 2023-10-06T02:28:50.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.573+0000 7f138db9d700 1 --2- 172.21.15.31:0/2495132351 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f138815e3d0 0x7f138815a030 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:50.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.573+0000 7f138db9d700 1 --2- 172.21.15.31:0/2495132351 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1378003a60 0x7f138814df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:50.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.573+0000 7f138db9d700 1 --2- 172.21.15.31:0/2495132351 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f136c067050 0x7f136c069500 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:50.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.573+0000 7f138db9d700 1 --2- 172.21.15.31:0/2495132351 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f13880afbe0 0x7f13880b0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:50.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.573+0000 7f138db9d700 1 -- 172.21.15.31:0/2495132351 >> 172.21.15.31:0/2495132351 conn(0x7f138806c6a0 msgr2=0x7f13880cb710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:50.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.573+0000 7f138db9d700 1 -- 172.21.15.31:0/2495132351 shutdown_connections 2023-10-06T02:28:50.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.573+0000 7f138db9d700 1 -- 172.21.15.31:0/2495132351 wait complete. 2023-10-06T02:28:50.591 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume info cephfs subvolume_0000000000011953 2023-10-06T02:28:50.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.780+0000 7f391a79d700 1 Processor -- start 2023-10-06T02:28:50.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.780+0000 7f391a79d700 1 -- start start 2023-10-06T02:28:50.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.781+0000 7f391a79d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f39140c7be0 0x7f391406abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:50.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.781+0000 7f391a79d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f391413f780 0x7f391413fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:50.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.781+0000 7f391a79d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39141400e0 con 0x7f39140c7fb0 2023-10-06T02:28:50.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.781+0000 7f391a79d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f391406bf10 con 0x7f39140c7be0 2023-10-06T02:28:50.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.781+0000 7f391a79d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f391406b290 con 0x7f391413f780 2023-10-06T02:28:50.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.781+0000 7f391979b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f39140c7be0 0x7f391406abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:50.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.781+0000 7f391979b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f39140c7be0 0x7f391406abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48850/0 (socket says 172.21.15.31:48850) 2023-10-06T02:28:50.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.781+0000 7f391979b700 1 -- 172.21.15.31:0/3719729166 learned_addr learned my addr 172.21.15.31:0/3719729166 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:50.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.782+0000 7f3919f9c700 1 --2- 172.21.15.31:0/3719729166 >> v2:172.21.15.62:3300/0 conn(0x7f391413f780 0x7f391413fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:50.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.782+0000 7f391979b700 1 -- 172.21.15.31:0/3719729166 >> v2:172.21.15.62:3300/0 conn(0x7f391413f780 msgr2=0x7f391413fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:50.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.782+0000 7f391979b700 1 --2- 172.21.15.31:0/3719729166 >> v2:172.21.15.62:3300/0 conn(0x7f391413f780 0x7f391413fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:50.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.782+0000 7f391979b700 1 -- 172.21.15.31:0/3719729166 >> v1:172.21.15.62:6789/0 conn(0x7f39140c7fb0 legacy=0x7f391415a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:50.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.782+0000 7f391979b700 1 -- 172.21.15.31:0/3719729166 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f391406b570 con 0x7f39140c7be0 2023-10-06T02:28:50.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.782+0000 7f391979b700 1 --2- 172.21.15.31:0/3719729166 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39140c7be0 0x7f391406abd0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f39140cba10 tx=0x7f3910009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2bd908d2bca8d1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:50.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.782+0000 7f390bfff700 1 -- 172.21.15.31:0/3719729166 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f391000b860 con 0x7f39140c7be0 2023-10-06T02:28:50.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.782+0000 7f390bfff700 1 -- 172.21.15.31:0/3719729166 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f391000b9c0 con 0x7f39140c7be0 2023-10-06T02:28:50.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.782+0000 7f390bfff700 1 -- 172.21.15.31:0/3719729166 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f391000bd60 con 0x7f39140c7be0 2023-10-06T02:28:50.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.783+0000 7f391a79d700 1 -- 172.21.15.31:0/3719729166 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39140c7be0 msgr2=0x7f391406abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:50.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.783+0000 7f391a79d700 1 --2- 172.21.15.31:0/3719729166 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39140c7be0 0x7f391406abd0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f39140cba10 tx=0x7f3910009250 comp rx=0 tx=0).stop 2023-10-06T02:28:50.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.783+0000 7f391a79d700 1 -- 172.21.15.31:0/3719729166 shutdown_connections 2023-10-06T02:28:50.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.783+0000 7f391a79d700 1 --2- 172.21.15.31:0/3719729166 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39140c7be0 0x7f391406abd0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:50.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.783+0000 7f391a79d700 1 --2- 172.21.15.31:0/3719729166 >> v2:172.21.15.62:3300/0 conn(0x7f391413f780 0x7f391413fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:50.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.783+0000 7f391a79d700 1 -- 172.21.15.31:0/3719729166 >> 172.21.15.31:0/3719729166 conn(0x7f391406d110 msgr2=0x7f39140cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:50.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.783+0000 7f391a79d700 1 -- 172.21.15.31:0/3719729166 shutdown_connections 2023-10-06T02:28:50.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.783+0000 7f391a79d700 1 -- 172.21.15.31:0/3719729166 wait complete. 2023-10-06T02:28:50.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.784+0000 7f391a79d700 1 Processor -- start 2023-10-06T02:28:50.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.785+0000 7f391a79d700 1 -- start start 2023-10-06T02:28:50.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.785+0000 7f391a79d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39140c7be0 0x7f39140aeed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:50.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.785+0000 7f391a79d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39140c7fb0 0x7f391416ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:50.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.785+0000 7f391a79d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f391413f780 0x7f39140c31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:50.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.785+0000 7f391a79d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39140c4920 con 0x7f39140c7be0 2023-10-06T02:28:50.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.785+0000 7f391a79d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39140c3d10 con 0x7f391413f780 2023-10-06T02:28:50.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.785+0000 7f391a79d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39140c3ff0 con 0x7f39140c7fb0 2023-10-06T02:28:50.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.786+0000 7f391979b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39140c7be0 0x7f39140aeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:50.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.786+0000 7f391979b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39140c7be0 0x7f39140aeed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48866/0 (socket says 172.21.15.31:48866) 2023-10-06T02:28:50.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.786+0000 7f391979b700 1 -- 172.21.15.31:0/4051074080 learned_addr learned my addr 172.21.15.31:0/4051074080 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:50.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.786+0000 7f3919f9c700 1 --2- 172.21.15.31:0/4051074080 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f391413f780 0x7f39140c31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:50.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.786+0000 7f391979b700 1 -- 172.21.15.31:0/4051074080 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39140c7fb0 msgr2=0x7f391416ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:50.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.786+0000 7f3918f9a700 1 --2- 172.21.15.31:0/4051074080 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39140c7fb0 0x7f391416ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:50.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.786+0000 7f391979b700 1 --2- 172.21.15.31:0/4051074080 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39140c7fb0 0x7f391416ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:50.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.786+0000 7f391979b700 1 -- 172.21.15.31:0/4051074080 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f391413f780 msgr2=0x7f39140c31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:50.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.786+0000 7f391979b700 1 --2- 172.21.15.31:0/4051074080 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f391413f780 0x7f39140c31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:50.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.786+0000 7f391979b700 1 -- 172.21.15.31:0/4051074080 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39100094f0 con 0x7f39140c7be0 2023-10-06T02:28:50.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.786+0000 7f3919f9c700 1 --2- 172.21.15.31:0/4051074080 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f391413f780 0x7f39140c31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:28:50.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.786+0000 7f3918f9a700 1 --2- 172.21.15.31:0/4051074080 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39140c7fb0 0x7f391416ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:28:50.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.786+0000 7f391979b700 1 --2- 172.21.15.31:0/4051074080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39140c7be0 0x7f39140aeed0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f39140693f0 tx=0x7f3910017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:50.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.787+0000 7f39097fa700 1 -- 172.21.15.31:0/4051074080 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3910011e60 con 0x7f39140c7be0 2023-10-06T02:28:50.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.787+0000 7f39097fa700 1 -- 172.21.15.31:0/4051074080 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f391001a630 con 0x7f39140c7be0 2023-10-06T02:28:50.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.787+0000 7f391a79d700 1 -- 172.21.15.31:0/4051074080 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39140c42d0 con 0x7f39140c7be0 2023-10-06T02:28:50.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.787+0000 7f39097fa700 1 -- 172.21.15.31:0/4051074080 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f391001a9f0 con 0x7f39140c7be0 2023-10-06T02:28:50.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.787+0000 7f391a79d700 1 -- 172.21.15.31:0/4051074080 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39141561e0 con 0x7f39140c7be0 2023-10-06T02:28:50.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.788+0000 7f391a79d700 1 -- 172.21.15.31:0/4051074080 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f391404c3c0 con 0x7f39140c7be0 2023-10-06T02:28:50.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.789+0000 7f39097fa700 1 -- 172.21.15.31:0/4051074080 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f391002d440 con 0x7f39140c7be0 2023-10-06T02:28:50.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.790+0000 7f39097fa700 1 --2- 172.21.15.31:0/4051074080 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f38f80670e0 0x7f38f8069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:50.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.790+0000 7f39097fa700 1 -- 172.21.15.31:0/4051074080 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f39100866c0 con 0x7f39140c7be0 2023-10-06T02:28:50.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.790+0000 7f3918f9a700 1 --2- 172.21.15.31:0/4051074080 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f38f80670e0 0x7f38f8069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:50.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.791+0000 7f3918f9a700 1 --2- 172.21.15.31:0/4051074080 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f38f80670e0 0x7f38f8069590 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f391413cae0 tx=0x7f3904009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:50.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:50.794+0000 7f39097fa700 1 -- 172.21.15.31:0/4051074080 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f39100534b0 con 0x7f39140c7be0 2023-10-06T02:28:51.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.034+0000 7f391a79d700 1 -- 172.21.15.31:0/4051074080 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "fs subvolume info", "vol_name": "cephfs", "sub_name": "subvolume_0000000000011953", "target": ["mon-mgr", ""]}) v1 -- 0x7f39140c4a60 con 0x7f38f80670e0 2023-10-06T02:28:51.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.064+0000 7f39097fa700 1 -- 172.21.15.31:0/4051074080 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+675 (secure 0 0 0) 0x7f390400d660 con 0x7f38f80670e0 2023-10-06T02:28:51.065 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T02:28:51.065 INFO:teuthology.orchestra.run.smithi031.stdout: "atime": "2023-10-06 02:28:47", 2023-10-06T02:28:51.065 INFO:teuthology.orchestra.run.smithi031.stdout: "bytes_pcent": "0.00", 2023-10-06T02:28:51.065 INFO:teuthology.orchestra.run.smithi031.stdout: "bytes_quota": 10485760, 2023-10-06T02:28:51.065 INFO:teuthology.orchestra.run.smithi031.stdout: "bytes_used": 0, 2023-10-06T02:28:51.066 INFO:teuthology.orchestra.run.smithi031.stdout: "created_at": "2023-10-06 02:28:47", 2023-10-06T02:28:51.066 INFO:teuthology.orchestra.run.smithi031.stdout: "ctime": "2023-10-06 02:28:50", 2023-10-06T02:28:51.066 INFO:teuthology.orchestra.run.smithi031.stdout: "data_pool": "cephfs_data", 2023-10-06T02:28:51.066 INFO:teuthology.orchestra.run.smithi031.stdout: "features": [ 2023-10-06T02:28:51.066 INFO:teuthology.orchestra.run.smithi031.stdout: "snapshot-clone", 2023-10-06T02:28:51.067 INFO:teuthology.orchestra.run.smithi031.stdout: "snapshot-autoprotect" 2023-10-06T02:28:51.067 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:28:51.067 INFO:teuthology.orchestra.run.smithi031.stdout: "gid": 0, 2023-10-06T02:28:51.067 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": 16877, 2023-10-06T02:28:51.067 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_addrs": [ 2023-10-06T02:28:51.067 INFO:teuthology.orchestra.run.smithi031.stdout: "172.21.15.31:6789", 2023-10-06T02:28:51.067 INFO:teuthology.orchestra.run.smithi031.stdout: "172.21.15.62:6789", 2023-10-06T02:28:51.068 INFO:teuthology.orchestra.run.smithi031.stdout: "172.21.15.62:6790" 2023-10-06T02:28:51.068 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:28:51.068 INFO:teuthology.orchestra.run.smithi031.stdout: "mtime": "2023-10-06 02:28:47", 2023-10-06T02:28:51.068 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/volumes/_nogroup/subvolume_0000000000011953/5172ced3-24b3-4d86-a22c-dcfea079edc7", 2023-10-06T02:28:51.068 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_namespace": "", 2023-10-06T02:28:51.068 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "complete", 2023-10-06T02:28:51.068 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "subvolume", 2023-10-06T02:28:51.069 INFO:teuthology.orchestra.run.smithi031.stdout: "uid": 0 2023-10-06T02:28:51.069 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T02:28:51.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.066+0000 7f391a79d700 1 -- 172.21.15.31:0/4051074080 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f38f80670e0 msgr2=0x7f38f8069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:51.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.066+0000 7f391a79d700 1 --2- 172.21.15.31:0/4051074080 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f38f80670e0 0x7f38f8069590 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f391413cae0 tx=0x7f3904009040 comp rx=0 tx=0).stop 2023-10-06T02:28:51.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.066+0000 7f391a79d700 1 -- 172.21.15.31:0/4051074080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39140c7be0 msgr2=0x7f39140aeed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:51.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.066+0000 7f391a79d700 1 --2- 172.21.15.31:0/4051074080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39140c7be0 0x7f39140aeed0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f39140693f0 tx=0x7f3910017000 comp rx=0 tx=0).stop 2023-10-06T02:28:51.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.066+0000 7f391a79d700 1 -- 172.21.15.31:0/4051074080 shutdown_connections 2023-10-06T02:28:51.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.066+0000 7f391a79d700 1 --2- 172.21.15.31:0/4051074080 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f391413f780 0x7f39140c31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:51.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.066+0000 7f391a79d700 1 --2- 172.21.15.31:0/4051074080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39140c7be0 0x7f39140aeed0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:51.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.066+0000 7f391a79d700 1 --2- 172.21.15.31:0/4051074080 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f38f80670e0 0x7f38f8069590 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:51.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.066+0000 7f391a79d700 1 --2- 172.21.15.31:0/4051074080 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39140c7fb0 0x7f391416ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:51.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.066+0000 7f391a79d700 1 -- 172.21.15.31:0/4051074080 >> 172.21.15.31:0/4051074080 conn(0x7f391406d110 msgr2=0x7f39140cb220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:51.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.066+0000 7f391a79d700 1 -- 172.21.15.31:0/4051074080 shutdown_connections 2023-10-06T02:28:51.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.066+0000 7f391a79d700 1 -- 172.21.15.31:0/4051074080 wait complete. 2023-10-06T02:28:51.085 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000011953 --mode 777 --uid 1000 --gid 1000 2023-10-06T02:28:51.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.284+0000 7fd5b5d9d700 1 Processor -- start 2023-10-06T02:28:51.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.284+0000 7fd5b5d9d700 1 -- start start 2023-10-06T02:28:51.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.285+0000 7fd5b5d9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd5b0140a70 0x7fd5b00c6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:51.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.285+0000 7fd5b5d9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd5b006de50 0x7fd5b00aeef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:51.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.285+0000 7fd5b5d9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5b00c6d50 con 0x7fd5b0140e40 2023-10-06T02:28:51.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.285+0000 7fd5b5d9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5b00c7030 con 0x7fd5b006de50 2023-10-06T02:28:51.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.285+0000 7fd5b5d9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5b00c7310 con 0x7fd5b0140a70 2023-10-06T02:28:51.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.285+0000 7fd5b559c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd5b006de50 0x7fd5b00aeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:51.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.285+0000 7fd5b559c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd5b006de50 0x7fd5b00aeef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48870/0 (socket says 172.21.15.31:48870) 2023-10-06T02:28:51.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.285+0000 7fd5b559c700 1 -- 172.21.15.31:0/3901556589 learned_addr learned my addr 172.21.15.31:0/3901556589 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:51.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.286+0000 7fd5b4d9b700 1 --2- 172.21.15.31:0/3901556589 >> v2:172.21.15.62:3300/0 conn(0x7fd5b0140a70 0x7fd5b00c6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:51.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.286+0000 7fd5b559c700 1 -- 172.21.15.31:0/3901556589 >> v2:172.21.15.62:3300/0 conn(0x7fd5b0140a70 msgr2=0x7fd5b00c6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:51.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.286+0000 7fd5b559c700 1 --2- 172.21.15.31:0/3901556589 >> v2:172.21.15.62:3300/0 conn(0x7fd5b0140a70 0x7fd5b00c6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:51.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.286+0000 7fd5b559c700 1 -- 172.21.15.31:0/3901556589 >> v1:172.21.15.62:6789/0 conn(0x7fd5b0140e40 legacy=0x7fd5b00c7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:51.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.286+0000 7fd5b559c700 1 -- 172.21.15.31:0/3901556589 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5b00c75f0 con 0x7fd5b006de50 2023-10-06T02:28:51.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.286+0000 7fd5b559c700 1 --2- 172.21.15.31:0/3901556589 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5b006de50 0x7fd5b00aeef0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0069230 tx=0x7fd5a8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee89e50504118943 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:51.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.286+0000 7fd5af7fe700 1 -- 172.21.15.31:0/3901556589 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a800e070 con 0x7fd5b006de50 2023-10-06T02:28:51.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.286+0000 7fd5af7fe700 1 -- 172.21.15.31:0/3901556589 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5a800ba50 con 0x7fd5b006de50 2023-10-06T02:28:51.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.286+0000 7fd5af7fe700 1 -- 172.21.15.31:0/3901556589 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a80092e0 con 0x7fd5b006de50 2023-10-06T02:28:51.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.287+0000 7fd5b5d9d700 1 -- 172.21.15.31:0/3901556589 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5b006de50 msgr2=0x7fd5b00aeef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:51.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.287+0000 7fd5b5d9d700 1 --2- 172.21.15.31:0/3901556589 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5b006de50 0x7fd5b00aeef0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0069230 tx=0x7fd5a8009840 comp rx=0 tx=0).stop 2023-10-06T02:28:51.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.287+0000 7fd5b5d9d700 1 -- 172.21.15.31:0/3901556589 shutdown_connections 2023-10-06T02:28:51.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.287+0000 7fd5b5d9d700 1 --2- 172.21.15.31:0/3901556589 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5b006de50 0x7fd5b00aeef0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:51.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.287+0000 7fd5b5d9d700 1 --2- 172.21.15.31:0/3901556589 >> v2:172.21.15.62:3300/0 conn(0x7fd5b0140a70 0x7fd5b00c6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:51.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.287+0000 7fd5b5d9d700 1 -- 172.21.15.31:0/3901556589 >> 172.21.15.31:0/3901556589 conn(0x7fd5b006d170 msgr2=0x7fd5b00cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:51.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.287+0000 7fd5b5d9d700 1 -- 172.21.15.31:0/3901556589 shutdown_connections 2023-10-06T02:28:51.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.287+0000 7fd5b5d9d700 1 -- 172.21.15.31:0/3901556589 wait complete. 2023-10-06T02:28:51.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.288+0000 7fd5b5d9d700 1 Processor -- start 2023-10-06T02:28:51.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.289+0000 7fd5b5d9d700 1 -- start start 2023-10-06T02:28:51.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.289+0000 7fd5b5d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5b006de50 0x7fd5b013bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:51.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.290+0000 7fd5b5d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5b0140a70 0x7fd5b00b0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:51.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.290+0000 7fd5b5d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd5b0140e40 0x7fd5b00b12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:51.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.290+0000 7fd5b5d9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b0168de0 con 0x7fd5b006de50 2023-10-06T02:28:51.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.290+0000 7fd5b5d9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b0169760 con 0x7fd5b0140a70 2023-10-06T02:28:51.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.290+0000 7fd5b5d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b013c330 con 0x7fd5b0140e40 2023-10-06T02:28:51.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.290+0000 7fd5b4d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5b006de50 0x7fd5b013bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:51.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.290+0000 7fd5b4d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5b006de50 0x7fd5b013bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48884/0 (socket says 172.21.15.31:48884) 2023-10-06T02:28:51.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.290+0000 7fd5b4d9b700 1 -- 172.21.15.31:0/3582725401 learned_addr learned my addr 172.21.15.31:0/3582725401 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:51.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.290+0000 7fd5b559c700 1 --2- 172.21.15.31:0/3582725401 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd5b0140e40 0x7fd5b00b12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:51.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.290+0000 7fd5affff700 1 --2- 172.21.15.31:0/3582725401 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5b0140a70 0x7fd5b00b0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:51.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.290+0000 7fd5b4d9b700 1 -- 172.21.15.31:0/3582725401 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd5b0140e40 msgr2=0x7fd5b00b12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:51.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.290+0000 7fd5b4d9b700 1 --2- 172.21.15.31:0/3582725401 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd5b0140e40 0x7fd5b00b12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:51.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.290+0000 7fd5b4d9b700 1 -- 172.21.15.31:0/3582725401 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5b0140a70 msgr2=0x7fd5b00b0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:51.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.290+0000 7fd5b4d9b700 1 --2- 172.21.15.31:0/3582725401 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5b0140a70 0x7fd5b00b0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:51.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.290+0000 7fd5b4d9b700 1 -- 172.21.15.31:0/3582725401 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5a8009ae0 con 0x7fd5b006de50 2023-10-06T02:28:51.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.291+0000 7fd5b4d9b700 1 --2- 172.21.15.31:0/3582725401 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5b006de50 0x7fd5b013bdf0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fd5a400d890 tx=0x7fd5a4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:51.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.291+0000 7fd5acff9700 1 -- 172.21.15.31:0/3582725401 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a40125f0 con 0x7fd5b006de50 2023-10-06T02:28:51.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.291+0000 7fd5acff9700 1 -- 172.21.15.31:0/3582725401 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5a4012770 con 0x7fd5b006de50 2023-10-06T02:28:51.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.291+0000 7fd5b5d9d700 1 -- 172.21.15.31:0/3582725401 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5b0168b00 con 0x7fd5b006de50 2023-10-06T02:28:51.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.291+0000 7fd5acff9700 1 -- 172.21.15.31:0/3582725401 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a4012b30 con 0x7fd5b006de50 2023-10-06T02:28:51.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.291+0000 7fd5b5d9d700 1 -- 172.21.15.31:0/3582725401 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5b01693f0 con 0x7fd5b006de50 2023-10-06T02:28:51.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.292+0000 7fd5b5d9d700 1 -- 172.21.15.31:0/3582725401 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5b0168f70 con 0x7fd5b006de50 2023-10-06T02:28:51.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.293+0000 7fd5acff9700 1 -- 172.21.15.31:0/3582725401 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) v1 ==== 84893+0+0 (secure 0 0 0) 0x7fd5a401e460 con 0x7fd5b006de50 2023-10-06T02:28:51.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.294+0000 7fd5acff9700 1 --2- 172.21.15.31:0/3582725401 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd5940670e0 0x7fd594069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:51.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.294+0000 7fd5acff9700 1 -- 172.21.15.31:0/3582725401 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd5a4084fc0 con 0x7fd5b006de50 2023-10-06T02:28:51.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.294+0000 7fd5affff700 1 --2- 172.21.15.31:0/3582725401 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd5940670e0 0x7fd594069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:51.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.295+0000 7fd5affff700 1 --2- 172.21.15.31:0/3582725401 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd5940670e0 0x7fd594069590 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd5b01412a0 tx=0x7fd59c007f10 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:51.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.298+0000 7fd5acff9700 1 -- 172.21.15.31:0/3582725401 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd5b0168f70 con 0x7fd5b006de50 2023-10-06T02:28:51.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.535+0000 7fd5b5d9d700 1 -- 172.21.15.31:0/3582725401 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000011953", "mode": "777", "uid": 1000, "gid": 1000, "target": ["mon-mgr", ""]}) v1 -- 0x7fd5b0168f70 con 0x7fd5940670e0 2023-10-06T02:28:51.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.568+0000 7fd5acff9700 1 -- 172.21.15.31:0/3582725401 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fd5b0168f70 con 0x7fd5940670e0 2023-10-06T02:28:51.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.571+0000 7fd5b5d9d700 1 -- 172.21.15.31:0/3582725401 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd5940670e0 msgr2=0x7fd594069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:51.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.571+0000 7fd5b5d9d700 1 --2- 172.21.15.31:0/3582725401 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd5940670e0 0x7fd594069590 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd5b01412a0 tx=0x7fd59c007f10 comp rx=0 tx=0).stop 2023-10-06T02:28:51.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.571+0000 7fd5b5d9d700 1 -- 172.21.15.31:0/3582725401 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5b006de50 msgr2=0x7fd5b013bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:51.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.571+0000 7fd5b5d9d700 1 --2- 172.21.15.31:0/3582725401 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5b006de50 0x7fd5b013bdf0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fd5a400d890 tx=0x7fd5a4008040 comp rx=0 tx=0).stop 2023-10-06T02:28:51.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.571+0000 7fd5b5d9d700 1 -- 172.21.15.31:0/3582725401 shutdown_connections 2023-10-06T02:28:51.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.571+0000 7fd5b5d9d700 1 --2- 172.21.15.31:0/3582725401 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd5940670e0 0x7fd594069590 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:51.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.571+0000 7fd5b5d9d700 1 --2- 172.21.15.31:0/3582725401 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd5b0140e40 0x7fd5b00b12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:51.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.571+0000 7fd5b5d9d700 1 --2- 172.21.15.31:0/3582725401 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5b006de50 0x7fd5b013bdf0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:51.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.571+0000 7fd5b5d9d700 1 --2- 172.21.15.31:0/3582725401 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5b0140a70 0x7fd5b00b0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:51.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.571+0000 7fd5b5d9d700 1 -- 172.21.15.31:0/3582725401 >> 172.21.15.31:0/3582725401 conn(0x7fd5b006d170 msgr2=0x7fd5b00c2410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:51.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.571+0000 7fd5b5d9d700 1 -- 172.21.15.31:0/3582725401 shutdown_connections 2023-10-06T02:28:51.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.571+0000 7fd5b5d9d700 1 -- 172.21.15.31:0/3582725401 wait complete. 2023-10-06T02:28:51.588 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000011953 2023-10-06T02:28:51.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.778+0000 7f6026f9d700 1 Processor -- start 2023-10-06T02:28:51.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.778+0000 7f6026f9d700 1 -- start start 2023-10-06T02:28:51.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.779+0000 7f6026f9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f6020153a00 0x7f6020153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:51.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.779+0000 7f6026f9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f60200cea00 0x7f60200cee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:51.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.779+0000 7f6026f9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f602015ac50 con 0x7f602015de50 2023-10-06T02:28:51.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.779+0000 7f6026f9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f602015d010 con 0x7f60200cea00 2023-10-06T02:28:51.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.779+0000 7f6026f9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60200cf3c0 con 0x7f6020153a00 2023-10-06T02:28:51.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.780+0000 7f602679c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f60200cea00 0x7f60200cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:51.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.780+0000 7f602679c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f60200cea00 0x7f60200cee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48888/0 (socket says 172.21.15.31:48888) 2023-10-06T02:28:51.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.780+0000 7f602679c700 1 -- 172.21.15.31:0/39681951 learned_addr learned my addr 172.21.15.31:0/39681951 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:51.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.780+0000 7f602579a700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f602015de50 0x7f602015a540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:39386/0 (socket says 172.21.15.31:39386) 2023-10-06T02:28:51.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.780+0000 7f6025f9b700 1 --2- 172.21.15.31:0/39681951 >> v2:172.21.15.62:3300/0 conn(0x7f6020153a00 0x7f6020153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:51.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.780+0000 7f602679c700 1 -- 172.21.15.31:0/39681951 >> v2:172.21.15.62:3300/0 conn(0x7f6020153a00 msgr2=0x7f6020153dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:51.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.780+0000 7f602679c700 1 --2- 172.21.15.31:0/39681951 >> v2:172.21.15.62:3300/0 conn(0x7f6020153a00 0x7f6020153dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:51.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.780+0000 7f602679c700 1 -- 172.21.15.31:0/39681951 >> v1:172.21.15.31:6789/0 conn(0x7f602015de50 legacy=0x7f602015a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:51.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.780+0000 7f602679c700 1 -- 172.21.15.31:0/39681951 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f602015af60 con 0x7f60200cea00 2023-10-06T02:28:51.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.780+0000 7f6024f99700 1 -- 172.21.15.31:0/39681951 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1026889143 0 0) 0x7f602015ac50 con 0x7f602015de50 2023-10-06T02:28:51.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.780+0000 7f602679c700 1 --2- 172.21.15.31:0/39681951 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60200cea00 0x7f60200cee80 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f60200cac20 tx=0x7f6018009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14e314cc3d48ffb1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:51.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.781+0000 7f6024f99700 1 -- 172.21.15.31:0/39681951 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601800e070 con 0x7f60200cea00 2023-10-06T02:28:51.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.781+0000 7f6024f99700 1 -- 172.21.15.31:0/39681951 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f601800ba20 con 0x7f60200cea00 2023-10-06T02:28:51.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.781+0000 7f6024f99700 1 -- 172.21.15.31:0/39681951 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60180092e0 con 0x7f60200cea00 2023-10-06T02:28:51.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.781+0000 7f6026f9d700 1 -- 172.21.15.31:0/39681951 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60200cea00 msgr2=0x7f60200cee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:51.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.781+0000 7f6026f9d700 1 --2- 172.21.15.31:0/39681951 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60200cea00 0x7f60200cee80 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f60200cac20 tx=0x7f6018009840 comp rx=0 tx=0).stop 2023-10-06T02:28:51.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.781+0000 7f6026f9d700 1 -- 172.21.15.31:0/39681951 shutdown_connections 2023-10-06T02:28:51.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.781+0000 7f6026f9d700 1 --2- 172.21.15.31:0/39681951 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60200cea00 0x7f60200cee80 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:51.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.781+0000 7f6026f9d700 1 --2- 172.21.15.31:0/39681951 >> v2:172.21.15.62:3300/0 conn(0x7f6020153a00 0x7f6020153dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:51.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.781+0000 7f6026f9d700 1 -- 172.21.15.31:0/39681951 >> 172.21.15.31:0/39681951 conn(0x7f60200703a0 msgr2=0x7f60200707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:51.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.781+0000 7f6026f9d700 1 -- 172.21.15.31:0/39681951 shutdown_connections 2023-10-06T02:28:51.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.781+0000 7f6026f9d700 1 -- 172.21.15.31:0/39681951 wait complete. 2023-10-06T02:28:51.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.783+0000 7f6026f9d700 1 Processor -- start 2023-10-06T02:28:51.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.783+0000 7f6026f9d700 1 -- start start 2023-10-06T02:28:51.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.784+0000 7f6026f9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60200cea00 0x7f6020153100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:51.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.784+0000 7f6026f9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6020153a00 0x7f60200cc290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:51.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.784+0000 7f6026f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f602015de50 0x7f60200cc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:51.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.784+0000 7f6026f9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6020153640 con 0x7f60200cea00 2023-10-06T02:28:51.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.784+0000 7f6026f9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60201542f0 con 0x7f6020153a00 2023-10-06T02:28:51.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.784+0000 7f6026f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6020158480 con 0x7f602015de50 2023-10-06T02:28:51.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.784+0000 7f6025f9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60200cea00 0x7f6020153100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:51.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.784+0000 7f6025f9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60200cea00 0x7f6020153100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48890/0 (socket says 172.21.15.31:48890) 2023-10-06T02:28:51.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.784+0000 7f6025f9b700 1 -- 172.21.15.31:0/1845701814 learned_addr learned my addr 172.21.15.31:0/1845701814 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:51.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.784+0000 7f6025f9b700 1 -- 172.21.15.31:0/1845701814 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f602015de50 msgr2=0x7f60200cc7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:51.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.784+0000 7f6025f9b700 1 --2- 172.21.15.31:0/1845701814 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f602015de50 0x7f60200cc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:51.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.784+0000 7f6025f9b700 1 -- 172.21.15.31:0/1845701814 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6020153a00 msgr2=0x7f60200cc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:51.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.784+0000 7f6025f9b700 1 --2- 172.21.15.31:0/1845701814 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6020153a00 0x7f60200cc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:51.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.784+0000 7f6025f9b700 1 -- 172.21.15.31:0/1845701814 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6018009ae0 con 0x7f60200cea00 2023-10-06T02:28:51.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.785+0000 7f6025f9b700 1 --2- 172.21.15.31:0/1845701814 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60200cea00 0x7f6020153100 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f601c00ec80 tx=0x7f601c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:51.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.785+0000 7f6015ffb700 1 -- 172.21.15.31:0/1845701814 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601c00c9d0 con 0x7f60200cea00 2023-10-06T02:28:51.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.785+0000 7f6015ffb700 1 -- 172.21.15.31:0/1845701814 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f601c00cb30 con 0x7f60200cea00 2023-10-06T02:28:51.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.785+0000 7f6015ffb700 1 -- 172.21.15.31:0/1845701814 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601c013470 con 0x7f60200cea00 2023-10-06T02:28:51.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.785+0000 7f6026f9d700 1 -- 172.21.15.31:0/1845701814 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6020158760 con 0x7f60200cea00 2023-10-06T02:28:51.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.785+0000 7f6026f9d700 1 -- 172.21.15.31:0/1845701814 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6020153f10 con 0x7f60200cea00 2023-10-06T02:28:51.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.786+0000 7f6026f9d700 1 -- 172.21.15.31:0/1845701814 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f602004c3c0 con 0x7f60200cea00 2023-10-06T02:28:51.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.787+0000 7f6015ffb700 1 -- 172.21.15.31:0/1845701814 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f601c013820 con 0x7f60200cea00 2023-10-06T02:28:51.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.788+0000 7f6015ffb700 1 --2- 172.21.15.31:0/1845701814 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f60040670e0 0x7f6004069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:51.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.788+0000 7f6015ffb700 1 -- 172.21.15.31:0/1845701814 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f601c084d30 con 0x7f60200cea00 2023-10-06T02:28:51.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.789+0000 7f602579a700 1 --2- 172.21.15.31:0/1845701814 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f60040670e0 0x7f6004069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:51.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.789+0000 7f602579a700 1 --2- 172.21.15.31:0/1845701814 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f60040670e0 0x7f6004069590 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f60100011e0 tx=0x7f60100079b0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:51.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:51.792+0000 7f6015ffb700 1 -- 172.21.15.31:0/1845701814 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f601c051bc0 con 0x7f60200cea00 2023-10-06T02:28:52.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.029+0000 7f6026f9d700 1 -- 172.21.15.31:0/1845701814 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000011953", "target": ["mon-mgr", ""]}) v1 -- 0x7f60200d0170 con 0x7f60040670e0 2023-10-06T02:28:52.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.064+0000 7f6015ffb700 1 -- 172.21.15.31:0/1845701814 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+82 (secure 0 0 0) 0x7f60200d0170 con 0x7f60040670e0 2023-10-06T02:28:52.065 INFO:teuthology.orchestra.run.smithi031.stdout:/volumes/_nogroup/subvolume_0000000000011953/5172ced3-24b3-4d86-a22c-dcfea079edc7 2023-10-06T02:28:52.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.066+0000 7f6026f9d700 1 -- 172.21.15.31:0/1845701814 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f60040670e0 msgr2=0x7f6004069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:52.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.066+0000 7f6026f9d700 1 --2- 172.21.15.31:0/1845701814 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f60040670e0 0x7f6004069590 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f60100011e0 tx=0x7f60100079b0 comp rx=0 tx=0).stop 2023-10-06T02:28:52.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.066+0000 7f6026f9d700 1 -- 172.21.15.31:0/1845701814 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60200cea00 msgr2=0x7f6020153100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:52.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.066+0000 7f6026f9d700 1 --2- 172.21.15.31:0/1845701814 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60200cea00 0x7f6020153100 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f601c00ec80 tx=0x7f601c0091a0 comp rx=0 tx=0).stop 2023-10-06T02:28:52.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.066+0000 7f6026f9d700 1 -- 172.21.15.31:0/1845701814 shutdown_connections 2023-10-06T02:28:52.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.066+0000 7f6026f9d700 1 --2- 172.21.15.31:0/1845701814 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f60040670e0 0x7f6004069590 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:52.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.066+0000 7f6026f9d700 1 --2- 172.21.15.31:0/1845701814 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f602015de50 0x7f60200cc7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:52.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.066+0000 7f6026f9d700 1 --2- 172.21.15.31:0/1845701814 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60200cea00 0x7f6020153100 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:52.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.066+0000 7f6026f9d700 1 --2- 172.21.15.31:0/1845701814 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6020153a00 0x7f60200cc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:52.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.066+0000 7f6026f9d700 1 -- 172.21.15.31:0/1845701814 >> 172.21.15.31:0/1845701814 conn(0x7f60200703a0 msgr2=0x7f60200b1000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:52.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.067+0000 7f6026f9d700 1 -- 172.21.15.31:0/1845701814 shutdown_connections 2023-10-06T02:28:52.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.067+0000 7f6026f9d700 1 -- 172.21.15.31:0/1845701814 wait complete. 2023-10-06T02:28:52.084 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000011953 with 8 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000011953/5172ced3-24b3-4d86-a22c-dcfea079edc7 2023-10-06T02:28:52.084 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:28:52.085 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000011953/5172ced3-24b3-4d86-a22c-dcfea079edc7/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0) 2023-10-06T02:28:52.146 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2023-10-06T02:28:52.146 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2023-10-06T02:28:52.146 INFO:teuthology.orchestra.run.smithi031.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184168 s, 56.9 MB/s 2023-10-06T02:28:52.147 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:28:52.147 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000011953/5172ced3-24b3-4d86-a22c-dcfea079edc7/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0) 2023-10-06T02:28:52.202 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2023-10-06T02:28:52.202 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2023-10-06T02:28:52.202 INFO:teuthology.orchestra.run.smithi031.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018202 s, 57.6 MB/s 2023-10-06T02:28:52.203 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:28:52.203 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000011953/5172ced3-24b3-4d86-a22c-dcfea079edc7/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0) 2023-10-06T02:28:52.258 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2023-10-06T02:28:52.258 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2023-10-06T02:28:52.258 INFO:teuthology.orchestra.run.smithi031.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017525 s, 59.8 MB/s 2023-10-06T02:28:52.259 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:28:52.259 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000011953/5172ced3-24b3-4d86-a22c-dcfea079edc7/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0) 2023-10-06T02:28:52.311 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2023-10-06T02:28:52.311 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2023-10-06T02:28:52.311 INFO:teuthology.orchestra.run.smithi031.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150046 s, 69.9 MB/s 2023-10-06T02:28:52.312 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:28:52.312 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000011953/5172ced3-24b3-4d86-a22c-dcfea079edc7/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0) 2023-10-06T02:28:52.370 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2023-10-06T02:28:52.370 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2023-10-06T02:28:52.371 INFO:teuthology.orchestra.run.smithi031.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206072 s, 50.9 MB/s 2023-10-06T02:28:52.372 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:28:52.372 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000011953/5172ced3-24b3-4d86-a22c-dcfea079edc7/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0) 2023-10-06T02:28:52.428 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2023-10-06T02:28:52.428 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2023-10-06T02:28:52.428 INFO:teuthology.orchestra.run.smithi031.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192616 s, 54.4 MB/s 2023-10-06T02:28:52.429 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:28:52.429 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000011953/5172ced3-24b3-4d86-a22c-dcfea079edc7/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0) 2023-10-06T02:28:52.480 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2023-10-06T02:28:52.480 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2023-10-06T02:28:52.480 INFO:teuthology.orchestra.run.smithi031.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141276 s, 74.2 MB/s 2023-10-06T02:28:52.481 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:28:52.481 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000011953/5172ced3-24b3-4d86-a22c-dcfea079edc7/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0) 2023-10-06T02:28:52.534 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2023-10-06T02:28:52.534 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2023-10-06T02:28:52.534 INFO:teuthology.orchestra.run.smithi031.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145189 s, 72.2 MB/s 2023-10-06T02:28:52.535 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs subvolume_0000000000011953 snapshot_0000000000915642 2023-10-06T02:28:52.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.727+0000 7f7d3219d700 1 Processor -- start 2023-10-06T02:28:52.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.727+0000 7f7d3219d700 1 -- start start 2023-10-06T02:28:52.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.729+0000 7f7d3219d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7d2c0c85d0 0x7f7d2c0cc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:52.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.729+0000 7f7d3219d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d2c1571b0 con 0x7f7d2c0c8200 2023-10-06T02:28:52.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.729+0000 7f7d3219d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d2c1579b0 con 0x7f7d2c0c2a90 2023-10-06T02:28:52.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.729+0000 7f7d3219d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2c0c1390 con 0x7f7d2c0c85d0 2023-10-06T02:28:52.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.729+0000 7f7d3119b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f7d2c0c8200 0x7f7d2c0c1820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:39400/0 (socket says 172.21.15.31:39400) 2023-10-06T02:28:52.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.729+0000 7f7d3119b700 1 -- 172.21.15.31:0/317055328 learned_addr learned my addr 172.21.15.31:0/317055328 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:28:52.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.729+0000 7f7d3199c700 1 --2- 172.21.15.31:0/317055328 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7d2c0c85d0 0x7f7d2c0cc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:52.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.729+0000 7f7d23fff700 1 -- 172.21.15.31:0/317055328 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1102349834 0 0) 0x7f7d28001110 con 0x7f7d2c0c8200 2023-10-06T02:28:52.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.730+0000 7f7d23fff700 1 -- 172.21.15.31:0/317055328 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d18003570 con 0x7f7d2c0c8200 2023-10-06T02:28:52.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.730+0000 7f7d23fff700 1 -- 172.21.15.31:0/317055328 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3147266885 0 0) 0x7f7d2c1579b0 con 0x7f7d2c0c2a90 2023-10-06T02:28:52.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.730+0000 7f7d23fff700 1 -- 172.21.15.31:0/317055328 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d28001110 con 0x7f7d2c0c2a90 2023-10-06T02:28:52.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.730+0000 7f7d23fff700 1 -- 172.21.15.31:0/317055328 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2544215979 0 0) 0x7f7d18003570 con 0x7f7d2c0c8200 2023-10-06T02:28:52.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.730+0000 7f7d23fff700 1 -- 172.21.15.31:0/317055328 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7d2c0c85d0 msgr2=0x7f7d2c0cc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:52.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.730+0000 7f7d23fff700 1 --2- 172.21.15.31:0/317055328 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7d2c0c85d0 0x7f7d2c0cc2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:52.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.730+0000 7f7d23fff700 1 -- 172.21.15.31:0/317055328 >> v1:172.21.15.62:6789/0 conn(0x7f7d2c0c2a90 legacy=0x7f7d2c156aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:52.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.730+0000 7f7d23fff700 1 -- 172.21.15.31:0/317055328 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d2c1573f0 con 0x7f7d2c0c8200 2023-10-06T02:28:52.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.730+0000 7f7d23fff700 1 -- 172.21.15.31:0/317055328 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7f7d28001600 con 0x7f7d2c0c8200 2023-10-06T02:28:52.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.730+0000 7f7d23fff700 1 -- 172.21.15.31:0/317055328 >> v1:172.21.15.31:6789/0 conn(0x7f7d2c0c8200 legacy=0x7f7d2c0c1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:52.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.730+0000 7f7d23fff700 1 --2- 172.21.15.31:0/317055328 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d18003c20 0x7f7d18004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:52.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.730+0000 7f7d23fff700 1 --2- 172.21.15.31:0/317055328 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d18004810 0x7f7d18004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:52.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.730+0000 7f7d23fff700 1 --2- 172.21.15.31:0/317055328 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7d2c0c85d0 0x7f7d18005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:52.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.730+0000 7f7d23fff700 1 -- 172.21.15.31:0/317055328 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2c0c1390 con 0x7f7d18003c20 2023-10-06T02:28:52.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.730+0000 7f7d23fff700 1 -- 172.21.15.31:0/317055328 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1800bbf0 con 0x7f7d18004810 2023-10-06T02:28:52.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.730+0000 7f7d23fff700 1 -- 172.21.15.31:0/317055328 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1800bdb0 con 0x7f7d2c0c85d0 2023-10-06T02:28:52.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.730+0000 7f7d3119b700 1 --2- 172.21.15.31:0/317055328 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d18003c20 0x7f7d18004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:52.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.730+0000 7f7d3099a700 1 --2- 172.21.15.31:0/317055328 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d18004810 0x7f7d18004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:52.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.731+0000 7f7d3199c700 1 --2- 172.21.15.31:0/317055328 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7d2c0c85d0 0x7f7d18005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:52.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.731+0000 7f7d3119b700 1 -- 172.21.15.31:0/317055328 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7d2c0c85d0 msgr2=0x7f7d18005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:52.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.731+0000 7f7d3119b700 1 --2- 172.21.15.31:0/317055328 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7d2c0c85d0 0x7f7d18005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:52.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.731+0000 7f7d3119b700 1 -- 172.21.15.31:0/317055328 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d18004810 msgr2=0x7f7d18004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:52.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.731+0000 7f7d3119b700 1 --2- 172.21.15.31:0/317055328 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d18004810 0x7f7d18004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:52.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.731+0000 7f7d3119b700 1 -- 172.21.15.31:0/317055328 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d18003570 con 0x7f7d18003c20 2023-10-06T02:28:52.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.731+0000 7f7d3119b700 1 --2- 172.21.15.31:0/317055328 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d18003c20 0x7f7d18004010 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f7d2800c3c0 tx=0x7f7d2c0c14d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a310ea0fcaac4b6f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:52.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.731+0000 7f7d23fff700 1 -- 172.21.15.31:0/317055328 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d2800cc30 con 0x7f7d18003c20 2023-10-06T02:28:52.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.731+0000 7f7d23fff700 1 -- 172.21.15.31:0/317055328 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d2800cdb0 con 0x7f7d18003c20 2023-10-06T02:28:52.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.731+0000 7f7d23fff700 1 -- 172.21.15.31:0/317055328 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d28005b70 con 0x7f7d18003c20 2023-10-06T02:28:52.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.732+0000 7f7d3219d700 1 -- 172.21.15.31:0/317055328 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d18003c20 msgr2=0x7f7d18004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:52.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.732+0000 7f7d3219d700 1 --2- 172.21.15.31:0/317055328 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d18003c20 0x7f7d18004010 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f7d2800c3c0 tx=0x7f7d2c0c14d0 comp rx=0 tx=0).stop 2023-10-06T02:28:52.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.732+0000 7f7d3199c700 1 -- 172.21.15.31:0/317055328 reap_dead start 2023-10-06T02:28:52.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.732+0000 7f7d3219d700 1 -- 172.21.15.31:0/317055328 shutdown_connections 2023-10-06T02:28:52.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.732+0000 7f7d3219d700 1 -- 172.21.15.31:0/317055328 >> 172.21.15.31:0/317055328 conn(0x7f7d2c06c6a0 msgr2=0x7f7d2c06caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:52.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.732+0000 7f7d3219d700 1 -- 172.21.15.31:0/317055328 shutdown_connections 2023-10-06T02:28:52.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.732+0000 7f7d3219d700 1 -- 172.21.15.31:0/317055328 wait complete. 2023-10-06T02:28:52.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.733+0000 7f7d3219d700 1 Processor -- start 2023-10-06T02:28:52.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.734+0000 7f7d3219d700 1 -- start start 2023-10-06T02:28:52.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.734+0000 7f7d3219d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d18003c20 0x7f7d2c14eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:52.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.734+0000 7f7d3219d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7d2c15ab90 0x7f7d2c15b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:52.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.734+0000 7f7d3219d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d2c15e400 0x7f7d2c15e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:52.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.734+0000 7f7d3219d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2c16a9a0 con 0x7f7d2c15e400 2023-10-06T02:28:52.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.734+0000 7f7d3219d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2c15b540 con 0x7f7d18003c20 2023-10-06T02:28:52.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.734+0000 7f7d3219d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2c15edb0 con 0x7f7d2c15ab90 2023-10-06T02:28:52.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.735+0000 7f7d3199c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d2c15e400 0x7f7d2c15e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:52.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.735+0000 7f7d3199c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d2c15e400 0x7f7d2c15e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48904/0 (socket says 172.21.15.31:48904) 2023-10-06T02:28:52.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.735+0000 7f7d3199c700 1 -- 172.21.15.31:0/953961097 learned_addr learned my addr 172.21.15.31:0/953961097 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:52.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.735+0000 7f7d3119b700 1 --2- 172.21.15.31:0/953961097 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d18003c20 0x7f7d2c14eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:52.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.735+0000 7f7d3099a700 1 --2- 172.21.15.31:0/953961097 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7d2c15ab90 0x7f7d2c15b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:52.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.735+0000 7f7d3199c700 1 -- 172.21.15.31:0/953961097 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7d2c15ab90 msgr2=0x7f7d2c15b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:52.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.735+0000 7f7d3199c700 1 --2- 172.21.15.31:0/953961097 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7d2c15ab90 0x7f7d2c15b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:52.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.735+0000 7f7d3199c700 1 -- 172.21.15.31:0/953961097 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d18003c20 msgr2=0x7f7d2c14eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:52.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.735+0000 7f7d3199c700 1 --2- 172.21.15.31:0/953961097 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d18003c20 0x7f7d2c14eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:52.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.735+0000 7f7d3199c700 1 -- 172.21.15.31:0/953961097 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d2c0ab740 con 0x7f7d2c15e400 2023-10-06T02:28:52.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.735+0000 7f7d3199c700 1 --2- 172.21.15.31:0/953961097 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d2c15e400 0x7f7d2c15e870 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c0c2840 tx=0x7f7d2400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:52.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.736+0000 7f7d217fa700 1 -- 172.21.15.31:0/953961097 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d2400e070 con 0x7f7d2c15e400 2023-10-06T02:28:52.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.736+0000 7f7d217fa700 1 -- 172.21.15.31:0/953961097 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d24011c30 con 0x7f7d2c15e400 2023-10-06T02:28:52.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.736+0000 7f7d217fa700 1 -- 172.21.15.31:0/953961097 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d24016470 con 0x7f7d2c15e400 2023-10-06T02:28:52.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.736+0000 7f7d3219d700 1 -- 172.21.15.31:0/953961097 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d2c0ac2c0 con 0x7f7d2c15e400 2023-10-06T02:28:52.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.736+0000 7f7d3219d700 1 -- 172.21.15.31:0/953961097 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d2c0abc70 con 0x7f7d2c15e400 2023-10-06T02:28:52.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.737+0000 7f7d3219d700 1 -- 172.21.15.31:0/953961097 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d2c0ac450 con 0x7f7d2c15e400 2023-10-06T02:28:52.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.738+0000 7f7d217fa700 1 -- 172.21.15.31:0/953961097 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f7d240168a0 con 0x7f7d2c15e400 2023-10-06T02:28:52.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.739+0000 7f7d217fa700 1 --2- 172.21.15.31:0/953961097 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7d10067110 0x7f7d100695c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:52.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.739+0000 7f7d217fa700 1 -- 172.21.15.31:0/953961097 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f7d240857c0 con 0x7f7d2c15e400 2023-10-06T02:28:52.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.739+0000 7f7d3119b700 1 --2- 172.21.15.31:0/953961097 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7d10067110 0x7f7d100695c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:52.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.740+0000 7f7d3119b700 1 --2- 172.21.15.31:0/953961097 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7d10067110 0x7f7d100695c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7d28007d40 tx=0x7f7d28015000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:52.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.743+0000 7f7d217fa700 1 -- 172.21.15.31:0/953961097 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7d240524a0 con 0x7f7d2c15e400 2023-10-06T02:28:52.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:52.981+0000 7f7d3219d700 1 -- 172.21.15.31:0/953961097 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000011953", "snap_name": "snapshot_0000000000915642", "target": ["mon-mgr", ""]}) v1 -- 0x7f7d2c04c3c0 con 0x7f7d10067110 2023-10-06T02:28:53.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.005+0000 7f7d217fa700 1 -- 172.21.15.31:0/953961097 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f7d2c04c3c0 con 0x7f7d10067110 2023-10-06T02:28:53.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.007+0000 7f7d3219d700 1 -- 172.21.15.31:0/953961097 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7d10067110 msgr2=0x7f7d100695c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:53.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.007+0000 7f7d3219d700 1 --2- 172.21.15.31:0/953961097 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7d10067110 0x7f7d100695c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7d28007d40 tx=0x7f7d28015000 comp rx=0 tx=0).stop 2023-10-06T02:28:53.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.007+0000 7f7d3219d700 1 -- 172.21.15.31:0/953961097 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d2c15e400 msgr2=0x7f7d2c15e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:53.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.007+0000 7f7d3219d700 1 --2- 172.21.15.31:0/953961097 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d2c15e400 0x7f7d2c15e870 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c0c2840 tx=0x7f7d2400b040 comp rx=0 tx=0).stop 2023-10-06T02:28:53.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.007+0000 7f7d3219d700 1 -- 172.21.15.31:0/953961097 shutdown_connections 2023-10-06T02:28:53.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.007+0000 7f7d3219d700 1 --2- 172.21.15.31:0/953961097 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d2c15e400 0x7f7d2c15e870 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:53.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.007+0000 7f7d3219d700 1 --2- 172.21.15.31:0/953961097 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d18003c20 0x7f7d2c14eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:53.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.007+0000 7f7d3219d700 1 --2- 172.21.15.31:0/953961097 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7d10067110 0x7f7d100695c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:53.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.007+0000 7f7d3219d700 1 --2- 172.21.15.31:0/953961097 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7d2c15ab90 0x7f7d2c15b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:53.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.007+0000 7f7d3219d700 1 -- 172.21.15.31:0/953961097 >> 172.21.15.31:0/953961097 conn(0x7f7d2c06c6a0 msgr2=0x7f7d2c14fe90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:53.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.007+0000 7f7d3219d700 1 -- 172.21.15.31:0/953961097 shutdown_connections 2023-10-06T02:28:53.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.007+0000 7f7d3219d700 1 -- 172.21.15.31:0/953961097 wait complete. 2023-10-06T02:28:53.024 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs subvolume_0000000000011953 snapshot_0000000000915642 clone_0000000000433051 2023-10-06T02:28:53.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.217+0000 7ffa8f19d700 1 Processor -- start 2023-10-06T02:28:53.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.217+0000 7ffa8f19d700 1 -- start start 2023-10-06T02:28:53.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.219+0000 7ffa8f19d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ffa880af600 0x7ffa880c6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:53.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.219+0000 7ffa8f19d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa880c6d50 con 0x7ffa88140a70 2023-10-06T02:28:53.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.219+0000 7ffa8f19d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa880c7030 con 0x7ffa88140e40 2023-10-06T02:28:53.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.219+0000 7ffa8f19d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa880c7310 con 0x7ffa880af600 2023-10-06T02:28:53.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.219+0000 7ffa8e19b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7ffa88140a70 0x7ffa880c7df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:39412/0 (socket says 172.21.15.31:39412) 2023-10-06T02:28:53.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.219+0000 7ffa8e19b700 1 -- 172.21.15.31:0/4017586530 learned_addr learned my addr 172.21.15.31:0/4017586530 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:28:53.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.219+0000 7ffa8e99c700 1 --2- 172.21.15.31:0/4017586530 >> v2:172.21.15.62:3300/0 conn(0x7ffa880af600 0x7ffa880c6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:53.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.219+0000 7ffa8d199700 1 -- 172.21.15.31:0/4017586530 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2604329744 0 0) 0x7ffa880c6d50 con 0x7ffa88140a70 2023-10-06T02:28:53.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.219+0000 7ffa8d199700 1 -- 172.21.15.31:0/4017586530 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa74003570 con 0x7ffa88140a70 2023-10-06T02:28:53.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.220+0000 7ffa8d199700 1 -- 172.21.15.31:0/4017586530 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1615803294 0 0) 0x7ffa880c7030 con 0x7ffa88140e40 2023-10-06T02:28:53.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.220+0000 7ffa8d199700 1 -- 172.21.15.31:0/4017586530 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa880c6d50 con 0x7ffa88140e40 2023-10-06T02:28:53.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.220+0000 7ffa8d199700 1 -- 172.21.15.31:0/4017586530 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1759189380 0 0) 0x7ffa74003570 con 0x7ffa88140a70 2023-10-06T02:28:53.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.220+0000 7ffa8d199700 1 -- 172.21.15.31:0/4017586530 >> v2:172.21.15.62:3300/0 conn(0x7ffa880af600 msgr2=0x7ffa880c6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:53.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.220+0000 7ffa8d199700 1 --2- 172.21.15.31:0/4017586530 >> v2:172.21.15.62:3300/0 conn(0x7ffa880af600 0x7ffa880c6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:53.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.220+0000 7ffa8d199700 1 -- 172.21.15.31:0/4017586530 >> v1:172.21.15.62:6789/0 conn(0x7ffa88140e40 legacy=0x7ffa880aeef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:53.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.220+0000 7ffa8d199700 1 -- 172.21.15.31:0/4017586530 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa880c75f0 con 0x7ffa88140a70 2023-10-06T02:28:53.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.220+0000 7ffa8d199700 1 -- 172.21.15.31:0/4017586530 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7ffa84001470 con 0x7ffa88140a70 2023-10-06T02:28:53.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.220+0000 7ffa8e99c700 1 --2- 172.21.15.31:0/4017586530 >> v2:172.21.15.62:3300/0 conn(0x7ffa880af600 0x7ffa880c6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:28:53.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.220+0000 7ffa8d199700 1 -- 172.21.15.31:0/4017586530 >> v1:172.21.15.31:6789/0 conn(0x7ffa88140a70 legacy=0x7ffa880c7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:53.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.220+0000 7ffa8d199700 1 --2- 172.21.15.31:0/4017586530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa74003bf0 0x7ffa74003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:53.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.220+0000 7ffa8d199700 1 --2- 172.21.15.31:0/4017586530 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffa74004810 0x7ffa74004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:53.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.220+0000 7ffa8d199700 1 --2- 172.21.15.31:0/4017586530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa740051c0 0x7ffa7400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:53.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.220+0000 7ffa8d199700 1 -- 172.21.15.31:0/4017586530 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa880c7310 con 0x7ffa740051c0 2023-10-06T02:28:53.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.220+0000 7ffa8d199700 1 -- 172.21.15.31:0/4017586530 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa7400c0b0 con 0x7ffa74003bf0 2023-10-06T02:28:53.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.220+0000 7ffa8d199700 1 -- 172.21.15.31:0/4017586530 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa7400c270 con 0x7ffa74004810 2023-10-06T02:28:53.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.220+0000 7ffa8e99c700 1 --2- 172.21.15.31:0/4017586530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa740051c0 0x7ffa7400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:53.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.220+0000 7ffa8e19b700 1 --2- 172.21.15.31:0/4017586530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa74003bf0 0x7ffa74003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:53.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.220+0000 7ffa8d99a700 1 --2- 172.21.15.31:0/4017586530 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffa74004810 0x7ffa74004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:53.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.221+0000 7ffa8e99c700 1 -- 172.21.15.31:0/4017586530 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffa74004810 msgr2=0x7ffa74004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:53.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.221+0000 7ffa8e99c700 1 --2- 172.21.15.31:0/4017586530 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffa74004810 0x7ffa74004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:53.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.221+0000 7ffa8e99c700 1 -- 172.21.15.31:0/4017586530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa74003bf0 msgr2=0x7ffa74003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:53.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.221+0000 7ffa8e99c700 1 --2- 172.21.15.31:0/4017586530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa74003bf0 0x7ffa74003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:53.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.221+0000 7ffa8e99c700 1 -- 172.21.15.31:0/4017586530 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa74003570 con 0x7ffa740051c0 2023-10-06T02:28:53.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.221+0000 7ffa8e19b700 1 --2- 172.21.15.31:0/4017586530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa74003bf0 0x7ffa74003fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:28:53.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.221+0000 7ffa8e99c700 1 -- 172.21.15.31:0/4017586530 reap_dead start 2023-10-06T02:28:53.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.221+0000 7ffa8e99c700 1 --2- 172.21.15.31:0/4017586530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa740051c0 0x7ffa7400ba60 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7ffa8000e3c0 tx=0x7ffa8000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=348191e0fb1b0732 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:53.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.221+0000 7ffa8d199700 1 -- 172.21.15.31:0/4017586530 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa80009c50 con 0x7ffa740051c0 2023-10-06T02:28:53.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.221+0000 7ffa8d199700 1 -- 172.21.15.31:0/4017586530 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa8000f040 con 0x7ffa740051c0 2023-10-06T02:28:53.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.221+0000 7ffa8d199700 1 -- 172.21.15.31:0/4017586530 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa8000eec0 con 0x7ffa740051c0 2023-10-06T02:28:53.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.222+0000 7ffa8f19d700 1 -- 172.21.15.31:0/4017586530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa740051c0 msgr2=0x7ffa7400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:53.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.222+0000 7ffa8f19d700 1 --2- 172.21.15.31:0/4017586530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa740051c0 0x7ffa7400ba60 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7ffa8000e3c0 tx=0x7ffa8000b040 comp rx=0 tx=0).stop 2023-10-06T02:28:53.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.222+0000 7ffa8f19d700 1 -- 172.21.15.31:0/4017586530 shutdown_connections 2023-10-06T02:28:53.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.222+0000 7ffa8f19d700 1 --2- 172.21.15.31:0/4017586530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa740051c0 0x7ffa7400ba60 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:53.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.222+0000 7ffa8f19d700 1 -- 172.21.15.31:0/4017586530 >> 172.21.15.31:0/4017586530 conn(0x7ffa8806d170 msgr2=0x7ffa880cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:53.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.222+0000 7ffa8f19d700 1 -- 172.21.15.31:0/4017586530 shutdown_connections 2023-10-06T02:28:53.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.222+0000 7ffa8f19d700 1 -- 172.21.15.31:0/4017586530 wait complete. 2023-10-06T02:28:53.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.223+0000 7ffa8f19d700 1 Processor -- start 2023-10-06T02:28:53.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.223+0000 7ffa8f19d700 1 -- start start 2023-10-06T02:28:53.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.224+0000 7ffa8f19d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa740051c0 0x7ffa8813e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:53.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.224+0000 7ffa8f19d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffa8815b360 0x7ffa8815b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:53.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.224+0000 7ffa8f19d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa880c3d40 0x7ffa880c41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:53.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.224+0000 7ffa8f19d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa880ae8f0 con 0x7ffa880c3d40 2023-10-06T02:28:53.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.224+0000 7ffa8f19d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa8815bd10 con 0x7ffa740051c0 2023-10-06T02:28:53.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.224+0000 7ffa8f19d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa88169e20 con 0x7ffa8815b360 2023-10-06T02:28:53.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.225+0000 7ffa8e99c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa880c3d40 0x7ffa880c41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:53.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.225+0000 7ffa8e99c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa880c3d40 0x7ffa880c41b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48922/0 (socket says 172.21.15.31:48922) 2023-10-06T02:28:53.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.225+0000 7ffa8e99c700 1 -- 172.21.15.31:0/1982283017 learned_addr learned my addr 172.21.15.31:0/1982283017 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:53.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.225+0000 7ffa8e19b700 1 --2- 172.21.15.31:0/1982283017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa740051c0 0x7ffa8813e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:53.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.225+0000 7ffa8d99a700 1 --2- 172.21.15.31:0/1982283017 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffa8815b360 0x7ffa8815b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:53.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.225+0000 7ffa8e99c700 1 -- 172.21.15.31:0/1982283017 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffa8815b360 msgr2=0x7ffa8815b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:53.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.225+0000 7ffa8e99c700 1 --2- 172.21.15.31:0/1982283017 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffa8815b360 0x7ffa8815b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:53.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.225+0000 7ffa8e99c700 1 -- 172.21.15.31:0/1982283017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa740051c0 msgr2=0x7ffa8813e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:53.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.225+0000 7ffa8e99c700 1 --2- 172.21.15.31:0/1982283017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa740051c0 0x7ffa8813e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:53.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.225+0000 7ffa8e99c700 1 -- 172.21.15.31:0/1982283017 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa880c4a70 con 0x7ffa880c3d40 2023-10-06T02:28:53.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.225+0000 7ffa8d99a700 1 --2- 172.21.15.31:0/1982283017 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffa8815b360 0x7ffa8815b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:28:53.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.225+0000 7ffa8e99c700 1 --2- 172.21.15.31:0/1982283017 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa880c3d40 0x7ffa880c41b0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7ffa800099f0 tx=0x7ffa8001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:53.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.226+0000 7ffa7e7fc700 1 -- 172.21.15.31:0/1982283017 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa8001d660 con 0x7ffa880c3d40 2023-10-06T02:28:53.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.226+0000 7ffa7e7fc700 1 -- 172.21.15.31:0/1982283017 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa8001d7e0 con 0x7ffa880c3d40 2023-10-06T02:28:53.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.226+0000 7ffa8f19d700 1 -- 172.21.15.31:0/1982283017 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa880c4d50 con 0x7ffa880c3d40 2023-10-06T02:28:53.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.226+0000 7ffa7e7fc700 1 -- 172.21.15.31:0/1982283017 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa8001dba0 con 0x7ffa880c3d40 2023-10-06T02:28:53.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.226+0000 7ffa8f19d700 1 -- 172.21.15.31:0/1982283017 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa880c5310 con 0x7ffa880c3d40 2023-10-06T02:28:53.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.227+0000 7ffa8f19d700 1 -- 172.21.15.31:0/1982283017 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa88169fb0 con 0x7ffa880c3d40 2023-10-06T02:28:53.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.228+0000 7ffa7e7fc700 1 -- 172.21.15.31:0/1982283017 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) v1 ==== 84893+0+0 (secure 0 0 0) 0x7ffa80003020 con 0x7ffa880c3d40 2023-10-06T02:28:53.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.228+0000 7ffa7e7fc700 1 --2- 172.21.15.31:0/1982283017 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ffa6c0670e0 0x7ffa6c069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:53.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.229+0000 7ffa7e7fc700 1 -- 172.21.15.31:0/1982283017 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ffa80086b90 con 0x7ffa880c3d40 2023-10-06T02:28:53.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.229+0000 7ffa8e19b700 1 --2- 172.21.15.31:0/1982283017 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ffa6c0670e0 0x7ffa6c069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:53.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.230+0000 7ffa8e19b700 1 --2- 172.21.15.31:0/1982283017 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ffa6c0670e0 0x7ffa6c069590 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ffa84005c90 tx=0x7ffa84007040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:53.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.233+0000 7ffa7e7fc700 1 -- 172.21.15.31:0/1982283017 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffa80053870 con 0x7ffa880c3d40 2023-10-06T02:28:53.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.480+0000 7ffa8f19d700 1 -- 172.21.15.31:0/1982283017 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "subvolume_0000000000011953", "snap_name": "snapshot_0000000000915642", "target_sub_name": "clone_0000000000433051", "target": ["mon-mgr", ""]}) v1 -- 0x7ffa88169c20 con 0x7ffa6c0670e0 2023-10-06T02:28:53.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.555+0000 7ffa7e7fc700 1 -- 172.21.15.31:0/1982283017 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7ffa88169c20 con 0x7ffa6c0670e0 2023-10-06T02:28:53.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.557+0000 7ffa8f19d700 1 -- 172.21.15.31:0/1982283017 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ffa6c0670e0 msgr2=0x7ffa6c069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:53.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.557+0000 7ffa8f19d700 1 --2- 172.21.15.31:0/1982283017 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ffa6c0670e0 0x7ffa6c069590 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ffa84005c90 tx=0x7ffa84007040 comp rx=0 tx=0).stop 2023-10-06T02:28:53.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.557+0000 7ffa8f19d700 1 -- 172.21.15.31:0/1982283017 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa880c3d40 msgr2=0x7ffa880c41b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:53.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.557+0000 7ffa8f19d700 1 --2- 172.21.15.31:0/1982283017 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa880c3d40 0x7ffa880c41b0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7ffa800099f0 tx=0x7ffa8001a000 comp rx=0 tx=0).stop 2023-10-06T02:28:53.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.557+0000 7ffa8f19d700 1 -- 172.21.15.31:0/1982283017 shutdown_connections 2023-10-06T02:28:53.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.557+0000 7ffa8f19d700 1 --2- 172.21.15.31:0/1982283017 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa880c3d40 0x7ffa880c41b0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:53.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.557+0000 7ffa8f19d700 1 --2- 172.21.15.31:0/1982283017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa740051c0 0x7ffa8813e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:53.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.557+0000 7ffa8f19d700 1 --2- 172.21.15.31:0/1982283017 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ffa6c0670e0 0x7ffa6c069590 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:53.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.557+0000 7ffa8f19d700 1 --2- 172.21.15.31:0/1982283017 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffa8815b360 0x7ffa8815b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:53.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.557+0000 7ffa8f19d700 1 -- 172.21.15.31:0/1982283017 >> 172.21.15.31:0/1982283017 conn(0x7ffa8806d170 msgr2=0x7ffa880638f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:53.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.558+0000 7ffa8f19d700 1 -- 172.21.15.31:0/1982283017 shutdown_connections 2023-10-06T02:28:53.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.558+0000 7ffa8f19d700 1 -- 172.21.15.31:0/1982283017 wait complete. 2023-10-06T02:28:53.561 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:28:53.560+0000 7f656b389700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:28:53.561 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:28:53.560+0000 7f656b389700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:28:53.561 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:28:53.560+0000 7f656b389700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:28:53.561 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:28:53.560+0000 7f656b389700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:28:53.561 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:28:53.560+0000 7f656b389700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:28:53.578 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs clone_0000000000433051 2023-10-06T02:28:53.581 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:28:53.580+0000 7f656ab88700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:28:53.581 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:28:53.580+0000 7f656ab88700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:28:53.581 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:28:53.580+0000 7f656ab88700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:28:53.582 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:28:53.580+0000 7f656ab88700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:28:53.582 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:28:53.580+0000 7f656ab88700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:28:53.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.788+0000 7faa4a59d700 1 Processor -- start 2023-10-06T02:28:53.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.788+0000 7faa4a59d700 1 -- start start 2023-10-06T02:28:53.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.789+0000 7faa4a59d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7faa44153a00 0x7faa44153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:53.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.789+0000 7faa4a59d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7faa4415de50 0x7faa4415a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:53.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.789+0000 7faa4a59d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa4415af60 con 0x7faa440cea00 2023-10-06T02:28:53.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.789+0000 7faa4a59d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa4415d010 con 0x7faa44153a00 2023-10-06T02:28:53.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.789+0000 7faa4a59d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa4415b240 con 0x7faa4415de50 2023-10-06T02:28:53.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.790+0000 7faa4959b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7faa44153a00 0x7faa44153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:53.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.790+0000 7faa4959b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7faa44153a00 0x7faa44153dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48934/0 (socket says 172.21.15.31:48934) 2023-10-06T02:28:53.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.790+0000 7faa4959b700 1 -- 172.21.15.31:0/907347636 learned_addr learned my addr 172.21.15.31:0/907347636 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:53.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.790+0000 7faa48d9a700 1 --2- 172.21.15.31:0/907347636 >> v2:172.21.15.62:3300/0 conn(0x7faa4415de50 0x7faa4415a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:53.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.790+0000 7faa4959b700 1 -- 172.21.15.31:0/907347636 >> v2:172.21.15.62:3300/0 conn(0x7faa4415de50 msgr2=0x7faa4415a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:53.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.790+0000 7faa4959b700 1 --2- 172.21.15.31:0/907347636 >> v2:172.21.15.62:3300/0 conn(0x7faa4415de50 0x7faa4415a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:53.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.790+0000 7faa4959b700 1 -- 172.21.15.31:0/907347636 >> v1:172.21.15.62:6789/0 conn(0x7faa440cea00 legacy=0x7faa440cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:53.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.790+0000 7faa4959b700 1 -- 172.21.15.31:0/907347636 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa4415b520 con 0x7faa44153a00 2023-10-06T02:28:53.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.790+0000 7faa4959b700 1 --2- 172.21.15.31:0/907347636 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa44153a00 0x7faa44153dd0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7faa440b3070 tx=0x7faa40009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=221ef219b2d5ae2b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:53.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.791+0000 7faa3bfff700 1 -- 172.21.15.31:0/907347636 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa40009e70 con 0x7faa44153a00 2023-10-06T02:28:53.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.791+0000 7faa3bfff700 1 -- 172.21.15.31:0/907347636 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa4000b860 con 0x7faa44153a00 2023-10-06T02:28:53.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.791+0000 7faa3bfff700 1 -- 172.21.15.31:0/907347636 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa4000bc00 con 0x7faa44153a00 2023-10-06T02:28:53.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.791+0000 7faa4a59d700 1 -- 172.21.15.31:0/907347636 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa44153a00 msgr2=0x7faa44153dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:53.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.791+0000 7faa4a59d700 1 --2- 172.21.15.31:0/907347636 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa44153a00 0x7faa44153dd0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7faa440b3070 tx=0x7faa40009250 comp rx=0 tx=0).stop 2023-10-06T02:28:53.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.791+0000 7faa4a59d700 1 -- 172.21.15.31:0/907347636 shutdown_connections 2023-10-06T02:28:53.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.791+0000 7faa4a59d700 1 --2- 172.21.15.31:0/907347636 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa44153a00 0x7faa44153dd0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:53.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.791+0000 7faa4a59d700 1 --2- 172.21.15.31:0/907347636 >> v2:172.21.15.62:3300/0 conn(0x7faa4415de50 0x7faa4415a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:53.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.791+0000 7faa4a59d700 1 -- 172.21.15.31:0/907347636 >> 172.21.15.31:0/907347636 conn(0x7faa440703a0 msgr2=0x7faa440707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:53.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.791+0000 7faa4a59d700 1 -- 172.21.15.31:0/907347636 shutdown_connections 2023-10-06T02:28:53.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.791+0000 7faa4a59d700 1 -- 172.21.15.31:0/907347636 wait complete. 2023-10-06T02:28:53.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.793+0000 7faa4a59d700 1 Processor -- start 2023-10-06T02:28:53.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.793+0000 7faa4a59d700 1 -- start start 2023-10-06T02:28:53.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.794+0000 7faa4a59d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa440cea00 0x7faa4416de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:53.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.794+0000 7faa4a59d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faa44153a00 0x7faa4414a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:53.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.794+0000 7faa4a59d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa4415de50 0x7faa44151850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:53.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.794+0000 7faa4a59d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa4414b500 con 0x7faa440cea00 2023-10-06T02:28:53.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.794+0000 7faa4a59d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa4416a8b0 con 0x7faa4415de50 2023-10-06T02:28:53.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.794+0000 7faa4a59d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa4416ab90 con 0x7faa44153a00 2023-10-06T02:28:53.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.794+0000 7faa4959b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa440cea00 0x7faa4416de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:53.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.794+0000 7faa4959b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa440cea00 0x7faa4416de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48938/0 (socket says 172.21.15.31:48938) 2023-10-06T02:28:53.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.794+0000 7faa4959b700 1 -- 172.21.15.31:0/2714864339 learned_addr learned my addr 172.21.15.31:0/2714864339 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:53.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.794+0000 7faa48d9a700 1 --2- 172.21.15.31:0/2714864339 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faa44153a00 0x7faa4414a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:53.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.794+0000 7faa49d9c700 1 --2- 172.21.15.31:0/2714864339 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa4415de50 0x7faa44151850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:53.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.795+0000 7faa4959b700 1 -- 172.21.15.31:0/2714864339 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faa44153a00 msgr2=0x7faa4414a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:53.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.795+0000 7faa4959b700 1 --2- 172.21.15.31:0/2714864339 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faa44153a00 0x7faa4414a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:53.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.795+0000 7faa4959b700 1 -- 172.21.15.31:0/2714864339 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa4415de50 msgr2=0x7faa44151850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:53.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.795+0000 7faa4959b700 1 --2- 172.21.15.31:0/2714864339 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa4415de50 0x7faa44151850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:53.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.795+0000 7faa4959b700 1 -- 172.21.15.31:0/2714864339 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa400094f0 con 0x7faa440cea00 2023-10-06T02:28:53.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.795+0000 7faa4959b700 1 --2- 172.21.15.31:0/2714864339 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa440cea00 0x7faa4416de20 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7faa4406eb00 tx=0x7faa40017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:53.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.795+0000 7faa397fa700 1 -- 172.21.15.31:0/2714864339 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa4000bbe0 con 0x7faa440cea00 2023-10-06T02:28:53.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.795+0000 7faa397fa700 1 -- 172.21.15.31:0/2714864339 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa40011420 con 0x7faa440cea00 2023-10-06T02:28:53.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.795+0000 7faa397fa700 1 -- 172.21.15.31:0/2714864339 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa400117e0 con 0x7faa440cea00 2023-10-06T02:28:53.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.795+0000 7faa4a59d700 1 -- 172.21.15.31:0/2714864339 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa4416d340 con 0x7faa440cea00 2023-10-06T02:28:53.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.796+0000 7faa4a59d700 1 -- 172.21.15.31:0/2714864339 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa4416d900 con 0x7faa440cea00 2023-10-06T02:28:53.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.796+0000 7faa4a59d700 1 -- 172.21.15.31:0/2714864339 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa4416db20 con 0x7faa440cea00 2023-10-06T02:28:53.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.797+0000 7faa397fa700 1 -- 172.21.15.31:0/2714864339 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) v1 ==== 84893+0+0 (secure 0 0 0) 0x7faa40011c10 con 0x7faa440cea00 2023-10-06T02:28:53.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.799+0000 7faa397fa700 1 --2- 172.21.15.31:0/2714864339 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7faa280670e0 0x7faa28069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:53.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.799+0000 7faa397fa700 1 -- 172.21.15.31:0/2714864339 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6058+0+0 (secure 0 0 0) 0x7faa40086270 con 0x7faa440cea00 2023-10-06T02:28:53.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.799+0000 7faa48d9a700 1 --2- 172.21.15.31:0/2714864339 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7faa280670e0 0x7faa28069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:53.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.800+0000 7faa48d9a700 1 --2- 172.21.15.31:0/2714864339 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7faa280670e0 0x7faa28069590 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7faa4406d760 tx=0x7faa340091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:53.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:53.803+0000 7faa397fa700 1 -- 172.21.15.31:0/2714864339 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faa40053060 con 0x7faa440cea00 2023-10-06T02:28:54.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:54.059+0000 7faa4a59d700 1 -- 172.21.15.31:0/2714864339 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "clone_0000000000433051", "target": ["mon-mgr", ""]}) v1 -- 0x7faa4404c3c0 con 0x7faa280670e0 2023-10-06T02:28:54.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:54.516+0000 7faa397fa700 1 -- 172.21.15.31:0/2714864339 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mgrmap(e 19) v1 ==== 84995+0+0 (secure 0 0 0) 0x7faa40053430 con 0x7faa440cea00 2023-10-06T02:28:54.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:54.880+0000 7faa397fa700 1 -- 172.21.15.31:0/2714864339 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+46 (secure 0 0 0) 0x7faa4404c3c0 con 0x7faa280670e0 2023-10-06T02:28:54.881 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T02:28:54.881 INFO:teuthology.orchestra.run.smithi031.stdout: "status": { 2023-10-06T02:28:54.881 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "complete" 2023-10-06T02:28:54.881 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:28:54.881 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T02:28:54.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:54.883+0000 7faa4a59d700 1 -- 172.21.15.31:0/2714864339 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7faa280670e0 msgr2=0x7faa28069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:54.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:54.883+0000 7faa4a59d700 1 --2- 172.21.15.31:0/2714864339 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7faa280670e0 0x7faa28069590 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7faa4406d760 tx=0x7faa340091a0 comp rx=0 tx=0).stop 2023-10-06T02:28:54.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:54.883+0000 7faa4a59d700 1 -- 172.21.15.31:0/2714864339 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa440cea00 msgr2=0x7faa4416de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:54.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:54.883+0000 7faa4a59d700 1 --2- 172.21.15.31:0/2714864339 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa440cea00 0x7faa4416de20 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7faa4406eb00 tx=0x7faa40017000 comp rx=0 tx=0).stop 2023-10-06T02:28:54.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:54.883+0000 7faa4a59d700 1 -- 172.21.15.31:0/2714864339 shutdown_connections 2023-10-06T02:28:54.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:54.883+0000 7faa4a59d700 1 --2- 172.21.15.31:0/2714864339 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa4415de50 0x7faa44151850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:54.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:54.883+0000 7faa4a59d700 1 --2- 172.21.15.31:0/2714864339 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa440cea00 0x7faa4416de20 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:54.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:54.883+0000 7faa4a59d700 1 --2- 172.21.15.31:0/2714864339 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7faa280670e0 0x7faa28069590 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:54.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:54.883+0000 7faa4a59d700 1 --2- 172.21.15.31:0/2714864339 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faa44153a00 0x7faa4414a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:54.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:54.883+0000 7faa4a59d700 1 -- 172.21.15.31:0/2714864339 >> 172.21.15.31:0/2714864339 conn(0x7faa440703a0 msgr2=0x7faa440d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:54.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:54.884+0000 7faa4a59d700 1 -- 172.21.15.31:0/2714864339 shutdown_connections 2023-10-06T02:28:54.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:54.884+0000 7faa4a59d700 1 -- 172.21.15.31:0/2714864339 wait complete. 2023-10-06T02:28:54.906 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:28:54.906 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/clone_0000000000433051/.meta) 2023-10-06T02:28:54.973 INFO:teuthology.orchestra.run.smithi031.stdout:[GLOBAL] 2023-10-06T02:28:54.973 INFO:teuthology.orchestra.run.smithi031.stdout:version = 2 2023-10-06T02:28:54.973 INFO:teuthology.orchestra.run.smithi031.stdout:type = clone 2023-10-06T02:28:54.973 INFO:teuthology.orchestra.run.smithi031.stdout:path = /volumes/_nogroup/clone_0000000000433051/6d3bb52c-1833-4d2b-b295-0332ce04e82e 2023-10-06T02:28:54.974 INFO:teuthology.orchestra.run.smithi031.stdout:state = complete 2023-10-06T02:28:54.974 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:28:54.976 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000011953 2023-10-06T02:28:55.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.168+0000 7f17d919d700 1 Processor -- start 2023-10-06T02:28:55.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.168+0000 7f17d919d700 1 -- start start 2023-10-06T02:28:55.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.169+0000 7f17d919d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f17d4153dd0 0x7f17d415de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:55.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.169+0000 7f17d919d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17d415ac50 con 0x7f17d4153a00 2023-10-06T02:28:55.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.169+0000 7f17d919d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17d415d010 con 0x7f17d40cea00 2023-10-06T02:28:55.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.169+0000 7f17d919d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17d415af60 con 0x7f17d4153dd0 2023-10-06T02:28:55.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.170+0000 7f17d3fff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f17d4153a00 0x7f17d415a540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:39420/0 (socket says 172.21.15.31:39420) 2023-10-06T02:28:55.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.170+0000 7f17d3fff700 1 -- 172.21.15.31:0/3823438430 learned_addr learned my addr 172.21.15.31:0/3823438430 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:28:55.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.170+0000 7f17d37fe700 1 --2- 172.21.15.31:0/3823438430 >> v2:172.21.15.31:3300/0 conn(0x7f17d4153dd0 0x7f17d415de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:55.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.170+0000 7f17d37fe700 1 -- 172.21.15.31:0/3823438430 >> v1:172.21.15.62:6789/0 conn(0x7f17d40cea00 legacy=0x7f17d40ceea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:55.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.170+0000 7f17d37fe700 1 -- 172.21.15.31:0/3823438430 >> v1:172.21.15.31:6789/0 conn(0x7f17d4153a00 legacy=0x7f17d415a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:55.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.170+0000 7f17d37fe700 1 -- 172.21.15.31:0/3823438430 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17d415b240 con 0x7f17d4153dd0 2023-10-06T02:28:55.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.170+0000 7f17d2ffd700 1 -- 172.21.15.31:0/3823438430 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 284866791 0 0) 0x7f17d415ac50 con 0x7f17d4153a00 2023-10-06T02:28:55.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.170+0000 7f17d37fe700 1 --2- 172.21.15.31:0/3823438430 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17d4153dd0 0x7f17d415de50 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f17c80098e0 tx=0x7f17c80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=625fbf556e977ab8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:55.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.171+0000 7f17d2ffd700 1 -- 172.21.15.31:0/3823438430 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c8009e80 con 0x7f17d4153dd0 2023-10-06T02:28:55.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.171+0000 7f17d2ffd700 1 -- 172.21.15.31:0/3823438430 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17c800b820 con 0x7f17d4153dd0 2023-10-06T02:28:55.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.171+0000 7f17d2ffd700 1 -- 172.21.15.31:0/3823438430 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c800bba0 con 0x7f17d4153dd0 2023-10-06T02:28:55.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.171+0000 7f17d919d700 1 -- 172.21.15.31:0/3823438430 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17d4153dd0 msgr2=0x7f17d415de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:55.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.171+0000 7f17d919d700 1 --2- 172.21.15.31:0/3823438430 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17d4153dd0 0x7f17d415de50 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f17c80098e0 tx=0x7f17c80091a0 comp rx=0 tx=0).stop 2023-10-06T02:28:55.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.171+0000 7f17d919d700 1 -- 172.21.15.31:0/3823438430 shutdown_connections 2023-10-06T02:28:55.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.171+0000 7f17d919d700 1 --2- 172.21.15.31:0/3823438430 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17d4153dd0 0x7f17d415de50 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:55.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.171+0000 7f17d919d700 1 -- 172.21.15.31:0/3823438430 >> 172.21.15.31:0/3823438430 conn(0x7f17d40703a0 msgr2=0x7f17d40707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:55.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.171+0000 7f17d919d700 1 -- 172.21.15.31:0/3823438430 shutdown_connections 2023-10-06T02:28:55.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.171+0000 7f17d919d700 1 -- 172.21.15.31:0/3823438430 wait complete. 2023-10-06T02:28:55.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.173+0000 7f17d919d700 1 Processor -- start 2023-10-06T02:28:55.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.173+0000 7f17d919d700 1 -- start start 2023-10-06T02:28:55.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.174+0000 7f17d919d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17d40cea00 0x7f17d4145e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:55.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.174+0000 7f17d919d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17d4153a00 0x7f17d41588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:55.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.174+0000 7f17d919d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f17d4153dd0 0x7f17d4158e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:55.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.174+0000 7f17d919d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17d40c8290 con 0x7f17d4153a00 2023-10-06T02:28:55.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.174+0000 7f17d919d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17d40c7680 con 0x7f17d40cea00 2023-10-06T02:28:55.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.174+0000 7f17d919d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17d40c7960 con 0x7f17d4153dd0 2023-10-06T02:28:55.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.174+0000 7f17d37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17d4153a00 0x7f17d41588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:55.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.174+0000 7f17d37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17d4153a00 0x7f17d41588c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48962/0 (socket says 172.21.15.31:48962) 2023-10-06T02:28:55.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.174+0000 7f17d37fe700 1 -- 172.21.15.31:0/3277502494 learned_addr learned my addr 172.21.15.31:0/3277502494 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:55.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.174+0000 7f17d3fff700 1 --2- 172.21.15.31:0/3277502494 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17d40cea00 0x7f17d4145e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:55.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.175+0000 7f17d37fe700 1 -- 172.21.15.31:0/3277502494 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f17d4153dd0 msgr2=0x7f17d4158e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:55.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.175+0000 7f17d899c700 1 --2- 172.21.15.31:0/3277502494 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f17d4153dd0 0x7f17d4158e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:55.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.175+0000 7f17d37fe700 1 --2- 172.21.15.31:0/3277502494 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f17d4153dd0 0x7f17d4158e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:55.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.175+0000 7f17d37fe700 1 -- 172.21.15.31:0/3277502494 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17d40cea00 msgr2=0x7f17d4145e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:55.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.175+0000 7f17d37fe700 1 --2- 172.21.15.31:0/3277502494 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17d40cea00 0x7f17d4145e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:55.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.175+0000 7f17d37fe700 1 -- 172.21.15.31:0/3277502494 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17c8009420 con 0x7f17d4153a00 2023-10-06T02:28:55.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.175+0000 7f17d3fff700 1 --2- 172.21.15.31:0/3277502494 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17d40cea00 0x7f17d4145e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:28:55.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.175+0000 7f17d899c700 1 --2- 172.21.15.31:0/3277502494 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f17d4153dd0 0x7f17d4158e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:28:55.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.175+0000 7f17d37fe700 1 --2- 172.21.15.31:0/3277502494 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17d4153a00 0x7f17d41588c0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f17c80030c0 tx=0x7f17c8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:55.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.175+0000 7f17bbfff700 1 -- 172.21.15.31:0/3277502494 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c800bae0 con 0x7f17d4153a00 2023-10-06T02:28:55.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.175+0000 7f17bbfff700 1 -- 172.21.15.31:0/3277502494 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17c8020070 con 0x7f17d4153a00 2023-10-06T02:28:55.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.175+0000 7f17bbfff700 1 -- 172.21.15.31:0/3277502494 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c801b6a0 con 0x7f17d4153a00 2023-10-06T02:28:55.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.175+0000 7f17d919d700 1 -- 172.21.15.31:0/3277502494 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17d40c7c40 con 0x7f17d4153a00 2023-10-06T02:28:55.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.175+0000 7f17d919d700 1 -- 172.21.15.31:0/3277502494 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17d41463d0 con 0x7f17d4153a00 2023-10-06T02:28:55.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.176+0000 7f17d919d700 1 -- 172.21.15.31:0/3277502494 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17d40c8400 con 0x7f17d4153a00 2023-10-06T02:28:55.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.177+0000 7f17bbfff700 1 -- 172.21.15.31:0/3277502494 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f17c8022030 con 0x7f17d4153a00 2023-10-06T02:28:55.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.178+0000 7f17bbfff700 1 --2- 172.21.15.31:0/3277502494 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f17c0067200 0x7f17c00696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:55.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.179+0000 7f17bbfff700 1 -- 172.21.15.31:0/3277502494 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f17c8086b50 con 0x7f17d4153a00 2023-10-06T02:28:55.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.179+0000 7f17d3fff700 1 --2- 172.21.15.31:0/3277502494 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f17c0067200 0x7f17c00696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:55.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.180+0000 7f17d3fff700 1 --2- 172.21.15.31:0/3277502494 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f17c0067200 0x7f17c00696b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f17c40018d0 tx=0x7f17c4009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:55.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.182+0000 7f17bbfff700 1 -- 172.21.15.31:0/3277502494 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f17c8053830 con 0x7f17d4153a00 2023-10-06T02:28:55.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.421+0000 7f17d919d700 1 -- 172.21.15.31:0/3277502494 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000011953", "target": ["mon-mgr", ""]}) v1 -- 0x7f17d40c83d0 con 0x7f17c0067200 2023-10-06T02:28:55.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.441+0000 7f17bbfff700 1 -- 172.21.15.31:0/3277502494 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+82 (secure 0 0 0) 0x7f17d40c83d0 con 0x7f17c0067200 2023-10-06T02:28:55.442 INFO:teuthology.orchestra.run.smithi031.stdout:/volumes/_nogroup/subvolume_0000000000011953/5172ced3-24b3-4d86-a22c-dcfea079edc7 2023-10-06T02:28:55.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.444+0000 7f17d919d700 1 -- 172.21.15.31:0/3277502494 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f17c0067200 msgr2=0x7f17c00696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:55.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.444+0000 7f17d919d700 1 --2- 172.21.15.31:0/3277502494 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f17c0067200 0x7f17c00696b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f17c40018d0 tx=0x7f17c4009040 comp rx=0 tx=0).stop 2023-10-06T02:28:55.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.444+0000 7f17d919d700 1 -- 172.21.15.31:0/3277502494 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17d4153a00 msgr2=0x7f17d41588c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:55.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.444+0000 7f17d919d700 1 --2- 172.21.15.31:0/3277502494 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17d4153a00 0x7f17d41588c0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f17c80030c0 tx=0x7f17c8016000 comp rx=0 tx=0).stop 2023-10-06T02:28:55.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.444+0000 7f17d919d700 1 -- 172.21.15.31:0/3277502494 shutdown_connections 2023-10-06T02:28:55.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.444+0000 7f17d919d700 1 --2- 172.21.15.31:0/3277502494 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f17c0067200 0x7f17c00696b0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:55.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.444+0000 7f17d919d700 1 --2- 172.21.15.31:0/3277502494 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f17d4153dd0 0x7f17d4158e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:55.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.444+0000 7f17d919d700 1 --2- 172.21.15.31:0/3277502494 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17d40cea00 0x7f17d4145e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:55.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.444+0000 7f17d919d700 1 --2- 172.21.15.31:0/3277502494 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17d4153a00 0x7f17d41588c0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:55.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.444+0000 7f17d919d700 1 -- 172.21.15.31:0/3277502494 >> 172.21.15.31:0/3277502494 conn(0x7f17d40703a0 msgr2=0x7f17d4152150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:55.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.444+0000 7f17d919d700 1 -- 172.21.15.31:0/3277502494 shutdown_connections 2023-10-06T02:28:55.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.444+0000 7f17d919d700 1 -- 172.21.15.31:0/3277502494 wait complete. 2023-10-06T02:28:55.462 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs clone_0000000000433051 2023-10-06T02:28:55.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.654+0000 7f0bf9b9d700 1 Processor -- start 2023-10-06T02:28:55.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.654+0000 7f0bf9b9d700 1 -- start start 2023-10-06T02:28:55.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.655+0000 7f0bf9b9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0bf4153c90 0x7f0bf415a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:55.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.655+0000 7f0bf9b9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0bf406db20 0x7f0bf40ca7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:55.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.655+0000 7f0bf9b9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0bf4154060 0x7f0bf40cacf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:55.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.655+0000 7f0bf9b9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bf415ada0 con 0x7f0bf4154060 2023-10-06T02:28:55.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.655+0000 7f0bf9b9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bf406ca80 con 0x7f0bf4153c90 2023-10-06T02:28:55.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.655+0000 7f0bf9b9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf406cd60 con 0x7f0bf406db20 2023-10-06T02:28:55.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.655+0000 7f0bf939c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0bf4154060 0x7f0bf40cacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:55.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.656+0000 7f0bf939c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0bf4154060 0x7f0bf40cacf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48970/0 (socket says 172.21.15.31:48970) 2023-10-06T02:28:55.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.656+0000 7f0bf939c700 1 -- 172.21.15.31:0/1223043407 learned_addr learned my addr 172.21.15.31:0/1223043407 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:55.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.656+0000 7f0bf8b9b700 1 --2- 172.21.15.31:0/1223043407 >> v2:172.21.15.62:3300/0 conn(0x7f0bf4153c90 0x7f0bf415a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:55.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.656+0000 7f0bf3fff700 1 --2- 172.21.15.31:0/1223043407 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0bf406db20 0x7f0bf40ca7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:55.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.656+0000 7f0bf939c700 1 -- 172.21.15.31:0/1223043407 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0bf406db20 msgr2=0x7f0bf40ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:55.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.656+0000 7f0bf939c700 1 --2- 172.21.15.31:0/1223043407 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0bf406db20 0x7f0bf40ca7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:55.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.656+0000 7f0bf939c700 1 -- 172.21.15.31:0/1223043407 >> v2:172.21.15.62:3300/0 conn(0x7f0bf4153c90 msgr2=0x7f0bf415a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:28:55.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.656+0000 7f0bf939c700 1 --2- 172.21.15.31:0/1223043407 >> v2:172.21.15.62:3300/0 conn(0x7f0bf4153c90 0x7f0bf415a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:55.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.656+0000 7f0bf939c700 1 -- 172.21.15.31:0/1223043407 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bf406d040 con 0x7f0bf4154060 2023-10-06T02:28:55.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.656+0000 7f0bf939c700 1 --2- 172.21.15.31:0/1223043407 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bf4154060 0x7f0bf40cacf0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f0bf40c37f0 tx=0x7f0bec009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17ab2ee2f07a108c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:55.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.656+0000 7f0bf37fe700 1 -- 172.21.15.31:0/1223043407 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bec00e070 con 0x7f0bf4154060 2023-10-06T02:28:55.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.657+0000 7f0bf37fe700 1 -- 172.21.15.31:0/1223043407 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bec00b970 con 0x7f0bf4154060 2023-10-06T02:28:55.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.657+0000 7f0bf37fe700 1 -- 172.21.15.31:0/1223043407 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bec0092e0 con 0x7f0bf4154060 2023-10-06T02:28:55.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.657+0000 7f0bf9b9d700 1 -- 172.21.15.31:0/1223043407 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bf4154060 msgr2=0x7f0bf40cacf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:55.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.657+0000 7f0bf9b9d700 1 --2- 172.21.15.31:0/1223043407 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bf4154060 0x7f0bf40cacf0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f0bf40c37f0 tx=0x7f0bec009840 comp rx=0 tx=0).stop 2023-10-06T02:28:55.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.657+0000 7f0bf9b9d700 1 -- 172.21.15.31:0/1223043407 shutdown_connections 2023-10-06T02:28:55.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.657+0000 7f0bf9b9d700 1 --2- 172.21.15.31:0/1223043407 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bf4154060 0x7f0bf40cacf0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:55.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.657+0000 7f0bf9b9d700 1 --2- 172.21.15.31:0/1223043407 >> v2:172.21.15.62:3300/0 conn(0x7f0bf4153c90 0x7f0bf415a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:55.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.657+0000 7f0bf9b9d700 1 --2- 172.21.15.31:0/1223043407 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0bf406db20 0x7f0bf40ca7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:55.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.657+0000 7f0bf9b9d700 1 -- 172.21.15.31:0/1223043407 >> 172.21.15.31:0/1223043407 conn(0x7f0bf406e720 msgr2=0x7f0bf406eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:55.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.657+0000 7f0bf9b9d700 1 -- 172.21.15.31:0/1223043407 shutdown_connections 2023-10-06T02:28:55.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.657+0000 7f0bf9b9d700 1 -- 172.21.15.31:0/1223043407 wait complete. 2023-10-06T02:28:55.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.659+0000 7f0bf9b9d700 1 Processor -- start 2023-10-06T02:28:55.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.659+0000 7f0bf9b9d700 1 -- start start 2023-10-06T02:28:55.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.660+0000 7f0bf9b9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bf406db20 0x7f0bf416d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:55.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.660+0000 7f0bf9b9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0bf4153c90 0x7f0bf40c6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:55.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.660+0000 7f0bf9b9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bf4154060 0x7f0bf414be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:55.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.660+0000 7f0bf9b9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf40c77f0 con 0x7f0bf406db20 2023-10-06T02:28:55.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.660+0000 7f0bf9b9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf40c7ad0 con 0x7f0bf4154060 2023-10-06T02:28:55.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.660+0000 7f0bf9b9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf40c7db0 con 0x7f0bf4153c90 2023-10-06T02:28:55.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.660+0000 7f0bf8b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bf406db20 0x7f0bf416d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:55.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.660+0000 7f0bf8b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bf406db20 0x7f0bf416d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48984/0 (socket says 172.21.15.31:48984) 2023-10-06T02:28:55.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.660+0000 7f0bf8b9b700 1 -- 172.21.15.31:0/1835851420 learned_addr learned my addr 172.21.15.31:0/1835851420 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:55.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.660+0000 7f0bf8b9b700 1 -- 172.21.15.31:0/1835851420 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0bf4153c90 msgr2=0x7f0bf40c6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:55.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.660+0000 7f0bf8b9b700 1 --2- 172.21.15.31:0/1835851420 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0bf4153c90 0x7f0bf40c6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:55.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.660+0000 7f0bf8b9b700 1 -- 172.21.15.31:0/1835851420 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bf4154060 msgr2=0x7f0bf414be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:55.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.660+0000 7f0bf939c700 1 --2- 172.21.15.31:0/1835851420 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bf4154060 0x7f0bf414be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:55.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.660+0000 7f0bf8b9b700 1 --2- 172.21.15.31:0/1835851420 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bf4154060 0x7f0bf414be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:55.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.660+0000 7f0bf8b9b700 1 -- 172.21.15.31:0/1835851420 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bec009ae0 con 0x7f0bf406db20 2023-10-06T02:28:55.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.661+0000 7f0bf939c700 1 --2- 172.21.15.31:0/1835851420 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bf4154060 0x7f0bf414be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:28:55.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.661+0000 7f0bf8b9b700 1 --2- 172.21.15.31:0/1835851420 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bf406db20 0x7f0bf416d9e0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f0be800ec80 tx=0x7f0be80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:55.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.661+0000 7f0bf0ff9700 1 -- 172.21.15.31:0/1835851420 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be800c9d0 con 0x7f0bf406db20 2023-10-06T02:28:55.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.661+0000 7f0bf0ff9700 1 -- 172.21.15.31:0/1835851420 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0be800cb30 con 0x7f0bf406db20 2023-10-06T02:28:55.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.661+0000 7f0bf0ff9700 1 -- 172.21.15.31:0/1835851420 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be8013470 con 0x7f0bf406db20 2023-10-06T02:28:55.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.661+0000 7f0bf9b9d700 1 -- 172.21.15.31:0/1835851420 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bf416d1e0 con 0x7f0bf406db20 2023-10-06T02:28:55.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.661+0000 7f0bf9b9d700 1 -- 172.21.15.31:0/1835851420 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bf416df20 con 0x7f0bf406db20 2023-10-06T02:28:55.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.662+0000 7f0bf9b9d700 1 -- 172.21.15.31:0/1835851420 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bf416d6e0 con 0x7f0bf406db20 2023-10-06T02:28:55.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.663+0000 7f0bf0ff9700 1 -- 172.21.15.31:0/1835851420 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f0be8013820 con 0x7f0bf406db20 2023-10-06T02:28:55.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.664+0000 7f0bf0ff9700 1 --2- 172.21.15.31:0/1835851420 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0bd8067200 0x7f0bd80696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:55.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.665+0000 7f0bf0ff9700 1 -- 172.21.15.31:0/1835851420 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0be8084eb0 con 0x7f0bf406db20 2023-10-06T02:28:55.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.665+0000 7f0bf3fff700 1 --2- 172.21.15.31:0/1835851420 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0bd8067200 0x7f0bd80696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:55.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.666+0000 7f0bf3fff700 1 --2- 172.21.15.31:0/1835851420 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0bd8067200 0x7f0bd80696b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0bf40c6f50 tx=0x7f0be00091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:55.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.669+0000 7f0bf0ff9700 1 -- 172.21.15.31:0/1835851420 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0be8051c30 con 0x7f0bf406db20 2023-10-06T02:28:55.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.904+0000 7f0bf9b9d700 1 -- 172.21.15.31:0/1835851420 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "clone_0000000000433051", "target": ["mon-mgr", ""]}) v1 -- 0x7f0bf404c3c0 con 0x7f0bd8067200 2023-10-06T02:28:55.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.911+0000 7f0bf0ff9700 1 -- 172.21.15.31:0/1835851420 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+78 (secure 0 0 0) 0x7f0bf404c3c0 con 0x7f0bd8067200 2023-10-06T02:28:55.911 INFO:teuthology.orchestra.run.smithi031.stdout:/volumes/_nogroup/clone_0000000000433051/6d3bb52c-1833-4d2b-b295-0332ce04e82e 2023-10-06T02:28:55.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.913+0000 7f0bf9b9d700 1 -- 172.21.15.31:0/1835851420 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0bd8067200 msgr2=0x7f0bd80696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:55.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.913+0000 7f0bf9b9d700 1 --2- 172.21.15.31:0/1835851420 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0bd8067200 0x7f0bd80696b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0bf40c6f50 tx=0x7f0be00091a0 comp rx=0 tx=0).stop 2023-10-06T02:28:55.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.913+0000 7f0bf9b9d700 1 -- 172.21.15.31:0/1835851420 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bf406db20 msgr2=0x7f0bf416d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:55.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.913+0000 7f0bf9b9d700 1 --2- 172.21.15.31:0/1835851420 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bf406db20 0x7f0bf416d9e0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f0be800ec80 tx=0x7f0be80091a0 comp rx=0 tx=0).stop 2023-10-06T02:28:55.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.913+0000 7f0bf9b9d700 1 -- 172.21.15.31:0/1835851420 shutdown_connections 2023-10-06T02:28:55.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.913+0000 7f0bf9b9d700 1 --2- 172.21.15.31:0/1835851420 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bf4154060 0x7f0bf414be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:55.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.913+0000 7f0bf9b9d700 1 --2- 172.21.15.31:0/1835851420 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bf406db20 0x7f0bf416d9e0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:55.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.913+0000 7f0bf9b9d700 1 --2- 172.21.15.31:0/1835851420 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0bd8067200 0x7f0bd80696b0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:55.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.913+0000 7f0bf9b9d700 1 --2- 172.21.15.31:0/1835851420 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0bf4153c90 0x7f0bf40c6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:55.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.913+0000 7f0bf9b9d700 1 -- 172.21.15.31:0/1835851420 >> 172.21.15.31:0/1835851420 conn(0x7f0bf406e720 msgr2=0x7f0bf415bc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:55.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.913+0000 7f0bf9b9d700 1 -- 172.21.15.31:0/1835851420 shutdown_connections 2023-10-06T02:28:55.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:55.913+0000 7f0bf9b9d700 1 -- 172.21.15.31:0/1835851420 wait complete. 2023-10-06T02:28:55.932 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:28:55.932 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.rentries volumes/_nogroup/subvolume_0000000000011953/5172ced3-24b3-4d86-a22c-dcfea079edc7/.snap/snapshot_0000000000915642) 2023-10-06T02:28:55.971 INFO:teuthology.orchestra.run.smithi031.stdout:9 2023-10-06T02:28:55.971 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:28:55.971 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.rentries volumes/_nogroup/clone_0000000000433051/6d3bb52c-1833-4d2b-b295-0332ce04e82e) 2023-10-06T02:28:56.049 INFO:teuthology.orchestra.run.smithi031.stdout:9 2023-10-06T02:28:56.049 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume info cephfs clone_0000000000433051 2023-10-06T02:28:56.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:56.281+0000 7f696299d700 1 Processor -- start 2023-10-06T02:28:56.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:56.281+0000 7f696299d700 1 -- start start 2023-10-06T02:28:56.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:56.283+0000 7f696299d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f695c0c7fb0 0x7f695c06abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:56.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:56.283+0000 7f696299d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f695c06b290 con 0x7f695c13f780 2023-10-06T02:28:56.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:56.283+0000 7f696299d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f695c06b570 con 0x7f695c0c7be0 2023-10-06T02:28:56.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:56.283+0000 7f696299d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f695c06bf10 con 0x7f695c0c7fb0 2023-10-06T02:28:56.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:56.283+0000 7f696219c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f695c13f780 0x7f695c13fba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:39436/0 (socket says 172.21.15.31:39436) 2023-10-06T02:28:56.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:56.283+0000 7f696219c700 1 -- 172.21.15.31:0/3262831809 learned_addr learned my addr 172.21.15.31:0/3262831809 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:28:56.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:56.283+0000 7f696119a700 1 --2- 172.21.15.31:0/3262831809 >> v2:172.21.15.62:3300/0 conn(0x7f695c0c7fb0 0x7f695c06abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:56.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:56.283+0000 7f6960999700 1 -- 172.21.15.31:0/3262831809 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2726727376 0 0) 0x7f695c06b290 con 0x7f695c13f780 2023-10-06T02:28:56.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:56.284+0000 7f6960999700 1 -- 172.21.15.31:0/3262831809 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6948003570 con 0x7f695c13f780 2023-10-06T02:28:56.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:56.284+0000 7f6960999700 1 -- 172.21.15.31:0/3262831809 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2518805658 0 0) 0x7f695c06b570 con 0x7f695c0c7be0 2023-10-06T02:28:56.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:56.284+0000 7f6960999700 1 -- 172.21.15.31:0/3262831809 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f695c06b290 con 0x7f695c0c7be0 2023-10-06T02:28:56.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:56.284+0000 7f696119a700 1 -- 172.21.15.31:0/3262831809 >> v1:172.21.15.62:6789/0 conn(0x7f695c0c7be0 legacy=0x7f695c15a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:56.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:56.284+0000 7f696119a700 1 -- 172.21.15.31:0/3262831809 >> v1:172.21.15.31:6789/0 conn(0x7f695c13f780 legacy=0x7f695c13fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:56.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:56.284+0000 7f6960999700 1 -- 172.21.15.31:0/3262831809 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 453767064 0 0) 0x7f6948003570 con 0x7f695c13f780 2023-10-06T02:28:56.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:56.284+0000 7f696119a700 1 -- 172.21.15.31:0/3262831809 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f695c06b850 con 0x7f695c0c7fb0 2023-10-06T02:28:56.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:56.284+0000 7f696119a700 1 --2- 172.21.15.31:0/3262831809 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f695c0c7fb0 0x7f695c06abd0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f694c0098e0 tx=0x7f694c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89529928f6429941 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:56.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:56.285+0000 7f6960999700 1 -- 172.21.15.31:0/3262831809 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69540017f0 con 0x7f695c0c7fb0 2023-10-06T02:28:56.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:56.285+0000 7f6960999700 1 -- 172.21.15.31:0/3262831809 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f694c011420 con 0x7f695c0c7fb0 2023-10-06T02:28:56.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:56.285+0000 7f6960999700 1 -- 172.21.15.31:0/3262831809 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f694c0117c0 con 0x7f695c0c7fb0 2023-10-06T02:28:56.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:56.285+0000 7f696299d700 1 -- 172.21.15.31:0/3262831809 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f695c0c7fb0 msgr2=0x7f695c06abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:56.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:56.285+0000 7f696299d700 1 --2- 172.21.15.31:0/3262831809 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f695c0c7fb0 0x7f695c06abd0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f694c0098e0 tx=0x7f694c0091a0 comp rx=0 tx=0).stop 2023-10-06T02:28:56.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:56.285+0000 7f696299d700 1 -- 172.21.15.31:0/3262831809 shutdown_connections 2023-10-06T02:28:56.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:56.285+0000 7f696299d700 1 --2- 172.21.15.31:0/3262831809 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f695c0c7fb0 0x7f695c06abd0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:56.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:56.285+0000 7f696299d700 1 -- 172.21.15.31:0/3262831809 >> 172.21.15.31:0/3262831809 conn(0x7f695c06d110 msgr2=0x7f695c0cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:56.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:56.285+0000 7f696299d700 1 -- 172.21.15.31:0/3262831809 shutdown_connections 2023-10-06T02:28:56.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:56.285+0000 7f696299d700 1 -- 172.21.15.31:0/3262831809 wait complete. 2023-10-06T02:28:56.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:56.287+0000 7f696299d700 1 Processor -- start 2023-10-06T02:28:56.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:56.287+0000 7f696299d700 1 -- start start 2023-10-06T02:28:56.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:56.288+0000 7f696299d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f695c0c7be0 0x7f695c153690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:56.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:56.288+0000 7f696299d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f695c0c7fb0 0x7f695c13de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:56.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:56.288+0000 7f696299d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f695c13f780 0x7f695c13e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:56.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:56.288+0000 7f696299d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f695c156ac0 con 0x7f695c13f780 2023-10-06T02:28:56.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:56.288+0000 7f696299d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f695c157360 con 0x7f695c0c7be0 2023-10-06T02:28:56.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:56.288+0000 7f696299d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f695c0c4960 con 0x7f695c0c7fb0 2023-10-06T02:28:56.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:56.288+0000 7f696219c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f695c13f780 0x7f695c13e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:56.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:56.288+0000 7f696219c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f695c13f780 0x7f695c13e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48990/0 (socket says 172.21.15.31:48990) 2023-10-06T02:28:56.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:56.288+0000 7f696219c700 1 -- 172.21.15.31:0/2760003599 learned_addr learned my addr 172.21.15.31:0/2760003599 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:28:56.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:56.288+0000 7f696119a700 1 --2- 172.21.15.31:0/2760003599 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f695c0c7fb0 0x7f695c13de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:56.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:56.288+0000 7f696199b700 1 --2- 172.21.15.31:0/2760003599 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f695c0c7be0 0x7f695c153690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:56.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:56.289+0000 7f696219c700 1 -- 172.21.15.31:0/2760003599 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f695c0c7fb0 msgr2=0x7f695c13de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:56.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:56.289+0000 7f696219c700 1 --2- 172.21.15.31:0/2760003599 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f695c0c7fb0 0x7f695c13de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:56.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:56.289+0000 7f696219c700 1 -- 172.21.15.31:0/2760003599 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f695c0c7be0 msgr2=0x7f695c153690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:56.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:56.289+0000 7f696219c700 1 --2- 172.21.15.31:0/2760003599 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f695c0c7be0 0x7f695c153690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:56.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:56.289+0000 7f696219c700 1 -- 172.21.15.31:0/2760003599 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f694c009420 con 0x7f695c13f780 2023-10-06T02:28:56.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:56.289+0000 7f696219c700 1 --2- 172.21.15.31:0/2760003599 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f695c13f780 0x7f695c13e3a0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f6954001560 tx=0x7f6954007d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:56.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:56.289+0000 7f6951ffb700 1 -- 172.21.15.31:0/2760003599 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6954004dc0 con 0x7f695c13f780 2023-10-06T02:28:56.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:56.289+0000 7f6951ffb700 1 -- 172.21.15.31:0/2760003599 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f695400a660 con 0x7f695c13f780 2023-10-06T02:28:56.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:56.289+0000 7f6951ffb700 1 -- 172.21.15.31:0/2760003599 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f695400aa20 con 0x7f695c13f780 2023-10-06T02:28:56.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:56.289+0000 7f696299d700 1 -- 172.21.15.31:0/2760003599 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f695c156d10 con 0x7f695c13f780 2023-10-06T02:28:56.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:56.289+0000 7f696299d700 1 -- 172.21.15.31:0/2760003599 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f695c153bd0 con 0x7f695c13f780 2023-10-06T02:28:56.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:56.290+0000 7f696299d700 1 -- 172.21.15.31:0/2760003599 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f695c1574d0 con 0x7f695c13f780 2023-10-06T02:28:56.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:56.291+0000 7f6951ffb700 1 -- 172.21.15.31:0/2760003599 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f6954007150 con 0x7f695c13f780 2023-10-06T02:28:56.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:56.292+0000 7f6951ffb700 1 --2- 172.21.15.31:0/2760003599 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6940067200 0x7f69400696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:28:56.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:56.292+0000 7f6951ffb700 1 -- 172.21.15.31:0/2760003599 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6954084620 con 0x7f695c13f780 2023-10-06T02:28:56.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:56.293+0000 7f696199b700 1 --2- 172.21.15.31:0/2760003599 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6940067200 0x7f69400696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:28:56.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:56.293+0000 7f696199b700 1 --2- 172.21.15.31:0/2760003599 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6940067200 0x7f69400696b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6958003870 tx=0x7f6958009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:28:56.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:56.296+0000 7f6951ffb700 1 -- 172.21.15.31:0/2760003599 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6954051460 con 0x7f695c13f780 2023-10-06T02:28:56.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:56.534+0000 7f696299d700 1 -- 172.21.15.31:0/2760003599 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "fs subvolume info", "vol_name": "cephfs", "sub_name": "clone_0000000000433051", "target": ["mon-mgr", ""]}) v1 -- 0x7f695c04c3c0 con 0x7f6940067200 2023-10-06T02:28:56.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:56.546+0000 7f6951ffb700 1 -- 172.21.15.31:0/2760003599 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+710 (secure 0 0 0) 0x7f695c04c3c0 con 0x7f6940067200 2023-10-06T02:28:56.547 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T02:28:56.547 INFO:teuthology.orchestra.run.smithi031.stdout: "atime": "2023-10-06 02:28:47", 2023-10-06T02:28:56.547 INFO:teuthology.orchestra.run.smithi031.stdout: "bytes_pcent": "80.00", 2023-10-06T02:28:56.548 INFO:teuthology.orchestra.run.smithi031.stdout: "bytes_quota": 10485760, 2023-10-06T02:28:56.548 INFO:teuthology.orchestra.run.smithi031.stdout: "bytes_used": 8388608, 2023-10-06T02:28:56.548 INFO:teuthology.orchestra.run.smithi031.stdout: "created_at": "2023-10-06 02:28:53", 2023-10-06T02:28:56.548 INFO:teuthology.orchestra.run.smithi031.stdout: "ctime": "2023-10-06 02:28:53", 2023-10-06T02:28:56.548 INFO:teuthology.orchestra.run.smithi031.stdout: "data_pool": "cephfs_data", 2023-10-06T02:28:56.549 INFO:teuthology.orchestra.run.smithi031.stdout: "features": [ 2023-10-06T02:28:56.549 INFO:teuthology.orchestra.run.smithi031.stdout: "snapshot-clone", 2023-10-06T02:28:56.549 INFO:teuthology.orchestra.run.smithi031.stdout: "snapshot-autoprotect", 2023-10-06T02:28:56.549 INFO:teuthology.orchestra.run.smithi031.stdout: "snapshot-retention" 2023-10-06T02:28:56.549 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:28:56.550 INFO:teuthology.orchestra.run.smithi031.stdout: "gid": 1000, 2023-10-06T02:28:56.550 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": 16895, 2023-10-06T02:28:56.550 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_addrs": [ 2023-10-06T02:28:56.550 INFO:teuthology.orchestra.run.smithi031.stdout: "172.21.15.31:6789", 2023-10-06T02:28:56.550 INFO:teuthology.orchestra.run.smithi031.stdout: "172.21.15.62:6789", 2023-10-06T02:28:56.550 INFO:teuthology.orchestra.run.smithi031.stdout: "172.21.15.62:6790" 2023-10-06T02:28:56.551 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T02:28:56.551 INFO:teuthology.orchestra.run.smithi031.stdout: "mtime": "2023-10-06 02:28:52", 2023-10-06T02:28:56.551 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/volumes/_nogroup/clone_0000000000433051/6d3bb52c-1833-4d2b-b295-0332ce04e82e", 2023-10-06T02:28:56.551 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_namespace": "", 2023-10-06T02:28:56.551 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "complete", 2023-10-06T02:28:56.552 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "clone", 2023-10-06T02:28:56.552 INFO:teuthology.orchestra.run.smithi031.stdout: "uid": 1000 2023-10-06T02:28:56.552 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T02:28:56.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:56.549+0000 7f696299d700 1 -- 172.21.15.31:0/2760003599 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6940067200 msgr2=0x7f69400696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:56.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:56.549+0000 7f696299d700 1 --2- 172.21.15.31:0/2760003599 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6940067200 0x7f69400696b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6958003870 tx=0x7f6958009040 comp rx=0 tx=0).stop 2023-10-06T02:28:56.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:56.549+0000 7f696299d700 1 -- 172.21.15.31:0/2760003599 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f695c13f780 msgr2=0x7f695c13e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:28:56.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:56.549+0000 7f696299d700 1 --2- 172.21.15.31:0/2760003599 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f695c13f780 0x7f695c13e3a0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f6954001560 tx=0x7f6954007d20 comp rx=0 tx=0).stop 2023-10-06T02:28:56.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:56.549+0000 7f696299d700 1 -- 172.21.15.31:0/2760003599 shutdown_connections 2023-10-06T02:28:56.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:56.549+0000 7f696299d700 1 --2- 172.21.15.31:0/2760003599 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f695c13f780 0x7f695c13e3a0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:56.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:56.549+0000 7f696299d700 1 --2- 172.21.15.31:0/2760003599 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f695c0c7be0 0x7f695c153690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:56.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:56.549+0000 7f696299d700 1 --2- 172.21.15.31:0/2760003599 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6940067200 0x7f69400696b0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:56.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:56.549+0000 7f696299d700 1 --2- 172.21.15.31:0/2760003599 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f695c0c7fb0 0x7f695c13de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:28:56.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:56.549+0000 7f696299d700 1 -- 172.21.15.31:0/2760003599 >> 172.21.15.31:0/2760003599 conn(0x7f695c06d110 msgr2=0x7f695c0ca8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:28:56.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:56.549+0000 7f696299d700 1 -- 172.21.15.31:0/2760003599 shutdown_connections 2023-10-06T02:28:56.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:28:56.549+0000 7f696299d700 1 -- 172.21.15.31:0/2760003599 wait complete. 2023-10-06T02:28:56.567 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:28:56.567 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.quota.max_bytes volumes/_nogroup/subvolume_0000000000011953/5172ced3-24b3-4d86-a22c-dcfea079edc7/.snap/snapshot_0000000000915642) 2023-10-06T02:28:56.604 INFO:teuthology.orchestra.run.smithi031.stderr:volumes/_nogroup/subvolume_0000000000011953/5172ced3-24b3-4d86-a22c-dcfea079edc7/.snap/snapshot_0000000000915642: ceph.quota.max_bytes: No such attribute 2023-10-06T02:28:56.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T02:28:56.605 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:28:56.605 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool volumes/_nogroup/subvolume_0000000000011953/5172ced3-24b3-4d86-a22c-dcfea079edc7/.snap/snapshot_0000000000915642) 2023-10-06T02:28:56.683 INFO:teuthology.orchestra.run.smithi031.stdout:cephfs_data 2023-10-06T02:28:56.683 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:28:56.683 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool_namespace volumes/_nogroup/subvolume_0000000000011953/5172ced3-24b3-4d86-a22c-dcfea079edc7/.snap/snapshot_0000000000915642) 2023-10-06T02:28:56.760 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:28:56.760 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec find volumes/_nogroup/subvolume_0000000000011953/5172ced3-24b3-4d86-a22c-dcfea079edc7/.snap/snapshot_0000000000915642) 2023-10-06T02:28:56.839 INFO:teuthology.orchestra.run.smithi031.stdout:volumes/_nogroup/subvolume_0000000000011953/5172ced3-24b3-4d86-a22c-dcfea079edc7/.snap/snapshot_0000000000915642 2023-10-06T02:28:56.839 INFO:teuthology.orchestra.run.smithi031.stdout:volumes/_nogroup/subvolume_0000000000011953/5172ced3-24b3-4d86-a22c-dcfea079edc7/.snap/snapshot_0000000000915642/subvolume_file.2 2023-10-06T02:28:56.839 INFO:teuthology.orchestra.run.smithi031.stdout:volumes/_nogroup/subvolume_0000000000011953/5172ced3-24b3-4d86-a22c-dcfea079edc7/.snap/snapshot_0000000000915642/subvolume_file.5 2023-10-06T02:28:56.839 INFO:teuthology.orchestra.run.smithi031.stdout:volumes/_nogroup/subvolume_0000000000011953/5172ced3-24b3-4d86-a22c-dcfea079edc7/.snap/snapshot_0000000000915642/subvolume_file.4 2023-10-06T02:28:56.839 INFO:teuthology.orchestra.run.smithi031.stdout:volumes/_nogroup/subvolume_0000000000011953/5172ced3-24b3-4d86-a22c-dcfea079edc7/.snap/snapshot_0000000000915642/subvolume_file.7 2023-10-06T02:28:56.839 INFO:teuthology.orchestra.run.smithi031.stdout:volumes/_nogroup/subvolume_0000000000011953/5172ced3-24b3-4d86-a22c-dcfea079edc7/.snap/snapshot_0000000000915642/subvolume_file.6 2023-10-06T02:28:56.839 INFO:teuthology.orchestra.run.smithi031.stdout:volumes/_nogroup/subvolume_0000000000011953/5172ced3-24b3-4d86-a22c-dcfea079edc7/.snap/snapshot_0000000000915642/subvolume_file.0 2023-10-06T02:28:56.840 INFO:teuthology.orchestra.run.smithi031.stdout:volumes/_nogroup/subvolume_0000000000011953/5172ced3-24b3-4d86-a22c-dcfea079edc7/.snap/snapshot_0000000000915642/subvolume_file.3 2023-10-06T02:28:56.840 INFO:teuthology.orchestra.run.smithi031.stdout:volumes/_nogroup/subvolume_0000000000011953/5172ced3-24b3-4d86-a22c-dcfea079edc7/.snap/snapshot_0000000000915642/subvolume_file.1 2023-10-06T02:28:56.841 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:28:56.841 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvolume_0000000000011953/5172ced3-24b3-4d86-a22c-dcfea079edc7/.snap/snapshot_0000000000915642) 2023-10-06T02:28:56.918 INFO:teuthology.orchestra.run.smithi031.stdout:41ff 2023-10-06T02:28:56.919 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:28:56.919 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/clone_0000000000433051/6d3bb52c-1833-4d2b-b295-0332ce04e82e/) 2023-10-06T02:28:56.996 INFO:teuthology.orchestra.run.smithi031.stdout:41ff 2023-10-06T02:28:56.997 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:28:56.997 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvolume_0000000000011953/5172ced3-24b3-4d86-a22c-dcfea079edc7/.snap/snapshot_0000000000915642) 2023-10-06T02:28:57.073 INFO:teuthology.orchestra.run.smithi031.stdout:1000 2023-10-06T02:28:57.074 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:28:57.075 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/clone_0000000000433051/6d3bb52c-1833-4d2b-b295-0332ce04e82e/) 2023-10-06T02:28:57.152 INFO:teuthology.orchestra.run.smithi031.stdout:1000 2023-10-06T02:28:57.153 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:28:57.153 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvolume_0000000000011953/5172ced3-24b3-4d86-a22c-dcfea079edc7/.snap/snapshot_0000000000915642) 2023-10-06T02:28:57.230 INFO:teuthology.orchestra.run.smithi031.stdout:1000 2023-10-06T02:28:57.231 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:28:57.231 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/clone_0000000000433051/6d3bb52c-1833-4d2b-b295-0332ce04e82e/) 2023-10-06T02:28:57.307 INFO:teuthology.orchestra.run.smithi031.stdout:1000 2023-10-06T02:28:57.308 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:28:57.309 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvolume_0000000000011953/5172ced3-24b3-4d86-a22c-dcfea079edc7/.snap/snapshot_0000000000915642) 2023-10-06T02:28:57.385 INFO:teuthology.orchestra.run.smithi031.stdout:1696559332 2023-10-06T02:28:57.386 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:28:57.386 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/clone_0000000000433051/6d3bb52c-1833-4d2b-b295-0332ce04e82e/) 2023-10-06T02:28:57.462 INFO:teuthology.orchestra.run.smithi031.stdout:1696559332 2023-10-06T02:28:57.463 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:28:57.463 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvolume_0000000000011953/5172ced3-24b3-4d86-a22c-dcfea079edc7/.snap/snapshot_0000000000915642/subvolume_file.2) 2023-10-06T02:28:57.546 INFO:teuthology.orchestra.run.smithi031.stdout:81b4 2023-10-06T02:28:57.547 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:28:57.547 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/clone_0000000000433051/6d3bb52c-1833-4d2b-b295-0332ce04e82e/subvolume_file.2) 2023-10-06T02:28:57.628 INFO:teuthology.orchestra.run.smithi031.stdout:81b4 2023-10-06T02:28:57.629 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:28:57.629 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvolume_0000000000011953/5172ced3-24b3-4d86-a22c-dcfea079edc7/.snap/snapshot_0000000000915642/subvolume_file.2) 2023-10-06T02:28:57.705 INFO:teuthology.orchestra.run.smithi031.stdout:1000 2023-10-06T02:28:57.706 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:28:57.707 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/clone_0000000000433051/6d3bb52c-1833-4d2b-b295-0332ce04e82e/subvolume_file.2) 2023-10-06T02:28:57.784 INFO:teuthology.orchestra.run.smithi031.stdout:1000 2023-10-06T02:28:57.785 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:28:57.785 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvolume_0000000000011953/5172ced3-24b3-4d86-a22c-dcfea079edc7/.snap/snapshot_0000000000915642/subvolume_file.2) 2023-10-06T02:28:57.862 INFO:teuthology.orchestra.run.smithi031.stdout:1275 2023-10-06T02:28:57.863 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:28:57.863 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/clone_0000000000433051/6d3bb52c-1833-4d2b-b295-0332ce04e82e/subvolume_file.2) 2023-10-06T02:28:57.940 INFO:teuthology.orchestra.run.smithi031.stdout:1275 2023-10-06T02:28:57.941 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:28:57.941 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvolume_0000000000011953/5172ced3-24b3-4d86-a22c-dcfea079edc7/.snap/snapshot_0000000000915642/subvolume_file.2) 2023-10-06T02:28:58.017 INFO:teuthology.orchestra.run.smithi031.stdout:1696559332 2023-10-06T02:28:58.018 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:28:58.018 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/clone_0000000000433051/6d3bb52c-1833-4d2b-b295-0332ce04e82e/subvolume_file.2) 2023-10-06T02:28:58.094 INFO:teuthology.orchestra.run.smithi031.stdout:1696559332 2023-10-06T02:28:58.096 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:28:58.096 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvolume_0000000000011953/5172ced3-24b3-4d86-a22c-dcfea079edc7/.snap/snapshot_0000000000915642/subvolume_file.5) 2023-10-06T02:28:58.177 INFO:teuthology.orchestra.run.smithi031.stdout:81b4 2023-10-06T02:28:58.178 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:28:58.178 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/clone_0000000000433051/6d3bb52c-1833-4d2b-b295-0332ce04e82e/subvolume_file.5) 2023-10-06T02:28:58.259 INFO:teuthology.orchestra.run.smithi031.stdout:81b4 2023-10-06T02:28:58.260 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:28:58.260 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvolume_0000000000011953/5172ced3-24b3-4d86-a22c-dcfea079edc7/.snap/snapshot_0000000000915642/subvolume_file.5) 2023-10-06T02:28:58.337 INFO:teuthology.orchestra.run.smithi031.stdout:1000 2023-10-06T02:28:58.338 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:28:58.338 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/clone_0000000000433051/6d3bb52c-1833-4d2b-b295-0332ce04e82e/subvolume_file.5) 2023-10-06T02:28:58.415 INFO:teuthology.orchestra.run.smithi031.stdout:1000 2023-10-06T02:28:58.416 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:28:58.416 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvolume_0000000000011953/5172ced3-24b3-4d86-a22c-dcfea079edc7/.snap/snapshot_0000000000915642/subvolume_file.5) 2023-10-06T02:28:58.493 INFO:teuthology.orchestra.run.smithi031.stdout:1275 2023-10-06T02:28:58.494 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:28:58.494 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/clone_0000000000433051/6d3bb52c-1833-4d2b-b295-0332ce04e82e/subvolume_file.5) 2023-10-06T02:28:58.572 INFO:teuthology.orchestra.run.smithi031.stdout:1275 2023-10-06T02:28:58.573 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:28:58.573 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvolume_0000000000011953/5172ced3-24b3-4d86-a22c-dcfea079edc7/.snap/snapshot_0000000000915642/subvolume_file.5) 2023-10-06T02:28:58.650 INFO:teuthology.orchestra.run.smithi031.stdout:1696559332 2023-10-06T02:28:58.651 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:28:58.651 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/clone_0000000000433051/6d3bb52c-1833-4d2b-b295-0332ce04e82e/subvolume_file.5) 2023-10-06T02:28:58.728 INFO:teuthology.orchestra.run.smithi031.stdout:1696559332 2023-10-06T02:28:58.729 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:28:58.729 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvolume_0000000000011953/5172ced3-24b3-4d86-a22c-dcfea079edc7/.snap/snapshot_0000000000915642/subvolume_file.4) 2023-10-06T02:28:58.811 INFO:teuthology.orchestra.run.smithi031.stdout:81b4 2023-10-06T02:28:58.812 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:28:58.812 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/clone_0000000000433051/6d3bb52c-1833-4d2b-b295-0332ce04e82e/subvolume_file.4) 2023-10-06T02:28:58.894 INFO:teuthology.orchestra.run.smithi031.stdout:81b4 2023-10-06T02:28:58.896 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:28:58.896 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvolume_0000000000011953/5172ced3-24b3-4d86-a22c-dcfea079edc7/.snap/snapshot_0000000000915642/subvolume_file.4) 2023-10-06T02:28:58.973 INFO:teuthology.orchestra.run.smithi031.stdout:1000 2023-10-06T02:28:58.974 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:28:58.974 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/clone_0000000000433051/6d3bb52c-1833-4d2b-b295-0332ce04e82e/subvolume_file.4) 2023-10-06T02:28:59.051 INFO:teuthology.orchestra.run.smithi031.stdout:1000 2023-10-06T02:28:59.052 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:28:59.052 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvolume_0000000000011953/5172ced3-24b3-4d86-a22c-dcfea079edc7/.snap/snapshot_0000000000915642/subvolume_file.4) 2023-10-06T02:28:59.130 INFO:teuthology.orchestra.run.smithi031.stdout:1275 2023-10-06T02:28:59.131 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:28:59.131 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/clone_0000000000433051/6d3bb52c-1833-4d2b-b295-0332ce04e82e/subvolume_file.4) 2023-10-06T02:28:59.208 INFO:teuthology.orchestra.run.smithi031.stdout:1275 2023-10-06T02:28:59.209 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:28:59.209 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvolume_0000000000011953/5172ced3-24b3-4d86-a22c-dcfea079edc7/.snap/snapshot_0000000000915642/subvolume_file.4) 2023-10-06T02:28:59.286 INFO:teuthology.orchestra.run.smithi031.stdout:1696559332 2023-10-06T02:28:59.287 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:28:59.288 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/clone_0000000000433051/6d3bb52c-1833-4d2b-b295-0332ce04e82e/subvolume_file.4) 2023-10-06T02:28:59.364 INFO:teuthology.orchestra.run.smithi031.stdout:1696559332 2023-10-06T02:28:59.365 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:28:59.365 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvolume_0000000000011953/5172ced3-24b3-4d86-a22c-dcfea079edc7/.snap/snapshot_0000000000915642/subvolume_file.7) 2023-10-06T02:28:59.446 INFO:teuthology.orchestra.run.smithi031.stdout:81b4 2023-10-06T02:28:59.447 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:28:59.447 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/clone_0000000000433051/6d3bb52c-1833-4d2b-b295-0332ce04e82e/subvolume_file.7) 2023-10-06T02:28:59.528 INFO:teuthology.orchestra.run.smithi031.stdout:81b4 2023-10-06T02:28:59.530 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:28:59.530 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvolume_0000000000011953/5172ced3-24b3-4d86-a22c-dcfea079edc7/.snap/snapshot_0000000000915642/subvolume_file.7) 2023-10-06T02:28:59.607 INFO:teuthology.orchestra.run.smithi031.stdout:1000 2023-10-06T02:28:59.608 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:28:59.608 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/clone_0000000000433051/6d3bb52c-1833-4d2b-b295-0332ce04e82e/subvolume_file.7) 2023-10-06T02:28:59.684 INFO:teuthology.orchestra.run.smithi031.stdout:1000 2023-10-06T02:28:59.685 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:28:59.686 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvolume_0000000000011953/5172ced3-24b3-4d86-a22c-dcfea079edc7/.snap/snapshot_0000000000915642/subvolume_file.7) 2023-10-06T02:28:59.763 INFO:teuthology.orchestra.run.smithi031.stdout:1275 2023-10-06T02:28:59.764 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:28:59.764 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/clone_0000000000433051/6d3bb52c-1833-4d2b-b295-0332ce04e82e/subvolume_file.7) 2023-10-06T02:28:59.841 INFO:teuthology.orchestra.run.smithi031.stdout:1275 2023-10-06T02:28:59.842 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:28:59.842 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvolume_0000000000011953/5172ced3-24b3-4d86-a22c-dcfea079edc7/.snap/snapshot_0000000000915642/subvolume_file.7) 2023-10-06T02:28:59.919 INFO:teuthology.orchestra.run.smithi031.stdout:1696559332 2023-10-06T02:28:59.920 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:28:59.920 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/clone_0000000000433051/6d3bb52c-1833-4d2b-b295-0332ce04e82e/subvolume_file.7) 2023-10-06T02:28:59.998 INFO:teuthology.orchestra.run.smithi031.stdout:1696559332 2023-10-06T02:28:59.999 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:28:59.999 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvolume_0000000000011953/5172ced3-24b3-4d86-a22c-dcfea079edc7/.snap/snapshot_0000000000915642/subvolume_file.6) 2023-10-06T02:29:00.081 INFO:teuthology.orchestra.run.smithi031.stdout:81b4 2023-10-06T02:29:00.082 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:29:00.083 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/clone_0000000000433051/6d3bb52c-1833-4d2b-b295-0332ce04e82e/subvolume_file.6) 2023-10-06T02:29:00.161 INFO:teuthology.orchestra.run.smithi031.stdout:81b4 2023-10-06T02:29:00.163 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:29:00.163 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvolume_0000000000011953/5172ced3-24b3-4d86-a22c-dcfea079edc7/.snap/snapshot_0000000000915642/subvolume_file.6) 2023-10-06T02:29:00.240 INFO:teuthology.orchestra.run.smithi031.stdout:1000 2023-10-06T02:29:00.241 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:29:00.241 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/clone_0000000000433051/6d3bb52c-1833-4d2b-b295-0332ce04e82e/subvolume_file.6) 2023-10-06T02:29:00.318 INFO:teuthology.orchestra.run.smithi031.stdout:1000 2023-10-06T02:29:00.319 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:29:00.319 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvolume_0000000000011953/5172ced3-24b3-4d86-a22c-dcfea079edc7/.snap/snapshot_0000000000915642/subvolume_file.6) 2023-10-06T02:29:00.396 INFO:teuthology.orchestra.run.smithi031.stdout:1275 2023-10-06T02:29:00.397 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:29:00.397 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/clone_0000000000433051/6d3bb52c-1833-4d2b-b295-0332ce04e82e/subvolume_file.6) 2023-10-06T02:29:00.473 INFO:teuthology.orchestra.run.smithi031.stdout:1275 2023-10-06T02:29:00.474 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:29:00.475 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvolume_0000000000011953/5172ced3-24b3-4d86-a22c-dcfea079edc7/.snap/snapshot_0000000000915642/subvolume_file.6) 2023-10-06T02:29:00.552 INFO:teuthology.orchestra.run.smithi031.stdout:1696559332 2023-10-06T02:29:00.553 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:29:00.553 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/clone_0000000000433051/6d3bb52c-1833-4d2b-b295-0332ce04e82e/subvolume_file.6) 2023-10-06T02:29:00.630 INFO:teuthology.orchestra.run.smithi031.stdout:1696559332 2023-10-06T02:29:00.631 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:29:00.631 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvolume_0000000000011953/5172ced3-24b3-4d86-a22c-dcfea079edc7/.snap/snapshot_0000000000915642/subvolume_file.0) 2023-10-06T02:29:00.711 INFO:teuthology.orchestra.run.smithi031.stdout:81b4 2023-10-06T02:29:00.712 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:29:00.713 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/clone_0000000000433051/6d3bb52c-1833-4d2b-b295-0332ce04e82e/subvolume_file.0) 2023-10-06T02:29:00.789 INFO:teuthology.orchestra.run.smithi031.stdout:81b4 2023-10-06T02:29:00.790 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:29:00.791 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvolume_0000000000011953/5172ced3-24b3-4d86-a22c-dcfea079edc7/.snap/snapshot_0000000000915642/subvolume_file.0) 2023-10-06T02:29:00.867 INFO:teuthology.orchestra.run.smithi031.stdout:1000 2023-10-06T02:29:00.868 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:29:00.869 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/clone_0000000000433051/6d3bb52c-1833-4d2b-b295-0332ce04e82e/subvolume_file.0) 2023-10-06T02:29:00.946 INFO:teuthology.orchestra.run.smithi031.stdout:1000 2023-10-06T02:29:00.947 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:29:00.947 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvolume_0000000000011953/5172ced3-24b3-4d86-a22c-dcfea079edc7/.snap/snapshot_0000000000915642/subvolume_file.0) 2023-10-06T02:29:01.023 INFO:teuthology.orchestra.run.smithi031.stdout:1275 2023-10-06T02:29:01.024 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:29:01.024 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/clone_0000000000433051/6d3bb52c-1833-4d2b-b295-0332ce04e82e/subvolume_file.0) 2023-10-06T02:29:01.102 INFO:teuthology.orchestra.run.smithi031.stdout:1275 2023-10-06T02:29:01.103 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:29:01.103 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvolume_0000000000011953/5172ced3-24b3-4d86-a22c-dcfea079edc7/.snap/snapshot_0000000000915642/subvolume_file.0) 2023-10-06T02:29:01.179 INFO:teuthology.orchestra.run.smithi031.stdout:1696559332 2023-10-06T02:29:01.180 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:29:01.180 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/clone_0000000000433051/6d3bb52c-1833-4d2b-b295-0332ce04e82e/subvolume_file.0) 2023-10-06T02:29:01.256 INFO:teuthology.orchestra.run.smithi031.stdout:1696559332 2023-10-06T02:29:01.257 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:29:01.257 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvolume_0000000000011953/5172ced3-24b3-4d86-a22c-dcfea079edc7/.snap/snapshot_0000000000915642/subvolume_file.3) 2023-10-06T02:29:01.338 INFO:teuthology.orchestra.run.smithi031.stdout:81b4 2023-10-06T02:29:01.339 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:29:01.340 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/clone_0000000000433051/6d3bb52c-1833-4d2b-b295-0332ce04e82e/subvolume_file.3) 2023-10-06T02:29:01.417 INFO:teuthology.orchestra.run.smithi031.stdout:81b4 2023-10-06T02:29:01.419 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:29:01.419 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvolume_0000000000011953/5172ced3-24b3-4d86-a22c-dcfea079edc7/.snap/snapshot_0000000000915642/subvolume_file.3) 2023-10-06T02:29:01.496 INFO:teuthology.orchestra.run.smithi031.stdout:1000 2023-10-06T02:29:01.497 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:29:01.497 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/clone_0000000000433051/6d3bb52c-1833-4d2b-b295-0332ce04e82e/subvolume_file.3) 2023-10-06T02:29:01.573 INFO:teuthology.orchestra.run.smithi031.stdout:1000 2023-10-06T02:29:01.574 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:29:01.574 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvolume_0000000000011953/5172ced3-24b3-4d86-a22c-dcfea079edc7/.snap/snapshot_0000000000915642/subvolume_file.3) 2023-10-06T02:29:01.650 INFO:teuthology.orchestra.run.smithi031.stdout:1275 2023-10-06T02:29:01.652 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:29:01.652 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/clone_0000000000433051/6d3bb52c-1833-4d2b-b295-0332ce04e82e/subvolume_file.3) 2023-10-06T02:29:01.705 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T02:29:01.728 INFO:teuthology.orchestra.run.smithi031.stdout:1275 2023-10-06T02:29:01.729 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:29:01.729 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvolume_0000000000011953/5172ced3-24b3-4d86-a22c-dcfea079edc7/.snap/snapshot_0000000000915642/subvolume_file.3) 2023-10-06T02:29:01.732 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T02:29:01.768 INFO:teuthology.orchestra.run.smithi031.stdout:1696559332 2023-10-06T02:29:01.768 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:29:01.769 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/clone_0000000000433051/6d3bb52c-1833-4d2b-b295-0332ce04e82e/subvolume_file.3) 2023-10-06T02:29:01.846 INFO:teuthology.orchestra.run.smithi031.stdout:1696559332 2023-10-06T02:29:01.847 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:29:01.847 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvolume_0000000000011953/5172ced3-24b3-4d86-a22c-dcfea079edc7/.snap/snapshot_0000000000915642/subvolume_file.1) 2023-10-06T02:29:01.936 INFO:teuthology.orchestra.run.smithi031.stdout:81b4 2023-10-06T02:29:01.937 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:29:01.937 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/clone_0000000000433051/6d3bb52c-1833-4d2b-b295-0332ce04e82e/subvolume_file.1) 2023-10-06T02:29:02.014 INFO:teuthology.orchestra.run.smithi031.stdout:81b4 2023-10-06T02:29:02.015 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:29:02.015 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvolume_0000000000011953/5172ced3-24b3-4d86-a22c-dcfea079edc7/.snap/snapshot_0000000000915642/subvolume_file.1) 2023-10-06T02:29:02.092 INFO:teuthology.orchestra.run.smithi031.stdout:1000 2023-10-06T02:29:02.093 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:29:02.093 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/clone_0000000000433051/6d3bb52c-1833-4d2b-b295-0332ce04e82e/subvolume_file.1) 2023-10-06T02:29:02.169 INFO:teuthology.orchestra.run.smithi031.stdout:1000 2023-10-06T02:29:02.170 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:29:02.170 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvolume_0000000000011953/5172ced3-24b3-4d86-a22c-dcfea079edc7/.snap/snapshot_0000000000915642/subvolume_file.1) 2023-10-06T02:29:02.249 INFO:teuthology.orchestra.run.smithi031.stdout:1275 2023-10-06T02:29:02.250 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:29:02.250 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/clone_0000000000433051/6d3bb52c-1833-4d2b-b295-0332ce04e82e/subvolume_file.1) 2023-10-06T02:29:02.326 INFO:teuthology.orchestra.run.smithi031.stdout:1275 2023-10-06T02:29:02.327 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:29:02.328 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvolume_0000000000011953/5172ced3-24b3-4d86-a22c-dcfea079edc7/.snap/snapshot_0000000000915642/subvolume_file.1) 2023-10-06T02:29:02.404 INFO:teuthology.orchestra.run.smithi031.stdout:1696559332 2023-10-06T02:29:02.405 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:29:02.406 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/clone_0000000000433051/6d3bb52c-1833-4d2b-b295-0332ce04e82e/subvolume_file.1) 2023-10-06T02:29:02.483 INFO:teuthology.orchestra.run.smithi031.stdout:1696559332 2023-10-06T02:29:02.484 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs subvolume_0000000000011953 fake clone_0000000000433052 2023-10-06T02:29:02.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:02.716+0000 7f8c7559d700 1 Processor -- start 2023-10-06T02:29:02.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:02.716+0000 7f8c7559d700 1 -- start start 2023-10-06T02:29:02.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:02.718+0000 7f8c7559d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8c70140a70 0x7f8c700c6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:02.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:02.718+0000 7f8c7559d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8c7006de50 0x7f8c700aeef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:02.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:02.718+0000 7f8c7559d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c700c6d50 con 0x7f8c70140e40 2023-10-06T02:29:02.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:02.718+0000 7f8c7559d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c700c7030 con 0x7f8c7006de50 2023-10-06T02:29:02.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:02.718+0000 7f8c7559d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c700c7310 con 0x7f8c70140a70 2023-10-06T02:29:02.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:02.718+0000 7f8c74d9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8c7006de50 0x7f8c700aeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:02.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:02.718+0000 7f8c74d9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8c7006de50 0x7f8c700aeef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47174/0 (socket says 172.21.15.31:47174) 2023-10-06T02:29:02.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:02.718+0000 7f8c74d9c700 1 -- 172.21.15.31:0/2821883956 learned_addr learned my addr 172.21.15.31:0/2821883956 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:02.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:02.718+0000 7f8c6ffff700 1 --2- 172.21.15.31:0/2821883956 >> v2:172.21.15.62:3300/0 conn(0x7f8c70140a70 0x7f8c700c6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:02.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:02.718+0000 7f8c74d9c700 1 -- 172.21.15.31:0/2821883956 >> v2:172.21.15.62:3300/0 conn(0x7f8c70140a70 msgr2=0x7f8c700c6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:02.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:02.718+0000 7f8c74d9c700 1 --2- 172.21.15.31:0/2821883956 >> v2:172.21.15.62:3300/0 conn(0x7f8c70140a70 0x7f8c700c6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:02.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:02.718+0000 7f8c74d9c700 1 -- 172.21.15.31:0/2821883956 >> v1:172.21.15.62:6789/0 conn(0x7f8c70140e40 legacy=0x7f8c700c7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:02.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:02.719+0000 7f8c74d9c700 1 -- 172.21.15.31:0/2821883956 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c700c75f0 con 0x7f8c7006de50 2023-10-06T02:29:02.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:02.719+0000 7f8c6ffff700 1 --2- 172.21.15.31:0/2821883956 >> v2:172.21.15.62:3300/0 conn(0x7f8c70140a70 0x7f8c700c6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:29:02.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:02.719+0000 7f8c74d9c700 1 --2- 172.21.15.31:0/2821883956 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c7006de50 0x7f8c700aeef0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f8c70069230 tx=0x7f8c68009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=634dcfca85d45e24 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:02.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:02.719+0000 7f8c6effd700 1 -- 172.21.15.31:0/2821883956 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c6800e070 con 0x7f8c7006de50 2023-10-06T02:29:02.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:02.719+0000 7f8c6effd700 1 -- 172.21.15.31:0/2821883956 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c6800ba50 con 0x7f8c7006de50 2023-10-06T02:29:02.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:02.719+0000 7f8c6effd700 1 -- 172.21.15.31:0/2821883956 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c680092e0 con 0x7f8c7006de50 2023-10-06T02:29:02.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:02.720+0000 7f8c7559d700 1 -- 172.21.15.31:0/2821883956 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c7006de50 msgr2=0x7f8c700aeef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:02.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:02.720+0000 7f8c7559d700 1 --2- 172.21.15.31:0/2821883956 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c7006de50 0x7f8c700aeef0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f8c70069230 tx=0x7f8c68009840 comp rx=0 tx=0).stop 2023-10-06T02:29:02.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:02.720+0000 7f8c7559d700 1 -- 172.21.15.31:0/2821883956 shutdown_connections 2023-10-06T02:29:02.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:02.720+0000 7f8c7559d700 1 --2- 172.21.15.31:0/2821883956 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c7006de50 0x7f8c700aeef0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:02.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:02.720+0000 7f8c7559d700 1 --2- 172.21.15.31:0/2821883956 >> v2:172.21.15.62:3300/0 conn(0x7f8c70140a70 0x7f8c700c6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:02.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:02.720+0000 7f8c7559d700 1 -- 172.21.15.31:0/2821883956 >> 172.21.15.31:0/2821883956 conn(0x7f8c7006d170 msgr2=0x7f8c700cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:02.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:02.720+0000 7f8c7559d700 1 -- 172.21.15.31:0/2821883956 shutdown_connections 2023-10-06T02:29:02.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:02.720+0000 7f8c7559d700 1 -- 172.21.15.31:0/2821883956 wait complete. 2023-10-06T02:29:02.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:02.721+0000 7f8c7559d700 1 Processor -- start 2023-10-06T02:29:02.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:02.721+0000 7f8c7559d700 1 -- start start 2023-10-06T02:29:02.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:02.722+0000 7f8c7559d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c7006de50 0x7f8c700a5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:02.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:02.722+0000 7f8c7559d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c70140a70 0x7f8c70158510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:02.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:02.722+0000 7f8c7559d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8c70140e40 0x7f8c7013e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:02.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:02.722+0000 7f8c7559d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c701595a0 con 0x7f8c70140a70 2023-10-06T02:29:02.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:02.722+0000 7f8c7559d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c70158a50 con 0x7f8c7006de50 2023-10-06T02:29:02.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:02.722+0000 7f8c7559d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c7013e7b0 con 0x7f8c70140e40 2023-10-06T02:29:02.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:02.722+0000 7f8c6f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c70140a70 0x7f8c70158510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:02.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:02.723+0000 7f8c6f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c70140a70 0x7f8c70158510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47182/0 (socket says 172.21.15.31:47182) 2023-10-06T02:29:02.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:02.723+0000 7f8c6f7fe700 1 -- 172.21.15.31:0/2261753888 learned_addr learned my addr 172.21.15.31:0/2261753888 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:02.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:02.723+0000 7f8c6ffff700 1 --2- 172.21.15.31:0/2261753888 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c7006de50 0x7f8c700a5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:02.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:02.723+0000 7f8c74d9c700 1 --2- 172.21.15.31:0/2261753888 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8c70140e40 0x7f8c7013e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:02.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:02.723+0000 7f8c6f7fe700 1 -- 172.21.15.31:0/2261753888 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8c70140e40 msgr2=0x7f8c7013e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:02.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:02.723+0000 7f8c6f7fe700 1 --2- 172.21.15.31:0/2261753888 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8c70140e40 0x7f8c7013e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:02.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:02.723+0000 7f8c6f7fe700 1 -- 172.21.15.31:0/2261753888 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c7006de50 msgr2=0x7f8c700a5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:02.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:02.723+0000 7f8c6f7fe700 1 --2- 172.21.15.31:0/2261753888 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c7006de50 0x7f8c700a5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:02.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:02.723+0000 7f8c6f7fe700 1 -- 172.21.15.31:0/2261753888 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c68009ae0 con 0x7f8c70140a70 2023-10-06T02:29:02.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:02.723+0000 7f8c74d9c700 1 --2- 172.21.15.31:0/2261753888 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8c70140e40 0x7f8c7013e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:29:02.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:02.723+0000 7f8c6f7fe700 1 --2- 172.21.15.31:0/2261753888 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c70140a70 0x7f8c70158510 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f8c6400c510 tx=0x7f8c64001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:02.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:02.723+0000 7f8c57fff700 1 -- 172.21.15.31:0/2261753888 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c64011590 con 0x7f8c70140a70 2023-10-06T02:29:02.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:02.723+0000 7f8c57fff700 1 -- 172.21.15.31:0/2261753888 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c640116f0 con 0x7f8c70140a70 2023-10-06T02:29:02.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:02.723+0000 7f8c57fff700 1 -- 172.21.15.31:0/2261753888 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c64011ab0 con 0x7f8c70140a70 2023-10-06T02:29:02.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:02.723+0000 7f8c7559d700 1 -- 172.21.15.31:0/2261753888 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c700a6570 con 0x7f8c70140a70 2023-10-06T02:29:02.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:02.724+0000 7f8c7559d700 1 -- 172.21.15.31:0/2261753888 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c70158f50 con 0x7f8c70140a70 2023-10-06T02:29:02.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:02.725+0000 7f8c7559d700 1 -- 172.21.15.31:0/2261753888 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c7004c3c0 con 0x7f8c70140a70 2023-10-06T02:29:02.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:02.726+0000 7f8c57fff700 1 -- 172.21.15.31:0/2261753888 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f8c6401b3b0 con 0x7f8c70140a70 2023-10-06T02:29:02.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:02.727+0000 7f8c57fff700 1 --2- 172.21.15.31:0/2261753888 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f8c5c067200 0x7f8c5c0696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:02.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:02.727+0000 7f8c57fff700 1 -- 172.21.15.31:0/2261753888 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8c64083e80 con 0x7f8c70140a70 2023-10-06T02:29:02.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:02.727+0000 7f8c6ffff700 1 --2- 172.21.15.31:0/2261753888 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f8c5c067200 0x7f8c5c0696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:02.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:02.728+0000 7f8c6ffff700 1 --2- 172.21.15.31:0/2261753888 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f8c5c067200 0x7f8c5c0696b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8c60006f00 tx=0x7f8c60008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:02.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:02.732+0000 7f8c57fff700 1 -- 172.21.15.31:0/2261753888 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8c64050cb0 con 0x7f8c70140a70 2023-10-06T02:29:02.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:02.976+0000 7f8c7559d700 1 -- 172.21.15.31:0/2261753888 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "subvolume_0000000000011953", "snap_name": "fake", "target_sub_name": "clone_0000000000433052", "target": ["mon-mgr", ""]}) v1 -- 0x7f8c700cc620 con 0x7f8c5c067200 2023-10-06T02:29:03.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:03.055+0000 7f8c57fff700 1 -- 172.21.15.31:0/2261753888 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f8c700cc620 con 0x7f8c5c067200 2023-10-06T02:29:03.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:03.057+0000 7f8c7559d700 1 -- 172.21.15.31:0/2261753888 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f8c5c067200 msgr2=0x7f8c5c0696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:03.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:03.057+0000 7f8c7559d700 1 --2- 172.21.15.31:0/2261753888 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f8c5c067200 0x7f8c5c0696b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8c60006f00 tx=0x7f8c60008040 comp rx=0 tx=0).stop 2023-10-06T02:29:03.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:03.057+0000 7f8c7559d700 1 -- 172.21.15.31:0/2261753888 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c70140a70 msgr2=0x7f8c70158510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:03.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:03.057+0000 7f8c7559d700 1 --2- 172.21.15.31:0/2261753888 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c70140a70 0x7f8c70158510 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f8c6400c510 tx=0x7f8c64001070 comp rx=0 tx=0).stop 2023-10-06T02:29:03.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:03.057+0000 7f8c7559d700 1 -- 172.21.15.31:0/2261753888 shutdown_connections 2023-10-06T02:29:03.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:03.057+0000 7f8c7559d700 1 --2- 172.21.15.31:0/2261753888 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f8c5c067200 0x7f8c5c0696b0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:03.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:03.057+0000 7f8c7559d700 1 --2- 172.21.15.31:0/2261753888 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8c70140e40 0x7f8c7013e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:03.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:03.057+0000 7f8c7559d700 1 --2- 172.21.15.31:0/2261753888 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c7006de50 0x7f8c700a5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:03.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:03.057+0000 7f8c7559d700 1 --2- 172.21.15.31:0/2261753888 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c70140a70 0x7f8c70158510 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:03.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:03.057+0000 7f8c7559d700 1 -- 172.21.15.31:0/2261753888 >> 172.21.15.31:0/2261753888 conn(0x7f8c7006d170 msgr2=0x7f8c700c1a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:03.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:03.057+0000 7f8c7559d700 1 -- 172.21.15.31:0/2261753888 shutdown_connections 2023-10-06T02:29:03.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:03.057+0000 7f8c7559d700 1 -- 172.21.15.31:0/2261753888 wait complete. 2023-10-06T02:29:03.076 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs clone_0000000000433052 2023-10-06T02:29:03.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:03.266+0000 7efbf219d700 1 Processor -- start 2023-10-06T02:29:03.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:03.267+0000 7efbf219d700 1 -- start start 2023-10-06T02:29:03.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:03.268+0000 7efbf219d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efbec0c7be0 0x7efbec06abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:03.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:03.268+0000 7efbf219d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7efbec13f780 0x7efbec13fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:03.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:03.268+0000 7efbf219d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efbec06b290 con 0x7efbec0c7fb0 2023-10-06T02:29:03.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:03.268+0000 7efbf219d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efbec06bf10 con 0x7efbec13f780 2023-10-06T02:29:03.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:03.268+0000 7efbf219d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efbec140140 con 0x7efbec0c7be0 2023-10-06T02:29:03.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:03.268+0000 7efbf099a700 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7efbec0c7fb0 0x7efbec15a990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:36528/0 (socket says 172.21.15.31:36528) 2023-10-06T02:29:03.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:03.268+0000 7efbf099a700 1 -- 172.21.15.31:0/3195242047 learned_addr learned my addr 172.21.15.31:0/3195242047 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:29:03.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:03.268+0000 7efbf119b700 1 --2- 172.21.15.31:0/3195242047 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efbec0c7be0 0x7efbec06abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:03.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:03.269+0000 7efbf199c700 1 --2- 172.21.15.31:0/3195242047 >> v2:172.21.15.62:3300/0 conn(0x7efbec13f780 0x7efbec13fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:03.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:03.269+0000 7efbe3fff700 1 -- 172.21.15.31:0/3195242047 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2298907912 0 0) 0x7efbec06b290 con 0x7efbec0c7fb0 2023-10-06T02:29:03.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:03.269+0000 7efbf119b700 1 -- 172.21.15.31:0/3195242047 >> v2:172.21.15.62:3300/0 conn(0x7efbec13f780 msgr2=0x7efbec13fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:03.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:03.269+0000 7efbf119b700 1 --2- 172.21.15.31:0/3195242047 >> v2:172.21.15.62:3300/0 conn(0x7efbec13f780 0x7efbec13fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:03.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:03.269+0000 7efbf119b700 1 -- 172.21.15.31:0/3195242047 >> v1:172.21.15.62:6789/0 conn(0x7efbec0c7fb0 legacy=0x7efbec15a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:03.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:03.269+0000 7efbf119b700 1 -- 172.21.15.31:0/3195242047 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbec06b570 con 0x7efbec0c7be0 2023-10-06T02:29:03.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:03.269+0000 7efbf119b700 1 --2- 172.21.15.31:0/3195242047 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efbec0c7be0 0x7efbec06abd0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7efbe80098e0 tx=0x7efbe80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5236f6c7642cc719 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:03.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:03.270+0000 7efbe3fff700 1 -- 172.21.15.31:0/3195242047 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbe800b640 con 0x7efbec0c7be0 2023-10-06T02:29:03.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:03.270+0000 7efbe3fff700 1 -- 172.21.15.31:0/3195242047 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbe800b7a0 con 0x7efbec0c7be0 2023-10-06T02:29:03.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:03.270+0000 7efbe3fff700 1 -- 172.21.15.31:0/3195242047 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbe800bb40 con 0x7efbec0c7be0 2023-10-06T02:29:03.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:03.270+0000 7efbf219d700 1 -- 172.21.15.31:0/3195242047 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efbec0c7be0 msgr2=0x7efbec06abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:03.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:03.270+0000 7efbf219d700 1 --2- 172.21.15.31:0/3195242047 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efbec0c7be0 0x7efbec06abd0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7efbe80098e0 tx=0x7efbe80091a0 comp rx=0 tx=0).stop 2023-10-06T02:29:03.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:03.270+0000 7efbf219d700 1 -- 172.21.15.31:0/3195242047 shutdown_connections 2023-10-06T02:29:03.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:03.270+0000 7efbf219d700 1 --2- 172.21.15.31:0/3195242047 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efbec0c7be0 0x7efbec06abd0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:03.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:03.270+0000 7efbf219d700 1 --2- 172.21.15.31:0/3195242047 >> v2:172.21.15.62:3300/0 conn(0x7efbec13f780 0x7efbec13fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:03.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:03.270+0000 7efbf219d700 1 -- 172.21.15.31:0/3195242047 >> 172.21.15.31:0/3195242047 conn(0x7efbec06d110 msgr2=0x7efbec0cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:03.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:03.270+0000 7efbf219d700 1 -- 172.21.15.31:0/3195242047 shutdown_connections 2023-10-06T02:29:03.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:03.270+0000 7efbf219d700 1 -- 172.21.15.31:0/3195242047 wait complete. 2023-10-06T02:29:03.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:03.272+0000 7efbf219d700 1 Processor -- start 2023-10-06T02:29:03.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:03.272+0000 7efbf219d700 1 -- start start 2023-10-06T02:29:03.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:03.273+0000 7efbf219d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efbec0c7be0 0x7efbec13c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:03.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:03.273+0000 7efbf219d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efbec0c7fb0 0x7efbec15b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:03.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:03.273+0000 7efbf219d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efbec13f780 0x7efbec1558c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:03.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:03.273+0000 7efbf219d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbec0c1ec0 con 0x7efbec13f780 2023-10-06T02:29:03.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:03.273+0000 7efbf219d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbec0c12b0 con 0x7efbec0c7fb0 2023-10-06T02:29:03.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:03.273+0000 7efbf219d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efbec0c1590 con 0x7efbec0c7be0 2023-10-06T02:29:03.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:03.273+0000 7efbf199c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efbec13f780 0x7efbec1558c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:03.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:03.273+0000 7efbf199c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efbec13f780 0x7efbec1558c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47194/0 (socket says 172.21.15.31:47194) 2023-10-06T02:29:03.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:03.273+0000 7efbf199c700 1 -- 172.21.15.31:0/2845130977 learned_addr learned my addr 172.21.15.31:0/2845130977 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:03.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:03.273+0000 7efbf099a700 1 --2- 172.21.15.31:0/2845130977 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efbec0c7fb0 0x7efbec15b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:03.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:03.273+0000 7efbf119b700 1 --2- 172.21.15.31:0/2845130977 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efbec0c7be0 0x7efbec13c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:03.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:03.274+0000 7efbf199c700 1 -- 172.21.15.31:0/2845130977 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efbec0c7be0 msgr2=0x7efbec13c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:03.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:03.274+0000 7efbf199c700 1 --2- 172.21.15.31:0/2845130977 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efbec0c7be0 0x7efbec13c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:03.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:03.274+0000 7efbf199c700 1 -- 172.21.15.31:0/2845130977 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efbec0c7fb0 msgr2=0x7efbec15b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:03.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:03.274+0000 7efbf199c700 1 --2- 172.21.15.31:0/2845130977 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efbec0c7fb0 0x7efbec15b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:03.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:03.274+0000 7efbf199c700 1 -- 172.21.15.31:0/2845130977 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbe8009420 con 0x7efbec13f780 2023-10-06T02:29:03.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:03.274+0000 7efbf199c700 1 --2- 172.21.15.31:0/2845130977 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efbec13f780 0x7efbec1558c0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7efbe4009df0 tx=0x7efbe400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:03.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:03.274+0000 7efbe17fa700 1 -- 172.21.15.31:0/2845130977 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbe400caa0 con 0x7efbec13f780 2023-10-06T02:29:03.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:03.274+0000 7efbe17fa700 1 -- 172.21.15.31:0/2845130977 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbe400f040 con 0x7efbec13f780 2023-10-06T02:29:03.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:03.274+0000 7efbe17fa700 1 -- 172.21.15.31:0/2845130977 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbe400ce70 con 0x7efbec13f780 2023-10-06T02:29:03.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:03.274+0000 7efbf219d700 1 -- 172.21.15.31:0/2845130977 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efbec0c1870 con 0x7efbec13f780 2023-10-06T02:29:03.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:03.274+0000 7efbf219d700 1 -- 172.21.15.31:0/2845130977 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efbec0c0ed0 con 0x7efbec13f780 2023-10-06T02:29:03.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:03.275+0000 7efbf219d700 1 -- 172.21.15.31:0/2845130977 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efbec156930 con 0x7efbec13f780 2023-10-06T02:29:03.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:03.276+0000 7efbe17fa700 1 -- 172.21.15.31:0/2845130977 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 84995+0+0 (secure 0 0 0) 0x7efbe4014690 con 0x7efbec13f780 2023-10-06T02:29:03.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:03.277+0000 7efbe17fa700 1 --2- 172.21.15.31:0/2845130977 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7efbd0067200 0x7efbd00696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:03.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:03.277+0000 7efbe17fa700 1 -- 172.21.15.31:0/2845130977 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6058+0+0 (secure 0 0 0) 0x7efbe4085750 con 0x7efbec13f780 2023-10-06T02:29:03.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:03.278+0000 7efbf119b700 1 --2- 172.21.15.31:0/2845130977 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7efbd0067200 0x7efbd00696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:03.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:03.278+0000 7efbf119b700 1 --2- 172.21.15.31:0/2845130977 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7efbd0067200 0x7efbd00696b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7efbe80091e0 tx=0x7efbe8017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:03.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:03.281+0000 7efbe17fa700 1 -- 172.21.15.31:0/2845130977 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efbe4055e50 con 0x7efbec13f780 2023-10-06T02:29:03.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:03.530+0000 7efbf219d700 1 -- 172.21.15.31:0/2845130977 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "clone_0000000000433052", "target": ["mon-mgr", ""]}) v1 -- 0x7efbec04c3c0 con 0x7efbd0067200 2023-10-06T02:29:05.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:05.386+0000 7efbe17fa700 1 -- 172.21.15.31:0/2845130977 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+168 (secure 0 0 0) 0x7efbec04c3c0 con 0x7efbd0067200 2023-10-06T02:29:05.387 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T02:29:05.387 INFO:teuthology.orchestra.run.smithi031.stdout: "status": { 2023-10-06T02:29:05.387 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "pending", 2023-10-06T02:29:05.387 INFO:teuthology.orchestra.run.smithi031.stdout: "source": { 2023-10-06T02:29:05.388 INFO:teuthology.orchestra.run.smithi031.stdout: "volume": "cephfs", 2023-10-06T02:29:05.388 INFO:teuthology.orchestra.run.smithi031.stdout: "subvolume": "subvolume_0000000000011953", 2023-10-06T02:29:05.388 INFO:teuthology.orchestra.run.smithi031.stdout: "snapshot": "fake" 2023-10-06T02:29:05.388 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:29:05.388 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:29:05.388 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T02:29:05.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:05.389+0000 7efbf219d700 1 -- 172.21.15.31:0/2845130977 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7efbd0067200 msgr2=0x7efbd00696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:05.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:05.389+0000 7efbf219d700 1 --2- 172.21.15.31:0/2845130977 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7efbd0067200 0x7efbd00696b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7efbe80091e0 tx=0x7efbe8017000 comp rx=0 tx=0).stop 2023-10-06T02:29:05.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:05.389+0000 7efbf219d700 1 -- 172.21.15.31:0/2845130977 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efbec13f780 msgr2=0x7efbec1558c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:05.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:05.389+0000 7efbf219d700 1 --2- 172.21.15.31:0/2845130977 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efbec13f780 0x7efbec1558c0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7efbe4009df0 tx=0x7efbe400b040 comp rx=0 tx=0).stop 2023-10-06T02:29:05.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:05.389+0000 7efbf219d700 1 -- 172.21.15.31:0/2845130977 shutdown_connections 2023-10-06T02:29:05.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:05.389+0000 7efbf219d700 1 --2- 172.21.15.31:0/2845130977 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efbec13f780 0x7efbec1558c0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:05.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:05.389+0000 7efbf219d700 1 --2- 172.21.15.31:0/2845130977 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7efbd0067200 0x7efbd00696b0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:05.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:05.389+0000 7efbf219d700 1 --2- 172.21.15.31:0/2845130977 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efbec0c7be0 0x7efbec13c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:05.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:05.389+0000 7efbf219d700 1 --2- 172.21.15.31:0/2845130977 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efbec0c7fb0 0x7efbec15b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:05.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:05.389+0000 7efbf219d700 1 -- 172.21.15.31:0/2845130977 >> 172.21.15.31:0/2845130977 conn(0x7efbec06d110 msgr2=0x7efbec0b1530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:05.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:05.389+0000 7efbf219d700 1 -- 172.21.15.31:0/2845130977 shutdown_connections 2023-10-06T02:29:05.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:05.389+0000 7efbf219d700 1 -- 172.21.15.31:0/2845130977 wait complete. 2023-10-06T02:29:06.415 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs clone_0000000000433052 2023-10-06T02:29:06.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:06.618+0000 7f8af8d9d700 1 Processor -- start 2023-10-06T02:29:06.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:06.618+0000 7f8af8d9d700 1 -- start start 2023-10-06T02:29:06.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:06.619+0000 7f8af8d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8af40cf110 0x7f8af415de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:06.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:06.619+0000 7f8af8d9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8af415ac50 con 0x7f8af4153a00 2023-10-06T02:29:06.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:06.619+0000 7f8af8d9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8af415d010 con 0x7f8af4153dd0 2023-10-06T02:29:06.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:06.619+0000 7f8af8d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8af415af60 con 0x7f8af40cf110 2023-10-06T02:29:06.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:06.620+0000 7f8af37fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f8af4153a00 0x7f8af415a540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:44492/0 (socket says 172.21.15.31:44492) 2023-10-06T02:29:06.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:06.620+0000 7f8af37fe700 1 -- 172.21.15.31:0/2252776129 learned_addr learned my addr 172.21.15.31:0/2252776129 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:29:06.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:06.620+0000 7f8af3fff700 1 --2- 172.21.15.31:0/2252776129 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8af40cf110 0x7f8af415de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:06.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:06.620+0000 7f8af27fc700 1 -- 172.21.15.31:0/2252776129 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2867743136 0 0) 0x7f8af415ac50 con 0x7f8af4153a00 2023-10-06T02:29:06.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:06.620+0000 7f8af27fc700 1 -- 172.21.15.31:0/2252776129 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8adc003570 con 0x7f8af4153a00 2023-10-06T02:29:06.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:06.620+0000 7f8af27fc700 1 -- 172.21.15.31:0/2252776129 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2250301169 0 0) 0x7f8af415d010 con 0x7f8af4153dd0 2023-10-06T02:29:06.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:06.620+0000 7f8af27fc700 1 -- 172.21.15.31:0/2252776129 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8af415ac50 con 0x7f8af4153dd0 2023-10-06T02:29:06.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:06.620+0000 7f8af27fc700 1 -- 172.21.15.31:0/2252776129 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 439940367 0 0) 0x7f8adc003570 con 0x7f8af4153a00 2023-10-06T02:29:06.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:06.620+0000 7f8af27fc700 1 -- 172.21.15.31:0/2252776129 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8af40cf110 msgr2=0x7f8af415de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:06.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:06.620+0000 7f8af27fc700 1 --2- 172.21.15.31:0/2252776129 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8af40cf110 0x7f8af415de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:06.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:06.620+0000 7f8af27fc700 1 -- 172.21.15.31:0/2252776129 >> v1:172.21.15.62:6789/0 conn(0x7f8af4153dd0 legacy=0x7f8af40cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:06.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:06.621+0000 7f8af27fc700 1 -- 172.21.15.31:0/2252776129 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8af415b240 con 0x7f8af4153a00 2023-10-06T02:29:06.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:06.621+0000 7f8af27fc700 1 -- 172.21.15.31:0/2252776129 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7f8ae4001470 con 0x7f8af4153a00 2023-10-06T02:29:06.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:06.621+0000 7f8af27fc700 1 -- 172.21.15.31:0/2252776129 >> v1:172.21.15.31:6789/0 conn(0x7f8af4153a00 legacy=0x7f8af415a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:06.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:06.621+0000 7f8af27fc700 1 --2- 172.21.15.31:0/2252776129 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8adc003c20 0x7f8adc004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:06.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:06.621+0000 7f8af27fc700 1 --2- 172.21.15.31:0/2252776129 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8adc004810 0x7f8adc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:06.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:06.621+0000 7f8af27fc700 1 --2- 172.21.15.31:0/2252776129 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8adc0051c0 0x7f8adc00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:06.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:06.621+0000 7f8af27fc700 1 -- 172.21.15.31:0/2252776129 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af415af60 con 0x7f8adc004810 2023-10-06T02:29:06.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:06.621+0000 7f8af27fc700 1 -- 172.21.15.31:0/2252776129 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8adc00c150 con 0x7f8adc003c20 2023-10-06T02:29:06.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:06.621+0000 7f8af27fc700 1 -- 172.21.15.31:0/2252776129 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8adc00c310 con 0x7f8adc0051c0 2023-10-06T02:29:06.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:06.621+0000 7f8af3fff700 1 --2- 172.21.15.31:0/2252776129 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8af40cf110 0x7f8af415de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:29:06.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:06.621+0000 7f8af2ffd700 1 --2- 172.21.15.31:0/2252776129 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8adc004810 0x7f8adc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:06.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:06.621+0000 7f8af37fe700 1 --2- 172.21.15.31:0/2252776129 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8adc003c20 0x7f8adc004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:06.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:06.621+0000 7f8af3fff700 1 --2- 172.21.15.31:0/2252776129 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8adc0051c0 0x7f8adc00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:06.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:06.621+0000 7f8af2ffd700 1 -- 172.21.15.31:0/2252776129 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8adc0051c0 msgr2=0x7f8adc00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:06.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:06.621+0000 7f8af2ffd700 1 --2- 172.21.15.31:0/2252776129 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8adc0051c0 0x7f8adc00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:06.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:06.621+0000 7f8af2ffd700 1 -- 172.21.15.31:0/2252776129 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8adc003c20 msgr2=0x7f8adc004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:06.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:06.621+0000 7f8af2ffd700 1 --2- 172.21.15.31:0/2252776129 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8adc003c20 0x7f8adc004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:06.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:06.621+0000 7f8af2ffd700 1 -- 172.21.15.31:0/2252776129 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8adc003570 con 0x7f8adc004810 2023-10-06T02:29:06.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:06.622+0000 7f8af2ffd700 1 --2- 172.21.15.31:0/2252776129 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8adc004810 0x7f8adc004c80 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8001a10 tx=0x7f8ae8007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd6bd782b077c87b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:06.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:06.622+0000 7f8af27fc700 1 -- 172.21.15.31:0/2252776129 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae800c5d0 con 0x7f8adc004810 2023-10-06T02:29:06.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:06.622+0000 7f8af27fc700 1 -- 172.21.15.31:0/2252776129 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ae800c730 con 0x7f8adc004810 2023-10-06T02:29:06.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:06.622+0000 7f8af27fc700 1 -- 172.21.15.31:0/2252776129 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae800cad0 con 0x7f8adc004810 2023-10-06T02:29:06.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:06.622+0000 7f8af8d9d700 1 -- 172.21.15.31:0/2252776129 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8adc004810 msgr2=0x7f8adc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:06.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:06.622+0000 7f8af8d9d700 1 --2- 172.21.15.31:0/2252776129 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8adc004810 0x7f8adc004c80 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8001a10 tx=0x7f8ae8007d30 comp rx=0 tx=0).stop 2023-10-06T02:29:06.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:06.622+0000 7f8af3fff700 1 -- 172.21.15.31:0/2252776129 reap_dead start 2023-10-06T02:29:06.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:06.623+0000 7f8af8d9d700 1 -- 172.21.15.31:0/2252776129 shutdown_connections 2023-10-06T02:29:06.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:06.623+0000 7f8af8d9d700 1 -- 172.21.15.31:0/2252776129 >> 172.21.15.31:0/2252776129 conn(0x7f8af40703a0 msgr2=0x7f8af40707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:06.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:06.623+0000 7f8af8d9d700 1 -- 172.21.15.31:0/2252776129 shutdown_connections 2023-10-06T02:29:06.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:06.623+0000 7f8af8d9d700 1 -- 172.21.15.31:0/2252776129 wait complete. 2023-10-06T02:29:06.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:06.624+0000 7f8af8d9d700 1 Processor -- start 2023-10-06T02:29:06.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:06.624+0000 7f8af8d9d700 1 -- start start 2023-10-06T02:29:06.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:06.625+0000 7f8af8d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8adc004810 0x7f8af414e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:06.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:06.625+0000 7f8af8d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8af40cc2e0 0x7f8af40b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:06.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:06.625+0000 7f8af8d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8af40cc750 0x7f8af40b1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:06.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:06.625+0000 7f8af8d9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af40b27c0 con 0x7f8af40cc2e0 2023-10-06T02:29:06.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:06.625+0000 7f8af8d9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af40cb240 con 0x7f8af40cc750 2023-10-06T02:29:06.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:06.625+0000 7f8af8d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8af40cb520 con 0x7f8adc004810 2023-10-06T02:29:06.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:06.625+0000 7f8af2ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8af40cc2e0 0x7f8af40b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:06.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:06.625+0000 7f8af2ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8af40cc2e0 0x7f8af40b12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47204/0 (socket says 172.21.15.31:47204) 2023-10-06T02:29:06.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:06.625+0000 7f8af2ffd700 1 -- 172.21.15.31:0/1067487246 learned_addr learned my addr 172.21.15.31:0/1067487246 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:06.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:06.626+0000 7f8af37fe700 1 --2- 172.21.15.31:0/1067487246 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8adc004810 0x7f8af414e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:06.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:06.626+0000 7f8af3fff700 1 --2- 172.21.15.31:0/1067487246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8af40cc750 0x7f8af40b1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:06.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:06.626+0000 7f8af2ffd700 1 -- 172.21.15.31:0/1067487246 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8adc004810 msgr2=0x7f8af414e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:06.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:06.626+0000 7f8af2ffd700 1 --2- 172.21.15.31:0/1067487246 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8adc004810 0x7f8af414e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:06.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:06.626+0000 7f8af2ffd700 1 -- 172.21.15.31:0/1067487246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8af40cc750 msgr2=0x7f8af40b1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:06.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:06.626+0000 7f8af2ffd700 1 --2- 172.21.15.31:0/1067487246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8af40cc750 0x7f8af40b1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:06.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:06.626+0000 7f8af2ffd700 1 -- 172.21.15.31:0/1067487246 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8af40cb800 con 0x7f8af40cc2e0 2023-10-06T02:29:06.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:06.626+0000 7f8af3fff700 1 --2- 172.21.15.31:0/1067487246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8af40cc750 0x7f8af40b1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:29:06.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:06.626+0000 7f8af2ffd700 1 --2- 172.21.15.31:0/1067487246 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8af40cc2e0 0x7f8af40b12d0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f8adc00c120 tx=0x7f8ae8015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:06.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:06.626+0000 7f8adb7fe700 1 -- 172.21.15.31:0/1067487246 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae800cab0 con 0x7f8af40cc2e0 2023-10-06T02:29:06.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:06.626+0000 7f8adb7fe700 1 -- 172.21.15.31:0/1067487246 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ae800ce40 con 0x7f8af40cc2e0 2023-10-06T02:29:06.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:06.626+0000 7f8adb7fe700 1 -- 172.21.15.31:0/1067487246 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae801e650 con 0x7f8af40cc2e0 2023-10-06T02:29:06.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:06.626+0000 7f8af8d9d700 1 -- 172.21.15.31:0/1067487246 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8af40cbae0 con 0x7f8af40cc2e0 2023-10-06T02:29:06.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:06.626+0000 7f8af8d9d700 1 -- 172.21.15.31:0/1067487246 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8af414e670 con 0x7f8af40cc2e0 2023-10-06T02:29:06.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:06.628+0000 7f8af8d9d700 1 -- 172.21.15.31:0/1067487246 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8af404c3c0 con 0x7f8af40cc2e0 2023-10-06T02:29:06.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:06.628+0000 7f8adb7fe700 1 -- 172.21.15.31:0/1067487246 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f8ae8019070 con 0x7f8af40cc2e0 2023-10-06T02:29:06.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:06.629+0000 7f8adb7fe700 1 --2- 172.21.15.31:0/1067487246 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f8ae0067200 0x7f8ae00696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:06.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:06.629+0000 7f8adb7fe700 1 -- 172.21.15.31:0/1067487246 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8ae8086510 con 0x7f8af40cc2e0 2023-10-06T02:29:06.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:06.630+0000 7f8af37fe700 1 --2- 172.21.15.31:0/1067487246 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f8ae0067200 0x7f8ae00696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:06.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:06.630+0000 7f8af37fe700 1 --2- 172.21.15.31:0/1067487246 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f8ae0067200 0x7f8ae00696b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8ae4001bb0 tx=0x7f8ae4009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:06.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:06.634+0000 7f8adb7fe700 1 -- 172.21.15.31:0/1067487246 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8ae8053290 con 0x7f8af40cc2e0 2023-10-06T02:29:06.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:06.880+0000 7f8af8d9d700 1 -- 172.21.15.31:0/1067487246 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "clone_0000000000433052", "target": ["mon-mgr", ""]}) v1 -- 0x7f8af40cb690 con 0x7f8ae0067200 2023-10-06T02:29:10.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.376+0000 7f8adb7fe700 1 -- 172.21.15.31:0/1067487246 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+46 (secure 0 0 0) 0x7f8af40cb690 con 0x7f8ae0067200 2023-10-06T02:29:10.376 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T02:29:10.377 INFO:teuthology.orchestra.run.smithi031.stdout: "status": { 2023-10-06T02:29:10.377 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "complete" 2023-10-06T02:29:10.377 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:29:10.377 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T02:29:10.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.379+0000 7f8af8d9d700 1 -- 172.21.15.31:0/1067487246 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f8ae0067200 msgr2=0x7f8ae00696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:10.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.379+0000 7f8af8d9d700 1 --2- 172.21.15.31:0/1067487246 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f8ae0067200 0x7f8ae00696b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8ae4001bb0 tx=0x7f8ae4009040 comp rx=0 tx=0).stop 2023-10-06T02:29:10.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.379+0000 7f8af8d9d700 1 -- 172.21.15.31:0/1067487246 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8af40cc2e0 msgr2=0x7f8af40b12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:10.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.379+0000 7f8af8d9d700 1 --2- 172.21.15.31:0/1067487246 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8af40cc2e0 0x7f8af40b12d0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f8adc00c120 tx=0x7f8ae8015000 comp rx=0 tx=0).stop 2023-10-06T02:29:10.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.379+0000 7f8af8d9d700 1 -- 172.21.15.31:0/1067487246 shutdown_connections 2023-10-06T02:29:10.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.379+0000 7f8af8d9d700 1 --2- 172.21.15.31:0/1067487246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8af40cc750 0x7f8af40b1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:10.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.379+0000 7f8af8d9d700 1 --2- 172.21.15.31:0/1067487246 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f8ae0067200 0x7f8ae00696b0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:10.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.379+0000 7f8af8d9d700 1 --2- 172.21.15.31:0/1067487246 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8adc004810 0x7f8af414e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:10.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.379+0000 7f8af8d9d700 1 --2- 172.21.15.31:0/1067487246 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8af40cc2e0 0x7f8af40b12d0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:10.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.379+0000 7f8af8d9d700 1 -- 172.21.15.31:0/1067487246 >> 172.21.15.31:0/1067487246 conn(0x7f8af40703a0 msgr2=0x7f8af4150d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:10.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.379+0000 7f8af8d9d700 1 -- 172.21.15.31:0/1067487246 shutdown_connections 2023-10-06T02:29:10.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.379+0000 7f8af8d9d700 1 -- 172.21.15.31:0/1067487246 wait complete. 2023-10-06T02:29:10.400 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:29:10.400 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/clone_0000000000433052/.meta) 2023-10-06T02:29:10.466 INFO:teuthology.orchestra.run.smithi031.stdout:[GLOBAL] 2023-10-06T02:29:10.466 INFO:teuthology.orchestra.run.smithi031.stdout:version = 2 2023-10-06T02:29:10.466 INFO:teuthology.orchestra.run.smithi031.stdout:type = clone 2023-10-06T02:29:10.466 INFO:teuthology.orchestra.run.smithi031.stdout:path = /volumes/_nogroup/clone_0000000000433052/49901b1a-c11a-40ae-bf70-74e3d9c04914 2023-10-06T02:29:10.466 INFO:teuthology.orchestra.run.smithi031.stdout:state = complete 2023-10-06T02:29:10.466 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:29:10.469 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot info cephfs subvolume_0000000000011953 snapshot_0000000000915642 2023-10-06T02:29:10.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.662+0000 7f7f93fff700 1 Processor -- start 2023-10-06T02:29:10.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.662+0000 7f7f93fff700 1 -- start start 2023-10-06T02:29:10.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.663+0000 7f7f93fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7f94157380 0x7f7f941585f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:10.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.663+0000 7f7f93fff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7f94157750 0x7f7f940ae080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:10.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.663+0000 7f7f93fff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f940c8680 con 0x7f7f9415b1a0 2023-10-06T02:29:10.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.663+0000 7f7f93fff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f9406a7f0 con 0x7f7f94157750 2023-10-06T02:29:10.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.663+0000 7f7f93fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f940c7270 con 0x7f7f94157380 2023-10-06T02:29:10.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.663+0000 7f7f937fe700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7f94157750 0x7f7f940ae080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:10.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.663+0000 7f7f92ffd700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7f94157380 0x7f7f941585f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:10.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.664+0000 7f7f937fe700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7f94157750 0x7f7f940ae080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:45496/0 (socket says 172.21.15.31:45496) 2023-10-06T02:29:10.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.664+0000 7f7f937fe700 1 -- 172.21.15.31:0/703319071 learned_addr learned my addr 172.21.15.31:0/703319071 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:10.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.664+0000 7f7f937fe700 1 -- 172.21.15.31:0/703319071 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7f94157380 msgr2=0x7f7f941585f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:10.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.664+0000 7f7f937fe700 1 --2- 172.21.15.31:0/703319071 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7f94157380 0x7f7f941585f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:10.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.664+0000 7f7f937fe700 1 -- 172.21.15.31:0/703319071 >> v1:172.21.15.62:6789/0 conn(0x7f7f9415b1a0 legacy=0x7f7f940c6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:10.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.664+0000 7f7f937fe700 1 -- 172.21.15.31:0/703319071 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f94158b30 con 0x7f7f94157750 2023-10-06T02:29:10.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.664+0000 7f7f937fe700 1 --2- 172.21.15.31:0/703319071 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f94157750 0x7f7f940ae080 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c009f20 tx=0x7f7f8c009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c64d1cc348057543 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:10.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.665+0000 7f7f91ffb700 1 -- 172.21.15.31:0/703319071 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8c00e070 con 0x7f7f94157750 2023-10-06T02:29:10.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.665+0000 7f7f91ffb700 1 -- 172.21.15.31:0/703319071 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f8c00b9d0 con 0x7f7f94157750 2023-10-06T02:29:10.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.665+0000 7f7f91ffb700 1 -- 172.21.15.31:0/703319071 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8c0092e0 con 0x7f7f94157750 2023-10-06T02:29:10.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.665+0000 7f7f93fff700 1 -- 172.21.15.31:0/703319071 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f94157750 msgr2=0x7f7f940ae080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:10.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.665+0000 7f7f93fff700 1 --2- 172.21.15.31:0/703319071 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f94157750 0x7f7f940ae080 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c009f20 tx=0x7f7f8c009840 comp rx=0 tx=0).stop 2023-10-06T02:29:10.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.665+0000 7f7f93fff700 1 -- 172.21.15.31:0/703319071 shutdown_connections 2023-10-06T02:29:10.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.665+0000 7f7f93fff700 1 --2- 172.21.15.31:0/703319071 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7f94157380 0x7f7f941585f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:10.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.665+0000 7f7f93fff700 1 --2- 172.21.15.31:0/703319071 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f94157750 0x7f7f940ae080 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:10.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.665+0000 7f7f93fff700 1 -- 172.21.15.31:0/703319071 >> 172.21.15.31:0/703319071 conn(0x7f7f9406c6a0 msgr2=0x7f7f9406caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:10.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.665+0000 7f7f93fff700 1 -- 172.21.15.31:0/703319071 shutdown_connections 2023-10-06T02:29:10.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.665+0000 7f7f93fff700 1 -- 172.21.15.31:0/703319071 wait complete. 2023-10-06T02:29:10.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.667+0000 7f7f93fff700 1 Processor -- start 2023-10-06T02:29:10.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.667+0000 7f7f93fff700 1 -- start start 2023-10-06T02:29:10.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.668+0000 7f7f93fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7f94157380 0x7f7f940aa870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:10.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.668+0000 7f7f93fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f94157750 0x7f7f9415dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:10.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.668+0000 7f7f93fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7f9415b1a0 0x7f7f941469f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:10.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.668+0000 7f7f93fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f940ab410 con 0x7f7f94157380 2023-10-06T02:29:10.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.668+0000 7f7f93fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9415d420 con 0x7f7f94157750 2023-10-06T02:29:10.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.668+0000 7f7f93fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f940aae50 con 0x7f7f9415b1a0 2023-10-06T02:29:10.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.668+0000 7f7f92ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7f94157380 0x7f7f940aa870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:10.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.668+0000 7f7f92ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7f94157380 0x7f7f940aa870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44674/0 (socket says 172.21.15.31:44674) 2023-10-06T02:29:10.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.668+0000 7f7f92ffd700 1 -- 172.21.15.31:0/3587139389 learned_addr learned my addr 172.21.15.31:0/3587139389 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:10.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.668+0000 7f7f937fe700 1 --2- 172.21.15.31:0/3587139389 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7f9415b1a0 0x7f7f941469f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:10.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.668+0000 7f7f927fc700 1 --2- 172.21.15.31:0/3587139389 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f94157750 0x7f7f9415dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:10.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.668+0000 7f7f92ffd700 1 -- 172.21.15.31:0/3587139389 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7f9415b1a0 msgr2=0x7f7f941469f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:10.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.668+0000 7f7f92ffd700 1 --2- 172.21.15.31:0/3587139389 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7f9415b1a0 0x7f7f941469f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:10.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.668+0000 7f7f92ffd700 1 -- 172.21.15.31:0/3587139389 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f94157750 msgr2=0x7f7f9415dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:10.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.668+0000 7f7f92ffd700 1 --2- 172.21.15.31:0/3587139389 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f94157750 0x7f7f9415dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:10.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.669+0000 7f7f92ffd700 1 -- 172.21.15.31:0/3587139389 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f8c009ae0 con 0x7f7f94157380 2023-10-06T02:29:10.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.669+0000 7f7f92ffd700 1 --2- 172.21.15.31:0/3587139389 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7f94157380 0x7f7f940aa870 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f7f8400ec80 tx=0x7f7f840091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:10.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.669+0000 7f7f7affd700 1 -- 172.21.15.31:0/3587139389 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8400c9d0 con 0x7f7f94157380 2023-10-06T02:29:10.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.669+0000 7f7f7affd700 1 -- 172.21.15.31:0/3587139389 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f8400cb30 con 0x7f7f94157380 2023-10-06T02:29:10.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.669+0000 7f7f93fff700 1 -- 172.21.15.31:0/3587139389 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f940ab130 con 0x7f7f94157380 2023-10-06T02:29:10.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.669+0000 7f7f7affd700 1 -- 172.21.15.31:0/3587139389 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f84013470 con 0x7f7f94157380 2023-10-06T02:29:10.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.669+0000 7f7f93fff700 1 -- 172.21.15.31:0/3587139389 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f94158ee0 con 0x7f7f94157380 2023-10-06T02:29:10.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.670+0000 7f7f93fff700 1 -- 172.21.15.31:0/3587139389 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f9415d280 con 0x7f7f94157380 2023-10-06T02:29:10.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.671+0000 7f7f7affd700 1 -- 172.21.15.31:0/3587139389 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f7f84013820 con 0x7f7f94157380 2023-10-06T02:29:10.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.672+0000 7f7f7affd700 1 --2- 172.21.15.31:0/3587139389 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7f80067200 0x7f7f800696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:10.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.672+0000 7f7f7affd700 1 -- 172.21.15.31:0/3587139389 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f7f84084eb0 con 0x7f7f94157380 2023-10-06T02:29:10.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.672+0000 7f7f927fc700 1 --2- 172.21.15.31:0/3587139389 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7f80067200 0x7f7f800696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:10.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.673+0000 7f7f927fc700 1 --2- 172.21.15.31:0/3587139389 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7f80067200 0x7f7f800696b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7f940b2070 tx=0x7f7f88009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:10.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.676+0000 7f7f7affd700 1 -- 172.21.15.31:0/3587139389 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7f9415d280 con 0x7f7f94157380 2023-10-06T02:29:10.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.917+0000 7f7f93fff700 1 -- 172.21.15.31:0/3587139389 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot info", "vol_name": "cephfs", "sub_name": "subvolume_0000000000011953", "snap_name": "snapshot_0000000000915642", "target": ["mon-mgr", ""]}) v1 -- 0x7f7f9415da30 con 0x7f7f80067200 2023-10-06T02:29:10.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.940+0000 7f7f7affd700 1 -- 172.21.15.31:0/3587139389 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+115 (secure 0 0 0) 0x7f7f9415da30 con 0x7f7f80067200 2023-10-06T02:29:10.941 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T02:29:10.941 INFO:teuthology.orchestra.run.smithi031.stdout: "created_at": "2023-10-06 02:28:53.000399", 2023-10-06T02:29:10.941 INFO:teuthology.orchestra.run.smithi031.stdout: "data_pool": "cephfs_data", 2023-10-06T02:29:10.941 INFO:teuthology.orchestra.run.smithi031.stdout: "has_pending_clones": "no" 2023-10-06T02:29:10.942 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T02:29:10.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.943+0000 7f7f93fff700 1 -- 172.21.15.31:0/3587139389 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7f80067200 msgr2=0x7f7f800696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:10.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.943+0000 7f7f93fff700 1 --2- 172.21.15.31:0/3587139389 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7f80067200 0x7f7f800696b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7f940b2070 tx=0x7f7f88009040 comp rx=0 tx=0).stop 2023-10-06T02:29:10.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.943+0000 7f7f93fff700 1 -- 172.21.15.31:0/3587139389 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7f94157380 msgr2=0x7f7f940aa870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:10.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.943+0000 7f7f93fff700 1 --2- 172.21.15.31:0/3587139389 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7f94157380 0x7f7f940aa870 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f7f8400ec80 tx=0x7f7f840091a0 comp rx=0 tx=0).stop 2023-10-06T02:29:10.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.943+0000 7f7f93fff700 1 -- 172.21.15.31:0/3587139389 shutdown_connections 2023-10-06T02:29:10.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.943+0000 7f7f93fff700 1 --2- 172.21.15.31:0/3587139389 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7f80067200 0x7f7f800696b0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:10.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.943+0000 7f7f93fff700 1 --2- 172.21.15.31:0/3587139389 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7f9415b1a0 0x7f7f941469f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:10.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.943+0000 7f7f93fff700 1 --2- 172.21.15.31:0/3587139389 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7f94157380 0x7f7f940aa870 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:10.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.943+0000 7f7f93fff700 1 --2- 172.21.15.31:0/3587139389 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f94157750 0x7f7f9415dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:10.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.943+0000 7f7f93fff700 1 -- 172.21.15.31:0/3587139389 >> 172.21.15.31:0/3587139389 conn(0x7f7f9406c6a0 msgr2=0x7f7f9406e680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:10.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.943+0000 7f7f93fff700 1 -- 172.21.15.31:0/3587139389 shutdown_connections 2023-10-06T02:29:10.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:10.943+0000 7f7f93fff700 1 -- 172.21.15.31:0/3587139389 wait complete. 2023-10-06T02:29:10.961 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs subvolume_0000000000011953 2023-10-06T02:29:11.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.154+0000 7fcb20f9d700 1 Processor -- start 2023-10-06T02:29:11.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.154+0000 7fcb20f9d700 1 -- start start 2023-10-06T02:29:11.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.156+0000 7fcb20f9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fcb1c0cd330 0x7fcb1c0cbb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:11.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.156+0000 7fcb20f9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb1c15d570 con 0x7fcb1c0cce90 2023-10-06T02:29:11.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.156+0000 7fcb20f9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb1c15d850 con 0x7fcb1c159fb0 2023-10-06T02:29:11.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.156+0000 7fcb20f9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb1c15e180 con 0x7fcb1c0cd330 2023-10-06T02:29:11.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.156+0000 7fcb1bfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fcb1c0cd330 0x7fcb1c0cbb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:11.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.156+0000 7fcb1affd700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fcb1c0cce90 0x7fcb1c0cb440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:51654/0 (socket says 172.21.15.31:51654) 2023-10-06T02:29:11.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.156+0000 7fcb1affd700 1 -- 172.21.15.31:0/2054194527 learned_addr learned my addr 172.21.15.31:0/2054194527 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:29:11.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.156+0000 7fcb1bfff700 1 -- 172.21.15.31:0/2054194527 >> v1:172.21.15.62:6789/0 conn(0x7fcb1c159fb0 legacy=0x7fcb1c15a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:11.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.156+0000 7fcb1bfff700 1 -- 172.21.15.31:0/2054194527 >> v1:172.21.15.31:6789/0 conn(0x7fcb1c0cce90 legacy=0x7fcb1c0cb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:11.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.156+0000 7fcb1bfff700 1 -- 172.21.15.31:0/2054194527 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb1c15db30 con 0x7fcb1c0cd330 2023-10-06T02:29:11.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.157+0000 7fcb1bfff700 1 --2- 172.21.15.31:0/2054194527 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb1c0cd330 0x7fcb1c0cbb50 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fcb14009e60 tx=0x7fcb14009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ee3c2675a538688 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:11.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.157+0000 7fcb1a7fc700 1 -- 172.21.15.31:0/2054194527 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb1400e070 con 0x7fcb1c0cd330 2023-10-06T02:29:11.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.157+0000 7fcb1a7fc700 1 -- 172.21.15.31:0/2054194527 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb1400bc30 con 0x7fcb1c0cd330 2023-10-06T02:29:11.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.157+0000 7fcb1a7fc700 1 -- 172.21.15.31:0/2054194527 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb14012430 con 0x7fcb1c0cd330 2023-10-06T02:29:11.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.157+0000 7fcb20f9d700 1 -- 172.21.15.31:0/2054194527 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb1c0cd330 msgr2=0x7fcb1c0cbb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:11.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.157+0000 7fcb20f9d700 1 --2- 172.21.15.31:0/2054194527 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb1c0cd330 0x7fcb1c0cbb50 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fcb14009e60 tx=0x7fcb14009780 comp rx=0 tx=0).stop 2023-10-06T02:29:11.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.158+0000 7fcb20f9d700 1 -- 172.21.15.31:0/2054194527 shutdown_connections 2023-10-06T02:29:11.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.158+0000 7fcb20f9d700 1 --2- 172.21.15.31:0/2054194527 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb1c0cd330 0x7fcb1c0cbb50 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:11.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.158+0000 7fcb20f9d700 1 -- 172.21.15.31:0/2054194527 >> 172.21.15.31:0/2054194527 conn(0x7fcb1c0703a0 msgr2=0x7fcb1c0707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:11.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.158+0000 7fcb20f9d700 1 -- 172.21.15.31:0/2054194527 shutdown_connections 2023-10-06T02:29:11.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.158+0000 7fcb20f9d700 1 -- 172.21.15.31:0/2054194527 wait complete. 2023-10-06T02:29:11.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.159+0000 7fcb20f9d700 1 Processor -- start 2023-10-06T02:29:11.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.159+0000 7fcb20f9d700 1 -- start start 2023-10-06T02:29:11.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.160+0000 7fcb20f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcb1c0cce90 0x7fcb1c16ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:11.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.160+0000 7fcb20f9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb1c0cd330 0x7fcb1c16c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:11.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.160+0000 7fcb20f9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb1c159fb0 0x7fcb1c16cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:11.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.160+0000 7fcb20f9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb1c0b0540 con 0x7fcb1c0cd330 2023-10-06T02:29:11.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.160+0000 7fcb20f9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb1c0b0820 con 0x7fcb1c159fb0 2023-10-06T02:29:11.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.160+0000 7fcb20f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb1c0b0b00 con 0x7fcb1c0cce90 2023-10-06T02:29:11.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.160+0000 7fcb1affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb1c0cd330 0x7fcb1c16c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:11.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.161+0000 7fcb1affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb1c0cd330 0x7fcb1c16c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44696/0 (socket says 172.21.15.31:44696) 2023-10-06T02:29:11.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.161+0000 7fcb1affd700 1 -- 172.21.15.31:0/3965617116 learned_addr learned my addr 172.21.15.31:0/3965617116 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:11.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.161+0000 7fcb1bfff700 1 --2- 172.21.15.31:0/3965617116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb1c159fb0 0x7fcb1c16cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:11.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.161+0000 7fcb1b7fe700 1 --2- 172.21.15.31:0/3965617116 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcb1c0cce90 0x7fcb1c16ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:11.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.161+0000 7fcb1affd700 1 -- 172.21.15.31:0/3965617116 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcb1c0cce90 msgr2=0x7fcb1c16ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:11.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.161+0000 7fcb1affd700 1 --2- 172.21.15.31:0/3965617116 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcb1c0cce90 0x7fcb1c16ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:11.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.161+0000 7fcb1affd700 1 -- 172.21.15.31:0/3965617116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb1c159fb0 msgr2=0x7fcb1c16cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:11.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.161+0000 7fcb1affd700 1 --2- 172.21.15.31:0/3965617116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb1c159fb0 0x7fcb1c16cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:11.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.161+0000 7fcb1affd700 1 -- 172.21.15.31:0/3965617116 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb14009a20 con 0x7fcb1c0cd330 2023-10-06T02:29:11.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.161+0000 7fcb1affd700 1 --2- 172.21.15.31:0/3965617116 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb1c0cd330 0x7fcb1c16c660 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fcb1000c590 tx=0x7fcb10007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:11.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.161+0000 7fcb037fe700 1 -- 172.21.15.31:0/3965617116 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb10005b30 con 0x7fcb1c0cd330 2023-10-06T02:29:11.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.161+0000 7fcb037fe700 1 -- 172.21.15.31:0/3965617116 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb10005cb0 con 0x7fcb1c0cd330 2023-10-06T02:29:11.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.162+0000 7fcb037fe700 1 -- 172.21.15.31:0/3965617116 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb10007320 con 0x7fcb1c0cd330 2023-10-06T02:29:11.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.162+0000 7fcb20f9d700 1 -- 172.21.15.31:0/3965617116 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb1c0b0de0 con 0x7fcb1c0cd330 2023-10-06T02:29:11.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.162+0000 7fcb20f9d700 1 -- 172.21.15.31:0/3965617116 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb1c16b2b0 con 0x7fcb1c0cd330 2023-10-06T02:29:11.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.163+0000 7fcb20f9d700 1 -- 172.21.15.31:0/3965617116 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb1c0b1300 con 0x7fcb1c0cd330 2023-10-06T02:29:11.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.164+0000 7fcb037fe700 1 -- 172.21.15.31:0/3965617116 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 84995+0+0 (secure 0 0 0) 0x7fcb1001b3b0 con 0x7fcb1c0cd330 2023-10-06T02:29:11.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.165+0000 7fcb037fe700 1 --2- 172.21.15.31:0/3965617116 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fcb08067200 0x7fcb080696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:11.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.165+0000 7fcb037fe700 1 -- 172.21.15.31:0/3965617116 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fcb10083ea0 con 0x7fcb1c0cd330 2023-10-06T02:29:11.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.165+0000 7fcb1b7fe700 1 --2- 172.21.15.31:0/3965617116 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fcb08067200 0x7fcb080696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:11.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.166+0000 7fcb1b7fe700 1 --2- 172.21.15.31:0/3965617116 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fcb08067200 0x7fcb080696b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c0b22c0 tx=0x7fcb0c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:11.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.169+0000 7fcb037fe700 1 -- 172.21.15.31:0/3965617116 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcb1c0b1300 con 0x7fcb1c0cd330 2023-10-06T02:29:11.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.408+0000 7fcb20f9d700 1 -- 172.21.15.31:0/3965617116 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "subvolume_0000000000011953", "target": ["mon-mgr", ""]}) v1 -- 0x7fcb1c0b1300 con 0x7fcb08067200 2023-10-06T02:29:11.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.425+0000 7fcb037fe700 1 -- 172.21.15.31:0/3965617116 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+96 (secure 0 0 0) 0x7fcb1c0b1300 con 0x7fcb08067200 2023-10-06T02:29:11.426 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2023-10-06T02:29:11.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:29:11.426 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "fake" 2023-10-06T02:29:11.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:29:11.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:29:11.427 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snapshot_0000000000915642" 2023-10-06T02:29:11.427 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:29:11.427 INFO:teuthology.orchestra.run.smithi031.stdout:] 2023-10-06T02:29:11.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.428+0000 7fcb20f9d700 1 -- 172.21.15.31:0/3965617116 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fcb08067200 msgr2=0x7fcb080696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:11.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.428+0000 7fcb20f9d700 1 --2- 172.21.15.31:0/3965617116 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fcb08067200 0x7fcb080696b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c0b22c0 tx=0x7fcb0c009040 comp rx=0 tx=0).stop 2023-10-06T02:29:11.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.428+0000 7fcb20f9d700 1 -- 172.21.15.31:0/3965617116 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb1c0cd330 msgr2=0x7fcb1c16c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:11.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.428+0000 7fcb20f9d700 1 --2- 172.21.15.31:0/3965617116 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb1c0cd330 0x7fcb1c16c660 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fcb1000c590 tx=0x7fcb10007f90 comp rx=0 tx=0).stop 2023-10-06T02:29:11.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.428+0000 7fcb20f9d700 1 -- 172.21.15.31:0/3965617116 shutdown_connections 2023-10-06T02:29:11.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.428+0000 7fcb20f9d700 1 --2- 172.21.15.31:0/3965617116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb1c159fb0 0x7fcb1c16cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:11.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.428+0000 7fcb20f9d700 1 --2- 172.21.15.31:0/3965617116 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fcb08067200 0x7fcb080696b0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:11.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.428+0000 7fcb20f9d700 1 --2- 172.21.15.31:0/3965617116 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcb1c0cce90 0x7fcb1c16ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:11.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.428+0000 7fcb20f9d700 1 --2- 172.21.15.31:0/3965617116 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb1c0cd330 0x7fcb1c16c660 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:11.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.428+0000 7fcb20f9d700 1 -- 172.21.15.31:0/3965617116 >> 172.21.15.31:0/3965617116 conn(0x7fcb1c0703a0 msgr2=0x7fcb1c14dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:11.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.428+0000 7fcb20f9d700 1 -- 172.21.15.31:0/3965617116 shutdown_connections 2023-10-06T02:29:11.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.428+0000 7fcb20f9d700 1 -- 172.21.15.31:0/3965617116 wait complete. 2023-10-06T02:29:11.446 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm cephfs subvolume_0000000000011953 snapshot_0000000000915642 2023-10-06T02:29:11.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.637+0000 7f6d3d59d700 1 Processor -- start 2023-10-06T02:29:11.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.638+0000 7f6d3d59d700 1 -- start start 2023-10-06T02:29:11.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.639+0000 7f6d3d59d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6d380c2a90 0x7f6d38156aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:11.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.639+0000 7f6d3d59d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6d380c85d0 0x7f6d380cc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:11.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.639+0000 7f6d3d59d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d381579b0 con 0x7f6d380c8200 2023-10-06T02:29:11.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.639+0000 7f6d3d59d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d380c1390 con 0x7f6d380c85d0 2023-10-06T02:29:11.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.639+0000 7f6d3d59d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d381573f0 con 0x7f6d380c2a90 2023-10-06T02:29:11.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.639+0000 7f6d3cd9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6d380c85d0 0x7f6d380cc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:11.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.639+0000 7f6d3cd9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6d380c85d0 0x7f6d380cc2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44710/0 (socket says 172.21.15.31:44710) 2023-10-06T02:29:11.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.639+0000 7f6d3cd9c700 1 -- 172.21.15.31:0/3681563793 learned_addr learned my addr 172.21.15.31:0/3681563793 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:11.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.639+0000 7f6d377fe700 1 --2- 172.21.15.31:0/3681563793 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6d380c2a90 0x7f6d38156aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:11.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.639+0000 7f6d3cd9c700 1 -- 172.21.15.31:0/3681563793 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6d380c2a90 msgr2=0x7f6d38156aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:11.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.640+0000 7f6d3cd9c700 1 --2- 172.21.15.31:0/3681563793 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6d380c2a90 0x7f6d38156aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:11.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.640+0000 7f6d3cd9c700 1 -- 172.21.15.31:0/3681563793 >> v1:172.21.15.62:6789/0 conn(0x7f6d380c8200 legacy=0x7f6d380c1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:11.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.640+0000 7f6d3cd9c700 1 -- 172.21.15.31:0/3681563793 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d381576d0 con 0x7f6d380c85d0 2023-10-06T02:29:11.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.640+0000 7f6d3cd9c700 1 --2- 172.21.15.31:0/3681563793 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d380c85d0 0x7f6d380cc2f0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f6d30009f20 tx=0x7f6d30009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c92d843871ec5a0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:11.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.640+0000 7f6d36ffd700 1 -- 172.21.15.31:0/3681563793 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d3000e070 con 0x7f6d380c85d0 2023-10-06T02:29:11.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.640+0000 7f6d36ffd700 1 -- 172.21.15.31:0/3681563793 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d3000ba80 con 0x7f6d380c85d0 2023-10-06T02:29:11.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.640+0000 7f6d36ffd700 1 -- 172.21.15.31:0/3681563793 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d300092e0 con 0x7f6d380c85d0 2023-10-06T02:29:11.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.641+0000 7f6d3d59d700 1 -- 172.21.15.31:0/3681563793 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d380c85d0 msgr2=0x7f6d380cc2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:11.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.641+0000 7f6d3d59d700 1 --2- 172.21.15.31:0/3681563793 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d380c85d0 0x7f6d380cc2f0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f6d30009f20 tx=0x7f6d30009840 comp rx=0 tx=0).stop 2023-10-06T02:29:11.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.641+0000 7f6d3d59d700 1 -- 172.21.15.31:0/3681563793 shutdown_connections 2023-10-06T02:29:11.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.641+0000 7f6d3d59d700 1 --2- 172.21.15.31:0/3681563793 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d380c85d0 0x7f6d380cc2f0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:11.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.641+0000 7f6d3d59d700 1 --2- 172.21.15.31:0/3681563793 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6d380c2a90 0x7f6d38156aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:11.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.641+0000 7f6d3d59d700 1 -- 172.21.15.31:0/3681563793 >> 172.21.15.31:0/3681563793 conn(0x7f6d3806c6a0 msgr2=0x7f6d3806caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:11.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.641+0000 7f6d3d59d700 1 -- 172.21.15.31:0/3681563793 shutdown_connections 2023-10-06T02:29:11.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.641+0000 7f6d3d59d700 1 -- 172.21.15.31:0/3681563793 wait complete. 2023-10-06T02:29:11.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.642+0000 7f6d3d59d700 1 Processor -- start 2023-10-06T02:29:11.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.642+0000 7f6d3d59d700 1 -- start start 2023-10-06T02:29:11.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.643+0000 7f6d3d59d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6d380c2a90 0x7f6d380acc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:11.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.643+0000 7f6d3d59d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d380c8200 0x7f6d3816b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:11.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.643+0000 7f6d3d59d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d380c85d0 0x7f6d380b0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:11.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.643+0000 7f6d3d59d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d381678e0 con 0x7f6d380c8200 2023-10-06T02:29:11.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.643+0000 7f6d3d59d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d38168150 con 0x7f6d380c85d0 2023-10-06T02:29:11.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.643+0000 7f6d3d59d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d380ad140 con 0x7f6d380c2a90 2023-10-06T02:29:11.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.644+0000 7f6d377fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d380c8200 0x7f6d3816b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:11.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.644+0000 7f6d377fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d380c8200 0x7f6d3816b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44724/0 (socket says 172.21.15.31:44724) 2023-10-06T02:29:11.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.644+0000 7f6d377fe700 1 -- 172.21.15.31:0/2080723384 learned_addr learned my addr 172.21.15.31:0/2080723384 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:11.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.644+0000 7f6d3cd9c700 1 --2- 172.21.15.31:0/2080723384 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d380c85d0 0x7f6d380b0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:11.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.644+0000 7f6d37fff700 1 --2- 172.21.15.31:0/2080723384 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6d380c2a90 0x7f6d380acc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:11.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.644+0000 7f6d377fe700 1 -- 172.21.15.31:0/2080723384 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6d380c2a90 msgr2=0x7f6d380acc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:11.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.644+0000 7f6d377fe700 1 --2- 172.21.15.31:0/2080723384 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6d380c2a90 0x7f6d380acc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:11.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.644+0000 7f6d377fe700 1 -- 172.21.15.31:0/2080723384 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d380c85d0 msgr2=0x7f6d380b0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:11.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.644+0000 7f6d377fe700 1 --2- 172.21.15.31:0/2080723384 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d380c85d0 0x7f6d380b0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:11.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.644+0000 7f6d377fe700 1 -- 172.21.15.31:0/2080723384 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d30009ae0 con 0x7f6d380c8200 2023-10-06T02:29:11.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.644+0000 7f6d377fe700 1 --2- 172.21.15.31:0/2080723384 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d380c8200 0x7f6d3816b7a0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c00d890 tx=0x7f6d2c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:11.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.645+0000 7f6d1ffff700 1 -- 172.21.15.31:0/2080723384 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d2c0125f0 con 0x7f6d380c8200 2023-10-06T02:29:11.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.645+0000 7f6d1ffff700 1 -- 172.21.15.31:0/2080723384 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d2c012770 con 0x7f6d380c8200 2023-10-06T02:29:11.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.645+0000 7f6d1ffff700 1 -- 172.21.15.31:0/2080723384 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d2c012b30 con 0x7f6d380c8200 2023-10-06T02:29:11.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.645+0000 7f6d3d59d700 1 -- 172.21.15.31:0/2080723384 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d380ad970 con 0x7f6d380c8200 2023-10-06T02:29:11.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.645+0000 7f6d3d59d700 1 -- 172.21.15.31:0/2080723384 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d38167de0 con 0x7f6d380c8200 2023-10-06T02:29:11.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.646+0000 7f6d3d59d700 1 -- 172.21.15.31:0/2080723384 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d381682e0 con 0x7f6d380c8200 2023-10-06T02:29:11.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.647+0000 7f6d1ffff700 1 -- 172.21.15.31:0/2080723384 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f6d2c01e460 con 0x7f6d380c8200 2023-10-06T02:29:11.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.648+0000 7f6d1ffff700 1 --2- 172.21.15.31:0/2080723384 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6d24067200 0x7f6d240696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:11.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.648+0000 7f6d1ffff700 1 -- 172.21.15.31:0/2080723384 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6d2c085140 con 0x7f6d380c8200 2023-10-06T02:29:11.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.648+0000 7f6d37fff700 1 --2- 172.21.15.31:0/2080723384 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6d24067200 0x7f6d240696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:11.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.649+0000 7f6d37fff700 1 --2- 172.21.15.31:0/2080723384 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6d24067200 0x7f6d240696b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6d380c7960 tx=0x7f6d28009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:11.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.652+0000 7f6d1ffff700 1 -- 172.21.15.31:0/2080723384 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6d2c051ec0 con 0x7f6d380c8200 2023-10-06T02:29:11.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.893+0000 7f6d3d59d700 1 -- 172.21.15.31:0/2080723384 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000011953", "snap_name": "snapshot_0000000000915642", "target": ["mon-mgr", ""]}) v1 -- 0x7f6d3804c3c0 con 0x7f6d24067200 2023-10-06T02:29:11.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.929+0000 7f6d1ffff700 1 -- 172.21.15.31:0/2080723384 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f6d3804c3c0 con 0x7f6d24067200 2023-10-06T02:29:11.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.931+0000 7f6d3d59d700 1 -- 172.21.15.31:0/2080723384 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6d24067200 msgr2=0x7f6d240696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:11.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.931+0000 7f6d3d59d700 1 --2- 172.21.15.31:0/2080723384 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6d24067200 0x7f6d240696b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6d380c7960 tx=0x7f6d28009040 comp rx=0 tx=0).stop 2023-10-06T02:29:11.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.931+0000 7f6d3d59d700 1 -- 172.21.15.31:0/2080723384 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d380c8200 msgr2=0x7f6d3816b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:11.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.931+0000 7f6d3d59d700 1 --2- 172.21.15.31:0/2080723384 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d380c8200 0x7f6d3816b7a0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c00d890 tx=0x7f6d2c008040 comp rx=0 tx=0).stop 2023-10-06T02:29:11.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.931+0000 7f6d3d59d700 1 -- 172.21.15.31:0/2080723384 shutdown_connections 2023-10-06T02:29:11.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.931+0000 7f6d3d59d700 1 --2- 172.21.15.31:0/2080723384 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d380c85d0 0x7f6d380b0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:11.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.931+0000 7f6d3d59d700 1 --2- 172.21.15.31:0/2080723384 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6d24067200 0x7f6d240696b0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:11.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.931+0000 7f6d3d59d700 1 --2- 172.21.15.31:0/2080723384 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6d380c2a90 0x7f6d380acc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:11.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.931+0000 7f6d3d59d700 1 --2- 172.21.15.31:0/2080723384 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d380c8200 0x7f6d3816b7a0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:11.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.931+0000 7f6d3d59d700 1 -- 172.21.15.31:0/2080723384 >> 172.21.15.31:0/2080723384 conn(0x7f6d3806c6a0 msgr2=0x7f6d38149b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:11.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.931+0000 7f6d3d59d700 1 -- 172.21.15.31:0/2080723384 shutdown_connections 2023-10-06T02:29:11.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:11.931+0000 7f6d3d59d700 1 -- 172.21.15.31:0/2080723384 wait complete. 2023-10-06T02:29:11.950 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm cephfs subvolume_0000000000011953 fake 2023-10-06T02:29:12.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:12.142+0000 7f4dddb9d700 1 Processor -- start 2023-10-06T02:29:12.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:12.142+0000 7f4dddb9d700 1 -- start start 2023-10-06T02:29:12.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:12.143+0000 7f4dddb9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4dd8153ac0 0x7f4dd815a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:12.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:12.143+0000 7f4dddb9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4dd8153e90 0x7f4dd80ca5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:12.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:12.143+0000 7f4dddb9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4dd815abf0 con 0x7f4dd80703a0 2023-10-06T02:29:12.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:12.143+0000 7f4dddb9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4dd8070940 con 0x7f4dd8153e90 2023-10-06T02:29:12.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:12.143+0000 7f4dddb9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4dd806a330 con 0x7f4dd8153ac0 2023-10-06T02:29:12.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:12.143+0000 7f4dcffff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4dd8153e90 0x7f4dd80ca5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:12.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:12.144+0000 7f4dcffff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4dd8153e90 0x7f4dd80ca5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44726/0 (socket says 172.21.15.31:44726) 2023-10-06T02:29:12.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:12.144+0000 7f4dcffff700 1 -- 172.21.15.31:0/3230996673 learned_addr learned my addr 172.21.15.31:0/3230996673 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:12.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:12.144+0000 7f4ddcb9b700 1 --2- 172.21.15.31:0/3230996673 >> v2:172.21.15.62:3300/0 conn(0x7f4dd8153ac0 0x7f4dd815a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:12.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:12.144+0000 7f4dcffff700 1 -- 172.21.15.31:0/3230996673 >> v2:172.21.15.62:3300/0 conn(0x7f4dd8153ac0 msgr2=0x7f4dd815a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:12.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:12.144+0000 7f4dcffff700 1 --2- 172.21.15.31:0/3230996673 >> v2:172.21.15.62:3300/0 conn(0x7f4dd8153ac0 0x7f4dd815a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:12.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:12.144+0000 7f4dcffff700 1 -- 172.21.15.31:0/3230996673 >> v1:172.21.15.62:6789/0 conn(0x7f4dd80703a0 legacy=0x7f4dd815d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:12.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:12.144+0000 7f4dcffff700 1 -- 172.21.15.31:0/3230996673 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dd806a610 con 0x7f4dd8153e90 2023-10-06T02:29:12.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:12.144+0000 7f4dcffff700 1 --2- 172.21.15.31:0/3230996673 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4dd8153e90 0x7f4dd80ca5a0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f4dd80cc930 tx=0x7f4dc8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16fdc41600b24998 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:12.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:12.145+0000 7f4dcf7fe700 1 -- 172.21.15.31:0/3230996673 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc8009e70 con 0x7f4dd8153e90 2023-10-06T02:29:12.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:12.145+0000 7f4dcf7fe700 1 -- 172.21.15.31:0/3230996673 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dc800b860 con 0x7f4dd8153e90 2023-10-06T02:29:12.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:12.145+0000 7f4dcf7fe700 1 -- 172.21.15.31:0/3230996673 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc800bc00 con 0x7f4dd8153e90 2023-10-06T02:29:12.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:12.145+0000 7f4dddb9d700 1 -- 172.21.15.31:0/3230996673 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4dd8153e90 msgr2=0x7f4dd80ca5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:12.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:12.145+0000 7f4dddb9d700 1 --2- 172.21.15.31:0/3230996673 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4dd8153e90 0x7f4dd80ca5a0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f4dd80cc930 tx=0x7f4dc8009250 comp rx=0 tx=0).stop 2023-10-06T02:29:12.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:12.145+0000 7f4dddb9d700 1 -- 172.21.15.31:0/3230996673 shutdown_connections 2023-10-06T02:29:12.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:12.145+0000 7f4dddb9d700 1 --2- 172.21.15.31:0/3230996673 >> v2:172.21.15.62:3300/0 conn(0x7f4dd8153ac0 0x7f4dd815a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:12.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:12.145+0000 7f4dddb9d700 1 --2- 172.21.15.31:0/3230996673 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4dd8153e90 0x7f4dd80ca5a0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:12.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:12.145+0000 7f4dddb9d700 1 -- 172.21.15.31:0/3230996673 >> 172.21.15.31:0/3230996673 conn(0x7f4dd806e790 msgr2=0x7f4dd806eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:12.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:12.145+0000 7f4dddb9d700 1 -- 172.21.15.31:0/3230996673 shutdown_connections 2023-10-06T02:29:12.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:12.145+0000 7f4dddb9d700 1 -- 172.21.15.31:0/3230996673 wait complete. 2023-10-06T02:29:12.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:12.147+0000 7f4dddb9d700 1 Processor -- start 2023-10-06T02:29:12.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:12.147+0000 7f4dddb9d700 1 -- start start 2023-10-06T02:29:12.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:12.148+0000 7f4dddb9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4dd80703a0 0x7f4dd814f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:12.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:12.148+0000 7f4dddb9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4dd8153ac0 0x7f4dd80c9a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:12.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:12.148+0000 7f4dddb9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4dd8153e90 0x7f4dd8151fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:12.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:12.148+0000 7f4dddb9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd814fe90 con 0x7f4dd8153e90 2023-10-06T02:29:12.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:12.148+0000 7f4dddb9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd815b260 con 0x7f4dd80703a0 2023-10-06T02:29:12.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:12.148+0000 7f4dddb9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd815c690 con 0x7f4dd8153ac0 2023-10-06T02:29:12.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:12.148+0000 7f4ddd39c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4dd8153e90 0x7f4dd8151fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:12.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:12.148+0000 7f4ddd39c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4dd8153e90 0x7f4dd8151fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44742/0 (socket says 172.21.15.31:44742) 2023-10-06T02:29:12.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:12.148+0000 7f4ddd39c700 1 -- 172.21.15.31:0/3820729006 learned_addr learned my addr 172.21.15.31:0/3820729006 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:12.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:12.148+0000 7f4ddcb9b700 1 --2- 172.21.15.31:0/3820729006 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4dd80703a0 0x7f4dd814f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:12.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:12.148+0000 7f4dcffff700 1 --2- 172.21.15.31:0/3820729006 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4dd8153ac0 0x7f4dd80c9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:12.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:12.148+0000 7f4ddd39c700 1 -- 172.21.15.31:0/3820729006 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4dd8153ac0 msgr2=0x7f4dd80c9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:12.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:12.148+0000 7f4ddd39c700 1 --2- 172.21.15.31:0/3820729006 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4dd8153ac0 0x7f4dd80c9a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:12.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:12.148+0000 7f4ddd39c700 1 -- 172.21.15.31:0/3820729006 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4dd80703a0 msgr2=0x7f4dd814f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:12.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:12.148+0000 7f4ddd39c700 1 --2- 172.21.15.31:0/3820729006 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4dd80703a0 0x7f4dd814f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:12.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:12.148+0000 7f4ddd39c700 1 -- 172.21.15.31:0/3820729006 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dc80094f0 con 0x7f4dd8153e90 2023-10-06T02:29:12.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:12.148+0000 7f4ddcb9b700 1 --2- 172.21.15.31:0/3820729006 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4dd80703a0 0x7f4dd814f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:29:12.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:12.149+0000 7f4ddd39c700 1 --2- 172.21.15.31:0/3820729006 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4dd8153e90 0x7f4dd8151fa0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f4dd000c8e0 tx=0x7f4dd0001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:12.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:12.149+0000 7f4dccff9700 1 -- 172.21.15.31:0/3820729006 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd000ce80 con 0x7f4dd8153e90 2023-10-06T02:29:12.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:12.149+0000 7f4dccff9700 1 -- 172.21.15.31:0/3820729006 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dd000a7e0 con 0x7f4dd8153e90 2023-10-06T02:29:12.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:12.149+0000 7f4dccff9700 1 -- 172.21.15.31:0/3820729006 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd000ab80 con 0x7f4dd8153e90 2023-10-06T02:29:12.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:12.149+0000 7f4dddb9d700 1 -- 172.21.15.31:0/3820729006 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dd816a670 con 0x7f4dd8153e90 2023-10-06T02:29:12.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:12.149+0000 7f4dddb9d700 1 -- 172.21.15.31:0/3820729006 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dd816acb0 con 0x7f4dd8153e90 2023-10-06T02:29:12.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:12.150+0000 7f4dddb9d700 1 -- 172.21.15.31:0/3820729006 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dd815c850 con 0x7f4dd8153e90 2023-10-06T02:29:12.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:12.151+0000 7f4dccff9700 1 -- 172.21.15.31:0/3820729006 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f4dd0012420 con 0x7f4dd8153e90 2023-10-06T02:29:12.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:12.152+0000 7f4dccff9700 1 --2- 172.21.15.31:0/3820729006 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f4dc4067200 0x7f4dc40696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:12.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:12.152+0000 7f4dccff9700 1 -- 172.21.15.31:0/3820729006 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f4dd0084e10 con 0x7f4dd8153e90 2023-10-06T02:29:12.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:12.152+0000 7f4ddcb9b700 1 --2- 172.21.15.31:0/3820729006 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f4dc4067200 0x7f4dc40696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:12.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:12.153+0000 7f4ddcb9b700 1 --2- 172.21.15.31:0/3820729006 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f4dc4067200 0x7f4dc40696b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4dd80aed70 tx=0x7f4dd4008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:12.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:12.156+0000 7f4dccff9700 1 -- 172.21.15.31:0/3820729006 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4dd0088030 con 0x7f4dd8153e90 2023-10-06T02:29:12.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:12.399+0000 7f4dddb9d700 1 -- 172.21.15.31:0/3820729006 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000011953", "snap_name": "fake", "target": ["mon-mgr", ""]}) v1 -- 0x7f4dd804c3c0 con 0x7f4dc4067200 2023-10-06T02:29:12.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:12.434+0000 7f4dccff9700 1 -- 172.21.15.31:0/3820729006 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f4dd804c3c0 con 0x7f4dc4067200 2023-10-06T02:29:12.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:12.436+0000 7f4dddb9d700 1 -- 172.21.15.31:0/3820729006 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f4dc4067200 msgr2=0x7f4dc40696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:12.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:12.436+0000 7f4dddb9d700 1 --2- 172.21.15.31:0/3820729006 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f4dc4067200 0x7f4dc40696b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4dd80aed70 tx=0x7f4dd4008040 comp rx=0 tx=0).stop 2023-10-06T02:29:12.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:12.436+0000 7f4dddb9d700 1 -- 172.21.15.31:0/3820729006 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4dd8153e90 msgr2=0x7f4dd8151fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:12.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:12.436+0000 7f4dddb9d700 1 --2- 172.21.15.31:0/3820729006 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4dd8153e90 0x7f4dd8151fa0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f4dd000c8e0 tx=0x7f4dd0001710 comp rx=0 tx=0).stop 2023-10-06T02:29:12.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:12.436+0000 7f4dddb9d700 1 -- 172.21.15.31:0/3820729006 shutdown_connections 2023-10-06T02:29:12.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:12.436+0000 7f4dddb9d700 1 --2- 172.21.15.31:0/3820729006 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4dd8153e90 0x7f4dd8151fa0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:12.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:12.436+0000 7f4dddb9d700 1 --2- 172.21.15.31:0/3820729006 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4dd80703a0 0x7f4dd814f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:12.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:12.436+0000 7f4dddb9d700 1 --2- 172.21.15.31:0/3820729006 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f4dc4067200 0x7f4dc40696b0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:12.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:12.436+0000 7f4dddb9d700 1 --2- 172.21.15.31:0/3820729006 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4dd8153ac0 0x7f4dd80c9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:12.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:12.436+0000 7f4dddb9d700 1 -- 172.21.15.31:0/3820729006 >> 172.21.15.31:0/3820729006 conn(0x7f4dd806e790 msgr2=0x7f4dd80c2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:12.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:12.436+0000 7f4dddb9d700 1 -- 172.21.15.31:0/3820729006 shutdown_connections 2023-10-06T02:29:12.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:12.436+0000 7f4dddb9d700 1 -- 172.21.15.31:0/3820729006 wait complete. 2023-10-06T02:29:12.454 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:29:12.454 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvolume_0000000000011953/.meta) 2023-10-06T02:29:15.371 INFO:teuthology.orchestra.run.smithi031.stdout:[GLOBAL] 2023-10-06T02:29:15.371 INFO:teuthology.orchestra.run.smithi031.stdout:version = 1 2023-10-06T02:29:15.371 INFO:teuthology.orchestra.run.smithi031.stdout:type = subvolume 2023-10-06T02:29:15.371 INFO:teuthology.orchestra.run.smithi031.stdout:path = /volumes/_nogroup/subvolume_0000000000011953/5172ced3-24b3-4d86-a22c-dcfea079edc7 2023-10-06T02:29:15.371 INFO:teuthology.orchestra.run.smithi031.stdout:state = complete 2023-10-06T02:29:15.371 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:29:15.375 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000011953 2023-10-06T02:29:15.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:15.575+0000 7fca60d9d700 1 Processor -- start 2023-10-06T02:29:15.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:15.576+0000 7fca60d9d700 1 -- start start 2023-10-06T02:29:15.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:15.577+0000 7fca60d9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fca5c153c90 0x7fca5c15a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:15.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:15.577+0000 7fca60d9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca5c15ada0 con 0x7fca5c06db20 2023-10-06T02:29:15.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:15.577+0000 7fca60d9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca5c06ca80 con 0x7fca5c0caec0 2023-10-06T02:29:15.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:15.577+0000 7fca60d9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca5c06cd60 con 0x7fca5c153c90 2023-10-06T02:29:15.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:15.577+0000 7fca5b7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fca5c153c90 0x7fca5c15a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:15.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:15.577+0000 7fca5affd700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fca5c06db20 0x7fca5c0ca7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:51658/0 (socket says 172.21.15.31:51658) 2023-10-06T02:29:15.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:15.577+0000 7fca5affd700 1 -- 172.21.15.31:0/3630611611 learned_addr learned my addr 172.21.15.31:0/3630611611 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:29:15.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:15.578+0000 7fca5b7fe700 1 -- 172.21.15.31:0/3630611611 >> v1:172.21.15.62:6789/0 conn(0x7fca5c0caec0 legacy=0x7fca5c15afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:15.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:15.578+0000 7fca5a7fc700 1 -- 172.21.15.31:0/3630611611 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3473707782 0 0) 0x7fca50001110 con 0x7fca5c06db20 2023-10-06T02:29:15.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:15.578+0000 7fca5b7fe700 1 -- 172.21.15.31:0/3630611611 >> v1:172.21.15.31:6789/0 conn(0x7fca5c06db20 legacy=0x7fca5c0ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:15.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:15.578+0000 7fca5b7fe700 1 -- 172.21.15.31:0/3630611611 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca5c06d040 con 0x7fca5c153c90 2023-10-06T02:29:15.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:15.578+0000 7fca5b7fe700 1 --2- 172.21.15.31:0/3630611611 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca5c153c90 0x7fca5c15a860 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fca4c0098e0 tx=0x7fca4c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c014e2bcb6135581 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:15.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:15.578+0000 7fca5a7fc700 1 -- 172.21.15.31:0/3630611611 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca4c009e80 con 0x7fca5c153c90 2023-10-06T02:29:15.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:15.578+0000 7fca5a7fc700 1 -- 172.21.15.31:0/3630611611 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca4c00b860 con 0x7fca5c153c90 2023-10-06T02:29:15.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:15.578+0000 7fca5a7fc700 1 -- 172.21.15.31:0/3630611611 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca4c00bbe0 con 0x7fca5c153c90 2023-10-06T02:29:15.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:15.579+0000 7fca60d9d700 1 -- 172.21.15.31:0/3630611611 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca5c153c90 msgr2=0x7fca5c15a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:15.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:15.579+0000 7fca60d9d700 1 --2- 172.21.15.31:0/3630611611 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca5c153c90 0x7fca5c15a860 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fca4c0098e0 tx=0x7fca4c0091a0 comp rx=0 tx=0).stop 2023-10-06T02:29:15.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:15.579+0000 7fca60d9d700 1 -- 172.21.15.31:0/3630611611 shutdown_connections 2023-10-06T02:29:15.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:15.579+0000 7fca60d9d700 1 --2- 172.21.15.31:0/3630611611 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca5c153c90 0x7fca5c15a860 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:15.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:15.579+0000 7fca60d9d700 1 -- 172.21.15.31:0/3630611611 >> 172.21.15.31:0/3630611611 conn(0x7fca5c06e720 msgr2=0x7fca5c06eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:15.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:15.579+0000 7fca60d9d700 1 -- 172.21.15.31:0/3630611611 shutdown_connections 2023-10-06T02:29:15.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:15.579+0000 7fca60d9d700 1 -- 172.21.15.31:0/3630611611 wait complete. 2023-10-06T02:29:15.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:15.580+0000 7fca60d9d700 1 Processor -- start 2023-10-06T02:29:15.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:15.581+0000 7fca60d9d700 1 -- start start 2023-10-06T02:29:15.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:15.582+0000 7fca60d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca5c06db20 0x7fca5c0b0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:15.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:15.582+0000 7fca60d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca5c0caec0 0x7fca5c152400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:15.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:15.582+0000 7fca60d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca5c153c90 0x7fca5c0c5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:15.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:15.582+0000 7fca60d9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca5c0ac230 con 0x7fca5c06db20 2023-10-06T02:29:15.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:15.582+0000 7fca60d9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca5c0afc80 con 0x7fca5c0caec0 2023-10-06T02:29:15.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:15.582+0000 7fca60d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca5c0aff60 con 0x7fca5c153c90 2023-10-06T02:29:15.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:15.582+0000 7fca5b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca5c06db20 0x7fca5c0b0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:15.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:15.582+0000 7fca5b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca5c06db20 0x7fca5c0b0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44758/0 (socket says 172.21.15.31:44758) 2023-10-06T02:29:15.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:15.582+0000 7fca5b7fe700 1 -- 172.21.15.31:0/3352333018 learned_addr learned my addr 172.21.15.31:0/3352333018 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:15.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:15.582+0000 7fca5affd700 1 --2- 172.21.15.31:0/3352333018 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca5c0caec0 0x7fca5c152400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:15.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:15.582+0000 7fca5bfff700 1 --2- 172.21.15.31:0/3352333018 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca5c153c90 0x7fca5c0c5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:15.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:15.582+0000 7fca5b7fe700 1 -- 172.21.15.31:0/3352333018 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca5c153c90 msgr2=0x7fca5c0c5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:15.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:15.582+0000 7fca5b7fe700 1 --2- 172.21.15.31:0/3352333018 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca5c153c90 0x7fca5c0c5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:15.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:15.582+0000 7fca5b7fe700 1 -- 172.21.15.31:0/3352333018 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca5c0caec0 msgr2=0x7fca5c152400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:15.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:15.582+0000 7fca5b7fe700 1 --2- 172.21.15.31:0/3352333018 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca5c0caec0 0x7fca5c152400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:15.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:15.582+0000 7fca5b7fe700 1 -- 172.21.15.31:0/3352333018 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca4c009420 con 0x7fca5c06db20 2023-10-06T02:29:15.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:15.582+0000 7fca5bfff700 1 --2- 172.21.15.31:0/3352333018 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca5c153c90 0x7fca5c0c5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:29:15.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:15.582+0000 7fca5affd700 1 --2- 172.21.15.31:0/3352333018 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca5c0caec0 0x7fca5c152400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:29:15.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:15.583+0000 7fca5b7fe700 1 --2- 172.21.15.31:0/3352333018 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca5c06db20 0x7fca5c0b0990 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fca5c0c3820 tx=0x7fca4c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:15.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:15.583+0000 7fca437fe700 1 -- 172.21.15.31:0/3352333018 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca4c01f070 con 0x7fca5c06db20 2023-10-06T02:29:15.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:15.583+0000 7fca437fe700 1 -- 172.21.15.31:0/3352333018 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca4c00b9d0 con 0x7fca5c06db20 2023-10-06T02:29:15.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:15.583+0000 7fca60d9d700 1 -- 172.21.15.31:0/3352333018 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca5c0b0240 con 0x7fca5c06db20 2023-10-06T02:29:15.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:15.583+0000 7fca437fe700 1 -- 172.21.15.31:0/3352333018 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca4c025670 con 0x7fca5c06db20 2023-10-06T02:29:15.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:15.583+0000 7fca60d9d700 1 -- 172.21.15.31:0/3352333018 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca5c0b0ed0 con 0x7fca5c06db20 2023-10-06T02:29:15.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:15.584+0000 7fca60d9d700 1 -- 172.21.15.31:0/3352333018 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca5c0b0760 con 0x7fca5c06db20 2023-10-06T02:29:15.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:15.585+0000 7fca437fe700 1 -- 172.21.15.31:0/3352333018 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 84995+0+0 (secure 0 0 0) 0x7fca4c023070 con 0x7fca5c06db20 2023-10-06T02:29:15.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:15.586+0000 7fca437fe700 1 --2- 172.21.15.31:0/3352333018 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fca48067200 0x7fca480696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:15.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:15.586+0000 7fca437fe700 1 -- 172.21.15.31:0/3352333018 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fca4c086270 con 0x7fca5c06db20 2023-10-06T02:29:15.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:15.586+0000 7fca5affd700 1 --2- 172.21.15.31:0/3352333018 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fca48067200 0x7fca480696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:15.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:15.587+0000 7fca5affd700 1 --2- 172.21.15.31:0/3352333018 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fca48067200 0x7fca480696b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fca5c15be60 tx=0x7fca50009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:15.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:15.590+0000 7fca437fe700 1 -- 172.21.15.31:0/3352333018 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fca5c0b0760 con 0x7fca5c06db20 2023-10-06T02:29:15.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:15.821+0000 7fca60d9d700 1 -- 172.21.15.31:0/3352333018 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000011953", "target": ["mon-mgr", ""]}) v1 -- 0x7fca5c0b0760 con 0x7fca48067200 2023-10-06T02:29:15.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:15.872+0000 7fca437fe700 1 -- 172.21.15.31:0/3352333018 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fca5c0b0760 con 0x7fca48067200 2023-10-06T02:29:15.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:15.874+0000 7fca60d9d700 1 -- 172.21.15.31:0/3352333018 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fca48067200 msgr2=0x7fca480696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:15.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:15.874+0000 7fca60d9d700 1 --2- 172.21.15.31:0/3352333018 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fca48067200 0x7fca480696b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fca5c15be60 tx=0x7fca50009040 comp rx=0 tx=0).stop 2023-10-06T02:29:15.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:15.874+0000 7fca60d9d700 1 -- 172.21.15.31:0/3352333018 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca5c06db20 msgr2=0x7fca5c0b0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:15.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:15.874+0000 7fca60d9d700 1 --2- 172.21.15.31:0/3352333018 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca5c06db20 0x7fca5c0b0990 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fca5c0c3820 tx=0x7fca4c017000 comp rx=0 tx=0).stop 2023-10-06T02:29:15.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:15.874+0000 7fca60d9d700 1 -- 172.21.15.31:0/3352333018 shutdown_connections 2023-10-06T02:29:15.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:15.874+0000 7fca60d9d700 1 --2- 172.21.15.31:0/3352333018 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fca48067200 0x7fca480696b0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:15.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:15.874+0000 7fca60d9d700 1 --2- 172.21.15.31:0/3352333018 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca5c153c90 0x7fca5c0c5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:15.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:15.874+0000 7fca60d9d700 1 --2- 172.21.15.31:0/3352333018 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca5c06db20 0x7fca5c0b0990 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:15.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:15.874+0000 7fca60d9d700 1 --2- 172.21.15.31:0/3352333018 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca5c0caec0 0x7fca5c152400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:15.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:15.874+0000 7fca60d9d700 1 -- 172.21.15.31:0/3352333018 >> 172.21.15.31:0/3352333018 conn(0x7fca5c06e720 msgr2=0x7fca5c0c1770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:15.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:15.875+0000 7fca60d9d700 1 -- 172.21.15.31:0/3352333018 shutdown_connections 2023-10-06T02:29:15.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:15.875+0000 7fca60d9d700 1 -- 172.21.15.31:0/3352333018 wait complete. 2023-10-06T02:29:15.878 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:29:15.877+0000 7f6567b82700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:29:15.878 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:29:15.877+0000 7f6567b82700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:29:15.878 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:29:15.877+0000 7f6567b82700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:29:15.879 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:29:15.877+0000 7f6567b82700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:29:15.879 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:29:15.877+0000 7f6567b82700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:29:15.894 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs clone_0000000000433051 2023-10-06T02:29:15.897 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:29:15.897+0000 7f6567381700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:29:15.898 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:29:15.897+0000 7f6567381700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:29:15.898 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:29:15.897+0000 7f6567381700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:29:15.898 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:29:15.897+0000 7f6567381700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:29:15.898 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:29:15.897+0000 7f6567381700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:29:16.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.092+0000 7fa1f9b9d700 1 Processor -- start 2023-10-06T02:29:16.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.092+0000 7fa1f9b9d700 1 -- start start 2023-10-06T02:29:16.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.093+0000 7fa1f9b9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa1f4142c00 0x7fa1f406b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:16.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.094+0000 7fa1f9b9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa1f406bd20 0x7fa1f40c7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:16.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.094+0000 7fa1f9b9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1f40ca480 con 0x7fa1f4142fd0 2023-10-06T02:29:16.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.094+0000 7fa1f9b9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1f415e240 con 0x7fa1f4142c00 2023-10-06T02:29:16.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.094+0000 7fa1f9b9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f406b350 con 0x7fa1f406bd20 2023-10-06T02:29:16.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.094+0000 7fa1f8b9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa1f4142c00 0x7fa1f406b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:16.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.094+0000 7fa1f8b9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa1f4142c00 0x7fa1f406b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44772/0 (socket says 172.21.15.31:44772) 2023-10-06T02:29:16.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.094+0000 7fa1f8b9b700 1 -- 172.21.15.31:0/2793493984 learned_addr learned my addr 172.21.15.31:0/2793493984 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:16.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.094+0000 7fa1f3fff700 1 --2- 172.21.15.31:0/2793493984 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa1f406bd20 0x7fa1f40c7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:16.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.094+0000 7fa1f8b9b700 1 -- 172.21.15.31:0/2793493984 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa1f406bd20 msgr2=0x7fa1f40c7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:16.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.094+0000 7fa1f8b9b700 1 --2- 172.21.15.31:0/2793493984 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa1f406bd20 0x7fa1f40c7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:16.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.094+0000 7fa1f8b9b700 1 -- 172.21.15.31:0/2793493984 >> v1:172.21.15.62:6789/0 conn(0x7fa1f4142fd0 legacy=0x7fa1f406c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:16.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.094+0000 7fa1f8b9b700 1 -- 172.21.15.31:0/2793493984 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1f415df60 con 0x7fa1f4142c00 2023-10-06T02:29:16.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.095+0000 7fa1f8b9b700 1 --2- 172.21.15.31:0/2793493984 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1f4142c00 0x7fa1f406b7e0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fa1e80099b0 tx=0x7fa1e8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5638a4ff68cfcf65 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:16.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.095+0000 7fa1f37fe700 1 -- 172.21.15.31:0/2793493984 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e800b860 con 0x7fa1f4142c00 2023-10-06T02:29:16.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.095+0000 7fa1f37fe700 1 -- 172.21.15.31:0/2793493984 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1e800b9c0 con 0x7fa1f4142c00 2023-10-06T02:29:16.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.095+0000 7fa1f37fe700 1 -- 172.21.15.31:0/2793493984 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e800bd80 con 0x7fa1f4142c00 2023-10-06T02:29:16.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.095+0000 7fa1f9b9d700 1 -- 172.21.15.31:0/2793493984 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1f4142c00 msgr2=0x7fa1f406b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:16.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.095+0000 7fa1f9b9d700 1 --2- 172.21.15.31:0/2793493984 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1f4142c00 0x7fa1f406b7e0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fa1e80099b0 tx=0x7fa1e8009250 comp rx=0 tx=0).stop 2023-10-06T02:29:16.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.096+0000 7fa1f9b9d700 1 -- 172.21.15.31:0/2793493984 shutdown_connections 2023-10-06T02:29:16.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.096+0000 7fa1f9b9d700 1 --2- 172.21.15.31:0/2793493984 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1f4142c00 0x7fa1f406b7e0 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:16.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.096+0000 7fa1f9b9d700 1 --2- 172.21.15.31:0/2793493984 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa1f406bd20 0x7fa1f40c7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:16.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.096+0000 7fa1f9b9d700 1 -- 172.21.15.31:0/2793493984 >> 172.21.15.31:0/2793493984 conn(0x7fa1f40675a0 msgr2=0x7fa1f40679a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:16.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.096+0000 7fa1f9b9d700 1 -- 172.21.15.31:0/2793493984 shutdown_connections 2023-10-06T02:29:16.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.096+0000 7fa1f9b9d700 1 -- 172.21.15.31:0/2793493984 wait complete. 2023-10-06T02:29:16.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.097+0000 7fa1f9b9d700 1 Processor -- start 2023-10-06T02:29:16.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.097+0000 7fa1f9b9d700 1 -- start start 2023-10-06T02:29:16.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.098+0000 7fa1f9b9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1f406bd20 0x7fa1f415ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:16.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.098+0000 7fa1f9b9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa1f4142c00 0x7fa1f413d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:16.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.098+0000 7fa1f9b9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa1f4142fd0 0x7fa1f40c3250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:16.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.098+0000 7fa1f9b9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f413f020 con 0x7fa1f406bd20 2023-10-06T02:29:16.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.098+0000 7fa1f9b9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f415cc70 con 0x7fa1f4142c00 2023-10-06T02:29:16.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.098+0000 7fa1f9b9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f415b080 con 0x7fa1f4142fd0 2023-10-06T02:29:16.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.098+0000 7fa1f8b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1f406bd20 0x7fa1f415ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:16.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.098+0000 7fa1f8b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1f406bd20 0x7fa1f415ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44778/0 (socket says 172.21.15.31:44778) 2023-10-06T02:29:16.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.098+0000 7fa1f8b9b700 1 -- 172.21.15.31:0/2736224405 learned_addr learned my addr 172.21.15.31:0/2736224405 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:16.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.099+0000 7fa1f3fff700 1 --2- 172.21.15.31:0/2736224405 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa1f4142c00 0x7fa1f413d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:16.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.099+0000 7fa1f939c700 1 --2- 172.21.15.31:0/2736224405 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa1f4142fd0 0x7fa1f40c3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:16.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.099+0000 7fa1f8b9b700 1 -- 172.21.15.31:0/2736224405 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa1f4142fd0 msgr2=0x7fa1f40c3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:16.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.099+0000 7fa1f8b9b700 1 --2- 172.21.15.31:0/2736224405 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa1f4142fd0 0x7fa1f40c3250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:16.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.099+0000 7fa1f8b9b700 1 -- 172.21.15.31:0/2736224405 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa1f4142c00 msgr2=0x7fa1f413d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:16.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.099+0000 7fa1f8b9b700 1 --2- 172.21.15.31:0/2736224405 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa1f4142c00 0x7fa1f413d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:16.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.099+0000 7fa1f8b9b700 1 -- 172.21.15.31:0/2736224405 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1e80094f0 con 0x7fa1f406bd20 2023-10-06T02:29:16.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.099+0000 7fa1f3fff700 1 --2- 172.21.15.31:0/2736224405 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa1f4142c00 0x7fa1f413d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:29:16.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.099+0000 7fa1f8b9b700 1 --2- 172.21.15.31:0/2736224405 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1f406bd20 0x7fa1f415ab40 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fa1e800b3c0 tx=0x7fa1e8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:16.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.099+0000 7fa1f0ff9700 1 -- 172.21.15.31:0/2736224405 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e801f070 con 0x7fa1f406bd20 2023-10-06T02:29:16.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.099+0000 7fa1f0ff9700 1 -- 172.21.15.31:0/2736224405 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1e8006b10 con 0x7fa1f406bd20 2023-10-06T02:29:16.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.099+0000 7fa1f9b9d700 1 -- 172.21.15.31:0/2736224405 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1f413ea60 con 0x7fa1f406bd20 2023-10-06T02:29:16.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.099+0000 7fa1f0ff9700 1 -- 172.21.15.31:0/2736224405 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e801a650 con 0x7fa1f406bd20 2023-10-06T02:29:16.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.099+0000 7fa1f9b9d700 1 -- 172.21.15.31:0/2736224405 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1f415c900 con 0x7fa1f406bd20 2023-10-06T02:29:16.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.100+0000 7fa1f9b9d700 1 -- 172.21.15.31:0/2736224405 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1f404c3c0 con 0x7fa1f406bd20 2023-10-06T02:29:16.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.101+0000 7fa1f0ff9700 1 -- 172.21.15.31:0/2736224405 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 84995+0+0 (secure 0 0 0) 0x7fa1e801aa40 con 0x7fa1f406bd20 2023-10-06T02:29:16.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.102+0000 7fa1f0ff9700 1 --2- 172.21.15.31:0/2736224405 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa1d8067200 0x7fa1d80696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:16.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.102+0000 7fa1f0ff9700 1 -- 172.21.15.31:0/2736224405 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6114+0+0 (secure 0 0 0) 0x7fa1e8086050 con 0x7fa1f406bd20 2023-10-06T02:29:16.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.103+0000 7fa1f3fff700 1 --2- 172.21.15.31:0/2736224405 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa1d8067200 0x7fa1d80696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:16.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.103+0000 7fa1f3fff700 1 --2- 172.21.15.31:0/2736224405 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa1d8067200 0x7fa1d80696b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4171460 tx=0x7fa1e0008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:16.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.107+0000 7fa1f0ff9700 1 -- 172.21.15.31:0/2736224405 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa1e80b75a0 con 0x7fa1f406bd20 2023-10-06T02:29:16.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.347+0000 7fa1f9b9d700 1 -- 172.21.15.31:0/2736224405 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "clone_0000000000433051", "target": ["mon-mgr", ""]}) v1 -- 0x7fa1f406ad00 con 0x7fa1d8067200 2023-10-06T02:29:16.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.379+0000 7fa1f0ff9700 1 -- 172.21.15.31:0/2736224405 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fa1f406ad00 con 0x7fa1d8067200 2023-10-06T02:29:16.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.381+0000 7fa1f9b9d700 1 -- 172.21.15.31:0/2736224405 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa1d8067200 msgr2=0x7fa1d80696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:16.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.381+0000 7fa1f9b9d700 1 --2- 172.21.15.31:0/2736224405 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa1d8067200 0x7fa1d80696b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4171460 tx=0x7fa1e0008040 comp rx=0 tx=0).stop 2023-10-06T02:29:16.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.381+0000 7fa1f9b9d700 1 -- 172.21.15.31:0/2736224405 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1f406bd20 msgr2=0x7fa1f415ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:16.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.381+0000 7fa1f9b9d700 1 --2- 172.21.15.31:0/2736224405 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1f406bd20 0x7fa1f415ab40 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fa1e800b3c0 tx=0x7fa1e8017000 comp rx=0 tx=0).stop 2023-10-06T02:29:16.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.381+0000 7fa1f9b9d700 1 -- 172.21.15.31:0/2736224405 shutdown_connections 2023-10-06T02:29:16.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.381+0000 7fa1f9b9d700 1 --2- 172.21.15.31:0/2736224405 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa1d8067200 0x7fa1d80696b0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:16.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.381+0000 7fa1f9b9d700 1 --2- 172.21.15.31:0/2736224405 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa1f4142fd0 0x7fa1f40c3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:16.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.381+0000 7fa1f9b9d700 1 --2- 172.21.15.31:0/2736224405 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1f406bd20 0x7fa1f415ab40 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:16.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.381+0000 7fa1f9b9d700 1 --2- 172.21.15.31:0/2736224405 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa1f4142c00 0x7fa1f413d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:16.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.381+0000 7fa1f9b9d700 1 -- 172.21.15.31:0/2736224405 >> 172.21.15.31:0/2736224405 conn(0x7fa1f40675a0 msgr2=0x7fa1f413f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:16.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.382+0000 7fa1f9b9d700 1 -- 172.21.15.31:0/2736224405 shutdown_connections 2023-10-06T02:29:16.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.382+0000 7fa1f9b9d700 1 -- 172.21.15.31:0/2736224405 wait complete. 2023-10-06T02:29:16.402 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs clone_0000000000433052 2023-10-06T02:29:16.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.595+0000 7fafa9b9d700 1 Processor -- start 2023-10-06T02:29:16.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.596+0000 7fafa9b9d700 1 -- start start 2023-10-06T02:29:16.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.597+0000 7fafa9b9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fafa4153c90 0x7fafa415a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:16.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.597+0000 7fafa9b9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fafa40caec0 0x7fafa415afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:16.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.597+0000 7fafa9b9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafa415ada0 con 0x7fafa406db20 2023-10-06T02:29:16.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.597+0000 7fafa9b9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafa406ca80 con 0x7fafa4153c90 2023-10-06T02:29:16.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.597+0000 7fafa9b9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafa406cd60 con 0x7fafa40caec0 2023-10-06T02:29:16.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.597+0000 7fafa8b9b700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fafa4153c90 0x7fafa415a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:16.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.597+0000 7fafa3fff700 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fafa406db20 0x7fafa40ca7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:48914/0 (socket says 172.21.15.31:48914) 2023-10-06T02:29:16.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.598+0000 7fafa3fff700 1 -- 172.21.15.31:0/1456950805 learned_addr learned my addr 172.21.15.31:0/1456950805 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:29:16.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.598+0000 7fafa939c700 1 --2- 172.21.15.31:0/1456950805 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fafa40caec0 0x7fafa415afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:16.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.598+0000 7fafa37fe700 1 -- 172.21.15.31:0/1456950805 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 384310543 0 0) 0x7faf90001110 con 0x7fafa406db20 2023-10-06T02:29:16.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.598+0000 7fafa37fe700 1 -- 172.21.15.31:0/1456950805 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf94003570 con 0x7fafa406db20 2023-10-06T02:29:16.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.598+0000 7fafa939c700 1 -- 172.21.15.31:0/1456950805 >> v2:172.21.15.62:3300/0 conn(0x7fafa4153c90 msgr2=0x7fafa415a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:16.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.598+0000 7fafa939c700 1 --2- 172.21.15.31:0/1456950805 >> v2:172.21.15.62:3300/0 conn(0x7fafa4153c90 0x7fafa415a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:16.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.598+0000 7fafa939c700 1 -- 172.21.15.31:0/1456950805 >> v1:172.21.15.62:6789/0 conn(0x7fafa406db20 legacy=0x7fafa40ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:16.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.598+0000 7fafa939c700 1 -- 172.21.15.31:0/1456950805 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafa406d040 con 0x7fafa40caec0 2023-10-06T02:29:16.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.598+0000 7fafa8b9b700 1 --2- 172.21.15.31:0/1456950805 >> v2:172.21.15.62:3300/0 conn(0x7fafa4153c90 0x7fafa415a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:29:16.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.599+0000 7fafa939c700 1 --2- 172.21.15.31:0/1456950805 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fafa40caec0 0x7fafa415afb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7faf9c009e60 tx=0x7faf9c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab8a27dd48585a07 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:16.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.599+0000 7fafa37fe700 1 -- 172.21.15.31:0/1456950805 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf9c00e070 con 0x7fafa40caec0 2023-10-06T02:29:16.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.599+0000 7fafa37fe700 1 -- 172.21.15.31:0/1456950805 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf9c00bab0 con 0x7fafa40caec0 2023-10-06T02:29:16.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.599+0000 7fafa37fe700 1 -- 172.21.15.31:0/1456950805 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf9c00be70 con 0x7fafa40caec0 2023-10-06T02:29:16.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.599+0000 7fafa9b9d700 1 -- 172.21.15.31:0/1456950805 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fafa40caec0 msgr2=0x7fafa415afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:16.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.599+0000 7fafa9b9d700 1 --2- 172.21.15.31:0/1456950805 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fafa40caec0 0x7fafa415afb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7faf9c009e60 tx=0x7faf9c009780 comp rx=0 tx=0).stop 2023-10-06T02:29:16.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.599+0000 7fafa9b9d700 1 -- 172.21.15.31:0/1456950805 shutdown_connections 2023-10-06T02:29:16.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.599+0000 7fafa9b9d700 1 --2- 172.21.15.31:0/1456950805 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fafa40caec0 0x7fafa415afb0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:16.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.599+0000 7fafa9b9d700 1 --2- 172.21.15.31:0/1456950805 >> v2:172.21.15.62:3300/0 conn(0x7fafa4153c90 0x7fafa415a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:16.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.599+0000 7fafa9b9d700 1 -- 172.21.15.31:0/1456950805 >> 172.21.15.31:0/1456950805 conn(0x7fafa406e720 msgr2=0x7fafa406eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:16.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.600+0000 7fafa9b9d700 1 -- 172.21.15.31:0/1456950805 shutdown_connections 2023-10-06T02:29:16.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.600+0000 7fafa9b9d700 1 -- 172.21.15.31:0/1456950805 wait complete. 2023-10-06T02:29:16.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.601+0000 7fafa9b9d700 1 Processor -- start 2023-10-06T02:29:16.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.601+0000 7fafa9b9d700 1 -- start start 2023-10-06T02:29:16.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.602+0000 7fafa9b9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fafa406db20 0x7fafa4152000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:16.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.602+0000 7fafa9b9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fafa40caec0 0x7fafa40b0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:16.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.602+0000 7fafa9b9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fafa4153c90 0x7fafa40c8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:16.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.602+0000 7fafa9b9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafa40b1900 con 0x7fafa406db20 2023-10-06T02:29:16.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.602+0000 7fafa9b9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafa40b1be0 con 0x7fafa40caec0 2023-10-06T02:29:16.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.602+0000 7fafa9b9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafa40b1ec0 con 0x7fafa4153c90 2023-10-06T02:29:16.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.602+0000 7fafa8b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fafa406db20 0x7fafa4152000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:16.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.602+0000 7fafa8b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fafa406db20 0x7fafa4152000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44780/0 (socket says 172.21.15.31:44780) 2023-10-06T02:29:16.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.602+0000 7fafa8b9b700 1 -- 172.21.15.31:0/4081735259 learned_addr learned my addr 172.21.15.31:0/4081735259 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:16.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.602+0000 7fafa3fff700 1 --2- 172.21.15.31:0/4081735259 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fafa40caec0 0x7fafa40b0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:16.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.602+0000 7fafa939c700 1 --2- 172.21.15.31:0/4081735259 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fafa4153c90 0x7fafa40c8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:16.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.603+0000 7fafa8b9b700 1 -- 172.21.15.31:0/4081735259 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fafa4153c90 msgr2=0x7fafa40c8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:16.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.603+0000 7fafa8b9b700 1 --2- 172.21.15.31:0/4081735259 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fafa4153c90 0x7fafa40c8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:16.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.603+0000 7fafa8b9b700 1 -- 172.21.15.31:0/4081735259 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fafa40caec0 msgr2=0x7fafa40b0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:16.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.603+0000 7fafa8b9b700 1 --2- 172.21.15.31:0/4081735259 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fafa40caec0 0x7fafa40b0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:16.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.603+0000 7fafa8b9b700 1 -- 172.21.15.31:0/4081735259 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf98009420 con 0x7fafa406db20 2023-10-06T02:29:16.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.603+0000 7fafa939c700 1 --2- 172.21.15.31:0/4081735259 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fafa4153c90 0x7fafa40c8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:29:16.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.603+0000 7fafa3fff700 1 --2- 172.21.15.31:0/4081735259 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fafa40caec0 0x7fafa40b0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:29:16.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.603+0000 7fafa8b9b700 1 --2- 172.21.15.31:0/4081735259 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fafa406db20 0x7fafa4152000 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7faf9800ec80 tx=0x7faf980091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:16.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.603+0000 7fafa0ff9700 1 -- 172.21.15.31:0/4081735259 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf9800c9d0 con 0x7fafa406db20 2023-10-06T02:29:16.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.603+0000 7fafa0ff9700 1 -- 172.21.15.31:0/4081735259 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf9800cb30 con 0x7fafa406db20 2023-10-06T02:29:16.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.603+0000 7fafa0ff9700 1 -- 172.21.15.31:0/4081735259 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf9800ceb0 con 0x7fafa406db20 2023-10-06T02:29:16.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.603+0000 7fafa9b9d700 1 -- 172.21.15.31:0/4081735259 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf9c009a20 con 0x7fafa406db20 2023-10-06T02:29:16.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.603+0000 7fafa9b9d700 1 -- 172.21.15.31:0/4081735259 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafa40b2480 con 0x7fafa406db20 2023-10-06T02:29:16.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.604+0000 7fafa9b9d700 1 -- 172.21.15.31:0/4081735259 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafa404c3c0 con 0x7fafa406db20 2023-10-06T02:29:16.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.605+0000 7fafa0ff9700 1 -- 172.21.15.31:0/4081735259 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 84995+0+0 (secure 0 0 0) 0x7faf98013680 con 0x7fafa406db20 2023-10-06T02:29:16.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.606+0000 7fafa0ff9700 1 --2- 172.21.15.31:0/4081735259 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7faf88067200 0x7faf880696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:16.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.606+0000 7fafa0ff9700 1 -- 172.21.15.31:0/4081735259 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6114+0+0 (secure 0 0 0) 0x7faf98084f00 con 0x7fafa406db20 2023-10-06T02:29:16.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.606+0000 7fafa3fff700 1 --2- 172.21.15.31:0/4081735259 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7faf88067200 0x7faf880696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:16.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.607+0000 7fafa3fff700 1 --2- 172.21.15.31:0/4081735259 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7faf88067200 0x7faf880696b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fafa40c6f50 tx=0x7faf90009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:16.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.610+0000 7fafa0ff9700 1 -- 172.21.15.31:0/4081735259 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faf98051c40 con 0x7fafa406db20 2023-10-06T02:29:16.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.848+0000 7fafa9b9d700 1 -- 172.21.15.31:0/4081735259 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "clone_0000000000433052", "target": ["mon-mgr", ""]}) v1 -- 0x7fafa40b1d50 con 0x7faf88067200 2023-10-06T02:29:16.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.871+0000 7fafa0ff9700 1 -- 172.21.15.31:0/4081735259 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fafa40b1d50 con 0x7faf88067200 2023-10-06T02:29:16.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.873+0000 7fafa9b9d700 1 -- 172.21.15.31:0/4081735259 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7faf88067200 msgr2=0x7faf880696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:16.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.873+0000 7fafa9b9d700 1 --2- 172.21.15.31:0/4081735259 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7faf88067200 0x7faf880696b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fafa40c6f50 tx=0x7faf90009040 comp rx=0 tx=0).stop 2023-10-06T02:29:16.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.873+0000 7fafa9b9d700 1 -- 172.21.15.31:0/4081735259 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fafa406db20 msgr2=0x7fafa4152000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:16.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.873+0000 7fafa9b9d700 1 --2- 172.21.15.31:0/4081735259 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fafa406db20 0x7fafa4152000 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7faf9800ec80 tx=0x7faf980091a0 comp rx=0 tx=0).stop 2023-10-06T02:29:16.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.873+0000 7fafa9b9d700 1 -- 172.21.15.31:0/4081735259 shutdown_connections 2023-10-06T02:29:16.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.873+0000 7fafa9b9d700 1 --2- 172.21.15.31:0/4081735259 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7faf88067200 0x7faf880696b0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:16.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.873+0000 7fafa9b9d700 1 --2- 172.21.15.31:0/4081735259 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fafa4153c90 0x7fafa40c8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:16.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.873+0000 7fafa9b9d700 1 --2- 172.21.15.31:0/4081735259 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fafa406db20 0x7fafa4152000 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:16.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.873+0000 7fafa9b9d700 1 --2- 172.21.15.31:0/4081735259 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fafa40caec0 0x7fafa40b0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:16.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.873+0000 7fafa9b9d700 1 -- 172.21.15.31:0/4081735259 >> 172.21.15.31:0/4081735259 conn(0x7fafa406e720 msgr2=0x7fafa415af40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:16.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.873+0000 7fafa9b9d700 1 -- 172.21.15.31:0/4081735259 shutdown_connections 2023-10-06T02:29:16.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:16.874+0000 7fafa9b9d700 1 -- 172.21.15.31:0/4081735259 wait complete. 2023-10-06T02:29:16.893 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:29:16.894 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2023-10-06T02:29:16.966 INFO:teuthology.orchestra.run.smithi031.stdout:2 2023-10-06T02:29:16.967 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T02:29:16.967 DEBUG:teuthology.orchestra.run.smithi031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T02:29:17.045 DEBUG:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T02:29:17.136 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T02:29:17.136 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T02:29:17.136 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:29:17.136 DEBUG:teuthology.orchestra.run.smithi031:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T02:29:17.240 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi031.front.sepia.ceph.com 2023-10-06T02:29:17.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:29:17.241 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T02:29:17.292 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-06T02:29:17.292 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:29:17.292 DEBUG:teuthology.orchestra.run.smithi062:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-06T02:29:17.376 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi062.front.sepia.ceph.com 2023-10-06T02:29:17.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:29:17.377 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T02:29:17.413 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:29:17.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:17.599+0000 7f3ac179d700 1 Processor -- start 2023-10-06T02:29:17.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:17.599+0000 7f3ac179d700 1 -- start start 2023-10-06T02:29:17.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:17.600+0000 7f3ac179d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3abc13fb20 0x7f3abc15a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:17.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:17.600+0000 7f3ac179d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3abc13fef0 0x7f3abc0c7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:17.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:17.600+0000 7f3ac179d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3abc15ae70 con 0x7f3abc06aaa0 2023-10-06T02:29:17.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:17.600+0000 7f3ac179d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3abc06af60 con 0x7f3abc13fb20 2023-10-06T02:29:17.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:17.600+0000 7f3ac179d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3abc06b240 con 0x7f3abc13fef0 2023-10-06T02:29:17.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:17.600+0000 7f3abbfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3abc13fb20 0x7f3abc15a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:17.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:17.600+0000 7f3abbfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3abc13fb20 0x7f3abc15a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44790/0 (socket says 172.21.15.31:44790) 2023-10-06T02:29:17.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:17.600+0000 7f3abbfff700 1 -- 172.21.15.31:0/497510038 learned_addr learned my addr 172.21.15.31:0/497510038 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:17.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:17.601+0000 7f3abb7fe700 1 --2- 172.21.15.31:0/497510038 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3abc13fef0 0x7f3abc0c7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:17.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:17.601+0000 7f3abbfff700 1 -- 172.21.15.31:0/497510038 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3abc13fef0 msgr2=0x7f3abc0c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:17.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:17.601+0000 7f3abbfff700 1 --2- 172.21.15.31:0/497510038 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3abc13fef0 0x7f3abc0c7990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:17.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:17.601+0000 7f3abbfff700 1 -- 172.21.15.31:0/497510038 >> v1:172.21.15.62:6789/0 conn(0x7f3abc06aaa0 legacy=0x7f3abc0c6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:17.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:17.601+0000 7f3abbfff700 1 -- 172.21.15.31:0/497510038 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3abc06b520 con 0x7f3abc13fb20 2023-10-06T02:29:17.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:17.601+0000 7f3abbfff700 1 --2- 172.21.15.31:0/497510038 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3abc13fb20 0x7f3abc15a930 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f3aac0099b0 tx=0x7f3aac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=728007961ce5b10d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:17.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:17.602+0000 7f3abaffd700 1 -- 172.21.15.31:0/497510038 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aac00b860 con 0x7f3abc13fb20 2023-10-06T02:29:17.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:17.602+0000 7f3abaffd700 1 -- 172.21.15.31:0/497510038 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3aac00b9c0 con 0x7f3abc13fb20 2023-10-06T02:29:17.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:17.602+0000 7f3abaffd700 1 -- 172.21.15.31:0/497510038 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aac00bd60 con 0x7f3abc13fb20 2023-10-06T02:29:17.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:17.602+0000 7f3ac179d700 1 -- 172.21.15.31:0/497510038 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3abc13fb20 msgr2=0x7f3abc15a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:17.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:17.602+0000 7f3ac179d700 1 --2- 172.21.15.31:0/497510038 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3abc13fb20 0x7f3abc15a930 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f3aac0099b0 tx=0x7f3aac009250 comp rx=0 tx=0).stop 2023-10-06T02:29:17.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:17.602+0000 7f3ac179d700 1 -- 172.21.15.31:0/497510038 shutdown_connections 2023-10-06T02:29:17.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:17.602+0000 7f3ac179d700 1 --2- 172.21.15.31:0/497510038 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3abc13fb20 0x7f3abc15a930 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:17.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:17.602+0000 7f3ac179d700 1 --2- 172.21.15.31:0/497510038 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3abc13fef0 0x7f3abc0c7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:17.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:17.602+0000 7f3ac179d700 1 -- 172.21.15.31:0/497510038 >> 172.21.15.31:0/497510038 conn(0x7f3abc06d170 msgr2=0x7f3abc0cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:17.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:17.602+0000 7f3ac179d700 1 -- 172.21.15.31:0/497510038 shutdown_connections 2023-10-06T02:29:17.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:17.602+0000 7f3ac179d700 1 -- 172.21.15.31:0/497510038 wait complete. 2023-10-06T02:29:17.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:17.604+0000 7f3ac179d700 1 Processor -- start 2023-10-06T02:29:17.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:17.604+0000 7f3ac179d700 1 -- start start 2023-10-06T02:29:17.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:17.605+0000 7f3ac179d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3abc06aaa0 0x7f3abc0add70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:17.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:17.605+0000 7f3ac179d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3abc13fb20 0x7f3abc155c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:17.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:17.605+0000 7f3ac179d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3abc13fef0 0x7f3abc0c2400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:17.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:17.605+0000 7f3ac179d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3abc157b40 con 0x7f3abc06aaa0 2023-10-06T02:29:17.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:17.605+0000 7f3ac179d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3abc156f30 con 0x7f3abc13fb20 2023-10-06T02:29:17.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:17.605+0000 7f3ac179d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3abc157210 con 0x7f3abc13fef0 2023-10-06T02:29:17.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:17.605+0000 7f3abbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3abc06aaa0 0x7f3abc0add70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:17.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:17.605+0000 7f3abbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3abc06aaa0 0x7f3abc0add70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44796/0 (socket says 172.21.15.31:44796) 2023-10-06T02:29:17.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:17.605+0000 7f3abbfff700 1 -- 172.21.15.31:0/4251542420 learned_addr learned my addr 172.21.15.31:0/4251542420 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:17.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:17.605+0000 7f3ac0f9c700 1 --2- 172.21.15.31:0/4251542420 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3abc13fef0 0x7f3abc0c2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:17.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:17.605+0000 7f3abb7fe700 1 --2- 172.21.15.31:0/4251542420 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3abc13fb20 0x7f3abc155c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:17.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:17.605+0000 7f3abbfff700 1 -- 172.21.15.31:0/4251542420 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3abc13fef0 msgr2=0x7f3abc0c2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:17.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:17.605+0000 7f3abbfff700 1 --2- 172.21.15.31:0/4251542420 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3abc13fef0 0x7f3abc0c2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:17.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:17.605+0000 7f3abbfff700 1 -- 172.21.15.31:0/4251542420 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3abc13fb20 msgr2=0x7f3abc155c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:17.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:17.605+0000 7f3abbfff700 1 --2- 172.21.15.31:0/4251542420 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3abc13fb20 0x7f3abc155c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:17.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:17.605+0000 7f3abbfff700 1 -- 172.21.15.31:0/4251542420 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aac0094f0 con 0x7f3abc06aaa0 2023-10-06T02:29:17.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:17.605+0000 7f3abb7fe700 1 --2- 172.21.15.31:0/4251542420 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3abc13fb20 0x7f3abc155c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:29:17.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:17.605+0000 7f3ac0f9c700 1 --2- 172.21.15.31:0/4251542420 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3abc13fef0 0x7f3abc0c2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:29:17.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:17.606+0000 7f3abbfff700 1 --2- 172.21.15.31:0/4251542420 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3abc06aaa0 0x7f3abc0add70 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f3abc071320 tx=0x7f3aac017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:17.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:17.606+0000 7f3aa3fff700 1 -- 172.21.15.31:0/4251542420 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aac024440 con 0x7f3abc06aaa0 2023-10-06T02:29:17.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:17.606+0000 7f3aa3fff700 1 -- 172.21.15.31:0/4251542420 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3aac0245c0 con 0x7f3abc06aaa0 2023-10-06T02:29:17.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:17.606+0000 7f3aa3fff700 1 -- 172.21.15.31:0/4251542420 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aac024980 con 0x7f3abc06aaa0 2023-10-06T02:29:17.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:17.606+0000 7f3ac179d700 1 -- 172.21.15.31:0/4251542420 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3abc1574f0 con 0x7f3abc06aaa0 2023-10-06T02:29:17.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:17.606+0000 7f3ac179d700 1 -- 172.21.15.31:0/4251542420 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3abc1561c0 con 0x7f3abc06aaa0 2023-10-06T02:29:17.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:17.607+0000 7f3ac179d700 1 -- 172.21.15.31:0/4251542420 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3abc157e20 con 0x7f3abc06aaa0 2023-10-06T02:29:17.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:17.608+0000 7f3aa3fff700 1 -- 172.21.15.31:0/4251542420 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f3aac01a420 con 0x7f3abc06aaa0 2023-10-06T02:29:17.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:17.609+0000 7f3aa3fff700 1 --2- 172.21.15.31:0/4251542420 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f3aa8067200 0x7f3aa80696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:17.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:17.609+0000 7f3aa3fff700 1 -- 172.21.15.31:0/4251542420 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f3aac086940 con 0x7f3abc06aaa0 2023-10-06T02:29:17.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:17.609+0000 7f3abb7fe700 1 --2- 172.21.15.31:0/4251542420 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f3aa8067200 0x7f3aa80696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:17.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:17.610+0000 7f3abb7fe700 1 --2- 172.21.15.31:0/4251542420 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f3aa8067200 0x7f3aa80696b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3ab0006aa0 tx=0x7f3ab0008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:17.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:17.613+0000 7f3aa3fff700 1 -- 172.21.15.31:0/4251542420 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3aac053670 con 0x7f3abc06aaa0 2023-10-06T02:29:17.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:17.859+0000 7f3ac179d700 1 -- 172.21.15.31:0/4251542420 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3abc04c3c0 con 0x7f3abc06aaa0 2023-10-06T02:29:17.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:17.860+0000 7f3aa3fff700 1 -- 172.21.15.31:0/4251542420 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f3aac056c90 con 0x7f3abc06aaa0 2023-10-06T02:29:17.862 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:29:17.862 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":72,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6026,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/3801334948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3801334948},{"type":"v1","addr":"172.21.15.62:6834","nonce":3801334948}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":6724,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/1804582836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":1804582836},{"type":"v1","addr":"172.21.15.31:6835","nonce":1804582836}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":6736,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1428432734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":1428432734},{"type":"v1","addr":"172.21.15.62:6835","nonce":1428432734}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:28:35.598388+0000","modified":"2023-10-06T02:28:39.126355+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7065},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7065":{"gid":7065,"name":"a","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6834/603135594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":603135594},{"type":"v1","addr":"172.21.15.31:6834","nonce":603135594}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-06T02:29:17.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:17.862+0000 7f3ac179d700 1 -- 172.21.15.31:0/4251542420 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f3aa8067200 msgr2=0x7f3aa80696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:17.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:17.862+0000 7f3ac179d700 1 --2- 172.21.15.31:0/4251542420 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f3aa8067200 0x7f3aa80696b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3ab0006aa0 tx=0x7f3ab0008040 comp rx=0 tx=0).stop 2023-10-06T02:29:17.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:17.862+0000 7f3ac179d700 1 -- 172.21.15.31:0/4251542420 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3abc06aaa0 msgr2=0x7f3abc0add70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:17.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:17.862+0000 7f3ac179d700 1 --2- 172.21.15.31:0/4251542420 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3abc06aaa0 0x7f3abc0add70 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f3abc071320 tx=0x7f3aac017000 comp rx=0 tx=0).stop 2023-10-06T02:29:17.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:17.862+0000 7f3ac179d700 1 -- 172.21.15.31:0/4251542420 shutdown_connections 2023-10-06T02:29:17.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:17.862+0000 7f3ac179d700 1 --2- 172.21.15.31:0/4251542420 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f3aa8067200 0x7f3aa80696b0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:17.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:17.862+0000 7f3ac179d700 1 --2- 172.21.15.31:0/4251542420 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3abc13fef0 0x7f3abc0c2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:17.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:17.862+0000 7f3ac179d700 1 --2- 172.21.15.31:0/4251542420 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3abc06aaa0 0x7f3abc0add70 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:17.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:17.862+0000 7f3ac179d700 1 --2- 172.21.15.31:0/4251542420 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3abc13fb20 0x7f3abc155c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:17.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:17.862+0000 7f3ac179d700 1 -- 172.21.15.31:0/4251542420 >> 172.21.15.31:0/4251542420 conn(0x7f3abc06d170 msgr2=0x7f3abc13e810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:17.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:17.862+0000 7f3ac179d700 1 -- 172.21.15.31:0/4251542420 shutdown_connections 2023-10-06T02:29:17.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:17.862+0000 7f3ac179d700 1 -- 172.21.15.31:0/4251542420 wait complete. 2023-10-06T02:29:17.865 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 72 2023-10-06T02:29:17.881 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:29:18.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.069+0000 7f68eed9d700 1 Processor -- start 2023-10-06T02:29:18.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.069+0000 7f68eed9d700 1 -- start start 2023-10-06T02:29:18.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.071+0000 7f68eed9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f68e8153c30 0x7f68e815a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:18.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.071+0000 7f68eed9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f68e815de50 0x7f68e80cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:18.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.071+0000 7f68eed9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f68e80cef40 0x7f68e806d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:18.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.071+0000 7f68eed9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68e80cf3c0 con 0x7f68e8153c30 2023-10-06T02:29:18.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.071+0000 7f68eed9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68e815afd0 con 0x7f68e815de50 2023-10-06T02:29:18.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.071+0000 7f68eed9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68e815b2b0 con 0x7f68e80cef40 2023-10-06T02:29:18.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.071+0000 7f68edd9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f68e8153c30 0x7f68e815a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:18.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.071+0000 7f68edd9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f68e8153c30 0x7f68e815a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52682/0 (socket says 172.21.15.31:52682) 2023-10-06T02:29:18.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.071+0000 7f68edd9b700 1 -- 172.21.15.31:0/3780998017 learned_addr learned my addr 172.21.15.31:0/3780998017 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:18.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.071+0000 7f68ee59c700 1 --2- 172.21.15.31:0/3780998017 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f68e80cef40 0x7f68e806d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:18.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.071+0000 7f68ed59a700 1 --2- 172.21.15.31:0/3780998017 >> v2:172.21.15.62:3300/0 conn(0x7f68e815de50 0x7f68e80cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:18.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.072+0000 7f68edd9b700 1 -- 172.21.15.31:0/3780998017 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f68e80cef40 msgr2=0x7f68e806d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:18.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.072+0000 7f68edd9b700 1 --2- 172.21.15.31:0/3780998017 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f68e80cef40 0x7f68e806d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:18.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.072+0000 7f68edd9b700 1 -- 172.21.15.31:0/3780998017 >> v2:172.21.15.62:3300/0 conn(0x7f68e815de50 msgr2=0x7f68e80cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:18.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.072+0000 7f68edd9b700 1 --2- 172.21.15.31:0/3780998017 >> v2:172.21.15.62:3300/0 conn(0x7f68e815de50 0x7f68e80cea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:18.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.072+0000 7f68edd9b700 1 -- 172.21.15.31:0/3780998017 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68e815b590 con 0x7f68e8153c30 2023-10-06T02:29:18.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.072+0000 7f68edd9b700 1 --2- 172.21.15.31:0/3780998017 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68e8153c30 0x7f68e815a5e0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f68e80713a0 tx=0x7f68e4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfaf0523983c0c48 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:18.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.075+0000 7f68ecd99700 1 -- 172.21.15.31:0/3780998017 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e400b860 con 0x7f68e8153c30 2023-10-06T02:29:18.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.076+0000 7f68ecd99700 1 -- 172.21.15.31:0/3780998017 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68e400b9c0 con 0x7f68e8153c30 2023-10-06T02:29:18.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.076+0000 7f68ecd99700 1 -- 172.21.15.31:0/3780998017 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e400bd60 con 0x7f68e8153c30 2023-10-06T02:29:18.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.076+0000 7f68eed9d700 1 -- 172.21.15.31:0/3780998017 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68e8153c30 msgr2=0x7f68e815a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:18.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.076+0000 7f68eed9d700 1 --2- 172.21.15.31:0/3780998017 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68e8153c30 0x7f68e815a5e0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f68e80713a0 tx=0x7f68e4009250 comp rx=0 tx=0).stop 2023-10-06T02:29:18.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.076+0000 7f68eed9d700 1 -- 172.21.15.31:0/3780998017 shutdown_connections 2023-10-06T02:29:18.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.076+0000 7f68eed9d700 1 --2- 172.21.15.31:0/3780998017 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f68e80cef40 0x7f68e806d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:18.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.076+0000 7f68eed9d700 1 --2- 172.21.15.31:0/3780998017 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68e8153c30 0x7f68e815a5e0 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:18.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.076+0000 7f68eed9d700 1 --2- 172.21.15.31:0/3780998017 >> v2:172.21.15.62:3300/0 conn(0x7f68e815de50 0x7f68e80cea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:18.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.076+0000 7f68eed9d700 1 -- 172.21.15.31:0/3780998017 >> 172.21.15.31:0/3780998017 conn(0x7f68e80703a0 msgr2=0x7f68e80707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:18.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.077+0000 7f68eed9d700 1 -- 172.21.15.31:0/3780998017 shutdown_connections 2023-10-06T02:29:18.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.077+0000 7f68eed9d700 1 -- 172.21.15.31:0/3780998017 wait complete. 2023-10-06T02:29:18.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.078+0000 7f68eed9d700 1 Processor -- start 2023-10-06T02:29:18.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.079+0000 7f68eed9d700 1 -- start start 2023-10-06T02:29:18.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.080+0000 7f68eed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68e80cef40 0x7f68e8169e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:18.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.080+0000 7f68eed9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68e8153c30 0x7f68e814c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:18.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.080+0000 7f68eed9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f68e815de50 0x7f68e8151520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:18.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.080+0000 7f68eed9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e8150230 con 0x7f68e80cef40 2023-10-06T02:29:18.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.080+0000 7f68eed9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e8151090 con 0x7f68e8153c30 2023-10-06T02:29:18.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.080+0000 7f68eed9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68e816a360 con 0x7f68e815de50 2023-10-06T02:29:18.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.080+0000 7f68edd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68e80cef40 0x7f68e8169e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:18.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.080+0000 7f68edd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68e80cef40 0x7f68e8169e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52688/0 (socket says 172.21.15.31:52688) 2023-10-06T02:29:18.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.080+0000 7f68edd9b700 1 -- 172.21.15.31:0/1737795620 learned_addr learned my addr 172.21.15.31:0/1737795620 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:18.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.080+0000 7f68ee59c700 1 --2- 172.21.15.31:0/1737795620 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f68e815de50 0x7f68e8151520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:18.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.080+0000 7f68ed59a700 1 --2- 172.21.15.31:0/1737795620 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68e8153c30 0x7f68e814c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:18.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.080+0000 7f68edd9b700 1 -- 172.21.15.31:0/1737795620 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f68e815de50 msgr2=0x7f68e8151520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:18.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.080+0000 7f68edd9b700 1 --2- 172.21.15.31:0/1737795620 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f68e815de50 0x7f68e8151520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:18.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.080+0000 7f68edd9b700 1 -- 172.21.15.31:0/1737795620 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68e8153c30 msgr2=0x7f68e814c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:18.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.080+0000 7f68edd9b700 1 --2- 172.21.15.31:0/1737795620 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68e8153c30 0x7f68e814c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:18.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.080+0000 7f68edd9b700 1 -- 172.21.15.31:0/1737795620 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68e40094f0 con 0x7f68e80cef40 2023-10-06T02:29:18.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.080+0000 7f68ed59a700 1 --2- 172.21.15.31:0/1737795620 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68e8153c30 0x7f68e814c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:29:18.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.081+0000 7f68edd9b700 1 --2- 172.21.15.31:0/1737795620 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68e80cef40 0x7f68e8169e20 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f68e4006fd0 tx=0x7f68e4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:18.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.081+0000 7f68ddffb700 1 -- 172.21.15.31:0/1737795620 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e4011e30 con 0x7f68e80cef40 2023-10-06T02:29:18.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.081+0000 7f68ddffb700 1 -- 172.21.15.31:0/1737795620 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68e4024440 con 0x7f68e80cef40 2023-10-06T02:29:18.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.081+0000 7f68ddffb700 1 -- 172.21.15.31:0/1737795620 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e4024800 con 0x7f68e80cef40 2023-10-06T02:29:18.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.081+0000 7f68eed9d700 1 -- 172.21.15.31:0/1737795620 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68e8150480 con 0x7f68e80cef40 2023-10-06T02:29:18.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.081+0000 7f68eed9d700 1 -- 172.21.15.31:0/1737795620 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68e8150a40 con 0x7f68e80cef40 2023-10-06T02:29:18.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.082+0000 7f68eed9d700 1 -- 172.21.15.31:0/1737795620 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68e804c3c0 con 0x7f68e80cef40 2023-10-06T02:29:18.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.083+0000 7f68ddffb700 1 -- 172.21.15.31:0/1737795620 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f68e401a420 con 0x7f68e80cef40 2023-10-06T02:29:18.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.084+0000 7f68ddffb700 1 --2- 172.21.15.31:0/1737795620 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f68cc067320 0x7f68cc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:18.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.084+0000 7f68ddffb700 1 -- 172.21.15.31:0/1737795620 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f68e4086cf0 con 0x7f68e80cef40 2023-10-06T02:29:18.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.084+0000 7f68ed59a700 1 --2- 172.21.15.31:0/1737795620 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f68cc067320 0x7f68cc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:18.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.085+0000 7f68ed59a700 1 --2- 172.21.15.31:0/1737795620 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f68cc067320 0x7f68cc0697d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f68d8006fd0 tx=0x7f68d8008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:18.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.088+0000 7f68ddffb700 1 -- 172.21.15.31:0/1737795620 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f68e4053790 con 0x7f68e80cef40 2023-10-06T02:29:18.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.298+0000 7f68eed9d700 1 -- 172.21.15.31:0/1737795620 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f68e815b870 con 0x7f68e80cef40 2023-10-06T02:29:18.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.299+0000 7f68ddffb700 1 -- 172.21.15.31:0/1737795620 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) v1 ==== 74+0+16966 (secure 0 0 0) 0x7f68e4015090 con 0x7f68e80cef40 2023-10-06T02:29:18.301 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:29:18.301 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":108,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:29:17.059670+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:28:33.803742+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-10-06T02:28:34.570331+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":108,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":18,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":18,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:29:18.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.301+0000 7f68eed9d700 1 -- 172.21.15.31:0/1737795620 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f68cc067320 msgr2=0x7f68cc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:18.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.301+0000 7f68eed9d700 1 --2- 172.21.15.31:0/1737795620 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f68cc067320 0x7f68cc0697d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f68d8006fd0 tx=0x7f68d8008040 comp rx=0 tx=0).stop 2023-10-06T02:29:18.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.301+0000 7f68eed9d700 1 -- 172.21.15.31:0/1737795620 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68e80cef40 msgr2=0x7f68e8169e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:18.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.301+0000 7f68eed9d700 1 --2- 172.21.15.31:0/1737795620 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68e80cef40 0x7f68e8169e20 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f68e4006fd0 tx=0x7f68e4017000 comp rx=0 tx=0).stop 2023-10-06T02:29:18.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.301+0000 7f68eed9d700 1 -- 172.21.15.31:0/1737795620 shutdown_connections 2023-10-06T02:29:18.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.301+0000 7f68eed9d700 1 --2- 172.21.15.31:0/1737795620 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f68cc067320 0x7f68cc0697d0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:18.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.301+0000 7f68eed9d700 1 --2- 172.21.15.31:0/1737795620 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f68e815de50 0x7f68e8151520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:18.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.301+0000 7f68eed9d700 1 --2- 172.21.15.31:0/1737795620 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68e80cef40 0x7f68e8169e20 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:18.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.301+0000 7f68eed9d700 1 --2- 172.21.15.31:0/1737795620 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68e8153c30 0x7f68e814c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:18.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.301+0000 7f68eed9d700 1 -- 172.21.15.31:0/1737795620 >> 172.21.15.31:0/1737795620 conn(0x7f68e80703a0 msgr2=0x7f68e80b0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:18.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.302+0000 7f68eed9d700 1 -- 172.21.15.31:0/1737795620 shutdown_connections 2023-10-06T02:29:18.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.302+0000 7f68eed9d700 1 -- 172.21.15.31:0/1737795620 wait complete. 2023-10-06T02:29:18.321 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T02:22:53.183425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T02:23:00.046967+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T02:28:33.803742+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '105', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T02:28:34.570331+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '108', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 5, 'snap_epoch': 108, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T02:29:18.321 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T02:29:18.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.510+0000 7fc9a9d9d700 1 Processor -- start 2023-10-06T02:29:18.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.510+0000 7fc9a9d9d700 1 -- start start 2023-10-06T02:29:18.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.511+0000 7fc9a9d9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc9a415dfd0 0x7fc9a40aedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:18.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.511+0000 7fc9a9d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc9a4159920 0x7fc9a40ad090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:18.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.511+0000 7fc9a9d9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9a40acc00 con 0x7fc9a4159550 2023-10-06T02:29:18.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.511+0000 7fc9a9d9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9a40abff0 con 0x7fc9a415dfd0 2023-10-06T02:29:18.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.511+0000 7fc9a9d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a40ac2d0 con 0x7fc9a4159920 2023-10-06T02:29:18.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.512+0000 7fc9a8d9b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fc9a4159550 0x7fc9a40ceac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:32990/0 (socket says 172.21.15.31:32990) 2023-10-06T02:29:18.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.512+0000 7fc9a8d9b700 1 -- 172.21.15.31:0/3782740407 learned_addr learned my addr 172.21.15.31:0/3782740407 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:29:18.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.512+0000 7fc9a959c700 1 --2- 172.21.15.31:0/3782740407 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc9a4159920 0x7fc9a40ad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:18.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.512+0000 7fc9a3fff700 1 --2- 172.21.15.31:0/3782740407 >> v2:172.21.15.62:3300/0 conn(0x7fc9a415dfd0 0x7fc9a40aedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:18.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.512+0000 7fc9a37fe700 1 -- 172.21.15.31:0/3782740407 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2257942251 0 0) 0x7fc9a40acc00 con 0x7fc9a4159550 2023-10-06T02:29:18.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.512+0000 7fc9a37fe700 1 -- 172.21.15.31:0/3782740407 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc994003570 con 0x7fc9a4159550 2023-10-06T02:29:18.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.512+0000 7fc9a37fe700 1 -- 172.21.15.31:0/3782740407 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4262049572 0 0) 0x7fc994003570 con 0x7fc9a4159550 2023-10-06T02:29:18.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.512+0000 7fc9a37fe700 1 -- 172.21.15.31:0/3782740407 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc9a4159920 msgr2=0x7fc9a40ad090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:18.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.512+0000 7fc9a37fe700 1 --2- 172.21.15.31:0/3782740407 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc9a4159920 0x7fc9a40ad090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:18.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.512+0000 7fc9a37fe700 1 -- 172.21.15.31:0/3782740407 >> v2:172.21.15.62:3300/0 conn(0x7fc9a415dfd0 msgr2=0x7fc9a40aedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:18.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.512+0000 7fc9a37fe700 1 --2- 172.21.15.31:0/3782740407 >> v2:172.21.15.62:3300/0 conn(0x7fc9a415dfd0 0x7fc9a40aedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:18.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.512+0000 7fc9a37fe700 1 -- 172.21.15.31:0/3782740407 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9a40ac5b0 con 0x7fc9a4159550 2023-10-06T02:29:18.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.512+0000 7fc9a37fe700 1 -- 172.21.15.31:0/3782740407 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7fc998001470 con 0x7fc9a4159550 2023-10-06T02:29:18.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.512+0000 7fc9a37fe700 1 -- 172.21.15.31:0/3782740407 >> v1:172.21.15.31:6789/0 conn(0x7fc9a4159550 legacy=0x7fc9a40ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:18.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.513+0000 7fc9a37fe700 1 --2- 172.21.15.31:0/3782740407 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc994003ae0 0x7fc994003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:18.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.513+0000 7fc9a37fe700 1 --2- 172.21.15.31:0/3782740407 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc9a4159920 0x7fc994004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:18.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.513+0000 7fc9a37fe700 1 --2- 172.21.15.31:0/3782740407 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc994004d50 0x7fc9940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:18.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.513+0000 7fc9a37fe700 1 -- 172.21.15.31:0/3782740407 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a40abff0 con 0x7fc994003ae0 2023-10-06T02:29:18.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.513+0000 7fc9a37fe700 1 -- 172.21.15.31:0/3782740407 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a40ac2d0 con 0x7fc994004d50 2023-10-06T02:29:18.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.513+0000 7fc9a37fe700 1 -- 172.21.15.31:0/3782740407 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc99400bb40 con 0x7fc9a4159920 2023-10-06T02:29:18.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.513+0000 7fc9a8d9b700 1 --2- 172.21.15.31:0/3782740407 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc994003ae0 0x7fc994003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:18.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.513+0000 7fc9a3fff700 1 --2- 172.21.15.31:0/3782740407 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc9a4159920 0x7fc994004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:18.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.513+0000 7fc9a959c700 1 --2- 172.21.15.31:0/3782740407 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc994004d50 0x7fc9940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:18.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.513+0000 7fc9a8d9b700 1 -- 172.21.15.31:0/3782740407 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc9a4159920 msgr2=0x7fc994004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:18.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.513+0000 7fc9a8d9b700 1 --2- 172.21.15.31:0/3782740407 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc9a4159920 0x7fc994004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:18.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.513+0000 7fc9a8d9b700 1 -- 172.21.15.31:0/3782740407 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc994004d50 msgr2=0x7fc9940051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:18.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.513+0000 7fc9a8d9b700 1 --2- 172.21.15.31:0/3782740407 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc994004d50 0x7fc9940051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:18.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.513+0000 7fc9a8d9b700 1 -- 172.21.15.31:0/3782740407 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc994003570 con 0x7fc994003ae0 2023-10-06T02:29:18.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.513+0000 7fc9a3fff700 1 --2- 172.21.15.31:0/3782740407 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc9a4159920 0x7fc994004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:29:18.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.513+0000 7fc9a8d9b700 1 --2- 172.21.15.31:0/3782740407 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc994003ae0 0x7fc994003ef0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fc99800c2d0 tx=0x7fc998001dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff93030b9b11ddc3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:18.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.514+0000 7fc9a37fe700 1 -- 172.21.15.31:0/3782740407 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc99800cb40 con 0x7fc994003ae0 2023-10-06T02:29:18.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.514+0000 7fc9a37fe700 1 -- 172.21.15.31:0/3782740407 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc99800cca0 con 0x7fc994003ae0 2023-10-06T02:29:18.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.514+0000 7fc9a37fe700 1 -- 172.21.15.31:0/3782740407 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc998005b70 con 0x7fc994003ae0 2023-10-06T02:29:18.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.514+0000 7fc9a9d9d700 1 -- 172.21.15.31:0/3782740407 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc994003ae0 msgr2=0x7fc994003ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:18.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.514+0000 7fc9a9d9d700 1 --2- 172.21.15.31:0/3782740407 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc994003ae0 0x7fc994003ef0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fc99800c2d0 tx=0x7fc998001dc0 comp rx=0 tx=0).stop 2023-10-06T02:29:18.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.514+0000 7fc9a959c700 1 -- 172.21.15.31:0/3782740407 reap_dead start 2023-10-06T02:29:18.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.514+0000 7fc9a9d9d700 1 -- 172.21.15.31:0/3782740407 shutdown_connections 2023-10-06T02:29:18.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.514+0000 7fc9a9d9d700 1 -- 172.21.15.31:0/3782740407 >> 172.21.15.31:0/3782740407 conn(0x7fc9a406ce80 msgr2=0x7fc9a40ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:18.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.514+0000 7fc9a9d9d700 1 -- 172.21.15.31:0/3782740407 shutdown_connections 2023-10-06T02:29:18.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.514+0000 7fc9a9d9d700 1 -- 172.21.15.31:0/3782740407 wait complete. 2023-10-06T02:29:18.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.516+0000 7fc9a9d9d700 1 Processor -- start 2023-10-06T02:29:18.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.516+0000 7fc9a9d9d700 1 -- start start 2023-10-06T02:29:18.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.517+0000 7fc9a9d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc994003ae0 0x7fc9a414e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:18.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.517+0000 7fc9a9d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9a40c81e0 0x7fc9a414d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:18.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.517+0000 7fc9a9d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9a40ca120 0x7fc9a406e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:18.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.517+0000 7fc9a9d9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a414f0f0 con 0x7fc9a40c81e0 2023-10-06T02:29:18.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.517+0000 7fc9a9d9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a40c7d50 con 0x7fc9a40ca120 2023-10-06T02:29:18.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.517+0000 7fc9a9d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a414ee10 con 0x7fc994003ae0 2023-10-06T02:29:18.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.517+0000 7fc9a3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9a40c81e0 0x7fc9a414d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:18.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.517+0000 7fc9a3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9a40c81e0 0x7fc9a414d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52718/0 (socket says 172.21.15.31:52718) 2023-10-06T02:29:18.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.517+0000 7fc9a3fff700 1 -- 172.21.15.31:0/1550845848 learned_addr learned my addr 172.21.15.31:0/1550845848 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:18.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.517+0000 7fc9a8d9b700 1 --2- 172.21.15.31:0/1550845848 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc994003ae0 0x7fc9a414e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:18.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.517+0000 7fc9a959c700 1 --2- 172.21.15.31:0/1550845848 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9a40ca120 0x7fc9a406e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:18.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.517+0000 7fc9a3fff700 1 -- 172.21.15.31:0/1550845848 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc994003ae0 msgr2=0x7fc9a414e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:18.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.517+0000 7fc9a3fff700 1 --2- 172.21.15.31:0/1550845848 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc994003ae0 0x7fc9a414e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:18.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.517+0000 7fc9a3fff700 1 -- 172.21.15.31:0/1550845848 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9a40ca120 msgr2=0x7fc9a406e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:18.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.517+0000 7fc9a3fff700 1 --2- 172.21.15.31:0/1550845848 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9a40ca120 0x7fc9a406e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:18.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.517+0000 7fc9a3fff700 1 -- 172.21.15.31:0/1550845848 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9a40c7700 con 0x7fc9a40c81e0 2023-10-06T02:29:18.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.517+0000 7fc9a8d9b700 1 --2- 172.21.15.31:0/1550845848 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc994003ae0 0x7fc9a414e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:29:18.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.518+0000 7fc9a3fff700 1 --2- 172.21.15.31:0/1550845848 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9a40c81e0 0x7fc9a414d050 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fc9900118e0 tx=0x7fc990008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:18.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.518+0000 7fc9a0ff9700 1 -- 172.21.15.31:0/1550845848 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc990011e80 con 0x7fc9a40c81e0 2023-10-06T02:29:18.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.518+0000 7fc9a0ff9700 1 -- 172.21.15.31:0/1550845848 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc99000f860 con 0x7fc9a40c81e0 2023-10-06T02:29:18.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.518+0000 7fc9a0ff9700 1 -- 172.21.15.31:0/1550845848 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc99000fc00 con 0x7fc9a40c81e0 2023-10-06T02:29:18.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.518+0000 7fc9a9d9d700 1 -- 172.21.15.31:0/1550845848 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9a40c79e0 con 0x7fc9a40c81e0 2023-10-06T02:29:18.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.518+0000 7fc9a9d9d700 1 -- 172.21.15.31:0/1550845848 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9a414d590 con 0x7fc9a40c81e0 2023-10-06T02:29:18.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.519+0000 7fc9a9d9d700 1 -- 172.21.15.31:0/1550845848 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9a40c7560 con 0x7fc9a40c81e0 2023-10-06T02:29:18.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.520+0000 7fc9a0ff9700 1 -- 172.21.15.31:0/1550845848 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fc990015460 con 0x7fc9a40c81e0 2023-10-06T02:29:18.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.521+0000 7fc9a0ff9700 1 --2- 172.21.15.31:0/1550845848 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc988067320 0x7fc9880697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:18.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.521+0000 7fc9a0ff9700 1 -- 172.21.15.31:0/1550845848 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6130+0+0 (secure 0 0 0) 0x7fc9900855c0 con 0x7fc9a40c81e0 2023-10-06T02:29:18.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.521+0000 7fc9a8d9b700 1 --2- 172.21.15.31:0/1550845848 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc988067320 0x7fc9880697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:18.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.522+0000 7fc9a8d9b700 1 --2- 172.21.15.31:0/1550845848 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc988067320 0x7fc9880697d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc998015040 tx=0x7fc998012000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:18.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.525+0000 7fc9a0ff9700 1 -- 172.21.15.31:0/1550845848 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc9a40c7560 con 0x7fc9a40c81e0 2023-10-06T02:29:18.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.728+0000 7fc9a9d9d700 1 -- 172.21.15.31:0/1550845848 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc9a40c7560 con 0x7fc9a40c81e0 2023-10-06T02:29:18.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.728+0000 7fc9a0ff9700 1 -- 172.21.15.31:0/1550845848 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v108) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc9a40c7560 con 0x7fc9a40c81e0 2023-10-06T02:29:18.730 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-10-06T02:29:18.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.731+0000 7fc9a9d9d700 1 -- 172.21.15.31:0/1550845848 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc988067320 msgr2=0x7fc9880697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:18.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.731+0000 7fc9a9d9d700 1 --2- 172.21.15.31:0/1550845848 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc988067320 0x7fc9880697d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc998015040 tx=0x7fc998012000 comp rx=0 tx=0).stop 2023-10-06T02:29:18.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.731+0000 7fc9a9d9d700 1 -- 172.21.15.31:0/1550845848 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9a40c81e0 msgr2=0x7fc9a414d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:18.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.731+0000 7fc9a9d9d700 1 --2- 172.21.15.31:0/1550845848 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9a40c81e0 0x7fc9a414d050 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fc9900118e0 tx=0x7fc990008040 comp rx=0 tx=0).stop 2023-10-06T02:29:18.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.731+0000 7fc9a9d9d700 1 -- 172.21.15.31:0/1550845848 shutdown_connections 2023-10-06T02:29:18.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.731+0000 7fc9a9d9d700 1 --2- 172.21.15.31:0/1550845848 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9a40ca120 0x7fc9a406e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:18.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.731+0000 7fc9a9d9d700 1 --2- 172.21.15.31:0/1550845848 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc988067320 0x7fc9880697d0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:18.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.731+0000 7fc9a9d9d700 1 --2- 172.21.15.31:0/1550845848 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc994003ae0 0x7fc9a414e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:18.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.731+0000 7fc9a9d9d700 1 --2- 172.21.15.31:0/1550845848 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9a40c81e0 0x7fc9a414d050 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:18.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.731+0000 7fc9a9d9d700 1 -- 172.21.15.31:0/1550845848 >> 172.21.15.31:0/1550845848 conn(0x7fc9a406ce80 msgr2=0x7fc9a40aee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:18.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.731+0000 7fc9a9d9d700 1 -- 172.21.15.31:0/1550845848 shutdown_connections 2023-10-06T02:29:18.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.731+0000 7fc9a9d9d700 1 -- 172.21.15.31:0/1550845848 wait complete. 2023-10-06T02:29:18.749 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T02:29:18.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.939+0000 7f51b2b9d700 1 Processor -- start 2023-10-06T02:29:18.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.940+0000 7f51b2b9d700 1 -- start start 2023-10-06T02:29:18.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.941+0000 7f51b2b9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f51ac155890 0x7f51ac0c7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:18.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.941+0000 7f51b2b9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f51ac155c60 0x7f51ac06a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:18.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.941+0000 7f51b2b9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51ac0c7e70 con 0x7f51ac06de50 2023-10-06T02:29:18.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.941+0000 7f51b2b9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51ac06b790 con 0x7f51ac155c60 2023-10-06T02:29:18.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.941+0000 7f51b2b9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51ac06a8e0 con 0x7f51ac155890 2023-10-06T02:29:18.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.941+0000 7f51b139a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f51ac155c60 0x7f51ac06a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:18.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.941+0000 7f51b139a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f51ac155c60 0x7f51ac06a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52726/0 (socket says 172.21.15.31:52726) 2023-10-06T02:29:18.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.941+0000 7f51b139a700 1 -- 172.21.15.31:0/2085713593 learned_addr learned my addr 172.21.15.31:0/2085713593 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:18.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.942+0000 7f51b1b9b700 1 --2- 172.21.15.31:0/2085713593 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f51ac155890 0x7f51ac0c7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:18.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.942+0000 7f51b139a700 1 -- 172.21.15.31:0/2085713593 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f51ac155890 msgr2=0x7f51ac0c7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:18.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.942+0000 7f51b139a700 1 --2- 172.21.15.31:0/2085713593 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f51ac155890 0x7f51ac0c7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:18.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.942+0000 7f51b139a700 1 -- 172.21.15.31:0/2085713593 >> v1:172.21.15.62:6789/0 conn(0x7f51ac06de50 legacy=0x7f51ac159720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:18.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.942+0000 7f51b139a700 1 -- 172.21.15.31:0/2085713593 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51ac06ab80 con 0x7f51ac155c60 2023-10-06T02:29:18.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.942+0000 7f51b139a700 1 --2- 172.21.15.31:0/2085713593 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51ac155c60 0x7f51ac06a3a0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f519c0099b0 tx=0x7f519c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73be369b4aef1a6b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:18.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.942+0000 7f51b0b99700 1 -- 172.21.15.31:0/2085713593 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f519c009ea0 con 0x7f51ac155c60 2023-10-06T02:29:18.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.942+0000 7f51b0b99700 1 -- 172.21.15.31:0/2085713593 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f519c00b860 con 0x7f51ac155c60 2023-10-06T02:29:18.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.942+0000 7f51b0b99700 1 -- 172.21.15.31:0/2085713593 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f519c00bc00 con 0x7f51ac155c60 2023-10-06T02:29:18.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.943+0000 7f51b2b9d700 1 -- 172.21.15.31:0/2085713593 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51ac155c60 msgr2=0x7f51ac06a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:18.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.943+0000 7f51b2b9d700 1 --2- 172.21.15.31:0/2085713593 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51ac155c60 0x7f51ac06a3a0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f519c0099b0 tx=0x7f519c009250 comp rx=0 tx=0).stop 2023-10-06T02:29:18.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.943+0000 7f51b2b9d700 1 -- 172.21.15.31:0/2085713593 shutdown_connections 2023-10-06T02:29:18.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.943+0000 7f51b2b9d700 1 --2- 172.21.15.31:0/2085713593 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f51ac155890 0x7f51ac0c7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:18.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.943+0000 7f51b2b9d700 1 --2- 172.21.15.31:0/2085713593 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51ac155c60 0x7f51ac06a3a0 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:18.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.943+0000 7f51b2b9d700 1 -- 172.21.15.31:0/2085713593 >> 172.21.15.31:0/2085713593 conn(0x7f51ac06d110 msgr2=0x7f51ac0cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:18.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.943+0000 7f51b2b9d700 1 -- 172.21.15.31:0/2085713593 shutdown_connections 2023-10-06T02:29:18.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.943+0000 7f51b2b9d700 1 -- 172.21.15.31:0/2085713593 wait complete. 2023-10-06T02:29:18.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.945+0000 7f51b2b9d700 1 Processor -- start 2023-10-06T02:29:18.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.945+0000 7f51b2b9d700 1 -- start start 2023-10-06T02:29:18.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.946+0000 7f51b2b9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51ac06de50 0x7f51ac141520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:18.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.946+0000 7f51b2b9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51ac155890 0x7f51ac0c1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:18.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.946+0000 7f51b2b9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f51ac155c60 0x7f51ac0c2200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:18.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.946+0000 7f51b2b9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51ac16c400 con 0x7f51ac155890 2023-10-06T02:29:18.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.946+0000 7f51b2b9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51ac141a60 con 0x7f51ac06de50 2023-10-06T02:29:18.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.946+0000 7f51b2b9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51ac16b7f0 con 0x7f51ac155c60 2023-10-06T02:29:18.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.946+0000 7f51b139a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51ac155890 0x7f51ac0c1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:18.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.946+0000 7f51b139a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51ac155890 0x7f51ac0c1cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52738/0 (socket says 172.21.15.31:52738) 2023-10-06T02:29:18.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.946+0000 7f51b139a700 1 -- 172.21.15.31:0/4153479810 learned_addr learned my addr 172.21.15.31:0/4153479810 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:18.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.946+0000 7f51b1b9b700 1 --2- 172.21.15.31:0/4153479810 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51ac06de50 0x7f51ac141520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:18.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.946+0000 7f51b239c700 1 --2- 172.21.15.31:0/4153479810 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f51ac155c60 0x7f51ac0c2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:18.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.946+0000 7f51b139a700 1 -- 172.21.15.31:0/4153479810 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f51ac155c60 msgr2=0x7f51ac0c2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:18.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.946+0000 7f51b139a700 1 --2- 172.21.15.31:0/4153479810 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f51ac155c60 0x7f51ac0c2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:18.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.946+0000 7f51b139a700 1 -- 172.21.15.31:0/4153479810 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51ac06de50 msgr2=0x7f51ac141520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:18.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.946+0000 7f51b139a700 1 --2- 172.21.15.31:0/4153479810 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51ac06de50 0x7f51ac141520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:18.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.946+0000 7f51b139a700 1 -- 172.21.15.31:0/4153479810 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f519c0094f0 con 0x7f51ac155890 2023-10-06T02:29:18.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.946+0000 7f51b1b9b700 1 --2- 172.21.15.31:0/4153479810 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51ac06de50 0x7f51ac141520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:29:18.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.947+0000 7f51b139a700 1 --2- 172.21.15.31:0/4153479810 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51ac155890 0x7f51ac0c1cc0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f51ac0cd1c0 tx=0x7f519c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:18.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.947+0000 7f51a1ffb700 1 -- 172.21.15.31:0/4153479810 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f519c00bbe0 con 0x7f51ac155890 2023-10-06T02:29:18.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.947+0000 7f51a1ffb700 1 -- 172.21.15.31:0/4153479810 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f519c01a450 con 0x7f51ac155890 2023-10-06T02:29:18.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.947+0000 7f51a1ffb700 1 -- 172.21.15.31:0/4153479810 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f519c01a7f0 con 0x7f51ac155890 2023-10-06T02:29:18.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.947+0000 7f51b2b9d700 1 -- 172.21.15.31:0/4153479810 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51ac16bad0 con 0x7f51ac155890 2023-10-06T02:29:18.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.947+0000 7f51b2b9d700 1 -- 172.21.15.31:0/4153479810 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51ac16c090 con 0x7f51ac155890 2023-10-06T02:29:18.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.948+0000 7f51b2b9d700 1 -- 172.21.15.31:0/4153479810 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51ac04c3c0 con 0x7f51ac155890 2023-10-06T02:29:18.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.949+0000 7f51a1ffb700 1 -- 172.21.15.31:0/4153479810 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f519c01ab60 con 0x7f51ac155890 2023-10-06T02:29:18.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.950+0000 7f51a1ffb700 1 --2- 172.21.15.31:0/4153479810 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5190067320 0x7f51900697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:18.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.950+0000 7f51a1ffb700 1 -- 172.21.15.31:0/4153479810 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f519c0867f0 con 0x7f51ac155890 2023-10-06T02:29:18.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.950+0000 7f51b1b9b700 1 --2- 172.21.15.31:0/4153479810 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5190067320 0x7f51900697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:18.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.951+0000 7f51b1b9b700 1 --2- 172.21.15.31:0/4153479810 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5190067320 0x7f51900697d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f51a8009d80 tx=0x7f51a8008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:18.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:18.954+0000 7f51a1ffb700 1 -- 172.21.15.31:0/4153479810 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f519c0532d0 con 0x7f51ac155890 2023-10-06T02:29:19.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.158+0000 7f51b2b9d700 1 -- 172.21.15.31:0/4153479810 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f51ac06f170 con 0x7f51ac155890 2023-10-06T02:29:19.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.158+0000 7f51a1ffb700 1 -- 172.21.15.31:0/4153479810 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v108) v1 ==== 92+0+10 (secure 0 0 0) 0x7f519c0568f0 con 0x7f51ac155890 2023-10-06T02:29:19.160 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 8 2023-10-06T02:29:19.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.161+0000 7f51b2b9d700 1 -- 172.21.15.31:0/4153479810 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5190067320 msgr2=0x7f51900697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:19.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.161+0000 7f51b2b9d700 1 --2- 172.21.15.31:0/4153479810 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5190067320 0x7f51900697d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f51a8009d80 tx=0x7f51a8008040 comp rx=0 tx=0).stop 2023-10-06T02:29:19.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.161+0000 7f51b2b9d700 1 -- 172.21.15.31:0/4153479810 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51ac155890 msgr2=0x7f51ac0c1cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:19.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.161+0000 7f51b2b9d700 1 --2- 172.21.15.31:0/4153479810 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51ac155890 0x7f51ac0c1cc0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f51ac0cd1c0 tx=0x7f519c017000 comp rx=0 tx=0).stop 2023-10-06T02:29:19.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.161+0000 7f51b2b9d700 1 -- 172.21.15.31:0/4153479810 shutdown_connections 2023-10-06T02:29:19.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.161+0000 7f51b2b9d700 1 --2- 172.21.15.31:0/4153479810 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5190067320 0x7f51900697d0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:19.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.161+0000 7f51b2b9d700 1 --2- 172.21.15.31:0/4153479810 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f51ac155c60 0x7f51ac0c2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:19.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.161+0000 7f51b2b9d700 1 --2- 172.21.15.31:0/4153479810 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51ac06de50 0x7f51ac141520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:19.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.161+0000 7f51b2b9d700 1 --2- 172.21.15.31:0/4153479810 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51ac155890 0x7f51ac0c1cc0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:19.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.161+0000 7f51b2b9d700 1 -- 172.21.15.31:0/4153479810 >> 172.21.15.31:0/4153479810 conn(0x7f51ac06d110 msgr2=0x7f51ac0c9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:19.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.161+0000 7f51b2b9d700 1 -- 172.21.15.31:0/4153479810 shutdown_connections 2023-10-06T02:29:19.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.161+0000 7f51b2b9d700 1 -- 172.21.15.31:0/4153479810 wait complete. 2023-10-06T02:29:19.179 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T02:29:19.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.369+0000 7f2a75f9d700 1 Processor -- start 2023-10-06T02:29:19.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.369+0000 7f2a75f9d700 1 -- start start 2023-10-06T02:29:19.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.370+0000 7f2a75f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2a700c7fb0 0x7f2a7006abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:19.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.370+0000 7f2a75f9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a7006b290 con 0x7f2a700c7be0 2023-10-06T02:29:19.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.370+0000 7f2a75f9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a7006b570 con 0x7f2a7013f780 2023-10-06T02:29:19.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.370+0000 7f2a75f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a7006bf10 con 0x7f2a700c7fb0 2023-10-06T02:29:19.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.371+0000 7f2a74f9b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f2a700c7be0 0x7f2a7015a990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:33004/0 (socket says 172.21.15.31:33004) 2023-10-06T02:29:19.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.371+0000 7f2a74f9b700 1 -- 172.21.15.31:0/3110064127 learned_addr learned my addr 172.21.15.31:0/3110064127 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:29:19.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.371+0000 7f2a6ffff700 1 --2- 172.21.15.31:0/3110064127 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2a700c7fb0 0x7f2a7006abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:19.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.371+0000 7f2a6f7fe700 1 -- 172.21.15.31:0/3110064127 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3488731081 0 0) 0x7f2a7006b290 con 0x7f2a700c7be0 2023-10-06T02:29:19.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.371+0000 7f2a6f7fe700 1 -- 172.21.15.31:0/3110064127 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a60003570 con 0x7f2a700c7be0 2023-10-06T02:29:19.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.371+0000 7f2a6f7fe700 1 -- 172.21.15.31:0/3110064127 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 216338811 0 0) 0x7f2a7006b570 con 0x7f2a7013f780 2023-10-06T02:29:19.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.371+0000 7f2a6f7fe700 1 -- 172.21.15.31:0/3110064127 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a7006b290 con 0x7f2a7013f780 2023-10-06T02:29:19.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.371+0000 7f2a6ffff700 1 -- 172.21.15.31:0/3110064127 >> v1:172.21.15.62:6789/0 conn(0x7f2a7013f780 legacy=0x7f2a700c9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:19.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.371+0000 7f2a6ffff700 1 -- 172.21.15.31:0/3110064127 >> v1:172.21.15.31:6789/0 conn(0x7f2a700c7be0 legacy=0x7f2a7015a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:19.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.371+0000 7f2a6ffff700 1 -- 172.21.15.31:0/3110064127 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a7006b850 con 0x7f2a700c7fb0 2023-10-06T02:29:19.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.372+0000 7f2a6ffff700 1 --2- 172.21.15.31:0/3110064127 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2a700c7fb0 0x7f2a7006abd0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c009910 tx=0x7f2a5c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57c6ff50b42da10d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:19.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.372+0000 7f2a6f7fe700 1 -- 172.21.15.31:0/3110064127 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c011630 con 0x7f2a700c7fb0 2023-10-06T02:29:19.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.372+0000 7f2a6f7fe700 1 -- 172.21.15.31:0/3110064127 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a5c011790 con 0x7f2a700c7fb0 2023-10-06T02:29:19.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.372+0000 7f2a6f7fe700 1 -- 172.21.15.31:0/3110064127 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c011b30 con 0x7f2a700c7fb0 2023-10-06T02:29:19.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.373+0000 7f2a75f9d700 1 -- 172.21.15.31:0/3110064127 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2a700c7fb0 msgr2=0x7f2a7006abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:19.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.373+0000 7f2a75f9d700 1 --2- 172.21.15.31:0/3110064127 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2a700c7fb0 0x7f2a7006abd0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c009910 tx=0x7f2a5c0091a0 comp rx=0 tx=0).stop 2023-10-06T02:29:19.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.373+0000 7f2a75f9d700 1 -- 172.21.15.31:0/3110064127 shutdown_connections 2023-10-06T02:29:19.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.373+0000 7f2a75f9d700 1 --2- 172.21.15.31:0/3110064127 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2a700c7fb0 0x7f2a7006abd0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:19.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.373+0000 7f2a75f9d700 1 -- 172.21.15.31:0/3110064127 >> 172.21.15.31:0/3110064127 conn(0x7f2a7006d110 msgr2=0x7f2a700cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:19.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.373+0000 7f2a75f9d700 1 -- 172.21.15.31:0/3110064127 shutdown_connections 2023-10-06T02:29:19.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.373+0000 7f2a75f9d700 1 -- 172.21.15.31:0/3110064127 wait complete. 2023-10-06T02:29:19.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.374+0000 7f2a75f9d700 1 Processor -- start 2023-10-06T02:29:19.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.374+0000 7f2a75f9d700 1 -- start start 2023-10-06T02:29:19.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.375+0000 7f2a75f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2a700c7be0 0x7f2a70155c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:19.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.375+0000 7f2a75f9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a700c7fb0 0x7f2a70156dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:19.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.375+0000 7f2a75f9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2a7013f780 0x7f2a7013ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:19.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.375+0000 7f2a75f9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a70140b60 con 0x7f2a7013f780 2023-10-06T02:29:19.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.375+0000 7f2a75f9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a70156190 con 0x7f2a700c7fb0 2023-10-06T02:29:19.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.375+0000 7f2a75f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a70159c60 con 0x7f2a700c7be0 2023-10-06T02:29:19.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.375+0000 7f2a7579c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2a7013f780 0x7f2a7013ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:19.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.376+0000 7f2a7579c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2a7013f780 0x7f2a7013ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52744/0 (socket says 172.21.15.31:52744) 2023-10-06T02:29:19.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.376+0000 7f2a7579c700 1 -- 172.21.15.31:0/109404280 learned_addr learned my addr 172.21.15.31:0/109404280 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:19.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.376+0000 7f2a6ffff700 1 --2- 172.21.15.31:0/109404280 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a700c7fb0 0x7f2a70156dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:19.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.376+0000 7f2a74f9b700 1 --2- 172.21.15.31:0/109404280 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2a700c7be0 0x7f2a70155c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:19.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.376+0000 7f2a7579c700 1 -- 172.21.15.31:0/109404280 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2a700c7be0 msgr2=0x7f2a70155c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:19.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.376+0000 7f2a7579c700 1 --2- 172.21.15.31:0/109404280 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2a700c7be0 0x7f2a70155c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:19.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.376+0000 7f2a7579c700 1 -- 172.21.15.31:0/109404280 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a700c7fb0 msgr2=0x7f2a70156dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:19.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.376+0000 7f2a7579c700 1 --2- 172.21.15.31:0/109404280 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a700c7fb0 0x7f2a70156dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:19.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.376+0000 7f2a7579c700 1 -- 172.21.15.31:0/109404280 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a5c009420 con 0x7f2a7013f780 2023-10-06T02:29:19.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.376+0000 7f2a6ffff700 1 --2- 172.21.15.31:0/109404280 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a700c7fb0 0x7f2a70156dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:29:19.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.376+0000 7f2a7579c700 1 --2- 172.21.15.31:0/109404280 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2a7013f780 0x7f2a7013ccc0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f2a6800c860 tx=0x7f2a68002700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:19.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.376+0000 7f2a6cff9700 1 -- 172.21.15.31:0/109404280 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a68002030 con 0x7f2a7013f780 2023-10-06T02:29:19.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.377+0000 7f2a6cff9700 1 -- 172.21.15.31:0/109404280 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a680021b0 con 0x7f2a7013f780 2023-10-06T02:29:19.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.377+0000 7f2a75f9d700 1 -- 172.21.15.31:0/109404280 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a7013ff50 con 0x7f2a7013f780 2023-10-06T02:29:19.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.377+0000 7f2a6cff9700 1 -- 172.21.15.31:0/109404280 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a68012630 con 0x7f2a7013f780 2023-10-06T02:29:19.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.377+0000 7f2a75f9d700 1 -- 172.21.15.31:0/109404280 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a70140510 con 0x7f2a7013f780 2023-10-06T02:29:19.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.378+0000 7f2a75f9d700 1 -- 172.21.15.31:0/109404280 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a7004c3c0 con 0x7f2a7013f780 2023-10-06T02:29:19.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.379+0000 7f2a6cff9700 1 -- 172.21.15.31:0/109404280 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f2a68012a20 con 0x7f2a7013f780 2023-10-06T02:29:19.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.379+0000 7f2a6cff9700 1 --2- 172.21.15.31:0/109404280 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f2a54067320 0x7f2a540697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:19.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.379+0000 7f2a6cff9700 1 -- 172.21.15.31:0/109404280 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f2a68084f00 con 0x7f2a7013f780 2023-10-06T02:29:19.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.380+0000 7f2a74f9b700 1 --2- 172.21.15.31:0/109404280 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f2a54067320 0x7f2a540697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:19.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.380+0000 7f2a74f9b700 1 --2- 172.21.15.31:0/109404280 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f2a54067320 0x7f2a540697d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2a640058d0 tx=0x7f2a64009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:19.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.383+0000 7f2a6cff9700 1 -- 172.21.15.31:0/109404280 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2a680519a0 con 0x7f2a7013f780 2023-10-06T02:29:19.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.585+0000 7f2a75f9d700 1 -- 172.21.15.31:0/109404280 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2a7006bb30 con 0x7f2a7013f780 2023-10-06T02:29:19.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.586+0000 7f2a6cff9700 1 -- 172.21.15.31:0/109404280 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v108) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2a68054fc0 con 0x7f2a7013f780 2023-10-06T02:29:19.588 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-10-06T02:29:19.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.588+0000 7f2a75f9d700 1 -- 172.21.15.31:0/109404280 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f2a54067320 msgr2=0x7f2a540697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:19.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.588+0000 7f2a75f9d700 1 --2- 172.21.15.31:0/109404280 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f2a54067320 0x7f2a540697d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2a640058d0 tx=0x7f2a64009040 comp rx=0 tx=0).stop 2023-10-06T02:29:19.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.588+0000 7f2a75f9d700 1 -- 172.21.15.31:0/109404280 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2a7013f780 msgr2=0x7f2a7013ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:19.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.588+0000 7f2a75f9d700 1 --2- 172.21.15.31:0/109404280 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2a7013f780 0x7f2a7013ccc0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f2a6800c860 tx=0x7f2a68002700 comp rx=0 tx=0).stop 2023-10-06T02:29:19.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.588+0000 7f2a75f9d700 1 -- 172.21.15.31:0/109404280 shutdown_connections 2023-10-06T02:29:19.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.588+0000 7f2a75f9d700 1 --2- 172.21.15.31:0/109404280 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2a7013f780 0x7f2a7013ccc0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:19.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.588+0000 7f2a75f9d700 1 --2- 172.21.15.31:0/109404280 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f2a54067320 0x7f2a540697d0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:19.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.588+0000 7f2a75f9d700 1 --2- 172.21.15.31:0/109404280 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2a700c7be0 0x7f2a70155c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:19.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.588+0000 7f2a75f9d700 1 --2- 172.21.15.31:0/109404280 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a700c7fb0 0x7f2a70156dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:19.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.588+0000 7f2a75f9d700 1 -- 172.21.15.31:0/109404280 >> 172.21.15.31:0/109404280 conn(0x7f2a7006d110 msgr2=0x7f2a7006e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:19.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.588+0000 7f2a75f9d700 1 -- 172.21.15.31:0/109404280 shutdown_connections 2023-10-06T02:29:19.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.588+0000 7f2a75f9d700 1 -- 172.21.15.31:0/109404280 wait complete. 2023-10-06T02:29:19.607 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T02:29:19.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.796+0000 7f683b19d700 1 Processor -- start 2023-10-06T02:29:19.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.796+0000 7f683b19d700 1 -- start start 2023-10-06T02:29:19.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.797+0000 7f683b19d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f68340c7be0 0x7f683406abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:19.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.797+0000 7f683b19d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f683406b290 con 0x7f68340c7fb0 2023-10-06T02:29:19.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.797+0000 7f683b19d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f683406b570 con 0x7f683413f780 2023-10-06T02:29:19.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.797+0000 7f683b19d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f683406bf10 con 0x7f68340c7be0 2023-10-06T02:29:19.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.797+0000 7f683999a700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f68340c7fb0 0x7f683415a990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:33016/0 (socket says 172.21.15.31:33016) 2023-10-06T02:29:19.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.797+0000 7f683999a700 1 -- 172.21.15.31:0/189544153 learned_addr learned my addr 172.21.15.31:0/189544153 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:29:19.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.798+0000 7f683a19b700 1 --2- 172.21.15.31:0/189544153 >> v2:172.21.15.62:3300/0 conn(0x7f68340c7be0 0x7f683406abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:19.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.798+0000 7f6839199700 1 -- 172.21.15.31:0/189544153 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2017684918 0 0) 0x7f683406b290 con 0x7f68340c7fb0 2023-10-06T02:29:19.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.798+0000 7f6839199700 1 -- 172.21.15.31:0/189544153 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6820003570 con 0x7f68340c7fb0 2023-10-06T02:29:19.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.798+0000 7f683a19b700 1 -- 172.21.15.31:0/189544153 >> v1:172.21.15.62:6789/0 conn(0x7f683413f780 legacy=0x7f683413fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:19.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.798+0000 7f683a19b700 1 -- 172.21.15.31:0/189544153 >> v1:172.21.15.31:6789/0 conn(0x7f68340c7fb0 legacy=0x7f683415a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:19.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.798+0000 7f683a19b700 1 -- 172.21.15.31:0/189544153 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f683406b850 con 0x7f68340c7be0 2023-10-06T02:29:19.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.798+0000 7f683a19b700 1 --2- 172.21.15.31:0/189544153 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68340c7be0 0x7f683406abd0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f68300098e0 tx=0x7f68300091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70d722a8b4edfa32 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:19.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.799+0000 7f6839199700 1 -- 172.21.15.31:0/189544153 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f683000b640 con 0x7f68340c7be0 2023-10-06T02:29:19.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.799+0000 7f6839199700 1 -- 172.21.15.31:0/189544153 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f683000b7a0 con 0x7f68340c7be0 2023-10-06T02:29:19.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.799+0000 7f6839199700 1 -- 172.21.15.31:0/189544153 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f683000bb40 con 0x7f68340c7be0 2023-10-06T02:29:19.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.799+0000 7f683b19d700 1 -- 172.21.15.31:0/189544153 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68340c7be0 msgr2=0x7f683406abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:19.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.799+0000 7f683b19d700 1 --2- 172.21.15.31:0/189544153 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68340c7be0 0x7f683406abd0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f68300098e0 tx=0x7f68300091a0 comp rx=0 tx=0).stop 2023-10-06T02:29:19.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.799+0000 7f683b19d700 1 -- 172.21.15.31:0/189544153 shutdown_connections 2023-10-06T02:29:19.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.799+0000 7f683b19d700 1 --2- 172.21.15.31:0/189544153 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68340c7be0 0x7f683406abd0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:19.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.799+0000 7f683b19d700 1 -- 172.21.15.31:0/189544153 >> 172.21.15.31:0/189544153 conn(0x7f683406d110 msgr2=0x7f68340cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:19.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.800+0000 7f683b19d700 1 -- 172.21.15.31:0/189544153 shutdown_connections 2023-10-06T02:29:19.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.800+0000 7f683b19d700 1 -- 172.21.15.31:0/189544153 wait complete. 2023-10-06T02:29:19.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.801+0000 7f683b19d700 1 Processor -- start 2023-10-06T02:29:19.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.801+0000 7f683b19d700 1 -- start start 2023-10-06T02:29:19.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.802+0000 7f683b19d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f68340c7be0 0x7f6834155dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:19.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.802+0000 7f683b19d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68340c7fb0 0x7f68340b1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:19.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.802+0000 7f683b19d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f683413f780 0x7f683413bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:19.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.802+0000 7f683b19d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6834157a00 con 0x7f683413f780 2023-10-06T02:29:19.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.802+0000 7f683b19d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6834156df0 con 0x7f68340c7fb0 2023-10-06T02:29:19.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.802+0000 7f683b19d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68341570d0 con 0x7f68340c7be0 2023-10-06T02:29:19.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.802+0000 7f683a99c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f683413f780 0x7f683413bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:19.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.802+0000 7f683a99c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f683413f780 0x7f683413bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52758/0 (socket says 172.21.15.31:52758) 2023-10-06T02:29:19.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.802+0000 7f683a99c700 1 -- 172.21.15.31:0/2762821534 learned_addr learned my addr 172.21.15.31:0/2762821534 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:19.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.802+0000 7f683999a700 1 --2- 172.21.15.31:0/2762821534 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68340c7fb0 0x7f68340b1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:19.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.802+0000 7f683a19b700 1 --2- 172.21.15.31:0/2762821534 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f68340c7be0 0x7f6834155dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:19.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.803+0000 7f683a99c700 1 -- 172.21.15.31:0/2762821534 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f68340c7be0 msgr2=0x7f6834155dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:19.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.803+0000 7f683a99c700 1 --2- 172.21.15.31:0/2762821534 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f68340c7be0 0x7f6834155dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:19.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.803+0000 7f683a99c700 1 -- 172.21.15.31:0/2762821534 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68340c7fb0 msgr2=0x7f68340b1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:19.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.803+0000 7f683a99c700 1 --2- 172.21.15.31:0/2762821534 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68340c7fb0 0x7f68340b1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:19.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.803+0000 7f683a99c700 1 -- 172.21.15.31:0/2762821534 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6830009420 con 0x7f683413f780 2023-10-06T02:29:19.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.803+0000 7f683a99c700 1 --2- 172.21.15.31:0/2762821534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f683413f780 0x7f683413bca0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f682c00c840 tx=0x7f682c001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:19.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.803+0000 7f682a7fc700 1 -- 172.21.15.31:0/2762821534 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f682c012630 con 0x7f683413f780 2023-10-06T02:29:19.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.803+0000 7f682a7fc700 1 -- 172.21.15.31:0/2762821534 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f682c0127b0 con 0x7f683413f780 2023-10-06T02:29:19.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.803+0000 7f682a7fc700 1 -- 172.21.15.31:0/2762821534 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f682c012b70 con 0x7f683413f780 2023-10-06T02:29:19.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.803+0000 7f683b19d700 1 -- 172.21.15.31:0/2762821534 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68341573b0 con 0x7f683413f780 2023-10-06T02:29:19.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.803+0000 7f683b19d700 1 -- 172.21.15.31:0/2762821534 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6834156ac0 con 0x7f683413f780 2023-10-06T02:29:19.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.804+0000 7f683b19d700 1 -- 172.21.15.31:0/2762821534 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f683404c3c0 con 0x7f683413f780 2023-10-06T02:29:19.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.805+0000 7f682a7fc700 1 -- 172.21.15.31:0/2762821534 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f682c01a3a0 con 0x7f683413f780 2023-10-06T02:29:19.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.806+0000 7f682a7fc700 1 --2- 172.21.15.31:0/2762821534 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6818067320 0x7f68180697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:19.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.806+0000 7f682a7fc700 1 -- 172.21.15.31:0/2762821534 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f682c084f90 con 0x7f683413f780 2023-10-06T02:29:19.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.807+0000 7f683a19b700 1 --2- 172.21.15.31:0/2762821534 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6818067320 0x7f68180697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:19.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.807+0000 7f683a19b700 1 --2- 172.21.15.31:0/2762821534 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6818067320 0x7f68180697d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f68300093f0 tx=0x7f6830017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:19.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:19.810+0000 7f682a7fc700 1 -- 172.21.15.31:0/2762821534 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f682c051a30 con 0x7f683413f780 2023-10-06T02:29:20.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.011+0000 7f683b19d700 1 -- 172.21.15.31:0/2762821534 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6834157b40 con 0x7f683413f780 2023-10-06T02:29:20.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.012+0000 7f682a7fc700 1 -- 172.21.15.31:0/2762821534 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v108) v1 ==== 100+0+11 (secure 0 0 0) 0x7f682c055050 con 0x7f683413f780 2023-10-06T02:29:20.014 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-10-06T02:29:20.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.014+0000 7f683b19d700 1 -- 172.21.15.31:0/2762821534 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6818067320 msgr2=0x7f68180697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:20.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.014+0000 7f683b19d700 1 --2- 172.21.15.31:0/2762821534 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6818067320 0x7f68180697d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f68300093f0 tx=0x7f6830017000 comp rx=0 tx=0).stop 2023-10-06T02:29:20.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.014+0000 7f683b19d700 1 -- 172.21.15.31:0/2762821534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f683413f780 msgr2=0x7f683413bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:20.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.014+0000 7f683b19d700 1 --2- 172.21.15.31:0/2762821534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f683413f780 0x7f683413bca0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f682c00c840 tx=0x7f682c001670 comp rx=0 tx=0).stop 2023-10-06T02:29:20.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.014+0000 7f683b19d700 1 -- 172.21.15.31:0/2762821534 shutdown_connections 2023-10-06T02:29:20.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.014+0000 7f683b19d700 1 --2- 172.21.15.31:0/2762821534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f683413f780 0x7f683413bca0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:20.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.014+0000 7f683b19d700 1 --2- 172.21.15.31:0/2762821534 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6818067320 0x7f68180697d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:20.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.014+0000 7f683b19d700 1 --2- 172.21.15.31:0/2762821534 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f68340c7be0 0x7f6834155dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:20.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.014+0000 7f683b19d700 1 --2- 172.21.15.31:0/2762821534 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68340c7fb0 0x7f68340b1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:20.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.014+0000 7f683b19d700 1 -- 172.21.15.31:0/2762821534 >> 172.21.15.31:0/2762821534 conn(0x7f683406d110 msgr2=0x7f68340cae60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:20.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.014+0000 7f683b19d700 1 -- 172.21.15.31:0/2762821534 shutdown_connections 2023-10-06T02:29:20.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.014+0000 7f683b19d700 1 -- 172.21.15.31:0/2762821534 wait complete. 2023-10-06T02:29:20.033 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:29:20.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.219+0000 7ff6fab9d700 1 Processor -- start 2023-10-06T02:29:20.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.219+0000 7ff6fab9d700 1 -- start start 2023-10-06T02:29:20.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.220+0000 7ff6fab9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff6f4159550 0x7ff6f40ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:20.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.220+0000 7ff6fab9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6f4159920 0x7ff6f40ad090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:20.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.220+0000 7ff6fab9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6f40acc00 con 0x7ff6f415dfd0 2023-10-06T02:29:20.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.220+0000 7ff6fab9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6f40abff0 con 0x7ff6f4159550 2023-10-06T02:29:20.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.220+0000 7ff6fab9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f40ac2d0 con 0x7ff6f4159920 2023-10-06T02:29:20.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.221+0000 7ff6fa39c700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6f4159920 0x7ff6f40ad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:20.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.221+0000 7ff6f939a700 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7ff6f415dfd0 0x7ff6f40aedb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:53776/0 (socket says 172.21.15.31:53776) 2023-10-06T02:29:20.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.221+0000 7ff6f939a700 1 -- 172.21.15.31:0/3693738325 learned_addr learned my addr 172.21.15.31:0/3693738325 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:29:20.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.221+0000 7ff6f9b9b700 1 --2- 172.21.15.31:0/3693738325 >> v2:172.21.15.62:3300/0 conn(0x7ff6f4159550 0x7ff6f40ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:20.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.221+0000 7ff6f8b99700 1 -- 172.21.15.31:0/3693738325 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3413111078 0 0) 0x7ff6f40acc00 con 0x7ff6f415dfd0 2023-10-06T02:29:20.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.221+0000 7ff6f8b99700 1 -- 172.21.15.31:0/3693738325 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6e0003570 con 0x7ff6f415dfd0 2023-10-06T02:29:20.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.222+0000 7ff6f9b9b700 1 -- 172.21.15.31:0/3693738325 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6f4159920 msgr2=0x7ff6f40ad090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:20.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.222+0000 7ff6f9b9b700 1 --2- 172.21.15.31:0/3693738325 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6f4159920 0x7ff6f40ad090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:20.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.222+0000 7ff6f9b9b700 1 -- 172.21.15.31:0/3693738325 >> v1:172.21.15.62:6789/0 conn(0x7ff6f415dfd0 legacy=0x7ff6f40aedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:20.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.222+0000 7ff6f9b9b700 1 -- 172.21.15.31:0/3693738325 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6f40ac5b0 con 0x7ff6f4159550 2023-10-06T02:29:20.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.222+0000 7ff6fa39c700 1 --2- 172.21.15.31:0/3693738325 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6f4159920 0x7ff6f40ad090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:29:20.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.222+0000 7ff6f9b9b700 1 --2- 172.21.15.31:0/3693738325 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6f4159550 0x7ff6f40ceac0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff6f00098e0 tx=0x7ff6f00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4081da4d18404d6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:20.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.222+0000 7ff6f8b99700 1 -- 172.21.15.31:0/3693738325 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f0011630 con 0x7ff6f4159550 2023-10-06T02:29:20.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.222+0000 7ff6f8b99700 1 -- 172.21.15.31:0/3693738325 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6f0011790 con 0x7ff6f4159550 2023-10-06T02:29:20.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.222+0000 7ff6f8b99700 1 -- 172.21.15.31:0/3693738325 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f0011b30 con 0x7ff6f4159550 2023-10-06T02:29:20.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.223+0000 7ff6fab9d700 1 -- 172.21.15.31:0/3693738325 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6f4159550 msgr2=0x7ff6f40ceac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:20.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.223+0000 7ff6fab9d700 1 --2- 172.21.15.31:0/3693738325 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6f4159550 0x7ff6f40ceac0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff6f00098e0 tx=0x7ff6f00091a0 comp rx=0 tx=0).stop 2023-10-06T02:29:20.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.223+0000 7ff6fab9d700 1 -- 172.21.15.31:0/3693738325 shutdown_connections 2023-10-06T02:29:20.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.223+0000 7ff6fab9d700 1 --2- 172.21.15.31:0/3693738325 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6f4159920 0x7ff6f40ad090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:20.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.223+0000 7ff6fab9d700 1 --2- 172.21.15.31:0/3693738325 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6f4159550 0x7ff6f40ceac0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:20.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.223+0000 7ff6fab9d700 1 -- 172.21.15.31:0/3693738325 >> 172.21.15.31:0/3693738325 conn(0x7ff6f406ce80 msgr2=0x7ff6f40ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:20.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.223+0000 7ff6fab9d700 1 -- 172.21.15.31:0/3693738325 shutdown_connections 2023-10-06T02:29:20.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.223+0000 7ff6fab9d700 1 -- 172.21.15.31:0/3693738325 wait complete. 2023-10-06T02:29:20.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.224+0000 7ff6fab9d700 1 Processor -- start 2023-10-06T02:29:20.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.225+0000 7ff6fab9d700 1 -- start start 2023-10-06T02:29:20.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.225+0000 7ff6fab9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6f4159550 0x7ff6f4151f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:20.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.225+0000 7ff6fab9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6f4159920 0x7ff6f4154890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:20.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.225+0000 7ff6fab9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6f415dfd0 0x7ff6f4154dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:20.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.225+0000 7ff6fab9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f40b4210 con 0x7ff6f4159550 2023-10-06T02:29:20.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.225+0000 7ff6fab9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f416b660 con 0x7ff6f4159920 2023-10-06T02:29:20.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.225+0000 7ff6fab9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f40b3200 con 0x7ff6f415dfd0 2023-10-06T02:29:20.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.226+0000 7ff6f9b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6f4159550 0x7ff6f4151f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:20.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.226+0000 7ff6f9b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6f4159550 0x7ff6f4151f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52766/0 (socket says 172.21.15.31:52766) 2023-10-06T02:29:20.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.226+0000 7ff6f9b9b700 1 -- 172.21.15.31:0/3981668368 learned_addr learned my addr 172.21.15.31:0/3981668368 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:20.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.226+0000 7ff6fa39c700 1 --2- 172.21.15.31:0/3981668368 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6f415dfd0 0x7ff6f4154dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:20.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.226+0000 7ff6f9b9b700 1 -- 172.21.15.31:0/3981668368 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6f415dfd0 msgr2=0x7ff6f4154dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:20.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.226+0000 7ff6f9b9b700 1 --2- 172.21.15.31:0/3981668368 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6f415dfd0 0x7ff6f4154dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:20.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.226+0000 7ff6f939a700 1 --2- 172.21.15.31:0/3981668368 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6f4159920 0x7ff6f4154890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:20.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.226+0000 7ff6f9b9b700 1 -- 172.21.15.31:0/3981668368 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6f4159920 msgr2=0x7ff6f4154890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:20.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.226+0000 7ff6f9b9b700 1 --2- 172.21.15.31:0/3981668368 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6f4159920 0x7ff6f4154890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:20.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.226+0000 7ff6f9b9b700 1 -- 172.21.15.31:0/3981668368 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6ec009a20 con 0x7ff6f4159550 2023-10-06T02:29:20.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.226+0000 7ff6fa39c700 1 --2- 172.21.15.31:0/3981668368 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6f415dfd0 0x7ff6f4154dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:29:20.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.226+0000 7ff6f939a700 1 --2- 172.21.15.31:0/3981668368 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6f4159920 0x7ff6f4154890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:29:20.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.227+0000 7ff6f9b9b700 1 --2- 172.21.15.31:0/3981668368 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6f4159550 0x7ff6f4151f00 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7ff6f406ad70 tx=0x7ff6f0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:20.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.227+0000 7ff6e9ffb700 1 -- 172.21.15.31:0/3981668368 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f0011b10 con 0x7ff6f4159550 2023-10-06T02:29:20.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.227+0000 7ff6e9ffb700 1 -- 172.21.15.31:0/3981668368 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6f00253f0 con 0x7ff6f4159550 2023-10-06T02:29:20.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.227+0000 7ff6e9ffb700 1 -- 172.21.15.31:0/3981668368 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f0025790 con 0x7ff6f4159550 2023-10-06T02:29:20.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.227+0000 7ff6fab9d700 1 -- 172.21.15.31:0/3981668368 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6f0009420 con 0x7ff6f4159550 2023-10-06T02:29:20.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.227+0000 7ff6fab9d700 1 -- 172.21.15.31:0/3981668368 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6f40b3d80 con 0x7ff6f4159550 2023-10-06T02:29:20.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.228+0000 7ff6fab9d700 1 -- 172.21.15.31:0/3981668368 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6f404c3c0 con 0x7ff6f4159550 2023-10-06T02:29:20.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.229+0000 7ff6e9ffb700 1 -- 172.21.15.31:0/3981668368 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) v1 ==== 85097+0+0 (secure 0 0 0) 0x7ff6f0025b40 con 0x7ff6f4159550 2023-10-06T02:29:20.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.230+0000 7ff6e9ffb700 1 --2- 172.21.15.31:0/3981668368 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff6d8067320 0x7ff6d80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:20.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.230+0000 7ff6e9ffb700 1 -- 172.21.15.31:0/3981668368 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6130+0+0 (secure 0 0 0) 0x7ff6f0086690 con 0x7ff6f4159550 2023-10-06T02:29:20.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.230+0000 7ff6f939a700 1 --2- 172.21.15.31:0/3981668368 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff6d8067320 0x7ff6d80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:20.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.231+0000 7ff6f939a700 1 --2- 172.21.15.31:0/3981668368 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff6d8067320 0x7ff6d80697d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff6e4007fd0 tx=0x7ff6e4009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:20.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.234+0000 7ff6e9ffb700 1 -- 172.21.15.31:0/3981668368 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff6f0053130 con 0x7ff6f4159550 2023-10-06T02:29:20.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.482+0000 7ff6fab9d700 1 -- 172.21.15.31:0/3981668368 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff6f40cab90 con 0x7ff6f4159550 2023-10-06T02:29:20.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.482+0000 7ff6e9ffb700 1 -- 172.21.15.31:0/3981668368 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+4596 (secure 0 0 0) 0x7ff6f0056750 con 0x7ff6f4159550 2023-10-06T02:29:20.484 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:29:20.484 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":72,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6026,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/3801334948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3801334948},{"type":"v1","addr":"172.21.15.62:6834","nonce":3801334948}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":6724,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/1804582836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":1804582836},{"type":"v1","addr":"172.21.15.31:6835","nonce":1804582836}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":6736,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1428432734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":1428432734},{"type":"v1","addr":"172.21.15.62:6835","nonce":1428432734}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:28:35.598388+0000","modified":"2023-10-06T02:28:39.126355+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7065},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7065":{"gid":7065,"name":"a","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6834/603135594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":603135594},{"type":"v1","addr":"172.21.15.31:6834","nonce":603135594}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-06T02:29:20.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.484+0000 7ff6fab9d700 1 -- 172.21.15.31:0/3981668368 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff6d8067320 msgr2=0x7ff6d80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:20.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.484+0000 7ff6fab9d700 1 --2- 172.21.15.31:0/3981668368 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff6d8067320 0x7ff6d80697d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff6e4007fd0 tx=0x7ff6e4009040 comp rx=0 tx=0).stop 2023-10-06T02:29:20.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.484+0000 7ff6fab9d700 1 -- 172.21.15.31:0/3981668368 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6f4159550 msgr2=0x7ff6f4151f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:20.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.484+0000 7ff6fab9d700 1 --2- 172.21.15.31:0/3981668368 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6f4159550 0x7ff6f4151f00 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7ff6f406ad70 tx=0x7ff6f0017000 comp rx=0 tx=0).stop 2023-10-06T02:29:20.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.484+0000 7ff6fab9d700 1 -- 172.21.15.31:0/3981668368 shutdown_connections 2023-10-06T02:29:20.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.484+0000 7ff6fab9d700 1 --2- 172.21.15.31:0/3981668368 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff6d8067320 0x7ff6d80697d0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:20.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.484+0000 7ff6fab9d700 1 --2- 172.21.15.31:0/3981668368 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6f415dfd0 0x7ff6f4154dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:20.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.484+0000 7ff6fab9d700 1 --2- 172.21.15.31:0/3981668368 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6f4159550 0x7ff6f4151f00 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:20.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.484+0000 7ff6fab9d700 1 --2- 172.21.15.31:0/3981668368 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6f4159920 0x7ff6f4154890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:20.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.484+0000 7ff6fab9d700 1 -- 172.21.15.31:0/3981668368 >> 172.21.15.31:0/3981668368 conn(0x7ff6f406ce80 msgr2=0x7ff6f4152c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:20.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.485+0000 7ff6fab9d700 1 -- 172.21.15.31:0/3981668368 shutdown_connections 2023-10-06T02:29:20.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.485+0000 7ff6fab9d700 1 -- 172.21.15.31:0/3981668368 wait complete. 2023-10-06T02:29:20.487 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 72 2023-10-06T02:29:20.503 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:29:20.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.689+0000 7fb23d19d700 1 Processor -- start 2023-10-06T02:29:20.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.689+0000 7fb23d19d700 1 -- start start 2023-10-06T02:29:20.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.690+0000 7fb23d19d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb238159570 0x7fb2380b23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:20.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.690+0000 7fb23d19d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb238159940 0x7fb23806d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:20.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.690+0000 7fb23d19d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2380b3270 con 0x7fb23815dea0 2023-10-06T02:29:20.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.690+0000 7fb23d19d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2380b3550 con 0x7fb238159570 2023-10-06T02:29:20.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.690+0000 7fb23d19d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2380b3830 con 0x7fb238159940 2023-10-06T02:29:20.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.691+0000 7fb23c99c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fb23815dea0 0x7fb2380b0080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:33032/0 (socket says 172.21.15.31:33032) 2023-10-06T02:29:20.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.691+0000 7fb23c99c700 1 -- 172.21.15.31:0/4257632117 learned_addr learned my addr 172.21.15.31:0/4257632117 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:29:20.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.691+0000 7fb237fff700 1 --2- 172.21.15.31:0/4257632117 >> v2:172.21.15.62:3300/0 conn(0x7fb238159570 0x7fb2380b23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:20.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.691+0000 7fb2377fe700 1 --2- 172.21.15.31:0/4257632117 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb238159940 0x7fb23806d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:20.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.691+0000 7fb236ffd700 1 -- 172.21.15.31:0/4257632117 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 358505609 0 0) 0x7fb2380b3270 con 0x7fb23815dea0 2023-10-06T02:29:20.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.691+0000 7fb236ffd700 1 -- 172.21.15.31:0/4257632117 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb220003570 con 0x7fb23815dea0 2023-10-06T02:29:20.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.691+0000 7fb237fff700 1 -- 172.21.15.31:0/4257632117 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb238159940 msgr2=0x7fb23806d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:20.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.691+0000 7fb237fff700 1 --2- 172.21.15.31:0/4257632117 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb238159940 0x7fb23806d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:20.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.691+0000 7fb237fff700 1 -- 172.21.15.31:0/4257632117 >> v1:172.21.15.31:6789/0 conn(0x7fb23815dea0 legacy=0x7fb2380b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:20.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.691+0000 7fb237fff700 1 -- 172.21.15.31:0/4257632117 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2380b3b10 con 0x7fb238159570 2023-10-06T02:29:20.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.692+0000 7fb237fff700 1 --2- 172.21.15.31:0/4257632117 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb238159570 0x7fb2380b23e0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fb2280098e0 tx=0x7fb2280091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bec7ef42f53d4340 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:20.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.692+0000 7fb236ffd700 1 -- 172.21.15.31:0/4257632117 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb228011630 con 0x7fb238159570 2023-10-06T02:29:20.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.692+0000 7fb236ffd700 1 -- 172.21.15.31:0/4257632117 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb228011790 con 0x7fb238159570 2023-10-06T02:29:20.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.692+0000 7fb236ffd700 1 -- 172.21.15.31:0/4257632117 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb228011b30 con 0x7fb238159570 2023-10-06T02:29:20.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.692+0000 7fb23d19d700 1 -- 172.21.15.31:0/4257632117 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb238159570 msgr2=0x7fb2380b23e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:20.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.692+0000 7fb23d19d700 1 --2- 172.21.15.31:0/4257632117 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb238159570 0x7fb2380b23e0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fb2280098e0 tx=0x7fb2280091a0 comp rx=0 tx=0).stop 2023-10-06T02:29:20.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.693+0000 7fb23d19d700 1 -- 172.21.15.31:0/4257632117 shutdown_connections 2023-10-06T02:29:20.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.693+0000 7fb23d19d700 1 --2- 172.21.15.31:0/4257632117 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb238159570 0x7fb2380b23e0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:20.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.693+0000 7fb23d19d700 1 --2- 172.21.15.31:0/4257632117 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb238159940 0x7fb23806d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:20.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.693+0000 7fb23d19d700 1 -- 172.21.15.31:0/4257632117 >> 172.21.15.31:0/4257632117 conn(0x7fb2380703a0 msgr2=0x7fb2380707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:20.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.693+0000 7fb23d19d700 1 -- 172.21.15.31:0/4257632117 shutdown_connections 2023-10-06T02:29:20.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.693+0000 7fb23d19d700 1 -- 172.21.15.31:0/4257632117 wait complete. 2023-10-06T02:29:20.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.694+0000 7fb23d19d700 1 Processor -- start 2023-10-06T02:29:20.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.694+0000 7fb23d19d700 1 -- start start 2023-10-06T02:29:20.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.695+0000 7fb23d19d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb238159570 0x7fb2380ce990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:20.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.695+0000 7fb23d19d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb238159940 0x7fb238149730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:20.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.695+0000 7fb23d19d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb23815dea0 0x7fb2380c9650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:20.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.695+0000 7fb23d19d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2380c7800 con 0x7fb23815dea0 2023-10-06T02:29:20.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.695+0000 7fb23d19d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2380cf9b0 con 0x7fb238159570 2023-10-06T02:29:20.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.695+0000 7fb23d19d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2380ceed0 con 0x7fb238159940 2023-10-06T02:29:20.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.695+0000 7fb23c99c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb23815dea0 0x7fb2380c9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:20.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.696+0000 7fb237fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb238159570 0x7fb2380ce990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:20.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.696+0000 7fb23c99c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb23815dea0 0x7fb2380c9650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52780/0 (socket says 172.21.15.31:52780) 2023-10-06T02:29:20.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.696+0000 7fb237fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb238159570 0x7fb2380ce990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:41850/0 (socket says 172.21.15.31:41850) 2023-10-06T02:29:20.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.696+0000 7fb23c99c700 1 -- 172.21.15.31:0/996478050 learned_addr learned my addr 172.21.15.31:0/996478050 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:20.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.696+0000 7fb2377fe700 1 --2- 172.21.15.31:0/996478050 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb238159940 0x7fb238149730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:20.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.696+0000 7fb23c99c700 1 -- 172.21.15.31:0/996478050 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb238159940 msgr2=0x7fb238149730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:20.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.696+0000 7fb23c99c700 1 --2- 172.21.15.31:0/996478050 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb238159940 0x7fb238149730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:20.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.696+0000 7fb23c99c700 1 -- 172.21.15.31:0/996478050 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb238159570 msgr2=0x7fb2380ce990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:20.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.696+0000 7fb23c99c700 1 --2- 172.21.15.31:0/996478050 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb238159570 0x7fb2380ce990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:20.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.696+0000 7fb23c99c700 1 -- 172.21.15.31:0/996478050 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb228009420 con 0x7fb23815dea0 2023-10-06T02:29:20.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.696+0000 7fb23c99c700 1 --2- 172.21.15.31:0/996478050 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb23815dea0 0x7fb2380c9650 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fb230001460 tx=0x7fb230001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:20.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.696+0000 7fb21ffff700 1 -- 172.21.15.31:0/996478050 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb23000a810 con 0x7fb23815dea0 2023-10-06T02:29:20.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.697+0000 7fb21ffff700 1 -- 172.21.15.31:0/996478050 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb23000a970 con 0x7fb23815dea0 2023-10-06T02:29:20.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.697+0000 7fb23d19d700 1 -- 172.21.15.31:0/996478050 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2380c7b90 con 0x7fb23815dea0 2023-10-06T02:29:20.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.697+0000 7fb21ffff700 1 -- 172.21.15.31:0/996478050 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb23000acf0 con 0x7fb23815dea0 2023-10-06T02:29:20.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.697+0000 7fb23d19d700 1 -- 172.21.15.31:0/996478050 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2380cf470 con 0x7fb23815dea0 2023-10-06T02:29:20.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.698+0000 7fb23d19d700 1 -- 172.21.15.31:0/996478050 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2380cf7f0 con 0x7fb23815dea0 2023-10-06T02:29:20.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.699+0000 7fb21ffff700 1 -- 172.21.15.31:0/996478050 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fb230007300 con 0x7fb23815dea0 2023-10-06T02:29:20.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.699+0000 7fb21ffff700 1 --2- 172.21.15.31:0/996478050 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb224067320 0x7fb2240697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:20.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.699+0000 7fb21ffff700 1 -- 172.21.15.31:0/996478050 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6130+0+0 (secure 0 0 0) 0x7fb230085250 con 0x7fb23815dea0 2023-10-06T02:29:20.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.700+0000 7fb237fff700 1 --2- 172.21.15.31:0/996478050 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb224067320 0x7fb2240697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:20.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.701+0000 7fb237fff700 1 --2- 172.21.15.31:0/996478050 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb224067320 0x7fb2240697d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb2280091e0 tx=0x7fb228017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:20.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.703+0000 7fb21ffff700 1 -- 172.21.15.31:0/996478050 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb230051c60 con 0x7fb23815dea0 2023-10-06T02:29:20.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.906+0000 7fb23d19d700 1 -- 172.21.15.31:0/996478050 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb2380c7d20 con 0x7fb23815dea0 2023-10-06T02:29:20.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.907+0000 7fb21ffff700 1 -- 172.21.15.31:0/996478050 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) v1 ==== 74+0+16966 (secure 0 0 0) 0x7fb230055280 con 0x7fb23815dea0 2023-10-06T02:29:20.909 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:29:20.909 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":108,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:29:17.059670+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:28:33.803742+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-10-06T02:28:34.570331+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":108,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":18,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":18,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:29:20.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.909+0000 7fb23d19d700 1 -- 172.21.15.31:0/996478050 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb224067320 msgr2=0x7fb2240697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:20.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.909+0000 7fb23d19d700 1 --2- 172.21.15.31:0/996478050 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb224067320 0x7fb2240697d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb2280091e0 tx=0x7fb228017000 comp rx=0 tx=0).stop 2023-10-06T02:29:20.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.909+0000 7fb23d19d700 1 -- 172.21.15.31:0/996478050 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb23815dea0 msgr2=0x7fb2380c9650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:20.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.909+0000 7fb23d19d700 1 --2- 172.21.15.31:0/996478050 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb23815dea0 0x7fb2380c9650 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fb230001460 tx=0x7fb230001a40 comp rx=0 tx=0).stop 2023-10-06T02:29:20.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.909+0000 7fb23d19d700 1 -- 172.21.15.31:0/996478050 shutdown_connections 2023-10-06T02:29:20.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.909+0000 7fb23d19d700 1 --2- 172.21.15.31:0/996478050 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb23815dea0 0x7fb2380c9650 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:20.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.909+0000 7fb23d19d700 1 --2- 172.21.15.31:0/996478050 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb238159570 0x7fb2380ce990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:20.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.909+0000 7fb23d19d700 1 --2- 172.21.15.31:0/996478050 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb224067320 0x7fb2240697d0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:20.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.909+0000 7fb23d19d700 1 --2- 172.21.15.31:0/996478050 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb238159940 0x7fb238149730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:20.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.909+0000 7fb23d19d700 1 -- 172.21.15.31:0/996478050 >> 172.21.15.31:0/996478050 conn(0x7fb2380703a0 msgr2=0x7fb2380b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:20.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.909+0000 7fb23d19d700 1 -- 172.21.15.31:0/996478050 shutdown_connections 2023-10-06T02:29:20.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:20.909+0000 7fb23d19d700 1 -- 172.21.15.31:0/996478050 wait complete. 2023-10-06T02:29:20.929 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T02:29:20.929 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:29:21.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.118+0000 7f247579d700 1 Processor -- start 2023-10-06T02:29:21.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.118+0000 7f247579d700 1 -- start start 2023-10-06T02:29:21.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.119+0000 7f247579d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f24700accb0 0x7f24700aaed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:21.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.119+0000 7f247579d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f24700cc340 0x7f247006ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:21.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.119+0000 7f247579d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24700ad780 con 0x7f24700cbea0 2023-10-06T02:29:21.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.119+0000 7f247579d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24700cee60 con 0x7f24700accb0 2023-10-06T02:29:21.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.119+0000 7f247579d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24700cf140 con 0x7f24700cc340 2023-10-06T02:29:21.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.119+0000 7f246ffff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f24700accb0 0x7f24700aaed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:21.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.119+0000 7f246ffff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f24700accb0 0x7f24700aaed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52792/0 (socket says 172.21.15.31:52792) 2023-10-06T02:29:21.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.119+0000 7f246ffff700 1 -- 172.21.15.31:0/3687745117 learned_addr learned my addr 172.21.15.31:0/3687745117 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:21.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.120+0000 7f246ffff700 1 -- 172.21.15.31:0/3687745117 >> v2:172.21.15.62:3300/0 conn(0x7f24700cc340 msgr2=0x7f247006ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:21.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.120+0000 7f246ffff700 1 --2- 172.21.15.31:0/3687745117 >> v2:172.21.15.62:3300/0 conn(0x7f24700cc340 0x7f247006ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:21.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.120+0000 7f246ffff700 1 -- 172.21.15.31:0/3687745117 >> v1:172.21.15.62:6789/0 conn(0x7f24700cbea0 legacy=0x7f247006e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:21.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.120+0000 7f246ffff700 1 -- 172.21.15.31:0/3687745117 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f247006d850 con 0x7f24700accb0 2023-10-06T02:29:21.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.120+0000 7f246ffff700 1 --2- 172.21.15.31:0/3687745117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24700accb0 0x7f24700aaed0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f247006ac60 tx=0x7f2460009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4d5fb6425818d2d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:21.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.120+0000 7f246effd700 1 -- 172.21.15.31:0/3687745117 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f246000b820 con 0x7f24700accb0 2023-10-06T02:29:21.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.120+0000 7f246effd700 1 -- 172.21.15.31:0/3687745117 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f246000b980 con 0x7f24700accb0 2023-10-06T02:29:21.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.120+0000 7f246effd700 1 -- 172.21.15.31:0/3687745117 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f246000bd20 con 0x7f24700accb0 2023-10-06T02:29:21.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.121+0000 7f247579d700 1 -- 172.21.15.31:0/3687745117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24700accb0 msgr2=0x7f24700aaed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:21.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.121+0000 7f247579d700 1 --2- 172.21.15.31:0/3687745117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24700accb0 0x7f24700aaed0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f247006ac60 tx=0x7f2460009250 comp rx=0 tx=0).stop 2023-10-06T02:29:21.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.121+0000 7f247579d700 1 -- 172.21.15.31:0/3687745117 shutdown_connections 2023-10-06T02:29:21.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.121+0000 7f247579d700 1 --2- 172.21.15.31:0/3687745117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24700accb0 0x7f24700aaed0 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:21.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.121+0000 7f247579d700 1 --2- 172.21.15.31:0/3687745117 >> v2:172.21.15.62:3300/0 conn(0x7f24700cc340 0x7f247006ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:21.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.121+0000 7f247579d700 1 -- 172.21.15.31:0/3687745117 >> 172.21.15.31:0/3687745117 conn(0x7f247006ce80 msgr2=0x7f24700ce370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:21.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.121+0000 7f247579d700 1 -- 172.21.15.31:0/3687745117 shutdown_connections 2023-10-06T02:29:21.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.121+0000 7f247579d700 1 -- 172.21.15.31:0/3687745117 wait complete. 2023-10-06T02:29:21.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.123+0000 7f247579d700 1 Processor -- start 2023-10-06T02:29:21.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.123+0000 7f247579d700 1 -- start start 2023-10-06T02:29:21.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.124+0000 7f247579d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24700accb0 0x7f2470169f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:21.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.124+0000 7f247579d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24700cbea0 0x7f24700ce330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:21.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.124+0000 7f247579d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f24700cc340 0x7f24700c4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:21.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.124+0000 7f247579d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f247016a800 con 0x7f24700cbea0 2023-10-06T02:29:21.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.124+0000 7f247579d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f247016aae0 con 0x7f24700accb0 2023-10-06T02:29:21.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.124+0000 7f247579d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f247016adc0 con 0x7f24700cc340 2023-10-06T02:29:21.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.124+0000 7f246f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24700cbea0 0x7f24700ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:21.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.124+0000 7f246f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24700cbea0 0x7f24700ce330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52808/0 (socket says 172.21.15.31:52808) 2023-10-06T02:29:21.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.124+0000 7f246f7fe700 1 -- 172.21.15.31:0/4657597 learned_addr learned my addr 172.21.15.31:0/4657597 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:21.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.124+0000 7f246ffff700 1 --2- 172.21.15.31:0/4657597 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24700accb0 0x7f2470169f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:21.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.124+0000 7f2474f9c700 1 --2- 172.21.15.31:0/4657597 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f24700cc340 0x7f24700c4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:21.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.124+0000 7f246f7fe700 1 -- 172.21.15.31:0/4657597 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f24700cc340 msgr2=0x7f24700c4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:21.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.124+0000 7f246f7fe700 1 --2- 172.21.15.31:0/4657597 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f24700cc340 0x7f24700c4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:21.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.124+0000 7f246f7fe700 1 -- 172.21.15.31:0/4657597 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24700accb0 msgr2=0x7f2470169f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:21.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.124+0000 7f246f7fe700 1 --2- 172.21.15.31:0/4657597 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24700accb0 0x7f2470169f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:21.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.124+0000 7f246f7fe700 1 -- 172.21.15.31:0/4657597 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24600094f0 con 0x7f24700cbea0 2023-10-06T02:29:21.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.124+0000 7f2474f9c700 1 --2- 172.21.15.31:0/4657597 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f24700cc340 0x7f24700c4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:29:21.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.125+0000 7f246f7fe700 1 --2- 172.21.15.31:0/4657597 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24700cbea0 0x7f24700ce330 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f246400b5e0 tx=0x7f24640090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:21.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.125+0000 7f2457fff700 1 -- 172.21.15.31:0/4657597 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f246400f510 con 0x7f24700cbea0 2023-10-06T02:29:21.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.125+0000 7f2457fff700 1 -- 172.21.15.31:0/4657597 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f246400f670 con 0x7f24700cbea0 2023-10-06T02:29:21.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.125+0000 7f2457fff700 1 -- 172.21.15.31:0/4657597 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f246400fa30 con 0x7f24700cbea0 2023-10-06T02:29:21.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.125+0000 7f247579d700 1 -- 172.21.15.31:0/4657597 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24700c3d70 con 0x7f24700cbea0 2023-10-06T02:29:21.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.125+0000 7f247579d700 1 -- 172.21.15.31:0/4657597 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f247016a480 con 0x7f24700cbea0 2023-10-06T02:29:21.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.126+0000 7f247579d700 1 -- 172.21.15.31:0/4657597 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f247004c3c0 con 0x7f24700cbea0 2023-10-06T02:29:21.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.128+0000 7f2457fff700 1 -- 172.21.15.31:0/4657597 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f246401a400 con 0x7f24700cbea0 2023-10-06T02:29:21.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.128+0000 7f2457fff700 1 --2- 172.21.15.31:0/4657597 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f245c067320 0x7f245c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:21.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.128+0000 7f2457fff700 1 -- 172.21.15.31:0/4657597 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f24640848d0 con 0x7f24700cbea0 2023-10-06T02:29:21.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.129+0000 7f246ffff700 1 --2- 172.21.15.31:0/4657597 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f245c067320 0x7f245c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:21.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.130+0000 7f246ffff700 1 --2- 172.21.15.31:0/4657597 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f245c067320 0x7f245c0697d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f247006ac60 tx=0x7f2460016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:21.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.132+0000 7f2457fff700 1 -- 172.21.15.31:0/4657597 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2464051450 con 0x7f24700cbea0 2023-10-06T02:29:21.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.378+0000 7f247579d700 1 -- 172.21.15.31:0/4657597 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f247006db30 con 0x7f24700cbea0 2023-10-06T02:29:21.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.379+0000 7f2457fff700 1 -- 172.21.15.31:0/4657597 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f2464054a70 con 0x7f24700cbea0 2023-10-06T02:29:21.380 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:29:21.381 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":72,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6026,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/3801334948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3801334948},{"type":"v1","addr":"172.21.15.62:6834","nonce":3801334948}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":6724,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/1804582836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":1804582836},{"type":"v1","addr":"172.21.15.31:6835","nonce":1804582836}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":6736,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1428432734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":1428432734},{"type":"v1","addr":"172.21.15.62:6835","nonce":1428432734}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:28:35.598388+0000","modified":"2023-10-06T02:28:39.126355+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7065},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7065":{"gid":7065,"name":"a","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6834/603135594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":603135594},{"type":"v1","addr":"172.21.15.31:6834","nonce":603135594}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-06T02:29:21.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.381+0000 7f247579d700 1 -- 172.21.15.31:0/4657597 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f245c067320 msgr2=0x7f245c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:21.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.381+0000 7f247579d700 1 --2- 172.21.15.31:0/4657597 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f245c067320 0x7f245c0697d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f247006ac60 tx=0x7f2460016000 comp rx=0 tx=0).stop 2023-10-06T02:29:21.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.381+0000 7f247579d700 1 -- 172.21.15.31:0/4657597 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24700cbea0 msgr2=0x7f24700ce330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:21.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.381+0000 7f247579d700 1 --2- 172.21.15.31:0/4657597 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24700cbea0 0x7f24700ce330 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f246400b5e0 tx=0x7f24640090f0 comp rx=0 tx=0).stop 2023-10-06T02:29:21.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.381+0000 7f247579d700 1 -- 172.21.15.31:0/4657597 shutdown_connections 2023-10-06T02:29:21.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.381+0000 7f247579d700 1 --2- 172.21.15.31:0/4657597 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f245c067320 0x7f245c0697d0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:21.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.381+0000 7f247579d700 1 --2- 172.21.15.31:0/4657597 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f24700cc340 0x7f24700c4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:21.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.381+0000 7f247579d700 1 --2- 172.21.15.31:0/4657597 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24700accb0 0x7f2470169f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:21.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.381+0000 7f247579d700 1 --2- 172.21.15.31:0/4657597 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24700cbea0 0x7f24700ce330 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:21.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.381+0000 7f247579d700 1 -- 172.21.15.31:0/4657597 >> 172.21.15.31:0/4657597 conn(0x7f247006ce80 msgr2=0x7f24701594b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:21.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.381+0000 7f247579d700 1 -- 172.21.15.31:0/4657597 shutdown_connections 2023-10-06T02:29:21.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.381+0000 7f247579d700 1 -- 172.21.15.31:0/4657597 wait complete. 2023-10-06T02:29:21.384 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 72 2023-10-06T02:29:21.400 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:29:21.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.585+0000 7f1e3f59e700 1 Processor -- start 2023-10-06T02:29:21.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.585+0000 7f1e3f59e700 1 -- start start 2023-10-06T02:29:21.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.586+0000 7f1e3f59e700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f1e40142c00 0x7f1e4006b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:21.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.586+0000 7f1e3f59e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e4006bd20 0x7f1e400c7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:21.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.587+0000 7f1e3f59e700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e400ca480 con 0x7f1e40142fd0 2023-10-06T02:29:21.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.587+0000 7f1e3f59e700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e4015e240 con 0x7f1e40142c00 2023-10-06T02:29:21.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.587+0000 7f1e3f59e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4006b350 con 0x7f1e4006bd20 2023-10-06T02:29:21.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.587+0000 7f1e3ed9d700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f1e40142fd0 0x7f1e4006c6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:33034/0 (socket says 172.21.15.31:33034) 2023-10-06T02:29:21.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.587+0000 7f1e3ed9d700 1 -- 172.21.15.31:0/42324880 learned_addr learned my addr 172.21.15.31:0/42324880 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:29:21.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.587+0000 7f1e3e59c700 1 --2- 172.21.15.31:0/42324880 >> v2:172.21.15.62:3300/0 conn(0x7f1e40142c00 0x7f1e4006b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:21.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.587+0000 7f1e3dd9b700 1 --2- 172.21.15.31:0/42324880 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e4006bd20 0x7f1e400c7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:21.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.587+0000 7f1e3d59a700 1 -- 172.21.15.31:0/42324880 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3246494619 0 0) 0x7f1e400ca480 con 0x7f1e40142fd0 2023-10-06T02:29:21.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.587+0000 7f1e3d59a700 1 -- 172.21.15.31:0/42324880 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e28003570 con 0x7f1e40142fd0 2023-10-06T02:29:21.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.587+0000 7f1e3e59c700 1 -- 172.21.15.31:0/42324880 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e4006bd20 msgr2=0x7f1e400c7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:21.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.587+0000 7f1e3e59c700 1 --2- 172.21.15.31:0/42324880 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e4006bd20 0x7f1e400c7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:21.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.587+0000 7f1e3e59c700 1 -- 172.21.15.31:0/42324880 >> v1:172.21.15.31:6789/0 conn(0x7f1e40142fd0 legacy=0x7f1e4006c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:21.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.588+0000 7f1e3e59c700 1 -- 172.21.15.31:0/42324880 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e4015df60 con 0x7f1e40142c00 2023-10-06T02:29:21.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.588+0000 7f1e3e59c700 1 --2- 172.21.15.31:0/42324880 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e40142c00 0x7f1e4006b7e0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f1e300098e0 tx=0x7f1e300091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a3a4317c52dbe07 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:21.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.588+0000 7f1e3d59a700 1 -- 172.21.15.31:0/42324880 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e30011630 con 0x7f1e40142c00 2023-10-06T02:29:21.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.588+0000 7f1e3d59a700 1 -- 172.21.15.31:0/42324880 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e30011790 con 0x7f1e40142c00 2023-10-06T02:29:21.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.588+0000 7f1e3d59a700 1 -- 172.21.15.31:0/42324880 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e30011b30 con 0x7f1e40142c00 2023-10-06T02:29:21.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.589+0000 7f1e3f59e700 1 -- 172.21.15.31:0/42324880 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e40142c00 msgr2=0x7f1e4006b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:21.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.589+0000 7f1e3f59e700 1 --2- 172.21.15.31:0/42324880 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e40142c00 0x7f1e4006b7e0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f1e300098e0 tx=0x7f1e300091a0 comp rx=0 tx=0).stop 2023-10-06T02:29:21.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.589+0000 7f1e3f59e700 1 -- 172.21.15.31:0/42324880 shutdown_connections 2023-10-06T02:29:21.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.589+0000 7f1e3f59e700 1 --2- 172.21.15.31:0/42324880 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e40142c00 0x7f1e4006b7e0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:21.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.589+0000 7f1e3f59e700 1 --2- 172.21.15.31:0/42324880 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e4006bd20 0x7f1e400c7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:21.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.589+0000 7f1e3f59e700 1 -- 172.21.15.31:0/42324880 >> 172.21.15.31:0/42324880 conn(0x7f1e400675a0 msgr2=0x7f1e400679a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:21.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.589+0000 7f1e3f59e700 1 -- 172.21.15.31:0/42324880 shutdown_connections 2023-10-06T02:29:21.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.589+0000 7f1e3f59e700 1 -- 172.21.15.31:0/42324880 wait complete. 2023-10-06T02:29:21.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.590+0000 7f1e3f59e700 1 Processor -- start 2023-10-06T02:29:21.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.590+0000 7f1e3f59e700 1 -- start start 2023-10-06T02:29:21.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.591+0000 7f1e3f59e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e4006bd20 0x7f1e4013c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:21.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.591+0000 7f1e3f59e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e40142c00 0x7f1e4015e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:21.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.591+0000 7f1e3f59e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e40142fd0 0x7f1e40158f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:21.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.591+0000 7f1e3f59e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4015e8c0 con 0x7f1e4006bd20 2023-10-06T02:29:21.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.591+0000 7f1e3f59e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4015eb20 con 0x7f1e40142fd0 2023-10-06T02:29:21.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.591+0000 7f1e3f59e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4015ee00 con 0x7f1e40142c00 2023-10-06T02:29:21.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.591+0000 7f1e3e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e4006bd20 0x7f1e4013c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:21.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.592+0000 7f1e3e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e4006bd20 0x7f1e4013c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52820/0 (socket says 172.21.15.31:52820) 2023-10-06T02:29:21.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.592+0000 7f1e3e59c700 1 -- 172.21.15.31:0/119473511 learned_addr learned my addr 172.21.15.31:0/119473511 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:21.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.592+0000 7f1e3dd9b700 1 --2- 172.21.15.31:0/119473511 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e40142c00 0x7f1e4015e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:21.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.592+0000 7f1e3ed9d700 1 --2- 172.21.15.31:0/119473511 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e40142fd0 0x7f1e40158f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:21.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.592+0000 7f1e3e59c700 1 -- 172.21.15.31:0/119473511 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e40142c00 msgr2=0x7f1e4015e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:21.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.592+0000 7f1e3e59c700 1 --2- 172.21.15.31:0/119473511 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e40142c00 0x7f1e4015e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:21.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.592+0000 7f1e3e59c700 1 -- 172.21.15.31:0/119473511 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e40142fd0 msgr2=0x7f1e40158f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:21.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.592+0000 7f1e3e59c700 1 --2- 172.21.15.31:0/119473511 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e40142fd0 0x7f1e40158f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:21.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.592+0000 7f1e3e59c700 1 -- 172.21.15.31:0/119473511 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e30009420 con 0x7f1e4006bd20 2023-10-06T02:29:21.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.592+0000 7f1e3dd9b700 1 --2- 172.21.15.31:0/119473511 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e40142c00 0x7f1e4015e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:29:21.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.592+0000 7f1e3e59c700 1 --2- 172.21.15.31:0/119473511 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e4006bd20 0x7f1e4013c840 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f1e30009ed0 tx=0x7f1e30017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:21.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.593+0000 7f1e2e7fc700 1 -- 172.21.15.31:0/119473511 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e30011b10 con 0x7f1e4006bd20 2023-10-06T02:29:21.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.593+0000 7f1e2e7fc700 1 -- 172.21.15.31:0/119473511 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e30006af0 con 0x7f1e4006bd20 2023-10-06T02:29:21.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.593+0000 7f1e3f59e700 1 -- 172.21.15.31:0/119473511 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e4015f0e0 con 0x7f1e4006bd20 2023-10-06T02:29:21.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.593+0000 7f1e2e7fc700 1 -- 172.21.15.31:0/119473511 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e30025640 con 0x7f1e4006bd20 2023-10-06T02:29:21.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.593+0000 7f1e3f59e700 1 -- 172.21.15.31:0/119473511 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e40169990 con 0x7f1e4006bd20 2023-10-06T02:29:21.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.594+0000 7f1e3f59e700 1 -- 172.21.15.31:0/119473511 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e40169ee0 con 0x7f1e4006bd20 2023-10-06T02:29:21.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.595+0000 7f1e2e7fc700 1 -- 172.21.15.31:0/119473511 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f1e30025a70 con 0x7f1e4006bd20 2023-10-06T02:29:21.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.596+0000 7f1e2e7fc700 1 --2- 172.21.15.31:0/119473511 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f1e20067320 0x7f1e200697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:21.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.597+0000 7f1e2e7fc700 1 -- 172.21.15.31:0/119473511 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f1e300866b0 con 0x7f1e4006bd20 2023-10-06T02:29:21.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.597+0000 7f1e3dd9b700 1 --2- 172.21.15.31:0/119473511 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f1e20067320 0x7f1e200697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:21.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.598+0000 7f1e3dd9b700 1 --2- 172.21.15.31:0/119473511 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f1e20067320 0x7f1e200697d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1e4006aff0 tx=0x7f1e34008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:21.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.601+0000 7f1e2e7fc700 1 -- 172.21.15.31:0/119473511 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1e40169ee0 con 0x7f1e4006bd20 2023-10-06T02:29:21.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.846+0000 7f1e3f59e700 1 -- 172.21.15.31:0/119473511 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1e40142490 con 0x7f1e4006bd20 2023-10-06T02:29:21.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.847+0000 7f1e2e7fc700 1 -- 172.21.15.31:0/119473511 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f1e30053150 con 0x7f1e4006bd20 2023-10-06T02:29:21.848 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:29:21.848 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":72,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6026,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/3801334948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3801334948},{"type":"v1","addr":"172.21.15.62:6834","nonce":3801334948}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":6724,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/1804582836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":1804582836},{"type":"v1","addr":"172.21.15.31:6835","nonce":1804582836}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":6736,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1428432734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":1428432734},{"type":"v1","addr":"172.21.15.62:6835","nonce":1428432734}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:28:35.598388+0000","modified":"2023-10-06T02:28:39.126355+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7065},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7065":{"gid":7065,"name":"a","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6834/603135594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":603135594},{"type":"v1","addr":"172.21.15.31:6834","nonce":603135594}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-06T02:29:21.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.849+0000 7f1e3f59e700 1 -- 172.21.15.31:0/119473511 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f1e20067320 msgr2=0x7f1e200697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:21.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.849+0000 7f1e3f59e700 1 --2- 172.21.15.31:0/119473511 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f1e20067320 0x7f1e200697d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1e4006aff0 tx=0x7f1e34008040 comp rx=0 tx=0).stop 2023-10-06T02:29:21.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.849+0000 7f1e3f59e700 1 -- 172.21.15.31:0/119473511 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e4006bd20 msgr2=0x7f1e4013c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:21.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.849+0000 7f1e3f59e700 1 --2- 172.21.15.31:0/119473511 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e4006bd20 0x7f1e4013c840 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f1e30009ed0 tx=0x7f1e30017000 comp rx=0 tx=0).stop 2023-10-06T02:29:21.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.849+0000 7f1e3f59e700 1 -- 172.21.15.31:0/119473511 shutdown_connections 2023-10-06T02:29:21.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.849+0000 7f1e3f59e700 1 --2- 172.21.15.31:0/119473511 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e40142fd0 0x7f1e40158f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:21.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.849+0000 7f1e3f59e700 1 --2- 172.21.15.31:0/119473511 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e4006bd20 0x7f1e4013c840 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:21.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.849+0000 7f1e3f59e700 1 --2- 172.21.15.31:0/119473511 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f1e20067320 0x7f1e200697d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:21.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.849+0000 7f1e3f59e700 1 --2- 172.21.15.31:0/119473511 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e40142c00 0x7f1e4015e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:21.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.849+0000 7f1e3f59e700 1 -- 172.21.15.31:0/119473511 >> 172.21.15.31:0/119473511 conn(0x7f1e400675a0 msgr2=0x7f1e400b03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:21.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.849+0000 7f1e3f59e700 1 -- 172.21.15.31:0/119473511 shutdown_connections 2023-10-06T02:29:21.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:21.849+0000 7f1e3f59e700 1 -- 172.21.15.31:0/119473511 wait complete. 2023-10-06T02:29:21.852 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 72 2023-10-06T02:29:21.868 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:29:22.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.053+0000 7ffa9bfff700 1 Processor -- start 2023-10-06T02:29:22.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.053+0000 7ffa9bfff700 1 -- start start 2023-10-06T02:29:22.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.054+0000 7ffa9bfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ffa9c15de50 0x7ffa9c0cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:22.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.054+0000 7ffa9bfff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa9c0cf3c0 con 0x7ffa9c0cef40 2023-10-06T02:29:22.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.054+0000 7ffa9bfff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa9c15afd0 con 0x7ffa9c153c30 2023-10-06T02:29:22.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.054+0000 7ffa9bfff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa9c15b2b0 con 0x7ffa9c15de50 2023-10-06T02:29:22.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.054+0000 7ffa9a7fc700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ffa9c15de50 0x7ffa9c0cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:22.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.055+0000 7ffa9a7fc700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ffa9c15de50 0x7ffa9c0cea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52824/0 (socket says 172.21.15.31:52824) 2023-10-06T02:29:22.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.055+0000 7ffa9b7fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7ffa9c0cef40 0x7ffa9c15c070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:33046/0 (socket says 172.21.15.31:33046) 2023-10-06T02:29:22.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.055+0000 7ffa9a7fc700 1 -- 172.21.15.31:0/2770477376 learned_addr learned my addr 172.21.15.31:0/2770477376 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:22.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.055+0000 7ffa9a7fc700 1 -- 172.21.15.31:0/2770477376 >> v1:172.21.15.62:6789/0 conn(0x7ffa9c153c30 legacy=0x7ffa9c15a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:22.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.055+0000 7ffa9a7fc700 1 -- 172.21.15.31:0/2770477376 >> v1:172.21.15.31:6789/0 conn(0x7ffa9c0cef40 legacy=0x7ffa9c15c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:22.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.055+0000 7ffa9a7fc700 1 -- 172.21.15.31:0/2770477376 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa9c15b590 con 0x7ffa9c15de50 2023-10-06T02:29:22.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.055+0000 7ffa99ffb700 1 -- 172.21.15.31:0/2770477376 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4105918660 0 0) 0x7ffa9c0cf3c0 con 0x7ffa9c0cef40 2023-10-06T02:29:22.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.055+0000 7ffa9a7fc700 1 --2- 172.21.15.31:0/2770477376 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa9c15de50 0x7ffa9c0cea00 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7ffa900099b0 tx=0x7ffa90009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66630387c36f0cef server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:22.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.055+0000 7ffa99ffb700 1 -- 172.21.15.31:0/2770477376 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa9000b860 con 0x7ffa9c15de50 2023-10-06T02:29:22.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.056+0000 7ffa99ffb700 1 -- 172.21.15.31:0/2770477376 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa9000b9c0 con 0x7ffa9c15de50 2023-10-06T02:29:22.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.056+0000 7ffa99ffb700 1 -- 172.21.15.31:0/2770477376 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa9000bd60 con 0x7ffa9c15de50 2023-10-06T02:29:22.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.056+0000 7ffa9bfff700 1 -- 172.21.15.31:0/2770477376 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa9c15de50 msgr2=0x7ffa9c0cea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:22.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.056+0000 7ffa9bfff700 1 --2- 172.21.15.31:0/2770477376 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa9c15de50 0x7ffa9c0cea00 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7ffa900099b0 tx=0x7ffa90009250 comp rx=0 tx=0).stop 2023-10-06T02:29:22.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.056+0000 7ffa9bfff700 1 -- 172.21.15.31:0/2770477376 shutdown_connections 2023-10-06T02:29:22.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.056+0000 7ffa9bfff700 1 --2- 172.21.15.31:0/2770477376 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa9c15de50 0x7ffa9c0cea00 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:22.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.056+0000 7ffa9bfff700 1 -- 172.21.15.31:0/2770477376 >> 172.21.15.31:0/2770477376 conn(0x7ffa9c0703a0 msgr2=0x7ffa9c0707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:22.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.056+0000 7ffa9bfff700 1 -- 172.21.15.31:0/2770477376 shutdown_connections 2023-10-06T02:29:22.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.056+0000 7ffa9bfff700 1 -- 172.21.15.31:0/2770477376 wait complete. 2023-10-06T02:29:22.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.058+0000 7ffa9bfff700 1 Processor -- start 2023-10-06T02:29:22.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.058+0000 7ffa9bfff700 1 -- start start 2023-10-06T02:29:22.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.059+0000 7ffa9bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa9c0cef40 0x7ffa9c16c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:22.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.059+0000 7ffa9bfff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffa9c153c30 0x7ffa9c0b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:22.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.059+0000 7ffa9bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa9c15de50 0x7ffa9c0d0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:22.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.059+0000 7ffa9bfff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa9c13eac0 con 0x7ffa9c15de50 2023-10-06T02:29:22.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.059+0000 7ffa9bfff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa9c0b5f80 con 0x7ffa9c0cef40 2023-10-06T02:29:22.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.059+0000 7ffa9bfff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa9c0b50d0 con 0x7ffa9c153c30 2023-10-06T02:29:22.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.059+0000 7ffa9b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa9c15de50 0x7ffa9c0d0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:22.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.059+0000 7ffa9b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa9c15de50 0x7ffa9c0d0450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52830/0 (socket says 172.21.15.31:52830) 2023-10-06T02:29:22.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.059+0000 7ffa9b7fe700 1 -- 172.21.15.31:0/3898369108 learned_addr learned my addr 172.21.15.31:0/3898369108 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:22.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.059+0000 7ffa9a7fc700 1 --2- 172.21.15.31:0/3898369108 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffa9c153c30 0x7ffa9c0b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:22.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.059+0000 7ffa9affd700 1 --2- 172.21.15.31:0/3898369108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa9c0cef40 0x7ffa9c16c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:22.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.059+0000 7ffa9b7fe700 1 -- 172.21.15.31:0/3898369108 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffa9c153c30 msgr2=0x7ffa9c0b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:22.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.059+0000 7ffa9b7fe700 1 --2- 172.21.15.31:0/3898369108 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffa9c153c30 0x7ffa9c0b3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:22.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.059+0000 7ffa9b7fe700 1 -- 172.21.15.31:0/3898369108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa9c0cef40 msgr2=0x7ffa9c16c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:22.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.059+0000 7ffa9b7fe700 1 --2- 172.21.15.31:0/3898369108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa9c0cef40 0x7ffa9c16c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:22.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.060+0000 7ffa9b7fe700 1 -- 172.21.15.31:0/3898369108 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa900094f0 con 0x7ffa9c15de50 2023-10-06T02:29:22.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.060+0000 7ffa9b7fe700 1 --2- 172.21.15.31:0/3898369108 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa9c15de50 0x7ffa9c0d0450 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7ffa9400ccc0 tx=0x7ffa94001a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:22.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.060+0000 7ffa82ffd700 1 -- 172.21.15.31:0/3898369108 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa94005df0 con 0x7ffa9c15de50 2023-10-06T02:29:22.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.060+0000 7ffa82ffd700 1 -- 172.21.15.31:0/3898369108 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa9400a670 con 0x7ffa9c15de50 2023-10-06T02:29:22.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.060+0000 7ffa82ffd700 1 -- 172.21.15.31:0/3898369108 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa9400aa30 con 0x7ffa9c15de50 2023-10-06T02:29:22.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.060+0000 7ffa9bfff700 1 -- 172.21.15.31:0/3898369108 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa9c0b5370 con 0x7ffa9c15de50 2023-10-06T02:29:22.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.060+0000 7ffa9bfff700 1 -- 172.21.15.31:0/3898369108 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa9c0b59b0 con 0x7ffa9c15de50 2023-10-06T02:29:22.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.061+0000 7ffa9bfff700 1 -- 172.21.15.31:0/3898369108 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa9c0b60f0 con 0x7ffa9c15de50 2023-10-06T02:29:22.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.062+0000 7ffa82ffd700 1 -- 172.21.15.31:0/3898369108 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) v1 ==== 85097+0+0 (secure 0 0 0) 0x7ffa940070f0 con 0x7ffa9c15de50 2023-10-06T02:29:22.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.063+0000 7ffa82ffd700 1 --2- 172.21.15.31:0/3898369108 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ffa88067320 0x7ffa880697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:22.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.063+0000 7ffa82ffd700 1 -- 172.21.15.31:0/3898369108 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6130+0+0 (secure 0 0 0) 0x7ffa94084e10 con 0x7ffa9c15de50 2023-10-06T02:29:22.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.064+0000 7ffa9affd700 1 --2- 172.21.15.31:0/3898369108 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ffa88067320 0x7ffa880697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:22.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.064+0000 7ffa9affd700 1 --2- 172.21.15.31:0/3898369108 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ffa88067320 0x7ffa880697d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c003a80 tx=0x7ffa8c007ec0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:22.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.067+0000 7ffa82ffd700 1 -- 172.21.15.31:0/3898369108 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffa94051930 con 0x7ffa9c15de50 2023-10-06T02:29:22.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.270+0000 7ffa9bfff700 1 -- 172.21.15.31:0/3898369108 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffa9c04c3c0 con 0x7ffa9c15de50 2023-10-06T02:29:22.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.271+0000 7ffa82ffd700 1 -- 172.21.15.31:0/3898369108 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) v1 ==== 74+0+16966 (secure 0 0 0) 0x7ffa94054f50 con 0x7ffa9c15de50 2023-10-06T02:29:22.273 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:29:22.273 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":108,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:29:17.059670+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:28:33.803742+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-10-06T02:28:34.570331+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":108,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":18,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":18,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:29:22.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.273+0000 7ffa9bfff700 1 -- 172.21.15.31:0/3898369108 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ffa88067320 msgr2=0x7ffa880697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:22.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.273+0000 7ffa9bfff700 1 --2- 172.21.15.31:0/3898369108 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ffa88067320 0x7ffa880697d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c003a80 tx=0x7ffa8c007ec0 comp rx=0 tx=0).stop 2023-10-06T02:29:22.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.273+0000 7ffa9bfff700 1 -- 172.21.15.31:0/3898369108 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa9c15de50 msgr2=0x7ffa9c0d0450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:22.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.273+0000 7ffa9bfff700 1 --2- 172.21.15.31:0/3898369108 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa9c15de50 0x7ffa9c0d0450 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7ffa9400ccc0 tx=0x7ffa94001a20 comp rx=0 tx=0).stop 2023-10-06T02:29:22.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.274+0000 7ffa9bfff700 1 -- 172.21.15.31:0/3898369108 shutdown_connections 2023-10-06T02:29:22.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.274+0000 7ffa9bfff700 1 --2- 172.21.15.31:0/3898369108 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa9c15de50 0x7ffa9c0d0450 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:22.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.274+0000 7ffa9bfff700 1 --2- 172.21.15.31:0/3898369108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa9c0cef40 0x7ffa9c16c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:22.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.274+0000 7ffa9bfff700 1 --2- 172.21.15.31:0/3898369108 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ffa88067320 0x7ffa880697d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:22.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.274+0000 7ffa9bfff700 1 --2- 172.21.15.31:0/3898369108 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffa9c153c30 0x7ffa9c0b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:22.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.274+0000 7ffa9bfff700 1 -- 172.21.15.31:0/3898369108 >> 172.21.15.31:0/3898369108 conn(0x7ffa9c0703a0 msgr2=0x7ffa9c0c5ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:22.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.274+0000 7ffa9bfff700 1 -- 172.21.15.31:0/3898369108 shutdown_connections 2023-10-06T02:29:22.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.274+0000 7ffa9bfff700 1 -- 172.21.15.31:0/3898369108 wait complete. 2023-10-06T02:29:22.293 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T02:29:22.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.480+0000 7f294a99d700 1 Processor -- start 2023-10-06T02:29:22.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.480+0000 7f294a99d700 1 -- start start 2023-10-06T02:29:22.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.481+0000 7f294a99d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f294406f3a0 0x7f294414fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:22.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.481+0000 7f294a99d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f294406a490 con 0x7f294406e470 2023-10-06T02:29:22.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.481+0000 7f294a99d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29440c7230 con 0x7f294406e0a0 2023-10-06T02:29:22.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.481+0000 7f294a99d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29440c7490 con 0x7f294406f3a0 2023-10-06T02:29:22.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.481+0000 7f294a19c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f294406e470 0x7f294406a920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:33054/0 (socket says 172.21.15.31:33054) 2023-10-06T02:29:22.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.481+0000 7f294a19c700 1 -- 172.21.15.31:0/1013372843 learned_addr learned my addr 172.21.15.31:0/1013372843 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:29:22.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.482+0000 7f294919a700 1 --2- 172.21.15.31:0/1013372843 >> v2:172.21.15.62:3300/0 conn(0x7f294406f3a0 0x7f294414fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:22.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.482+0000 7f2948999700 1 -- 172.21.15.31:0/1013372843 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4092126486 0 0) 0x7f294406a490 con 0x7f294406e470 2023-10-06T02:29:22.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.482+0000 7f2948999700 1 -- 172.21.15.31:0/1013372843 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2930003570 con 0x7f294406e470 2023-10-06T02:29:22.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.482+0000 7f2948999700 1 -- 172.21.15.31:0/1013372843 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2076766946 0 0) 0x7f2930003570 con 0x7f294406e470 2023-10-06T02:29:22.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.482+0000 7f2948999700 1 -- 172.21.15.31:0/1013372843 >> v2:172.21.15.62:3300/0 conn(0x7f294406f3a0 msgr2=0x7f294414fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:22.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.482+0000 7f2948999700 1 --2- 172.21.15.31:0/1013372843 >> v2:172.21.15.62:3300/0 conn(0x7f294406f3a0 0x7f294414fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:22.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.482+0000 7f2948999700 1 -- 172.21.15.31:0/1013372843 >> v1:172.21.15.62:6789/0 conn(0x7f294406e0a0 legacy=0x7f29440c6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:22.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.482+0000 7f2948999700 1 -- 172.21.15.31:0/1013372843 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29440c7770 con 0x7f294406e470 2023-10-06T02:29:22.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.482+0000 7f2948999700 1 -- 172.21.15.31:0/1013372843 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7f293c0017f0 con 0x7f294406e470 2023-10-06T02:29:22.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.482+0000 7f2948999700 1 -- 172.21.15.31:0/1013372843 >> v1:172.21.15.31:6789/0 conn(0x7f294406e470 legacy=0x7f294406a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:22.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.482+0000 7f2948999700 1 --2- 172.21.15.31:0/1013372843 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2930003a60 0x7f2930003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:22.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.482+0000 7f2948999700 1 --2- 172.21.15.31:0/1013372843 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2930004810 0x7f2930004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:22.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.482+0000 7f2948999700 1 --2- 172.21.15.31:0/1013372843 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f29300051c0 0x7f293000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:22.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.482+0000 7f2948999700 1 -- 172.21.15.31:0/1013372843 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29440c7490 con 0x7f2930003a60 2023-10-06T02:29:22.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.482+0000 7f2948999700 1 -- 172.21.15.31:0/1013372843 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f293000c0d0 con 0x7f2930004810 2023-10-06T02:29:22.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.482+0000 7f2948999700 1 -- 172.21.15.31:0/1013372843 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f293000c290 con 0x7f29300051c0 2023-10-06T02:29:22.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.482+0000 7f294999b700 1 --2- 172.21.15.31:0/1013372843 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2930003a60 0x7f2930003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:22.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.483+0000 7f294919a700 1 --2- 172.21.15.31:0/1013372843 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2930004810 0x7f2930004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:22.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.483+0000 7f294a19c700 1 --2- 172.21.15.31:0/1013372843 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f29300051c0 0x7f293000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:22.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.483+0000 7f294999b700 1 -- 172.21.15.31:0/1013372843 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f29300051c0 msgr2=0x7f293000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:22.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.483+0000 7f294999b700 1 --2- 172.21.15.31:0/1013372843 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f29300051c0 0x7f293000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:22.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.483+0000 7f294999b700 1 -- 172.21.15.31:0/1013372843 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2930004810 msgr2=0x7f2930004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:22.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.483+0000 7f294999b700 1 --2- 172.21.15.31:0/1013372843 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2930004810 0x7f2930004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:22.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.483+0000 7f294999b700 1 -- 172.21.15.31:0/1013372843 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2930003570 con 0x7f2930003a60 2023-10-06T02:29:22.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.483+0000 7f294a19c700 1 -- 172.21.15.31:0/1013372843 reap_dead start 2023-10-06T02:29:22.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.483+0000 7f294999b700 1 --2- 172.21.15.31:0/1013372843 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2930003a60 0x7f2930003e70 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f29400018b0 tx=0x7f2940007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc767d54bc65da24 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:22.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.483+0000 7f2948999700 1 -- 172.21.15.31:0/1013372843 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2940012420 con 0x7f2930003a60 2023-10-06T02:29:22.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.483+0000 7f2948999700 1 -- 172.21.15.31:0/1013372843 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29400125a0 con 0x7f2930003a60 2023-10-06T02:29:22.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.483+0000 7f2948999700 1 -- 172.21.15.31:0/1013372843 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2940012960 con 0x7f2930003a60 2023-10-06T02:29:22.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.484+0000 7f294a99d700 1 -- 172.21.15.31:0/1013372843 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2930003a60 msgr2=0x7f2930003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:22.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.484+0000 7f294a99d700 1 --2- 172.21.15.31:0/1013372843 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2930003a60 0x7f2930003e70 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f29400018b0 tx=0x7f2940007d30 comp rx=0 tx=0).stop 2023-10-06T02:29:22.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.484+0000 7f294a99d700 1 -- 172.21.15.31:0/1013372843 shutdown_connections 2023-10-06T02:29:22.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.484+0000 7f294a99d700 1 --2- 172.21.15.31:0/1013372843 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2930003a60 0x7f2930003e70 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:22.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.484+0000 7f294a99d700 1 -- 172.21.15.31:0/1013372843 >> 172.21.15.31:0/1013372843 conn(0x7f294406d170 msgr2=0x7f29440cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:22.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.484+0000 7f294a99d700 1 -- 172.21.15.31:0/1013372843 shutdown_connections 2023-10-06T02:29:22.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.484+0000 7f294a99d700 1 -- 172.21.15.31:0/1013372843 wait complete. 2023-10-06T02:29:22.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.485+0000 7f294a99d700 1 Processor -- start 2023-10-06T02:29:22.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.486+0000 7f294a99d700 1 -- start start 2023-10-06T02:29:22.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.486+0000 7f294a99d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2930003a60 0x7f29440b1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:22.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.486+0000 7f294a99d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2944165e40 0x7f294414b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:22.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.486+0000 7f294a99d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f29440c2250 0x7f29440c26c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:22.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.487+0000 7f294a99d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f294414b5b0 con 0x7f2944165e40 2023-10-06T02:29:22.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.487+0000 7f294a99d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29440c11b0 con 0x7f2930003a60 2023-10-06T02:29:22.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.487+0000 7f294a99d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29440c1490 con 0x7f29440c2250 2023-10-06T02:29:22.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.487+0000 7f294919a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2944165e40 0x7f294414b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:22.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.487+0000 7f294919a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2944165e40 0x7f294414b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52850/0 (socket says 172.21.15.31:52850) 2023-10-06T02:29:22.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.487+0000 7f294919a700 1 -- 172.21.15.31:0/123944518 learned_addr learned my addr 172.21.15.31:0/123944518 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:22.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.487+0000 7f294999b700 1 --2- 172.21.15.31:0/123944518 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2930003a60 0x7f29440b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:22.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.487+0000 7f294a19c700 1 --2- 172.21.15.31:0/123944518 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f29440c2250 0x7f29440c26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:22.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.487+0000 7f294919a700 1 -- 172.21.15.31:0/123944518 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f29440c2250 msgr2=0x7f29440c26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:22.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.487+0000 7f294919a700 1 --2- 172.21.15.31:0/123944518 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f29440c2250 0x7f29440c26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:22.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.487+0000 7f294919a700 1 -- 172.21.15.31:0/123944518 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2930003a60 msgr2=0x7f29440b1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:22.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.487+0000 7f294919a700 1 --2- 172.21.15.31:0/123944518 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2930003a60 0x7f29440b1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:22.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.487+0000 7f294919a700 1 -- 172.21.15.31:0/123944518 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29440c1770 con 0x7f2944165e40 2023-10-06T02:29:22.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.487+0000 7f294999b700 1 --2- 172.21.15.31:0/123944518 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2930003a60 0x7f29440b1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:29:22.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.487+0000 7f294a19c700 1 --2- 172.21.15.31:0/123944518 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f29440c2250 0x7f29440c26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:29:22.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.487+0000 7f294919a700 1 --2- 172.21.15.31:0/123944518 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2944165e40 0x7f294414b070 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f2934006530 tx=0x7f293400b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:22.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.488+0000 7f2939ffb700 1 -- 172.21.15.31:0/123944518 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2934011c20 con 0x7f2944165e40 2023-10-06T02:29:22.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.488+0000 7f2939ffb700 1 -- 172.21.15.31:0/123944518 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2934011d80 con 0x7f2944165e40 2023-10-06T02:29:22.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.488+0000 7f2939ffb700 1 -- 172.21.15.31:0/123944518 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f293401a690 con 0x7f2944165e40 2023-10-06T02:29:22.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.488+0000 7f294a99d700 1 -- 172.21.15.31:0/123944518 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29440c1a50 con 0x7f2944165e40 2023-10-06T02:29:22.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.488+0000 7f294a99d700 1 -- 172.21.15.31:0/123944518 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29441662b0 con 0x7f2944165e40 2023-10-06T02:29:22.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.489+0000 7f294a99d700 1 -- 172.21.15.31:0/123944518 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29440c1ef0 con 0x7f2944165e40 2023-10-06T02:29:22.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.490+0000 7f2939ffb700 1 -- 172.21.15.31:0/123944518 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f2934023440 con 0x7f2944165e40 2023-10-06T02:29:22.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.490+0000 7f2939ffb700 1 --2- 172.21.15.31:0/123944518 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f2928067320 0x7f29280697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:22.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.490+0000 7f2939ffb700 1 -- 172.21.15.31:0/123944518 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f2934085a80 con 0x7f2944165e40 2023-10-06T02:29:22.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.491+0000 7f294999b700 1 --2- 172.21.15.31:0/123944518 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f2928067320 0x7f29280697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:22.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.491+0000 7f294999b700 1 --2- 172.21.15.31:0/123944518 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f2928067320 0x7f29280697d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2944158ea0 tx=0x7f2940015000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:22.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.494+0000 7f2939ffb700 1 -- 172.21.15.31:0/123944518 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2934052410 con 0x7f2944165e40 2023-10-06T02:29:22.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.741+0000 7f294a99d700 1 -- 172.21.15.31:0/123944518 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f294404c3c0 con 0x7f2944165e40 2023-10-06T02:29:22.743 INFO:tasks.ceph.mon.a.smithi031.stderr:2023-10-06T02:29:22.741+0000 7f6de01f1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T02:29:22.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.746+0000 7f2939ffb700 1 -- 172.21.15.31:0/123944518 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v73) v1 ==== 161+0+0 (secure 0 0 0) 0x7f2934055a30 con 0x7f2944165e40 2023-10-06T02:29:22.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.748+0000 7f294a99d700 1 -- 172.21.15.31:0/123944518 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f2928067320 msgr2=0x7f29280697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:22.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.748+0000 7f294a99d700 1 --2- 172.21.15.31:0/123944518 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f2928067320 0x7f29280697d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2944158ea0 tx=0x7f2940015000 comp rx=0 tx=0).stop 2023-10-06T02:29:22.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.748+0000 7f294a99d700 1 -- 172.21.15.31:0/123944518 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2944165e40 msgr2=0x7f294414b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:22.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.748+0000 7f294a99d700 1 --2- 172.21.15.31:0/123944518 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2944165e40 0x7f294414b070 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f2934006530 tx=0x7f293400b3e0 comp rx=0 tx=0).stop 2023-10-06T02:29:22.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.749+0000 7f294a99d700 1 -- 172.21.15.31:0/123944518 shutdown_connections 2023-10-06T02:29:22.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.749+0000 7f294a99d700 1 --2- 172.21.15.31:0/123944518 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f2928067320 0x7f29280697d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:22.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.749+0000 7f294a99d700 1 --2- 172.21.15.31:0/123944518 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f29440c2250 0x7f29440c26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:22.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.749+0000 7f294a99d700 1 --2- 172.21.15.31:0/123944518 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2930003a60 0x7f29440b1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:22.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.749+0000 7f294a99d700 1 --2- 172.21.15.31:0/123944518 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2944165e40 0x7f294414b070 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:22.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.749+0000 7f294a99d700 1 -- 172.21.15.31:0/123944518 >> 172.21.15.31:0/123944518 conn(0x7f294406d170 msgr2=0x7f294414bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:22.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.749+0000 7f294a99d700 1 -- 172.21.15.31:0/123944518 shutdown_connections 2023-10-06T02:29:22.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.749+0000 7f294a99d700 1 -- 172.21.15.31:0/123944518 wait complete. 2023-10-06T02:29:22.751 INFO:teuthology.orchestra.run.smithi031.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T02:29:22.768 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T02:29:22.811 INFO:tasks.ceph.mds.a.smithi031.stdout:starting mds.a at 2023-10-06T02:29:22.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.958+0000 7fb798b9d700 1 Processor -- start 2023-10-06T02:29:22.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.958+0000 7fb798b9d700 1 -- start start 2023-10-06T02:29:22.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.959+0000 7fb798b9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb7940c7fb0 0x7fb79406abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:22.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.959+0000 7fb798b9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb79406b290 con 0x7fb7940c7be0 2023-10-06T02:29:22.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.959+0000 7fb798b9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb79406b570 con 0x7fb79413f780 2023-10-06T02:29:22.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.960+0000 7fb798b9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb79406bf10 con 0x7fb7940c7fb0 2023-10-06T02:29:22.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.960+0000 7fb7937fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fb7940c7be0 0x7fb79415a990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:33076/0 (socket says 172.21.15.31:33076) 2023-10-06T02:29:22.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.960+0000 7fb7937fe700 1 -- 172.21.15.31:0/3884753739 learned_addr learned my addr 172.21.15.31:0/3884753739 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:29:22.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.960+0000 7fb792ffd700 1 --2- 172.21.15.31:0/3884753739 >> v2:172.21.15.62:3300/0 conn(0x7fb7940c7fb0 0x7fb79406abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:22.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.960+0000 7fb7927fc700 1 -- 172.21.15.31:0/3884753739 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 111332316 0 0) 0x7fb79406b290 con 0x7fb7940c7be0 2023-10-06T02:29:22.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.960+0000 7fb7927fc700 1 -- 172.21.15.31:0/3884753739 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb77c003570 con 0x7fb7940c7be0 2023-10-06T02:29:22.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.960+0000 7fb7927fc700 1 -- 172.21.15.31:0/3884753739 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3503295429 0 0) 0x7fb79406b570 con 0x7fb79413f780 2023-10-06T02:29:22.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.961+0000 7fb7927fc700 1 -- 172.21.15.31:0/3884753739 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb79406b290 con 0x7fb79413f780 2023-10-06T02:29:22.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.961+0000 7fb792ffd700 1 -- 172.21.15.31:0/3884753739 >> v1:172.21.15.62:6789/0 conn(0x7fb79413f780 legacy=0x7fb79413fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:22.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.961+0000 7fb792ffd700 1 -- 172.21.15.31:0/3884753739 >> v1:172.21.15.31:6789/0 conn(0x7fb7940c7be0 legacy=0x7fb79415a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:22.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.961+0000 7fb7927fc700 1 -- 172.21.15.31:0/3884753739 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1941495346 0 0) 0x7fb77c003570 con 0x7fb7940c7be0 2023-10-06T02:29:22.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.961+0000 7fb792ffd700 1 -- 172.21.15.31:0/3884753739 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb79406b850 con 0x7fb7940c7fb0 2023-10-06T02:29:22.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.961+0000 7fb792ffd700 1 --2- 172.21.15.31:0/3884753739 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7940c7fb0 0x7fb79406abd0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fb7880098e0 tx=0x7fb7880091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c9e83b32338b476 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:22.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.961+0000 7fb7927fc700 1 -- 172.21.15.31:0/3884753739 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb784001470 con 0x7fb7940c7fb0 2023-10-06T02:29:22.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.961+0000 7fb7927fc700 1 -- 172.21.15.31:0/3884753739 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb788011420 con 0x7fb7940c7fb0 2023-10-06T02:29:22.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.961+0000 7fb7927fc700 1 -- 172.21.15.31:0/3884753739 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7880117c0 con 0x7fb7940c7fb0 2023-10-06T02:29:22.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.962+0000 7fb798b9d700 1 -- 172.21.15.31:0/3884753739 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7940c7fb0 msgr2=0x7fb79406abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:22.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.962+0000 7fb798b9d700 1 --2- 172.21.15.31:0/3884753739 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7940c7fb0 0x7fb79406abd0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fb7880098e0 tx=0x7fb7880091a0 comp rx=0 tx=0).stop 2023-10-06T02:29:22.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.962+0000 7fb798b9d700 1 -- 172.21.15.31:0/3884753739 shutdown_connections 2023-10-06T02:29:22.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.962+0000 7fb798b9d700 1 --2- 172.21.15.31:0/3884753739 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7940c7fb0 0x7fb79406abd0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:22.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.962+0000 7fb798b9d700 1 -- 172.21.15.31:0/3884753739 >> 172.21.15.31:0/3884753739 conn(0x7fb79406d110 msgr2=0x7fb7940cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:22.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.962+0000 7fb798b9d700 1 -- 172.21.15.31:0/3884753739 shutdown_connections 2023-10-06T02:29:22.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.962+0000 7fb798b9d700 1 -- 172.21.15.31:0/3884753739 wait complete. 2023-10-06T02:29:22.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.963+0000 7fb798b9d700 1 Processor -- start 2023-10-06T02:29:22.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.964+0000 7fb798b9d700 1 -- start start 2023-10-06T02:29:22.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.965+0000 7fb798b9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7940c7be0 0x7fb794155a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:22.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.965+0000 7fb798b9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7940c7fb0 0x7fb7940a98e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:22.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.965+0000 7fb798b9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb79413f780 0x7fb7940a9e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:22.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.965+0000 7fb798b9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb794155f40 con 0x7fb7940c7be0 2023-10-06T02:29:22.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.965+0000 7fb798b9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7940c5ae0 con 0x7fb7940c7fb0 2023-10-06T02:29:22.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.965+0000 7fb798b9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7940abe00 con 0x7fb79413f780 2023-10-06T02:29:22.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.965+0000 7fb7937fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7940c7be0 0x7fb794155a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:22.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.965+0000 7fb7937fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7940c7be0 0x7fb794155a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52880/0 (socket says 172.21.15.31:52880) 2023-10-06T02:29:22.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.965+0000 7fb7937fe700 1 -- 172.21.15.31:0/3255091000 learned_addr learned my addr 172.21.15.31:0/3255091000 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:22.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.965+0000 7fb792ffd700 1 --2- 172.21.15.31:0/3255091000 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7940c7fb0 0x7fb7940a98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:22.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.965+0000 7fb7937fe700 1 -- 172.21.15.31:0/3255091000 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb79413f780 msgr2=0x7fb7940a9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:22.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.965+0000 7fb793fff700 1 --2- 172.21.15.31:0/3255091000 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb79413f780 0x7fb7940a9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:22.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.965+0000 7fb7937fe700 1 --2- 172.21.15.31:0/3255091000 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb79413f780 0x7fb7940a9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:22.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.965+0000 7fb7937fe700 1 -- 172.21.15.31:0/3255091000 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7940c7fb0 msgr2=0x7fb7940a98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:22.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.965+0000 7fb7937fe700 1 --2- 172.21.15.31:0/3255091000 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7940c7fb0 0x7fb7940a98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:22.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.965+0000 7fb7937fe700 1 -- 172.21.15.31:0/3255091000 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb788009420 con 0x7fb7940c7be0 2023-10-06T02:29:22.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.965+0000 7fb793fff700 1 --2- 172.21.15.31:0/3255091000 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb79413f780 0x7fb7940a9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:29:22.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.965+0000 7fb792ffd700 1 --2- 172.21.15.31:0/3255091000 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7940c7fb0 0x7fb7940a98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:29:22.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.966+0000 7fb7937fe700 1 --2- 172.21.15.31:0/3255091000 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7940c7be0 0x7fb794155a00 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fb78400cb90 tx=0x7fb784001ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:22.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.966+0000 7fb77b7fe700 1 -- 172.21.15.31:0/3255091000 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb784001d00 con 0x7fb7940c7be0 2023-10-06T02:29:22.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.966+0000 7fb77b7fe700 1 -- 172.21.15.31:0/3255091000 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb784007360 con 0x7fb7940c7be0 2023-10-06T02:29:22.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.966+0000 7fb77b7fe700 1 -- 172.21.15.31:0/3255091000 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb784004830 con 0x7fb7940c7be0 2023-10-06T02:29:22.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.966+0000 7fb798b9d700 1 -- 172.21.15.31:0/3255091000 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7940ac0e0 con 0x7fb7940c7be0 2023-10-06T02:29:22.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.966+0000 7fb798b9d700 1 -- 172.21.15.31:0/3255091000 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7940abad0 con 0x7fb7940c7be0 2023-10-06T02:29:22.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.967+0000 7fb798b9d700 1 -- 172.21.15.31:0/3255091000 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb79404c3c0 con 0x7fb7940c7be0 2023-10-06T02:29:22.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.968+0000 7fb77b7fe700 1 -- 172.21.15.31:0/3255091000 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fb784004be0 con 0x7fb7940c7be0 2023-10-06T02:29:22.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.969+0000 7fb77b7fe700 1 --2- 172.21.15.31:0/3255091000 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb780067320 0x7fb7800697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:22.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.969+0000 7fb77b7fe700 1 -- 172.21.15.31:0/3255091000 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6172+0+0 (secure 0 0 0) 0x7fb784084720 con 0x7fb7940c7be0 2023-10-06T02:29:22.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.969+0000 7fb792ffd700 1 --2- 172.21.15.31:0/3255091000 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb780067320 0x7fb7800697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:22.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.970+0000 7fb792ffd700 1 --2- 172.21.15.31:0/3255091000 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb780067320 0x7fb7800697d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb7880093f0 tx=0x7fb788016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:22.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:22.973+0000 7fb77b7fe700 1 -- 172.21.15.31:0/3255091000 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb784051220 con 0x7fb7940c7be0 2023-10-06T02:29:23.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:23.220+0000 7fb798b9d700 1 -- 172.21.15.31:0/3255091000 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fb79406bb30 con 0x7fb7940c7be0 2023-10-06T02:29:23.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:23.748+0000 7fb77b7fe700 1 -- 172.21.15.31:0/3255091000 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v74) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb784054840 con 0x7fb7940c7be0 2023-10-06T02:29:23.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:23.751+0000 7fb798b9d700 1 -- 172.21.15.31:0/3255091000 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb780067320 msgr2=0x7fb7800697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:23.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:23.751+0000 7fb798b9d700 1 --2- 172.21.15.31:0/3255091000 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb780067320 0x7fb7800697d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb7880093f0 tx=0x7fb788016000 comp rx=0 tx=0).stop 2023-10-06T02:29:23.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:23.751+0000 7fb798b9d700 1 -- 172.21.15.31:0/3255091000 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7940c7be0 msgr2=0x7fb794155a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:23.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:23.751+0000 7fb798b9d700 1 --2- 172.21.15.31:0/3255091000 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7940c7be0 0x7fb794155a00 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fb78400cb90 tx=0x7fb784001ab0 comp rx=0 tx=0).stop 2023-10-06T02:29:23.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:23.751+0000 7fb798b9d700 1 -- 172.21.15.31:0/3255091000 shutdown_connections 2023-10-06T02:29:23.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:23.751+0000 7fb798b9d700 1 --2- 172.21.15.31:0/3255091000 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb780067320 0x7fb7800697d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:23.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:23.751+0000 7fb798b9d700 1 --2- 172.21.15.31:0/3255091000 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb79413f780 0x7fb7940a9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:23.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:23.751+0000 7fb798b9d700 1 --2- 172.21.15.31:0/3255091000 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7940c7be0 0x7fb794155a00 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:23.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:23.751+0000 7fb798b9d700 1 --2- 172.21.15.31:0/3255091000 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7940c7fb0 0x7fb7940a98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:23.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:23.751+0000 7fb798b9d700 1 -- 172.21.15.31:0/3255091000 >> 172.21.15.31:0/3255091000 conn(0x7fb79406d110 msgr2=0x7fb7940caaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:23.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:23.751+0000 7fb798b9d700 1 -- 172.21.15.31:0/3255091000 shutdown_connections 2023-10-06T02:29:23.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:23.751+0000 7fb798b9d700 1 -- 172.21.15.31:0/3255091000 wait complete. 2023-10-06T02:29:23.888 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T02:29:24.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:24.128+0000 7f0d1979d700 1 Processor -- start 2023-10-06T02:29:25.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:24.129+0000 7f0d1979d700 1 -- start start 2023-10-06T02:29:25.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:24.130+0000 7f0d1979d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0d141539d0 0x7f0d14153da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:25.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:24.130+0000 7f0d1979d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0d1415a740 0x7f0d140ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:25.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:24.130+0000 7f0d1979d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d140cd180 con 0x7f0d140cb470 2023-10-06T02:29:25.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:24.130+0000 7f0d1979d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d140ce930 con 0x7f0d141539d0 2023-10-06T02:29:25.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:24.130+0000 7f0d1979d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d1415abe0 con 0x7f0d1415a740 2023-10-06T02:29:25.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:24.130+0000 7f0d13fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0d141539d0 0x7f0d14153da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:25.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:24.130+0000 7f0d13fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0d141539d0 0x7f0d14153da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52896/0 (socket says 172.21.15.31:52896) 2023-10-06T02:29:25.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:24.130+0000 7f0d13fff700 1 -- 172.21.15.31:0/2043236583 learned_addr learned my addr 172.21.15.31:0/2043236583 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:25.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:24.131+0000 7f0d137fe700 1 --2- 172.21.15.31:0/2043236583 >> v2:172.21.15.62:3300/0 conn(0x7f0d1415a740 0x7f0d140ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:25.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:24.131+0000 7f0d13fff700 1 -- 172.21.15.31:0/2043236583 >> v2:172.21.15.62:3300/0 conn(0x7f0d1415a740 msgr2=0x7f0d140ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:25.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:24.131+0000 7f0d13fff700 1 --2- 172.21.15.31:0/2043236583 >> v2:172.21.15.62:3300/0 conn(0x7f0d1415a740 0x7f0d140ccc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:25.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:24.131+0000 7f0d13fff700 1 -- 172.21.15.31:0/2043236583 >> v1:172.21.15.62:6789/0 conn(0x7f0d140cb470 legacy=0x7f0d1415d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:25.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:24.131+0000 7f0d13fff700 1 -- 172.21.15.31:0/2043236583 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d140cd3e0 con 0x7f0d141539d0 2023-10-06T02:29:25.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:24.131+0000 7f0d13fff700 1 --2- 172.21.15.31:0/2043236583 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d141539d0 0x7f0d14153da0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f0d140ce4b0 tx=0x7f0d04009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f3ebe79e39d4808 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:25.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:24.131+0000 7f0d12ffd700 1 -- 172.21.15.31:0/2043236583 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d04009e70 con 0x7f0d141539d0 2023-10-06T02:29:25.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:24.132+0000 7f0d12ffd700 1 -- 172.21.15.31:0/2043236583 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d0400b860 con 0x7f0d141539d0 2023-10-06T02:29:25.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:24.132+0000 7f0d12ffd700 1 -- 172.21.15.31:0/2043236583 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0400bc00 con 0x7f0d141539d0 2023-10-06T02:29:25.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:24.132+0000 7f0d1979d700 1 -- 172.21.15.31:0/2043236583 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d141539d0 msgr2=0x7f0d14153da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:25.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:24.132+0000 7f0d1979d700 1 --2- 172.21.15.31:0/2043236583 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d141539d0 0x7f0d14153da0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f0d140ce4b0 tx=0x7f0d04009250 comp rx=0 tx=0).stop 2023-10-06T02:29:25.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:24.132+0000 7f0d1979d700 1 -- 172.21.15.31:0/2043236583 shutdown_connections 2023-10-06T02:29:25.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:24.132+0000 7f0d1979d700 1 --2- 172.21.15.31:0/2043236583 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d141539d0 0x7f0d14153da0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:25.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:24.132+0000 7f0d1979d700 1 --2- 172.21.15.31:0/2043236583 >> v2:172.21.15.62:3300/0 conn(0x7f0d1415a740 0x7f0d140ccc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:25.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:24.132+0000 7f0d1979d700 1 -- 172.21.15.31:0/2043236583 >> 172.21.15.31:0/2043236583 conn(0x7f0d140703a0 msgr2=0x7f0d140707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:25.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:24.132+0000 7f0d1979d700 1 -- 172.21.15.31:0/2043236583 shutdown_connections 2023-10-06T02:29:25.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:24.132+0000 7f0d1979d700 1 -- 172.21.15.31:0/2043236583 wait complete. 2023-10-06T02:29:25.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:24.134+0000 7f0d1979d700 1 Processor -- start 2023-10-06T02:29:25.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:24.134+0000 7f0d1979d700 1 -- start start 2023-10-06T02:29:25.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:24.135+0000 7f0d1979d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0d140cb470 0x7f0d14152fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:25.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:24.135+0000 7f0d1979d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d141539d0 0x7f0d140b41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:25.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:24.135+0000 7f0d1979d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d1415a740 0x7f0d1415d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:25.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:24.135+0000 7f0d1979d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1415d5e0 con 0x7f0d1415a740 2023-10-06T02:29:25.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:24.135+0000 7f0d1979d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d140b4ee0 con 0x7f0d141539d0 2023-10-06T02:29:25.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:24.135+0000 7f0d1979d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1415d7d0 con 0x7f0d140cb470 2023-10-06T02:29:25.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:24.135+0000 7f0d18f9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d1415a740 0x7f0d1415d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:25.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:24.135+0000 7f0d13fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0d140cb470 0x7f0d14152fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:25.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:24.135+0000 7f0d18f9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d1415a740 0x7f0d1415d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52908/0 (socket says 172.21.15.31:52908) 2023-10-06T02:29:25.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:24.135+0000 7f0d18f9c700 1 -- 172.21.15.31:0/1530902100 learned_addr learned my addr 172.21.15.31:0/1530902100 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:25.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:24.135+0000 7f0d137fe700 1 --2- 172.21.15.31:0/1530902100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d141539d0 0x7f0d140b41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:25.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:24.136+0000 7f0d18f9c700 1 -- 172.21.15.31:0/1530902100 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0d140cb470 msgr2=0x7f0d14152fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:25.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:24.136+0000 7f0d18f9c700 1 --2- 172.21.15.31:0/1530902100 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0d140cb470 0x7f0d14152fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:25.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:24.136+0000 7f0d18f9c700 1 -- 172.21.15.31:0/1530902100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d141539d0 msgr2=0x7f0d140b41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:25.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:24.136+0000 7f0d18f9c700 1 --2- 172.21.15.31:0/1530902100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d141539d0 0x7f0d140b41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:25.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:24.136+0000 7f0d18f9c700 1 -- 172.21.15.31:0/1530902100 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d040094f0 con 0x7f0d1415a740 2023-10-06T02:29:25.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:24.136+0000 7f0d18f9c700 1 --2- 172.21.15.31:0/1530902100 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d1415a740 0x7f0d1415d0a0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c00c910 tx=0x7f0d0c001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:25.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:24.136+0000 7f0cfbfff700 1 -- 172.21.15.31:0/1530902100 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c012630 con 0x7f0d1415a740 2023-10-06T02:29:25.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:24.136+0000 7f0cfbfff700 1 -- 172.21.15.31:0/1530902100 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d0c012790 con 0x7f0d1415a740 2023-10-06T02:29:25.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:24.136+0000 7f0d1979d700 1 -- 172.21.15.31:0/1530902100 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d1415dab0 con 0x7f0d1415a740 2023-10-06T02:29:25.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:24.136+0000 7f0cfbfff700 1 -- 172.21.15.31:0/1530902100 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c012b50 con 0x7f0d1415a740 2023-10-06T02:29:25.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:24.137+0000 7f0d1979d700 1 -- 172.21.15.31:0/1530902100 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d1415e070 con 0x7f0d1415a740 2023-10-06T02:29:25.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:24.138+0000 7f0d1979d700 1 -- 172.21.15.31:0/1530902100 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d1404c3c0 con 0x7f0d1415a740 2023-10-06T02:29:25.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:24.139+0000 7f0cfbfff700 1 -- 172.21.15.31:0/1530902100 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f0d0c01a3a0 con 0x7f0d1415a740 2023-10-06T02:29:25.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:24.140+0000 7f0cfbfff700 1 --2- 172.21.15.31:0/1530902100 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0d00067320 0x7f0d000697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:25.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:24.140+0000 7f0cfbfff700 1 -- 172.21.15.31:0/1530902100 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6172+0+0 (secure 0 0 0) 0x7f0d0c085080 con 0x7f0d1415a740 2023-10-06T02:29:25.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:24.141+0000 7f0d13fff700 1 --2- 172.21.15.31:0/1530902100 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0d00067320 0x7f0d000697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:25.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:24.141+0000 7f0d13fff700 1 --2- 172.21.15.31:0/1530902100 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0d00067320 0x7f0d000697d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0d140ce4b0 tx=0x7f0d04017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:25.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:24.144+0000 7f0cfbfff700 1 -- 172.21.15.31:0/1530902100 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0d0c0b65a0 con 0x7f0d1415a740 2023-10-06T02:29:25.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:24.350+0000 7f0d1979d700 1 -- 172.21.15.31:0/1530902100 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0d140b2bf0 con 0x7f0d1415a740 2023-10-06T02:29:25.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:25.064+0000 7f0cfbfff700 1 -- 172.21.15.31:0/1530902100 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v110) v1 ==== 181+0+0 (secure 0 0 0) 0x7f0d0c0555d0 con 0x7f0d1415a740 2023-10-06T02:29:25.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:25.069+0000 7f0d1979d700 1 -- 172.21.15.31:0/1530902100 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0d00067320 msgr2=0x7f0d000697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:25.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:25.069+0000 7f0d1979d700 1 --2- 172.21.15.31:0/1530902100 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0d00067320 0x7f0d000697d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0d140ce4b0 tx=0x7f0d04017000 comp rx=0 tx=0).stop 2023-10-06T02:29:25.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:25.069+0000 7f0d1979d700 1 -- 172.21.15.31:0/1530902100 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d1415a740 msgr2=0x7f0d1415d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:25.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:25.069+0000 7f0d1979d700 1 --2- 172.21.15.31:0/1530902100 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d1415a740 0x7f0d1415d0a0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c00c910 tx=0x7f0d0c001710 comp rx=0 tx=0).stop 2023-10-06T02:29:25.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:25.069+0000 7f0d1979d700 1 -- 172.21.15.31:0/1530902100 shutdown_connections 2023-10-06T02:29:25.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:25.069+0000 7f0d1979d700 1 --2- 172.21.15.31:0/1530902100 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d1415a740 0x7f0d1415d0a0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:25.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:25.069+0000 7f0d1979d700 1 --2- 172.21.15.31:0/1530902100 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0d00067320 0x7f0d000697d0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:25.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:25.069+0000 7f0d1979d700 1 --2- 172.21.15.31:0/1530902100 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0d140cb470 0x7f0d14152fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:25.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:25.069+0000 7f0d1979d700 1 --2- 172.21.15.31:0/1530902100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d141539d0 0x7f0d140b41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:25.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:25.069+0000 7f0d1979d700 1 -- 172.21.15.31:0/1530902100 >> 172.21.15.31:0/1530902100 conn(0x7f0d140703a0 msgr2=0x7f0d1415fab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:25.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:25.069+0000 7f0d1979d700 1 -- 172.21.15.31:0/1530902100 shutdown_connections 2023-10-06T02:29:25.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:25.069+0000 7f0d1979d700 1 -- 172.21.15.31:0/1530902100 wait complete. 2023-10-06T02:29:25.081 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' removed 2023-10-06T02:29:25.091 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T02:29:25.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:25.289+0000 7f39d759d700 1 Processor -- start 2023-10-06T02:29:25.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:25.289+0000 7f39d759d700 1 -- start start 2023-10-06T02:29:25.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:25.290+0000 7f39d759d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f39d015cb90 0x7f39d00c7870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:25.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:25.290+0000 7f39d759d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39d015dc10 0x7f39d006c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:25.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:25.290+0000 7f39d759d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39d006d040 con 0x7f39d006cbe0 2023-10-06T02:29:25.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:25.290+0000 7f39d759d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39d00ccdf0 con 0x7f39d015cb90 2023-10-06T02:29:25.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:25.290+0000 7f39d759d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39d00c9b20 con 0x7f39d015dc10 2023-10-06T02:29:25.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:25.290+0000 7f39d6d9c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f39d006cbe0 0x7f39d00c3430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:33078/0 (socket says 172.21.15.31:33078) 2023-10-06T02:29:25.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:25.290+0000 7f39d6d9c700 1 -- 172.21.15.31:0/3896103597 learned_addr learned my addr 172.21.15.31:0/3896103597 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:29:25.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:25.291+0000 7f39d659b700 1 --2- 172.21.15.31:0/3896103597 >> v2:172.21.15.62:3300/0 conn(0x7f39d015cb90 0x7f39d00c7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:25.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:25.291+0000 7f39d5d9a700 1 --2- 172.21.15.31:0/3896103597 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39d015dc10 0x7f39d006c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:25.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:25.291+0000 7f39d5599700 1 -- 172.21.15.31:0/3896103597 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 979277334 0 0) 0x7f39c8001490 con 0x7f39d006cbe0 2023-10-06T02:29:25.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:25.291+0000 7f39d5599700 1 -- 172.21.15.31:0/3896103597 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39bc003570 con 0x7f39d006cbe0 2023-10-06T02:29:25.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:25.291+0000 7f39d659b700 1 -- 172.21.15.31:0/3896103597 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39d015dc10 msgr2=0x7f39d006c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:25.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:25.291+0000 7f39d659b700 1 --2- 172.21.15.31:0/3896103597 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39d015dc10 0x7f39d006c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:25.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:25.291+0000 7f39d659b700 1 -- 172.21.15.31:0/3896103597 >> v1:172.21.15.31:6789/0 conn(0x7f39d006cbe0 legacy=0x7f39d00c3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:25.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:25.291+0000 7f39d659b700 1 -- 172.21.15.31:0/3896103597 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39d00c2fa0 con 0x7f39d015cb90 2023-10-06T02:29:25.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:25.291+0000 7f39d659b700 1 --2- 172.21.15.31:0/3896103597 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39d015cb90 0x7f39d00c7870 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f39cc0098e0 tx=0x7f39cc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e70001e9c725406 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:25.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:25.292+0000 7f39d5599700 1 -- 172.21.15.31:0/3896103597 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39cc011630 con 0x7f39d015cb90 2023-10-06T02:29:25.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:25.292+0000 7f39d5599700 1 -- 172.21.15.31:0/3896103597 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39cc011790 con 0x7f39d015cb90 2023-10-06T02:29:25.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:25.292+0000 7f39d5599700 1 -- 172.21.15.31:0/3896103597 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39cc011b30 con 0x7f39d015cb90 2023-10-06T02:29:25.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:25.292+0000 7f39d759d700 1 -- 172.21.15.31:0/3896103597 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39d015cb90 msgr2=0x7f39d00c7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:25.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:25.292+0000 7f39d759d700 1 --2- 172.21.15.31:0/3896103597 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39d015cb90 0x7f39d00c7870 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f39cc0098e0 tx=0x7f39cc0091a0 comp rx=0 tx=0).stop 2023-10-06T02:29:25.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:25.292+0000 7f39d759d700 1 -- 172.21.15.31:0/3896103597 shutdown_connections 2023-10-06T02:29:25.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:25.292+0000 7f39d759d700 1 --2- 172.21.15.31:0/3896103597 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39d015cb90 0x7f39d00c7870 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:25.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:25.292+0000 7f39d759d700 1 --2- 172.21.15.31:0/3896103597 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39d015dc10 0x7f39d006c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:25.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:25.292+0000 7f39d759d700 1 -- 172.21.15.31:0/3896103597 >> 172.21.15.31:0/3896103597 conn(0x7f39d00cd230 msgr2=0x7f39d00cd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:25.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:25.292+0000 7f39d759d700 1 -- 172.21.15.31:0/3896103597 shutdown_connections 2023-10-06T02:29:25.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:25.292+0000 7f39d759d700 1 -- 172.21.15.31:0/3896103597 wait complete. 2023-10-06T02:29:25.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:25.294+0000 7f39d759d700 1 Processor -- start 2023-10-06T02:29:25.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:25.294+0000 7f39d759d700 1 -- start start 2023-10-06T02:29:25.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:25.295+0000 7f39d759d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39d006cbe0 0x7f39d015cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:25.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:25.295+0000 7f39d759d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39d015cb90 0x7f39d015d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:25.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:25.295+0000 7f39d759d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39d015dc10 0x7f39d016a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:25.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:25.295+0000 7f39d759d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d016afc0 con 0x7f39d015dc10 2023-10-06T02:29:25.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:25.295+0000 7f39d759d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d00692a0 con 0x7f39d015cb90 2023-10-06T02:29:25.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:25.295+0000 7f39d759d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39d013fae0 con 0x7f39d006cbe0 2023-10-06T02:29:25.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:25.295+0000 7f39d6d9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39d015dc10 0x7f39d016a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:25.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:25.295+0000 7f39d6d9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39d015dc10 0x7f39d016a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52912/0 (socket says 172.21.15.31:52912) 2023-10-06T02:29:25.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:25.295+0000 7f39d659b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39d006cbe0 0x7f39d015cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:25.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:25.295+0000 7f39d6d9c700 1 -- 172.21.15.31:0/821054002 learned_addr learned my addr 172.21.15.31:0/821054002 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:25.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:25.295+0000 7f39d5d9a700 1 --2- 172.21.15.31:0/821054002 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39d015cb90 0x7f39d015d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:25.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:25.295+0000 7f39d6d9c700 1 -- 172.21.15.31:0/821054002 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39d006cbe0 msgr2=0x7f39d015cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:25.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:25.295+0000 7f39d6d9c700 1 --2- 172.21.15.31:0/821054002 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39d006cbe0 0x7f39d015cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:25.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:25.295+0000 7f39d6d9c700 1 -- 172.21.15.31:0/821054002 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39d015cb90 msgr2=0x7f39d015d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:25.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:25.295+0000 7f39d6d9c700 1 --2- 172.21.15.31:0/821054002 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39d015cb90 0x7f39d015d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:25.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:25.295+0000 7f39d6d9c700 1 -- 172.21.15.31:0/821054002 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39cc009420 con 0x7f39d015dc10 2023-10-06T02:29:25.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:25.296+0000 7f39d6d9c700 1 --2- 172.21.15.31:0/821054002 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39d015dc10 0x7f39d016a000 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f39c800cd00 tx=0x7f39c8001ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:25.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:25.296+0000 7f39c67fc700 1 -- 172.21.15.31:0/821054002 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c800ce00 con 0x7f39d015dc10 2023-10-06T02:29:25.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:25.296+0000 7f39c67fc700 1 -- 172.21.15.31:0/821054002 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39c800aac0 con 0x7f39d015dc10 2023-10-06T02:29:25.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:25.296+0000 7f39c67fc700 1 -- 172.21.15.31:0/821054002 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c800ae40 con 0x7f39d015dc10 2023-10-06T02:29:25.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:25.296+0000 7f39d759d700 1 -- 172.21.15.31:0/821054002 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39d0068750 con 0x7f39d015dc10 2023-10-06T02:29:25.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:25.296+0000 7f39d759d700 1 -- 172.21.15.31:0/821054002 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39d0068d10 con 0x7f39d015dc10 2023-10-06T02:29:25.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:25.297+0000 7f39d759d700 1 -- 172.21.15.31:0/821054002 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39d006d570 con 0x7f39d015dc10 2023-10-06T02:29:25.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:25.298+0000 7f39c67fc700 1 -- 172.21.15.31:0/821054002 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f39c8007300 con 0x7f39d015dc10 2023-10-06T02:29:25.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:25.299+0000 7f39c67fc700 1 --2- 172.21.15.31:0/821054002 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f39b40673f0 0x7f39b40698a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:25.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:25.299+0000 7f39c67fc700 1 -- 172.21.15.31:0/821054002 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5734+0+0 (secure 0 0 0) 0x7f39c8084e90 con 0x7f39d015dc10 2023-10-06T02:29:25.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:25.299+0000 7f39d659b700 1 --2- 172.21.15.31:0/821054002 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f39b40673f0 0x7f39b40698a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:25.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:25.300+0000 7f39d659b700 1 --2- 172.21.15.31:0/821054002 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f39b40673f0 0x7f39b40698a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f39cc0091e0 tx=0x7f39cc017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:25.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:25.303+0000 7f39c67fc700 1 -- 172.21.15.31:0/821054002 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f39c8051b30 con 0x7f39d015dc10 2023-10-06T02:29:25.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:25.503+0000 7f39d759d700 1 -- 172.21.15.31:0/821054002 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f39d006d040 con 0x7f39d015dc10 2023-10-06T02:29:26.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.111+0000 7f39c67fc700 1 -- 172.21.15.31:0/821054002 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v111) v1 ==== 169+0+0 (secure 0 0 0) 0x7f39d006d040 con 0x7f39d015dc10 2023-10-06T02:29:26.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.116+0000 7f39d759d700 1 -- 172.21.15.31:0/821054002 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f39b40673f0 msgr2=0x7f39b40698a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:26.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.116+0000 7f39d759d700 1 --2- 172.21.15.31:0/821054002 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f39b40673f0 0x7f39b40698a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f39cc0091e0 tx=0x7f39cc017000 comp rx=0 tx=0).stop 2023-10-06T02:29:26.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.116+0000 7f39d759d700 1 -- 172.21.15.31:0/821054002 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39d015dc10 msgr2=0x7f39d016a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:26.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.116+0000 7f39d759d700 1 --2- 172.21.15.31:0/821054002 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39d015dc10 0x7f39d016a000 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f39c800cd00 tx=0x7f39c8001ad0 comp rx=0 tx=0).stop 2023-10-06T02:29:26.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.116+0000 7f39d759d700 1 -- 172.21.15.31:0/821054002 shutdown_connections 2023-10-06T02:29:26.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.116+0000 7f39d759d700 1 --2- 172.21.15.31:0/821054002 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39d015dc10 0x7f39d016a000 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:26.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.116+0000 7f39d759d700 1 --2- 172.21.15.31:0/821054002 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f39b40673f0 0x7f39b40698a0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:26.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.116+0000 7f39d759d700 1 --2- 172.21.15.31:0/821054002 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39d006cbe0 0x7f39d015cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:26.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.116+0000 7f39d759d700 1 --2- 172.21.15.31:0/821054002 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39d015cb90 0x7f39d015d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:26.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.116+0000 7f39d759d700 1 -- 172.21.15.31:0/821054002 >> 172.21.15.31:0/821054002 conn(0x7f39d00cd230 msgr2=0x7f39d00b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:26.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.116+0000 7f39d759d700 1 -- 172.21.15.31:0/821054002 shutdown_connections 2023-10-06T02:29:26.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.116+0000 7f39d759d700 1 -- 172.21.15.31:0/821054002 wait complete. 2023-10-06T02:29:26.119 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' removed 2023-10-06T02:29:26.150 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T02:29:26.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.340+0000 7f9bbdf9d700 1 Processor -- start 2023-10-06T02:29:26.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.340+0000 7f9bbdf9d700 1 -- start start 2023-10-06T02:29:26.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.341+0000 7f9bbdf9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9bb80c7be0 0x7f9bb806abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:26.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.341+0000 7f9bbdf9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bb806b290 con 0x7f9bb813f780 2023-10-06T02:29:26.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.341+0000 7f9bbdf9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bb806b570 con 0x7f9bb80c7fb0 2023-10-06T02:29:26.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.341+0000 7f9bbdf9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb806bf10 con 0x7f9bb80c7be0 2023-10-06T02:29:26.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.342+0000 7f9bbd79c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f9bb813f780 0x7f9bb80c9eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:33092/0 (socket says 172.21.15.31:33092) 2023-10-06T02:29:26.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.342+0000 7f9bbd79c700 1 -- 172.21.15.31:0/3285656469 learned_addr learned my addr 172.21.15.31:0/3285656469 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:29:26.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.342+0000 7f9bbcf9b700 1 --2- 172.21.15.31:0/3285656469 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9bb80c7be0 0x7f9bb806abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:26.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.342+0000 7f9bb77fe700 1 -- 172.21.15.31:0/3285656469 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3042298247 0 0) 0x7f9bb806b290 con 0x7f9bb813f780 2023-10-06T02:29:26.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.342+0000 7f9bb77fe700 1 -- 172.21.15.31:0/3285656469 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ba8003570 con 0x7f9bb813f780 2023-10-06T02:29:26.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.342+0000 7f9bb77fe700 1 -- 172.21.15.31:0/3285656469 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 156172337 0 0) 0x7f9bb806b570 con 0x7f9bb80c7fb0 2023-10-06T02:29:26.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.342+0000 7f9bb77fe700 1 -- 172.21.15.31:0/3285656469 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9bb806b290 con 0x7f9bb80c7fb0 2023-10-06T02:29:26.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.343+0000 7f9bb77fe700 1 -- 172.21.15.31:0/3285656469 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3954329146 0 0) 0x7f9ba8003570 con 0x7f9bb813f780 2023-10-06T02:29:26.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.343+0000 7f9bb77fe700 1 -- 172.21.15.31:0/3285656469 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9bb80c7be0 msgr2=0x7f9bb806abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:26.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.343+0000 7f9bb77fe700 1 --2- 172.21.15.31:0/3285656469 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9bb80c7be0 0x7f9bb806abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:26.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.343+0000 7f9bb77fe700 1 -- 172.21.15.31:0/3285656469 >> v1:172.21.15.62:6789/0 conn(0x7f9bb80c7fb0 legacy=0x7f9bb815a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:26.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.343+0000 7f9bb77fe700 1 -- 172.21.15.31:0/3285656469 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bb806b850 con 0x7f9bb813f780 2023-10-06T02:29:26.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.343+0000 7f9bb77fe700 1 -- 172.21.15.31:0/3285656469 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7f9bb00017f0 con 0x7f9bb813f780 2023-10-06T02:29:26.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.343+0000 7f9bbcf9b700 1 --2- 172.21.15.31:0/3285656469 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9bb80c7be0 0x7f9bb806abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:29:26.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.343+0000 7f9bb77fe700 1 -- 172.21.15.31:0/3285656469 >> v1:172.21.15.31:6789/0 conn(0x7f9bb813f780 legacy=0x7f9bb80c9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:26.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.343+0000 7f9bb77fe700 1 --2- 172.21.15.31:0/3285656469 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ba8003c20 0x7f9ba8004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:26.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.343+0000 7f9bb77fe700 1 --2- 172.21.15.31:0/3285656469 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ba8004810 0x7f9ba8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:26.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.343+0000 7f9bb77fe700 1 --2- 172.21.15.31:0/3285656469 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9bb80c7be0 0x7f9ba80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:26.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.343+0000 7f9bb77fe700 1 -- 172.21.15.31:0/3285656469 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb806bf10 con 0x7f9ba8004810 2023-10-06T02:29:26.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.343+0000 7f9bb77fe700 1 -- 172.21.15.31:0/3285656469 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba800bb10 con 0x7f9ba8003c20 2023-10-06T02:29:26.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.343+0000 7f9bb77fe700 1 -- 172.21.15.31:0/3285656469 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba800bcd0 con 0x7f9bb80c7be0 2023-10-06T02:29:26.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.343+0000 7f9bb7fff700 1 --2- 172.21.15.31:0/3285656469 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ba8004810 0x7f9ba8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:26.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.343+0000 7f9bbd79c700 1 --2- 172.21.15.31:0/3285656469 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9bb80c7be0 0x7f9ba80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:26.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.343+0000 7f9bbcf9b700 1 --2- 172.21.15.31:0/3285656469 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ba8003c20 0x7f9ba8004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:26.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.343+0000 7f9bb7fff700 1 -- 172.21.15.31:0/3285656469 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9bb80c7be0 msgr2=0x7f9ba80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:26.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.343+0000 7f9bb7fff700 1 --2- 172.21.15.31:0/3285656469 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9bb80c7be0 0x7f9ba80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:26.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.343+0000 7f9bb7fff700 1 -- 172.21.15.31:0/3285656469 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ba8003c20 msgr2=0x7f9ba8004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:26.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.343+0000 7f9bb7fff700 1 --2- 172.21.15.31:0/3285656469 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ba8003c20 0x7f9ba8004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:26.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.343+0000 7f9bb7fff700 1 -- 172.21.15.31:0/3285656469 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bac009420 con 0x7f9ba8004810 2023-10-06T02:29:26.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.344+0000 7f9bb7fff700 1 --2- 172.21.15.31:0/3285656469 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ba8004810 0x7f9ba8004c80 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f9ba40019e0 tx=0x7f9ba4007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9fc48567bf6db1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:26.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.344+0000 7f9bb77fe700 1 -- 172.21.15.31:0/3285656469 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ba400c5d0 con 0x7f9ba8004810 2023-10-06T02:29:26.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.344+0000 7f9bb77fe700 1 -- 172.21.15.31:0/3285656469 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ba400c730 con 0x7f9ba8004810 2023-10-06T02:29:26.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.344+0000 7f9bb77fe700 1 -- 172.21.15.31:0/3285656469 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ba400cad0 con 0x7f9ba8004810 2023-10-06T02:29:26.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.344+0000 7f9bbdf9d700 1 -- 172.21.15.31:0/3285656469 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ba8004810 msgr2=0x7f9ba8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:26.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.344+0000 7f9bbdf9d700 1 --2- 172.21.15.31:0/3285656469 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ba8004810 0x7f9ba8004c80 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f9ba40019e0 tx=0x7f9ba4007d30 comp rx=0 tx=0).stop 2023-10-06T02:29:26.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.344+0000 7f9bbd79c700 1 -- 172.21.15.31:0/3285656469 reap_dead start 2023-10-06T02:29:26.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.345+0000 7f9bbdf9d700 1 -- 172.21.15.31:0/3285656469 shutdown_connections 2023-10-06T02:29:26.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.345+0000 7f9bbdf9d700 1 -- 172.21.15.31:0/3285656469 >> 172.21.15.31:0/3285656469 conn(0x7f9bb806d110 msgr2=0x7f9bb80cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:26.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.345+0000 7f9bbdf9d700 1 -- 172.21.15.31:0/3285656469 shutdown_connections 2023-10-06T02:29:26.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.345+0000 7f9bbdf9d700 1 -- 172.21.15.31:0/3285656469 wait complete. 2023-10-06T02:29:26.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.346+0000 7f9bbdf9d700 1 Processor -- start 2023-10-06T02:29:26.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.346+0000 7f9bbdf9d700 1 -- start start 2023-10-06T02:29:26.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.347+0000 7f9bbdf9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ba8004810 0x7f9bb8155790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:26.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.347+0000 7f9bbdf9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9bb80b0b70 0x7f9bb81337f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:26.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.347+0000 7f9bbdf9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9bb80c5530 0x7f9bb80c59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:26.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.347+0000 7f9bbdf9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb80b1690 con 0x7f9bb80b0b70 2023-10-06T02:29:26.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.347+0000 7f9bbdf9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb80c50a0 con 0x7f9ba8004810 2023-10-06T02:29:26.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.347+0000 7f9bbdf9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb8155cd0 con 0x7f9bb80c5530 2023-10-06T02:29:26.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.347+0000 7f9bb7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9bb80b0b70 0x7f9bb81337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:26.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.348+0000 7f9bb7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9bb80b0b70 0x7f9bb81337f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52924/0 (socket says 172.21.15.31:52924) 2023-10-06T02:29:26.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.348+0000 7f9bb7fff700 1 -- 172.21.15.31:0/1222523063 learned_addr learned my addr 172.21.15.31:0/1222523063 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:26.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.348+0000 7f9bbcf9b700 1 --2- 172.21.15.31:0/1222523063 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ba8004810 0x7f9bb8155790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:26.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.348+0000 7f9bbd79c700 1 --2- 172.21.15.31:0/1222523063 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9bb80c5530 0x7f9bb80c59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:26.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.348+0000 7f9bb7fff700 1 -- 172.21.15.31:0/1222523063 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9bb80c5530 msgr2=0x7f9bb80c59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:26.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.348+0000 7f9bb7fff700 1 --2- 172.21.15.31:0/1222523063 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9bb80c5530 0x7f9bb80c59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:26.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.348+0000 7f9bb7fff700 1 -- 172.21.15.31:0/1222523063 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ba8004810 msgr2=0x7f9bb8155790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:26.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.348+0000 7f9bb7fff700 1 --2- 172.21.15.31:0/1222523063 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ba8004810 0x7f9bb8155790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:26.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.348+0000 7f9bb7fff700 1 -- 172.21.15.31:0/1222523063 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bb80c5ee0 con 0x7f9bb80b0b70 2023-10-06T02:29:26.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.348+0000 7f9bb7fff700 1 --2- 172.21.15.31:0/1222523063 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9bb80b0b70 0x7f9bb81337f0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f9ba4012f60 tx=0x7f9ba4015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:26.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.348+0000 7f9bb4ff9700 1 -- 172.21.15.31:0/1222523063 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ba400c5d0 con 0x7f9bb80b0b70 2023-10-06T02:29:26.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.348+0000 7f9bb4ff9700 1 -- 172.21.15.31:0/1222523063 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ba400cab0 con 0x7f9bb80b0b70 2023-10-06T02:29:26.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.348+0000 7f9bb4ff9700 1 -- 172.21.15.31:0/1222523063 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ba4006a70 con 0x7f9bb80b0b70 2023-10-06T02:29:26.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.348+0000 7f9bbdf9d700 1 -- 172.21.15.31:0/1222523063 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bb80b13b0 con 0x7f9bb80b0b70 2023-10-06T02:29:26.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.349+0000 7f9bbdf9d700 1 -- 172.21.15.31:0/1222523063 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bb80c4d30 con 0x7f9bb80b0b70 2023-10-06T02:29:26.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.350+0000 7f9bbdf9d700 1 -- 172.21.15.31:0/1222523063 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bb804c3c0 con 0x7f9bb80b0b70 2023-10-06T02:29:26.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.350+0000 7f9bb4ff9700 1 -- 172.21.15.31:0/1222523063 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f9ba4019070 con 0x7f9bb80b0b70 2023-10-06T02:29:26.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.351+0000 7f9bb4ff9700 1 --2- 172.21.15.31:0/1222523063 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9b9c067320 0x7f9b9c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:26.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.351+0000 7f9bb4ff9700 1 -- 172.21.15.31:0/1222523063 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f9ba4085ec0 con 0x7f9bb80b0b70 2023-10-06T02:29:26.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.352+0000 7f9bbcf9b700 1 --2- 172.21.15.31:0/1222523063 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9b9c067320 0x7f9b9c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:26.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.352+0000 7f9bbcf9b700 1 --2- 172.21.15.31:0/1222523063 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9b9c067320 0x7f9b9c0697d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9bac006f00 tx=0x7f9bac0091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:26.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.356+0000 7f9bb4ff9700 1 -- 172.21.15.31:0/1222523063 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9ba4052cb0 con 0x7f9bb80b0b70 2023-10-06T02:29:26.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.560+0000 7f9bbdf9d700 1 -- 172.21.15.31:0/1222523063 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f9bb80c51e0 con 0x7f9bb80b0b70 2023-10-06T02:29:26.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.564+0000 7f9bb4ff9700 1 -- 172.21.15.31:0/1222523063 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) v1 ==== 104+0+0 (secure 0 0 0) 0x7f9ba40170a0 con 0x7f9bb80b0b70 2023-10-06T02:29:26.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.567+0000 7f9bbdf9d700 1 -- 172.21.15.31:0/1222523063 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9b9c067320 msgr2=0x7f9b9c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:26.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.567+0000 7f9bbdf9d700 1 --2- 172.21.15.31:0/1222523063 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9b9c067320 0x7f9b9c0697d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9bac006f00 tx=0x7f9bac0091a0 comp rx=0 tx=0).stop 2023-10-06T02:29:26.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.567+0000 7f9bbdf9d700 1 -- 172.21.15.31:0/1222523063 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9bb80b0b70 msgr2=0x7f9bb81337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:26.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.567+0000 7f9bbdf9d700 1 --2- 172.21.15.31:0/1222523063 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9bb80b0b70 0x7f9bb81337f0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f9ba4012f60 tx=0x7f9ba4015000 comp rx=0 tx=0).stop 2023-10-06T02:29:26.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.567+0000 7f9bbdf9d700 1 -- 172.21.15.31:0/1222523063 shutdown_connections 2023-10-06T02:29:26.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.567+0000 7f9bbdf9d700 1 --2- 172.21.15.31:0/1222523063 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9b9c067320 0x7f9b9c0697d0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:26.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.567+0000 7f9bbdf9d700 1 --2- 172.21.15.31:0/1222523063 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9bb80c5530 0x7f9bb80c59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:26.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.567+0000 7f9bbdf9d700 1 --2- 172.21.15.31:0/1222523063 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ba8004810 0x7f9bb8155790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:26.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.567+0000 7f9bbdf9d700 1 --2- 172.21.15.31:0/1222523063 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9bb80b0b70 0x7f9bb81337f0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:26.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.567+0000 7f9bbdf9d700 1 -- 172.21.15.31:0/1222523063 >> 172.21.15.31:0/1222523063 conn(0x7f9bb806d110 msgr2=0x7f9bb80c10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:26.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.567+0000 7f9bbdf9d700 1 -- 172.21.15.31:0/1222523063 shutdown_connections 2023-10-06T02:29:26.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.567+0000 7f9bbdf9d700 1 -- 172.21.15.31:0/1222523063 wait complete. 2023-10-06T02:29:26.586 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity' 2023-10-06T02:29:26.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.772+0000 7fba47fff700 1 Processor -- start 2023-10-06T02:29:26.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.773+0000 7fba47fff700 1 -- start start 2023-10-06T02:29:26.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.774+0000 7fba47fff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fba4815ca70 0x7fba4806aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:26.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.774+0000 7fba47fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fba480c7e60 0x7fba4806b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:26.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.774+0000 7fba47fff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba4806d5b0 con 0x7fba4815ce40 2023-10-06T02:29:26.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.774+0000 7fba47fff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba480b00b0 con 0x7fba480c7e60 2023-10-06T02:29:26.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.774+0000 7fba47fff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba4806b6b0 con 0x7fba4815ca70 2023-10-06T02:29:26.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.774+0000 7fba477fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fba480c7e60 0x7fba4806b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:26.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.774+0000 7fba477fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fba480c7e60 0x7fba4806b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52932/0 (socket says 172.21.15.31:52932) 2023-10-06T02:29:26.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.774+0000 7fba477fe700 1 -- 172.21.15.31:0/2588962822 learned_addr learned my addr 172.21.15.31:0/2588962822 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:26.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.774+0000 7fba46ffd700 1 --2- 172.21.15.31:0/2588962822 >> v2:172.21.15.62:3300/0 conn(0x7fba4815ca70 0x7fba4806aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:26.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.774+0000 7fba477fe700 1 -- 172.21.15.31:0/2588962822 >> v2:172.21.15.62:3300/0 conn(0x7fba4815ca70 msgr2=0x7fba4806aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:26.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.774+0000 7fba477fe700 1 --2- 172.21.15.31:0/2588962822 >> v2:172.21.15.62:3300/0 conn(0x7fba4815ca70 0x7fba4806aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:26.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.774+0000 7fba477fe700 1 -- 172.21.15.31:0/2588962822 >> v1:172.21.15.62:6789/0 conn(0x7fba4815ce40 legacy=0x7fba480c7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:26.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.774+0000 7fba477fe700 1 -- 172.21.15.31:0/2588962822 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba4806b990 con 0x7fba480c7e60 2023-10-06T02:29:26.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.774+0000 7fba46ffd700 1 --2- 172.21.15.31:0/2588962822 >> v2:172.21.15.62:3300/0 conn(0x7fba4815ca70 0x7fba4806aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:29:26.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.775+0000 7fba477fe700 1 --2- 172.21.15.31:0/2588962822 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba480c7e60 0x7fba4806b010 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fba4806cc70 tx=0x7fba40009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d98a3bc51e8ec739 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:26.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.775+0000 7fba45ffb700 1 -- 172.21.15.31:0/2588962822 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba4000e070 con 0x7fba480c7e60 2023-10-06T02:29:26.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.775+0000 7fba45ffb700 1 -- 172.21.15.31:0/2588962822 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba4000ba10 con 0x7fba480c7e60 2023-10-06T02:29:26.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.775+0000 7fba45ffb700 1 -- 172.21.15.31:0/2588962822 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba400092e0 con 0x7fba480c7e60 2023-10-06T02:29:26.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.775+0000 7fba47fff700 1 -- 172.21.15.31:0/2588962822 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba480c7e60 msgr2=0x7fba4806b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:26.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.775+0000 7fba47fff700 1 --2- 172.21.15.31:0/2588962822 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba480c7e60 0x7fba4806b010 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fba4806cc70 tx=0x7fba40009840 comp rx=0 tx=0).stop 2023-10-06T02:29:26.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.776+0000 7fba47fff700 1 -- 172.21.15.31:0/2588962822 shutdown_connections 2023-10-06T02:29:26.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.776+0000 7fba47fff700 1 --2- 172.21.15.31:0/2588962822 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba480c7e60 0x7fba4806b010 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:26.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.776+0000 7fba47fff700 1 --2- 172.21.15.31:0/2588962822 >> v2:172.21.15.62:3300/0 conn(0x7fba4815ca70 0x7fba4806aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:26.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.776+0000 7fba47fff700 1 -- 172.21.15.31:0/2588962822 >> 172.21.15.31:0/2588962822 conn(0x7fba480675a0 msgr2=0x7fba480679a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:26.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.776+0000 7fba47fff700 1 -- 172.21.15.31:0/2588962822 shutdown_connections 2023-10-06T02:29:26.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.776+0000 7fba47fff700 1 -- 172.21.15.31:0/2588962822 wait complete. 2023-10-06T02:29:26.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.777+0000 7fba47fff700 1 Processor -- start 2023-10-06T02:29:26.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.777+0000 7fba47fff700 1 -- start start 2023-10-06T02:29:26.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.778+0000 7fba47fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba480c7e60 0x7fba480c0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:26.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.778+0000 7fba47fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba4815ca70 0x7fba4816a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:26.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.778+0000 7fba47fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fba4815ce40 0x7fba480c28b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:26.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.778+0000 7fba47fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba4816a830 con 0x7fba480c7e60 2023-10-06T02:29:26.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.778+0000 7fba47fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba480c1e90 con 0x7fba4815ca70 2023-10-06T02:29:26.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.778+0000 7fba47fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba480c1310 con 0x7fba4815ce40 2023-10-06T02:29:26.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.778+0000 7fba46ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba480c7e60 0x7fba480c0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:26.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.779+0000 7fba46ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba480c7e60 0x7fba480c0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52938/0 (socket says 172.21.15.31:52938) 2023-10-06T02:29:26.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.779+0000 7fba46ffd700 1 -- 172.21.15.31:0/2102462098 learned_addr learned my addr 172.21.15.31:0/2102462098 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:26.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.779+0000 7fba467fc700 1 --2- 172.21.15.31:0/2102462098 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba4815ca70 0x7fba4816a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:26.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.779+0000 7fba477fe700 1 --2- 172.21.15.31:0/2102462098 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fba4815ce40 0x7fba480c28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:26.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.779+0000 7fba46ffd700 1 -- 172.21.15.31:0/2102462098 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fba4815ce40 msgr2=0x7fba480c28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:26.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.779+0000 7fba46ffd700 1 --2- 172.21.15.31:0/2102462098 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fba4815ce40 0x7fba480c28b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:26.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.779+0000 7fba46ffd700 1 -- 172.21.15.31:0/2102462098 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba4815ca70 msgr2=0x7fba4816a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:26.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.779+0000 7fba46ffd700 1 --2- 172.21.15.31:0/2102462098 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba4815ca70 0x7fba4816a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:26.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.779+0000 7fba46ffd700 1 -- 172.21.15.31:0/2102462098 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba40009ae0 con 0x7fba480c7e60 2023-10-06T02:29:26.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.779+0000 7fba46ffd700 1 --2- 172.21.15.31:0/2102462098 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba480c7e60 0x7fba480c0c10 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fba3800c830 tx=0x7fba38008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:26.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.779+0000 7fba2effd700 1 -- 172.21.15.31:0/2102462098 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba38011600 con 0x7fba480c7e60 2023-10-06T02:29:26.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.780+0000 7fba2effd700 1 -- 172.21.15.31:0/2102462098 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba38011760 con 0x7fba480c7e60 2023-10-06T02:29:26.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.780+0000 7fba47fff700 1 -- 172.21.15.31:0/2102462098 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba480c15f0 con 0x7fba480c7e60 2023-10-06T02:29:26.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.780+0000 7fba2effd700 1 -- 172.21.15.31:0/2102462098 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba38011b20 con 0x7fba480c7e60 2023-10-06T02:29:26.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.780+0000 7fba47fff700 1 -- 172.21.15.31:0/2102462098 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba480c1bb0 con 0x7fba480c7e60 2023-10-06T02:29:26.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.780+0000 7fba47fff700 1 -- 172.21.15.31:0/2102462098 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba4804c3c0 con 0x7fba480c7e60 2023-10-06T02:29:26.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.781+0000 7fba2effd700 1 -- 172.21.15.31:0/2102462098 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fba3800a3e0 con 0x7fba480c7e60 2023-10-06T02:29:26.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.782+0000 7fba2effd700 1 --2- 172.21.15.31:0/2102462098 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fba34067320 0x7fba340697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:26.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.782+0000 7fba2effd700 1 -- 172.21.15.31:0/2102462098 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5292+0+0 (secure 0 0 0) 0x7fba38084c80 con 0x7fba480c7e60 2023-10-06T02:29:26.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.783+0000 7fba467fc700 1 --2- 172.21.15.31:0/2102462098 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fba34067320 0x7fba340697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:26.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.783+0000 7fba467fc700 1 --2- 172.21.15.31:0/2102462098 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fba34067320 0x7fba340697d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fba3c006fd0 tx=0x7fba3c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:26.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:26.786+0000 7fba2effd700 1 -- 172.21.15.31:0/2102462098 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fba380519a0 con 0x7fba480c7e60 2023-10-06T02:29:27.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.069+0000 7fba47fff700 1 -- 172.21.15.31:0/2102462098 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity"]} v 0) v1 -- 0x7fba4806bc70 con 0x7fba480c7e60 2023-10-06T02:29:27.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.117+0000 7fba2effd700 1 -- 172.21.15.31:0/2102462098 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity"]}]=0 v352) v1 ==== 149+0+0 (secure 0 0 0) 0x7fba380150d0 con 0x7fba480c7e60 2023-10-06T02:29:27.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.119+0000 7fba47fff700 1 -- 172.21.15.31:0/2102462098 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fba34067320 msgr2=0x7fba340697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:27.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.119+0000 7fba47fff700 1 --2- 172.21.15.31:0/2102462098 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fba34067320 0x7fba340697d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fba3c006fd0 tx=0x7fba3c008040 comp rx=0 tx=0).stop 2023-10-06T02:29:27.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.119+0000 7fba47fff700 1 -- 172.21.15.31:0/2102462098 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba480c7e60 msgr2=0x7fba480c0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:27.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.119+0000 7fba47fff700 1 --2- 172.21.15.31:0/2102462098 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba480c7e60 0x7fba480c0c10 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fba3800c830 tx=0x7fba38008040 comp rx=0 tx=0).stop 2023-10-06T02:29:27.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.119+0000 7fba47fff700 1 -- 172.21.15.31:0/2102462098 shutdown_connections 2023-10-06T02:29:27.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.119+0000 7fba47fff700 1 --2- 172.21.15.31:0/2102462098 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fba34067320 0x7fba340697d0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:27.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.119+0000 7fba47fff700 1 --2- 172.21.15.31:0/2102462098 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fba4815ce40 0x7fba480c28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:27.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.119+0000 7fba47fff700 1 --2- 172.21.15.31:0/2102462098 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba480c7e60 0x7fba480c0c10 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:27.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.119+0000 7fba47fff700 1 --2- 172.21.15.31:0/2102462098 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba4815ca70 0x7fba4816a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:27.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.119+0000 7fba47fff700 1 -- 172.21.15.31:0/2102462098 >> 172.21.15.31:0/2102462098 conn(0x7fba480675a0 msgr2=0x7fba480b22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:27.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.120+0000 7fba47fff700 1 -- 172.21.15.31:0/2102462098 shutdown_connections 2023-10-06T02:29:27.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.120+0000 7fba47fff700 1 -- 172.21.15.31:0/2102462098 wait complete. 2023-10-06T02:29:27.138 INFO:tasks.cephfs_test_runner:poor man's upgrade test -- theme continues... ... ok 2023-10-06T02:29:27.138 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_no_upgrade_v1_to_v2 (tasks.cephfs.test_volumes.TestMisc) poor man's upgrade test -- theme continues... 2023-10-06T02:29:27.138 INFO:tasks.cephfs_test_runner:test_subvolume_no_upgrade_v1_to_v2 (tasks.cephfs.test_volumes.TestMisc) 2023-10-06T02:29:27.139 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2' 2023-10-06T02:29:27.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.325+0000 7f2198b9d700 1 Processor -- start 2023-10-06T02:29:27.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.325+0000 7f2198b9d700 1 -- start start 2023-10-06T02:29:27.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.326+0000 7f2198b9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f219406aaa0 0x7f219415a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:27.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.326+0000 7f2198b9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f219415ae70 con 0x7f219413fb20 2023-10-06T02:29:27.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.326+0000 7f2198b9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f219406af60 con 0x7f219413fef0 2023-10-06T02:29:27.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.326+0000 7f2198b9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f219406b240 con 0x7f219406aaa0 2023-10-06T02:29:27.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.326+0000 7f21937fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f219413fb20 0x7f21940c7990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:33106/0 (socket says 172.21.15.31:33106) 2023-10-06T02:29:27.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.326+0000 7f21937fe700 1 -- 172.21.15.31:0/1051780010 learned_addr learned my addr 172.21.15.31:0/1051780010 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:29:27.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.327+0000 7f2193fff700 1 --2- 172.21.15.31:0/1051780010 >> v2:172.21.15.62:3300/0 conn(0x7f219406aaa0 0x7f219415a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:27.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.327+0000 7f21927fc700 1 -- 172.21.15.31:0/1051780010 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 540295783 0 0) 0x7f2184001110 con 0x7f219413fb20 2023-10-06T02:29:27.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.327+0000 7f21927fc700 1 -- 172.21.15.31:0/1051780010 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f217c003570 con 0x7f219413fb20 2023-10-06T02:29:27.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.327+0000 7f21927fc700 1 -- 172.21.15.31:0/1051780010 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2550287546 0 0) 0x7f219406af60 con 0x7f219413fef0 2023-10-06T02:29:27.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.327+0000 7f21927fc700 1 -- 172.21.15.31:0/1051780010 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2184001110 con 0x7f219413fef0 2023-10-06T02:29:27.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.327+0000 7f21927fc700 1 -- 172.21.15.31:0/1051780010 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1077783999 0 0) 0x7f217c003570 con 0x7f219413fb20 2023-10-06T02:29:27.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.327+0000 7f21927fc700 1 -- 172.21.15.31:0/1051780010 >> v2:172.21.15.62:3300/0 conn(0x7f219406aaa0 msgr2=0x7f219415a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:27.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.327+0000 7f21927fc700 1 --2- 172.21.15.31:0/1051780010 >> v2:172.21.15.62:3300/0 conn(0x7f219406aaa0 0x7f219415a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:27.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.327+0000 7f21927fc700 1 -- 172.21.15.31:0/1051780010 >> v1:172.21.15.62:6789/0 conn(0x7f219413fef0 legacy=0x7f21940c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:27.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.327+0000 7f21927fc700 1 -- 172.21.15.31:0/1051780010 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f219406b520 con 0x7f219413fb20 2023-10-06T02:29:27.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.327+0000 7f21927fc700 1 -- 172.21.15.31:0/1051780010 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7f2184001580 con 0x7f219413fb20 2023-10-06T02:29:27.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.327+0000 7f21927fc700 1 -- 172.21.15.31:0/1051780010 >> v1:172.21.15.31:6789/0 conn(0x7f219413fb20 legacy=0x7f21940c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:27.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.327+0000 7f21927fc700 1 --2- 172.21.15.31:0/1051780010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f217c003c40 0x7f217c004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:27.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.327+0000 7f21927fc700 1 --2- 172.21.15.31:0/1051780010 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f217c004810 0x7f217c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:27.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.327+0000 7f21927fc700 1 --2- 172.21.15.31:0/1051780010 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f217c005210 0x7f217c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:27.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.327+0000 7f21927fc700 1 -- 172.21.15.31:0/1051780010 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f219406b240 con 0x7f217c005210 2023-10-06T02:29:27.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.327+0000 7f21927fc700 1 -- 172.21.15.31:0/1051780010 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f217c00c130 con 0x7f217c003c40 2023-10-06T02:29:27.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.327+0000 7f21927fc700 1 -- 172.21.15.31:0/1051780010 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f217c00c2c0 con 0x7f217c004810 2023-10-06T02:29:27.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.328+0000 7f2193fff700 1 --2- 172.21.15.31:0/1051780010 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f217c005210 0x7f217c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:27.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.328+0000 7f21937fe700 1 --2- 172.21.15.31:0/1051780010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f217c003c40 0x7f217c004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:27.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.328+0000 7f2192ffd700 1 --2- 172.21.15.31:0/1051780010 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f217c004810 0x7f217c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:27.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.328+0000 7f2193fff700 1 -- 172.21.15.31:0/1051780010 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f217c004810 msgr2=0x7f217c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:27.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.328+0000 7f2193fff700 1 --2- 172.21.15.31:0/1051780010 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f217c004810 0x7f217c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:27.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.328+0000 7f2193fff700 1 -- 172.21.15.31:0/1051780010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f217c003c40 msgr2=0x7f217c004030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:27.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.328+0000 7f2193fff700 1 --2- 172.21.15.31:0/1051780010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f217c003c40 0x7f217c004030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:27.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.328+0000 7f2193fff700 1 -- 172.21.15.31:0/1051780010 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f217c003570 con 0x7f217c005210 2023-10-06T02:29:27.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.328+0000 7f2193fff700 1 -- 172.21.15.31:0/1051780010 reap_dead start 2023-10-06T02:29:27.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.328+0000 7f2193fff700 1 --2- 172.21.15.31:0/1051780010 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f217c005210 0x7f217c00ba60 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f218c00e210 tx=0x7f218c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba250951c27c73d5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:27.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.328+0000 7f21927fc700 1 -- 172.21.15.31:0/1051780010 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f218c009ac0 con 0x7f217c005210 2023-10-06T02:29:27.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.328+0000 7f21927fc700 1 -- 172.21.15.31:0/1051780010 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f218c00f040 con 0x7f217c005210 2023-10-06T02:29:27.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.329+0000 7f21927fc700 1 -- 172.21.15.31:0/1051780010 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f218c00ed10 con 0x7f217c005210 2023-10-06T02:29:27.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.329+0000 7f2198b9d700 1 -- 172.21.15.31:0/1051780010 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f217c005210 msgr2=0x7f217c00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:27.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.329+0000 7f2198b9d700 1 --2- 172.21.15.31:0/1051780010 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f217c005210 0x7f217c00ba60 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f218c00e210 tx=0x7f218c00b040 comp rx=0 tx=0).stop 2023-10-06T02:29:27.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.329+0000 7f2198b9d700 1 -- 172.21.15.31:0/1051780010 shutdown_connections 2023-10-06T02:29:27.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.329+0000 7f2198b9d700 1 --2- 172.21.15.31:0/1051780010 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f217c005210 0x7f217c00ba60 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:27.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.329+0000 7f2198b9d700 1 -- 172.21.15.31:0/1051780010 >> 172.21.15.31:0/1051780010 conn(0x7f219406d170 msgr2=0x7f21940cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:27.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.329+0000 7f2198b9d700 1 -- 172.21.15.31:0/1051780010 shutdown_connections 2023-10-06T02:29:27.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.329+0000 7f2198b9d700 1 -- 172.21.15.31:0/1051780010 wait complete. 2023-10-06T02:29:27.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.331+0000 7f2198b9d700 1 Processor -- start 2023-10-06T02:29:27.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.331+0000 7f2198b9d700 1 -- start start 2023-10-06T02:29:27.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.332+0000 7f2198b9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f217c005210 0x7f219416dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:27.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.332+0000 7f2198b9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21940ae1e0 0x7f2194155870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:27.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.332+0000 7f2198b9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21940ae650 0x7f2194155db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:27.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.332+0000 7f2198b9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f219416e210 con 0x7f21940ae1e0 2023-10-06T02:29:27.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.332+0000 7f2198b9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2194157940 con 0x7f21940ae650 2023-10-06T02:29:27.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.332+0000 7f2198b9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2194156d30 con 0x7f217c005210 2023-10-06T02:29:27.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.332+0000 7f2192ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21940ae1e0 0x7f2194155870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:27.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.332+0000 7f2192ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21940ae1e0 0x7f2194155870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52950/0 (socket says 172.21.15.31:52950) 2023-10-06T02:29:27.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.332+0000 7f2192ffd700 1 -- 172.21.15.31:0/3294019530 learned_addr learned my addr 172.21.15.31:0/3294019530 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:27.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.332+0000 7f2192ffd700 1 -- 172.21.15.31:0/3294019530 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f217c005210 msgr2=0x7f219416dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:27.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.332+0000 7f21937fe700 1 --2- 172.21.15.31:0/3294019530 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f217c005210 0x7f219416dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:27.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.332+0000 7f2193fff700 1 --2- 172.21.15.31:0/3294019530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21940ae650 0x7f2194155db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:27.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.332+0000 7f2192ffd700 1 --2- 172.21.15.31:0/3294019530 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f217c005210 0x7f219416dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:27.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.332+0000 7f2192ffd700 1 -- 172.21.15.31:0/3294019530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21940ae650 msgr2=0x7f2194155db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:27.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.332+0000 7f2192ffd700 1 --2- 172.21.15.31:0/3294019530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21940ae650 0x7f2194155db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:27.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.332+0000 7f2192ffd700 1 -- 172.21.15.31:0/3294019530 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2194157010 con 0x7f21940ae1e0 2023-10-06T02:29:27.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.332+0000 7f21937fe700 1 --2- 172.21.15.31:0/3294019530 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f217c005210 0x7f219416dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:29:27.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.332+0000 7f2193fff700 1 --2- 172.21.15.31:0/3294019530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21940ae650 0x7f2194155db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:29:27.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.333+0000 7f2192ffd700 1 --2- 172.21.15.31:0/3294019530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21940ae1e0 0x7f2194155870 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f218800e5a0 tx=0x7f2188009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:27.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.333+0000 7f217b7fe700 1 -- 172.21.15.31:0/3294019530 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2188002690 con 0x7f21940ae1e0 2023-10-06T02:29:27.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.333+0000 7f217b7fe700 1 -- 172.21.15.31:0/3294019530 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2188010040 con 0x7f21940ae1e0 2023-10-06T02:29:27.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.333+0000 7f217b7fe700 1 -- 172.21.15.31:0/3294019530 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2188002a40 con 0x7f21940ae1e0 2023-10-06T02:29:27.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.333+0000 7f2198b9d700 1 -- 172.21.15.31:0/3294019530 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21941572f0 con 0x7f21940ae1e0 2023-10-06T02:29:27.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.333+0000 7f2198b9d700 1 -- 172.21.15.31:0/3294019530 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2194157dd0 con 0x7f21940ae1e0 2023-10-06T02:29:27.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.334+0000 7f2198b9d700 1 -- 172.21.15.31:0/3294019530 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f219406f6e0 con 0x7f21940ae1e0 2023-10-06T02:29:27.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.335+0000 7f217b7fe700 1 -- 172.21.15.31:0/3294019530 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f218801d3b0 con 0x7f21940ae1e0 2023-10-06T02:29:27.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.336+0000 7f217b7fe700 1 --2- 172.21.15.31:0/3294019530 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f2180067320 0x7f21800697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:27.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.336+0000 7f217b7fe700 1 -- 172.21.15.31:0/3294019530 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f2188084750 con 0x7f21940ae1e0 2023-10-06T02:29:27.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.336+0000 7f21937fe700 1 --2- 172.21.15.31:0/3294019530 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f2180067320 0x7f21800697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:27.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.337+0000 7f21937fe700 1 --2- 172.21.15.31:0/3294019530 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f2180067320 0x7f21800697d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f21840010e0 tx=0x7f2184009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:27.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.340+0000 7f217b7fe700 1 -- 172.21.15.31:0/3294019530 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2188051430 con 0x7f21940ae1e0 2023-10-06T02:29:27.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:27.620+0000 7f2198b9d700 1 -- 172.21.15.31:0/3294019530 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2"]} v 0) v1 -- 0x7f21940c1910 con 0x7f21940ae1e0 2023-10-06T02:29:28.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.119+0000 7f217b7fe700 1 -- 172.21.15.31:0/3294019530 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2"]}]=0 v353) v1 ==== 151+0+0 (secure 0 0 0) 0x7f2188054a50 con 0x7f21940ae1e0 2023-10-06T02:29:28.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.122+0000 7f2198b9d700 1 -- 172.21.15.31:0/3294019530 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f2180067320 msgr2=0x7f21800697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:28.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.122+0000 7f2198b9d700 1 --2- 172.21.15.31:0/3294019530 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f2180067320 0x7f21800697d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f21840010e0 tx=0x7f2184009040 comp rx=0 tx=0).stop 2023-10-06T02:29:28.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.122+0000 7f2198b9d700 1 -- 172.21.15.31:0/3294019530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21940ae1e0 msgr2=0x7f2194155870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:28.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.122+0000 7f2198b9d700 1 --2- 172.21.15.31:0/3294019530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21940ae1e0 0x7f2194155870 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f218800e5a0 tx=0x7f2188009040 comp rx=0 tx=0).stop 2023-10-06T02:29:28.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.122+0000 7f2198b9d700 1 -- 172.21.15.31:0/3294019530 shutdown_connections 2023-10-06T02:29:28.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.122+0000 7f2198b9d700 1 --2- 172.21.15.31:0/3294019530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21940ae650 0x7f2194155db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:28.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.122+0000 7f2198b9d700 1 --2- 172.21.15.31:0/3294019530 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f2180067320 0x7f21800697d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:28.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.122+0000 7f2198b9d700 1 --2- 172.21.15.31:0/3294019530 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f217c005210 0x7f219416dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:28.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.122+0000 7f2198b9d700 1 --2- 172.21.15.31:0/3294019530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21940ae1e0 0x7f2194155870 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:28.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.122+0000 7f2198b9d700 1 -- 172.21.15.31:0/3294019530 >> 172.21.15.31:0/3294019530 conn(0x7f219406d170 msgr2=0x7f21940cae50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:28.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.122+0000 7f2198b9d700 1 -- 172.21.15.31:0/3294019530 shutdown_connections 2023-10-06T02:29:28.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.122+0000 7f2198b9d700 1 -- 172.21.15.31:0/3294019530 wait complete. 2023-10-06T02:29:28.143 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T02:29:28.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.336+0000 7fbdfd99d700 1 Processor -- start 2023-10-06T02:29:28.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.337+0000 7fbdfd99d700 1 -- start start 2023-10-06T02:29:28.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.338+0000 7fbdfd99d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbdf806a210 0x7fbdf80c6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:28.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.338+0000 7fbdfd99d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdf815b640 con 0x7fbdf806de50 2023-10-06T02:29:28.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.338+0000 7fbdfd99d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdf815b920 con 0x7fbdf8140a70 2023-10-06T02:29:28.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.338+0000 7fbdfd99d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf80c6800 con 0x7fbdf806a210 2023-10-06T02:29:28.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.338+0000 7fbdfd19c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fbdf806de50 0x7fbdf80c7190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:36316/0 (socket says 172.21.15.31:36316) 2023-10-06T02:29:28.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.338+0000 7fbdfd19c700 1 -- 172.21.15.31:0/3401772156 learned_addr learned my addr 172.21.15.31:0/3401772156 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:29:28.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.339+0000 7fbdf7fff700 1 --2- 172.21.15.31:0/3401772156 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbdf806a210 0x7fbdf80c6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:28.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.339+0000 7fbdf77fe700 1 -- 172.21.15.31:0/3401772156 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4292687014 0 0) 0x7fbdf815b640 con 0x7fbdf806de50 2023-10-06T02:29:28.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.339+0000 7fbdf77fe700 1 -- 172.21.15.31:0/3401772156 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbde8003570 con 0x7fbdf806de50 2023-10-06T02:29:28.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.339+0000 7fbdf77fe700 1 -- 172.21.15.31:0/3401772156 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 808691106 0 0) 0x7fbdf815b920 con 0x7fbdf8140a70 2023-10-06T02:29:28.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.339+0000 7fbdf77fe700 1 -- 172.21.15.31:0/3401772156 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbdf815b640 con 0x7fbdf8140a70 2023-10-06T02:29:28.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.339+0000 7fbdf77fe700 1 -- 172.21.15.31:0/3401772156 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 605137314 0 0) 0x7fbde8003570 con 0x7fbdf806de50 2023-10-06T02:29:28.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.339+0000 7fbdf7fff700 1 -- 172.21.15.31:0/3401772156 >> v1:172.21.15.62:6789/0 conn(0x7fbdf8140a70 legacy=0x7fbdf8069b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:28.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.339+0000 7fbdf7fff700 1 -- 172.21.15.31:0/3401772156 >> v1:172.21.15.31:6789/0 conn(0x7fbdf806de50 legacy=0x7fbdf80c7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:28.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.339+0000 7fbdf7fff700 1 -- 172.21.15.31:0/3401772156 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdf815bc00 con 0x7fbdf806a210 2023-10-06T02:29:28.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.340+0000 7fbdf7fff700 1 --2- 172.21.15.31:0/3401772156 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbdf806a210 0x7fbdf80c6c50 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fbde4009910 tx=0x7fbde40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e70e30748ce95e51 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:28.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.340+0000 7fbdf77fe700 1 -- 172.21.15.31:0/3401772156 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf00017f0 con 0x7fbdf806a210 2023-10-06T02:29:28.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.340+0000 7fbdf77fe700 1 -- 172.21.15.31:0/3401772156 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbde4011420 con 0x7fbdf806a210 2023-10-06T02:29:28.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.340+0000 7fbdf77fe700 1 -- 172.21.15.31:0/3401772156 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde40117c0 con 0x7fbdf806a210 2023-10-06T02:29:28.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.340+0000 7fbdfd99d700 1 -- 172.21.15.31:0/3401772156 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbdf806a210 msgr2=0x7fbdf80c6c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:28.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.340+0000 7fbdfd99d700 1 --2- 172.21.15.31:0/3401772156 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbdf806a210 0x7fbdf80c6c50 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fbde4009910 tx=0x7fbde40091a0 comp rx=0 tx=0).stop 2023-10-06T02:29:28.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.340+0000 7fbdfd99d700 1 -- 172.21.15.31:0/3401772156 shutdown_connections 2023-10-06T02:29:28.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.340+0000 7fbdfd99d700 1 --2- 172.21.15.31:0/3401772156 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbdf806a210 0x7fbdf80c6c50 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:28.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.340+0000 7fbdfd99d700 1 -- 172.21.15.31:0/3401772156 >> 172.21.15.31:0/3401772156 conn(0x7fbdf806d110 msgr2=0x7fbdf80cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:28.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.340+0000 7fbdfd99d700 1 -- 172.21.15.31:0/3401772156 shutdown_connections 2023-10-06T02:29:28.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.341+0000 7fbdfd99d700 1 -- 172.21.15.31:0/3401772156 wait complete. 2023-10-06T02:29:28.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.342+0000 7fbdfd99d700 1 Processor -- start 2023-10-06T02:29:28.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.342+0000 7fbdfd99d700 1 -- start start 2023-10-06T02:29:28.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.343+0000 7fbdfd99d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbdf806a210 0x7fbdf8157bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:28.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.343+0000 7fbdfd99d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbdf806de50 0x7fbdf80cc030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:28.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.343+0000 7fbdfd99d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdf8140a70 0x7fbdf80cc570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:28.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.343+0000 7fbdfd99d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf8169780 con 0x7fbdf8140a70 2023-10-06T02:29:28.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.343+0000 7fbdfd99d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf8168bb0 con 0x7fbdf806a210 2023-10-06T02:29:28.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.343+0000 7fbdfd99d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf8165580 con 0x7fbdf806de50 2023-10-06T02:29:28.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.343+0000 7fbdfd19c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdf8140a70 0x7fbdf80cc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:28.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.343+0000 7fbdfd19c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdf8140a70 0x7fbdf80cc570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34230/0 (socket says 172.21.15.31:34230) 2023-10-06T02:29:28.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.343+0000 7fbdfd19c700 1 -- 172.21.15.31:0/1935331530 learned_addr learned my addr 172.21.15.31:0/1935331530 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:28.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.343+0000 7fbdf7fff700 1 --2- 172.21.15.31:0/1935331530 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbdf806de50 0x7fbdf80cc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:28.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.343+0000 7fbdfc99b700 1 --2- 172.21.15.31:0/1935331530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbdf806a210 0x7fbdf8157bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:28.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.344+0000 7fbdfd19c700 1 -- 172.21.15.31:0/1935331530 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbdf806de50 msgr2=0x7fbdf80cc030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:28.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.344+0000 7fbdfd19c700 1 --2- 172.21.15.31:0/1935331530 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbdf806de50 0x7fbdf80cc030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:28.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.344+0000 7fbdfd19c700 1 -- 172.21.15.31:0/1935331530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbdf806a210 msgr2=0x7fbdf8157bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:28.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.344+0000 7fbdfd19c700 1 --2- 172.21.15.31:0/1935331530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbdf806a210 0x7fbdf8157bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:28.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.344+0000 7fbdfd19c700 1 -- 172.21.15.31:0/1935331530 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbde4009420 con 0x7fbdf8140a70 2023-10-06T02:29:28.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.344+0000 7fbdfd19c700 1 --2- 172.21.15.31:0/1935331530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdf8140a70 0x7fbdf80cc570 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fbdf0002250 tx=0x7fbdf0007d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:28.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.344+0000 7fbdf4ff9700 1 -- 172.21.15.31:0/1935331530 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf0004dc0 con 0x7fbdf8140a70 2023-10-06T02:29:28.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.344+0000 7fbdf4ff9700 1 -- 172.21.15.31:0/1935331530 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdf000a660 con 0x7fbdf8140a70 2023-10-06T02:29:28.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.344+0000 7fbdf4ff9700 1 -- 172.21.15.31:0/1935331530 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf000aa20 con 0x7fbdf8140a70 2023-10-06T02:29:28.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.344+0000 7fbdfd99d700 1 -- 172.21.15.31:0/1935331530 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdf8168e50 con 0x7fbdf8140a70 2023-10-06T02:29:28.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.345+0000 7fbdfd99d700 1 -- 172.21.15.31:0/1935331530 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdf8169410 con 0x7fbdf8140a70 2023-10-06T02:29:28.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.346+0000 7fbdfd99d700 1 -- 172.21.15.31:0/1935331530 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdf804c3c0 con 0x7fbdf8140a70 2023-10-06T02:29:28.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.346+0000 7fbdf4ff9700 1 -- 172.21.15.31:0/1935331530 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fbdf0007150 con 0x7fbdf8140a70 2023-10-06T02:29:28.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.347+0000 7fbdf4ff9700 1 --2- 172.21.15.31:0/1935331530 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fbddc067320 0x7fbddc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:28.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.347+0000 7fbdf4ff9700 1 -- 172.21.15.31:0/1935331530 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5292+0+0 (secure 0 0 0) 0x7fbdf0084280 con 0x7fbdf8140a70 2023-10-06T02:29:28.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.348+0000 7fbdfc99b700 1 --2- 172.21.15.31:0/1935331530 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fbddc067320 0x7fbddc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:28.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.348+0000 7fbdfc99b700 1 --2- 172.21.15.31:0/1935331530 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fbddc067320 0x7fbddc0697d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fbdec009310 tx=0x7fbdec0095c0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:28.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.351+0000 7fbdf4ff9700 1 -- 172.21.15.31:0/1935331530 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbdf00510f0 con 0x7fbdf8140a70 2023-10-06T02:29:28.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.552+0000 7fbdfd99d700 1 -- 172.21.15.31:0/1935331530 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fbdf815bee0 con 0x7fbdf8140a70 2023-10-06T02:29:28.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.556+0000 7fbdf4ff9700 1 -- 172.21.15.31:0/1935331530 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7fbdf0054710 con 0x7fbdf8140a70 2023-10-06T02:29:28.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.559+0000 7fbdfd99d700 1 -- 172.21.15.31:0/1935331530 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fbddc067320 msgr2=0x7fbddc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:28.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.559+0000 7fbdfd99d700 1 --2- 172.21.15.31:0/1935331530 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fbddc067320 0x7fbddc0697d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fbdec009310 tx=0x7fbdec0095c0 comp rx=0 tx=0).stop 2023-10-06T02:29:28.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.559+0000 7fbdfd99d700 1 -- 172.21.15.31:0/1935331530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdf8140a70 msgr2=0x7fbdf80cc570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:28.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.559+0000 7fbdfd99d700 1 --2- 172.21.15.31:0/1935331530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdf8140a70 0x7fbdf80cc570 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fbdf0002250 tx=0x7fbdf0007d50 comp rx=0 tx=0).stop 2023-10-06T02:29:28.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.559+0000 7fbdfd99d700 1 -- 172.21.15.31:0/1935331530 shutdown_connections 2023-10-06T02:29:28.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.559+0000 7fbdfd99d700 1 --2- 172.21.15.31:0/1935331530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdf8140a70 0x7fbdf80cc570 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:28.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.559+0000 7fbdfd99d700 1 --2- 172.21.15.31:0/1935331530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbdf806a210 0x7fbdf8157bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:28.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.559+0000 7fbdfd99d700 1 --2- 172.21.15.31:0/1935331530 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fbddc067320 0x7fbddc0697d0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:28.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.559+0000 7fbdfd99d700 1 --2- 172.21.15.31:0/1935331530 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbdf806de50 0x7fbdf80cc030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:28.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.559+0000 7fbdfd99d700 1 -- 172.21.15.31:0/1935331530 >> 172.21.15.31:0/1935331530 conn(0x7fbdf806d110 msgr2=0x7fbdf813d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:28.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.559+0000 7fbdfd99d700 1 -- 172.21.15.31:0/1935331530 shutdown_connections 2023-10-06T02:29:28.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.559+0000 7fbdfd99d700 1 -- 172.21.15.31:0/1935331530 wait complete. 2023-10-06T02:29:28.578 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T02:29:28.579 DEBUG:teuthology.orchestra.run.smithi031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T02:29:28.657 DEBUG:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T02:29:28.746 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T02:29:28.746 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T02:29:28.746 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:29:28.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.932+0000 7f71c6d9d700 1 Processor -- start 2023-10-06T02:29:28.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.932+0000 7f71c6d9d700 1 -- start start 2023-10-06T02:29:28.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.934+0000 7f71c6d9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f71c013fef0 0x7f71c015a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:28.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.934+0000 7f71c6d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f71c006aaa0 0x7f71c00c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:28.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.934+0000 7f71c6d9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71c015ae70 con 0x7f71c013fb20 2023-10-06T02:29:28.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.934+0000 7f71c6d9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71c006af60 con 0x7f71c013fef0 2023-10-06T02:29:28.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.934+0000 7f71c6d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71c006b240 con 0x7f71c006aaa0 2023-10-06T02:29:28.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.934+0000 7f71c559a700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f71c013fef0 0x7f71c015a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:28.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.934+0000 7f71c659c700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f71c006aaa0 0x7f71c00c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:28.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.934+0000 7f71c5d9b700 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f71c013fb20 0x7f71c00c7990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:52300/0 (socket says 172.21.15.31:52300) 2023-10-06T02:29:28.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.934+0000 7f71c5d9b700 1 -- 172.21.15.31:0/3165018161 learned_addr learned my addr 172.21.15.31:0/3165018161 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:29:28.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.935+0000 7f71c559a700 1 -- 172.21.15.31:0/3165018161 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f71c006aaa0 msgr2=0x7f71c00c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:28.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.935+0000 7f71c559a700 1 --2- 172.21.15.31:0/3165018161 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f71c006aaa0 0x7f71c00c9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:28.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.935+0000 7f71c559a700 1 -- 172.21.15.31:0/3165018161 >> v1:172.21.15.62:6789/0 conn(0x7f71c013fb20 legacy=0x7f71c00c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:28.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.935+0000 7f71c559a700 1 -- 172.21.15.31:0/3165018161 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71c006b520 con 0x7f71c013fef0 2023-10-06T02:29:28.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.935+0000 7f71c559a700 1 --2- 172.21.15.31:0/3165018161 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71c013fef0 0x7f71c015a930 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f71b00098d0 tx=0x7f71b0009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f26aade4a9fae9c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:28.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.935+0000 7f71c4d99700 1 -- 172.21.15.31:0/3165018161 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b0005c10 con 0x7f71c013fef0 2023-10-06T02:29:28.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.935+0000 7f71c4d99700 1 -- 172.21.15.31:0/3165018161 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71b0005d70 con 0x7f71c013fef0 2023-10-06T02:29:28.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.935+0000 7f71c4d99700 1 -- 172.21.15.31:0/3165018161 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b0006280 con 0x7f71c013fef0 2023-10-06T02:29:28.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.936+0000 7f71c6d9d700 1 -- 172.21.15.31:0/3165018161 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71c013fef0 msgr2=0x7f71c015a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:28.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.936+0000 7f71c6d9d700 1 --2- 172.21.15.31:0/3165018161 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71c013fef0 0x7f71c015a930 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f71b00098d0 tx=0x7f71b0009120 comp rx=0 tx=0).stop 2023-10-06T02:29:28.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.936+0000 7f71c6d9d700 1 -- 172.21.15.31:0/3165018161 shutdown_connections 2023-10-06T02:29:28.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.936+0000 7f71c6d9d700 1 --2- 172.21.15.31:0/3165018161 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f71c006aaa0 0x7f71c00c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:28.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.936+0000 7f71c6d9d700 1 --2- 172.21.15.31:0/3165018161 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71c013fef0 0x7f71c015a930 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:28.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.936+0000 7f71c6d9d700 1 -- 172.21.15.31:0/3165018161 >> 172.21.15.31:0/3165018161 conn(0x7f71c006d170 msgr2=0x7f71c00cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:28.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.936+0000 7f71c6d9d700 1 -- 172.21.15.31:0/3165018161 shutdown_connections 2023-10-06T02:29:28.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.936+0000 7f71c6d9d700 1 -- 172.21.15.31:0/3165018161 wait complete. 2023-10-06T02:29:28.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.937+0000 7f71c6d9d700 1 Processor -- start 2023-10-06T02:29:28.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.938+0000 7f71c6d9d700 1 -- start start 2023-10-06T02:29:28.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.939+0000 7f71c6d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71c006aaa0 0x7f71c0159240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:28.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.939+0000 7f71c6d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f71c013fb20 0x7f71c00ab940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:28.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.939+0000 7f71c6d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71c013fef0 0x7f71c00ad3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:28.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.939+0000 7f71c6d9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c01373f0 con 0x7f71c013fef0 2023-10-06T02:29:28.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.939+0000 7f71c6d9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c0157130 con 0x7f71c006aaa0 2023-10-06T02:29:28.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.939+0000 7f71c6d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71c013c6a0 con 0x7f71c013fb20 2023-10-06T02:29:28.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.939+0000 7f71c659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71c013fef0 0x7f71c00ad3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:28.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.939+0000 7f71c659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71c013fef0 0x7f71c00ad3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34234/0 (socket says 172.21.15.31:34234) 2023-10-06T02:29:28.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.939+0000 7f71c659c700 1 -- 172.21.15.31:0/2589546154 learned_addr learned my addr 172.21.15.31:0/2589546154 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:28.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.939+0000 7f71c5d9b700 1 --2- 172.21.15.31:0/2589546154 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71c006aaa0 0x7f71c0159240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:28.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.939+0000 7f71c559a700 1 --2- 172.21.15.31:0/2589546154 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f71c013fb20 0x7f71c00ab940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:28.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.940+0000 7f71c659c700 1 -- 172.21.15.31:0/2589546154 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f71c013fb20 msgr2=0x7f71c00ab940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:28.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.940+0000 7f71c659c700 1 --2- 172.21.15.31:0/2589546154 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f71c013fb20 0x7f71c00ab940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:28.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.940+0000 7f71c659c700 1 -- 172.21.15.31:0/2589546154 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71c006aaa0 msgr2=0x7f71c0159240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:28.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.940+0000 7f71c659c700 1 --2- 172.21.15.31:0/2589546154 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71c006aaa0 0x7f71c0159240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:28.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.940+0000 7f71c659c700 1 -- 172.21.15.31:0/2589546154 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71b00093c0 con 0x7f71c013fef0 2023-10-06T02:29:28.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.940+0000 7f71c659c700 1 --2- 172.21.15.31:0/2589546154 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71c013fef0 0x7f71c00ad3b0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f71b8009dd0 tx=0x7f71b800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:28.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.940+0000 7f71b5ffb700 1 -- 172.21.15.31:0/2589546154 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b800caa0 con 0x7f71c013fef0 2023-10-06T02:29:28.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.940+0000 7f71b5ffb700 1 -- 172.21.15.31:0/2589546154 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71b800f040 con 0x7f71c013fef0 2023-10-06T02:29:28.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.940+0000 7f71c6d9d700 1 -- 172.21.15.31:0/2589546154 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71c0159780 con 0x7f71c013fef0 2023-10-06T02:29:28.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.940+0000 7f71b5ffb700 1 -- 172.21.15.31:0/2589546154 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b800ce50 con 0x7f71c013fef0 2023-10-06T02:29:28.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.940+0000 7f71c6d9d700 1 -- 172.21.15.31:0/2589546154 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71c013c3c0 con 0x7f71c013fef0 2023-10-06T02:29:28.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.941+0000 7f71c6d9d700 1 -- 172.21.15.31:0/2589546154 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71c01572c0 con 0x7f71c013fef0 2023-10-06T02:29:28.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.942+0000 7f71b5ffb700 1 -- 172.21.15.31:0/2589546154 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f71b8014690 con 0x7f71c013fef0 2023-10-06T02:29:28.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.944+0000 7f71b5ffb700 1 --2- 172.21.15.31:0/2589546154 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f71a4067320 0x7f71a40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:28.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.944+0000 7f71b5ffb700 1 -- 172.21.15.31:0/2589546154 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f71b8085720 con 0x7f71c013fef0 2023-10-06T02:29:28.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.944+0000 7f71c5d9b700 1 --2- 172.21.15.31:0/2589546154 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f71a4067320 0x7f71a40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:28.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.945+0000 7f71c5d9b700 1 --2- 172.21.15.31:0/2589546154 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f71a4067320 0x7f71a40697d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f71bc007130 tx=0x7f71bc007eb0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:28.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:28.948+0000 7f71b5ffb700 1 -- 172.21.15.31:0/2589546154 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f71b8052400 con 0x7f71c013fef0 2023-10-06T02:29:29.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:29.200+0000 7f71c6d9d700 1 -- 172.21.15.31:0/2589546154 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f71c00648e0 con 0x7f71c013fef0 2023-10-06T02:29:29.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:29.200+0000 7f71b5ffb700 1 -- 172.21.15.31:0/2589546154 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+3526 (secure 0 0 0) 0x7f71b8018de0 con 0x7f71c013fef0 2023-10-06T02:29:29.202 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:29:29.202 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":74,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6026,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/3801334948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3801334948},{"type":"v1","addr":"172.21.15.62:6834","nonce":3801334948}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":6724,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/1804582836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":1804582836},{"type":"v1","addr":"172.21.15.31:6835","nonce":1804582836}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":6736,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1428432734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":1428432734},{"type":"v1","addr":"172.21.15.62:6835","nonce":1428432734}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":7512,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/166531985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":166531985},{"type":"v1","addr":"172.21.15.31:6834","nonce":166531985}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74}],"filesystems":[]} 2023-10-06T02:29:29.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:29.203+0000 7f71c6d9d700 1 -- 172.21.15.31:0/2589546154 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f71a4067320 msgr2=0x7f71a40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:29.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:29.203+0000 7f71c6d9d700 1 --2- 172.21.15.31:0/2589546154 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f71a4067320 0x7f71a40697d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f71bc007130 tx=0x7f71bc007eb0 comp rx=0 tx=0).stop 2023-10-06T02:29:29.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:29.203+0000 7f71c6d9d700 1 -- 172.21.15.31:0/2589546154 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71c013fef0 msgr2=0x7f71c00ad3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:29.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:29.203+0000 7f71c6d9d700 1 --2- 172.21.15.31:0/2589546154 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71c013fef0 0x7f71c00ad3b0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f71b8009dd0 tx=0x7f71b800b040 comp rx=0 tx=0).stop 2023-10-06T02:29:29.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:29.203+0000 7f71c6d9d700 1 -- 172.21.15.31:0/2589546154 shutdown_connections 2023-10-06T02:29:29.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:29.203+0000 7f71c6d9d700 1 --2- 172.21.15.31:0/2589546154 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71c013fef0 0x7f71c00ad3b0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:29.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:29.203+0000 7f71c6d9d700 1 --2- 172.21.15.31:0/2589546154 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71c006aaa0 0x7f71c0159240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:29.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:29.203+0000 7f71c6d9d700 1 --2- 172.21.15.31:0/2589546154 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f71a4067320 0x7f71a40697d0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:29.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:29.203+0000 7f71c6d9d700 1 --2- 172.21.15.31:0/2589546154 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f71c013fb20 0x7f71c00ab940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:29.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:29.203+0000 7f71c6d9d700 1 -- 172.21.15.31:0/2589546154 >> 172.21.15.31:0/2589546154 conn(0x7f71c006d170 msgr2=0x7f71c00c1580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:29.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:29.203+0000 7f71c6d9d700 1 -- 172.21.15.31:0/2589546154 shutdown_connections 2023-10-06T02:29:29.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:29.203+0000 7f71c6d9d700 1 -- 172.21.15.31:0/2589546154 wait complete. 2023-10-06T02:29:29.206 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 74 2023-10-06T02:29:29.222 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T02:29:29.222 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T02:29:29.223 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T02:29:29.223 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T02:29:29.223 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T02:29:29.223 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T02:29:29.223 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T02:29:29.223 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T02:29:29.223 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T02:29:29.224 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T02:29:29.224 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T02:29:29.224 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T02:29:29.224 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T02:29:29.224 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T02:29:29.224 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T02:29:29.224 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T02:29:29.236 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T02:29:29.236 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T02:29:29.237 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T02:29:29.237 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T02:29:29.241 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T02:29:29.241 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T02:29:29.242 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T02:29:29.242 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T02:29:29.281 INFO:tasks.ceph.mds.a:Started 2023-10-06T02:29:29.282 DEBUG:teuthology.parallel:result is None 2023-10-06T02:29:29.285 INFO:tasks.ceph.mds.c:Started 2023-10-06T02:29:29.285 DEBUG:teuthology.parallel:result is None 2023-10-06T02:29:29.286 INFO:tasks.ceph.mds.b:Started 2023-10-06T02:29:29.286 DEBUG:teuthology.parallel:result is None 2023-10-06T02:29:29.289 INFO:tasks.ceph.mds.d:Started 2023-10-06T02:29:29.290 DEBUG:teuthology.parallel:result is None 2023-10-06T02:29:29.290 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T02:29:29.418 INFO:tasks.ceph.mds.a.smithi031.stdout:starting mds.a at 2023-10-06T02:29:29.423 INFO:tasks.ceph.mds.c.smithi031.stdout:starting mds.c at 2023-10-06T02:29:29.427 INFO:tasks.ceph.mds.b.smithi062.stdout:starting mds.b at 2023-10-06T02:29:29.427 INFO:tasks.ceph.mds.d.smithi062.stdout:starting mds.d at 2023-10-06T02:29:29.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:29.486+0000 7f65edd9d700 1 Processor -- start 2023-10-06T02:29:29.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:29.486+0000 7f65edd9d700 1 -- start start 2023-10-06T02:29:29.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:29.487+0000 7f65edd9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f65e8153c30 0x7f65e815a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:29.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:29.487+0000 7f65edd9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f65e815de50 0x7f65e80cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:29.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:29.487+0000 7f65edd9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65e80cf3c0 con 0x7f65e80cef40 2023-10-06T02:29:29.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:29.487+0000 7f65edd9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65e815afd0 con 0x7f65e8153c30 2023-10-06T02:29:29.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:29.487+0000 7f65edd9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65e815b2b0 con 0x7f65e815de50 2023-10-06T02:29:29.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:29.488+0000 7f65ed59c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f65e80cef40 0x7f65e806d790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:36334/0 (socket says 172.21.15.31:36334) 2023-10-06T02:29:29.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:29.488+0000 7f65ed59c700 1 -- 172.21.15.31:0/2986147427 learned_addr learned my addr 172.21.15.31:0/2986147427 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:29:29.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:29.488+0000 7f65ecd9b700 1 --2- 172.21.15.31:0/2986147427 >> v2:172.21.15.31:3300/0 conn(0x7f65e8153c30 0x7f65e815a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:29.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:29.488+0000 7f65e7fff700 1 --2- 172.21.15.31:0/2986147427 >> v2:172.21.15.62:3300/0 conn(0x7f65e815de50 0x7f65e80cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:29.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:29.488+0000 7f65ecd9b700 1 -- 172.21.15.31:0/2986147427 >> v2:172.21.15.62:3300/0 conn(0x7f65e815de50 msgr2=0x7f65e80cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:29.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:29.488+0000 7f65e77fe700 1 -- 172.21.15.31:0/2986147427 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3114109420 0 0) 0x7f65e80cf3c0 con 0x7f65e80cef40 2023-10-06T02:29:29.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:29.488+0000 7f65ecd9b700 1 --2- 172.21.15.31:0/2986147427 >> v2:172.21.15.62:3300/0 conn(0x7f65e815de50 0x7f65e80cea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:29.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:29.488+0000 7f65ecd9b700 1 -- 172.21.15.31:0/2986147427 >> v1:172.21.15.31:6789/0 conn(0x7f65e80cef40 legacy=0x7f65e806d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:29.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:29.488+0000 7f65ecd9b700 1 -- 172.21.15.31:0/2986147427 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65e815b590 con 0x7f65e8153c30 2023-10-06T02:29:29.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:29.488+0000 7f65ecd9b700 1 --2- 172.21.15.31:0/2986147427 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65e8153c30 0x7f65e815a5e0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f65e80713a0 tx=0x7f65dc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e903b339bfcbfc45 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:29.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:29.489+0000 7f65e77fe700 1 -- 172.21.15.31:0/2986147427 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65dc011630 con 0x7f65e8153c30 2023-10-06T02:29:29.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:29.489+0000 7f65e77fe700 1 -- 172.21.15.31:0/2986147427 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65dc011790 con 0x7f65e8153c30 2023-10-06T02:29:29.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:29.489+0000 7f65e77fe700 1 -- 172.21.15.31:0/2986147427 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65dc011b30 con 0x7f65e8153c30 2023-10-06T02:29:29.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:29.489+0000 7f65edd9d700 1 -- 172.21.15.31:0/2986147427 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65e8153c30 msgr2=0x7f65e815a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:29.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:29.489+0000 7f65edd9d700 1 --2- 172.21.15.31:0/2986147427 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65e8153c30 0x7f65e815a5e0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f65e80713a0 tx=0x7f65dc0091a0 comp rx=0 tx=0).stop 2023-10-06T02:29:29.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:29.489+0000 7f65edd9d700 1 -- 172.21.15.31:0/2986147427 shutdown_connections 2023-10-06T02:29:29.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:29.489+0000 7f65edd9d700 1 --2- 172.21.15.31:0/2986147427 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65e8153c30 0x7f65e815a5e0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:29.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:29.489+0000 7f65edd9d700 1 --2- 172.21.15.31:0/2986147427 >> v2:172.21.15.62:3300/0 conn(0x7f65e815de50 0x7f65e80cea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:29.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:29.489+0000 7f65edd9d700 1 -- 172.21.15.31:0/2986147427 >> 172.21.15.31:0/2986147427 conn(0x7f65e80703a0 msgr2=0x7f65e80707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:29.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:29.489+0000 7f65edd9d700 1 -- 172.21.15.31:0/2986147427 shutdown_connections 2023-10-06T02:29:29.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:29.489+0000 7f65edd9d700 1 -- 172.21.15.31:0/2986147427 wait complete. 2023-10-06T02:29:29.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:29.491+0000 7f65edd9d700 1 Processor -- start 2023-10-06T02:29:29.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:29.491+0000 7f65edd9d700 1 -- start start 2023-10-06T02:29:29.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:29.492+0000 7f65edd9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65e80cef40 0x7f65e8168e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:29.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:29.492+0000 7f65edd9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f65e8153c30 0x7f65e80b3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:29.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:29.492+0000 7f65edd9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65e815de50 0x7f65e80c7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:29.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:29.492+0000 7f65edd9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65e80c9490 con 0x7f65e815de50 2023-10-06T02:29:29.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:29.492+0000 7f65edd9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65e80cb1b0 con 0x7f65e80cef40 2023-10-06T02:29:29.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:29.492+0000 7f65edd9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65e80cb490 con 0x7f65e8153c30 2023-10-06T02:29:29.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:29.492+0000 7f65ed59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65e815de50 0x7f65e80c7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:29.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:29.492+0000 7f65ed59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65e815de50 0x7f65e80c7f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34300/0 (socket says 172.21.15.31:34300) 2023-10-06T02:29:29.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:29.492+0000 7f65ed59c700 1 -- 172.21.15.31:0/834396017 learned_addr learned my addr 172.21.15.31:0/834396017 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:29.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:29.492+0000 7f65e7fff700 1 --2- 172.21.15.31:0/834396017 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f65e8153c30 0x7f65e80b3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:29.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:29.492+0000 7f65ecd9b700 1 --2- 172.21.15.31:0/834396017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65e80cef40 0x7f65e8168e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:29.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:29.492+0000 7f65ed59c700 1 -- 172.21.15.31:0/834396017 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f65e8153c30 msgr2=0x7f65e80b3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:29.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:29.492+0000 7f65ed59c700 1 --2- 172.21.15.31:0/834396017 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f65e8153c30 0x7f65e80b3e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:29.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:29.492+0000 7f65ed59c700 1 -- 172.21.15.31:0/834396017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65e80cef40 msgr2=0x7f65e8168e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:29.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:29.492+0000 7f65ed59c700 1 --2- 172.21.15.31:0/834396017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65e80cef40 0x7f65e8168e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:29.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:29.492+0000 7f65ed59c700 1 -- 172.21.15.31:0/834396017 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65dc009420 con 0x7f65e815de50 2023-10-06T02:29:29.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:29.493+0000 7f65ed59c700 1 --2- 172.21.15.31:0/834396017 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65e815de50 0x7f65e80c7f90 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f65e000cbf0 tx=0x7f65e00019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:29.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:29.493+0000 7f65e4ff9700 1 -- 172.21.15.31:0/834396017 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e000a8b0 con 0x7f65e815de50 2023-10-06T02:29:29.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:29.493+0000 7f65e4ff9700 1 -- 172.21.15.31:0/834396017 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65e000aa10 con 0x7f65e815de50 2023-10-06T02:29:29.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:29.493+0000 7f65edd9d700 1 -- 172.21.15.31:0/834396017 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65e80cb770 con 0x7f65e815de50 2023-10-06T02:29:29.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:29.493+0000 7f65e4ff9700 1 -- 172.21.15.31:0/834396017 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e000add0 con 0x7f65e815de50 2023-10-06T02:29:29.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:29.493+0000 7f65edd9d700 1 -- 172.21.15.31:0/834396017 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65e80c8be0 con 0x7f65e815de50 2023-10-06T02:29:29.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:29.494+0000 7f65edd9d700 1 -- 172.21.15.31:0/834396017 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65e80cbc90 con 0x7f65e815de50 2023-10-06T02:29:29.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:29.495+0000 7f65e4ff9700 1 -- 172.21.15.31:0/834396017 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f65e0007300 con 0x7f65e815de50 2023-10-06T02:29:29.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:29.496+0000 7f65e4ff9700 1 --2- 172.21.15.31:0/834396017 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f65cc067320 0x7f65cc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:29.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:29.496+0000 7f65e4ff9700 1 -- 172.21.15.31:0/834396017 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f65e0084d80 con 0x7f65e815de50 2023-10-06T02:29:29.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:29.496+0000 7f65ecd9b700 1 --2- 172.21.15.31:0/834396017 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f65cc067320 0x7f65cc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:29.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:29.497+0000 7f65ecd9b700 1 --2- 172.21.15.31:0/834396017 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f65cc067320 0x7f65cc0697d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f65dc0093f0 tx=0x7f65dc017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:29.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:29.500+0000 7f65e4ff9700 1 -- 172.21.15.31:0/834396017 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f65e0051bf0 con 0x7f65e815de50 2023-10-06T02:29:29.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:29.718+0000 7f65edd9d700 1 -- 172.21.15.31:0/834396017 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f65e804c3c0 con 0x7f65e815de50 2023-10-06T02:29:31.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.081+0000 7f65e4ff9700 1 -- 172.21.15.31:0/834396017 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v116) v1 ==== 97+0+0 (secure 0 0 0) 0x7f65e0055210 con 0x7f65e815de50 2023-10-06T02:29:31.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.085+0000 7f65edd9d700 1 -- 172.21.15.31:0/834396017 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f65cc067320 msgr2=0x7f65cc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:31.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.085+0000 7f65edd9d700 1 --2- 172.21.15.31:0/834396017 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f65cc067320 0x7f65cc0697d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f65dc0093f0 tx=0x7f65dc017000 comp rx=0 tx=0).stop 2023-10-06T02:29:31.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.085+0000 7f65edd9d700 1 -- 172.21.15.31:0/834396017 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65e815de50 msgr2=0x7f65e80c7f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:31.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.085+0000 7f65edd9d700 1 --2- 172.21.15.31:0/834396017 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65e815de50 0x7f65e80c7f90 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f65e000cbf0 tx=0x7f65e00019b0 comp rx=0 tx=0).stop 2023-10-06T02:29:31.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.085+0000 7f65edd9d700 1 -- 172.21.15.31:0/834396017 shutdown_connections 2023-10-06T02:29:31.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.085+0000 7f65edd9d700 1 --2- 172.21.15.31:0/834396017 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65e815de50 0x7f65e80c7f90 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:31.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.085+0000 7f65edd9d700 1 --2- 172.21.15.31:0/834396017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65e80cef40 0x7f65e8168e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:31.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.085+0000 7f65edd9d700 1 --2- 172.21.15.31:0/834396017 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f65cc067320 0x7f65cc0697d0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:31.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.085+0000 7f65edd9d700 1 --2- 172.21.15.31:0/834396017 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f65e8153c30 0x7f65e80b3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:31.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.085+0000 7f65edd9d700 1 -- 172.21.15.31:0/834396017 >> 172.21.15.31:0/834396017 conn(0x7f65e80703a0 msgr2=0x7f65e8070770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:31.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.085+0000 7f65edd9d700 1 -- 172.21.15.31:0/834396017 shutdown_connections 2023-10-06T02:29:31.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.085+0000 7f65edd9d700 1 -- 172.21.15.31:0/834396017 wait complete. 2023-10-06T02:29:31.088 INFO:teuthology.orchestra.run.smithi031.stderr: removed all blocklist entries 2023-10-06T02:29:31.107 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T02:29:31.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.299+0000 7fb31eb9d700 1 Processor -- start 2023-10-06T02:29:31.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.299+0000 7fb31eb9d700 1 -- start start 2023-10-06T02:29:31.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.301+0000 7fb31eb9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb31813fb20 0x7fb31815a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:31.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.301+0000 7fb31eb9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb31813fef0 0x7fb3180c7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:31.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.301+0000 7fb31eb9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb31815ae70 con 0x7fb31806aaa0 2023-10-06T02:29:31.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.301+0000 7fb31eb9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb31806af60 con 0x7fb31813fb20 2023-10-06T02:29:31.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.301+0000 7fb31eb9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb31806b240 con 0x7fb31813fef0 2023-10-06T02:29:31.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.301+0000 7fb31e39c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fb31806aaa0 0x7fb3180c6770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:36340/0 (socket says 172.21.15.31:36340) 2023-10-06T02:29:31.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.301+0000 7fb31e39c700 1 -- 172.21.15.31:0/298315323 learned_addr learned my addr 172.21.15.31:0/298315323 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:29:31.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.301+0000 7fb31d39a700 1 --2- 172.21.15.31:0/298315323 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb31813fef0 0x7fb3180c7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:31.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.301+0000 7fb31db9b700 1 --2- 172.21.15.31:0/298315323 >> v2:172.21.15.62:3300/0 conn(0x7fb31813fb20 0x7fb31815a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:31.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.301+0000 7fb31cb99700 1 -- 172.21.15.31:0/298315323 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1453198365 0 0) 0x7fb310001490 con 0x7fb31806aaa0 2023-10-06T02:29:31.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.301+0000 7fb31cb99700 1 -- 172.21.15.31:0/298315323 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb304003570 con 0x7fb31806aaa0 2023-10-06T02:29:31.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.302+0000 7fb31cb99700 1 -- 172.21.15.31:0/298315323 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3525508120 0 0) 0x7fb304003570 con 0x7fb31806aaa0 2023-10-06T02:29:31.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.302+0000 7fb31cb99700 1 -- 172.21.15.31:0/298315323 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb31813fef0 msgr2=0x7fb3180c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:31.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.302+0000 7fb31cb99700 1 --2- 172.21.15.31:0/298315323 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb31813fef0 0x7fb3180c7990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:31.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.302+0000 7fb31cb99700 1 -- 172.21.15.31:0/298315323 >> v2:172.21.15.62:3300/0 conn(0x7fb31813fb20 msgr2=0x7fb31815a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:31.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.302+0000 7fb31cb99700 1 --2- 172.21.15.31:0/298315323 >> v2:172.21.15.62:3300/0 conn(0x7fb31813fb20 0x7fb31815a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:31.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.302+0000 7fb31cb99700 1 -- 172.21.15.31:0/298315323 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb31806b520 con 0x7fb31806aaa0 2023-10-06T02:29:31.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.302+0000 7fb31cb99700 1 -- 172.21.15.31:0/298315323 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7fb310001980 con 0x7fb31806aaa0 2023-10-06T02:29:31.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.302+0000 7fb31cb99700 1 -- 172.21.15.31:0/298315323 >> v1:172.21.15.31:6789/0 conn(0x7fb31806aaa0 legacy=0x7fb3180c6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:31.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.302+0000 7fb31cb99700 1 --2- 172.21.15.31:0/298315323 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb304003ae0 0x7fb304003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:31.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.302+0000 7fb31cb99700 1 --2- 172.21.15.31:0/298315323 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb304004810 0x7fb304004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:31.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.302+0000 7fb31cb99700 1 --2- 172.21.15.31:0/298315323 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb31813fef0 0x7fb3040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:31.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.302+0000 7fb31cb99700 1 -- 172.21.15.31:0/298315323 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb31806af60 con 0x7fb304004810 2023-10-06T02:29:31.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.302+0000 7fb31cb99700 1 -- 172.21.15.31:0/298315323 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb31806b240 con 0x7fb304003ae0 2023-10-06T02:29:31.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.302+0000 7fb31cb99700 1 -- 172.21.15.31:0/298315323 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb30400dd50 con 0x7fb31813fef0 2023-10-06T02:29:31.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.302+0000 7fb31d39a700 1 --2- 172.21.15.31:0/298315323 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb304004810 0x7fb304004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:31.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.302+0000 7fb31db9b700 1 --2- 172.21.15.31:0/298315323 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb304003ae0 0x7fb304003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:31.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.302+0000 7fb31e39c700 1 --2- 172.21.15.31:0/298315323 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb31813fef0 0x7fb3040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:31.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.303+0000 7fb31d39a700 1 -- 172.21.15.31:0/298315323 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb31813fef0 msgr2=0x7fb3040051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:31.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.303+0000 7fb31d39a700 1 --2- 172.21.15.31:0/298315323 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb31813fef0 0x7fb3040051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:31.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.303+0000 7fb31d39a700 1 -- 172.21.15.31:0/298315323 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb304003ae0 msgr2=0x7fb304003ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:31.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.303+0000 7fb31d39a700 1 --2- 172.21.15.31:0/298315323 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb304003ae0 0x7fb304003ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:31.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.303+0000 7fb31d39a700 1 -- 172.21.15.31:0/298315323 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb304003570 con 0x7fb304004810 2023-10-06T02:29:31.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.303+0000 7fb31e39c700 1 --2- 172.21.15.31:0/298315323 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb31813fef0 0x7fb3040051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:29:31.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.303+0000 7fb31d39a700 1 --2- 172.21.15.31:0/298315323 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb304004810 0x7fb304004c80 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fb318069580 tx=0x7fb308009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7a227160ee35e81 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:31.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.303+0000 7fb31cb99700 1 -- 172.21.15.31:0/298315323 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb308005a00 con 0x7fb304004810 2023-10-06T02:29:31.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.303+0000 7fb31cb99700 1 -- 172.21.15.31:0/298315323 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb308005b60 con 0x7fb304004810 2023-10-06T02:29:31.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.303+0000 7fb31cb99700 1 -- 172.21.15.31:0/298315323 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb30800e640 con 0x7fb304004810 2023-10-06T02:29:31.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.304+0000 7fb31eb9d700 1 -- 172.21.15.31:0/298315323 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb304004810 msgr2=0x7fb304004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:31.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.304+0000 7fb31eb9d700 1 --2- 172.21.15.31:0/298315323 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb304004810 0x7fb304004c80 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fb318069580 tx=0x7fb308009120 comp rx=0 tx=0).stop 2023-10-06T02:29:31.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.304+0000 7fb31e39c700 1 -- 172.21.15.31:0/298315323 reap_dead start 2023-10-06T02:29:31.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.304+0000 7fb31eb9d700 1 -- 172.21.15.31:0/298315323 shutdown_connections 2023-10-06T02:29:31.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.304+0000 7fb31eb9d700 1 -- 172.21.15.31:0/298315323 >> 172.21.15.31:0/298315323 conn(0x7fb31806d170 msgr2=0x7fb3180cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:31.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.304+0000 7fb31eb9d700 1 -- 172.21.15.31:0/298315323 shutdown_connections 2023-10-06T02:29:31.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.304+0000 7fb31eb9d700 1 -- 172.21.15.31:0/298315323 wait complete. 2023-10-06T02:29:31.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.305+0000 7fb31eb9d700 1 Processor -- start 2023-10-06T02:29:31.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.305+0000 7fb31eb9d700 1 -- start start 2023-10-06T02:29:31.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.306+0000 7fb31eb9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb304004810 0x7fb318157e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:31.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.306+0000 7fb31eb9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb318156920 0x7fb31813e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:31.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.306+0000 7fb31eb9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb318155b10 0x7fb3180addc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:31.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.306+0000 7fb31eb9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3181583c0 con 0x7fb318156920 2023-10-06T02:29:31.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.306+0000 7fb31eb9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb318156de0 con 0x7fb318155b10 2023-10-06T02:29:31.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.306+0000 7fb31eb9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3181570c0 con 0x7fb304004810 2023-10-06T02:29:31.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.306+0000 7fb31d39a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb318156920 0x7fb31813e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:31.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.306+0000 7fb31d39a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb318156920 0x7fb31813e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34312/0 (socket says 172.21.15.31:34312) 2023-10-06T02:29:31.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.306+0000 7fb31d39a700 1 -- 172.21.15.31:0/3406885874 learned_addr learned my addr 172.21.15.31:0/3406885874 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:31.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.307+0000 7fb31db9b700 1 --2- 172.21.15.31:0/3406885874 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb304004810 0x7fb318157e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:31.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.307+0000 7fb31e39c700 1 --2- 172.21.15.31:0/3406885874 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb318155b10 0x7fb3180addc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:31.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.307+0000 7fb31d39a700 1 -- 172.21.15.31:0/3406885874 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb304004810 msgr2=0x7fb318157e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:31.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.307+0000 7fb31d39a700 1 --2- 172.21.15.31:0/3406885874 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb304004810 0x7fb318157e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:31.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.307+0000 7fb31d39a700 1 -- 172.21.15.31:0/3406885874 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb318155b10 msgr2=0x7fb3180addc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:31.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.307+0000 7fb31d39a700 1 --2- 172.21.15.31:0/3406885874 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb318155b10 0x7fb3180addc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:31.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.307+0000 7fb31d39a700 1 -- 172.21.15.31:0/3406885874 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3181573a0 con 0x7fb318156920 2023-10-06T02:29:31.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.307+0000 7fb31d39a700 1 --2- 172.21.15.31:0/3406885874 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb318156920 0x7fb31813e0f0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fb30800e4b0 tx=0x7fb308019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:31.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.307+0000 7fb30dffb700 1 -- 172.21.15.31:0/3406885874 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb308005a00 con 0x7fb318156920 2023-10-06T02:29:31.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.308+0000 7fb30dffb700 1 -- 172.21.15.31:0/3406885874 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb308006240 con 0x7fb318156920 2023-10-06T02:29:31.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.308+0000 7fb30dffb700 1 -- 172.21.15.31:0/3406885874 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb308006620 con 0x7fb318156920 2023-10-06T02:29:31.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.308+0000 7fb31eb9d700 1 -- 172.21.15.31:0/3406885874 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb318157680 con 0x7fb318156920 2023-10-06T02:29:31.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.308+0000 7fb31eb9d700 1 -- 172.21.15.31:0/3406885874 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb31813e630 con 0x7fb318156920 2023-10-06T02:29:31.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.309+0000 7fb31eb9d700 1 -- 172.21.15.31:0/3406885874 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb318157b20 con 0x7fb318156920 2023-10-06T02:29:31.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.310+0000 7fb30dffb700 1 -- 172.21.15.31:0/3406885874 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fb308013470 con 0x7fb318156920 2023-10-06T02:29:31.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.311+0000 7fb30dffb700 1 --2- 172.21.15.31:0/3406885874 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb2fc067320 0x7fb2fc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:31.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.311+0000 7fb30dffb700 1 -- 172.21.15.31:0/3406885874 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fb308085eb0 con 0x7fb318156920 2023-10-06T02:29:31.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.311+0000 7fb31db9b700 1 --2- 172.21.15.31:0/3406885874 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb2fc067320 0x7fb2fc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:31.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.312+0000 7fb31db9b700 1 --2- 172.21.15.31:0/3406885874 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb2fc067320 0x7fb2fc0697d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb314006f00 tx=0x7fb3140091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:31.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.315+0000 7fb30dffb700 1 -- 172.21.15.31:0/3406885874 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb308052f00 con 0x7fb318156920 2023-10-06T02:29:31.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.593+0000 7fb31eb9d700 1 -- 172.21.15.31:0/3406885874 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fb31804c3c0 con 0x7fb318156920 2023-10-06T02:29:31.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.594+0000 7fb30dffb700 1 -- 172.21.15.31:0/3406885874 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fb308056520 con 0x7fb318156920 2023-10-06T02:29:31.595 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:29:31.596 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T02:29:31.596 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_dump": [ 2023-10-06T02:29:31.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:29:31.596 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.a", 2023-10-06T02:29:31.596 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBLbx9lbjrOGxAAVoSOoDaZb1temL41F4Nong==", 2023-10-06T02:29:31.596 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:29:31.596 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:29:31.596 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T02:29:31.596 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T02:29:31.596 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:29:31.596 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:29:31.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:29:31.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:29:31.597 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.b", 2023-10-06T02:29:31.597 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBLbx9lFgsTMRAATTklzroiyQbTLiZc9j1BHQ==", 2023-10-06T02:29:31.597 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:29:31.597 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:29:31.597 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T02:29:31.597 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T02:29:31.597 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:29:31.597 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:29:31.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:29:31.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:29:31.598 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.c", 2023-10-06T02:29:31.598 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBLbx9lezVtJhAA8wK7+dJ1OmIc+iXJkexK0w==", 2023-10-06T02:29:31.598 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:29:31.598 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:29:31.598 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T02:29:31.598 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T02:29:31.598 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:29:31.598 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:29:31.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:29:31.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:29:31.599 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.d", 2023-10-06T02:29:31.599 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBMbx9l74gWABAApBu2nVv12rGA+OBfBnEc3A==", 2023-10-06T02:29:31.599 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:29:31.599 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:29:31.599 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T02:29:31.599 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T02:29:31.599 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:29:31.599 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:29:31.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:29:31.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:29:31.599 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.0", 2023-10-06T02:29:31.599 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBPbx9lgj6+MBAAXSUD0GCG+rO/9DQBZYt/8w==", 2023-10-06T02:29:31.600 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:29:31.600 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:29:31.600 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:29:31.600 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:29:31.600 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:29:31.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:29:31.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:29:31.600 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.1", 2023-10-06T02:29:31.600 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBSbx9lXhMTNBAAeEdPnMP71ZGb6k2Ddg5raQ==", 2023-10-06T02:29:31.601 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:29:31.601 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:29:31.601 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:29:31.601 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:29:31.601 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:29:31.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:29:31.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:29:31.601 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.2", 2023-10-06T02:29:31.601 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBVbx9l4Ly0NhAATYXYDAjYuP53zjqK0yMboA==", 2023-10-06T02:29:31.601 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:29:31.601 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:29:31.601 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:29:31.602 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:29:31.602 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:29:31.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:29:31.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:29:31.602 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.3", 2023-10-06T02:29:31.602 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBZbx9lzsi4ARAAbGnNV0QyL/7itzy827+chg==", 2023-10-06T02:29:31.602 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:29:31.602 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:29:31.602 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:29:31.602 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:29:31.602 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:29:31.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:29:31.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:29:31.603 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.4", 2023-10-06T02:29:31.603 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBfbx9lW4OSEhAAj7HnVh/yq12zzCaXd6BHCA==", 2023-10-06T02:29:31.603 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:29:31.603 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:29:31.603 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:29:31.603 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:29:31.603 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:29:31.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:29:31.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:29:31.603 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.5", 2023-10-06T02:29:31.604 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBibx9lBcaIFRAAV8wq8QjsCOkI9Pzf/E+9bA==", 2023-10-06T02:29:31.604 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:29:31.604 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:29:31.604 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:29:31.604 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:29:31.604 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:29:31.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:29:31.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:29:31.604 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.6", 2023-10-06T02:29:31.604 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBlbx9lA7S+FxAA38mlSAurT+DpwMzH53zFcw==", 2023-10-06T02:29:31.604 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:29:31.604 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:29:31.605 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:29:31.605 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:29:31.605 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:29:31.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:29:31.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:29:31.605 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.7", 2023-10-06T02:29:31.605 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBobx9l5RkdHhAAxn0Z8IAE/o9oe2LRp3O9gg==", 2023-10-06T02:29:31.605 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:29:31.605 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:29:31.605 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:29:31.605 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:29:31.606 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:29:31.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:29:31.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:29:31.606 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.0", 2023-10-06T02:29:31.606 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBMbx9lh44fCRAA7sMISvk1t7dEJg4ZcBiMcA==", 2023-10-06T02:29:31.606 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:29:31.606 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:29:31.606 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow r", 2023-10-06T02:29:31.606 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T02:29:31.606 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:29:31.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:29:31.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:29:31.607 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.1", 2023-10-06T02:29:31.607 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBMbx9lP0zqFhAAlov2y7ipSvS3AkOdQkhg6g==", 2023-10-06T02:29:31.607 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:29:31.607 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:29:31.607 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-10-06T02:29:31.607 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-10-06T02:29:31.607 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-10-06T02:29:31.607 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:29:31.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:29:31.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:29:31.608 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.2", 2023-10-06T02:29:31.608 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBMbx9lpxYAEBAACtiI1XgQXM8yGGcpFyNK1w==", 2023-10-06T02:29:31.608 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:29:31.608 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:29:31.608 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-10-06T02:29:31.608 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-10-06T02:29:31.608 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-10-06T02:29:31.608 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:29:31.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:29:31.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:29:31.608 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.3", 2023-10-06T02:29:31.608 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBMbx9l81/PHRAAYFOoaezu0pwE0NdtnlYfVA==", 2023-10-06T02:29:31.609 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:29:31.609 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:29:31.609 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-10-06T02:29:31.609 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-10-06T02:29:31.609 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-10-06T02:29:31.609 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:29:31.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:29:31.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:29:31.609 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.admin", 2023-10-06T02:29:31.609 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBKbx9liFl1JBAAFyyHYNLVx1/Ex5a2MKm8rA==", 2023-10-06T02:29:31.609 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:29:31.609 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-10-06T02:29:31.609 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T02:29:31.610 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T02:29:31.610 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:29:31.610 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:29:31.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:29:31.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:29:31.610 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mgr.x", 2023-10-06T02:29:31.610 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBLbx9lTBbIFBAAX2wsioKL4Nx4qz/qwsCsNA==", 2023-10-06T02:29:31.610 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:29:31.610 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-10-06T02:29:31.610 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile mgr", 2023-10-06T02:29:31.610 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:29:31.610 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:29:31.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:29:31.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:29:31.611 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mgr.y", 2023-10-06T02:29:31.611 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBLbx9l4+oLDRAA5YVo3q6FI9e9W/rJYROIsg==", 2023-10-06T02:29:31.611 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:29:31.611 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-10-06T02:29:31.611 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile mgr", 2023-10-06T02:29:31.611 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:29:31.611 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:29:31.611 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:29:31.611 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-10-06T02:29:31.611 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T02:29:31.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.596+0000 7fb31eb9d700 1 -- 172.21.15.31:0/3406885874 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb2fc067320 msgr2=0x7fb2fc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:31.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.596+0000 7fb31eb9d700 1 --2- 172.21.15.31:0/3406885874 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb2fc067320 0x7fb2fc0697d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb314006f00 tx=0x7fb3140091a0 comp rx=0 tx=0).stop 2023-10-06T02:29:31.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.596+0000 7fb31eb9d700 1 -- 172.21.15.31:0/3406885874 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb318156920 msgr2=0x7fb31813e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:31.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.596+0000 7fb31eb9d700 1 --2- 172.21.15.31:0/3406885874 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb318156920 0x7fb31813e0f0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fb30800e4b0 tx=0x7fb308019000 comp rx=0 tx=0).stop 2023-10-06T02:29:31.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.596+0000 7fb31eb9d700 1 -- 172.21.15.31:0/3406885874 shutdown_connections 2023-10-06T02:29:31.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.596+0000 7fb31eb9d700 1 --2- 172.21.15.31:0/3406885874 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb318155b10 0x7fb3180addc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:31.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.596+0000 7fb31eb9d700 1 --2- 172.21.15.31:0/3406885874 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb2fc067320 0x7fb2fc0697d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:31.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.596+0000 7fb31eb9d700 1 --2- 172.21.15.31:0/3406885874 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb304004810 0x7fb318157e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:31.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.596+0000 7fb31eb9d700 1 --2- 172.21.15.31:0/3406885874 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb318156920 0x7fb31813e0f0 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:31.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.596+0000 7fb31eb9d700 1 -- 172.21.15.31:0/3406885874 >> 172.21.15.31:0/3406885874 conn(0x7fb31806d170 msgr2=0x7fb3180b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:31.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.596+0000 7fb31eb9d700 1 -- 172.21.15.31:0/3406885874 shutdown_connections 2023-10-06T02:29:31.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.596+0000 7fb31eb9d700 1 -- 172.21.15.31:0/3406885874 wait complete. 2023-10-06T02:29:31.615 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:29:31.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.800+0000 7fc71999d700 1 Processor -- start 2023-10-06T02:29:31.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.800+0000 7fc71999d700 1 -- start start 2023-10-06T02:29:31.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.801+0000 7fc71999d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc714159550 0x7fc7140ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:31.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.801+0000 7fc71999d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc71415dfd0 0x7fc7140aedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:31.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.801+0000 7fc71999d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc714159920 0x7fc7140ad090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:31.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.802+0000 7fc71999d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7140acc00 con 0x7fc71415dfd0 2023-10-06T02:29:31.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.802+0000 7fc71999d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7140abff0 con 0x7fc714159550 2023-10-06T02:29:31.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.802+0000 7fc71999d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7140ac2d0 con 0x7fc714159920 2023-10-06T02:29:31.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.802+0000 7fc713fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc71415dfd0 0x7fc7140aedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:31.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.802+0000 7fc713fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc71415dfd0 0x7fc7140aedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34326/0 (socket says 172.21.15.31:34326) 2023-10-06T02:29:31.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.802+0000 7fc713fff700 1 -- 172.21.15.31:0/764460544 learned_addr learned my addr 172.21.15.31:0/764460544 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:31.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.802+0000 7fc71899b700 1 --2- 172.21.15.31:0/764460544 >> v2:172.21.15.62:3300/0 conn(0x7fc714159550 0x7fc7140ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:31.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.802+0000 7fc71919c700 1 --2- 172.21.15.31:0/764460544 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc714159920 0x7fc7140ad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:31.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.802+0000 7fc713fff700 1 -- 172.21.15.31:0/764460544 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc714159920 msgr2=0x7fc7140ad090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:31.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.802+0000 7fc713fff700 1 --2- 172.21.15.31:0/764460544 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc714159920 0x7fc7140ad090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:31.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.802+0000 7fc713fff700 1 -- 172.21.15.31:0/764460544 >> v2:172.21.15.62:3300/0 conn(0x7fc714159550 msgr2=0x7fc7140ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:31.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.802+0000 7fc713fff700 1 --2- 172.21.15.31:0/764460544 >> v2:172.21.15.62:3300/0 conn(0x7fc714159550 0x7fc7140ceac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:31.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.802+0000 7fc713fff700 1 -- 172.21.15.31:0/764460544 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7140ac5b0 con 0x7fc71415dfd0 2023-10-06T02:29:31.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.802+0000 7fc71899b700 1 --2- 172.21.15.31:0/764460544 >> v2:172.21.15.62:3300/0 conn(0x7fc714159550 0x7fc7140ceac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:29:31.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.803+0000 7fc713fff700 1 --2- 172.21.15.31:0/764460544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc71415dfd0 0x7fc7140aedb0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fc714064270 tx=0x7fc700009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eabbafdde5433316 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:31.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.803+0000 7fc7137fe700 1 -- 172.21.15.31:0/764460544 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc70000b860 con 0x7fc71415dfd0 2023-10-06T02:29:31.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.803+0000 7fc7137fe700 1 -- 172.21.15.31:0/764460544 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc70000b9c0 con 0x7fc71415dfd0 2023-10-06T02:29:31.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.803+0000 7fc7137fe700 1 -- 172.21.15.31:0/764460544 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc70000bd60 con 0x7fc71415dfd0 2023-10-06T02:29:31.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.803+0000 7fc71999d700 1 -- 172.21.15.31:0/764460544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc71415dfd0 msgr2=0x7fc7140aedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:31.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.803+0000 7fc71999d700 1 --2- 172.21.15.31:0/764460544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc71415dfd0 0x7fc7140aedb0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fc714064270 tx=0x7fc700009250 comp rx=0 tx=0).stop 2023-10-06T02:29:31.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.804+0000 7fc71999d700 1 -- 172.21.15.31:0/764460544 shutdown_connections 2023-10-06T02:29:31.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.804+0000 7fc71999d700 1 --2- 172.21.15.31:0/764460544 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc714159920 0x7fc7140ad090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:31.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.804+0000 7fc71999d700 1 --2- 172.21.15.31:0/764460544 >> v2:172.21.15.62:3300/0 conn(0x7fc714159550 0x7fc7140ceac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:31.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.804+0000 7fc71999d700 1 --2- 172.21.15.31:0/764460544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc71415dfd0 0x7fc7140aedb0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:31.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.804+0000 7fc71999d700 1 -- 172.21.15.31:0/764460544 >> 172.21.15.31:0/764460544 conn(0x7fc71406ce80 msgr2=0x7fc7140ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:31.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.804+0000 7fc71999d700 1 -- 172.21.15.31:0/764460544 shutdown_connections 2023-10-06T02:29:31.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.804+0000 7fc71999d700 1 -- 172.21.15.31:0/764460544 wait complete. 2023-10-06T02:29:31.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.805+0000 7fc71999d700 1 Processor -- start 2023-10-06T02:29:31.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.805+0000 7fc71999d700 1 -- start start 2023-10-06T02:29:31.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.806+0000 7fc71999d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc714159550 0x7fc7140c9f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:31.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.806+0000 7fc71999d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc714159920 0x7fc71416a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:31.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.806+0000 7fc71999d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc71415dfd0 0x7fc71414e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:31.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.806+0000 7fc71999d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc71416acf0 con 0x7fc714159550 2023-10-06T02:29:31.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.806+0000 7fc71999d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc71414db30 con 0x7fc71415dfd0 2023-10-06T02:29:31.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.806+0000 7fc71999d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc71416d880 con 0x7fc714159920 2023-10-06T02:29:31.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.806+0000 7fc71899b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc714159550 0x7fc7140c9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:31.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.806+0000 7fc71899b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc714159550 0x7fc7140c9f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34338/0 (socket says 172.21.15.31:34338) 2023-10-06T02:29:31.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.806+0000 7fc71899b700 1 -- 172.21.15.31:0/2301994161 learned_addr learned my addr 172.21.15.31:0/2301994161 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:31.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.807+0000 7fc71919c700 1 --2- 172.21.15.31:0/2301994161 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc71415dfd0 0x7fc71414e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:31.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.807+0000 7fc713fff700 1 --2- 172.21.15.31:0/2301994161 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc714159920 0x7fc71416a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:31.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.807+0000 7fc71899b700 1 -- 172.21.15.31:0/2301994161 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc714159920 msgr2=0x7fc71416a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:31.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.807+0000 7fc71899b700 1 --2- 172.21.15.31:0/2301994161 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc714159920 0x7fc71416a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:31.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.807+0000 7fc71899b700 1 -- 172.21.15.31:0/2301994161 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc71415dfd0 msgr2=0x7fc71414e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:31.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.807+0000 7fc71899b700 1 --2- 172.21.15.31:0/2301994161 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc71415dfd0 0x7fc71414e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:31.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.807+0000 7fc71899b700 1 -- 172.21.15.31:0/2301994161 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7000094f0 con 0x7fc714159550 2023-10-06T02:29:31.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.807+0000 7fc71919c700 1 --2- 172.21.15.31:0/2301994161 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc71415dfd0 0x7fc71414e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:29:31.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.807+0000 7fc71899b700 1 --2- 172.21.15.31:0/2301994161 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc714159550 0x7fc7140c9f60 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fc70800d860 tx=0x7fc708008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:31.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.807+0000 7fc710ff9700 1 -- 172.21.15.31:0/2301994161 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7080125f0 con 0x7fc714159550 2023-10-06T02:29:31.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.807+0000 7fc710ff9700 1 -- 172.21.15.31:0/2301994161 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc708012750 con 0x7fc714159550 2023-10-06T02:29:31.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.807+0000 7fc710ff9700 1 -- 172.21.15.31:0/2301994161 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc708012ad0 con 0x7fc714159550 2023-10-06T02:29:31.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.807+0000 7fc71999d700 1 -- 172.21.15.31:0/2301994161 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc71416db60 con 0x7fc714159550 2023-10-06T02:29:31.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.807+0000 7fc71999d700 1 -- 172.21.15.31:0/2301994161 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7140cb390 con 0x7fc714159550 2023-10-06T02:29:31.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.808+0000 7fc71999d700 1 -- 172.21.15.31:0/2301994161 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7140cb700 con 0x7fc714159550 2023-10-06T02:29:31.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.809+0000 7fc710ff9700 1 -- 172.21.15.31:0/2301994161 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fc708014430 con 0x7fc714159550 2023-10-06T02:29:31.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.810+0000 7fc710ff9700 1 --2- 172.21.15.31:0/2301994161 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc6f8067320 0x7fc6f80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:31.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.810+0000 7fc710ff9700 1 -- 172.21.15.31:0/2301994161 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fc708084f10 con 0x7fc714159550 2023-10-06T02:29:31.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.811+0000 7fc713fff700 1 --2- 172.21.15.31:0/2301994161 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc6f8067320 0x7fc6f80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:31.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.811+0000 7fc713fff700 1 --2- 172.21.15.31:0/2301994161 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc6f8067320 0x7fc6f80697d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc714064270 tx=0x7fc700017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:31.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:31.814+0000 7fc710ff9700 1 -- 172.21.15.31:0/2301994161 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc7080557e0 con 0x7fc714159550 2023-10-06T02:29:31.857 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T02:29:31.862 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T02:29:32.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.019+0000 7fc71999d700 1 -- 172.21.15.31:0/2301994161 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc71404c3c0 con 0x7fc714159550 2023-10-06T02:29:32.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.020+0000 7fc710ff9700 1 -- 172.21.15.31:0/2301994161 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) v1 ==== 74+0+13612 (secure 0 0 0) 0x7fc708055370 con 0x7fc714159550 2023-10-06T02:29:32.021 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:29:32.022 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":116,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:29:31.077927+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:29:32.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.022+0000 7fc71999d700 1 -- 172.21.15.31:0/2301994161 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc6f8067320 msgr2=0x7fc6f80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:32.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.022+0000 7fc71999d700 1 --2- 172.21.15.31:0/2301994161 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc6f8067320 0x7fc6f80697d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc714064270 tx=0x7fc700017000 comp rx=0 tx=0).stop 2023-10-06T02:29:32.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.022+0000 7fc71999d700 1 -- 172.21.15.31:0/2301994161 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc714159550 msgr2=0x7fc7140c9f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:32.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.022+0000 7fc71999d700 1 --2- 172.21.15.31:0/2301994161 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc714159550 0x7fc7140c9f60 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fc70800d860 tx=0x7fc708008040 comp rx=0 tx=0).stop 2023-10-06T02:29:32.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.022+0000 7fc71999d700 1 -- 172.21.15.31:0/2301994161 shutdown_connections 2023-10-06T02:29:32.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.022+0000 7fc71999d700 1 --2- 172.21.15.31:0/2301994161 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc71415dfd0 0x7fc71414e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:32.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.022+0000 7fc71999d700 1 --2- 172.21.15.31:0/2301994161 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc714159550 0x7fc7140c9f60 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:32.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.022+0000 7fc71999d700 1 --2- 172.21.15.31:0/2301994161 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc6f8067320 0x7fc6f80697d0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:32.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.022+0000 7fc71999d700 1 --2- 172.21.15.31:0/2301994161 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc714159920 0x7fc71416a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:32.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.022+0000 7fc71999d700 1 -- 172.21.15.31:0/2301994161 >> 172.21.15.31:0/2301994161 conn(0x7fc71406ce80 msgr2=0x7fc71406ef70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:32.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.022+0000 7fc71999d700 1 -- 172.21.15.31:0/2301994161 shutdown_connections 2023-10-06T02:29:32.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.022+0000 7fc71999d700 1 -- 172.21.15.31:0/2301994161 wait complete. 2023-10-06T02:29:32.042 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T02:22:53.183425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T02:23:00.046967+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-10-06T02:29:32.042 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T02:29:32.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.231+0000 7fa5b7fff700 1 Processor -- start 2023-10-06T02:29:32.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.231+0000 7fa5b7fff700 1 -- start start 2023-10-06T02:29:32.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.233+0000 7fa5b7fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa5b8153a00 0x7fa5b8153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:32.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.233+0000 7fa5b7fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa5b80cea00 0x7fa5b80cee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:32.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.233+0000 7fa5b7fff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5b815ac50 con 0x7fa5b815de50 2023-10-06T02:29:32.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.233+0000 7fa5b7fff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5b815d010 con 0x7fa5b80cea00 2023-10-06T02:29:32.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.233+0000 7fa5b7fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b815af60 con 0x7fa5b8153a00 2023-10-06T02:29:32.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.233+0000 7fa5b67fc700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fa5b815de50 0x7fa5b815a540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:36344/0 (socket says 172.21.15.31:36344) 2023-10-06T02:29:32.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.233+0000 7fa5b67fc700 1 -- 172.21.15.31:0/1491379704 learned_addr learned my addr 172.21.15.31:0/1491379704 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:29:32.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.233+0000 7fa5b77fe700 1 --2- 172.21.15.31:0/1491379704 >> v2:172.21.15.31:3300/0 conn(0x7fa5b80cea00 0x7fa5b80cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:32.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.233+0000 7fa5b6ffd700 1 --2- 172.21.15.31:0/1491379704 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa5b8153a00 0x7fa5b8153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:32.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.233+0000 7fa5b5ffb700 1 -- 172.21.15.31:0/1491379704 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2977201970 0 0) 0x7fa5b815ac50 con 0x7fa5b815de50 2023-10-06T02:29:32.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.233+0000 7fa5b5ffb700 1 -- 172.21.15.31:0/1491379704 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5a0003570 con 0x7fa5b815de50 2023-10-06T02:29:32.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.234+0000 7fa5b77fe700 1 -- 172.21.15.31:0/1491379704 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa5b8153a00 msgr2=0x7fa5b8153dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:32.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.234+0000 7fa5b77fe700 1 --2- 172.21.15.31:0/1491379704 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa5b8153a00 0x7fa5b8153dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:32.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.234+0000 7fa5b77fe700 1 -- 172.21.15.31:0/1491379704 >> v1:172.21.15.31:6789/0 conn(0x7fa5b815de50 legacy=0x7fa5b815a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:32.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.234+0000 7fa5b77fe700 1 -- 172.21.15.31:0/1491379704 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5b815b240 con 0x7fa5b80cea00 2023-10-06T02:29:32.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.234+0000 7fa5b77fe700 1 --2- 172.21.15.31:0/1491379704 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5b80cea00 0x7fa5b80cee40 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fa5b0009e60 tx=0x7fa5b0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c23760da7ab63b1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:32.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.234+0000 7fa5b5ffb700 1 -- 172.21.15.31:0/1491379704 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b000e070 con 0x7fa5b80cea00 2023-10-06T02:29:32.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.234+0000 7fa5b5ffb700 1 -- 172.21.15.31:0/1491379704 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5b000bab0 con 0x7fa5b80cea00 2023-10-06T02:29:32.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.234+0000 7fa5b5ffb700 1 -- 172.21.15.31:0/1491379704 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b000be30 con 0x7fa5b80cea00 2023-10-06T02:29:32.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.235+0000 7fa5b7fff700 1 -- 172.21.15.31:0/1491379704 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5b80cea00 msgr2=0x7fa5b80cee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:32.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.235+0000 7fa5b7fff700 1 --2- 172.21.15.31:0/1491379704 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5b80cea00 0x7fa5b80cee40 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fa5b0009e60 tx=0x7fa5b0009780 comp rx=0 tx=0).stop 2023-10-06T02:29:32.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.235+0000 7fa5b7fff700 1 -- 172.21.15.31:0/1491379704 shutdown_connections 2023-10-06T02:29:32.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.235+0000 7fa5b7fff700 1 --2- 172.21.15.31:0/1491379704 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5b80cea00 0x7fa5b80cee40 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:32.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.235+0000 7fa5b7fff700 1 --2- 172.21.15.31:0/1491379704 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa5b8153a00 0x7fa5b8153dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:32.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.235+0000 7fa5b7fff700 1 -- 172.21.15.31:0/1491379704 >> 172.21.15.31:0/1491379704 conn(0x7fa5b80703a0 msgr2=0x7fa5b80707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:32.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.235+0000 7fa5b7fff700 1 -- 172.21.15.31:0/1491379704 shutdown_connections 2023-10-06T02:29:32.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.235+0000 7fa5b7fff700 1 -- 172.21.15.31:0/1491379704 wait complete. 2023-10-06T02:29:32.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.236+0000 7fa5b7fff700 1 Processor -- start 2023-10-06T02:29:32.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.236+0000 7fa5b7fff700 1 -- start start 2023-10-06T02:29:32.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.237+0000 7fa5b7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa5b80cea00 0x7fa5b80c67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:32.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.237+0000 7fa5b7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5b8153a00 0x7fa5b8152ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:32.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.237+0000 7fa5b7fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa5b815de50 0x7fa5b80b3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:32.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.237+0000 7fa5b7fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b80c75e0 con 0x7fa5b8153a00 2023-10-06T02:29:32.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.237+0000 7fa5b7fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b80c78a0 con 0x7fa5b80cea00 2023-10-06T02:29:32.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.237+0000 7fa5b7fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b80c7b80 con 0x7fa5b815de50 2023-10-06T02:29:32.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.238+0000 7fa5b67fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5b8153a00 0x7fa5b8152ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:32.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.238+0000 7fa5b67fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5b8153a00 0x7fa5b8152ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34352/0 (socket says 172.21.15.31:34352) 2023-10-06T02:29:32.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.238+0000 7fa5b67fc700 1 -- 172.21.15.31:0/2528831163 learned_addr learned my addr 172.21.15.31:0/2528831163 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:32.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.238+0000 7fa5b6ffd700 1 --2- 172.21.15.31:0/2528831163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa5b80cea00 0x7fa5b80c67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:32.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.238+0000 7fa5b77fe700 1 --2- 172.21.15.31:0/2528831163 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa5b815de50 0x7fa5b80b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:32.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.238+0000 7fa5b67fc700 1 -- 172.21.15.31:0/2528831163 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa5b815de50 msgr2=0x7fa5b80b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:32.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.238+0000 7fa5b67fc700 1 --2- 172.21.15.31:0/2528831163 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa5b815de50 0x7fa5b80b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:32.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.238+0000 7fa5b67fc700 1 -- 172.21.15.31:0/2528831163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa5b80cea00 msgr2=0x7fa5b80c67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:32.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.238+0000 7fa5b67fc700 1 --2- 172.21.15.31:0/2528831163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa5b80cea00 0x7fa5b80c67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:32.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.238+0000 7fa5b67fc700 1 -- 172.21.15.31:0/2528831163 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5b0009a20 con 0x7fa5b8153a00 2023-10-06T02:29:32.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.238+0000 7fa5b67fc700 1 --2- 172.21.15.31:0/2528831163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5b8153a00 0x7fa5b8152ff0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac0011e0 tx=0x7fa5ac007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:32.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.239+0000 7fa59effd700 1 -- 172.21.15.31:0/2528831163 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5ac005b30 con 0x7fa5b8153a00 2023-10-06T02:29:32.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.239+0000 7fa59effd700 1 -- 172.21.15.31:0/2528831163 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5ac005c90 con 0x7fa5b8153a00 2023-10-06T02:29:32.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.239+0000 7fa59effd700 1 -- 172.21.15.31:0/2528831163 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5ac007320 con 0x7fa5b8153a00 2023-10-06T02:29:32.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.239+0000 7fa5b7fff700 1 -- 172.21.15.31:0/2528831163 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5b80c7e60 con 0x7fa5b8153a00 2023-10-06T02:29:32.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.239+0000 7fa5b7fff700 1 -- 172.21.15.31:0/2528831163 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5b80c6d30 con 0x7fa5b8153a00 2023-10-06T02:29:32.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.240+0000 7fa5b7fff700 1 -- 172.21.15.31:0/2528831163 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5b80c8380 con 0x7fa5b8153a00 2023-10-06T02:29:32.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.241+0000 7fa59effd700 1 -- 172.21.15.31:0/2528831163 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fa5ac007690 con 0x7fa5b8153a00 2023-10-06T02:29:32.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.242+0000 7fa59effd700 1 --2- 172.21.15.31:0/2528831163 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa5a4067320 0x7fa5a40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:32.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.242+0000 7fa59effd700 1 -- 172.21.15.31:0/2528831163 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fa5ac083e90 con 0x7fa5b8153a00 2023-10-06T02:29:32.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.242+0000 7fa5b6ffd700 1 --2- 172.21.15.31:0/2528831163 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa5a4067320 0x7fa5a40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:32.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.243+0000 7fa5b6ffd700 1 --2- 172.21.15.31:0/2528831163 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa5a4067320 0x7fa5a40697d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa5a8009db0 tx=0x7fa5a80091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:32.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.246+0000 7fa59effd700 1 -- 172.21.15.31:0/2528831163 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa5b80c8380 con 0x7fa5b8153a00 2023-10-06T02:29:32.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.445+0000 7fa5b7fff700 1 -- 172.21.15.31:0/2528831163 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa5b80c8380 con 0x7fa5b8153a00 2023-10-06T02:29:32.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.445+0000 7fa59effd700 1 -- 172.21.15.31:0/2528831163 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v116) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa5b80c8380 con 0x7fa5b8153a00 2023-10-06T02:29:32.447 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-10-06T02:29:32.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.447+0000 7fa5b7fff700 1 -- 172.21.15.31:0/2528831163 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa5a4067320 msgr2=0x7fa5a40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:32.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.447+0000 7fa5b7fff700 1 --2- 172.21.15.31:0/2528831163 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa5a4067320 0x7fa5a40697d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa5a8009db0 tx=0x7fa5a80091a0 comp rx=0 tx=0).stop 2023-10-06T02:29:32.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.448+0000 7fa5b7fff700 1 -- 172.21.15.31:0/2528831163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5b8153a00 msgr2=0x7fa5b8152ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:32.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.448+0000 7fa5b7fff700 1 --2- 172.21.15.31:0/2528831163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5b8153a00 0x7fa5b8152ff0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac0011e0 tx=0x7fa5ac007df0 comp rx=0 tx=0).stop 2023-10-06T02:29:32.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.448+0000 7fa5b7fff700 1 -- 172.21.15.31:0/2528831163 shutdown_connections 2023-10-06T02:29:32.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.448+0000 7fa5b7fff700 1 --2- 172.21.15.31:0/2528831163 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa5a4067320 0x7fa5a40697d0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:32.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.448+0000 7fa5b7fff700 1 --2- 172.21.15.31:0/2528831163 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa5b815de50 0x7fa5b80b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:32.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.448+0000 7fa5b7fff700 1 --2- 172.21.15.31:0/2528831163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa5b80cea00 0x7fa5b80c67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:32.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.448+0000 7fa5b7fff700 1 --2- 172.21.15.31:0/2528831163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5b8153a00 0x7fa5b8152ff0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:32.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.448+0000 7fa5b7fff700 1 -- 172.21.15.31:0/2528831163 >> 172.21.15.31:0/2528831163 conn(0x7fa5b80703a0 msgr2=0x7fa5b815bf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:32.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.448+0000 7fa5b7fff700 1 -- 172.21.15.31:0/2528831163 shutdown_connections 2023-10-06T02:29:32.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.448+0000 7fa5b7fff700 1 -- 172.21.15.31:0/2528831163 wait complete. 2023-10-06T02:29:32.466 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T02:29:32.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.655+0000 7efc2df9d700 1 Processor -- start 2023-10-06T02:29:32.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.655+0000 7efc2df9d700 1 -- start start 2023-10-06T02:29:32.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.656+0000 7efc2df9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7efc28153a00 0x7efc28153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:32.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.656+0000 7efc2df9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7efc280cea00 0x7efc280cee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:32.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.656+0000 7efc2df9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc2815ac50 con 0x7efc2815de50 2023-10-06T02:29:32.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.656+0000 7efc2df9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc2815d010 con 0x7efc280cea00 2023-10-06T02:29:32.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.656+0000 7efc2df9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc280cf3c0 con 0x7efc28153a00 2023-10-06T02:29:32.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.657+0000 7efc2d79c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7efc280cea00 0x7efc280cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:32.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.657+0000 7efc27fff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7efc2815de50 0x7efc2815a540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:36360/0 (socket says 172.21.15.31:36360) 2023-10-06T02:29:32.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.657+0000 7efc27fff700 1 -- 172.21.15.31:0/803469761 learned_addr learned my addr 172.21.15.31:0/803469761 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:29:32.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.657+0000 7efc2cf9b700 1 --2- 172.21.15.31:0/803469761 >> v2:172.21.15.62:3300/0 conn(0x7efc28153a00 0x7efc28153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:32.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.657+0000 7efc277fe700 1 -- 172.21.15.31:0/803469761 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3893539695 0 0) 0x7efc2815ac50 con 0x7efc2815de50 2023-10-06T02:29:32.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.657+0000 7efc277fe700 1 -- 172.21.15.31:0/803469761 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc18003570 con 0x7efc2815de50 2023-10-06T02:29:32.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.657+0000 7efc2d79c700 1 -- 172.21.15.31:0/803469761 >> v2:172.21.15.62:3300/0 conn(0x7efc28153a00 msgr2=0x7efc28153dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:32.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.657+0000 7efc2d79c700 1 --2- 172.21.15.31:0/803469761 >> v2:172.21.15.62:3300/0 conn(0x7efc28153a00 0x7efc28153dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:32.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.657+0000 7efc2d79c700 1 -- 172.21.15.31:0/803469761 >> v1:172.21.15.31:6789/0 conn(0x7efc2815de50 legacy=0x7efc2815a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:32.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.657+0000 7efc2d79c700 1 -- 172.21.15.31:0/803469761 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc2815af60 con 0x7efc280cea00 2023-10-06T02:29:32.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.658+0000 7efc2d79c700 1 --2- 172.21.15.31:0/803469761 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc280cea00 0x7efc280cee80 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7efc280cac20 tx=0x7efc20009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c5b0bb0894d73ba server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:32.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.658+0000 7efc277fe700 1 -- 172.21.15.31:0/803469761 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc2000e070 con 0x7efc280cea00 2023-10-06T02:29:32.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.658+0000 7efc277fe700 1 -- 172.21.15.31:0/803469761 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc2000bab0 con 0x7efc280cea00 2023-10-06T02:29:32.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.658+0000 7efc277fe700 1 -- 172.21.15.31:0/803469761 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc2000be70 con 0x7efc280cea00 2023-10-06T02:29:32.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.658+0000 7efc2df9d700 1 -- 172.21.15.31:0/803469761 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc280cea00 msgr2=0x7efc280cee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:32.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.658+0000 7efc2df9d700 1 --2- 172.21.15.31:0/803469761 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc280cea00 0x7efc280cee80 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7efc280cac20 tx=0x7efc20009780 comp rx=0 tx=0).stop 2023-10-06T02:29:32.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.658+0000 7efc2df9d700 1 -- 172.21.15.31:0/803469761 shutdown_connections 2023-10-06T02:29:32.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.658+0000 7efc2df9d700 1 --2- 172.21.15.31:0/803469761 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc280cea00 0x7efc280cee80 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:32.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.658+0000 7efc2df9d700 1 --2- 172.21.15.31:0/803469761 >> v2:172.21.15.62:3300/0 conn(0x7efc28153a00 0x7efc28153dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:32.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.658+0000 7efc2df9d700 1 -- 172.21.15.31:0/803469761 >> 172.21.15.31:0/803469761 conn(0x7efc280703a0 msgr2=0x7efc280707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:32.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.659+0000 7efc2df9d700 1 -- 172.21.15.31:0/803469761 shutdown_connections 2023-10-06T02:29:32.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.659+0000 7efc2df9d700 1 -- 172.21.15.31:0/803469761 wait complete. 2023-10-06T02:29:32.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.660+0000 7efc2df9d700 1 Processor -- start 2023-10-06T02:29:32.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.660+0000 7efc2df9d700 1 -- start start 2023-10-06T02:29:32.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.661+0000 7efc2df9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc280cea00 0x7efc2814bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:32.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.661+0000 7efc2df9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc28153a00 0x7efc2815e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:32.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.661+0000 7efc2df9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efc2815de50 0x7efc280cc290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:32.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.661+0000 7efc2df9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc281506d0 con 0x7efc280cea00 2023-10-06T02:29:32.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.661+0000 7efc2df9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc2815f5b0 con 0x7efc28153a00 2023-10-06T02:29:32.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.661+0000 7efc2df9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc2814c190 con 0x7efc2815de50 2023-10-06T02:29:32.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.661+0000 7efc2cf9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc280cea00 0x7efc2814bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:32.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.661+0000 7efc2cf9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc280cea00 0x7efc2814bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34364/0 (socket says 172.21.15.31:34364) 2023-10-06T02:29:32.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.661+0000 7efc2cf9b700 1 -- 172.21.15.31:0/2815279791 learned_addr learned my addr 172.21.15.31:0/2815279791 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:32.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.662+0000 7efc27fff700 1 --2- 172.21.15.31:0/2815279791 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc28153a00 0x7efc2815e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:32.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.662+0000 7efc2d79c700 1 --2- 172.21.15.31:0/2815279791 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efc2815de50 0x7efc280cc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:32.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.662+0000 7efc2cf9b700 1 -- 172.21.15.31:0/2815279791 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efc2815de50 msgr2=0x7efc280cc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:32.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.662+0000 7efc2cf9b700 1 --2- 172.21.15.31:0/2815279791 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efc2815de50 0x7efc280cc290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:32.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.662+0000 7efc2cf9b700 1 -- 172.21.15.31:0/2815279791 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc28153a00 msgr2=0x7efc2815e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:32.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.662+0000 7efc2cf9b700 1 --2- 172.21.15.31:0/2815279791 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc28153a00 0x7efc2815e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:32.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.662+0000 7efc2cf9b700 1 -- 172.21.15.31:0/2815279791 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc20009a20 con 0x7efc280cea00 2023-10-06T02:29:32.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.662+0000 7efc27fff700 1 --2- 172.21.15.31:0/2815279791 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc28153a00 0x7efc2815e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:29:32.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.662+0000 7efc2cf9b700 1 --2- 172.21.15.31:0/2815279791 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc280cea00 0x7efc2814bc50 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7efc1c00ecb0 tx=0x7efc1c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:32.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.662+0000 7efc24ff9700 1 -- 172.21.15.31:0/2815279791 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc1c00c9d0 con 0x7efc280cea00 2023-10-06T02:29:32.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.662+0000 7efc24ff9700 1 -- 172.21.15.31:0/2815279791 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc1c00cb30 con 0x7efc280cea00 2023-10-06T02:29:32.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.662+0000 7efc2df9d700 1 -- 172.21.15.31:0/2815279791 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc2815ed10 con 0x7efc280cea00 2023-10-06T02:29:32.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.662+0000 7efc24ff9700 1 -- 172.21.15.31:0/2815279791 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc1c00ced0 con 0x7efc280cea00 2023-10-06T02:29:32.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.663+0000 7efc2df9d700 1 -- 172.21.15.31:0/2815279791 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc2815f2d0 con 0x7efc280cea00 2023-10-06T02:29:32.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.664+0000 7efc2df9d700 1 -- 172.21.15.31:0/2815279791 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc28150970 con 0x7efc280cea00 2023-10-06T02:29:32.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.664+0000 7efc24ff9700 1 -- 172.21.15.31:0/2815279791 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) v1 ==== 85097+0+0 (secure 0 0 0) 0x7efc1c013680 con 0x7efc280cea00 2023-10-06T02:29:32.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.665+0000 7efc24ff9700 1 --2- 172.21.15.31:0/2815279791 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7efc0c067320 0x7efc0c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:32.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.665+0000 7efc24ff9700 1 -- 172.21.15.31:0/2815279791 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5206+0+0 (secure 0 0 0) 0x7efc1c084e40 con 0x7efc280cea00 2023-10-06T02:29:32.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.666+0000 7efc27fff700 1 --2- 172.21.15.31:0/2815279791 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7efc0c067320 0x7efc0c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:32.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.666+0000 7efc27fff700 1 --2- 172.21.15.31:0/2815279791 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7efc0c067320 0x7efc0c0697d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7efc281598f0 tx=0x7efc14009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:32.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.669+0000 7efc24ff9700 1 -- 172.21.15.31:0/2815279791 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efc1c051cc0 con 0x7efc280cea00 2023-10-06T02:29:32.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.870+0000 7efc2df9d700 1 -- 172.21.15.31:0/2815279791 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7efc281504a0 con 0x7efc280cea00 2023-10-06T02:29:32.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.870+0000 7efc24ff9700 1 -- 172.21.15.31:0/2815279791 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v116) v1 ==== 92+0+10 (secure 0 0 0) 0x7efc1c0552e0 con 0x7efc280cea00 2023-10-06T02:29:32.872 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 8 2023-10-06T02:29:32.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.873+0000 7efc2df9d700 1 -- 172.21.15.31:0/2815279791 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7efc0c067320 msgr2=0x7efc0c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:32.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.873+0000 7efc2df9d700 1 --2- 172.21.15.31:0/2815279791 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7efc0c067320 0x7efc0c0697d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7efc281598f0 tx=0x7efc14009040 comp rx=0 tx=0).stop 2023-10-06T02:29:32.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.873+0000 7efc2df9d700 1 -- 172.21.15.31:0/2815279791 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc280cea00 msgr2=0x7efc2814bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:32.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.873+0000 7efc2df9d700 1 --2- 172.21.15.31:0/2815279791 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc280cea00 0x7efc2814bc50 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7efc1c00ecb0 tx=0x7efc1c0091a0 comp rx=0 tx=0).stop 2023-10-06T02:29:32.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.873+0000 7efc2df9d700 1 -- 172.21.15.31:0/2815279791 shutdown_connections 2023-10-06T02:29:32.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.873+0000 7efc2df9d700 1 --2- 172.21.15.31:0/2815279791 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7efc0c067320 0x7efc0c0697d0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:32.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.873+0000 7efc2df9d700 1 --2- 172.21.15.31:0/2815279791 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efc2815de50 0x7efc280cc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:32.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.873+0000 7efc2df9d700 1 --2- 172.21.15.31:0/2815279791 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc280cea00 0x7efc2814bc50 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:32.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.873+0000 7efc2df9d700 1 --2- 172.21.15.31:0/2815279791 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc28153a00 0x7efc2815e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:32.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.873+0000 7efc2df9d700 1 -- 172.21.15.31:0/2815279791 >> 172.21.15.31:0/2815279791 conn(0x7efc280703a0 msgr2=0x7efc28159c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:32.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.873+0000 7efc2df9d700 1 -- 172.21.15.31:0/2815279791 shutdown_connections 2023-10-06T02:29:32.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:32.873+0000 7efc2df9d700 1 -- 172.21.15.31:0/2815279791 wait complete. 2023-10-06T02:29:32.892 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T02:29:33.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.078+0000 7f25b919d700 1 Processor -- start 2023-10-06T02:29:33.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.078+0000 7f25b919d700 1 -- start start 2023-10-06T02:29:33.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.079+0000 7f25b919d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f25b406e0a0 0x7f25b40c6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:33.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.079+0000 7f25b919d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f25b406f3a0 0x7f25b414fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:33.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.079+0000 7f25b919d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25b406a490 con 0x7f25b406e470 2023-10-06T02:29:33.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.079+0000 7f25b919d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25b406b930 con 0x7f25b406e0a0 2023-10-06T02:29:33.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.079+0000 7f25b919d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25b40c7490 con 0x7f25b406f3a0 2023-10-06T02:29:33.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.080+0000 7f25b3fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f25b406e0a0 0x7f25b40c6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:33.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.080+0000 7f25b3fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f25b406e0a0 0x7f25b40c6b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34380/0 (socket says 172.21.15.31:34380) 2023-10-06T02:29:33.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.080+0000 7f25b3fff700 1 -- 172.21.15.31:0/1581991172 learned_addr learned my addr 172.21.15.31:0/1581991172 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:33.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.080+0000 7f25b37fe700 1 --2- 172.21.15.31:0/1581991172 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f25b406f3a0 0x7f25b414fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:33.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.080+0000 7f25b3fff700 1 -- 172.21.15.31:0/1581991172 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f25b406f3a0 msgr2=0x7f25b414fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:33.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.080+0000 7f25b3fff700 1 --2- 172.21.15.31:0/1581991172 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f25b406f3a0 0x7f25b414fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:33.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.080+0000 7f25b3fff700 1 -- 172.21.15.31:0/1581991172 >> v1:172.21.15.62:6789/0 conn(0x7f25b406e470 legacy=0x7f25b4159740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:33.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.080+0000 7f25b3fff700 1 -- 172.21.15.31:0/1581991172 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25b40c7770 con 0x7f25b406e0a0 2023-10-06T02:29:33.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.080+0000 7f25b3fff700 1 --2- 172.21.15.31:0/1581991172 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25b406e0a0 0x7f25b40c6b20 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f25a40099b0 tx=0x7f25a4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44d40448637b6202 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:33.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.081+0000 7f25b2ffd700 1 -- 172.21.15.31:0/1581991172 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a400b820 con 0x7f25b406e0a0 2023-10-06T02:29:33.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.081+0000 7f25b2ffd700 1 -- 172.21.15.31:0/1581991172 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25a400b980 con 0x7f25b406e0a0 2023-10-06T02:29:33.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.081+0000 7f25b2ffd700 1 -- 172.21.15.31:0/1581991172 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a400bd20 con 0x7f25b406e0a0 2023-10-06T02:29:33.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.081+0000 7f25b919d700 1 -- 172.21.15.31:0/1581991172 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25b406e0a0 msgr2=0x7f25b40c6b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:33.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.081+0000 7f25b919d700 1 --2- 172.21.15.31:0/1581991172 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25b406e0a0 0x7f25b40c6b20 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f25a40099b0 tx=0x7f25a4009250 comp rx=0 tx=0).stop 2023-10-06T02:29:33.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.081+0000 7f25b919d700 1 -- 172.21.15.31:0/1581991172 shutdown_connections 2023-10-06T02:29:33.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.081+0000 7f25b919d700 1 --2- 172.21.15.31:0/1581991172 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25b406e0a0 0x7f25b40c6b20 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:33.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.081+0000 7f25b919d700 1 --2- 172.21.15.31:0/1581991172 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f25b406f3a0 0x7f25b414fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:33.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.081+0000 7f25b919d700 1 -- 172.21.15.31:0/1581991172 >> 172.21.15.31:0/1581991172 conn(0x7f25b406d170 msgr2=0x7f25b40cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:33.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.081+0000 7f25b919d700 1 -- 172.21.15.31:0/1581991172 shutdown_connections 2023-10-06T02:29:33.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.081+0000 7f25b919d700 1 -- 172.21.15.31:0/1581991172 wait complete. 2023-10-06T02:29:33.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.083+0000 7f25b919d700 1 Processor -- start 2023-10-06T02:29:33.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.083+0000 7f25b919d700 1 -- start start 2023-10-06T02:29:33.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.084+0000 7f25b919d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25b406e0a0 0x7f25b414b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:33.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.084+0000 7f25b919d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f25b406e470 0x7f25b4145cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:33.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.084+0000 7f25b919d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f25b406f3a0 0x7f25b40c31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:33.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.084+0000 7f25b919d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b4146230 con 0x7f25b406e0a0 2023-10-06T02:29:33.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.084+0000 7f25b919d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b40afda0 con 0x7f25b406e470 2023-10-06T02:29:33.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.084+0000 7f25b919d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25b40c3720 con 0x7f25b406f3a0 2023-10-06T02:29:33.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.084+0000 7f25b3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25b406e0a0 0x7f25b414b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:33.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.084+0000 7f25b3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25b406e0a0 0x7f25b414b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34392/0 (socket says 172.21.15.31:34392) 2023-10-06T02:29:33.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.084+0000 7f25b3fff700 1 -- 172.21.15.31:0/967938782 learned_addr learned my addr 172.21.15.31:0/967938782 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:33.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.085+0000 7f25b37fe700 1 --2- 172.21.15.31:0/967938782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f25b406e470 0x7f25b4145cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:33.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.085+0000 7f25b899c700 1 --2- 172.21.15.31:0/967938782 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f25b406f3a0 0x7f25b40c31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:33.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.085+0000 7f25b3fff700 1 -- 172.21.15.31:0/967938782 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f25b406f3a0 msgr2=0x7f25b40c31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:33.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.085+0000 7f25b3fff700 1 --2- 172.21.15.31:0/967938782 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f25b406f3a0 0x7f25b40c31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:33.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.085+0000 7f25b3fff700 1 -- 172.21.15.31:0/967938782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f25b406e470 msgr2=0x7f25b4145cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:33.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.085+0000 7f25b3fff700 1 --2- 172.21.15.31:0/967938782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f25b406e470 0x7f25b4145cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:33.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.085+0000 7f25b3fff700 1 -- 172.21.15.31:0/967938782 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25a40094f0 con 0x7f25b406e0a0 2023-10-06T02:29:33.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.085+0000 7f25b3fff700 1 --2- 172.21.15.31:0/967938782 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25b406e0a0 0x7f25b414b7e0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f25a4009290 tx=0x7f25a4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:33.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.085+0000 7f259bfff700 1 -- 172.21.15.31:0/967938782 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a4011ca0 con 0x7f25b406e0a0 2023-10-06T02:29:33.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.085+0000 7f259bfff700 1 -- 172.21.15.31:0/967938782 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25a4020070 con 0x7f25b406e0a0 2023-10-06T02:29:33.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.085+0000 7f259bfff700 1 -- 172.21.15.31:0/967938782 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a401b6a0 con 0x7f25b406e0a0 2023-10-06T02:29:33.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.085+0000 7f25b919d700 1 -- 172.21.15.31:0/967938782 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25b40c4500 con 0x7f25b406e0a0 2023-10-06T02:29:33.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.086+0000 7f25b919d700 1 -- 172.21.15.31:0/967938782 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25b40c3c60 con 0x7f25b406e0a0 2023-10-06T02:29:33.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.087+0000 7f25b919d700 1 -- 172.21.15.31:0/967938782 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25b40aff30 con 0x7f25b406e0a0 2023-10-06T02:29:33.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.087+0000 7f259bfff700 1 -- 172.21.15.31:0/967938782 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f25a4022030 con 0x7f25b406e0a0 2023-10-06T02:29:33.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.088+0000 7f259bfff700 1 --2- 172.21.15.31:0/967938782 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f25a0067320 0x7f25a00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:33.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.088+0000 7f259bfff700 1 -- 172.21.15.31:0/967938782 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f25a40868e0 con 0x7f25b406e0a0 2023-10-06T02:29:33.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.089+0000 7f25b37fe700 1 --2- 172.21.15.31:0/967938782 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f25a0067320 0x7f25a00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:33.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.089+0000 7f25b37fe700 1 --2- 172.21.15.31:0/967938782 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f25a0067320 0x7f25a00697d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f25b415c6d0 tx=0x7f25a8008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:33.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.092+0000 7f259bfff700 1 -- 172.21.15.31:0/967938782 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f25b40aff30 con 0x7f25b406e0a0 2023-10-06T02:29:33.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.299+0000 7f25b919d700 1 -- 172.21.15.31:0/967938782 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f25b40cc030 con 0x7f25b406e0a0 2023-10-06T02:29:33.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.300+0000 7f259bfff700 1 -- 172.21.15.31:0/967938782 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v116) v1 ==== 84+0+126 (secure 0 0 0) 0x7f25a4053830 con 0x7f25b406e0a0 2023-10-06T02:29:33.301 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:29:33.302 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2023-10-06T02:29:33.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:29:33.302 INFO:teuthology.orchestra.run.smithi031.stdout: "poolnum": 1, 2023-10-06T02:29:33.302 INFO:teuthology.orchestra.run.smithi031.stdout: "poolname": ".mgr" 2023-10-06T02:29:33.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:29:33.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:29:33.302 INFO:teuthology.orchestra.run.smithi031.stdout: "poolnum": 2, 2023-10-06T02:29:33.302 INFO:teuthology.orchestra.run.smithi031.stdout: "poolname": "rbd" 2023-10-06T02:29:33.302 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:29:33.302 INFO:teuthology.orchestra.run.smithi031.stdout:] 2023-10-06T02:29:33.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.302+0000 7f25b919d700 1 -- 172.21.15.31:0/967938782 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f25a0067320 msgr2=0x7f25a00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:33.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.302+0000 7f25b919d700 1 --2- 172.21.15.31:0/967938782 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f25a0067320 0x7f25a00697d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f25b415c6d0 tx=0x7f25a8008040 comp rx=0 tx=0).stop 2023-10-06T02:29:33.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.302+0000 7f25b919d700 1 -- 172.21.15.31:0/967938782 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25b406e0a0 msgr2=0x7f25b414b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:33.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.302+0000 7f25b919d700 1 --2- 172.21.15.31:0/967938782 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25b406e0a0 0x7f25b414b7e0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f25a4009290 tx=0x7f25a4016000 comp rx=0 tx=0).stop 2023-10-06T02:29:33.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.302+0000 7f25b919d700 1 -- 172.21.15.31:0/967938782 shutdown_connections 2023-10-06T02:29:33.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.302+0000 7f25b919d700 1 --2- 172.21.15.31:0/967938782 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f25a0067320 0x7f25a00697d0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:33.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.302+0000 7f25b919d700 1 --2- 172.21.15.31:0/967938782 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f25b406f3a0 0x7f25b40c31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:33.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.302+0000 7f25b919d700 1 --2- 172.21.15.31:0/967938782 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25b406e0a0 0x7f25b414b7e0 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:33.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.302+0000 7f25b919d700 1 --2- 172.21.15.31:0/967938782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f25b406e470 0x7f25b4145cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:33.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.302+0000 7f25b919d700 1 -- 172.21.15.31:0/967938782 >> 172.21.15.31:0/967938782 conn(0x7f25b406d170 msgr2=0x7f25b40cb3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:33.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.302+0000 7f25b919d700 1 -- 172.21.15.31:0/967938782 shutdown_connections 2023-10-06T02:29:33.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.302+0000 7f25b919d700 1 -- 172.21.15.31:0/967938782 wait complete. 2023-10-06T02:29:33.321 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T02:29:33.321 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T02:29:33.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.511+0000 7f39bf19d700 1 Processor -- start 2023-10-06T02:29:33.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.511+0000 7f39bf19d700 1 -- start start 2023-10-06T02:29:33.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.512+0000 7f39bf19d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f39b8159fb0 0x7f39b815a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:33.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.512+0000 7f39bf19d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39b80cd5a0 0x7f39b80cb440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:33.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.512+0000 7f39bf19d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39b815d570 con 0x7f39b815a8c0 2023-10-06T02:29:33.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.512+0000 7f39bf19d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39b815e180 con 0x7f39b8159fb0 2023-10-06T02:29:33.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.512+0000 7f39bf19d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39b815d850 con 0x7f39b80cd5a0 2023-10-06T02:29:33.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.513+0000 7f39be19b700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f39b8159fb0 0x7f39b815a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:33.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.513+0000 7f39be99c700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39b80cd5a0 0x7f39b80cb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:33.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.513+0000 7f39bd99a700 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f39b815a8c0 0x7f39b80cce90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:52316/0 (socket says 172.21.15.31:52316) 2023-10-06T02:29:33.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.513+0000 7f39bd99a700 1 -- 172.21.15.31:0/3103337533 learned_addr learned my addr 172.21.15.31:0/3103337533 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:29:33.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.513+0000 7f39be99c700 1 -- 172.21.15.31:0/3103337533 >> v2:172.21.15.62:3300/0 conn(0x7f39b8159fb0 msgr2=0x7f39b815a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:33.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.513+0000 7f39be99c700 1 --2- 172.21.15.31:0/3103337533 >> v2:172.21.15.62:3300/0 conn(0x7f39b8159fb0 0x7f39b815a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:33.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.513+0000 7f39be99c700 1 -- 172.21.15.31:0/3103337533 >> v1:172.21.15.62:6789/0 conn(0x7f39b815a8c0 legacy=0x7f39b80cce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:33.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.513+0000 7f39bd199700 1 -- 172.21.15.31:0/3103337533 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 442877661 0 0) 0x7f39b815d570 con 0x7f39b815a8c0 2023-10-06T02:29:33.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.513+0000 7f39be99c700 1 -- 172.21.15.31:0/3103337533 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39b815db30 con 0x7f39b80cd5a0 2023-10-06T02:29:33.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.514+0000 7f39be99c700 1 --2- 172.21.15.31:0/3103337533 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39b80cd5a0 0x7f39b80cb440 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f39b0009e60 tx=0x7f39b0009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92a2a6cb9e613ea server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:33.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.514+0000 7f39bd199700 1 -- 172.21.15.31:0/3103337533 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b000e070 con 0x7f39b80cd5a0 2023-10-06T02:29:33.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.514+0000 7f39bd199700 1 -- 172.21.15.31:0/3103337533 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39b000bb60 con 0x7f39b80cd5a0 2023-10-06T02:29:33.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.514+0000 7f39bd199700 1 -- 172.21.15.31:0/3103337533 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b0012430 con 0x7f39b80cd5a0 2023-10-06T02:29:33.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.515+0000 7f39bf19d700 1 -- 172.21.15.31:0/3103337533 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39b80cd5a0 msgr2=0x7f39b80cb440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:33.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.515+0000 7f39bf19d700 1 --2- 172.21.15.31:0/3103337533 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39b80cd5a0 0x7f39b80cb440 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f39b0009e60 tx=0x7f39b0009780 comp rx=0 tx=0).stop 2023-10-06T02:29:33.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.515+0000 7f39bf19d700 1 -- 172.21.15.31:0/3103337533 shutdown_connections 2023-10-06T02:29:33.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.515+0000 7f39bf19d700 1 --2- 172.21.15.31:0/3103337533 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39b80cd5a0 0x7f39b80cb440 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:33.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.515+0000 7f39bf19d700 1 --2- 172.21.15.31:0/3103337533 >> v2:172.21.15.62:3300/0 conn(0x7f39b8159fb0 0x7f39b815a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:33.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.515+0000 7f39bf19d700 1 -- 172.21.15.31:0/3103337533 >> 172.21.15.31:0/3103337533 conn(0x7f39b80703a0 msgr2=0x7f39b80707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:33.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.515+0000 7f39bf19d700 1 -- 172.21.15.31:0/3103337533 shutdown_connections 2023-10-06T02:29:33.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.515+0000 7f39bf19d700 1 -- 172.21.15.31:0/3103337533 wait complete. 2023-10-06T02:29:33.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.516+0000 7f39bf19d700 1 Processor -- start 2023-10-06T02:29:33.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.516+0000 7f39bf19d700 1 -- start start 2023-10-06T02:29:33.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.517+0000 7f39bf19d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39b80cd5a0 0x7f39b816a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:33.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.517+0000 7f39bf19d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39b8159fb0 0x7f39b80cc810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:33.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.517+0000 7f39bf19d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39b815a8c0 0x7f39b816a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:33.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.517+0000 7f39bf19d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39b815eb10 con 0x7f39b80cd5a0 2023-10-06T02:29:33.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.517+0000 7f39bf19d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39b816a550 con 0x7f39b815a8c0 2023-10-06T02:29:33.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.517+0000 7f39bf19d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39b815e830 con 0x7f39b8159fb0 2023-10-06T02:29:33.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.517+0000 7f39be19b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39b80cd5a0 0x7f39b816a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:33.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.517+0000 7f39be19b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39b80cd5a0 0x7f39b816a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34400/0 (socket says 172.21.15.31:34400) 2023-10-06T02:29:33.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.517+0000 7f39be19b700 1 -- 172.21.15.31:0/361474613 learned_addr learned my addr 172.21.15.31:0/361474613 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:33.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.518+0000 7f39bd99a700 1 --2- 172.21.15.31:0/361474613 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39b8159fb0 0x7f39b80cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:33.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.518+0000 7f39be99c700 1 --2- 172.21.15.31:0/361474613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39b815a8c0 0x7f39b816a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:33.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.518+0000 7f39be19b700 1 -- 172.21.15.31:0/361474613 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39b8159fb0 msgr2=0x7f39b80cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:33.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.518+0000 7f39be19b700 1 --2- 172.21.15.31:0/361474613 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39b8159fb0 0x7f39b80cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:33.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.518+0000 7f39be19b700 1 -- 172.21.15.31:0/361474613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39b815a8c0 msgr2=0x7f39b816a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:33.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.518+0000 7f39be19b700 1 --2- 172.21.15.31:0/361474613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39b815a8c0 0x7f39b816a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:33.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.518+0000 7f39be19b700 1 -- 172.21.15.31:0/361474613 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39b0009a20 con 0x7f39b80cd5a0 2023-10-06T02:29:33.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.518+0000 7f39be19b700 1 --2- 172.21.15.31:0/361474613 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39b80cd5a0 0x7f39b816a010 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f39b400ec80 tx=0x7f39b40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:33.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.518+0000 7f39ae7fc700 1 -- 172.21.15.31:0/361474613 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b400c7f0 con 0x7f39b80cd5a0 2023-10-06T02:29:33.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.518+0000 7f39ae7fc700 1 -- 172.21.15.31:0/361474613 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39b400c950 con 0x7f39b80cd5a0 2023-10-06T02:29:33.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.518+0000 7f39ae7fc700 1 -- 172.21.15.31:0/361474613 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b400ccf0 con 0x7f39b80cd5a0 2023-10-06T02:29:33.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.519+0000 7f39bf19d700 1 -- 172.21.15.31:0/361474613 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39b814bd50 con 0x7f39b80cd5a0 2023-10-06T02:29:33.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.519+0000 7f39bf19d700 1 -- 172.21.15.31:0/361474613 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39b814c310 con 0x7f39b80cd5a0 2023-10-06T02:29:33.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.520+0000 7f39bf19d700 1 -- 172.21.15.31:0/361474613 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39b814bbb0 con 0x7f39b80cd5a0 2023-10-06T02:29:33.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.520+0000 7f39ae7fc700 1 -- 172.21.15.31:0/361474613 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f39b4013680 con 0x7f39b80cd5a0 2023-10-06T02:29:33.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.522+0000 7f39ae7fc700 1 --2- 172.21.15.31:0/361474613 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f399c067320 0x7f399c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:33.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.522+0000 7f39ae7fc700 1 -- 172.21.15.31:0/361474613 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f39b4084c50 con 0x7f39b80cd5a0 2023-10-06T02:29:33.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.522+0000 7f39bd99a700 1 --2- 172.21.15.31:0/361474613 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f399c067320 0x7f399c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:33.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.523+0000 7f39bd99a700 1 --2- 172.21.15.31:0/361474613 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f399c067320 0x7f399c0697d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f39b815ad20 tx=0x7f39a8007f50 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:33.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.526+0000 7f39ae7fc700 1 -- 172.21.15.31:0/361474613 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f39b814bbb0 con 0x7f39b80cd5a0 2023-10-06T02:29:33.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:33.728+0000 7f39bf19d700 1 -- 172.21.15.31:0/361474613 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f39b814bbb0 con 0x7f39b80cd5a0 2023-10-06T02:29:34.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:34.159+0000 7f39ae7fc700 1 -- 172.21.15.31:0/361474613 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v117) v1 ==== 138+0+0 (secure 0 0 0) 0x7f39b814bbb0 con 0x7f39b80cd5a0 2023-10-06T02:29:34.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:34.165+0000 7f39bf19d700 1 -- 172.21.15.31:0/361474613 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f399c067320 msgr2=0x7f399c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:34.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:34.165+0000 7f39bf19d700 1 --2- 172.21.15.31:0/361474613 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f399c067320 0x7f399c0697d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f39b815ad20 tx=0x7f39a8007f50 comp rx=0 tx=0).stop 2023-10-06T02:29:34.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:34.165+0000 7f39bf19d700 1 -- 172.21.15.31:0/361474613 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39b80cd5a0 msgr2=0x7f39b816a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:34.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:34.165+0000 7f39bf19d700 1 --2- 172.21.15.31:0/361474613 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39b80cd5a0 0x7f39b816a010 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f39b400ec80 tx=0x7f39b40091a0 comp rx=0 tx=0).stop 2023-10-06T02:29:34.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:34.165+0000 7f39bf19d700 1 -- 172.21.15.31:0/361474613 shutdown_connections 2023-10-06T02:29:34.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:34.165+0000 7f39bf19d700 1 --2- 172.21.15.31:0/361474613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39b815a8c0 0x7f39b816a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:34.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:34.165+0000 7f39bf19d700 1 --2- 172.21.15.31:0/361474613 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39b80cd5a0 0x7f39b816a010 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:34.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:34.165+0000 7f39bf19d700 1 --2- 172.21.15.31:0/361474613 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f399c067320 0x7f399c0697d0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:34.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:34.165+0000 7f39bf19d700 1 --2- 172.21.15.31:0/361474613 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39b8159fb0 0x7f39b80cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:34.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:34.165+0000 7f39bf19d700 1 -- 172.21.15.31:0/361474613 >> 172.21.15.31:0/361474613 conn(0x7f39b80703a0 msgr2=0x7f39b814f140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:34.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:34.165+0000 7f39bf19d700 1 -- 172.21.15.31:0/361474613 shutdown_connections 2023-10-06T02:29:34.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:34.165+0000 7f39bf19d700 1 -- 172.21.15.31:0/361474613 wait complete. 2023-10-06T02:29:34.168 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' created 2023-10-06T02:29:34.186 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T02:29:34.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:34.380+0000 7f97aa59d700 1 Processor -- start 2023-10-06T02:29:34.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:34.380+0000 7f97aa59d700 1 -- start start 2023-10-06T02:29:34.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:34.382+0000 7f97aa59d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f97a4140a70 0x7f97a40c6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:34.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:34.382+0000 7f97aa59d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f97a406de50 0x7f97a40aeef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:34.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:34.382+0000 7f97aa59d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97a40c6d50 con 0x7f97a4140e40 2023-10-06T02:29:34.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:34.382+0000 7f97aa59d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97a40c7030 con 0x7f97a4140a70 2023-10-06T02:29:34.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:34.382+0000 7f97aa59d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97a40c7310 con 0x7f97a406de50 2023-10-06T02:29:34.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:34.382+0000 7f97a8d9a700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f97a4140e40 0x7f97a40c7df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:36372/0 (socket says 172.21.15.31:36372) 2023-10-06T02:29:34.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:34.382+0000 7f97a8d9a700 1 -- 172.21.15.31:0/7996703 learned_addr learned my addr 172.21.15.31:0/7996703 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:29:34.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:34.382+0000 7f97a9d9c700 1 --2- 172.21.15.31:0/7996703 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f97a406de50 0x7f97a40aeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:34.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:34.382+0000 7f97a959b700 1 --2- 172.21.15.31:0/7996703 >> v2:172.21.15.62:3300/0 conn(0x7f97a4140a70 0x7f97a40c6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:34.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:34.383+0000 7f979bfff700 1 -- 172.21.15.31:0/7996703 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 74566216 0 0) 0x7f97a40c6d50 con 0x7f97a4140e40 2023-10-06T02:29:34.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:34.383+0000 7f979bfff700 1 -- 172.21.15.31:0/7996703 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9790003570 con 0x7f97a4140e40 2023-10-06T02:29:34.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:34.383+0000 7f97a9d9c700 1 -- 172.21.15.31:0/7996703 >> v2:172.21.15.62:3300/0 conn(0x7f97a4140a70 msgr2=0x7f97a40c6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:34.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:34.383+0000 7f97a9d9c700 1 --2- 172.21.15.31:0/7996703 >> v2:172.21.15.62:3300/0 conn(0x7f97a4140a70 0x7f97a40c6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:34.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:34.383+0000 7f97a9d9c700 1 -- 172.21.15.31:0/7996703 >> v1:172.21.15.31:6789/0 conn(0x7f97a4140e40 legacy=0x7f97a40c7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:34.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:34.383+0000 7f97a9d9c700 1 -- 172.21.15.31:0/7996703 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97a40c75f0 con 0x7f97a406de50 2023-10-06T02:29:34.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:34.383+0000 7f97a9d9c700 1 --2- 172.21.15.31:0/7996703 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f97a406de50 0x7f97a40aeef0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f979c009e60 tx=0x7f979c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29e16eee76eecab3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:34.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:34.383+0000 7f979bfff700 1 -- 172.21.15.31:0/7996703 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f979c00e070 con 0x7f97a406de50 2023-10-06T02:29:34.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:34.383+0000 7f979bfff700 1 -- 172.21.15.31:0/7996703 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f979c00bab0 con 0x7f97a406de50 2023-10-06T02:29:34.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:34.383+0000 7f979bfff700 1 -- 172.21.15.31:0/7996703 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f979c00be70 con 0x7f97a406de50 2023-10-06T02:29:34.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:34.384+0000 7f97aa59d700 1 -- 172.21.15.31:0/7996703 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f97a406de50 msgr2=0x7f97a40aeef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:34.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:34.384+0000 7f97aa59d700 1 --2- 172.21.15.31:0/7996703 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f97a406de50 0x7f97a40aeef0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f979c009e60 tx=0x7f979c009780 comp rx=0 tx=0).stop 2023-10-06T02:29:34.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:34.384+0000 7f97aa59d700 1 -- 172.21.15.31:0/7996703 shutdown_connections 2023-10-06T02:29:34.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:34.384+0000 7f97aa59d700 1 --2- 172.21.15.31:0/7996703 >> v2:172.21.15.62:3300/0 conn(0x7f97a4140a70 0x7f97a40c6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:34.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:34.384+0000 7f97aa59d700 1 --2- 172.21.15.31:0/7996703 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f97a406de50 0x7f97a40aeef0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:34.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:34.384+0000 7f97aa59d700 1 -- 172.21.15.31:0/7996703 >> 172.21.15.31:0/7996703 conn(0x7f97a406d170 msgr2=0x7f97a40cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:34.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:34.384+0000 7f97aa59d700 1 -- 172.21.15.31:0/7996703 shutdown_connections 2023-10-06T02:29:34.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:34.384+0000 7f97aa59d700 1 -- 172.21.15.31:0/7996703 wait complete. 2023-10-06T02:29:34.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:34.385+0000 7f97aa59d700 1 Processor -- start 2023-10-06T02:29:34.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:34.386+0000 7f97aa59d700 1 -- start start 2023-10-06T02:29:34.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:34.386+0000 7f97aa59d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f97a406de50 0x7f97a40ae3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:34.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:34.386+0000 7f97aa59d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f97a4140a70 0x7f97a413dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:34.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:34.387+0000 7f97aa59d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f97a4140e40 0x7f97a40cae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:34.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:34.387+0000 7f97aa59d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a40c4ff0 con 0x7f97a4140a70 2023-10-06T02:29:34.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:34.387+0000 7f97aa59d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a40ae900 con 0x7f97a4140e40 2023-10-06T02:29:34.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:34.387+0000 7f97aa59d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97a41578e0 con 0x7f97a406de50 2023-10-06T02:29:34.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:34.387+0000 7f97a8d9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f97a4140a70 0x7f97a413dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:34.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:34.387+0000 7f97a8d9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f97a4140a70 0x7f97a413dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34406/0 (socket says 172.21.15.31:34406) 2023-10-06T02:29:34.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:34.387+0000 7f97a8d9a700 1 -- 172.21.15.31:0/3272644397 learned_addr learned my addr 172.21.15.31:0/3272644397 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:34.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:34.387+0000 7f97a959b700 1 --2- 172.21.15.31:0/3272644397 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f97a406de50 0x7f97a40ae3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:34.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:34.387+0000 7f97a9d9c700 1 --2- 172.21.15.31:0/3272644397 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f97a4140e40 0x7f97a40cae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:34.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:34.387+0000 7f97a8d9a700 1 -- 172.21.15.31:0/3272644397 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f97a406de50 msgr2=0x7f97a40ae3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:34.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:34.387+0000 7f97a8d9a700 1 --2- 172.21.15.31:0/3272644397 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f97a406de50 0x7f97a40ae3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:34.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:34.387+0000 7f97a8d9a700 1 -- 172.21.15.31:0/3272644397 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f97a4140e40 msgr2=0x7f97a40cae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:34.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:34.387+0000 7f97a8d9a700 1 --2- 172.21.15.31:0/3272644397 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f97a4140e40 0x7f97a40cae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:34.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:34.387+0000 7f97a8d9a700 1 -- 172.21.15.31:0/3272644397 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f979c009a20 con 0x7f97a4140a70 2023-10-06T02:29:34.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:34.387+0000 7f97a959b700 1 --2- 172.21.15.31:0/3272644397 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f97a406de50 0x7f97a40ae3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:29:34.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:34.387+0000 7f97a9d9c700 1 --2- 172.21.15.31:0/3272644397 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f97a4140e40 0x7f97a40cae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:29:34.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:34.388+0000 7f97a8d9a700 1 --2- 172.21.15.31:0/3272644397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f97a4140a70 0x7f97a413dfa0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f979400c7f0 tx=0x7f9794007c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:34.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:34.388+0000 7f97997fa700 1 -- 172.21.15.31:0/3272644397 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97940072e0 con 0x7f97a4140a70 2023-10-06T02:29:34.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:34.388+0000 7f97997fa700 1 -- 172.21.15.31:0/3272644397 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9794007440 con 0x7f97a4140a70 2023-10-06T02:29:34.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:34.388+0000 7f97997fa700 1 -- 172.21.15.31:0/3272644397 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97940057f0 con 0x7f97a4140a70 2023-10-06T02:29:34.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:34.388+0000 7f97aa59d700 1 -- 172.21.15.31:0/3272644397 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97a40c36e0 con 0x7f97a4140a70 2023-10-06T02:29:34.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:34.388+0000 7f97aa59d700 1 -- 172.21.15.31:0/3272644397 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97a40c3ca0 con 0x7f97a4140a70 2023-10-06T02:29:34.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:34.389+0000 7f97aa59d700 1 -- 172.21.15.31:0/3272644397 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97a4157a70 con 0x7f97a4140a70 2023-10-06T02:29:34.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:34.390+0000 7f97997fa700 1 -- 172.21.15.31:0/3272644397 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f9794005c40 con 0x7f97a4140a70 2023-10-06T02:29:34.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:34.392+0000 7f97997fa700 1 --2- 172.21.15.31:0/3272644397 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9788067320 0x7f97880697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:34.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:34.392+0000 7f97997fa700 1 -- 172.21.15.31:0/3272644397 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5576+0+0 (secure 0 0 0) 0x7f97940840a0 con 0x7f97a4140a70 2023-10-06T02:29:34.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:34.392+0000 7f97a959b700 1 --2- 172.21.15.31:0/3272644397 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9788067320 0x7f97880697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:34.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:34.393+0000 7f97a959b700 1 --2- 172.21.15.31:0/3272644397 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9788067320 0x7f97880697d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f97a0009d80 tx=0x7f97a0008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:34.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:34.396+0000 7f97997fa700 1 -- 172.21.15.31:0/3272644397 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9794050d70 con 0x7f97a4140a70 2023-10-06T02:29:34.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:34.602+0000 7f97aa59d700 1 -- 172.21.15.31:0/3272644397 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f97a404c3c0 con 0x7f97a4140a70 2023-10-06T02:29:35.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:35.165+0000 7f97997fa700 1 -- 172.21.15.31:0/3272644397 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v118) v1 ==== 170+0+0 (secure 0 0 0) 0x7f9794017070 con 0x7f97a4140a70 2023-10-06T02:29:35.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:35.176+0000 7f97aa59d700 1 -- 172.21.15.31:0/3272644397 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9788067320 msgr2=0x7f97880697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:35.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:35.176+0000 7f97aa59d700 1 --2- 172.21.15.31:0/3272644397 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9788067320 0x7f97880697d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f97a0009d80 tx=0x7f97a0008040 comp rx=0 tx=0).stop 2023-10-06T02:29:35.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:35.176+0000 7f97aa59d700 1 -- 172.21.15.31:0/3272644397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f97a4140a70 msgr2=0x7f97a413dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:35.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:35.176+0000 7f97aa59d700 1 --2- 172.21.15.31:0/3272644397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f97a4140a70 0x7f97a413dfa0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f979400c7f0 tx=0x7f9794007c40 comp rx=0 tx=0).stop 2023-10-06T02:29:35.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:35.178+0000 7f97aa59d700 1 -- 172.21.15.31:0/3272644397 shutdown_connections 2023-10-06T02:29:35.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:35.178+0000 7f97aa59d700 1 --2- 172.21.15.31:0/3272644397 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f97a4140e40 0x7f97a40cae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:35.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:35.178+0000 7f97aa59d700 1 --2- 172.21.15.31:0/3272644397 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9788067320 0x7f97880697d0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:35.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:35.178+0000 7f97aa59d700 1 --2- 172.21.15.31:0/3272644397 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f97a406de50 0x7f97a40ae3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:35.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:35.178+0000 7f97aa59d700 1 --2- 172.21.15.31:0/3272644397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f97a4140a70 0x7f97a413dfa0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:35.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:35.178+0000 7f97aa59d700 1 -- 172.21.15.31:0/3272644397 >> 172.21.15.31:0/3272644397 conn(0x7f97a406d170 msgr2=0x7f97a413ee90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:35.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:35.179+0000 7f97aa59d700 1 -- 172.21.15.31:0/3272644397 shutdown_connections 2023-10-06T02:29:35.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:35.180+0000 7f97aa59d700 1 -- 172.21.15.31:0/3272644397 wait complete. 2023-10-06T02:29:35.188 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' created 2023-10-06T02:29:35.207 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T02:29:35.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:35.397+0000 7f02aed9d700 1 Processor -- start 2023-10-06T02:29:35.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:35.397+0000 7f02aed9d700 1 -- start start 2023-10-06T02:29:35.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:35.398+0000 7f02aed9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f02a8153dd0 0x7f02a815de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:35.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:35.398+0000 7f02aed9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f02a80cea00 0x7f02a80ceea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:35.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:35.398+0000 7f02aed9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02a815ac50 con 0x7f02a8153a00 2023-10-06T02:29:35.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:35.398+0000 7f02aed9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02a815d010 con 0x7f02a8153dd0 2023-10-06T02:29:35.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:35.398+0000 7f02aed9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02a80cf3e0 con 0x7f02a80cea00 2023-10-06T02:29:35.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:35.399+0000 7f02ad59a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f02a8153dd0 0x7f02a815de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:35.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:35.399+0000 7f02ad59a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f02a8153dd0 0x7f02a815de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34422/0 (socket says 172.21.15.31:34422) 2023-10-06T02:29:35.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:35.399+0000 7f02ad59a700 1 -- 172.21.15.31:0/1369109564 learned_addr learned my addr 172.21.15.31:0/1369109564 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:35.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:35.399+0000 7f02ae59c700 1 --2- 172.21.15.31:0/1369109564 >> v2:172.21.15.62:3300/0 conn(0x7f02a80cea00 0x7f02a80ceea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:35.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:35.399+0000 7f02ad59a700 1 -- 172.21.15.31:0/1369109564 >> v2:172.21.15.62:3300/0 conn(0x7f02a80cea00 msgr2=0x7f02a80ceea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:35.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:35.399+0000 7f02ad59a700 1 --2- 172.21.15.31:0/1369109564 >> v2:172.21.15.62:3300/0 conn(0x7f02a80cea00 0x7f02a80ceea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:35.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:35.399+0000 7f02ad59a700 1 -- 172.21.15.31:0/1369109564 >> v1:172.21.15.62:6789/0 conn(0x7f02a8153a00 legacy=0x7f02a815a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:35.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:35.399+0000 7f02ad59a700 1 -- 172.21.15.31:0/1369109564 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02a815af60 con 0x7f02a8153dd0 2023-10-06T02:29:35.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:35.399+0000 7f02ad59a700 1 --2- 172.21.15.31:0/1369109564 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02a8153dd0 0x7f02a815de50 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f02a80d03b0 tx=0x7f0298009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aede0fa12e32a819 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:35.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:35.400+0000 7f02acd99700 1 -- 172.21.15.31:0/1369109564 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0298009e70 con 0x7f02a8153dd0 2023-10-06T02:29:35.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:35.400+0000 7f02acd99700 1 -- 172.21.15.31:0/1369109564 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f029800b860 con 0x7f02a8153dd0 2023-10-06T02:29:35.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:35.400+0000 7f02acd99700 1 -- 172.21.15.31:0/1369109564 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f029800bc00 con 0x7f02a8153dd0 2023-10-06T02:29:35.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:35.400+0000 7f02aed9d700 1 -- 172.21.15.31:0/1369109564 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02a8153dd0 msgr2=0x7f02a815de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:35.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:35.400+0000 7f02aed9d700 1 --2- 172.21.15.31:0/1369109564 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02a8153dd0 0x7f02a815de50 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f02a80d03b0 tx=0x7f0298009250 comp rx=0 tx=0).stop 2023-10-06T02:29:35.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:35.400+0000 7f02aed9d700 1 -- 172.21.15.31:0/1369109564 shutdown_connections 2023-10-06T02:29:35.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:35.400+0000 7f02aed9d700 1 --2- 172.21.15.31:0/1369109564 >> v2:172.21.15.62:3300/0 conn(0x7f02a80cea00 0x7f02a80ceea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:35.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:35.400+0000 7f02aed9d700 1 --2- 172.21.15.31:0/1369109564 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02a8153dd0 0x7f02a815de50 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:35.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:35.400+0000 7f02aed9d700 1 -- 172.21.15.31:0/1369109564 >> 172.21.15.31:0/1369109564 conn(0x7f02a80703a0 msgr2=0x7f02a80707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:35.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:35.400+0000 7f02aed9d700 1 -- 172.21.15.31:0/1369109564 shutdown_connections 2023-10-06T02:29:35.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:35.400+0000 7f02aed9d700 1 -- 172.21.15.31:0/1369109564 wait complete. 2023-10-06T02:29:35.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:35.402+0000 7f02aed9d700 1 Processor -- start 2023-10-06T02:29:35.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:35.402+0000 7f02aed9d700 1 -- start start 2023-10-06T02:29:35.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:35.403+0000 7f02aed9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02a80cea00 0x7f02a816cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:35.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:35.403+0000 7f02aed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02a8153a00 0x7f02a80c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:35.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:35.403+0000 7f02aed9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f02a8153dd0 0x7f02a8144e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:35.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:35.403+0000 7f02aed9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02a80c85d0 con 0x7f02a8153a00 2023-10-06T02:29:35.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:35.403+0000 7f02aed9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02a816d610 con 0x7f02a80cea00 2023-10-06T02:29:35.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:35.403+0000 7f02aed9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02a816d8f0 con 0x7f02a8153dd0 2023-10-06T02:29:35.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:35.403+0000 7f02ad59a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02a8153a00 0x7f02a80c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:35.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:35.403+0000 7f02ad59a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02a8153a00 0x7f02a80c8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34430/0 (socket says 172.21.15.31:34430) 2023-10-06T02:29:35.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:35.403+0000 7f02ad59a700 1 -- 172.21.15.31:0/2012802515 learned_addr learned my addr 172.21.15.31:0/2012802515 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:35.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:35.403+0000 7f02add9b700 1 --2- 172.21.15.31:0/2012802515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02a80cea00 0x7f02a816cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:35.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:35.403+0000 7f02ae59c700 1 --2- 172.21.15.31:0/2012802515 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f02a8153dd0 0x7f02a8144e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:35.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:35.403+0000 7f02ad59a700 1 -- 172.21.15.31:0/2012802515 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f02a8153dd0 msgr2=0x7f02a8144e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:35.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:35.403+0000 7f02ad59a700 1 --2- 172.21.15.31:0/2012802515 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f02a8153dd0 0x7f02a8144e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:35.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:35.403+0000 7f02ad59a700 1 -- 172.21.15.31:0/2012802515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02a80cea00 msgr2=0x7f02a816cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:35.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:35.403+0000 7f02ad59a700 1 --2- 172.21.15.31:0/2012802515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02a80cea00 0x7f02a816cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:35.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:35.403+0000 7f02ad59a700 1 -- 172.21.15.31:0/2012802515 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02980094f0 con 0x7f02a8153a00 2023-10-06T02:29:35.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:35.403+0000 7f02add9b700 1 --2- 172.21.15.31:0/2012802515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02a80cea00 0x7f02a816cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:29:35.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:35.403+0000 7f02ae59c700 1 --2- 172.21.15.31:0/2012802515 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f02a8153dd0 0x7f02a8144e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:29:35.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:35.404+0000 7f02ad59a700 1 --2- 172.21.15.31:0/2012802515 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02a8153a00 0x7f02a80c8090 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f02a806eb00 tx=0x7f0298017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:35.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:35.404+0000 7f029dffb700 1 -- 172.21.15.31:0/2012802515 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0298011420 con 0x7f02a8153a00 2023-10-06T02:29:35.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:35.404+0000 7f029dffb700 1 -- 172.21.15.31:0/2012802515 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0298011580 con 0x7f02a8153a00 2023-10-06T02:29:35.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:35.404+0000 7f029dffb700 1 -- 172.21.15.31:0/2012802515 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0298011940 con 0x7f02a8153a00 2023-10-06T02:29:35.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:35.404+0000 7f02aed9d700 1 -- 172.21.15.31:0/2012802515 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02a816dbd0 con 0x7f02a8153a00 2023-10-06T02:29:35.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:35.404+0000 7f02aed9d700 1 -- 172.21.15.31:0/2012802515 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02a81453b0 con 0x7f02a8153a00 2023-10-06T02:29:35.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:35.405+0000 7f02aed9d700 1 -- 172.21.15.31:0/2012802515 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02a80cbd40 con 0x7f02a8153a00 2023-10-06T02:29:35.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:35.406+0000 7f029dffb700 1 -- 172.21.15.31:0/2012802515 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f029802c3b0 con 0x7f02a8153a00 2023-10-06T02:29:35.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:35.407+0000 7f029dffb700 1 --2- 172.21.15.31:0/2012802515 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f028c067320 0x7f028c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:35.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:35.407+0000 7f029dffb700 1 -- 172.21.15.31:0/2012802515 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5958+0+0 (secure 0 0 0) 0x7f0298086380 con 0x7f02a8153a00 2023-10-06T02:29:35.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:35.408+0000 7f02add9b700 1 --2- 172.21.15.31:0/2012802515 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f028c067320 0x7f028c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:35.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:35.409+0000 7f02add9b700 1 --2- 172.21.15.31:0/2012802515 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f028c067320 0x7f028c0697d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f02a81536e0 tx=0x7f02a4009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:35.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:35.413+0000 7f029dffb700 1 -- 172.21.15.31:0/2012802515 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f02a80cbd40 con 0x7f02a8153a00 2023-10-06T02:29:35.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:35.662+0000 7f02aed9d700 1 -- 172.21.15.31:0/2012802515 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f02a80cbd40 con 0x7f02a8153a00 2023-10-06T02:29:36.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.169+0000 7f029dffb700 1 -- 172.21.15.31:0/2012802515 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 19 and data pool 20 v76) v1 ==== 174+0+0 (secure 0 0 0) 0x7f02a80cbd40 con 0x7f02a8153a00 2023-10-06T02:29:36.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.177+0000 7f02aed9d700 1 -- 172.21.15.31:0/2012802515 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f028c067320 msgr2=0x7f028c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:36.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.177+0000 7f02aed9d700 1 --2- 172.21.15.31:0/2012802515 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f028c067320 0x7f028c0697d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f02a81536e0 tx=0x7f02a4009040 comp rx=0 tx=0).stop 2023-10-06T02:29:36.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.177+0000 7f02aed9d700 1 -- 172.21.15.31:0/2012802515 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02a8153a00 msgr2=0x7f02a80c8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:36.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.177+0000 7f02aed9d700 1 --2- 172.21.15.31:0/2012802515 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02a8153a00 0x7f02a80c8090 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f02a806eb00 tx=0x7f0298017000 comp rx=0 tx=0).stop 2023-10-06T02:29:36.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.178+0000 7f02aed9d700 1 -- 172.21.15.31:0/2012802515 shutdown_connections 2023-10-06T02:29:36.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.178+0000 7f02aed9d700 1 --2- 172.21.15.31:0/2012802515 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f028c067320 0x7f028c0697d0 secure :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f02a81536e0 tx=0x7f02a4009040 comp rx=0 tx=0).stop 2023-10-06T02:29:36.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.178+0000 7f02aed9d700 1 --2- 172.21.15.31:0/2012802515 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f02a8153dd0 0x7f02a8144e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:36.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.178+0000 7f02aed9d700 1 --2- 172.21.15.31:0/2012802515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02a80cea00 0x7f02a816cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:36.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.178+0000 7f02aed9d700 1 --2- 172.21.15.31:0/2012802515 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02a8153a00 0x7f02a80c8090 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:36.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.178+0000 7f02aed9d700 1 -- 172.21.15.31:0/2012802515 >> 172.21.15.31:0/2012802515 conn(0x7f02a80703a0 msgr2=0x7f02a81509b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:36.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.184+0000 7f02aed9d700 1 -- 172.21.15.31:0/2012802515 shutdown_connections 2023-10-06T02:29:36.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.185+0000 7f02aed9d700 1 -- 172.21.15.31:0/2012802515 wait complete. 2023-10-06T02:29:36.197 INFO:teuthology.orchestra.run.smithi031.stderr:new fs with metadata pool 19 and data pool 20 2023-10-06T02:29:36.216 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:29:36.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.402+0000 7f5569b9d700 1 Processor -- start 2023-10-06T02:29:36.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.402+0000 7f5569b9d700 1 -- start start 2023-10-06T02:29:36.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.403+0000 7f5569b9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5564159550 0x7f55640ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:36.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.403+0000 7f5569b9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55640acc00 con 0x7f556415dfd0 2023-10-06T02:29:36.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.403+0000 7f5569b9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55640abff0 con 0x7f5564159920 2023-10-06T02:29:36.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.403+0000 7f5569b9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55640ac2d0 con 0x7f5564159550 2023-10-06T02:29:36.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.404+0000 7f5568b9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5564159550 0x7f55640ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:36.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.404+0000 7f5563fff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f556415dfd0 0x7f55640aedb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:36388/0 (socket says 172.21.15.31:36388) 2023-10-06T02:29:36.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.404+0000 7f5563fff700 1 -- 172.21.15.31:0/432650518 learned_addr learned my addr 172.21.15.31:0/432650518 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:29:36.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.404+0000 7f5568b9b700 1 -- 172.21.15.31:0/432650518 >> v1:172.21.15.62:6789/0 conn(0x7f5564159920 legacy=0x7f55640ad090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:36.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.404+0000 7f5568b9b700 1 -- 172.21.15.31:0/432650518 >> v1:172.21.15.31:6789/0 conn(0x7f556415dfd0 legacy=0x7f55640aedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:36.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.404+0000 7f5568b9b700 1 -- 172.21.15.31:0/432650518 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55640ac5b0 con 0x7f5564159550 2023-10-06T02:29:36.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.404+0000 7f5568b9b700 1 --2- 172.21.15.31:0/432650518 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5564159550 0x7f55640ceac0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f55580098e0 tx=0x7f55580091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c90c3d0960e556f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:36.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.405+0000 7f55637fe700 1 -- 172.21.15.31:0/432650518 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5558009e80 con 0x7f5564159550 2023-10-06T02:29:36.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.405+0000 7f55637fe700 1 -- 172.21.15.31:0/432650518 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f555800b860 con 0x7f5564159550 2023-10-06T02:29:36.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.405+0000 7f55637fe700 1 -- 172.21.15.31:0/432650518 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f555800bbe0 con 0x7f5564159550 2023-10-06T02:29:36.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.405+0000 7f5569b9d700 1 -- 172.21.15.31:0/432650518 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5564159550 msgr2=0x7f55640ceac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:36.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.405+0000 7f5569b9d700 1 --2- 172.21.15.31:0/432650518 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5564159550 0x7f55640ceac0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f55580098e0 tx=0x7f55580091a0 comp rx=0 tx=0).stop 2023-10-06T02:29:36.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.405+0000 7f5569b9d700 1 -- 172.21.15.31:0/432650518 shutdown_connections 2023-10-06T02:29:36.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.405+0000 7f5569b9d700 1 --2- 172.21.15.31:0/432650518 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5564159550 0x7f55640ceac0 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:36.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.405+0000 7f5569b9d700 1 -- 172.21.15.31:0/432650518 >> 172.21.15.31:0/432650518 conn(0x7f556406ce80 msgr2=0x7f55640ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:36.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.405+0000 7f5569b9d700 1 -- 172.21.15.31:0/432650518 shutdown_connections 2023-10-06T02:29:36.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.405+0000 7f5569b9d700 1 -- 172.21.15.31:0/432650518 wait complete. 2023-10-06T02:29:36.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.407+0000 7f5569b9d700 1 Processor -- start 2023-10-06T02:29:36.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.407+0000 7f5569b9d700 1 -- start start 2023-10-06T02:29:36.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.408+0000 7f5569b9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5564159550 0x7f55640cb250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:36.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.408+0000 7f5569b9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5564159920 0x7f556416c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:36.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.408+0000 7f5569b9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f556415dfd0 0x7f55640aed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:36.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.408+0000 7f5569b9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55640cc690 con 0x7f5564159920 2023-10-06T02:29:36.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.408+0000 7f5569b9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55640cb790 con 0x7f556415dfd0 2023-10-06T02:29:36.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.408+0000 7f5569b9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55640cba10 con 0x7f5564159550 2023-10-06T02:29:36.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.408+0000 7f5563fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5564159920 0x7f556416c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:36.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.408+0000 7f5563fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5564159920 0x7f556416c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34444/0 (socket says 172.21.15.31:34444) 2023-10-06T02:29:36.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.408+0000 7f5563fff700 1 -- 172.21.15.31:0/1015251926 learned_addr learned my addr 172.21.15.31:0/1015251926 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:36.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.408+0000 7f5563fff700 1 -- 172.21.15.31:0/1015251926 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5564159550 msgr2=0x7f55640cb250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:36.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.408+0000 7f5563fff700 1 --2- 172.21.15.31:0/1015251926 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5564159550 0x7f55640cb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:36.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.408+0000 7f5563fff700 1 -- 172.21.15.31:0/1015251926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f556415dfd0 msgr2=0x7f55640aed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:36.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.408+0000 7f556939c700 1 --2- 172.21.15.31:0/1015251926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f556415dfd0 0x7f55640aed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:36.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.409+0000 7f5563fff700 1 --2- 172.21.15.31:0/1015251926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f556415dfd0 0x7f55640aed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:36.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.409+0000 7f5563fff700 1 -- 172.21.15.31:0/1015251926 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5558009420 con 0x7f5564159920 2023-10-06T02:29:36.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.409+0000 7f556939c700 1 --2- 172.21.15.31:0/1015251926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f556415dfd0 0x7f55640aed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:29:36.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.409+0000 7f5563fff700 1 --2- 172.21.15.31:0/1015251926 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5564159920 0x7f556416c7b0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f555000c7c0 tx=0x7f5550007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:36.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.409+0000 7f5560ff9700 1 -- 172.21.15.31:0/1015251926 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5550005b30 con 0x7f5564159920 2023-10-06T02:29:36.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.409+0000 7f5560ff9700 1 -- 172.21.15.31:0/1015251926 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5550005cb0 con 0x7f5564159920 2023-10-06T02:29:36.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.409+0000 7f5560ff9700 1 -- 172.21.15.31:0/1015251926 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5550004a80 con 0x7f5564159920 2023-10-06T02:29:36.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.409+0000 7f5569b9d700 1 -- 172.21.15.31:0/1015251926 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55640cbcf0 con 0x7f5564159920 2023-10-06T02:29:36.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.409+0000 7f5569b9d700 1 -- 172.21.15.31:0/1015251926 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55640cc2b0 con 0x7f5564159920 2023-10-06T02:29:36.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.410+0000 7f5569b9d700 1 -- 172.21.15.31:0/1015251926 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f556404c3c0 con 0x7f5564159920 2023-10-06T02:29:36.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.411+0000 7f5560ff9700 1 -- 172.21.15.31:0/1015251926 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f55500070f0 con 0x7f5564159920 2023-10-06T02:29:36.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.412+0000 7f5560ff9700 1 --2- 172.21.15.31:0/1015251926 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5548067320 0x7f55480697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:36.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.412+0000 7f5560ff9700 1 -- 172.21.15.31:0/1015251926 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f5550084220 con 0x7f5564159920 2023-10-06T02:29:36.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.412+0000 7f5568b9b700 1 --2- 172.21.15.31:0/1015251926 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5548067320 0x7f55480697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:36.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.413+0000 7f5568b9b700 1 --2- 172.21.15.31:0/1015251926 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5548067320 0x7f55480697d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f55580091e0 tx=0x7f5558017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:36.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.416+0000 7f5560ff9700 1 -- 172.21.15.31:0/1015251926 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5550050d90 con 0x7f5564159920 2023-10-06T02:29:36.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.622+0000 7f5569b9d700 1 -- 172.21.15.31:0/1015251926 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5564169ad0 con 0x7f5564159920 2023-10-06T02:29:36.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.623+0000 7f5560ff9700 1 -- 172.21.15.31:0/1015251926 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) v1 ==== 74+0+16845 (secure 0 0 0) 0x7f5564169ad0 con 0x7f5564159920 2023-10-06T02:29:36.625 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:29:36.625 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":119,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:29:35.664604+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:29:33.769454+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-10-06T02:29:34.643236+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:29:36.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.625+0000 7f5569b9d700 1 -- 172.21.15.31:0/1015251926 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5548067320 msgr2=0x7f55480697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:36.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.625+0000 7f5569b9d700 1 --2- 172.21.15.31:0/1015251926 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5548067320 0x7f55480697d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f55580091e0 tx=0x7f5558017000 comp rx=0 tx=0).stop 2023-10-06T02:29:36.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.625+0000 7f5569b9d700 1 -- 172.21.15.31:0/1015251926 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5564159920 msgr2=0x7f556416c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:36.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.625+0000 7f5569b9d700 1 --2- 172.21.15.31:0/1015251926 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5564159920 0x7f556416c7b0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f555000c7c0 tx=0x7f5550007f70 comp rx=0 tx=0).stop 2023-10-06T02:29:36.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.626+0000 7f5569b9d700 1 -- 172.21.15.31:0/1015251926 shutdown_connections 2023-10-06T02:29:36.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.626+0000 7f5569b9d700 1 --2- 172.21.15.31:0/1015251926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f556415dfd0 0x7f55640aed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:36.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.626+0000 7f5569b9d700 1 --2- 172.21.15.31:0/1015251926 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5548067320 0x7f55480697d0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:36.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.626+0000 7f5569b9d700 1 --2- 172.21.15.31:0/1015251926 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5564159550 0x7f55640cb250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:36.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.626+0000 7f5569b9d700 1 --2- 172.21.15.31:0/1015251926 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5564159920 0x7f556416c7b0 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:36.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.626+0000 7f5569b9d700 1 -- 172.21.15.31:0/1015251926 >> 172.21.15.31:0/1015251926 conn(0x7f556406ce80 msgr2=0x7f55640b3310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:36.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.626+0000 7f5569b9d700 1 -- 172.21.15.31:0/1015251926 shutdown_connections 2023-10-06T02:29:36.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.626+0000 7f5569b9d700 1 -- 172.21.15.31:0/1015251926 wait complete. 2023-10-06T02:29:36.645 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:29:36.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.831+0000 7ff9da39d700 1 Processor -- start 2023-10-06T02:29:36.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.831+0000 7ff9da39d700 1 -- start start 2023-10-06T02:29:36.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.832+0000 7ff9da39d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff9d4159550 0x7ff9d40ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:36.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.832+0000 7ff9da39d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff9d4159920 0x7ff9d40ad090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:36.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.832+0000 7ff9da39d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9d40acc00 con 0x7ff9d415dfd0 2023-10-06T02:29:36.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.832+0000 7ff9da39d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9d40abff0 con 0x7ff9d4159550 2023-10-06T02:29:36.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.832+0000 7ff9da39d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d40ac2d0 con 0x7ff9d4159920 2023-10-06T02:29:36.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.833+0000 7ff9d8b9a700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7ff9d415dfd0 0x7ff9d40aedb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:36394/0 (socket says 172.21.15.31:36394) 2023-10-06T02:29:36.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.833+0000 7ff9d8b9a700 1 -- 172.21.15.31:0/1057397828 learned_addr learned my addr 172.21.15.31:0/1057397828 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:29:36.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.833+0000 7ff9d939b700 1 --2- 172.21.15.31:0/1057397828 >> v2:172.21.15.62:3300/0 conn(0x7ff9d4159550 0x7ff9d40ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:36.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.833+0000 7ff9d9b9c700 1 --2- 172.21.15.31:0/1057397828 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff9d4159920 0x7ff9d40ad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:36.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.833+0000 7ff9cbfff700 1 -- 172.21.15.31:0/1057397828 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3393135758 0 0) 0x7ff9d40acc00 con 0x7ff9d415dfd0 2023-10-06T02:29:36.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.833+0000 7ff9cbfff700 1 -- 172.21.15.31:0/1057397828 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9c0003570 con 0x7ff9d415dfd0 2023-10-06T02:29:36.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.833+0000 7ff9d939b700 1 -- 172.21.15.31:0/1057397828 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff9d4159920 msgr2=0x7ff9d40ad090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:36.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.833+0000 7ff9d939b700 1 --2- 172.21.15.31:0/1057397828 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff9d4159920 0x7ff9d40ad090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:36.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.833+0000 7ff9cbfff700 1 -- 172.21.15.31:0/1057397828 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 380856172 0 0) 0x7ff9c0003570 con 0x7ff9d415dfd0 2023-10-06T02:29:36.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.834+0000 7ff9d939b700 1 -- 172.21.15.31:0/1057397828 >> v1:172.21.15.31:6789/0 conn(0x7ff9d415dfd0 legacy=0x7ff9d40aedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:36.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.834+0000 7ff9d939b700 1 -- 172.21.15.31:0/1057397828 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9d40ac5b0 con 0x7ff9d4159550 2023-10-06T02:29:36.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.834+0000 7ff9d939b700 1 --2- 172.21.15.31:0/1057397828 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff9d4159550 0x7ff9d40ceac0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ff9d00098e0 tx=0x7ff9d00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7648c2e6ecc64d59 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:36.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.834+0000 7ff9cbfff700 1 -- 172.21.15.31:0/1057397828 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c4001470 con 0x7ff9d4159550 2023-10-06T02:29:36.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.834+0000 7ff9cbfff700 1 -- 172.21.15.31:0/1057397828 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9d0011420 con 0x7ff9d4159550 2023-10-06T02:29:36.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.834+0000 7ff9cbfff700 1 -- 172.21.15.31:0/1057397828 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d00117c0 con 0x7ff9d4159550 2023-10-06T02:29:36.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.835+0000 7ff9da39d700 1 -- 172.21.15.31:0/1057397828 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff9d4159550 msgr2=0x7ff9d40ceac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:36.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.835+0000 7ff9da39d700 1 --2- 172.21.15.31:0/1057397828 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff9d4159550 0x7ff9d40ceac0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ff9d00098e0 tx=0x7ff9d00091a0 comp rx=0 tx=0).stop 2023-10-06T02:29:36.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.835+0000 7ff9da39d700 1 -- 172.21.15.31:0/1057397828 shutdown_connections 2023-10-06T02:29:36.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.835+0000 7ff9da39d700 1 --2- 172.21.15.31:0/1057397828 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff9d4159550 0x7ff9d40ceac0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:36.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.835+0000 7ff9da39d700 1 --2- 172.21.15.31:0/1057397828 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff9d4159920 0x7ff9d40ad090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:36.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.835+0000 7ff9da39d700 1 -- 172.21.15.31:0/1057397828 >> 172.21.15.31:0/1057397828 conn(0x7ff9d406ce80 msgr2=0x7ff9d40ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:36.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.835+0000 7ff9da39d700 1 -- 172.21.15.31:0/1057397828 shutdown_connections 2023-10-06T02:29:36.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.835+0000 7ff9da39d700 1 -- 172.21.15.31:0/1057397828 wait complete. 2023-10-06T02:29:36.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.836+0000 7ff9da39d700 1 Processor -- start 2023-10-06T02:29:36.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.836+0000 7ff9da39d700 1 -- start start 2023-10-06T02:29:36.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.837+0000 7ff9da39d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff9d4159550 0x7ff9d40cd400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:36.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.837+0000 7ff9da39d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9d4159920 0x7ff9d4157a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:36.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.837+0000 7ff9da39d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff9d415dfd0 0x7ff9d4157fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:36.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.837+0000 7ff9da39d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d40b1150 con 0x7ff9d4159920 2023-10-06T02:29:36.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.837+0000 7ff9da39d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d40cd940 con 0x7ff9d415dfd0 2023-10-06T02:29:36.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.837+0000 7ff9da39d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d4140bc0 con 0x7ff9d4159550 2023-10-06T02:29:36.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.838+0000 7ff9d8b9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9d4159920 0x7ff9d4157a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:36.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.838+0000 7ff9d8b9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9d4159920 0x7ff9d4157a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34454/0 (socket says 172.21.15.31:34454) 2023-10-06T02:29:36.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.838+0000 7ff9d8b9a700 1 -- 172.21.15.31:0/1014867823 learned_addr learned my addr 172.21.15.31:0/1014867823 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:36.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.838+0000 7ff9d939b700 1 --2- 172.21.15.31:0/1014867823 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff9d4159550 0x7ff9d40cd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:36.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.838+0000 7ff9d9b9c700 1 --2- 172.21.15.31:0/1014867823 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff9d415dfd0 0x7ff9d4157fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:36.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.838+0000 7ff9d8b9a700 1 -- 172.21.15.31:0/1014867823 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff9d4159550 msgr2=0x7ff9d40cd400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:36.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.838+0000 7ff9d8b9a700 1 --2- 172.21.15.31:0/1014867823 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff9d4159550 0x7ff9d40cd400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:36.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.838+0000 7ff9d8b9a700 1 -- 172.21.15.31:0/1014867823 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff9d415dfd0 msgr2=0x7ff9d4157fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:36.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.838+0000 7ff9d8b9a700 1 --2- 172.21.15.31:0/1014867823 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff9d415dfd0 0x7ff9d4157fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:36.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.838+0000 7ff9d8b9a700 1 -- 172.21.15.31:0/1014867823 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9d0009420 con 0x7ff9d4159920 2023-10-06T02:29:36.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.838+0000 7ff9d8b9a700 1 --2- 172.21.15.31:0/1014867823 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9d4159920 0x7ff9d4157a80 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7ff9c400c7f0 tx=0x7ff9c40019e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:36.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.839+0000 7ff9c97fa700 1 -- 172.21.15.31:0/1014867823 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c4007360 con 0x7ff9d4159920 2023-10-06T02:29:36.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.839+0000 7ff9c97fa700 1 -- 172.21.15.31:0/1014867823 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9c40074e0 con 0x7ff9d4159920 2023-10-06T02:29:36.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.839+0000 7ff9c97fa700 1 -- 172.21.15.31:0/1014867823 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c4004830 con 0x7ff9d4159920 2023-10-06T02:29:36.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.839+0000 7ff9da39d700 1 -- 172.21.15.31:0/1014867823 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9d40cdbc0 con 0x7ff9d4159920 2023-10-06T02:29:36.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.839+0000 7ff9da39d700 1 -- 172.21.15.31:0/1014867823 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9d40b1680 con 0x7ff9d4159920 2023-10-06T02:29:36.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.840+0000 7ff9da39d700 1 -- 172.21.15.31:0/1014867823 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9d404c3c0 con 0x7ff9d4159920 2023-10-06T02:29:36.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.841+0000 7ff9c97fa700 1 -- 172.21.15.31:0/1014867823 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) v1 ==== 85097+0+0 (secure 0 0 0) 0x7ff9c4004c60 con 0x7ff9d4159920 2023-10-06T02:29:36.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.842+0000 7ff9c97fa700 1 --2- 172.21.15.31:0/1014867823 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff9b8067320 0x7ff9b80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:36.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.842+0000 7ff9c97fa700 1 -- 172.21.15.31:0/1014867823 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff9c40845b0 con 0x7ff9d4159920 2023-10-06T02:29:36.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.843+0000 7ff9d939b700 1 --2- 172.21.15.31:0/1014867823 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff9b8067320 0x7ff9b80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:36.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.843+0000 7ff9d939b700 1 --2- 172.21.15.31:0/1014867823 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff9b8067320 0x7ff9b80697d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff9d00030c0 tx=0x7ff9d0016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:36.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:36.847+0000 7ff9c97fa700 1 -- 172.21.15.31:0/1014867823 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff9c4051120 con 0x7ff9d4159920 2023-10-06T02:29:37.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:37.051+0000 7ff9da39d700 1 -- 172.21.15.31:0/1014867823 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff9d406e7c0 con 0x7ff9d4159920 2023-10-06T02:29:37.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:37.052+0000 7ff9c97fa700 1 -- 172.21.15.31:0/1014867823 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) v1 ==== 74+0+16845 (secure 0 0 0) 0x7ff9c4054740 con 0x7ff9d4159920 2023-10-06T02:29:37.054 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:29:37.054 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":119,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:29:35.664604+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:29:33.769454+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-10-06T02:29:34.643236+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:29:37.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:37.055+0000 7ff9da39d700 1 -- 172.21.15.31:0/1014867823 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff9b8067320 msgr2=0x7ff9b80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:37.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:37.055+0000 7ff9da39d700 1 --2- 172.21.15.31:0/1014867823 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff9b8067320 0x7ff9b80697d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff9d00030c0 tx=0x7ff9d0016000 comp rx=0 tx=0).stop 2023-10-06T02:29:37.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:37.055+0000 7ff9da39d700 1 -- 172.21.15.31:0/1014867823 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9d4159920 msgr2=0x7ff9d4157a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:37.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:37.055+0000 7ff9da39d700 1 --2- 172.21.15.31:0/1014867823 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9d4159920 0x7ff9d4157a80 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7ff9c400c7f0 tx=0x7ff9c40019e0 comp rx=0 tx=0).stop 2023-10-06T02:29:37.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:37.055+0000 7ff9da39d700 1 -- 172.21.15.31:0/1014867823 shutdown_connections 2023-10-06T02:29:37.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:37.055+0000 7ff9da39d700 1 --2- 172.21.15.31:0/1014867823 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff9d415dfd0 0x7ff9d4157fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:37.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:37.055+0000 7ff9da39d700 1 --2- 172.21.15.31:0/1014867823 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff9b8067320 0x7ff9b80697d0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:37.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:37.055+0000 7ff9da39d700 1 --2- 172.21.15.31:0/1014867823 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff9d4159550 0x7ff9d40cd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:37.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:37.055+0000 7ff9da39d700 1 --2- 172.21.15.31:0/1014867823 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9d4159920 0x7ff9d4157a80 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:37.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:37.055+0000 7ff9da39d700 1 -- 172.21.15.31:0/1014867823 >> 172.21.15.31:0/1014867823 conn(0x7ff9d406ce80 msgr2=0x7ff9d4152b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:37.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:37.055+0000 7ff9da39d700 1 -- 172.21.15.31:0/1014867823 shutdown_connections 2023-10-06T02:29:37.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:37.055+0000 7ff9da39d700 1 -- 172.21.15.31:0/1014867823 wait complete. 2023-10-06T02:29:37.074 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T02:29:37.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:37.267+0000 7fadf5d9d700 1 Processor -- start 2023-10-06T02:29:37.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:37.267+0000 7fadf5d9d700 1 -- start start 2023-10-06T02:29:37.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:37.269+0000 7fadf5d9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fadf006a210 0x7fadf00c6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:37.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:37.269+0000 7fadf5d9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fadf006de50 0x7fadf00c7190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:37.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:37.269+0000 7fadf5d9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadf015b640 con 0x7fadf0140a70 2023-10-06T02:29:37.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:37.269+0000 7fadf5d9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadf00c6800 con 0x7fadf006de50 2023-10-06T02:29:37.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:37.269+0000 7fadf5d9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadf015b920 con 0x7fadf006a210 2023-10-06T02:29:37.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:37.269+0000 7fadf559c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fadf006de50 0x7fadf00c7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:37.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:37.269+0000 7fadf559c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fadf006de50 0x7fadf00c7190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34456/0 (socket says 172.21.15.31:34456) 2023-10-06T02:29:37.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:37.269+0000 7fadf559c700 1 -- 172.21.15.31:0/3360024807 learned_addr learned my addr 172.21.15.31:0/3360024807 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:37.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:37.269+0000 7fadeffff700 1 --2- 172.21.15.31:0/3360024807 >> v2:172.21.15.62:3300/0 conn(0x7fadf006a210 0x7fadf00c6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:37.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:37.270+0000 7fadf559c700 1 -- 172.21.15.31:0/3360024807 >> v2:172.21.15.62:3300/0 conn(0x7fadf006a210 msgr2=0x7fadf00c6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:37.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:37.270+0000 7fadf559c700 1 --2- 172.21.15.31:0/3360024807 >> v2:172.21.15.62:3300/0 conn(0x7fadf006a210 0x7fadf00c6c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:37.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:37.270+0000 7fadf559c700 1 -- 172.21.15.31:0/3360024807 >> v1:172.21.15.62:6789/0 conn(0x7fadf0140a70 legacy=0x7fadf0069b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:37.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:37.270+0000 7fadf559c700 1 -- 172.21.15.31:0/3360024807 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadf015bc00 con 0x7fadf006de50 2023-10-06T02:29:37.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:37.270+0000 7fadf559c700 1 --2- 172.21.15.31:0/3360024807 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fadf006de50 0x7fadf00c7190 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fadf0158bf0 tx=0x7fade8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebedb15654784aa5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:37.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:37.270+0000 7fadef7fe700 1 -- 172.21.15.31:0/3360024807 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade800e070 con 0x7fadf006de50 2023-10-06T02:29:37.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:37.270+0000 7fadef7fe700 1 -- 172.21.15.31:0/3360024807 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fade800ba10 con 0x7fadf006de50 2023-10-06T02:29:37.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:37.270+0000 7fadef7fe700 1 -- 172.21.15.31:0/3360024807 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade80092e0 con 0x7fadf006de50 2023-10-06T02:29:37.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:37.271+0000 7fadf5d9d700 1 -- 172.21.15.31:0/3360024807 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fadf006de50 msgr2=0x7fadf00c7190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:37.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:37.271+0000 7fadf5d9d700 1 --2- 172.21.15.31:0/3360024807 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fadf006de50 0x7fadf00c7190 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fadf0158bf0 tx=0x7fade8009840 comp rx=0 tx=0).stop 2023-10-06T02:29:37.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:37.271+0000 7fadf5d9d700 1 -- 172.21.15.31:0/3360024807 shutdown_connections 2023-10-06T02:29:37.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:37.271+0000 7fadf5d9d700 1 --2- 172.21.15.31:0/3360024807 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fadf006de50 0x7fadf00c7190 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:37.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:37.271+0000 7fadf5d9d700 1 --2- 172.21.15.31:0/3360024807 >> v2:172.21.15.62:3300/0 conn(0x7fadf006a210 0x7fadf00c6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:37.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:37.271+0000 7fadf5d9d700 1 -- 172.21.15.31:0/3360024807 >> 172.21.15.31:0/3360024807 conn(0x7fadf006d110 msgr2=0x7fadf00cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:37.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:37.271+0000 7fadf5d9d700 1 -- 172.21.15.31:0/3360024807 shutdown_connections 2023-10-06T02:29:37.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:37.271+0000 7fadf5d9d700 1 -- 172.21.15.31:0/3360024807 wait complete. 2023-10-06T02:29:37.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:37.272+0000 7fadf5d9d700 1 Processor -- start 2023-10-06T02:29:37.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:37.272+0000 7fadf5d9d700 1 -- start start 2023-10-06T02:29:37.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:37.273+0000 7fadf5d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fadf006a210 0x7fadf013db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:37.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:37.273+0000 7fadf5d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fadf006de50 0x7fadf00c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:37.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:37.273+0000 7fadf5d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fadf0140a70 0x7fadf016cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:37.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:37.273+0000 7fadf5d9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf013e0b0 con 0x7fadf006a210 2023-10-06T02:29:37.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:37.273+0000 7fadf5d9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf00ca750 con 0x7fadf0140a70 2023-10-06T02:29:37.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:37.273+0000 7fadf5d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadf00caa30 con 0x7fadf006de50 2023-10-06T02:29:37.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:37.274+0000 7fadf4d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fadf006a210 0x7fadf013db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:37.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:37.274+0000 7fadf4d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fadf006a210 0x7fadf013db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34458/0 (socket says 172.21.15.31:34458) 2023-10-06T02:29:37.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:37.274+0000 7fadf4d9b700 1 -- 172.21.15.31:0/3360555921 learned_addr learned my addr 172.21.15.31:0/3360555921 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:37.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:37.274+0000 7fadf559c700 1 --2- 172.21.15.31:0/3360555921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fadf0140a70 0x7fadf016cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:37.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:37.274+0000 7fadeffff700 1 --2- 172.21.15.31:0/3360555921 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fadf006de50 0x7fadf00c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:37.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:37.274+0000 7fadf4d9b700 1 -- 172.21.15.31:0/3360555921 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fadf006de50 msgr2=0x7fadf00c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:37.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:37.274+0000 7fadf4d9b700 1 --2- 172.21.15.31:0/3360555921 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fadf006de50 0x7fadf00c9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:37.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:37.274+0000 7fadf4d9b700 1 -- 172.21.15.31:0/3360555921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fadf0140a70 msgr2=0x7fadf016cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:37.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:37.274+0000 7fadf4d9b700 1 --2- 172.21.15.31:0/3360555921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fadf0140a70 0x7fadf016cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:37.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:37.274+0000 7fadf4d9b700 1 -- 172.21.15.31:0/3360555921 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fade8009ae0 con 0x7fadf006a210 2023-10-06T02:29:37.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:37.274+0000 7fadeffff700 1 --2- 172.21.15.31:0/3360555921 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fadf006de50 0x7fadf00c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:29:37.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:37.274+0000 7fadf4d9b700 1 --2- 172.21.15.31:0/3360555921 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fadf006a210 0x7fadf013db70 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fade400b5e0 tx=0x7fade4009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:37.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:37.275+0000 7fadecff9700 1 -- 172.21.15.31:0/3360555921 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade400f510 con 0x7fadf006a210 2023-10-06T02:29:37.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:37.275+0000 7fadecff9700 1 -- 172.21.15.31:0/3360555921 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fade400f670 con 0x7fadf006a210 2023-10-06T02:29:37.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:37.275+0000 7fadecff9700 1 -- 172.21.15.31:0/3360555921 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade400fa30 con 0x7fadf006a210 2023-10-06T02:29:37.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:37.275+0000 7fadf5d9d700 1 -- 172.21.15.31:0/3360555921 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadf00cad10 con 0x7fadf006a210 2023-10-06T02:29:37.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:37.275+0000 7fadf5d9d700 1 -- 172.21.15.31:0/3360555921 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadf00cb2d0 con 0x7fadf006a210 2023-10-06T02:29:37.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:37.276+0000 7fadf5d9d700 1 -- 172.21.15.31:0/3360555921 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadf00cb4f0 con 0x7fadf006a210 2023-10-06T02:29:37.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:37.277+0000 7fadecff9700 1 -- 172.21.15.31:0/3360555921 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fade401a400 con 0x7fadf006a210 2023-10-06T02:29:37.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:37.278+0000 7fadecff9700 1 --2- 172.21.15.31:0/3360555921 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fadd4067350 0x7fadd4069800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:37.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:37.278+0000 7fadecff9700 1 -- 172.21.15.31:0/3360555921 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fade40848b0 con 0x7fadf006a210 2023-10-06T02:29:37.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:37.278+0000 7fadeffff700 1 --2- 172.21.15.31:0/3360555921 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fadd4067350 0x7fadd4069800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:37.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:37.279+0000 7fadeffff700 1 --2- 172.21.15.31:0/3360555921 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fadd4067350 0x7fadd4069800 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fadf015afe0 tx=0x7faddc008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:37.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:37.282+0000 7fadecff9700 1 -- 172.21.15.31:0/3360555921 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fade4051480 con 0x7fadf006a210 2023-10-06T02:29:37.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:37.529+0000 7fadf5d9d700 1 -- 172.21.15.31:0/3360555921 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fadf004c3c0 con 0x7fadf006a210 2023-10-06T02:29:38.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:38.189+0000 7fadecff9700 1 -- 172.21.15.31:0/3360555921 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v79) v1 ==== 118+0+0 (secure 0 0 0) 0x7fade4054aa0 con 0x7fadf006a210 2023-10-06T02:29:38.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:38.193+0000 7fadf5d9d700 1 -- 172.21.15.31:0/3360555921 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fadd4067350 msgr2=0x7fadd4069800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:38.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:38.193+0000 7fadf5d9d700 1 --2- 172.21.15.31:0/3360555921 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fadd4067350 0x7fadd4069800 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fadf015afe0 tx=0x7faddc008040 comp rx=0 tx=0).stop 2023-10-06T02:29:38.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:38.193+0000 7fadf5d9d700 1 -- 172.21.15.31:0/3360555921 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fadf006a210 msgr2=0x7fadf013db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:38.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:38.193+0000 7fadf5d9d700 1 --2- 172.21.15.31:0/3360555921 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fadf006a210 0x7fadf013db70 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fade400b5e0 tx=0x7fade4009120 comp rx=0 tx=0).stop 2023-10-06T02:29:38.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:38.193+0000 7fadf5d9d700 1 -- 172.21.15.31:0/3360555921 shutdown_connections 2023-10-06T02:29:38.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:38.193+0000 7fadf5d9d700 1 --2- 172.21.15.31:0/3360555921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fadf0140a70 0x7fadf016cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:38.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:38.193+0000 7fadf5d9d700 1 --2- 172.21.15.31:0/3360555921 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fadf006a210 0x7fadf013db70 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:38.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:38.193+0000 7fadf5d9d700 1 --2- 172.21.15.31:0/3360555921 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fadd4067350 0x7fadd4069800 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:38.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:38.193+0000 7fadf5d9d700 1 --2- 172.21.15.31:0/3360555921 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fadf006de50 0x7fadf00c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:38.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:38.193+0000 7fadf5d9d700 1 -- 172.21.15.31:0/3360555921 >> 172.21.15.31:0/3360555921 conn(0x7fadf006d110 msgr2=0x7fadf00cc030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:38.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:38.193+0000 7fadf5d9d700 1 -- 172.21.15.31:0/3360555921 shutdown_connections 2023-10-06T02:29:38.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:38.193+0000 7fadf5d9d700 1 -- 172.21.15.31:0/3360555921 wait complete. 2023-10-06T02:29:38.216 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T02:29:38.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:38.412+0000 7ff48ab9d700 1 Processor -- start 2023-10-06T02:29:38.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:38.412+0000 7ff48ab9d700 1 -- start start 2023-10-06T02:29:38.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:38.413+0000 7ff48ab9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff484155890 0x7ff4840c7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:38.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:38.413+0000 7ff48ab9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff484155c60 0x7ff48406a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:38.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:38.413+0000 7ff48ab9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4840c7e70 con 0x7ff48406de50 2023-10-06T02:29:38.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:38.413+0000 7ff48ab9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff48406b790 con 0x7ff484155890 2023-10-06T02:29:38.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:38.413+0000 7ff48ab9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff48406a8e0 con 0x7ff484155c60 2023-10-06T02:29:38.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:38.413+0000 7ff48939a700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff484155c60 0x7ff48406a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:38.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:38.413+0000 7ff489b9b700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff484155890 0x7ff4840c7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:38.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:38.413+0000 7ff48a39c700 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7ff48406de50 0x7ff484159720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:57372/0 (socket says 172.21.15.31:57372) 2023-10-06T02:29:38.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:38.414+0000 7ff489b9b700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff484155890 0x7ff4840c7930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:43396/0 (socket says 172.21.15.31:43396) 2023-10-06T02:29:38.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:38.414+0000 7ff48a39c700 1 -- 172.21.15.31:0/1291081291 learned_addr learned my addr 172.21.15.31:0/1291081291 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:29:38.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:38.414+0000 7ff489b9b700 1 -- 172.21.15.31:0/1291081291 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff484155c60 msgr2=0x7ff48406a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:38.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:38.414+0000 7ff489b9b700 1 --2- 172.21.15.31:0/1291081291 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff484155c60 0x7ff48406a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:38.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:38.414+0000 7ff489b9b700 1 -- 172.21.15.31:0/1291081291 >> v1:172.21.15.62:6789/0 conn(0x7ff48406de50 legacy=0x7ff484159720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:38.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:38.414+0000 7ff489b9b700 1 -- 172.21.15.31:0/1291081291 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff48406ab80 con 0x7ff484155890 2023-10-06T02:29:38.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:38.414+0000 7ff489b9b700 1 --2- 172.21.15.31:0/1291081291 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff484155890 0x7ff4840c7930 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ff4800098e0 tx=0x7ff4800091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8221d5a16e1b7490 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:38.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:38.415+0000 7ff488b99700 1 -- 172.21.15.31:0/1291081291 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff48000b640 con 0x7ff484155890 2023-10-06T02:29:38.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:38.415+0000 7ff488b99700 1 -- 172.21.15.31:0/1291081291 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff48000b7a0 con 0x7ff484155890 2023-10-06T02:29:38.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:38.415+0000 7ff488b99700 1 -- 172.21.15.31:0/1291081291 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff48000bb40 con 0x7ff484155890 2023-10-06T02:29:38.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:38.415+0000 7ff48ab9d700 1 -- 172.21.15.31:0/1291081291 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff484155890 msgr2=0x7ff4840c7930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:38.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:38.415+0000 7ff48ab9d700 1 --2- 172.21.15.31:0/1291081291 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff484155890 0x7ff4840c7930 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ff4800098e0 tx=0x7ff4800091a0 comp rx=0 tx=0).stop 2023-10-06T02:29:38.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:38.415+0000 7ff48ab9d700 1 -- 172.21.15.31:0/1291081291 shutdown_connections 2023-10-06T02:29:38.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:38.415+0000 7ff48ab9d700 1 --2- 172.21.15.31:0/1291081291 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff484155890 0x7ff4840c7930 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:38.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:38.415+0000 7ff48ab9d700 1 --2- 172.21.15.31:0/1291081291 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff484155c60 0x7ff48406a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:38.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:38.415+0000 7ff48ab9d700 1 -- 172.21.15.31:0/1291081291 >> 172.21.15.31:0/1291081291 conn(0x7ff48406d110 msgr2=0x7ff4840cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:38.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:38.415+0000 7ff48ab9d700 1 -- 172.21.15.31:0/1291081291 shutdown_connections 2023-10-06T02:29:38.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:38.415+0000 7ff48ab9d700 1 -- 172.21.15.31:0/1291081291 wait complete. 2023-10-06T02:29:38.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:38.417+0000 7ff48ab9d700 1 Processor -- start 2023-10-06T02:29:38.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:38.417+0000 7ff48ab9d700 1 -- start start 2023-10-06T02:29:38.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:38.418+0000 7ff48ab9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff48406de50 0x7ff484147e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:38.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:38.418+0000 7ff48ab9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff484155890 0x7ff4840c31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:38.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:38.418+0000 7ff48ab9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff484155c60 0x7ff4840c1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:38.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:38.418+0000 7ff48ab9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4841483c0 con 0x7ff484155c60 2023-10-06T02:29:38.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:38.418+0000 7ff48ab9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4840c3bc0 con 0x7ff484155890 2023-10-06T02:29:38.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:38.418+0000 7ff48ab9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4840c3ea0 con 0x7ff48406de50 2023-10-06T02:29:38.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:38.418+0000 7ff48a39c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff484155c60 0x7ff4840c1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:38.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:38.418+0000 7ff48a39c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff484155c60 0x7ff4840c1080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55334/0 (socket says 172.21.15.31:55334) 2023-10-06T02:29:38.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:38.418+0000 7ff48a39c700 1 -- 172.21.15.31:0/1238662551 learned_addr learned my addr 172.21.15.31:0/1238662551 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:38.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:38.418+0000 7ff489b9b700 1 --2- 172.21.15.31:0/1238662551 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff48406de50 0x7ff484147e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:38.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:38.419+0000 7ff48939a700 1 --2- 172.21.15.31:0/1238662551 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff484155890 0x7ff4840c31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:38.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:38.419+0000 7ff48a39c700 1 -- 172.21.15.31:0/1238662551 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff48406de50 msgr2=0x7ff484147e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:38.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:38.419+0000 7ff48a39c700 1 --2- 172.21.15.31:0/1238662551 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff48406de50 0x7ff484147e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:38.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:38.419+0000 7ff48a39c700 1 -- 172.21.15.31:0/1238662551 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff484155890 msgr2=0x7ff4840c31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:38.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:38.419+0000 7ff48a39c700 1 --2- 172.21.15.31:0/1238662551 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff484155890 0x7ff4840c31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:38.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:38.419+0000 7ff48a39c700 1 -- 172.21.15.31:0/1238662551 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff480009420 con 0x7ff484155c60 2023-10-06T02:29:38.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:38.419+0000 7ff48939a700 1 --2- 172.21.15.31:0/1238662551 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff484155890 0x7ff4840c31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:29:38.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:38.419+0000 7ff48a39c700 1 --2- 172.21.15.31:0/1238662551 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff484155c60 0x7ff4840c1080 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7ff47c00cbc0 tx=0x7ff47c001960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:38.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:38.419+0000 7ff479ffb700 1 -- 172.21.15.31:0/1238662551 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff47c004dc0 con 0x7ff484155c60 2023-10-06T02:29:38.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:38.419+0000 7ff479ffb700 1 -- 172.21.15.31:0/1238662551 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff47c00a620 con 0x7ff484155c60 2023-10-06T02:29:38.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:38.419+0000 7ff479ffb700 1 -- 172.21.15.31:0/1238662551 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff47c00a9e0 con 0x7ff484155c60 2023-10-06T02:29:38.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:38.419+0000 7ff48ab9d700 1 -- 172.21.15.31:0/1238662551 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4840c4180 con 0x7ff484155c60 2023-10-06T02:29:38.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:38.419+0000 7ff48ab9d700 1 -- 172.21.15.31:0/1238662551 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4840c4740 con 0x7ff484155c60 2023-10-06T02:29:38.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:38.420+0000 7ff48ab9d700 1 -- 172.21.15.31:0/1238662551 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff48404c3c0 con 0x7ff484155c60 2023-10-06T02:29:38.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:38.421+0000 7ff479ffb700 1 -- 172.21.15.31:0/1238662551 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) v1 ==== 85097+0+0 (secure 0 0 0) 0x7ff47c00ad50 con 0x7ff484155c60 2023-10-06T02:29:38.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:38.422+0000 7ff479ffb700 1 --2- 172.21.15.31:0/1238662551 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff468067320 0x7ff4680697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:38.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:38.422+0000 7ff479ffb700 1 -- 172.21.15.31:0/1238662551 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff47c084d30 con 0x7ff484155c60 2023-10-06T02:29:38.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:38.423+0000 7ff489b9b700 1 --2- 172.21.15.31:0/1238662551 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff468067320 0x7ff4680697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:38.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:38.423+0000 7ff489b9b700 1 --2- 172.21.15.31:0/1238662551 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff468067320 0x7ff4680697d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff4800093f0 tx=0x7ff480017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:38.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:38.426+0000 7ff479ffb700 1 -- 172.21.15.31:0/1238662551 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff47c0518a0 con 0x7ff484155c60 2023-10-06T02:29:38.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:38.677+0000 7ff48ab9d700 1 -- 172.21.15.31:0/1238662551 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7ff48406f170 con 0x7ff484155c60 2023-10-06T02:29:39.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.192+0000 7ff479ffb700 1 -- 172.21.15.31:0/1238662551 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v80) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff47c054ec0 con 0x7ff484155c60 2023-10-06T02:29:39.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.195+0000 7ff48ab9d700 1 -- 172.21.15.31:0/1238662551 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff468067320 msgr2=0x7ff4680697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:39.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.195+0000 7ff48ab9d700 1 --2- 172.21.15.31:0/1238662551 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff468067320 0x7ff4680697d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff4800093f0 tx=0x7ff480017000 comp rx=0 tx=0).stop 2023-10-06T02:29:39.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.195+0000 7ff48ab9d700 1 -- 172.21.15.31:0/1238662551 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff484155c60 msgr2=0x7ff4840c1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:39.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.195+0000 7ff48ab9d700 1 --2- 172.21.15.31:0/1238662551 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff484155c60 0x7ff4840c1080 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7ff47c00cbc0 tx=0x7ff47c001960 comp rx=0 tx=0).stop 2023-10-06T02:29:39.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.195+0000 7ff48ab9d700 1 -- 172.21.15.31:0/1238662551 shutdown_connections 2023-10-06T02:29:39.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.195+0000 7ff48ab9d700 1 --2- 172.21.15.31:0/1238662551 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff484155c60 0x7ff4840c1080 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:39.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.195+0000 7ff48ab9d700 1 --2- 172.21.15.31:0/1238662551 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff468067320 0x7ff4680697d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:39.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.195+0000 7ff48ab9d700 1 --2- 172.21.15.31:0/1238662551 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff48406de50 0x7ff484147e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:39.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.195+0000 7ff48ab9d700 1 --2- 172.21.15.31:0/1238662551 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff484155890 0x7ff4840c31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:39.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.195+0000 7ff48ab9d700 1 -- 172.21.15.31:0/1238662551 >> 172.21.15.31:0/1238662551 conn(0x7ff48406d110 msgr2=0x7ff4840c21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:39.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.195+0000 7ff48ab9d700 1 -- 172.21.15.31:0/1238662551 shutdown_connections 2023-10-06T02:29:39.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.195+0000 7ff48ab9d700 1 -- 172.21.15.31:0/1238662551 wait complete. 2023-10-06T02:29:39.216 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:29:39.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.404+0000 7f388c9fe700 1 Processor -- start 2023-10-06T02:29:39.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.404+0000 7f388c9fe700 1 -- start start 2023-10-06T02:29:39.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.405+0000 7f388c9fe700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f388813fb20 0x7f388815a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:39.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.405+0000 7f388c9fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f388806aaa0 0x7f38880c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:39.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.405+0000 7f388c9fe700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f388815ae70 con 0x7f388813fef0 2023-10-06T02:29:39.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.405+0000 7f388c9fe700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f388806af60 con 0x7f388806aaa0 2023-10-06T02:29:39.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.405+0000 7f388c9fe700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f388806b240 con 0x7f388813fb20 2023-10-06T02:29:39.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.406+0000 7f388759e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f388806aaa0 0x7f38880c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:39.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.406+0000 7f388759e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f388806aaa0 0x7f38880c9d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55348/0 (socket says 172.21.15.31:55348) 2023-10-06T02:29:39.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.406+0000 7f388759e700 1 -- 172.21.15.31:0/4144781851 learned_addr learned my addr 172.21.15.31:0/4144781851 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:39.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.406+0000 7f3886d9d700 1 --2- 172.21.15.31:0/4144781851 >> v2:172.21.15.62:3300/0 conn(0x7f388813fb20 0x7f388815a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:39.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.406+0000 7f388759e700 1 -- 172.21.15.31:0/4144781851 >> v2:172.21.15.62:3300/0 conn(0x7f388813fb20 msgr2=0x7f388815a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:39.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.406+0000 7f388759e700 1 --2- 172.21.15.31:0/4144781851 >> v2:172.21.15.62:3300/0 conn(0x7f388813fb20 0x7f388815a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:39.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.406+0000 7f388759e700 1 -- 172.21.15.31:0/4144781851 >> v1:172.21.15.62:6789/0 conn(0x7f388813fef0 legacy=0x7f38880c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:39.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.406+0000 7f388759e700 1 -- 172.21.15.31:0/4144781851 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f388806b520 con 0x7f388806aaa0 2023-10-06T02:29:39.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.406+0000 7f3886d9d700 1 --2- 172.21.15.31:0/4144781851 >> v2:172.21.15.62:3300/0 conn(0x7f388813fb20 0x7f388815a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:29:39.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.407+0000 7f388759e700 1 --2- 172.21.15.31:0/4144781851 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f388806aaa0 0x7f38880c9d00 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f38880695b0 tx=0x7f3880009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a835f18c3a4599c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:39.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.407+0000 7f3885d9b700 1 -- 172.21.15.31:0/4144781851 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f388000e070 con 0x7f388806aaa0 2023-10-06T02:29:39.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.407+0000 7f3885d9b700 1 -- 172.21.15.31:0/4144781851 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f388000ba10 con 0x7f388806aaa0 2023-10-06T02:29:39.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.407+0000 7f3885d9b700 1 -- 172.21.15.31:0/4144781851 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38800092e0 con 0x7f388806aaa0 2023-10-06T02:29:39.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.407+0000 7f388c9fe700 1 -- 172.21.15.31:0/4144781851 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f388806aaa0 msgr2=0x7f38880c9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:39.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.407+0000 7f388c9fe700 1 --2- 172.21.15.31:0/4144781851 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f388806aaa0 0x7f38880c9d00 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f38880695b0 tx=0x7f3880009840 comp rx=0 tx=0).stop 2023-10-06T02:29:39.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.408+0000 7f388c9fe700 1 -- 172.21.15.31:0/4144781851 shutdown_connections 2023-10-06T02:29:39.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.408+0000 7f388c9fe700 1 --2- 172.21.15.31:0/4144781851 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f388806aaa0 0x7f38880c9d00 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:39.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.408+0000 7f388c9fe700 1 --2- 172.21.15.31:0/4144781851 >> v2:172.21.15.62:3300/0 conn(0x7f388813fb20 0x7f388815a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:39.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.408+0000 7f388c9fe700 1 -- 172.21.15.31:0/4144781851 >> 172.21.15.31:0/4144781851 conn(0x7f388806d170 msgr2=0x7f38880cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:39.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.408+0000 7f388c9fe700 1 -- 172.21.15.31:0/4144781851 shutdown_connections 2023-10-06T02:29:39.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.408+0000 7f388c9fe700 1 -- 172.21.15.31:0/4144781851 wait complete. 2023-10-06T02:29:39.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.409+0000 7f388c9fe700 1 Processor -- start 2023-10-06T02:29:39.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.409+0000 7f388c9fe700 1 -- start start 2023-10-06T02:29:39.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.410+0000 7f388c9fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f388806aaa0 0x7f3888155df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:39.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.410+0000 7f388c9fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f388813fb20 0x7f38880c23f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:39.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.410+0000 7f388c9fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f388813fef0 0x7f38880c2930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:39.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.410+0000 7f388c9fe700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38881598e0 con 0x7f388813fb20 2023-10-06T02:29:39.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.410+0000 7f388c9fe700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38880ab900 con 0x7f388813fef0 2023-10-06T02:29:39.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.410+0000 7f388c9fe700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3888159040 con 0x7f388806aaa0 2023-10-06T02:29:39.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.410+0000 7f388659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f388813fb20 0x7f38880c23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:39.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.410+0000 7f388659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f388813fb20 0x7f38880c23f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55354/0 (socket says 172.21.15.31:55354) 2023-10-06T02:29:39.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.410+0000 7f388659c700 1 -- 172.21.15.31:0/2233727330 learned_addr learned my addr 172.21.15.31:0/2233727330 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:39.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.411+0000 7f3886d9d700 1 --2- 172.21.15.31:0/2233727330 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f388806aaa0 0x7f3888155df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:39.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.411+0000 7f388759e700 1 --2- 172.21.15.31:0/2233727330 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f388813fef0 0x7f38880c2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:39.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.411+0000 7f388659c700 1 -- 172.21.15.31:0/2233727330 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f388806aaa0 msgr2=0x7f3888155df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:39.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.411+0000 7f388659c700 1 --2- 172.21.15.31:0/2233727330 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f388806aaa0 0x7f3888155df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:39.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.411+0000 7f388659c700 1 -- 172.21.15.31:0/2233727330 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f388813fef0 msgr2=0x7f38880c2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:39.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.411+0000 7f388659c700 1 --2- 172.21.15.31:0/2233727330 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f388813fef0 0x7f38880c2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:39.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.411+0000 7f388659c700 1 -- 172.21.15.31:0/2233727330 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3880009ae0 con 0x7f388813fb20 2023-10-06T02:29:39.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.411+0000 7f388759e700 1 --2- 172.21.15.31:0/2233727330 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f388813fef0 0x7f38880c2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:29:39.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.411+0000 7f388659c700 1 --2- 172.21.15.31:0/2233727330 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f388813fb20 0x7f38880c23f0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f387c00b560 tx=0x7f387c0090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:39.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.411+0000 7f386effd700 1 -- 172.21.15.31:0/2233727330 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f387c00f510 con 0x7f388813fb20 2023-10-06T02:29:39.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.411+0000 7f386effd700 1 -- 172.21.15.31:0/2233727330 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f387c00f670 con 0x7f388813fb20 2023-10-06T02:29:39.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.411+0000 7f386effd700 1 -- 172.21.15.31:0/2233727330 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f387c00fa30 con 0x7f388813fb20 2023-10-06T02:29:39.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.411+0000 7f388c9fe700 1 -- 172.21.15.31:0/2233727330 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3888159320 con 0x7f388813fb20 2023-10-06T02:29:39.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.412+0000 7f388c9fe700 1 -- 172.21.15.31:0/2233727330 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3888156330 con 0x7f388813fb20 2023-10-06T02:29:39.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.413+0000 7f388c9fe700 1 -- 172.21.15.31:0/2233727330 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38880aba90 con 0x7f388813fb20 2023-10-06T02:29:39.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.413+0000 7f386effd700 1 -- 172.21.15.31:0/2233727330 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f387c01a400 con 0x7f388813fb20 2023-10-06T02:29:39.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.415+0000 7f386effd700 1 --2- 172.21.15.31:0/2233727330 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f3874067320 0x7f38740697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:39.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.415+0000 7f386effd700 1 -- 172.21.15.31:0/2233727330 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f387c0846e0 con 0x7f388813fb20 2023-10-06T02:29:39.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.415+0000 7f3886d9d700 1 --2- 172.21.15.31:0/2233727330 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f3874067320 0x7f38740697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:39.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.416+0000 7f3886d9d700 1 --2- 172.21.15.31:0/2233727330 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f3874067320 0x7f38740697d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f387800adc0 tx=0x7f3878006f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:39.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.419+0000 7f386effd700 1 -- 172.21.15.31:0/2233727330 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f387c051290 con 0x7f388813fb20 2023-10-06T02:29:39.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.667+0000 7f388c9fe700 1 -- 172.21.15.31:0/2233727330 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f388804c3c0 con 0x7f388813fb20 2023-10-06T02:29:39.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.667+0000 7f386effd700 1 -- 172.21.15.31:0/2233727330 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+4590 (secure 0 0 0) 0x7f387c0548b0 con 0x7f388813fb20 2023-10-06T02:29:39.669 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:29:39.669 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":80,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6338,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/1906951645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":1906951645},{"type":"v1","addr":"172.21.15.62:6834","nonce":1906951645}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75},{"gid":6341,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3788411327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3788411327},{"type":"v1","addr":"172.21.15.62:6835","nonce":3788411327}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75},{"gid":7569,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6833/581939695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":581939695},{"type":"v1","addr":"172.21.15.31:6833","nonce":581939695}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:29:35.664920+0000","modified":"2023-10-06T02:29:39.189818+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7575},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7575":{"gid":7575,"name":"c","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/40412586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":40412586},{"type":"v1","addr":"172.21.15.31:6835","nonce":40412586}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-06T02:29:39.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.670+0000 7f388c9fe700 1 -- 172.21.15.31:0/2233727330 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f3874067320 msgr2=0x7f38740697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:39.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.670+0000 7f388c9fe700 1 --2- 172.21.15.31:0/2233727330 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f3874067320 0x7f38740697d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f387800adc0 tx=0x7f3878006f90 comp rx=0 tx=0).stop 2023-10-06T02:29:39.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.670+0000 7f388c9fe700 1 -- 172.21.15.31:0/2233727330 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f388813fb20 msgr2=0x7f38880c23f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:39.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.670+0000 7f388c9fe700 1 --2- 172.21.15.31:0/2233727330 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f388813fb20 0x7f38880c23f0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f387c00b560 tx=0x7f387c0090f0 comp rx=0 tx=0).stop 2023-10-06T02:29:39.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.670+0000 7f388c9fe700 1 -- 172.21.15.31:0/2233727330 shutdown_connections 2023-10-06T02:29:39.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.670+0000 7f388c9fe700 1 --2- 172.21.15.31:0/2233727330 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f388813fef0 0x7f38880c2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:39.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.670+0000 7f388c9fe700 1 --2- 172.21.15.31:0/2233727330 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f3874067320 0x7f38740697d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:39.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.670+0000 7f388c9fe700 1 --2- 172.21.15.31:0/2233727330 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f388806aaa0 0x7f3888155df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:39.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.670+0000 7f388c9fe700 1 --2- 172.21.15.31:0/2233727330 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f388813fb20 0x7f38880c23f0 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:39.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.670+0000 7f388c9fe700 1 -- 172.21.15.31:0/2233727330 >> 172.21.15.31:0/2233727330 conn(0x7f388806d170 msgr2=0x7f388806de50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:39.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.670+0000 7f388c9fe700 1 -- 172.21.15.31:0/2233727330 shutdown_connections 2023-10-06T02:29:39.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.670+0000 7f388c9fe700 1 -- 172.21.15.31:0/2233727330 wait complete. 2023-10-06T02:29:39.673 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 80 2023-10-06T02:29:39.689 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:29:39.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.873+0000 7fb2ddb9d700 1 Processor -- start 2023-10-06T02:29:39.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.873+0000 7fb2ddb9d700 1 -- start start 2023-10-06T02:29:39.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.875+0000 7fb2ddb9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb2d806e0a0 0x7fb2d80c6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:39.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.875+0000 7fb2ddb9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb2d806f3a0 0x7fb2d814fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:39.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.875+0000 7fb2ddb9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2d806a490 con 0x7fb2d806e470 2023-10-06T02:29:39.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.875+0000 7fb2ddb9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2d806b930 con 0x7fb2d806f3a0 2023-10-06T02:29:39.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.875+0000 7fb2ddb9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2d80c7490 con 0x7fb2d806e0a0 2023-10-06T02:29:39.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.875+0000 7fb2d7fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb2d806f3a0 0x7fb2d814fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:39.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.875+0000 7fb2d7fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb2d806f3a0 0x7fb2d814fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55368/0 (socket says 172.21.15.31:55368) 2023-10-06T02:29:39.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.875+0000 7fb2d7fff700 1 -- 172.21.15.31:0/3777698470 learned_addr learned my addr 172.21.15.31:0/3777698470 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:39.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.875+0000 7fb2dcb9b700 1 --2- 172.21.15.31:0/3777698470 >> v2:172.21.15.62:3300/0 conn(0x7fb2d806e0a0 0x7fb2d80c6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:39.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.875+0000 7fb2d7fff700 1 -- 172.21.15.31:0/3777698470 >> v2:172.21.15.62:3300/0 conn(0x7fb2d806e0a0 msgr2=0x7fb2d80c6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:39.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.875+0000 7fb2d7fff700 1 --2- 172.21.15.31:0/3777698470 >> v2:172.21.15.62:3300/0 conn(0x7fb2d806e0a0 0x7fb2d80c6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:39.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.875+0000 7fb2d7fff700 1 -- 172.21.15.31:0/3777698470 >> v1:172.21.15.62:6789/0 conn(0x7fb2d806e470 legacy=0x7fb2d8159740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:39.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.875+0000 7fb2d7fff700 1 -- 172.21.15.31:0/3777698470 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2d80c7770 con 0x7fb2d806f3a0 2023-10-06T02:29:39.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.876+0000 7fb2d7fff700 1 --2- 172.21.15.31:0/3777698470 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2d806f3a0 0x7fb2d814fc70 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fb2d8069230 tx=0x7fb2c4009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20f3df82d1df0852 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:39.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.876+0000 7fb2d77fe700 1 -- 172.21.15.31:0/3777698470 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c4005c10 con 0x7fb2d806f3a0 2023-10-06T02:29:39.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.876+0000 7fb2d77fe700 1 -- 172.21.15.31:0/3777698470 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2c4005d70 con 0x7fb2d806f3a0 2023-10-06T02:29:39.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.876+0000 7fb2d77fe700 1 -- 172.21.15.31:0/3777698470 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c400b840 con 0x7fb2d806f3a0 2023-10-06T02:29:39.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.876+0000 7fb2ddb9d700 1 -- 172.21.15.31:0/3777698470 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2d806f3a0 msgr2=0x7fb2d814fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:39.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.876+0000 7fb2ddb9d700 1 --2- 172.21.15.31:0/3777698470 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2d806f3a0 0x7fb2d814fc70 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fb2d8069230 tx=0x7fb2c4009220 comp rx=0 tx=0).stop 2023-10-06T02:29:39.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.877+0000 7fb2ddb9d700 1 -- 172.21.15.31:0/3777698470 shutdown_connections 2023-10-06T02:29:39.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.877+0000 7fb2ddb9d700 1 --2- 172.21.15.31:0/3777698470 >> v2:172.21.15.62:3300/0 conn(0x7fb2d806e0a0 0x7fb2d80c6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:39.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.877+0000 7fb2ddb9d700 1 --2- 172.21.15.31:0/3777698470 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2d806f3a0 0x7fb2d814fc70 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:39.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.877+0000 7fb2ddb9d700 1 -- 172.21.15.31:0/3777698470 >> 172.21.15.31:0/3777698470 conn(0x7fb2d806d170 msgr2=0x7fb2d80cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:39.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.877+0000 7fb2ddb9d700 1 -- 172.21.15.31:0/3777698470 shutdown_connections 2023-10-06T02:29:39.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.877+0000 7fb2ddb9d700 1 -- 172.21.15.31:0/3777698470 wait complete. 2023-10-06T02:29:39.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.878+0000 7fb2ddb9d700 1 Processor -- start 2023-10-06T02:29:39.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.878+0000 7fb2ddb9d700 1 -- start start 2023-10-06T02:29:39.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.879+0000 7fb2ddb9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb2d806e0a0 0x7fb2d814bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:39.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.879+0000 7fb2ddb9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2d806e470 0x7fb2d80c5f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:39.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.879+0000 7fb2ddb9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2d806f3a0 0x7fb2d80c0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:39.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.879+0000 7fb2ddb9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d80c4c30 con 0x7fb2d806e470 2023-10-06T02:29:39.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.879+0000 7fb2ddb9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d80a5980 con 0x7fb2d806f3a0 2023-10-06T02:29:39.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.879+0000 7fb2ddb9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d814c450 con 0x7fb2d806e0a0 2023-10-06T02:29:39.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.880+0000 7fb2d7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2d806e470 0x7fb2d80c5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:39.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.880+0000 7fb2d7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2d806e470 0x7fb2d80c5f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55370/0 (socket says 172.21.15.31:55370) 2023-10-06T02:29:39.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.880+0000 7fb2d7fff700 1 -- 172.21.15.31:0/528182702 learned_addr learned my addr 172.21.15.31:0/528182702 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:39.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.880+0000 7fb2dcb9b700 1 --2- 172.21.15.31:0/528182702 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb2d806e0a0 0x7fb2d814bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:39.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.880+0000 7fb2dd39c700 1 --2- 172.21.15.31:0/528182702 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2d806f3a0 0x7fb2d80c0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:39.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.880+0000 7fb2d7fff700 1 -- 172.21.15.31:0/528182702 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb2d806e0a0 msgr2=0x7fb2d814bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:39.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.880+0000 7fb2d7fff700 1 --2- 172.21.15.31:0/528182702 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb2d806e0a0 0x7fb2d814bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:39.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.880+0000 7fb2d7fff700 1 -- 172.21.15.31:0/528182702 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2d806f3a0 msgr2=0x7fb2d80c0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:39.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.880+0000 7fb2d7fff700 1 --2- 172.21.15.31:0/528182702 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2d806f3a0 0x7fb2d80c0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:39.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.880+0000 7fb2d7fff700 1 -- 172.21.15.31:0/528182702 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2c4009480 con 0x7fb2d806e470 2023-10-06T02:29:39.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.880+0000 7fb2d7fff700 1 --2- 172.21.15.31:0/528182702 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2d806e470 0x7fb2d80c5f80 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fb2c40030c0 tx=0x7fb2c4015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:39.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.881+0000 7fb2d4ff9700 1 -- 172.21.15.31:0/528182702 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c4011a10 con 0x7fb2d806e470 2023-10-06T02:29:39.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.881+0000 7fb2d4ff9700 1 -- 172.21.15.31:0/528182702 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2c4011b90 con 0x7fb2d806e470 2023-10-06T02:29:39.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.881+0000 7fb2d4ff9700 1 -- 172.21.15.31:0/528182702 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c401c420 con 0x7fb2d806e470 2023-10-06T02:29:39.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.881+0000 7fb2ddb9d700 1 -- 172.21.15.31:0/528182702 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2d80a5f00 con 0x7fb2d806e470 2023-10-06T02:29:39.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.881+0000 7fb2ddb9d700 1 -- 172.21.15.31:0/528182702 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2d80c5120 con 0x7fb2d806e470 2023-10-06T02:29:39.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.882+0000 7fb2ddb9d700 1 -- 172.21.15.31:0/528182702 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2d80a6090 con 0x7fb2d806e470 2023-10-06T02:29:39.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.882+0000 7fb2d4ff9700 1 -- 172.21.15.31:0/528182702 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fb2c401c810 con 0x7fb2d806e470 2023-10-06T02:29:39.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.883+0000 7fb2d4ff9700 1 --2- 172.21.15.31:0/528182702 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb2bc067320 0x7fb2bc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:39.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.883+0000 7fb2d4ff9700 1 -- 172.21.15.31:0/528182702 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb2c4086760 con 0x7fb2d806e470 2023-10-06T02:29:39.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.884+0000 7fb2dcb9b700 1 --2- 172.21.15.31:0/528182702 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb2bc067320 0x7fb2bc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:39.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.885+0000 7fb2dcb9b700 1 --2- 172.21.15.31:0/528182702 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb2bc067320 0x7fb2bc0697d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc006fd0 tx=0x7fb2cc008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:39.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:39.888+0000 7fb2d4ff9700 1 -- 172.21.15.31:0/528182702 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb2c4053390 con 0x7fb2d806e470 2023-10-06T02:29:40.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.092+0000 7fb2ddb9d700 1 -- 172.21.15.31:0/528182702 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb2d804c3c0 con 0x7fb2d806e470 2023-10-06T02:29:40.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.093+0000 7fb2d4ff9700 1 -- 172.21.15.31:0/528182702 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v120) v1 ==== 74+0+16832 (secure 0 0 0) 0x7fb2c40569b0 con 0x7fb2d806e470 2023-10-06T02:29:40.094 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:29:40.094 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":120,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:29:37.176295+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:29:33.769454+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-10-06T02:29:34.643236+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:29:40.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.095+0000 7fb2ddb9d700 1 -- 172.21.15.31:0/528182702 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb2bc067320 msgr2=0x7fb2bc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:40.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.095+0000 7fb2ddb9d700 1 --2- 172.21.15.31:0/528182702 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb2bc067320 0x7fb2bc0697d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc006fd0 tx=0x7fb2cc008040 comp rx=0 tx=0).stop 2023-10-06T02:29:40.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.095+0000 7fb2ddb9d700 1 -- 172.21.15.31:0/528182702 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2d806e470 msgr2=0x7fb2d80c5f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:40.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.095+0000 7fb2ddb9d700 1 --2- 172.21.15.31:0/528182702 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2d806e470 0x7fb2d80c5f80 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fb2c40030c0 tx=0x7fb2c4015000 comp rx=0 tx=0).stop 2023-10-06T02:29:40.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.095+0000 7fb2ddb9d700 1 -- 172.21.15.31:0/528182702 shutdown_connections 2023-10-06T02:29:40.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.095+0000 7fb2ddb9d700 1 --2- 172.21.15.31:0/528182702 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2d806f3a0 0x7fb2d80c0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:40.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.095+0000 7fb2ddb9d700 1 --2- 172.21.15.31:0/528182702 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb2bc067320 0x7fb2bc0697d0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:40.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.095+0000 7fb2ddb9d700 1 --2- 172.21.15.31:0/528182702 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb2d806e0a0 0x7fb2d814bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:40.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.095+0000 7fb2ddb9d700 1 --2- 172.21.15.31:0/528182702 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2d806e470 0x7fb2d80c5f80 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:40.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.095+0000 7fb2ddb9d700 1 -- 172.21.15.31:0/528182702 >> 172.21.15.31:0/528182702 conn(0x7fb2d806d170 msgr2=0x7fb2d814ecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:40.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.095+0000 7fb2ddb9d700 1 -- 172.21.15.31:0/528182702 shutdown_connections 2023-10-06T02:29:40.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.095+0000 7fb2ddb9d700 1 -- 172.21.15.31:0/528182702 wait complete. 2023-10-06T02:29:40.114 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T02:29:40.115 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T02:29:40.137 INFO:tasks.ceph.mds.c.smithi031.stderr:2023-10-06T02:29:40.136+0000 7fe67a7f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T02:29:40.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.302+0000 7f6c5f59e700 1 Processor -- start 2023-10-06T02:29:40.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.302+0000 7f6c5f59e700 1 -- start start 2023-10-06T02:29:40.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.303+0000 7f6c5f59e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6c6015de50 0x7f6c600cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:40.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.303+0000 7f6c5f59e700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f6c600cef40 0x7f6c6015c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:40.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.303+0000 7f6c5f59e700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c600cf3c0 con 0x7f6c60153c30 2023-10-06T02:29:40.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.303+0000 7f6c5f59e700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c6015afd0 con 0x7f6c6015de50 2023-10-06T02:29:40.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.303+0000 7f6c5f59e700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c6015b2b0 con 0x7f6c600cef40 2023-10-06T02:29:40.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.304+0000 7f6c5dd9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6c6015de50 0x7f6c600cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:40.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.304+0000 7f6c5dd9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6c6015de50 0x7f6c600cea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55384/0 (socket says 172.21.15.31:55384) 2023-10-06T02:29:40.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.304+0000 7f6c5dd9b700 1 -- 172.21.15.31:0/2719402534 learned_addr learned my addr 172.21.15.31:0/2719402534 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:40.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.304+0000 7f6c5ed9d700 1 --2- 172.21.15.31:0/2719402534 >> v2:172.21.15.62:3300/0 conn(0x7f6c600cef40 0x7f6c6015c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:40.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.304+0000 7f6c5dd9b700 1 -- 172.21.15.31:0/2719402534 >> v2:172.21.15.62:3300/0 conn(0x7f6c600cef40 msgr2=0x7f6c6015c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:40.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.304+0000 7f6c5dd9b700 1 --2- 172.21.15.31:0/2719402534 >> v2:172.21.15.62:3300/0 conn(0x7f6c600cef40 0x7f6c6015c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:40.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.304+0000 7f6c5dd9b700 1 -- 172.21.15.31:0/2719402534 >> v1:172.21.15.62:6789/0 conn(0x7f6c60153c30 legacy=0x7f6c6015a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:40.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.304+0000 7f6c5dd9b700 1 -- 172.21.15.31:0/2719402534 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c6015b590 con 0x7f6c6015de50 2023-10-06T02:29:40.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.305+0000 7f6c5dd9b700 1 --2- 172.21.15.31:0/2719402534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c6015de50 0x7f6c600cea00 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f6c600cdab0 tx=0x7f6c54009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cf0f27449b822ed server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:40.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.305+0000 7f6c5d59a700 1 -- 172.21.15.31:0/2719402534 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c54009e70 con 0x7f6c6015de50 2023-10-06T02:29:40.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.305+0000 7f6c5d59a700 1 -- 172.21.15.31:0/2719402534 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c5400b860 con 0x7f6c6015de50 2023-10-06T02:29:40.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.305+0000 7f6c5d59a700 1 -- 172.21.15.31:0/2719402534 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c5400bc00 con 0x7f6c6015de50 2023-10-06T02:29:40.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.305+0000 7f6c5f59e700 1 -- 172.21.15.31:0/2719402534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c6015de50 msgr2=0x7f6c600cea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:40.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.305+0000 7f6c5f59e700 1 --2- 172.21.15.31:0/2719402534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c6015de50 0x7f6c600cea00 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f6c600cdab0 tx=0x7f6c54009250 comp rx=0 tx=0).stop 2023-10-06T02:29:40.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.305+0000 7f6c5f59e700 1 -- 172.21.15.31:0/2719402534 shutdown_connections 2023-10-06T02:29:40.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.305+0000 7f6c5f59e700 1 --2- 172.21.15.31:0/2719402534 >> v2:172.21.15.62:3300/0 conn(0x7f6c600cef40 0x7f6c6015c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:40.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.305+0000 7f6c5f59e700 1 --2- 172.21.15.31:0/2719402534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c6015de50 0x7f6c600cea00 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:40.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.305+0000 7f6c5f59e700 1 -- 172.21.15.31:0/2719402534 >> 172.21.15.31:0/2719402534 conn(0x7f6c600703a0 msgr2=0x7f6c600707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:40.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.306+0000 7f6c5f59e700 1 -- 172.21.15.31:0/2719402534 shutdown_connections 2023-10-06T02:29:40.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.306+0000 7f6c5f59e700 1 -- 172.21.15.31:0/2719402534 wait complete. 2023-10-06T02:29:40.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.307+0000 7f6c5f59e700 1 Processor -- start 2023-10-06T02:29:40.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.307+0000 7f6c5f59e700 1 -- start start 2023-10-06T02:29:40.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.308+0000 7f6c5f59e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c600cef40 0x7f6c600cba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:40.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.308+0000 7f6c5f59e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6c60153c30 0x7f6c600c75e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:40.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.308+0000 7f6c5f59e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c6015de50 0x7f6c6014ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:40.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.308+0000 7f6c5f59e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c600ca4d0 con 0x7f6c6015de50 2023-10-06T02:29:40.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.308+0000 7f6c5f59e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c600ccc20 con 0x7f6c600cef40 2023-10-06T02:29:40.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.308+0000 7f6c5f59e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c600cc010 con 0x7f6c60153c30 2023-10-06T02:29:40.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.308+0000 7f6c5ed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c6015de50 0x7f6c6014ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:40.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.308+0000 7f6c5ed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c6015de50 0x7f6c6014ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55392/0 (socket says 172.21.15.31:55392) 2023-10-06T02:29:40.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.308+0000 7f6c5ed9d700 1 -- 172.21.15.31:0/1222779892 learned_addr learned my addr 172.21.15.31:0/1222779892 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:40.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.309+0000 7f6c5ed9d700 1 -- 172.21.15.31:0/1222779892 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6c60153c30 msgr2=0x7f6c600c75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:40.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.309+0000 7f6c5ed9d700 1 --2- 172.21.15.31:0/1222779892 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6c60153c30 0x7f6c600c75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:40.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.309+0000 7f6c5e59c700 1 --2- 172.21.15.31:0/1222779892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c600cef40 0x7f6c600cba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:40.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.309+0000 7f6c5ed9d700 1 -- 172.21.15.31:0/1222779892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c600cef40 msgr2=0x7f6c600cba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:40.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.309+0000 7f6c5ed9d700 1 --2- 172.21.15.31:0/1222779892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c600cef40 0x7f6c600cba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:40.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.309+0000 7f6c5ed9d700 1 -- 172.21.15.31:0/1222779892 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c540094f0 con 0x7f6c6015de50 2023-10-06T02:29:40.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.309+0000 7f6c5e59c700 1 --2- 172.21.15.31:0/1222779892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c600cef40 0x7f6c600cba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:29:40.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.309+0000 7f6c5ed9d700 1 --2- 172.21.15.31:0/1222779892 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c6015de50 0x7f6c6014ffd0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f6c58009df0 tx=0x7f6c5800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:40.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.309+0000 7f6c4e7fc700 1 -- 172.21.15.31:0/1222779892 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c5800cb50 con 0x7f6c6015de50 2023-10-06T02:29:40.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.309+0000 7f6c4e7fc700 1 -- 172.21.15.31:0/1222779892 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c5800f040 con 0x7f6c6015de50 2023-10-06T02:29:40.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.309+0000 7f6c4e7fc700 1 -- 172.21.15.31:0/1222779892 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c58014480 con 0x7f6c6015de50 2023-10-06T02:29:40.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.309+0000 7f6c5f59e700 1 -- 172.21.15.31:0/1222779892 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c600cc2f0 con 0x7f6c6015de50 2023-10-06T02:29:40.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.309+0000 7f6c5f59e700 1 -- 172.21.15.31:0/1222779892 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c600cc8b0 con 0x7f6c6015de50 2023-10-06T02:29:40.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.310+0000 7f6c5f59e700 1 -- 172.21.15.31:0/1222779892 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c600ccdb0 con 0x7f6c6015de50 2023-10-06T02:29:40.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.311+0000 7f6c4e7fc700 1 -- 172.21.15.31:0/1222779892 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f6c58014830 con 0x7f6c6015de50 2023-10-06T02:29:40.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.312+0000 7f6c4e7fc700 1 --2- 172.21.15.31:0/1222779892 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6c40067320 0x7f6c400697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:40.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.312+0000 7f6c4e7fc700 1 -- 172.21.15.31:0/1222779892 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6c580858c0 con 0x7f6c6015de50 2023-10-06T02:29:40.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.313+0000 7f6c5e59c700 1 --2- 172.21.15.31:0/1222779892 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6c40067320 0x7f6c400697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:40.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.313+0000 7f6c5e59c700 1 --2- 172.21.15.31:0/1222779892 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6c40067320 0x7f6c400697d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6c500017a0 tx=0x7f6c50009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:40.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.316+0000 7f6c4e7fc700 1 -- 172.21.15.31:0/1222779892 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6c58052300 con 0x7f6c6015de50 2023-10-06T02:29:40.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.516+0000 7f6c5f59e700 1 -- 172.21.15.31:0/1222779892 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6c6004c3c0 con 0x7f6c40067320 2023-10-06T02:29:40.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.526+0000 7f6c4e7fc700 1 -- 172.21.15.31:0/1222779892 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311338 (secure 0 0 0) 0x7f6c6004c3c0 con 0x7f6c40067320 2023-10-06T02:29:40.533 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:29:40.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.530+0000 7f6c5f59e700 1 -- 172.21.15.31:0/1222779892 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6c40067320 msgr2=0x7f6c400697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:40.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.530+0000 7f6c5f59e700 1 --2- 172.21.15.31:0/1222779892 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6c40067320 0x7f6c400697d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6c500017a0 tx=0x7f6c50009040 comp rx=0 tx=0).stop 2023-10-06T02:29:40.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.531+0000 7f6c5f59e700 1 -- 172.21.15.31:0/1222779892 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c6015de50 msgr2=0x7f6c6014ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:40.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.531+0000 7f6c5f59e700 1 --2- 172.21.15.31:0/1222779892 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c6015de50 0x7f6c6014ffd0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f6c58009df0 tx=0x7f6c5800b040 comp rx=0 tx=0).stop 2023-10-06T02:29:40.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.531+0000 7f6c5f59e700 1 -- 172.21.15.31:0/1222779892 shutdown_connections 2023-10-06T02:29:40.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.531+0000 7f6c5f59e700 1 --2- 172.21.15.31:0/1222779892 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c6015de50 0x7f6c6014ffd0 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:40.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.531+0000 7f6c5f59e700 1 --2- 172.21.15.31:0/1222779892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c600cef40 0x7f6c600cba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:40.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.531+0000 7f6c5f59e700 1 --2- 172.21.15.31:0/1222779892 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6c40067320 0x7f6c400697d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:40.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.531+0000 7f6c5f59e700 1 --2- 172.21.15.31:0/1222779892 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6c60153c30 0x7f6c600c75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:40.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.531+0000 7f6c5f59e700 1 -- 172.21.15.31:0/1222779892 >> 172.21.15.31:0/1222779892 conn(0x7f6c600703a0 msgr2=0x7f6c600c5420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:40.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.531+0000 7f6c5f59e700 1 -- 172.21.15.31:0/1222779892 shutdown_connections 2023-10-06T02:29:40.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.531+0000 7f6c5f59e700 1 -- 172.21.15.31:0/1222779892 wait complete. 2023-10-06T02:29:40.536 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-10-06T02:29:40.550 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":109,"stamp":"2023-10-06T02:29:40.482535+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":224,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":167,"ondisk_log_size":167,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":169,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2424008,"kb_used_data":12936,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752550712,"statfs":{"total":773094113280,"available":770611929088,"internally_reserved":0,"allocated":13246464,"data_stored":6756972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001634"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.181634+0000","last_change":"2023-10-06T02:29:36.180448+0000","last_active":"2023-10-06T02:29:36.181634+0000","last_peered":"2023-10-06T02:29:36.181634+0000","last_clean":"2023-10-06T02:29:36.181634+0000","last_became_active":"2023-10-06T02:29:36.180182+0000","last_became_peered":"2023-10-06T02:29:36.180182+0000","last_unstale":"2023-10-06T02:29:36.181634+0000","last_undegraded":"2023-10-06T02:29:36.181634+0000","last_fullsized":"2023-10-06T02:29:36.181634+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:13:33.504425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.191454+0000","last_change":"2023-10-06T02:29:36.185203+0000","last_active":"2023-10-06T02:29:36.191454+0000","last_peered":"2023-10-06T02:29:36.191454+0000","last_clean":"2023-10-06T02:29:36.191454+0000","last_became_active":"2023-10-06T02:29:36.184889+0000","last_became_peered":"2023-10-06T02:29:36.184889+0000","last_unstale":"2023-10-06T02:29:36.191454+0000","last_undegraded":"2023-10-06T02:29:36.191454+0000","last_fullsized":"2023-10-06T02:29:36.191454+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:10:07.892520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.196410+0000","last_change":"2023-10-06T02:29:36.189362+0000","last_active":"2023-10-06T02:29:36.196410+0000","last_peered":"2023-10-06T02:29:36.196410+0000","last_clean":"2023-10-06T02:29:36.196410+0000","last_became_active":"2023-10-06T02:29:36.188505+0000","last_became_peered":"2023-10-06T02:29:36.188505+0000","last_unstale":"2023-10-06T02:29:36.196410+0000","last_undegraded":"2023-10-06T02:29:36.196410+0000","last_fullsized":"2023-10-06T02:29:36.196410+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:14:52.577434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.191521+0000","last_change":"2023-10-06T02:29:36.186503+0000","last_active":"2023-10-06T02:29:36.191521+0000","last_peered":"2023-10-06T02:29:36.191521+0000","last_clean":"2023-10-06T02:29:36.191521+0000","last_became_active":"2023-10-06T02:29:36.186137+0000","last_became_peered":"2023-10-06T02:29:36.186137+0000","last_unstale":"2023-10-06T02:29:36.191521+0000","last_undegraded":"2023-10-06T02:29:36.191521+0000","last_fullsized":"2023-10-06T02:29:36.191521+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:28:27.243954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.197725+0000","last_change":"2023-10-06T02:29:36.189063+0000","last_active":"2023-10-06T02:29:36.197725+0000","last_peered":"2023-10-06T02:29:36.197725+0000","last_clean":"2023-10-06T02:29:36.197725+0000","last_became_active":"2023-10-06T02:29:36.188758+0000","last_became_peered":"2023-10-06T02:29:36.188758+0000","last_unstale":"2023-10-06T02:29:36.197725+0000","last_undegraded":"2023-10-06T02:29:36.197725+0000","last_fullsized":"2023-10-06T02:29:36.197725+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:14:59.301511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.202542+0000","last_change":"2023-10-06T02:29:36.201440+0000","last_active":"2023-10-06T02:29:36.202542+0000","last_peered":"2023-10-06T02:29:36.202542+0000","last_clean":"2023-10-06T02:29:36.202542+0000","last_became_active":"2023-10-06T02:29:36.199748+0000","last_became_peered":"2023-10-06T02:29:36.199748+0000","last_unstale":"2023-10-06T02:29:36.202542+0000","last_undegraded":"2023-10-06T02:29:36.202542+0000","last_fullsized":"2023-10-06T02:29:36.202542+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:28:27.243954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.205145+0000","last_change":"2023-10-06T02:29:36.201694+0000","last_active":"2023-10-06T02:29:36.205145+0000","last_peered":"2023-10-06T02:29:36.205145+0000","last_clean":"2023-10-06T02:29:36.205145+0000","last_became_active":"2023-10-06T02:29:36.199613+0000","last_became_peered":"2023-10-06T02:29:36.199613+0000","last_unstale":"2023-10-06T02:29:36.205145+0000","last_undegraded":"2023-10-06T02:29:36.205145+0000","last_fullsized":"2023-10-06T02:29:36.205145+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:11:52.582273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.199056+0000","last_change":"2023-10-06T02:29:36.198176+0000","last_active":"2023-10-06T02:29:36.199056+0000","last_peered":"2023-10-06T02:29:36.199056+0000","last_clean":"2023-10-06T02:29:36.199056+0000","last_became_active":"2023-10-06T02:29:36.197788+0000","last_became_peered":"2023-10-06T02:29:36.197788+0000","last_unstale":"2023-10-06T02:29:36.199056+0000","last_undegraded":"2023-10-06T02:29:36.199056+0000","last_fullsized":"2023-10-06T02:29:36.199056+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:02:43.415532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.196802+0000","last_change":"2023-10-06T02:29:36.195172+0000","last_active":"2023-10-06T02:29:36.196802+0000","last_peered":"2023-10-06T02:29:36.196802+0000","last_clean":"2023-10-06T02:29:36.196802+0000","last_became_active":"2023-10-06T02:29:36.194899+0000","last_became_peered":"2023-10-06T02:29:36.194899+0000","last_unstale":"2023-10-06T02:29:36.196802+0000","last_undegraded":"2023-10-06T02:29:36.196802+0000","last_fullsized":"2023-10-06T02:29:36.196802+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:19:49.845748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.189203+0000","last_change":"2023-10-06T02:29:36.185097+0000","last_active":"2023-10-06T02:29:36.189203+0000","last_peered":"2023-10-06T02:29:36.189203+0000","last_clean":"2023-10-06T02:29:36.189203+0000","last_became_active":"2023-10-06T02:29:36.184831+0000","last_became_peered":"2023-10-06T02:29:36.184831+0000","last_unstale":"2023-10-06T02:29:36.189203+0000","last_undegraded":"2023-10-06T02:29:36.189203+0000","last_fullsized":"2023-10-06T02:29:36.189203+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:50:11.198660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.196296+0000","last_change":"2023-10-06T02:29:36.192247+0000","last_active":"2023-10-06T02:29:36.196296+0000","last_peered":"2023-10-06T02:29:36.196296+0000","last_clean":"2023-10-06T02:29:36.196296+0000","last_became_active":"2023-10-06T02:29:36.191926+0000","last_became_peered":"2023-10-06T02:29:36.191926+0000","last_unstale":"2023-10-06T02:29:36.196296+0000","last_undegraded":"2023-10-06T02:29:36.196296+0000","last_fullsized":"2023-10-06T02:29:36.196296+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:23:38.004436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.181311+0000","last_change":"2023-10-06T02:29:36.179316+0000","last_active":"2023-10-06T02:29:36.181311+0000","last_peered":"2023-10-06T02:29:36.181311+0000","last_clean":"2023-10-06T02:29:36.181311+0000","last_became_active":"2023-10-06T02:29:36.179058+0000","last_became_peered":"2023-10-06T02:29:36.179058+0000","last_unstale":"2023-10-06T02:29:36.181311+0000","last_undegraded":"2023-10-06T02:29:36.181311+0000","last_fullsized":"2023-10-06T02:29:36.181311+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:46:32.857433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.202541+0000","last_change":"2023-10-06T02:29:36.201465+0000","last_active":"2023-10-06T02:29:36.202541+0000","last_peered":"2023-10-06T02:29:36.202541+0000","last_clean":"2023-10-06T02:29:36.202541+0000","last_became_active":"2023-10-06T02:29:36.201182+0000","last_became_peered":"2023-10-06T02:29:36.201182+0000","last_unstale":"2023-10-06T02:29:36.202541+0000","last_undegraded":"2023-10-06T02:29:36.202541+0000","last_fullsized":"2023-10-06T02:29:36.202541+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:05:07.231083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.194786+0000","last_change":"2023-10-06T02:29:36.192619+0000","last_active":"2023-10-06T02:29:36.194786+0000","last_peered":"2023-10-06T02:29:36.194786+0000","last_clean":"2023-10-06T02:29:36.194786+0000","last_became_active":"2023-10-06T02:29:36.192345+0000","last_became_peered":"2023-10-06T02:29:36.192345+0000","last_unstale":"2023-10-06T02:29:36.194786+0000","last_undegraded":"2023-10-06T02:29:36.194786+0000","last_fullsized":"2023-10-06T02:29:36.194786+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:48:20.630360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.197308+0000","last_change":"2023-10-06T02:29:36.193299+0000","last_active":"2023-10-06T02:29:36.197308+0000","last_peered":"2023-10-06T02:29:36.197308+0000","last_clean":"2023-10-06T02:29:36.197308+0000","last_became_active":"2023-10-06T02:29:36.192936+0000","last_became_peered":"2023-10-06T02:29:36.192936+0000","last_unstale":"2023-10-06T02:29:36.197308+0000","last_undegraded":"2023-10-06T02:29:36.197308+0000","last_fullsized":"2023-10-06T02:29:36.197308+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:31:14.878470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.193227+0000","last_change":"2023-10-06T02:29:36.189557+0000","last_active":"2023-10-06T02:29:36.193227+0000","last_peered":"2023-10-06T02:29:36.193227+0000","last_clean":"2023-10-06T02:29:36.193227+0000","last_became_active":"2023-10-06T02:29:36.189276+0000","last_became_peered":"2023-10-06T02:29:36.189276+0000","last_unstale":"2023-10-06T02:29:36.193227+0000","last_undegraded":"2023-10-06T02:29:36.193227+0000","last_fullsized":"2023-10-06T02:29:36.193227+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:50:05.271603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.193829+0000","last_change":"2023-10-06T02:29:36.190332+0000","last_active":"2023-10-06T02:29:36.193829+0000","last_peered":"2023-10-06T02:29:36.193829+0000","last_clean":"2023-10-06T02:29:36.193829+0000","last_became_active":"2023-10-06T02:29:36.189914+0000","last_became_peered":"2023-10-06T02:29:36.189914+0000","last_unstale":"2023-10-06T02:29:36.193829+0000","last_undegraded":"2023-10-06T02:29:36.193829+0000","last_fullsized":"2023-10-06T02:29:36.193829+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:36:47.544763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.188447+0000","last_change":"2023-10-06T02:29:36.185191+0000","last_active":"2023-10-06T02:29:36.188447+0000","last_peered":"2023-10-06T02:29:36.188447+0000","last_clean":"2023-10-06T02:29:36.188447+0000","last_became_active":"2023-10-06T02:29:36.184910+0000","last_became_peered":"2023-10-06T02:29:36.184910+0000","last_unstale":"2023-10-06T02:29:36.188447+0000","last_undegraded":"2023-10-06T02:29:36.188447+0000","last_fullsized":"2023-10-06T02:29:36.188447+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:40:48.871334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.204007+0000","last_change":"2023-10-06T02:29:36.193886+0000","last_active":"2023-10-06T02:29:36.204007+0000","last_peered":"2023-10-06T02:29:36.204007+0000","last_clean":"2023-10-06T02:29:36.204007+0000","last_became_active":"2023-10-06T02:29:36.191083+0000","last_became_peered":"2023-10-06T02:29:36.191083+0000","last_unstale":"2023-10-06T02:29:36.204007+0000","last_undegraded":"2023-10-06T02:29:36.204007+0000","last_fullsized":"2023-10-06T02:29:36.204007+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:33:36.484989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.200470+0000","last_change":"2023-10-06T02:29:36.198198+0000","last_active":"2023-10-06T02:29:36.200470+0000","last_peered":"2023-10-06T02:29:36.200470+0000","last_clean":"2023-10-06T02:29:36.200470+0000","last_became_active":"2023-10-06T02:29:36.197950+0000","last_became_peered":"2023-10-06T02:29:36.197950+0000","last_unstale":"2023-10-06T02:29:36.200470+0000","last_undegraded":"2023-10-06T02:29:36.200470+0000","last_fullsized":"2023-10-06T02:29:36.200470+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:34:31.991692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.196349+0000","last_change":"2023-10-06T02:29:36.190553+0000","last_active":"2023-10-06T02:29:36.196349+0000","last_peered":"2023-10-06T02:29:36.196349+0000","last_clean":"2023-10-06T02:29:36.196349+0000","last_became_active":"2023-10-06T02:29:36.190253+0000","last_became_peered":"2023-10-06T02:29:36.190253+0000","last_unstale":"2023-10-06T02:29:36.196349+0000","last_undegraded":"2023-10-06T02:29:36.196349+0000","last_fullsized":"2023-10-06T02:29:36.196349+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:05:07.231083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.180983+0000","last_change":"2023-10-06T02:29:36.176630+0000","last_active":"2023-10-06T02:29:36.180983+0000","last_peered":"2023-10-06T02:29:36.180983+0000","last_clean":"2023-10-06T02:29:36.180983+0000","last_became_active":"2023-10-06T02:29:36.176301+0000","last_became_peered":"2023-10-06T02:29:36.176301+0000","last_unstale":"2023-10-06T02:29:36.180983+0000","last_undegraded":"2023-10-06T02:29:36.180983+0000","last_fullsized":"2023-10-06T02:29:36.180983+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:38:29.470165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.197329+0000","last_change":"2023-10-06T02:29:36.192712+0000","last_active":"2023-10-06T02:29:36.197329+0000","last_peered":"2023-10-06T02:29:36.197329+0000","last_clean":"2023-10-06T02:29:36.197329+0000","last_became_active":"2023-10-06T02:29:36.192439+0000","last_became_peered":"2023-10-06T02:29:36.192439+0000","last_unstale":"2023-10-06T02:29:36.197329+0000","last_undegraded":"2023-10-06T02:29:36.197329+0000","last_fullsized":"2023-10-06T02:29:36.197329+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:50:05.271603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.193780+0000","last_change":"2023-10-06T02:29:36.192232+0000","last_active":"2023-10-06T02:29:36.193780+0000","last_peered":"2023-10-06T02:29:36.193780+0000","last_clean":"2023-10-06T02:29:36.193780+0000","last_became_active":"2023-10-06T02:29:36.191969+0000","last_became_peered":"2023-10-06T02:29:36.191969+0000","last_unstale":"2023-10-06T02:29:36.193780+0000","last_undegraded":"2023-10-06T02:29:36.193780+0000","last_fullsized":"2023-10-06T02:29:36.193780+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:01:04.373806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.198281+0000","last_change":"2023-10-06T02:29:36.189286+0000","last_active":"2023-10-06T02:29:36.198281+0000","last_peered":"2023-10-06T02:29:36.198281+0000","last_clean":"2023-10-06T02:29:36.198281+0000","last_became_active":"2023-10-06T02:29:36.188437+0000","last_became_peered":"2023-10-06T02:29:36.188437+0000","last_unstale":"2023-10-06T02:29:36.198281+0000","last_undegraded":"2023-10-06T02:29:36.198281+0000","last_fullsized":"2023-10-06T02:29:36.198281+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:21:14.154163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.203339+0000","last_change":"2023-10-06T02:29:36.196695+0000","last_active":"2023-10-06T02:29:36.203339+0000","last_peered":"2023-10-06T02:29:36.203339+0000","last_clean":"2023-10-06T02:29:36.203339+0000","last_became_active":"2023-10-06T02:29:36.196337+0000","last_became_peered":"2023-10-06T02:29:36.196337+0000","last_unstale":"2023-10-06T02:29:36.203339+0000","last_undegraded":"2023-10-06T02:29:36.203339+0000","last_fullsized":"2023-10-06T02:29:36.203339+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:18:19.492112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.205081+0000","last_change":"2023-10-06T02:29:36.202975+0000","last_active":"2023-10-06T02:29:36.205081+0000","last_peered":"2023-10-06T02:29:36.205081+0000","last_clean":"2023-10-06T02:29:36.205081+0000","last_became_active":"2023-10-06T02:29:36.201854+0000","last_became_peered":"2023-10-06T02:29:36.201854+0000","last_unstale":"2023-10-06T02:29:36.205081+0000","last_undegraded":"2023-10-06T02:29:36.205081+0000","last_fullsized":"2023-10-06T02:29:36.205081+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:43:21.797680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.187675+0000","last_change":"2023-10-06T02:29:36.185022+0000","last_active":"2023-10-06T02:29:36.187675+0000","last_peered":"2023-10-06T02:29:36.187675+0000","last_clean":"2023-10-06T02:29:36.187675+0000","last_became_active":"2023-10-06T02:29:36.184733+0000","last_became_peered":"2023-10-06T02:29:36.184733+0000","last_unstale":"2023-10-06T02:29:36.187675+0000","last_undegraded":"2023-10-06T02:29:36.187675+0000","last_fullsized":"2023-10-06T02:29:36.187675+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:28.316081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.196292+0000","last_change":"2023-10-06T02:29:36.192870+0000","last_active":"2023-10-06T02:29:36.196292+0000","last_peered":"2023-10-06T02:29:36.196292+0000","last_clean":"2023-10-06T02:29:36.196292+0000","last_became_active":"2023-10-06T02:29:36.192604+0000","last_became_peered":"2023-10-06T02:29:36.192604+0000","last_unstale":"2023-10-06T02:29:36.196292+0000","last_undegraded":"2023-10-06T02:29:36.196292+0000","last_fullsized":"2023-10-06T02:29:36.196292+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:45:06.604436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.195036+0000","last_change":"2023-10-06T02:29:36.186963+0000","last_active":"2023-10-06T02:29:36.195036+0000","last_peered":"2023-10-06T02:29:36.195036+0000","last_clean":"2023-10-06T02:29:36.195036+0000","last_became_active":"2023-10-06T02:29:36.186689+0000","last_became_peered":"2023-10-06T02:29:36.186689+0000","last_unstale":"2023-10-06T02:29:36.195036+0000","last_undegraded":"2023-10-06T02:29:36.195036+0000","last_fullsized":"2023-10-06T02:29:36.195036+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:21:34.832535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.197437+0000","last_change":"2023-10-06T02:29:36.189860+0000","last_active":"2023-10-06T02:29:36.197437+0000","last_peered":"2023-10-06T02:29:36.197437+0000","last_clean":"2023-10-06T02:29:36.197437+0000","last_became_active":"2023-10-06T02:29:36.189431+0000","last_became_peered":"2023-10-06T02:29:36.189431+0000","last_unstale":"2023-10-06T02:29:36.197437+0000","last_undegraded":"2023-10-06T02:29:36.197437+0000","last_fullsized":"2023-10-06T02:29:36.197437+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:44:02.050446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.195498+0000","last_change":"2023-10-06T02:29:36.192171+0000","last_active":"2023-10-06T02:29:36.195498+0000","last_peered":"2023-10-06T02:29:36.195498+0000","last_clean":"2023-10-06T02:29:36.195498+0000","last_became_active":"2023-10-06T02:29:36.191906+0000","last_became_peered":"2023-10-06T02:29:36.191906+0000","last_unstale":"2023-10-06T02:29:36.195498+0000","last_undegraded":"2023-10-06T02:29:36.195498+0000","last_fullsized":"2023-10-06T02:29:36.195498+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:26:51.911921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.198283+0000","last_change":"2023-10-06T02:29:36.192293+0000","last_active":"2023-10-06T02:29:36.198283+0000","last_peered":"2023-10-06T02:29:36.198283+0000","last_clean":"2023-10-06T02:29:36.198283+0000","last_became_active":"2023-10-06T02:29:36.192021+0000","last_became_peered":"2023-10-06T02:29:36.192021+0000","last_unstale":"2023-10-06T02:29:36.198283+0000","last_undegraded":"2023-10-06T02:29:36.198283+0000","last_fullsized":"2023-10-06T02:29:36.198283+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:54:48.281815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1f","version":"119'4","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.203852+0000","last_change":"2023-10-06T02:29:35.187018+0000","last_active":"2023-10-06T02:29:36.203852+0000","last_peered":"2023-10-06T02:29:36.203852+0000","last_clean":"2023-10-06T02:29:36.203852+0000","last_became_active":"2023-10-06T02:29:35.186239+0000","last_became_peered":"2023-10-06T02:29:35.186239+0000","last_unstale":"2023-10-06T02:29:36.203852+0000","last_undegraded":"2023-10-06T02:29:36.203852+0000","last_fullsized":"2023-10-06T02:29:36.203852+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_clean_scrub_stamp":"2023-10-06T02:29:34.151749+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:12:51.337692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.180620+0000","last_change":"2023-10-06T02:29:36.175948+0000","last_active":"2023-10-06T02:29:36.180620+0000","last_peered":"2023-10-06T02:29:36.180620+0000","last_clean":"2023-10-06T02:29:36.180620+0000","last_became_active":"2023-10-06T02:29:36.175662+0000","last_became_peered":"2023-10-06T02:29:36.175662+0000","last_unstale":"2023-10-06T02:29:36.180620+0000","last_undegraded":"2023-10-06T02:29:36.180620+0000","last_fullsized":"2023-10-06T02:29:36.180620+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:55:24.441198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1e","version":"119'1","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.195630+0000","last_change":"2023-10-06T02:29:35.186934+0000","last_active":"2023-10-06T02:29:36.195630+0000","last_peered":"2023-10-06T02:29:36.195630+0000","last_clean":"2023-10-06T02:29:36.195630+0000","last_became_active":"2023-10-06T02:29:35.186673+0000","last_became_peered":"2023-10-06T02:29:35.186673+0000","last_unstale":"2023-10-06T02:29:36.195630+0000","last_undegraded":"2023-10-06T02:29:36.195630+0000","last_fullsized":"2023-10-06T02:29:36.195630+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_clean_scrub_stamp":"2023-10-06T02:29:34.151749+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:02:42.408863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.196247+0000","last_change":"2023-10-06T02:29:36.192264+0000","last_active":"2023-10-06T02:29:36.196247+0000","last_peered":"2023-10-06T02:29:36.196247+0000","last_clean":"2023-10-06T02:29:36.196247+0000","last_became_active":"2023-10-06T02:29:36.191984+0000","last_became_peered":"2023-10-06T02:29:36.191984+0000","last_unstale":"2023-10-06T02:29:36.196247+0000","last_undegraded":"2023-10-06T02:29:36.196247+0000","last_fullsized":"2023-10-06T02:29:36.196247+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:09:06.367106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"119'2","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.202233+0000","last_change":"2023-10-06T02:29:35.186954+0000","last_active":"2023-10-06T02:29:36.202233+0000","last_peered":"2023-10-06T02:29:36.202233+0000","last_clean":"2023-10-06T02:29:36.202233+0000","last_became_active":"2023-10-06T02:29:35.186630+0000","last_became_peered":"2023-10-06T02:29:35.186630+0000","last_unstale":"2023-10-06T02:29:36.202233+0000","last_undegraded":"2023-10-06T02:29:36.202233+0000","last_fullsized":"2023-10-06T02:29:36.202233+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_clean_scrub_stamp":"2023-10-06T02:29:34.151749+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:23:36.997767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.198465+0000","last_change":"2023-10-06T02:29:36.194504+0000","last_active":"2023-10-06T02:29:36.198465+0000","last_peered":"2023-10-06T02:29:36.198465+0000","last_clean":"2023-10-06T02:29:36.198465+0000","last_became_active":"2023-10-06T02:29:36.194064+0000","last_became_peered":"2023-10-06T02:29:36.194064+0000","last_unstale":"2023-10-06T02:29:36.198465+0000","last_undegraded":"2023-10-06T02:29:36.198465+0000","last_fullsized":"2023-10-06T02:29:36.198465+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:06:39.730288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1c","version":"119'2","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.204197+0000","last_change":"2023-10-06T02:29:35.191543+0000","last_active":"2023-10-06T02:29:36.204197+0000","last_peered":"2023-10-06T02:29:36.204197+0000","last_clean":"2023-10-06T02:29:36.204197+0000","last_became_active":"2023-10-06T02:29:35.191284+0000","last_became_peered":"2023-10-06T02:29:35.191284+0000","last_unstale":"2023-10-06T02:29:36.204197+0000","last_undegraded":"2023-10-06T02:29:36.204197+0000","last_fullsized":"2023-10-06T02:29:36.204197+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_clean_scrub_stamp":"2023-10-06T02:29:34.151749+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:54:47.275146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.197474+0000","last_change":"2023-10-06T02:29:36.190541+0000","last_active":"2023-10-06T02:29:36.197474+0000","last_peered":"2023-10-06T02:29:36.197474+0000","last_clean":"2023-10-06T02:29:36.197474+0000","last_became_active":"2023-10-06T02:29:36.190237+0000","last_became_peered":"2023-10-06T02:29:36.190237+0000","last_unstale":"2023-10-06T02:29:36.197474+0000","last_undegraded":"2023-10-06T02:29:36.197474+0000","last_fullsized":"2023-10-06T02:29:36.197474+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:10:07.892520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"119'1","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.203978+0000","last_change":"2023-10-06T02:29:35.190653+0000","last_active":"2023-10-06T02:29:36.203978+0000","last_peered":"2023-10-06T02:29:36.203978+0000","last_clean":"2023-10-06T02:29:36.203978+0000","last_became_active":"2023-10-06T02:29:35.190239+0000","last_became_peered":"2023-10-06T02:29:35.190239+0000","last_unstale":"2023-10-06T02:29:36.203978+0000","last_undegraded":"2023-10-06T02:29:36.203978+0000","last_fullsized":"2023-10-06T02:29:36.203978+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_clean_scrub_stamp":"2023-10-06T02:29:34.151749+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:40:48.229921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.198373+0000","last_change":"2023-10-06T02:29:36.197433+0000","last_active":"2023-10-06T02:29:36.198373+0000","last_peered":"2023-10-06T02:29:36.198373+0000","last_clean":"2023-10-06T02:29:36.198373+0000","last_became_active":"2023-10-06T02:29:36.197107+0000","last_became_peered":"2023-10-06T02:29:36.197107+0000","last_unstale":"2023-10-06T02:29:36.198373+0000","last_undegraded":"2023-10-06T02:29:36.198373+0000","last_fullsized":"2023-10-06T02:29:36.198373+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:18:41.713149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"119'1","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.188668+0000","last_change":"2023-10-06T02:29:35.188191+0000","last_active":"2023-10-06T02:29:36.188668+0000","last_peered":"2023-10-06T02:29:36.188668+0000","last_clean":"2023-10-06T02:29:36.188668+0000","last_became_active":"2023-10-06T02:29:35.187955+0000","last_became_peered":"2023-10-06T02:29:35.187955+0000","last_unstale":"2023-10-06T02:29:36.188668+0000","last_undegraded":"2023-10-06T02:29:36.188668+0000","last_fullsized":"2023-10-06T02:29:36.188668+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_clean_scrub_stamp":"2023-10-06T02:29:34.151749+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:29:18.462793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.200162+0000","last_change":"2023-10-06T02:29:36.195807+0000","last_active":"2023-10-06T02:29:36.200162+0000","last_peered":"2023-10-06T02:29:36.200162+0000","last_clean":"2023-10-06T02:29:36.200162+0000","last_became_active":"2023-10-06T02:29:36.195576+0000","last_became_peered":"2023-10-06T02:29:36.195576+0000","last_unstale":"2023-10-06T02:29:36.200162+0000","last_undegraded":"2023-10-06T02:29:36.200162+0000","last_fullsized":"2023-10-06T02:29:36.200162+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:03:45.223339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.184471+0000","last_change":"2023-10-06T02:29:35.181416+0000","last_active":"2023-10-06T02:29:36.184471+0000","last_peered":"2023-10-06T02:29:36.184471+0000","last_clean":"2023-10-06T02:29:36.184471+0000","last_became_active":"2023-10-06T02:29:35.181097+0000","last_became_peered":"2023-10-06T02:29:35.181097+0000","last_unstale":"2023-10-06T02:29:36.184471+0000","last_undegraded":"2023-10-06T02:29:36.184471+0000","last_fullsized":"2023-10-06T02:29:36.184471+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_clean_scrub_stamp":"2023-10-06T02:29:34.151749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:18:18.485443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.204050+0000","last_change":"2023-10-06T02:29:36.195444+0000","last_active":"2023-10-06T02:29:36.204050+0000","last_peered":"2023-10-06T02:29:36.204050+0000","last_clean":"2023-10-06T02:29:36.204050+0000","last_became_active":"2023-10-06T02:29:36.195135+0000","last_became_peered":"2023-10-06T02:29:36.195135+0000","last_unstale":"2023-10-06T02:29:36.204050+0000","last_undegraded":"2023-10-06T02:29:36.204050+0000","last_fullsized":"2023-10-06T02:29:36.204050+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:03:45.223339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.171902+0000","last_change":"2023-10-06T02:29:35.182468+0000","last_active":"2023-10-06T02:29:36.171902+0000","last_peered":"2023-10-06T02:29:36.171902+0000","last_clean":"2023-10-06T02:29:36.171902+0000","last_became_active":"2023-10-06T02:29:35.181783+0000","last_became_peered":"2023-10-06T02:29:35.181783+0000","last_unstale":"2023-10-06T02:29:36.171902+0000","last_undegraded":"2023-10-06T02:29:36.171902+0000","last_fullsized":"2023-10-06T02:29:36.171902+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_clean_scrub_stamp":"2023-10-06T02:29:34.151749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:31:13.871801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.193056+0000","last_change":"2023-10-06T02:29:36.181840+0000","last_active":"2023-10-06T02:29:36.193056+0000","last_peered":"2023-10-06T02:29:36.193056+0000","last_clean":"2023-10-06T02:29:36.193056+0000","last_became_active":"2023-10-06T02:29:36.181482+0000","last_became_peered":"2023-10-06T02:29:36.181482+0000","last_unstale":"2023-10-06T02:29:36.193056+0000","last_undegraded":"2023-10-06T02:29:36.193056+0000","last_fullsized":"2023-10-06T02:29:36.193056+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:53:28.589732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.f","version":"119'1","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.195193+0000","last_change":"2023-10-06T02:29:35.180337+0000","last_active":"2023-10-06T02:29:36.195193+0000","last_peered":"2023-10-06T02:29:36.195193+0000","last_clean":"2023-10-06T02:29:36.195193+0000","last_became_active":"2023-10-06T02:29:35.180005+0000","last_became_peered":"2023-10-06T02:29:35.180005+0000","last_unstale":"2023-10-06T02:29:36.195193+0000","last_undegraded":"2023-10-06T02:29:36.195193+0000","last_fullsized":"2023-10-06T02:29:36.195193+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_clean_scrub_stamp":"2023-10-06T02:29:34.151749+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:10:52.886547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.203580+0000","last_change":"2023-10-06T02:29:36.189641+0000","last_active":"2023-10-06T02:29:36.203580+0000","last_peered":"2023-10-06T02:29:36.203580+0000","last_clean":"2023-10-06T02:29:36.203580+0000","last_became_active":"2023-10-06T02:29:36.189282+0000","last_became_peered":"2023-10-06T02:29:36.189282+0000","last_unstale":"2023-10-06T02:29:36.203580+0000","last_undegraded":"2023-10-06T02:29:36.203580+0000","last_fullsized":"2023-10-06T02:29:36.203580+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:36:48.422482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.e","version":"119'2","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.203297+0000","last_change":"2023-10-06T02:29:35.185825+0000","last_active":"2023-10-06T02:29:36.203297+0000","last_peered":"2023-10-06T02:29:36.203297+0000","last_clean":"2023-10-06T02:29:36.203297+0000","last_became_active":"2023-10-06T02:29:35.185541+0000","last_became_peered":"2023-10-06T02:29:35.185541+0000","last_unstale":"2023-10-06T02:29:36.203297+0000","last_undegraded":"2023-10-06T02:29:36.203297+0000","last_fullsized":"2023-10-06T02:29:36.203297+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_clean_scrub_stamp":"2023-10-06T02:29:34.151749+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:06:38.723619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.204884+0000","last_change":"2023-10-06T02:29:36.201345+0000","last_active":"2023-10-06T02:29:36.204884+0000","last_peered":"2023-10-06T02:29:36.204884+0000","last_clean":"2023-10-06T02:29:36.204884+0000","last_became_active":"2023-10-06T02:29:36.200076+0000","last_became_peered":"2023-10-06T02:29:36.200076+0000","last_unstale":"2023-10-06T02:29:36.204884+0000","last_undegraded":"2023-10-06T02:29:36.204884+0000","last_fullsized":"2023-10-06T02:29:36.204884+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:34:31.991692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.d","version":"119'1","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.201791+0000","last_change":"2023-10-06T02:29:35.181838+0000","last_active":"2023-10-06T02:29:36.201791+0000","last_peered":"2023-10-06T02:29:36.201791+0000","last_clean":"2023-10-06T02:29:36.201791+0000","last_became_active":"2023-10-06T02:29:35.181156+0000","last_became_peered":"2023-10-06T02:29:35.181156+0000","last_unstale":"2023-10-06T02:29:36.201791+0000","last_undegraded":"2023-10-06T02:29:36.201791+0000","last_fullsized":"2023-10-06T02:29:36.201791+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_clean_scrub_stamp":"2023-10-06T02:29:34.151749+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:00:44.429148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.198318+0000","last_change":"2023-10-06T02:29:36.196495+0000","last_active":"2023-10-06T02:29:36.198318+0000","last_peered":"2023-10-06T02:29:36.198318+0000","last_clean":"2023-10-06T02:29:36.198318+0000","last_became_active":"2023-10-06T02:29:36.196109+0000","last_became_peered":"2023-10-06T02:29:36.196109+0000","last_unstale":"2023-10-06T02:29:36.198318+0000","last_undegraded":"2023-10-06T02:29:36.198318+0000","last_fullsized":"2023-10-06T02:29:36.198318+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:08:43.848065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.175404+0000","last_change":"2023-10-06T02:29:35.184270+0000","last_active":"2023-10-06T02:29:36.175404+0000","last_peered":"2023-10-06T02:29:36.175404+0000","last_clean":"2023-10-06T02:29:36.175404+0000","last_became_active":"2023-10-06T02:29:35.183938+0000","last_became_peered":"2023-10-06T02:29:35.183938+0000","last_unstale":"2023-10-06T02:29:36.175404+0000","last_undegraded":"2023-10-06T02:29:36.175404+0000","last_fullsized":"2023-10-06T02:29:36.175404+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_clean_scrub_stamp":"2023-10-06T02:29:34.151749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:18:40.706480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.186736+0000","last_change":"2023-10-06T02:29:36.182350+0000","last_active":"2023-10-06T02:29:36.186736+0000","last_peered":"2023-10-06T02:29:36.186736+0000","last_clean":"2023-10-06T02:29:36.186736+0000","last_became_active":"2023-10-06T02:29:36.182013+0000","last_became_peered":"2023-10-06T02:29:36.182013+0000","last_unstale":"2023-10-06T02:29:36.186736+0000","last_undegraded":"2023-10-06T02:29:36.186736+0000","last_fullsized":"2023-10-06T02:29:36.186736+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:36:48.422482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"119'2","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.205867+0000","last_change":"2023-10-06T02:29:35.186895+0000","last_active":"2023-10-06T02:29:36.205867+0000","last_peered":"2023-10-06T02:29:36.205867+0000","last_clean":"2023-10-06T02:29:36.205867+0000","last_became_active":"2023-10-06T02:29:35.186599+0000","last_became_peered":"2023-10-06T02:29:35.186599+0000","last_unstale":"2023-10-06T02:29:36.205867+0000","last_undegraded":"2023-10-06T02:29:36.205867+0000","last_fullsized":"2023-10-06T02:29:36.205867+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_clean_scrub_stamp":"2023-10-06T02:29:34.151749+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:39:16.941655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.197316+0000","last_change":"2023-10-06T02:29:36.193620+0000","last_active":"2023-10-06T02:29:36.197316+0000","last_peered":"2023-10-06T02:29:36.197316+0000","last_clean":"2023-10-06T02:29:36.197316+0000","last_became_active":"2023-10-06T02:29:36.193001+0000","last_became_peered":"2023-10-06T02:29:36.193001+0000","last_unstale":"2023-10-06T02:29:36.197316+0000","last_undegraded":"2023-10-06T02:29:36.197316+0000","last_fullsized":"2023-10-06T02:29:36.197316+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:13:52.435753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.168415+0000","last_change":"2023-10-06T02:29:35.185588+0000","last_active":"2023-10-06T02:29:36.168415+0000","last_peered":"2023-10-06T02:29:36.168415+0000","last_clean":"2023-10-06T02:29:36.168415+0000","last_became_active":"2023-10-06T02:29:35.185306+0000","last_became_peered":"2023-10-06T02:29:35.185306+0000","last_unstale":"2023-10-06T02:29:36.168415+0000","last_undegraded":"2023-10-06T02:29:36.168415+0000","last_fullsized":"2023-10-06T02:29:36.168415+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_clean_scrub_stamp":"2023-10-06T02:29:34.151749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:59:35.165543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.201597+0000","last_change":"2023-10-06T02:29:36.200293+0000","last_active":"2023-10-06T02:29:36.201597+0000","last_peered":"2023-10-06T02:29:36.201597+0000","last_clean":"2023-10-06T02:29:36.201597+0000","last_became_active":"2023-10-06T02:29:36.199983+0000","last_became_peered":"2023-10-06T02:29:36.199983+0000","last_unstale":"2023-10-06T02:29:36.201597+0000","last_undegraded":"2023-10-06T02:29:36.201597+0000","last_fullsized":"2023-10-06T02:29:36.201597+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:44:14.781598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.176666+0000","last_change":"2023-10-06T02:29:35.188766+0000","last_active":"2023-10-06T02:29:36.176666+0000","last_peered":"2023-10-06T02:29:36.176666+0000","last_clean":"2023-10-06T02:29:36.176666+0000","last_became_active":"2023-10-06T02:29:35.188353+0000","last_became_peered":"2023-10-06T02:29:35.188353+0000","last_unstale":"2023-10-06T02:29:36.176666+0000","last_undegraded":"2023-10-06T02:29:36.176666+0000","last_fullsized":"2023-10-06T02:29:36.176666+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_clean_scrub_stamp":"2023-10-06T02:29:34.151749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:55:16.710103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.191436+0000","last_change":"2023-10-06T02:29:36.185674+0000","last_active":"2023-10-06T02:29:36.191436+0000","last_peered":"2023-10-06T02:29:36.191436+0000","last_clean":"2023-10-06T02:29:36.191436+0000","last_became_active":"2023-10-06T02:29:36.185391+0000","last_became_peered":"2023-10-06T02:29:36.185391+0000","last_unstale":"2023-10-06T02:29:36.191436+0000","last_undegraded":"2023-10-06T02:29:36.191436+0000","last_fullsized":"2023-10-06T02:29:36.191436+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:55:17.716772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.184145+0000","last_change":"2023-10-06T02:29:35.179848+0000","last_active":"2023-10-06T02:29:36.184145+0000","last_peered":"2023-10-06T02:29:36.184145+0000","last_clean":"2023-10-06T02:29:36.184145+0000","last_became_active":"2023-10-06T02:29:35.179588+0000","last_became_peered":"2023-10-06T02:29:35.179588+0000","last_unstale":"2023-10-06T02:29:36.184145+0000","last_undegraded":"2023-10-06T02:29:36.184145+0000","last_fullsized":"2023-10-06T02:29:36.184145+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_clean_scrub_stamp":"2023-10-06T02:29:34.151749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:30:35.677183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.197087+0000","last_change":"2023-10-06T02:29:36.191908+0000","last_active":"2023-10-06T02:29:36.197087+0000","last_peered":"2023-10-06T02:29:36.197087+0000","last_clean":"2023-10-06T02:29:36.197087+0000","last_became_active":"2023-10-06T02:29:36.191641+0000","last_became_peered":"2023-10-06T02:29:36.191641+0000","last_unstale":"2023-10-06T02:29:36.197087+0000","last_undegraded":"2023-10-06T02:29:36.197087+0000","last_fullsized":"2023-10-06T02:29:36.197087+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:43:21.797680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.7","version":"119'2","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.200612+0000","last_change":"2023-10-06T02:29:35.186014+0000","last_active":"2023-10-06T02:29:36.200612+0000","last_peered":"2023-10-06T02:29:36.200612+0000","last_clean":"2023-10-06T02:29:36.200612+0000","last_became_active":"2023-10-06T02:29:35.185766+0000","last_became_peered":"2023-10-06T02:29:35.185766+0000","last_unstale":"2023-10-06T02:29:36.200612+0000","last_undegraded":"2023-10-06T02:29:36.200612+0000","last_fullsized":"2023-10-06T02:29:36.200612+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_clean_scrub_stamp":"2023-10-06T02:29:34.151749+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:26:07.237072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.193171+0000","last_change":"2023-10-06T02:29:36.183950+0000","last_active":"2023-10-06T02:29:36.193171+0000","last_peered":"2023-10-06T02:29:36.193171+0000","last_clean":"2023-10-06T02:29:36.193171+0000","last_became_active":"2023-10-06T02:29:36.183468+0000","last_became_peered":"2023-10-06T02:29:36.183468+0000","last_unstale":"2023-10-06T02:29:36.193171+0000","last_undegraded":"2023-10-06T02:29:36.193171+0000","last_fullsized":"2023-10-06T02:29:36.193171+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:38:49.300110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.6","version":"120'1","reported_seq":11,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-10-06T02:29:38.629801+0000","last_change":"2023-10-06T02:29:35.191390+0000","last_active":"2023-10-06T02:29:38.629801+0000","last_peered":"2023-10-06T02:29:38.629801+0000","last_clean":"2023-10-06T02:29:38.629801+0000","last_became_active":"2023-10-06T02:29:35.191055+0000","last_became_peered":"2023-10-06T02:29:35.191055+0000","last_unstale":"2023-10-06T02:29:38.629801+0000","last_undegraded":"2023-10-06T02:29:38.629801+0000","last_fullsized":"2023-10-06T02:29:38.629801+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_clean_scrub_stamp":"2023-10-06T02:29:34.151749+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:36:46.538094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.204904+0000","last_change":"2023-10-06T02:29:36.201724+0000","last_active":"2023-10-06T02:29:36.204904+0000","last_peered":"2023-10-06T02:29:36.204904+0000","last_clean":"2023-10-06T02:29:36.204904+0000","last_became_active":"2023-10-06T02:29:36.199808+0000","last_became_peered":"2023-10-06T02:29:36.199808+0000","last_unstale":"2023-10-06T02:29:36.204904+0000","last_undegraded":"2023-10-06T02:29:36.204904+0000","last_fullsized":"2023-10-06T02:29:36.204904+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:14:59.301511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.180618+0000","last_change":"2023-10-06T02:29:35.190464+0000","last_active":"2023-10-06T02:29:36.180618+0000","last_peered":"2023-10-06T02:29:36.180618+0000","last_clean":"2023-10-06T02:29:36.180618+0000","last_became_active":"2023-10-06T02:29:35.190162+0000","last_became_peered":"2023-10-06T02:29:35.190162+0000","last_unstale":"2023-10-06T02:29:36.180618+0000","last_undegraded":"2023-10-06T02:29:36.180618+0000","last_fullsized":"2023-10-06T02:29:36.180618+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_clean_scrub_stamp":"2023-10-06T02:29:34.151749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:19:48.839079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.196296+0000","last_change":"2023-10-06T02:29:36.189747+0000","last_active":"2023-10-06T02:29:36.196296+0000","last_peered":"2023-10-06T02:29:36.196296+0000","last_clean":"2023-10-06T02:29:36.196296+0000","last_became_active":"2023-10-06T02:29:36.189417+0000","last_became_peered":"2023-10-06T02:29:36.189417+0000","last_unstale":"2023-10-06T02:29:36.196296+0000","last_undegraded":"2023-10-06T02:29:36.196296+0000","last_fullsized":"2023-10-06T02:29:36.196296+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:44:27.074320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.186603+0000","last_change":"2023-10-06T02:29:35.181721+0000","last_active":"2023-10-06T02:29:36.186603+0000","last_peered":"2023-10-06T02:29:36.186603+0000","last_clean":"2023-10-06T02:29:36.186603+0000","last_became_active":"2023-10-06T02:29:35.181450+0000","last_became_peered":"2023-10-06T02:29:35.181450+0000","last_unstale":"2023-10-06T02:29:36.186603+0000","last_undegraded":"2023-10-06T02:29:36.186603+0000","last_fullsized":"2023-10-06T02:29:36.186603+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_clean_scrub_stamp":"2023-10-06T02:29:34.151749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:14:51.570765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.195671+0000","last_change":"2023-10-06T02:29:36.194729+0000","last_active":"2023-10-06T02:29:36.195671+0000","last_peered":"2023-10-06T02:29:36.195671+0000","last_clean":"2023-10-06T02:29:36.195671+0000","last_became_active":"2023-10-06T02:29:36.194394+0000","last_became_peered":"2023-10-06T02:29:36.194394+0000","last_unstale":"2023-10-06T02:29:36.195671+0000","last_undegraded":"2023-10-06T02:29:36.195671+0000","last_fullsized":"2023-10-06T02:29:36.195671+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:56:31.959275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.172798+0000","last_change":"2023-10-06T02:29:35.178750+0000","last_active":"2023-10-06T02:29:36.172798+0000","last_peered":"2023-10-06T02:29:36.172798+0000","last_clean":"2023-10-06T02:29:36.172798+0000","last_became_active":"2023-10-06T02:29:35.178427+0000","last_became_peered":"2023-10-06T02:29:35.178427+0000","last_unstale":"2023-10-06T02:29:36.172798+0000","last_undegraded":"2023-10-06T02:29:36.172798+0000","last_fullsized":"2023-10-06T02:29:36.172798+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_clean_scrub_stamp":"2023-10-06T02:29:34.151749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:52:23.438197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.196824+0000","last_change":"2023-10-06T02:29:36.195414+0000","last_active":"2023-10-06T02:29:36.196824+0000","last_peered":"2023-10-06T02:29:36.196824+0000","last_clean":"2023-10-06T02:29:36.196824+0000","last_became_active":"2023-10-06T02:29:36.195174+0000","last_became_peered":"2023-10-06T02:29:36.195174+0000","last_unstale":"2023-10-06T02:29:36.196824+0000","last_undegraded":"2023-10-06T02:29:36.196824+0000","last_fullsized":"2023-10-06T02:29:36.196824+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:30:23.636577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.177136+0000","last_change":"2023-10-06T02:29:35.191801+0000","last_active":"2023-10-06T02:29:36.177136+0000","last_peered":"2023-10-06T02:29:36.177136+0000","last_clean":"2023-10-06T02:29:36.177136+0000","last_became_active":"2023-10-06T02:29:35.191260+0000","last_became_peered":"2023-10-06T02:29:35.191260+0000","last_unstale":"2023-10-06T02:29:36.177136+0000","last_undegraded":"2023-10-06T02:29:36.177136+0000","last_fullsized":"2023-10-06T02:29:36.177136+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_clean_scrub_stamp":"2023-10-06T02:29:34.151749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:10:06.885851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.205562+0000","last_change":"2023-10-06T02:29:36.205054+0000","last_active":"2023-10-06T02:29:36.205562+0000","last_peered":"2023-10-06T02:29:36.205562+0000","last_clean":"2023-10-06T02:29:36.205562+0000","last_became_active":"2023-10-06T02:29:36.204829+0000","last_became_peered":"2023-10-06T02:29:36.204829+0000","last_unstale":"2023-10-06T02:29:36.205562+0000","last_undegraded":"2023-10-06T02:29:36.205562+0000","last_fullsized":"2023-10-06T02:29:36.205562+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:39:17.948324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.184586+0000","last_change":"2023-10-06T02:29:35.178708+0000","last_active":"2023-10-06T02:29:36.184586+0000","last_peered":"2023-10-06T02:29:36.184586+0000","last_clean":"2023-10-06T02:29:36.184586+0000","last_became_active":"2023-10-06T02:29:35.178434+0000","last_became_peered":"2023-10-06T02:29:35.178434+0000","last_unstale":"2023-10-06T02:29:36.184586+0000","last_undegraded":"2023-10-06T02:29:36.184586+0000","last_fullsized":"2023-10-06T02:29:36.184586+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_clean_scrub_stamp":"2023-10-06T02:29:34.151749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:22:09.674761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.204252+0000","last_change":"2023-10-06T02:29:36.198675+0000","last_active":"2023-10-06T02:29:36.204252+0000","last_peered":"2023-10-06T02:29:36.204252+0000","last_clean":"2023-10-06T02:29:36.204252+0000","last_became_active":"2023-10-06T02:29:36.198384+0000","last_became_peered":"2023-10-06T02:29:36.198384+0000","last_unstale":"2023-10-06T02:29:36.204252+0000","last_undegraded":"2023-10-06T02:29:36.204252+0000","last_fullsized":"2023-10-06T02:29:36.204252+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:33:43.513554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.173907+0000","last_change":"2023-10-06T02:29:35.187993+0000","last_active":"2023-10-06T02:29:36.173907+0000","last_peered":"2023-10-06T02:29:36.173907+0000","last_clean":"2023-10-06T02:29:36.173907+0000","last_became_active":"2023-10-06T02:29:35.187730+0000","last_became_peered":"2023-10-06T02:29:35.187730+0000","last_unstale":"2023-10-06T02:29:36.173907+0000","last_undegraded":"2023-10-06T02:29:36.173907+0000","last_fullsized":"2023-10-06T02:29:36.173907+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_clean_scrub_stamp":"2023-10-06T02:29:34.151749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:25:18.945648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.199652+0000","last_change":"2023-10-06T02:29:36.196164+0000","last_active":"2023-10-06T02:29:36.199652+0000","last_peered":"2023-10-06T02:29:36.199652+0000","last_clean":"2023-10-06T02:29:36.199652+0000","last_became_active":"2023-10-06T02:29:36.195922+0000","last_became_peered":"2023-10-06T02:29:36.195922+0000","last_unstale":"2023-10-06T02:29:36.199652+0000","last_undegraded":"2023-10-06T02:29:36.199652+0000","last_fullsized":"2023-10-06T02:29:36.199652+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:40:49.236590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.169161+0000","last_change":"2023-10-06T02:29:35.179569+0000","last_active":"2023-10-06T02:29:36.169161+0000","last_peered":"2023-10-06T02:29:36.169161+0000","last_clean":"2023-10-06T02:29:36.169161+0000","last_became_active":"2023-10-06T02:29:35.178995+0000","last_became_peered":"2023-10-06T02:29:35.178995+0000","last_unstale":"2023-10-06T02:29:36.169161+0000","last_undegraded":"2023-10-06T02:29:36.169161+0000","last_fullsized":"2023-10-06T02:29:36.169161+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_clean_scrub_stamp":"2023-10-06T02:29:34.151749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:22:17.305684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":96,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.173434+0000","last_change":"2023-10-06T02:23:01.992935+0000","last_active":"2023-10-06T02:29:36.173434+0000","last_peered":"2023-10-06T02:29:36.173434+0000","last_clean":"2023-10-06T02:29:36.173434+0000","last_became_active":"2023-10-06T02:23:01.992568+0000","last_became_peered":"2023-10-06T02:23:01.992568+0000","last_unstale":"2023-10-06T02:29:36.173434+0000","last_undegraded":"2023-10-06T02:29:36.173434+0000","last_fullsized":"2023-10-06T02:29:36.173434+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:06.212251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.193804+0000","last_change":"2023-10-06T02:29:36.188159+0000","last_active":"2023-10-06T02:29:36.193804+0000","last_peered":"2023-10-06T02:29:36.193804+0000","last_clean":"2023-10-06T02:29:36.193804+0000","last_became_active":"2023-10-06T02:29:36.187799+0000","last_became_peered":"2023-10-06T02:29:36.187799+0000","last_unstale":"2023-10-06T02:29:36.193804+0000","last_undegraded":"2023-10-06T02:29:36.193804+0000","last_fullsized":"2023-10-06T02:29:36.193804+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:12:52.344361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.171388+0000","last_change":"2023-10-06T02:29:35.176590+0000","last_active":"2023-10-06T02:29:36.171388+0000","last_peered":"2023-10-06T02:29:36.171388+0000","last_clean":"2023-10-06T02:29:36.171388+0000","last_became_active":"2023-10-06T02:29:35.176154+0000","last_became_peered":"2023-10-06T02:29:35.176154+0000","last_unstale":"2023-10-06T02:29:36.171388+0000","last_undegraded":"2023-10-06T02:29:36.171388+0000","last_fullsized":"2023-10-06T02:29:36.171388+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_clean_scrub_stamp":"2023-10-06T02:29:34.151749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:17:46.181194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":95,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.184808+0000","last_change":"2023-10-06T02:23:01.989934+0000","last_active":"2023-10-06T02:29:36.184808+0000","last_peered":"2023-10-06T02:29:36.184808+0000","last_clean":"2023-10-06T02:29:36.184808+0000","last_became_active":"2023-10-06T02:23:01.989706+0000","last_became_peered":"2023-10-06T02:23:01.989706+0000","last_unstale":"2023-10-06T02:29:36.184808+0000","last_undegraded":"2023-10-06T02:29:36.184808+0000","last_fullsized":"2023-10-06T02:29:36.184808+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:06.212251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.193269+0000","last_change":"2023-10-06T02:29:36.180993+0000","last_active":"2023-10-06T02:29:36.193269+0000","last_peered":"2023-10-06T02:29:36.193269+0000","last_clean":"2023-10-06T02:29:36.193269+0000","last_became_active":"2023-10-06T02:29:36.180697+0000","last_became_peered":"2023-10-06T02:29:36.180697+0000","last_unstale":"2023-10-06T02:29:36.193269+0000","last_undegraded":"2023-10-06T02:29:36.193269+0000","last_fullsized":"2023-10-06T02:29:36.193269+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:22:10.681430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.14","version":"119'6","reported_seq":16,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.207057+0000","last_change":"2023-10-06T02:29:35.189634+0000","last_active":"2023-10-06T02:29:36.207057+0000","last_peered":"2023-10-06T02:29:36.207057+0000","last_clean":"2023-10-06T02:29:36.207057+0000","last_became_active":"2023-10-06T02:29:35.189353+0000","last_became_peered":"2023-10-06T02:29:35.189353+0000","last_unstale":"2023-10-06T02:29:36.207057+0000","last_undegraded":"2023-10-06T02:29:36.207057+0000","last_fullsized":"2023-10-06T02:29:36.207057+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_clean_scrub_stamp":"2023-10-06T02:29:34.151749+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:02:42.408863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":91,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.180100+0000","last_change":"2023-10-06T02:23:02.054069+0000","last_active":"2023-10-06T02:29:36.180100+0000","last_peered":"2023-10-06T02:29:36.180100+0000","last_clean":"2023-10-06T02:29:36.180100+0000","last_became_active":"2023-10-06T02:23:02.053739+0000","last_became_peered":"2023-10-06T02:23:02.053739+0000","last_unstale":"2023-10-06T02:29:36.180100+0000","last_undegraded":"2023-10-06T02:29:36.180100+0000","last_fullsized":"2023-10-06T02:29:36.180100+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:57:48.822608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.203957+0000","last_change":"2023-10-06T02:29:36.190327+0000","last_active":"2023-10-06T02:29:36.203957+0000","last_peered":"2023-10-06T02:29:36.203957+0000","last_clean":"2023-10-06T02:29:36.203957+0000","last_became_active":"2023-10-06T02:29:36.190056+0000","last_became_peered":"2023-10-06T02:29:36.190056+0000","last_unstale":"2023-10-06T02:29:36.203957+0000","last_undegraded":"2023-10-06T02:29:36.203957+0000","last_fullsized":"2023-10-06T02:29:36.203957+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:44:14.781598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.172812+0000","last_change":"2023-10-06T02:29:35.187050+0000","last_active":"2023-10-06T02:29:36.172812+0000","last_peered":"2023-10-06T02:29:36.172812+0000","last_clean":"2023-10-06T02:29:36.172812+0000","last_became_active":"2023-10-06T02:29:35.186770+0000","last_became_peered":"2023-10-06T02:29:35.186770+0000","last_unstale":"2023-10-06T02:29:36.172812+0000","last_undegraded":"2023-10-06T02:29:36.172812+0000","last_fullsized":"2023-10-06T02:29:36.172812+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_clean_scrub_stamp":"2023-10-06T02:29:34.151749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:48:19.623691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":91,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.176343+0000","last_change":"2023-10-06T02:23:01.990773+0000","last_active":"2023-10-06T02:29:36.176343+0000","last_peered":"2023-10-06T02:29:36.176343+0000","last_clean":"2023-10-06T02:29:36.176343+0000","last_became_active":"2023-10-06T02:23:01.990511+0000","last_became_peered":"2023-10-06T02:23:01.990511+0000","last_unstale":"2023-10-06T02:29:36.176343+0000","last_undegraded":"2023-10-06T02:29:36.176343+0000","last_fullsized":"2023-10-06T02:29:36.176343+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:42:54.797731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.196432+0000","last_change":"2023-10-06T02:29:36.189566+0000","last_active":"2023-10-06T02:29:36.196432+0000","last_peered":"2023-10-06T02:29:36.196432+0000","last_clean":"2023-10-06T02:29:36.196432+0000","last_became_active":"2023-10-06T02:29:36.188988+0000","last_became_peered":"2023-10-06T02:29:36.188988+0000","last_unstale":"2023-10-06T02:29:36.196432+0000","last_undegraded":"2023-10-06T02:29:36.196432+0000","last_fullsized":"2023-10-06T02:29:36.196432+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:52:11.695224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.177015+0000","last_change":"2023-10-06T02:29:35.185086+0000","last_active":"2023-10-06T02:29:36.177015+0000","last_peered":"2023-10-06T02:29:36.177015+0000","last_clean":"2023-10-06T02:29:36.177015+0000","last_became_active":"2023-10-06T02:29:35.184418+0000","last_became_peered":"2023-10-06T02:29:35.184418+0000","last_unstale":"2023-10-06T02:29:36.177015+0000","last_undegraded":"2023-10-06T02:29:36.177015+0000","last_fullsized":"2023-10-06T02:29:36.177015+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_clean_scrub_stamp":"2023-10-06T02:29:34.151749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:44:01.043777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"76'128","reported_seq":283,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.176447+0000","last_change":"2023-10-06T02:22:55.032519+0000","last_active":"2023-10-06T02:29:36.176447+0000","last_peered":"2023-10-06T02:29:36.176447+0000","last_clean":"2023-10-06T02:29:36.176447+0000","last_became_active":"2023-10-06T02:22:55.031367+0000","last_became_peered":"2023-10-06T02:22:55.031367+0000","last_unstale":"2023-10-06T02:29:36.176447+0000","last_undegraded":"2023-10-06T02:29:36.176447+0000","last_fullsized":"2023-10-06T02:29:36.176447+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:22:53.309097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:22:53.309097+0000","last_clean_scrub_stamp":"2023-10-06T02:22:53.309097+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:53:46.520909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":110,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.173524+0000","last_change":"2023-10-06T02:23:01.988140+0000","last_active":"2023-10-06T02:29:36.173524+0000","last_peered":"2023-10-06T02:29:36.173524+0000","last_clean":"2023-10-06T02:29:36.173524+0000","last_became_active":"2023-10-06T02:23:01.987769+0000","last_became_peered":"2023-10-06T02:23:01.987769+0000","last_unstale":"2023-10-06T02:29:36.173524+0000","last_undegraded":"2023-10-06T02:29:36.173524+0000","last_fullsized":"2023-10-06T02:29:36.173524+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:20:08.391912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.201090+0000","last_change":"2023-10-06T02:29:36.189439+0000","last_active":"2023-10-06T02:29:36.201090+0000","last_peered":"2023-10-06T02:29:36.201090+0000","last_clean":"2023-10-06T02:29:36.201090+0000","last_became_active":"2023-10-06T02:29:36.189289+0000","last_became_peered":"2023-10-06T02:29:36.189289+0000","last_unstale":"2023-10-06T02:29:36.201090+0000","last_undegraded":"2023-10-06T02:29:36.201090+0000","last_fullsized":"2023-10-06T02:29:36.201090+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:33:36.484989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.10","version":"119'8","reported_seq":18,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.204747+0000","last_change":"2023-10-06T02:29:35.190087+0000","last_active":"2023-10-06T02:29:36.204747+0000","last_peered":"2023-10-06T02:29:36.204747+0000","last_clean":"2023-10-06T02:29:36.204747+0000","last_became_active":"2023-10-06T02:29:35.189741+0000","last_became_peered":"2023-10-06T02:29:35.189741+0000","last_unstale":"2023-10-06T02:29:36.204747+0000","last_undegraded":"2023-10-06T02:29:36.204747+0000","last_fullsized":"2023-10-06T02:29:36.204747+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_clean_scrub_stamp":"2023-10-06T02:29:34.151749+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:53:27.583063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":102,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.168303+0000","last_change":"2023-10-06T02:23:02.054338+0000","last_active":"2023-10-06T02:29:36.168303+0000","last_peered":"2023-10-06T02:29:36.168303+0000","last_clean":"2023-10-06T02:29:36.168303+0000","last_became_active":"2023-10-06T02:23:02.053991+0000","last_became_peered":"2023-10-06T02:23:02.053991+0000","last_unstale":"2023-10-06T02:29:36.168303+0000","last_undegraded":"2023-10-06T02:29:36.168303+0000","last_fullsized":"2023-10-06T02:29:36.168303+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:39:30.192838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.185878+0000","last_change":"2023-10-06T02:29:36.182601+0000","last_active":"2023-10-06T02:29:36.185878+0000","last_peered":"2023-10-06T02:29:36.185878+0000","last_clean":"2023-10-06T02:29:36.185878+0000","last_became_active":"2023-10-06T02:29:36.182333+0000","last_became_peered":"2023-10-06T02:29:36.182333+0000","last_unstale":"2023-10-06T02:29:36.185878+0000","last_undegraded":"2023-10-06T02:29:36.185878+0000","last_fullsized":"2023-10-06T02:29:36.185878+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:41:12.745924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"119'2","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.198540+0000","last_change":"2023-10-06T02:29:35.178863+0000","last_active":"2023-10-06T02:29:36.198540+0000","last_peered":"2023-10-06T02:29:36.198540+0000","last_clean":"2023-10-06T02:29:36.198540+0000","last_became_active":"2023-10-06T02:29:35.178607+0000","last_became_peered":"2023-10-06T02:29:35.178607+0000","last_unstale":"2023-10-06T02:29:36.198540+0000","last_undegraded":"2023-10-06T02:29:36.198540+0000","last_fullsized":"2023-10-06T02:29:36.198540+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_clean_scrub_stamp":"2023-10-06T02:29:34.151749+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:21:08.149327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":110,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.172653+0000","last_change":"2023-10-06T02:23:01.988631+0000","last_active":"2023-10-06T02:29:36.172653+0000","last_peered":"2023-10-06T02:29:36.172653+0000","last_clean":"2023-10-06T02:29:36.172653+0000","last_became_active":"2023-10-06T02:23:01.988303+0000","last_became_peered":"2023-10-06T02:23:01.988303+0000","last_unstale":"2023-10-06T02:29:36.172653+0000","last_undegraded":"2023-10-06T02:29:36.172653+0000","last_fullsized":"2023-10-06T02:29:36.172653+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:38.211466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.195014+0000","last_change":"2023-10-06T02:29:36.186370+0000","last_active":"2023-10-06T02:29:36.195014+0000","last_peered":"2023-10-06T02:29:36.195014+0000","last_clean":"2023-10-06T02:29:36.195014+0000","last_became_active":"2023-10-06T02:29:36.183661+0000","last_became_peered":"2023-10-06T02:29:36.183661+0000","last_unstale":"2023-10-06T02:29:36.195014+0000","last_undegraded":"2023-10-06T02:29:36.195014+0000","last_fullsized":"2023-10-06T02:29:36.195014+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:42:32.320089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"119'3","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.206116+0000","last_change":"2023-10-06T02:29:35.193263+0000","last_active":"2023-10-06T02:29:36.206116+0000","last_peered":"2023-10-06T02:29:36.206116+0000","last_clean":"2023-10-06T02:29:36.206116+0000","last_became_active":"2023-10-06T02:29:35.192895+0000","last_became_peered":"2023-10-06T02:29:35.192895+0000","last_unstale":"2023-10-06T02:29:36.206116+0000","last_undegraded":"2023-10-06T02:29:36.206116+0000","last_fullsized":"2023-10-06T02:29:36.206116+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_clean_scrub_stamp":"2023-10-06T02:29:34.151749+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:18:40.706480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":95,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.186684+0000","last_change":"2023-10-06T02:23:02.054039+0000","last_active":"2023-10-06T02:29:36.186684+0000","last_peered":"2023-10-06T02:29:36.186684+0000","last_clean":"2023-10-06T02:29:36.186684+0000","last_became_active":"2023-10-06T02:23:02.053742+0000","last_became_peered":"2023-10-06T02:23:02.053742+0000","last_unstale":"2023-10-06T02:29:36.186684+0000","last_undegraded":"2023-10-06T02:29:36.186684+0000","last_fullsized":"2023-10-06T02:29:36.186684+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:58:18.846475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574934,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303444,"kb_used_data":2060,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068396,"statfs":{"total":96636764160,"available":96326037504,"internally_reserved":0,"allocated":2109440,"data_stored":1288447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:29:06 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.63900000000000001},"min":{"1min":0.46300000000000002,"5min":0.318,"15min":0.318},"max":{"1min":0.90000000000000002,"5min":1.169,"15min":1.169},"last":0.63},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63800000000000001,"15min":0.61099999999999999},"min":{"1min":0.437,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.83399999999999996,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.84699999999999998}]},{"osd":1,"last update":"Fri Oct 6 02:29:06 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.72699999999999998,"15min":0.69099999999999995},"min":{"1min":0.55200000000000005,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.1020000000000001,"5min":1.147,"15min":1.147},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67600000000000005,"15min":0.67300000000000004},"min":{"1min":0.45100000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.82399999999999995}]},{"osd":2,"last update":"Fri Oct 6 02:29:06 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.68200000000000005,"15min":0.69099999999999995},"min":{"1min":0.5,"5min":0.375,"15min":0.375},"max":{"1min":0.95999999999999996,"5min":1.238,"15min":1.238},"last":0.87},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.68600000000000005,"15min":0.68999999999999995},"min":{"1min":0.47099999999999997,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.95099999999999996,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.69599999999999995}]},{"osd":3,"last update":"Fri Oct 6 02:29:06 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.74299999999999999,"15min":0.73099999999999998},"min":{"1min":0.54500000000000004,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.97899999999999998,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.76400000000000001,"15min":0.77400000000000002},"min":{"1min":0.58499999999999996,"5min":0.497,"15min":0.497},"max":{"1min":0.94499999999999995,"5min":1.23,"15min":1.23},"last":1.004}]},{"osd":4,"last update":"Fri Oct 6 02:29:06 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.60499999999999998,"15min":0.68100000000000005},"min":{"1min":0.45600000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.88400000000000001,"5min":1.002,"15min":1.2849999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.68799999999999994,"15min":0.75700000000000001},"min":{"1min":0.42299999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.84699999999999998,"5min":1.51,"15min":1.51},"last":0.54400000000000004}]},{"osd":5,"last update":"Fri Oct 6 02:29:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.69899999999999995,"15min":0.72999999999999998},"min":{"1min":0.46300000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.97999999999999998,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.69099999999999995,"15min":0.72699999999999998},"min":{"1min":0.46800000000000003,"5min":0.442,"15min":0.442},"max":{"1min":0.96599999999999997,"5min":0.98099999999999998,"15min":1.0289999999999999},"last":0.66500000000000004}]},{"osd":6,"last update":"Fri Oct 6 02:29:06 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.76800000000000002,"15min":0.78500000000000003},"min":{"1min":0.626,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0089999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.80100000000000005,"15min":0.81399999999999995},"min":{"1min":0.55100000000000005,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.92800000000000005,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.80100000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574934,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302856,"kb_used_data":1472,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068984,"statfs":{"total":96636764160,"available":96326639616,"internally_reserved":0,"allocated":1507328,"data_stored":697097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:29:05 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.67900000000000005,"15min":0.66700000000000004},"min":{"1min":0.44,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.505},{"interface":"front","average":{"1min":0.749,"5min":0.72899999999999998,"15min":0.70199999999999996},"min":{"1min":0.497,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0229999999999999,"5min":1.119,"15min":1.119},"last":0.56000000000000005}]},{"osd":1,"last update":"Fri Oct 6 02:29:05 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.65900000000000003,"15min":0.65000000000000002},"min":{"1min":0.53400000000000003,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.90200000000000002,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.68899999999999995,"15min":0.67100000000000004},"min":{"1min":0.53500000000000003,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.86699999999999999,"5min":0.997,"15min":0.997},"last":0.66600000000000004}]},{"osd":2,"last update":"Fri Oct 6 02:29:05 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.73099999999999998,"15min":0.70899999999999996},"min":{"1min":0.48899999999999999,"5min":0.437,"15min":0.437},"max":{"1min":1.0449999999999999,"5min":1.137,"15min":1.137},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.74199999999999999,"15min":0.71599999999999997},"min":{"1min":0.47199999999999998,"5min":0.45300000000000001,"15min":0.40300000000000002},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.80200000000000005}]},{"osd":3,"last update":"Fri Oct 6 02:29:05 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70599999999999996,"15min":0.66800000000000004},"min":{"1min":0.51300000000000001,"5min":0.49099999999999999,"15min":0.45700000000000002},"max":{"1min":0.95399999999999996,"5min":1.056,"15min":1.056},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.749,"5min":0.73099999999999998,"15min":0.70899999999999996},"min":{"1min":0.59599999999999997,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.97999999999999998,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.95699999999999996}]},{"osd":4,"last update":"Fri Oct 6 02:29:05 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66600000000000004,"15min":0.64300000000000002},"min":{"1min":0.40999999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.90700000000000003,"5min":1.095,"15min":1.095},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71199999999999997,"15min":0.66800000000000004},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.40999999999999998},"max":{"1min":0.92800000000000005,"5min":1.29,"15min":1.29},"last":0.91900000000000004}]},{"osd":5,"last update":"Fri Oct 6 02:29:05 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69099999999999995,"15min":0.70099999999999996},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.39700000000000002},"max":{"1min":0.89300000000000002,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":1.014},{"interface":"front","average":{"1min":0.745,"5min":0.72599999999999998,"15min":0.70799999999999996},"min":{"1min":0.49099999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.93799999999999994,"5min":1,"15min":1.02},"last":0.88700000000000001}]},{"osd":7,"last update":"Fri Oct 6 02:29:05 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.749,"15min":0.749},"min":{"1min":0.42399999999999999,"5min":0.41499999999999998,"15min":0.36799999999999999},"max":{"1min":0.95599999999999996,"5min":1.173,"15min":1.458},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.77500000000000002,"15min":0.76400000000000001},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.45900000000000002},"max":{"1min":1.103,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":1.046}]}]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302852,"kb_used_data":1468,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068988,"statfs":{"total":96636764160,"available":96326643712,"internally_reserved":0,"allocated":1503232,"data_stored":696523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:29:01 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.434,"15min":0.45700000000000002},"min":{"1min":0.29899999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.63600000000000001,"5min":0.83099999999999996,"15min":1.2230000000000001},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.46400000000000002,"15min":0.51900000000000002},"min":{"1min":0.34100000000000003,"5min":0.31,"15min":0.31},"max":{"1min":0.55600000000000005,"5min":0.71799999999999997,"15min":1.788},"last":0.35099999999999998}]},{"osd":2,"last update":"Fri Oct 6 02:29:01 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.44,"15min":0.51400000000000001},"min":{"1min":0.33300000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.58799999999999997,"5min":0.66200000000000003,"15min":2.0840000000000001},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.499,"15min":0.55100000000000005},"min":{"1min":0.375,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.61299999999999999,"5min":0.69299999999999995,"15min":1.6699999999999999},"last":0.52100000000000002}]},{"osd":3,"last update":"Fri Oct 6 02:29:01 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.59799999999999998,"15min":0.61399999999999999},"min":{"1min":0.46400000000000002,"5min":0.44600000000000001,"15min":0.42999999999999999},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":1.266},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.53300000000000003,"15min":0.628},"min":{"1min":0.40999999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.67200000000000004,"5min":0.79400000000000004,"15min":1.861},"last":0.59299999999999997}]},{"osd":4,"last update":"Fri Oct 6 02:29:01 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.77700000000000002,"15min":0.84699999999999998},"min":{"1min":0.59999999999999998,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.97199999999999998,"5min":1.3580000000000001,"15min":1.9279999999999999},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.78600000000000003,"15min":0.84799999999999998},"min":{"1min":0.56200000000000006,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.94499999999999995,"5min":1.4339999999999999,"15min":2.0179999999999998},"last":0.751}]},{"osd":5,"last update":"Fri Oct 6 02:29:01 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.746,"15min":0.84799999999999998},"min":{"1min":0.53900000000000003,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.9830000000000001},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.753,"15min":0.81499999999999995},"min":{"1min":0.57099999999999995,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.327,"5min":1.327,"15min":1.3380000000000001},"last":0.68600000000000005}]},{"osd":6,"last update":"Fri Oct 6 02:29:01 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.79100000000000004,"15min":0.85799999999999998},"min":{"1min":0.57899999999999996,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0009999999999999,"5min":1.1240000000000001,"15min":1.829},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.81000000000000005,"15min":0.90100000000000002},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.351,"5min":1.351,"15min":2.052},"last":0.61099999999999999}]},{"osd":7,"last update":"Fri Oct 6 02:29:01 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.77200000000000002,"15min":0.82799999999999996},"min":{"1min":0.55900000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.96199999999999997,"5min":1.3120000000000001,"15min":1.3340000000000001},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.78400000000000003,"15min":0.86799999999999999},"min":{"1min":0.57099999999999995,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.93400000000000005,"5min":1.2110000000000001,"15min":1.895},"last":0.75800000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574934,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303436,"kb_used_data":2052,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068404,"statfs":{"total":96636764160,"available":96326045696,"internally_reserved":0,"allocated":2101248,"data_stored":1286877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 02:29:13 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.42699999999999999,"15min":0.45200000000000001},"min":{"1min":0.32700000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":1.0129999999999999},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.504,"5min":0.46500000000000002,"15min":0.47899999999999998},"min":{"1min":0.29799999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.9830000000000001,"5min":1.9830000000000001,"15min":1.9830000000000001},"last":0.40600000000000003}]},{"osd":2,"last update":"Fri Oct 6 02:29:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.505,"15min":0.51600000000000001},"min":{"1min":0.38,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":3.0339999999999998,"5min":3.0339999999999998,"15min":3.0339999999999998},"last":0.626},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.503,"15min":0.51900000000000002},"min":{"1min":0.39200000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.44800000000000001}]},{"osd":3,"last update":"Fri Oct 6 02:29:13 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.48699999999999999,"15min":0.503},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.59399999999999997},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.028,"5min":1.028,"15min":1.1359999999999999},"last":0.68300000000000005}]},{"osd":4,"last update":"Fri Oct 6 02:29:13 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.755,"15min":0.79800000000000004},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":2.9940000000000002,"5min":2.9940000000000002,"15min":2.9940000000000002},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.75900000000000001,"15min":0.79700000000000004},"min":{"1min":0.56599999999999995,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.88700000000000001}]},{"osd":5,"last update":"Fri Oct 6 02:29:13 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.76000000000000001,"15min":0.78600000000000003},"min":{"1min":0.61599999999999999,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.79300000000000004,"15min":0.80300000000000005},"min":{"1min":0.58299999999999996,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":3.1110000000000002,"5min":3.1110000000000002,"15min":3.1110000000000002},"last":0.90700000000000003}]},{"osd":6,"last update":"Fri Oct 6 02:29:13 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.80700000000000005,"15min":0.82399999999999995},"min":{"1min":0.61399999999999999,"5min":0.57599999999999996,"15min":0.54600000000000004},"max":{"1min":3.1600000000000001,"5min":3.1600000000000001,"15min":3.1600000000000001},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77300000000000002,"15min":0.82399999999999995},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.80600000000000005}]},{"osd":7,"last update":"Fri Oct 6 02:29:13 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.79700000000000004,"15min":0.82599999999999996},"min":{"1min":0.60199999999999998,"5min":0.56899999999999995,"15min":0.56000000000000005},"max":{"1min":2.9369999999999998,"5min":2.9369999999999998,"15min":2.9369999999999998},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.77300000000000002,"15min":0.82099999999999995},"min":{"1min":0.60499999999999998,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.93700000000000006}]}]},{"osd":2,"up_from":13,"seq":55834574935,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302856,"kb_used_data":1472,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068984,"statfs":{"total":96636764160,"available":96326639616,"internally_reserved":0,"allocated":1507328,"data_stored":697041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:29:09 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.42899999999999999,"15min":0.439},"min":{"1min":0.29899999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.46999999999999997,"15min":0.48399999999999999},"min":{"1min":0.32700000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.69199999999999995,"5min":0.88700000000000001,"15min":1.0800000000000001},"last":0.45100000000000001}]},{"osd":1,"last update":"Fri Oct 6 02:29:09 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.48199999999999998,"15min":0.501},"min":{"1min":0.36299999999999999,"5min":0.33600000000000002,"15min":0.32400000000000001},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.52600000000000002,"15min":0.59999999999999998},"min":{"1min":0.379,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.86299999999999999,"5min":1.778,"15min":1.778},"last":0.52900000000000003}]},{"osd":3,"last update":"Fri Oct 6 02:29:09 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.51400000000000001,"15min":0.53400000000000003},"min":{"1min":0.33800000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.97499999999999998},"last":0.437},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.58999999999999997,"15min":0.61799999999999999},"min":{"1min":0.46600000000000003,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":1.028},"last":0.51800000000000002}]},{"osd":4,"last update":"Fri Oct 6 02:29:09 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.747,"15min":0.77500000000000002},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.0649999999999999,"5min":1.109,"15min":1.1319999999999999},"last":0.97499999999999998},{"interface":"front","average":{"1min":0.747,"5min":0.78300000000000003,"15min":0.84099999999999997},"min":{"1min":0.55900000000000005,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.014,"5min":1.907,"15min":1.907},"last":0.81799999999999995}]},{"osd":5,"last update":"Fri Oct 6 02:29:09 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.73799999999999999,"15min":0.79600000000000004},"min":{"1min":0.53600000000000003,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.077,"5min":1.109,"15min":1.1850000000000001},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76000000000000001,"15min":0.81599999999999995},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.98699999999999999,"5min":1.153,"15min":1.258},"last":0.90900000000000003}]},{"osd":6,"last update":"Fri Oct 6 02:29:09 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.79100000000000004,"15min":0.84299999999999997},"min":{"1min":0.59399999999999997,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.008,"5min":1.827,"15min":1.827},"last":0.98699999999999999},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.76800000000000002,"15min":0.79200000000000004},"min":{"1min":0.51200000000000001,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.052,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.92100000000000004}]},{"osd":7,"last update":"Fri Oct 6 02:29:09 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.80500000000000005,"15min":0.876},"min":{"1min":0.55100000000000005,"5min":0.499,"15min":0.499},"max":{"1min":1.0780000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.78400000000000003,"15min":0.82799999999999996},"min":{"1min":0.55700000000000005,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.125,"5min":1.125,"15min":1.1479999999999999},"last":0.85699999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574932,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302864,"kb_used_data":1480,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068976,"statfs":{"total":96636764160,"available":96326631424,"internally_reserved":0,"allocated":1515520,"data_stored":698059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:29:08 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.45100000000000001,"15min":0.504},"min":{"1min":0.26000000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.60399999999999998,"5min":1.0489999999999999,"15min":2.024},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.48499999999999999,"15min":0.51900000000000002},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.63,"5min":0.84699999999999998,"15min":2.181},"last":0.41199999999999998}]},{"osd":1,"last update":"Fri Oct 6 02:29:08 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.46300000000000002,"15min":0.66100000000000003},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.59099999999999997,"5min":0.66100000000000003,"15min":4.492},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.52800000000000002,"15min":0.70799999999999996},"min":{"1min":0.34100000000000003,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.72799999999999998,"5min":1.079,"15min":4.569},"last":0.53400000000000003}]},{"osd":2,"last update":"Fri Oct 6 02:29:08 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.624,"15min":0.66100000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.76600000000000001,"5min":1.024,"15min":2.0870000000000002},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.58399999999999996,"15min":0.64600000000000002},"min":{"1min":0.41699999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.753,"5min":0.94199999999999995,"15min":2.4260000000000002},"last":0.76800000000000002}]},{"osd":4,"last update":"Fri Oct 6 02:29:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.72999999999999998,"15min":0.81999999999999995},"min":{"1min":0.56799999999999995,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.94099999999999995,"5min":1.0149999999999999,"15min":2.214},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.749,"15min":0.80500000000000005},"min":{"1min":0.51800000000000002,"5min":0.5,"15min":0.5},"max":{"1min":0.95799999999999996,"5min":1.079,"15min":1.2869999999999999},"last":0.94999999999999996}]},{"osd":5,"last update":"Fri Oct 6 02:29:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.76200000000000001,"15min":0.81399999999999995},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.98399999999999999,"5min":1.0780000000000001,"15min":1.3029999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.75,"15min":0.84499999999999997},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.94999999999999996,"5min":1.0580000000000001,"15min":2.242},"last":0.78500000000000003}]},{"osd":6,"last update":"Fri Oct 6 02:29:08 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.76200000000000001,"15min":0.81399999999999995},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.038,"5min":1.0589999999999999,"15min":1.357},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.79400000000000004,"15min":0.82899999999999996},"min":{"1min":0.54700000000000004,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.94599999999999995,"5min":1.123,"15min":1.3100000000000001},"last":0.70499999999999996}]},{"osd":7,"last update":"Fri Oct 6 02:29:08 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.78100000000000003,"15min":0.85299999999999998},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.004,"5min":1.077,"15min":2.27},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.79400000000000004,"15min":0.85199999999999998},"min":{"1min":0.55800000000000005,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.0329999999999999,"5min":1.105,"15min":1.325},"last":0.68700000000000006}]}]},{"osd":4,"up_from":13,"seq":55834574934,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302848,"kb_used_data":1464,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068992,"statfs":{"total":96636764160,"available":96326647808,"internally_reserved":0,"allocated":1499136,"data_stored":696453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:29:06 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.66900000000000004,"15min":0.98899999999999999},"min":{"1min":0.48499999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":12.337},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.67400000000000004,"15min":0.90400000000000003},"min":{"1min":0.45100000000000001,"5min":0.438,"15min":0.438},"max":{"1min":0.98699999999999999,"5min":1.3520000000000001,"15min":9.6539999999999999},"last":0.52600000000000002}]},{"osd":1,"last update":"Fri Oct 6 02:29:06 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.68000000000000005,"15min":0.69899999999999995},"min":{"1min":0.48499999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.129,"5min":1.129,"15min":3.181},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.751,"5min":0.68999999999999995,"15min":1.012},"min":{"1min":0.52600000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.995,"5min":0.995,"15min":12.946999999999999},"last":0.57499999999999996}]},{"osd":2,"last update":"Fri Oct 6 02:29:06 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.70599999999999996,"15min":1.04},"min":{"1min":0.57299999999999995,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.89400000000000002,"5min":1.2989999999999999,"15min":12.377000000000001},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.72299999999999998,"15min":0.86899999999999999},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":6.3220000000000001},"last":0.86899999999999999}]},{"osd":3,"last update":"Fri Oct 6 02:29:06 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.72599999999999998,"15min":0.98199999999999998},"min":{"1min":0.55000000000000004,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.96599999999999997,"5min":1.5229999999999999,"15min":9.7720000000000002},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.70999999999999996,"15min":0.77400000000000002},"min":{"1min":0.53300000000000003,"5min":0.498,"15min":0.47999999999999998},"max":{"1min":1.212,"5min":1.212,"15min":3.0840000000000001},"last":0.79400000000000004}]},{"osd":5,"last update":"Fri Oct 6 02:29:06 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.68300000000000005,"15min":0.84299999999999997},"min":{"1min":0.40400000000000003,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.006,"5min":1.02,"15min":6.2690000000000001},"last":0.99199999999999999},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.70299999999999996,"15min":1.0269999999999999},"min":{"1min":0.53500000000000003,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.0700000000000001,"5min":1.4359999999999999,"15min":10.769},"last":0.76800000000000002}]},{"osd":6,"last update":"Fri Oct 6 02:29:06 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.68200000000000005,"15min":0.76000000000000001},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.024,"5min":1.4590000000000001,"15min":3.1349999999999998},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.75,"15min":0.96799999999999997},"min":{"1min":0.47799999999999998,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.103,"5min":1.3839999999999999,"15min":8.0899999999999999},"last":1.0329999999999999}]},{"osd":7,"last update":"Fri Oct 6 02:29:06 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.76300000000000001,"15min":1.0589999999999999},"min":{"1min":0.501,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.1599999999999999,"5min":1.494,"15min":9.7089999999999996},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.75900000000000001,"15min":0.99099999999999999},"min":{"1min":0.56399999999999995,"5min":0.374,"15min":0.374},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":7.6719999999999997},"last":0.89500000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574934,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302852,"kb_used_data":1468,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068988,"statfs":{"total":96636764160,"available":96326643712,"internally_reserved":0,"allocated":1503232,"data_stored":696475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:29:03 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.64500000000000002,"15min":0.627},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.97599999999999998,"5min":1.016,"15min":1.4279999999999999},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.66600000000000004,"15min":0.61399999999999999},"min":{"1min":0.443,"5min":0.41699999999999998,"15min":0.38700000000000001},"max":{"1min":0.92900000000000005,"5min":1.099,"15min":1.099},"last":0.79200000000000004}]},{"osd":1,"last update":"Fri Oct 6 02:29:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.67700000000000005,"15min":0.65300000000000002},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.004,"5min":1.0660000000000001,"15min":1.089},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.68799999999999994,"15min":0.67400000000000004},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40500000000000003},"max":{"1min":0.95699999999999996,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.52500000000000002}]},{"osd":2,"last update":"Fri Oct 6 02:29:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.70299999999999996,"15min":0.67100000000000004},"min":{"1min":0.48299999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.98799999999999999,"5min":1.016,"15min":1.016},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.70399999999999996,"15min":0.68400000000000005},"min":{"1min":0.52000000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.91800000000000004,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.495}]},{"osd":3,"last update":"Fri Oct 6 02:29:03 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.75800000000000001,"15min":0.72999999999999998},"min":{"1min":0.53200000000000003,"5min":0.503,"15min":0.48799999999999999},"max":{"1min":1.052,"5min":1.159,"15min":1.159},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.79700000000000004,"15min":0.76900000000000002},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.54100000000000004},"max":{"1min":0.97599999999999998,"5min":1.194,"15min":1.194},"last":0.75600000000000001}]},{"osd":4,"last update":"Fri Oct 6 02:29:03 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.66900000000000004,"15min":0.67500000000000004},"min":{"1min":0.42899999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.012,"5min":1.038,"15min":1.1499999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.67600000000000005,"15min":0.68400000000000005},"min":{"1min":0.45600000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.99299999999999999,"5min":1.25,"15min":1.25},"last":0.67900000000000005}]},{"osd":6,"last update":"Fri Oct 6 02:29:03 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.745,"15min":0.745},"min":{"1min":0.47799999999999998,"5min":0.45700000000000002,"15min":0.40300000000000002},"max":{"1min":0.95299999999999996,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.81200000000000006,"15min":0.81999999999999995},"min":{"1min":0.56999999999999995,"5min":0.373,"15min":0.373},"max":{"1min":1.1080000000000001,"5min":1.1200000000000001,"15min":1.2170000000000001},"last":0.93899999999999995}]},{"osd":7,"last update":"Fri Oct 6 02:29:03 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.76400000000000001,"15min":0.76500000000000001},"min":{"1min":0.503,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.97399999999999998,"5min":1.23,"15min":1.23},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.80800000000000005,"15min":0.83199999999999996},"min":{"1min":0.54800000000000004,"5min":0.437,"15min":0.437},"max":{"1min":1.0660000000000001,"5min":1.2050000000000001,"15min":1.5640000000000001},"last":0.90300000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T02:29:40.556 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T02:29:40.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.776+0000 7f58289fe700 1 Processor -- start 2023-10-06T02:29:40.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.776+0000 7f58289fe700 1 -- start start 2023-10-06T02:29:40.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.778+0000 7f58289fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f582413fef0 0x7f582415a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:40.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.778+0000 7f58289fe700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f582406aaa0 0x7f58240c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:40.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.778+0000 7f58289fe700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f582415ae70 con 0x7f582413fb20 2023-10-06T02:29:40.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.778+0000 7f58289fe700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f582406af60 con 0x7f582406aaa0 2023-10-06T02:29:40.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.778+0000 7f58289fe700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f582406b240 con 0x7f582413fef0 2023-10-06T02:29:40.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.778+0000 7f5822d9d700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f582413fb20 0x7f58240c7990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:54058/0 (socket says 172.21.15.31:54058) 2023-10-06T02:29:40.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.778+0000 7f5822d9d700 1 -- 172.21.15.31:0/2456934353 learned_addr learned my addr 172.21.15.31:0/2456934353 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:29:40.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.778+0000 7f582359e700 1 --2- 172.21.15.31:0/2456934353 >> v2:172.21.15.62:3300/0 conn(0x7f582406aaa0 0x7f58240c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:40.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.778+0000 7f582259c700 1 --2- 172.21.15.31:0/2456934353 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f582413fef0 0x7f582415a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:40.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.778+0000 7f5821d9b700 1 -- 172.21.15.31:0/2456934353 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3932414144 0 0) 0x7f5814001110 con 0x7f582413fb20 2023-10-06T02:29:40.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.778+0000 7f5821d9b700 1 -- 172.21.15.31:0/2456934353 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f580c003570 con 0x7f582413fb20 2023-10-06T02:29:40.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.779+0000 7f5821d9b700 1 -- 172.21.15.31:0/2456934353 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 138093144 0 0) 0x7f580c003570 con 0x7f582413fb20 2023-10-06T02:29:40.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.779+0000 7f5821d9b700 1 -- 172.21.15.31:0/2456934353 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f582413fef0 msgr2=0x7f582415a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:40.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.779+0000 7f5821d9b700 1 --2- 172.21.15.31:0/2456934353 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f582413fef0 0x7f582415a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:40.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.779+0000 7f5821d9b700 1 -- 172.21.15.31:0/2456934353 >> v2:172.21.15.62:3300/0 conn(0x7f582406aaa0 msgr2=0x7f58240c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:40.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.779+0000 7f5821d9b700 1 --2- 172.21.15.31:0/2456934353 >> v2:172.21.15.62:3300/0 conn(0x7f582406aaa0 0x7f58240c9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:40.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.779+0000 7f5821d9b700 1 -- 172.21.15.31:0/2456934353 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f582406b520 con 0x7f582413fb20 2023-10-06T02:29:40.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.779+0000 7f5821d9b700 1 -- 172.21.15.31:0/2456934353 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7f5814001600 con 0x7f582413fb20 2023-10-06T02:29:40.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.779+0000 7f5821d9b700 1 -- 172.21.15.31:0/2456934353 >> v1:172.21.15.31:6789/0 conn(0x7f582413fb20 legacy=0x7f58240c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:40.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.779+0000 7f5821d9b700 1 --2- 172.21.15.31:0/2456934353 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f580c003ae0 0x7f580c003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:40.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.779+0000 7f5821d9b700 1 --2- 172.21.15.31:0/2456934353 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f580c004810 0x7f580c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:40.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.779+0000 7f5821d9b700 1 --2- 172.21.15.31:0/2456934353 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f582413fef0 0x7f580c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:40.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.779+0000 7f5821d9b700 1 -- 172.21.15.31:0/2456934353 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f582406af60 con 0x7f580c004810 2023-10-06T02:29:40.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.779+0000 7f5821d9b700 1 -- 172.21.15.31:0/2456934353 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f582406b240 con 0x7f580c003ae0 2023-10-06T02:29:40.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.779+0000 7f5821d9b700 1 -- 172.21.15.31:0/2456934353 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f580c00bb40 con 0x7f582413fef0 2023-10-06T02:29:40.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.779+0000 7f582259c700 1 --2- 172.21.15.31:0/2456934353 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f580c004810 0x7f580c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:40.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.779+0000 7f5822d9d700 1 --2- 172.21.15.31:0/2456934353 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f580c003ae0 0x7f580c003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:40.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.779+0000 7f582359e700 1 --2- 172.21.15.31:0/2456934353 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f582413fef0 0x7f580c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:40.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.780+0000 7f582259c700 1 -- 172.21.15.31:0/2456934353 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f582413fef0 msgr2=0x7f580c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:40.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.780+0000 7f582259c700 1 --2- 172.21.15.31:0/2456934353 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f582413fef0 0x7f580c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:40.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.780+0000 7f582259c700 1 -- 172.21.15.31:0/2456934353 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f580c003ae0 msgr2=0x7f580c003ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:40.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.780+0000 7f582259c700 1 --2- 172.21.15.31:0/2456934353 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f580c003ae0 0x7f580c003ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:40.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.780+0000 7f582259c700 1 -- 172.21.15.31:0/2456934353 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f580c003570 con 0x7f580c004810 2023-10-06T02:29:40.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.780+0000 7f582259c700 1 --2- 172.21.15.31:0/2456934353 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f580c004810 0x7f580c004c80 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f5824069580 tx=0x7f5818009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41d9b1146e8bb5ec server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:40.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.780+0000 7f5821d9b700 1 -- 172.21.15.31:0/2456934353 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5818005a00 con 0x7f580c004810 2023-10-06T02:29:40.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.780+0000 7f5821d9b700 1 -- 172.21.15.31:0/2456934353 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5818005b60 con 0x7f580c004810 2023-10-06T02:29:40.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.780+0000 7f5821d9b700 1 -- 172.21.15.31:0/2456934353 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f581800e640 con 0x7f580c004810 2023-10-06T02:29:40.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.781+0000 7f58289fe700 1 -- 172.21.15.31:0/2456934353 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f580c004810 msgr2=0x7f580c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:40.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.781+0000 7f58289fe700 1 --2- 172.21.15.31:0/2456934353 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f580c004810 0x7f580c004c80 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f5824069580 tx=0x7f5818009120 comp rx=0 tx=0).stop 2023-10-06T02:29:40.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.781+0000 7f582359e700 1 -- 172.21.15.31:0/2456934353 reap_dead start 2023-10-06T02:29:40.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.781+0000 7f58289fe700 1 -- 172.21.15.31:0/2456934353 shutdown_connections 2023-10-06T02:29:40.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.781+0000 7f58289fe700 1 -- 172.21.15.31:0/2456934353 >> 172.21.15.31:0/2456934353 conn(0x7f582406d170 msgr2=0x7f58240cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:40.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.781+0000 7f58289fe700 1 -- 172.21.15.31:0/2456934353 shutdown_connections 2023-10-06T02:29:40.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.781+0000 7f58289fe700 1 -- 172.21.15.31:0/2456934353 wait complete. 2023-10-06T02:29:40.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.782+0000 7f58289fe700 1 Processor -- start 2023-10-06T02:29:40.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.782+0000 7f58289fe700 1 -- start start 2023-10-06T02:29:40.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.783+0000 7f58289fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f580c004810 0x7f5824159550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:40.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.783+0000 7f58289fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5824155d80 0x7f58240ad270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:40.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.783+0000 7f58289fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58241561f0 0x7f58240ad7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:40.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.783+0000 7f58289fe700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58240ab740 con 0x7f5824155d80 2023-10-06T02:29:40.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.783+0000 7f58289fe700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58240ab990 con 0x7f58241561f0 2023-10-06T02:29:40.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.783+0000 7f58289fe700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58240abc70 con 0x7f580c004810 2023-10-06T02:29:40.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.784+0000 7f582259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5824155d80 0x7f58240ad270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:40.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.784+0000 7f5822d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f580c004810 0x7f5824159550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:40.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.784+0000 7f582259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5824155d80 0x7f58240ad270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55412/0 (socket says 172.21.15.31:55412) 2023-10-06T02:29:40.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.784+0000 7f582259c700 1 -- 172.21.15.31:0/2408130715 learned_addr learned my addr 172.21.15.31:0/2408130715 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:40.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.784+0000 7f582359e700 1 --2- 172.21.15.31:0/2408130715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58241561f0 0x7f58240ad7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:40.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.784+0000 7f582259c700 1 -- 172.21.15.31:0/2408130715 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f580c004810 msgr2=0x7f5824159550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:40.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.784+0000 7f582259c700 1 --2- 172.21.15.31:0/2408130715 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f580c004810 0x7f5824159550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:40.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.784+0000 7f582259c700 1 -- 172.21.15.31:0/2408130715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58241561f0 msgr2=0x7f58240ad7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:40.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.784+0000 7f582259c700 1 --2- 172.21.15.31:0/2408130715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58241561f0 0x7f58240ad7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:40.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.784+0000 7f582259c700 1 -- 172.21.15.31:0/2408130715 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58240abf50 con 0x7f5824155d80 2023-10-06T02:29:40.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.784+0000 7f582259c700 1 --2- 172.21.15.31:0/2408130715 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5824155d80 0x7f58240ad270 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f580c00ba90 tx=0x7f5818019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:40.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.785+0000 7f580affd700 1 -- 172.21.15.31:0/2408130715 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f581800eeb0 con 0x7f5824155d80 2023-10-06T02:29:40.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.785+0000 7f580affd700 1 -- 172.21.15.31:0/2408130715 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5818006240 con 0x7f5824155d80 2023-10-06T02:29:40.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.785+0000 7f58289fe700 1 -- 172.21.15.31:0/2408130715 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58240ac230 con 0x7f5824155d80 2023-10-06T02:29:40.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.785+0000 7f580affd700 1 -- 172.21.15.31:0/2408130715 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5818006600 con 0x7f5824155d80 2023-10-06T02:29:40.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.785+0000 7f58289fe700 1 -- 172.21.15.31:0/2408130715 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5824159a90 con 0x7f5824155d80 2023-10-06T02:29:40.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.786+0000 7f58289fe700 1 -- 172.21.15.31:0/2408130715 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58240ac880 con 0x7f5824155d80 2023-10-06T02:29:40.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.786+0000 7f580affd700 1 -- 172.21.15.31:0/2408130715 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f5818013470 con 0x7f5824155d80 2023-10-06T02:29:40.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.787+0000 7f580affd700 1 --2- 172.21.15.31:0/2408130715 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5810067320 0x7f58100697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:40.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.787+0000 7f580affd700 1 -- 172.21.15.31:0/2408130715 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f5818086150 con 0x7f5824155d80 2023-10-06T02:29:40.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.788+0000 7f5822d9d700 1 --2- 172.21.15.31:0/2408130715 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5810067320 0x7f58100697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:40.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.788+0000 7f5822d9d700 1 --2- 172.21.15.31:0/2408130715 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5810067320 0x7f58100697d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5814007e90 tx=0x7f58140079d0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:40.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.791+0000 7f580affd700 1 -- 172.21.15.31:0/2408130715 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5818052d90 con 0x7f5824155d80 2023-10-06T02:29:40.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:40.990+0000 7f58289fe700 1 -- 172.21.15.31:0/2408130715 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f58240ab740 con 0x7f5810067320 2023-10-06T02:29:41.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.000+0000 7f580affd700 1 -- 172.21.15.31:0/2408130715 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311338 (secure 0 0 0) 0x7f58240ab740 con 0x7f5810067320 2023-10-06T02:29:41.005 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:29:41.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.004+0000 7f58289fe700 1 -- 172.21.15.31:0/2408130715 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5810067320 msgr2=0x7f58100697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:41.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.004+0000 7f58289fe700 1 --2- 172.21.15.31:0/2408130715 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5810067320 0x7f58100697d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5814007e90 tx=0x7f58140079d0 comp rx=0 tx=0).stop 2023-10-06T02:29:41.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.004+0000 7f58289fe700 1 -- 172.21.15.31:0/2408130715 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5824155d80 msgr2=0x7f58240ad270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:41.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.004+0000 7f58289fe700 1 --2- 172.21.15.31:0/2408130715 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5824155d80 0x7f58240ad270 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f580c00ba90 tx=0x7f5818019000 comp rx=0 tx=0).stop 2023-10-06T02:29:41.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.004+0000 7f58289fe700 1 -- 172.21.15.31:0/2408130715 shutdown_connections 2023-10-06T02:29:41.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.004+0000 7f58289fe700 1 --2- 172.21.15.31:0/2408130715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58241561f0 0x7f58240ad7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:41.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.004+0000 7f58289fe700 1 --2- 172.21.15.31:0/2408130715 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5810067320 0x7f58100697d0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:41.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.004+0000 7f58289fe700 1 --2- 172.21.15.31:0/2408130715 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f580c004810 0x7f5824159550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:41.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.004+0000 7f58289fe700 1 --2- 172.21.15.31:0/2408130715 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5824155d80 0x7f58240ad270 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:41.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.004+0000 7f58289fe700 1 -- 172.21.15.31:0/2408130715 >> 172.21.15.31:0/2408130715 conn(0x7f582406d170 msgr2=0x7f58240b1770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:41.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.005+0000 7f58289fe700 1 -- 172.21.15.31:0/2408130715 shutdown_connections 2023-10-06T02:29:41.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.005+0000 7f58289fe700 1 -- 172.21.15.31:0/2408130715 wait complete. 2023-10-06T02:29:41.012 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-10-06T02:29:41.023 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":109,"stamp":"2023-10-06T02:29:40.482535+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":224,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":167,"ondisk_log_size":167,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":169,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2424008,"kb_used_data":12936,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752550712,"statfs":{"total":773094113280,"available":770611929088,"internally_reserved":0,"allocated":13246464,"data_stored":6756972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001634"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.181634+0000","last_change":"2023-10-06T02:29:36.180448+0000","last_active":"2023-10-06T02:29:36.181634+0000","last_peered":"2023-10-06T02:29:36.181634+0000","last_clean":"2023-10-06T02:29:36.181634+0000","last_became_active":"2023-10-06T02:29:36.180182+0000","last_became_peered":"2023-10-06T02:29:36.180182+0000","last_unstale":"2023-10-06T02:29:36.181634+0000","last_undegraded":"2023-10-06T02:29:36.181634+0000","last_fullsized":"2023-10-06T02:29:36.181634+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:13:33.504425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.191454+0000","last_change":"2023-10-06T02:29:36.185203+0000","last_active":"2023-10-06T02:29:36.191454+0000","last_peered":"2023-10-06T02:29:36.191454+0000","last_clean":"2023-10-06T02:29:36.191454+0000","last_became_active":"2023-10-06T02:29:36.184889+0000","last_became_peered":"2023-10-06T02:29:36.184889+0000","last_unstale":"2023-10-06T02:29:36.191454+0000","last_undegraded":"2023-10-06T02:29:36.191454+0000","last_fullsized":"2023-10-06T02:29:36.191454+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:10:07.892520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.196410+0000","last_change":"2023-10-06T02:29:36.189362+0000","last_active":"2023-10-06T02:29:36.196410+0000","last_peered":"2023-10-06T02:29:36.196410+0000","last_clean":"2023-10-06T02:29:36.196410+0000","last_became_active":"2023-10-06T02:29:36.188505+0000","last_became_peered":"2023-10-06T02:29:36.188505+0000","last_unstale":"2023-10-06T02:29:36.196410+0000","last_undegraded":"2023-10-06T02:29:36.196410+0000","last_fullsized":"2023-10-06T02:29:36.196410+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:14:52.577434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.191521+0000","last_change":"2023-10-06T02:29:36.186503+0000","last_active":"2023-10-06T02:29:36.191521+0000","last_peered":"2023-10-06T02:29:36.191521+0000","last_clean":"2023-10-06T02:29:36.191521+0000","last_became_active":"2023-10-06T02:29:36.186137+0000","last_became_peered":"2023-10-06T02:29:36.186137+0000","last_unstale":"2023-10-06T02:29:36.191521+0000","last_undegraded":"2023-10-06T02:29:36.191521+0000","last_fullsized":"2023-10-06T02:29:36.191521+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:28:27.243954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.197725+0000","last_change":"2023-10-06T02:29:36.189063+0000","last_active":"2023-10-06T02:29:36.197725+0000","last_peered":"2023-10-06T02:29:36.197725+0000","last_clean":"2023-10-06T02:29:36.197725+0000","last_became_active":"2023-10-06T02:29:36.188758+0000","last_became_peered":"2023-10-06T02:29:36.188758+0000","last_unstale":"2023-10-06T02:29:36.197725+0000","last_undegraded":"2023-10-06T02:29:36.197725+0000","last_fullsized":"2023-10-06T02:29:36.197725+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:14:59.301511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.202542+0000","last_change":"2023-10-06T02:29:36.201440+0000","last_active":"2023-10-06T02:29:36.202542+0000","last_peered":"2023-10-06T02:29:36.202542+0000","last_clean":"2023-10-06T02:29:36.202542+0000","last_became_active":"2023-10-06T02:29:36.199748+0000","last_became_peered":"2023-10-06T02:29:36.199748+0000","last_unstale":"2023-10-06T02:29:36.202542+0000","last_undegraded":"2023-10-06T02:29:36.202542+0000","last_fullsized":"2023-10-06T02:29:36.202542+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:28:27.243954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.205145+0000","last_change":"2023-10-06T02:29:36.201694+0000","last_active":"2023-10-06T02:29:36.205145+0000","last_peered":"2023-10-06T02:29:36.205145+0000","last_clean":"2023-10-06T02:29:36.205145+0000","last_became_active":"2023-10-06T02:29:36.199613+0000","last_became_peered":"2023-10-06T02:29:36.199613+0000","last_unstale":"2023-10-06T02:29:36.205145+0000","last_undegraded":"2023-10-06T02:29:36.205145+0000","last_fullsized":"2023-10-06T02:29:36.205145+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:11:52.582273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.199056+0000","last_change":"2023-10-06T02:29:36.198176+0000","last_active":"2023-10-06T02:29:36.199056+0000","last_peered":"2023-10-06T02:29:36.199056+0000","last_clean":"2023-10-06T02:29:36.199056+0000","last_became_active":"2023-10-06T02:29:36.197788+0000","last_became_peered":"2023-10-06T02:29:36.197788+0000","last_unstale":"2023-10-06T02:29:36.199056+0000","last_undegraded":"2023-10-06T02:29:36.199056+0000","last_fullsized":"2023-10-06T02:29:36.199056+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:02:43.415532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.196802+0000","last_change":"2023-10-06T02:29:36.195172+0000","last_active":"2023-10-06T02:29:36.196802+0000","last_peered":"2023-10-06T02:29:36.196802+0000","last_clean":"2023-10-06T02:29:36.196802+0000","last_became_active":"2023-10-06T02:29:36.194899+0000","last_became_peered":"2023-10-06T02:29:36.194899+0000","last_unstale":"2023-10-06T02:29:36.196802+0000","last_undegraded":"2023-10-06T02:29:36.196802+0000","last_fullsized":"2023-10-06T02:29:36.196802+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:19:49.845748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.189203+0000","last_change":"2023-10-06T02:29:36.185097+0000","last_active":"2023-10-06T02:29:36.189203+0000","last_peered":"2023-10-06T02:29:36.189203+0000","last_clean":"2023-10-06T02:29:36.189203+0000","last_became_active":"2023-10-06T02:29:36.184831+0000","last_became_peered":"2023-10-06T02:29:36.184831+0000","last_unstale":"2023-10-06T02:29:36.189203+0000","last_undegraded":"2023-10-06T02:29:36.189203+0000","last_fullsized":"2023-10-06T02:29:36.189203+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:50:11.198660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.196296+0000","last_change":"2023-10-06T02:29:36.192247+0000","last_active":"2023-10-06T02:29:36.196296+0000","last_peered":"2023-10-06T02:29:36.196296+0000","last_clean":"2023-10-06T02:29:36.196296+0000","last_became_active":"2023-10-06T02:29:36.191926+0000","last_became_peered":"2023-10-06T02:29:36.191926+0000","last_unstale":"2023-10-06T02:29:36.196296+0000","last_undegraded":"2023-10-06T02:29:36.196296+0000","last_fullsized":"2023-10-06T02:29:36.196296+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:23:38.004436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.181311+0000","last_change":"2023-10-06T02:29:36.179316+0000","last_active":"2023-10-06T02:29:36.181311+0000","last_peered":"2023-10-06T02:29:36.181311+0000","last_clean":"2023-10-06T02:29:36.181311+0000","last_became_active":"2023-10-06T02:29:36.179058+0000","last_became_peered":"2023-10-06T02:29:36.179058+0000","last_unstale":"2023-10-06T02:29:36.181311+0000","last_undegraded":"2023-10-06T02:29:36.181311+0000","last_fullsized":"2023-10-06T02:29:36.181311+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:46:32.857433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.202541+0000","last_change":"2023-10-06T02:29:36.201465+0000","last_active":"2023-10-06T02:29:36.202541+0000","last_peered":"2023-10-06T02:29:36.202541+0000","last_clean":"2023-10-06T02:29:36.202541+0000","last_became_active":"2023-10-06T02:29:36.201182+0000","last_became_peered":"2023-10-06T02:29:36.201182+0000","last_unstale":"2023-10-06T02:29:36.202541+0000","last_undegraded":"2023-10-06T02:29:36.202541+0000","last_fullsized":"2023-10-06T02:29:36.202541+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:05:07.231083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.194786+0000","last_change":"2023-10-06T02:29:36.192619+0000","last_active":"2023-10-06T02:29:36.194786+0000","last_peered":"2023-10-06T02:29:36.194786+0000","last_clean":"2023-10-06T02:29:36.194786+0000","last_became_active":"2023-10-06T02:29:36.192345+0000","last_became_peered":"2023-10-06T02:29:36.192345+0000","last_unstale":"2023-10-06T02:29:36.194786+0000","last_undegraded":"2023-10-06T02:29:36.194786+0000","last_fullsized":"2023-10-06T02:29:36.194786+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:48:20.630360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.197308+0000","last_change":"2023-10-06T02:29:36.193299+0000","last_active":"2023-10-06T02:29:36.197308+0000","last_peered":"2023-10-06T02:29:36.197308+0000","last_clean":"2023-10-06T02:29:36.197308+0000","last_became_active":"2023-10-06T02:29:36.192936+0000","last_became_peered":"2023-10-06T02:29:36.192936+0000","last_unstale":"2023-10-06T02:29:36.197308+0000","last_undegraded":"2023-10-06T02:29:36.197308+0000","last_fullsized":"2023-10-06T02:29:36.197308+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:31:14.878470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.193227+0000","last_change":"2023-10-06T02:29:36.189557+0000","last_active":"2023-10-06T02:29:36.193227+0000","last_peered":"2023-10-06T02:29:36.193227+0000","last_clean":"2023-10-06T02:29:36.193227+0000","last_became_active":"2023-10-06T02:29:36.189276+0000","last_became_peered":"2023-10-06T02:29:36.189276+0000","last_unstale":"2023-10-06T02:29:36.193227+0000","last_undegraded":"2023-10-06T02:29:36.193227+0000","last_fullsized":"2023-10-06T02:29:36.193227+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:50:05.271603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.193829+0000","last_change":"2023-10-06T02:29:36.190332+0000","last_active":"2023-10-06T02:29:36.193829+0000","last_peered":"2023-10-06T02:29:36.193829+0000","last_clean":"2023-10-06T02:29:36.193829+0000","last_became_active":"2023-10-06T02:29:36.189914+0000","last_became_peered":"2023-10-06T02:29:36.189914+0000","last_unstale":"2023-10-06T02:29:36.193829+0000","last_undegraded":"2023-10-06T02:29:36.193829+0000","last_fullsized":"2023-10-06T02:29:36.193829+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:36:47.544763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.188447+0000","last_change":"2023-10-06T02:29:36.185191+0000","last_active":"2023-10-06T02:29:36.188447+0000","last_peered":"2023-10-06T02:29:36.188447+0000","last_clean":"2023-10-06T02:29:36.188447+0000","last_became_active":"2023-10-06T02:29:36.184910+0000","last_became_peered":"2023-10-06T02:29:36.184910+0000","last_unstale":"2023-10-06T02:29:36.188447+0000","last_undegraded":"2023-10-06T02:29:36.188447+0000","last_fullsized":"2023-10-06T02:29:36.188447+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:40:48.871334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.204007+0000","last_change":"2023-10-06T02:29:36.193886+0000","last_active":"2023-10-06T02:29:36.204007+0000","last_peered":"2023-10-06T02:29:36.204007+0000","last_clean":"2023-10-06T02:29:36.204007+0000","last_became_active":"2023-10-06T02:29:36.191083+0000","last_became_peered":"2023-10-06T02:29:36.191083+0000","last_unstale":"2023-10-06T02:29:36.204007+0000","last_undegraded":"2023-10-06T02:29:36.204007+0000","last_fullsized":"2023-10-06T02:29:36.204007+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:33:36.484989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.200470+0000","last_change":"2023-10-06T02:29:36.198198+0000","last_active":"2023-10-06T02:29:36.200470+0000","last_peered":"2023-10-06T02:29:36.200470+0000","last_clean":"2023-10-06T02:29:36.200470+0000","last_became_active":"2023-10-06T02:29:36.197950+0000","last_became_peered":"2023-10-06T02:29:36.197950+0000","last_unstale":"2023-10-06T02:29:36.200470+0000","last_undegraded":"2023-10-06T02:29:36.200470+0000","last_fullsized":"2023-10-06T02:29:36.200470+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:34:31.991692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.196349+0000","last_change":"2023-10-06T02:29:36.190553+0000","last_active":"2023-10-06T02:29:36.196349+0000","last_peered":"2023-10-06T02:29:36.196349+0000","last_clean":"2023-10-06T02:29:36.196349+0000","last_became_active":"2023-10-06T02:29:36.190253+0000","last_became_peered":"2023-10-06T02:29:36.190253+0000","last_unstale":"2023-10-06T02:29:36.196349+0000","last_undegraded":"2023-10-06T02:29:36.196349+0000","last_fullsized":"2023-10-06T02:29:36.196349+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:05:07.231083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.180983+0000","last_change":"2023-10-06T02:29:36.176630+0000","last_active":"2023-10-06T02:29:36.180983+0000","last_peered":"2023-10-06T02:29:36.180983+0000","last_clean":"2023-10-06T02:29:36.180983+0000","last_became_active":"2023-10-06T02:29:36.176301+0000","last_became_peered":"2023-10-06T02:29:36.176301+0000","last_unstale":"2023-10-06T02:29:36.180983+0000","last_undegraded":"2023-10-06T02:29:36.180983+0000","last_fullsized":"2023-10-06T02:29:36.180983+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:38:29.470165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.197329+0000","last_change":"2023-10-06T02:29:36.192712+0000","last_active":"2023-10-06T02:29:36.197329+0000","last_peered":"2023-10-06T02:29:36.197329+0000","last_clean":"2023-10-06T02:29:36.197329+0000","last_became_active":"2023-10-06T02:29:36.192439+0000","last_became_peered":"2023-10-06T02:29:36.192439+0000","last_unstale":"2023-10-06T02:29:36.197329+0000","last_undegraded":"2023-10-06T02:29:36.197329+0000","last_fullsized":"2023-10-06T02:29:36.197329+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:50:05.271603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.193780+0000","last_change":"2023-10-06T02:29:36.192232+0000","last_active":"2023-10-06T02:29:36.193780+0000","last_peered":"2023-10-06T02:29:36.193780+0000","last_clean":"2023-10-06T02:29:36.193780+0000","last_became_active":"2023-10-06T02:29:36.191969+0000","last_became_peered":"2023-10-06T02:29:36.191969+0000","last_unstale":"2023-10-06T02:29:36.193780+0000","last_undegraded":"2023-10-06T02:29:36.193780+0000","last_fullsized":"2023-10-06T02:29:36.193780+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:01:04.373806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.198281+0000","last_change":"2023-10-06T02:29:36.189286+0000","last_active":"2023-10-06T02:29:36.198281+0000","last_peered":"2023-10-06T02:29:36.198281+0000","last_clean":"2023-10-06T02:29:36.198281+0000","last_became_active":"2023-10-06T02:29:36.188437+0000","last_became_peered":"2023-10-06T02:29:36.188437+0000","last_unstale":"2023-10-06T02:29:36.198281+0000","last_undegraded":"2023-10-06T02:29:36.198281+0000","last_fullsized":"2023-10-06T02:29:36.198281+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:21:14.154163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.203339+0000","last_change":"2023-10-06T02:29:36.196695+0000","last_active":"2023-10-06T02:29:36.203339+0000","last_peered":"2023-10-06T02:29:36.203339+0000","last_clean":"2023-10-06T02:29:36.203339+0000","last_became_active":"2023-10-06T02:29:36.196337+0000","last_became_peered":"2023-10-06T02:29:36.196337+0000","last_unstale":"2023-10-06T02:29:36.203339+0000","last_undegraded":"2023-10-06T02:29:36.203339+0000","last_fullsized":"2023-10-06T02:29:36.203339+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:18:19.492112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.205081+0000","last_change":"2023-10-06T02:29:36.202975+0000","last_active":"2023-10-06T02:29:36.205081+0000","last_peered":"2023-10-06T02:29:36.205081+0000","last_clean":"2023-10-06T02:29:36.205081+0000","last_became_active":"2023-10-06T02:29:36.201854+0000","last_became_peered":"2023-10-06T02:29:36.201854+0000","last_unstale":"2023-10-06T02:29:36.205081+0000","last_undegraded":"2023-10-06T02:29:36.205081+0000","last_fullsized":"2023-10-06T02:29:36.205081+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:43:21.797680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.187675+0000","last_change":"2023-10-06T02:29:36.185022+0000","last_active":"2023-10-06T02:29:36.187675+0000","last_peered":"2023-10-06T02:29:36.187675+0000","last_clean":"2023-10-06T02:29:36.187675+0000","last_became_active":"2023-10-06T02:29:36.184733+0000","last_became_peered":"2023-10-06T02:29:36.184733+0000","last_unstale":"2023-10-06T02:29:36.187675+0000","last_undegraded":"2023-10-06T02:29:36.187675+0000","last_fullsized":"2023-10-06T02:29:36.187675+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:28.316081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.196292+0000","last_change":"2023-10-06T02:29:36.192870+0000","last_active":"2023-10-06T02:29:36.196292+0000","last_peered":"2023-10-06T02:29:36.196292+0000","last_clean":"2023-10-06T02:29:36.196292+0000","last_became_active":"2023-10-06T02:29:36.192604+0000","last_became_peered":"2023-10-06T02:29:36.192604+0000","last_unstale":"2023-10-06T02:29:36.196292+0000","last_undegraded":"2023-10-06T02:29:36.196292+0000","last_fullsized":"2023-10-06T02:29:36.196292+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:45:06.604436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.195036+0000","last_change":"2023-10-06T02:29:36.186963+0000","last_active":"2023-10-06T02:29:36.195036+0000","last_peered":"2023-10-06T02:29:36.195036+0000","last_clean":"2023-10-06T02:29:36.195036+0000","last_became_active":"2023-10-06T02:29:36.186689+0000","last_became_peered":"2023-10-06T02:29:36.186689+0000","last_unstale":"2023-10-06T02:29:36.195036+0000","last_undegraded":"2023-10-06T02:29:36.195036+0000","last_fullsized":"2023-10-06T02:29:36.195036+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:21:34.832535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.197437+0000","last_change":"2023-10-06T02:29:36.189860+0000","last_active":"2023-10-06T02:29:36.197437+0000","last_peered":"2023-10-06T02:29:36.197437+0000","last_clean":"2023-10-06T02:29:36.197437+0000","last_became_active":"2023-10-06T02:29:36.189431+0000","last_became_peered":"2023-10-06T02:29:36.189431+0000","last_unstale":"2023-10-06T02:29:36.197437+0000","last_undegraded":"2023-10-06T02:29:36.197437+0000","last_fullsized":"2023-10-06T02:29:36.197437+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:44:02.050446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.195498+0000","last_change":"2023-10-06T02:29:36.192171+0000","last_active":"2023-10-06T02:29:36.195498+0000","last_peered":"2023-10-06T02:29:36.195498+0000","last_clean":"2023-10-06T02:29:36.195498+0000","last_became_active":"2023-10-06T02:29:36.191906+0000","last_became_peered":"2023-10-06T02:29:36.191906+0000","last_unstale":"2023-10-06T02:29:36.195498+0000","last_undegraded":"2023-10-06T02:29:36.195498+0000","last_fullsized":"2023-10-06T02:29:36.195498+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:26:51.911921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.198283+0000","last_change":"2023-10-06T02:29:36.192293+0000","last_active":"2023-10-06T02:29:36.198283+0000","last_peered":"2023-10-06T02:29:36.198283+0000","last_clean":"2023-10-06T02:29:36.198283+0000","last_became_active":"2023-10-06T02:29:36.192021+0000","last_became_peered":"2023-10-06T02:29:36.192021+0000","last_unstale":"2023-10-06T02:29:36.198283+0000","last_undegraded":"2023-10-06T02:29:36.198283+0000","last_fullsized":"2023-10-06T02:29:36.198283+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:54:48.281815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1f","version":"119'4","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.203852+0000","last_change":"2023-10-06T02:29:35.187018+0000","last_active":"2023-10-06T02:29:36.203852+0000","last_peered":"2023-10-06T02:29:36.203852+0000","last_clean":"2023-10-06T02:29:36.203852+0000","last_became_active":"2023-10-06T02:29:35.186239+0000","last_became_peered":"2023-10-06T02:29:35.186239+0000","last_unstale":"2023-10-06T02:29:36.203852+0000","last_undegraded":"2023-10-06T02:29:36.203852+0000","last_fullsized":"2023-10-06T02:29:36.203852+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_clean_scrub_stamp":"2023-10-06T02:29:34.151749+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:12:51.337692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.180620+0000","last_change":"2023-10-06T02:29:36.175948+0000","last_active":"2023-10-06T02:29:36.180620+0000","last_peered":"2023-10-06T02:29:36.180620+0000","last_clean":"2023-10-06T02:29:36.180620+0000","last_became_active":"2023-10-06T02:29:36.175662+0000","last_became_peered":"2023-10-06T02:29:36.175662+0000","last_unstale":"2023-10-06T02:29:36.180620+0000","last_undegraded":"2023-10-06T02:29:36.180620+0000","last_fullsized":"2023-10-06T02:29:36.180620+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:55:24.441198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1e","version":"119'1","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.195630+0000","last_change":"2023-10-06T02:29:35.186934+0000","last_active":"2023-10-06T02:29:36.195630+0000","last_peered":"2023-10-06T02:29:36.195630+0000","last_clean":"2023-10-06T02:29:36.195630+0000","last_became_active":"2023-10-06T02:29:35.186673+0000","last_became_peered":"2023-10-06T02:29:35.186673+0000","last_unstale":"2023-10-06T02:29:36.195630+0000","last_undegraded":"2023-10-06T02:29:36.195630+0000","last_fullsized":"2023-10-06T02:29:36.195630+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_clean_scrub_stamp":"2023-10-06T02:29:34.151749+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:02:42.408863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.196247+0000","last_change":"2023-10-06T02:29:36.192264+0000","last_active":"2023-10-06T02:29:36.196247+0000","last_peered":"2023-10-06T02:29:36.196247+0000","last_clean":"2023-10-06T02:29:36.196247+0000","last_became_active":"2023-10-06T02:29:36.191984+0000","last_became_peered":"2023-10-06T02:29:36.191984+0000","last_unstale":"2023-10-06T02:29:36.196247+0000","last_undegraded":"2023-10-06T02:29:36.196247+0000","last_fullsized":"2023-10-06T02:29:36.196247+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:09:06.367106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"119'2","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.202233+0000","last_change":"2023-10-06T02:29:35.186954+0000","last_active":"2023-10-06T02:29:36.202233+0000","last_peered":"2023-10-06T02:29:36.202233+0000","last_clean":"2023-10-06T02:29:36.202233+0000","last_became_active":"2023-10-06T02:29:35.186630+0000","last_became_peered":"2023-10-06T02:29:35.186630+0000","last_unstale":"2023-10-06T02:29:36.202233+0000","last_undegraded":"2023-10-06T02:29:36.202233+0000","last_fullsized":"2023-10-06T02:29:36.202233+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_clean_scrub_stamp":"2023-10-06T02:29:34.151749+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:23:36.997767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.198465+0000","last_change":"2023-10-06T02:29:36.194504+0000","last_active":"2023-10-06T02:29:36.198465+0000","last_peered":"2023-10-06T02:29:36.198465+0000","last_clean":"2023-10-06T02:29:36.198465+0000","last_became_active":"2023-10-06T02:29:36.194064+0000","last_became_peered":"2023-10-06T02:29:36.194064+0000","last_unstale":"2023-10-06T02:29:36.198465+0000","last_undegraded":"2023-10-06T02:29:36.198465+0000","last_fullsized":"2023-10-06T02:29:36.198465+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:06:39.730288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1c","version":"119'2","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.204197+0000","last_change":"2023-10-06T02:29:35.191543+0000","last_active":"2023-10-06T02:29:36.204197+0000","last_peered":"2023-10-06T02:29:36.204197+0000","last_clean":"2023-10-06T02:29:36.204197+0000","last_became_active":"2023-10-06T02:29:35.191284+0000","last_became_peered":"2023-10-06T02:29:35.191284+0000","last_unstale":"2023-10-06T02:29:36.204197+0000","last_undegraded":"2023-10-06T02:29:36.204197+0000","last_fullsized":"2023-10-06T02:29:36.204197+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_clean_scrub_stamp":"2023-10-06T02:29:34.151749+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:54:47.275146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.197474+0000","last_change":"2023-10-06T02:29:36.190541+0000","last_active":"2023-10-06T02:29:36.197474+0000","last_peered":"2023-10-06T02:29:36.197474+0000","last_clean":"2023-10-06T02:29:36.197474+0000","last_became_active":"2023-10-06T02:29:36.190237+0000","last_became_peered":"2023-10-06T02:29:36.190237+0000","last_unstale":"2023-10-06T02:29:36.197474+0000","last_undegraded":"2023-10-06T02:29:36.197474+0000","last_fullsized":"2023-10-06T02:29:36.197474+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:10:07.892520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"119'1","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.203978+0000","last_change":"2023-10-06T02:29:35.190653+0000","last_active":"2023-10-06T02:29:36.203978+0000","last_peered":"2023-10-06T02:29:36.203978+0000","last_clean":"2023-10-06T02:29:36.203978+0000","last_became_active":"2023-10-06T02:29:35.190239+0000","last_became_peered":"2023-10-06T02:29:35.190239+0000","last_unstale":"2023-10-06T02:29:36.203978+0000","last_undegraded":"2023-10-06T02:29:36.203978+0000","last_fullsized":"2023-10-06T02:29:36.203978+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_clean_scrub_stamp":"2023-10-06T02:29:34.151749+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:40:48.229921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.198373+0000","last_change":"2023-10-06T02:29:36.197433+0000","last_active":"2023-10-06T02:29:36.198373+0000","last_peered":"2023-10-06T02:29:36.198373+0000","last_clean":"2023-10-06T02:29:36.198373+0000","last_became_active":"2023-10-06T02:29:36.197107+0000","last_became_peered":"2023-10-06T02:29:36.197107+0000","last_unstale":"2023-10-06T02:29:36.198373+0000","last_undegraded":"2023-10-06T02:29:36.198373+0000","last_fullsized":"2023-10-06T02:29:36.198373+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:18:41.713149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"119'1","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.188668+0000","last_change":"2023-10-06T02:29:35.188191+0000","last_active":"2023-10-06T02:29:36.188668+0000","last_peered":"2023-10-06T02:29:36.188668+0000","last_clean":"2023-10-06T02:29:36.188668+0000","last_became_active":"2023-10-06T02:29:35.187955+0000","last_became_peered":"2023-10-06T02:29:35.187955+0000","last_unstale":"2023-10-06T02:29:36.188668+0000","last_undegraded":"2023-10-06T02:29:36.188668+0000","last_fullsized":"2023-10-06T02:29:36.188668+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_clean_scrub_stamp":"2023-10-06T02:29:34.151749+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:29:18.462793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.200162+0000","last_change":"2023-10-06T02:29:36.195807+0000","last_active":"2023-10-06T02:29:36.200162+0000","last_peered":"2023-10-06T02:29:36.200162+0000","last_clean":"2023-10-06T02:29:36.200162+0000","last_became_active":"2023-10-06T02:29:36.195576+0000","last_became_peered":"2023-10-06T02:29:36.195576+0000","last_unstale":"2023-10-06T02:29:36.200162+0000","last_undegraded":"2023-10-06T02:29:36.200162+0000","last_fullsized":"2023-10-06T02:29:36.200162+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:03:45.223339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.184471+0000","last_change":"2023-10-06T02:29:35.181416+0000","last_active":"2023-10-06T02:29:36.184471+0000","last_peered":"2023-10-06T02:29:36.184471+0000","last_clean":"2023-10-06T02:29:36.184471+0000","last_became_active":"2023-10-06T02:29:35.181097+0000","last_became_peered":"2023-10-06T02:29:35.181097+0000","last_unstale":"2023-10-06T02:29:36.184471+0000","last_undegraded":"2023-10-06T02:29:36.184471+0000","last_fullsized":"2023-10-06T02:29:36.184471+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_clean_scrub_stamp":"2023-10-06T02:29:34.151749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:18:18.485443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.204050+0000","last_change":"2023-10-06T02:29:36.195444+0000","last_active":"2023-10-06T02:29:36.204050+0000","last_peered":"2023-10-06T02:29:36.204050+0000","last_clean":"2023-10-06T02:29:36.204050+0000","last_became_active":"2023-10-06T02:29:36.195135+0000","last_became_peered":"2023-10-06T02:29:36.195135+0000","last_unstale":"2023-10-06T02:29:36.204050+0000","last_undegraded":"2023-10-06T02:29:36.204050+0000","last_fullsized":"2023-10-06T02:29:36.204050+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:03:45.223339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.171902+0000","last_change":"2023-10-06T02:29:35.182468+0000","last_active":"2023-10-06T02:29:36.171902+0000","last_peered":"2023-10-06T02:29:36.171902+0000","last_clean":"2023-10-06T02:29:36.171902+0000","last_became_active":"2023-10-06T02:29:35.181783+0000","last_became_peered":"2023-10-06T02:29:35.181783+0000","last_unstale":"2023-10-06T02:29:36.171902+0000","last_undegraded":"2023-10-06T02:29:36.171902+0000","last_fullsized":"2023-10-06T02:29:36.171902+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_clean_scrub_stamp":"2023-10-06T02:29:34.151749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:31:13.871801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.193056+0000","last_change":"2023-10-06T02:29:36.181840+0000","last_active":"2023-10-06T02:29:36.193056+0000","last_peered":"2023-10-06T02:29:36.193056+0000","last_clean":"2023-10-06T02:29:36.193056+0000","last_became_active":"2023-10-06T02:29:36.181482+0000","last_became_peered":"2023-10-06T02:29:36.181482+0000","last_unstale":"2023-10-06T02:29:36.193056+0000","last_undegraded":"2023-10-06T02:29:36.193056+0000","last_fullsized":"2023-10-06T02:29:36.193056+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:53:28.589732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.f","version":"119'1","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.195193+0000","last_change":"2023-10-06T02:29:35.180337+0000","last_active":"2023-10-06T02:29:36.195193+0000","last_peered":"2023-10-06T02:29:36.195193+0000","last_clean":"2023-10-06T02:29:36.195193+0000","last_became_active":"2023-10-06T02:29:35.180005+0000","last_became_peered":"2023-10-06T02:29:35.180005+0000","last_unstale":"2023-10-06T02:29:36.195193+0000","last_undegraded":"2023-10-06T02:29:36.195193+0000","last_fullsized":"2023-10-06T02:29:36.195193+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_clean_scrub_stamp":"2023-10-06T02:29:34.151749+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:10:52.886547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.203580+0000","last_change":"2023-10-06T02:29:36.189641+0000","last_active":"2023-10-06T02:29:36.203580+0000","last_peered":"2023-10-06T02:29:36.203580+0000","last_clean":"2023-10-06T02:29:36.203580+0000","last_became_active":"2023-10-06T02:29:36.189282+0000","last_became_peered":"2023-10-06T02:29:36.189282+0000","last_unstale":"2023-10-06T02:29:36.203580+0000","last_undegraded":"2023-10-06T02:29:36.203580+0000","last_fullsized":"2023-10-06T02:29:36.203580+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:36:48.422482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.e","version":"119'2","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.203297+0000","last_change":"2023-10-06T02:29:35.185825+0000","last_active":"2023-10-06T02:29:36.203297+0000","last_peered":"2023-10-06T02:29:36.203297+0000","last_clean":"2023-10-06T02:29:36.203297+0000","last_became_active":"2023-10-06T02:29:35.185541+0000","last_became_peered":"2023-10-06T02:29:35.185541+0000","last_unstale":"2023-10-06T02:29:36.203297+0000","last_undegraded":"2023-10-06T02:29:36.203297+0000","last_fullsized":"2023-10-06T02:29:36.203297+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_clean_scrub_stamp":"2023-10-06T02:29:34.151749+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:06:38.723619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.204884+0000","last_change":"2023-10-06T02:29:36.201345+0000","last_active":"2023-10-06T02:29:36.204884+0000","last_peered":"2023-10-06T02:29:36.204884+0000","last_clean":"2023-10-06T02:29:36.204884+0000","last_became_active":"2023-10-06T02:29:36.200076+0000","last_became_peered":"2023-10-06T02:29:36.200076+0000","last_unstale":"2023-10-06T02:29:36.204884+0000","last_undegraded":"2023-10-06T02:29:36.204884+0000","last_fullsized":"2023-10-06T02:29:36.204884+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:34:31.991692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.d","version":"119'1","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.201791+0000","last_change":"2023-10-06T02:29:35.181838+0000","last_active":"2023-10-06T02:29:36.201791+0000","last_peered":"2023-10-06T02:29:36.201791+0000","last_clean":"2023-10-06T02:29:36.201791+0000","last_became_active":"2023-10-06T02:29:35.181156+0000","last_became_peered":"2023-10-06T02:29:35.181156+0000","last_unstale":"2023-10-06T02:29:36.201791+0000","last_undegraded":"2023-10-06T02:29:36.201791+0000","last_fullsized":"2023-10-06T02:29:36.201791+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_clean_scrub_stamp":"2023-10-06T02:29:34.151749+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:00:44.429148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.198318+0000","last_change":"2023-10-06T02:29:36.196495+0000","last_active":"2023-10-06T02:29:36.198318+0000","last_peered":"2023-10-06T02:29:36.198318+0000","last_clean":"2023-10-06T02:29:36.198318+0000","last_became_active":"2023-10-06T02:29:36.196109+0000","last_became_peered":"2023-10-06T02:29:36.196109+0000","last_unstale":"2023-10-06T02:29:36.198318+0000","last_undegraded":"2023-10-06T02:29:36.198318+0000","last_fullsized":"2023-10-06T02:29:36.198318+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:08:43.848065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.175404+0000","last_change":"2023-10-06T02:29:35.184270+0000","last_active":"2023-10-06T02:29:36.175404+0000","last_peered":"2023-10-06T02:29:36.175404+0000","last_clean":"2023-10-06T02:29:36.175404+0000","last_became_active":"2023-10-06T02:29:35.183938+0000","last_became_peered":"2023-10-06T02:29:35.183938+0000","last_unstale":"2023-10-06T02:29:36.175404+0000","last_undegraded":"2023-10-06T02:29:36.175404+0000","last_fullsized":"2023-10-06T02:29:36.175404+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_clean_scrub_stamp":"2023-10-06T02:29:34.151749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:18:40.706480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.186736+0000","last_change":"2023-10-06T02:29:36.182350+0000","last_active":"2023-10-06T02:29:36.186736+0000","last_peered":"2023-10-06T02:29:36.186736+0000","last_clean":"2023-10-06T02:29:36.186736+0000","last_became_active":"2023-10-06T02:29:36.182013+0000","last_became_peered":"2023-10-06T02:29:36.182013+0000","last_unstale":"2023-10-06T02:29:36.186736+0000","last_undegraded":"2023-10-06T02:29:36.186736+0000","last_fullsized":"2023-10-06T02:29:36.186736+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:36:48.422482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"119'2","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.205867+0000","last_change":"2023-10-06T02:29:35.186895+0000","last_active":"2023-10-06T02:29:36.205867+0000","last_peered":"2023-10-06T02:29:36.205867+0000","last_clean":"2023-10-06T02:29:36.205867+0000","last_became_active":"2023-10-06T02:29:35.186599+0000","last_became_peered":"2023-10-06T02:29:35.186599+0000","last_unstale":"2023-10-06T02:29:36.205867+0000","last_undegraded":"2023-10-06T02:29:36.205867+0000","last_fullsized":"2023-10-06T02:29:36.205867+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_clean_scrub_stamp":"2023-10-06T02:29:34.151749+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:39:16.941655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.197316+0000","last_change":"2023-10-06T02:29:36.193620+0000","last_active":"2023-10-06T02:29:36.197316+0000","last_peered":"2023-10-06T02:29:36.197316+0000","last_clean":"2023-10-06T02:29:36.197316+0000","last_became_active":"2023-10-06T02:29:36.193001+0000","last_became_peered":"2023-10-06T02:29:36.193001+0000","last_unstale":"2023-10-06T02:29:36.197316+0000","last_undegraded":"2023-10-06T02:29:36.197316+0000","last_fullsized":"2023-10-06T02:29:36.197316+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:13:52.435753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.168415+0000","last_change":"2023-10-06T02:29:35.185588+0000","last_active":"2023-10-06T02:29:36.168415+0000","last_peered":"2023-10-06T02:29:36.168415+0000","last_clean":"2023-10-06T02:29:36.168415+0000","last_became_active":"2023-10-06T02:29:35.185306+0000","last_became_peered":"2023-10-06T02:29:35.185306+0000","last_unstale":"2023-10-06T02:29:36.168415+0000","last_undegraded":"2023-10-06T02:29:36.168415+0000","last_fullsized":"2023-10-06T02:29:36.168415+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_clean_scrub_stamp":"2023-10-06T02:29:34.151749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:59:35.165543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.201597+0000","last_change":"2023-10-06T02:29:36.200293+0000","last_active":"2023-10-06T02:29:36.201597+0000","last_peered":"2023-10-06T02:29:36.201597+0000","last_clean":"2023-10-06T02:29:36.201597+0000","last_became_active":"2023-10-06T02:29:36.199983+0000","last_became_peered":"2023-10-06T02:29:36.199983+0000","last_unstale":"2023-10-06T02:29:36.201597+0000","last_undegraded":"2023-10-06T02:29:36.201597+0000","last_fullsized":"2023-10-06T02:29:36.201597+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:44:14.781598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.176666+0000","last_change":"2023-10-06T02:29:35.188766+0000","last_active":"2023-10-06T02:29:36.176666+0000","last_peered":"2023-10-06T02:29:36.176666+0000","last_clean":"2023-10-06T02:29:36.176666+0000","last_became_active":"2023-10-06T02:29:35.188353+0000","last_became_peered":"2023-10-06T02:29:35.188353+0000","last_unstale":"2023-10-06T02:29:36.176666+0000","last_undegraded":"2023-10-06T02:29:36.176666+0000","last_fullsized":"2023-10-06T02:29:36.176666+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_clean_scrub_stamp":"2023-10-06T02:29:34.151749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:55:16.710103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.191436+0000","last_change":"2023-10-06T02:29:36.185674+0000","last_active":"2023-10-06T02:29:36.191436+0000","last_peered":"2023-10-06T02:29:36.191436+0000","last_clean":"2023-10-06T02:29:36.191436+0000","last_became_active":"2023-10-06T02:29:36.185391+0000","last_became_peered":"2023-10-06T02:29:36.185391+0000","last_unstale":"2023-10-06T02:29:36.191436+0000","last_undegraded":"2023-10-06T02:29:36.191436+0000","last_fullsized":"2023-10-06T02:29:36.191436+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:55:17.716772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.184145+0000","last_change":"2023-10-06T02:29:35.179848+0000","last_active":"2023-10-06T02:29:36.184145+0000","last_peered":"2023-10-06T02:29:36.184145+0000","last_clean":"2023-10-06T02:29:36.184145+0000","last_became_active":"2023-10-06T02:29:35.179588+0000","last_became_peered":"2023-10-06T02:29:35.179588+0000","last_unstale":"2023-10-06T02:29:36.184145+0000","last_undegraded":"2023-10-06T02:29:36.184145+0000","last_fullsized":"2023-10-06T02:29:36.184145+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_clean_scrub_stamp":"2023-10-06T02:29:34.151749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:30:35.677183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.197087+0000","last_change":"2023-10-06T02:29:36.191908+0000","last_active":"2023-10-06T02:29:36.197087+0000","last_peered":"2023-10-06T02:29:36.197087+0000","last_clean":"2023-10-06T02:29:36.197087+0000","last_became_active":"2023-10-06T02:29:36.191641+0000","last_became_peered":"2023-10-06T02:29:36.191641+0000","last_unstale":"2023-10-06T02:29:36.197087+0000","last_undegraded":"2023-10-06T02:29:36.197087+0000","last_fullsized":"2023-10-06T02:29:36.197087+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:43:21.797680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.7","version":"119'2","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.200612+0000","last_change":"2023-10-06T02:29:35.186014+0000","last_active":"2023-10-06T02:29:36.200612+0000","last_peered":"2023-10-06T02:29:36.200612+0000","last_clean":"2023-10-06T02:29:36.200612+0000","last_became_active":"2023-10-06T02:29:35.185766+0000","last_became_peered":"2023-10-06T02:29:35.185766+0000","last_unstale":"2023-10-06T02:29:36.200612+0000","last_undegraded":"2023-10-06T02:29:36.200612+0000","last_fullsized":"2023-10-06T02:29:36.200612+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_clean_scrub_stamp":"2023-10-06T02:29:34.151749+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:26:07.237072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.193171+0000","last_change":"2023-10-06T02:29:36.183950+0000","last_active":"2023-10-06T02:29:36.193171+0000","last_peered":"2023-10-06T02:29:36.193171+0000","last_clean":"2023-10-06T02:29:36.193171+0000","last_became_active":"2023-10-06T02:29:36.183468+0000","last_became_peered":"2023-10-06T02:29:36.183468+0000","last_unstale":"2023-10-06T02:29:36.193171+0000","last_undegraded":"2023-10-06T02:29:36.193171+0000","last_fullsized":"2023-10-06T02:29:36.193171+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:38:49.300110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.6","version":"120'1","reported_seq":11,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-10-06T02:29:38.629801+0000","last_change":"2023-10-06T02:29:35.191390+0000","last_active":"2023-10-06T02:29:38.629801+0000","last_peered":"2023-10-06T02:29:38.629801+0000","last_clean":"2023-10-06T02:29:38.629801+0000","last_became_active":"2023-10-06T02:29:35.191055+0000","last_became_peered":"2023-10-06T02:29:35.191055+0000","last_unstale":"2023-10-06T02:29:38.629801+0000","last_undegraded":"2023-10-06T02:29:38.629801+0000","last_fullsized":"2023-10-06T02:29:38.629801+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_clean_scrub_stamp":"2023-10-06T02:29:34.151749+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:36:46.538094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.204904+0000","last_change":"2023-10-06T02:29:36.201724+0000","last_active":"2023-10-06T02:29:36.204904+0000","last_peered":"2023-10-06T02:29:36.204904+0000","last_clean":"2023-10-06T02:29:36.204904+0000","last_became_active":"2023-10-06T02:29:36.199808+0000","last_became_peered":"2023-10-06T02:29:36.199808+0000","last_unstale":"2023-10-06T02:29:36.204904+0000","last_undegraded":"2023-10-06T02:29:36.204904+0000","last_fullsized":"2023-10-06T02:29:36.204904+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:14:59.301511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.180618+0000","last_change":"2023-10-06T02:29:35.190464+0000","last_active":"2023-10-06T02:29:36.180618+0000","last_peered":"2023-10-06T02:29:36.180618+0000","last_clean":"2023-10-06T02:29:36.180618+0000","last_became_active":"2023-10-06T02:29:35.190162+0000","last_became_peered":"2023-10-06T02:29:35.190162+0000","last_unstale":"2023-10-06T02:29:36.180618+0000","last_undegraded":"2023-10-06T02:29:36.180618+0000","last_fullsized":"2023-10-06T02:29:36.180618+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_clean_scrub_stamp":"2023-10-06T02:29:34.151749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:19:48.839079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.196296+0000","last_change":"2023-10-06T02:29:36.189747+0000","last_active":"2023-10-06T02:29:36.196296+0000","last_peered":"2023-10-06T02:29:36.196296+0000","last_clean":"2023-10-06T02:29:36.196296+0000","last_became_active":"2023-10-06T02:29:36.189417+0000","last_became_peered":"2023-10-06T02:29:36.189417+0000","last_unstale":"2023-10-06T02:29:36.196296+0000","last_undegraded":"2023-10-06T02:29:36.196296+0000","last_fullsized":"2023-10-06T02:29:36.196296+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:44:27.074320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.186603+0000","last_change":"2023-10-06T02:29:35.181721+0000","last_active":"2023-10-06T02:29:36.186603+0000","last_peered":"2023-10-06T02:29:36.186603+0000","last_clean":"2023-10-06T02:29:36.186603+0000","last_became_active":"2023-10-06T02:29:35.181450+0000","last_became_peered":"2023-10-06T02:29:35.181450+0000","last_unstale":"2023-10-06T02:29:36.186603+0000","last_undegraded":"2023-10-06T02:29:36.186603+0000","last_fullsized":"2023-10-06T02:29:36.186603+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_clean_scrub_stamp":"2023-10-06T02:29:34.151749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:14:51.570765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.195671+0000","last_change":"2023-10-06T02:29:36.194729+0000","last_active":"2023-10-06T02:29:36.195671+0000","last_peered":"2023-10-06T02:29:36.195671+0000","last_clean":"2023-10-06T02:29:36.195671+0000","last_became_active":"2023-10-06T02:29:36.194394+0000","last_became_peered":"2023-10-06T02:29:36.194394+0000","last_unstale":"2023-10-06T02:29:36.195671+0000","last_undegraded":"2023-10-06T02:29:36.195671+0000","last_fullsized":"2023-10-06T02:29:36.195671+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:56:31.959275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.172798+0000","last_change":"2023-10-06T02:29:35.178750+0000","last_active":"2023-10-06T02:29:36.172798+0000","last_peered":"2023-10-06T02:29:36.172798+0000","last_clean":"2023-10-06T02:29:36.172798+0000","last_became_active":"2023-10-06T02:29:35.178427+0000","last_became_peered":"2023-10-06T02:29:35.178427+0000","last_unstale":"2023-10-06T02:29:36.172798+0000","last_undegraded":"2023-10-06T02:29:36.172798+0000","last_fullsized":"2023-10-06T02:29:36.172798+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_clean_scrub_stamp":"2023-10-06T02:29:34.151749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:52:23.438197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.196824+0000","last_change":"2023-10-06T02:29:36.195414+0000","last_active":"2023-10-06T02:29:36.196824+0000","last_peered":"2023-10-06T02:29:36.196824+0000","last_clean":"2023-10-06T02:29:36.196824+0000","last_became_active":"2023-10-06T02:29:36.195174+0000","last_became_peered":"2023-10-06T02:29:36.195174+0000","last_unstale":"2023-10-06T02:29:36.196824+0000","last_undegraded":"2023-10-06T02:29:36.196824+0000","last_fullsized":"2023-10-06T02:29:36.196824+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:30:23.636577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.177136+0000","last_change":"2023-10-06T02:29:35.191801+0000","last_active":"2023-10-06T02:29:36.177136+0000","last_peered":"2023-10-06T02:29:36.177136+0000","last_clean":"2023-10-06T02:29:36.177136+0000","last_became_active":"2023-10-06T02:29:35.191260+0000","last_became_peered":"2023-10-06T02:29:35.191260+0000","last_unstale":"2023-10-06T02:29:36.177136+0000","last_undegraded":"2023-10-06T02:29:36.177136+0000","last_fullsized":"2023-10-06T02:29:36.177136+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_clean_scrub_stamp":"2023-10-06T02:29:34.151749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:10:06.885851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.205562+0000","last_change":"2023-10-06T02:29:36.205054+0000","last_active":"2023-10-06T02:29:36.205562+0000","last_peered":"2023-10-06T02:29:36.205562+0000","last_clean":"2023-10-06T02:29:36.205562+0000","last_became_active":"2023-10-06T02:29:36.204829+0000","last_became_peered":"2023-10-06T02:29:36.204829+0000","last_unstale":"2023-10-06T02:29:36.205562+0000","last_undegraded":"2023-10-06T02:29:36.205562+0000","last_fullsized":"2023-10-06T02:29:36.205562+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:39:17.948324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.184586+0000","last_change":"2023-10-06T02:29:35.178708+0000","last_active":"2023-10-06T02:29:36.184586+0000","last_peered":"2023-10-06T02:29:36.184586+0000","last_clean":"2023-10-06T02:29:36.184586+0000","last_became_active":"2023-10-06T02:29:35.178434+0000","last_became_peered":"2023-10-06T02:29:35.178434+0000","last_unstale":"2023-10-06T02:29:36.184586+0000","last_undegraded":"2023-10-06T02:29:36.184586+0000","last_fullsized":"2023-10-06T02:29:36.184586+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_clean_scrub_stamp":"2023-10-06T02:29:34.151749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:22:09.674761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.204252+0000","last_change":"2023-10-06T02:29:36.198675+0000","last_active":"2023-10-06T02:29:36.204252+0000","last_peered":"2023-10-06T02:29:36.204252+0000","last_clean":"2023-10-06T02:29:36.204252+0000","last_became_active":"2023-10-06T02:29:36.198384+0000","last_became_peered":"2023-10-06T02:29:36.198384+0000","last_unstale":"2023-10-06T02:29:36.204252+0000","last_undegraded":"2023-10-06T02:29:36.204252+0000","last_fullsized":"2023-10-06T02:29:36.204252+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:33:43.513554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.173907+0000","last_change":"2023-10-06T02:29:35.187993+0000","last_active":"2023-10-06T02:29:36.173907+0000","last_peered":"2023-10-06T02:29:36.173907+0000","last_clean":"2023-10-06T02:29:36.173907+0000","last_became_active":"2023-10-06T02:29:35.187730+0000","last_became_peered":"2023-10-06T02:29:35.187730+0000","last_unstale":"2023-10-06T02:29:36.173907+0000","last_undegraded":"2023-10-06T02:29:36.173907+0000","last_fullsized":"2023-10-06T02:29:36.173907+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_clean_scrub_stamp":"2023-10-06T02:29:34.151749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:25:18.945648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.199652+0000","last_change":"2023-10-06T02:29:36.196164+0000","last_active":"2023-10-06T02:29:36.199652+0000","last_peered":"2023-10-06T02:29:36.199652+0000","last_clean":"2023-10-06T02:29:36.199652+0000","last_became_active":"2023-10-06T02:29:36.195922+0000","last_became_peered":"2023-10-06T02:29:36.195922+0000","last_unstale":"2023-10-06T02:29:36.199652+0000","last_undegraded":"2023-10-06T02:29:36.199652+0000","last_fullsized":"2023-10-06T02:29:36.199652+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:40:49.236590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.169161+0000","last_change":"2023-10-06T02:29:35.179569+0000","last_active":"2023-10-06T02:29:36.169161+0000","last_peered":"2023-10-06T02:29:36.169161+0000","last_clean":"2023-10-06T02:29:36.169161+0000","last_became_active":"2023-10-06T02:29:35.178995+0000","last_became_peered":"2023-10-06T02:29:35.178995+0000","last_unstale":"2023-10-06T02:29:36.169161+0000","last_undegraded":"2023-10-06T02:29:36.169161+0000","last_fullsized":"2023-10-06T02:29:36.169161+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_clean_scrub_stamp":"2023-10-06T02:29:34.151749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:22:17.305684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":96,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.173434+0000","last_change":"2023-10-06T02:23:01.992935+0000","last_active":"2023-10-06T02:29:36.173434+0000","last_peered":"2023-10-06T02:29:36.173434+0000","last_clean":"2023-10-06T02:29:36.173434+0000","last_became_active":"2023-10-06T02:23:01.992568+0000","last_became_peered":"2023-10-06T02:23:01.992568+0000","last_unstale":"2023-10-06T02:29:36.173434+0000","last_undegraded":"2023-10-06T02:29:36.173434+0000","last_fullsized":"2023-10-06T02:29:36.173434+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:06.212251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.193804+0000","last_change":"2023-10-06T02:29:36.188159+0000","last_active":"2023-10-06T02:29:36.193804+0000","last_peered":"2023-10-06T02:29:36.193804+0000","last_clean":"2023-10-06T02:29:36.193804+0000","last_became_active":"2023-10-06T02:29:36.187799+0000","last_became_peered":"2023-10-06T02:29:36.187799+0000","last_unstale":"2023-10-06T02:29:36.193804+0000","last_undegraded":"2023-10-06T02:29:36.193804+0000","last_fullsized":"2023-10-06T02:29:36.193804+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:12:52.344361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.171388+0000","last_change":"2023-10-06T02:29:35.176590+0000","last_active":"2023-10-06T02:29:36.171388+0000","last_peered":"2023-10-06T02:29:36.171388+0000","last_clean":"2023-10-06T02:29:36.171388+0000","last_became_active":"2023-10-06T02:29:35.176154+0000","last_became_peered":"2023-10-06T02:29:35.176154+0000","last_unstale":"2023-10-06T02:29:36.171388+0000","last_undegraded":"2023-10-06T02:29:36.171388+0000","last_fullsized":"2023-10-06T02:29:36.171388+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_clean_scrub_stamp":"2023-10-06T02:29:34.151749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:17:46.181194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":95,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.184808+0000","last_change":"2023-10-06T02:23:01.989934+0000","last_active":"2023-10-06T02:29:36.184808+0000","last_peered":"2023-10-06T02:29:36.184808+0000","last_clean":"2023-10-06T02:29:36.184808+0000","last_became_active":"2023-10-06T02:23:01.989706+0000","last_became_peered":"2023-10-06T02:23:01.989706+0000","last_unstale":"2023-10-06T02:29:36.184808+0000","last_undegraded":"2023-10-06T02:29:36.184808+0000","last_fullsized":"2023-10-06T02:29:36.184808+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:06.212251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.193269+0000","last_change":"2023-10-06T02:29:36.180993+0000","last_active":"2023-10-06T02:29:36.193269+0000","last_peered":"2023-10-06T02:29:36.193269+0000","last_clean":"2023-10-06T02:29:36.193269+0000","last_became_active":"2023-10-06T02:29:36.180697+0000","last_became_peered":"2023-10-06T02:29:36.180697+0000","last_unstale":"2023-10-06T02:29:36.193269+0000","last_undegraded":"2023-10-06T02:29:36.193269+0000","last_fullsized":"2023-10-06T02:29:36.193269+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:22:10.681430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.14","version":"119'6","reported_seq":16,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.207057+0000","last_change":"2023-10-06T02:29:35.189634+0000","last_active":"2023-10-06T02:29:36.207057+0000","last_peered":"2023-10-06T02:29:36.207057+0000","last_clean":"2023-10-06T02:29:36.207057+0000","last_became_active":"2023-10-06T02:29:35.189353+0000","last_became_peered":"2023-10-06T02:29:35.189353+0000","last_unstale":"2023-10-06T02:29:36.207057+0000","last_undegraded":"2023-10-06T02:29:36.207057+0000","last_fullsized":"2023-10-06T02:29:36.207057+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_clean_scrub_stamp":"2023-10-06T02:29:34.151749+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:02:42.408863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":91,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.180100+0000","last_change":"2023-10-06T02:23:02.054069+0000","last_active":"2023-10-06T02:29:36.180100+0000","last_peered":"2023-10-06T02:29:36.180100+0000","last_clean":"2023-10-06T02:29:36.180100+0000","last_became_active":"2023-10-06T02:23:02.053739+0000","last_became_peered":"2023-10-06T02:23:02.053739+0000","last_unstale":"2023-10-06T02:29:36.180100+0000","last_undegraded":"2023-10-06T02:29:36.180100+0000","last_fullsized":"2023-10-06T02:29:36.180100+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:57:48.822608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.203957+0000","last_change":"2023-10-06T02:29:36.190327+0000","last_active":"2023-10-06T02:29:36.203957+0000","last_peered":"2023-10-06T02:29:36.203957+0000","last_clean":"2023-10-06T02:29:36.203957+0000","last_became_active":"2023-10-06T02:29:36.190056+0000","last_became_peered":"2023-10-06T02:29:36.190056+0000","last_unstale":"2023-10-06T02:29:36.203957+0000","last_undegraded":"2023-10-06T02:29:36.203957+0000","last_fullsized":"2023-10-06T02:29:36.203957+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:44:14.781598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.172812+0000","last_change":"2023-10-06T02:29:35.187050+0000","last_active":"2023-10-06T02:29:36.172812+0000","last_peered":"2023-10-06T02:29:36.172812+0000","last_clean":"2023-10-06T02:29:36.172812+0000","last_became_active":"2023-10-06T02:29:35.186770+0000","last_became_peered":"2023-10-06T02:29:35.186770+0000","last_unstale":"2023-10-06T02:29:36.172812+0000","last_undegraded":"2023-10-06T02:29:36.172812+0000","last_fullsized":"2023-10-06T02:29:36.172812+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_clean_scrub_stamp":"2023-10-06T02:29:34.151749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:48:19.623691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":91,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.176343+0000","last_change":"2023-10-06T02:23:01.990773+0000","last_active":"2023-10-06T02:29:36.176343+0000","last_peered":"2023-10-06T02:29:36.176343+0000","last_clean":"2023-10-06T02:29:36.176343+0000","last_became_active":"2023-10-06T02:23:01.990511+0000","last_became_peered":"2023-10-06T02:23:01.990511+0000","last_unstale":"2023-10-06T02:29:36.176343+0000","last_undegraded":"2023-10-06T02:29:36.176343+0000","last_fullsized":"2023-10-06T02:29:36.176343+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:42:54.797731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.196432+0000","last_change":"2023-10-06T02:29:36.189566+0000","last_active":"2023-10-06T02:29:36.196432+0000","last_peered":"2023-10-06T02:29:36.196432+0000","last_clean":"2023-10-06T02:29:36.196432+0000","last_became_active":"2023-10-06T02:29:36.188988+0000","last_became_peered":"2023-10-06T02:29:36.188988+0000","last_unstale":"2023-10-06T02:29:36.196432+0000","last_undegraded":"2023-10-06T02:29:36.196432+0000","last_fullsized":"2023-10-06T02:29:36.196432+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:52:11.695224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.177015+0000","last_change":"2023-10-06T02:29:35.185086+0000","last_active":"2023-10-06T02:29:36.177015+0000","last_peered":"2023-10-06T02:29:36.177015+0000","last_clean":"2023-10-06T02:29:36.177015+0000","last_became_active":"2023-10-06T02:29:35.184418+0000","last_became_peered":"2023-10-06T02:29:35.184418+0000","last_unstale":"2023-10-06T02:29:36.177015+0000","last_undegraded":"2023-10-06T02:29:36.177015+0000","last_fullsized":"2023-10-06T02:29:36.177015+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_clean_scrub_stamp":"2023-10-06T02:29:34.151749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:44:01.043777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"76'128","reported_seq":283,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.176447+0000","last_change":"2023-10-06T02:22:55.032519+0000","last_active":"2023-10-06T02:29:36.176447+0000","last_peered":"2023-10-06T02:29:36.176447+0000","last_clean":"2023-10-06T02:29:36.176447+0000","last_became_active":"2023-10-06T02:22:55.031367+0000","last_became_peered":"2023-10-06T02:22:55.031367+0000","last_unstale":"2023-10-06T02:29:36.176447+0000","last_undegraded":"2023-10-06T02:29:36.176447+0000","last_fullsized":"2023-10-06T02:29:36.176447+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:22:53.309097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:22:53.309097+0000","last_clean_scrub_stamp":"2023-10-06T02:22:53.309097+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:53:46.520909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":110,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.173524+0000","last_change":"2023-10-06T02:23:01.988140+0000","last_active":"2023-10-06T02:29:36.173524+0000","last_peered":"2023-10-06T02:29:36.173524+0000","last_clean":"2023-10-06T02:29:36.173524+0000","last_became_active":"2023-10-06T02:23:01.987769+0000","last_became_peered":"2023-10-06T02:23:01.987769+0000","last_unstale":"2023-10-06T02:29:36.173524+0000","last_undegraded":"2023-10-06T02:29:36.173524+0000","last_fullsized":"2023-10-06T02:29:36.173524+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:20:08.391912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.201090+0000","last_change":"2023-10-06T02:29:36.189439+0000","last_active":"2023-10-06T02:29:36.201090+0000","last_peered":"2023-10-06T02:29:36.201090+0000","last_clean":"2023-10-06T02:29:36.201090+0000","last_became_active":"2023-10-06T02:29:36.189289+0000","last_became_peered":"2023-10-06T02:29:36.189289+0000","last_unstale":"2023-10-06T02:29:36.201090+0000","last_undegraded":"2023-10-06T02:29:36.201090+0000","last_fullsized":"2023-10-06T02:29:36.201090+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:33:36.484989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.10","version":"119'8","reported_seq":18,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.204747+0000","last_change":"2023-10-06T02:29:35.190087+0000","last_active":"2023-10-06T02:29:36.204747+0000","last_peered":"2023-10-06T02:29:36.204747+0000","last_clean":"2023-10-06T02:29:36.204747+0000","last_became_active":"2023-10-06T02:29:35.189741+0000","last_became_peered":"2023-10-06T02:29:35.189741+0000","last_unstale":"2023-10-06T02:29:36.204747+0000","last_undegraded":"2023-10-06T02:29:36.204747+0000","last_fullsized":"2023-10-06T02:29:36.204747+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_clean_scrub_stamp":"2023-10-06T02:29:34.151749+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:53:27.583063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":102,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.168303+0000","last_change":"2023-10-06T02:23:02.054338+0000","last_active":"2023-10-06T02:29:36.168303+0000","last_peered":"2023-10-06T02:29:36.168303+0000","last_clean":"2023-10-06T02:29:36.168303+0000","last_became_active":"2023-10-06T02:23:02.053991+0000","last_became_peered":"2023-10-06T02:23:02.053991+0000","last_unstale":"2023-10-06T02:29:36.168303+0000","last_undegraded":"2023-10-06T02:29:36.168303+0000","last_fullsized":"2023-10-06T02:29:36.168303+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:39:30.192838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.185878+0000","last_change":"2023-10-06T02:29:36.182601+0000","last_active":"2023-10-06T02:29:36.185878+0000","last_peered":"2023-10-06T02:29:36.185878+0000","last_clean":"2023-10-06T02:29:36.185878+0000","last_became_active":"2023-10-06T02:29:36.182333+0000","last_became_peered":"2023-10-06T02:29:36.182333+0000","last_unstale":"2023-10-06T02:29:36.185878+0000","last_undegraded":"2023-10-06T02:29:36.185878+0000","last_fullsized":"2023-10-06T02:29:36.185878+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:41:12.745924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"119'2","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.198540+0000","last_change":"2023-10-06T02:29:35.178863+0000","last_active":"2023-10-06T02:29:36.198540+0000","last_peered":"2023-10-06T02:29:36.198540+0000","last_clean":"2023-10-06T02:29:36.198540+0000","last_became_active":"2023-10-06T02:29:35.178607+0000","last_became_peered":"2023-10-06T02:29:35.178607+0000","last_unstale":"2023-10-06T02:29:36.198540+0000","last_undegraded":"2023-10-06T02:29:36.198540+0000","last_fullsized":"2023-10-06T02:29:36.198540+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_clean_scrub_stamp":"2023-10-06T02:29:34.151749+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:21:08.149327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":110,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.172653+0000","last_change":"2023-10-06T02:23:01.988631+0000","last_active":"2023-10-06T02:29:36.172653+0000","last_peered":"2023-10-06T02:29:36.172653+0000","last_clean":"2023-10-06T02:29:36.172653+0000","last_became_active":"2023-10-06T02:23:01.988303+0000","last_became_peered":"2023-10-06T02:23:01.988303+0000","last_unstale":"2023-10-06T02:29:36.172653+0000","last_undegraded":"2023-10-06T02:29:36.172653+0000","last_fullsized":"2023-10-06T02:29:36.172653+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:38.211466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.195014+0000","last_change":"2023-10-06T02:29:36.186370+0000","last_active":"2023-10-06T02:29:36.195014+0000","last_peered":"2023-10-06T02:29:36.195014+0000","last_clean":"2023-10-06T02:29:36.195014+0000","last_became_active":"2023-10-06T02:29:36.183661+0000","last_became_peered":"2023-10-06T02:29:36.183661+0000","last_unstale":"2023-10-06T02:29:36.195014+0000","last_undegraded":"2023-10-06T02:29:36.195014+0000","last_fullsized":"2023-10-06T02:29:36.195014+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:35.158418+0000","last_clean_scrub_stamp":"2023-10-06T02:29:35.158418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:42:32.320089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"119'3","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.206116+0000","last_change":"2023-10-06T02:29:35.193263+0000","last_active":"2023-10-06T02:29:36.206116+0000","last_peered":"2023-10-06T02:29:36.206116+0000","last_clean":"2023-10-06T02:29:36.206116+0000","last_became_active":"2023-10-06T02:29:35.192895+0000","last_became_peered":"2023-10-06T02:29:35.192895+0000","last_unstale":"2023-10-06T02:29:36.206116+0000","last_undegraded":"2023-10-06T02:29:36.206116+0000","last_fullsized":"2023-10-06T02:29:36.206116+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:29:34.151749+0000","last_clean_scrub_stamp":"2023-10-06T02:29:34.151749+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:18:40.706480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":95,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T02:29:36.186684+0000","last_change":"2023-10-06T02:23:02.054039+0000","last_active":"2023-10-06T02:29:36.186684+0000","last_peered":"2023-10-06T02:29:36.186684+0000","last_clean":"2023-10-06T02:29:36.186684+0000","last_became_active":"2023-10-06T02:23:02.053742+0000","last_became_peered":"2023-10-06T02:23:02.053742+0000","last_unstale":"2023-10-06T02:29:36.186684+0000","last_undegraded":"2023-10-06T02:29:36.186684+0000","last_fullsized":"2023-10-06T02:29:36.186684+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:58:18.846475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574934,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303444,"kb_used_data":2060,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068396,"statfs":{"total":96636764160,"available":96326037504,"internally_reserved":0,"allocated":2109440,"data_stored":1288447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:29:06 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.63900000000000001},"min":{"1min":0.46300000000000002,"5min":0.318,"15min":0.318},"max":{"1min":0.90000000000000002,"5min":1.169,"15min":1.169},"last":0.63},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63800000000000001,"15min":0.61099999999999999},"min":{"1min":0.437,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.83399999999999996,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.84699999999999998}]},{"osd":1,"last update":"Fri Oct 6 02:29:06 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.72699999999999998,"15min":0.69099999999999995},"min":{"1min":0.55200000000000005,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.1020000000000001,"5min":1.147,"15min":1.147},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67600000000000005,"15min":0.67300000000000004},"min":{"1min":0.45100000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.82399999999999995}]},{"osd":2,"last update":"Fri Oct 6 02:29:06 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.68200000000000005,"15min":0.69099999999999995},"min":{"1min":0.5,"5min":0.375,"15min":0.375},"max":{"1min":0.95999999999999996,"5min":1.238,"15min":1.238},"last":0.87},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.68600000000000005,"15min":0.68999999999999995},"min":{"1min":0.47099999999999997,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.95099999999999996,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.69599999999999995}]},{"osd":3,"last update":"Fri Oct 6 02:29:06 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.74299999999999999,"15min":0.73099999999999998},"min":{"1min":0.54500000000000004,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.97899999999999998,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.76400000000000001,"15min":0.77400000000000002},"min":{"1min":0.58499999999999996,"5min":0.497,"15min":0.497},"max":{"1min":0.94499999999999995,"5min":1.23,"15min":1.23},"last":1.004}]},{"osd":4,"last update":"Fri Oct 6 02:29:06 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.60499999999999998,"15min":0.68100000000000005},"min":{"1min":0.45600000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.88400000000000001,"5min":1.002,"15min":1.2849999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.68799999999999994,"15min":0.75700000000000001},"min":{"1min":0.42299999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.84699999999999998,"5min":1.51,"15min":1.51},"last":0.54400000000000004}]},{"osd":5,"last update":"Fri Oct 6 02:29:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.69899999999999995,"15min":0.72999999999999998},"min":{"1min":0.46300000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.97999999999999998,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.69099999999999995,"15min":0.72699999999999998},"min":{"1min":0.46800000000000003,"5min":0.442,"15min":0.442},"max":{"1min":0.96599999999999997,"5min":0.98099999999999998,"15min":1.0289999999999999},"last":0.66500000000000004}]},{"osd":6,"last update":"Fri Oct 6 02:29:06 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.76800000000000002,"15min":0.78500000000000003},"min":{"1min":0.626,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0089999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.80100000000000005,"15min":0.81399999999999995},"min":{"1min":0.55100000000000005,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.92800000000000005,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.80100000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574934,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302856,"kb_used_data":1472,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068984,"statfs":{"total":96636764160,"available":96326639616,"internally_reserved":0,"allocated":1507328,"data_stored":697097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:29:05 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.67900000000000005,"15min":0.66700000000000004},"min":{"1min":0.44,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.505},{"interface":"front","average":{"1min":0.749,"5min":0.72899999999999998,"15min":0.70199999999999996},"min":{"1min":0.497,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0229999999999999,"5min":1.119,"15min":1.119},"last":0.56000000000000005}]},{"osd":1,"last update":"Fri Oct 6 02:29:05 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.65900000000000003,"15min":0.65000000000000002},"min":{"1min":0.53400000000000003,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.90200000000000002,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.68899999999999995,"15min":0.67100000000000004},"min":{"1min":0.53500000000000003,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.86699999999999999,"5min":0.997,"15min":0.997},"last":0.66600000000000004}]},{"osd":2,"last update":"Fri Oct 6 02:29:05 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.73099999999999998,"15min":0.70899999999999996},"min":{"1min":0.48899999999999999,"5min":0.437,"15min":0.437},"max":{"1min":1.0449999999999999,"5min":1.137,"15min":1.137},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.74199999999999999,"15min":0.71599999999999997},"min":{"1min":0.47199999999999998,"5min":0.45300000000000001,"15min":0.40300000000000002},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.80200000000000005}]},{"osd":3,"last update":"Fri Oct 6 02:29:05 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70599999999999996,"15min":0.66800000000000004},"min":{"1min":0.51300000000000001,"5min":0.49099999999999999,"15min":0.45700000000000002},"max":{"1min":0.95399999999999996,"5min":1.056,"15min":1.056},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.749,"5min":0.73099999999999998,"15min":0.70899999999999996},"min":{"1min":0.59599999999999997,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.97999999999999998,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.95699999999999996}]},{"osd":4,"last update":"Fri Oct 6 02:29:05 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66600000000000004,"15min":0.64300000000000002},"min":{"1min":0.40999999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.90700000000000003,"5min":1.095,"15min":1.095},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71199999999999997,"15min":0.66800000000000004},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.40999999999999998},"max":{"1min":0.92800000000000005,"5min":1.29,"15min":1.29},"last":0.91900000000000004}]},{"osd":5,"last update":"Fri Oct 6 02:29:05 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69099999999999995,"15min":0.70099999999999996},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.39700000000000002},"max":{"1min":0.89300000000000002,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":1.014},{"interface":"front","average":{"1min":0.745,"5min":0.72599999999999998,"15min":0.70799999999999996},"min":{"1min":0.49099999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.93799999999999994,"5min":1,"15min":1.02},"last":0.88700000000000001}]},{"osd":7,"last update":"Fri Oct 6 02:29:05 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.749,"15min":0.749},"min":{"1min":0.42399999999999999,"5min":0.41499999999999998,"15min":0.36799999999999999},"max":{"1min":0.95599999999999996,"5min":1.173,"15min":1.458},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.77500000000000002,"15min":0.76400000000000001},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.45900000000000002},"max":{"1min":1.103,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":1.046}]}]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302852,"kb_used_data":1468,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068988,"statfs":{"total":96636764160,"available":96326643712,"internally_reserved":0,"allocated":1503232,"data_stored":696523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:29:01 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.434,"15min":0.45700000000000002},"min":{"1min":0.29899999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.63600000000000001,"5min":0.83099999999999996,"15min":1.2230000000000001},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.46400000000000002,"15min":0.51900000000000002},"min":{"1min":0.34100000000000003,"5min":0.31,"15min":0.31},"max":{"1min":0.55600000000000005,"5min":0.71799999999999997,"15min":1.788},"last":0.35099999999999998}]},{"osd":2,"last update":"Fri Oct 6 02:29:01 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.44,"15min":0.51400000000000001},"min":{"1min":0.33300000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.58799999999999997,"5min":0.66200000000000003,"15min":2.0840000000000001},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.499,"15min":0.55100000000000005},"min":{"1min":0.375,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.61299999999999999,"5min":0.69299999999999995,"15min":1.6699999999999999},"last":0.52100000000000002}]},{"osd":3,"last update":"Fri Oct 6 02:29:01 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.59799999999999998,"15min":0.61399999999999999},"min":{"1min":0.46400000000000002,"5min":0.44600000000000001,"15min":0.42999999999999999},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":1.266},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.53300000000000003,"15min":0.628},"min":{"1min":0.40999999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.67200000000000004,"5min":0.79400000000000004,"15min":1.861},"last":0.59299999999999997}]},{"osd":4,"last update":"Fri Oct 6 02:29:01 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.77700000000000002,"15min":0.84699999999999998},"min":{"1min":0.59999999999999998,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.97199999999999998,"5min":1.3580000000000001,"15min":1.9279999999999999},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.78600000000000003,"15min":0.84799999999999998},"min":{"1min":0.56200000000000006,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.94499999999999995,"5min":1.4339999999999999,"15min":2.0179999999999998},"last":0.751}]},{"osd":5,"last update":"Fri Oct 6 02:29:01 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.746,"15min":0.84799999999999998},"min":{"1min":0.53900000000000003,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.9830000000000001},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.753,"15min":0.81499999999999995},"min":{"1min":0.57099999999999995,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.327,"5min":1.327,"15min":1.3380000000000001},"last":0.68600000000000005}]},{"osd":6,"last update":"Fri Oct 6 02:29:01 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.79100000000000004,"15min":0.85799999999999998},"min":{"1min":0.57899999999999996,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0009999999999999,"5min":1.1240000000000001,"15min":1.829},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.81000000000000005,"15min":0.90100000000000002},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.351,"5min":1.351,"15min":2.052},"last":0.61099999999999999}]},{"osd":7,"last update":"Fri Oct 6 02:29:01 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.77200000000000002,"15min":0.82799999999999996},"min":{"1min":0.55900000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.96199999999999997,"5min":1.3120000000000001,"15min":1.3340000000000001},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.78400000000000003,"15min":0.86799999999999999},"min":{"1min":0.57099999999999995,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.93400000000000005,"5min":1.2110000000000001,"15min":1.895},"last":0.75800000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574934,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303436,"kb_used_data":2052,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068404,"statfs":{"total":96636764160,"available":96326045696,"internally_reserved":0,"allocated":2101248,"data_stored":1286877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 02:29:13 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.42699999999999999,"15min":0.45200000000000001},"min":{"1min":0.32700000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":1.0129999999999999},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.504,"5min":0.46500000000000002,"15min":0.47899999999999998},"min":{"1min":0.29799999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.9830000000000001,"5min":1.9830000000000001,"15min":1.9830000000000001},"last":0.40600000000000003}]},{"osd":2,"last update":"Fri Oct 6 02:29:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.505,"15min":0.51600000000000001},"min":{"1min":0.38,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":3.0339999999999998,"5min":3.0339999999999998,"15min":3.0339999999999998},"last":0.626},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.503,"15min":0.51900000000000002},"min":{"1min":0.39200000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.44800000000000001}]},{"osd":3,"last update":"Fri Oct 6 02:29:13 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.48699999999999999,"15min":0.503},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.59399999999999997},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.028,"5min":1.028,"15min":1.1359999999999999},"last":0.68300000000000005}]},{"osd":4,"last update":"Fri Oct 6 02:29:13 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.755,"15min":0.79800000000000004},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":2.9940000000000002,"5min":2.9940000000000002,"15min":2.9940000000000002},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.75900000000000001,"15min":0.79700000000000004},"min":{"1min":0.56599999999999995,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.88700000000000001}]},{"osd":5,"last update":"Fri Oct 6 02:29:13 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.76000000000000001,"15min":0.78600000000000003},"min":{"1min":0.61599999999999999,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.79300000000000004,"15min":0.80300000000000005},"min":{"1min":0.58299999999999996,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":3.1110000000000002,"5min":3.1110000000000002,"15min":3.1110000000000002},"last":0.90700000000000003}]},{"osd":6,"last update":"Fri Oct 6 02:29:13 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.80700000000000005,"15min":0.82399999999999995},"min":{"1min":0.61399999999999999,"5min":0.57599999999999996,"15min":0.54600000000000004},"max":{"1min":3.1600000000000001,"5min":3.1600000000000001,"15min":3.1600000000000001},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77300000000000002,"15min":0.82399999999999995},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.80600000000000005}]},{"osd":7,"last update":"Fri Oct 6 02:29:13 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.79700000000000004,"15min":0.82599999999999996},"min":{"1min":0.60199999999999998,"5min":0.56899999999999995,"15min":0.56000000000000005},"max":{"1min":2.9369999999999998,"5min":2.9369999999999998,"15min":2.9369999999999998},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.77300000000000002,"15min":0.82099999999999995},"min":{"1min":0.60499999999999998,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.93700000000000006}]}]},{"osd":2,"up_from":13,"seq":55834574935,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302856,"kb_used_data":1472,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068984,"statfs":{"total":96636764160,"available":96326639616,"internally_reserved":0,"allocated":1507328,"data_stored":697041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:29:09 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.42899999999999999,"15min":0.439},"min":{"1min":0.29899999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.46999999999999997,"15min":0.48399999999999999},"min":{"1min":0.32700000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.69199999999999995,"5min":0.88700000000000001,"15min":1.0800000000000001},"last":0.45100000000000001}]},{"osd":1,"last update":"Fri Oct 6 02:29:09 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.48199999999999998,"15min":0.501},"min":{"1min":0.36299999999999999,"5min":0.33600000000000002,"15min":0.32400000000000001},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.52600000000000002,"15min":0.59999999999999998},"min":{"1min":0.379,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.86299999999999999,"5min":1.778,"15min":1.778},"last":0.52900000000000003}]},{"osd":3,"last update":"Fri Oct 6 02:29:09 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.51400000000000001,"15min":0.53400000000000003},"min":{"1min":0.33800000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.97499999999999998},"last":0.437},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.58999999999999997,"15min":0.61799999999999999},"min":{"1min":0.46600000000000003,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":1.028},"last":0.51800000000000002}]},{"osd":4,"last update":"Fri Oct 6 02:29:09 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.747,"15min":0.77500000000000002},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.0649999999999999,"5min":1.109,"15min":1.1319999999999999},"last":0.97499999999999998},{"interface":"front","average":{"1min":0.747,"5min":0.78300000000000003,"15min":0.84099999999999997},"min":{"1min":0.55900000000000005,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.014,"5min":1.907,"15min":1.907},"last":0.81799999999999995}]},{"osd":5,"last update":"Fri Oct 6 02:29:09 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.73799999999999999,"15min":0.79600000000000004},"min":{"1min":0.53600000000000003,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.077,"5min":1.109,"15min":1.1850000000000001},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76000000000000001,"15min":0.81599999999999995},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.98699999999999999,"5min":1.153,"15min":1.258},"last":0.90900000000000003}]},{"osd":6,"last update":"Fri Oct 6 02:29:09 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.79100000000000004,"15min":0.84299999999999997},"min":{"1min":0.59399999999999997,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.008,"5min":1.827,"15min":1.827},"last":0.98699999999999999},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.76800000000000002,"15min":0.79200000000000004},"min":{"1min":0.51200000000000001,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.052,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.92100000000000004}]},{"osd":7,"last update":"Fri Oct 6 02:29:09 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.80500000000000005,"15min":0.876},"min":{"1min":0.55100000000000005,"5min":0.499,"15min":0.499},"max":{"1min":1.0780000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.78400000000000003,"15min":0.82799999999999996},"min":{"1min":0.55700000000000005,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.125,"5min":1.125,"15min":1.1479999999999999},"last":0.85699999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574932,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302864,"kb_used_data":1480,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068976,"statfs":{"total":96636764160,"available":96326631424,"internally_reserved":0,"allocated":1515520,"data_stored":698059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:29:08 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.45100000000000001,"15min":0.504},"min":{"1min":0.26000000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.60399999999999998,"5min":1.0489999999999999,"15min":2.024},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.48499999999999999,"15min":0.51900000000000002},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.63,"5min":0.84699999999999998,"15min":2.181},"last":0.41199999999999998}]},{"osd":1,"last update":"Fri Oct 6 02:29:08 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.46300000000000002,"15min":0.66100000000000003},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.59099999999999997,"5min":0.66100000000000003,"15min":4.492},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.52800000000000002,"15min":0.70799999999999996},"min":{"1min":0.34100000000000003,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.72799999999999998,"5min":1.079,"15min":4.569},"last":0.53400000000000003}]},{"osd":2,"last update":"Fri Oct 6 02:29:08 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.624,"15min":0.66100000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.76600000000000001,"5min":1.024,"15min":2.0870000000000002},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.58399999999999996,"15min":0.64600000000000002},"min":{"1min":0.41699999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.753,"5min":0.94199999999999995,"15min":2.4260000000000002},"last":0.76800000000000002}]},{"osd":4,"last update":"Fri Oct 6 02:29:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.72999999999999998,"15min":0.81999999999999995},"min":{"1min":0.56799999999999995,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.94099999999999995,"5min":1.0149999999999999,"15min":2.214},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.749,"15min":0.80500000000000005},"min":{"1min":0.51800000000000002,"5min":0.5,"15min":0.5},"max":{"1min":0.95799999999999996,"5min":1.079,"15min":1.2869999999999999},"last":0.94999999999999996}]},{"osd":5,"last update":"Fri Oct 6 02:29:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.76200000000000001,"15min":0.81399999999999995},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.98399999999999999,"5min":1.0780000000000001,"15min":1.3029999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.75,"15min":0.84499999999999997},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.94999999999999996,"5min":1.0580000000000001,"15min":2.242},"last":0.78500000000000003}]},{"osd":6,"last update":"Fri Oct 6 02:29:08 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.76200000000000001,"15min":0.81399999999999995},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.038,"5min":1.0589999999999999,"15min":1.357},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.79400000000000004,"15min":0.82899999999999996},"min":{"1min":0.54700000000000004,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.94599999999999995,"5min":1.123,"15min":1.3100000000000001},"last":0.70499999999999996}]},{"osd":7,"last update":"Fri Oct 6 02:29:08 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.78100000000000003,"15min":0.85299999999999998},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.004,"5min":1.077,"15min":2.27},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.79400000000000004,"15min":0.85199999999999998},"min":{"1min":0.55800000000000005,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.0329999999999999,"5min":1.105,"15min":1.325},"last":0.68700000000000006}]}]},{"osd":4,"up_from":13,"seq":55834574934,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302848,"kb_used_data":1464,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068992,"statfs":{"total":96636764160,"available":96326647808,"internally_reserved":0,"allocated":1499136,"data_stored":696453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:29:06 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.66900000000000004,"15min":0.98899999999999999},"min":{"1min":0.48499999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":12.337},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.67400000000000004,"15min":0.90400000000000003},"min":{"1min":0.45100000000000001,"5min":0.438,"15min":0.438},"max":{"1min":0.98699999999999999,"5min":1.3520000000000001,"15min":9.6539999999999999},"last":0.52600000000000002}]},{"osd":1,"last update":"Fri Oct 6 02:29:06 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.68000000000000005,"15min":0.69899999999999995},"min":{"1min":0.48499999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.129,"5min":1.129,"15min":3.181},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.751,"5min":0.68999999999999995,"15min":1.012},"min":{"1min":0.52600000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.995,"5min":0.995,"15min":12.946999999999999},"last":0.57499999999999996}]},{"osd":2,"last update":"Fri Oct 6 02:29:06 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.70599999999999996,"15min":1.04},"min":{"1min":0.57299999999999995,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.89400000000000002,"5min":1.2989999999999999,"15min":12.377000000000001},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.72299999999999998,"15min":0.86899999999999999},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":6.3220000000000001},"last":0.86899999999999999}]},{"osd":3,"last update":"Fri Oct 6 02:29:06 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.72599999999999998,"15min":0.98199999999999998},"min":{"1min":0.55000000000000004,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.96599999999999997,"5min":1.5229999999999999,"15min":9.7720000000000002},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.70999999999999996,"15min":0.77400000000000002},"min":{"1min":0.53300000000000003,"5min":0.498,"15min":0.47999999999999998},"max":{"1min":1.212,"5min":1.212,"15min":3.0840000000000001},"last":0.79400000000000004}]},{"osd":5,"last update":"Fri Oct 6 02:29:06 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.68300000000000005,"15min":0.84299999999999997},"min":{"1min":0.40400000000000003,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.006,"5min":1.02,"15min":6.2690000000000001},"last":0.99199999999999999},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.70299999999999996,"15min":1.0269999999999999},"min":{"1min":0.53500000000000003,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.0700000000000001,"5min":1.4359999999999999,"15min":10.769},"last":0.76800000000000002}]},{"osd":6,"last update":"Fri Oct 6 02:29:06 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.68200000000000005,"15min":0.76000000000000001},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.024,"5min":1.4590000000000001,"15min":3.1349999999999998},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.75,"15min":0.96799999999999997},"min":{"1min":0.47799999999999998,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.103,"5min":1.3839999999999999,"15min":8.0899999999999999},"last":1.0329999999999999}]},{"osd":7,"last update":"Fri Oct 6 02:29:06 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.76300000000000001,"15min":1.0589999999999999},"min":{"1min":0.501,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.1599999999999999,"5min":1.494,"15min":9.7089999999999996},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.75900000000000001,"15min":0.99099999999999999},"min":{"1min":0.56399999999999995,"5min":0.374,"15min":0.374},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":7.6719999999999997},"last":0.89500000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574934,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302852,"kb_used_data":1468,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068988,"statfs":{"total":96636764160,"available":96326643712,"internally_reserved":0,"allocated":1503232,"data_stored":696475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:29:03 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.64500000000000002,"15min":0.627},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.97599999999999998,"5min":1.016,"15min":1.4279999999999999},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.66600000000000004,"15min":0.61399999999999999},"min":{"1min":0.443,"5min":0.41699999999999998,"15min":0.38700000000000001},"max":{"1min":0.92900000000000005,"5min":1.099,"15min":1.099},"last":0.79200000000000004}]},{"osd":1,"last update":"Fri Oct 6 02:29:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.67700000000000005,"15min":0.65300000000000002},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.004,"5min":1.0660000000000001,"15min":1.089},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.68799999999999994,"15min":0.67400000000000004},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40500000000000003},"max":{"1min":0.95699999999999996,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.52500000000000002}]},{"osd":2,"last update":"Fri Oct 6 02:29:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.70299999999999996,"15min":0.67100000000000004},"min":{"1min":0.48299999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.98799999999999999,"5min":1.016,"15min":1.016},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.70399999999999996,"15min":0.68400000000000005},"min":{"1min":0.52000000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.91800000000000004,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.495}]},{"osd":3,"last update":"Fri Oct 6 02:29:03 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.75800000000000001,"15min":0.72999999999999998},"min":{"1min":0.53200000000000003,"5min":0.503,"15min":0.48799999999999999},"max":{"1min":1.052,"5min":1.159,"15min":1.159},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.79700000000000004,"15min":0.76900000000000002},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.54100000000000004},"max":{"1min":0.97599999999999998,"5min":1.194,"15min":1.194},"last":0.75600000000000001}]},{"osd":4,"last update":"Fri Oct 6 02:29:03 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.66900000000000004,"15min":0.67500000000000004},"min":{"1min":0.42899999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.012,"5min":1.038,"15min":1.1499999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.67600000000000005,"15min":0.68400000000000005},"min":{"1min":0.45600000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.99299999999999999,"5min":1.25,"15min":1.25},"last":0.67900000000000005}]},{"osd":6,"last update":"Fri Oct 6 02:29:03 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.745,"15min":0.745},"min":{"1min":0.47799999999999998,"5min":0.45700000000000002,"15min":0.40300000000000002},"max":{"1min":0.95299999999999996,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.81200000000000006,"15min":0.81999999999999995},"min":{"1min":0.56999999999999995,"5min":0.373,"15min":0.373},"max":{"1min":1.1080000000000001,"5min":1.1200000000000001,"15min":1.2170000000000001},"last":0.93899999999999995}]},{"osd":7,"last update":"Fri Oct 6 02:29:03 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.76400000000000001,"15min":0.76500000000000001},"min":{"1min":0.503,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.97399999999999998,"5min":1.23,"15min":1.23},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.80800000000000005,"15min":0.83199999999999996},"min":{"1min":0.54800000000000004,"5min":0.437,"15min":0.437},"max":{"1min":1.0660000000000001,"5min":1.2050000000000001,"15min":1.5640000000000001},"last":0.90300000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T02:29:41.031 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T02:29:41.031 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T02:29:41.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.259+0000 7f5f7eb9d700 1 Processor -- start 2023-10-06T02:29:41.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.259+0000 7f5f7eb9d700 1 -- start start 2023-10-06T02:29:41.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.260+0000 7f5f7eb9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5f78159380 0x7f5f78159750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:41.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.260+0000 7f5f7eb9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5f7815dc30 0x7f5f780cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:41.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.260+0000 7f5f7eb9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f5f780cef40 0x7f5f7806d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:41.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.260+0000 7f5f7eb9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f780cc350 con 0x7f5f78159380 2023-10-06T02:29:41.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.260+0000 7f5f7eb9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f780cf380 con 0x7f5f780cef40 2023-10-06T02:29:41.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.260+0000 7f5f7eb9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f780cb740 con 0x7f5f7815dc30 2023-10-06T02:29:41.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.261+0000 7f5f7db9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5f78159380 0x7f5f78159750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:41.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.261+0000 7f5f7db9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5f78159380 0x7f5f78159750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55420/0 (socket says 172.21.15.31:55420) 2023-10-06T02:29:41.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.261+0000 7f5f7db9b700 1 -- 172.21.15.31:0/2804027946 learned_addr learned my addr 172.21.15.31:0/2804027946 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:41.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.261+0000 7f5f7d39a700 1 --2- 172.21.15.31:0/2804027946 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5f7815dc30 0x7f5f780cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:41.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.261+0000 7f5f7e39c700 1 --2- 172.21.15.31:0/2804027946 >> v2:172.21.15.62:3300/0 conn(0x7f5f780cef40 0x7f5f7806d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:41.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.261+0000 7f5f7db9b700 1 -- 172.21.15.31:0/2804027946 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5f7815dc30 msgr2=0x7f5f780cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:41.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.261+0000 7f5f7db9b700 1 --2- 172.21.15.31:0/2804027946 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5f7815dc30 0x7f5f780cea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:41.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.261+0000 7f5f7db9b700 1 -- 172.21.15.31:0/2804027946 >> v2:172.21.15.62:3300/0 conn(0x7f5f780cef40 msgr2=0x7f5f7806d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:41.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.261+0000 7f5f7db9b700 1 --2- 172.21.15.31:0/2804027946 >> v2:172.21.15.62:3300/0 conn(0x7f5f780cef40 0x7f5f7806d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:41.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.261+0000 7f5f7db9b700 1 -- 172.21.15.31:0/2804027946 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f780cba20 con 0x7f5f78159380 2023-10-06T02:29:41.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.261+0000 7f5f7e39c700 1 --2- 172.21.15.31:0/2804027946 >> v2:172.21.15.62:3300/0 conn(0x7f5f780cef40 0x7f5f7806d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:29:41.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.262+0000 7f5f7db9b700 1 --2- 172.21.15.31:0/2804027946 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f78159380 0x7f5f78159750 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f5f780ce650 tx=0x7f5f74009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef64ecb7d8525ce2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:41.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.262+0000 7f5f7cb99700 1 -- 172.21.15.31:0/2804027946 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f74009e70 con 0x7f5f78159380 2023-10-06T02:29:41.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.262+0000 7f5f7cb99700 1 -- 172.21.15.31:0/2804027946 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f7400b860 con 0x7f5f78159380 2023-10-06T02:29:41.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.262+0000 7f5f7cb99700 1 -- 172.21.15.31:0/2804027946 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f7400bc00 con 0x7f5f78159380 2023-10-06T02:29:41.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.262+0000 7f5f7eb9d700 1 -- 172.21.15.31:0/2804027946 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f78159380 msgr2=0x7f5f78159750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:41.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.262+0000 7f5f7eb9d700 1 --2- 172.21.15.31:0/2804027946 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f78159380 0x7f5f78159750 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f5f780ce650 tx=0x7f5f74009250 comp rx=0 tx=0).stop 2023-10-06T02:29:41.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.263+0000 7f5f7eb9d700 1 -- 172.21.15.31:0/2804027946 shutdown_connections 2023-10-06T02:29:41.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.263+0000 7f5f7eb9d700 1 --2- 172.21.15.31:0/2804027946 >> v2:172.21.15.62:3300/0 conn(0x7f5f780cef40 0x7f5f7806d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:41.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.263+0000 7f5f7eb9d700 1 --2- 172.21.15.31:0/2804027946 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f78159380 0x7f5f78159750 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:41.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.263+0000 7f5f7eb9d700 1 --2- 172.21.15.31:0/2804027946 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5f7815dc30 0x7f5f780cea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:41.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.263+0000 7f5f7eb9d700 1 -- 172.21.15.31:0/2804027946 >> 172.21.15.31:0/2804027946 conn(0x7f5f780703a0 msgr2=0x7f5f780707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:41.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.263+0000 7f5f7eb9d700 1 -- 172.21.15.31:0/2804027946 shutdown_connections 2023-10-06T02:29:41.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.263+0000 7f5f7eb9d700 1 -- 172.21.15.31:0/2804027946 wait complete. 2023-10-06T02:29:41.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.264+0000 7f5f7eb9d700 1 Processor -- start 2023-10-06T02:29:41.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.264+0000 7f5f7eb9d700 1 -- start start 2023-10-06T02:29:41.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.265+0000 7f5f7eb9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5f780cef40 0x7f5f78154390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:41.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.265+0000 7f5f7eb9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f78159380 0x7f5f781533b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:41.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.265+0000 7f5f7eb9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5f7815dc30 0x7f5f78159bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:41.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.265+0000 7f5f7eb9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f780b53d0 con 0x7f5f78159380 2023-10-06T02:29:41.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.265+0000 7f5f7eb9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f780b56b0 con 0x7f5f7815dc30 2023-10-06T02:29:41.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.265+0000 7f5f7eb9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f780b5990 con 0x7f5f780cef40 2023-10-06T02:29:41.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.265+0000 7f5f7d39a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f78159380 0x7f5f781533b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:41.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.265+0000 7f5f7d39a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f78159380 0x7f5f781533b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55432/0 (socket says 172.21.15.31:55432) 2023-10-06T02:29:41.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.265+0000 7f5f7d39a700 1 -- 172.21.15.31:0/754635935 learned_addr learned my addr 172.21.15.31:0/754635935 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:41.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.266+0000 7f5f7d39a700 1 -- 172.21.15.31:0/754635935 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5f780cef40 msgr2=0x7f5f78154390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:41.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.266+0000 7f5f7d39a700 1 --2- 172.21.15.31:0/754635935 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5f780cef40 0x7f5f78154390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:41.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.266+0000 7f5f7d39a700 1 -- 172.21.15.31:0/754635935 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5f7815dc30 msgr2=0x7f5f78159bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:41.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.266+0000 7f5f7d39a700 1 --2- 172.21.15.31:0/754635935 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5f7815dc30 0x7f5f78159bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:41.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.266+0000 7f5f7d39a700 1 -- 172.21.15.31:0/754635935 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f740094f0 con 0x7f5f78159380 2023-10-06T02:29:41.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.266+0000 7f5f7d39a700 1 --2- 172.21.15.31:0/754635935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f78159380 0x7f5f781533b0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f5f6800e950 tx=0x7f5f680091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:41.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.266+0000 7f5f6dffb700 1 -- 172.21.15.31:0/754635935 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6800c850 con 0x7f5f78159380 2023-10-06T02:29:41.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.266+0000 7f5f6dffb700 1 -- 172.21.15.31:0/754635935 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f6800c9b0 con 0x7f5f78159380 2023-10-06T02:29:41.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.266+0000 7f5f7eb9d700 1 -- 172.21.15.31:0/754635935 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f780b5c70 con 0x7f5f78159380 2023-10-06T02:29:41.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.266+0000 7f5f6dffb700 1 -- 172.21.15.31:0/754635935 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6800cdb0 con 0x7f5f78159380 2023-10-06T02:29:41.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.266+0000 7f5f7eb9d700 1 -- 172.21.15.31:0/754635935 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f781538f0 con 0x7f5f78159380 2023-10-06T02:29:41.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.268+0000 7f5f7eb9d700 1 -- 172.21.15.31:0/754635935 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f7804c3c0 con 0x7f5f78159380 2023-10-06T02:29:41.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.268+0000 7f5f6dffb700 1 -- 172.21.15.31:0/754635935 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f5f68013680 con 0x7f5f78159380 2023-10-06T02:29:41.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.269+0000 7f5f6dffb700 1 --2- 172.21.15.31:0/754635935 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5f5c067320 0x7f5f5c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:41.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.269+0000 7f5f6dffb700 1 -- 172.21.15.31:0/754635935 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f5f68085040 con 0x7f5f78159380 2023-10-06T02:29:41.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.270+0000 7f5f7db9b700 1 --2- 172.21.15.31:0/754635935 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5f5c067320 0x7f5f5c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:41.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.271+0000 7f5f7db9b700 1 --2- 172.21.15.31:0/754635935 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5f5c067320 0x7f5f5c0697d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5f780ce650 tx=0x7f5f74017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:41.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.273+0000 7f5f6dffb700 1 -- 172.21.15.31:0/754635935 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5f68051b30 con 0x7f5f78159380 2023-10-06T02:29:41.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.552+0000 7f5f7eb9d700 1 -- 172.21.15.31:0/754635935 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f5f78155cb0 con 0x7f5f78159380 2023-10-06T02:29:41.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.556+0000 7f5f6dffb700 1 -- 172.21.15.31:0/754635935 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) v1 ==== 190+0+0 (secure 0 0 0) 0x7f5f78155cb0 con 0x7f5f78159380 2023-10-06T02:29:41.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.559+0000 7f5f7eb9d700 1 -- 172.21.15.31:0/754635935 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5f5c067320 msgr2=0x7f5f5c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:41.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.559+0000 7f5f7eb9d700 1 --2- 172.21.15.31:0/754635935 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5f5c067320 0x7f5f5c0697d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5f780ce650 tx=0x7f5f74017000 comp rx=0 tx=0).stop 2023-10-06T02:29:41.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.559+0000 7f5f7eb9d700 1 -- 172.21.15.31:0/754635935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f78159380 msgr2=0x7f5f781533b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:41.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.559+0000 7f5f7eb9d700 1 --2- 172.21.15.31:0/754635935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f78159380 0x7f5f781533b0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f5f6800e950 tx=0x7f5f680091a0 comp rx=0 tx=0).stop 2023-10-06T02:29:41.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.559+0000 7f5f7eb9d700 1 -- 172.21.15.31:0/754635935 shutdown_connections 2023-10-06T02:29:41.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.559+0000 7f5f7eb9d700 1 --2- 172.21.15.31:0/754635935 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5f7815dc30 0x7f5f78159bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:41.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.559+0000 7f5f7eb9d700 1 --2- 172.21.15.31:0/754635935 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5f5c067320 0x7f5f5c0697d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:41.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.559+0000 7f5f7eb9d700 1 --2- 172.21.15.31:0/754635935 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5f780cef40 0x7f5f78154390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:41.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.559+0000 7f5f7eb9d700 1 --2- 172.21.15.31:0/754635935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f78159380 0x7f5f781533b0 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:41.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.559+0000 7f5f7eb9d700 1 -- 172.21.15.31:0/754635935 >> 172.21.15.31:0/754635935 conn(0x7f5f780703a0 msgr2=0x7f5f780cd0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:41.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.559+0000 7f5f7eb9d700 1 -- 172.21.15.31:0/754635935 shutdown_connections 2023-10-06T02:29:41.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.559+0000 7f5f7eb9d700 1 -- 172.21.15.31:0/754635935 wait complete. 2023-10-06T02:29:41.562 INFO:teuthology.orchestra.run.smithi031.stderr:updated caps for client.0 2023-10-06T02:29:41.578 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:29:41.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.763+0000 7f19b49fe700 1 Processor -- start 2023-10-06T02:29:41.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.763+0000 7f19b49fe700 1 -- start start 2023-10-06T02:29:41.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.764+0000 7f19b49fe700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f19b00cf110 0x7f19b0164600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:41.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.764+0000 7f19b49fe700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19b015afd0 con 0x7f19b015de50 2023-10-06T02:29:41.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.764+0000 7f19b49fe700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19b015b2b0 con 0x7f19b0153c30 2023-10-06T02:29:41.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.764+0000 7f19b49fe700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19b015b590 con 0x7f19b00cf110 2023-10-06T02:29:41.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.765+0000 7f19ae59c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f19b015de50 0x7f19b00cea00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:54072/0 (socket says 172.21.15.31:54072) 2023-10-06T02:29:41.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.765+0000 7f19ae59c700 1 -- 172.21.15.31:0/2255989080 learned_addr learned my addr 172.21.15.31:0/2255989080 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:29:41.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.765+0000 7f19af59e700 1 --2- 172.21.15.31:0/2255989080 >> v2:172.21.15.62:3300/0 conn(0x7f19b00cf110 0x7f19b0164600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:41.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.765+0000 7f19add9b700 1 -- 172.21.15.31:0/2255989080 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1495124572 0 0) 0x7f19b015afd0 con 0x7f19b015de50 2023-10-06T02:29:41.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.765+0000 7f19add9b700 1 -- 172.21.15.31:0/2255989080 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1998003570 con 0x7f19b015de50 2023-10-06T02:29:41.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.765+0000 7f19add9b700 1 -- 172.21.15.31:0/2255989080 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3206797809 0 0) 0x7f19b015b2b0 con 0x7f19b0153c30 2023-10-06T02:29:41.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.765+0000 7f19add9b700 1 -- 172.21.15.31:0/2255989080 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19b015afd0 con 0x7f19b0153c30 2023-10-06T02:29:41.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.765+0000 7f19af59e700 1 -- 172.21.15.31:0/2255989080 >> v1:172.21.15.62:6789/0 conn(0x7f19b0153c30 legacy=0x7f19b015a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:41.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.765+0000 7f19af59e700 1 -- 172.21.15.31:0/2255989080 >> v1:172.21.15.31:6789/0 conn(0x7f19b015de50 legacy=0x7f19b00cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:41.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.765+0000 7f19af59e700 1 -- 172.21.15.31:0/2255989080 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19b015b870 con 0x7f19b00cf110 2023-10-06T02:29:41.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.766+0000 7f19af59e700 1 --2- 172.21.15.31:0/2255989080 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19b00cf110 0x7f19b0164600 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f19a8009e60 tx=0x7f19a8009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=892606749a4dd366 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:41.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.766+0000 7f19add9b700 1 -- 172.21.15.31:0/2255989080 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a800e070 con 0x7f19b00cf110 2023-10-06T02:29:41.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.766+0000 7f19add9b700 1 -- 172.21.15.31:0/2255989080 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19a800bab0 con 0x7f19b00cf110 2023-10-06T02:29:41.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.766+0000 7f19add9b700 1 -- 172.21.15.31:0/2255989080 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a800be70 con 0x7f19b00cf110 2023-10-06T02:29:41.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.766+0000 7f19b49fe700 1 -- 172.21.15.31:0/2255989080 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19b00cf110 msgr2=0x7f19b0164600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:41.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.766+0000 7f19b49fe700 1 --2- 172.21.15.31:0/2255989080 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19b00cf110 0x7f19b0164600 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f19a8009e60 tx=0x7f19a8009780 comp rx=0 tx=0).stop 2023-10-06T02:29:41.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.767+0000 7f19b49fe700 1 -- 172.21.15.31:0/2255989080 shutdown_connections 2023-10-06T02:29:41.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.767+0000 7f19b49fe700 1 --2- 172.21.15.31:0/2255989080 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19b00cf110 0x7f19b0164600 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:41.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.767+0000 7f19b49fe700 1 -- 172.21.15.31:0/2255989080 >> 172.21.15.31:0/2255989080 conn(0x7f19b00703a0 msgr2=0x7f19b00707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:41.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.767+0000 7f19b49fe700 1 -- 172.21.15.31:0/2255989080 shutdown_connections 2023-10-06T02:29:41.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.767+0000 7f19b49fe700 1 -- 172.21.15.31:0/2255989080 wait complete. 2023-10-06T02:29:41.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.768+0000 7f19b49fe700 1 Processor -- start 2023-10-06T02:29:41.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.768+0000 7f19b49fe700 1 -- start start 2023-10-06T02:29:41.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.769+0000 7f19b49fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19b00cf110 0x7f19b0169ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:41.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.769+0000 7f19b49fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f19b0153c30 0x7f19b0151eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:41.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.769+0000 7f19b49fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19b015de50 0x7f19b00c7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:41.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.769+0000 7f19b49fe700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19b00c8940 con 0x7f19b015de50 2023-10-06T02:29:41.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.769+0000 7f19b49fe700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19b00b4900 con 0x7f19b00cf110 2023-10-06T02:29:41.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.769+0000 7f19b49fe700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19b00b4be0 con 0x7f19b0153c30 2023-10-06T02:29:41.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.769+0000 7f19af59e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19b015de50 0x7f19b00c7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:41.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.769+0000 7f19af59e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19b015de50 0x7f19b00c7fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55436/0 (socket says 172.21.15.31:55436) 2023-10-06T02:29:41.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.769+0000 7f19af59e700 1 -- 172.21.15.31:0/116502201 learned_addr learned my addr 172.21.15.31:0/116502201 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:41.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.770+0000 7f19af59e700 1 -- 172.21.15.31:0/116502201 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f19b0153c30 msgr2=0x7f19b0151eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:41.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.770+0000 7f19af59e700 1 --2- 172.21.15.31:0/116502201 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f19b0153c30 0x7f19b0151eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:41.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.770+0000 7f19af59e700 1 -- 172.21.15.31:0/116502201 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19b00cf110 msgr2=0x7f19b0169ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:41.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.770+0000 7f19aed9d700 1 --2- 172.21.15.31:0/116502201 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19b00cf110 0x7f19b0169ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:41.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.770+0000 7f19af59e700 1 --2- 172.21.15.31:0/116502201 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19b00cf110 0x7f19b0169ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:41.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.770+0000 7f19af59e700 1 -- 172.21.15.31:0/116502201 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19a8009a20 con 0x7f19b015de50 2023-10-06T02:29:41.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.770+0000 7f19aed9d700 1 --2- 172.21.15.31:0/116502201 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19b00cf110 0x7f19b0169ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:29:41.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.770+0000 7f19af59e700 1 --2- 172.21.15.31:0/116502201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19b015de50 0x7f19b00c7fa0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f19a8009460 tx=0x7f19a8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:41.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.770+0000 7f1996ffd700 1 -- 172.21.15.31:0/116502201 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a8026480 con 0x7f19b015de50 2023-10-06T02:29:41.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.770+0000 7f1996ffd700 1 -- 172.21.15.31:0/116502201 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19a80265e0 con 0x7f19b015de50 2023-10-06T02:29:41.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.770+0000 7f1996ffd700 1 -- 172.21.15.31:0/116502201 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a80269a0 con 0x7f19b015de50 2023-10-06T02:29:41.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.770+0000 7f19b49fe700 1 -- 172.21.15.31:0/116502201 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19b00b4ec0 con 0x7f19b015de50 2023-10-06T02:29:41.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.771+0000 7f19b49fe700 1 -- 172.21.15.31:0/116502201 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19b00c8c20 con 0x7f19b015de50 2023-10-06T02:29:41.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.771+0000 7f19b49fe700 1 -- 172.21.15.31:0/116502201 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19b00b4760 con 0x7f19b015de50 2023-10-06T02:29:41.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.772+0000 7f1996ffd700 1 -- 172.21.15.31:0/116502201 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f19a8026d50 con 0x7f19b015de50 2023-10-06T02:29:41.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.773+0000 7f1996ffd700 1 --2- 172.21.15.31:0/116502201 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f199c067320 0x7f199c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:41.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.774+0000 7f1996ffd700 1 -- 172.21.15.31:0/116502201 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f19a8086ff0 con 0x7f19b015de50 2023-10-06T02:29:41.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.774+0000 7f19aed9d700 1 --2- 172.21.15.31:0/116502201 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f199c067320 0x7f199c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:41.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.775+0000 7f19aed9d700 1 --2- 172.21.15.31:0/116502201 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f199c067320 0x7f199c0697d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f19a0007fd0 tx=0x7f19a0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:41.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:41.777+0000 7f1996ffd700 1 -- 172.21.15.31:0/116502201 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f19b00b4760 con 0x7f19b015de50 2023-10-06T02:29:42.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.023+0000 7f19b49fe700 1 -- 172.21.15.31:0/116502201 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f19b00b4760 con 0x7f19b015de50 2023-10-06T02:29:42.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.023+0000 7f1996ffd700 1 -- 172.21.15.31:0/116502201 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+4590 (secure 0 0 0) 0x7f19b00b4760 con 0x7f19b015de50 2023-10-06T02:29:42.025 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:29:42.025 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":80,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6338,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/1906951645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":1906951645},{"type":"v1","addr":"172.21.15.62:6834","nonce":1906951645}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75},{"gid":6341,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3788411327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3788411327},{"type":"v1","addr":"172.21.15.62:6835","nonce":3788411327}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75},{"gid":7569,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6833/581939695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":581939695},{"type":"v1","addr":"172.21.15.31:6833","nonce":581939695}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:29:35.664920+0000","modified":"2023-10-06T02:29:39.189818+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7575},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7575":{"gid":7575,"name":"c","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/40412586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":40412586},{"type":"v1","addr":"172.21.15.31:6835","nonce":40412586}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-06T02:29:42.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.026+0000 7f19b49fe700 1 -- 172.21.15.31:0/116502201 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f199c067320 msgr2=0x7f199c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:42.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.026+0000 7f19b49fe700 1 --2- 172.21.15.31:0/116502201 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f199c067320 0x7f199c0697d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f19a0007fd0 tx=0x7f19a0009040 comp rx=0 tx=0).stop 2023-10-06T02:29:42.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.026+0000 7f19b49fe700 1 -- 172.21.15.31:0/116502201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19b015de50 msgr2=0x7f19b00c7fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:42.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.026+0000 7f19b49fe700 1 --2- 172.21.15.31:0/116502201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19b015de50 0x7f19b00c7fa0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f19a8009460 tx=0x7f19a8019000 comp rx=0 tx=0).stop 2023-10-06T02:29:42.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.026+0000 7f19b49fe700 1 -- 172.21.15.31:0/116502201 shutdown_connections 2023-10-06T02:29:42.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.026+0000 7f19b49fe700 1 --2- 172.21.15.31:0/116502201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19b015de50 0x7f19b00c7fa0 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:42.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.026+0000 7f19b49fe700 1 --2- 172.21.15.31:0/116502201 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19b00cf110 0x7f19b0169ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:42.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.026+0000 7f19b49fe700 1 --2- 172.21.15.31:0/116502201 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f199c067320 0x7f199c0697d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:42.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.026+0000 7f19b49fe700 1 --2- 172.21.15.31:0/116502201 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f19b0153c30 0x7f19b0151eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:42.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.026+0000 7f19b49fe700 1 -- 172.21.15.31:0/116502201 >> 172.21.15.31:0/116502201 conn(0x7f19b00703a0 msgr2=0x7f19b00ce960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:42.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.026+0000 7f19b49fe700 1 -- 172.21.15.31:0/116502201 shutdown_connections 2023-10-06T02:29:42.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.026+0000 7f19b49fe700 1 -- 172.21.15.31:0/116502201 wait complete. 2023-10-06T02:29:42.029 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 80 2023-10-06T02:29:42.447 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 80, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T02:29:35.664920+0000', 'modified': '2023-10-06T02:29:39.189818+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7575}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7575': {'gid': 7575, 'name': 'c', 'rank': 0, 'incarnation': 76, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.31:6835/40412586', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6834', 'nonce': 40412586}, {'type': 'v1', 'addr': '172.21.15.31:6835', 'nonce': 40412586}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T02:29:42.447 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T02:29:42.448 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-06T02:29:42.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.637+0000 7f041b19d700 1 Processor -- start 2023-10-06T02:29:42.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.637+0000 7f041b19d700 1 -- start start 2023-10-06T02:29:42.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.638+0000 7f041b19d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0414159a10 0x7f04140ca7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:42.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.638+0000 7f041b19d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0414070410 0x7f04140cacf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:42.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.638+0000 7f041b19d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04140c7420 con 0x7f0414159640 2023-10-06T02:29:42.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.638+0000 7f041b19d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f041406d820 con 0x7f0414159a10 2023-10-06T02:29:42.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.638+0000 7f041b19d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f041406db00 con 0x7f0414070410 2023-10-06T02:29:42.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.638+0000 7f041999a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0414159a10 0x7f04140ca7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:42.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.638+0000 7f041a19b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f0414159640 0x7f041406d060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:54074/0 (socket says 172.21.15.31:54074) 2023-10-06T02:29:42.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.638+0000 7f041a19b700 1 -- 172.21.15.31:0/2146837402 learned_addr learned my addr 172.21.15.31:0/2146837402 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:29:42.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.639+0000 7f041a99c700 1 --2- 172.21.15.31:0/2146837402 >> v2:172.21.15.62:3300/0 conn(0x7f0414070410 0x7f04140cacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:42.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.639+0000 7f041999a700 1 -- 172.21.15.31:0/2146837402 >> v2:172.21.15.62:3300/0 conn(0x7f0414070410 msgr2=0x7f04140cacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:42.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.639+0000 7f041999a700 1 --2- 172.21.15.31:0/2146837402 >> v2:172.21.15.62:3300/0 conn(0x7f0414070410 0x7f04140cacf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:42.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.639+0000 7f041999a700 1 -- 172.21.15.31:0/2146837402 >> v1:172.21.15.31:6789/0 conn(0x7f0414159640 legacy=0x7f041406d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:42.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.639+0000 7f041999a700 1 -- 172.21.15.31:0/2146837402 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f041406dde0 con 0x7f0414159a10 2023-10-06T02:29:42.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.639+0000 7f0419199700 1 -- 172.21.15.31:0/2146837402 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3788849911 0 0) 0x7f04140c7420 con 0x7f0414159640 2023-10-06T02:29:42.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.639+0000 7f041999a700 1 --2- 172.21.15.31:0/2146837402 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0414159a10 0x7f04140ca7b0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f04140cb820 tx=0x7f04040091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b22dd81ef6f5421 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:42.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.639+0000 7f0419199700 1 -- 172.21.15.31:0/2146837402 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f040400b6b0 con 0x7f0414159a10 2023-10-06T02:29:42.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.639+0000 7f0419199700 1 -- 172.21.15.31:0/2146837402 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f040400b810 con 0x7f0414159a10 2023-10-06T02:29:42.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.639+0000 7f0419199700 1 -- 172.21.15.31:0/2146837402 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f040400bbb0 con 0x7f0414159a10 2023-10-06T02:29:42.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.640+0000 7f041b19d700 1 -- 172.21.15.31:0/2146837402 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0414159a10 msgr2=0x7f04140ca7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:42.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.640+0000 7f041b19d700 1 --2- 172.21.15.31:0/2146837402 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0414159a10 0x7f04140ca7b0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f04140cb820 tx=0x7f04040091a0 comp rx=0 tx=0).stop 2023-10-06T02:29:42.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.640+0000 7f041b19d700 1 -- 172.21.15.31:0/2146837402 shutdown_connections 2023-10-06T02:29:42.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.640+0000 7f041b19d700 1 --2- 172.21.15.31:0/2146837402 >> v2:172.21.15.62:3300/0 conn(0x7f0414070410 0x7f04140cacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:42.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.640+0000 7f041b19d700 1 --2- 172.21.15.31:0/2146837402 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0414159a10 0x7f04140ca7b0 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:42.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.640+0000 7f041b19d700 1 -- 172.21.15.31:0/2146837402 >> 172.21.15.31:0/2146837402 conn(0x7f041406e720 msgr2=0x7f041406eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:42.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.640+0000 7f041b19d700 1 -- 172.21.15.31:0/2146837402 shutdown_connections 2023-10-06T02:29:42.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.640+0000 7f041b19d700 1 -- 172.21.15.31:0/2146837402 wait complete. 2023-10-06T02:29:42.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.642+0000 7f041b19d700 1 Processor -- start 2023-10-06T02:29:42.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.642+0000 7f041b19d700 1 -- start start 2023-10-06T02:29:42.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.643+0000 7f041b19d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0414070410 0x7f041414b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:42.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.643+0000 7f041b19d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0414159640 0x7f04140c1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:42.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.643+0000 7f041b19d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0414159a10 0x7f04140c24f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:42.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.643+0000 7f041b19d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f041416eb10 con 0x7f0414159a10 2023-10-06T02:29:42.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.643+0000 7f041b19d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f041416df00 con 0x7f0414159640 2023-10-06T02:29:42.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.643+0000 7f041b19d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f041416e1e0 con 0x7f0414070410 2023-10-06T02:29:42.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.643+0000 7f041a99c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0414159a10 0x7f04140c24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:42.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.643+0000 7f041a99c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0414159a10 0x7f04140c24f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55454/0 (socket says 172.21.15.31:55454) 2023-10-06T02:29:42.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.643+0000 7f041a99c700 1 -- 172.21.15.31:0/1179657478 learned_addr learned my addr 172.21.15.31:0/1179657478 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:42.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.643+0000 7f041a19b700 1 --2- 172.21.15.31:0/1179657478 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0414070410 0x7f041414b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:42.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.643+0000 7f041999a700 1 --2- 172.21.15.31:0/1179657478 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0414159640 0x7f04140c1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:42.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.643+0000 7f041a99c700 1 -- 172.21.15.31:0/1179657478 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0414070410 msgr2=0x7f041414b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:42.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.643+0000 7f041a99c700 1 --2- 172.21.15.31:0/1179657478 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0414070410 0x7f041414b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:42.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.643+0000 7f041a99c700 1 -- 172.21.15.31:0/1179657478 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0414159640 msgr2=0x7f04140c1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:42.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.643+0000 7f041a99c700 1 --2- 172.21.15.31:0/1179657478 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0414159640 0x7f04140c1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:42.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.643+0000 7f041a99c700 1 -- 172.21.15.31:0/1179657478 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0404009420 con 0x7f0414159a10 2023-10-06T02:29:42.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.643+0000 7f041a19b700 1 --2- 172.21.15.31:0/1179657478 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0414070410 0x7f041414b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:29:42.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.644+0000 7f041a99c700 1 --2- 172.21.15.31:0/1179657478 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0414159a10 0x7f04140c24f0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f040c00d890 tx=0x7f040c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:42.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.644+0000 7f040a7fc700 1 -- 172.21.15.31:0/1179657478 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f040c0125f0 con 0x7f0414159a10 2023-10-06T02:29:42.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.644+0000 7f040a7fc700 1 -- 172.21.15.31:0/1179657478 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f040c012770 con 0x7f0414159a10 2023-10-06T02:29:42.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.644+0000 7f040a7fc700 1 -- 172.21.15.31:0/1179657478 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f040c012b30 con 0x7f0414159a10 2023-10-06T02:29:42.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.644+0000 7f041b19d700 1 -- 172.21.15.31:0/1179657478 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f041416e4c0 con 0x7f0414159a10 2023-10-06T02:29:42.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.644+0000 7f041b19d700 1 -- 172.21.15.31:0/1179657478 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f041414b8d0 con 0x7f0414159a10 2023-10-06T02:29:42.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.646+0000 7f040a7fc700 1 -- 172.21.15.31:0/1179657478 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f040c01e460 con 0x7f0414159a10 2023-10-06T02:29:42.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.647+0000 7f040a7fc700 1 --2- 172.21.15.31:0/1179657478 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f03f8067320 0x7f03f80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:42.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.647+0000 7f040a7fc700 1 -- 172.21.15.31:0/1179657478 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f040c085570 con 0x7f0414159a10 2023-10-06T02:29:42.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.648+0000 7f041a19b700 1 --2- 172.21.15.31:0/1179657478 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f03f8067320 0x7f03f80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:42.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.648+0000 7f041a19b700 1 --2- 172.21.15.31:0/1179657478 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f03f8067320 0x7f03f80697d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0410001560 tx=0x7f0410009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:42.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.649+0000 7f042aabcb80 1 Processor -- start 2023-10-06T02:29:42.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.649+0000 7f042aabcb80 1 -- start start 2023-10-06T02:29:42.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.650+0000 7f042aabcb80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5584dcd7ae50 0x5584dcd55cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:42.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.650+0000 7f042aabcb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584dcd5b2c0 0x5584dcd56200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:42.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.650+0000 7f042aabcb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5584dcd5ca10 0x5584dcd56740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:42.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.650+0000 7f042aabcb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584dccb0a70 con 0x5584dcd5ca10 2023-10-06T02:29:42.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.650+0000 7f042aabcb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584dccae270 con 0x5584dcd5b2c0 2023-10-06T02:29:42.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.650+0000 7f042aabcb80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x5584dccaa060 con 0x5584dcd7ae50 2023-10-06T02:29:42.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.650+0000 7f041a99c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5584dcd5ca10 0x5584dcd56740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:42.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.650+0000 7f041a99c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5584dcd5ca10 0x5584dcd56740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55470/0 (socket says 172.21.15.31:55470) 2023-10-06T02:29:42.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.650+0000 7f041a99c700 1 -- 172.21.15.31:0/1416693926 learned_addr learned my addr 172.21.15.31:0/1416693926 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:42.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.651+0000 7f041a19b700 1 --2- 172.21.15.31:0/1416693926 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5584dcd7ae50 0x5584dcd55cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:42.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.651+0000 7f041999a700 1 --2- 172.21.15.31:0/1416693926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584dcd5b2c0 0x5584dcd56200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:42.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.651+0000 7f041a99c700 1 -- 172.21.15.31:0/1416693926 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5584dcd7ae50 msgr2=0x5584dcd55cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:42.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.651+0000 7f041a99c700 1 --2- 172.21.15.31:0/1416693926 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5584dcd7ae50 0x5584dcd55cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:42.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.651+0000 7f041a99c700 1 -- 172.21.15.31:0/1416693926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584dcd5b2c0 msgr2=0x5584dcd56200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:42.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.651+0000 7f041a99c700 1 --2- 172.21.15.31:0/1416693926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584dcd5b2c0 0x5584dcd56200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:42.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.651+0000 7f041a99c700 1 -- 172.21.15.31:0/1416693926 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5584dcb44270 con 0x5584dcd5ca10 2023-10-06T02:29:42.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.651+0000 7f041999a700 1 --2- 172.21.15.31:0/1416693926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584dcd5b2c0 0x5584dcd56200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:29:42.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.651+0000 7f041a99c700 1 --2- 172.21.15.31:0/1416693926 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5584dcd5ca10 0x5584dcd56740 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f040c055a20 tx=0x7f040c055b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33ac3e6a93bb4296 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:42.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.651+0000 7f03f77fe700 1 -- 172.21.15.31:0/1416693926 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f040c055da0 con 0x5584dcd5ca10 2023-10-06T02:29:42.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.651+0000 7f03f77fe700 1 -- 172.21.15.31:0/1416693926 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f040c06c170 con 0x5584dcd5ca10 2023-10-06T02:29:42.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.651+0000 7f03f77fe700 1 -- 172.21.15.31:0/1416693926 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f040c055860 con 0x5584dcd5ca10 2023-10-06T02:29:42.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.652+0000 7f042aabcb80 1 -- 172.21.15.31:0/1416693926 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5584dcd5ca10 msgr2=0x5584dcd56740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:42.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.652+0000 7f042aabcb80 1 --2- 172.21.15.31:0/1416693926 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5584dcd5ca10 0x5584dcd56740 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f040c055a20 tx=0x7f040c055b80 comp rx=0 tx=0).stop 2023-10-06T02:29:42.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.652+0000 7f042aabcb80 1 -- 172.21.15.31:0/1416693926 shutdown_connections 2023-10-06T02:29:42.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.652+0000 7f042aabcb80 1 --2- 172.21.15.31:0/1416693926 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5584dcd5ca10 0x5584dcd56740 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:42.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.652+0000 7f042aabcb80 1 --2- 172.21.15.31:0/1416693926 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5584dcd7ae50 0x5584dcd55cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:42.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.652+0000 7f042aabcb80 1 --2- 172.21.15.31:0/1416693926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584dcd5b2c0 0x5584dcd56200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:42.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.652+0000 7f042aabcb80 1 -- 172.21.15.31:0/1416693926 >> 172.21.15.31:0/1416693926 conn(0x5584dcd5ae80 msgr2=0x5584dcd5b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:42.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.652+0000 7f042aabcb80 1 -- 172.21.15.31:0/1416693926 shutdown_connections 2023-10-06T02:29:42.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.652+0000 7f042aabcb80 1 -- 172.21.15.31:0/1416693926 wait complete. 2023-10-06T02:29:42.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.653+0000 7f042aabcb80 1 -- start start 2023-10-06T02:29:42.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.653+0000 7f042aabcb80 1 Processor -- start 2023-10-06T02:29:42.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.654+0000 7f042aabcb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584dcd5b2c0 0x5584dcd78c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:42.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.654+0000 7f042aabcb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5584dcd5ca10 0x5584dcd83480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:42.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.654+0000 7f042aabcb80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5584dcd7ae50 0x5584dcdb34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:42.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.654+0000 7f042aabcb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584dcc55890 con 0x5584dcd5ca10 2023-10-06T02:29:42.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.654+0000 7f042aabcb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584dcc70ea0 con 0x5584dcd5b2c0 2023-10-06T02:29:42.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.654+0000 7f042aabcb80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x5584dcb2ce30 con 0x5584dcd7ae50 2023-10-06T02:29:42.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.654+0000 7f042aabcb80 10 client.0 _ll_register_callbacks cb 0x5584dcd1d020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T02:29:42.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.654+0000 7f041999a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5584dcd5ca10 0x5584dcd83480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:42.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.654+0000 7f041999a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5584dcd5ca10 0x5584dcd83480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55474/0 (socket says 172.21.15.31:55474) 2023-10-06T02:29:42.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.654+0000 7f041999a700 1 -- 172.21.15.31:0/1180651951 learned_addr learned my addr 172.21.15.31:0/1180651951 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:42.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.654+0000 7f041a19b700 1 --2- 172.21.15.31:0/1180651951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584dcd5b2c0 0x5584dcd78c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:42.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.654+0000 7f041a99c700 1 --2- 172.21.15.31:0/1180651951 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5584dcd7ae50 0x5584dcdb34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:42.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.654+0000 7f041999a700 1 -- 172.21.15.31:0/1180651951 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5584dcd7ae50 msgr2=0x5584dcdb34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:42.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.654+0000 7f041999a700 1 --2- 172.21.15.31:0/1180651951 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5584dcd7ae50 0x5584dcdb34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:42.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.655+0000 7f041999a700 1 -- 172.21.15.31:0/1180651951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584dcd5b2c0 msgr2=0x5584dcd78c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:42.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.655+0000 7f041999a700 1 --2- 172.21.15.31:0/1180651951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584dcd5b2c0 0x5584dcd78c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:42.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.655+0000 7f041999a700 1 -- 172.21.15.31:0/1180651951 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f041414b8d0 con 0x5584dcd5ca10 2023-10-06T02:29:42.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.655+0000 7f041999a700 1 --2- 172.21.15.31:0/1180651951 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5584dcd5ca10 0x5584dcd83480 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f0404013000 tx=0x7f0404017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6df11d1029934fa4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:42.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.655+0000 7f03f4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T02:29:42.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.655+0000 7f03f4ff9700 1 -- 172.21.15.31:0/1180651951 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f040401a420 con 0x5584dcd5ca10 2023-10-06T02:29:42.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.655+0000 7f03f4ff9700 1 -- 172.21.15.31:0/1180651951 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f040401a580 con 0x5584dcd5ca10 2023-10-06T02:29:42.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.655+0000 7f042aabcb80 1 -- 172.21.15.31:0/1180651951 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5584dcd4e940 con 0x5584dcd5ca10 2023-10-06T02:29:42.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.655+0000 7f03f4ff9700 1 -- 172.21.15.31:0/1180651951 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f040401a950 con 0x5584dcd5ca10 2023-10-06T02:29:42.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.655+0000 7f042aabcb80 10 client.7716 fetch_fsmap learned FSMap version 80 2023-10-06T02:29:42.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.655+0000 7f042aabcb80 1 -- 172.21.15.31:0/1180651951 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=2+}) v3 -- 0x5584dcae6fe0 con 0x5584dcd5ca10 2023-10-06T02:29:42.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.656+0000 7f03f4ff9700 1 -- 172.21.15.31:0/1180651951 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5584dcd4e940 con 0x5584dcd5ca10 2023-10-06T02:29:42.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.656+0000 7f03f4ff9700 1 -- 172.21.15.31:0/1180651951 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 80) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f0404023070 con 0x5584dcd5ca10 2023-10-06T02:29:42.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.656+0000 7f042aabcb80 10 client.7716 fetch_fsmap finished waiting for FSMap version 80 2023-10-06T02:29:42.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.656+0000 7f042aabcb80 10 client.7716 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T02:29:42.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.656+0000 7f042aabcb80 20 client.7716 populate_metadata read hostname 'smithi031' 2023-10-06T02:29:42.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.656+0000 7f042aabcb80 1 --2- 172.21.15.31:0/1180651951 >> [v2:172.21.15.31:6834/40412586,v1:172.21.15.31:6835/40412586] conn(0x5584dcda4750 0x5584dcda4b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:42.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.656+0000 7f042aabcb80 4 client.7716 mds_command: new command op to 7575 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T02:29:42.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.656+0000 7f042aabcb80 1 -- 172.21.15.31:0/1180651951 --> [v2:172.21.15.31:6834/40412586,v1:172.21.15.31:6835/40412586] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5584dcd2b870 con 0x5584dcda4750 2023-10-06T02:29:42.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.656+0000 7f041a19b700 1 --2- 172.21.15.31:0/1180651951 >> [v2:172.21.15.31:6834/40412586,v1:172.21.15.31:6835/40412586] conn(0x5584dcda4750 0x5584dcda4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:42.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.657+0000 7f041a19b700 1 --2- 172.21.15.31:0/1180651951 >> [v2:172.21.15.31:6834/40412586,v1:172.21.15.31:6835/40412586] conn(0x5584dcda4750 0x5584dcda4b20 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b036956c5e4078ce server_cookie=aecda72ac3ffd247 in_seq=0 out_seq=0 2023-10-06T02:29:42.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.657+0000 7f03f4ff9700 10 client.7716 ms_handle_connect on v2:172.21.15.31:6834/40412586 2023-10-06T02:29:42.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.657+0000 7f03f4ff9700 1 -- 172.21.15.31:0/1180651951 <== mds.0 v2:172.21.15.31:6834/40412586 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x5584dcd2b870 con 0x5584dcda4750 2023-10-06T02:29:42.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.657+0000 7f03f4ff9700 10 client.7716 handle_command_reply: tid=0 2023-10-06T02:29:42.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.657+0000 7f042aabcb80 1 client.7716 shutdown 2023-10-06T02:29:42.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.658+0000 7f042aabcb80 1 -- 172.21.15.31:0/1180651951 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5584dcd5ca10 msgr2=0x5584dcd83480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:42.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.658+0000 7f042aabcb80 1 --2- 172.21.15.31:0/1180651951 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5584dcd5ca10 0x5584dcd83480 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f0404013000 tx=0x7f0404017000 comp rx=0 tx=0).stop 2023-10-06T02:29:42.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.658+0000 7f042aabcb80 1 -- 172.21.15.31:0/1180651951 shutdown_connections 2023-10-06T02:29:42.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.658+0000 7f042aabcb80 1 --2- 172.21.15.31:0/1180651951 >> [v2:172.21.15.31:6834/40412586,v1:172.21.15.31:6835/40412586] conn(0x5584dcda4750 0x5584dcda4b20 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:42.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.658+0000 7f042aabcb80 1 --2- 172.21.15.31:0/1180651951 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5584dcd7ae50 0x5584dcdb34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:42.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.658+0000 7f042aabcb80 1 --2- 172.21.15.31:0/1180651951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584dcd5b2c0 0x5584dcd78c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:42.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.658+0000 7f03f4ff9700 0 client.7716 ms_handle_reset on v2:172.21.15.31:6834/40412586 2023-10-06T02:29:42.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.658+0000 7f042aabcb80 1 --2- 172.21.15.31:0/1180651951 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5584dcd5ca10 0x5584dcd83480 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:42.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.658+0000 7f042aabcb80 1 -- 172.21.15.31:0/1180651951 >> 172.21.15.31:0/1180651951 conn(0x5584dcd5ae80 msgr2=0x5584dcd59c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:42.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.658+0000 7f042aabcb80 1 -- 172.21.15.31:0/1180651951 shutdown_connections 2023-10-06T02:29:42.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.658+0000 7f042aabcb80 1 -- 172.21.15.31:0/1180651951 wait complete. 2023-10-06T02:29:42.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.658+0000 7f042aabcb80 20 client.7716 trim_cache size 0 max 16384 2023-10-06T02:29:42.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.675+0000 7f042aabcb80 1 Processor -- start 2023-10-06T02:29:42.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.676+0000 7f042aabcb80 1 -- start start 2023-10-06T02:29:42.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.676+0000 7f042aabcb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5584dcd5b2c0 0x5584dcd5b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:42.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.676+0000 7f042aabcb80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5584dcd5ca10 0x5584dcd5bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:42.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.676+0000 7f042aabcb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584dcd7ae50 0x5584dcdb1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:42.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.676+0000 7f042aabcb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584dcd87bb0 con 0x5584dcd5b2c0 2023-10-06T02:29:42.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.676+0000 7f042aabcb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584dcda2ab0 con 0x5584dcd7ae50 2023-10-06T02:29:42.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.676+0000 7f042aabcb80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x5584dcd89c20 con 0x5584dcd5ca10 2023-10-06T02:29:42.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.677+0000 7f041a19b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5584dcd5b2c0 0x5584dcd5b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:42.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.677+0000 7f041a19b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5584dcd5b2c0 0x5584dcd5b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55478/0 (socket says 172.21.15.31:55478) 2023-10-06T02:29:42.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.677+0000 7f041a19b700 1 -- 172.21.15.31:0/3368829788 learned_addr learned my addr 172.21.15.31:0/3368829788 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:42.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.677+0000 7f041999a700 1 --2- 172.21.15.31:0/3368829788 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5584dcd5ca10 0x5584dcd5bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:42.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.677+0000 7f041a99c700 1 --2- 172.21.15.31:0/3368829788 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584dcd7ae50 0x5584dcdb1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:42.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.677+0000 7f041a19b700 1 -- 172.21.15.31:0/3368829788 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5584dcd5ca10 msgr2=0x5584dcd5bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:42.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.677+0000 7f041a19b700 1 --2- 172.21.15.31:0/3368829788 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5584dcd5ca10 0x5584dcd5bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:42.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.677+0000 7f041a19b700 1 -- 172.21.15.31:0/3368829788 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584dcd7ae50 msgr2=0x5584dcdb1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:42.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.677+0000 7f041a19b700 1 --2- 172.21.15.31:0/3368829788 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584dcd7ae50 0x5584dcdb1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:42.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.677+0000 7f041a19b700 1 -- 172.21.15.31:0/3368829788 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5584dcc78780 con 0x5584dcd5b2c0 2023-10-06T02:29:42.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.677+0000 7f041a99c700 1 --2- 172.21.15.31:0/3368829788 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584dcd7ae50 0x5584dcdb1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:29:42.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.677+0000 7f041a19b700 1 --2- 172.21.15.31:0/3368829788 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5584dcd5b2c0 0x5584dcd5b690 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f0410009010 tx=0x7f0410017040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b1eed4f2a62b224 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:42.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.677+0000 7f03f7fff700 1 -- 172.21.15.31:0/3368829788 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04100150d0 con 0x5584dcd5b2c0 2023-10-06T02:29:42.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.678+0000 7f03f7fff700 1 -- 172.21.15.31:0/3368829788 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0410015230 con 0x5584dcd5b2c0 2023-10-06T02:29:42.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.678+0000 7f03f7fff700 1 -- 172.21.15.31:0/3368829788 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0410015770 con 0x5584dcd5b2c0 2023-10-06T02:29:42.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.678+0000 7f042aabcb80 1 -- 172.21.15.31:0/3368829788 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5584dcd5b2c0 msgr2=0x5584dcd5b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:42.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.678+0000 7f042aabcb80 1 --2- 172.21.15.31:0/3368829788 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5584dcd5b2c0 0x5584dcd5b690 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f0410009010 tx=0x7f0410017040 comp rx=0 tx=0).stop 2023-10-06T02:29:42.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.678+0000 7f042aabcb80 1 -- 172.21.15.31:0/3368829788 shutdown_connections 2023-10-06T02:29:42.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.678+0000 7f042aabcb80 1 --2- 172.21.15.31:0/3368829788 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584dcd7ae50 0x5584dcdb1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:42.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.678+0000 7f042aabcb80 1 --2- 172.21.15.31:0/3368829788 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5584dcd5b2c0 0x5584dcd5b690 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:42.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.678+0000 7f042aabcb80 1 --2- 172.21.15.31:0/3368829788 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5584dcd5ca10 0x5584dcd5bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:42.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.678+0000 7f042aabcb80 1 -- 172.21.15.31:0/3368829788 >> 172.21.15.31:0/3368829788 conn(0x5584dcd5ae80 msgr2=0x5584dcda4b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:42.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.678+0000 7f042aabcb80 1 -- 172.21.15.31:0/3368829788 shutdown_connections 2023-10-06T02:29:42.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.678+0000 7f042aabcb80 1 -- 172.21.15.31:0/3368829788 wait complete. 2023-10-06T02:29:42.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.679+0000 7f042aabcb80 1 -- start start 2023-10-06T02:29:42.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.679+0000 7f042aabcb80 1 Processor -- start 2023-10-06T02:29:42.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.680+0000 7f042aabcb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584dcd5b2c0 0x5584dcdffe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:42.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.680+0000 7f042aabcb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5584dcd5ca10 0x5584dcd799a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:42.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.680+0000 7f042aabcb80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5584dcd7ae50 0x5584dcd7b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:42.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.680+0000 7f042aabcb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584dcd78f90 con 0x5584dcd5ca10 2023-10-06T02:29:42.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.680+0000 7f042aabcb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584dcd7bf40 con 0x5584dcd5b2c0 2023-10-06T02:29:42.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.680+0000 7f042aabcb80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x5584dcd82750 con 0x5584dcd7ae50 2023-10-06T02:29:42.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.680+0000 7f042aabcb80 10 client.0 _ll_register_callbacks cb 0x5584dcd1d020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T02:29:42.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.680+0000 7f041999a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5584dcd5ca10 0x5584dcd799a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:42.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.680+0000 7f041999a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5584dcd5ca10 0x5584dcd799a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55490/0 (socket says 172.21.15.31:55490) 2023-10-06T02:29:42.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.680+0000 7f041a19b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584dcd5b2c0 0x5584dcdffe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:42.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.680+0000 7f041999a700 1 -- 172.21.15.31:0/4017415424 learned_addr learned my addr 172.21.15.31:0/4017415424 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:42.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.680+0000 7f041a99c700 1 --2- 172.21.15.31:0/4017415424 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5584dcd7ae50 0x5584dcd7b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:42.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.681+0000 7f041999a700 1 -- 172.21.15.31:0/4017415424 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5584dcd7ae50 msgr2=0x5584dcd7b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:42.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.681+0000 7f041999a700 1 --2- 172.21.15.31:0/4017415424 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5584dcd7ae50 0x5584dcd7b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:42.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.681+0000 7f041999a700 1 -- 172.21.15.31:0/4017415424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584dcd5b2c0 msgr2=0x5584dcdffe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:42.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.681+0000 7f041999a700 1 --2- 172.21.15.31:0/4017415424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584dcd5b2c0 0x5584dcdffe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:42.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.681+0000 7f041999a700 1 -- 172.21.15.31:0/4017415424 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f041000f040 con 0x5584dcd5ca10 2023-10-06T02:29:42.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.681+0000 7f041999a700 1 --2- 172.21.15.31:0/4017415424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5584dcd5ca10 0x5584dcd799a0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x5584dcdb3430 tx=0x7f040401c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64af6132d9709f9e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:42.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.681+0000 7f03f4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T02:29:42.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.681+0000 7f03f4ff9700 1 -- 172.21.15.31:0/4017415424 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f040401ab00 con 0x5584dcd5ca10 2023-10-06T02:29:42.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.681+0000 7f03f4ff9700 1 -- 172.21.15.31:0/4017415424 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f04040115e0 con 0x5584dcd5ca10 2023-10-06T02:29:42.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.681+0000 7f042aabcb80 1 -- 172.21.15.31:0/4017415424 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f040c06c170 con 0x5584dcd5ca10 2023-10-06T02:29:42.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.682+0000 7f03f4ff9700 1 -- 172.21.15.31:0/4017415424 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f04040119b0 con 0x5584dcd5ca10 2023-10-06T02:29:42.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.682+0000 7f042aabcb80 10 client.7722 fetch_fsmap learned FSMap version 80 2023-10-06T02:29:42.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.682+0000 7f042aabcb80 1 -- 172.21.15.31:0/4017415424 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=2+}) v3 -- 0x5584dcc5f470 con 0x5584dcd5ca10 2023-10-06T02:29:42.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.682+0000 7f03f4ff9700 1 -- 172.21.15.31:0/4017415424 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f040c06c170 con 0x5584dcd5ca10 2023-10-06T02:29:42.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.682+0000 7f03f4ff9700 1 -- 172.21.15.31:0/4017415424 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 80) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f0404009d20 con 0x5584dcd5ca10 2023-10-06T02:29:42.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.682+0000 7f042aabcb80 10 client.7722 fetch_fsmap finished waiting for FSMap version 80 2023-10-06T02:29:42.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.682+0000 7f042aabcb80 10 client.7722 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T02:29:42.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.682+0000 7f042aabcb80 20 client.7722 populate_metadata read hostname 'smithi031' 2023-10-06T02:29:42.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.682+0000 7f042aabcb80 1 --2- 172.21.15.31:0/4017415424 >> [v2:172.21.15.31:6834/40412586,v1:172.21.15.31:6835/40412586] conn(0x5584dcd55ad0 0x5584dcd55ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:42.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.682+0000 7f042aabcb80 4 client.7722 mds_command: new command op to 7575 tid=0[{"prefix": "status"}] 2023-10-06T02:29:42.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.682+0000 7f042aabcb80 1 -- 172.21.15.31:0/4017415424 --> [v2:172.21.15.31:6834/40412586,v1:172.21.15.31:6835/40412586] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5584dcc4ce00 con 0x5584dcd55ad0 2023-10-06T02:29:42.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.682+0000 7f041a19b700 1 --2- 172.21.15.31:0/4017415424 >> [v2:172.21.15.31:6834/40412586,v1:172.21.15.31:6835/40412586] conn(0x5584dcd55ad0 0x5584dcd55ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:42.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.683+0000 7f03f4ff9700 10 client.7722 ms_handle_connect on v2:172.21.15.31:6834/40412586 2023-10-06T02:29:42.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.683+0000 7f041a19b700 1 --2- 172.21.15.31:0/4017415424 >> [v2:172.21.15.31:6834/40412586,v1:172.21.15.31:6835/40412586] conn(0x5584dcd55ad0 0x5584dcd55ea0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc2b29521aa2499e server_cookie=cef887d9815ef629 in_seq=0 out_seq=0 2023-10-06T02:29:42.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.683+0000 7f03f4ff9700 1 -- 172.21.15.31:0/4017415424 <== mds.0 v2:172.21.15.31:6834/40412586 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x5584dcc4ce00 con 0x5584dcd55ad0 2023-10-06T02:29:42.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.683+0000 7f03f4ff9700 10 client.7722 handle_command_reply: tid=0 2023-10-06T02:29:42.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.683+0000 7f042aabcb80 1 client.7722 shutdown 2023-10-06T02:29:42.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.683+0000 7f042aabcb80 1 -- 172.21.15.31:0/4017415424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5584dcd5ca10 msgr2=0x5584dcd799a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:42.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.683+0000 7f042aabcb80 1 --2- 172.21.15.31:0/4017415424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5584dcd5ca10 0x5584dcd799a0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x5584dcdb3430 tx=0x7f040401c070 comp rx=0 tx=0).stop 2023-10-06T02:29:42.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.683+0000 7f042aabcb80 1 -- 172.21.15.31:0/4017415424 shutdown_connections 2023-10-06T02:29:42.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.683+0000 7f042aabcb80 1 --2- 172.21.15.31:0/4017415424 >> [v2:172.21.15.31:6834/40412586,v1:172.21.15.31:6835/40412586] conn(0x5584dcd55ad0 0x5584dcd55ea0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:42.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.683+0000 7f042aabcb80 1 --2- 172.21.15.31:0/4017415424 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5584dcd7ae50 0x5584dcd7b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:42.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.683+0000 7f042aabcb80 1 --2- 172.21.15.31:0/4017415424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584dcd5b2c0 0x5584dcdffe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:42.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.683+0000 7f03f4ff9700 0 client.7722 ms_handle_reset on v2:172.21.15.31:6834/40412586 2023-10-06T02:29:42.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.683+0000 7f042aabcb80 1 --2- 172.21.15.31:0/4017415424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5584dcd5ca10 0x5584dcd799a0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:42.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.683+0000 7f042aabcb80 1 -- 172.21.15.31:0/4017415424 >> 172.21.15.31:0/4017415424 conn(0x5584dcd5ae80 msgr2=0x5584dcd5b690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:42.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.683+0000 7f042aabcb80 1 -- 172.21.15.31:0/4017415424 shutdown_connections 2023-10-06T02:29:42.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.683+0000 7f042aabcb80 1 -- 172.21.15.31:0/4017415424 wait complete. 2023-10-06T02:29:42.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.684+0000 7f042aabcb80 20 client.7722 trim_cache size 0 max 16384 2023-10-06T02:29:42.690 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T02:29:42.691 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "d6fcf512-874b-4259-b90a-30ae070f00a9", 2023-10-06T02:29:42.691 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 0, 2023-10-06T02:29:42.691 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 7575, 2023-10-06T02:29:42.691 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2023-10-06T02:29:42.691 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2023-10-06T02:29:42.691 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2023-10-06T02:29:42.691 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 6.5190865609999999, 2023-10-06T02:29:42.691 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 80, 2023-10-06T02:29:42.691 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 120, 2023-10-06T02:29:42.691 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 120, 2023-10-06T02:29:42.691 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 13.255254269 2023-10-06T02:29:42.692 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T02:29:42.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.684+0000 7f041b19d700 1 -- 172.21.15.31:0/1179657478 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f03f8067320 msgr2=0x7f03f80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:42.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.684+0000 7f041b19d700 1 --2- 172.21.15.31:0/1179657478 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f03f8067320 0x7f03f80697d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0410001560 tx=0x7f0410009040 comp rx=0 tx=0).stop 2023-10-06T02:29:42.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.684+0000 7f041b19d700 1 -- 172.21.15.31:0/1179657478 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0414159a10 msgr2=0x7f04140c24f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:42.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.684+0000 7f041b19d700 1 --2- 172.21.15.31:0/1179657478 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0414159a10 0x7f04140c24f0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f040c00d890 tx=0x7f040c008040 comp rx=0 tx=0).stop 2023-10-06T02:29:42.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.684+0000 7f041b19d700 1 -- 172.21.15.31:0/1179657478 shutdown_connections 2023-10-06T02:29:42.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.684+0000 7f041b19d700 1 --2- 172.21.15.31:0/1179657478 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0414159a10 0x7f04140c24f0 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:42.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.684+0000 7f041b19d700 1 --2- 172.21.15.31:0/1179657478 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f03f8067320 0x7f03f80697d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:42.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.684+0000 7f041b19d700 1 --2- 172.21.15.31:0/1179657478 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0414070410 0x7f041414b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:42.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.684+0000 7f041b19d700 1 --2- 172.21.15.31:0/1179657478 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0414159640 0x7f04140c1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:42.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.684+0000 7f041b19d700 1 -- 172.21.15.31:0/1179657478 >> 172.21.15.31:0/1179657478 conn(0x7f041406e720 msgr2=0x7f041414bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:42.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.684+0000 7f041b19d700 1 -- 172.21.15.31:0/1179657478 shutdown_connections 2023-10-06T02:29:42.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.684+0000 7f041b19d700 1 -- 172.21.15.31:0/1179657478 wait complete. 2023-10-06T02:29:42.702 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T02:29:42.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.887+0000 7fa6b0b9d700 1 Processor -- start 2023-10-06T02:29:42.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.887+0000 7fa6b0b9d700 1 -- start start 2023-10-06T02:29:42.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.889+0000 7fa6b0b9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa6ac159a10 0x7fa6ac0ca7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:42.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.889+0000 7fa6b0b9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6ac0c7420 con 0x7fa6ac070410 2023-10-06T02:29:42.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.889+0000 7fa6b0b9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6ac06d820 con 0x7fa6ac159640 2023-10-06T02:29:42.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.889+0000 7fa6b0b9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ac06db00 con 0x7fa6ac159a10 2023-10-06T02:29:42.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.889+0000 7fa6abfff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fa6ac070410 0x7fa6ac0c78b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:54088/0 (socket says 172.21.15.31:54088) 2023-10-06T02:29:42.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.889+0000 7fa6abfff700 1 -- 172.21.15.31:0/2814921750 learned_addr learned my addr 172.21.15.31:0/2814921750 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:29:42.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.889+0000 7fa6aaffd700 1 --2- 172.21.15.31:0/2814921750 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa6ac159a10 0x7fa6ac0ca7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:42.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.890+0000 7fa6aa7fc700 1 -- 172.21.15.31:0/2814921750 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1464991807 0 0) 0x7fa6ac0c7420 con 0x7fa6ac070410 2023-10-06T02:29:42.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.890+0000 7fa6aa7fc700 1 -- 172.21.15.31:0/2814921750 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa694003570 con 0x7fa6ac070410 2023-10-06T02:29:42.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.890+0000 7fa6aa7fc700 1 -- 172.21.15.31:0/2814921750 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3354286478 0 0) 0x7fa6ac06d820 con 0x7fa6ac159640 2023-10-06T02:29:42.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.890+0000 7fa6aa7fc700 1 -- 172.21.15.31:0/2814921750 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6ac0c7420 con 0x7fa6ac159640 2023-10-06T02:29:42.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.890+0000 7fa6aaffd700 1 -- 172.21.15.31:0/2814921750 >> v1:172.21.15.62:6789/0 conn(0x7fa6ac159640 legacy=0x7fa6ac06d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:42.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.890+0000 7fa6aa7fc700 1 -- 172.21.15.31:0/2814921750 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3410197293 0 0) 0x7fa694003570 con 0x7fa6ac070410 2023-10-06T02:29:42.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.890+0000 7fa6aaffd700 1 -- 172.21.15.31:0/2814921750 >> v1:172.21.15.31:6789/0 conn(0x7fa6ac070410 legacy=0x7fa6ac0c78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:42.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.890+0000 7fa6aaffd700 1 -- 172.21.15.31:0/2814921750 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6ac06dde0 con 0x7fa6ac159a10 2023-10-06T02:29:42.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.890+0000 7fa6aaffd700 1 --2- 172.21.15.31:0/2814921750 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa6ac159a10 0x7fa6ac0ca7b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa6a0009910 tx=0x7fa6a00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5aff2bb6e15da70 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:42.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.891+0000 7fa6aa7fc700 1 -- 172.21.15.31:0/2814921750 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a40017f0 con 0x7fa6ac159a10 2023-10-06T02:29:42.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.891+0000 7fa6aa7fc700 1 -- 172.21.15.31:0/2814921750 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6a0011420 con 0x7fa6ac159a10 2023-10-06T02:29:42.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.891+0000 7fa6aa7fc700 1 -- 172.21.15.31:0/2814921750 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a00117c0 con 0x7fa6ac159a10 2023-10-06T02:29:42.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.891+0000 7fa6b0b9d700 1 -- 172.21.15.31:0/2814921750 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa6ac159a10 msgr2=0x7fa6ac0ca7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:42.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.891+0000 7fa6b0b9d700 1 --2- 172.21.15.31:0/2814921750 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa6ac159a10 0x7fa6ac0ca7b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa6a0009910 tx=0x7fa6a00091a0 comp rx=0 tx=0).stop 2023-10-06T02:29:42.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.891+0000 7fa6b0b9d700 1 -- 172.21.15.31:0/2814921750 shutdown_connections 2023-10-06T02:29:42.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.891+0000 7fa6b0b9d700 1 --2- 172.21.15.31:0/2814921750 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa6ac159a10 0x7fa6ac0ca7b0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:42.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.891+0000 7fa6b0b9d700 1 -- 172.21.15.31:0/2814921750 >> 172.21.15.31:0/2814921750 conn(0x7fa6ac06e720 msgr2=0x7fa6ac06eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:42.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.891+0000 7fa6b0b9d700 1 -- 172.21.15.31:0/2814921750 shutdown_connections 2023-10-06T02:29:42.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.891+0000 7fa6b0b9d700 1 -- 172.21.15.31:0/2814921750 wait complete. 2023-10-06T02:29:42.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.893+0000 7fa6b0b9d700 1 Processor -- start 2023-10-06T02:29:42.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.893+0000 7fa6b0b9d700 1 -- start start 2023-10-06T02:29:42.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.894+0000 7fa6b0b9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6ac070410 0x7fa6ac14b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:42.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.894+0000 7fa6b0b9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa6ac159640 0x7fa6ac0b1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:42.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.894+0000 7fa6b0b9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6ac159a10 0x7fa6ac0c2840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:42.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.894+0000 7fa6b0b9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ac15c370 con 0x7fa6ac159a10 2023-10-06T02:29:42.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.894+0000 7fa6b0b9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ac15c800 con 0x7fa6ac070410 2023-10-06T02:29:42.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.894+0000 7fa6b0b9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ac15b760 con 0x7fa6ac159640 2023-10-06T02:29:42.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.894+0000 7fa6abfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6ac159a10 0x7fa6ac0c2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:42.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.894+0000 7fa6abfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6ac159a10 0x7fa6ac0c2840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55494/0 (socket says 172.21.15.31:55494) 2023-10-06T02:29:42.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.894+0000 7fa6abfff700 1 -- 172.21.15.31:0/3426827293 learned_addr learned my addr 172.21.15.31:0/3426827293 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:42.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.894+0000 7fa6aaffd700 1 --2- 172.21.15.31:0/3426827293 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa6ac159640 0x7fa6ac0b1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:42.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.894+0000 7fa6ab7fe700 1 --2- 172.21.15.31:0/3426827293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6ac070410 0x7fa6ac14b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:42.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.894+0000 7fa6abfff700 1 -- 172.21.15.31:0/3426827293 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa6ac159640 msgr2=0x7fa6ac0b1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:42.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.894+0000 7fa6abfff700 1 --2- 172.21.15.31:0/3426827293 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa6ac159640 0x7fa6ac0b1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:42.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.894+0000 7fa6abfff700 1 -- 172.21.15.31:0/3426827293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6ac070410 msgr2=0x7fa6ac14b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:42.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.894+0000 7fa6abfff700 1 --2- 172.21.15.31:0/3426827293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6ac070410 0x7fa6ac14b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:42.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.894+0000 7fa6abfff700 1 -- 172.21.15.31:0/3426827293 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6a0009420 con 0x7fa6ac159a10 2023-10-06T02:29:42.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.895+0000 7fa6abfff700 1 --2- 172.21.15.31:0/3426827293 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6ac159a10 0x7fa6ac0c2840 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fa6a400cca0 tx=0x7fa6a4007d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:42.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.895+0000 7fa6937fe700 1 -- 172.21.15.31:0/3426827293 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a400ce50 con 0x7fa6ac159a10 2023-10-06T02:29:42.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.895+0000 7fa6937fe700 1 -- 172.21.15.31:0/3426827293 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6a4004dc0 con 0x7fa6ac159a10 2023-10-06T02:29:42.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.895+0000 7fa6937fe700 1 -- 172.21.15.31:0/3426827293 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a400ab70 con 0x7fa6ac159a10 2023-10-06T02:29:42.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.895+0000 7fa6b0b9d700 1 -- 172.21.15.31:0/3426827293 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6ac15ba40 con 0x7fa6ac159a10 2023-10-06T02:29:42.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.895+0000 7fa6b0b9d700 1 -- 172.21.15.31:0/3426827293 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6ac15c000 con 0x7fa6ac159a10 2023-10-06T02:29:42.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.896+0000 7fa6b0b9d700 1 -- 172.21.15.31:0/3426827293 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6ac15c4e0 con 0x7fa6ac159a10 2023-10-06T02:29:42.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.897+0000 7fa6937fe700 1 -- 172.21.15.31:0/3426827293 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fa6a4007150 con 0x7fa6ac159a10 2023-10-06T02:29:42.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.898+0000 7fa6937fe700 1 --2- 172.21.15.31:0/3426827293 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa698067320 0x7fa6980697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:42.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.898+0000 7fa6937fe700 1 -- 172.21.15.31:0/3426827293 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa6a40846c0 con 0x7fa6ac159a10 2023-10-06T02:29:42.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.899+0000 7fa6ab7fe700 1 --2- 172.21.15.31:0/3426827293 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa698067320 0x7fa6980697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:42.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.899+0000 7fa6ab7fe700 1 --2- 172.21.15.31:0/3426827293 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa698067320 0x7fa6980697d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa69c003870 tx=0x7fa69c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:42.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:42.903+0000 7fa6937fe700 1 -- 172.21.15.31:0/3426827293 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa6a4051230 con 0x7fa6ac159a10 2023-10-06T02:29:43.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:43.150+0000 7fa6b0b9d700 1 -- 172.21.15.31:0/3426827293 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa6ac04c3c0 con 0x7fa6ac159a10 2023-10-06T02:29:43.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:43.151+0000 7fa6937fe700 1 -- 172.21.15.31:0/3426827293 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v80) v1 ==== 53+0+73 (secure 0 0 0) 0x7fa6a4082e60 con 0x7fa6ac159a10 2023-10-06T02:29:43.153 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T02:29:43.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:43.153+0000 7fa6b0b9d700 1 -- 172.21.15.31:0/3426827293 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa698067320 msgr2=0x7fa6980697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:43.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:43.153+0000 7fa6b0b9d700 1 --2- 172.21.15.31:0/3426827293 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa698067320 0x7fa6980697d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa69c003870 tx=0x7fa69c009040 comp rx=0 tx=0).stop 2023-10-06T02:29:43.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:43.153+0000 7fa6b0b9d700 1 -- 172.21.15.31:0/3426827293 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6ac159a10 msgr2=0x7fa6ac0c2840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:43.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:43.153+0000 7fa6b0b9d700 1 --2- 172.21.15.31:0/3426827293 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6ac159a10 0x7fa6ac0c2840 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fa6a400cca0 tx=0x7fa6a4007d20 comp rx=0 tx=0).stop 2023-10-06T02:29:43.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:43.153+0000 7fa6b0b9d700 1 -- 172.21.15.31:0/3426827293 shutdown_connections 2023-10-06T02:29:43.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:43.153+0000 7fa6b0b9d700 1 --2- 172.21.15.31:0/3426827293 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6ac159a10 0x7fa6ac0c2840 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:43.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:43.153+0000 7fa6b0b9d700 1 --2- 172.21.15.31:0/3426827293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6ac070410 0x7fa6ac14b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:43.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:43.153+0000 7fa6b0b9d700 1 --2- 172.21.15.31:0/3426827293 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa698067320 0x7fa6980697d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:43.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:43.153+0000 7fa6b0b9d700 1 --2- 172.21.15.31:0/3426827293 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa6ac159640 0x7fa6ac0b1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:43.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:43.153+0000 7fa6b0b9d700 1 -- 172.21.15.31:0/3426827293 >> 172.21.15.31:0/3426827293 conn(0x7fa6ac06e720 msgr2=0x7fa6ac157b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:43.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:43.153+0000 7fa6b0b9d700 1 -- 172.21.15.31:0/3426827293 shutdown_connections 2023-10-06T02:29:43.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:43.153+0000 7fa6b0b9d700 1 -- 172.21.15.31:0/3426827293 wait complete. 2023-10-06T02:29:43.171 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T02:29:43.172 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T02:29:43.172 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi031.front.sepia.ceph.com 2023-10-06T02:29:43.172 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T02:29:43.172 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T02:29:43.172 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T02:29:43.172 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T02:29:43.172 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T02:29:43.172 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T02:29:43.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:29:43.173 DEBUG:teuthology.orchestra.run.smithi031:> ip addr 2023-10-06T02:29:43.211 INFO:teuthology.orchestra.run.smithi031.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T02:29:43.212 INFO:teuthology.orchestra.run.smithi031.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T02:29:43.212 INFO:teuthology.orchestra.run.smithi031.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T02:29:43.212 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:29:43.212 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 ::1/128 scope host 2023-10-06T02:29:43.212 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:29:43.212 INFO:teuthology.orchestra.run.smithi031.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T02:29:43.212 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cc brd ff:ff:ff:ff:ff:ff 2023-10-06T02:29:43.212 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f0 2023-10-06T02:29:43.212 INFO:teuthology.orchestra.run.smithi031.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T02:29:43.212 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cd brd ff:ff:ff:ff:ff:ff 2023-10-06T02:29:43.212 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f1 2023-10-06T02:29:43.213 INFO:teuthology.orchestra.run.smithi031.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T02:29:43.213 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:ba brd ff:ff:ff:ff:ff:ff 2023-10-06T02:29:43.213 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f0 2023-10-06T02:29:43.213 INFO:teuthology.orchestra.run.smithi031.stdout: inet 172.21.15.31/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T02:29:43.213 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft 41873sec preferred_lft 41873sec 2023-10-06T02:29:43.213 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::ae1f:6bff:fea5:caba/64 scope link 2023-10-06T02:29:43.213 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:29:43.213 INFO:teuthology.orchestra.run.smithi031.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T02:29:43.213 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:bb brd ff:ff:ff:ff:ff:ff 2023-10-06T02:29:43.213 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f1 2023-10-06T02:29:43.213 INFO:teuthology.orchestra.run.smithi031.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T02:29:43.214 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 16:d7:88:2d:d0:c2 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:29:43.214 INFO:teuthology.orchestra.run.smithi031.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T02:29:43.214 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:29:43.214 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::408:52ff:fe3c:ba8f/64 scope link 2023-10-06T02:29:43.214 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:29:43.214 INFO:teuthology.orchestra.run.smithi031.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T02:29:43.214 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 16:d7:88:2d:d0:c2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T02:29:43.214 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::14d7:88ff:fe2d:d0c2/64 scope link 2023-10-06T02:29:43.214 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:29:43.214 INFO:teuthology.orchestra.run.smithi031.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T02:29:43.214 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 82:16:f1:00:cf:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T02:29:43.214 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::8016:f1ff:fe00:cf2f/64 scope link 2023-10-06T02:29:43.215 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:29:43.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:29:43.216 DEBUG:teuthology.orchestra.run.smithi031:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T02:29:43.315 INFO:teuthology.orchestra.run.smithi031.stdout:1 2023-10-06T02:29:43.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:29:43.319 DEBUG:teuthology.orchestra.run.smithi031:> route 2023-10-06T02:29:43.357 INFO:teuthology.orchestra.run.smithi031.stdout:Kernel IP routing table 2023-10-06T02:29:43.357 INFO:teuthology.orchestra.run.smithi031.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T02:29:43.357 INFO:teuthology.orchestra.run.smithi031.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T02:29:43.357 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T02:29:43.357 INFO:teuthology.orchestra.run.smithi031.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T02:29:43.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:29:43.359 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2023-10-06T02:29:43.359 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2023-10-06T02:29:43.359 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T02:29:43.359 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T02:29:43.359 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T02:29:43.359 DEBUG:teuthology.orchestra.run.smithi031:> ') 2023-10-06T02:29:43.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:29:43.557 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2023-10-06T02:29:43.594 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T02:29:43.594 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T02:29:43.594 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T02:29:43.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:29:43.594 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set brx.0 up 2023-10-06T02:29:43.697 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T02:29:43.697 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T02:29:43.733 INFO:teuthology.orchestra.run.smithi031.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T02:29:43.734 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T02:29:43.734 DEBUG:teuthology.orchestra.run.smithi031:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T02:29:43.810 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-06T02:29:43.810 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T02:29:43.810 DEBUG:teuthology.orchestra.run.smithi031:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T02:29:43.943 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T02:29:43.943 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T02:29:43.972 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T02:29:43.972 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T02:29:43.972 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.62:3300,172.21.15.62:3301" 2023-10-06T02:29:43.972 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: trying mount with new device syntax: 0@d6fcf512-874b-4259-b90a-30ae070f00a9.cephfs=/ 2023-10-06T02:29:43.972 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.31:3300/172.21.15.62:3300/172.21.15.62:3301" will pass to kernel 2023-10-06T02:29:43.976 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T02:29:43.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:29:43.977 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T02:29:44.039 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T02:29:44.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:44.227+0000 7fb52759e700 1 Processor -- start 2023-10-06T02:29:44.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:44.227+0000 7fb52759e700 1 -- start start 2023-10-06T02:29:44.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:44.229+0000 7fb52759e700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb528140e40 0x7fb5280c6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:44.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:44.229+0000 7fb52759e700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5280c6d50 con 0x7fb528140a70 2023-10-06T02:29:44.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:44.229+0000 7fb52759e700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5280c7030 con 0x7fb52806de50 2023-10-06T02:29:44.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:44.229+0000 7fb52759e700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5280c7310 con 0x7fb528140e40 2023-10-06T02:29:44.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:44.229+0000 7fb52659c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fb528140a70 0x7fb5280c7df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:54102/0 (socket says 172.21.15.31:54102) 2023-10-06T02:29:44.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:44.229+0000 7fb52659c700 1 -- 172.21.15.31:0/532253712 learned_addr learned my addr 172.21.15.31:0/532253712 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:29:44.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:44.229+0000 7fb525d9b700 1 --2- 172.21.15.31:0/532253712 >> v2:172.21.15.62:3300/0 conn(0x7fb528140e40 0x7fb5280c6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:44.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:44.229+0000 7fb52559a700 1 -- 172.21.15.31:0/532253712 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4238828206 0 0) 0x7fb5280c6d50 con 0x7fb528140a70 2023-10-06T02:29:44.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:44.229+0000 7fb52559a700 1 -- 172.21.15.31:0/532253712 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb510003570 con 0x7fb528140a70 2023-10-06T02:29:44.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:44.230+0000 7fb52559a700 1 -- 172.21.15.31:0/532253712 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2665685422 0 0) 0x7fb510003570 con 0x7fb528140a70 2023-10-06T02:29:44.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:44.230+0000 7fb525d9b700 1 -- 172.21.15.31:0/532253712 >> v1:172.21.15.62:6789/0 conn(0x7fb52806de50 legacy=0x7fb5280aeef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:44.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:44.230+0000 7fb525d9b700 1 -- 172.21.15.31:0/532253712 >> v1:172.21.15.31:6789/0 conn(0x7fb528140a70 legacy=0x7fb5280c7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:44.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:44.230+0000 7fb525d9b700 1 -- 172.21.15.31:0/532253712 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5280c75f0 con 0x7fb528140e40 2023-10-06T02:29:44.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:44.230+0000 7fb525d9b700 1 --2- 172.21.15.31:0/532253712 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb528140e40 0x7fb5280c6770 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fb51c0098d0 tx=0x7fb51c009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7121f09e91f2659b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:44.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:44.230+0000 7fb52559a700 1 -- 172.21.15.31:0/532253712 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb518001470 con 0x7fb528140e40 2023-10-06T02:29:44.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:44.230+0000 7fb52559a700 1 -- 172.21.15.31:0/532253712 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb51c011420 con 0x7fb528140e40 2023-10-06T02:29:44.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:44.230+0000 7fb52559a700 1 -- 172.21.15.31:0/532253712 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51c0117c0 con 0x7fb528140e40 2023-10-06T02:29:44.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:44.231+0000 7fb52759e700 1 -- 172.21.15.31:0/532253712 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb528140e40 msgr2=0x7fb5280c6770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:44.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:44.231+0000 7fb52759e700 1 --2- 172.21.15.31:0/532253712 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb528140e40 0x7fb5280c6770 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fb51c0098d0 tx=0x7fb51c009120 comp rx=0 tx=0).stop 2023-10-06T02:29:44.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:44.231+0000 7fb52759e700 1 -- 172.21.15.31:0/532253712 shutdown_connections 2023-10-06T02:29:44.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:44.231+0000 7fb52759e700 1 --2- 172.21.15.31:0/532253712 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb528140e40 0x7fb5280c6770 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:44.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:44.231+0000 7fb52759e700 1 -- 172.21.15.31:0/532253712 >> 172.21.15.31:0/532253712 conn(0x7fb52806d170 msgr2=0x7fb5280cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:44.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:44.231+0000 7fb52759e700 1 -- 172.21.15.31:0/532253712 shutdown_connections 2023-10-06T02:29:44.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:44.231+0000 7fb52759e700 1 -- 172.21.15.31:0/532253712 wait complete. 2023-10-06T02:29:44.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:44.232+0000 7fb52759e700 1 Processor -- start 2023-10-06T02:29:44.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:44.232+0000 7fb52759e700 1 -- start start 2023-10-06T02:29:44.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:44.233+0000 7fb52759e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb52806de50 0x7fb5280add60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:44.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:44.233+0000 7fb52759e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb528140a70 0x7fb528158af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:44.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:44.233+0000 7fb52759e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb528140e40 0x7fb528157af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:44.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:44.233+0000 7fb52759e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5280c5440 con 0x7fb52806de50 2023-10-06T02:29:44.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:44.233+0000 7fb52759e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb52816b170 con 0x7fb528140a70 2023-10-06T02:29:44.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:44.233+0000 7fb52759e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5280c56e0 con 0x7fb528140e40 2023-10-06T02:29:44.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:44.234+0000 7fb52659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb52806de50 0x7fb5280add60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:44.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:44.234+0000 7fb525d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb528140a70 0x7fb528158af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:44.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:44.234+0000 7fb52659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb52806de50 0x7fb5280add60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55500/0 (socket says 172.21.15.31:55500) 2023-10-06T02:29:44.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:44.234+0000 7fb52659c700 1 -- 172.21.15.31:0/4223644887 learned_addr learned my addr 172.21.15.31:0/4223644887 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:44.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:44.234+0000 7fb526d9d700 1 --2- 172.21.15.31:0/4223644887 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb528140e40 0x7fb528157af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:44.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:44.234+0000 7fb52659c700 1 -- 172.21.15.31:0/4223644887 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb528140e40 msgr2=0x7fb528157af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:44.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:44.234+0000 7fb52659c700 1 --2- 172.21.15.31:0/4223644887 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb528140e40 0x7fb528157af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:44.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:44.234+0000 7fb52659c700 1 -- 172.21.15.31:0/4223644887 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb528140a70 msgr2=0x7fb528158af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:44.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:44.234+0000 7fb52659c700 1 --2- 172.21.15.31:0/4223644887 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb528140a70 0x7fb528158af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:44.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:44.234+0000 7fb52659c700 1 -- 172.21.15.31:0/4223644887 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb51c0093c0 con 0x7fb52806de50 2023-10-06T02:29:44.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:44.234+0000 7fb526d9d700 1 --2- 172.21.15.31:0/4223644887 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb528140e40 0x7fb528157af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:29:44.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:44.234+0000 7fb525d9b700 1 --2- 172.21.15.31:0/4223644887 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb528140a70 0x7fb528158af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:29:44.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:44.234+0000 7fb52659c700 1 --2- 172.21.15.31:0/4223644887 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb52806de50 0x7fb5280add60 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fb51800cb90 tx=0x7fb518001ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:44.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:44.235+0000 7fb5167fc700 1 -- 172.21.15.31:0/4223644887 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb518001d60 con 0x7fb52806de50 2023-10-06T02:29:44.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:44.235+0000 7fb5167fc700 1 -- 172.21.15.31:0/4223644887 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb518007360 con 0x7fb52806de50 2023-10-06T02:29:44.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:44.235+0000 7fb52759e700 1 -- 172.21.15.31:0/4223644887 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5280c59c0 con 0x7fb52806de50 2023-10-06T02:29:44.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:44.235+0000 7fb5167fc700 1 -- 172.21.15.31:0/4223644887 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb518007700 con 0x7fb52806de50 2023-10-06T02:29:44.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:44.235+0000 7fb52759e700 1 -- 172.21.15.31:0/4223644887 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5280c5f80 con 0x7fb52806de50 2023-10-06T02:29:44.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:44.236+0000 7fb52759e700 1 -- 172.21.15.31:0/4223644887 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb52816b330 con 0x7fb52806de50 2023-10-06T02:29:44.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:44.237+0000 7fb5167fc700 1 -- 172.21.15.31:0/4223644887 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fb518004a40 con 0x7fb52806de50 2023-10-06T02:29:44.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:44.238+0000 7fb5167fc700 1 --2- 172.21.15.31:0/4223644887 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb508067320 0x7fb5080697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:44.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:44.238+0000 7fb5167fc700 1 -- 172.21.15.31:0/4223644887 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb5180841d0 con 0x7fb52806de50 2023-10-06T02:29:44.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:44.238+0000 7fb525d9b700 1 --2- 172.21.15.31:0/4223644887 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb508067320 0x7fb5080697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:44.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:44.239+0000 7fb525d9b700 1 --2- 172.21.15.31:0/4223644887 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb508067320 0x7fb5080697d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb5281412a0 tx=0x7fb51c016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:44.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:44.242+0000 7fb5167fc700 1 -- 172.21.15.31:0/4223644887 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb518050d00 con 0x7fb52806de50 2023-10-06T02:29:44.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:44.491+0000 7fb52759e700 1 -- 172.21.15.31:0/4223644887 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb52804c3c0 con 0x7fb52806de50 2023-10-06T02:29:44.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:44.492+0000 7fb5167fc700 1 -- 172.21.15.31:0/4223644887 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v80) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb518054320 con 0x7fb52806de50 2023-10-06T02:29:44.493 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T02:29:44.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:44.494+0000 7fb52759e700 1 -- 172.21.15.31:0/4223644887 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb508067320 msgr2=0x7fb5080697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:44.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:44.494+0000 7fb52759e700 1 --2- 172.21.15.31:0/4223644887 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb508067320 0x7fb5080697d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb5281412a0 tx=0x7fb51c016000 comp rx=0 tx=0).stop 2023-10-06T02:29:44.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:44.494+0000 7fb52759e700 1 -- 172.21.15.31:0/4223644887 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb52806de50 msgr2=0x7fb5280add60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:44.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:44.494+0000 7fb52759e700 1 --2- 172.21.15.31:0/4223644887 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb52806de50 0x7fb5280add60 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fb51800cb90 tx=0x7fb518001ad0 comp rx=0 tx=0).stop 2023-10-06T02:29:44.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:44.494+0000 7fb52759e700 1 -- 172.21.15.31:0/4223644887 shutdown_connections 2023-10-06T02:29:44.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:44.494+0000 7fb52759e700 1 --2- 172.21.15.31:0/4223644887 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb508067320 0x7fb5080697d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:44.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:44.494+0000 7fb52759e700 1 --2- 172.21.15.31:0/4223644887 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb528140e40 0x7fb528157af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:44.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:44.494+0000 7fb52759e700 1 --2- 172.21.15.31:0/4223644887 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb52806de50 0x7fb5280add60 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:44.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:44.494+0000 7fb52759e700 1 --2- 172.21.15.31:0/4223644887 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb528140a70 0x7fb528158af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:44.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:44.494+0000 7fb52759e700 1 -- 172.21.15.31:0/4223644887 >> 172.21.15.31:0/4223644887 conn(0x7fb52806d170 msgr2=0x7fb528069a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:44.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:44.494+0000 7fb52759e700 1 -- 172.21.15.31:0/4223644887 shutdown_connections 2023-10-06T02:29:44.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:44.494+0000 7fb52759e700 1 -- 172.21.15.31:0/4223644887 wait complete. 2023-10-06T02:29:44.513 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T02:29:44.513 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T02:29:44.513 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi062.front.sepia.ceph.com 2023-10-06T02:29:44.513 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-06T02:29:44.514 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-06T02:29:44.514 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T02:29:44.514 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T02:29:44.514 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T02:29:44.514 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T02:29:44.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:29:44.514 DEBUG:teuthology.orchestra.run.smithi062:> ip addr 2023-10-06T02:29:44.556 INFO:teuthology.orchestra.run.smithi062.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T02:29:44.556 INFO:teuthology.orchestra.run.smithi062.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T02:29:44.556 INFO:teuthology.orchestra.run.smithi062.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T02:29:44.557 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:29:44.557 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 ::1/128 scope host 2023-10-06T02:29:44.557 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:29:44.557 INFO:teuthology.orchestra.run.smithi062.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T02:29:44.557 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:14 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:29:44.557 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f0 2023-10-06T02:29:44.557 INFO:teuthology.orchestra.run.smithi062.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T02:29:44.557 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:15 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:29:44.558 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f1 2023-10-06T02:29:44.558 INFO:teuthology.orchestra.run.smithi062.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T02:29:44.558 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c2 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:29:44.558 INFO:teuthology.orchestra.run.smithi062.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T02:29:44.558 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c3 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:29:44.558 INFO:teuthology.orchestra.run.smithi062.stdout: inet 172.21.15.62/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T02:29:44.558 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft 41864sec preferred_lft 41864sec 2023-10-06T02:29:44.559 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::ec4:7aff:febb:d1c3/64 scope link 2023-10-06T02:29:44.559 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:29:44.559 INFO:teuthology.orchestra.run.smithi062.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T02:29:44.559 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether ae:42:a8:dd:f7:9b brd ff:ff:ff:ff:ff:ff 2023-10-06T02:29:44.559 INFO:teuthology.orchestra.run.smithi062.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T02:29:44.559 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:29:44.559 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::1098:a4ff:fe0d:389/64 scope link 2023-10-06T02:29:44.559 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:29:44.559 INFO:teuthology.orchestra.run.smithi062.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T02:29:44.559 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether fe:60:2b:8e:af:28 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T02:29:44.559 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::fc60:2bff:fe8e:af28/64 scope link 2023-10-06T02:29:44.559 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:29:44.559 INFO:teuthology.orchestra.run.smithi062.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T02:29:44.559 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether ae:42:a8:dd:f7:9b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T02:29:44.560 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::ac42:a8ff:fedd:f79b/64 scope link 2023-10-06T02:29:44.560 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:29:44.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:29:44.560 DEBUG:teuthology.orchestra.run.smithi062:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T02:29:44.661 INFO:teuthology.orchestra.run.smithi062.stdout:1 2023-10-06T02:29:44.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:29:44.664 DEBUG:teuthology.orchestra.run.smithi062:> route 2023-10-06T02:29:44.704 INFO:teuthology.orchestra.run.smithi062.stdout:Kernel IP routing table 2023-10-06T02:29:44.704 INFO:teuthology.orchestra.run.smithi062.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T02:29:44.705 INFO:teuthology.orchestra.run.smithi062.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T02:29:44.705 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T02:29:44.705 INFO:teuthology.orchestra.run.smithi062.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T02:29:44.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:29:44.706 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2023-10-06T02:29:44.706 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2023-10-06T02:29:44.706 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T02:29:44.706 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T02:29:44.706 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T02:29:44.707 DEBUG:teuthology.orchestra.run.smithi062:> ') 2023-10-06T02:29:44.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:29:44.904 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2023-10-06T02:29:44.941 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-06T02:29:44.941 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T02:29:44.942 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T02:29:44.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:29:44.942 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set brx.0 up 2023-10-06T02:29:45.044 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T02:29:45.044 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-06T02:29:45.081 INFO:teuthology.orchestra.run.smithi062.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-06T02:29:45.082 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T02:29:45.082 DEBUG:teuthology.orchestra.run.smithi062:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-06T02:29:45.157 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-06T02:29:45.157 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T02:29:45.158 DEBUG:teuthology.orchestra.run.smithi062:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T02:29:45.291 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T02:29:45.291 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T02:29:45.319 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T02:29:45.319 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T02:29:45.319 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.62:3300,172.21.15.62:3301" 2023-10-06T02:29:45.319 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: trying mount with new device syntax: 1@d6fcf512-874b-4259-b90a-30ae070f00a9.cephfs=/ 2023-10-06T02:29:45.319 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.31:3300/172.21.15.62:3300/172.21.15.62:3301" will pass to kernel 2023-10-06T02:29:45.323 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T02:29:45.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:29:45.324 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-06T02:29:45.386 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-10-06T02:29:45.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:45.586+0000 7f05a619d700 1 Processor -- start 2023-10-06T02:29:45.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:45.587+0000 7f05a619d700 1 -- start start 2023-10-06T02:29:45.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:45.588+0000 7f05a619d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f05a015b1a0 0x7f05a00c6b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:45.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:45.588+0000 7f05a619d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f05a0157750 0x7f05a00ae080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:45.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:45.588+0000 7f05a619d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05a00c8680 con 0x7f05a0157380 2023-10-06T02:29:45.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:45.588+0000 7f05a619d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05a006a7f0 con 0x7f05a0157750 2023-10-06T02:29:45.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:45.588+0000 7f05a619d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05a015b6a0 con 0x7f05a015b1a0 2023-10-06T02:29:45.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:45.588+0000 7f05a599c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f05a0157750 0x7f05a00ae080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:45.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:45.588+0000 7f05a599c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f05a0157750 0x7f05a00ae080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55504/0 (socket says 172.21.15.31:55504) 2023-10-06T02:29:45.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:45.588+0000 7f05a599c700 1 -- 172.21.15.31:0/2266153069 learned_addr learned my addr 172.21.15.31:0/2266153069 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:45.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:45.588+0000 7f05a499a700 1 --2- 172.21.15.31:0/2266153069 >> v2:172.21.15.62:3300/0 conn(0x7f05a015b1a0 0x7f05a00c6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:45.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:45.589+0000 7f05a599c700 1 -- 172.21.15.31:0/2266153069 >> v2:172.21.15.62:3300/0 conn(0x7f05a015b1a0 msgr2=0x7f05a00c6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:45.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:45.589+0000 7f05a599c700 1 --2- 172.21.15.31:0/2266153069 >> v2:172.21.15.62:3300/0 conn(0x7f05a015b1a0 0x7f05a00c6b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:45.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:45.589+0000 7f05a599c700 1 -- 172.21.15.31:0/2266153069 >> v1:172.21.15.62:6789/0 conn(0x7f05a0157380 legacy=0x7f05a01585f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:45.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:45.589+0000 7f05a599c700 1 -- 172.21.15.31:0/2266153069 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05a015b980 con 0x7f05a0157750 2023-10-06T02:29:45.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:45.589+0000 7f05a499a700 1 --2- 172.21.15.31:0/2266153069 >> v2:172.21.15.62:3300/0 conn(0x7f05a015b1a0 0x7f05a00c6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:29:45.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:45.589+0000 7f05a599c700 1 --2- 172.21.15.31:0/2266153069 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05a0157750 0x7f05a00ae080 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f05a00c6720 tx=0x7f0598009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fbfab67821f8f59 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:45.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:45.589+0000 7f0597fff700 1 -- 172.21.15.31:0/2266153069 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f059800e070 con 0x7f05a0157750 2023-10-06T02:29:45.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:45.589+0000 7f0597fff700 1 -- 172.21.15.31:0/2266153069 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f059800ba50 con 0x7f05a0157750 2023-10-06T02:29:45.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:45.589+0000 7f0597fff700 1 -- 172.21.15.31:0/2266153069 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05980092e0 con 0x7f05a0157750 2023-10-06T02:29:45.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:45.590+0000 7f05a619d700 1 -- 172.21.15.31:0/2266153069 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05a0157750 msgr2=0x7f05a00ae080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:45.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:45.590+0000 7f05a619d700 1 --2- 172.21.15.31:0/2266153069 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05a0157750 0x7f05a00ae080 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f05a00c6720 tx=0x7f0598009840 comp rx=0 tx=0).stop 2023-10-06T02:29:45.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:45.590+0000 7f05a619d700 1 -- 172.21.15.31:0/2266153069 shutdown_connections 2023-10-06T02:29:45.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:45.590+0000 7f05a619d700 1 --2- 172.21.15.31:0/2266153069 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05a0157750 0x7f05a00ae080 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:45.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:45.590+0000 7f05a619d700 1 --2- 172.21.15.31:0/2266153069 >> v2:172.21.15.62:3300/0 conn(0x7f05a015b1a0 0x7f05a00c6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:45.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:45.590+0000 7f05a619d700 1 -- 172.21.15.31:0/2266153069 >> 172.21.15.31:0/2266153069 conn(0x7f05a006c6a0 msgr2=0x7f05a006caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:45.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:45.590+0000 7f05a619d700 1 -- 172.21.15.31:0/2266153069 shutdown_connections 2023-10-06T02:29:45.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:45.590+0000 7f05a619d700 1 -- 172.21.15.31:0/2266153069 wait complete. 2023-10-06T02:29:45.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:45.591+0000 7f05a619d700 1 Processor -- start 2023-10-06T02:29:45.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:45.591+0000 7f05a619d700 1 -- start start 2023-10-06T02:29:45.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:45.592+0000 7f05a619d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05a0157380 0x7f05a015ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:45.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:45.592+0000 7f05a619d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05a0157750 0x7f05a01684e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:45.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:45.592+0000 7f05a619d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f05a015b1a0 0x7f05a014eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:45.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:45.592+0000 7f05a619d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a0151160 con 0x7f05a0157750 2023-10-06T02:29:45.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:45.592+0000 7f05a619d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a0151440 con 0x7f05a0157380 2023-10-06T02:29:45.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:45.592+0000 7f05a619d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05a0159b50 con 0x7f05a015b1a0 2023-10-06T02:29:45.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:45.593+0000 7f05a499a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05a0157750 0x7f05a01684e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:45.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:45.593+0000 7f05a499a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05a0157750 0x7f05a01684e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55510/0 (socket says 172.21.15.31:55510) 2023-10-06T02:29:45.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:45.593+0000 7f05a499a700 1 -- 172.21.15.31:0/2864308799 learned_addr learned my addr 172.21.15.31:0/2864308799 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:45.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:45.593+0000 7f05a519b700 1 --2- 172.21.15.31:0/2864308799 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05a0157380 0x7f05a015ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:45.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:45.593+0000 7f05a599c700 1 --2- 172.21.15.31:0/2864308799 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f05a015b1a0 0x7f05a014eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:45.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:45.593+0000 7f05a499a700 1 -- 172.21.15.31:0/2864308799 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f05a015b1a0 msgr2=0x7f05a014eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:45.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:45.593+0000 7f05a499a700 1 --2- 172.21.15.31:0/2864308799 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f05a015b1a0 0x7f05a014eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:45.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:45.593+0000 7f05a499a700 1 -- 172.21.15.31:0/2864308799 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05a0157380 msgr2=0x7f05a015ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:45.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:45.593+0000 7f05a499a700 1 --2- 172.21.15.31:0/2864308799 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05a0157380 0x7f05a015ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:45.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:45.593+0000 7f05a499a700 1 -- 172.21.15.31:0/2864308799 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0598009ae0 con 0x7f05a0157750 2023-10-06T02:29:45.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:45.593+0000 7f05a499a700 1 --2- 172.21.15.31:0/2864308799 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05a0157750 0x7f05a01684e0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f059000e900 tx=0x7f0590008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:45.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:45.593+0000 7f05957fa700 1 -- 172.21.15.31:0/2864308799 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0590013410 con 0x7f05a0157750 2023-10-06T02:29:45.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:45.594+0000 7f05957fa700 1 -- 172.21.15.31:0/2864308799 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0590013570 con 0x7f05a0157750 2023-10-06T02:29:45.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:45.594+0000 7f05957fa700 1 -- 172.21.15.31:0/2864308799 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0590013910 con 0x7f05a0157750 2023-10-06T02:29:45.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:45.594+0000 7f05a619d700 1 -- 172.21.15.31:0/2864308799 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05a0159e30 con 0x7f05a0157750 2023-10-06T02:29:45.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:45.594+0000 7f05a619d700 1 -- 172.21.15.31:0/2864308799 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05a0168a20 con 0x7f05a0157750 2023-10-06T02:29:45.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:45.595+0000 7f05a619d700 1 -- 172.21.15.31:0/2864308799 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05a004c3c0 con 0x7f05a0157750 2023-10-06T02:29:45.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:45.596+0000 7f05957fa700 1 -- 172.21.15.31:0/2864308799 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f0590013cc0 con 0x7f05a0157750 2023-10-06T02:29:45.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:45.596+0000 7f05957fa700 1 --2- 172.21.15.31:0/2864308799 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0584067320 0x7f05840697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:45.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:45.597+0000 7f05957fa700 1 -- 172.21.15.31:0/2864308799 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0590084c00 con 0x7f05a0157750 2023-10-06T02:29:45.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:45.597+0000 7f05a519b700 1 --2- 172.21.15.31:0/2864308799 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0584067320 0x7f05840697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:45.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:45.597+0000 7f05a519b700 1 --2- 172.21.15.31:0/2864308799 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0584067320 0x7f05840697d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f05a014fe60 tx=0x7f059c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:45.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:45.601+0000 7f05957fa700 1 -- 172.21.15.31:0/2864308799 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0590051620 con 0x7f05a0157750 2023-10-06T02:29:45.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:45.848+0000 7f05a619d700 1 -- 172.21.15.31:0/2864308799 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f05a0151580 con 0x7f05a0157750 2023-10-06T02:29:46.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:46.558+0000 7f05957fa700 1 -- 172.21.15.31:0/2864308799 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v81) v1 ==== 136+0+0 (secure 0 0 0) 0x7f0590054c40 con 0x7f05a0157750 2023-10-06T02:29:46.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:46.561+0000 7f05a619d700 1 -- 172.21.15.31:0/2864308799 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0584067320 msgr2=0x7f05840697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:46.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:46.561+0000 7f05a619d700 1 --2- 172.21.15.31:0/2864308799 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0584067320 0x7f05840697d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f05a014fe60 tx=0x7f059c009040 comp rx=0 tx=0).stop 2023-10-06T02:29:46.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:46.561+0000 7f05a619d700 1 -- 172.21.15.31:0/2864308799 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05a0157750 msgr2=0x7f05a01684e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:46.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:46.561+0000 7f05a619d700 1 --2- 172.21.15.31:0/2864308799 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05a0157750 0x7f05a01684e0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f059000e900 tx=0x7f0590008040 comp rx=0 tx=0).stop 2023-10-06T02:29:46.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:46.561+0000 7f05a619d700 1 -- 172.21.15.31:0/2864308799 shutdown_connections 2023-10-06T02:29:46.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:46.561+0000 7f05a619d700 1 --2- 172.21.15.31:0/2864308799 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0584067320 0x7f05840697d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:46.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:46.561+0000 7f05a619d700 1 --2- 172.21.15.31:0/2864308799 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f05a015b1a0 0x7f05a014eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:46.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:46.561+0000 7f05a619d700 1 --2- 172.21.15.31:0/2864308799 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05a0157380 0x7f05a015ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:46.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:46.561+0000 7f05a619d700 1 --2- 172.21.15.31:0/2864308799 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05a0157750 0x7f05a01684e0 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:46.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:46.562+0000 7f05a619d700 1 -- 172.21.15.31:0/2864308799 >> 172.21.15.31:0/2864308799 conn(0x7f05a006c6a0 msgr2=0x7f05a0158090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:46.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:46.562+0000 7f05a619d700 1 -- 172.21.15.31:0/2864308799 shutdown_connections 2023-10-06T02:29:46.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:46.562+0000 7f05a619d700 1 -- 172.21.15.31:0/2864308799 wait complete. 2023-10-06T02:29:46.584 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-06T02:29:46.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:46.775+0000 7fcba4d9d700 1 Processor -- start 2023-10-06T02:29:46.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:46.776+0000 7fcba4d9d700 1 -- start start 2023-10-06T02:29:46.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:46.777+0000 7fcba4d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcba013fb20 0x7fcba015a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:46.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:46.777+0000 7fcba4d9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fcba013fef0 0x7fcba00c7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:46.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:46.777+0000 7fcba4d9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcba015ae70 con 0x7fcba006aaa0 2023-10-06T02:29:46.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:46.777+0000 7fcba4d9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcba006af60 con 0x7fcba013fef0 2023-10-06T02:29:46.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:46.777+0000 7fcba4d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcba006b240 con 0x7fcba013fb20 2023-10-06T02:29:46.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:46.777+0000 7fcb9effd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fcba013fef0 0x7fcba00c7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:46.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:46.777+0000 7fcb9ffff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fcba006aaa0 0x7fcba00c6770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:54114/0 (socket says 172.21.15.31:54114) 2023-10-06T02:29:46.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:46.777+0000 7fcb9ffff700 1 -- 172.21.15.31:0/1546522382 learned_addr learned my addr 172.21.15.31:0/1546522382 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:29:46.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:46.777+0000 7fcb9f7fe700 1 --2- 172.21.15.31:0/1546522382 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcba013fb20 0x7fcba015a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:46.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:46.778+0000 7fcb9effd700 1 -- 172.21.15.31:0/1546522382 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcba013fb20 msgr2=0x7fcba015a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:46.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:46.778+0000 7fcb9effd700 1 --2- 172.21.15.31:0/1546522382 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcba013fb20 0x7fcba015a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:46.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:46.778+0000 7fcb9effd700 1 -- 172.21.15.31:0/1546522382 >> v1:172.21.15.31:6789/0 conn(0x7fcba006aaa0 legacy=0x7fcba00c6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:46.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:46.778+0000 7fcb9effd700 1 -- 172.21.15.31:0/1546522382 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcba006b520 con 0x7fcba013fef0 2023-10-06T02:29:46.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:46.778+0000 7fcb9e7fc700 1 -- 172.21.15.31:0/1546522382 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 193308254 0 0) 0x7fcb98001490 con 0x7fcba006aaa0 2023-10-06T02:29:46.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:46.778+0000 7fcb9effd700 1 --2- 172.21.15.31:0/1546522382 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcba013fef0 0x7fcba00c7990 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fcb940098d0 tx=0x7fcb94009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d910fea87145bbc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:46.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:46.778+0000 7fcb9e7fc700 1 -- 172.21.15.31:0/1546522382 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb94005c10 con 0x7fcba013fef0 2023-10-06T02:29:46.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:46.778+0000 7fcb9e7fc700 1 -- 172.21.15.31:0/1546522382 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb94005d70 con 0x7fcba013fef0 2023-10-06T02:29:46.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:46.778+0000 7fcb9e7fc700 1 -- 172.21.15.31:0/1546522382 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb94006280 con 0x7fcba013fef0 2023-10-06T02:29:46.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:46.779+0000 7fcba4d9d700 1 -- 172.21.15.31:0/1546522382 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcba013fef0 msgr2=0x7fcba00c7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:46.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:46.779+0000 7fcba4d9d700 1 --2- 172.21.15.31:0/1546522382 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcba013fef0 0x7fcba00c7990 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fcb940098d0 tx=0x7fcb94009120 comp rx=0 tx=0).stop 2023-10-06T02:29:46.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:46.779+0000 7fcba4d9d700 1 -- 172.21.15.31:0/1546522382 shutdown_connections 2023-10-06T02:29:46.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:46.779+0000 7fcba4d9d700 1 --2- 172.21.15.31:0/1546522382 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcba013fb20 0x7fcba015a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:46.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:46.779+0000 7fcba4d9d700 1 --2- 172.21.15.31:0/1546522382 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcba013fef0 0x7fcba00c7990 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:46.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:46.779+0000 7fcba4d9d700 1 -- 172.21.15.31:0/1546522382 >> 172.21.15.31:0/1546522382 conn(0x7fcba006d170 msgr2=0x7fcba00cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:46.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:46.779+0000 7fcba4d9d700 1 -- 172.21.15.31:0/1546522382 shutdown_connections 2023-10-06T02:29:46.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:46.779+0000 7fcba4d9d700 1 -- 172.21.15.31:0/1546522382 wait complete. 2023-10-06T02:29:46.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:46.780+0000 7fcba4d9d700 1 Processor -- start 2023-10-06T02:29:46.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:46.781+0000 7fcba4d9d700 1 -- start start 2023-10-06T02:29:46.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:46.781+0000 7fcba4d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcba006aaa0 0x7fcba0155b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:46.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:46.782+0000 7fcba4d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcba013fb20 0x7fcba0158fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:46.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:46.782+0000 7fcba4d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcba013fef0 0x7fcba00b0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:46.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:46.782+0000 7fcba4d9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba0156070 con 0x7fcba013fb20 2023-10-06T02:29:46.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:46.782+0000 7fcba4d9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba00c12b0 con 0x7fcba013fef0 2023-10-06T02:29:46.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:46.782+0000 7fcba4d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcba00c1590 con 0x7fcba006aaa0 2023-10-06T02:29:46.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:46.782+0000 7fcb9effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcba013fb20 0x7fcba0158fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:46.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:46.782+0000 7fcb9effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcba013fb20 0x7fcba0158fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55522/0 (socket says 172.21.15.31:55522) 2023-10-06T02:29:46.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:46.782+0000 7fcb9effd700 1 -- 172.21.15.31:0/4246294444 learned_addr learned my addr 172.21.15.31:0/4246294444 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:46.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:46.782+0000 7fcb9f7fe700 1 --2- 172.21.15.31:0/4246294444 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcba006aaa0 0x7fcba0155b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:46.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:46.782+0000 7fcb9ffff700 1 --2- 172.21.15.31:0/4246294444 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcba013fef0 0x7fcba00b0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:46.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:46.782+0000 7fcb9effd700 1 -- 172.21.15.31:0/4246294444 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcba006aaa0 msgr2=0x7fcba0155b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:46.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:46.782+0000 7fcb9effd700 1 --2- 172.21.15.31:0/4246294444 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcba006aaa0 0x7fcba0155b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:46.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:46.782+0000 7fcb9effd700 1 -- 172.21.15.31:0/4246294444 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcba013fef0 msgr2=0x7fcba00b0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:46.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:46.782+0000 7fcb9effd700 1 --2- 172.21.15.31:0/4246294444 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcba013fef0 0x7fcba00b0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:46.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:46.782+0000 7fcb9effd700 1 -- 172.21.15.31:0/4246294444 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb940093c0 con 0x7fcba013fb20 2023-10-06T02:29:46.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:46.783+0000 7fcb9effd700 1 --2- 172.21.15.31:0/4246294444 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcba013fb20 0x7fcba0158fd0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fcb94009f70 tx=0x7fcb94016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:46.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:46.783+0000 7fcb877fe700 1 -- 172.21.15.31:0/4246294444 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb9401d880 con 0x7fcba013fb20 2023-10-06T02:29:46.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:46.783+0000 7fcb877fe700 1 -- 172.21.15.31:0/4246294444 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb9401d9e0 con 0x7fcba013fb20 2023-10-06T02:29:46.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:46.783+0000 7fcb877fe700 1 -- 172.21.15.31:0/4246294444 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb9401dda0 con 0x7fcba013fb20 2023-10-06T02:29:46.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:46.783+0000 7fcba4d9d700 1 -- 172.21.15.31:0/4246294444 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcba00c1870 con 0x7fcba013fb20 2023-10-06T02:29:46.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:46.783+0000 7fcba4d9d700 1 -- 172.21.15.31:0/4246294444 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcba00c1e30 con 0x7fcba013fb20 2023-10-06T02:29:46.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:46.784+0000 7fcba4d9d700 1 -- 172.21.15.31:0/4246294444 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcba00c2020 con 0x7fcba013fb20 2023-10-06T02:29:46.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:46.785+0000 7fcb877fe700 1 -- 172.21.15.31:0/4246294444 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fcb9401b690 con 0x7fcba013fb20 2023-10-06T02:29:46.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:46.786+0000 7fcb877fe700 1 --2- 172.21.15.31:0/4246294444 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fcb8c067320 0x7fcb8c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:46.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:46.786+0000 7fcb877fe700 1 -- 172.21.15.31:0/4246294444 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fcb94086b50 con 0x7fcba013fb20 2023-10-06T02:29:46.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:46.786+0000 7fcb9f7fe700 1 --2- 172.21.15.31:0/4246294444 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fcb8c067320 0x7fcb8c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:46.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:46.787+0000 7fcb9f7fe700 1 --2- 172.21.15.31:0/4246294444 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fcb8c067320 0x7fcb8c0697d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fcb90009d40 tx=0x7fcb90008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:46.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:46.790+0000 7fcb877fe700 1 -- 172.21.15.31:0/4246294444 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcb94053680 con 0x7fcba013fb20 2023-10-06T02:29:47.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.022+0000 7fcba4d9d700 1 -- 172.21.15.31:0/4246294444 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fcba004c3c0 con 0x7fcb8c067320 2023-10-06T02:29:47.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.025+0000 7fcb877fe700 1 -- 172.21.15.31:0/4246294444 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fcba004c3c0 con 0x7fcb8c067320 2023-10-06T02:29:47.025 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2023-10-06T02:29:47.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:29:47.026 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephfs" 2023-10-06T02:29:47.026 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:29:47.026 INFO:teuthology.orchestra.run.smithi031.stdout:] 2023-10-06T02:29:47.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.026+0000 7fcba4d9d700 1 -- 172.21.15.31:0/4246294444 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fcb8c067320 msgr2=0x7fcb8c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:47.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.026+0000 7fcba4d9d700 1 --2- 172.21.15.31:0/4246294444 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fcb8c067320 0x7fcb8c0697d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fcb90009d40 tx=0x7fcb90008040 comp rx=0 tx=0).stop 2023-10-06T02:29:47.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.026+0000 7fcba4d9d700 1 -- 172.21.15.31:0/4246294444 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcba013fb20 msgr2=0x7fcba0158fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:47.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.026+0000 7fcba4d9d700 1 --2- 172.21.15.31:0/4246294444 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcba013fb20 0x7fcba0158fd0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fcb94009f70 tx=0x7fcb94016000 comp rx=0 tx=0).stop 2023-10-06T02:29:47.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.026+0000 7fcba4d9d700 1 -- 172.21.15.31:0/4246294444 shutdown_connections 2023-10-06T02:29:47.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.026+0000 7fcba4d9d700 1 --2- 172.21.15.31:0/4246294444 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcba013fef0 0x7fcba00b0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:47.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.026+0000 7fcba4d9d700 1 --2- 172.21.15.31:0/4246294444 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fcb8c067320 0x7fcb8c0697d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:47.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.026+0000 7fcba4d9d700 1 --2- 172.21.15.31:0/4246294444 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcba006aaa0 0x7fcba0155b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:47.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.027+0000 7fcba4d9d700 1 --2- 172.21.15.31:0/4246294444 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcba013fb20 0x7fcba0158fd0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:47.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.027+0000 7fcba4d9d700 1 -- 172.21.15.31:0/4246294444 >> 172.21.15.31:0/4246294444 conn(0x7fcba006d170 msgr2=0x7fcba00c9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:47.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.027+0000 7fcba4d9d700 1 -- 172.21.15.31:0/4246294444 shutdown_connections 2023-10-06T02:29:47.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.027+0000 7fcba4d9d700 1 -- 172.21.15.31:0/4246294444 wait complete. 2023-10-06T02:29:47.046 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T02:29:47.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.235+0000 7f9079f9d700 1 Processor -- start 2023-10-06T02:29:47.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.236+0000 7f9079f9d700 1 -- start start 2023-10-06T02:29:47.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.237+0000 7f9079f9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9074159550 0x7f90740ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:47.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.237+0000 7f9079f9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f907415dfd0 0x7f90740aedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:47.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.237+0000 7f9079f9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90740acc00 con 0x7f9074159920 2023-10-06T02:29:47.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.237+0000 7f9079f9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90740abff0 con 0x7f907415dfd0 2023-10-06T02:29:47.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.237+0000 7f9079f9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90740ac2d0 con 0x7f9074159550 2023-10-06T02:29:47.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.237+0000 7f9073fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f907415dfd0 0x7f90740aedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:47.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.237+0000 7f9073fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f907415dfd0 0x7f90740aedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55534/0 (socket says 172.21.15.31:55534) 2023-10-06T02:29:47.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.237+0000 7f9073fff700 1 -- 172.21.15.31:0/1333484133 learned_addr learned my addr 172.21.15.31:0/1333484133 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:47.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.237+0000 7f9078f9b700 1 --2- 172.21.15.31:0/1333484133 >> v2:172.21.15.62:3300/0 conn(0x7f9074159550 0x7f90740ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:47.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.238+0000 7f9073fff700 1 -- 172.21.15.31:0/1333484133 >> v2:172.21.15.62:3300/0 conn(0x7f9074159550 msgr2=0x7f90740ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:47.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.238+0000 7f9073fff700 1 --2- 172.21.15.31:0/1333484133 >> v2:172.21.15.62:3300/0 conn(0x7f9074159550 0x7f90740ceac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:47.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.238+0000 7f9073fff700 1 -- 172.21.15.31:0/1333484133 >> v1:172.21.15.62:6789/0 conn(0x7f9074159920 legacy=0x7f90740ad090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:47.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.238+0000 7f9073fff700 1 -- 172.21.15.31:0/1333484133 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90740ac5b0 con 0x7f907415dfd0 2023-10-06T02:29:47.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.238+0000 7f9073fff700 1 --2- 172.21.15.31:0/1333484133 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f907415dfd0 0x7f90740aedb0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f9074064270 tx=0x7f9060009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16bfea3548536cf4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:47.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.238+0000 7f90737fe700 1 -- 172.21.15.31:0/1333484133 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9060009e70 con 0x7f907415dfd0 2023-10-06T02:29:47.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.238+0000 7f90737fe700 1 -- 172.21.15.31:0/1333484133 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f906000b860 con 0x7f907415dfd0 2023-10-06T02:29:47.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.238+0000 7f90737fe700 1 -- 172.21.15.31:0/1333484133 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f906000bc00 con 0x7f907415dfd0 2023-10-06T02:29:47.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.239+0000 7f9079f9d700 1 -- 172.21.15.31:0/1333484133 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f907415dfd0 msgr2=0x7f90740aedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:47.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.239+0000 7f9079f9d700 1 --2- 172.21.15.31:0/1333484133 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f907415dfd0 0x7f90740aedb0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f9074064270 tx=0x7f9060009250 comp rx=0 tx=0).stop 2023-10-06T02:29:47.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.239+0000 7f9079f9d700 1 -- 172.21.15.31:0/1333484133 shutdown_connections 2023-10-06T02:29:47.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.239+0000 7f9079f9d700 1 --2- 172.21.15.31:0/1333484133 >> v2:172.21.15.62:3300/0 conn(0x7f9074159550 0x7f90740ceac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:47.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.239+0000 7f9079f9d700 1 --2- 172.21.15.31:0/1333484133 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f907415dfd0 0x7f90740aedb0 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:47.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.239+0000 7f9079f9d700 1 -- 172.21.15.31:0/1333484133 >> 172.21.15.31:0/1333484133 conn(0x7f907406ce80 msgr2=0x7f90740ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:47.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.239+0000 7f9079f9d700 1 -- 172.21.15.31:0/1333484133 shutdown_connections 2023-10-06T02:29:47.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.239+0000 7f9079f9d700 1 -- 172.21.15.31:0/1333484133 wait complete. 2023-10-06T02:29:47.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.240+0000 7f9079f9d700 1 Processor -- start 2023-10-06T02:29:47.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.241+0000 7f9079f9d700 1 -- start start 2023-10-06T02:29:47.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.241+0000 7f9079f9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9074159550 0x7f907416aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:47.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.241+0000 7f9079f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9074159920 0x7f90740c9900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:47.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.242+0000 7f9079f9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f907415dfd0 0x7f90741530f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:47.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.242+0000 7f9079f9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f907416af40 con 0x7f9074159550 2023-10-06T02:29:47.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.242+0000 7f9079f9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90740cbca0 con 0x7f907415dfd0 2023-10-06T02:29:47.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.242+0000 7f9079f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90740cadb0 con 0x7f9074159920 2023-10-06T02:29:47.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.242+0000 7f9078f9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9074159550 0x7f907416aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:47.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.242+0000 7f9078f9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9074159550 0x7f907416aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55538/0 (socket says 172.21.15.31:55538) 2023-10-06T02:29:47.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.242+0000 7f9078f9b700 1 -- 172.21.15.31:0/433414221 learned_addr learned my addr 172.21.15.31:0/433414221 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:47.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.242+0000 7f9073fff700 1 --2- 172.21.15.31:0/433414221 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9074159920 0x7f90740c9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:47.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.242+0000 7f907979c700 1 --2- 172.21.15.31:0/433414221 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f907415dfd0 0x7f90741530f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:47.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.242+0000 7f9078f9b700 1 -- 172.21.15.31:0/433414221 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9074159920 msgr2=0x7f90740c9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:47.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.242+0000 7f9078f9b700 1 --2- 172.21.15.31:0/433414221 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9074159920 0x7f90740c9900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:47.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.242+0000 7f9078f9b700 1 -- 172.21.15.31:0/433414221 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f907415dfd0 msgr2=0x7f90741530f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:47.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.242+0000 7f9078f9b700 1 --2- 172.21.15.31:0/433414221 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f907415dfd0 0x7f90741530f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:47.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.242+0000 7f9078f9b700 1 -- 172.21.15.31:0/433414221 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90600094f0 con 0x7f9074159550 2023-10-06T02:29:47.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.242+0000 7f907979c700 1 --2- 172.21.15.31:0/433414221 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f907415dfd0 0x7f90741530f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:29:47.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.243+0000 7f9078f9b700 1 --2- 172.21.15.31:0/433414221 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9074159550 0x7f907416aa00 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f906800ec80 tx=0x7f90680091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:47.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.243+0000 7f9070ff9700 1 -- 172.21.15.31:0/433414221 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f906800c9d0 con 0x7f9074159550 2023-10-06T02:29:47.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.243+0000 7f9070ff9700 1 -- 172.21.15.31:0/433414221 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f906800cb30 con 0x7f9074159550 2023-10-06T02:29:47.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.243+0000 7f9070ff9700 1 -- 172.21.15.31:0/433414221 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9068013470 con 0x7f9074159550 2023-10-06T02:29:47.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.243+0000 7f9079f9d700 1 -- 172.21.15.31:0/433414221 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90740cb090 con 0x7f9074159550 2023-10-06T02:29:47.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.243+0000 7f9079f9d700 1 -- 172.21.15.31:0/433414221 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90740cb650 con 0x7f9074159550 2023-10-06T02:29:47.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.244+0000 7f9079f9d700 1 -- 172.21.15.31:0/433414221 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9074070870 con 0x7f9074159550 2023-10-06T02:29:47.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.245+0000 7f9070ff9700 1 -- 172.21.15.31:0/433414221 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f9068013820 con 0x7f9074159550 2023-10-06T02:29:47.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.246+0000 7f9070ff9700 1 --2- 172.21.15.31:0/433414221 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9058067320 0x7f90580697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:47.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.246+0000 7f9070ff9700 1 -- 172.21.15.31:0/433414221 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f90680851d0 con 0x7f9074159550 2023-10-06T02:29:47.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.247+0000 7f9073fff700 1 --2- 172.21.15.31:0/433414221 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9058067320 0x7f90580697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:47.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.247+0000 7f9073fff700 1 --2- 172.21.15.31:0/433414221 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9058067320 0x7f90580697d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9074064270 tx=0x7f9060017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:47.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.250+0000 7f9070ff9700 1 -- 172.21.15.31:0/433414221 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9068051cc0 con 0x7f9074159550 2023-10-06T02:29:47.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.457+0000 7f9079f9d700 1 -- 172.21.15.31:0/433414221 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f907409a420 con 0x7f9074159550 2023-10-06T02:29:47.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.458+0000 7f9070ff9700 1 -- 172.21.15.31:0/433414221 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7f90680552e0 con 0x7f9074159550 2023-10-06T02:29:47.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.460+0000 7f9079f9d700 1 -- 172.21.15.31:0/433414221 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9058067320 msgr2=0x7f90580697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:47.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.460+0000 7f9079f9d700 1 --2- 172.21.15.31:0/433414221 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9058067320 0x7f90580697d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9074064270 tx=0x7f9060017000 comp rx=0 tx=0).stop 2023-10-06T02:29:47.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.460+0000 7f9079f9d700 1 -- 172.21.15.31:0/433414221 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9074159550 msgr2=0x7f907416aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:47.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.460+0000 7f9079f9d700 1 --2- 172.21.15.31:0/433414221 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9074159550 0x7f907416aa00 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f906800ec80 tx=0x7f90680091a0 comp rx=0 tx=0).stop 2023-10-06T02:29:47.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.460+0000 7f9079f9d700 1 -- 172.21.15.31:0/433414221 shutdown_connections 2023-10-06T02:29:47.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.460+0000 7f9079f9d700 1 --2- 172.21.15.31:0/433414221 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f907415dfd0 0x7f90741530f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:47.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.460+0000 7f9079f9d700 1 --2- 172.21.15.31:0/433414221 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9074159550 0x7f907416aa00 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:47.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.460+0000 7f9079f9d700 1 --2- 172.21.15.31:0/433414221 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9058067320 0x7f90580697d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:47.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.460+0000 7f9079f9d700 1 --2- 172.21.15.31:0/433414221 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9074159920 0x7f90740c9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:47.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.460+0000 7f9079f9d700 1 -- 172.21.15.31:0/433414221 >> 172.21.15.31:0/433414221 conn(0x7f907406ce80 msgr2=0x7f90740b3310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:47.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.460+0000 7f9079f9d700 1 -- 172.21.15.31:0/433414221 shutdown_connections 2023-10-06T02:29:47.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:47.460+0000 7f9079f9d700 1 -- 172.21.15.31:0/433414221 wait complete. 2023-10-06T02:29:47.479 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:29:47.479 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvolume_0000000000554351/de2ab1bd-640c-4194-8b09-3a11819808e2) 2023-10-06T02:29:47.550 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:29:47.550 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvolume_0000000000554351/de2ab1bd-640c-4194-8b09-3a11819808e2/.snap/fake) 2023-10-06T02:29:47.623 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:29:47.623 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2023-10-06T02:29:47.660 INFO:teuthology.orchestra.run.smithi031.stdout:cephfs_data 2023-10-06T02:29:47.660 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:29:47.661 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/_nogroup/subvolume_0000000000554351/de2ab1bd-640c-4194-8b09-3a11819808e2) 2023-10-06T02:29:47.766 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T02:29:47.767 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/_nogroup/subvolume_0000000000554351/.meta 2023-10-06T02:29:47.832 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:29:47.832 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/group_0000000000489015/subvolume_0000000000554352/b65f3321-3c24-47fd-ada1-51bab8363193) 2023-10-06T02:29:47.938 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:29:47.938 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/group_0000000000489015/subvolume_0000000000554352/b65f3321-3c24-47fd-ada1-51bab8363193/.snap/fake) 2023-10-06T02:29:48.010 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:29:48.010 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2023-10-06T02:29:48.047 INFO:teuthology.orchestra.run.smithi031.stdout:cephfs_data 2023-10-06T02:29:48.048 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:29:48.048 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/group_0000000000489015/subvolume_0000000000554352/b65f3321-3c24-47fd-ada1-51bab8363193) 2023-10-06T02:29:48.155 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T02:29:48.155 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/group_0000000000489015/subvolume_0000000000554352/.meta 2023-10-06T02:29:48.217 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:29:48.218 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvolume_0000000000554353/8137d096-da5f-4840-9c08-3fd3bbefc549) 2023-10-06T02:29:48.323 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:29:48.323 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2023-10-06T02:29:48.360 INFO:teuthology.orchestra.run.smithi031.stdout:cephfs_data 2023-10-06T02:29:48.360 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:29:48.361 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/_nogroup/subvolume_0000000000554353/8137d096-da5f-4840-9c08-3fd3bbefc549) 2023-10-06T02:29:48.468 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T02:29:48.468 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/_nogroup/subvolume_0000000000554353/.meta 2023-10-06T02:29:48.532 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000554351 2023-10-06T02:29:48.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:48.762+0000 7f5ca7fff700 1 Processor -- start 2023-10-06T02:29:48.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:48.762+0000 7f5ca7fff700 1 -- start start 2023-10-06T02:29:48.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:48.762+0000 7f5ca7fff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f5ca8155c60 0x7f5ca80c7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:48.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:48.762+0000 7f5ca7fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5ca806de50 0x7f5ca806e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:48.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:48.762+0000 7f5ca7fff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ca80c7e70 con 0x7f5ca8155890 2023-10-06T02:29:48.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:48.762+0000 7f5ca7fff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ca806e7f0 con 0x7f5ca806de50 2023-10-06T02:29:48.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:48.762+0000 7f5ca7fff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ca806b790 con 0x7f5ca8155c60 2023-10-06T02:29:48.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:48.763+0000 7f5ca77fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5ca806de50 0x7f5ca806e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:48.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:48.763+0000 7f5ca6ffd700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f5ca8155890 0x7f5ca806a3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:54490/0 (socket says 172.21.15.31:54490) 2023-10-06T02:29:48.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:48.763+0000 7f5ca6ffd700 1 -- 172.21.15.31:0/2952720786 learned_addr learned my addr 172.21.15.31:0/2952720786 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:29:48.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:48.763+0000 7f5ca67fc700 1 --2- 172.21.15.31:0/2952720786 >> v2:172.21.15.62:3300/0 conn(0x7f5ca8155c60 0x7f5ca80c7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:48.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:48.763+0000 7f5ca5ffb700 1 -- 172.21.15.31:0/2952720786 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3623111285 0 0) 0x7f5ca80c7e70 con 0x7f5ca8155890 2023-10-06T02:29:48.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:48.763+0000 7f5ca77fe700 1 -- 172.21.15.31:0/2952720786 >> v2:172.21.15.62:3300/0 conn(0x7f5ca8155c60 msgr2=0x7f5ca80c7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:48.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:48.763+0000 7f5ca77fe700 1 --2- 172.21.15.31:0/2952720786 >> v2:172.21.15.62:3300/0 conn(0x7f5ca8155c60 0x7f5ca80c7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:48.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:48.763+0000 7f5ca77fe700 1 -- 172.21.15.31:0/2952720786 >> v1:172.21.15.31:6789/0 conn(0x7f5ca8155890 legacy=0x7f5ca806a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:48.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:48.763+0000 7f5ca77fe700 1 -- 172.21.15.31:0/2952720786 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ca806ab80 con 0x7f5ca806de50 2023-10-06T02:29:48.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:48.763+0000 7f5ca67fc700 1 --2- 172.21.15.31:0/2952720786 >> v2:172.21.15.62:3300/0 conn(0x7f5ca8155c60 0x7f5ca80c7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:29:48.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:48.763+0000 7f5ca77fe700 1 --2- 172.21.15.31:0/2952720786 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ca806de50 0x7f5ca806e2b0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f5ca80697f0 tx=0x7f5ca0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7460061246def877 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:48.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:48.764+0000 7f5ca5ffb700 1 -- 172.21.15.31:0/2952720786 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ca000e070 con 0x7f5ca806de50 2023-10-06T02:29:48.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:48.764+0000 7f5ca5ffb700 1 -- 172.21.15.31:0/2952720786 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ca000bc30 con 0x7f5ca806de50 2023-10-06T02:29:48.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:48.764+0000 7f5ca5ffb700 1 -- 172.21.15.31:0/2952720786 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ca0012430 con 0x7f5ca806de50 2023-10-06T02:29:48.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:48.764+0000 7f5ca7fff700 1 -- 172.21.15.31:0/2952720786 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ca806de50 msgr2=0x7f5ca806e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:48.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:48.764+0000 7f5ca7fff700 1 --2- 172.21.15.31:0/2952720786 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ca806de50 0x7f5ca806e2b0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f5ca80697f0 tx=0x7f5ca0009780 comp rx=0 tx=0).stop 2023-10-06T02:29:48.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:48.764+0000 7f5ca7fff700 1 -- 172.21.15.31:0/2952720786 shutdown_connections 2023-10-06T02:29:48.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:48.764+0000 7f5ca7fff700 1 --2- 172.21.15.31:0/2952720786 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ca806de50 0x7f5ca806e2b0 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:48.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:48.764+0000 7f5ca7fff700 1 --2- 172.21.15.31:0/2952720786 >> v2:172.21.15.62:3300/0 conn(0x7f5ca8155c60 0x7f5ca80c7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:48.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:48.764+0000 7f5ca7fff700 1 -- 172.21.15.31:0/2952720786 >> 172.21.15.31:0/2952720786 conn(0x7f5ca806d110 msgr2=0x7f5ca80cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:48.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:48.764+0000 7f5ca7fff700 1 -- 172.21.15.31:0/2952720786 shutdown_connections 2023-10-06T02:29:48.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:48.764+0000 7f5ca7fff700 1 -- 172.21.15.31:0/2952720786 wait complete. 2023-10-06T02:29:48.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:48.766+0000 7f5ca7fff700 1 Processor -- start 2023-10-06T02:29:48.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:48.766+0000 7f5ca7fff700 1 -- start start 2023-10-06T02:29:48.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:48.768+0000 7f5ca7fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5ca806de50 0x7f5ca8157500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:48.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:48.768+0000 7f5ca7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ca8155890 0x7f5ca8146fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:48.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:48.768+0000 7f5ca7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ca8155c60 0x7f5ca8148d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:48.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:48.768+0000 7f5ca7fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca81571a0 con 0x7f5ca8155c60 2023-10-06T02:29:48.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:48.768+0000 7f5ca7fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca80af6c0 con 0x7f5ca8155890 2023-10-06T02:29:48.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:48.768+0000 7f5ca7fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca8157a40 con 0x7f5ca806de50 2023-10-06T02:29:48.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:48.768+0000 7f5ca77fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ca8155c60 0x7f5ca8148d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:48.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:48.768+0000 7f5ca77fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ca8155c60 0x7f5ca8148d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33888/0 (socket says 172.21.15.31:33888) 2023-10-06T02:29:48.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:48.768+0000 7f5ca77fe700 1 -- 172.21.15.31:0/695505153 learned_addr learned my addr 172.21.15.31:0/695505153 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:48.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:48.768+0000 7f5ca6ffd700 1 --2- 172.21.15.31:0/695505153 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5ca806de50 0x7f5ca8157500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:48.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:48.768+0000 7f5ca67fc700 1 --2- 172.21.15.31:0/695505153 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ca8155890 0x7f5ca8146fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:48.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:48.768+0000 7f5ca77fe700 1 -- 172.21.15.31:0/695505153 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5ca806de50 msgr2=0x7f5ca8157500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:48.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:48.768+0000 7f5ca77fe700 1 --2- 172.21.15.31:0/695505153 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5ca806de50 0x7f5ca8157500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:48.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:48.768+0000 7f5ca77fe700 1 -- 172.21.15.31:0/695505153 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ca8155890 msgr2=0x7f5ca8146fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:48.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:48.768+0000 7f5ca77fe700 1 --2- 172.21.15.31:0/695505153 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ca8155890 0x7f5ca8146fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:48.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:48.768+0000 7f5ca77fe700 1 -- 172.21.15.31:0/695505153 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ca0009a20 con 0x7f5ca8155c60 2023-10-06T02:29:48.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:48.768+0000 7f5ca77fe700 1 --2- 172.21.15.31:0/695505153 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ca8155c60 0x7f5ca8148d40 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f5ca80cd1c0 tx=0x7f5ca0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:48.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:48.768+0000 7f5c8effd700 1 -- 172.21.15.31:0/695505153 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ca0027470 con 0x7f5ca8155c60 2023-10-06T02:29:48.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:48.768+0000 7f5c8effd700 1 -- 172.21.15.31:0/695505153 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ca00275d0 con 0x7f5ca8155c60 2023-10-06T02:29:48.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:48.768+0000 7f5ca7fff700 1 -- 172.21.15.31:0/695505153 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ca8156ec0 con 0x7f5ca8155c60 2023-10-06T02:29:48.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:48.768+0000 7f5c8effd700 1 -- 172.21.15.31:0/695505153 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ca0027970 con 0x7f5ca8155c60 2023-10-06T02:29:48.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:48.769+0000 7f5ca7fff700 1 -- 172.21.15.31:0/695505153 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ca80af350 con 0x7f5ca8155c60 2023-10-06T02:29:48.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:48.769+0000 7f5ca7fff700 1 -- 172.21.15.31:0/695505153 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ca80aeed0 con 0x7f5ca8155c60 2023-10-06T02:29:48.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:48.770+0000 7f5c8effd700 1 -- 172.21.15.31:0/695505153 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f5ca0024070 con 0x7f5ca8155c60 2023-10-06T02:29:48.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:48.771+0000 7f5c8effd700 1 --2- 172.21.15.31:0/695505153 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5c94067320 0x7f5c940697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:48.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:48.771+0000 7f5c8effd700 1 -- 172.21.15.31:0/695505153 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f5ca0087480 con 0x7f5ca8155c60 2023-10-06T02:29:48.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:48.773+0000 7f5ca6ffd700 1 --2- 172.21.15.31:0/695505153 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5c94067320 0x7f5c940697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:48.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:48.773+0000 7f5ca6ffd700 1 --2- 172.21.15.31:0/695505153 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5c94067320 0x7f5c940697d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5ca80cc9a0 tx=0x7f5c98007d30 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:48.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:48.776+0000 7f5c8effd700 1 -- 172.21.15.31:0/695505153 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5ca80aeed0 con 0x7f5ca8155c60 2023-10-06T02:29:49.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.013+0000 7f5ca7fff700 1 -- 172.21.15.31:0/695505153 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000554351", "target": ["mon-mgr", ""]}) v1 -- 0x7f5ca80aeed0 con 0x7f5c94067320 2023-10-06T02:29:49.022 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:29:49.022+0000 7f656537d700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:29:49.023 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:29:49.022+0000 7f656537d700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:29:49.023 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:29:49.022+0000 7f656537d700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:29:49.023 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:29:49.022+0000 7f656537d700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:29:49.023 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:29:49.022+0000 7f656537d700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:29:49.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.072+0000 7f5c8effd700 1 -- 172.21.15.31:0/695505153 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+82 (secure 0 0 0) 0x7f5ca80aeed0 con 0x7f5c94067320 2023-10-06T02:29:49.074 INFO:teuthology.orchestra.run.smithi031.stdout:/volumes/_nogroup/subvolume_0000000000554351/de2ab1bd-640c-4194-8b09-3a11819808e2 2023-10-06T02:29:49.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.075+0000 7f5ca7fff700 1 -- 172.21.15.31:0/695505153 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5c94067320 msgr2=0x7f5c940697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:49.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.075+0000 7f5ca7fff700 1 --2- 172.21.15.31:0/695505153 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5c94067320 0x7f5c940697d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5ca80cc9a0 tx=0x7f5c98007d30 comp rx=0 tx=0).stop 2023-10-06T02:29:49.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.075+0000 7f5ca7fff700 1 -- 172.21.15.31:0/695505153 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ca8155c60 msgr2=0x7f5ca8148d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:49.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.075+0000 7f5ca7fff700 1 --2- 172.21.15.31:0/695505153 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ca8155c60 0x7f5ca8148d40 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f5ca80cd1c0 tx=0x7f5ca0019000 comp rx=0 tx=0).stop 2023-10-06T02:29:49.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.075+0000 7f5ca7fff700 1 -- 172.21.15.31:0/695505153 shutdown_connections 2023-10-06T02:29:49.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.075+0000 7f5ca7fff700 1 --2- 172.21.15.31:0/695505153 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ca8155c60 0x7f5ca8148d40 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:49.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.075+0000 7f5ca7fff700 1 --2- 172.21.15.31:0/695505153 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5c94067320 0x7f5c940697d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:49.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.075+0000 7f5ca7fff700 1 --2- 172.21.15.31:0/695505153 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5ca806de50 0x7f5ca8157500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:49.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.075+0000 7f5ca7fff700 1 --2- 172.21.15.31:0/695505153 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ca8155890 0x7f5ca8146fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:49.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.075+0000 7f5ca7fff700 1 -- 172.21.15.31:0/695505153 >> 172.21.15.31:0/695505153 conn(0x7f5ca806d110 msgr2=0x7f5ca80c20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:49.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.075+0000 7f5ca7fff700 1 -- 172.21.15.31:0/695505153 shutdown_connections 2023-10-06T02:29:49.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.075+0000 7f5ca7fff700 1 -- 172.21.15.31:0/695505153 wait complete. 2023-10-06T02:29:49.094 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000554352 group_0000000000489015 2023-10-06T02:29:49.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.285+0000 7f118979d700 1 Processor -- start 2023-10-06T02:29:49.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.285+0000 7f118979d700 1 -- start start 2023-10-06T02:29:49.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.287+0000 7f118979d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f11840c8200 0x7f11840c1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:49.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.287+0000 7f118979d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f11840c2a90 0x7f1184156aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:49.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.287+0000 7f118979d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11841579b0 con 0x7f11840c85d0 2023-10-06T02:29:49.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.287+0000 7f118979d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11840c1390 con 0x7f11840c2a90 2023-10-06T02:29:49.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.287+0000 7f118979d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11841573f0 con 0x7f11840c8200 2023-10-06T02:29:49.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.287+0000 7f1188f9c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f11840c85d0 0x7f11840cc2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:54502/0 (socket says 172.21.15.31:54502) 2023-10-06T02:29:49.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.287+0000 7f1188f9c700 1 -- 172.21.15.31:0/2510190948 learned_addr learned my addr 172.21.15.31:0/2510190948 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:29:49.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.287+0000 7f11837fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f11840c2a90 0x7f1184156aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:49.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.287+0000 7f1183fff700 1 --2- 172.21.15.31:0/2510190948 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f11840c8200 0x7f11840c1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:49.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.287+0000 7f1182ffd700 1 -- 172.21.15.31:0/2510190948 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 75397267 0 0) 0x7f11841579b0 con 0x7f11840c85d0 2023-10-06T02:29:49.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.287+0000 7f1182ffd700 1 -- 172.21.15.31:0/2510190948 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f116c003570 con 0x7f11840c85d0 2023-10-06T02:29:49.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.287+0000 7f11837fe700 1 -- 172.21.15.31:0/2510190948 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f11840c8200 msgr2=0x7f11840c1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:49.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.287+0000 7f11837fe700 1 --2- 172.21.15.31:0/2510190948 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f11840c8200 0x7f11840c1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:49.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.287+0000 7f11837fe700 1 -- 172.21.15.31:0/2510190948 >> v1:172.21.15.31:6789/0 conn(0x7f11840c85d0 legacy=0x7f11840cc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:49.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.287+0000 7f11837fe700 1 -- 172.21.15.31:0/2510190948 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11841576d0 con 0x7f11840c2a90 2023-10-06T02:29:49.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.287+0000 7f11837fe700 1 --2- 172.21.15.31:0/2510190948 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11840c2a90 0x7f1184156aa0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f11780098e0 tx=0x7f11780091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d77a3275d1311db server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:49.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.287+0000 7f1182ffd700 1 -- 172.21.15.31:0/2510190948 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1178011630 con 0x7f11840c2a90 2023-10-06T02:29:49.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.287+0000 7f1182ffd700 1 -- 172.21.15.31:0/2510190948 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1178011790 con 0x7f11840c2a90 2023-10-06T02:29:49.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.287+0000 7f1182ffd700 1 -- 172.21.15.31:0/2510190948 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1178011b30 con 0x7f11840c2a90 2023-10-06T02:29:49.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.288+0000 7f118979d700 1 -- 172.21.15.31:0/2510190948 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11840c2a90 msgr2=0x7f1184156aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:49.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.288+0000 7f118979d700 1 --2- 172.21.15.31:0/2510190948 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11840c2a90 0x7f1184156aa0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f11780098e0 tx=0x7f11780091a0 comp rx=0 tx=0).stop 2023-10-06T02:29:49.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.288+0000 7f118979d700 1 -- 172.21.15.31:0/2510190948 shutdown_connections 2023-10-06T02:29:49.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.288+0000 7f118979d700 1 --2- 172.21.15.31:0/2510190948 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f11840c8200 0x7f11840c1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:49.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.288+0000 7f118979d700 1 --2- 172.21.15.31:0/2510190948 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11840c2a90 0x7f1184156aa0 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:49.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.288+0000 7f118979d700 1 -- 172.21.15.31:0/2510190948 >> 172.21.15.31:0/2510190948 conn(0x7f118406c6a0 msgr2=0x7f118406caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:49.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.288+0000 7f118979d700 1 -- 172.21.15.31:0/2510190948 shutdown_connections 2023-10-06T02:29:49.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.288+0000 7f118979d700 1 -- 172.21.15.31:0/2510190948 wait complete. 2023-10-06T02:29:49.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.289+0000 7f118979d700 1 Processor -- start 2023-10-06T02:29:49.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.289+0000 7f118979d700 1 -- start start 2023-10-06T02:29:49.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.290+0000 7f118979d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11840c2a90 0x7f118413cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:49.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.290+0000 7f118979d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11840c8200 0x7f11841590d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:49.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.290+0000 7f118979d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f11840c85d0 0x7f1184159610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:49.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.290+0000 7f118979d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11840ab3c0 con 0x7f11840c2a90 2023-10-06T02:29:49.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.290+0000 7f118979d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11840abc50 con 0x7f11840c8200 2023-10-06T02:29:49.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.290+0000 7f118979d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11840aae00 con 0x7f11840c85d0 2023-10-06T02:29:49.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.291+0000 7f1183fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11840c2a90 0x7f118413cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:49.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.291+0000 7f1183fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11840c2a90 0x7f118413cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33908/0 (socket says 172.21.15.31:33908) 2023-10-06T02:29:49.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.291+0000 7f1183fff700 1 -- 172.21.15.31:0/3126996876 learned_addr learned my addr 172.21.15.31:0/3126996876 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:49.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.291+0000 7f1183fff700 1 -- 172.21.15.31:0/3126996876 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f11840c85d0 msgr2=0x7f1184159610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:49.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.291+0000 7f1183fff700 1 --2- 172.21.15.31:0/3126996876 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f11840c85d0 0x7f1184159610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:49.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.291+0000 7f1183fff700 1 -- 172.21.15.31:0/3126996876 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11840c8200 msgr2=0x7f11841590d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:49.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.291+0000 7f11837fe700 1 --2- 172.21.15.31:0/3126996876 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11840c8200 0x7f11841590d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:49.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.291+0000 7f1183fff700 1 --2- 172.21.15.31:0/3126996876 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11840c8200 0x7f11841590d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:49.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.291+0000 7f1183fff700 1 -- 172.21.15.31:0/3126996876 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1178009420 con 0x7f11840c2a90 2023-10-06T02:29:49.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.291+0000 7f11837fe700 1 --2- 172.21.15.31:0/3126996876 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11840c8200 0x7f11841590d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:29:49.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.291+0000 7f1183fff700 1 --2- 172.21.15.31:0/3126996876 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11840c2a90 0x7f118413cb10 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f117400ec80 tx=0x7f11740091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:49.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.292+0000 7f116bfff700 1 -- 172.21.15.31:0/3126996876 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117400c9d0 con 0x7f11840c2a90 2023-10-06T02:29:49.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.292+0000 7f116bfff700 1 -- 172.21.15.31:0/3126996876 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f117400cb30 con 0x7f11840c2a90 2023-10-06T02:29:49.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.292+0000 7f118979d700 1 -- 172.21.15.31:0/3126996876 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11840ab0e0 con 0x7f11840c2a90 2023-10-06T02:29:49.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.292+0000 7f116bfff700 1 -- 172.21.15.31:0/3126996876 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1174013470 con 0x7f11840c2a90 2023-10-06T02:29:49.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.292+0000 7f118979d700 1 -- 172.21.15.31:0/3126996876 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f118413d050 con 0x7f11840c2a90 2023-10-06T02:29:49.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.293+0000 7f118979d700 1 -- 172.21.15.31:0/3126996876 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11840abf30 con 0x7f11840c2a90 2023-10-06T02:29:49.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.295+0000 7f116bfff700 1 -- 172.21.15.31:0/3126996876 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f1174013820 con 0x7f11840c2a90 2023-10-06T02:29:49.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.296+0000 7f116bfff700 1 --2- 172.21.15.31:0/3126996876 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f1170067320 0x7f11700697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:49.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.296+0000 7f116bfff700 1 -- 172.21.15.31:0/3126996876 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1174085160 con 0x7f11840c2a90 2023-10-06T02:29:49.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.296+0000 7f11837fe700 1 --2- 172.21.15.31:0/3126996876 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f1170067320 0x7f11700697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:49.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.297+0000 7f11837fe700 1 --2- 172.21.15.31:0/3126996876 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f1170067320 0x7f11700697d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1184147860 tx=0x7f1178017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:49.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.300+0000 7f116bfff700 1 -- 172.21.15.31:0/3126996876 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1174051c90 con 0x7f11840c2a90 2023-10-06T02:29:49.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.539+0000 7f118979d700 1 -- 172.21.15.31:0/3126996876 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000554352", "group_name": "group_0000000000489015", "target": ["mon-mgr", ""]}) v1 -- 0x7f11840ab740 con 0x7f1170067320 2023-10-06T02:29:49.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.579+0000 7f116bfff700 1 -- 172.21.15.31:0/3126996876 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+96 (secure 0 0 0) 0x7f11840ab740 con 0x7f1170067320 2023-10-06T02:29:49.580 INFO:teuthology.orchestra.run.smithi031.stdout:/volumes/group_0000000000489015/subvolume_0000000000554352/b65f3321-3c24-47fd-ada1-51bab8363193 2023-10-06T02:29:49.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.582+0000 7f118979d700 1 -- 172.21.15.31:0/3126996876 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f1170067320 msgr2=0x7f11700697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:49.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.582+0000 7f118979d700 1 --2- 172.21.15.31:0/3126996876 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f1170067320 0x7f11700697d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1184147860 tx=0x7f1178017000 comp rx=0 tx=0).stop 2023-10-06T02:29:49.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.582+0000 7f118979d700 1 -- 172.21.15.31:0/3126996876 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11840c2a90 msgr2=0x7f118413cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:49.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.582+0000 7f118979d700 1 --2- 172.21.15.31:0/3126996876 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11840c2a90 0x7f118413cb10 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f117400ec80 tx=0x7f11740091a0 comp rx=0 tx=0).stop 2023-10-06T02:29:49.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.582+0000 7f118979d700 1 -- 172.21.15.31:0/3126996876 shutdown_connections 2023-10-06T02:29:49.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.582+0000 7f118979d700 1 --2- 172.21.15.31:0/3126996876 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f1170067320 0x7f11700697d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:49.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.582+0000 7f118979d700 1 --2- 172.21.15.31:0/3126996876 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f11840c85d0 0x7f1184159610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:49.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.582+0000 7f118979d700 1 --2- 172.21.15.31:0/3126996876 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11840c2a90 0x7f118413cb10 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:49.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.582+0000 7f118979d700 1 --2- 172.21.15.31:0/3126996876 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11840c8200 0x7f11841590d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:49.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.582+0000 7f118979d700 1 -- 172.21.15.31:0/3126996876 >> 172.21.15.31:0/3126996876 conn(0x7f118406c6a0 msgr2=0x7f118414d920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:49.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.582+0000 7f118979d700 1 -- 172.21.15.31:0/3126996876 shutdown_connections 2023-10-06T02:29:49.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.582+0000 7f118979d700 1 -- 172.21.15.31:0/3126996876 wait complete. 2023-10-06T02:29:49.600 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvolume_0000000000554353 2023-10-06T02:29:49.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.790+0000 7f817f39d700 1 Processor -- start 2023-10-06T02:29:49.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.790+0000 7f817f39d700 1 -- start start 2023-10-06T02:29:49.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.791+0000 7f817f39d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f81780c7fb0 0x7f817806abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:49.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.792+0000 7f817f39d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f817813f780 0x7f817813fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:49.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.792+0000 7f817f39d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8178140120 con 0x7f81780c7be0 2023-10-06T02:29:49.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.792+0000 7f817f39d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f817806bf10 con 0x7f817813f780 2023-10-06T02:29:49.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.792+0000 7f817f39d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f817806b290 con 0x7f81780c7fb0 2023-10-06T02:29:49.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.792+0000 7f817eb9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f817813f780 0x7f817813fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:49.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.792+0000 7f817eb9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f817813f780 0x7f817813fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33922/0 (socket says 172.21.15.31:33922) 2023-10-06T02:29:49.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.792+0000 7f817eb9c700 1 -- 172.21.15.31:0/614850790 learned_addr learned my addr 172.21.15.31:0/614850790 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:49.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.792+0000 7f817db9a700 1 --2- 172.21.15.31:0/614850790 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f81780c7fb0 0x7f817806abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:49.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.792+0000 7f817eb9c700 1 -- 172.21.15.31:0/614850790 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f81780c7fb0 msgr2=0x7f817806abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:49.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.792+0000 7f817eb9c700 1 --2- 172.21.15.31:0/614850790 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f81780c7fb0 0x7f817806abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:49.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.792+0000 7f817eb9c700 1 -- 172.21.15.31:0/614850790 >> v1:172.21.15.62:6789/0 conn(0x7f81780c7be0 legacy=0x7f817815a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:49.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.792+0000 7f817eb9c700 1 -- 172.21.15.31:0/614850790 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f817806b570 con 0x7f817813f780 2023-10-06T02:29:49.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.792+0000 7f817db9a700 1 --2- 172.21.15.31:0/614850790 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f81780c7fb0 0x7f817806abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:29:49.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.793+0000 7f817eb9c700 1 --2- 172.21.15.31:0/614850790 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f817813f780 0x7f817813fbe0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f8170009f20 tx=0x7f8170009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=733262be9519c4f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:49.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.793+0000 7f817d399700 1 -- 172.21.15.31:0/614850790 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f817000e070 con 0x7f817813f780 2023-10-06T02:29:49.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.793+0000 7f817d399700 1 -- 172.21.15.31:0/614850790 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f817000ba80 con 0x7f817813f780 2023-10-06T02:29:49.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.793+0000 7f817d399700 1 -- 172.21.15.31:0/614850790 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81700092e0 con 0x7f817813f780 2023-10-06T02:29:49.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.794+0000 7f817f39d700 1 -- 172.21.15.31:0/614850790 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f817813f780 msgr2=0x7f817813fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:49.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.794+0000 7f817f39d700 1 --2- 172.21.15.31:0/614850790 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f817813f780 0x7f817813fbe0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f8170009f20 tx=0x7f8170009840 comp rx=0 tx=0).stop 2023-10-06T02:29:49.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.794+0000 7f817f39d700 1 -- 172.21.15.31:0/614850790 shutdown_connections 2023-10-06T02:29:49.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.794+0000 7f817f39d700 1 --2- 172.21.15.31:0/614850790 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f817813f780 0x7f817813fbe0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:49.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.794+0000 7f817f39d700 1 --2- 172.21.15.31:0/614850790 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f81780c7fb0 0x7f817806abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:49.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.794+0000 7f817f39d700 1 -- 172.21.15.31:0/614850790 >> 172.21.15.31:0/614850790 conn(0x7f817806d110 msgr2=0x7f81780cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:49.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.794+0000 7f817f39d700 1 -- 172.21.15.31:0/614850790 shutdown_connections 2023-10-06T02:29:49.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.794+0000 7f817f39d700 1 -- 172.21.15.31:0/614850790 wait complete. 2023-10-06T02:29:49.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.795+0000 7f817f39d700 1 Processor -- start 2023-10-06T02:29:49.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.795+0000 7f817f39d700 1 -- start start 2023-10-06T02:29:49.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.796+0000 7f817f39d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81780c7be0 0x7f8178136580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:49.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.796+0000 7f817f39d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f81780c7fb0 0x7f81780c3430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:49.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.796+0000 7f817f39d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f817813f780 0x7f8178155770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:49.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.796+0000 7f817f39d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81780afd30 con 0x7f817813f780 2023-10-06T02:29:49.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.796+0000 7f817f39d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81780b0010 con 0x7f81780c7be0 2023-10-06T02:29:49.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.796+0000 7f817f39d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81780b02f0 con 0x7f81780c7fb0 2023-10-06T02:29:49.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.797+0000 7f817eb9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f817813f780 0x7f8178155770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:49.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.797+0000 7f817eb9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f817813f780 0x7f8178155770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33938/0 (socket says 172.21.15.31:33938) 2023-10-06T02:29:49.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.797+0000 7f817eb9c700 1 -- 172.21.15.31:0/1125743994 learned_addr learned my addr 172.21.15.31:0/1125743994 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:49.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.797+0000 7f817e39b700 1 --2- 172.21.15.31:0/1125743994 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81780c7be0 0x7f8178136580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:49.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.797+0000 7f817db9a700 1 --2- 172.21.15.31:0/1125743994 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f81780c7fb0 0x7f81780c3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:49.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.797+0000 7f817eb9c700 1 -- 172.21.15.31:0/1125743994 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f81780c7fb0 msgr2=0x7f81780c3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:49.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.797+0000 7f817eb9c700 1 --2- 172.21.15.31:0/1125743994 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f81780c7fb0 0x7f81780c3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:49.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.797+0000 7f817eb9c700 1 -- 172.21.15.31:0/1125743994 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81780c7be0 msgr2=0x7f8178136580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:49.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.797+0000 7f817eb9c700 1 --2- 172.21.15.31:0/1125743994 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81780c7be0 0x7f8178136580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:49.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.797+0000 7f817eb9c700 1 -- 172.21.15.31:0/1125743994 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8170009ae0 con 0x7f817813f780 2023-10-06T02:29:49.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.797+0000 7f817db9a700 1 --2- 172.21.15.31:0/1125743994 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f81780c7fb0 0x7f81780c3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:29:49.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.797+0000 7f817eb9c700 1 --2- 172.21.15.31:0/1125743994 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f817813f780 0x7f8178155770 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f81780693f0 tx=0x7f8170018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:49.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.798+0000 7f816e7fc700 1 -- 172.21.15.31:0/1125743994 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81700092e0 con 0x7f817813f780 2023-10-06T02:29:49.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.798+0000 7f816e7fc700 1 -- 172.21.15.31:0/1125743994 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8170008dd0 con 0x7f817813f780 2023-10-06T02:29:49.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.798+0000 7f816e7fc700 1 -- 172.21.15.31:0/1125743994 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8170026690 con 0x7f817813f780 2023-10-06T02:29:49.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.798+0000 7f817f39d700 1 -- 172.21.15.31:0/1125743994 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81780b05d0 con 0x7f817813f780 2023-10-06T02:29:49.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.798+0000 7f817f39d700 1 -- 172.21.15.31:0/1125743994 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81780c3f90 con 0x7f817813f780 2023-10-06T02:29:49.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.799+0000 7f817f39d700 1 -- 172.21.15.31:0/1125743994 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8178133c30 con 0x7f817813f780 2023-10-06T02:29:49.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.800+0000 7f816e7fc700 1 -- 172.21.15.31:0/1125743994 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f8170003040 con 0x7f817813f780 2023-10-06T02:29:49.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.801+0000 7f816e7fc700 1 --2- 172.21.15.31:0/1125743994 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f815c067320 0x7f815c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:49.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.801+0000 7f816e7fc700 1 -- 172.21.15.31:0/1125743994 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8170086c30 con 0x7f817813f780 2023-10-06T02:29:49.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.801+0000 7f817e39b700 1 --2- 172.21.15.31:0/1125743994 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f815c067320 0x7f815c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:49.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.802+0000 7f817e39b700 1 --2- 172.21.15.31:0/1125743994 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f815c067320 0x7f815c0697d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f81740039e0 tx=0x7f8174007ec0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:49.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:49.805+0000 7f816e7fc700 1 -- 172.21.15.31:0/1125743994 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f81700536c0 con 0x7f817813f780 2023-10-06T02:29:50.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.054+0000 7f817f39d700 1 -- 172.21.15.31:0/1125743994 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvolume_0000000000554353", "target": ["mon-mgr", ""]}) v1 -- 0x7f81780b0a70 con 0x7f815c067320 2023-10-06T02:29:50.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.089+0000 7f816e7fc700 1 -- 172.21.15.31:0/1125743994 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+144 (secure 0 0 0) 0x7f81780b0a70 con 0x7f815c067320 2023-10-06T02:29:50.090 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T02:29:50.090 INFO:teuthology.orchestra.run.smithi031.stdout: "status": { 2023-10-06T02:29:50.090 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "pending", 2023-10-06T02:29:50.090 INFO:teuthology.orchestra.run.smithi031.stdout: "source": { 2023-10-06T02:29:50.091 INFO:teuthology.orchestra.run.smithi031.stdout: "volume": "fake", 2023-10-06T02:29:50.091 INFO:teuthology.orchestra.run.smithi031.stdout: "subvolume": "fake", 2023-10-06T02:29:50.091 INFO:teuthology.orchestra.run.smithi031.stdout: "snapshot": "fake" 2023-10-06T02:29:50.091 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:29:50.091 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:29:50.091 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T02:29:50.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.092+0000 7f817f39d700 1 -- 172.21.15.31:0/1125743994 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f815c067320 msgr2=0x7f815c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:50.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.092+0000 7f817f39d700 1 --2- 172.21.15.31:0/1125743994 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f815c067320 0x7f815c0697d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f81740039e0 tx=0x7f8174007ec0 comp rx=0 tx=0).stop 2023-10-06T02:29:50.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.092+0000 7f817f39d700 1 -- 172.21.15.31:0/1125743994 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f817813f780 msgr2=0x7f8178155770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:50.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.092+0000 7f817f39d700 1 --2- 172.21.15.31:0/1125743994 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f817813f780 0x7f8178155770 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f81780693f0 tx=0x7f8170018000 comp rx=0 tx=0).stop 2023-10-06T02:29:50.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.092+0000 7f817f39d700 1 -- 172.21.15.31:0/1125743994 shutdown_connections 2023-10-06T02:29:50.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.092+0000 7f817f39d700 1 --2- 172.21.15.31:0/1125743994 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f817813f780 0x7f8178155770 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:50.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.092+0000 7f817f39d700 1 --2- 172.21.15.31:0/1125743994 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81780c7be0 0x7f8178136580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:50.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.092+0000 7f817f39d700 1 --2- 172.21.15.31:0/1125743994 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f815c067320 0x7f815c0697d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:50.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.092+0000 7f817f39d700 1 --2- 172.21.15.31:0/1125743994 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f81780c7fb0 0x7f81780c3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:50.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.092+0000 7f817f39d700 1 -- 172.21.15.31:0/1125743994 >> 172.21.15.31:0/1125743994 conn(0x7f817806d110 msgr2=0x7f81780cb040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:50.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.092+0000 7f817f39d700 1 -- 172.21.15.31:0/1125743994 shutdown_connections 2023-10-06T02:29:50.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.092+0000 7f817f39d700 1 -- 172.21.15.31:0/1125743994 wait complete. 2023-10-06T02:29:50.109 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm cephfs subvolume_0000000000554351 fake 2023-10-06T02:29:50.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.301+0000 7faa2e39d700 1 Processor -- start 2023-10-06T02:29:50.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.301+0000 7faa2e39d700 1 -- start start 2023-10-06T02:29:50.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.303+0000 7faa2e39d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7faa280c2a90 0x7faa28156aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:50.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.303+0000 7faa2e39d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7faa280c85d0 0x7faa280cc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:50.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.303+0000 7faa2e39d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa281579b0 con 0x7faa280c8200 2023-10-06T02:29:50.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.303+0000 7faa2e39d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa280c1390 con 0x7faa280c2a90 2023-10-06T02:29:50.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.303+0000 7faa2e39d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa281573f0 con 0x7faa280c85d0 2023-10-06T02:29:50.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.303+0000 7faa2cb9a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7faa280c2a90 0x7faa28156aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:50.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.303+0000 7faa2cb9a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7faa280c2a90 0x7faa28156aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33944/0 (socket says 172.21.15.31:33944) 2023-10-06T02:29:50.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.303+0000 7faa2cb9a700 1 -- 172.21.15.31:0/3746583374 learned_addr learned my addr 172.21.15.31:0/3746583374 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:50.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.303+0000 7faa2db9c700 1 --2- 172.21.15.31:0/3746583374 >> v2:172.21.15.62:3300/0 conn(0x7faa280c85d0 0x7faa280cc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:50.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.304+0000 7faa2cb9a700 1 -- 172.21.15.31:0/3746583374 >> v2:172.21.15.62:3300/0 conn(0x7faa280c85d0 msgr2=0x7faa280cc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:50.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.304+0000 7faa2cb9a700 1 --2- 172.21.15.31:0/3746583374 >> v2:172.21.15.62:3300/0 conn(0x7faa280c85d0 0x7faa280cc2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:50.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.304+0000 7faa2cb9a700 1 -- 172.21.15.31:0/3746583374 >> v1:172.21.15.62:6789/0 conn(0x7faa280c8200 legacy=0x7faa280c1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:50.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.304+0000 7faa2cb9a700 1 -- 172.21.15.31:0/3746583374 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa281576d0 con 0x7faa280c2a90 2023-10-06T02:29:50.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.304+0000 7faa2cb9a700 1 --2- 172.21.15.31:0/3746583374 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa280c2a90 0x7faa28156aa0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7faa280c1300 tx=0x7faa1800b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c7c9132ba4fd316 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:50.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.304+0000 7faa1ffff700 1 -- 172.21.15.31:0/3746583374 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa1800dba0 con 0x7faa280c2a90 2023-10-06T02:29:50.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.304+0000 7faa1ffff700 1 -- 172.21.15.31:0/3746583374 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa1800dd00 con 0x7faa280c2a90 2023-10-06T02:29:50.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.304+0000 7faa1ffff700 1 -- 172.21.15.31:0/3746583374 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa180196d0 con 0x7faa280c2a90 2023-10-06T02:29:50.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.305+0000 7faa2e39d700 1 -- 172.21.15.31:0/3746583374 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa280c2a90 msgr2=0x7faa28156aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:50.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.305+0000 7faa2e39d700 1 --2- 172.21.15.31:0/3746583374 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa280c2a90 0x7faa28156aa0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7faa280c1300 tx=0x7faa1800b4f0 comp rx=0 tx=0).stop 2023-10-06T02:29:50.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.305+0000 7faa2e39d700 1 -- 172.21.15.31:0/3746583374 shutdown_connections 2023-10-06T02:29:50.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.305+0000 7faa2e39d700 1 --2- 172.21.15.31:0/3746583374 >> v2:172.21.15.62:3300/0 conn(0x7faa280c85d0 0x7faa280cc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:50.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.305+0000 7faa2e39d700 1 --2- 172.21.15.31:0/3746583374 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa280c2a90 0x7faa28156aa0 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:50.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.305+0000 7faa2e39d700 1 -- 172.21.15.31:0/3746583374 >> 172.21.15.31:0/3746583374 conn(0x7faa2806c6a0 msgr2=0x7faa2806caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:50.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.305+0000 7faa2e39d700 1 -- 172.21.15.31:0/3746583374 shutdown_connections 2023-10-06T02:29:50.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.305+0000 7faa2e39d700 1 -- 172.21.15.31:0/3746583374 wait complete. 2023-10-06T02:29:50.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.306+0000 7faa2e39d700 1 Processor -- start 2023-10-06T02:29:50.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.306+0000 7faa2e39d700 1 -- start start 2023-10-06T02:29:50.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.307+0000 7faa2e39d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa280c2a90 0x7faa28141de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:50.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.307+0000 7faa2e39d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faa280c8200 0x7faa2815cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:50.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.307+0000 7faa2e39d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa280c85d0 0x7faa280af8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:50.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.307+0000 7faa2e39d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa28142320 con 0x7faa280c2a90 2023-10-06T02:29:50.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.307+0000 7faa2e39d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa280b0330 con 0x7faa280c85d0 2023-10-06T02:29:50.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.307+0000 7faa2e39d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa2815d110 con 0x7faa280c8200 2023-10-06T02:29:50.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.308+0000 7faa2d39b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa280c2a90 0x7faa28141de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:50.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.308+0000 7faa2d39b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa280c2a90 0x7faa28141de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33958/0 (socket says 172.21.15.31:33958) 2023-10-06T02:29:50.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.308+0000 7faa2d39b700 1 -- 172.21.15.31:0/2427013289 learned_addr learned my addr 172.21.15.31:0/2427013289 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:50.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.308+0000 7faa2cb9a700 1 --2- 172.21.15.31:0/2427013289 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faa280c8200 0x7faa2815cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:50.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.308+0000 7faa2db9c700 1 --2- 172.21.15.31:0/2427013289 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa280c85d0 0x7faa280af8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:50.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.308+0000 7faa2d39b700 1 -- 172.21.15.31:0/2427013289 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faa280c8200 msgr2=0x7faa2815cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:50.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.308+0000 7faa2d39b700 1 --2- 172.21.15.31:0/2427013289 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faa280c8200 0x7faa2815cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:50.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.308+0000 7faa2d39b700 1 -- 172.21.15.31:0/2427013289 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa280c85d0 msgr2=0x7faa280af8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:50.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.308+0000 7faa2d39b700 1 --2- 172.21.15.31:0/2427013289 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa280c85d0 0x7faa280af8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:50.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.308+0000 7faa2d39b700 1 -- 172.21.15.31:0/2427013289 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa1800b790 con 0x7faa280c2a90 2023-10-06T02:29:50.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.308+0000 7faa2db9c700 1 --2- 172.21.15.31:0/2427013289 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa280c85d0 0x7faa280af8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:29:50.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.308+0000 7faa2d39b700 1 --2- 172.21.15.31:0/2427013289 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa280c2a90 0x7faa28141de0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7faa2400c590 tx=0x7faa2400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:50.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.308+0000 7faa1d7fa700 1 -- 172.21.15.31:0/2427013289 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa24011590 con 0x7faa280c2a90 2023-10-06T02:29:50.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.308+0000 7faa1d7fa700 1 -- 172.21.15.31:0/2427013289 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa24011710 con 0x7faa280c2a90 2023-10-06T02:29:50.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.309+0000 7faa2e39d700 1 -- 172.21.15.31:0/2427013289 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa280b0050 con 0x7faa280c2a90 2023-10-06T02:29:50.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.309+0000 7faa1d7fa700 1 -- 172.21.15.31:0/2427013289 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa24011ad0 con 0x7faa280c2a90 2023-10-06T02:29:50.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.309+0000 7faa2e39d700 1 -- 172.21.15.31:0/2427013289 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa280abb50 con 0x7faa280c2a90 2023-10-06T02:29:50.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.310+0000 7faa2e39d700 1 -- 172.21.15.31:0/2427013289 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa2804c3c0 con 0x7faa280c2a90 2023-10-06T02:29:50.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.311+0000 7faa1d7fa700 1 -- 172.21.15.31:0/2427013289 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) v1 ==== 85097+0+0 (secure 0 0 0) 0x7faa2401b3b0 con 0x7faa280c2a90 2023-10-06T02:29:50.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.312+0000 7faa1d7fa700 1 --2- 172.21.15.31:0/2427013289 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7faa0c067320 0x7faa0c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:50.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.312+0000 7faa1d7fa700 1 -- 172.21.15.31:0/2427013289 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6058+0+0 (secure 0 0 0) 0x7faa24084300 con 0x7faa280c2a90 2023-10-06T02:29:50.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.313+0000 7faa2cb9a700 1 --2- 172.21.15.31:0/2427013289 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7faa0c067320 0x7faa0c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:50.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.313+0000 7faa2cb9a700 1 --2- 172.21.15.31:0/2427013289 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7faa0c067320 0x7faa0c0697d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7faa280c1300 tx=0x7faa1801b000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:50.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.316+0000 7faa1d7fa700 1 -- 172.21.15.31:0/2427013289 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faa24050e30 con 0x7faa280c2a90 2023-10-06T02:29:50.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.559+0000 7faa2e39d700 1 -- 172.21.15.31:0/2427013289 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000554351", "snap_name": "fake", "target": ["mon-mgr", ""]}) v1 -- 0x7faa280c1010 con 0x7faa0c067320 2023-10-06T02:29:50.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.575+0000 7faa1d7fa700 1 -- 172.21.15.31:0/2427013289 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mgrmap(e 21) v1 ==== 85148+0+0 (secure 0 0 0) 0x7faa24051200 con 0x7faa280c2a90 2023-10-06T02:29:50.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.613+0000 7faa1d7fa700 1 -- 172.21.15.31:0/2427013289 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7faa280c1010 con 0x7faa0c067320 2023-10-06T02:29:50.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.616+0000 7faa2e39d700 1 -- 172.21.15.31:0/2427013289 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7faa0c067320 msgr2=0x7faa0c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:50.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.616+0000 7faa2e39d700 1 --2- 172.21.15.31:0/2427013289 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7faa0c067320 0x7faa0c0697d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7faa280c1300 tx=0x7faa1801b000 comp rx=0 tx=0).stop 2023-10-06T02:29:50.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.616+0000 7faa2e39d700 1 -- 172.21.15.31:0/2427013289 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa280c2a90 msgr2=0x7faa28141de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:50.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.616+0000 7faa2e39d700 1 --2- 172.21.15.31:0/2427013289 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa280c2a90 0x7faa28141de0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7faa2400c590 tx=0x7faa2400c8a0 comp rx=0 tx=0).stop 2023-10-06T02:29:50.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.616+0000 7faa2e39d700 1 -- 172.21.15.31:0/2427013289 shutdown_connections 2023-10-06T02:29:50.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.616+0000 7faa2e39d700 1 --2- 172.21.15.31:0/2427013289 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa280c85d0 0x7faa280af8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:50.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.616+0000 7faa2e39d700 1 --2- 172.21.15.31:0/2427013289 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa280c2a90 0x7faa28141de0 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:50.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.616+0000 7faa2e39d700 1 --2- 172.21.15.31:0/2427013289 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7faa0c067320 0x7faa0c0697d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:50.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.616+0000 7faa2e39d700 1 --2- 172.21.15.31:0/2427013289 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faa280c8200 0x7faa2815cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:50.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.616+0000 7faa2e39d700 1 -- 172.21.15.31:0/2427013289 >> 172.21.15.31:0/2427013289 conn(0x7faa2806c6a0 msgr2=0x7faa28150140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:50.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.616+0000 7faa2e39d700 1 -- 172.21.15.31:0/2427013289 shutdown_connections 2023-10-06T02:29:50.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.616+0000 7faa2e39d700 1 -- 172.21.15.31:0/2427013289 wait complete. 2023-10-06T02:29:50.633 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm cephfs subvolume_0000000000554352 fake group_0000000000489015 2023-10-06T02:29:50.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.827+0000 7f98ab59e700 1 Processor -- start 2023-10-06T02:29:50.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.827+0000 7f98ab59e700 1 -- start start 2023-10-06T02:29:50.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.828+0000 7f98ab59e700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f98ac13fb20 0x7f98ac15a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:50.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.828+0000 7f98ab59e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f98ac06aaa0 0x7f98ac0c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:50.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.828+0000 7f98ab59e700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98ac15ae70 con 0x7f98ac13fef0 2023-10-06T02:29:50.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.829+0000 7f98ab59e700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98ac06af60 con 0x7f98ac06aaa0 2023-10-06T02:29:50.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.829+0000 7f98ab59e700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98ac06b240 con 0x7f98ac13fb20 2023-10-06T02:29:50.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.829+0000 7f98aad9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f98ac06aaa0 0x7f98ac0c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:50.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.829+0000 7f98a9d9b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f98ac13fef0 0x7f98ac0c7990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:54510/0 (socket says 172.21.15.31:54510) 2023-10-06T02:29:50.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.829+0000 7f98a9d9b700 1 -- 172.21.15.31:0/281825705 learned_addr learned my addr 172.21.15.31:0/281825705 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:29:50.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.829+0000 7f98aa59c700 1 --2- 172.21.15.31:0/281825705 >> v2:172.21.15.62:3300/0 conn(0x7f98ac13fb20 0x7f98ac15a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:50.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.829+0000 7f98aad9d700 1 -- 172.21.15.31:0/281825705 >> v2:172.21.15.62:3300/0 conn(0x7f98ac13fb20 msgr2=0x7f98ac15a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:50.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.829+0000 7f98aad9d700 1 --2- 172.21.15.31:0/281825705 >> v2:172.21.15.62:3300/0 conn(0x7f98ac13fb20 0x7f98ac15a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:50.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.829+0000 7f98a959a700 1 -- 172.21.15.31:0/281825705 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2139300831 0 0) 0x7f98a0001290 con 0x7f98ac13fef0 2023-10-06T02:29:50.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.829+0000 7f98aad9d700 1 -- 172.21.15.31:0/281825705 >> v1:172.21.15.31:6789/0 conn(0x7f98ac13fef0 legacy=0x7f98ac0c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:50.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.829+0000 7f98aad9d700 1 -- 172.21.15.31:0/281825705 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98ac06b520 con 0x7f98ac06aaa0 2023-10-06T02:29:50.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.830+0000 7f98aad9d700 1 --2- 172.21.15.31:0/281825705 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98ac06aaa0 0x7f98ac0c9d00 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f98ac0695b0 tx=0x7f98a4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3383c6ab52c1e837 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:50.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.830+0000 7f98a959a700 1 -- 172.21.15.31:0/281825705 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a400e070 con 0x7f98ac06aaa0 2023-10-06T02:29:50.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.830+0000 7f98a959a700 1 -- 172.21.15.31:0/281825705 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98a400bb60 con 0x7f98ac06aaa0 2023-10-06T02:29:50.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.830+0000 7f98a959a700 1 -- 172.21.15.31:0/281825705 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a4012430 con 0x7f98ac06aaa0 2023-10-06T02:29:50.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.830+0000 7f98ab59e700 1 -- 172.21.15.31:0/281825705 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98ac06aaa0 msgr2=0x7f98ac0c9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:50.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.830+0000 7f98ab59e700 1 --2- 172.21.15.31:0/281825705 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98ac06aaa0 0x7f98ac0c9d00 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f98ac0695b0 tx=0x7f98a4009780 comp rx=0 tx=0).stop 2023-10-06T02:29:50.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.831+0000 7f98ab59e700 1 -- 172.21.15.31:0/281825705 shutdown_connections 2023-10-06T02:29:50.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.831+0000 7f98ab59e700 1 --2- 172.21.15.31:0/281825705 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98ac06aaa0 0x7f98ac0c9d00 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:50.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.831+0000 7f98ab59e700 1 --2- 172.21.15.31:0/281825705 >> v2:172.21.15.62:3300/0 conn(0x7f98ac13fb20 0x7f98ac15a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:50.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.831+0000 7f98ab59e700 1 -- 172.21.15.31:0/281825705 >> 172.21.15.31:0/281825705 conn(0x7f98ac06d170 msgr2=0x7f98ac0cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:50.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.831+0000 7f98ab59e700 1 -- 172.21.15.31:0/281825705 shutdown_connections 2023-10-06T02:29:50.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.831+0000 7f98ab59e700 1 -- 172.21.15.31:0/281825705 wait complete. 2023-10-06T02:29:50.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.832+0000 7f98ab59e700 1 Processor -- start 2023-10-06T02:29:50.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.832+0000 7f98ab59e700 1 -- start start 2023-10-06T02:29:50.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.833+0000 7f98ab59e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f98ac06aaa0 0x7f98ac155d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:50.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.833+0000 7f98ab59e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98ac13fb20 0x7f98ac0adf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:50.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.833+0000 7f98ab59e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98ac13fef0 0x7f98ac12d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:50.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.833+0000 7f98ab59e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98ac0ac4f0 con 0x7f98ac13fb20 2023-10-06T02:29:50.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.833+0000 7f98ab59e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98ac1562a0 con 0x7f98ac13fef0 2023-10-06T02:29:50.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.833+0000 7f98ab59e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98ac0ab8e0 con 0x7f98ac06aaa0 2023-10-06T02:29:50.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.834+0000 7f98a9d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98ac13fb20 0x7f98ac0adf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:50.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.834+0000 7f98aa59c700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f98ac06aaa0 0x7f98ac155d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:50.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.834+0000 7f98a9d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98ac13fb20 0x7f98ac0adf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33974/0 (socket says 172.21.15.31:33974) 2023-10-06T02:29:50.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.834+0000 7f98a9d9b700 1 -- 172.21.15.31:0/2304924623 learned_addr learned my addr 172.21.15.31:0/2304924623 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:50.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.834+0000 7f98aad9d700 1 --2- 172.21.15.31:0/2304924623 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98ac13fef0 0x7f98ac12d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:50.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.834+0000 7f98a9d9b700 1 -- 172.21.15.31:0/2304924623 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f98ac06aaa0 msgr2=0x7f98ac155d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:50.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.834+0000 7f98a9d9b700 1 --2- 172.21.15.31:0/2304924623 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f98ac06aaa0 0x7f98ac155d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:50.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.834+0000 7f98a9d9b700 1 -- 172.21.15.31:0/2304924623 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98ac13fef0 msgr2=0x7f98ac12d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:50.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.834+0000 7f98a9d9b700 1 --2- 172.21.15.31:0/2304924623 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98ac13fef0 0x7f98ac12d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:50.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.834+0000 7f98a9d9b700 1 -- 172.21.15.31:0/2304924623 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98a4009a20 con 0x7f98ac13fb20 2023-10-06T02:29:50.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.834+0000 7f98a9d9b700 1 --2- 172.21.15.31:0/2304924623 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98ac13fb20 0x7f98ac0adf20 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f98a000c5e0 tx=0x7f98a0007e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:50.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.834+0000 7f989a7fc700 1 -- 172.21.15.31:0/2304924623 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a0001ea0 con 0x7f98ac13fb20 2023-10-06T02:29:50.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.834+0000 7f989a7fc700 1 -- 172.21.15.31:0/2304924623 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98a00072e0 con 0x7f98ac13fb20 2023-10-06T02:29:50.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.834+0000 7f98ab59e700 1 -- 172.21.15.31:0/2304924623 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98ac0abbc0 con 0x7f98ac13fb20 2023-10-06T02:29:50.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.834+0000 7f989a7fc700 1 -- 172.21.15.31:0/2304924623 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a00057f0 con 0x7f98ac13fb20 2023-10-06T02:29:50.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.835+0000 7f98ab59e700 1 -- 172.21.15.31:0/2304924623 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98ac0ac180 con 0x7f98ac13fb20 2023-10-06T02:29:50.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.836+0000 7f98ab59e700 1 -- 172.21.15.31:0/2304924623 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98ac0ac660 con 0x7f98ac13fb20 2023-10-06T02:29:50.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.837+0000 7f989a7fc700 1 -- 172.21.15.31:0/2304924623 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) v1 ==== 85148+0+0 (secure 0 0 0) 0x7f98a0005c20 con 0x7f98ac13fb20 2023-10-06T02:29:50.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.838+0000 7f989a7fc700 1 --2- 172.21.15.31:0/2304924623 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f988c0673e0 0x7f988c069890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:50.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.838+0000 7f989a7fc700 1 -- 172.21.15.31:0/2304924623 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f98a0083fb0 con 0x7f98ac13fb20 2023-10-06T02:29:50.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.838+0000 7f98aa59c700 1 --2- 172.21.15.31:0/2304924623 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f988c0673e0 0x7f988c069890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:50.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.839+0000 7f98aa59c700 1 --2- 172.21.15.31:0/2304924623 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f988c0673e0 0x7f988c069890 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f989c010e50 tx=0x7f989c012040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:50.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:50.842+0000 7f989a7fc700 1 -- 172.21.15.31:0/2304924623 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f98a004b5a0 con 0x7f98ac13fb20 2023-10-06T02:29:51.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.084+0000 7f98ab59e700 1 -- 172.21.15.31:0/2304924623 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000554352", "snap_name": "fake", "group_name": "group_0000000000489015", "target": ["mon-mgr", ""]}) v1 -- 0x7f98ac04c3c0 con 0x7f988c0673e0 2023-10-06T02:29:51.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.137+0000 7f989a7fc700 1 -- 172.21.15.31:0/2304924623 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f98ac04c3c0 con 0x7f988c0673e0 2023-10-06T02:29:51.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.139+0000 7f98ab59e700 1 -- 172.21.15.31:0/2304924623 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f988c0673e0 msgr2=0x7f988c069890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:51.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.139+0000 7f98ab59e700 1 --2- 172.21.15.31:0/2304924623 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f988c0673e0 0x7f988c069890 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f989c010e50 tx=0x7f989c012040 comp rx=0 tx=0).stop 2023-10-06T02:29:51.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.139+0000 7f98ab59e700 1 -- 172.21.15.31:0/2304924623 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98ac13fb20 msgr2=0x7f98ac0adf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:51.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.139+0000 7f98ab59e700 1 --2- 172.21.15.31:0/2304924623 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98ac13fb20 0x7f98ac0adf20 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f98a000c5e0 tx=0x7f98a0007e20 comp rx=0 tx=0).stop 2023-10-06T02:29:51.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.140+0000 7f98ab59e700 1 -- 172.21.15.31:0/2304924623 shutdown_connections 2023-10-06T02:29:51.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.140+0000 7f98ab59e700 1 --2- 172.21.15.31:0/2304924623 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98ac13fef0 0x7f98ac12d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:51.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.140+0000 7f98ab59e700 1 --2- 172.21.15.31:0/2304924623 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f988c0673e0 0x7f988c069890 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:51.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.140+0000 7f98ab59e700 1 --2- 172.21.15.31:0/2304924623 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f98ac06aaa0 0x7f98ac155d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:51.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.140+0000 7f98ab59e700 1 --2- 172.21.15.31:0/2304924623 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98ac13fb20 0x7f98ac0adf20 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:51.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.140+0000 7f98ab59e700 1 -- 172.21.15.31:0/2304924623 >> 172.21.15.31:0/2304924623 conn(0x7f98ac06d170 msgr2=0x7f98ac0aeed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:51.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.140+0000 7f98ab59e700 1 -- 172.21.15.31:0/2304924623 shutdown_connections 2023-10-06T02:29:51.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.140+0000 7f98ab59e700 1 -- 172.21.15.31:0/2304924623 wait complete. 2023-10-06T02:29:51.157 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:29:51.158 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvolume_0000000000554351/.meta) 2023-10-06T02:29:51.221 INFO:teuthology.orchestra.run.smithi031.stdout:[GLOBAL] 2023-10-06T02:29:51.222 INFO:teuthology.orchestra.run.smithi031.stdout:version = 1 2023-10-06T02:29:51.222 INFO:teuthology.orchestra.run.smithi031.stdout:type = subvolume 2023-10-06T02:29:51.222 INFO:teuthology.orchestra.run.smithi031.stdout:path = /volumes/_nogroup/subvolume_0000000000554351/de2ab1bd-640c-4194-8b09-3a11819808e2 2023-10-06T02:29:51.222 INFO:teuthology.orchestra.run.smithi031.stdout:state = complete 2023-10-06T02:29:51.222 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:29:51.225 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:29:51.225 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/group_0000000000489015/subvolume_0000000000554352/.meta) 2023-10-06T02:29:51.287 INFO:teuthology.orchestra.run.smithi031.stdout:[GLOBAL] 2023-10-06T02:29:51.288 INFO:teuthology.orchestra.run.smithi031.stdout:version = 1 2023-10-06T02:29:51.288 INFO:teuthology.orchestra.run.smithi031.stdout:type = subvolume 2023-10-06T02:29:51.288 INFO:teuthology.orchestra.run.smithi031.stdout:path = /volumes/group_0000000000489015/subvolume_0000000000554352/b65f3321-3c24-47fd-ada1-51bab8363193 2023-10-06T02:29:51.288 INFO:teuthology.orchestra.run.smithi031.stdout:state = complete 2023-10-06T02:29:51.288 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:29:51.291 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000554351 2023-10-06T02:29:51.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.482+0000 7f5c90f9d700 1 Processor -- start 2023-10-06T02:29:51.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.482+0000 7f5c90f9d700 1 -- start start 2023-10-06T02:29:51.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.483+0000 7f5c90f9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f5c8c153a00 0x7f5c8c153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:51.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.483+0000 7f5c90f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5c8c15de50 0x7f5c8c15a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:51.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.483+0000 7f5c90f9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c8c15af60 con 0x7f5c8c0cea00 2023-10-06T02:29:51.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.483+0000 7f5c90f9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c8c15d010 con 0x7f5c8c153a00 2023-10-06T02:29:51.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.483+0000 7f5c90f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c8c15b240 con 0x7f5c8c15de50 2023-10-06T02:29:51.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.484+0000 7f5c8bfff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f5c8c0cea00 0x7f5c8c0cee80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:54524/0 (socket says 172.21.15.31:54524) 2023-10-06T02:29:51.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.484+0000 7f5c8bfff700 1 -- 172.21.15.31:0/3526353127 learned_addr learned my addr 172.21.15.31:0/3526353127 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:29:51.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.484+0000 7f5c8affd700 1 --2- 172.21.15.31:0/3526353127 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5c8c15de50 0x7f5c8c15a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:51.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.484+0000 7f5c8b7fe700 1 --2- 172.21.15.31:0/3526353127 >> v2:172.21.15.62:3300/0 conn(0x7f5c8c153a00 0x7f5c8c153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:51.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.484+0000 7f5c8a7fc700 1 -- 172.21.15.31:0/3526353127 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1113893390 0 0) 0x7f5c8c15af60 con 0x7f5c8c0cea00 2023-10-06T02:29:51.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.484+0000 7f5c8a7fc700 1 -- 172.21.15.31:0/3526353127 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c74003570 con 0x7f5c8c0cea00 2023-10-06T02:29:51.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.484+0000 7f5c8b7fe700 1 -- 172.21.15.31:0/3526353127 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5c8c15de50 msgr2=0x7f5c8c15a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:51.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.484+0000 7f5c8b7fe700 1 --2- 172.21.15.31:0/3526353127 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5c8c15de50 0x7f5c8c15a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:51.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.484+0000 7f5c8b7fe700 1 -- 172.21.15.31:0/3526353127 >> v1:172.21.15.31:6789/0 conn(0x7f5c8c0cea00 legacy=0x7f5c8c0cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:51.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.484+0000 7f5c8b7fe700 1 -- 172.21.15.31:0/3526353127 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c8c15b520 con 0x7f5c8c153a00 2023-10-06T02:29:51.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.484+0000 7f5c8affd700 1 --2- 172.21.15.31:0/3526353127 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5c8c15de50 0x7f5c8c15a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:29:51.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.485+0000 7f5c8b7fe700 1 --2- 172.21.15.31:0/3526353127 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c8c153a00 0x7f5c8c153dd0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c0098e0 tx=0x7f5c7c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c61a6ba8c8bdb3e6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:51.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.485+0000 7f5c8a7fc700 1 -- 172.21.15.31:0/3526353127 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7c011630 con 0x7f5c8c153a00 2023-10-06T02:29:51.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.485+0000 7f5c8a7fc700 1 -- 172.21.15.31:0/3526353127 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c7c011790 con 0x7f5c8c153a00 2023-10-06T02:29:51.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.485+0000 7f5c8a7fc700 1 -- 172.21.15.31:0/3526353127 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7c011b30 con 0x7f5c8c153a00 2023-10-06T02:29:51.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.485+0000 7f5c90f9d700 1 -- 172.21.15.31:0/3526353127 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c8c153a00 msgr2=0x7f5c8c153dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:51.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.485+0000 7f5c90f9d700 1 --2- 172.21.15.31:0/3526353127 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c8c153a00 0x7f5c8c153dd0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c0098e0 tx=0x7f5c7c0091a0 comp rx=0 tx=0).stop 2023-10-06T02:29:51.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.486+0000 7f5c90f9d700 1 -- 172.21.15.31:0/3526353127 shutdown_connections 2023-10-06T02:29:51.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.486+0000 7f5c90f9d700 1 --2- 172.21.15.31:0/3526353127 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c8c153a00 0x7f5c8c153dd0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:51.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.486+0000 7f5c90f9d700 1 --2- 172.21.15.31:0/3526353127 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5c8c15de50 0x7f5c8c15a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:51.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.486+0000 7f5c90f9d700 1 -- 172.21.15.31:0/3526353127 >> 172.21.15.31:0/3526353127 conn(0x7f5c8c0703a0 msgr2=0x7f5c8c0707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:51.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.486+0000 7f5c90f9d700 1 -- 172.21.15.31:0/3526353127 shutdown_connections 2023-10-06T02:29:51.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.486+0000 7f5c90f9d700 1 -- 172.21.15.31:0/3526353127 wait complete. 2023-10-06T02:29:51.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.487+0000 7f5c90f9d700 1 Processor -- start 2023-10-06T02:29:51.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.487+0000 7f5c90f9d700 1 -- start start 2023-10-06T02:29:51.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.488+0000 7f5c90f9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c8c0cea00 0x7f5c8c153190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:51.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.488+0000 7f5c90f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5c8c153a00 0x7f5c8c0c9d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:51.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.488+0000 7f5c90f9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c8c15de50 0x7f5c8c0b1c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:51.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.488+0000 7f5c90f9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c8c0b58b0 con 0x7f5c8c15de50 2023-10-06T02:29:51.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.488+0000 7f5c90f9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c8c0b49c0 con 0x7f5c8c0cea00 2023-10-06T02:29:51.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.488+0000 7f5c90f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c8c0b4ca0 con 0x7f5c8c153a00 2023-10-06T02:29:51.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.489+0000 7f5c8bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c8c15de50 0x7f5c8c0b1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:51.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.489+0000 7f5c8b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c8c0cea00 0x7f5c8c153190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:51.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.489+0000 7f5c8affd700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5c8c153a00 0x7f5c8c0c9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:51.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.489+0000 7f5c8b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c8c0cea00 0x7f5c8c153190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:55428/0 (socket says 172.21.15.31:55428) 2023-10-06T02:29:51.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.489+0000 7f5c8bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c8c15de50 0x7f5c8c0b1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33988/0 (socket says 172.21.15.31:33988) 2023-10-06T02:29:51.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.489+0000 7f5c8b7fe700 1 -- 172.21.15.31:0/3555063294 learned_addr learned my addr 172.21.15.31:0/3555063294 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:51.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.489+0000 7f5c8b7fe700 1 -- 172.21.15.31:0/3555063294 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5c8c153a00 msgr2=0x7f5c8c0c9d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:51.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.489+0000 7f5c8b7fe700 1 --2- 172.21.15.31:0/3555063294 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5c8c153a00 0x7f5c8c0c9d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:51.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.489+0000 7f5c8b7fe700 1 -- 172.21.15.31:0/3555063294 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c8c15de50 msgr2=0x7f5c8c0b1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:51.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.489+0000 7f5c8b7fe700 1 --2- 172.21.15.31:0/3555063294 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c8c15de50 0x7f5c8c0b1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:51.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.489+0000 7f5c8b7fe700 1 -- 172.21.15.31:0/3555063294 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c80009420 con 0x7f5c8c0cea00 2023-10-06T02:29:51.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.489+0000 7f5c8bfff700 1 --2- 172.21.15.31:0/3555063294 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c8c15de50 0x7f5c8c0b1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:29:51.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.489+0000 7f5c8b7fe700 1 --2- 172.21.15.31:0/3555063294 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c8c0cea00 0x7f5c8c153190 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c06eb00 tx=0x7f5c7c017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:51.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.490+0000 7f5c737fe700 1 -- 172.21.15.31:0/3555063294 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7c011900 con 0x7f5c8c0cea00 2023-10-06T02:29:51.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.490+0000 7f5c737fe700 1 -- 172.21.15.31:0/3555063294 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c7c011a60 con 0x7f5c8c0cea00 2023-10-06T02:29:51.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.490+0000 7f5c737fe700 1 -- 172.21.15.31:0/3555063294 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7c025640 con 0x7f5c8c0cea00 2023-10-06T02:29:51.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.490+0000 7f5c90f9d700 1 -- 172.21.15.31:0/3555063294 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c7c009420 con 0x7f5c8c0cea00 2023-10-06T02:29:51.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.490+0000 7f5c90f9d700 1 -- 172.21.15.31:0/3555063294 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c8c0b5260 con 0x7f5c8c0cea00 2023-10-06T02:29:51.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.491+0000 7f5c90f9d700 1 -- 172.21.15.31:0/3555063294 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c8c04c3c0 con 0x7f5c8c0cea00 2023-10-06T02:29:51.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.492+0000 7f5c737fe700 1 -- 172.21.15.31:0/3555063294 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 21) v1 ==== 85148+0+0 (secure 0 0 0) 0x7f5c7c025a30 con 0x7f5c8c0cea00 2023-10-06T02:29:51.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.493+0000 7f5c737fe700 1 --2- 172.21.15.31:0/3555063294 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5c780673b0 0x7f5c78069860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:51.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.493+0000 7f5c737fe700 1 -- 172.21.15.31:0/3555063294 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f5c7c0868c0 con 0x7f5c8c0cea00 2023-10-06T02:29:51.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.493+0000 7f5c8affd700 1 --2- 172.21.15.31:0/3555063294 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5c780673b0 0x7f5c78069860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:51.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.494+0000 7f5c8affd700 1 --2- 172.21.15.31:0/3555063294 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5c780673b0 0x7f5c78069860 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c0c8610 tx=0x7f5c800091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:51.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.497+0000 7f5c737fe700 1 -- 172.21.15.31:0/3555063294 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5c7c0533b0 con 0x7f5c8c0cea00 2023-10-06T02:29:51.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.735+0000 7f5c90f9d700 1 -- 172.21.15.31:0/3555063294 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000554351", "target": ["mon-mgr", ""]}) v1 -- 0x7f5c8c154650 con 0x7f5c780673b0 2023-10-06T02:29:51.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.791+0000 7f5c737fe700 1 -- 172.21.15.31:0/3555063294 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f5c8c154650 con 0x7f5c780673b0 2023-10-06T02:29:51.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.794+0000 7f5c90f9d700 1 -- 172.21.15.31:0/3555063294 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5c780673b0 msgr2=0x7f5c78069860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:51.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.794+0000 7f5c90f9d700 1 --2- 172.21.15.31:0/3555063294 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5c780673b0 0x7f5c78069860 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c0c8610 tx=0x7f5c800091a0 comp rx=0 tx=0).stop 2023-10-06T02:29:51.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.794+0000 7f5c90f9d700 1 -- 172.21.15.31:0/3555063294 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c8c0cea00 msgr2=0x7f5c8c153190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:51.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.794+0000 7f5c90f9d700 1 --2- 172.21.15.31:0/3555063294 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c8c0cea00 0x7f5c8c153190 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c06eb00 tx=0x7f5c7c017000 comp rx=0 tx=0).stop 2023-10-06T02:29:51.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.794+0000 7f5c90f9d700 1 -- 172.21.15.31:0/3555063294 shutdown_connections 2023-10-06T02:29:51.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.794+0000 7f5c90f9d700 1 --2- 172.21.15.31:0/3555063294 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c8c15de50 0x7f5c8c0b1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:51.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.794+0000 7f5c90f9d700 1 --2- 172.21.15.31:0/3555063294 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c8c0cea00 0x7f5c8c153190 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:51.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.794+0000 7f5c90f9d700 1 --2- 172.21.15.31:0/3555063294 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5c780673b0 0x7f5c78069860 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:51.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.794+0000 7f5c90f9d700 1 --2- 172.21.15.31:0/3555063294 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5c8c153a00 0x7f5c8c0c9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:51.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.794+0000 7f5c90f9d700 1 -- 172.21.15.31:0/3555063294 >> 172.21.15.31:0/3555063294 conn(0x7f5c8c0703a0 msgr2=0x7f5c8c0b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:51.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.794+0000 7f5c90f9d700 1 -- 172.21.15.31:0/3555063294 shutdown_connections 2023-10-06T02:29:51.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:51.794+0000 7f5c90f9d700 1 -- 172.21.15.31:0/3555063294 wait complete. 2023-10-06T02:29:51.800 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:29:51.800+0000 7f6566b80700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:29:51.800 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:29:51.800+0000 7f6566b80700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:29:51.800 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:29:51.800+0000 7f6566b80700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:29:51.800 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:29:51.800+0000 7f6566b80700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:29:51.800 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:29:51.800+0000 7f6566b80700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:29:51.814 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000554352 group_0000000000489015 2023-10-06T02:29:51.818 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:29:51.818+0000 7f6568383700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:29:51.819 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:29:51.818+0000 7f6568383700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:29:51.819 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:29:51.818+0000 7f6568383700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:29:51.819 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:29:51.818+0000 7f6568383700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:29:51.819 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:29:51.818+0000 7f6568383700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:29:52.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.006+0000 7f451dd9d700 1 Processor -- start 2023-10-06T02:29:52.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.006+0000 7f451dd9d700 1 -- start start 2023-10-06T02:29:52.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.007+0000 7f451dd9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f45180c8200 0x7f45180c1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:52.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.008+0000 7f451dd9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45181571b0 con 0x7f45180c2a90 2023-10-06T02:29:52.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.008+0000 7f451dd9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45181579b0 con 0x7f45180c85d0 2023-10-06T02:29:52.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.008+0000 7f451dd9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45180c1390 con 0x7f45180c8200 2023-10-06T02:29:52.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.008+0000 7f4517fff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f45180c2a90 0x7f4518156aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:54530/0 (socket says 172.21.15.31:54530) 2023-10-06T02:29:52.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.008+0000 7f4517fff700 1 -- 172.21.15.31:0/4031448746 learned_addr learned my addr 172.21.15.31:0/4031448746 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:29:52.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.008+0000 7f451cd9b700 1 --2- 172.21.15.31:0/4031448746 >> v2:172.21.15.62:3300/0 conn(0x7f45180c8200 0x7f45180c1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:52.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.008+0000 7f45177fe700 1 -- 172.21.15.31:0/4031448746 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2798385828 0 0) 0x7f4504001110 con 0x7f45180c2a90 2023-10-06T02:29:52.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.008+0000 7f45177fe700 1 -- 172.21.15.31:0/4031448746 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4508003570 con 0x7f45180c2a90 2023-10-06T02:29:52.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.009+0000 7f45177fe700 1 -- 172.21.15.31:0/4031448746 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2778631386 0 0) 0x7f4508003570 con 0x7f45180c2a90 2023-10-06T02:29:52.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.009+0000 7f45177fe700 1 -- 172.21.15.31:0/4031448746 >> v2:172.21.15.62:3300/0 conn(0x7f45180c8200 msgr2=0x7f45180c1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:52.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.009+0000 7f45177fe700 1 --2- 172.21.15.31:0/4031448746 >> v2:172.21.15.62:3300/0 conn(0x7f45180c8200 0x7f45180c1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:52.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.009+0000 7f45177fe700 1 -- 172.21.15.31:0/4031448746 >> v1:172.21.15.62:6789/0 conn(0x7f45180c85d0 legacy=0x7f45180cc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:52.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.009+0000 7f45177fe700 1 -- 172.21.15.31:0/4031448746 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45181573f0 con 0x7f45180c2a90 2023-10-06T02:29:52.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.009+0000 7f45177fe700 1 -- 172.21.15.31:0/4031448746 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7f4504001600 con 0x7f45180c2a90 2023-10-06T02:29:52.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.009+0000 7f45177fe700 1 -- 172.21.15.31:0/4031448746 >> v1:172.21.15.31:6789/0 conn(0x7f45180c2a90 legacy=0x7f4518156aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:52.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.009+0000 7f45177fe700 1 --2- 172.21.15.31:0/4031448746 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4508003ab0 0x7f4508003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:52.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.009+0000 7f45177fe700 1 --2- 172.21.15.31:0/4031448746 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4508004810 0x7f4508004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:52.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.009+0000 7f45177fe700 1 --2- 172.21.15.31:0/4031448746 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4508005210 0x7f450800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:52.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.009+0000 7f45177fe700 1 -- 172.21.15.31:0/4031448746 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45180c1390 con 0x7f4508005210 2023-10-06T02:29:52.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.009+0000 7f45177fe700 1 -- 172.21.15.31:0/4031448746 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f450800c1b0 con 0x7f4508003ab0 2023-10-06T02:29:52.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.009+0000 7f45177fe700 1 -- 172.21.15.31:0/4031448746 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f450800c370 con 0x7f4508004810 2023-10-06T02:29:52.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.009+0000 7f451d59c700 1 --2- 172.21.15.31:0/4031448746 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4508005210 0x7f450800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:52.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.009+0000 7f451cd9b700 1 --2- 172.21.15.31:0/4031448746 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4508003ab0 0x7f4508003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:52.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.009+0000 7f4517fff700 1 --2- 172.21.15.31:0/4031448746 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4508004810 0x7f4508004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:52.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.010+0000 7f451d59c700 1 -- 172.21.15.31:0/4031448746 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4508004810 msgr2=0x7f4508004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:52.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.010+0000 7f451d59c700 1 --2- 172.21.15.31:0/4031448746 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4508004810 0x7f4508004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:52.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.010+0000 7f451d59c700 1 -- 172.21.15.31:0/4031448746 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4508003ab0 msgr2=0x7f4508003ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:52.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.010+0000 7f451d59c700 1 --2- 172.21.15.31:0/4031448746 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4508003ab0 0x7f4508003ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:52.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.010+0000 7f451d59c700 1 -- 172.21.15.31:0/4031448746 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4508003570 con 0x7f4508005210 2023-10-06T02:29:52.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.010+0000 7f451d59c700 1 -- 172.21.15.31:0/4031448746 reap_dead start 2023-10-06T02:29:52.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.010+0000 7f451d59c700 1 --2- 172.21.15.31:0/4031448746 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4508005210 0x7f450800ba60 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f4510001fc0 tx=0x7f4510001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9994f1379c7fcb45 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:52.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.010+0000 7f45177fe700 1 -- 172.21.15.31:0/4031448746 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4510001950 con 0x7f4508005210 2023-10-06T02:29:52.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.010+0000 7f45177fe700 1 -- 172.21.15.31:0/4031448746 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f451000c8b0 con 0x7f4508005210 2023-10-06T02:29:52.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.010+0000 7f45177fe700 1 -- 172.21.15.31:0/4031448746 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f451000cc50 con 0x7f4508005210 2023-10-06T02:29:52.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.010+0000 7f451dd9d700 1 -- 172.21.15.31:0/4031448746 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4508005210 msgr2=0x7f450800ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:52.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.011+0000 7f451dd9d700 1 --2- 172.21.15.31:0/4031448746 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4508005210 0x7f450800ba60 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f4510001fc0 tx=0x7f4510001640 comp rx=0 tx=0).stop 2023-10-06T02:29:52.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.011+0000 7f451dd9d700 1 -- 172.21.15.31:0/4031448746 shutdown_connections 2023-10-06T02:29:52.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.011+0000 7f451dd9d700 1 --2- 172.21.15.31:0/4031448746 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4508005210 0x7f450800ba60 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:52.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.011+0000 7f451dd9d700 1 -- 172.21.15.31:0/4031448746 >> 172.21.15.31:0/4031448746 conn(0x7f451806c6a0 msgr2=0x7f451806caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:52.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.011+0000 7f451dd9d700 1 -- 172.21.15.31:0/4031448746 shutdown_connections 2023-10-06T02:29:52.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.011+0000 7f451dd9d700 1 -- 172.21.15.31:0/4031448746 wait complete. 2023-10-06T02:29:52.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.012+0000 7f451dd9d700 1 Processor -- start 2023-10-06T02:29:52.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.012+0000 7f451dd9d700 1 -- start start 2023-10-06T02:29:52.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.013+0000 7f451dd9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4508005210 0x7f451816e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:52.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.013+0000 7f451dd9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45180c33c0 0x7f451814cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:52.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.013+0000 7f451dd9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f45180c4e00 0x7f45180c5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:52.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.013+0000 7f451dd9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f451814d4f0 con 0x7f4508005210 2023-10-06T02:29:52.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.013+0000 7f451dd9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45180afc90 con 0x7f45180c33c0 2023-10-06T02:29:52.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.013+0000 7f451dd9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45180c3830 con 0x7f45180c4e00 2023-10-06T02:29:52.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.013+0000 7f451cd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4508005210 0x7f451816e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:52.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.014+0000 7f451cd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4508005210 0x7f451816e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34002/0 (socket says 172.21.15.31:34002) 2023-10-06T02:29:52.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.014+0000 7f451cd9b700 1 -- 172.21.15.31:0/1169106252 learned_addr learned my addr 172.21.15.31:0/1169106252 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:52.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.014+0000 7f4517fff700 1 --2- 172.21.15.31:0/1169106252 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45180c33c0 0x7f451814cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:52.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.014+0000 7f451d59c700 1 --2- 172.21.15.31:0/1169106252 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f45180c4e00 0x7f45180c5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:52.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.014+0000 7f451cd9b700 1 -- 172.21.15.31:0/1169106252 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f45180c4e00 msgr2=0x7f45180c5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:52.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.014+0000 7f451cd9b700 1 --2- 172.21.15.31:0/1169106252 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f45180c4e00 0x7f45180c5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:52.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.014+0000 7f451cd9b700 1 -- 172.21.15.31:0/1169106252 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45180c33c0 msgr2=0x7f451814cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:52.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.014+0000 7f451cd9b700 1 --2- 172.21.15.31:0/1169106252 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45180c33c0 0x7f451814cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:52.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.014+0000 7f451cd9b700 1 -- 172.21.15.31:0/1169106252 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f451816ea30 con 0x7f4508005210 2023-10-06T02:29:52.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.014+0000 7f451cd9b700 1 --2- 172.21.15.31:0/1169106252 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4508005210 0x7f451816e3a0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f450c011b10 tx=0x7f450c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:52.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.014+0000 7f4514ff9700 1 -- 172.21.15.31:0/1169106252 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f450c00f990 con 0x7f4508005210 2023-10-06T02:29:52.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.014+0000 7f4514ff9700 1 -- 172.21.15.31:0/1169106252 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f450c00faf0 con 0x7f4508005210 2023-10-06T02:29:52.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.014+0000 7f451dd9d700 1 -- 172.21.15.31:0/1169106252 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f451816ed10 con 0x7f4508005210 2023-10-06T02:29:52.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.014+0000 7f4514ff9700 1 -- 172.21.15.31:0/1169106252 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f450c00fe70 con 0x7f4508005210 2023-10-06T02:29:52.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.015+0000 7f451dd9d700 1 -- 172.21.15.31:0/1169106252 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f451816f2d0 con 0x7f4508005210 2023-10-06T02:29:52.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.015+0000 7f451dd9d700 1 -- 172.21.15.31:0/1169106252 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45180c7450 con 0x7f4508005210 2023-10-06T02:29:52.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.016+0000 7f4514ff9700 1 -- 172.21.15.31:0/1169106252 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) v1 ==== 85148+0+0 (secure 0 0 0) 0x7f450c014650 con 0x7f4508005210 2023-10-06T02:29:52.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.017+0000 7f4514ff9700 1 --2- 172.21.15.31:0/1169106252 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f44fc0673b0 0x7f44fc069860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:52.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.017+0000 7f4514ff9700 1 -- 172.21.15.31:0/1169106252 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f450c018030 con 0x7f4508005210 2023-10-06T02:29:52.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.018+0000 7f4517fff700 1 --2- 172.21.15.31:0/1169106252 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f44fc0673b0 0x7f44fc069860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:52.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.018+0000 7f4517fff700 1 --2- 172.21.15.31:0/1169106252 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f44fc0673b0 0x7f44fc069860 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4504001da0 tx=0x7f4504009be0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:52.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.021+0000 7f4514ff9700 1 -- 172.21.15.31:0/1169106252 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f450c055e30 con 0x7f4508005210 2023-10-06T02:29:52.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.258+0000 7f451dd9d700 1 -- 172.21.15.31:0/1169106252 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000554352", "group_name": "group_0000000000489015", "target": ["mon-mgr", ""]}) v1 -- 0x7f45180afdd0 con 0x7f44fc0673b0 2023-10-06T02:29:52.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.304+0000 7f4514ff9700 1 -- 172.21.15.31:0/1169106252 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f4504009150 con 0x7f44fc0673b0 2023-10-06T02:29:52.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.306+0000 7f451dd9d700 1 -- 172.21.15.31:0/1169106252 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f44fc0673b0 msgr2=0x7f44fc069860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:52.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.306+0000 7f451dd9d700 1 --2- 172.21.15.31:0/1169106252 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f44fc0673b0 0x7f44fc069860 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4504001da0 tx=0x7f4504009be0 comp rx=0 tx=0).stop 2023-10-06T02:29:52.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.306+0000 7f451dd9d700 1 -- 172.21.15.31:0/1169106252 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4508005210 msgr2=0x7f451816e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:52.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.306+0000 7f451dd9d700 1 --2- 172.21.15.31:0/1169106252 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4508005210 0x7f451816e3a0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f450c011b10 tx=0x7f450c0091a0 comp rx=0 tx=0).stop 2023-10-06T02:29:52.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.306+0000 7f451dd9d700 1 -- 172.21.15.31:0/1169106252 shutdown_connections 2023-10-06T02:29:52.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.306+0000 7f451dd9d700 1 --2- 172.21.15.31:0/1169106252 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f44fc0673b0 0x7f44fc069860 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:52.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.306+0000 7f451dd9d700 1 --2- 172.21.15.31:0/1169106252 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f45180c4e00 0x7f45180c5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:52.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.306+0000 7f451dd9d700 1 --2- 172.21.15.31:0/1169106252 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4508005210 0x7f451816e3a0 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:52.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.306+0000 7f451dd9d700 1 --2- 172.21.15.31:0/1169106252 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45180c33c0 0x7f451814cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:52.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.306+0000 7f451dd9d700 1 -- 172.21.15.31:0/1169106252 >> 172.21.15.31:0/1169106252 conn(0x7f451806c6a0 msgr2=0x7f451814dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:52.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.306+0000 7f451dd9d700 1 -- 172.21.15.31:0/1169106252 shutdown_connections 2023-10-06T02:29:52.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.307+0000 7f451dd9d700 1 -- 172.21.15.31:0/1169106252 wait complete. 2023-10-06T02:29:52.325 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000554353 2023-10-06T02:29:52.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.518+0000 7f9c8539d700 1 Processor -- start 2023-10-06T02:29:52.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.518+0000 7f9c8539d700 1 -- start start 2023-10-06T02:29:52.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.519+0000 7f9c8539d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9c80155890 0x7f9c800c7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:52.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.519+0000 7f9c8539d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9c80155c60 0x7f9c8006a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:52.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.519+0000 7f9c8539d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c800c7e70 con 0x7f9c8006de50 2023-10-06T02:29:52.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.519+0000 7f9c8539d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c8006b790 con 0x7f9c80155c60 2023-10-06T02:29:52.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.519+0000 7f9c8539d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8006a8e0 con 0x7f9c80155890 2023-10-06T02:29:52.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.519+0000 7f9c84b9c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f9c8006de50 0x7f9c80159720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:54538/0 (socket says 172.21.15.31:54538) 2023-10-06T02:29:52.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.519+0000 7f9c84b9c700 1 -- 172.21.15.31:0/3555808968 learned_addr learned my addr 172.21.15.31:0/3555808968 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:29:52.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.520+0000 7f9c7ffff700 1 --2- 172.21.15.31:0/3555808968 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9c80155890 0x7f9c800c7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:52.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.520+0000 7f9c7f7fe700 1 --2- 172.21.15.31:0/3555808968 >> v2:172.21.15.62:3300/0 conn(0x7f9c80155c60 0x7f9c8006a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:52.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.520+0000 7f9c7effd700 1 -- 172.21.15.31:0/3555808968 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1534124401 0 0) 0x7f9c800c7e70 con 0x7f9c8006de50 2023-10-06T02:29:52.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.520+0000 7f9c7effd700 1 -- 172.21.15.31:0/3555808968 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c68003570 con 0x7f9c8006de50 2023-10-06T02:29:52.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.520+0000 7f9c7effd700 1 -- 172.21.15.31:0/3555808968 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 391337585 0 0) 0x7f9c68003570 con 0x7f9c8006de50 2023-10-06T02:29:52.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.520+0000 7f9c7effd700 1 -- 172.21.15.31:0/3555808968 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9c80155890 msgr2=0x7f9c800c7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:52.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.520+0000 7f9c7effd700 1 --2- 172.21.15.31:0/3555808968 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9c80155890 0x7f9c800c7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:52.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.520+0000 7f9c7effd700 1 -- 172.21.15.31:0/3555808968 >> v2:172.21.15.62:3300/0 conn(0x7f9c80155c60 msgr2=0x7f9c8006a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:52.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.520+0000 7f9c7effd700 1 --2- 172.21.15.31:0/3555808968 >> v2:172.21.15.62:3300/0 conn(0x7f9c80155c60 0x7f9c8006a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:52.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.520+0000 7f9c7effd700 1 -- 172.21.15.31:0/3555808968 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c8006ab80 con 0x7f9c8006de50 2023-10-06T02:29:52.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.520+0000 7f9c7effd700 1 -- 172.21.15.31:0/3555808968 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7f9c780017f0 con 0x7f9c8006de50 2023-10-06T02:29:52.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.520+0000 7f9c7effd700 1 -- 172.21.15.31:0/3555808968 >> v1:172.21.15.31:6789/0 conn(0x7f9c8006de50 legacy=0x7f9c80159720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:52.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.520+0000 7f9c7effd700 1 --2- 172.21.15.31:0/3555808968 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c68003b10 0x7f9c68003f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:52.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.520+0000 7f9c7effd700 1 --2- 172.21.15.31:0/3555808968 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9c80155890 0x7f9c68004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:52.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.520+0000 7f9c7effd700 1 --2- 172.21.15.31:0/3555808968 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c68004d50 0x7f9c680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:52.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.520+0000 7f9c7effd700 1 -- 172.21.15.31:0/3555808968 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8006b790 con 0x7f9c68004d50 2023-10-06T02:29:52.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.520+0000 7f9c7effd700 1 -- 172.21.15.31:0/3555808968 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8006a8e0 con 0x7f9c68003b10 2023-10-06T02:29:52.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.520+0000 7f9c7effd700 1 -- 172.21.15.31:0/3555808968 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c6800dd50 con 0x7f9c80155890 2023-10-06T02:29:52.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.521+0000 7f9c7ffff700 1 --2- 172.21.15.31:0/3555808968 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c68003b10 0x7f9c68003f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:52.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.521+0000 7f9c7f7fe700 1 --2- 172.21.15.31:0/3555808968 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9c80155890 0x7f9c68004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:52.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.521+0000 7f9c84b9c700 1 --2- 172.21.15.31:0/3555808968 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c68004d50 0x7f9c680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:52.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.521+0000 7f9c84b9c700 1 -- 172.21.15.31:0/3555808968 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9c80155890 msgr2=0x7f9c68004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:52.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.521+0000 7f9c84b9c700 1 --2- 172.21.15.31:0/3555808968 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9c80155890 0x7f9c68004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:52.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.521+0000 7f9c84b9c700 1 -- 172.21.15.31:0/3555808968 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c68003b10 msgr2=0x7f9c68003f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:52.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.521+0000 7f9c84b9c700 1 --2- 172.21.15.31:0/3555808968 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c68003b10 0x7f9c68003f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:52.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.521+0000 7f9c84b9c700 1 -- 172.21.15.31:0/3555808968 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c68003570 con 0x7f9c68004d50 2023-10-06T02:29:52.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.521+0000 7f9c7ffff700 1 --2- 172.21.15.31:0/3555808968 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c68003b10 0x7f9c68003f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:29:52.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.521+0000 7f9c84b9c700 1 --2- 172.21.15.31:0/3555808968 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c68004d50 0x7f9c680051c0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f9c7800a830 tx=0x7f9c8006b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e612a7b68d3a9c63 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:52.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.522+0000 7f9c7effd700 1 -- 172.21.15.31:0/3555808968 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c780070d0 con 0x7f9c68004d50 2023-10-06T02:29:52.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.522+0000 7f9c7effd700 1 -- 172.21.15.31:0/3555808968 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c78007250 con 0x7f9c68004d50 2023-10-06T02:29:52.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.522+0000 7f9c7effd700 1 -- 172.21.15.31:0/3555808968 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c78007610 con 0x7f9c68004d50 2023-10-06T02:29:52.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.522+0000 7f9c8539d700 1 -- 172.21.15.31:0/3555808968 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c68004d50 msgr2=0x7f9c680051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:52.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.522+0000 7f9c8539d700 1 --2- 172.21.15.31:0/3555808968 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c68004d50 0x7f9c680051c0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f9c7800a830 tx=0x7f9c8006b8d0 comp rx=0 tx=0).stop 2023-10-06T02:29:52.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.522+0000 7f9c84b9c700 1 -- 172.21.15.31:0/3555808968 reap_dead start 2023-10-06T02:29:52.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.522+0000 7f9c8539d700 1 -- 172.21.15.31:0/3555808968 shutdown_connections 2023-10-06T02:29:52.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.522+0000 7f9c8539d700 1 -- 172.21.15.31:0/3555808968 >> 172.21.15.31:0/3555808968 conn(0x7f9c8006d110 msgr2=0x7f9c800cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:52.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.522+0000 7f9c8539d700 1 -- 172.21.15.31:0/3555808968 shutdown_connections 2023-10-06T02:29:52.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.522+0000 7f9c8539d700 1 -- 172.21.15.31:0/3555808968 wait complete. 2023-10-06T02:29:52.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.524+0000 7f9c8539d700 1 Processor -- start 2023-10-06T02:29:52.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.524+0000 7f9c8539d700 1 -- start start 2023-10-06T02:29:52.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.525+0000 7f9c8539d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c80152630 0x7f9c800af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:52.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.525+0000 7f9c8539d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c800afe20 0x7f9c800ca2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:52.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.525+0000 7f9c8539d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9c800ca7e0 0x7f9c800c31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:52.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.525+0000 7f9c8539d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c800ae370 con 0x7f9c800afe20 2023-10-06T02:29:52.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.525+0000 7f9c8539d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c80165440 con 0x7f9c80152630 2023-10-06T02:29:52.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.525+0000 7f9c8539d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c800cac50 con 0x7f9c800ca7e0 2023-10-06T02:29:52.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.525+0000 7f9c7f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c800afe20 0x7f9c800ca2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:52.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.525+0000 7f9c7f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c800afe20 0x7f9c800ca2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34026/0 (socket says 172.21.15.31:34026) 2023-10-06T02:29:52.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.525+0000 7f9c7f7fe700 1 -- 172.21.15.31:0/3426414885 learned_addr learned my addr 172.21.15.31:0/3426414885 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:52.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.525+0000 7f9c7f7fe700 1 -- 172.21.15.31:0/3426414885 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9c800ca7e0 msgr2=0x7f9c800c31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:52.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.525+0000 7f9c7f7fe700 1 --2- 172.21.15.31:0/3426414885 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9c800ca7e0 0x7f9c800c31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:52.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.525+0000 7f9c7f7fe700 1 -- 172.21.15.31:0/3426414885 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c80152630 msgr2=0x7f9c800af8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:52.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.525+0000 7f9c7ffff700 1 --2- 172.21.15.31:0/3426414885 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c80152630 0x7f9c800af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:52.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.525+0000 7f9c7f7fe700 1 --2- 172.21.15.31:0/3426414885 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c80152630 0x7f9c800af8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:52.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.525+0000 7f9c7f7fe700 1 -- 172.21.15.31:0/3426414885 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c800c4d50 con 0x7f9c800afe20 2023-10-06T02:29:52.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.525+0000 7f9c7f7fe700 1 --2- 172.21.15.31:0/3426414885 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c800afe20 0x7f9c800ca2a0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f9c74011a40 tx=0x7f9c740091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:52.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.526+0000 7f9c67fff700 1 -- 172.21.15.31:0/3426414885 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c7400f940 con 0x7f9c800afe20 2023-10-06T02:29:52.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.526+0000 7f9c67fff700 1 -- 172.21.15.31:0/3426414885 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c7400faa0 con 0x7f9c800afe20 2023-10-06T02:29:52.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.526+0000 7f9c8539d700 1 -- 172.21.15.31:0/3426414885 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c800b03a0 con 0x7f9c800afe20 2023-10-06T02:29:52.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.526+0000 7f9c67fff700 1 -- 172.21.15.31:0/3426414885 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c7400fe60 con 0x7f9c800afe20 2023-10-06T02:29:52.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.526+0000 7f9c8539d700 1 -- 172.21.15.31:0/3426414885 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c800b0960 con 0x7f9c800afe20 2023-10-06T02:29:52.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.527+0000 7f9c8539d700 1 -- 172.21.15.31:0/3426414885 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c801655b0 con 0x7f9c800afe20 2023-10-06T02:29:52.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.528+0000 7f9c67fff700 1 -- 172.21.15.31:0/3426414885 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) v1 ==== 85148+0+0 (secure 0 0 0) 0x7f9c74014650 con 0x7f9c800afe20 2023-10-06T02:29:52.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.529+0000 7f9c67fff700 1 --2- 172.21.15.31:0/3426414885 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9c6c0673b0 0x7f9c6c069860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:52.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.529+0000 7f9c67fff700 1 -- 172.21.15.31:0/3426414885 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9c74018030 con 0x7f9c800afe20 2023-10-06T02:29:52.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.529+0000 7f9c7ffff700 1 --2- 172.21.15.31:0/3426414885 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9c6c0673b0 0x7f9c6c069860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:52.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.530+0000 7f9c7ffff700 1 --2- 172.21.15.31:0/3426414885 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9c6c0673b0 0x7f9c6c069860 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9c70009720 tx=0x7f9c70008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:52.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.533+0000 7f9c67fff700 1 -- 172.21.15.31:0/3426414885 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9c801655b0 con 0x7f9c800afe20 2023-10-06T02:29:52.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.768+0000 7f9c8539d700 1 -- 172.21.15.31:0/3426414885 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000554353", "target": ["mon-mgr", ""]}) v1 -- 0x7f9c801655b0 con 0x7f9c6c0673b0 2023-10-06T02:29:52.793 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:29:52.793+0000 7f656537d700 -1 mgr.server reply reply (11) Resource temporarily unavailable subvolume 'subvolume_0000000000554353' is not ready for operation rm (use --force to override) 2023-10-06T02:29:52.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:52.792+0000 7f9c67fff700 1 -- 172.21.15.31:0/3426414885 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: -11 subvolume 'subvolume_0000000000554353' is not ready for operation rm (use --force to override)) v1 ==== 102+0+0 (secure 0 0 0) 0x7f9c801655b0 con 0x7f9c6c0673b0 2023-10-06T02:29:52.794 INFO:teuthology.orchestra.run.smithi031.stderr:Error EAGAIN: subvolume 'subvolume_0000000000554353' is not ready for operation rm (use --force to override) 2023-10-06T02:29:52.802 DEBUG:teuthology.orchestra.run:got remote process result: 11 2023-10-06T02:29:52.802 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:29:52.803 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvolume_0000000000554353/.meta) 2023-10-06T02:29:52.866 INFO:teuthology.orchestra.run.smithi031.stdout:[GLOBAL] 2023-10-06T02:29:52.866 INFO:teuthology.orchestra.run.smithi031.stdout:version = 1 2023-10-06T02:29:52.866 INFO:teuthology.orchestra.run.smithi031.stdout:type = clone 2023-10-06T02:29:52.866 INFO:teuthology.orchestra.run.smithi031.stdout:path = /volumes/_nogroup/subvolume_0000000000554353/8137d096-da5f-4840-9c08-3fd3bbefc549 2023-10-06T02:29:52.866 INFO:teuthology.orchestra.run.smithi031.stdout:state = pending 2023-10-06T02:29:52.866 INFO:teuthology.orchestra.run.smithi031.stdout:[source] 2023-10-06T02:29:52.866 INFO:teuthology.orchestra.run.smithi031.stdout:volume = fake 2023-10-06T02:29:52.866 INFO:teuthology.orchestra.run.smithi031.stdout:subvolume = fake 2023-10-06T02:29:52.866 INFO:teuthology.orchestra.run.smithi031.stdout:snapshot = fake 2023-10-06T02:29:52.870 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000554353 --force 2023-10-06T02:29:53.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.062+0000 7f6ba159d700 1 Processor -- start 2023-10-06T02:29:53.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.062+0000 7f6ba159d700 1 -- start start 2023-10-06T02:29:53.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.064+0000 7f6ba159d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6b9c0c8200 0x7f6b9c0c1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:53.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.064+0000 7f6ba159d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6b9c0c2a90 0x7f6b9c156aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:53.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.064+0000 7f6ba159d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b9c1579b0 con 0x7f6b9c0c85d0 2023-10-06T02:29:53.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.064+0000 7f6ba159d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b9c0c1390 con 0x7f6b9c0c8200 2023-10-06T02:29:53.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.064+0000 7f6ba159d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b9c1573f0 con 0x7f6b9c0c2a90 2023-10-06T02:29:53.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.064+0000 7f6b9bfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6b9c0c8200 0x7f6b9c0c1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:53.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.064+0000 7f6b9bfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6b9c0c8200 0x7f6b9c0c1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34036/0 (socket says 172.21.15.31:34036) 2023-10-06T02:29:53.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.064+0000 7f6b9bfff700 1 -- 172.21.15.31:0/307654131 learned_addr learned my addr 172.21.15.31:0/307654131 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:53.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.064+0000 7f6b9b7fe700 1 --2- 172.21.15.31:0/307654131 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6b9c0c2a90 0x7f6b9c156aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:53.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.064+0000 7f6b9bfff700 1 -- 172.21.15.31:0/307654131 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6b9c0c2a90 msgr2=0x7f6b9c156aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:53.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.064+0000 7f6b9bfff700 1 --2- 172.21.15.31:0/307654131 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6b9c0c2a90 0x7f6b9c156aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:53.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.064+0000 7f6b9bfff700 1 -- 172.21.15.31:0/307654131 >> v1:172.21.15.62:6789/0 conn(0x7f6b9c0c85d0 legacy=0x7f6b9c0cc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:53.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.064+0000 7f6b9bfff700 1 -- 172.21.15.31:0/307654131 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b9c1576d0 con 0x7f6b9c0c8200 2023-10-06T02:29:53.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.065+0000 7f6b9bfff700 1 --2- 172.21.15.31:0/307654131 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b9c0c8200 0x7f6b9c0c1820 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c0099b0 tx=0x7f6b8c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d138858866b649e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:53.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.065+0000 7f6b9affd700 1 -- 172.21.15.31:0/307654131 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c00b860 con 0x7f6b9c0c8200 2023-10-06T02:29:53.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.065+0000 7f6b9affd700 1 -- 172.21.15.31:0/307654131 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b8c00b9c0 con 0x7f6b9c0c8200 2023-10-06T02:29:53.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.065+0000 7f6b9affd700 1 -- 172.21.15.31:0/307654131 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c00bd60 con 0x7f6b9c0c8200 2023-10-06T02:29:53.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.065+0000 7f6ba159d700 1 -- 172.21.15.31:0/307654131 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b9c0c8200 msgr2=0x7f6b9c0c1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:53.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.065+0000 7f6ba159d700 1 --2- 172.21.15.31:0/307654131 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b9c0c8200 0x7f6b9c0c1820 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c0099b0 tx=0x7f6b8c009250 comp rx=0 tx=0).stop 2023-10-06T02:29:53.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.066+0000 7f6ba159d700 1 -- 172.21.15.31:0/307654131 shutdown_connections 2023-10-06T02:29:53.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.066+0000 7f6ba159d700 1 --2- 172.21.15.31:0/307654131 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b9c0c8200 0x7f6b9c0c1820 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:53.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.066+0000 7f6ba159d700 1 --2- 172.21.15.31:0/307654131 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6b9c0c2a90 0x7f6b9c156aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:53.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.066+0000 7f6ba159d700 1 -- 172.21.15.31:0/307654131 >> 172.21.15.31:0/307654131 conn(0x7f6b9c06c6a0 msgr2=0x7f6b9c06caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:53.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.066+0000 7f6ba159d700 1 -- 172.21.15.31:0/307654131 shutdown_connections 2023-10-06T02:29:53.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.066+0000 7f6ba159d700 1 -- 172.21.15.31:0/307654131 wait complete. 2023-10-06T02:29:53.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.067+0000 7f6ba159d700 1 Processor -- start 2023-10-06T02:29:53.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.067+0000 7f6ba159d700 1 -- start start 2023-10-06T02:29:53.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.068+0000 7f6ba159d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b9c0c2a90 0x7f6b9c0ae4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:53.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.068+0000 7f6ba159d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b9c0c8200 0x7f6b9c14e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:53.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.068+0000 7f6ba159d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6b9c0c85d0 0x7f6b9c14eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:53.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.068+0000 7f6ba159d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b9c169860 con 0x7f6b9c0c2a90 2023-10-06T02:29:53.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.068+0000 7f6ba159d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b9c16a3b0 con 0x7f6b9c0c8200 2023-10-06T02:29:53.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.068+0000 7f6ba159d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b9c169a80 con 0x7f6b9c0c85d0 2023-10-06T02:29:53.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.068+0000 7f6b9bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b9c0c2a90 0x7f6b9c0ae4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:53.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.068+0000 7f6b9bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b9c0c2a90 0x7f6b9c0ae4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34046/0 (socket says 172.21.15.31:34046) 2023-10-06T02:29:53.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.068+0000 7f6b9bfff700 1 -- 172.21.15.31:0/2197482733 learned_addr learned my addr 172.21.15.31:0/2197482733 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:53.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.069+0000 7f6b9b7fe700 1 --2- 172.21.15.31:0/2197482733 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b9c0c8200 0x7f6b9c14e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:53.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.069+0000 7f6ba0d9c700 1 --2- 172.21.15.31:0/2197482733 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6b9c0c85d0 0x7f6b9c14eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:53.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.069+0000 7f6b9bfff700 1 -- 172.21.15.31:0/2197482733 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6b9c0c85d0 msgr2=0x7f6b9c14eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:53.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.069+0000 7f6b9bfff700 1 --2- 172.21.15.31:0/2197482733 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6b9c0c85d0 0x7f6b9c14eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:53.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.069+0000 7f6b9bfff700 1 -- 172.21.15.31:0/2197482733 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b9c0c8200 msgr2=0x7f6b9c14e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:53.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.069+0000 7f6b9bfff700 1 --2- 172.21.15.31:0/2197482733 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b9c0c8200 0x7f6b9c14e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:53.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.069+0000 7f6b9bfff700 1 -- 172.21.15.31:0/2197482733 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b8c0094f0 con 0x7f6b9c0c2a90 2023-10-06T02:29:53.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.069+0000 7f6b9bfff700 1 --2- 172.21.15.31:0/2197482733 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b9c0c2a90 0x7f6b9c0ae4d0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c0697c0 tx=0x7f6b8c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:53.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.069+0000 7f6b83fff700 1 -- 172.21.15.31:0/2197482733 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c011e40 con 0x7f6b9c0c2a90 2023-10-06T02:29:53.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.069+0000 7f6b83fff700 1 -- 172.21.15.31:0/2197482733 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b8c024440 con 0x7f6b9c0c2a90 2023-10-06T02:29:53.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.070+0000 7f6b83fff700 1 -- 172.21.15.31:0/2197482733 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c024800 con 0x7f6b9c0c2a90 2023-10-06T02:29:53.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.070+0000 7f6ba159d700 1 -- 172.21.15.31:0/2197482733 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b9c169d60 con 0x7f6b9c0c2a90 2023-10-06T02:29:53.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.070+0000 7f6ba159d700 1 -- 172.21.15.31:0/2197482733 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b9c0aea10 con 0x7f6b9c0c2a90 2023-10-06T02:29:53.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.071+0000 7f6ba159d700 1 -- 172.21.15.31:0/2197482733 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b9c04c3c0 con 0x7f6b9c0c2a90 2023-10-06T02:29:53.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.072+0000 7f6b83fff700 1 -- 172.21.15.31:0/2197482733 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) v1 ==== 85148+0+0 (secure 0 0 0) 0x7f6b8c02d440 con 0x7f6b9c0c2a90 2023-10-06T02:29:53.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.072+0000 7f6b83fff700 1 --2- 172.21.15.31:0/2197482733 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6b880673b0 0x7f6b88069860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:53.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.073+0000 7f6b83fff700 1 -- 172.21.15.31:0/2197482733 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6b8c086720 con 0x7f6b9c0c2a90 2023-10-06T02:29:53.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.073+0000 7f6b9b7fe700 1 --2- 172.21.15.31:0/2197482733 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6b880673b0 0x7f6b88069860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:53.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.074+0000 7f6b9b7fe700 1 --2- 172.21.15.31:0/2197482733 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6b880673b0 0x7f6b88069860 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6b90006fd0 tx=0x7f6b90008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:53.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.077+0000 7f6b83fff700 1 -- 172.21.15.31:0/2197482733 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6b8c053270 con 0x7f6b9c0c2a90 2023-10-06T02:29:53.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.313+0000 7f6ba159d700 1 -- 172.21.15.31:0/2197482733 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000554353", "force": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f6b9c0c2280 con 0x7f6b880673b0 2023-10-06T02:29:53.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.346+0000 7f6b83fff700 1 -- 172.21.15.31:0/2197482733 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f6b9c0c2280 con 0x7f6b880673b0 2023-10-06T02:29:53.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.349+0000 7f6ba159d700 1 -- 172.21.15.31:0/2197482733 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6b880673b0 msgr2=0x7f6b88069860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:53.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.349+0000 7f6ba159d700 1 --2- 172.21.15.31:0/2197482733 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6b880673b0 0x7f6b88069860 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6b90006fd0 tx=0x7f6b90008040 comp rx=0 tx=0).stop 2023-10-06T02:29:53.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.349+0000 7f6ba159d700 1 -- 172.21.15.31:0/2197482733 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b9c0c2a90 msgr2=0x7f6b9c0ae4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:53.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.349+0000 7f6ba159d700 1 --2- 172.21.15.31:0/2197482733 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b9c0c2a90 0x7f6b9c0ae4d0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c0697c0 tx=0x7f6b8c017000 comp rx=0 tx=0).stop 2023-10-06T02:29:53.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.349+0000 7f6ba159d700 1 -- 172.21.15.31:0/2197482733 shutdown_connections 2023-10-06T02:29:53.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.349+0000 7f6ba159d700 1 --2- 172.21.15.31:0/2197482733 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6b880673b0 0x7f6b88069860 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:53.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.349+0000 7f6ba159d700 1 --2- 172.21.15.31:0/2197482733 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6b9c0c85d0 0x7f6b9c14eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:53.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.349+0000 7f6ba159d700 1 --2- 172.21.15.31:0/2197482733 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b9c0c2a90 0x7f6b9c0ae4d0 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:53.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.349+0000 7f6ba159d700 1 --2- 172.21.15.31:0/2197482733 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b9c0c8200 0x7f6b9c14e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:53.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.349+0000 7f6ba159d700 1 -- 172.21.15.31:0/2197482733 >> 172.21.15.31:0/2197482733 conn(0x7f6b9c06c6a0 msgr2=0x7f6b9c14fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:53.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.349+0000 7f6ba159d700 1 -- 172.21.15.31:0/2197482733 shutdown_connections 2023-10-06T02:29:53.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.349+0000 7f6ba159d700 1 -- 172.21.15.31:0/2197482733 wait complete. 2023-10-06T02:29:53.370 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs 2023-10-06T02:29:53.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.558+0000 7f4d0ed9d700 1 Processor -- start 2023-10-06T02:29:53.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.559+0000 7f4d0ed9d700 1 -- start start 2023-10-06T02:29:53.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.560+0000 7f4d0ed9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4d0813fe90 0x7f4d0806abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:53.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.560+0000 7f4d0ed9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d0806b290 con 0x7f4d080c7fb0 2023-10-06T02:29:53.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.560+0000 7f4d0ed9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d0806b570 con 0x7f4d080c7be0 2023-10-06T02:29:53.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.560+0000 7f4d0ed9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d0806bf10 con 0x7f4d0813fe90 2023-10-06T02:29:53.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.560+0000 7f4d0d59a700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f4d080c7fb0 0x7f4d0813f780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:54548/0 (socket says 172.21.15.31:54548) 2023-10-06T02:29:53.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.560+0000 7f4d0d59a700 1 -- 172.21.15.31:0/745383381 learned_addr learned my addr 172.21.15.31:0/745383381 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:29:53.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.560+0000 7f4d0e59c700 1 --2- 172.21.15.31:0/745383381 >> v2:172.21.15.31:3300/0 conn(0x7f4d0813fe90 0x7f4d0806abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:53.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.561+0000 7f4d0e59c700 1 -- 172.21.15.31:0/745383381 >> v1:172.21.15.62:6789/0 conn(0x7f4d080c7be0 legacy=0x7f4d0815a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:53.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.561+0000 7f4d0e59c700 1 -- 172.21.15.31:0/745383381 >> v1:172.21.15.31:6789/0 conn(0x7f4d080c7fb0 legacy=0x7f4d0813f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:53.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.561+0000 7f4d0e59c700 1 -- 172.21.15.31:0/745383381 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d0806b850 con 0x7f4d0813fe90 2023-10-06T02:29:53.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.561+0000 7f4d0e59c700 1 --2- 172.21.15.31:0/745383381 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d0813fe90 0x7f4d0806abd0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f4d00009e60 tx=0x7f4d00009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=787c89f58e9a454c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:53.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.561+0000 7f4d0cd99700 1 -- 172.21.15.31:0/745383381 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d0000e070 con 0x7f4d0813fe90 2023-10-06T02:29:53.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.561+0000 7f4d0cd99700 1 -- 172.21.15.31:0/745383381 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d0000bc30 con 0x7f4d0813fe90 2023-10-06T02:29:53.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.561+0000 7f4d0cd99700 1 -- 172.21.15.31:0/745383381 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d00012430 con 0x7f4d0813fe90 2023-10-06T02:29:53.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.562+0000 7f4d0ed9d700 1 -- 172.21.15.31:0/745383381 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d0813fe90 msgr2=0x7f4d0806abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:53.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.562+0000 7f4d0ed9d700 1 --2- 172.21.15.31:0/745383381 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d0813fe90 0x7f4d0806abd0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f4d00009e60 tx=0x7f4d00009780 comp rx=0 tx=0).stop 2023-10-06T02:29:53.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.562+0000 7f4d0ed9d700 1 -- 172.21.15.31:0/745383381 shutdown_connections 2023-10-06T02:29:53.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.562+0000 7f4d0ed9d700 1 --2- 172.21.15.31:0/745383381 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d0813fe90 0x7f4d0806abd0 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:53.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.562+0000 7f4d0ed9d700 1 -- 172.21.15.31:0/745383381 >> 172.21.15.31:0/745383381 conn(0x7f4d0806d110 msgr2=0x7f4d080cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:53.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.562+0000 7f4d0ed9d700 1 -- 172.21.15.31:0/745383381 shutdown_connections 2023-10-06T02:29:53.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.562+0000 7f4d0ed9d700 1 -- 172.21.15.31:0/745383381 wait complete. 2023-10-06T02:29:53.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.563+0000 7f4d0ed9d700 1 Processor -- start 2023-10-06T02:29:53.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.564+0000 7f4d0ed9d700 1 -- start start 2023-10-06T02:29:53.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.564+0000 7f4d0ed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d080c7be0 0x7f4d08155910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:53.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.564+0000 7f4d0ed9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4d080c7fb0 0x7f4d0813baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:53.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.565+0000 7f4d0ed9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d0813fe90 0x7f4d080b12b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:53.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.565+0000 7f4d0ed9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d081353b0 con 0x7f4d080c7be0 2023-10-06T02:29:53.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.565+0000 7f4d0ed9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d081372c0 con 0x7f4d0813fe90 2023-10-06T02:29:53.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.565+0000 7f4d0ed9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d081366b0 con 0x7f4d080c7fb0 2023-10-06T02:29:53.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.565+0000 7f4d0dd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d080c7be0 0x7f4d08155910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:53.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.565+0000 7f4d0dd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d080c7be0 0x7f4d08155910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34056/0 (socket says 172.21.15.31:34056) 2023-10-06T02:29:53.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.565+0000 7f4d0dd9b700 1 -- 172.21.15.31:0/3968276571 learned_addr learned my addr 172.21.15.31:0/3968276571 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:53.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.565+0000 7f4d0d59a700 1 --2- 172.21.15.31:0/3968276571 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4d080c7fb0 0x7f4d0813baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:53.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.565+0000 7f4d0e59c700 1 --2- 172.21.15.31:0/3968276571 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d0813fe90 0x7f4d080b12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:53.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.565+0000 7f4d0dd9b700 1 -- 172.21.15.31:0/3968276571 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4d080c7fb0 msgr2=0x7f4d0813baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:53.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.565+0000 7f4d0dd9b700 1 --2- 172.21.15.31:0/3968276571 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4d080c7fb0 0x7f4d0813baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:53.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.565+0000 7f4d0dd9b700 1 -- 172.21.15.31:0/3968276571 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d0813fe90 msgr2=0x7f4d080b12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:53.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.565+0000 7f4d0dd9b700 1 --2- 172.21.15.31:0/3968276571 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d0813fe90 0x7f4d080b12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:53.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.565+0000 7f4d0dd9b700 1 -- 172.21.15.31:0/3968276571 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d00009a20 con 0x7f4d080c7be0 2023-10-06T02:29:53.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.565+0000 7f4d0dd9b700 1 --2- 172.21.15.31:0/3968276571 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d080c7be0 0x7f4d08155910 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f4d0400c790 tx=0x7f4d04007f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:53.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.566+0000 7f4cfdffb700 1 -- 172.21.15.31:0/3968276571 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d04011590 con 0x7f4d080c7be0 2023-10-06T02:29:53.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.566+0000 7f4cfdffb700 1 -- 172.21.15.31:0/3968276571 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d04011710 con 0x7f4d080c7be0 2023-10-06T02:29:53.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.566+0000 7f4cfdffb700 1 -- 172.21.15.31:0/3968276571 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d04011ad0 con 0x7f4d080c7be0 2023-10-06T02:29:53.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.566+0000 7f4d0ed9d700 1 -- 172.21.15.31:0/3968276571 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d08136990 con 0x7f4d080c7be0 2023-10-06T02:29:53.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.566+0000 7f4d0ed9d700 1 -- 172.21.15.31:0/3968276571 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d08136f50 con 0x7f4d080c7be0 2023-10-06T02:29:53.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.567+0000 7f4d0ed9d700 1 -- 172.21.15.31:0/3968276571 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d0804c3c0 con 0x7f4d080c7be0 2023-10-06T02:29:53.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.568+0000 7f4cfdffb700 1 -- 172.21.15.31:0/3968276571 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) v1 ==== 85148+0+0 (secure 0 0 0) 0x7f4d0401b3b0 con 0x7f4d080c7be0 2023-10-06T02:29:53.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.568+0000 7f4cfdffb700 1 --2- 172.21.15.31:0/3968276571 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f4cec0673b0 0x7f4cec069860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:53.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.569+0000 7f4cfdffb700 1 -- 172.21.15.31:0/3968276571 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f4d04084400 con 0x7f4d080c7be0 2023-10-06T02:29:53.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.569+0000 7f4d0d59a700 1 --2- 172.21.15.31:0/3968276571 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f4cec0673b0 0x7f4cec069860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:53.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.570+0000 7f4d0d59a700 1 --2- 172.21.15.31:0/3968276571 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f4cec0673b0 0x7f4cec069860 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4d080af3d0 tx=0x7f4cf8009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:53.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.572+0000 7f4cfdffb700 1 -- 172.21.15.31:0/3968276571 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4d04050e50 con 0x7f4d080c7be0 2023-10-06T02:29:53.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.807+0000 7f4d0ed9d700 1 -- 172.21.15.31:0/3968276571 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""]}) v1 -- 0x7f4d08137400 con 0x7f4cec0673b0 2023-10-06T02:29:53.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.813+0000 7f4cfdffb700 1 -- 172.21.15.31:0/3968276571 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3 (secure 0 0 0) 0x7f4cf8001cc0 con 0x7f4cec0673b0 2023-10-06T02:29:53.813 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2023-10-06T02:29:53.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.815+0000 7f4d0ed9d700 1 -- 172.21.15.31:0/3968276571 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f4cec0673b0 msgr2=0x7f4cec069860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:53.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.815+0000 7f4d0ed9d700 1 --2- 172.21.15.31:0/3968276571 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f4cec0673b0 0x7f4cec069860 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4d080af3d0 tx=0x7f4cf8009040 comp rx=0 tx=0).stop 2023-10-06T02:29:53.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.815+0000 7f4d0ed9d700 1 -- 172.21.15.31:0/3968276571 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d080c7be0 msgr2=0x7f4d08155910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:53.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.815+0000 7f4d0ed9d700 1 --2- 172.21.15.31:0/3968276571 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d080c7be0 0x7f4d08155910 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f4d0400c790 tx=0x7f4d04007f40 comp rx=0 tx=0).stop 2023-10-06T02:29:53.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.815+0000 7f4d0ed9d700 1 -- 172.21.15.31:0/3968276571 shutdown_connections 2023-10-06T02:29:53.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.815+0000 7f4d0ed9d700 1 --2- 172.21.15.31:0/3968276571 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d0813fe90 0x7f4d080b12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:53.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.815+0000 7f4d0ed9d700 1 --2- 172.21.15.31:0/3968276571 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d080c7be0 0x7f4d08155910 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:53.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.815+0000 7f4d0ed9d700 1 --2- 172.21.15.31:0/3968276571 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f4cec0673b0 0x7f4cec069860 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:53.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.815+0000 7f4d0ed9d700 1 --2- 172.21.15.31:0/3968276571 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4d080c7fb0 0x7f4d0813baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:53.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.815+0000 7f4d0ed9d700 1 -- 172.21.15.31:0/3968276571 >> 172.21.15.31:0/3968276571 conn(0x7f4d0806d110 msgr2=0x7f4d080c9460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:53.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.816+0000 7f4d0ed9d700 1 -- 172.21.15.31:0/3968276571 shutdown_connections 2023-10-06T02:29:53.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:53.816+0000 7f4d0ed9d700 1 -- 172.21.15.31:0/3968276571 wait complete. 2023-10-06T02:29:53.833 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:29:53.833 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2023-10-06T02:29:53.908 INFO:teuthology.orchestra.run.smithi031.stdout:2 2023-10-06T02:29:53.909 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T02:29:53.909 DEBUG:teuthology.orchestra.run.smithi031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T02:29:53.988 DEBUG:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T02:29:54.076 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T02:29:54.076 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T02:29:54.076 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:29:54.076 DEBUG:teuthology.orchestra.run.smithi031:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T02:29:54.187 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi031.front.sepia.ceph.com 2023-10-06T02:29:54.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:29:54.188 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T02:29:54.227 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-06T02:29:54.227 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:29:54.227 DEBUG:teuthology.orchestra.run.smithi062:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-06T02:29:54.310 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi062.front.sepia.ceph.com 2023-10-06T02:29:54.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:29:54.310 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T02:29:54.347 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:29:54.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:54.536+0000 7f79e499d700 1 Processor -- start 2023-10-06T02:29:54.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:54.536+0000 7f79e499d700 1 -- start start 2023-10-06T02:29:54.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:54.538+0000 7f79e499d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f79e0140a70 0x7f79e00c6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:54.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:54.538+0000 7f79e499d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f79e0140e40 0x7f79e00c7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:54.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:54.538+0000 7f79e499d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f79e00c8330 0x7f79e00aeef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:54.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:54.538+0000 7f79e499d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79e00c6d50 con 0x7f79e00c8330 2023-10-06T02:29:54.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:54.538+0000 7f79e499d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79e00c7030 con 0x7f79e0140e40 2023-10-06T02:29:54.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:54.538+0000 7f79e499d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79e00c7310 con 0x7f79e0140a70 2023-10-06T02:29:54.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:54.538+0000 7f79dffff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f79e00c8330 0x7f79e00aeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:54.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:54.538+0000 7f79df7fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f79e0140a70 0x7f79e00c6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:54.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:54.538+0000 7f79dffff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f79e00c8330 0x7f79e00aeef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34058/0 (socket says 172.21.15.31:34058) 2023-10-06T02:29:54.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:54.538+0000 7f79deffd700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f79e0140e40 0x7f79e00c7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:54.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:54.538+0000 7f79dffff700 1 -- 172.21.15.31:0/3816799823 learned_addr learned my addr 172.21.15.31:0/3816799823 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:54.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:54.538+0000 7f79dffff700 1 -- 172.21.15.31:0/3816799823 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f79e0140a70 msgr2=0x7f79e00c6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:54.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:54.538+0000 7f79dffff700 1 --2- 172.21.15.31:0/3816799823 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f79e0140a70 0x7f79e00c6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:54.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:54.539+0000 7f79dffff700 1 -- 172.21.15.31:0/3816799823 >> v2:172.21.15.62:3300/0 conn(0x7f79e0140e40 msgr2=0x7f79e00c7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:54.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:54.539+0000 7f79dffff700 1 --2- 172.21.15.31:0/3816799823 >> v2:172.21.15.62:3300/0 conn(0x7f79e0140e40 0x7f79e00c7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:54.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:54.539+0000 7f79dffff700 1 -- 172.21.15.31:0/3816799823 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79e00c75f0 con 0x7f79e00c8330 2023-10-06T02:29:54.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:54.539+0000 7f79deffd700 1 --2- 172.21.15.31:0/3816799823 >> v2:172.21.15.62:3300/0 conn(0x7f79e0140e40 0x7f79e00c7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:29:54.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:54.539+0000 7f79dffff700 1 --2- 172.21.15.31:0/3816799823 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79e00c8330 0x7f79e00aeef0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f79e00712f0 tx=0x7f79d8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46ed5619f3ab80de server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:54.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:54.539+0000 7f79de7fc700 1 -- 172.21.15.31:0/3816799823 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d800e070 con 0x7f79e00c8330 2023-10-06T02:29:54.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:54.539+0000 7f79de7fc700 1 -- 172.21.15.31:0/3816799823 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79d800b970 con 0x7f79e00c8330 2023-10-06T02:29:54.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:54.539+0000 7f79de7fc700 1 -- 172.21.15.31:0/3816799823 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d80092e0 con 0x7f79e00c8330 2023-10-06T02:29:54.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:54.540+0000 7f79e499d700 1 -- 172.21.15.31:0/3816799823 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79e00c8330 msgr2=0x7f79e00aeef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:54.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:54.540+0000 7f79e499d700 1 --2- 172.21.15.31:0/3816799823 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79e00c8330 0x7f79e00aeef0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f79e00712f0 tx=0x7f79d8009840 comp rx=0 tx=0).stop 2023-10-06T02:29:54.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:54.540+0000 7f79e499d700 1 -- 172.21.15.31:0/3816799823 shutdown_connections 2023-10-06T02:29:54.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:54.540+0000 7f79e499d700 1 --2- 172.21.15.31:0/3816799823 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79e00c8330 0x7f79e00aeef0 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:54.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:54.540+0000 7f79e499d700 1 --2- 172.21.15.31:0/3816799823 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f79e0140a70 0x7f79e00c6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:54.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:54.540+0000 7f79e499d700 1 --2- 172.21.15.31:0/3816799823 >> v2:172.21.15.62:3300/0 conn(0x7f79e0140e40 0x7f79e00c7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:54.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:54.540+0000 7f79e499d700 1 -- 172.21.15.31:0/3816799823 >> 172.21.15.31:0/3816799823 conn(0x7f79e006d170 msgr2=0x7f79e00cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:54.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:54.540+0000 7f79e499d700 1 -- 172.21.15.31:0/3816799823 shutdown_connections 2023-10-06T02:29:54.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:54.540+0000 7f79e499d700 1 -- 172.21.15.31:0/3816799823 wait complete. 2023-10-06T02:29:54.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:54.541+0000 7f79e499d700 1 Processor -- start 2023-10-06T02:29:54.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:54.541+0000 7f79e499d700 1 -- start start 2023-10-06T02:29:54.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:54.542+0000 7f79e499d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f79e00c8330 0x7f79e0156b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:54.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:54.542+0000 7f79e499d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79e0140a70 0x7f79e01582f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:54.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:54.542+0000 7f79e499d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79e0140e40 0x7f79e013e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:54.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:54.542+0000 7f79e499d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79e0159950 con 0x7f79e0140a70 2023-10-06T02:29:54.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:54.542+0000 7f79e499d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79e0158d40 con 0x7f79e0140e40 2023-10-06T02:29:54.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:54.542+0000 7f79e499d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79e0159020 con 0x7f79e00c8330 2023-10-06T02:29:54.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:54.543+0000 7f79deffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79e0140a70 0x7f79e01582f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:54.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:54.543+0000 7f79deffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79e0140a70 0x7f79e01582f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34068/0 (socket says 172.21.15.31:34068) 2023-10-06T02:29:54.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:54.543+0000 7f79deffd700 1 -- 172.21.15.31:0/2076655819 learned_addr learned my addr 172.21.15.31:0/2076655819 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:54.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:54.543+0000 7f79df7fe700 1 --2- 172.21.15.31:0/2076655819 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f79e00c8330 0x7f79e0156b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:54.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:54.543+0000 7f79dffff700 1 --2- 172.21.15.31:0/2076655819 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79e0140e40 0x7f79e013e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:54.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:54.543+0000 7f79deffd700 1 -- 172.21.15.31:0/2076655819 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f79e00c8330 msgr2=0x7f79e0156b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:54.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:54.543+0000 7f79deffd700 1 --2- 172.21.15.31:0/2076655819 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f79e00c8330 0x7f79e0156b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:54.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:54.543+0000 7f79deffd700 1 -- 172.21.15.31:0/2076655819 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79e0140e40 msgr2=0x7f79e013e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:54.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:54.543+0000 7f79deffd700 1 --2- 172.21.15.31:0/2076655819 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79e0140e40 0x7f79e013e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:54.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:54.543+0000 7f79deffd700 1 -- 172.21.15.31:0/2076655819 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79d40093c0 con 0x7f79e0140a70 2023-10-06T02:29:54.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:54.543+0000 7f79deffd700 1 --2- 172.21.15.31:0/2076655819 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79e0140a70 0x7f79e01582f0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f79d400ec80 tx=0x7f79d4009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:54.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:54.543+0000 7f79c77fe700 1 -- 172.21.15.31:0/2076655819 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d4005a00 con 0x7f79e0140a70 2023-10-06T02:29:54.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:54.544+0000 7f79c77fe700 1 -- 172.21.15.31:0/2076655819 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79d4005b60 con 0x7f79e0140a70 2023-10-06T02:29:54.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:54.544+0000 7f79e499d700 1 -- 172.21.15.31:0/2076655819 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79d8009ae0 con 0x7f79e0140a70 2023-10-06T02:29:54.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:54.544+0000 7f79c77fe700 1 -- 172.21.15.31:0/2076655819 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d400c850 con 0x7f79e0140a70 2023-10-06T02:29:54.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:54.544+0000 7f79e499d700 1 -- 172.21.15.31:0/2076655819 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79e01595e0 con 0x7f79e0140a70 2023-10-06T02:29:54.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:54.545+0000 7f79e499d700 1 -- 172.21.15.31:0/2076655819 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79e0140090 con 0x7f79e0140a70 2023-10-06T02:29:54.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:54.546+0000 7f79c77fe700 1 -- 172.21.15.31:0/2076655819 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) v1 ==== 85148+0+0 (secure 0 0 0) 0x7f79d400cbc0 con 0x7f79e0140a70 2023-10-06T02:29:54.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:54.546+0000 7f79c77fe700 1 --2- 172.21.15.31:0/2076655819 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f79cc0673b0 0x7f79cc069860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:54.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:54.547+0000 7f79c77fe700 1 -- 172.21.15.31:0/2076655819 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f79d40852b0 con 0x7f79e0140a70 2023-10-06T02:29:54.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:54.547+0000 7f79df7fe700 1 --2- 172.21.15.31:0/2076655819 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f79cc0673b0 0x7f79cc069860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:54.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:54.547+0000 7f79df7fe700 1 --2- 172.21.15.31:0/2076655819 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f79cc0673b0 0x7f79cc069860 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f79d0009db0 tx=0x7f79d00091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:54.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:54.551+0000 7f79c77fe700 1 -- 172.21.15.31:0/2076655819 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f79d4051d00 con 0x7f79e0140a70 2023-10-06T02:29:54.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:54.608+0000 7f79c77fe700 1 -- 172.21.15.31:0/2076655819 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f79d40520d0 con 0x7f79e0140a70 2023-10-06T02:29:54.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:54.804+0000 7f79e499d700 1 -- 172.21.15.31:0/2076655819 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f79e004c3c0 con 0x7f79e0140a70 2023-10-06T02:29:54.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:54.805+0000 7f79c77fe700 1 -- 172.21.15.31:0/2076655819 <== mon.0 v2:172.21.15.31:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+4590 (secure 0 0 0) 0x7f79d4055790 con 0x7f79e0140a70 2023-10-06T02:29:54.805 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:29:54.806 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":81,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6338,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/1906951645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":1906951645},{"type":"v1","addr":"172.21.15.62:6834","nonce":1906951645}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75},{"gid":6341,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3788411327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3788411327},{"type":"v1","addr":"172.21.15.62:6835","nonce":3788411327}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75},{"gid":7569,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6833/581939695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":581939695},{"type":"v1","addr":"172.21.15.31:6833","nonce":581939695}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:29:35.664920+0000","modified":"2023-10-06T02:29:39.189818+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7575},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7575":{"gid":7575,"name":"c","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/40412586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":40412586},{"type":"v1","addr":"172.21.15.31:6835","nonce":40412586}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-06T02:29:54.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:54.807+0000 7f79e499d700 1 -- 172.21.15.31:0/2076655819 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f79cc0673b0 msgr2=0x7f79cc069860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:54.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:54.807+0000 7f79e499d700 1 --2- 172.21.15.31:0/2076655819 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f79cc0673b0 0x7f79cc069860 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f79d0009db0 tx=0x7f79d00091a0 comp rx=0 tx=0).stop 2023-10-06T02:29:54.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:54.807+0000 7f79e499d700 1 -- 172.21.15.31:0/2076655819 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79e0140a70 msgr2=0x7f79e01582f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:54.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:54.807+0000 7f79e499d700 1 --2- 172.21.15.31:0/2076655819 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79e0140a70 0x7f79e01582f0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f79d400ec80 tx=0x7f79d4009120 comp rx=0 tx=0).stop 2023-10-06T02:29:54.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:54.807+0000 7f79e499d700 1 -- 172.21.15.31:0/2076655819 shutdown_connections 2023-10-06T02:29:54.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:54.807+0000 7f79e499d700 1 --2- 172.21.15.31:0/2076655819 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79e0140e40 0x7f79e013e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:54.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:54.807+0000 7f79e499d700 1 --2- 172.21.15.31:0/2076655819 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f79cc0673b0 0x7f79cc069860 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:54.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:54.807+0000 7f79e499d700 1 --2- 172.21.15.31:0/2076655819 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f79e00c8330 0x7f79e0156b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:54.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:54.807+0000 7f79e499d700 1 --2- 172.21.15.31:0/2076655819 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79e0140a70 0x7f79e01582f0 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:54.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:54.807+0000 7f79e499d700 1 -- 172.21.15.31:0/2076655819 >> 172.21.15.31:0/2076655819 conn(0x7f79e006d170 msgr2=0x7f79e00cb520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:54.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:54.807+0000 7f79e499d700 1 -- 172.21.15.31:0/2076655819 shutdown_connections 2023-10-06T02:29:54.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:54.807+0000 7f79e499d700 1 -- 172.21.15.31:0/2076655819 wait complete. 2023-10-06T02:29:54.809 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 81 2023-10-06T02:29:54.825 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:29:55.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.012+0000 7f2e7f59e700 1 Processor -- start 2023-10-06T02:29:55.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.012+0000 7f2e7f59e700 1 -- start start 2023-10-06T02:29:55.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.013+0000 7f2e7f59e700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2e8006d250 0x7f2e800aefa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:55.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.013+0000 7f2e7f59e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2e800ceac0 0x7f2e800af4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:55.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.013+0000 7f2e7f59e700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e800ac840 con 0x7f2e80157420 2023-10-06T02:29:55.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.013+0000 7f2e7f59e700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e800abf50 con 0x7f2e8006d250 2023-10-06T02:29:55.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.013+0000 7f2e7f59e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e800aea60 con 0x7f2e800ceac0 2023-10-06T02:29:55.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.014+0000 7f2e7dd9b700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2e8006d250 0x7f2e800aefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:55.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.014+0000 7f2e7ed9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2e800ceac0 0x7f2e800af4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:55.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.014+0000 7f2e7dd9b700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2e8006d250 0x7f2e800aefa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:55546/0 (socket says 172.21.15.31:55546) 2023-10-06T02:29:55.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.014+0000 7f2e7dd9b700 1 -- 172.21.15.31:0/4052275331 learned_addr learned my addr 172.21.15.31:0/4052275331 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:55.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.014+0000 7f2e7ed9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2e800ceac0 0x7f2e800af4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:43408/0 (socket says 172.21.15.31:43408) 2023-10-06T02:29:55.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.014+0000 7f2e7e59c700 1 --1- 172.21.15.31:0/4052275331 >> v1:172.21.15.62:6789/0 conn(0x7f2e80157420 0x7f2e800b13b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:57620/0 (socket says 172.21.15.31:57620) 2023-10-06T02:29:55.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.014+0000 7f2e7ed9d700 1 -- 172.21.15.31:0/4052275331 >> v2:172.21.15.62:3300/0 conn(0x7f2e8006d250 msgr2=0x7f2e800aefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:55.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.014+0000 7f2e7ed9d700 1 --2- 172.21.15.31:0/4052275331 >> v2:172.21.15.62:3300/0 conn(0x7f2e8006d250 0x7f2e800aefa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:55.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.014+0000 7f2e7ed9d700 1 -- 172.21.15.31:0/4052275331 >> v1:172.21.15.62:6789/0 conn(0x7f2e80157420 legacy=0x7f2e800b13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:55.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.014+0000 7f2e7ed9d700 1 -- 172.21.15.31:0/4052275331 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e800ac1f0 con 0x7f2e800ceac0 2023-10-06T02:29:55.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.014+0000 7f2e7dd9b700 1 --2- 172.21.15.31:0/4052275331 >> v2:172.21.15.62:3300/0 conn(0x7f2e8006d250 0x7f2e800aefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:29:55.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.014+0000 7f2e7d59a700 1 -- 172.21.15.31:0/4052275331 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 593556990 0 0) 0x7f2e800ac840 con 0x7f2e80157420 2023-10-06T02:29:55.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.015+0000 7f2e7ed9d700 1 --2- 172.21.15.31:0/4052275331 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2e800ceac0 0x7f2e800af4e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2e78009e60 tx=0x7f2e78009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83537a262da5133c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:55.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.015+0000 7f2e7d59a700 1 -- 172.21.15.31:0/4052275331 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e7800e070 con 0x7f2e800ceac0 2023-10-06T02:29:55.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.015+0000 7f2e7d59a700 1 -- 172.21.15.31:0/4052275331 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e7800bb60 con 0x7f2e800ceac0 2023-10-06T02:29:55.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.015+0000 7f2e7d59a700 1 -- 172.21.15.31:0/4052275331 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e78012430 con 0x7f2e800ceac0 2023-10-06T02:29:55.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.016+0000 7f2e7f59e700 1 -- 172.21.15.31:0/4052275331 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2e800ceac0 msgr2=0x7f2e800af4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:55.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.016+0000 7f2e7f59e700 1 --2- 172.21.15.31:0/4052275331 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2e800ceac0 0x7f2e800af4e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2e78009e60 tx=0x7f2e78009780 comp rx=0 tx=0).stop 2023-10-06T02:29:55.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.016+0000 7f2e7f59e700 1 -- 172.21.15.31:0/4052275331 shutdown_connections 2023-10-06T02:29:55.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.016+0000 7f2e7f59e700 1 --2- 172.21.15.31:0/4052275331 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2e800ceac0 0x7f2e800af4e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:55.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.016+0000 7f2e7f59e700 1 --2- 172.21.15.31:0/4052275331 >> v2:172.21.15.62:3300/0 conn(0x7f2e8006d250 0x7f2e800aefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:55.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.016+0000 7f2e7f59e700 1 -- 172.21.15.31:0/4052275331 >> 172.21.15.31:0/4052275331 conn(0x7f2e8006ce80 msgr2=0x7f2e800ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:55.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.016+0000 7f2e7f59e700 1 -- 172.21.15.31:0/4052275331 shutdown_connections 2023-10-06T02:29:55.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.016+0000 7f2e7f59e700 1 -- 172.21.15.31:0/4052275331 wait complete. 2023-10-06T02:29:55.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.017+0000 7f2e7f59e700 1 Processor -- start 2023-10-06T02:29:55.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.017+0000 7f2e7f59e700 1 -- start start 2023-10-06T02:29:55.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.018+0000 7f2e7f59e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e8006d250 0x7f2e80158aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:55.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.018+0000 7f2e7f59e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2e800ceac0 0x7f2e80155e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:55.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.018+0000 7f2e7f59e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e80157420 0x7f2e80156340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:55.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.018+0000 7f2e7f59e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e801577f0 con 0x7f2e80157420 2023-10-06T02:29:55.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.018+0000 7f2e7f59e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e80158610 con 0x7f2e8006d250 2023-10-06T02:29:55.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.018+0000 7f2e7f59e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e801521b0 con 0x7f2e800ceac0 2023-10-06T02:29:55.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.019+0000 7f2e7ed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e80157420 0x7f2e80156340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:55.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.019+0000 7f2e7ed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e80157420 0x7f2e80156340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34080/0 (socket says 172.21.15.31:34080) 2023-10-06T02:29:55.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.019+0000 7f2e7ed9d700 1 -- 172.21.15.31:0/29108635 learned_addr learned my addr 172.21.15.31:0/29108635 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:55.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.019+0000 7f2e7e59c700 1 --2- 172.21.15.31:0/29108635 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e8006d250 0x7f2e80158aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:55.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.019+0000 7f2e7dd9b700 1 --2- 172.21.15.31:0/29108635 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2e800ceac0 0x7f2e80155e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:55.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.019+0000 7f2e7ed9d700 1 -- 172.21.15.31:0/29108635 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2e800ceac0 msgr2=0x7f2e80155e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:55.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.019+0000 7f2e7ed9d700 1 --2- 172.21.15.31:0/29108635 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2e800ceac0 0x7f2e80155e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:55.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.019+0000 7f2e7ed9d700 1 -- 172.21.15.31:0/29108635 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e8006d250 msgr2=0x7f2e80158aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:55.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.019+0000 7f2e7ed9d700 1 --2- 172.21.15.31:0/29108635 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e8006d250 0x7f2e80158aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:55.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.019+0000 7f2e7ed9d700 1 -- 172.21.15.31:0/29108635 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e740094f0 con 0x7f2e80157420 2023-10-06T02:29:55.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.019+0000 7f2e7ed9d700 1 --2- 172.21.15.31:0/29108635 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e80157420 0x7f2e80156340 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f2e7800b6c0 tx=0x7f2e78019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:55.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.020+0000 7f2e6e7fc700 1 -- 172.21.15.31:0/29108635 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e78027470 con 0x7f2e80157420 2023-10-06T02:29:55.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.020+0000 7f2e6e7fc700 1 -- 172.21.15.31:0/29108635 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e780275d0 con 0x7f2e80157420 2023-10-06T02:29:55.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.020+0000 7f2e6e7fc700 1 -- 172.21.15.31:0/29108635 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e78027990 con 0x7f2e80157420 2023-10-06T02:29:55.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.020+0000 7f2e7f59e700 1 -- 172.21.15.31:0/29108635 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e78009a20 con 0x7f2e80157420 2023-10-06T02:29:55.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.020+0000 7f2e7f59e700 1 -- 172.21.15.31:0/29108635 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e80157ce0 con 0x7f2e80157420 2023-10-06T02:29:55.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.021+0000 7f2e7f59e700 1 -- 172.21.15.31:0/29108635 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e801587c0 con 0x7f2e80157420 2023-10-06T02:29:55.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.022+0000 7f2e6e7fc700 1 -- 172.21.15.31:0/29108635 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f2e78024070 con 0x7f2e80157420 2023-10-06T02:29:55.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.023+0000 7f2e6e7fc700 1 --2- 172.21.15.31:0/29108635 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f2e600674d0 0x7f2e60069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:55.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.023+0000 7f2e6e7fc700 1 -- 172.21.15.31:0/29108635 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6114+0+0 (secure 0 0 0) 0x7f2e78087b10 con 0x7f2e80157420 2023-10-06T02:29:55.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.023+0000 7f2e7e59c700 1 --2- 172.21.15.31:0/29108635 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f2e600674d0 0x7f2e60069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:55.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.024+0000 7f2e7e59c700 1 --2- 172.21.15.31:0/29108635 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f2e600674d0 0x7f2e60069980 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2e700011e0 tx=0x7f2e70007bd0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:55.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.027+0000 7f2e6e7fc700 1 -- 172.21.15.31:0/29108635 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2e801587c0 con 0x7f2e80157420 2023-10-06T02:29:55.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.248+0000 7f2e7f59e700 1 -- 172.21.15.31:0/29108635 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2e80158780 con 0x7f2e80157420 2023-10-06T02:29:55.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.249+0000 7f2e6e7fc700 1 -- 172.21.15.31:0/29108635 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) v1 ==== 74+0+16943 (secure 0 0 0) 0x7f2e78054120 con 0x7f2e80157420 2023-10-06T02:29:55.249 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:29:55.250 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":121,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:29:54.592304+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:29:33.769454+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-10-06T02:29:34.643236+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":121,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":20,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":20,"snaps":[{"begin":2,"length":4}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:29:55.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.251+0000 7f2e7f59e700 1 -- 172.21.15.31:0/29108635 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f2e600674d0 msgr2=0x7f2e60069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:55.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.251+0000 7f2e7f59e700 1 --2- 172.21.15.31:0/29108635 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f2e600674d0 0x7f2e60069980 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2e700011e0 tx=0x7f2e70007bd0 comp rx=0 tx=0).stop 2023-10-06T02:29:55.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.251+0000 7f2e7f59e700 1 -- 172.21.15.31:0/29108635 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e80157420 msgr2=0x7f2e80156340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:55.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.251+0000 7f2e7f59e700 1 --2- 172.21.15.31:0/29108635 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e80157420 0x7f2e80156340 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f2e7800b6c0 tx=0x7f2e78019000 comp rx=0 tx=0).stop 2023-10-06T02:29:55.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.251+0000 7f2e7f59e700 1 -- 172.21.15.31:0/29108635 shutdown_connections 2023-10-06T02:29:55.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.251+0000 7f2e7f59e700 1 --2- 172.21.15.31:0/29108635 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e80157420 0x7f2e80156340 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:55.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.251+0000 7f2e7f59e700 1 --2- 172.21.15.31:0/29108635 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e8006d250 0x7f2e80158aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:55.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.251+0000 7f2e7f59e700 1 --2- 172.21.15.31:0/29108635 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f2e600674d0 0x7f2e60069980 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:55.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.251+0000 7f2e7f59e700 1 --2- 172.21.15.31:0/29108635 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2e800ceac0 0x7f2e80155e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:55.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.251+0000 7f2e7f59e700 1 -- 172.21.15.31:0/29108635 >> 172.21.15.31:0/29108635 conn(0x7f2e8006ce80 msgr2=0x7f2e8015b180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:55.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.251+0000 7f2e7f59e700 1 -- 172.21.15.31:0/29108635 shutdown_connections 2023-10-06T02:29:55.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.251+0000 7f2e7f59e700 1 -- 172.21.15.31:0/29108635 wait complete. 2023-10-06T02:29:55.269 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T02:22:53.183425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T02:23:00.046967+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 19, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T02:29:33.769454+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '119', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 20, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T02:29:34.643236+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 5, 'snap_epoch': 121, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T02:29:55.270 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T02:29:55.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.459+0000 7f5cb89fe700 1 Processor -- start 2023-10-06T02:29:55.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.459+0000 7f5cb89fe700 1 -- start start 2023-10-06T02:29:55.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.461+0000 7f5cb89fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5cb40c7be0 0x7f5cb406abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:55.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.461+0000 7f5cb89fe700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cb406b290 con 0x7f5cb40c7fb0 2023-10-06T02:29:55.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.461+0000 7f5cb89fe700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cb406b570 con 0x7f5cb413f780 2023-10-06T02:29:55.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.461+0000 7f5cb89fe700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb406bf10 con 0x7f5cb40c7be0 2023-10-06T02:29:55.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.461+0000 7f5cb259c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f5cb40c7fb0 0x7f5cb415a990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:54560/0 (socket says 172.21.15.31:54560) 2023-10-06T02:29:55.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.461+0000 7f5cb259c700 1 -- 172.21.15.31:0/1640417509 learned_addr learned my addr 172.21.15.31:0/1640417509 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:29:55.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.461+0000 7f5cb2d9d700 1 --2- 172.21.15.31:0/1640417509 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5cb40c7be0 0x7f5cb406abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:55.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.461+0000 7f5cb1d9b700 1 -- 172.21.15.31:0/1640417509 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1093325142 0 0) 0x7f5cb406b290 con 0x7f5cb40c7fb0 2023-10-06T02:29:55.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.461+0000 7f5cb1d9b700 1 -- 172.21.15.31:0/1640417509 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c9c003570 con 0x7f5cb40c7fb0 2023-10-06T02:29:55.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.462+0000 7f5cb1d9b700 1 -- 172.21.15.31:0/1640417509 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 601272339 0 0) 0x7f5cb406b570 con 0x7f5cb413f780 2023-10-06T02:29:55.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.462+0000 7f5cb1d9b700 1 -- 172.21.15.31:0/1640417509 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5cb406b290 con 0x7f5cb413f780 2023-10-06T02:29:55.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.462+0000 7f5cb2d9d700 1 -- 172.21.15.31:0/1640417509 >> v1:172.21.15.62:6789/0 conn(0x7f5cb413f780 legacy=0x7f5cb40c9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:55.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.462+0000 7f5cb2d9d700 1 -- 172.21.15.31:0/1640417509 >> v1:172.21.15.31:6789/0 conn(0x7f5cb40c7fb0 legacy=0x7f5cb415a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:55.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.462+0000 7f5cb1d9b700 1 -- 172.21.15.31:0/1640417509 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 454299453 0 0) 0x7f5c9c003570 con 0x7f5cb40c7fb0 2023-10-06T02:29:55.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.462+0000 7f5cb2d9d700 1 -- 172.21.15.31:0/1640417509 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cb406b850 con 0x7f5cb40c7be0 2023-10-06T02:29:55.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.462+0000 7f5cb2d9d700 1 --2- 172.21.15.31:0/1640417509 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5cb40c7be0 0x7f5cb406abd0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5ca4009910 tx=0x7f5ca40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c247d16b68efa1b3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:55.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.462+0000 7f5cb1d9b700 1 -- 172.21.15.31:0/1640417509 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ca8001470 con 0x7f5cb40c7be0 2023-10-06T02:29:55.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.462+0000 7f5cb1d9b700 1 -- 172.21.15.31:0/1640417509 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ca4011420 con 0x7f5cb40c7be0 2023-10-06T02:29:55.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.462+0000 7f5cb1d9b700 1 -- 172.21.15.31:0/1640417509 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ca40117a0 con 0x7f5cb40c7be0 2023-10-06T02:29:55.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.463+0000 7f5cb89fe700 1 -- 172.21.15.31:0/1640417509 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5cb40c7be0 msgr2=0x7f5cb406abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:55.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.463+0000 7f5cb89fe700 1 --2- 172.21.15.31:0/1640417509 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5cb40c7be0 0x7f5cb406abd0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5ca4009910 tx=0x7f5ca40091a0 comp rx=0 tx=0).stop 2023-10-06T02:29:55.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.463+0000 7f5cb89fe700 1 -- 172.21.15.31:0/1640417509 shutdown_connections 2023-10-06T02:29:55.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.463+0000 7f5cb89fe700 1 --2- 172.21.15.31:0/1640417509 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5cb40c7be0 0x7f5cb406abd0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:55.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.463+0000 7f5cb89fe700 1 -- 172.21.15.31:0/1640417509 >> 172.21.15.31:0/1640417509 conn(0x7f5cb406d110 msgr2=0x7f5cb40cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:55.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.463+0000 7f5cb89fe700 1 -- 172.21.15.31:0/1640417509 shutdown_connections 2023-10-06T02:29:55.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.463+0000 7f5cb89fe700 1 -- 172.21.15.31:0/1640417509 wait complete. 2023-10-06T02:29:55.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.464+0000 7f5cb89fe700 1 Processor -- start 2023-10-06T02:29:55.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.465+0000 7f5cb89fe700 1 -- start start 2023-10-06T02:29:55.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.465+0000 7f5cb89fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5cb40c7be0 0x7f5cb413ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:55.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.465+0000 7f5cb89fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5cb40c7fb0 0x7f5cb4133140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:55.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.465+0000 7f5cb89fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5cb413f780 0x7f5cb40c4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:55.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.465+0000 7f5cb89fe700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb4156bb0 con 0x7f5cb40c7fb0 2023-10-06T02:29:55.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.465+0000 7f5cb89fe700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb4156e90 con 0x7f5cb40c7be0 2023-10-06T02:29:55.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.465+0000 7f5cb89fe700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb4157170 con 0x7f5cb413f780 2023-10-06T02:29:55.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.466+0000 7f5cb259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5cb40c7fb0 0x7f5cb4133140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:55.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.466+0000 7f5cb259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5cb40c7fb0 0x7f5cb4133140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34094/0 (socket says 172.21.15.31:34094) 2023-10-06T02:29:55.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.466+0000 7f5cb259c700 1 -- 172.21.15.31:0/285790490 learned_addr learned my addr 172.21.15.31:0/285790490 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:55.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.466+0000 7f5cb2d9d700 1 --2- 172.21.15.31:0/285790490 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5cb40c7be0 0x7f5cb413ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:55.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.466+0000 7f5cb259c700 1 -- 172.21.15.31:0/285790490 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5cb413f780 msgr2=0x7f5cb40c4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:55.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.466+0000 7f5cb359e700 1 --2- 172.21.15.31:0/285790490 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5cb413f780 0x7f5cb40c4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:55.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.466+0000 7f5cb259c700 1 --2- 172.21.15.31:0/285790490 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5cb413f780 0x7f5cb40c4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:55.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.466+0000 7f5cb259c700 1 -- 172.21.15.31:0/285790490 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5cb40c7be0 msgr2=0x7f5cb413ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:55.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.466+0000 7f5cb259c700 1 --2- 172.21.15.31:0/285790490 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5cb40c7be0 0x7f5cb413ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:55.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.466+0000 7f5cb259c700 1 -- 172.21.15.31:0/285790490 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ca4009420 con 0x7f5cb40c7fb0 2023-10-06T02:29:55.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.466+0000 7f5cb2d9d700 1 --2- 172.21.15.31:0/285790490 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5cb40c7be0 0x7f5cb413ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:29:55.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.466+0000 7f5cb359e700 1 --2- 172.21.15.31:0/285790490 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5cb413f780 0x7f5cb40c4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:29:55.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.466+0000 7f5cb259c700 1 --2- 172.21.15.31:0/285790490 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5cb40c7fb0 0x7f5cb4133140 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f5ca8001ed0 tx=0x7f5ca8001a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:55.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.467+0000 7f5c9affd700 1 -- 172.21.15.31:0/285790490 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ca800ce40 con 0x7f5cb40c7fb0 2023-10-06T02:29:55.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.467+0000 7f5c9affd700 1 -- 172.21.15.31:0/285790490 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ca8005b30 con 0x7f5cb40c7fb0 2023-10-06T02:29:55.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.467+0000 7f5c9affd700 1 -- 172.21.15.31:0/285790490 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ca80070f0 con 0x7f5cb40c7fb0 2023-10-06T02:29:55.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.467+0000 7f5cb89fe700 1 -- 172.21.15.31:0/285790490 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cb4157450 con 0x7f5cb40c7fb0 2023-10-06T02:29:55.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.467+0000 7f5cb89fe700 1 -- 172.21.15.31:0/285790490 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cb4158f80 con 0x7f5cb40c7fb0 2023-10-06T02:29:55.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.468+0000 7f5cb89fe700 1 -- 172.21.15.31:0/285790490 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cb4157670 con 0x7f5cb40c7fb0 2023-10-06T02:29:55.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.469+0000 7f5c9affd700 1 -- 172.21.15.31:0/285790490 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f5ca8007460 con 0x7f5cb40c7fb0 2023-10-06T02:29:55.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.470+0000 7f5c9affd700 1 --2- 172.21.15.31:0/285790490 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5ca00674d0 0x7f5ca0069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:55.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.470+0000 7f5c9affd700 1 -- 172.21.15.31:0/285790490 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6114+0+0 (secure 0 0 0) 0x7f5ca80847a0 con 0x7f5cb40c7fb0 2023-10-06T02:29:55.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.470+0000 7f5cb2d9d700 1 --2- 172.21.15.31:0/285790490 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5ca00674d0 0x7f5ca0069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:55.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.471+0000 7f5cb2d9d700 1 --2- 172.21.15.31:0/285790490 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5ca00674d0 0x7f5ca0069980 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5cb40c9960 tx=0x7f5ca4016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:55.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.474+0000 7f5c9affd700 1 -- 172.21.15.31:0/285790490 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5ca804b5a0 con 0x7f5cb40c7fb0 2023-10-06T02:29:55.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.675+0000 7f5cb89fe700 1 -- 172.21.15.31:0/285790490 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5cb4156cf0 con 0x7f5cb40c7fb0 2023-10-06T02:29:55.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.676+0000 7f5c9affd700 1 -- 172.21.15.31:0/285790490 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v121) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5cb4156cf0 con 0x7f5cb40c7fb0 2023-10-06T02:29:55.677 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-10-06T02:29:55.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.678+0000 7f5cb89fe700 1 -- 172.21.15.31:0/285790490 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5ca00674d0 msgr2=0x7f5ca0069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:55.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.678+0000 7f5cb89fe700 1 --2- 172.21.15.31:0/285790490 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5ca00674d0 0x7f5ca0069980 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5cb40c9960 tx=0x7f5ca4016000 comp rx=0 tx=0).stop 2023-10-06T02:29:55.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.678+0000 7f5cb89fe700 1 -- 172.21.15.31:0/285790490 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5cb40c7fb0 msgr2=0x7f5cb4133140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:55.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.678+0000 7f5cb89fe700 1 --2- 172.21.15.31:0/285790490 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5cb40c7fb0 0x7f5cb4133140 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f5ca8001ed0 tx=0x7f5ca8001a20 comp rx=0 tx=0).stop 2023-10-06T02:29:55.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.678+0000 7f5cb89fe700 1 -- 172.21.15.31:0/285790490 shutdown_connections 2023-10-06T02:29:55.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.678+0000 7f5cb89fe700 1 --2- 172.21.15.31:0/285790490 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5ca00674d0 0x7f5ca0069980 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:55.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.678+0000 7f5cb89fe700 1 --2- 172.21.15.31:0/285790490 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5cb413f780 0x7f5cb40c4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:55.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.678+0000 7f5cb89fe700 1 --2- 172.21.15.31:0/285790490 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5cb40c7be0 0x7f5cb413ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:55.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.678+0000 7f5cb89fe700 1 --2- 172.21.15.31:0/285790490 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5cb40c7fb0 0x7f5cb4133140 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:55.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.678+0000 7f5cb89fe700 1 -- 172.21.15.31:0/285790490 >> 172.21.15.31:0/285790490 conn(0x7f5cb406d110 msgr2=0x7f5cb40cb220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:55.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.678+0000 7f5cb89fe700 1 -- 172.21.15.31:0/285790490 shutdown_connections 2023-10-06T02:29:55.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.678+0000 7f5cb89fe700 1 -- 172.21.15.31:0/285790490 wait complete. 2023-10-06T02:29:55.696 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T02:29:55.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.886+0000 7f48cdd9d700 1 Processor -- start 2023-10-06T02:29:55.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.886+0000 7f48cdd9d700 1 -- start start 2023-10-06T02:29:55.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.887+0000 7f48cdd9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f48c80cf110 0x7f48c815de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:55.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.887+0000 7f48cdd9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48c815ac50 con 0x7f48c8153a00 2023-10-06T02:29:55.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.887+0000 7f48cdd9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48c815d010 con 0x7f48c8153dd0 2023-10-06T02:29:55.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.887+0000 7f48cdd9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48c815af60 con 0x7f48c80cf110 2023-10-06T02:29:55.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.888+0000 7f48ccd9b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f48c8153a00 0x7f48c815a540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:54564/0 (socket says 172.21.15.31:54564) 2023-10-06T02:29:55.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.888+0000 7f48ccd9b700 1 -- 172.21.15.31:0/2496305586 learned_addr learned my addr 172.21.15.31:0/2496305586 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:29:55.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.888+0000 7f48cd59c700 1 --2- 172.21.15.31:0/2496305586 >> v2:172.21.15.62:3300/0 conn(0x7f48c80cf110 0x7f48c815de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:55.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.888+0000 7f48c77fe700 1 -- 172.21.15.31:0/2496305586 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2301570152 0 0) 0x7f48c815ac50 con 0x7f48c8153a00 2023-10-06T02:29:55.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.888+0000 7f48c77fe700 1 -- 172.21.15.31:0/2496305586 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48b8003570 con 0x7f48c8153a00 2023-10-06T02:29:55.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.888+0000 7f48c77fe700 1 -- 172.21.15.31:0/2496305586 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2518654515 0 0) 0x7f48b8003570 con 0x7f48c8153a00 2023-10-06T02:29:55.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.888+0000 7f48c77fe700 1 -- 172.21.15.31:0/2496305586 >> v2:172.21.15.62:3300/0 conn(0x7f48c80cf110 msgr2=0x7f48c815de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:55.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.888+0000 7f48c77fe700 1 --2- 172.21.15.31:0/2496305586 >> v2:172.21.15.62:3300/0 conn(0x7f48c80cf110 0x7f48c815de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:55.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.888+0000 7f48c77fe700 1 -- 172.21.15.31:0/2496305586 >> v1:172.21.15.62:6789/0 conn(0x7f48c8153dd0 legacy=0x7f48c80cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:55.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.888+0000 7f48c77fe700 1 -- 172.21.15.31:0/2496305586 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48c815b240 con 0x7f48c8153a00 2023-10-06T02:29:55.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.888+0000 7f48c77fe700 1 -- 172.21.15.31:0/2496305586 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7f48bc001470 con 0x7f48c8153a00 2023-10-06T02:29:55.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.888+0000 7f48cd59c700 1 --2- 172.21.15.31:0/2496305586 >> v2:172.21.15.62:3300/0 conn(0x7f48c80cf110 0x7f48c815de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:29:55.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.888+0000 7f48c77fe700 1 -- 172.21.15.31:0/2496305586 >> v1:172.21.15.31:6789/0 conn(0x7f48c8153a00 legacy=0x7f48c815a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:55.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.889+0000 7f48c77fe700 1 --2- 172.21.15.31:0/2496305586 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48b8003a60 0x7f48b8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:55.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.889+0000 7f48c77fe700 1 --2- 172.21.15.31:0/2496305586 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48b8004810 0x7f48b8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:55.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.889+0000 7f48c77fe700 1 --2- 172.21.15.31:0/2496305586 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f48b80051c0 0x7f48b800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:55.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.889+0000 7f48c77fe700 1 -- 172.21.15.31:0/2496305586 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c815af60 con 0x7f48b8004810 2023-10-06T02:29:55.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.889+0000 7f48c77fe700 1 -- 172.21.15.31:0/2496305586 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b800c160 con 0x7f48b8003a60 2023-10-06T02:29:55.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.889+0000 7f48c77fe700 1 -- 172.21.15.31:0/2496305586 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48b800c320 con 0x7f48b80051c0 2023-10-06T02:29:55.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.889+0000 7f48c7fff700 1 --2- 172.21.15.31:0/2496305586 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48b8004810 0x7f48b8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:55.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.889+0000 7f48ccd9b700 1 --2- 172.21.15.31:0/2496305586 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48b8003a60 0x7f48b8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:55.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.889+0000 7f48cd59c700 1 --2- 172.21.15.31:0/2496305586 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f48b80051c0 0x7f48b800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:55.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.889+0000 7f48c7fff700 1 -- 172.21.15.31:0/2496305586 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f48b80051c0 msgr2=0x7f48b800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:55.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.889+0000 7f48c7fff700 1 --2- 172.21.15.31:0/2496305586 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f48b80051c0 0x7f48b800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:55.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.889+0000 7f48c7fff700 1 -- 172.21.15.31:0/2496305586 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48b8003a60 msgr2=0x7f48b8003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:55.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.889+0000 7f48c7fff700 1 --2- 172.21.15.31:0/2496305586 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48b8003a60 0x7f48b8003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:55.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.889+0000 7f48c7fff700 1 -- 172.21.15.31:0/2496305586 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48b8003570 con 0x7f48b8004810 2023-10-06T02:29:55.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.889+0000 7f48cd59c700 1 --2- 172.21.15.31:0/2496305586 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f48b80051c0 0x7f48b800ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:29:55.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.889+0000 7f48cd59c700 1 -- 172.21.15.31:0/2496305586 reap_dead start 2023-10-06T02:29:55.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.890+0000 7f48c7fff700 1 --2- 172.21.15.31:0/2496305586 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48b8004810 0x7f48b8004c80 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f48b40018b0 tx=0x7f48b4007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e234ebb4f7ed7b7d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:55.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.890+0000 7f48c77fe700 1 -- 172.21.15.31:0/2496305586 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48b4012420 con 0x7f48b8004810 2023-10-06T02:29:55.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.890+0000 7f48c77fe700 1 -- 172.21.15.31:0/2496305586 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48b40125a0 con 0x7f48b8004810 2023-10-06T02:29:55.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.890+0000 7f48c77fe700 1 -- 172.21.15.31:0/2496305586 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48b4012960 con 0x7f48b8004810 2023-10-06T02:29:55.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.890+0000 7f48cdd9d700 1 -- 172.21.15.31:0/2496305586 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48b8004810 msgr2=0x7f48b8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:55.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.890+0000 7f48cdd9d700 1 --2- 172.21.15.31:0/2496305586 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48b8004810 0x7f48b8004c80 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f48b40018b0 tx=0x7f48b4007d30 comp rx=0 tx=0).stop 2023-10-06T02:29:55.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.890+0000 7f48cdd9d700 1 -- 172.21.15.31:0/2496305586 shutdown_connections 2023-10-06T02:29:55.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.890+0000 7f48cdd9d700 1 --2- 172.21.15.31:0/2496305586 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48b8004810 0x7f48b8004c80 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:55.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.890+0000 7f48cdd9d700 1 -- 172.21.15.31:0/2496305586 >> 172.21.15.31:0/2496305586 conn(0x7f48c80703a0 msgr2=0x7f48c80707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:55.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.891+0000 7f48cdd9d700 1 -- 172.21.15.31:0/2496305586 shutdown_connections 2023-10-06T02:29:55.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.891+0000 7f48cdd9d700 1 -- 172.21.15.31:0/2496305586 wait complete. 2023-10-06T02:29:55.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.892+0000 7f48cdd9d700 1 Processor -- start 2023-10-06T02:29:55.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.892+0000 7f48cdd9d700 1 -- start start 2023-10-06T02:29:55.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.893+0000 7f48cdd9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48b8004810 0x7f48c80c9e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:55.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.893+0000 7f48cdd9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48c80c75e0 0x7f48c80c7a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:55.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.893+0000 7f48cdd9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f48c80d0410 0x7f48c80d0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:55.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.893+0000 7f48cdd9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c80c81c0 con 0x7f48b8004810 2023-10-06T02:29:55.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.893+0000 7f48cdd9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c80c7f90 con 0x7f48c80c75e0 2023-10-06T02:29:55.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.893+0000 7f48cdd9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48c80d0dc0 con 0x7f48c80d0410 2023-10-06T02:29:55.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.893+0000 7f48ccd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48b8004810 0x7f48c80c9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:55.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.893+0000 7f48ccd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48b8004810 0x7f48c80c9e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34110/0 (socket says 172.21.15.31:34110) 2023-10-06T02:29:55.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.893+0000 7f48ccd9b700 1 -- 172.21.15.31:0/633792623 learned_addr learned my addr 172.21.15.31:0/633792623 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:55.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.893+0000 7f48cd59c700 1 --2- 172.21.15.31:0/633792623 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f48c80d0410 0x7f48c80d0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:55.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.893+0000 7f48c7fff700 1 --2- 172.21.15.31:0/633792623 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48c80c75e0 0x7f48c80c7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:55.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.894+0000 7f48ccd9b700 1 -- 172.21.15.31:0/633792623 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f48c80d0410 msgr2=0x7f48c80d0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:55.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.894+0000 7f48ccd9b700 1 --2- 172.21.15.31:0/633792623 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f48c80d0410 0x7f48c80d0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:55.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.894+0000 7f48ccd9b700 1 -- 172.21.15.31:0/633792623 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48c80c75e0 msgr2=0x7f48c80c7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:55.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.894+0000 7f48ccd9b700 1 --2- 172.21.15.31:0/633792623 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48c80c75e0 0x7f48c80c7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:55.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.894+0000 7f48ccd9b700 1 -- 172.21.15.31:0/633792623 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48c80ca3d0 con 0x7f48b8004810 2023-10-06T02:29:55.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.894+0000 7f48cd59c700 1 --2- 172.21.15.31:0/633792623 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f48c80d0410 0x7f48c80d0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:29:55.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.894+0000 7f48c7fff700 1 --2- 172.21.15.31:0/633792623 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48c80c75e0 0x7f48c80c7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:29:55.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.894+0000 7f48ccd9b700 1 --2- 172.21.15.31:0/633792623 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48b8004810 0x7f48c80c9e90 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f48bc007390 tx=0x7f48bc001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:55.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.894+0000 7f48c4ff9700 1 -- 172.21.15.31:0/633792623 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48bc018030 con 0x7f48b8004810 2023-10-06T02:29:55.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.894+0000 7f48c4ff9700 1 -- 172.21.15.31:0/633792623 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48bc00f8d0 con 0x7f48b8004810 2023-10-06T02:29:55.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.894+0000 7f48cdd9d700 1 -- 172.21.15.31:0/633792623 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48c8169ab0 con 0x7f48b8004810 2023-10-06T02:29:55.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.894+0000 7f48c4ff9700 1 -- 172.21.15.31:0/633792623 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48bc00fcb0 con 0x7f48b8004810 2023-10-06T02:29:55.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.894+0000 7f48cdd9d700 1 -- 172.21.15.31:0/633792623 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48c816a070 con 0x7f48b8004810 2023-10-06T02:29:55.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.895+0000 7f48cdd9d700 1 -- 172.21.15.31:0/633792623 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48c804c3c0 con 0x7f48b8004810 2023-10-06T02:29:55.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.896+0000 7f48c4ff9700 1 -- 172.21.15.31:0/633792623 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f48bc01b070 con 0x7f48b8004810 2023-10-06T02:29:55.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.897+0000 7f48c4ff9700 1 --2- 172.21.15.31:0/633792623 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f48ac0674d0 0x7f48ac069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:55.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.897+0000 7f48c4ff9700 1 -- 172.21.15.31:0/633792623 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6114+0+0 (secure 0 0 0) 0x7f48bc019070 con 0x7f48b8004810 2023-10-06T02:29:55.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.897+0000 7f48c7fff700 1 --2- 172.21.15.31:0/633792623 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f48ac0674d0 0x7f48ac069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:55.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.898+0000 7f48c7fff700 1 --2- 172.21.15.31:0/633792623 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f48ac0674d0 0x7f48ac069980 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f48c806e0b0 tx=0x7f48b4015000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:55.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:55.901+0000 7f48c4ff9700 1 -- 172.21.15.31:0/633792623 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f48bc089070 con 0x7f48b8004810 2023-10-06T02:29:56.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.106+0000 7f48cdd9d700 1 -- 172.21.15.31:0/633792623 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f48c815b520 con 0x7f48b8004810 2023-10-06T02:29:56.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.106+0000 7f48c4ff9700 1 -- 172.21.15.31:0/633792623 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v121) v1 ==== 92+0+10 (secure 0 0 0) 0x7f48bc051910 con 0x7f48b8004810 2023-10-06T02:29:56.107 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 8 2023-10-06T02:29:56.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.108+0000 7f48cdd9d700 1 -- 172.21.15.31:0/633792623 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f48ac0674d0 msgr2=0x7f48ac069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:56.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.108+0000 7f48cdd9d700 1 --2- 172.21.15.31:0/633792623 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f48ac0674d0 0x7f48ac069980 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f48c806e0b0 tx=0x7f48b4015000 comp rx=0 tx=0).stop 2023-10-06T02:29:56.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.108+0000 7f48cdd9d700 1 -- 172.21.15.31:0/633792623 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48b8004810 msgr2=0x7f48c80c9e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:56.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.108+0000 7f48cdd9d700 1 --2- 172.21.15.31:0/633792623 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48b8004810 0x7f48c80c9e90 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f48bc007390 tx=0x7f48bc001e50 comp rx=0 tx=0).stop 2023-10-06T02:29:56.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.108+0000 7f48cdd9d700 1 -- 172.21.15.31:0/633792623 shutdown_connections 2023-10-06T02:29:56.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.108+0000 7f48cdd9d700 1 --2- 172.21.15.31:0/633792623 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f48ac0674d0 0x7f48ac069980 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:56.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.108+0000 7f48cdd9d700 1 --2- 172.21.15.31:0/633792623 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f48c80d0410 0x7f48c80d0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:56.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.108+0000 7f48cdd9d700 1 --2- 172.21.15.31:0/633792623 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48b8004810 0x7f48c80c9e90 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:56.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.108+0000 7f48cdd9d700 1 --2- 172.21.15.31:0/633792623 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48c80c75e0 0x7f48c80c7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:56.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.109+0000 7f48cdd9d700 1 -- 172.21.15.31:0/633792623 >> 172.21.15.31:0/633792623 conn(0x7f48c80703a0 msgr2=0x7f48c8150910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:56.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.109+0000 7f48cdd9d700 1 -- 172.21.15.31:0/633792623 shutdown_connections 2023-10-06T02:29:56.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.109+0000 7f48cdd9d700 1 -- 172.21.15.31:0/633792623 wait complete. 2023-10-06T02:29:56.126 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T02:29:56.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.317+0000 7f363599d700 1 Processor -- start 2023-10-06T02:29:56.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.317+0000 7f363599d700 1 -- start start 2023-10-06T02:29:56.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.318+0000 7f363599d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3630153a00 0x7f3630153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:56.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.318+0000 7f363599d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f363015ac50 con 0x7f363015de50 2023-10-06T02:29:56.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.318+0000 7f363599d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f363015d010 con 0x7f36300cea00 2023-10-06T02:29:56.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.318+0000 7f363599d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f363015af60 con 0x7f3630153a00 2023-10-06T02:29:56.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.318+0000 7f362ffff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f363015de50 0x7f363015a540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:54580/0 (socket says 172.21.15.31:54580) 2023-10-06T02:29:56.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.319+0000 7f362ffff700 1 -- 172.21.15.31:0/2224720161 learned_addr learned my addr 172.21.15.31:0/2224720161 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:29:56.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.319+0000 7f363499b700 1 --2- 172.21.15.31:0/2224720161 >> v2:172.21.15.62:3300/0 conn(0x7f3630153a00 0x7f3630153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:56.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.319+0000 7f362f7fe700 1 -- 172.21.15.31:0/2224720161 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1986353899 0 0) 0x7f363015ac50 con 0x7f363015de50 2023-10-06T02:29:56.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.319+0000 7f362f7fe700 1 -- 172.21.15.31:0/2224720161 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3620003570 con 0x7f363015de50 2023-10-06T02:29:56.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.319+0000 7f363499b700 1 -- 172.21.15.31:0/2224720161 >> v1:172.21.15.62:6789/0 conn(0x7f36300cea00 legacy=0x7f36300cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:56.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.319+0000 7f362f7fe700 1 -- 172.21.15.31:0/2224720161 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4091080346 0 0) 0x7f3620003570 con 0x7f363015de50 2023-10-06T02:29:56.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.319+0000 7f363499b700 1 -- 172.21.15.31:0/2224720161 >> v1:172.21.15.31:6789/0 conn(0x7f363015de50 legacy=0x7f363015a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:56.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.319+0000 7f363499b700 1 -- 172.21.15.31:0/2224720161 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f363015b240 con 0x7f3630153a00 2023-10-06T02:29:56.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.320+0000 7f363499b700 1 --2- 172.21.15.31:0/2224720161 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3630153a00 0x7f3630153dd0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f36240098e0 tx=0x7f36240091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97947c6fcc9baf67 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:56.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.320+0000 7f362f7fe700 1 -- 172.21.15.31:0/2224720161 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361c001470 con 0x7f3630153a00 2023-10-06T02:29:56.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.320+0000 7f362f7fe700 1 -- 172.21.15.31:0/2224720161 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3624011420 con 0x7f3630153a00 2023-10-06T02:29:56.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.320+0000 7f362f7fe700 1 -- 172.21.15.31:0/2224720161 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36240117a0 con 0x7f3630153a00 2023-10-06T02:29:56.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.321+0000 7f363599d700 1 -- 172.21.15.31:0/2224720161 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3630153a00 msgr2=0x7f3630153dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:56.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.321+0000 7f363599d700 1 --2- 172.21.15.31:0/2224720161 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3630153a00 0x7f3630153dd0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f36240098e0 tx=0x7f36240091a0 comp rx=0 tx=0).stop 2023-10-06T02:29:56.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.321+0000 7f363599d700 1 -- 172.21.15.31:0/2224720161 shutdown_connections 2023-10-06T02:29:56.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.321+0000 7f363599d700 1 --2- 172.21.15.31:0/2224720161 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3630153a00 0x7f3630153dd0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:56.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.321+0000 7f363599d700 1 -- 172.21.15.31:0/2224720161 >> 172.21.15.31:0/2224720161 conn(0x7f36300703a0 msgr2=0x7f36300707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:56.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.321+0000 7f363599d700 1 -- 172.21.15.31:0/2224720161 shutdown_connections 2023-10-06T02:29:56.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.321+0000 7f363599d700 1 -- 172.21.15.31:0/2224720161 wait complete. 2023-10-06T02:29:56.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.322+0000 7f363599d700 1 Processor -- start 2023-10-06T02:29:56.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.322+0000 7f363599d700 1 -- start start 2023-10-06T02:29:56.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.323+0000 7f363599d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f36300cea00 0x7f36300c5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:56.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.323+0000 7f363599d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3630153a00 0x7f36300b1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:56.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.323+0000 7f363599d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f363015de50 0x7f36300b1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:56.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.323+0000 7f363599d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f363016cfb0 con 0x7f3630153a00 2023-10-06T02:29:56.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.323+0000 7f363599d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f363016d290 con 0x7f363015de50 2023-10-06T02:29:56.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.323+0000 7f363599d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f363016d570 con 0x7f36300cea00 2023-10-06T02:29:56.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.324+0000 7f362ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3630153a00 0x7f36300b1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:56.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.324+0000 7f362ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3630153a00 0x7f36300b1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34120/0 (socket says 172.21.15.31:34120) 2023-10-06T02:29:56.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.324+0000 7f362ffff700 1 -- 172.21.15.31:0/3248196282 learned_addr learned my addr 172.21.15.31:0/3248196282 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:56.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.324+0000 7f363499b700 1 --2- 172.21.15.31:0/3248196282 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f36300cea00 0x7f36300c5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:56.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.324+0000 7f363519c700 1 --2- 172.21.15.31:0/3248196282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f363015de50 0x7f36300b1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:56.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.324+0000 7f362ffff700 1 -- 172.21.15.31:0/3248196282 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f36300cea00 msgr2=0x7f36300c5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:56.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.324+0000 7f362ffff700 1 --2- 172.21.15.31:0/3248196282 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f36300cea00 0x7f36300c5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:56.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.324+0000 7f362ffff700 1 -- 172.21.15.31:0/3248196282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f363015de50 msgr2=0x7f36300b1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:56.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.324+0000 7f362ffff700 1 --2- 172.21.15.31:0/3248196282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f363015de50 0x7f36300b1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:56.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.324+0000 7f362ffff700 1 -- 172.21.15.31:0/3248196282 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3624009420 con 0x7f3630153a00 2023-10-06T02:29:56.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.324+0000 7f363519c700 1 --2- 172.21.15.31:0/3248196282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f363015de50 0x7f36300b1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:29:56.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.324+0000 7f362ffff700 1 --2- 172.21.15.31:0/3248196282 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3630153a00 0x7f36300b1580 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f361c00cb90 tx=0x7f361c001aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:56.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.325+0000 7f362cff9700 1 -- 172.21.15.31:0/3248196282 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361c007150 con 0x7f3630153a00 2023-10-06T02:29:56.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.325+0000 7f362cff9700 1 -- 172.21.15.31:0/3248196282 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f361c0072d0 con 0x7f3630153a00 2023-10-06T02:29:56.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.325+0000 7f363599d700 1 -- 172.21.15.31:0/3248196282 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f363016d850 con 0x7f3630153a00 2023-10-06T02:29:56.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.325+0000 7f362cff9700 1 -- 172.21.15.31:0/3248196282 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361c007690 con 0x7f3630153a00 2023-10-06T02:29:56.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.325+0000 7f363599d700 1 -- 172.21.15.31:0/3248196282 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36300c59d0 con 0x7f3630153a00 2023-10-06T02:29:56.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.326+0000 7f363599d700 1 -- 172.21.15.31:0/3248196282 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f363016dd70 con 0x7f3630153a00 2023-10-06T02:29:56.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.327+0000 7f362cff9700 1 -- 172.21.15.31:0/3248196282 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f361c004a40 con 0x7f3630153a00 2023-10-06T02:29:56.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.328+0000 7f362cff9700 1 --2- 172.21.15.31:0/3248196282 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f36140674d0 0x7f3614069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:56.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.328+0000 7f362cff9700 1 -- 172.21.15.31:0/3248196282 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6114+0+0 (secure 0 0 0) 0x7f361c084830 con 0x7f3630153a00 2023-10-06T02:29:56.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.328+0000 7f363499b700 1 --2- 172.21.15.31:0/3248196282 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f36140674d0 0x7f3614069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:56.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.329+0000 7f363499b700 1 --2- 172.21.15.31:0/3248196282 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f36140674d0 0x7f3614069980 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3624009fd0 tx=0x7f3624016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:56.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.332+0000 7f362cff9700 1 -- 172.21.15.31:0/3248196282 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f363016dd70 con 0x7f3630153a00 2023-10-06T02:29:56.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.533+0000 7f363599d700 1 -- 172.21.15.31:0/3248196282 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f363016dd70 con 0x7f3630153a00 2023-10-06T02:29:56.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.533+0000 7f362cff9700 1 -- 172.21.15.31:0/3248196282 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v121) v1 ==== 104+0+11 (secure 0 0 0) 0x7f363016dd70 con 0x7f3630153a00 2023-10-06T02:29:56.535 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-10-06T02:29:56.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.535+0000 7f363599d700 1 -- 172.21.15.31:0/3248196282 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f36140674d0 msgr2=0x7f3614069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:56.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.535+0000 7f363599d700 1 --2- 172.21.15.31:0/3248196282 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f36140674d0 0x7f3614069980 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3624009fd0 tx=0x7f3624016000 comp rx=0 tx=0).stop 2023-10-06T02:29:56.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.535+0000 7f363599d700 1 -- 172.21.15.31:0/3248196282 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3630153a00 msgr2=0x7f36300b1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:56.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.535+0000 7f363599d700 1 --2- 172.21.15.31:0/3248196282 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3630153a00 0x7f36300b1580 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f361c00cb90 tx=0x7f361c001aa0 comp rx=0 tx=0).stop 2023-10-06T02:29:56.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.536+0000 7f363599d700 1 -- 172.21.15.31:0/3248196282 shutdown_connections 2023-10-06T02:29:56.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.536+0000 7f363599d700 1 --2- 172.21.15.31:0/3248196282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f363015de50 0x7f36300b1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:56.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.536+0000 7f363599d700 1 --2- 172.21.15.31:0/3248196282 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f36140674d0 0x7f3614069980 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:56.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.536+0000 7f363599d700 1 --2- 172.21.15.31:0/3248196282 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f36300cea00 0x7f36300c5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:56.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.536+0000 7f363599d700 1 --2- 172.21.15.31:0/3248196282 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3630153a00 0x7f36300b1580 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:56.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.536+0000 7f363599d700 1 -- 172.21.15.31:0/3248196282 >> 172.21.15.31:0/3248196282 conn(0x7f36300703a0 msgr2=0x7f36300c60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:56.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.536+0000 7f363599d700 1 -- 172.21.15.31:0/3248196282 shutdown_connections 2023-10-06T02:29:56.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.536+0000 7f363599d700 1 -- 172.21.15.31:0/3248196282 wait complete. 2023-10-06T02:29:56.554 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T02:29:56.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.743+0000 7f0a3d19d700 1 Processor -- start 2023-10-06T02:29:56.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.744+0000 7f0a3d19d700 1 -- start start 2023-10-06T02:29:56.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.745+0000 7f0a3d19d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0a38142c00 0x7f0a3806b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:56.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.745+0000 7f0a3d19d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0a380c7f90 0x7f0a3806c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:56.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.745+0000 7f0a3d19d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a380ca480 con 0x7f0a3806bd20 2023-10-06T02:29:56.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.745+0000 7f0a3d19d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a3815e240 con 0x7f0a38142c00 2023-10-06T02:29:56.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.745+0000 7f0a3d19d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a3806b350 con 0x7f0a380c7f90 2023-10-06T02:29:56.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.745+0000 7f0a37fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0a38142c00 0x7f0a3806b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:56.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.745+0000 7f0a37fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0a38142c00 0x7f0a3806b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34128/0 (socket says 172.21.15.31:34128) 2023-10-06T02:29:56.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.745+0000 7f0a37fff700 1 -- 172.21.15.31:0/2420576850 learned_addr learned my addr 172.21.15.31:0/2420576850 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:56.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.745+0000 7f0a3c99c700 1 --2- 172.21.15.31:0/2420576850 >> v2:172.21.15.62:3300/0 conn(0x7f0a380c7f90 0x7f0a3806c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:56.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.746+0000 7f0a37fff700 1 -- 172.21.15.31:0/2420576850 >> v2:172.21.15.62:3300/0 conn(0x7f0a380c7f90 msgr2=0x7f0a3806c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:56.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.746+0000 7f0a37fff700 1 --2- 172.21.15.31:0/2420576850 >> v2:172.21.15.62:3300/0 conn(0x7f0a380c7f90 0x7f0a3806c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:56.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.746+0000 7f0a37fff700 1 -- 172.21.15.31:0/2420576850 >> v1:172.21.15.62:6789/0 conn(0x7f0a3806bd20 legacy=0x7f0a380c7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:56.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.746+0000 7f0a37fff700 1 -- 172.21.15.31:0/2420576850 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a3815df60 con 0x7f0a38142c00 2023-10-06T02:29:56.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.746+0000 7f0a37fff700 1 --2- 172.21.15.31:0/2420576850 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a38142c00 0x7f0a3806b7e0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f0a3806d780 tx=0x7f0a28009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5eccb2242682746 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:56.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.746+0000 7f0a36ffd700 1 -- 172.21.15.31:0/2420576850 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2800b820 con 0x7f0a38142c00 2023-10-06T02:29:56.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.746+0000 7f0a36ffd700 1 -- 172.21.15.31:0/2420576850 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a2800b980 con 0x7f0a38142c00 2023-10-06T02:29:56.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.746+0000 7f0a36ffd700 1 -- 172.21.15.31:0/2420576850 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2800bd40 con 0x7f0a38142c00 2023-10-06T02:29:56.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.747+0000 7f0a3d19d700 1 -- 172.21.15.31:0/2420576850 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a38142c00 msgr2=0x7f0a3806b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:56.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.747+0000 7f0a3d19d700 1 --2- 172.21.15.31:0/2420576850 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a38142c00 0x7f0a3806b7e0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f0a3806d780 tx=0x7f0a28009250 comp rx=0 tx=0).stop 2023-10-06T02:29:56.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.747+0000 7f0a3d19d700 1 -- 172.21.15.31:0/2420576850 shutdown_connections 2023-10-06T02:29:56.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.747+0000 7f0a3d19d700 1 --2- 172.21.15.31:0/2420576850 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a38142c00 0x7f0a3806b7e0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:56.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.747+0000 7f0a3d19d700 1 --2- 172.21.15.31:0/2420576850 >> v2:172.21.15.62:3300/0 conn(0x7f0a380c7f90 0x7f0a3806c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:56.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.747+0000 7f0a3d19d700 1 -- 172.21.15.31:0/2420576850 >> 172.21.15.31:0/2420576850 conn(0x7f0a380675a0 msgr2=0x7f0a380679a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:56.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.747+0000 7f0a3d19d700 1 -- 172.21.15.31:0/2420576850 shutdown_connections 2023-10-06T02:29:56.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.747+0000 7f0a3d19d700 1 -- 172.21.15.31:0/2420576850 wait complete. 2023-10-06T02:29:56.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.748+0000 7f0a3d19d700 1 Processor -- start 2023-10-06T02:29:56.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.748+0000 7f0a3d19d700 1 -- start start 2023-10-06T02:29:56.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.749+0000 7f0a3d19d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a3806bd20 0x7f0a381692b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:56.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.749+0000 7f0a3d19d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a380c7f90 0x7f0a3815be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:56.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.749+0000 7f0a3d19d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0a38142c00 0x7f0a3816a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:56.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.749+0000 7f0a3d19d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3815cdd0 con 0x7f0a380c7f90 2023-10-06T02:29:56.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.749+0000 7f0a3d19d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a380c7600 con 0x7f0a3806bd20 2023-10-06T02:29:56.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.749+0000 7f0a3d19d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3815c530 con 0x7f0a38142c00 2023-10-06T02:29:56.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.749+0000 7f0a377fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a380c7f90 0x7f0a3815be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:56.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.750+0000 7f0a377fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a380c7f90 0x7f0a3815be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34134/0 (socket says 172.21.15.31:34134) 2023-10-06T02:29:56.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.750+0000 7f0a377fe700 1 -- 172.21.15.31:0/805891889 learned_addr learned my addr 172.21.15.31:0/805891889 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:56.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.750+0000 7f0a37fff700 1 --2- 172.21.15.31:0/805891889 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a3806bd20 0x7f0a381692b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:56.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.750+0000 7f0a3c99c700 1 --2- 172.21.15.31:0/805891889 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0a38142c00 0x7f0a3816a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:56.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.750+0000 7f0a377fe700 1 -- 172.21.15.31:0/805891889 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0a38142c00 msgr2=0x7f0a3816a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:56.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.750+0000 7f0a377fe700 1 --2- 172.21.15.31:0/805891889 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0a38142c00 0x7f0a3816a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:56.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.750+0000 7f0a377fe700 1 -- 172.21.15.31:0/805891889 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a3806bd20 msgr2=0x7f0a381692b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:56.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.750+0000 7f0a377fe700 1 --2- 172.21.15.31:0/805891889 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a3806bd20 0x7f0a381692b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:56.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.750+0000 7f0a377fe700 1 -- 172.21.15.31:0/805891889 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a280094f0 con 0x7f0a380c7f90 2023-10-06T02:29:56.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.750+0000 7f0a377fe700 1 --2- 172.21.15.31:0/805891889 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a380c7f90 0x7f0a3815be30 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c00c5c0 tx=0x7f0a2c007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:56.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.750+0000 7f0a1ffff700 1 -- 172.21.15.31:0/805891889 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c011590 con 0x7f0a380c7f90 2023-10-06T02:29:56.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.750+0000 7f0a1ffff700 1 -- 172.21.15.31:0/805891889 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a2c0116f0 con 0x7f0a380c7f90 2023-10-06T02:29:56.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.750+0000 7f0a1ffff700 1 -- 172.21.15.31:0/805891889 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c011ab0 con 0x7f0a380c7f90 2023-10-06T02:29:56.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.751+0000 7f0a3d19d700 1 -- 172.21.15.31:0/805891889 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a3815c810 con 0x7f0a380c7f90 2023-10-06T02:29:56.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.751+0000 7f0a3d19d700 1 -- 172.21.15.31:0/805891889 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a3815a3b0 con 0x7f0a380c7f90 2023-10-06T02:29:56.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.752+0000 7f0a3d19d700 1 -- 172.21.15.31:0/805891889 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a3804c3c0 con 0x7f0a380c7f90 2023-10-06T02:29:56.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.753+0000 7f0a1ffff700 1 -- 172.21.15.31:0/805891889 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f0a2c01c460 con 0x7f0a380c7f90 2023-10-06T02:29:56.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.754+0000 7f0a1ffff700 1 --2- 172.21.15.31:0/805891889 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0a240674d0 0x7f0a24069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:56.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.754+0000 7f0a1ffff700 1 -- 172.21.15.31:0/805891889 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6114+0+0 (secure 0 0 0) 0x7f0a2c084e00 con 0x7f0a380c7f90 2023-10-06T02:29:56.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.754+0000 7f0a37fff700 1 --2- 172.21.15.31:0/805891889 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0a240674d0 0x7f0a24069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:56.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.755+0000 7f0a37fff700 1 --2- 172.21.15.31:0/805891889 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0a240674d0 0x7f0a24069980 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0a3806d780 tx=0x7f0a28016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:56.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.758+0000 7f0a1ffff700 1 -- 172.21.15.31:0/805891889 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0a2c051620 con 0x7f0a380c7f90 2023-10-06T02:29:56.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.959+0000 7f0a3d19d700 1 -- 172.21.15.31:0/805891889 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0a3806ad00 con 0x7f0a380c7f90 2023-10-06T02:29:56.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.959+0000 7f0a1ffff700 1 -- 172.21.15.31:0/805891889 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v121) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0a2c054c40 con 0x7f0a380c7f90 2023-10-06T02:29:56.960 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-10-06T02:29:56.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.962+0000 7f0a3d19d700 1 -- 172.21.15.31:0/805891889 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0a240674d0 msgr2=0x7f0a24069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:56.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.962+0000 7f0a3d19d700 1 --2- 172.21.15.31:0/805891889 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0a240674d0 0x7f0a24069980 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0a3806d780 tx=0x7f0a28016000 comp rx=0 tx=0).stop 2023-10-06T02:29:56.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.962+0000 7f0a3d19d700 1 -- 172.21.15.31:0/805891889 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a380c7f90 msgr2=0x7f0a3815be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:56.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.962+0000 7f0a3d19d700 1 --2- 172.21.15.31:0/805891889 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a380c7f90 0x7f0a3815be30 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c00c5c0 tx=0x7f0a2c007f30 comp rx=0 tx=0).stop 2023-10-06T02:29:56.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.962+0000 7f0a3d19d700 1 -- 172.21.15.31:0/805891889 shutdown_connections 2023-10-06T02:29:56.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.962+0000 7f0a3d19d700 1 --2- 172.21.15.31:0/805891889 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0a240674d0 0x7f0a24069980 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:56.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.962+0000 7f0a3d19d700 1 --2- 172.21.15.31:0/805891889 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0a38142c00 0x7f0a3816a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:56.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.962+0000 7f0a3d19d700 1 --2- 172.21.15.31:0/805891889 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a3806bd20 0x7f0a381692b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:56.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.962+0000 7f0a3d19d700 1 --2- 172.21.15.31:0/805891889 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a380c7f90 0x7f0a3815be30 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:56.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.962+0000 7f0a3d19d700 1 -- 172.21.15.31:0/805891889 >> 172.21.15.31:0/805891889 conn(0x7f0a380675a0 msgr2=0x7f0a380c1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:56.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.962+0000 7f0a3d19d700 1 -- 172.21.15.31:0/805891889 shutdown_connections 2023-10-06T02:29:56.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:56.962+0000 7f0a3d19d700 1 -- 172.21.15.31:0/805891889 wait complete. 2023-10-06T02:29:56.979 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:29:57.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.166+0000 7fdbe759e700 1 Processor -- start 2023-10-06T02:29:57.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.166+0000 7fdbe759e700 1 -- start start 2023-10-06T02:29:57.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.167+0000 7fdbe759e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdbe8153c30 0x7fdbe815a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:57.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.167+0000 7fdbe759e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fdbe815de50 0x7fdbe80cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:57.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.167+0000 7fdbe759e700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbe815afd0 con 0x7fdbe80cef40 2023-10-06T02:29:57.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.167+0000 7fdbe759e700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbe815b2b0 con 0x7fdbe815de50 2023-10-06T02:29:57.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.167+0000 7fdbe759e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe815b590 con 0x7fdbe8153c30 2023-10-06T02:29:57.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.168+0000 7fdbe5d9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fdbe815de50 0x7fdbe80cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:57.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.168+0000 7fdbe5d9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fdbe815de50 0x7fdbe80cea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34136/0 (socket says 172.21.15.31:34136) 2023-10-06T02:29:57.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.168+0000 7fdbe5d9b700 1 -- 172.21.15.31:0/106204673 learned_addr learned my addr 172.21.15.31:0/106204673 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:57.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.168+0000 7fdbe659c700 1 --2- 172.21.15.31:0/106204673 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdbe8153c30 0x7fdbe815a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:57.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.168+0000 7fdbe5d9b700 1 -- 172.21.15.31:0/106204673 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdbe8153c30 msgr2=0x7fdbe815a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:57.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.168+0000 7fdbe5d9b700 1 --2- 172.21.15.31:0/106204673 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdbe8153c30 0x7fdbe815a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:57.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.168+0000 7fdbe5d9b700 1 -- 172.21.15.31:0/106204673 >> v1:172.21.15.62:6789/0 conn(0x7fdbe80cef40 legacy=0x7fdbe806d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:57.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.168+0000 7fdbe5d9b700 1 -- 172.21.15.31:0/106204673 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbe815b870 con 0x7fdbe815de50 2023-10-06T02:29:57.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.168+0000 7fdbe5d9b700 1 --2- 172.21.15.31:0/106204673 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdbe815de50 0x7fdbe80cea00 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc0099b0 tx=0x7fdbdc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78edeac1b3e8e65e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:57.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.169+0000 7fdbe559a700 1 -- 172.21.15.31:0/106204673 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbdc009ea0 con 0x7fdbe815de50 2023-10-06T02:29:57.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.169+0000 7fdbe559a700 1 -- 172.21.15.31:0/106204673 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbdc00b860 con 0x7fdbe815de50 2023-10-06T02:29:57.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.169+0000 7fdbe559a700 1 -- 172.21.15.31:0/106204673 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbdc00bc00 con 0x7fdbe815de50 2023-10-06T02:29:57.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.169+0000 7fdbe759e700 1 -- 172.21.15.31:0/106204673 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdbe815de50 msgr2=0x7fdbe80cea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:57.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.169+0000 7fdbe759e700 1 --2- 172.21.15.31:0/106204673 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdbe815de50 0x7fdbe80cea00 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc0099b0 tx=0x7fdbdc009250 comp rx=0 tx=0).stop 2023-10-06T02:29:57.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.169+0000 7fdbe759e700 1 -- 172.21.15.31:0/106204673 shutdown_connections 2023-10-06T02:29:57.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.169+0000 7fdbe759e700 1 --2- 172.21.15.31:0/106204673 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdbe8153c30 0x7fdbe815a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:57.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.169+0000 7fdbe759e700 1 --2- 172.21.15.31:0/106204673 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdbe815de50 0x7fdbe80cea00 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:57.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.169+0000 7fdbe759e700 1 -- 172.21.15.31:0/106204673 >> 172.21.15.31:0/106204673 conn(0x7fdbe80703a0 msgr2=0x7fdbe80707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:57.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.169+0000 7fdbe759e700 1 -- 172.21.15.31:0/106204673 shutdown_connections 2023-10-06T02:29:57.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.169+0000 7fdbe759e700 1 -- 172.21.15.31:0/106204673 wait complete. 2023-10-06T02:29:57.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.171+0000 7fdbe759e700 1 Processor -- start 2023-10-06T02:29:57.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.171+0000 7fdbe759e700 1 -- start start 2023-10-06T02:29:57.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.172+0000 7fdbe759e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdbe80cef40 0x7fdbe80cd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:57.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.172+0000 7fdbe759e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdbe8153c30 0x7fdbe80cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:57.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.172+0000 7fdbe759e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdbe815de50 0x7fdbe80b32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:57.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.172+0000 7fdbe759e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe80c75e0 con 0x7fdbe80cef40 2023-10-06T02:29:57.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.172+0000 7fdbe759e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe816d6f0 con 0x7fdbe8153c30 2023-10-06T02:29:57.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.172+0000 7fdbe759e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe816d9d0 con 0x7fdbe815de50 2023-10-06T02:29:57.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.172+0000 7fdbe659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdbe80cef40 0x7fdbe80cd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:57.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.172+0000 7fdbe659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdbe80cef40 0x7fdbe80cd1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34144/0 (socket says 172.21.15.31:34144) 2023-10-06T02:29:57.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.172+0000 7fdbe659c700 1 -- 172.21.15.31:0/2969447054 learned_addr learned my addr 172.21.15.31:0/2969447054 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:57.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.172+0000 7fdbe5d9b700 1 --2- 172.21.15.31:0/2969447054 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdbe8153c30 0x7fdbe80cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:57.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.173+0000 7fdbe6d9d700 1 --2- 172.21.15.31:0/2969447054 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdbe815de50 0x7fdbe80b32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:57.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.173+0000 7fdbe659c700 1 -- 172.21.15.31:0/2969447054 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdbe815de50 msgr2=0x7fdbe80b32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:57.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.173+0000 7fdbe659c700 1 --2- 172.21.15.31:0/2969447054 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdbe815de50 0x7fdbe80b32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:57.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.173+0000 7fdbe659c700 1 -- 172.21.15.31:0/2969447054 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdbe8153c30 msgr2=0x7fdbe80cc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:57.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.173+0000 7fdbe659c700 1 --2- 172.21.15.31:0/2969447054 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdbe8153c30 0x7fdbe80cc6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:57.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.173+0000 7fdbe659c700 1 -- 172.21.15.31:0/2969447054 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbdc0094f0 con 0x7fdbe80cef40 2023-10-06T02:29:57.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.173+0000 7fdbe6d9d700 1 --2- 172.21.15.31:0/2969447054 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdbe815de50 0x7fdbe80b32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:29:57.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.173+0000 7fdbe659c700 1 --2- 172.21.15.31:0/2969447054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdbe80cef40 0x7fdbe80cd1a0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fdbd800ec80 tx=0x7fdbd80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:57.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.173+0000 7fdbd67fc700 1 -- 172.21.15.31:0/2969447054 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd800c850 con 0x7fdbe80cef40 2023-10-06T02:29:57.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.173+0000 7fdbd67fc700 1 -- 172.21.15.31:0/2969447054 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbd800c9b0 con 0x7fdbe80cef40 2023-10-06T02:29:57.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.173+0000 7fdbd67fc700 1 -- 172.21.15.31:0/2969447054 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd800cd50 con 0x7fdbe80cef40 2023-10-06T02:29:57.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.173+0000 7fdbe759e700 1 -- 172.21.15.31:0/2969447054 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbe816dcb0 con 0x7fdbe80cef40 2023-10-06T02:29:57.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.174+0000 7fdbe759e700 1 -- 172.21.15.31:0/2969447054 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbe80cd6e0 con 0x7fdbe80cef40 2023-10-06T02:29:57.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.175+0000 7fdbe759e700 1 -- 172.21.15.31:0/2969447054 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbe80b6390 con 0x7fdbe80cef40 2023-10-06T02:29:57.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.175+0000 7fdbd67fc700 1 -- 172.21.15.31:0/2969447054 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7fdbd8013680 con 0x7fdbe80cef40 2023-10-06T02:29:57.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.176+0000 7fdbd67fc700 1 --2- 172.21.15.31:0/2969447054 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fdbc80674d0 0x7fdbc8069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:57.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.176+0000 7fdbd67fc700 1 -- 172.21.15.31:0/2969447054 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6114+0+0 (secure 0 0 0) 0x7fdbd8085770 con 0x7fdbe80cef40 2023-10-06T02:29:57.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.177+0000 7fdbe5d9b700 1 --2- 172.21.15.31:0/2969447054 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fdbc80674d0 0x7fdbc8069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:57.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.178+0000 7fdbe5d9b700 1 --2- 172.21.15.31:0/2969447054 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fdbc80674d0 0x7fdbc8069980 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc0099b0 tx=0x7fdbdc017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:57.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.181+0000 7fdbd67fc700 1 -- 172.21.15.31:0/2969447054 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdbd8051fd0 con 0x7fdbe80cef40 2023-10-06T02:29:57.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.425+0000 7fdbe759e700 1 -- 172.21.15.31:0/2969447054 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdbe804c3c0 con 0x7fdbe80cef40 2023-10-06T02:29:57.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.426+0000 7fdbd67fc700 1 -- 172.21.15.31:0/2969447054 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+4590 (secure 0 0 0) 0x7fdbd80555f0 con 0x7fdbe80cef40 2023-10-06T02:29:57.426 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:29:57.427 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":81,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6338,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/1906951645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":1906951645},{"type":"v1","addr":"172.21.15.62:6834","nonce":1906951645}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75},{"gid":6341,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3788411327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3788411327},{"type":"v1","addr":"172.21.15.62:6835","nonce":3788411327}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75},{"gid":7569,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6833/581939695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":581939695},{"type":"v1","addr":"172.21.15.31:6833","nonce":581939695}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:29:35.664920+0000","modified":"2023-10-06T02:29:39.189818+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7575},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7575":{"gid":7575,"name":"c","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/40412586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":40412586},{"type":"v1","addr":"172.21.15.31:6835","nonce":40412586}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-06T02:29:57.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.428+0000 7fdbe759e700 1 -- 172.21.15.31:0/2969447054 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fdbc80674d0 msgr2=0x7fdbc8069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:57.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.428+0000 7fdbe759e700 1 --2- 172.21.15.31:0/2969447054 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fdbc80674d0 0x7fdbc8069980 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc0099b0 tx=0x7fdbdc017000 comp rx=0 tx=0).stop 2023-10-06T02:29:57.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.428+0000 7fdbe759e700 1 -- 172.21.15.31:0/2969447054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdbe80cef40 msgr2=0x7fdbe80cd1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:57.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.428+0000 7fdbe759e700 1 --2- 172.21.15.31:0/2969447054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdbe80cef40 0x7fdbe80cd1a0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fdbd800ec80 tx=0x7fdbd80091a0 comp rx=0 tx=0).stop 2023-10-06T02:29:57.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.428+0000 7fdbe759e700 1 -- 172.21.15.31:0/2969447054 shutdown_connections 2023-10-06T02:29:57.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.428+0000 7fdbe759e700 1 --2- 172.21.15.31:0/2969447054 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fdbc80674d0 0x7fdbc8069980 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:57.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.428+0000 7fdbe759e700 1 --2- 172.21.15.31:0/2969447054 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdbe815de50 0x7fdbe80b32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:57.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.428+0000 7fdbe759e700 1 --2- 172.21.15.31:0/2969447054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdbe80cef40 0x7fdbe80cd1a0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:57.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.428+0000 7fdbe759e700 1 --2- 172.21.15.31:0/2969447054 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdbe8153c30 0x7fdbe80cc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:57.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.428+0000 7fdbe759e700 1 -- 172.21.15.31:0/2969447054 >> 172.21.15.31:0/2969447054 conn(0x7fdbe80703a0 msgr2=0x7fdbe80b1870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:57.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.428+0000 7fdbe759e700 1 -- 172.21.15.31:0/2969447054 shutdown_connections 2023-10-06T02:29:57.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.428+0000 7fdbe759e700 1 -- 172.21.15.31:0/2969447054 wait complete. 2023-10-06T02:29:57.430 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 81 2023-10-06T02:29:57.447 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:29:57.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.635+0000 7f0a74d9d700 1 Processor -- start 2023-10-06T02:29:57.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.635+0000 7f0a74d9d700 1 -- start start 2023-10-06T02:29:57.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.637+0000 7f0a74d9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0a7013fb20 0x7f0a7015a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:57.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.637+0000 7f0a74d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0a7006aaa0 0x7f0a700c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:57.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.637+0000 7f0a74d9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a7015ae70 con 0x7f0a7013fef0 2023-10-06T02:29:57.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.637+0000 7f0a74d9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a7006af60 con 0x7f0a7013fb20 2023-10-06T02:29:57.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.637+0000 7f0a74d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7006b240 con 0x7f0a7006aaa0 2023-10-06T02:29:57.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.637+0000 7f0a6effd700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f0a7013fef0 0x7f0a700c7990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:54594/0 (socket says 172.21.15.31:54594) 2023-10-06T02:29:57.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.637+0000 7f0a6effd700 1 -- 172.21.15.31:0/825577792 learned_addr learned my addr 172.21.15.31:0/825577792 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:29:57.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.637+0000 7f0a6f7fe700 1 --2- 172.21.15.31:0/825577792 >> v2:172.21.15.62:3300/0 conn(0x7f0a7013fb20 0x7f0a7015a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:57.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.637+0000 7f0a6ffff700 1 --2- 172.21.15.31:0/825577792 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0a7006aaa0 0x7f0a700c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:57.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.637+0000 7f0a6e7fc700 1 -- 172.21.15.31:0/825577792 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1454508614 0 0) 0x7f0a64004b80 con 0x7f0a7013fef0 2023-10-06T02:29:57.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.637+0000 7f0a6e7fc700 1 -- 172.21.15.31:0/825577792 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a58003570 con 0x7f0a7013fef0 2023-10-06T02:29:57.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.638+0000 7f0a6ffff700 1 -- 172.21.15.31:0/825577792 >> v2:172.21.15.62:3300/0 conn(0x7f0a7013fb20 msgr2=0x7f0a7015a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:57.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.638+0000 7f0a6ffff700 1 --2- 172.21.15.31:0/825577792 >> v2:172.21.15.62:3300/0 conn(0x7f0a7013fb20 0x7f0a7015a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:57.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.638+0000 7f0a6ffff700 1 -- 172.21.15.31:0/825577792 >> v1:172.21.15.31:6789/0 conn(0x7f0a7013fef0 legacy=0x7f0a700c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:57.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.638+0000 7f0a6ffff700 1 -- 172.21.15.31:0/825577792 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a7006b520 con 0x7f0a7006aaa0 2023-10-06T02:29:57.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.638+0000 7f0a6f7fe700 1 --2- 172.21.15.31:0/825577792 >> v2:172.21.15.62:3300/0 conn(0x7f0a7013fb20 0x7f0a7015a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:29:57.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.638+0000 7f0a6ffff700 1 --2- 172.21.15.31:0/825577792 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0a7006aaa0 0x7f0a700c9d00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0a68009e60 tx=0x7f0a68009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a126248519b94fef server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:57.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.638+0000 7f0a6e7fc700 1 -- 172.21.15.31:0/825577792 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a6800e070 con 0x7f0a7006aaa0 2023-10-06T02:29:57.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.638+0000 7f0a6e7fc700 1 -- 172.21.15.31:0/825577792 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a6800bab0 con 0x7f0a7006aaa0 2023-10-06T02:29:57.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.638+0000 7f0a6e7fc700 1 -- 172.21.15.31:0/825577792 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a6800be70 con 0x7f0a7006aaa0 2023-10-06T02:29:57.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.639+0000 7f0a74d9d700 1 -- 172.21.15.31:0/825577792 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0a7006aaa0 msgr2=0x7f0a700c9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:57.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.639+0000 7f0a74d9d700 1 --2- 172.21.15.31:0/825577792 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0a7006aaa0 0x7f0a700c9d00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0a68009e60 tx=0x7f0a68009780 comp rx=0 tx=0).stop 2023-10-06T02:29:57.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.639+0000 7f0a74d9d700 1 -- 172.21.15.31:0/825577792 shutdown_connections 2023-10-06T02:29:57.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.639+0000 7f0a74d9d700 1 --2- 172.21.15.31:0/825577792 >> v2:172.21.15.62:3300/0 conn(0x7f0a7013fb20 0x7f0a7015a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:57.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.639+0000 7f0a74d9d700 1 --2- 172.21.15.31:0/825577792 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0a7006aaa0 0x7f0a700c9d00 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:57.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.639+0000 7f0a74d9d700 1 -- 172.21.15.31:0/825577792 >> 172.21.15.31:0/825577792 conn(0x7f0a7006d170 msgr2=0x7f0a700cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:57.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.639+0000 7f0a74d9d700 1 -- 172.21.15.31:0/825577792 shutdown_connections 2023-10-06T02:29:57.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.639+0000 7f0a74d9d700 1 -- 172.21.15.31:0/825577792 wait complete. 2023-10-06T02:29:57.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.640+0000 7f0a74d9d700 1 Processor -- start 2023-10-06T02:29:57.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.641+0000 7f0a74d9d700 1 -- start start 2023-10-06T02:29:57.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.641+0000 7f0a74d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a7006aaa0 0x7f0a70156b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:57.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.641+0000 7f0a74d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0a7013fb20 0x7f0a70155e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:57.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.641+0000 7f0a74d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a7013fef0 0x7f0a70158450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:57.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.641+0000 7f0a74d9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7013e440 con 0x7f0a7013fef0 2023-10-06T02:29:57.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.641+0000 7f0a74d9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a70157fc0 con 0x7f0a7006aaa0 2023-10-06T02:29:57.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.641+0000 7f0a74d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a70159260 con 0x7f0a7013fb20 2023-10-06T02:29:57.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.642+0000 7f0a6ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a7013fef0 0x7f0a70158450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:57.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.642+0000 7f0a6ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a7013fef0 0x7f0a70158450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34146/0 (socket says 172.21.15.31:34146) 2023-10-06T02:29:57.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.642+0000 7f0a6ffff700 1 -- 172.21.15.31:0/2168987448 learned_addr learned my addr 172.21.15.31:0/2168987448 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:57.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.642+0000 7f0a6f7fe700 1 --2- 172.21.15.31:0/2168987448 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a7006aaa0 0x7f0a70156b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:57.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.642+0000 7f0a6effd700 1 --2- 172.21.15.31:0/2168987448 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0a7013fb20 0x7f0a70155e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:57.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.642+0000 7f0a6ffff700 1 -- 172.21.15.31:0/2168987448 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0a7013fb20 msgr2=0x7f0a70155e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:57.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.642+0000 7f0a6ffff700 1 --2- 172.21.15.31:0/2168987448 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0a7013fb20 0x7f0a70155e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:57.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.642+0000 7f0a6ffff700 1 -- 172.21.15.31:0/2168987448 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a7006aaa0 msgr2=0x7f0a70156b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:57.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.642+0000 7f0a6ffff700 1 --2- 172.21.15.31:0/2168987448 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a7006aaa0 0x7f0a70156b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:57.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.642+0000 7f0a6ffff700 1 -- 172.21.15.31:0/2168987448 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a60009420 con 0x7f0a7013fef0 2023-10-06T02:29:57.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.642+0000 7f0a6ffff700 1 --2- 172.21.15.31:0/2168987448 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a7013fef0 0x7f0a70158450 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f0a68013000 tx=0x7f0a68019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:57.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.643+0000 7f0a577fe700 1 -- 172.21.15.31:0/2168987448 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a68012e10 con 0x7f0a7013fef0 2023-10-06T02:29:57.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.643+0000 7f0a577fe700 1 -- 172.21.15.31:0/2168987448 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a68027470 con 0x7f0a7013fef0 2023-10-06T02:29:57.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.643+0000 7f0a577fe700 1 -- 172.21.15.31:0/2168987448 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a68027830 con 0x7f0a7013fef0 2023-10-06T02:29:57.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.643+0000 7f0a74d9d700 1 -- 172.21.15.31:0/2168987448 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a68009a20 con 0x7f0a7013fef0 2023-10-06T02:29:57.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.643+0000 7f0a74d9d700 1 -- 172.21.15.31:0/2168987448 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a70159820 con 0x7f0a7013fef0 2023-10-06T02:29:57.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.644+0000 7f0a74d9d700 1 -- 172.21.15.31:0/2168987448 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a7004c3c0 con 0x7f0a7013fef0 2023-10-06T02:29:57.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.645+0000 7f0a577fe700 1 -- 172.21.15.31:0/2168987448 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f0a68024070 con 0x7f0a7013fef0 2023-10-06T02:29:57.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.646+0000 7f0a577fe700 1 --2- 172.21.15.31:0/2168987448 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0a5c0674d0 0x7f0a5c069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:57.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.646+0000 7f0a577fe700 1 -- 172.21.15.31:0/2168987448 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6114+0+0 (secure 0 0 0) 0x7f0a68087ba0 con 0x7f0a7013fef0 2023-10-06T02:29:57.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.646+0000 7f0a6f7fe700 1 --2- 172.21.15.31:0/2168987448 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0a5c0674d0 0x7f0a5c069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:57.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.647+0000 7f0a6f7fe700 1 --2- 172.21.15.31:0/2168987448 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0a5c0674d0 0x7f0a5c069980 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0a60009f40 tx=0x7f0a600091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:57.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.650+0000 7f0a577fe700 1 -- 172.21.15.31:0/2168987448 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0a680543c0 con 0x7f0a7013fef0 2023-10-06T02:29:57.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.851+0000 7f0a74d9d700 1 -- 172.21.15.31:0/2168987448 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0a7006ed20 con 0x7f0a7013fef0 2023-10-06T02:29:57.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.852+0000 7f0a577fe700 1 -- 172.21.15.31:0/2168987448 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) v1 ==== 74+0+16943 (secure 0 0 0) 0x7f0a6802c020 con 0x7f0a7013fef0 2023-10-06T02:29:57.853 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:29:57.853 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":121,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:29:54.592304+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:29:33.769454+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-10-06T02:29:34.643236+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":121,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":20,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":20,"snaps":[{"begin":2,"length":4}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:29:57.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.854+0000 7f0a74d9d700 1 -- 172.21.15.31:0/2168987448 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0a5c0674d0 msgr2=0x7f0a5c069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:57.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.854+0000 7f0a74d9d700 1 --2- 172.21.15.31:0/2168987448 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0a5c0674d0 0x7f0a5c069980 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0a60009f40 tx=0x7f0a600091a0 comp rx=0 tx=0).stop 2023-10-06T02:29:57.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.854+0000 7f0a74d9d700 1 -- 172.21.15.31:0/2168987448 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a7013fef0 msgr2=0x7f0a70158450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:57.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.854+0000 7f0a74d9d700 1 --2- 172.21.15.31:0/2168987448 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a7013fef0 0x7f0a70158450 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f0a68013000 tx=0x7f0a68019000 comp rx=0 tx=0).stop 2023-10-06T02:29:57.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.854+0000 7f0a74d9d700 1 -- 172.21.15.31:0/2168987448 shutdown_connections 2023-10-06T02:29:57.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.854+0000 7f0a74d9d700 1 --2- 172.21.15.31:0/2168987448 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a7013fef0 0x7f0a70158450 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:57.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.854+0000 7f0a74d9d700 1 --2- 172.21.15.31:0/2168987448 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a7006aaa0 0x7f0a70156b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:57.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.854+0000 7f0a74d9d700 1 --2- 172.21.15.31:0/2168987448 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0a5c0674d0 0x7f0a5c069980 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:57.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.854+0000 7f0a74d9d700 1 --2- 172.21.15.31:0/2168987448 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0a7013fb20 0x7f0a70155e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:57.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.854+0000 7f0a74d9d700 1 -- 172.21.15.31:0/2168987448 >> 172.21.15.31:0/2168987448 conn(0x7f0a7006d170 msgr2=0x7f0a700c9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:57.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.854+0000 7f0a74d9d700 1 -- 172.21.15.31:0/2168987448 shutdown_connections 2023-10-06T02:29:57.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:57.854+0000 7f0a74d9d700 1 -- 172.21.15.31:0/2168987448 wait complete. 2023-10-06T02:29:57.872 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T02:29:57.873 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:29:58.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.060+0000 7fd17659d700 1 Processor -- start 2023-10-06T02:29:58.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.060+0000 7fd17659d700 1 -- start start 2023-10-06T02:29:58.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.061+0000 7fd17659d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd1700cf110 0x7fd170164600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:58.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.061+0000 7fd17659d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd17015afd0 con 0x7fd17015de50 2023-10-06T02:29:58.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.061+0000 7fd17659d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd17015b2b0 con 0x7fd170153c30 2023-10-06T02:29:58.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.061+0000 7fd17659d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd17015b590 con 0x7fd1700cf110 2023-10-06T02:29:58.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.061+0000 7fd174d9a700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fd17015de50 0x7fd1700cea00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56080/0 (socket says 172.21.15.31:56080) 2023-10-06T02:29:58.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.061+0000 7fd174d9a700 1 -- 172.21.15.31:0/3942397245 learned_addr learned my addr 172.21.15.31:0/3942397245 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:29:58.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.062+0000 7fd175d9c700 1 --2- 172.21.15.31:0/3942397245 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd1700cf110 0x7fd170164600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:58.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.062+0000 7fd167fff700 1 -- 172.21.15.31:0/3942397245 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2484673473 0 0) 0x7fd17015afd0 con 0x7fd17015de50 2023-10-06T02:29:58.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.062+0000 7fd167fff700 1 -- 172.21.15.31:0/3942397245 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd15c003570 con 0x7fd17015de50 2023-10-06T02:29:58.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.062+0000 7fd167fff700 1 -- 172.21.15.31:0/3942397245 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 30599673 0 0) 0x7fd15c003570 con 0x7fd17015de50 2023-10-06T02:29:58.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.062+0000 7fd167fff700 1 -- 172.21.15.31:0/3942397245 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd1700cf110 msgr2=0x7fd170164600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:58.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.062+0000 7fd167fff700 1 --2- 172.21.15.31:0/3942397245 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd1700cf110 0x7fd170164600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:58.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.062+0000 7fd167fff700 1 -- 172.21.15.31:0/3942397245 >> v1:172.21.15.62:6789/0 conn(0x7fd170153c30 legacy=0x7fd17015a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:58.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.062+0000 7fd167fff700 1 -- 172.21.15.31:0/3942397245 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd17015b870 con 0x7fd17015de50 2023-10-06T02:29:58.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.062+0000 7fd167fff700 1 -- 172.21.15.31:0/3942397245 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7fd160001470 con 0x7fd17015de50 2023-10-06T02:29:58.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.062+0000 7fd175d9c700 1 --2- 172.21.15.31:0/3942397245 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd1700cf110 0x7fd170164600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:29:58.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.062+0000 7fd167fff700 1 -- 172.21.15.31:0/3942397245 >> v1:172.21.15.31:6789/0 conn(0x7fd17015de50 legacy=0x7fd1700cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:58.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.062+0000 7fd167fff700 1 --2- 172.21.15.31:0/3942397245 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd1700cf110 0x7fd15c003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:58.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.062+0000 7fd167fff700 1 --2- 172.21.15.31:0/3942397245 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd15c003fd0 0x7fd15c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:58.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.062+0000 7fd167fff700 1 --2- 172.21.15.31:0/3942397245 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd15c004d50 0x7fd15c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:58.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.062+0000 7fd167fff700 1 -- 172.21.15.31:0/3942397245 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd17015b590 con 0x7fd15c004d50 2023-10-06T02:29:58.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.062+0000 7fd167fff700 1 -- 172.21.15.31:0/3942397245 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd15c00bb40 con 0x7fd15c003fd0 2023-10-06T02:29:58.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.062+0000 7fd167fff700 1 -- 172.21.15.31:0/3942397245 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd15c00bd00 con 0x7fd1700cf110 2023-10-06T02:29:58.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.063+0000 7fd175d9c700 1 --2- 172.21.15.31:0/3942397245 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd15c004d50 0x7fd15c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:58.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.063+0000 7fd17559b700 1 --2- 172.21.15.31:0/3942397245 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd1700cf110 0x7fd15c003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:58.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.063+0000 7fd174d9a700 1 --2- 172.21.15.31:0/3942397245 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd15c003fd0 0x7fd15c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:58.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.063+0000 7fd175d9c700 1 -- 172.21.15.31:0/3942397245 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd1700cf110 msgr2=0x7fd15c003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:58.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.063+0000 7fd175d9c700 1 --2- 172.21.15.31:0/3942397245 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd1700cf110 0x7fd15c003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:58.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.063+0000 7fd175d9c700 1 -- 172.21.15.31:0/3942397245 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd15c003fd0 msgr2=0x7fd15c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:58.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.063+0000 7fd175d9c700 1 --2- 172.21.15.31:0/3942397245 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd15c003fd0 0x7fd15c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:58.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.063+0000 7fd175d9c700 1 -- 172.21.15.31:0/3942397245 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd168009a20 con 0x7fd15c004d50 2023-10-06T02:29:58.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.063+0000 7fd175d9c700 1 --2- 172.21.15.31:0/3942397245 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd15c004d50 0x7fd15c0051c0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fd16800e3c0 tx=0x7fd16800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41c4a0faa139cd9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:58.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.063+0000 7fd167fff700 1 -- 172.21.15.31:0/3942397245 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd168009c50 con 0x7fd15c004d50 2023-10-06T02:29:58.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.063+0000 7fd167fff700 1 -- 172.21.15.31:0/3942397245 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd16800f040 con 0x7fd15c004d50 2023-10-06T02:29:58.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.063+0000 7fd167fff700 1 -- 172.21.15.31:0/3942397245 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd16800ee60 con 0x7fd15c004d50 2023-10-06T02:29:58.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.064+0000 7fd17659d700 1 -- 172.21.15.31:0/3942397245 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd15c004d50 msgr2=0x7fd15c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:58.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.064+0000 7fd17659d700 1 --2- 172.21.15.31:0/3942397245 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd15c004d50 0x7fd15c0051c0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fd16800e3c0 tx=0x7fd16800b040 comp rx=0 tx=0).stop 2023-10-06T02:29:58.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.064+0000 7fd175d9c700 1 -- 172.21.15.31:0/3942397245 reap_dead start 2023-10-06T02:29:58.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.064+0000 7fd17659d700 1 -- 172.21.15.31:0/3942397245 shutdown_connections 2023-10-06T02:29:58.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.064+0000 7fd17659d700 1 -- 172.21.15.31:0/3942397245 >> 172.21.15.31:0/3942397245 conn(0x7fd1700703a0 msgr2=0x7fd1700707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:58.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.064+0000 7fd17659d700 1 -- 172.21.15.31:0/3942397245 shutdown_connections 2023-10-06T02:29:58.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.064+0000 7fd17659d700 1 -- 172.21.15.31:0/3942397245 wait complete. 2023-10-06T02:29:58.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.066+0000 7fd17659d700 1 Processor -- start 2023-10-06T02:29:58.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.066+0000 7fd17659d700 1 -- start start 2023-10-06T02:29:58.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.067+0000 7fd17659d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd15c004d50 0x7fd170169620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:58.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.067+0000 7fd17659d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1700d06a0 0x7fd17016cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:58.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.067+0000 7fd17659d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd17016d140 0x7fd170152d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:58.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.067+0000 7fd17659d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1700b32d0 con 0x7fd1700d06a0 2023-10-06T02:29:58.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.067+0000 7fd17659d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1700b3820 con 0x7fd15c004d50 2023-10-06T02:29:58.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.067+0000 7fd17659d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd170169b60 con 0x7fd17016d140 2023-10-06T02:29:58.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.067+0000 7fd174d9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1700d06a0 0x7fd17016cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:58.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.067+0000 7fd174d9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1700d06a0 0x7fd17016cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50722/0 (socket says 172.21.15.31:50722) 2023-10-06T02:29:58.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.067+0000 7fd174d9a700 1 -- 172.21.15.31:0/2662448287 learned_addr learned my addr 172.21.15.31:0/2662448287 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:58.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.067+0000 7fd175d9c700 1 --2- 172.21.15.31:0/2662448287 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd17016d140 0x7fd170152d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:58.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.067+0000 7fd17559b700 1 --2- 172.21.15.31:0/2662448287 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd15c004d50 0x7fd170169620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:58.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.068+0000 7fd174d9a700 1 -- 172.21.15.31:0/2662448287 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd17016d140 msgr2=0x7fd170152d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:58.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.068+0000 7fd174d9a700 1 --2- 172.21.15.31:0/2662448287 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd17016d140 0x7fd170152d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:58.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.068+0000 7fd174d9a700 1 -- 172.21.15.31:0/2662448287 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd15c004d50 msgr2=0x7fd170169620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:58.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.068+0000 7fd174d9a700 1 --2- 172.21.15.31:0/2662448287 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd15c004d50 0x7fd170169620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:58.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.068+0000 7fd174d9a700 1 -- 172.21.15.31:0/2662448287 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd17016d650 con 0x7fd1700d06a0 2023-10-06T02:29:58.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.068+0000 7fd17559b700 1 --2- 172.21.15.31:0/2662448287 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd15c004d50 0x7fd170169620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:29:58.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.068+0000 7fd174d9a700 1 --2- 172.21.15.31:0/2662448287 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1700d06a0 0x7fd17016cc00 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fd16000e800 tx=0x7fd170159c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:58.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.068+0000 7fd1657fa700 1 -- 172.21.15.31:0/2662448287 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd160012500 con 0x7fd1700d06a0 2023-10-06T02:29:58.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.068+0000 7fd1657fa700 1 -- 172.21.15.31:0/2662448287 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd160010040 con 0x7fd1700d06a0 2023-10-06T02:29:58.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.068+0000 7fd1657fa700 1 -- 172.21.15.31:0/2662448287 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1600128b0 con 0x7fd1700d06a0 2023-10-06T02:29:58.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.068+0000 7fd17659d700 1 -- 172.21.15.31:0/2662448287 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd17016d930 con 0x7fd1700d06a0 2023-10-06T02:29:58.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.068+0000 7fd17659d700 1 -- 172.21.15.31:0/2662448287 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1700ca2f0 con 0x7fd1700d06a0 2023-10-06T02:29:58.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.069+0000 7fd17659d700 1 -- 172.21.15.31:0/2662448287 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd17004c3c0 con 0x7fd1700d06a0 2023-10-06T02:29:58.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.070+0000 7fd1657fa700 1 -- 172.21.15.31:0/2662448287 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7fd160012c20 con 0x7fd1700d06a0 2023-10-06T02:29:58.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.071+0000 7fd1657fa700 1 --2- 172.21.15.31:0/2662448287 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd1540674d0 0x7fd154069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:58.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.072+0000 7fd1657fa700 1 -- 172.21.15.31:0/2662448287 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6114+0+0 (secure 0 0 0) 0x7fd160084760 con 0x7fd1700d06a0 2023-10-06T02:29:58.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.072+0000 7fd17559b700 1 --2- 172.21.15.31:0/2662448287 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd1540674d0 0x7fd154069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:58.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.072+0000 7fd17559b700 1 --2- 172.21.15.31:0/2662448287 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd1540674d0 0x7fd154069980 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd16c007e20 tx=0x7fd16c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:58.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.076+0000 7fd1657fa700 1 -- 172.21.15.31:0/2662448287 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd160050e70 con 0x7fd1700d06a0 2023-10-06T02:29:58.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.326+0000 7fd17659d700 1 -- 172.21.15.31:0/2662448287 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd1700b3960 con 0x7fd1700d06a0 2023-10-06T02:29:58.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.327+0000 7fd1657fa700 1 -- 172.21.15.31:0/2662448287 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+4590 (secure 0 0 0) 0x7fd160019090 con 0x7fd1700d06a0 2023-10-06T02:29:58.328 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:29:58.328 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":81,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6338,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/1906951645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":1906951645},{"type":"v1","addr":"172.21.15.62:6834","nonce":1906951645}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75},{"gid":6341,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3788411327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3788411327},{"type":"v1","addr":"172.21.15.62:6835","nonce":3788411327}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75},{"gid":7569,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6833/581939695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":581939695},{"type":"v1","addr":"172.21.15.31:6833","nonce":581939695}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:29:35.664920+0000","modified":"2023-10-06T02:29:39.189818+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7575},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7575":{"gid":7575,"name":"c","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/40412586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":40412586},{"type":"v1","addr":"172.21.15.31:6835","nonce":40412586}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-06T02:29:58.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.329+0000 7fd17659d700 1 -- 172.21.15.31:0/2662448287 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd1540674d0 msgr2=0x7fd154069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:58.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.329+0000 7fd17659d700 1 --2- 172.21.15.31:0/2662448287 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd1540674d0 0x7fd154069980 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd16c007e20 tx=0x7fd16c009040 comp rx=0 tx=0).stop 2023-10-06T02:29:58.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.329+0000 7fd17659d700 1 -- 172.21.15.31:0/2662448287 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1700d06a0 msgr2=0x7fd17016cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:58.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.329+0000 7fd17659d700 1 --2- 172.21.15.31:0/2662448287 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1700d06a0 0x7fd17016cc00 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fd16000e800 tx=0x7fd170159c60 comp rx=0 tx=0).stop 2023-10-06T02:29:58.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.330+0000 7fd17659d700 1 -- 172.21.15.31:0/2662448287 shutdown_connections 2023-10-06T02:29:58.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.330+0000 7fd17659d700 1 --2- 172.21.15.31:0/2662448287 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd1540674d0 0x7fd154069980 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:58.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.330+0000 7fd17659d700 1 --2- 172.21.15.31:0/2662448287 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd17016d140 0x7fd170152d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:58.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.330+0000 7fd17659d700 1 --2- 172.21.15.31:0/2662448287 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd15c004d50 0x7fd170169620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:58.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.330+0000 7fd17659d700 1 --2- 172.21.15.31:0/2662448287 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1700d06a0 0x7fd17016cc00 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:58.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.330+0000 7fd17659d700 1 -- 172.21.15.31:0/2662448287 >> 172.21.15.31:0/2662448287 conn(0x7fd1700703a0 msgr2=0x7fd1700cd120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:58.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.330+0000 7fd17659d700 1 -- 172.21.15.31:0/2662448287 shutdown_connections 2023-10-06T02:29:58.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.330+0000 7fd17659d700 1 -- 172.21.15.31:0/2662448287 wait complete. 2023-10-06T02:29:58.331 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 81 2023-10-06T02:29:58.348 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:29:58.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.535+0000 7f74f759e700 1 Processor -- start 2023-10-06T02:29:58.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.535+0000 7f74f759e700 1 -- start start 2023-10-06T02:29:58.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.537+0000 7f74f759e700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f74f813fef0 0x7f74f815a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:58.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.537+0000 7f74f759e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f74f806aaa0 0x7f74f80c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:58.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.537+0000 7f74f759e700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74f815ae70 con 0x7f74f813fb20 2023-10-06T02:29:58.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.537+0000 7f74f759e700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74f806af60 con 0x7f74f813fef0 2023-10-06T02:29:58.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.537+0000 7f74f759e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74f806b240 con 0x7f74f806aaa0 2023-10-06T02:29:58.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.537+0000 7f74f6d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f74f806aaa0 0x7f74f80c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:58.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.537+0000 7f74f5d9b700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f74f813fef0 0x7f74f815a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:58.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.538+0000 7f74f6d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f74f806aaa0 0x7f74f80c9d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:52026/0 (socket says 172.21.15.31:52026) 2023-10-06T02:29:58.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.538+0000 7f74f6d9d700 1 -- 172.21.15.31:0/3959681285 learned_addr learned my addr 172.21.15.31:0/3959681285 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:58.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.538+0000 7f74f5d9b700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f74f813fef0 0x7f74f815a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:39538/0 (socket says 172.21.15.31:39538) 2023-10-06T02:29:58.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.538+0000 7f74f6d9d700 1 -- 172.21.15.31:0/3959681285 >> v2:172.21.15.62:3300/0 conn(0x7f74f813fef0 msgr2=0x7f74f815a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:58.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.538+0000 7f74f6d9d700 1 --2- 172.21.15.31:0/3959681285 >> v2:172.21.15.62:3300/0 conn(0x7f74f813fef0 0x7f74f815a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:58.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.538+0000 7f74f6d9d700 1 -- 172.21.15.31:0/3959681285 >> v1:172.21.15.62:6789/0 conn(0x7f74f813fb20 legacy=0x7f74f80c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:58.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.538+0000 7f74f6d9d700 1 -- 172.21.15.31:0/3959681285 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74f806b520 con 0x7f74f806aaa0 2023-10-06T02:29:58.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.538+0000 7f74f6d9d700 1 --2- 172.21.15.31:0/3959681285 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f74f806aaa0 0x7f74f80c9d00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f74f000d1a0 tx=0x7f74f000ba50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0037afe08f438bf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:58.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.539+0000 7f74f559a700 1 -- 172.21.15.31:0/3959681285 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74f0010070 con 0x7f74f806aaa0 2023-10-06T02:29:58.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.539+0000 7f74f559a700 1 -- 172.21.15.31:0/3959681285 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74f000de10 con 0x7f74f806aaa0 2023-10-06T02:29:58.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.539+0000 7f74f559a700 1 -- 172.21.15.31:0/3959681285 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74f000b320 con 0x7f74f806aaa0 2023-10-06T02:29:58.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.539+0000 7f74f759e700 1 -- 172.21.15.31:0/3959681285 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f74f806aaa0 msgr2=0x7f74f80c9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:58.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.539+0000 7f74f759e700 1 --2- 172.21.15.31:0/3959681285 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f74f806aaa0 0x7f74f80c9d00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f74f000d1a0 tx=0x7f74f000ba50 comp rx=0 tx=0).stop 2023-10-06T02:29:58.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.539+0000 7f74f759e700 1 -- 172.21.15.31:0/3959681285 shutdown_connections 2023-10-06T02:29:58.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.539+0000 7f74f759e700 1 --2- 172.21.15.31:0/3959681285 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f74f806aaa0 0x7f74f80c9d00 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:58.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.539+0000 7f74f759e700 1 --2- 172.21.15.31:0/3959681285 >> v2:172.21.15.62:3300/0 conn(0x7f74f813fef0 0x7f74f815a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:58.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.539+0000 7f74f759e700 1 -- 172.21.15.31:0/3959681285 >> 172.21.15.31:0/3959681285 conn(0x7f74f806d170 msgr2=0x7f74f80cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:58.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.539+0000 7f74f759e700 1 -- 172.21.15.31:0/3959681285 shutdown_connections 2023-10-06T02:29:58.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.539+0000 7f74f759e700 1 -- 172.21.15.31:0/3959681285 wait complete. 2023-10-06T02:29:58.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.541+0000 7f74f759e700 1 Processor -- start 2023-10-06T02:29:58.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.541+0000 7f74f759e700 1 -- start start 2023-10-06T02:29:58.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.542+0000 7f74f759e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74f806aaa0 0x7f74f813cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:58.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.542+0000 7f74f759e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74f813fb20 0x7f74f80ad0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:58.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.542+0000 7f74f759e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f74f813fef0 0x7f74f80c75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:58.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.542+0000 7f74f759e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74f813d0a0 con 0x7f74f806aaa0 2023-10-06T02:29:58.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.542+0000 7f74f759e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74f8157320 con 0x7f74f813fb20 2023-10-06T02:29:58.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.542+0000 7f74f759e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74f81569f0 con 0x7f74f813fef0 2023-10-06T02:29:58.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.542+0000 7f74f659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74f806aaa0 0x7f74f813cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:58.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.542+0000 7f74f659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74f806aaa0 0x7f74f813cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50738/0 (socket says 172.21.15.31:50738) 2023-10-06T02:29:58.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.542+0000 7f74f659c700 1 -- 172.21.15.31:0/3930774792 learned_addr learned my addr 172.21.15.31:0/3930774792 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:58.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.542+0000 7f74f5d9b700 1 --2- 172.21.15.31:0/3930774792 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74f813fb20 0x7f74f80ad0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:58.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.542+0000 7f74f6d9d700 1 --2- 172.21.15.31:0/3930774792 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f74f813fef0 0x7f74f80c75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:58.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.542+0000 7f74f659c700 1 -- 172.21.15.31:0/3930774792 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f74f813fef0 msgr2=0x7f74f80c75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:58.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.542+0000 7f74f659c700 1 --2- 172.21.15.31:0/3930774792 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f74f813fef0 0x7f74f80c75f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:58.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.543+0000 7f74f659c700 1 -- 172.21.15.31:0/3930774792 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74f813fb20 msgr2=0x7f74f80ad0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:58.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.543+0000 7f74f659c700 1 --2- 172.21.15.31:0/3930774792 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74f813fb20 0x7f74f80ad0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:58.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.543+0000 7f74f659c700 1 -- 172.21.15.31:0/3930774792 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74f000bcf0 con 0x7f74f806aaa0 2023-10-06T02:29:58.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.543+0000 7f74f659c700 1 --2- 172.21.15.31:0/3930774792 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74f806aaa0 0x7f74f813cb60 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f74e800c590 tx=0x7f74e800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:58.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.543+0000 7f74e67fc700 1 -- 172.21.15.31:0/3930774792 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e8011590 con 0x7f74f806aaa0 2023-10-06T02:29:58.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.543+0000 7f74e67fc700 1 -- 172.21.15.31:0/3930774792 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74e8011710 con 0x7f74f806aaa0 2023-10-06T02:29:58.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.543+0000 7f74e67fc700 1 -- 172.21.15.31:0/3930774792 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e8011ad0 con 0x7f74f806aaa0 2023-10-06T02:29:58.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.543+0000 7f74f759e700 1 -- 172.21.15.31:0/3930774792 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74f8156cd0 con 0x7f74f806aaa0 2023-10-06T02:29:58.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.543+0000 7f74f759e700 1 -- 172.21.15.31:0/3930774792 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74f80adba0 con 0x7f74f806aaa0 2023-10-06T02:29:58.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.544+0000 7f74f759e700 1 -- 172.21.15.31:0/3930774792 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74f804c3c0 con 0x7f74f806aaa0 2023-10-06T02:29:58.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.545+0000 7f74e67fc700 1 -- 172.21.15.31:0/3930774792 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f74e801b3b0 con 0x7f74f806aaa0 2023-10-06T02:29:58.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.546+0000 7f74e67fc700 1 --2- 172.21.15.31:0/3930774792 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f74d80674d0 0x7f74d8069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:58.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.546+0000 7f74e67fc700 1 -- 172.21.15.31:0/3930774792 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6114+0+0 (secure 0 0 0) 0x7f74e8084400 con 0x7f74f806aaa0 2023-10-06T02:29:58.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.546+0000 7f74f5d9b700 1 --2- 172.21.15.31:0/3930774792 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f74d80674d0 0x7f74d8069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:58.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.547+0000 7f74f5d9b700 1 --2- 172.21.15.31:0/3930774792 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f74d80674d0 0x7f74d8069980 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f74f80c29e0 tx=0x7f74ec009120 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:58.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.550+0000 7f74e67fc700 1 -- 172.21.15.31:0/3930774792 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f74e8050c60 con 0x7f74f806aaa0 2023-10-06T02:29:58.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.802+0000 7f74f759e700 1 -- 172.21.15.31:0/3930774792 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f74f815ae70 con 0x7f74f806aaa0 2023-10-06T02:29:58.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.803+0000 7f74e67fc700 1 -- 172.21.15.31:0/3930774792 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+4590 (secure 0 0 0) 0x7f74f815ae70 con 0x7f74f806aaa0 2023-10-06T02:29:58.803 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:29:58.803 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":81,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6338,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/1906951645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":1906951645},{"type":"v1","addr":"172.21.15.62:6834","nonce":1906951645}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75},{"gid":6341,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3788411327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3788411327},{"type":"v1","addr":"172.21.15.62:6835","nonce":3788411327}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75},{"gid":7569,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6833/581939695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":581939695},{"type":"v1","addr":"172.21.15.31:6833","nonce":581939695}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:29:35.664920+0000","modified":"2023-10-06T02:29:39.189818+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7575},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7575":{"gid":7575,"name":"c","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/40412586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":40412586},{"type":"v1","addr":"172.21.15.31:6835","nonce":40412586}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-06T02:29:58.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.805+0000 7f74f759e700 1 -- 172.21.15.31:0/3930774792 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f74d80674d0 msgr2=0x7f74d8069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:58.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.805+0000 7f74f759e700 1 --2- 172.21.15.31:0/3930774792 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f74d80674d0 0x7f74d8069980 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f74f80c29e0 tx=0x7f74ec009120 comp rx=0 tx=0).stop 2023-10-06T02:29:58.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.805+0000 7f74f759e700 1 -- 172.21.15.31:0/3930774792 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74f806aaa0 msgr2=0x7f74f813cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:58.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.805+0000 7f74f759e700 1 --2- 172.21.15.31:0/3930774792 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74f806aaa0 0x7f74f813cb60 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f74e800c590 tx=0x7f74e800c8a0 comp rx=0 tx=0).stop 2023-10-06T02:29:58.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.805+0000 7f74f759e700 1 -- 172.21.15.31:0/3930774792 shutdown_connections 2023-10-06T02:29:58.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.805+0000 7f74f759e700 1 --2- 172.21.15.31:0/3930774792 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f74d80674d0 0x7f74d8069980 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:58.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.805+0000 7f74f759e700 1 --2- 172.21.15.31:0/3930774792 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f74f813fef0 0x7f74f80c75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:58.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.805+0000 7f74f759e700 1 --2- 172.21.15.31:0/3930774792 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74f806aaa0 0x7f74f813cb60 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:58.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.805+0000 7f74f759e700 1 --2- 172.21.15.31:0/3930774792 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74f813fb20 0x7f74f80ad0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:58.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.805+0000 7f74f759e700 1 -- 172.21.15.31:0/3930774792 >> 172.21.15.31:0/3930774792 conn(0x7f74f806d170 msgr2=0x7f74f80c9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:58.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.805+0000 7f74f759e700 1 -- 172.21.15.31:0/3930774792 shutdown_connections 2023-10-06T02:29:58.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:58.805+0000 7f74f759e700 1 -- 172.21.15.31:0/3930774792 wait complete. 2023-10-06T02:29:58.807 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 81 2023-10-06T02:29:58.823 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:29:59.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.010+0000 7f910c9fe700 1 Processor -- start 2023-10-06T02:29:59.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.010+0000 7f910c9fe700 1 -- start start 2023-10-06T02:29:59.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.012+0000 7f910c9fe700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f910806aaa0 0x7f910815a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:59.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.012+0000 7f910c9fe700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f910815ae70 con 0x7f910813fb20 2023-10-06T02:29:59.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.012+0000 7f910c9fe700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f910806af60 con 0x7f910813fef0 2023-10-06T02:29:59.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.012+0000 7f910c9fe700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f910806b240 con 0x7f910806aaa0 2023-10-06T02:29:59.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.012+0000 7f9106d9d700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f910813fb20 0x7f91080c7990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56094/0 (socket says 172.21.15.31:56094) 2023-10-06T02:29:59.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.012+0000 7f9106d9d700 1 -- 172.21.15.31:0/1326024700 learned_addr learned my addr 172.21.15.31:0/1326024700 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:29:59.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.012+0000 7f910759e700 1 --2- 172.21.15.31:0/1326024700 >> v2:172.21.15.62:3300/0 conn(0x7f910806aaa0 0x7f910815a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:59.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.012+0000 7f9105d9b700 1 -- 172.21.15.31:0/1326024700 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4011359050 0 0) 0x7f90f8001110 con 0x7f910813fb20 2023-10-06T02:29:59.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.012+0000 7f9105d9b700 1 -- 172.21.15.31:0/1326024700 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90f0003570 con 0x7f910813fb20 2023-10-06T02:29:59.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.013+0000 7f9105d9b700 1 -- 172.21.15.31:0/1326024700 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1330497007 0 0) 0x7f90f0003570 con 0x7f910813fb20 2023-10-06T02:29:59.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.013+0000 7f9105d9b700 1 -- 172.21.15.31:0/1326024700 >> v2:172.21.15.62:3300/0 conn(0x7f910806aaa0 msgr2=0x7f910815a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:59.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.013+0000 7f9105d9b700 1 --2- 172.21.15.31:0/1326024700 >> v2:172.21.15.62:3300/0 conn(0x7f910806aaa0 0x7f910815a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:59.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.013+0000 7f9105d9b700 1 -- 172.21.15.31:0/1326024700 >> v1:172.21.15.62:6789/0 conn(0x7f910813fef0 legacy=0x7f91080c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:59.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.013+0000 7f9105d9b700 1 -- 172.21.15.31:0/1326024700 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f910806b520 con 0x7f910813fb20 2023-10-06T02:29:59.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.013+0000 7f9105d9b700 1 -- 172.21.15.31:0/1326024700 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7f90f8001600 con 0x7f910813fb20 2023-10-06T02:29:59.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.013+0000 7f9105d9b700 1 -- 172.21.15.31:0/1326024700 >> v1:172.21.15.31:6789/0 conn(0x7f910813fb20 legacy=0x7f91080c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:59.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.013+0000 7f9105d9b700 1 --2- 172.21.15.31:0/1326024700 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f90f0003a60 0x7f90f0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:59.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.013+0000 7f9105d9b700 1 --2- 172.21.15.31:0/1326024700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90f0004810 0x7f90f0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:59.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.013+0000 7f9105d9b700 1 --2- 172.21.15.31:0/1326024700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90f00051c0 0x7f90f000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:59.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.013+0000 7f9105d9b700 1 -- 172.21.15.31:0/1326024700 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f910806b240 con 0x7f90f00051c0 2023-10-06T02:29:59.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.013+0000 7f9105d9b700 1 -- 172.21.15.31:0/1326024700 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f000c0d0 con 0x7f90f0004810 2023-10-06T02:29:59.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.013+0000 7f9105d9b700 1 -- 172.21.15.31:0/1326024700 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90f000c290 con 0x7f90f0003a60 2023-10-06T02:29:59.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.013+0000 7f910759e700 1 --2- 172.21.15.31:0/1326024700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90f00051c0 0x7f90f000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:59.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.013+0000 7f9106d9d700 1 --2- 172.21.15.31:0/1326024700 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f90f0003a60 0x7f90f0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:59.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.013+0000 7f910659c700 1 --2- 172.21.15.31:0/1326024700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90f0004810 0x7f90f0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:59.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.013+0000 7f910759e700 1 -- 172.21.15.31:0/1326024700 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f90f0003a60 msgr2=0x7f90f0003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:59.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.013+0000 7f910759e700 1 --2- 172.21.15.31:0/1326024700 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f90f0003a60 0x7f90f0003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:59.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.014+0000 7f910759e700 1 -- 172.21.15.31:0/1326024700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90f0004810 msgr2=0x7f90f0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:59.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.014+0000 7f910759e700 1 --2- 172.21.15.31:0/1326024700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90f0004810 0x7f90f0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:59.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.014+0000 7f910759e700 1 -- 172.21.15.31:0/1326024700 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90f0003570 con 0x7f90f00051c0 2023-10-06T02:29:59.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.014+0000 7f910759e700 1 -- 172.21.15.31:0/1326024700 reap_dead start 2023-10-06T02:29:59.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.014+0000 7f910759e700 1 --2- 172.21.15.31:0/1326024700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90f00051c0 0x7f90f000ba60 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f9108069db0 tx=0x7f910000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2a10f12cd487d09 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:59.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.014+0000 7f9105d9b700 1 -- 172.21.15.31:0/1326024700 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9100010ca0 con 0x7f90f00051c0 2023-10-06T02:29:59.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.014+0000 7f9105d9b700 1 -- 172.21.15.31:0/1326024700 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9100010e00 con 0x7f90f00051c0 2023-10-06T02:29:59.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.014+0000 7f9105d9b700 1 -- 172.21.15.31:0/1326024700 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f910001a680 con 0x7f90f00051c0 2023-10-06T02:29:59.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.014+0000 7f910c9fe700 1 -- 172.21.15.31:0/1326024700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90f00051c0 msgr2=0x7f90f000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:59.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.014+0000 7f910c9fe700 1 --2- 172.21.15.31:0/1326024700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90f00051c0 0x7f90f000ba60 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f9108069db0 tx=0x7f910000b040 comp rx=0 tx=0).stop 2023-10-06T02:29:59.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.015+0000 7f910c9fe700 1 -- 172.21.15.31:0/1326024700 shutdown_connections 2023-10-06T02:29:59.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.015+0000 7f910c9fe700 1 --2- 172.21.15.31:0/1326024700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90f00051c0 0x7f90f000ba60 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:59.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.015+0000 7f910c9fe700 1 -- 172.21.15.31:0/1326024700 >> 172.21.15.31:0/1326024700 conn(0x7f910806d170 msgr2=0x7f91080cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:59.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.015+0000 7f910c9fe700 1 -- 172.21.15.31:0/1326024700 shutdown_connections 2023-10-06T02:29:59.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.015+0000 7f910c9fe700 1 -- 172.21.15.31:0/1326024700 wait complete. 2023-10-06T02:29:59.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.016+0000 7f910c9fe700 1 Processor -- start 2023-10-06T02:29:59.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.016+0000 7f910c9fe700 1 -- start start 2023-10-06T02:29:59.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.017+0000 7f910c9fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90f00051c0 0x7f9108157cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:59.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.017+0000 7f910c9fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9108149d30 0x7f91080b00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:59.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.017+0000 7f910c9fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91080b05e0 0x7f91081656d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:59.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.017+0000 7f910c9fe700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91080b0a50 con 0x7f91080b05e0 2023-10-06T02:29:59.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.017+0000 7f910c9fe700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f910814a1a0 con 0x7f90f00051c0 2023-10-06T02:29:59.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.017+0000 7f910c9fe700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9108156950 con 0x7f9108149d30 2023-10-06T02:29:59.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.017+0000 7f910759e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91080b05e0 0x7f91081656d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:59.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.017+0000 7f910759e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91080b05e0 0x7f91081656d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50752/0 (socket says 172.21.15.31:50752) 2023-10-06T02:29:59.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.017+0000 7f910759e700 1 -- 172.21.15.31:0/2369958170 learned_addr learned my addr 172.21.15.31:0/2369958170 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:59.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.018+0000 7f9106d9d700 1 --2- 172.21.15.31:0/2369958170 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90f00051c0 0x7f9108157cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:59.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.018+0000 7f910659c700 1 --2- 172.21.15.31:0/2369958170 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9108149d30 0x7f91080b00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:59.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.018+0000 7f910759e700 1 -- 172.21.15.31:0/2369958170 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9108149d30 msgr2=0x7f91080b00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:59.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.018+0000 7f910759e700 1 --2- 172.21.15.31:0/2369958170 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9108149d30 0x7f91080b00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:59.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.018+0000 7f910759e700 1 -- 172.21.15.31:0/2369958170 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90f00051c0 msgr2=0x7f9108157cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:59.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.018+0000 7f910759e700 1 --2- 172.21.15.31:0/2369958170 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90f00051c0 0x7f9108157cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:59.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.018+0000 7f910759e700 1 -- 172.21.15.31:0/2369958170 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9108156c30 con 0x7f91080b05e0 2023-10-06T02:29:59.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.018+0000 7f910759e700 1 --2- 172.21.15.31:0/2369958170 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91080b05e0 0x7f91081656d0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f910000afd0 tx=0x7f910001b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:59.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.018+0000 7f90eeffd700 1 -- 172.21.15.31:0/2369958170 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9100010ca0 con 0x7f91080b05e0 2023-10-06T02:29:59.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.018+0000 7f90eeffd700 1 -- 172.21.15.31:0/2369958170 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9100012c90 con 0x7f91080b05e0 2023-10-06T02:29:59.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.018+0000 7f90eeffd700 1 -- 172.21.15.31:0/2369958170 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9100027690 con 0x7f91080b05e0 2023-10-06T02:29:59.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.018+0000 7f910c9fe700 1 -- 172.21.15.31:0/2369958170 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9108156f10 con 0x7f91080b05e0 2023-10-06T02:29:59.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.019+0000 7f910c9fe700 1 -- 172.21.15.31:0/2369958170 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91081574d0 con 0x7f91080b05e0 2023-10-06T02:29:59.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.019+0000 7f910c9fe700 1 -- 172.21.15.31:0/2369958170 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f910804c3c0 con 0x7f91080b05e0 2023-10-06T02:29:59.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.020+0000 7f90eeffd700 1 -- 172.21.15.31:0/2369958170 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f9100003020 con 0x7f91080b05e0 2023-10-06T02:29:59.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.021+0000 7f90eeffd700 1 --2- 172.21.15.31:0/2369958170 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f90f40674d0 0x7f90f4069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:59.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.021+0000 7f90eeffd700 1 -- 172.21.15.31:0/2369958170 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6114+0+0 (secure 0 0 0) 0x7f9100016070 con 0x7f91080b05e0 2023-10-06T02:29:59.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.022+0000 7f9106d9d700 1 --2- 172.21.15.31:0/2369958170 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f90f40674d0 0x7f90f4069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:59.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.023+0000 7f9106d9d700 1 --2- 172.21.15.31:0/2369958170 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f90f40674d0 0x7f90f4069980 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f90f8000e10 tx=0x7f90f8009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:59.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.025+0000 7f90eeffd700 1 -- 172.21.15.31:0/2369958170 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9100055d70 con 0x7f91080b05e0 2023-10-06T02:29:59.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.228+0000 7f910c9fe700 1 -- 172.21.15.31:0/2369958170 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f910806b800 con 0x7f91080b05e0 2023-10-06T02:29:59.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.228+0000 7f90eeffd700 1 -- 172.21.15.31:0/2369958170 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) v1 ==== 74+0+16943 (secure 0 0 0) 0x7f9100059390 con 0x7f91080b05e0 2023-10-06T02:29:59.229 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:29:59.229 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":121,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:29:54.592304+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:29:33.769454+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-10-06T02:29:34.643236+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":121,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":20,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":20,"snaps":[{"begin":2,"length":4}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:29:59.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.231+0000 7f910c9fe700 1 -- 172.21.15.31:0/2369958170 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f90f40674d0 msgr2=0x7f90f4069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:59.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.231+0000 7f910c9fe700 1 --2- 172.21.15.31:0/2369958170 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f90f40674d0 0x7f90f4069980 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f90f8000e10 tx=0x7f90f8009040 comp rx=0 tx=0).stop 2023-10-06T02:29:59.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.231+0000 7f910c9fe700 1 -- 172.21.15.31:0/2369958170 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91080b05e0 msgr2=0x7f91081656d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:59.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.231+0000 7f910c9fe700 1 --2- 172.21.15.31:0/2369958170 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91080b05e0 0x7f91081656d0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f910000afd0 tx=0x7f910001b000 comp rx=0 tx=0).stop 2023-10-06T02:29:59.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.231+0000 7f910c9fe700 1 -- 172.21.15.31:0/2369958170 shutdown_connections 2023-10-06T02:29:59.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.231+0000 7f910c9fe700 1 --2- 172.21.15.31:0/2369958170 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91080b05e0 0x7f91081656d0 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:59.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.231+0000 7f910c9fe700 1 --2- 172.21.15.31:0/2369958170 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90f00051c0 0x7f9108157cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:59.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.231+0000 7f910c9fe700 1 --2- 172.21.15.31:0/2369958170 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f90f40674d0 0x7f90f4069980 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:59.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.231+0000 7f910c9fe700 1 --2- 172.21.15.31:0/2369958170 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9108149d30 0x7f91080b00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:59.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.231+0000 7f910c9fe700 1 -- 172.21.15.31:0/2369958170 >> 172.21.15.31:0/2369958170 conn(0x7f910806d170 msgr2=0x7f91080c2540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:59.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.231+0000 7f910c9fe700 1 -- 172.21.15.31:0/2369958170 shutdown_connections 2023-10-06T02:29:59.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.231+0000 7f910c9fe700 1 -- 172.21.15.31:0/2369958170 wait complete. 2023-10-06T02:29:59.250 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T02:29:59.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.438+0000 7fd0c3fff700 1 Processor -- start 2023-10-06T02:29:59.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.438+0000 7fd0c3fff700 1 -- start start 2023-10-06T02:29:59.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.439+0000 7fd0c3fff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd0c4159640 0x7fd0c406d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:59.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.439+0000 7fd0c3fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd0c40caec0 0x7fd0c40c78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:59.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.439+0000 7fd0c3fff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0c40c7420 con 0x7fd0c4159a10 2023-10-06T02:29:59.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.439+0000 7fd0c3fff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0c406d5a0 con 0x7fd0c40caec0 2023-10-06T02:29:59.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.439+0000 7fd0c3fff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0c406d820 con 0x7fd0c4159640 2023-10-06T02:29:59.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.440+0000 7fd0c37fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd0c40caec0 0x7fd0c40c78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:59.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.440+0000 7fd0c37fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd0c40caec0 0x7fd0c40c78b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50762/0 (socket says 172.21.15.31:50762) 2023-10-06T02:29:59.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.440+0000 7fd0c2ffd700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd0c4159640 0x7fd0c406d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:59.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.440+0000 7fd0c37fe700 1 -- 172.21.15.31:0/1879980890 learned_addr learned my addr 172.21.15.31:0/1879980890 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:59.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.440+0000 7fd0c27fc700 1 --1- 172.21.15.31:0/1879980890 >> v1:172.21.15.31:6789/0 conn(0x7fd0c4159a10 0x7fd0c40ca7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56110/0 (socket says 172.21.15.31:56110) 2023-10-06T02:29:59.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.440+0000 7fd0c37fe700 1 -- 172.21.15.31:0/1879980890 >> v2:172.21.15.62:3300/0 conn(0x7fd0c4159640 msgr2=0x7fd0c406d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:59.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.440+0000 7fd0c37fe700 1 --2- 172.21.15.31:0/1879980890 >> v2:172.21.15.62:3300/0 conn(0x7fd0c4159640 0x7fd0c406d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:59.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.440+0000 7fd0c37fe700 1 -- 172.21.15.31:0/1879980890 >> v1:172.21.15.31:6789/0 conn(0x7fd0c4159a10 legacy=0x7fd0c40ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:59.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.440+0000 7fd0c37fe700 1 -- 172.21.15.31:0/1879980890 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0c406db00 con 0x7fd0c40caec0 2023-10-06T02:29:59.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.441+0000 7fd0c37fe700 1 --2- 172.21.15.31:0/1879980890 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0c40caec0 0x7fd0c40c78b0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc009f20 tx=0x7fd0bc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7867cf17ed30f76 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:59.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.441+0000 7fd0c1ffb700 1 -- 172.21.15.31:0/1879980890 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0bc00e070 con 0x7fd0c40caec0 2023-10-06T02:29:59.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.441+0000 7fd0c1ffb700 1 -- 172.21.15.31:0/1879980890 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0bc00b990 con 0x7fd0c40caec0 2023-10-06T02:29:59.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.441+0000 7fd0c1ffb700 1 -- 172.21.15.31:0/1879980890 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0bc0092e0 con 0x7fd0c40caec0 2023-10-06T02:29:59.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.441+0000 7fd0c3fff700 1 -- 172.21.15.31:0/1879980890 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0c40caec0 msgr2=0x7fd0c40c78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:59.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.441+0000 7fd0c3fff700 1 --2- 172.21.15.31:0/1879980890 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0c40caec0 0x7fd0c40c78b0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc009f20 tx=0x7fd0bc009840 comp rx=0 tx=0).stop 2023-10-06T02:29:59.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.441+0000 7fd0c3fff700 1 -- 172.21.15.31:0/1879980890 shutdown_connections 2023-10-06T02:29:59.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.441+0000 7fd0c3fff700 1 --2- 172.21.15.31:0/1879980890 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0c40caec0 0x7fd0c40c78b0 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:59.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.441+0000 7fd0c3fff700 1 --2- 172.21.15.31:0/1879980890 >> v2:172.21.15.62:3300/0 conn(0x7fd0c4159640 0x7fd0c406d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:59.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.441+0000 7fd0c3fff700 1 -- 172.21.15.31:0/1879980890 >> 172.21.15.31:0/1879980890 conn(0x7fd0c406e720 msgr2=0x7fd0c406eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:59.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.442+0000 7fd0c3fff700 1 -- 172.21.15.31:0/1879980890 shutdown_connections 2023-10-06T02:29:59.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.442+0000 7fd0c3fff700 1 -- 172.21.15.31:0/1879980890 wait complete. 2023-10-06T02:29:59.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.443+0000 7fd0c3fff700 1 Processor -- start 2023-10-06T02:29:59.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.443+0000 7fd0c3fff700 1 -- start start 2023-10-06T02:29:59.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.444+0000 7fd0c3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0c40caec0 0x7fd0c416ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:59.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.444+0000 7fd0c3fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0c4159640 0x7fd0c413bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:59.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.444+0000 7fd0c3fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd0c4159a10 0x7fd0c414f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:59.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.444+0000 7fd0c3fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c4152da0 con 0x7fd0c40caec0 2023-10-06T02:29:59.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.444+0000 7fd0c3fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c414e9b0 con 0x7fd0c4159640 2023-10-06T02:29:59.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.444+0000 7fd0c3fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c40c1f60 con 0x7fd0c4159a10 2023-10-06T02:29:59.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.444+0000 7fd0c2ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0c40caec0 0x7fd0c416ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:59.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.444+0000 7fd0c2ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0c40caec0 0x7fd0c416ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50764/0 (socket says 172.21.15.31:50764) 2023-10-06T02:29:59.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.444+0000 7fd0c2ffd700 1 -- 172.21.15.31:0/1235497579 learned_addr learned my addr 172.21.15.31:0/1235497579 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:59.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.444+0000 7fd0c27fc700 1 --2- 172.21.15.31:0/1235497579 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0c4159640 0x7fd0c413bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:59.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.445+0000 7fd0c37fe700 1 --2- 172.21.15.31:0/1235497579 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd0c4159a10 0x7fd0c414f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:59.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.445+0000 7fd0c2ffd700 1 -- 172.21.15.31:0/1235497579 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd0c4159a10 msgr2=0x7fd0c414f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:59.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.445+0000 7fd0c2ffd700 1 --2- 172.21.15.31:0/1235497579 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd0c4159a10 0x7fd0c414f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:59.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.445+0000 7fd0c2ffd700 1 -- 172.21.15.31:0/1235497579 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0c4159640 msgr2=0x7fd0c413bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:59.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.445+0000 7fd0c2ffd700 1 --2- 172.21.15.31:0/1235497579 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0c4159640 0x7fd0c413bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:59.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.445+0000 7fd0c2ffd700 1 -- 172.21.15.31:0/1235497579 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0bc009ae0 con 0x7fd0c40caec0 2023-10-06T02:29:59.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.445+0000 7fd0c27fc700 1 --2- 172.21.15.31:0/1235497579 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0c4159640 0x7fd0c413bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:29:59.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.445+0000 7fd0c2ffd700 1 --2- 172.21.15.31:0/1235497579 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0c40caec0 0x7fd0c416ad10 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fd0b400ec80 tx=0x7fd0b40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:59.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.445+0000 7fd0aaffd700 1 -- 172.21.15.31:0/1235497579 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b400c9d0 con 0x7fd0c40caec0 2023-10-06T02:29:59.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.445+0000 7fd0aaffd700 1 -- 172.21.15.31:0/1235497579 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0b400cb30 con 0x7fd0c40caec0 2023-10-06T02:29:59.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.445+0000 7fd0aaffd700 1 -- 172.21.15.31:0/1235497579 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b4013470 con 0x7fd0c40caec0 2023-10-06T02:29:59.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.445+0000 7fd0c3fff700 1 -- 172.21.15.31:0/1235497579 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0c4152ac0 con 0x7fd0c40caec0 2023-10-06T02:29:59.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.445+0000 7fd0c3fff700 1 -- 172.21.15.31:0/1235497579 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0c40c1bf0 con 0x7fd0c40caec0 2023-10-06T02:29:59.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.446+0000 7fd0c3fff700 1 -- 172.21.15.31:0/1235497579 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0c40c2240 con 0x7fd0c40caec0 2023-10-06T02:29:59.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.447+0000 7fd0aaffd700 1 -- 172.21.15.31:0/1235497579 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7fd0b4013820 con 0x7fd0c40caec0 2023-10-06T02:29:59.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.448+0000 7fd0aaffd700 1 --2- 172.21.15.31:0/1235497579 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd0b00674d0 0x7fd0b0069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:59.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.448+0000 7fd0aaffd700 1 -- 172.21.15.31:0/1235497579 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6114+0+0 (secure 0 0 0) 0x7fd0b4085510 con 0x7fd0c40caec0 2023-10-06T02:29:59.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.449+0000 7fd0c27fc700 1 --2- 172.21.15.31:0/1235497579 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd0b00674d0 0x7fd0b0069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:59.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.449+0000 7fd0c27fc700 1 --2- 172.21.15.31:0/1235497579 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd0b00674d0 0x7fd0b0069980 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd0b8007fd0 tx=0x7fd0b8007e10 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:59.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.452+0000 7fd0aaffd700 1 -- 172.21.15.31:0/1235497579 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd0b4051d30 con 0x7fd0c40caec0 2023-10-06T02:29:59.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.699+0000 7fd0c3fff700 1 -- 172.21.15.31:0/1235497579 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fd0c404c3c0 con 0x7fd0c40caec0 2023-10-06T02:29:59.700 INFO:tasks.ceph.mon.a.smithi031.stderr:2023-10-06T02:29:59.700+0000 7f6de01f1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T02:29:59.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.708+0000 7fd0aaffd700 1 -- 172.21.15.31:0/1235497579 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v82) v1 ==== 161+0+0 (secure 0 0 0) 0x7fd0b4055350 con 0x7fd0c40caec0 2023-10-06T02:29:59.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.711+0000 7fd0c3fff700 1 -- 172.21.15.31:0/1235497579 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd0b00674d0 msgr2=0x7fd0b0069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:59.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.711+0000 7fd0c3fff700 1 --2- 172.21.15.31:0/1235497579 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd0b00674d0 0x7fd0b0069980 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd0b8007fd0 tx=0x7fd0b8007e10 comp rx=0 tx=0).stop 2023-10-06T02:29:59.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.711+0000 7fd0c3fff700 1 -- 172.21.15.31:0/1235497579 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0c40caec0 msgr2=0x7fd0c416ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:59.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.711+0000 7fd0c3fff700 1 --2- 172.21.15.31:0/1235497579 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0c40caec0 0x7fd0c416ad10 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fd0b400ec80 tx=0x7fd0b40091a0 comp rx=0 tx=0).stop 2023-10-06T02:29:59.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.711+0000 7fd0c3fff700 1 -- 172.21.15.31:0/1235497579 shutdown_connections 2023-10-06T02:29:59.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.711+0000 7fd0c3fff700 1 --2- 172.21.15.31:0/1235497579 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd0b00674d0 0x7fd0b0069980 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:59.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.711+0000 7fd0c3fff700 1 --2- 172.21.15.31:0/1235497579 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd0c4159a10 0x7fd0c414f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:59.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.711+0000 7fd0c3fff700 1 --2- 172.21.15.31:0/1235497579 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0c40caec0 0x7fd0c416ad10 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:59.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.711+0000 7fd0c3fff700 1 --2- 172.21.15.31:0/1235497579 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0c4159640 0x7fd0c413bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:59.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.711+0000 7fd0c3fff700 1 -- 172.21.15.31:0/1235497579 >> 172.21.15.31:0/1235497579 conn(0x7fd0c406e720 msgr2=0x7fd0c41587f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:59.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.711+0000 7fd0c3fff700 1 -- 172.21.15.31:0/1235497579 shutdown_connections 2023-10-06T02:29:59.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.711+0000 7fd0c3fff700 1 -- 172.21.15.31:0/1235497579 wait complete. 2023-10-06T02:29:59.713 INFO:teuthology.orchestra.run.smithi031.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T02:29:59.731 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T02:29:59.771 INFO:tasks.ceph.mds.c.smithi031.stdout:starting mds.c at 2023-10-06T02:29:59.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.923+0000 7f7d5359e700 1 Processor -- start 2023-10-06T02:29:59.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.923+0000 7f7d5359e700 1 -- start start 2023-10-06T02:29:59.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.924+0000 7f7d5359e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7d54153c90 0x7f7d5415a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:59.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.924+0000 7f7d5359e700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7d5406db20 0x7f7d540ca7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:59.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.924+0000 7f7d5359e700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d5415ada0 con 0x7f7d54154060 2023-10-06T02:29:59.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.924+0000 7f7d5359e700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d5406ca80 con 0x7f7d5406db20 2023-10-06T02:29:59.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.924+0000 7f7d5359e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5406cd60 con 0x7f7d54153c90 2023-10-06T02:29:59.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.925+0000 7f7d51d9b700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7d5406db20 0x7f7d540ca7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:59.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.925+0000 7f7d5259c700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7d54153c90 0x7f7d5415a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:59.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.925+0000 7f7d51d9b700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7d5406db20 0x7f7d540ca7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:39604/0 (socket says 172.21.15.31:39604) 2023-10-06T02:29:59.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.925+0000 7f7d51d9b700 1 -- 172.21.15.31:0/1082904352 learned_addr learned my addr 172.21.15.31:0/1082904352 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:59.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.925+0000 7f7d51d9b700 1 -- 172.21.15.31:0/1082904352 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7d54153c90 msgr2=0x7f7d5415a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:29:59.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.925+0000 7f7d51d9b700 1 --2- 172.21.15.31:0/1082904352 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7d54153c90 0x7f7d5415a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:59.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.925+0000 7f7d51d9b700 1 -- 172.21.15.31:0/1082904352 >> v1:172.21.15.62:6789/0 conn(0x7f7d54154060 legacy=0x7f7d5406a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:59.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.925+0000 7f7d51d9b700 1 -- 172.21.15.31:0/1082904352 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d5406d040 con 0x7f7d5406db20 2023-10-06T02:29:59.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.925+0000 7f7d5259c700 1 --2- 172.21.15.31:0/1082904352 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7d54153c90 0x7f7d5415a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:29:59.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.926+0000 7f7d51d9b700 1 --2- 172.21.15.31:0/1082904352 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d5406db20 0x7f7d540ca7b0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f7d480099b0 tx=0x7f7d48009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2fa0c74a0a752d8b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:59.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.926+0000 7f7d5159a700 1 -- 172.21.15.31:0/1082904352 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d48009ea0 con 0x7f7d5406db20 2023-10-06T02:29:59.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.926+0000 7f7d5159a700 1 -- 172.21.15.31:0/1082904352 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d4800b860 con 0x7f7d5406db20 2023-10-06T02:29:59.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.926+0000 7f7d5159a700 1 -- 172.21.15.31:0/1082904352 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d4800bc00 con 0x7f7d5406db20 2023-10-06T02:29:59.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.927+0000 7f7d5359e700 1 -- 172.21.15.31:0/1082904352 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d5406db20 msgr2=0x7f7d540ca7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:59.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.927+0000 7f7d5359e700 1 --2- 172.21.15.31:0/1082904352 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d5406db20 0x7f7d540ca7b0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f7d480099b0 tx=0x7f7d48009250 comp rx=0 tx=0).stop 2023-10-06T02:29:59.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.927+0000 7f7d5359e700 1 -- 172.21.15.31:0/1082904352 shutdown_connections 2023-10-06T02:29:59.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.927+0000 7f7d5359e700 1 --2- 172.21.15.31:0/1082904352 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7d54153c90 0x7f7d5415a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:59.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.927+0000 7f7d5359e700 1 --2- 172.21.15.31:0/1082904352 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d5406db20 0x7f7d540ca7b0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:59.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.927+0000 7f7d5359e700 1 -- 172.21.15.31:0/1082904352 >> 172.21.15.31:0/1082904352 conn(0x7f7d5406e720 msgr2=0x7f7d5406eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:29:59.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.927+0000 7f7d5359e700 1 -- 172.21.15.31:0/1082904352 shutdown_connections 2023-10-06T02:29:59.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.927+0000 7f7d5359e700 1 -- 172.21.15.31:0/1082904352 wait complete. 2023-10-06T02:29:59.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.928+0000 7f7d5359e700 1 Processor -- start 2023-10-06T02:29:59.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.928+0000 7f7d5359e700 1 -- start start 2023-10-06T02:29:59.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.929+0000 7f7d5359e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d5406db20 0x7f7d540b1340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:59.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.929+0000 7f7d5359e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d54153c90 0x7f7d540b02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:59.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.929+0000 7f7d5359e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7d54154060 0x7f7d5414c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:59.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.929+0000 7f7d5359e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d540b1880 con 0x7f7d54153c90 2023-10-06T02:29:59.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.929+0000 7f7d5359e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d540b26c0 con 0x7f7d5406db20 2023-10-06T02:29:59.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.929+0000 7f7d5359e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d540b07f0 con 0x7f7d54154060 2023-10-06T02:29:59.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.930+0000 7f7d51d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d54153c90 0x7f7d540b02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:59.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.930+0000 7f7d51d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d54153c90 0x7f7d540b02b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50790/0 (socket says 172.21.15.31:50790) 2023-10-06T02:29:59.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.930+0000 7f7d51d9b700 1 -- 172.21.15.31:0/2718138224 learned_addr learned my addr 172.21.15.31:0/2718138224 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:29:59.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.930+0000 7f7d5259c700 1 --2- 172.21.15.31:0/2718138224 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d5406db20 0x7f7d540b1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:59.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.930+0000 7f7d52d9d700 1 --2- 172.21.15.31:0/2718138224 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7d54154060 0x7f7d5414c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:59.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.930+0000 7f7d51d9b700 1 -- 172.21.15.31:0/2718138224 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7d54154060 msgr2=0x7f7d5414c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:59.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.930+0000 7f7d51d9b700 1 --2- 172.21.15.31:0/2718138224 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7d54154060 0x7f7d5414c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:59.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.930+0000 7f7d51d9b700 1 -- 172.21.15.31:0/2718138224 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d5406db20 msgr2=0x7f7d540b1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:29:59.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.930+0000 7f7d51d9b700 1 --2- 172.21.15.31:0/2718138224 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d5406db20 0x7f7d540b1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:29:59.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.930+0000 7f7d51d9b700 1 -- 172.21.15.31:0/2718138224 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d44009420 con 0x7f7d54153c90 2023-10-06T02:29:59.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.930+0000 7f7d52d9d700 1 --2- 172.21.15.31:0/2718138224 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7d54154060 0x7f7d5414c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:29:59.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.930+0000 7f7d51d9b700 1 --2- 172.21.15.31:0/2718138224 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d54153c90 0x7f7d540b02b0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f7d540c3820 tx=0x7f7d48017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:59.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.931+0000 7f7d427fc700 1 -- 172.21.15.31:0/2718138224 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d4800bbe0 con 0x7f7d54153c90 2023-10-06T02:29:59.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.931+0000 7f7d427fc700 1 -- 172.21.15.31:0/2718138224 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d48011420 con 0x7f7d54153c90 2023-10-06T02:29:59.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.931+0000 7f7d427fc700 1 -- 172.21.15.31:0/2718138224 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d480117c0 con 0x7f7d54153c90 2023-10-06T02:29:59.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.931+0000 7f7d5359e700 1 -- 172.21.15.31:0/2718138224 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d480094f0 con 0x7f7d54153c90 2023-10-06T02:29:59.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.931+0000 7f7d5359e700 1 -- 172.21.15.31:0/2718138224 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d540b1ab0 con 0x7f7d54153c90 2023-10-06T02:29:59.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.932+0000 7f7d5359e700 1 -- 172.21.15.31:0/2718138224 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d540b2830 con 0x7f7d54153c90 2023-10-06T02:29:59.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.933+0000 7f7d427fc700 1 -- 172.21.15.31:0/2718138224 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f7d48011bf0 con 0x7f7d54153c90 2023-10-06T02:29:59.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.934+0000 7f7d427fc700 1 --2- 172.21.15.31:0/2718138224 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7d340674d0 0x7f7d34069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:29:59.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.934+0000 7f7d427fc700 1 -- 172.21.15.31:0/2718138224 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6172+0+0 (secure 0 0 0) 0x7f7d480868a0 con 0x7f7d54153c90 2023-10-06T02:29:59.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.934+0000 7f7d5259c700 1 --2- 172.21.15.31:0/2718138224 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7d340674d0 0x7f7d34069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:29:59.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.935+0000 7f7d5259c700 1 --2- 172.21.15.31:0/2718138224 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7d340674d0 0x7f7d34069980 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7d540c6b80 tx=0x7f7d440091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:29:59.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:29:59.938+0000 7f7d427fc700 1 -- 172.21.15.31:0/2718138224 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7d48053170 con 0x7f7d54153c90 2023-10-06T02:30:00.000 INFO:tasks.ceph.mon.a.smithi031.stderr:2023-10-06T02:29:59.999+0000 7f6de29f6700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2023-10-06T02:30:00.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:00.186+0000 7f7d5359e700 1 -- 172.21.15.31:0/2718138224 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f7d540ca450 con 0x7f7d54153c90 2023-10-06T02:30:00.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:00.707+0000 7f7d427fc700 1 -- 172.21.15.31:0/2718138224 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v83) v1 ==== 104+0+0 (secure 0 0 0) 0x7f7d48056790 con 0x7f7d54153c90 2023-10-06T02:30:00.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:00.710+0000 7f7d5359e700 1 -- 172.21.15.31:0/2718138224 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7d340674d0 msgr2=0x7f7d34069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:00.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:00.710+0000 7f7d5359e700 1 --2- 172.21.15.31:0/2718138224 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7d340674d0 0x7f7d34069980 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7d540c6b80 tx=0x7f7d440091a0 comp rx=0 tx=0).stop 2023-10-06T02:30:00.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:00.710+0000 7f7d5359e700 1 -- 172.21.15.31:0/2718138224 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d54153c90 msgr2=0x7f7d540b02b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:00.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:00.710+0000 7f7d5359e700 1 --2- 172.21.15.31:0/2718138224 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d54153c90 0x7f7d540b02b0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f7d540c3820 tx=0x7f7d48017000 comp rx=0 tx=0).stop 2023-10-06T02:30:00.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:00.710+0000 7f7d5359e700 1 -- 172.21.15.31:0/2718138224 shutdown_connections 2023-10-06T02:30:00.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:00.710+0000 7f7d5359e700 1 --2- 172.21.15.31:0/2718138224 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7d340674d0 0x7f7d34069980 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:00.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:00.710+0000 7f7d5359e700 1 --2- 172.21.15.31:0/2718138224 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7d54154060 0x7f7d5414c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:00.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:00.710+0000 7f7d5359e700 1 --2- 172.21.15.31:0/2718138224 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d5406db20 0x7f7d540b1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:00.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:00.710+0000 7f7d5359e700 1 --2- 172.21.15.31:0/2718138224 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d54153c90 0x7f7d540b02b0 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:00.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:00.710+0000 7f7d5359e700 1 -- 172.21.15.31:0/2718138224 >> 172.21.15.31:0/2718138224 conn(0x7f7d5406e720 msgr2=0x7f7d5415feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:00.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:00.711+0000 7f7d5359e700 1 -- 172.21.15.31:0/2718138224 shutdown_connections 2023-10-06T02:30:00.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:00.711+0000 7f7d5359e700 1 -- 172.21.15.31:0/2718138224 wait complete. 2023-10-06T02:30:00.731 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T02:30:00.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:00.977+0000 7f63e1f9d700 1 Processor -- start 2023-10-06T02:30:00.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:00.977+0000 7f63e1f9d700 1 -- start start 2023-10-06T02:30:00.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:00.978+0000 7f63e1f9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f63dc153ac0 0x7f63dc15a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:00.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:00.978+0000 7f63e1f9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63dc15abf0 con 0x7f63dc153e90 2023-10-06T02:30:00.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:00.978+0000 7f63e1f9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63dc070940 con 0x7f63dc0cacb0 2023-10-06T02:30:00.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:00.978+0000 7f63e1f9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63dc06a330 con 0x7f63dc153ac0 2023-10-06T02:30:00.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:00.978+0000 7f63dbfff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f63dc153e90 0x7f63dc0ca5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56120/0 (socket says 172.21.15.31:56120) 2023-10-06T02:30:00.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:00.978+0000 7f63dbfff700 1 -- 172.21.15.31:0/2546301269 learned_addr learned my addr 172.21.15.31:0/2546301269 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:30:00.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:00.979+0000 7f63e0f9b700 1 --2- 172.21.15.31:0/2546301269 >> v2:172.21.15.62:3300/0 conn(0x7f63dc153ac0 0x7f63dc15a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:00.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:00.979+0000 7f63db7fe700 1 -- 172.21.15.31:0/2546301269 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2006978105 0 0) 0x7f63dc15abf0 con 0x7f63dc153e90 2023-10-06T02:30:00.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:00.979+0000 7f63db7fe700 1 -- 172.21.15.31:0/2546301269 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63cc003570 con 0x7f63dc153e90 2023-10-06T02:30:00.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:00.979+0000 7f63e0f9b700 1 -- 172.21.15.31:0/2546301269 >> v1:172.21.15.62:6789/0 conn(0x7f63dc0cacb0 legacy=0x7f63dc06c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:00.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:00.979+0000 7f63e0f9b700 1 -- 172.21.15.31:0/2546301269 >> v1:172.21.15.31:6789/0 conn(0x7f63dc153e90 legacy=0x7f63dc0ca5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:00.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:00.979+0000 7f63e0f9b700 1 -- 172.21.15.31:0/2546301269 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63dc06a610 con 0x7f63dc153ac0 2023-10-06T02:30:00.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:00.979+0000 7f63e0f9b700 1 --2- 172.21.15.31:0/2546301269 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63dc153ac0 0x7f63dc15a6b0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f63d0009e60 tx=0x7f63d0009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca0bddd279f43616 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:00.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:00.980+0000 7f63db7fe700 1 -- 172.21.15.31:0/2546301269 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d000e070 con 0x7f63dc153ac0 2023-10-06T02:30:00.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:00.980+0000 7f63db7fe700 1 -- 172.21.15.31:0/2546301269 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63d000bb60 con 0x7f63dc153ac0 2023-10-06T02:30:00.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:00.980+0000 7f63db7fe700 1 -- 172.21.15.31:0/2546301269 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d0012430 con 0x7f63dc153ac0 2023-10-06T02:30:00.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:00.981+0000 7f63e1f9d700 1 -- 172.21.15.31:0/2546301269 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63dc153ac0 msgr2=0x7f63dc15a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:00.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:00.981+0000 7f63e1f9d700 1 --2- 172.21.15.31:0/2546301269 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63dc153ac0 0x7f63dc15a6b0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f63d0009e60 tx=0x7f63d0009780 comp rx=0 tx=0).stop 2023-10-06T02:30:00.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:00.981+0000 7f63e1f9d700 1 -- 172.21.15.31:0/2546301269 shutdown_connections 2023-10-06T02:30:00.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:00.981+0000 7f63e1f9d700 1 --2- 172.21.15.31:0/2546301269 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63dc153ac0 0x7f63dc15a6b0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:00.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:00.981+0000 7f63e1f9d700 1 -- 172.21.15.31:0/2546301269 >> 172.21.15.31:0/2546301269 conn(0x7f63dc06e790 msgr2=0x7f63dc06eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:00.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:00.981+0000 7f63e1f9d700 1 -- 172.21.15.31:0/2546301269 shutdown_connections 2023-10-06T02:30:00.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:00.981+0000 7f63e1f9d700 1 -- 172.21.15.31:0/2546301269 wait complete. 2023-10-06T02:30:00.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:00.982+0000 7f63e1f9d700 1 Processor -- start 2023-10-06T02:30:00.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:00.982+0000 7f63e1f9d700 1 -- start start 2023-10-06T02:30:00.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:00.983+0000 7f63e1f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f63dc0cacb0 0x7f63dc0c6f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:00.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:00.983+0000 7f63e1f9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63dc153ac0 0x7f63dc0c9a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:00.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:00.983+0000 7f63e1f9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63dc153e90 0x7f63dc15cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:00.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:00.983+0000 7f63e1f9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63dc0c9f60 con 0x7f63dc153ac0 2023-10-06T02:30:00.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:00.983+0000 7f63e1f9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63dc16d4a0 con 0x7f63dc153e90 2023-10-06T02:30:00.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:00.983+0000 7f63e1f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63dc0c7620 con 0x7f63dc0cacb0 2023-10-06T02:30:00.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:00.984+0000 7f63dbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63dc153ac0 0x7f63dc0c9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:00.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:00.984+0000 7f63dbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63dc153ac0 0x7f63dc0c9a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50798/0 (socket says 172.21.15.31:50798) 2023-10-06T02:30:00.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:00.984+0000 7f63dbfff700 1 -- 172.21.15.31:0/3128257530 learned_addr learned my addr 172.21.15.31:0/3128257530 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:00.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:00.984+0000 7f63e0f9b700 1 --2- 172.21.15.31:0/3128257530 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f63dc0cacb0 0x7f63dc0c6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:00.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:00.984+0000 7f63e179c700 1 --2- 172.21.15.31:0/3128257530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63dc153e90 0x7f63dc15cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:00.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:00.984+0000 7f63dbfff700 1 -- 172.21.15.31:0/3128257530 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f63dc0cacb0 msgr2=0x7f63dc0c6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:00.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:00.984+0000 7f63dbfff700 1 --2- 172.21.15.31:0/3128257530 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f63dc0cacb0 0x7f63dc0c6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:00.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:00.984+0000 7f63dbfff700 1 -- 172.21.15.31:0/3128257530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63dc153e90 msgr2=0x7f63dc15cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:00.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:00.984+0000 7f63dbfff700 1 --2- 172.21.15.31:0/3128257530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63dc153e90 0x7f63dc15cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:00.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:00.984+0000 7f63dbfff700 1 -- 172.21.15.31:0/3128257530 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63d0009a20 con 0x7f63dc153ac0 2023-10-06T02:30:00.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:00.984+0000 7f63dbfff700 1 --2- 172.21.15.31:0/3128257530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63dc153ac0 0x7f63dc0c9a20 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f63c800c560 tx=0x7f63c8007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:00.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:00.985+0000 7f63d8ff9700 1 -- 172.21.15.31:0/3128257530 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c8005b30 con 0x7f63dc153ac0 2023-10-06T02:30:00.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:00.985+0000 7f63d8ff9700 1 -- 172.21.15.31:0/3128257530 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63c8005c90 con 0x7f63dc153ac0 2023-10-06T02:30:00.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:00.985+0000 7f63d8ff9700 1 -- 172.21.15.31:0/3128257530 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c8007300 con 0x7f63dc153ac0 2023-10-06T02:30:00.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:00.985+0000 7f63e1f9d700 1 -- 172.21.15.31:0/3128257530 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63dc0c7900 con 0x7f63dc153ac0 2023-10-06T02:30:00.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:00.985+0000 7f63e1f9d700 1 -- 172.21.15.31:0/3128257530 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63dc15d060 con 0x7f63dc153ac0 2023-10-06T02:30:00.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:00.986+0000 7f63e1f9d700 1 -- 172.21.15.31:0/3128257530 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63dc0c7e20 con 0x7f63dc153ac0 2023-10-06T02:30:00.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:00.987+0000 7f63d8ff9700 1 -- 172.21.15.31:0/3128257530 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f63c801b3b0 con 0x7f63dc153ac0 2023-10-06T02:30:00.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:00.988+0000 7f63d8ff9700 1 --2- 172.21.15.31:0/3128257530 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f63c00674d0 0x7f63c0069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:00.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:00.988+0000 7f63d8ff9700 1 -- 172.21.15.31:0/3128257530 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6172+0+0 (secure 0 0 0) 0x7f63c8084540 con 0x7f63dc153ac0 2023-10-06T02:30:00.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:00.988+0000 7f63e0f9b700 1 --2- 172.21.15.31:0/3128257530 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f63c00674d0 0x7f63c0069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:00.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:00.989+0000 7f63e0f9b700 1 --2- 172.21.15.31:0/3128257530 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f63c00674d0 0x7f63c0069980 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f63d00097c0 tx=0x7f63d0019000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:00.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:00.992+0000 7f63d8ff9700 1 -- 172.21.15.31:0/3128257530 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f63dc0c7e20 con 0x7f63dc153ac0 2023-10-06T02:30:01.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:01.201+0000 7f63e1f9d700 1 -- 172.21.15.31:0/3128257530 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f63dc16d270 con 0x7f63dc153ac0 2023-10-06T02:30:01.963 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T02:30:01.968 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T02:30:02.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:02.077+0000 7f63d8ff9700 1 -- 172.21.15.31:0/3128257530 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v123) v1 ==== 181+0+0 (secure 0 0 0) 0x7f63dc16d270 con 0x7f63dc153ac0 2023-10-06T02:30:02.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:02.081+0000 7f63e1f9d700 1 -- 172.21.15.31:0/3128257530 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f63c00674d0 msgr2=0x7f63c0069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:02.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:02.081+0000 7f63e1f9d700 1 --2- 172.21.15.31:0/3128257530 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f63c00674d0 0x7f63c0069980 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f63d00097c0 tx=0x7f63d0019000 comp rx=0 tx=0).stop 2023-10-06T02:30:02.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:02.081+0000 7f63e1f9d700 1 -- 172.21.15.31:0/3128257530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63dc153ac0 msgr2=0x7f63dc0c9a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:02.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:02.081+0000 7f63e1f9d700 1 --2- 172.21.15.31:0/3128257530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63dc153ac0 0x7f63dc0c9a20 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f63c800c560 tx=0x7f63c8007dd0 comp rx=0 tx=0).stop 2023-10-06T02:30:02.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:02.081+0000 7f63e1f9d700 1 -- 172.21.15.31:0/3128257530 shutdown_connections 2023-10-06T02:30:02.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:02.081+0000 7f63e1f9d700 1 --2- 172.21.15.31:0/3128257530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63dc153e90 0x7f63dc15cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:02.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:02.081+0000 7f63e1f9d700 1 --2- 172.21.15.31:0/3128257530 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f63c00674d0 0x7f63c0069980 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:02.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:02.081+0000 7f63e1f9d700 1 --2- 172.21.15.31:0/3128257530 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f63dc0cacb0 0x7f63dc0c6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:02.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:02.081+0000 7f63e1f9d700 1 --2- 172.21.15.31:0/3128257530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63dc153ac0 0x7f63dc0c9a20 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:02.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:02.081+0000 7f63e1f9d700 1 -- 172.21.15.31:0/3128257530 >> 172.21.15.31:0/3128257530 conn(0x7f63dc06e790 msgr2=0x7f63dc0c5240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:02.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:02.081+0000 7f63e1f9d700 1 -- 172.21.15.31:0/3128257530 shutdown_connections 2023-10-06T02:30:02.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:02.081+0000 7f63e1f9d700 1 -- 172.21.15.31:0/3128257530 wait complete. 2023-10-06T02:30:02.083 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' removed 2023-10-06T02:30:02.101 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T02:30:02.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:02.334+0000 7f0fc159d700 1 Processor -- start 2023-10-06T02:30:02.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:02.334+0000 7f0fc159d700 1 -- start start 2023-10-06T02:30:02.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:02.336+0000 7f0fc159d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0fbc0c8200 0x7f0fbc0c1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:02.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:02.336+0000 7f0fc159d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0fbc0c85d0 0x7f0fbc0cc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:02.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:02.336+0000 7f0fc159d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fbc1571b0 con 0x7f0fbc0c2a90 2023-10-06T02:30:02.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:02.336+0000 7f0fc159d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fbc1579b0 con 0x7f0fbc0c85d0 2023-10-06T02:30:02.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:02.336+0000 7f0fc159d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fbc0c1390 con 0x7f0fbc0c8200 2023-10-06T02:30:02.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:02.336+0000 7f0fc0d9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0fbc0c85d0 0x7f0fbc0cc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:02.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:02.336+0000 7f0fc0d9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0fbc0c85d0 0x7f0fbc0cc2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50804/0 (socket says 172.21.15.31:50804) 2023-10-06T02:30:02.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:02.336+0000 7f0fc0d9c700 1 -- 172.21.15.31:0/3424692083 learned_addr learned my addr 172.21.15.31:0/3424692083 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:02.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:02.336+0000 7f0fbbfff700 1 --2- 172.21.15.31:0/3424692083 >> v2:172.21.15.62:3300/0 conn(0x7f0fbc0c8200 0x7f0fbc0c1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:02.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:02.337+0000 7f0fc0d9c700 1 -- 172.21.15.31:0/3424692083 >> v2:172.21.15.62:3300/0 conn(0x7f0fbc0c8200 msgr2=0x7f0fbc0c1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:02.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:02.337+0000 7f0fc0d9c700 1 --2- 172.21.15.31:0/3424692083 >> v2:172.21.15.62:3300/0 conn(0x7f0fbc0c8200 0x7f0fbc0c1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:02.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:02.337+0000 7f0fc0d9c700 1 -- 172.21.15.31:0/3424692083 >> v1:172.21.15.62:6789/0 conn(0x7f0fbc0c2a90 legacy=0x7f0fbc156aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:02.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:02.337+0000 7f0fc0d9c700 1 -- 172.21.15.31:0/3424692083 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fbc1573f0 con 0x7f0fbc0c85d0 2023-10-06T02:30:02.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:02.337+0000 7f0fc0d9c700 1 --2- 172.21.15.31:0/3424692083 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0fbc0c85d0 0x7f0fbc0cc2f0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc0683d0 tx=0x7f0fb4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc85760a09feaf94 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:02.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:02.337+0000 7f0fbaffd700 1 -- 172.21.15.31:0/3424692083 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb40092e0 con 0x7f0fbc0c85d0 2023-10-06T02:30:02.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:02.337+0000 7f0fbaffd700 1 -- 172.21.15.31:0/3424692083 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fb400ba10 con 0x7f0fbc0c85d0 2023-10-06T02:30:02.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:02.337+0000 7f0fbaffd700 1 -- 172.21.15.31:0/3424692083 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb400bdd0 con 0x7f0fbc0c85d0 2023-10-06T02:30:02.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:02.338+0000 7f0fc159d700 1 -- 172.21.15.31:0/3424692083 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0fbc0c85d0 msgr2=0x7f0fbc0cc2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:02.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:02.338+0000 7f0fc159d700 1 --2- 172.21.15.31:0/3424692083 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0fbc0c85d0 0x7f0fbc0cc2f0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc0683d0 tx=0x7f0fb4009840 comp rx=0 tx=0).stop 2023-10-06T02:30:02.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:02.338+0000 7f0fc159d700 1 -- 172.21.15.31:0/3424692083 shutdown_connections 2023-10-06T02:30:02.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:02.338+0000 7f0fc159d700 1 --2- 172.21.15.31:0/3424692083 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0fbc0c85d0 0x7f0fbc0cc2f0 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:02.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:02.338+0000 7f0fc159d700 1 --2- 172.21.15.31:0/3424692083 >> v2:172.21.15.62:3300/0 conn(0x7f0fbc0c8200 0x7f0fbc0c1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:02.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:02.338+0000 7f0fc159d700 1 -- 172.21.15.31:0/3424692083 >> 172.21.15.31:0/3424692083 conn(0x7f0fbc06c6a0 msgr2=0x7f0fbc06caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:02.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:02.338+0000 7f0fc159d700 1 -- 172.21.15.31:0/3424692083 shutdown_connections 2023-10-06T02:30:02.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:02.338+0000 7f0fc159d700 1 -- 172.21.15.31:0/3424692083 wait complete. 2023-10-06T02:30:02.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:02.339+0000 7f0fc159d700 1 Processor -- start 2023-10-06T02:30:02.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:02.339+0000 7f0fc159d700 1 -- start start 2023-10-06T02:30:02.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:02.340+0000 7f0fc159d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0fbc0c2a90 0x7f0fbc14a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:02.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:02.340+0000 7f0fc159d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0fbc0c8200 0x7f0fbc0ac0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:02.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:02.340+0000 7f0fc159d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0fbc0c85d0 0x7f0fbc0adc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:02.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:02.340+0000 7f0fc159d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fbc0aa290 con 0x7f0fbc0c2a90 2023-10-06T02:30:02.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:02.340+0000 7f0fc159d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fbc0ab0a0 con 0x7f0fbc0c85d0 2023-10-06T02:30:02.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:02.340+0000 7f0fc159d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fbc0aa490 con 0x7f0fbc0c8200 2023-10-06T02:30:02.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:02.341+0000 7f0fbbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0fbc0c2a90 0x7f0fbc14a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:02.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:02.341+0000 7f0fbbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0fbc0c2a90 0x7f0fbc14a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50818/0 (socket says 172.21.15.31:50818) 2023-10-06T02:30:02.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:02.341+0000 7f0fbbfff700 1 -- 172.21.15.31:0/3708125755 learned_addr learned my addr 172.21.15.31:0/3708125755 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:02.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:02.341+0000 7f0fc0d9c700 1 --2- 172.21.15.31:0/3708125755 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0fbc0c85d0 0x7f0fbc0adc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:02.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:02.341+0000 7f0fbb7fe700 1 --2- 172.21.15.31:0/3708125755 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0fbc0c8200 0x7f0fbc0ac0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:02.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:02.341+0000 7f0fbbfff700 1 -- 172.21.15.31:0/3708125755 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0fbc0c8200 msgr2=0x7f0fbc0ac0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:02.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:02.341+0000 7f0fbbfff700 1 --2- 172.21.15.31:0/3708125755 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0fbc0c8200 0x7f0fbc0ac0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:02.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:02.341+0000 7f0fbbfff700 1 -- 172.21.15.31:0/3708125755 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0fbc0c85d0 msgr2=0x7f0fbc0adc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:02.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:02.341+0000 7f0fbbfff700 1 --2- 172.21.15.31:0/3708125755 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0fbc0c85d0 0x7f0fbc0adc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:02.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:02.341+0000 7f0fbbfff700 1 -- 172.21.15.31:0/3708125755 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fb4009ae0 con 0x7f0fbc0c2a90 2023-10-06T02:30:02.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:02.341+0000 7f0fc0d9c700 1 --2- 172.21.15.31:0/3708125755 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0fbc0c85d0 0x7f0fbc0adc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:30:02.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:02.341+0000 7f0fbbfff700 1 --2- 172.21.15.31:0/3708125755 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0fbc0c2a90 0x7f0fbc14a5d0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f0fac00d890 tx=0x7f0fac008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:02.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:02.342+0000 7f0fa3fff700 1 -- 172.21.15.31:0/3708125755 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fac0125f0 con 0x7f0fbc0c2a90 2023-10-06T02:30:02.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:02.342+0000 7f0fa3fff700 1 -- 172.21.15.31:0/3708125755 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fac012770 con 0x7f0fbc0c2a90 2023-10-06T02:30:02.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:02.342+0000 7f0fa3fff700 1 -- 172.21.15.31:0/3708125755 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fac012b30 con 0x7f0fbc0c2a90 2023-10-06T02:30:02.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:02.342+0000 7f0fc159d700 1 -- 172.21.15.31:0/3708125755 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fbc0aa770 con 0x7f0fbc0c2a90 2023-10-06T02:30:02.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:02.342+0000 7f0fc159d700 1 -- 172.21.15.31:0/3708125755 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fbc0aad30 con 0x7f0fbc0c2a90 2023-10-06T02:30:02.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:02.343+0000 7f0fc159d700 1 -- 172.21.15.31:0/3708125755 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fbc0c25d0 con 0x7f0fbc0c2a90 2023-10-06T02:30:02.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:02.344+0000 7f0fa3fff700 1 -- 172.21.15.31:0/3708125755 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f0fac01e460 con 0x7f0fbc0c2a90 2023-10-06T02:30:02.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:02.345+0000 7f0fa3fff700 1 --2- 172.21.15.31:0/3708125755 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0fa80674d0 0x7f0fa8069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:02.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:02.345+0000 7f0fa3fff700 1 -- 172.21.15.31:0/3708125755 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5734+0+0 (secure 0 0 0) 0x7f0fac085680 con 0x7f0fbc0c2a90 2023-10-06T02:30:02.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:02.345+0000 7f0fbb7fe700 1 --2- 172.21.15.31:0/3708125755 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0fa80674d0 0x7f0fa8069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:02.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:02.346+0000 7f0fbb7fe700 1 --2- 172.21.15.31:0/3708125755 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0fa80674d0 0x7f0fa8069980 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc158d40 tx=0x7f0fb0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:02.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:02.349+0000 7f0fa3fff700 1 -- 172.21.15.31:0/3708125755 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0fac0520e0 con 0x7f0fbc0c2a90 2023-10-06T02:30:02.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:02.560+0000 7f0fc159d700 1 -- 172.21.15.31:0/3708125755 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0fbc04c3c0 con 0x7f0fbc0c2a90 2023-10-06T02:30:03.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.084+0000 7f0fa3fff700 1 -- 172.21.15.31:0/3708125755 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v124) v1 ==== 169+0+0 (secure 0 0 0) 0x7f0fac055700 con 0x7f0fbc0c2a90 2023-10-06T02:30:03.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.087+0000 7f0fc159d700 1 -- 172.21.15.31:0/3708125755 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0fa80674d0 msgr2=0x7f0fa8069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:03.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.087+0000 7f0fc159d700 1 --2- 172.21.15.31:0/3708125755 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0fa80674d0 0x7f0fa8069980 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc158d40 tx=0x7f0fb0009040 comp rx=0 tx=0).stop 2023-10-06T02:30:03.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.087+0000 7f0fc159d700 1 -- 172.21.15.31:0/3708125755 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0fbc0c2a90 msgr2=0x7f0fbc14a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:03.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.087+0000 7f0fc159d700 1 --2- 172.21.15.31:0/3708125755 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0fbc0c2a90 0x7f0fbc14a5d0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f0fac00d890 tx=0x7f0fac008040 comp rx=0 tx=0).stop 2023-10-06T02:30:03.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.088+0000 7f0fc159d700 1 -- 172.21.15.31:0/3708125755 shutdown_connections 2023-10-06T02:30:03.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.088+0000 7f0fc159d700 1 --2- 172.21.15.31:0/3708125755 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0fbc0c85d0 0x7f0fbc0adc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:03.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.088+0000 7f0fc159d700 1 --2- 172.21.15.31:0/3708125755 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0fbc0c2a90 0x7f0fbc14a5d0 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:03.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.088+0000 7f0fc159d700 1 --2- 172.21.15.31:0/3708125755 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0fa80674d0 0x7f0fa8069980 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:03.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.088+0000 7f0fc159d700 1 --2- 172.21.15.31:0/3708125755 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0fbc0c8200 0x7f0fbc0ac0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:03.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.088+0000 7f0fc159d700 1 -- 172.21.15.31:0/3708125755 >> 172.21.15.31:0/3708125755 conn(0x7f0fbc06c6a0 msgr2=0x7f0fbc06b0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:03.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.088+0000 7f0fc159d700 1 -- 172.21.15.31:0/3708125755 shutdown_connections 2023-10-06T02:30:03.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.088+0000 7f0fc159d700 1 -- 172.21.15.31:0/3708125755 wait complete. 2023-10-06T02:30:03.090 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' removed 2023-10-06T02:30:03.108 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T02:30:03.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.298+0000 7f99d9d9d700 1 Processor -- start 2023-10-06T02:30:03.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.298+0000 7f99d9d9d700 1 -- start start 2023-10-06T02:30:03.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.300+0000 7f99d9d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f99d4159540 0x7f99d40c1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:03.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.300+0000 7f99d9d9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f99d40c6be0 0x7f99d40c7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:03.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.300+0000 7f99d9d9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99d40c7580 con 0x7f99d4159910 2023-10-06T02:30:03.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.300+0000 7f99d9d9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99d4068250 con 0x7f99d40c6be0 2023-10-06T02:30:03.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.300+0000 7f99d9d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99d4068530 con 0x7f99d4159540 2023-10-06T02:30:03.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.300+0000 7f99d959c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f99d40c6be0 0x7f99d40c7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:03.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.300+0000 7f99d959c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f99d40c6be0 0x7f99d40c7040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50824/0 (socket says 172.21.15.31:50824) 2023-10-06T02:30:03.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.300+0000 7f99d959c700 1 -- 172.21.15.31:0/270332665 learned_addr learned my addr 172.21.15.31:0/270332665 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:03.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.300+0000 7f99d8d9b700 1 --2- 172.21.15.31:0/270332665 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f99d4159540 0x7f99d40c1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:03.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.300+0000 7f99d959c700 1 -- 172.21.15.31:0/270332665 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f99d4159540 msgr2=0x7f99d40c1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:03.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.300+0000 7f99d959c700 1 --2- 172.21.15.31:0/270332665 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f99d4159540 0x7f99d40c1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:03.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.301+0000 7f99d959c700 1 -- 172.21.15.31:0/270332665 >> v1:172.21.15.62:6789/0 conn(0x7f99d4159910 legacy=0x7f99d40677b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:03.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.301+0000 7f99d959c700 1 -- 172.21.15.31:0/270332665 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99d4068810 con 0x7f99d40c6be0 2023-10-06T02:30:03.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.301+0000 7f99d8d9b700 1 --2- 172.21.15.31:0/270332665 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f99d4159540 0x7f99d40c1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:30:03.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.301+0000 7f99d959c700 1 --2- 172.21.15.31:0/270332665 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99d40c6be0 0x7f99d40c7040 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f99cc009f20 tx=0x7f99cc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfcbd734c00f263f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:03.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.301+0000 7f99d37fe700 1 -- 172.21.15.31:0/270332665 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99cc00e070 con 0x7f99d40c6be0 2023-10-06T02:30:03.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.301+0000 7f99d37fe700 1 -- 172.21.15.31:0/270332665 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99cc00ba40 con 0x7f99d40c6be0 2023-10-06T02:30:03.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.301+0000 7f99d37fe700 1 -- 172.21.15.31:0/270332665 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99cc0092e0 con 0x7f99d40c6be0 2023-10-06T02:30:03.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.302+0000 7f99d9d9d700 1 -- 172.21.15.31:0/270332665 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99d40c6be0 msgr2=0x7f99d40c7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:03.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.302+0000 7f99d9d9d700 1 --2- 172.21.15.31:0/270332665 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99d40c6be0 0x7f99d40c7040 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f99cc009f20 tx=0x7f99cc009840 comp rx=0 tx=0).stop 2023-10-06T02:30:03.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.302+0000 7f99d9d9d700 1 -- 172.21.15.31:0/270332665 shutdown_connections 2023-10-06T02:30:03.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.302+0000 7f99d9d9d700 1 --2- 172.21.15.31:0/270332665 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99d40c6be0 0x7f99d40c7040 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:03.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.302+0000 7f99d9d9d700 1 --2- 172.21.15.31:0/270332665 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f99d4159540 0x7f99d40c1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:03.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.302+0000 7f99d9d9d700 1 -- 172.21.15.31:0/270332665 >> 172.21.15.31:0/270332665 conn(0x7f99d406f330 msgr2=0x7f99d40cc5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:03.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.302+0000 7f99d9d9d700 1 -- 172.21.15.31:0/270332665 shutdown_connections 2023-10-06T02:30:03.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.302+0000 7f99d9d9d700 1 -- 172.21.15.31:0/270332665 wait complete. 2023-10-06T02:30:03.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.303+0000 7f99d9d9d700 1 Processor -- start 2023-10-06T02:30:03.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.303+0000 7f99d9d9d700 1 -- start start 2023-10-06T02:30:03.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.304+0000 7f99d9d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f99d40c6be0 0x7f99d4069ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:03.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.304+0000 7f99d9d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99d4159540 0x7f99d414ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:03.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.304+0000 7f99d9d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99d4159910 0x7f99d40af590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:03.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.304+0000 7f99d9d9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99d406b400 con 0x7f99d4159910 2023-10-06T02:30:03.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.304+0000 7f99d9d9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99d406a7f0 con 0x7f99d4159540 2023-10-06T02:30:03.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.304+0000 7f99d9d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99d406aad0 con 0x7f99d40c6be0 2023-10-06T02:30:03.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.305+0000 7f99d959c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99d4159910 0x7f99d40af590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:03.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.305+0000 7f99d959c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99d4159910 0x7f99d40af590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50832/0 (socket says 172.21.15.31:50832) 2023-10-06T02:30:03.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.305+0000 7f99d959c700 1 -- 172.21.15.31:0/3991511702 learned_addr learned my addr 172.21.15.31:0/3991511702 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:03.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.305+0000 7f99d8d9b700 1 --2- 172.21.15.31:0/3991511702 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f99d40c6be0 0x7f99d4069ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:03.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.305+0000 7f99d3fff700 1 --2- 172.21.15.31:0/3991511702 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99d4159540 0x7f99d414ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:03.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.305+0000 7f99d959c700 1 -- 172.21.15.31:0/3991511702 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f99d40c6be0 msgr2=0x7f99d4069ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:03.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.305+0000 7f99d959c700 1 --2- 172.21.15.31:0/3991511702 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f99d40c6be0 0x7f99d4069ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:03.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.305+0000 7f99d959c700 1 -- 172.21.15.31:0/3991511702 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99d4159540 msgr2=0x7f99d414ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:03.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.305+0000 7f99d959c700 1 --2- 172.21.15.31:0/3991511702 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99d4159540 0x7f99d414ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:03.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.305+0000 7f99d959c700 1 -- 172.21.15.31:0/3991511702 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99cc009ae0 con 0x7f99d4159910 2023-10-06T02:30:03.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.305+0000 7f99d959c700 1 --2- 172.21.15.31:0/3991511702 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99d4159910 0x7f99d40af590 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f99d4071320 tx=0x7f99cc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:03.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.306+0000 7f99d0ff9700 1 -- 172.21.15.31:0/3991511702 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99cc0032f0 con 0x7f99d4159910 2023-10-06T02:30:03.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.306+0000 7f99d0ff9700 1 -- 172.21.15.31:0/3991511702 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99cc012e60 con 0x7f99d4159910 2023-10-06T02:30:03.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.306+0000 7f99d9d9d700 1 -- 172.21.15.31:0/3991511702 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99d406adb0 con 0x7f99d4159910 2023-10-06T02:30:03.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.306+0000 7f99d0ff9700 1 -- 172.21.15.31:0/3991511702 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99cc01d670 con 0x7f99d4159910 2023-10-06T02:30:03.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.306+0000 7f99d9d9d700 1 -- 172.21.15.31:0/3991511702 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99d406a430 con 0x7f99d4159910 2023-10-06T02:30:03.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.307+0000 7f99d9d9d700 1 -- 172.21.15.31:0/3991511702 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99d406b570 con 0x7f99d4159910 2023-10-06T02:30:03.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.308+0000 7f99d0ff9700 1 -- 172.21.15.31:0/3991511702 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f99cc01da60 con 0x7f99d4159910 2023-10-06T02:30:03.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.309+0000 7f99d0ff9700 1 --2- 172.21.15.31:0/3991511702 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f99b80674d0 0x7f99b8069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:03.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.309+0000 7f99d0ff9700 1 -- 172.21.15.31:0/3991511702 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5320+0+0 (secure 0 0 0) 0x7f99cc086710 con 0x7f99d4159910 2023-10-06T02:30:03.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.309+0000 7f99d8d9b700 1 --2- 172.21.15.31:0/3991511702 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f99b80674d0 0x7f99b8069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:03.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.310+0000 7f99d8d9b700 1 --2- 172.21.15.31:0/3991511702 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f99b80674d0 0x7f99b8069980 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f99c800ac00 tx=0x7f99c8006f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:03.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.313+0000 7f99d0ff9700 1 -- 172.21.15.31:0/3991511702 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f99cc053140 con 0x7f99d4159910 2023-10-06T02:30:03.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.533+0000 7f99d9d9d700 1 -- 172.21.15.31:0/3991511702 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f99d40c7580 con 0x7f99d4159910 2023-10-06T02:30:03.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.537+0000 7f99d0ff9700 1 -- 172.21.15.31:0/3991511702 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v23) v1 ==== 104+0+0 (secure 0 0 0) 0x7f99d40c7580 con 0x7f99d4159910 2023-10-06T02:30:03.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.539+0000 7f99d9d9d700 1 -- 172.21.15.31:0/3991511702 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f99b80674d0 msgr2=0x7f99b8069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:03.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.539+0000 7f99d9d9d700 1 --2- 172.21.15.31:0/3991511702 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f99b80674d0 0x7f99b8069980 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f99c800ac00 tx=0x7f99c8006f90 comp rx=0 tx=0).stop 2023-10-06T02:30:03.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.539+0000 7f99d9d9d700 1 -- 172.21.15.31:0/3991511702 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99d4159910 msgr2=0x7f99d40af590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:03.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.539+0000 7f99d9d9d700 1 --2- 172.21.15.31:0/3991511702 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99d4159910 0x7f99d40af590 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f99d4071320 tx=0x7f99cc017000 comp rx=0 tx=0).stop 2023-10-06T02:30:03.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.539+0000 7f99d9d9d700 1 -- 172.21.15.31:0/3991511702 shutdown_connections 2023-10-06T02:30:03.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.539+0000 7f99d9d9d700 1 --2- 172.21.15.31:0/3991511702 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99d4159910 0x7f99d40af590 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:03.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.539+0000 7f99d9d9d700 1 --2- 172.21.15.31:0/3991511702 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f99b80674d0 0x7f99b8069980 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:03.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.539+0000 7f99d9d9d700 1 --2- 172.21.15.31:0/3991511702 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f99d40c6be0 0x7f99d4069ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:03.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.539+0000 7f99d9d9d700 1 --2- 172.21.15.31:0/3991511702 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99d4159540 0x7f99d414ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:03.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.539+0000 7f99d9d9d700 1 -- 172.21.15.31:0/3991511702 >> 172.21.15.31:0/3991511702 conn(0x7f99d406f330 msgr2=0x7f99d415a780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:03.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.539+0000 7f99d9d9d700 1 -- 172.21.15.31:0/3991511702 shutdown_connections 2023-10-06T02:30:03.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.539+0000 7f99d9d9d700 1 -- 172.21.15.31:0/3991511702 wait complete. 2023-10-06T02:30:03.558 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2' 2023-10-06T02:30:03.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.744+0000 7fb43cb9d700 1 Processor -- start 2023-10-06T02:30:03.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.744+0000 7fb43cb9d700 1 -- start start 2023-10-06T02:30:03.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.746+0000 7fb43cb9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb43806f3a0 0x7fb4380cc840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:03.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.746+0000 7fb43cb9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb4380c6b20 0x7fb43806e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:03.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.746+0000 7fb43cb9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb43806a490 con 0x7fb4380c6f70 2023-10-06T02:30:03.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.746+0000 7fb43cb9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4380c7490 con 0x7fb43806f3a0 2023-10-06T02:30:03.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.746+0000 7fb43cb9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4380c7770 con 0x7fb4380c6b20 2023-10-06T02:30:03.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.746+0000 7fb4377fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb43806f3a0 0x7fb4380cc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:03.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.746+0000 7fb4377fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb43806f3a0 0x7fb4380cc840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50844/0 (socket says 172.21.15.31:50844) 2023-10-06T02:30:03.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.746+0000 7fb4377fe700 1 -- 172.21.15.31:0/2095197403 learned_addr learned my addr 172.21.15.31:0/2095197403 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:03.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.746+0000 7fb436ffd700 1 --2- 172.21.15.31:0/2095197403 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb4380c6b20 0x7fb43806e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:03.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.746+0000 7fb4377fe700 1 -- 172.21.15.31:0/2095197403 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb4380c6b20 msgr2=0x7fb43806e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:03.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.746+0000 7fb4377fe700 1 --2- 172.21.15.31:0/2095197403 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb4380c6b20 0x7fb43806e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:03.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.746+0000 7fb4377fe700 1 -- 172.21.15.31:0/2095197403 >> v1:172.21.15.31:6789/0 conn(0x7fb4380c6f70 legacy=0x7fb43814fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:03.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.746+0000 7fb4367fc700 1 -- 172.21.15.31:0/2095197403 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3727237587 0 0) 0x7fb43806a490 con 0x7fb4380c6f70 2023-10-06T02:30:03.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.746+0000 7fb4377fe700 1 -- 172.21.15.31:0/2095197403 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4380c7a50 con 0x7fb43806f3a0 2023-10-06T02:30:03.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.747+0000 7fb4377fe700 1 --2- 172.21.15.31:0/2095197403 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb43806f3a0 0x7fb4380cc840 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fb4280099b0 tx=0x7fb428009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59a9203576601288 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:03.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.747+0000 7fb4367fc700 1 -- 172.21.15.31:0/2095197403 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb42800b860 con 0x7fb43806f3a0 2023-10-06T02:30:03.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.747+0000 7fb4367fc700 1 -- 172.21.15.31:0/2095197403 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb42800b9c0 con 0x7fb43806f3a0 2023-10-06T02:30:03.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.747+0000 7fb4367fc700 1 -- 172.21.15.31:0/2095197403 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb42800bd60 con 0x7fb43806f3a0 2023-10-06T02:30:03.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.747+0000 7fb43cb9d700 1 -- 172.21.15.31:0/2095197403 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb43806f3a0 msgr2=0x7fb4380cc840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:03.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.747+0000 7fb43cb9d700 1 --2- 172.21.15.31:0/2095197403 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb43806f3a0 0x7fb4380cc840 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fb4280099b0 tx=0x7fb428009250 comp rx=0 tx=0).stop 2023-10-06T02:30:03.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.748+0000 7fb43cb9d700 1 -- 172.21.15.31:0/2095197403 shutdown_connections 2023-10-06T02:30:03.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.748+0000 7fb43cb9d700 1 --2- 172.21.15.31:0/2095197403 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb43806f3a0 0x7fb4380cc840 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:03.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.748+0000 7fb43cb9d700 1 --2- 172.21.15.31:0/2095197403 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb4380c6b20 0x7fb43806e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:03.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.748+0000 7fb43cb9d700 1 -- 172.21.15.31:0/2095197403 >> 172.21.15.31:0/2095197403 conn(0x7fb43806d170 msgr2=0x7fb4380cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:03.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.748+0000 7fb43cb9d700 1 -- 172.21.15.31:0/2095197403 shutdown_connections 2023-10-06T02:30:03.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.748+0000 7fb43cb9d700 1 -- 172.21.15.31:0/2095197403 wait complete. 2023-10-06T02:30:03.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.749+0000 7fb43cb9d700 1 Processor -- start 2023-10-06T02:30:03.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.749+0000 7fb43cb9d700 1 -- start start 2023-10-06T02:30:03.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.750+0000 7fb43cb9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb43806f3a0 0x7fb4380c1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:03.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.750+0000 7fb43cb9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4380c6b20 0x7fb4381586c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:03.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.750+0000 7fb43cb9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb4380c6f70 0x7fb438149900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:03.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.750+0000 7fb43cb9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4380b1370 con 0x7fb4380c6b20 2023-10-06T02:30:03.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.750+0000 7fb43cb9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4380b07c0 con 0x7fb43806f3a0 2023-10-06T02:30:03.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.750+0000 7fb43cb9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb438158eb0 con 0x7fb4380c6f70 2023-10-06T02:30:03.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.750+0000 7fb436ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4380c6b20 0x7fb4381586c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:03.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.750+0000 7fb436ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4380c6b20 0x7fb4381586c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50856/0 (socket says 172.21.15.31:50856) 2023-10-06T02:30:03.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.750+0000 7fb436ffd700 1 -- 172.21.15.31:0/477557299 learned_addr learned my addr 172.21.15.31:0/477557299 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:03.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.751+0000 7fb4377fe700 1 --2- 172.21.15.31:0/477557299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb43806f3a0 0x7fb4380c1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:03.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.751+0000 7fb437fff700 1 --2- 172.21.15.31:0/477557299 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb4380c6f70 0x7fb438149900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:03.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.751+0000 7fb436ffd700 1 -- 172.21.15.31:0/477557299 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb4380c6f70 msgr2=0x7fb438149900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:03.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.751+0000 7fb436ffd700 1 --2- 172.21.15.31:0/477557299 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb4380c6f70 0x7fb438149900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:03.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.751+0000 7fb436ffd700 1 -- 172.21.15.31:0/477557299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb43806f3a0 msgr2=0x7fb4380c1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:03.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.751+0000 7fb436ffd700 1 --2- 172.21.15.31:0/477557299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb43806f3a0 0x7fb4380c1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:03.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.751+0000 7fb436ffd700 1 -- 172.21.15.31:0/477557299 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4280094f0 con 0x7fb4380c6b20 2023-10-06T02:30:03.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.751+0000 7fb4377fe700 1 --2- 172.21.15.31:0/477557299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb43806f3a0 0x7fb4380c1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:30:03.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.751+0000 7fb436ffd700 1 --2- 172.21.15.31:0/477557299 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4380c6b20 0x7fb4381586c0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fb42c00d810 tx=0x7fb42c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:03.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.751+0000 7fb41f7fe700 1 -- 172.21.15.31:0/477557299 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb42c0125f0 con 0x7fb4380c6b20 2023-10-06T02:30:03.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.751+0000 7fb41f7fe700 1 -- 172.21.15.31:0/477557299 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb42c012750 con 0x7fb4380c6b20 2023-10-06T02:30:03.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.751+0000 7fb41f7fe700 1 -- 172.21.15.31:0/477557299 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb42c012b10 con 0x7fb4380c6b20 2023-10-06T02:30:03.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.751+0000 7fb43cb9d700 1 -- 172.21.15.31:0/477557299 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb438158c00 con 0x7fb4380c6b20 2023-10-06T02:30:03.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.752+0000 7fb43cb9d700 1 -- 172.21.15.31:0/477557299 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4380b0cb0 con 0x7fb4380c6b20 2023-10-06T02:30:03.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.753+0000 7fb43cb9d700 1 -- 172.21.15.31:0/477557299 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4380b14b0 con 0x7fb4380c6b20 2023-10-06T02:30:03.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.754+0000 7fb41f7fe700 1 -- 172.21.15.31:0/477557299 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7fb42c01e460 con 0x7fb4380c6b20 2023-10-06T02:30:03.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.755+0000 7fb41f7fe700 1 --2- 172.21.15.31:0/477557299 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb4240675a0 0x7fb424069a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:03.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.755+0000 7fb41f7fe700 1 -- 172.21.15.31:0/477557299 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5320+0+0 (secure 0 0 0) 0x7fb42c0853a0 con 0x7fb4380c6b20 2023-10-06T02:30:03.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.755+0000 7fb4377fe700 1 --2- 172.21.15.31:0/477557299 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb4240675a0 0x7fb424069a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:03.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.756+0000 7fb4377fe700 1 --2- 172.21.15.31:0/477557299 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb4240675a0 0x7fb424069a50 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb4280099b0 tx=0x7fb428017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:03.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:03.759+0000 7fb41f7fe700 1 -- 172.21.15.31:0/477557299 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb42c051fa0 con 0x7fb4380c6b20 2023-10-06T02:30:04.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:04.042+0000 7fb43cb9d700 1 -- 172.21.15.31:0/477557299 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2"]} v 0) v1 -- 0x7fb43804c3c0 con 0x7fb4380c6b20 2023-10-06T02:30:04.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:04.083+0000 7fb41f7fe700 1 -- 172.21.15.31:0/477557299 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2"]}]=0 v388) v1 ==== 148+0+0 (secure 0 0 0) 0x7fb42c083e60 con 0x7fb4380c6b20 2023-10-06T02:30:04.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:04.085+0000 7fb43cb9d700 1 -- 172.21.15.31:0/477557299 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb4240675a0 msgr2=0x7fb424069a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:04.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:04.085+0000 7fb43cb9d700 1 --2- 172.21.15.31:0/477557299 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb4240675a0 0x7fb424069a50 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb4280099b0 tx=0x7fb428017000 comp rx=0 tx=0).stop 2023-10-06T02:30:04.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:04.085+0000 7fb43cb9d700 1 -- 172.21.15.31:0/477557299 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4380c6b20 msgr2=0x7fb4381586c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:04.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:04.085+0000 7fb43cb9d700 1 --2- 172.21.15.31:0/477557299 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4380c6b20 0x7fb4381586c0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fb42c00d810 tx=0x7fb42c008040 comp rx=0 tx=0).stop 2023-10-06T02:30:04.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:04.086+0000 7fb43cb9d700 1 -- 172.21.15.31:0/477557299 shutdown_connections 2023-10-06T02:30:04.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:04.086+0000 7fb43cb9d700 1 --2- 172.21.15.31:0/477557299 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb4240675a0 0x7fb424069a50 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:04.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:04.086+0000 7fb43cb9d700 1 --2- 172.21.15.31:0/477557299 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb4380c6f70 0x7fb438149900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:04.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:04.086+0000 7fb43cb9d700 1 --2- 172.21.15.31:0/477557299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb43806f3a0 0x7fb4380c1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:04.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:04.086+0000 7fb43cb9d700 1 --2- 172.21.15.31:0/477557299 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4380c6b20 0x7fb4381586c0 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:04.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:04.086+0000 7fb43cb9d700 1 -- 172.21.15.31:0/477557299 >> 172.21.15.31:0/477557299 conn(0x7fb43806d170 msgr2=0x7fb43814eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:04.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:04.086+0000 7fb43cb9d700 1 -- 172.21.15.31:0/477557299 shutdown_connections 2023-10-06T02:30:04.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:04.086+0000 7fb43cb9d700 1 -- 172.21.15.31:0/477557299 wait complete. 2023-10-06T02:30:04.104 INFO:tasks.cephfs_test_runner:poor man's upgrade test -- theme continues... ... ok 2023-10-06T02:30:04.104 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_ops_on_nonexistent_vol (tasks.cephfs.test_volumes.TestMisc) 2023-10-06T02:30:04.104 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol' 2023-10-06T02:30:04.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:04.290+0000 7f3bea39d700 1 Processor -- start 2023-10-06T02:30:04.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:04.291+0000 7f3bea39d700 1 -- start start 2023-10-06T02:30:04.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:04.292+0000 7f3bea39d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3be413fb20 0x7f3be415a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:04.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:04.292+0000 7f3bea39d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3be413fef0 0x7f3be40c7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:04.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:04.292+0000 7f3bea39d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3be415ae70 con 0x7f3be406aaa0 2023-10-06T02:30:04.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:04.292+0000 7f3bea39d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3be406af60 con 0x7f3be413fb20 2023-10-06T02:30:04.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:04.292+0000 7f3bea39d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3be406b240 con 0x7f3be413fef0 2023-10-06T02:30:04.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:04.292+0000 7f3be939b700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3be413fb20 0x7f3be415a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:04.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:04.292+0000 7f3be8b9a700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3be413fef0 0x7f3be40c7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:04.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:04.293+0000 7f3be8b9a700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3be413fef0 0x7f3be40c7990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:52170/0 (socket says 172.21.15.31:52170) 2023-10-06T02:30:04.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:04.293+0000 7f3be939b700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3be413fb20 0x7f3be415a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:39666/0 (socket says 172.21.15.31:39666) 2023-10-06T02:30:04.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:04.293+0000 7f3be8b9a700 1 -- 172.21.15.31:0/1653913791 learned_addr learned my addr 172.21.15.31:0/1653913791 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:04.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:04.293+0000 7f3be8b9a700 1 -- 172.21.15.31:0/1653913791 >> v2:172.21.15.62:3300/0 conn(0x7f3be413fb20 msgr2=0x7f3be415a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:04.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:04.293+0000 7f3be8b9a700 1 --2- 172.21.15.31:0/1653913791 >> v2:172.21.15.62:3300/0 conn(0x7f3be413fb20 0x7f3be415a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:04.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:04.293+0000 7f3be8b9a700 1 -- 172.21.15.31:0/1653913791 >> v1:172.21.15.62:6789/0 conn(0x7f3be406aaa0 legacy=0x7f3be40c6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:04.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:04.293+0000 7f3be8b9a700 1 -- 172.21.15.31:0/1653913791 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3be406b520 con 0x7f3be413fef0 2023-10-06T02:30:04.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:04.293+0000 7f3be939b700 1 --2- 172.21.15.31:0/1653913791 >> v2:172.21.15.62:3300/0 conn(0x7f3be413fb20 0x7f3be415a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:30:04.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:04.293+0000 7f3be8b9a700 1 --2- 172.21.15.31:0/1653913791 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3be413fef0 0x7f3be40c7990 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3bd4009990 tx=0x7f3bd4009220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cabaacde20891ac server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:04.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:04.294+0000 7f3bdbfff700 1 -- 172.21.15.31:0/1653913791 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd4009e80 con 0x7f3be413fef0 2023-10-06T02:30:04.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:04.294+0000 7f3bdbfff700 1 -- 172.21.15.31:0/1653913791 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bd4005c10 con 0x7f3be413fef0 2023-10-06T02:30:04.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:04.294+0000 7f3bdbfff700 1 -- 172.21.15.31:0/1653913791 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd400b650 con 0x7f3be413fef0 2023-10-06T02:30:04.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:04.294+0000 7f3bea39d700 1 -- 172.21.15.31:0/1653913791 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3be413fef0 msgr2=0x7f3be40c7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:04.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:04.294+0000 7f3bea39d700 1 --2- 172.21.15.31:0/1653913791 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3be413fef0 0x7f3be40c7990 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3bd4009990 tx=0x7f3bd4009220 comp rx=0 tx=0).stop 2023-10-06T02:30:04.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:04.294+0000 7f3bea39d700 1 -- 172.21.15.31:0/1653913791 shutdown_connections 2023-10-06T02:30:04.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:04.294+0000 7f3bea39d700 1 --2- 172.21.15.31:0/1653913791 >> v2:172.21.15.62:3300/0 conn(0x7f3be413fb20 0x7f3be415a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:04.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:04.294+0000 7f3bea39d700 1 --2- 172.21.15.31:0/1653913791 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3be413fef0 0x7f3be40c7990 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:04.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:04.294+0000 7f3bea39d700 1 -- 172.21.15.31:0/1653913791 >> 172.21.15.31:0/1653913791 conn(0x7f3be406d170 msgr2=0x7f3be40cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:04.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:04.294+0000 7f3bea39d700 1 -- 172.21.15.31:0/1653913791 shutdown_connections 2023-10-06T02:30:04.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:04.294+0000 7f3bea39d700 1 -- 172.21.15.31:0/1653913791 wait complete. 2023-10-06T02:30:04.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:04.296+0000 7f3bea39d700 1 Processor -- start 2023-10-06T02:30:04.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:04.296+0000 7f3bea39d700 1 -- start start 2023-10-06T02:30:04.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:04.297+0000 7f3bea39d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3be406aaa0 0x7f3be41583c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:04.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:04.297+0000 7f3bea39d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3be413fb20 0x7f3be4169740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:04.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:04.297+0000 7f3bea39d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3be413fef0 0x7f3be40b02c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:04.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:04.297+0000 7f3bea39d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3be40c1e70 con 0x7f3be406aaa0 2023-10-06T02:30:04.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:04.297+0000 7f3bea39d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3be4158d30 con 0x7f3be413fef0 2023-10-06T02:30:04.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:04.297+0000 7f3bea39d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3be40c1820 con 0x7f3be413fb20 2023-10-06T02:30:04.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:04.297+0000 7f3be939b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3be406aaa0 0x7f3be41583c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:04.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:04.297+0000 7f3be939b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3be406aaa0 0x7f3be41583c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50872/0 (socket says 172.21.15.31:50872) 2023-10-06T02:30:04.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:04.297+0000 7f3be939b700 1 -- 172.21.15.31:0/805973441 learned_addr learned my addr 172.21.15.31:0/805973441 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:04.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:04.297+0000 7f3be8b9a700 1 --2- 172.21.15.31:0/805973441 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3be413fb20 0x7f3be4169740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:04.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:04.297+0000 7f3be9b9c700 1 --2- 172.21.15.31:0/805973441 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3be413fef0 0x7f3be40b02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:04.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:04.298+0000 7f3be939b700 1 -- 172.21.15.31:0/805973441 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3be413fb20 msgr2=0x7f3be4169740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:04.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:04.298+0000 7f3be939b700 1 --2- 172.21.15.31:0/805973441 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3be413fb20 0x7f3be4169740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:04.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:04.298+0000 7f3be939b700 1 -- 172.21.15.31:0/805973441 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3be413fef0 msgr2=0x7f3be40b02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:04.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:04.298+0000 7f3be939b700 1 --2- 172.21.15.31:0/805973441 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3be413fef0 0x7f3be40b02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:04.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:04.298+0000 7f3be939b700 1 -- 172.21.15.31:0/805973441 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3be0009420 con 0x7f3be406aaa0 2023-10-06T02:30:04.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:04.298+0000 7f3be939b700 1 --2- 172.21.15.31:0/805973441 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3be406aaa0 0x7f3be41583c0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f3be000ec80 tx=0x7f3be00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:04.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:04.298+0000 7f3bd97fa700 1 -- 172.21.15.31:0/805973441 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be000c850 con 0x7f3be406aaa0 2023-10-06T02:30:04.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:04.298+0000 7f3bd97fa700 1 -- 172.21.15.31:0/805973441 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3be000c9b0 con 0x7f3be406aaa0 2023-10-06T02:30:04.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:04.298+0000 7f3bd97fa700 1 -- 172.21.15.31:0/805973441 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be000cd30 con 0x7f3be406aaa0 2023-10-06T02:30:04.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:04.298+0000 7f3bea39d700 1 -- 172.21.15.31:0/805973441 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bd4009480 con 0x7f3be406aaa0 2023-10-06T02:30:04.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:04.298+0000 7f3bea39d700 1 -- 172.21.15.31:0/805973441 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3be416d990 con 0x7f3be406aaa0 2023-10-06T02:30:04.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:04.299+0000 7f3bea39d700 1 -- 172.21.15.31:0/805973441 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3be40c1680 con 0x7f3be406aaa0 2023-10-06T02:30:04.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:04.300+0000 7f3bd97fa700 1 -- 172.21.15.31:0/805973441 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f3be0013680 con 0x7f3be406aaa0 2023-10-06T02:30:04.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:04.301+0000 7f3bd97fa700 1 --2- 172.21.15.31:0/805973441 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f3bc80674d0 0x7f3bc8069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:04.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:04.301+0000 7f3bd97fa700 1 -- 172.21.15.31:0/805973441 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5320+0+0 (secure 0 0 0) 0x7f3be0085170 con 0x7f3be406aaa0 2023-10-06T02:30:04.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:04.302+0000 7f3be8b9a700 1 --2- 172.21.15.31:0/805973441 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f3bc80674d0 0x7f3bc8069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:04.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:04.302+0000 7f3be8b9a700 1 --2- 172.21.15.31:0/805973441 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f3bc80674d0 0x7f3bc8069980 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3be4069580 tx=0x7f3bd4016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:04.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:04.305+0000 7f3bd97fa700 1 -- 172.21.15.31:0/805973441 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3be40c1680 con 0x7f3be406aaa0 2023-10-06T02:30:04.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:04.587+0000 7f3bea39d700 1 -- 172.21.15.31:0/805973441 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol"]} v 0) v1 -- 0x7f3be40c1680 con 0x7f3be406aaa0 2023-10-06T02:30:05.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.087+0000 7f3bd97fa700 1 -- 172.21.15.31:0/805973441 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol"]}]=0 v389) v1 ==== 154+0+0 (secure 0 0 0) 0x7f3be40c1680 con 0x7f3be406aaa0 2023-10-06T02:30:05.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.090+0000 7f3bea39d700 1 -- 172.21.15.31:0/805973441 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f3bc80674d0 msgr2=0x7f3bc8069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:05.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.090+0000 7f3bea39d700 1 --2- 172.21.15.31:0/805973441 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f3bc80674d0 0x7f3bc8069980 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3be4069580 tx=0x7f3bd4016000 comp rx=0 tx=0).stop 2023-10-06T02:30:05.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.090+0000 7f3bea39d700 1 -- 172.21.15.31:0/805973441 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3be406aaa0 msgr2=0x7f3be41583c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:05.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.090+0000 7f3bea39d700 1 --2- 172.21.15.31:0/805973441 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3be406aaa0 0x7f3be41583c0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f3be000ec80 tx=0x7f3be00091a0 comp rx=0 tx=0).stop 2023-10-06T02:30:05.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.090+0000 7f3bea39d700 1 -- 172.21.15.31:0/805973441 shutdown_connections 2023-10-06T02:30:05.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.090+0000 7f3bea39d700 1 --2- 172.21.15.31:0/805973441 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3be413fef0 0x7f3be40b02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:05.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.090+0000 7f3bea39d700 1 --2- 172.21.15.31:0/805973441 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3be406aaa0 0x7f3be41583c0 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:05.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.090+0000 7f3bea39d700 1 --2- 172.21.15.31:0/805973441 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f3bc80674d0 0x7f3bc8069980 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:05.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.090+0000 7f3bea39d700 1 --2- 172.21.15.31:0/805973441 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3be413fb20 0x7f3be4169740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:05.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.090+0000 7f3bea39d700 1 -- 172.21.15.31:0/805973441 >> 172.21.15.31:0/805973441 conn(0x7f3be406d170 msgr2=0x7f3be40af800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:05.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.090+0000 7f3bea39d700 1 -- 172.21.15.31:0/805973441 shutdown_connections 2023-10-06T02:30:05.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.090+0000 7f3bea39d700 1 -- 172.21.15.31:0/805973441 wait complete. 2023-10-06T02:30:05.109 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T02:30:05.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.300+0000 7fde0ed9d700 1 Processor -- start 2023-10-06T02:30:05.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.301+0000 7fde0ed9d700 1 -- start start 2023-10-06T02:30:05.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.302+0000 7fde0ed9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fde08140a70 0x7fde08069b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:05.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.302+0000 7fde0ed9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fde0806de50 0x7fde0806a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:05.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.302+0000 7fde0ed9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde0815b640 con 0x7fde0815b0a0 2023-10-06T02:30:05.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.302+0000 7fde0ed9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde080c6800 con 0x7fde08140a70 2023-10-06T02:30:05.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.302+0000 7fde0ed9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde0815b920 con 0x7fde0806de50 2023-10-06T02:30:05.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.302+0000 7fde0dd9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fde08140a70 0x7fde08069b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:05.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.302+0000 7fde0dd9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fde08140a70 0x7fde08069b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50874/0 (socket says 172.21.15.31:50874) 2023-10-06T02:30:05.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.302+0000 7fde0dd9b700 1 -- 172.21.15.31:0/1866819204 learned_addr learned my addr 172.21.15.31:0/1866819204 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:05.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.302+0000 7fde0d59a700 1 --2- 172.21.15.31:0/1866819204 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fde0806de50 0x7fde0806a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:05.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.303+0000 7fde0dd9b700 1 -- 172.21.15.31:0/1866819204 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fde0806de50 msgr2=0x7fde0806a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:05.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.303+0000 7fde0dd9b700 1 --2- 172.21.15.31:0/1866819204 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fde0806de50 0x7fde0806a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:05.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.303+0000 7fde0dd9b700 1 -- 172.21.15.31:0/1866819204 >> v1:172.21.15.62:6789/0 conn(0x7fde0815b0a0 legacy=0x7fde08157a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:05.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.303+0000 7fde0dd9b700 1 -- 172.21.15.31:0/1866819204 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde0815bc00 con 0x7fde08140a70 2023-10-06T02:30:05.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.303+0000 7fde0dd9b700 1 --2- 172.21.15.31:0/1866819204 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde08140a70 0x7fde08069b00 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fde040099b0 tx=0x7fde04009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=862d32de6a1ae4fe server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:05.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.303+0000 7fde0cd99700 1 -- 172.21.15.31:0/1866819204 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde04009ea0 con 0x7fde08140a70 2023-10-06T02:30:05.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.303+0000 7fde0cd99700 1 -- 172.21.15.31:0/1866819204 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde0400b860 con 0x7fde08140a70 2023-10-06T02:30:05.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.303+0000 7fde0cd99700 1 -- 172.21.15.31:0/1866819204 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde0400bc00 con 0x7fde08140a70 2023-10-06T02:30:05.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.304+0000 7fde0ed9d700 1 -- 172.21.15.31:0/1866819204 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde08140a70 msgr2=0x7fde08069b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:05.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.304+0000 7fde0ed9d700 1 --2- 172.21.15.31:0/1866819204 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde08140a70 0x7fde08069b00 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fde040099b0 tx=0x7fde04009250 comp rx=0 tx=0).stop 2023-10-06T02:30:05.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.304+0000 7fde0ed9d700 1 -- 172.21.15.31:0/1866819204 shutdown_connections 2023-10-06T02:30:05.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.304+0000 7fde0ed9d700 1 --2- 172.21.15.31:0/1866819204 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde08140a70 0x7fde08069b00 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:05.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.304+0000 7fde0ed9d700 1 --2- 172.21.15.31:0/1866819204 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fde0806de50 0x7fde0806a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:05.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.304+0000 7fde0ed9d700 1 -- 172.21.15.31:0/1866819204 >> 172.21.15.31:0/1866819204 conn(0x7fde0806d110 msgr2=0x7fde080cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:05.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.304+0000 7fde0ed9d700 1 -- 172.21.15.31:0/1866819204 shutdown_connections 2023-10-06T02:30:05.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.304+0000 7fde0ed9d700 1 -- 172.21.15.31:0/1866819204 wait complete. 2023-10-06T02:30:05.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.305+0000 7fde0ed9d700 1 Processor -- start 2023-10-06T02:30:05.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.305+0000 7fde0ed9d700 1 -- start start 2023-10-06T02:30:05.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.306+0000 7fde0ed9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde0806de50 0x7fde080aff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:05.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.306+0000 7fde0ed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde08140a70 0x7fde0815c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:05.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.306+0000 7fde0ed9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fde0815b0a0 0x7fde0815c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:05.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.306+0000 7fde0ed9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde080b04d0 con 0x7fde08140a70 2023-10-06T02:30:05.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.306+0000 7fde0ed9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde080cb9a0 con 0x7fde0806de50 2023-10-06T02:30:05.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.306+0000 7fde0ed9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde080cad90 con 0x7fde0815b0a0 2023-10-06T02:30:05.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.307+0000 7fde0d59a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde08140a70 0x7fde0815c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:05.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.307+0000 7fde0d59a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde08140a70 0x7fde0815c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50882/0 (socket says 172.21.15.31:50882) 2023-10-06T02:30:05.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.307+0000 7fde0d59a700 1 -- 172.21.15.31:0/3354889679 learned_addr learned my addr 172.21.15.31:0/3354889679 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:05.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.307+0000 7fde0dd9b700 1 --2- 172.21.15.31:0/3354889679 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde0806de50 0x7fde080aff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:05.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.307+0000 7fde0e59c700 1 --2- 172.21.15.31:0/3354889679 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fde0815b0a0 0x7fde0815c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:05.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.307+0000 7fde0d59a700 1 -- 172.21.15.31:0/3354889679 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fde0815b0a0 msgr2=0x7fde0815c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:05.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.307+0000 7fde0d59a700 1 --2- 172.21.15.31:0/3354889679 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fde0815b0a0 0x7fde0815c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:05.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.307+0000 7fde0d59a700 1 -- 172.21.15.31:0/3354889679 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde0806de50 msgr2=0x7fde080aff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:05.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.307+0000 7fde0d59a700 1 --2- 172.21.15.31:0/3354889679 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde0806de50 0x7fde080aff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:05.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.307+0000 7fde0d59a700 1 -- 172.21.15.31:0/3354889679 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde040094f0 con 0x7fde08140a70 2023-10-06T02:30:05.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.307+0000 7fde0e59c700 1 --2- 172.21.15.31:0/3354889679 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fde0815b0a0 0x7fde0815c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:30:05.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.307+0000 7fde0d59a700 1 --2- 172.21.15.31:0/3354889679 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde08140a70 0x7fde0815c120 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fddf800ec80 tx=0x7fddf80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:05.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.307+0000 7fddfdffb700 1 -- 172.21.15.31:0/3354889679 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf800c850 con 0x7fde08140a70 2023-10-06T02:30:05.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.307+0000 7fddfdffb700 1 -- 172.21.15.31:0/3354889679 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddf800c9b0 con 0x7fde08140a70 2023-10-06T02:30:05.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.307+0000 7fde0ed9d700 1 -- 172.21.15.31:0/3354889679 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde080cb070 con 0x7fde08140a70 2023-10-06T02:30:05.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.308+0000 7fddfdffb700 1 -- 172.21.15.31:0/3354889679 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf800cd50 con 0x7fde08140a70 2023-10-06T02:30:05.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.308+0000 7fde0ed9d700 1 -- 172.21.15.31:0/3354889679 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde080cb630 con 0x7fde08140a70 2023-10-06T02:30:05.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.309+0000 7fde0ed9d700 1 -- 172.21.15.31:0/3354889679 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde0804c3c0 con 0x7fde08140a70 2023-10-06T02:30:05.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.310+0000 7fddfdffb700 1 -- 172.21.15.31:0/3354889679 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7fddf8013680 con 0x7fde08140a70 2023-10-06T02:30:05.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.310+0000 7fddfdffb700 1 --2- 172.21.15.31:0/3354889679 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fddec0674d0 0x7fddec069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:05.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.311+0000 7fddfdffb700 1 -- 172.21.15.31:0/3354889679 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5320+0+0 (secure 0 0 0) 0x7fddf8085100 con 0x7fde08140a70 2023-10-06T02:30:05.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.311+0000 7fde0dd9b700 1 --2- 172.21.15.31:0/3354889679 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fddec0674d0 0x7fddec069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:05.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.312+0000 7fde0dd9b700 1 --2- 172.21.15.31:0/3354889679 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fddec0674d0 0x7fddec069980 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fde040099b0 tx=0x7fde04017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:05.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.315+0000 7fddfdffb700 1 -- 172.21.15.31:0/3354889679 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fddf8051c40 con 0x7fde08140a70 2023-10-06T02:30:05.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.516+0000 7fde0ed9d700 1 -- 172.21.15.31:0/3354889679 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fde0815bee0 con 0x7fde08140a70 2023-10-06T02:30:05.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.521+0000 7fddfdffb700 1 -- 172.21.15.31:0/3354889679 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v24)=0 v24) v1 ==== 122+0+0 (secure 0 0 0) 0x7fddf8055260 con 0x7fde08140a70 2023-10-06T02:30:05.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.523+0000 7fde0ed9d700 1 -- 172.21.15.31:0/3354889679 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fddec0674d0 msgr2=0x7fddec069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:05.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.523+0000 7fde0ed9d700 1 --2- 172.21.15.31:0/3354889679 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fddec0674d0 0x7fddec069980 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fde040099b0 tx=0x7fde04017000 comp rx=0 tx=0).stop 2023-10-06T02:30:05.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.523+0000 7fde0ed9d700 1 -- 172.21.15.31:0/3354889679 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde08140a70 msgr2=0x7fde0815c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:05.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.523+0000 7fde0ed9d700 1 --2- 172.21.15.31:0/3354889679 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde08140a70 0x7fde0815c120 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fddf800ec80 tx=0x7fddf80091a0 comp rx=0 tx=0).stop 2023-10-06T02:30:05.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.523+0000 7fde0ed9d700 1 -- 172.21.15.31:0/3354889679 shutdown_connections 2023-10-06T02:30:05.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.523+0000 7fde0ed9d700 1 --2- 172.21.15.31:0/3354889679 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fddec0674d0 0x7fddec069980 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:05.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.523+0000 7fde0ed9d700 1 --2- 172.21.15.31:0/3354889679 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fde0815b0a0 0x7fde0815c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:05.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.523+0000 7fde0ed9d700 1 --2- 172.21.15.31:0/3354889679 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde0806de50 0x7fde080aff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:05.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.523+0000 7fde0ed9d700 1 --2- 172.21.15.31:0/3354889679 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde08140a70 0x7fde0815c120 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:05.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.523+0000 7fde0ed9d700 1 -- 172.21.15.31:0/3354889679 >> 172.21.15.31:0/3354889679 conn(0x7fde0806d110 msgr2=0x7fde0806ee40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:05.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.523+0000 7fde0ed9d700 1 -- 172.21.15.31:0/3354889679 shutdown_connections 2023-10-06T02:30:05.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.523+0000 7fde0ed9d700 1 -- 172.21.15.31:0/3354889679 wait complete. 2023-10-06T02:30:05.541 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T02:30:05.541 DEBUG:teuthology.orchestra.run.smithi031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T02:30:05.621 DEBUG:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T02:30:05.708 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T02:30:05.709 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T02:30:05.709 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:30:05.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.896+0000 7f95ce79d700 1 Processor -- start 2023-10-06T02:30:05.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.896+0000 7f95ce79d700 1 -- start start 2023-10-06T02:30:05.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.898+0000 7f95ce79d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f95c8142fd0 0x7f95c80c7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:05.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.898+0000 7f95ce79d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95c80ca480 con 0x7f95c80c7dc0 2023-10-06T02:30:05.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.898+0000 7f95ce79d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95c815e240 con 0x7f95c8142c00 2023-10-06T02:30:05.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.898+0000 7f95ce79d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95c806b350 con 0x7f95c8142fd0 2023-10-06T02:30:05.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.898+0000 7f95cdf9c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f95c80c7dc0 0x7f95c806c6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56132/0 (socket says 172.21.15.31:56132) 2023-10-06T02:30:05.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.898+0000 7f95cdf9c700 1 -- 172.21.15.31:0/4100750117 learned_addr learned my addr 172.21.15.31:0/4100750117 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:30:05.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.898+0000 7f95ccf9a700 1 --2- 172.21.15.31:0/4100750117 >> v2:172.21.15.62:3300/0 conn(0x7f95c8142fd0 0x7f95c80c7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:05.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.898+0000 7f95bffff700 1 -- 172.21.15.31:0/4100750117 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3535520534 0 0) 0x7f95c80ca480 con 0x7f95c80c7dc0 2023-10-06T02:30:05.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.898+0000 7f95bffff700 1 -- 172.21.15.31:0/4100750117 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95b4003570 con 0x7f95c80c7dc0 2023-10-06T02:30:05.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.899+0000 7f95ccf9a700 1 -- 172.21.15.31:0/4100750117 >> v1:172.21.15.62:6789/0 conn(0x7f95c8142c00 legacy=0x7f95c806b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:05.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.899+0000 7f95ccf9a700 1 -- 172.21.15.31:0/4100750117 >> v1:172.21.15.31:6789/0 conn(0x7f95c80c7dc0 legacy=0x7f95c806c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:05.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.899+0000 7f95ccf9a700 1 -- 172.21.15.31:0/4100750117 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95c80c81d0 con 0x7f95c8142fd0 2023-10-06T02:30:05.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.899+0000 7f95ccf9a700 1 --2- 172.21.15.31:0/4100750117 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95c8142fd0 0x7f95c80c7880 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f95b80098e0 tx=0x7f95b80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7fec01c83f23e8eb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:05.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.899+0000 7f95bffff700 1 -- 172.21.15.31:0/4100750117 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b800b640 con 0x7f95c8142fd0 2023-10-06T02:30:05.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.899+0000 7f95bffff700 1 -- 172.21.15.31:0/4100750117 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95b800b7a0 con 0x7f95c8142fd0 2023-10-06T02:30:05.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.899+0000 7f95bffff700 1 -- 172.21.15.31:0/4100750117 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b800bb40 con 0x7f95c8142fd0 2023-10-06T02:30:05.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.900+0000 7f95ce79d700 1 -- 172.21.15.31:0/4100750117 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95c8142fd0 msgr2=0x7f95c80c7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:05.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.900+0000 7f95ce79d700 1 --2- 172.21.15.31:0/4100750117 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95c8142fd0 0x7f95c80c7880 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f95b80098e0 tx=0x7f95b80091a0 comp rx=0 tx=0).stop 2023-10-06T02:30:05.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.900+0000 7f95ce79d700 1 -- 172.21.15.31:0/4100750117 shutdown_connections 2023-10-06T02:30:05.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.900+0000 7f95ce79d700 1 --2- 172.21.15.31:0/4100750117 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95c8142fd0 0x7f95c80c7880 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:05.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.900+0000 7f95ce79d700 1 -- 172.21.15.31:0/4100750117 >> 172.21.15.31:0/4100750117 conn(0x7f95c80675a0 msgr2=0x7f95c80679a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:05.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.900+0000 7f95ce79d700 1 -- 172.21.15.31:0/4100750117 shutdown_connections 2023-10-06T02:30:05.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.900+0000 7f95ce79d700 1 -- 172.21.15.31:0/4100750117 wait complete. 2023-10-06T02:30:05.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.901+0000 7f95ce79d700 1 Processor -- start 2023-10-06T02:30:05.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.902+0000 7f95ce79d700 1 -- start start 2023-10-06T02:30:05.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.902+0000 7f95ce79d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95c80c7dc0 0x7f95c8140620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:05.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.902+0000 7f95ce79d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95c8142c00 0x7f95c80b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:05.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.902+0000 7f95ce79d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f95c8142fd0 0x7f95c80c61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:05.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.902+0000 7f95ce79d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c80affe0 con 0x7f95c8142c00 2023-10-06T02:30:05.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.902+0000 7f95ce79d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c80b0e40 con 0x7f95c80c7dc0 2023-10-06T02:30:05.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.902+0000 7f95ce79d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95c80b0230 con 0x7f95c8142fd0 2023-10-06T02:30:05.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.903+0000 7f95ccf9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95c8142c00 0x7f95c80b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:05.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.903+0000 7f95ccf9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95c8142c00 0x7f95c80b12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50896/0 (socket says 172.21.15.31:50896) 2023-10-06T02:30:05.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.903+0000 7f95ccf9a700 1 -- 172.21.15.31:0/2282829601 learned_addr learned my addr 172.21.15.31:0/2282829601 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:05.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.903+0000 7f95cd79b700 1 --2- 172.21.15.31:0/2282829601 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95c80c7dc0 0x7f95c8140620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:05.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.903+0000 7f95cdf9c700 1 --2- 172.21.15.31:0/2282829601 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f95c8142fd0 0x7f95c80c61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:05.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.903+0000 7f95ccf9a700 1 -- 172.21.15.31:0/2282829601 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f95c8142fd0 msgr2=0x7f95c80c61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:05.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.903+0000 7f95ccf9a700 1 --2- 172.21.15.31:0/2282829601 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f95c8142fd0 0x7f95c80c61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:05.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.903+0000 7f95ccf9a700 1 -- 172.21.15.31:0/2282829601 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95c80c7dc0 msgr2=0x7f95c8140620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:05.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.903+0000 7f95ccf9a700 1 --2- 172.21.15.31:0/2282829601 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95c80c7dc0 0x7f95c8140620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:05.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.903+0000 7f95ccf9a700 1 -- 172.21.15.31:0/2282829601 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95b8009420 con 0x7f95c8142c00 2023-10-06T02:30:05.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.903+0000 7f95ccf9a700 1 --2- 172.21.15.31:0/2282829601 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95c8142c00 0x7f95c80b12d0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f95c806f290 tx=0x7f95b8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:05.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.904+0000 7f95bd7fa700 1 -- 172.21.15.31:0/2282829601 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b8000d10 con 0x7f95c8142c00 2023-10-06T02:30:05.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.904+0000 7f95bd7fa700 1 -- 172.21.15.31:0/2282829601 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95b800bb20 con 0x7f95c8142c00 2023-10-06T02:30:05.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.904+0000 7f95bd7fa700 1 -- 172.21.15.31:0/2282829601 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b8011670 con 0x7f95c8142c00 2023-10-06T02:30:05.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.904+0000 7f95ce79d700 1 -- 172.21.15.31:0/2282829601 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95c80b0510 con 0x7f95c8142c00 2023-10-06T02:30:05.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.904+0000 7f95ce79d700 1 -- 172.21.15.31:0/2282829601 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95c80b0ad0 con 0x7f95c8142c00 2023-10-06T02:30:05.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.905+0000 7f95ce79d700 1 -- 172.21.15.31:0/2282829601 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95c80b0fd0 con 0x7f95c8142c00 2023-10-06T02:30:05.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.906+0000 7f95bd7fa700 1 -- 172.21.15.31:0/2282829601 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f95b8011aa0 con 0x7f95c8142c00 2023-10-06T02:30:05.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.907+0000 7f95bd7fa700 1 --2- 172.21.15.31:0/2282829601 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f95ac0674d0 0x7f95ac069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:05.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.907+0000 7f95bd7fa700 1 -- 172.21.15.31:0/2282829601 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5320+0+0 (secure 0 0 0) 0x7f95b8086240 con 0x7f95c8142c00 2023-10-06T02:30:05.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.907+0000 7f95cd79b700 1 --2- 172.21.15.31:0/2282829601 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f95ac0674d0 0x7f95ac069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:05.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.908+0000 7f95cd79b700 1 --2- 172.21.15.31:0/2282829601 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f95ac0674d0 0x7f95ac069980 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f95c4003a80 tx=0x7f95c4007ec0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:05.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:05.911+0000 7f95bd7fa700 1 -- 172.21.15.31:0/2282829601 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f95b8052e00 con 0x7f95c8142c00 2023-10-06T02:30:06.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:06.159+0000 7f95ce79d700 1 -- 172.21.15.31:0/2282829601 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f95c804c3c0 con 0x7f95c8142c00 2023-10-06T02:30:06.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:06.159+0000 7f95bd7fa700 1 -- 172.21.15.31:0/2282829601 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+3526 (secure 0 0 0) 0x7f95b8015de0 con 0x7f95c8142c00 2023-10-06T02:30:06.160 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:30:06.160 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":83,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6338,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/1906951645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":1906951645},{"type":"v1","addr":"172.21.15.62:6834","nonce":1906951645}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75},{"gid":6341,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3788411327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3788411327},{"type":"v1","addr":"172.21.15.62:6835","nonce":3788411327}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75},{"gid":7569,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6833/581939695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":581939695},{"type":"v1","addr":"172.21.15.31:6833","nonce":581939695}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75},{"gid":7914,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/1879581192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":1879581192},{"type":"v1","addr":"172.21.15.31:6835","nonce":1879581192}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":83}],"filesystems":[]} 2023-10-06T02:30:06.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:06.162+0000 7f95ce79d700 1 -- 172.21.15.31:0/2282829601 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f95ac0674d0 msgr2=0x7f95ac069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:06.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:06.162+0000 7f95ce79d700 1 --2- 172.21.15.31:0/2282829601 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f95ac0674d0 0x7f95ac069980 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f95c4003a80 tx=0x7f95c4007ec0 comp rx=0 tx=0).stop 2023-10-06T02:30:06.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:06.162+0000 7f95ce79d700 1 -- 172.21.15.31:0/2282829601 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95c8142c00 msgr2=0x7f95c80b12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:06.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:06.162+0000 7f95ce79d700 1 --2- 172.21.15.31:0/2282829601 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95c8142c00 0x7f95c80b12d0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f95c806f290 tx=0x7f95b8017000 comp rx=0 tx=0).stop 2023-10-06T02:30:06.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:06.162+0000 7f95ce79d700 1 -- 172.21.15.31:0/2282829601 shutdown_connections 2023-10-06T02:30:06.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:06.162+0000 7f95ce79d700 1 --2- 172.21.15.31:0/2282829601 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f95ac0674d0 0x7f95ac069980 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:06.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:06.162+0000 7f95ce79d700 1 --2- 172.21.15.31:0/2282829601 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f95c8142fd0 0x7f95c80c61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:06.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:06.162+0000 7f95ce79d700 1 --2- 172.21.15.31:0/2282829601 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95c80c7dc0 0x7f95c8140620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:06.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:06.162+0000 7f95ce79d700 1 --2- 172.21.15.31:0/2282829601 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95c8142c00 0x7f95c80b12d0 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:06.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:06.162+0000 7f95ce79d700 1 -- 172.21.15.31:0/2282829601 >> 172.21.15.31:0/2282829601 conn(0x7f95c80675a0 msgr2=0x7f95c80c3120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:06.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:06.162+0000 7f95ce79d700 1 -- 172.21.15.31:0/2282829601 shutdown_connections 2023-10-06T02:30:06.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:06.162+0000 7f95ce79d700 1 -- 172.21.15.31:0/2282829601 wait complete. 2023-10-06T02:30:06.164 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 83 2023-10-06T02:30:06.181 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T02:30:06.181 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T02:30:06.181 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T02:30:06.181 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T02:30:06.181 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T02:30:06.181 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T02:30:06.182 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T02:30:06.182 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T02:30:06.182 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T02:30:06.182 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T02:30:06.182 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T02:30:06.182 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T02:30:06.183 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T02:30:06.183 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T02:30:06.183 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T02:30:06.183 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T02:30:06.196 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T02:30:06.196 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T02:30:06.197 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T02:30:06.197 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T02:30:06.198 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T02:30:06.199 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T02:30:06.199 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T02:30:06.199 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T02:30:06.241 INFO:tasks.ceph.mds.a:Started 2023-10-06T02:30:06.241 DEBUG:teuthology.parallel:result is None 2023-10-06T02:30:06.244 INFO:tasks.ceph.mds.b:Started 2023-10-06T02:30:06.244 INFO:tasks.ceph.mds.c:Started 2023-10-06T02:30:06.244 DEBUG:teuthology.parallel:result is None 2023-10-06T02:30:06.244 DEBUG:teuthology.parallel:result is None 2023-10-06T02:30:06.246 INFO:tasks.ceph.mds.d:Started 2023-10-06T02:30:06.247 DEBUG:teuthology.parallel:result is None 2023-10-06T02:30:06.247 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T02:30:06.379 INFO:tasks.ceph.mds.a.smithi031.stdout:starting mds.a at 2023-10-06T02:30:06.379 INFO:tasks.ceph.mds.c.smithi031.stdout:starting mds.c at 2023-10-06T02:30:06.389 INFO:tasks.ceph.mds.b.smithi062.stdout:starting mds.b at 2023-10-06T02:30:06.389 INFO:tasks.ceph.mds.d.smithi062.stdout:starting mds.d at 2023-10-06T02:30:06.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:06.447+0000 7f056d99d700 1 Processor -- start 2023-10-06T02:30:06.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:06.447+0000 7f056d99d700 1 -- start start 2023-10-06T02:30:06.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:06.448+0000 7f056d99d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f056815de50 0x7f05680cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:06.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:06.448+0000 7f056d99d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f05680cef40 0x7f056815c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:06.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:06.448+0000 7f056d99d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05680cf3c0 con 0x7f0568153c30 2023-10-06T02:30:06.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:06.448+0000 7f056d99d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f056815afd0 con 0x7f056815de50 2023-10-06T02:30:06.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:06.448+0000 7f056d99d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f056815b2b0 con 0x7f05680cef40 2023-10-06T02:30:06.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:06.448+0000 7f0567fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f056815de50 0x7f05680cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:06.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:06.448+0000 7f056c99b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f0568153c30 0x7f056815a5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56142/0 (socket says 172.21.15.31:56142) 2023-10-06T02:30:06.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:06.448+0000 7f056c99b700 1 -- 172.21.15.31:0/2604920207 learned_addr learned my addr 172.21.15.31:0/2604920207 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:30:06.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:06.449+0000 7f056d19c700 1 --2- 172.21.15.31:0/2604920207 >> v2:172.21.15.62:3300/0 conn(0x7f05680cef40 0x7f056815c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:06.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:06.449+0000 7f05677fe700 1 -- 172.21.15.31:0/2604920207 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1720512334 0 0) 0x7f05680cf3c0 con 0x7f0568153c30 2023-10-06T02:30:06.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:06.449+0000 7f0567fff700 1 -- 172.21.15.31:0/2604920207 >> v2:172.21.15.62:3300/0 conn(0x7f05680cef40 msgr2=0x7f056815c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:06.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:06.449+0000 7f0567fff700 1 --2- 172.21.15.31:0/2604920207 >> v2:172.21.15.62:3300/0 conn(0x7f05680cef40 0x7f056815c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:06.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:06.449+0000 7f0567fff700 1 -- 172.21.15.31:0/2604920207 >> v1:172.21.15.31:6789/0 conn(0x7f0568153c30 legacy=0x7f056815a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:06.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:06.449+0000 7f0567fff700 1 -- 172.21.15.31:0/2604920207 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f056815b590 con 0x7f056815de50 2023-10-06T02:30:06.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:06.449+0000 7f0567fff700 1 --2- 172.21.15.31:0/2604920207 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f056815de50 0x7f05680cea00 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f05680cdab0 tx=0x7f05540091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ead41fe86daf1d5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:06.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:06.449+0000 7f05677fe700 1 -- 172.21.15.31:0/2604920207 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0554011630 con 0x7f056815de50 2023-10-06T02:30:06.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:06.450+0000 7f05677fe700 1 -- 172.21.15.31:0/2604920207 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0554011790 con 0x7f056815de50 2023-10-06T02:30:06.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:06.450+0000 7f05677fe700 1 -- 172.21.15.31:0/2604920207 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0554011b30 con 0x7f056815de50 2023-10-06T02:30:06.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:06.450+0000 7f056d99d700 1 -- 172.21.15.31:0/2604920207 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f056815de50 msgr2=0x7f05680cea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:06.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:06.450+0000 7f056d99d700 1 --2- 172.21.15.31:0/2604920207 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f056815de50 0x7f05680cea00 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f05680cdab0 tx=0x7f05540091a0 comp rx=0 tx=0).stop 2023-10-06T02:30:06.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:06.450+0000 7f056d99d700 1 -- 172.21.15.31:0/2604920207 shutdown_connections 2023-10-06T02:30:06.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:06.450+0000 7f056d99d700 1 --2- 172.21.15.31:0/2604920207 >> v2:172.21.15.62:3300/0 conn(0x7f05680cef40 0x7f056815c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:06.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:06.450+0000 7f056d99d700 1 --2- 172.21.15.31:0/2604920207 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f056815de50 0x7f05680cea00 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:06.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:06.450+0000 7f056d99d700 1 -- 172.21.15.31:0/2604920207 >> 172.21.15.31:0/2604920207 conn(0x7f05680703a0 msgr2=0x7f05680707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:06.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:06.450+0000 7f056d99d700 1 -- 172.21.15.31:0/2604920207 shutdown_connections 2023-10-06T02:30:06.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:06.450+0000 7f056d99d700 1 -- 172.21.15.31:0/2604920207 wait complete. 2023-10-06T02:30:06.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:06.452+0000 7f056d99d700 1 Processor -- start 2023-10-06T02:30:06.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:06.452+0000 7f056d99d700 1 -- start start 2023-10-06T02:30:06.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:06.453+0000 7f056d99d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05680cef40 0x7f0568143d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:06.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:06.453+0000 7f056d99d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0568153c30 0x7f05680c9950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:06.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:06.453+0000 7f056d99d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f056815de50 0x7f05680b4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:06.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:06.453+0000 7f056d99d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05680b4080 con 0x7f056815de50 2023-10-06T02:30:06.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:06.453+0000 7f056d99d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05680b3470 con 0x7f05680cef40 2023-10-06T02:30:06.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:06.453+0000 7f056d99d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05680b3750 con 0x7f0568153c30 2023-10-06T02:30:06.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:06.453+0000 7f056d19c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f056815de50 0x7f05680b4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:06.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:06.453+0000 7f056d19c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f056815de50 0x7f05680b4fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50936/0 (socket says 172.21.15.31:50936) 2023-10-06T02:30:06.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:06.453+0000 7f056d19c700 1 -- 172.21.15.31:0/4227338354 learned_addr learned my addr 172.21.15.31:0/4227338354 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:06.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:06.453+0000 7f0567fff700 1 --2- 172.21.15.31:0/4227338354 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0568153c30 0x7f05680c9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:06.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:06.453+0000 7f056c99b700 1 --2- 172.21.15.31:0/4227338354 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05680cef40 0x7f0568143d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:06.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:06.453+0000 7f056d19c700 1 -- 172.21.15.31:0/4227338354 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0568153c30 msgr2=0x7f05680c9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:06.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:06.453+0000 7f056d19c700 1 --2- 172.21.15.31:0/4227338354 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0568153c30 0x7f05680c9950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:06.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:06.453+0000 7f056d19c700 1 -- 172.21.15.31:0/4227338354 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05680cef40 msgr2=0x7f0568143d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:06.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:06.453+0000 7f056d19c700 1 --2- 172.21.15.31:0/4227338354 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05680cef40 0x7f0568143d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:06.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:06.453+0000 7f056d19c700 1 -- 172.21.15.31:0/4227338354 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0554009420 con 0x7f056815de50 2023-10-06T02:30:06.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:06.454+0000 7f056d19c700 1 --2- 172.21.15.31:0/4227338354 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f056815de50 0x7f05680b4fc0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f0560009df0 tx=0x7f056000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:06.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:06.454+0000 7f0564ff9700 1 -- 172.21.15.31:0/4227338354 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f056000cb50 con 0x7f056815de50 2023-10-06T02:30:06.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:06.454+0000 7f0564ff9700 1 -- 172.21.15.31:0/4227338354 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f056000f040 con 0x7f056815de50 2023-10-06T02:30:06.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:06.454+0000 7f0564ff9700 1 -- 172.21.15.31:0/4227338354 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0560014480 con 0x7f056815de50 2023-10-06T02:30:06.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:06.454+0000 7f056d99d700 1 -- 172.21.15.31:0/4227338354 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05680b3a30 con 0x7f056815de50 2023-10-06T02:30:06.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:06.454+0000 7f056d99d700 1 -- 172.21.15.31:0/4227338354 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f056816abe0 con 0x7f056815de50 2023-10-06T02:30:06.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:06.455+0000 7f056d99d700 1 -- 172.21.15.31:0/4227338354 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f056804c3c0 con 0x7f056815de50 2023-10-06T02:30:06.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:06.456+0000 7f0564ff9700 1 -- 172.21.15.31:0/4227338354 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f0560014830 con 0x7f056815de50 2023-10-06T02:30:06.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:06.457+0000 7f0564ff9700 1 --2- 172.21.15.31:0/4227338354 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f054c0674d0 0x7f054c069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:06.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:06.457+0000 7f0564ff9700 1 -- 172.21.15.31:0/4227338354 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5320+0+0 (secure 0 0 0) 0x7f05600858f0 con 0x7f056815de50 2023-10-06T02:30:06.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:06.457+0000 7f056c99b700 1 --2- 172.21.15.31:0/4227338354 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f054c0674d0 0x7f054c069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:06.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:06.458+0000 7f056c99b700 1 --2- 172.21.15.31:0/4227338354 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f054c0674d0 0x7f054c069980 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f05680709d0 tx=0x7f055c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:06.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:06.461+0000 7f0564ff9700 1 -- 172.21.15.31:0/4227338354 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0560052320 con 0x7f056815de50 2023-10-06T02:30:06.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:06.679+0000 7f056d99d700 1 -- 172.21.15.31:0/4227338354 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f05680cc070 con 0x7f056815de50 2023-10-06T02:30:07.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.103+0000 7f0564ff9700 1 -- 172.21.15.31:0/4227338354 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v126) v1 ==== 97+0+0 (secure 0 0 0) 0x7f0560055940 con 0x7f056815de50 2023-10-06T02:30:07.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.106+0000 7f056d99d700 1 -- 172.21.15.31:0/4227338354 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f054c0674d0 msgr2=0x7f054c069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:07.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.106+0000 7f056d99d700 1 --2- 172.21.15.31:0/4227338354 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f054c0674d0 0x7f054c069980 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f05680709d0 tx=0x7f055c009040 comp rx=0 tx=0).stop 2023-10-06T02:30:07.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.106+0000 7f056d99d700 1 -- 172.21.15.31:0/4227338354 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f056815de50 msgr2=0x7f05680b4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:07.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.106+0000 7f056d99d700 1 --2- 172.21.15.31:0/4227338354 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f056815de50 0x7f05680b4fc0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f0560009df0 tx=0x7f056000b040 comp rx=0 tx=0).stop 2023-10-06T02:30:07.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.106+0000 7f056d99d700 1 -- 172.21.15.31:0/4227338354 shutdown_connections 2023-10-06T02:30:07.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.106+0000 7f056d99d700 1 --2- 172.21.15.31:0/4227338354 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f056815de50 0x7f05680b4fc0 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:07.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.106+0000 7f056d99d700 1 --2- 172.21.15.31:0/4227338354 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05680cef40 0x7f0568143d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:07.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.106+0000 7f056d99d700 1 --2- 172.21.15.31:0/4227338354 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f054c0674d0 0x7f054c069980 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:07.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.106+0000 7f056d99d700 1 --2- 172.21.15.31:0/4227338354 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0568153c30 0x7f05680c9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:07.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.106+0000 7f056d99d700 1 -- 172.21.15.31:0/4227338354 >> 172.21.15.31:0/4227338354 conn(0x7f05680703a0 msgr2=0x7f0568152fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:07.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.106+0000 7f056d99d700 1 -- 172.21.15.31:0/4227338354 shutdown_connections 2023-10-06T02:30:07.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.106+0000 7f056d99d700 1 -- 172.21.15.31:0/4227338354 wait complete. 2023-10-06T02:30:07.108 INFO:teuthology.orchestra.run.smithi031.stderr: removed all blocklist entries 2023-10-06T02:30:07.126 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T02:30:07.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.314+0000 7f7cb6b9d700 1 Processor -- start 2023-10-06T02:30:07.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.314+0000 7f7cb6b9d700 1 -- start start 2023-10-06T02:30:07.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.315+0000 7f7cb6b9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7cb0159aa0 0x7f7cb00703a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:07.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.315+0000 7f7cb6b9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7cb00ca7b0 0x7f7cb00cabe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:07.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.315+0000 7f7cb6b9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cb006d3e0 con 0x7f7cb01596d0 2023-10-06T02:30:07.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.315+0000 7f7cb6b9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cb00cb120 con 0x7f7cb00ca7b0 2023-10-06T02:30:07.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.315+0000 7f7cb6b9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb006c7d0 con 0x7f7cb0159aa0 2023-10-06T02:30:07.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.316+0000 7f7cb639c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7cb00ca7b0 0x7f7cb00cabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:07.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.316+0000 7f7cb639c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7cb00ca7b0 0x7f7cb00cabe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50950/0 (socket says 172.21.15.31:50950) 2023-10-06T02:30:07.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.316+0000 7f7cb639c700 1 -- 172.21.15.31:0/1811556488 learned_addr learned my addr 172.21.15.31:0/1811556488 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:07.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.316+0000 7f7cb539a700 1 --2- 172.21.15.31:0/1811556488 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7cb0159aa0 0x7f7cb00703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:07.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.316+0000 7f7cb639c700 1 -- 172.21.15.31:0/1811556488 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7cb0159aa0 msgr2=0x7f7cb00703a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:07.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.316+0000 7f7cb639c700 1 --2- 172.21.15.31:0/1811556488 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7cb0159aa0 0x7f7cb00703a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:07.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.316+0000 7f7cb639c700 1 -- 172.21.15.31:0/1811556488 >> v1:172.21.15.62:6789/0 conn(0x7f7cb01596d0 legacy=0x7f7cb006d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:07.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.316+0000 7f7cb639c700 1 -- 172.21.15.31:0/1811556488 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cb006cab0 con 0x7f7cb00ca7b0 2023-10-06T02:30:07.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.316+0000 7f7cb639c700 1 --2- 172.21.15.31:0/1811556488 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7cb00ca7b0 0x7f7cb00cabe0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f7ca8009f20 tx=0x7f7ca8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34f1ded912f42f55 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:07.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.317+0000 7f7cb4b99700 1 -- 172.21.15.31:0/1811556488 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca800e070 con 0x7f7cb00ca7b0 2023-10-06T02:30:07.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.317+0000 7f7cb4b99700 1 -- 172.21.15.31:0/1811556488 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ca800b9d0 con 0x7f7cb00ca7b0 2023-10-06T02:30:07.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.317+0000 7f7cb4b99700 1 -- 172.21.15.31:0/1811556488 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca80092e0 con 0x7f7cb00ca7b0 2023-10-06T02:30:07.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.317+0000 7f7cb6b9d700 1 -- 172.21.15.31:0/1811556488 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7cb00ca7b0 msgr2=0x7f7cb00cabe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:07.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.317+0000 7f7cb6b9d700 1 --2- 172.21.15.31:0/1811556488 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7cb00ca7b0 0x7f7cb00cabe0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f7ca8009f20 tx=0x7f7ca8009840 comp rx=0 tx=0).stop 2023-10-06T02:30:07.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.317+0000 7f7cb6b9d700 1 -- 172.21.15.31:0/1811556488 shutdown_connections 2023-10-06T02:30:07.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.317+0000 7f7cb6b9d700 1 --2- 172.21.15.31:0/1811556488 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7cb00ca7b0 0x7f7cb00cabe0 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:07.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.317+0000 7f7cb6b9d700 1 --2- 172.21.15.31:0/1811556488 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7cb0159aa0 0x7f7cb00703a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:07.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.317+0000 7f7cb6b9d700 1 -- 172.21.15.31:0/1811556488 >> 172.21.15.31:0/1811556488 conn(0x7f7cb006e720 msgr2=0x7f7cb006eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:07.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.317+0000 7f7cb6b9d700 1 -- 172.21.15.31:0/1811556488 shutdown_connections 2023-10-06T02:30:07.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.317+0000 7f7cb6b9d700 1 -- 172.21.15.31:0/1811556488 wait complete. 2023-10-06T02:30:07.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.319+0000 7f7cb6b9d700 1 Processor -- start 2023-10-06T02:30:07.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.319+0000 7f7cb6b9d700 1 -- start start 2023-10-06T02:30:07.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.320+0000 7f7cb6b9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7cb00ca7b0 0x7f7cb016c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:07.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.320+0000 7f7cb6b9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7cb01596d0 0x7f7cb0157a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:07.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.320+0000 7f7cb6b9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7cb0159aa0 0x7f7cb0157fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:07.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.320+0000 7f7cb6b9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb01695e0 con 0x7f7cb00ca7b0 2023-10-06T02:30:07.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.320+0000 7f7cb6b9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb016a430 con 0x7f7cb01596d0 2023-10-06T02:30:07.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.320+0000 7f7cb6b9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb0169820 con 0x7f7cb0159aa0 2023-10-06T02:30:07.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.320+0000 7f7cb5b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7cb00ca7b0 0x7f7cb016c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:07.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.320+0000 7f7cb5b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7cb00ca7b0 0x7f7cb016c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50964/0 (socket says 172.21.15.31:50964) 2023-10-06T02:30:07.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.320+0000 7f7cb5b9b700 1 -- 172.21.15.31:0/2697477651 learned_addr learned my addr 172.21.15.31:0/2697477651 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:07.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.320+0000 7f7cb639c700 1 --2- 172.21.15.31:0/2697477651 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7cb0159aa0 0x7f7cb0157fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:07.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.320+0000 7f7cb539a700 1 --2- 172.21.15.31:0/2697477651 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7cb01596d0 0x7f7cb0157a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:07.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.321+0000 7f7cb5b9b700 1 -- 172.21.15.31:0/2697477651 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7cb0159aa0 msgr2=0x7f7cb0157fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:07.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.321+0000 7f7cb5b9b700 1 --2- 172.21.15.31:0/2697477651 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7cb0159aa0 0x7f7cb0157fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:07.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.321+0000 7f7cb5b9b700 1 -- 172.21.15.31:0/2697477651 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7cb01596d0 msgr2=0x7f7cb0157a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:07.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.321+0000 7f7cb5b9b700 1 --2- 172.21.15.31:0/2697477651 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7cb01596d0 0x7f7cb0157a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:07.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.321+0000 7f7cb5b9b700 1 -- 172.21.15.31:0/2697477651 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ca8009ae0 con 0x7f7cb00ca7b0 2023-10-06T02:30:07.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.321+0000 7f7cb5b9b700 1 --2- 172.21.15.31:0/2697477651 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7cb00ca7b0 0x7f7cb016c3d0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f7cac00c590 tx=0x7f7cac00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:07.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.321+0000 7f7ca5ffb700 1 -- 172.21.15.31:0/2697477651 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cac011590 con 0x7f7cb00ca7b0 2023-10-06T02:30:07.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.321+0000 7f7ca5ffb700 1 -- 172.21.15.31:0/2697477651 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cac0116f0 con 0x7f7cb00ca7b0 2023-10-06T02:30:07.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.321+0000 7f7cb6b9d700 1 -- 172.21.15.31:0/2697477651 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cb0169b00 con 0x7f7cb00ca7b0 2023-10-06T02:30:07.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.321+0000 7f7ca5ffb700 1 -- 172.21.15.31:0/2697477651 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cac011ab0 con 0x7f7cb00ca7b0 2023-10-06T02:30:07.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.321+0000 7f7cb6b9d700 1 -- 172.21.15.31:0/2697477651 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cb016a0c0 con 0x7f7cb00ca7b0 2023-10-06T02:30:07.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.322+0000 7f7cb6b9d700 1 -- 172.21.15.31:0/2697477651 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cb004c3c0 con 0x7f7cb00ca7b0 2023-10-06T02:30:07.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.323+0000 7f7ca5ffb700 1 -- 172.21.15.31:0/2697477651 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f7cac01b3b0 con 0x7f7cb00ca7b0 2023-10-06T02:30:07.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.324+0000 7f7ca5ffb700 1 --2- 172.21.15.31:0/2697477651 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7c940674d0 0x7f7c94069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:07.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.324+0000 7f7ca5ffb700 1 -- 172.21.15.31:0/2697477651 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5234+0+0 (secure 0 0 0) 0x7f7cac084320 con 0x7f7cb00ca7b0 2023-10-06T02:30:07.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.325+0000 7f7cb539a700 1 --2- 172.21.15.31:0/2697477651 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7c940674d0 0x7f7c94069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:07.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.325+0000 7f7cb539a700 1 --2- 172.21.15.31:0/2697477651 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7c940674d0 0x7f7c94069980 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7cb00cd2c0 tx=0x7f7ca0008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:07.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.328+0000 7f7ca5ffb700 1 -- 172.21.15.31:0/2697477651 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7cac050eb0 con 0x7f7cb00ca7b0 2023-10-06T02:30:07.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.611+0000 7f7cb6b9d700 1 -- 172.21.15.31:0/2697477651 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f7cb006cd90 con 0x7f7cb00ca7b0 2023-10-06T02:30:07.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.611+0000 7f7ca5ffb700 1 -- 172.21.15.31:0/2697477651 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f7cac017090 con 0x7f7cb00ca7b0 2023-10-06T02:30:07.612 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:30:07.612 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T02:30:07.612 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_dump": [ 2023-10-06T02:30:07.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:30:07.612 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.a", 2023-10-06T02:30:07.613 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBLbx9lbjrOGxAAVoSOoDaZb1temL41F4Nong==", 2023-10-06T02:30:07.613 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:30:07.613 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:30:07.613 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T02:30:07.613 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T02:30:07.613 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:30:07.613 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:30:07.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:30:07.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:30:07.613 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.b", 2023-10-06T02:30:07.613 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBLbx9lFgsTMRAATTklzroiyQbTLiZc9j1BHQ==", 2023-10-06T02:30:07.613 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:30:07.614 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:30:07.614 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T02:30:07.614 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T02:30:07.614 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:30:07.614 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:30:07.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:30:07.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:30:07.614 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.c", 2023-10-06T02:30:07.614 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBLbx9lezVtJhAA8wK7+dJ1OmIc+iXJkexK0w==", 2023-10-06T02:30:07.614 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:30:07.614 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:30:07.615 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T02:30:07.615 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T02:30:07.615 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:30:07.615 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:30:07.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:30:07.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:30:07.615 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.d", 2023-10-06T02:30:07.615 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBMbx9l74gWABAApBu2nVv12rGA+OBfBnEc3A==", 2023-10-06T02:30:07.615 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:30:07.615 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:30:07.615 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T02:30:07.615 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T02:30:07.616 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:30:07.616 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:30:07.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:30:07.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:30:07.616 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.0", 2023-10-06T02:30:07.616 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBPbx9lgj6+MBAAXSUD0GCG+rO/9DQBZYt/8w==", 2023-10-06T02:30:07.616 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:30:07.616 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:30:07.616 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:30:07.616 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:30:07.616 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:30:07.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:30:07.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:30:07.617 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.1", 2023-10-06T02:30:07.617 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBSbx9lXhMTNBAAeEdPnMP71ZGb6k2Ddg5raQ==", 2023-10-06T02:30:07.617 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:30:07.617 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:30:07.617 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:30:07.617 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:30:07.617 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:30:07.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:30:07.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:30:07.617 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.2", 2023-10-06T02:30:07.617 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBVbx9l4Ly0NhAATYXYDAjYuP53zjqK0yMboA==", 2023-10-06T02:30:07.617 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:30:07.617 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:30:07.618 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:30:07.618 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:30:07.618 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:30:07.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:30:07.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:30:07.618 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.3", 2023-10-06T02:30:07.618 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBZbx9lzsi4ARAAbGnNV0QyL/7itzy827+chg==", 2023-10-06T02:30:07.618 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:30:07.618 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:30:07.618 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:30:07.618 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:30:07.619 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:30:07.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:30:07.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:30:07.619 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.4", 2023-10-06T02:30:07.619 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBfbx9lW4OSEhAAj7HnVh/yq12zzCaXd6BHCA==", 2023-10-06T02:30:07.619 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:30:07.619 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:30:07.619 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:30:07.619 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:30:07.619 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:30:07.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:30:07.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:30:07.620 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.5", 2023-10-06T02:30:07.620 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBibx9lBcaIFRAAV8wq8QjsCOkI9Pzf/E+9bA==", 2023-10-06T02:30:07.620 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:30:07.620 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:30:07.620 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:30:07.620 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:30:07.620 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:30:07.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:30:07.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:30:07.620 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.6", 2023-10-06T02:30:07.620 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBlbx9lA7S+FxAA38mlSAurT+DpwMzH53zFcw==", 2023-10-06T02:30:07.621 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:30:07.621 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:30:07.621 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:30:07.621 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:30:07.621 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:30:07.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:30:07.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:30:07.621 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.7", 2023-10-06T02:30:07.621 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBobx9l5RkdHhAAxn0Z8IAE/o9oe2LRp3O9gg==", 2023-10-06T02:30:07.621 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:30:07.621 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:30:07.621 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:30:07.622 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:30:07.622 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:30:07.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:30:07.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:30:07.622 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.0", 2023-10-06T02:30:07.622 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBMbx9lh44fCRAA7sMISvk1t7dEJg4ZcBiMcA==", 2023-10-06T02:30:07.622 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:30:07.622 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:30:07.622 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow r", 2023-10-06T02:30:07.622 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T02:30:07.622 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:30:07.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:30:07.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:30:07.623 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.1", 2023-10-06T02:30:07.623 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBMbx9lP0zqFhAAlov2y7ipSvS3AkOdQkhg6g==", 2023-10-06T02:30:07.623 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:30:07.623 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:30:07.623 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-10-06T02:30:07.623 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-10-06T02:30:07.623 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-10-06T02:30:07.623 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:30:07.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:30:07.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:30:07.623 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.2", 2023-10-06T02:30:07.624 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBMbx9lpxYAEBAACtiI1XgQXM8yGGcpFyNK1w==", 2023-10-06T02:30:07.624 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:30:07.624 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:30:07.624 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-10-06T02:30:07.624 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-10-06T02:30:07.624 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-10-06T02:30:07.624 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:30:07.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:30:07.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:30:07.624 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.3", 2023-10-06T02:30:07.624 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBMbx9l81/PHRAAYFOoaezu0pwE0NdtnlYfVA==", 2023-10-06T02:30:07.624 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:30:07.624 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:30:07.625 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-10-06T02:30:07.625 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-10-06T02:30:07.625 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-10-06T02:30:07.625 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:30:07.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:30:07.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:30:07.625 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.admin", 2023-10-06T02:30:07.625 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBKbx9liFl1JBAAFyyHYNLVx1/Ex5a2MKm8rA==", 2023-10-06T02:30:07.625 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:30:07.625 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-10-06T02:30:07.625 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T02:30:07.625 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T02:30:07.626 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:30:07.626 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:30:07.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:30:07.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:30:07.626 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mgr.x", 2023-10-06T02:30:07.626 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBLbx9lTBbIFBAAX2wsioKL4Nx4qz/qwsCsNA==", 2023-10-06T02:30:07.626 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:30:07.626 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-10-06T02:30:07.626 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile mgr", 2023-10-06T02:30:07.626 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:30:07.626 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:30:07.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:30:07.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:30:07.627 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mgr.y", 2023-10-06T02:30:07.627 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBLbx9l4+oLDRAA5YVo3q6FI9e9W/rJYROIsg==", 2023-10-06T02:30:07.627 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:30:07.627 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-10-06T02:30:07.627 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile mgr", 2023-10-06T02:30:07.627 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:30:07.627 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:30:07.627 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:30:07.627 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-10-06T02:30:07.627 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T02:30:07.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.614+0000 7f7cb6b9d700 1 -- 172.21.15.31:0/2697477651 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7c940674d0 msgr2=0x7f7c94069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:07.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.614+0000 7f7cb6b9d700 1 --2- 172.21.15.31:0/2697477651 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7c940674d0 0x7f7c94069980 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7cb00cd2c0 tx=0x7f7ca0008040 comp rx=0 tx=0).stop 2023-10-06T02:30:07.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.614+0000 7f7cb6b9d700 1 -- 172.21.15.31:0/2697477651 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7cb00ca7b0 msgr2=0x7f7cb016c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:07.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.614+0000 7f7cb6b9d700 1 --2- 172.21.15.31:0/2697477651 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7cb00ca7b0 0x7f7cb016c3d0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f7cac00c590 tx=0x7f7cac00c950 comp rx=0 tx=0).stop 2023-10-06T02:30:07.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.614+0000 7f7cb6b9d700 1 -- 172.21.15.31:0/2697477651 shutdown_connections 2023-10-06T02:30:07.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.614+0000 7f7cb6b9d700 1 --2- 172.21.15.31:0/2697477651 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7c940674d0 0x7f7c94069980 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:07.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.614+0000 7f7cb6b9d700 1 --2- 172.21.15.31:0/2697477651 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7cb0159aa0 0x7f7cb0157fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:07.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.614+0000 7f7cb6b9d700 1 --2- 172.21.15.31:0/2697477651 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7cb00ca7b0 0x7f7cb016c3d0 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:07.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.614+0000 7f7cb6b9d700 1 --2- 172.21.15.31:0/2697477651 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7cb01596d0 0x7f7cb0157a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:07.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.614+0000 7f7cb6b9d700 1 -- 172.21.15.31:0/2697477651 >> 172.21.15.31:0/2697477651 conn(0x7f7cb006e720 msgr2=0x7f7cb015afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:07.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.614+0000 7f7cb6b9d700 1 -- 172.21.15.31:0/2697477651 shutdown_connections 2023-10-06T02:30:07.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.614+0000 7f7cb6b9d700 1 -- 172.21.15.31:0/2697477651 wait complete. 2023-10-06T02:30:07.632 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:30:07.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.819+0000 7f35b299d700 1 Processor -- start 2023-10-06T02:30:07.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.819+0000 7f35b299d700 1 -- start start 2023-10-06T02:30:07.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.820+0000 7f35b299d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f35ac0c9ab0 0x7f35ac06a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:07.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.820+0000 7f35b299d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f35ac06db40 0x7f35ac161230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:07.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.820+0000 7f35b299d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35ac06a800 con 0x7f35ac157410 2023-10-06T02:30:07.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.820+0000 7f35b299d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35ac06c6a0 con 0x7f35ac0c9ab0 2023-10-06T02:30:07.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.820+0000 7f35b299d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35ac06aa30 con 0x7f35ac06db40 2023-10-06T02:30:07.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.821+0000 7f35b119a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f35ac0c9ab0 0x7f35ac06a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:07.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.821+0000 7f35b119a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f35ac0c9ab0 0x7f35ac06a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50966/0 (socket says 172.21.15.31:50966) 2023-10-06T02:30:07.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.821+0000 7f35b119a700 1 -- 172.21.15.31:0/3901848667 learned_addr learned my addr 172.21.15.31:0/3901848667 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:07.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.821+0000 7f35b219c700 1 --2- 172.21.15.31:0/3901848667 >> v2:172.21.15.62:3300/0 conn(0x7f35ac06db40 0x7f35ac161230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:07.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.821+0000 7f35b119a700 1 -- 172.21.15.31:0/3901848667 >> v2:172.21.15.62:3300/0 conn(0x7f35ac06db40 msgr2=0x7f35ac161230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:07.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.821+0000 7f35b119a700 1 --2- 172.21.15.31:0/3901848667 >> v2:172.21.15.62:3300/0 conn(0x7f35ac06db40 0x7f35ac161230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:07.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.821+0000 7f35b119a700 1 -- 172.21.15.31:0/3901848667 >> v1:172.21.15.62:6789/0 conn(0x7f35ac157410 legacy=0x7f35ac0c93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:07.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.821+0000 7f35b119a700 1 -- 172.21.15.31:0/3901848667 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35ac06ad10 con 0x7f35ac0c9ab0 2023-10-06T02:30:07.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.821+0000 7f35b119a700 1 --2- 172.21.15.31:0/3901848667 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35ac0c9ab0 0x7f35ac06a2c0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f35ac0c7e70 tx=0x7f359c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c40e77601360165 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:07.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.822+0000 7f35b0999700 1 -- 172.21.15.31:0/3901848667 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f359c00b820 con 0x7f35ac0c9ab0 2023-10-06T02:30:07.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.822+0000 7f35b0999700 1 -- 172.21.15.31:0/3901848667 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f359c00b980 con 0x7f35ac0c9ab0 2023-10-06T02:30:07.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.822+0000 7f35b0999700 1 -- 172.21.15.31:0/3901848667 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f359c00bd20 con 0x7f35ac0c9ab0 2023-10-06T02:30:07.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.822+0000 7f35b299d700 1 -- 172.21.15.31:0/3901848667 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35ac0c9ab0 msgr2=0x7f35ac06a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:07.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.822+0000 7f35b299d700 1 --2- 172.21.15.31:0/3901848667 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35ac0c9ab0 0x7f35ac06a2c0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f35ac0c7e70 tx=0x7f359c009250 comp rx=0 tx=0).stop 2023-10-06T02:30:07.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.822+0000 7f35b299d700 1 -- 172.21.15.31:0/3901848667 shutdown_connections 2023-10-06T02:30:07.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.822+0000 7f35b299d700 1 --2- 172.21.15.31:0/3901848667 >> v2:172.21.15.62:3300/0 conn(0x7f35ac06db40 0x7f35ac161230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:07.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.822+0000 7f35b299d700 1 --2- 172.21.15.31:0/3901848667 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35ac0c9ab0 0x7f35ac06a2c0 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:07.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.822+0000 7f35b299d700 1 -- 172.21.15.31:0/3901848667 >> 172.21.15.31:0/3901848667 conn(0x7f35ac06e720 msgr2=0x7f35ac06eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:07.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.822+0000 7f35b299d700 1 -- 172.21.15.31:0/3901848667 shutdown_connections 2023-10-06T02:30:07.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.822+0000 7f35b299d700 1 -- 172.21.15.31:0/3901848667 wait complete. 2023-10-06T02:30:07.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.824+0000 7f35b299d700 1 Processor -- start 2023-10-06T02:30:07.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.824+0000 7f35b299d700 1 -- start start 2023-10-06T02:30:07.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.825+0000 7f35b299d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f35ac06db40 0x7f35ac15a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:07.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.825+0000 7f35b299d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35ac0c9ab0 0x7f35ac1530a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:07.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.825+0000 7f35b299d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35ac157410 0x7f35ac0c2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:07.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.825+0000 7f35b299d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35ac0c4f40 con 0x7f35ac157410 2023-10-06T02:30:07.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.825+0000 7f35b299d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35ac15ab60 con 0x7f35ac0c9ab0 2023-10-06T02:30:07.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.825+0000 7f35b299d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35ac0c4610 con 0x7f35ac06db40 2023-10-06T02:30:07.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.825+0000 7f35b219c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35ac157410 0x7f35ac0c2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:07.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.825+0000 7f35b219c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35ac157410 0x7f35ac0c2c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50974/0 (socket says 172.21.15.31:50974) 2023-10-06T02:30:07.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.825+0000 7f35b219c700 1 -- 172.21.15.31:0/4266783844 learned_addr learned my addr 172.21.15.31:0/4266783844 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:07.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.825+0000 7f35b119a700 1 --2- 172.21.15.31:0/4266783844 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35ac0c9ab0 0x7f35ac1530a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:07.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.825+0000 7f35b199b700 1 --2- 172.21.15.31:0/4266783844 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f35ac06db40 0x7f35ac15a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:07.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.825+0000 7f35b219c700 1 -- 172.21.15.31:0/4266783844 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f35ac06db40 msgr2=0x7f35ac15a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:07.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.825+0000 7f35b219c700 1 --2- 172.21.15.31:0/4266783844 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f35ac06db40 0x7f35ac15a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:07.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.825+0000 7f35b219c700 1 -- 172.21.15.31:0/4266783844 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35ac0c9ab0 msgr2=0x7f35ac1530a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:07.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.825+0000 7f35b219c700 1 --2- 172.21.15.31:0/4266783844 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35ac0c9ab0 0x7f35ac1530a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:07.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.825+0000 7f35b219c700 1 -- 172.21.15.31:0/4266783844 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f359c0094f0 con 0x7f35ac157410 2023-10-06T02:30:07.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.826+0000 7f35b219c700 1 --2- 172.21.15.31:0/4266783844 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35ac157410 0x7f35ac0c2c00 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f35a4007f20 tx=0x7f35a4009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:07.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.826+0000 7f35a1ffb700 1 -- 172.21.15.31:0/4266783844 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a400ec40 con 0x7f35ac157410 2023-10-06T02:30:07.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.826+0000 7f35a1ffb700 1 -- 172.21.15.31:0/4266783844 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35a400edc0 con 0x7f35ac157410 2023-10-06T02:30:07.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.826+0000 7f35a1ffb700 1 -- 172.21.15.31:0/4266783844 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a4014670 con 0x7f35ac157410 2023-10-06T02:30:07.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.826+0000 7f35b299d700 1 -- 172.21.15.31:0/4266783844 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35ac0c48f0 con 0x7f35ac157410 2023-10-06T02:30:07.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.826+0000 7f35b299d700 1 -- 172.21.15.31:0/4266783844 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35ac15cdd0 con 0x7f35ac157410 2023-10-06T02:30:07.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.827+0000 7f35b299d700 1 -- 172.21.15.31:0/4266783844 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35ac04c3c0 con 0x7f35ac157410 2023-10-06T02:30:07.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.828+0000 7f35a1ffb700 1 -- 172.21.15.31:0/4266783844 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f35a40149e0 con 0x7f35ac157410 2023-10-06T02:30:07.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.829+0000 7f35a1ffb700 1 --2- 172.21.15.31:0/4266783844 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f35900674d0 0x7f3590069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:07.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.829+0000 7f35a1ffb700 1 -- 172.21.15.31:0/4266783844 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5234+0+0 (secure 0 0 0) 0x7f35a40854c0 con 0x7f35ac157410 2023-10-06T02:30:07.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.830+0000 7f35b199b700 1 --2- 172.21.15.31:0/4266783844 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f35900674d0 0x7f3590069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:07.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.830+0000 7f35b199b700 1 --2- 172.21.15.31:0/4266783844 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f35900674d0 0x7f3590069980 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f35ac0c2510 tx=0x7f35a8009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:07.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:07.834+0000 7f35a1ffb700 1 -- 172.21.15.31:0/4266783844 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f35a4052090 con 0x7f35ac157410 2023-10-06T02:30:08.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.038+0000 7f35b299d700 1 -- 172.21.15.31:0/4266783844 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f35ac0c1f80 con 0x7f35ac157410 2023-10-06T02:30:08.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.039+0000 7f35a1ffb700 1 -- 172.21.15.31:0/4266783844 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v128) v1 ==== 74+0+13656 (secure 0 0 0) 0x7f35a40556b0 con 0x7f35ac157410 2023-10-06T02:30:08.039 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:30:08.040 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":128,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:30:07.109246+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":20,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:30:08.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.041+0000 7f35b299d700 1 -- 172.21.15.31:0/4266783844 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f35900674d0 msgr2=0x7f3590069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:08.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.041+0000 7f35b299d700 1 --2- 172.21.15.31:0/4266783844 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f35900674d0 0x7f3590069980 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f35ac0c2510 tx=0x7f35a8009040 comp rx=0 tx=0).stop 2023-10-06T02:30:08.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.041+0000 7f35b299d700 1 -- 172.21.15.31:0/4266783844 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35ac157410 msgr2=0x7f35ac0c2c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:08.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.041+0000 7f35b299d700 1 --2- 172.21.15.31:0/4266783844 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35ac157410 0x7f35ac0c2c00 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f35a4007f20 tx=0x7f35a4009860 comp rx=0 tx=0).stop 2023-10-06T02:30:08.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.042+0000 7f35b299d700 1 -- 172.21.15.31:0/4266783844 shutdown_connections 2023-10-06T02:30:08.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.042+0000 7f35b299d700 1 --2- 172.21.15.31:0/4266783844 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35ac157410 0x7f35ac0c2c00 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:08.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.042+0000 7f35b299d700 1 --2- 172.21.15.31:0/4266783844 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f35900674d0 0x7f3590069980 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:08.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.042+0000 7f35b299d700 1 --2- 172.21.15.31:0/4266783844 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f35ac06db40 0x7f35ac15a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:08.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.042+0000 7f35b299d700 1 --2- 172.21.15.31:0/4266783844 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35ac0c9ab0 0x7f35ac1530a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:08.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.042+0000 7f35b299d700 1 -- 172.21.15.31:0/4266783844 >> 172.21.15.31:0/4266783844 conn(0x7f35ac06e720 msgr2=0x7f35ac0af840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:08.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.042+0000 7f35b299d700 1 -- 172.21.15.31:0/4266783844 shutdown_connections 2023-10-06T02:30:08.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.042+0000 7f35b299d700 1 -- 172.21.15.31:0/4266783844 wait complete. 2023-10-06T02:30:08.060 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T02:22:53.183425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T02:23:00.046967+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-10-06T02:30:08.060 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T02:30:08.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.251+0000 7f79349fe700 1 Processor -- start 2023-10-06T02:30:08.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.251+0000 7f79349fe700 1 -- start start 2023-10-06T02:30:08.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.252+0000 7f79349fe700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7930153a00 0x7f7930153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:08.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.252+0000 7f79349fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f79300cea00 0x7f79300cee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:08.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.252+0000 7f79349fe700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f793015ac50 con 0x7f793015de50 2023-10-06T02:30:08.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.252+0000 7f79349fe700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f793015d010 con 0x7f79300cea00 2023-10-06T02:30:08.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.252+0000 7f79349fe700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79300cf3c0 con 0x7f7930153a00 2023-10-06T02:30:08.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.252+0000 7f792f59e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f79300cea00 0x7f79300cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:08.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.252+0000 7f792f59e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f79300cea00 0x7f79300cee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39064/0 (socket says 172.21.15.31:39064) 2023-10-06T02:30:08.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.252+0000 7f792f59e700 1 -- 172.21.15.31:0/286981440 learned_addr learned my addr 172.21.15.31:0/286981440 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:08.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.253+0000 7f792ed9d700 1 --2- 172.21.15.31:0/286981440 >> v2:172.21.15.62:3300/0 conn(0x7f7930153a00 0x7f7930153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:08.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.253+0000 7f792f59e700 1 -- 172.21.15.31:0/286981440 >> v2:172.21.15.62:3300/0 conn(0x7f7930153a00 msgr2=0x7f7930153dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:08.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.253+0000 7f792f59e700 1 --2- 172.21.15.31:0/286981440 >> v2:172.21.15.62:3300/0 conn(0x7f7930153a00 0x7f7930153dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:08.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.253+0000 7f792f59e700 1 -- 172.21.15.31:0/286981440 >> v1:172.21.15.62:6789/0 conn(0x7f793015de50 legacy=0x7f793015a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:08.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.253+0000 7f792f59e700 1 -- 172.21.15.31:0/286981440 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f793015af60 con 0x7f79300cea00 2023-10-06T02:30:08.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.253+0000 7f792f59e700 1 --2- 172.21.15.31:0/286981440 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79300cea00 0x7f79300cee80 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f79300cac20 tx=0x7f7928009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c31dc9c6a4fa260 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:08.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.253+0000 7f792dd9b700 1 -- 172.21.15.31:0/286981440 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f792800e070 con 0x7f79300cea00 2023-10-06T02:30:08.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.254+0000 7f792dd9b700 1 -- 172.21.15.31:0/286981440 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f792800ba50 con 0x7f79300cea00 2023-10-06T02:30:08.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.254+0000 7f792dd9b700 1 -- 172.21.15.31:0/286981440 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79280092e0 con 0x7f79300cea00 2023-10-06T02:30:08.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.254+0000 7f79349fe700 1 -- 172.21.15.31:0/286981440 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79300cea00 msgr2=0x7f79300cee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:08.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.254+0000 7f79349fe700 1 --2- 172.21.15.31:0/286981440 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79300cea00 0x7f79300cee80 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f79300cac20 tx=0x7f7928009840 comp rx=0 tx=0).stop 2023-10-06T02:30:08.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.254+0000 7f79349fe700 1 -- 172.21.15.31:0/286981440 shutdown_connections 2023-10-06T02:30:08.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.254+0000 7f79349fe700 1 --2- 172.21.15.31:0/286981440 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79300cea00 0x7f79300cee80 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:08.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.254+0000 7f79349fe700 1 --2- 172.21.15.31:0/286981440 >> v2:172.21.15.62:3300/0 conn(0x7f7930153a00 0x7f7930153dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:08.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.254+0000 7f79349fe700 1 -- 172.21.15.31:0/286981440 >> 172.21.15.31:0/286981440 conn(0x7f79300703a0 msgr2=0x7f79300707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:08.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.254+0000 7f79349fe700 1 -- 172.21.15.31:0/286981440 shutdown_connections 2023-10-06T02:30:08.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.254+0000 7f79349fe700 1 -- 172.21.15.31:0/286981440 wait complete. 2023-10-06T02:30:08.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.256+0000 7f79349fe700 1 Processor -- start 2023-10-06T02:30:08.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.256+0000 7f79349fe700 1 -- start start 2023-10-06T02:30:08.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.257+0000 7f79349fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79300cea00 0x7f79300c9e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:08.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.257+0000 7f79349fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7930153a00 0x7f793014f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:08.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.257+0000 7f79349fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f793015de50 0x7f79300c7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:08.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.257+0000 7f79349fe700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79300ccaa0 con 0x7f79300cea00 2023-10-06T02:30:08.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.257+0000 7f79349fe700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79300cd250 con 0x7f793015de50 2023-10-06T02:30:08.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.257+0000 7f79349fe700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7930157a20 con 0x7f7930153a00 2023-10-06T02:30:08.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.257+0000 7f792ed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79300cea00 0x7f79300c9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:08.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.257+0000 7f792ed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79300cea00 0x7f79300c9e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39070/0 (socket says 172.21.15.31:39070) 2023-10-06T02:30:08.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.257+0000 7f792ed9d700 1 -- 172.21.15.31:0/1936091086 learned_addr learned my addr 172.21.15.31:0/1936091086 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:08.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.257+0000 7f792f59e700 1 --2- 172.21.15.31:0/1936091086 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f793015de50 0x7f79300c7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:08.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.257+0000 7f792e59c700 1 --2- 172.21.15.31:0/1936091086 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7930153a00 0x7f793014f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:08.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.257+0000 7f792ed9d700 1 -- 172.21.15.31:0/1936091086 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7930153a00 msgr2=0x7f793014f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:08.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.257+0000 7f792ed9d700 1 --2- 172.21.15.31:0/1936091086 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7930153a00 0x7f793014f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:08.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.257+0000 7f792ed9d700 1 -- 172.21.15.31:0/1936091086 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f793015de50 msgr2=0x7f79300c7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:08.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.257+0000 7f792ed9d700 1 --2- 172.21.15.31:0/1936091086 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f793015de50 0x7f79300c7ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:08.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.257+0000 7f792ed9d700 1 -- 172.21.15.31:0/1936091086 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7928009ae0 con 0x7f79300cea00 2023-10-06T02:30:08.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.257+0000 7f792f59e700 1 --2- 172.21.15.31:0/1936091086 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f793015de50 0x7f79300c7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:30:08.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.258+0000 7f792ed9d700 1 --2- 172.21.15.31:0/1936091086 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79300cea00 0x7f79300c9e60 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f792000d890 tx=0x7f7920008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:08.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.258+0000 7f7916ffd700 1 -- 172.21.15.31:0/1936091086 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79200125f0 con 0x7f79300cea00 2023-10-06T02:30:08.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.258+0000 7f7916ffd700 1 -- 172.21.15.31:0/1936091086 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7920012770 con 0x7f79300cea00 2023-10-06T02:30:08.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.258+0000 7f79349fe700 1 -- 172.21.15.31:0/1936091086 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79301576f0 con 0x7f79300cea00 2023-10-06T02:30:08.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.258+0000 7f7916ffd700 1 -- 172.21.15.31:0/1936091086 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7920012b30 con 0x7f79300cea00 2023-10-06T02:30:08.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.258+0000 7f79349fe700 1 -- 172.21.15.31:0/1936091086 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79300ccc90 con 0x7f79300cea00 2023-10-06T02:30:08.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.259+0000 7f79349fe700 1 -- 172.21.15.31:0/1936091086 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f793015d520 con 0x7f79300cea00 2023-10-06T02:30:08.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.260+0000 7f7916ffd700 1 -- 172.21.15.31:0/1936091086 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f792001e460 con 0x7f79300cea00 2023-10-06T02:30:08.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.261+0000 7f7916ffd700 1 --2- 172.21.15.31:0/1936091086 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f791c0674d0 0x7f791c069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:08.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.261+0000 7f7916ffd700 1 -- 172.21.15.31:0/1936091086 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5234+0+0 (secure 0 0 0) 0x7f7920085490 con 0x7f79300cea00 2023-10-06T02:30:08.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.261+0000 7f792e59c700 1 --2- 172.21.15.31:0/1936091086 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f791c0674d0 0x7f791c069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:08.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.262+0000 7f792e59c700 1 --2- 172.21.15.31:0/1936091086 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f791c0674d0 0x7f791c069980 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f79300cdd50 tx=0x7f7924009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:08.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.265+0000 7f7916ffd700 1 -- 172.21.15.31:0/1936091086 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f79200520e0 con 0x7f79300cea00 2023-10-06T02:30:08.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.465+0000 7f79349fe700 1 -- 172.21.15.31:0/1936091086 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f793004c3c0 con 0x7f79300cea00 2023-10-06T02:30:08.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.465+0000 7f7916ffd700 1 -- 172.21.15.31:0/1936091086 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v128) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7920055700 con 0x7f79300cea00 2023-10-06T02:30:08.466 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-10-06T02:30:08.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.468+0000 7f79349fe700 1 -- 172.21.15.31:0/1936091086 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f791c0674d0 msgr2=0x7f791c069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:08.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.468+0000 7f79349fe700 1 --2- 172.21.15.31:0/1936091086 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f791c0674d0 0x7f791c069980 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f79300cdd50 tx=0x7f7924009040 comp rx=0 tx=0).stop 2023-10-06T02:30:08.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.468+0000 7f79349fe700 1 -- 172.21.15.31:0/1936091086 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79300cea00 msgr2=0x7f79300c9e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:08.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.468+0000 7f79349fe700 1 --2- 172.21.15.31:0/1936091086 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79300cea00 0x7f79300c9e60 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f792000d890 tx=0x7f7920008040 comp rx=0 tx=0).stop 2023-10-06T02:30:08.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.468+0000 7f79349fe700 1 -- 172.21.15.31:0/1936091086 shutdown_connections 2023-10-06T02:30:08.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.468+0000 7f79349fe700 1 --2- 172.21.15.31:0/1936091086 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f793015de50 0x7f79300c7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:08.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.468+0000 7f79349fe700 1 --2- 172.21.15.31:0/1936091086 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79300cea00 0x7f79300c9e60 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:08.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.468+0000 7f79349fe700 1 --2- 172.21.15.31:0/1936091086 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f791c0674d0 0x7f791c069980 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:08.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.468+0000 7f79349fe700 1 --2- 172.21.15.31:0/1936091086 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7930153a00 0x7f793014f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:08.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.468+0000 7f79349fe700 1 -- 172.21.15.31:0/1936091086 >> 172.21.15.31:0/1936091086 conn(0x7f79300703a0 msgr2=0x7f79300d0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:08.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.468+0000 7f79349fe700 1 -- 172.21.15.31:0/1936091086 shutdown_connections 2023-10-06T02:30:08.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.468+0000 7f79349fe700 1 -- 172.21.15.31:0/1936091086 wait complete. 2023-10-06T02:30:08.486 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T02:30:08.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.677+0000 7fb00dd9d700 1 Processor -- start 2023-10-06T02:30:08.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.678+0000 7fb00dd9d700 1 -- start start 2023-10-06T02:30:08.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.679+0000 7fb00dd9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb0080c7be0 0x7fb00806abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:08.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.679+0000 7fb00dd9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb0080c7fb0 0x7fb00815a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:08.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.679+0000 7fb00dd9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb00815aed0 con 0x7fb00813f780 2023-10-06T02:30:08.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.679+0000 7fb00dd9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb00806bf10 con 0x7fb0080c7fb0 2023-10-06T02:30:08.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.679+0000 7fb00dd9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb00806b290 con 0x7fb0080c7be0 2023-10-06T02:30:08.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.679+0000 7fb007fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb0080c7fb0 0x7fb00815a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:08.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.679+0000 7fb007fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb0080c7fb0 0x7fb00815a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39076/0 (socket says 172.21.15.31:39076) 2023-10-06T02:30:08.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.679+0000 7fb007fff700 1 -- 172.21.15.31:0/1780491679 learned_addr learned my addr 172.21.15.31:0/1780491679 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:08.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.679+0000 7fb00cd9b700 1 --2- 172.21.15.31:0/1780491679 >> v2:172.21.15.62:3300/0 conn(0x7fb0080c7be0 0x7fb00806abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:08.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.680+0000 7fb007fff700 1 -- 172.21.15.31:0/1780491679 >> v2:172.21.15.62:3300/0 conn(0x7fb0080c7be0 msgr2=0x7fb00806abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:08.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.680+0000 7fb007fff700 1 --2- 172.21.15.31:0/1780491679 >> v2:172.21.15.62:3300/0 conn(0x7fb0080c7be0 0x7fb00806abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:08.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.680+0000 7fb007fff700 1 -- 172.21.15.31:0/1780491679 >> v1:172.21.15.62:6789/0 conn(0x7fb00813f780 legacy=0x7fb00813fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:08.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.680+0000 7fb007fff700 1 -- 172.21.15.31:0/1780491679 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb00806b570 con 0x7fb0080c7fb0 2023-10-06T02:30:08.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.680+0000 7fb007fff700 1 --2- 172.21.15.31:0/1780491679 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0080c7fb0 0x7fb00815a990 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fb0080c8b00 tx=0x7faff4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ec6aa52ae9626a0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:08.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.680+0000 7fb0077fe700 1 -- 172.21.15.31:0/1780491679 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff4009e70 con 0x7fb0080c7fb0 2023-10-06T02:30:08.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.680+0000 7fb0077fe700 1 -- 172.21.15.31:0/1780491679 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faff400b860 con 0x7fb0080c7fb0 2023-10-06T02:30:08.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.680+0000 7fb0077fe700 1 -- 172.21.15.31:0/1780491679 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff400bc00 con 0x7fb0080c7fb0 2023-10-06T02:30:08.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.681+0000 7fb00dd9d700 1 -- 172.21.15.31:0/1780491679 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0080c7fb0 msgr2=0x7fb00815a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:08.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.681+0000 7fb00dd9d700 1 --2- 172.21.15.31:0/1780491679 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0080c7fb0 0x7fb00815a990 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fb0080c8b00 tx=0x7faff4009250 comp rx=0 tx=0).stop 2023-10-06T02:30:08.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.681+0000 7fb00dd9d700 1 -- 172.21.15.31:0/1780491679 shutdown_connections 2023-10-06T02:30:08.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.681+0000 7fb00dd9d700 1 --2- 172.21.15.31:0/1780491679 >> v2:172.21.15.62:3300/0 conn(0x7fb0080c7be0 0x7fb00806abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:08.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.681+0000 7fb00dd9d700 1 --2- 172.21.15.31:0/1780491679 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0080c7fb0 0x7fb00815a990 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:08.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.681+0000 7fb00dd9d700 1 -- 172.21.15.31:0/1780491679 >> 172.21.15.31:0/1780491679 conn(0x7fb00806d110 msgr2=0x7fb0080cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:08.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.681+0000 7fb00dd9d700 1 -- 172.21.15.31:0/1780491679 shutdown_connections 2023-10-06T02:30:08.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.681+0000 7fb00dd9d700 1 -- 172.21.15.31:0/1780491679 wait complete. 2023-10-06T02:30:08.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.682+0000 7fb00dd9d700 1 Processor -- start 2023-10-06T02:30:08.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.683+0000 7fb00dd9d700 1 -- start start 2023-10-06T02:30:08.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.683+0000 7fb00dd9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0080c7be0 0x7fb0080c31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:08.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.683+0000 7fb00dd9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0080c7fb0 0x7fb008156a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:08.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.683+0000 7fb00dd9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb00813f780 0x7fb008155bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:08.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.683+0000 7fb00dd9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb008157140 con 0x7fb0080c7fb0 2023-10-06T02:30:08.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.683+0000 7fb00dd9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb008157420 con 0x7fb0080c7be0 2023-10-06T02:30:08.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.683+0000 7fb00dd9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb008157700 con 0x7fb00813f780 2023-10-06T02:30:08.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.684+0000 7fb007fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0080c7fb0 0x7fb008156a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:08.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.684+0000 7fb007fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0080c7fb0 0x7fb008156a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39080/0 (socket says 172.21.15.31:39080) 2023-10-06T02:30:08.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.684+0000 7fb007fff700 1 -- 172.21.15.31:0/4243261865 learned_addr learned my addr 172.21.15.31:0/4243261865 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:08.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.684+0000 7fb00cd9b700 1 --2- 172.21.15.31:0/4243261865 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0080c7be0 0x7fb0080c31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:08.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.684+0000 7fb00d59c700 1 --2- 172.21.15.31:0/4243261865 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb00813f780 0x7fb008155bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:08.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.684+0000 7fb007fff700 1 -- 172.21.15.31:0/4243261865 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb00813f780 msgr2=0x7fb008155bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:08.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.684+0000 7fb007fff700 1 --2- 172.21.15.31:0/4243261865 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb00813f780 0x7fb008155bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:08.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.684+0000 7fb007fff700 1 -- 172.21.15.31:0/4243261865 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0080c7be0 msgr2=0x7fb0080c31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:08.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.684+0000 7fb007fff700 1 --2- 172.21.15.31:0/4243261865 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0080c7be0 0x7fb0080c31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:08.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.684+0000 7fb007fff700 1 -- 172.21.15.31:0/4243261865 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faff40094f0 con 0x7fb0080c7fb0 2023-10-06T02:30:08.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.684+0000 7fb00cd9b700 1 --2- 172.21.15.31:0/4243261865 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0080c7be0 0x7fb0080c31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:30:08.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.684+0000 7fb00d59c700 1 --2- 172.21.15.31:0/4243261865 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb00813f780 0x7fb008155bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:30:08.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.684+0000 7fb007fff700 1 --2- 172.21.15.31:0/4243261865 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0080c7fb0 0x7fb008156a80 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fb0080693f0 tx=0x7faff4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:08.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.685+0000 7fb004ff9700 1 -- 172.21.15.31:0/4243261865 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff4011420 con 0x7fb0080c7fb0 2023-10-06T02:30:08.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.685+0000 7fb004ff9700 1 -- 172.21.15.31:0/4243261865 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faff40115a0 con 0x7fb0080c7fb0 2023-10-06T02:30:08.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.685+0000 7fb00dd9d700 1 -- 172.21.15.31:0/4243261865 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0081579e0 con 0x7fb0080c7fb0 2023-10-06T02:30:08.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.685+0000 7fb004ff9700 1 -- 172.21.15.31:0/4243261865 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff4011960 con 0x7fb0080c7fb0 2023-10-06T02:30:08.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.685+0000 7fb00dd9d700 1 -- 172.21.15.31:0/4243261865 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb008156100 con 0x7fb0080c7fb0 2023-10-06T02:30:08.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.686+0000 7fb00dd9d700 1 -- 172.21.15.31:0/4243261865 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb008158160 con 0x7fb0080c7fb0 2023-10-06T02:30:08.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.687+0000 7fb004ff9700 1 -- 172.21.15.31:0/4243261865 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7faff40253f0 con 0x7fb0080c7fb0 2023-10-06T02:30:08.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.687+0000 7fb004ff9700 1 --2- 172.21.15.31:0/4243261865 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fafec0674d0 0x7fafec069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:08.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.688+0000 7fb004ff9700 1 -- 172.21.15.31:0/4243261865 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5234+0+0 (secure 0 0 0) 0x7faff4086800 con 0x7fb0080c7fb0 2023-10-06T02:30:08.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.688+0000 7fb00cd9b700 1 --2- 172.21.15.31:0/4243261865 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fafec0674d0 0x7fafec069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:08.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.689+0000 7fb00cd9b700 1 --2- 172.21.15.31:0/4243261865 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fafec0674d0 0x7fafec069980 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7faffc006f00 tx=0x7faffc008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:08.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.692+0000 7fb004ff9700 1 -- 172.21.15.31:0/4243261865 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb008158160 con 0x7fb0080c7fb0 2023-10-06T02:30:08.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.894+0000 7fb00dd9d700 1 -- 172.21.15.31:0/4243261865 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb008158160 con 0x7fb0080c7fb0 2023-10-06T02:30:08.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.895+0000 7fb004ff9700 1 -- 172.21.15.31:0/4243261865 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v128) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb008158160 con 0x7fb0080c7fb0 2023-10-06T02:30:08.895 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 8 2023-10-06T02:30:08.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.897+0000 7fb00dd9d700 1 -- 172.21.15.31:0/4243261865 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fafec0674d0 msgr2=0x7fafec069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:08.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.897+0000 7fb00dd9d700 1 --2- 172.21.15.31:0/4243261865 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fafec0674d0 0x7fafec069980 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7faffc006f00 tx=0x7faffc008040 comp rx=0 tx=0).stop 2023-10-06T02:30:08.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.897+0000 7fb00dd9d700 1 -- 172.21.15.31:0/4243261865 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0080c7fb0 msgr2=0x7fb008156a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:08.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.897+0000 7fb00dd9d700 1 --2- 172.21.15.31:0/4243261865 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0080c7fb0 0x7fb008156a80 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fb0080693f0 tx=0x7faff4017000 comp rx=0 tx=0).stop 2023-10-06T02:30:08.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.897+0000 7fb00dd9d700 1 -- 172.21.15.31:0/4243261865 shutdown_connections 2023-10-06T02:30:08.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.897+0000 7fb00dd9d700 1 --2- 172.21.15.31:0/4243261865 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fafec0674d0 0x7fafec069980 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:08.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.897+0000 7fb00dd9d700 1 --2- 172.21.15.31:0/4243261865 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb00813f780 0x7fb008155bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:08.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.897+0000 7fb00dd9d700 1 --2- 172.21.15.31:0/4243261865 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0080c7be0 0x7fb0080c31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:08.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.897+0000 7fb00dd9d700 1 --2- 172.21.15.31:0/4243261865 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0080c7fb0 0x7fb008156a80 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:08.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.897+0000 7fb00dd9d700 1 -- 172.21.15.31:0/4243261865 >> 172.21.15.31:0/4243261865 conn(0x7fb00806d110 msgr2=0x7fb00813eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:08.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.898+0000 7fb00dd9d700 1 -- 172.21.15.31:0/4243261865 shutdown_connections 2023-10-06T02:30:08.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:08.898+0000 7fb00dd9d700 1 -- 172.21.15.31:0/4243261865 wait complete. 2023-10-06T02:30:08.915 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T02:30:09.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.102+0000 7f9f9539d700 1 Processor -- start 2023-10-06T02:30:09.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.102+0000 7f9f9539d700 1 -- start start 2023-10-06T02:30:09.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.103+0000 7f9f9539d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9f900b2980 0x7f9f900cce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:09.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.103+0000 7f9f9539d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9f9015a8c0 0x7f9f9006d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:09.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.103+0000 7f9f9539d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9f9006ddd0 0x7f9f9015d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:09.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.103+0000 7f9f9539d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f9006e270 con 0x7f9f9006ddd0 2023-10-06T02:30:09.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.103+0000 7f9f9539d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f900cec30 con 0x7f9f900b2980 2023-10-06T02:30:09.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.103+0000 7f9f9539d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f900ce930 con 0x7f9f9015a8c0 2023-10-06T02:30:09.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.104+0000 7f9f94b9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9f9006ddd0 0x7f9f9015d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:09.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.104+0000 7f9f94b9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9f9006ddd0 0x7f9f9015d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39088/0 (socket says 172.21.15.31:39088) 2023-10-06T02:30:09.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.104+0000 7f9f94b9c700 1 -- 172.21.15.31:0/401046324 learned_addr learned my addr 172.21.15.31:0/401046324 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:09.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.104+0000 7f9f8ffff700 1 --2- 172.21.15.31:0/401046324 >> v2:172.21.15.62:3300/0 conn(0x7f9f900b2980 0x7f9f900cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:09.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.104+0000 7f9f8f7fe700 1 --2- 172.21.15.31:0/401046324 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9f9015a8c0 0x7f9f9006d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:09.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.104+0000 7f9f94b9c700 1 -- 172.21.15.31:0/401046324 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9f9015a8c0 msgr2=0x7f9f9006d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:09.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.104+0000 7f9f94b9c700 1 --2- 172.21.15.31:0/401046324 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9f9015a8c0 0x7f9f9006d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:09.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.104+0000 7f9f94b9c700 1 -- 172.21.15.31:0/401046324 >> v2:172.21.15.62:3300/0 conn(0x7f9f900b2980 msgr2=0x7f9f900cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:09.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.104+0000 7f9f94b9c700 1 --2- 172.21.15.31:0/401046324 >> v2:172.21.15.62:3300/0 conn(0x7f9f900b2980 0x7f9f900cce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:09.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.104+0000 7f9f94b9c700 1 -- 172.21.15.31:0/401046324 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f900cd7a0 con 0x7f9f9006ddd0 2023-10-06T02:30:09.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.104+0000 7f9f8f7fe700 1 --2- 172.21.15.31:0/401046324 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9f9015a8c0 0x7f9f9006d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:30:09.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.104+0000 7f9f94b9c700 1 --2- 172.21.15.31:0/401046324 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f9006ddd0 0x7f9f9015d280 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f9f900d15e0 tx=0x7f9f88009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=565f77fad1ff7ec0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:09.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.105+0000 7f9f8effd700 1 -- 172.21.15.31:0/401046324 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f8800e070 con 0x7f9f9006ddd0 2023-10-06T02:30:09.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.105+0000 7f9f8effd700 1 -- 172.21.15.31:0/401046324 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f8800ba20 con 0x7f9f9006ddd0 2023-10-06T02:30:09.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.105+0000 7f9f8effd700 1 -- 172.21.15.31:0/401046324 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f880092e0 con 0x7f9f9006ddd0 2023-10-06T02:30:09.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.106+0000 7f9f9539d700 1 -- 172.21.15.31:0/401046324 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f9006ddd0 msgr2=0x7f9f9015d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:09.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.106+0000 7f9f9539d700 1 --2- 172.21.15.31:0/401046324 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f9006ddd0 0x7f9f9015d280 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f9f900d15e0 tx=0x7f9f88009840 comp rx=0 tx=0).stop 2023-10-06T02:30:09.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.106+0000 7f9f9539d700 1 -- 172.21.15.31:0/401046324 shutdown_connections 2023-10-06T02:30:09.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.106+0000 7f9f9539d700 1 --2- 172.21.15.31:0/401046324 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f9006ddd0 0x7f9f9015d280 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:09.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.106+0000 7f9f9539d700 1 --2- 172.21.15.31:0/401046324 >> v2:172.21.15.62:3300/0 conn(0x7f9f900b2980 0x7f9f900cce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:09.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.106+0000 7f9f9539d700 1 --2- 172.21.15.31:0/401046324 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9f9015a8c0 0x7f9f9006d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:09.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.106+0000 7f9f9539d700 1 -- 172.21.15.31:0/401046324 >> 172.21.15.31:0/401046324 conn(0x7f9f900703a0 msgr2=0x7f9f900707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:09.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.106+0000 7f9f9539d700 1 -- 172.21.15.31:0/401046324 shutdown_connections 2023-10-06T02:30:09.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.106+0000 7f9f9539d700 1 -- 172.21.15.31:0/401046324 wait complete. 2023-10-06T02:30:09.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.108+0000 7f9f9539d700 1 Processor -- start 2023-10-06T02:30:09.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.108+0000 7f9f9539d700 1 -- start start 2023-10-06T02:30:09.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.109+0000 7f9f9539d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9f9006ddd0 0x7f9f900ca160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:09.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.109+0000 7f9f9539d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f900b2980 0x7f9f90169540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:09.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.109+0000 7f9f9539d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f9015a8c0 0x7f9f9014d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:09.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.109+0000 7f9f9539d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f9014e570 con 0x7f9f9015a8c0 2023-10-06T02:30:09.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.109+0000 7f9f9539d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f900ca960 con 0x7f9f900b2980 2023-10-06T02:30:09.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.109+0000 7f9f9539d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f90152da0 con 0x7f9f9006ddd0 2023-10-06T02:30:09.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.109+0000 7f9f94b9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f9015a8c0 0x7f9f9014d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:09.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.109+0000 7f9f94b9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f9015a8c0 0x7f9f9014d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39096/0 (socket says 172.21.15.31:39096) 2023-10-06T02:30:09.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.109+0000 7f9f94b9c700 1 -- 172.21.15.31:0/3250182199 learned_addr learned my addr 172.21.15.31:0/3250182199 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:09.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.109+0000 7f9f8ffff700 1 --2- 172.21.15.31:0/3250182199 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9f9006ddd0 0x7f9f900ca160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:09.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.109+0000 7f9f8f7fe700 1 --2- 172.21.15.31:0/3250182199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f900b2980 0x7f9f90169540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:09.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.110+0000 7f9f94b9c700 1 -- 172.21.15.31:0/3250182199 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9f9006ddd0 msgr2=0x7f9f900ca160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:09.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.110+0000 7f9f94b9c700 1 --2- 172.21.15.31:0/3250182199 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9f9006ddd0 0x7f9f900ca160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:09.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.110+0000 7f9f94b9c700 1 -- 172.21.15.31:0/3250182199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f900b2980 msgr2=0x7f9f90169540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:09.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.110+0000 7f9f94b9c700 1 --2- 172.21.15.31:0/3250182199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f900b2980 0x7f9f90169540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:09.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.110+0000 7f9f94b9c700 1 -- 172.21.15.31:0/3250182199 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f88009ae0 con 0x7f9f9015a8c0 2023-10-06T02:30:09.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.110+0000 7f9f8f7fe700 1 --2- 172.21.15.31:0/3250182199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f900b2980 0x7f9f90169540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:30:09.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.110+0000 7f9f8ffff700 1 --2- 172.21.15.31:0/3250182199 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9f9006ddd0 0x7f9f900ca160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:30:09.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.110+0000 7f9f94b9c700 1 --2- 172.21.15.31:0/3250182199 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f9015a8c0 0x7f9f9014d420 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f9f8800b6c0 tx=0x7f9f88018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:09.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.110+0000 7f9f77fff700 1 -- 172.21.15.31:0/3250182199 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f880032f0 con 0x7f9f9015a8c0 2023-10-06T02:30:09.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.110+0000 7f9f77fff700 1 -- 172.21.15.31:0/3250182199 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f880092e0 con 0x7f9f9015a8c0 2023-10-06T02:30:09.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.110+0000 7f9f77fff700 1 -- 172.21.15.31:0/3250182199 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f88025670 con 0x7f9f9015a8c0 2023-10-06T02:30:09.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.110+0000 7f9f9539d700 1 -- 172.21.15.31:0/3250182199 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f900ca6a0 con 0x7f9f9015a8c0 2023-10-06T02:30:09.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.110+0000 7f9f9539d700 1 -- 172.21.15.31:0/3250182199 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f90152750 con 0x7f9f9015a8c0 2023-10-06T02:30:09.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.111+0000 7f9f9539d700 1 -- 172.21.15.31:0/3250182199 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f9004c3c0 con 0x7f9f9015a8c0 2023-10-06T02:30:09.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.112+0000 7f9f77fff700 1 -- 172.21.15.31:0/3250182199 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f9f88003040 con 0x7f9f9015a8c0 2023-10-06T02:30:09.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.113+0000 7f9f77fff700 1 --2- 172.21.15.31:0/3250182199 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9f7c0674d0 0x7f9f7c069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:09.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.113+0000 7f9f77fff700 1 -- 172.21.15.31:0/3250182199 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5234+0+0 (secure 0 0 0) 0x7f9f88086c80 con 0x7f9f9015a8c0 2023-10-06T02:30:09.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.114+0000 7f9f8ffff700 1 --2- 172.21.15.31:0/3250182199 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9f7c0674d0 0x7f9f7c069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:09.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.114+0000 7f9f8ffff700 1 --2- 172.21.15.31:0/3250182199 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9f7c0674d0 0x7f9f7c069980 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9f80009d80 tx=0x7f9f80008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:09.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.118+0000 7f9f77fff700 1 -- 172.21.15.31:0/3250182199 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9f88053700 con 0x7f9f9015a8c0 2023-10-06T02:30:09.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.323+0000 7f9f9539d700 1 -- 172.21.15.31:0/3250182199 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f9f900ce4a0 con 0x7f9f9015a8c0 2023-10-06T02:30:09.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.323+0000 7f9f77fff700 1 -- 172.21.15.31:0/3250182199 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v128) v1 ==== 84+0+126 (secure 0 0 0) 0x7f9f88056d20 con 0x7f9f9015a8c0 2023-10-06T02:30:09.324 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:30:09.324 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2023-10-06T02:30:09.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:30:09.324 INFO:teuthology.orchestra.run.smithi031.stdout: "poolnum": 1, 2023-10-06T02:30:09.324 INFO:teuthology.orchestra.run.smithi031.stdout: "poolname": ".mgr" 2023-10-06T02:30:09.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:30:09.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:30:09.325 INFO:teuthology.orchestra.run.smithi031.stdout: "poolnum": 2, 2023-10-06T02:30:09.325 INFO:teuthology.orchestra.run.smithi031.stdout: "poolname": "rbd" 2023-10-06T02:30:09.325 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:30:09.325 INFO:teuthology.orchestra.run.smithi031.stdout:] 2023-10-06T02:30:09.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.325+0000 7f9f9539d700 1 -- 172.21.15.31:0/3250182199 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9f7c0674d0 msgr2=0x7f9f7c069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:09.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.325+0000 7f9f9539d700 1 --2- 172.21.15.31:0/3250182199 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9f7c0674d0 0x7f9f7c069980 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9f80009d80 tx=0x7f9f80008040 comp rx=0 tx=0).stop 2023-10-06T02:30:09.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.325+0000 7f9f9539d700 1 -- 172.21.15.31:0/3250182199 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f9015a8c0 msgr2=0x7f9f9014d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:09.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.325+0000 7f9f9539d700 1 --2- 172.21.15.31:0/3250182199 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f9015a8c0 0x7f9f9014d420 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f9f8800b6c0 tx=0x7f9f88018000 comp rx=0 tx=0).stop 2023-10-06T02:30:09.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.326+0000 7f9f9539d700 1 -- 172.21.15.31:0/3250182199 shutdown_connections 2023-10-06T02:30:09.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.326+0000 7f9f9539d700 1 --2- 172.21.15.31:0/3250182199 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f9015a8c0 0x7f9f9014d420 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:09.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.326+0000 7f9f9539d700 1 --2- 172.21.15.31:0/3250182199 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9f7c0674d0 0x7f9f7c069980 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:09.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.326+0000 7f9f9539d700 1 --2- 172.21.15.31:0/3250182199 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9f9006ddd0 0x7f9f900ca160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:09.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.326+0000 7f9f9539d700 1 --2- 172.21.15.31:0/3250182199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f900b2980 0x7f9f90169540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:09.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.326+0000 7f9f9539d700 1 -- 172.21.15.31:0/3250182199 >> 172.21.15.31:0/3250182199 conn(0x7f9f900703a0 msgr2=0x7f9f9014fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:09.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.326+0000 7f9f9539d700 1 -- 172.21.15.31:0/3250182199 shutdown_connections 2023-10-06T02:30:09.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.326+0000 7f9f9539d700 1 -- 172.21.15.31:0/3250182199 wait complete. 2023-10-06T02:30:09.344 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T02:30:09.344 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T02:30:09.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.534+0000 7f42a51ff700 1 Processor -- start 2023-10-06T02:30:09.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.535+0000 7f42a51ff700 1 -- start start 2023-10-06T02:30:09.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.536+0000 7f42a51ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f42a01575a0 0x7f42a0069dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:09.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.536+0000 7f42a51ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f42a006eaa0 0x7f42a00ae1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:09.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.536+0000 7f42a51ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42a00af090 con 0x7f42a00ae6f0 2023-10-06T02:30:09.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.536+0000 7f42a51ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42a006a950 con 0x7f42a006eaa0 2023-10-06T02:30:09.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.536+0000 7f42a51ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42a00aeb70 con 0x7f42a01575a0 2023-10-06T02:30:09.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.536+0000 7f429e59c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f42a006eaa0 0x7f42a00ae1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:09.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.536+0000 7f429f59e700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f42a00ae6f0 0x7f42a015b240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:47802/0 (socket says 172.21.15.31:47802) 2023-10-06T02:30:09.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.536+0000 7f429f59e700 1 -- 172.21.15.31:0/950770992 learned_addr learned my addr 172.21.15.31:0/950770992 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:30:09.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.536+0000 7f429ed9d700 1 --2- 172.21.15.31:0/950770992 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f42a01575a0 0x7f42a0069dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:09.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.537+0000 7f429e59c700 1 -- 172.21.15.31:0/950770992 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f42a01575a0 msgr2=0x7f42a0069dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:09.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.537+0000 7f429e59c700 1 --2- 172.21.15.31:0/950770992 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f42a01575a0 0x7f42a0069dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:09.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.537+0000 7f429e59c700 1 -- 172.21.15.31:0/950770992 >> v1:172.21.15.31:6789/0 conn(0x7f42a00ae6f0 legacy=0x7f42a015b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:09.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.537+0000 7f429e59c700 1 -- 172.21.15.31:0/950770992 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42a00c7100 con 0x7f42a006eaa0 2023-10-06T02:30:09.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.537+0000 7f429dd9b700 1 -- 172.21.15.31:0/950770992 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2483118465 0 0) 0x7f42a00af090 con 0x7f42a00ae6f0 2023-10-06T02:30:09.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.537+0000 7f429e59c700 1 --2- 172.21.15.31:0/950770992 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42a006eaa0 0x7f42a00ae1b0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f4294009910 tx=0x7f42940091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2c8426527dfa4c5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:09.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.537+0000 7f429dd9b700 1 -- 172.21.15.31:0/950770992 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429400b820 con 0x7f42a006eaa0 2023-10-06T02:30:09.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.537+0000 7f429dd9b700 1 -- 172.21.15.31:0/950770992 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f429400b980 con 0x7f42a006eaa0 2023-10-06T02:30:09.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.537+0000 7f429dd9b700 1 -- 172.21.15.31:0/950770992 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429400bd20 con 0x7f42a006eaa0 2023-10-06T02:30:09.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.538+0000 7f42a51ff700 1 -- 172.21.15.31:0/950770992 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42a006eaa0 msgr2=0x7f42a00ae1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:09.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.538+0000 7f42a51ff700 1 --2- 172.21.15.31:0/950770992 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42a006eaa0 0x7f42a00ae1b0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f4294009910 tx=0x7f42940091a0 comp rx=0 tx=0).stop 2023-10-06T02:30:09.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.538+0000 7f42a51ff700 1 -- 172.21.15.31:0/950770992 shutdown_connections 2023-10-06T02:30:09.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.538+0000 7f42a51ff700 1 --2- 172.21.15.31:0/950770992 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f42a01575a0 0x7f42a0069dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:09.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.538+0000 7f42a51ff700 1 --2- 172.21.15.31:0/950770992 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42a006eaa0 0x7f42a00ae1b0 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:09.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.538+0000 7f42a51ff700 1 -- 172.21.15.31:0/950770992 >> 172.21.15.31:0/950770992 conn(0x7f42a006d170 msgr2=0x7f42a00cc2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:09.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.538+0000 7f42a51ff700 1 -- 172.21.15.31:0/950770992 shutdown_connections 2023-10-06T02:30:09.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.538+0000 7f42a51ff700 1 -- 172.21.15.31:0/950770992 wait complete. 2023-10-06T02:30:09.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.539+0000 7f42a51ff700 1 Processor -- start 2023-10-06T02:30:09.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.540+0000 7f42a51ff700 1 -- start start 2023-10-06T02:30:09.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.540+0000 7f42a51ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42a006eaa0 0x7f42a0151c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:09.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.540+0000 7f42a51ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f42a00ae6f0 0x7f42a00c5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:09.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.540+0000 7f42a51ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f42a01575a0 0x7f42a00c62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:09.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.540+0000 7f42a51ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a0168b90 con 0x7f42a006eaa0 2023-10-06T02:30:09.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.540+0000 7f42a51ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a00a6f20 con 0x7f42a00ae6f0 2023-10-06T02:30:09.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.540+0000 7f42a51ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42a00a7200 con 0x7f42a01575a0 2023-10-06T02:30:09.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.541+0000 7f429ed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42a006eaa0 0x7f42a0151c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:09.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.541+0000 7f429ed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42a006eaa0 0x7f42a0151c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39106/0 (socket says 172.21.15.31:39106) 2023-10-06T02:30:09.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.541+0000 7f429ed9d700 1 -- 172.21.15.31:0/3171909366 learned_addr learned my addr 172.21.15.31:0/3171909366 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:09.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.541+0000 7f429e59c700 1 --2- 172.21.15.31:0/3171909366 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f42a00ae6f0 0x7f42a00c5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:09.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.541+0000 7f429f59e700 1 --2- 172.21.15.31:0/3171909366 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f42a01575a0 0x7f42a00c62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:09.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.541+0000 7f429ed9d700 1 -- 172.21.15.31:0/3171909366 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f42a01575a0 msgr2=0x7f42a00c62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:09.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.541+0000 7f429ed9d700 1 --2- 172.21.15.31:0/3171909366 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f42a01575a0 0x7f42a00c62b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:09.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.541+0000 7f429ed9d700 1 -- 172.21.15.31:0/3171909366 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f42a00ae6f0 msgr2=0x7f42a00c5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:09.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.541+0000 7f429ed9d700 1 --2- 172.21.15.31:0/3171909366 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f42a00ae6f0 0x7f42a00c5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:09.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.541+0000 7f429ed9d700 1 -- 172.21.15.31:0/3171909366 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4294009420 con 0x7f42a006eaa0 2023-10-06T02:30:09.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.541+0000 7f429f59e700 1 --2- 172.21.15.31:0/3171909366 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f42a01575a0 0x7f42a00c62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:30:09.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.541+0000 7f429ed9d700 1 --2- 172.21.15.31:0/3171909366 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42a006eaa0 0x7f42a0151c90 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f429000d890 tx=0x7f4290008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:09.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.542+0000 7f4286ffd700 1 -- 172.21.15.31:0/3171909366 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42900125f0 con 0x7f42a006eaa0 2023-10-06T02:30:09.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.542+0000 7f4286ffd700 1 -- 172.21.15.31:0/3171909366 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4290012750 con 0x7f42a006eaa0 2023-10-06T02:30:09.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.542+0000 7f4286ffd700 1 -- 172.21.15.31:0/3171909366 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4290012ad0 con 0x7f42a006eaa0 2023-10-06T02:30:09.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.542+0000 7f42a51ff700 1 -- 172.21.15.31:0/3171909366 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42a00a74e0 con 0x7f42a006eaa0 2023-10-06T02:30:09.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.542+0000 7f42a51ff700 1 -- 172.21.15.31:0/3171909366 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42a00a7aa0 con 0x7f42a006eaa0 2023-10-06T02:30:09.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.543+0000 7f42a51ff700 1 -- 172.21.15.31:0/3171909366 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42a00a7c60 con 0x7f42a006eaa0 2023-10-06T02:30:09.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.544+0000 7f4286ffd700 1 -- 172.21.15.31:0/3171909366 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f429001e460 con 0x7f42a006eaa0 2023-10-06T02:30:09.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.545+0000 7f4286ffd700 1 --2- 172.21.15.31:0/3171909366 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f428c0674d0 0x7f428c069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:09.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.545+0000 7f4286ffd700 1 -- 172.21.15.31:0/3171909366 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5234+0+0 (secure 0 0 0) 0x7f4290085420 con 0x7f42a006eaa0 2023-10-06T02:30:09.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.545+0000 7f429e59c700 1 --2- 172.21.15.31:0/3171909366 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f428c0674d0 0x7f428c069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:09.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.546+0000 7f429e59c700 1 --2- 172.21.15.31:0/3171909366 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f428c0674d0 0x7f428c069980 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4294009860 tx=0x7f4294017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:09.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.549+0000 7f4286ffd700 1 -- 172.21.15.31:0/3171909366 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4290051fb0 con 0x7f42a006eaa0 2023-10-06T02:30:09.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:09.751+0000 7f42a51ff700 1 -- 172.21.15.31:0/3171909366 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f42a00a7e10 con 0x7f42a006eaa0 2023-10-06T02:30:10.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:10.119+0000 7f4286ffd700 1 -- 172.21.15.31:0/3171909366 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v129) v1 ==== 138+0+0 (secure 0 0 0) 0x7f42900555d0 con 0x7f42a006eaa0 2023-10-06T02:30:10.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:10.122+0000 7f42a51ff700 1 -- 172.21.15.31:0/3171909366 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f428c0674d0 msgr2=0x7f428c069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:10.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:10.122+0000 7f42a51ff700 1 --2- 172.21.15.31:0/3171909366 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f428c0674d0 0x7f428c069980 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4294009860 tx=0x7f4294017000 comp rx=0 tx=0).stop 2023-10-06T02:30:10.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:10.122+0000 7f42a51ff700 1 -- 172.21.15.31:0/3171909366 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42a006eaa0 msgr2=0x7f42a0151c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:10.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:10.122+0000 7f42a51ff700 1 --2- 172.21.15.31:0/3171909366 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42a006eaa0 0x7f42a0151c90 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f429000d890 tx=0x7f4290008040 comp rx=0 tx=0).stop 2023-10-06T02:30:10.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:10.123+0000 7f42a51ff700 1 -- 172.21.15.31:0/3171909366 shutdown_connections 2023-10-06T02:30:10.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:10.123+0000 7f42a51ff700 1 --2- 172.21.15.31:0/3171909366 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f428c0674d0 0x7f428c069980 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:10.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:10.123+0000 7f42a51ff700 1 --2- 172.21.15.31:0/3171909366 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f42a01575a0 0x7f42a00c62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:10.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:10.123+0000 7f42a51ff700 1 --2- 172.21.15.31:0/3171909366 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42a006eaa0 0x7f42a0151c90 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:10.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:10.123+0000 7f42a51ff700 1 --2- 172.21.15.31:0/3171909366 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f42a00ae6f0 0x7f42a00c5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:10.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:10.123+0000 7f42a51ff700 1 -- 172.21.15.31:0/3171909366 >> 172.21.15.31:0/3171909366 conn(0x7f42a006d170 msgr2=0x7f42a006e0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:10.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:10.123+0000 7f42a51ff700 1 -- 172.21.15.31:0/3171909366 shutdown_connections 2023-10-06T02:30:10.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:10.123+0000 7f42a51ff700 1 -- 172.21.15.31:0/3171909366 wait complete. 2023-10-06T02:30:10.125 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' created 2023-10-06T02:30:10.143 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T02:30:10.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:10.337+0000 7f201f59d700 1 Processor -- start 2023-10-06T02:30:10.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:10.337+0000 7f201f59d700 1 -- start start 2023-10-06T02:30:10.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:10.338+0000 7f201f59d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2018159920 0x7f20180ad090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:10.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:10.338+0000 7f201f59d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20180acc00 con 0x7f2018159550 2023-10-06T02:30:10.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:10.338+0000 7f201f59d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20180abff0 con 0x7f201815dfd0 2023-10-06T02:30:10.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:10.338+0000 7f201f59d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20180ac2d0 con 0x7f2018159920 2023-10-06T02:30:10.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:10.338+0000 7f201e59b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f2018159550 0x7f20180ceac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:47812/0 (socket says 172.21.15.31:47812) 2023-10-06T02:30:10.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:10.339+0000 7f201e59b700 1 -- 172.21.15.31:0/2091487144 learned_addr learned my addr 172.21.15.31:0/2091487144 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:30:10.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:10.339+0000 7f201ed9c700 1 --2- 172.21.15.31:0/2091487144 >> v2:172.21.15.31:3300/0 conn(0x7f2018159920 0x7f20180ad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:10.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:10.339+0000 7f201d599700 1 -- 172.21.15.31:0/2091487144 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3717661002 0 0) 0x7f20180acc00 con 0x7f2018159550 2023-10-06T02:30:10.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:10.339+0000 7f201d599700 1 -- 172.21.15.31:0/2091487144 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2004003570 con 0x7f2018159550 2023-10-06T02:30:10.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:10.339+0000 7f201d599700 1 -- 172.21.15.31:0/2091487144 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1981357781 0 0) 0x7f20180abff0 con 0x7f201815dfd0 2023-10-06T02:30:10.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:10.339+0000 7f201ed9c700 1 -- 172.21.15.31:0/2091487144 >> v1:172.21.15.62:6789/0 conn(0x7f201815dfd0 legacy=0x7f20180aedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:10.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:10.339+0000 7f201ed9c700 1 -- 172.21.15.31:0/2091487144 >> v1:172.21.15.31:6789/0 conn(0x7f2018159550 legacy=0x7f20180ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:10.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:10.339+0000 7f201ed9c700 1 -- 172.21.15.31:0/2091487144 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20180ac5b0 con 0x7f2018159920 2023-10-06T02:30:10.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:10.339+0000 7f201ed9c700 1 --2- 172.21.15.31:0/2091487144 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2018159920 0x7f20180ad090 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f2010009e60 tx=0x7f2010009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1881cd2d81bc5d2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:10.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:10.340+0000 7f201d599700 1 -- 172.21.15.31:0/2091487144 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201000e070 con 0x7f2018159920 2023-10-06T02:30:10.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:10.340+0000 7f201d599700 1 -- 172.21.15.31:0/2091487144 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f201000bb60 con 0x7f2018159920 2023-10-06T02:30:10.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:10.340+0000 7f201d599700 1 -- 172.21.15.31:0/2091487144 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2010012430 con 0x7f2018159920 2023-10-06T02:30:10.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:10.340+0000 7f201f59d700 1 -- 172.21.15.31:0/2091487144 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2018159920 msgr2=0x7f20180ad090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:10.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:10.340+0000 7f201f59d700 1 --2- 172.21.15.31:0/2091487144 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2018159920 0x7f20180ad090 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f2010009e60 tx=0x7f2010009780 comp rx=0 tx=0).stop 2023-10-06T02:30:10.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:10.340+0000 7f201f59d700 1 -- 172.21.15.31:0/2091487144 shutdown_connections 2023-10-06T02:30:10.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:10.340+0000 7f201f59d700 1 --2- 172.21.15.31:0/2091487144 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2018159920 0x7f20180ad090 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:10.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:10.340+0000 7f201f59d700 1 -- 172.21.15.31:0/2091487144 >> 172.21.15.31:0/2091487144 conn(0x7f201806ce80 msgr2=0x7f20180ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:10.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:10.340+0000 7f201f59d700 1 -- 172.21.15.31:0/2091487144 shutdown_connections 2023-10-06T02:30:10.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:10.340+0000 7f201f59d700 1 -- 172.21.15.31:0/2091487144 wait complete. 2023-10-06T02:30:10.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:10.342+0000 7f201f59d700 1 Processor -- start 2023-10-06T02:30:10.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:10.342+0000 7f201f59d700 1 -- start start 2023-10-06T02:30:10.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:10.343+0000 7f201f59d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2018159550 0x7f201814c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:10.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:10.343+0000 7f201f59d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2018159920 0x7f20180b01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:10.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:10.343+0000 7f201f59d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f201815dfd0 0x7f20180b06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:10.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:10.343+0000 7f201f59d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20180c8200 con 0x7f2018159550 2023-10-06T02:30:10.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:10.343+0000 7f201f59d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2018150140 con 0x7f2018159920 2023-10-06T02:30:10.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:10.343+0000 7f201f59d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2018150420 con 0x7f201815dfd0 2023-10-06T02:30:10.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:10.343+0000 7f201e59b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2018159550 0x7f201814c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:10.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:10.343+0000 7f201e59b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2018159550 0x7f201814c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39120/0 (socket says 172.21.15.31:39120) 2023-10-06T02:30:10.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:10.343+0000 7f201e59b700 1 -- 172.21.15.31:0/42203639 learned_addr learned my addr 172.21.15.31:0/42203639 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:10.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:10.343+0000 7f201ed9c700 1 --2- 172.21.15.31:0/42203639 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f201815dfd0 0x7f20180b06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:10.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:10.343+0000 7f201dd9a700 1 --2- 172.21.15.31:0/42203639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2018159920 0x7f20180b01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:10.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:10.344+0000 7f201e59b700 1 -- 172.21.15.31:0/42203639 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f201815dfd0 msgr2=0x7f20180b06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:10.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:10.344+0000 7f201e59b700 1 --2- 172.21.15.31:0/42203639 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f201815dfd0 0x7f20180b06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:10.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:10.344+0000 7f201e59b700 1 -- 172.21.15.31:0/42203639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2018159920 msgr2=0x7f20180b01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:10.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:10.344+0000 7f201e59b700 1 --2- 172.21.15.31:0/42203639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2018159920 0x7f20180b01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:10.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:10.344+0000 7f201e59b700 1 -- 172.21.15.31:0/42203639 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2010009a20 con 0x7f2018159550 2023-10-06T02:30:10.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:10.344+0000 7f201e59b700 1 --2- 172.21.15.31:0/42203639 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2018159550 0x7f201814c610 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f20140011e0 tx=0x7f2014007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:10.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:10.344+0000 7f200e7fc700 1 -- 172.21.15.31:0/42203639 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2014005b30 con 0x7f2018159550 2023-10-06T02:30:10.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:10.344+0000 7f200e7fc700 1 -- 172.21.15.31:0/42203639 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2014005c90 con 0x7f2018159550 2023-10-06T02:30:10.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:10.344+0000 7f200e7fc700 1 -- 172.21.15.31:0/42203639 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20140072e0 con 0x7f2018159550 2023-10-06T02:30:10.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:10.344+0000 7f201f59d700 1 -- 172.21.15.31:0/42203639 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2018150700 con 0x7f2018159550 2023-10-06T02:30:10.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:10.344+0000 7f201f59d700 1 -- 172.21.15.31:0/42203639 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f201814d150 con 0x7f2018159550 2023-10-06T02:30:10.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:10.345+0000 7f201f59d700 1 -- 172.21.15.31:0/42203639 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f201804c3c0 con 0x7f2018159550 2023-10-06T02:30:10.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:10.346+0000 7f200e7fc700 1 -- 172.21.15.31:0/42203639 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f2014007650 con 0x7f2018159550 2023-10-06T02:30:10.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:10.347+0000 7f200e7fc700 1 --2- 172.21.15.31:0/42203639 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f1ffc0674d0 0x7f1ffc069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:10.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:10.347+0000 7f200e7fc700 1 -- 172.21.15.31:0/42203639 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5604+0+0 (secure 0 0 0) 0x7f2014084400 con 0x7f2018159550 2023-10-06T02:30:10.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:10.348+0000 7f201dd9a700 1 --2- 172.21.15.31:0/42203639 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f1ffc0674d0 0x7f1ffc069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:10.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:10.348+0000 7f201dd9a700 1 --2- 172.21.15.31:0/42203639 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f1ffc0674d0 0x7f1ffc069980 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f201814d620 tx=0x7f2008009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:10.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:10.352+0000 7f200e7fc700 1 -- 172.21.15.31:0/42203639 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f201404b5a0 con 0x7f2018159550 2023-10-06T02:30:10.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:10.561+0000 7f201f59d700 1 -- 172.21.15.31:0/42203639 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f201806e7c0 con 0x7f2018159550 2023-10-06T02:30:11.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:11.133+0000 7f200e7fc700 1 -- 172.21.15.31:0/42203639 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v130) v1 ==== 170+0+0 (secure 0 0 0) 0x7f2014054960 con 0x7f2018159550 2023-10-06T02:30:11.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:11.139+0000 7f201f59d700 1 -- 172.21.15.31:0/42203639 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f1ffc0674d0 msgr2=0x7f1ffc069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:11.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:11.139+0000 7f201f59d700 1 --2- 172.21.15.31:0/42203639 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f1ffc0674d0 0x7f1ffc069980 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f201814d620 tx=0x7f2008009040 comp rx=0 tx=0).stop 2023-10-06T02:30:11.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:11.140+0000 7f201f59d700 1 -- 172.21.15.31:0/42203639 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2018159550 msgr2=0x7f201814c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:11.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:11.140+0000 7f201f59d700 1 --2- 172.21.15.31:0/42203639 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2018159550 0x7f201814c610 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f20140011e0 tx=0x7f2014007df0 comp rx=0 tx=0).stop 2023-10-06T02:30:11.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:11.141+0000 7f201f59d700 1 -- 172.21.15.31:0/42203639 shutdown_connections 2023-10-06T02:30:11.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:11.141+0000 7f201f59d700 1 --2- 172.21.15.31:0/42203639 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f1ffc0674d0 0x7f1ffc069980 secure :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f201814d620 tx=0x7f2008009040 comp rx=0 tx=0).stop 2023-10-06T02:30:11.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:11.141+0000 7f201f59d700 1 --2- 172.21.15.31:0/42203639 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f201815dfd0 0x7f20180b06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:11.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:11.141+0000 7f201f59d700 1 --2- 172.21.15.31:0/42203639 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2018159550 0x7f201814c610 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:11.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:11.141+0000 7f201f59d700 1 --2- 172.21.15.31:0/42203639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2018159920 0x7f20180b01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:11.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:11.141+0000 7f201f59d700 1 -- 172.21.15.31:0/42203639 >> 172.21.15.31:0/42203639 conn(0x7f201806ce80 msgr2=0x7f2018154230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:11.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:11.142+0000 7f201f59d700 1 -- 172.21.15.31:0/42203639 shutdown_connections 2023-10-06T02:30:11.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:11.143+0000 7f201f59d700 1 -- 172.21.15.31:0/42203639 wait complete. 2023-10-06T02:30:11.151 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' created 2023-10-06T02:30:11.170 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T02:30:11.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:11.360+0000 7fdc3759d700 1 Processor -- start 2023-10-06T02:30:11.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:11.360+0000 7fdc3759d700 1 -- start start 2023-10-06T02:30:11.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:11.361+0000 7fdc3759d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdc300c7be0 0x7fdc3006abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:11.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:11.361+0000 7fdc3759d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fdc300c7fb0 0x7fdc3015a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:11.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:11.361+0000 7fdc3759d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fdc3013f780 0x7fdc3013fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:11.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:11.361+0000 7fdc3759d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc3006bf10 con 0x7fdc300c7fb0 2023-10-06T02:30:11.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:11.361+0000 7fdc3759d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc30140100 con 0x7fdc3013f780 2023-10-06T02:30:11.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:11.361+0000 7fdc3759d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc3015aed0 con 0x7fdc300c7be0 2023-10-06T02:30:11.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:11.361+0000 7fdc35d9a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fdc300c7fb0 0x7fdc3015a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:11.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:11.362+0000 7fdc35d9a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fdc300c7fb0 0x7fdc3015a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39126/0 (socket says 172.21.15.31:39126) 2023-10-06T02:30:11.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:11.362+0000 7fdc35d9a700 1 -- 172.21.15.31:0/1133736363 learned_addr learned my addr 172.21.15.31:0/1133736363 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:11.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:11.362+0000 7fdc3659b700 1 --2- 172.21.15.31:0/1133736363 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdc300c7be0 0x7fdc3006abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:11.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:11.362+0000 7fdc36d9c700 1 --2- 172.21.15.31:0/1133736363 >> v2:172.21.15.62:3300/0 conn(0x7fdc3013f780 0x7fdc3013fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:11.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:11.362+0000 7fdc35d9a700 1 -- 172.21.15.31:0/1133736363 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdc300c7be0 msgr2=0x7fdc3006abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:11.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:11.362+0000 7fdc35d9a700 1 --2- 172.21.15.31:0/1133736363 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdc300c7be0 0x7fdc3006abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:11.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:11.362+0000 7fdc35d9a700 1 -- 172.21.15.31:0/1133736363 >> v2:172.21.15.62:3300/0 conn(0x7fdc3013f780 msgr2=0x7fdc3013fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:11.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:11.362+0000 7fdc35d9a700 1 --2- 172.21.15.31:0/1133736363 >> v2:172.21.15.62:3300/0 conn(0x7fdc3013f780 0x7fdc3013fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:11.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:11.362+0000 7fdc35d9a700 1 -- 172.21.15.31:0/1133736363 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc3006b290 con 0x7fdc300c7fb0 2023-10-06T02:30:11.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:11.362+0000 7fdc35d9a700 1 --2- 172.21.15.31:0/1133736363 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc300c7fb0 0x7fdc3015a990 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fdc300c6700 tx=0x7fdc20009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e1944336135dafa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:11.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:11.363+0000 7fdc35599700 1 -- 172.21.15.31:0/1133736363 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc2000b640 con 0x7fdc300c7fb0 2023-10-06T02:30:11.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:11.363+0000 7fdc35599700 1 -- 172.21.15.31:0/1133736363 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc2000b7a0 con 0x7fdc300c7fb0 2023-10-06T02:30:11.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:11.363+0000 7fdc35599700 1 -- 172.21.15.31:0/1133736363 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc2000bb40 con 0x7fdc300c7fb0 2023-10-06T02:30:11.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:11.363+0000 7fdc3759d700 1 -- 172.21.15.31:0/1133736363 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc300c7fb0 msgr2=0x7fdc3015a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:11.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:11.363+0000 7fdc3759d700 1 --2- 172.21.15.31:0/1133736363 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc300c7fb0 0x7fdc3015a990 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fdc300c6700 tx=0x7fdc20009250 comp rx=0 tx=0).stop 2023-10-06T02:30:11.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:11.363+0000 7fdc3759d700 1 -- 172.21.15.31:0/1133736363 shutdown_connections 2023-10-06T02:30:11.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:11.363+0000 7fdc3759d700 1 --2- 172.21.15.31:0/1133736363 >> v2:172.21.15.62:3300/0 conn(0x7fdc3013f780 0x7fdc3013fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:11.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:11.363+0000 7fdc3759d700 1 --2- 172.21.15.31:0/1133736363 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdc300c7be0 0x7fdc3006abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:11.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:11.363+0000 7fdc3759d700 1 --2- 172.21.15.31:0/1133736363 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc300c7fb0 0x7fdc3015a990 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:11.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:11.363+0000 7fdc3759d700 1 -- 172.21.15.31:0/1133736363 >> 172.21.15.31:0/1133736363 conn(0x7fdc3006d110 msgr2=0x7fdc300cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:11.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:11.363+0000 7fdc3759d700 1 -- 172.21.15.31:0/1133736363 shutdown_connections 2023-10-06T02:30:11.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:11.363+0000 7fdc3759d700 1 -- 172.21.15.31:0/1133736363 wait complete. 2023-10-06T02:30:11.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:11.365+0000 7fdc3759d700 1 Processor -- start 2023-10-06T02:30:11.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:11.365+0000 7fdc3759d700 1 -- start start 2023-10-06T02:30:11.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:11.366+0000 7fdc3759d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc300c7be0 0x7fdc300b0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:11.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:11.366+0000 7fdc3759d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdc300c7fb0 0x7fdc30154900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:11.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:11.366+0000 7fdc3759d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc3013f780 0x7fdc30155d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:11.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:11.366+0000 7fdc3759d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc30154e40 con 0x7fdc3013f780 2023-10-06T02:30:11.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:11.366+0000 7fdc3759d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc300b06a0 con 0x7fdc300c7be0 2023-10-06T02:30:11.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:11.366+0000 7fdc3759d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc300b0980 con 0x7fdc300c7fb0 2023-10-06T02:30:11.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:11.366+0000 7fdc36d9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc3013f780 0x7fdc30155d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:11.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:11.366+0000 7fdc36d9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc3013f780 0x7fdc30155d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39142/0 (socket says 172.21.15.31:39142) 2023-10-06T02:30:11.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:11.366+0000 7fdc36d9c700 1 -- 172.21.15.31:0/676904499 learned_addr learned my addr 172.21.15.31:0/676904499 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:11.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:11.366+0000 7fdc3659b700 1 --2- 172.21.15.31:0/676904499 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc300c7be0 0x7fdc300b0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:11.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:11.366+0000 7fdc35d9a700 1 --2- 172.21.15.31:0/676904499 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdc300c7fb0 0x7fdc30154900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:11.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:11.366+0000 7fdc36d9c700 1 -- 172.21.15.31:0/676904499 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdc300c7fb0 msgr2=0x7fdc30154900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:11.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:11.366+0000 7fdc36d9c700 1 --2- 172.21.15.31:0/676904499 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdc300c7fb0 0x7fdc30154900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:11.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:11.366+0000 7fdc36d9c700 1 -- 172.21.15.31:0/676904499 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc300c7be0 msgr2=0x7fdc300b0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:11.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:11.366+0000 7fdc36d9c700 1 --2- 172.21.15.31:0/676904499 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc300c7be0 0x7fdc300b0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:11.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:11.366+0000 7fdc36d9c700 1 -- 172.21.15.31:0/676904499 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc200094f0 con 0x7fdc3013f780 2023-10-06T02:30:11.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:11.367+0000 7fdc36d9c700 1 --2- 172.21.15.31:0/676904499 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc3013f780 0x7fdc30155d80 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fdc28009df0 tx=0x7fdc2800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:11.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:11.367+0000 7fdc267fc700 1 -- 172.21.15.31:0/676904499 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc2800caa0 con 0x7fdc3013f780 2023-10-06T02:30:11.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:11.367+0000 7fdc267fc700 1 -- 172.21.15.31:0/676904499 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc2800f040 con 0x7fdc3013f780 2023-10-06T02:30:11.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:11.367+0000 7fdc267fc700 1 -- 172.21.15.31:0/676904499 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc2800ce70 con 0x7fdc3013f780 2023-10-06T02:30:11.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:11.367+0000 7fdc3759d700 1 -- 172.21.15.31:0/676904499 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc300b0c60 con 0x7fdc3013f780 2023-10-06T02:30:11.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:11.367+0000 7fdc3759d700 1 -- 172.21.15.31:0/676904499 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc300b1220 con 0x7fdc3013f780 2023-10-06T02:30:11.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:11.368+0000 7fdc3759d700 1 -- 172.21.15.31:0/676904499 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc300b1440 con 0x7fdc3013f780 2023-10-06T02:30:11.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:11.369+0000 7fdc267fc700 1 -- 172.21.15.31:0/676904499 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7fdc28014690 con 0x7fdc3013f780 2023-10-06T02:30:11.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:11.371+0000 7fdc267fc700 1 --2- 172.21.15.31:0/676904499 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fdc140674d0 0x7fdc14069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:11.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:11.371+0000 7fdc267fc700 1 -- 172.21.15.31:0/676904499 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5986+0+0 (secure 0 0 0) 0x7fdc280861b0 con 0x7fdc3013f780 2023-10-06T02:30:11.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:11.371+0000 7fdc3659b700 1 --2- 172.21.15.31:0/676904499 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fdc140674d0 0x7fdc14069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:11.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:11.372+0000 7fdc3659b700 1 --2- 172.21.15.31:0/676904499 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fdc140674d0 0x7fdc14069980 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c009db0 tx=0x7fdc2c0091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:11.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:11.375+0000 7fdc267fc700 1 -- 172.21.15.31:0/676904499 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdc28056450 con 0x7fdc3013f780 2023-10-06T02:30:11.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:11.623+0000 7fdc3759d700 1 -- 172.21.15.31:0/676904499 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fdc3004c3c0 con 0x7fdc3013f780 2023-10-06T02:30:12.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.142+0000 7fdc267fc700 1 -- 172.21.15.31:0/676904499 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 21 and data pool 22 v86) v1 ==== 174+0+0 (secure 0 0 0) 0x7fdc280180c0 con 0x7fdc3013f780 2023-10-06T02:30:12.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.155+0000 7fdc3759d700 1 -- 172.21.15.31:0/676904499 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fdc140674d0 msgr2=0x7fdc14069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:12.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.155+0000 7fdc3759d700 1 --2- 172.21.15.31:0/676904499 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fdc140674d0 0x7fdc14069980 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c009db0 tx=0x7fdc2c0091a0 comp rx=0 tx=0).stop 2023-10-06T02:30:12.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.155+0000 7fdc3759d700 1 -- 172.21.15.31:0/676904499 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc3013f780 msgr2=0x7fdc30155d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:12.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.155+0000 7fdc3759d700 1 --2- 172.21.15.31:0/676904499 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc3013f780 0x7fdc30155d80 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fdc28009df0 tx=0x7fdc2800b040 comp rx=0 tx=0).stop 2023-10-06T02:30:12.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.156+0000 7fdc3759d700 1 -- 172.21.15.31:0/676904499 shutdown_connections 2023-10-06T02:30:12.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.157+0000 7fdc3759d700 1 --2- 172.21.15.31:0/676904499 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc3013f780 0x7fdc30155d80 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:12.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.157+0000 7fdc3759d700 1 --2- 172.21.15.31:0/676904499 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc300c7be0 0x7fdc300b0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:12.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.157+0000 7fdc3759d700 1 --2- 172.21.15.31:0/676904499 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fdc140674d0 0x7fdc14069980 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:12.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.157+0000 7fdc3759d700 1 --2- 172.21.15.31:0/676904499 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdc300c7fb0 0x7fdc30154900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:12.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.157+0000 7fdc3759d700 1 -- 172.21.15.31:0/676904499 >> 172.21.15.31:0/676904499 conn(0x7fdc3006d110 msgr2=0x7fdc30139130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:12.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.160+0000 7fdc3759d700 1 -- 172.21.15.31:0/676904499 shutdown_connections 2023-10-06T02:30:12.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.160+0000 7fdc3759d700 1 -- 172.21.15.31:0/676904499 wait complete. 2023-10-06T02:30:12.163 INFO:teuthology.orchestra.run.smithi031.stderr:new fs with metadata pool 21 and data pool 22 2023-10-06T02:30:12.184 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:30:12.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.372+0000 7f3b1899d700 1 Processor -- start 2023-10-06T02:30:12.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.372+0000 7f3b1899d700 1 -- start start 2023-10-06T02:30:12.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.374+0000 7f3b1899d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3b14159550 0x7f3b140ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:12.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.374+0000 7f3b1899d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b140acc00 con 0x7f3b1415dfd0 2023-10-06T02:30:12.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.374+0000 7f3b1899d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b140abff0 con 0x7f3b14159920 2023-10-06T02:30:12.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.374+0000 7f3b1899d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b140ac2d0 con 0x7f3b14159550 2023-10-06T02:30:12.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.374+0000 7f3b12ffd700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f3b1415dfd0 0x7f3b140aedb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:47814/0 (socket says 172.21.15.31:47814) 2023-10-06T02:30:12.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.374+0000 7f3b12ffd700 1 -- 172.21.15.31:0/586569646 learned_addr learned my addr 172.21.15.31:0/586569646 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:30:12.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.374+0000 7f3b137fe700 1 --2- 172.21.15.31:0/586569646 >> v2:172.21.15.62:3300/0 conn(0x7f3b14159550 0x7f3b140ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:12.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.375+0000 7f3b127fc700 1 -- 172.21.15.31:0/586569646 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1318732374 0 0) 0x7f3b140acc00 con 0x7f3b1415dfd0 2023-10-06T02:30:12.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.375+0000 7f3b127fc700 1 -- 172.21.15.31:0/586569646 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3afc003570 con 0x7f3b1415dfd0 2023-10-06T02:30:12.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.375+0000 7f3b127fc700 1 -- 172.21.15.31:0/586569646 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1525390288 0 0) 0x7f3b140abff0 con 0x7f3b14159920 2023-10-06T02:30:12.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.375+0000 7f3b127fc700 1 -- 172.21.15.31:0/586569646 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b140acc00 con 0x7f3b14159920 2023-10-06T02:30:12.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.375+0000 7f3b137fe700 1 -- 172.21.15.31:0/586569646 >> v1:172.21.15.62:6789/0 conn(0x7f3b14159920 legacy=0x7f3b140ad090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:12.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.375+0000 7f3b137fe700 1 -- 172.21.15.31:0/586569646 >> v1:172.21.15.31:6789/0 conn(0x7f3b1415dfd0 legacy=0x7f3b140aedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:12.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.375+0000 7f3b137fe700 1 -- 172.21.15.31:0/586569646 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b140ac5b0 con 0x7f3b14159550 2023-10-06T02:30:12.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.375+0000 7f3b137fe700 1 --2- 172.21.15.31:0/586569646 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b14159550 0x7f3b140ceac0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f3b040098e0 tx=0x7f3b040091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a48c91777f14e905 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:12.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.375+0000 7f3b127fc700 1 -- 172.21.15.31:0/586569646 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b04011630 con 0x7f3b14159550 2023-10-06T02:30:12.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.376+0000 7f3b127fc700 1 -- 172.21.15.31:0/586569646 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b04011790 con 0x7f3b14159550 2023-10-06T02:30:12.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.376+0000 7f3b127fc700 1 -- 172.21.15.31:0/586569646 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b04011b30 con 0x7f3b14159550 2023-10-06T02:30:12.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.376+0000 7f3b1899d700 1 -- 172.21.15.31:0/586569646 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b14159550 msgr2=0x7f3b140ceac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:12.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.376+0000 7f3b1899d700 1 --2- 172.21.15.31:0/586569646 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b14159550 0x7f3b140ceac0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f3b040098e0 tx=0x7f3b040091a0 comp rx=0 tx=0).stop 2023-10-06T02:30:12.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.376+0000 7f3b1899d700 1 -- 172.21.15.31:0/586569646 shutdown_connections 2023-10-06T02:30:12.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.376+0000 7f3b1899d700 1 --2- 172.21.15.31:0/586569646 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b14159550 0x7f3b140ceac0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:12.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.376+0000 7f3b1899d700 1 -- 172.21.15.31:0/586569646 >> 172.21.15.31:0/586569646 conn(0x7f3b1406ce80 msgr2=0x7f3b140ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:12.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.376+0000 7f3b1899d700 1 -- 172.21.15.31:0/586569646 shutdown_connections 2023-10-06T02:30:12.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.376+0000 7f3b1899d700 1 -- 172.21.15.31:0/586569646 wait complete. 2023-10-06T02:30:12.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.378+0000 7f3b1899d700 1 Processor -- start 2023-10-06T02:30:12.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.378+0000 7f3b1899d700 1 -- start start 2023-10-06T02:30:12.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.379+0000 7f3b1899d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b14159550 0x7f3b140c9150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:12.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.379+0000 7f3b1899d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3b14159920 0x7f3b14157fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:12.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.379+0000 7f3b1899d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b1415dfd0 0x7f3b140cc060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:12.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.379+0000 7f3b1899d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b140cadb0 con 0x7f3b14159550 2023-10-06T02:30:12.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.379+0000 7f3b1899d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b140cbbd0 con 0x7f3b1415dfd0 2023-10-06T02:30:12.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.379+0000 7f3b1899d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b140cafc0 con 0x7f3b14159920 2023-10-06T02:30:12.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.379+0000 7f3b137fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b14159550 0x7f3b140c9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:12.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.379+0000 7f3b137fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b14159550 0x7f3b140c9150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39148/0 (socket says 172.21.15.31:39148) 2023-10-06T02:30:12.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.379+0000 7f3b137fe700 1 -- 172.21.15.31:0/1717135157 learned_addr learned my addr 172.21.15.31:0/1717135157 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:12.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.379+0000 7f3b137fe700 1 -- 172.21.15.31:0/1717135157 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3b14159920 msgr2=0x7f3b14157fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:12.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.379+0000 7f3b13fff700 1 --2- 172.21.15.31:0/1717135157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b1415dfd0 0x7f3b140cc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:12.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.380+0000 7f3b12ffd700 1 --2- 172.21.15.31:0/1717135157 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3b14159920 0x7f3b14157fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:12.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.380+0000 7f3b137fe700 1 --2- 172.21.15.31:0/1717135157 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3b14159920 0x7f3b14157fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:12.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.380+0000 7f3b137fe700 1 -- 172.21.15.31:0/1717135157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b1415dfd0 msgr2=0x7f3b140cc060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:12.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.380+0000 7f3b137fe700 1 --2- 172.21.15.31:0/1717135157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b1415dfd0 0x7f3b140cc060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:12.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.380+0000 7f3b137fe700 1 -- 172.21.15.31:0/1717135157 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b04009420 con 0x7f3b14159550 2023-10-06T02:30:12.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.380+0000 7f3b12ffd700 1 --2- 172.21.15.31:0/1717135157 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3b14159920 0x7f3b14157fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:30:12.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.380+0000 7f3b13fff700 1 --2- 172.21.15.31:0/1717135157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b1415dfd0 0x7f3b140cc060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:30:12.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.380+0000 7f3b137fe700 1 --2- 172.21.15.31:0/1717135157 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b14159550 0x7f3b140c9150 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f3b1406ad70 tx=0x7f3b04017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:12.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.380+0000 7f3afb7fe700 1 -- 172.21.15.31:0/1717135157 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b0401f070 con 0x7f3b14159550 2023-10-06T02:30:12.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.380+0000 7f3afb7fe700 1 -- 172.21.15.31:0/1717135157 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b04000c90 con 0x7f3b14159550 2023-10-06T02:30:12.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.380+0000 7f3afb7fe700 1 -- 172.21.15.31:0/1717135157 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b04025420 con 0x7f3b14159550 2023-10-06T02:30:12.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.380+0000 7f3b1899d700 1 -- 172.21.15.31:0/1717135157 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b140cb2a0 con 0x7f3b14159550 2023-10-06T02:30:12.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.380+0000 7f3b1899d700 1 -- 172.21.15.31:0/1717135157 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b140cb860 con 0x7f3b14159550 2023-10-06T02:30:12.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.381+0000 7f3b1899d700 1 -- 172.21.15.31:0/1717135157 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b140cbeb0 con 0x7f3b14159550 2023-10-06T02:30:12.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.382+0000 7f3afb7fe700 1 -- 172.21.15.31:0/1717135157 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f3b04023070 con 0x7f3b14159550 2023-10-06T02:30:12.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.383+0000 7f3afb7fe700 1 --2- 172.21.15.31:0/1717135157 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f3b000674d0 0x7f3b00069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:12.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.383+0000 7f3afb7fe700 1 -- 172.21.15.31:0/1717135157 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f3b040864d0 con 0x7f3b14159550 2023-10-06T02:30:12.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.383+0000 7f3b12ffd700 1 --2- 172.21.15.31:0/1717135157 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f3b000674d0 0x7f3b00069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:12.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.384+0000 7f3b12ffd700 1 --2- 172.21.15.31:0/1717135157 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f3b000674d0 0x7f3b00069980 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3b08007fd0 tx=0x7f3b08009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:12.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.387+0000 7f3afb7fe700 1 -- 172.21.15.31:0/1717135157 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3b04052d10 con 0x7f3b14159550 2023-10-06T02:30:12.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.592+0000 7f3b1899d700 1 -- 172.21.15.31:0/1717135157 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3b1404c3c0 con 0x7f3b14159550 2023-10-06T02:30:12.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.593+0000 7f3afb7fe700 1 -- 172.21.15.31:0/1717135157 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v131) v1 ==== 74+0+16889 (secure 0 0 0) 0x7f3b04056330 con 0x7f3b14159550 2023-10-06T02:30:12.594 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:30:12.594 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":131,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:30:11.624705+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:30:09.791433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-10-06T02:30:10.601699+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":20,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:30:12.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.595+0000 7f3b1899d700 1 -- 172.21.15.31:0/1717135157 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f3b000674d0 msgr2=0x7f3b00069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:12.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.595+0000 7f3b1899d700 1 --2- 172.21.15.31:0/1717135157 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f3b000674d0 0x7f3b00069980 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3b08007fd0 tx=0x7f3b08009040 comp rx=0 tx=0).stop 2023-10-06T02:30:12.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.595+0000 7f3b1899d700 1 -- 172.21.15.31:0/1717135157 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b14159550 msgr2=0x7f3b140c9150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:12.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.595+0000 7f3b1899d700 1 --2- 172.21.15.31:0/1717135157 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b14159550 0x7f3b140c9150 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f3b1406ad70 tx=0x7f3b04017000 comp rx=0 tx=0).stop 2023-10-06T02:30:12.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.595+0000 7f3b1899d700 1 -- 172.21.15.31:0/1717135157 shutdown_connections 2023-10-06T02:30:12.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.595+0000 7f3b1899d700 1 --2- 172.21.15.31:0/1717135157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b1415dfd0 0x7f3b140cc060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:12.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.595+0000 7f3b1899d700 1 --2- 172.21.15.31:0/1717135157 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b14159550 0x7f3b140c9150 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:12.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.595+0000 7f3b1899d700 1 --2- 172.21.15.31:0/1717135157 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f3b000674d0 0x7f3b00069980 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:12.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.595+0000 7f3b1899d700 1 --2- 172.21.15.31:0/1717135157 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3b14159920 0x7f3b14157fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:12.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.595+0000 7f3b1899d700 1 -- 172.21.15.31:0/1717135157 >> 172.21.15.31:0/1717135157 conn(0x7f3b1406ce80 msgr2=0x7f3b141535c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:12.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.595+0000 7f3b1899d700 1 -- 172.21.15.31:0/1717135157 shutdown_connections 2023-10-06T02:30:12.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.595+0000 7f3b1899d700 1 -- 172.21.15.31:0/1717135157 wait complete. 2023-10-06T02:30:12.613 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:30:12.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.852+0000 7f619bfff700 1 Processor -- start 2023-10-06T02:30:12.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.852+0000 7f619bfff700 1 -- start start 2023-10-06T02:30:12.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.853+0000 7f619bfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f619c13fb20 0x7f619c15a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:12.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.853+0000 7f619bfff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f619c06aaa0 0x7f619c0c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:12.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.853+0000 7f619bfff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f619c15ae70 con 0x7f619c13fef0 2023-10-06T02:30:12.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.853+0000 7f619bfff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f619c06af60 con 0x7f619c13fb20 2023-10-06T02:30:12.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.853+0000 7f619bfff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f619c06b240 con 0x7f619c06aaa0 2023-10-06T02:30:12.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.853+0000 7f619affd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f619c13fb20 0x7f619c15a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:12.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.853+0000 7f619a7fc700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f619c13fef0 0x7f619c0c7990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:47824/0 (socket says 172.21.15.31:47824) 2023-10-06T02:30:12.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.853+0000 7f619a7fc700 1 -- 172.21.15.31:0/1181283364 learned_addr learned my addr 172.21.15.31:0/1181283364 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:30:12.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.854+0000 7f619b7fe700 1 --2- 172.21.15.31:0/1181283364 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f619c06aaa0 0x7f619c0c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:12.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.854+0000 7f619affd700 1 -- 172.21.15.31:0/1181283364 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f619c06aaa0 msgr2=0x7f619c0c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:12.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.854+0000 7f619affd700 1 --2- 172.21.15.31:0/1181283364 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f619c06aaa0 0x7f619c0c9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:12.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.854+0000 7f619affd700 1 -- 172.21.15.31:0/1181283364 >> v1:172.21.15.31:6789/0 conn(0x7f619c13fef0 legacy=0x7f619c0c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:12.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.854+0000 7f619affd700 1 -- 172.21.15.31:0/1181283364 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f619c06b520 con 0x7f619c13fb20 2023-10-06T02:30:12.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.854+0000 7f619b7fe700 1 --2- 172.21.15.31:0/1181283364 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f619c06aaa0 0x7f619c0c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:30:12.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.854+0000 7f619affd700 1 --2- 172.21.15.31:0/1181283364 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f619c13fb20 0x7f619c15a930 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f618c0098e0 tx=0x7f618c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f61a3b250fdcd645 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:12.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.854+0000 7f6199ffb700 1 -- 172.21.15.31:0/1181283364 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f618c009e80 con 0x7f619c13fb20 2023-10-06T02:30:12.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.855+0000 7f6199ffb700 1 -- 172.21.15.31:0/1181283364 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f618c00b860 con 0x7f619c13fb20 2023-10-06T02:30:12.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.855+0000 7f6199ffb700 1 -- 172.21.15.31:0/1181283364 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f618c00bbe0 con 0x7f619c13fb20 2023-10-06T02:30:12.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.855+0000 7f619bfff700 1 -- 172.21.15.31:0/1181283364 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f619c13fb20 msgr2=0x7f619c15a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:12.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.855+0000 7f619bfff700 1 --2- 172.21.15.31:0/1181283364 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f619c13fb20 0x7f619c15a930 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f618c0098e0 tx=0x7f618c0091a0 comp rx=0 tx=0).stop 2023-10-06T02:30:12.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.855+0000 7f619bfff700 1 -- 172.21.15.31:0/1181283364 shutdown_connections 2023-10-06T02:30:12.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.855+0000 7f619bfff700 1 --2- 172.21.15.31:0/1181283364 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f619c13fb20 0x7f619c15a930 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:12.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.855+0000 7f619bfff700 1 --2- 172.21.15.31:0/1181283364 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f619c06aaa0 0x7f619c0c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:12.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.855+0000 7f619bfff700 1 -- 172.21.15.31:0/1181283364 >> 172.21.15.31:0/1181283364 conn(0x7f619c06d170 msgr2=0x7f619c0cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:12.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.855+0000 7f619bfff700 1 -- 172.21.15.31:0/1181283364 shutdown_connections 2023-10-06T02:30:12.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.855+0000 7f619bfff700 1 -- 172.21.15.31:0/1181283364 wait complete. 2023-10-06T02:30:12.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.857+0000 7f619bfff700 1 Processor -- start 2023-10-06T02:30:12.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.857+0000 7f619bfff700 1 -- start start 2023-10-06T02:30:12.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.858+0000 7f619bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f619c06aaa0 0x7f619c155cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:12.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.858+0000 7f619bfff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f619c13fb20 0x7f619c13e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:12.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.858+0000 7f619bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f619c13fef0 0x7f619c13ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:12.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.858+0000 7f619bfff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f619c159440 con 0x7f619c13fef0 2023-10-06T02:30:12.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.858+0000 7f619bfff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f619c156230 con 0x7f619c06aaa0 2023-10-06T02:30:12.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.858+0000 7f619bfff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f619c158e80 con 0x7f619c13fb20 2023-10-06T02:30:12.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.858+0000 7f619b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f619c13fef0 0x7f619c13ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:12.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.858+0000 7f619b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f619c13fef0 0x7f619c13ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39160/0 (socket says 172.21.15.31:39160) 2023-10-06T02:30:12.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.858+0000 7f619b7fe700 1 -- 172.21.15.31:0/698406312 learned_addr learned my addr 172.21.15.31:0/698406312 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:12.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.858+0000 7f619affd700 1 --2- 172.21.15.31:0/698406312 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f619c06aaa0 0x7f619c155cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:12.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.858+0000 7f619a7fc700 1 --2- 172.21.15.31:0/698406312 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f619c13fb20 0x7f619c13e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:12.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.858+0000 7f619b7fe700 1 -- 172.21.15.31:0/698406312 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f619c13fb20 msgr2=0x7f619c13e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:12.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.858+0000 7f619b7fe700 1 --2- 172.21.15.31:0/698406312 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f619c13fb20 0x7f619c13e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:12.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.858+0000 7f619b7fe700 1 -- 172.21.15.31:0/698406312 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f619c06aaa0 msgr2=0x7f619c155cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:12.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.858+0000 7f619b7fe700 1 --2- 172.21.15.31:0/698406312 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f619c06aaa0 0x7f619c155cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:12.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.858+0000 7f619b7fe700 1 -- 172.21.15.31:0/698406312 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f618c009420 con 0x7f619c13fef0 2023-10-06T02:30:12.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.858+0000 7f619a7fc700 1 --2- 172.21.15.31:0/698406312 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f619c13fb20 0x7f619c13e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:30:12.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.859+0000 7f619b7fe700 1 --2- 172.21.15.31:0/698406312 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f619c13fef0 0x7f619c13ecc0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f6194009f90 tx=0x7f61940099b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:12.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.859+0000 7f6182ffd700 1 -- 172.21.15.31:0/698406312 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f619400dd40 con 0x7f619c13fef0 2023-10-06T02:30:12.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.859+0000 7f6182ffd700 1 -- 172.21.15.31:0/698406312 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f619400f040 con 0x7f619c13fef0 2023-10-06T02:30:12.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.859+0000 7f6182ffd700 1 -- 172.21.15.31:0/698406312 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6194015690 con 0x7f619c13fef0 2023-10-06T02:30:12.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.859+0000 7f619bfff700 1 -- 172.21.15.31:0/698406312 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f619c159160 con 0x7f619c13fef0 2023-10-06T02:30:12.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.859+0000 7f619bfff700 1 -- 172.21.15.31:0/698406312 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f619c0abb50 con 0x7f619c13fef0 2023-10-06T02:30:12.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.860+0000 7f619bfff700 1 -- 172.21.15.31:0/698406312 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f619c04c3c0 con 0x7f619c13fef0 2023-10-06T02:30:12.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.861+0000 7f6182ffd700 1 -- 172.21.15.31:0/698406312 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f619401e3b0 con 0x7f619c13fef0 2023-10-06T02:30:12.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.862+0000 7f6182ffd700 1 --2- 172.21.15.31:0/698406312 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f61880674d0 0x7f6188069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:12.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.862+0000 7f6182ffd700 1 -- 172.21.15.31:0/698406312 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f6194085760 con 0x7f619c13fef0 2023-10-06T02:30:12.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.863+0000 7f619affd700 1 --2- 172.21.15.31:0/698406312 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f61880674d0 0x7f6188069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:12.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.863+0000 7f619affd700 1 --2- 172.21.15.31:0/698406312 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f61880674d0 0x7f6188069980 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f618c0093f0 tx=0x7f618c019000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:12.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:12.866+0000 7f6182ffd700 1 -- 172.21.15.31:0/698406312 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6194051ed0 con 0x7f619c13fef0 2023-10-06T02:30:13.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:13.068+0000 7f619bfff700 1 -- 172.21.15.31:0/698406312 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f619c0abd10 con 0x7f619c13fef0 2023-10-06T02:30:13.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:13.069+0000 7f6182ffd700 1 -- 172.21.15.31:0/698406312 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v131) v1 ==== 74+0+16889 (secure 0 0 0) 0x7f61940554f0 con 0x7f619c13fef0 2023-10-06T02:30:13.070 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:30:13.070 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":131,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:30:11.624705+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:30:09.791433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-10-06T02:30:10.601699+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":20,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:30:13.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:13.071+0000 7f619bfff700 1 -- 172.21.15.31:0/698406312 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f61880674d0 msgr2=0x7f6188069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:13.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:13.071+0000 7f619bfff700 1 --2- 172.21.15.31:0/698406312 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f61880674d0 0x7f6188069980 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f618c0093f0 tx=0x7f618c019000 comp rx=0 tx=0).stop 2023-10-06T02:30:13.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:13.071+0000 7f619bfff700 1 -- 172.21.15.31:0/698406312 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f619c13fef0 msgr2=0x7f619c13ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:13.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:13.071+0000 7f619bfff700 1 --2- 172.21.15.31:0/698406312 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f619c13fef0 0x7f619c13ecc0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f6194009f90 tx=0x7f61940099b0 comp rx=0 tx=0).stop 2023-10-06T02:30:13.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:13.071+0000 7f619bfff700 1 -- 172.21.15.31:0/698406312 shutdown_connections 2023-10-06T02:30:13.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:13.071+0000 7f619bfff700 1 --2- 172.21.15.31:0/698406312 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f619c13fef0 0x7f619c13ecc0 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:13.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:13.071+0000 7f619bfff700 1 --2- 172.21.15.31:0/698406312 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f619c06aaa0 0x7f619c155cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:13.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:13.071+0000 7f619bfff700 1 --2- 172.21.15.31:0/698406312 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f61880674d0 0x7f6188069980 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:13.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:13.071+0000 7f619bfff700 1 --2- 172.21.15.31:0/698406312 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f619c13fb20 0x7f619c13e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:13.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:13.071+0000 7f619bfff700 1 -- 172.21.15.31:0/698406312 >> 172.21.15.31:0/698406312 conn(0x7f619c06d170 msgr2=0x7f619c13b900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:13.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:13.072+0000 7f619bfff700 1 -- 172.21.15.31:0/698406312 shutdown_connections 2023-10-06T02:30:13.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:13.072+0000 7f619bfff700 1 -- 172.21.15.31:0/698406312 wait complete. 2023-10-06T02:30:13.090 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T02:30:13.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:13.285+0000 7fb385d9d700 1 Processor -- start 2023-10-06T02:30:13.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:13.285+0000 7fb385d9d700 1 -- start start 2023-10-06T02:30:13.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:13.286+0000 7fb385d9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb3800c7930 0x7fb38006a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:13.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:13.286+0000 7fb385d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb3800ccce0 0x7fb38006de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:13.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:13.287+0000 7fb385d9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb38006b790 con 0x7fb3800cc840 2023-10-06T02:30:13.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:13.287+0000 7fb385d9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb380155fa0 con 0x7fb3800c7930 2023-10-06T02:30:13.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:13.287+0000 7fb385d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb38006a8e0 con 0x7fb3800ccce0 2023-10-06T02:30:13.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:13.287+0000 7fb384d9b700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb3800c7930 0x7fb38006a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:13.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:13.287+0000 7fb38559c700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb3800ccce0 0x7fb38006de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:13.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:13.287+0000 7fb384d9b700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb3800c7930 0x7fb38006a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:43682/0 (socket says 172.21.15.31:43682) 2023-10-06T02:30:13.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:13.287+0000 7fb384d9b700 1 -- 172.21.15.31:0/1713508838 learned_addr learned my addr 172.21.15.31:0/1713508838 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:13.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:13.288+0000 7fb38559c700 1 -- 172.21.15.31:0/1713508838 >> v2:172.21.15.62:3300/0 conn(0x7fb3800c7930 msgr2=0x7fb38006a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:13.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:13.288+0000 7fb38559c700 1 --2- 172.21.15.31:0/1713508838 >> v2:172.21.15.62:3300/0 conn(0x7fb3800c7930 0x7fb38006a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:13.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:13.288+0000 7fb38559c700 1 -- 172.21.15.31:0/1713508838 >> v1:172.21.15.62:6789/0 conn(0x7fb3800cc840 legacy=0x7fb380155890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:13.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:13.288+0000 7fb38559c700 1 -- 172.21.15.31:0/1713508838 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb38006ab80 con 0x7fb3800ccce0 2023-10-06T02:30:13.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:13.288+0000 7fb384d9b700 1 --2- 172.21.15.31:0/1713508838 >> v2:172.21.15.62:3300/0 conn(0x7fb3800c7930 0x7fb38006a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:30:13.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:13.288+0000 7fb38559c700 1 --2- 172.21.15.31:0/1713508838 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb3800ccce0 0x7fb38006de50 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb378009e60 tx=0x7fb378009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3b394e2171c9c5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:13.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:13.288+0000 7fb37f7fe700 1 -- 172.21.15.31:0/1713508838 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37800e070 con 0x7fb3800ccce0 2023-10-06T02:30:13.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:13.289+0000 7fb37f7fe700 1 -- 172.21.15.31:0/1713508838 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb37800bb60 con 0x7fb3800ccce0 2023-10-06T02:30:13.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:13.289+0000 7fb37f7fe700 1 -- 172.21.15.31:0/1713508838 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb378012430 con 0x7fb3800ccce0 2023-10-06T02:30:13.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:13.289+0000 7fb385d9d700 1 -- 172.21.15.31:0/1713508838 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb3800ccce0 msgr2=0x7fb38006de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:13.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:13.289+0000 7fb385d9d700 1 --2- 172.21.15.31:0/1713508838 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb3800ccce0 0x7fb38006de50 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb378009e60 tx=0x7fb378009780 comp rx=0 tx=0).stop 2023-10-06T02:30:13.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:13.289+0000 7fb385d9d700 1 -- 172.21.15.31:0/1713508838 shutdown_connections 2023-10-06T02:30:13.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:13.289+0000 7fb385d9d700 1 --2- 172.21.15.31:0/1713508838 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb3800ccce0 0x7fb38006de50 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:13.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:13.289+0000 7fb385d9d700 1 --2- 172.21.15.31:0/1713508838 >> v2:172.21.15.62:3300/0 conn(0x7fb3800c7930 0x7fb38006a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:13.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:13.289+0000 7fb385d9d700 1 -- 172.21.15.31:0/1713508838 >> 172.21.15.31:0/1713508838 conn(0x7fb38006d110 msgr2=0x7fb3800cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:13.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:13.289+0000 7fb385d9d700 1 -- 172.21.15.31:0/1713508838 shutdown_connections 2023-10-06T02:30:13.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:13.289+0000 7fb385d9d700 1 -- 172.21.15.31:0/1713508838 wait complete. 2023-10-06T02:30:13.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:13.291+0000 7fb385d9d700 1 Processor -- start 2023-10-06T02:30:13.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:13.291+0000 7fb385d9d700 1 -- start start 2023-10-06T02:30:13.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:13.292+0000 7fb385d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3800c7930 0x7fb38014a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:13.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:13.292+0000 7fb385d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb3800cc840 0x7fb3800b1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:13.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:13.292+0000 7fb385d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3800ccce0 0x7fb380157650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:13.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:13.292+0000 7fb385d9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3800b0910 con 0x7fb3800c7930 2023-10-06T02:30:13.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:13.292+0000 7fb385d9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3800c4bf0 con 0x7fb3800ccce0 2023-10-06T02:30:13.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:13.292+0000 7fb385d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3800af700 con 0x7fb3800cc840 2023-10-06T02:30:13.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:13.292+0000 7fb384d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3800c7930 0x7fb38014a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:13.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:13.292+0000 7fb384d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3800c7930 0x7fb38014a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39168/0 (socket says 172.21.15.31:39168) 2023-10-06T02:30:13.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:13.292+0000 7fb384d9b700 1 -- 172.21.15.31:0/187679478 learned_addr learned my addr 172.21.15.31:0/187679478 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:13.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:13.292+0000 7fb384d9b700 1 -- 172.21.15.31:0/187679478 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb3800cc840 msgr2=0x7fb3800b1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:13.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:13.292+0000 7fb384d9b700 1 --2- 172.21.15.31:0/187679478 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb3800cc840 0x7fb3800b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:13.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:13.292+0000 7fb384d9b700 1 -- 172.21.15.31:0/187679478 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3800ccce0 msgr2=0x7fb380157650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:13.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:13.292+0000 7fb384d9b700 1 --2- 172.21.15.31:0/187679478 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3800ccce0 0x7fb380157650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:13.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:13.292+0000 7fb384d9b700 1 -- 172.21.15.31:0/187679478 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb37400b790 con 0x7fb3800c7930 2023-10-06T02:30:13.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:13.293+0000 7fb384d9b700 1 --2- 172.21.15.31:0/187679478 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3800c7930 0x7fb38014a120 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fb37400ede0 tx=0x7fb37400b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:13.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:13.293+0000 7fb37cff9700 1 -- 172.21.15.31:0/187679478 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37400dd80 con 0x7fb3800c7930 2023-10-06T02:30:13.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:13.293+0000 7fb37cff9700 1 -- 172.21.15.31:0/187679478 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb374010440 con 0x7fb3800c7930 2023-10-06T02:30:13.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:13.293+0000 7fb37cff9700 1 -- 172.21.15.31:0/187679478 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3740107c0 con 0x7fb3800c7930 2023-10-06T02:30:13.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:13.293+0000 7fb385d9d700 1 -- 172.21.15.31:0/187679478 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb378009a20 con 0x7fb3800c7930 2023-10-06T02:30:13.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:13.293+0000 7fb385d9d700 1 -- 172.21.15.31:0/187679478 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb38014a660 con 0x7fb3800c7930 2023-10-06T02:30:13.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:13.294+0000 7fb385d9d700 1 -- 172.21.15.31:0/187679478 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3800c5750 con 0x7fb3800c7930 2023-10-06T02:30:13.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:13.295+0000 7fb37cff9700 1 -- 172.21.15.31:0/187679478 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7fb374010b70 con 0x7fb3800c7930 2023-10-06T02:30:13.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:13.295+0000 7fb37cff9700 1 --2- 172.21.15.31:0/187679478 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb3640674d0 0x7fb364069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:13.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:13.296+0000 7fb37cff9700 1 -- 172.21.15.31:0/187679478 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fb374085ba0 con 0x7fb3800c7930 2023-10-06T02:30:13.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:13.296+0000 7fb37ffff700 1 --2- 172.21.15.31:0/187679478 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb3640674d0 0x7fb364069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:13.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:13.297+0000 7fb37ffff700 1 --2- 172.21.15.31:0/187679478 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb3640674d0 0x7fb364069980 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb380067650 tx=0x7fb36c007b40 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:13.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:13.299+0000 7fb37cff9700 1 -- 172.21.15.31:0/187679478 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb3740522d0 con 0x7fb3800c7930 2023-10-06T02:30:13.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:13.546+0000 7fb385d9d700 1 -- 172.21.15.31:0/187679478 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fb38004c3c0 con 0x7fb3800c7930 2023-10-06T02:30:14.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:14.150+0000 7fb37cff9700 1 -- 172.21.15.31:0/187679478 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v89) v1 ==== 118+0+0 (secure 0 0 0) 0x7fb3740558f0 con 0x7fb3800c7930 2023-10-06T02:30:14.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:14.152+0000 7fb385d9d700 1 -- 172.21.15.31:0/187679478 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb3640674d0 msgr2=0x7fb364069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:14.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:14.152+0000 7fb385d9d700 1 --2- 172.21.15.31:0/187679478 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb3640674d0 0x7fb364069980 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb380067650 tx=0x7fb36c007b40 comp rx=0 tx=0).stop 2023-10-06T02:30:14.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:14.152+0000 7fb385d9d700 1 -- 172.21.15.31:0/187679478 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3800c7930 msgr2=0x7fb38014a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:14.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:14.152+0000 7fb385d9d700 1 --2- 172.21.15.31:0/187679478 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3800c7930 0x7fb38014a120 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fb37400ede0 tx=0x7fb37400b4f0 comp rx=0 tx=0).stop 2023-10-06T02:30:14.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:14.152+0000 7fb385d9d700 1 -- 172.21.15.31:0/187679478 shutdown_connections 2023-10-06T02:30:14.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:14.152+0000 7fb385d9d700 1 --2- 172.21.15.31:0/187679478 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3800ccce0 0x7fb380157650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:14.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:14.152+0000 7fb385d9d700 1 --2- 172.21.15.31:0/187679478 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3800c7930 0x7fb38014a120 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:14.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:14.152+0000 7fb385d9d700 1 --2- 172.21.15.31:0/187679478 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb3640674d0 0x7fb364069980 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:14.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:14.152+0000 7fb385d9d700 1 --2- 172.21.15.31:0/187679478 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb3800cc840 0x7fb3800b1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:14.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:14.152+0000 7fb385d9d700 1 -- 172.21.15.31:0/187679478 >> 172.21.15.31:0/187679478 conn(0x7fb38006d110 msgr2=0x7fb38006de50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:14.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:14.153+0000 7fb385d9d700 1 -- 172.21.15.31:0/187679478 shutdown_connections 2023-10-06T02:30:14.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:14.153+0000 7fb385d9d700 1 -- 172.21.15.31:0/187679478 wait complete. 2023-10-06T02:30:14.172 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T02:30:14.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:14.365+0000 7f93b679d700 1 Processor -- start 2023-10-06T02:30:14.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:14.365+0000 7f93b679d700 1 -- start start 2023-10-06T02:30:14.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:14.366+0000 7f93b679d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f93b00cf110 0x7f93b015de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:14.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:14.366+0000 7f93b679d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93b015ac50 con 0x7f93b0153a00 2023-10-06T02:30:14.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:14.366+0000 7f93b679d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93b015d010 con 0x7f93b0153dd0 2023-10-06T02:30:14.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:14.366+0000 7f93b679d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93b015af60 con 0x7f93b00cf110 2023-10-06T02:30:14.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:14.366+0000 7f93b579b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f93b0153a00 0x7f93b015a540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:47834/0 (socket says 172.21.15.31:47834) 2023-10-06T02:30:14.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:14.366+0000 7f93b579b700 1 -- 172.21.15.31:0/789027238 learned_addr learned my addr 172.21.15.31:0/789027238 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:30:14.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:14.367+0000 7f93b5f9c700 1 --2- 172.21.15.31:0/789027238 >> v2:172.21.15.31:3300/0 conn(0x7f93b00cf110 0x7f93b015de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:14.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:14.367+0000 7f93a7fff700 1 -- 172.21.15.31:0/789027238 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1152135105 0 0) 0x7f93b015ac50 con 0x7f93b0153a00 2023-10-06T02:30:14.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:14.367+0000 7f93a7fff700 1 -- 172.21.15.31:0/789027238 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f939c003570 con 0x7f93b0153a00 2023-10-06T02:30:14.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:14.367+0000 7f93a7fff700 1 -- 172.21.15.31:0/789027238 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3224983125 0 0) 0x7f93b015d010 con 0x7f93b0153dd0 2023-10-06T02:30:14.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:14.367+0000 7f93b5f9c700 1 -- 172.21.15.31:0/789027238 >> v1:172.21.15.62:6789/0 conn(0x7f93b0153dd0 legacy=0x7f93b00cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:14.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:14.367+0000 7f93b5f9c700 1 -- 172.21.15.31:0/789027238 >> v1:172.21.15.31:6789/0 conn(0x7f93b0153a00 legacy=0x7f93b015a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:14.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:14.367+0000 7f93b5f9c700 1 -- 172.21.15.31:0/789027238 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93b015b240 con 0x7f93b00cf110 2023-10-06T02:30:14.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:14.367+0000 7f93b5f9c700 1 --2- 172.21.15.31:0/789027238 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93b00cf110 0x7f93b015de50 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f93a8009e60 tx=0x7f93a8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e052c257317ac70 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:14.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:14.368+0000 7f93a7fff700 1 -- 172.21.15.31:0/789027238 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a800e070 con 0x7f93b00cf110 2023-10-06T02:30:14.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:14.368+0000 7f93a7fff700 1 -- 172.21.15.31:0/789027238 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93a800bb60 con 0x7f93b00cf110 2023-10-06T02:30:14.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:14.368+0000 7f93a7fff700 1 -- 172.21.15.31:0/789027238 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a8012430 con 0x7f93b00cf110 2023-10-06T02:30:14.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:14.368+0000 7f93b679d700 1 -- 172.21.15.31:0/789027238 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93b00cf110 msgr2=0x7f93b015de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:14.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:14.368+0000 7f93b679d700 1 --2- 172.21.15.31:0/789027238 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93b00cf110 0x7f93b015de50 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f93a8009e60 tx=0x7f93a8009780 comp rx=0 tx=0).stop 2023-10-06T02:30:14.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:14.368+0000 7f93b679d700 1 -- 172.21.15.31:0/789027238 shutdown_connections 2023-10-06T02:30:14.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:14.368+0000 7f93b679d700 1 --2- 172.21.15.31:0/789027238 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93b00cf110 0x7f93b015de50 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:14.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:14.368+0000 7f93b679d700 1 -- 172.21.15.31:0/789027238 >> 172.21.15.31:0/789027238 conn(0x7f93b00703a0 msgr2=0x7f93b00707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:14.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:14.368+0000 7f93b679d700 1 -- 172.21.15.31:0/789027238 shutdown_connections 2023-10-06T02:30:14.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:14.368+0000 7f93b679d700 1 -- 172.21.15.31:0/789027238 wait complete. 2023-10-06T02:30:14.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:14.370+0000 7f93b679d700 1 Processor -- start 2023-10-06T02:30:14.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:14.370+0000 7f93b679d700 1 -- start start 2023-10-06T02:30:14.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:14.371+0000 7f93b679d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93b00cf110 0x7f93b00cbc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:14.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:14.371+0000 7f93b679d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93b0153a00 0x7f93b0151c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:14.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:14.371+0000 7f93b679d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f93b0153dd0 0x7f93b016cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:14.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:14.371+0000 7f93b679d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b0156eb0 con 0x7f93b00cf110 2023-10-06T02:30:14.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:14.371+0000 7f93b679d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b00cc160 con 0x7f93b0153a00 2023-10-06T02:30:14.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:14.371+0000 7f93b679d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93b00cb790 con 0x7f93b0153dd0 2023-10-06T02:30:14.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:14.371+0000 7f93b579b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93b00cf110 0x7f93b00cbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:14.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:14.371+0000 7f93b579b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93b00cf110 0x7f93b00cbc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39180/0 (socket says 172.21.15.31:39180) 2023-10-06T02:30:14.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:14.371+0000 7f93b579b700 1 -- 172.21.15.31:0/3364630085 learned_addr learned my addr 172.21.15.31:0/3364630085 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:14.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:14.371+0000 7f93b5f9c700 1 --2- 172.21.15.31:0/3364630085 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f93b0153dd0 0x7f93b016cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:14.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:14.371+0000 7f93b579b700 1 -- 172.21.15.31:0/3364630085 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f93b0153dd0 msgr2=0x7f93b016cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:14.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:14.372+0000 7f93b579b700 1 --2- 172.21.15.31:0/3364630085 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f93b0153dd0 0x7f93b016cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:14.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:14.372+0000 7f93b579b700 1 -- 172.21.15.31:0/3364630085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93b0153a00 msgr2=0x7f93b0151c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:14.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:14.372+0000 7f93b4f9a700 1 --2- 172.21.15.31:0/3364630085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93b0153a00 0x7f93b0151c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:14.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:14.372+0000 7f93b579b700 1 --2- 172.21.15.31:0/3364630085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93b0153a00 0x7f93b0151c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:14.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:14.372+0000 7f93b579b700 1 -- 172.21.15.31:0/3364630085 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93a8009a20 con 0x7f93b00cf110 2023-10-06T02:30:14.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:14.372+0000 7f93b5f9c700 1 --2- 172.21.15.31:0/3364630085 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f93b0153dd0 0x7f93b016cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:30:14.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:14.372+0000 7f93b4f9a700 1 --2- 172.21.15.31:0/3364630085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93b0153a00 0x7f93b0151c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:30:14.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:14.372+0000 7f93b579b700 1 --2- 172.21.15.31:0/3364630085 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93b00cf110 0x7f93b00cbc20 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f93ac0011e0 tx=0x7f93ac007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:14.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:14.372+0000 7f93a57fa700 1 -- 172.21.15.31:0/3364630085 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93ac005b30 con 0x7f93b00cf110 2023-10-06T02:30:14.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:14.372+0000 7f93a57fa700 1 -- 172.21.15.31:0/3364630085 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93ac005c90 con 0x7f93b00cf110 2023-10-06T02:30:14.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:14.372+0000 7f93a57fa700 1 -- 172.21.15.31:0/3364630085 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93ac0072e0 con 0x7f93b00cf110 2023-10-06T02:30:14.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:14.372+0000 7f93b679d700 1 -- 172.21.15.31:0/3364630085 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93b00ccee0 con 0x7f93b00cf110 2023-10-06T02:30:14.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:14.372+0000 7f93b679d700 1 -- 172.21.15.31:0/3364630085 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93b00cb420 con 0x7f93b00cf110 2023-10-06T02:30:14.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:14.373+0000 7f93b679d700 1 -- 172.21.15.31:0/3364630085 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93b004c3c0 con 0x7f93b00cf110 2023-10-06T02:30:14.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:14.374+0000 7f93a57fa700 1 -- 172.21.15.31:0/3364630085 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f93ac007650 con 0x7f93b00cf110 2023-10-06T02:30:14.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:14.375+0000 7f93a57fa700 1 --2- 172.21.15.31:0/3364630085 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f93940674d0 0x7f9394069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:14.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:14.375+0000 7f93a57fa700 1 -- 172.21.15.31:0/3364630085 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f93ac084570 con 0x7f93b00cf110 2023-10-06T02:30:14.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:14.375+0000 7f93b4f9a700 1 --2- 172.21.15.31:0/3364630085 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f93940674d0 0x7f9394069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:14.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:14.376+0000 7f93b4f9a700 1 --2- 172.21.15.31:0/3364630085 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f93940674d0 0x7f9394069980 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f93b00afe70 tx=0x7f93a0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:14.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:14.379+0000 7f93a57fa700 1 -- 172.21.15.31:0/3364630085 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f93ac04b5a0 con 0x7f93b00cf110 2023-10-06T02:30:14.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:14.628+0000 7f93b679d700 1 -- 172.21.15.31:0/3364630085 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f93b015b520 con 0x7f93b00cf110 2023-10-06T02:30:15.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.158+0000 7f93a57fa700 1 -- 172.21.15.31:0/3364630085 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v90) v1 ==== 122+0+0 (secure 0 0 0) 0x7f93ac050e60 con 0x7f93b00cf110 2023-10-06T02:30:15.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.161+0000 7f93b679d700 1 -- 172.21.15.31:0/3364630085 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f93940674d0 msgr2=0x7f9394069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:15.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.161+0000 7f93b679d700 1 --2- 172.21.15.31:0/3364630085 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f93940674d0 0x7f9394069980 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f93b00afe70 tx=0x7f93a0009040 comp rx=0 tx=0).stop 2023-10-06T02:30:15.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.161+0000 7f93b679d700 1 -- 172.21.15.31:0/3364630085 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93b00cf110 msgr2=0x7f93b00cbc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:15.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.161+0000 7f93b679d700 1 --2- 172.21.15.31:0/3364630085 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93b00cf110 0x7f93b00cbc20 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f93ac0011e0 tx=0x7f93ac007df0 comp rx=0 tx=0).stop 2023-10-06T02:30:15.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.161+0000 7f93b679d700 1 -- 172.21.15.31:0/3364630085 shutdown_connections 2023-10-06T02:30:15.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.161+0000 7f93b679d700 1 --2- 172.21.15.31:0/3364630085 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f93940674d0 0x7f9394069980 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:15.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.161+0000 7f93b679d700 1 --2- 172.21.15.31:0/3364630085 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f93b0153dd0 0x7f93b016cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:15.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.161+0000 7f93b679d700 1 --2- 172.21.15.31:0/3364630085 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93b00cf110 0x7f93b00cbc20 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:15.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.161+0000 7f93b679d700 1 --2- 172.21.15.31:0/3364630085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93b0153a00 0x7f93b0151c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:15.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.161+0000 7f93b679d700 1 -- 172.21.15.31:0/3364630085 >> 172.21.15.31:0/3364630085 conn(0x7f93b00703a0 msgr2=0x7f93b015cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:15.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.162+0000 7f93b679d700 1 -- 172.21.15.31:0/3364630085 shutdown_connections 2023-10-06T02:30:15.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.162+0000 7f93b679d700 1 -- 172.21.15.31:0/3364630085 wait complete. 2023-10-06T02:30:15.182 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:30:15.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.372+0000 7fd05599d700 1 Processor -- start 2023-10-06T02:30:15.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.372+0000 7fd05599d700 1 -- start start 2023-10-06T02:30:15.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.374+0000 7fd05599d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd05006e0a0 0x7fd0500c6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:15.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.374+0000 7fd05599d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd050150380 0x7fd05006a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:15.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.374+0000 7fd05599d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd05006a490 con 0x7fd05006f3a0 2023-10-06T02:30:15.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.374+0000 7fd05599d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0500c7490 con 0x7fd050150380 2023-10-06T02:30:15.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.374+0000 7fd05599d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0500c7770 con 0x7fd05006e0a0 2023-10-06T02:30:15.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.374+0000 7fd05519c700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd050150380 0x7fd05006a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:15.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.374+0000 7fd05499b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd05006e0a0 0x7fd0500c6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:15.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.375+0000 7fd05519c700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd050150380 0x7fd05006a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:43708/0 (socket says 172.21.15.31:43708) 2023-10-06T02:30:15.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.375+0000 7fd05519c700 1 -- 172.21.15.31:0/3336253716 learned_addr learned my addr 172.21.15.31:0/3336253716 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:15.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.375+0000 7fd04ffff700 1 --1- 172.21.15.31:0/3336253716 >> v1:172.21.15.62:6789/0 conn(0x7fd05006f3a0 0x7fd05014fc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:35770/0 (socket says 172.21.15.31:35770) 2023-10-06T02:30:15.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.375+0000 7fd04f7fe700 1 -- 172.21.15.31:0/3336253716 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1875918859 0 0) 0x7fd05006a490 con 0x7fd05006f3a0 2023-10-06T02:30:15.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.375+0000 7fd04f7fe700 1 -- 172.21.15.31:0/3336253716 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd040003570 con 0x7fd05006f3a0 2023-10-06T02:30:15.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.375+0000 7fd05519c700 1 -- 172.21.15.31:0/3336253716 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd05006e0a0 msgr2=0x7fd0500c6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:15.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.375+0000 7fd05519c700 1 --2- 172.21.15.31:0/3336253716 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd05006e0a0 0x7fd0500c6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:15.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.375+0000 7fd05519c700 1 -- 172.21.15.31:0/3336253716 >> v1:172.21.15.62:6789/0 conn(0x7fd05006f3a0 legacy=0x7fd05014fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:15.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.375+0000 7fd05519c700 1 -- 172.21.15.31:0/3336253716 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0500c7a50 con 0x7fd050150380 2023-10-06T02:30:15.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.375+0000 7fd05499b700 1 --2- 172.21.15.31:0/3336253716 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd05006e0a0 0x7fd0500c6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:30:15.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.375+0000 7fd05519c700 1 --2- 172.21.15.31:0/3336253716 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd050150380 0x7fd05006a920 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fd048009f20 tx=0x7fd048009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c4e6840ce03c47a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:15.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.376+0000 7fd04f7fe700 1 -- 172.21.15.31:0/3336253716 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd04800e070 con 0x7fd050150380 2023-10-06T02:30:15.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.376+0000 7fd04f7fe700 1 -- 172.21.15.31:0/3336253716 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd04800b970 con 0x7fd050150380 2023-10-06T02:30:15.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.376+0000 7fd04f7fe700 1 -- 172.21.15.31:0/3336253716 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0480092e0 con 0x7fd050150380 2023-10-06T02:30:15.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.376+0000 7fd05599d700 1 -- 172.21.15.31:0/3336253716 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd050150380 msgr2=0x7fd05006a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:15.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.376+0000 7fd05599d700 1 --2- 172.21.15.31:0/3336253716 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd050150380 0x7fd05006a920 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fd048009f20 tx=0x7fd048009840 comp rx=0 tx=0).stop 2023-10-06T02:30:15.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.376+0000 7fd05599d700 1 -- 172.21.15.31:0/3336253716 shutdown_connections 2023-10-06T02:30:15.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.376+0000 7fd05599d700 1 --2- 172.21.15.31:0/3336253716 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd05006e0a0 0x7fd0500c6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:15.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.376+0000 7fd05599d700 1 --2- 172.21.15.31:0/3336253716 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd050150380 0x7fd05006a920 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:15.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.376+0000 7fd05599d700 1 -- 172.21.15.31:0/3336253716 >> 172.21.15.31:0/3336253716 conn(0x7fd05006d170 msgr2=0x7fd0500cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:15.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.376+0000 7fd05599d700 1 -- 172.21.15.31:0/3336253716 shutdown_connections 2023-10-06T02:30:15.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.376+0000 7fd05599d700 1 -- 172.21.15.31:0/3336253716 wait complete. 2023-10-06T02:30:15.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.378+0000 7fd05599d700 1 Processor -- start 2023-10-06T02:30:15.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.378+0000 7fd05599d700 1 -- start start 2023-10-06T02:30:15.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.379+0000 7fd05599d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd05006e0a0 0x7fd0501696c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:15.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.379+0000 7fd05599d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd05006f3a0 0x7fd0500c31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:15.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.379+0000 7fd05599d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd050150380 0x7fd050164ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:15.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.379+0000 7fd05599d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd050156f40 con 0x7fd050150380 2023-10-06T02:30:15.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.379+0000 7fd05599d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd050169c00 con 0x7fd05006e0a0 2023-10-06T02:30:15.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.379+0000 7fd05599d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd050156330 con 0x7fd05006f3a0 2023-10-06T02:30:15.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.379+0000 7fd05519c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd050150380 0x7fd050164ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:15.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.379+0000 7fd05519c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd050150380 0x7fd050164ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39196/0 (socket says 172.21.15.31:39196) 2023-10-06T02:30:15.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.379+0000 7fd05519c700 1 -- 172.21.15.31:0/3631176539 learned_addr learned my addr 172.21.15.31:0/3631176539 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:15.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.379+0000 7fd05499b700 1 --2- 172.21.15.31:0/3631176539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd05006e0a0 0x7fd0501696c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:15.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.379+0000 7fd04ffff700 1 --2- 172.21.15.31:0/3631176539 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd05006f3a0 0x7fd0500c31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:15.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.380+0000 7fd05519c700 1 -- 172.21.15.31:0/3631176539 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd05006f3a0 msgr2=0x7fd0500c31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:15.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.380+0000 7fd05519c700 1 --2- 172.21.15.31:0/3631176539 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd05006f3a0 0x7fd0500c31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:15.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.380+0000 7fd05519c700 1 -- 172.21.15.31:0/3631176539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd05006e0a0 msgr2=0x7fd0501696c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:15.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.380+0000 7fd05519c700 1 --2- 172.21.15.31:0/3631176539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd05006e0a0 0x7fd0501696c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:15.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.380+0000 7fd05519c700 1 -- 172.21.15.31:0/3631176539 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd04400a5d0 con 0x7fd050150380 2023-10-06T02:30:15.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.380+0000 7fd05519c700 1 --2- 172.21.15.31:0/3631176539 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd050150380 0x7fd050164ec0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fd050071320 tx=0x7fd048018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:15.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.380+0000 7fd04cff9700 1 -- 172.21.15.31:0/3631176539 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0480032f0 con 0x7fd050150380 2023-10-06T02:30:15.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.380+0000 7fd04cff9700 1 -- 172.21.15.31:0/3631176539 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd048008c30 con 0x7fd050150380 2023-10-06T02:30:15.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.380+0000 7fd04cff9700 1 -- 172.21.15.31:0/3631176539 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd048025460 con 0x7fd050150380 2023-10-06T02:30:15.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.380+0000 7fd05599d700 1 -- 172.21.15.31:0/3631176539 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd048009ae0 con 0x7fd050150380 2023-10-06T02:30:15.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.380+0000 7fd05599d700 1 -- 172.21.15.31:0/3631176539 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0501568f0 con 0x7fd050150380 2023-10-06T02:30:15.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.381+0000 7fd05599d700 1 -- 172.21.15.31:0/3631176539 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0501570b0 con 0x7fd050150380 2023-10-06T02:30:15.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.382+0000 7fd04cff9700 1 -- 172.21.15.31:0/3631176539 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7fd048003040 con 0x7fd050150380 2023-10-06T02:30:15.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.383+0000 7fd04cff9700 1 --2- 172.21.15.31:0/3631176539 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd0340674d0 0x7fd034069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:15.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.383+0000 7fd04cff9700 1 -- 172.21.15.31:0/3631176539 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fd0480870d0 con 0x7fd050150380 2023-10-06T02:30:15.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.384+0000 7fd05499b700 1 --2- 172.21.15.31:0/3631176539 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd0340674d0 0x7fd034069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:15.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.384+0000 7fd05499b700 1 --2- 172.21.15.31:0/3631176539 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd0340674d0 0x7fd034069980 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd04400af50 tx=0x7fd044008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:15.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.388+0000 7fd04cff9700 1 -- 172.21.15.31:0/3631176539 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd048053800 con 0x7fd050150380 2023-10-06T02:30:15.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.635+0000 7fd05599d700 1 -- 172.21.15.31:0/3631176539 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd05004c3c0 con 0x7fd050150380 2023-10-06T02:30:15.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.635+0000 7fd04cff9700 1 -- 172.21.15.31:0/3631176539 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4601 (secure 0 0 0) 0x7fd048009940 con 0x7fd050150380 2023-10-06T02:30:15.636 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:30:15.636 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6611,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/1574790962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1574790962},{"type":"v1","addr":"172.21.15.62:6834","nonce":1574790962}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":85},{"gid":7591,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/2840240410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":2840240410},{"type":"v1","addr":"172.21.15.31:6835","nonce":2840240410}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":85},{"gid":7597,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2500071914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2500071914},{"type":"v1","addr":"172.21.15.62:6835","nonce":2500071914}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:30:11.625173+0000","modified":"2023-10-06T02:30:15.149386+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7971},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7971":{"gid":7971,"name":"a","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6834/1197111339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1197111339},{"type":"v1","addr":"172.21.15.31:6834","nonce":1197111339}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-10-06T02:30:15.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.637+0000 7fd05599d700 1 -- 172.21.15.31:0/3631176539 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd0340674d0 msgr2=0x7fd034069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:15.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.637+0000 7fd05599d700 1 --2- 172.21.15.31:0/3631176539 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd0340674d0 0x7fd034069980 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd04400af50 tx=0x7fd044008040 comp rx=0 tx=0).stop 2023-10-06T02:30:15.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.637+0000 7fd05599d700 1 -- 172.21.15.31:0/3631176539 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd050150380 msgr2=0x7fd050164ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:15.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.637+0000 7fd05599d700 1 --2- 172.21.15.31:0/3631176539 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd050150380 0x7fd050164ec0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fd050071320 tx=0x7fd048018000 comp rx=0 tx=0).stop 2023-10-06T02:30:15.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.638+0000 7fd05599d700 1 -- 172.21.15.31:0/3631176539 shutdown_connections 2023-10-06T02:30:15.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.638+0000 7fd05599d700 1 --2- 172.21.15.31:0/3631176539 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd050150380 0x7fd050164ec0 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:15.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.638+0000 7fd05599d700 1 --2- 172.21.15.31:0/3631176539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd05006e0a0 0x7fd0501696c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:15.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.638+0000 7fd05599d700 1 --2- 172.21.15.31:0/3631176539 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd0340674d0 0x7fd034069980 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:15.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.638+0000 7fd05599d700 1 --2- 172.21.15.31:0/3631176539 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd05006f3a0 0x7fd0500c31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:15.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.638+0000 7fd05599d700 1 -- 172.21.15.31:0/3631176539 >> 172.21.15.31:0/3631176539 conn(0x7fd05006d170 msgr2=0x7fd0500cb0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:15.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.638+0000 7fd05599d700 1 -- 172.21.15.31:0/3631176539 shutdown_connections 2023-10-06T02:30:15.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.638+0000 7fd05599d700 1 -- 172.21.15.31:0/3631176539 wait complete. 2023-10-06T02:30:15.640 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 90 2023-10-06T02:30:15.655 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:30:15.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.841+0000 7f2900d9d700 1 Processor -- start 2023-10-06T02:30:15.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.841+0000 7f2900d9d700 1 -- start start 2023-10-06T02:30:15.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.843+0000 7f2900d9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f28fc06aaa0 0x7f28fc15a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:15.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.843+0000 7f2900d9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28fc15ae70 con 0x7f28fc13fb20 2023-10-06T02:30:15.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.843+0000 7f2900d9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28fc06af60 con 0x7f28fc13fef0 2023-10-06T02:30:15.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.843+0000 7f2900d9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28fc06b240 con 0x7f28fc06aaa0 2023-10-06T02:30:15.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.843+0000 7f28fb7fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f28fc13fb20 0x7f28fc0c7990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:47838/0 (socket says 172.21.15.31:47838) 2023-10-06T02:30:15.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.843+0000 7f28fb7fe700 1 -- 172.21.15.31:0/2642009806 learned_addr learned my addr 172.21.15.31:0/2642009806 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:30:15.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.843+0000 7f28fbfff700 1 --2- 172.21.15.31:0/2642009806 >> v2:172.21.15.62:3300/0 conn(0x7f28fc06aaa0 0x7f28fc15a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:15.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.843+0000 7f28fa7fc700 1 -- 172.21.15.31:0/2642009806 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3627285859 0 0) 0x7f28ec001110 con 0x7f28fc13fb20 2023-10-06T02:30:15.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.843+0000 7f28fa7fc700 1 -- 172.21.15.31:0/2642009806 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28e4003570 con 0x7f28fc13fb20 2023-10-06T02:30:15.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.844+0000 7f28fa7fc700 1 -- 172.21.15.31:0/2642009806 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3597434038 0 0) 0x7f28e4003570 con 0x7f28fc13fb20 2023-10-06T02:30:15.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.844+0000 7f28fa7fc700 1 -- 172.21.15.31:0/2642009806 >> v2:172.21.15.62:3300/0 conn(0x7f28fc06aaa0 msgr2=0x7f28fc15a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:15.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.844+0000 7f28fa7fc700 1 --2- 172.21.15.31:0/2642009806 >> v2:172.21.15.62:3300/0 conn(0x7f28fc06aaa0 0x7f28fc15a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:15.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.844+0000 7f28fa7fc700 1 -- 172.21.15.31:0/2642009806 >> v1:172.21.15.62:6789/0 conn(0x7f28fc13fef0 legacy=0x7f28fc0c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:15.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.844+0000 7f28fa7fc700 1 -- 172.21.15.31:0/2642009806 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28fc06b520 con 0x7f28fc13fb20 2023-10-06T02:30:15.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.844+0000 7f28fa7fc700 1 -- 172.21.15.31:0/2642009806 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7f28ec001600 con 0x7f28fc13fb20 2023-10-06T02:30:15.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.844+0000 7f28fa7fc700 1 -- 172.21.15.31:0/2642009806 >> v1:172.21.15.31:6789/0 conn(0x7f28fc13fb20 legacy=0x7f28fc0c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:15.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.844+0000 7f28fa7fc700 1 --2- 172.21.15.31:0/2642009806 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28e4003a60 0x7f28e4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:15.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.844+0000 7f28fa7fc700 1 --2- 172.21.15.31:0/2642009806 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28e4004810 0x7f28e4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:15.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.844+0000 7f28fa7fc700 1 --2- 172.21.15.31:0/2642009806 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f28e40051c0 0x7f28e400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:15.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.844+0000 7f28fa7fc700 1 -- 172.21.15.31:0/2642009806 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28fc06b240 con 0x7f28e4004810 2023-10-06T02:30:15.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.844+0000 7f28fa7fc700 1 -- 172.21.15.31:0/2642009806 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28e400e2e0 con 0x7f28e4003a60 2023-10-06T02:30:15.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.844+0000 7f28fa7fc700 1 -- 172.21.15.31:0/2642009806 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28e400e4a0 con 0x7f28e40051c0 2023-10-06T02:30:15.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.844+0000 7f28faffd700 1 --2- 172.21.15.31:0/2642009806 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28e4004810 0x7f28e4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:15.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.844+0000 7f28fb7fe700 1 --2- 172.21.15.31:0/2642009806 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28e4003a60 0x7f28e4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:15.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.844+0000 7f28fbfff700 1 --2- 172.21.15.31:0/2642009806 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f28e40051c0 0x7f28e400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:15.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.845+0000 7f28faffd700 1 -- 172.21.15.31:0/2642009806 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f28e40051c0 msgr2=0x7f28e400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:15.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.845+0000 7f28faffd700 1 --2- 172.21.15.31:0/2642009806 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f28e40051c0 0x7f28e400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:15.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.845+0000 7f28faffd700 1 -- 172.21.15.31:0/2642009806 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28e4003a60 msgr2=0x7f28e4003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:15.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.845+0000 7f28faffd700 1 --2- 172.21.15.31:0/2642009806 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28e4003a60 0x7f28e4003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:15.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.845+0000 7f28faffd700 1 -- 172.21.15.31:0/2642009806 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28e4003570 con 0x7f28e4004810 2023-10-06T02:30:15.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.845+0000 7f28fbfff700 1 -- 172.21.15.31:0/2642009806 reap_dead start 2023-10-06T02:30:15.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.845+0000 7f28faffd700 1 --2- 172.21.15.31:0/2642009806 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28e4004810 0x7f28e4004c80 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f28fc0cd200 tx=0x7f28f0007d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e88ba6992ea233e5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:15.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.845+0000 7f28fa7fc700 1 -- 172.21.15.31:0/2642009806 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f0001290 con 0x7f28e4004810 2023-10-06T02:30:15.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.845+0000 7f28fa7fc700 1 -- 172.21.15.31:0/2642009806 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28f0012420 con 0x7f28e4004810 2023-10-06T02:30:15.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.845+0000 7f28fa7fc700 1 -- 172.21.15.31:0/2642009806 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f00127e0 con 0x7f28e4004810 2023-10-06T02:30:15.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.846+0000 7f2900d9d700 1 -- 172.21.15.31:0/2642009806 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28e4004810 msgr2=0x7f28e4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:15.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.846+0000 7f2900d9d700 1 --2- 172.21.15.31:0/2642009806 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28e4004810 0x7f28e4004c80 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f28fc0cd200 tx=0x7f28f0007d80 comp rx=0 tx=0).stop 2023-10-06T02:30:15.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.846+0000 7f2900d9d700 1 -- 172.21.15.31:0/2642009806 shutdown_connections 2023-10-06T02:30:15.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.846+0000 7f2900d9d700 1 --2- 172.21.15.31:0/2642009806 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28e4004810 0x7f28e4004c80 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:15.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.846+0000 7f2900d9d700 1 -- 172.21.15.31:0/2642009806 >> 172.21.15.31:0/2642009806 conn(0x7f28fc06d170 msgr2=0x7f28fc0cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:15.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.846+0000 7f2900d9d700 1 -- 172.21.15.31:0/2642009806 shutdown_connections 2023-10-06T02:30:15.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.846+0000 7f2900d9d700 1 -- 172.21.15.31:0/2642009806 wait complete. 2023-10-06T02:30:15.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.847+0000 7f2900d9d700 1 Processor -- start 2023-10-06T02:30:15.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.847+0000 7f2900d9d700 1 -- start start 2023-10-06T02:30:15.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.848+0000 7f2900d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28e4004810 0x7f28fc0acaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:15.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.848+0000 7f2900d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28fc1389b0 0x7f28fc138e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:15.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.848+0000 7f2900d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f28fc0af1b0 0x7f28fc0af620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:15.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.848+0000 7f2900d9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28fc0ab7f0 con 0x7f28fc1389b0 2023-10-06T02:30:15.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.848+0000 7f2900d9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28fc0ac610 con 0x7f28e4004810 2023-10-06T02:30:15.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.848+0000 7f2900d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28fc0acfe0 con 0x7f28fc0af1b0 2023-10-06T02:30:15.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.848+0000 7f28faffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28fc1389b0 0x7f28fc138e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:15.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.848+0000 7f28faffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28fc1389b0 0x7f28fc138e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39210/0 (socket says 172.21.15.31:39210) 2023-10-06T02:30:15.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.848+0000 7f28faffd700 1 -- 172.21.15.31:0/368154499 learned_addr learned my addr 172.21.15.31:0/368154499 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:15.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.849+0000 7f28fb7fe700 1 --2- 172.21.15.31:0/368154499 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28e4004810 0x7f28fc0acaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:15.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.849+0000 7f28fbfff700 1 --2- 172.21.15.31:0/368154499 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f28fc0af1b0 0x7f28fc0af620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:15.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.849+0000 7f28faffd700 1 -- 172.21.15.31:0/368154499 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f28fc0af1b0 msgr2=0x7f28fc0af620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:15.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.849+0000 7f28faffd700 1 --2- 172.21.15.31:0/368154499 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f28fc0af1b0 0x7f28fc0af620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:15.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.849+0000 7f28faffd700 1 -- 172.21.15.31:0/368154499 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28e4004810 msgr2=0x7f28fc0acaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:15.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.849+0000 7f28faffd700 1 --2- 172.21.15.31:0/368154499 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28e4004810 0x7f28fc0acaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:15.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.849+0000 7f28faffd700 1 -- 172.21.15.31:0/368154499 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28fc139360 con 0x7f28fc1389b0 2023-10-06T02:30:15.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.849+0000 7f28faffd700 1 --2- 172.21.15.31:0/368154499 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28fc1389b0 0x7f28fc138e20 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f28fc068320 tx=0x7f28f0015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:15.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.849+0000 7f28e37fe700 1 -- 172.21.15.31:0/368154499 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f001e610 con 0x7f28fc1389b0 2023-10-06T02:30:15.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.849+0000 7f28e37fe700 1 -- 172.21.15.31:0/368154499 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28f001e770 con 0x7f28fc1389b0 2023-10-06T02:30:15.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.849+0000 7f2900d9d700 1 -- 172.21.15.31:0/368154499 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28fc0aba00 con 0x7f28fc1389b0 2023-10-06T02:30:15.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.849+0000 7f28e37fe700 1 -- 172.21.15.31:0/368154499 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f001eb10 con 0x7f28fc1389b0 2023-10-06T02:30:15.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.850+0000 7f2900d9d700 1 -- 172.21.15.31:0/368154499 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28fc0abfc0 con 0x7f28fc1389b0 2023-10-06T02:30:15.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.851+0000 7f2900d9d700 1 -- 172.21.15.31:0/368154499 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28fc0ac780 con 0x7f28fc1389b0 2023-10-06T02:30:15.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.852+0000 7f28e37fe700 1 -- 172.21.15.31:0/368154499 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f28f0019030 con 0x7f28fc1389b0 2023-10-06T02:30:15.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.853+0000 7f28e37fe700 1 --2- 172.21.15.31:0/368154499 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f28e80674d0 0x7f28e8069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:15.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.853+0000 7f28e37fe700 1 -- 172.21.15.31:0/368154499 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f28f0086570 con 0x7f28fc1389b0 2023-10-06T02:30:15.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.853+0000 7f28fb7fe700 1 --2- 172.21.15.31:0/368154499 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f28e80674d0 0x7f28e8069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:15.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.854+0000 7f28fb7fe700 1 --2- 172.21.15.31:0/368154499 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f28e80674d0 0x7f28e8069980 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f28ec001050 tx=0x7f28ec007a50 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:15.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:15.857+0000 7f28e37fe700 1 -- 172.21.15.31:0/368154499 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f28f0052df0 con 0x7f28fc1389b0 2023-10-06T02:30:16.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.064+0000 7f2900d9d700 1 -- 172.21.15.31:0/368154499 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f28fc04c3c0 con 0x7f28fc1389b0 2023-10-06T02:30:16.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.065+0000 7f28e37fe700 1 -- 172.21.15.31:0/368154499 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v132) v1 ==== 74+0+16876 (secure 0 0 0) 0x7f28f0056410 con 0x7f28fc1389b0 2023-10-06T02:30:16.065 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:30:16.065 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":132,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:30:13.138416+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:30:09.791433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-10-06T02:30:10.601699+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":20,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:30:16.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.067+0000 7f2900d9d700 1 -- 172.21.15.31:0/368154499 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f28e80674d0 msgr2=0x7f28e8069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:16.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.067+0000 7f2900d9d700 1 --2- 172.21.15.31:0/368154499 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f28e80674d0 0x7f28e8069980 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f28ec001050 tx=0x7f28ec007a50 comp rx=0 tx=0).stop 2023-10-06T02:30:16.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.067+0000 7f2900d9d700 1 -- 172.21.15.31:0/368154499 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28fc1389b0 msgr2=0x7f28fc138e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:16.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.067+0000 7f2900d9d700 1 --2- 172.21.15.31:0/368154499 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28fc1389b0 0x7f28fc138e20 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f28fc068320 tx=0x7f28f0015000 comp rx=0 tx=0).stop 2023-10-06T02:30:16.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.067+0000 7f2900d9d700 1 -- 172.21.15.31:0/368154499 shutdown_connections 2023-10-06T02:30:16.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.067+0000 7f2900d9d700 1 --2- 172.21.15.31:0/368154499 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f28e80674d0 0x7f28e8069980 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:16.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.067+0000 7f2900d9d700 1 --2- 172.21.15.31:0/368154499 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f28fc0af1b0 0x7f28fc0af620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:16.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.067+0000 7f2900d9d700 1 --2- 172.21.15.31:0/368154499 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28e4004810 0x7f28fc0acaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:16.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.067+0000 7f2900d9d700 1 --2- 172.21.15.31:0/368154499 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28fc1389b0 0x7f28fc138e20 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:16.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.067+0000 7f2900d9d700 1 -- 172.21.15.31:0/368154499 >> 172.21.15.31:0/368154499 conn(0x7f28fc06d170 msgr2=0x7f28fc0c2360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:16.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.067+0000 7f2900d9d700 1 -- 172.21.15.31:0/368154499 shutdown_connections 2023-10-06T02:30:16.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.067+0000 7f2900d9d700 1 -- 172.21.15.31:0/368154499 wait complete. 2023-10-06T02:30:16.085 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T02:30:16.086 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T02:30:16.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.273+0000 7f7e5919d700 1 Processor -- start 2023-10-06T02:30:16.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.273+0000 7f7e5919d700 1 -- start start 2023-10-06T02:30:16.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.274+0000 7f7e5919d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7e54153c30 0x7f7e5415a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:16.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.274+0000 7f7e5919d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7e5415de50 0x7f7e540cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:16.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.274+0000 7f7e5919d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e5415afd0 con 0x7f7e540cef40 2023-10-06T02:30:16.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.274+0000 7f7e5919d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e5415b2b0 con 0x7f7e5415de50 2023-10-06T02:30:16.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.274+0000 7f7e5919d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5415b590 con 0x7f7e54153c30 2023-10-06T02:30:16.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.275+0000 7f7e53fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7e54153c30 0x7f7e5415a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:16.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.275+0000 7f7e537fe700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7e5415de50 0x7f7e540cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:16.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.275+0000 7f7e537fe700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7e5415de50 0x7f7e540cea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:43768/0 (socket says 172.21.15.31:43768) 2023-10-06T02:30:16.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.275+0000 7f7e53fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7e54153c30 0x7f7e5415a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:40898/0 (socket says 172.21.15.31:40898) 2023-10-06T02:30:16.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.275+0000 7f7e537fe700 1 -- 172.21.15.31:0/1304352118 learned_addr learned my addr 172.21.15.31:0/1304352118 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:16.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.275+0000 7f7e53fff700 1 -- 172.21.15.31:0/1304352118 >> v2:172.21.15.62:3300/0 conn(0x7f7e5415de50 msgr2=0x7f7e540cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:16.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.275+0000 7f7e53fff700 1 --2- 172.21.15.31:0/1304352118 >> v2:172.21.15.62:3300/0 conn(0x7f7e5415de50 0x7f7e540cea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:16.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.275+0000 7f7e53fff700 1 -- 172.21.15.31:0/1304352118 >> v1:172.21.15.62:6789/0 conn(0x7f7e540cef40 legacy=0x7f7e5406d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:16.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.275+0000 7f7e53fff700 1 -- 172.21.15.31:0/1304352118 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e5415b870 con 0x7f7e54153c30 2023-10-06T02:30:16.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.275+0000 7f7e537fe700 1 --2- 172.21.15.31:0/1304352118 >> v2:172.21.15.62:3300/0 conn(0x7f7e5415de50 0x7f7e540cea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:30:16.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.275+0000 7f7e53fff700 1 --2- 172.21.15.31:0/1304352118 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7e54153c30 0x7f7e5415a5e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7e440098e0 tx=0x7f7e440091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e441a6b8134e1575 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:16.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.276+0000 7f7e52ffd700 1 -- 172.21.15.31:0/1304352118 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e4400b640 con 0x7f7e54153c30 2023-10-06T02:30:16.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.276+0000 7f7e52ffd700 1 -- 172.21.15.31:0/1304352118 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e4400b7a0 con 0x7f7e54153c30 2023-10-06T02:30:16.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.276+0000 7f7e52ffd700 1 -- 172.21.15.31:0/1304352118 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e4400bb40 con 0x7f7e54153c30 2023-10-06T02:30:16.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.276+0000 7f7e5919d700 1 -- 172.21.15.31:0/1304352118 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7e54153c30 msgr2=0x7f7e5415a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:16.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.276+0000 7f7e5919d700 1 --2- 172.21.15.31:0/1304352118 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7e54153c30 0x7f7e5415a5e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7e440098e0 tx=0x7f7e440091a0 comp rx=0 tx=0).stop 2023-10-06T02:30:16.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.276+0000 7f7e5919d700 1 -- 172.21.15.31:0/1304352118 shutdown_connections 2023-10-06T02:30:16.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.276+0000 7f7e5919d700 1 --2- 172.21.15.31:0/1304352118 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7e54153c30 0x7f7e5415a5e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:16.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.276+0000 7f7e5919d700 1 --2- 172.21.15.31:0/1304352118 >> v2:172.21.15.62:3300/0 conn(0x7f7e5415de50 0x7f7e540cea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:16.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.276+0000 7f7e5919d700 1 -- 172.21.15.31:0/1304352118 >> 172.21.15.31:0/1304352118 conn(0x7f7e540703a0 msgr2=0x7f7e540707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:16.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.277+0000 7f7e5919d700 1 -- 172.21.15.31:0/1304352118 shutdown_connections 2023-10-06T02:30:16.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.277+0000 7f7e5919d700 1 -- 172.21.15.31:0/1304352118 wait complete. 2023-10-06T02:30:16.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.278+0000 7f7e5919d700 1 Processor -- start 2023-10-06T02:30:16.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.278+0000 7f7e5919d700 1 -- start start 2023-10-06T02:30:16.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.279+0000 7f7e5919d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7e540cef40 0x7f7e54169920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:16.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.279+0000 7f7e5919d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e54153c30 0x7f7e54146fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:16.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.279+0000 7f7e5919d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e5415de50 0x7f7e540b4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:16.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.279+0000 7f7e5919d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e540c7f10 con 0x7f7e54153c30 2023-10-06T02:30:16.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.279+0000 7f7e5919d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e540b5590 con 0x7f7e5415de50 2023-10-06T02:30:16.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.279+0000 7f7e5919d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e540c75e0 con 0x7f7e540cef40 2023-10-06T02:30:16.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.279+0000 7f7e537fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e54153c30 0x7f7e54146fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:16.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.280+0000 7f7e537fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e54153c30 0x7f7e54146fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39214/0 (socket says 172.21.15.31:39214) 2023-10-06T02:30:16.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.280+0000 7f7e537fe700 1 -- 172.21.15.31:0/3434121397 learned_addr learned my addr 172.21.15.31:0/3434121397 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:16.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.280+0000 7f7e53fff700 1 --2- 172.21.15.31:0/3434121397 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7e540cef40 0x7f7e54169920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:16.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.280+0000 7f7e5899c700 1 --2- 172.21.15.31:0/3434121397 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e5415de50 0x7f7e540b4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:16.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.280+0000 7f7e537fe700 1 -- 172.21.15.31:0/3434121397 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7e540cef40 msgr2=0x7f7e54169920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:16.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.280+0000 7f7e537fe700 1 --2- 172.21.15.31:0/3434121397 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7e540cef40 0x7f7e54169920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:16.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.280+0000 7f7e537fe700 1 -- 172.21.15.31:0/3434121397 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e5415de50 msgr2=0x7f7e540b4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:16.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.280+0000 7f7e537fe700 1 --2- 172.21.15.31:0/3434121397 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e5415de50 0x7f7e540b4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:16.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.280+0000 7f7e537fe700 1 -- 172.21.15.31:0/3434121397 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e480094f0 con 0x7f7e54153c30 2023-10-06T02:30:16.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.280+0000 7f7e537fe700 1 --2- 172.21.15.31:0/3434121397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e54153c30 0x7f7e54146fc0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f7e4800ec50 tx=0x7f7e48009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:16.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.280+0000 7f7e3bfff700 1 -- 172.21.15.31:0/3434121397 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e4800c850 con 0x7f7e54153c30 2023-10-06T02:30:16.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.280+0000 7f7e3bfff700 1 -- 172.21.15.31:0/3434121397 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e4800c9b0 con 0x7f7e54153c30 2023-10-06T02:30:16.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.280+0000 7f7e3bfff700 1 -- 172.21.15.31:0/3434121397 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e4800cd50 con 0x7f7e54153c30 2023-10-06T02:30:16.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.280+0000 7f7e5919d700 1 -- 172.21.15.31:0/3434121397 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e44009420 con 0x7f7e54153c30 2023-10-06T02:30:16.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.281+0000 7f7e5919d700 1 -- 172.21.15.31:0/3434121397 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e540c7ba0 con 0x7f7e54153c30 2023-10-06T02:30:16.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.282+0000 7f7e5919d700 1 -- 172.21.15.31:0/3434121397 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e540c8080 con 0x7f7e54153c30 2023-10-06T02:30:16.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.283+0000 7f7e3bfff700 1 -- 172.21.15.31:0/3434121397 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f7e48013680 con 0x7f7e54153c30 2023-10-06T02:30:16.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.284+0000 7f7e3bfff700 1 --2- 172.21.15.31:0/3434121397 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7e400674d0 0x7f7e40069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:16.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.284+0000 7f7e3bfff700 1 -- 172.21.15.31:0/3434121397 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f7e480851e0 con 0x7f7e54153c30 2023-10-06T02:30:16.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.284+0000 7f7e53fff700 1 --2- 172.21.15.31:0/3434121397 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7e400674d0 0x7f7e40069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:16.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.285+0000 7f7e53fff700 1 --2- 172.21.15.31:0/3434121397 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7e400674d0 0x7f7e40069980 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7e440093f0 tx=0x7f7e44017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:16.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.288+0000 7f7e3bfff700 1 -- 172.21.15.31:0/3434121397 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7e48051a20 con 0x7f7e54153c30 2023-10-06T02:30:16.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.483+0000 7f7e5919d700 1 -- 172.21.15.31:0/3434121397 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7e5404c3c0 con 0x7f7e400674d0 2023-10-06T02:30:16.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.494+0000 7f7e3bfff700 1 -- 172.21.15.31:0/3434121397 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311315 (secure 0 0 0) 0x7f7e5404c3c0 con 0x7f7e400674d0 2023-10-06T02:30:16.498 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:30:16.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.498+0000 7f7e5919d700 1 -- 172.21.15.31:0/3434121397 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7e400674d0 msgr2=0x7f7e40069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:16.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.498+0000 7f7e5919d700 1 --2- 172.21.15.31:0/3434121397 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7e400674d0 0x7f7e40069980 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7e440093f0 tx=0x7f7e44017000 comp rx=0 tx=0).stop 2023-10-06T02:30:16.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.498+0000 7f7e5919d700 1 -- 172.21.15.31:0/3434121397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e54153c30 msgr2=0x7f7e54146fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:16.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.498+0000 7f7e5919d700 1 --2- 172.21.15.31:0/3434121397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e54153c30 0x7f7e54146fc0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f7e4800ec50 tx=0x7f7e48009250 comp rx=0 tx=0).stop 2023-10-06T02:30:16.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.498+0000 7f7e5919d700 1 -- 172.21.15.31:0/3434121397 shutdown_connections 2023-10-06T02:30:16.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.498+0000 7f7e5919d700 1 --2- 172.21.15.31:0/3434121397 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e5415de50 0x7f7e540b4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:16.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.498+0000 7f7e5919d700 1 --2- 172.21.15.31:0/3434121397 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7e400674d0 0x7f7e40069980 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:16.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.498+0000 7f7e5919d700 1 --2- 172.21.15.31:0/3434121397 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7e540cef40 0x7f7e54169920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:16.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.498+0000 7f7e5919d700 1 --2- 172.21.15.31:0/3434121397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e54153c30 0x7f7e54146fc0 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:16.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.498+0000 7f7e5919d700 1 -- 172.21.15.31:0/3434121397 >> 172.21.15.31:0/3434121397 conn(0x7f7e540703a0 msgr2=0x7f7e5415bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:16.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.498+0000 7f7e5919d700 1 -- 172.21.15.31:0/3434121397 shutdown_connections 2023-10-06T02:30:16.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.498+0000 7f7e5919d700 1 -- 172.21.15.31:0/3434121397 wait complete. 2023-10-06T02:30:16.502 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-10-06T02:30:16.516 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":138,"stamp":"2023-10-06T02:30:14.491582+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":223,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":166,"ondisk_log_size":166,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":143,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2425224,"kb_used_data":14152,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752549496,"statfs":{"total":773094113280,"available":770610683904,"internally_reserved":0,"allocated":14491648,"data_stored":7348844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.366025"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.149042+0000","last_change":"2023-10-06T02:30:12.144840+0000","last_active":"2023-10-06T02:30:12.149042+0000","last_peered":"2023-10-06T02:30:12.149042+0000","last_clean":"2023-10-06T02:30:12.149042+0000","last_became_active":"2023-10-06T02:30:12.144454+0000","last_became_peered":"2023-10-06T02:30:12.144454+0000","last_unstale":"2023-10-06T02:30:12.149042+0000","last_undegraded":"2023-10-06T02:30:12.149042+0000","last_fullsized":"2023-10-06T02:30:12.149042+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:08:48.276730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.156080+0000","last_change":"2023-10-06T02:30:12.148461+0000","last_active":"2023-10-06T02:30:12.156080+0000","last_peered":"2023-10-06T02:30:12.156080+0000","last_clean":"2023-10-06T02:30:12.156080+0000","last_became_active":"2023-10-06T02:30:12.148182+0000","last_became_peered":"2023-10-06T02:30:12.148182+0000","last_unstale":"2023-10-06T02:30:12.156080+0000","last_undegraded":"2023-10-06T02:30:12.156080+0000","last_fullsized":"2023-10-06T02:30:12.156080+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:22:43.565715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.163875+0000","last_change":"2023-10-06T02:30:12.159696+0000","last_active":"2023-10-06T02:30:12.163875+0000","last_peered":"2023-10-06T02:30:12.163875+0000","last_clean":"2023-10-06T02:30:12.163875+0000","last_became_active":"2023-10-06T02:30:12.159416+0000","last_became_peered":"2023-10-06T02:30:12.159416+0000","last_unstale":"2023-10-06T02:30:12.163875+0000","last_undegraded":"2023-10-06T02:30:12.163875+0000","last_fullsized":"2023-10-06T02:30:12.163875+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:41:55.119067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.159437+0000","last_change":"2023-10-06T02:30:12.156128+0000","last_active":"2023-10-06T02:30:12.159437+0000","last_peered":"2023-10-06T02:30:12.159437+0000","last_clean":"2023-10-06T02:30:12.159437+0000","last_became_active":"2023-10-06T02:30:12.155703+0000","last_became_peered":"2023-10-06T02:30:12.155703+0000","last_unstale":"2023-10-06T02:30:12.159437+0000","last_undegraded":"2023-10-06T02:30:12.159437+0000","last_fullsized":"2023-10-06T02:30:12.159437+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:20.790424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.145182+0000","last_change":"2023-10-06T02:30:12.138126+0000","last_active":"2023-10-06T02:30:12.145182+0000","last_peered":"2023-10-06T02:30:12.145182+0000","last_clean":"2023-10-06T02:30:12.145182+0000","last_became_active":"2023-10-06T02:30:12.137801+0000","last_became_peered":"2023-10-06T02:30:12.137801+0000","last_unstale":"2023-10-06T02:30:12.145182+0000","last_undegraded":"2023-10-06T02:30:12.145182+0000","last_fullsized":"2023-10-06T02:30:12.145182+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:29:37.190584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.145225+0000","last_change":"2023-10-06T02:30:12.138750+0000","last_active":"2023-10-06T02:30:12.145225+0000","last_peered":"2023-10-06T02:30:12.145225+0000","last_clean":"2023-10-06T02:30:12.145225+0000","last_became_active":"2023-10-06T02:30:12.138398+0000","last_became_peered":"2023-10-06T02:30:12.138398+0000","last_unstale":"2023-10-06T02:30:12.145225+0000","last_undegraded":"2023-10-06T02:30:12.145225+0000","last_fullsized":"2023-10-06T02:30:12.145225+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:12:51.002294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.156930+0000","last_change":"2023-10-06T02:30:12.155555+0000","last_active":"2023-10-06T02:30:12.156930+0000","last_peered":"2023-10-06T02:30:12.156930+0000","last_clean":"2023-10-06T02:30:12.156930+0000","last_became_active":"2023-10-06T02:30:12.155287+0000","last_became_peered":"2023-10-06T02:30:12.155287+0000","last_unstale":"2023-10-06T02:30:12.156930+0000","last_undegraded":"2023-10-06T02:30:12.156930+0000","last_fullsized":"2023-10-06T02:30:12.156930+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:33:02.804709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.153470+0000","last_change":"2023-10-06T02:30:12.148777+0000","last_active":"2023-10-06T02:30:12.153470+0000","last_peered":"2023-10-06T02:30:12.153470+0000","last_clean":"2023-10-06T02:30:12.153470+0000","last_became_active":"2023-10-06T02:30:12.148461+0000","last_became_peered":"2023-10-06T02:30:12.148461+0000","last_unstale":"2023-10-06T02:30:12.153470+0000","last_undegraded":"2023-10-06T02:30:12.153470+0000","last_fullsized":"2023-10-06T02:30:12.153470+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:46.828782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.164998+0000","last_change":"2023-10-06T02:30:12.159510+0000","last_active":"2023-10-06T02:30:12.164998+0000","last_peered":"2023-10-06T02:30:12.164998+0000","last_clean":"2023-10-06T02:30:12.164998+0000","last_became_active":"2023-10-06T02:30:12.159185+0000","last_became_peered":"2023-10-06T02:30:12.159185+0000","last_unstale":"2023-10-06T02:30:12.164998+0000","last_undegraded":"2023-10-06T02:30:12.164998+0000","last_fullsized":"2023-10-06T02:30:12.164998+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:39:36.644245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.152624+0000","last_change":"2023-10-06T02:30:12.142557+0000","last_active":"2023-10-06T02:30:12.152624+0000","last_peered":"2023-10-06T02:30:12.152624+0000","last_clean":"2023-10-06T02:30:12.152624+0000","last_became_active":"2023-10-06T02:30:12.141736+0000","last_became_peered":"2023-10-06T02:30:12.141736+0000","last_unstale":"2023-10-06T02:30:12.152624+0000","last_undegraded":"2023-10-06T02:30:12.152624+0000","last_fullsized":"2023-10-06T02:30:12.152624+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:10:46.375670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.162628+0000","last_change":"2023-10-06T02:30:12.161151+0000","last_active":"2023-10-06T02:30:12.162628+0000","last_peered":"2023-10-06T02:30:12.162628+0000","last_clean":"2023-10-06T02:30:12.162628+0000","last_became_active":"2023-10-06T02:30:12.160897+0000","last_became_peered":"2023-10-06T02:30:12.160897+0000","last_unstale":"2023-10-06T02:30:12.162628+0000","last_undegraded":"2023-10-06T02:30:12.162628+0000","last_fullsized":"2023-10-06T02:30:12.162628+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:40:26.757057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.163348+0000","last_change":"2023-10-06T02:30:12.161868+0000","last_active":"2023-10-06T02:30:12.163348+0000","last_peered":"2023-10-06T02:30:12.163348+0000","last_clean":"2023-10-06T02:30:12.163348+0000","last_became_active":"2023-10-06T02:30:12.161586+0000","last_became_peered":"2023-10-06T02:30:12.161586+0000","last_unstale":"2023-10-06T02:30:12.163348+0000","last_undegraded":"2023-10-06T02:30:12.163348+0000","last_fullsized":"2023-10-06T02:30:12.163348+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:47:13.733592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.153445+0000","last_change":"2023-10-06T02:30:12.143405+0000","last_active":"2023-10-06T02:30:12.153445+0000","last_peered":"2023-10-06T02:30:12.153445+0000","last_clean":"2023-10-06T02:30:12.153445+0000","last_became_active":"2023-10-06T02:30:12.142467+0000","last_became_peered":"2023-10-06T02:30:12.142467+0000","last_unstale":"2023-10-06T02:30:12.153445+0000","last_undegraded":"2023-10-06T02:30:12.153445+0000","last_fullsized":"2023-10-06T02:30:12.153445+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:42:36.144572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.162327+0000","last_change":"2023-10-06T02:30:12.159390+0000","last_active":"2023-10-06T02:30:12.162327+0000","last_peered":"2023-10-06T02:30:12.162327+0000","last_clean":"2023-10-06T02:30:12.162327+0000","last_became_active":"2023-10-06T02:30:12.159038+0000","last_became_peered":"2023-10-06T02:30:12.159038+0000","last_unstale":"2023-10-06T02:30:12.162327+0000","last_undegraded":"2023-10-06T02:30:12.162327+0000","last_fullsized":"2023-10-06T02:30:12.162327+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:33:27.094350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.155010+0000","last_change":"2023-10-06T02:30:12.147928+0000","last_active":"2023-10-06T02:30:12.155010+0000","last_peered":"2023-10-06T02:30:12.155010+0000","last_clean":"2023-10-06T02:30:12.155010+0000","last_became_active":"2023-10-06T02:30:12.147595+0000","last_became_peered":"2023-10-06T02:30:12.147595+0000","last_unstale":"2023-10-06T02:30:12.155010+0000","last_undegraded":"2023-10-06T02:30:12.155010+0000","last_fullsized":"2023-10-06T02:30:12.155010+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:02:46.429440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.152655+0000","last_change":"2023-10-06T02:30:12.141268+0000","last_active":"2023-10-06T02:30:12.152655+0000","last_peered":"2023-10-06T02:30:12.152655+0000","last_clean":"2023-10-06T02:30:12.152655+0000","last_became_active":"2023-10-06T02:30:12.141017+0000","last_became_peered":"2023-10-06T02:30:12.141017+0000","last_unstale":"2023-10-06T02:30:12.152655+0000","last_undegraded":"2023-10-06T02:30:12.152655+0000","last_fullsized":"2023-10-06T02:30:12.152655+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:02:17.503754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.158851+0000","last_change":"2023-10-06T02:30:12.153384+0000","last_active":"2023-10-06T02:30:12.158851+0000","last_peered":"2023-10-06T02:30:12.158851+0000","last_clean":"2023-10-06T02:30:12.158851+0000","last_became_active":"2023-10-06T02:30:12.153067+0000","last_became_peered":"2023-10-06T02:30:12.153067+0000","last_unstale":"2023-10-06T02:30:12.158851+0000","last_undegraded":"2023-10-06T02:30:12.158851+0000","last_fullsized":"2023-10-06T02:30:12.158851+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:25:27.483151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.154754+0000","last_change":"2023-10-06T02:30:12.144337+0000","last_active":"2023-10-06T02:30:12.154754+0000","last_peered":"2023-10-06T02:30:12.154754+0000","last_clean":"2023-10-06T02:30:12.154754+0000","last_became_active":"2023-10-06T02:30:12.144068+0000","last_became_peered":"2023-10-06T02:30:12.144068+0000","last_unstale":"2023-10-06T02:30:12.154754+0000","last_undegraded":"2023-10-06T02:30:12.154754+0000","last_fullsized":"2023-10-06T02:30:12.154754+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:08:30.193069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.166823+0000","last_change":"2023-10-06T02:30:12.163812+0000","last_active":"2023-10-06T02:30:12.166823+0000","last_peered":"2023-10-06T02:30:12.166823+0000","last_clean":"2023-10-06T02:30:12.166823+0000","last_became_active":"2023-10-06T02:30:12.162912+0000","last_became_peered":"2023-10-06T02:30:12.162912+0000","last_unstale":"2023-10-06T02:30:12.166823+0000","last_undegraded":"2023-10-06T02:30:12.166823+0000","last_fullsized":"2023-10-06T02:30:12.166823+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:19:15.502284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.152518+0000","last_change":"2023-10-06T02:30:12.150200+0000","last_active":"2023-10-06T02:30:12.152518+0000","last_peered":"2023-10-06T02:30:12.152518+0000","last_clean":"2023-10-06T02:30:12.152518+0000","last_became_active":"2023-10-06T02:30:12.149884+0000","last_became_peered":"2023-10-06T02:30:12.149884+0000","last_unstale":"2023-10-06T02:30:12.152518+0000","last_undegraded":"2023-10-06T02:30:12.152518+0000","last_fullsized":"2023-10-06T02:30:12.152518+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:49:23.533957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.164863+0000","last_change":"2023-10-06T02:30:12.159653+0000","last_active":"2023-10-06T02:30:12.164863+0000","last_peered":"2023-10-06T02:30:12.164863+0000","last_clean":"2023-10-06T02:30:12.164863+0000","last_became_active":"2023-10-06T02:30:12.159195+0000","last_became_peered":"2023-10-06T02:30:12.159195+0000","last_unstale":"2023-10-06T02:30:12.164863+0000","last_undegraded":"2023-10-06T02:30:12.164863+0000","last_fullsized":"2023-10-06T02:30:12.164863+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:48:59.244317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.155451+0000","last_change":"2023-10-06T02:30:12.154533+0000","last_active":"2023-10-06T02:30:12.155451+0000","last_peered":"2023-10-06T02:30:12.155451+0000","last_clean":"2023-10-06T02:30:12.155451+0000","last_became_active":"2023-10-06T02:30:12.154252+0000","last_became_peered":"2023-10-06T02:30:12.154252+0000","last_unstale":"2023-10-06T02:30:12.155451+0000","last_undegraded":"2023-10-06T02:30:12.155451+0000","last_fullsized":"2023-10-06T02:30:12.155451+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:25:44.270467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.152557+0000","last_change":"2023-10-06T02:30:12.147146+0000","last_active":"2023-10-06T02:30:12.152557+0000","last_peered":"2023-10-06T02:30:12.152557+0000","last_clean":"2023-10-06T02:30:12.152557+0000","last_became_active":"2023-10-06T02:30:12.146880+0000","last_became_peered":"2023-10-06T02:30:12.146880+0000","last_unstale":"2023-10-06T02:30:12.152557+0000","last_undegraded":"2023-10-06T02:30:12.152557+0000","last_fullsized":"2023-10-06T02:30:12.152557+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:16:11.270014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.159832+0000","last_change":"2023-10-06T02:30:12.158538+0000","last_active":"2023-10-06T02:30:12.159832+0000","last_peered":"2023-10-06T02:30:12.159832+0000","last_clean":"2023-10-06T02:30:12.159832+0000","last_became_active":"2023-10-06T02:30:12.158175+0000","last_became_peered":"2023-10-06T02:30:12.158175+0000","last_unstale":"2023-10-06T02:30:12.159832+0000","last_undegraded":"2023-10-06T02:30:12.159832+0000","last_fullsized":"2023-10-06T02:30:12.159832+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:24.831338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.147982+0000","last_change":"2023-10-06T02:30:12.143290+0000","last_active":"2023-10-06T02:30:12.147982+0000","last_peered":"2023-10-06T02:30:12.147982+0000","last_clean":"2023-10-06T02:30:12.147982+0000","last_became_active":"2023-10-06T02:30:12.142811+0000","last_became_peered":"2023-10-06T02:30:12.142811+0000","last_unstale":"2023-10-06T02:30:12.147982+0000","last_undegraded":"2023-10-06T02:30:12.147982+0000","last_fullsized":"2023-10-06T02:30:12.147982+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:15:45.887824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.144456+0000","last_change":"2023-10-06T02:30:12.142626+0000","last_active":"2023-10-06T02:30:12.144456+0000","last_peered":"2023-10-06T02:30:12.144456+0000","last_clean":"2023-10-06T02:30:12.144456+0000","last_became_active":"2023-10-06T02:30:12.142327+0000","last_became_peered":"2023-10-06T02:30:12.142327+0000","last_unstale":"2023-10-06T02:30:12.144456+0000","last_undegraded":"2023-10-06T02:30:12.144456+0000","last_fullsized":"2023-10-06T02:30:12.144456+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:40:09.048569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.164955+0000","last_change":"2023-10-06T02:30:12.162084+0000","last_active":"2023-10-06T02:30:12.164955+0000","last_peered":"2023-10-06T02:30:12.164955+0000","last_clean":"2023-10-06T02:30:12.164955+0000","last_became_active":"2023-10-06T02:30:12.160260+0000","last_became_peered":"2023-10-06T02:30:12.160260+0000","last_unstale":"2023-10-06T02:30:12.164955+0000","last_undegraded":"2023-10-06T02:30:12.164955+0000","last_fullsized":"2023-10-06T02:30:12.164955+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:19:39.260439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.156466+0000","last_change":"2023-10-06T02:30:12.146173+0000","last_active":"2023-10-06T02:30:12.156466+0000","last_peered":"2023-10-06T02:30:12.156466+0000","last_clean":"2023-10-06T02:30:12.156466+0000","last_became_active":"2023-10-06T02:30:12.145862+0000","last_became_peered":"2023-10-06T02:30:12.145862+0000","last_unstale":"2023-10-06T02:30:12.156466+0000","last_undegraded":"2023-10-06T02:30:12.156466+0000","last_fullsized":"2023-10-06T02:30:12.156466+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:35.261516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.159429+0000","last_change":"2023-10-06T02:30:12.155658+0000","last_active":"2023-10-06T02:30:12.159429+0000","last_peered":"2023-10-06T02:30:12.159429+0000","last_clean":"2023-10-06T02:30:12.159429+0000","last_became_active":"2023-10-06T02:30:12.154950+0000","last_became_peered":"2023-10-06T02:30:12.154950+0000","last_unstale":"2023-10-06T02:30:12.159429+0000","last_undegraded":"2023-10-06T02:30:12.159429+0000","last_fullsized":"2023-10-06T02:30:12.159429+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:08:48.276730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.156351+0000","last_change":"2023-10-06T02:30:12.146097+0000","last_active":"2023-10-06T02:30:12.156351+0000","last_peered":"2023-10-06T02:30:12.156351+0000","last_clean":"2023-10-06T02:30:12.156351+0000","last_became_active":"2023-10-06T02:30:12.145821+0000","last_became_peered":"2023-10-06T02:30:12.145821+0000","last_unstale":"2023-10-06T02:30:12.156351+0000","last_undegraded":"2023-10-06T02:30:12.156351+0000","last_fullsized":"2023-10-06T02:30:12.156351+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:01:54.944524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.152765+0000","last_change":"2023-10-06T02:30:12.140575+0000","last_active":"2023-10-06T02:30:12.152765+0000","last_peered":"2023-10-06T02:30:12.152765+0000","last_clean":"2023-10-06T02:30:12.152765+0000","last_became_active":"2023-10-06T02:30:12.140288+0000","last_became_peered":"2023-10-06T02:30:12.140288+0000","last_unstale":"2023-10-06T02:30:12.152765+0000","last_undegraded":"2023-10-06T02:30:12.152765+0000","last_fullsized":"2023-10-06T02:30:12.152765+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:56:35.993001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.157442+0000","last_change":"2023-10-06T02:30:12.149331+0000","last_active":"2023-10-06T02:30:12.157442+0000","last_peered":"2023-10-06T02:30:12.157442+0000","last_clean":"2023-10-06T02:30:12.157442+0000","last_became_active":"2023-10-06T02:30:12.147293+0000","last_became_peered":"2023-10-06T02:30:12.147293+0000","last_unstale":"2023-10-06T02:30:12.157442+0000","last_undegraded":"2023-10-06T02:30:12.157442+0000","last_fullsized":"2023-10-06T02:30:12.157442+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:36:59.376547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.161039+0000","last_change":"2023-10-06T02:30:12.158156+0000","last_active":"2023-10-06T02:30:12.161039+0000","last_peered":"2023-10-06T02:30:12.161039+0000","last_clean":"2023-10-06T02:30:12.161039+0000","last_became_active":"2023-10-06T02:30:12.157856+0000","last_became_peered":"2023-10-06T02:30:12.157856+0000","last_unstale":"2023-10-06T02:30:12.161039+0000","last_undegraded":"2023-10-06T02:30:12.161039+0000","last_fullsized":"2023-10-06T02:30:12.161039+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:09.847602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"131'4","reported_seq":14,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.167766+0000","last_change":"2023-10-06T02:30:11.143280+0000","last_active":"2023-10-06T02:30:12.167766+0000","last_peered":"2023-10-06T02:30:12.167766+0000","last_clean":"2023-10-06T02:30:12.167766+0000","last_became_active":"2023-10-06T02:30:11.143032+0000","last_became_peered":"2023-10-06T02:30:11.143032+0000","last_unstale":"2023-10-06T02:30:12.167766+0000","last_undegraded":"2023-10-06T02:30:12.167766+0000","last_fullsized":"2023-10-06T02:30:12.167766+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_clean_scrub_stamp":"2023-10-06T02:30:10.108419+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:50:30.974609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.155528+0000","last_change":"2023-10-06T02:30:12.149152+0000","last_active":"2023-10-06T02:30:12.155528+0000","last_peered":"2023-10-06T02:30:12.155528+0000","last_clean":"2023-10-06T02:30:12.155528+0000","last_became_active":"2023-10-06T02:30:12.148787+0000","last_became_peered":"2023-10-06T02:30:12.148787+0000","last_unstale":"2023-10-06T02:30:12.155528+0000","last_undegraded":"2023-10-06T02:30:12.155528+0000","last_fullsized":"2023-10-06T02:30:12.155528+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:19:22.533732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1e","version":"131'1","reported_seq":11,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.169833+0000","last_change":"2023-10-06T02:30:11.156964+0000","last_active":"2023-10-06T02:30:12.169833+0000","last_peered":"2023-10-06T02:30:12.169833+0000","last_clean":"2023-10-06T02:30:12.169833+0000","last_became_active":"2023-10-06T02:30:11.156574+0000","last_became_peered":"2023-10-06T02:30:11.156574+0000","last_unstale":"2023-10-06T02:30:12.169833+0000","last_undegraded":"2023-10-06T02:30:12.169833+0000","last_fullsized":"2023-10-06T02:30:12.169833+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_clean_scrub_stamp":"2023-10-06T02:30:10.108419+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:50:30.974609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.152601+0000","last_change":"2023-10-06T02:30:12.145807+0000","last_active":"2023-10-06T02:30:12.152601+0000","last_peered":"2023-10-06T02:30:12.152601+0000","last_clean":"2023-10-06T02:30:12.152601+0000","last_became_active":"2023-10-06T02:30:12.145534+0000","last_became_peered":"2023-10-06T02:30:12.145534+0000","last_unstale":"2023-10-06T02:30:12.152601+0000","last_undegraded":"2023-10-06T02:30:12.152601+0000","last_fullsized":"2023-10-06T02:30:12.152601+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:31:03.185895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"131'2","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.170087+0000","last_change":"2023-10-06T02:30:11.156936+0000","last_active":"2023-10-06T02:30:12.170087+0000","last_peered":"2023-10-06T02:30:12.170087+0000","last_clean":"2023-10-06T02:30:12.170087+0000","last_became_active":"2023-10-06T02:30:11.156581+0000","last_became_peered":"2023-10-06T02:30:11.156581+0000","last_unstale":"2023-10-06T02:30:12.170087+0000","last_undegraded":"2023-10-06T02:30:12.170087+0000","last_fullsized":"2023-10-06T02:30:12.170087+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_clean_scrub_stamp":"2023-10-06T02:30:10.108419+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:25:43.260463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.162657+0000","last_change":"2023-10-06T02:30:12.157805+0000","last_active":"2023-10-06T02:30:12.162657+0000","last_peered":"2023-10-06T02:30:12.162657+0000","last_clean":"2023-10-06T02:30:12.162657+0000","last_became_active":"2023-10-06T02:30:12.157525+0000","last_became_peered":"2023-10-06T02:30:12.157525+0000","last_unstale":"2023-10-06T02:30:12.162657+0000","last_undegraded":"2023-10-06T02:30:12.162657+0000","last_fullsized":"2023-10-06T02:30:12.162657+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:48:56.362100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"131'2","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.161955+0000","last_change":"2023-10-06T02:30:11.156553+0000","last_active":"2023-10-06T02:30:12.161955+0000","last_peered":"2023-10-06T02:30:12.161955+0000","last_clean":"2023-10-06T02:30:12.161955+0000","last_became_active":"2023-10-06T02:30:11.156286+0000","last_became_peered":"2023-10-06T02:30:11.156286+0000","last_unstale":"2023-10-06T02:30:12.161955+0000","last_undegraded":"2023-10-06T02:30:12.161955+0000","last_fullsized":"2023-10-06T02:30:12.161955+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_clean_scrub_stamp":"2023-10-06T02:30:10.108419+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:19.780420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.149449+0000","last_change":"2023-10-06T02:30:12.141626+0000","last_active":"2023-10-06T02:30:12.149449+0000","last_peered":"2023-10-06T02:30:12.149449+0000","last_clean":"2023-10-06T02:30:12.149449+0000","last_became_active":"2023-10-06T02:30:12.141323+0000","last_became_peered":"2023-10-06T02:30:12.141323+0000","last_unstale":"2023-10-06T02:30:12.149449+0000","last_undegraded":"2023-10-06T02:30:12.149449+0000","last_fullsized":"2023-10-06T02:30:12.149449+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:12:51.002294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"131'1","reported_seq":11,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.162946+0000","last_change":"2023-10-06T02:30:11.147508+0000","last_active":"2023-10-06T02:30:12.162946+0000","last_peered":"2023-10-06T02:30:12.162946+0000","last_clean":"2023-10-06T02:30:12.162946+0000","last_became_active":"2023-10-06T02:30:11.147084+0000","last_became_peered":"2023-10-06T02:30:11.147084+0000","last_unstale":"2023-10-06T02:30:12.162946+0000","last_undegraded":"2023-10-06T02:30:12.162946+0000","last_fullsized":"2023-10-06T02:30:12.162946+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_clean_scrub_stamp":"2023-10-06T02:30:10.108419+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:39:22.980561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.159569+0000","last_change":"2023-10-06T02:30:12.154322+0000","last_active":"2023-10-06T02:30:12.159569+0000","last_peered":"2023-10-06T02:30:12.159569+0000","last_clean":"2023-10-06T02:30:12.159569+0000","last_became_active":"2023-10-06T02:30:12.154020+0000","last_became_peered":"2023-10-06T02:30:12.154020+0000","last_unstale":"2023-10-06T02:30:12.159569+0000","last_undegraded":"2023-10-06T02:30:12.159569+0000","last_fullsized":"2023-10-06T02:30:12.159569+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:34:32.671541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1a","version":"131'1","reported_seq":11,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.166635+0000","last_change":"2023-10-06T02:30:11.146699+0000","last_active":"2023-10-06T02:30:12.166635+0000","last_peered":"2023-10-06T02:30:12.166635+0000","last_clean":"2023-10-06T02:30:12.166635+0000","last_became_active":"2023-10-06T02:30:11.146468+0000","last_became_peered":"2023-10-06T02:30:11.146468+0000","last_unstale":"2023-10-06T02:30:12.166635+0000","last_undegraded":"2023-10-06T02:30:12.166635+0000","last_fullsized":"2023-10-06T02:30:12.166635+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_clean_scrub_stamp":"2023-10-06T02:30:10.108419+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:50:04.409854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.147874+0000","last_change":"2023-10-06T02:30:12.146356+0000","last_active":"2023-10-06T02:30:12.147874+0000","last_peered":"2023-10-06T02:30:12.147874+0000","last_clean":"2023-10-06T02:30:12.147874+0000","last_became_active":"2023-10-06T02:30:12.146093+0000","last_became_peered":"2023-10-06T02:30:12.146093+0000","last_unstale":"2023-10-06T02:30:12.147874+0000","last_undegraded":"2023-10-06T02:30:12.147874+0000","last_fullsized":"2023-10-06T02:30:12.147874+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:26:15.763758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.132982+0000","last_change":"2023-10-06T02:30:11.150906+0000","last_active":"2023-10-06T02:30:12.132982+0000","last_peered":"2023-10-06T02:30:12.132982+0000","last_clean":"2023-10-06T02:30:12.132982+0000","last_became_active":"2023-10-06T02:30:11.150588+0000","last_became_peered":"2023-10-06T02:30:11.150588+0000","last_unstale":"2023-10-06T02:30:12.132982+0000","last_undegraded":"2023-10-06T02:30:12.132982+0000","last_fullsized":"2023-10-06T02:30:12.132982+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_clean_scrub_stamp":"2023-10-06T02:30:10.108419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:49:11.284716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.148546+0000","last_change":"2023-10-06T02:30:12.143757+0000","last_active":"2023-10-06T02:30:12.148546+0000","last_peered":"2023-10-06T02:30:12.148546+0000","last_clean":"2023-10-06T02:30:12.148546+0000","last_became_active":"2023-10-06T02:30:12.143472+0000","last_became_peered":"2023-10-06T02:30:12.143472+0000","last_unstale":"2023-10-06T02:30:12.148546+0000","last_undegraded":"2023-10-06T02:30:12.148546+0000","last_fullsized":"2023-10-06T02:30:12.148546+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:26:19.550623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.129940+0000","last_change":"2023-10-06T02:30:11.153621+0000","last_active":"2023-10-06T02:30:12.129940+0000","last_peered":"2023-10-06T02:30:12.129940+0000","last_clean":"2023-10-06T02:30:12.129940+0000","last_became_active":"2023-10-06T02:30:11.153323+0000","last_became_peered":"2023-10-06T02:30:11.153323+0000","last_unstale":"2023-10-06T02:30:12.129940+0000","last_undegraded":"2023-10-06T02:30:12.129940+0000","last_fullsized":"2023-10-06T02:30:12.129940+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_clean_scrub_stamp":"2023-10-06T02:30:10.108419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:30:04.813580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.170263+0000","last_change":"2023-10-06T02:30:12.167908+0000","last_active":"2023-10-06T02:30:12.170263+0000","last_peered":"2023-10-06T02:30:12.170263+0000","last_clean":"2023-10-06T02:30:12.170263+0000","last_became_active":"2023-10-06T02:30:12.167328+0000","last_became_peered":"2023-10-06T02:30:12.167328+0000","last_unstale":"2023-10-06T02:30:12.170263+0000","last_undegraded":"2023-10-06T02:30:12.170263+0000","last_fullsized":"2023-10-06T02:30:12.170263+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:11:47.949557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"131'1","reported_seq":11,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.162998+0000","last_change":"2023-10-06T02:30:11.148080+0000","last_active":"2023-10-06T02:30:12.162998+0000","last_peered":"2023-10-06T02:30:12.162998+0000","last_clean":"2023-10-06T02:30:12.162998+0000","last_became_active":"2023-10-06T02:30:11.147435+0000","last_became_peered":"2023-10-06T02:30:11.147435+0000","last_unstale":"2023-10-06T02:30:12.162998+0000","last_undegraded":"2023-10-06T02:30:12.162998+0000","last_fullsized":"2023-10-06T02:30:12.162998+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_clean_scrub_stamp":"2023-10-06T02:30:10.108419+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:12:06.291697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.166800+0000","last_change":"2023-10-06T02:30:12.163464+0000","last_active":"2023-10-06T02:30:12.166800+0000","last_peered":"2023-10-06T02:30:12.166800+0000","last_clean":"2023-10-06T02:30:12.166800+0000","last_became_active":"2023-10-06T02:30:12.163180+0000","last_became_peered":"2023-10-06T02:30:12.163180+0000","last_unstale":"2023-10-06T02:30:12.166800+0000","last_undegraded":"2023-10-06T02:30:12.166800+0000","last_fullsized":"2023-10-06T02:30:12.166800+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:58:19.872024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"131'2","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.169584+0000","last_change":"2023-10-06T02:30:11.151792+0000","last_active":"2023-10-06T02:30:12.169584+0000","last_peered":"2023-10-06T02:30:12.169584+0000","last_clean":"2023-10-06T02:30:12.169584+0000","last_became_active":"2023-10-06T02:30:11.151477+0000","last_became_peered":"2023-10-06T02:30:11.151477+0000","last_unstale":"2023-10-06T02:30:12.169584+0000","last_undegraded":"2023-10-06T02:30:12.169584+0000","last_fullsized":"2023-10-06T02:30:12.169584+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_clean_scrub_stamp":"2023-10-06T02:30:10.108419+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:26:18.540619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.149076+0000","last_change":"2023-10-06T02:30:12.145890+0000","last_active":"2023-10-06T02:30:12.149076+0000","last_peered":"2023-10-06T02:30:12.149076+0000","last_clean":"2023-10-06T02:30:12.149076+0000","last_became_active":"2023-10-06T02:30:12.145447+0000","last_became_peered":"2023-10-06T02:30:12.145447+0000","last_unstale":"2023-10-06T02:30:12.149076+0000","last_undegraded":"2023-10-06T02:30:12.149076+0000","last_fullsized":"2023-10-06T02:30:12.149076+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:36:59.376547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"131'1","reported_seq":11,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.161831+0000","last_change":"2023-10-06T02:30:11.149042+0000","last_active":"2023-10-06T02:30:12.161831+0000","last_peered":"2023-10-06T02:30:12.161831+0000","last_clean":"2023-10-06T02:30:12.161831+0000","last_became_active":"2023-10-06T02:30:11.148792+0000","last_became_peered":"2023-10-06T02:30:11.148792+0000","last_unstale":"2023-10-06T02:30:12.161831+0000","last_undegraded":"2023-10-06T02:30:12.161831+0000","last_fullsized":"2023-10-06T02:30:12.161831+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_clean_scrub_stamp":"2023-10-06T02:30:10.108419+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:33:26.084346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.150367+0000","last_change":"2023-10-06T02:30:12.148273+0000","last_active":"2023-10-06T02:30:12.150367+0000","last_peered":"2023-10-06T02:30:12.150367+0000","last_clean":"2023-10-06T02:30:12.150367+0000","last_became_active":"2023-10-06T02:30:12.147817+0000","last_became_peered":"2023-10-06T02:30:12.147817+0000","last_unstale":"2023-10-06T02:30:12.150367+0000","last_undegraded":"2023-10-06T02:30:12.150367+0000","last_fullsized":"2023-10-06T02:30:12.150367+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:46.828782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.138074+0000","last_change":"2023-10-06T02:30:11.153165+0000","last_active":"2023-10-06T02:30:12.138074+0000","last_peered":"2023-10-06T02:30:12.138074+0000","last_clean":"2023-10-06T02:30:12.138074+0000","last_became_active":"2023-10-06T02:30:11.152879+0000","last_became_peered":"2023-10-06T02:30:11.152879+0000","last_unstale":"2023-10-06T02:30:12.138074+0000","last_undegraded":"2023-10-06T02:30:12.138074+0000","last_fullsized":"2023-10-06T02:30:12.138074+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_clean_scrub_stamp":"2023-10-06T02:30:10.108419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:40:25.747053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.162884+0000","last_change":"2023-10-06T02:30:12.159732+0000","last_active":"2023-10-06T02:30:12.162884+0000","last_peered":"2023-10-06T02:30:12.162884+0000","last_clean":"2023-10-06T02:30:12.162884+0000","last_became_active":"2023-10-06T02:30:12.159433+0000","last_became_peered":"2023-10-06T02:30:12.159433+0000","last_unstale":"2023-10-06T02:30:12.162884+0000","last_undegraded":"2023-10-06T02:30:12.162884+0000","last_fullsized":"2023-10-06T02:30:12.162884+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:47:28.496930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"131'2","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.165135+0000","last_change":"2023-10-06T02:30:11.140625+0000","last_active":"2023-10-06T02:30:12.165135+0000","last_peered":"2023-10-06T02:30:12.165135+0000","last_clean":"2023-10-06T02:30:12.165135+0000","last_became_active":"2023-10-06T02:30:11.140327+0000","last_became_peered":"2023-10-06T02:30:11.140327+0000","last_unstale":"2023-10-06T02:30:12.165135+0000","last_undegraded":"2023-10-06T02:30:12.165135+0000","last_fullsized":"2023-10-06T02:30:12.165135+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_clean_scrub_stamp":"2023-10-06T02:30:10.108419+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:59:01.365002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.158258+0000","last_change":"2023-10-06T02:30:12.151174+0000","last_active":"2023-10-06T02:30:12.158258+0000","last_peered":"2023-10-06T02:30:12.158258+0000","last_clean":"2023-10-06T02:30:12.158258+0000","last_became_active":"2023-10-06T02:30:12.150890+0000","last_became_peered":"2023-10-06T02:30:12.150890+0000","last_unstale":"2023-10-06T02:30:12.158258+0000","last_undegraded":"2023-10-06T02:30:12.158258+0000","last_fullsized":"2023-10-06T02:30:12.158258+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:18:54.516532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.138534+0000","last_change":"2023-10-06T02:30:11.148034+0000","last_active":"2023-10-06T02:30:12.138534+0000","last_peered":"2023-10-06T02:30:12.138534+0000","last_clean":"2023-10-06T02:30:12.138534+0000","last_became_active":"2023-10-06T02:30:11.147753+0000","last_became_peered":"2023-10-06T02:30:11.147753+0000","last_unstale":"2023-10-06T02:30:12.138534+0000","last_undegraded":"2023-10-06T02:30:12.138534+0000","last_fullsized":"2023-10-06T02:30:12.138534+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_clean_scrub_stamp":"2023-10-06T02:30:10.108419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:27.763054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.162784+0000","last_change":"2023-10-06T02:30:12.158709+0000","last_active":"2023-10-06T02:30:12.162784+0000","last_peered":"2023-10-06T02:30:12.162784+0000","last_clean":"2023-10-06T02:30:12.162784+0000","last_became_active":"2023-10-06T02:30:12.158441+0000","last_became_peered":"2023-10-06T02:30:12.158441+0000","last_unstale":"2023-10-06T02:30:12.162784+0000","last_undegraded":"2023-10-06T02:30:12.162784+0000","last_fullsized":"2023-10-06T02:30:12.162784+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:37:15.648699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.135106+0000","last_change":"2023-10-06T02:30:11.140734+0000","last_active":"2023-10-06T02:30:12.135106+0000","last_peered":"2023-10-06T02:30:12.135106+0000","last_clean":"2023-10-06T02:30:12.135106+0000","last_became_active":"2023-10-06T02:30:11.140405+0000","last_became_peered":"2023-10-06T02:30:11.140405+0000","last_unstale":"2023-10-06T02:30:12.135106+0000","last_undegraded":"2023-10-06T02:30:12.135106+0000","last_fullsized":"2023-10-06T02:30:12.135106+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_clean_scrub_stamp":"2023-10-06T02:30:10.108419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:02:16.493749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.164550+0000","last_change":"2023-10-06T02:30:12.162226+0000","last_active":"2023-10-06T02:30:12.164550+0000","last_peered":"2023-10-06T02:30:12.164550+0000","last_clean":"2023-10-06T02:30:12.164550+0000","last_became_active":"2023-10-06T02:30:12.161941+0000","last_became_peered":"2023-10-06T02:30:12.161941+0000","last_unstale":"2023-10-06T02:30:12.164550+0000","last_undegraded":"2023-10-06T02:30:12.164550+0000","last_fullsized":"2023-10-06T02:30:12.164550+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:09:06.895098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.138333+0000","last_change":"2023-10-06T02:30:11.146212+0000","last_active":"2023-10-06T02:30:12.138333+0000","last_peered":"2023-10-06T02:30:12.138333+0000","last_clean":"2023-10-06T02:30:12.138333+0000","last_became_active":"2023-10-06T02:30:11.145902+0000","last_became_peered":"2023-10-06T02:30:11.145902+0000","last_unstale":"2023-10-06T02:30:12.138333+0000","last_undegraded":"2023-10-06T02:30:12.138333+0000","last_fullsized":"2023-10-06T02:30:12.138333+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_clean_scrub_stamp":"2023-10-06T02:30:10.108419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:00:07.252457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.145544+0000","last_change":"2023-10-06T02:30:12.139280+0000","last_active":"2023-10-06T02:30:12.145544+0000","last_peered":"2023-10-06T02:30:12.145544+0000","last_clean":"2023-10-06T02:30:12.145544+0000","last_became_active":"2023-10-06T02:30:12.139010+0000","last_became_peered":"2023-10-06T02:30:12.139010+0000","last_unstale":"2023-10-06T02:30:12.145544+0000","last_undegraded":"2023-10-06T02:30:12.145544+0000","last_fullsized":"2023-10-06T02:30:12.145544+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:27:16.232913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.7","version":"131'2","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.163679+0000","last_change":"2023-10-06T02:30:11.153790+0000","last_active":"2023-10-06T02:30:12.163679+0000","last_peered":"2023-10-06T02:30:12.163679+0000","last_clean":"2023-10-06T02:30:12.163679+0000","last_became_active":"2023-10-06T02:30:11.153482+0000","last_became_peered":"2023-10-06T02:30:11.153482+0000","last_unstale":"2023-10-06T02:30:12.163679+0000","last_undegraded":"2023-10-06T02:30:12.163679+0000","last_fullsized":"2023-10-06T02:30:12.163679+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_clean_scrub_stamp":"2023-10-06T02:30:10.108419+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:13:34.004147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.159524+0000","last_change":"2023-10-06T02:30:12.157967+0000","last_active":"2023-10-06T02:30:12.159524+0000","last_peered":"2023-10-06T02:30:12.159524+0000","last_clean":"2023-10-06T02:30:12.159524+0000","last_became_active":"2023-10-06T02:30:12.157656+0000","last_became_peered":"2023-10-06T02:30:12.157656+0000","last_unstale":"2023-10-06T02:30:12.159524+0000","last_undegraded":"2023-10-06T02:30:12.159524+0000","last_fullsized":"2023-10-06T02:30:12.159524+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:40:34.054488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.135582+0000","last_change":"2023-10-06T02:30:11.149211+0000","last_active":"2023-10-06T02:30:12.135582+0000","last_peered":"2023-10-06T02:30:12.135582+0000","last_clean":"2023-10-06T02:30:12.135582+0000","last_became_active":"2023-10-06T02:30:11.148961+0000","last_became_peered":"2023-10-06T02:30:11.148961+0000","last_unstale":"2023-10-06T02:30:12.135582+0000","last_undegraded":"2023-10-06T02:30:12.135582+0000","last_fullsized":"2023-10-06T02:30:12.135582+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_clean_scrub_stamp":"2023-10-06T02:30:10.108419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:39:22.980561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.148989+0000","last_change":"2023-10-06T02:30:12.145707+0000","last_active":"2023-10-06T02:30:12.148989+0000","last_peered":"2023-10-06T02:30:12.148989+0000","last_clean":"2023-10-06T02:30:12.148989+0000","last_became_active":"2023-10-06T02:30:12.145090+0000","last_became_peered":"2023-10-06T02:30:12.145090+0000","last_unstale":"2023-10-06T02:30:12.148989+0000","last_undegraded":"2023-10-06T02:30:12.148989+0000","last_fullsized":"2023-10-06T02:30:12.148989+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:31:03.185895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.140076+0000","last_change":"2023-10-06T02:30:11.154287+0000","last_active":"2023-10-06T02:30:12.140076+0000","last_peered":"2023-10-06T02:30:12.140076+0000","last_clean":"2023-10-06T02:30:12.140076+0000","last_became_active":"2023-10-06T02:30:11.153901+0000","last_became_peered":"2023-10-06T02:30:11.153901+0000","last_unstale":"2023-10-06T02:30:12.140076+0000","last_undegraded":"2023-10-06T02:30:12.140076+0000","last_fullsized":"2023-10-06T02:30:12.140076+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_clean_scrub_stamp":"2023-10-06T02:30:10.108419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:25:26.473147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.163559+0000","last_change":"2023-10-06T02:30:12.156545+0000","last_active":"2023-10-06T02:30:12.163559+0000","last_peered":"2023-10-06T02:30:12.163559+0000","last_clean":"2023-10-06T02:30:12.163559+0000","last_became_active":"2023-10-06T02:30:12.156236+0000","last_became_peered":"2023-10-06T02:30:12.156236+0000","last_unstale":"2023-10-06T02:30:12.163559+0000","last_undegraded":"2023-10-06T02:30:12.163559+0000","last_fullsized":"2023-10-06T02:30:12.163559+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:43:28.212045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.130185+0000","last_change":"2023-10-06T02:30:11.145279+0000","last_active":"2023-10-06T02:30:12.130185+0000","last_peered":"2023-10-06T02:30:12.130185+0000","last_clean":"2023-10-06T02:30:12.130185+0000","last_became_active":"2023-10-06T02:30:11.144903+0000","last_became_peered":"2023-10-06T02:30:11.144903+0000","last_unstale":"2023-10-06T02:30:12.130185+0000","last_undegraded":"2023-10-06T02:30:12.130185+0000","last_fullsized":"2023-10-06T02:30:12.130185+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_clean_scrub_stamp":"2023-10-06T02:30:10.108419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:35:03.975595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.153445+0000","last_change":"2023-10-06T02:30:12.146323+0000","last_active":"2023-10-06T02:30:12.153445+0000","last_peered":"2023-10-06T02:30:12.153445+0000","last_clean":"2023-10-06T02:30:12.153445+0000","last_became_active":"2023-10-06T02:30:12.145996+0000","last_became_peered":"2023-10-06T02:30:12.145996+0000","last_unstale":"2023-10-06T02:30:12.153445+0000","last_undegraded":"2023-10-06T02:30:12.153445+0000","last_fullsized":"2023-10-06T02:30:12.153445+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:41:55.119067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.133902+0000","last_change":"2023-10-06T02:30:11.148696+0000","last_active":"2023-10-06T02:30:12.133902+0000","last_peered":"2023-10-06T02:30:12.133902+0000","last_clean":"2023-10-06T02:30:12.133902+0000","last_became_active":"2023-10-06T02:30:11.148290+0000","last_became_peered":"2023-10-06T02:30:11.148290+0000","last_unstale":"2023-10-06T02:30:12.133902+0000","last_undegraded":"2023-10-06T02:30:12.133902+0000","last_fullsized":"2023-10-06T02:30:12.133902+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_clean_scrub_stamp":"2023-10-06T02:30:10.108419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:00:07.252457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.162145+0000","last_change":"2023-10-06T02:30:12.158638+0000","last_active":"2023-10-06T02:30:12.162145+0000","last_peered":"2023-10-06T02:30:12.162145+0000","last_clean":"2023-10-06T02:30:12.162145+0000","last_became_active":"2023-10-06T02:30:12.158365+0000","last_became_peered":"2023-10-06T02:30:12.158365+0000","last_unstale":"2023-10-06T02:30:12.162145+0000","last_undegraded":"2023-10-06T02:30:12.162145+0000","last_fullsized":"2023-10-06T02:30:12.162145+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:29:35.838267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.137724+0000","last_change":"2023-10-06T02:30:11.150259+0000","last_active":"2023-10-06T02:30:12.137724+0000","last_peered":"2023-10-06T02:30:12.137724+0000","last_clean":"2023-10-06T02:30:12.137724+0000","last_became_active":"2023-10-06T02:30:11.149139+0000","last_became_peered":"2023-10-06T02:30:11.149139+0000","last_unstale":"2023-10-06T02:30:12.137724+0000","last_undegraded":"2023-10-06T02:30:12.137724+0000","last_fullsized":"2023-10-06T02:30:12.137724+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_clean_scrub_stamp":"2023-10-06T02:30:10.108419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:12:27.532273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.149793+0000","last_change":"2023-10-06T02:30:12.147136+0000","last_active":"2023-10-06T02:30:12.149793+0000","last_peered":"2023-10-06T02:30:12.149793+0000","last_clean":"2023-10-06T02:30:12.149793+0000","last_became_active":"2023-10-06T02:30:12.146817+0000","last_became_peered":"2023-10-06T02:30:12.146817+0000","last_unstale":"2023-10-06T02:30:12.149793+0000","last_undegraded":"2023-10-06T02:30:12.149793+0000","last_fullsized":"2023-10-06T02:30:12.149793+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:16:11.270014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.132118+0000","last_change":"2023-10-06T02:30:11.152607+0000","last_active":"2023-10-06T02:30:12.132118+0000","last_peered":"2023-10-06T02:30:12.132118+0000","last_clean":"2023-10-06T02:30:12.132118+0000","last_became_active":"2023-10-06T02:30:11.152343+0000","last_became_peered":"2023-10-06T02:30:11.152343+0000","last_unstale":"2023-10-06T02:30:12.132118+0000","last_undegraded":"2023-10-06T02:30:12.132118+0000","last_fullsized":"2023-10-06T02:30:12.132118+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_clean_scrub_stamp":"2023-10-06T02:30:10.108419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:44:23.481085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.159448+0000","last_change":"2023-10-06T02:30:12.155769+0000","last_active":"2023-10-06T02:30:12.159448+0000","last_peered":"2023-10-06T02:30:12.159448+0000","last_clean":"2023-10-06T02:30:12.159448+0000","last_became_active":"2023-10-06T02:30:12.155381+0000","last_became_peered":"2023-10-06T02:30:12.155381+0000","last_unstale":"2023-10-06T02:30:12.159448+0000","last_undegraded":"2023-10-06T02:30:12.159448+0000","last_fullsized":"2023-10-06T02:30:12.159448+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:15:45.887824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.150680+0000","last_change":"2023-10-06T02:30:11.152195+0000","last_active":"2023-10-06T02:30:12.150680+0000","last_peered":"2023-10-06T02:30:12.150680+0000","last_clean":"2023-10-06T02:30:12.150680+0000","last_became_active":"2023-10-06T02:30:11.151899+0000","last_became_peered":"2023-10-06T02:30:11.151899+0000","last_unstale":"2023-10-06T02:30:12.150680+0000","last_undegraded":"2023-10-06T02:30:12.150680+0000","last_fullsized":"2023-10-06T02:30:12.150680+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_clean_scrub_stamp":"2023-10-06T02:30:10.108419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:33:01.794705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.162876+0000","last_change":"2023-10-06T02:30:12.158963+0000","last_active":"2023-10-06T02:30:12.162876+0000","last_peered":"2023-10-06T02:30:12.162876+0000","last_clean":"2023-10-06T02:30:12.162876+0000","last_became_active":"2023-10-06T02:30:12.158657+0000","last_became_peered":"2023-10-06T02:30:12.158657+0000","last_unstale":"2023-10-06T02:30:12.162876+0000","last_undegraded":"2023-10-06T02:30:12.162876+0000","last_fullsized":"2023-10-06T02:30:12.162876+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:31:38.983594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"131'8","reported_seq":18,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.170847+0000","last_change":"2023-10-06T02:30:11.149212+0000","last_active":"2023-10-06T02:30:12.170847+0000","last_peered":"2023-10-06T02:30:12.170847+0000","last_clean":"2023-10-06T02:30:12.170847+0000","last_became_active":"2023-10-06T02:30:11.148926+0000","last_became_peered":"2023-10-06T02:30:11.148926+0000","last_unstale":"2023-10-06T02:30:12.170847+0000","last_undegraded":"2023-10-06T02:30:12.170847+0000","last_fullsized":"2023-10-06T02:30:12.170847+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_clean_scrub_stamp":"2023-10-06T02:30:10.108419+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:27.763054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":106,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.144885+0000","last_change":"2023-10-06T02:23:01.992935+0000","last_active":"2023-10-06T02:30:12.144885+0000","last_peered":"2023-10-06T02:30:12.144885+0000","last_clean":"2023-10-06T02:30:12.144885+0000","last_became_active":"2023-10-06T02:23:01.992568+0000","last_became_peered":"2023-10-06T02:23:01.992568+0000","last_unstale":"2023-10-06T02:30:12.144885+0000","last_undegraded":"2023-10-06T02:30:12.144885+0000","last_fullsized":"2023-10-06T02:30:12.144885+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:06.212251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.153199+0000","last_change":"2023-10-06T02:30:12.143202+0000","last_active":"2023-10-06T02:30:12.153199+0000","last_peered":"2023-10-06T02:30:12.153199+0000","last_clean":"2023-10-06T02:30:12.153199+0000","last_became_active":"2023-10-06T02:30:12.142895+0000","last_became_peered":"2023-10-06T02:30:12.142895+0000","last_unstale":"2023-10-06T02:30:12.153199+0000","last_undegraded":"2023-10-06T02:30:12.153199+0000","last_fullsized":"2023-10-06T02:30:12.153199+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:49:23.533957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.140286+0000","last_change":"2023-10-06T02:30:11.143304+0000","last_active":"2023-10-06T02:30:12.140286+0000","last_peered":"2023-10-06T02:30:12.140286+0000","last_clean":"2023-10-06T02:30:12.140286+0000","last_became_active":"2023-10-06T02:30:11.142989+0000","last_became_peered":"2023-10-06T02:30:11.142989+0000","last_unstale":"2023-10-06T02:30:12.140286+0000","last_undegraded":"2023-10-06T02:30:12.140286+0000","last_fullsized":"2023-10-06T02:30:12.140286+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_clean_scrub_stamp":"2023-10-06T02:30:10.108419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:42:03.384901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":106,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.140109+0000","last_change":"2023-10-06T02:23:01.989934+0000","last_active":"2023-10-06T02:30:12.140109+0000","last_peered":"2023-10-06T02:30:12.140109+0000","last_clean":"2023-10-06T02:30:12.140109+0000","last_became_active":"2023-10-06T02:23:01.989706+0000","last_became_peered":"2023-10-06T02:23:01.989706+0000","last_unstale":"2023-10-06T02:30:12.140109+0000","last_undegraded":"2023-10-06T02:30:12.140109+0000","last_fullsized":"2023-10-06T02:30:12.140109+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:06.212251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.149839+0000","last_change":"2023-10-06T02:30:12.147222+0000","last_active":"2023-10-06T02:30:12.149839+0000","last_peered":"2023-10-06T02:30:12.149839+0000","last_clean":"2023-10-06T02:30:12.149839+0000","last_became_active":"2023-10-06T02:30:12.146914+0000","last_became_peered":"2023-10-06T02:30:12.146914+0000","last_unstale":"2023-10-06T02:30:12.149839+0000","last_undegraded":"2023-10-06T02:30:12.149839+0000","last_fullsized":"2023-10-06T02:30:12.149839+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:43:32.260076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.139875+0000","last_change":"2023-10-06T02:30:11.149874+0000","last_active":"2023-10-06T02:30:12.139875+0000","last_peered":"2023-10-06T02:30:12.139875+0000","last_clean":"2023-10-06T02:30:12.139875+0000","last_became_active":"2023-10-06T02:30:11.149557+0000","last_became_peered":"2023-10-06T02:30:11.149557+0000","last_unstale":"2023-10-06T02:30:12.139875+0000","last_undegraded":"2023-10-06T02:30:12.139875+0000","last_fullsized":"2023-10-06T02:30:12.139875+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_clean_scrub_stamp":"2023-10-06T02:30:10.108419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:33:42.306461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":101,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.151028+0000","last_change":"2023-10-06T02:23:02.054069+0000","last_active":"2023-10-06T02:30:12.151028+0000","last_peered":"2023-10-06T02:30:12.151028+0000","last_clean":"2023-10-06T02:30:12.151028+0000","last_became_active":"2023-10-06T02:23:02.053739+0000","last_became_peered":"2023-10-06T02:23:02.053739+0000","last_unstale":"2023-10-06T02:30:12.151028+0000","last_undegraded":"2023-10-06T02:30:12.151028+0000","last_fullsized":"2023-10-06T02:30:12.151028+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:57:48.822608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.161236+0000","last_change":"2023-10-06T02:30:12.159090+0000","last_active":"2023-10-06T02:30:12.161236+0000","last_peered":"2023-10-06T02:30:12.161236+0000","last_clean":"2023-10-06T02:30:12.161236+0000","last_became_active":"2023-10-06T02:30:12.158737+0000","last_became_peered":"2023-10-06T02:30:12.158737+0000","last_unstale":"2023-10-06T02:30:12.161236+0000","last_undegraded":"2023-10-06T02:30:12.161236+0000","last_fullsized":"2023-10-06T02:30:12.161236+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:26:19.550623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.146192+0000","last_change":"2023-10-06T02:30:11.153794+0000","last_active":"2023-10-06T02:30:12.146192+0000","last_peered":"2023-10-06T02:30:12.146192+0000","last_clean":"2023-10-06T02:30:12.146192+0000","last_became_active":"2023-10-06T02:30:11.153528+0000","last_became_peered":"2023-10-06T02:30:11.153528+0000","last_unstale":"2023-10-06T02:30:12.146192+0000","last_undegraded":"2023-10-06T02:30:12.146192+0000","last_fullsized":"2023-10-06T02:30:12.146192+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_clean_scrub_stamp":"2023-10-06T02:30:10.108419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:40:08.038564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":101,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.150799+0000","last_change":"2023-10-06T02:23:01.990773+0000","last_active":"2023-10-06T02:30:12.150799+0000","last_peered":"2023-10-06T02:30:12.150799+0000","last_clean":"2023-10-06T02:30:12.150799+0000","last_became_active":"2023-10-06T02:23:01.990511+0000","last_became_peered":"2023-10-06T02:23:01.990511+0000","last_unstale":"2023-10-06T02:30:12.150799+0000","last_undegraded":"2023-10-06T02:30:12.150799+0000","last_fullsized":"2023-10-06T02:30:12.150799+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:42:54.797731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.158423+0000","last_change":"2023-10-06T02:30:12.151106+0000","last_active":"2023-10-06T02:30:12.158423+0000","last_peered":"2023-10-06T02:30:12.158423+0000","last_clean":"2023-10-06T02:30:12.158423+0000","last_became_active":"2023-10-06T02:30:12.148744+0000","last_became_peered":"2023-10-06T02:30:12.148744+0000","last_unstale":"2023-10-06T02:30:12.158423+0000","last_undegraded":"2023-10-06T02:30:12.158423+0000","last_fullsized":"2023-10-06T02:30:12.158423+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:40:09.048569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.14","version":"131'6","reported_seq":16,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.169574+0000","last_change":"2023-10-06T02:30:11.145555+0000","last_active":"2023-10-06T02:30:12.169574+0000","last_peered":"2023-10-06T02:30:12.169574+0000","last_clean":"2023-10-06T02:30:12.169574+0000","last_became_active":"2023-10-06T02:30:11.144779+0000","last_became_peered":"2023-10-06T02:30:11.144779+0000","last_unstale":"2023-10-06T02:30:12.169574+0000","last_undegraded":"2023-10-06T02:30:12.169574+0000","last_fullsized":"2023-10-06T02:30:12.169574+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_clean_scrub_stamp":"2023-10-06T02:30:10.108419+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:56:34.982997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"76'128","reported_seq":293,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.150890+0000","last_change":"2023-10-06T02:22:55.032519+0000","last_active":"2023-10-06T02:30:12.150890+0000","last_peered":"2023-10-06T02:30:12.150890+0000","last_clean":"2023-10-06T02:30:12.150890+0000","last_became_active":"2023-10-06T02:22:55.031367+0000","last_became_peered":"2023-10-06T02:22:55.031367+0000","last_unstale":"2023-10-06T02:30:12.150890+0000","last_undegraded":"2023-10-06T02:30:12.150890+0000","last_fullsized":"2023-10-06T02:30:12.150890+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:22:53.309097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:22:53.309097+0000","last_clean_scrub_stamp":"2023-10-06T02:22:53.309097+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:53:46.520909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":122,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.134958+0000","last_change":"2023-10-06T02:23:01.988140+0000","last_active":"2023-10-06T02:30:12.134958+0000","last_peered":"2023-10-06T02:30:12.134958+0000","last_clean":"2023-10-06T02:30:12.134958+0000","last_became_active":"2023-10-06T02:23:01.987769+0000","last_became_peered":"2023-10-06T02:23:01.987769+0000","last_unstale":"2023-10-06T02:30:12.134958+0000","last_undegraded":"2023-10-06T02:30:12.134958+0000","last_fullsized":"2023-10-06T02:30:12.134958+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:20:08.391912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.162608+0000","last_change":"2023-10-06T02:30:12.160607+0000","last_active":"2023-10-06T02:30:12.162608+0000","last_peered":"2023-10-06T02:30:12.162608+0000","last_clean":"2023-10-06T02:30:12.162608+0000","last_became_active":"2023-10-06T02:30:12.160328+0000","last_became_peered":"2023-10-06T02:30:12.160328+0000","last_unstale":"2023-10-06T02:30:12.162608+0000","last_undegraded":"2023-10-06T02:30:12.162608+0000","last_fullsized":"2023-10-06T02:30:12.162608+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:27:37.502858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.135672+0000","last_change":"2023-10-06T02:30:11.148714+0000","last_active":"2023-10-06T02:30:12.135672+0000","last_peered":"2023-10-06T02:30:12.135672+0000","last_clean":"2023-10-06T02:30:12.135672+0000","last_became_active":"2023-10-06T02:30:11.148415+0000","last_became_peered":"2023-10-06T02:30:11.148415+0000","last_unstale":"2023-10-06T02:30:12.135672+0000","last_undegraded":"2023-10-06T02:30:12.135672+0000","last_fullsized":"2023-10-06T02:30:12.135672+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_clean_scrub_stamp":"2023-10-06T02:30:10.108419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:34:11.589155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":113,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.137202+0000","last_change":"2023-10-06T02:23:02.054338+0000","last_active":"2023-10-06T02:30:12.137202+0000","last_peered":"2023-10-06T02:30:12.137202+0000","last_clean":"2023-10-06T02:30:12.137202+0000","last_became_active":"2023-10-06T02:23:02.053991+0000","last_became_peered":"2023-10-06T02:23:02.053991+0000","last_unstale":"2023-10-06T02:30:12.137202+0000","last_undegraded":"2023-10-06T02:30:12.137202+0000","last_fullsized":"2023-10-06T02:30:12.137202+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:39:30.192838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.154733+0000","last_change":"2023-10-06T02:30:12.143667+0000","last_active":"2023-10-06T02:30:12.154733+0000","last_peered":"2023-10-06T02:30:12.154733+0000","last_clean":"2023-10-06T02:30:12.154733+0000","last_became_active":"2023-10-06T02:30:12.143157+0000","last_became_peered":"2023-10-06T02:30:12.143157+0000","last_unstale":"2023-10-06T02:30:12.154733+0000","last_undegraded":"2023-10-06T02:30:12.154733+0000","last_fullsized":"2023-10-06T02:30:12.154733+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:59:54.022935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"131'3","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.163645+0000","last_change":"2023-10-06T02:30:11.138337+0000","last_active":"2023-10-06T02:30:12.163645+0000","last_peered":"2023-10-06T02:30:12.163645+0000","last_clean":"2023-10-06T02:30:12.163645+0000","last_became_active":"2023-10-06T02:30:11.138046+0000","last_became_peered":"2023-10-06T02:30:11.138046+0000","last_unstale":"2023-10-06T02:30:12.163645+0000","last_undegraded":"2023-10-06T02:30:12.163645+0000","last_fullsized":"2023-10-06T02:30:12.163645+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_clean_scrub_stamp":"2023-10-06T02:30:10.108419+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:10:34.557245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":122,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.132709+0000","last_change":"2023-10-06T02:23:01.988631+0000","last_active":"2023-10-06T02:30:12.132709+0000","last_peered":"2023-10-06T02:30:12.132709+0000","last_clean":"2023-10-06T02:30:12.132709+0000","last_became_active":"2023-10-06T02:23:01.988303+0000","last_became_peered":"2023-10-06T02:23:01.988303+0000","last_unstale":"2023-10-06T02:30:12.132709+0000","last_undegraded":"2023-10-06T02:30:12.132709+0000","last_fullsized":"2023-10-06T02:30:12.132709+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:38.211466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.161337+0000","last_change":"2023-10-06T02:30:12.155413+0000","last_active":"2023-10-06T02:30:12.161337+0000","last_peered":"2023-10-06T02:30:12.161337+0000","last_clean":"2023-10-06T02:30:12.161337+0000","last_became_active":"2023-10-06T02:30:12.155129+0000","last_became_peered":"2023-10-06T02:30:12.155129+0000","last_unstale":"2023-10-06T02:30:12.161337+0000","last_undegraded":"2023-10-06T02:30:12.161337+0000","last_fullsized":"2023-10-06T02:30:12.161337+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:33:02.804709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.13","version":"131'2","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.167649+0000","last_change":"2023-10-06T02:30:11.144283+0000","last_active":"2023-10-06T02:30:12.167649+0000","last_peered":"2023-10-06T02:30:12.167649+0000","last_clean":"2023-10-06T02:30:12.167649+0000","last_became_active":"2023-10-06T02:30:11.144037+0000","last_became_peered":"2023-10-06T02:30:11.144037+0000","last_unstale":"2023-10-06T02:30:12.167649+0000","last_undegraded":"2023-10-06T02:30:12.167649+0000","last_fullsized":"2023-10-06T02:30:12.167649+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_clean_scrub_stamp":"2023-10-06T02:30:10.108419+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:25:43.260463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":106,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.137700+0000","last_change":"2023-10-06T02:23:02.054039+0000","last_active":"2023-10-06T02:30:12.137700+0000","last_peered":"2023-10-06T02:30:12.137700+0000","last_clean":"2023-10-06T02:30:12.137700+0000","last_became_active":"2023-10-06T02:23:02.053742+0000","last_became_peered":"2023-10-06T02:23:02.053742+0000","last_unstale":"2023-10-06T02:30:12.137700+0000","last_undegraded":"2023-10-06T02:30:12.137700+0000","last_fullsized":"2023-10-06T02:30:12.137700+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:58:18.846475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574941,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303584,"kb_used_data":2200,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068256,"statfs":{"total":96636764160,"available":96325894144,"internally_reserved":0,"allocated":2252800,"data_stored":1360827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:30:12 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.64100000000000001,"15min":0.63600000000000001},"min":{"1min":0.375,"5min":0.318,"15min":0.318},"max":{"1min":0.78600000000000003,"5min":1.169,"15min":1.169},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.63900000000000001,"15min":0.61199999999999999},"min":{"1min":0.45300000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.84699999999999998,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.70699999999999996}]},{"osd":1,"last update":"Fri Oct 6 02:30:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.73199999999999998,"15min":0.69399999999999995},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.40300000000000002},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.147},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.67400000000000004,"15min":0.67200000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.82399999999999995,"5min":1.129,"15min":1.129},"last":0.754}]},{"osd":2,"last update":"Fri Oct 6 02:30:12 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.67500000000000004,"15min":0.68700000000000006},"min":{"1min":0.44,"5min":0.375,"15min":0.375},"max":{"1min":0.87,"5min":1.238,"15min":1.238},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.69099999999999995,"15min":0.68999999999999995},"min":{"1min":0.54000000000000004,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.77700000000000002}]},{"osd":3,"last update":"Fri Oct 6 02:30:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.45700000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.79400000000000004,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.76300000000000001,"15min":0.77300000000000002},"min":{"1min":0.55900000000000005,"5min":0.497,"15min":0.497},"max":{"1min":1.383,"5min":1.383,"15min":1.383},"last":0.86099999999999999}]},{"osd":4,"last update":"Fri Oct 6 02:30:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.59199999999999997,"15min":0.67200000000000004},"min":{"1min":0.42899999999999999,"5min":0.41099999999999998,"15min":0.35099999999999998},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.2849999999999999},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.67200000000000004,"15min":0.747},"min":{"1min":0.41199999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.96499999999999997,"5min":1.51,"15min":1.51},"last":0.52100000000000002}]},{"osd":5,"last update":"Fri Oct 6 02:30:12 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.69199999999999995,"15min":0.72699999999999998},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.67600000000000005,"15min":0.71999999999999997},"min":{"1min":0.47199999999999998,"5min":0.442,"15min":0.442},"max":{"1min":0.874,"5min":0.98099999999999998,"15min":1.0289999999999999},"last":0.48599999999999999}]},{"osd":6,"last update":"Fri Oct 6 02:30:12 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.76100000000000001,"15min":0.78200000000000003},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.97599999999999998,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.77800000000000002,"15min":0.80600000000000005},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.872,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.46000000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574941,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303004,"kb_used_data":1620,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068836,"statfs":{"total":96636764160,"available":96326488064,"internally_reserved":0,"allocated":1658880,"data_stored":770507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:30:06 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.68700000000000006,"15min":0.66700000000000004},"min":{"1min":0.501,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.88500000000000001,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.72999999999999998,"15min":0.70299999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.91100000000000003,"5min":1.0229999999999999,"15min":1.119},"last":0.66100000000000003}]},{"osd":1,"last update":"Fri Oct 6 02:30:06 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.68100000000000005,"15min":0.65100000000000002},"min":{"1min":0.45200000000000001,"5min":0.40999999999999998,"15min":0.40699999999999997},"max":{"1min":0.96499999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.68500000000000005,"15min":0.67100000000000004},"min":{"1min":0.437,"5min":0.41899999999999998,"15min":0.40999999999999998},"max":{"1min":0.84399999999999997,"5min":0.997,"15min":0.997},"last":0.49399999999999999}]},{"osd":2,"last update":"Fri Oct 6 02:30:06 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.72899999999999998,"15min":0.70799999999999996},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.92100000000000004,"5min":1.0449999999999999,"15min":1.137},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.74399999999999999,"15min":0.71799999999999997},"min":{"1min":0.55600000000000005,"5min":0.45300000000000001,"15min":0.40300000000000002},"max":{"1min":0.95299999999999996,"5min":1.077,"15min":1.077},"last":0.68799999999999994}]},{"osd":3,"last update":"Fri Oct 6 02:30:06 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.71199999999999997,"15min":0.67100000000000004},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.94099999999999995,"5min":1.056,"15min":1.056},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.73299999999999998,"15min":0.70899999999999996},"min":{"1min":0.56699999999999995,"5min":0.49299999999999999,"15min":0.45900000000000002},"max":{"1min":0.97799999999999998,"5min":0.97999999999999998,"15min":0.98499999999999999},"last":0.56399999999999995}]},{"osd":4,"last update":"Fri Oct 6 02:30:06 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.65900000000000003,"15min":0.64200000000000002},"min":{"1min":0.42499999999999999,"5min":0.36199999999999999,"15min":0.34300000000000003},"max":{"1min":0.93000000000000005,"5min":0.94799999999999995,"15min":1.095},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.69999999999999996,"15min":0.66900000000000004},"min":{"1min":0.52200000000000002,"5min":0.41699999999999998,"15min":0.40999999999999998},"max":{"1min":0.91900000000000004,"5min":1.002,"15min":1.29},"last":0.51600000000000001}]},{"osd":5,"last update":"Fri Oct 6 02:30:06 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.67700000000000005,"15min":0.69799999999999995},"min":{"1min":0.438,"5min":0.438,"15min":0.39700000000000002},"max":{"1min":1.014,"5min":1.014,"15min":1.2929999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.70799999999999996,"15min":0.70599999999999996},"min":{"1min":0.48699999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.01,"5min":1.01,"15min":1.02},"last":0.59099999999999997}]},{"osd":7,"last update":"Fri Oct 6 02:30:06 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.74099999999999999,"15min":0.749},"min":{"1min":0.48399999999999999,"5min":0.41499999999999998,"15min":0.36799999999999999},"max":{"1min":1,"5min":1,"15min":1.458},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.76500000000000001,"15min":0.76300000000000001},"min":{"1min":0.50700000000000001,"5min":0.47299999999999998,"15min":0.45900000000000002},"max":{"1min":1.046,"5min":1.103,"15min":1.2470000000000001},"last":0.72999999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303004,"kb_used_data":1620,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068836,"statfs":{"total":96636764160,"available":96326488064,"internally_reserved":0,"allocated":1658880,"data_stored":770507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:30:02 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.42999999999999999,"15min":0.45300000000000001},"min":{"1min":0.30599999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.59599999999999997,"5min":0.65500000000000003,"15min":1.2230000000000001},"last":0.432},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.46999999999999997,"15min":0.51400000000000001},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.58899999999999997,"5min":0.71799999999999997,"15min":1.788},"last":0.40000000000000002}]},{"osd":2,"last update":"Fri Oct 6 02:30:02 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.44,"15min":0.50600000000000001},"min":{"1min":0.33800000000000002,"5min":0.307,"15min":0.28299999999999997},"max":{"1min":0.53800000000000003,"5min":0.60899999999999999,"15min":2.0840000000000001},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.504,"15min":0.54700000000000004},"min":{"1min":0.38700000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":1.6699999999999999},"last":0.48999999999999999}]},{"osd":3,"last update":"Fri Oct 6 02:30:02 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.59799999999999998,"15min":0.60999999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.72499999999999998,"5min":0.89900000000000002,"15min":1.266},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.54200000000000004,"15min":0.62},"min":{"1min":0.46899999999999997,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.69999999999999996,"5min":0.79400000000000004,"15min":1.861},"last":0.60099999999999998}]},{"osd":4,"last update":"Fri Oct 6 02:30:02 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.77300000000000002,"15min":0.83799999999999997},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.55900000000000005},"max":{"1min":0.93999999999999995,"5min":1.3580000000000001,"15min":1.9279999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.78400000000000003,"15min":0.84099999999999997},"min":{"1min":0.57399999999999995,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.90200000000000002,"5min":1.4339999999999999,"15min":2.0179999999999998},"last":0.67300000000000004}]},{"osd":5,"last update":"Fri Oct 6 02:30:02 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.73699999999999999,"15min":0.83699999999999997},"min":{"1min":0.54400000000000004,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.874,"5min":1.3069999999999999,"15min":1.9830000000000001},"last":0.622},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.748,"15min":0.80700000000000005},"min":{"1min":0.55400000000000005,"5min":0.53600000000000003,"15min":0.47299999999999998},"max":{"1min":0.91300000000000003,"5min":1.327,"15min":1.3380000000000001},"last":0.58499999999999996}]},{"osd":6,"last update":"Fri Oct 6 02:30:02 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.78400000000000003,"15min":0.84799999999999998},"min":{"1min":0.60899999999999999,"5min":0.52700000000000002,"15min":0.45600000000000002},"max":{"1min":1.012,"5min":1.0609999999999999,"15min":1.829},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.80000000000000004,"15min":0.88900000000000001},"min":{"1min":0.61099999999999999,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.93100000000000005,"5min":1.351,"15min":2.052},"last":0.71199999999999997}]},{"osd":7,"last update":"Fri Oct 6 02:30:02 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.76300000000000001,"15min":0.81999999999999995},"min":{"1min":0.59399999999999997,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.90400000000000003,"5min":1.3120000000000001,"15min":1.3340000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.77800000000000002,"15min":0.85699999999999998},"min":{"1min":0.51600000000000001,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.018,"5min":1.2110000000000001,"15min":1.895},"last":0.63900000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574941,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303604,"kb_used_data":2220,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068236,"statfs":{"total":96636764160,"available":96325873664,"internally_reserved":0,"allocated":2273280,"data_stored":1363019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 02:29:13 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.42699999999999999,"15min":0.45200000000000001},"min":{"1min":0.32700000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":1.0129999999999999},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.504,"5min":0.46500000000000002,"15min":0.47899999999999998},"min":{"1min":0.29799999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.9830000000000001,"5min":1.9830000000000001,"15min":1.9830000000000001},"last":0.44600000000000001}]},{"osd":2,"last update":"Fri Oct 6 02:29:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.505,"15min":0.51600000000000001},"min":{"1min":0.38,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":3.0339999999999998,"5min":3.0339999999999998,"15min":3.0339999999999998},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.503,"15min":0.51900000000000002},"min":{"1min":0.39200000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.51700000000000002}]},{"osd":3,"last update":"Fri Oct 6 02:29:13 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.48699999999999999,"15min":0.503},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.59399999999999997},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.028,"5min":1.028,"15min":1.1359999999999999},"last":0.57999999999999996}]},{"osd":4,"last update":"Fri Oct 6 02:29:13 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.755,"15min":0.79800000000000004},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":2.9940000000000002,"5min":2.9940000000000002,"15min":2.9940000000000002},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.75900000000000001,"15min":0.79700000000000004},"min":{"1min":0.56599999999999995,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.54800000000000004}]},{"osd":5,"last update":"Fri Oct 6 02:29:13 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.76000000000000001,"15min":0.78600000000000003},"min":{"1min":0.61599999999999999,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.503},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.79300000000000004,"15min":0.80300000000000005},"min":{"1min":0.58299999999999996,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":3.1110000000000002,"5min":3.1110000000000002,"15min":3.1110000000000002},"last":0.66000000000000003}]},{"osd":6,"last update":"Fri Oct 6 02:29:13 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.80700000000000005,"15min":0.82399999999999995},"min":{"1min":0.61399999999999999,"5min":0.57599999999999996,"15min":0.54600000000000004},"max":{"1min":3.1600000000000001,"5min":3.1600000000000001,"15min":3.1600000000000001},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77300000000000002,"15min":0.82399999999999995},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.56499999999999995}]},{"osd":7,"last update":"Fri Oct 6 02:29:13 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.79700000000000004,"15min":0.82599999999999996},"min":{"1min":0.60199999999999998,"5min":0.56899999999999995,"15min":0.56000000000000005},"max":{"1min":2.9369999999999998,"5min":2.9369999999999998,"15min":2.9369999999999998},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.77300000000000002,"15min":0.82099999999999995},"min":{"1min":0.60499999999999998,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.59899999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574942,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303008,"kb_used_data":1624,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068832,"statfs":{"total":96636764160,"available":96326483968,"internally_reserved":0,"allocated":1662976,"data_stored":770549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:30:10 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.43099999999999999,"15min":0.437},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.502,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.47299999999999998,"15min":0.48199999999999998},"min":{"1min":0.35799999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.64100000000000001,"5min":0.88700000000000001,"15min":1.0800000000000001},"last":0.52600000000000002}]},{"osd":1,"last update":"Fri Oct 6 02:30:10 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.47799999999999998,"15min":0.498},"min":{"1min":0.36299999999999999,"5min":0.33600000000000002,"15min":0.32400000000000001},"max":{"1min":0.55500000000000005,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.52700000000000002,"15min":0.58999999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34599999999999997},"max":{"1min":0.66000000000000003,"5min":1.778,"15min":1.778},"last":0.48399999999999999}]},{"osd":3,"last update":"Fri Oct 6 02:30:10 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.50900000000000001,"15min":0.52900000000000003},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.79500000000000004,"5min":0.83699999999999997,"15min":0.97499999999999998},"last":0.438},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.59299999999999997,"15min":0.61399999999999999},"min":{"1min":0.48099999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.77100000000000002,"5min":0.84699999999999998,"15min":1.028},"last":0.505}]},{"osd":4,"last update":"Fri Oct 6 02:30:10 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.751,"15min":0.77300000000000002},"min":{"1min":0.51500000000000001,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.97499999999999998,"5min":1.109,"15min":1.1319999999999999},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.78400000000000003,"15min":0.83499999999999996},"min":{"1min":0.57999999999999996,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.96099999999999997,"5min":1.907,"15min":1.907},"last":0.63800000000000001}]},{"osd":5,"last update":"Fri Oct 6 02:30:10 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.73999999999999999,"15min":0.79000000000000004},"min":{"1min":0.53200000000000003,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.97199999999999998,"5min":1.109,"15min":1.1850000000000001},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.76300000000000001,"15min":0.81000000000000005},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0129999999999999,"5min":1.153,"15min":1.258},"last":0.60499999999999998}]},{"osd":6,"last update":"Fri Oct 6 02:30:10 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.79500000000000004,"15min":0.83799999999999997},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.98699999999999999,"5min":1.827,"15min":1.827},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.77500000000000002,"15min":0.79300000000000004},"min":{"1min":0.60299999999999998,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0660000000000001,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.65600000000000003}]},{"osd":7,"last update":"Fri Oct 6 02:30:10 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.80500000000000005,"15min":0.86799999999999999},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0369999999999999,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.78600000000000003,"15min":0.82399999999999995},"min":{"1min":0.66100000000000003,"5min":0.55300000000000005,"15min":0.54800000000000004},"max":{"1min":1.01,"5min":1.125,"15min":1.1479999999999999},"last":0.61799999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574939,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303008,"kb_used_data":1624,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068832,"statfs":{"total":96636764160,"available":96326483968,"internally_reserved":0,"allocated":1662976,"data_stored":771025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:30:10 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.443,"15min":0.49399999999999999},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.56200000000000006,"5min":1.0489999999999999,"15min":2.024},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.49199999999999999,"15min":0.51500000000000001},"min":{"1min":0.311,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.754,"5min":0.84699999999999998,"15min":2.181},"last":0.41799999999999998}]},{"osd":1,"last update":"Fri Oct 6 02:30:10 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.46899999999999997,"15min":0.64100000000000001},"min":{"1min":0.34799999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.65100000000000002,"5min":0.66100000000000003,"15min":4.492},"last":0.443},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.52600000000000002,"15min":0.68700000000000006},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.76200000000000001,"5min":1.079,"15min":4.569},"last":0.67100000000000004}]},{"osd":2,"last update":"Fri Oct 6 02:30:10 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.627,"15min":0.65500000000000003},"min":{"1min":0.372,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.82599999999999996,"5min":1.024,"15min":2.0870000000000002},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.59299999999999997,"15min":0.63700000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.76800000000000002,"5min":0.94199999999999995,"15min":2.4260000000000002},"last":0.50900000000000001}]},{"osd":4,"last update":"Fri Oct 6 02:30:10 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.73799999999999999,"15min":0.81100000000000005},"min":{"1min":0.55000000000000004,"5min":0.502,"15min":0.44800000000000001},"max":{"1min":0.94199999999999995,"5min":1.0149999999999999,"15min":2.214},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.76400000000000001,"15min":0.80000000000000004},"min":{"1min":0.54400000000000004,"5min":0.50800000000000001,"15min":0.5},"max":{"1min":1.034,"5min":1.079,"15min":1.2869999999999999},"last":0.53000000000000003}]},{"osd":5,"last update":"Fri Oct 6 02:30:10 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.76300000000000001,"15min":0.80900000000000005},"min":{"1min":0.497,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.077,"5min":1.077,"15min":1.3029999999999999},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.752,"5min":0.76000000000000001,"15min":0.83499999999999996},"min":{"1min":0.60099999999999998,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.0029999999999999,"5min":1.0580000000000001,"15min":2.242},"last":0.64900000000000002}]},{"osd":6,"last update":"Fri Oct 6 02:30:10 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.77400000000000002,"15min":0.81100000000000005},"min":{"1min":0.625,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.98199999999999998,"5min":1.0489999999999999,"15min":1.357},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.80300000000000005,"15min":0.82499999999999996},"min":{"1min":0.60599999999999998,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.014,"5min":1.0529999999999999,"15min":1.3100000000000001},"last":0.75700000000000001}]},{"osd":7,"last update":"Fri Oct 6 02:30:10 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.78200000000000003,"15min":0.84399999999999997},"min":{"1min":0.57299999999999995,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.88400000000000001,"5min":1.077,"15min":2.27},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.79800000000000004,"15min":0.84399999999999997},"min":{"1min":0.57499999999999996,"5min":0.55600000000000005,"15min":0.53200000000000003},"max":{"1min":1.0029999999999999,"5min":1.0760000000000001,"15min":1.325},"last":0.65700000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574941,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303012,"kb_used_data":1628,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068828,"statfs":{"total":96636764160,"available":96326479872,"internally_reserved":0,"allocated":1667072,"data_stored":771973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:30:06 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68100000000000005,"15min":0.95799999999999996},"min":{"1min":0.46200000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":12.337},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68400000000000005,"15min":0.88200000000000001},"min":{"1min":0.495,"5min":0.44400000000000001,"15min":0.438},"max":{"1min":0.95299999999999996,"5min":1.3520000000000001,"15min":9.6539999999999999},"last":0.80700000000000005}]},{"osd":1,"last update":"Fri Oct 6 02:30:06 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.68000000000000005,"15min":0.69399999999999995},"min":{"1min":0.47799999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.042,"5min":1.129,"15min":3.181},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.70999999999999996,"15min":0.98299999999999998},"min":{"1min":0.46300000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.98799999999999999,"5min":0.995,"15min":12.946999999999999},"last":0.85199999999999998}]},{"osd":2,"last update":"Fri Oct 6 02:30:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.71599999999999997,"15min":1.006},"min":{"1min":0.47999999999999998,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.1319999999999999,"5min":1.2989999999999999,"15min":12.377000000000001},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.745,"5min":0.73799999999999999,"15min":0.85599999999999998},"min":{"1min":0.56899999999999995,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.93700000000000006,"5min":1.0529999999999999,"15min":6.3220000000000001},"last":0.79000000000000004}]},{"osd":3,"last update":"Fri Oct 6 02:30:06 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72899999999999998,"15min":0.95599999999999996},"min":{"1min":0.55300000000000005,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.96299999999999997,"5min":1.5229999999999999,"15min":9.7720000000000002},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71799999999999997,"15min":0.76800000000000002},"min":{"1min":0.54600000000000004,"5min":0.498,"15min":0.47999999999999998},"max":{"1min":0.90700000000000003,"5min":1.212,"15min":3.0840000000000001},"last":0.75}]},{"osd":5,"last update":"Fri Oct 6 02:30:06 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.68700000000000006,"15min":0.82899999999999996},"min":{"1min":0.42099999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.99199999999999999,"5min":1.02,"15min":6.2690000000000001},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.745,"5min":0.71199999999999997,"15min":0.997},"min":{"1min":0.45700000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.0589999999999999,"5min":1.4359999999999999,"15min":10.769},"last":0.53900000000000003}]},{"osd":6,"last update":"Fri Oct 6 02:30:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.68799999999999994,"15min":0.753},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.95699999999999996,"5min":1.4590000000000001,"15min":3.1349999999999998},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.76900000000000002,"15min":0.94999999999999996},"min":{"1min":0.47999999999999998,"5min":0.47799999999999998,"15min":0.45700000000000002},"max":{"1min":1.1579999999999999,"5min":1.3839999999999999,"15min":8.0899999999999999},"last":0.90800000000000003}]},{"osd":7,"last update":"Fri Oct 6 02:30:06 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.76800000000000002,"15min":1.032},"min":{"1min":0.50900000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.1870000000000001,"5min":1.494,"15min":9.7089999999999996},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.76600000000000001,"15min":0.97099999999999997},"min":{"1min":0.44400000000000001,"5min":0.374,"15min":0.374},"max":{"1min":1.133,"5min":1.2669999999999999,"15min":7.6719999999999997},"last":0.77700000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574941,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303000,"kb_used_data":1616,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068840,"statfs":{"total":96636764160,"available":96326492160,"internally_reserved":0,"allocated":1654784,"data_stored":770437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:30:04 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63900000000000001,"15min":0.627},"min":{"1min":0.40100000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.88200000000000001,"5min":0.97599999999999998,"15min":1.4279999999999999},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.66000000000000003,"15min":0.61599999999999999},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.38700000000000001},"max":{"1min":0.92700000000000005,"5min":1.099,"15min":1.099},"last":0.57799999999999996}]},{"osd":1,"last update":"Fri Oct 6 02:30:04 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.67000000000000004,"15min":0.65100000000000002},"min":{"1min":0.44500000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.85599999999999998,"5min":1.0660000000000001,"15min":1.089},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.68999999999999995,"15min":0.67500000000000004},"min":{"1min":0.49099999999999999,"5min":0.40699999999999997,"15min":0.40500000000000003},"max":{"1min":0.93600000000000005,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.78200000000000003}]},{"osd":2,"last update":"Fri Oct 6 02:30:04 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.69799999999999995,"15min":0.67300000000000004},"min":{"1min":0.41299999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.88500000000000001,"5min":1.016,"15min":1.016},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.63,"5min":0.68899999999999995,"15min":0.68100000000000005},"min":{"1min":0.48799999999999999,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.80500000000000005,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.65900000000000003}]},{"osd":3,"last update":"Fri Oct 6 02:30:04 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.752,"15min":0.73199999999999998},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.48799999999999999},"max":{"1min":0.98499999999999999,"5min":1.159,"15min":1.159},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.753,"5min":0.79000000000000004,"15min":0.76900000000000002},"min":{"1min":0.56999999999999995,"5min":0.56200000000000006,"15min":0.54100000000000004},"max":{"1min":0.86799999999999999,"5min":1.194,"15min":1.194},"last":0.79600000000000004}]},{"osd":4,"last update":"Fri Oct 6 02:30:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.65200000000000002,"15min":0.67500000000000004},"min":{"1min":0.439,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.93000000000000005,"5min":1.012,"15min":1.1499999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.66600000000000004,"15min":0.68500000000000005},"min":{"1min":0.498,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.92700000000000005,"5min":1.25,"15min":1.25},"last":0.69299999999999995}]},{"osd":6,"last update":"Fri Oct 6 02:30:04 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.72799999999999998,"15min":0.74299999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.40300000000000002},"max":{"1min":0.95699999999999996,"5min":1.0549999999999999,"15min":1.1719999999999999},"last":0.75},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.80100000000000005,"15min":0.81899999999999995},"min":{"1min":0.502,"5min":0.373,"15min":0.373},"max":{"1min":0.98299999999999998,"5min":1.1200000000000001,"15min":1.2170000000000001},"last":0.85599999999999998}]},{"osd":7,"last update":"Fri Oct 6 02:30:04 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.747,"15min":0.76500000000000001},"min":{"1min":0.57199999999999995,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.0169999999999999,"5min":1.23,"15min":1.23},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79800000000000004,"15min":0.82899999999999996},"min":{"1min":0.58499999999999996,"5min":0.437,"15min":0.437},"max":{"1min":0.93300000000000005,"5min":1.1040000000000001,"15min":1.5640000000000001},"last":0.83399999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T02:30:16.523 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T02:30:16.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.742+0000 7f219359e700 1 Processor -- start 2023-10-06T02:30:16.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.743+0000 7f219359e700 1 -- start start 2023-10-06T02:30:16.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.744+0000 7f219359e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2194140e40 0x7f21940c6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:16.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.744+0000 7f219359e700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21940c6d50 con 0x7f219406de50 2023-10-06T02:30:16.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.744+0000 7f219359e700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21940c7030 con 0x7f2194140a70 2023-10-06T02:30:16.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.744+0000 7f219359e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21940c7310 con 0x7f2194140e40 2023-10-06T02:30:16.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.744+0000 7f2192d9d700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f219406de50 0x7f21940aeef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:47844/0 (socket says 172.21.15.31:47844) 2023-10-06T02:30:16.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.744+0000 7f2192d9d700 1 -- 172.21.15.31:0/1085621265 learned_addr learned my addr 172.21.15.31:0/1085621265 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:30:16.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.745+0000 7f2191d9b700 1 --2- 172.21.15.31:0/1085621265 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2194140e40 0x7f21940c6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:16.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.745+0000 7f219159a700 1 -- 172.21.15.31:0/1085621265 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4266045786 0 0) 0x7f21940c6d50 con 0x7f219406de50 2023-10-06T02:30:16.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.745+0000 7f219159a700 1 -- 172.21.15.31:0/1085621265 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2180003570 con 0x7f219406de50 2023-10-06T02:30:16.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.745+0000 7f219159a700 1 -- 172.21.15.31:0/1085621265 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3708880067 0 0) 0x7f21940c7030 con 0x7f2194140a70 2023-10-06T02:30:16.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.745+0000 7f219159a700 1 -- 172.21.15.31:0/1085621265 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21940c6d50 con 0x7f2194140a70 2023-10-06T02:30:16.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.745+0000 7f219159a700 1 -- 172.21.15.31:0/1085621265 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1639298634 0 0) 0x7f2180003570 con 0x7f219406de50 2023-10-06T02:30:16.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.745+0000 7f219159a700 1 -- 172.21.15.31:0/1085621265 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2194140e40 msgr2=0x7f21940c6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:16.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.745+0000 7f219159a700 1 --2- 172.21.15.31:0/1085621265 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2194140e40 0x7f21940c6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:16.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.745+0000 7f219159a700 1 -- 172.21.15.31:0/1085621265 >> v1:172.21.15.62:6789/0 conn(0x7f2194140a70 legacy=0x7f21940c7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:16.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.745+0000 7f219159a700 1 -- 172.21.15.31:0/1085621265 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21940c75f0 con 0x7f219406de50 2023-10-06T02:30:16.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.745+0000 7f219159a700 1 -- 172.21.15.31:0/1085621265 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7f218c0017f0 con 0x7f219406de50 2023-10-06T02:30:16.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.745+0000 7f219159a700 1 -- 172.21.15.31:0/1085621265 >> v1:172.21.15.31:6789/0 conn(0x7f219406de50 legacy=0x7f21940aeef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:16.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.745+0000 7f219159a700 1 --2- 172.21.15.31:0/1085621265 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2194140e40 0x7f2180003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:16.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.745+0000 7f219159a700 1 --2- 172.21.15.31:0/1085621265 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2180004180 0x7f2180004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:16.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.745+0000 7f219159a700 1 --2- 172.21.15.31:0/1085621265 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2180004d50 0x7f21800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:16.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.745+0000 7f219159a700 1 -- 172.21.15.31:0/1085621265 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21940c7310 con 0x7f2180004180 2023-10-06T02:30:16.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.745+0000 7f219159a700 1 -- 172.21.15.31:0/1085621265 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f218000bba0 con 0x7f2180004d50 2023-10-06T02:30:16.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.745+0000 7f219159a700 1 -- 172.21.15.31:0/1085621265 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2180010170 con 0x7f2194140e40 2023-10-06T02:30:16.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.746+0000 7f2191d9b700 1 --2- 172.21.15.31:0/1085621265 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2180004180 0x7f2180004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:16.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.746+0000 7f219259c700 1 --2- 172.21.15.31:0/1085621265 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2194140e40 0x7f2180003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:16.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.746+0000 7f2192d9d700 1 --2- 172.21.15.31:0/1085621265 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2180004d50 0x7f21800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:16.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.746+0000 7f2191d9b700 1 -- 172.21.15.31:0/1085621265 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2194140e40 msgr2=0x7f2180003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:16.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.746+0000 7f2191d9b700 1 --2- 172.21.15.31:0/1085621265 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2194140e40 0x7f2180003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:16.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.746+0000 7f2191d9b700 1 -- 172.21.15.31:0/1085621265 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2180004d50 msgr2=0x7f21800051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:16.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.746+0000 7f2191d9b700 1 --2- 172.21.15.31:0/1085621265 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2180004d50 0x7f21800051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:16.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.746+0000 7f2191d9b700 1 -- 172.21.15.31:0/1085621265 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2180003570 con 0x7f2180004180 2023-10-06T02:30:16.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.746+0000 7f2191d9b700 1 --2- 172.21.15.31:0/1085621265 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2180004180 0x7f2180004810 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f2184009bf0 tx=0x7f2184009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b72fb1645b975d0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:16.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.746+0000 7f219159a700 1 -- 172.21.15.31:0/1085621265 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2184005a00 con 0x7f2180004180 2023-10-06T02:30:16.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.746+0000 7f219159a700 1 -- 172.21.15.31:0/1085621265 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2184005b60 con 0x7f2180004180 2023-10-06T02:30:16.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.746+0000 7f219159a700 1 -- 172.21.15.31:0/1085621265 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f218400e710 con 0x7f2180004180 2023-10-06T02:30:16.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.747+0000 7f219359e700 1 -- 172.21.15.31:0/1085621265 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2180004180 msgr2=0x7f2180004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:16.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.747+0000 7f219359e700 1 --2- 172.21.15.31:0/1085621265 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2180004180 0x7f2180004810 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f2184009bf0 tx=0x7f2184009120 comp rx=0 tx=0).stop 2023-10-06T02:30:16.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.747+0000 7f2192d9d700 1 -- 172.21.15.31:0/1085621265 reap_dead start 2023-10-06T02:30:16.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.747+0000 7f219359e700 1 -- 172.21.15.31:0/1085621265 shutdown_connections 2023-10-06T02:30:16.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.747+0000 7f219359e700 1 -- 172.21.15.31:0/1085621265 >> 172.21.15.31:0/1085621265 conn(0x7f219406d170 msgr2=0x7f21940cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:16.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.747+0000 7f219359e700 1 -- 172.21.15.31:0/1085621265 shutdown_connections 2023-10-06T02:30:16.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.747+0000 7f219359e700 1 -- 172.21.15.31:0/1085621265 wait complete. 2023-10-06T02:30:16.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.749+0000 7f219359e700 1 Processor -- start 2023-10-06T02:30:16.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.749+0000 7f219359e700 1 -- start start 2023-10-06T02:30:16.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.749+0000 7f219359e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2180004180 0x7f21940c43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:16.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.750+0000 7f219359e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f21940c4c40 0x7f219413be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:16.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.750+0000 7f219359e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f219413f6a0 0x7f219413fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:16.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.750+0000 7f219359e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21940c48e0 con 0x7f219413f6a0 2023-10-06T02:30:16.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.750+0000 7f219359e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f219413c3c0 con 0x7f2180004180 2023-10-06T02:30:16.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.750+0000 7f219359e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2194140060 con 0x7f21940c4c40 2023-10-06T02:30:16.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.750+0000 7f2192d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f219413f6a0 0x7f219413fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:16.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.750+0000 7f2192d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f219413f6a0 0x7f219413fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39222/0 (socket says 172.21.15.31:39222) 2023-10-06T02:30:16.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.750+0000 7f2192d9d700 1 -- 172.21.15.31:0/2027391940 learned_addr learned my addr 172.21.15.31:0/2027391940 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:16.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.750+0000 7f2192d9d700 1 -- 172.21.15.31:0/2027391940 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f21940c4c40 msgr2=0x7f219413be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:16.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.750+0000 7f2192d9d700 1 --2- 172.21.15.31:0/2027391940 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f21940c4c40 0x7f219413be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:16.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.750+0000 7f2192d9d700 1 -- 172.21.15.31:0/2027391940 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2180004180 msgr2=0x7f21940c43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:16.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.750+0000 7f2192d9d700 1 --2- 172.21.15.31:0/2027391940 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2180004180 0x7f21940c43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:16.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.750+0000 7f2192d9d700 1 -- 172.21.15.31:0/2027391940 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21940c5d10 con 0x7f219413f6a0 2023-10-06T02:30:16.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.750+0000 7f2192d9d700 1 --2- 172.21.15.31:0/2027391940 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f219413f6a0 0x7f219413fb20 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f219415afe0 tx=0x7f218c002170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:16.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.751+0000 7f218a7fc700 1 -- 172.21.15.31:0/2027391940 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f218c007b40 con 0x7f219413f6a0 2023-10-06T02:30:16.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.751+0000 7f218a7fc700 1 -- 172.21.15.31:0/2027391940 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f218c00ad40 con 0x7f219413f6a0 2023-10-06T02:30:16.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.751+0000 7f218a7fc700 1 -- 172.21.15.31:0/2027391940 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f218c013640 con 0x7f219413f6a0 2023-10-06T02:30:16.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.751+0000 7f219359e700 1 -- 172.21.15.31:0/2027391940 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21940c5100 con 0x7f219413f6a0 2023-10-06T02:30:16.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.751+0000 7f219359e700 1 -- 172.21.15.31:0/2027391940 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21940c56c0 con 0x7f219413f6a0 2023-10-06T02:30:16.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.752+0000 7f219359e700 1 -- 172.21.15.31:0/2027391940 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f219404c3c0 con 0x7f219413f6a0 2023-10-06T02:30:16.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.753+0000 7f218a7fc700 1 -- 172.21.15.31:0/2027391940 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f218c013a70 con 0x7f219413f6a0 2023-10-06T02:30:16.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.753+0000 7f218a7fc700 1 --2- 172.21.15.31:0/2027391940 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f21740674d0 0x7f2174069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:16.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.754+0000 7f218a7fc700 1 -- 172.21.15.31:0/2027391940 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f218c085aa0 con 0x7f219413f6a0 2023-10-06T02:30:16.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.754+0000 7f219259c700 1 --2- 172.21.15.31:0/2027391940 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f21740674d0 0x7f2174069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:16.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.755+0000 7f219259c700 1 --2- 172.21.15.31:0/2027391940 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f21740674d0 0x7f2174069980 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f217c003ac0 tx=0x7f217c007d30 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:16.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.758+0000 7f218a7fc700 1 -- 172.21.15.31:0/2027391940 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f218c0522e0 con 0x7f219413f6a0 2023-10-06T02:30:16.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.955+0000 7f219359e700 1 -- 172.21.15.31:0/2027391940 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f21940c60a0 con 0x7f21740674d0 2023-10-06T02:30:16.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.965+0000 7f218a7fc700 1 -- 172.21.15.31:0/2027391940 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311315 (secure 0 0 0) 0x7f21940c60a0 con 0x7f21740674d0 2023-10-06T02:30:16.969 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:30:16.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.969+0000 7f219359e700 1 -- 172.21.15.31:0/2027391940 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f21740674d0 msgr2=0x7f2174069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:16.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.969+0000 7f219359e700 1 --2- 172.21.15.31:0/2027391940 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f21740674d0 0x7f2174069980 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f217c003ac0 tx=0x7f217c007d30 comp rx=0 tx=0).stop 2023-10-06T02:30:16.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.969+0000 7f219359e700 1 -- 172.21.15.31:0/2027391940 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f219413f6a0 msgr2=0x7f219413fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:16.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.969+0000 7f219359e700 1 --2- 172.21.15.31:0/2027391940 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f219413f6a0 0x7f219413fb20 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f219415afe0 tx=0x7f218c002170 comp rx=0 tx=0).stop 2023-10-06T02:30:16.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.969+0000 7f219359e700 1 -- 172.21.15.31:0/2027391940 shutdown_connections 2023-10-06T02:30:16.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.969+0000 7f219359e700 1 --2- 172.21.15.31:0/2027391940 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f219413f6a0 0x7f219413fb20 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:16.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.969+0000 7f219359e700 1 --2- 172.21.15.31:0/2027391940 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2180004180 0x7f21940c43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:16.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.969+0000 7f219359e700 1 --2- 172.21.15.31:0/2027391940 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f21740674d0 0x7f2174069980 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:16.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.969+0000 7f219359e700 1 --2- 172.21.15.31:0/2027391940 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f21940c4c40 0x7f219413be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:16.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.969+0000 7f219359e700 1 -- 172.21.15.31:0/2027391940 >> 172.21.15.31:0/2027391940 conn(0x7f219406d170 msgr2=0x7f21940c9c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:16.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.969+0000 7f219359e700 1 -- 172.21.15.31:0/2027391940 shutdown_connections 2023-10-06T02:30:16.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:16.969+0000 7f219359e700 1 -- 172.21.15.31:0/2027391940 wait complete. 2023-10-06T02:30:16.973 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-10-06T02:30:16.987 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":139,"stamp":"2023-10-06T02:30:16.493673+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":223,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":166,"ondisk_log_size":166,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":143,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2425224,"kb_used_data":14152,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752549496,"statfs":{"total":773094113280,"available":770610683904,"internally_reserved":0,"allocated":14491648,"data_stored":7348844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.003021"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.149042+0000","last_change":"2023-10-06T02:30:12.144840+0000","last_active":"2023-10-06T02:30:12.149042+0000","last_peered":"2023-10-06T02:30:12.149042+0000","last_clean":"2023-10-06T02:30:12.149042+0000","last_became_active":"2023-10-06T02:30:12.144454+0000","last_became_peered":"2023-10-06T02:30:12.144454+0000","last_unstale":"2023-10-06T02:30:12.149042+0000","last_undegraded":"2023-10-06T02:30:12.149042+0000","last_fullsized":"2023-10-06T02:30:12.149042+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:08:48.276730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.156080+0000","last_change":"2023-10-06T02:30:12.148461+0000","last_active":"2023-10-06T02:30:12.156080+0000","last_peered":"2023-10-06T02:30:12.156080+0000","last_clean":"2023-10-06T02:30:12.156080+0000","last_became_active":"2023-10-06T02:30:12.148182+0000","last_became_peered":"2023-10-06T02:30:12.148182+0000","last_unstale":"2023-10-06T02:30:12.156080+0000","last_undegraded":"2023-10-06T02:30:12.156080+0000","last_fullsized":"2023-10-06T02:30:12.156080+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:22:43.565715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.163875+0000","last_change":"2023-10-06T02:30:12.159696+0000","last_active":"2023-10-06T02:30:12.163875+0000","last_peered":"2023-10-06T02:30:12.163875+0000","last_clean":"2023-10-06T02:30:12.163875+0000","last_became_active":"2023-10-06T02:30:12.159416+0000","last_became_peered":"2023-10-06T02:30:12.159416+0000","last_unstale":"2023-10-06T02:30:12.163875+0000","last_undegraded":"2023-10-06T02:30:12.163875+0000","last_fullsized":"2023-10-06T02:30:12.163875+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:41:55.119067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.159437+0000","last_change":"2023-10-06T02:30:12.156128+0000","last_active":"2023-10-06T02:30:12.159437+0000","last_peered":"2023-10-06T02:30:12.159437+0000","last_clean":"2023-10-06T02:30:12.159437+0000","last_became_active":"2023-10-06T02:30:12.155703+0000","last_became_peered":"2023-10-06T02:30:12.155703+0000","last_unstale":"2023-10-06T02:30:12.159437+0000","last_undegraded":"2023-10-06T02:30:12.159437+0000","last_fullsized":"2023-10-06T02:30:12.159437+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:20.790424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.145182+0000","last_change":"2023-10-06T02:30:12.138126+0000","last_active":"2023-10-06T02:30:12.145182+0000","last_peered":"2023-10-06T02:30:12.145182+0000","last_clean":"2023-10-06T02:30:12.145182+0000","last_became_active":"2023-10-06T02:30:12.137801+0000","last_became_peered":"2023-10-06T02:30:12.137801+0000","last_unstale":"2023-10-06T02:30:12.145182+0000","last_undegraded":"2023-10-06T02:30:12.145182+0000","last_fullsized":"2023-10-06T02:30:12.145182+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:29:37.190584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.145225+0000","last_change":"2023-10-06T02:30:12.138750+0000","last_active":"2023-10-06T02:30:12.145225+0000","last_peered":"2023-10-06T02:30:12.145225+0000","last_clean":"2023-10-06T02:30:12.145225+0000","last_became_active":"2023-10-06T02:30:12.138398+0000","last_became_peered":"2023-10-06T02:30:12.138398+0000","last_unstale":"2023-10-06T02:30:12.145225+0000","last_undegraded":"2023-10-06T02:30:12.145225+0000","last_fullsized":"2023-10-06T02:30:12.145225+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:12:51.002294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.156930+0000","last_change":"2023-10-06T02:30:12.155555+0000","last_active":"2023-10-06T02:30:12.156930+0000","last_peered":"2023-10-06T02:30:12.156930+0000","last_clean":"2023-10-06T02:30:12.156930+0000","last_became_active":"2023-10-06T02:30:12.155287+0000","last_became_peered":"2023-10-06T02:30:12.155287+0000","last_unstale":"2023-10-06T02:30:12.156930+0000","last_undegraded":"2023-10-06T02:30:12.156930+0000","last_fullsized":"2023-10-06T02:30:12.156930+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:33:02.804709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.153470+0000","last_change":"2023-10-06T02:30:12.148777+0000","last_active":"2023-10-06T02:30:12.153470+0000","last_peered":"2023-10-06T02:30:12.153470+0000","last_clean":"2023-10-06T02:30:12.153470+0000","last_became_active":"2023-10-06T02:30:12.148461+0000","last_became_peered":"2023-10-06T02:30:12.148461+0000","last_unstale":"2023-10-06T02:30:12.153470+0000","last_undegraded":"2023-10-06T02:30:12.153470+0000","last_fullsized":"2023-10-06T02:30:12.153470+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:46.828782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.164998+0000","last_change":"2023-10-06T02:30:12.159510+0000","last_active":"2023-10-06T02:30:12.164998+0000","last_peered":"2023-10-06T02:30:12.164998+0000","last_clean":"2023-10-06T02:30:12.164998+0000","last_became_active":"2023-10-06T02:30:12.159185+0000","last_became_peered":"2023-10-06T02:30:12.159185+0000","last_unstale":"2023-10-06T02:30:12.164998+0000","last_undegraded":"2023-10-06T02:30:12.164998+0000","last_fullsized":"2023-10-06T02:30:12.164998+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:39:36.644245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.152624+0000","last_change":"2023-10-06T02:30:12.142557+0000","last_active":"2023-10-06T02:30:12.152624+0000","last_peered":"2023-10-06T02:30:12.152624+0000","last_clean":"2023-10-06T02:30:12.152624+0000","last_became_active":"2023-10-06T02:30:12.141736+0000","last_became_peered":"2023-10-06T02:30:12.141736+0000","last_unstale":"2023-10-06T02:30:12.152624+0000","last_undegraded":"2023-10-06T02:30:12.152624+0000","last_fullsized":"2023-10-06T02:30:12.152624+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:10:46.375670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.162628+0000","last_change":"2023-10-06T02:30:12.161151+0000","last_active":"2023-10-06T02:30:12.162628+0000","last_peered":"2023-10-06T02:30:12.162628+0000","last_clean":"2023-10-06T02:30:12.162628+0000","last_became_active":"2023-10-06T02:30:12.160897+0000","last_became_peered":"2023-10-06T02:30:12.160897+0000","last_unstale":"2023-10-06T02:30:12.162628+0000","last_undegraded":"2023-10-06T02:30:12.162628+0000","last_fullsized":"2023-10-06T02:30:12.162628+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:40:26.757057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.163348+0000","last_change":"2023-10-06T02:30:12.161868+0000","last_active":"2023-10-06T02:30:12.163348+0000","last_peered":"2023-10-06T02:30:12.163348+0000","last_clean":"2023-10-06T02:30:12.163348+0000","last_became_active":"2023-10-06T02:30:12.161586+0000","last_became_peered":"2023-10-06T02:30:12.161586+0000","last_unstale":"2023-10-06T02:30:12.163348+0000","last_undegraded":"2023-10-06T02:30:12.163348+0000","last_fullsized":"2023-10-06T02:30:12.163348+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:47:13.733592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.153445+0000","last_change":"2023-10-06T02:30:12.143405+0000","last_active":"2023-10-06T02:30:12.153445+0000","last_peered":"2023-10-06T02:30:12.153445+0000","last_clean":"2023-10-06T02:30:12.153445+0000","last_became_active":"2023-10-06T02:30:12.142467+0000","last_became_peered":"2023-10-06T02:30:12.142467+0000","last_unstale":"2023-10-06T02:30:12.153445+0000","last_undegraded":"2023-10-06T02:30:12.153445+0000","last_fullsized":"2023-10-06T02:30:12.153445+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:42:36.144572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.162327+0000","last_change":"2023-10-06T02:30:12.159390+0000","last_active":"2023-10-06T02:30:12.162327+0000","last_peered":"2023-10-06T02:30:12.162327+0000","last_clean":"2023-10-06T02:30:12.162327+0000","last_became_active":"2023-10-06T02:30:12.159038+0000","last_became_peered":"2023-10-06T02:30:12.159038+0000","last_unstale":"2023-10-06T02:30:12.162327+0000","last_undegraded":"2023-10-06T02:30:12.162327+0000","last_fullsized":"2023-10-06T02:30:12.162327+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:33:27.094350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.155010+0000","last_change":"2023-10-06T02:30:12.147928+0000","last_active":"2023-10-06T02:30:12.155010+0000","last_peered":"2023-10-06T02:30:12.155010+0000","last_clean":"2023-10-06T02:30:12.155010+0000","last_became_active":"2023-10-06T02:30:12.147595+0000","last_became_peered":"2023-10-06T02:30:12.147595+0000","last_unstale":"2023-10-06T02:30:12.155010+0000","last_undegraded":"2023-10-06T02:30:12.155010+0000","last_fullsized":"2023-10-06T02:30:12.155010+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:02:46.429440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.152655+0000","last_change":"2023-10-06T02:30:12.141268+0000","last_active":"2023-10-06T02:30:12.152655+0000","last_peered":"2023-10-06T02:30:12.152655+0000","last_clean":"2023-10-06T02:30:12.152655+0000","last_became_active":"2023-10-06T02:30:12.141017+0000","last_became_peered":"2023-10-06T02:30:12.141017+0000","last_unstale":"2023-10-06T02:30:12.152655+0000","last_undegraded":"2023-10-06T02:30:12.152655+0000","last_fullsized":"2023-10-06T02:30:12.152655+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:02:17.503754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.158851+0000","last_change":"2023-10-06T02:30:12.153384+0000","last_active":"2023-10-06T02:30:12.158851+0000","last_peered":"2023-10-06T02:30:12.158851+0000","last_clean":"2023-10-06T02:30:12.158851+0000","last_became_active":"2023-10-06T02:30:12.153067+0000","last_became_peered":"2023-10-06T02:30:12.153067+0000","last_unstale":"2023-10-06T02:30:12.158851+0000","last_undegraded":"2023-10-06T02:30:12.158851+0000","last_fullsized":"2023-10-06T02:30:12.158851+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:25:27.483151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.154754+0000","last_change":"2023-10-06T02:30:12.144337+0000","last_active":"2023-10-06T02:30:12.154754+0000","last_peered":"2023-10-06T02:30:12.154754+0000","last_clean":"2023-10-06T02:30:12.154754+0000","last_became_active":"2023-10-06T02:30:12.144068+0000","last_became_peered":"2023-10-06T02:30:12.144068+0000","last_unstale":"2023-10-06T02:30:12.154754+0000","last_undegraded":"2023-10-06T02:30:12.154754+0000","last_fullsized":"2023-10-06T02:30:12.154754+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:08:30.193069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.166823+0000","last_change":"2023-10-06T02:30:12.163812+0000","last_active":"2023-10-06T02:30:12.166823+0000","last_peered":"2023-10-06T02:30:12.166823+0000","last_clean":"2023-10-06T02:30:12.166823+0000","last_became_active":"2023-10-06T02:30:12.162912+0000","last_became_peered":"2023-10-06T02:30:12.162912+0000","last_unstale":"2023-10-06T02:30:12.166823+0000","last_undegraded":"2023-10-06T02:30:12.166823+0000","last_fullsized":"2023-10-06T02:30:12.166823+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:19:15.502284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.152518+0000","last_change":"2023-10-06T02:30:12.150200+0000","last_active":"2023-10-06T02:30:12.152518+0000","last_peered":"2023-10-06T02:30:12.152518+0000","last_clean":"2023-10-06T02:30:12.152518+0000","last_became_active":"2023-10-06T02:30:12.149884+0000","last_became_peered":"2023-10-06T02:30:12.149884+0000","last_unstale":"2023-10-06T02:30:12.152518+0000","last_undegraded":"2023-10-06T02:30:12.152518+0000","last_fullsized":"2023-10-06T02:30:12.152518+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:49:23.533957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.164863+0000","last_change":"2023-10-06T02:30:12.159653+0000","last_active":"2023-10-06T02:30:12.164863+0000","last_peered":"2023-10-06T02:30:12.164863+0000","last_clean":"2023-10-06T02:30:12.164863+0000","last_became_active":"2023-10-06T02:30:12.159195+0000","last_became_peered":"2023-10-06T02:30:12.159195+0000","last_unstale":"2023-10-06T02:30:12.164863+0000","last_undegraded":"2023-10-06T02:30:12.164863+0000","last_fullsized":"2023-10-06T02:30:12.164863+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:48:59.244317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.155451+0000","last_change":"2023-10-06T02:30:12.154533+0000","last_active":"2023-10-06T02:30:12.155451+0000","last_peered":"2023-10-06T02:30:12.155451+0000","last_clean":"2023-10-06T02:30:12.155451+0000","last_became_active":"2023-10-06T02:30:12.154252+0000","last_became_peered":"2023-10-06T02:30:12.154252+0000","last_unstale":"2023-10-06T02:30:12.155451+0000","last_undegraded":"2023-10-06T02:30:12.155451+0000","last_fullsized":"2023-10-06T02:30:12.155451+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:25:44.270467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.152557+0000","last_change":"2023-10-06T02:30:12.147146+0000","last_active":"2023-10-06T02:30:12.152557+0000","last_peered":"2023-10-06T02:30:12.152557+0000","last_clean":"2023-10-06T02:30:12.152557+0000","last_became_active":"2023-10-06T02:30:12.146880+0000","last_became_peered":"2023-10-06T02:30:12.146880+0000","last_unstale":"2023-10-06T02:30:12.152557+0000","last_undegraded":"2023-10-06T02:30:12.152557+0000","last_fullsized":"2023-10-06T02:30:12.152557+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:16:11.270014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.159832+0000","last_change":"2023-10-06T02:30:12.158538+0000","last_active":"2023-10-06T02:30:12.159832+0000","last_peered":"2023-10-06T02:30:12.159832+0000","last_clean":"2023-10-06T02:30:12.159832+0000","last_became_active":"2023-10-06T02:30:12.158175+0000","last_became_peered":"2023-10-06T02:30:12.158175+0000","last_unstale":"2023-10-06T02:30:12.159832+0000","last_undegraded":"2023-10-06T02:30:12.159832+0000","last_fullsized":"2023-10-06T02:30:12.159832+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:24.831338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.147982+0000","last_change":"2023-10-06T02:30:12.143290+0000","last_active":"2023-10-06T02:30:12.147982+0000","last_peered":"2023-10-06T02:30:12.147982+0000","last_clean":"2023-10-06T02:30:12.147982+0000","last_became_active":"2023-10-06T02:30:12.142811+0000","last_became_peered":"2023-10-06T02:30:12.142811+0000","last_unstale":"2023-10-06T02:30:12.147982+0000","last_undegraded":"2023-10-06T02:30:12.147982+0000","last_fullsized":"2023-10-06T02:30:12.147982+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:15:45.887824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.144456+0000","last_change":"2023-10-06T02:30:12.142626+0000","last_active":"2023-10-06T02:30:12.144456+0000","last_peered":"2023-10-06T02:30:12.144456+0000","last_clean":"2023-10-06T02:30:12.144456+0000","last_became_active":"2023-10-06T02:30:12.142327+0000","last_became_peered":"2023-10-06T02:30:12.142327+0000","last_unstale":"2023-10-06T02:30:12.144456+0000","last_undegraded":"2023-10-06T02:30:12.144456+0000","last_fullsized":"2023-10-06T02:30:12.144456+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:40:09.048569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.164955+0000","last_change":"2023-10-06T02:30:12.162084+0000","last_active":"2023-10-06T02:30:12.164955+0000","last_peered":"2023-10-06T02:30:12.164955+0000","last_clean":"2023-10-06T02:30:12.164955+0000","last_became_active":"2023-10-06T02:30:12.160260+0000","last_became_peered":"2023-10-06T02:30:12.160260+0000","last_unstale":"2023-10-06T02:30:12.164955+0000","last_undegraded":"2023-10-06T02:30:12.164955+0000","last_fullsized":"2023-10-06T02:30:12.164955+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:19:39.260439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.156466+0000","last_change":"2023-10-06T02:30:12.146173+0000","last_active":"2023-10-06T02:30:12.156466+0000","last_peered":"2023-10-06T02:30:12.156466+0000","last_clean":"2023-10-06T02:30:12.156466+0000","last_became_active":"2023-10-06T02:30:12.145862+0000","last_became_peered":"2023-10-06T02:30:12.145862+0000","last_unstale":"2023-10-06T02:30:12.156466+0000","last_undegraded":"2023-10-06T02:30:12.156466+0000","last_fullsized":"2023-10-06T02:30:12.156466+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:35.261516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.159429+0000","last_change":"2023-10-06T02:30:12.155658+0000","last_active":"2023-10-06T02:30:12.159429+0000","last_peered":"2023-10-06T02:30:12.159429+0000","last_clean":"2023-10-06T02:30:12.159429+0000","last_became_active":"2023-10-06T02:30:12.154950+0000","last_became_peered":"2023-10-06T02:30:12.154950+0000","last_unstale":"2023-10-06T02:30:12.159429+0000","last_undegraded":"2023-10-06T02:30:12.159429+0000","last_fullsized":"2023-10-06T02:30:12.159429+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:08:48.276730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.156351+0000","last_change":"2023-10-06T02:30:12.146097+0000","last_active":"2023-10-06T02:30:12.156351+0000","last_peered":"2023-10-06T02:30:12.156351+0000","last_clean":"2023-10-06T02:30:12.156351+0000","last_became_active":"2023-10-06T02:30:12.145821+0000","last_became_peered":"2023-10-06T02:30:12.145821+0000","last_unstale":"2023-10-06T02:30:12.156351+0000","last_undegraded":"2023-10-06T02:30:12.156351+0000","last_fullsized":"2023-10-06T02:30:12.156351+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:01:54.944524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.152765+0000","last_change":"2023-10-06T02:30:12.140575+0000","last_active":"2023-10-06T02:30:12.152765+0000","last_peered":"2023-10-06T02:30:12.152765+0000","last_clean":"2023-10-06T02:30:12.152765+0000","last_became_active":"2023-10-06T02:30:12.140288+0000","last_became_peered":"2023-10-06T02:30:12.140288+0000","last_unstale":"2023-10-06T02:30:12.152765+0000","last_undegraded":"2023-10-06T02:30:12.152765+0000","last_fullsized":"2023-10-06T02:30:12.152765+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:56:35.993001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.157442+0000","last_change":"2023-10-06T02:30:12.149331+0000","last_active":"2023-10-06T02:30:12.157442+0000","last_peered":"2023-10-06T02:30:12.157442+0000","last_clean":"2023-10-06T02:30:12.157442+0000","last_became_active":"2023-10-06T02:30:12.147293+0000","last_became_peered":"2023-10-06T02:30:12.147293+0000","last_unstale":"2023-10-06T02:30:12.157442+0000","last_undegraded":"2023-10-06T02:30:12.157442+0000","last_fullsized":"2023-10-06T02:30:12.157442+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:36:59.376547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.161039+0000","last_change":"2023-10-06T02:30:12.158156+0000","last_active":"2023-10-06T02:30:12.161039+0000","last_peered":"2023-10-06T02:30:12.161039+0000","last_clean":"2023-10-06T02:30:12.161039+0000","last_became_active":"2023-10-06T02:30:12.157856+0000","last_became_peered":"2023-10-06T02:30:12.157856+0000","last_unstale":"2023-10-06T02:30:12.161039+0000","last_undegraded":"2023-10-06T02:30:12.161039+0000","last_fullsized":"2023-10-06T02:30:12.161039+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:09.847602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"131'4","reported_seq":14,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.167766+0000","last_change":"2023-10-06T02:30:11.143280+0000","last_active":"2023-10-06T02:30:12.167766+0000","last_peered":"2023-10-06T02:30:12.167766+0000","last_clean":"2023-10-06T02:30:12.167766+0000","last_became_active":"2023-10-06T02:30:11.143032+0000","last_became_peered":"2023-10-06T02:30:11.143032+0000","last_unstale":"2023-10-06T02:30:12.167766+0000","last_undegraded":"2023-10-06T02:30:12.167766+0000","last_fullsized":"2023-10-06T02:30:12.167766+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_clean_scrub_stamp":"2023-10-06T02:30:10.108419+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:50:30.974609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.155528+0000","last_change":"2023-10-06T02:30:12.149152+0000","last_active":"2023-10-06T02:30:12.155528+0000","last_peered":"2023-10-06T02:30:12.155528+0000","last_clean":"2023-10-06T02:30:12.155528+0000","last_became_active":"2023-10-06T02:30:12.148787+0000","last_became_peered":"2023-10-06T02:30:12.148787+0000","last_unstale":"2023-10-06T02:30:12.155528+0000","last_undegraded":"2023-10-06T02:30:12.155528+0000","last_fullsized":"2023-10-06T02:30:12.155528+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:19:22.533732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1e","version":"131'1","reported_seq":11,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.169833+0000","last_change":"2023-10-06T02:30:11.156964+0000","last_active":"2023-10-06T02:30:12.169833+0000","last_peered":"2023-10-06T02:30:12.169833+0000","last_clean":"2023-10-06T02:30:12.169833+0000","last_became_active":"2023-10-06T02:30:11.156574+0000","last_became_peered":"2023-10-06T02:30:11.156574+0000","last_unstale":"2023-10-06T02:30:12.169833+0000","last_undegraded":"2023-10-06T02:30:12.169833+0000","last_fullsized":"2023-10-06T02:30:12.169833+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_clean_scrub_stamp":"2023-10-06T02:30:10.108419+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:50:30.974609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.152601+0000","last_change":"2023-10-06T02:30:12.145807+0000","last_active":"2023-10-06T02:30:12.152601+0000","last_peered":"2023-10-06T02:30:12.152601+0000","last_clean":"2023-10-06T02:30:12.152601+0000","last_became_active":"2023-10-06T02:30:12.145534+0000","last_became_peered":"2023-10-06T02:30:12.145534+0000","last_unstale":"2023-10-06T02:30:12.152601+0000","last_undegraded":"2023-10-06T02:30:12.152601+0000","last_fullsized":"2023-10-06T02:30:12.152601+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:31:03.185895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"131'2","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.170087+0000","last_change":"2023-10-06T02:30:11.156936+0000","last_active":"2023-10-06T02:30:12.170087+0000","last_peered":"2023-10-06T02:30:12.170087+0000","last_clean":"2023-10-06T02:30:12.170087+0000","last_became_active":"2023-10-06T02:30:11.156581+0000","last_became_peered":"2023-10-06T02:30:11.156581+0000","last_unstale":"2023-10-06T02:30:12.170087+0000","last_undegraded":"2023-10-06T02:30:12.170087+0000","last_fullsized":"2023-10-06T02:30:12.170087+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_clean_scrub_stamp":"2023-10-06T02:30:10.108419+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:25:43.260463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.162657+0000","last_change":"2023-10-06T02:30:12.157805+0000","last_active":"2023-10-06T02:30:12.162657+0000","last_peered":"2023-10-06T02:30:12.162657+0000","last_clean":"2023-10-06T02:30:12.162657+0000","last_became_active":"2023-10-06T02:30:12.157525+0000","last_became_peered":"2023-10-06T02:30:12.157525+0000","last_unstale":"2023-10-06T02:30:12.162657+0000","last_undegraded":"2023-10-06T02:30:12.162657+0000","last_fullsized":"2023-10-06T02:30:12.162657+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:48:56.362100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"131'2","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.161955+0000","last_change":"2023-10-06T02:30:11.156553+0000","last_active":"2023-10-06T02:30:12.161955+0000","last_peered":"2023-10-06T02:30:12.161955+0000","last_clean":"2023-10-06T02:30:12.161955+0000","last_became_active":"2023-10-06T02:30:11.156286+0000","last_became_peered":"2023-10-06T02:30:11.156286+0000","last_unstale":"2023-10-06T02:30:12.161955+0000","last_undegraded":"2023-10-06T02:30:12.161955+0000","last_fullsized":"2023-10-06T02:30:12.161955+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_clean_scrub_stamp":"2023-10-06T02:30:10.108419+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:19.780420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.149449+0000","last_change":"2023-10-06T02:30:12.141626+0000","last_active":"2023-10-06T02:30:12.149449+0000","last_peered":"2023-10-06T02:30:12.149449+0000","last_clean":"2023-10-06T02:30:12.149449+0000","last_became_active":"2023-10-06T02:30:12.141323+0000","last_became_peered":"2023-10-06T02:30:12.141323+0000","last_unstale":"2023-10-06T02:30:12.149449+0000","last_undegraded":"2023-10-06T02:30:12.149449+0000","last_fullsized":"2023-10-06T02:30:12.149449+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:12:51.002294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"131'1","reported_seq":11,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.162946+0000","last_change":"2023-10-06T02:30:11.147508+0000","last_active":"2023-10-06T02:30:12.162946+0000","last_peered":"2023-10-06T02:30:12.162946+0000","last_clean":"2023-10-06T02:30:12.162946+0000","last_became_active":"2023-10-06T02:30:11.147084+0000","last_became_peered":"2023-10-06T02:30:11.147084+0000","last_unstale":"2023-10-06T02:30:12.162946+0000","last_undegraded":"2023-10-06T02:30:12.162946+0000","last_fullsized":"2023-10-06T02:30:12.162946+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_clean_scrub_stamp":"2023-10-06T02:30:10.108419+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:39:22.980561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.159569+0000","last_change":"2023-10-06T02:30:12.154322+0000","last_active":"2023-10-06T02:30:12.159569+0000","last_peered":"2023-10-06T02:30:12.159569+0000","last_clean":"2023-10-06T02:30:12.159569+0000","last_became_active":"2023-10-06T02:30:12.154020+0000","last_became_peered":"2023-10-06T02:30:12.154020+0000","last_unstale":"2023-10-06T02:30:12.159569+0000","last_undegraded":"2023-10-06T02:30:12.159569+0000","last_fullsized":"2023-10-06T02:30:12.159569+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:34:32.671541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1a","version":"131'1","reported_seq":11,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.166635+0000","last_change":"2023-10-06T02:30:11.146699+0000","last_active":"2023-10-06T02:30:12.166635+0000","last_peered":"2023-10-06T02:30:12.166635+0000","last_clean":"2023-10-06T02:30:12.166635+0000","last_became_active":"2023-10-06T02:30:11.146468+0000","last_became_peered":"2023-10-06T02:30:11.146468+0000","last_unstale":"2023-10-06T02:30:12.166635+0000","last_undegraded":"2023-10-06T02:30:12.166635+0000","last_fullsized":"2023-10-06T02:30:12.166635+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_clean_scrub_stamp":"2023-10-06T02:30:10.108419+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:50:04.409854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.147874+0000","last_change":"2023-10-06T02:30:12.146356+0000","last_active":"2023-10-06T02:30:12.147874+0000","last_peered":"2023-10-06T02:30:12.147874+0000","last_clean":"2023-10-06T02:30:12.147874+0000","last_became_active":"2023-10-06T02:30:12.146093+0000","last_became_peered":"2023-10-06T02:30:12.146093+0000","last_unstale":"2023-10-06T02:30:12.147874+0000","last_undegraded":"2023-10-06T02:30:12.147874+0000","last_fullsized":"2023-10-06T02:30:12.147874+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:26:15.763758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.132982+0000","last_change":"2023-10-06T02:30:11.150906+0000","last_active":"2023-10-06T02:30:12.132982+0000","last_peered":"2023-10-06T02:30:12.132982+0000","last_clean":"2023-10-06T02:30:12.132982+0000","last_became_active":"2023-10-06T02:30:11.150588+0000","last_became_peered":"2023-10-06T02:30:11.150588+0000","last_unstale":"2023-10-06T02:30:12.132982+0000","last_undegraded":"2023-10-06T02:30:12.132982+0000","last_fullsized":"2023-10-06T02:30:12.132982+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_clean_scrub_stamp":"2023-10-06T02:30:10.108419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:49:11.284716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.148546+0000","last_change":"2023-10-06T02:30:12.143757+0000","last_active":"2023-10-06T02:30:12.148546+0000","last_peered":"2023-10-06T02:30:12.148546+0000","last_clean":"2023-10-06T02:30:12.148546+0000","last_became_active":"2023-10-06T02:30:12.143472+0000","last_became_peered":"2023-10-06T02:30:12.143472+0000","last_unstale":"2023-10-06T02:30:12.148546+0000","last_undegraded":"2023-10-06T02:30:12.148546+0000","last_fullsized":"2023-10-06T02:30:12.148546+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:26:19.550623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.129940+0000","last_change":"2023-10-06T02:30:11.153621+0000","last_active":"2023-10-06T02:30:12.129940+0000","last_peered":"2023-10-06T02:30:12.129940+0000","last_clean":"2023-10-06T02:30:12.129940+0000","last_became_active":"2023-10-06T02:30:11.153323+0000","last_became_peered":"2023-10-06T02:30:11.153323+0000","last_unstale":"2023-10-06T02:30:12.129940+0000","last_undegraded":"2023-10-06T02:30:12.129940+0000","last_fullsized":"2023-10-06T02:30:12.129940+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_clean_scrub_stamp":"2023-10-06T02:30:10.108419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:30:04.813580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.170263+0000","last_change":"2023-10-06T02:30:12.167908+0000","last_active":"2023-10-06T02:30:12.170263+0000","last_peered":"2023-10-06T02:30:12.170263+0000","last_clean":"2023-10-06T02:30:12.170263+0000","last_became_active":"2023-10-06T02:30:12.167328+0000","last_became_peered":"2023-10-06T02:30:12.167328+0000","last_unstale":"2023-10-06T02:30:12.170263+0000","last_undegraded":"2023-10-06T02:30:12.170263+0000","last_fullsized":"2023-10-06T02:30:12.170263+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:11:47.949557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"131'1","reported_seq":11,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.162998+0000","last_change":"2023-10-06T02:30:11.148080+0000","last_active":"2023-10-06T02:30:12.162998+0000","last_peered":"2023-10-06T02:30:12.162998+0000","last_clean":"2023-10-06T02:30:12.162998+0000","last_became_active":"2023-10-06T02:30:11.147435+0000","last_became_peered":"2023-10-06T02:30:11.147435+0000","last_unstale":"2023-10-06T02:30:12.162998+0000","last_undegraded":"2023-10-06T02:30:12.162998+0000","last_fullsized":"2023-10-06T02:30:12.162998+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_clean_scrub_stamp":"2023-10-06T02:30:10.108419+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:12:06.291697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.166800+0000","last_change":"2023-10-06T02:30:12.163464+0000","last_active":"2023-10-06T02:30:12.166800+0000","last_peered":"2023-10-06T02:30:12.166800+0000","last_clean":"2023-10-06T02:30:12.166800+0000","last_became_active":"2023-10-06T02:30:12.163180+0000","last_became_peered":"2023-10-06T02:30:12.163180+0000","last_unstale":"2023-10-06T02:30:12.166800+0000","last_undegraded":"2023-10-06T02:30:12.166800+0000","last_fullsized":"2023-10-06T02:30:12.166800+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:58:19.872024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"131'2","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.169584+0000","last_change":"2023-10-06T02:30:11.151792+0000","last_active":"2023-10-06T02:30:12.169584+0000","last_peered":"2023-10-06T02:30:12.169584+0000","last_clean":"2023-10-06T02:30:12.169584+0000","last_became_active":"2023-10-06T02:30:11.151477+0000","last_became_peered":"2023-10-06T02:30:11.151477+0000","last_unstale":"2023-10-06T02:30:12.169584+0000","last_undegraded":"2023-10-06T02:30:12.169584+0000","last_fullsized":"2023-10-06T02:30:12.169584+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_clean_scrub_stamp":"2023-10-06T02:30:10.108419+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:26:18.540619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.149076+0000","last_change":"2023-10-06T02:30:12.145890+0000","last_active":"2023-10-06T02:30:12.149076+0000","last_peered":"2023-10-06T02:30:12.149076+0000","last_clean":"2023-10-06T02:30:12.149076+0000","last_became_active":"2023-10-06T02:30:12.145447+0000","last_became_peered":"2023-10-06T02:30:12.145447+0000","last_unstale":"2023-10-06T02:30:12.149076+0000","last_undegraded":"2023-10-06T02:30:12.149076+0000","last_fullsized":"2023-10-06T02:30:12.149076+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:36:59.376547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"131'1","reported_seq":11,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.161831+0000","last_change":"2023-10-06T02:30:11.149042+0000","last_active":"2023-10-06T02:30:12.161831+0000","last_peered":"2023-10-06T02:30:12.161831+0000","last_clean":"2023-10-06T02:30:12.161831+0000","last_became_active":"2023-10-06T02:30:11.148792+0000","last_became_peered":"2023-10-06T02:30:11.148792+0000","last_unstale":"2023-10-06T02:30:12.161831+0000","last_undegraded":"2023-10-06T02:30:12.161831+0000","last_fullsized":"2023-10-06T02:30:12.161831+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_clean_scrub_stamp":"2023-10-06T02:30:10.108419+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:33:26.084346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.150367+0000","last_change":"2023-10-06T02:30:12.148273+0000","last_active":"2023-10-06T02:30:12.150367+0000","last_peered":"2023-10-06T02:30:12.150367+0000","last_clean":"2023-10-06T02:30:12.150367+0000","last_became_active":"2023-10-06T02:30:12.147817+0000","last_became_peered":"2023-10-06T02:30:12.147817+0000","last_unstale":"2023-10-06T02:30:12.150367+0000","last_undegraded":"2023-10-06T02:30:12.150367+0000","last_fullsized":"2023-10-06T02:30:12.150367+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:46.828782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.138074+0000","last_change":"2023-10-06T02:30:11.153165+0000","last_active":"2023-10-06T02:30:12.138074+0000","last_peered":"2023-10-06T02:30:12.138074+0000","last_clean":"2023-10-06T02:30:12.138074+0000","last_became_active":"2023-10-06T02:30:11.152879+0000","last_became_peered":"2023-10-06T02:30:11.152879+0000","last_unstale":"2023-10-06T02:30:12.138074+0000","last_undegraded":"2023-10-06T02:30:12.138074+0000","last_fullsized":"2023-10-06T02:30:12.138074+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_clean_scrub_stamp":"2023-10-06T02:30:10.108419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:40:25.747053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.162884+0000","last_change":"2023-10-06T02:30:12.159732+0000","last_active":"2023-10-06T02:30:12.162884+0000","last_peered":"2023-10-06T02:30:12.162884+0000","last_clean":"2023-10-06T02:30:12.162884+0000","last_became_active":"2023-10-06T02:30:12.159433+0000","last_became_peered":"2023-10-06T02:30:12.159433+0000","last_unstale":"2023-10-06T02:30:12.162884+0000","last_undegraded":"2023-10-06T02:30:12.162884+0000","last_fullsized":"2023-10-06T02:30:12.162884+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:47:28.496930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"131'2","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.165135+0000","last_change":"2023-10-06T02:30:11.140625+0000","last_active":"2023-10-06T02:30:12.165135+0000","last_peered":"2023-10-06T02:30:12.165135+0000","last_clean":"2023-10-06T02:30:12.165135+0000","last_became_active":"2023-10-06T02:30:11.140327+0000","last_became_peered":"2023-10-06T02:30:11.140327+0000","last_unstale":"2023-10-06T02:30:12.165135+0000","last_undegraded":"2023-10-06T02:30:12.165135+0000","last_fullsized":"2023-10-06T02:30:12.165135+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_clean_scrub_stamp":"2023-10-06T02:30:10.108419+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:59:01.365002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.158258+0000","last_change":"2023-10-06T02:30:12.151174+0000","last_active":"2023-10-06T02:30:12.158258+0000","last_peered":"2023-10-06T02:30:12.158258+0000","last_clean":"2023-10-06T02:30:12.158258+0000","last_became_active":"2023-10-06T02:30:12.150890+0000","last_became_peered":"2023-10-06T02:30:12.150890+0000","last_unstale":"2023-10-06T02:30:12.158258+0000","last_undegraded":"2023-10-06T02:30:12.158258+0000","last_fullsized":"2023-10-06T02:30:12.158258+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:18:54.516532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.138534+0000","last_change":"2023-10-06T02:30:11.148034+0000","last_active":"2023-10-06T02:30:12.138534+0000","last_peered":"2023-10-06T02:30:12.138534+0000","last_clean":"2023-10-06T02:30:12.138534+0000","last_became_active":"2023-10-06T02:30:11.147753+0000","last_became_peered":"2023-10-06T02:30:11.147753+0000","last_unstale":"2023-10-06T02:30:12.138534+0000","last_undegraded":"2023-10-06T02:30:12.138534+0000","last_fullsized":"2023-10-06T02:30:12.138534+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_clean_scrub_stamp":"2023-10-06T02:30:10.108419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:27.763054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.162784+0000","last_change":"2023-10-06T02:30:12.158709+0000","last_active":"2023-10-06T02:30:12.162784+0000","last_peered":"2023-10-06T02:30:12.162784+0000","last_clean":"2023-10-06T02:30:12.162784+0000","last_became_active":"2023-10-06T02:30:12.158441+0000","last_became_peered":"2023-10-06T02:30:12.158441+0000","last_unstale":"2023-10-06T02:30:12.162784+0000","last_undegraded":"2023-10-06T02:30:12.162784+0000","last_fullsized":"2023-10-06T02:30:12.162784+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:37:15.648699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.135106+0000","last_change":"2023-10-06T02:30:11.140734+0000","last_active":"2023-10-06T02:30:12.135106+0000","last_peered":"2023-10-06T02:30:12.135106+0000","last_clean":"2023-10-06T02:30:12.135106+0000","last_became_active":"2023-10-06T02:30:11.140405+0000","last_became_peered":"2023-10-06T02:30:11.140405+0000","last_unstale":"2023-10-06T02:30:12.135106+0000","last_undegraded":"2023-10-06T02:30:12.135106+0000","last_fullsized":"2023-10-06T02:30:12.135106+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_clean_scrub_stamp":"2023-10-06T02:30:10.108419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:02:16.493749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.164550+0000","last_change":"2023-10-06T02:30:12.162226+0000","last_active":"2023-10-06T02:30:12.164550+0000","last_peered":"2023-10-06T02:30:12.164550+0000","last_clean":"2023-10-06T02:30:12.164550+0000","last_became_active":"2023-10-06T02:30:12.161941+0000","last_became_peered":"2023-10-06T02:30:12.161941+0000","last_unstale":"2023-10-06T02:30:12.164550+0000","last_undegraded":"2023-10-06T02:30:12.164550+0000","last_fullsized":"2023-10-06T02:30:12.164550+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:09:06.895098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.138333+0000","last_change":"2023-10-06T02:30:11.146212+0000","last_active":"2023-10-06T02:30:12.138333+0000","last_peered":"2023-10-06T02:30:12.138333+0000","last_clean":"2023-10-06T02:30:12.138333+0000","last_became_active":"2023-10-06T02:30:11.145902+0000","last_became_peered":"2023-10-06T02:30:11.145902+0000","last_unstale":"2023-10-06T02:30:12.138333+0000","last_undegraded":"2023-10-06T02:30:12.138333+0000","last_fullsized":"2023-10-06T02:30:12.138333+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_clean_scrub_stamp":"2023-10-06T02:30:10.108419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:00:07.252457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.145544+0000","last_change":"2023-10-06T02:30:12.139280+0000","last_active":"2023-10-06T02:30:12.145544+0000","last_peered":"2023-10-06T02:30:12.145544+0000","last_clean":"2023-10-06T02:30:12.145544+0000","last_became_active":"2023-10-06T02:30:12.139010+0000","last_became_peered":"2023-10-06T02:30:12.139010+0000","last_unstale":"2023-10-06T02:30:12.145544+0000","last_undegraded":"2023-10-06T02:30:12.145544+0000","last_fullsized":"2023-10-06T02:30:12.145544+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:27:16.232913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.7","version":"131'2","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.163679+0000","last_change":"2023-10-06T02:30:11.153790+0000","last_active":"2023-10-06T02:30:12.163679+0000","last_peered":"2023-10-06T02:30:12.163679+0000","last_clean":"2023-10-06T02:30:12.163679+0000","last_became_active":"2023-10-06T02:30:11.153482+0000","last_became_peered":"2023-10-06T02:30:11.153482+0000","last_unstale":"2023-10-06T02:30:12.163679+0000","last_undegraded":"2023-10-06T02:30:12.163679+0000","last_fullsized":"2023-10-06T02:30:12.163679+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_clean_scrub_stamp":"2023-10-06T02:30:10.108419+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:13:34.004147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.159524+0000","last_change":"2023-10-06T02:30:12.157967+0000","last_active":"2023-10-06T02:30:12.159524+0000","last_peered":"2023-10-06T02:30:12.159524+0000","last_clean":"2023-10-06T02:30:12.159524+0000","last_became_active":"2023-10-06T02:30:12.157656+0000","last_became_peered":"2023-10-06T02:30:12.157656+0000","last_unstale":"2023-10-06T02:30:12.159524+0000","last_undegraded":"2023-10-06T02:30:12.159524+0000","last_fullsized":"2023-10-06T02:30:12.159524+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:40:34.054488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.135582+0000","last_change":"2023-10-06T02:30:11.149211+0000","last_active":"2023-10-06T02:30:12.135582+0000","last_peered":"2023-10-06T02:30:12.135582+0000","last_clean":"2023-10-06T02:30:12.135582+0000","last_became_active":"2023-10-06T02:30:11.148961+0000","last_became_peered":"2023-10-06T02:30:11.148961+0000","last_unstale":"2023-10-06T02:30:12.135582+0000","last_undegraded":"2023-10-06T02:30:12.135582+0000","last_fullsized":"2023-10-06T02:30:12.135582+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_clean_scrub_stamp":"2023-10-06T02:30:10.108419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:39:22.980561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.148989+0000","last_change":"2023-10-06T02:30:12.145707+0000","last_active":"2023-10-06T02:30:12.148989+0000","last_peered":"2023-10-06T02:30:12.148989+0000","last_clean":"2023-10-06T02:30:12.148989+0000","last_became_active":"2023-10-06T02:30:12.145090+0000","last_became_peered":"2023-10-06T02:30:12.145090+0000","last_unstale":"2023-10-06T02:30:12.148989+0000","last_undegraded":"2023-10-06T02:30:12.148989+0000","last_fullsized":"2023-10-06T02:30:12.148989+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:31:03.185895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.140076+0000","last_change":"2023-10-06T02:30:11.154287+0000","last_active":"2023-10-06T02:30:12.140076+0000","last_peered":"2023-10-06T02:30:12.140076+0000","last_clean":"2023-10-06T02:30:12.140076+0000","last_became_active":"2023-10-06T02:30:11.153901+0000","last_became_peered":"2023-10-06T02:30:11.153901+0000","last_unstale":"2023-10-06T02:30:12.140076+0000","last_undegraded":"2023-10-06T02:30:12.140076+0000","last_fullsized":"2023-10-06T02:30:12.140076+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_clean_scrub_stamp":"2023-10-06T02:30:10.108419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:25:26.473147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.163559+0000","last_change":"2023-10-06T02:30:12.156545+0000","last_active":"2023-10-06T02:30:12.163559+0000","last_peered":"2023-10-06T02:30:12.163559+0000","last_clean":"2023-10-06T02:30:12.163559+0000","last_became_active":"2023-10-06T02:30:12.156236+0000","last_became_peered":"2023-10-06T02:30:12.156236+0000","last_unstale":"2023-10-06T02:30:12.163559+0000","last_undegraded":"2023-10-06T02:30:12.163559+0000","last_fullsized":"2023-10-06T02:30:12.163559+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:43:28.212045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.130185+0000","last_change":"2023-10-06T02:30:11.145279+0000","last_active":"2023-10-06T02:30:12.130185+0000","last_peered":"2023-10-06T02:30:12.130185+0000","last_clean":"2023-10-06T02:30:12.130185+0000","last_became_active":"2023-10-06T02:30:11.144903+0000","last_became_peered":"2023-10-06T02:30:11.144903+0000","last_unstale":"2023-10-06T02:30:12.130185+0000","last_undegraded":"2023-10-06T02:30:12.130185+0000","last_fullsized":"2023-10-06T02:30:12.130185+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_clean_scrub_stamp":"2023-10-06T02:30:10.108419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:35:03.975595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.153445+0000","last_change":"2023-10-06T02:30:12.146323+0000","last_active":"2023-10-06T02:30:12.153445+0000","last_peered":"2023-10-06T02:30:12.153445+0000","last_clean":"2023-10-06T02:30:12.153445+0000","last_became_active":"2023-10-06T02:30:12.145996+0000","last_became_peered":"2023-10-06T02:30:12.145996+0000","last_unstale":"2023-10-06T02:30:12.153445+0000","last_undegraded":"2023-10-06T02:30:12.153445+0000","last_fullsized":"2023-10-06T02:30:12.153445+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:41:55.119067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.133902+0000","last_change":"2023-10-06T02:30:11.148696+0000","last_active":"2023-10-06T02:30:12.133902+0000","last_peered":"2023-10-06T02:30:12.133902+0000","last_clean":"2023-10-06T02:30:12.133902+0000","last_became_active":"2023-10-06T02:30:11.148290+0000","last_became_peered":"2023-10-06T02:30:11.148290+0000","last_unstale":"2023-10-06T02:30:12.133902+0000","last_undegraded":"2023-10-06T02:30:12.133902+0000","last_fullsized":"2023-10-06T02:30:12.133902+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_clean_scrub_stamp":"2023-10-06T02:30:10.108419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:00:07.252457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.162145+0000","last_change":"2023-10-06T02:30:12.158638+0000","last_active":"2023-10-06T02:30:12.162145+0000","last_peered":"2023-10-06T02:30:12.162145+0000","last_clean":"2023-10-06T02:30:12.162145+0000","last_became_active":"2023-10-06T02:30:12.158365+0000","last_became_peered":"2023-10-06T02:30:12.158365+0000","last_unstale":"2023-10-06T02:30:12.162145+0000","last_undegraded":"2023-10-06T02:30:12.162145+0000","last_fullsized":"2023-10-06T02:30:12.162145+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:29:35.838267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.137724+0000","last_change":"2023-10-06T02:30:11.150259+0000","last_active":"2023-10-06T02:30:12.137724+0000","last_peered":"2023-10-06T02:30:12.137724+0000","last_clean":"2023-10-06T02:30:12.137724+0000","last_became_active":"2023-10-06T02:30:11.149139+0000","last_became_peered":"2023-10-06T02:30:11.149139+0000","last_unstale":"2023-10-06T02:30:12.137724+0000","last_undegraded":"2023-10-06T02:30:12.137724+0000","last_fullsized":"2023-10-06T02:30:12.137724+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_clean_scrub_stamp":"2023-10-06T02:30:10.108419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:12:27.532273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.149793+0000","last_change":"2023-10-06T02:30:12.147136+0000","last_active":"2023-10-06T02:30:12.149793+0000","last_peered":"2023-10-06T02:30:12.149793+0000","last_clean":"2023-10-06T02:30:12.149793+0000","last_became_active":"2023-10-06T02:30:12.146817+0000","last_became_peered":"2023-10-06T02:30:12.146817+0000","last_unstale":"2023-10-06T02:30:12.149793+0000","last_undegraded":"2023-10-06T02:30:12.149793+0000","last_fullsized":"2023-10-06T02:30:12.149793+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:16:11.270014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.132118+0000","last_change":"2023-10-06T02:30:11.152607+0000","last_active":"2023-10-06T02:30:12.132118+0000","last_peered":"2023-10-06T02:30:12.132118+0000","last_clean":"2023-10-06T02:30:12.132118+0000","last_became_active":"2023-10-06T02:30:11.152343+0000","last_became_peered":"2023-10-06T02:30:11.152343+0000","last_unstale":"2023-10-06T02:30:12.132118+0000","last_undegraded":"2023-10-06T02:30:12.132118+0000","last_fullsized":"2023-10-06T02:30:12.132118+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_clean_scrub_stamp":"2023-10-06T02:30:10.108419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:44:23.481085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.159448+0000","last_change":"2023-10-06T02:30:12.155769+0000","last_active":"2023-10-06T02:30:12.159448+0000","last_peered":"2023-10-06T02:30:12.159448+0000","last_clean":"2023-10-06T02:30:12.159448+0000","last_became_active":"2023-10-06T02:30:12.155381+0000","last_became_peered":"2023-10-06T02:30:12.155381+0000","last_unstale":"2023-10-06T02:30:12.159448+0000","last_undegraded":"2023-10-06T02:30:12.159448+0000","last_fullsized":"2023-10-06T02:30:12.159448+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:15:45.887824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.150680+0000","last_change":"2023-10-06T02:30:11.152195+0000","last_active":"2023-10-06T02:30:12.150680+0000","last_peered":"2023-10-06T02:30:12.150680+0000","last_clean":"2023-10-06T02:30:12.150680+0000","last_became_active":"2023-10-06T02:30:11.151899+0000","last_became_peered":"2023-10-06T02:30:11.151899+0000","last_unstale":"2023-10-06T02:30:12.150680+0000","last_undegraded":"2023-10-06T02:30:12.150680+0000","last_fullsized":"2023-10-06T02:30:12.150680+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_clean_scrub_stamp":"2023-10-06T02:30:10.108419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:33:01.794705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.162876+0000","last_change":"2023-10-06T02:30:12.158963+0000","last_active":"2023-10-06T02:30:12.162876+0000","last_peered":"2023-10-06T02:30:12.162876+0000","last_clean":"2023-10-06T02:30:12.162876+0000","last_became_active":"2023-10-06T02:30:12.158657+0000","last_became_peered":"2023-10-06T02:30:12.158657+0000","last_unstale":"2023-10-06T02:30:12.162876+0000","last_undegraded":"2023-10-06T02:30:12.162876+0000","last_fullsized":"2023-10-06T02:30:12.162876+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:31:38.983594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"131'8","reported_seq":18,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.170847+0000","last_change":"2023-10-06T02:30:11.149212+0000","last_active":"2023-10-06T02:30:12.170847+0000","last_peered":"2023-10-06T02:30:12.170847+0000","last_clean":"2023-10-06T02:30:12.170847+0000","last_became_active":"2023-10-06T02:30:11.148926+0000","last_became_peered":"2023-10-06T02:30:11.148926+0000","last_unstale":"2023-10-06T02:30:12.170847+0000","last_undegraded":"2023-10-06T02:30:12.170847+0000","last_fullsized":"2023-10-06T02:30:12.170847+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_clean_scrub_stamp":"2023-10-06T02:30:10.108419+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:27.763054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":106,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.144885+0000","last_change":"2023-10-06T02:23:01.992935+0000","last_active":"2023-10-06T02:30:12.144885+0000","last_peered":"2023-10-06T02:30:12.144885+0000","last_clean":"2023-10-06T02:30:12.144885+0000","last_became_active":"2023-10-06T02:23:01.992568+0000","last_became_peered":"2023-10-06T02:23:01.992568+0000","last_unstale":"2023-10-06T02:30:12.144885+0000","last_undegraded":"2023-10-06T02:30:12.144885+0000","last_fullsized":"2023-10-06T02:30:12.144885+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:06.212251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.153199+0000","last_change":"2023-10-06T02:30:12.143202+0000","last_active":"2023-10-06T02:30:12.153199+0000","last_peered":"2023-10-06T02:30:12.153199+0000","last_clean":"2023-10-06T02:30:12.153199+0000","last_became_active":"2023-10-06T02:30:12.142895+0000","last_became_peered":"2023-10-06T02:30:12.142895+0000","last_unstale":"2023-10-06T02:30:12.153199+0000","last_undegraded":"2023-10-06T02:30:12.153199+0000","last_fullsized":"2023-10-06T02:30:12.153199+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:49:23.533957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.140286+0000","last_change":"2023-10-06T02:30:11.143304+0000","last_active":"2023-10-06T02:30:12.140286+0000","last_peered":"2023-10-06T02:30:12.140286+0000","last_clean":"2023-10-06T02:30:12.140286+0000","last_became_active":"2023-10-06T02:30:11.142989+0000","last_became_peered":"2023-10-06T02:30:11.142989+0000","last_unstale":"2023-10-06T02:30:12.140286+0000","last_undegraded":"2023-10-06T02:30:12.140286+0000","last_fullsized":"2023-10-06T02:30:12.140286+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_clean_scrub_stamp":"2023-10-06T02:30:10.108419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:42:03.384901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":106,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.140109+0000","last_change":"2023-10-06T02:23:01.989934+0000","last_active":"2023-10-06T02:30:12.140109+0000","last_peered":"2023-10-06T02:30:12.140109+0000","last_clean":"2023-10-06T02:30:12.140109+0000","last_became_active":"2023-10-06T02:23:01.989706+0000","last_became_peered":"2023-10-06T02:23:01.989706+0000","last_unstale":"2023-10-06T02:30:12.140109+0000","last_undegraded":"2023-10-06T02:30:12.140109+0000","last_fullsized":"2023-10-06T02:30:12.140109+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:06.212251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.149839+0000","last_change":"2023-10-06T02:30:12.147222+0000","last_active":"2023-10-06T02:30:12.149839+0000","last_peered":"2023-10-06T02:30:12.149839+0000","last_clean":"2023-10-06T02:30:12.149839+0000","last_became_active":"2023-10-06T02:30:12.146914+0000","last_became_peered":"2023-10-06T02:30:12.146914+0000","last_unstale":"2023-10-06T02:30:12.149839+0000","last_undegraded":"2023-10-06T02:30:12.149839+0000","last_fullsized":"2023-10-06T02:30:12.149839+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:43:32.260076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.139875+0000","last_change":"2023-10-06T02:30:11.149874+0000","last_active":"2023-10-06T02:30:12.139875+0000","last_peered":"2023-10-06T02:30:12.139875+0000","last_clean":"2023-10-06T02:30:12.139875+0000","last_became_active":"2023-10-06T02:30:11.149557+0000","last_became_peered":"2023-10-06T02:30:11.149557+0000","last_unstale":"2023-10-06T02:30:12.139875+0000","last_undegraded":"2023-10-06T02:30:12.139875+0000","last_fullsized":"2023-10-06T02:30:12.139875+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_clean_scrub_stamp":"2023-10-06T02:30:10.108419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:33:42.306461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":101,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.151028+0000","last_change":"2023-10-06T02:23:02.054069+0000","last_active":"2023-10-06T02:30:12.151028+0000","last_peered":"2023-10-06T02:30:12.151028+0000","last_clean":"2023-10-06T02:30:12.151028+0000","last_became_active":"2023-10-06T02:23:02.053739+0000","last_became_peered":"2023-10-06T02:23:02.053739+0000","last_unstale":"2023-10-06T02:30:12.151028+0000","last_undegraded":"2023-10-06T02:30:12.151028+0000","last_fullsized":"2023-10-06T02:30:12.151028+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:57:48.822608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.161236+0000","last_change":"2023-10-06T02:30:12.159090+0000","last_active":"2023-10-06T02:30:12.161236+0000","last_peered":"2023-10-06T02:30:12.161236+0000","last_clean":"2023-10-06T02:30:12.161236+0000","last_became_active":"2023-10-06T02:30:12.158737+0000","last_became_peered":"2023-10-06T02:30:12.158737+0000","last_unstale":"2023-10-06T02:30:12.161236+0000","last_undegraded":"2023-10-06T02:30:12.161236+0000","last_fullsized":"2023-10-06T02:30:12.161236+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:26:19.550623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.146192+0000","last_change":"2023-10-06T02:30:11.153794+0000","last_active":"2023-10-06T02:30:12.146192+0000","last_peered":"2023-10-06T02:30:12.146192+0000","last_clean":"2023-10-06T02:30:12.146192+0000","last_became_active":"2023-10-06T02:30:11.153528+0000","last_became_peered":"2023-10-06T02:30:11.153528+0000","last_unstale":"2023-10-06T02:30:12.146192+0000","last_undegraded":"2023-10-06T02:30:12.146192+0000","last_fullsized":"2023-10-06T02:30:12.146192+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_clean_scrub_stamp":"2023-10-06T02:30:10.108419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:40:08.038564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":101,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.150799+0000","last_change":"2023-10-06T02:23:01.990773+0000","last_active":"2023-10-06T02:30:12.150799+0000","last_peered":"2023-10-06T02:30:12.150799+0000","last_clean":"2023-10-06T02:30:12.150799+0000","last_became_active":"2023-10-06T02:23:01.990511+0000","last_became_peered":"2023-10-06T02:23:01.990511+0000","last_unstale":"2023-10-06T02:30:12.150799+0000","last_undegraded":"2023-10-06T02:30:12.150799+0000","last_fullsized":"2023-10-06T02:30:12.150799+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:42:54.797731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.158423+0000","last_change":"2023-10-06T02:30:12.151106+0000","last_active":"2023-10-06T02:30:12.158423+0000","last_peered":"2023-10-06T02:30:12.158423+0000","last_clean":"2023-10-06T02:30:12.158423+0000","last_became_active":"2023-10-06T02:30:12.148744+0000","last_became_peered":"2023-10-06T02:30:12.148744+0000","last_unstale":"2023-10-06T02:30:12.158423+0000","last_undegraded":"2023-10-06T02:30:12.158423+0000","last_fullsized":"2023-10-06T02:30:12.158423+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:40:09.048569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.14","version":"131'6","reported_seq":16,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.169574+0000","last_change":"2023-10-06T02:30:11.145555+0000","last_active":"2023-10-06T02:30:12.169574+0000","last_peered":"2023-10-06T02:30:12.169574+0000","last_clean":"2023-10-06T02:30:12.169574+0000","last_became_active":"2023-10-06T02:30:11.144779+0000","last_became_peered":"2023-10-06T02:30:11.144779+0000","last_unstale":"2023-10-06T02:30:12.169574+0000","last_undegraded":"2023-10-06T02:30:12.169574+0000","last_fullsized":"2023-10-06T02:30:12.169574+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_clean_scrub_stamp":"2023-10-06T02:30:10.108419+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:56:34.982997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"76'128","reported_seq":293,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.150890+0000","last_change":"2023-10-06T02:22:55.032519+0000","last_active":"2023-10-06T02:30:12.150890+0000","last_peered":"2023-10-06T02:30:12.150890+0000","last_clean":"2023-10-06T02:30:12.150890+0000","last_became_active":"2023-10-06T02:22:55.031367+0000","last_became_peered":"2023-10-06T02:22:55.031367+0000","last_unstale":"2023-10-06T02:30:12.150890+0000","last_undegraded":"2023-10-06T02:30:12.150890+0000","last_fullsized":"2023-10-06T02:30:12.150890+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:22:53.309097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:22:53.309097+0000","last_clean_scrub_stamp":"2023-10-06T02:22:53.309097+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:53:46.520909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":122,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.134958+0000","last_change":"2023-10-06T02:23:01.988140+0000","last_active":"2023-10-06T02:30:12.134958+0000","last_peered":"2023-10-06T02:30:12.134958+0000","last_clean":"2023-10-06T02:30:12.134958+0000","last_became_active":"2023-10-06T02:23:01.987769+0000","last_became_peered":"2023-10-06T02:23:01.987769+0000","last_unstale":"2023-10-06T02:30:12.134958+0000","last_undegraded":"2023-10-06T02:30:12.134958+0000","last_fullsized":"2023-10-06T02:30:12.134958+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:20:08.391912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.162608+0000","last_change":"2023-10-06T02:30:12.160607+0000","last_active":"2023-10-06T02:30:12.162608+0000","last_peered":"2023-10-06T02:30:12.162608+0000","last_clean":"2023-10-06T02:30:12.162608+0000","last_became_active":"2023-10-06T02:30:12.160328+0000","last_became_peered":"2023-10-06T02:30:12.160328+0000","last_unstale":"2023-10-06T02:30:12.162608+0000","last_undegraded":"2023-10-06T02:30:12.162608+0000","last_fullsized":"2023-10-06T02:30:12.162608+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:27:37.502858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.135672+0000","last_change":"2023-10-06T02:30:11.148714+0000","last_active":"2023-10-06T02:30:12.135672+0000","last_peered":"2023-10-06T02:30:12.135672+0000","last_clean":"2023-10-06T02:30:12.135672+0000","last_became_active":"2023-10-06T02:30:11.148415+0000","last_became_peered":"2023-10-06T02:30:11.148415+0000","last_unstale":"2023-10-06T02:30:12.135672+0000","last_undegraded":"2023-10-06T02:30:12.135672+0000","last_fullsized":"2023-10-06T02:30:12.135672+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_clean_scrub_stamp":"2023-10-06T02:30:10.108419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:34:11.589155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":113,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.137202+0000","last_change":"2023-10-06T02:23:02.054338+0000","last_active":"2023-10-06T02:30:12.137202+0000","last_peered":"2023-10-06T02:30:12.137202+0000","last_clean":"2023-10-06T02:30:12.137202+0000","last_became_active":"2023-10-06T02:23:02.053991+0000","last_became_peered":"2023-10-06T02:23:02.053991+0000","last_unstale":"2023-10-06T02:30:12.137202+0000","last_undegraded":"2023-10-06T02:30:12.137202+0000","last_fullsized":"2023-10-06T02:30:12.137202+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:39:30.192838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.154733+0000","last_change":"2023-10-06T02:30:12.143667+0000","last_active":"2023-10-06T02:30:12.154733+0000","last_peered":"2023-10-06T02:30:12.154733+0000","last_clean":"2023-10-06T02:30:12.154733+0000","last_became_active":"2023-10-06T02:30:12.143157+0000","last_became_peered":"2023-10-06T02:30:12.143157+0000","last_unstale":"2023-10-06T02:30:12.154733+0000","last_undegraded":"2023-10-06T02:30:12.154733+0000","last_fullsized":"2023-10-06T02:30:12.154733+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:59:54.022935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"131'3","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.163645+0000","last_change":"2023-10-06T02:30:11.138337+0000","last_active":"2023-10-06T02:30:12.163645+0000","last_peered":"2023-10-06T02:30:12.163645+0000","last_clean":"2023-10-06T02:30:12.163645+0000","last_became_active":"2023-10-06T02:30:11.138046+0000","last_became_peered":"2023-10-06T02:30:11.138046+0000","last_unstale":"2023-10-06T02:30:12.163645+0000","last_undegraded":"2023-10-06T02:30:12.163645+0000","last_fullsized":"2023-10-06T02:30:12.163645+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_clean_scrub_stamp":"2023-10-06T02:30:10.108419+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:10:34.557245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":122,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.132709+0000","last_change":"2023-10-06T02:23:01.988631+0000","last_active":"2023-10-06T02:30:12.132709+0000","last_peered":"2023-10-06T02:30:12.132709+0000","last_clean":"2023-10-06T02:30:12.132709+0000","last_became_active":"2023-10-06T02:23:01.988303+0000","last_became_peered":"2023-10-06T02:23:01.988303+0000","last_unstale":"2023-10-06T02:30:12.132709+0000","last_undegraded":"2023-10-06T02:30:12.132709+0000","last_fullsized":"2023-10-06T02:30:12.132709+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:38.211466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.161337+0000","last_change":"2023-10-06T02:30:12.155413+0000","last_active":"2023-10-06T02:30:12.161337+0000","last_peered":"2023-10-06T02:30:12.161337+0000","last_clean":"2023-10-06T02:30:12.161337+0000","last_became_active":"2023-10-06T02:30:12.155129+0000","last_became_peered":"2023-10-06T02:30:12.155129+0000","last_unstale":"2023-10-06T02:30:12.161337+0000","last_undegraded":"2023-10-06T02:30:12.161337+0000","last_fullsized":"2023-10-06T02:30:12.161337+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:11.118423+0000","last_clean_scrub_stamp":"2023-10-06T02:30:11.118423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:33:02.804709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.13","version":"131'2","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.167649+0000","last_change":"2023-10-06T02:30:11.144283+0000","last_active":"2023-10-06T02:30:12.167649+0000","last_peered":"2023-10-06T02:30:12.167649+0000","last_clean":"2023-10-06T02:30:12.167649+0000","last_became_active":"2023-10-06T02:30:11.144037+0000","last_became_peered":"2023-10-06T02:30:11.144037+0000","last_unstale":"2023-10-06T02:30:12.167649+0000","last_undegraded":"2023-10-06T02:30:12.167649+0000","last_fullsized":"2023-10-06T02:30:12.167649+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:10.108419+0000","last_clean_scrub_stamp":"2023-10-06T02:30:10.108419+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:25:43.260463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":106,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T02:30:12.137700+0000","last_change":"2023-10-06T02:23:02.054039+0000","last_active":"2023-10-06T02:30:12.137700+0000","last_peered":"2023-10-06T02:30:12.137700+0000","last_clean":"2023-10-06T02:30:12.137700+0000","last_became_active":"2023-10-06T02:23:02.053742+0000","last_became_peered":"2023-10-06T02:23:02.053742+0000","last_unstale":"2023-10-06T02:30:12.137700+0000","last_undegraded":"2023-10-06T02:30:12.137700+0000","last_fullsized":"2023-10-06T02:30:12.137700+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:58:18.846475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574941,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303584,"kb_used_data":2200,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068256,"statfs":{"total":96636764160,"available":96325894144,"internally_reserved":0,"allocated":2252800,"data_stored":1360827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:30:12 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.64100000000000001,"15min":0.63600000000000001},"min":{"1min":0.375,"5min":0.318,"15min":0.318},"max":{"1min":0.78600000000000003,"5min":1.169,"15min":1.169},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.63900000000000001,"15min":0.61199999999999999},"min":{"1min":0.45300000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.84699999999999998,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.70699999999999996}]},{"osd":1,"last update":"Fri Oct 6 02:30:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.73199999999999998,"15min":0.69399999999999995},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.40300000000000002},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.147},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.67400000000000004,"15min":0.67200000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.82399999999999995,"5min":1.129,"15min":1.129},"last":0.754}]},{"osd":2,"last update":"Fri Oct 6 02:30:12 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.67500000000000004,"15min":0.68700000000000006},"min":{"1min":0.44,"5min":0.375,"15min":0.375},"max":{"1min":0.87,"5min":1.238,"15min":1.238},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.69099999999999995,"15min":0.68999999999999995},"min":{"1min":0.54000000000000004,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.77700000000000002}]},{"osd":3,"last update":"Fri Oct 6 02:30:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.45700000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.79400000000000004,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.76300000000000001,"15min":0.77300000000000002},"min":{"1min":0.55900000000000005,"5min":0.497,"15min":0.497},"max":{"1min":1.383,"5min":1.383,"15min":1.383},"last":0.86099999999999999}]},{"osd":4,"last update":"Fri Oct 6 02:30:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.59199999999999997,"15min":0.67200000000000004},"min":{"1min":0.42899999999999999,"5min":0.41099999999999998,"15min":0.35099999999999998},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.2849999999999999},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.67200000000000004,"15min":0.747},"min":{"1min":0.41199999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.96499999999999997,"5min":1.51,"15min":1.51},"last":0.52100000000000002}]},{"osd":5,"last update":"Fri Oct 6 02:30:12 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.69199999999999995,"15min":0.72699999999999998},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.67600000000000005,"15min":0.71999999999999997},"min":{"1min":0.47199999999999998,"5min":0.442,"15min":0.442},"max":{"1min":0.874,"5min":0.98099999999999998,"15min":1.0289999999999999},"last":0.48599999999999999}]},{"osd":6,"last update":"Fri Oct 6 02:30:12 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.76100000000000001,"15min":0.78200000000000003},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.97599999999999998,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.77800000000000002,"15min":0.80600000000000005},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.872,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.46000000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574941,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303004,"kb_used_data":1620,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068836,"statfs":{"total":96636764160,"available":96326488064,"internally_reserved":0,"allocated":1658880,"data_stored":770507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:30:06 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.68700000000000006,"15min":0.66700000000000004},"min":{"1min":0.501,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.88500000000000001,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.72999999999999998,"15min":0.70299999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.91100000000000003,"5min":1.0229999999999999,"15min":1.119},"last":0.66100000000000003}]},{"osd":1,"last update":"Fri Oct 6 02:30:06 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.68100000000000005,"15min":0.65100000000000002},"min":{"1min":0.45200000000000001,"5min":0.40999999999999998,"15min":0.40699999999999997},"max":{"1min":0.96499999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.68500000000000005,"15min":0.67100000000000004},"min":{"1min":0.437,"5min":0.41899999999999998,"15min":0.40999999999999998},"max":{"1min":0.84399999999999997,"5min":0.997,"15min":0.997},"last":0.49399999999999999}]},{"osd":2,"last update":"Fri Oct 6 02:30:06 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.72899999999999998,"15min":0.70799999999999996},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.92100000000000004,"5min":1.0449999999999999,"15min":1.137},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.74399999999999999,"15min":0.71799999999999997},"min":{"1min":0.55600000000000005,"5min":0.45300000000000001,"15min":0.40300000000000002},"max":{"1min":0.95299999999999996,"5min":1.077,"15min":1.077},"last":0.68799999999999994}]},{"osd":3,"last update":"Fri Oct 6 02:30:06 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.71199999999999997,"15min":0.67100000000000004},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.94099999999999995,"5min":1.056,"15min":1.056},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.73299999999999998,"15min":0.70899999999999996},"min":{"1min":0.56699999999999995,"5min":0.49299999999999999,"15min":0.45900000000000002},"max":{"1min":0.97799999999999998,"5min":0.97999999999999998,"15min":0.98499999999999999},"last":0.56399999999999995}]},{"osd":4,"last update":"Fri Oct 6 02:30:06 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.65900000000000003,"15min":0.64200000000000002},"min":{"1min":0.42499999999999999,"5min":0.36199999999999999,"15min":0.34300000000000003},"max":{"1min":0.93000000000000005,"5min":0.94799999999999995,"15min":1.095},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.69999999999999996,"15min":0.66900000000000004},"min":{"1min":0.52200000000000002,"5min":0.41699999999999998,"15min":0.40999999999999998},"max":{"1min":0.91900000000000004,"5min":1.002,"15min":1.29},"last":0.51600000000000001}]},{"osd":5,"last update":"Fri Oct 6 02:30:06 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.67700000000000005,"15min":0.69799999999999995},"min":{"1min":0.438,"5min":0.438,"15min":0.39700000000000002},"max":{"1min":1.014,"5min":1.014,"15min":1.2929999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.70799999999999996,"15min":0.70599999999999996},"min":{"1min":0.48699999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.01,"5min":1.01,"15min":1.02},"last":0.59099999999999997}]},{"osd":7,"last update":"Fri Oct 6 02:30:06 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.74099999999999999,"15min":0.749},"min":{"1min":0.48399999999999999,"5min":0.41499999999999998,"15min":0.36799999999999999},"max":{"1min":1,"5min":1,"15min":1.458},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.76500000000000001,"15min":0.76300000000000001},"min":{"1min":0.50700000000000001,"5min":0.47299999999999998,"15min":0.45900000000000002},"max":{"1min":1.046,"5min":1.103,"15min":1.2470000000000001},"last":0.72999999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303004,"kb_used_data":1620,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068836,"statfs":{"total":96636764160,"available":96326488064,"internally_reserved":0,"allocated":1658880,"data_stored":770507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:30:02 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.42999999999999999,"15min":0.45300000000000001},"min":{"1min":0.30599999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.59599999999999997,"5min":0.65500000000000003,"15min":1.2230000000000001},"last":0.432},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.46999999999999997,"15min":0.51400000000000001},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.58899999999999997,"5min":0.71799999999999997,"15min":1.788},"last":0.40000000000000002}]},{"osd":2,"last update":"Fri Oct 6 02:30:02 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.44,"15min":0.50600000000000001},"min":{"1min":0.33800000000000002,"5min":0.307,"15min":0.28299999999999997},"max":{"1min":0.53800000000000003,"5min":0.60899999999999999,"15min":2.0840000000000001},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.504,"15min":0.54700000000000004},"min":{"1min":0.38700000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":1.6699999999999999},"last":0.48999999999999999}]},{"osd":3,"last update":"Fri Oct 6 02:30:02 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.59799999999999998,"15min":0.60999999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.72499999999999998,"5min":0.89900000000000002,"15min":1.266},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.54200000000000004,"15min":0.62},"min":{"1min":0.46899999999999997,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.69999999999999996,"5min":0.79400000000000004,"15min":1.861},"last":0.60099999999999998}]},{"osd":4,"last update":"Fri Oct 6 02:30:02 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.77300000000000002,"15min":0.83799999999999997},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.55900000000000005},"max":{"1min":0.93999999999999995,"5min":1.3580000000000001,"15min":1.9279999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.78400000000000003,"15min":0.84099999999999997},"min":{"1min":0.57399999999999995,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.90200000000000002,"5min":1.4339999999999999,"15min":2.0179999999999998},"last":0.67300000000000004}]},{"osd":5,"last update":"Fri Oct 6 02:30:02 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.73699999999999999,"15min":0.83699999999999997},"min":{"1min":0.54400000000000004,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.874,"5min":1.3069999999999999,"15min":1.9830000000000001},"last":0.622},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.748,"15min":0.80700000000000005},"min":{"1min":0.55400000000000005,"5min":0.53600000000000003,"15min":0.47299999999999998},"max":{"1min":0.91300000000000003,"5min":1.327,"15min":1.3380000000000001},"last":0.58499999999999996}]},{"osd":6,"last update":"Fri Oct 6 02:30:02 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.78400000000000003,"15min":0.84799999999999998},"min":{"1min":0.60899999999999999,"5min":0.52700000000000002,"15min":0.45600000000000002},"max":{"1min":1.012,"5min":1.0609999999999999,"15min":1.829},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.80000000000000004,"15min":0.88900000000000001},"min":{"1min":0.61099999999999999,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.93100000000000005,"5min":1.351,"15min":2.052},"last":0.71199999999999997}]},{"osd":7,"last update":"Fri Oct 6 02:30:02 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.76300000000000001,"15min":0.81999999999999995},"min":{"1min":0.59399999999999997,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.90400000000000003,"5min":1.3120000000000001,"15min":1.3340000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.77800000000000002,"15min":0.85699999999999998},"min":{"1min":0.51600000000000001,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.018,"5min":1.2110000000000001,"15min":1.895},"last":0.63900000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574941,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303604,"kb_used_data":2220,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068236,"statfs":{"total":96636764160,"available":96325873664,"internally_reserved":0,"allocated":2273280,"data_stored":1363019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 02:29:13 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.42699999999999999,"15min":0.45200000000000001},"min":{"1min":0.32700000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":1.0129999999999999},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.504,"5min":0.46500000000000002,"15min":0.47899999999999998},"min":{"1min":0.29799999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.9830000000000001,"5min":1.9830000000000001,"15min":1.9830000000000001},"last":0.44600000000000001}]},{"osd":2,"last update":"Fri Oct 6 02:29:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.505,"15min":0.51600000000000001},"min":{"1min":0.38,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":3.0339999999999998,"5min":3.0339999999999998,"15min":3.0339999999999998},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.503,"15min":0.51900000000000002},"min":{"1min":0.39200000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.51700000000000002}]},{"osd":3,"last update":"Fri Oct 6 02:29:13 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.48699999999999999,"15min":0.503},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.59399999999999997},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.028,"5min":1.028,"15min":1.1359999999999999},"last":0.57999999999999996}]},{"osd":4,"last update":"Fri Oct 6 02:29:13 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.755,"15min":0.79800000000000004},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":2.9940000000000002,"5min":2.9940000000000002,"15min":2.9940000000000002},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.75900000000000001,"15min":0.79700000000000004},"min":{"1min":0.56599999999999995,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.54800000000000004}]},{"osd":5,"last update":"Fri Oct 6 02:29:13 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.76000000000000001,"15min":0.78600000000000003},"min":{"1min":0.61599999999999999,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.503},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.79300000000000004,"15min":0.80300000000000005},"min":{"1min":0.58299999999999996,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":3.1110000000000002,"5min":3.1110000000000002,"15min":3.1110000000000002},"last":0.66000000000000003}]},{"osd":6,"last update":"Fri Oct 6 02:29:13 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.80700000000000005,"15min":0.82399999999999995},"min":{"1min":0.61399999999999999,"5min":0.57599999999999996,"15min":0.54600000000000004},"max":{"1min":3.1600000000000001,"5min":3.1600000000000001,"15min":3.1600000000000001},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77300000000000002,"15min":0.82399999999999995},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.56499999999999995}]},{"osd":7,"last update":"Fri Oct 6 02:29:13 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.79700000000000004,"15min":0.82599999999999996},"min":{"1min":0.60199999999999998,"5min":0.56899999999999995,"15min":0.56000000000000005},"max":{"1min":2.9369999999999998,"5min":2.9369999999999998,"15min":2.9369999999999998},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.77300000000000002,"15min":0.82099999999999995},"min":{"1min":0.60499999999999998,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.59899999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574942,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303008,"kb_used_data":1624,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068832,"statfs":{"total":96636764160,"available":96326483968,"internally_reserved":0,"allocated":1662976,"data_stored":770549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:30:10 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.43099999999999999,"15min":0.437},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.502,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.47299999999999998,"15min":0.48199999999999998},"min":{"1min":0.35799999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.64100000000000001,"5min":0.88700000000000001,"15min":1.0800000000000001},"last":0.52600000000000002}]},{"osd":1,"last update":"Fri Oct 6 02:30:10 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.47799999999999998,"15min":0.498},"min":{"1min":0.36299999999999999,"5min":0.33600000000000002,"15min":0.32400000000000001},"max":{"1min":0.55500000000000005,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.52700000000000002,"15min":0.58999999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34599999999999997},"max":{"1min":0.66000000000000003,"5min":1.778,"15min":1.778},"last":0.48399999999999999}]},{"osd":3,"last update":"Fri Oct 6 02:30:10 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.50900000000000001,"15min":0.52900000000000003},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.79500000000000004,"5min":0.83699999999999997,"15min":0.97499999999999998},"last":0.438},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.59299999999999997,"15min":0.61399999999999999},"min":{"1min":0.48099999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.77100000000000002,"5min":0.84699999999999998,"15min":1.028},"last":0.505}]},{"osd":4,"last update":"Fri Oct 6 02:30:10 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.751,"15min":0.77300000000000002},"min":{"1min":0.51500000000000001,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.97499999999999998,"5min":1.109,"15min":1.1319999999999999},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.78400000000000003,"15min":0.83499999999999996},"min":{"1min":0.57999999999999996,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.96099999999999997,"5min":1.907,"15min":1.907},"last":0.63800000000000001}]},{"osd":5,"last update":"Fri Oct 6 02:30:10 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.73999999999999999,"15min":0.79000000000000004},"min":{"1min":0.53200000000000003,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.97199999999999998,"5min":1.109,"15min":1.1850000000000001},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.76300000000000001,"15min":0.81000000000000005},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0129999999999999,"5min":1.153,"15min":1.258},"last":0.60499999999999998}]},{"osd":6,"last update":"Fri Oct 6 02:30:10 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.79500000000000004,"15min":0.83799999999999997},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.98699999999999999,"5min":1.827,"15min":1.827},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.77500000000000002,"15min":0.79300000000000004},"min":{"1min":0.60299999999999998,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0660000000000001,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.65600000000000003}]},{"osd":7,"last update":"Fri Oct 6 02:30:10 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.80500000000000005,"15min":0.86799999999999999},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0369999999999999,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.78600000000000003,"15min":0.82399999999999995},"min":{"1min":0.66100000000000003,"5min":0.55300000000000005,"15min":0.54800000000000004},"max":{"1min":1.01,"5min":1.125,"15min":1.1479999999999999},"last":0.61799999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574939,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303008,"kb_used_data":1624,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068832,"statfs":{"total":96636764160,"available":96326483968,"internally_reserved":0,"allocated":1662976,"data_stored":771025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:30:10 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.443,"15min":0.49399999999999999},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.56200000000000006,"5min":1.0489999999999999,"15min":2.024},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.49199999999999999,"15min":0.51500000000000001},"min":{"1min":0.311,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.754,"5min":0.84699999999999998,"15min":2.181},"last":0.41799999999999998}]},{"osd":1,"last update":"Fri Oct 6 02:30:10 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.46899999999999997,"15min":0.64100000000000001},"min":{"1min":0.34799999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.65100000000000002,"5min":0.66100000000000003,"15min":4.492},"last":0.443},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.52600000000000002,"15min":0.68700000000000006},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.76200000000000001,"5min":1.079,"15min":4.569},"last":0.67100000000000004}]},{"osd":2,"last update":"Fri Oct 6 02:30:10 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.627,"15min":0.65500000000000003},"min":{"1min":0.372,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.82599999999999996,"5min":1.024,"15min":2.0870000000000002},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.59299999999999997,"15min":0.63700000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.76800000000000002,"5min":0.94199999999999995,"15min":2.4260000000000002},"last":0.50900000000000001}]},{"osd":4,"last update":"Fri Oct 6 02:30:10 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.73799999999999999,"15min":0.81100000000000005},"min":{"1min":0.55000000000000004,"5min":0.502,"15min":0.44800000000000001},"max":{"1min":0.94199999999999995,"5min":1.0149999999999999,"15min":2.214},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.76400000000000001,"15min":0.80000000000000004},"min":{"1min":0.54400000000000004,"5min":0.50800000000000001,"15min":0.5},"max":{"1min":1.034,"5min":1.079,"15min":1.2869999999999999},"last":0.53000000000000003}]},{"osd":5,"last update":"Fri Oct 6 02:30:10 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.76300000000000001,"15min":0.80900000000000005},"min":{"1min":0.497,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.077,"5min":1.077,"15min":1.3029999999999999},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.752,"5min":0.76000000000000001,"15min":0.83499999999999996},"min":{"1min":0.60099999999999998,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.0029999999999999,"5min":1.0580000000000001,"15min":2.242},"last":0.64900000000000002}]},{"osd":6,"last update":"Fri Oct 6 02:30:10 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.77400000000000002,"15min":0.81100000000000005},"min":{"1min":0.625,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.98199999999999998,"5min":1.0489999999999999,"15min":1.357},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.80300000000000005,"15min":0.82499999999999996},"min":{"1min":0.60599999999999998,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.014,"5min":1.0529999999999999,"15min":1.3100000000000001},"last":0.75700000000000001}]},{"osd":7,"last update":"Fri Oct 6 02:30:10 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.78200000000000003,"15min":0.84399999999999997},"min":{"1min":0.57299999999999995,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.88400000000000001,"5min":1.077,"15min":2.27},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.79800000000000004,"15min":0.84399999999999997},"min":{"1min":0.57499999999999996,"5min":0.55600000000000005,"15min":0.53200000000000003},"max":{"1min":1.0029999999999999,"5min":1.0760000000000001,"15min":1.325},"last":0.65700000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574941,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303012,"kb_used_data":1628,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068828,"statfs":{"total":96636764160,"available":96326479872,"internally_reserved":0,"allocated":1667072,"data_stored":771973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:30:06 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68100000000000005,"15min":0.95799999999999996},"min":{"1min":0.46200000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":12.337},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68400000000000005,"15min":0.88200000000000001},"min":{"1min":0.495,"5min":0.44400000000000001,"15min":0.438},"max":{"1min":0.95299999999999996,"5min":1.3520000000000001,"15min":9.6539999999999999},"last":0.80700000000000005}]},{"osd":1,"last update":"Fri Oct 6 02:30:06 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.68000000000000005,"15min":0.69399999999999995},"min":{"1min":0.47799999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.042,"5min":1.129,"15min":3.181},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.70999999999999996,"15min":0.98299999999999998},"min":{"1min":0.46300000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.98799999999999999,"5min":0.995,"15min":12.946999999999999},"last":0.85199999999999998}]},{"osd":2,"last update":"Fri Oct 6 02:30:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.71599999999999997,"15min":1.006},"min":{"1min":0.47999999999999998,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.1319999999999999,"5min":1.2989999999999999,"15min":12.377000000000001},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.745,"5min":0.73799999999999999,"15min":0.85599999999999998},"min":{"1min":0.56899999999999995,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.93700000000000006,"5min":1.0529999999999999,"15min":6.3220000000000001},"last":0.79000000000000004}]},{"osd":3,"last update":"Fri Oct 6 02:30:06 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72899999999999998,"15min":0.95599999999999996},"min":{"1min":0.55300000000000005,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.96299999999999997,"5min":1.5229999999999999,"15min":9.7720000000000002},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71799999999999997,"15min":0.76800000000000002},"min":{"1min":0.54600000000000004,"5min":0.498,"15min":0.47999999999999998},"max":{"1min":0.90700000000000003,"5min":1.212,"15min":3.0840000000000001},"last":0.75}]},{"osd":5,"last update":"Fri Oct 6 02:30:06 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.68700000000000006,"15min":0.82899999999999996},"min":{"1min":0.42099999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.99199999999999999,"5min":1.02,"15min":6.2690000000000001},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.745,"5min":0.71199999999999997,"15min":0.997},"min":{"1min":0.45700000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.0589999999999999,"5min":1.4359999999999999,"15min":10.769},"last":0.53900000000000003}]},{"osd":6,"last update":"Fri Oct 6 02:30:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.68799999999999994,"15min":0.753},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.95699999999999996,"5min":1.4590000000000001,"15min":3.1349999999999998},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.76900000000000002,"15min":0.94999999999999996},"min":{"1min":0.47999999999999998,"5min":0.47799999999999998,"15min":0.45700000000000002},"max":{"1min":1.1579999999999999,"5min":1.3839999999999999,"15min":8.0899999999999999},"last":0.90800000000000003}]},{"osd":7,"last update":"Fri Oct 6 02:30:06 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.76800000000000002,"15min":1.032},"min":{"1min":0.50900000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.1870000000000001,"5min":1.494,"15min":9.7089999999999996},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.76600000000000001,"15min":0.97099999999999997},"min":{"1min":0.44400000000000001,"5min":0.374,"15min":0.374},"max":{"1min":1.133,"5min":1.2669999999999999,"15min":7.6719999999999997},"last":0.77700000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574941,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303000,"kb_used_data":1616,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068840,"statfs":{"total":96636764160,"available":96326492160,"internally_reserved":0,"allocated":1654784,"data_stored":770437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:30:04 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63900000000000001,"15min":0.627},"min":{"1min":0.40100000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.88200000000000001,"5min":0.97599999999999998,"15min":1.4279999999999999},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.66000000000000003,"15min":0.61599999999999999},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.38700000000000001},"max":{"1min":0.92700000000000005,"5min":1.099,"15min":1.099},"last":0.57799999999999996}]},{"osd":1,"last update":"Fri Oct 6 02:30:04 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.67000000000000004,"15min":0.65100000000000002},"min":{"1min":0.44500000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.85599999999999998,"5min":1.0660000000000001,"15min":1.089},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.68999999999999995,"15min":0.67500000000000004},"min":{"1min":0.49099999999999999,"5min":0.40699999999999997,"15min":0.40500000000000003},"max":{"1min":0.93600000000000005,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.78200000000000003}]},{"osd":2,"last update":"Fri Oct 6 02:30:04 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.69799999999999995,"15min":0.67300000000000004},"min":{"1min":0.41299999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.88500000000000001,"5min":1.016,"15min":1.016},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.63,"5min":0.68899999999999995,"15min":0.68100000000000005},"min":{"1min":0.48799999999999999,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.80500000000000005,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.65900000000000003}]},{"osd":3,"last update":"Fri Oct 6 02:30:04 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.752,"15min":0.73199999999999998},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.48799999999999999},"max":{"1min":0.98499999999999999,"5min":1.159,"15min":1.159},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.753,"5min":0.79000000000000004,"15min":0.76900000000000002},"min":{"1min":0.56999999999999995,"5min":0.56200000000000006,"15min":0.54100000000000004},"max":{"1min":0.86799999999999999,"5min":1.194,"15min":1.194},"last":0.79600000000000004}]},{"osd":4,"last update":"Fri Oct 6 02:30:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.65200000000000002,"15min":0.67500000000000004},"min":{"1min":0.439,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.93000000000000005,"5min":1.012,"15min":1.1499999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.66600000000000004,"15min":0.68500000000000005},"min":{"1min":0.498,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.92700000000000005,"5min":1.25,"15min":1.25},"last":0.69299999999999995}]},{"osd":6,"last update":"Fri Oct 6 02:30:04 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.72799999999999998,"15min":0.74299999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.40300000000000002},"max":{"1min":0.95699999999999996,"5min":1.0549999999999999,"15min":1.1719999999999999},"last":0.75},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.80100000000000005,"15min":0.81899999999999995},"min":{"1min":0.502,"5min":0.373,"15min":0.373},"max":{"1min":0.98299999999999998,"5min":1.1200000000000001,"15min":1.2170000000000001},"last":0.85599999999999998}]},{"osd":7,"last update":"Fri Oct 6 02:30:04 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.747,"15min":0.76500000000000001},"min":{"1min":0.57199999999999995,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.0169999999999999,"5min":1.23,"15min":1.23},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79800000000000004,"15min":0.82899999999999996},"min":{"1min":0.58499999999999996,"5min":0.437,"15min":0.437},"max":{"1min":0.93300000000000005,"5min":1.1040000000000001,"15min":1.5640000000000001},"last":0.83399999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T02:30:16.994 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T02:30:16.994 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T02:30:17.097 INFO:tasks.ceph.mds.a.smithi031.stderr:2023-10-06T02:30:17.097+0000 7f70a29f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T02:30:17.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.221+0000 7fdc2679d700 1 Processor -- start 2023-10-06T02:30:17.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.221+0000 7fdc2679d700 1 -- start start 2023-10-06T02:30:17.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.223+0000 7fdc2679d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdc2013fb20 0x7fdc2015a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:17.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.223+0000 7fdc2679d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fdc2013fef0 0x7fdc200c7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:17.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.223+0000 7fdc2679d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fdc2006aaa0 0x7fdc200c7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:17.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.223+0000 7fdc2679d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc200c8410 con 0x7fdc2013fef0 2023-10-06T02:30:17.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.223+0000 7fdc2679d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc2015ae70 con 0x7fdc2006aaa0 2023-10-06T02:30:17.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.223+0000 7fdc2679d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2006af60 con 0x7fdc2013fb20 2023-10-06T02:30:17.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.223+0000 7fdc24f9a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fdc2013fef0 0x7fdc200c7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:17.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.223+0000 7fdc24f9a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fdc2013fef0 0x7fdc200c7990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39228/0 (socket says 172.21.15.31:39228) 2023-10-06T02:30:17.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.223+0000 7fdc24f9a700 1 -- 172.21.15.31:0/2585580328 learned_addr learned my addr 172.21.15.31:0/2585580328 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:17.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.223+0000 7fdc2579b700 1 --2- 172.21.15.31:0/2585580328 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdc2013fb20 0x7fdc2015a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:17.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.223+0000 7fdc25f9c700 1 --2- 172.21.15.31:0/2585580328 >> v2:172.21.15.62:3300/0 conn(0x7fdc2006aaa0 0x7fdc200c7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:17.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.223+0000 7fdc24f9a700 1 -- 172.21.15.31:0/2585580328 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdc2013fb20 msgr2=0x7fdc2015a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:17.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.223+0000 7fdc24f9a700 1 --2- 172.21.15.31:0/2585580328 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdc2013fb20 0x7fdc2015a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:17.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.223+0000 7fdc24f9a700 1 -- 172.21.15.31:0/2585580328 >> v2:172.21.15.62:3300/0 conn(0x7fdc2006aaa0 msgr2=0x7fdc200c7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:17.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.224+0000 7fdc24f9a700 1 --2- 172.21.15.31:0/2585580328 >> v2:172.21.15.62:3300/0 conn(0x7fdc2006aaa0 0x7fdc200c7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:17.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.224+0000 7fdc24f9a700 1 -- 172.21.15.31:0/2585580328 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc2006b240 con 0x7fdc2013fef0 2023-10-06T02:30:17.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.224+0000 7fdc25f9c700 1 --2- 172.21.15.31:0/2585580328 >> v2:172.21.15.62:3300/0 conn(0x7fdc2006aaa0 0x7fdc200c7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:30:17.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.224+0000 7fdc24f9a700 1 --2- 172.21.15.31:0/2585580328 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc2013fef0 0x7fdc200c7990 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fdc20156920 tx=0x7fdc1000b4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c823c31e27321e74 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:17.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.224+0000 7fdc17fff700 1 -- 172.21.15.31:0/2585580328 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc10005c10 con 0x7fdc2013fef0 2023-10-06T02:30:17.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.224+0000 7fdc17fff700 1 -- 172.21.15.31:0/2585580328 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc10005d70 con 0x7fdc2013fef0 2023-10-06T02:30:17.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.224+0000 7fdc17fff700 1 -- 172.21.15.31:0/2585580328 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1000dc30 con 0x7fdc2013fef0 2023-10-06T02:30:17.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.225+0000 7fdc2679d700 1 -- 172.21.15.31:0/2585580328 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc2013fef0 msgr2=0x7fdc200c7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:17.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.225+0000 7fdc2679d700 1 --2- 172.21.15.31:0/2585580328 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc2013fef0 0x7fdc200c7990 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fdc20156920 tx=0x7fdc1000b4c0 comp rx=0 tx=0).stop 2023-10-06T02:30:17.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.225+0000 7fdc2679d700 1 -- 172.21.15.31:0/2585580328 shutdown_connections 2023-10-06T02:30:17.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.225+0000 7fdc2679d700 1 --2- 172.21.15.31:0/2585580328 >> v2:172.21.15.62:3300/0 conn(0x7fdc2006aaa0 0x7fdc200c7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:17.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.225+0000 7fdc2679d700 1 --2- 172.21.15.31:0/2585580328 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdc2013fb20 0x7fdc2015a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:17.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.225+0000 7fdc2679d700 1 --2- 172.21.15.31:0/2585580328 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc2013fef0 0x7fdc200c7990 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:17.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.225+0000 7fdc2679d700 1 -- 172.21.15.31:0/2585580328 >> 172.21.15.31:0/2585580328 conn(0x7fdc2006d170 msgr2=0x7fdc200cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:17.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.225+0000 7fdc2679d700 1 -- 172.21.15.31:0/2585580328 shutdown_connections 2023-10-06T02:30:17.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.225+0000 7fdc2679d700 1 -- 172.21.15.31:0/2585580328 wait complete. 2023-10-06T02:30:17.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.226+0000 7fdc2679d700 1 Processor -- start 2023-10-06T02:30:17.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.226+0000 7fdc2679d700 1 -- start start 2023-10-06T02:30:17.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.227+0000 7fdc2679d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdc2006aaa0 0x7fdc20157aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:17.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.227+0000 7fdc2679d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc2013fb20 0x7fdc200b17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:17.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.227+0000 7fdc2679d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc2013fef0 0x7fdc2013e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:17.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.227+0000 7fdc2679d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc20157fe0 con 0x7fdc2013fef0 2023-10-06T02:30:17.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.227+0000 7fdc2679d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc200b1d20 con 0x7fdc2013fb20 2023-10-06T02:30:17.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.227+0000 7fdc2679d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc20158de0 con 0x7fdc2006aaa0 2023-10-06T02:30:17.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.228+0000 7fdc25f9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc2013fef0 0x7fdc2013e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:17.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.228+0000 7fdc25f9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc2013fef0 0x7fdc2013e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39232/0 (socket says 172.21.15.31:39232) 2023-10-06T02:30:17.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.228+0000 7fdc25f9c700 1 -- 172.21.15.31:0/3147136996 learned_addr learned my addr 172.21.15.31:0/3147136996 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:17.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.228+0000 7fdc2579b700 1 --2- 172.21.15.31:0/3147136996 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdc2006aaa0 0x7fdc20157aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:17.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.228+0000 7fdc24f9a700 1 --2- 172.21.15.31:0/3147136996 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc2013fb20 0x7fdc200b17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:17.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.228+0000 7fdc25f9c700 1 -- 172.21.15.31:0/3147136996 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdc2006aaa0 msgr2=0x7fdc20157aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:17.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.228+0000 7fdc25f9c700 1 --2- 172.21.15.31:0/3147136996 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdc2006aaa0 0x7fdc20157aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:17.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.228+0000 7fdc25f9c700 1 -- 172.21.15.31:0/3147136996 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc2013fb20 msgr2=0x7fdc200b17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:17.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.228+0000 7fdc25f9c700 1 --2- 172.21.15.31:0/3147136996 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc2013fb20 0x7fdc200b17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:17.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.228+0000 7fdc25f9c700 1 -- 172.21.15.31:0/3147136996 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc1000b720 con 0x7fdc2013fef0 2023-10-06T02:30:17.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.228+0000 7fdc25f9c700 1 --2- 172.21.15.31:0/3147136996 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc2013fef0 0x7fdc2013e9d0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7fdc1800d150 tx=0x7fdc18009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:17.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.229+0000 7fdc157fa700 1 -- 172.21.15.31:0/3147136996 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1800dc90 con 0x7fdc2013fef0 2023-10-06T02:30:17.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.229+0000 7fdc157fa700 1 -- 172.21.15.31:0/3147136996 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc1800f040 con 0x7fdc2013fef0 2023-10-06T02:30:17.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.229+0000 7fdc2679d700 1 -- 172.21.15.31:0/3147136996 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc201581d0 con 0x7fdc2013fef0 2023-10-06T02:30:17.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.229+0000 7fdc157fa700 1 -- 172.21.15.31:0/3147136996 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc18015690 con 0x7fdc2013fef0 2023-10-06T02:30:17.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.229+0000 7fdc2679d700 1 -- 172.21.15.31:0/3147136996 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc20158a70 con 0x7fdc2013fef0 2023-10-06T02:30:17.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.230+0000 7fdc2679d700 1 -- 172.21.15.31:0/3147136996 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc201585d0 con 0x7fdc2013fef0 2023-10-06T02:30:17.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.231+0000 7fdc157fa700 1 -- 172.21.15.31:0/3147136996 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7fdc1801e3b0 con 0x7fdc2013fef0 2023-10-06T02:30:17.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.231+0000 7fdc157fa700 1 --2- 172.21.15.31:0/3147136996 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fdc040674d0 0x7fdc04069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:17.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.232+0000 7fdc157fa700 1 -- 172.21.15.31:0/3147136996 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fdc180856d0 con 0x7fdc2013fef0 2023-10-06T02:30:17.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.232+0000 7fdc2579b700 1 --2- 172.21.15.31:0/3147136996 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fdc040674d0 0x7fdc04069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:17.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.233+0000 7fdc2579b700 1 --2- 172.21.15.31:0/3147136996 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fdc040674d0 0x7fdc04069980 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c006fd0 tx=0x7fdc1c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:17.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.236+0000 7fdc157fa700 1 -- 172.21.15.31:0/3147136996 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdc18051e00 con 0x7fdc2013fef0 2023-10-06T02:30:17.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.518+0000 7fdc2679d700 1 -- 172.21.15.31:0/3147136996 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fdc200648e0 con 0x7fdc2013fef0 2023-10-06T02:30:17.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.524+0000 7fdc157fa700 1 -- 172.21.15.31:0/3147136996 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) v1 ==== 190+0+0 (secure 0 0 0) 0x7fdc200648e0 con 0x7fdc2013fef0 2023-10-06T02:30:17.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.526+0000 7fdc2679d700 1 -- 172.21.15.31:0/3147136996 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fdc040674d0 msgr2=0x7fdc04069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:17.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.526+0000 7fdc2679d700 1 --2- 172.21.15.31:0/3147136996 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fdc040674d0 0x7fdc04069980 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c006fd0 tx=0x7fdc1c008040 comp rx=0 tx=0).stop 2023-10-06T02:30:17.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.526+0000 7fdc2679d700 1 -- 172.21.15.31:0/3147136996 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc2013fef0 msgr2=0x7fdc2013e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:17.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.526+0000 7fdc2679d700 1 --2- 172.21.15.31:0/3147136996 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc2013fef0 0x7fdc2013e9d0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7fdc1800d150 tx=0x7fdc18009a50 comp rx=0 tx=0).stop 2023-10-06T02:30:17.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.526+0000 7fdc2679d700 1 -- 172.21.15.31:0/3147136996 shutdown_connections 2023-10-06T02:30:17.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.526+0000 7fdc2679d700 1 --2- 172.21.15.31:0/3147136996 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc2013fef0 0x7fdc2013e9d0 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:17.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.526+0000 7fdc2679d700 1 --2- 172.21.15.31:0/3147136996 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fdc040674d0 0x7fdc04069980 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:17.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.526+0000 7fdc2679d700 1 --2- 172.21.15.31:0/3147136996 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdc2006aaa0 0x7fdc20157aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:17.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.526+0000 7fdc2679d700 1 --2- 172.21.15.31:0/3147136996 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc2013fb20 0x7fdc200b17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:17.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.526+0000 7fdc2679d700 1 -- 172.21.15.31:0/3147136996 >> 172.21.15.31:0/3147136996 conn(0x7fdc2006d170 msgr2=0x7fdc200c1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:17.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.526+0000 7fdc2679d700 1 -- 172.21.15.31:0/3147136996 shutdown_connections 2023-10-06T02:30:17.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.527+0000 7fdc2679d700 1 -- 172.21.15.31:0/3147136996 wait complete. 2023-10-06T02:30:17.528 INFO:teuthology.orchestra.run.smithi031.stderr:updated caps for client.0 2023-10-06T02:30:17.544 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:30:17.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.730+0000 7f5a1239d700 1 Processor -- start 2023-10-06T02:30:17.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.730+0000 7f5a1239d700 1 -- start start 2023-10-06T02:30:17.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.732+0000 7f5a1239d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f5a0c13fb20 0x7f5a0c15a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:17.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.732+0000 7f5a1239d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5a0c06aaa0 0x7f5a0c0c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:17.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.732+0000 7f5a1239d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a0c15ae70 con 0x7f5a0c13fef0 2023-10-06T02:30:17.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.732+0000 7f5a1239d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a0c06af60 con 0x7f5a0c06aaa0 2023-10-06T02:30:17.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.732+0000 7f5a1239d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a0c06b240 con 0x7f5a0c13fb20 2023-10-06T02:30:17.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.732+0000 7f5a11b9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5a0c06aaa0 0x7f5a0c0c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:17.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.732+0000 7f5a11b9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5a0c06aaa0 0x7f5a0c0c9d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39242/0 (socket says 172.21.15.31:39242) 2023-10-06T02:30:17.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.732+0000 7f5a11b9c700 1 -- 172.21.15.31:0/765104590 learned_addr learned my addr 172.21.15.31:0/765104590 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:17.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.732+0000 7f5a1139b700 1 --2- 172.21.15.31:0/765104590 >> v2:172.21.15.62:3300/0 conn(0x7f5a0c13fb20 0x7f5a0c15a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:17.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.732+0000 7f5a11b9c700 1 -- 172.21.15.31:0/765104590 >> v2:172.21.15.62:3300/0 conn(0x7f5a0c13fb20 msgr2=0x7f5a0c15a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:17.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.732+0000 7f5a11b9c700 1 --2- 172.21.15.31:0/765104590 >> v2:172.21.15.62:3300/0 conn(0x7f5a0c13fb20 0x7f5a0c15a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:17.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.732+0000 7f5a11b9c700 1 -- 172.21.15.31:0/765104590 >> v1:172.21.15.62:6789/0 conn(0x7f5a0c13fef0 legacy=0x7f5a0c0c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:17.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.733+0000 7f5a11b9c700 1 -- 172.21.15.31:0/765104590 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a0c06b520 con 0x7f5a0c06aaa0 2023-10-06T02:30:17.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.733+0000 7f5a1139b700 1 --2- 172.21.15.31:0/765104590 >> v2:172.21.15.62:3300/0 conn(0x7f5a0c13fb20 0x7f5a0c15a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:30:17.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.733+0000 7f5a11b9c700 1 --2- 172.21.15.31:0/765104590 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a0c06aaa0 0x7f5a0c0c9d00 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c0695b0 tx=0x7f5a04009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1de2c305ad58d20b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:17.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.733+0000 7f5a03fff700 1 -- 172.21.15.31:0/765104590 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a0400e070 con 0x7f5a0c06aaa0 2023-10-06T02:30:17.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.733+0000 7f5a03fff700 1 -- 172.21.15.31:0/765104590 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a0400ba50 con 0x7f5a0c06aaa0 2023-10-06T02:30:17.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.733+0000 7f5a03fff700 1 -- 172.21.15.31:0/765104590 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a040092e0 con 0x7f5a0c06aaa0 2023-10-06T02:30:17.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.734+0000 7f5a1239d700 1 -- 172.21.15.31:0/765104590 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a0c06aaa0 msgr2=0x7f5a0c0c9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:17.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.734+0000 7f5a1239d700 1 --2- 172.21.15.31:0/765104590 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a0c06aaa0 0x7f5a0c0c9d00 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c0695b0 tx=0x7f5a04009840 comp rx=0 tx=0).stop 2023-10-06T02:30:17.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.734+0000 7f5a1239d700 1 -- 172.21.15.31:0/765104590 shutdown_connections 2023-10-06T02:30:17.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.734+0000 7f5a1239d700 1 --2- 172.21.15.31:0/765104590 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a0c06aaa0 0x7f5a0c0c9d00 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:17.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.734+0000 7f5a1239d700 1 --2- 172.21.15.31:0/765104590 >> v2:172.21.15.62:3300/0 conn(0x7f5a0c13fb20 0x7f5a0c15a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:17.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.734+0000 7f5a1239d700 1 -- 172.21.15.31:0/765104590 >> 172.21.15.31:0/765104590 conn(0x7f5a0c06d170 msgr2=0x7f5a0c0cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:17.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.734+0000 7f5a1239d700 1 -- 172.21.15.31:0/765104590 shutdown_connections 2023-10-06T02:30:17.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.734+0000 7f5a1239d700 1 -- 172.21.15.31:0/765104590 wait complete. 2023-10-06T02:30:17.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.735+0000 7f5a1239d700 1 Processor -- start 2023-10-06T02:30:17.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.735+0000 7f5a1239d700 1 -- start start 2023-10-06T02:30:17.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.736+0000 7f5a1239d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a0c06aaa0 0x7f5a0c155e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:17.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.736+0000 7f5a1239d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a0c13fb20 0x7f5a0c1387a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:17.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.736+0000 7f5a1239d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5a0c13fef0 0x7f5a0c138ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:17.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.736+0000 7f5a1239d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0c156350 con 0x7f5a0c13fb20 2023-10-06T02:30:17.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.736+0000 7f5a1239d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0c15ba50 con 0x7f5a0c06aaa0 2023-10-06T02:30:17.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.736+0000 7f5a1239d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0c139be0 con 0x7f5a0c13fef0 2023-10-06T02:30:17.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.737+0000 7f5a10b9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a0c13fb20 0x7f5a0c1387a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:17.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.737+0000 7f5a10b9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a0c13fb20 0x7f5a0c1387a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39256/0 (socket says 172.21.15.31:39256) 2023-10-06T02:30:17.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.737+0000 7f5a10b9a700 1 -- 172.21.15.31:0/3567774124 learned_addr learned my addr 172.21.15.31:0/3567774124 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:17.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.737+0000 7f5a11b9c700 1 --2- 172.21.15.31:0/3567774124 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5a0c13fef0 0x7f5a0c138ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:17.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.737+0000 7f5a1139b700 1 --2- 172.21.15.31:0/3567774124 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a0c06aaa0 0x7f5a0c155e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:17.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.737+0000 7f5a10b9a700 1 -- 172.21.15.31:0/3567774124 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5a0c13fef0 msgr2=0x7f5a0c138ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:17.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.737+0000 7f5a10b9a700 1 --2- 172.21.15.31:0/3567774124 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5a0c13fef0 0x7f5a0c138ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:17.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.737+0000 7f5a10b9a700 1 -- 172.21.15.31:0/3567774124 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a0c06aaa0 msgr2=0x7f5a0c155e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:17.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.737+0000 7f5a10b9a700 1 --2- 172.21.15.31:0/3567774124 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a0c06aaa0 0x7f5a0c155e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:17.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.737+0000 7f5a10b9a700 1 -- 172.21.15.31:0/3567774124 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a04009ae0 con 0x7f5a0c13fb20 2023-10-06T02:30:17.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.737+0000 7f5a10b9a700 1 --2- 172.21.15.31:0/3567774124 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a0c13fb20 0x7f5a0c1387a0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f59fc00c4e0 tx=0x7f59fc007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:17.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.738+0000 7f5a017fa700 1 -- 172.21.15.31:0/3567774124 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59fc011590 con 0x7f5a0c13fb20 2023-10-06T02:30:17.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.738+0000 7f5a017fa700 1 -- 172.21.15.31:0/3567774124 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59fc011710 con 0x7f5a0c13fb20 2023-10-06T02:30:17.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.738+0000 7f5a1239d700 1 -- 172.21.15.31:0/3567774124 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a0c139340 con 0x7f5a0c13fb20 2023-10-06T02:30:17.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.738+0000 7f5a017fa700 1 -- 172.21.15.31:0/3567774124 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59fc011ad0 con 0x7f5a0c13fb20 2023-10-06T02:30:17.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.738+0000 7f5a1239d700 1 -- 172.21.15.31:0/3567774124 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a0c139900 con 0x7f5a0c13fb20 2023-10-06T02:30:17.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.739+0000 7f5a1239d700 1 -- 172.21.15.31:0/3567774124 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a0c15bbe0 con 0x7f5a0c13fb20 2023-10-06T02:30:17.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.739+0000 7f5a017fa700 1 -- 172.21.15.31:0/3567774124 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f59fc01b3b0 con 0x7f5a0c13fb20 2023-10-06T02:30:17.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.740+0000 7f5a017fa700 1 --2- 172.21.15.31:0/3567774124 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f59f00674d0 0x7f59f0069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:17.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.740+0000 7f5a017fa700 1 -- 172.21.15.31:0/3567774124 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f59fc0844d0 con 0x7f5a0c13fb20 2023-10-06T02:30:17.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.741+0000 7f5a1139b700 1 --2- 172.21.15.31:0/3567774124 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f59f00674d0 0x7f59f0069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:17.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.741+0000 7f5a1139b700 1 --2- 172.21.15.31:0/3567774124 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f59f00674d0 0x7f59f0069980 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5a08006f00 tx=0x7f5a08008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:17.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.744+0000 7f5a017fa700 1 -- 172.21.15.31:0/3567774124 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f59fc050dc0 con 0x7f5a0c13fb20 2023-10-06T02:30:17.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.997+0000 7f5a1239d700 1 -- 172.21.15.31:0/3567774124 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5a0c15ae70 con 0x7f5a0c13fb20 2023-10-06T02:30:17.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.997+0000 7f5a017fa700 1 -- 172.21.15.31:0/3567774124 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4601 (secure 0 0 0) 0x7f5a0c15ae70 con 0x7f5a0c13fb20 2023-10-06T02:30:17.998 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:30:17.998 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6611,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/1574790962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1574790962},{"type":"v1","addr":"172.21.15.62:6834","nonce":1574790962}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":85},{"gid":7591,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/2840240410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":2840240410},{"type":"v1","addr":"172.21.15.31:6835","nonce":2840240410}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":85},{"gid":7597,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2500071914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2500071914},{"type":"v1","addr":"172.21.15.62:6835","nonce":2500071914}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:30:11.625173+0000","modified":"2023-10-06T02:30:15.149386+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7971},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7971":{"gid":7971,"name":"a","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6834/1197111339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1197111339},{"type":"v1","addr":"172.21.15.31:6834","nonce":1197111339}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-10-06T02:30:18.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.999+0000 7f5a1239d700 1 -- 172.21.15.31:0/3567774124 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f59f00674d0 msgr2=0x7f59f0069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:18.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.999+0000 7f5a1239d700 1 --2- 172.21.15.31:0/3567774124 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f59f00674d0 0x7f59f0069980 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5a08006f00 tx=0x7f5a08008040 comp rx=0 tx=0).stop 2023-10-06T02:30:18.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.999+0000 7f5a1239d700 1 -- 172.21.15.31:0/3567774124 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a0c13fb20 msgr2=0x7f5a0c1387a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:18.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.999+0000 7f5a1239d700 1 --2- 172.21.15.31:0/3567774124 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a0c13fb20 0x7f5a0c1387a0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f59fc00c4e0 tx=0x7f59fc007f70 comp rx=0 tx=0).stop 2023-10-06T02:30:18.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.999+0000 7f5a1239d700 1 -- 172.21.15.31:0/3567774124 shutdown_connections 2023-10-06T02:30:18.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.999+0000 7f5a1239d700 1 --2- 172.21.15.31:0/3567774124 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f59f00674d0 0x7f59f0069980 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:18.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.999+0000 7f5a1239d700 1 --2- 172.21.15.31:0/3567774124 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5a0c13fef0 0x7f5a0c138ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:18.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.999+0000 7f5a1239d700 1 --2- 172.21.15.31:0/3567774124 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a0c06aaa0 0x7f5a0c155e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:18.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.999+0000 7f5a1239d700 1 --2- 172.21.15.31:0/3567774124 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a0c13fb20 0x7f5a0c1387a0 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:18.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:17.999+0000 7f5a1239d700 1 -- 172.21.15.31:0/3567774124 >> 172.21.15.31:0/3567774124 conn(0x7f5a0c06d170 msgr2=0x7f5a0c0aeed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:18.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.000+0000 7f5a1239d700 1 -- 172.21.15.31:0/3567774124 shutdown_connections 2023-10-06T02:30:18.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.000+0000 7f5a1239d700 1 -- 172.21.15.31:0/3567774124 wait complete. 2023-10-06T02:30:18.001 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 90 2023-10-06T02:30:18.420 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T02:30:11.625173+0000', 'modified': '2023-10-06T02:30:15.149386+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7971}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7971': {'gid': 7971, 'name': 'a', 'rank': 0, 'incarnation': 86, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.31:6834/1197111339', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6832', 'nonce': 1197111339}, {'type': 'v1', 'addr': '172.21.15.31:6834', 'nonce': 1197111339}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T02:30:18.420 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T02:30:18.421 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-06T02:30:18.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.611+0000 7fe5cf19d700 1 Processor -- start 2023-10-06T02:30:18.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.611+0000 7fe5cf19d700 1 -- start start 2023-10-06T02:30:18.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.612+0000 7fe5cf19d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe5c8159a10 0x7fe5c80ca7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:18.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.612+0000 7fe5cf19d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe5c8070410 0x7fe5c80cacf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:18.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.612+0000 7fe5cf19d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5c80c7420 con 0x7fe5c8159640 2023-10-06T02:30:18.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.612+0000 7fe5cf19d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5c806d820 con 0x7fe5c8070410 2023-10-06T02:30:18.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.612+0000 7fe5cf19d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c806db00 con 0x7fe5c8159a10 2023-10-06T02:30:18.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.613+0000 7fe5cd99a700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe5c8159a10 0x7fe5c80ca7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:18.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.613+0000 7fe5ce99c700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe5c8070410 0x7fe5c80cacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:18.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.613+0000 7fe5cd99a700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe5c8159a10 0x7fe5c80ca7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:55212/0 (socket says 172.21.15.31:55212) 2023-10-06T02:30:18.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.613+0000 7fe5cd99a700 1 -- 172.21.15.31:0/1557118796 learned_addr learned my addr 172.21.15.31:0/1557118796 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:18.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.613+0000 7fe5cd99a700 1 -- 172.21.15.31:0/1557118796 >> v2:172.21.15.62:3300/0 conn(0x7fe5c8070410 msgr2=0x7fe5c80cacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:18.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.613+0000 7fe5cd99a700 1 --2- 172.21.15.31:0/1557118796 >> v2:172.21.15.62:3300/0 conn(0x7fe5c8070410 0x7fe5c80cacf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:18.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.613+0000 7fe5cd99a700 1 -- 172.21.15.31:0/1557118796 >> v1:172.21.15.62:6789/0 conn(0x7fe5c8159640 legacy=0x7fe5c806d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:18.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.613+0000 7fe5cd99a700 1 -- 172.21.15.31:0/1557118796 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5c806dde0 con 0x7fe5c8159a10 2023-10-06T02:30:18.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.613+0000 7fe5ce99c700 1 --2- 172.21.15.31:0/1557118796 >> v2:172.21.15.62:3300/0 conn(0x7fe5c8070410 0x7fe5c80cacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:30:18.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.614+0000 7fe5cd99a700 1 --2- 172.21.15.31:0/1557118796 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe5c8159a10 0x7fe5c80ca7b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe5b80099b0 tx=0x7fe5b8009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63527814694a9fe7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:18.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.614+0000 7fe5cd199700 1 -- 172.21.15.31:0/1557118796 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b8009ea0 con 0x7fe5c8159a10 2023-10-06T02:30:18.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.614+0000 7fe5cd199700 1 -- 172.21.15.31:0/1557118796 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5b800b860 con 0x7fe5c8159a10 2023-10-06T02:30:18.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.614+0000 7fe5cd199700 1 -- 172.21.15.31:0/1557118796 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b800bc00 con 0x7fe5c8159a10 2023-10-06T02:30:18.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.615+0000 7fe5cf19d700 1 -- 172.21.15.31:0/1557118796 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe5c8159a10 msgr2=0x7fe5c80ca7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:18.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.615+0000 7fe5cf19d700 1 --2- 172.21.15.31:0/1557118796 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe5c8159a10 0x7fe5c80ca7b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe5b80099b0 tx=0x7fe5b8009250 comp rx=0 tx=0).stop 2023-10-06T02:30:18.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.615+0000 7fe5cf19d700 1 -- 172.21.15.31:0/1557118796 shutdown_connections 2023-10-06T02:30:18.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.615+0000 7fe5cf19d700 1 --2- 172.21.15.31:0/1557118796 >> v2:172.21.15.62:3300/0 conn(0x7fe5c8070410 0x7fe5c80cacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:18.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.615+0000 7fe5cf19d700 1 --2- 172.21.15.31:0/1557118796 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe5c8159a10 0x7fe5c80ca7b0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:18.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.615+0000 7fe5cf19d700 1 -- 172.21.15.31:0/1557118796 >> 172.21.15.31:0/1557118796 conn(0x7fe5c806e720 msgr2=0x7fe5c806eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:18.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.615+0000 7fe5cf19d700 1 -- 172.21.15.31:0/1557118796 shutdown_connections 2023-10-06T02:30:18.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.615+0000 7fe5cf19d700 1 -- 172.21.15.31:0/1557118796 wait complete. 2023-10-06T02:30:18.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.616+0000 7fe5cf19d700 1 Processor -- start 2023-10-06T02:30:18.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.617+0000 7fe5cf19d700 1 -- start start 2023-10-06T02:30:18.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.617+0000 7fe5cf19d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5c8070410 0x7fe5c81569c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:18.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.617+0000 7fe5cf19d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe5c8159640 0x7fe5c80b0370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:18.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.617+0000 7fe5cf19d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5c8159a10 0x7fe5c816b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:18.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.617+0000 7fe5cf19d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c80afee0 con 0x7fe5c8070410 2023-10-06T02:30:18.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.617+0000 7fe5cf19d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c81545c0 con 0x7fe5c8159a10 2023-10-06T02:30:18.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.617+0000 7fe5cf19d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c816abb0 con 0x7fe5c8159640 2023-10-06T02:30:18.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.618+0000 7fe5ce19b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5c8070410 0x7fe5c81569c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:18.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.618+0000 7fe5ce19b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5c8070410 0x7fe5c81569c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35462/0 (socket says 172.21.15.31:35462) 2023-10-06T02:30:18.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.618+0000 7fe5ce19b700 1 -- 172.21.15.31:0/4228497754 learned_addr learned my addr 172.21.15.31:0/4228497754 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:18.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.618+0000 7fe5cd99a700 1 --2- 172.21.15.31:0/4228497754 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe5c8159640 0x7fe5c80b0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:18.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.618+0000 7fe5ce99c700 1 --2- 172.21.15.31:0/4228497754 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5c8159a10 0x7fe5c816b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:18.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.618+0000 7fe5ce19b700 1 -- 172.21.15.31:0/4228497754 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe5c8159640 msgr2=0x7fe5c80b0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:18.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.618+0000 7fe5ce19b700 1 --2- 172.21.15.31:0/4228497754 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe5c8159640 0x7fe5c80b0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:18.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.618+0000 7fe5ce19b700 1 -- 172.21.15.31:0/4228497754 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5c8159a10 msgr2=0x7fe5c816b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:18.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.618+0000 7fe5ce19b700 1 --2- 172.21.15.31:0/4228497754 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5c8159a10 0x7fe5c816b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:18.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.618+0000 7fe5ce19b700 1 -- 172.21.15.31:0/4228497754 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5c0009a20 con 0x7fe5c8070410 2023-10-06T02:30:18.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.618+0000 7fe5ce99c700 1 --2- 172.21.15.31:0/4228497754 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5c8159a10 0x7fe5c816b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:30:18.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.619+0000 7fe5ce19b700 1 --2- 172.21.15.31:0/4228497754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5c8070410 0x7fe5c81569c0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fe5c400c590 tx=0x7fe5c400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:18.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.619+0000 7fe5be7fc700 1 -- 172.21.15.31:0/4228497754 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c4011590 con 0x7fe5c8070410 2023-10-06T02:30:18.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.619+0000 7fe5be7fc700 1 -- 172.21.15.31:0/4228497754 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5c40116f0 con 0x7fe5c8070410 2023-10-06T02:30:18.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.619+0000 7fe5be7fc700 1 -- 172.21.15.31:0/4228497754 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c4011ab0 con 0x7fe5c8070410 2023-10-06T02:30:18.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.619+0000 7fe5cf19d700 1 -- 172.21.15.31:0/4228497754 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5b80094f0 con 0x7fe5c8070410 2023-10-06T02:30:18.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.619+0000 7fe5cf19d700 1 -- 172.21.15.31:0/4228497754 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5c80afb70 con 0x7fe5c8070410 2023-10-06T02:30:18.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.621+0000 7fe5be7fc700 1 -- 172.21.15.31:0/4228497754 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7fe5c401b3b0 con 0x7fe5c8070410 2023-10-06T02:30:18.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.622+0000 7fe5be7fc700 1 --2- 172.21.15.31:0/4228497754 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe5ac0674d0 0x7fe5ac069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:18.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.622+0000 7fe5be7fc700 1 -- 172.21.15.31:0/4228497754 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fe5c4084710 con 0x7fe5c8070410 2023-10-06T02:30:18.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.623+0000 7fe5cd99a700 1 --2- 172.21.15.31:0/4228497754 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe5ac0674d0 0x7fe5ac069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:18.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.623+0000 7fe5cd99a700 1 --2- 172.21.15.31:0/4228497754 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe5ac0674d0 0x7fe5ac069980 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe5b80099b0 tx=0x7fe5b8017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:18.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.624+0000 7fe5deb2db80 1 Processor -- start 2023-10-06T02:30:18.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.624+0000 7fe5deb2db80 1 -- start start 2023-10-06T02:30:18.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.625+0000 7fe5deb2db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55de4f9fbe50 0x55de4f9d6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:18.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.625+0000 7fe5deb2db80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55de4f9dc2c0 0x55de4f9d7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:18.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.625+0000 7fe5deb2db80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55de4f9dda10 0x55de4f9d7740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:18.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.625+0000 7fe5deb2db80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de4f931a70 con 0x55de4f9dda10 2023-10-06T02:30:18.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.625+0000 7fe5deb2db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de4f92f270 con 0x55de4f9fbe50 2023-10-06T02:30:18.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.625+0000 7fe5deb2db80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x55de4f92b060 con 0x55de4f9dc2c0 2023-10-06T02:30:18.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.625+0000 7fe5ce99c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55de4f9dda10 0x55de4f9d7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:18.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.625+0000 7fe5ce99c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55de4f9dda10 0x55de4f9d7740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35464/0 (socket says 172.21.15.31:35464) 2023-10-06T02:30:18.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.625+0000 7fe5ce99c700 1 -- 172.21.15.31:0/1082999553 learned_addr learned my addr 172.21.15.31:0/1082999553 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:18.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.625+0000 7fe5cd99a700 1 --2- 172.21.15.31:0/1082999553 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55de4f9dc2c0 0x55de4f9d7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:18.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.625+0000 7fe5ce19b700 1 --2- 172.21.15.31:0/1082999553 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55de4f9fbe50 0x55de4f9d6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:18.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.626+0000 7fe5ce99c700 1 -- 172.21.15.31:0/1082999553 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55de4f9dc2c0 msgr2=0x55de4f9d7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:18.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.626+0000 7fe5ce99c700 1 --2- 172.21.15.31:0/1082999553 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55de4f9dc2c0 0x55de4f9d7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:18.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.626+0000 7fe5ce99c700 1 -- 172.21.15.31:0/1082999553 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55de4f9fbe50 msgr2=0x55de4f9d6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:18.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.626+0000 7fe5ce99c700 1 --2- 172.21.15.31:0/1082999553 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55de4f9fbe50 0x55de4f9d6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:18.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.626+0000 7fe5ce99c700 1 -- 172.21.15.31:0/1082999553 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de4f7c5270 con 0x55de4f9dda10 2023-10-06T02:30:18.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.626+0000 7fe5ce99c700 1 --2- 172.21.15.31:0/1082999553 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55de4f9dda10 0x55de4f9d7740 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fe5c0007730 tx=0x7fe5c000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e08e0c9999866cb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:18.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.626+0000 7fe5ab7fe700 1 -- 172.21.15.31:0/1082999553 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c000e070 con 0x55de4f9dda10 2023-10-06T02:30:18.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.626+0000 7fe5ab7fe700 1 -- 172.21.15.31:0/1082999553 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5c000f040 con 0x55de4f9dda10 2023-10-06T02:30:18.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.626+0000 7fe5ab7fe700 1 -- 172.21.15.31:0/1082999553 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c0015860 con 0x55de4f9dda10 2023-10-06T02:30:18.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.627+0000 7fe5deb2db80 1 -- 172.21.15.31:0/1082999553 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55de4f9dda10 msgr2=0x55de4f9d7740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:18.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.627+0000 7fe5deb2db80 1 --2- 172.21.15.31:0/1082999553 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55de4f9dda10 0x55de4f9d7740 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fe5c0007730 tx=0x7fe5c000b040 comp rx=0 tx=0).stop 2023-10-06T02:30:18.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.627+0000 7fe5deb2db80 1 -- 172.21.15.31:0/1082999553 shutdown_connections 2023-10-06T02:30:18.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.627+0000 7fe5deb2db80 1 --2- 172.21.15.31:0/1082999553 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55de4f9dda10 0x55de4f9d7740 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:18.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.627+0000 7fe5deb2db80 1 --2- 172.21.15.31:0/1082999553 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55de4f9fbe50 0x55de4f9d6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:18.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.627+0000 7fe5deb2db80 1 --2- 172.21.15.31:0/1082999553 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55de4f9dc2c0 0x55de4f9d7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:18.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.627+0000 7fe5deb2db80 1 -- 172.21.15.31:0/1082999553 >> 172.21.15.31:0/1082999553 conn(0x55de4f9dbe80 msgr2=0x55de4f9dc6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:18.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.627+0000 7fe5deb2db80 1 -- 172.21.15.31:0/1082999553 shutdown_connections 2023-10-06T02:30:18.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.627+0000 7fe5deb2db80 1 -- 172.21.15.31:0/1082999553 wait complete. 2023-10-06T02:30:18.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.628+0000 7fe5deb2db80 1 -- start start 2023-10-06T02:30:18.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.628+0000 7fe5deb2db80 1 Processor -- start 2023-10-06T02:30:18.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.629+0000 7fe5deb2db80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55de4f9dc2c0 0x55de4fa22f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:18.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.629+0000 7fe5deb2db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55de4f9dda10 0x55de4f9f9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:18.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.629+0000 7fe5deb2db80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55de4f9fbe50 0x55de4fa23fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:18.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.629+0000 7fe5deb2db80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de4f8d6890 con 0x55de4f9dc2c0 2023-10-06T02:30:18.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.629+0000 7fe5deb2db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de4f8f1ea0 con 0x55de4f9dda10 2023-10-06T02:30:18.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.629+0000 7fe5deb2db80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x55de4f7ade30 con 0x55de4f9fbe50 2023-10-06T02:30:18.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.629+0000 7fe5deb2db80 10 client.0 _ll_register_callbacks cb 0x55de4f99e020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T02:30:18.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.629+0000 7fe5ce19b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55de4f9dc2c0 0x55de4fa22f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:18.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.629+0000 7fe5ce19b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55de4f9dc2c0 0x55de4fa22f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35474/0 (socket says 172.21.15.31:35474) 2023-10-06T02:30:18.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.629+0000 7fe5ce19b700 1 -- 172.21.15.31:0/3490925246 learned_addr learned my addr 172.21.15.31:0/3490925246 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:18.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.629+0000 7fe5ce19b700 1 -- 172.21.15.31:0/3490925246 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55de4f9fbe50 msgr2=0x55de4fa23fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:18.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.629+0000 7fe5ce19b700 1 --2- 172.21.15.31:0/3490925246 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55de4f9fbe50 0x55de4fa23fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:18.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.629+0000 7fe5ce19b700 1 -- 172.21.15.31:0/3490925246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55de4f9dda10 msgr2=0x55de4f9f9ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:18.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.629+0000 7fe5ce19b700 1 --2- 172.21.15.31:0/3490925246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55de4f9dda10 0x55de4f9f9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:18.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.629+0000 7fe5ce19b700 1 -- 172.21.15.31:0/3490925246 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5c0007460 con 0x55de4f9dc2c0 2023-10-06T02:30:18.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.630+0000 7fe5ce19b700 1 --2- 172.21.15.31:0/3490925246 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55de4f9dc2c0 0x55de4fa22f80 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fe5c400cd50 tx=0x7fe5c405a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b79bdf4ad654b635 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:18.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.630+0000 7fe5a8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T02:30:18.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.630+0000 7fe5a8ff9700 1 -- 172.21.15.31:0/3490925246 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c4082160 con 0x55de4f9dc2c0 2023-10-06T02:30:18.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.630+0000 7fe5a8ff9700 1 -- 172.21.15.31:0/3490925246 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(132..132 src has 1..132) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fe5c405bbb0 con 0x55de4f9dc2c0 2023-10-06T02:30:18.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.630+0000 7fe5deb2db80 1 -- 172.21.15.31:0/3490925246 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55de4f9cf940 con 0x55de4f9dc2c0 2023-10-06T02:30:18.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.630+0000 7fe5a8ff9700 1 -- 172.21.15.31:0/3490925246 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe5c4054310 con 0x55de4f9dc2c0 2023-10-06T02:30:18.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.630+0000 7fe5deb2db80 10 client.8112 fetch_fsmap learned FSMap version 90 2023-10-06T02:30:18.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.630+0000 7fe5deb2db80 1 -- 172.21.15.31:0/3490925246 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) v3 -- 0x55de4f767fe0 con 0x55de4f9dc2c0 2023-10-06T02:30:18.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.630+0000 7fe5a8ff9700 1 -- 172.21.15.31:0/3490925246 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5c4080020 con 0x55de4f9dc2c0 2023-10-06T02:30:18.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.630+0000 7fe5a8ff9700 1 -- 172.21.15.31:0/3490925246 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 90) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fe5c405b720 con 0x55de4f9dc2c0 2023-10-06T02:30:18.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.630+0000 7fe5deb2db80 10 client.8112 fetch_fsmap finished waiting for FSMap version 90 2023-10-06T02:30:18.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.630+0000 7fe5deb2db80 10 client.8112 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T02:30:18.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.630+0000 7fe5deb2db80 20 client.8112 populate_metadata read hostname 'smithi031' 2023-10-06T02:30:18.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.630+0000 7fe5deb2db80 1 --2- 172.21.15.31:0/3490925246 >> [v2:172.21.15.31:6832/1197111339,v1:172.21.15.31:6834/1197111339] conn(0x55de4fa314b0 0x55de4fa31880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:18.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.630+0000 7fe5deb2db80 4 client.8112 mds_command: new command op to 7971 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T02:30:18.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.630+0000 7fe5deb2db80 1 -- 172.21.15.31:0/3490925246 --> [v2:172.21.15.31:6832/1197111339,v1:172.21.15.31:6834/1197111339] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55de4f9ac870 con 0x55de4fa314b0 2023-10-06T02:30:18.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.631+0000 7fe5cd99a700 1 --2- 172.21.15.31:0/3490925246 >> [v2:172.21.15.31:6832/1197111339,v1:172.21.15.31:6834/1197111339] conn(0x55de4fa314b0 0x55de4fa31880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:18.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.631+0000 7fe5cd99a700 1 --2- 172.21.15.31:0/3490925246 >> [v2:172.21.15.31:6832/1197111339,v1:172.21.15.31:6834/1197111339] conn(0x55de4fa314b0 0x55de4fa31880 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ef7567724bf9d77 server_cookie=7bc91c23b18b1aa in_seq=0 out_seq=0 2023-10-06T02:30:18.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.631+0000 7fe5a8ff9700 10 client.8112 ms_handle_connect on v2:172.21.15.31:6832/1197111339 2023-10-06T02:30:18.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.632+0000 7fe5a8ff9700 1 -- 172.21.15.31:0/3490925246 <== mds.0 v2:172.21.15.31:6832/1197111339 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55de4f9ac870 con 0x55de4fa314b0 2023-10-06T02:30:18.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.632+0000 7fe5a8ff9700 10 client.8112 handle_command_reply: tid=0 2023-10-06T02:30:18.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.632+0000 7fe5deb2db80 1 client.8112 shutdown 2023-10-06T02:30:18.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.632+0000 7fe5deb2db80 1 -- 172.21.15.31:0/3490925246 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55de4f9dc2c0 msgr2=0x55de4fa22f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:18.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.632+0000 7fe5deb2db80 1 --2- 172.21.15.31:0/3490925246 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55de4f9dc2c0 0x55de4fa22f80 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fe5c400cd50 tx=0x7fe5c405a040 comp rx=0 tx=0).stop 2023-10-06T02:30:18.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.632+0000 7fe5deb2db80 1 -- 172.21.15.31:0/3490925246 shutdown_connections 2023-10-06T02:30:18.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.632+0000 7fe5deb2db80 1 --2- 172.21.15.31:0/3490925246 >> [v2:172.21.15.31:6832/1197111339,v1:172.21.15.31:6834/1197111339] conn(0x55de4fa314b0 0x55de4fa31880 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:18.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.632+0000 7fe5deb2db80 1 --2- 172.21.15.31:0/3490925246 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55de4f9fbe50 0x55de4fa23fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:18.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.632+0000 7fe5deb2db80 1 --2- 172.21.15.31:0/3490925246 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55de4f9dc2c0 0x55de4fa22f80 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:18.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.632+0000 7fe5a8ff9700 0 client.8112 ms_handle_reset on v2:172.21.15.31:6832/1197111339 2023-10-06T02:30:18.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.632+0000 7fe5deb2db80 1 --2- 172.21.15.31:0/3490925246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55de4f9dda10 0x55de4f9f9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:18.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.632+0000 7fe5deb2db80 1 -- 172.21.15.31:0/3490925246 >> 172.21.15.31:0/3490925246 conn(0x55de4f9dbe80 msgr2=0x55de4fa06b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:18.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.632+0000 7fe5deb2db80 1 -- 172.21.15.31:0/3490925246 shutdown_connections 2023-10-06T02:30:18.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.632+0000 7fe5deb2db80 1 -- 172.21.15.31:0/3490925246 wait complete. 2023-10-06T02:30:18.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.633+0000 7fe5deb2db80 20 client.8112 trim_cache size 0 max 16384 2023-10-06T02:30:18.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.650+0000 7fe5deb2db80 1 Processor -- start 2023-10-06T02:30:18.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.650+0000 7fe5deb2db80 1 -- start start 2023-10-06T02:30:18.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.651+0000 7fe5deb2db80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55de4f9dc2c0 0x55de4f9d14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:18.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.651+0000 7fe5deb2db80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55de4f9dda10 0x55de4fa30180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:18.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.651+0000 7fe5deb2db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55de4f9fbe50 0x55de4fa306c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:18.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.651+0000 7fe5deb2db80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de4f9de190 con 0x55de4f9dda10 2023-10-06T02:30:18.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.651+0000 7fe5deb2db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de4fa21b10 con 0x55de4f9fbe50 2023-10-06T02:30:18.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.651+0000 7fe5deb2db80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x55de4f9ba2e0 con 0x55de4f9dc2c0 2023-10-06T02:30:18.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.651+0000 7fe5cd99a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55de4f9dda10 0x55de4fa30180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:18.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.651+0000 7fe5cd99a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55de4f9dda10 0x55de4fa30180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35484/0 (socket says 172.21.15.31:35484) 2023-10-06T02:30:18.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.651+0000 7fe5cd99a700 1 -- 172.21.15.31:0/1565459095 learned_addr learned my addr 172.21.15.31:0/1565459095 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:18.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.651+0000 7fe5ce19b700 1 --2- 172.21.15.31:0/1565459095 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55de4f9dc2c0 0x55de4f9d14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:18.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.651+0000 7fe5ce99c700 1 --2- 172.21.15.31:0/1565459095 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55de4f9fbe50 0x55de4fa306c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:18.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.651+0000 7fe5cd99a700 1 -- 172.21.15.31:0/1565459095 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55de4f9dc2c0 msgr2=0x55de4f9d14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:18.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.652+0000 7fe5cd99a700 1 --2- 172.21.15.31:0/1565459095 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55de4f9dc2c0 0x55de4f9d14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:18.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.652+0000 7fe5cd99a700 1 -- 172.21.15.31:0/1565459095 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55de4f9fbe50 msgr2=0x55de4fa306c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:18.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.652+0000 7fe5cd99a700 1 --2- 172.21.15.31:0/1565459095 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55de4f9fbe50 0x55de4fa306c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:18.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.652+0000 7fe5cd99a700 1 -- 172.21.15.31:0/1565459095 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de4f8f9780 con 0x55de4f9dda10 2023-10-06T02:30:18.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.652+0000 7fe5ce99c700 1 --2- 172.21.15.31:0/1565459095 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55de4f9fbe50 0x55de4fa306c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:30:18.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.652+0000 7fe5cd99a700 1 --2- 172.21.15.31:0/1565459095 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55de4f9dda10 0x55de4fa30180 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fe5b80090b0 tx=0x7fe5b802b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82764f4d40c9a49f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:18.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.652+0000 7fe5abfff700 1 -- 172.21.15.31:0/1565459095 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b802f070 con 0x55de4f9dda10 2023-10-06T02:30:18.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.652+0000 7fe5abfff700 1 -- 172.21.15.31:0/1565459095 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5b801ac10 con 0x55de4f9dda10 2023-10-06T02:30:18.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.652+0000 7fe5abfff700 1 -- 172.21.15.31:0/1565459095 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b8033250 con 0x55de4f9dda10 2023-10-06T02:30:18.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.653+0000 7fe5deb2db80 1 -- 172.21.15.31:0/1565459095 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55de4f9dda10 msgr2=0x55de4fa30180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:18.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.653+0000 7fe5deb2db80 1 --2- 172.21.15.31:0/1565459095 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55de4f9dda10 0x55de4fa30180 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fe5b80090b0 tx=0x7fe5b802b040 comp rx=0 tx=0).stop 2023-10-06T02:30:18.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.653+0000 7fe5deb2db80 1 -- 172.21.15.31:0/1565459095 shutdown_connections 2023-10-06T02:30:18.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.653+0000 7fe5deb2db80 1 --2- 172.21.15.31:0/1565459095 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55de4f9fbe50 0x55de4fa306c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:18.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.653+0000 7fe5deb2db80 1 --2- 172.21.15.31:0/1565459095 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55de4f9dc2c0 0x55de4f9d14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:18.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.653+0000 7fe5deb2db80 1 --2- 172.21.15.31:0/1565459095 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55de4f9dda10 0x55de4fa30180 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:18.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.653+0000 7fe5deb2db80 1 -- 172.21.15.31:0/1565459095 >> 172.21.15.31:0/1565459095 conn(0x55de4f9dbe80 msgr2=0x55de4f9fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:18.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.653+0000 7fe5deb2db80 1 -- 172.21.15.31:0/1565459095 shutdown_connections 2023-10-06T02:30:18.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.653+0000 7fe5deb2db80 1 -- 172.21.15.31:0/1565459095 wait complete. 2023-10-06T02:30:18.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.653+0000 7fe5deb2db80 1 -- start start 2023-10-06T02:30:18.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.654+0000 7fe5deb2db80 1 Processor -- start 2023-10-06T02:30:18.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.654+0000 7fe5deb2db80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55de4f9dc2c0 0x55de4fa87120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:18.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.654+0000 7fe5deb2db80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55de4f9dda10 0x55de4f9da740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:18.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.654+0000 7fe5deb2db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55de4f9fbe50 0x55de4f9dac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:18.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.654+0000 7fe5deb2db80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de4fa02100 con 0x55de4f9dda10 2023-10-06T02:30:18.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.654+0000 7fe5deb2db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de4fa022a0 con 0x55de4f9fbe50 2023-10-06T02:30:18.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.654+0000 7fe5deb2db80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x55de4f9fd180 con 0x55de4f9dc2c0 2023-10-06T02:30:18.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.655+0000 7fe5deb2db80 10 client.0 _ll_register_callbacks cb 0x55de4f99e020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T02:30:18.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.655+0000 7fe5cd99a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55de4f9dda10 0x55de4f9da740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:18.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.655+0000 7fe5cd99a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55de4f9dda10 0x55de4f9da740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35490/0 (socket says 172.21.15.31:35490) 2023-10-06T02:30:18.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.655+0000 7fe5cd99a700 1 -- 172.21.15.31:0/2324851892 learned_addr learned my addr 172.21.15.31:0/2324851892 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:18.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.655+0000 7fe5cd99a700 1 -- 172.21.15.31:0/2324851892 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55de4f9dc2c0 msgr2=0x55de4fa87120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:18.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.655+0000 7fe5cd99a700 1 --2- 172.21.15.31:0/2324851892 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55de4f9dc2c0 0x55de4fa87120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:18.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.655+0000 7fe5cd99a700 1 -- 172.21.15.31:0/2324851892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55de4f9fbe50 msgr2=0x55de4f9dac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:18.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.655+0000 7fe5cd99a700 1 --2- 172.21.15.31:0/2324851892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55de4f9fbe50 0x55de4f9dac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:18.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.655+0000 7fe5cd99a700 1 -- 172.21.15.31:0/2324851892 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5c806dde0 con 0x55de4f9dda10 2023-10-06T02:30:18.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.655+0000 7fe5cd99a700 1 --2- 172.21.15.31:0/2324851892 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55de4f9dda10 0x55de4f9da740 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fe5c806dc40 tx=0x7fe5b802f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f26c61bf895d0aa8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:18.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.655+0000 7fe5a8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T02:30:18.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.655+0000 7fe5a8ff9700 1 -- 172.21.15.31:0/2324851892 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b8020070 con 0x55de4f9dda10 2023-10-06T02:30:18.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.656+0000 7fe5deb2db80 1 -- 172.21.15.31:0/2324851892 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe5c000f040 con 0x55de4f9dda10 2023-10-06T02:30:18.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.656+0000 7fe5a8ff9700 1 -- 172.21.15.31:0/2324851892 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(132..132 src has 1..132) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fe5b8028aa0 con 0x55de4f9dda10 2023-10-06T02:30:18.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.656+0000 7fe5a8ff9700 1 -- 172.21.15.31:0/2324851892 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe5b8028e70 con 0x55de4f9dda10 2023-10-06T02:30:18.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.656+0000 7fe5deb2db80 10 client.8118 fetch_fsmap learned FSMap version 90 2023-10-06T02:30:18.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.656+0000 7fe5deb2db80 1 -- 172.21.15.31:0/2324851892 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) v3 -- 0x55de4f8e0470 con 0x55de4f9dda10 2023-10-06T02:30:18.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.656+0000 7fe5a8ff9700 1 -- 172.21.15.31:0/2324851892 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5c000f040 con 0x55de4f9dda10 2023-10-06T02:30:18.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.656+0000 7fe5a8ff9700 1 -- 172.21.15.31:0/2324851892 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 90) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fe5b8034420 con 0x55de4f9dda10 2023-10-06T02:30:18.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.656+0000 7fe5deb2db80 10 client.8118 fetch_fsmap finished waiting for FSMap version 90 2023-10-06T02:30:18.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.656+0000 7fe5deb2db80 10 client.8118 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T02:30:18.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.656+0000 7fe5deb2db80 20 client.8118 populate_metadata read hostname 'smithi031' 2023-10-06T02:30:18.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.656+0000 7fe5deb2db80 1 --2- 172.21.15.31:0/2324851892 >> [v2:172.21.15.31:6832/1197111339,v1:172.21.15.31:6834/1197111339] conn(0x55de4f9d6ea0 0x55de4f9d7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:18.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.656+0000 7fe5deb2db80 4 client.8118 mds_command: new command op to 7971 tid=0[{"prefix": "status"}] 2023-10-06T02:30:18.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.656+0000 7fe5deb2db80 1 -- 172.21.15.31:0/2324851892 --> [v2:172.21.15.31:6832/1197111339,v1:172.21.15.31:6834/1197111339] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55de4f8cde00 con 0x55de4f9d6ea0 2023-10-06T02:30:18.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.656+0000 7fe5ce19b700 1 --2- 172.21.15.31:0/2324851892 >> [v2:172.21.15.31:6832/1197111339,v1:172.21.15.31:6834/1197111339] conn(0x55de4f9d6ea0 0x55de4f9d7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:18.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.657+0000 7fe5ce19b700 1 --2- 172.21.15.31:0/2324851892 >> [v2:172.21.15.31:6832/1197111339,v1:172.21.15.31:6834/1197111339] conn(0x55de4f9d6ea0 0x55de4f9d7270 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a95ebe37f0bc5f1c server_cookie=28a8433a5b0ac9c4 in_seq=0 out_seq=0 2023-10-06T02:30:18.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.657+0000 7fe5a8ff9700 10 client.8118 ms_handle_connect on v2:172.21.15.31:6832/1197111339 2023-10-06T02:30:18.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.657+0000 7fe5a8ff9700 1 -- 172.21.15.31:0/2324851892 <== mds.0 v2:172.21.15.31:6832/1197111339 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x7fe5c4080150 con 0x55de4f9d6ea0 2023-10-06T02:30:18.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.657+0000 7fe5a8ff9700 10 client.8118 handle_command_reply: tid=0 2023-10-06T02:30:18.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.657+0000 7fe5deb2db80 1 client.8118 shutdown 2023-10-06T02:30:18.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.657+0000 7fe5deb2db80 1 -- 172.21.15.31:0/2324851892 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55de4f9dda10 msgr2=0x55de4f9da740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:18.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.657+0000 7fe5deb2db80 1 --2- 172.21.15.31:0/2324851892 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55de4f9dda10 0x55de4f9da740 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fe5c806dc40 tx=0x7fe5b802f000 comp rx=0 tx=0).stop 2023-10-06T02:30:18.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.657+0000 7fe5deb2db80 1 -- 172.21.15.31:0/2324851892 shutdown_connections 2023-10-06T02:30:18.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.657+0000 7fe5deb2db80 1 --2- 172.21.15.31:0/2324851892 >> [v2:172.21.15.31:6832/1197111339,v1:172.21.15.31:6834/1197111339] conn(0x55de4f9d6ea0 0x55de4f9d7270 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:18.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.657+0000 7fe5deb2db80 1 --2- 172.21.15.31:0/2324851892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55de4f9fbe50 0x55de4f9dac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:18.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.657+0000 7fe5deb2db80 1 --2- 172.21.15.31:0/2324851892 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55de4f9dc2c0 0x55de4fa87120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:18.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.657+0000 7fe5deb2db80 1 --2- 172.21.15.31:0/2324851892 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55de4f9dda10 0x55de4f9da740 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:18.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.657+0000 7fe5a8ff9700 0 client.8118 ms_handle_reset on v2:172.21.15.31:6832/1197111339 2023-10-06T02:30:18.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.657+0000 7fe5deb2db80 1 -- 172.21.15.31:0/2324851892 >> 172.21.15.31:0/2324851892 conn(0x55de4f9dbe80 msgr2=0x55de4fa30180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:18.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.657+0000 7fe5deb2db80 1 -- 172.21.15.31:0/2324851892 shutdown_connections 2023-10-06T02:30:18.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.658+0000 7fe5deb2db80 1 -- 172.21.15.31:0/2324851892 wait complete. 2023-10-06T02:30:18.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.658+0000 7fe5deb2db80 20 client.8118 trim_cache size 0 max 16384 2023-10-06T02:30:18.663 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T02:30:18.663 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "d6fcf512-874b-4259-b90a-30ae070f00a9", 2023-10-06T02:30:18.663 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 0, 2023-10-06T02:30:18.663 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 7971, 2023-10-06T02:30:18.663 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2023-10-06T02:30:18.663 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2023-10-06T02:30:18.663 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2023-10-06T02:30:18.664 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 6.5181877940000001, 2023-10-06T02:30:18.664 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 90, 2023-10-06T02:30:18.664 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 132, 2023-10-06T02:30:18.664 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 132, 2023-10-06T02:30:18.664 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 12.274584509 2023-10-06T02:30:18.664 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T02:30:18.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.658+0000 7fe5cf19d700 1 -- 172.21.15.31:0/4228497754 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe5ac0674d0 msgr2=0x7fe5ac069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:18.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.658+0000 7fe5cf19d700 1 --2- 172.21.15.31:0/4228497754 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe5ac0674d0 0x7fe5ac069980 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe5b80099b0 tx=0x7fe5b8017000 comp rx=0 tx=0).stop 2023-10-06T02:30:18.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.658+0000 7fe5cf19d700 1 -- 172.21.15.31:0/4228497754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5c8070410 msgr2=0x7fe5c81569c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:18.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.658+0000 7fe5cf19d700 1 --2- 172.21.15.31:0/4228497754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5c8070410 0x7fe5c81569c0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fe5c400c590 tx=0x7fe5c400c950 comp rx=0 tx=0).stop 2023-10-06T02:30:18.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.658+0000 7fe5cf19d700 1 -- 172.21.15.31:0/4228497754 shutdown_connections 2023-10-06T02:30:18.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.658+0000 7fe5cf19d700 1 --2- 172.21.15.31:0/4228497754 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5c8159a10 0x7fe5c816b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:18.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.658+0000 7fe5cf19d700 1 --2- 172.21.15.31:0/4228497754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5c8070410 0x7fe5c81569c0 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:18.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.658+0000 7fe5cf19d700 1 --2- 172.21.15.31:0/4228497754 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe5ac0674d0 0x7fe5ac069980 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:18.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.658+0000 7fe5cf19d700 1 --2- 172.21.15.31:0/4228497754 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe5c8159640 0x7fe5c80b0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:18.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.658+0000 7fe5cf19d700 1 -- 172.21.15.31:0/4228497754 >> 172.21.15.31:0/4228497754 conn(0x7fe5c806e720 msgr2=0x7fe5c815feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:18.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.658+0000 7fe5cf19d700 1 -- 172.21.15.31:0/4228497754 shutdown_connections 2023-10-06T02:30:18.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.658+0000 7fe5cf19d700 1 -- 172.21.15.31:0/4228497754 wait complete. 2023-10-06T02:30:18.676 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T02:30:18.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.862+0000 7f07e099d700 1 Processor -- start 2023-10-06T02:30:18.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.862+0000 7f07e099d700 1 -- start start 2023-10-06T02:30:18.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.863+0000 7f07e099d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f07dc140b70 0x7f07dc0c42f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:18.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.863+0000 7f07e099d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f07dc06d370 0x7f07dc0c7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:18.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.863+0000 7f07e099d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f07dc140f40 0x7f07dc0c7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:18.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.864+0000 7f07e099d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07dc0c8300 con 0x7f07dc140b70 2023-10-06T02:30:18.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.864+0000 7f07e099d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07dc0cd170 con 0x7f07dc140f40 2023-10-06T02:30:18.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.864+0000 7f07e099d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07dc0c9a20 con 0x7f07dc06d370 2023-10-06T02:30:18.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.864+0000 7f07db7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f07dc140b70 0x7f07dc0c42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:18.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.864+0000 7f07db7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f07dc140b70 0x7f07dc0c42f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35498/0 (socket says 172.21.15.31:35498) 2023-10-06T02:30:18.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.864+0000 7f07db7fe700 1 -- 172.21.15.31:0/1271588857 learned_addr learned my addr 172.21.15.31:0/1271588857 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:18.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.864+0000 7f07daffd700 1 --2- 172.21.15.31:0/1271588857 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f07dc06d370 0x7f07dc0c7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:18.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.864+0000 7f07dbfff700 1 --2- 172.21.15.31:0/1271588857 >> v2:172.21.15.62:3300/0 conn(0x7f07dc140f40 0x7f07dc0c7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:18.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.864+0000 7f07db7fe700 1 -- 172.21.15.31:0/1271588857 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f07dc06d370 msgr2=0x7f07dc0c7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:18.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.864+0000 7f07db7fe700 1 --2- 172.21.15.31:0/1271588857 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f07dc06d370 0x7f07dc0c7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:18.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.864+0000 7f07db7fe700 1 -- 172.21.15.31:0/1271588857 >> v2:172.21.15.62:3300/0 conn(0x7f07dc140f40 msgr2=0x7f07dc0c7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:18.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.864+0000 7f07db7fe700 1 --2- 172.21.15.31:0/1271588857 >> v2:172.21.15.62:3300/0 conn(0x7f07dc140f40 0x7f07dc0c7dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:18.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.864+0000 7f07db7fe700 1 -- 172.21.15.31:0/1271588857 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07dc0cd400 con 0x7f07dc140b70 2023-10-06T02:30:18.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.864+0000 7f07daffd700 1 --2- 172.21.15.31:0/1271588857 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f07dc06d370 0x7f07dc0c7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:30:18.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.864+0000 7f07dbfff700 1 --2- 172.21.15.31:0/1271588857 >> v2:172.21.15.62:3300/0 conn(0x7f07dc140f40 0x7f07dc0c7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:30:18.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.865+0000 7f07db7fe700 1 --2- 172.21.15.31:0/1271588857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07dc140b70 0x7f07dc0c42f0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f07dc069a30 tx=0x7f07cc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=840113cab572aafb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:18.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.865+0000 7f07da7fc700 1 -- 172.21.15.31:0/1271588857 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07cc00b860 con 0x7f07dc140b70 2023-10-06T02:30:18.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.865+0000 7f07da7fc700 1 -- 172.21.15.31:0/1271588857 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07cc00b9c0 con 0x7f07dc140b70 2023-10-06T02:30:18.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.865+0000 7f07da7fc700 1 -- 172.21.15.31:0/1271588857 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07cc00bd60 con 0x7f07dc140b70 2023-10-06T02:30:18.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.865+0000 7f07e099d700 1 -- 172.21.15.31:0/1271588857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07dc140b70 msgr2=0x7f07dc0c42f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:18.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.865+0000 7f07e099d700 1 --2- 172.21.15.31:0/1271588857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07dc140b70 0x7f07dc0c42f0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f07dc069a30 tx=0x7f07cc009250 comp rx=0 tx=0).stop 2023-10-06T02:30:18.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.866+0000 7f07e099d700 1 -- 172.21.15.31:0/1271588857 shutdown_connections 2023-10-06T02:30:18.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.866+0000 7f07e099d700 1 --2- 172.21.15.31:0/1271588857 >> v2:172.21.15.62:3300/0 conn(0x7f07dc140f40 0x7f07dc0c7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:18.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.866+0000 7f07e099d700 1 --2- 172.21.15.31:0/1271588857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07dc140b70 0x7f07dc0c42f0 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:18.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.866+0000 7f07e099d700 1 --2- 172.21.15.31:0/1271588857 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f07dc06d370 0x7f07dc0c7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:18.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.866+0000 7f07e099d700 1 -- 172.21.15.31:0/1271588857 >> 172.21.15.31:0/1271588857 conn(0x7f07dc0675a0 msgr2=0x7f07dc0679a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:18.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.866+0000 7f07e099d700 1 -- 172.21.15.31:0/1271588857 shutdown_connections 2023-10-06T02:30:18.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.866+0000 7f07e099d700 1 -- 172.21.15.31:0/1271588857 wait complete. 2023-10-06T02:30:18.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.867+0000 7f07e099d700 1 Processor -- start 2023-10-06T02:30:18.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.867+0000 7f07e099d700 1 -- start start 2023-10-06T02:30:18.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.868+0000 7f07e099d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07dc06d370 0x7f07dc0c5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:18.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.868+0000 7f07e099d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07dc140b70 0x7f07dc15a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:18.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.868+0000 7f07e099d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f07dc140f40 0x7f07dc1569e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:18.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.868+0000 7f07e099d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07dc16ad70 con 0x7f07dc140b70 2023-10-06T02:30:18.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.868+0000 7f07e099d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07dc16a160 con 0x7f07dc06d370 2023-10-06T02:30:18.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.868+0000 7f07e099d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07dc16a440 con 0x7f07dc140f40 2023-10-06T02:30:18.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.868+0000 7f07daffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07dc140b70 0x7f07dc15a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:18.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.869+0000 7f07daffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07dc140b70 0x7f07dc15a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35504/0 (socket says 172.21.15.31:35504) 2023-10-06T02:30:18.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.869+0000 7f07daffd700 1 -- 172.21.15.31:0/1026313327 learned_addr learned my addr 172.21.15.31:0/1026313327 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:18.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.869+0000 7f07db7fe700 1 --2- 172.21.15.31:0/1026313327 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07dc06d370 0x7f07dc0c5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:18.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.869+0000 7f07dbfff700 1 --2- 172.21.15.31:0/1026313327 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f07dc140f40 0x7f07dc1569e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:18.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.869+0000 7f07daffd700 1 -- 172.21.15.31:0/1026313327 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f07dc140f40 msgr2=0x7f07dc1569e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:18.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.869+0000 7f07daffd700 1 --2- 172.21.15.31:0/1026313327 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f07dc140f40 0x7f07dc1569e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:18.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.869+0000 7f07daffd700 1 -- 172.21.15.31:0/1026313327 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07dc06d370 msgr2=0x7f07dc0c5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:18.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.869+0000 7f07daffd700 1 --2- 172.21.15.31:0/1026313327 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07dc06d370 0x7f07dc0c5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:18.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.869+0000 7f07daffd700 1 -- 172.21.15.31:0/1026313327 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07cc0094f0 con 0x7f07dc140b70 2023-10-06T02:30:18.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.869+0000 7f07daffd700 1 --2- 172.21.15.31:0/1026313327 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07dc140b70 0x7f07dc15a3d0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f07d000d890 tx=0x7f07d0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:18.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.869+0000 7f07c37fe700 1 -- 172.21.15.31:0/1026313327 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d00125f0 con 0x7f07dc140b70 2023-10-06T02:30:18.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.869+0000 7f07c37fe700 1 -- 172.21.15.31:0/1026313327 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07d0012750 con 0x7f07dc140b70 2023-10-06T02:30:18.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.869+0000 7f07e099d700 1 -- 172.21.15.31:0/1026313327 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07dc16a720 con 0x7f07dc140b70 2023-10-06T02:30:18.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.869+0000 7f07c37fe700 1 -- 172.21.15.31:0/1026313327 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d0012af0 con 0x7f07dc140b70 2023-10-06T02:30:18.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.870+0000 7f07e099d700 1 -- 172.21.15.31:0/1026313327 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07dc0c61d0 con 0x7f07dc140b70 2023-10-06T02:30:18.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.870+0000 7f07e099d700 1 -- 172.21.15.31:0/1026313327 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07dc16af40 con 0x7f07dc140b70 2023-10-06T02:30:18.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.872+0000 7f07c37fe700 1 -- 172.21.15.31:0/1026313327 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f07d0014430 con 0x7f07dc140b70 2023-10-06T02:30:18.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.872+0000 7f07c37fe700 1 --2- 172.21.15.31:0/1026313327 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f07c80674d0 0x7f07c8069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:18.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.872+0000 7f07c37fe700 1 -- 172.21.15.31:0/1026313327 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f07d00854b0 con 0x7f07dc140b70 2023-10-06T02:30:18.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.873+0000 7f07db7fe700 1 --2- 172.21.15.31:0/1026313327 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f07c80674d0 0x7f07c8069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:18.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.874+0000 7f07db7fe700 1 --2- 172.21.15.31:0/1026313327 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f07c80674d0 0x7f07c8069980 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f07dc069a30 tx=0x7f07cc017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:18.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:18.877+0000 7f07c37fe700 1 -- 172.21.15.31:0/1026313327 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f07dc16af40 con 0x7f07dc140b70 2023-10-06T02:30:19.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:19.126+0000 7f07e099d700 1 -- 172.21.15.31:0/1026313327 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f07dc04c3c0 con 0x7f07dc140b70 2023-10-06T02:30:19.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:19.127+0000 7f07c37fe700 1 -- 172.21.15.31:0/1026313327 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v90) v1 ==== 53+0+73 (secure 0 0 0) 0x7f07d0055830 con 0x7f07dc140b70 2023-10-06T02:30:19.127 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T02:30:19.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:19.129+0000 7f07e099d700 1 -- 172.21.15.31:0/1026313327 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f07c80674d0 msgr2=0x7f07c8069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:19.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:19.129+0000 7f07e099d700 1 --2- 172.21.15.31:0/1026313327 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f07c80674d0 0x7f07c8069980 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f07dc069a30 tx=0x7f07cc017000 comp rx=0 tx=0).stop 2023-10-06T02:30:19.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:19.129+0000 7f07e099d700 1 -- 172.21.15.31:0/1026313327 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07dc140b70 msgr2=0x7f07dc15a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:19.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:19.129+0000 7f07e099d700 1 --2- 172.21.15.31:0/1026313327 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07dc140b70 0x7f07dc15a3d0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f07d000d890 tx=0x7f07d0008040 comp rx=0 tx=0).stop 2023-10-06T02:30:19.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:19.129+0000 7f07e099d700 1 -- 172.21.15.31:0/1026313327 shutdown_connections 2023-10-06T02:30:19.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:19.129+0000 7f07e099d700 1 --2- 172.21.15.31:0/1026313327 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f07c80674d0 0x7f07c8069980 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:19.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:19.129+0000 7f07e099d700 1 --2- 172.21.15.31:0/1026313327 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f07dc140f40 0x7f07dc1569e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:19.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:19.129+0000 7f07e099d700 1 --2- 172.21.15.31:0/1026313327 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07dc06d370 0x7f07dc0c5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:19.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:19.129+0000 7f07e099d700 1 --2- 172.21.15.31:0/1026313327 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07dc140b70 0x7f07dc15a3d0 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:19.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:19.129+0000 7f07e099d700 1 -- 172.21.15.31:0/1026313327 >> 172.21.15.31:0/1026313327 conn(0x7f07dc0675a0 msgr2=0x7f07dc13e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:19.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:19.129+0000 7f07e099d700 1 -- 172.21.15.31:0/1026313327 shutdown_connections 2023-10-06T02:30:19.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:19.129+0000 7f07e099d700 1 -- 172.21.15.31:0/1026313327 wait complete. 2023-10-06T02:30:19.147 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T02:30:19.147 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T02:30:19.147 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi031.front.sepia.ceph.com 2023-10-06T02:30:19.147 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T02:30:19.149 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T02:30:19.149 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T02:30:19.149 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T02:30:19.149 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T02:30:19.149 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T02:30:19.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:30:19.150 DEBUG:teuthology.orchestra.run.smithi031:> ip addr 2023-10-06T02:30:19.188 INFO:teuthology.orchestra.run.smithi031.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T02:30:19.188 INFO:teuthology.orchestra.run.smithi031.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T02:30:19.188 INFO:teuthology.orchestra.run.smithi031.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T02:30:19.188 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:30:19.188 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 ::1/128 scope host 2023-10-06T02:30:19.188 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:30:19.188 INFO:teuthology.orchestra.run.smithi031.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T02:30:19.188 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cc brd ff:ff:ff:ff:ff:ff 2023-10-06T02:30:19.188 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f0 2023-10-06T02:30:19.188 INFO:teuthology.orchestra.run.smithi031.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T02:30:19.189 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cd brd ff:ff:ff:ff:ff:ff 2023-10-06T02:30:19.189 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f1 2023-10-06T02:30:19.189 INFO:teuthology.orchestra.run.smithi031.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T02:30:19.189 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:ba brd ff:ff:ff:ff:ff:ff 2023-10-06T02:30:19.189 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f0 2023-10-06T02:30:19.189 INFO:teuthology.orchestra.run.smithi031.stdout: inet 172.21.15.31/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T02:30:19.189 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft 41837sec preferred_lft 41837sec 2023-10-06T02:30:19.189 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::ae1f:6bff:fea5:caba/64 scope link 2023-10-06T02:30:19.189 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:30:19.189 INFO:teuthology.orchestra.run.smithi031.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T02:30:19.189 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:bb brd ff:ff:ff:ff:ff:ff 2023-10-06T02:30:19.189 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f1 2023-10-06T02:30:19.190 INFO:teuthology.orchestra.run.smithi031.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T02:30:19.190 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 16:d7:88:2d:d0:c2 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:30:19.190 INFO:teuthology.orchestra.run.smithi031.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T02:30:19.190 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:30:19.190 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::408:52ff:fe3c:ba8f/64 scope link 2023-10-06T02:30:19.190 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:30:19.190 INFO:teuthology.orchestra.run.smithi031.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T02:30:19.190 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 16:d7:88:2d:d0:c2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T02:30:19.190 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::14d7:88ff:fe2d:d0c2/64 scope link 2023-10-06T02:30:19.190 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:30:19.190 INFO:teuthology.orchestra.run.smithi031.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T02:30:19.190 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 82:16:f1:00:cf:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T02:30:19.191 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::8016:f1ff:fe00:cf2f/64 scope link 2023-10-06T02:30:19.191 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:30:19.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:30:19.191 DEBUG:teuthology.orchestra.run.smithi031:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T02:30:19.288 INFO:teuthology.orchestra.run.smithi031.stdout:1 2023-10-06T02:30:19.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:30:19.292 DEBUG:teuthology.orchestra.run.smithi031:> route 2023-10-06T02:30:19.330 INFO:teuthology.orchestra.run.smithi031.stdout:Kernel IP routing table 2023-10-06T02:30:19.330 INFO:teuthology.orchestra.run.smithi031.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T02:30:19.330 INFO:teuthology.orchestra.run.smithi031.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T02:30:19.330 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T02:30:19.331 INFO:teuthology.orchestra.run.smithi031.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T02:30:19.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:30:19.332 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2023-10-06T02:30:19.332 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2023-10-06T02:30:19.332 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T02:30:19.332 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T02:30:19.332 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T02:30:19.332 DEBUG:teuthology.orchestra.run.smithi031:> ') 2023-10-06T02:30:19.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:30:19.532 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2023-10-06T02:30:19.569 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T02:30:19.569 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T02:30:19.569 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T02:30:19.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:30:19.569 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set brx.0 up 2023-10-06T02:30:19.672 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T02:30:19.672 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T02:30:19.708 INFO:teuthology.orchestra.run.smithi031.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T02:30:19.709 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T02:30:19.709 DEBUG:teuthology.orchestra.run.smithi031:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T02:30:19.785 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-06T02:30:19.785 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T02:30:19.785 DEBUG:teuthology.orchestra.run.smithi031:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T02:30:19.917 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T02:30:19.917 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T02:30:19.946 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T02:30:19.946 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T02:30:19.946 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.62:3300,172.21.15.62:3301" 2023-10-06T02:30:19.946 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: trying mount with new device syntax: 0@d6fcf512-874b-4259-b90a-30ae070f00a9.cephfs=/ 2023-10-06T02:30:19.946 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.31:3300/172.21.15.62:3300/172.21.15.62:3301" will pass to kernel 2023-10-06T02:30:19.950 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T02:30:19.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:30:19.950 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T02:30:20.013 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T02:30:20.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:20.203+0000 7f32ab19d700 1 Processor -- start 2023-10-06T02:30:20.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:20.203+0000 7f32ab19d700 1 -- start start 2023-10-06T02:30:20.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:20.204+0000 7f32ab19d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f32a4142c00 0x7f32a406b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:20.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:20.204+0000 7f32ab19d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f32a406bd20 0x7f32a40c7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:20.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:20.204+0000 7f32ab19d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f32a4142fd0 0x7f32a40c7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:20.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:20.204+0000 7f32ab19d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32a40ca480 con 0x7f32a4142c00 2023-10-06T02:30:20.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:20.204+0000 7f32ab19d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32a415e240 con 0x7f32a4142fd0 2023-10-06T02:30:20.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:20.204+0000 7f32ab19d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32a406b350 con 0x7f32a406bd20 2023-10-06T02:30:20.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:20.204+0000 7f32aa19b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f32a4142c00 0x7f32a406b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:20.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:20.204+0000 7f32aa19b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f32a4142c00 0x7f32a406b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35508/0 (socket says 172.21.15.31:35508) 2023-10-06T02:30:20.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:20.204+0000 7f32aa19b700 1 -- 172.21.15.31:0/1999196598 learned_addr learned my addr 172.21.15.31:0/1999196598 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:20.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:20.205+0000 7f32aa19b700 1 -- 172.21.15.31:0/1999196598 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f32a406bd20 msgr2=0x7f32a40c7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:20.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:20.205+0000 7f32aa99c700 1 --2- 172.21.15.31:0/1999196598 >> v2:172.21.15.62:3300/0 conn(0x7f32a4142fd0 0x7f32a40c7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:20.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:20.205+0000 7f32a999a700 1 --2- 172.21.15.31:0/1999196598 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f32a406bd20 0x7f32a40c7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:20.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:20.205+0000 7f32aa19b700 1 --2- 172.21.15.31:0/1999196598 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f32a406bd20 0x7f32a40c7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:20.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:20.205+0000 7f32aa19b700 1 -- 172.21.15.31:0/1999196598 >> v2:172.21.15.62:3300/0 conn(0x7f32a4142fd0 msgr2=0x7f32a40c7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:20.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:20.205+0000 7f32aa19b700 1 --2- 172.21.15.31:0/1999196598 >> v2:172.21.15.62:3300/0 conn(0x7f32a4142fd0 0x7f32a40c7dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:20.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:20.205+0000 7f32aa19b700 1 -- 172.21.15.31:0/1999196598 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32a415df60 con 0x7f32a4142c00 2023-10-06T02:30:20.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:20.205+0000 7f32aa99c700 1 --2- 172.21.15.31:0/1999196598 >> v2:172.21.15.62:3300/0 conn(0x7f32a4142fd0 0x7f32a40c7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:30:20.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:20.205+0000 7f32a999a700 1 --2- 172.21.15.31:0/1999196598 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f32a406bd20 0x7f32a40c7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:30:20.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:20.205+0000 7f32aa19b700 1 --2- 172.21.15.31:0/1999196598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32a4142c00 0x7f32a406b7e0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f32a40ca8c0 tx=0x7f32a0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fff291f59cd2ebc9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:20.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:20.205+0000 7f32a9199700 1 -- 172.21.15.31:0/1999196598 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a000b860 con 0x7f32a4142c00 2023-10-06T02:30:20.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:20.205+0000 7f32a9199700 1 -- 172.21.15.31:0/1999196598 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32a000b9c0 con 0x7f32a4142c00 2023-10-06T02:30:20.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:20.205+0000 7f32a9199700 1 -- 172.21.15.31:0/1999196598 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a000bd80 con 0x7f32a4142c00 2023-10-06T02:30:20.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:20.206+0000 7f32ab19d700 1 -- 172.21.15.31:0/1999196598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32a4142c00 msgr2=0x7f32a406b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:20.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:20.206+0000 7f32ab19d700 1 --2- 172.21.15.31:0/1999196598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32a4142c00 0x7f32a406b7e0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f32a40ca8c0 tx=0x7f32a0009250 comp rx=0 tx=0).stop 2023-10-06T02:30:20.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:20.206+0000 7f32ab19d700 1 -- 172.21.15.31:0/1999196598 shutdown_connections 2023-10-06T02:30:20.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:20.206+0000 7f32ab19d700 1 --2- 172.21.15.31:0/1999196598 >> v2:172.21.15.62:3300/0 conn(0x7f32a4142fd0 0x7f32a40c7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:20.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:20.206+0000 7f32ab19d700 1 --2- 172.21.15.31:0/1999196598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32a4142c00 0x7f32a406b7e0 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:20.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:20.206+0000 7f32ab19d700 1 --2- 172.21.15.31:0/1999196598 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f32a406bd20 0x7f32a40c7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:20.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:20.206+0000 7f32ab19d700 1 -- 172.21.15.31:0/1999196598 >> 172.21.15.31:0/1999196598 conn(0x7f32a40675a0 msgr2=0x7f32a40679a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:20.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:20.206+0000 7f32ab19d700 1 -- 172.21.15.31:0/1999196598 shutdown_connections 2023-10-06T02:30:20.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:20.206+0000 7f32ab19d700 1 -- 172.21.15.31:0/1999196598 wait complete. 2023-10-06T02:30:20.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:20.207+0000 7f32ab19d700 1 Processor -- start 2023-10-06T02:30:20.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:20.208+0000 7f32ab19d700 1 -- start start 2023-10-06T02:30:20.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:20.208+0000 7f32ab19d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f32a406bd20 0x7f32a4159e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:20.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:20.209+0000 7f32ab19d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32a4142c00 0x7f32a40c6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:20.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:20.209+0000 7f32ab19d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32a4142fd0 0x7f32a413f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:20.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:20.209+0000 7f32ab19d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32a40c5790 con 0x7f32a4142c00 2023-10-06T02:30:20.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:20.209+0000 7f32ab19d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32a40c5a30 con 0x7f32a4142fd0 2023-10-06T02:30:20.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:20.209+0000 7f32ab19d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32a40c5d10 con 0x7f32a406bd20 2023-10-06T02:30:20.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:20.209+0000 7f32a999a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32a4142c00 0x7f32a40c6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:20.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:20.209+0000 7f32a999a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32a4142c00 0x7f32a40c6ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35520/0 (socket says 172.21.15.31:35520) 2023-10-06T02:30:20.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:20.209+0000 7f32a999a700 1 -- 172.21.15.31:0/491611606 learned_addr learned my addr 172.21.15.31:0/491611606 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:20.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:20.209+0000 7f32aa19b700 1 --2- 172.21.15.31:0/491611606 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f32a406bd20 0x7f32a4159e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:20.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:20.209+0000 7f32aa99c700 1 --2- 172.21.15.31:0/491611606 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32a4142fd0 0x7f32a413f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:20.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:20.209+0000 7f32a999a700 1 -- 172.21.15.31:0/491611606 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f32a406bd20 msgr2=0x7f32a4159e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:20.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:20.209+0000 7f32a999a700 1 --2- 172.21.15.31:0/491611606 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f32a406bd20 0x7f32a4159e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:20.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:20.209+0000 7f32a999a700 1 -- 172.21.15.31:0/491611606 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32a4142fd0 msgr2=0x7f32a413f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:20.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:20.209+0000 7f32a999a700 1 --2- 172.21.15.31:0/491611606 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32a4142fd0 0x7f32a413f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:20.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:20.209+0000 7f32a999a700 1 -- 172.21.15.31:0/491611606 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32a00094f0 con 0x7f32a4142c00 2023-10-06T02:30:20.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:20.209+0000 7f32aa19b700 1 --2- 172.21.15.31:0/491611606 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f32a406bd20 0x7f32a4159e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:30:20.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:20.210+0000 7f32a999a700 1 --2- 172.21.15.31:0/491611606 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32a4142c00 0x7f32a40c6ee0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f329400c880 tx=0x7f3294008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:20.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:20.210+0000 7f329a7fc700 1 -- 172.21.15.31:0/491611606 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3294011660 con 0x7f32a4142c00 2023-10-06T02:30:20.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:20.210+0000 7f329a7fc700 1 -- 172.21.15.31:0/491611606 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32940117c0 con 0x7f32a4142c00 2023-10-06T02:30:20.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:20.210+0000 7f329a7fc700 1 -- 172.21.15.31:0/491611606 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3294011b60 con 0x7f32a4142c00 2023-10-06T02:30:20.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:20.210+0000 7f32ab19d700 1 -- 172.21.15.31:0/491611606 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32a40c5ff0 con 0x7f32a4142c00 2023-10-06T02:30:20.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:20.210+0000 7f32ab19d700 1 -- 172.21.15.31:0/491611606 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32a415a490 con 0x7f32a4142c00 2023-10-06T02:30:20.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:20.211+0000 7f32ab19d700 1 -- 172.21.15.31:0/491611606 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32a40c6230 con 0x7f32a4142c00 2023-10-06T02:30:20.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:20.212+0000 7f329a7fc700 1 -- 172.21.15.31:0/491611606 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f3294021350 con 0x7f32a4142c00 2023-10-06T02:30:20.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:20.213+0000 7f329a7fc700 1 --2- 172.21.15.31:0/491611606 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f32880674d0 0x7f3288069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:20.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:20.213+0000 7f329a7fc700 1 -- 172.21.15.31:0/491611606 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f3294085180 con 0x7f32a4142c00 2023-10-06T02:30:20.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:20.213+0000 7f32aa19b700 1 --2- 172.21.15.31:0/491611606 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f32880674d0 0x7f3288069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:20.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:20.214+0000 7f32aa19b700 1 --2- 172.21.15.31:0/491611606 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f32880674d0 0x7f3288069980 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f32a40ca8c0 tx=0x7f32a0017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:20.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:20.217+0000 7f329a7fc700 1 -- 172.21.15.31:0/491611606 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f32a40c6230 con 0x7f32a4142c00 2023-10-06T02:30:20.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:20.465+0000 7f32ab19d700 1 -- 172.21.15.31:0/491611606 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f32a40c6230 con 0x7f32a4142c00 2023-10-06T02:30:20.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:20.465+0000 7f329a7fc700 1 -- 172.21.15.31:0/491611606 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v90) v1 ==== 53+0+73 (secure 0 0 0) 0x7f32a40c6230 con 0x7f32a4142c00 2023-10-06T02:30:20.466 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T02:30:20.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:20.467+0000 7f32ab19d700 1 -- 172.21.15.31:0/491611606 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f32880674d0 msgr2=0x7f3288069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:20.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:20.467+0000 7f32ab19d700 1 --2- 172.21.15.31:0/491611606 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f32880674d0 0x7f3288069980 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f32a40ca8c0 tx=0x7f32a0017000 comp rx=0 tx=0).stop 2023-10-06T02:30:20.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:20.467+0000 7f32ab19d700 1 -- 172.21.15.31:0/491611606 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32a4142c00 msgr2=0x7f32a40c6ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:20.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:20.467+0000 7f32ab19d700 1 --2- 172.21.15.31:0/491611606 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32a4142c00 0x7f32a40c6ee0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f329400c880 tx=0x7f3294008040 comp rx=0 tx=0).stop 2023-10-06T02:30:20.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:20.468+0000 7f32ab19d700 1 -- 172.21.15.31:0/491611606 shutdown_connections 2023-10-06T02:30:20.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:20.468+0000 7f32ab19d700 1 --2- 172.21.15.31:0/491611606 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32a4142fd0 0x7f32a413f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:20.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:20.468+0000 7f32ab19d700 1 --2- 172.21.15.31:0/491611606 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f32880674d0 0x7f3288069980 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:20.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:20.468+0000 7f32ab19d700 1 --2- 172.21.15.31:0/491611606 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f32a406bd20 0x7f32a4159e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:20.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:20.468+0000 7f32ab19d700 1 --2- 172.21.15.31:0/491611606 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32a4142c00 0x7f32a40c6ee0 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:20.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:20.468+0000 7f32ab19d700 1 -- 172.21.15.31:0/491611606 >> 172.21.15.31:0/491611606 conn(0x7f32a40675a0 msgr2=0x7f32a40c2190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:20.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:20.468+0000 7f32ab19d700 1 -- 172.21.15.31:0/491611606 shutdown_connections 2023-10-06T02:30:20.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:20.468+0000 7f32ab19d700 1 -- 172.21.15.31:0/491611606 wait complete. 2023-10-06T02:30:20.486 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T02:30:20.486 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T02:30:20.486 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi062.front.sepia.ceph.com 2023-10-06T02:30:20.486 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-06T02:30:20.486 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-06T02:30:20.486 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T02:30:20.486 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T02:30:20.486 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T02:30:20.486 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T02:30:20.486 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:30:20.487 DEBUG:teuthology.orchestra.run.smithi062:> ip addr 2023-10-06T02:30:20.529 INFO:teuthology.orchestra.run.smithi062.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T02:30:20.529 INFO:teuthology.orchestra.run.smithi062.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T02:30:20.529 INFO:teuthology.orchestra.run.smithi062.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T02:30:20.529 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:30:20.529 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 ::1/128 scope host 2023-10-06T02:30:20.529 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:30:20.529 INFO:teuthology.orchestra.run.smithi062.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T02:30:20.530 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:14 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:30:20.530 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f0 2023-10-06T02:30:20.530 INFO:teuthology.orchestra.run.smithi062.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T02:30:20.530 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:15 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:30:20.530 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f1 2023-10-06T02:30:20.530 INFO:teuthology.orchestra.run.smithi062.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T02:30:20.530 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c2 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:30:20.530 INFO:teuthology.orchestra.run.smithi062.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T02:30:20.530 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c3 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:30:20.530 INFO:teuthology.orchestra.run.smithi062.stdout: inet 172.21.15.62/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T02:30:20.530 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft 41828sec preferred_lft 41828sec 2023-10-06T02:30:20.531 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::ec4:7aff:febb:d1c3/64 scope link 2023-10-06T02:30:20.531 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:30:20.531 INFO:teuthology.orchestra.run.smithi062.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T02:30:20.531 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether ae:42:a8:dd:f7:9b brd ff:ff:ff:ff:ff:ff 2023-10-06T02:30:20.531 INFO:teuthology.orchestra.run.smithi062.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T02:30:20.531 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:30:20.531 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::1098:a4ff:fe0d:389/64 scope link 2023-10-06T02:30:20.531 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:30:20.531 INFO:teuthology.orchestra.run.smithi062.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T02:30:20.531 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether fe:60:2b:8e:af:28 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T02:30:20.531 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::fc60:2bff:fe8e:af28/64 scope link 2023-10-06T02:30:20.531 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:30:20.531 INFO:teuthology.orchestra.run.smithi062.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T02:30:20.532 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether ae:42:a8:dd:f7:9b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T02:30:20.532 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::ac42:a8ff:fedd:f79b/64 scope link 2023-10-06T02:30:20.532 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:30:20.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:30:20.532 DEBUG:teuthology.orchestra.run.smithi062:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T02:30:20.633 INFO:teuthology.orchestra.run.smithi062.stdout:1 2023-10-06T02:30:20.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:30:20.636 DEBUG:teuthology.orchestra.run.smithi062:> route 2023-10-06T02:30:20.675 INFO:teuthology.orchestra.run.smithi062.stdout:Kernel IP routing table 2023-10-06T02:30:20.675 INFO:teuthology.orchestra.run.smithi062.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T02:30:20.675 INFO:teuthology.orchestra.run.smithi062.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T02:30:20.675 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T02:30:20.676 INFO:teuthology.orchestra.run.smithi062.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T02:30:20.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:30:20.677 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2023-10-06T02:30:20.677 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2023-10-06T02:30:20.677 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T02:30:20.677 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T02:30:20.677 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T02:30:20.677 DEBUG:teuthology.orchestra.run.smithi062:> ') 2023-10-06T02:30:20.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:30:20.876 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2023-10-06T02:30:20.912 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-06T02:30:20.912 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T02:30:20.913 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T02:30:20.913 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:30:20.913 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set brx.0 up 2023-10-06T02:30:21.015 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T02:30:21.015 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-06T02:30:21.052 INFO:teuthology.orchestra.run.smithi062.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-06T02:30:21.053 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T02:30:21.053 DEBUG:teuthology.orchestra.run.smithi062:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-06T02:30:21.129 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-06T02:30:21.130 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T02:30:21.130 DEBUG:teuthology.orchestra.run.smithi062:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T02:30:21.262 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T02:30:21.262 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T02:30:21.291 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T02:30:21.291 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T02:30:21.291 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.62:3300,172.21.15.62:3301" 2023-10-06T02:30:21.291 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: trying mount with new device syntax: 1@d6fcf512-874b-4259-b90a-30ae070f00a9.cephfs=/ 2023-10-06T02:30:21.291 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.31:3300/172.21.15.62:3300/172.21.15.62:3301" will pass to kernel 2023-10-06T02:30:21.295 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T02:30:21.296 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:30:21.296 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-06T02:30:21.357 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-10-06T02:30:21.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:21.558+0000 7f95f5d9d700 1 Processor -- start 2023-10-06T02:30:21.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:21.558+0000 7f95f5d9d700 1 -- start start 2023-10-06T02:30:21.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:21.559+0000 7f95f5d9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f95f00b24f0 0x7f95f00b0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:21.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:21.559+0000 7f95f5d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f95f00b1d40 0x7f95f01590f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:21.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:21.559+0000 7f95f5d9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95f00b0c80 con 0x7f95f00b28c0 2023-10-06T02:30:21.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:21.559+0000 7f95f5d9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95f00d10d0 con 0x7f95f00b24f0 2023-10-06T02:30:21.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:21.559+0000 7f95f5d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95f00b0f60 con 0x7f95f00b1d40 2023-10-06T02:30:21.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:21.559+0000 7f95f559c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f95f00b28c0 0x7f95f015d260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:48454/0 (socket says 172.21.15.31:48454) 2023-10-06T02:30:21.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:21.560+0000 7f95f559c700 1 -- 172.21.15.31:0/1558294111 learned_addr learned my addr 172.21.15.31:0/1558294111 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:30:21.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:21.560+0000 7f95effff700 1 --2- 172.21.15.31:0/1558294111 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f95f00b1d40 0x7f95f01590f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:21.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:21.560+0000 7f95f4d9b700 1 --2- 172.21.15.31:0/1558294111 >> v2:172.21.15.62:3300/0 conn(0x7f95f00b24f0 0x7f95f00b0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:21.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:21.560+0000 7f95ef7fe700 1 -- 172.21.15.31:0/1558294111 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 918328960 0 0) 0x7f95f00b0c80 con 0x7f95f00b28c0 2023-10-06T02:30:21.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:21.560+0000 7f95ef7fe700 1 -- 172.21.15.31:0/1558294111 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95e0003570 con 0x7f95f00b28c0 2023-10-06T02:30:21.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:21.560+0000 7f95effff700 1 -- 172.21.15.31:0/1558294111 >> v2:172.21.15.62:3300/0 conn(0x7f95f00b24f0 msgr2=0x7f95f00b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:21.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:21.560+0000 7f95effff700 1 --2- 172.21.15.31:0/1558294111 >> v2:172.21.15.62:3300/0 conn(0x7f95f00b24f0 0x7f95f00b0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:21.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:21.560+0000 7f95effff700 1 -- 172.21.15.31:0/1558294111 >> v1:172.21.15.31:6789/0 conn(0x7f95f00b28c0 legacy=0x7f95f015d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:21.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:21.560+0000 7f95effff700 1 -- 172.21.15.31:0/1558294111 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95f00b1240 con 0x7f95f00b1d40 2023-10-06T02:30:21.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:21.560+0000 7f95f4d9b700 1 --2- 172.21.15.31:0/1558294111 >> v2:172.21.15.62:3300/0 conn(0x7f95f00b24f0 0x7f95f00b0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:30:21.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:21.560+0000 7f95effff700 1 --2- 172.21.15.31:0/1558294111 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f95f00b1d40 0x7f95f01590f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f95dc0098e0 tx=0x7f95dc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a993e17fb126287d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:21.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:21.561+0000 7f95ef7fe700 1 -- 172.21.15.31:0/1558294111 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95dc011630 con 0x7f95f00b1d40 2023-10-06T02:30:21.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:21.561+0000 7f95ef7fe700 1 -- 172.21.15.31:0/1558294111 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95dc011790 con 0x7f95f00b1d40 2023-10-06T02:30:21.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:21.561+0000 7f95ef7fe700 1 -- 172.21.15.31:0/1558294111 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95dc011b30 con 0x7f95f00b1d40 2023-10-06T02:30:21.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:21.561+0000 7f95f5d9d700 1 -- 172.21.15.31:0/1558294111 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f95f00b1d40 msgr2=0x7f95f01590f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:21.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:21.561+0000 7f95f5d9d700 1 --2- 172.21.15.31:0/1558294111 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f95f00b1d40 0x7f95f01590f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f95dc0098e0 tx=0x7f95dc0091a0 comp rx=0 tx=0).stop 2023-10-06T02:30:21.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:21.561+0000 7f95f5d9d700 1 -- 172.21.15.31:0/1558294111 shutdown_connections 2023-10-06T02:30:21.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:21.561+0000 7f95f5d9d700 1 --2- 172.21.15.31:0/1558294111 >> v2:172.21.15.62:3300/0 conn(0x7f95f00b24f0 0x7f95f00b0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:21.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:21.561+0000 7f95f5d9d700 1 --2- 172.21.15.31:0/1558294111 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f95f00b1d40 0x7f95f01590f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:21.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:21.561+0000 7f95f5d9d700 1 -- 172.21.15.31:0/1558294111 >> 172.21.15.31:0/1558294111 conn(0x7f95f00703a0 msgr2=0x7f95f00707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:21.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:21.561+0000 7f95f5d9d700 1 -- 172.21.15.31:0/1558294111 shutdown_connections 2023-10-06T02:30:21.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:21.562+0000 7f95f5d9d700 1 -- 172.21.15.31:0/1558294111 wait complete. 2023-10-06T02:30:21.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:21.563+0000 7f95f5d9d700 1 Processor -- start 2023-10-06T02:30:21.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:21.563+0000 7f95f5d9d700 1 -- start start 2023-10-06T02:30:21.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:21.564+0000 7f95f5d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95f00b1d40 0x7f95f0154960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:21.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:21.564+0000 7f95f5d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95f00b24f0 0x7f95f016d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:21.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:21.564+0000 7f95f5d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f95f00b28c0 0x7f95f014bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:21.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:21.564+0000 7f95f5d9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95f0150250 con 0x7f95f00b1d40 2023-10-06T02:30:21.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:21.564+0000 7f95f5d9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95f016e300 con 0x7f95f00b24f0 2023-10-06T02:30:21.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:21.564+0000 7f95f5d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95f016dd40 con 0x7f95f00b28c0 2023-10-06T02:30:21.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:21.564+0000 7f95f4d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95f00b1d40 0x7f95f0154960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:21.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:21.564+0000 7f95f4d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95f00b1d40 0x7f95f0154960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35534/0 (socket says 172.21.15.31:35534) 2023-10-06T02:30:21.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:21.564+0000 7f95f4d9b700 1 -- 172.21.15.31:0/1854892257 learned_addr learned my addr 172.21.15.31:0/1854892257 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:21.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:21.564+0000 7f95effff700 1 --2- 172.21.15.31:0/1854892257 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95f00b24f0 0x7f95f016d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:21.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:21.564+0000 7f95f559c700 1 --2- 172.21.15.31:0/1854892257 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f95f00b28c0 0x7f95f014bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:21.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:21.565+0000 7f95f4d9b700 1 -- 172.21.15.31:0/1854892257 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f95f00b28c0 msgr2=0x7f95f014bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:21.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:21.565+0000 7f95f4d9b700 1 --2- 172.21.15.31:0/1854892257 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f95f00b28c0 0x7f95f014bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:21.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:21.565+0000 7f95f4d9b700 1 -- 172.21.15.31:0/1854892257 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95f00b24f0 msgr2=0x7f95f016d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:21.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:21.565+0000 7f95f4d9b700 1 --2- 172.21.15.31:0/1854892257 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95f00b24f0 0x7f95f016d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:21.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:21.565+0000 7f95f4d9b700 1 -- 172.21.15.31:0/1854892257 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95e4009420 con 0x7f95f00b1d40 2023-10-06T02:30:21.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:21.565+0000 7f95f4d9b700 1 --2- 172.21.15.31:0/1854892257 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95f00b1d40 0x7f95f0154960 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f95e400ec80 tx=0x7f95e40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:21.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:21.565+0000 7f95ecff9700 1 -- 172.21.15.31:0/1854892257 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e400c920 con 0x7f95f00b1d40 2023-10-06T02:30:21.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:21.565+0000 7f95ecff9700 1 -- 172.21.15.31:0/1854892257 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95e400ca80 con 0x7f95f00b1d40 2023-10-06T02:30:21.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:21.565+0000 7f95ecff9700 1 -- 172.21.15.31:0/1854892257 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e400ce00 con 0x7f95f00b1d40 2023-10-06T02:30:21.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:21.565+0000 7f95f5d9d700 1 -- 172.21.15.31:0/1854892257 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95dc009420 con 0x7f95f00b1d40 2023-10-06T02:30:21.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:21.565+0000 7f95f5d9d700 1 -- 172.21.15.31:0/1854892257 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95f014fee0 con 0x7f95f00b1d40 2023-10-06T02:30:21.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:21.566+0000 7f95f5d9d700 1 -- 172.21.15.31:0/1854892257 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95f0150530 con 0x7f95f00b1d40 2023-10-06T02:30:21.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:21.567+0000 7f95ecff9700 1 -- 172.21.15.31:0/1854892257 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f95e4013680 con 0x7f95f00b1d40 2023-10-06T02:30:21.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:21.568+0000 7f95ecff9700 1 --2- 172.21.15.31:0/1854892257 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f95d40674d0 0x7f95d4069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:21.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:21.568+0000 7f95ecff9700 1 -- 172.21.15.31:0/1854892257 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f95e4085810 con 0x7f95f00b1d40 2023-10-06T02:30:21.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:21.569+0000 7f95effff700 1 --2- 172.21.15.31:0/1854892257 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f95d40674d0 0x7f95d4069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:21.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:21.569+0000 7f95effff700 1 --2- 172.21.15.31:0/1854892257 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f95d40674d0 0x7f95d4069980 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f95dc0091e0 tx=0x7f95dc017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:21.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:21.572+0000 7f95ecff9700 1 -- 172.21.15.31:0/1854892257 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f95e4052090 con 0x7f95f00b1d40 2023-10-06T02:30:21.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:21.823+0000 7f95f5d9d700 1 -- 172.21.15.31:0/1854892257 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f95f0064f90 con 0x7f95f00b1d40 2023-10-06T02:30:22.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.084+0000 7f95ecff9700 1 -- 172.21.15.31:0/1854892257 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v91) v1 ==== 136+0+0 (secure 0 0 0) 0x7f95e40556b0 con 0x7f95f00b1d40 2023-10-06T02:30:22.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.087+0000 7f95f5d9d700 1 -- 172.21.15.31:0/1854892257 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f95d40674d0 msgr2=0x7f95d4069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:22.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.087+0000 7f95f5d9d700 1 --2- 172.21.15.31:0/1854892257 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f95d40674d0 0x7f95d4069980 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f95dc0091e0 tx=0x7f95dc017000 comp rx=0 tx=0).stop 2023-10-06T02:30:22.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.087+0000 7f95f5d9d700 1 -- 172.21.15.31:0/1854892257 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95f00b1d40 msgr2=0x7f95f0154960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:22.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.087+0000 7f95f5d9d700 1 --2- 172.21.15.31:0/1854892257 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95f00b1d40 0x7f95f0154960 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f95e400ec80 tx=0x7f95e40091a0 comp rx=0 tx=0).stop 2023-10-06T02:30:22.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.087+0000 7f95f5d9d700 1 -- 172.21.15.31:0/1854892257 shutdown_connections 2023-10-06T02:30:22.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.087+0000 7f95f5d9d700 1 --2- 172.21.15.31:0/1854892257 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f95d40674d0 0x7f95d4069980 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:22.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.087+0000 7f95f5d9d700 1 --2- 172.21.15.31:0/1854892257 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f95f00b28c0 0x7f95f014bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:22.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.087+0000 7f95f5d9d700 1 --2- 172.21.15.31:0/1854892257 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95f00b1d40 0x7f95f0154960 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:22.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.087+0000 7f95f5d9d700 1 --2- 172.21.15.31:0/1854892257 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95f00b24f0 0x7f95f016d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:22.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.087+0000 7f95f5d9d700 1 -- 172.21.15.31:0/1854892257 >> 172.21.15.31:0/1854892257 conn(0x7f95f00703a0 msgr2=0x7f95f00cf3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:22.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.087+0000 7f95f5d9d700 1 -- 172.21.15.31:0/1854892257 shutdown_connections 2023-10-06T02:30:22.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.087+0000 7f95f5d9d700 1 -- 172.21.15.31:0/1854892257 wait complete. 2023-10-06T02:30:22.106 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-06T02:30:22.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.292+0000 7f709759e700 1 Processor -- start 2023-10-06T02:30:22.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.293+0000 7f709759e700 1 -- start start 2023-10-06T02:30:22.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.294+0000 7f709759e700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7098140a70 0x7f70980c6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:22.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.294+0000 7f709759e700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70980c6d50 con 0x7f709806de50 2023-10-06T02:30:22.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.294+0000 7f709759e700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70980c7030 con 0x7f7098140e40 2023-10-06T02:30:22.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.294+0000 7f709759e700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70980c7310 con 0x7f7098140a70 2023-10-06T02:30:22.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.294+0000 7f7096d9d700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f709806de50 0x7f70980aeef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:48456/0 (socket says 172.21.15.31:48456) 2023-10-06T02:30:22.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.294+0000 7f7096d9d700 1 -- 172.21.15.31:0/2881552348 learned_addr learned my addr 172.21.15.31:0/2881552348 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:30:22.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.295+0000 7f709659c700 1 --2- 172.21.15.31:0/2881552348 >> v2:172.21.15.62:3300/0 conn(0x7f7098140a70 0x7f70980c6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:22.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.295+0000 7f709559a700 1 -- 172.21.15.31:0/2881552348 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3691258200 0 0) 0x7f70980c6d50 con 0x7f709806de50 2023-10-06T02:30:22.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.295+0000 7f709559a700 1 -- 172.21.15.31:0/2881552348 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7080003570 con 0x7f709806de50 2023-10-06T02:30:22.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.295+0000 7f709559a700 1 -- 172.21.15.31:0/2881552348 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3414534424 0 0) 0x7f7080003570 con 0x7f709806de50 2023-10-06T02:30:22.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.295+0000 7f709559a700 1 -- 172.21.15.31:0/2881552348 >> v2:172.21.15.62:3300/0 conn(0x7f7098140a70 msgr2=0x7f70980c6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:22.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.295+0000 7f709559a700 1 --2- 172.21.15.31:0/2881552348 >> v2:172.21.15.62:3300/0 conn(0x7f7098140a70 0x7f70980c6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:22.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.295+0000 7f709559a700 1 -- 172.21.15.31:0/2881552348 >> v1:172.21.15.62:6789/0 conn(0x7f7098140e40 legacy=0x7f70980c7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:22.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.295+0000 7f709559a700 1 -- 172.21.15.31:0/2881552348 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70980c75f0 con 0x7f709806de50 2023-10-06T02:30:22.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.295+0000 7f709559a700 1 -- 172.21.15.31:0/2881552348 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7f70900017f0 con 0x7f709806de50 2023-10-06T02:30:22.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.295+0000 7f709559a700 1 -- 172.21.15.31:0/2881552348 >> v1:172.21.15.31:6789/0 conn(0x7f709806de50 legacy=0x7f70980aeef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:22.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.295+0000 7f709559a700 1 --2- 172.21.15.31:0/2881552348 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7080003a60 0x7f7080003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:22.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.295+0000 7f709559a700 1 --2- 172.21.15.31:0/2881552348 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7080004810 0x7f7080004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:22.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.295+0000 7f709559a700 1 --2- 172.21.15.31:0/2881552348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70800051c0 0x7f708000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:22.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.295+0000 7f709559a700 1 -- 172.21.15.31:0/2881552348 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70980c7310 con 0x7f70800051c0 2023-10-06T02:30:22.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.295+0000 7f709559a700 1 -- 172.21.15.31:0/2881552348 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f708000c0b0 con 0x7f7080004810 2023-10-06T02:30:22.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.295+0000 7f709559a700 1 -- 172.21.15.31:0/2881552348 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f708000c270 con 0x7f7080003a60 2023-10-06T02:30:22.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.295+0000 7f7096d9d700 1 --2- 172.21.15.31:0/2881552348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70800051c0 0x7f708000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:22.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.296+0000 7f7095d9b700 1 --2- 172.21.15.31:0/2881552348 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7080004810 0x7f7080004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:22.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.296+0000 7f709659c700 1 --2- 172.21.15.31:0/2881552348 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7080003a60 0x7f7080003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:22.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.296+0000 7f7096d9d700 1 -- 172.21.15.31:0/2881552348 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7080003a60 msgr2=0x7f7080003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:22.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.296+0000 7f7096d9d700 1 --2- 172.21.15.31:0/2881552348 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7080003a60 0x7f7080003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:22.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.296+0000 7f7096d9d700 1 -- 172.21.15.31:0/2881552348 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7080004810 msgr2=0x7f7080004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:22.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.296+0000 7f7096d9d700 1 --2- 172.21.15.31:0/2881552348 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7080004810 0x7f7080004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:22.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.296+0000 7f7096d9d700 1 -- 172.21.15.31:0/2881552348 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7080003570 con 0x7f70800051c0 2023-10-06T02:30:22.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.296+0000 7f7096d9d700 1 -- 172.21.15.31:0/2881552348 reap_dead start 2023-10-06T02:30:22.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.296+0000 7f7096d9d700 1 --2- 172.21.15.31:0/2881552348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70800051c0 0x7f708000ba60 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f709000a880 tx=0x7f7090002120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a96d4d03e824dd3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:22.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.296+0000 7f709559a700 1 -- 172.21.15.31:0/2881552348 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70900070f0 con 0x7f70800051c0 2023-10-06T02:30:22.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.296+0000 7f709559a700 1 -- 172.21.15.31:0/2881552348 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7090007250 con 0x7f70800051c0 2023-10-06T02:30:22.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.296+0000 7f709559a700 1 -- 172.21.15.31:0/2881552348 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7090007610 con 0x7f70800051c0 2023-10-06T02:30:22.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.297+0000 7f709759e700 1 -- 172.21.15.31:0/2881552348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70800051c0 msgr2=0x7f708000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:22.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.297+0000 7f709759e700 1 --2- 172.21.15.31:0/2881552348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70800051c0 0x7f708000ba60 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f709000a880 tx=0x7f7090002120 comp rx=0 tx=0).stop 2023-10-06T02:30:22.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.297+0000 7f709759e700 1 -- 172.21.15.31:0/2881552348 shutdown_connections 2023-10-06T02:30:22.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.297+0000 7f709759e700 1 --2- 172.21.15.31:0/2881552348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70800051c0 0x7f708000ba60 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:22.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.297+0000 7f709759e700 1 -- 172.21.15.31:0/2881552348 >> 172.21.15.31:0/2881552348 conn(0x7f709806d170 msgr2=0x7f70980cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:22.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.297+0000 7f709759e700 1 -- 172.21.15.31:0/2881552348 shutdown_connections 2023-10-06T02:30:22.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.297+0000 7f709759e700 1 -- 172.21.15.31:0/2881552348 wait complete. 2023-10-06T02:30:22.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.299+0000 7f709759e700 1 Processor -- start 2023-10-06T02:30:22.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.299+0000 7f709759e700 1 -- start start 2023-10-06T02:30:22.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.300+0000 7f709759e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70800051c0 0x7f70980c3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:22.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.300+0000 7f709759e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70980c45e0 0x7f70980b0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:22.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.300+0000 7f709759e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7098156900 0x7f7098156d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:22.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.300+0000 7f709759e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70980c4e30 con 0x7f70980c45e0 2023-10-06T02:30:22.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.300+0000 7f709759e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70980c5110 con 0x7f70800051c0 2023-10-06T02:30:22.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.300+0000 7f709759e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70980c53f0 con 0x7f7098156900 2023-10-06T02:30:22.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.300+0000 7f7095d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70980c45e0 0x7f70980b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:22.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.300+0000 7f7095d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70980c45e0 0x7f70980b0950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35544/0 (socket says 172.21.15.31:35544) 2023-10-06T02:30:22.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.300+0000 7f7095d9b700 1 -- 172.21.15.31:0/1619030871 learned_addr learned my addr 172.21.15.31:0/1619030871 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:22.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.300+0000 7f709659c700 1 --2- 172.21.15.31:0/1619030871 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70800051c0 0x7f70980c3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:22.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.300+0000 7f7096d9d700 1 --2- 172.21.15.31:0/1619030871 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7098156900 0x7f7098156d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:22.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.300+0000 7f7095d9b700 1 -- 172.21.15.31:0/1619030871 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7098156900 msgr2=0x7f7098156d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:22.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.300+0000 7f7095d9b700 1 --2- 172.21.15.31:0/1619030871 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7098156900 0x7f7098156d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:22.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.300+0000 7f7095d9b700 1 -- 172.21.15.31:0/1619030871 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70800051c0 msgr2=0x7f70980c3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:22.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.300+0000 7f7095d9b700 1 --2- 172.21.15.31:0/1619030871 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70800051c0 0x7f70980c3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:22.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.300+0000 7f7095d9b700 1 -- 172.21.15.31:0/1619030871 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70980c56d0 con 0x7f70980c45e0 2023-10-06T02:30:22.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.300+0000 7f7096d9d700 1 --2- 172.21.15.31:0/1619030871 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7098156900 0x7f7098156d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:30:22.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.301+0000 7f7095d9b700 1 --2- 172.21.15.31:0/1619030871 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70980c45e0 0x7f70980b0950 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f708c002a60 tx=0x7f708c007d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:22.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.301+0000 7f70867fc700 1 -- 172.21.15.31:0/1619030871 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f708c00f8d0 con 0x7f70980c45e0 2023-10-06T02:30:22.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.301+0000 7f70867fc700 1 -- 172.21.15.31:0/1619030871 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f708c00fa50 con 0x7f70980c45e0 2023-10-06T02:30:22.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.301+0000 7f70867fc700 1 -- 172.21.15.31:0/1619030871 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f708c00fe10 con 0x7f70980c45e0 2023-10-06T02:30:22.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.301+0000 7f709759e700 1 -- 172.21.15.31:0/1619030871 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70980c59b0 con 0x7f70980c45e0 2023-10-06T02:30:22.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.301+0000 7f709759e700 1 -- 172.21.15.31:0/1619030871 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70980c4a50 con 0x7f70980c45e0 2023-10-06T02:30:22.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.302+0000 7f709759e700 1 -- 172.21.15.31:0/1619030871 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f709804c3c0 con 0x7f70980c45e0 2023-10-06T02:30:22.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.303+0000 7f70867fc700 1 -- 172.21.15.31:0/1619030871 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f708c013600 con 0x7f70980c45e0 2023-10-06T02:30:22.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.304+0000 7f70867fc700 1 --2- 172.21.15.31:0/1619030871 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f70780674d0 0x7f7078069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:22.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.304+0000 7f70867fc700 1 -- 172.21.15.31:0/1619030871 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f708c085780 con 0x7f70980c45e0 2023-10-06T02:30:22.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.304+0000 7f709659c700 1 --2- 172.21.15.31:0/1619030871 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f70780674d0 0x7f7078069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:22.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.305+0000 7f709659c700 1 --2- 172.21.15.31:0/1619030871 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f70780674d0 0x7f7078069980 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7088006f00 tx=0x7f70880091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:22.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.308+0000 7f70867fc700 1 -- 172.21.15.31:0/1619030871 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f708c051fc0 con 0x7f70980c45e0 2023-10-06T02:30:22.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.540+0000 7f709759e700 1 -- 172.21.15.31:0/1619030871 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f70980c78d0 con 0x7f70780674d0 2023-10-06T02:30:22.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.542+0000 7f70867fc700 1 -- 172.21.15.31:0/1619030871 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f70980c78d0 con 0x7f70780674d0 2023-10-06T02:30:22.543 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2023-10-06T02:30:22.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:30:22.544 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephfs" 2023-10-06T02:30:22.544 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:30:22.544 INFO:teuthology.orchestra.run.smithi031.stdout:] 2023-10-06T02:30:22.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.544+0000 7f709759e700 1 -- 172.21.15.31:0/1619030871 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f70780674d0 msgr2=0x7f7078069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:22.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.544+0000 7f709759e700 1 --2- 172.21.15.31:0/1619030871 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f70780674d0 0x7f7078069980 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7088006f00 tx=0x7f70880091a0 comp rx=0 tx=0).stop 2023-10-06T02:30:22.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.544+0000 7f709759e700 1 -- 172.21.15.31:0/1619030871 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70980c45e0 msgr2=0x7f70980b0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:22.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.544+0000 7f709759e700 1 --2- 172.21.15.31:0/1619030871 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70980c45e0 0x7f70980b0950 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f708c002a60 tx=0x7f708c007d60 comp rx=0 tx=0).stop 2023-10-06T02:30:22.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.544+0000 7f709759e700 1 -- 172.21.15.31:0/1619030871 shutdown_connections 2023-10-06T02:30:22.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.544+0000 7f709759e700 1 --2- 172.21.15.31:0/1619030871 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f70780674d0 0x7f7078069980 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:22.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.544+0000 7f709759e700 1 --2- 172.21.15.31:0/1619030871 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7098156900 0x7f7098156d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:22.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.544+0000 7f709759e700 1 --2- 172.21.15.31:0/1619030871 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70800051c0 0x7f70980c3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:22.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.544+0000 7f709759e700 1 --2- 172.21.15.31:0/1619030871 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70980c45e0 0x7f70980b0950 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:22.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.544+0000 7f709759e700 1 -- 172.21.15.31:0/1619030871 >> 172.21.15.31:0/1619030871 conn(0x7f709806d170 msgr2=0x7f709813f780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:22.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.545+0000 7f709759e700 1 -- 172.21.15.31:0/1619030871 shutdown_connections 2023-10-06T02:30:22.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.545+0000 7f709759e700 1 -- 172.21.15.31:0/1619030871 wait complete. 2023-10-06T02:30:22.563 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T02:30:22.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.751+0000 7f2659d9d700 1 Processor -- start 2023-10-06T02:30:22.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.751+0000 7f2659d9d700 1 -- start start 2023-10-06T02:30:22.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.753+0000 7f2659d9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2654140a70 0x7f2654069b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:22.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.753+0000 7f2659d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f265406de50 0x7f265406a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:22.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.753+0000 7f2659d9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f265415b640 con 0x7f265415b0a0 2023-10-06T02:30:22.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.753+0000 7f2659d9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26540c6800 con 0x7f2654140a70 2023-10-06T02:30:22.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.753+0000 7f2659d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f265415b920 con 0x7f265406de50 2023-10-06T02:30:22.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.753+0000 7f2658d9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2654140a70 0x7f2654069b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:22.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.753+0000 7f2658d9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2654140a70 0x7f2654069b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35552/0 (socket says 172.21.15.31:35552) 2023-10-06T02:30:22.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.753+0000 7f2658d9b700 1 -- 172.21.15.31:0/4227738598 learned_addr learned my addr 172.21.15.31:0/4227738598 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:22.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.753+0000 7f265959c700 1 --1- 172.21.15.31:0/4227738598 >> v1:172.21.15.31:6789/0 conn(0x7f265415b0a0 0x7f2654157a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:48464/0 (socket says 172.21.15.31:48464) 2023-10-06T02:30:22.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.753+0000 7f2653fff700 1 --2- 172.21.15.31:0/4227738598 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f265406de50 0x7f265406a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:22.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.753+0000 7f2658d9b700 1 -- 172.21.15.31:0/4227738598 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f265406de50 msgr2=0x7f265406a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:22.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.753+0000 7f2658d9b700 1 --2- 172.21.15.31:0/4227738598 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f265406de50 0x7f265406a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:22.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.753+0000 7f2658d9b700 1 -- 172.21.15.31:0/4227738598 >> v1:172.21.15.31:6789/0 conn(0x7f265415b0a0 legacy=0x7f2654157a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:22.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.753+0000 7f2658d9b700 1 -- 172.21.15.31:0/4227738598 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f265415bc00 con 0x7f2654140a70 2023-10-06T02:30:22.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.753+0000 7f2653fff700 1 --2- 172.21.15.31:0/4227738598 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f265406de50 0x7f265406a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:30:22.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.753+0000 7f26537fe700 1 -- 172.21.15.31:0/4227738598 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4292914760 0 0) 0x7f265415b640 con 0x7f265415b0a0 2023-10-06T02:30:22.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.754+0000 7f2658d9b700 1 --2- 172.21.15.31:0/4227738598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2654140a70 0x7f2654069b00 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f26480099b0 tx=0x7f2648009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85940fdae0ebb938 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:22.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.754+0000 7f26537fe700 1 -- 172.21.15.31:0/4227738598 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f264800b860 con 0x7f2654140a70 2023-10-06T02:30:22.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.754+0000 7f26537fe700 1 -- 172.21.15.31:0/4227738598 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f264800b9c0 con 0x7f2654140a70 2023-10-06T02:30:22.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.754+0000 7f26537fe700 1 -- 172.21.15.31:0/4227738598 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f264800bd60 con 0x7f2654140a70 2023-10-06T02:30:22.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.754+0000 7f2659d9d700 1 -- 172.21.15.31:0/4227738598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2654140a70 msgr2=0x7f2654069b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:22.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.754+0000 7f2659d9d700 1 --2- 172.21.15.31:0/4227738598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2654140a70 0x7f2654069b00 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f26480099b0 tx=0x7f2648009250 comp rx=0 tx=0).stop 2023-10-06T02:30:22.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.755+0000 7f2659d9d700 1 -- 172.21.15.31:0/4227738598 shutdown_connections 2023-10-06T02:30:22.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.755+0000 7f2659d9d700 1 --2- 172.21.15.31:0/4227738598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2654140a70 0x7f2654069b00 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:22.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.755+0000 7f2659d9d700 1 --2- 172.21.15.31:0/4227738598 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f265406de50 0x7f265406a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:22.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.755+0000 7f2659d9d700 1 -- 172.21.15.31:0/4227738598 >> 172.21.15.31:0/4227738598 conn(0x7f265406d110 msgr2=0x7f26540cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:22.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.755+0000 7f2659d9d700 1 -- 172.21.15.31:0/4227738598 shutdown_connections 2023-10-06T02:30:22.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.755+0000 7f2659d9d700 1 -- 172.21.15.31:0/4227738598 wait complete. 2023-10-06T02:30:22.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.756+0000 7f2659d9d700 1 Processor -- start 2023-10-06T02:30:22.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.756+0000 7f2659d9d700 1 -- start start 2023-10-06T02:30:22.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.757+0000 7f2659d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f265406de50 0x7f2654158430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:22.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.757+0000 7f2659d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2654140a70 0x7f26540c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:22.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.757+0000 7f2659d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f265415b0a0 0x7f26540ca240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:22.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.757+0000 7f2659d9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26540af1c0 con 0x7f265415b0a0 2023-10-06T02:30:22.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.757+0000 7f2659d9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26540afa60 con 0x7f265406de50 2023-10-06T02:30:22.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.757+0000 7f2659d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26540c6320 con 0x7f2654140a70 2023-10-06T02:30:22.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.758+0000 7f265959c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f265415b0a0 0x7f26540ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:22.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.758+0000 7f265959c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f265415b0a0 0x7f26540ca240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35560/0 (socket says 172.21.15.31:35560) 2023-10-06T02:30:22.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.758+0000 7f265959c700 1 -- 172.21.15.31:0/1975026869 learned_addr learned my addr 172.21.15.31:0/1975026869 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:22.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.758+0000 7f2653fff700 1 --2- 172.21.15.31:0/1975026869 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2654140a70 0x7f26540c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:22.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.758+0000 7f2658d9b700 1 --2- 172.21.15.31:0/1975026869 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f265406de50 0x7f2654158430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:22.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.758+0000 7f265959c700 1 -- 172.21.15.31:0/1975026869 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2654140a70 msgr2=0x7f26540c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:22.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.758+0000 7f265959c700 1 --2- 172.21.15.31:0/1975026869 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2654140a70 0x7f26540c9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:22.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.758+0000 7f265959c700 1 -- 172.21.15.31:0/1975026869 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f265406de50 msgr2=0x7f2654158430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:22.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.758+0000 7f265959c700 1 --2- 172.21.15.31:0/1975026869 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f265406de50 0x7f2654158430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:22.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.758+0000 7f265959c700 1 -- 172.21.15.31:0/1975026869 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26480094f0 con 0x7f265415b0a0 2023-10-06T02:30:22.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.758+0000 7f265959c700 1 --2- 172.21.15.31:0/1975026869 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f265415b0a0 0x7f26540ca240 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f264c00ccf0 tx=0x7f264c001a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:22.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.758+0000 7f2650ff9700 1 -- 172.21.15.31:0/1975026869 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f264c00ce30 con 0x7f265415b0a0 2023-10-06T02:30:22.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.758+0000 7f2650ff9700 1 -- 172.21.15.31:0/1975026869 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f264c005df0 con 0x7f265415b0a0 2023-10-06T02:30:22.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.758+0000 7f2650ff9700 1 -- 172.21.15.31:0/1975026869 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f264c00ab70 con 0x7f265415b0a0 2023-10-06T02:30:22.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.759+0000 7f2659d9d700 1 -- 172.21.15.31:0/1975026869 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26540af410 con 0x7f265415b0a0 2023-10-06T02:30:22.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.759+0000 7f2659d9d700 1 -- 172.21.15.31:0/1975026869 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2654158970 con 0x7f265415b0a0 2023-10-06T02:30:22.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.760+0000 7f2659d9d700 1 -- 172.21.15.31:0/1975026869 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26540afbf0 con 0x7f265415b0a0 2023-10-06T02:30:22.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.761+0000 7f2650ff9700 1 -- 172.21.15.31:0/1975026869 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f264c0070f0 con 0x7f265415b0a0 2023-10-06T02:30:22.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.761+0000 7f2650ff9700 1 --2- 172.21.15.31:0/1975026869 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f26380674d0 0x7f2638069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:22.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.762+0000 7f2650ff9700 1 -- 172.21.15.31:0/1975026869 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f264c085290 con 0x7f265415b0a0 2023-10-06T02:30:22.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.762+0000 7f2658d9b700 1 --2- 172.21.15.31:0/1975026869 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f26380674d0 0x7f2638069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:22.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.763+0000 7f2658d9b700 1 --2- 172.21.15.31:0/1975026869 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f26380674d0 0x7f2638069980 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f26480099b0 tx=0x7f2648017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:22.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.766+0000 7f2650ff9700 1 -- 172.21.15.31:0/1975026869 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f264c051b50 con 0x7f265415b0a0 2023-10-06T02:30:22.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.969+0000 7f2659d9d700 1 -- 172.21.15.31:0/1975026869 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f265404c3c0 con 0x7f265415b0a0 2023-10-06T02:30:22.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.969+0000 7f2650ff9700 1 -- 172.21.15.31:0/1975026869 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v24)=0 v24) v1 ==== 122+0+0 (secure 0 0 0) 0x7f264c055170 con 0x7f265415b0a0 2023-10-06T02:30:22.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.971+0000 7f2659d9d700 1 -- 172.21.15.31:0/1975026869 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f26380674d0 msgr2=0x7f2638069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:22.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.971+0000 7f2659d9d700 1 --2- 172.21.15.31:0/1975026869 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f26380674d0 0x7f2638069980 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f26480099b0 tx=0x7f2648017000 comp rx=0 tx=0).stop 2023-10-06T02:30:22.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.971+0000 7f2659d9d700 1 -- 172.21.15.31:0/1975026869 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f265415b0a0 msgr2=0x7f26540ca240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:22.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.971+0000 7f2659d9d700 1 --2- 172.21.15.31:0/1975026869 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f265415b0a0 0x7f26540ca240 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f264c00ccf0 tx=0x7f264c001a70 comp rx=0 tx=0).stop 2023-10-06T02:30:22.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.972+0000 7f2659d9d700 1 -- 172.21.15.31:0/1975026869 shutdown_connections 2023-10-06T02:30:22.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.972+0000 7f2659d9d700 1 --2- 172.21.15.31:0/1975026869 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f265415b0a0 0x7f26540ca240 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:22.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.972+0000 7f2659d9d700 1 --2- 172.21.15.31:0/1975026869 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f265406de50 0x7f2654158430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:22.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.972+0000 7f2659d9d700 1 --2- 172.21.15.31:0/1975026869 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f26380674d0 0x7f2638069980 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:22.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.972+0000 7f2659d9d700 1 --2- 172.21.15.31:0/1975026869 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2654140a70 0x7f26540c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:22.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.972+0000 7f2659d9d700 1 -- 172.21.15.31:0/1975026869 >> 172.21.15.31:0/1975026869 conn(0x7f265406d110 msgr2=0x7f26540c6c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:22.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.972+0000 7f2659d9d700 1 -- 172.21.15.31:0/1975026869 shutdown_connections 2023-10-06T02:30:22.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:22.972+0000 7f2659d9d700 1 -- 172.21.15.31:0/1975026869 wait complete. 2023-10-06T02:30:22.990 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create non_existent_subvolume subvolume_1 2023-10-06T02:30:23.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.179+0000 7f9d0cf9d700 1 Processor -- start 2023-10-06T02:30:23.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.180+0000 7f9d0cf9d700 1 -- start start 2023-10-06T02:30:23.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.181+0000 7f9d0cf9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9d08159540 0x7f9d080c1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:23.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.181+0000 7f9d0cf9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9d080c6be0 0x7f9d080c6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:23.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.181+0000 7f9d0cf9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d080c7520 con 0x7f9d08159910 2023-10-06T02:30:23.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.181+0000 7f9d0cf9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d08068250 con 0x7f9d08159540 2023-10-06T02:30:23.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.181+0000 7f9d0cf9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d08068530 con 0x7f9d080c6be0 2023-10-06T02:30:23.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.181+0000 7f9d077fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9d08159540 0x7f9d080c1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:23.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.181+0000 7f9d077fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9d08159540 0x7f9d080c1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35564/0 (socket says 172.21.15.31:35564) 2023-10-06T02:30:23.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.181+0000 7f9d077fe700 1 -- 172.21.15.31:0/2346513364 learned_addr learned my addr 172.21.15.31:0/2346513364 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:23.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.181+0000 7f9d06ffd700 1 --1- 172.21.15.31:0/2346513364 >> v1:172.21.15.31:6789/0 conn(0x7f9d08159910 0x7f9d080677b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:48470/0 (socket says 172.21.15.31:48470) 2023-10-06T02:30:23.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.182+0000 7f9d07fff700 1 --2- 172.21.15.31:0/2346513364 >> v2:172.21.15.62:3300/0 conn(0x7f9d080c6be0 0x7f9d080c6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:23.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.182+0000 7f9d077fe700 1 -- 172.21.15.31:0/2346513364 >> v2:172.21.15.62:3300/0 conn(0x7f9d080c6be0 msgr2=0x7f9d080c6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:23.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.182+0000 7f9d077fe700 1 --2- 172.21.15.31:0/2346513364 >> v2:172.21.15.62:3300/0 conn(0x7f9d080c6be0 0x7f9d080c6fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:23.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.182+0000 7f9d077fe700 1 -- 172.21.15.31:0/2346513364 >> v1:172.21.15.31:6789/0 conn(0x7f9d08159910 legacy=0x7f9d080677b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:23.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.182+0000 7f9d067fc700 1 -- 172.21.15.31:0/2346513364 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 38851424 0 0) 0x7f9d080c7520 con 0x7f9d08159910 2023-10-06T02:30:23.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.182+0000 7f9d077fe700 1 -- 172.21.15.31:0/2346513364 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d08068810 con 0x7f9d08159540 2023-10-06T02:30:23.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.182+0000 7f9d07fff700 1 --2- 172.21.15.31:0/2346513364 >> v2:172.21.15.62:3300/0 conn(0x7f9d080c6be0 0x7f9d080c6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:30:23.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.182+0000 7f9d077fe700 1 --2- 172.21.15.31:0/2346513364 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d08159540 0x7f9d080c1340 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f9d08069ec0 tx=0x7f9cf8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a74e683dfc0494ba server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:23.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.182+0000 7f9d067fc700 1 -- 172.21.15.31:0/2346513364 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf800b860 con 0x7f9d08159540 2023-10-06T02:30:23.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.182+0000 7f9d067fc700 1 -- 172.21.15.31:0/2346513364 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cf800b9c0 con 0x7f9d08159540 2023-10-06T02:30:23.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.182+0000 7f9d067fc700 1 -- 172.21.15.31:0/2346513364 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf800bd60 con 0x7f9d08159540 2023-10-06T02:30:23.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.183+0000 7f9d0cf9d700 1 -- 172.21.15.31:0/2346513364 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d08159540 msgr2=0x7f9d080c1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:23.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.183+0000 7f9d0cf9d700 1 --2- 172.21.15.31:0/2346513364 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d08159540 0x7f9d080c1340 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f9d08069ec0 tx=0x7f9cf8009250 comp rx=0 tx=0).stop 2023-10-06T02:30:23.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.183+0000 7f9d0cf9d700 1 -- 172.21.15.31:0/2346513364 shutdown_connections 2023-10-06T02:30:23.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.183+0000 7f9d0cf9d700 1 --2- 172.21.15.31:0/2346513364 >> v2:172.21.15.62:3300/0 conn(0x7f9d080c6be0 0x7f9d080c6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:23.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.183+0000 7f9d0cf9d700 1 --2- 172.21.15.31:0/2346513364 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d08159540 0x7f9d080c1340 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:23.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.183+0000 7f9d0cf9d700 1 -- 172.21.15.31:0/2346513364 >> 172.21.15.31:0/2346513364 conn(0x7f9d0806f330 msgr2=0x7f9d080cc5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:23.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.183+0000 7f9d0cf9d700 1 -- 172.21.15.31:0/2346513364 shutdown_connections 2023-10-06T02:30:23.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.183+0000 7f9d0cf9d700 1 -- 172.21.15.31:0/2346513364 wait complete. 2023-10-06T02:30:23.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.185+0000 7f9d0cf9d700 1 Processor -- start 2023-10-06T02:30:23.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.185+0000 7f9d0cf9d700 1 -- start start 2023-10-06T02:30:23.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.185+0000 7f9d0cf9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9d080c6be0 0x7f9d08144040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:23.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.186+0000 7f9d0cf9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d08159540 0x7f9d080af340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:23.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.186+0000 7f9d0cf9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d08159910 0x7f9d080c6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:23.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.186+0000 7f9d0cf9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0815aba0 con 0x7f9d08159540 2023-10-06T02:30:23.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.186+0000 7f9d0cf9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d08144580 con 0x7f9d08159910 2023-10-06T02:30:23.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.186+0000 7f9d0cf9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0815a870 con 0x7f9d080c6be0 2023-10-06T02:30:23.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.186+0000 7f9d06ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d08159540 0x7f9d080af340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:23.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.186+0000 7f9d06ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d08159540 0x7f9d080af340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35578/0 (socket says 172.21.15.31:35578) 2023-10-06T02:30:23.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.186+0000 7f9d06ffd700 1 -- 172.21.15.31:0/1850479727 learned_addr learned my addr 172.21.15.31:0/1850479727 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:23.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.186+0000 7f9d07fff700 1 --2- 172.21.15.31:0/1850479727 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d08159910 0x7f9d080c6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:23.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.186+0000 7f9d077fe700 1 --2- 172.21.15.31:0/1850479727 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9d080c6be0 0x7f9d08144040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:23.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.186+0000 7f9d06ffd700 1 -- 172.21.15.31:0/1850479727 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9d080c6be0 msgr2=0x7f9d08144040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:23.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.186+0000 7f9d06ffd700 1 --2- 172.21.15.31:0/1850479727 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9d080c6be0 0x7f9d08144040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:23.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.186+0000 7f9d06ffd700 1 -- 172.21.15.31:0/1850479727 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d08159910 msgr2=0x7f9d080c6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:23.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.186+0000 7f9d06ffd700 1 --2- 172.21.15.31:0/1850479727 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d08159910 0x7f9d080c6fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:23.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.186+0000 7f9d06ffd700 1 -- 172.21.15.31:0/1850479727 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf80094f0 con 0x7f9d08159540 2023-10-06T02:30:23.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.186+0000 7f9d07fff700 1 --2- 172.21.15.31:0/1850479727 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d08159910 0x7f9d080c6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:30:23.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.186+0000 7f9d077fe700 1 --2- 172.21.15.31:0/1850479727 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9d080c6be0 0x7f9d08144040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:30:23.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.186+0000 7f9d06ffd700 1 --2- 172.21.15.31:0/1850479727 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d08159540 0x7f9d080af340 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc00c5e0 tx=0x7f9cfc001040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:23.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.187+0000 7f9cef7fe700 1 -- 172.21.15.31:0/1850479727 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc005a70 con 0x7f9d08159540 2023-10-06T02:30:23.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.187+0000 7f9cef7fe700 1 -- 172.21.15.31:0/1850479727 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cfc005bd0 con 0x7f9d08159540 2023-10-06T02:30:23.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.187+0000 7f9cef7fe700 1 -- 172.21.15.31:0/1850479727 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc0070d0 con 0x7f9d08159540 2023-10-06T02:30:23.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.187+0000 7f9d0cf9d700 1 -- 172.21.15.31:0/1850479727 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d080afb90 con 0x7f9d08159540 2023-10-06T02:30:23.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.187+0000 7f9d0cf9d700 1 -- 172.21.15.31:0/1850479727 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d080b0150 con 0x7f9d08159540 2023-10-06T02:30:23.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.188+0000 7f9d0cf9d700 1 -- 172.21.15.31:0/1850479727 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d080b0370 con 0x7f9d08159540 2023-10-06T02:30:23.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.189+0000 7f9cef7fe700 1 -- 172.21.15.31:0/1850479727 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f9cfc007480 con 0x7f9d08159540 2023-10-06T02:30:23.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.190+0000 7f9cef7fe700 1 --2- 172.21.15.31:0/1850479727 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9cf40674d0 0x7f9cf4069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:23.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.190+0000 7f9cef7fe700 1 -- 172.21.15.31:0/1850479727 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f9cfc084580 con 0x7f9d08159540 2023-10-06T02:30:23.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.190+0000 7f9d077fe700 1 --2- 172.21.15.31:0/1850479727 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9cf40674d0 0x7f9cf4069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:23.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.191+0000 7f9d077fe700 1 --2- 172.21.15.31:0/1850479727 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9cf40674d0 0x7f9cf4069980 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f9d08069ec0 tx=0x7f9cf8017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:23.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.193+0000 7f9cef7fe700 1 -- 172.21.15.31:0/1850479727 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9cfc04b5a0 con 0x7f9d08159540 2023-10-06T02:30:23.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.427+0000 7f9d0cf9d700 1 -- 172.21.15.31:0/1850479727 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f9d0804c3c0 con 0x7f9cf40674d0 2023-10-06T02:30:23.430 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:30:23.429+0000 7f656537d700 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2023-10-06T02:30:23.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.429+0000 7f9cef7fe700 1 -- 172.21.15.31:0/1850479727 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) v1 ==== 45+0+0 (secure 0 0 0) 0x7f9d0804c3c0 con 0x7f9cf40674d0 2023-10-06T02:30:23.432 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2023-10-06T02:30:23.439 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-06T02:30:23.439 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm non_existent_subvolume subvolume_1 2023-10-06T02:30:23.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.627+0000 7f630619d700 1 Processor -- start 2023-10-06T02:30:23.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.627+0000 7f630619d700 1 -- start start 2023-10-06T02:30:23.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.628+0000 7f630619d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f63000cf110 0x7f630015de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:23.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.628+0000 7f630619d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f630015ac50 con 0x7f6300153a00 2023-10-06T02:30:23.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.628+0000 7f630619d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f630015d010 con 0x7f6300153dd0 2023-10-06T02:30:23.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.628+0000 7f630619d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f630015af60 con 0x7f63000cf110 2023-10-06T02:30:23.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.629+0000 7f630599c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f63000cf110 0x7f630015de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:23.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.629+0000 7f630519b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f6300153a00 0x7f630015a540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:48476/0 (socket says 172.21.15.31:48476) 2023-10-06T02:30:23.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.629+0000 7f630599c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f63000cf110 0x7f630015de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35590/0 (socket says 172.21.15.31:35590) 2023-10-06T02:30:23.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.629+0000 7f630519b700 1 -- 172.21.15.31:0/2815552807 learned_addr learned my addr 172.21.15.31:0/2815552807 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:30:23.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.629+0000 7f630599c700 1 -- 172.21.15.31:0/2815552807 >> v1:172.21.15.62:6789/0 conn(0x7f6300153dd0 legacy=0x7f63000cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:23.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.629+0000 7f630599c700 1 -- 172.21.15.31:0/2815552807 >> v1:172.21.15.31:6789/0 conn(0x7f6300153a00 legacy=0x7f630015a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:23.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.629+0000 7f630599c700 1 -- 172.21.15.31:0/2815552807 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f630015b240 con 0x7f63000cf110 2023-10-06T02:30:23.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.629+0000 7f62f7fff700 1 -- 172.21.15.31:0/2815552807 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 558522723 0 0) 0x7f630015ac50 con 0x7f6300153a00 2023-10-06T02:30:23.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.629+0000 7f630599c700 1 --2- 172.21.15.31:0/2815552807 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63000cf110 0x7f630015de50 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f62f8009e60 tx=0x7f62f8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbb9bcf76f66a781 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:23.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.630+0000 7f62f7fff700 1 -- 172.21.15.31:0/2815552807 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f800e070 con 0x7f63000cf110 2023-10-06T02:30:23.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.630+0000 7f62f7fff700 1 -- 172.21.15.31:0/2815552807 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62f800bc30 con 0x7f63000cf110 2023-10-06T02:30:23.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.630+0000 7f62f7fff700 1 -- 172.21.15.31:0/2815552807 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f8012430 con 0x7f63000cf110 2023-10-06T02:30:23.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.630+0000 7f630619d700 1 -- 172.21.15.31:0/2815552807 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63000cf110 msgr2=0x7f630015de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:23.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.630+0000 7f630619d700 1 --2- 172.21.15.31:0/2815552807 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63000cf110 0x7f630015de50 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f62f8009e60 tx=0x7f62f8009780 comp rx=0 tx=0).stop 2023-10-06T02:30:23.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.630+0000 7f630619d700 1 -- 172.21.15.31:0/2815552807 shutdown_connections 2023-10-06T02:30:23.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.630+0000 7f630619d700 1 --2- 172.21.15.31:0/2815552807 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63000cf110 0x7f630015de50 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:23.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.630+0000 7f630619d700 1 -- 172.21.15.31:0/2815552807 >> 172.21.15.31:0/2815552807 conn(0x7f63000703a0 msgr2=0x7f63000707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:23.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.630+0000 7f630619d700 1 -- 172.21.15.31:0/2815552807 shutdown_connections 2023-10-06T02:30:23.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.630+0000 7f630619d700 1 -- 172.21.15.31:0/2815552807 wait complete. 2023-10-06T02:30:23.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.632+0000 7f630619d700 1 Processor -- start 2023-10-06T02:30:23.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.632+0000 7f630619d700 1 -- start start 2023-10-06T02:30:23.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.633+0000 7f630619d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63000cf110 0x7f6300147b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:23.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.633+0000 7f630619d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6300153a00 0x7f630015bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:23.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.633+0000 7f630619d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6300153dd0 0x7f630015c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:23.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.633+0000 7f630619d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6300152580 con 0x7f63000cf110 2023-10-06T02:30:23.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.633+0000 7f630619d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6300158160 con 0x7f6300153a00 2023-10-06T02:30:23.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.633+0000 7f630619d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6300157550 con 0x7f6300153dd0 2023-10-06T02:30:23.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.633+0000 7f630519b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63000cf110 0x7f6300147b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:23.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.633+0000 7f630519b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63000cf110 0x7f6300147b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35592/0 (socket says 172.21.15.31:35592) 2023-10-06T02:30:23.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.633+0000 7f630519b700 1 -- 172.21.15.31:0/3963229665 learned_addr learned my addr 172.21.15.31:0/3963229665 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:23.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.633+0000 7f630599c700 1 --2- 172.21.15.31:0/3963229665 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6300153dd0 0x7f630015c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:23.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.633+0000 7f630499a700 1 --2- 172.21.15.31:0/3963229665 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6300153a00 0x7f630015bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:23.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.633+0000 7f630519b700 1 -- 172.21.15.31:0/3963229665 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6300153dd0 msgr2=0x7f630015c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:23.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.633+0000 7f630519b700 1 --2- 172.21.15.31:0/3963229665 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6300153dd0 0x7f630015c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:23.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.633+0000 7f630519b700 1 -- 172.21.15.31:0/3963229665 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6300153a00 msgr2=0x7f630015bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:23.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.633+0000 7f630519b700 1 --2- 172.21.15.31:0/3963229665 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6300153a00 0x7f630015bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:23.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.633+0000 7f630519b700 1 -- 172.21.15.31:0/3963229665 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62f8009a20 con 0x7f63000cf110 2023-10-06T02:30:23.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.633+0000 7f630599c700 1 --2- 172.21.15.31:0/3963229665 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6300153dd0 0x7f630015c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:30:23.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.633+0000 7f630499a700 1 --2- 172.21.15.31:0/3963229665 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6300153a00 0x7f630015bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:30:23.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.634+0000 7f630519b700 1 --2- 172.21.15.31:0/3963229665 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63000cf110 0x7f6300147b70 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f62fc00c7e0 tx=0x7f62fc007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:23.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.634+0000 7f62f57fa700 1 -- 172.21.15.31:0/3963229665 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62fc005b30 con 0x7f63000cf110 2023-10-06T02:30:23.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.634+0000 7f62f57fa700 1 -- 172.21.15.31:0/3963229665 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62fc005c90 con 0x7f63000cf110 2023-10-06T02:30:23.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.634+0000 7f630619d700 1 -- 172.21.15.31:0/3963229665 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6300157830 con 0x7f63000cf110 2023-10-06T02:30:23.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.634+0000 7f62f57fa700 1 -- 172.21.15.31:0/3963229665 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62fc0072e0 con 0x7f63000cf110 2023-10-06T02:30:23.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.634+0000 7f630619d700 1 -- 172.21.15.31:0/3963229665 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6300157df0 con 0x7f63000cf110 2023-10-06T02:30:23.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.635+0000 7f630619d700 1 -- 172.21.15.31:0/3963229665 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f630004c3c0 con 0x7f63000cf110 2023-10-06T02:30:23.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.636+0000 7f62f57fa700 1 -- 172.21.15.31:0/3963229665 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f62fc01b3b0 con 0x7f63000cf110 2023-10-06T02:30:23.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.637+0000 7f62f57fa700 1 --2- 172.21.15.31:0/3963229665 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f62e40674d0 0x7f62e4069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:23.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.637+0000 7f62f57fa700 1 -- 172.21.15.31:0/3963229665 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f62fc084520 con 0x7f63000cf110 2023-10-06T02:30:23.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.637+0000 7f630499a700 1 --2- 172.21.15.31:0/3963229665 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f62e40674d0 0x7f62e4069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:23.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.638+0000 7f630499a700 1 --2- 172.21.15.31:0/3963229665 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f62e40674d0 0x7f62e4069980 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f630014ccf0 tx=0x7f62f0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:23.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.641+0000 7f62f57fa700 1 -- 172.21.15.31:0/3963229665 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f62fc04b5a0 con 0x7f63000cf110 2023-10-06T02:30:23.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.873+0000 7f630619d700 1 -- 172.21.15.31:0/3963229665 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f630015b520 con 0x7f62e40674d0 2023-10-06T02:30:23.876 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:30:23.875+0000 7f656537d700 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2023-10-06T02:30:23.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:23.875+0000 7f62f57fa700 1 -- 172.21.15.31:0/3963229665 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) v1 ==== 45+0+0 (secure 0 0 0) 0x7f630015b520 con 0x7f62e40674d0 2023-10-06T02:30:23.878 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2023-10-06T02:30:23.885 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-06T02:30:23.886 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath non_existent_subvolume subvolume_1 2023-10-06T02:30:24.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.075+0000 7f49a719d700 1 Processor -- start 2023-10-06T02:30:24.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.075+0000 7f49a719d700 1 -- start start 2023-10-06T02:30:24.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.076+0000 7f49a719d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f49a0159540 0x7f49a00c1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:24.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.076+0000 7f49a719d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f49a0159910 0x7f49a00677b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:24.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.076+0000 7f49a719d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f49a0067cf0 0x7f49a00c6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:24.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.076+0000 7f49a719d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49a0068250 con 0x7f49a0159910 2023-10-06T02:30:24.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.076+0000 7f49a719d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49a0068530 con 0x7f49a0067cf0 2023-10-06T02:30:24.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.076+0000 7f49a719d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49a0068810 con 0x7f49a0159540 2023-10-06T02:30:24.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.076+0000 7f49a599a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f49a0159910 0x7f49a00677b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:24.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.076+0000 7f49a599a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f49a0159910 0x7f49a00677b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35596/0 (socket says 172.21.15.31:35596) 2023-10-06T02:30:24.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.076+0000 7f49a599a700 1 -- 172.21.15.31:0/2924368132 learned_addr learned my addr 172.21.15.31:0/2924368132 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:24.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.076+0000 7f49a699c700 1 --2- 172.21.15.31:0/2924368132 >> v2:172.21.15.62:3300/0 conn(0x7f49a0067cf0 0x7f49a00c6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:24.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.076+0000 7f49a619b700 1 --2- 172.21.15.31:0/2924368132 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f49a0159540 0x7f49a00c1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:24.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.077+0000 7f49a599a700 1 -- 172.21.15.31:0/2924368132 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f49a0159540 msgr2=0x7f49a00c1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:24.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.077+0000 7f49a599a700 1 --2- 172.21.15.31:0/2924368132 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f49a0159540 0x7f49a00c1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:24.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.077+0000 7f49a599a700 1 -- 172.21.15.31:0/2924368132 >> v2:172.21.15.62:3300/0 conn(0x7f49a0067cf0 msgr2=0x7f49a00c6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:24.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.077+0000 7f49a599a700 1 --2- 172.21.15.31:0/2924368132 >> v2:172.21.15.62:3300/0 conn(0x7f49a0067cf0 0x7f49a00c6be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:24.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.077+0000 7f49a599a700 1 -- 172.21.15.31:0/2924368132 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49a00a9ca0 con 0x7f49a0159910 2023-10-06T02:30:24.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.077+0000 7f49a619b700 1 --2- 172.21.15.31:0/2924368132 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f49a0159540 0x7f49a00c1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:30:24.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.077+0000 7f49a599a700 1 --2- 172.21.15.31:0/2924368132 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49a0159910 0x7f49a00677b0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f49a006c4a0 tx=0x7f4990009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e69200f15cda104 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:24.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.077+0000 7f49a5199700 1 -- 172.21.15.31:0/2924368132 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4990009e70 con 0x7f49a0159910 2023-10-06T02:30:24.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.077+0000 7f49a5199700 1 -- 172.21.15.31:0/2924368132 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f499000b860 con 0x7f49a0159910 2023-10-06T02:30:24.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.077+0000 7f49a5199700 1 -- 172.21.15.31:0/2924368132 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f499000bc00 con 0x7f49a0159910 2023-10-06T02:30:24.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.078+0000 7f49a719d700 1 -- 172.21.15.31:0/2924368132 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49a0159910 msgr2=0x7f49a00677b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:24.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.078+0000 7f49a719d700 1 --2- 172.21.15.31:0/2924368132 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49a0159910 0x7f49a00677b0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f49a006c4a0 tx=0x7f4990009250 comp rx=0 tx=0).stop 2023-10-06T02:30:24.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.078+0000 7f49a719d700 1 -- 172.21.15.31:0/2924368132 shutdown_connections 2023-10-06T02:30:24.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.078+0000 7f49a719d700 1 --2- 172.21.15.31:0/2924368132 >> v2:172.21.15.62:3300/0 conn(0x7f49a0067cf0 0x7f49a00c6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:24.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.078+0000 7f49a719d700 1 --2- 172.21.15.31:0/2924368132 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f49a0159540 0x7f49a00c1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:24.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.078+0000 7f49a719d700 1 --2- 172.21.15.31:0/2924368132 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49a0159910 0x7f49a00677b0 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:24.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.078+0000 7f49a719d700 1 -- 172.21.15.31:0/2924368132 >> 172.21.15.31:0/2924368132 conn(0x7f49a006f330 msgr2=0x7f49a00cc5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:24.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.078+0000 7f49a719d700 1 -- 172.21.15.31:0/2924368132 shutdown_connections 2023-10-06T02:30:24.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.078+0000 7f49a719d700 1 -- 172.21.15.31:0/2924368132 wait complete. 2023-10-06T02:30:24.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.080+0000 7f49a719d700 1 Processor -- start 2023-10-06T02:30:24.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.080+0000 7f49a719d700 1 -- start start 2023-10-06T02:30:24.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.081+0000 7f49a719d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49a0067cf0 0x7f49a016a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:24.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.081+0000 7f49a719d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49a0159540 0x7f49a00af680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:24.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.081+0000 7f49a719d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f49a0159910 0x7f49a014f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:24.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.081+0000 7f49a719d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49a014ba10 con 0x7f49a0159540 2023-10-06T02:30:24.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.081+0000 7f49a719d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49a006a650 con 0x7f49a0067cf0 2023-10-06T02:30:24.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.081+0000 7f49a719d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49a00afbc0 con 0x7f49a0159910 2023-10-06T02:30:24.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.081+0000 7f49a599a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49a0159540 0x7f49a00af680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:24.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.081+0000 7f49a599a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49a0159540 0x7f49a00af680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35608/0 (socket says 172.21.15.31:35608) 2023-10-06T02:30:24.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.081+0000 7f49a599a700 1 -- 172.21.15.31:0/660373142 learned_addr learned my addr 172.21.15.31:0/660373142 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:24.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.081+0000 7f49a619b700 1 --2- 172.21.15.31:0/660373142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49a0067cf0 0x7f49a016a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:24.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.081+0000 7f49a699c700 1 --2- 172.21.15.31:0/660373142 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f49a0159910 0x7f49a014f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:24.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.081+0000 7f49a599a700 1 -- 172.21.15.31:0/660373142 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f49a0159910 msgr2=0x7f49a014f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:24.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.081+0000 7f49a599a700 1 --2- 172.21.15.31:0/660373142 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f49a0159910 0x7f49a014f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:24.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.081+0000 7f49a599a700 1 -- 172.21.15.31:0/660373142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49a0067cf0 msgr2=0x7f49a016a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:24.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.081+0000 7f49a599a700 1 --2- 172.21.15.31:0/660373142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49a0067cf0 0x7f49a016a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:24.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.082+0000 7f49a599a700 1 -- 172.21.15.31:0/660373142 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49900094f0 con 0x7f49a0159540 2023-10-06T02:30:24.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.082+0000 7f49a599a700 1 --2- 172.21.15.31:0/660373142 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49a0159540 0x7f49a00af680 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f49900090b0 tx=0x7f4990017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:24.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.082+0000 7f49967fc700 1 -- 172.21.15.31:0/660373142 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f499000bbe0 con 0x7f49a0159540 2023-10-06T02:30:24.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.082+0000 7f49967fc700 1 -- 172.21.15.31:0/660373142 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4990011420 con 0x7f49a0159540 2023-10-06T02:30:24.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.082+0000 7f49a719d700 1 -- 172.21.15.31:0/660373142 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49a014f8d0 con 0x7f49a0159540 2023-10-06T02:30:24.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.082+0000 7f49967fc700 1 -- 172.21.15.31:0/660373142 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4990011820 con 0x7f49a0159540 2023-10-06T02:30:24.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.082+0000 7f49a719d700 1 -- 172.21.15.31:0/660373142 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49a00b00b0 con 0x7f49a0159540 2023-10-06T02:30:24.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.083+0000 7f49a719d700 1 -- 172.21.15.31:0/660373142 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49a004c3c0 con 0x7f49a0159540 2023-10-06T02:30:24.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.084+0000 7f49967fc700 1 -- 172.21.15.31:0/660373142 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f4990011c50 con 0x7f49a0159540 2023-10-06T02:30:24.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.085+0000 7f49967fc700 1 --2- 172.21.15.31:0/660373142 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f49840674d0 0x7f4984069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:24.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.085+0000 7f49967fc700 1 -- 172.21.15.31:0/660373142 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f49900867f0 con 0x7f49a0159540 2023-10-06T02:30:24.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.085+0000 7f49a619b700 1 --2- 172.21.15.31:0/660373142 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f49840674d0 0x7f4984069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:24.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.086+0000 7f49a619b700 1 --2- 172.21.15.31:0/660373142 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f49840674d0 0x7f4984069980 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f499c006f00 tx=0x7f499c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:24.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.089+0000 7f49967fc700 1 -- 172.21.15.31:0/660373142 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f49900530f0 con 0x7f49a0159540 2023-10-06T02:30:24.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.328+0000 7f49a719d700 1 -- 172.21.15.31:0/660373142 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f49a014f8d0 con 0x7f49840674d0 2023-10-06T02:30:24.330 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:30:24.330+0000 7f656537d700 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2023-10-06T02:30:24.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.330+0000 7f49967fc700 1 -- 172.21.15.31:0/660373142 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) v1 ==== 45+0+0 (secure 0 0 0) 0x7f49a014f8d0 con 0x7f49840674d0 2023-10-06T02:30:24.333 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2023-10-06T02:30:24.340 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-06T02:30:24.341 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume info non_existent_subvolume subvolume_1 2023-10-06T02:30:24.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.529+0000 7f9593fff700 1 Processor -- start 2023-10-06T02:30:24.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.529+0000 7f9593fff700 1 -- start start 2023-10-06T02:30:24.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.530+0000 7f9593fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f95940ccce0 0x7f959406de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:24.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.530+0000 7f9593fff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f959406b790 con 0x7f95940cc840 2023-10-06T02:30:24.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.530+0000 7f9593fff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9594155fa0 con 0x7f95940c7930 2023-10-06T02:30:24.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.530+0000 7f9593fff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f959406ab80 con 0x7f95940ccce0 2023-10-06T02:30:24.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.530+0000 7f95937fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f95940ccce0 0x7f959406de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:24.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.530+0000 7f95927fc700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f95940cc840 0x7f9594155890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:48490/0 (socket says 172.21.15.31:48490) 2023-10-06T02:30:24.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.530+0000 7f95937fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f95940ccce0 0x7f959406de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35622/0 (socket says 172.21.15.31:35622) 2023-10-06T02:30:24.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.530+0000 7f95927fc700 1 -- 172.21.15.31:0/154802210 learned_addr learned my addr 172.21.15.31:0/154802210 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:30:24.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.531+0000 7f95937fe700 1 -- 172.21.15.31:0/154802210 >> v1:172.21.15.62:6789/0 conn(0x7f95940c7930 legacy=0x7f959406a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:24.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.531+0000 7f95937fe700 1 -- 172.21.15.31:0/154802210 >> v1:172.21.15.31:6789/0 conn(0x7f95940cc840 legacy=0x7f9594155890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:24.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.531+0000 7f95937fe700 1 -- 172.21.15.31:0/154802210 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f959406ae60 con 0x7f95940ccce0 2023-10-06T02:30:24.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.531+0000 7f9591ffb700 1 -- 172.21.15.31:0/154802210 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4119357641 0 0) 0x7f959406b790 con 0x7f95940cc840 2023-10-06T02:30:24.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.531+0000 7f95937fe700 1 --2- 172.21.15.31:0/154802210 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95940ccce0 0x7f959406de50 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f958c009e60 tx=0x7f958c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f55b953cc9db2872 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:24.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.531+0000 7f9591ffb700 1 -- 172.21.15.31:0/154802210 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f958c00e070 con 0x7f95940ccce0 2023-10-06T02:30:24.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.531+0000 7f9591ffb700 1 -- 172.21.15.31:0/154802210 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f958c00bc30 con 0x7f95940ccce0 2023-10-06T02:30:24.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.531+0000 7f9591ffb700 1 -- 172.21.15.31:0/154802210 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f958c012430 con 0x7f95940ccce0 2023-10-06T02:30:24.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.532+0000 7f9593fff700 1 -- 172.21.15.31:0/154802210 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95940ccce0 msgr2=0x7f959406de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:24.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.532+0000 7f9593fff700 1 --2- 172.21.15.31:0/154802210 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95940ccce0 0x7f959406de50 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f958c009e60 tx=0x7f958c009780 comp rx=0 tx=0).stop 2023-10-06T02:30:24.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.532+0000 7f9593fff700 1 -- 172.21.15.31:0/154802210 shutdown_connections 2023-10-06T02:30:24.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.532+0000 7f9593fff700 1 --2- 172.21.15.31:0/154802210 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95940ccce0 0x7f959406de50 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:24.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.532+0000 7f9593fff700 1 -- 172.21.15.31:0/154802210 >> 172.21.15.31:0/154802210 conn(0x7f959406d110 msgr2=0x7f95940cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:24.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.532+0000 7f9593fff700 1 -- 172.21.15.31:0/154802210 shutdown_connections 2023-10-06T02:30:24.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.532+0000 7f9593fff700 1 -- 172.21.15.31:0/154802210 wait complete. 2023-10-06T02:30:24.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.533+0000 7f9593fff700 1 Processor -- start 2023-10-06T02:30:24.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.534+0000 7f9593fff700 1 -- start start 2023-10-06T02:30:24.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.534+0000 7f9593fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95940c7930 0x7f9594165100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:24.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.534+0000 7f9593fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95940cc840 0x7f95940c5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:24.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.534+0000 7f9593fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f95940ccce0 0x7f95940c4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:24.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.534+0000 7f9593fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95940c3280 con 0x7f95940cc840 2023-10-06T02:30:24.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.535+0000 7f9593fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95940c3560 con 0x7f95940c7930 2023-10-06T02:30:24.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.535+0000 7f9593fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95940c3840 con 0x7f95940ccce0 2023-10-06T02:30:24.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.535+0000 7f95927fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95940cc840 0x7f95940c5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:24.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.535+0000 7f95927fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95940cc840 0x7f95940c5b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35624/0 (socket says 172.21.15.31:35624) 2023-10-06T02:30:24.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.535+0000 7f95927fc700 1 -- 172.21.15.31:0/4167587179 learned_addr learned my addr 172.21.15.31:0/4167587179 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:24.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.535+0000 7f9592ffd700 1 --2- 172.21.15.31:0/4167587179 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95940c7930 0x7f9594165100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:24.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.535+0000 7f95937fe700 1 --2- 172.21.15.31:0/4167587179 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f95940ccce0 0x7f95940c4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:24.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.535+0000 7f95927fc700 1 -- 172.21.15.31:0/4167587179 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f95940ccce0 msgr2=0x7f95940c4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:24.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.535+0000 7f95927fc700 1 --2- 172.21.15.31:0/4167587179 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f95940ccce0 0x7f95940c4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:24.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.535+0000 7f95927fc700 1 -- 172.21.15.31:0/4167587179 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95940c7930 msgr2=0x7f9594165100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:24.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.535+0000 7f95927fc700 1 --2- 172.21.15.31:0/4167587179 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95940c7930 0x7f9594165100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:24.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.535+0000 7f95927fc700 1 -- 172.21.15.31:0/4167587179 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f958c009a20 con 0x7f95940cc840 2023-10-06T02:30:24.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.535+0000 7f95927fc700 1 --2- 172.21.15.31:0/4167587179 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95940cc840 0x7f95940c5b10 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f958800c5b0 tx=0x7f9588001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:24.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.536+0000 7f957affd700 1 -- 172.21.15.31:0/4167587179 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9588005b30 con 0x7f95940cc840 2023-10-06T02:30:24.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.536+0000 7f957affd700 1 -- 172.21.15.31:0/4167587179 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9588005c90 con 0x7f95940cc840 2023-10-06T02:30:24.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.536+0000 7f9593fff700 1 -- 172.21.15.31:0/4167587179 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95940c3b20 con 0x7f95940cc840 2023-10-06T02:30:24.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.536+0000 7f957affd700 1 -- 172.21.15.31:0/4167587179 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9588007300 con 0x7f95940cc840 2023-10-06T02:30:24.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.536+0000 7f9593fff700 1 -- 172.21.15.31:0/4167587179 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9594165640 con 0x7f95940cc840 2023-10-06T02:30:24.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.537+0000 7f9593fff700 1 -- 172.21.15.31:0/4167587179 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95940c4040 con 0x7f95940cc840 2023-10-06T02:30:24.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.538+0000 7f957affd700 1 -- 172.21.15.31:0/4167587179 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f958801b3b0 con 0x7f95940cc840 2023-10-06T02:30:24.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.539+0000 7f957affd700 1 --2- 172.21.15.31:0/4167587179 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f95800674d0 0x7f9580069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:24.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.539+0000 7f957affd700 1 -- 172.21.15.31:0/4167587179 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f95880846a0 con 0x7f95940cc840 2023-10-06T02:30:24.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.539+0000 7f9592ffd700 1 --2- 172.21.15.31:0/4167587179 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f95800674d0 0x7f9580069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:24.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.540+0000 7f9592ffd700 1 --2- 172.21.15.31:0/4167587179 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f95800674d0 0x7f9580069980 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f95840037c0 tx=0x7f9584009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:24.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.543+0000 7f957affd700 1 -- 172.21.15.31:0/4167587179 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f95940c4040 con 0x7f95940cc840 2023-10-06T02:30:24.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.779+0000 7f9593fff700 1 -- 172.21.15.31:0/4167587179 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "fs subvolume info", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f95940c4040 con 0x7f95800674d0 2023-10-06T02:30:24.782 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:30:24.781+0000 7f656537d700 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2023-10-06T02:30:24.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.781+0000 7f957affd700 1 -- 172.21.15.31:0/4167587179 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) v1 ==== 45+0+0 (secure 0 0 0) 0x7f95940c4040 con 0x7f95800674d0 2023-10-06T02:30:24.785 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2023-10-06T02:30:24.792 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-06T02:30:24.792 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume resize non_existent_subvolume subvolname_1 inf 2023-10-06T02:30:24.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.982+0000 7f178a59d700 1 Processor -- start 2023-10-06T02:30:24.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.982+0000 7f178a59d700 1 -- start start 2023-10-06T02:30:24.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.983+0000 7f178a59d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1784142ca0 0x7f178415dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:24.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.983+0000 7f178a59d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1784143070 0x7f17840cbf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:24.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.983+0000 7f178a59d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f178415a6f0 0x7f17840cad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:24.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.983+0000 7f178a59d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f178415d8d0 con 0x7f1784143070 2023-10-06T02:30:24.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.983+0000 7f178a59d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f178415d2f0 con 0x7f178415a6f0 2023-10-06T02:30:24.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.983+0000 7f178a59d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f178415ab90 con 0x7f1784142ca0 2023-10-06T02:30:24.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.983+0000 7f1788d9a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1784143070 0x7f17840cbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:24.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.983+0000 7f1788d9a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1784143070 0x7f17840cbf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35628/0 (socket says 172.21.15.31:35628) 2023-10-06T02:30:24.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.983+0000 7f1788d9a700 1 -- 172.21.15.31:0/972766683 learned_addr learned my addr 172.21.15.31:0/972766683 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:24.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.984+0000 7f1789d9c700 1 --2- 172.21.15.31:0/972766683 >> v2:172.21.15.62:3300/0 conn(0x7f178415a6f0 0x7f17840cad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:24.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.984+0000 7f1788d9a700 1 -- 172.21.15.31:0/972766683 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1784142ca0 msgr2=0x7f178415dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:24.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.984+0000 7f178959b700 1 --2- 172.21.15.31:0/972766683 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1784142ca0 0x7f178415dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:24.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.984+0000 7f1788d9a700 1 --2- 172.21.15.31:0/972766683 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1784142ca0 0x7f178415dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:24.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.984+0000 7f1788d9a700 1 -- 172.21.15.31:0/972766683 >> v2:172.21.15.62:3300/0 conn(0x7f178415a6f0 msgr2=0x7f17840cad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:24.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.984+0000 7f1788d9a700 1 --2- 172.21.15.31:0/972766683 >> v2:172.21.15.62:3300/0 conn(0x7f178415a6f0 0x7f17840cad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:24.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.984+0000 7f1788d9a700 1 -- 172.21.15.31:0/972766683 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17840cc630 con 0x7f1784143070 2023-10-06T02:30:24.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.984+0000 7f1789d9c700 1 --2- 172.21.15.31:0/972766683 >> v2:172.21.15.62:3300/0 conn(0x7f178415a6f0 0x7f17840cad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:30:24.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.984+0000 7f178959b700 1 --2- 172.21.15.31:0/972766683 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1784142ca0 0x7f178415dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:30:24.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.984+0000 7f1788d9a700 1 --2- 172.21.15.31:0/972766683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1784143070 0x7f17840cbf30 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f17840ce330 tx=0x7f1774009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c42a9417fe1a70a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:24.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.984+0000 7f177bfff700 1 -- 172.21.15.31:0/972766683 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f177400b860 con 0x7f1784143070 2023-10-06T02:30:24.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.985+0000 7f177bfff700 1 -- 172.21.15.31:0/972766683 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f177400b9c0 con 0x7f1784143070 2023-10-06T02:30:24.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.985+0000 7f177bfff700 1 -- 172.21.15.31:0/972766683 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f177400bd60 con 0x7f1784143070 2023-10-06T02:30:24.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.985+0000 7f178a59d700 1 -- 172.21.15.31:0/972766683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1784143070 msgr2=0x7f17840cbf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:24.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.985+0000 7f178a59d700 1 --2- 172.21.15.31:0/972766683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1784143070 0x7f17840cbf30 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f17840ce330 tx=0x7f1774009250 comp rx=0 tx=0).stop 2023-10-06T02:30:24.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.985+0000 7f178a59d700 1 -- 172.21.15.31:0/972766683 shutdown_connections 2023-10-06T02:30:24.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.985+0000 7f178a59d700 1 --2- 172.21.15.31:0/972766683 >> v2:172.21.15.62:3300/0 conn(0x7f178415a6f0 0x7f17840cad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:24.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.985+0000 7f178a59d700 1 --2- 172.21.15.31:0/972766683 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1784142ca0 0x7f178415dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:24.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.985+0000 7f178a59d700 1 --2- 172.21.15.31:0/972766683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1784143070 0x7f17840cbf30 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:24.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.985+0000 7f178a59d700 1 -- 172.21.15.31:0/972766683 >> 172.21.15.31:0/972766683 conn(0x7f1784067c80 msgr2=0x7f1784068080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:24.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.985+0000 7f178a59d700 1 -- 172.21.15.31:0/972766683 shutdown_connections 2023-10-06T02:30:24.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.985+0000 7f178a59d700 1 -- 172.21.15.31:0/972766683 wait complete. 2023-10-06T02:30:24.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.987+0000 7f178a59d700 1 Processor -- start 2023-10-06T02:30:24.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.987+0000 7f178a59d700 1 -- start start 2023-10-06T02:30:24.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.988+0000 7f178a59d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1784142ca0 0x7f17840b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:24.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.988+0000 7f178a59d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1784143070 0x7f178413fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:24.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.988+0000 7f178a59d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f178415a6f0 0x7f178413ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:24.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.988+0000 7f178a59d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17841404b0 con 0x7f1784142ca0 2023-10-06T02:30:24.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.988+0000 7f178a59d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17840af0c0 con 0x7f1784143070 2023-10-06T02:30:24.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.988+0000 7f178a59d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17840ae4f0 con 0x7f178415a6f0 2023-10-06T02:30:24.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.988+0000 7f178959b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1784142ca0 0x7f17840b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:24.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.988+0000 7f178959b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1784142ca0 0x7f17840b1350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35634/0 (socket says 172.21.15.31:35634) 2023-10-06T02:30:24.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.988+0000 7f178959b700 1 -- 172.21.15.31:0/2899820707 learned_addr learned my addr 172.21.15.31:0/2899820707 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:24.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.988+0000 7f1789d9c700 1 --2- 172.21.15.31:0/2899820707 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f178415a6f0 0x7f178413ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:24.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.988+0000 7f1788d9a700 1 --2- 172.21.15.31:0/2899820707 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1784143070 0x7f178413fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:24.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.988+0000 7f178959b700 1 -- 172.21.15.31:0/2899820707 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f178415a6f0 msgr2=0x7f178413ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:24.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.988+0000 7f178959b700 1 --2- 172.21.15.31:0/2899820707 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f178415a6f0 0x7f178413ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:24.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.988+0000 7f178959b700 1 -- 172.21.15.31:0/2899820707 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1784143070 msgr2=0x7f178413fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:24.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.988+0000 7f178959b700 1 --2- 172.21.15.31:0/2899820707 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1784143070 0x7f178413fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:24.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.988+0000 7f178959b700 1 -- 172.21.15.31:0/2899820707 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17740094f0 con 0x7f1784142ca0 2023-10-06T02:30:24.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.988+0000 7f1788d9a700 1 --2- 172.21.15.31:0/2899820707 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1784143070 0x7f178413fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:30:24.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.989+0000 7f178959b700 1 --2- 172.21.15.31:0/2899820707 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1784142ca0 0x7f17840b1350 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f178000c880 tx=0x7f1780008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:24.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.989+0000 7f17797fa700 1 -- 172.21.15.31:0/2899820707 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1780011600 con 0x7f1784142ca0 2023-10-06T02:30:24.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.989+0000 7f17797fa700 1 -- 172.21.15.31:0/2899820707 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1780011760 con 0x7f1784142ca0 2023-10-06T02:30:24.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.989+0000 7f178a59d700 1 -- 172.21.15.31:0/2899820707 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17840af550 con 0x7f1784142ca0 2023-10-06T02:30:24.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.989+0000 7f17797fa700 1 -- 172.21.15.31:0/2899820707 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1780011b20 con 0x7f1784142ca0 2023-10-06T02:30:24.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.989+0000 7f178a59d700 1 -- 172.21.15.31:0/2899820707 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17840aed50 con 0x7f1784142ca0 2023-10-06T02:30:24.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.990+0000 7f178a59d700 1 -- 172.21.15.31:0/2899820707 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f178404c3c0 con 0x7f1784142ca0 2023-10-06T02:30:24.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.991+0000 7f17797fa700 1 -- 172.21.15.31:0/2899820707 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f17800193b0 con 0x7f1784142ca0 2023-10-06T02:30:24.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.992+0000 7f17797fa700 1 --2- 172.21.15.31:0/2899820707 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f17680674d0 0x7f1768069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:24.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.992+0000 7f17797fa700 1 -- 172.21.15.31:0/2899820707 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f17800851f0 con 0x7f1784142ca0 2023-10-06T02:30:24.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.992+0000 7f1788d9a700 1 --2- 172.21.15.31:0/2899820707 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f17680674d0 0x7f1768069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:24.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.993+0000 7f1788d9a700 1 --2- 172.21.15.31:0/2899820707 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f17680674d0 0x7f1768069980 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f17840ce330 tx=0x7f1774017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:24.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:24.996+0000 7f17797fa700 1 -- 172.21.15.31:0/2899820707 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f17800553b0 con 0x7f1784142ca0 2023-10-06T02:30:25.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.240+0000 7f178a59d700 1 -- 172.21.15.31:0/2899820707 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "fs subvolume resize", "vol_name": "non_existent_subvolume", "sub_name": "subvolname_1", "new_size": "inf", "target": ["mon-mgr", ""]}) v1 -- 0x7f17840cb910 con 0x7f17680674d0 2023-10-06T02:30:25.243 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:30:25.243+0000 7f656537d700 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2023-10-06T02:30:25.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.243+0000 7f17797fa700 1 -- 172.21.15.31:0/2899820707 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) v1 ==== 45+0+0 (secure 0 0 0) 0x7f17840cb910 con 0x7f17680674d0 2023-10-06T02:30:25.246 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2023-10-06T02:30:25.253 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-06T02:30:25.253 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume pin non_existent_subvolume subvolname_1 export 1 2023-10-06T02:30:25.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.445+0000 7ff443fff700 1 Processor -- start 2023-10-06T02:30:25.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.445+0000 7ff443fff700 1 -- start start 2023-10-06T02:30:25.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.446+0000 7ff443fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff444142ca0 0x7ff44415dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:25.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.446+0000 7ff443fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff444143070 0x7ff4440cbf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:25.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.446+0000 7ff443fff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff44415a6f0 0x7ff4440cad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:25.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.446+0000 7ff443fff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff44415d8d0 con 0x7ff444142ca0 2023-10-06T02:30:25.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.446+0000 7ff443fff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff44415d2f0 con 0x7ff44415a6f0 2023-10-06T02:30:25.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.446+0000 7ff443fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4440cc630 con 0x7ff444143070 2023-10-06T02:30:25.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.447+0000 7ff442ffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff444142ca0 0x7ff44415dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:25.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.447+0000 7ff442ffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff444142ca0 0x7ff44415dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35646/0 (socket says 172.21.15.31:35646) 2023-10-06T02:30:25.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.447+0000 7ff442ffd700 1 -- 172.21.15.31:0/979174886 learned_addr learned my addr 172.21.15.31:0/979174886 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:25.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.447+0000 7ff4427fc700 1 --2- 172.21.15.31:0/979174886 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff444143070 0x7ff4440cbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:25.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.447+0000 7ff4437fe700 1 --2- 172.21.15.31:0/979174886 >> v2:172.21.15.62:3300/0 conn(0x7ff44415a6f0 0x7ff4440cad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:25.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.447+0000 7ff442ffd700 1 -- 172.21.15.31:0/979174886 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff444143070 msgr2=0x7ff4440cbf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:25.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.447+0000 7ff442ffd700 1 --2- 172.21.15.31:0/979174886 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff444143070 0x7ff4440cbf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:25.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.447+0000 7ff442ffd700 1 -- 172.21.15.31:0/979174886 >> v2:172.21.15.62:3300/0 conn(0x7ff44415a6f0 msgr2=0x7ff4440cad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:25.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.447+0000 7ff442ffd700 1 --2- 172.21.15.31:0/979174886 >> v2:172.21.15.62:3300/0 conn(0x7ff44415a6f0 0x7ff4440cad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:25.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.447+0000 7ff442ffd700 1 -- 172.21.15.31:0/979174886 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff44415d560 con 0x7ff444142ca0 2023-10-06T02:30:25.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.447+0000 7ff4437fe700 1 --2- 172.21.15.31:0/979174886 >> v2:172.21.15.62:3300/0 conn(0x7ff44415a6f0 0x7ff4440cad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:30:25.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.447+0000 7ff442ffd700 1 --2- 172.21.15.31:0/979174886 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff444142ca0 0x7ff44415dd60 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7ff44406a760 tx=0x7ff434009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31c4aef108609b03 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:25.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.448+0000 7ff441ffb700 1 -- 172.21.15.31:0/979174886 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff43400b860 con 0x7ff444142ca0 2023-10-06T02:30:25.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.448+0000 7ff441ffb700 1 -- 172.21.15.31:0/979174886 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff43400b9c0 con 0x7ff444142ca0 2023-10-06T02:30:25.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.448+0000 7ff441ffb700 1 -- 172.21.15.31:0/979174886 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff43400bd60 con 0x7ff444142ca0 2023-10-06T02:30:25.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.448+0000 7ff443fff700 1 -- 172.21.15.31:0/979174886 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff444142ca0 msgr2=0x7ff44415dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:25.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.448+0000 7ff443fff700 1 --2- 172.21.15.31:0/979174886 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff444142ca0 0x7ff44415dd60 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7ff44406a760 tx=0x7ff434009250 comp rx=0 tx=0).stop 2023-10-06T02:30:25.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.448+0000 7ff443fff700 1 -- 172.21.15.31:0/979174886 shutdown_connections 2023-10-06T02:30:25.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.448+0000 7ff443fff700 1 --2- 172.21.15.31:0/979174886 >> v2:172.21.15.62:3300/0 conn(0x7ff44415a6f0 0x7ff4440cad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:25.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.448+0000 7ff443fff700 1 --2- 172.21.15.31:0/979174886 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff444142ca0 0x7ff44415dd60 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:25.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.448+0000 7ff443fff700 1 --2- 172.21.15.31:0/979174886 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff444143070 0x7ff4440cbf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:25.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.448+0000 7ff443fff700 1 -- 172.21.15.31:0/979174886 >> 172.21.15.31:0/979174886 conn(0x7ff444067c80 msgr2=0x7ff444068080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:25.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.448+0000 7ff443fff700 1 -- 172.21.15.31:0/979174886 shutdown_connections 2023-10-06T02:30:25.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.448+0000 7ff443fff700 1 -- 172.21.15.31:0/979174886 wait complete. 2023-10-06T02:30:25.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.450+0000 7ff443fff700 1 Processor -- start 2023-10-06T02:30:25.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.450+0000 7ff443fff700 1 -- start start 2023-10-06T02:30:25.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.451+0000 7ff443fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff444142ca0 0x7ff444158f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:25.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.451+0000 7ff443fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff444143070 0x7ff4440ad150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:25.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.451+0000 7ff443fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff44415a6f0 0x7ff4440c3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:25.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.451+0000 7ff443fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4440aab20 con 0x7ff444142ca0 2023-10-06T02:30:25.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.451+0000 7ff443fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4440b1ac0 con 0x7ff444143070 2023-10-06T02:30:25.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.451+0000 7ff443fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4441594a0 con 0x7ff44415a6f0 2023-10-06T02:30:25.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.451+0000 7ff442ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff444142ca0 0x7ff444158f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:25.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.451+0000 7ff442ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff444142ca0 0x7ff444158f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35648/0 (socket says 172.21.15.31:35648) 2023-10-06T02:30:25.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.451+0000 7ff442ffd700 1 -- 172.21.15.31:0/2919061681 learned_addr learned my addr 172.21.15.31:0/2919061681 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:25.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.451+0000 7ff4427fc700 1 --2- 172.21.15.31:0/2919061681 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff444143070 0x7ff4440ad150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:25.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.451+0000 7ff4437fe700 1 --2- 172.21.15.31:0/2919061681 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff44415a6f0 0x7ff4440c3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:25.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.452+0000 7ff442ffd700 1 -- 172.21.15.31:0/2919061681 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff44415a6f0 msgr2=0x7ff4440c3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:25.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.452+0000 7ff442ffd700 1 --2- 172.21.15.31:0/2919061681 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff44415a6f0 0x7ff4440c3620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:25.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.452+0000 7ff442ffd700 1 -- 172.21.15.31:0/2919061681 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff444143070 msgr2=0x7ff4440ad150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:25.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.452+0000 7ff442ffd700 1 --2- 172.21.15.31:0/2919061681 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff444143070 0x7ff4440ad150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:25.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.452+0000 7ff442ffd700 1 -- 172.21.15.31:0/2919061681 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4340094f0 con 0x7ff444142ca0 2023-10-06T02:30:25.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.452+0000 7ff4427fc700 1 --2- 172.21.15.31:0/2919061681 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff444143070 0x7ff4440ad150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:30:25.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.452+0000 7ff442ffd700 1 --2- 172.21.15.31:0/2919061681 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff444142ca0 0x7ff444158f60 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7ff434006fd0 tx=0x7ff434017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:25.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.452+0000 7ff42affd700 1 -- 172.21.15.31:0/2919061681 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff434024440 con 0x7ff444142ca0 2023-10-06T02:30:25.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.452+0000 7ff42affd700 1 -- 172.21.15.31:0/2919061681 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4340245a0 con 0x7ff444142ca0 2023-10-06T02:30:25.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.452+0000 7ff42affd700 1 -- 172.21.15.31:0/2919061681 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff434024960 con 0x7ff444142ca0 2023-10-06T02:30:25.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.452+0000 7ff443fff700 1 -- 172.21.15.31:0/2919061681 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4440b2300 con 0x7ff444142ca0 2023-10-06T02:30:25.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.452+0000 7ff443fff700 1 -- 172.21.15.31:0/2919061681 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4440b1cb0 con 0x7ff444142ca0 2023-10-06T02:30:25.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.453+0000 7ff443fff700 1 -- 172.21.15.31:0/2919061681 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff44404c3c0 con 0x7ff444142ca0 2023-10-06T02:30:25.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.454+0000 7ff42affd700 1 -- 172.21.15.31:0/2919061681 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7ff43401a420 con 0x7ff444142ca0 2023-10-06T02:30:25.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.455+0000 7ff42affd700 1 --2- 172.21.15.31:0/2919061681 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff430067500 0x7ff4300699b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:25.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.455+0000 7ff42affd700 1 -- 172.21.15.31:0/2919061681 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6086+0+0 (secure 0 0 0) 0x7ff434086d30 con 0x7ff444142ca0 2023-10-06T02:30:25.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.456+0000 7ff4427fc700 1 --2- 172.21.15.31:0/2919061681 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff430067500 0x7ff4300699b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:25.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.456+0000 7ff4427fc700 1 --2- 172.21.15.31:0/2919061681 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff430067500 0x7ff4300699b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff44415de90 tx=0x7ff438008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:25.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.460+0000 7ff42affd700 1 -- 172.21.15.31:0/2919061681 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff434053570 con 0x7ff444142ca0 2023-10-06T02:30:25.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.703+0000 7ff443fff700 1 -- 172.21.15.31:0/2919061681 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "fs subvolume pin", "vol_name": "non_existent_subvolume", "sub_name": "subvolname_1", "pin_type": "export", "pin_setting": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7ff4440b2050 con 0x7ff430067500 2023-10-06T02:30:25.706 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:30:25.705+0000 7f656537d700 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2023-10-06T02:30:25.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.706+0000 7ff42affd700 1 -- 172.21.15.31:0/2919061681 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) v1 ==== 45+0+0 (secure 0 0 0) 0x7ff4440b2050 con 0x7ff430067500 2023-10-06T02:30:25.708 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2023-10-06T02:30:25.716 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-06T02:30:25.716 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls non_existent_subvolume 2023-10-06T02:30:25.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.905+0000 7fc40219d700 1 Processor -- start 2023-10-06T02:30:25.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.905+0000 7fc40219d700 1 -- start start 2023-10-06T02:30:25.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.906+0000 7fc40219d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc3fc153a00 0x7fc3fc153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:25.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.906+0000 7fc40219d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc3fc15de50 0x7fc3fc15a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:25.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.906+0000 7fc40219d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc3fc0cea00 0x7fc3fc0cee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:25.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.906+0000 7fc40219d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3fc15af60 con 0x7fc3fc0cea00 2023-10-06T02:30:25.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.906+0000 7fc40219d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3fc15b240 con 0x7fc3fc15de50 2023-10-06T02:30:25.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.906+0000 7fc40219d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3fc15b520 con 0x7fc3fc153a00 2023-10-06T02:30:25.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.906+0000 7fc40199c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc3fc0cea00 0x7fc3fc0cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:25.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.907+0000 7fc40199c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc3fc0cea00 0x7fc3fc0cee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35654/0 (socket says 172.21.15.31:35654) 2023-10-06T02:30:25.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.907+0000 7fc40199c700 1 -- 172.21.15.31:0/671869167 learned_addr learned my addr 172.21.15.31:0/671869167 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:25.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.907+0000 7fc40119b700 1 --2- 172.21.15.31:0/671869167 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc3fc153a00 0x7fc3fc153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:25.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.907+0000 7fc40099a700 1 --2- 172.21.15.31:0/671869167 >> v2:172.21.15.62:3300/0 conn(0x7fc3fc15de50 0x7fc3fc15a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:25.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.907+0000 7fc40199c700 1 -- 172.21.15.31:0/671869167 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc3fc153a00 msgr2=0x7fc3fc153dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:25.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.907+0000 7fc40199c700 1 --2- 172.21.15.31:0/671869167 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc3fc153a00 0x7fc3fc153dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:25.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.907+0000 7fc40199c700 1 -- 172.21.15.31:0/671869167 >> v2:172.21.15.62:3300/0 conn(0x7fc3fc15de50 msgr2=0x7fc3fc15a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:25.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.907+0000 7fc40199c700 1 --2- 172.21.15.31:0/671869167 >> v2:172.21.15.62:3300/0 conn(0x7fc3fc15de50 0x7fc3fc15a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:25.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.907+0000 7fc40199c700 1 -- 172.21.15.31:0/671869167 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3fc15b800 con 0x7fc3fc0cea00 2023-10-06T02:30:25.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.907+0000 7fc40099a700 1 --2- 172.21.15.31:0/671869167 >> v2:172.21.15.62:3300/0 conn(0x7fc3fc15de50 0x7fc3fc15a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:30:25.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.907+0000 7fc40199c700 1 --2- 172.21.15.31:0/671869167 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3fc0cea00 0x7fc3fc0cee40 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc06e800 tx=0x7fc3f4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6da0010190c5c7a0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:25.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.908+0000 7fc3f3fff700 1 -- 172.21.15.31:0/671869167 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3f400e070 con 0x7fc3fc0cea00 2023-10-06T02:30:25.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.908+0000 7fc3f3fff700 1 -- 172.21.15.31:0/671869167 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3f400ba50 con 0x7fc3fc0cea00 2023-10-06T02:30:25.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.908+0000 7fc3f3fff700 1 -- 172.21.15.31:0/671869167 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3f40092e0 con 0x7fc3fc0cea00 2023-10-06T02:30:25.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.908+0000 7fc40219d700 1 -- 172.21.15.31:0/671869167 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3fc0cea00 msgr2=0x7fc3fc0cee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:25.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.908+0000 7fc40219d700 1 --2- 172.21.15.31:0/671869167 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3fc0cea00 0x7fc3fc0cee40 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc06e800 tx=0x7fc3f4009840 comp rx=0 tx=0).stop 2023-10-06T02:30:25.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.908+0000 7fc40219d700 1 -- 172.21.15.31:0/671869167 shutdown_connections 2023-10-06T02:30:25.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.908+0000 7fc40219d700 1 --2- 172.21.15.31:0/671869167 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3fc0cea00 0x7fc3fc0cee40 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:25.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.908+0000 7fc40219d700 1 --2- 172.21.15.31:0/671869167 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc3fc153a00 0x7fc3fc153dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:25.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.908+0000 7fc40219d700 1 --2- 172.21.15.31:0/671869167 >> v2:172.21.15.62:3300/0 conn(0x7fc3fc15de50 0x7fc3fc15a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:25.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.908+0000 7fc40219d700 1 -- 172.21.15.31:0/671869167 >> 172.21.15.31:0/671869167 conn(0x7fc3fc0703a0 msgr2=0x7fc3fc0707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:25.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.908+0000 7fc40219d700 1 -- 172.21.15.31:0/671869167 shutdown_connections 2023-10-06T02:30:25.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.908+0000 7fc40219d700 1 -- 172.21.15.31:0/671869167 wait complete. 2023-10-06T02:30:25.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.910+0000 7fc40219d700 1 Processor -- start 2023-10-06T02:30:25.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.910+0000 7fc40219d700 1 -- start start 2023-10-06T02:30:25.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.911+0000 7fc40219d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc3fc0cea00 0x7fc3fc14ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:25.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.911+0000 7fc40219d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3fc153a00 0x7fc3fc0cd420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:25.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.911+0000 7fc40219d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3fc15de50 0x7fc3fc0b32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:25.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.911+0000 7fc40219d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3fc0c85e0 con 0x7fc3fc153a00 2023-10-06T02:30:25.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.911+0000 7fc40219d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3fc0cc130 con 0x7fc3fc15de50 2023-10-06T02:30:25.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.911+0000 7fc40219d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3fc0ccf90 con 0x7fc3fc0cea00 2023-10-06T02:30:25.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.911+0000 7fc40099a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3fc153a00 0x7fc3fc0cd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:25.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.911+0000 7fc40099a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3fc153a00 0x7fc3fc0cd420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35662/0 (socket says 172.21.15.31:35662) 2023-10-06T02:30:25.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.911+0000 7fc40099a700 1 -- 172.21.15.31:0/3320576756 learned_addr learned my addr 172.21.15.31:0/3320576756 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:25.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.911+0000 7fc40119b700 1 --2- 172.21.15.31:0/3320576756 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc3fc0cea00 0x7fc3fc14ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:25.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.911+0000 7fc40199c700 1 --2- 172.21.15.31:0/3320576756 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3fc15de50 0x7fc3fc0b32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:25.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.912+0000 7fc40099a700 1 -- 172.21.15.31:0/3320576756 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc3fc0cea00 msgr2=0x7fc3fc14ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:25.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.912+0000 7fc40099a700 1 --2- 172.21.15.31:0/3320576756 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc3fc0cea00 0x7fc3fc14ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:25.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.912+0000 7fc40099a700 1 -- 172.21.15.31:0/3320576756 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3fc15de50 msgr2=0x7fc3fc0b32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:25.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.912+0000 7fc40099a700 1 --2- 172.21.15.31:0/3320576756 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3fc15de50 0x7fc3fc0b32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:25.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.912+0000 7fc40099a700 1 -- 172.21.15.31:0/3320576756 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3f4009ae0 con 0x7fc3fc153a00 2023-10-06T02:30:25.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.912+0000 7fc40199c700 1 --2- 172.21.15.31:0/3320576756 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3fc15de50 0x7fc3fc0b32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:30:25.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.912+0000 7fc40099a700 1 --2- 172.21.15.31:0/3320576756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3fc153a00 0x7fc3fc0cd420 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec00d860 tx=0x7fc3ec008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:25.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.912+0000 7fc3f17fa700 1 -- 172.21.15.31:0/3320576756 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec0125f0 con 0x7fc3fc153a00 2023-10-06T02:30:25.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.912+0000 7fc3f17fa700 1 -- 172.21.15.31:0/3320576756 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3ec012750 con 0x7fc3fc153a00 2023-10-06T02:30:25.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.912+0000 7fc3f17fa700 1 -- 172.21.15.31:0/3320576756 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec012ad0 con 0x7fc3fc153a00 2023-10-06T02:30:25.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.912+0000 7fc40219d700 1 -- 172.21.15.31:0/3320576756 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3fc0cc3e0 con 0x7fc3fc153a00 2023-10-06T02:30:25.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.912+0000 7fc40219d700 1 -- 172.21.15.31:0/3320576756 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3fc0ccc20 con 0x7fc3fc153a00 2023-10-06T02:30:25.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.913+0000 7fc40219d700 1 -- 172.21.15.31:0/3320576756 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3fc0cd100 con 0x7fc3fc153a00 2023-10-06T02:30:25.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.914+0000 7fc3f17fa700 1 -- 172.21.15.31:0/3320576756 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7fc3ec014430 con 0x7fc3fc153a00 2023-10-06T02:30:25.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.915+0000 7fc3f17fa700 1 --2- 172.21.15.31:0/3320576756 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc3e00674d0 0x7fc3e0069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:25.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.915+0000 7fc3f17fa700 1 -- 172.21.15.31:0/3320576756 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fc3ec0855b0 con 0x7fc3fc153a00 2023-10-06T02:30:25.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.915+0000 7fc40119b700 1 --2- 172.21.15.31:0/3320576756 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc3e00674d0 0x7fc3e0069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:25.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.916+0000 7fc40119b700 1 --2- 172.21.15.31:0/3320576756 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc3e00674d0 0x7fc3e0069980 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc3f8006fd0 tx=0x7fc3f8008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:25.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:25.919+0000 7fc3f17fa700 1 -- 172.21.15.31:0/3320576756 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc3ec055880 con 0x7fc3fc153a00 2023-10-06T02:30:26.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.152+0000 7fc40219d700 1 -- 172.21.15.31:0/3320576756 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "non_existent_subvolume", "target": ["mon-mgr", ""]}) v1 -- 0x7fc3fc04c3c0 con 0x7fc3e00674d0 2023-10-06T02:30:26.155 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:30:26.154+0000 7f656537d700 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2023-10-06T02:30:26.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.154+0000 7fc3f17fa700 1 -- 172.21.15.31:0/3320576756 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) v1 ==== 45+0+0 (secure 0 0 0) 0x7fc3fc04c3c0 con 0x7fc3e00674d0 2023-10-06T02:30:26.157 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2023-10-06T02:30:26.165 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-06T02:30:26.165 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create non_existent_subvolume subvolume_1 snapshot_1 2023-10-06T02:30:26.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.355+0000 7f199b39d700 1 Processor -- start 2023-10-06T02:30:26.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.355+0000 7f199b39d700 1 -- start start 2023-10-06T02:30:26.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.356+0000 7f199b39d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f19940c2a90 0x7f1994156aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:26.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.356+0000 7f199b39d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f19940c85d0 0x7f19940cc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:26.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.356+0000 7f199b39d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19941579b0 con 0x7f19940c8200 2023-10-06T02:30:26.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.356+0000 7f199b39d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19940c1390 con 0x7f19940c85d0 2023-10-06T02:30:26.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.356+0000 7f199b39d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19941573f0 con 0x7f19940c2a90 2023-10-06T02:30:26.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.357+0000 7f199ab9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f19940c85d0 0x7f19940cc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:26.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.357+0000 7f199ab9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f19940c85d0 0x7f19940cc2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35674/0 (socket says 172.21.15.31:35674) 2023-10-06T02:30:26.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.357+0000 7f199ab9c700 1 -- 172.21.15.31:0/2340035418 learned_addr learned my addr 172.21.15.31:0/2340035418 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:26.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.357+0000 7f1999b9a700 1 --2- 172.21.15.31:0/2340035418 >> v2:172.21.15.62:3300/0 conn(0x7f19940c2a90 0x7f1994156aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:26.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.357+0000 7f199ab9c700 1 -- 172.21.15.31:0/2340035418 >> v2:172.21.15.62:3300/0 conn(0x7f19940c2a90 msgr2=0x7f1994156aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:26.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.357+0000 7f199ab9c700 1 --2- 172.21.15.31:0/2340035418 >> v2:172.21.15.62:3300/0 conn(0x7f19940c2a90 0x7f1994156aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:26.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.357+0000 7f199ab9c700 1 -- 172.21.15.31:0/2340035418 >> v1:172.21.15.62:6789/0 conn(0x7f19940c8200 legacy=0x7f19940c1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:26.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.357+0000 7f199ab9c700 1 -- 172.21.15.31:0/2340035418 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19941576d0 con 0x7f19940c85d0 2023-10-06T02:30:26.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.357+0000 7f199ab9c700 1 --2- 172.21.15.31:0/2340035418 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19940c85d0 0x7f19940cc2f0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f199415b1d0 tx=0x7f198c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=196165402cf7a1d5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:26.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.358+0000 7f1999399700 1 -- 172.21.15.31:0/2340035418 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f198c0092e0 con 0x7f19940c85d0 2023-10-06T02:30:26.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.358+0000 7f1999399700 1 -- 172.21.15.31:0/2340035418 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f198c00ba20 con 0x7f19940c85d0 2023-10-06T02:30:26.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.358+0000 7f1999399700 1 -- 172.21.15.31:0/2340035418 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f198c00bde0 con 0x7f19940c85d0 2023-10-06T02:30:26.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.358+0000 7f199b39d700 1 -- 172.21.15.31:0/2340035418 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19940c85d0 msgr2=0x7f19940cc2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:26.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.358+0000 7f199b39d700 1 --2- 172.21.15.31:0/2340035418 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19940c85d0 0x7f19940cc2f0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f199415b1d0 tx=0x7f198c009840 comp rx=0 tx=0).stop 2023-10-06T02:30:26.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.358+0000 7f199b39d700 1 -- 172.21.15.31:0/2340035418 shutdown_connections 2023-10-06T02:30:26.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.358+0000 7f199b39d700 1 --2- 172.21.15.31:0/2340035418 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19940c85d0 0x7f19940cc2f0 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:26.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.358+0000 7f199b39d700 1 --2- 172.21.15.31:0/2340035418 >> v2:172.21.15.62:3300/0 conn(0x7f19940c2a90 0x7f1994156aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:26.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.358+0000 7f199b39d700 1 -- 172.21.15.31:0/2340035418 >> 172.21.15.31:0/2340035418 conn(0x7f199406c6a0 msgr2=0x7f199406caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:26.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.358+0000 7f199b39d700 1 -- 172.21.15.31:0/2340035418 shutdown_connections 2023-10-06T02:30:26.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.358+0000 7f199b39d700 1 -- 172.21.15.31:0/2340035418 wait complete. 2023-10-06T02:30:26.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.360+0000 7f199b39d700 1 Processor -- start 2023-10-06T02:30:26.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.360+0000 7f199b39d700 1 -- start start 2023-10-06T02:30:26.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.361+0000 7f199b39d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19940c2a90 0x7f19940b1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:26.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.361+0000 7f199b39d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f19940c8200 0x7f19940cc310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:26.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.361+0000 7f199b39d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19940c85d0 0x7f19940cc850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:26.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.361+0000 7f199b39d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19940b1580 con 0x7f19940c2a90 2023-10-06T02:30:26.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.361+0000 7f199b39d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19940a9c50 con 0x7f19940c85d0 2023-10-06T02:30:26.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.361+0000 7f199b39d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19940a9f30 con 0x7f19940c8200 2023-10-06T02:30:26.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.361+0000 7f199a39b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19940c2a90 0x7f19940b1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:26.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.361+0000 7f199a39b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19940c2a90 0x7f19940b1880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35682/0 (socket says 172.21.15.31:35682) 2023-10-06T02:30:26.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.361+0000 7f199a39b700 1 -- 172.21.15.31:0/3648281133 learned_addr learned my addr 172.21.15.31:0/3648281133 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:26.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.361+0000 7f199ab9c700 1 --2- 172.21.15.31:0/3648281133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19940c85d0 0x7f19940cc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:26.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.362+0000 7f1999b9a700 1 --2- 172.21.15.31:0/3648281133 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f19940c8200 0x7f19940cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:26.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.362+0000 7f199a39b700 1 -- 172.21.15.31:0/3648281133 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f19940c8200 msgr2=0x7f19940cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:26.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.362+0000 7f199a39b700 1 --2- 172.21.15.31:0/3648281133 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f19940c8200 0x7f19940cc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:26.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.362+0000 7f199a39b700 1 -- 172.21.15.31:0/3648281133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19940c85d0 msgr2=0x7f19940cc850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:26.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.362+0000 7f199a39b700 1 --2- 172.21.15.31:0/3648281133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19940c85d0 0x7f19940cc850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:26.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.362+0000 7f199a39b700 1 -- 172.21.15.31:0/3648281133 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f198c009ae0 con 0x7f19940c2a90 2023-10-06T02:30:26.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.362+0000 7f199a39b700 1 --2- 172.21.15.31:0/3648281133 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19940c2a90 0x7f19940b1880 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f199000c590 tx=0x7f199000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:26.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.362+0000 7f198a7fc700 1 -- 172.21.15.31:0/3648281133 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1990011590 con 0x7f19940c2a90 2023-10-06T02:30:26.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.362+0000 7f198a7fc700 1 -- 172.21.15.31:0/3648281133 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1990011710 con 0x7f19940c2a90 2023-10-06T02:30:26.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.362+0000 7f198a7fc700 1 -- 172.21.15.31:0/3648281133 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1990011ad0 con 0x7f19940c2a90 2023-10-06T02:30:26.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.362+0000 7f199b39d700 1 -- 172.21.15.31:0/3648281133 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19940aa210 con 0x7f19940c2a90 2023-10-06T02:30:26.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.363+0000 7f199b39d700 1 -- 172.21.15.31:0/3648281133 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19940aa7d0 con 0x7f19940c2a90 2023-10-06T02:30:26.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.363+0000 7f199b39d700 1 -- 172.21.15.31:0/3648281133 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f199404c3c0 con 0x7f19940c2a90 2023-10-06T02:30:26.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.364+0000 7f198a7fc700 1 -- 172.21.15.31:0/3648281133 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f199001b3b0 con 0x7f19940c2a90 2023-10-06T02:30:26.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.365+0000 7f198a7fc700 1 --2- 172.21.15.31:0/3648281133 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f19780674d0 0x7f1978069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:26.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.365+0000 7f198a7fc700 1 -- 172.21.15.31:0/3648281133 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f1990084590 con 0x7f19940c2a90 2023-10-06T02:30:26.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.366+0000 7f1999b9a700 1 --2- 172.21.15.31:0/3648281133 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f19780674d0 0x7f1978069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:26.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.366+0000 7f1999b9a700 1 --2- 172.21.15.31:0/3648281133 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f19780674d0 0x7f1978069980 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f19940ccfa0 tx=0x7f19840091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:26.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.369+0000 7f198a7fc700 1 -- 172.21.15.31:0/3648281133 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1990050e10 con 0x7f19940c2a90 2023-10-06T02:30:26.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.607+0000 7f199b39d700 1 -- 172.21.15.31:0/3648281133 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f19940ab150 con 0x7f19780674d0 2023-10-06T02:30:26.610 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:30:26.609+0000 7f656537d700 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2023-10-06T02:30:26.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.609+0000 7f198a7fc700 1 -- 172.21.15.31:0/3648281133 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) v1 ==== 45+0+0 (secure 0 0 0) 0x7f19940ab150 con 0x7f19780674d0 2023-10-06T02:30:26.612 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2023-10-06T02:30:26.619 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-06T02:30:26.620 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm non_existent_subvolume subvolume_1 snapshot_1 2023-10-06T02:30:26.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.811+0000 7f2e62d9d700 1 Processor -- start 2023-10-06T02:30:26.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.811+0000 7f2e62d9d700 1 -- start start 2023-10-06T02:30:26.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.813+0000 7f2e62d9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2e5c15b1a0 0x7f2e5c0c6b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:26.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.813+0000 7f2e62d9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e5c0c8680 con 0x7f2e5c157380 2023-10-06T02:30:26.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.813+0000 7f2e62d9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e5c15b6a0 con 0x7f2e5c157750 2023-10-06T02:30:26.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.813+0000 7f2e62d9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e5c06a7f0 con 0x7f2e5c15b1a0 2023-10-06T02:30:26.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.813+0000 7f2e6159a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2e5c15b1a0 0x7f2e5c0c6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:26.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.813+0000 7f2e6159a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2e5c15b1a0 0x7f2e5c0c6b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35688/0 (socket says 172.21.15.31:35688) 2023-10-06T02:30:26.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.813+0000 7f2e6159a700 1 -- 172.21.15.31:0/3586682763 learned_addr learned my addr 172.21.15.31:0/3586682763 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:26.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.813+0000 7f2e61d9b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f2e5c157380 0x7f2e5c1585f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:48504/0 (socket says 172.21.15.31:48504) 2023-10-06T02:30:26.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.813+0000 7f2e6159a700 1 -- 172.21.15.31:0/3586682763 >> v1:172.21.15.62:6789/0 conn(0x7f2e5c157750 legacy=0x7f2e5c0ae080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:26.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.813+0000 7f2e6159a700 1 -- 172.21.15.31:0/3586682763 >> v1:172.21.15.31:6789/0 conn(0x7f2e5c157380 legacy=0x7f2e5c1585f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:26.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.813+0000 7f2e6159a700 1 -- 172.21.15.31:0/3586682763 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e5c15b980 con 0x7f2e5c15b1a0 2023-10-06T02:30:26.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.813+0000 7f2e60d99700 1 -- 172.21.15.31:0/3586682763 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3207989506 0 0) 0x7f2e5c0c8680 con 0x7f2e5c157380 2023-10-06T02:30:26.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.814+0000 7f2e6159a700 1 --2- 172.21.15.31:0/3586682763 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e5c15b1a0 0x7f2e5c0c6b60 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c0099b0 tx=0x7f2e4c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eddb9abc21acb476 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:26.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.814+0000 7f2e60d99700 1 -- 172.21.15.31:0/3586682763 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c00b860 con 0x7f2e5c15b1a0 2023-10-06T02:30:26.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.814+0000 7f2e60d99700 1 -- 172.21.15.31:0/3586682763 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e4c00b9c0 con 0x7f2e5c15b1a0 2023-10-06T02:30:26.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.814+0000 7f2e60d99700 1 -- 172.21.15.31:0/3586682763 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c00bd60 con 0x7f2e5c15b1a0 2023-10-06T02:30:26.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.814+0000 7f2e62d9d700 1 -- 172.21.15.31:0/3586682763 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e5c15b1a0 msgr2=0x7f2e5c0c6b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:26.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.814+0000 7f2e62d9d700 1 --2- 172.21.15.31:0/3586682763 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e5c15b1a0 0x7f2e5c0c6b60 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c0099b0 tx=0x7f2e4c009250 comp rx=0 tx=0).stop 2023-10-06T02:30:26.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.815+0000 7f2e62d9d700 1 -- 172.21.15.31:0/3586682763 shutdown_connections 2023-10-06T02:30:26.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.815+0000 7f2e62d9d700 1 --2- 172.21.15.31:0/3586682763 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e5c15b1a0 0x7f2e5c0c6b60 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:26.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.815+0000 7f2e62d9d700 1 -- 172.21.15.31:0/3586682763 >> 172.21.15.31:0/3586682763 conn(0x7f2e5c06c6a0 msgr2=0x7f2e5c06caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:26.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.815+0000 7f2e62d9d700 1 -- 172.21.15.31:0/3586682763 shutdown_connections 2023-10-06T02:30:26.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.815+0000 7f2e62d9d700 1 -- 172.21.15.31:0/3586682763 wait complete. 2023-10-06T02:30:26.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.816+0000 7f2e62d9d700 1 Processor -- start 2023-10-06T02:30:26.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.816+0000 7f2e62d9d700 1 -- start start 2023-10-06T02:30:26.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.817+0000 7f2e62d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e5c157380 0x7f2e5c14f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:26.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.817+0000 7f2e62d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2e5c157750 0x7f2e5c14f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:26.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.817+0000 7f2e62d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e5c15b1a0 0x7f2e5c0c2340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:26.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.817+0000 7f2e62d9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e5c0c1eb0 con 0x7f2e5c157380 2023-10-06T02:30:26.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.817+0000 7f2e62d9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e5c0c12a0 con 0x7f2e5c15b1a0 2023-10-06T02:30:26.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.817+0000 7f2e62d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e5c0c1580 con 0x7f2e5c157750 2023-10-06T02:30:26.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.817+0000 7f2e61d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e5c157380 0x7f2e5c14f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:26.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.817+0000 7f2e61d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e5c157380 0x7f2e5c14f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35690/0 (socket says 172.21.15.31:35690) 2023-10-06T02:30:26.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.817+0000 7f2e61d9b700 1 -- 172.21.15.31:0/985157577 learned_addr learned my addr 172.21.15.31:0/985157577 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:26.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.818+0000 7f2e6259c700 1 --2- 172.21.15.31:0/985157577 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e5c15b1a0 0x7f2e5c0c2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:26.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.818+0000 7f2e6159a700 1 --2- 172.21.15.31:0/985157577 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2e5c157750 0x7f2e5c14f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:26.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.818+0000 7f2e61d9b700 1 -- 172.21.15.31:0/985157577 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2e5c157750 msgr2=0x7f2e5c14f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:26.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.818+0000 7f2e61d9b700 1 --2- 172.21.15.31:0/985157577 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2e5c157750 0x7f2e5c14f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:26.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.818+0000 7f2e61d9b700 1 -- 172.21.15.31:0/985157577 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e5c15b1a0 msgr2=0x7f2e5c0c2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:26.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.818+0000 7f2e61d9b700 1 --2- 172.21.15.31:0/985157577 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e5c15b1a0 0x7f2e5c0c2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:26.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.818+0000 7f2e61d9b700 1 -- 172.21.15.31:0/985157577 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e4c0094f0 con 0x7f2e5c157380 2023-10-06T02:30:26.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.818+0000 7f2e61d9b700 1 --2- 172.21.15.31:0/985157577 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e5c157380 0x7f2e5c14f0d0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f2e5800c560 tx=0x7f2e58007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:26.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.818+0000 7f2e51ffb700 1 -- 172.21.15.31:0/985157577 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e58005a70 con 0x7f2e5c157380 2023-10-06T02:30:26.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.818+0000 7f2e51ffb700 1 -- 172.21.15.31:0/985157577 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e58005bd0 con 0x7f2e5c157380 2023-10-06T02:30:26.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.818+0000 7f2e51ffb700 1 -- 172.21.15.31:0/985157577 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e580070d0 con 0x7f2e5c157380 2023-10-06T02:30:26.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.818+0000 7f2e62d9d700 1 -- 172.21.15.31:0/985157577 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e5c0c1860 con 0x7f2e5c157380 2023-10-06T02:30:26.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.819+0000 7f2e62d9d700 1 -- 172.21.15.31:0/985157577 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e5c14f610 con 0x7f2e5c157380 2023-10-06T02:30:26.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.819+0000 7f2e62d9d700 1 -- 172.21.15.31:0/985157577 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e5c04c3c0 con 0x7f2e5c157380 2023-10-06T02:30:26.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.820+0000 7f2e51ffb700 1 -- 172.21.15.31:0/985157577 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f2e580074c0 con 0x7f2e5c157380 2023-10-06T02:30:26.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.821+0000 7f2e51ffb700 1 --2- 172.21.15.31:0/985157577 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f2e400674d0 0x7f2e40069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:26.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.821+0000 7f2e51ffb700 1 -- 172.21.15.31:0/985157577 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f2e58084520 con 0x7f2e5c157380 2023-10-06T02:30:26.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.822+0000 7f2e6159a700 1 --2- 172.21.15.31:0/985157577 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f2e400674d0 0x7f2e40069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:26.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.822+0000 7f2e6159a700 1 --2- 172.21.15.31:0/985157577 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f2e400674d0 0x7f2e40069980 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c0099b0 tx=0x7f2e4c017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:26.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:26.825+0000 7f2e51ffb700 1 -- 172.21.15.31:0/985157577 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2e58050da0 con 0x7f2e5c157380 2023-10-06T02:30:27.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.067+0000 7f2e62d9d700 1 -- 172.21.15.31:0/985157577 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f2e5c0c1ff0 con 0x7f2e400674d0 2023-10-06T02:30:27.070 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:30:27.070+0000 7f656537d700 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2023-10-06T02:30:27.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.070+0000 7f2e51ffb700 1 -- 172.21.15.31:0/985157577 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) v1 ==== 45+0+0 (secure 0 0 0) 0x7f2e4c020040 con 0x7f2e400674d0 2023-10-06T02:30:27.073 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2023-10-06T02:30:27.080 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-06T02:30:27.081 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot info non_existent_subvolume subvolume_1 snapshot_1 2023-10-06T02:30:27.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.274+0000 7f2ca0b9d700 1 Processor -- start 2023-10-06T02:30:27.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.274+0000 7f2ca0b9d700 1 -- start start 2023-10-06T02:30:27.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.276+0000 7f2ca0b9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2c9c15cb90 0x7f2c9c0c7870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:27.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.276+0000 7f2ca0b9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2c9c06cdb0 0x7f2c9c0c3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:27.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.276+0000 7f2ca0b9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c9c0c2fa0 con 0x7f2c9c15dc10 2023-10-06T02:30:27.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.276+0000 7f2ca0b9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c9c0ccdf0 con 0x7f2c9c06cdb0 2023-10-06T02:30:27.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.276+0000 7f2ca0b9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9c0c9b20 con 0x7f2c9c15cb90 2023-10-06T02:30:27.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.276+0000 7f2c9affd700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f2c9c15dc10 0x7f2c9c06c6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:48518/0 (socket says 172.21.15.31:48518) 2023-10-06T02:30:27.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.276+0000 7f2c9affd700 1 -- 172.21.15.31:0/3680370116 learned_addr learned my addr 172.21.15.31:0/3680370116 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:30:27.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.276+0000 7f2c9b7fe700 1 --2- 172.21.15.31:0/3680370116 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2c9c15cb90 0x7f2c9c0c7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:27.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.276+0000 7f2c9bfff700 1 --2- 172.21.15.31:0/3680370116 >> v2:172.21.15.62:3300/0 conn(0x7f2c9c06cdb0 0x7f2c9c0c3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:27.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.277+0000 7f2c9a7fc700 1 -- 172.21.15.31:0/3680370116 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3673375197 0 0) 0x7f2c9c0c2fa0 con 0x7f2c9c15dc10 2023-10-06T02:30:27.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.277+0000 7f2c9a7fc700 1 -- 172.21.15.31:0/3680370116 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c84003570 con 0x7f2c9c15dc10 2023-10-06T02:30:27.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.277+0000 7f2c9a7fc700 1 -- 172.21.15.31:0/3680370116 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 108502122 0 0) 0x7f2c84003570 con 0x7f2c9c15dc10 2023-10-06T02:30:27.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.277+0000 7f2c9a7fc700 1 -- 172.21.15.31:0/3680370116 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2c9c15cb90 msgr2=0x7f2c9c0c7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:27.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.277+0000 7f2c9a7fc700 1 --2- 172.21.15.31:0/3680370116 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2c9c15cb90 0x7f2c9c0c7870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:27.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.277+0000 7f2c9a7fc700 1 -- 172.21.15.31:0/3680370116 >> v2:172.21.15.62:3300/0 conn(0x7f2c9c06cdb0 msgr2=0x7f2c9c0c3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:27.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.277+0000 7f2c9a7fc700 1 --2- 172.21.15.31:0/3680370116 >> v2:172.21.15.62:3300/0 conn(0x7f2c9c06cdb0 0x7f2c9c0c3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:27.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.277+0000 7f2c9a7fc700 1 -- 172.21.15.31:0/3680370116 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c9c0c2390 con 0x7f2c9c15dc10 2023-10-06T02:30:27.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.277+0000 7f2c9a7fc700 1 -- 172.21.15.31:0/3680370116 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7f2c90001470 con 0x7f2c9c15dc10 2023-10-06T02:30:27.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.277+0000 7f2c9a7fc700 1 -- 172.21.15.31:0/3680370116 >> v1:172.21.15.31:6789/0 conn(0x7f2c9c15dc10 legacy=0x7f2c9c06c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:27.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.277+0000 7f2c9a7fc700 1 --2- 172.21.15.31:0/3680370116 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c84003ae0 0x7f2c84003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:27.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.277+0000 7f2c9a7fc700 1 --2- 172.21.15.31:0/3680370116 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2c9c15cb90 0x7f2c84004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:27.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.277+0000 7f2c9a7fc700 1 --2- 172.21.15.31:0/3680370116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c84004d50 0x7f2c840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:27.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.277+0000 7f2c9a7fc700 1 -- 172.21.15.31:0/3680370116 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9c0ccdf0 con 0x7f2c84003ae0 2023-10-06T02:30:27.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.277+0000 7f2c9a7fc700 1 -- 172.21.15.31:0/3680370116 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9c0c9b20 con 0x7f2c84004d50 2023-10-06T02:30:27.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.277+0000 7f2c9a7fc700 1 -- 172.21.15.31:0/3680370116 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c8400bb10 con 0x7f2c9c15cb90 2023-10-06T02:30:27.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.277+0000 7f2c9b7fe700 1 --2- 172.21.15.31:0/3680370116 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c84003ae0 0x7f2c84003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:27.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.277+0000 7f2c9affd700 1 --2- 172.21.15.31:0/3680370116 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2c9c15cb90 0x7f2c84004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:27.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.277+0000 7f2c9bfff700 1 --2- 172.21.15.31:0/3680370116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c84004d50 0x7f2c840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:27.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.278+0000 7f2c9b7fe700 1 -- 172.21.15.31:0/3680370116 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2c9c15cb90 msgr2=0x7f2c84004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:27.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.278+0000 7f2c9b7fe700 1 --2- 172.21.15.31:0/3680370116 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2c9c15cb90 0x7f2c84004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:27.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.278+0000 7f2c9b7fe700 1 -- 172.21.15.31:0/3680370116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c84004d50 msgr2=0x7f2c840051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:27.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.278+0000 7f2c9b7fe700 1 --2- 172.21.15.31:0/3680370116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c84004d50 0x7f2c840051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:27.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.278+0000 7f2c9b7fe700 1 -- 172.21.15.31:0/3680370116 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c84003570 con 0x7f2c84003ae0 2023-10-06T02:30:27.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.278+0000 7f2c9bfff700 1 --2- 172.21.15.31:0/3680370116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c84004d50 0x7f2c840051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:30:27.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.278+0000 7f2c9b7fe700 1 --2- 172.21.15.31:0/3680370116 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c84003ae0 0x7f2c84003ef0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c009c30 tx=0x7f2c8c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d66f77f1869cacc1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:27.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.278+0000 7f2c9a7fc700 1 -- 172.21.15.31:0/3680370116 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c8c00e710 con 0x7f2c84003ae0 2023-10-06T02:30:27.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.278+0000 7f2c9a7fc700 1 -- 172.21.15.31:0/3680370116 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c8c00e870 con 0x7f2c84003ae0 2023-10-06T02:30:27.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.278+0000 7f2c9a7fc700 1 -- 172.21.15.31:0/3680370116 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c8c00ec30 con 0x7f2c84003ae0 2023-10-06T02:30:27.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.279+0000 7f2ca0b9d700 1 -- 172.21.15.31:0/3680370116 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c84003ae0 msgr2=0x7f2c84003ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:27.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.279+0000 7f2ca0b9d700 1 --2- 172.21.15.31:0/3680370116 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c84003ae0 0x7f2c84003ef0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c009c30 tx=0x7f2c8c0091a0 comp rx=0 tx=0).stop 2023-10-06T02:30:27.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.279+0000 7f2c9bfff700 1 -- 172.21.15.31:0/3680370116 reap_dead start 2023-10-06T02:30:27.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.279+0000 7f2ca0b9d700 1 -- 172.21.15.31:0/3680370116 shutdown_connections 2023-10-06T02:30:27.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.279+0000 7f2ca0b9d700 1 -- 172.21.15.31:0/3680370116 >> 172.21.15.31:0/3680370116 conn(0x7f2c9c0cd230 msgr2=0x7f2c9c0cd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:27.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.279+0000 7f2ca0b9d700 1 -- 172.21.15.31:0/3680370116 shutdown_connections 2023-10-06T02:30:27.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.279+0000 7f2ca0b9d700 1 -- 172.21.15.31:0/3680370116 wait complete. 2023-10-06T02:30:27.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.280+0000 7f2ca0b9d700 1 Processor -- start 2023-10-06T02:30:27.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.280+0000 7f2ca0b9d700 1 -- start start 2023-10-06T02:30:27.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.281+0000 7f2ca0b9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c84003ae0 0x7f2c9c0c71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:27.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.281+0000 7f2ca0b9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c9c0c9eb0 0x7f2c9c16c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:27.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.281+0000 7f2ca0b9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2c9c0ca320 0x7f2c9c16c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:27.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.281+0000 7f2ca0b9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9c0b19e0 con 0x7f2c9c0c9eb0 2023-10-06T02:30:27.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.281+0000 7f2ca0b9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9c0b1cc0 con 0x7f2c84003ae0 2023-10-06T02:30:27.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.281+0000 7f2ca0b9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9c0b1fa0 con 0x7f2c9c0ca320 2023-10-06T02:30:27.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.281+0000 7f2c9affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c9c0c9eb0 0x7f2c9c16c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:27.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.282+0000 7f2c9b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c84003ae0 0x7f2c9c0c71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:27.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.282+0000 7f2c9affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c9c0c9eb0 0x7f2c9c16c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35710/0 (socket says 172.21.15.31:35710) 2023-10-06T02:30:27.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.282+0000 7f2c9affd700 1 -- 172.21.15.31:0/1639646462 learned_addr learned my addr 172.21.15.31:0/1639646462 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:27.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.282+0000 7f2c9bfff700 1 --2- 172.21.15.31:0/1639646462 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2c9c0ca320 0x7f2c9c16c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:27.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.282+0000 7f2c9affd700 1 -- 172.21.15.31:0/1639646462 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2c9c0ca320 msgr2=0x7f2c9c16c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:27.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.282+0000 7f2c9affd700 1 --2- 172.21.15.31:0/1639646462 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2c9c0ca320 0x7f2c9c16c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:27.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.282+0000 7f2c9affd700 1 -- 172.21.15.31:0/1639646462 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c84003ae0 msgr2=0x7f2c9c0c71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:27.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.282+0000 7f2c9affd700 1 --2- 172.21.15.31:0/1639646462 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c84003ae0 0x7f2c9c0c71a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:27.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.282+0000 7f2c9affd700 1 -- 172.21.15.31:0/1639646462 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c9c0b2280 con 0x7f2c9c0c9eb0 2023-10-06T02:30:27.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.282+0000 7f2c9affd700 1 --2- 172.21.15.31:0/1639646462 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c9c0c9eb0 0x7f2c9c16c3b0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f2c9000eb90 tx=0x7f2c9c0c2f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:27.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.282+0000 7f2c837fe700 1 -- 172.21.15.31:0/1639646462 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c9000c550 con 0x7f2c9c0c9eb0 2023-10-06T02:30:27.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.282+0000 7f2c837fe700 1 -- 172.21.15.31:0/1639646462 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c9000c6b0 con 0x7f2c9c0c9eb0 2023-10-06T02:30:27.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.282+0000 7f2c837fe700 1 -- 172.21.15.31:0/1639646462 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c9000ca70 con 0x7f2c9c0c9eb0 2023-10-06T02:30:27.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.282+0000 7f2ca0b9d700 1 -- 172.21.15.31:0/1639646462 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c9c0b2560 con 0x7f2c9c0c9eb0 2023-10-06T02:30:27.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.283+0000 7f2ca0b9d700 1 -- 172.21.15.31:0/1639646462 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c9c0affd0 con 0x7f2c9c0c9eb0 2023-10-06T02:30:27.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.284+0000 7f2ca0b9d700 1 -- 172.21.15.31:0/1639646462 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c9c04c3c0 con 0x7f2c9c0c9eb0 2023-10-06T02:30:27.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.285+0000 7f2c837fe700 1 -- 172.21.15.31:0/1639646462 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f2c9001d460 con 0x7f2c9c0c9eb0 2023-10-06T02:30:27.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.285+0000 7f2c837fe700 1 --2- 172.21.15.31:0/1639646462 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f2c880674d0 0x7f2c88069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:27.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.286+0000 7f2c837fe700 1 -- 172.21.15.31:0/1639646462 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f2c90084f90 con 0x7f2c9c0c9eb0 2023-10-06T02:30:27.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.286+0000 7f2c9b7fe700 1 --2- 172.21.15.31:0/1639646462 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f2c880674d0 0x7f2c88069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:27.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.287+0000 7f2c9b7fe700 1 --2- 172.21.15.31:0/1639646462 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f2c880674d0 0x7f2c88069980 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c0099c0 tx=0x7f2c8c01a000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:27.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.290+0000 7f2c837fe700 1 -- 172.21.15.31:0/1639646462 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2c90051810 con 0x7f2c9c0c9eb0 2023-10-06T02:30:27.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.530+0000 7f2ca0b9d700 1 -- 172.21.15.31:0/1639646462 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot info", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f2c9c0c2670 con 0x7f2c880674d0 2023-10-06T02:30:27.533 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:30:27.532+0000 7f656537d700 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2023-10-06T02:30:27.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.532+0000 7f2c837fe700 1 -- 172.21.15.31:0/1639646462 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) v1 ==== 45+0+0 (secure 0 0 0) 0x7f2c9c0c2670 con 0x7f2c880674d0 2023-10-06T02:30:27.535 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2023-10-06T02:30:27.543 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-06T02:30:27.543 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot protect non_existent_subvolume subvolume_1 snapshot_1 2023-10-06T02:30:27.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.733+0000 7f0a53fff700 1 Processor -- start 2023-10-06T02:30:27.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.733+0000 7f0a53fff700 1 -- start start 2023-10-06T02:30:27.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.735+0000 7f0a53fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0a54142ca0 0x7f0a5415dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:27.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.735+0000 7f0a53fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0a54143070 0x7f0a540cbf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:27.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.735+0000 7f0a53fff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a5415d8d0 con 0x7f0a5415a6f0 2023-10-06T02:30:27.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.735+0000 7f0a53fff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a5415d2f0 con 0x7f0a54143070 2023-10-06T02:30:27.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.735+0000 7f0a53fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5415ab90 con 0x7f0a54142ca0 2023-10-06T02:30:27.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.735+0000 7f0a527fc700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0a54143070 0x7f0a540cbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:27.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.735+0000 7f0a527fc700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0a54143070 0x7f0a540cbf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35718/0 (socket says 172.21.15.31:35718) 2023-10-06T02:30:27.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.735+0000 7f0a527fc700 1 -- 172.21.15.31:0/1271902175 learned_addr learned my addr 172.21.15.31:0/1271902175 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:27.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.735+0000 7f0a52ffd700 1 --2- 172.21.15.31:0/1271902175 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0a54142ca0 0x7f0a5415dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:27.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.736+0000 7f0a527fc700 1 -- 172.21.15.31:0/1271902175 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0a54142ca0 msgr2=0x7f0a5415dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:27.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.736+0000 7f0a527fc700 1 --2- 172.21.15.31:0/1271902175 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0a54142ca0 0x7f0a5415dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:27.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.736+0000 7f0a527fc700 1 -- 172.21.15.31:0/1271902175 >> v1:172.21.15.62:6789/0 conn(0x7f0a5415a6f0 legacy=0x7f0a540cad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:27.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.736+0000 7f0a527fc700 1 -- 172.21.15.31:0/1271902175 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a540cc630 con 0x7f0a54143070 2023-10-06T02:30:27.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.736+0000 7f0a527fc700 1 --2- 172.21.15.31:0/1271902175 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a54143070 0x7f0a540cbf30 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f0a480099b0 tx=0x7f0a48009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=333187e9a4de7dce server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:27.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.736+0000 7f0a51ffb700 1 -- 172.21.15.31:0/1271902175 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a4800b860 con 0x7f0a54143070 2023-10-06T02:30:27.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.736+0000 7f0a51ffb700 1 -- 172.21.15.31:0/1271902175 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a4800b9c0 con 0x7f0a54143070 2023-10-06T02:30:27.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.736+0000 7f0a51ffb700 1 -- 172.21.15.31:0/1271902175 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a4800bd60 con 0x7f0a54143070 2023-10-06T02:30:27.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.737+0000 7f0a53fff700 1 -- 172.21.15.31:0/1271902175 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a54143070 msgr2=0x7f0a540cbf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:27.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.737+0000 7f0a53fff700 1 --2- 172.21.15.31:0/1271902175 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a54143070 0x7f0a540cbf30 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f0a480099b0 tx=0x7f0a48009250 comp rx=0 tx=0).stop 2023-10-06T02:30:27.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.737+0000 7f0a53fff700 1 -- 172.21.15.31:0/1271902175 shutdown_connections 2023-10-06T02:30:27.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.737+0000 7f0a53fff700 1 --2- 172.21.15.31:0/1271902175 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0a54142ca0 0x7f0a5415dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:27.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.737+0000 7f0a53fff700 1 --2- 172.21.15.31:0/1271902175 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a54143070 0x7f0a540cbf30 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:27.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.737+0000 7f0a53fff700 1 -- 172.21.15.31:0/1271902175 >> 172.21.15.31:0/1271902175 conn(0x7f0a54067c80 msgr2=0x7f0a54068080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:27.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.737+0000 7f0a53fff700 1 -- 172.21.15.31:0/1271902175 shutdown_connections 2023-10-06T02:30:27.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.737+0000 7f0a53fff700 1 -- 172.21.15.31:0/1271902175 wait complete. 2023-10-06T02:30:27.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.738+0000 7f0a53fff700 1 Processor -- start 2023-10-06T02:30:27.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.739+0000 7f0a53fff700 1 -- start start 2023-10-06T02:30:27.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.739+0000 7f0a53fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a54142ca0 0x7f0a5413fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:27.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.740+0000 7f0a53fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0a54143070 0x7f0a5406e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:27.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.740+0000 7f0a53fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a5415a6f0 0x7f0a5406e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:27.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.740+0000 7f0a53fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a54169f60 con 0x7f0a5415a6f0 2023-10-06T02:30:27.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.740+0000 7f0a53fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a540b2ca0 con 0x7f0a54142ca0 2023-10-06T02:30:27.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.740+0000 7f0a53fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5416aa70 con 0x7f0a54143070 2023-10-06T02:30:27.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.740+0000 7f0a537fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a5415a6f0 0x7f0a5406e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:27.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.740+0000 7f0a537fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a5415a6f0 0x7f0a5406e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35720/0 (socket says 172.21.15.31:35720) 2023-10-06T02:30:27.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.740+0000 7f0a537fe700 1 -- 172.21.15.31:0/1515279569 learned_addr learned my addr 172.21.15.31:0/1515279569 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:27.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.740+0000 7f0a527fc700 1 --2- 172.21.15.31:0/1515279569 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0a54143070 0x7f0a5406e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:27.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.740+0000 7f0a52ffd700 1 --2- 172.21.15.31:0/1515279569 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a54142ca0 0x7f0a5413fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:27.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.740+0000 7f0a537fe700 1 -- 172.21.15.31:0/1515279569 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0a54143070 msgr2=0x7f0a5406e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:27.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.740+0000 7f0a537fe700 1 --2- 172.21.15.31:0/1515279569 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0a54143070 0x7f0a5406e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:27.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.740+0000 7f0a537fe700 1 -- 172.21.15.31:0/1515279569 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a54142ca0 msgr2=0x7f0a5413fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:27.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.740+0000 7f0a537fe700 1 --2- 172.21.15.31:0/1515279569 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a54142ca0 0x7f0a5413fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:27.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.740+0000 7f0a537fe700 1 -- 172.21.15.31:0/1515279569 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a480094f0 con 0x7f0a5415a6f0 2023-10-06T02:30:27.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.741+0000 7f0a537fe700 1 --2- 172.21.15.31:0/1515279569 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a5415a6f0 0x7f0a5406e7d0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c00c870 tx=0x7f0a4c001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:27.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.741+0000 7f0a3affd700 1 -- 172.21.15.31:0/1515279569 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a4c012630 con 0x7f0a5415a6f0 2023-10-06T02:30:27.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.741+0000 7f0a3affd700 1 -- 172.21.15.31:0/1515279569 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a4c0127b0 con 0x7f0a5415a6f0 2023-10-06T02:30:27.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.741+0000 7f0a3affd700 1 -- 172.21.15.31:0/1515279569 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a4c012b70 con 0x7f0a5415a6f0 2023-10-06T02:30:27.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.741+0000 7f0a53fff700 1 -- 172.21.15.31:0/1515279569 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a5416a140 con 0x7f0a5415a6f0 2023-10-06T02:30:27.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.741+0000 7f0a53fff700 1 -- 172.21.15.31:0/1515279569 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a5416a700 con 0x7f0a5415a6f0 2023-10-06T02:30:27.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.742+0000 7f0a53fff700 1 -- 172.21.15.31:0/1515279569 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a5404c3c0 con 0x7f0a5415a6f0 2023-10-06T02:30:27.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.743+0000 7f0a3affd700 1 -- 172.21.15.31:0/1515279569 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f0a4c003030 con 0x7f0a5415a6f0 2023-10-06T02:30:27.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.744+0000 7f0a3affd700 1 --2- 172.21.15.31:0/1515279569 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0a400674d0 0x7f0a40069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:27.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.744+0000 7f0a3affd700 1 -- 172.21.15.31:0/1515279569 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f0a4c084fb0 con 0x7f0a5415a6f0 2023-10-06T02:30:27.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.745+0000 7f0a52ffd700 1 --2- 172.21.15.31:0/1515279569 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0a400674d0 0x7f0a40069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:27.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.745+0000 7f0a52ffd700 1 --2- 172.21.15.31:0/1515279569 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0a400674d0 0x7f0a40069980 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0a44006fd0 tx=0x7f0a44008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:27.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.748+0000 7f0a3affd700 1 -- 172.21.15.31:0/1515279569 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0a4c0517f0 con 0x7f0a5415a6f0 2023-10-06T02:30:27.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.991+0000 7f0a53fff700 1 -- 172.21.15.31:0/1515279569 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot protect", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f0a5416abb0 con 0x7f0a400674d0 2023-10-06T02:30:27.994 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:30:27.993+0000 7f656537d700 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2023-10-06T02:30:27.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:27.993+0000 7f0a3affd700 1 -- 172.21.15.31:0/1515279569 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) v1 ==== 45+0+0 (secure 0 0 0) 0x7f0a5416abb0 con 0x7f0a400674d0 2023-10-06T02:30:27.996 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2023-10-06T02:30:28.003 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-06T02:30:28.004 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot unprotect non_existent_subvolume subvolume_1 snapshot_1 2023-10-06T02:30:28.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.198+0000 7fab7759d700 1 Processor -- start 2023-10-06T02:30:28.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.198+0000 7fab7759d700 1 -- start start 2023-10-06T02:30:28.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.199+0000 7fab7759d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fab70159770 0x7fab7006dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:28.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.199+0000 7fab7759d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fab7006c6a0 0x7fab700ca5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:28.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.200+0000 7fab7759d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab7006d740 con 0x7fab7006a330 2023-10-06T02:30:28.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.200+0000 7fab7759d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab70159b40 con 0x7fab7006c6a0 2023-10-06T02:30:28.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.200+0000 7fab7759d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab7006cb30 con 0x7fab70159770 2023-10-06T02:30:28.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.200+0000 7fab75d9a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fab7006c6a0 0x7fab700ca5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:28.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.200+0000 7fab75d9a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fab7006c6a0 0x7fab700ca5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46884/0 (socket says 172.21.15.31:46884) 2023-10-06T02:30:28.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.200+0000 7fab75d9a700 1 -- 172.21.15.31:0/4218944132 learned_addr learned my addr 172.21.15.31:0/4218944132 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:28.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.200+0000 7fab7659b700 1 --2- 172.21.15.31:0/4218944132 >> v2:172.21.15.62:3300/0 conn(0x7fab70159770 0x7fab7006dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:28.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.200+0000 7fab75d9a700 1 -- 172.21.15.31:0/4218944132 >> v2:172.21.15.62:3300/0 conn(0x7fab70159770 msgr2=0x7fab7006dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:28.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.200+0000 7fab75d9a700 1 --2- 172.21.15.31:0/4218944132 >> v2:172.21.15.62:3300/0 conn(0x7fab70159770 0x7fab7006dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:28.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.200+0000 7fab75d9a700 1 -- 172.21.15.31:0/4218944132 >> v1:172.21.15.62:6789/0 conn(0x7fab7006a330 legacy=0x7fab700c7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:28.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.200+0000 7fab75d9a700 1 -- 172.21.15.31:0/4218944132 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab7006ce10 con 0x7fab7006c6a0 2023-10-06T02:30:28.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.200+0000 7fab7659b700 1 --2- 172.21.15.31:0/4218944132 >> v2:172.21.15.62:3300/0 conn(0x7fab70159770 0x7fab7006dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:30:28.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.201+0000 7fab75d9a700 1 --2- 172.21.15.31:0/4218944132 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab7006c6a0 0x7fab700ca5a0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7fab7006f640 tx=0x7fab60009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=194186bd54e446d6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:28.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.201+0000 7fab75599700 1 -- 172.21.15.31:0/4218944132 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab6000b820 con 0x7fab7006c6a0 2023-10-06T02:30:28.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.201+0000 7fab75599700 1 -- 172.21.15.31:0/4218944132 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab6000b980 con 0x7fab7006c6a0 2023-10-06T02:30:28.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.201+0000 7fab75599700 1 -- 172.21.15.31:0/4218944132 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab6000bd20 con 0x7fab7006c6a0 2023-10-06T02:30:28.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.201+0000 7fab7759d700 1 -- 172.21.15.31:0/4218944132 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab7006c6a0 msgr2=0x7fab700ca5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:28.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.201+0000 7fab7759d700 1 --2- 172.21.15.31:0/4218944132 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab7006c6a0 0x7fab700ca5a0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7fab7006f640 tx=0x7fab60009250 comp rx=0 tx=0).stop 2023-10-06T02:30:28.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.202+0000 7fab7759d700 1 -- 172.21.15.31:0/4218944132 shutdown_connections 2023-10-06T02:30:28.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.202+0000 7fab7759d700 1 --2- 172.21.15.31:0/4218944132 >> v2:172.21.15.62:3300/0 conn(0x7fab70159770 0x7fab7006dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:28.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.202+0000 7fab7759d700 1 --2- 172.21.15.31:0/4218944132 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab7006c6a0 0x7fab700ca5a0 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:28.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.202+0000 7fab7759d700 1 -- 172.21.15.31:0/4218944132 >> 172.21.15.31:0/4218944132 conn(0x7fab7006e790 msgr2=0x7fab7006eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:28.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.202+0000 7fab7759d700 1 -- 172.21.15.31:0/4218944132 shutdown_connections 2023-10-06T02:30:28.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.202+0000 7fab7759d700 1 -- 172.21.15.31:0/4218944132 wait complete. 2023-10-06T02:30:28.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.203+0000 7fab7759d700 1 Processor -- start 2023-10-06T02:30:28.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.203+0000 7fab7759d700 1 -- start start 2023-10-06T02:30:28.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.204+0000 7fab7759d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab7006a330 0x7fab70151220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:28.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.204+0000 7fab7759d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab7006c6a0 0x7fab700c5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:28.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.204+0000 7fab7759d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab70159770 0x7fab7015d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:28.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.204+0000 7fab7759d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab7015bba0 con 0x7fab70159770 2023-10-06T02:30:28.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.204+0000 7fab7759d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab70151760 con 0x7fab7006c6a0 2023-10-06T02:30:28.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.204+0000 7fab7759d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab7015af90 con 0x7fab7006a330 2023-10-06T02:30:28.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.204+0000 7fab76d9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab70159770 0x7fab7015d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:28.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.204+0000 7fab76d9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab70159770 0x7fab7015d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46886/0 (socket says 172.21.15.31:46886) 2023-10-06T02:30:28.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.204+0000 7fab76d9c700 1 -- 172.21.15.31:0/1918706937 learned_addr learned my addr 172.21.15.31:0/1918706937 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:28.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.205+0000 7fab75d9a700 1 --2- 172.21.15.31:0/1918706937 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab7006c6a0 0x7fab700c5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:28.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.205+0000 7fab7659b700 1 --2- 172.21.15.31:0/1918706937 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab7006a330 0x7fab70151220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:28.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.205+0000 7fab76d9c700 1 -- 172.21.15.31:0/1918706937 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab7006a330 msgr2=0x7fab70151220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:28.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.205+0000 7fab76d9c700 1 --2- 172.21.15.31:0/1918706937 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab7006a330 0x7fab70151220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:28.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.205+0000 7fab76d9c700 1 -- 172.21.15.31:0/1918706937 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab7006c6a0 msgr2=0x7fab700c5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:28.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.205+0000 7fab76d9c700 1 --2- 172.21.15.31:0/1918706937 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab7006c6a0 0x7fab700c5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:28.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.205+0000 7fab76d9c700 1 -- 172.21.15.31:0/1918706937 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab600094f0 con 0x7fab70159770 2023-10-06T02:30:28.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.205+0000 7fab76d9c700 1 --2- 172.21.15.31:0/1918706937 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab70159770 0x7fab7015d5e0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fab6800c960 tx=0x7fab680026e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:28.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.205+0000 7fab667fc700 1 -- 172.21.15.31:0/1918706937 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab68012630 con 0x7fab70159770 2023-10-06T02:30:28.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.205+0000 7fab667fc700 1 -- 172.21.15.31:0/1918706937 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab68012790 con 0x7fab70159770 2023-10-06T02:30:28.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.205+0000 7fab7759d700 1 -- 172.21.15.31:0/1918706937 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab7015b270 con 0x7fab70159770 2023-10-06T02:30:28.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.206+0000 7fab667fc700 1 -- 172.21.15.31:0/1918706937 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab68012b50 con 0x7fab70159770 2023-10-06T02:30:28.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.206+0000 7fab7759d700 1 -- 172.21.15.31:0/1918706937 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab7015b830 con 0x7fab70159770 2023-10-06T02:30:28.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.207+0000 7fab7759d700 1 -- 172.21.15.31:0/1918706937 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab7015bd50 con 0x7fab70159770 2023-10-06T02:30:28.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.208+0000 7fab667fc700 1 -- 172.21.15.31:0/1918706937 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7fab6801b400 con 0x7fab70159770 2023-10-06T02:30:28.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.208+0000 7fab667fc700 1 --2- 172.21.15.31:0/1918706937 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fab540674d0 0x7fab54069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:28.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.208+0000 7fab667fc700 1 -- 172.21.15.31:0/1918706937 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fab680853d0 con 0x7fab70159770 2023-10-06T02:30:28.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.209+0000 7fab7659b700 1 --2- 172.21.15.31:0/1918706937 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fab540674d0 0x7fab54069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:28.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.209+0000 7fab7659b700 1 --2- 172.21.15.31:0/1918706937 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fab540674d0 0x7fab54069980 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fab6c006fd0 tx=0x7fab6c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:28.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.212+0000 7fab667fc700 1 -- 172.21.15.31:0/1918706937 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fab7015bd50 con 0x7fab70159770 2023-10-06T02:30:28.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.458+0000 7fab7759d700 1 -- 172.21.15.31:0/1918706937 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot unprotect", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fab7015bd50 con 0x7fab540674d0 2023-10-06T02:30:28.461 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:30:28.460+0000 7f656537d700 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2023-10-06T02:30:28.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.460+0000 7fab667fc700 1 -- 172.21.15.31:0/1918706937 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) v1 ==== 45+0+0 (secure 0 0 0) 0x7fab7015bd50 con 0x7fab540674d0 2023-10-06T02:30:28.463 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2023-10-06T02:30:28.471 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-06T02:30:28.471 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls non_existent_subvolume subvolume_1 2023-10-06T02:30:28.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.660+0000 7f562219d700 1 Processor -- start 2023-10-06T02:30:28.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.660+0000 7f562219d700 1 -- start start 2023-10-06T02:30:28.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.662+0000 7f562219d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f561c140f40 0x7f561c0cb8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:28.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.662+0000 7f562219d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f561c0cbfb0 con 0x7f561c06cdb0 2023-10-06T02:30:28.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.662+0000 7f562219d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f561c0cc290 con 0x7f561c140b70 2023-10-06T02:30:28.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.662+0000 7f562219d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f561c0b0220 con 0x7f561c140f40 2023-10-06T02:30:28.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.662+0000 7f562099a700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f561c06cdb0 0x7f561c0cb1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49618/0 (socket says 172.21.15.31:49618) 2023-10-06T02:30:28.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.662+0000 7f562099a700 1 -- 172.21.15.31:0/4166719220 learned_addr learned my addr 172.21.15.31:0/4166719220 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:30:28.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.662+0000 7f562199c700 1 --2- 172.21.15.31:0/4166719220 >> v2:172.21.15.31:3300/0 conn(0x7f561c140f40 0x7f561c0cb8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:28.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.663+0000 7f562199c700 1 -- 172.21.15.31:0/4166719220 >> v1:172.21.15.62:6789/0 conn(0x7f561c140b70 legacy=0x7f561c06c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:28.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.663+0000 7f562199c700 1 -- 172.21.15.31:0/4166719220 >> v1:172.21.15.31:6789/0 conn(0x7f561c06cdb0 legacy=0x7f561c0cb1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:28.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.662+0000 7f5613fff700 1 -- 172.21.15.31:0/4166719220 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1772694231 0 0) 0x7f561c0cbfb0 con 0x7f561c06cdb0 2023-10-06T02:30:28.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.663+0000 7f562199c700 1 -- 172.21.15.31:0/4166719220 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f561c0cc570 con 0x7f561c140f40 2023-10-06T02:30:28.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.663+0000 7f562199c700 1 --2- 172.21.15.31:0/4166719220 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f561c140f40 0x7f561c0cb8f0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f5614009e60 tx=0x7f5614009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68c5f4761552bfd3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:28.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.663+0000 7f5613fff700 1 -- 172.21.15.31:0/4166719220 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f561400e070 con 0x7f561c140f40 2023-10-06T02:30:28.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.663+0000 7f5613fff700 1 -- 172.21.15.31:0/4166719220 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f561400bc30 con 0x7f561c140f40 2023-10-06T02:30:28.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.663+0000 7f5613fff700 1 -- 172.21.15.31:0/4166719220 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5614012430 con 0x7f561c140f40 2023-10-06T02:30:28.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.663+0000 7f562219d700 1 -- 172.21.15.31:0/4166719220 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f561c140f40 msgr2=0x7f561c0cb8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:28.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.663+0000 7f562219d700 1 --2- 172.21.15.31:0/4166719220 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f561c140f40 0x7f561c0cb8f0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f5614009e60 tx=0x7f5614009780 comp rx=0 tx=0).stop 2023-10-06T02:30:28.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.664+0000 7f562219d700 1 -- 172.21.15.31:0/4166719220 shutdown_connections 2023-10-06T02:30:28.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.664+0000 7f562219d700 1 --2- 172.21.15.31:0/4166719220 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f561c140f40 0x7f561c0cb8f0 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:28.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.664+0000 7f562219d700 1 -- 172.21.15.31:0/4166719220 >> 172.21.15.31:0/4166719220 conn(0x7f561c0cd230 msgr2=0x7f561c0cd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:28.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.664+0000 7f562219d700 1 -- 172.21.15.31:0/4166719220 shutdown_connections 2023-10-06T02:30:28.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.664+0000 7f562219d700 1 -- 172.21.15.31:0/4166719220 wait complete. 2023-10-06T02:30:28.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.665+0000 7f562219d700 1 Processor -- start 2023-10-06T02:30:28.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.665+0000 7f562219d700 1 -- start start 2023-10-06T02:30:28.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.666+0000 7f562219d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f561c06cdb0 0x7f561c0c4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:28.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.666+0000 7f562219d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f561c140b70 0x7f561c0c6650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:28.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.666+0000 7f562219d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f561c140f40 0x7f561c0c6b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:28.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.666+0000 7f562219d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f561c0c5170 con 0x7f561c140f40 2023-10-06T02:30:28.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.666+0000 7f562219d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f561c0c70d0 con 0x7f561c06cdb0 2023-10-06T02:30:28.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.666+0000 7f562219d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f561c0c1fd0 con 0x7f561c140b70 2023-10-06T02:30:28.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.666+0000 7f562199c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f561c140f40 0x7f561c0c6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:28.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.666+0000 7f562199c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f561c140f40 0x7f561c0c6b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46902/0 (socket says 172.21.15.31:46902) 2023-10-06T02:30:28.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.666+0000 7f562199c700 1 -- 172.21.15.31:0/886221400 learned_addr learned my addr 172.21.15.31:0/886221400 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:28.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.667+0000 7f562119b700 1 --2- 172.21.15.31:0/886221400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f561c06cdb0 0x7f561c0c4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:28.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.667+0000 7f562099a700 1 --2- 172.21.15.31:0/886221400 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f561c140b70 0x7f561c0c6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:28.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.667+0000 7f562199c700 1 -- 172.21.15.31:0/886221400 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f561c140b70 msgr2=0x7f561c0c6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:28.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.667+0000 7f562199c700 1 --2- 172.21.15.31:0/886221400 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f561c140b70 0x7f561c0c6650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:28.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.667+0000 7f562199c700 1 -- 172.21.15.31:0/886221400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f561c06cdb0 msgr2=0x7f561c0c4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:28.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.667+0000 7f562199c700 1 --2- 172.21.15.31:0/886221400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f561c06cdb0 0x7f561c0c4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:28.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.667+0000 7f562199c700 1 -- 172.21.15.31:0/886221400 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5614009a20 con 0x7f561c140f40 2023-10-06T02:30:28.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.667+0000 7f562199c700 1 --2- 172.21.15.31:0/886221400 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f561c140f40 0x7f561c0c6b90 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f5614013000 tx=0x7f5614019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:28.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.667+0000 7f56117fa700 1 -- 172.21.15.31:0/886221400 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f561401b630 con 0x7f561c140f40 2023-10-06T02:30:28.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.667+0000 7f56117fa700 1 -- 172.21.15.31:0/886221400 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f561401b790 con 0x7f561c140f40 2023-10-06T02:30:28.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.667+0000 7f56117fa700 1 -- 172.21.15.31:0/886221400 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f561401bb10 con 0x7f561c140f40 2023-10-06T02:30:28.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.667+0000 7f562219d700 1 -- 172.21.15.31:0/886221400 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f561c0c75f0 con 0x7f561c140f40 2023-10-06T02:30:28.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.668+0000 7f562219d700 1 -- 172.21.15.31:0/886221400 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f561c0c7bb0 con 0x7f561c140f40 2023-10-06T02:30:28.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.668+0000 7f562219d700 1 -- 172.21.15.31:0/886221400 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f561c04c3c0 con 0x7f561c140f40 2023-10-06T02:30:28.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.669+0000 7f56117fa700 1 -- 172.21.15.31:0/886221400 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f561401c400 con 0x7f561c140f40 2023-10-06T02:30:28.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.670+0000 7f56117fa700 1 --2- 172.21.15.31:0/886221400 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f56000674d0 0x7f5600069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:28.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.670+0000 7f56117fa700 1 -- 172.21.15.31:0/886221400 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f5614087250 con 0x7f561c140f40 2023-10-06T02:30:28.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.671+0000 7f562119b700 1 --2- 172.21.15.31:0/886221400 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f56000674d0 0x7f5600069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:28.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.671+0000 7f562119b700 1 --2- 172.21.15.31:0/886221400 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f56000674d0 0x7f5600069980 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f5618003860 tx=0x7f5618009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:28.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.674+0000 7f56117fa700 1 -- 172.21.15.31:0/886221400 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5614053a30 con 0x7f561c140f40 2023-10-06T02:30:28.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.911+0000 7f562219d700 1 -- 172.21.15.31:0/886221400 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f561c0c73d0 con 0x7f56000674d0 2023-10-06T02:30:28.914 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:30:28.913+0000 7f656537d700 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2023-10-06T02:30:28.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:28.913+0000 7f56117fa700 1 -- 172.21.15.31:0/886221400 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) v1 ==== 45+0+0 (secure 0 0 0) 0x7f5618003490 con 0x7f56000674d0 2023-10-06T02:30:28.916 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2023-10-06T02:30:28.923 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-06T02:30:28.924 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone non_existent_subvolume subvolume_1 snapshot_1 clone_1 2023-10-06T02:30:29.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.120+0000 7f994f59e700 1 Processor -- start 2023-10-06T02:30:29.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.120+0000 7f994f59e700 1 -- start start 2023-10-06T02:30:29.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.121+0000 7f994f59e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f99500cb3f0 0x7f9950159750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:29.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.121+0000 7f994f59e700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f99500cea00 0x7f99500cee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:29.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.121+0000 7f994f59e700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99500ccfa0 con 0x7f9950159380 2023-10-06T02:30:29.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.121+0000 7f994f59e700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99500cf3c0 con 0x7f99500cb3f0 2023-10-06T02:30:29.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.121+0000 7f994f59e700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99500cb880 con 0x7f99500cea00 2023-10-06T02:30:29.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.121+0000 7f994e59c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f9950159380 0x7f995015dc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49634/0 (socket says 172.21.15.31:49634) 2023-10-06T02:30:29.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.121+0000 7f994e59c700 1 -- 172.21.15.31:0/2057754522 learned_addr learned my addr 172.21.15.31:0/2057754522 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:30:29.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.121+0000 7f994dd9b700 1 --2- 172.21.15.31:0/2057754522 >> v2:172.21.15.31:3300/0 conn(0x7f99500cb3f0 0x7f9950159750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:29.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.122+0000 7f994d59a700 1 -- 172.21.15.31:0/2057754522 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3585497016 0 0) 0x7f99500ccfa0 con 0x7f9950159380 2023-10-06T02:30:29.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.122+0000 7f994ed9d700 1 --2- 172.21.15.31:0/2057754522 >> v2:172.21.15.62:3300/0 conn(0x7f99500cea00 0x7f99500cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:29.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.122+0000 7f994d59a700 1 -- 172.21.15.31:0/2057754522 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9938003570 con 0x7f9950159380 2023-10-06T02:30:29.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.122+0000 7f994dd9b700 1 -- 172.21.15.31:0/2057754522 >> v2:172.21.15.62:3300/0 conn(0x7f99500cea00 msgr2=0x7f99500cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:29.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.122+0000 7f994dd9b700 1 --2- 172.21.15.31:0/2057754522 >> v2:172.21.15.62:3300/0 conn(0x7f99500cea00 0x7f99500cee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:29.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.122+0000 7f994dd9b700 1 -- 172.21.15.31:0/2057754522 >> v1:172.21.15.31:6789/0 conn(0x7f9950159380 legacy=0x7f995015dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:29.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.122+0000 7f994dd9b700 1 -- 172.21.15.31:0/2057754522 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99500cbb60 con 0x7f99500cb3f0 2023-10-06T02:30:29.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.122+0000 7f994ed9d700 1 --2- 172.21.15.31:0/2057754522 >> v2:172.21.15.62:3300/0 conn(0x7f99500cea00 0x7f99500cee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:30:29.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.122+0000 7f994dd9b700 1 --2- 172.21.15.31:0/2057754522 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99500cb3f0 0x7f9950159750 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f99500d0420 tx=0x7f99440091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2bcbb22be54cc3e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:29.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.122+0000 7f994d59a700 1 -- 172.21.15.31:0/2057754522 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9944011630 con 0x7f99500cb3f0 2023-10-06T02:30:29.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.122+0000 7f994d59a700 1 -- 172.21.15.31:0/2057754522 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9944011790 con 0x7f99500cb3f0 2023-10-06T02:30:29.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.122+0000 7f994d59a700 1 -- 172.21.15.31:0/2057754522 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9944011b30 con 0x7f99500cb3f0 2023-10-06T02:30:29.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.123+0000 7f994f59e700 1 -- 172.21.15.31:0/2057754522 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99500cb3f0 msgr2=0x7f9950159750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:29.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.123+0000 7f994f59e700 1 --2- 172.21.15.31:0/2057754522 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99500cb3f0 0x7f9950159750 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f99500d0420 tx=0x7f99440091a0 comp rx=0 tx=0).stop 2023-10-06T02:30:29.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.123+0000 7f994f59e700 1 -- 172.21.15.31:0/2057754522 shutdown_connections 2023-10-06T02:30:29.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.123+0000 7f994f59e700 1 --2- 172.21.15.31:0/2057754522 >> v2:172.21.15.62:3300/0 conn(0x7f99500cea00 0x7f99500cee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:29.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.123+0000 7f994f59e700 1 --2- 172.21.15.31:0/2057754522 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99500cb3f0 0x7f9950159750 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:29.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.123+0000 7f994f59e700 1 -- 172.21.15.31:0/2057754522 >> 172.21.15.31:0/2057754522 conn(0x7f99500703a0 msgr2=0x7f99500707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:29.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.123+0000 7f994f59e700 1 -- 172.21.15.31:0/2057754522 shutdown_connections 2023-10-06T02:30:29.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.123+0000 7f994f59e700 1 -- 172.21.15.31:0/2057754522 wait complete. 2023-10-06T02:30:29.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.125+0000 7f994f59e700 1 Processor -- start 2023-10-06T02:30:29.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.125+0000 7f994f59e700 1 -- start start 2023-10-06T02:30:29.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.126+0000 7f994f59e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99500cb3f0 0x7f995015c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:29.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.126+0000 7f994f59e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99500cea00 0x7f9950159cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:29.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.126+0000 7f994f59e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9950159380 0x7f9950169820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:29.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.126+0000 7f994f59e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f995015b0a0 con 0x7f99500cea00 2023-10-06T02:30:29.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.126+0000 7f994f59e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f995015a1f0 con 0x7f99500cb3f0 2023-10-06T02:30:29.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.126+0000 7f994f59e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f995015a490 con 0x7f9950159380 2023-10-06T02:30:29.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.126+0000 7f994dd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99500cea00 0x7f9950159cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:29.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.126+0000 7f994dd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99500cea00 0x7f9950159cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46914/0 (socket says 172.21.15.31:46914) 2023-10-06T02:30:29.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.126+0000 7f994dd9b700 1 -- 172.21.15.31:0/2920657403 learned_addr learned my addr 172.21.15.31:0/2920657403 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:29.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.126+0000 7f994e59c700 1 --2- 172.21.15.31:0/2920657403 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99500cb3f0 0x7f995015c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:29.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.126+0000 7f994ed9d700 1 --2- 172.21.15.31:0/2920657403 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9950159380 0x7f9950169820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:29.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.126+0000 7f994dd9b700 1 -- 172.21.15.31:0/2920657403 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9950159380 msgr2=0x7f9950169820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:29.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.126+0000 7f994dd9b700 1 --2- 172.21.15.31:0/2920657403 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9950159380 0x7f9950169820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:29.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.126+0000 7f994dd9b700 1 -- 172.21.15.31:0/2920657403 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99500cb3f0 msgr2=0x7f995015c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:29.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.127+0000 7f994dd9b700 1 --2- 172.21.15.31:0/2920657403 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99500cb3f0 0x7f995015c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:29.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.127+0000 7f994dd9b700 1 -- 172.21.15.31:0/2920657403 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9944009420 con 0x7f99500cea00 2023-10-06T02:30:29.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.127+0000 7f994dd9b700 1 --2- 172.21.15.31:0/2920657403 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99500cea00 0x7f9950159cb0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f9944013000 tx=0x7f9944017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:29.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.127+0000 7f993e7fc700 1 -- 172.21.15.31:0/2920657403 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9944000d10 con 0x7f99500cea00 2023-10-06T02:30:29.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.127+0000 7f993e7fc700 1 -- 172.21.15.31:0/2920657403 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9944011b10 con 0x7f99500cea00 2023-10-06T02:30:29.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.127+0000 7f993e7fc700 1 -- 172.21.15.31:0/2920657403 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9944025640 con 0x7f99500cea00 2023-10-06T02:30:29.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.127+0000 7f994f59e700 1 -- 172.21.15.31:0/2920657403 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f995015a770 con 0x7f99500cea00 2023-10-06T02:30:29.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.127+0000 7f994f59e700 1 -- 172.21.15.31:0/2920657403 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f995015ad30 con 0x7f99500cea00 2023-10-06T02:30:29.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.128+0000 7f994f59e700 1 -- 172.21.15.31:0/2920657403 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f995004c3c0 con 0x7f99500cea00 2023-10-06T02:30:29.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.129+0000 7f993e7fc700 1 -- 172.21.15.31:0/2920657403 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f99440259f0 con 0x7f99500cea00 2023-10-06T02:30:29.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.130+0000 7f993e7fc700 1 --2- 172.21.15.31:0/2920657403 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f99300674d0 0x7f9930069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:29.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.130+0000 7f993e7fc700 1 -- 172.21.15.31:0/2920657403 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f9944086880 con 0x7f99500cea00 2023-10-06T02:30:29.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.131+0000 7f994e59c700 1 --2- 172.21.15.31:0/2920657403 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f99300674d0 0x7f9930069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:29.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.131+0000 7f994e59c700 1 --2- 172.21.15.31:0/2920657403 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f99300674d0 0x7f9930069980 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9940007fd0 tx=0x7f9940009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:29.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.134+0000 7f993e7fc700 1 -- 172.21.15.31:0/2920657403 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f99440531a0 con 0x7f99500cea00 2023-10-06T02:30:29.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.378+0000 7f994f59e700 1 -- 172.21.15.31:0/2920657403 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target_sub_name": "clone_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f99500b4310 con 0x7f99300674d0 2023-10-06T02:30:29.381 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:30:29.380+0000 7f656537d700 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2023-10-06T02:30:29.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.380+0000 7f993e7fc700 1 -- 172.21.15.31:0/2920657403 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) v1 ==== 45+0+0 (secure 0 0 0) 0x7f99500b4310 con 0x7f99300674d0 2023-10-06T02:30:29.384 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2023-10-06T02:30:29.391 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-06T02:30:29.391 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status non_existent_subvolume clone_1 2023-10-06T02:30:29.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.580+0000 7f646dd9d700 1 Processor -- start 2023-10-06T02:30:29.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.580+0000 7f646dd9d700 1 -- start start 2023-10-06T02:30:29.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.582+0000 7f646dd9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f64680c7be0 0x7f646806abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:29.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.582+0000 7f646dd9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f64680c7fb0 0x7f646815a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:29.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.582+0000 7f646dd9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f646813f780 0x7f646813fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:29.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.582+0000 7f646dd9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f646806bf10 con 0x7f646813f780 2023-10-06T02:30:29.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.582+0000 7f646dd9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6468140100 con 0x7f64680c7fb0 2023-10-06T02:30:29.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.582+0000 7f646dd9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f646815aed0 con 0x7f64680c7be0 2023-10-06T02:30:29.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.582+0000 7f646d59c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f646813f780 0x7f646813fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:29.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.582+0000 7f646d59c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f646813f780 0x7f646813fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46928/0 (socket says 172.21.15.31:46928) 2023-10-06T02:30:29.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.582+0000 7f646d59c700 1 -- 172.21.15.31:0/3593310240 learned_addr learned my addr 172.21.15.31:0/3593310240 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:29.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.582+0000 7f6467fff700 1 --2- 172.21.15.31:0/3593310240 >> v2:172.21.15.62:3300/0 conn(0x7f64680c7fb0 0x7f646815a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:29.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.582+0000 7f646cd9b700 1 --2- 172.21.15.31:0/3593310240 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f64680c7be0 0x7f646806abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:29.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.583+0000 7f646d59c700 1 -- 172.21.15.31:0/3593310240 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f64680c7be0 msgr2=0x7f646806abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:29.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.583+0000 7f646d59c700 1 --2- 172.21.15.31:0/3593310240 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f64680c7be0 0x7f646806abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:29.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.583+0000 7f646d59c700 1 -- 172.21.15.31:0/3593310240 >> v2:172.21.15.62:3300/0 conn(0x7f64680c7fb0 msgr2=0x7f646815a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:29.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.583+0000 7f646d59c700 1 --2- 172.21.15.31:0/3593310240 >> v2:172.21.15.62:3300/0 conn(0x7f64680c7fb0 0x7f646815a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:29.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.583+0000 7f646d59c700 1 -- 172.21.15.31:0/3593310240 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f646806b290 con 0x7f646813f780 2023-10-06T02:30:29.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.583+0000 7f646d59c700 1 --2- 172.21.15.31:0/3593310240 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f646813f780 0x7f646813fbc0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f64680697f0 tx=0x7f6460009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc8afdcba00f8f8f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:29.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.583+0000 7f64677fe700 1 -- 172.21.15.31:0/3593310240 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f646000e070 con 0x7f646813f780 2023-10-06T02:30:29.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.583+0000 7f64677fe700 1 -- 172.21.15.31:0/3593310240 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f646000b970 con 0x7f646813f780 2023-10-06T02:30:29.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.583+0000 7f64677fe700 1 -- 172.21.15.31:0/3593310240 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64600092e0 con 0x7f646813f780 2023-10-06T02:30:29.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.584+0000 7f646dd9d700 1 -- 172.21.15.31:0/3593310240 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f646813f780 msgr2=0x7f646813fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:29.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.584+0000 7f646dd9d700 1 --2- 172.21.15.31:0/3593310240 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f646813f780 0x7f646813fbc0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f64680697f0 tx=0x7f6460009840 comp rx=0 tx=0).stop 2023-10-06T02:30:29.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.584+0000 7f646dd9d700 1 -- 172.21.15.31:0/3593310240 shutdown_connections 2023-10-06T02:30:29.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.584+0000 7f646dd9d700 1 --2- 172.21.15.31:0/3593310240 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f646813f780 0x7f646813fbc0 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:29.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.584+0000 7f646dd9d700 1 --2- 172.21.15.31:0/3593310240 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f64680c7be0 0x7f646806abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:29.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.584+0000 7f646dd9d700 1 --2- 172.21.15.31:0/3593310240 >> v2:172.21.15.62:3300/0 conn(0x7f64680c7fb0 0x7f646815a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:29.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.584+0000 7f646dd9d700 1 -- 172.21.15.31:0/3593310240 >> 172.21.15.31:0/3593310240 conn(0x7f646806d110 msgr2=0x7f64680cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:29.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.584+0000 7f646dd9d700 1 -- 172.21.15.31:0/3593310240 shutdown_connections 2023-10-06T02:30:29.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.584+0000 7f646dd9d700 1 -- 172.21.15.31:0/3593310240 wait complete. 2023-10-06T02:30:29.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.585+0000 7f646dd9d700 1 Processor -- start 2023-10-06T02:30:29.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.585+0000 7f646dd9d700 1 -- start start 2023-10-06T02:30:29.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.586+0000 7f646dd9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f64680c7be0 0x7f6468159890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:29.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.586+0000 7f646dd9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64680c7fb0 0x7f646816a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:29.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.586+0000 7f646dd9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f646813f780 0x7f646816cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:29.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.586+0000 7f646dd9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f646816deb0 con 0x7f64680c7fb0 2023-10-06T02:30:29.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.586+0000 7f646dd9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6468159dd0 con 0x7f646813f780 2023-10-06T02:30:29.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.586+0000 7f646dd9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64681699e0 con 0x7f64680c7be0 2023-10-06T02:30:29.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.587+0000 7f6467fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64680c7fb0 0x7f646816a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:29.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.587+0000 7f6467fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64680c7fb0 0x7f646816a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46938/0 (socket says 172.21.15.31:46938) 2023-10-06T02:30:29.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.587+0000 7f6467fff700 1 -- 172.21.15.31:0/280679399 learned_addr learned my addr 172.21.15.31:0/280679399 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:29.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.587+0000 7f646d59c700 1 --2- 172.21.15.31:0/280679399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f646813f780 0x7f646816cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:29.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.587+0000 7f646cd9b700 1 --2- 172.21.15.31:0/280679399 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f64680c7be0 0x7f6468159890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:29.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.587+0000 7f6467fff700 1 -- 172.21.15.31:0/280679399 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f64680c7be0 msgr2=0x7f6468159890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:29.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.587+0000 7f6467fff700 1 --2- 172.21.15.31:0/280679399 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f64680c7be0 0x7f6468159890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:29.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.587+0000 7f6467fff700 1 -- 172.21.15.31:0/280679399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f646813f780 msgr2=0x7f646816cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:29.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.587+0000 7f6467fff700 1 --2- 172.21.15.31:0/280679399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f646813f780 0x7f646816cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:29.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.587+0000 7f6467fff700 1 -- 172.21.15.31:0/280679399 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6460009ae0 con 0x7f64680c7fb0 2023-10-06T02:30:29.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.587+0000 7f646d59c700 1 --2- 172.21.15.31:0/280679399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f646813f780 0x7f646816cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:30:29.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.587+0000 7f6467fff700 1 --2- 172.21.15.31:0/280679399 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64680c7fb0 0x7f646816a3f0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f645400ec80 tx=0x7f64540091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:29.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.587+0000 7f6464ff9700 1 -- 172.21.15.31:0/280679399 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f645400c850 con 0x7f64680c7fb0 2023-10-06T02:30:29.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.588+0000 7f6464ff9700 1 -- 172.21.15.31:0/280679399 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f645400c9b0 con 0x7f64680c7fb0 2023-10-06T02:30:29.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.588+0000 7f6464ff9700 1 -- 172.21.15.31:0/280679399 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f645400cd50 con 0x7f64680c7fb0 2023-10-06T02:30:29.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.588+0000 7f646dd9d700 1 -- 172.21.15.31:0/280679399 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6468169f60 con 0x7f64680c7fb0 2023-10-06T02:30:29.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.588+0000 7f646dd9d700 1 -- 172.21.15.31:0/280679399 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f646816a990 con 0x7f64680c7fb0 2023-10-06T02:30:29.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.589+0000 7f646dd9d700 1 -- 172.21.15.31:0/280679399 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f646816dd20 con 0x7f64680c7fb0 2023-10-06T02:30:29.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.590+0000 7f6464ff9700 1 -- 172.21.15.31:0/280679399 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f6454013680 con 0x7f64680c7fb0 2023-10-06T02:30:29.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.590+0000 7f6464ff9700 1 --2- 172.21.15.31:0/280679399 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f644c0674d0 0x7f644c069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:29.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.591+0000 7f6464ff9700 1 -- 172.21.15.31:0/280679399 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f64540853e0 con 0x7f64680c7fb0 2023-10-06T02:30:29.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.591+0000 7f646cd9b700 1 --2- 172.21.15.31:0/280679399 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f644c0674d0 0x7f644c069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:29.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.592+0000 7f646cd9b700 1 --2- 172.21.15.31:0/280679399 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f644c0674d0 0x7f644c069980 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f645c009db0 tx=0x7f645c0091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:29.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.594+0000 7f6464ff9700 1 -- 172.21.15.31:0/280679399 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6454051c60 con 0x7f64680c7fb0 2023-10-06T02:30:29.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.843+0000 7f646dd9d700 1 -- 172.21.15.31:0/280679399 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "non_existent_subvolume", "clone_name": "clone_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f646806b570 con 0x7f644c0674d0 2023-10-06T02:30:29.846 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:30:29.845+0000 7f656537d700 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2023-10-06T02:30:29.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:29.846+0000 7f6464ff9700 1 -- 172.21.15.31:0/280679399 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) v1 ==== 45+0+0 (secure 0 0 0) 0x7f646806b570 con 0x7f644c0674d0 2023-10-06T02:30:29.848 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2023-10-06T02:30:29.856 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-06T02:30:29.857 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create non_existent_subvolume group_1 2023-10-06T02:30:30.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.046+0000 7ff23759e700 1 Processor -- start 2023-10-06T02:30:30.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.046+0000 7ff23759e700 1 -- start start 2023-10-06T02:30:30.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.047+0000 7ff23759e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff238153dd0 0x7ff23815de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:30.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.047+0000 7ff23759e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff2380cea00 0x7ff2380cee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:30.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.047+0000 7ff23759e700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff23815ac50 con 0x7ff238153a00 2023-10-06T02:30:30.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.047+0000 7ff23759e700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff23815d010 con 0x7ff2380cea00 2023-10-06T02:30:30.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.047+0000 7ff23759e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff23815af60 con 0x7ff238153dd0 2023-10-06T02:30:30.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.048+0000 7ff236d9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff2380cea00 0x7ff2380cee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:30.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.048+0000 7ff236d9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff2380cea00 0x7ff2380cee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46946/0 (socket says 172.21.15.31:46946) 2023-10-06T02:30:30.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.048+0000 7ff236d9d700 1 -- 172.21.15.31:0/3245115763 learned_addr learned my addr 172.21.15.31:0/3245115763 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:30.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.048+0000 7ff235d9b700 1 --2- 172.21.15.31:0/3245115763 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff238153dd0 0x7ff23815de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:30.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.048+0000 7ff236d9d700 1 -- 172.21.15.31:0/3245115763 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff238153dd0 msgr2=0x7ff23815de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:30.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.048+0000 7ff236d9d700 1 --2- 172.21.15.31:0/3245115763 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff238153dd0 0x7ff23815de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:30.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.048+0000 7ff236d9d700 1 -- 172.21.15.31:0/3245115763 >> v1:172.21.15.62:6789/0 conn(0x7ff238153a00 legacy=0x7ff23815a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:30.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.048+0000 7ff236d9d700 1 -- 172.21.15.31:0/3245115763 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff23815b240 con 0x7ff2380cea00 2023-10-06T02:30:30.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.048+0000 7ff236d9d700 1 --2- 172.21.15.31:0/3245115763 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2380cea00 0x7ff2380cee60 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7ff230009f50 tx=0x7ff230009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a23a46a1b6ffc2c7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:30.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.049+0000 7ff23559a700 1 -- 172.21.15.31:0/3245115763 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff23000e070 con 0x7ff2380cea00 2023-10-06T02:30:30.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.049+0000 7ff23559a700 1 -- 172.21.15.31:0/3245115763 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff23000b9c0 con 0x7ff2380cea00 2023-10-06T02:30:30.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.049+0000 7ff23559a700 1 -- 172.21.15.31:0/3245115763 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2300092e0 con 0x7ff2380cea00 2023-10-06T02:30:30.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.049+0000 7ff23759e700 1 -- 172.21.15.31:0/3245115763 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2380cea00 msgr2=0x7ff2380cee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:30.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.049+0000 7ff23759e700 1 --2- 172.21.15.31:0/3245115763 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2380cea00 0x7ff2380cee60 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7ff230009f50 tx=0x7ff230009840 comp rx=0 tx=0).stop 2023-10-06T02:30:30.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.049+0000 7ff23759e700 1 -- 172.21.15.31:0/3245115763 shutdown_connections 2023-10-06T02:30:30.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.049+0000 7ff23759e700 1 --2- 172.21.15.31:0/3245115763 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2380cea00 0x7ff2380cee60 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:30.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.049+0000 7ff23759e700 1 --2- 172.21.15.31:0/3245115763 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff238153dd0 0x7ff23815de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:30.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.049+0000 7ff23759e700 1 -- 172.21.15.31:0/3245115763 >> 172.21.15.31:0/3245115763 conn(0x7ff2380703a0 msgr2=0x7ff2380707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:30.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.050+0000 7ff23759e700 1 -- 172.21.15.31:0/3245115763 shutdown_connections 2023-10-06T02:30:30.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.050+0000 7ff23759e700 1 -- 172.21.15.31:0/3245115763 wait complete. 2023-10-06T02:30:30.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.051+0000 7ff23759e700 1 Processor -- start 2023-10-06T02:30:30.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.051+0000 7ff23759e700 1 -- start start 2023-10-06T02:30:30.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.052+0000 7ff23759e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2380cea00 0x7ff2380c9db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:30.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.052+0000 7ff23759e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff238153a00 0x7ff238154310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:30.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.052+0000 7ff23759e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff238153dd0 0x7ff238154850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:30.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.052+0000 7ff23759e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2380ca2f0 con 0x7ff2380cea00 2023-10-06T02:30:30.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.052+0000 7ff23759e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff23815e560 con 0x7ff238153dd0 2023-10-06T02:30:30.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.052+0000 7ff23759e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff23815e840 con 0x7ff238153a00 2023-10-06T02:30:30.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.052+0000 7ff23659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2380cea00 0x7ff2380c9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:30.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.052+0000 7ff23659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2380cea00 0x7ff2380c9db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46954/0 (socket says 172.21.15.31:46954) 2023-10-06T02:30:30.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.052+0000 7ff23659c700 1 -- 172.21.15.31:0/3302597827 learned_addr learned my addr 172.21.15.31:0/3302597827 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:30.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.052+0000 7ff235d9b700 1 --2- 172.21.15.31:0/3302597827 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff238153a00 0x7ff238154310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:30.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.053+0000 7ff236d9d700 1 --2- 172.21.15.31:0/3302597827 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff238153dd0 0x7ff238154850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:30.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.053+0000 7ff235d9b700 1 -- 172.21.15.31:0/3302597827 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff238153dd0 msgr2=0x7ff238154850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:30.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.053+0000 7ff235d9b700 1 --2- 172.21.15.31:0/3302597827 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff238153dd0 0x7ff238154850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:30.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.053+0000 7ff235d9b700 1 -- 172.21.15.31:0/3302597827 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2380cea00 msgr2=0x7ff2380c9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:30.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.053+0000 7ff235d9b700 1 --2- 172.21.15.31:0/3302597827 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2380cea00 0x7ff2380c9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:30.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.053+0000 7ff235d9b700 1 -- 172.21.15.31:0/3302597827 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff230009ae0 con 0x7ff238153a00 2023-10-06T02:30:30.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.053+0000 7ff23659c700 1 --2- 172.21.15.31:0/3302597827 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2380cea00 0x7ff2380c9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:30:30.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.053+0000 7ff235d9b700 1 --2- 172.21.15.31:0/3302597827 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff238153a00 0x7ff238154310 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff22c00e8d0 tx=0x7ff22c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:30.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.054+0000 7ff2267fc700 1 -- 172.21.15.31:0/3302597827 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff22c013680 con 0x7ff238153a00 2023-10-06T02:30:30.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.054+0000 7ff2267fc700 1 -- 172.21.15.31:0/3302597827 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff22c0137e0 con 0x7ff238153a00 2023-10-06T02:30:30.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.054+0000 7ff2267fc700 1 -- 172.21.15.31:0/3302597827 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff22c013ba0 con 0x7ff238153a00 2023-10-06T02:30:30.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.054+0000 7ff23759e700 1 -- 172.21.15.31:0/3302597827 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff23815eb20 con 0x7ff238153a00 2023-10-06T02:30:30.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.054+0000 7ff23759e700 1 -- 172.21.15.31:0/3302597827 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff23815f320 con 0x7ff238153a00 2023-10-06T02:30:30.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.055+0000 7ff23759e700 1 -- 172.21.15.31:0/3302597827 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff23804c3c0 con 0x7ff238153a00 2023-10-06T02:30:30.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.056+0000 7ff2267fc700 1 -- 172.21.15.31:0/3302597827 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7ff22c005a00 con 0x7ff238153a00 2023-10-06T02:30:30.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.057+0000 7ff2267fc700 1 --2- 172.21.15.31:0/3302597827 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff2180674d0 0x7ff218069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:30.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.057+0000 7ff2267fc700 1 -- 172.21.15.31:0/3302597827 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6086+0+0 (secure 0 0 0) 0x7ff22c084f10 con 0x7ff238153a00 2023-10-06T02:30:30.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.057+0000 7ff23659c700 1 --2- 172.21.15.31:0/3302597827 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff2180674d0 0x7ff218069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:30.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.058+0000 7ff23659c700 1 --2- 172.21.15.31:0/3302597827 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff2180674d0 0x7ff218069980 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff228001e90 tx=0x7ff228009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:30.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.061+0000 7ff2267fc700 1 -- 172.21.15.31:0/3302597827 <== mon.2 v2:172.21.15.62:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff22c051750 con 0x7ff238153a00 2023-10-06T02:30:30.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.295+0000 7ff23759e700 1 -- 172.21.15.31:0/3302597827 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "non_existent_subvolume", "group_name": "group_1", "target": ["mon-mgr", ""]}) v1 -- 0x7ff23815e6d0 con 0x7ff2180674d0 2023-10-06T02:30:30.298 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:30:30.297+0000 7f656537d700 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2023-10-06T02:30:30.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.297+0000 7ff2267fc700 1 -- 172.21.15.31:0/3302597827 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) v1 ==== 45+0+0 (secure 0 0 0) 0x7ff23815e6d0 con 0x7ff2180674d0 2023-10-06T02:30:30.300 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2023-10-06T02:30:30.308 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-06T02:30:30.308 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm non_existent_subvolume group_1 2023-10-06T02:30:30.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.497+0000 7f934339d700 1 Processor -- start 2023-10-06T02:30:30.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.497+0000 7f934339d700 1 -- start start 2023-10-06T02:30:30.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.498+0000 7f934339d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f933c15dfd0 0x7f933c0aedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:30.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.498+0000 7f934339d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f933c159920 0x7f933c0ad090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:30.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.499+0000 7f934339d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f933c0acc00 con 0x7f933c159550 2023-10-06T02:30:30.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.499+0000 7f934339d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f933c0abff0 con 0x7f933c159920 2023-10-06T02:30:30.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.499+0000 7f934339d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f933c0ac2d0 con 0x7f933c15dfd0 2023-10-06T02:30:30.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.499+0000 7f9342b9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f933c159920 0x7f933c0ad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:30.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.499+0000 7f9342b9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f933c159920 0x7f933c0ad090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46968/0 (socket says 172.21.15.31:46968) 2023-10-06T02:30:30.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.499+0000 7f9342b9c700 1 -- 172.21.15.31:0/3280453814 learned_addr learned my addr 172.21.15.31:0/3280453814 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:30.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.499+0000 7f9341b9a700 1 --2- 172.21.15.31:0/3280453814 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f933c15dfd0 0x7f933c0aedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:30.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.499+0000 7f9342b9c700 1 -- 172.21.15.31:0/3280453814 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f933c15dfd0 msgr2=0x7f933c0aedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:30.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.499+0000 7f9342b9c700 1 --2- 172.21.15.31:0/3280453814 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f933c15dfd0 0x7f933c0aedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:30.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.499+0000 7f9342b9c700 1 -- 172.21.15.31:0/3280453814 >> v1:172.21.15.62:6789/0 conn(0x7f933c159550 legacy=0x7f933c0ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:30.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.499+0000 7f9342b9c700 1 -- 172.21.15.31:0/3280453814 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f933c0ac5b0 con 0x7f933c159920 2023-10-06T02:30:30.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.499+0000 7f9341b9a700 1 --2- 172.21.15.31:0/3280453814 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f933c15dfd0 0x7f933c0aedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:30:30.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.499+0000 7f9342b9c700 1 --2- 172.21.15.31:0/3280453814 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f933c159920 0x7f933c0ad090 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f9334009f20 tx=0x7f9334009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ada2b22c7236c41 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:30.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.500+0000 7f9341399700 1 -- 172.21.15.31:0/3280453814 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f933400e070 con 0x7f933c159920 2023-10-06T02:30:30.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.500+0000 7f9341399700 1 -- 172.21.15.31:0/3280453814 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f933400ba80 con 0x7f933c159920 2023-10-06T02:30:30.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.500+0000 7f9341399700 1 -- 172.21.15.31:0/3280453814 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93340092e0 con 0x7f933c159920 2023-10-06T02:30:30.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.500+0000 7f934339d700 1 -- 172.21.15.31:0/3280453814 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f933c159920 msgr2=0x7f933c0ad090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:30.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.500+0000 7f934339d700 1 --2- 172.21.15.31:0/3280453814 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f933c159920 0x7f933c0ad090 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f9334009f20 tx=0x7f9334009840 comp rx=0 tx=0).stop 2023-10-06T02:30:30.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.500+0000 7f934339d700 1 -- 172.21.15.31:0/3280453814 shutdown_connections 2023-10-06T02:30:30.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.500+0000 7f934339d700 1 --2- 172.21.15.31:0/3280453814 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f933c159920 0x7f933c0ad090 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:30.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.500+0000 7f934339d700 1 --2- 172.21.15.31:0/3280453814 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f933c15dfd0 0x7f933c0aedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:30.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.500+0000 7f934339d700 1 -- 172.21.15.31:0/3280453814 >> 172.21.15.31:0/3280453814 conn(0x7f933c06ce80 msgr2=0x7f933c0ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:30.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.501+0000 7f934339d700 1 -- 172.21.15.31:0/3280453814 shutdown_connections 2023-10-06T02:30:30.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.501+0000 7f934339d700 1 -- 172.21.15.31:0/3280453814 wait complete. 2023-10-06T02:30:30.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.502+0000 7f934339d700 1 Processor -- start 2023-10-06T02:30:30.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.502+0000 7f934339d700 1 -- start start 2023-10-06T02:30:30.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.503+0000 7f934339d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f933c159550 0x7f933c0c7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:30.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.503+0000 7f934339d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f933c159920 0x7f933c1531d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:30.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.503+0000 7f934339d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f933c15dfd0 0x7f933c153710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:30.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.503+0000 7f934339d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f933c15f720 con 0x7f933c15dfd0 2023-10-06T02:30:30.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.503+0000 7f934339d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f933c0b2170 con 0x7f933c159550 2023-10-06T02:30:30.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.503+0000 7f934339d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f933c0b1840 con 0x7f933c159920 2023-10-06T02:30:30.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.503+0000 7f9342b9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f933c15dfd0 0x7f933c153710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:30.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.503+0000 7f9342b9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f933c15dfd0 0x7f933c153710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46982/0 (socket says 172.21.15.31:46982) 2023-10-06T02:30:30.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.503+0000 7f9342b9c700 1 -- 172.21.15.31:0/3879600227 learned_addr learned my addr 172.21.15.31:0/3879600227 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:30.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.503+0000 7f934239b700 1 --2- 172.21.15.31:0/3879600227 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f933c159550 0x7f933c0c7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:30.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.503+0000 7f9341b9a700 1 --2- 172.21.15.31:0/3879600227 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f933c159920 0x7f933c1531d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:30.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.504+0000 7f9342b9c700 1 -- 172.21.15.31:0/3879600227 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f933c159920 msgr2=0x7f933c1531d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:30.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.504+0000 7f9342b9c700 1 --2- 172.21.15.31:0/3879600227 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f933c159920 0x7f933c1531d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:30.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.504+0000 7f9342b9c700 1 -- 172.21.15.31:0/3879600227 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f933c159550 msgr2=0x7f933c0c7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:30.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.504+0000 7f9342b9c700 1 --2- 172.21.15.31:0/3879600227 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f933c159550 0x7f933c0c7ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:30.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.504+0000 7f9342b9c700 1 -- 172.21.15.31:0/3879600227 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9334009ae0 con 0x7f933c15dfd0 2023-10-06T02:30:30.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.504+0000 7f934239b700 1 --2- 172.21.15.31:0/3879600227 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f933c159550 0x7f933c0c7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:30:30.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.504+0000 7f9341b9a700 1 --2- 172.21.15.31:0/3879600227 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f933c159920 0x7f933c1531d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:30:30.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.504+0000 7f9342b9c700 1 --2- 172.21.15.31:0/3879600227 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f933c15dfd0 0x7f933c153710 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f9334008950 tx=0x7f9334018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:30.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.504+0000 7f93327fc700 1 -- 172.21.15.31:0/3879600227 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9334008ea0 con 0x7f933c15dfd0 2023-10-06T02:30:30.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.504+0000 7f93327fc700 1 -- 172.21.15.31:0/3879600227 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93340032f0 con 0x7f933c15dfd0 2023-10-06T02:30:30.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.504+0000 7f93327fc700 1 -- 172.21.15.31:0/3879600227 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93340266b0 con 0x7f933c15dfd0 2023-10-06T02:30:30.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.504+0000 7f934339d700 1 -- 172.21.15.31:0/3879600227 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f933c0b1b20 con 0x7f933c15dfd0 2023-10-06T02:30:30.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.504+0000 7f934339d700 1 -- 172.21.15.31:0/3879600227 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f933c0c8020 con 0x7f933c15dfd0 2023-10-06T02:30:30.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.505+0000 7f934339d700 1 -- 172.21.15.31:0/3879600227 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f933c0b2300 con 0x7f933c15dfd0 2023-10-06T02:30:30.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.506+0000 7f93327fc700 1 -- 172.21.15.31:0/3879600227 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f9334003040 con 0x7f933c15dfd0 2023-10-06T02:30:30.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.507+0000 7f93327fc700 1 --2- 172.21.15.31:0/3879600227 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f93200674d0 0x7f9320069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:30.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.507+0000 7f93327fc700 1 -- 172.21.15.31:0/3879600227 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f93340871f0 con 0x7f933c15dfd0 2023-10-06T02:30:30.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.507+0000 7f934239b700 1 --2- 172.21.15.31:0/3879600227 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f93200674d0 0x7f9320069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:30.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.508+0000 7f934239b700 1 --2- 172.21.15.31:0/3879600227 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f93200674d0 0x7f9320069980 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9338007fd0 tx=0x7f9338009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:30.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.511+0000 7f93327fc700 1 -- 172.21.15.31:0/3879600227 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9334053920 con 0x7f933c15dfd0 2023-10-06T02:30:30.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.742+0000 7f934339d700 1 -- 172.21.15.31:0/3879600227 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "non_existent_subvolume", "group_name": "group_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f933c04c3c0 con 0x7f93200674d0 2023-10-06T02:30:30.745 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:30:30.744+0000 7f656537d700 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2023-10-06T02:30:30.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.744+0000 7f93327fc700 1 -- 172.21.15.31:0/3879600227 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) v1 ==== 45+0+0 (secure 0 0 0) 0x7f933c04c3c0 con 0x7f93200674d0 2023-10-06T02:30:30.748 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2023-10-06T02:30:30.755 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-06T02:30:30.755 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup getpath non_existent_subvolume group_1 2023-10-06T02:30:30.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.944+0000 7fda27dff700 1 Processor -- start 2023-10-06T02:30:30.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.944+0000 7fda27dff700 1 -- start start 2023-10-06T02:30:30.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.946+0000 7fda27dff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fda200c8800 0x7fda200c94e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:30.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.946+0000 7fda27dff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fda200c8bd0 0x7fda200b0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:30.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.946+0000 7fda27dff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda200adbc0 con 0x7fda200b0f80 2023-10-06T02:30:30.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.946+0000 7fda27dff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda200b13d0 con 0x7fda200c8800 2023-10-06T02:30:30.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.946+0000 7fda27dff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda200c9d30 con 0x7fda200c8bd0 2023-10-06T02:30:30.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.946+0000 7fda25b9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fda200c8800 0x7fda200c94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:30.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.946+0000 7fda25b9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fda200c8800 0x7fda200c94e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46986/0 (socket says 172.21.15.31:46986) 2023-10-06T02:30:30.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.946+0000 7fda25b9b700 1 -- 172.21.15.31:0/25509460 learned_addr learned my addr 172.21.15.31:0/25509460 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:30.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.947+0000 7fda2539a700 1 --2- 172.21.15.31:0/25509460 >> v2:172.21.15.62:3300/0 conn(0x7fda200c8bd0 0x7fda200b0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:30.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.947+0000 7fda25b9b700 1 -- 172.21.15.31:0/25509460 >> v2:172.21.15.62:3300/0 conn(0x7fda200c8bd0 msgr2=0x7fda200b0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:30.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.947+0000 7fda25b9b700 1 --2- 172.21.15.31:0/25509460 >> v2:172.21.15.62:3300/0 conn(0x7fda200c8bd0 0x7fda200b0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:30.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.947+0000 7fda25b9b700 1 -- 172.21.15.31:0/25509460 >> v1:172.21.15.62:6789/0 conn(0x7fda200b0f80 legacy=0x7fda200ae050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:30.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.947+0000 7fda25b9b700 1 -- 172.21.15.31:0/25509460 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda200acfb0 con 0x7fda200c8800 2023-10-06T02:30:30.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.947+0000 7fda2539a700 1 --2- 172.21.15.31:0/25509460 >> v2:172.21.15.62:3300/0 conn(0x7fda200c8bd0 0x7fda200b0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:30:30.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.947+0000 7fda25b9b700 1 --2- 172.21.15.31:0/25509460 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda200c8800 0x7fda200c94e0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fda20069850 tx=0x7fda1c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b317da70f9c1b00 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:30.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.947+0000 7fda24b99700 1 -- 172.21.15.31:0/25509460 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda1c00b820 con 0x7fda200c8800 2023-10-06T02:30:30.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.947+0000 7fda24b99700 1 -- 172.21.15.31:0/25509460 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda1c00b980 con 0x7fda200c8800 2023-10-06T02:30:30.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.947+0000 7fda24b99700 1 -- 172.21.15.31:0/25509460 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda1c00bd20 con 0x7fda200c8800 2023-10-06T02:30:30.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.948+0000 7fda27dff700 1 -- 172.21.15.31:0/25509460 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda200c8800 msgr2=0x7fda200c94e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:30.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.948+0000 7fda27dff700 1 --2- 172.21.15.31:0/25509460 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda200c8800 0x7fda200c94e0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fda20069850 tx=0x7fda1c009250 comp rx=0 tx=0).stop 2023-10-06T02:30:30.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.948+0000 7fda27dff700 1 -- 172.21.15.31:0/25509460 shutdown_connections 2023-10-06T02:30:30.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.948+0000 7fda27dff700 1 --2- 172.21.15.31:0/25509460 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda200c8800 0x7fda200c94e0 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:30.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.948+0000 7fda27dff700 1 --2- 172.21.15.31:0/25509460 >> v2:172.21.15.62:3300/0 conn(0x7fda200c8bd0 0x7fda200b0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:30.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.948+0000 7fda27dff700 1 -- 172.21.15.31:0/25509460 >> 172.21.15.31:0/25509460 conn(0x7fda2006d170 msgr2=0x7fda200cc400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:30.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.948+0000 7fda27dff700 1 -- 172.21.15.31:0/25509460 shutdown_connections 2023-10-06T02:30:30.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.948+0000 7fda27dff700 1 -- 172.21.15.31:0/25509460 wait complete. 2023-10-06T02:30:30.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.949+0000 7fda27dff700 1 Processor -- start 2023-10-06T02:30:30.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.950+0000 7fda27dff700 1 -- start start 2023-10-06T02:30:30.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.950+0000 7fda27dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda200b0f80 0x7fda20147e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:30.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.950+0000 7fda27dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fda200c8800 0x7fda2015d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:30.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.950+0000 7fda27dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda200c8bd0 0x7fda2015b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:30.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.950+0000 7fda27dff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda200c59a0 con 0x7fda200b0f80 2023-10-06T02:30:30.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.950+0000 7fda27dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda2015de60 con 0x7fda200c8bd0 2023-10-06T02:30:30.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.950+0000 7fda27dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda2015e140 con 0x7fda200c8800 2023-10-06T02:30:30.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.951+0000 7fda25b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda200b0f80 0x7fda20147e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:30.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.951+0000 7fda25b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda200b0f80 0x7fda20147e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47000/0 (socket says 172.21.15.31:47000) 2023-10-06T02:30:30.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.951+0000 7fda25b9b700 1 -- 172.21.15.31:0/2499341976 learned_addr learned my addr 172.21.15.31:0/2499341976 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:30.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.951+0000 7fda2639c700 1 --2- 172.21.15.31:0/2499341976 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda200c8bd0 0x7fda2015b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:30.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.951+0000 7fda2539a700 1 --2- 172.21.15.31:0/2499341976 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fda200c8800 0x7fda2015d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:30.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.951+0000 7fda25b9b700 1 -- 172.21.15.31:0/2499341976 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fda200c8800 msgr2=0x7fda2015d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:30.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.951+0000 7fda25b9b700 1 --2- 172.21.15.31:0/2499341976 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fda200c8800 0x7fda2015d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:30.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.951+0000 7fda25b9b700 1 -- 172.21.15.31:0/2499341976 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda200c8bd0 msgr2=0x7fda2015b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:30.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.951+0000 7fda25b9b700 1 --2- 172.21.15.31:0/2499341976 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda200c8bd0 0x7fda2015b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:30.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.951+0000 7fda25b9b700 1 -- 172.21.15.31:0/2499341976 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda1c0094f0 con 0x7fda200b0f80 2023-10-06T02:30:30.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.951+0000 7fda25b9b700 1 --2- 172.21.15.31:0/2499341976 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda200b0f80 0x7fda20147e70 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fda1c009290 tx=0x7fda1c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:30.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.952+0000 7fda15ffb700 1 -- 172.21.15.31:0/2499341976 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda1c011e20 con 0x7fda200b0f80 2023-10-06T02:30:30.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.952+0000 7fda15ffb700 1 -- 172.21.15.31:0/2499341976 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda1c020070 con 0x7fda200b0f80 2023-10-06T02:30:30.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.952+0000 7fda27dff700 1 -- 172.21.15.31:0/2499341976 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda200c5bf0 con 0x7fda200b0f80 2023-10-06T02:30:30.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.952+0000 7fda15ffb700 1 -- 172.21.15.31:0/2499341976 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda1c01b6a0 con 0x7fda200b0f80 2023-10-06T02:30:30.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.952+0000 7fda27dff700 1 -- 172.21.15.31:0/2499341976 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda201483b0 con 0x7fda200b0f80 2023-10-06T02:30:30.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.953+0000 7fda27dff700 1 -- 172.21.15.31:0/2499341976 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda2004c3c0 con 0x7fda200b0f80 2023-10-06T02:30:30.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.954+0000 7fda15ffb700 1 -- 172.21.15.31:0/2499341976 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7fda1c022030 con 0x7fda200b0f80 2023-10-06T02:30:30.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.955+0000 7fda15ffb700 1 --2- 172.21.15.31:0/2499341976 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fda040674d0 0x7fda04069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:30.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.955+0000 7fda15ffb700 1 -- 172.21.15.31:0/2499341976 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fda1c086fd0 con 0x7fda200b0f80 2023-10-06T02:30:30.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.955+0000 7fda2539a700 1 --2- 172.21.15.31:0/2499341976 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fda040674d0 0x7fda04069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:30.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.956+0000 7fda2539a700 1 --2- 172.21.15.31:0/2499341976 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fda040674d0 0x7fda04069980 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fda200c81e0 tx=0x7fda10006f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:30.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:30.959+0000 7fda15ffb700 1 -- 172.21.15.31:0/2499341976 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fda1c053850 con 0x7fda200b0f80 2023-10-06T02:30:31.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.193+0000 7fda27dff700 1 -- 172.21.15.31:0/2499341976 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup getpath", "vol_name": "non_existent_subvolume", "group_name": "group_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fda2004c3c0 con 0x7fda040674d0 2023-10-06T02:30:31.197 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:30:31.196+0000 7f656537d700 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2023-10-06T02:30:31.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.196+0000 7fda15ffb700 1 -- 172.21.15.31:0/2499341976 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) v1 ==== 45+0+0 (secure 0 0 0) 0x7fda2004c3c0 con 0x7fda040674d0 2023-10-06T02:30:31.199 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2023-10-06T02:30:31.206 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-06T02:30:31.208 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup pin non_existent_subvolume group_1 export 0 2023-10-06T02:30:31.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.398+0000 7f4f2f59d700 1 Processor -- start 2023-10-06T02:30:31.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.398+0000 7f4f2f59d700 1 -- start start 2023-10-06T02:30:31.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.400+0000 7f4f2f59d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4f280c2a90 0x7f4f28156aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:31.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.400+0000 7f4f2f59d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4f280c85d0 0x7f4f280cc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:31.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.400+0000 7f4f2f59d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f281579b0 con 0x7f4f280c8200 2023-10-06T02:30:31.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.400+0000 7f4f2f59d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f280c1390 con 0x7f4f280c85d0 2023-10-06T02:30:31.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.400+0000 7f4f2f59d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f281573f0 con 0x7f4f280c2a90 2023-10-06T02:30:31.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.400+0000 7f4f2e59b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f4f280c8200 0x7f4f280c1820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49644/0 (socket says 172.21.15.31:49644) 2023-10-06T02:30:31.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.400+0000 7f4f2e59b700 1 -- 172.21.15.31:0/2698270485 learned_addr learned my addr 172.21.15.31:0/2698270485 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:30:31.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.400+0000 7f4f2ed9c700 1 --2- 172.21.15.31:0/2698270485 >> v2:172.21.15.31:3300/0 conn(0x7f4f280c85d0 0x7f4f280cc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:31.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.400+0000 7f4f2dd9a700 1 --2- 172.21.15.31:0/2698270485 >> v2:172.21.15.62:3300/0 conn(0x7f4f280c2a90 0x7f4f28156aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:31.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.401+0000 7f4f2ed9c700 1 -- 172.21.15.31:0/2698270485 >> v2:172.21.15.62:3300/0 conn(0x7f4f280c2a90 msgr2=0x7f4f28156aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:31.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.401+0000 7f4f2ed9c700 1 --2- 172.21.15.31:0/2698270485 >> v2:172.21.15.62:3300/0 conn(0x7f4f280c2a90 0x7f4f28156aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:31.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.401+0000 7f4f2ed9c700 1 -- 172.21.15.31:0/2698270485 >> v1:172.21.15.31:6789/0 conn(0x7f4f280c8200 legacy=0x7f4f280c1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:31.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.401+0000 7f4f2ed9c700 1 -- 172.21.15.31:0/2698270485 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f281576d0 con 0x7f4f280c85d0 2023-10-06T02:30:31.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.401+0000 7f4f2d599700 1 -- 172.21.15.31:0/2698270485 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3045947875 0 0) 0x7f4f281579b0 con 0x7f4f280c8200 2023-10-06T02:30:31.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.401+0000 7f4f2ed9c700 1 --2- 172.21.15.31:0/2698270485 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f280c85d0 0x7f4f280cc2f0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f4f2815b1d0 tx=0x7f4f20009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6990b1ba8ec783f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:31.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.401+0000 7f4f2d599700 1 -- 172.21.15.31:0/2698270485 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f2000e070 con 0x7f4f280c85d0 2023-10-06T02:30:31.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.401+0000 7f4f2d599700 1 -- 172.21.15.31:0/2698270485 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f2000bc30 con 0x7f4f280c85d0 2023-10-06T02:30:31.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.401+0000 7f4f2d599700 1 -- 172.21.15.31:0/2698270485 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f20012430 con 0x7f4f280c85d0 2023-10-06T02:30:31.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.402+0000 7f4f2f59d700 1 -- 172.21.15.31:0/2698270485 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f280c85d0 msgr2=0x7f4f280cc2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:31.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.402+0000 7f4f2f59d700 1 --2- 172.21.15.31:0/2698270485 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f280c85d0 0x7f4f280cc2f0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f4f2815b1d0 tx=0x7f4f20009780 comp rx=0 tx=0).stop 2023-10-06T02:30:31.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.402+0000 7f4f2f59d700 1 -- 172.21.15.31:0/2698270485 shutdown_connections 2023-10-06T02:30:31.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.402+0000 7f4f2f59d700 1 --2- 172.21.15.31:0/2698270485 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f280c85d0 0x7f4f280cc2f0 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:31.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.402+0000 7f4f2f59d700 1 --2- 172.21.15.31:0/2698270485 >> v2:172.21.15.62:3300/0 conn(0x7f4f280c2a90 0x7f4f28156aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:31.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.402+0000 7f4f2f59d700 1 -- 172.21.15.31:0/2698270485 >> 172.21.15.31:0/2698270485 conn(0x7f4f2806c6a0 msgr2=0x7f4f2806caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:31.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.402+0000 7f4f2f59d700 1 -- 172.21.15.31:0/2698270485 shutdown_connections 2023-10-06T02:30:31.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.402+0000 7f4f2f59d700 1 -- 172.21.15.31:0/2698270485 wait complete. 2023-10-06T02:30:31.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.403+0000 7f4f2f59d700 1 Processor -- start 2023-10-06T02:30:31.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.403+0000 7f4f2f59d700 1 -- start start 2023-10-06T02:30:31.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.404+0000 7f4f2f59d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f280c2a90 0x7f4f28148740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:31.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.404+0000 7f4f2f59d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f280c8200 0x7f4f2814a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:31.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.404+0000 7f4f2f59d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4f280c85d0 0x7f4f2814a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:31.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.404+0000 7f4f2f59d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f2814b6f0 con 0x7f4f280c8200 2023-10-06T02:30:31.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.404+0000 7f4f2f59d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f28148c80 con 0x7f4f280c2a90 2023-10-06T02:30:31.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.404+0000 7f4f2f59d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f2814b410 con 0x7f4f280c85d0 2023-10-06T02:30:31.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.405+0000 7f4f2dd9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f280c8200 0x7f4f2814a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:31.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.405+0000 7f4f2dd9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f280c8200 0x7f4f2814a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47020/0 (socket says 172.21.15.31:47020) 2023-10-06T02:30:31.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.405+0000 7f4f2dd9a700 1 -- 172.21.15.31:0/1452907826 learned_addr learned my addr 172.21.15.31:0/1452907826 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:31.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.405+0000 7f4f2e59b700 1 --2- 172.21.15.31:0/1452907826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f280c2a90 0x7f4f28148740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:31.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.405+0000 7f4f2ed9c700 1 --2- 172.21.15.31:0/1452907826 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4f280c85d0 0x7f4f2814a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:31.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.405+0000 7f4f2dd9a700 1 -- 172.21.15.31:0/1452907826 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4f280c85d0 msgr2=0x7f4f2814a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:31.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.405+0000 7f4f2dd9a700 1 --2- 172.21.15.31:0/1452907826 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4f280c85d0 0x7f4f2814a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:31.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.405+0000 7f4f2dd9a700 1 -- 172.21.15.31:0/1452907826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f280c2a90 msgr2=0x7f4f28148740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:31.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.405+0000 7f4f2dd9a700 1 --2- 172.21.15.31:0/1452907826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f280c2a90 0x7f4f28148740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:31.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.405+0000 7f4f2dd9a700 1 -- 172.21.15.31:0/1452907826 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f20009a20 con 0x7f4f280c8200 2023-10-06T02:30:31.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.405+0000 7f4f2ed9c700 1 --2- 172.21.15.31:0/1452907826 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4f280c85d0 0x7f4f2814a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:30:31.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.405+0000 7f4f2dd9a700 1 --2- 172.21.15.31:0/1452907826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f280c8200 0x7f4f2814a460 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f4f1800d860 tx=0x7f4f18008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:31.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.405+0000 7f4f1e7fc700 1 -- 172.21.15.31:0/1452907826 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f180125f0 con 0x7f4f280c8200 2023-10-06T02:30:31.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.406+0000 7f4f1e7fc700 1 -- 172.21.15.31:0/1452907826 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f18012750 con 0x7f4f280c8200 2023-10-06T02:30:31.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.406+0000 7f4f1e7fc700 1 -- 172.21.15.31:0/1452907826 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f18012af0 con 0x7f4f280c8200 2023-10-06T02:30:31.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.406+0000 7f4f2f59d700 1 -- 172.21.15.31:0/1452907826 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f280abf70 con 0x7f4f280c8200 2023-10-06T02:30:31.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.406+0000 7f4f2f59d700 1 -- 172.21.15.31:0/1452907826 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f280ac530 con 0x7f4f280c8200 2023-10-06T02:30:31.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.407+0000 7f4f2f59d700 1 -- 172.21.15.31:0/1452907826 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f280ac6c0 con 0x7f4f280c8200 2023-10-06T02:30:31.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.408+0000 7f4f1e7fc700 1 -- 172.21.15.31:0/1452907826 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f4f1801e460 con 0x7f4f280c8200 2023-10-06T02:30:31.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.408+0000 7f4f1e7fc700 1 --2- 172.21.15.31:0/1452907826 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f4f0c0674d0 0x7f4f0c069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:31.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.408+0000 7f4f1e7fc700 1 -- 172.21.15.31:0/1452907826 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f4f18085750 con 0x7f4f280c8200 2023-10-06T02:30:31.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.409+0000 7f4f2e59b700 1 --2- 172.21.15.31:0/1452907826 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f4f0c0674d0 0x7f4f0c069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:31.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.410+0000 7f4f2e59b700 1 --2- 172.21.15.31:0/1452907826 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f4f0c0674d0 0x7f4f0c069980 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f4f24007fd0 tx=0x7f4f24007d30 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:31.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.412+0000 7f4f1e7fc700 1 -- 172.21.15.31:0/1452907826 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4f18055a20 con 0x7f4f280c8200 2023-10-06T02:30:31.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.644+0000 7f4f2f59d700 1 -- 172.21.15.31:0/1452907826 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "non_existent_subvolume", "group_name": "group_1", "pin_type": "export", "pin_setting": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f4f2804c3c0 con 0x7f4f0c0674d0 2023-10-06T02:30:31.646 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:30:31.646+0000 7f656537d700 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2023-10-06T02:30:31.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.646+0000 7f4f1e7fc700 1 -- 172.21.15.31:0/1452907826 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) v1 ==== 45+0+0 (secure 0 0 0) 0x7f4f2804c3c0 con 0x7f4f0c0674d0 2023-10-06T02:30:31.649 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2023-10-06T02:30:31.656 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-06T02:30:31.657 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup ls non_existent_subvolume 2023-10-06T02:30:31.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.844+0000 7f25db59d700 1 Processor -- start 2023-10-06T02:30:31.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.845+0000 7f25db59d700 1 -- start start 2023-10-06T02:30:31.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.846+0000 7f25db59d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f25d4153c90 0x7f25d415a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:31.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.846+0000 7f25db59d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f25d406db20 0x7f25d40ca7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:31.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.846+0000 7f25db59d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25d415ada0 con 0x7f25d4154060 2023-10-06T02:30:31.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.846+0000 7f25db59d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25d406ca80 con 0x7f25d4153c90 2023-10-06T02:30:31.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.846+0000 7f25db59d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25d406cd60 con 0x7f25d406db20 2023-10-06T02:30:31.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.846+0000 7f25dad9c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f25d4154060 0x7f25d406a2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49654/0 (socket says 172.21.15.31:49654) 2023-10-06T02:30:31.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.846+0000 7f25dad9c700 1 -- 172.21.15.31:0/175798270 learned_addr learned my addr 172.21.15.31:0/175798270 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:30:31.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.846+0000 7f25d9d9a700 1 --2- 172.21.15.31:0/175798270 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f25d406db20 0x7f25d40ca7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:31.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.847+0000 7f25da59b700 1 --2- 172.21.15.31:0/175798270 >> v2:172.21.15.62:3300/0 conn(0x7f25d4153c90 0x7f25d415a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:31.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.847+0000 7f25d9599700 1 -- 172.21.15.31:0/175798270 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 196768345 0 0) 0x7f25cc001490 con 0x7f25d4154060 2023-10-06T02:30:31.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.847+0000 7f25d9599700 1 -- 172.21.15.31:0/175798270 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25c0003570 con 0x7f25d4154060 2023-10-06T02:30:31.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.847+0000 7f25d9599700 1 -- 172.21.15.31:0/175798270 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2217154998 0 0) 0x7f25c0003570 con 0x7f25d4154060 2023-10-06T02:30:31.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.847+0000 7f25d9599700 1 -- 172.21.15.31:0/175798270 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f25d406db20 msgr2=0x7f25d40ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:31.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.847+0000 7f25d9599700 1 --2- 172.21.15.31:0/175798270 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f25d406db20 0x7f25d40ca7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:31.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.847+0000 7f25d9599700 1 -- 172.21.15.31:0/175798270 >> v2:172.21.15.62:3300/0 conn(0x7f25d4153c90 msgr2=0x7f25d415a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:31.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.847+0000 7f25d9599700 1 --2- 172.21.15.31:0/175798270 >> v2:172.21.15.62:3300/0 conn(0x7f25d4153c90 0x7f25d415a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:31.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.847+0000 7f25d9599700 1 -- 172.21.15.31:0/175798270 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25d406d040 con 0x7f25d4154060 2023-10-06T02:30:31.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.847+0000 7f25d9599700 1 -- 172.21.15.31:0/175798270 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7f25cc001980 con 0x7f25d4154060 2023-10-06T02:30:31.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.847+0000 7f25d9599700 1 -- 172.21.15.31:0/175798270 >> v1:172.21.15.31:6789/0 conn(0x7f25d4154060 legacy=0x7f25d406a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:31.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.847+0000 7f25d9599700 1 --2- 172.21.15.31:0/175798270 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f25c0003ae0 0x7f25c0003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:31.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.847+0000 7f25d9599700 1 --2- 172.21.15.31:0/175798270 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25c0004810 0x7f25c0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:31.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.847+0000 7f25d9599700 1 --2- 172.21.15.31:0/175798270 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f25d406db20 0x7f25c00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:31.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.847+0000 7f25d9599700 1 -- 172.21.15.31:0/175798270 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25d406ca80 con 0x7f25c0004810 2023-10-06T02:30:31.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.847+0000 7f25d9599700 1 -- 172.21.15.31:0/175798270 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25d406cd60 con 0x7f25c0003ae0 2023-10-06T02:30:31.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.847+0000 7f25d9599700 1 -- 172.21.15.31:0/175798270 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25c000bb10 con 0x7f25d406db20 2023-10-06T02:30:31.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.847+0000 7f25d9d9a700 1 --2- 172.21.15.31:0/175798270 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25c0004810 0x7f25c0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:31.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.847+0000 7f25da59b700 1 --2- 172.21.15.31:0/175798270 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f25c0003ae0 0x7f25c0003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:31.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.848+0000 7f25dad9c700 1 --2- 172.21.15.31:0/175798270 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f25d406db20 0x7f25c00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:31.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.848+0000 7f25d9d9a700 1 -- 172.21.15.31:0/175798270 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f25d406db20 msgr2=0x7f25c00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:31.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.848+0000 7f25d9d9a700 1 --2- 172.21.15.31:0/175798270 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f25d406db20 0x7f25c00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:31.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.848+0000 7f25d9d9a700 1 -- 172.21.15.31:0/175798270 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f25c0003ae0 msgr2=0x7f25c0003ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:31.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.848+0000 7f25d9d9a700 1 --2- 172.21.15.31:0/175798270 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f25c0003ae0 0x7f25c0003ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:31.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.848+0000 7f25d9d9a700 1 -- 172.21.15.31:0/175798270 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25c0003570 con 0x7f25c0004810 2023-10-06T02:30:31.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.848+0000 7f25d9d9a700 1 --2- 172.21.15.31:0/175798270 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25c0004810 0x7f25c0004c80 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f25d406df10 tx=0x7f25c40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c60d009e6716634f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:31.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.848+0000 7f25d9599700 1 -- 172.21.15.31:0/175798270 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c400e710 con 0x7f25c0004810 2023-10-06T02:30:31.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.848+0000 7f25d9599700 1 -- 172.21.15.31:0/175798270 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25c400e870 con 0x7f25c0004810 2023-10-06T02:30:31.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.848+0000 7f25d9599700 1 -- 172.21.15.31:0/175798270 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c400ec30 con 0x7f25c0004810 2023-10-06T02:30:31.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.849+0000 7f25db59d700 1 -- 172.21.15.31:0/175798270 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25c0004810 msgr2=0x7f25c0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:31.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.849+0000 7f25db59d700 1 --2- 172.21.15.31:0/175798270 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25c0004810 0x7f25c0004c80 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f25d406df10 tx=0x7f25c40091a0 comp rx=0 tx=0).stop 2023-10-06T02:30:31.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.849+0000 7f25dad9c700 1 -- 172.21.15.31:0/175798270 reap_dead start 2023-10-06T02:30:31.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.849+0000 7f25db59d700 1 -- 172.21.15.31:0/175798270 shutdown_connections 2023-10-06T02:30:31.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.849+0000 7f25db59d700 1 -- 172.21.15.31:0/175798270 >> 172.21.15.31:0/175798270 conn(0x7f25d406e720 msgr2=0x7f25d406eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:31.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.849+0000 7f25db59d700 1 -- 172.21.15.31:0/175798270 shutdown_connections 2023-10-06T02:30:31.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.849+0000 7f25db59d700 1 -- 172.21.15.31:0/175798270 wait complete. 2023-10-06T02:30:31.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.850+0000 7f25db59d700 1 Processor -- start 2023-10-06T02:30:31.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.850+0000 7f25db59d700 1 -- start start 2023-10-06T02:30:31.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.851+0000 7f25db59d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25c0004810 0x7f25d414efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:31.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.851+0000 7f25db59d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f25d415ea90 0x7f25d415ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:31.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.851+0000 7f25db59d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f25d4151110 0x7f25d4151590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:31.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.851+0000 7f25db59d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25d40b0f60 con 0x7f25c0004810 2023-10-06T02:30:31.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.851+0000 7f25db59d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25d4151ad0 con 0x7f25d415ea90 2023-10-06T02:30:31.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.851+0000 7f25db59d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25d41505d0 con 0x7f25d4151110 2023-10-06T02:30:31.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.852+0000 7f25da59b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25c0004810 0x7f25d414efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:31.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.852+0000 7f25da59b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25c0004810 0x7f25d414efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47034/0 (socket says 172.21.15.31:47034) 2023-10-06T02:30:31.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.852+0000 7f25da59b700 1 -- 172.21.15.31:0/1908310295 learned_addr learned my addr 172.21.15.31:0/1908310295 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:31.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.852+0000 7f25dad9c700 1 --2- 172.21.15.31:0/1908310295 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f25d4151110 0x7f25d4151590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:31.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.852+0000 7f25d9d9a700 1 --2- 172.21.15.31:0/1908310295 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f25d415ea90 0x7f25d415ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:31.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.852+0000 7f25da59b700 1 -- 172.21.15.31:0/1908310295 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f25d4151110 msgr2=0x7f25d4151590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:31.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.852+0000 7f25da59b700 1 --2- 172.21.15.31:0/1908310295 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f25d4151110 0x7f25d4151590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:31.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.852+0000 7f25da59b700 1 -- 172.21.15.31:0/1908310295 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f25d415ea90 msgr2=0x7f25d415ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:31.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.852+0000 7f25da59b700 1 --2- 172.21.15.31:0/1908310295 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f25d415ea90 0x7f25d415ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:31.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.852+0000 7f25da59b700 1 -- 172.21.15.31:0/1908310295 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25d414f510 con 0x7f25c0004810 2023-10-06T02:30:31.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.852+0000 7f25da59b700 1 --2- 172.21.15.31:0/1908310295 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25c0004810 0x7f25d414efd0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f25d0011d40 tx=0x7f25d00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:31.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.852+0000 7f25ca7fc700 1 -- 172.21.15.31:0/1908310295 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d000fac0 con 0x7f25c0004810 2023-10-06T02:30:31.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.852+0000 7f25ca7fc700 1 -- 172.21.15.31:0/1908310295 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25d000fc20 con 0x7f25c0004810 2023-10-06T02:30:31.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.852+0000 7f25ca7fc700 1 -- 172.21.15.31:0/1908310295 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d0014440 con 0x7f25c0004810 2023-10-06T02:30:31.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.852+0000 7f25db59d700 1 -- 172.21.15.31:0/1908310295 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25d414fa50 con 0x7f25c0004810 2023-10-06T02:30:31.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.853+0000 7f25db59d700 1 -- 172.21.15.31:0/1908310295 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25d4150010 con 0x7f25c0004810 2023-10-06T02:30:31.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.853+0000 7f25db59d700 1 -- 172.21.15.31:0/1908310295 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25d404c3c0 con 0x7f25c0004810 2023-10-06T02:30:31.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.854+0000 7f25ca7fc700 1 -- 172.21.15.31:0/1908310295 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f25d0014a80 con 0x7f25c0004810 2023-10-06T02:30:31.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.855+0000 7f25ca7fc700 1 --2- 172.21.15.31:0/1908310295 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f25b80674d0 0x7f25b8069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:31.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.855+0000 7f25ca7fc700 1 -- 172.21.15.31:0/1908310295 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f25d0018030 con 0x7f25c0004810 2023-10-06T02:30:31.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.856+0000 7f25d9d9a700 1 --2- 172.21.15.31:0/1908310295 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f25b80674d0 0x7f25b8069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:31.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.857+0000 7f25d9d9a700 1 --2- 172.21.15.31:0/1908310295 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f25b80674d0 0x7f25b8069980 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f25c40091e0 tx=0x7f25c401a000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:31.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:31.860+0000 7f25ca7fc700 1 -- 172.21.15.31:0/1908310295 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f25d0052720 con 0x7f25c0004810 2023-10-06T02:30:32.079 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T02:30:32.084 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T02:30:32.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.091+0000 7f25db59d700 1 -- 172.21.15.31:0/1908310295 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup ls", "vol_name": "non_existent_subvolume", "target": ["mon-mgr", ""]}) v1 -- 0x7f25d4159bc0 con 0x7f25b80674d0 2023-10-06T02:30:32.093 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:30:32.093+0000 7f656537d700 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2023-10-06T02:30:32.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.093+0000 7f25ca7fc700 1 -- 172.21.15.31:0/1908310295 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) v1 ==== 45+0+0 (secure 0 0 0) 0x7f25d4159bc0 con 0x7f25b80674d0 2023-10-06T02:30:32.096 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2023-10-06T02:30:32.104 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-06T02:30:32.104 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup snapshot create non_existent_subvolume group_1 snapshot_1 2023-10-06T02:30:32.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.337+0000 7f78cb59d700 1 Processor -- start 2023-10-06T02:30:32.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.338+0000 7f78cb59d700 1 -- start start 2023-10-06T02:30:32.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.339+0000 7f78cb59d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f78c40c8200 0x7f78c40c1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:32.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.339+0000 7f78cb59d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f78c40c85d0 0x7f78c40cc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:32.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.339+0000 7f78cb59d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78c41571b0 con 0x7f78c40c2a90 2023-10-06T02:30:32.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.339+0000 7f78cb59d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78c41579b0 con 0x7f78c40c85d0 2023-10-06T02:30:32.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.339+0000 7f78cb59d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78c40c1390 con 0x7f78c40c8200 2023-10-06T02:30:32.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.339+0000 7f78cad9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f78c40c85d0 0x7f78c40cc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:32.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.339+0000 7f78ca59b700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f78c40c8200 0x7f78c40c1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:32.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.339+0000 7f78ca59b700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f78c40c8200 0x7f78c40c1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:52910/0 (socket says 172.21.15.31:52910) 2023-10-06T02:30:32.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.339+0000 7f78ca59b700 1 -- 172.21.15.31:0/3372228214 learned_addr learned my addr 172.21.15.31:0/3372228214 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:32.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.339+0000 7f78c9d9a700 1 --1- 172.21.15.31:0/3372228214 >> v1:172.21.15.62:6789/0 conn(0x7f78c40c2a90 0x7f78c4156aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:46590/0 (socket says 172.21.15.31:46590) 2023-10-06T02:30:32.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.340+0000 7f78cad9c700 1 -- 172.21.15.31:0/3372228214 >> v2:172.21.15.62:3300/0 conn(0x7f78c40c8200 msgr2=0x7f78c40c1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:32.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.340+0000 7f78cad9c700 1 --2- 172.21.15.31:0/3372228214 >> v2:172.21.15.62:3300/0 conn(0x7f78c40c8200 0x7f78c40c1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:32.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.340+0000 7f78cad9c700 1 -- 172.21.15.31:0/3372228214 >> v1:172.21.15.62:6789/0 conn(0x7f78c40c2a90 legacy=0x7f78c4156aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:32.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.340+0000 7f78cad9c700 1 -- 172.21.15.31:0/3372228214 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78c41573f0 con 0x7f78c40c85d0 2023-10-06T02:30:32.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.340+0000 7f78ca59b700 1 --2- 172.21.15.31:0/3372228214 >> v2:172.21.15.62:3300/0 conn(0x7f78c40c8200 0x7f78c40c1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:30:32.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.340+0000 7f78cad9c700 1 --2- 172.21.15.31:0/3372228214 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78c40c85d0 0x7f78c40cc2f0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f78c40683d0 tx=0x7f78bc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=570ea1e9a716eb8a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:32.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.340+0000 7f78c9599700 1 -- 172.21.15.31:0/3372228214 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78bc00e070 con 0x7f78c40c85d0 2023-10-06T02:30:32.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.341+0000 7f78c9599700 1 -- 172.21.15.31:0/3372228214 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78bc00bb60 con 0x7f78c40c85d0 2023-10-06T02:30:32.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.341+0000 7f78c9599700 1 -- 172.21.15.31:0/3372228214 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78bc012430 con 0x7f78c40c85d0 2023-10-06T02:30:32.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.341+0000 7f78cb59d700 1 -- 172.21.15.31:0/3372228214 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78c40c85d0 msgr2=0x7f78c40cc2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:32.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.341+0000 7f78cb59d700 1 --2- 172.21.15.31:0/3372228214 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78c40c85d0 0x7f78c40cc2f0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f78c40683d0 tx=0x7f78bc009780 comp rx=0 tx=0).stop 2023-10-06T02:30:32.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.341+0000 7f78cb59d700 1 -- 172.21.15.31:0/3372228214 shutdown_connections 2023-10-06T02:30:32.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.341+0000 7f78cb59d700 1 --2- 172.21.15.31:0/3372228214 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78c40c85d0 0x7f78c40cc2f0 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:32.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.341+0000 7f78cb59d700 1 --2- 172.21.15.31:0/3372228214 >> v2:172.21.15.62:3300/0 conn(0x7f78c40c8200 0x7f78c40c1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:32.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.341+0000 7f78cb59d700 1 -- 172.21.15.31:0/3372228214 >> 172.21.15.31:0/3372228214 conn(0x7f78c406c6a0 msgr2=0x7f78c406caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:32.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.341+0000 7f78cb59d700 1 -- 172.21.15.31:0/3372228214 shutdown_connections 2023-10-06T02:30:32.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.341+0000 7f78cb59d700 1 -- 172.21.15.31:0/3372228214 wait complete. 2023-10-06T02:30:32.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.343+0000 7f78cb59d700 1 Processor -- start 2023-10-06T02:30:32.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.343+0000 7f78cb59d700 1 -- start start 2023-10-06T02:30:32.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.344+0000 7f78cb59d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78c40c2a90 0x7f78c40b1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:32.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.344+0000 7f78cb59d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78c40c8200 0x7f78c414e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:32.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.344+0000 7f78cb59d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f78c40c85d0 0x7f78c40ac500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:32.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.344+0000 7f78cb59d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78c40ac070 con 0x7f78c40c2a90 2023-10-06T02:30:32.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.344+0000 7f78cb59d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78c40b1ce0 con 0x7f78c40c8200 2023-10-06T02:30:32.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.344+0000 7f78cb59d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78c40ab740 con 0x7f78c40c85d0 2023-10-06T02:30:32.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.344+0000 7f78ca59b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78c40c2a90 0x7f78c40b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:32.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.344+0000 7f78ca59b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78c40c2a90 0x7f78c40b1490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47046/0 (socket says 172.21.15.31:47046) 2023-10-06T02:30:32.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.344+0000 7f78ca59b700 1 -- 172.21.15.31:0/3957797149 learned_addr learned my addr 172.21.15.31:0/3957797149 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:32.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.344+0000 7f78c9d9a700 1 --2- 172.21.15.31:0/3957797149 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78c40c8200 0x7f78c414e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:32.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.344+0000 7f78cad9c700 1 --2- 172.21.15.31:0/3957797149 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f78c40c85d0 0x7f78c40ac500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:32.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.344+0000 7f78ca59b700 1 -- 172.21.15.31:0/3957797149 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f78c40c85d0 msgr2=0x7f78c40ac500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:32.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.344+0000 7f78ca59b700 1 --2- 172.21.15.31:0/3957797149 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f78c40c85d0 0x7f78c40ac500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:32.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.344+0000 7f78ca59b700 1 -- 172.21.15.31:0/3957797149 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78c40c8200 msgr2=0x7f78c414e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:32.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.344+0000 7f78ca59b700 1 --2- 172.21.15.31:0/3957797149 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78c40c8200 0x7f78c414e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:32.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.345+0000 7f78ca59b700 1 -- 172.21.15.31:0/3957797149 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78c00094f0 con 0x7f78c40c2a90 2023-10-06T02:30:32.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.345+0000 7f78c9d9a700 1 --2- 172.21.15.31:0/3957797149 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78c40c8200 0x7f78c414e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:30:32.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.345+0000 7f78ca59b700 1 --2- 172.21.15.31:0/3957797149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78c40c2a90 0x7f78c40b1490 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f78c000ec50 tx=0x7f78c0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:32.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.345+0000 7f78ba7fc700 1 -- 172.21.15.31:0/3957797149 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c000c9d0 con 0x7f78c40c2a90 2023-10-06T02:30:32.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.345+0000 7f78ba7fc700 1 -- 172.21.15.31:0/3957797149 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78c000cb30 con 0x7f78c40c2a90 2023-10-06T02:30:32.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.345+0000 7f78cb59d700 1 -- 172.21.15.31:0/3957797149 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78bc009a20 con 0x7f78c40c2a90 2023-10-06T02:30:32.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.345+0000 7f78ba7fc700 1 -- 172.21.15.31:0/3957797149 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c0013470 con 0x7f78c40c2a90 2023-10-06T02:30:32.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.345+0000 7f78cb59d700 1 -- 172.21.15.31:0/3957797149 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78c40abd80 con 0x7f78c40c2a90 2023-10-06T02:30:32.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.346+0000 7f78cb59d700 1 -- 172.21.15.31:0/3957797149 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78c406d300 con 0x7f78c40c2a90 2023-10-06T02:30:32.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.347+0000 7f78ba7fc700 1 -- 172.21.15.31:0/3957797149 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f78c0013860 con 0x7f78c40c2a90 2023-10-06T02:30:32.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.348+0000 7f78ba7fc700 1 --2- 172.21.15.31:0/3957797149 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f78a80674d0 0x7f78a8069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:32.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.348+0000 7f78ba7fc700 1 -- 172.21.15.31:0/3957797149 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f78c0085830 con 0x7f78c40c2a90 2023-10-06T02:30:32.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.349+0000 7f78c9d9a700 1 --2- 172.21.15.31:0/3957797149 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f78a80674d0 0x7f78a8069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:32.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.349+0000 7f78c9d9a700 1 --2- 172.21.15.31:0/3957797149 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f78a80674d0 0x7f78a8069980 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f78b4007fd0 tx=0x7f78b4007e10 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:32.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.352+0000 7f78ba7fc700 1 -- 172.21.15.31:0/3957797149 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f78c0052130 con 0x7f78c40c2a90 2023-10-06T02:30:32.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.590+0000 7f78cb59d700 1 -- 172.21.15.31:0/3957797149 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup snapshot create", "vol_name": "non_existent_subvolume", "group_name": "group_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f78c40ac1b0 con 0x7f78a80674d0 2023-10-06T02:30:32.593 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:30:32.593+0000 7f656537d700 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2023-10-06T02:30:32.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.593+0000 7f78ba7fc700 1 -- 172.21.15.31:0/3957797149 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) v1 ==== 45+0+0 (secure 0 0 0) 0x7f78b4005880 con 0x7f78a80674d0 2023-10-06T02:30:32.596 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2023-10-06T02:30:32.603 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-06T02:30:32.604 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup snapshot rm non_existent_subvolume group_1 snapshot_1 2023-10-06T02:30:32.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.795+0000 7f5dde39d700 1 Processor -- start 2023-10-06T02:30:32.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.795+0000 7f5dde39d700 1 -- start start 2023-10-06T02:30:32.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.796+0000 7f5dde39d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5dd815cb90 0x7f5dd80c7870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:32.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.796+0000 7f5dde39d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5dd806cdb0 0x7f5dd80c3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:32.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.796+0000 7f5dde39d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dd80c2fa0 con 0x7f5dd815dc10 2023-10-06T02:30:32.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.797+0000 7f5dde39d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dd80ccdf0 con 0x7f5dd806cdb0 2023-10-06T02:30:32.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.797+0000 7f5dde39d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd80c9b20 con 0x7f5dd815cb90 2023-10-06T02:30:32.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.797+0000 7f5dddb9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5dd806cdb0 0x7f5dd80c3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:32.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.797+0000 7f5dddb9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5dd806cdb0 0x7f5dd80c3430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47062/0 (socket says 172.21.15.31:47062) 2023-10-06T02:30:32.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.797+0000 7f5dddb9c700 1 -- 172.21.15.31:0/1138445097 learned_addr learned my addr 172.21.15.31:0/1138445097 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:32.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.797+0000 7f5ddd39b700 1 --2- 172.21.15.31:0/1138445097 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5dd815cb90 0x7f5dd80c7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:32.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.797+0000 7f5dddb9c700 1 -- 172.21.15.31:0/1138445097 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5dd815cb90 msgr2=0x7f5dd80c7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:32.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.797+0000 7f5dddb9c700 1 --2- 172.21.15.31:0/1138445097 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5dd815cb90 0x7f5dd80c7870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:32.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.797+0000 7f5dddb9c700 1 -- 172.21.15.31:0/1138445097 >> v1:172.21.15.62:6789/0 conn(0x7f5dd815dc10 legacy=0x7f5dd806c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:32.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.797+0000 7f5dddb9c700 1 -- 172.21.15.31:0/1138445097 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dd80c2390 con 0x7f5dd806cdb0 2023-10-06T02:30:32.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.797+0000 7f5ddd39b700 1 --2- 172.21.15.31:0/1138445097 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5dd815cb90 0x7f5dd80c7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:30:32.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.798+0000 7f5dddb9c700 1 --2- 172.21.15.31:0/1138445097 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5dd806cdb0 0x7f5dd80c3430 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f5dd0009f20 tx=0x7f5dd0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28f78c5a8463dde server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:32.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.798+0000 7f5dcffff700 1 -- 172.21.15.31:0/1138445097 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd000e070 con 0x7f5dd806cdb0 2023-10-06T02:30:32.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.798+0000 7f5dcffff700 1 -- 172.21.15.31:0/1138445097 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dd000ba40 con 0x7f5dd806cdb0 2023-10-06T02:30:32.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.798+0000 7f5dcffff700 1 -- 172.21.15.31:0/1138445097 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd00092e0 con 0x7f5dd806cdb0 2023-10-06T02:30:32.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.798+0000 7f5dde39d700 1 -- 172.21.15.31:0/1138445097 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5dd806cdb0 msgr2=0x7f5dd80c3430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:32.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.798+0000 7f5dde39d700 1 --2- 172.21.15.31:0/1138445097 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5dd806cdb0 0x7f5dd80c3430 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f5dd0009f20 tx=0x7f5dd0009840 comp rx=0 tx=0).stop 2023-10-06T02:30:32.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.799+0000 7f5dde39d700 1 -- 172.21.15.31:0/1138445097 shutdown_connections 2023-10-06T02:30:32.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.799+0000 7f5dde39d700 1 --2- 172.21.15.31:0/1138445097 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5dd806cdb0 0x7f5dd80c3430 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:32.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.799+0000 7f5dde39d700 1 --2- 172.21.15.31:0/1138445097 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5dd815cb90 0x7f5dd80c7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:32.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.799+0000 7f5dde39d700 1 -- 172.21.15.31:0/1138445097 >> 172.21.15.31:0/1138445097 conn(0x7f5dd80cd230 msgr2=0x7f5dd80cd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:32.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.799+0000 7f5dde39d700 1 -- 172.21.15.31:0/1138445097 shutdown_connections 2023-10-06T02:30:32.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.799+0000 7f5dde39d700 1 -- 172.21.15.31:0/1138445097 wait complete. 2023-10-06T02:30:32.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.800+0000 7f5dde39d700 1 Processor -- start 2023-10-06T02:30:32.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.800+0000 7f5dde39d700 1 -- start start 2023-10-06T02:30:32.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.801+0000 7f5dde39d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5dd806cdb0 0x7f5dd8145010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:32.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.801+0000 7f5dde39d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5dd815cb90 0x7f5dd80c42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:32.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.801+0000 7f5dde39d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5dd815dc10 0x7f5dd80affd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:32.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.801+0000 7f5dde39d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd80b0f00 con 0x7f5dd815dc10 2023-10-06T02:30:32.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.801+0000 7f5dde39d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd80b1e80 con 0x7f5dd806cdb0 2023-10-06T02:30:32.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.801+0000 7f5dde39d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd80b0940 con 0x7f5dd815cb90 2023-10-06T02:30:32.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.801+0000 7f5dddb9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5dd815dc10 0x7f5dd80affd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:32.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.801+0000 7f5dddb9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5dd815dc10 0x7f5dd80affd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47066/0 (socket says 172.21.15.31:47066) 2023-10-06T02:30:32.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.801+0000 7f5dddb9c700 1 -- 172.21.15.31:0/1310572411 learned_addr learned my addr 172.21.15.31:0/1310572411 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:32.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.802+0000 7f5ddd39b700 1 --2- 172.21.15.31:0/1310572411 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5dd806cdb0 0x7f5dd8145010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:32.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.802+0000 7f5ddcb9a700 1 --2- 172.21.15.31:0/1310572411 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5dd815cb90 0x7f5dd80c42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:32.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.802+0000 7f5dddb9c700 1 -- 172.21.15.31:0/1310572411 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5dd815cb90 msgr2=0x7f5dd80c42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:32.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.802+0000 7f5dddb9c700 1 --2- 172.21.15.31:0/1310572411 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5dd815cb90 0x7f5dd80c42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:32.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.802+0000 7f5dddb9c700 1 -- 172.21.15.31:0/1310572411 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5dd806cdb0 msgr2=0x7f5dd8145010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:32.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.802+0000 7f5dddb9c700 1 --2- 172.21.15.31:0/1310572411 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5dd806cdb0 0x7f5dd8145010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:32.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.802+0000 7f5dddb9c700 1 -- 172.21.15.31:0/1310572411 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dd0009ae0 con 0x7f5dd815dc10 2023-10-06T02:30:32.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.802+0000 7f5ddd39b700 1 --2- 172.21.15.31:0/1310572411 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5dd806cdb0 0x7f5dd8145010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:30:32.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.802+0000 7f5ddcb9a700 1 --2- 172.21.15.31:0/1310572411 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5dd815cb90 0x7f5dd80c42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:30:32.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.802+0000 7f5dddb9c700 1 --2- 172.21.15.31:0/1310572411 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5dd815dc10 0x7f5dd80affd0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f5dd0003000 tx=0x7f5dd0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:32.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.802+0000 7f5dcd7fa700 1 -- 172.21.15.31:0/1310572411 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd0012e60 con 0x7f5dd815dc10 2023-10-06T02:30:32.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.802+0000 7f5dcd7fa700 1 -- 172.21.15.31:0/1310572411 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dd001e430 con 0x7f5dd815dc10 2023-10-06T02:30:32.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.802+0000 7f5dcd7fa700 1 -- 172.21.15.31:0/1310572411 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd001e7f0 con 0x7f5dd815dc10 2023-10-06T02:30:32.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.802+0000 7f5dde39d700 1 -- 172.21.15.31:0/1310572411 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dd80b0c20 con 0x7f5dd815dc10 2023-10-06T02:30:32.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.803+0000 7f5dde39d700 1 -- 172.21.15.31:0/1310572411 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dd80c4820 con 0x7f5dd815dc10 2023-10-06T02:30:32.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.803+0000 7f5dde39d700 1 -- 172.21.15.31:0/1310572411 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dd815d8f0 con 0x7f5dd815dc10 2023-10-06T02:30:32.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.804+0000 7f5dcd7fa700 1 -- 172.21.15.31:0/1310572411 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f5dd001eba0 con 0x7f5dd815dc10 2023-10-06T02:30:32.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.805+0000 7f5dcd7fa700 1 --2- 172.21.15.31:0/1310572411 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5dbc0674d0 0x7f5dbc069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:32.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.805+0000 7f5dcd7fa700 1 -- 172.21.15.31:0/1310572411 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f5dd0086ed0 con 0x7f5dd815dc10 2023-10-06T02:30:32.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.806+0000 7f5ddd39b700 1 --2- 172.21.15.31:0/1310572411 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5dbc0674d0 0x7f5dbc069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:32.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.806+0000 7f5ddd39b700 1 --2- 172.21.15.31:0/1310572411 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5dbc0674d0 0x7f5dbc069980 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f5dd4008b20 tx=0x7f5dd4008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:32.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:32.809+0000 7f5dcd7fa700 1 -- 172.21.15.31:0/1310572411 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5dd0053600 con 0x7f5dd815dc10 2023-10-06T02:30:33.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:33.042+0000 7f5dde39d700 1 -- 172.21.15.31:0/1310572411 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup snapshot rm", "vol_name": "non_existent_subvolume", "group_name": "group_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f5dd815d8f0 con 0x7f5dbc0674d0 2023-10-06T02:30:33.045 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:30:33.044+0000 7f656537d700 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2023-10-06T02:30:33.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:33.044+0000 7f5dcd7fa700 1 -- 172.21.15.31:0/1310572411 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) v1 ==== 45+0+0 (secure 0 0 0) 0x7f5dd815d8f0 con 0x7f5dbc0674d0 2023-10-06T02:30:33.047 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2023-10-06T02:30:33.054 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-06T02:30:33.055 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup snapshot ls non_existent_subvolume group_1 2023-10-06T02:30:33.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:33.245+0000 7fd9b939d700 1 Processor -- start 2023-10-06T02:30:33.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:33.245+0000 7fd9b939d700 1 -- start start 2023-10-06T02:30:33.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:33.246+0000 7fd9b939d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd9b415b1a0 0x7fd9b40c6b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:33.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:33.246+0000 7fd9b939d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd9b4157750 0x7fd9b40ae080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:33.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:33.246+0000 7fd9b939d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9b40c8680 con 0x7fd9b4157380 2023-10-06T02:30:33.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:33.246+0000 7fd9b939d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9b406a7f0 con 0x7fd9b415b1a0 2023-10-06T02:30:33.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:33.246+0000 7fd9b939d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b415b6a0 con 0x7fd9b4157750 2023-10-06T02:30:33.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:33.247+0000 7fd9b3fff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fd9b4157380 0x7fd9b41585f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49666/0 (socket says 172.21.15.31:49666) 2023-10-06T02:30:33.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:33.247+0000 7fd9b3fff700 1 -- 172.21.15.31:0/706984846 learned_addr learned my addr 172.21.15.31:0/706984846 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:30:33.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:33.247+0000 7fd9b8b9c700 1 --2- 172.21.15.31:0/706984846 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd9b4157750 0x7fd9b40ae080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:33.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:33.247+0000 7fd9b37fe700 1 --2- 172.21.15.31:0/706984846 >> v2:172.21.15.62:3300/0 conn(0x7fd9b415b1a0 0x7fd9b40c6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:33.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:33.247+0000 7fd9b2ffd700 1 -- 172.21.15.31:0/706984846 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 836071146 0 0) 0x7fd9b40c8680 con 0x7fd9b4157380 2023-10-06T02:30:33.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:33.247+0000 7fd9b2ffd700 1 -- 172.21.15.31:0/706984846 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd99c003570 con 0x7fd9b4157380 2023-10-06T02:30:33.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:33.247+0000 7fd9b2ffd700 1 -- 172.21.15.31:0/706984846 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3897414211 0 0) 0x7fd99c003570 con 0x7fd9b4157380 2023-10-06T02:30:33.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:33.247+0000 7fd9b2ffd700 1 -- 172.21.15.31:0/706984846 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd9b4157750 msgr2=0x7fd9b40ae080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:33.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:33.247+0000 7fd9b2ffd700 1 --2- 172.21.15.31:0/706984846 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd9b4157750 0x7fd9b40ae080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:33.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:33.247+0000 7fd9b2ffd700 1 -- 172.21.15.31:0/706984846 >> v2:172.21.15.62:3300/0 conn(0x7fd9b415b1a0 msgr2=0x7fd9b40c6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:33.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:33.247+0000 7fd9b2ffd700 1 --2- 172.21.15.31:0/706984846 >> v2:172.21.15.62:3300/0 conn(0x7fd9b415b1a0 0x7fd9b40c6b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:33.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:33.247+0000 7fd9b2ffd700 1 -- 172.21.15.31:0/706984846 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9b415b980 con 0x7fd9b4157380 2023-10-06T02:30:33.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:33.247+0000 7fd9b2ffd700 1 -- 172.21.15.31:0/706984846 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7fd9a4001470 con 0x7fd9b4157380 2023-10-06T02:30:33.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:33.247+0000 7fd9b2ffd700 1 -- 172.21.15.31:0/706984846 >> v1:172.21.15.31:6789/0 conn(0x7fd9b4157380 legacy=0x7fd9b41585f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:33.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:33.248+0000 7fd9b2ffd700 1 --2- 172.21.15.31:0/706984846 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd9b4157750 0x7fd99c003ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:33.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:33.248+0000 7fd9b2ffd700 1 --2- 172.21.15.31:0/706984846 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd99c0040b0 0x7fd99c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:33.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:33.248+0000 7fd9b2ffd700 1 --2- 172.21.15.31:0/706984846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd99c004d50 0x7fd99c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:33.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:33.248+0000 7fd9b2ffd700 1 -- 172.21.15.31:0/706984846 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b406a7f0 con 0x7fd99c0040b0 2023-10-06T02:30:33.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:33.248+0000 7fd9b2ffd700 1 -- 172.21.15.31:0/706984846 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b415b6a0 con 0x7fd99c004d50 2023-10-06T02:30:33.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:33.248+0000 7fd9b2ffd700 1 -- 172.21.15.31:0/706984846 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd99c00bb60 con 0x7fd9b4157750 2023-10-06T02:30:33.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:33.248+0000 7fd9b37fe700 1 --2- 172.21.15.31:0/706984846 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd99c0040b0 0x7fd99c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:33.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:33.248+0000 7fd9b3fff700 1 --2- 172.21.15.31:0/706984846 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd9b4157750 0x7fd99c003ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:33.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:33.248+0000 7fd9b8b9c700 1 --2- 172.21.15.31:0/706984846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd99c004d50 0x7fd99c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:33.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:33.248+0000 7fd9b37fe700 1 -- 172.21.15.31:0/706984846 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd9b4157750 msgr2=0x7fd99c003ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:33.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:33.248+0000 7fd9b37fe700 1 --2- 172.21.15.31:0/706984846 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd9b4157750 0x7fd99c003ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:33.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:33.248+0000 7fd9b37fe700 1 -- 172.21.15.31:0/706984846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd99c004d50 msgr2=0x7fd99c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:33.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:33.248+0000 7fd9b37fe700 1 --2- 172.21.15.31:0/706984846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd99c004d50 0x7fd99c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:33.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:33.248+0000 7fd9b37fe700 1 -- 172.21.15.31:0/706984846 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd99c003570 con 0x7fd99c0040b0 2023-10-06T02:30:33.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:33.248+0000 7fd9b8b9c700 1 --2- 172.21.15.31:0/706984846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd99c004d50 0x7fd99c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:30:33.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:33.249+0000 7fd9b37fe700 1 --2- 172.21.15.31:0/706984846 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd99c0040b0 0x7fd99c004810 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fd9a8006f00 tx=0x7fd9a80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84274b9fcf10cdb4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:33.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:33.249+0000 7fd9b2ffd700 1 -- 172.21.15.31:0/706984846 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a800e710 con 0x7fd99c0040b0 2023-10-06T02:30:33.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:33.249+0000 7fd9b2ffd700 1 -- 172.21.15.31:0/706984846 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9a800e870 con 0x7fd99c0040b0 2023-10-06T02:30:33.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:33.249+0000 7fd9b2ffd700 1 -- 172.21.15.31:0/706984846 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a800ec30 con 0x7fd99c0040b0 2023-10-06T02:30:33.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:33.249+0000 7fd9b939d700 1 -- 172.21.15.31:0/706984846 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd99c0040b0 msgr2=0x7fd99c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:33.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:33.249+0000 7fd9b939d700 1 --2- 172.21.15.31:0/706984846 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd99c0040b0 0x7fd99c004810 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fd9a8006f00 tx=0x7fd9a80091a0 comp rx=0 tx=0).stop 2023-10-06T02:30:33.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:33.249+0000 7fd9b8b9c700 1 -- 172.21.15.31:0/706984846 reap_dead start 2023-10-06T02:30:33.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:33.249+0000 7fd9b939d700 1 -- 172.21.15.31:0/706984846 shutdown_connections 2023-10-06T02:30:33.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:33.249+0000 7fd9b939d700 1 -- 172.21.15.31:0/706984846 >> 172.21.15.31:0/706984846 conn(0x7fd9b406c6a0 msgr2=0x7fd9b406caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:33.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:33.249+0000 7fd9b939d700 1 -- 172.21.15.31:0/706984846 shutdown_connections 2023-10-06T02:30:33.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:33.249+0000 7fd9b939d700 1 -- 172.21.15.31:0/706984846 wait complete. 2023-10-06T02:30:33.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:33.251+0000 7fd9b939d700 1 Processor -- start 2023-10-06T02:30:33.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:33.251+0000 7fd9b939d700 1 -- start start 2023-10-06T02:30:33.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:33.252+0000 7fd9b939d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd99c0040b0 0x7fd9b413ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:33.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:33.252+0000 7fd9b939d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9b416d910 0x7fd9b40cc840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:33.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:33.252+0000 7fd9b939d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd9b4151bc0 0x7fd9b4152060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:33.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:33.252+0000 7fd9b939d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b41525a0 con 0x7fd9b416d910 2023-10-06T02:30:33.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:33.252+0000 7fd9b939d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b416dda0 con 0x7fd99c0040b0 2023-10-06T02:30:33.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:33.252+0000 7fd9b939d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b416e080 con 0x7fd9b4151bc0 2023-10-06T02:30:33.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:33.252+0000 7fd9b37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9b416d910 0x7fd9b40cc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:33.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:33.252+0000 7fd9b37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9b416d910 0x7fd9b40cc840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47096/0 (socket says 172.21.15.31:47096) 2023-10-06T02:30:33.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:33.252+0000 7fd9b37fe700 1 -- 172.21.15.31:0/3849971599 learned_addr learned my addr 172.21.15.31:0/3849971599 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:33.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:33.252+0000 7fd9b3fff700 1 --2- 172.21.15.31:0/3849971599 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd99c0040b0 0x7fd9b413ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:33.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:33.252+0000 7fd9b8b9c700 1 --2- 172.21.15.31:0/3849971599 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd9b4151bc0 0x7fd9b4152060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:33.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:33.253+0000 7fd9b37fe700 1 -- 172.21.15.31:0/3849971599 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd9b4151bc0 msgr2=0x7fd9b4152060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:33.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:33.253+0000 7fd9b37fe700 1 --2- 172.21.15.31:0/3849971599 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd9b4151bc0 0x7fd9b4152060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:33.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:33.253+0000 7fd9b37fe700 1 -- 172.21.15.31:0/3849971599 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd99c0040b0 msgr2=0x7fd9b413ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:33.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:33.253+0000 7fd9b37fe700 1 --2- 172.21.15.31:0/3849971599 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd99c0040b0 0x7fd9b413ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:33.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:33.253+0000 7fd9b37fe700 1 -- 172.21.15.31:0/3849971599 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9b416e360 con 0x7fd9b416d910 2023-10-06T02:30:33.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:33.253+0000 7fd9b37fe700 1 --2- 172.21.15.31:0/3849971599 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9b416d910 0x7fd9b40cc840 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fd99c004760 tx=0x7fd9a801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:33.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:33.253+0000 7fd99bfff700 1 -- 172.21.15.31:0/3849971599 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a8022070 con 0x7fd9b416d910 2023-10-06T02:30:33.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:33.253+0000 7fd99bfff700 1 -- 172.21.15.31:0/3849971599 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9a800e9e0 con 0x7fd9b416d910 2023-10-06T02:30:33.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:33.253+0000 7fd99bfff700 1 -- 172.21.15.31:0/3849971599 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a800e710 con 0x7fd9b416d910 2023-10-06T02:30:33.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:33.253+0000 7fd9b939d700 1 -- 172.21.15.31:0/3849971599 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9b416e640 con 0x7fd9b416d910 2023-10-06T02:30:33.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:33.253+0000 7fd9b939d700 1 -- 172.21.15.31:0/3849971599 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9b40ccd80 con 0x7fd9b416d910 2023-10-06T02:30:33.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:33.254+0000 7fd9b939d700 1 -- 172.21.15.31:0/3849971599 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9b40ccf70 con 0x7fd9b416d910 2023-10-06T02:30:33.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:33.255+0000 7fd99bfff700 1 -- 172.21.15.31:0/3849971599 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7fd9a801c660 con 0x7fd9b416d910 2023-10-06T02:30:33.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:33.256+0000 7fd99bfff700 1 --2- 172.21.15.31:0/3849971599 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd9a00674d0 0x7fd9a0069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:33.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:33.257+0000 7fd99bfff700 1 -- 172.21.15.31:0/3849971599 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fd9a80867d0 con 0x7fd9b416d910 2023-10-06T02:30:33.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:33.257+0000 7fd9b3fff700 1 --2- 172.21.15.31:0/3849971599 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd9a00674d0 0x7fd9a0069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:33.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:33.258+0000 7fd9b3fff700 1 --2- 172.21.15.31:0/3849971599 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd9a00674d0 0x7fd9a0069980 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4001660 tx=0x7fd9a4001dc0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:33.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:33.261+0000 7fd99bfff700 1 -- 172.21.15.31:0/3849971599 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd9b40ccf70 con 0x7fd9b416d910 2023-10-06T02:30:33.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:33.495+0000 7fd9b939d700 1 -- 172.21.15.31:0/3849971599 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup snapshot ls", "vol_name": "non_existent_subvolume", "group_name": "group_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fd9b416e1f0 con 0x7fd9a00674d0 2023-10-06T02:30:33.498 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:30:33.497+0000 7f656537d700 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2023-10-06T02:30:33.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:33.497+0000 7fd99bfff700 1 -- 172.21.15.31:0/3849971599 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) v1 ==== 45+0+0 (secure 0 0 0) 0x7fd9b416e1f0 con 0x7fd9a00674d0 2023-10-06T02:30:33.500 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2023-10-06T02:30:33.508 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-06T02:30:33.508 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T02:30:33.508 DEBUG:teuthology.orchestra.run.smithi031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T02:30:33.587 DEBUG:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T02:30:33.686 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T02:30:33.686 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T02:30:33.686 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:30:33.686 DEBUG:teuthology.orchestra.run.smithi031:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T02:30:33.767 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi031.front.sepia.ceph.com 2023-10-06T02:30:33.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:30:33.768 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T02:30:33.805 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-06T02:30:33.805 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:30:33.805 DEBUG:teuthology.orchestra.run.smithi062:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-06T02:30:33.902 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi062.front.sepia.ceph.com 2023-10-06T02:30:33.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:30:33.902 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T02:30:33.940 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:30:34.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.128+0000 7f52f2b9d700 1 Processor -- start 2023-10-06T02:30:34.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.128+0000 7f52f2b9d700 1 -- start start 2023-10-06T02:30:34.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.129+0000 7f52f2b9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f52ec153c30 0x7f52ec15a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:34.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.129+0000 7f52f2b9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f52ec15de50 0x7f52ec0cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:34.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.129+0000 7f52f2b9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52ec15afd0 con 0x7f52ec0cef40 2023-10-06T02:30:34.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.129+0000 7f52f2b9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52ec15b2b0 con 0x7f52ec15de50 2023-10-06T02:30:34.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.129+0000 7f52f2b9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52ec15b590 con 0x7f52ec153c30 2023-10-06T02:30:34.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.130+0000 7f52f239c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f52ec0cef40 0x7f52ec06d790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49682/0 (socket says 172.21.15.31:49682) 2023-10-06T02:30:34.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.130+0000 7f52f239c700 1 -- 172.21.15.31:0/2591175867 learned_addr learned my addr 172.21.15.31:0/2591175867 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:30:34.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.130+0000 7f52f139a700 1 --2- 172.21.15.31:0/2591175867 >> v2:172.21.15.31:3300/0 conn(0x7f52ec15de50 0x7f52ec0cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:34.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.130+0000 7f52f1b9b700 1 --2- 172.21.15.31:0/2591175867 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f52ec153c30 0x7f52ec15a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:34.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.130+0000 7f52f0b99700 1 -- 172.21.15.31:0/2591175867 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 198021350 0 0) 0x7f52ec15afd0 con 0x7f52ec0cef40 2023-10-06T02:30:34.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.130+0000 7f52f0b99700 1 -- 172.21.15.31:0/2591175867 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52d8003570 con 0x7f52ec0cef40 2023-10-06T02:30:34.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.130+0000 7f52f139a700 1 -- 172.21.15.31:0/2591175867 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f52ec153c30 msgr2=0x7f52ec15a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:34.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.130+0000 7f52f139a700 1 --2- 172.21.15.31:0/2591175867 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f52ec153c30 0x7f52ec15a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:34.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.130+0000 7f52f139a700 1 -- 172.21.15.31:0/2591175867 >> v1:172.21.15.31:6789/0 conn(0x7f52ec0cef40 legacy=0x7f52ec06d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:34.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.130+0000 7f52f139a700 1 -- 172.21.15.31:0/2591175867 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52ec15b870 con 0x7f52ec15de50 2023-10-06T02:30:34.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.130+0000 7f52f139a700 1 --2- 172.21.15.31:0/2591175867 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52ec15de50 0x7f52ec0cea00 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f52dc0098e0 tx=0x7f52dc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c16c2d90c4997429 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:34.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.131+0000 7f52f0b99700 1 -- 172.21.15.31:0/2591175867 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52dc011630 con 0x7f52ec15de50 2023-10-06T02:30:34.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.131+0000 7f52f0b99700 1 -- 172.21.15.31:0/2591175867 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52dc011790 con 0x7f52ec15de50 2023-10-06T02:30:34.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.131+0000 7f52f0b99700 1 -- 172.21.15.31:0/2591175867 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52dc011b30 con 0x7f52ec15de50 2023-10-06T02:30:34.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.131+0000 7f52f2b9d700 1 -- 172.21.15.31:0/2591175867 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52ec15de50 msgr2=0x7f52ec0cea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:34.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.131+0000 7f52f2b9d700 1 --2- 172.21.15.31:0/2591175867 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52ec15de50 0x7f52ec0cea00 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f52dc0098e0 tx=0x7f52dc0091a0 comp rx=0 tx=0).stop 2023-10-06T02:30:34.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.131+0000 7f52f2b9d700 1 -- 172.21.15.31:0/2591175867 shutdown_connections 2023-10-06T02:30:34.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.131+0000 7f52f2b9d700 1 --2- 172.21.15.31:0/2591175867 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f52ec153c30 0x7f52ec15a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:34.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.131+0000 7f52f2b9d700 1 --2- 172.21.15.31:0/2591175867 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52ec15de50 0x7f52ec0cea00 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:34.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.131+0000 7f52f2b9d700 1 -- 172.21.15.31:0/2591175867 >> 172.21.15.31:0/2591175867 conn(0x7f52ec0703a0 msgr2=0x7f52ec0707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:34.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.131+0000 7f52f2b9d700 1 -- 172.21.15.31:0/2591175867 shutdown_connections 2023-10-06T02:30:34.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.131+0000 7f52f2b9d700 1 -- 172.21.15.31:0/2591175867 wait complete. 2023-10-06T02:30:34.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.133+0000 7f52f2b9d700 1 Processor -- start 2023-10-06T02:30:34.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.133+0000 7f52f2b9d700 1 -- start start 2023-10-06T02:30:34.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.134+0000 7f52f2b9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f52ec0cef40 0x7f52ec16d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:34.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.134+0000 7f52f2b9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52ec153c30 0x7f52ec1523a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:34.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.134+0000 7f52f2b9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52ec15de50 0x7f52ec0cc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:34.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.134+0000 7f52f2b9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52ec16de70 con 0x7f52ec153c30 2023-10-06T02:30:34.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.134+0000 7f52f2b9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52ec0c85b0 con 0x7f52ec15de50 2023-10-06T02:30:34.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.134+0000 7f52f2b9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52ec1528e0 con 0x7f52ec0cef40 2023-10-06T02:30:34.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.134+0000 7f52f139a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52ec153c30 0x7f52ec1523a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:34.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.134+0000 7f52f1b9b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f52ec0cef40 0x7f52ec16d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:34.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.134+0000 7f52f139a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52ec153c30 0x7f52ec1523a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47126/0 (socket says 172.21.15.31:47126) 2023-10-06T02:30:34.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.134+0000 7f52f139a700 1 -- 172.21.15.31:0/2727422467 learned_addr learned my addr 172.21.15.31:0/2727422467 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:34.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.134+0000 7f52f239c700 1 --2- 172.21.15.31:0/2727422467 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52ec15de50 0x7f52ec0cc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:34.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.134+0000 7f52f139a700 1 -- 172.21.15.31:0/2727422467 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f52ec0cef40 msgr2=0x7f52ec16d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:34.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.134+0000 7f52f139a700 1 --2- 172.21.15.31:0/2727422467 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f52ec0cef40 0x7f52ec16d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:34.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.134+0000 7f52f139a700 1 -- 172.21.15.31:0/2727422467 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52ec15de50 msgr2=0x7f52ec0cc5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:34.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.134+0000 7f52f139a700 1 --2- 172.21.15.31:0/2727422467 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52ec15de50 0x7f52ec0cc5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:34.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.134+0000 7f52f139a700 1 -- 172.21.15.31:0/2727422467 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52dc009420 con 0x7f52ec153c30 2023-10-06T02:30:34.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.135+0000 7f52f239c700 1 --2- 172.21.15.31:0/2727422467 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52ec15de50 0x7f52ec0cc5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:30:34.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.135+0000 7f52f139a700 1 --2- 172.21.15.31:0/2727422467 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52ec153c30 0x7f52ec1523a0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f52ec06eb00 tx=0x7f52dc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:34.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.137+0000 7f52e1ffb700 1 -- 172.21.15.31:0/2727422467 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52dc011900 con 0x7f52ec153c30 2023-10-06T02:30:34.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.137+0000 7f52e1ffb700 1 -- 172.21.15.31:0/2727422467 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52dc011a80 con 0x7f52ec153c30 2023-10-06T02:30:34.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.137+0000 7f52e1ffb700 1 -- 172.21.15.31:0/2727422467 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52dc025670 con 0x7f52ec153c30 2023-10-06T02:30:34.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.137+0000 7f52f2b9d700 1 -- 172.21.15.31:0/2727422467 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52ec0c79a0 con 0x7f52ec153c30 2023-10-06T02:30:34.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.137+0000 7f52f2b9d700 1 -- 172.21.15.31:0/2727422467 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52ec0c7f60 con 0x7f52ec153c30 2023-10-06T02:30:34.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.138+0000 7f52f2b9d700 1 -- 172.21.15.31:0/2727422467 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52ec04c3c0 con 0x7f52ec153c30 2023-10-06T02:30:34.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.139+0000 7f52e1ffb700 1 -- 172.21.15.31:0/2727422467 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f52dc023070 con 0x7f52ec153c30 2023-10-06T02:30:34.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.140+0000 7f52e1ffb700 1 --2- 172.21.15.31:0/2727422467 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f52d00674d0 0x7f52d0069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:34.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.140+0000 7f52e1ffb700 1 -- 172.21.15.31:0/2727422467 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f52dc086580 con 0x7f52ec153c30 2023-10-06T02:30:34.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.140+0000 7f52f1b9b700 1 --2- 172.21.15.31:0/2727422467 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f52d00674d0 0x7f52d0069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:34.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.141+0000 7f52f1b9b700 1 --2- 172.21.15.31:0/2727422467 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f52d00674d0 0x7f52d0069980 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f52ec0c5ea0 tx=0x7f52e80091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:34.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.144+0000 7f52e1ffb700 1 -- 172.21.15.31:0/2727422467 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f52dc052dc0 con 0x7f52ec153c30 2023-10-06T02:30:34.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.391+0000 7f52f2b9d700 1 -- 172.21.15.31:0/2727422467 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f52ec0c86f0 con 0x7f52ec153c30 2023-10-06T02:30:34.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.392+0000 7f52e1ffb700 1 -- 172.21.15.31:0/2727422467 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4601 (secure 0 0 0) 0x7f52dc0563e0 con 0x7f52ec153c30 2023-10-06T02:30:34.393 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:30:34.393 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":91,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6611,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/1574790962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1574790962},{"type":"v1","addr":"172.21.15.62:6834","nonce":1574790962}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":85},{"gid":7591,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/2840240410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":2840240410},{"type":"v1","addr":"172.21.15.31:6835","nonce":2840240410}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":85},{"gid":7597,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2500071914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2500071914},{"type":"v1","addr":"172.21.15.62:6835","nonce":2500071914}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:30:11.625173+0000","modified":"2023-10-06T02:30:15.149386+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7971},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7971":{"gid":7971,"name":"a","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6834/1197111339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1197111339},{"type":"v1","addr":"172.21.15.31:6834","nonce":1197111339}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-10-06T02:30:34.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.394+0000 7f52f2b9d700 1 -- 172.21.15.31:0/2727422467 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f52d00674d0 msgr2=0x7f52d0069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:34.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.394+0000 7f52f2b9d700 1 --2- 172.21.15.31:0/2727422467 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f52d00674d0 0x7f52d0069980 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f52ec0c5ea0 tx=0x7f52e80091a0 comp rx=0 tx=0).stop 2023-10-06T02:30:34.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.394+0000 7f52f2b9d700 1 -- 172.21.15.31:0/2727422467 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52ec153c30 msgr2=0x7f52ec1523a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:34.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.394+0000 7f52f2b9d700 1 --2- 172.21.15.31:0/2727422467 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52ec153c30 0x7f52ec1523a0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f52ec06eb00 tx=0x7f52dc017000 comp rx=0 tx=0).stop 2023-10-06T02:30:34.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.394+0000 7f52f2b9d700 1 -- 172.21.15.31:0/2727422467 shutdown_connections 2023-10-06T02:30:34.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.394+0000 7f52f2b9d700 1 --2- 172.21.15.31:0/2727422467 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52ec15de50 0x7f52ec0cc5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:34.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.394+0000 7f52f2b9d700 1 --2- 172.21.15.31:0/2727422467 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f52d00674d0 0x7f52d0069980 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:34.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.394+0000 7f52f2b9d700 1 --2- 172.21.15.31:0/2727422467 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f52ec0cef40 0x7f52ec16d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:34.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.394+0000 7f52f2b9d700 1 --2- 172.21.15.31:0/2727422467 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52ec153c30 0x7f52ec1523a0 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:34.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.394+0000 7f52f2b9d700 1 -- 172.21.15.31:0/2727422467 >> 172.21.15.31:0/2727422467 conn(0x7f52ec0703a0 msgr2=0x7f52ec0c52d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:34.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.394+0000 7f52f2b9d700 1 -- 172.21.15.31:0/2727422467 shutdown_connections 2023-10-06T02:30:34.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.394+0000 7f52f2b9d700 1 -- 172.21.15.31:0/2727422467 wait complete. 2023-10-06T02:30:34.397 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 91 2023-10-06T02:30:34.413 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:30:34.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.598+0000 7efe5319d700 1 Processor -- start 2023-10-06T02:30:34.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.598+0000 7efe5319d700 1 -- start start 2023-10-06T02:30:34.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.599+0000 7efe5319d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efe4c15de50 0x7efe4c0cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:34.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.599+0000 7efe5319d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7efe4c0cef40 0x7efe4c15c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:34.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.599+0000 7efe5319d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe4c15afd0 con 0x7efe4c153c30 2023-10-06T02:30:34.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.599+0000 7efe5319d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe4c15b2b0 con 0x7efe4c0cef40 2023-10-06T02:30:34.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.599+0000 7efe5319d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe4c15b590 con 0x7efe4c15de50 2023-10-06T02:30:34.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.599+0000 7efe5219b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7efe4c153c30 0x7efe4c15a5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49694/0 (socket says 172.21.15.31:49694) 2023-10-06T02:30:34.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.600+0000 7efe5219b700 1 -- 172.21.15.31:0/1059134131 learned_addr learned my addr 172.21.15.31:0/1059134131 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:30:34.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.600+0000 7efe5299c700 1 --2- 172.21.15.31:0/1059134131 >> v2:172.21.15.31:3300/0 conn(0x7efe4c0cef40 0x7efe4c15c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:34.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.600+0000 7efe5199a700 1 --2- 172.21.15.31:0/1059134131 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efe4c15de50 0x7efe4c0cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:34.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.600+0000 7efe51199700 1 -- 172.21.15.31:0/1059134131 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4142559474 0 0) 0x7efe4c15afd0 con 0x7efe4c153c30 2023-10-06T02:30:34.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.600+0000 7efe51199700 1 -- 172.21.15.31:0/1059134131 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe38003570 con 0x7efe4c153c30 2023-10-06T02:30:34.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.600+0000 7efe5299c700 1 -- 172.21.15.31:0/1059134131 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efe4c15de50 msgr2=0x7efe4c0cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:34.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.600+0000 7efe5299c700 1 --2- 172.21.15.31:0/1059134131 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efe4c15de50 0x7efe4c0cea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:34.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.600+0000 7efe5299c700 1 -- 172.21.15.31:0/1059134131 >> v1:172.21.15.31:6789/0 conn(0x7efe4c153c30 legacy=0x7efe4c15a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:34.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.600+0000 7efe5299c700 1 -- 172.21.15.31:0/1059134131 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe4c15b870 con 0x7efe4c0cef40 2023-10-06T02:30:34.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.600+0000 7efe5299c700 1 --2- 172.21.15.31:0/1059134131 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe4c0cef40 0x7efe4c15c070 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7efe44009e60 tx=0x7efe44009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a1820c603bf79b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:34.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.601+0000 7efe51199700 1 -- 172.21.15.31:0/1059134131 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe4400e070 con 0x7efe4c0cef40 2023-10-06T02:30:34.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.601+0000 7efe51199700 1 -- 172.21.15.31:0/1059134131 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe4400bb60 con 0x7efe4c0cef40 2023-10-06T02:30:34.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.601+0000 7efe51199700 1 -- 172.21.15.31:0/1059134131 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe44012430 con 0x7efe4c0cef40 2023-10-06T02:30:34.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.601+0000 7efe5319d700 1 -- 172.21.15.31:0/1059134131 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe4c0cef40 msgr2=0x7efe4c15c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:34.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.601+0000 7efe5319d700 1 --2- 172.21.15.31:0/1059134131 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe4c0cef40 0x7efe4c15c070 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7efe44009e60 tx=0x7efe44009780 comp rx=0 tx=0).stop 2023-10-06T02:30:34.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.601+0000 7efe5319d700 1 -- 172.21.15.31:0/1059134131 shutdown_connections 2023-10-06T02:30:34.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.601+0000 7efe5319d700 1 --2- 172.21.15.31:0/1059134131 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe4c0cef40 0x7efe4c15c070 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:34.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.601+0000 7efe5319d700 1 --2- 172.21.15.31:0/1059134131 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efe4c15de50 0x7efe4c0cea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:34.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.601+0000 7efe5319d700 1 -- 172.21.15.31:0/1059134131 >> 172.21.15.31:0/1059134131 conn(0x7efe4c0703a0 msgr2=0x7efe4c0707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:34.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.601+0000 7efe5319d700 1 -- 172.21.15.31:0/1059134131 shutdown_connections 2023-10-06T02:30:34.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.601+0000 7efe5319d700 1 -- 172.21.15.31:0/1059134131 wait complete. 2023-10-06T02:30:34.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.603+0000 7efe5319d700 1 Processor -- start 2023-10-06T02:30:34.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.603+0000 7efe5319d700 1 -- start start 2023-10-06T02:30:34.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.604+0000 7efe5319d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efe4c0cef40 0x7efe4c1519f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:34.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.604+0000 7efe5319d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe4c153c30 0x7efe4c0c8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:34.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.604+0000 7efe5319d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe4c15de50 0x7efe4c169950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:34.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.604+0000 7efe5319d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe4c0b3ff0 con 0x7efe4c153c30 2023-10-06T02:30:34.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.604+0000 7efe5319d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe4c151f30 con 0x7efe4c15de50 2023-10-06T02:30:34.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.604+0000 7efe5319d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe4c0b33e0 con 0x7efe4c0cef40 2023-10-06T02:30:34.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.604+0000 7efe5199a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe4c153c30 0x7efe4c0c8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:34.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.604+0000 7efe5199a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe4c153c30 0x7efe4c0c8c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47142/0 (socket says 172.21.15.31:47142) 2023-10-06T02:30:34.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.604+0000 7efe5199a700 1 -- 172.21.15.31:0/1698231975 learned_addr learned my addr 172.21.15.31:0/1698231975 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:34.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.604+0000 7efe5219b700 1 --2- 172.21.15.31:0/1698231975 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efe4c0cef40 0x7efe4c1519f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:34.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.604+0000 7efe5299c700 1 --2- 172.21.15.31:0/1698231975 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe4c15de50 0x7efe4c169950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:34.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.604+0000 7efe5199a700 1 -- 172.21.15.31:0/1698231975 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efe4c0cef40 msgr2=0x7efe4c1519f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:34.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.604+0000 7efe5199a700 1 --2- 172.21.15.31:0/1698231975 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efe4c0cef40 0x7efe4c1519f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:34.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.604+0000 7efe5199a700 1 -- 172.21.15.31:0/1698231975 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe4c15de50 msgr2=0x7efe4c169950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:34.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.604+0000 7efe5199a700 1 --2- 172.21.15.31:0/1698231975 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe4c15de50 0x7efe4c169950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:34.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.604+0000 7efe5199a700 1 -- 172.21.15.31:0/1698231975 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe44009a20 con 0x7efe4c153c30 2023-10-06T02:30:34.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.604+0000 7efe5299c700 1 --2- 172.21.15.31:0/1698231975 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe4c15de50 0x7efe4c169950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:30:34.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.604+0000 7efe5219b700 1 --2- 172.21.15.31:0/1698231975 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efe4c0cef40 0x7efe4c1519f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:30:34.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.605+0000 7efe5199a700 1 --2- 172.21.15.31:0/1698231975 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe4c153c30 0x7efe4c0c8c50 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7efe3c00d890 tx=0x7efe3c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:34.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.605+0000 7efe427fc700 1 -- 172.21.15.31:0/1698231975 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe3c00b600 con 0x7efe4c153c30 2023-10-06T02:30:34.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.605+0000 7efe427fc700 1 -- 172.21.15.31:0/1698231975 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe3c00b760 con 0x7efe4c153c30 2023-10-06T02:30:34.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.605+0000 7efe5319d700 1 -- 172.21.15.31:0/1698231975 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe4c0b36c0 con 0x7efe4c153c30 2023-10-06T02:30:34.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.605+0000 7efe427fc700 1 -- 172.21.15.31:0/1698231975 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe3c00bae0 con 0x7efe4c153c30 2023-10-06T02:30:34.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.605+0000 7efe5319d700 1 -- 172.21.15.31:0/1698231975 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe4c0b3c80 con 0x7efe4c153c30 2023-10-06T02:30:34.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.606+0000 7efe5319d700 1 -- 172.21.15.31:0/1698231975 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe4c0b4160 con 0x7efe4c153c30 2023-10-06T02:30:34.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.607+0000 7efe427fc700 1 -- 172.21.15.31:0/1698231975 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7efe3c01e460 con 0x7efe4c153c30 2023-10-06T02:30:34.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.608+0000 7efe427fc700 1 --2- 172.21.15.31:0/1698231975 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7efe300674d0 0x7efe30069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:34.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.608+0000 7efe427fc700 1 -- 172.21.15.31:0/1698231975 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6086+0+0 (secure 0 0 0) 0x7efe3c085800 con 0x7efe4c153c30 2023-10-06T02:30:34.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.608+0000 7efe5219b700 1 --2- 172.21.15.31:0/1698231975 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7efe300674d0 0x7efe30069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:34.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.609+0000 7efe5219b700 1 --2- 172.21.15.31:0/1698231975 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7efe300674d0 0x7efe30069980 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7efe48007fd0 tx=0x7efe48009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:34.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.612+0000 7efe427fc700 1 -- 172.21.15.31:0/1698231975 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efe3c052080 con 0x7efe4c153c30 2023-10-06T02:30:34.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.814+0000 7efe5319d700 1 -- 172.21.15.31:0/1698231975 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efe4c153270 con 0x7efe4c153c30 2023-10-06T02:30:34.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.815+0000 7efe427fc700 1 -- 172.21.15.31:0/1698231975 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v132) v1 ==== 74+0+16876 (secure 0 0 0) 0x7efe3c0556a0 con 0x7efe4c153c30 2023-10-06T02:30:34.817 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:30:34.817 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":132,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:30:13.138416+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:30:09.791433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-10-06T02:30:10.601699+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":20,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:30:34.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.817+0000 7efe5319d700 1 -- 172.21.15.31:0/1698231975 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7efe300674d0 msgr2=0x7efe30069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:34.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.817+0000 7efe5319d700 1 --2- 172.21.15.31:0/1698231975 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7efe300674d0 0x7efe30069980 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7efe48007fd0 tx=0x7efe48009040 comp rx=0 tx=0).stop 2023-10-06T02:30:34.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.817+0000 7efe5319d700 1 -- 172.21.15.31:0/1698231975 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe4c153c30 msgr2=0x7efe4c0c8c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:34.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.817+0000 7efe5319d700 1 --2- 172.21.15.31:0/1698231975 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe4c153c30 0x7efe4c0c8c50 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7efe3c00d890 tx=0x7efe3c008040 comp rx=0 tx=0).stop 2023-10-06T02:30:34.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.817+0000 7efe5319d700 1 -- 172.21.15.31:0/1698231975 shutdown_connections 2023-10-06T02:30:34.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.817+0000 7efe5319d700 1 --2- 172.21.15.31:0/1698231975 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe4c15de50 0x7efe4c169950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:34.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.817+0000 7efe5319d700 1 --2- 172.21.15.31:0/1698231975 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7efe300674d0 0x7efe30069980 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:34.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.817+0000 7efe5319d700 1 --2- 172.21.15.31:0/1698231975 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efe4c0cef40 0x7efe4c1519f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:34.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.817+0000 7efe5319d700 1 --2- 172.21.15.31:0/1698231975 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe4c153c30 0x7efe4c0c8c50 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:34.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.817+0000 7efe5319d700 1 -- 172.21.15.31:0/1698231975 >> 172.21.15.31:0/1698231975 conn(0x7efe4c0703a0 msgr2=0x7efe4c0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:34.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.818+0000 7efe5319d700 1 -- 172.21.15.31:0/1698231975 shutdown_connections 2023-10-06T02:30:34.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:34.818+0000 7efe5319d700 1 -- 172.21.15.31:0/1698231975 wait complete. 2023-10-06T02:30:34.837 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T02:22:53.183425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T02:23:00.046967+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 21, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T02:30:09.791433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '131', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 22, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T02:30:10.601699+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '132', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T02:30:34.837 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T02:30:35.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.027+0000 7fca2f39d700 1 Processor -- start 2023-10-06T02:30:35.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.027+0000 7fca2f39d700 1 -- start start 2023-10-06T02:30:35.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.028+0000 7fca2f39d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fca28142c00 0x7fca2806b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:35.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.028+0000 7fca2f39d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca280c7f90 0x7fca2806c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:35.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.028+0000 7fca2f39d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca280ca480 con 0x7fca2806bd20 2023-10-06T02:30:35.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.028+0000 7fca2f39d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca2815e240 con 0x7fca28142c00 2023-10-06T02:30:35.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.028+0000 7fca2f39d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca2806b350 con 0x7fca280c7f90 2023-10-06T02:30:35.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.028+0000 7fca2e39b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fca28142c00 0x7fca2806b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:35.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.028+0000 7fca2db9a700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fca2806bd20 0x7fca280c7880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49698/0 (socket says 172.21.15.31:49698) 2023-10-06T02:30:35.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.028+0000 7fca2db9a700 1 -- 172.21.15.31:0/3967612450 learned_addr learned my addr 172.21.15.31:0/3967612450 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:30:35.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.029+0000 7fca2eb9c700 1 --2- 172.21.15.31:0/3967612450 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca280c7f90 0x7fca2806c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:35.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.029+0000 7fca2e39b700 1 -- 172.21.15.31:0/3967612450 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca280c7f90 msgr2=0x7fca2806c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:35.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.029+0000 7fca2e39b700 1 --2- 172.21.15.31:0/3967612450 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca280c7f90 0x7fca2806c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:35.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.029+0000 7fca2e39b700 1 -- 172.21.15.31:0/3967612450 >> v1:172.21.15.31:6789/0 conn(0x7fca2806bd20 legacy=0x7fca280c7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:35.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.029+0000 7fca2d399700 1 -- 172.21.15.31:0/3967612450 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3510832420 0 0) 0x7fca280ca480 con 0x7fca2806bd20 2023-10-06T02:30:35.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.029+0000 7fca2e39b700 1 -- 172.21.15.31:0/3967612450 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca2815df60 con 0x7fca28142c00 2023-10-06T02:30:35.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.029+0000 7fca2eb9c700 1 --2- 172.21.15.31:0/3967612450 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca280c7f90 0x7fca2806c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:30:35.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.029+0000 7fca2e39b700 1 --2- 172.21.15.31:0/3967612450 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca28142c00 0x7fca2806b7e0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fca240098e0 tx=0x7fca240091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18e1b414b065283f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:35.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.029+0000 7fca2d399700 1 -- 172.21.15.31:0/3967612450 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca24009e80 con 0x7fca28142c00 2023-10-06T02:30:35.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.030+0000 7fca2d399700 1 -- 172.21.15.31:0/3967612450 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca2400b860 con 0x7fca28142c00 2023-10-06T02:30:35.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.030+0000 7fca2d399700 1 -- 172.21.15.31:0/3967612450 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca2400bbe0 con 0x7fca28142c00 2023-10-06T02:30:35.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.030+0000 7fca2f39d700 1 -- 172.21.15.31:0/3967612450 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca28142c00 msgr2=0x7fca2806b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:35.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.030+0000 7fca2f39d700 1 --2- 172.21.15.31:0/3967612450 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca28142c00 0x7fca2806b7e0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fca240098e0 tx=0x7fca240091a0 comp rx=0 tx=0).stop 2023-10-06T02:30:35.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.030+0000 7fca2f39d700 1 -- 172.21.15.31:0/3967612450 shutdown_connections 2023-10-06T02:30:35.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.030+0000 7fca2f39d700 1 --2- 172.21.15.31:0/3967612450 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca28142c00 0x7fca2806b7e0 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:35.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.030+0000 7fca2f39d700 1 --2- 172.21.15.31:0/3967612450 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca280c7f90 0x7fca2806c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:35.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.030+0000 7fca2f39d700 1 -- 172.21.15.31:0/3967612450 >> 172.21.15.31:0/3967612450 conn(0x7fca280675a0 msgr2=0x7fca280679a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:35.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.030+0000 7fca2f39d700 1 -- 172.21.15.31:0/3967612450 shutdown_connections 2023-10-06T02:30:35.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.030+0000 7fca2f39d700 1 -- 172.21.15.31:0/3967612450 wait complete. 2023-10-06T02:30:35.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.032+0000 7fca2f39d700 1 Processor -- start 2023-10-06T02:30:35.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.032+0000 7fca2f39d700 1 -- start start 2023-10-06T02:30:35.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.033+0000 7fca2f39d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca2806bd20 0x7fca2813e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:35.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.033+0000 7fca2f39d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca280c7f90 0x7fca280c42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:35.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.033+0000 7fca2f39d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca28142c00 0x7fca28140ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:35.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.033+0000 7fca2f39d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca280c5980 con 0x7fca280c7f90 2023-10-06T02:30:35.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.033+0000 7fca2f39d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca280c4d70 con 0x7fca2806bd20 2023-10-06T02:30:35.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.033+0000 7fca2f39d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca280c5050 con 0x7fca28142c00 2023-10-06T02:30:35.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.033+0000 7fca2db9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca280c7f90 0x7fca280c42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:35.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.033+0000 7fca2db9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca280c7f90 0x7fca280c42f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47152/0 (socket says 172.21.15.31:47152) 2023-10-06T02:30:35.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.033+0000 7fca2db9a700 1 -- 172.21.15.31:0/3824573904 learned_addr learned my addr 172.21.15.31:0/3824573904 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:35.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.033+0000 7fca2eb9c700 1 --2- 172.21.15.31:0/3824573904 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca28142c00 0x7fca28140ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:35.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.033+0000 7fca2e39b700 1 --2- 172.21.15.31:0/3824573904 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca2806bd20 0x7fca2813e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:35.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.033+0000 7fca2db9a700 1 -- 172.21.15.31:0/3824573904 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca28142c00 msgr2=0x7fca28140ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:35.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.033+0000 7fca2db9a700 1 --2- 172.21.15.31:0/3824573904 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca28142c00 0x7fca28140ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:35.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.033+0000 7fca2db9a700 1 -- 172.21.15.31:0/3824573904 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca2806bd20 msgr2=0x7fca2813e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:35.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.033+0000 7fca2db9a700 1 --2- 172.21.15.31:0/3824573904 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca2806bd20 0x7fca2813e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:35.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.033+0000 7fca2db9a700 1 -- 172.21.15.31:0/3824573904 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca24009420 con 0x7fca280c7f90 2023-10-06T02:30:35.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.033+0000 7fca2e39b700 1 --2- 172.21.15.31:0/3824573904 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca2806bd20 0x7fca2813e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:30:35.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.034+0000 7fca2eb9c700 1 --2- 172.21.15.31:0/3824573904 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca28142c00 0x7fca28140ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:30:35.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.034+0000 7fca2db9a700 1 --2- 172.21.15.31:0/3824573904 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca280c7f90 0x7fca280c42f0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7fca1800c810 tx=0x7fca18007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:35.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.034+0000 7fca1e7fc700 1 -- 172.21.15.31:0/3824573904 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca18005c00 con 0x7fca280c7f90 2023-10-06T02:30:35.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.034+0000 7fca1e7fc700 1 -- 172.21.15.31:0/3824573904 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca18005d60 con 0x7fca280c7f90 2023-10-06T02:30:35.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.034+0000 7fca2f39d700 1 -- 172.21.15.31:0/3824573904 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca280c5330 con 0x7fca280c7f90 2023-10-06T02:30:35.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.034+0000 7fca1e7fc700 1 -- 172.21.15.31:0/3824573904 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca18007320 con 0x7fca280c7f90 2023-10-06T02:30:35.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.034+0000 7fca2f39d700 1 -- 172.21.15.31:0/3824573904 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca280b1ae0 con 0x7fca280c7f90 2023-10-06T02:30:35.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.035+0000 7fca2f39d700 1 -- 172.21.15.31:0/3824573904 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca2804c3c0 con 0x7fca280c7f90 2023-10-06T02:30:35.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.036+0000 7fca1e7fc700 1 -- 172.21.15.31:0/3824573904 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7fca1801b400 con 0x7fca280c7f90 2023-10-06T02:30:35.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.037+0000 7fca1e7fc700 1 --2- 172.21.15.31:0/3824573904 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fca0c0674d0 0x7fca0c069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:35.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.037+0000 7fca1e7fc700 1 -- 172.21.15.31:0/3824573904 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fca18084c20 con 0x7fca280c7f90 2023-10-06T02:30:35.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.038+0000 7fca2e39b700 1 --2- 172.21.15.31:0/3824573904 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fca0c0674d0 0x7fca0c069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:35.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.038+0000 7fca2e39b700 1 --2- 172.21.15.31:0/3824573904 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fca0c0674d0 0x7fca0c069980 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fca24003080 tx=0x7fca24017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:35.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.041+0000 7fca1e7fc700 1 -- 172.21.15.31:0/3824573904 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fca180514e0 con 0x7fca280c7f90 2023-10-06T02:30:35.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.245+0000 7fca2f39d700 1 -- 172.21.15.31:0/3824573904 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fca2806ad00 con 0x7fca280c7f90 2023-10-06T02:30:35.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.246+0000 7fca1e7fc700 1 -- 172.21.15.31:0/3824573904 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v132) v1 ==== 93+0+10 (secure 0 0 0) 0x7fca18054b00 con 0x7fca280c7f90 2023-10-06T02:30:35.248 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-10-06T02:30:35.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.248+0000 7fca2f39d700 1 -- 172.21.15.31:0/3824573904 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fca0c0674d0 msgr2=0x7fca0c069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:35.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.248+0000 7fca2f39d700 1 --2- 172.21.15.31:0/3824573904 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fca0c0674d0 0x7fca0c069980 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fca24003080 tx=0x7fca24017000 comp rx=0 tx=0).stop 2023-10-06T02:30:35.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.248+0000 7fca2f39d700 1 -- 172.21.15.31:0/3824573904 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca280c7f90 msgr2=0x7fca280c42f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:35.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.248+0000 7fca2f39d700 1 --2- 172.21.15.31:0/3824573904 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca280c7f90 0x7fca280c42f0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7fca1800c810 tx=0x7fca18007db0 comp rx=0 tx=0).stop 2023-10-06T02:30:35.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.248+0000 7fca2f39d700 1 -- 172.21.15.31:0/3824573904 shutdown_connections 2023-10-06T02:30:35.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.248+0000 7fca2f39d700 1 --2- 172.21.15.31:0/3824573904 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fca0c0674d0 0x7fca0c069980 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:35.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.248+0000 7fca2f39d700 1 --2- 172.21.15.31:0/3824573904 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca28142c00 0x7fca28140ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:35.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.248+0000 7fca2f39d700 1 --2- 172.21.15.31:0/3824573904 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca2806bd20 0x7fca2813e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:35.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.248+0000 7fca2f39d700 1 --2- 172.21.15.31:0/3824573904 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca280c7f90 0x7fca280c42f0 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:35.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.248+0000 7fca2f39d700 1 -- 172.21.15.31:0/3824573904 >> 172.21.15.31:0/3824573904 conn(0x7fca280675a0 msgr2=0x7fca280c1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:35.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.248+0000 7fca2f39d700 1 -- 172.21.15.31:0/3824573904 shutdown_connections 2023-10-06T02:30:35.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.248+0000 7fca2f39d700 1 -- 172.21.15.31:0/3824573904 wait complete. 2023-10-06T02:30:35.267 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T02:30:35.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.455+0000 7f6fd359d700 1 Processor -- start 2023-10-06T02:30:35.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.455+0000 7f6fd359d700 1 -- start start 2023-10-06T02:30:35.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.457+0000 7f6fd359d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f6fcc142c00 0x7f6fcc06b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:35.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.457+0000 7f6fd359d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6fcc0c7f90 0x7f6fcc06c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:35.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.457+0000 7f6fd359d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fcc0ca480 con 0x7f6fcc06bd20 2023-10-06T02:30:35.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.457+0000 7f6fd359d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fcc15e240 con 0x7f6fcc142c00 2023-10-06T02:30:35.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.457+0000 7f6fd359d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fcc06b350 con 0x7f6fcc0c7f90 2023-10-06T02:30:35.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.457+0000 7f6fd1d9a700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f6fcc06bd20 0x7f6fcc0c7880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49706/0 (socket says 172.21.15.31:49706) 2023-10-06T02:30:35.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.457+0000 7f6fd1d9a700 1 -- 172.21.15.31:0/334684844 learned_addr learned my addr 172.21.15.31:0/334684844 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:30:35.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.457+0000 7f6fd259b700 1 --2- 172.21.15.31:0/334684844 >> v2:172.21.15.62:3300/0 conn(0x7f6fcc142c00 0x7f6fcc06b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:35.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.457+0000 7f6fd2d9c700 1 --2- 172.21.15.31:0/334684844 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6fcc0c7f90 0x7f6fcc06c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:35.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.457+0000 7f6fd1599700 1 -- 172.21.15.31:0/334684844 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3328309324 0 0) 0x7f6fcc0ca480 con 0x7f6fcc06bd20 2023-10-06T02:30:35.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.457+0000 7f6fd1599700 1 -- 172.21.15.31:0/334684844 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fb8003570 con 0x7f6fcc06bd20 2023-10-06T02:30:35.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.458+0000 7f6fd259b700 1 -- 172.21.15.31:0/334684844 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6fcc0c7f90 msgr2=0x7f6fcc06c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:35.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.458+0000 7f6fd259b700 1 --2- 172.21.15.31:0/334684844 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6fcc0c7f90 0x7f6fcc06c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:35.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.458+0000 7f6fd259b700 1 -- 172.21.15.31:0/334684844 >> v1:172.21.15.31:6789/0 conn(0x7f6fcc06bd20 legacy=0x7f6fcc0c7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:35.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.458+0000 7f6fd259b700 1 -- 172.21.15.31:0/334684844 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fcc15df60 con 0x7f6fcc142c00 2023-10-06T02:30:35.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.458+0000 7f6fd259b700 1 --2- 172.21.15.31:0/334684844 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6fcc142c00 0x7f6fcc06b7e0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f6fc80098e0 tx=0x7f6fc80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a9634d8ac77c5a8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:35.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.458+0000 7f6fd1599700 1 -- 172.21.15.31:0/334684844 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc8011630 con 0x7f6fcc142c00 2023-10-06T02:30:35.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.458+0000 7f6fd1599700 1 -- 172.21.15.31:0/334684844 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fc8011790 con 0x7f6fcc142c00 2023-10-06T02:30:35.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.458+0000 7f6fd1599700 1 -- 172.21.15.31:0/334684844 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc8011b30 con 0x7f6fcc142c00 2023-10-06T02:30:35.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.459+0000 7f6fd359d700 1 -- 172.21.15.31:0/334684844 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6fcc142c00 msgr2=0x7f6fcc06b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:35.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.459+0000 7f6fd359d700 1 --2- 172.21.15.31:0/334684844 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6fcc142c00 0x7f6fcc06b7e0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f6fc80098e0 tx=0x7f6fc80091a0 comp rx=0 tx=0).stop 2023-10-06T02:30:35.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.459+0000 7f6fd359d700 1 -- 172.21.15.31:0/334684844 shutdown_connections 2023-10-06T02:30:35.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.459+0000 7f6fd359d700 1 --2- 172.21.15.31:0/334684844 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6fcc142c00 0x7f6fcc06b7e0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:35.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.459+0000 7f6fd359d700 1 --2- 172.21.15.31:0/334684844 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6fcc0c7f90 0x7f6fcc06c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:35.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.459+0000 7f6fd359d700 1 -- 172.21.15.31:0/334684844 >> 172.21.15.31:0/334684844 conn(0x7f6fcc0675a0 msgr2=0x7f6fcc0679a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:35.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.459+0000 7f6fd359d700 1 -- 172.21.15.31:0/334684844 shutdown_connections 2023-10-06T02:30:35.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.459+0000 7f6fd359d700 1 -- 172.21.15.31:0/334684844 wait complete. 2023-10-06T02:30:35.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.460+0000 7f6fd359d700 1 Processor -- start 2023-10-06T02:30:35.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.461+0000 7f6fd359d700 1 -- start start 2023-10-06T02:30:35.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.461+0000 7f6fd359d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fcc06bd20 0x7f6fcc0c6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:35.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.461+0000 7f6fd359d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6fcc0c7f90 0x7f6fcc0c6d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:35.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.461+0000 7f6fd359d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6fcc142c00 0x7f6fcc0c42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:35.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.461+0000 7f6fd359d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fcc0c5c30 con 0x7f6fcc06bd20 2023-10-06T02:30:35.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.461+0000 7f6fd359d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fcc0c5020 con 0x7f6fcc142c00 2023-10-06T02:30:35.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.462+0000 7f6fd359d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fcc0c5300 con 0x7f6fcc0c7f90 2023-10-06T02:30:35.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.462+0000 7f6fd259b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fcc06bd20 0x7f6fcc0c6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:35.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.462+0000 7f6fd259b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fcc06bd20 0x7f6fcc0c6810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47160/0 (socket says 172.21.15.31:47160) 2023-10-06T02:30:35.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.462+0000 7f6fd259b700 1 -- 172.21.15.31:0/888908869 learned_addr learned my addr 172.21.15.31:0/888908869 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:35.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.462+0000 7f6fd2d9c700 1 --2- 172.21.15.31:0/888908869 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6fcc142c00 0x7f6fcc0c42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:35.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.462+0000 7f6fd1d9a700 1 --2- 172.21.15.31:0/888908869 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6fcc0c7f90 0x7f6fcc0c6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:35.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.462+0000 7f6fd259b700 1 -- 172.21.15.31:0/888908869 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6fcc0c7f90 msgr2=0x7f6fcc0c6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:35.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.462+0000 7f6fd259b700 1 --2- 172.21.15.31:0/888908869 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6fcc0c7f90 0x7f6fcc0c6d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:35.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.462+0000 7f6fd259b700 1 -- 172.21.15.31:0/888908869 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6fcc142c00 msgr2=0x7f6fcc0c42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:35.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.462+0000 7f6fd259b700 1 --2- 172.21.15.31:0/888908869 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6fcc142c00 0x7f6fcc0c42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:35.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.462+0000 7f6fd259b700 1 -- 172.21.15.31:0/888908869 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fc8009420 con 0x7f6fcc06bd20 2023-10-06T02:30:35.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.462+0000 7f6fd1d9a700 1 --2- 172.21.15.31:0/888908869 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6fcc0c7f90 0x7f6fcc0c6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:30:35.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.463+0000 7f6fd259b700 1 --2- 172.21.15.31:0/888908869 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fcc06bd20 0x7f6fcc0c6810 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc06f290 tx=0x7f6fc8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:35.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.463+0000 7f6fc27fc700 1 -- 172.21.15.31:0/888908869 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc8011b10 con 0x7f6fcc06bd20 2023-10-06T02:30:35.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.463+0000 7f6fc27fc700 1 -- 172.21.15.31:0/888908869 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fc8000cc0 con 0x7f6fcc06bd20 2023-10-06T02:30:35.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.463+0000 7f6fc27fc700 1 -- 172.21.15.31:0/888908869 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc8025620 con 0x7f6fcc06bd20 2023-10-06T02:30:35.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.463+0000 7f6fd359d700 1 -- 172.21.15.31:0/888908869 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fcc0c55e0 con 0x7f6fcc06bd20 2023-10-06T02:30:35.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.463+0000 7f6fd359d700 1 -- 172.21.15.31:0/888908869 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fcc159180 con 0x7f6fcc06bd20 2023-10-06T02:30:35.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.464+0000 7f6fd359d700 1 -- 172.21.15.31:0/888908869 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fcc04c3c0 con 0x7f6fcc06bd20 2023-10-06T02:30:35.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.465+0000 7f6fc27fc700 1 -- 172.21.15.31:0/888908869 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f6fc8025a10 con 0x7f6fcc06bd20 2023-10-06T02:30:35.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.466+0000 7f6fc27fc700 1 --2- 172.21.15.31:0/888908869 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6fb00674d0 0x7f6fb0069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:35.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.466+0000 7f6fc27fc700 1 -- 172.21.15.31:0/888908869 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f6fc80867d0 con 0x7f6fcc06bd20 2023-10-06T02:30:35.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.466+0000 7f6fd1d9a700 1 --2- 172.21.15.31:0/888908869 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6fb00674d0 0x7f6fb0069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:35.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.467+0000 7f6fd1d9a700 1 --2- 172.21.15.31:0/888908869 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6fb00674d0 0x7f6fb0069980 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc0cd950 tx=0x7f6fbc009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:35.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.470+0000 7f6fc27fc700 1 -- 172.21.15.31:0/888908869 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6fc80530d0 con 0x7f6fcc06bd20 2023-10-06T02:30:35.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.671+0000 7f6fd359d700 1 -- 172.21.15.31:0/888908869 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f6fcc0c75f0 con 0x7f6fcc06bd20 2023-10-06T02:30:35.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.671+0000 7f6fc27fc700 1 -- 172.21.15.31:0/888908869 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v132) v1 ==== 92+0+10 (secure 0 0 0) 0x7f6fc80566f0 con 0x7f6fcc06bd20 2023-10-06T02:30:35.673 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 8 2023-10-06T02:30:35.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.674+0000 7f6fd359d700 1 -- 172.21.15.31:0/888908869 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6fb00674d0 msgr2=0x7f6fb0069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:35.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.674+0000 7f6fd359d700 1 --2- 172.21.15.31:0/888908869 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6fb00674d0 0x7f6fb0069980 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc0cd950 tx=0x7f6fbc009040 comp rx=0 tx=0).stop 2023-10-06T02:30:35.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.674+0000 7f6fd359d700 1 -- 172.21.15.31:0/888908869 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fcc06bd20 msgr2=0x7f6fcc0c6810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:35.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.674+0000 7f6fd359d700 1 --2- 172.21.15.31:0/888908869 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fcc06bd20 0x7f6fcc0c6810 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc06f290 tx=0x7f6fc8017000 comp rx=0 tx=0).stop 2023-10-06T02:30:35.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.674+0000 7f6fd359d700 1 -- 172.21.15.31:0/888908869 shutdown_connections 2023-10-06T02:30:35.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.674+0000 7f6fd359d700 1 --2- 172.21.15.31:0/888908869 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6fcc142c00 0x7f6fcc0c42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:35.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.674+0000 7f6fd359d700 1 --2- 172.21.15.31:0/888908869 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fcc06bd20 0x7f6fcc0c6810 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:35.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.674+0000 7f6fd359d700 1 --2- 172.21.15.31:0/888908869 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6fb00674d0 0x7f6fb0069980 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:35.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.674+0000 7f6fd359d700 1 --2- 172.21.15.31:0/888908869 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6fcc0c7f90 0x7f6fcc0c6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:35.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.674+0000 7f6fd359d700 1 -- 172.21.15.31:0/888908869 >> 172.21.15.31:0/888908869 conn(0x7f6fcc0675a0 msgr2=0x7f6fcc0cd010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:35.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.674+0000 7f6fd359d700 1 -- 172.21.15.31:0/888908869 shutdown_connections 2023-10-06T02:30:35.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.674+0000 7f6fd359d700 1 -- 172.21.15.31:0/888908869 wait complete. 2023-10-06T02:30:35.692 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T02:30:35.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.881+0000 7f58d219d700 1 Processor -- start 2023-10-06T02:30:35.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.881+0000 7f58d219d700 1 -- start start 2023-10-06T02:30:35.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.883+0000 7f58d219d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f58cc140a70 0x7f58cc069b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:35.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.883+0000 7f58d219d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f58cc06de50 0x7f58cc06a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:35.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.883+0000 7f58d219d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58cc15b640 con 0x7f58cc15b0a0 2023-10-06T02:30:35.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.883+0000 7f58d219d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58cc0c6800 con 0x7f58cc140a70 2023-10-06T02:30:35.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.883+0000 7f58d219d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58cc15b920 con 0x7f58cc06de50 2023-10-06T02:30:35.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.883+0000 7f58d119b700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f58cc140a70 0x7f58cc069b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:35.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.883+0000 7f58d099a700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f58cc06de50 0x7f58cc06a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:35.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.883+0000 7f58d099a700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f58cc06de50 0x7f58cc06a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:55576/0 (socket says 172.21.15.31:55576) 2023-10-06T02:30:35.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.883+0000 7f58d099a700 1 -- 172.21.15.31:0/3158462396 learned_addr learned my addr 172.21.15.31:0/3158462396 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:35.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.883+0000 7f58d199c700 1 --1- 172.21.15.31:0/3158462396 >> v1:172.21.15.62:6789/0 conn(0x7f58cc15b0a0 0x7f58cc157a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:46600/0 (socket says 172.21.15.31:46600) 2023-10-06T02:30:35.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.884+0000 7f58d099a700 1 -- 172.21.15.31:0/3158462396 >> v2:172.21.15.62:3300/0 conn(0x7f58cc140a70 msgr2=0x7f58cc069b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:35.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.884+0000 7f58d099a700 1 --2- 172.21.15.31:0/3158462396 >> v2:172.21.15.62:3300/0 conn(0x7f58cc140a70 0x7f58cc069b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:35.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.884+0000 7f58d099a700 1 -- 172.21.15.31:0/3158462396 >> v1:172.21.15.62:6789/0 conn(0x7f58cc15b0a0 legacy=0x7f58cc157a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:35.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.884+0000 7f58d099a700 1 -- 172.21.15.31:0/3158462396 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58cc15bc00 con 0x7f58cc06de50 2023-10-06T02:30:35.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.884+0000 7f58d119b700 1 --2- 172.21.15.31:0/3158462396 >> v2:172.21.15.62:3300/0 conn(0x7f58cc140a70 0x7f58cc069b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:30:35.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.884+0000 7f58d099a700 1 --2- 172.21.15.31:0/3158462396 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f58cc06de50 0x7f58cc06a040 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f58bc0099b0 tx=0x7f58bc009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b387b3be563735e9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:35.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.884+0000 7f58c3fff700 1 -- 172.21.15.31:0/3158462396 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58bc009ea0 con 0x7f58cc06de50 2023-10-06T02:30:35.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.884+0000 7f58c3fff700 1 -- 172.21.15.31:0/3158462396 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58bc00b860 con 0x7f58cc06de50 2023-10-06T02:30:35.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.884+0000 7f58c3fff700 1 -- 172.21.15.31:0/3158462396 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58bc00bc00 con 0x7f58cc06de50 2023-10-06T02:30:35.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.885+0000 7f58d219d700 1 -- 172.21.15.31:0/3158462396 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f58cc06de50 msgr2=0x7f58cc06a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:35.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.885+0000 7f58d219d700 1 --2- 172.21.15.31:0/3158462396 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f58cc06de50 0x7f58cc06a040 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f58bc0099b0 tx=0x7f58bc009250 comp rx=0 tx=0).stop 2023-10-06T02:30:35.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.885+0000 7f58d219d700 1 -- 172.21.15.31:0/3158462396 shutdown_connections 2023-10-06T02:30:35.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.885+0000 7f58d219d700 1 --2- 172.21.15.31:0/3158462396 >> v2:172.21.15.62:3300/0 conn(0x7f58cc140a70 0x7f58cc069b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:35.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.885+0000 7f58d219d700 1 --2- 172.21.15.31:0/3158462396 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f58cc06de50 0x7f58cc06a040 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:35.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.885+0000 7f58d219d700 1 -- 172.21.15.31:0/3158462396 >> 172.21.15.31:0/3158462396 conn(0x7f58cc06d110 msgr2=0x7f58cc0cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:35.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.885+0000 7f58d219d700 1 -- 172.21.15.31:0/3158462396 shutdown_connections 2023-10-06T02:30:35.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.885+0000 7f58d219d700 1 -- 172.21.15.31:0/3158462396 wait complete. 2023-10-06T02:30:35.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.886+0000 7f58d219d700 1 Processor -- start 2023-10-06T02:30:35.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.887+0000 7f58d219d700 1 -- start start 2023-10-06T02:30:35.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.887+0000 7f58d219d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f58cc06de50 0x7f58cc13cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:35.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.887+0000 7f58d219d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58cc140a70 0x7f58cc0b13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:35.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.887+0000 7f58d219d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58cc15b0a0 0x7f58cc0b18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:35.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.888+0000 7f58d219d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58cc15eb00 con 0x7f58cc140a70 2023-10-06T02:30:35.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.888+0000 7f58d219d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58cc13d110 con 0x7f58cc15b0a0 2023-10-06T02:30:35.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.888+0000 7f58d219d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58cc0af050 con 0x7f58cc06de50 2023-10-06T02:30:35.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.888+0000 7f58d099a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58cc140a70 0x7f58cc0b13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:35.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.888+0000 7f58d099a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58cc140a70 0x7f58cc0b13a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47168/0 (socket says 172.21.15.31:47168) 2023-10-06T02:30:35.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.888+0000 7f58d099a700 1 -- 172.21.15.31:0/2750478745 learned_addr learned my addr 172.21.15.31:0/2750478745 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:35.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.888+0000 7f58d119b700 1 --2- 172.21.15.31:0/2750478745 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f58cc06de50 0x7f58cc13cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:35.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.888+0000 7f58d199c700 1 --2- 172.21.15.31:0/2750478745 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58cc15b0a0 0x7f58cc0b18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:35.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.888+0000 7f58d099a700 1 -- 172.21.15.31:0/2750478745 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f58cc06de50 msgr2=0x7f58cc13cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:35.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.888+0000 7f58d099a700 1 --2- 172.21.15.31:0/2750478745 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f58cc06de50 0x7f58cc13cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:35.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.888+0000 7f58d099a700 1 -- 172.21.15.31:0/2750478745 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58cc15b0a0 msgr2=0x7f58cc0b18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:35.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.888+0000 7f58d099a700 1 --2- 172.21.15.31:0/2750478745 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58cc15b0a0 0x7f58cc0b18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:35.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.888+0000 7f58d099a700 1 -- 172.21.15.31:0/2750478745 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58c8009420 con 0x7f58cc140a70 2023-10-06T02:30:35.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.889+0000 7f58d099a700 1 --2- 172.21.15.31:0/2750478745 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58cc140a70 0x7f58cc0b13a0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f58cc0c6940 tx=0x7f58bc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:35.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.889+0000 7f58c17fa700 1 -- 172.21.15.31:0/2750478745 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58bc00bbe0 con 0x7f58cc140a70 2023-10-06T02:30:35.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.889+0000 7f58c17fa700 1 -- 172.21.15.31:0/2750478745 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58bc011420 con 0x7f58cc140a70 2023-10-06T02:30:35.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.889+0000 7f58c17fa700 1 -- 172.21.15.31:0/2750478745 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58bc0117c0 con 0x7f58cc140a70 2023-10-06T02:30:35.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.889+0000 7f58d219d700 1 -- 172.21.15.31:0/2750478745 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58bc0094f0 con 0x7f58cc140a70 2023-10-06T02:30:35.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.889+0000 7f58d219d700 1 -- 172.21.15.31:0/2750478745 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58cc0af310 con 0x7f58cc140a70 2023-10-06T02:30:35.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.890+0000 7f58d219d700 1 -- 172.21.15.31:0/2750478745 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58cc04c3c0 con 0x7f58cc140a70 2023-10-06T02:30:35.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.891+0000 7f58c17fa700 1 -- 172.21.15.31:0/2750478745 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f58bc011bf0 con 0x7f58cc140a70 2023-10-06T02:30:35.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.892+0000 7f58c17fa700 1 --2- 172.21.15.31:0/2750478745 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f58b00674d0 0x7f58b0069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:35.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.892+0000 7f58c17fa700 1 -- 172.21.15.31:0/2750478745 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f58bc086aa0 con 0x7f58cc140a70 2023-10-06T02:30:35.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.892+0000 7f58d119b700 1 --2- 172.21.15.31:0/2750478745 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f58b00674d0 0x7f58b0069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:35.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.893+0000 7f58d119b700 1 --2- 172.21.15.31:0/2750478745 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f58b00674d0 0x7f58b0069980 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f58c80093f0 tx=0x7f58c8008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:35.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:35.896+0000 7f58c17fa700 1 -- 172.21.15.31:0/2750478745 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f58bc0532e0 con 0x7f58cc140a70 2023-10-06T02:30:36.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.099+0000 7f58d219d700 1 -- 172.21.15.31:0/2750478745 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f58cc15bee0 con 0x7f58cc140a70 2023-10-06T02:30:36.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.099+0000 7f58c17fa700 1 -- 172.21.15.31:0/2750478745 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v132) v1 ==== 104+0+11 (secure 0 0 0) 0x7f58bc056900 con 0x7f58cc140a70 2023-10-06T02:30:36.235 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-10-06T02:30:36.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.101+0000 7f58d219d700 1 -- 172.21.15.31:0/2750478745 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f58b00674d0 msgr2=0x7f58b0069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:36.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.101+0000 7f58d219d700 1 --2- 172.21.15.31:0/2750478745 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f58b00674d0 0x7f58b0069980 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f58c80093f0 tx=0x7f58c8008040 comp rx=0 tx=0).stop 2023-10-06T02:30:36.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.101+0000 7f58d219d700 1 -- 172.21.15.31:0/2750478745 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58cc140a70 msgr2=0x7f58cc0b13a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:36.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.101+0000 7f58d219d700 1 --2- 172.21.15.31:0/2750478745 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58cc140a70 0x7f58cc0b13a0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f58cc0c6940 tx=0x7f58bc017000 comp rx=0 tx=0).stop 2023-10-06T02:30:36.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.101+0000 7f58d219d700 1 -- 172.21.15.31:0/2750478745 shutdown_connections 2023-10-06T02:30:36.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.101+0000 7f58d219d700 1 --2- 172.21.15.31:0/2750478745 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58cc15b0a0 0x7f58cc0b18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:36.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.101+0000 7f58d219d700 1 --2- 172.21.15.31:0/2750478745 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f58b00674d0 0x7f58b0069980 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:36.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.101+0000 7f58d219d700 1 --2- 172.21.15.31:0/2750478745 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f58cc06de50 0x7f58cc13cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:36.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.101+0000 7f58d219d700 1 --2- 172.21.15.31:0/2750478745 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58cc140a70 0x7f58cc0b13a0 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:36.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.101+0000 7f58d219d700 1 -- 172.21.15.31:0/2750478745 >> 172.21.15.31:0/2750478745 conn(0x7f58cc06d110 msgr2=0x7f58cc06f210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:36.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.102+0000 7f58d219d700 1 -- 172.21.15.31:0/2750478745 shutdown_connections 2023-10-06T02:30:36.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.102+0000 7f58d219d700 1 -- 172.21.15.31:0/2750478745 wait complete. 2023-10-06T02:30:36.240 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T02:30:36.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.429+0000 7f04909fe700 1 Processor -- start 2023-10-06T02:30:36.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.429+0000 7f04909fe700 1 -- start start 2023-10-06T02:30:36.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.430+0000 7f04909fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f048c153a00 0x7f048c153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:36.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.430+0000 7f04909fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f048c15de50 0x7f048c15a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:36.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.430+0000 7f04909fe700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f048c15af60 con 0x7f048c0cea00 2023-10-06T02:30:36.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.430+0000 7f04909fe700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f048c15d010 con 0x7f048c15de50 2023-10-06T02:30:36.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.430+0000 7f04909fe700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f048c15b240 con 0x7f048c153a00 2023-10-06T02:30:36.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.430+0000 7f048a59c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f048c15de50 0x7f048c15a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:36.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.430+0000 7f048b59e700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f048c0cea00 0x7f048c0cee40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49716/0 (socket says 172.21.15.31:49716) 2023-10-06T02:30:36.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.430+0000 7f048b59e700 1 -- 172.21.15.31:0/3722223076 learned_addr learned my addr 172.21.15.31:0/3722223076 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:30:36.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.431+0000 7f048ad9d700 1 --2- 172.21.15.31:0/3722223076 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f048c153a00 0x7f048c153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:36.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.431+0000 7f048a59c700 1 -- 172.21.15.31:0/3722223076 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f048c153a00 msgr2=0x7f048c153dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:36.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.431+0000 7f048a59c700 1 --2- 172.21.15.31:0/3722223076 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f048c153a00 0x7f048c153dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:36.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.431+0000 7f048a59c700 1 -- 172.21.15.31:0/3722223076 >> v1:172.21.15.31:6789/0 conn(0x7f048c0cea00 legacy=0x7f048c0cee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:36.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.431+0000 7f048a59c700 1 -- 172.21.15.31:0/3722223076 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f048c15b520 con 0x7f048c15de50 2023-10-06T02:30:36.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.431+0000 7f048ad9d700 1 --2- 172.21.15.31:0/3722223076 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f048c153a00 0x7f048c153dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:30:36.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.431+0000 7f0489d9b700 1 -- 172.21.15.31:0/3722223076 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 233643170 0 0) 0x7f048c15af60 con 0x7f048c0cea00 2023-10-06T02:30:36.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.431+0000 7f048a59c700 1 --2- 172.21.15.31:0/3722223076 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f048c15de50 0x7f048c15a540 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f04800098e0 tx=0x7f04800091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=663f7144054e3ac8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:36.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.431+0000 7f0489d9b700 1 -- 172.21.15.31:0/3722223076 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0480009e80 con 0x7f048c15de50 2023-10-06T02:30:36.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.431+0000 7f0489d9b700 1 -- 172.21.15.31:0/3722223076 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f048000b860 con 0x7f048c15de50 2023-10-06T02:30:36.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.432+0000 7f0489d9b700 1 -- 172.21.15.31:0/3722223076 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f048000bbe0 con 0x7f048c15de50 2023-10-06T02:30:36.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.432+0000 7f04909fe700 1 -- 172.21.15.31:0/3722223076 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f048c15de50 msgr2=0x7f048c15a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:36.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.432+0000 7f04909fe700 1 --2- 172.21.15.31:0/3722223076 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f048c15de50 0x7f048c15a540 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f04800098e0 tx=0x7f04800091a0 comp rx=0 tx=0).stop 2023-10-06T02:30:36.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.432+0000 7f04909fe700 1 -- 172.21.15.31:0/3722223076 shutdown_connections 2023-10-06T02:30:36.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.432+0000 7f04909fe700 1 --2- 172.21.15.31:0/3722223076 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f048c153a00 0x7f048c153dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:36.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.432+0000 7f04909fe700 1 --2- 172.21.15.31:0/3722223076 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f048c15de50 0x7f048c15a540 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:36.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.432+0000 7f04909fe700 1 -- 172.21.15.31:0/3722223076 >> 172.21.15.31:0/3722223076 conn(0x7f048c0703a0 msgr2=0x7f048c0707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:36.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.432+0000 7f04909fe700 1 -- 172.21.15.31:0/3722223076 shutdown_connections 2023-10-06T02:30:36.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.432+0000 7f04909fe700 1 -- 172.21.15.31:0/3722223076 wait complete. 2023-10-06T02:30:36.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.434+0000 7f04909fe700 1 Processor -- start 2023-10-06T02:30:36.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.434+0000 7f04909fe700 1 -- start start 2023-10-06T02:30:36.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.434+0000 7f04909fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f048c0cea00 0x7f048c0c9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:36.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.435+0000 7f04909fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f048c153a00 0x7f048c169b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:36.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.435+0000 7f04909fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f048c15de50 0x7f048c16a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:36.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.435+0000 7f04909fe700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f048c0ca3e0 con 0x7f048c0cea00 2023-10-06T02:30:36.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.435+0000 7f04909fe700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f048c0c9530 con 0x7f048c153a00 2023-10-06T02:30:36.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.435+0000 7f04909fe700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f048c0c75e0 con 0x7f048c15de50 2023-10-06T02:30:36.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.435+0000 7f048ad9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f048c0cea00 0x7f048c0c9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:36.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.435+0000 7f048ad9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f048c0cea00 0x7f048c0c9ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47180/0 (socket says 172.21.15.31:47180) 2023-10-06T02:30:36.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.435+0000 7f048ad9d700 1 -- 172.21.15.31:0/3509085099 learned_addr learned my addr 172.21.15.31:0/3509085099 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:36.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.435+0000 7f048ad9d700 1 -- 172.21.15.31:0/3509085099 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f048c15de50 msgr2=0x7f048c16a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:36.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.435+0000 7f048ad9d700 1 --2- 172.21.15.31:0/3509085099 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f048c15de50 0x7f048c16a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:36.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.435+0000 7f048ad9d700 1 -- 172.21.15.31:0/3509085099 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f048c153a00 msgr2=0x7f048c169b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:36.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.435+0000 7f048ad9d700 1 --2- 172.21.15.31:0/3509085099 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f048c153a00 0x7f048c169b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:36.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.435+0000 7f048ad9d700 1 -- 172.21.15.31:0/3509085099 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0480009420 con 0x7f048c0cea00 2023-10-06T02:30:36.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.435+0000 7f048ad9d700 1 --2- 172.21.15.31:0/3509085099 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f048c0cea00 0x7f048c0c9ea0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f047c00e8d0 tx=0x7f047c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:36.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.436+0000 7f0472ffd700 1 -- 172.21.15.31:0/3509085099 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f047c00ee70 con 0x7f048c0cea00 2023-10-06T02:30:36.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.436+0000 7f0472ffd700 1 -- 172.21.15.31:0/3509085099 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f047c00c5e0 con 0x7f048c0cea00 2023-10-06T02:30:36.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.436+0000 7f04909fe700 1 -- 172.21.15.31:0/3509085099 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f048c0cb520 con 0x7f048c0cea00 2023-10-06T02:30:36.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.436+0000 7f0472ffd700 1 -- 172.21.15.31:0/3509085099 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f047c00c980 con 0x7f048c0cea00 2023-10-06T02:30:36.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.436+0000 7f04909fe700 1 -- 172.21.15.31:0/3509085099 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f048c0b5b20 con 0x7f048c0cea00 2023-10-06T02:30:36.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.437+0000 7f04909fe700 1 -- 172.21.15.31:0/3509085099 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f048c1576a0 con 0x7f048c0cea00 2023-10-06T02:30:36.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.438+0000 7f0472ffd700 1 -- 172.21.15.31:0/3509085099 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f047c013410 con 0x7f048c0cea00 2023-10-06T02:30:36.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.439+0000 7f0472ffd700 1 --2- 172.21.15.31:0/3509085099 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f04780674d0 0x7f0478069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:36.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.439+0000 7f0472ffd700 1 -- 172.21.15.31:0/3509085099 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f047c0850f0 con 0x7f048c0cea00 2023-10-06T02:30:36.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.439+0000 7f048a59c700 1 --2- 172.21.15.31:0/3509085099 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f04780674d0 0x7f0478069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:36.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.440+0000 7f048a59c700 1 --2- 172.21.15.31:0/3509085099 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f04780674d0 0x7f0478069980 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f048c15c3a0 tx=0x7f0480017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:36.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.443+0000 7f0472ffd700 1 -- 172.21.15.31:0/3509085099 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f047c051820 con 0x7f048c0cea00 2023-10-06T02:30:36.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.643+0000 7f04909fe700 1 -- 172.21.15.31:0/3509085099 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f048c04c3c0 con 0x7f048c0cea00 2023-10-06T02:30:36.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.643+0000 7f0472ffd700 1 -- 172.21.15.31:0/3509085099 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v132) v1 ==== 100+0+11 (secure 0 0 0) 0x7f047c054e40 con 0x7f048c0cea00 2023-10-06T02:30:36.868 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-10-06T02:30:36.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.645+0000 7f04909fe700 1 -- 172.21.15.31:0/3509085099 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f04780674d0 msgr2=0x7f0478069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:36.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.645+0000 7f04909fe700 1 --2- 172.21.15.31:0/3509085099 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f04780674d0 0x7f0478069980 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f048c15c3a0 tx=0x7f0480017000 comp rx=0 tx=0).stop 2023-10-06T02:30:36.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.645+0000 7f04909fe700 1 -- 172.21.15.31:0/3509085099 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f048c0cea00 msgr2=0x7f048c0c9ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:36.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.645+0000 7f04909fe700 1 --2- 172.21.15.31:0/3509085099 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f048c0cea00 0x7f048c0c9ea0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f047c00e8d0 tx=0x7f047c008040 comp rx=0 tx=0).stop 2023-10-06T02:30:36.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.646+0000 7f04909fe700 1 -- 172.21.15.31:0/3509085099 shutdown_connections 2023-10-06T02:30:36.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.646+0000 7f04909fe700 1 --2- 172.21.15.31:0/3509085099 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f04780674d0 0x7f0478069980 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:36.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.646+0000 7f04909fe700 1 --2- 172.21.15.31:0/3509085099 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f048c15de50 0x7f048c16a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:36.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.646+0000 7f04909fe700 1 --2- 172.21.15.31:0/3509085099 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f048c0cea00 0x7f048c0c9ea0 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:36.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.646+0000 7f04909fe700 1 --2- 172.21.15.31:0/3509085099 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f048c153a00 0x7f048c169b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:36.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.646+0000 7f04909fe700 1 -- 172.21.15.31:0/3509085099 >> 172.21.15.31:0/3509085099 conn(0x7f048c0703a0 msgr2=0x7f048c0d0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:36.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.646+0000 7f04909fe700 1 -- 172.21.15.31:0/3509085099 shutdown_connections 2023-10-06T02:30:36.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:36.646+0000 7f04909fe700 1 -- 172.21.15.31:0/3509085099 wait complete. 2023-10-06T02:30:36.871 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:30:37.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.057+0000 7fb00a39d700 1 Processor -- start 2023-10-06T02:30:37.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.057+0000 7fb00a39d700 1 -- start start 2023-10-06T02:30:37.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.059+0000 7fb00a39d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb00406aaa0 0x7fb00415a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:37.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.059+0000 7fb00a39d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb00415ae70 con 0x7fb00413fb20 2023-10-06T02:30:37.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.059+0000 7fb00a39d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb00406af60 con 0x7fb00413fef0 2023-10-06T02:30:37.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.059+0000 7fb00a39d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb00406b240 con 0x7fb00406aaa0 2023-10-06T02:30:37.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.059+0000 7fb00939b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fb00413fb20 0x7fb0040c7990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49730/0 (socket says 172.21.15.31:49730) 2023-10-06T02:30:37.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.059+0000 7fb00939b700 1 -- 172.21.15.31:0/1955240519 learned_addr learned my addr 172.21.15.31:0/1955240519 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:30:37.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.059+0000 7fb009b9c700 1 --2- 172.21.15.31:0/1955240519 >> v2:172.21.15.31:3300/0 conn(0x7fb00406aaa0 0x7fb00415a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:37.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.059+0000 7faffbfff700 1 -- 172.21.15.31:0/1955240519 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2421430300 0 0) 0x7fb000001110 con 0x7fb00413fb20 2023-10-06T02:30:37.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.060+0000 7faffbfff700 1 -- 172.21.15.31:0/1955240519 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faff0003570 con 0x7fb00413fb20 2023-10-06T02:30:37.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.060+0000 7fb009b9c700 1 -- 172.21.15.31:0/1955240519 >> v1:172.21.15.62:6789/0 conn(0x7fb00413fef0 legacy=0x7fb0040c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:37.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.060+0000 7faffbfff700 1 -- 172.21.15.31:0/1955240519 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1392943440 0 0) 0x7fb00406af60 con 0x7fb00413fef0 2023-10-06T02:30:37.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.060+0000 7fb009b9c700 1 -- 172.21.15.31:0/1955240519 >> v1:172.21.15.31:6789/0 conn(0x7fb00413fb20 legacy=0x7fb0040c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:37.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.060+0000 7fb009b9c700 1 -- 172.21.15.31:0/1955240519 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb00406b520 con 0x7fb00406aaa0 2023-10-06T02:30:37.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.060+0000 7fb009b9c700 1 --2- 172.21.15.31:0/1955240519 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb00406aaa0 0x7fb00415a930 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7faffc009e60 tx=0x7faffc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8228d46b048278fc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:37.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.060+0000 7faffbfff700 1 -- 172.21.15.31:0/1955240519 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faffc00bb60 con 0x7fb00406aaa0 2023-10-06T02:30:37.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.060+0000 7faffbfff700 1 -- 172.21.15.31:0/1955240519 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faffc00bcc0 con 0x7fb00406aaa0 2023-10-06T02:30:37.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.060+0000 7faffbfff700 1 -- 172.21.15.31:0/1955240519 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faffc011460 con 0x7fb00406aaa0 2023-10-06T02:30:37.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.061+0000 7fb00a39d700 1 -- 172.21.15.31:0/1955240519 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb00406aaa0 msgr2=0x7fb00415a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:37.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.061+0000 7fb00a39d700 1 --2- 172.21.15.31:0/1955240519 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb00406aaa0 0x7fb00415a930 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7faffc009e60 tx=0x7faffc009780 comp rx=0 tx=0).stop 2023-10-06T02:30:37.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.061+0000 7fb00a39d700 1 -- 172.21.15.31:0/1955240519 shutdown_connections 2023-10-06T02:30:37.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.061+0000 7fb00a39d700 1 --2- 172.21.15.31:0/1955240519 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb00406aaa0 0x7fb00415a930 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:37.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.061+0000 7fb00a39d700 1 -- 172.21.15.31:0/1955240519 >> 172.21.15.31:0/1955240519 conn(0x7fb00406d170 msgr2=0x7fb0040cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:37.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.061+0000 7fb00a39d700 1 -- 172.21.15.31:0/1955240519 shutdown_connections 2023-10-06T02:30:37.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.061+0000 7fb00a39d700 1 -- 172.21.15.31:0/1955240519 wait complete. 2023-10-06T02:30:37.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.062+0000 7fb00a39d700 1 Processor -- start 2023-10-06T02:30:37.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.063+0000 7fb00a39d700 1 -- start start 2023-10-06T02:30:37.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.063+0000 7fb00a39d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb00406aaa0 0x7fb00413c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:37.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.063+0000 7fb00a39d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb00413fb20 0x7fb0040c75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:37.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.063+0000 7fb00a39d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb00413fef0 0x7fb0040c7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:37.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.063+0000 7fb00a39d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb00415b120 con 0x7fb00413fef0 2023-10-06T02:30:37.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.063+0000 7fb00a39d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb004169710 con 0x7fb00413fb20 2023-10-06T02:30:37.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.063+0000 7fb00a39d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0040ad190 con 0x7fb00406aaa0 2023-10-06T02:30:37.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.064+0000 7fb009b9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb00413fef0 0x7fb0040c7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:37.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.064+0000 7fb009b9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb00413fef0 0x7fb0040c7b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47194/0 (socket says 172.21.15.31:47194) 2023-10-06T02:30:37.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.064+0000 7fb009b9c700 1 -- 172.21.15.31:0/3756602766 learned_addr learned my addr 172.21.15.31:0/3756602766 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:37.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.064+0000 7fb00939b700 1 --2- 172.21.15.31:0/3756602766 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb00406aaa0 0x7fb00413c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:37.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.064+0000 7fb008b9a700 1 --2- 172.21.15.31:0/3756602766 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb00413fb20 0x7fb0040c75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:37.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.064+0000 7fb009b9c700 1 -- 172.21.15.31:0/3756602766 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb00406aaa0 msgr2=0x7fb00413c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:37.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.064+0000 7fb009b9c700 1 --2- 172.21.15.31:0/3756602766 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb00406aaa0 0x7fb00413c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:37.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.064+0000 7fb009b9c700 1 -- 172.21.15.31:0/3756602766 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb00413fb20 msgr2=0x7fb0040c75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:37.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.064+0000 7fb009b9c700 1 --2- 172.21.15.31:0/3756602766 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb00413fb20 0x7fb0040c75f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:37.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.064+0000 7fb009b9c700 1 -- 172.21.15.31:0/3756602766 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faffc009a20 con 0x7fb00413fef0 2023-10-06T02:30:37.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.064+0000 7fb009b9c700 1 --2- 172.21.15.31:0/3756602766 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb00413fef0 0x7fb0040c7b30 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fb004071320 tx=0x7faffc01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:37.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.065+0000 7faff97fa700 1 -- 172.21.15.31:0/3756602766 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faffc011460 con 0x7fb00413fef0 2023-10-06T02:30:37.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.065+0000 7faff97fa700 1 -- 172.21.15.31:0/3756602766 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faffc009420 con 0x7fb00413fef0 2023-10-06T02:30:37.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.065+0000 7faff97fa700 1 -- 172.21.15.31:0/3756602766 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faffc00bb60 con 0x7fb00413fef0 2023-10-06T02:30:37.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.065+0000 7fb00a39d700 1 -- 172.21.15.31:0/3756602766 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0040ad470 con 0x7fb00413fef0 2023-10-06T02:30:37.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.065+0000 7fb00a39d700 1 -- 172.21.15.31:0/3756602766 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb004169960 con 0x7fb00413fef0 2023-10-06T02:30:37.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.066+0000 7fb00a39d700 1 -- 172.21.15.31:0/3756602766 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb00404c3c0 con 0x7fb00413fef0 2023-10-06T02:30:37.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.067+0000 7faff97fa700 1 -- 172.21.15.31:0/3756602766 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7faffc02c3b0 con 0x7fb00413fef0 2023-10-06T02:30:37.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.068+0000 7faff97fa700 1 --2- 172.21.15.31:0/3756602766 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fafe80674d0 0x7fafe8069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:37.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.068+0000 7faff97fa700 1 -- 172.21.15.31:0/3756602766 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6086+0+0 (secure 0 0 0) 0x7faffc087090 con 0x7fb00413fef0 2023-10-06T02:30:37.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.068+0000 7fb00939b700 1 --2- 172.21.15.31:0/3756602766 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fafe80674d0 0x7fafe8069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:37.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.069+0000 7fb00939b700 1 --2- 172.21.15.31:0/3756602766 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fafe80674d0 0x7fafe8069980 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb000001270 tx=0x7fb000000bc0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:37.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.072+0000 7faff97fa700 1 -- 172.21.15.31:0/3756602766 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faffc0537c0 con 0x7fb00413fef0 2023-10-06T02:30:37.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.320+0000 7fb00a39d700 1 -- 172.21.15.31:0/3756602766 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb00406b800 con 0x7fb00413fef0 2023-10-06T02:30:37.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.321+0000 7faff97fa700 1 -- 172.21.15.31:0/3756602766 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4601 (secure 0 0 0) 0x7faffc028090 con 0x7fb00413fef0 2023-10-06T02:30:37.322 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:30:37.323 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":91,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6611,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/1574790962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1574790962},{"type":"v1","addr":"172.21.15.62:6834","nonce":1574790962}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":85},{"gid":7591,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/2840240410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":2840240410},{"type":"v1","addr":"172.21.15.31:6835","nonce":2840240410}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":85},{"gid":7597,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2500071914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2500071914},{"type":"v1","addr":"172.21.15.62:6835","nonce":2500071914}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:30:11.625173+0000","modified":"2023-10-06T02:30:15.149386+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7971},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7971":{"gid":7971,"name":"a","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6834/1197111339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1197111339},{"type":"v1","addr":"172.21.15.31:6834","nonce":1197111339}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-10-06T02:30:37.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.323+0000 7fb00a39d700 1 -- 172.21.15.31:0/3756602766 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fafe80674d0 msgr2=0x7fafe8069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:37.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.323+0000 7fb00a39d700 1 --2- 172.21.15.31:0/3756602766 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fafe80674d0 0x7fafe8069980 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb000001270 tx=0x7fb000000bc0 comp rx=0 tx=0).stop 2023-10-06T02:30:37.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.323+0000 7fb00a39d700 1 -- 172.21.15.31:0/3756602766 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb00413fef0 msgr2=0x7fb0040c7b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:37.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.323+0000 7fb00a39d700 1 --2- 172.21.15.31:0/3756602766 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb00413fef0 0x7fb0040c7b30 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fb004071320 tx=0x7faffc01b000 comp rx=0 tx=0).stop 2023-10-06T02:30:37.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.323+0000 7fb00a39d700 1 -- 172.21.15.31:0/3756602766 shutdown_connections 2023-10-06T02:30:37.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.323+0000 7fb00a39d700 1 --2- 172.21.15.31:0/3756602766 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb00413fef0 0x7fb0040c7b30 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:37.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.323+0000 7fb00a39d700 1 --2- 172.21.15.31:0/3756602766 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fafe80674d0 0x7fafe8069980 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:37.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.323+0000 7fb00a39d700 1 --2- 172.21.15.31:0/3756602766 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb00406aaa0 0x7fb00413c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:37.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.323+0000 7fb00a39d700 1 --2- 172.21.15.31:0/3756602766 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb00413fb20 0x7fb0040c75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:37.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.323+0000 7fb00a39d700 1 -- 172.21.15.31:0/3756602766 >> 172.21.15.31:0/3756602766 conn(0x7fb00406d170 msgr2=0x7fb00413e810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:37.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.323+0000 7fb00a39d700 1 -- 172.21.15.31:0/3756602766 shutdown_connections 2023-10-06T02:30:37.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.323+0000 7fb00a39d700 1 -- 172.21.15.31:0/3756602766 wait complete. 2023-10-06T02:30:37.326 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 91 2023-10-06T02:30:37.342 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:30:37.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.528+0000 7f1e34d9d700 1 Processor -- start 2023-10-06T02:30:37.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.528+0000 7f1e34d9d700 1 -- start start 2023-10-06T02:30:37.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.530+0000 7f1e34d9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1e300caec0 0x7f1e30164450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:37.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.530+0000 7f1e34d9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e300c7420 con 0x7f1e30159a10 2023-10-06T02:30:37.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.530+0000 7f1e34d9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e3006d820 con 0x7f1e30159640 2023-10-06T02:30:37.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.530+0000 7f1e34d9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e3006db00 con 0x7f1e300caec0 2023-10-06T02:30:37.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.530+0000 7f1e2ffff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1e300caec0 0x7f1e30164450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:37.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.530+0000 7f1e2effd700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f1e30159a10 0x7f1e300ca7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49732/0 (socket says 172.21.15.31:49732) 2023-10-06T02:30:37.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.530+0000 7f1e2effd700 1 -- 172.21.15.31:0/3675497975 learned_addr learned my addr 172.21.15.31:0/3675497975 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:30:37.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.530+0000 7f1e2ffff700 1 -- 172.21.15.31:0/3675497975 >> v1:172.21.15.62:6789/0 conn(0x7f1e30159640 legacy=0x7f1e3006d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:37.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.530+0000 7f1e2ffff700 1 -- 172.21.15.31:0/3675497975 >> v1:172.21.15.31:6789/0 conn(0x7f1e30159a10 legacy=0x7f1e300ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:37.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.530+0000 7f1e2e7fc700 1 -- 172.21.15.31:0/3675497975 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2722839887 0 0) 0x7f1e300c7420 con 0x7f1e30159a10 2023-10-06T02:30:37.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.530+0000 7f1e2ffff700 1 -- 172.21.15.31:0/3675497975 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e3006dde0 con 0x7f1e300caec0 2023-10-06T02:30:37.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.531+0000 7f1e2ffff700 1 --2- 172.21.15.31:0/3675497975 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e300caec0 0x7f1e30164450 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f1e28009e90 tx=0x7f1e28009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11e1e78244c50738 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:37.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.531+0000 7f1e2e7fc700 1 -- 172.21.15.31:0/3675497975 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e2800e070 con 0x7f1e300caec0 2023-10-06T02:30:37.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.531+0000 7f1e2e7fc700 1 -- 172.21.15.31:0/3675497975 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e2800bbf0 con 0x7f1e300caec0 2023-10-06T02:30:37.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.531+0000 7f1e2e7fc700 1 -- 172.21.15.31:0/3675497975 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e28012430 con 0x7f1e300caec0 2023-10-06T02:30:37.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.531+0000 7f1e34d9d700 1 -- 172.21.15.31:0/3675497975 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e300caec0 msgr2=0x7f1e30164450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:37.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.531+0000 7f1e34d9d700 1 --2- 172.21.15.31:0/3675497975 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e300caec0 0x7f1e30164450 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f1e28009e90 tx=0x7f1e28009780 comp rx=0 tx=0).stop 2023-10-06T02:30:37.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.532+0000 7f1e34d9d700 1 -- 172.21.15.31:0/3675497975 shutdown_connections 2023-10-06T02:30:37.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.532+0000 7f1e34d9d700 1 --2- 172.21.15.31:0/3675497975 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e300caec0 0x7f1e30164450 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:37.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.532+0000 7f1e34d9d700 1 -- 172.21.15.31:0/3675497975 >> 172.21.15.31:0/3675497975 conn(0x7f1e3006e720 msgr2=0x7f1e3006eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:37.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.532+0000 7f1e34d9d700 1 -- 172.21.15.31:0/3675497975 shutdown_connections 2023-10-06T02:30:37.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.532+0000 7f1e34d9d700 1 -- 172.21.15.31:0/3675497975 wait complete. 2023-10-06T02:30:37.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.533+0000 7f1e34d9d700 1 Processor -- start 2023-10-06T02:30:37.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.533+0000 7f1e34d9d700 1 -- start start 2023-10-06T02:30:37.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.534+0000 7f1e34d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e300caec0 0x7f1e30141560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:37.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.534+0000 7f1e34d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e30159640 0x7f1e301512b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:37.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.534+0000 7f1e34d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e30159a10 0x7f1e301517f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:37.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.534+0000 7f1e34d9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e30155d40 con 0x7f1e30159a10 2023-10-06T02:30:37.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.534+0000 7f1e34d9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e30155130 con 0x7f1e300caec0 2023-10-06T02:30:37.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.534+0000 7f1e34d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e30155410 con 0x7f1e30159640 2023-10-06T02:30:37.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.534+0000 7f1e2ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e30159a10 0x7f1e301517f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:37.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.534+0000 7f1e2ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e30159a10 0x7f1e301517f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47218/0 (socket says 172.21.15.31:47218) 2023-10-06T02:30:37.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.534+0000 7f1e2ffff700 1 -- 172.21.15.31:0/2824818546 learned_addr learned my addr 172.21.15.31:0/2824818546 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:37.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.535+0000 7f1e2f7fe700 1 --2- 172.21.15.31:0/2824818546 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e300caec0 0x7f1e30141560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:37.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.535+0000 7f1e2effd700 1 --2- 172.21.15.31:0/2824818546 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e30159640 0x7f1e301512b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:37.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.535+0000 7f1e2ffff700 1 -- 172.21.15.31:0/2824818546 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e30159640 msgr2=0x7f1e301512b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:37.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.535+0000 7f1e2ffff700 1 --2- 172.21.15.31:0/2824818546 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e30159640 0x7f1e301512b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:37.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.535+0000 7f1e2ffff700 1 -- 172.21.15.31:0/2824818546 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e300caec0 msgr2=0x7f1e30141560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:37.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.535+0000 7f1e2ffff700 1 --2- 172.21.15.31:0/2824818546 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e300caec0 0x7f1e30141560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:37.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.535+0000 7f1e2ffff700 1 -- 172.21.15.31:0/2824818546 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e28009a20 con 0x7f1e30159a10 2023-10-06T02:30:37.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.535+0000 7f1e2ffff700 1 --2- 172.21.15.31:0/2824818546 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e30159a10 0x7f1e301517f0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f1e28013000 tx=0x7f1e28019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:37.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.535+0000 7f1e177fe700 1 -- 172.21.15.31:0/2824818546 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e2801b420 con 0x7f1e30159a10 2023-10-06T02:30:37.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.536+0000 7f1e177fe700 1 -- 172.21.15.31:0/2824818546 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e2801b580 con 0x7f1e30159a10 2023-10-06T02:30:37.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.536+0000 7f1e177fe700 1 -- 172.21.15.31:0/2824818546 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e2801b940 con 0x7f1e30159a10 2023-10-06T02:30:37.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.536+0000 7f1e34d9d700 1 -- 172.21.15.31:0/2824818546 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e301556f0 con 0x7f1e30159a10 2023-10-06T02:30:37.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.536+0000 7f1e34d9d700 1 -- 172.21.15.31:0/2824818546 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e30154370 con 0x7f1e30159a10 2023-10-06T02:30:37.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.537+0000 7f1e34d9d700 1 -- 172.21.15.31:0/2824818546 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e30155b90 con 0x7f1e30159a10 2023-10-06T02:30:37.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.537+0000 7f1e177fe700 1 -- 172.21.15.31:0/2824818546 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f1e2801bcb0 con 0x7f1e30159a10 2023-10-06T02:30:37.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.538+0000 7f1e177fe700 1 --2- 172.21.15.31:0/2824818546 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f1e1c0674d0 0x7f1e1c069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:37.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.538+0000 7f1e177fe700 1 -- 172.21.15.31:0/2824818546 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f1e28086d20 con 0x7f1e30159a10 2023-10-06T02:30:37.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.539+0000 7f1e2f7fe700 1 --2- 172.21.15.31:0/2824818546 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f1e1c0674d0 0x7f1e1c069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:37.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.540+0000 7f1e2f7fe700 1 --2- 172.21.15.31:0/2824818546 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f1e1c0674d0 0x7f1e1c069980 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f1e3016d380 tx=0x7f1e20009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:37.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.543+0000 7f1e177fe700 1 -- 172.21.15.31:0/2824818546 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1e28053450 con 0x7f1e30159a10 2023-10-06T02:30:37.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.747+0000 7f1e34d9d700 1 -- 172.21.15.31:0/2824818546 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1e30155eb0 con 0x7f1e30159a10 2023-10-06T02:30:37.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.748+0000 7f1e177fe700 1 -- 172.21.15.31:0/2824818546 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v132) v1 ==== 74+0+16876 (secure 0 0 0) 0x7f1e2802a090 con 0x7f1e30159a10 2023-10-06T02:30:37.750 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:30:37.750 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":132,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:30:13.138416+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:30:09.791433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-10-06T02:30:10.601699+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":20,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:30:37.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.750+0000 7f1e34d9d700 1 -- 172.21.15.31:0/2824818546 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f1e1c0674d0 msgr2=0x7f1e1c069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:37.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.750+0000 7f1e34d9d700 1 --2- 172.21.15.31:0/2824818546 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f1e1c0674d0 0x7f1e1c069980 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f1e3016d380 tx=0x7f1e20009040 comp rx=0 tx=0).stop 2023-10-06T02:30:37.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.750+0000 7f1e34d9d700 1 -- 172.21.15.31:0/2824818546 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e30159a10 msgr2=0x7f1e301517f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:37.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.750+0000 7f1e34d9d700 1 --2- 172.21.15.31:0/2824818546 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e30159a10 0x7f1e301517f0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f1e28013000 tx=0x7f1e28019000 comp rx=0 tx=0).stop 2023-10-06T02:30:37.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.750+0000 7f1e34d9d700 1 -- 172.21.15.31:0/2824818546 shutdown_connections 2023-10-06T02:30:37.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.750+0000 7f1e34d9d700 1 --2- 172.21.15.31:0/2824818546 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e30159a10 0x7f1e301517f0 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:37.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.750+0000 7f1e34d9d700 1 --2- 172.21.15.31:0/2824818546 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e300caec0 0x7f1e30141560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:37.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.750+0000 7f1e34d9d700 1 --2- 172.21.15.31:0/2824818546 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f1e1c0674d0 0x7f1e1c069980 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:37.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.750+0000 7f1e34d9d700 1 --2- 172.21.15.31:0/2824818546 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e30159640 0x7f1e301512b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:37.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.750+0000 7f1e34d9d700 1 -- 172.21.15.31:0/2824818546 >> 172.21.15.31:0/2824818546 conn(0x7f1e3006e720 msgr2=0x7f1e300aeb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:37.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.750+0000 7f1e34d9d700 1 -- 172.21.15.31:0/2824818546 shutdown_connections 2023-10-06T02:30:37.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.751+0000 7f1e34d9d700 1 -- 172.21.15.31:0/2824818546 wait complete. 2023-10-06T02:30:37.769 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T02:30:37.770 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:30:37.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.955+0000 7fc0bd79d700 1 Processor -- start 2023-10-06T02:30:37.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.955+0000 7fc0bd79d700 1 -- start start 2023-10-06T02:30:37.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.956+0000 7fc0bd79d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc0b813fb20 0x7fc0b815a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:37.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.956+0000 7fc0bd79d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc0b806aaa0 0x7fc0b80c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:37.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.956+0000 7fc0bd79d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0b815ae70 con 0x7fc0b813fef0 2023-10-06T02:30:37.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.956+0000 7fc0bd79d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0b806af60 con 0x7fc0b813fb20 2023-10-06T02:30:37.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.956+0000 7fc0bd79d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b806b240 con 0x7fc0b806aaa0 2023-10-06T02:30:37.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.957+0000 7fc0b77fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fc0b813fef0 0x7fc0b80c7990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60726/0 (socket says 172.21.15.31:60726) 2023-10-06T02:30:37.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.957+0000 7fc0b77fe700 1 -- 172.21.15.31:0/3679759898 learned_addr learned my addr 172.21.15.31:0/3679759898 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:30:37.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.957+0000 7fc0b7fff700 1 --2- 172.21.15.31:0/3679759898 >> v2:172.21.15.62:3300/0 conn(0x7fc0b813fb20 0x7fc0b815a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:37.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.957+0000 7fc0bcf9c700 1 --2- 172.21.15.31:0/3679759898 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc0b806aaa0 0x7fc0b80c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:37.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.957+0000 7fc0b6ffd700 1 -- 172.21.15.31:0/3679759898 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1739980702 0 0) 0x7fc0ac001290 con 0x7fc0b813fef0 2023-10-06T02:30:37.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.957+0000 7fc0b6ffd700 1 -- 172.21.15.31:0/3679759898 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0a0003570 con 0x7fc0b813fef0 2023-10-06T02:30:37.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.958+0000 7fc0bcf9c700 1 -- 172.21.15.31:0/3679759898 >> v2:172.21.15.62:3300/0 conn(0x7fc0b813fb20 msgr2=0x7fc0b815a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:37.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.958+0000 7fc0bcf9c700 1 --2- 172.21.15.31:0/3679759898 >> v2:172.21.15.62:3300/0 conn(0x7fc0b813fb20 0x7fc0b815a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:37.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.958+0000 7fc0b6ffd700 1 -- 172.21.15.31:0/3679759898 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3218463430 0 0) 0x7fc0a0003570 con 0x7fc0b813fef0 2023-10-06T02:30:37.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.958+0000 7fc0bcf9c700 1 -- 172.21.15.31:0/3679759898 >> v1:172.21.15.31:6789/0 conn(0x7fc0b813fef0 legacy=0x7fc0b80c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:37.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.958+0000 7fc0bcf9c700 1 -- 172.21.15.31:0/3679759898 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0b806b520 con 0x7fc0b806aaa0 2023-10-06T02:30:37.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.958+0000 7fc0bcf9c700 1 --2- 172.21.15.31:0/3679759898 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc0b806aaa0 0x7fc0b80c9d00 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc0b0009e60 tx=0x7fc0b0009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c814e31c86543e1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:37.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.958+0000 7fc0b6ffd700 1 -- 172.21.15.31:0/3679759898 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b000e070 con 0x7fc0b806aaa0 2023-10-06T02:30:37.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.958+0000 7fc0b6ffd700 1 -- 172.21.15.31:0/3679759898 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0b000b7a0 con 0x7fc0b806aaa0 2023-10-06T02:30:37.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.958+0000 7fc0b6ffd700 1 -- 172.21.15.31:0/3679759898 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0ac001780 con 0x7fc0b806aaa0 2023-10-06T02:30:37.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.959+0000 7fc0bd79d700 1 -- 172.21.15.31:0/3679759898 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc0b806aaa0 msgr2=0x7fc0b80c9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:37.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.959+0000 7fc0bd79d700 1 --2- 172.21.15.31:0/3679759898 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc0b806aaa0 0x7fc0b80c9d00 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc0b0009e60 tx=0x7fc0b0009780 comp rx=0 tx=0).stop 2023-10-06T02:30:37.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.959+0000 7fc0bd79d700 1 -- 172.21.15.31:0/3679759898 shutdown_connections 2023-10-06T02:30:37.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.959+0000 7fc0bd79d700 1 --2- 172.21.15.31:0/3679759898 >> v2:172.21.15.62:3300/0 conn(0x7fc0b813fb20 0x7fc0b815a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:37.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.959+0000 7fc0bd79d700 1 --2- 172.21.15.31:0/3679759898 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc0b806aaa0 0x7fc0b80c9d00 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:37.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.959+0000 7fc0bd79d700 1 -- 172.21.15.31:0/3679759898 >> 172.21.15.31:0/3679759898 conn(0x7fc0b806d170 msgr2=0x7fc0b80cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:37.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.959+0000 7fc0bd79d700 1 -- 172.21.15.31:0/3679759898 shutdown_connections 2023-10-06T02:30:37.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.959+0000 7fc0bd79d700 1 -- 172.21.15.31:0/3679759898 wait complete. 2023-10-06T02:30:37.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.960+0000 7fc0bd79d700 1 Processor -- start 2023-10-06T02:30:37.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.961+0000 7fc0bd79d700 1 -- start start 2023-10-06T02:30:37.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.961+0000 7fc0bd79d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0b806aaa0 0x7fc0b8156950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:37.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.961+0000 7fc0bd79d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc0b813fb20 0x7fc0b81576d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:37.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.961+0000 7fc0bd79d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0b813fef0 0x7fc0b80c1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:37.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.961+0000 7fc0bd79d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b806e560 con 0x7fc0b813fef0 2023-10-06T02:30:37.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.961+0000 7fc0bd79d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b8156e90 con 0x7fc0b806aaa0 2023-10-06T02:30:37.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.961+0000 7fc0bd79d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b816e1f0 con 0x7fc0b813fb20 2023-10-06T02:30:37.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.962+0000 7fc0bcf9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0b813fef0 0x7fc0b80c1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:37.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.962+0000 7fc0bcf9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0b813fef0 0x7fc0b80c1140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35970/0 (socket says 172.21.15.31:35970) 2023-10-06T02:30:37.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.962+0000 7fc0bcf9c700 1 -- 172.21.15.31:0/132793170 learned_addr learned my addr 172.21.15.31:0/132793170 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:37.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.962+0000 7fc0b7fff700 1 --2- 172.21.15.31:0/132793170 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0b806aaa0 0x7fc0b8156950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:37.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.962+0000 7fc0b77fe700 1 --2- 172.21.15.31:0/132793170 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc0b813fb20 0x7fc0b81576d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:37.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.962+0000 7fc0bcf9c700 1 -- 172.21.15.31:0/132793170 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc0b813fb20 msgr2=0x7fc0b81576d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:37.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.962+0000 7fc0bcf9c700 1 --2- 172.21.15.31:0/132793170 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc0b813fb20 0x7fc0b81576d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:37.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.962+0000 7fc0bcf9c700 1 -- 172.21.15.31:0/132793170 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0b806aaa0 msgr2=0x7fc0b8156950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:37.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.962+0000 7fc0bcf9c700 1 --2- 172.21.15.31:0/132793170 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0b806aaa0 0x7fc0b8156950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:37.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.962+0000 7fc0bcf9c700 1 -- 172.21.15.31:0/132793170 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0b0009a20 con 0x7fc0b813fef0 2023-10-06T02:30:37.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.962+0000 7fc0b77fe700 1 --2- 172.21.15.31:0/132793170 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc0b813fb20 0x7fc0b81576d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:30:37.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.962+0000 7fc0bcf9c700 1 --2- 172.21.15.31:0/132793170 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0b813fef0 0x7fc0b80c1140 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fc0b0003000 tx=0x7fc0b0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:37.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.963+0000 7fc09ffff700 1 -- 172.21.15.31:0/132793170 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b0023070 con 0x7fc0b813fef0 2023-10-06T02:30:37.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.963+0000 7fc09ffff700 1 -- 172.21.15.31:0/132793170 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0b0012c70 con 0x7fc0b813fef0 2023-10-06T02:30:37.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.963+0000 7fc0bd79d700 1 -- 172.21.15.31:0/132793170 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0b816d320 con 0x7fc0b813fef0 2023-10-06T02:30:37.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.963+0000 7fc09ffff700 1 -- 172.21.15.31:0/132793170 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b001e430 con 0x7fc0b813fef0 2023-10-06T02:30:37.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.963+0000 7fc0bd79d700 1 -- 172.21.15.31:0/132793170 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0b816d8c0 con 0x7fc0b813fef0 2023-10-06T02:30:37.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.964+0000 7fc0bd79d700 1 -- 172.21.15.31:0/132793170 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0b804c3c0 con 0x7fc0b813fef0 2023-10-06T02:30:37.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.965+0000 7fc09ffff700 1 -- 172.21.15.31:0/132793170 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7fc0b001e7a0 con 0x7fc0b813fef0 2023-10-06T02:30:37.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.966+0000 7fc09ffff700 1 --2- 172.21.15.31:0/132793170 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc0a40674d0 0x7fc0a4069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:37.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.966+0000 7fc09ffff700 1 -- 172.21.15.31:0/132793170 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fc0b0086eb0 con 0x7fc0b813fef0 2023-10-06T02:30:37.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.966+0000 7fc0b7fff700 1 --2- 172.21.15.31:0/132793170 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc0a40674d0 0x7fc0a4069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:37.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.967+0000 7fc0b7fff700 1 --2- 172.21.15.31:0/132793170 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc0a40674d0 0x7fc0a4069980 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc0b806e960 tx=0x7fc0a80091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:37.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:37.970+0000 7fc09ffff700 1 -- 172.21.15.31:0/132793170 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc0b00535e0 con 0x7fc0b813fef0 2023-10-06T02:30:38.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.216+0000 7fc0bd79d700 1 -- 172.21.15.31:0/132793170 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc0b80c2600 con 0x7fc0b813fef0 2023-10-06T02:30:38.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.217+0000 7fc09ffff700 1 -- 172.21.15.31:0/132793170 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4601 (secure 0 0 0) 0x7fc0b0056c00 con 0x7fc0b813fef0 2023-10-06T02:30:38.218 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:30:38.218 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":91,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6611,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/1574790962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1574790962},{"type":"v1","addr":"172.21.15.62:6834","nonce":1574790962}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":85},{"gid":7591,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/2840240410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":2840240410},{"type":"v1","addr":"172.21.15.31:6835","nonce":2840240410}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":85},{"gid":7597,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2500071914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2500071914},{"type":"v1","addr":"172.21.15.62:6835","nonce":2500071914}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:30:11.625173+0000","modified":"2023-10-06T02:30:15.149386+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7971},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7971":{"gid":7971,"name":"a","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6834/1197111339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1197111339},{"type":"v1","addr":"172.21.15.31:6834","nonce":1197111339}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-10-06T02:30:38.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.219+0000 7fc0bd79d700 1 -- 172.21.15.31:0/132793170 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc0a40674d0 msgr2=0x7fc0a4069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:38.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.219+0000 7fc0bd79d700 1 --2- 172.21.15.31:0/132793170 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc0a40674d0 0x7fc0a4069980 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc0b806e960 tx=0x7fc0a80091a0 comp rx=0 tx=0).stop 2023-10-06T02:30:38.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.219+0000 7fc0bd79d700 1 -- 172.21.15.31:0/132793170 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0b813fef0 msgr2=0x7fc0b80c1140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:38.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.219+0000 7fc0bd79d700 1 --2- 172.21.15.31:0/132793170 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0b813fef0 0x7fc0b80c1140 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fc0b0003000 tx=0x7fc0b0018000 comp rx=0 tx=0).stop 2023-10-06T02:30:38.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.219+0000 7fc0bd79d700 1 -- 172.21.15.31:0/132793170 shutdown_connections 2023-10-06T02:30:38.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.219+0000 7fc0bd79d700 1 --2- 172.21.15.31:0/132793170 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0b813fef0 0x7fc0b80c1140 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:38.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.219+0000 7fc0bd79d700 1 --2- 172.21.15.31:0/132793170 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0b806aaa0 0x7fc0b8156950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:38.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.219+0000 7fc0bd79d700 1 --2- 172.21.15.31:0/132793170 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc0a40674d0 0x7fc0a4069980 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:38.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.219+0000 7fc0bd79d700 1 --2- 172.21.15.31:0/132793170 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc0b813fb20 0x7fc0b81576d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:38.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.219+0000 7fc0bd79d700 1 -- 172.21.15.31:0/132793170 >> 172.21.15.31:0/132793170 conn(0x7fc0b806d170 msgr2=0x7fc0b8155b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:38.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.219+0000 7fc0bd79d700 1 -- 172.21.15.31:0/132793170 shutdown_connections 2023-10-06T02:30:38.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.219+0000 7fc0bd79d700 1 -- 172.21.15.31:0/132793170 wait complete. 2023-10-06T02:30:38.222 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 91 2023-10-06T02:30:38.238 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:30:38.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.423+0000 7fd1d3dff700 1 Processor -- start 2023-10-06T02:30:38.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.424+0000 7fd1d3dff700 1 -- start start 2023-10-06T02:30:38.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.425+0000 7fd1d3dff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd1cc0b07d0 0x7fd1cc0ccac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:38.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.425+0000 7fd1d3dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd1cc0cf200 0x7fd1cc0cd000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:38.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.425+0000 7fd1d3dff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1cc15a280 con 0x7fd1cc0b0ba0 2023-10-06T02:30:38.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.425+0000 7fd1d3dff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1cc06e180 con 0x7fd1cc0b07d0 2023-10-06T02:30:38.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.425+0000 7fd1d3dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1cc0cd540 con 0x7fd1cc0cf200 2023-10-06T02:30:38.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.425+0000 7fd1d239c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fd1cc0b0ba0 0x7fd1cc06e610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60738/0 (socket says 172.21.15.31:60738) 2023-10-06T02:30:38.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.425+0000 7fd1d1b9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd1cc0b07d0 0x7fd1cc0ccac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:38.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.425+0000 7fd1d239c700 1 -- 172.21.15.31:0/399854509 learned_addr learned my addr 172.21.15.31:0/399854509 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:30:38.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.426+0000 7fd1d139a700 1 --2- 172.21.15.31:0/399854509 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd1cc0cf200 0x7fd1cc0cd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:38.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.426+0000 7fd1d0b99700 1 -- 172.21.15.31:0/399854509 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1332816048 0 0) 0x7fd1cc15a280 con 0x7fd1cc0b0ba0 2023-10-06T02:30:38.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.426+0000 7fd1d0b99700 1 -- 172.21.15.31:0/399854509 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1b8003570 con 0x7fd1cc0b0ba0 2023-10-06T02:30:38.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.426+0000 7fd1d1b9b700 1 -- 172.21.15.31:0/399854509 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd1cc0cf200 msgr2=0x7fd1cc0cd000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:38.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.426+0000 7fd1d1b9b700 1 --2- 172.21.15.31:0/399854509 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd1cc0cf200 0x7fd1cc0cd000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:38.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.426+0000 7fd1d1b9b700 1 -- 172.21.15.31:0/399854509 >> v1:172.21.15.31:6789/0 conn(0x7fd1cc0b0ba0 legacy=0x7fd1cc06e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:38.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.426+0000 7fd1d1b9b700 1 -- 172.21.15.31:0/399854509 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1cc06d850 con 0x7fd1cc0b07d0 2023-10-06T02:30:38.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.426+0000 7fd1d139a700 1 --2- 172.21.15.31:0/399854509 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd1cc0cf200 0x7fd1cc0cd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:30:38.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.426+0000 7fd1d1b9b700 1 --2- 172.21.15.31:0/399854509 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1cc0b07d0 0x7fd1cc0ccac0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fd1c80098d0 tx=0x7fd1c8009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=727ee3ae8731d9a1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:38.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.426+0000 7fd1d0b99700 1 -- 172.21.15.31:0/399854509 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c8005c10 con 0x7fd1cc0b07d0 2023-10-06T02:30:38.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.426+0000 7fd1d0b99700 1 -- 172.21.15.31:0/399854509 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1c8005d70 con 0x7fd1cc0b07d0 2023-10-06T02:30:38.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.426+0000 7fd1d0b99700 1 -- 172.21.15.31:0/399854509 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c8006280 con 0x7fd1cc0b07d0 2023-10-06T02:30:38.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.427+0000 7fd1d3dff700 1 -- 172.21.15.31:0/399854509 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1cc0b07d0 msgr2=0x7fd1cc0ccac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:38.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.427+0000 7fd1d3dff700 1 --2- 172.21.15.31:0/399854509 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1cc0b07d0 0x7fd1cc0ccac0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fd1c80098d0 tx=0x7fd1c8009120 comp rx=0 tx=0).stop 2023-10-06T02:30:38.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.427+0000 7fd1d3dff700 1 -- 172.21.15.31:0/399854509 shutdown_connections 2023-10-06T02:30:38.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.427+0000 7fd1d3dff700 1 --2- 172.21.15.31:0/399854509 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1cc0b07d0 0x7fd1cc0ccac0 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:38.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.427+0000 7fd1d3dff700 1 --2- 172.21.15.31:0/399854509 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd1cc0cf200 0x7fd1cc0cd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:38.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.427+0000 7fd1d3dff700 1 -- 172.21.15.31:0/399854509 >> 172.21.15.31:0/399854509 conn(0x7fd1cc06ce80 msgr2=0x7fd1cc0ce510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:38.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.427+0000 7fd1d3dff700 1 -- 172.21.15.31:0/399854509 shutdown_connections 2023-10-06T02:30:38.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.427+0000 7fd1d3dff700 1 -- 172.21.15.31:0/399854509 wait complete. 2023-10-06T02:30:38.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.428+0000 7fd1d3dff700 1 Processor -- start 2023-10-06T02:30:38.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.429+0000 7fd1d3dff700 1 -- start start 2023-10-06T02:30:38.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.429+0000 7fd1d3dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd1cc0b07d0 0x7fd1cc0b1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:38.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.429+0000 7fd1d3dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1cc0b0ba0 0x7fd1cc14c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:38.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.430+0000 7fd1d3dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1cc0cf200 0x7fd1cc0b2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:38.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.430+0000 7fd1d3dff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1cc0b28b0 con 0x7fd1cc0b0ba0 2023-10-06T02:30:38.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.430+0000 7fd1d3dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1cc0b19d0 con 0x7fd1cc0cf200 2023-10-06T02:30:38.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.430+0000 7fd1d3dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1cc0b1ca0 con 0x7fd1cc0b07d0 2023-10-06T02:30:38.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.430+0000 7fd1d139a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1cc0b0ba0 0x7fd1cc14c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:38.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.430+0000 7fd1d139a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1cc0b0ba0 0x7fd1cc14c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35988/0 (socket says 172.21.15.31:35988) 2023-10-06T02:30:38.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.430+0000 7fd1d139a700 1 -- 172.21.15.31:0/431118669 learned_addr learned my addr 172.21.15.31:0/431118669 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:38.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.430+0000 7fd1d1b9b700 1 --2- 172.21.15.31:0/431118669 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd1cc0b07d0 0x7fd1cc0b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:38.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.430+0000 7fd1d239c700 1 --2- 172.21.15.31:0/431118669 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1cc0cf200 0x7fd1cc0b2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:38.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.430+0000 7fd1d139a700 1 -- 172.21.15.31:0/431118669 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd1cc0b07d0 msgr2=0x7fd1cc0b1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:38.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.430+0000 7fd1d139a700 1 --2- 172.21.15.31:0/431118669 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd1cc0b07d0 0x7fd1cc0b1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:38.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.430+0000 7fd1d139a700 1 -- 172.21.15.31:0/431118669 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1cc0cf200 msgr2=0x7fd1cc0b2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:38.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.430+0000 7fd1d139a700 1 --2- 172.21.15.31:0/431118669 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1cc0cf200 0x7fd1cc0b2ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:38.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.430+0000 7fd1d139a700 1 -- 172.21.15.31:0/431118669 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1c80093c0 con 0x7fd1cc0b0ba0 2023-10-06T02:30:38.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.430+0000 7fd1d1b9b700 1 --2- 172.21.15.31:0/431118669 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd1cc0b07d0 0x7fd1cc0b1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:30:38.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.430+0000 7fd1d239c700 1 --2- 172.21.15.31:0/431118669 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1cc0cf200 0x7fd1cc0b2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:30:38.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.431+0000 7fd1d139a700 1 --2- 172.21.15.31:0/431118669 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1cc0b0ba0 0x7fd1cc14c8a0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc00d860 tx=0x7fd1bc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:38.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.431+0000 7fd1c1ffb700 1 -- 172.21.15.31:0/431118669 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1bc0125f0 con 0x7fd1cc0b0ba0 2023-10-06T02:30:38.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.431+0000 7fd1c1ffb700 1 -- 172.21.15.31:0/431118669 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1bc012750 con 0x7fd1cc0b0ba0 2023-10-06T02:30:38.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.431+0000 7fd1c1ffb700 1 -- 172.21.15.31:0/431118669 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1bc012ad0 con 0x7fd1cc0b0ba0 2023-10-06T02:30:38.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.431+0000 7fd1d3dff700 1 -- 172.21.15.31:0/431118669 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1cc0b1f80 con 0x7fd1cc0b0ba0 2023-10-06T02:30:38.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.431+0000 7fd1d3dff700 1 -- 172.21.15.31:0/431118669 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1cc0b2540 con 0x7fd1cc0b0ba0 2023-10-06T02:30:38.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.432+0000 7fd1d3dff700 1 -- 172.21.15.31:0/431118669 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1cc0b2a60 con 0x7fd1cc0b0ba0 2023-10-06T02:30:38.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.433+0000 7fd1c1ffb700 1 -- 172.21.15.31:0/431118669 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7fd1bc014430 con 0x7fd1cc0b0ba0 2023-10-06T02:30:38.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.434+0000 7fd1c1ffb700 1 --2- 172.21.15.31:0/431118669 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd1b00674d0 0x7fd1b0069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:38.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.434+0000 7fd1c1ffb700 1 -- 172.21.15.31:0/431118669 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fd1bc085640 con 0x7fd1cc0b0ba0 2023-10-06T02:30:38.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.434+0000 7fd1d1b9b700 1 --2- 172.21.15.31:0/431118669 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd1b00674d0 0x7fd1b0069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:38.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.435+0000 7fd1d1b9b700 1 --2- 172.21.15.31:0/431118669 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd1b00674d0 0x7fd1b0069980 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd1c8009160 tx=0x7fd1c8016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:38.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.438+0000 7fd1c1ffb700 1 -- 172.21.15.31:0/431118669 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd1cc0b2a60 con 0x7fd1cc0b0ba0 2023-10-06T02:30:38.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.682+0000 7fd1d3dff700 1 -- 172.21.15.31:0/431118669 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd1cc04c3c0 con 0x7fd1cc0b0ba0 2023-10-06T02:30:38.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.683+0000 7fd1c1ffb700 1 -- 172.21.15.31:0/431118669 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4601 (secure 0 0 0) 0x7fd1bc055910 con 0x7fd1cc0b0ba0 2023-10-06T02:30:38.685 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:30:38.685 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":91,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6611,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/1574790962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1574790962},{"type":"v1","addr":"172.21.15.62:6834","nonce":1574790962}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":85},{"gid":7591,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/2840240410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":2840240410},{"type":"v1","addr":"172.21.15.31:6835","nonce":2840240410}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":85},{"gid":7597,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2500071914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2500071914},{"type":"v1","addr":"172.21.15.62:6835","nonce":2500071914}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:30:11.625173+0000","modified":"2023-10-06T02:30:15.149386+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7971},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7971":{"gid":7971,"name":"a","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6834/1197111339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1197111339},{"type":"v1","addr":"172.21.15.31:6834","nonce":1197111339}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-10-06T02:30:38.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.685+0000 7fd1d3dff700 1 -- 172.21.15.31:0/431118669 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd1b00674d0 msgr2=0x7fd1b0069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:38.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.685+0000 7fd1d3dff700 1 --2- 172.21.15.31:0/431118669 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd1b00674d0 0x7fd1b0069980 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd1c8009160 tx=0x7fd1c8016000 comp rx=0 tx=0).stop 2023-10-06T02:30:38.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.685+0000 7fd1d3dff700 1 -- 172.21.15.31:0/431118669 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1cc0b0ba0 msgr2=0x7fd1cc14c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:38.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.685+0000 7fd1d3dff700 1 --2- 172.21.15.31:0/431118669 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1cc0b0ba0 0x7fd1cc14c8a0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc00d860 tx=0x7fd1bc008040 comp rx=0 tx=0).stop 2023-10-06T02:30:38.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.685+0000 7fd1d3dff700 1 -- 172.21.15.31:0/431118669 shutdown_connections 2023-10-06T02:30:38.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.685+0000 7fd1d3dff700 1 --2- 172.21.15.31:0/431118669 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1cc0cf200 0x7fd1cc0b2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:38.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.685+0000 7fd1d3dff700 1 --2- 172.21.15.31:0/431118669 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd1b00674d0 0x7fd1b0069980 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:38.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.685+0000 7fd1d3dff700 1 --2- 172.21.15.31:0/431118669 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd1cc0b07d0 0x7fd1cc0b1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:38.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.685+0000 7fd1d3dff700 1 --2- 172.21.15.31:0/431118669 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1cc0b0ba0 0x7fd1cc14c8a0 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:38.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.685+0000 7fd1d3dff700 1 -- 172.21.15.31:0/431118669 >> 172.21.15.31:0/431118669 conn(0x7fd1cc06ce80 msgr2=0x7fd1cc0c8df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:38.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.686+0000 7fd1d3dff700 1 -- 172.21.15.31:0/431118669 shutdown_connections 2023-10-06T02:30:38.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.686+0000 7fd1d3dff700 1 -- 172.21.15.31:0/431118669 wait complete. 2023-10-06T02:30:38.689 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 91 2023-10-06T02:30:38.705 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:30:38.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.892+0000 7f188e19d700 1 Processor -- start 2023-10-06T02:30:38.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.892+0000 7f188e19d700 1 -- start start 2023-10-06T02:30:38.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.893+0000 7f188e19d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1888140e40 0x7f18880c6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:38.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.893+0000 7f188e19d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18880c6d50 con 0x7f188806de50 2023-10-06T02:30:38.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.893+0000 7f188e19d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18880c7030 con 0x7f1888140a70 2023-10-06T02:30:38.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.893+0000 7f188e19d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18880c7310 con 0x7f1888140e40 2023-10-06T02:30:38.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.893+0000 7f188c99a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1888140e40 0x7f18880c6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:38.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.893+0000 7f188d99c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f188806de50 0x7f18880aeef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60754/0 (socket says 172.21.15.31:60754) 2023-10-06T02:30:38.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.893+0000 7f188d99c700 1 -- 172.21.15.31:0/2607729909 learned_addr learned my addr 172.21.15.31:0/2607729909 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:30:38.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.894+0000 7f188c99a700 1 -- 172.21.15.31:0/2607729909 >> v1:172.21.15.62:6789/0 conn(0x7f1888140a70 legacy=0x7f18880c7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:38.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.894+0000 7f188c99a700 1 -- 172.21.15.31:0/2607729909 >> v1:172.21.15.31:6789/0 conn(0x7f188806de50 legacy=0x7f18880aeef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:38.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.894+0000 7f188c99a700 1 -- 172.21.15.31:0/2607729909 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18880c75f0 con 0x7f1888140e40 2023-10-06T02:30:38.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.894+0000 7f187ffff700 1 -- 172.21.15.31:0/2607729909 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2342233066 0 0) 0x7f18880c6d50 con 0x7f188806de50 2023-10-06T02:30:38.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.894+0000 7f188c99a700 1 --2- 172.21.15.31:0/2607729909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1888140e40 0x7f18880c6770 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f18780098d0 tx=0x7f1878009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30206fb302899768 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:38.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.894+0000 7f187ffff700 1 -- 172.21.15.31:0/2607729909 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1878009e70 con 0x7f1888140e40 2023-10-06T02:30:38.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.894+0000 7f187ffff700 1 -- 172.21.15.31:0/2607729909 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1878005c10 con 0x7f1888140e40 2023-10-06T02:30:38.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.894+0000 7f187ffff700 1 -- 172.21.15.31:0/2607729909 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f187800b650 con 0x7f1888140e40 2023-10-06T02:30:38.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.895+0000 7f188e19d700 1 -- 172.21.15.31:0/2607729909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1888140e40 msgr2=0x7f18880c6770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:38.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.895+0000 7f188e19d700 1 --2- 172.21.15.31:0/2607729909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1888140e40 0x7f18880c6770 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f18780098d0 tx=0x7f1878009120 comp rx=0 tx=0).stop 2023-10-06T02:30:38.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.895+0000 7f188e19d700 1 -- 172.21.15.31:0/2607729909 shutdown_connections 2023-10-06T02:30:38.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.895+0000 7f188e19d700 1 --2- 172.21.15.31:0/2607729909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1888140e40 0x7f18880c6770 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:38.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.895+0000 7f188e19d700 1 -- 172.21.15.31:0/2607729909 >> 172.21.15.31:0/2607729909 conn(0x7f188806d170 msgr2=0x7f18880cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:38.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.895+0000 7f188e19d700 1 -- 172.21.15.31:0/2607729909 shutdown_connections 2023-10-06T02:30:38.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.895+0000 7f188e19d700 1 -- 172.21.15.31:0/2607729909 wait complete. 2023-10-06T02:30:38.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.896+0000 7f188e19d700 1 Processor -- start 2023-10-06T02:30:38.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.897+0000 7f188e19d700 1 -- start start 2023-10-06T02:30:38.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.897+0000 7f188e19d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f188806de50 0x7f188813d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:38.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.897+0000 7f188e19d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1888140a70 0x7f18880c3f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:38.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.897+0000 7f188e19d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1888140e40 0x7f188813e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:38.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.897+0000 7f188e19d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f188816ace0 con 0x7f1888140e40 2023-10-06T02:30:38.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.897+0000 7f188e19d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f188813d700 con 0x7f1888140a70 2023-10-06T02:30:38.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.897+0000 7f188e19d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1888169e00 con 0x7f188806de50 2023-10-06T02:30:38.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.898+0000 7f188d99c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1888140e40 0x7f188813e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:38.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.898+0000 7f188d99c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1888140e40 0x7f188813e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36008/0 (socket says 172.21.15.31:36008) 2023-10-06T02:30:38.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.898+0000 7f188d99c700 1 -- 172.21.15.31:0/3861889472 learned_addr learned my addr 172.21.15.31:0/3861889472 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:38.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.898+0000 7f188c99a700 1 --2- 172.21.15.31:0/3861889472 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1888140a70 0x7f18880c3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:38.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.898+0000 7f188d19b700 1 --2- 172.21.15.31:0/3861889472 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f188806de50 0x7f188813d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:38.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.898+0000 7f188d99c700 1 -- 172.21.15.31:0/3861889472 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f188806de50 msgr2=0x7f188813d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:38.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.898+0000 7f188d99c700 1 --2- 172.21.15.31:0/3861889472 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f188806de50 0x7f188813d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:38.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.898+0000 7f188d99c700 1 -- 172.21.15.31:0/3861889472 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1888140a70 msgr2=0x7f18880c3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:38.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.898+0000 7f188d99c700 1 --2- 172.21.15.31:0/3861889472 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1888140a70 0x7f18880c3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:38.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.898+0000 7f188d99c700 1 -- 172.21.15.31:0/3861889472 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18780093c0 con 0x7f1888140e40 2023-10-06T02:30:38.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.898+0000 7f188c99a700 1 --2- 172.21.15.31:0/3861889472 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1888140a70 0x7f18880c3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:30:38.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.898+0000 7f188d99c700 1 --2- 172.21.15.31:0/3861889472 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1888140e40 0x7f188813e3a0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f188000ccc0 tx=0x7f1880001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:38.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.899+0000 7f187d7fa700 1 -- 172.21.15.31:0/3861889472 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f188000a950 con 0x7f1888140e40 2023-10-06T02:30:38.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.899+0000 7f187d7fa700 1 -- 172.21.15.31:0/3861889472 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f188000aab0 con 0x7f1888140e40 2023-10-06T02:30:38.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.899+0000 7f188e19d700 1 -- 172.21.15.31:0/3861889472 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f188816a0d0 con 0x7f1888140e40 2023-10-06T02:30:38.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.899+0000 7f187d7fa700 1 -- 172.21.15.31:0/3861889472 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f188000ae70 con 0x7f1888140e40 2023-10-06T02:30:38.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.899+0000 7f188e19d700 1 -- 172.21.15.31:0/3861889472 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f188816a690 con 0x7f1888140e40 2023-10-06T02:30:38.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.900+0000 7f188e19d700 1 -- 172.21.15.31:0/3861889472 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f188816ae20 con 0x7f1888140e40 2023-10-06T02:30:38.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.901+0000 7f187d7fa700 1 -- 172.21.15.31:0/3861889472 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f1880007300 con 0x7f1888140e40 2023-10-06T02:30:38.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.901+0000 7f187d7fa700 1 --2- 172.21.15.31:0/3861889472 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f186c0674d0 0x7f186c069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:38.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.902+0000 7f187d7fa700 1 -- 172.21.15.31:0/3861889472 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f1880085570 con 0x7f1888140e40 2023-10-06T02:30:38.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.902+0000 7f188d19b700 1 --2- 172.21.15.31:0/3861889472 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f186c0674d0 0x7f186c069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:38.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.903+0000 7f188d19b700 1 --2- 172.21.15.31:0/3861889472 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f186c0674d0 0x7f186c069980 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f1884003a80 tx=0x7f1884007ec0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:38.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:38.906+0000 7f187d7fa700 1 -- 172.21.15.31:0/3861889472 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1880051e30 con 0x7f1888140e40 2023-10-06T02:30:39.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.111+0000 7f188e19d700 1 -- 172.21.15.31:0/3861889472 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f188804c3c0 con 0x7f1888140e40 2023-10-06T02:30:39.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.112+0000 7f187d7fa700 1 -- 172.21.15.31:0/3861889472 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v132) v1 ==== 74+0+16876 (secure 0 0 0) 0x7f1880055450 con 0x7f1888140e40 2023-10-06T02:30:39.114 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:30:39.114 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":132,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:30:13.138416+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:30:09.791433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-10-06T02:30:10.601699+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":20,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:30:39.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.115+0000 7f188e19d700 1 -- 172.21.15.31:0/3861889472 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f186c0674d0 msgr2=0x7f186c069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:39.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.115+0000 7f188e19d700 1 --2- 172.21.15.31:0/3861889472 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f186c0674d0 0x7f186c069980 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f1884003a80 tx=0x7f1884007ec0 comp rx=0 tx=0).stop 2023-10-06T02:30:39.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.115+0000 7f188e19d700 1 -- 172.21.15.31:0/3861889472 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1888140e40 msgr2=0x7f188813e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:39.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.115+0000 7f188e19d700 1 --2- 172.21.15.31:0/3861889472 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1888140e40 0x7f188813e3a0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f188000ccc0 tx=0x7f1880001a80 comp rx=0 tx=0).stop 2023-10-06T02:30:39.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.115+0000 7f188e19d700 1 -- 172.21.15.31:0/3861889472 shutdown_connections 2023-10-06T02:30:39.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.115+0000 7f188e19d700 1 --2- 172.21.15.31:0/3861889472 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1888140e40 0x7f188813e3a0 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:39.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.115+0000 7f188e19d700 1 --2- 172.21.15.31:0/3861889472 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f186c0674d0 0x7f186c069980 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:39.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.115+0000 7f188e19d700 1 --2- 172.21.15.31:0/3861889472 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f188806de50 0x7f188813d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:39.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.115+0000 7f188e19d700 1 --2- 172.21.15.31:0/3861889472 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1888140a70 0x7f18880c3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:39.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.115+0000 7f188e19d700 1 -- 172.21.15.31:0/3861889472 >> 172.21.15.31:0/3861889472 conn(0x7f188806d170 msgr2=0x7f18880aef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:39.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.115+0000 7f188e19d700 1 -- 172.21.15.31:0/3861889472 shutdown_connections 2023-10-06T02:30:39.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.115+0000 7f188e19d700 1 -- 172.21.15.31:0/3861889472 wait complete. 2023-10-06T02:30:39.134 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T02:30:39.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.326+0000 7f186519d700 1 Processor -- start 2023-10-06T02:30:39.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.326+0000 7f186519d700 1 -- start start 2023-10-06T02:30:39.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.328+0000 7f186519d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f1860142c00 0x7f186006b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:39.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.328+0000 7f186519d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f18600c7f90 0x7f186006c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:39.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.328+0000 7f186519d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18600ca480 con 0x7f186006bd20 2023-10-06T02:30:39.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.328+0000 7f186519d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f186015e240 con 0x7f18600c7f90 2023-10-06T02:30:39.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.328+0000 7f186519d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f186006b350 con 0x7f1860142c00 2023-10-06T02:30:39.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.328+0000 7f186499c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f18600c7f90 0x7f186006c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:39.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.328+0000 7f186499c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f18600c7f90 0x7f186006c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36018/0 (socket says 172.21.15.31:36018) 2023-10-06T02:30:39.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.328+0000 7f186499c700 1 -- 172.21.15.31:0/3436353840 learned_addr learned my addr 172.21.15.31:0/3436353840 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:39.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.328+0000 7f185ffff700 1 --2- 172.21.15.31:0/3436353840 >> v2:172.21.15.62:3300/0 conn(0x7f1860142c00 0x7f186006b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:39.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.329+0000 7f186499c700 1 -- 172.21.15.31:0/3436353840 >> v2:172.21.15.62:3300/0 conn(0x7f1860142c00 msgr2=0x7f186006b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:39.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.329+0000 7f186499c700 1 --2- 172.21.15.31:0/3436353840 >> v2:172.21.15.62:3300/0 conn(0x7f1860142c00 0x7f186006b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:39.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.329+0000 7f186499c700 1 -- 172.21.15.31:0/3436353840 >> v1:172.21.15.62:6789/0 conn(0x7f186006bd20 legacy=0x7f18600c7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:39.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.329+0000 7f186499c700 1 -- 172.21.15.31:0/3436353840 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f186015df60 con 0x7f18600c7f90 2023-10-06T02:30:39.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.329+0000 7f186499c700 1 --2- 172.21.15.31:0/3436353840 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18600c7f90 0x7f186006c6a0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f186006f490 tx=0x7f1858009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ce7659403deda8d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:39.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.329+0000 7f185effd700 1 -- 172.21.15.31:0/3436353840 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185800e070 con 0x7f18600c7f90 2023-10-06T02:30:39.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.329+0000 7f185effd700 1 -- 172.21.15.31:0/3436353840 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f185800ba20 con 0x7f18600c7f90 2023-10-06T02:30:39.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.329+0000 7f185effd700 1 -- 172.21.15.31:0/3436353840 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18580092e0 con 0x7f18600c7f90 2023-10-06T02:30:39.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.330+0000 7f186519d700 1 -- 172.21.15.31:0/3436353840 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18600c7f90 msgr2=0x7f186006c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:39.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.330+0000 7f186519d700 1 --2- 172.21.15.31:0/3436353840 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18600c7f90 0x7f186006c6a0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f186006f490 tx=0x7f1858009840 comp rx=0 tx=0).stop 2023-10-06T02:30:39.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.330+0000 7f186519d700 1 -- 172.21.15.31:0/3436353840 shutdown_connections 2023-10-06T02:30:39.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.330+0000 7f186519d700 1 --2- 172.21.15.31:0/3436353840 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18600c7f90 0x7f186006c6a0 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:39.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.330+0000 7f186519d700 1 --2- 172.21.15.31:0/3436353840 >> v2:172.21.15.62:3300/0 conn(0x7f1860142c00 0x7f186006b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:39.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.330+0000 7f186519d700 1 -- 172.21.15.31:0/3436353840 >> 172.21.15.31:0/3436353840 conn(0x7f18600675a0 msgr2=0x7f18600679a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:39.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.330+0000 7f186519d700 1 -- 172.21.15.31:0/3436353840 shutdown_connections 2023-10-06T02:30:39.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.330+0000 7f186519d700 1 -- 172.21.15.31:0/3436353840 wait complete. 2023-10-06T02:30:39.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.331+0000 7f186519d700 1 Processor -- start 2023-10-06T02:30:39.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.331+0000 7f186519d700 1 -- start start 2023-10-06T02:30:39.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.332+0000 7f186519d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f186006bd20 0x7f1860159180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:39.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.332+0000 7f186519d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18600c7f90 0x7f186013f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:39.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.332+0000 7f186519d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1860142c00 0x7f18600b0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:39.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.332+0000 7f186519d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18600b03f0 con 0x7f186006bd20 2023-10-06T02:30:39.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.332+0000 7f186519d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f186013e300 con 0x7f18600c7f90 2023-10-06T02:30:39.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.332+0000 7f186519d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f186013e5e0 con 0x7f1860142c00 2023-10-06T02:30:39.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.333+0000 7f185ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f186006bd20 0x7f1860159180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:39.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.333+0000 7f185ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f186006bd20 0x7f1860159180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36032/0 (socket says 172.21.15.31:36032) 2023-10-06T02:30:39.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.333+0000 7f185ffff700 1 -- 172.21.15.31:0/2443151597 learned_addr learned my addr 172.21.15.31:0/2443151597 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:39.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.333+0000 7f186499c700 1 --2- 172.21.15.31:0/2443151597 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1860142c00 0x7f18600b0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:39.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.333+0000 7f185f7fe700 1 --2- 172.21.15.31:0/2443151597 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18600c7f90 0x7f186013f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:39.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.333+0000 7f185ffff700 1 -- 172.21.15.31:0/2443151597 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1860142c00 msgr2=0x7f18600b0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:39.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.333+0000 7f185ffff700 1 --2- 172.21.15.31:0/2443151597 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1860142c00 0x7f18600b0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:39.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.333+0000 7f185ffff700 1 -- 172.21.15.31:0/2443151597 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18600c7f90 msgr2=0x7f186013f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:39.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.333+0000 7f185ffff700 1 --2- 172.21.15.31:0/2443151597 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18600c7f90 0x7f186013f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:39.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.333+0000 7f185ffff700 1 -- 172.21.15.31:0/2443151597 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1858009ae0 con 0x7f186006bd20 2023-10-06T02:30:39.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.333+0000 7f185ffff700 1 --2- 172.21.15.31:0/2443151597 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f186006bd20 0x7f1860159180 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f185000e930 tx=0x7f18500091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:39.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.334+0000 7f1847fff700 1 -- 172.21.15.31:0/2443151597 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1850013680 con 0x7f186006bd20 2023-10-06T02:30:39.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.334+0000 7f1847fff700 1 -- 172.21.15.31:0/2443151597 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18500137e0 con 0x7f186006bd20 2023-10-06T02:30:39.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.334+0000 7f1847fff700 1 -- 172.21.15.31:0/2443151597 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1850013b80 con 0x7f186006bd20 2023-10-06T02:30:39.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.334+0000 7f186519d700 1 -- 172.21.15.31:0/2443151597 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f186013e8c0 con 0x7f186006bd20 2023-10-06T02:30:39.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.334+0000 7f186519d700 1 -- 172.21.15.31:0/2443151597 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18601596c0 con 0x7f186006bd20 2023-10-06T02:30:39.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.335+0000 7f186519d700 1 -- 172.21.15.31:0/2443151597 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f186004c3c0 con 0x7f186006bd20 2023-10-06T02:30:39.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.336+0000 7f1847fff700 1 -- 172.21.15.31:0/2443151597 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f185000c3e0 con 0x7f186006bd20 2023-10-06T02:30:39.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.337+0000 7f1847fff700 1 --2- 172.21.15.31:0/2443151597 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f184c0674d0 0x7f184c069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:39.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.337+0000 7f1847fff700 1 -- 172.21.15.31:0/2443151597 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6173+0+0 (secure 0 0 0) 0x7f1850085220 con 0x7f186006bd20 2023-10-06T02:30:39.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.337+0000 7f185f7fe700 1 --2- 172.21.15.31:0/2443151597 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f184c0674d0 0x7f184c069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:39.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.338+0000 7f185f7fe700 1 --2- 172.21.15.31:0/2443151597 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f184c0674d0 0x7f184c069980 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1860140070 tx=0x7f1854007ec0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:39.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.341+0000 7f1847fff700 1 -- 172.21.15.31:0/2443151597 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1850051a50 con 0x7f186006bd20 2023-10-06T02:30:39.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.592+0000 7f186519d700 1 -- 172.21.15.31:0/2443151597 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f18600b06d0 con 0x7f186006bd20 2023-10-06T02:30:39.594 INFO:tasks.ceph.mon.a.smithi031.stderr:2023-10-06T02:30:39.592+0000 7f6de01f1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T02:30:39.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.597+0000 7f1847fff700 1 -- 172.21.15.31:0/2443151597 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v92) v1 ==== 161+0+0 (secure 0 0 0) 0x7f1850055070 con 0x7f186006bd20 2023-10-06T02:30:39.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.600+0000 7f186519d700 1 -- 172.21.15.31:0/2443151597 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f184c0674d0 msgr2=0x7f184c069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:39.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.601+0000 7f186519d700 1 --2- 172.21.15.31:0/2443151597 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f184c0674d0 0x7f184c069980 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1860140070 tx=0x7f1854007ec0 comp rx=0 tx=0).stop 2023-10-06T02:30:39.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.601+0000 7f186519d700 1 -- 172.21.15.31:0/2443151597 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f186006bd20 msgr2=0x7f1860159180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:39.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.601+0000 7f186519d700 1 --2- 172.21.15.31:0/2443151597 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f186006bd20 0x7f1860159180 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f185000e930 tx=0x7f18500091a0 comp rx=0 tx=0).stop 2023-10-06T02:30:39.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.601+0000 7f186519d700 1 -- 172.21.15.31:0/2443151597 shutdown_connections 2023-10-06T02:30:39.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.601+0000 7f186519d700 1 --2- 172.21.15.31:0/2443151597 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f184c0674d0 0x7f184c069980 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:39.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.601+0000 7f186519d700 1 --2- 172.21.15.31:0/2443151597 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1860142c00 0x7f18600b0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:39.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.601+0000 7f186519d700 1 --2- 172.21.15.31:0/2443151597 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f186006bd20 0x7f1860159180 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:39.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.601+0000 7f186519d700 1 --2- 172.21.15.31:0/2443151597 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18600c7f90 0x7f186013f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:39.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.601+0000 7f186519d700 1 -- 172.21.15.31:0/2443151597 >> 172.21.15.31:0/2443151597 conn(0x7f18600675a0 msgr2=0x7f18600ca8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:39.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.601+0000 7f186519d700 1 -- 172.21.15.31:0/2443151597 shutdown_connections 2023-10-06T02:30:39.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.601+0000 7f186519d700 1 -- 172.21.15.31:0/2443151597 wait complete. 2023-10-06T02:30:39.605 INFO:teuthology.orchestra.run.smithi031.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T02:30:39.606 INFO:tasks.ceph.mds.a.smithi031.stderr: -1242> 2023-10-06T02:30:17.097+0000 7f70a29f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T02:30:39.621 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T02:30:39.644 INFO:tasks.ceph.mds.a.smithi031.stdout:starting mds.a at 2023-10-06T02:30:39.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.814+0000 7f9be49ff700 1 Processor -- start 2023-10-06T02:30:39.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.814+0000 7f9be49ff700 1 -- start start 2023-10-06T02:30:39.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.815+0000 7f9be49ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9be013c9b0 0x7f9be0069a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:39.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.815+0000 7f9be49ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9be00cc6f0 0x7f9be0069f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:39.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.815+0000 7f9be49ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9be006d7b0 con 0x7f9be00cc320 2023-10-06T02:30:39.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.815+0000 7f9be49ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9be006f7d0 con 0x7f9be00cc6f0 2023-10-06T02:30:39.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.815+0000 7f9be49ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9be006e330 con 0x7f9be013c9b0 2023-10-06T02:30:39.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.815+0000 7f9bde59c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f9be00cc320 0x7f9be013c2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60766/0 (socket says 172.21.15.31:60766) 2023-10-06T02:30:39.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.815+0000 7f9bde59c700 1 -- 172.21.15.31:0/1341315781 learned_addr learned my addr 172.21.15.31:0/1341315781 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:30:39.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.816+0000 7f9bddd9b700 1 --2- 172.21.15.31:0/1341315781 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9be013c9b0 0x7f9be0069a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:39.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.816+0000 7f9bded9d700 1 --2- 172.21.15.31:0/1341315781 >> v2:172.21.15.62:3300/0 conn(0x7f9be00cc6f0 0x7f9be0069f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:39.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.816+0000 7f9bdd59a700 1 -- 172.21.15.31:0/1341315781 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3961069492 0 0) 0x7f9be006d7b0 con 0x7f9be00cc320 2023-10-06T02:30:39.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.816+0000 7f9bdd59a700 1 -- 172.21.15.31:0/1341315781 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9bc8003570 con 0x7f9be00cc320 2023-10-06T02:30:39.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.816+0000 7f9bded9d700 1 -- 172.21.15.31:0/1341315781 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9be013c9b0 msgr2=0x7f9be0069a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:39.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.816+0000 7f9bded9d700 1 --2- 172.21.15.31:0/1341315781 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9be013c9b0 0x7f9be0069a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:39.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.816+0000 7f9bded9d700 1 -- 172.21.15.31:0/1341315781 >> v1:172.21.15.31:6789/0 conn(0x7f9be00cc320 legacy=0x7f9be013c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:39.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.816+0000 7f9bded9d700 1 -- 172.21.15.31:0/1341315781 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9be006da00 con 0x7f9be00cc6f0 2023-10-06T02:30:39.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.816+0000 7f9bddd9b700 1 --2- 172.21.15.31:0/1341315781 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9be013c9b0 0x7f9be0069a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:30:39.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.817+0000 7f9bded9d700 1 --2- 172.21.15.31:0/1341315781 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9be00cc6f0 0x7f9be0069f40 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f9bd8009e60 tx=0x7f9bd8009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2179cc2f00b1b839 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:39.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.817+0000 7f9bdd59a700 1 -- 172.21.15.31:0/1341315781 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd800e070 con 0x7f9be00cc6f0 2023-10-06T02:30:39.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.817+0000 7f9bdd59a700 1 -- 172.21.15.31:0/1341315781 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bd800bab0 con 0x7f9be00cc6f0 2023-10-06T02:30:39.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.817+0000 7f9bdd59a700 1 -- 172.21.15.31:0/1341315781 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd800be70 con 0x7f9be00cc6f0 2023-10-06T02:30:39.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.817+0000 7f9be49ff700 1 -- 172.21.15.31:0/1341315781 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9be00cc6f0 msgr2=0x7f9be0069f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:39.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.817+0000 7f9be49ff700 1 --2- 172.21.15.31:0/1341315781 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9be00cc6f0 0x7f9be0069f40 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f9bd8009e60 tx=0x7f9bd8009780 comp rx=0 tx=0).stop 2023-10-06T02:30:39.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.817+0000 7f9be49ff700 1 -- 172.21.15.31:0/1341315781 shutdown_connections 2023-10-06T02:30:39.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.817+0000 7f9be49ff700 1 --2- 172.21.15.31:0/1341315781 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9be00cc6f0 0x7f9be0069f40 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:39.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.817+0000 7f9be49ff700 1 --2- 172.21.15.31:0/1341315781 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9be013c9b0 0x7f9be0069a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:39.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.817+0000 7f9be49ff700 1 -- 172.21.15.31:0/1341315781 >> 172.21.15.31:0/1341315781 conn(0x7f9be0067f00 msgr2=0x7f9be0068300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:39.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.818+0000 7f9be49ff700 1 -- 172.21.15.31:0/1341315781 shutdown_connections 2023-10-06T02:30:39.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.818+0000 7f9be49ff700 1 -- 172.21.15.31:0/1341315781 wait complete. 2023-10-06T02:30:39.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.819+0000 7f9be49ff700 1 Processor -- start 2023-10-06T02:30:39.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.819+0000 7f9be49ff700 1 -- start start 2023-10-06T02:30:39.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.820+0000 7f9be49ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9be00cc320 0x7f9be00c7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:39.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.820+0000 7f9be49ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9be00cc6f0 0x7f9be013b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:39.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.820+0000 7f9be49ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9be013c9b0 0x7f9be0138790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:39.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.820+0000 7f9be49ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be013aa20 con 0x7f9be00cc6f0 2023-10-06T02:30:39.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.820+0000 7f9be49ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be013b2a0 con 0x7f9be013c9b0 2023-10-06T02:30:39.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.820+0000 7f9be49ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9be016e030 con 0x7f9be00cc320 2023-10-06T02:30:39.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.820+0000 7f9bddd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9be00cc6f0 0x7f9be013b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:39.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.820+0000 7f9bddd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9be00cc6f0 0x7f9be013b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36050/0 (socket says 172.21.15.31:36050) 2023-10-06T02:30:39.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.820+0000 7f9bddd9b700 1 -- 172.21.15.31:0/621295189 learned_addr learned my addr 172.21.15.31:0/621295189 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:39.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.821+0000 7f9bde59c700 1 --2- 172.21.15.31:0/621295189 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9be00cc320 0x7f9be00c7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:39.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.821+0000 7f9bded9d700 1 --2- 172.21.15.31:0/621295189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9be013c9b0 0x7f9be0138790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:39.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.821+0000 7f9bddd9b700 1 -- 172.21.15.31:0/621295189 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9be00cc320 msgr2=0x7f9be00c7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:39.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.821+0000 7f9bddd9b700 1 --2- 172.21.15.31:0/621295189 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9be00cc320 0x7f9be00c7920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:39.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.821+0000 7f9bddd9b700 1 -- 172.21.15.31:0/621295189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9be013c9b0 msgr2=0x7f9be0138790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:39.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.821+0000 7f9bddd9b700 1 --2- 172.21.15.31:0/621295189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9be013c9b0 0x7f9be0138790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:39.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.821+0000 7f9bddd9b700 1 -- 172.21.15.31:0/621295189 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bd4009420 con 0x7f9be00cc6f0 2023-10-06T02:30:39.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.821+0000 7f9bded9d700 1 --2- 172.21.15.31:0/621295189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9be013c9b0 0x7f9be0138790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:30:39.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.821+0000 7f9bddd9b700 1 --2- 172.21.15.31:0/621295189 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9be00cc6f0 0x7f9be013b730 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f9bd400ec80 tx=0x7f9bd40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:39.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.821+0000 7f9bce7fc700 1 -- 172.21.15.31:0/621295189 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd400c9d0 con 0x7f9be00cc6f0 2023-10-06T02:30:39.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.821+0000 7f9bce7fc700 1 -- 172.21.15.31:0/621295189 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bd400cb30 con 0x7f9be00cc6f0 2023-10-06T02:30:39.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.821+0000 7f9be49ff700 1 -- 172.21.15.31:0/621295189 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bd8009a20 con 0x7f9be00cc6f0 2023-10-06T02:30:39.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.821+0000 7f9bce7fc700 1 -- 172.21.15.31:0/621295189 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd4013470 con 0x7f9be00cc6f0 2023-10-06T02:30:39.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.821+0000 7f9be49ff700 1 -- 172.21.15.31:0/621295189 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9be016dd50 con 0x7f9be00cc6f0 2023-10-06T02:30:39.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.823+0000 7f9be49ff700 1 -- 172.21.15.31:0/621295189 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9be013b450 con 0x7f9be00cc6f0 2023-10-06T02:30:39.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.823+0000 7f9bce7fc700 1 -- 172.21.15.31:0/621295189 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f9bd4013820 con 0x7f9be00cc6f0 2023-10-06T02:30:39.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.824+0000 7f9bce7fc700 1 --2- 172.21.15.31:0/621295189 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9bc00674d0 0x7f9bc0069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:39.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.825+0000 7f9bce7fc700 1 -- 172.21.15.31:0/621295189 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6259+0+0 (secure 0 0 0) 0x7f9bd40852b0 con 0x7f9be00cc6f0 2023-10-06T02:30:39.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.825+0000 7f9bde59c700 1 --2- 172.21.15.31:0/621295189 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9bc00674d0 0x7f9bc0069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:39.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.826+0000 7f9bde59c700 1 --2- 172.21.15.31:0/621295189 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9bc00674d0 0x7f9bc0069980 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9bd00058d0 tx=0x7f9bd0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:39.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:39.828+0000 7f9bce7fc700 1 -- 172.21.15.31:0/621295189 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9bd4051a40 con 0x7f9be00cc6f0 2023-10-06T02:30:40.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.076+0000 7f9be49ff700 1 -- 172.21.15.31:0/621295189 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f9be004c3c0 con 0x7f9be00cc6f0 2023-10-06T02:30:40.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.598+0000 7f9bce7fc700 1 -- 172.21.15.31:0/621295189 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v93) v1 ==== 104+0+0 (secure 0 0 0) 0x7f9bd4055060 con 0x7f9be00cc6f0 2023-10-06T02:30:40.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.601+0000 7f9be49ff700 1 -- 172.21.15.31:0/621295189 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9bc00674d0 msgr2=0x7f9bc0069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:40.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.601+0000 7f9be49ff700 1 --2- 172.21.15.31:0/621295189 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9bc00674d0 0x7f9bc0069980 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9bd00058d0 tx=0x7f9bd0009040 comp rx=0 tx=0).stop 2023-10-06T02:30:40.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.601+0000 7f9be49ff700 1 -- 172.21.15.31:0/621295189 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9be00cc6f0 msgr2=0x7f9be013b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:40.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.601+0000 7f9be49ff700 1 --2- 172.21.15.31:0/621295189 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9be00cc6f0 0x7f9be013b730 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f9bd400ec80 tx=0x7f9bd40091a0 comp rx=0 tx=0).stop 2023-10-06T02:30:40.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.601+0000 7f9be49ff700 1 -- 172.21.15.31:0/621295189 shutdown_connections 2023-10-06T02:30:40.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.601+0000 7f9be49ff700 1 --2- 172.21.15.31:0/621295189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9be013c9b0 0x7f9be0138790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:40.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.601+0000 7f9be49ff700 1 --2- 172.21.15.31:0/621295189 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9bc00674d0 0x7f9bc0069980 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:40.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.601+0000 7f9be49ff700 1 --2- 172.21.15.31:0/621295189 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9be00cc320 0x7f9be00c7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:40.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.602+0000 7f9be49ff700 1 --2- 172.21.15.31:0/621295189 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9be00cc6f0 0x7f9be013b730 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:40.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.602+0000 7f9be49ff700 1 -- 172.21.15.31:0/621295189 >> 172.21.15.31:0/621295189 conn(0x7f9be0067f00 msgr2=0x7f9be00b1880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:40.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.602+0000 7f9be49ff700 1 -- 172.21.15.31:0/621295189 shutdown_connections 2023-10-06T02:30:40.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.602+0000 7f9be49ff700 1 -- 172.21.15.31:0/621295189 wait complete. 2023-10-06T02:30:40.624 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T02:30:40.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.861+0000 7f22eb59e700 1 Processor -- start 2023-10-06T02:30:40.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.861+0000 7f22eb59e700 1 -- start start 2023-10-06T02:30:40.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.862+0000 7f22eb59e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f22ec153660 0x7f22ec0c0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:40.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.862+0000 7f22eb59e700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f22ec153a30 0x7f22ec156d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:40.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.862+0000 7f22eb59e700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22ec0c2200 con 0x7f22ec0c8980 2023-10-06T02:30:40.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.862+0000 7f22eb59e700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22ec157250 con 0x7f22ec153a30 2023-10-06T02:30:40.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.862+0000 7f22eb59e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22ec0c15f0 con 0x7f22ec153660 2023-10-06T02:30:40.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.862+0000 7f22ead9d700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f22ec0c8980 0x7f22ec0c8dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60776/0 (socket says 172.21.15.31:60776) 2023-10-06T02:30:40.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.862+0000 7f22ead9d700 1 -- 172.21.15.31:0/4172918825 learned_addr learned my addr 172.21.15.31:0/4172918825 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:30:40.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.863+0000 7f22e9d9b700 1 --2- 172.21.15.31:0/4172918825 >> v2:172.21.15.62:3300/0 conn(0x7f22ec153a30 0x7f22ec156d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:40.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.863+0000 7f22ea59c700 1 --2- 172.21.15.31:0/4172918825 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f22ec153660 0x7f22ec0c0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:40.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.863+0000 7f22e959a700 1 -- 172.21.15.31:0/4172918825 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2443800363 0 0) 0x7f22ec0c2200 con 0x7f22ec0c8980 2023-10-06T02:30:40.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.863+0000 7f22e959a700 1 -- 172.21.15.31:0/4172918825 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22d4003570 con 0x7f22ec0c8980 2023-10-06T02:30:40.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.863+0000 7f22e959a700 1 -- 172.21.15.31:0/4172918825 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 76067626 0 0) 0x7f22d4003570 con 0x7f22ec0c8980 2023-10-06T02:30:40.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.863+0000 7f22e959a700 1 -- 172.21.15.31:0/4172918825 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f22ec153660 msgr2=0x7f22ec0c0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:40.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.863+0000 7f22e959a700 1 --2- 172.21.15.31:0/4172918825 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f22ec153660 0x7f22ec0c0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:40.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.863+0000 7f22e959a700 1 -- 172.21.15.31:0/4172918825 >> v2:172.21.15.62:3300/0 conn(0x7f22ec153a30 msgr2=0x7f22ec156d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:40.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.863+0000 7f22e959a700 1 --2- 172.21.15.31:0/4172918825 >> v2:172.21.15.62:3300/0 conn(0x7f22ec153a30 0x7f22ec156d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:40.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.863+0000 7f22e959a700 1 -- 172.21.15.31:0/4172918825 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22ec0c18d0 con 0x7f22ec0c8980 2023-10-06T02:30:40.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.863+0000 7f22e959a700 1 -- 172.21.15.31:0/4172918825 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7f22e40017f0 con 0x7f22ec0c8980 2023-10-06T02:30:40.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.863+0000 7f22e959a700 1 -- 172.21.15.31:0/4172918825 >> v1:172.21.15.31:6789/0 conn(0x7f22ec0c8980 legacy=0x7f22ec0c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:40.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.863+0000 7f22e959a700 1 --2- 172.21.15.31:0/4172918825 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22d4003a60 0x7f22d4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:40.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.863+0000 7f22e959a700 1 --2- 172.21.15.31:0/4172918825 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f22ec153660 0x7f22d4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:40.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.863+0000 7f22e959a700 1 --2- 172.21.15.31:0/4172918825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22d4004d50 0x7f22d40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:40.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.863+0000 7f22e959a700 1 -- 172.21.15.31:0/4172918825 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22ec157250 con 0x7f22d4004d50 2023-10-06T02:30:40.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.863+0000 7f22e959a700 1 -- 172.21.15.31:0/4172918825 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22ec0c15f0 con 0x7f22d4003a60 2023-10-06T02:30:40.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.863+0000 7f22e959a700 1 -- 172.21.15.31:0/4172918825 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22d400dc70 con 0x7f22ec153660 2023-10-06T02:30:40.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.864+0000 7f22ea59c700 1 --2- 172.21.15.31:0/4172918825 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22d4003a60 0x7f22d4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:40.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.864+0000 7f22ead9d700 1 --2- 172.21.15.31:0/4172918825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22d4004d50 0x7f22d40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:40.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.864+0000 7f22e9d9b700 1 --2- 172.21.15.31:0/4172918825 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f22ec153660 0x7f22d4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:40.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.864+0000 7f22ea59c700 1 -- 172.21.15.31:0/4172918825 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f22ec153660 msgr2=0x7f22d4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:40.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.864+0000 7f22ea59c700 1 --2- 172.21.15.31:0/4172918825 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f22ec153660 0x7f22d4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:40.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.864+0000 7f22ea59c700 1 -- 172.21.15.31:0/4172918825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22d4004d50 msgr2=0x7f22d40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:40.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.864+0000 7f22ea59c700 1 --2- 172.21.15.31:0/4172918825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22d4004d50 0x7f22d40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:40.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.864+0000 7f22ea59c700 1 -- 172.21.15.31:0/4172918825 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d4003570 con 0x7f22d4003a60 2023-10-06T02:30:40.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.864+0000 7f22ead9d700 1 --2- 172.21.15.31:0/4172918825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22d4004d50 0x7f22d40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:30:40.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.864+0000 7f22ea59c700 1 --2- 172.21.15.31:0/4172918825 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22d4003a60 0x7f22d4003e70 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f22dc009c00 tx=0x7f22dc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80a94ea3543073a2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:40.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.865+0000 7f22e959a700 1 -- 172.21.15.31:0/4172918825 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22dc00e710 con 0x7f22d4003a60 2023-10-06T02:30:40.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.865+0000 7f22e959a700 1 -- 172.21.15.31:0/4172918825 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22dc00e870 con 0x7f22d4003a60 2023-10-06T02:30:40.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.865+0000 7f22e959a700 1 -- 172.21.15.31:0/4172918825 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22dc00ec30 con 0x7f22d4003a60 2023-10-06T02:30:40.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.865+0000 7f22eb59e700 1 -- 172.21.15.31:0/4172918825 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22d4003a60 msgr2=0x7f22d4003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:40.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.865+0000 7f22eb59e700 1 --2- 172.21.15.31:0/4172918825 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22d4003a60 0x7f22d4003e70 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f22dc009c00 tx=0x7f22dc0091a0 comp rx=0 tx=0).stop 2023-10-06T02:30:40.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.865+0000 7f22ead9d700 1 -- 172.21.15.31:0/4172918825 reap_dead start 2023-10-06T02:30:40.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.865+0000 7f22eb59e700 1 -- 172.21.15.31:0/4172918825 shutdown_connections 2023-10-06T02:30:40.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.865+0000 7f22eb59e700 1 -- 172.21.15.31:0/4172918825 >> 172.21.15.31:0/4172918825 conn(0x7f22ec06c6a0 msgr2=0x7f22ec06caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:40.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.866+0000 7f22eb59e700 1 -- 172.21.15.31:0/4172918825 shutdown_connections 2023-10-06T02:30:40.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.866+0000 7f22eb59e700 1 -- 172.21.15.31:0/4172918825 wait complete. 2023-10-06T02:30:40.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.867+0000 7f22eb59e700 1 Processor -- start 2023-10-06T02:30:40.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.867+0000 7f22eb59e700 1 -- start start 2023-10-06T02:30:40.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.868+0000 7f22eb59e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f22d4003a60 0x7f22ec165620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:40.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.868+0000 7f22eb59e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22ec165b60 0x7f22ec151c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:40.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.868+0000 7f22eb59e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22ec14e610 0x7f22ec152140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:40.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.868+0000 7f22eb59e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22ec1665c0 con 0x7f22ec14e610 2023-10-06T02:30:40.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.868+0000 7f22eb59e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22ec169130 con 0x7f22ec165b60 2023-10-06T02:30:40.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.868+0000 7f22eb59e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22ec166000 con 0x7f22d4003a60 2023-10-06T02:30:40.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.868+0000 7f22ead9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22ec14e610 0x7f22ec152140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:40.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.868+0000 7f22ead9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22ec14e610 0x7f22ec152140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36070/0 (socket says 172.21.15.31:36070) 2023-10-06T02:30:40.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.868+0000 7f22ead9d700 1 -- 172.21.15.31:0/2128217518 learned_addr learned my addr 172.21.15.31:0/2128217518 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:40.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.868+0000 7f22ea59c700 1 --2- 172.21.15.31:0/2128217518 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f22d4003a60 0x7f22ec165620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:40.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.868+0000 7f22e9d9b700 1 --2- 172.21.15.31:0/2128217518 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22ec165b60 0x7f22ec151c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:40.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.869+0000 7f22ead9d700 1 -- 172.21.15.31:0/2128217518 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f22d4003a60 msgr2=0x7f22ec165620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:40.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.869+0000 7f22ead9d700 1 --2- 172.21.15.31:0/2128217518 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f22d4003a60 0x7f22ec165620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:40.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.869+0000 7f22ead9d700 1 -- 172.21.15.31:0/2128217518 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22ec165b60 msgr2=0x7f22ec151c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:40.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.869+0000 7f22ead9d700 1 --2- 172.21.15.31:0/2128217518 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22ec165b60 0x7f22ec151c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:40.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.869+0000 7f22ead9d700 1 -- 172.21.15.31:0/2128217518 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22ec1662e0 con 0x7f22ec14e610 2023-10-06T02:30:40.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.869+0000 7f22ead9d700 1 --2- 172.21.15.31:0/2128217518 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22ec14e610 0x7f22ec152140 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f22ec0ccd30 tx=0x7f22e4002210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:40.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.869+0000 7f22da7fc700 1 -- 172.21.15.31:0/2128217518 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e4007a80 con 0x7f22ec14e610 2023-10-06T02:30:40.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.869+0000 7f22da7fc700 1 -- 172.21.15.31:0/2128217518 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22e40133f0 con 0x7f22ec14e610 2023-10-06T02:30:40.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.869+0000 7f22da7fc700 1 -- 172.21.15.31:0/2128217518 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e40137b0 con 0x7f22ec14e610 2023-10-06T02:30:40.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.869+0000 7f22eb59e700 1 -- 172.21.15.31:0/2128217518 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22ec168dc0 con 0x7f22ec14e610 2023-10-06T02:30:40.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.869+0000 7f22eb59e700 1 -- 172.21.15.31:0/2128217518 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22ec1586a0 con 0x7f22ec14e610 2023-10-06T02:30:40.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.871+0000 7f22eb59e700 1 -- 172.21.15.31:0/2128217518 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22ec168c20 con 0x7f22ec14e610 2023-10-06T02:30:40.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.871+0000 7f22da7fc700 1 -- 172.21.15.31:0/2128217518 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f22e4013b20 con 0x7f22ec14e610 2023-10-06T02:30:40.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.873+0000 7f22da7fc700 1 --2- 172.21.15.31:0/2128217518 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f22cc0674d0 0x7f22cc069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:40.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.873+0000 7f22da7fc700 1 -- 172.21.15.31:0/2128217518 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6259+0+0 (secure 0 0 0) 0x7f22e4085c30 con 0x7f22ec14e610 2023-10-06T02:30:40.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.873+0000 7f22ea59c700 1 --2- 172.21.15.31:0/2128217518 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f22cc0674d0 0x7f22cc069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:40.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.874+0000 7f22ea59c700 1 --2- 172.21.15.31:0/2128217518 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f22cc0674d0 0x7f22cc069980 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f22dc0091e0 tx=0x7f22dc01a000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:40.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:40.877+0000 7f22da7fc700 1 -- 172.21.15.31:0/2128217518 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f22ec168c20 con 0x7f22ec14e610 2023-10-06T02:30:41.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:41.083+0000 7f22eb59e700 1 -- 172.21.15.31:0/2128217518 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f22ec168c20 con 0x7f22ec14e610 2023-10-06T02:30:41.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:41.601+0000 7f22da7fc700 1 -- 172.21.15.31:0/2128217518 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v135) v1 ==== 181+0+0 (secure 0 0 0) 0x7f22ec168c20 con 0x7f22ec14e610 2023-10-06T02:30:41.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:41.604+0000 7f22eb59e700 1 -- 172.21.15.31:0/2128217518 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f22cc0674d0 msgr2=0x7f22cc069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:41.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:41.604+0000 7f22eb59e700 1 --2- 172.21.15.31:0/2128217518 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f22cc0674d0 0x7f22cc069980 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f22dc0091e0 tx=0x7f22dc01a000 comp rx=0 tx=0).stop 2023-10-06T02:30:41.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:41.604+0000 7f22eb59e700 1 -- 172.21.15.31:0/2128217518 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22ec14e610 msgr2=0x7f22ec152140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:41.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:41.604+0000 7f22eb59e700 1 --2- 172.21.15.31:0/2128217518 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22ec14e610 0x7f22ec152140 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f22ec0ccd30 tx=0x7f22e4002210 comp rx=0 tx=0).stop 2023-10-06T02:30:41.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:41.605+0000 7f22eb59e700 1 -- 172.21.15.31:0/2128217518 shutdown_connections 2023-10-06T02:30:41.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:41.605+0000 7f22eb59e700 1 --2- 172.21.15.31:0/2128217518 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22ec14e610 0x7f22ec152140 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:41.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:41.605+0000 7f22eb59e700 1 --2- 172.21.15.31:0/2128217518 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f22cc0674d0 0x7f22cc069980 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:41.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:41.605+0000 7f22eb59e700 1 --2- 172.21.15.31:0/2128217518 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f22d4003a60 0x7f22ec165620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:41.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:41.605+0000 7f22eb59e700 1 --2- 172.21.15.31:0/2128217518 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22ec165b60 0x7f22ec151c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:41.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:41.605+0000 7f22eb59e700 1 -- 172.21.15.31:0/2128217518 >> 172.21.15.31:0/2128217518 conn(0x7f22ec06c6a0 msgr2=0x7f22ec152c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:41.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:41.605+0000 7f22eb59e700 1 -- 172.21.15.31:0/2128217518 shutdown_connections 2023-10-06T02:30:41.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:41.605+0000 7f22eb59e700 1 -- 172.21.15.31:0/2128217518 wait complete. 2023-10-06T02:30:41.608 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' removed 2023-10-06T02:30:41.626 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T02:30:41.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:41.819+0000 7f569219d700 1 Processor -- start 2023-10-06T02:30:41.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:41.819+0000 7f569219d700 1 -- start start 2023-10-06T02:30:41.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:41.820+0000 7f569219d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f568c153660 0x7f568c0c0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:41.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:41.820+0000 7f569219d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f568c0c9090 0x7f568c156d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:41.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:41.820+0000 7f569219d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f568c0c2200 con 0x7f568c153a30 2023-10-06T02:30:41.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:41.820+0000 7f569219d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f568c157250 con 0x7f568c0c9090 2023-10-06T02:30:41.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:41.820+0000 7f569219d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f568c0c15f0 con 0x7f568c153660 2023-10-06T02:30:41.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:41.820+0000 7f569099a700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f568c153a30 0x7f568c0c8980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60784/0 (socket says 172.21.15.31:60784) 2023-10-06T02:30:41.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:41.820+0000 7f569099a700 1 -- 172.21.15.31:0/715154603 learned_addr learned my addr 172.21.15.31:0/715154603 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:30:41.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:41.821+0000 7f569199c700 1 --2- 172.21.15.31:0/715154603 >> v2:172.21.15.31:3300/0 conn(0x7f568c0c9090 0x7f568c156d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:41.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:41.821+0000 7f569119b700 1 --2- 172.21.15.31:0/715154603 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f568c153660 0x7f568c0c0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:41.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:41.821+0000 7f5683fff700 1 -- 172.21.15.31:0/715154603 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2663627870 0 0) 0x7f568c0c2200 con 0x7f568c153a30 2023-10-06T02:30:41.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:41.821+0000 7f5683fff700 1 -- 172.21.15.31:0/715154603 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5678003570 con 0x7f568c153a30 2023-10-06T02:30:41.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:41.821+0000 7f569199c700 1 -- 172.21.15.31:0/715154603 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f568c153660 msgr2=0x7f568c0c0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:41.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:41.821+0000 7f569199c700 1 --2- 172.21.15.31:0/715154603 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f568c153660 0x7f568c0c0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:41.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:41.821+0000 7f569199c700 1 -- 172.21.15.31:0/715154603 >> v1:172.21.15.31:6789/0 conn(0x7f568c153a30 legacy=0x7f568c0c8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:41.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:41.821+0000 7f569199c700 1 -- 172.21.15.31:0/715154603 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f568c0c18d0 con 0x7f568c0c9090 2023-10-06T02:30:41.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:41.821+0000 7f569119b700 1 --2- 172.21.15.31:0/715154603 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f568c153660 0x7f568c0c0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:30:41.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:41.821+0000 7f569199c700 1 --2- 172.21.15.31:0/715154603 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f568c0c9090 0x7f568c156d10 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f5684009e60 tx=0x7f5684009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7fb5d4e4f6aee1a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:41.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:41.822+0000 7f5683fff700 1 -- 172.21.15.31:0/715154603 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f568400e070 con 0x7f568c0c9090 2023-10-06T02:30:41.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:41.822+0000 7f5683fff700 1 -- 172.21.15.31:0/715154603 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f568400bb60 con 0x7f568c0c9090 2023-10-06T02:30:41.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:41.822+0000 7f5683fff700 1 -- 172.21.15.31:0/715154603 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5684012430 con 0x7f568c0c9090 2023-10-06T02:30:41.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:41.822+0000 7f569219d700 1 -- 172.21.15.31:0/715154603 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f568c0c9090 msgr2=0x7f568c156d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:41.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:41.822+0000 7f569219d700 1 --2- 172.21.15.31:0/715154603 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f568c0c9090 0x7f568c156d10 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f5684009e60 tx=0x7f5684009780 comp rx=0 tx=0).stop 2023-10-06T02:30:41.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:41.822+0000 7f569219d700 1 -- 172.21.15.31:0/715154603 shutdown_connections 2023-10-06T02:30:41.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:41.822+0000 7f569219d700 1 --2- 172.21.15.31:0/715154603 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f568c0c9090 0x7f568c156d10 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:41.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:41.822+0000 7f569219d700 1 --2- 172.21.15.31:0/715154603 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f568c153660 0x7f568c0c0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:41.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:41.822+0000 7f569219d700 1 -- 172.21.15.31:0/715154603 >> 172.21.15.31:0/715154603 conn(0x7f568c06c6a0 msgr2=0x7f568c06caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:41.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:41.822+0000 7f569219d700 1 -- 172.21.15.31:0/715154603 shutdown_connections 2023-10-06T02:30:41.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:41.822+0000 7f569219d700 1 -- 172.21.15.31:0/715154603 wait complete. 2023-10-06T02:30:41.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:41.824+0000 7f569219d700 1 Processor -- start 2023-10-06T02:30:41.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:41.824+0000 7f569219d700 1 -- start start 2023-10-06T02:30:41.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:41.825+0000 7f569219d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f568c0c9090 0x7f568c152b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:41.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:41.825+0000 7f569219d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f568c153660 0x7f568c14e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:41.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:41.825+0000 7f569219d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f568c153a30 0x7f568c14b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:41.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:41.825+0000 7f569219d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f568c157be0 con 0x7f568c0c9090 2023-10-06T02:30:41.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:41.825+0000 7f569219d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f568c157ec0 con 0x7f568c153a30 2023-10-06T02:30:41.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:41.825+0000 7f569219d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f568c1581a0 con 0x7f568c153660 2023-10-06T02:30:41.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:41.825+0000 7f569119b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f568c0c9090 0x7f568c152b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:41.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:41.825+0000 7f569119b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f568c0c9090 0x7f568c152b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36080/0 (socket says 172.21.15.31:36080) 2023-10-06T02:30:41.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:41.825+0000 7f569119b700 1 -- 172.21.15.31:0/3219229261 learned_addr learned my addr 172.21.15.31:0/3219229261 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:41.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:41.825+0000 7f569099a700 1 --2- 172.21.15.31:0/3219229261 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f568c153660 0x7f568c14e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:41.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:41.825+0000 7f569199c700 1 --2- 172.21.15.31:0/3219229261 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f568c153a30 0x7f568c14b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:41.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:41.825+0000 7f569119b700 1 -- 172.21.15.31:0/3219229261 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f568c153660 msgr2=0x7f568c14e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:41.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:41.825+0000 7f569119b700 1 --2- 172.21.15.31:0/3219229261 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f568c153660 0x7f568c14e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:41.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:41.825+0000 7f569119b700 1 -- 172.21.15.31:0/3219229261 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f568c153a30 msgr2=0x7f568c14b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:41.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:41.825+0000 7f569119b700 1 --2- 172.21.15.31:0/3219229261 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f568c153a30 0x7f568c14b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:41.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:41.825+0000 7f569119b700 1 -- 172.21.15.31:0/3219229261 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5684009a20 con 0x7f568c0c9090 2023-10-06T02:30:41.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:41.826+0000 7f569119b700 1 --2- 172.21.15.31:0/3219229261 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f568c0c9090 0x7f568c152b00 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f568800e8d0 tx=0x7f5688008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:41.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:41.826+0000 7f56817fa700 1 -- 172.21.15.31:0/3219229261 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f568800ee70 con 0x7f568c0c9090 2023-10-06T02:30:41.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:41.826+0000 7f56817fa700 1 -- 172.21.15.31:0/3219229261 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f568800c5e0 con 0x7f568c0c9090 2023-10-06T02:30:41.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:41.826+0000 7f56817fa700 1 -- 172.21.15.31:0/3219229261 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f568800c980 con 0x7f568c0c9090 2023-10-06T02:30:41.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:41.826+0000 7f569219d700 1 -- 172.21.15.31:0/3219229261 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f568c158480 con 0x7f568c0c9090 2023-10-06T02:30:41.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:41.826+0000 7f569219d700 1 -- 172.21.15.31:0/3219229261 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f568c153040 con 0x7f568c0c9090 2023-10-06T02:30:41.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:41.827+0000 7f569219d700 1 -- 172.21.15.31:0/3219229261 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f568c04c3c0 con 0x7f568c0c9090 2023-10-06T02:30:41.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:41.828+0000 7f56817fa700 1 -- 172.21.15.31:0/3219229261 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f5688013410 con 0x7f568c0c9090 2023-10-06T02:30:41.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:41.829+0000 7f56817fa700 1 --2- 172.21.15.31:0/3219229261 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f56700674d0 0x7f5670069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:41.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:41.829+0000 7f56817fa700 1 -- 172.21.15.31:0/3219229261 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5821+0+0 (secure 0 0 0) 0x7f5688084f50 con 0x7f568c0c9090 2023-10-06T02:30:41.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:41.830+0000 7f569099a700 1 --2- 172.21.15.31:0/3219229261 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f56700674d0 0x7f5670069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:41.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:41.830+0000 7f569099a700 1 --2- 172.21.15.31:0/3219229261 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f56700674d0 0x7f5670069980 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f567c0058d0 tx=0x7f567c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:41.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:41.833+0000 7f56817fa700 1 -- 172.21.15.31:0/3219229261 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5688051790 con 0x7f568c0c9090 2023-10-06T02:30:42.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:42.036+0000 7f569219d700 1 -- 172.21.15.31:0/3219229261 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f568c0ccd60 con 0x7f568c0c9090 2023-10-06T02:30:42.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:42.608+0000 7f56817fa700 1 -- 172.21.15.31:0/3219229261 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v136) v1 ==== 169+0+0 (secure 0 0 0) 0x7f5688054db0 con 0x7f568c0c9090 2023-10-06T02:30:42.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:42.613+0000 7f569219d700 1 -- 172.21.15.31:0/3219229261 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f56700674d0 msgr2=0x7f5670069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:42.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:42.613+0000 7f569219d700 1 --2- 172.21.15.31:0/3219229261 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f56700674d0 0x7f5670069980 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f567c0058d0 tx=0x7f567c009040 comp rx=0 tx=0).stop 2023-10-06T02:30:42.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:42.613+0000 7f569219d700 1 -- 172.21.15.31:0/3219229261 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f568c0c9090 msgr2=0x7f568c152b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:42.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:42.613+0000 7f569219d700 1 --2- 172.21.15.31:0/3219229261 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f568c0c9090 0x7f568c152b00 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f568800e8d0 tx=0x7f5688008040 comp rx=0 tx=0).stop 2023-10-06T02:30:42.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:42.613+0000 7f569219d700 1 -- 172.21.15.31:0/3219229261 shutdown_connections 2023-10-06T02:30:42.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:42.613+0000 7f569219d700 1 --2- 172.21.15.31:0/3219229261 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f568c153a30 0x7f568c14b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:42.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:42.613+0000 7f569219d700 1 --2- 172.21.15.31:0/3219229261 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f568c0c9090 0x7f568c152b00 secure :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f568800e8d0 tx=0x7f5688008040 comp rx=0 tx=0).stop 2023-10-06T02:30:42.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:42.613+0000 7f569219d700 1 --2- 172.21.15.31:0/3219229261 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f56700674d0 0x7f5670069980 secure :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f567c0058d0 tx=0x7f567c009040 comp rx=0 tx=0).stop 2023-10-06T02:30:42.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:42.613+0000 7f569219d700 1 --2- 172.21.15.31:0/3219229261 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f568c153660 0x7f568c14e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:42.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:42.613+0000 7f569219d700 1 -- 172.21.15.31:0/3219229261 >> 172.21.15.31:0/3219229261 conn(0x7f568c06c6a0 msgr2=0x7f568c0cc360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:42.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:42.614+0000 7f569219d700 1 -- 172.21.15.31:0/3219229261 shutdown_connections 2023-10-06T02:30:42.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:42.614+0000 7f569219d700 1 -- 172.21.15.31:0/3219229261 wait complete. 2023-10-06T02:30:42.617 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' removed 2023-10-06T02:30:42.637 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T02:30:42.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:42.831+0000 7f654959d700 1 Processor -- start 2023-10-06T02:30:42.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:42.831+0000 7f654959d700 1 -- start start 2023-10-06T02:30:42.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:42.832+0000 7f654959d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f654415dfd0 0x7f65440aedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:42.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:42.832+0000 7f654959d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6544159920 0x7f65440ad090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:42.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:42.832+0000 7f654959d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65440acc00 con 0x7f6544159550 2023-10-06T02:30:42.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:42.832+0000 7f654959d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65440abff0 con 0x7f6544159920 2023-10-06T02:30:42.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:42.832+0000 7f654959d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65440ac2d0 con 0x7f654415dfd0 2023-10-06T02:30:42.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:42.833+0000 7f6548d9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6544159920 0x7f65440ad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:42.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:42.833+0000 7f6548d9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6544159920 0x7f65440ad090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36094/0 (socket says 172.21.15.31:36094) 2023-10-06T02:30:42.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:42.833+0000 7f6548d9c700 1 -- 172.21.15.31:0/1216114902 learned_addr learned my addr 172.21.15.31:0/1216114902 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:42.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:42.833+0000 7f65437fe700 1 --2- 172.21.15.31:0/1216114902 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f654415dfd0 0x7f65440aedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:42.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:42.833+0000 7f6548d9c700 1 -- 172.21.15.31:0/1216114902 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f654415dfd0 msgr2=0x7f65440aedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:42.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:42.833+0000 7f6548d9c700 1 --2- 172.21.15.31:0/1216114902 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f654415dfd0 0x7f65440aedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:42.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:42.833+0000 7f6548d9c700 1 -- 172.21.15.31:0/1216114902 >> v1:172.21.15.62:6789/0 conn(0x7f6544159550 legacy=0x7f65440ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:42.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:42.833+0000 7f6548d9c700 1 -- 172.21.15.31:0/1216114902 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65440ac5b0 con 0x7f6544159920 2023-10-06T02:30:42.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:42.833+0000 7f6548d9c700 1 --2- 172.21.15.31:0/1216114902 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6544159920 0x7f65440ad090 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f653c009f20 tx=0x7f653c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78a377ec05a03235 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:42.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:42.834+0000 7f6542ffd700 1 -- 172.21.15.31:0/1216114902 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f653c00e070 con 0x7f6544159920 2023-10-06T02:30:42.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:42.834+0000 7f6542ffd700 1 -- 172.21.15.31:0/1216114902 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f653c00ba80 con 0x7f6544159920 2023-10-06T02:30:42.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:42.834+0000 7f6542ffd700 1 -- 172.21.15.31:0/1216114902 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f653c0092e0 con 0x7f6544159920 2023-10-06T02:30:42.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:42.834+0000 7f654959d700 1 -- 172.21.15.31:0/1216114902 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6544159920 msgr2=0x7f65440ad090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:42.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:42.834+0000 7f654959d700 1 --2- 172.21.15.31:0/1216114902 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6544159920 0x7f65440ad090 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f653c009f20 tx=0x7f653c009840 comp rx=0 tx=0).stop 2023-10-06T02:30:42.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:42.834+0000 7f654959d700 1 -- 172.21.15.31:0/1216114902 shutdown_connections 2023-10-06T02:30:42.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:42.834+0000 7f654959d700 1 --2- 172.21.15.31:0/1216114902 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6544159920 0x7f65440ad090 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:42.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:42.834+0000 7f654959d700 1 --2- 172.21.15.31:0/1216114902 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f654415dfd0 0x7f65440aedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:42.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:42.834+0000 7f654959d700 1 -- 172.21.15.31:0/1216114902 >> 172.21.15.31:0/1216114902 conn(0x7f654406ce80 msgr2=0x7f65440ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:42.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:42.834+0000 7f654959d700 1 -- 172.21.15.31:0/1216114902 shutdown_connections 2023-10-06T02:30:42.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:42.834+0000 7f654959d700 1 -- 172.21.15.31:0/1216114902 wait complete. 2023-10-06T02:30:42.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:42.836+0000 7f654959d700 1 Processor -- start 2023-10-06T02:30:42.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:42.836+0000 7f654959d700 1 -- start start 2023-10-06T02:30:42.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:42.837+0000 7f654959d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6544159550 0x7f654409b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:42.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:42.837+0000 7f654959d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6544159920 0x7f65440c7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:42.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:42.837+0000 7f654959d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f654415dfd0 0x7f65440c8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:42.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:42.837+0000 7f654959d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f654409bab0 con 0x7f654415dfd0 2023-10-06T02:30:42.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:42.837+0000 7f654959d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f654416b4c0 con 0x7f6544159920 2023-10-06T02:30:42.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:42.837+0000 7f654959d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f654416e140 con 0x7f6544159550 2023-10-06T02:30:42.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:42.837+0000 7f6548d9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f654415dfd0 0x7f65440c8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:42.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:42.837+0000 7f6548d9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f654415dfd0 0x7f65440c8450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36102/0 (socket says 172.21.15.31:36102) 2023-10-06T02:30:42.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:42.837+0000 7f6548d9c700 1 -- 172.21.15.31:0/3808383472 learned_addr learned my addr 172.21.15.31:0/3808383472 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:42.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:42.837+0000 7f6543fff700 1 --2- 172.21.15.31:0/3808383472 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6544159550 0x7f654409b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:42.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:42.837+0000 7f65437fe700 1 --2- 172.21.15.31:0/3808383472 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6544159920 0x7f65440c7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:42.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:42.838+0000 7f6548d9c700 1 -- 172.21.15.31:0/3808383472 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6544159550 msgr2=0x7f654409b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:42.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:42.838+0000 7f6548d9c700 1 --2- 172.21.15.31:0/3808383472 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6544159550 0x7f654409b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:42.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:42.838+0000 7f6548d9c700 1 -- 172.21.15.31:0/3808383472 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6544159920 msgr2=0x7f65440c7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:42.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:42.838+0000 7f6548d9c700 1 --2- 172.21.15.31:0/3808383472 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6544159920 0x7f65440c7f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:42.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:42.838+0000 7f6548d9c700 1 -- 172.21.15.31:0/3808383472 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f653c009ae0 con 0x7f654415dfd0 2023-10-06T02:30:42.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:42.838+0000 7f6548d9c700 1 --2- 172.21.15.31:0/3808383472 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f654415dfd0 0x7f65440c8450 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f653c009430 tx=0x7f653c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:42.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:42.838+0000 7f652bfff700 1 -- 172.21.15.31:0/3808383472 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f653c0032f0 con 0x7f654415dfd0 2023-10-06T02:30:42.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:42.838+0000 7f652bfff700 1 -- 172.21.15.31:0/3808383472 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f653c008d10 con 0x7f654415dfd0 2023-10-06T02:30:42.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:42.838+0000 7f654959d700 1 -- 172.21.15.31:0/3808383472 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65440c8ae0 con 0x7f654415dfd0 2023-10-06T02:30:42.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:42.838+0000 7f652bfff700 1 -- 172.21.15.31:0/3808383472 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f653c026670 con 0x7f654415dfd0 2023-10-06T02:30:42.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:42.838+0000 7f654959d700 1 -- 172.21.15.31:0/3808383472 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f654416b150 con 0x7f654415dfd0 2023-10-06T02:30:42.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:42.840+0000 7f654959d700 1 -- 172.21.15.31:0/3808383472 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f654404c3c0 con 0x7f654415dfd0 2023-10-06T02:30:42.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:42.840+0000 7f652bfff700 1 -- 172.21.15.31:0/3808383472 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f653c003040 con 0x7f654415dfd0 2023-10-06T02:30:42.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:42.841+0000 7f652bfff700 1 --2- 172.21.15.31:0/3808383472 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f65300674d0 0x7f6530069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:42.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:42.841+0000 7f652bfff700 1 -- 172.21.15.31:0/3808383472 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5320+0+0 (secure 0 0 0) 0x7f653c086ea0 con 0x7f654415dfd0 2023-10-06T02:30:42.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:42.841+0000 7f6543fff700 1 --2- 172.21.15.31:0/3808383472 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f65300674d0 0x7f6530069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:42.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:42.842+0000 7f6543fff700 1 --2- 172.21.15.31:0/3808383472 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f65300674d0 0x7f6530069980 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6544147710 tx=0x7f6534009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:42.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:42.845+0000 7f652bfff700 1 -- 172.21.15.31:0/3808383472 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f653c053980 con 0x7f654415dfd0 2023-10-06T02:30:43.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.051+0000 7f654959d700 1 -- 172.21.15.31:0/3808383472 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f65440b3ad0 con 0x7f654415dfd0 2023-10-06T02:30:43.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.055+0000 7f652bfff700 1 -- 172.21.15.31:0/3808383472 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v25) v1 ==== 104+0+0 (secure 0 0 0) 0x7f653c056fa0 con 0x7f654415dfd0 2023-10-06T02:30:43.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.058+0000 7f654959d700 1 -- 172.21.15.31:0/3808383472 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f65300674d0 msgr2=0x7f6530069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:43.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.058+0000 7f654959d700 1 --2- 172.21.15.31:0/3808383472 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f65300674d0 0x7f6530069980 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6544147710 tx=0x7f6534009040 comp rx=0 tx=0).stop 2023-10-06T02:30:43.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.058+0000 7f654959d700 1 -- 172.21.15.31:0/3808383472 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f654415dfd0 msgr2=0x7f65440c8450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:43.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.058+0000 7f654959d700 1 --2- 172.21.15.31:0/3808383472 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f654415dfd0 0x7f65440c8450 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f653c009430 tx=0x7f653c018000 comp rx=0 tx=0).stop 2023-10-06T02:30:43.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.058+0000 7f654959d700 1 -- 172.21.15.31:0/3808383472 shutdown_connections 2023-10-06T02:30:43.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.058+0000 7f654959d700 1 --2- 172.21.15.31:0/3808383472 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f654415dfd0 0x7f65440c8450 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:43.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.058+0000 7f654959d700 1 --2- 172.21.15.31:0/3808383472 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f65300674d0 0x7f6530069980 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:43.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.058+0000 7f654959d700 1 --2- 172.21.15.31:0/3808383472 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6544159550 0x7f654409b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:43.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.058+0000 7f654959d700 1 --2- 172.21.15.31:0/3808383472 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6544159920 0x7f65440c7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:43.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.058+0000 7f654959d700 1 -- 172.21.15.31:0/3808383472 >> 172.21.15.31:0/3808383472 conn(0x7f654406ce80 msgr2=0x7f65440b0620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:43.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.058+0000 7f654959d700 1 -- 172.21.15.31:0/3808383472 shutdown_connections 2023-10-06T02:30:43.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.058+0000 7f654959d700 1 -- 172.21.15.31:0/3808383472 wait complete. 2023-10-06T02:30:43.077 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol' 2023-10-06T02:30:43.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.262+0000 7fa37759e700 1 Processor -- start 2023-10-06T02:30:43.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.262+0000 7fa37759e700 1 -- start start 2023-10-06T02:30:43.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.263+0000 7fa37759e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa37813fb20 0x7fa37815a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:43.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.263+0000 7fa37759e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa37813fef0 0x7fa3780c7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:43.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.263+0000 7fa37759e700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa37815ae70 con 0x7fa37806aaa0 2023-10-06T02:30:43.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.263+0000 7fa37759e700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa37806af60 con 0x7fa37813fef0 2023-10-06T02:30:43.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.263+0000 7fa37759e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa37806b240 con 0x7fa37813fb20 2023-10-06T02:30:43.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.263+0000 7fa375d9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa37813fef0 0x7fa3780c7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:43.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.263+0000 7fa375d9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa37813fef0 0x7fa3780c7990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36116/0 (socket says 172.21.15.31:36116) 2023-10-06T02:30:43.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.263+0000 7fa375d9b700 1 -- 172.21.15.31:0/1247580549 learned_addr learned my addr 172.21.15.31:0/1247580549 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:43.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.263+0000 7fa37659c700 1 --2- 172.21.15.31:0/1247580549 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa37813fb20 0x7fa37815a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:43.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.264+0000 7fa375d9b700 1 -- 172.21.15.31:0/1247580549 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa37813fb20 msgr2=0x7fa37815a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:43.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.264+0000 7fa375d9b700 1 --2- 172.21.15.31:0/1247580549 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa37813fb20 0x7fa37815a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:43.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.264+0000 7fa375d9b700 1 -- 172.21.15.31:0/1247580549 >> v1:172.21.15.62:6789/0 conn(0x7fa37806aaa0 legacy=0x7fa3780c6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:43.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.264+0000 7fa375d9b700 1 -- 172.21.15.31:0/1247580549 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa37806b520 con 0x7fa37813fef0 2023-10-06T02:30:43.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.264+0000 7fa375d9b700 1 --2- 172.21.15.31:0/1247580549 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa37813fef0 0x7fa3780c7990 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7fa36c009990 tx=0x7fa36c009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be4dd4095602f7a4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:43.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.264+0000 7fa37559a700 1 -- 172.21.15.31:0/1247580549 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa36c009e80 con 0x7fa37813fef0 2023-10-06T02:30:43.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.264+0000 7fa37559a700 1 -- 172.21.15.31:0/1247580549 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa36c005c10 con 0x7fa37813fef0 2023-10-06T02:30:43.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.264+0000 7fa37559a700 1 -- 172.21.15.31:0/1247580549 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa36c00b650 con 0x7fa37813fef0 2023-10-06T02:30:43.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.265+0000 7fa37759e700 1 -- 172.21.15.31:0/1247580549 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa37813fef0 msgr2=0x7fa3780c7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:43.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.265+0000 7fa37759e700 1 --2- 172.21.15.31:0/1247580549 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa37813fef0 0x7fa3780c7990 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7fa36c009990 tx=0x7fa36c009220 comp rx=0 tx=0).stop 2023-10-06T02:30:43.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.265+0000 7fa37759e700 1 -- 172.21.15.31:0/1247580549 shutdown_connections 2023-10-06T02:30:43.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.265+0000 7fa37759e700 1 --2- 172.21.15.31:0/1247580549 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa37813fb20 0x7fa37815a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:43.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.265+0000 7fa37759e700 1 --2- 172.21.15.31:0/1247580549 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa37813fef0 0x7fa3780c7990 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:43.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.265+0000 7fa37759e700 1 -- 172.21.15.31:0/1247580549 >> 172.21.15.31:0/1247580549 conn(0x7fa37806d170 msgr2=0x7fa3780cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:43.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.265+0000 7fa37759e700 1 -- 172.21.15.31:0/1247580549 shutdown_connections 2023-10-06T02:30:43.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.265+0000 7fa37759e700 1 -- 172.21.15.31:0/1247580549 wait complete. 2023-10-06T02:30:43.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.266+0000 7fa37759e700 1 Processor -- start 2023-10-06T02:30:43.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.267+0000 7fa37759e700 1 -- start start 2023-10-06T02:30:43.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.267+0000 7fa37759e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa37806aaa0 0x7fa37816a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:43.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.268+0000 7fa37759e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa37813fb20 0x7fa3780b0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:43.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.268+0000 7fa37759e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa37813fef0 0x7fa378158370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:43.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.268+0000 7fa37759e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa37816d130 con 0x7fa37813fb20 2023-10-06T02:30:43.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.268+0000 7fa37759e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3780b1200 con 0x7fa37813fef0 2023-10-06T02:30:43.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.268+0000 7fa37759e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa37816d380 con 0x7fa37806aaa0 2023-10-06T02:30:43.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.268+0000 7fa375d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa37813fb20 0x7fa3780b0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:43.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.268+0000 7fa375d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa37813fb20 0x7fa3780b0cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36126/0 (socket says 172.21.15.31:36126) 2023-10-06T02:30:43.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.268+0000 7fa375d9b700 1 -- 172.21.15.31:0/4157090512 learned_addr learned my addr 172.21.15.31:0/4157090512 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:43.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.268+0000 7fa37659c700 1 --2- 172.21.15.31:0/4157090512 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa37806aaa0 0x7fa37816a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:43.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.268+0000 7fa376d9d700 1 --2- 172.21.15.31:0/4157090512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa37813fef0 0x7fa378158370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:43.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.268+0000 7fa375d9b700 1 -- 172.21.15.31:0/4157090512 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa37806aaa0 msgr2=0x7fa37816a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:43.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.268+0000 7fa375d9b700 1 --2- 172.21.15.31:0/4157090512 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa37806aaa0 0x7fa37816a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:43.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.268+0000 7fa375d9b700 1 -- 172.21.15.31:0/4157090512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa37813fef0 msgr2=0x7fa378158370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:43.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.268+0000 7fa375d9b700 1 --2- 172.21.15.31:0/4157090512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa37813fef0 0x7fa378158370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:43.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.268+0000 7fa375d9b700 1 -- 172.21.15.31:0/4157090512 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa36c009480 con 0x7fa37813fb20 2023-10-06T02:30:43.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.269+0000 7fa375d9b700 1 --2- 172.21.15.31:0/4157090512 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa37813fb20 0x7fa3780b0cc0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7fa36c00b3c0 tx=0x7fa36c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:43.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.269+0000 7fa3667fc700 1 -- 172.21.15.31:0/4157090512 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa36c011ae0 con 0x7fa37813fb20 2023-10-06T02:30:43.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.269+0000 7fa3667fc700 1 -- 172.21.15.31:0/4157090512 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa36c020070 con 0x7fa37813fb20 2023-10-06T02:30:43.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.269+0000 7fa37759e700 1 -- 172.21.15.31:0/4157090512 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa37816d660 con 0x7fa37813fb20 2023-10-06T02:30:43.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.269+0000 7fa3667fc700 1 -- 172.21.15.31:0/4157090512 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa36c011e70 con 0x7fa37813fb20 2023-10-06T02:30:43.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.269+0000 7fa37759e700 1 -- 172.21.15.31:0/4157090512 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa37816dc20 con 0x7fa37813fb20 2023-10-06T02:30:43.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.270+0000 7fa37759e700 1 -- 172.21.15.31:0/4157090512 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa37816de40 con 0x7fa37813fb20 2023-10-06T02:30:43.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.271+0000 7fa3667fc700 1 -- 172.21.15.31:0/4157090512 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7fa36c022030 con 0x7fa37813fb20 2023-10-06T02:30:43.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.272+0000 7fa3667fc700 1 --2- 172.21.15.31:0/4157090512 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa3580674d0 0x7fa358069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:43.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.272+0000 7fa3667fc700 1 -- 172.21.15.31:0/4157090512 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5320+0+0 (secure 0 0 0) 0x7fa36c086a80 con 0x7fa37813fb20 2023-10-06T02:30:43.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.273+0000 7fa37659c700 1 --2- 172.21.15.31:0/4157090512 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa3580674d0 0x7fa358069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:43.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.273+0000 7fa37659c700 1 --2- 172.21.15.31:0/4157090512 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa3580674d0 0x7fa358069980 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa3780ccbc0 tx=0x7fa368008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:43.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.276+0000 7fa3667fc700 1 -- 172.21.15.31:0/4157090512 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa36c0535c0 con 0x7fa37813fb20 2023-10-06T02:30:43.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.563+0000 7fa37759e700 1 -- 172.21.15.31:0/4157090512 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol"]} v 0) v1 -- 0x7fa37804c3c0 con 0x7fa37813fb20 2023-10-06T02:30:43.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.608+0000 7fa3667fc700 1 -- 172.21.15.31:0/4157090512 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol"]}]=0 v426) v1 ==== 151+0+0 (secure 0 0 0) 0x7fa36c056be0 con 0x7fa37813fb20 2023-10-06T02:30:43.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.611+0000 7fa37759e700 1 -- 172.21.15.31:0/4157090512 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa3580674d0 msgr2=0x7fa358069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:43.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.611+0000 7fa37759e700 1 --2- 172.21.15.31:0/4157090512 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa3580674d0 0x7fa358069980 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa3780ccbc0 tx=0x7fa368008040 comp rx=0 tx=0).stop 2023-10-06T02:30:43.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.611+0000 7fa37759e700 1 -- 172.21.15.31:0/4157090512 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa37813fb20 msgr2=0x7fa3780b0cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:43.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.611+0000 7fa37759e700 1 --2- 172.21.15.31:0/4157090512 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa37813fb20 0x7fa3780b0cc0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7fa36c00b3c0 tx=0x7fa36c016000 comp rx=0 tx=0).stop 2023-10-06T02:30:43.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.611+0000 7fa37759e700 1 -- 172.21.15.31:0/4157090512 shutdown_connections 2023-10-06T02:30:43.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.611+0000 7fa37759e700 1 --2- 172.21.15.31:0/4157090512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa37813fef0 0x7fa378158370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:43.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.611+0000 7fa37759e700 1 --2- 172.21.15.31:0/4157090512 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa3580674d0 0x7fa358069980 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:43.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.611+0000 7fa37759e700 1 --2- 172.21.15.31:0/4157090512 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa37806aaa0 0x7fa37816a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:43.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.611+0000 7fa37759e700 1 --2- 172.21.15.31:0/4157090512 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa37813fb20 0x7fa3780b0cc0 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:43.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.611+0000 7fa37759e700 1 -- 172.21.15.31:0/4157090512 >> 172.21.15.31:0/4157090512 conn(0x7fa37806d170 msgr2=0x7fa3780c0ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:43.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.611+0000 7fa37759e700 1 -- 172.21.15.31:0/4157090512 shutdown_connections 2023-10-06T02:30:43.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.611+0000 7fa37759e700 1 -- 172.21.15.31:0/4157090512 wait complete. 2023-10-06T02:30:43.630 INFO:tasks.cephfs_test_runner:test_subvolume_ops_on_nonexistent_vol (tasks.cephfs.test_volumes.TestMisc) ... ok 2023-10-06T02:30:43.630 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_upgrade_legacy_to_v1 (tasks.cephfs.test_volumes.TestMisc) poor man's upgrade test -- rather than going through a full upgrade cycle, 2023-10-06T02:30:43.630 INFO:tasks.cephfs_test_runner:test_subvolume_upgrade_legacy_to_v1 (tasks.cephfs.test_volumes.TestMisc) 2023-10-06T02:30:43.631 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1' 2023-10-06T02:30:43.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.816+0000 7f8046d9d700 1 Processor -- start 2023-10-06T02:30:43.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.816+0000 7f8046d9d700 1 -- start start 2023-10-06T02:30:43.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.817+0000 7f8046d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f804015de50 0x7f80400cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:43.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.817+0000 7f8046d9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f80400cef40 0x7f804015c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:43.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.817+0000 7f8046d9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f804015afd0 con 0x7f8040153c30 2023-10-06T02:30:43.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.817+0000 7f8046d9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f804015b2b0 con 0x7f80400cef40 2023-10-06T02:30:43.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.817+0000 7f8046d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f804015b590 con 0x7f804015de50 2023-10-06T02:30:43.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.818+0000 7f8045d9b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f8040153c30 0x7f804015a5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60788/0 (socket says 172.21.15.31:60788) 2023-10-06T02:30:43.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.818+0000 7f8045d9b700 1 -- 172.21.15.31:0/2684486268 learned_addr learned my addr 172.21.15.31:0/2684486268 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:30:43.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.818+0000 7f804559a700 1 --2- 172.21.15.31:0/2684486268 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f804015de50 0x7f80400cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:43.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.818+0000 7f804659c700 1 --2- 172.21.15.31:0/2684486268 >> v2:172.21.15.62:3300/0 conn(0x7f80400cef40 0x7f804015c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:43.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.818+0000 7f8044d99700 1 -- 172.21.15.31:0/2684486268 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 305768530 0 0) 0x7f804015afd0 con 0x7f8040153c30 2023-10-06T02:30:43.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.818+0000 7f8044d99700 1 -- 172.21.15.31:0/2684486268 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f802c003570 con 0x7f8040153c30 2023-10-06T02:30:43.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.818+0000 7f804559a700 1 -- 172.21.15.31:0/2684486268 >> v2:172.21.15.62:3300/0 conn(0x7f80400cef40 msgr2=0x7f804015c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:43.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.818+0000 7f804559a700 1 --2- 172.21.15.31:0/2684486268 >> v2:172.21.15.62:3300/0 conn(0x7f80400cef40 0x7f804015c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:43.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.818+0000 7f804559a700 1 -- 172.21.15.31:0/2684486268 >> v1:172.21.15.31:6789/0 conn(0x7f8040153c30 legacy=0x7f804015a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:43.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.818+0000 7f804559a700 1 -- 172.21.15.31:0/2684486268 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f804015b870 con 0x7f804015de50 2023-10-06T02:30:43.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.819+0000 7f804559a700 1 --2- 172.21.15.31:0/2684486268 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f804015de50 0x7f80400cea00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f80300098e0 tx=0x7f80300091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ac1616cd0c2649c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:43.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.819+0000 7f8044d99700 1 -- 172.21.15.31:0/2684486268 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8030011630 con 0x7f804015de50 2023-10-06T02:30:43.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.819+0000 7f8044d99700 1 -- 172.21.15.31:0/2684486268 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8030011790 con 0x7f804015de50 2023-10-06T02:30:43.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.819+0000 7f8044d99700 1 -- 172.21.15.31:0/2684486268 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8030011b30 con 0x7f804015de50 2023-10-06T02:30:43.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.819+0000 7f8046d9d700 1 -- 172.21.15.31:0/2684486268 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f804015de50 msgr2=0x7f80400cea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:43.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.819+0000 7f8046d9d700 1 --2- 172.21.15.31:0/2684486268 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f804015de50 0x7f80400cea00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f80300098e0 tx=0x7f80300091a0 comp rx=0 tx=0).stop 2023-10-06T02:30:43.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.819+0000 7f8046d9d700 1 -- 172.21.15.31:0/2684486268 shutdown_connections 2023-10-06T02:30:43.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.819+0000 7f8046d9d700 1 --2- 172.21.15.31:0/2684486268 >> v2:172.21.15.62:3300/0 conn(0x7f80400cef40 0x7f804015c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:43.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.819+0000 7f8046d9d700 1 --2- 172.21.15.31:0/2684486268 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f804015de50 0x7f80400cea00 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:43.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.819+0000 7f8046d9d700 1 -- 172.21.15.31:0/2684486268 >> 172.21.15.31:0/2684486268 conn(0x7f80400703a0 msgr2=0x7f80400707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:43.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.820+0000 7f8046d9d700 1 -- 172.21.15.31:0/2684486268 shutdown_connections 2023-10-06T02:30:43.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.820+0000 7f8046d9d700 1 -- 172.21.15.31:0/2684486268 wait complete. 2023-10-06T02:30:43.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.821+0000 7f8046d9d700 1 Processor -- start 2023-10-06T02:30:43.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.821+0000 7f8046d9d700 1 -- start start 2023-10-06T02:30:43.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.822+0000 7f8046d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f80400cef40 0x7f80400b1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:43.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.822+0000 7f8046d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8040153c30 0x7f80400ccce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:43.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.822+0000 7f8046d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f804015de50 0x7f8040151a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:43.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.822+0000 7f8046d9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80400b1490 con 0x7f804015de50 2023-10-06T02:30:43.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.822+0000 7f8046d9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80400b0880 con 0x7f8040153c30 2023-10-06T02:30:43.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.822+0000 7f8046d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80400b0b60 con 0x7f80400cef40 2023-10-06T02:30:43.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.822+0000 7f804659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f804015de50 0x7f8040151a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:43.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.822+0000 7f804659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f804015de50 0x7f8040151a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36134/0 (socket says 172.21.15.31:36134) 2023-10-06T02:30:43.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.822+0000 7f804659c700 1 -- 172.21.15.31:0/2466650770 learned_addr learned my addr 172.21.15.31:0/2466650770 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:43.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.822+0000 7f8045d9b700 1 --2- 172.21.15.31:0/2466650770 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f80400cef40 0x7f80400b1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:43.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.822+0000 7f804559a700 1 --2- 172.21.15.31:0/2466650770 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8040153c30 0x7f80400ccce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:43.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.823+0000 7f804659c700 1 -- 172.21.15.31:0/2466650770 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f80400cef40 msgr2=0x7f80400b1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:43.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.823+0000 7f804659c700 1 --2- 172.21.15.31:0/2466650770 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f80400cef40 0x7f80400b1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:43.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.823+0000 7f804659c700 1 -- 172.21.15.31:0/2466650770 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8040153c30 msgr2=0x7f80400ccce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:43.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.823+0000 7f804659c700 1 --2- 172.21.15.31:0/2466650770 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8040153c30 0x7f80400ccce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:43.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.823+0000 7f804659c700 1 -- 172.21.15.31:0/2466650770 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8030009420 con 0x7f804015de50 2023-10-06T02:30:43.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.823+0000 7f8045d9b700 1 --2- 172.21.15.31:0/2466650770 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f80400cef40 0x7f80400b1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:30:43.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.823+0000 7f804659c700 1 --2- 172.21.15.31:0/2466650770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f804015de50 0x7f8040151a20 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f8038009df0 tx=0x7f803800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:43.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.823+0000 7f8035ffb700 1 -- 172.21.15.31:0/2466650770 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803800cb50 con 0x7f804015de50 2023-10-06T02:30:43.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.823+0000 7f8035ffb700 1 -- 172.21.15.31:0/2466650770 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f803800f040 con 0x7f804015de50 2023-10-06T02:30:43.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.823+0000 7f8035ffb700 1 -- 172.21.15.31:0/2466650770 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8038014480 con 0x7f804015de50 2023-10-06T02:30:43.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.823+0000 7f8046d9d700 1 -- 172.21.15.31:0/2466650770 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80400b0e40 con 0x7f804015de50 2023-10-06T02:30:43.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.823+0000 7f8046d9d700 1 -- 172.21.15.31:0/2466650770 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80400cd220 con 0x7f804015de50 2023-10-06T02:30:43.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.824+0000 7f8046d9d700 1 -- 172.21.15.31:0/2466650770 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f804004c3c0 con 0x7f804015de50 2023-10-06T02:30:43.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.825+0000 7f8035ffb700 1 -- 172.21.15.31:0/2466650770 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f8038014830 con 0x7f804015de50 2023-10-06T02:30:43.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.826+0000 7f8035ffb700 1 --2- 172.21.15.31:0/2466650770 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f80240674d0 0x7f8024069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:43.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.826+0000 7f8035ffb700 1 -- 172.21.15.31:0/2466650770 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5320+0+0 (secure 0 0 0) 0x7f80380858c0 con 0x7f804015de50 2023-10-06T02:30:43.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.827+0000 7f8045d9b700 1 --2- 172.21.15.31:0/2466650770 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f80240674d0 0x7f8024069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:43.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.827+0000 7f8045d9b700 1 --2- 172.21.15.31:0/2466650770 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f80240674d0 0x7f8024069980 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f803c0058d0 tx=0x7f803c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:43.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:43.830+0000 7f8035ffb700 1 -- 172.21.15.31:0/2466650770 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f80380522f0 con 0x7f804015de50 2023-10-06T02:30:44.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:44.113+0000 7f8046d9d700 1 -- 172.21.15.31:0/2466650770 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1"]} v 0) v1 -- 0x7f80400cdaf0 con 0x7f804015de50 2023-10-06T02:30:44.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:44.613+0000 7f8035ffb700 1 -- 172.21.15.31:0/2466650770 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1"]}]=0 v427) v1 ==== 152+0+0 (secure 0 0 0) 0x7f8038055910 con 0x7f804015de50 2023-10-06T02:30:44.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:44.616+0000 7f8046d9d700 1 -- 172.21.15.31:0/2466650770 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f80240674d0 msgr2=0x7f8024069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:44.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:44.616+0000 7f8046d9d700 1 --2- 172.21.15.31:0/2466650770 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f80240674d0 0x7f8024069980 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f803c0058d0 tx=0x7f803c009040 comp rx=0 tx=0).stop 2023-10-06T02:30:44.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:44.616+0000 7f8046d9d700 1 -- 172.21.15.31:0/2466650770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f804015de50 msgr2=0x7f8040151a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:44.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:44.616+0000 7f8046d9d700 1 --2- 172.21.15.31:0/2466650770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f804015de50 0x7f8040151a20 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f8038009df0 tx=0x7f803800b040 comp rx=0 tx=0).stop 2023-10-06T02:30:44.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:44.616+0000 7f8046d9d700 1 -- 172.21.15.31:0/2466650770 shutdown_connections 2023-10-06T02:30:44.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:44.616+0000 7f8046d9d700 1 --2- 172.21.15.31:0/2466650770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f804015de50 0x7f8040151a20 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:44.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:44.616+0000 7f8046d9d700 1 --2- 172.21.15.31:0/2466650770 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f80240674d0 0x7f8024069980 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:44.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:44.616+0000 7f8046d9d700 1 --2- 172.21.15.31:0/2466650770 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f80400cef40 0x7f80400b1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:44.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:44.616+0000 7f8046d9d700 1 --2- 172.21.15.31:0/2466650770 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8040153c30 0x7f80400ccce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:44.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:44.616+0000 7f8046d9d700 1 -- 172.21.15.31:0/2466650770 >> 172.21.15.31:0/2466650770 conn(0x7f80400703a0 msgr2=0x7f80400c68d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:44.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:44.617+0000 7f8046d9d700 1 -- 172.21.15.31:0/2466650770 shutdown_connections 2023-10-06T02:30:44.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:44.617+0000 7f8046d9d700 1 -- 172.21.15.31:0/2466650770 wait complete. 2023-10-06T02:30:44.638 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T02:30:44.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:44.829+0000 7f2be359e700 1 Processor -- start 2023-10-06T02:30:44.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:44.829+0000 7f2be359e700 1 -- start start 2023-10-06T02:30:44.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:44.830+0000 7f2be359e700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2be4159550 0x7f2be40ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:44.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:44.830+0000 7f2be359e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2be4159920 0x7f2be40ad090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:44.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:44.830+0000 7f2be359e700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2be40acc00 con 0x7f2be415dfd0 2023-10-06T02:30:44.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:44.830+0000 7f2be359e700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2be40abff0 con 0x7f2be4159550 2023-10-06T02:30:44.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:44.830+0000 7f2be359e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2be40ac2d0 con 0x7f2be4159920 2023-10-06T02:30:44.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:44.831+0000 7f2be2d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2be4159920 0x7f2be40ad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:44.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:44.831+0000 7f2be259c700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2be4159550 0x7f2be40ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:44.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:44.831+0000 7f2be1d9b700 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f2be415dfd0 0x7f2be40aedb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:45530/0 (socket says 172.21.15.31:45530) 2023-10-06T02:30:44.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:44.831+0000 7f2be1d9b700 1 -- 172.21.15.31:0/3324086458 learned_addr learned my addr 172.21.15.31:0/3324086458 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:30:44.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:44.831+0000 7f2be159a700 1 -- 172.21.15.31:0/3324086458 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1855474944 0 0) 0x7f2be40acc00 con 0x7f2be415dfd0 2023-10-06T02:30:44.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:44.831+0000 7f2be159a700 1 -- 172.21.15.31:0/3324086458 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2bcc003570 con 0x7f2be415dfd0 2023-10-06T02:30:44.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:44.832+0000 7f2be259c700 1 -- 172.21.15.31:0/3324086458 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2be4159920 msgr2=0x7f2be40ad090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:44.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:44.832+0000 7f2be259c700 1 --2- 172.21.15.31:0/3324086458 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2be4159920 0x7f2be40ad090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:44.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:44.832+0000 7f2be259c700 1 -- 172.21.15.31:0/3324086458 >> v1:172.21.15.62:6789/0 conn(0x7f2be415dfd0 legacy=0x7f2be40aedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:44.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:44.832+0000 7f2be259c700 1 -- 172.21.15.31:0/3324086458 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2be40ac5b0 con 0x7f2be4159550 2023-10-06T02:30:44.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:44.832+0000 7f2be259c700 1 --2- 172.21.15.31:0/3324086458 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2be4159550 0x7f2be40ceac0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f2bd40098e0 tx=0x7f2bd40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7aae9c1a505c753f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:44.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:44.832+0000 7f2be159a700 1 -- 172.21.15.31:0/3324086458 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd4011630 con 0x7f2be4159550 2023-10-06T02:30:44.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:44.832+0000 7f2be159a700 1 -- 172.21.15.31:0/3324086458 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bd4011790 con 0x7f2be4159550 2023-10-06T02:30:44.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:44.832+0000 7f2be159a700 1 -- 172.21.15.31:0/3324086458 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd4011b10 con 0x7f2be4159550 2023-10-06T02:30:44.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:44.833+0000 7f2be359e700 1 -- 172.21.15.31:0/3324086458 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2be4159550 msgr2=0x7f2be40ceac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:44.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:44.833+0000 7f2be359e700 1 --2- 172.21.15.31:0/3324086458 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2be4159550 0x7f2be40ceac0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f2bd40098e0 tx=0x7f2bd40091a0 comp rx=0 tx=0).stop 2023-10-06T02:30:44.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:44.833+0000 7f2be359e700 1 -- 172.21.15.31:0/3324086458 shutdown_connections 2023-10-06T02:30:44.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:44.833+0000 7f2be359e700 1 --2- 172.21.15.31:0/3324086458 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2be4159920 0x7f2be40ad090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:44.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:44.833+0000 7f2be359e700 1 --2- 172.21.15.31:0/3324086458 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2be4159550 0x7f2be40ceac0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:44.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:44.833+0000 7f2be359e700 1 -- 172.21.15.31:0/3324086458 >> 172.21.15.31:0/3324086458 conn(0x7f2be406ce80 msgr2=0x7f2be40ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:44.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:44.833+0000 7f2be359e700 1 -- 172.21.15.31:0/3324086458 shutdown_connections 2023-10-06T02:30:44.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:44.833+0000 7f2be359e700 1 -- 172.21.15.31:0/3324086458 wait complete. 2023-10-06T02:30:44.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:44.834+0000 7f2be359e700 1 Processor -- start 2023-10-06T02:30:44.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:44.834+0000 7f2be359e700 1 -- start start 2023-10-06T02:30:44.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:44.835+0000 7f2be359e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2be4159550 0x7f2be40c9310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:44.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:44.835+0000 7f2be359e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2be4159920 0x7f2be4153f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:44.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:44.835+0000 7f2be359e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2be415dfd0 0x7f2be40b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:44.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:44.835+0000 7f2be359e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be40ca820 con 0x7f2be4159920 2023-10-06T02:30:44.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:44.835+0000 7f2be359e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be40c9c10 con 0x7f2be4159550 2023-10-06T02:30:44.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:44.835+0000 7f2be359e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2be40c9ef0 con 0x7f2be415dfd0 2023-10-06T02:30:44.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:44.836+0000 7f2be1d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2be4159920 0x7f2be4153f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:44.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:44.836+0000 7f2be1d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2be4159920 0x7f2be4153f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36136/0 (socket says 172.21.15.31:36136) 2023-10-06T02:30:44.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:44.836+0000 7f2be1d9b700 1 -- 172.21.15.31:0/1122623451 learned_addr learned my addr 172.21.15.31:0/1122623451 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:44.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:44.836+0000 7f2be2d9d700 1 --2- 172.21.15.31:0/1122623451 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2be415dfd0 0x7f2be40b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:44.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:44.836+0000 7f2be259c700 1 --2- 172.21.15.31:0/1122623451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2be4159550 0x7f2be40c9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:44.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:44.836+0000 7f2be1d9b700 1 -- 172.21.15.31:0/1122623451 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2be415dfd0 msgr2=0x7f2be40b2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:44.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:44.836+0000 7f2be1d9b700 1 --2- 172.21.15.31:0/1122623451 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2be415dfd0 0x7f2be40b2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:44.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:44.836+0000 7f2be1d9b700 1 -- 172.21.15.31:0/1122623451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2be4159550 msgr2=0x7f2be40c9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:44.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:44.836+0000 7f2be1d9b700 1 --2- 172.21.15.31:0/1122623451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2be4159550 0x7f2be40c9310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:44.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:44.836+0000 7f2be1d9b700 1 -- 172.21.15.31:0/1122623451 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bd4009420 con 0x7f2be4159920 2023-10-06T02:30:44.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:44.836+0000 7f2be1d9b700 1 --2- 172.21.15.31:0/1122623451 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2be4159920 0x7f2be4153f20 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f2bd80011e0 tx=0x7f2bd8007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:44.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:44.837+0000 7f2bd27fc700 1 -- 172.21.15.31:0/1122623451 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd8005b30 con 0x7f2be4159920 2023-10-06T02:30:44.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:44.837+0000 7f2bd27fc700 1 -- 172.21.15.31:0/1122623451 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bd8005c90 con 0x7f2be4159920 2023-10-06T02:30:44.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:44.837+0000 7f2bd27fc700 1 -- 172.21.15.31:0/1122623451 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd8007320 con 0x7f2be4159920 2023-10-06T02:30:44.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:44.837+0000 7f2be359e700 1 -- 172.21.15.31:0/1122623451 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2be40ca1d0 con 0x7f2be4159920 2023-10-06T02:30:44.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:44.837+0000 7f2be359e700 1 -- 172.21.15.31:0/1122623451 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2be40c9850 con 0x7f2be4159920 2023-10-06T02:30:44.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:44.838+0000 7f2be359e700 1 -- 172.21.15.31:0/1122623451 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2be4169c50 con 0x7f2be4159920 2023-10-06T02:30:44.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:44.839+0000 7f2bd27fc700 1 -- 172.21.15.31:0/1122623451 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f2bd8007690 con 0x7f2be4159920 2023-10-06T02:30:44.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:44.840+0000 7f2bd27fc700 1 --2- 172.21.15.31:0/1122623451 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f2bc40674d0 0x7f2bc4069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:44.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:44.840+0000 7f2bd27fc700 1 -- 172.21.15.31:0/1122623451 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5320+0+0 (secure 0 0 0) 0x7f2bd80841e0 con 0x7f2be4159920 2023-10-06T02:30:44.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:44.840+0000 7f2be259c700 1 --2- 172.21.15.31:0/1122623451 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f2bc40674d0 0x7f2bc4069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:44.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:44.841+0000 7f2be259c700 1 --2- 172.21.15.31:0/1122623451 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f2bc40674d0 0x7f2bc4069980 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2bd40091e0 tx=0x7f2bd4017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:44.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:44.844+0000 7f2bd27fc700 1 -- 172.21.15.31:0/1122623451 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2bd8050d20 con 0x7f2be4159920 2023-10-06T02:30:45.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.046+0000 7f2be359e700 1 -- 172.21.15.31:0/1122623451 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f2be404c3c0 con 0x7f2be4159920 2023-10-06T02:30:45.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.050+0000 7f2bd27fc700 1 -- 172.21.15.31:0/1122623451 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v26)=0 v26) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2bd8054340 con 0x7f2be4159920 2023-10-06T02:30:45.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.052+0000 7f2be359e700 1 -- 172.21.15.31:0/1122623451 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f2bc40674d0 msgr2=0x7f2bc4069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:45.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.052+0000 7f2be359e700 1 --2- 172.21.15.31:0/1122623451 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f2bc40674d0 0x7f2bc4069980 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2bd40091e0 tx=0x7f2bd4017000 comp rx=0 tx=0).stop 2023-10-06T02:30:45.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.052+0000 7f2be359e700 1 -- 172.21.15.31:0/1122623451 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2be4159920 msgr2=0x7f2be4153f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:45.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.052+0000 7f2be359e700 1 --2- 172.21.15.31:0/1122623451 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2be4159920 0x7f2be4153f20 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f2bd80011e0 tx=0x7f2bd8007df0 comp rx=0 tx=0).stop 2023-10-06T02:30:45.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.053+0000 7f2be359e700 1 -- 172.21.15.31:0/1122623451 shutdown_connections 2023-10-06T02:30:45.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.053+0000 7f2be359e700 1 --2- 172.21.15.31:0/1122623451 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f2bc40674d0 0x7f2bc4069980 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:45.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.053+0000 7f2be359e700 1 --2- 172.21.15.31:0/1122623451 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2be415dfd0 0x7f2be40b2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:45.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.053+0000 7f2be359e700 1 --2- 172.21.15.31:0/1122623451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2be4159550 0x7f2be40c9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:45.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.053+0000 7f2be359e700 1 --2- 172.21.15.31:0/1122623451 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2be4159920 0x7f2be4153f20 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:45.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.053+0000 7f2be359e700 1 -- 172.21.15.31:0/1122623451 >> 172.21.15.31:0/1122623451 conn(0x7f2be406ce80 msgr2=0x7f2be40ce2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:45.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.053+0000 7f2be359e700 1 -- 172.21.15.31:0/1122623451 shutdown_connections 2023-10-06T02:30:45.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.053+0000 7f2be359e700 1 -- 172.21.15.31:0/1122623451 wait complete. 2023-10-06T02:30:45.072 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T02:30:45.072 DEBUG:teuthology.orchestra.run.smithi031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T02:30:45.154 DEBUG:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T02:30:45.243 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T02:30:45.243 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T02:30:45.243 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:30:45.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.428+0000 7f198f19d700 1 Processor -- start 2023-10-06T02:30:45.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.428+0000 7f198f19d700 1 -- start start 2023-10-06T02:30:45.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.429+0000 7f198f19d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f198813fb20 0x7f198815a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:45.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.430+0000 7f198f19d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f198813fef0 0x7f19880c7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:45.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.430+0000 7f198f19d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f198806aaa0 0x7f19880c7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:45.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.430+0000 7f198f19d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19880c8410 con 0x7f198806aaa0 2023-10-06T02:30:45.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.430+0000 7f198f19d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f198815ae70 con 0x7f198813fef0 2023-10-06T02:30:45.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.430+0000 7f198f19d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f198806af60 con 0x7f198813fb20 2023-10-06T02:30:45.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.430+0000 7f198e99c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f198806aaa0 0x7f19880c7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:45.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.430+0000 7f198e99c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f198806aaa0 0x7f19880c7ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36150/0 (socket says 172.21.15.31:36150) 2023-10-06T02:30:45.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.430+0000 7f198e99c700 1 -- 172.21.15.31:0/2169684911 learned_addr learned my addr 172.21.15.31:0/2169684911 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:45.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.430+0000 7f198d99a700 1 --2- 172.21.15.31:0/2169684911 >> v2:172.21.15.62:3300/0 conn(0x7f198813fef0 0x7f19880c7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:45.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.430+0000 7f198e19b700 1 --2- 172.21.15.31:0/2169684911 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f198813fb20 0x7f198815a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:45.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.430+0000 7f198e99c700 1 -- 172.21.15.31:0/2169684911 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f198813fb20 msgr2=0x7f198815a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:45.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.430+0000 7f198e99c700 1 --2- 172.21.15.31:0/2169684911 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f198813fb20 0x7f198815a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:45.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.430+0000 7f198e99c700 1 -- 172.21.15.31:0/2169684911 >> v2:172.21.15.62:3300/0 conn(0x7f198813fef0 msgr2=0x7f19880c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:45.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.430+0000 7f198e99c700 1 --2- 172.21.15.31:0/2169684911 >> v2:172.21.15.62:3300/0 conn(0x7f198813fef0 0x7f19880c7990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:45.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.430+0000 7f198e99c700 1 -- 172.21.15.31:0/2169684911 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f198806b240 con 0x7f198806aaa0 2023-10-06T02:30:45.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.430+0000 7f198e19b700 1 --2- 172.21.15.31:0/2169684911 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f198813fb20 0x7f198815a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:30:45.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.431+0000 7f198e99c700 1 --2- 172.21.15.31:0/2169684911 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f198806aaa0 0x7f19880c7ed0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f19880cc090 tx=0x7f1980009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2954e09f617b3a4d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:45.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.431+0000 7f198d199700 1 -- 172.21.15.31:0/2169684911 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f198000e070 con 0x7f198806aaa0 2023-10-06T02:30:45.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.431+0000 7f198d199700 1 -- 172.21.15.31:0/2169684911 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f198000ba20 con 0x7f198806aaa0 2023-10-06T02:30:45.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.431+0000 7f198d199700 1 -- 172.21.15.31:0/2169684911 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19800092e0 con 0x7f198806aaa0 2023-10-06T02:30:45.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.431+0000 7f198f19d700 1 -- 172.21.15.31:0/2169684911 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f198806aaa0 msgr2=0x7f19880c7ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:45.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.431+0000 7f198f19d700 1 --2- 172.21.15.31:0/2169684911 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f198806aaa0 0x7f19880c7ed0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f19880cc090 tx=0x7f1980009840 comp rx=0 tx=0).stop 2023-10-06T02:30:45.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.432+0000 7f198f19d700 1 -- 172.21.15.31:0/2169684911 shutdown_connections 2023-10-06T02:30:45.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.432+0000 7f198f19d700 1 --2- 172.21.15.31:0/2169684911 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f198806aaa0 0x7f19880c7ed0 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:45.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.432+0000 7f198f19d700 1 --2- 172.21.15.31:0/2169684911 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f198813fb20 0x7f198815a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:45.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.432+0000 7f198f19d700 1 --2- 172.21.15.31:0/2169684911 >> v2:172.21.15.62:3300/0 conn(0x7f198813fef0 0x7f19880c7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:45.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.432+0000 7f198f19d700 1 -- 172.21.15.31:0/2169684911 >> 172.21.15.31:0/2169684911 conn(0x7f198806d170 msgr2=0x7f19880cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:45.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.432+0000 7f198f19d700 1 -- 172.21.15.31:0/2169684911 shutdown_connections 2023-10-06T02:30:45.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.432+0000 7f198f19d700 1 -- 172.21.15.31:0/2169684911 wait complete. 2023-10-06T02:30:45.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.433+0000 7f198f19d700 1 Processor -- start 2023-10-06T02:30:45.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.433+0000 7f198f19d700 1 -- start start 2023-10-06T02:30:45.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.434+0000 7f198f19d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f198806aaa0 0x7f198815bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:45.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.434+0000 7f198f19d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f198813fb20 0x7f19880c7840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:45.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.434+0000 7f198f19d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f198813fef0 0x7f198816cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:45.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.434+0000 7f198f19d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f198815c410 con 0x7f198813fb20 2023-10-06T02:30:45.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.434+0000 7f198f19d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1988169d40 con 0x7f198806aaa0 2023-10-06T02:30:45.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.434+0000 7f198f19d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f198815b430 con 0x7f198813fef0 2023-10-06T02:30:45.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.435+0000 7f198d99a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f198813fb20 0x7f19880c7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:45.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.435+0000 7f198d99a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f198813fb20 0x7f19880c7840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36166/0 (socket says 172.21.15.31:36166) 2023-10-06T02:30:45.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.435+0000 7f198d99a700 1 -- 172.21.15.31:0/1512311813 learned_addr learned my addr 172.21.15.31:0/1512311813 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:45.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.435+0000 7f198e19b700 1 --2- 172.21.15.31:0/1512311813 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f198806aaa0 0x7f198815bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:45.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.435+0000 7f198d99a700 1 -- 172.21.15.31:0/1512311813 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f198813fef0 msgr2=0x7f198816cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:45.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.435+0000 7f198e99c700 1 --2- 172.21.15.31:0/1512311813 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f198813fef0 0x7f198816cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:45.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.435+0000 7f198d99a700 1 --2- 172.21.15.31:0/1512311813 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f198813fef0 0x7f198816cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:45.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.435+0000 7f198d99a700 1 -- 172.21.15.31:0/1512311813 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f198806aaa0 msgr2=0x7f198815bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:45.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.435+0000 7f198d99a700 1 --2- 172.21.15.31:0/1512311813 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f198806aaa0 0x7f198815bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:45.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.435+0000 7f198d99a700 1 -- 172.21.15.31:0/1512311813 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1980009ae0 con 0x7f198813fb20 2023-10-06T02:30:45.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.435+0000 7f198e19b700 1 --2- 172.21.15.31:0/1512311813 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f198806aaa0 0x7f198815bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:30:45.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.435+0000 7f198e99c700 1 --2- 172.21.15.31:0/1512311813 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f198813fef0 0x7f198816cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:30:45.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.435+0000 7f198d99a700 1 --2- 172.21.15.31:0/1512311813 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f198813fb20 0x7f19880c7840 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f197800ec80 tx=0x7f1978009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:45.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.436+0000 7f197e7fc700 1 -- 172.21.15.31:0/1512311813 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f197800c9d0 con 0x7f198813fb20 2023-10-06T02:30:45.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.436+0000 7f197e7fc700 1 -- 172.21.15.31:0/1512311813 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f197800cb30 con 0x7f198813fb20 2023-10-06T02:30:45.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.436+0000 7f197e7fc700 1 -- 172.21.15.31:0/1512311813 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1978013470 con 0x7f198813fb20 2023-10-06T02:30:45.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.436+0000 7f198f19d700 1 -- 172.21.15.31:0/1512311813 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f198816da40 con 0x7f198813fb20 2023-10-06T02:30:45.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.436+0000 7f198f19d700 1 -- 172.21.15.31:0/1512311813 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f198815b6d0 con 0x7f198813fb20 2023-10-06T02:30:45.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.437+0000 7f198f19d700 1 -- 172.21.15.31:0/1512311813 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f198815a250 con 0x7f198813fb20 2023-10-06T02:30:45.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.438+0000 7f197e7fc700 1 -- 172.21.15.31:0/1512311813 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f1978013860 con 0x7f198813fb20 2023-10-06T02:30:45.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.439+0000 7f197e7fc700 1 --2- 172.21.15.31:0/1512311813 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f196c0674d0 0x7f196c069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:45.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.439+0000 7f197e7fc700 1 -- 172.21.15.31:0/1512311813 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5320+0+0 (secure 0 0 0) 0x7f1978085530 con 0x7f198813fb20 2023-10-06T02:30:45.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.439+0000 7f198e19b700 1 --2- 172.21.15.31:0/1512311813 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f196c0674d0 0x7f196c069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:45.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.440+0000 7f198e19b700 1 --2- 172.21.15.31:0/1512311813 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f196c0674d0 0x7f196c069980 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f1984006f00 tx=0x7f1984008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:45.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.443+0000 7f197e7fc700 1 -- 172.21.15.31:0/1512311813 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1978052150 con 0x7f198813fb20 2023-10-06T02:30:45.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.690+0000 7f198f19d700 1 -- 172.21.15.31:0/1512311813 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f198804c3c0 con 0x7f198813fb20 2023-10-06T02:30:45.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.690+0000 7f197e7fc700 1 -- 172.21.15.31:0/1512311813 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+3529 (secure 0 0 0) 0x7f1978017de0 con 0x7f198813fb20 2023-10-06T02:30:45.692 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:30:45.692 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":93,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6611,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/1574790962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1574790962},{"type":"v1","addr":"172.21.15.62:6834","nonce":1574790962}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":85},{"gid":7591,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/2840240410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":2840240410},{"type":"v1","addr":"172.21.15.31:6835","nonce":2840240410}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":85},{"gid":7597,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2500071914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2500071914},{"type":"v1","addr":"172.21.15.62:6835","nonce":2500071914}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":85},{"gid":8409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/3749503957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":3749503957},{"type":"v1","addr":"172.21.15.31:6834","nonce":3749503957}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":93}],"filesystems":[]} 2023-10-06T02:30:45.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.692+0000 7f198f19d700 1 -- 172.21.15.31:0/1512311813 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f196c0674d0 msgr2=0x7f196c069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:45.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.692+0000 7f198f19d700 1 --2- 172.21.15.31:0/1512311813 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f196c0674d0 0x7f196c069980 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f1984006f00 tx=0x7f1984008040 comp rx=0 tx=0).stop 2023-10-06T02:30:45.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.692+0000 7f198f19d700 1 -- 172.21.15.31:0/1512311813 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f198813fb20 msgr2=0x7f19880c7840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:45.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.692+0000 7f198f19d700 1 --2- 172.21.15.31:0/1512311813 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f198813fb20 0x7f19880c7840 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f197800ec80 tx=0x7f1978009120 comp rx=0 tx=0).stop 2023-10-06T02:30:45.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.692+0000 7f198f19d700 1 -- 172.21.15.31:0/1512311813 shutdown_connections 2023-10-06T02:30:45.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.692+0000 7f198f19d700 1 --2- 172.21.15.31:0/1512311813 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f196c0674d0 0x7f196c069980 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:45.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.692+0000 7f198f19d700 1 --2- 172.21.15.31:0/1512311813 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f198813fef0 0x7f198816cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:45.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.692+0000 7f198f19d700 1 --2- 172.21.15.31:0/1512311813 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f198806aaa0 0x7f198815bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:45.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.692+0000 7f198f19d700 1 --2- 172.21.15.31:0/1512311813 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f198813fb20 0x7f19880c7840 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:45.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.692+0000 7f198f19d700 1 -- 172.21.15.31:0/1512311813 >> 172.21.15.31:0/1512311813 conn(0x7f198806d170 msgr2=0x7f198806de50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:45.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.693+0000 7f198f19d700 1 -- 172.21.15.31:0/1512311813 shutdown_connections 2023-10-06T02:30:45.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.693+0000 7f198f19d700 1 -- 172.21.15.31:0/1512311813 wait complete. 2023-10-06T02:30:45.697 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 93 2023-10-06T02:30:45.711 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T02:30:45.711 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T02:30:45.712 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T02:30:45.712 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T02:30:45.712 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T02:30:45.712 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T02:30:45.713 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T02:30:45.713 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T02:30:45.713 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T02:30:45.713 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T02:30:45.714 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T02:30:45.714 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T02:30:45.714 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T02:30:45.714 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T02:30:45.714 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T02:30:45.714 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T02:30:45.725 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T02:30:45.725 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T02:30:45.727 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T02:30:45.727 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T02:30:45.734 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T02:30:45.734 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T02:30:45.735 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T02:30:45.735 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T02:30:45.772 INFO:tasks.ceph.mds.a:Started 2023-10-06T02:30:45.772 DEBUG:teuthology.parallel:result is None 2023-10-06T02:30:45.776 INFO:tasks.ceph.mds.c:Started 2023-10-06T02:30:45.776 DEBUG:teuthology.parallel:result is None 2023-10-06T02:30:45.778 INFO:tasks.ceph.mds.d:Started 2023-10-06T02:30:45.779 DEBUG:teuthology.parallel:result is None 2023-10-06T02:30:45.781 INFO:tasks.ceph.mds.b:Started 2023-10-06T02:30:45.782 DEBUG:teuthology.parallel:result is None 2023-10-06T02:30:45.782 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T02:30:45.907 INFO:tasks.ceph.mds.a.smithi031.stdout:starting mds.a at 2023-10-06T02:30:45.912 INFO:tasks.ceph.mds.c.smithi031.stdout:starting mds.c at 2023-10-06T02:30:45.920 INFO:tasks.ceph.mds.b.smithi062.stdout:starting mds.b at 2023-10-06T02:30:45.920 INFO:tasks.ceph.mds.d.smithi062.stdout:starting mds.d at 2023-10-06T02:30:45.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.979+0000 7f1f1519d700 1 Processor -- start 2023-10-06T02:30:45.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.980+0000 7f1f1519d700 1 -- start start 2023-10-06T02:30:45.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.981+0000 7f1f1519d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f1f1013fb20 0x7f1f1015a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:45.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.981+0000 7f1f1519d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1f1013fef0 0x7f1f100c7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:45.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.981+0000 7f1f1519d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1f1006aaa0 0x7f1f100c7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:45.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.981+0000 7f1f1519d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f100c8410 con 0x7f1f1013fef0 2023-10-06T02:30:45.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.981+0000 7f1f1519d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f1015ae70 con 0x7f1f1013fb20 2023-10-06T02:30:45.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.981+0000 7f1f1519d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1006af60 con 0x7f1f1006aaa0 2023-10-06T02:30:45.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.981+0000 7f1f0f7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1f1013fef0 0x7f1f100c7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:45.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.981+0000 7f1f0f7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1f1013fef0 0x7f1f100c7990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36208/0 (socket says 172.21.15.31:36208) 2023-10-06T02:30:45.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.981+0000 7f1f0f7fe700 1 -- 172.21.15.31:0/2196838561 learned_addr learned my addr 172.21.15.31:0/2196838561 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:45.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.981+0000 7f1f0ffff700 1 --2- 172.21.15.31:0/2196838561 >> v2:172.21.15.62:3300/0 conn(0x7f1f1013fb20 0x7f1f1015a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:45.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.981+0000 7f1f1499c700 1 --2- 172.21.15.31:0/2196838561 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1f1006aaa0 0x7f1f100c7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:45.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.982+0000 7f1f0f7fe700 1 -- 172.21.15.31:0/2196838561 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1f1006aaa0 msgr2=0x7f1f100c7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:45.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.982+0000 7f1f0f7fe700 1 --2- 172.21.15.31:0/2196838561 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1f1006aaa0 0x7f1f100c7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:45.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.982+0000 7f1f0f7fe700 1 -- 172.21.15.31:0/2196838561 >> v2:172.21.15.62:3300/0 conn(0x7f1f1013fb20 msgr2=0x7f1f1015a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:45.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.982+0000 7f1f0f7fe700 1 --2- 172.21.15.31:0/2196838561 >> v2:172.21.15.62:3300/0 conn(0x7f1f1013fb20 0x7f1f1015a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:45.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.982+0000 7f1f0f7fe700 1 -- 172.21.15.31:0/2196838561 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f1006b240 con 0x7f1f1013fef0 2023-10-06T02:30:45.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.982+0000 7f1f1499c700 1 --2- 172.21.15.31:0/2196838561 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1f1006aaa0 0x7f1f100c7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:30:45.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.982+0000 7f1f0f7fe700 1 --2- 172.21.15.31:0/2196838561 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f1013fef0 0x7f1f100c7990 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f1f100695b0 tx=0x7f1f04009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf7ab25d42c0ccd7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:45.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.982+0000 7f1f0effd700 1 -- 172.21.15.31:0/2196838561 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f04005c10 con 0x7f1f1013fef0 2023-10-06T02:30:45.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.982+0000 7f1f0effd700 1 -- 172.21.15.31:0/2196838561 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f04005d70 con 0x7f1f1013fef0 2023-10-06T02:30:45.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.982+0000 7f1f0effd700 1 -- 172.21.15.31:0/2196838561 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f04006260 con 0x7f1f1013fef0 2023-10-06T02:30:45.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.983+0000 7f1f1519d700 1 -- 172.21.15.31:0/2196838561 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f1013fef0 msgr2=0x7f1f100c7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:45.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.983+0000 7f1f1519d700 1 --2- 172.21.15.31:0/2196838561 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f1013fef0 0x7f1f100c7990 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f1f100695b0 tx=0x7f1f04009220 comp rx=0 tx=0).stop 2023-10-06T02:30:45.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.983+0000 7f1f1519d700 1 -- 172.21.15.31:0/2196838561 shutdown_connections 2023-10-06T02:30:45.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.983+0000 7f1f1519d700 1 --2- 172.21.15.31:0/2196838561 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1f1006aaa0 0x7f1f100c7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:45.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.983+0000 7f1f1519d700 1 --2- 172.21.15.31:0/2196838561 >> v2:172.21.15.62:3300/0 conn(0x7f1f1013fb20 0x7f1f1015a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:45.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.983+0000 7f1f1519d700 1 --2- 172.21.15.31:0/2196838561 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f1013fef0 0x7f1f100c7990 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:45.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.983+0000 7f1f1519d700 1 -- 172.21.15.31:0/2196838561 >> 172.21.15.31:0/2196838561 conn(0x7f1f1006d170 msgr2=0x7f1f100cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:45.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.983+0000 7f1f1519d700 1 -- 172.21.15.31:0/2196838561 shutdown_connections 2023-10-06T02:30:45.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.983+0000 7f1f1519d700 1 -- 172.21.15.31:0/2196838561 wait complete. 2023-10-06T02:30:45.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.984+0000 7f1f1519d700 1 Processor -- start 2023-10-06T02:30:45.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.985+0000 7f1f1519d700 1 -- start start 2023-10-06T02:30:45.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.985+0000 7f1f1519d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f1006aaa0 0x7f1f10155d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:45.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.985+0000 7f1f1519d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1f1013fb20 0x7f1f10156950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:45.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.986+0000 7f1f1519d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f1013fef0 0x7f1f100cbd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:45.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.986+0000 7f1f1519d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f101581b0 con 0x7f1f1013fef0 2023-10-06T02:30:45.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.986+0000 7f1f1519d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f10157330 con 0x7f1f1006aaa0 2023-10-06T02:30:45.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.986+0000 7f1f1519d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f10157610 con 0x7f1f1013fb20 2023-10-06T02:30:45.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.986+0000 7f1f1499c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f1013fef0 0x7f1f100cbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:45.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.986+0000 7f1f1499c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f1013fef0 0x7f1f100cbd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36216/0 (socket says 172.21.15.31:36216) 2023-10-06T02:30:45.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.986+0000 7f1f1499c700 1 -- 172.21.15.31:0/822957856 learned_addr learned my addr 172.21.15.31:0/822957856 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:45.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.986+0000 7f1f0f7fe700 1 --2- 172.21.15.31:0/822957856 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1f1013fb20 0x7f1f10156950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:45.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.986+0000 7f1f0ffff700 1 --2- 172.21.15.31:0/822957856 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f1006aaa0 0x7f1f10155d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:45.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.986+0000 7f1f1499c700 1 -- 172.21.15.31:0/822957856 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1f1013fb20 msgr2=0x7f1f10156950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:45.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.986+0000 7f1f1499c700 1 --2- 172.21.15.31:0/822957856 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1f1013fb20 0x7f1f10156950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:45.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.986+0000 7f1f1499c700 1 -- 172.21.15.31:0/822957856 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f1006aaa0 msgr2=0x7f1f10155d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:45.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.986+0000 7f1f1499c700 1 --2- 172.21.15.31:0/822957856 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f1006aaa0 0x7f1f10155d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:45.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.986+0000 7f1f1499c700 1 -- 172.21.15.31:0/822957856 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f04009480 con 0x7f1f1013fef0 2023-10-06T02:30:45.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.986+0000 7f1f0ffff700 1 --2- 172.21.15.31:0/822957856 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f1006aaa0 0x7f1f10155d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:30:45.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.986+0000 7f1f0f7fe700 1 --2- 172.21.15.31:0/822957856 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1f1013fb20 0x7f1f10156950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:30:45.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.987+0000 7f1f1499c700 1 --2- 172.21.15.31:0/822957856 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f1013fef0 0x7f1f100cbd30 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f1f08009f90 tx=0x7f1f080099b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:45.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.987+0000 7f1ef7fff700 1 -- 172.21.15.31:0/822957856 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f0800dd40 con 0x7f1f1013fef0 2023-10-06T02:30:45.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.987+0000 7f1ef7fff700 1 -- 172.21.15.31:0/822957856 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f0800f040 con 0x7f1f1013fef0 2023-10-06T02:30:45.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.987+0000 7f1ef7fff700 1 -- 172.21.15.31:0/822957856 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f08015690 con 0x7f1f1013fef0 2023-10-06T02:30:45.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.987+0000 7f1f1519d700 1 -- 172.21.15.31:0/822957856 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f101578f0 con 0x7f1f1013fef0 2023-10-06T02:30:45.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.987+0000 7f1f1519d700 1 -- 172.21.15.31:0/822957856 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f10156240 con 0x7f1f1013fef0 2023-10-06T02:30:45.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.988+0000 7f1f1519d700 1 -- 172.21.15.31:0/822957856 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f10158320 con 0x7f1f1013fef0 2023-10-06T02:30:45.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.989+0000 7f1ef7fff700 1 -- 172.21.15.31:0/822957856 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f1f0801e3b0 con 0x7f1f1013fef0 2023-10-06T02:30:45.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.990+0000 7f1ef7fff700 1 --2- 172.21.15.31:0/822957856 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f1efc0674d0 0x7f1efc069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:45.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.990+0000 7f1ef7fff700 1 -- 172.21.15.31:0/822957856 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5320+0+0 (secure 0 0 0) 0x7f1f080854a0 con 0x7f1f1013fef0 2023-10-06T02:30:45.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.990+0000 7f1f0ffff700 1 --2- 172.21.15.31:0/822957856 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f1efc0674d0 0x7f1efc069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:45.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.991+0000 7f1f0ffff700 1 --2- 172.21.15.31:0/822957856 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f1efc0674d0 0x7f1efc069980 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f1f00006f00 tx=0x7f1f00008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:45.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:45.994+0000 7f1ef7fff700 1 -- 172.21.15.31:0/822957856 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1f08051ed0 con 0x7f1f1013fef0 2023-10-06T02:30:46.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:46.211+0000 7f1f1519d700 1 -- 172.21.15.31:0/822957856 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f1f1004c3c0 con 0x7f1f1013fef0 2023-10-06T02:30:46.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:46.631+0000 7f1ef7fff700 1 -- 172.21.15.31:0/822957856 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v138) v1 ==== 97+0+0 (secure 0 0 0) 0x7f1f080554f0 con 0x7f1f1013fef0 2023-10-06T02:30:46.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:46.634+0000 7f1f1519d700 1 -- 172.21.15.31:0/822957856 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f1efc0674d0 msgr2=0x7f1efc069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:46.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:46.634+0000 7f1f1519d700 1 --2- 172.21.15.31:0/822957856 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f1efc0674d0 0x7f1efc069980 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f1f00006f00 tx=0x7f1f00008040 comp rx=0 tx=0).stop 2023-10-06T02:30:46.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:46.634+0000 7f1f1519d700 1 -- 172.21.15.31:0/822957856 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f1013fef0 msgr2=0x7f1f100cbd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:46.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:46.634+0000 7f1f1519d700 1 --2- 172.21.15.31:0/822957856 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f1013fef0 0x7f1f100cbd30 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f1f08009f90 tx=0x7f1f080099b0 comp rx=0 tx=0).stop 2023-10-06T02:30:46.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:46.634+0000 7f1f1519d700 1 -- 172.21.15.31:0/822957856 shutdown_connections 2023-10-06T02:30:46.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:46.634+0000 7f1f1519d700 1 --2- 172.21.15.31:0/822957856 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f1013fef0 0x7f1f100cbd30 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:46.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:46.634+0000 7f1f1519d700 1 --2- 172.21.15.31:0/822957856 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f1006aaa0 0x7f1f10155d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:46.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:46.634+0000 7f1f1519d700 1 --2- 172.21.15.31:0/822957856 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f1efc0674d0 0x7f1efc069980 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:46.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:46.634+0000 7f1f1519d700 1 --2- 172.21.15.31:0/822957856 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1f1013fb20 0x7f1f10156950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:46.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:46.634+0000 7f1f1519d700 1 -- 172.21.15.31:0/822957856 >> 172.21.15.31:0/822957856 conn(0x7f1f1006d170 msgr2=0x7f1f100af020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:46.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:46.634+0000 7f1f1519d700 1 -- 172.21.15.31:0/822957856 shutdown_connections 2023-10-06T02:30:46.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:46.634+0000 7f1f1519d700 1 -- 172.21.15.31:0/822957856 wait complete. 2023-10-06T02:30:46.637 INFO:teuthology.orchestra.run.smithi031.stderr: removed all blocklist entries 2023-10-06T02:30:46.655 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T02:30:46.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:46.842+0000 7f8c5359e700 1 Processor -- start 2023-10-06T02:30:46.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:46.842+0000 7f8c5359e700 1 -- start start 2023-10-06T02:30:46.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:46.843+0000 7f8c5359e700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8c54142fd0 0x7f8c540c7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:46.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:46.843+0000 7f8c5359e700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c540ca480 con 0x7f8c540c7dc0 2023-10-06T02:30:46.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:46.843+0000 7f8c5359e700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c5415e240 con 0x7f8c54142c00 2023-10-06T02:30:46.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:46.843+0000 7f8c5359e700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c5406b350 con 0x7f8c54142fd0 2023-10-06T02:30:46.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:46.844+0000 7f8c52d9d700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f8c540c7dc0 0x7f8c5406c6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60808/0 (socket says 172.21.15.31:60808) 2023-10-06T02:30:46.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:46.844+0000 7f8c52d9d700 1 -- 172.21.15.31:0/4031922829 learned_addr learned my addr 172.21.15.31:0/4031922829 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:30:46.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:46.844+0000 7f8c51d9b700 1 --2- 172.21.15.31:0/4031922829 >> v2:172.21.15.62:3300/0 conn(0x7f8c54142fd0 0x7f8c540c7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:46.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:46.844+0000 7f8c5159a700 1 -- 172.21.15.31:0/4031922829 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2707485170 0 0) 0x7f8c540ca480 con 0x7f8c540c7dc0 2023-10-06T02:30:46.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:46.844+0000 7f8c5159a700 1 -- 172.21.15.31:0/4031922829 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c3c003570 con 0x7f8c540c7dc0 2023-10-06T02:30:46.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:46.844+0000 7f8c5159a700 1 -- 172.21.15.31:0/4031922829 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1622622705 0 0) 0x7f8c5415e240 con 0x7f8c54142c00 2023-10-06T02:30:46.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:46.844+0000 7f8c5159a700 1 -- 172.21.15.31:0/4031922829 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c540ca480 con 0x7f8c54142c00 2023-10-06T02:30:46.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:46.844+0000 7f8c51d9b700 1 -- 172.21.15.31:0/4031922829 >> v1:172.21.15.62:6789/0 conn(0x7f8c54142c00 legacy=0x7f8c5406b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:46.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:46.844+0000 7f8c51d9b700 1 -- 172.21.15.31:0/4031922829 >> v1:172.21.15.31:6789/0 conn(0x7f8c540c7dc0 legacy=0x7f8c5406c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:46.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:46.844+0000 7f8c51d9b700 1 -- 172.21.15.31:0/4031922829 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c540c81d0 con 0x7f8c54142fd0 2023-10-06T02:30:46.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:46.845+0000 7f8c51d9b700 1 --2- 172.21.15.31:0/4031922829 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c54142fd0 0x7f8c540c7880 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f8c480098e0 tx=0x7f8c480091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8bf433956afec4d1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:46.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:46.845+0000 7f8c5159a700 1 -- 172.21.15.31:0/4031922829 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c48011630 con 0x7f8c54142fd0 2023-10-06T02:30:46.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:46.845+0000 7f8c5159a700 1 -- 172.21.15.31:0/4031922829 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c48011790 con 0x7f8c54142fd0 2023-10-06T02:30:46.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:46.845+0000 7f8c5159a700 1 -- 172.21.15.31:0/4031922829 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c48011b30 con 0x7f8c54142fd0 2023-10-06T02:30:46.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:46.845+0000 7f8c5359e700 1 -- 172.21.15.31:0/4031922829 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c54142fd0 msgr2=0x7f8c540c7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:46.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:46.845+0000 7f8c5359e700 1 --2- 172.21.15.31:0/4031922829 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c54142fd0 0x7f8c540c7880 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f8c480098e0 tx=0x7f8c480091a0 comp rx=0 tx=0).stop 2023-10-06T02:30:46.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:46.845+0000 7f8c5359e700 1 -- 172.21.15.31:0/4031922829 shutdown_connections 2023-10-06T02:30:46.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:46.846+0000 7f8c5359e700 1 --2- 172.21.15.31:0/4031922829 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c54142fd0 0x7f8c540c7880 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:46.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:46.846+0000 7f8c5359e700 1 -- 172.21.15.31:0/4031922829 >> 172.21.15.31:0/4031922829 conn(0x7f8c540675a0 msgr2=0x7f8c540679a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:46.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:46.846+0000 7f8c5359e700 1 -- 172.21.15.31:0/4031922829 shutdown_connections 2023-10-06T02:30:46.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:46.846+0000 7f8c5359e700 1 -- 172.21.15.31:0/4031922829 wait complete. 2023-10-06T02:30:46.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:46.847+0000 7f8c5359e700 1 Processor -- start 2023-10-06T02:30:46.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:46.847+0000 7f8c5359e700 1 -- start start 2023-10-06T02:30:46.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:46.848+0000 7f8c5359e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8c540c7dc0 0x7f8c54158eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:46.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:46.848+0000 7f8c5359e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c54142c00 0x7f8c54159e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:46.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:46.848+0000 7f8c5359e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c54142fd0 0x7f8c5415a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:46.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:46.848+0000 7f8c5359e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c5415bf50 con 0x7f8c54142fd0 2023-10-06T02:30:46.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:46.848+0000 7f8c5359e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c540b2980 con 0x7f8c54142c00 2023-10-06T02:30:46.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:46.848+0000 7f8c5359e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c540b1e00 con 0x7f8c540c7dc0 2023-10-06T02:30:46.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:46.848+0000 7f8c52d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c54142fd0 0x7f8c5415a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:46.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:46.848+0000 7f8c52d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c54142fd0 0x7f8c5415a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36220/0 (socket says 172.21.15.31:36220) 2023-10-06T02:30:46.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:46.848+0000 7f8c52d9d700 1 -- 172.21.15.31:0/4258068390 learned_addr learned my addr 172.21.15.31:0/4258068390 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:46.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:46.849+0000 7f8c51d9b700 1 --2- 172.21.15.31:0/4258068390 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c54142c00 0x7f8c54159e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:46.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:46.849+0000 7f8c5259c700 1 --2- 172.21.15.31:0/4258068390 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8c540c7dc0 0x7f8c54158eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:46.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:46.849+0000 7f8c52d9d700 1 -- 172.21.15.31:0/4258068390 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8c540c7dc0 msgr2=0x7f8c54158eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:46.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:46.849+0000 7f8c52d9d700 1 --2- 172.21.15.31:0/4258068390 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8c540c7dc0 0x7f8c54158eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:46.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:46.849+0000 7f8c52d9d700 1 -- 172.21.15.31:0/4258068390 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c54142c00 msgr2=0x7f8c54159e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:46.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:46.849+0000 7f8c52d9d700 1 --2- 172.21.15.31:0/4258068390 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c54142c00 0x7f8c54159e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:46.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:46.849+0000 7f8c52d9d700 1 -- 172.21.15.31:0/4258068390 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c48009420 con 0x7f8c54142fd0 2023-10-06T02:30:46.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:46.849+0000 7f8c51d9b700 1 --2- 172.21.15.31:0/4258068390 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c54142c00 0x7f8c54159e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:30:46.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:46.849+0000 7f8c52d9d700 1 --2- 172.21.15.31:0/4258068390 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c54142fd0 0x7f8c5415a380 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c00cc60 tx=0x7f8c4c001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:46.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:46.849+0000 7f8c427fc700 1 -- 172.21.15.31:0/4258068390 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c00ce70 con 0x7f8c54142fd0 2023-10-06T02:30:46.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:46.849+0000 7f8c427fc700 1 -- 172.21.15.31:0/4258068390 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c4c00ab70 con 0x7f8c54142fd0 2023-10-06T02:30:46.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:46.849+0000 7f8c427fc700 1 -- 172.21.15.31:0/4258068390 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c0070f0 con 0x7f8c54142fd0 2023-10-06T02:30:46.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:46.849+0000 7f8c5359e700 1 -- 172.21.15.31:0/4258068390 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c540b20e0 con 0x7f8c54142fd0 2023-10-06T02:30:46.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:46.850+0000 7f8c5359e700 1 -- 172.21.15.31:0/4258068390 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c540b26a0 con 0x7f8c54142fd0 2023-10-06T02:30:46.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:46.851+0000 7f8c5359e700 1 -- 172.21.15.31:0/4258068390 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c540ca100 con 0x7f8c54142fd0 2023-10-06T02:30:46.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:46.851+0000 7f8c427fc700 1 -- 172.21.15.31:0/4258068390 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f8c4c0074a0 con 0x7f8c54142fd0 2023-10-06T02:30:46.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:46.853+0000 7f8c427fc700 1 --2- 172.21.15.31:0/4258068390 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f8c340674d0 0x7f8c34069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:46.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:46.853+0000 7f8c427fc700 1 -- 172.21.15.31:0/4258068390 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5234+0+0 (secure 0 0 0) 0x7f8c4c085270 con 0x7f8c54142fd0 2023-10-06T02:30:46.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:46.853+0000 7f8c5259c700 1 --2- 172.21.15.31:0/4258068390 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f8c340674d0 0x7f8c34069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:46.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:46.854+0000 7f8c5259c700 1 --2- 172.21.15.31:0/4258068390 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f8c340674d0 0x7f8c34069980 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f8c5415de90 tx=0x7f8c44009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:46.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:46.856+0000 7f8c427fc700 1 -- 172.21.15.31:0/4258068390 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8c4c055950 con 0x7f8c54142fd0 2023-10-06T02:30:47.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.136+0000 7f8c5359e700 1 -- 172.21.15.31:0/4258068390 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f8c5404c3c0 con 0x7f8c54142fd0 2023-10-06T02:30:47.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.136+0000 7f8c427fc700 1 -- 172.21.15.31:0/4258068390 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v21) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f8c4c0554e0 con 0x7f8c54142fd0 2023-10-06T02:30:47.138 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:30:47.138 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T02:30:47.138 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_dump": [ 2023-10-06T02:30:47.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:30:47.138 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.a", 2023-10-06T02:30:47.138 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBLbx9lbjrOGxAAVoSOoDaZb1temL41F4Nong==", 2023-10-06T02:30:47.138 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:30:47.138 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:30:47.138 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T02:30:47.139 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T02:30:47.139 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:30:47.139 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:30:47.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:30:47.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:30:47.139 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.b", 2023-10-06T02:30:47.139 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBLbx9lFgsTMRAATTklzroiyQbTLiZc9j1BHQ==", 2023-10-06T02:30:47.139 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:30:47.139 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:30:47.139 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T02:30:47.139 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T02:30:47.139 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:30:47.140 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:30:47.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:30:47.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:30:47.140 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.c", 2023-10-06T02:30:47.140 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBLbx9lezVtJhAA8wK7+dJ1OmIc+iXJkexK0w==", 2023-10-06T02:30:47.140 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:30:47.140 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:30:47.140 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T02:30:47.140 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T02:30:47.140 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:30:47.140 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:30:47.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:30:47.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:30:47.141 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.d", 2023-10-06T02:30:47.141 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBMbx9l74gWABAApBu2nVv12rGA+OBfBnEc3A==", 2023-10-06T02:30:47.141 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:30:47.141 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:30:47.141 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T02:30:47.141 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T02:30:47.141 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:30:47.141 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:30:47.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:30:47.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:30:47.141 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.0", 2023-10-06T02:30:47.141 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBPbx9lgj6+MBAAXSUD0GCG+rO/9DQBZYt/8w==", 2023-10-06T02:30:47.142 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:30:47.142 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:30:47.142 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:30:47.142 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:30:47.142 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:30:47.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:30:47.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:30:47.142 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.1", 2023-10-06T02:30:47.142 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBSbx9lXhMTNBAAeEdPnMP71ZGb6k2Ddg5raQ==", 2023-10-06T02:30:47.142 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:30:47.142 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:30:47.142 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:30:47.142 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:30:47.143 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:30:47.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:30:47.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:30:47.143 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.2", 2023-10-06T02:30:47.143 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBVbx9l4Ly0NhAATYXYDAjYuP53zjqK0yMboA==", 2023-10-06T02:30:47.143 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:30:47.143 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:30:47.143 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:30:47.143 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:30:47.143 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:30:47.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:30:47.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:30:47.144 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.3", 2023-10-06T02:30:47.144 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBZbx9lzsi4ARAAbGnNV0QyL/7itzy827+chg==", 2023-10-06T02:30:47.144 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:30:47.144 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:30:47.144 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:30:47.144 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:30:47.144 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:30:47.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:30:47.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:30:47.144 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.4", 2023-10-06T02:30:47.144 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBfbx9lW4OSEhAAj7HnVh/yq12zzCaXd6BHCA==", 2023-10-06T02:30:47.144 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:30:47.144 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:30:47.145 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:30:47.145 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:30:47.145 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:30:47.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:30:47.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:30:47.145 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.5", 2023-10-06T02:30:47.145 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBibx9lBcaIFRAAV8wq8QjsCOkI9Pzf/E+9bA==", 2023-10-06T02:30:47.145 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:30:47.145 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:30:47.145 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:30:47.145 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:30:47.145 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:30:47.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:30:47.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:30:47.146 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.6", 2023-10-06T02:30:47.146 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBlbx9lA7S+FxAA38mlSAurT+DpwMzH53zFcw==", 2023-10-06T02:30:47.146 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:30:47.146 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:30:47.146 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:30:47.146 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:30:47.146 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:30:47.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:30:47.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:30:47.146 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.7", 2023-10-06T02:30:47.146 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBobx9l5RkdHhAAxn0Z8IAE/o9oe2LRp3O9gg==", 2023-10-06T02:30:47.147 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:30:47.147 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:30:47.147 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:30:47.147 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:30:47.147 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:30:47.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:30:47.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:30:47.147 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.0", 2023-10-06T02:30:47.147 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBMbx9lh44fCRAA7sMISvk1t7dEJg4ZcBiMcA==", 2023-10-06T02:30:47.147 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:30:47.147 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:30:47.147 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow r", 2023-10-06T02:30:47.148 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T02:30:47.148 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:30:47.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:30:47.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:30:47.148 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.1", 2023-10-06T02:30:47.148 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBMbx9lP0zqFhAAlov2y7ipSvS3AkOdQkhg6g==", 2023-10-06T02:30:47.148 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:30:47.148 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:30:47.148 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-10-06T02:30:47.148 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-10-06T02:30:47.148 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-10-06T02:30:47.148 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:30:47.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:30:47.149 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:30:47.149 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.2", 2023-10-06T02:30:47.149 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBMbx9lpxYAEBAACtiI1XgQXM8yGGcpFyNK1w==", 2023-10-06T02:30:47.149 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:30:47.149 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:30:47.149 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-10-06T02:30:47.149 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-10-06T02:30:47.149 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-10-06T02:30:47.149 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:30:47.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:30:47.149 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:30:47.149 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.3", 2023-10-06T02:30:47.149 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBMbx9l81/PHRAAYFOoaezu0pwE0NdtnlYfVA==", 2023-10-06T02:30:47.150 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:30:47.150 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:30:47.150 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-10-06T02:30:47.150 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-10-06T02:30:47.150 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-10-06T02:30:47.150 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:30:47.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:30:47.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:30:47.150 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.admin", 2023-10-06T02:30:47.150 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBKbx9liFl1JBAAFyyHYNLVx1/Ex5a2MKm8rA==", 2023-10-06T02:30:47.150 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:30:47.150 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-10-06T02:30:47.151 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T02:30:47.151 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T02:30:47.151 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:30:47.151 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:30:47.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:30:47.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:30:47.151 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mgr.x", 2023-10-06T02:30:47.151 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBLbx9lTBbIFBAAX2wsioKL4Nx4qz/qwsCsNA==", 2023-10-06T02:30:47.151 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:30:47.151 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-10-06T02:30:47.151 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile mgr", 2023-10-06T02:30:47.151 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:30:47.152 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:30:47.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:30:47.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:30:47.152 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mgr.y", 2023-10-06T02:30:47.152 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBLbx9l4+oLDRAA5YVo3q6FI9e9W/rJYROIsg==", 2023-10-06T02:30:47.152 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:30:47.152 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-10-06T02:30:47.152 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile mgr", 2023-10-06T02:30:47.152 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:30:47.152 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:30:47.152 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:30:47.152 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-10-06T02:30:47.152 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T02:30:47.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.138+0000 7f8c5359e700 1 -- 172.21.15.31:0/4258068390 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f8c340674d0 msgr2=0x7f8c34069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:47.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.138+0000 7f8c5359e700 1 --2- 172.21.15.31:0/4258068390 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f8c340674d0 0x7f8c34069980 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f8c5415de90 tx=0x7f8c44009040 comp rx=0 tx=0).stop 2023-10-06T02:30:47.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.138+0000 7f8c5359e700 1 -- 172.21.15.31:0/4258068390 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c54142fd0 msgr2=0x7f8c5415a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:47.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.138+0000 7f8c5359e700 1 --2- 172.21.15.31:0/4258068390 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c54142fd0 0x7f8c5415a380 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c00cc60 tx=0x7f8c4c001ae0 comp rx=0 tx=0).stop 2023-10-06T02:30:47.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.139+0000 7f8c5359e700 1 -- 172.21.15.31:0/4258068390 shutdown_connections 2023-10-06T02:30:47.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.139+0000 7f8c5359e700 1 --2- 172.21.15.31:0/4258068390 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c54142fd0 0x7f8c5415a380 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:47.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.139+0000 7f8c5359e700 1 --2- 172.21.15.31:0/4258068390 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f8c340674d0 0x7f8c34069980 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:47.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.139+0000 7f8c5359e700 1 --2- 172.21.15.31:0/4258068390 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8c540c7dc0 0x7f8c54158eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:47.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.139+0000 7f8c5359e700 1 --2- 172.21.15.31:0/4258068390 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c54142c00 0x7f8c54159e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:47.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.139+0000 7f8c5359e700 1 -- 172.21.15.31:0/4258068390 >> 172.21.15.31:0/4258068390 conn(0x7f8c540675a0 msgr2=0x7f8c540c1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:47.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.139+0000 7f8c5359e700 1 -- 172.21.15.31:0/4258068390 shutdown_connections 2023-10-06T02:30:47.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.139+0000 7f8c5359e700 1 -- 172.21.15.31:0/4258068390 wait complete. 2023-10-06T02:30:47.158 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:30:47.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.343+0000 7f356e39d700 1 Processor -- start 2023-10-06T02:30:47.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.344+0000 7f356e39d700 1 -- start start 2023-10-06T02:30:47.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.345+0000 7f356e39d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3568140e40 0x7f35680c6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:47.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.345+0000 7f356e39d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35680c6d50 con 0x7f3568140a70 2023-10-06T02:30:47.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.345+0000 7f356e39d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35680c7030 con 0x7f356806de50 2023-10-06T02:30:47.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.345+0000 7f356e39d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35680c7310 con 0x7f3568140e40 2023-10-06T02:30:47.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.345+0000 7f356cb9a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3568140e40 0x7f35680c6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:47.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.345+0000 7f356cb9a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3568140e40 0x7f35680c6770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36236/0 (socket says 172.21.15.31:36236) 2023-10-06T02:30:47.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.345+0000 7f356cb9a700 1 -- 172.21.15.31:0/2406654803 learned_addr learned my addr 172.21.15.31:0/2406654803 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:47.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.346+0000 7f356cb9a700 1 -- 172.21.15.31:0/2406654803 >> v1:172.21.15.62:6789/0 conn(0x7f356806de50 legacy=0x7f35680aeef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:47.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.346+0000 7f356cb9a700 1 -- 172.21.15.31:0/2406654803 >> v1:172.21.15.31:6789/0 conn(0x7f3568140a70 legacy=0x7f35680c7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:47.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.346+0000 7f356cb9a700 1 -- 172.21.15.31:0/2406654803 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35680c75f0 con 0x7f3568140e40 2023-10-06T02:30:47.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.346+0000 7f356cb9a700 1 --2- 172.21.15.31:0/2406654803 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3568140e40 0x7f35680c6770 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f3558009990 tx=0x7f3558009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d391b1c13616cff1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:47.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.346+0000 7f355ffff700 1 -- 172.21.15.31:0/2406654803 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3558005c10 con 0x7f3568140e40 2023-10-06T02:30:47.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.346+0000 7f355ffff700 1 -- 172.21.15.31:0/2406654803 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3558005d70 con 0x7f3568140e40 2023-10-06T02:30:47.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.346+0000 7f355ffff700 1 -- 172.21.15.31:0/2406654803 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f355800b840 con 0x7f3568140e40 2023-10-06T02:30:47.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.347+0000 7f356e39d700 1 -- 172.21.15.31:0/2406654803 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3568140e40 msgr2=0x7f35680c6770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:47.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.347+0000 7f356e39d700 1 --2- 172.21.15.31:0/2406654803 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3568140e40 0x7f35680c6770 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f3558009990 tx=0x7f3558009220 comp rx=0 tx=0).stop 2023-10-06T02:30:47.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.347+0000 7f356e39d700 1 -- 172.21.15.31:0/2406654803 shutdown_connections 2023-10-06T02:30:47.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.347+0000 7f356e39d700 1 --2- 172.21.15.31:0/2406654803 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3568140e40 0x7f35680c6770 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:47.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.347+0000 7f356e39d700 1 -- 172.21.15.31:0/2406654803 >> 172.21.15.31:0/2406654803 conn(0x7f356806d170 msgr2=0x7f35680cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:47.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.347+0000 7f356e39d700 1 -- 172.21.15.31:0/2406654803 shutdown_connections 2023-10-06T02:30:47.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.347+0000 7f356e39d700 1 -- 172.21.15.31:0/2406654803 wait complete. 2023-10-06T02:30:47.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.349+0000 7f356e39d700 1 Processor -- start 2023-10-06T02:30:47.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.349+0000 7f356e39d700 1 -- start start 2023-10-06T02:30:47.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.350+0000 7f356e39d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f356806de50 0x7f356815a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:47.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.350+0000 7f356e39d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3568140a70 0x7f3568158390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:47.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.350+0000 7f356e39d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3568140e40 0x7f3568169ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:47.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.350+0000 7f356e39d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3568157f00 con 0x7f3568140a70 2023-10-06T02:30:47.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.350+0000 7f356e39d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f356816a8a0 con 0x7f3568140e40 2023-10-06T02:30:47.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.350+0000 7f356e39d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f356816ab80 con 0x7f356806de50 2023-10-06T02:30:47.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.350+0000 7f356cb9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3568140a70 0x7f3568158390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:47.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.350+0000 7f356cb9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3568140a70 0x7f3568158390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36238/0 (socket says 172.21.15.31:36238) 2023-10-06T02:30:47.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.350+0000 7f356cb9a700 1 -- 172.21.15.31:0/1119656475 learned_addr learned my addr 172.21.15.31:0/1119656475 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:47.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.350+0000 7f356db9c700 1 --2- 172.21.15.31:0/1119656475 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3568140e40 0x7f3568169ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:47.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.350+0000 7f356d39b700 1 --2- 172.21.15.31:0/1119656475 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f356806de50 0x7f356815a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:47.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.350+0000 7f356cb9a700 1 -- 172.21.15.31:0/1119656475 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f356806de50 msgr2=0x7f356815a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:47.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.350+0000 7f356cb9a700 1 --2- 172.21.15.31:0/1119656475 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f356806de50 0x7f356815a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:47.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.350+0000 7f356cb9a700 1 -- 172.21.15.31:0/1119656475 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3568140e40 msgr2=0x7f3568169ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:47.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.350+0000 7f356cb9a700 1 --2- 172.21.15.31:0/1119656475 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3568140e40 0x7f3568169ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:47.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.350+0000 7f356cb9a700 1 -- 172.21.15.31:0/1119656475 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3558009480 con 0x7f3568140a70 2023-10-06T02:30:47.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.351+0000 7f356cb9a700 1 --2- 172.21.15.31:0/1119656475 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3568140a70 0x7f3568158390 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f3558006f10 tx=0x7f3558015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:47.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.351+0000 7f355d7fa700 1 -- 172.21.15.31:0/1119656475 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3558011cc0 con 0x7f3568140a70 2023-10-06T02:30:47.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.351+0000 7f355d7fa700 1 -- 172.21.15.31:0/1119656475 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3558011e20 con 0x7f3568140a70 2023-10-06T02:30:47.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.351+0000 7f355d7fa700 1 -- 172.21.15.31:0/1119656475 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f355801c670 con 0x7f3568140a70 2023-10-06T02:30:47.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.351+0000 7f356e39d700 1 -- 172.21.15.31:0/1119656475 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f356816ae60 con 0x7f3568140a70 2023-10-06T02:30:47.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.351+0000 7f356e39d700 1 -- 172.21.15.31:0/1119656475 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f356815a9c0 con 0x7f3568140a70 2023-10-06T02:30:47.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.352+0000 7f356e39d700 1 -- 172.21.15.31:0/1119656475 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f356815abb0 con 0x7f3568140a70 2023-10-06T02:30:47.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.353+0000 7f355d7fa700 1 -- 172.21.15.31:0/1119656475 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f355801caa0 con 0x7f3568140a70 2023-10-06T02:30:47.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.354+0000 7f355d7fa700 1 --2- 172.21.15.31:0/1119656475 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f354c0674d0 0x7f354c069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:47.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.354+0000 7f355d7fa700 1 -- 172.21.15.31:0/1119656475 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5234+0+0 (secure 0 0 0) 0x7f3558086650 con 0x7f3568140a70 2023-10-06T02:30:47.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.354+0000 7f356d39b700 1 --2- 172.21.15.31:0/1119656475 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f354c0674d0 0x7f354c069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:47.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.355+0000 7f356d39b700 1 --2- 172.21.15.31:0/1119656475 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f354c0674d0 0x7f354c069980 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f3564003860 tx=0x7f3564009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:47.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.358+0000 7f355d7fa700 1 -- 172.21.15.31:0/1119656475 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f35580531e0 con 0x7f3568140a70 2023-10-06T02:30:47.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.560+0000 7f356e39d700 1 -- 172.21.15.31:0/1119656475 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f356804c3c0 con 0x7f3568140a70 2023-10-06T02:30:47.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.561+0000 7f355d7fa700 1 -- 172.21.15.31:0/1119656475 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v140) v1 ==== 74+0+13656 (secure 0 0 0) 0x7f3558056800 con 0x7f3568140a70 2023-10-06T02:30:47.562 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:30:47.562 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":140,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:30:46.636942+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":20,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:30:47.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.563+0000 7f356e39d700 1 -- 172.21.15.31:0/1119656475 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f354c0674d0 msgr2=0x7f354c069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:47.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.563+0000 7f356e39d700 1 --2- 172.21.15.31:0/1119656475 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f354c0674d0 0x7f354c069980 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f3564003860 tx=0x7f3564009040 comp rx=0 tx=0).stop 2023-10-06T02:30:47.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.563+0000 7f356e39d700 1 -- 172.21.15.31:0/1119656475 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3568140a70 msgr2=0x7f3568158390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:47.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.563+0000 7f356e39d700 1 --2- 172.21.15.31:0/1119656475 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3568140a70 0x7f3568158390 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f3558006f10 tx=0x7f3558015000 comp rx=0 tx=0).stop 2023-10-06T02:30:47.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.563+0000 7f356e39d700 1 -- 172.21.15.31:0/1119656475 shutdown_connections 2023-10-06T02:30:47.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.563+0000 7f356e39d700 1 --2- 172.21.15.31:0/1119656475 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3568140e40 0x7f3568169ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:47.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.563+0000 7f356e39d700 1 --2- 172.21.15.31:0/1119656475 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f354c0674d0 0x7f354c069980 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:47.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.563+0000 7f356e39d700 1 --2- 172.21.15.31:0/1119656475 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f356806de50 0x7f356815a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:47.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.563+0000 7f356e39d700 1 --2- 172.21.15.31:0/1119656475 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3568140a70 0x7f3568158390 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:47.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.563+0000 7f356e39d700 1 -- 172.21.15.31:0/1119656475 >> 172.21.15.31:0/1119656475 conn(0x7f356806d170 msgr2=0x7f35680ca590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:47.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.563+0000 7f356e39d700 1 -- 172.21.15.31:0/1119656475 shutdown_connections 2023-10-06T02:30:47.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.563+0000 7f356e39d700 1 -- 172.21.15.31:0/1119656475 wait complete. 2023-10-06T02:30:47.582 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T02:22:53.183425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T02:23:00.046967+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-10-06T02:30:47.583 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T02:30:47.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.774+0000 7fa97ed9d700 1 Processor -- start 2023-10-06T02:30:47.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.774+0000 7fa97ed9d700 1 -- start start 2023-10-06T02:30:47.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.775+0000 7fa97ed9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa9780c7be0 0x7fa97806abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:47.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.775+0000 7fa97ed9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa97813f780 0x7fa97813fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:47.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.775+0000 7fa97ed9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa97806b290 con 0x7fa9780c7fb0 2023-10-06T02:30:47.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.775+0000 7fa97ed9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa97806bf10 con 0x7fa97813f780 2023-10-06T02:30:47.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.775+0000 7fa97ed9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa978140140 con 0x7fa9780c7be0 2023-10-06T02:30:47.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.776+0000 7fa97dd9b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa9780c7be0 0x7fa97806abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:47.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.776+0000 7fa97e59c700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa97813f780 0x7fa97813fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:47.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.776+0000 7fa97dd9b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa9780c7be0 0x7fa97806abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:37026/0 (socket says 172.21.15.31:37026) 2023-10-06T02:30:47.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.776+0000 7fa97dd9b700 1 -- 172.21.15.31:0/2329579264 learned_addr learned my addr 172.21.15.31:0/2329579264 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:47.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.776+0000 7fa97dd9b700 1 -- 172.21.15.31:0/2329579264 >> v2:172.21.15.62:3300/0 conn(0x7fa97813f780 msgr2=0x7fa97813fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:47.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.777+0000 7fa97dd9b700 1 --2- 172.21.15.31:0/2329579264 >> v2:172.21.15.62:3300/0 conn(0x7fa97813f780 0x7fa97813fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:47.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.777+0000 7fa97dd9b700 1 -- 172.21.15.31:0/2329579264 >> v1:172.21.15.62:6789/0 conn(0x7fa9780c7fb0 legacy=0x7fa97815a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:47.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.777+0000 7fa97dd9b700 1 -- 172.21.15.31:0/2329579264 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa97806b570 con 0x7fa9780c7be0 2023-10-06T02:30:47.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.777+0000 7fa97e59c700 1 --2- 172.21.15.31:0/2329579264 >> v2:172.21.15.62:3300/0 conn(0x7fa97813f780 0x7fa97813fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:30:47.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.777+0000 7fa97dd9b700 1 --2- 172.21.15.31:0/2329579264 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa9780c7be0 0x7fa97806abd0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa9740099b0 tx=0x7fa974009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=514aecf7d6ecc1ac server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:47.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.777+0000 7fa97cd99700 1 -- 172.21.15.31:0/2329579264 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa974009ea0 con 0x7fa9780c7be0 2023-10-06T02:30:47.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.777+0000 7fa97cd99700 1 -- 172.21.15.31:0/2329579264 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa97400b860 con 0x7fa9780c7be0 2023-10-06T02:30:47.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.777+0000 7fa97cd99700 1 -- 172.21.15.31:0/2329579264 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa97400bc00 con 0x7fa9780c7be0 2023-10-06T02:30:47.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.778+0000 7fa97ed9d700 1 -- 172.21.15.31:0/2329579264 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa9780c7be0 msgr2=0x7fa97806abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:47.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.778+0000 7fa97ed9d700 1 --2- 172.21.15.31:0/2329579264 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa9780c7be0 0x7fa97806abd0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa9740099b0 tx=0x7fa974009250 comp rx=0 tx=0).stop 2023-10-06T02:30:47.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.778+0000 7fa97ed9d700 1 -- 172.21.15.31:0/2329579264 shutdown_connections 2023-10-06T02:30:47.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.778+0000 7fa97ed9d700 1 --2- 172.21.15.31:0/2329579264 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa9780c7be0 0x7fa97806abd0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:47.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.778+0000 7fa97ed9d700 1 --2- 172.21.15.31:0/2329579264 >> v2:172.21.15.62:3300/0 conn(0x7fa97813f780 0x7fa97813fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:47.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.778+0000 7fa97ed9d700 1 -- 172.21.15.31:0/2329579264 >> 172.21.15.31:0/2329579264 conn(0x7fa97806d110 msgr2=0x7fa9780cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:47.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.778+0000 7fa97ed9d700 1 -- 172.21.15.31:0/2329579264 shutdown_connections 2023-10-06T02:30:47.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.778+0000 7fa97ed9d700 1 -- 172.21.15.31:0/2329579264 wait complete. 2023-10-06T02:30:47.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.779+0000 7fa97ed9d700 1 Processor -- start 2023-10-06T02:30:47.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.779+0000 7fa97ed9d700 1 -- start start 2023-10-06T02:30:47.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.780+0000 7fa97ed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9780c7be0 0x7fa9781677c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:47.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.780+0000 7fa97ed9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa9780c7fb0 0x7fa978156930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:47.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.780+0000 7fa97ed9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa97813f780 0x7fa978130970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:47.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.780+0000 7fa97ed9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa978167d00 con 0x7fa9780c7be0 2023-10-06T02:30:47.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.780+0000 7fa97ed9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9780c4b60 con 0x7fa97813f780 2023-10-06T02:30:47.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.780+0000 7fa97ed9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa978157760 con 0x7fa9780c7fb0 2023-10-06T02:30:47.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.781+0000 7fa97dd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9780c7be0 0x7fa9781677c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:47.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.781+0000 7fa97dd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9780c7be0 0x7fa9781677c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36250/0 (socket says 172.21.15.31:36250) 2023-10-06T02:30:47.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.781+0000 7fa97dd9b700 1 -- 172.21.15.31:0/663026545 learned_addr learned my addr 172.21.15.31:0/663026545 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:47.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.781+0000 7fa97e59c700 1 --2- 172.21.15.31:0/663026545 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa97813f780 0x7fa978130970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:47.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.781+0000 7fa97d59a700 1 --2- 172.21.15.31:0/663026545 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa9780c7fb0 0x7fa978156930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:47.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.781+0000 7fa97dd9b700 1 -- 172.21.15.31:0/663026545 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa9780c7fb0 msgr2=0x7fa978156930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:47.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.781+0000 7fa97dd9b700 1 --2- 172.21.15.31:0/663026545 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa9780c7fb0 0x7fa978156930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:47.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.781+0000 7fa97dd9b700 1 -- 172.21.15.31:0/663026545 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa97813f780 msgr2=0x7fa978130970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:47.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.781+0000 7fa97dd9b700 1 --2- 172.21.15.31:0/663026545 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa97813f780 0x7fa978130970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:47.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.781+0000 7fa97dd9b700 1 -- 172.21.15.31:0/663026545 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa970009a20 con 0x7fa9780c7be0 2023-10-06T02:30:47.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.781+0000 7fa97d59a700 1 --2- 172.21.15.31:0/663026545 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa9780c7fb0 0x7fa978156930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:30:47.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.781+0000 7fa97dd9b700 1 --2- 172.21.15.31:0/663026545 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9780c7be0 0x7fa9781677c0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fa974006fd0 tx=0x7fa974017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:47.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.781+0000 7fa96dffb700 1 -- 172.21.15.31:0/663026545 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa97400bbe0 con 0x7fa9780c7be0 2023-10-06T02:30:47.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.782+0000 7fa96dffb700 1 -- 172.21.15.31:0/663026545 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa974011420 con 0x7fa9780c7be0 2023-10-06T02:30:47.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.782+0000 7fa96dffb700 1 -- 172.21.15.31:0/663026545 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9740117e0 con 0x7fa9780c7be0 2023-10-06T02:30:47.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.782+0000 7fa97ed9d700 1 -- 172.21.15.31:0/663026545 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9740094f0 con 0x7fa9780c7be0 2023-10-06T02:30:47.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.782+0000 7fa97ed9d700 1 -- 172.21.15.31:0/663026545 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa978157d20 con 0x7fa9780c7be0 2023-10-06T02:30:47.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.783+0000 7fa97ed9d700 1 -- 172.21.15.31:0/663026545 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa97804c3c0 con 0x7fa9780c7be0 2023-10-06T02:30:47.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.784+0000 7fa96dffb700 1 -- 172.21.15.31:0/663026545 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7fa974011c10 con 0x7fa9780c7be0 2023-10-06T02:30:47.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.784+0000 7fa96dffb700 1 --2- 172.21.15.31:0/663026545 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa95c0674d0 0x7fa95c069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:47.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.784+0000 7fa96dffb700 1 -- 172.21.15.31:0/663026545 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5234+0+0 (secure 0 0 0) 0x7fa9740867a0 con 0x7fa9780c7be0 2023-10-06T02:30:47.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.785+0000 7fa97d59a700 1 --2- 172.21.15.31:0/663026545 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa95c0674d0 0x7fa95c069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:47.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.786+0000 7fa97d59a700 1 --2- 172.21.15.31:0/663026545 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa95c0674d0 0x7fa95c069980 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa968007fd0 tx=0x7fa968009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:47.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.789+0000 7fa96dffb700 1 -- 172.21.15.31:0/663026545 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa9740533f0 con 0x7fa9780c7be0 2023-10-06T02:30:47.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.993+0000 7fa97ed9d700 1 -- 172.21.15.31:0/663026545 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa9780b1990 con 0x7fa9780c7be0 2023-10-06T02:30:47.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.994+0000 7fa96dffb700 1 -- 172.21.15.31:0/663026545 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v140) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa9740150a0 con 0x7fa9780c7be0 2023-10-06T02:30:47.996 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-10-06T02:30:47.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.996+0000 7fa97ed9d700 1 -- 172.21.15.31:0/663026545 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa95c0674d0 msgr2=0x7fa95c069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:47.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.996+0000 7fa97ed9d700 1 --2- 172.21.15.31:0/663026545 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa95c0674d0 0x7fa95c069980 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa968007fd0 tx=0x7fa968009040 comp rx=0 tx=0).stop 2023-10-06T02:30:47.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.996+0000 7fa97ed9d700 1 -- 172.21.15.31:0/663026545 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9780c7be0 msgr2=0x7fa9781677c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:47.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.996+0000 7fa97ed9d700 1 --2- 172.21.15.31:0/663026545 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9780c7be0 0x7fa9781677c0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fa974006fd0 tx=0x7fa974017000 comp rx=0 tx=0).stop 2023-10-06T02:30:47.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.996+0000 7fa97ed9d700 1 -- 172.21.15.31:0/663026545 shutdown_connections 2023-10-06T02:30:47.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.996+0000 7fa97ed9d700 1 --2- 172.21.15.31:0/663026545 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa97813f780 0x7fa978130970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:47.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.996+0000 7fa97ed9d700 1 --2- 172.21.15.31:0/663026545 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9780c7be0 0x7fa9781677c0 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:47.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.996+0000 7fa97ed9d700 1 --2- 172.21.15.31:0/663026545 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa95c0674d0 0x7fa95c069980 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:47.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.996+0000 7fa97ed9d700 1 --2- 172.21.15.31:0/663026545 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa9780c7fb0 0x7fa978156930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:47.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.996+0000 7fa97ed9d700 1 -- 172.21.15.31:0/663026545 >> 172.21.15.31:0/663026545 conn(0x7fa97806d110 msgr2=0x7fa9780cb060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:48.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.997+0000 7fa97ed9d700 1 -- 172.21.15.31:0/663026545 shutdown_connections 2023-10-06T02:30:48.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:47.997+0000 7fa97ed9d700 1 -- 172.21.15.31:0/663026545 wait complete. 2023-10-06T02:30:48.015 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T02:30:48.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.205+0000 7f7f19f9d700 1 Processor -- start 2023-10-06T02:30:48.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.205+0000 7f7f19f9d700 1 -- start start 2023-10-06T02:30:48.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.206+0000 7f7f19f9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7f14153a00 0x7f7f14153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:48.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.206+0000 7f7f19f9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7f140cea00 0x7f7f140cee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:48.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.206+0000 7f7f19f9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f1415ac50 con 0x7f7f1415de50 2023-10-06T02:30:48.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.206+0000 7f7f19f9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f1415d010 con 0x7f7f14153a00 2023-10-06T02:30:48.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.206+0000 7f7f19f9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f140cf3c0 con 0x7f7f140cea00 2023-10-06T02:30:48.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.207+0000 7f7f18f9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7f14153a00 0x7f7f14153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:48.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.207+0000 7f7f13fff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f7f1415de50 0x7f7f1415a540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:47874/0 (socket says 172.21.15.31:47874) 2023-10-06T02:30:48.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.207+0000 7f7f18f9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7f14153a00 0x7f7f14153dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54624/0 (socket says 172.21.15.31:54624) 2023-10-06T02:30:48.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.207+0000 7f7f13fff700 1 -- 172.21.15.31:0/364727131 learned_addr learned my addr 172.21.15.31:0/364727131 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:30:48.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.207+0000 7f7f1979c700 1 --2- 172.21.15.31:0/364727131 >> v2:172.21.15.62:3300/0 conn(0x7f7f140cea00 0x7f7f140cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:48.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.207+0000 7f7f18f9b700 1 -- 172.21.15.31:0/364727131 >> v2:172.21.15.62:3300/0 conn(0x7f7f140cea00 msgr2=0x7f7f140cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:48.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.207+0000 7f7f18f9b700 1 --2- 172.21.15.31:0/364727131 >> v2:172.21.15.62:3300/0 conn(0x7f7f140cea00 0x7f7f140cee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:48.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.207+0000 7f7f18f9b700 1 -- 172.21.15.31:0/364727131 >> v1:172.21.15.31:6789/0 conn(0x7f7f1415de50 legacy=0x7f7f1415a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:48.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.207+0000 7f7f18f9b700 1 -- 172.21.15.31:0/364727131 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f1415af60 con 0x7f7f14153a00 2023-10-06T02:30:48.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.207+0000 7f7f1979c700 1 --2- 172.21.15.31:0/364727131 >> v2:172.21.15.62:3300/0 conn(0x7f7f140cea00 0x7f7f140cee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:30:48.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.208+0000 7f7f18f9b700 1 --2- 172.21.15.31:0/364727131 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7f14153a00 0x7f7f14153dd0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f7f140d03e0 tx=0x7f7f080091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c99048fc19116c16 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:48.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.208+0000 7f7f137fe700 1 -- 172.21.15.31:0/364727131 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f0800b6b0 con 0x7f7f14153a00 2023-10-06T02:30:48.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.208+0000 7f7f137fe700 1 -- 172.21.15.31:0/364727131 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f0800b810 con 0x7f7f14153a00 2023-10-06T02:30:48.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.208+0000 7f7f137fe700 1 -- 172.21.15.31:0/364727131 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f0800bbb0 con 0x7f7f14153a00 2023-10-06T02:30:48.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.208+0000 7f7f19f9d700 1 -- 172.21.15.31:0/364727131 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7f14153a00 msgr2=0x7f7f14153dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:48.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.208+0000 7f7f19f9d700 1 --2- 172.21.15.31:0/364727131 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7f14153a00 0x7f7f14153dd0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f7f140d03e0 tx=0x7f7f080091a0 comp rx=0 tx=0).stop 2023-10-06T02:30:48.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.208+0000 7f7f19f9d700 1 -- 172.21.15.31:0/364727131 shutdown_connections 2023-10-06T02:30:48.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.208+0000 7f7f19f9d700 1 --2- 172.21.15.31:0/364727131 >> v2:172.21.15.62:3300/0 conn(0x7f7f140cea00 0x7f7f140cee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:48.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.208+0000 7f7f19f9d700 1 --2- 172.21.15.31:0/364727131 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7f14153a00 0x7f7f14153dd0 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:48.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.208+0000 7f7f19f9d700 1 -- 172.21.15.31:0/364727131 >> 172.21.15.31:0/364727131 conn(0x7f7f140703a0 msgr2=0x7f7f140707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:48.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.209+0000 7f7f19f9d700 1 -- 172.21.15.31:0/364727131 shutdown_connections 2023-10-06T02:30:48.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.209+0000 7f7f19f9d700 1 -- 172.21.15.31:0/364727131 wait complete. 2023-10-06T02:30:48.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.210+0000 7f7f19f9d700 1 Processor -- start 2023-10-06T02:30:48.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.210+0000 7f7f19f9d700 1 -- start start 2023-10-06T02:30:48.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.211+0000 7f7f19f9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7f140cea00 0x7f7f141594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:48.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.211+0000 7f7f19f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7f14153a00 0x7f7f140cc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:48.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.211+0000 7f7f19f9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f1415de50 0x7f7f140c6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:48.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.211+0000 7f7f19f9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f14159030 con 0x7f7f140cea00 2023-10-06T02:30:48.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.211+0000 7f7f19f9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f14158700 con 0x7f7f1415de50 2023-10-06T02:30:48.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.211+0000 7f7f19f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f141589e0 con 0x7f7f14153a00 2023-10-06T02:30:48.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.211+0000 7f7f18f9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7f140cea00 0x7f7f141594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:48.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.211+0000 7f7f18f9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7f140cea00 0x7f7f141594c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54628/0 (socket says 172.21.15.31:54628) 2023-10-06T02:30:48.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.211+0000 7f7f18f9b700 1 -- 172.21.15.31:0/3338260069 learned_addr learned my addr 172.21.15.31:0/3338260069 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:48.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.212+0000 7f7f18f9b700 1 -- 172.21.15.31:0/3338260069 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7f14153a00 msgr2=0x7f7f140cc4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:48.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.212+0000 7f7f18f9b700 1 --2- 172.21.15.31:0/3338260069 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7f14153a00 0x7f7f140cc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:48.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.212+0000 7f7f18f9b700 1 -- 172.21.15.31:0/3338260069 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f1415de50 msgr2=0x7f7f140c6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:48.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.212+0000 7f7f18f9b700 1 --2- 172.21.15.31:0/3338260069 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f1415de50 0x7f7f140c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:48.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.212+0000 7f7f18f9b700 1 -- 172.21.15.31:0/3338260069 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f08009420 con 0x7f7f140cea00 2023-10-06T02:30:48.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.212+0000 7f7f18f9b700 1 --2- 172.21.15.31:0/3338260069 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7f140cea00 0x7f7f141594c0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f7f1406eb00 tx=0x7f7f08017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:48.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.212+0000 7f7f10ff9700 1 -- 172.21.15.31:0/3338260069 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f0800bb90 con 0x7f7f140cea00 2023-10-06T02:30:48.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.212+0000 7f7f10ff9700 1 -- 172.21.15.31:0/3338260069 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f08011420 con 0x7f7f140cea00 2023-10-06T02:30:48.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.212+0000 7f7f10ff9700 1 -- 172.21.15.31:0/3338260069 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f080117a0 con 0x7f7f140cea00 2023-10-06T02:30:48.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.212+0000 7f7f19f9d700 1 -- 172.21.15.31:0/3338260069 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f14158cc0 con 0x7f7f140cea00 2023-10-06T02:30:48.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.212+0000 7f7f19f9d700 1 -- 172.21.15.31:0/3338260069 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f140cca20 con 0x7f7f140cea00 2023-10-06T02:30:48.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.213+0000 7f7f19f9d700 1 -- 172.21.15.31:0/3338260069 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f1404c3c0 con 0x7f7f140cea00 2023-10-06T02:30:48.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.214+0000 7f7f10ff9700 1 -- 172.21.15.31:0/3338260069 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f7f08011b90 con 0x7f7f140cea00 2023-10-06T02:30:48.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.215+0000 7f7f10ff9700 1 --2- 172.21.15.31:0/3338260069 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7ef80674d0 0x7f7ef8069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:48.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.216+0000 7f7f10ff9700 1 -- 172.21.15.31:0/3338260069 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5234+0+0 (secure 0 0 0) 0x7f7f08086240 con 0x7f7f140cea00 2023-10-06T02:30:48.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.216+0000 7f7f13fff700 1 --2- 172.21.15.31:0/3338260069 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7ef80674d0 0x7f7ef8069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:48.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.217+0000 7f7f13fff700 1 --2- 172.21.15.31:0/3338260069 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7ef80674d0 0x7f7ef8069980 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f7f140c5620 tx=0x7f7f00007b30 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:48.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.220+0000 7f7f10ff9700 1 -- 172.21.15.31:0/3338260069 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7f08052e50 con 0x7f7f140cea00 2023-10-06T02:30:48.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.423+0000 7f7f19f9d700 1 -- 172.21.15.31:0/3338260069 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f7f140cc070 con 0x7f7f140cea00 2023-10-06T02:30:48.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.424+0000 7f7f10ff9700 1 -- 172.21.15.31:0/3338260069 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v140) v1 ==== 92+0+10 (secure 0 0 0) 0x7f7f08056470 con 0x7f7f140cea00 2023-10-06T02:30:48.426 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 8 2023-10-06T02:30:48.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.426+0000 7f7f19f9d700 1 -- 172.21.15.31:0/3338260069 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7ef80674d0 msgr2=0x7f7ef8069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:48.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.426+0000 7f7f19f9d700 1 --2- 172.21.15.31:0/3338260069 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7ef80674d0 0x7f7ef8069980 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f7f140c5620 tx=0x7f7f00007b30 comp rx=0 tx=0).stop 2023-10-06T02:30:48.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.426+0000 7f7f19f9d700 1 -- 172.21.15.31:0/3338260069 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7f140cea00 msgr2=0x7f7f141594c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:48.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.426+0000 7f7f19f9d700 1 --2- 172.21.15.31:0/3338260069 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7f140cea00 0x7f7f141594c0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f7f1406eb00 tx=0x7f7f08017000 comp rx=0 tx=0).stop 2023-10-06T02:30:48.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.426+0000 7f7f19f9d700 1 -- 172.21.15.31:0/3338260069 shutdown_connections 2023-10-06T02:30:48.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.426+0000 7f7f19f9d700 1 --2- 172.21.15.31:0/3338260069 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f1415de50 0x7f7f140c6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:48.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.426+0000 7f7f19f9d700 1 --2- 172.21.15.31:0/3338260069 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7f140cea00 0x7f7f141594c0 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:48.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.426+0000 7f7f19f9d700 1 --2- 172.21.15.31:0/3338260069 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7ef80674d0 0x7f7ef8069980 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:48.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.426+0000 7f7f19f9d700 1 --2- 172.21.15.31:0/3338260069 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7f14153a00 0x7f7f140cc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:48.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.426+0000 7f7f19f9d700 1 -- 172.21.15.31:0/3338260069 >> 172.21.15.31:0/3338260069 conn(0x7f7f140703a0 msgr2=0x7f7f14152ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:48.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.427+0000 7f7f19f9d700 1 -- 172.21.15.31:0/3338260069 shutdown_connections 2023-10-06T02:30:48.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.427+0000 7f7f19f9d700 1 -- 172.21.15.31:0/3338260069 wait complete. 2023-10-06T02:30:48.446 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T02:30:48.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.631+0000 7f0ab939d700 1 Processor -- start 2023-10-06T02:30:48.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.631+0000 7f0ab939d700 1 -- start start 2023-10-06T02:30:48.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.632+0000 7f0ab939d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0ab415de50 0x7f0ab40cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:48.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.633+0000 7f0ab939d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ab40cf3c0 con 0x7f0ab4153c30 2023-10-06T02:30:48.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.633+0000 7f0ab939d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ab415afd0 con 0x7f0ab40cef40 2023-10-06T02:30:48.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.633+0000 7f0ab939d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ab415b2b0 con 0x7f0ab415de50 2023-10-06T02:30:48.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.633+0000 7f0ab3fff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f0ab4153c30 0x7f0ab415a5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:47878/0 (socket says 172.21.15.31:47878) 2023-10-06T02:30:48.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.633+0000 7f0ab3fff700 1 -- 172.21.15.31:0/3950988852 learned_addr learned my addr 172.21.15.31:0/3950988852 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:30:48.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.633+0000 7f0ab37fe700 1 --2- 172.21.15.31:0/3950988852 >> v2:172.21.15.31:3300/0 conn(0x7f0ab415de50 0x7f0ab40cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:48.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.633+0000 7f0ab37fe700 1 -- 172.21.15.31:0/3950988852 >> v1:172.21.15.62:6789/0 conn(0x7f0ab40cef40 legacy=0x7f0ab415c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:48.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.633+0000 7f0ab37fe700 1 -- 172.21.15.31:0/3950988852 >> v1:172.21.15.31:6789/0 conn(0x7f0ab4153c30 legacy=0x7f0ab415a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:48.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.633+0000 7f0ab37fe700 1 -- 172.21.15.31:0/3950988852 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab415b590 con 0x7f0ab415de50 2023-10-06T02:30:48.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.634+0000 7f0ab37fe700 1 --2- 172.21.15.31:0/3950988852 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ab415de50 0x7f0ab40cea00 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f0aa80098e0 tx=0x7f0aa80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be91776c062bdb88 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:48.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.634+0000 7f0ab2ffd700 1 -- 172.21.15.31:0/3950988852 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa8009e80 con 0x7f0ab415de50 2023-10-06T02:30:48.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.634+0000 7f0ab2ffd700 1 -- 172.21.15.31:0/3950988852 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aa800b860 con 0x7f0ab415de50 2023-10-06T02:30:48.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.634+0000 7f0ab2ffd700 1 -- 172.21.15.31:0/3950988852 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa800bbe0 con 0x7f0ab415de50 2023-10-06T02:30:48.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.635+0000 7f0ab939d700 1 -- 172.21.15.31:0/3950988852 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ab415de50 msgr2=0x7f0ab40cea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:48.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.635+0000 7f0ab939d700 1 --2- 172.21.15.31:0/3950988852 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ab415de50 0x7f0ab40cea00 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f0aa80098e0 tx=0x7f0aa80091a0 comp rx=0 tx=0).stop 2023-10-06T02:30:48.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.635+0000 7f0ab939d700 1 -- 172.21.15.31:0/3950988852 shutdown_connections 2023-10-06T02:30:48.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.635+0000 7f0ab939d700 1 --2- 172.21.15.31:0/3950988852 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ab415de50 0x7f0ab40cea00 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:48.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.635+0000 7f0ab939d700 1 -- 172.21.15.31:0/3950988852 >> 172.21.15.31:0/3950988852 conn(0x7f0ab40703a0 msgr2=0x7f0ab40707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:48.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.635+0000 7f0ab939d700 1 -- 172.21.15.31:0/3950988852 shutdown_connections 2023-10-06T02:30:48.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.635+0000 7f0ab939d700 1 -- 172.21.15.31:0/3950988852 wait complete. 2023-10-06T02:30:48.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.637+0000 7f0ab939d700 1 Processor -- start 2023-10-06T02:30:48.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.637+0000 7f0ab939d700 1 -- start start 2023-10-06T02:30:48.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.638+0000 7f0ab939d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0ab40cef40 0x7f0ab40cb110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:48.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.638+0000 7f0ab939d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ab4153c30 0x7f0ab413eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:48.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.638+0000 7f0ab939d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ab415de50 0x7f0ab40b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:48.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.638+0000 7f0ab939d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab40cc300 con 0x7f0ab415de50 2023-10-06T02:30:48.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.638+0000 7f0ab939d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab413f020 con 0x7f0ab4153c30 2023-10-06T02:30:48.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.638+0000 7f0ab939d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab40cb6f0 con 0x7f0ab40cef40 2023-10-06T02:30:48.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.639+0000 7f0ab8b9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ab415de50 0x7f0ab40b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:48.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.639+0000 7f0ab3fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0ab40cef40 0x7f0ab40cb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:48.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.639+0000 7f0ab8b9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ab415de50 0x7f0ab40b5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54652/0 (socket says 172.21.15.31:54652) 2023-10-06T02:30:48.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.639+0000 7f0ab8b9c700 1 -- 172.21.15.31:0/890688809 learned_addr learned my addr 172.21.15.31:0/890688809 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:48.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.639+0000 7f0ab37fe700 1 --2- 172.21.15.31:0/890688809 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ab4153c30 0x7f0ab413eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:48.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.639+0000 7f0ab8b9c700 1 -- 172.21.15.31:0/890688809 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0ab40cef40 msgr2=0x7f0ab40cb110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:48.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.639+0000 7f0ab8b9c700 1 --2- 172.21.15.31:0/890688809 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0ab40cef40 0x7f0ab40cb110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:48.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.639+0000 7f0ab8b9c700 1 -- 172.21.15.31:0/890688809 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ab4153c30 msgr2=0x7f0ab413eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:48.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.639+0000 7f0ab8b9c700 1 --2- 172.21.15.31:0/890688809 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ab4153c30 0x7f0ab413eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:48.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.639+0000 7f0ab8b9c700 1 -- 172.21.15.31:0/890688809 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aa8009420 con 0x7f0ab415de50 2023-10-06T02:30:48.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.640+0000 7f0ab8b9c700 1 --2- 172.21.15.31:0/890688809 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ab415de50 0x7f0ab40b5ab0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f0aac00c840 tx=0x7f0aac001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:48.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.640+0000 7f0a9bfff700 1 -- 172.21.15.31:0/890688809 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aac012630 con 0x7f0ab415de50 2023-10-06T02:30:48.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.640+0000 7f0a9bfff700 1 -- 172.21.15.31:0/890688809 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aac0127b0 con 0x7f0ab415de50 2023-10-06T02:30:48.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.640+0000 7f0a9bfff700 1 -- 172.21.15.31:0/890688809 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aac012b70 con 0x7f0ab415de50 2023-10-06T02:30:48.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.640+0000 7f0ab939d700 1 -- 172.21.15.31:0/890688809 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ab40cb9d0 con 0x7f0ab415de50 2023-10-06T02:30:48.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.640+0000 7f0ab939d700 1 -- 172.21.15.31:0/890688809 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ab40cbf90 con 0x7f0ab415de50 2023-10-06T02:30:48.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.641+0000 7f0ab939d700 1 -- 172.21.15.31:0/890688809 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ab404c3c0 con 0x7f0ab415de50 2023-10-06T02:30:48.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.642+0000 7f0a9bfff700 1 -- 172.21.15.31:0/890688809 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f0aac01a3a0 con 0x7f0ab415de50 2023-10-06T02:30:48.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.643+0000 7f0a9bfff700 1 --2- 172.21.15.31:0/890688809 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0aa00674d0 0x7f0aa0069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:48.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.643+0000 7f0a9bfff700 1 -- 172.21.15.31:0/890688809 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5234+0+0 (secure 0 0 0) 0x7f0aac084f40 con 0x7f0ab415de50 2023-10-06T02:30:48.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.643+0000 7f0ab3fff700 1 --2- 172.21.15.31:0/890688809 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0aa00674d0 0x7f0aa0069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:48.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.644+0000 7f0ab3fff700 1 --2- 172.21.15.31:0/890688809 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0aa00674d0 0x7f0aa0069980 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0aa4007130 tx=0x7f0aa4007eb0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:48.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.647+0000 7f0a9bfff700 1 -- 172.21.15.31:0/890688809 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0aac051ad0 con 0x7f0ab415de50 2023-10-06T02:30:48.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.850+0000 7f0ab939d700 1 -- 172.21.15.31:0/890688809 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f0ab40cc440 con 0x7f0ab415de50 2023-10-06T02:30:48.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.851+0000 7f0a9bfff700 1 -- 172.21.15.31:0/890688809 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v140) v1 ==== 84+0+126 (secure 0 0 0) 0x7f0aac0550f0 con 0x7f0ab415de50 2023-10-06T02:30:48.852 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:30:48.853 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2023-10-06T02:30:48.853 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:30:48.853 INFO:teuthology.orchestra.run.smithi031.stdout: "poolnum": 1, 2023-10-06T02:30:48.853 INFO:teuthology.orchestra.run.smithi031.stdout: "poolname": ".mgr" 2023-10-06T02:30:48.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:30:48.853 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:30:48.853 INFO:teuthology.orchestra.run.smithi031.stdout: "poolnum": 2, 2023-10-06T02:30:48.853 INFO:teuthology.orchestra.run.smithi031.stdout: "poolname": "rbd" 2023-10-06T02:30:48.853 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:30:48.853 INFO:teuthology.orchestra.run.smithi031.stdout:] 2023-10-06T02:30:48.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.853+0000 7f0ab939d700 1 -- 172.21.15.31:0/890688809 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0aa00674d0 msgr2=0x7f0aa0069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:48.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.853+0000 7f0ab939d700 1 --2- 172.21.15.31:0/890688809 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0aa00674d0 0x7f0aa0069980 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0aa4007130 tx=0x7f0aa4007eb0 comp rx=0 tx=0).stop 2023-10-06T02:30:48.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.853+0000 7f0ab939d700 1 -- 172.21.15.31:0/890688809 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ab415de50 msgr2=0x7f0ab40b5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:48.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.853+0000 7f0ab939d700 1 --2- 172.21.15.31:0/890688809 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ab415de50 0x7f0ab40b5ab0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f0aac00c840 tx=0x7f0aac001670 comp rx=0 tx=0).stop 2023-10-06T02:30:48.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.853+0000 7f0ab939d700 1 -- 172.21.15.31:0/890688809 shutdown_connections 2023-10-06T02:30:48.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.853+0000 7f0ab939d700 1 --2- 172.21.15.31:0/890688809 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ab415de50 0x7f0ab40b5ab0 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:48.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.853+0000 7f0ab939d700 1 --2- 172.21.15.31:0/890688809 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0aa00674d0 0x7f0aa0069980 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:48.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.853+0000 7f0ab939d700 1 --2- 172.21.15.31:0/890688809 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0ab40cef40 0x7f0ab40cb110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:48.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.853+0000 7f0ab939d700 1 --2- 172.21.15.31:0/890688809 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ab4153c30 0x7f0ab413eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:48.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.853+0000 7f0ab939d700 1 -- 172.21.15.31:0/890688809 >> 172.21.15.31:0/890688809 conn(0x7f0ab40703a0 msgr2=0x7f0ab40d0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:48.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.853+0000 7f0ab939d700 1 -- 172.21.15.31:0/890688809 shutdown_connections 2023-10-06T02:30:48.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:48.853+0000 7f0ab939d700 1 -- 172.21.15.31:0/890688809 wait complete. 2023-10-06T02:30:48.872 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T02:30:48.872 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T02:30:49.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.063+0000 7fd3c5f9d700 1 Processor -- start 2023-10-06T02:30:49.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.063+0000 7fd3c5f9d700 1 -- start start 2023-10-06T02:30:49.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.064+0000 7fd3c5f9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd3c0159770 0x7fd3c006dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:49.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.064+0000 7fd3c5f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3c006c6a0 0x7fd3c00ca5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:49.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.064+0000 7fd3c5f9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3c006d740 con 0x7fd3c006a330 2023-10-06T02:30:49.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.064+0000 7fd3c5f9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3c0159b40 con 0x7fd3c0159770 2023-10-06T02:30:49.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.064+0000 7fd3c5f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c006cb30 con 0x7fd3c006c6a0 2023-10-06T02:30:49.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.064+0000 7fd3c4f9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd3c0159770 0x7fd3c006dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:49.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.064+0000 7fd3c4f9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd3c0159770 0x7fd3c006dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54662/0 (socket says 172.21.15.31:54662) 2023-10-06T02:30:49.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.064+0000 7fd3c4f9b700 1 -- 172.21.15.31:0/2875858960 learned_addr learned my addr 172.21.15.31:0/2875858960 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:49.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.065+0000 7fd3bffff700 1 --2- 172.21.15.31:0/2875858960 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3c006c6a0 0x7fd3c00ca5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:49.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.065+0000 7fd3c4f9b700 1 -- 172.21.15.31:0/2875858960 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3c006c6a0 msgr2=0x7fd3c00ca5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:49.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.065+0000 7fd3c4f9b700 1 --2- 172.21.15.31:0/2875858960 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3c006c6a0 0x7fd3c00ca5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:49.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.065+0000 7fd3c4f9b700 1 -- 172.21.15.31:0/2875858960 >> v1:172.21.15.31:6789/0 conn(0x7fd3c006a330 legacy=0x7fd3c00c7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:49.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.065+0000 7fd3c4f9b700 1 -- 172.21.15.31:0/2875858960 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3c006ce10 con 0x7fd3c0159770 2023-10-06T02:30:49.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.065+0000 7fd3c4f9b700 1 --2- 172.21.15.31:0/2875858960 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3c0159770 0x7fd3c006dbd0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7fd3b40099b0 tx=0x7fd3b4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0317f78628c9651 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:49.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.065+0000 7fd3bf7fe700 1 -- 172.21.15.31:0/2875858960 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b4009ea0 con 0x7fd3c0159770 2023-10-06T02:30:49.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.066+0000 7fd3bf7fe700 1 -- 172.21.15.31:0/2875858960 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3b400b860 con 0x7fd3c0159770 2023-10-06T02:30:49.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.066+0000 7fd3bf7fe700 1 -- 172.21.15.31:0/2875858960 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b400bc00 con 0x7fd3c0159770 2023-10-06T02:30:49.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.066+0000 7fd3c5f9d700 1 -- 172.21.15.31:0/2875858960 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3c0159770 msgr2=0x7fd3c006dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:49.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.066+0000 7fd3c5f9d700 1 --2- 172.21.15.31:0/2875858960 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3c0159770 0x7fd3c006dbd0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7fd3b40099b0 tx=0x7fd3b4009250 comp rx=0 tx=0).stop 2023-10-06T02:30:49.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.066+0000 7fd3c5f9d700 1 -- 172.21.15.31:0/2875858960 shutdown_connections 2023-10-06T02:30:49.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.066+0000 7fd3c5f9d700 1 --2- 172.21.15.31:0/2875858960 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3c0159770 0x7fd3c006dbd0 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:49.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.066+0000 7fd3c5f9d700 1 --2- 172.21.15.31:0/2875858960 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3c006c6a0 0x7fd3c00ca5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:49.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.066+0000 7fd3c5f9d700 1 -- 172.21.15.31:0/2875858960 >> 172.21.15.31:0/2875858960 conn(0x7fd3c006e790 msgr2=0x7fd3c006eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:49.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.066+0000 7fd3c5f9d700 1 -- 172.21.15.31:0/2875858960 shutdown_connections 2023-10-06T02:30:49.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.066+0000 7fd3c5f9d700 1 -- 172.21.15.31:0/2875858960 wait complete. 2023-10-06T02:30:49.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.068+0000 7fd3c5f9d700 1 Processor -- start 2023-10-06T02:30:49.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.068+0000 7fd3c5f9d700 1 -- start start 2023-10-06T02:30:49.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.069+0000 7fd3c5f9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3c006a330 0x7fd3c014d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:49.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.069+0000 7fd3c5f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3c006c6a0 0x7fd3c01693a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:49.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.069+0000 7fd3c5f9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3c0159770 0x7fd3c01698e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:49.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.069+0000 7fd3c5f9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c00ad320 con 0x7fd3c0159770 2023-10-06T02:30:49.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.069+0000 7fd3c5f9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c00ad600 con 0x7fd3c006a330 2023-10-06T02:30:49.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.069+0000 7fd3c5f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c00ad8e0 con 0x7fd3c006c6a0 2023-10-06T02:30:49.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.069+0000 7fd3c579c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3c0159770 0x7fd3c01698e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:49.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.069+0000 7fd3c579c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3c0159770 0x7fd3c01698e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54676/0 (socket says 172.21.15.31:54676) 2023-10-06T02:30:49.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.069+0000 7fd3c579c700 1 -- 172.21.15.31:0/1555193367 learned_addr learned my addr 172.21.15.31:0/1555193367 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:49.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.069+0000 7fd3c4f9b700 1 --2- 172.21.15.31:0/1555193367 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3c006a330 0x7fd3c014d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:49.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.069+0000 7fd3bffff700 1 --2- 172.21.15.31:0/1555193367 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3c006c6a0 0x7fd3c01693a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:49.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.070+0000 7fd3c579c700 1 -- 172.21.15.31:0/1555193367 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3c006c6a0 msgr2=0x7fd3c01693a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:49.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.070+0000 7fd3c579c700 1 --2- 172.21.15.31:0/1555193367 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3c006c6a0 0x7fd3c01693a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:49.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.070+0000 7fd3c579c700 1 -- 172.21.15.31:0/1555193367 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3c006a330 msgr2=0x7fd3c014d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:49.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.070+0000 7fd3c579c700 1 --2- 172.21.15.31:0/1555193367 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3c006a330 0x7fd3c014d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:49.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.070+0000 7fd3c579c700 1 -- 172.21.15.31:0/1555193367 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3b40094f0 con 0x7fd3c0159770 2023-10-06T02:30:49.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.070+0000 7fd3bffff700 1 --2- 172.21.15.31:0/1555193367 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3c006c6a0 0x7fd3c01693a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:30:49.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.070+0000 7fd3c579c700 1 --2- 172.21.15.31:0/1555193367 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3c0159770 0x7fd3c01698e0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fd3b800c840 tx=0x7fd3b8001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:49.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.070+0000 7fd3bcff9700 1 -- 172.21.15.31:0/1555193367 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b8012630 con 0x7fd3c0159770 2023-10-06T02:30:49.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.070+0000 7fd3bcff9700 1 -- 172.21.15.31:0/1555193367 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3b80127b0 con 0x7fd3c0159770 2023-10-06T02:30:49.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.070+0000 7fd3bcff9700 1 -- 172.21.15.31:0/1555193367 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b8012b70 con 0x7fd3c0159770 2023-10-06T02:30:49.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.070+0000 7fd3c5f9d700 1 -- 172.21.15.31:0/1555193367 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3c00adbc0 con 0x7fd3c0159770 2023-10-06T02:30:49.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.070+0000 7fd3c5f9d700 1 -- 172.21.15.31:0/1555193367 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3c014d9b0 con 0x7fd3c0159770 2023-10-06T02:30:49.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.071+0000 7fd3c5f9d700 1 -- 172.21.15.31:0/1555193367 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3c004c3c0 con 0x7fd3c0159770 2023-10-06T02:30:49.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.072+0000 7fd3bcff9700 1 -- 172.21.15.31:0/1555193367 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7fd3b8003030 con 0x7fd3c0159770 2023-10-06T02:30:49.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.073+0000 7fd3bcff9700 1 --2- 172.21.15.31:0/1555193367 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd3a40674d0 0x7fd3a4069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:49.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.074+0000 7fd3bcff9700 1 -- 172.21.15.31:0/1555193367 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5234+0+0 (secure 0 0 0) 0x7fd3b8084c80 con 0x7fd3c0159770 2023-10-06T02:30:49.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.074+0000 7fd3c4f9b700 1 --2- 172.21.15.31:0/1555193367 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd3a40674d0 0x7fd3a4069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:49.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.075+0000 7fd3c4f9b700 1 --2- 172.21.15.31:0/1555193367 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd3a40674d0 0x7fd3a4069980 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd3b40099b0 tx=0x7fd3b4017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:49.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.078+0000 7fd3bcff9700 1 -- 172.21.15.31:0/1555193367 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd3b8051810 con 0x7fd3c0159770 2023-10-06T02:30:49.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.284+0000 7fd3c5f9d700 1 -- 172.21.15.31:0/1555193367 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fd3c006d0f0 con 0x7fd3c0159770 2023-10-06T02:30:49.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.646+0000 7fd3bcff9700 1 -- 172.21.15.31:0/1555193367 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v141) v1 ==== 138+0+0 (secure 0 0 0) 0x7fd3b8054e30 con 0x7fd3c0159770 2023-10-06T02:30:49.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.653+0000 7fd3c5f9d700 1 -- 172.21.15.31:0/1555193367 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd3a40674d0 msgr2=0x7fd3a4069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:49.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.653+0000 7fd3c5f9d700 1 --2- 172.21.15.31:0/1555193367 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd3a40674d0 0x7fd3a4069980 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd3b40099b0 tx=0x7fd3b4017000 comp rx=0 tx=0).stop 2023-10-06T02:30:49.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.653+0000 7fd3c5f9d700 1 -- 172.21.15.31:0/1555193367 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3c0159770 msgr2=0x7fd3c01698e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:49.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.653+0000 7fd3c5f9d700 1 --2- 172.21.15.31:0/1555193367 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3c0159770 0x7fd3c01698e0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fd3b800c840 tx=0x7fd3b8001670 comp rx=0 tx=0).stop 2023-10-06T02:30:49.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.653+0000 7fd3c5f9d700 1 -- 172.21.15.31:0/1555193367 shutdown_connections 2023-10-06T02:30:49.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.653+0000 7fd3c5f9d700 1 --2- 172.21.15.31:0/1555193367 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3c0159770 0x7fd3c01698e0 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:49.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.653+0000 7fd3c5f9d700 1 --2- 172.21.15.31:0/1555193367 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3c006a330 0x7fd3c014d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:49.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.653+0000 7fd3c5f9d700 1 --2- 172.21.15.31:0/1555193367 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd3a40674d0 0x7fd3a4069980 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:49.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.653+0000 7fd3c5f9d700 1 --2- 172.21.15.31:0/1555193367 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3c006c6a0 0x7fd3c01693a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:49.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.653+0000 7fd3c5f9d700 1 -- 172.21.15.31:0/1555193367 >> 172.21.15.31:0/1555193367 conn(0x7fd3c006e790 msgr2=0x7fd3c0158940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:49.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.653+0000 7fd3c5f9d700 1 -- 172.21.15.31:0/1555193367 shutdown_connections 2023-10-06T02:30:49.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.653+0000 7fd3c5f9d700 1 -- 172.21.15.31:0/1555193367 wait complete. 2023-10-06T02:30:49.657 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' created 2023-10-06T02:30:49.675 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T02:30:49.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.868+0000 7f1ae119d700 1 Processor -- start 2023-10-06T02:30:49.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.868+0000 7f1ae119d700 1 -- start start 2023-10-06T02:30:49.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.869+0000 7f1ae119d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1adc13fb20 0x7f1adc15a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:49.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.870+0000 7f1ae119d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1adc13fef0 0x7f1adc0c7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:49.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.870+0000 7f1ae119d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f1adc06aaa0 0x7f1adc0c7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:49.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.870+0000 7f1ae119d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1adc0c8410 con 0x7f1adc13fef0 2023-10-06T02:30:49.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.870+0000 7f1ae119d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1adc15ae70 con 0x7f1adc06aaa0 2023-10-06T02:30:49.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.870+0000 7f1ae119d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1adc06af60 con 0x7f1adc13fb20 2023-10-06T02:30:49.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.870+0000 7f1adb7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1adc13fef0 0x7f1adc0c7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:49.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.870+0000 7f1adb7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1adc13fef0 0x7f1adc0c7990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54692/0 (socket says 172.21.15.31:54692) 2023-10-06T02:30:49.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.870+0000 7f1adb7fe700 1 -- 172.21.15.31:0/1362275704 learned_addr learned my addr 172.21.15.31:0/1362275704 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:49.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.870+0000 7f1adb7fe700 1 -- 172.21.15.31:0/1362275704 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1adc13fb20 msgr2=0x7f1adc15a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:49.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.870+0000 7f1ae099c700 1 --2- 172.21.15.31:0/1362275704 >> v2:172.21.15.62:3300/0 conn(0x7f1adc06aaa0 0x7f1adc0c7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:49.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.870+0000 7f1adbfff700 1 --2- 172.21.15.31:0/1362275704 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1adc13fb20 0x7f1adc15a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:49.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.870+0000 7f1adb7fe700 1 --2- 172.21.15.31:0/1362275704 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1adc13fb20 0x7f1adc15a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:49.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.870+0000 7f1adb7fe700 1 -- 172.21.15.31:0/1362275704 >> v2:172.21.15.62:3300/0 conn(0x7f1adc06aaa0 msgr2=0x7f1adc0c7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:49.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.870+0000 7f1adb7fe700 1 --2- 172.21.15.31:0/1362275704 >> v2:172.21.15.62:3300/0 conn(0x7f1adc06aaa0 0x7f1adc0c7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:49.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.870+0000 7f1adb7fe700 1 -- 172.21.15.31:0/1362275704 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1adc06b240 con 0x7f1adc13fef0 2023-10-06T02:30:49.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.870+0000 7f1adbfff700 1 --2- 172.21.15.31:0/1362275704 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1adc13fb20 0x7f1adc15a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:30:49.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.870+0000 7f1ae099c700 1 --2- 172.21.15.31:0/1362275704 >> v2:172.21.15.62:3300/0 conn(0x7f1adc06aaa0 0x7f1adc0c7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:30:49.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.871+0000 7f1adb7fe700 1 --2- 172.21.15.31:0/1362275704 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1adc13fef0 0x7f1adc0c7990 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f1adc156920 tx=0x7f1ad0009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc181a6eb48245cf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:49.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.871+0000 7f1adaffd700 1 -- 172.21.15.31:0/1362275704 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad0005c10 con 0x7f1adc13fef0 2023-10-06T02:30:49.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.871+0000 7f1adaffd700 1 -- 172.21.15.31:0/1362275704 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ad0005d70 con 0x7f1adc13fef0 2023-10-06T02:30:49.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.871+0000 7f1adaffd700 1 -- 172.21.15.31:0/1362275704 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad000b880 con 0x7f1adc13fef0 2023-10-06T02:30:49.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.871+0000 7f1ae119d700 1 -- 172.21.15.31:0/1362275704 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1adc13fef0 msgr2=0x7f1adc0c7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:49.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.871+0000 7f1ae119d700 1 --2- 172.21.15.31:0/1362275704 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1adc13fef0 0x7f1adc0c7990 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f1adc156920 tx=0x7f1ad0009220 comp rx=0 tx=0).stop 2023-10-06T02:30:49.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.871+0000 7f1ae119d700 1 -- 172.21.15.31:0/1362275704 shutdown_connections 2023-10-06T02:30:49.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.871+0000 7f1ae119d700 1 --2- 172.21.15.31:0/1362275704 >> v2:172.21.15.62:3300/0 conn(0x7f1adc06aaa0 0x7f1adc0c7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:49.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.871+0000 7f1ae119d700 1 --2- 172.21.15.31:0/1362275704 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1adc13fb20 0x7f1adc15a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:49.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.871+0000 7f1ae119d700 1 --2- 172.21.15.31:0/1362275704 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1adc13fef0 0x7f1adc0c7990 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:49.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.871+0000 7f1ae119d700 1 -- 172.21.15.31:0/1362275704 >> 172.21.15.31:0/1362275704 conn(0x7f1adc06d170 msgr2=0x7f1adc0cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:49.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.872+0000 7f1ae119d700 1 -- 172.21.15.31:0/1362275704 shutdown_connections 2023-10-06T02:30:49.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.872+0000 7f1ae119d700 1 -- 172.21.15.31:0/1362275704 wait complete. 2023-10-06T02:30:49.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.873+0000 7f1ae119d700 1 Processor -- start 2023-10-06T02:30:49.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.873+0000 7f1ae119d700 1 -- start start 2023-10-06T02:30:49.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.874+0000 7f1ae119d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1adc06aaa0 0x7f1adc137c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:49.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.874+0000 7f1ae119d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1adc13fb20 0x7f1adc151040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:49.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.874+0000 7f1ae119d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1adc13fef0 0x7f1adc13e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:49.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.874+0000 7f1ae119d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1adc156d70 con 0x7f1adc13fb20 2023-10-06T02:30:49.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.874+0000 7f1ae119d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1adc0adae0 con 0x7f1adc06aaa0 2023-10-06T02:30:49.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.874+0000 7f1ae119d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1adc0addc0 con 0x7f1adc13fef0 2023-10-06T02:30:49.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.874+0000 7f1adb7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1adc13fb20 0x7f1adc151040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:49.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.874+0000 7f1adb7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1adc13fb20 0x7f1adc151040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54698/0 (socket says 172.21.15.31:54698) 2023-10-06T02:30:49.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.874+0000 7f1adb7fe700 1 -- 172.21.15.31:0/3041946084 learned_addr learned my addr 172.21.15.31:0/3041946084 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:49.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.875+0000 7f1adbfff700 1 --2- 172.21.15.31:0/3041946084 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1adc06aaa0 0x7f1adc137c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:49.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.875+0000 7f1adb7fe700 1 -- 172.21.15.31:0/3041946084 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1adc13fef0 msgr2=0x7f1adc13e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:49.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.875+0000 7f1adb7fe700 1 --2- 172.21.15.31:0/3041946084 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1adc13fef0 0x7f1adc13e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:49.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.875+0000 7f1adb7fe700 1 -- 172.21.15.31:0/3041946084 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1adc06aaa0 msgr2=0x7f1adc137c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:49.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.875+0000 7f1adb7fe700 1 --2- 172.21.15.31:0/3041946084 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1adc06aaa0 0x7f1adc137c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:49.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.875+0000 7f1adb7fe700 1 -- 172.21.15.31:0/3041946084 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ad0009480 con 0x7f1adc13fb20 2023-10-06T02:30:49.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.875+0000 7f1adbfff700 1 --2- 172.21.15.31:0/3041946084 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1adc06aaa0 0x7f1adc137c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:30:49.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.875+0000 7f1adb7fe700 1 --2- 172.21.15.31:0/3041946084 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1adc13fb20 0x7f1adc151040 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f1ad000b3c0 tx=0x7f1ad0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:49.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.875+0000 7f1ac3fff700 1 -- 172.21.15.31:0/3041946084 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad0011920 con 0x7f1adc13fb20 2023-10-06T02:30:49.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.875+0000 7f1ac3fff700 1 -- 172.21.15.31:0/3041946084 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ad0020070 con 0x7f1adc13fb20 2023-10-06T02:30:49.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.875+0000 7f1ac3fff700 1 -- 172.21.15.31:0/3041946084 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad0011cb0 con 0x7f1adc13fb20 2023-10-06T02:30:49.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.875+0000 7f1ae119d700 1 -- 172.21.15.31:0/3041946084 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1adc0ae0a0 con 0x7f1adc13fb20 2023-10-06T02:30:49.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.876+0000 7f1ae119d700 1 -- 172.21.15.31:0/3041946084 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1adc1378b0 con 0x7f1adc13fb20 2023-10-06T02:30:49.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.876+0000 7f1ae119d700 1 -- 172.21.15.31:0/3041946084 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1adc04c3c0 con 0x7f1adc13fb20 2023-10-06T02:30:49.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.878+0000 7f1ac3fff700 1 -- 172.21.15.31:0/3041946084 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f1ad0022030 con 0x7f1adc13fb20 2023-10-06T02:30:49.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.878+0000 7f1ac3fff700 1 --2- 172.21.15.31:0/3041946084 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f1ac80674d0 0x7f1ac8069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:49.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.879+0000 7f1ac3fff700 1 -- 172.21.15.31:0/3041946084 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5604+0+0 (secure 0 0 0) 0x7f1ad0087010 con 0x7f1adc13fb20 2023-10-06T02:30:49.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.879+0000 7f1adbfff700 1 --2- 172.21.15.31:0/3041946084 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f1ac80674d0 0x7f1ac8069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:49.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.880+0000 7f1adbfff700 1 --2- 172.21.15.31:0/3041946084 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f1ac80674d0 0x7f1ac8069980 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f1acc006fd0 tx=0x7f1acc008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:49.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:49.882+0000 7f1ac3fff700 1 -- 172.21.15.31:0/3041946084 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1ad0053ab0 con 0x7f1adc13fb20 2023-10-06T02:30:50.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.086+0000 7f1ae119d700 1 -- 172.21.15.31:0/3041946084 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f1adc0b0ef0 con 0x7f1adc13fb20 2023-10-06T02:30:50.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.656+0000 7f1ac3fff700 1 -- 172.21.15.31:0/3041946084 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v142) v1 ==== 170+0+0 (secure 0 0 0) 0x7f1ad00570d0 con 0x7f1adc13fb20 2023-10-06T02:30:50.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.662+0000 7f1ae119d700 1 -- 172.21.15.31:0/3041946084 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f1ac80674d0 msgr2=0x7f1ac8069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:50.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.662+0000 7f1ae119d700 1 --2- 172.21.15.31:0/3041946084 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f1ac80674d0 0x7f1ac8069980 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f1acc006fd0 tx=0x7f1acc008040 comp rx=0 tx=0).stop 2023-10-06T02:30:50.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.663+0000 7f1ae119d700 1 -- 172.21.15.31:0/3041946084 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1adc13fb20 msgr2=0x7f1adc151040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:50.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.663+0000 7f1ae119d700 1 --2- 172.21.15.31:0/3041946084 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1adc13fb20 0x7f1adc151040 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f1ad000b3c0 tx=0x7f1ad0016000 comp rx=0 tx=0).stop 2023-10-06T02:30:50.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.666+0000 7f1ae119d700 1 -- 172.21.15.31:0/3041946084 shutdown_connections 2023-10-06T02:30:50.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.667+0000 7f1ae119d700 1 --2- 172.21.15.31:0/3041946084 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f1ac80674d0 0x7f1ac8069980 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:50.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.667+0000 7f1ae119d700 1 --2- 172.21.15.31:0/3041946084 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1adc13fef0 0x7f1adc13e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:50.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.667+0000 7f1ae119d700 1 --2- 172.21.15.31:0/3041946084 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1adc06aaa0 0x7f1adc137c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:50.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.667+0000 7f1ae119d700 1 --2- 172.21.15.31:0/3041946084 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1adc13fb20 0x7f1adc151040 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:50.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.667+0000 7f1ae119d700 1 -- 172.21.15.31:0/3041946084 >> 172.21.15.31:0/3041946084 conn(0x7f1adc06d170 msgr2=0x7f1adc06ef80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:50.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.671+0000 7f1ae119d700 1 -- 172.21.15.31:0/3041946084 shutdown_connections 2023-10-06T02:30:50.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.671+0000 7f1ae119d700 1 -- 172.21.15.31:0/3041946084 wait complete. 2023-10-06T02:30:50.676 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' created 2023-10-06T02:30:50.698 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T02:30:50.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.888+0000 7fe71119d700 1 Processor -- start 2023-10-06T02:30:50.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.888+0000 7fe71119d700 1 -- start start 2023-10-06T02:30:50.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.890+0000 7fe71119d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe70c0c7fb0 0x7fe70c06abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:50.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.890+0000 7fe71119d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe70c06b290 con 0x7fe70c0c7be0 2023-10-06T02:30:50.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.890+0000 7fe71119d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe70c06b570 con 0x7fe70c13f780 2023-10-06T02:30:50.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.890+0000 7fe71119d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe70c06bf10 con 0x7fe70c0c7fb0 2023-10-06T02:30:50.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.890+0000 7fe70bfff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fe70c0c7be0 0x7fe70c15a990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:47900/0 (socket says 172.21.15.31:47900) 2023-10-06T02:30:50.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.890+0000 7fe70bfff700 1 -- 172.21.15.31:0/1423966068 learned_addr learned my addr 172.21.15.31:0/1423966068 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:30:50.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.890+0000 7fe70b7fe700 1 --2- 172.21.15.31:0/1423966068 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe70c0c7fb0 0x7fe70c06abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:50.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.890+0000 7fe70affd700 1 -- 172.21.15.31:0/1423966068 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4137615216 0 0) 0x7fe70c06b290 con 0x7fe70c0c7be0 2023-10-06T02:30:50.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.890+0000 7fe70affd700 1 -- 172.21.15.31:0/1423966068 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6f4003570 con 0x7fe70c0c7be0 2023-10-06T02:30:50.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.891+0000 7fe70affd700 1 -- 172.21.15.31:0/1423966068 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 206187248 0 0) 0x7fe70c06b570 con 0x7fe70c13f780 2023-10-06T02:30:50.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.891+0000 7fe70affd700 1 -- 172.21.15.31:0/1423966068 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe70c06b290 con 0x7fe70c13f780 2023-10-06T02:30:50.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.891+0000 7fe70affd700 1 -- 172.21.15.31:0/1423966068 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2165786812 0 0) 0x7fe6f4003570 con 0x7fe70c0c7be0 2023-10-06T02:30:50.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.891+0000 7fe70affd700 1 -- 172.21.15.31:0/1423966068 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe70c0c7fb0 msgr2=0x7fe70c06abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:50.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.891+0000 7fe70affd700 1 --2- 172.21.15.31:0/1423966068 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe70c0c7fb0 0x7fe70c06abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:50.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.891+0000 7fe70affd700 1 -- 172.21.15.31:0/1423966068 >> v1:172.21.15.62:6789/0 conn(0x7fe70c13f780 legacy=0x7fe70c0c9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:50.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.891+0000 7fe70affd700 1 -- 172.21.15.31:0/1423966068 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe70c06b850 con 0x7fe70c0c7be0 2023-10-06T02:30:50.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.891+0000 7fe70affd700 1 -- 172.21.15.31:0/1423966068 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7fe6fc001470 con 0x7fe70c0c7be0 2023-10-06T02:30:50.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.891+0000 7fe70affd700 1 -- 172.21.15.31:0/1423966068 >> v1:172.21.15.31:6789/0 conn(0x7fe70c0c7be0 legacy=0x7fe70c15a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:50.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.891+0000 7fe70affd700 1 --2- 172.21.15.31:0/1423966068 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe6f4003c20 0x7fe6f4004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:50.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.891+0000 7fe70affd700 1 --2- 172.21.15.31:0/1423966068 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6f4004810 0x7fe6f4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:50.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.891+0000 7fe70affd700 1 --2- 172.21.15.31:0/1423966068 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe70c0c7fb0 0x7fe6f40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:50.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.891+0000 7fe70affd700 1 -- 172.21.15.31:0/1423966068 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe70c06bf10 con 0x7fe6f4003c20 2023-10-06T02:30:50.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.891+0000 7fe70affd700 1 -- 172.21.15.31:0/1423966068 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f400bb40 con 0x7fe6f4004810 2023-10-06T02:30:50.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.891+0000 7fe70affd700 1 -- 172.21.15.31:0/1423966068 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f400df10 con 0x7fe70c0c7fb0 2023-10-06T02:30:50.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.891+0000 7fe70bfff700 1 --2- 172.21.15.31:0/1423966068 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe6f4003c20 0x7fe6f4004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:50.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.891+0000 7fe70b7fe700 1 --2- 172.21.15.31:0/1423966068 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6f4004810 0x7fe6f4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:50.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.891+0000 7fe71099c700 1 --2- 172.21.15.31:0/1423966068 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe70c0c7fb0 0x7fe6f40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:50.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.891+0000 7fe70bfff700 1 -- 172.21.15.31:0/1423966068 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe70c0c7fb0 msgr2=0x7fe6f40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:50.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.892+0000 7fe70bfff700 1 --2- 172.21.15.31:0/1423966068 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe70c0c7fb0 0x7fe6f40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:50.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.892+0000 7fe70bfff700 1 -- 172.21.15.31:0/1423966068 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6f4004810 msgr2=0x7fe6f4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:50.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.892+0000 7fe70bfff700 1 --2- 172.21.15.31:0/1423966068 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6f4004810 0x7fe6f4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:50.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.892+0000 7fe70bfff700 1 -- 172.21.15.31:0/1423966068 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6f4003570 con 0x7fe6f4003c20 2023-10-06T02:30:50.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.892+0000 7fe70bfff700 1 --2- 172.21.15.31:0/1423966068 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe6f4003c20 0x7fe6f4004010 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc00c2d0 tx=0x7fe70c06be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ec1a22e79f70e24 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:50.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.892+0000 7fe70affd700 1 -- 172.21.15.31:0/1423966068 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6fc00cc00 con 0x7fe6f4003c20 2023-10-06T02:30:50.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.892+0000 7fe70affd700 1 -- 172.21.15.31:0/1423966068 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6fc00cda0 con 0x7fe6f4003c20 2023-10-06T02:30:50.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.892+0000 7fe70affd700 1 -- 172.21.15.31:0/1423966068 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6fc005b70 con 0x7fe6f4003c20 2023-10-06T02:30:50.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.892+0000 7fe71119d700 1 -- 172.21.15.31:0/1423966068 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe6f4003c20 msgr2=0x7fe6f4004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:50.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.893+0000 7fe71119d700 1 --2- 172.21.15.31:0/1423966068 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe6f4003c20 0x7fe6f4004010 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc00c2d0 tx=0x7fe70c06be10 comp rx=0 tx=0).stop 2023-10-06T02:30:50.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.893+0000 7fe71099c700 1 -- 172.21.15.31:0/1423966068 reap_dead start 2023-10-06T02:30:50.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.893+0000 7fe71119d700 1 -- 172.21.15.31:0/1423966068 shutdown_connections 2023-10-06T02:30:50.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.893+0000 7fe71119d700 1 -- 172.21.15.31:0/1423966068 >> 172.21.15.31:0/1423966068 conn(0x7fe70c06d110 msgr2=0x7fe70c0cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:50.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.893+0000 7fe71119d700 1 -- 172.21.15.31:0/1423966068 shutdown_connections 2023-10-06T02:30:50.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.893+0000 7fe71119d700 1 -- 172.21.15.31:0/1423966068 wait complete. 2023-10-06T02:30:50.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.894+0000 7fe71119d700 1 Processor -- start 2023-10-06T02:30:50.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.894+0000 7fe71119d700 1 -- start start 2023-10-06T02:30:50.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.895+0000 7fe71119d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe6f4003c20 0x7fe70c0aa6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:50.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.895+0000 7fe71119d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe70c13e9c0 0x7fe70c0b1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:50.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.895+0000 7fe71119d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe70c13ee30 0x7fe70c0b1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:50.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.895+0000 7fe71119d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe70c156a80 con 0x7fe70c13ee30 2023-10-06T02:30:50.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.895+0000 7fe71119d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe70c1578e0 con 0x7fe70c13e9c0 2023-10-06T02:30:50.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.895+0000 7fe71119d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe70c156cd0 con 0x7fe6f4003c20 2023-10-06T02:30:50.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.896+0000 7fe71099c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe70c13ee30 0x7fe70c0b1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:50.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.896+0000 7fe71099c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe70c13ee30 0x7fe70c0b1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54718/0 (socket says 172.21.15.31:54718) 2023-10-06T02:30:50.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.896+0000 7fe71099c700 1 -- 172.21.15.31:0/1047238535 learned_addr learned my addr 172.21.15.31:0/1047238535 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:50.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.896+0000 7fe70bfff700 1 --2- 172.21.15.31:0/1047238535 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe6f4003c20 0x7fe70c0aa6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:50.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.896+0000 7fe70b7fe700 1 --2- 172.21.15.31:0/1047238535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe70c13e9c0 0x7fe70c0b1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:50.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.896+0000 7fe71099c700 1 -- 172.21.15.31:0/1047238535 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe6f4003c20 msgr2=0x7fe70c0aa6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:50.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.896+0000 7fe71099c700 1 --2- 172.21.15.31:0/1047238535 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe6f4003c20 0x7fe70c0aa6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:50.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.896+0000 7fe71099c700 1 -- 172.21.15.31:0/1047238535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe70c13e9c0 msgr2=0x7fe70c0b1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:50.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.896+0000 7fe71099c700 1 --2- 172.21.15.31:0/1047238535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe70c13e9c0 0x7fe70c0b1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:50.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.896+0000 7fe71099c700 1 -- 172.21.15.31:0/1047238535 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe70c156fb0 con 0x7fe70c13ee30 2023-10-06T02:30:50.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.896+0000 7fe70b7fe700 1 --2- 172.21.15.31:0/1047238535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe70c13e9c0 0x7fe70c0b1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:30:50.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.896+0000 7fe70bfff700 1 --2- 172.21.15.31:0/1047238535 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe6f4003c20 0x7fe70c0aa6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:30:50.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.896+0000 7fe71099c700 1 --2- 172.21.15.31:0/1047238535 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe70c13ee30 0x7fe70c0b1560 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fe70c13fb50 tx=0x7fe704009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:50.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.897+0000 7fe6f3fff700 1 -- 172.21.15.31:0/1047238535 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7040032e0 con 0x7fe70c13ee30 2023-10-06T02:30:50.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.897+0000 7fe6f3fff700 1 -- 172.21.15.31:0/1047238535 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe704003440 con 0x7fe70c13ee30 2023-10-06T02:30:50.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.897+0000 7fe6f3fff700 1 -- 172.21.15.31:0/1047238535 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe704001930 con 0x7fe70c13ee30 2023-10-06T02:30:50.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.897+0000 7fe71119d700 1 -- 172.21.15.31:0/1047238535 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe70c157290 con 0x7fe70c13ee30 2023-10-06T02:30:50.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.897+0000 7fe71119d700 1 -- 172.21.15.31:0/1047238535 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe70c0aac20 con 0x7fe70c13ee30 2023-10-06T02:30:50.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.898+0000 7fe71119d700 1 -- 172.21.15.31:0/1047238535 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe70c0c6540 con 0x7fe70c13ee30 2023-10-06T02:30:50.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.899+0000 7fe6f3fff700 1 -- 172.21.15.31:0/1047238535 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7fe704001d20 con 0x7fe70c13ee30 2023-10-06T02:30:50.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.900+0000 7fe6f3fff700 1 --2- 172.21.15.31:0/1047238535 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe6f80674d0 0x7fe6f8069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:50.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.900+0000 7fe6f3fff700 1 -- 172.21.15.31:0/1047238535 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5986+0+0 (secure 0 0 0) 0x7fe704085bb0 con 0x7fe70c13ee30 2023-10-06T02:30:50.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.900+0000 7fe70bfff700 1 --2- 172.21.15.31:0/1047238535 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe6f80674d0 0x7fe6f8069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:50.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.901+0000 7fe70bfff700 1 --2- 172.21.15.31:0/1047238535 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe6f80674d0 0x7fe6f8069980 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc007c90 tx=0x7fe6fc015000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:50.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:50.904+0000 7fe6f3fff700 1 -- 172.21.15.31:0/1047238535 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe7040524d0 con 0x7fe70c13ee30 2023-10-06T02:30:51.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:51.151+0000 7fe71119d700 1 -- 172.21.15.31:0/1047238535 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fe70c157a50 con 0x7fe70c13ee30 2023-10-06T02:30:52.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.084+0000 7fe6f3fff700 1 -- 172.21.15.31:0/1047238535 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 23 and data pool 24 v96) v1 ==== 174+0+0 (secure 0 0 0) 0x7fe704055af0 con 0x7fe70c13ee30 2023-10-06T02:30:52.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.088+0000 7fe71119d700 1 -- 172.21.15.31:0/1047238535 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe6f80674d0 msgr2=0x7fe6f8069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:52.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.088+0000 7fe71119d700 1 --2- 172.21.15.31:0/1047238535 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe6f80674d0 0x7fe6f8069980 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc007c90 tx=0x7fe6fc015000 comp rx=0 tx=0).stop 2023-10-06T02:30:52.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.088+0000 7fe71119d700 1 -- 172.21.15.31:0/1047238535 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe70c13ee30 msgr2=0x7fe70c0b1560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:52.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.088+0000 7fe71119d700 1 --2- 172.21.15.31:0/1047238535 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe70c13ee30 0x7fe70c0b1560 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fe70c13fb50 tx=0x7fe704009040 comp rx=0 tx=0).stop 2023-10-06T02:30:52.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.089+0000 7fe71119d700 1 -- 172.21.15.31:0/1047238535 shutdown_connections 2023-10-06T02:30:52.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.089+0000 7fe71119d700 1 --2- 172.21.15.31:0/1047238535 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe70c13ee30 0x7fe70c0b1560 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:52.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.089+0000 7fe71119d700 1 --2- 172.21.15.31:0/1047238535 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe6f80674d0 0x7fe6f8069980 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:52.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.089+0000 7fe71119d700 1 --2- 172.21.15.31:0/1047238535 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe6f4003c20 0x7fe70c0aa6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:52.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.089+0000 7fe71119d700 1 --2- 172.21.15.31:0/1047238535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe70c13e9c0 0x7fe70c0b1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:52.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.089+0000 7fe71119d700 1 -- 172.21.15.31:0/1047238535 >> 172.21.15.31:0/1047238535 conn(0x7fe70c06d110 msgr2=0x7fe70c0c0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:52.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.089+0000 7fe71119d700 1 -- 172.21.15.31:0/1047238535 shutdown_connections 2023-10-06T02:30:52.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.089+0000 7fe71119d700 1 -- 172.21.15.31:0/1047238535 wait complete. 2023-10-06T02:30:52.098 INFO:teuthology.orchestra.run.smithi031.stderr:new fs with metadata pool 23 and data pool 24 2023-10-06T02:30:52.120 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:30:52.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.313+0000 7f17c9f9d700 1 Processor -- start 2023-10-06T02:30:52.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.313+0000 7f17c9f9d700 1 -- start start 2023-10-06T02:30:52.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.314+0000 7f17c9f9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f17c413fb20 0x7f17c415a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:52.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.314+0000 7f17c9f9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f17c406aaa0 0x7f17c40c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:52.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.314+0000 7f17c9f9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17c415ae70 con 0x7f17c413fef0 2023-10-06T02:30:52.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.314+0000 7f17c9f9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17c406af60 con 0x7f17c413fb20 2023-10-06T02:30:52.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.314+0000 7f17c9f9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17c406b240 con 0x7f17c406aaa0 2023-10-06T02:30:52.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.315+0000 7f17c8f9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f17c413fb20 0x7f17c415a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:52.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.315+0000 7f17c8f9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f17c413fb20 0x7f17c415a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54726/0 (socket says 172.21.15.31:54726) 2023-10-06T02:30:52.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.315+0000 7f17c8f9b700 1 -- 172.21.15.31:0/1064037516 learned_addr learned my addr 172.21.15.31:0/1064037516 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:52.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.315+0000 7f17c979c700 1 --2- 172.21.15.31:0/1064037516 >> v2:172.21.15.62:3300/0 conn(0x7f17c406aaa0 0x7f17c40c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:52.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.315+0000 7f17c8f9b700 1 -- 172.21.15.31:0/1064037516 >> v2:172.21.15.62:3300/0 conn(0x7f17c406aaa0 msgr2=0x7f17c40c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:52.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.315+0000 7f17c8f9b700 1 --2- 172.21.15.31:0/1064037516 >> v2:172.21.15.62:3300/0 conn(0x7f17c406aaa0 0x7f17c40c9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:52.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.315+0000 7f17c8f9b700 1 -- 172.21.15.31:0/1064037516 >> v1:172.21.15.31:6789/0 conn(0x7f17c413fef0 legacy=0x7f17c40c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:52.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.315+0000 7f17c8f9b700 1 -- 172.21.15.31:0/1064037516 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17c406b520 con 0x7f17c413fb20 2023-10-06T02:30:52.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.316+0000 7f17c8f9b700 1 --2- 172.21.15.31:0/1064037516 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17c413fb20 0x7f17c415a930 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f17c4069d40 tx=0x7f17b8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39c0194ffe7bffa9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:52.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.316+0000 7f17c37fe700 1 -- 172.21.15.31:0/1064037516 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b800b860 con 0x7f17c413fb20 2023-10-06T02:30:52.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.316+0000 7f17c37fe700 1 -- 172.21.15.31:0/1064037516 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17b800b9c0 con 0x7f17c413fb20 2023-10-06T02:30:52.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.316+0000 7f17c37fe700 1 -- 172.21.15.31:0/1064037516 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b800bd60 con 0x7f17c413fb20 2023-10-06T02:30:52.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.316+0000 7f17c9f9d700 1 -- 172.21.15.31:0/1064037516 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17c413fb20 msgr2=0x7f17c415a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:52.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.316+0000 7f17c9f9d700 1 --2- 172.21.15.31:0/1064037516 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17c413fb20 0x7f17c415a930 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f17c4069d40 tx=0x7f17b8009250 comp rx=0 tx=0).stop 2023-10-06T02:30:52.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.316+0000 7f17c9f9d700 1 -- 172.21.15.31:0/1064037516 shutdown_connections 2023-10-06T02:30:52.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.316+0000 7f17c9f9d700 1 --2- 172.21.15.31:0/1064037516 >> v2:172.21.15.62:3300/0 conn(0x7f17c406aaa0 0x7f17c40c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:52.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.316+0000 7f17c9f9d700 1 --2- 172.21.15.31:0/1064037516 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17c413fb20 0x7f17c415a930 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:52.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.316+0000 7f17c9f9d700 1 -- 172.21.15.31:0/1064037516 >> 172.21.15.31:0/1064037516 conn(0x7f17c406d170 msgr2=0x7f17c40cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:52.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.317+0000 7f17c9f9d700 1 -- 172.21.15.31:0/1064037516 shutdown_connections 2023-10-06T02:30:52.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.317+0000 7f17c9f9d700 1 -- 172.21.15.31:0/1064037516 wait complete. 2023-10-06T02:30:52.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.318+0000 7f17c9f9d700 1 Processor -- start 2023-10-06T02:30:52.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.318+0000 7f17c9f9d700 1 -- start start 2023-10-06T02:30:52.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.319+0000 7f17c9f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f17c406aaa0 0x7f17c4153cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:52.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.319+0000 7f17c9f9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17c413fb20 0x7f17c4157b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:52.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.319+0000 7f17c9f9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17c413fef0 0x7f17c4156950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:52.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.319+0000 7f17c9f9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c41578f0 con 0x7f17c413fef0 2023-10-06T02:30:52.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.319+0000 7f17c9f9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c4157330 con 0x7f17c413fb20 2023-10-06T02:30:52.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.319+0000 7f17c9f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17c4157610 con 0x7f17c406aaa0 2023-10-06T02:30:52.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.319+0000 7f17c979c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17c413fef0 0x7f17c4156950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:52.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.319+0000 7f17c979c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17c413fef0 0x7f17c4156950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54742/0 (socket says 172.21.15.31:54742) 2023-10-06T02:30:52.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.319+0000 7f17c979c700 1 -- 172.21.15.31:0/3374020853 learned_addr learned my addr 172.21.15.31:0/3374020853 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:52.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.319+0000 7f17c3fff700 1 --2- 172.21.15.31:0/3374020853 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17c413fb20 0x7f17c4157b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:52.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.319+0000 7f17c8f9b700 1 --2- 172.21.15.31:0/3374020853 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f17c406aaa0 0x7f17c4153cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:52.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.320+0000 7f17c979c700 1 -- 172.21.15.31:0/3374020853 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f17c406aaa0 msgr2=0x7f17c4153cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:52.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.320+0000 7f17c979c700 1 --2- 172.21.15.31:0/3374020853 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f17c406aaa0 0x7f17c4153cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:52.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.320+0000 7f17c979c700 1 -- 172.21.15.31:0/3374020853 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17c413fb20 msgr2=0x7f17c4157b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:52.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.320+0000 7f17c979c700 1 --2- 172.21.15.31:0/3374020853 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17c413fb20 0x7f17c4157b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:52.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.320+0000 7f17c979c700 1 -- 172.21.15.31:0/3374020853 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17b80094f0 con 0x7f17c413fef0 2023-10-06T02:30:52.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.320+0000 7f17c3fff700 1 --2- 172.21.15.31:0/3374020853 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17c413fb20 0x7f17c4157b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:30:52.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.320+0000 7f17c979c700 1 --2- 172.21.15.31:0/3374020853 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17c413fef0 0x7f17c4156950 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f17bc007f30 tx=0x7f17bc0098e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:52.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.320+0000 7f17c0ff9700 1 -- 172.21.15.31:0/3374020853 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17bc00ecf0 con 0x7f17c413fef0 2023-10-06T02:30:52.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.320+0000 7f17c0ff9700 1 -- 172.21.15.31:0/3374020853 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17bc00ee50 con 0x7f17c413fef0 2023-10-06T02:30:52.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.320+0000 7f17c0ff9700 1 -- 172.21.15.31:0/3374020853 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17bc014670 con 0x7f17c413fef0 2023-10-06T02:30:52.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.320+0000 7f17c9f9d700 1 -- 172.21.15.31:0/3374020853 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17c40abb90 con 0x7f17c413fef0 2023-10-06T02:30:52.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.320+0000 7f17c9f9d700 1 -- 172.21.15.31:0/3374020853 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17c4154210 con 0x7f17c413fef0 2023-10-06T02:30:52.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.321+0000 7f17c9f9d700 1 -- 172.21.15.31:0/3374020853 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17c404c3c0 con 0x7f17c413fef0 2023-10-06T02:30:52.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.322+0000 7f17c0ff9700 1 -- 172.21.15.31:0/3374020853 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f17bc0149e0 con 0x7f17c413fef0 2023-10-06T02:30:52.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.323+0000 7f17c0ff9700 1 --2- 172.21.15.31:0/3374020853 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f17a80674d0 0x7f17a8069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:52.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.323+0000 7f17c0ff9700 1 -- 172.21.15.31:0/3374020853 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f17bc085820 con 0x7f17c413fef0 2023-10-06T02:30:52.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.324+0000 7f17c8f9b700 1 --2- 172.21.15.31:0/3374020853 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f17a80674d0 0x7f17a8069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:52.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.324+0000 7f17c8f9b700 1 --2- 172.21.15.31:0/3374020853 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f17a80674d0 0x7f17a8069980 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f17c4069d40 tx=0x7f17b8017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:52.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.327+0000 7f17c0ff9700 1 -- 172.21.15.31:0/3374020853 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f17bc052120 con 0x7f17c413fef0 2023-10-06T02:30:52.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.534+0000 7f17c9f9d700 1 -- 172.21.15.31:0/3374020853 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f17c406b800 con 0x7f17c413fef0 2023-10-06T02:30:52.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.535+0000 7f17c0ff9700 1 -- 172.21.15.31:0/3374020853 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v143) v1 ==== 74+0+16889 (secure 0 0 0) 0x7f17bc055740 con 0x7f17c413fef0 2023-10-06T02:30:52.537 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:30:52.537 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":143,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:30:51.153401+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:30:49.325914+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":24,"pool_name":"cephfs_data","create_time":"2023-10-06T02:30:50.127083+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":20,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:30:52.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.538+0000 7f17c9f9d700 1 -- 172.21.15.31:0/3374020853 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f17a80674d0 msgr2=0x7f17a8069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:52.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.538+0000 7f17c9f9d700 1 --2- 172.21.15.31:0/3374020853 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f17a80674d0 0x7f17a8069980 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f17c4069d40 tx=0x7f17b8017000 comp rx=0 tx=0).stop 2023-10-06T02:30:52.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.538+0000 7f17c9f9d700 1 -- 172.21.15.31:0/3374020853 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17c413fef0 msgr2=0x7f17c4156950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:52.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.538+0000 7f17c9f9d700 1 --2- 172.21.15.31:0/3374020853 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17c413fef0 0x7f17c4156950 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f17bc007f30 tx=0x7f17bc0098e0 comp rx=0 tx=0).stop 2023-10-06T02:30:52.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.538+0000 7f17c9f9d700 1 -- 172.21.15.31:0/3374020853 shutdown_connections 2023-10-06T02:30:52.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.538+0000 7f17c9f9d700 1 --2- 172.21.15.31:0/3374020853 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17c413fef0 0x7f17c4156950 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:52.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.538+0000 7f17c9f9d700 1 --2- 172.21.15.31:0/3374020853 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f17a80674d0 0x7f17a8069980 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:52.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.538+0000 7f17c9f9d700 1 --2- 172.21.15.31:0/3374020853 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f17c406aaa0 0x7f17c4153cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:52.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.538+0000 7f17c9f9d700 1 --2- 172.21.15.31:0/3374020853 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17c413fb20 0x7f17c4157b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:52.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.538+0000 7f17c9f9d700 1 -- 172.21.15.31:0/3374020853 >> 172.21.15.31:0/3374020853 conn(0x7f17c406d170 msgr2=0x7f17c4155ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:52.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.538+0000 7f17c9f9d700 1 -- 172.21.15.31:0/3374020853 shutdown_connections 2023-10-06T02:30:52.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.538+0000 7f17c9f9d700 1 -- 172.21.15.31:0/3374020853 wait complete. 2023-10-06T02:30:52.557 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:30:52.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.745+0000 7feabdf9d700 1 Processor -- start 2023-10-06T02:30:52.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.745+0000 7feabdf9d700 1 -- start start 2023-10-06T02:30:52.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.747+0000 7feabdf9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feab80b2940 0x7feab80a98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:52.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.747+0000 7feabdf9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7feab80cd9d0 0x7feab806e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:52.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.747+0000 7feabdf9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feab80b0110 con 0x7feab80b2d10 2023-10-06T02:30:52.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.747+0000 7feabdf9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feab80b0920 con 0x7feab80cd9d0 2023-10-06T02:30:52.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.747+0000 7feabdf9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feab806cc70 con 0x7feab80b2940 2023-10-06T02:30:52.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.747+0000 7feabd79c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7feab80b2d10 0x7feab8159250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:47916/0 (socket says 172.21.15.31:47916) 2023-10-06T02:30:52.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.747+0000 7feabd79c700 1 -- 172.21.15.31:0/699821360 learned_addr learned my addr 172.21.15.31:0/699821360 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:30:52.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.747+0000 7feab7fff700 1 --2- 172.21.15.31:0/699821360 >> v2:172.21.15.62:3300/0 conn(0x7feab80cd9d0 0x7feab806e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:52.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.747+0000 7feabcf9b700 1 --2- 172.21.15.31:0/699821360 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feab80b2940 0x7feab80a98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:52.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.747+0000 7feab77fe700 1 -- 172.21.15.31:0/699821360 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4216706690 0 0) 0x7feab80b0110 con 0x7feab80b2d10 2023-10-06T02:30:52.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.748+0000 7feab77fe700 1 -- 172.21.15.31:0/699821360 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feaa8003570 con 0x7feab80b2d10 2023-10-06T02:30:52.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.748+0000 7feab7fff700 1 -- 172.21.15.31:0/699821360 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feab80b2940 msgr2=0x7feab80a98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:52.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.748+0000 7feab7fff700 1 --2- 172.21.15.31:0/699821360 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feab80b2940 0x7feab80a98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:52.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.748+0000 7feab7fff700 1 -- 172.21.15.31:0/699821360 >> v1:172.21.15.31:6789/0 conn(0x7feab80b2d10 legacy=0x7feab8159250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:52.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.748+0000 7feab7fff700 1 -- 172.21.15.31:0/699821360 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feab80b0360 con 0x7feab80cd9d0 2023-10-06T02:30:52.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.748+0000 7feab7fff700 1 --2- 172.21.15.31:0/699821360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feab80cd9d0 0x7feab806e020 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7feaa40098e0 tx=0x7feaa40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1185837e62e211b1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:52.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.749+0000 7feab77fe700 1 -- 172.21.15.31:0/699821360 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa4011630 con 0x7feab80cd9d0 2023-10-06T02:30:52.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.749+0000 7feab77fe700 1 -- 172.21.15.31:0/699821360 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaa4011790 con 0x7feab80cd9d0 2023-10-06T02:30:52.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.749+0000 7feab77fe700 1 -- 172.21.15.31:0/699821360 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa4011b30 con 0x7feab80cd9d0 2023-10-06T02:30:52.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.749+0000 7feabdf9d700 1 -- 172.21.15.31:0/699821360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feab80cd9d0 msgr2=0x7feab806e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:52.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.749+0000 7feabdf9d700 1 --2- 172.21.15.31:0/699821360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feab80cd9d0 0x7feab806e020 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7feaa40098e0 tx=0x7feaa40091a0 comp rx=0 tx=0).stop 2023-10-06T02:30:52.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.749+0000 7feabdf9d700 1 -- 172.21.15.31:0/699821360 shutdown_connections 2023-10-06T02:30:52.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.749+0000 7feabdf9d700 1 --2- 172.21.15.31:0/699821360 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feab80b2940 0x7feab80a98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:52.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.749+0000 7feabdf9d700 1 --2- 172.21.15.31:0/699821360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feab80cd9d0 0x7feab806e020 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:52.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.749+0000 7feabdf9d700 1 -- 172.21.15.31:0/699821360 >> 172.21.15.31:0/699821360 conn(0x7feab80675a0 msgr2=0x7feab80679a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:52.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.749+0000 7feabdf9d700 1 -- 172.21.15.31:0/699821360 shutdown_connections 2023-10-06T02:30:52.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.749+0000 7feabdf9d700 1 -- 172.21.15.31:0/699821360 wait complete. 2023-10-06T02:30:52.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.751+0000 7feabdf9d700 1 Processor -- start 2023-10-06T02:30:52.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.751+0000 7feabdf9d700 1 -- start start 2023-10-06T02:30:52.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.752+0000 7feabdf9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feab80b2940 0x7feab8169680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:52.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.752+0000 7feabdf9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feab80b2d10 0x7feab8144bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:52.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.752+0000 7feabdf9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feab80cd9d0 0x7feab815c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:52.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.752+0000 7feabdf9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab8151a30 con 0x7feab80b2d10 2023-10-06T02:30:52.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.752+0000 7feabdf9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab8150e60 con 0x7feab80b2940 2023-10-06T02:30:52.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.752+0000 7feabdf9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feab8151100 con 0x7feab80cd9d0 2023-10-06T02:30:52.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.752+0000 7feab7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feab80b2d10 0x7feab8144bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:52.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.752+0000 7feab7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feab80b2d10 0x7feab8144bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54744/0 (socket says 172.21.15.31:54744) 2023-10-06T02:30:52.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.752+0000 7feab7fff700 1 -- 172.21.15.31:0/1338509016 learned_addr learned my addr 172.21.15.31:0/1338509016 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:52.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.752+0000 7feabd79c700 1 --2- 172.21.15.31:0/1338509016 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feab80cd9d0 0x7feab815c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:52.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.752+0000 7feabcf9b700 1 --2- 172.21.15.31:0/1338509016 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feab80b2940 0x7feab8169680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:52.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.752+0000 7feab7fff700 1 -- 172.21.15.31:0/1338509016 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feab80cd9d0 msgr2=0x7feab815c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:52.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.752+0000 7feab7fff700 1 --2- 172.21.15.31:0/1338509016 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feab80cd9d0 0x7feab815c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:52.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.752+0000 7feab7fff700 1 -- 172.21.15.31:0/1338509016 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feab80b2940 msgr2=0x7feab8169680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:52.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.752+0000 7feab7fff700 1 --2- 172.21.15.31:0/1338509016 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feab80b2940 0x7feab8169680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:52.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.752+0000 7feab7fff700 1 -- 172.21.15.31:0/1338509016 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaa4009420 con 0x7feab80b2d10 2023-10-06T02:30:52.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.753+0000 7feab7fff700 1 --2- 172.21.15.31:0/1338509016 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feab80b2d10 0x7feab8144bd0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7feaa4009ed0 tx=0x7feaa4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:52.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.753+0000 7feab4ff9700 1 -- 172.21.15.31:0/1338509016 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa4011b10 con 0x7feab80b2d10 2023-10-06T02:30:52.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.753+0000 7feab4ff9700 1 -- 172.21.15.31:0/1338509016 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaa40253f0 con 0x7feab80b2d10 2023-10-06T02:30:52.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.753+0000 7feab4ff9700 1 -- 172.21.15.31:0/1338509016 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa40257b0 con 0x7feab80b2d10 2023-10-06T02:30:52.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.753+0000 7feabdf9d700 1 -- 172.21.15.31:0/1338509016 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feab81513e0 con 0x7feab80b2d10 2023-10-06T02:30:52.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.753+0000 7feabdf9d700 1 -- 172.21.15.31:0/1338509016 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feab8145110 con 0x7feab80b2d10 2023-10-06T02:30:52.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.754+0000 7feabdf9d700 1 -- 172.21.15.31:0/1338509016 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feab8151ba0 con 0x7feab80b2d10 2023-10-06T02:30:52.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.755+0000 7feab4ff9700 1 -- 172.21.15.31:0/1338509016 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7feaa4025ba0 con 0x7feab80b2d10 2023-10-06T02:30:52.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.756+0000 7feab4ff9700 1 --2- 172.21.15.31:0/1338509016 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fea9c0674d0 0x7fea9c069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:52.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.756+0000 7feab4ff9700 1 -- 172.21.15.31:0/1338509016 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6086+0+0 (secure 0 0 0) 0x7feaa40869c0 con 0x7feab80b2d10 2023-10-06T02:30:52.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.756+0000 7feabcf9b700 1 --2- 172.21.15.31:0/1338509016 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fea9c0674d0 0x7fea9c069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:52.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.757+0000 7feabcf9b700 1 --2- 172.21.15.31:0/1338509016 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fea9c0674d0 0x7fea9c069980 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7feaac009c90 tx=0x7feaac008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:52.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.760+0000 7feab4ff9700 1 -- 172.21.15.31:0/1338509016 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7feaa4053200 con 0x7feab80b2d10 2023-10-06T02:30:52.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.979+0000 7feabdf9d700 1 -- 172.21.15.31:0/1338509016 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feab804c3c0 con 0x7feab80b2d10 2023-10-06T02:30:52.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.980+0000 7feab4ff9700 1 -- 172.21.15.31:0/1338509016 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v144) v1 ==== 74+0+16876 (secure 0 0 0) 0x7feaa4056820 con 0x7feab80b2d10 2023-10-06T02:30:52.981 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:30:52.982 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":144,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:30:52.650397+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:30:49.325914+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":24,"pool_name":"cephfs_data","create_time":"2023-10-06T02:30:50.127083+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":20,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:30:52.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.982+0000 7feabdf9d700 1 -- 172.21.15.31:0/1338509016 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fea9c0674d0 msgr2=0x7fea9c069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:52.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.982+0000 7feabdf9d700 1 --2- 172.21.15.31:0/1338509016 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fea9c0674d0 0x7fea9c069980 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7feaac009c90 tx=0x7feaac008040 comp rx=0 tx=0).stop 2023-10-06T02:30:52.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.982+0000 7feabdf9d700 1 -- 172.21.15.31:0/1338509016 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feab80b2d10 msgr2=0x7feab8144bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:52.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.982+0000 7feabdf9d700 1 --2- 172.21.15.31:0/1338509016 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feab80b2d10 0x7feab8144bd0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7feaa4009ed0 tx=0x7feaa4017000 comp rx=0 tx=0).stop 2023-10-06T02:30:52.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.982+0000 7feabdf9d700 1 -- 172.21.15.31:0/1338509016 shutdown_connections 2023-10-06T02:30:52.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.982+0000 7feabdf9d700 1 --2- 172.21.15.31:0/1338509016 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fea9c0674d0 0x7fea9c069980 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:52.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.982+0000 7feabdf9d700 1 --2- 172.21.15.31:0/1338509016 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feab80cd9d0 0x7feab815c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:52.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.982+0000 7feabdf9d700 1 --2- 172.21.15.31:0/1338509016 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feab80b2940 0x7feab8169680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:52.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.982+0000 7feabdf9d700 1 --2- 172.21.15.31:0/1338509016 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feab80b2d10 0x7feab8144bd0 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:52.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.982+0000 7feabdf9d700 1 -- 172.21.15.31:0/1338509016 >> 172.21.15.31:0/1338509016 conn(0x7feab80675a0 msgr2=0x7feab80ab000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:52.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.982+0000 7feabdf9d700 1 -- 172.21.15.31:0/1338509016 shutdown_connections 2023-10-06T02:30:52.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:52.982+0000 7feabdf9d700 1 -- 172.21.15.31:0/1338509016 wait complete. 2023-10-06T02:30:53.002 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T02:30:53.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:53.191+0000 7fec4f59e700 1 Processor -- start 2023-10-06T02:30:53.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:53.191+0000 7fec4f59e700 1 -- start start 2023-10-06T02:30:53.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:53.192+0000 7fec4f59e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fec500adb30 0x7fec500adf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:53.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:53.192+0000 7fec4f59e700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fec500abf10 0x7fec5006de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:53.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:53.192+0000 7fec4f59e700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec500c6700 con 0x7fec5006e3b0 2023-10-06T02:30:53.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:53.192+0000 7fec4f59e700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec5006e830 con 0x7fec500abf10 2023-10-06T02:30:53.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:53.192+0000 7fec4f59e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec500c69a0 con 0x7fec500adb30 2023-10-06T02:30:53.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:53.193+0000 7fec4e59c700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fec500adb30 0x7fec500adf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:53.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:53.193+0000 7fec4dd9b700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fec500abf10 0x7fec5006de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:53.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:53.193+0000 7fec4ed9d700 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fec5006e3b0 0x7fec5006a560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:56170/0 (socket says 172.21.15.31:56170) 2023-10-06T02:30:53.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:53.193+0000 7fec4dd9b700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fec500abf10 0x7fec5006de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:33134/0 (socket says 172.21.15.31:33134) 2023-10-06T02:30:53.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:53.193+0000 7fec4ed9d700 1 -- 172.21.15.31:0/3937747111 learned_addr learned my addr 172.21.15.31:0/3937747111 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:30:53.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:53.194+0000 7fec4dd9b700 1 -- 172.21.15.31:0/3937747111 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fec500adb30 msgr2=0x7fec500adf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:53.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:53.194+0000 7fec4dd9b700 1 --2- 172.21.15.31:0/3937747111 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fec500adb30 0x7fec500adf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:53.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:53.194+0000 7fec4dd9b700 1 -- 172.21.15.31:0/3937747111 >> v1:172.21.15.62:6789/0 conn(0x7fec5006e3b0 legacy=0x7fec5006a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:53.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:53.194+0000 7fec4dd9b700 1 -- 172.21.15.31:0/3937747111 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec500c6c80 con 0x7fec500abf10 2023-10-06T02:30:53.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:53.194+0000 7fec4e59c700 1 --2- 172.21.15.31:0/3937747111 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fec500adb30 0x7fec500adf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:30:53.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:53.194+0000 7fec4d59a700 1 -- 172.21.15.31:0/3937747111 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1182351516 0 0) 0x7fec500c6700 con 0x7fec5006e3b0 2023-10-06T02:30:53.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:53.194+0000 7fec4dd9b700 1 --2- 172.21.15.31:0/3937747111 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec500abf10 0x7fec5006de70 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fec440098e0 tx=0x7fec440091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccebb250d864611a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:53.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:53.194+0000 7fec4d59a700 1 -- 172.21.15.31:0/3937747111 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec4400b640 con 0x7fec500abf10 2023-10-06T02:30:53.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:53.194+0000 7fec4d59a700 1 -- 172.21.15.31:0/3937747111 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec4400b7a0 con 0x7fec500abf10 2023-10-06T02:30:53.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:53.194+0000 7fec4d59a700 1 -- 172.21.15.31:0/3937747111 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec4400bb40 con 0x7fec500abf10 2023-10-06T02:30:53.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:53.195+0000 7fec4f59e700 1 -- 172.21.15.31:0/3937747111 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec500abf10 msgr2=0x7fec5006de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:53.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:53.195+0000 7fec4f59e700 1 --2- 172.21.15.31:0/3937747111 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec500abf10 0x7fec5006de70 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fec440098e0 tx=0x7fec440091a0 comp rx=0 tx=0).stop 2023-10-06T02:30:53.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:53.195+0000 7fec4f59e700 1 -- 172.21.15.31:0/3937747111 shutdown_connections 2023-10-06T02:30:53.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:53.195+0000 7fec4f59e700 1 --2- 172.21.15.31:0/3937747111 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fec500adb30 0x7fec500adf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:53.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:53.195+0000 7fec4f59e700 1 --2- 172.21.15.31:0/3937747111 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec500abf10 0x7fec5006de70 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:53.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:53.195+0000 7fec4f59e700 1 -- 172.21.15.31:0/3937747111 >> 172.21.15.31:0/3937747111 conn(0x7fec500c9600 msgr2=0x7fec500cc0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:53.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:53.195+0000 7fec4f59e700 1 -- 172.21.15.31:0/3937747111 shutdown_connections 2023-10-06T02:30:53.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:53.195+0000 7fec4f59e700 1 -- 172.21.15.31:0/3937747111 wait complete. 2023-10-06T02:30:53.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:53.196+0000 7fec4f59e700 1 Processor -- start 2023-10-06T02:30:53.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:53.196+0000 7fec4f59e700 1 -- start start 2023-10-06T02:30:53.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:53.197+0000 7fec4f59e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec5006e3b0 0x7fec500c59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:53.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:53.197+0000 7fec4f59e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec500abf10 0x7fec5014ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:53.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:53.197+0000 7fec4f59e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fec500adb30 0x7fec500b0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:53.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:53.197+0000 7fec4f59e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec500c5f30 con 0x7fec5006e3b0 2023-10-06T02:30:53.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:53.197+0000 7fec4f59e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec50151a70 con 0x7fec500abf10 2023-10-06T02:30:53.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:53.197+0000 7fec4f59e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec50151d50 con 0x7fec500adb30 2023-10-06T02:30:53.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:53.197+0000 7fec4e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec5006e3b0 0x7fec500c59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:53.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:53.198+0000 7fec4e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec5006e3b0 0x7fec500c59f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54752/0 (socket says 172.21.15.31:54752) 2023-10-06T02:30:53.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:53.198+0000 7fec4e59c700 1 -- 172.21.15.31:0/4238409125 learned_addr learned my addr 172.21.15.31:0/4238409125 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:53.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:53.198+0000 7fec4dd9b700 1 --2- 172.21.15.31:0/4238409125 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec500abf10 0x7fec5014ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:53.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:53.198+0000 7fec4ed9d700 1 --2- 172.21.15.31:0/4238409125 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fec500adb30 0x7fec500b0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:53.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:53.198+0000 7fec4e59c700 1 -- 172.21.15.31:0/4238409125 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fec500adb30 msgr2=0x7fec500b0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:53.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:53.198+0000 7fec4e59c700 1 --2- 172.21.15.31:0/4238409125 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fec500adb30 0x7fec500b0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:53.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:53.198+0000 7fec4e59c700 1 -- 172.21.15.31:0/4238409125 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec500abf10 msgr2=0x7fec5014ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:53.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:53.198+0000 7fec4e59c700 1 --2- 172.21.15.31:0/4238409125 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec500abf10 0x7fec5014ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:53.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:53.198+0000 7fec4e59c700 1 -- 172.21.15.31:0/4238409125 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec40009420 con 0x7fec5006e3b0 2023-10-06T02:30:53.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:53.198+0000 7fec4e59c700 1 --2- 172.21.15.31:0/4238409125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec5006e3b0 0x7fec500c59f0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7fec4000ec80 tx=0x7fec400091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:53.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:53.199+0000 7fec3e7fc700 1 -- 172.21.15.31:0/4238409125 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec4000c920 con 0x7fec5006e3b0 2023-10-06T02:30:53.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:53.199+0000 7fec3e7fc700 1 -- 172.21.15.31:0/4238409125 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec4000ca80 con 0x7fec5006e3b0 2023-10-06T02:30:53.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:53.199+0000 7fec3e7fc700 1 -- 172.21.15.31:0/4238409125 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec4000ce00 con 0x7fec5006e3b0 2023-10-06T02:30:53.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:53.199+0000 7fec4f59e700 1 -- 172.21.15.31:0/4238409125 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec44009420 con 0x7fec5006e3b0 2023-10-06T02:30:53.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:53.199+0000 7fec4f59e700 1 -- 172.21.15.31:0/4238409125 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec50152310 con 0x7fec5006e3b0 2023-10-06T02:30:53.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:53.200+0000 7fec4f59e700 1 -- 172.21.15.31:0/4238409125 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec50068d40 con 0x7fec5006e3b0 2023-10-06T02:30:53.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:53.201+0000 7fec3e7fc700 1 -- 172.21.15.31:0/4238409125 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7fec40013680 con 0x7fec5006e3b0 2023-10-06T02:30:53.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:53.202+0000 7fec3e7fc700 1 --2- 172.21.15.31:0/4238409125 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fec300674d0 0x7fec30069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:53.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:53.202+0000 7fec3e7fc700 1 -- 172.21.15.31:0/4238409125 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fec40085500 con 0x7fec5006e3b0 2023-10-06T02:30:53.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:53.202+0000 7fec4dd9b700 1 --2- 172.21.15.31:0/4238409125 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fec300674d0 0x7fec30069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:53.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:53.203+0000 7fec4dd9b700 1 --2- 172.21.15.31:0/4238409125 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fec300674d0 0x7fec30069980 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fec440093f0 tx=0x7fec44017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:53.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:53.206+0000 7fec3e7fc700 1 -- 172.21.15.31:0/4238409125 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fec40051d80 con 0x7fec5006e3b0 2023-10-06T02:30:53.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:53.453+0000 7fec4f59e700 1 -- 172.21.15.31:0/4238409125 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fec500c6f60 con 0x7fec5006e3b0 2023-10-06T02:30:54.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:54.097+0000 7fec3e7fc700 1 -- 172.21.15.31:0/4238409125 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v99) v1 ==== 118+0+0 (secure 0 0 0) 0x7fec400553a0 con 0x7fec5006e3b0 2023-10-06T02:30:54.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:54.100+0000 7fec4f59e700 1 -- 172.21.15.31:0/4238409125 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fec300674d0 msgr2=0x7fec30069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:54.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:54.100+0000 7fec4f59e700 1 --2- 172.21.15.31:0/4238409125 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fec300674d0 0x7fec30069980 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fec440093f0 tx=0x7fec44017000 comp rx=0 tx=0).stop 2023-10-06T02:30:54.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:54.100+0000 7fec4f59e700 1 -- 172.21.15.31:0/4238409125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec5006e3b0 msgr2=0x7fec500c59f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:54.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:54.100+0000 7fec4f59e700 1 --2- 172.21.15.31:0/4238409125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec5006e3b0 0x7fec500c59f0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7fec4000ec80 tx=0x7fec400091a0 comp rx=0 tx=0).stop 2023-10-06T02:30:54.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:54.100+0000 7fec4f59e700 1 -- 172.21.15.31:0/4238409125 shutdown_connections 2023-10-06T02:30:54.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:54.100+0000 7fec4f59e700 1 --2- 172.21.15.31:0/4238409125 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fec300674d0 0x7fec30069980 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:54.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:54.100+0000 7fec4f59e700 1 --2- 172.21.15.31:0/4238409125 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fec500adb30 0x7fec500b0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:54.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:54.100+0000 7fec4f59e700 1 --2- 172.21.15.31:0/4238409125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec5006e3b0 0x7fec500c59f0 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:54.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:54.100+0000 7fec4f59e700 1 --2- 172.21.15.31:0/4238409125 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec500abf10 0x7fec5014ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:54.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:54.100+0000 7fec4f59e700 1 -- 172.21.15.31:0/4238409125 >> 172.21.15.31:0/4238409125 conn(0x7fec500c9600 msgr2=0x7fec500a9980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:54.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:54.100+0000 7fec4f59e700 1 -- 172.21.15.31:0/4238409125 shutdown_connections 2023-10-06T02:30:54.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:54.100+0000 7fec4f59e700 1 -- 172.21.15.31:0/4238409125 wait complete. 2023-10-06T02:30:54.122 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T02:30:54.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:54.317+0000 7f60cb59e700 1 Processor -- start 2023-10-06T02:30:54.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:54.317+0000 7f60cb59e700 1 -- start start 2023-10-06T02:30:54.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:54.319+0000 7f60cb59e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f60cc06ad00 0x7f60cc06b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:54.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:54.319+0000 7f60cb59e700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f60cc0acbb0 0x7f60cc0ad030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:54.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:54.319+0000 7f60cb59e700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60cc0c8060 con 0x7f60cc156670 2023-10-06T02:30:54.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:54.319+0000 7f60cb59e700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60cc156a40 con 0x7f60cc0acbb0 2023-10-06T02:30:54.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:54.319+0000 7f60cb59e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60cc06b690 con 0x7f60cc06ad00 2023-10-06T02:30:54.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:54.319+0000 7f60ca59c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f60cc156670 0x7f60cc067f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:47920/0 (socket says 172.21.15.31:47920) 2023-10-06T02:30:54.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:54.319+0000 7f60ca59c700 1 -- 172.21.15.31:0/2646899632 learned_addr learned my addr 172.21.15.31:0/2646899632 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:30:54.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:54.319+0000 7f60c9d9b700 1 --2- 172.21.15.31:0/2646899632 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f60cc06ad00 0x7f60cc06b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:54.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:54.319+0000 7f60cad9d700 1 --2- 172.21.15.31:0/2646899632 >> v2:172.21.15.62:3300/0 conn(0x7f60cc0acbb0 0x7f60cc0ad030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:54.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:54.319+0000 7f60c959a700 1 -- 172.21.15.31:0/2646899632 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3884800931 0 0) 0x7f60cc0c8060 con 0x7f60cc156670 2023-10-06T02:30:54.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:54.319+0000 7f60c959a700 1 -- 172.21.15.31:0/2646899632 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f60b4003570 con 0x7f60cc156670 2023-10-06T02:30:54.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:54.319+0000 7f60c9d9b700 1 -- 172.21.15.31:0/2646899632 >> v2:172.21.15.62:3300/0 conn(0x7f60cc0acbb0 msgr2=0x7f60cc0ad030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:54.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:54.319+0000 7f60c9d9b700 1 --2- 172.21.15.31:0/2646899632 >> v2:172.21.15.62:3300/0 conn(0x7f60cc0acbb0 0x7f60cc0ad030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:54.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:54.319+0000 7f60c9d9b700 1 -- 172.21.15.31:0/2646899632 >> v1:172.21.15.31:6789/0 conn(0x7f60cc156670 legacy=0x7f60cc067f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:54.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:54.319+0000 7f60c9d9b700 1 -- 172.21.15.31:0/2646899632 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60cc0abaa0 con 0x7f60cc06ad00 2023-10-06T02:30:54.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:54.319+0000 7f60cad9d700 1 --2- 172.21.15.31:0/2646899632 >> v2:172.21.15.62:3300/0 conn(0x7f60cc0acbb0 0x7f60cc0ad030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:30:54.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:54.319+0000 7f60c9d9b700 1 --2- 172.21.15.31:0/2646899632 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f60cc06ad00 0x7f60cc06b150 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f60c00098e0 tx=0x7f60c00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30f4d0da6da4e42d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:54.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:54.319+0000 7f60c959a700 1 -- 172.21.15.31:0/2646899632 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60c0011630 con 0x7f60cc06ad00 2023-10-06T02:30:54.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:54.319+0000 7f60c959a700 1 -- 172.21.15.31:0/2646899632 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60c0011790 con 0x7f60cc06ad00 2023-10-06T02:30:54.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:54.319+0000 7f60c959a700 1 -- 172.21.15.31:0/2646899632 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60c0011b30 con 0x7f60cc06ad00 2023-10-06T02:30:54.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:54.320+0000 7f60cb59e700 1 -- 172.21.15.31:0/2646899632 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f60cc06ad00 msgr2=0x7f60cc06b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:54.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:54.320+0000 7f60cb59e700 1 --2- 172.21.15.31:0/2646899632 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f60cc06ad00 0x7f60cc06b150 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f60c00098e0 tx=0x7f60c00091a0 comp rx=0 tx=0).stop 2023-10-06T02:30:54.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:54.320+0000 7f60cb59e700 1 -- 172.21.15.31:0/2646899632 shutdown_connections 2023-10-06T02:30:54.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:54.320+0000 7f60cb59e700 1 --2- 172.21.15.31:0/2646899632 >> v2:172.21.15.62:3300/0 conn(0x7f60cc0acbb0 0x7f60cc0ad030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:54.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:54.320+0000 7f60cb59e700 1 --2- 172.21.15.31:0/2646899632 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f60cc06ad00 0x7f60cc06b150 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:54.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:54.320+0000 7f60cb59e700 1 -- 172.21.15.31:0/2646899632 >> 172.21.15.31:0/2646899632 conn(0x7f60cc0c9600 msgr2=0x7f60cc0cc0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:54.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:54.320+0000 7f60cb59e700 1 -- 172.21.15.31:0/2646899632 shutdown_connections 2023-10-06T02:30:54.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:54.320+0000 7f60cb59e700 1 -- 172.21.15.31:0/2646899632 wait complete. 2023-10-06T02:30:54.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:54.323+0000 7f60cb59e700 1 Processor -- start 2023-10-06T02:30:54.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:54.323+0000 7f60cb59e700 1 -- start start 2023-10-06T02:30:54.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:54.324+0000 7f60cb59e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60cc06ad00 0x7f60cc0bf950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:54.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:54.324+0000 7f60cb59e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f60cc0acbb0 0x7f60cc0c1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:54.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:54.324+0000 7f60cb59e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60cc156670 0x7f60cc153640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:54.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:54.324+0000 7f60cb59e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60cc0c0bb0 con 0x7f60cc06ad00 2023-10-06T02:30:54.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:54.324+0000 7f60cb59e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60cc0c2910 con 0x7f60cc156670 2023-10-06T02:30:54.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:54.324+0000 7f60cb59e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60cc0bff30 con 0x7f60cc0acbb0 2023-10-06T02:30:54.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:54.324+0000 7f60ca59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60cc06ad00 0x7f60cc0bf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:54.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:54.324+0000 7f60ca59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60cc06ad00 0x7f60cc0bf950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54758/0 (socket says 172.21.15.31:54758) 2023-10-06T02:30:54.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:54.324+0000 7f60ca59c700 1 -- 172.21.15.31:0/4173884311 learned_addr learned my addr 172.21.15.31:0/4173884311 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:54.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:54.324+0000 7f60cad9d700 1 --2- 172.21.15.31:0/4173884311 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60cc156670 0x7f60cc153640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:54.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:54.324+0000 7f60c9d9b700 1 --2- 172.21.15.31:0/4173884311 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f60cc0acbb0 0x7f60cc0c1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:54.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:54.324+0000 7f60ca59c700 1 -- 172.21.15.31:0/4173884311 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f60cc0acbb0 msgr2=0x7f60cc0c1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:54.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:54.324+0000 7f60ca59c700 1 --2- 172.21.15.31:0/4173884311 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f60cc0acbb0 0x7f60cc0c1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:54.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:54.324+0000 7f60ca59c700 1 -- 172.21.15.31:0/4173884311 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60cc156670 msgr2=0x7f60cc153640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:54.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:54.324+0000 7f60ca59c700 1 --2- 172.21.15.31:0/4173884311 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60cc156670 0x7f60cc153640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:54.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:54.324+0000 7f60ca59c700 1 -- 172.21.15.31:0/4173884311 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60c4009a20 con 0x7f60cc06ad00 2023-10-06T02:30:54.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:54.324+0000 7f60cad9d700 1 --2- 172.21.15.31:0/4173884311 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60cc156670 0x7f60cc153640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:30:54.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:54.324+0000 7f60ca59c700 1 --2- 172.21.15.31:0/4173884311 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60cc06ad00 0x7f60cc0bf950 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f60bc00c590 tx=0x7f60bc007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:54.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:54.324+0000 7f60ba7fc700 1 -- 172.21.15.31:0/4173884311 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60bc005b30 con 0x7f60cc06ad00 2023-10-06T02:30:54.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:54.324+0000 7f60ba7fc700 1 -- 172.21.15.31:0/4173884311 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60bc005cb0 con 0x7f60cc06ad00 2023-10-06T02:30:54.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:54.324+0000 7f60cb59e700 1 -- 172.21.15.31:0/4173884311 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60c0009420 con 0x7f60cc06ad00 2023-10-06T02:30:54.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:54.324+0000 7f60ba7fc700 1 -- 172.21.15.31:0/4173884311 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60bc007320 con 0x7f60cc06ad00 2023-10-06T02:30:54.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:54.324+0000 7f60cb59e700 1 -- 172.21.15.31:0/4173884311 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60cc0c04f0 con 0x7f60cc06ad00 2023-10-06T02:30:54.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:54.325+0000 7f60cb59e700 1 -- 172.21.15.31:0/4173884311 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60cc0c5440 con 0x7f60cc06ad00 2023-10-06T02:30:54.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:54.327+0000 7f60ba7fc700 1 -- 172.21.15.31:0/4173884311 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f60bc01b3b0 con 0x7f60cc06ad00 2023-10-06T02:30:54.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:54.328+0000 7f60ba7fc700 1 --2- 172.21.15.31:0/4173884311 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f60ac0674d0 0x7f60ac069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:54.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:54.328+0000 7f60ba7fc700 1 -- 172.21.15.31:0/4173884311 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f60bc084720 con 0x7f60cc06ad00 2023-10-06T02:30:54.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:54.328+0000 7f60c9d9b700 1 --2- 172.21.15.31:0/4173884311 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f60ac0674d0 0x7f60ac069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:54.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:54.329+0000 7f60c9d9b700 1 --2- 172.21.15.31:0/4173884311 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f60ac0674d0 0x7f60ac069980 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f60c00091e0 tx=0x7f60c0017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:54.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:54.332+0000 7f60ba7fc700 1 -- 172.21.15.31:0/4173884311 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f60bc050f60 con 0x7f60cc06ad00 2023-10-06T02:30:54.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:54.579+0000 7f60cb59e700 1 -- 172.21.15.31:0/4173884311 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f60cc0c7450 con 0x7f60cc06ad00 2023-10-06T02:30:55.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.103+0000 7f60ba7fc700 1 -- 172.21.15.31:0/4173884311 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v100) v1 ==== 122+0+0 (secure 0 0 0) 0x7f60bc054580 con 0x7f60cc06ad00 2023-10-06T02:30:55.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.105+0000 7f60cb59e700 1 -- 172.21.15.31:0/4173884311 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f60ac0674d0 msgr2=0x7f60ac069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:55.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.105+0000 7f60cb59e700 1 --2- 172.21.15.31:0/4173884311 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f60ac0674d0 0x7f60ac069980 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f60c00091e0 tx=0x7f60c0017000 comp rx=0 tx=0).stop 2023-10-06T02:30:55.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.105+0000 7f60cb59e700 1 -- 172.21.15.31:0/4173884311 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60cc06ad00 msgr2=0x7f60cc0bf950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:55.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.105+0000 7f60cb59e700 1 --2- 172.21.15.31:0/4173884311 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60cc06ad00 0x7f60cc0bf950 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f60bc00c590 tx=0x7f60bc007e00 comp rx=0 tx=0).stop 2023-10-06T02:30:55.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.105+0000 7f60cb59e700 1 -- 172.21.15.31:0/4173884311 shutdown_connections 2023-10-06T02:30:55.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.105+0000 7f60cb59e700 1 --2- 172.21.15.31:0/4173884311 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60cc156670 0x7f60cc153640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:55.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.105+0000 7f60cb59e700 1 --2- 172.21.15.31:0/4173884311 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60cc06ad00 0x7f60cc0bf950 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:55.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.105+0000 7f60cb59e700 1 --2- 172.21.15.31:0/4173884311 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f60ac0674d0 0x7f60ac069980 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:55.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.105+0000 7f60cb59e700 1 --2- 172.21.15.31:0/4173884311 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f60cc0acbb0 0x7f60cc0c1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:55.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.105+0000 7f60cb59e700 1 -- 172.21.15.31:0/4173884311 >> 172.21.15.31:0/4173884311 conn(0x7f60cc0c9600 msgr2=0x7f60cc0cb930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:55.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.105+0000 7f60cb59e700 1 -- 172.21.15.31:0/4173884311 shutdown_connections 2023-10-06T02:30:55.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.106+0000 7f60cb59e700 1 -- 172.21.15.31:0/4173884311 wait complete. 2023-10-06T02:30:55.125 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:30:55.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.313+0000 7fa447fff700 1 Processor -- start 2023-10-06T02:30:55.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.313+0000 7fa447fff700 1 -- start start 2023-10-06T02:30:55.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.315+0000 7fa447fff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa4480cbbe0 0x7fa4480aee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:55.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.315+0000 7fa447fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa448156030 0x7fa44806ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:55.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.315+0000 7fa447fff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4480b1f10 con 0x7fa448157570 2023-10-06T02:30:55.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.315+0000 7fa447fff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa448157ad0 con 0x7fa448156030 2023-10-06T02:30:55.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.315+0000 7fa447fff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa448156570 con 0x7fa4480cbbe0 2023-10-06T02:30:55.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.315+0000 7fa4467fc700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa448156030 0x7fa44806ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:55.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.315+0000 7fa4467fc700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa448156030 0x7fa44806ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54762/0 (socket says 172.21.15.31:54762) 2023-10-06T02:30:55.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.315+0000 7fa4467fc700 1 -- 172.21.15.31:0/3656267139 learned_addr learned my addr 172.21.15.31:0/3656267139 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:55.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.315+0000 7fa446ffd700 1 --2- 172.21.15.31:0/3656267139 >> v2:172.21.15.62:3300/0 conn(0x7fa4480cbbe0 0x7fa4480aee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:55.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.315+0000 7fa4467fc700 1 -- 172.21.15.31:0/3656267139 >> v2:172.21.15.62:3300/0 conn(0x7fa4480cbbe0 msgr2=0x7fa4480aee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:55.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.315+0000 7fa4467fc700 1 --2- 172.21.15.31:0/3656267139 >> v2:172.21.15.62:3300/0 conn(0x7fa4480cbbe0 0x7fa4480aee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:55.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.316+0000 7fa4467fc700 1 -- 172.21.15.31:0/3656267139 >> v1:172.21.15.62:6789/0 conn(0x7fa448157570 legacy=0x7fa4480adae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:55.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.316+0000 7fa4467fc700 1 -- 172.21.15.31:0/3656267139 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa448156850 con 0x7fa448156030 2023-10-06T02:30:55.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.316+0000 7fa4467fc700 1 --2- 172.21.15.31:0/3656267139 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa448156030 0x7fa44806ed40 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7fa448067770 tx=0x7fa43c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96edf1b6e37aea42 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:55.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.316+0000 7fa445ffb700 1 -- 172.21.15.31:0/3656267139 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa43c00b860 con 0x7fa448156030 2023-10-06T02:30:55.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.316+0000 7fa445ffb700 1 -- 172.21.15.31:0/3656267139 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa43c00b9c0 con 0x7fa448156030 2023-10-06T02:30:55.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.316+0000 7fa445ffb700 1 -- 172.21.15.31:0/3656267139 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa43c00bd60 con 0x7fa448156030 2023-10-06T02:30:55.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.317+0000 7fa447fff700 1 -- 172.21.15.31:0/3656267139 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa448156030 msgr2=0x7fa44806ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:55.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.317+0000 7fa447fff700 1 --2- 172.21.15.31:0/3656267139 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa448156030 0x7fa44806ed40 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7fa448067770 tx=0x7fa43c009250 comp rx=0 tx=0).stop 2023-10-06T02:30:55.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.317+0000 7fa447fff700 1 -- 172.21.15.31:0/3656267139 shutdown_connections 2023-10-06T02:30:55.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.317+0000 7fa447fff700 1 --2- 172.21.15.31:0/3656267139 >> v2:172.21.15.62:3300/0 conn(0x7fa4480cbbe0 0x7fa4480aee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:55.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.317+0000 7fa447fff700 1 --2- 172.21.15.31:0/3656267139 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa448156030 0x7fa44806ed40 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:55.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.317+0000 7fa447fff700 1 -- 172.21.15.31:0/3656267139 >> 172.21.15.31:0/3656267139 conn(0x7fa44806d170 msgr2=0x7fa4480cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:55.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.317+0000 7fa447fff700 1 -- 172.21.15.31:0/3656267139 shutdown_connections 2023-10-06T02:30:55.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.317+0000 7fa447fff700 1 -- 172.21.15.31:0/3656267139 wait complete. 2023-10-06T02:30:55.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.318+0000 7fa447fff700 1 Processor -- start 2023-10-06T02:30:55.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.318+0000 7fa447fff700 1 -- start start 2023-10-06T02:30:55.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.319+0000 7fa447fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa4480cbbe0 0x7fa4480a98e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:55.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.319+0000 7fa447fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa448156030 0x7fa44815ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:55.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.319+0000 7fa447fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa448157570 0x7fa44814db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:55.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.319+0000 7fa447fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa44815f180 con 0x7fa448156030 2023-10-06T02:30:55.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.319+0000 7fa447fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4480a9fe0 con 0x7fa448157570 2023-10-06T02:30:55.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.319+0000 7fa447fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4480aa2c0 con 0x7fa4480cbbe0 2023-10-06T02:30:55.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.320+0000 7fa4467fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa448156030 0x7fa44815ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:55.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.320+0000 7fa4467fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa448156030 0x7fa44815ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54768/0 (socket says 172.21.15.31:54768) 2023-10-06T02:30:55.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.320+0000 7fa4467fc700 1 -- 172.21.15.31:0/1230048687 learned_addr learned my addr 172.21.15.31:0/1230048687 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:55.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.320+0000 7fa446ffd700 1 --2- 172.21.15.31:0/1230048687 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa4480cbbe0 0x7fa4480a98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:55.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.320+0000 7fa4477fe700 1 --2- 172.21.15.31:0/1230048687 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa448157570 0x7fa44814db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:55.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.320+0000 7fa4467fc700 1 -- 172.21.15.31:0/1230048687 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa4480cbbe0 msgr2=0x7fa4480a98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:55.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.320+0000 7fa4467fc700 1 --2- 172.21.15.31:0/1230048687 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa4480cbbe0 0x7fa4480a98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:55.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.320+0000 7fa4467fc700 1 -- 172.21.15.31:0/1230048687 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa448157570 msgr2=0x7fa44814db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:55.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.320+0000 7fa4467fc700 1 --2- 172.21.15.31:0/1230048687 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa448157570 0x7fa44814db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:55.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.320+0000 7fa4467fc700 1 -- 172.21.15.31:0/1230048687 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa43c0094f0 con 0x7fa448156030 2023-10-06T02:30:55.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.320+0000 7fa4467fc700 1 --2- 172.21.15.31:0/1230048687 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa448156030 0x7fa44815ec40 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7fa4480acfc0 tx=0x7fa43c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:55.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.321+0000 7fa42effd700 1 -- 172.21.15.31:0/1230048687 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa43c024440 con 0x7fa448156030 2023-10-06T02:30:55.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.321+0000 7fa42effd700 1 -- 172.21.15.31:0/1230048687 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa43c0245a0 con 0x7fa448156030 2023-10-06T02:30:55.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.321+0000 7fa447fff700 1 -- 172.21.15.31:0/1230048687 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4480aa5a0 con 0x7fa448156030 2023-10-06T02:30:55.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.321+0000 7fa42effd700 1 -- 172.21.15.31:0/1230048687 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa43c024960 con 0x7fa448156030 2023-10-06T02:30:55.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.321+0000 7fa447fff700 1 -- 172.21.15.31:0/1230048687 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4480aab60 con 0x7fa448156030 2023-10-06T02:30:55.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.322+0000 7fa447fff700 1 -- 172.21.15.31:0/1230048687 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4480aaea0 con 0x7fa448156030 2023-10-06T02:30:55.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.323+0000 7fa42effd700 1 -- 172.21.15.31:0/1230048687 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7fa43c01a420 con 0x7fa448156030 2023-10-06T02:30:55.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.324+0000 7fa42effd700 1 --2- 172.21.15.31:0/1230048687 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa4340674d0 0x7fa434069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:55.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.324+0000 7fa42effd700 1 -- 172.21.15.31:0/1230048687 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fa43c087010 con 0x7fa448156030 2023-10-06T02:30:55.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.324+0000 7fa446ffd700 1 --2- 172.21.15.31:0/1230048687 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa4340674d0 0x7fa434069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:55.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.325+0000 7fa446ffd700 1 --2- 172.21.15.31:0/1230048687 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa4340674d0 0x7fa434069980 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa438006fd0 tx=0x7fa438008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:55.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.328+0000 7fa42effd700 1 -- 172.21.15.31:0/1230048687 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa43c053910 con 0x7fa448156030 2023-10-06T02:30:55.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.574+0000 7fa447fff700 1 -- 172.21.15.31:0/1230048687 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa44804c3c0 con 0x7fa448156030 2023-10-06T02:30:55.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.575+0000 7fa42effd700 1 -- 172.21.15.31:0/1230048687 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) v1 ==== 95+0+4603 (secure 0 0 0) 0x7fa43c056f30 con 0x7fa448156030 2023-10-06T02:30:55.576 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:30:55.576 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":100,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6923,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/4169296779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":4169296779},{"type":"v1","addr":"172.21.15.31:6835","nonce":4169296779}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95},{"gid":6926,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/3032245108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3032245108},{"type":"v1","addr":"172.21.15.62:6834","nonce":3032245108}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95},{"gid":8469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6833/1522769465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1522769465},{"type":"v1","addr":"172.21.15.31:6833","nonce":1522769465}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:30:51.153736+0000","modified":"2023-10-06T02:30:55.096858+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8487":{"gid":8487,"name":"d","rank":0,"incarnation":96,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/2236458886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2236458886},{"type":"v1","addr":"172.21.15.62:6835","nonce":2236458886}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2023-10-06T02:30:55.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.577+0000 7fa447fff700 1 -- 172.21.15.31:0/1230048687 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa4340674d0 msgr2=0x7fa434069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:55.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.577+0000 7fa447fff700 1 --2- 172.21.15.31:0/1230048687 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa4340674d0 0x7fa434069980 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa438006fd0 tx=0x7fa438008040 comp rx=0 tx=0).stop 2023-10-06T02:30:55.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.577+0000 7fa447fff700 1 -- 172.21.15.31:0/1230048687 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa448156030 msgr2=0x7fa44815ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:55.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.577+0000 7fa447fff700 1 --2- 172.21.15.31:0/1230048687 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa448156030 0x7fa44815ec40 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7fa4480acfc0 tx=0x7fa43c017000 comp rx=0 tx=0).stop 2023-10-06T02:30:55.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.577+0000 7fa447fff700 1 -- 172.21.15.31:0/1230048687 shutdown_connections 2023-10-06T02:30:55.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.577+0000 7fa447fff700 1 --2- 172.21.15.31:0/1230048687 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa448157570 0x7fa44814db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:55.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.577+0000 7fa447fff700 1 --2- 172.21.15.31:0/1230048687 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa4340674d0 0x7fa434069980 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:55.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.577+0000 7fa447fff700 1 --2- 172.21.15.31:0/1230048687 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa4480cbbe0 0x7fa4480a98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:55.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.577+0000 7fa447fff700 1 --2- 172.21.15.31:0/1230048687 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa448156030 0x7fa44815ec40 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:55.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.577+0000 7fa447fff700 1 -- 172.21.15.31:0/1230048687 >> 172.21.15.31:0/1230048687 conn(0x7fa44806d170 msgr2=0x7fa44814f960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:55.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.577+0000 7fa447fff700 1 -- 172.21.15.31:0/1230048687 shutdown_connections 2023-10-06T02:30:55.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.577+0000 7fa447fff700 1 -- 172.21.15.31:0/1230048687 wait complete. 2023-10-06T02:30:55.579 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 100 2023-10-06T02:30:55.595 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:30:55.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.782+0000 7fb1bd99d700 1 Processor -- start 2023-10-06T02:30:55.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.782+0000 7fb1bd99d700 1 -- start start 2023-10-06T02:30:55.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.783+0000 7fb1bd99d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb1b806e0a0 0x7fb1b80c6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:55.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.783+0000 7fb1bd99d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb1b806f3a0 0x7fb1b814fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:55.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.783+0000 7fb1bd99d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1b806a490 con 0x7fb1b806e470 2023-10-06T02:30:55.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.783+0000 7fb1bd99d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1b806b930 con 0x7fb1b806e0a0 2023-10-06T02:30:55.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.783+0000 7fb1bd99d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1b80c7490 con 0x7fb1b806f3a0 2023-10-06T02:30:55.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.784+0000 7fb1bc99b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb1b806e0a0 0x7fb1b80c6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:55.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.784+0000 7fb1bc99b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb1b806e0a0 0x7fb1b80c6b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54778/0 (socket says 172.21.15.31:54778) 2023-10-06T02:30:55.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.784+0000 7fb1bc99b700 1 -- 172.21.15.31:0/3670222756 learned_addr learned my addr 172.21.15.31:0/3670222756 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:55.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.784+0000 7fb1b7fff700 1 --2- 172.21.15.31:0/3670222756 >> v2:172.21.15.62:3300/0 conn(0x7fb1b806f3a0 0x7fb1b814fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:55.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.784+0000 7fb1bc99b700 1 -- 172.21.15.31:0/3670222756 >> v2:172.21.15.62:3300/0 conn(0x7fb1b806f3a0 msgr2=0x7fb1b814fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:55.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.784+0000 7fb1bc99b700 1 --2- 172.21.15.31:0/3670222756 >> v2:172.21.15.62:3300/0 conn(0x7fb1b806f3a0 0x7fb1b814fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:55.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.784+0000 7fb1bc99b700 1 -- 172.21.15.31:0/3670222756 >> v1:172.21.15.31:6789/0 conn(0x7fb1b806e470 legacy=0x7fb1b8159740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:55.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.784+0000 7fb1bc99b700 1 -- 172.21.15.31:0/3670222756 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1b80c7770 con 0x7fb1b806e0a0 2023-10-06T02:30:55.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.784+0000 7fb1bc99b700 1 --2- 172.21.15.31:0/3670222756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1b806e0a0 0x7fb1b80c6b20 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8156140 tx=0x7fb1ac00b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dad2392cd423a3d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:55.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.785+0000 7fb1b77fe700 1 -- 172.21.15.31:0/3670222756 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1ac00dba0 con 0x7fb1b806e0a0 2023-10-06T02:30:55.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.785+0000 7fb1b77fe700 1 -- 172.21.15.31:0/3670222756 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1ac00dd00 con 0x7fb1b806e0a0 2023-10-06T02:30:55.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.785+0000 7fb1b77fe700 1 -- 172.21.15.31:0/3670222756 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1ac0196d0 con 0x7fb1b806e0a0 2023-10-06T02:30:55.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.785+0000 7fb1bd99d700 1 -- 172.21.15.31:0/3670222756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1b806e0a0 msgr2=0x7fb1b80c6b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:55.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.785+0000 7fb1bd99d700 1 --2- 172.21.15.31:0/3670222756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1b806e0a0 0x7fb1b80c6b20 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8156140 tx=0x7fb1ac00b4f0 comp rx=0 tx=0).stop 2023-10-06T02:30:55.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.785+0000 7fb1bd99d700 1 -- 172.21.15.31:0/3670222756 shutdown_connections 2023-10-06T02:30:55.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.785+0000 7fb1bd99d700 1 --2- 172.21.15.31:0/3670222756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1b806e0a0 0x7fb1b80c6b20 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:55.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.785+0000 7fb1bd99d700 1 --2- 172.21.15.31:0/3670222756 >> v2:172.21.15.62:3300/0 conn(0x7fb1b806f3a0 0x7fb1b814fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:55.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.785+0000 7fb1bd99d700 1 -- 172.21.15.31:0/3670222756 >> 172.21.15.31:0/3670222756 conn(0x7fb1b806d170 msgr2=0x7fb1b80cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:55.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.785+0000 7fb1bd99d700 1 -- 172.21.15.31:0/3670222756 shutdown_connections 2023-10-06T02:30:55.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.785+0000 7fb1bd99d700 1 -- 172.21.15.31:0/3670222756 wait complete. 2023-10-06T02:30:55.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.787+0000 7fb1bd99d700 1 Processor -- start 2023-10-06T02:30:55.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.787+0000 7fb1bd99d700 1 -- start start 2023-10-06T02:30:55.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.788+0000 7fb1bd99d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb1b806e0a0 0x7fb1b8149210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:55.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.788+0000 7fb1bd99d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1b806e470 0x7fb1b8153e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:55.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.788+0000 7fb1bd99d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1b806f3a0 0x7fb1b80cc180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:55.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.788+0000 7fb1bd99d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b8155120 con 0x7fb1b806f3a0 2023-10-06T02:30:55.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.788+0000 7fb1bd99d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b8154390 con 0x7fb1b806e470 2023-10-06T02:30:55.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.788+0000 7fb1bd99d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b81545a0 con 0x7fb1b806e0a0 2023-10-06T02:30:55.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.788+0000 7fb1bd19c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1b806f3a0 0x7fb1b80cc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:55.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.788+0000 7fb1bd19c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1b806f3a0 0x7fb1b80cc180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54782/0 (socket says 172.21.15.31:54782) 2023-10-06T02:30:55.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.788+0000 7fb1bd19c700 1 -- 172.21.15.31:0/1883605643 learned_addr learned my addr 172.21.15.31:0/1883605643 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:55.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.788+0000 7fb1bc99b700 1 --2- 172.21.15.31:0/1883605643 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb1b806e0a0 0x7fb1b8149210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:55.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.788+0000 7fb1b7fff700 1 --2- 172.21.15.31:0/1883605643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1b806e470 0x7fb1b8153e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:55.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.789+0000 7fb1bd19c700 1 -- 172.21.15.31:0/1883605643 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb1b806e0a0 msgr2=0x7fb1b8149210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:55.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.789+0000 7fb1bd19c700 1 --2- 172.21.15.31:0/1883605643 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb1b806e0a0 0x7fb1b8149210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:55.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.789+0000 7fb1bd19c700 1 -- 172.21.15.31:0/1883605643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1b806e470 msgr2=0x7fb1b8153e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:55.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.789+0000 7fb1bd19c700 1 --2- 172.21.15.31:0/1883605643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1b806e470 0x7fb1b8153e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:55.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.789+0000 7fb1bd19c700 1 -- 172.21.15.31:0/1883605643 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1ac00b790 con 0x7fb1b806f3a0 2023-10-06T02:30:55.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.789+0000 7fb1bd19c700 1 --2- 172.21.15.31:0/1883605643 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1b806f3a0 0x7fb1b80cc180 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7fb1b000c8e0 tx=0x7fb1b0001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:55.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.789+0000 7fb1b4ff9700 1 -- 172.21.15.31:0/1883605643 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b0012630 con 0x7fb1b806f3a0 2023-10-06T02:30:55.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.789+0000 7fb1b4ff9700 1 -- 172.21.15.31:0/1883605643 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1b0012790 con 0x7fb1b806f3a0 2023-10-06T02:30:55.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.789+0000 7fb1b4ff9700 1 -- 172.21.15.31:0/1883605643 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b0012b50 con 0x7fb1b806f3a0 2023-10-06T02:30:55.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.789+0000 7fb1bd99d700 1 -- 172.21.15.31:0/1883605643 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1b8154880 con 0x7fb1b806f3a0 2023-10-06T02:30:55.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.790+0000 7fb1bd99d700 1 -- 172.21.15.31:0/1883605643 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1b8154e40 con 0x7fb1b806f3a0 2023-10-06T02:30:55.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.790+0000 7fb1bd99d700 1 -- 172.21.15.31:0/1883605643 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1b804c3c0 con 0x7fb1b806f3a0 2023-10-06T02:30:55.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.791+0000 7fb1b4ff9700 1 -- 172.21.15.31:0/1883605643 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7fb1b001a3a0 con 0x7fb1b806f3a0 2023-10-06T02:30:55.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.792+0000 7fb1b4ff9700 1 --2- 172.21.15.31:0/1883605643 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb19c0674d0 0x7fb19c069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:55.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.792+0000 7fb1b4ff9700 1 -- 172.21.15.31:0/1883605643 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fb1b00852d0 con 0x7fb1b806f3a0 2023-10-06T02:30:55.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.793+0000 7fb1bc99b700 1 --2- 172.21.15.31:0/1883605643 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb19c0674d0 0x7fb19c069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:55.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.793+0000 7fb1bc99b700 1 --2- 172.21.15.31:0/1883605643 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb19c0674d0 0x7fb19c069980 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8156140 tx=0x7fb1ac01b000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:55.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.796+0000 7fb1b4ff9700 1 -- 172.21.15.31:0/1883605643 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb1b00555a0 con 0x7fb1b806f3a0 2023-10-06T02:30:56.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:55.999+0000 7fb1bd99d700 1 -- 172.21.15.31:0/1883605643 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb1b80c7a50 con 0x7fb1b806f3a0 2023-10-06T02:30:56.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.000+0000 7fb1b4ff9700 1 -- 172.21.15.31:0/1883605643 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v144) v1 ==== 74+0+16876 (secure 0 0 0) 0x7fb1b0055130 con 0x7fb1b806f3a0 2023-10-06T02:30:56.001 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:30:56.001 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":144,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:30:52.650397+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:30:49.325914+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":24,"pool_name":"cephfs_data","create_time":"2023-10-06T02:30:50.127083+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":20,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:30:56.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.002+0000 7fb1bd99d700 1 -- 172.21.15.31:0/1883605643 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb19c0674d0 msgr2=0x7fb19c069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:56.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.002+0000 7fb1bd99d700 1 --2- 172.21.15.31:0/1883605643 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb19c0674d0 0x7fb19c069980 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8156140 tx=0x7fb1ac01b000 comp rx=0 tx=0).stop 2023-10-06T02:30:56.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.003+0000 7fb1bd99d700 1 -- 172.21.15.31:0/1883605643 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1b806f3a0 msgr2=0x7fb1b80cc180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:56.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.003+0000 7fb1bd99d700 1 --2- 172.21.15.31:0/1883605643 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1b806f3a0 0x7fb1b80cc180 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7fb1b000c8e0 tx=0x7fb1b0001710 comp rx=0 tx=0).stop 2023-10-06T02:30:56.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.003+0000 7fb1bd99d700 1 -- 172.21.15.31:0/1883605643 shutdown_connections 2023-10-06T02:30:56.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.003+0000 7fb1bd99d700 1 --2- 172.21.15.31:0/1883605643 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1b806f3a0 0x7fb1b80cc180 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:56.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.003+0000 7fb1bd99d700 1 --2- 172.21.15.31:0/1883605643 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb19c0674d0 0x7fb19c069980 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:56.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.003+0000 7fb1bd99d700 1 --2- 172.21.15.31:0/1883605643 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb1b806e0a0 0x7fb1b8149210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:56.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.003+0000 7fb1bd99d700 1 --2- 172.21.15.31:0/1883605643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1b806e470 0x7fb1b8153e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:56.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.003+0000 7fb1bd99d700 1 -- 172.21.15.31:0/1883605643 >> 172.21.15.31:0/1883605643 conn(0x7fb1b806d170 msgr2=0x7fb1b80ca730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:56.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.003+0000 7fb1bd99d700 1 -- 172.21.15.31:0/1883605643 shutdown_connections 2023-10-06T02:30:56.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.003+0000 7fb1bd99d700 1 -- 172.21.15.31:0/1883605643 wait complete. 2023-10-06T02:30:56.021 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T02:30:56.021 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T02:30:56.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.208+0000 7f86a49fe700 1 Processor -- start 2023-10-06T02:30:56.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.208+0000 7f86a49fe700 1 -- start start 2023-10-06T02:30:56.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.210+0000 7f86a49fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f86a006e0a0 0x7f86a00c6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:56.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.210+0000 7f86a49fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f86a0150380 0x7f86a006a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:56.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.210+0000 7f86a49fe700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86a006a490 con 0x7f86a006f3a0 2023-10-06T02:30:56.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.210+0000 7f86a49fe700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86a00c7490 con 0x7f86a006e0a0 2023-10-06T02:30:56.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.210+0000 7f86a49fe700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86a00c7770 con 0x7f86a0150380 2023-10-06T02:30:56.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.210+0000 7f869ed9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f86a006e0a0 0x7f86a00c6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:56.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.210+0000 7f869ed9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f86a006e0a0 0x7f86a00c6b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54798/0 (socket says 172.21.15.31:54798) 2023-10-06T02:30:56.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.210+0000 7f869ed9d700 1 -- 172.21.15.31:0/4184377653 learned_addr learned my addr 172.21.15.31:0/4184377653 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:56.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.210+0000 7f869f59e700 1 --2- 172.21.15.31:0/4184377653 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f86a0150380 0x7f86a006a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:56.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.211+0000 7f869ed9d700 1 -- 172.21.15.31:0/4184377653 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f86a0150380 msgr2=0x7f86a006a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:56.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.211+0000 7f869ed9d700 1 --2- 172.21.15.31:0/4184377653 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f86a0150380 0x7f86a006a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:56.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.211+0000 7f869ed9d700 1 -- 172.21.15.31:0/4184377653 >> v1:172.21.15.31:6789/0 conn(0x7f86a006f3a0 legacy=0x7f86a014fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:56.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.211+0000 7f869ed9d700 1 -- 172.21.15.31:0/4184377653 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86a00c7a50 con 0x7f86a006e0a0 2023-10-06T02:30:56.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.211+0000 7f869ed9d700 1 --2- 172.21.15.31:0/4184377653 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86a006e0a0 0x7f86a00c6b20 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f869000bc40 tx=0x7f869000b4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43008cc3a6320656 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:56.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.211+0000 7f869dd9b700 1 -- 172.21.15.31:0/4184377653 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f869000da90 con 0x7f86a006e0a0 2023-10-06T02:30:56.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.211+0000 7f869dd9b700 1 -- 172.21.15.31:0/4184377653 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f869000dbf0 con 0x7f86a006e0a0 2023-10-06T02:30:56.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.211+0000 7f869dd9b700 1 -- 172.21.15.31:0/4184377653 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8690017410 con 0x7f86a006e0a0 2023-10-06T02:30:56.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.212+0000 7f86a49fe700 1 -- 172.21.15.31:0/4184377653 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86a006e0a0 msgr2=0x7f86a00c6b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:56.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.212+0000 7f86a49fe700 1 --2- 172.21.15.31:0/4184377653 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86a006e0a0 0x7f86a00c6b20 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f869000bc40 tx=0x7f869000b4e0 comp rx=0 tx=0).stop 2023-10-06T02:30:56.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.212+0000 7f86a49fe700 1 -- 172.21.15.31:0/4184377653 shutdown_connections 2023-10-06T02:30:56.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.212+0000 7f86a49fe700 1 --2- 172.21.15.31:0/4184377653 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86a006e0a0 0x7f86a00c6b20 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:56.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.212+0000 7f86a49fe700 1 --2- 172.21.15.31:0/4184377653 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f86a0150380 0x7f86a006a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:56.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.212+0000 7f86a49fe700 1 -- 172.21.15.31:0/4184377653 >> 172.21.15.31:0/4184377653 conn(0x7f86a006d170 msgr2=0x7f86a00cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:56.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.212+0000 7f86a49fe700 1 -- 172.21.15.31:0/4184377653 shutdown_connections 2023-10-06T02:30:56.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.212+0000 7f86a49fe700 1 -- 172.21.15.31:0/4184377653 wait complete. 2023-10-06T02:30:56.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.213+0000 7f86a49fe700 1 Processor -- start 2023-10-06T02:30:56.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.214+0000 7f86a49fe700 1 -- start start 2023-10-06T02:30:56.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.214+0000 7f86a49fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86a006e0a0 0x7f86a0149cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:56.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.214+0000 7f86a49fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86a006f3a0 0x7f86a01698a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:56.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.214+0000 7f86a49fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f86a0150380 0x7f86a00b0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:56.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.215+0000 7f86a49fe700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86a0148710 con 0x7f86a006f3a0 2023-10-06T02:30:56.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.215+0000 7f86a49fe700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86a01489b0 con 0x7f86a006e0a0 2023-10-06T02:30:56.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.215+0000 7f86a49fe700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86a016a100 con 0x7f86a0150380 2023-10-06T02:30:56.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.215+0000 7f869e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86a006f3a0 0x7f86a01698a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:56.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.215+0000 7f869e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86a006f3a0 0x7f86a01698a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54800/0 (socket says 172.21.15.31:54800) 2023-10-06T02:30:56.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.215+0000 7f869e59c700 1 -- 172.21.15.31:0/3036425877 learned_addr learned my addr 172.21.15.31:0/3036425877 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:56.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.215+0000 7f869ed9d700 1 --2- 172.21.15.31:0/3036425877 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86a006e0a0 0x7f86a0149cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:56.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.215+0000 7f869f59e700 1 --2- 172.21.15.31:0/3036425877 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f86a0150380 0x7f86a00b0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:56.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.215+0000 7f869e59c700 1 -- 172.21.15.31:0/3036425877 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f86a0150380 msgr2=0x7f86a00b0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:56.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.215+0000 7f869e59c700 1 --2- 172.21.15.31:0/3036425877 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f86a0150380 0x7f86a00b0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:56.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.215+0000 7f869e59c700 1 -- 172.21.15.31:0/3036425877 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86a006e0a0 msgr2=0x7f86a0149cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:56.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.215+0000 7f869e59c700 1 --2- 172.21.15.31:0/3036425877 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86a006e0a0 0x7f86a0149cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:56.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.215+0000 7f869e59c700 1 -- 172.21.15.31:0/3036425877 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f869000b780 con 0x7f86a006f3a0 2023-10-06T02:30:56.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.215+0000 7f869f59e700 1 --2- 172.21.15.31:0/3036425877 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f86a0150380 0x7f86a00b0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:30:56.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.215+0000 7f869e59c700 1 --2- 172.21.15.31:0/3036425877 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86a006f3a0 0x7f86a01698a0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f869400c7c0 tx=0x7f8694007f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:56.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.216+0000 7f8686ffd700 1 -- 172.21.15.31:0/3036425877 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f869400e4a0 con 0x7f86a006f3a0 2023-10-06T02:30:56.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.216+0000 7f8686ffd700 1 -- 172.21.15.31:0/3036425877 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f869400e620 con 0x7f86a006f3a0 2023-10-06T02:30:56.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.216+0000 7f86a49fe700 1 -- 172.21.15.31:0/3036425877 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86a016a3e0 con 0x7f86a006f3a0 2023-10-06T02:30:56.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.216+0000 7f8686ffd700 1 -- 172.21.15.31:0/3036425877 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f869400e9e0 con 0x7f86a006f3a0 2023-10-06T02:30:56.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.216+0000 7f86a49fe700 1 -- 172.21.15.31:0/3036425877 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86a0169de0 con 0x7f86a006f3a0 2023-10-06T02:30:56.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.217+0000 7f86a49fe700 1 -- 172.21.15.31:0/3036425877 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86a0148b60 con 0x7f86a006f3a0 2023-10-06T02:30:56.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.218+0000 7f8686ffd700 1 -- 172.21.15.31:0/3036425877 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f869401b3b0 con 0x7f86a006f3a0 2023-10-06T02:30:56.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.219+0000 7f8686ffd700 1 --2- 172.21.15.31:0/3036425877 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f868c0674d0 0x7f868c069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:56.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.219+0000 7f8686ffd700 1 -- 172.21.15.31:0/3036425877 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f86940846a0 con 0x7f86a006f3a0 2023-10-06T02:30:56.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.219+0000 7f869ed9d700 1 --2- 172.21.15.31:0/3036425877 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f868c0674d0 0x7f868c069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:56.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.220+0000 7f869ed9d700 1 --2- 172.21.15.31:0/3036425877 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f868c0674d0 0x7f868c069980 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f869000b520 tx=0x7f8690019000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:56.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.223+0000 7f8686ffd700 1 -- 172.21.15.31:0/3036425877 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f86a0148b60 con 0x7f86a006f3a0 2023-10-06T02:30:56.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.424+0000 7f86a49fe700 1 -- 172.21.15.31:0/3036425877 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f86a0148b20 con 0x7f868c0674d0 2023-10-06T02:30:56.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.434+0000 7f8686ffd700 1 -- 172.21.15.31:0/3036425877 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311352 (secure 0 0 0) 0x7f86a0148b20 con 0x7f868c0674d0 2023-10-06T02:30:56.438 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:30:56.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.438+0000 7f86a49fe700 1 -- 172.21.15.31:0/3036425877 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f868c0674d0 msgr2=0x7f868c069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:56.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.438+0000 7f86a49fe700 1 --2- 172.21.15.31:0/3036425877 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f868c0674d0 0x7f868c069980 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f869000b520 tx=0x7f8690019000 comp rx=0 tx=0).stop 2023-10-06T02:30:56.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.438+0000 7f86a49fe700 1 -- 172.21.15.31:0/3036425877 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86a006f3a0 msgr2=0x7f86a01698a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:56.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.438+0000 7f86a49fe700 1 --2- 172.21.15.31:0/3036425877 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86a006f3a0 0x7f86a01698a0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f869400c7c0 tx=0x7f8694007f00 comp rx=0 tx=0).stop 2023-10-06T02:30:56.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.438+0000 7f86a49fe700 1 -- 172.21.15.31:0/3036425877 shutdown_connections 2023-10-06T02:30:56.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.438+0000 7f86a49fe700 1 --2- 172.21.15.31:0/3036425877 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f868c0674d0 0x7f868c069980 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:56.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.438+0000 7f86a49fe700 1 --2- 172.21.15.31:0/3036425877 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f86a0150380 0x7f86a00b0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:56.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.438+0000 7f86a49fe700 1 --2- 172.21.15.31:0/3036425877 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86a006e0a0 0x7f86a0149cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:56.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.438+0000 7f86a49fe700 1 --2- 172.21.15.31:0/3036425877 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86a006f3a0 0x7f86a01698a0 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:56.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.438+0000 7f86a49fe700 1 -- 172.21.15.31:0/3036425877 >> 172.21.15.31:0/3036425877 conn(0x7f86a006d170 msgr2=0x7f86a014ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:56.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.438+0000 7f86a49fe700 1 -- 172.21.15.31:0/3036425877 shutdown_connections 2023-10-06T02:30:56.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.438+0000 7f86a49fe700 1 -- 172.21.15.31:0/3036425877 wait complete. 2023-10-06T02:30:56.442 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-10-06T02:30:56.457 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":170,"stamp":"2023-10-06T02:30:54.503483+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":223,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":166,"ondisk_log_size":166,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":169,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2426404,"kb_used_data":15332,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752548316,"statfs":{"total":773094113280,"available":770609475584,"internally_reserved":0,"allocated":15699968,"data_stored":7920389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.855962"},"pg_stats":[{"pgid":"24.3f","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.677664+0000","last_change":"2023-10-06T02:30:51.677664+0000","last_active":"2023-10-06T02:30:51.677664+0000","last_peered":"2023-10-06T02:30:51.677664+0000","last_clean":"2023-10-06T02:30:51.677664+0000","last_became_active":"2023-10-06T02:30:51.674070+0000","last_became_peered":"2023-10-06T02:30:51.674070+0000","last_unstale":"2023-10-06T02:30:51.677664+0000","last_undegraded":"2023-10-06T02:30:51.677664+0000","last_fullsized":"2023-10-06T02:30:51.677664+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:15:10.732838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3e","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.680336+0000","last_change":"2023-10-06T02:30:51.675712+0000","last_active":"2023-10-06T02:30:51.680336+0000","last_peered":"2023-10-06T02:30:51.680336+0000","last_clean":"2023-10-06T02:30:51.680336+0000","last_became_active":"2023-10-06T02:30:51.675164+0000","last_became_peered":"2023-10-06T02:30:51.675164+0000","last_unstale":"2023-10-06T02:30:51.680336+0000","last_undegraded":"2023-10-06T02:30:51.680336+0000","last_fullsized":"2023-10-06T02:30:51.680336+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:15:14.213699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3d","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.682966+0000","last_change":"2023-10-06T02:30:51.682966+0000","last_active":"2023-10-06T02:30:51.682966+0000","last_peered":"2023-10-06T02:30:51.682966+0000","last_clean":"2023-10-06T02:30:51.682966+0000","last_became_active":"2023-10-06T02:30:51.682495+0000","last_became_peered":"2023-10-06T02:30:51.682495+0000","last_unstale":"2023-10-06T02:30:51.682966+0000","last_undegraded":"2023-10-06T02:30:51.682966+0000","last_fullsized":"2023-10-06T02:30:51.682966+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:27:26.681762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3c","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.677717+0000","last_change":"2023-10-06T02:30:51.677717+0000","last_active":"2023-10-06T02:30:51.677717+0000","last_peered":"2023-10-06T02:30:51.677717+0000","last_clean":"2023-10-06T02:30:51.677717+0000","last_became_active":"2023-10-06T02:30:51.677415+0000","last_became_peered":"2023-10-06T02:30:51.677415+0000","last_unstale":"2023-10-06T02:30:51.677717+0000","last_undegraded":"2023-10-06T02:30:51.677717+0000","last_fullsized":"2023-10-06T02:30:51.677717+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:13:59.000364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3b","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.671702+0000","last_change":"2023-10-06T02:30:51.669101+0000","last_active":"2023-10-06T02:30:51.671702+0000","last_peered":"2023-10-06T02:30:51.671702+0000","last_clean":"2023-10-06T02:30:51.671702+0000","last_became_active":"2023-10-06T02:30:51.668833+0000","last_became_peered":"2023-10-06T02:30:51.668833+0000","last_unstale":"2023-10-06T02:30:51.671702+0000","last_undegraded":"2023-10-06T02:30:51.671702+0000","last_fullsized":"2023-10-06T02:30:51.671702+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:56:38.263976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3a","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.675549+0000","last_change":"2023-10-06T02:30:51.670883+0000","last_active":"2023-10-06T02:30:51.675549+0000","last_peered":"2023-10-06T02:30:51.675549+0000","last_clean":"2023-10-06T02:30:51.675549+0000","last_became_active":"2023-10-06T02:30:51.670589+0000","last_became_peered":"2023-10-06T02:30:51.670589+0000","last_unstale":"2023-10-06T02:30:51.675549+0000","last_undegraded":"2023-10-06T02:30:51.675549+0000","last_fullsized":"2023-10-06T02:30:51.675549+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:15:15.716156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.39","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.672548+0000","last_change":"2023-10-06T02:30:51.665493+0000","last_active":"2023-10-06T02:30:51.672548+0000","last_peered":"2023-10-06T02:30:51.672548+0000","last_clean":"2023-10-06T02:30:51.672548+0000","last_became_active":"2023-10-06T02:30:51.664664+0000","last_became_peered":"2023-10-06T02:30:51.664664+0000","last_unstale":"2023-10-06T02:30:51.672548+0000","last_undegraded":"2023-10-06T02:30:51.672548+0000","last_fullsized":"2023-10-06T02:30:51.672548+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:04:59.566737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.38","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.675824+0000","last_change":"2023-10-06T02:30:51.670371+0000","last_active":"2023-10-06T02:30:51.675824+0000","last_peered":"2023-10-06T02:30:51.675824+0000","last_clean":"2023-10-06T02:30:51.675824+0000","last_became_active":"2023-10-06T02:30:51.669441+0000","last_became_peered":"2023-10-06T02:30:51.669441+0000","last_unstale":"2023-10-06T02:30:51.675824+0000","last_undegraded":"2023-10-06T02:30:51.675824+0000","last_fullsized":"2023-10-06T02:30:51.675824+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:50:09.735917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.37","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.675714+0000","last_change":"2023-10-06T02:30:51.670562+0000","last_active":"2023-10-06T02:30:51.675714+0000","last_peered":"2023-10-06T02:30:51.675714+0000","last_clean":"2023-10-06T02:30:51.675714+0000","last_became_active":"2023-10-06T02:30:51.670274+0000","last_became_peered":"2023-10-06T02:30:51.670274+0000","last_unstale":"2023-10-06T02:30:51.675714+0000","last_undegraded":"2023-10-06T02:30:51.675714+0000","last_fullsized":"2023-10-06T02:30:51.675714+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:15:10.732838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.36","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.676073+0000","last_change":"2023-10-06T02:30:51.673118+0000","last_active":"2023-10-06T02:30:51.676073+0000","last_peered":"2023-10-06T02:30:51.676073+0000","last_clean":"2023-10-06T02:30:51.676073+0000","last_became_active":"2023-10-06T02:30:51.672802+0000","last_became_peered":"2023-10-06T02:30:51.672802+0000","last_unstale":"2023-10-06T02:30:51.676073+0000","last_undegraded":"2023-10-06T02:30:51.676073+0000","last_fullsized":"2023-10-06T02:30:51.676073+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:13:59.000364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.35","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.677497+0000","last_change":"2023-10-06T02:30:51.673626+0000","last_active":"2023-10-06T02:30:51.677497+0000","last_peered":"2023-10-06T02:30:51.677497+0000","last_clean":"2023-10-06T02:30:51.677497+0000","last_became_active":"2023-10-06T02:30:51.673136+0000","last_became_peered":"2023-10-06T02:30:51.673136+0000","last_unstale":"2023-10-06T02:30:51.677497+0000","last_undegraded":"2023-10-06T02:30:51.677497+0000","last_fullsized":"2023-10-06T02:30:51.677497+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:20:07.242511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.34","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.677007+0000","last_change":"2023-10-06T02:30:51.671300+0000","last_active":"2023-10-06T02:30:51.677007+0000","last_peered":"2023-10-06T02:30:51.677007+0000","last_clean":"2023-10-06T02:30:51.677007+0000","last_became_active":"2023-10-06T02:30:51.670975+0000","last_became_peered":"2023-10-06T02:30:51.670975+0000","last_unstale":"2023-10-06T02:30:51.677007+0000","last_undegraded":"2023-10-06T02:30:51.677007+0000","last_fullsized":"2023-10-06T02:30:51.677007+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:21:59.543623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.33","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.674045+0000","last_change":"2023-10-06T02:30:51.674045+0000","last_active":"2023-10-06T02:30:51.674045+0000","last_peered":"2023-10-06T02:30:51.674045+0000","last_clean":"2023-10-06T02:30:51.674045+0000","last_became_active":"2023-10-06T02:30:51.673496+0000","last_became_peered":"2023-10-06T02:30:51.673496+0000","last_unstale":"2023-10-06T02:30:51.674045+0000","last_undegraded":"2023-10-06T02:30:51.674045+0000","last_fullsized":"2023-10-06T02:30:51.674045+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:22:16.120421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.32","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.670659+0000","last_change":"2023-10-06T02:30:51.666903+0000","last_active":"2023-10-06T02:30:51.670659+0000","last_peered":"2023-10-06T02:30:51.670659+0000","last_clean":"2023-10-06T02:30:51.670659+0000","last_became_active":"2023-10-06T02:30:51.666639+0000","last_became_peered":"2023-10-06T02:30:51.666639+0000","last_unstale":"2023-10-06T02:30:51.670659+0000","last_undegraded":"2023-10-06T02:30:51.670659+0000","last_fullsized":"2023-10-06T02:30:51.670659+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:50:51.184941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.31","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.674353+0000","last_change":"2023-10-06T02:30:51.662058+0000","last_active":"2023-10-06T02:30:51.674353+0000","last_peered":"2023-10-06T02:30:51.674353+0000","last_clean":"2023-10-06T02:30:51.674353+0000","last_became_active":"2023-10-06T02:30:51.660270+0000","last_became_peered":"2023-10-06T02:30:51.660270+0000","last_unstale":"2023-10-06T02:30:51.674353+0000","last_undegraded":"2023-10-06T02:30:51.674353+0000","last_fullsized":"2023-10-06T02:30:51.674353+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:04:04.504486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.30","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.679630+0000","last_change":"2023-10-06T02:30:51.679630+0000","last_active":"2023-10-06T02:30:51.679630+0000","last_peered":"2023-10-06T02:30:51.679630+0000","last_clean":"2023-10-06T02:30:51.679630+0000","last_became_active":"2023-10-06T02:30:51.678736+0000","last_became_peered":"2023-10-06T02:30:51.678736+0000","last_unstale":"2023-10-06T02:30:51.679630+0000","last_undegraded":"2023-10-06T02:30:51.679630+0000","last_fullsized":"2023-10-06T02:30:51.679630+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:39:22.478358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.2f","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.670682+0000","last_change":"2023-10-06T02:30:51.659731+0000","last_active":"2023-10-06T02:30:51.670682+0000","last_peered":"2023-10-06T02:30:51.670682+0000","last_clean":"2023-10-06T02:30:51.670682+0000","last_became_active":"2023-10-06T02:30:51.659420+0000","last_became_peered":"2023-10-06T02:30:51.659420+0000","last_unstale":"2023-10-06T02:30:51.670682+0000","last_undegraded":"2023-10-06T02:30:51.670682+0000","last_fullsized":"2023-10-06T02:30:51.670682+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:27:54.865078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2e","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.677114+0000","last_change":"2023-10-06T02:30:51.675616+0000","last_active":"2023-10-06T02:30:51.677114+0000","last_peered":"2023-10-06T02:30:51.677114+0000","last_clean":"2023-10-06T02:30:51.677114+0000","last_became_active":"2023-10-06T02:30:51.675266+0000","last_became_peered":"2023-10-06T02:30:51.675266+0000","last_unstale":"2023-10-06T02:30:51.677114+0000","last_undegraded":"2023-10-06T02:30:51.677114+0000","last_fullsized":"2023-10-06T02:30:51.677114+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:12:24.760338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.2d","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.673493+0000","last_change":"2023-10-06T02:30:51.661855+0000","last_active":"2023-10-06T02:30:51.673493+0000","last_peered":"2023-10-06T02:30:51.673493+0000","last_clean":"2023-10-06T02:30:51.673493+0000","last_became_active":"2023-10-06T02:30:51.660147+0000","last_became_peered":"2023-10-06T02:30:51.660147+0000","last_unstale":"2023-10-06T02:30:51.673493+0000","last_undegraded":"2023-10-06T02:30:51.673493+0000","last_fullsized":"2023-10-06T02:30:51.673493+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:46:15.913987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.2c","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.681982+0000","last_change":"2023-10-06T02:30:51.673971+0000","last_active":"2023-10-06T02:30:51.681982+0000","last_peered":"2023-10-06T02:30:51.681982+0000","last_clean":"2023-10-06T02:30:51.681982+0000","last_became_active":"2023-10-06T02:30:51.673710+0000","last_became_peered":"2023-10-06T02:30:51.673710+0000","last_unstale":"2023-10-06T02:30:51.681982+0000","last_undegraded":"2023-10-06T02:30:51.681982+0000","last_fullsized":"2023-10-06T02:30:51.681982+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:50:37.585095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.2b","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.673815+0000","last_change":"2023-10-06T02:30:51.673815+0000","last_active":"2023-10-06T02:30:51.673815+0000","last_peered":"2023-10-06T02:30:51.673815+0000","last_clean":"2023-10-06T02:30:51.673815+0000","last_became_active":"2023-10-06T02:30:51.673116+0000","last_became_peered":"2023-10-06T02:30:51.673116+0000","last_unstale":"2023-10-06T02:30:51.673815+0000","last_undegraded":"2023-10-06T02:30:51.673815+0000","last_fullsized":"2023-10-06T02:30:51.673815+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:24:36.962341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2a","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.675529+0000","last_change":"2023-10-06T02:30:51.669380+0000","last_active":"2023-10-06T02:30:51.675529+0000","last_peered":"2023-10-06T02:30:51.675529+0000","last_clean":"2023-10-06T02:30:51.675529+0000","last_became_active":"2023-10-06T02:30:51.669075+0000","last_became_peered":"2023-10-06T02:30:51.669075+0000","last_unstale":"2023-10-06T02:30:51.675529+0000","last_undegraded":"2023-10-06T02:30:51.675529+0000","last_fullsized":"2023-10-06T02:30:51.675529+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:53:10.241998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.29","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.676976+0000","last_change":"2023-10-06T02:30:51.671846+0000","last_active":"2023-10-06T02:30:51.676976+0000","last_peered":"2023-10-06T02:30:51.676976+0000","last_clean":"2023-10-06T02:30:51.676976+0000","last_became_active":"2023-10-06T02:30:51.671563+0000","last_became_peered":"2023-10-06T02:30:51.671563+0000","last_unstale":"2023-10-06T02:30:51.676976+0000","last_undegraded":"2023-10-06T02:30:51.676976+0000","last_fullsized":"2023-10-06T02:30:51.676976+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:01:17.186989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.28","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.679466+0000","last_change":"2023-10-06T02:30:51.679466+0000","last_active":"2023-10-06T02:30:51.679466+0000","last_peered":"2023-10-06T02:30:51.679466+0000","last_clean":"2023-10-06T02:30:51.679466+0000","last_became_active":"2023-10-06T02:30:51.678760+0000","last_became_peered":"2023-10-06T02:30:51.678760+0000","last_unstale":"2023-10-06T02:30:51.679466+0000","last_undegraded":"2023-10-06T02:30:51.679466+0000","last_fullsized":"2023-10-06T02:30:51.679466+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:45:16.352909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.27","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.674321+0000","last_change":"2023-10-06T02:30:51.674321+0000","last_active":"2023-10-06T02:30:51.674321+0000","last_peered":"2023-10-06T02:30:51.674321+0000","last_clean":"2023-10-06T02:30:51.674321+0000","last_became_active":"2023-10-06T02:30:51.674038+0000","last_became_peered":"2023-10-06T02:30:51.674038+0000","last_unstale":"2023-10-06T02:30:51.674321+0000","last_undegraded":"2023-10-06T02:30:51.674321+0000","last_fullsized":"2023-10-06T02:30:51.674321+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:20:18.598786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.26","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.681757+0000","last_change":"2023-10-06T02:30:51.669654+0000","last_active":"2023-10-06T02:30:51.681757+0000","last_peered":"2023-10-06T02:30:51.681757+0000","last_clean":"2023-10-06T02:30:51.681757+0000","last_became_active":"2023-10-06T02:30:51.669314+0000","last_became_peered":"2023-10-06T02:30:51.669314+0000","last_unstale":"2023-10-06T02:30:51.681757+0000","last_undegraded":"2023-10-06T02:30:51.681757+0000","last_fullsized":"2023-10-06T02:30:51.681757+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:30:16.718924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.25","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.680380+0000","last_change":"2023-10-06T02:30:51.675681+0000","last_active":"2023-10-06T02:30:51.680380+0000","last_peered":"2023-10-06T02:30:51.680380+0000","last_clean":"2023-10-06T02:30:51.680380+0000","last_became_active":"2023-10-06T02:30:51.672584+0000","last_became_peered":"2023-10-06T02:30:51.672584+0000","last_unstale":"2023-10-06T02:30:51.680380+0000","last_undegraded":"2023-10-06T02:30:51.680380+0000","last_fullsized":"2023-10-06T02:30:51.680380+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:08:49.141270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.24","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.672631+0000","last_change":"2023-10-06T02:30:51.672631+0000","last_active":"2023-10-06T02:30:51.672631+0000","last_peered":"2023-10-06T02:30:51.672631+0000","last_clean":"2023-10-06T02:30:51.672631+0000","last_became_active":"2023-10-06T02:30:51.672353+0000","last_became_peered":"2023-10-06T02:30:51.672353+0000","last_unstale":"2023-10-06T02:30:51.672631+0000","last_undegraded":"2023-10-06T02:30:51.672631+0000","last_fullsized":"2023-10-06T02:30:51.672631+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:45:54.859704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.23","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.680329+0000","last_change":"2023-10-06T02:30:51.676512+0000","last_active":"2023-10-06T02:30:51.680329+0000","last_peered":"2023-10-06T02:30:51.680329+0000","last_clean":"2023-10-06T02:30:51.680329+0000","last_became_active":"2023-10-06T02:30:51.676213+0000","last_became_peered":"2023-10-06T02:30:51.676213+0000","last_unstale":"2023-10-06T02:30:51.680329+0000","last_undegraded":"2023-10-06T02:30:51.680329+0000","last_fullsized":"2023-10-06T02:30:51.680329+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:06:59.430925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.22","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.679708+0000","last_change":"2023-10-06T02:30:51.678340+0000","last_active":"2023-10-06T02:30:51.679708+0000","last_peered":"2023-10-06T02:30:51.679708+0000","last_clean":"2023-10-06T02:30:51.679708+0000","last_became_active":"2023-10-06T02:30:51.678056+0000","last_became_peered":"2023-10-06T02:30:51.678056+0000","last_unstale":"2023-10-06T02:30:51.679708+0000","last_undegraded":"2023-10-06T02:30:51.679708+0000","last_fullsized":"2023-10-06T02:30:51.679708+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:50:37.585095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.21","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.677539+0000","last_change":"2023-10-06T02:30:51.674249+0000","last_active":"2023-10-06T02:30:51.677539+0000","last_peered":"2023-10-06T02:30:51.677539+0000","last_clean":"2023-10-06T02:30:51.677539+0000","last_became_active":"2023-10-06T02:30:51.673942+0000","last_became_peered":"2023-10-06T02:30:51.673942+0000","last_unstale":"2023-10-06T02:30:51.677539+0000","last_undegraded":"2023-10-06T02:30:51.677539+0000","last_fullsized":"2023-10-06T02:30:51.677539+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:00:18.060628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.20","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.682678+0000","last_change":"2023-10-06T02:30:51.681496+0000","last_active":"2023-10-06T02:30:51.682678+0000","last_peered":"2023-10-06T02:30:51.682678+0000","last_clean":"2023-10-06T02:30:51.682678+0000","last_became_active":"2023-10-06T02:30:51.681157+0000","last_became_peered":"2023-10-06T02:30:51.681157+0000","last_unstale":"2023-10-06T02:30:51.682678+0000","last_undegraded":"2023-10-06T02:30:51.682678+0000","last_fullsized":"2023-10-06T02:30:51.682678+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:52:30.458923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.677131+0000","last_change":"2023-10-06T02:30:51.675021+0000","last_active":"2023-10-06T02:30:51.677131+0000","last_peered":"2023-10-06T02:30:51.677131+0000","last_clean":"2023-10-06T02:30:51.677131+0000","last_became_active":"2023-10-06T02:30:51.674717+0000","last_became_peered":"2023-10-06T02:30:51.674717+0000","last_unstale":"2023-10-06T02:30:51.677131+0000","last_undegraded":"2023-10-06T02:30:51.677131+0000","last_fullsized":"2023-10-06T02:30:51.677131+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:34:36.018098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"143'4","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:52.094943+0000","last_change":"2023-10-06T02:30:50.674860+0000","last_active":"2023-10-06T02:30:52.094943+0000","last_peered":"2023-10-06T02:30:52.094943+0000","last_clean":"2023-10-06T02:30:52.094943+0000","last_became_active":"2023-10-06T02:30:50.674477+0000","last_became_peered":"2023-10-06T02:30:50.674477+0000","last_unstale":"2023-10-06T02:30:52.094943+0000","last_undegraded":"2023-10-06T02:30:52.094943+0000","last_fullsized":"2023-10-06T02:30:52.094943+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_clean_scrub_stamp":"2023-10-06T02:30:49.637381+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:54:31.897719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.671148+0000","last_change":"2023-10-06T02:30:51.671148+0000","last_active":"2023-10-06T02:30:51.671148+0000","last_peered":"2023-10-06T02:30:51.671148+0000","last_clean":"2023-10-06T02:30:51.671148+0000","last_became_active":"2023-10-06T02:30:51.670838+0000","last_became_peered":"2023-10-06T02:30:51.670838+0000","last_unstale":"2023-10-06T02:30:51.671148+0000","last_undegraded":"2023-10-06T02:30:51.671148+0000","last_fullsized":"2023-10-06T02:30:51.671148+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:19:08.717427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"143'1","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:52.099776+0000","last_change":"2023-10-06T02:30:50.679429+0000","last_active":"2023-10-06T02:30:52.099776+0000","last_peered":"2023-10-06T02:30:52.099776+0000","last_clean":"2023-10-06T02:30:52.099776+0000","last_became_active":"2023-10-06T02:30:50.679166+0000","last_became_peered":"2023-10-06T02:30:50.679166+0000","last_unstale":"2023-10-06T02:30:52.099776+0000","last_undegraded":"2023-10-06T02:30:52.099776+0000","last_fullsized":"2023-10-06T02:30:52.099776+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_clean_scrub_stamp":"2023-10-06T02:30:49.637381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:40:20.143016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.671598+0000","last_change":"2023-10-06T02:30:51.667929+0000","last_active":"2023-10-06T02:30:51.671598+0000","last_peered":"2023-10-06T02:30:51.671598+0000","last_clean":"2023-10-06T02:30:51.671598+0000","last_became_active":"2023-10-06T02:30:51.667670+0000","last_became_peered":"2023-10-06T02:30:51.667670+0000","last_unstale":"2023-10-06T02:30:51.671598+0000","last_undegraded":"2023-10-06T02:30:51.671598+0000","last_fullsized":"2023-10-06T02:30:51.671598+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:45:51.919765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1d","version":"143'2","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:52.096386+0000","last_change":"2023-10-06T02:30:50.672392+0000","last_active":"2023-10-06T02:30:52.096386+0000","last_peered":"2023-10-06T02:30:52.096386+0000","last_clean":"2023-10-06T02:30:52.096386+0000","last_became_active":"2023-10-06T02:30:50.672091+0000","last_became_peered":"2023-10-06T02:30:50.672091+0000","last_unstale":"2023-10-06T02:30:52.096386+0000","last_undegraded":"2023-10-06T02:30:52.096386+0000","last_fullsized":"2023-10-06T02:30:52.096386+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_clean_scrub_stamp":"2023-10-06T02:30:49.637381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:43:26.111876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.677895+0000","last_change":"2023-10-06T02:30:51.674762+0000","last_active":"2023-10-06T02:30:51.677895+0000","last_peered":"2023-10-06T02:30:51.677895+0000","last_clean":"2023-10-06T02:30:51.677895+0000","last_became_active":"2023-10-06T02:30:51.672125+0000","last_became_peered":"2023-10-06T02:30:51.672125+0000","last_unstale":"2023-10-06T02:30:51.677895+0000","last_undegraded":"2023-10-06T02:30:51.677895+0000","last_fullsized":"2023-10-06T02:30:51.677895+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:24:42.744456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1c","version":"143'2","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:52.096776+0000","last_change":"2023-10-06T02:30:50.666494+0000","last_active":"2023-10-06T02:30:52.096776+0000","last_peered":"2023-10-06T02:30:52.096776+0000","last_clean":"2023-10-06T02:30:52.096776+0000","last_became_active":"2023-10-06T02:30:50.666203+0000","last_became_peered":"2023-10-06T02:30:50.666203+0000","last_unstale":"2023-10-06T02:30:52.096776+0000","last_undegraded":"2023-10-06T02:30:52.096776+0000","last_fullsized":"2023-10-06T02:30:52.096776+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_clean_scrub_stamp":"2023-10-06T02:30:49.637381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:37:54.167657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.675779+0000","last_change":"2023-10-06T02:30:51.672331+0000","last_active":"2023-10-06T02:30:51.675779+0000","last_peered":"2023-10-06T02:30:51.675779+0000","last_clean":"2023-10-06T02:30:51.675779+0000","last_became_active":"2023-10-06T02:30:51.671546+0000","last_became_peered":"2023-10-06T02:30:51.671546+0000","last_unstale":"2023-10-06T02:30:51.675779+0000","last_undegraded":"2023-10-06T02:30:51.675779+0000","last_fullsized":"2023-10-06T02:30:51.675779+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:39:22.478358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1b","version":"143'1","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:52.100083+0000","last_change":"2023-10-06T02:30:50.682246+0000","last_active":"2023-10-06T02:30:52.100083+0000","last_peered":"2023-10-06T02:30:52.100083+0000","last_clean":"2023-10-06T02:30:52.100083+0000","last_became_active":"2023-10-06T02:30:50.681989+0000","last_became_peered":"2023-10-06T02:30:50.681989+0000","last_unstale":"2023-10-06T02:30:52.100083+0000","last_undegraded":"2023-10-06T02:30:52.100083+0000","last_fullsized":"2023-10-06T02:30:52.100083+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_clean_scrub_stamp":"2023-10-06T02:30:49.637381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:17:34.046237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.681994+0000","last_change":"2023-10-06T02:30:51.674180+0000","last_active":"2023-10-06T02:30:51.681994+0000","last_peered":"2023-10-06T02:30:51.681994+0000","last_clean":"2023-10-06T02:30:51.681994+0000","last_became_active":"2023-10-06T02:30:51.673893+0000","last_became_peered":"2023-10-06T02:30:51.673893+0000","last_unstale":"2023-10-06T02:30:51.681994+0000","last_undegraded":"2023-10-06T02:30:51.681994+0000","last_fullsized":"2023-10-06T02:30:51.681994+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:20:18.788778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1a","version":"143'1","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:52.099856+0000","last_change":"2023-10-06T02:30:50.674496+0000","last_active":"2023-10-06T02:30:52.099856+0000","last_peered":"2023-10-06T02:30:52.099856+0000","last_clean":"2023-10-06T02:30:52.099856+0000","last_became_active":"2023-10-06T02:30:50.674199+0000","last_became_peered":"2023-10-06T02:30:50.674199+0000","last_unstale":"2023-10-06T02:30:52.099856+0000","last_undegraded":"2023-10-06T02:30:52.099856+0000","last_fullsized":"2023-10-06T02:30:52.099856+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_clean_scrub_stamp":"2023-10-06T02:30:49.637381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:35:54.781726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.681909+0000","last_change":"2023-10-06T02:30:51.671717+0000","last_active":"2023-10-06T02:30:51.681909+0000","last_peered":"2023-10-06T02:30:51.681909+0000","last_clean":"2023-10-06T02:30:51.681909+0000","last_became_active":"2023-10-06T02:30:51.671434+0000","last_became_peered":"2023-10-06T02:30:51.671434+0000","last_unstale":"2023-10-06T02:30:51.681909+0000","last_undegraded":"2023-10-06T02:30:51.681909+0000","last_fullsized":"2023-10-06T02:30:51.681909+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:40:16.172585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.662466+0000","last_change":"2023-10-06T02:30:50.675638+0000","last_active":"2023-10-06T02:30:51.662466+0000","last_peered":"2023-10-06T02:30:51.662466+0000","last_clean":"2023-10-06T02:30:51.662466+0000","last_became_active":"2023-10-06T02:30:50.675374+0000","last_became_peered":"2023-10-06T02:30:50.675374+0000","last_unstale":"2023-10-06T02:30:51.662466+0000","last_undegraded":"2023-10-06T02:30:51.662466+0000","last_fullsized":"2023-10-06T02:30:51.662466+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_clean_scrub_stamp":"2023-10-06T02:30:49.637381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:27:05.582947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.678936+0000","last_change":"2023-10-06T02:30:51.678936+0000","last_active":"2023-10-06T02:30:51.678936+0000","last_peered":"2023-10-06T02:30:51.678936+0000","last_clean":"2023-10-06T02:30:51.678936+0000","last_became_active":"2023-10-06T02:30:51.678595+0000","last_became_peered":"2023-10-06T02:30:51.678595+0000","last_unstale":"2023-10-06T02:30:51.678936+0000","last_undegraded":"2023-10-06T02:30:51.678936+0000","last_fullsized":"2023-10-06T02:30:51.678936+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:27:54.865078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.656356+0000","last_change":"2023-10-06T02:30:50.680339+0000","last_active":"2023-10-06T02:30:51.656356+0000","last_peered":"2023-10-06T02:30:51.656356+0000","last_clean":"2023-10-06T02:30:51.656356+0000","last_became_active":"2023-10-06T02:30:50.680067+0000","last_became_peered":"2023-10-06T02:30:50.680067+0000","last_unstale":"2023-10-06T02:30:51.656356+0000","last_undegraded":"2023-10-06T02:30:51.656356+0000","last_fullsized":"2023-10-06T02:30:51.656356+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_clean_scrub_stamp":"2023-10-06T02:30:49.637381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:05:09.020108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.672520+0000","last_change":"2023-10-06T02:30:51.665202+0000","last_active":"2023-10-06T02:30:51.672520+0000","last_peered":"2023-10-06T02:30:51.672520+0000","last_clean":"2023-10-06T02:30:51.672520+0000","last_became_active":"2023-10-06T02:30:51.664129+0000","last_became_peered":"2023-10-06T02:30:51.664129+0000","last_unstale":"2023-10-06T02:30:51.672520+0000","last_undegraded":"2023-10-06T02:30:51.672520+0000","last_fullsized":"2023-10-06T02:30:51.672520+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:27:26.681762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"143'1","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.664309+0000","last_change":"2023-10-06T02:30:50.674863+0000","last_active":"2023-10-06T02:30:51.664309+0000","last_peered":"2023-10-06T02:30:51.664309+0000","last_clean":"2023-10-06T02:30:51.664309+0000","last_became_active":"2023-10-06T02:30:50.674542+0000","last_became_peered":"2023-10-06T02:30:50.674542+0000","last_unstale":"2023-10-06T02:30:51.664309+0000","last_undegraded":"2023-10-06T02:30:51.664309+0000","last_fullsized":"2023-10-06T02:30:51.664309+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_clean_scrub_stamp":"2023-10-06T02:30:49.637381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:06:58.421543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.684384+0000","last_change":"2023-10-06T02:30:51.681564+0000","last_active":"2023-10-06T02:30:51.684384+0000","last_peered":"2023-10-06T02:30:51.684384+0000","last_clean":"2023-10-06T02:30:51.684384+0000","last_became_active":"2023-10-06T02:30:51.680841+0000","last_became_peered":"2023-10-06T02:30:51.680841+0000","last_unstale":"2023-10-06T02:30:51.684384+0000","last_undegraded":"2023-10-06T02:30:51.684384+0000","last_fullsized":"2023-10-06T02:30:51.684384+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:30:16.718924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.e","version":"143'2","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:52.100357+0000","last_change":"2023-10-06T02:30:50.684634+0000","last_active":"2023-10-06T02:30:52.100357+0000","last_peered":"2023-10-06T02:30:52.100357+0000","last_clean":"2023-10-06T02:30:52.100357+0000","last_became_active":"2023-10-06T02:30:50.684238+0000","last_became_peered":"2023-10-06T02:30:50.684238+0000","last_unstale":"2023-10-06T02:30:52.100357+0000","last_undegraded":"2023-10-06T02:30:52.100357+0000","last_fullsized":"2023-10-06T02:30:52.100357+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_clean_scrub_stamp":"2023-10-06T02:30:49.637381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:54:59.387855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.677303+0000","last_change":"2023-10-06T02:30:51.672736+0000","last_active":"2023-10-06T02:30:51.677303+0000","last_peered":"2023-10-06T02:30:51.677303+0000","last_clean":"2023-10-06T02:30:51.677303+0000","last_became_active":"2023-10-06T02:30:51.672477+0000","last_became_peered":"2023-10-06T02:30:51.672477+0000","last_unstale":"2023-10-06T02:30:51.677303+0000","last_undegraded":"2023-10-06T02:30:51.677303+0000","last_fullsized":"2023-10-06T02:30:51.677303+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:54:10.422135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.d","version":"143'1","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:52.098816+0000","last_change":"2023-10-06T02:30:50.672218+0000","last_active":"2023-10-06T02:30:52.098816+0000","last_peered":"2023-10-06T02:30:52.098816+0000","last_clean":"2023-10-06T02:30:52.098816+0000","last_became_active":"2023-10-06T02:30:50.671901+0000","last_became_peered":"2023-10-06T02:30:50.671901+0000","last_unstale":"2023-10-06T02:30:52.098816+0000","last_undegraded":"2023-10-06T02:30:52.098816+0000","last_fullsized":"2023-10-06T02:30:52.098816+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_clean_scrub_stamp":"2023-10-06T02:30:49.637381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:06:58.421543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.674838+0000","last_change":"2023-10-06T02:30:51.671644+0000","last_active":"2023-10-06T02:30:51.674838+0000","last_peered":"2023-10-06T02:30:51.674838+0000","last_clean":"2023-10-06T02:30:51.674838+0000","last_became_active":"2023-10-06T02:30:51.671363+0000","last_became_peered":"2023-10-06T02:30:51.671363+0000","last_unstale":"2023-10-06T02:30:51.674838+0000","last_undegraded":"2023-10-06T02:30:51.674838+0000","last_fullsized":"2023-10-06T02:30:51.674838+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:08:16.365846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.657952+0000","last_change":"2023-10-06T02:30:50.682089+0000","last_active":"2023-10-06T02:30:51.657952+0000","last_peered":"2023-10-06T02:30:51.657952+0000","last_clean":"2023-10-06T02:30:51.657952+0000","last_became_active":"2023-10-06T02:30:50.681804+0000","last_became_peered":"2023-10-06T02:30:50.681804+0000","last_unstale":"2023-10-06T02:30:51.657952+0000","last_undegraded":"2023-10-06T02:30:51.657952+0000","last_fullsized":"2023-10-06T02:30:51.657952+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_clean_scrub_stamp":"2023-10-06T02:30:49.637381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:11:52.013827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.672003+0000","last_change":"2023-10-06T02:30:51.672003+0000","last_active":"2023-10-06T02:30:51.672003+0000","last_peered":"2023-10-06T02:30:51.672003+0000","last_clean":"2023-10-06T02:30:51.672003+0000","last_became_active":"2023-10-06T02:30:51.671755+0000","last_became_peered":"2023-10-06T02:30:51.671755+0000","last_unstale":"2023-10-06T02:30:51.672003+0000","last_undegraded":"2023-10-06T02:30:51.672003+0000","last_fullsized":"2023-10-06T02:30:51.672003+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:01:17.186989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.b","version":"143'2","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:52.097268+0000","last_change":"2023-10-06T02:30:50.671943+0000","last_active":"2023-10-06T02:30:52.097268+0000","last_peered":"2023-10-06T02:30:52.097268+0000","last_clean":"2023-10-06T02:30:52.097268+0000","last_became_active":"2023-10-06T02:30:50.671424+0000","last_became_peered":"2023-10-06T02:30:50.671424+0000","last_unstale":"2023-10-06T02:30:52.097268+0000","last_undegraded":"2023-10-06T02:30:52.097268+0000","last_fullsized":"2023-10-06T02:30:52.097268+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_clean_scrub_stamp":"2023-10-06T02:30:49.637381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:24:41.735075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.662205+0000","last_change":"2023-10-06T02:30:51.660403+0000","last_active":"2023-10-06T02:30:51.662205+0000","last_peered":"2023-10-06T02:30:51.662205+0000","last_clean":"2023-10-06T02:30:51.662205+0000","last_became_active":"2023-10-06T02:30:51.660081+0000","last_became_peered":"2023-10-06T02:30:51.660081+0000","last_unstale":"2023-10-06T02:30:51.662205+0000","last_undegraded":"2023-10-06T02:30:51.662205+0000","last_fullsized":"2023-10-06T02:30:51.662205+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:45:54.859704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.655128+0000","last_change":"2023-10-06T02:30:50.665481+0000","last_active":"2023-10-06T02:30:51.655128+0000","last_peered":"2023-10-06T02:30:51.655128+0000","last_clean":"2023-10-06T02:30:51.655128+0000","last_became_active":"2023-10-06T02:30:50.665189+0000","last_became_peered":"2023-10-06T02:30:50.665189+0000","last_unstale":"2023-10-06T02:30:51.655128+0000","last_undegraded":"2023-10-06T02:30:51.655128+0000","last_fullsized":"2023-10-06T02:30:51.655128+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_clean_scrub_stamp":"2023-10-06T02:30:49.637381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:40:16.665661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.670757+0000","last_change":"2023-10-06T02:30:51.664848+0000","last_active":"2023-10-06T02:30:51.670757+0000","last_peered":"2023-10-06T02:30:51.670757+0000","last_clean":"2023-10-06T02:30:51.670757+0000","last_became_active":"2023-10-06T02:30:51.664589+0000","last_became_peered":"2023-10-06T02:30:51.664589+0000","last_unstale":"2023-10-06T02:30:51.670757+0000","last_undegraded":"2023-10-06T02:30:51.670757+0000","last_fullsized":"2023-10-06T02:30:51.670757+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:20:18.598786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.662575+0000","last_change":"2023-10-06T02:30:50.667806+0000","last_active":"2023-10-06T02:30:51.662575+0000","last_peered":"2023-10-06T02:30:51.662575+0000","last_clean":"2023-10-06T02:30:51.662575+0000","last_became_active":"2023-10-06T02:30:50.667526+0000","last_became_peered":"2023-10-06T02:30:50.667526+0000","last_unstale":"2023-10-06T02:30:51.662575+0000","last_undegraded":"2023-10-06T02:30:51.662575+0000","last_fullsized":"2023-10-06T02:30:51.662575+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_clean_scrub_stamp":"2023-10-06T02:30:49.637381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:49:34.881059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.683354+0000","last_change":"2023-10-06T02:30:51.679150+0000","last_active":"2023-10-06T02:30:51.683354+0000","last_peered":"2023-10-06T02:30:51.683354+0000","last_clean":"2023-10-06T02:30:51.683354+0000","last_became_active":"2023-10-06T02:30:51.678870+0000","last_became_peered":"2023-10-06T02:30:51.678870+0000","last_unstale":"2023-10-06T02:30:51.683354+0000","last_undegraded":"2023-10-06T02:30:51.683354+0000","last_fullsized":"2023-10-06T02:30:51.683354+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:29:12.818783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.663318+0000","last_change":"2023-10-06T02:30:50.671628+0000","last_active":"2023-10-06T02:30:51.663318+0000","last_peered":"2023-10-06T02:30:51.663318+0000","last_clean":"2023-10-06T02:30:51.663318+0000","last_became_active":"2023-10-06T02:30:50.671300+0000","last_became_peered":"2023-10-06T02:30:50.671300+0000","last_unstale":"2023-10-06T02:30:51.663318+0000","last_undegraded":"2023-10-06T02:30:51.663318+0000","last_fullsized":"2023-10-06T02:30:51.663318+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_clean_scrub_stamp":"2023-10-06T02:30:49.637381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:39:21.468976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.674803+0000","last_change":"2023-10-06T02:30:51.671977+0000","last_active":"2023-10-06T02:30:51.674803+0000","last_peered":"2023-10-06T02:30:51.674803+0000","last_clean":"2023-10-06T02:30:51.674803+0000","last_became_active":"2023-10-06T02:30:51.671663+0000","last_became_peered":"2023-10-06T02:30:51.671663+0000","last_unstale":"2023-10-06T02:30:51.674803+0000","last_undegraded":"2023-10-06T02:30:51.674803+0000","last_fullsized":"2023-10-06T02:30:51.674803+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:04:59.566737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.7","version":"143'2","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:52.097289+0000","last_change":"2023-10-06T02:30:50.671032+0000","last_active":"2023-10-06T02:30:52.097289+0000","last_peered":"2023-10-06T02:30:52.097289+0000","last_clean":"2023-10-06T02:30:52.097289+0000","last_became_active":"2023-10-06T02:30:50.670761+0000","last_became_peered":"2023-10-06T02:30:50.670761+0000","last_unstale":"2023-10-06T02:30:52.097289+0000","last_undegraded":"2023-10-06T02:30:52.097289+0000","last_fullsized":"2023-10-06T02:30:52.097289+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_clean_scrub_stamp":"2023-10-06T02:30:49.637381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:26:19.617860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.672567+0000","last_change":"2023-10-06T02:30:51.665661+0000","last_active":"2023-10-06T02:30:51.672567+0000","last_peered":"2023-10-06T02:30:51.672567+0000","last_clean":"2023-10-06T02:30:51.672567+0000","last_became_active":"2023-10-06T02:30:51.664972+0000","last_became_peered":"2023-10-06T02:30:51.664972+0000","last_unstale":"2023-10-06T02:30:51.672567+0000","last_undegraded":"2023-10-06T02:30:51.672567+0000","last_fullsized":"2023-10-06T02:30:51.672567+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:08:16.365846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T02:30:50.683680+0000","last_change":"2023-10-06T02:30:50.678983+0000","last_active":"2023-10-06T02:30:50.683680+0000","last_peered":"2023-10-06T02:30:50.683680+0000","last_clean":"2023-10-06T02:30:50.683680+0000","last_became_active":"2023-10-06T02:30:50.678549+0000","last_became_peered":"2023-10-06T02:30:50.678549+0000","last_unstale":"2023-10-06T02:30:50.683680+0000","last_undegraded":"2023-10-06T02:30:50.683680+0000","last_fullsized":"2023-10-06T02:30:50.683680+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_clean_scrub_stamp":"2023-10-06T02:30:49.637381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:34:35.008716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.681955+0000","last_change":"2023-10-06T02:30:51.673424+0000","last_active":"2023-10-06T02:30:51.681955+0000","last_peered":"2023-10-06T02:30:51.681955+0000","last_clean":"2023-10-06T02:30:51.681955+0000","last_became_active":"2023-10-06T02:30:51.673159+0000","last_became_peered":"2023-10-06T02:30:51.673159+0000","last_unstale":"2023-10-06T02:30:51.681955+0000","last_undegraded":"2023-10-06T02:30:51.681955+0000","last_fullsized":"2023-10-06T02:30:51.681955+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:43:15.672912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T02:30:50.673429+0000","last_change":"2023-10-06T02:30:50.672021+0000","last_active":"2023-10-06T02:30:50.673429+0000","last_peered":"2023-10-06T02:30:50.673429+0000","last_clean":"2023-10-06T02:30:50.673429+0000","last_became_active":"2023-10-06T02:30:50.671736+0000","last_became_peered":"2023-10-06T02:30:50.671736+0000","last_unstale":"2023-10-06T02:30:50.673429+0000","last_undegraded":"2023-10-06T02:30:50.673429+0000","last_fullsized":"2023-10-06T02:30:50.673429+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_clean_scrub_stamp":"2023-10-06T02:30:49.637381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:21:58.534242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.672063+0000","last_change":"2023-10-06T02:30:51.668121+0000","last_active":"2023-10-06T02:30:51.672063+0000","last_peered":"2023-10-06T02:30:51.672063+0000","last_clean":"2023-10-06T02:30:51.672063+0000","last_became_active":"2023-10-06T02:30:51.664156+0000","last_became_peered":"2023-10-06T02:30:51.664156+0000","last_unstale":"2023-10-06T02:30:51.672063+0000","last_undegraded":"2023-10-06T02:30:51.672063+0000","last_fullsized":"2023-10-06T02:30:51.672063+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:05:57.018432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.662135+0000","last_change":"2023-10-06T02:30:50.683019+0000","last_active":"2023-10-06T02:30:51.662135+0000","last_peered":"2023-10-06T02:30:51.662135+0000","last_clean":"2023-10-06T02:30:51.662135+0000","last_became_active":"2023-10-06T02:30:50.682766+0000","last_became_peered":"2023-10-06T02:30:50.682766+0000","last_unstale":"2023-10-06T02:30:51.662135+0000","last_undegraded":"2023-10-06T02:30:51.662135+0000","last_fullsized":"2023-10-06T02:30:51.662135+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_clean_scrub_stamp":"2023-10-06T02:30:49.637381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:12:26.468515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.674078+0000","last_change":"2023-10-06T02:30:51.674078+0000","last_active":"2023-10-06T02:30:51.674078+0000","last_peered":"2023-10-06T02:30:51.674078+0000","last_clean":"2023-10-06T02:30:51.674078+0000","last_became_active":"2023-10-06T02:30:51.672332+0000","last_became_peered":"2023-10-06T02:30:51.672332+0000","last_unstale":"2023-10-06T02:30:51.674078+0000","last_undegraded":"2023-10-06T02:30:51.674078+0000","last_fullsized":"2023-10-06T02:30:51.674078+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:16:50.121627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T02:30:50.678742+0000","last_change":"2023-10-06T02:30:50.674949+0000","last_active":"2023-10-06T02:30:50.678742+0000","last_peered":"2023-10-06T02:30:50.678742+0000","last_clean":"2023-10-06T02:30:50.678742+0000","last_became_active":"2023-10-06T02:30:50.674664+0000","last_became_peered":"2023-10-06T02:30:50.674664+0000","last_unstale":"2023-10-06T02:30:50.678742+0000","last_undegraded":"2023-10-06T02:30:50.678742+0000","last_fullsized":"2023-10-06T02:30:50.678742+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_clean_scrub_stamp":"2023-10-06T02:30:49.637381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:54:47.060617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.680365+0000","last_change":"2023-10-06T02:30:51.675370+0000","last_active":"2023-10-06T02:30:51.680365+0000","last_peered":"2023-10-06T02:30:51.680365+0000","last_clean":"2023-10-06T02:30:51.680365+0000","last_became_active":"2023-10-06T02:30:51.675066+0000","last_became_peered":"2023-10-06T02:30:51.675066+0000","last_unstale":"2023-10-06T02:30:51.680365+0000","last_undegraded":"2023-10-06T02:30:51.680365+0000","last_fullsized":"2023-10-06T02:30:51.680365+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:39:08.450582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T02:30:50.664499+0000","last_change":"2023-10-06T02:30:50.662598+0000","last_active":"2023-10-06T02:30:50.664499+0000","last_peered":"2023-10-06T02:30:50.664499+0000","last_clean":"2023-10-06T02:30:50.664499+0000","last_became_active":"2023-10-06T02:30:50.662284+0000","last_became_peered":"2023-10-06T02:30:50.662284+0000","last_unstale":"2023-10-06T02:30:50.664499+0000","last_undegraded":"2023-10-06T02:30:50.664499+0000","last_fullsized":"2023-10-06T02:30:50.664499+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_clean_scrub_stamp":"2023-10-06T02:30:49.637381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:35:54.781726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.675498+0000","last_change":"2023-10-06T02:30:51.675498+0000","last_active":"2023-10-06T02:30:51.675498+0000","last_peered":"2023-10-06T02:30:51.675498+0000","last_clean":"2023-10-06T02:30:51.675498+0000","last_became_active":"2023-10-06T02:30:51.675217+0000","last_became_peered":"2023-10-06T02:30:51.675217+0000","last_unstale":"2023-10-06T02:30:51.675498+0000","last_undegraded":"2023-10-06T02:30:51.675498+0000","last_fullsized":"2023-10-06T02:30:51.675498+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:50:51.184941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.655775+0000","last_change":"2023-10-06T02:30:50.672835+0000","last_active":"2023-10-06T02:30:51.655775+0000","last_peered":"2023-10-06T02:30:51.655775+0000","last_clean":"2023-10-06T02:30:51.655775+0000","last_became_active":"2023-10-06T02:30:50.672540+0000","last_became_peered":"2023-10-06T02:30:50.672540+0000","last_unstale":"2023-10-06T02:30:51.655775+0000","last_undegraded":"2023-10-06T02:30:51.655775+0000","last_fullsized":"2023-10-06T02:30:51.655775+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_clean_scrub_stamp":"2023-10-06T02:30:49.637381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:35:14.998650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.681931+0000","last_change":"2023-10-06T02:30:51.672400+0000","last_active":"2023-10-06T02:30:51.681931+0000","last_peered":"2023-10-06T02:30:51.681931+0000","last_clean":"2023-10-06T02:30:51.681931+0000","last_became_active":"2023-10-06T02:30:51.672140+0000","last_became_peered":"2023-10-06T02:30:51.672140+0000","last_unstale":"2023-10-06T02:30:51.681931+0000","last_undegraded":"2023-10-06T02:30:51.681931+0000","last_fullsized":"2023-10-06T02:30:51.681931+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:52:30.458923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.655129+0000","last_change":"2023-10-06T02:30:50.672044+0000","last_active":"2023-10-06T02:30:51.655129+0000","last_peered":"2023-10-06T02:30:51.655129+0000","last_clean":"2023-10-06T02:30:51.655129+0000","last_became_active":"2023-10-06T02:30:50.671760+0000","last_became_peered":"2023-10-06T02:30:50.671760+0000","last_unstale":"2023-10-06T02:30:51.655129+0000","last_undegraded":"2023-10-06T02:30:51.655129+0000","last_fullsized":"2023-10-06T02:30:51.655129+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_clean_scrub_stamp":"2023-10-06T02:30:49.637381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:34:35.008716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.668851+0000","last_change":"2023-10-06T02:30:51.663438+0000","last_active":"2023-10-06T02:30:51.668851+0000","last_peered":"2023-10-06T02:30:51.668851+0000","last_clean":"2023-10-06T02:30:51.668851+0000","last_became_active":"2023-10-06T02:30:51.663194+0000","last_became_peered":"2023-10-06T02:30:51.663194+0000","last_unstale":"2023-10-06T02:30:51.668851+0000","last_undegraded":"2023-10-06T02:30:51.668851+0000","last_fullsized":"2023-10-06T02:30:51.668851+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:16:02.882307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T02:30:50.677591+0000","last_change":"2023-10-06T02:30:50.673162+0000","last_active":"2023-10-06T02:30:50.677591+0000","last_peered":"2023-10-06T02:30:50.677591+0000","last_clean":"2023-10-06T02:30:50.677591+0000","last_became_active":"2023-10-06T02:30:50.672891+0000","last_became_peered":"2023-10-06T02:30:50.672891+0000","last_unstale":"2023-10-06T02:30:50.677591+0000","last_undegraded":"2023-10-06T02:30:50.677591+0000","last_fullsized":"2023-10-06T02:30:50.677591+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_clean_scrub_stamp":"2023-10-06T02:30:49.637381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:24:41.735075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":115,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.662765+0000","last_change":"2023-10-06T02:23:01.992935+0000","last_active":"2023-10-06T02:30:51.662765+0000","last_peered":"2023-10-06T02:30:51.662765+0000","last_clean":"2023-10-06T02:30:51.662765+0000","last_became_active":"2023-10-06T02:23:01.992568+0000","last_became_peered":"2023-10-06T02:23:01.992568+0000","last_unstale":"2023-10-06T02:30:51.662765+0000","last_undegraded":"2023-10-06T02:30:51.662765+0000","last_fullsized":"2023-10-06T02:30:51.662765+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:06.212251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.686892+0000","last_change":"2023-10-06T02:30:51.684069+0000","last_active":"2023-10-06T02:30:51.686892+0000","last_peered":"2023-10-06T02:30:51.686892+0000","last_clean":"2023-10-06T02:30:51.686892+0000","last_became_active":"2023-10-06T02:30:51.682151+0000","last_became_peered":"2023-10-06T02:30:51.682151+0000","last_unstale":"2023-10-06T02:30:51.686892+0000","last_undegraded":"2023-10-06T02:30:51.686892+0000","last_fullsized":"2023-10-06T02:30:51.686892+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:35:49.324609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.13","version":"143'2","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:52.099045+0000","last_change":"2023-10-06T02:30:50.672607+0000","last_active":"2023-10-06T02:30:52.099045+0000","last_peered":"2023-10-06T02:30:52.099045+0000","last_clean":"2023-10-06T02:30:52.099045+0000","last_became_active":"2023-10-06T02:30:50.672343+0000","last_became_peered":"2023-10-06T02:30:50.672343+0000","last_unstale":"2023-10-06T02:30:52.099045+0000","last_undegraded":"2023-10-06T02:30:52.099045+0000","last_fullsized":"2023-10-06T02:30:52.099045+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_clean_scrub_stamp":"2023-10-06T02:30:49.637381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:39:07.441201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":116,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.657910+0000","last_change":"2023-10-06T02:23:01.989934+0000","last_active":"2023-10-06T02:30:51.657910+0000","last_peered":"2023-10-06T02:30:51.657910+0000","last_clean":"2023-10-06T02:30:51.657910+0000","last_became_active":"2023-10-06T02:23:01.989706+0000","last_became_peered":"2023-10-06T02:23:01.989706+0000","last_unstale":"2023-10-06T02:30:51.657910+0000","last_undegraded":"2023-10-06T02:30:51.657910+0000","last_fullsized":"2023-10-06T02:30:51.657910+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:06.212251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.681850+0000","last_change":"2023-10-06T02:30:51.670323+0000","last_active":"2023-10-06T02:30:51.681850+0000","last_peered":"2023-10-06T02:30:51.681850+0000","last_clean":"2023-10-06T02:30:51.681850+0000","last_became_active":"2023-10-06T02:30:51.670054+0000","last_became_peered":"2023-10-06T02:30:51.670054+0000","last_unstale":"2023-10-06T02:30:51.681850+0000","last_undegraded":"2023-10-06T02:30:51.681850+0000","last_fullsized":"2023-10-06T02:30:51.681850+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:49:38.772657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.10","version":"143'8","reported_seq":18,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:52.099165+0000","last_change":"2023-10-06T02:30:50.674130+0000","last_active":"2023-10-06T02:30:52.099165+0000","last_peered":"2023-10-06T02:30:52.099165+0000","last_clean":"2023-10-06T02:30:52.099165+0000","last_became_active":"2023-10-06T02:30:50.673875+0000","last_became_peered":"2023-10-06T02:30:50.673875+0000","last_unstale":"2023-10-06T02:30:52.099165+0000","last_undegraded":"2023-10-06T02:30:52.099165+0000","last_fullsized":"2023-10-06T02:30:52.099165+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_clean_scrub_stamp":"2023-10-06T02:30:49.637381+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:14:46.899357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":110,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.664119+0000","last_change":"2023-10-06T02:23:02.054069+0000","last_active":"2023-10-06T02:30:51.664119+0000","last_peered":"2023-10-06T02:30:51.664119+0000","last_clean":"2023-10-06T02:30:51.664119+0000","last_became_active":"2023-10-06T02:23:02.053739+0000","last_became_peered":"2023-10-06T02:23:02.053739+0000","last_unstale":"2023-10-06T02:30:51.664119+0000","last_undegraded":"2023-10-06T02:30:51.664119+0000","last_fullsized":"2023-10-06T02:30:51.664119+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:57:48.822608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.678381+0000","last_change":"2023-10-06T02:30:51.678381+0000","last_active":"2023-10-06T02:30:51.678381+0000","last_peered":"2023-10-06T02:30:51.678381+0000","last_clean":"2023-10-06T02:30:51.678381+0000","last_became_active":"2023-10-06T02:30:51.678011+0000","last_became_peered":"2023-10-06T02:30:51.678011+0000","last_unstale":"2023-10-06T02:30:51.678381+0000","last_undegraded":"2023-10-06T02:30:51.678381+0000","last_fullsized":"2023-10-06T02:30:51.678381+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:13:55.763105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.15","version":"143'3","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:52.093508+0000","last_change":"2023-10-06T02:30:50.676649+0000","last_active":"2023-10-06T02:30:52.093508+0000","last_peered":"2023-10-06T02:30:52.093508+0000","last_clean":"2023-10-06T02:30:52.093508+0000","last_became_active":"2023-10-06T02:30:50.676245+0000","last_became_peered":"2023-10-06T02:30:50.676245+0000","last_unstale":"2023-10-06T02:30:52.093508+0000","last_undegraded":"2023-10-06T02:30:52.093508+0000","last_fullsized":"2023-10-06T02:30:52.093508+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_clean_scrub_stamp":"2023-10-06T02:30:49.637381+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:17:34.046237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":110,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.663412+0000","last_change":"2023-10-06T02:23:01.990773+0000","last_active":"2023-10-06T02:30:51.663412+0000","last_peered":"2023-10-06T02:30:51.663412+0000","last_clean":"2023-10-06T02:30:51.663412+0000","last_became_active":"2023-10-06T02:23:01.990511+0000","last_became_peered":"2023-10-06T02:23:01.990511+0000","last_unstale":"2023-10-06T02:30:51.663412+0000","last_undegraded":"2023-10-06T02:30:51.663412+0000","last_fullsized":"2023-10-06T02:30:51.663412+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:42:54.797731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.677117+0000","last_change":"2023-10-06T02:30:51.672599+0000","last_active":"2023-10-06T02:30:51.677117+0000","last_peered":"2023-10-06T02:30:51.677117+0000","last_clean":"2023-10-06T02:30:51.677117+0000","last_became_active":"2023-10-06T02:30:51.672259+0000","last_became_peered":"2023-10-06T02:30:51.672259+0000","last_unstale":"2023-10-06T02:30:51.677117+0000","last_undegraded":"2023-10-06T02:30:51.677117+0000","last_fullsized":"2023-10-06T02:30:51.677117+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:35:55.791108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T02:30:50.683659+0000","last_change":"2023-10-06T02:30:50.678752+0000","last_active":"2023-10-06T02:30:50.683659+0000","last_peered":"2023-10-06T02:30:50.683659+0000","last_clean":"2023-10-06T02:30:50.683659+0000","last_became_active":"2023-10-06T02:30:50.678431+0000","last_became_peered":"2023-10-06T02:30:50.678431+0000","last_unstale":"2023-10-06T02:30:50.683659+0000","last_undegraded":"2023-10-06T02:30:50.683659+0000","last_fullsized":"2023-10-06T02:30:50.683659+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_clean_scrub_stamp":"2023-10-06T02:30:49.637381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:39:07.441201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"76'128","reported_seq":302,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.663445+0000","last_change":"2023-10-06T02:22:55.032519+0000","last_active":"2023-10-06T02:30:51.663445+0000","last_peered":"2023-10-06T02:30:51.663445+0000","last_clean":"2023-10-06T02:30:51.663445+0000","last_became_active":"2023-10-06T02:22:55.031367+0000","last_became_peered":"2023-10-06T02:22:55.031367+0000","last_unstale":"2023-10-06T02:30:51.663445+0000","last_undegraded":"2023-10-06T02:30:51.663445+0000","last_fullsized":"2023-10-06T02:30:51.663445+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:22:53.309097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:22:53.309097+0000","last_clean_scrub_stamp":"2023-10-06T02:22:53.309097+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:53:46.520909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":134,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.655209+0000","last_change":"2023-10-06T02:23:01.988140+0000","last_active":"2023-10-06T02:30:51.655209+0000","last_peered":"2023-10-06T02:30:51.655209+0000","last_clean":"2023-10-06T02:30:51.655209+0000","last_became_active":"2023-10-06T02:23:01.987769+0000","last_became_peered":"2023-10-06T02:23:01.987769+0000","last_unstale":"2023-10-06T02:30:51.655209+0000","last_undegraded":"2023-10-06T02:30:51.655209+0000","last_fullsized":"2023-10-06T02:30:51.655209+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:20:08.391912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.682189+0000","last_change":"2023-10-06T02:30:51.677753+0000","last_active":"2023-10-06T02:30:51.682189+0000","last_peered":"2023-10-06T02:30:51.682189+0000","last_clean":"2023-10-06T02:30:51.682189+0000","last_became_active":"2023-10-06T02:30:51.677358+0000","last_became_peered":"2023-10-06T02:30:51.677358+0000","last_unstale":"2023-10-06T02:30:51.682189+0000","last_undegraded":"2023-10-06T02:30:51.682189+0000","last_fullsized":"2023-10-06T02:30:51.682189+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:35:49.324609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"143'6","reported_seq":15,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:52.100309+0000","last_change":"2023-10-06T02:30:50.679318+0000","last_active":"2023-10-06T02:30:52.100309+0000","last_peered":"2023-10-06T02:30:52.100309+0000","last_clean":"2023-10-06T02:30:52.100309+0000","last_became_active":"2023-10-06T02:30:50.679061+0000","last_became_peered":"2023-10-06T02:30:50.679061+0000","last_unstale":"2023-10-06T02:30:52.100309+0000","last_undegraded":"2023-10-06T02:30:52.100309+0000","last_fullsized":"2023-10-06T02:30:52.100309+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_clean_scrub_stamp":"2023-10-06T02:30:49.637381+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:40:20.143016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":123,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.662666+0000","last_change":"2023-10-06T02:23:02.054338+0000","last_active":"2023-10-06T02:30:51.662666+0000","last_peered":"2023-10-06T02:30:51.662666+0000","last_clean":"2023-10-06T02:30:51.662666+0000","last_became_active":"2023-10-06T02:23:02.053991+0000","last_became_peered":"2023-10-06T02:23:02.053991+0000","last_unstale":"2023-10-06T02:30:51.662666+0000","last_undegraded":"2023-10-06T02:30:51.662666+0000","last_fullsized":"2023-10-06T02:30:51.662666+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:39:30.192838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.680365+0000","last_change":"2023-10-06T02:30:51.675353+0000","last_active":"2023-10-06T02:30:51.680365+0000","last_peered":"2023-10-06T02:30:51.680365+0000","last_clean":"2023-10-06T02:30:51.680365+0000","last_became_active":"2023-10-06T02:30:51.672735+0000","last_became_peered":"2023-10-06T02:30:51.672735+0000","last_unstale":"2023-10-06T02:30:51.680365+0000","last_undegraded":"2023-10-06T02:30:51.680365+0000","last_fullsized":"2023-10-06T02:30:51.680365+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:40:21.152397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T02:30:50.683712+0000","last_change":"2023-10-06T02:30:50.679780+0000","last_active":"2023-10-06T02:30:50.683712+0000","last_peered":"2023-10-06T02:30:50.683712+0000","last_clean":"2023-10-06T02:30:50.683712+0000","last_became_active":"2023-10-06T02:30:50.679465+0000","last_became_peered":"2023-10-06T02:30:50.679465+0000","last_unstale":"2023-10-06T02:30:50.683712+0000","last_undegraded":"2023-10-06T02:30:50.683712+0000","last_fullsized":"2023-10-06T02:30:50.683712+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_clean_scrub_stamp":"2023-10-06T02:30:49.637381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:26:19.617860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":134,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.655229+0000","last_change":"2023-10-06T02:23:01.988631+0000","last_active":"2023-10-06T02:30:51.655229+0000","last_peered":"2023-10-06T02:30:51.655229+0000","last_clean":"2023-10-06T02:30:51.655229+0000","last_became_active":"2023-10-06T02:23:01.988303+0000","last_became_peered":"2023-10-06T02:23:01.988303+0000","last_unstale":"2023-10-06T02:30:51.655229+0000","last_undegraded":"2023-10-06T02:30:51.655229+0000","last_fullsized":"2023-10-06T02:30:51.655229+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:38.211466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.677333+0000","last_change":"2023-10-06T02:30:51.673852+0000","last_active":"2023-10-06T02:30:51.677333+0000","last_peered":"2023-10-06T02:30:51.677333+0000","last_clean":"2023-10-06T02:30:51.677333+0000","last_became_active":"2023-10-06T02:30:51.673521+0000","last_became_peered":"2023-10-06T02:30:51.673521+0000","last_unstale":"2023-10-06T02:30:51.677333+0000","last_undegraded":"2023-10-06T02:30:51.677333+0000","last_fullsized":"2023-10-06T02:30:51.677333+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:16:02.882307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.655645+0000","last_change":"2023-10-06T02:30:50.671581+0000","last_active":"2023-10-06T02:30:51.655645+0000","last_peered":"2023-10-06T02:30:51.655645+0000","last_clean":"2023-10-06T02:30:51.655645+0000","last_became_active":"2023-10-06T02:30:50.671278+0000","last_became_peered":"2023-10-06T02:30:50.671278+0000","last_unstale":"2023-10-06T02:30:51.655645+0000","last_undegraded":"2023-10-06T02:30:51.655645+0000","last_fullsized":"2023-10-06T02:30:51.655645+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_clean_scrub_stamp":"2023-10-06T02:30:49.637381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:45:15.343527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":116,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.658042+0000","last_change":"2023-10-06T02:23:02.054039+0000","last_active":"2023-10-06T02:30:51.658042+0000","last_peered":"2023-10-06T02:30:51.658042+0000","last_clean":"2023-10-06T02:30:51.658042+0000","last_became_active":"2023-10-06T02:23:02.053742+0000","last_became_peered":"2023-10-06T02:23:02.053742+0000","last_unstale":"2023-10-06T02:30:51.658042+0000","last_undegraded":"2023-10-06T02:30:51.658042+0000","last_fullsized":"2023-10-06T02:30:51.658042+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:58:18.846475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":24,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2},{"poolid":23,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574949,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303720,"kb_used_data":2336,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068120,"statfs":{"total":96636764160,"available":96325754880,"internally_reserved":0,"allocated":2392064,"data_stored":1428080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:30:12 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.64100000000000001,"15min":0.63600000000000001},"min":{"1min":0.375,"5min":0.318,"15min":0.318},"max":{"1min":0.78600000000000003,"5min":1.169,"15min":1.169},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.63900000000000001,"15min":0.61199999999999999},"min":{"1min":0.45300000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.84699999999999998,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.91800000000000004}]},{"osd":1,"last update":"Fri Oct 6 02:30:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.73199999999999998,"15min":0.69399999999999995},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.40300000000000002},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.147},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.67400000000000004,"15min":0.67200000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.82399999999999995,"5min":1.129,"15min":1.129},"last":0.71599999999999997}]},{"osd":2,"last update":"Fri Oct 6 02:30:12 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.67500000000000004,"15min":0.68700000000000006},"min":{"1min":0.44,"5min":0.375,"15min":0.375},"max":{"1min":0.87,"5min":1.238,"15min":1.238},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.69099999999999995,"15min":0.68999999999999995},"min":{"1min":0.54000000000000004,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.75700000000000001}]},{"osd":3,"last update":"Fri Oct 6 02:30:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.45700000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.79400000000000004,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.76300000000000001,"15min":0.77300000000000002},"min":{"1min":0.55900000000000005,"5min":0.497,"15min":0.497},"max":{"1min":1.383,"5min":1.383,"15min":1.383},"last":0.877}]},{"osd":4,"last update":"Fri Oct 6 02:30:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.59199999999999997,"15min":0.67200000000000004},"min":{"1min":0.42899999999999999,"5min":0.41099999999999998,"15min":0.35099999999999998},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.2849999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.67200000000000004,"15min":0.747},"min":{"1min":0.41199999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.96499999999999997,"5min":1.51,"15min":1.51},"last":0.78300000000000003}]},{"osd":5,"last update":"Fri Oct 6 02:30:12 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.69199999999999995,"15min":0.72699999999999998},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.67600000000000005,"15min":0.71999999999999997},"min":{"1min":0.47199999999999998,"5min":0.442,"15min":0.442},"max":{"1min":0.874,"5min":0.98099999999999998,"15min":1.0289999999999999},"last":0.80700000000000005}]},{"osd":6,"last update":"Fri Oct 6 02:30:12 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.76100000000000001,"15min":0.78200000000000003},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.97599999999999998,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.77800000000000002,"15min":0.80600000000000005},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.872,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.89700000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574949,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303152,"kb_used_data":1768,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068688,"statfs":{"total":96636764160,"available":96326336512,"internally_reserved":0,"allocated":1810432,"data_stored":844381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:30:06 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.68700000000000006,"15min":0.66700000000000004},"min":{"1min":0.501,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.88500000000000001,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.72999999999999998,"15min":0.70299999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.91100000000000003,"5min":1.0229999999999999,"15min":1.119},"last":1.04}]},{"osd":1,"last update":"Fri Oct 6 02:30:06 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.68100000000000005,"15min":0.65100000000000002},"min":{"1min":0.45200000000000001,"5min":0.40999999999999998,"15min":0.40699999999999997},"max":{"1min":0.96499999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.68500000000000005,"15min":0.67100000000000004},"min":{"1min":0.437,"5min":0.41899999999999998,"15min":0.40999999999999998},"max":{"1min":0.84399999999999997,"5min":0.997,"15min":0.997},"last":0.78000000000000003}]},{"osd":2,"last update":"Fri Oct 6 02:30:06 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.72899999999999998,"15min":0.70799999999999996},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.92100000000000004,"5min":1.0449999999999999,"15min":1.137},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.74399999999999999,"15min":0.71799999999999997},"min":{"1min":0.55600000000000005,"5min":0.45300000000000001,"15min":0.40300000000000002},"max":{"1min":0.95299999999999996,"5min":1.077,"15min":1.077},"last":0.88200000000000001}]},{"osd":3,"last update":"Fri Oct 6 02:30:06 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.71199999999999997,"15min":0.67100000000000004},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.94099999999999995,"5min":1.056,"15min":1.056},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.73299999999999998,"15min":0.70899999999999996},"min":{"1min":0.56699999999999995,"5min":0.49299999999999999,"15min":0.45900000000000002},"max":{"1min":0.97799999999999998,"5min":0.97999999999999998,"15min":0.98499999999999999},"last":0.80400000000000005}]},{"osd":4,"last update":"Fri Oct 6 02:30:06 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.65900000000000003,"15min":0.64200000000000002},"min":{"1min":0.42499999999999999,"5min":0.36199999999999999,"15min":0.34300000000000003},"max":{"1min":0.93000000000000005,"5min":0.94799999999999995,"15min":1.095},"last":0.96799999999999997},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.69999999999999996,"15min":0.66900000000000004},"min":{"1min":0.52200000000000002,"5min":0.41699999999999998,"15min":0.40999999999999998},"max":{"1min":0.91900000000000004,"5min":1.002,"15min":1.29},"last":0.998}]},{"osd":5,"last update":"Fri Oct 6 02:30:06 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.67700000000000005,"15min":0.69799999999999995},"min":{"1min":0.438,"5min":0.438,"15min":0.39700000000000002},"max":{"1min":1.014,"5min":1.014,"15min":1.2929999999999999},"last":0.98699999999999999},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.70799999999999996,"15min":0.70599999999999996},"min":{"1min":0.48699999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.01,"5min":1.01,"15min":1.02},"last":1.1180000000000001}]},{"osd":7,"last update":"Fri Oct 6 02:30:06 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.74099999999999999,"15min":0.749},"min":{"1min":0.48399999999999999,"5min":0.41499999999999998,"15min":0.36799999999999999},"max":{"1min":1,"5min":1,"15min":1.458},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.76500000000000001,"15min":0.76300000000000001},"min":{"1min":0.50700000000000001,"5min":0.47299999999999998,"15min":0.45900000000000002},"max":{"1min":1.046,"5min":1.103,"15min":1.2470000000000001},"last":0.93799999999999994}]}]},{"osd":1,"up_from":13,"seq":55834574948,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303164,"kb_used_data":1780,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068676,"statfs":{"total":96636764160,"available":96326324224,"internally_reserved":0,"allocated":1822720,"data_stored":845917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:30:02 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.42999999999999999,"15min":0.45300000000000001},"min":{"1min":0.30599999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.59599999999999997,"5min":0.65500000000000003,"15min":1.2230000000000001},"last":0.314},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.46999999999999997,"15min":0.51400000000000001},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.58899999999999997,"5min":0.71799999999999997,"15min":1.788},"last":0.64500000000000002}]},{"osd":2,"last update":"Fri Oct 6 02:30:02 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.44,"15min":0.50600000000000001},"min":{"1min":0.33800000000000002,"5min":0.307,"15min":0.28299999999999997},"max":{"1min":0.53800000000000003,"5min":0.60899999999999999,"15min":2.0840000000000001},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.504,"15min":0.54700000000000004},"min":{"1min":0.38700000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":1.6699999999999999},"last":0.45300000000000001}]},{"osd":3,"last update":"Fri Oct 6 02:30:02 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.59799999999999998,"15min":0.60999999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.72499999999999998,"5min":0.89900000000000002,"15min":1.266},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.54200000000000004,"15min":0.62},"min":{"1min":0.46899999999999997,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.69999999999999996,"5min":0.79400000000000004,"15min":1.861},"last":0.41999999999999998}]},{"osd":4,"last update":"Fri Oct 6 02:30:02 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.77300000000000002,"15min":0.83799999999999997},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.55900000000000005},"max":{"1min":0.93999999999999995,"5min":1.3580000000000001,"15min":1.9279999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.78400000000000003,"15min":0.84099999999999997},"min":{"1min":0.57399999999999995,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.90200000000000002,"5min":1.4339999999999999,"15min":2.0179999999999998},"last":0.71099999999999997}]},{"osd":5,"last update":"Fri Oct 6 02:30:02 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.73699999999999999,"15min":0.83699999999999997},"min":{"1min":0.54400000000000004,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.874,"5min":1.3069999999999999,"15min":1.9830000000000001},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.748,"15min":0.80700000000000005},"min":{"1min":0.55400000000000005,"5min":0.53600000000000003,"15min":0.47299999999999998},"max":{"1min":0.91300000000000003,"5min":1.327,"15min":1.3380000000000001},"last":0.67200000000000004}]},{"osd":6,"last update":"Fri Oct 6 02:30:02 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.78400000000000003,"15min":0.84799999999999998},"min":{"1min":0.60899999999999999,"5min":0.52700000000000002,"15min":0.45600000000000002},"max":{"1min":1.012,"5min":1.0609999999999999,"15min":1.829},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.80000000000000004,"15min":0.88900000000000001},"min":{"1min":0.61099999999999999,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.93100000000000005,"5min":1.351,"15min":2.052},"last":0.74299999999999999}]},{"osd":7,"last update":"Fri Oct 6 02:30:02 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.76300000000000001,"15min":0.81999999999999995},"min":{"1min":0.59399999999999997,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.90400000000000003,"5min":1.3120000000000001,"15min":1.3340000000000001},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.77800000000000002,"15min":0.85699999999999998},"min":{"1min":0.51600000000000001,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.018,"5min":1.2110000000000001,"15min":1.895},"last":0.58799999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574949,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303740,"kb_used_data":2356,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068100,"statfs":{"total":96636764160,"available":96325734400,"internally_reserved":0,"allocated":2412544,"data_stored":1434853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 02:30:14 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.44900000000000001,"15min":0.45600000000000002},"min":{"1min":0.30199999999999999,"5min":0.28899999999999998,"15min":0.28000000000000003},"max":{"1min":2.6120000000000001,"5min":2.6120000000000001,"15min":2.6120000000000001},"last":0.376},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.48699999999999999,"15min":0.48399999999999999},"min":{"1min":0.32000000000000001,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":3.4199999999999999,"5min":3.4199999999999999,"15min":3.4199999999999999},"last":0.51400000000000001}]},{"osd":2,"last update":"Fri Oct 6 02:30:14 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.52900000000000003,"15min":0.52000000000000002},"min":{"1min":0.375,"5min":0.35999999999999999,"15min":0.34699999999999998},"max":{"1min":2.5699999999999998,"5min":3.0339999999999998,"15min":3.0339999999999998},"last":0.432},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.52700000000000002,"15min":0.52300000000000002},"min":{"1min":0.36499999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":3.343,"5min":3.343,"15min":3.343},"last":0.57599999999999996}]},{"osd":3,"last update":"Fri Oct 6 02:30:14 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.50900000000000001,"15min":0.50800000000000001},"min":{"1min":0.33900000000000002,"5min":0.31,"15min":0.31},"max":{"1min":3.6579999999999999,"5min":3.6579999999999999,"15min":3.6579999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.59699999999999998,"15min":0.59599999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":2.6890000000000001,"5min":2.6890000000000001,"15min":2.6890000000000001},"last":0.56899999999999995}]},{"osd":4,"last update":"Fri Oct 6 02:30:14 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.77900000000000003,"15min":0.79800000000000004},"min":{"1min":0.56100000000000005,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":3.2999999999999998,"5min":3.2999999999999998,"15min":3.2999999999999998},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.76700000000000002,"15min":0.79500000000000004},"min":{"1min":0.51600000000000001,"5min":0.51000000000000001,"15min":0.48499999999999999},"max":{"1min":2.2930000000000001,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":0.93899999999999995}]},{"osd":5,"last update":"Fri Oct 6 02:30:14 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.76000000000000001,"15min":0.78000000000000003},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.012,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":1.038},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.80400000000000005,"15min":0.80200000000000005},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":2.5070000000000001,"5min":3.1110000000000002,"15min":3.1110000000000002},"last":1.02}]},{"osd":6,"last update":"Fri Oct 6 02:30:14 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":2.6459999999999999,"5min":3.1600000000000001,"15min":3.1600000000000001},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.78400000000000003,"15min":0.81999999999999995},"min":{"1min":0.56499999999999995,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":2.3250000000000002,"5min":2.3250000000000002,"15min":2.3250000000000002},"last":0.98199999999999998}]},{"osd":7,"last update":"Fri Oct 6 02:30:14 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.82699999999999996,"15min":0.82899999999999996},"min":{"1min":0.621,"5min":0.56899999999999995,"15min":0.56000000000000005},"max":{"1min":3.3809999999999998,"5min":3.3809999999999998,"15min":3.3809999999999998},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.78000000000000003,"15min":0.81799999999999995},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.53300000000000003},"max":{"1min":2.3740000000000001,"5min":2.3740000000000001,"15min":2.3740000000000001},"last":0.98999999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574950,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303152,"kb_used_data":1768,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068688,"statfs":{"total":96636764160,"available":96326336512,"internally_reserved":0,"allocated":1810432,"data_stored":844381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:30:10 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.43099999999999999,"15min":0.437},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.502,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.307},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.47299999999999998,"15min":0.48199999999999998},"min":{"1min":0.35799999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.64100000000000001,"5min":0.88700000000000001,"15min":1.0800000000000001},"last":0.36899999999999999}]},{"osd":1,"last update":"Fri Oct 6 02:30:10 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.47799999999999998,"15min":0.498},"min":{"1min":0.36299999999999999,"5min":0.33600000000000002,"15min":0.32400000000000001},"max":{"1min":0.55500000000000005,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.52700000000000002,"15min":0.58999999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34599999999999997},"max":{"1min":0.66000000000000003,"5min":1.778,"15min":1.778},"last":0.40000000000000002}]},{"osd":3,"last update":"Fri Oct 6 02:30:10 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.50900000000000001,"15min":0.52900000000000003},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.79500000000000004,"5min":0.83699999999999997,"15min":0.97499999999999998},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.59299999999999997,"15min":0.61399999999999999},"min":{"1min":0.48099999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.77100000000000002,"5min":0.84699999999999998,"15min":1.028},"last":0.45300000000000001}]},{"osd":4,"last update":"Fri Oct 6 02:30:10 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.751,"15min":0.77300000000000002},"min":{"1min":0.51500000000000001,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.97499999999999998,"5min":1.109,"15min":1.1319999999999999},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.78400000000000003,"15min":0.83499999999999996},"min":{"1min":0.57999999999999996,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.96099999999999997,"5min":1.907,"15min":1.907},"last":0.57199999999999995}]},{"osd":5,"last update":"Fri Oct 6 02:30:10 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.73999999999999999,"15min":0.79000000000000004},"min":{"1min":0.53200000000000003,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.97199999999999998,"5min":1.109,"15min":1.1850000000000001},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.76300000000000001,"15min":0.81000000000000005},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0129999999999999,"5min":1.153,"15min":1.258},"last":0.72299999999999998}]},{"osd":6,"last update":"Fri Oct 6 02:30:10 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.79500000000000004,"15min":0.83799999999999997},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.98699999999999999,"5min":1.827,"15min":1.827},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.77500000000000002,"15min":0.79300000000000004},"min":{"1min":0.60299999999999998,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0660000000000001,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.57999999999999996}]},{"osd":7,"last update":"Fri Oct 6 02:30:10 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.80500000000000005,"15min":0.86799999999999999},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0369999999999999,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.78600000000000003,"15min":0.82399999999999995},"min":{"1min":0.66100000000000003,"5min":0.55300000000000005,"15min":0.54800000000000004},"max":{"1min":1.01,"5min":1.125,"15min":1.1479999999999999},"last":0.624}]}]},{"osd":3,"up_from":13,"seq":55834574947,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303160,"kb_used_data":1776,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068680,"statfs":{"total":96636764160,"available":96326328320,"internally_reserved":0,"allocated":1818624,"data_stored":839824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:30:10 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.443,"15min":0.49399999999999999},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.56200000000000006,"5min":1.0489999999999999,"15min":2.024},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.49199999999999999,"15min":0.51500000000000001},"min":{"1min":0.311,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.754,"5min":0.84699999999999998,"15min":2.181},"last":0.31900000000000001}]},{"osd":1,"last update":"Fri Oct 6 02:30:10 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.46899999999999997,"15min":0.64100000000000001},"min":{"1min":0.34799999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.65100000000000002,"5min":0.66100000000000003,"15min":4.492},"last":0.38},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.52600000000000002,"15min":0.68700000000000006},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.76200000000000001,"5min":1.079,"15min":4.569},"last":0.53600000000000003}]},{"osd":2,"last update":"Fri Oct 6 02:30:10 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.627,"15min":0.65500000000000003},"min":{"1min":0.372,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.82599999999999996,"5min":1.024,"15min":2.0870000000000002},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.59299999999999997,"15min":0.63700000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.76800000000000002,"5min":0.94199999999999995,"15min":2.4260000000000002},"last":0.42999999999999999}]},{"osd":4,"last update":"Fri Oct 6 02:30:10 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.73799999999999999,"15min":0.81100000000000005},"min":{"1min":0.55000000000000004,"5min":0.502,"15min":0.44800000000000001},"max":{"1min":0.94199999999999995,"5min":1.0149999999999999,"15min":2.214},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.76400000000000001,"15min":0.80000000000000004},"min":{"1min":0.54400000000000004,"5min":0.50800000000000001,"15min":0.5},"max":{"1min":1.034,"5min":1.079,"15min":1.2869999999999999},"last":0.85299999999999998}]},{"osd":5,"last update":"Fri Oct 6 02:30:10 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.76300000000000001,"15min":0.80900000000000005},"min":{"1min":0.497,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.077,"5min":1.077,"15min":1.3029999999999999},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.752,"5min":0.76000000000000001,"15min":0.83499999999999996},"min":{"1min":0.60099999999999998,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.0029999999999999,"5min":1.0580000000000001,"15min":2.242},"last":0.71499999999999997}]},{"osd":6,"last update":"Fri Oct 6 02:30:10 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.77400000000000002,"15min":0.81100000000000005},"min":{"1min":0.625,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.98199999999999998,"5min":1.0489999999999999,"15min":1.357},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.80300000000000005,"15min":0.82499999999999996},"min":{"1min":0.60599999999999998,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.014,"5min":1.0529999999999999,"15min":1.3100000000000001},"last":0.69699999999999995}]},{"osd":7,"last update":"Fri Oct 6 02:30:10 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.78200000000000003,"15min":0.84399999999999997},"min":{"1min":0.57299999999999995,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.88400000000000001,"5min":1.077,"15min":2.27},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.79800000000000004,"15min":0.84399999999999997},"min":{"1min":0.57499999999999996,"5min":0.55600000000000005,"15min":0.53200000000000003},"max":{"1min":1.0029999999999999,"5min":1.0760000000000001,"15min":1.325},"last":0.877}]}]},{"osd":4,"up_from":13,"seq":55834574949,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303156,"kb_used_data":1772,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068684,"statfs":{"total":96636764160,"available":96326332416,"internally_reserved":0,"allocated":1814528,"data_stored":838448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:30:06 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68100000000000005,"15min":0.95799999999999996},"min":{"1min":0.46200000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":12.337},"last":0.751},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68400000000000005,"15min":0.88200000000000001},"min":{"1min":0.495,"5min":0.44400000000000001,"15min":0.438},"max":{"1min":0.95299999999999996,"5min":1.3520000000000001,"15min":9.6539999999999999},"last":0.66200000000000003}]},{"osd":1,"last update":"Fri Oct 6 02:30:06 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.68000000000000005,"15min":0.69399999999999995},"min":{"1min":0.47799999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.042,"5min":1.129,"15min":3.181},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.70999999999999996,"15min":0.98299999999999998},"min":{"1min":0.46300000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.98799999999999999,"5min":0.995,"15min":12.946999999999999},"last":0.69999999999999996}]},{"osd":2,"last update":"Fri Oct 6 02:30:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.71599999999999997,"15min":1.006},"min":{"1min":0.47999999999999998,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.1319999999999999,"5min":1.2989999999999999,"15min":12.377000000000001},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.745,"5min":0.73799999999999999,"15min":0.85599999999999998},"min":{"1min":0.56899999999999995,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.93700000000000006,"5min":1.0529999999999999,"15min":6.3220000000000001},"last":0.626}]},{"osd":3,"last update":"Fri Oct 6 02:30:06 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72899999999999998,"15min":0.95599999999999996},"min":{"1min":0.55300000000000005,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.96299999999999997,"5min":1.5229999999999999,"15min":9.7720000000000002},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71799999999999997,"15min":0.76800000000000002},"min":{"1min":0.54600000000000004,"5min":0.498,"15min":0.47999999999999998},"max":{"1min":0.90700000000000003,"5min":1.212,"15min":3.0840000000000001},"last":0.76700000000000002}]},{"osd":5,"last update":"Fri Oct 6 02:30:06 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.68700000000000006,"15min":0.82899999999999996},"min":{"1min":0.42099999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.99199999999999999,"5min":1.02,"15min":6.2690000000000001},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.745,"5min":0.71199999999999997,"15min":0.997},"min":{"1min":0.45700000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.0589999999999999,"5min":1.4359999999999999,"15min":10.769},"last":0.72699999999999998}]},{"osd":6,"last update":"Fri Oct 6 02:30:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.68799999999999994,"15min":0.753},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.95699999999999996,"5min":1.4590000000000001,"15min":3.1349999999999998},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.76900000000000002,"15min":0.94999999999999996},"min":{"1min":0.47999999999999998,"5min":0.47799999999999998,"15min":0.45700000000000002},"max":{"1min":1.1579999999999999,"5min":1.3839999999999999,"15min":8.0899999999999999},"last":0.79600000000000004}]},{"osd":7,"last update":"Fri Oct 6 02:30:06 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.76800000000000002,"15min":1.032},"min":{"1min":0.50900000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.1870000000000001,"5min":1.494,"15min":9.7089999999999996},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.76600000000000001,"15min":0.97099999999999997},"min":{"1min":0.44400000000000001,"5min":0.374,"15min":0.374},"max":{"1min":1.133,"5min":1.2669999999999999,"15min":7.6719999999999997},"last":0.84899999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574949,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303160,"kb_used_data":1776,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068680,"statfs":{"total":96636764160,"available":96326328320,"internally_reserved":0,"allocated":1818624,"data_stored":844505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:30:04 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63900000000000001,"15min":0.627},"min":{"1min":0.40100000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.88200000000000001,"5min":0.97599999999999998,"15min":1.4279999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.626,"5min":0.66000000000000003,"15min":0.61599999999999999},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.38700000000000001},"max":{"1min":0.92700000000000005,"5min":1.099,"15min":1.099},"last":0.55100000000000005}]},{"osd":1,"last update":"Fri Oct 6 02:30:04 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.67000000000000004,"15min":0.65100000000000002},"min":{"1min":0.44500000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.85599999999999998,"5min":1.0660000000000001,"15min":1.089},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.68999999999999995,"15min":0.67500000000000004},"min":{"1min":0.49099999999999999,"5min":0.40699999999999997,"15min":0.40500000000000003},"max":{"1min":0.93600000000000005,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.65000000000000002}]},{"osd":2,"last update":"Fri Oct 6 02:30:04 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.69799999999999995,"15min":0.67300000000000004},"min":{"1min":0.41299999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.88500000000000001,"5min":1.016,"15min":1.016},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.63,"5min":0.68899999999999995,"15min":0.68100000000000005},"min":{"1min":0.48799999999999999,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.80500000000000005,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.56999999999999995}]},{"osd":3,"last update":"Fri Oct 6 02:30:04 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.752,"15min":0.73199999999999998},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.48799999999999999},"max":{"1min":0.98499999999999999,"5min":1.159,"15min":1.159},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.753,"5min":0.79000000000000004,"15min":0.76900000000000002},"min":{"1min":0.56999999999999995,"5min":0.56200000000000006,"15min":0.54100000000000004},"max":{"1min":0.86799999999999999,"5min":1.194,"15min":1.194},"last":0.62}]},{"osd":4,"last update":"Fri Oct 6 02:30:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.65200000000000002,"15min":0.67500000000000004},"min":{"1min":0.439,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.93000000000000005,"5min":1.012,"15min":1.1499999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.66600000000000004,"15min":0.68500000000000005},"min":{"1min":0.498,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.92700000000000005,"5min":1.25,"15min":1.25},"last":0.60699999999999998}]},{"osd":6,"last update":"Fri Oct 6 02:30:04 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.72799999999999998,"15min":0.74299999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.40300000000000002},"max":{"1min":0.95699999999999996,"5min":1.0549999999999999,"15min":1.1719999999999999},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.80100000000000005,"15min":0.81899999999999995},"min":{"1min":0.502,"5min":0.373,"15min":0.373},"max":{"1min":0.98299999999999998,"5min":1.1200000000000001,"15min":1.2170000000000001},"last":0.755}]},{"osd":7,"last update":"Fri Oct 6 02:30:04 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.747,"15min":0.76500000000000001},"min":{"1min":0.57199999999999995,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.0169999999999999,"5min":1.23,"15min":1.23},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79800000000000004,"15min":0.82899999999999996},"min":{"1min":0.58499999999999996,"5min":0.437,"15min":0.437},"max":{"1min":0.93300000000000005,"5min":1.1040000000000001,"15min":1.5640000000000001},"last":0.69499999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T02:30:56.507 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T02:30:56.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.694+0000 7f6ce0b9d700 1 Processor -- start 2023-10-06T02:30:56.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.694+0000 7f6ce0b9d700 1 -- start start 2023-10-06T02:30:56.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.695+0000 7f6ce0b9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f6cdc153c30 0x7f6cdc15a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:56.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.695+0000 7f6ce0b9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6cdc0cf110 0x7f6cdc15c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:56.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.695+0000 7f6ce0b9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cdc15afd0 con 0x7f6cdc15de50 2023-10-06T02:30:56.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.695+0000 7f6ce0b9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cdc15b2b0 con 0x7f6cdc0cf110 2023-10-06T02:30:56.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.695+0000 7f6ce0b9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cdc15b590 con 0x7f6cdc153c30 2023-10-06T02:30:56.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.695+0000 7f6cdbfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6cdc0cf110 0x7f6cdc15c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:56.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.695+0000 7f6cdbfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6cdc0cf110 0x7f6cdc15c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54806/0 (socket says 172.21.15.31:54806) 2023-10-06T02:30:56.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.695+0000 7f6cdbfff700 1 -- 172.21.15.31:0/1745404182 learned_addr learned my addr 172.21.15.31:0/1745404182 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:56.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.696+0000 7f6cdb7fe700 1 --2- 172.21.15.31:0/1745404182 >> v2:172.21.15.62:3300/0 conn(0x7f6cdc153c30 0x7f6cdc15a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:56.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.696+0000 7f6cdbfff700 1 -- 172.21.15.31:0/1745404182 >> v2:172.21.15.62:3300/0 conn(0x7f6cdc153c30 msgr2=0x7f6cdc15a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:56.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.696+0000 7f6cdbfff700 1 --2- 172.21.15.31:0/1745404182 >> v2:172.21.15.62:3300/0 conn(0x7f6cdc153c30 0x7f6cdc15a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:56.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.696+0000 7f6cdbfff700 1 -- 172.21.15.31:0/1745404182 >> v1:172.21.15.62:6789/0 conn(0x7f6cdc15de50 legacy=0x7f6cdc0cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:56.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.696+0000 7f6cdbfff700 1 -- 172.21.15.31:0/1745404182 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cdc15b870 con 0x7f6cdc0cf110 2023-10-06T02:30:56.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.696+0000 7f6cdbfff700 1 --2- 172.21.15.31:0/1745404182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cdc0cf110 0x7f6cdc15c070 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc06ead0 tx=0x7f6cd4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b382cb22d1c46671 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:56.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.696+0000 7f6cda7fc700 1 -- 172.21.15.31:0/1745404182 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd400e070 con 0x7f6cdc0cf110 2023-10-06T02:30:56.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.697+0000 7f6cda7fc700 1 -- 172.21.15.31:0/1745404182 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cd400ba20 con 0x7f6cdc0cf110 2023-10-06T02:30:56.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.697+0000 7f6cda7fc700 1 -- 172.21.15.31:0/1745404182 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd40092e0 con 0x7f6cdc0cf110 2023-10-06T02:30:56.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.697+0000 7f6ce0b9d700 1 -- 172.21.15.31:0/1745404182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cdc0cf110 msgr2=0x7f6cdc15c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:56.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.697+0000 7f6ce0b9d700 1 --2- 172.21.15.31:0/1745404182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cdc0cf110 0x7f6cdc15c070 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc06ead0 tx=0x7f6cd4009840 comp rx=0 tx=0).stop 2023-10-06T02:30:56.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.697+0000 7f6ce0b9d700 1 -- 172.21.15.31:0/1745404182 shutdown_connections 2023-10-06T02:30:56.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.697+0000 7f6ce0b9d700 1 --2- 172.21.15.31:0/1745404182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cdc0cf110 0x7f6cdc15c070 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:56.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.697+0000 7f6ce0b9d700 1 --2- 172.21.15.31:0/1745404182 >> v2:172.21.15.62:3300/0 conn(0x7f6cdc153c30 0x7f6cdc15a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:56.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.697+0000 7f6ce0b9d700 1 -- 172.21.15.31:0/1745404182 >> 172.21.15.31:0/1745404182 conn(0x7f6cdc0703a0 msgr2=0x7f6cdc0707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:56.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.697+0000 7f6ce0b9d700 1 -- 172.21.15.31:0/1745404182 shutdown_connections 2023-10-06T02:30:56.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.697+0000 7f6ce0b9d700 1 -- 172.21.15.31:0/1745404182 wait complete. 2023-10-06T02:30:56.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.699+0000 7f6ce0b9d700 1 Processor -- start 2023-10-06T02:30:56.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.699+0000 7f6ce0b9d700 1 -- start start 2023-10-06T02:30:56.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.700+0000 7f6ce0b9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6cdc0cf110 0x7f6cdc0c96b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:56.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.700+0000 7f6ce0b9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6cdc153c30 0x7f6cdc16c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:56.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.700+0000 7f6ce0b9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cdc15de50 0x7f6cdc0c52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:56.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.700+0000 7f6ce0b9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cdc0c9bf0 con 0x7f6cdc15de50 2023-10-06T02:30:56.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.700+0000 7f6ce0b9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cdc0b4450 con 0x7f6cdc153c30 2023-10-06T02:30:56.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.700+0000 7f6ce0b9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cdc0b3840 con 0x7f6cdc0cf110 2023-10-06T02:30:56.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.700+0000 7f6cdbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cdc15de50 0x7f6cdc0c52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:56.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.700+0000 7f6cdbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cdc15de50 0x7f6cdc0c52d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54816/0 (socket says 172.21.15.31:54816) 2023-10-06T02:30:56.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.700+0000 7f6cdbfff700 1 -- 172.21.15.31:0/3794681954 learned_addr learned my addr 172.21.15.31:0/3794681954 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:56.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.700+0000 7f6cdb7fe700 1 --2- 172.21.15.31:0/3794681954 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6cdc0cf110 0x7f6cdc0c96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:56.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.700+0000 7f6cdaffd700 1 --2- 172.21.15.31:0/3794681954 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6cdc153c30 0x7f6cdc16c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:56.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.700+0000 7f6cdbfff700 1 -- 172.21.15.31:0/3794681954 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6cdc0cf110 msgr2=0x7f6cdc0c96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:56.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.700+0000 7f6cdbfff700 1 --2- 172.21.15.31:0/3794681954 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6cdc0cf110 0x7f6cdc0c96b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:56.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.700+0000 7f6cdbfff700 1 -- 172.21.15.31:0/3794681954 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6cdc153c30 msgr2=0x7f6cdc16c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:56.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.700+0000 7f6cdbfff700 1 --2- 172.21.15.31:0/3794681954 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6cdc153c30 0x7f6cdc16c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:56.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.700+0000 7f6cdbfff700 1 -- 172.21.15.31:0/3794681954 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cd4009ae0 con 0x7f6cdc15de50 2023-10-06T02:30:56.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.700+0000 7f6cdb7fe700 1 --2- 172.21.15.31:0/3794681954 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6cdc0cf110 0x7f6cdc0c96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:30:56.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.700+0000 7f6cdaffd700 1 --2- 172.21.15.31:0/3794681954 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6cdc153c30 0x7f6cdc16c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:30:56.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.701+0000 7f6cdbfff700 1 --2- 172.21.15.31:0/3794681954 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cdc15de50 0x7f6cdc0c52d0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f6cd40072f0 tx=0x7f6cd4018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:56.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.701+0000 7f6cc37fe700 1 -- 172.21.15.31:0/3794681954 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd4008c60 con 0x7f6cdc15de50 2023-10-06T02:30:56.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.701+0000 7f6cc37fe700 1 -- 172.21.15.31:0/3794681954 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cd4008dc0 con 0x7f6cdc15de50 2023-10-06T02:30:56.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.701+0000 7f6cc37fe700 1 -- 172.21.15.31:0/3794681954 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd40266b0 con 0x7f6cdc15de50 2023-10-06T02:30:56.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.701+0000 7f6ce0b9d700 1 -- 172.21.15.31:0/3794681954 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cdc0b3b20 con 0x7f6cdc15de50 2023-10-06T02:30:56.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.701+0000 7f6ce0b9d700 1 -- 172.21.15.31:0/3794681954 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cdc0b40e0 con 0x7f6cdc15de50 2023-10-06T02:30:56.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.702+0000 7f6ce0b9d700 1 -- 172.21.15.31:0/3794681954 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cdc04c3c0 con 0x7f6cdc15de50 2023-10-06T02:30:56.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.703+0000 7f6cc37fe700 1 -- 172.21.15.31:0/3794681954 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f6cd4003040 con 0x7f6cdc15de50 2023-10-06T02:30:56.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.704+0000 7f6cc37fe700 1 --2- 172.21.15.31:0/3794681954 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6cc80674d0 0x7f6cc8069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:56.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.704+0000 7f6cc37fe700 1 -- 172.21.15.31:0/3794681954 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f6cd4087230 con 0x7f6cdc15de50 2023-10-06T02:30:56.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.704+0000 7f6cdb7fe700 1 --2- 172.21.15.31:0/3794681954 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6cc80674d0 0x7f6cc8069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:56.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.705+0000 7f6cdb7fe700 1 --2- 172.21.15.31:0/3794681954 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6cc80674d0 0x7f6cc8069980 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc009d80 tx=0x7f6ccc008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:56.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.708+0000 7f6cc37fe700 1 -- 172.21.15.31:0/3794681954 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6cd4053960 con 0x7f6cdc15de50 2023-10-06T02:30:56.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.906+0000 7f6ce0b9d700 1 -- 172.21.15.31:0/3794681954 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6cdc15bb50 con 0x7f6cc80674d0 2023-10-06T02:30:56.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.916+0000 7f6cc37fe700 1 -- 172.21.15.31:0/3794681954 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311352 (secure 0 0 0) 0x7f6cdc15bb50 con 0x7f6cc80674d0 2023-10-06T02:30:56.920 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:30:56.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.920+0000 7f6ce0b9d700 1 -- 172.21.15.31:0/3794681954 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6cc80674d0 msgr2=0x7f6cc8069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:56.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.920+0000 7f6ce0b9d700 1 --2- 172.21.15.31:0/3794681954 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6cc80674d0 0x7f6cc8069980 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc009d80 tx=0x7f6ccc008040 comp rx=0 tx=0).stop 2023-10-06T02:30:56.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.920+0000 7f6ce0b9d700 1 -- 172.21.15.31:0/3794681954 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cdc15de50 msgr2=0x7f6cdc0c52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:56.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.920+0000 7f6ce0b9d700 1 --2- 172.21.15.31:0/3794681954 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cdc15de50 0x7f6cdc0c52d0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f6cd40072f0 tx=0x7f6cd4018000 comp rx=0 tx=0).stop 2023-10-06T02:30:56.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.920+0000 7f6ce0b9d700 1 -- 172.21.15.31:0/3794681954 shutdown_connections 2023-10-06T02:30:56.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.920+0000 7f6ce0b9d700 1 --2- 172.21.15.31:0/3794681954 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cdc15de50 0x7f6cdc0c52d0 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:56.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.920+0000 7f6ce0b9d700 1 --2- 172.21.15.31:0/3794681954 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6cc80674d0 0x7f6cc8069980 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:56.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.920+0000 7f6ce0b9d700 1 --2- 172.21.15.31:0/3794681954 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6cdc0cf110 0x7f6cdc0c96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:56.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.920+0000 7f6ce0b9d700 1 --2- 172.21.15.31:0/3794681954 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6cdc153c30 0x7f6cdc16c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:56.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.920+0000 7f6ce0b9d700 1 -- 172.21.15.31:0/3794681954 >> 172.21.15.31:0/3794681954 conn(0x7f6cdc0703a0 msgr2=0x7f6cdc152fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:56.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.920+0000 7f6ce0b9d700 1 -- 172.21.15.31:0/3794681954 shutdown_connections 2023-10-06T02:30:56.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:56.920+0000 7f6ce0b9d700 1 -- 172.21.15.31:0/3794681954 wait complete. 2023-10-06T02:30:56.924 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-10-06T02:30:56.938 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":171,"stamp":"2023-10-06T02:30:56.504001+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":223,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":166,"ondisk_log_size":166,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":169,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2426404,"kb_used_data":15332,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752548316,"statfs":{"total":773094113280,"available":770609475584,"internally_reserved":0,"allocated":15699968,"data_stored":7920389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001387"},"pg_stats":[{"pgid":"24.3f","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.677664+0000","last_change":"2023-10-06T02:30:51.677664+0000","last_active":"2023-10-06T02:30:51.677664+0000","last_peered":"2023-10-06T02:30:51.677664+0000","last_clean":"2023-10-06T02:30:51.677664+0000","last_became_active":"2023-10-06T02:30:51.674070+0000","last_became_peered":"2023-10-06T02:30:51.674070+0000","last_unstale":"2023-10-06T02:30:51.677664+0000","last_undegraded":"2023-10-06T02:30:51.677664+0000","last_fullsized":"2023-10-06T02:30:51.677664+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:15:10.732838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3e","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.680336+0000","last_change":"2023-10-06T02:30:51.675712+0000","last_active":"2023-10-06T02:30:51.680336+0000","last_peered":"2023-10-06T02:30:51.680336+0000","last_clean":"2023-10-06T02:30:51.680336+0000","last_became_active":"2023-10-06T02:30:51.675164+0000","last_became_peered":"2023-10-06T02:30:51.675164+0000","last_unstale":"2023-10-06T02:30:51.680336+0000","last_undegraded":"2023-10-06T02:30:51.680336+0000","last_fullsized":"2023-10-06T02:30:51.680336+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:15:14.213699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3d","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.682966+0000","last_change":"2023-10-06T02:30:51.682966+0000","last_active":"2023-10-06T02:30:51.682966+0000","last_peered":"2023-10-06T02:30:51.682966+0000","last_clean":"2023-10-06T02:30:51.682966+0000","last_became_active":"2023-10-06T02:30:51.682495+0000","last_became_peered":"2023-10-06T02:30:51.682495+0000","last_unstale":"2023-10-06T02:30:51.682966+0000","last_undegraded":"2023-10-06T02:30:51.682966+0000","last_fullsized":"2023-10-06T02:30:51.682966+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:27:26.681762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3c","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.677717+0000","last_change":"2023-10-06T02:30:51.677717+0000","last_active":"2023-10-06T02:30:51.677717+0000","last_peered":"2023-10-06T02:30:51.677717+0000","last_clean":"2023-10-06T02:30:51.677717+0000","last_became_active":"2023-10-06T02:30:51.677415+0000","last_became_peered":"2023-10-06T02:30:51.677415+0000","last_unstale":"2023-10-06T02:30:51.677717+0000","last_undegraded":"2023-10-06T02:30:51.677717+0000","last_fullsized":"2023-10-06T02:30:51.677717+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:13:59.000364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3b","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.671702+0000","last_change":"2023-10-06T02:30:51.669101+0000","last_active":"2023-10-06T02:30:51.671702+0000","last_peered":"2023-10-06T02:30:51.671702+0000","last_clean":"2023-10-06T02:30:51.671702+0000","last_became_active":"2023-10-06T02:30:51.668833+0000","last_became_peered":"2023-10-06T02:30:51.668833+0000","last_unstale":"2023-10-06T02:30:51.671702+0000","last_undegraded":"2023-10-06T02:30:51.671702+0000","last_fullsized":"2023-10-06T02:30:51.671702+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:56:38.263976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3a","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.675549+0000","last_change":"2023-10-06T02:30:51.670883+0000","last_active":"2023-10-06T02:30:51.675549+0000","last_peered":"2023-10-06T02:30:51.675549+0000","last_clean":"2023-10-06T02:30:51.675549+0000","last_became_active":"2023-10-06T02:30:51.670589+0000","last_became_peered":"2023-10-06T02:30:51.670589+0000","last_unstale":"2023-10-06T02:30:51.675549+0000","last_undegraded":"2023-10-06T02:30:51.675549+0000","last_fullsized":"2023-10-06T02:30:51.675549+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:15:15.716156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.39","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.672548+0000","last_change":"2023-10-06T02:30:51.665493+0000","last_active":"2023-10-06T02:30:51.672548+0000","last_peered":"2023-10-06T02:30:51.672548+0000","last_clean":"2023-10-06T02:30:51.672548+0000","last_became_active":"2023-10-06T02:30:51.664664+0000","last_became_peered":"2023-10-06T02:30:51.664664+0000","last_unstale":"2023-10-06T02:30:51.672548+0000","last_undegraded":"2023-10-06T02:30:51.672548+0000","last_fullsized":"2023-10-06T02:30:51.672548+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:04:59.566737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.38","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.675824+0000","last_change":"2023-10-06T02:30:51.670371+0000","last_active":"2023-10-06T02:30:51.675824+0000","last_peered":"2023-10-06T02:30:51.675824+0000","last_clean":"2023-10-06T02:30:51.675824+0000","last_became_active":"2023-10-06T02:30:51.669441+0000","last_became_peered":"2023-10-06T02:30:51.669441+0000","last_unstale":"2023-10-06T02:30:51.675824+0000","last_undegraded":"2023-10-06T02:30:51.675824+0000","last_fullsized":"2023-10-06T02:30:51.675824+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:50:09.735917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.37","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.675714+0000","last_change":"2023-10-06T02:30:51.670562+0000","last_active":"2023-10-06T02:30:51.675714+0000","last_peered":"2023-10-06T02:30:51.675714+0000","last_clean":"2023-10-06T02:30:51.675714+0000","last_became_active":"2023-10-06T02:30:51.670274+0000","last_became_peered":"2023-10-06T02:30:51.670274+0000","last_unstale":"2023-10-06T02:30:51.675714+0000","last_undegraded":"2023-10-06T02:30:51.675714+0000","last_fullsized":"2023-10-06T02:30:51.675714+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:15:10.732838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.36","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.676073+0000","last_change":"2023-10-06T02:30:51.673118+0000","last_active":"2023-10-06T02:30:51.676073+0000","last_peered":"2023-10-06T02:30:51.676073+0000","last_clean":"2023-10-06T02:30:51.676073+0000","last_became_active":"2023-10-06T02:30:51.672802+0000","last_became_peered":"2023-10-06T02:30:51.672802+0000","last_unstale":"2023-10-06T02:30:51.676073+0000","last_undegraded":"2023-10-06T02:30:51.676073+0000","last_fullsized":"2023-10-06T02:30:51.676073+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:13:59.000364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.35","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.677497+0000","last_change":"2023-10-06T02:30:51.673626+0000","last_active":"2023-10-06T02:30:51.677497+0000","last_peered":"2023-10-06T02:30:51.677497+0000","last_clean":"2023-10-06T02:30:51.677497+0000","last_became_active":"2023-10-06T02:30:51.673136+0000","last_became_peered":"2023-10-06T02:30:51.673136+0000","last_unstale":"2023-10-06T02:30:51.677497+0000","last_undegraded":"2023-10-06T02:30:51.677497+0000","last_fullsized":"2023-10-06T02:30:51.677497+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:20:07.242511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.34","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.677007+0000","last_change":"2023-10-06T02:30:51.671300+0000","last_active":"2023-10-06T02:30:51.677007+0000","last_peered":"2023-10-06T02:30:51.677007+0000","last_clean":"2023-10-06T02:30:51.677007+0000","last_became_active":"2023-10-06T02:30:51.670975+0000","last_became_peered":"2023-10-06T02:30:51.670975+0000","last_unstale":"2023-10-06T02:30:51.677007+0000","last_undegraded":"2023-10-06T02:30:51.677007+0000","last_fullsized":"2023-10-06T02:30:51.677007+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:21:59.543623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.33","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.674045+0000","last_change":"2023-10-06T02:30:51.674045+0000","last_active":"2023-10-06T02:30:51.674045+0000","last_peered":"2023-10-06T02:30:51.674045+0000","last_clean":"2023-10-06T02:30:51.674045+0000","last_became_active":"2023-10-06T02:30:51.673496+0000","last_became_peered":"2023-10-06T02:30:51.673496+0000","last_unstale":"2023-10-06T02:30:51.674045+0000","last_undegraded":"2023-10-06T02:30:51.674045+0000","last_fullsized":"2023-10-06T02:30:51.674045+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:22:16.120421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.32","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.670659+0000","last_change":"2023-10-06T02:30:51.666903+0000","last_active":"2023-10-06T02:30:51.670659+0000","last_peered":"2023-10-06T02:30:51.670659+0000","last_clean":"2023-10-06T02:30:51.670659+0000","last_became_active":"2023-10-06T02:30:51.666639+0000","last_became_peered":"2023-10-06T02:30:51.666639+0000","last_unstale":"2023-10-06T02:30:51.670659+0000","last_undegraded":"2023-10-06T02:30:51.670659+0000","last_fullsized":"2023-10-06T02:30:51.670659+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:50:51.184941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.31","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.674353+0000","last_change":"2023-10-06T02:30:51.662058+0000","last_active":"2023-10-06T02:30:51.674353+0000","last_peered":"2023-10-06T02:30:51.674353+0000","last_clean":"2023-10-06T02:30:51.674353+0000","last_became_active":"2023-10-06T02:30:51.660270+0000","last_became_peered":"2023-10-06T02:30:51.660270+0000","last_unstale":"2023-10-06T02:30:51.674353+0000","last_undegraded":"2023-10-06T02:30:51.674353+0000","last_fullsized":"2023-10-06T02:30:51.674353+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:04:04.504486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.30","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.679630+0000","last_change":"2023-10-06T02:30:51.679630+0000","last_active":"2023-10-06T02:30:51.679630+0000","last_peered":"2023-10-06T02:30:51.679630+0000","last_clean":"2023-10-06T02:30:51.679630+0000","last_became_active":"2023-10-06T02:30:51.678736+0000","last_became_peered":"2023-10-06T02:30:51.678736+0000","last_unstale":"2023-10-06T02:30:51.679630+0000","last_undegraded":"2023-10-06T02:30:51.679630+0000","last_fullsized":"2023-10-06T02:30:51.679630+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:39:22.478358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.2f","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.670682+0000","last_change":"2023-10-06T02:30:51.659731+0000","last_active":"2023-10-06T02:30:51.670682+0000","last_peered":"2023-10-06T02:30:51.670682+0000","last_clean":"2023-10-06T02:30:51.670682+0000","last_became_active":"2023-10-06T02:30:51.659420+0000","last_became_peered":"2023-10-06T02:30:51.659420+0000","last_unstale":"2023-10-06T02:30:51.670682+0000","last_undegraded":"2023-10-06T02:30:51.670682+0000","last_fullsized":"2023-10-06T02:30:51.670682+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:27:54.865078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2e","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.677114+0000","last_change":"2023-10-06T02:30:51.675616+0000","last_active":"2023-10-06T02:30:51.677114+0000","last_peered":"2023-10-06T02:30:51.677114+0000","last_clean":"2023-10-06T02:30:51.677114+0000","last_became_active":"2023-10-06T02:30:51.675266+0000","last_became_peered":"2023-10-06T02:30:51.675266+0000","last_unstale":"2023-10-06T02:30:51.677114+0000","last_undegraded":"2023-10-06T02:30:51.677114+0000","last_fullsized":"2023-10-06T02:30:51.677114+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:12:24.760338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.2d","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.673493+0000","last_change":"2023-10-06T02:30:51.661855+0000","last_active":"2023-10-06T02:30:51.673493+0000","last_peered":"2023-10-06T02:30:51.673493+0000","last_clean":"2023-10-06T02:30:51.673493+0000","last_became_active":"2023-10-06T02:30:51.660147+0000","last_became_peered":"2023-10-06T02:30:51.660147+0000","last_unstale":"2023-10-06T02:30:51.673493+0000","last_undegraded":"2023-10-06T02:30:51.673493+0000","last_fullsized":"2023-10-06T02:30:51.673493+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:46:15.913987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.2c","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.681982+0000","last_change":"2023-10-06T02:30:51.673971+0000","last_active":"2023-10-06T02:30:51.681982+0000","last_peered":"2023-10-06T02:30:51.681982+0000","last_clean":"2023-10-06T02:30:51.681982+0000","last_became_active":"2023-10-06T02:30:51.673710+0000","last_became_peered":"2023-10-06T02:30:51.673710+0000","last_unstale":"2023-10-06T02:30:51.681982+0000","last_undegraded":"2023-10-06T02:30:51.681982+0000","last_fullsized":"2023-10-06T02:30:51.681982+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:50:37.585095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.2b","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.673815+0000","last_change":"2023-10-06T02:30:51.673815+0000","last_active":"2023-10-06T02:30:51.673815+0000","last_peered":"2023-10-06T02:30:51.673815+0000","last_clean":"2023-10-06T02:30:51.673815+0000","last_became_active":"2023-10-06T02:30:51.673116+0000","last_became_peered":"2023-10-06T02:30:51.673116+0000","last_unstale":"2023-10-06T02:30:51.673815+0000","last_undegraded":"2023-10-06T02:30:51.673815+0000","last_fullsized":"2023-10-06T02:30:51.673815+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:24:36.962341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2a","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.675529+0000","last_change":"2023-10-06T02:30:51.669380+0000","last_active":"2023-10-06T02:30:51.675529+0000","last_peered":"2023-10-06T02:30:51.675529+0000","last_clean":"2023-10-06T02:30:51.675529+0000","last_became_active":"2023-10-06T02:30:51.669075+0000","last_became_peered":"2023-10-06T02:30:51.669075+0000","last_unstale":"2023-10-06T02:30:51.675529+0000","last_undegraded":"2023-10-06T02:30:51.675529+0000","last_fullsized":"2023-10-06T02:30:51.675529+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:53:10.241998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.29","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.676976+0000","last_change":"2023-10-06T02:30:51.671846+0000","last_active":"2023-10-06T02:30:51.676976+0000","last_peered":"2023-10-06T02:30:51.676976+0000","last_clean":"2023-10-06T02:30:51.676976+0000","last_became_active":"2023-10-06T02:30:51.671563+0000","last_became_peered":"2023-10-06T02:30:51.671563+0000","last_unstale":"2023-10-06T02:30:51.676976+0000","last_undegraded":"2023-10-06T02:30:51.676976+0000","last_fullsized":"2023-10-06T02:30:51.676976+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:01:17.186989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.28","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.679466+0000","last_change":"2023-10-06T02:30:51.679466+0000","last_active":"2023-10-06T02:30:51.679466+0000","last_peered":"2023-10-06T02:30:51.679466+0000","last_clean":"2023-10-06T02:30:51.679466+0000","last_became_active":"2023-10-06T02:30:51.678760+0000","last_became_peered":"2023-10-06T02:30:51.678760+0000","last_unstale":"2023-10-06T02:30:51.679466+0000","last_undegraded":"2023-10-06T02:30:51.679466+0000","last_fullsized":"2023-10-06T02:30:51.679466+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:45:16.352909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.27","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.674321+0000","last_change":"2023-10-06T02:30:51.674321+0000","last_active":"2023-10-06T02:30:51.674321+0000","last_peered":"2023-10-06T02:30:51.674321+0000","last_clean":"2023-10-06T02:30:51.674321+0000","last_became_active":"2023-10-06T02:30:51.674038+0000","last_became_peered":"2023-10-06T02:30:51.674038+0000","last_unstale":"2023-10-06T02:30:51.674321+0000","last_undegraded":"2023-10-06T02:30:51.674321+0000","last_fullsized":"2023-10-06T02:30:51.674321+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:20:18.598786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.26","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.681757+0000","last_change":"2023-10-06T02:30:51.669654+0000","last_active":"2023-10-06T02:30:51.681757+0000","last_peered":"2023-10-06T02:30:51.681757+0000","last_clean":"2023-10-06T02:30:51.681757+0000","last_became_active":"2023-10-06T02:30:51.669314+0000","last_became_peered":"2023-10-06T02:30:51.669314+0000","last_unstale":"2023-10-06T02:30:51.681757+0000","last_undegraded":"2023-10-06T02:30:51.681757+0000","last_fullsized":"2023-10-06T02:30:51.681757+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:30:16.718924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.25","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.680380+0000","last_change":"2023-10-06T02:30:51.675681+0000","last_active":"2023-10-06T02:30:51.680380+0000","last_peered":"2023-10-06T02:30:51.680380+0000","last_clean":"2023-10-06T02:30:51.680380+0000","last_became_active":"2023-10-06T02:30:51.672584+0000","last_became_peered":"2023-10-06T02:30:51.672584+0000","last_unstale":"2023-10-06T02:30:51.680380+0000","last_undegraded":"2023-10-06T02:30:51.680380+0000","last_fullsized":"2023-10-06T02:30:51.680380+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:08:49.141270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.24","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.672631+0000","last_change":"2023-10-06T02:30:51.672631+0000","last_active":"2023-10-06T02:30:51.672631+0000","last_peered":"2023-10-06T02:30:51.672631+0000","last_clean":"2023-10-06T02:30:51.672631+0000","last_became_active":"2023-10-06T02:30:51.672353+0000","last_became_peered":"2023-10-06T02:30:51.672353+0000","last_unstale":"2023-10-06T02:30:51.672631+0000","last_undegraded":"2023-10-06T02:30:51.672631+0000","last_fullsized":"2023-10-06T02:30:51.672631+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:45:54.859704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.23","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.680329+0000","last_change":"2023-10-06T02:30:51.676512+0000","last_active":"2023-10-06T02:30:51.680329+0000","last_peered":"2023-10-06T02:30:51.680329+0000","last_clean":"2023-10-06T02:30:51.680329+0000","last_became_active":"2023-10-06T02:30:51.676213+0000","last_became_peered":"2023-10-06T02:30:51.676213+0000","last_unstale":"2023-10-06T02:30:51.680329+0000","last_undegraded":"2023-10-06T02:30:51.680329+0000","last_fullsized":"2023-10-06T02:30:51.680329+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:06:59.430925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.22","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.679708+0000","last_change":"2023-10-06T02:30:51.678340+0000","last_active":"2023-10-06T02:30:51.679708+0000","last_peered":"2023-10-06T02:30:51.679708+0000","last_clean":"2023-10-06T02:30:51.679708+0000","last_became_active":"2023-10-06T02:30:51.678056+0000","last_became_peered":"2023-10-06T02:30:51.678056+0000","last_unstale":"2023-10-06T02:30:51.679708+0000","last_undegraded":"2023-10-06T02:30:51.679708+0000","last_fullsized":"2023-10-06T02:30:51.679708+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:50:37.585095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.21","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.677539+0000","last_change":"2023-10-06T02:30:51.674249+0000","last_active":"2023-10-06T02:30:51.677539+0000","last_peered":"2023-10-06T02:30:51.677539+0000","last_clean":"2023-10-06T02:30:51.677539+0000","last_became_active":"2023-10-06T02:30:51.673942+0000","last_became_peered":"2023-10-06T02:30:51.673942+0000","last_unstale":"2023-10-06T02:30:51.677539+0000","last_undegraded":"2023-10-06T02:30:51.677539+0000","last_fullsized":"2023-10-06T02:30:51.677539+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:00:18.060628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.20","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.682678+0000","last_change":"2023-10-06T02:30:51.681496+0000","last_active":"2023-10-06T02:30:51.682678+0000","last_peered":"2023-10-06T02:30:51.682678+0000","last_clean":"2023-10-06T02:30:51.682678+0000","last_became_active":"2023-10-06T02:30:51.681157+0000","last_became_peered":"2023-10-06T02:30:51.681157+0000","last_unstale":"2023-10-06T02:30:51.682678+0000","last_undegraded":"2023-10-06T02:30:51.682678+0000","last_fullsized":"2023-10-06T02:30:51.682678+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:52:30.458923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.677131+0000","last_change":"2023-10-06T02:30:51.675021+0000","last_active":"2023-10-06T02:30:51.677131+0000","last_peered":"2023-10-06T02:30:51.677131+0000","last_clean":"2023-10-06T02:30:51.677131+0000","last_became_active":"2023-10-06T02:30:51.674717+0000","last_became_peered":"2023-10-06T02:30:51.674717+0000","last_unstale":"2023-10-06T02:30:51.677131+0000","last_undegraded":"2023-10-06T02:30:51.677131+0000","last_fullsized":"2023-10-06T02:30:51.677131+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:34:36.018098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"143'4","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:52.094943+0000","last_change":"2023-10-06T02:30:50.674860+0000","last_active":"2023-10-06T02:30:52.094943+0000","last_peered":"2023-10-06T02:30:52.094943+0000","last_clean":"2023-10-06T02:30:52.094943+0000","last_became_active":"2023-10-06T02:30:50.674477+0000","last_became_peered":"2023-10-06T02:30:50.674477+0000","last_unstale":"2023-10-06T02:30:52.094943+0000","last_undegraded":"2023-10-06T02:30:52.094943+0000","last_fullsized":"2023-10-06T02:30:52.094943+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_clean_scrub_stamp":"2023-10-06T02:30:49.637381+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:54:31.897719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.671148+0000","last_change":"2023-10-06T02:30:51.671148+0000","last_active":"2023-10-06T02:30:51.671148+0000","last_peered":"2023-10-06T02:30:51.671148+0000","last_clean":"2023-10-06T02:30:51.671148+0000","last_became_active":"2023-10-06T02:30:51.670838+0000","last_became_peered":"2023-10-06T02:30:51.670838+0000","last_unstale":"2023-10-06T02:30:51.671148+0000","last_undegraded":"2023-10-06T02:30:51.671148+0000","last_fullsized":"2023-10-06T02:30:51.671148+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:19:08.717427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"143'1","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:52.099776+0000","last_change":"2023-10-06T02:30:50.679429+0000","last_active":"2023-10-06T02:30:52.099776+0000","last_peered":"2023-10-06T02:30:52.099776+0000","last_clean":"2023-10-06T02:30:52.099776+0000","last_became_active":"2023-10-06T02:30:50.679166+0000","last_became_peered":"2023-10-06T02:30:50.679166+0000","last_unstale":"2023-10-06T02:30:52.099776+0000","last_undegraded":"2023-10-06T02:30:52.099776+0000","last_fullsized":"2023-10-06T02:30:52.099776+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_clean_scrub_stamp":"2023-10-06T02:30:49.637381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:40:20.143016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.671598+0000","last_change":"2023-10-06T02:30:51.667929+0000","last_active":"2023-10-06T02:30:51.671598+0000","last_peered":"2023-10-06T02:30:51.671598+0000","last_clean":"2023-10-06T02:30:51.671598+0000","last_became_active":"2023-10-06T02:30:51.667670+0000","last_became_peered":"2023-10-06T02:30:51.667670+0000","last_unstale":"2023-10-06T02:30:51.671598+0000","last_undegraded":"2023-10-06T02:30:51.671598+0000","last_fullsized":"2023-10-06T02:30:51.671598+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:45:51.919765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1d","version":"143'2","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:52.096386+0000","last_change":"2023-10-06T02:30:50.672392+0000","last_active":"2023-10-06T02:30:52.096386+0000","last_peered":"2023-10-06T02:30:52.096386+0000","last_clean":"2023-10-06T02:30:52.096386+0000","last_became_active":"2023-10-06T02:30:50.672091+0000","last_became_peered":"2023-10-06T02:30:50.672091+0000","last_unstale":"2023-10-06T02:30:52.096386+0000","last_undegraded":"2023-10-06T02:30:52.096386+0000","last_fullsized":"2023-10-06T02:30:52.096386+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_clean_scrub_stamp":"2023-10-06T02:30:49.637381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:43:26.111876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.677895+0000","last_change":"2023-10-06T02:30:51.674762+0000","last_active":"2023-10-06T02:30:51.677895+0000","last_peered":"2023-10-06T02:30:51.677895+0000","last_clean":"2023-10-06T02:30:51.677895+0000","last_became_active":"2023-10-06T02:30:51.672125+0000","last_became_peered":"2023-10-06T02:30:51.672125+0000","last_unstale":"2023-10-06T02:30:51.677895+0000","last_undegraded":"2023-10-06T02:30:51.677895+0000","last_fullsized":"2023-10-06T02:30:51.677895+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:24:42.744456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1c","version":"143'2","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:52.096776+0000","last_change":"2023-10-06T02:30:50.666494+0000","last_active":"2023-10-06T02:30:52.096776+0000","last_peered":"2023-10-06T02:30:52.096776+0000","last_clean":"2023-10-06T02:30:52.096776+0000","last_became_active":"2023-10-06T02:30:50.666203+0000","last_became_peered":"2023-10-06T02:30:50.666203+0000","last_unstale":"2023-10-06T02:30:52.096776+0000","last_undegraded":"2023-10-06T02:30:52.096776+0000","last_fullsized":"2023-10-06T02:30:52.096776+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_clean_scrub_stamp":"2023-10-06T02:30:49.637381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:37:54.167657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.675779+0000","last_change":"2023-10-06T02:30:51.672331+0000","last_active":"2023-10-06T02:30:51.675779+0000","last_peered":"2023-10-06T02:30:51.675779+0000","last_clean":"2023-10-06T02:30:51.675779+0000","last_became_active":"2023-10-06T02:30:51.671546+0000","last_became_peered":"2023-10-06T02:30:51.671546+0000","last_unstale":"2023-10-06T02:30:51.675779+0000","last_undegraded":"2023-10-06T02:30:51.675779+0000","last_fullsized":"2023-10-06T02:30:51.675779+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:39:22.478358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1b","version":"143'1","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:52.100083+0000","last_change":"2023-10-06T02:30:50.682246+0000","last_active":"2023-10-06T02:30:52.100083+0000","last_peered":"2023-10-06T02:30:52.100083+0000","last_clean":"2023-10-06T02:30:52.100083+0000","last_became_active":"2023-10-06T02:30:50.681989+0000","last_became_peered":"2023-10-06T02:30:50.681989+0000","last_unstale":"2023-10-06T02:30:52.100083+0000","last_undegraded":"2023-10-06T02:30:52.100083+0000","last_fullsized":"2023-10-06T02:30:52.100083+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_clean_scrub_stamp":"2023-10-06T02:30:49.637381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:17:34.046237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.681994+0000","last_change":"2023-10-06T02:30:51.674180+0000","last_active":"2023-10-06T02:30:51.681994+0000","last_peered":"2023-10-06T02:30:51.681994+0000","last_clean":"2023-10-06T02:30:51.681994+0000","last_became_active":"2023-10-06T02:30:51.673893+0000","last_became_peered":"2023-10-06T02:30:51.673893+0000","last_unstale":"2023-10-06T02:30:51.681994+0000","last_undegraded":"2023-10-06T02:30:51.681994+0000","last_fullsized":"2023-10-06T02:30:51.681994+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:20:18.788778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1a","version":"143'1","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:52.099856+0000","last_change":"2023-10-06T02:30:50.674496+0000","last_active":"2023-10-06T02:30:52.099856+0000","last_peered":"2023-10-06T02:30:52.099856+0000","last_clean":"2023-10-06T02:30:52.099856+0000","last_became_active":"2023-10-06T02:30:50.674199+0000","last_became_peered":"2023-10-06T02:30:50.674199+0000","last_unstale":"2023-10-06T02:30:52.099856+0000","last_undegraded":"2023-10-06T02:30:52.099856+0000","last_fullsized":"2023-10-06T02:30:52.099856+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_clean_scrub_stamp":"2023-10-06T02:30:49.637381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:35:54.781726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.681909+0000","last_change":"2023-10-06T02:30:51.671717+0000","last_active":"2023-10-06T02:30:51.681909+0000","last_peered":"2023-10-06T02:30:51.681909+0000","last_clean":"2023-10-06T02:30:51.681909+0000","last_became_active":"2023-10-06T02:30:51.671434+0000","last_became_peered":"2023-10-06T02:30:51.671434+0000","last_unstale":"2023-10-06T02:30:51.681909+0000","last_undegraded":"2023-10-06T02:30:51.681909+0000","last_fullsized":"2023-10-06T02:30:51.681909+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:40:16.172585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.662466+0000","last_change":"2023-10-06T02:30:50.675638+0000","last_active":"2023-10-06T02:30:51.662466+0000","last_peered":"2023-10-06T02:30:51.662466+0000","last_clean":"2023-10-06T02:30:51.662466+0000","last_became_active":"2023-10-06T02:30:50.675374+0000","last_became_peered":"2023-10-06T02:30:50.675374+0000","last_unstale":"2023-10-06T02:30:51.662466+0000","last_undegraded":"2023-10-06T02:30:51.662466+0000","last_fullsized":"2023-10-06T02:30:51.662466+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_clean_scrub_stamp":"2023-10-06T02:30:49.637381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:27:05.582947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.678936+0000","last_change":"2023-10-06T02:30:51.678936+0000","last_active":"2023-10-06T02:30:51.678936+0000","last_peered":"2023-10-06T02:30:51.678936+0000","last_clean":"2023-10-06T02:30:51.678936+0000","last_became_active":"2023-10-06T02:30:51.678595+0000","last_became_peered":"2023-10-06T02:30:51.678595+0000","last_unstale":"2023-10-06T02:30:51.678936+0000","last_undegraded":"2023-10-06T02:30:51.678936+0000","last_fullsized":"2023-10-06T02:30:51.678936+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:27:54.865078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.656356+0000","last_change":"2023-10-06T02:30:50.680339+0000","last_active":"2023-10-06T02:30:51.656356+0000","last_peered":"2023-10-06T02:30:51.656356+0000","last_clean":"2023-10-06T02:30:51.656356+0000","last_became_active":"2023-10-06T02:30:50.680067+0000","last_became_peered":"2023-10-06T02:30:50.680067+0000","last_unstale":"2023-10-06T02:30:51.656356+0000","last_undegraded":"2023-10-06T02:30:51.656356+0000","last_fullsized":"2023-10-06T02:30:51.656356+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_clean_scrub_stamp":"2023-10-06T02:30:49.637381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:05:09.020108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.672520+0000","last_change":"2023-10-06T02:30:51.665202+0000","last_active":"2023-10-06T02:30:51.672520+0000","last_peered":"2023-10-06T02:30:51.672520+0000","last_clean":"2023-10-06T02:30:51.672520+0000","last_became_active":"2023-10-06T02:30:51.664129+0000","last_became_peered":"2023-10-06T02:30:51.664129+0000","last_unstale":"2023-10-06T02:30:51.672520+0000","last_undegraded":"2023-10-06T02:30:51.672520+0000","last_fullsized":"2023-10-06T02:30:51.672520+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:27:26.681762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"143'1","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.664309+0000","last_change":"2023-10-06T02:30:50.674863+0000","last_active":"2023-10-06T02:30:51.664309+0000","last_peered":"2023-10-06T02:30:51.664309+0000","last_clean":"2023-10-06T02:30:51.664309+0000","last_became_active":"2023-10-06T02:30:50.674542+0000","last_became_peered":"2023-10-06T02:30:50.674542+0000","last_unstale":"2023-10-06T02:30:51.664309+0000","last_undegraded":"2023-10-06T02:30:51.664309+0000","last_fullsized":"2023-10-06T02:30:51.664309+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_clean_scrub_stamp":"2023-10-06T02:30:49.637381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:06:58.421543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.684384+0000","last_change":"2023-10-06T02:30:51.681564+0000","last_active":"2023-10-06T02:30:51.684384+0000","last_peered":"2023-10-06T02:30:51.684384+0000","last_clean":"2023-10-06T02:30:51.684384+0000","last_became_active":"2023-10-06T02:30:51.680841+0000","last_became_peered":"2023-10-06T02:30:51.680841+0000","last_unstale":"2023-10-06T02:30:51.684384+0000","last_undegraded":"2023-10-06T02:30:51.684384+0000","last_fullsized":"2023-10-06T02:30:51.684384+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:30:16.718924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.e","version":"143'2","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:52.100357+0000","last_change":"2023-10-06T02:30:50.684634+0000","last_active":"2023-10-06T02:30:52.100357+0000","last_peered":"2023-10-06T02:30:52.100357+0000","last_clean":"2023-10-06T02:30:52.100357+0000","last_became_active":"2023-10-06T02:30:50.684238+0000","last_became_peered":"2023-10-06T02:30:50.684238+0000","last_unstale":"2023-10-06T02:30:52.100357+0000","last_undegraded":"2023-10-06T02:30:52.100357+0000","last_fullsized":"2023-10-06T02:30:52.100357+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_clean_scrub_stamp":"2023-10-06T02:30:49.637381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:54:59.387855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.677303+0000","last_change":"2023-10-06T02:30:51.672736+0000","last_active":"2023-10-06T02:30:51.677303+0000","last_peered":"2023-10-06T02:30:51.677303+0000","last_clean":"2023-10-06T02:30:51.677303+0000","last_became_active":"2023-10-06T02:30:51.672477+0000","last_became_peered":"2023-10-06T02:30:51.672477+0000","last_unstale":"2023-10-06T02:30:51.677303+0000","last_undegraded":"2023-10-06T02:30:51.677303+0000","last_fullsized":"2023-10-06T02:30:51.677303+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:54:10.422135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.d","version":"143'1","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:52.098816+0000","last_change":"2023-10-06T02:30:50.672218+0000","last_active":"2023-10-06T02:30:52.098816+0000","last_peered":"2023-10-06T02:30:52.098816+0000","last_clean":"2023-10-06T02:30:52.098816+0000","last_became_active":"2023-10-06T02:30:50.671901+0000","last_became_peered":"2023-10-06T02:30:50.671901+0000","last_unstale":"2023-10-06T02:30:52.098816+0000","last_undegraded":"2023-10-06T02:30:52.098816+0000","last_fullsized":"2023-10-06T02:30:52.098816+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_clean_scrub_stamp":"2023-10-06T02:30:49.637381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:06:58.421543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.674838+0000","last_change":"2023-10-06T02:30:51.671644+0000","last_active":"2023-10-06T02:30:51.674838+0000","last_peered":"2023-10-06T02:30:51.674838+0000","last_clean":"2023-10-06T02:30:51.674838+0000","last_became_active":"2023-10-06T02:30:51.671363+0000","last_became_peered":"2023-10-06T02:30:51.671363+0000","last_unstale":"2023-10-06T02:30:51.674838+0000","last_undegraded":"2023-10-06T02:30:51.674838+0000","last_fullsized":"2023-10-06T02:30:51.674838+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:08:16.365846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.657952+0000","last_change":"2023-10-06T02:30:50.682089+0000","last_active":"2023-10-06T02:30:51.657952+0000","last_peered":"2023-10-06T02:30:51.657952+0000","last_clean":"2023-10-06T02:30:51.657952+0000","last_became_active":"2023-10-06T02:30:50.681804+0000","last_became_peered":"2023-10-06T02:30:50.681804+0000","last_unstale":"2023-10-06T02:30:51.657952+0000","last_undegraded":"2023-10-06T02:30:51.657952+0000","last_fullsized":"2023-10-06T02:30:51.657952+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_clean_scrub_stamp":"2023-10-06T02:30:49.637381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:11:52.013827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.672003+0000","last_change":"2023-10-06T02:30:51.672003+0000","last_active":"2023-10-06T02:30:51.672003+0000","last_peered":"2023-10-06T02:30:51.672003+0000","last_clean":"2023-10-06T02:30:51.672003+0000","last_became_active":"2023-10-06T02:30:51.671755+0000","last_became_peered":"2023-10-06T02:30:51.671755+0000","last_unstale":"2023-10-06T02:30:51.672003+0000","last_undegraded":"2023-10-06T02:30:51.672003+0000","last_fullsized":"2023-10-06T02:30:51.672003+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:01:17.186989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.b","version":"143'2","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:52.097268+0000","last_change":"2023-10-06T02:30:50.671943+0000","last_active":"2023-10-06T02:30:52.097268+0000","last_peered":"2023-10-06T02:30:52.097268+0000","last_clean":"2023-10-06T02:30:52.097268+0000","last_became_active":"2023-10-06T02:30:50.671424+0000","last_became_peered":"2023-10-06T02:30:50.671424+0000","last_unstale":"2023-10-06T02:30:52.097268+0000","last_undegraded":"2023-10-06T02:30:52.097268+0000","last_fullsized":"2023-10-06T02:30:52.097268+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_clean_scrub_stamp":"2023-10-06T02:30:49.637381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:24:41.735075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.662205+0000","last_change":"2023-10-06T02:30:51.660403+0000","last_active":"2023-10-06T02:30:51.662205+0000","last_peered":"2023-10-06T02:30:51.662205+0000","last_clean":"2023-10-06T02:30:51.662205+0000","last_became_active":"2023-10-06T02:30:51.660081+0000","last_became_peered":"2023-10-06T02:30:51.660081+0000","last_unstale":"2023-10-06T02:30:51.662205+0000","last_undegraded":"2023-10-06T02:30:51.662205+0000","last_fullsized":"2023-10-06T02:30:51.662205+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:45:54.859704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.655128+0000","last_change":"2023-10-06T02:30:50.665481+0000","last_active":"2023-10-06T02:30:51.655128+0000","last_peered":"2023-10-06T02:30:51.655128+0000","last_clean":"2023-10-06T02:30:51.655128+0000","last_became_active":"2023-10-06T02:30:50.665189+0000","last_became_peered":"2023-10-06T02:30:50.665189+0000","last_unstale":"2023-10-06T02:30:51.655128+0000","last_undegraded":"2023-10-06T02:30:51.655128+0000","last_fullsized":"2023-10-06T02:30:51.655128+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_clean_scrub_stamp":"2023-10-06T02:30:49.637381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:40:16.665661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.670757+0000","last_change":"2023-10-06T02:30:51.664848+0000","last_active":"2023-10-06T02:30:51.670757+0000","last_peered":"2023-10-06T02:30:51.670757+0000","last_clean":"2023-10-06T02:30:51.670757+0000","last_became_active":"2023-10-06T02:30:51.664589+0000","last_became_peered":"2023-10-06T02:30:51.664589+0000","last_unstale":"2023-10-06T02:30:51.670757+0000","last_undegraded":"2023-10-06T02:30:51.670757+0000","last_fullsized":"2023-10-06T02:30:51.670757+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:20:18.598786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.662575+0000","last_change":"2023-10-06T02:30:50.667806+0000","last_active":"2023-10-06T02:30:51.662575+0000","last_peered":"2023-10-06T02:30:51.662575+0000","last_clean":"2023-10-06T02:30:51.662575+0000","last_became_active":"2023-10-06T02:30:50.667526+0000","last_became_peered":"2023-10-06T02:30:50.667526+0000","last_unstale":"2023-10-06T02:30:51.662575+0000","last_undegraded":"2023-10-06T02:30:51.662575+0000","last_fullsized":"2023-10-06T02:30:51.662575+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_clean_scrub_stamp":"2023-10-06T02:30:49.637381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:49:34.881059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.683354+0000","last_change":"2023-10-06T02:30:51.679150+0000","last_active":"2023-10-06T02:30:51.683354+0000","last_peered":"2023-10-06T02:30:51.683354+0000","last_clean":"2023-10-06T02:30:51.683354+0000","last_became_active":"2023-10-06T02:30:51.678870+0000","last_became_peered":"2023-10-06T02:30:51.678870+0000","last_unstale":"2023-10-06T02:30:51.683354+0000","last_undegraded":"2023-10-06T02:30:51.683354+0000","last_fullsized":"2023-10-06T02:30:51.683354+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:29:12.818783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.663318+0000","last_change":"2023-10-06T02:30:50.671628+0000","last_active":"2023-10-06T02:30:51.663318+0000","last_peered":"2023-10-06T02:30:51.663318+0000","last_clean":"2023-10-06T02:30:51.663318+0000","last_became_active":"2023-10-06T02:30:50.671300+0000","last_became_peered":"2023-10-06T02:30:50.671300+0000","last_unstale":"2023-10-06T02:30:51.663318+0000","last_undegraded":"2023-10-06T02:30:51.663318+0000","last_fullsized":"2023-10-06T02:30:51.663318+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_clean_scrub_stamp":"2023-10-06T02:30:49.637381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:39:21.468976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.674803+0000","last_change":"2023-10-06T02:30:51.671977+0000","last_active":"2023-10-06T02:30:51.674803+0000","last_peered":"2023-10-06T02:30:51.674803+0000","last_clean":"2023-10-06T02:30:51.674803+0000","last_became_active":"2023-10-06T02:30:51.671663+0000","last_became_peered":"2023-10-06T02:30:51.671663+0000","last_unstale":"2023-10-06T02:30:51.674803+0000","last_undegraded":"2023-10-06T02:30:51.674803+0000","last_fullsized":"2023-10-06T02:30:51.674803+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:04:59.566737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.7","version":"143'2","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:52.097289+0000","last_change":"2023-10-06T02:30:50.671032+0000","last_active":"2023-10-06T02:30:52.097289+0000","last_peered":"2023-10-06T02:30:52.097289+0000","last_clean":"2023-10-06T02:30:52.097289+0000","last_became_active":"2023-10-06T02:30:50.670761+0000","last_became_peered":"2023-10-06T02:30:50.670761+0000","last_unstale":"2023-10-06T02:30:52.097289+0000","last_undegraded":"2023-10-06T02:30:52.097289+0000","last_fullsized":"2023-10-06T02:30:52.097289+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_clean_scrub_stamp":"2023-10-06T02:30:49.637381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:26:19.617860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.672567+0000","last_change":"2023-10-06T02:30:51.665661+0000","last_active":"2023-10-06T02:30:51.672567+0000","last_peered":"2023-10-06T02:30:51.672567+0000","last_clean":"2023-10-06T02:30:51.672567+0000","last_became_active":"2023-10-06T02:30:51.664972+0000","last_became_peered":"2023-10-06T02:30:51.664972+0000","last_unstale":"2023-10-06T02:30:51.672567+0000","last_undegraded":"2023-10-06T02:30:51.672567+0000","last_fullsized":"2023-10-06T02:30:51.672567+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:08:16.365846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T02:30:50.683680+0000","last_change":"2023-10-06T02:30:50.678983+0000","last_active":"2023-10-06T02:30:50.683680+0000","last_peered":"2023-10-06T02:30:50.683680+0000","last_clean":"2023-10-06T02:30:50.683680+0000","last_became_active":"2023-10-06T02:30:50.678549+0000","last_became_peered":"2023-10-06T02:30:50.678549+0000","last_unstale":"2023-10-06T02:30:50.683680+0000","last_undegraded":"2023-10-06T02:30:50.683680+0000","last_fullsized":"2023-10-06T02:30:50.683680+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_clean_scrub_stamp":"2023-10-06T02:30:49.637381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:34:35.008716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.681955+0000","last_change":"2023-10-06T02:30:51.673424+0000","last_active":"2023-10-06T02:30:51.681955+0000","last_peered":"2023-10-06T02:30:51.681955+0000","last_clean":"2023-10-06T02:30:51.681955+0000","last_became_active":"2023-10-06T02:30:51.673159+0000","last_became_peered":"2023-10-06T02:30:51.673159+0000","last_unstale":"2023-10-06T02:30:51.681955+0000","last_undegraded":"2023-10-06T02:30:51.681955+0000","last_fullsized":"2023-10-06T02:30:51.681955+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:43:15.672912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T02:30:50.673429+0000","last_change":"2023-10-06T02:30:50.672021+0000","last_active":"2023-10-06T02:30:50.673429+0000","last_peered":"2023-10-06T02:30:50.673429+0000","last_clean":"2023-10-06T02:30:50.673429+0000","last_became_active":"2023-10-06T02:30:50.671736+0000","last_became_peered":"2023-10-06T02:30:50.671736+0000","last_unstale":"2023-10-06T02:30:50.673429+0000","last_undegraded":"2023-10-06T02:30:50.673429+0000","last_fullsized":"2023-10-06T02:30:50.673429+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_clean_scrub_stamp":"2023-10-06T02:30:49.637381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:21:58.534242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.672063+0000","last_change":"2023-10-06T02:30:51.668121+0000","last_active":"2023-10-06T02:30:51.672063+0000","last_peered":"2023-10-06T02:30:51.672063+0000","last_clean":"2023-10-06T02:30:51.672063+0000","last_became_active":"2023-10-06T02:30:51.664156+0000","last_became_peered":"2023-10-06T02:30:51.664156+0000","last_unstale":"2023-10-06T02:30:51.672063+0000","last_undegraded":"2023-10-06T02:30:51.672063+0000","last_fullsized":"2023-10-06T02:30:51.672063+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:05:57.018432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.662135+0000","last_change":"2023-10-06T02:30:50.683019+0000","last_active":"2023-10-06T02:30:51.662135+0000","last_peered":"2023-10-06T02:30:51.662135+0000","last_clean":"2023-10-06T02:30:51.662135+0000","last_became_active":"2023-10-06T02:30:50.682766+0000","last_became_peered":"2023-10-06T02:30:50.682766+0000","last_unstale":"2023-10-06T02:30:51.662135+0000","last_undegraded":"2023-10-06T02:30:51.662135+0000","last_fullsized":"2023-10-06T02:30:51.662135+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_clean_scrub_stamp":"2023-10-06T02:30:49.637381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:12:26.468515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.674078+0000","last_change":"2023-10-06T02:30:51.674078+0000","last_active":"2023-10-06T02:30:51.674078+0000","last_peered":"2023-10-06T02:30:51.674078+0000","last_clean":"2023-10-06T02:30:51.674078+0000","last_became_active":"2023-10-06T02:30:51.672332+0000","last_became_peered":"2023-10-06T02:30:51.672332+0000","last_unstale":"2023-10-06T02:30:51.674078+0000","last_undegraded":"2023-10-06T02:30:51.674078+0000","last_fullsized":"2023-10-06T02:30:51.674078+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:16:50.121627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T02:30:50.678742+0000","last_change":"2023-10-06T02:30:50.674949+0000","last_active":"2023-10-06T02:30:50.678742+0000","last_peered":"2023-10-06T02:30:50.678742+0000","last_clean":"2023-10-06T02:30:50.678742+0000","last_became_active":"2023-10-06T02:30:50.674664+0000","last_became_peered":"2023-10-06T02:30:50.674664+0000","last_unstale":"2023-10-06T02:30:50.678742+0000","last_undegraded":"2023-10-06T02:30:50.678742+0000","last_fullsized":"2023-10-06T02:30:50.678742+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_clean_scrub_stamp":"2023-10-06T02:30:49.637381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:54:47.060617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.680365+0000","last_change":"2023-10-06T02:30:51.675370+0000","last_active":"2023-10-06T02:30:51.680365+0000","last_peered":"2023-10-06T02:30:51.680365+0000","last_clean":"2023-10-06T02:30:51.680365+0000","last_became_active":"2023-10-06T02:30:51.675066+0000","last_became_peered":"2023-10-06T02:30:51.675066+0000","last_unstale":"2023-10-06T02:30:51.680365+0000","last_undegraded":"2023-10-06T02:30:51.680365+0000","last_fullsized":"2023-10-06T02:30:51.680365+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:39:08.450582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T02:30:50.664499+0000","last_change":"2023-10-06T02:30:50.662598+0000","last_active":"2023-10-06T02:30:50.664499+0000","last_peered":"2023-10-06T02:30:50.664499+0000","last_clean":"2023-10-06T02:30:50.664499+0000","last_became_active":"2023-10-06T02:30:50.662284+0000","last_became_peered":"2023-10-06T02:30:50.662284+0000","last_unstale":"2023-10-06T02:30:50.664499+0000","last_undegraded":"2023-10-06T02:30:50.664499+0000","last_fullsized":"2023-10-06T02:30:50.664499+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_clean_scrub_stamp":"2023-10-06T02:30:49.637381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:35:54.781726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.675498+0000","last_change":"2023-10-06T02:30:51.675498+0000","last_active":"2023-10-06T02:30:51.675498+0000","last_peered":"2023-10-06T02:30:51.675498+0000","last_clean":"2023-10-06T02:30:51.675498+0000","last_became_active":"2023-10-06T02:30:51.675217+0000","last_became_peered":"2023-10-06T02:30:51.675217+0000","last_unstale":"2023-10-06T02:30:51.675498+0000","last_undegraded":"2023-10-06T02:30:51.675498+0000","last_fullsized":"2023-10-06T02:30:51.675498+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:50:51.184941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.655775+0000","last_change":"2023-10-06T02:30:50.672835+0000","last_active":"2023-10-06T02:30:51.655775+0000","last_peered":"2023-10-06T02:30:51.655775+0000","last_clean":"2023-10-06T02:30:51.655775+0000","last_became_active":"2023-10-06T02:30:50.672540+0000","last_became_peered":"2023-10-06T02:30:50.672540+0000","last_unstale":"2023-10-06T02:30:51.655775+0000","last_undegraded":"2023-10-06T02:30:51.655775+0000","last_fullsized":"2023-10-06T02:30:51.655775+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_clean_scrub_stamp":"2023-10-06T02:30:49.637381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:35:14.998650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.681931+0000","last_change":"2023-10-06T02:30:51.672400+0000","last_active":"2023-10-06T02:30:51.681931+0000","last_peered":"2023-10-06T02:30:51.681931+0000","last_clean":"2023-10-06T02:30:51.681931+0000","last_became_active":"2023-10-06T02:30:51.672140+0000","last_became_peered":"2023-10-06T02:30:51.672140+0000","last_unstale":"2023-10-06T02:30:51.681931+0000","last_undegraded":"2023-10-06T02:30:51.681931+0000","last_fullsized":"2023-10-06T02:30:51.681931+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:52:30.458923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.655129+0000","last_change":"2023-10-06T02:30:50.672044+0000","last_active":"2023-10-06T02:30:51.655129+0000","last_peered":"2023-10-06T02:30:51.655129+0000","last_clean":"2023-10-06T02:30:51.655129+0000","last_became_active":"2023-10-06T02:30:50.671760+0000","last_became_peered":"2023-10-06T02:30:50.671760+0000","last_unstale":"2023-10-06T02:30:51.655129+0000","last_undegraded":"2023-10-06T02:30:51.655129+0000","last_fullsized":"2023-10-06T02:30:51.655129+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_clean_scrub_stamp":"2023-10-06T02:30:49.637381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:34:35.008716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.668851+0000","last_change":"2023-10-06T02:30:51.663438+0000","last_active":"2023-10-06T02:30:51.668851+0000","last_peered":"2023-10-06T02:30:51.668851+0000","last_clean":"2023-10-06T02:30:51.668851+0000","last_became_active":"2023-10-06T02:30:51.663194+0000","last_became_peered":"2023-10-06T02:30:51.663194+0000","last_unstale":"2023-10-06T02:30:51.668851+0000","last_undegraded":"2023-10-06T02:30:51.668851+0000","last_fullsized":"2023-10-06T02:30:51.668851+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:16:02.882307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T02:30:50.677591+0000","last_change":"2023-10-06T02:30:50.673162+0000","last_active":"2023-10-06T02:30:50.677591+0000","last_peered":"2023-10-06T02:30:50.677591+0000","last_clean":"2023-10-06T02:30:50.677591+0000","last_became_active":"2023-10-06T02:30:50.672891+0000","last_became_peered":"2023-10-06T02:30:50.672891+0000","last_unstale":"2023-10-06T02:30:50.677591+0000","last_undegraded":"2023-10-06T02:30:50.677591+0000","last_fullsized":"2023-10-06T02:30:50.677591+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_clean_scrub_stamp":"2023-10-06T02:30:49.637381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:24:41.735075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":115,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.662765+0000","last_change":"2023-10-06T02:23:01.992935+0000","last_active":"2023-10-06T02:30:51.662765+0000","last_peered":"2023-10-06T02:30:51.662765+0000","last_clean":"2023-10-06T02:30:51.662765+0000","last_became_active":"2023-10-06T02:23:01.992568+0000","last_became_peered":"2023-10-06T02:23:01.992568+0000","last_unstale":"2023-10-06T02:30:51.662765+0000","last_undegraded":"2023-10-06T02:30:51.662765+0000","last_fullsized":"2023-10-06T02:30:51.662765+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:06.212251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.686892+0000","last_change":"2023-10-06T02:30:51.684069+0000","last_active":"2023-10-06T02:30:51.686892+0000","last_peered":"2023-10-06T02:30:51.686892+0000","last_clean":"2023-10-06T02:30:51.686892+0000","last_became_active":"2023-10-06T02:30:51.682151+0000","last_became_peered":"2023-10-06T02:30:51.682151+0000","last_unstale":"2023-10-06T02:30:51.686892+0000","last_undegraded":"2023-10-06T02:30:51.686892+0000","last_fullsized":"2023-10-06T02:30:51.686892+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:35:49.324609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.13","version":"143'2","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:52.099045+0000","last_change":"2023-10-06T02:30:50.672607+0000","last_active":"2023-10-06T02:30:52.099045+0000","last_peered":"2023-10-06T02:30:52.099045+0000","last_clean":"2023-10-06T02:30:52.099045+0000","last_became_active":"2023-10-06T02:30:50.672343+0000","last_became_peered":"2023-10-06T02:30:50.672343+0000","last_unstale":"2023-10-06T02:30:52.099045+0000","last_undegraded":"2023-10-06T02:30:52.099045+0000","last_fullsized":"2023-10-06T02:30:52.099045+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_clean_scrub_stamp":"2023-10-06T02:30:49.637381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:39:07.441201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":116,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.657910+0000","last_change":"2023-10-06T02:23:01.989934+0000","last_active":"2023-10-06T02:30:51.657910+0000","last_peered":"2023-10-06T02:30:51.657910+0000","last_clean":"2023-10-06T02:30:51.657910+0000","last_became_active":"2023-10-06T02:23:01.989706+0000","last_became_peered":"2023-10-06T02:23:01.989706+0000","last_unstale":"2023-10-06T02:30:51.657910+0000","last_undegraded":"2023-10-06T02:30:51.657910+0000","last_fullsized":"2023-10-06T02:30:51.657910+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:06.212251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.681850+0000","last_change":"2023-10-06T02:30:51.670323+0000","last_active":"2023-10-06T02:30:51.681850+0000","last_peered":"2023-10-06T02:30:51.681850+0000","last_clean":"2023-10-06T02:30:51.681850+0000","last_became_active":"2023-10-06T02:30:51.670054+0000","last_became_peered":"2023-10-06T02:30:51.670054+0000","last_unstale":"2023-10-06T02:30:51.681850+0000","last_undegraded":"2023-10-06T02:30:51.681850+0000","last_fullsized":"2023-10-06T02:30:51.681850+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:49:38.772657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.10","version":"143'8","reported_seq":18,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:52.099165+0000","last_change":"2023-10-06T02:30:50.674130+0000","last_active":"2023-10-06T02:30:52.099165+0000","last_peered":"2023-10-06T02:30:52.099165+0000","last_clean":"2023-10-06T02:30:52.099165+0000","last_became_active":"2023-10-06T02:30:50.673875+0000","last_became_peered":"2023-10-06T02:30:50.673875+0000","last_unstale":"2023-10-06T02:30:52.099165+0000","last_undegraded":"2023-10-06T02:30:52.099165+0000","last_fullsized":"2023-10-06T02:30:52.099165+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_clean_scrub_stamp":"2023-10-06T02:30:49.637381+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:14:46.899357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":110,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.664119+0000","last_change":"2023-10-06T02:23:02.054069+0000","last_active":"2023-10-06T02:30:51.664119+0000","last_peered":"2023-10-06T02:30:51.664119+0000","last_clean":"2023-10-06T02:30:51.664119+0000","last_became_active":"2023-10-06T02:23:02.053739+0000","last_became_peered":"2023-10-06T02:23:02.053739+0000","last_unstale":"2023-10-06T02:30:51.664119+0000","last_undegraded":"2023-10-06T02:30:51.664119+0000","last_fullsized":"2023-10-06T02:30:51.664119+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:57:48.822608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.678381+0000","last_change":"2023-10-06T02:30:51.678381+0000","last_active":"2023-10-06T02:30:51.678381+0000","last_peered":"2023-10-06T02:30:51.678381+0000","last_clean":"2023-10-06T02:30:51.678381+0000","last_became_active":"2023-10-06T02:30:51.678011+0000","last_became_peered":"2023-10-06T02:30:51.678011+0000","last_unstale":"2023-10-06T02:30:51.678381+0000","last_undegraded":"2023-10-06T02:30:51.678381+0000","last_fullsized":"2023-10-06T02:30:51.678381+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:13:55.763105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.15","version":"143'3","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:52.093508+0000","last_change":"2023-10-06T02:30:50.676649+0000","last_active":"2023-10-06T02:30:52.093508+0000","last_peered":"2023-10-06T02:30:52.093508+0000","last_clean":"2023-10-06T02:30:52.093508+0000","last_became_active":"2023-10-06T02:30:50.676245+0000","last_became_peered":"2023-10-06T02:30:50.676245+0000","last_unstale":"2023-10-06T02:30:52.093508+0000","last_undegraded":"2023-10-06T02:30:52.093508+0000","last_fullsized":"2023-10-06T02:30:52.093508+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_clean_scrub_stamp":"2023-10-06T02:30:49.637381+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:17:34.046237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":110,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.663412+0000","last_change":"2023-10-06T02:23:01.990773+0000","last_active":"2023-10-06T02:30:51.663412+0000","last_peered":"2023-10-06T02:30:51.663412+0000","last_clean":"2023-10-06T02:30:51.663412+0000","last_became_active":"2023-10-06T02:23:01.990511+0000","last_became_peered":"2023-10-06T02:23:01.990511+0000","last_unstale":"2023-10-06T02:30:51.663412+0000","last_undegraded":"2023-10-06T02:30:51.663412+0000","last_fullsized":"2023-10-06T02:30:51.663412+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:42:54.797731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.677117+0000","last_change":"2023-10-06T02:30:51.672599+0000","last_active":"2023-10-06T02:30:51.677117+0000","last_peered":"2023-10-06T02:30:51.677117+0000","last_clean":"2023-10-06T02:30:51.677117+0000","last_became_active":"2023-10-06T02:30:51.672259+0000","last_became_peered":"2023-10-06T02:30:51.672259+0000","last_unstale":"2023-10-06T02:30:51.677117+0000","last_undegraded":"2023-10-06T02:30:51.677117+0000","last_fullsized":"2023-10-06T02:30:51.677117+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:35:55.791108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T02:30:50.683659+0000","last_change":"2023-10-06T02:30:50.678752+0000","last_active":"2023-10-06T02:30:50.683659+0000","last_peered":"2023-10-06T02:30:50.683659+0000","last_clean":"2023-10-06T02:30:50.683659+0000","last_became_active":"2023-10-06T02:30:50.678431+0000","last_became_peered":"2023-10-06T02:30:50.678431+0000","last_unstale":"2023-10-06T02:30:50.683659+0000","last_undegraded":"2023-10-06T02:30:50.683659+0000","last_fullsized":"2023-10-06T02:30:50.683659+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_clean_scrub_stamp":"2023-10-06T02:30:49.637381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:39:07.441201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"76'128","reported_seq":302,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.663445+0000","last_change":"2023-10-06T02:22:55.032519+0000","last_active":"2023-10-06T02:30:51.663445+0000","last_peered":"2023-10-06T02:30:51.663445+0000","last_clean":"2023-10-06T02:30:51.663445+0000","last_became_active":"2023-10-06T02:22:55.031367+0000","last_became_peered":"2023-10-06T02:22:55.031367+0000","last_unstale":"2023-10-06T02:30:51.663445+0000","last_undegraded":"2023-10-06T02:30:51.663445+0000","last_fullsized":"2023-10-06T02:30:51.663445+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:22:53.309097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:22:53.309097+0000","last_clean_scrub_stamp":"2023-10-06T02:22:53.309097+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:53:46.520909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":134,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.655209+0000","last_change":"2023-10-06T02:23:01.988140+0000","last_active":"2023-10-06T02:30:51.655209+0000","last_peered":"2023-10-06T02:30:51.655209+0000","last_clean":"2023-10-06T02:30:51.655209+0000","last_became_active":"2023-10-06T02:23:01.987769+0000","last_became_peered":"2023-10-06T02:23:01.987769+0000","last_unstale":"2023-10-06T02:30:51.655209+0000","last_undegraded":"2023-10-06T02:30:51.655209+0000","last_fullsized":"2023-10-06T02:30:51.655209+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:20:08.391912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.682189+0000","last_change":"2023-10-06T02:30:51.677753+0000","last_active":"2023-10-06T02:30:51.682189+0000","last_peered":"2023-10-06T02:30:51.682189+0000","last_clean":"2023-10-06T02:30:51.682189+0000","last_became_active":"2023-10-06T02:30:51.677358+0000","last_became_peered":"2023-10-06T02:30:51.677358+0000","last_unstale":"2023-10-06T02:30:51.682189+0000","last_undegraded":"2023-10-06T02:30:51.682189+0000","last_fullsized":"2023-10-06T02:30:51.682189+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:35:49.324609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"143'6","reported_seq":15,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:52.100309+0000","last_change":"2023-10-06T02:30:50.679318+0000","last_active":"2023-10-06T02:30:52.100309+0000","last_peered":"2023-10-06T02:30:52.100309+0000","last_clean":"2023-10-06T02:30:52.100309+0000","last_became_active":"2023-10-06T02:30:50.679061+0000","last_became_peered":"2023-10-06T02:30:50.679061+0000","last_unstale":"2023-10-06T02:30:52.100309+0000","last_undegraded":"2023-10-06T02:30:52.100309+0000","last_fullsized":"2023-10-06T02:30:52.100309+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_clean_scrub_stamp":"2023-10-06T02:30:49.637381+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:40:20.143016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":123,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.662666+0000","last_change":"2023-10-06T02:23:02.054338+0000","last_active":"2023-10-06T02:30:51.662666+0000","last_peered":"2023-10-06T02:30:51.662666+0000","last_clean":"2023-10-06T02:30:51.662666+0000","last_became_active":"2023-10-06T02:23:02.053991+0000","last_became_peered":"2023-10-06T02:23:02.053991+0000","last_unstale":"2023-10-06T02:30:51.662666+0000","last_undegraded":"2023-10-06T02:30:51.662666+0000","last_fullsized":"2023-10-06T02:30:51.662666+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:39:30.192838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.680365+0000","last_change":"2023-10-06T02:30:51.675353+0000","last_active":"2023-10-06T02:30:51.680365+0000","last_peered":"2023-10-06T02:30:51.680365+0000","last_clean":"2023-10-06T02:30:51.680365+0000","last_became_active":"2023-10-06T02:30:51.672735+0000","last_became_peered":"2023-10-06T02:30:51.672735+0000","last_unstale":"2023-10-06T02:30:51.680365+0000","last_undegraded":"2023-10-06T02:30:51.680365+0000","last_fullsized":"2023-10-06T02:30:51.680365+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:40:21.152397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T02:30:50.683712+0000","last_change":"2023-10-06T02:30:50.679780+0000","last_active":"2023-10-06T02:30:50.683712+0000","last_peered":"2023-10-06T02:30:50.683712+0000","last_clean":"2023-10-06T02:30:50.683712+0000","last_became_active":"2023-10-06T02:30:50.679465+0000","last_became_peered":"2023-10-06T02:30:50.679465+0000","last_unstale":"2023-10-06T02:30:50.683712+0000","last_undegraded":"2023-10-06T02:30:50.683712+0000","last_fullsized":"2023-10-06T02:30:50.683712+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_clean_scrub_stamp":"2023-10-06T02:30:49.637381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:26:19.617860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":134,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.655229+0000","last_change":"2023-10-06T02:23:01.988631+0000","last_active":"2023-10-06T02:30:51.655229+0000","last_peered":"2023-10-06T02:30:51.655229+0000","last_clean":"2023-10-06T02:30:51.655229+0000","last_became_active":"2023-10-06T02:23:01.988303+0000","last_became_peered":"2023-10-06T02:23:01.988303+0000","last_unstale":"2023-10-06T02:30:51.655229+0000","last_undegraded":"2023-10-06T02:30:51.655229+0000","last_fullsized":"2023-10-06T02:30:51.655229+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:38.211466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.677333+0000","last_change":"2023-10-06T02:30:51.673852+0000","last_active":"2023-10-06T02:30:51.677333+0000","last_peered":"2023-10-06T02:30:51.677333+0000","last_clean":"2023-10-06T02:30:51.677333+0000","last_became_active":"2023-10-06T02:30:51.673521+0000","last_became_peered":"2023-10-06T02:30:51.673521+0000","last_unstale":"2023-10-06T02:30:51.677333+0000","last_undegraded":"2023-10-06T02:30:51.677333+0000","last_fullsized":"2023-10-06T02:30:51.677333+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:50.646763+0000","last_clean_scrub_stamp":"2023-10-06T02:30:50.646763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:16:02.882307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.655645+0000","last_change":"2023-10-06T02:30:50.671581+0000","last_active":"2023-10-06T02:30:51.655645+0000","last_peered":"2023-10-06T02:30:51.655645+0000","last_clean":"2023-10-06T02:30:51.655645+0000","last_became_active":"2023-10-06T02:30:50.671278+0000","last_became_peered":"2023-10-06T02:30:50.671278+0000","last_unstale":"2023-10-06T02:30:51.655645+0000","last_undegraded":"2023-10-06T02:30:51.655645+0000","last_fullsized":"2023-10-06T02:30:51.655645+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:30:49.637381+0000","last_clean_scrub_stamp":"2023-10-06T02:30:49.637381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:45:15.343527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":116,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T02:30:51.658042+0000","last_change":"2023-10-06T02:23:02.054039+0000","last_active":"2023-10-06T02:30:51.658042+0000","last_peered":"2023-10-06T02:30:51.658042+0000","last_clean":"2023-10-06T02:30:51.658042+0000","last_became_active":"2023-10-06T02:23:02.053742+0000","last_became_peered":"2023-10-06T02:23:02.053742+0000","last_unstale":"2023-10-06T02:30:51.658042+0000","last_undegraded":"2023-10-06T02:30:51.658042+0000","last_fullsized":"2023-10-06T02:30:51.658042+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:58:18.846475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":24,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2},{"poolid":23,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574949,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303720,"kb_used_data":2336,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068120,"statfs":{"total":96636764160,"available":96325754880,"internally_reserved":0,"allocated":2392064,"data_stored":1428080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:30:12 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.64100000000000001,"15min":0.63600000000000001},"min":{"1min":0.375,"5min":0.318,"15min":0.318},"max":{"1min":0.78600000000000003,"5min":1.169,"15min":1.169},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.63900000000000001,"15min":0.61199999999999999},"min":{"1min":0.45300000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.84699999999999998,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.91800000000000004}]},{"osd":1,"last update":"Fri Oct 6 02:30:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.73199999999999998,"15min":0.69399999999999995},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.40300000000000002},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.147},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.67400000000000004,"15min":0.67200000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.82399999999999995,"5min":1.129,"15min":1.129},"last":0.71599999999999997}]},{"osd":2,"last update":"Fri Oct 6 02:30:12 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.67500000000000004,"15min":0.68700000000000006},"min":{"1min":0.44,"5min":0.375,"15min":0.375},"max":{"1min":0.87,"5min":1.238,"15min":1.238},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.69099999999999995,"15min":0.68999999999999995},"min":{"1min":0.54000000000000004,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.75700000000000001}]},{"osd":3,"last update":"Fri Oct 6 02:30:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.45700000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.79400000000000004,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.76300000000000001,"15min":0.77300000000000002},"min":{"1min":0.55900000000000005,"5min":0.497,"15min":0.497},"max":{"1min":1.383,"5min":1.383,"15min":1.383},"last":0.877}]},{"osd":4,"last update":"Fri Oct 6 02:30:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.59199999999999997,"15min":0.67200000000000004},"min":{"1min":0.42899999999999999,"5min":0.41099999999999998,"15min":0.35099999999999998},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.2849999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.67200000000000004,"15min":0.747},"min":{"1min":0.41199999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.96499999999999997,"5min":1.51,"15min":1.51},"last":0.78300000000000003}]},{"osd":5,"last update":"Fri Oct 6 02:30:12 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.69199999999999995,"15min":0.72699999999999998},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.67600000000000005,"15min":0.71999999999999997},"min":{"1min":0.47199999999999998,"5min":0.442,"15min":0.442},"max":{"1min":0.874,"5min":0.98099999999999998,"15min":1.0289999999999999},"last":0.80700000000000005}]},{"osd":6,"last update":"Fri Oct 6 02:30:12 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.76100000000000001,"15min":0.78200000000000003},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.97599999999999998,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.77800000000000002,"15min":0.80600000000000005},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.872,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.89700000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574949,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303152,"kb_used_data":1768,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068688,"statfs":{"total":96636764160,"available":96326336512,"internally_reserved":0,"allocated":1810432,"data_stored":844381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:30:06 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.68700000000000006,"15min":0.66700000000000004},"min":{"1min":0.501,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.88500000000000001,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.72999999999999998,"15min":0.70299999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.91100000000000003,"5min":1.0229999999999999,"15min":1.119},"last":1.04}]},{"osd":1,"last update":"Fri Oct 6 02:30:06 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.68100000000000005,"15min":0.65100000000000002},"min":{"1min":0.45200000000000001,"5min":0.40999999999999998,"15min":0.40699999999999997},"max":{"1min":0.96499999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.68500000000000005,"15min":0.67100000000000004},"min":{"1min":0.437,"5min":0.41899999999999998,"15min":0.40999999999999998},"max":{"1min":0.84399999999999997,"5min":0.997,"15min":0.997},"last":0.78000000000000003}]},{"osd":2,"last update":"Fri Oct 6 02:30:06 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.72899999999999998,"15min":0.70799999999999996},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.92100000000000004,"5min":1.0449999999999999,"15min":1.137},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.74399999999999999,"15min":0.71799999999999997},"min":{"1min":0.55600000000000005,"5min":0.45300000000000001,"15min":0.40300000000000002},"max":{"1min":0.95299999999999996,"5min":1.077,"15min":1.077},"last":0.88200000000000001}]},{"osd":3,"last update":"Fri Oct 6 02:30:06 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.71199999999999997,"15min":0.67100000000000004},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.94099999999999995,"5min":1.056,"15min":1.056},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.73299999999999998,"15min":0.70899999999999996},"min":{"1min":0.56699999999999995,"5min":0.49299999999999999,"15min":0.45900000000000002},"max":{"1min":0.97799999999999998,"5min":0.97999999999999998,"15min":0.98499999999999999},"last":0.80400000000000005}]},{"osd":4,"last update":"Fri Oct 6 02:30:06 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.65900000000000003,"15min":0.64200000000000002},"min":{"1min":0.42499999999999999,"5min":0.36199999999999999,"15min":0.34300000000000003},"max":{"1min":0.93000000000000005,"5min":0.94799999999999995,"15min":1.095},"last":0.96799999999999997},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.69999999999999996,"15min":0.66900000000000004},"min":{"1min":0.52200000000000002,"5min":0.41699999999999998,"15min":0.40999999999999998},"max":{"1min":0.91900000000000004,"5min":1.002,"15min":1.29},"last":0.998}]},{"osd":5,"last update":"Fri Oct 6 02:30:06 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.67700000000000005,"15min":0.69799999999999995},"min":{"1min":0.438,"5min":0.438,"15min":0.39700000000000002},"max":{"1min":1.014,"5min":1.014,"15min":1.2929999999999999},"last":0.98699999999999999},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.70799999999999996,"15min":0.70599999999999996},"min":{"1min":0.48699999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.01,"5min":1.01,"15min":1.02},"last":1.1180000000000001}]},{"osd":7,"last update":"Fri Oct 6 02:30:06 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.74099999999999999,"15min":0.749},"min":{"1min":0.48399999999999999,"5min":0.41499999999999998,"15min":0.36799999999999999},"max":{"1min":1,"5min":1,"15min":1.458},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.76500000000000001,"15min":0.76300000000000001},"min":{"1min":0.50700000000000001,"5min":0.47299999999999998,"15min":0.45900000000000002},"max":{"1min":1.046,"5min":1.103,"15min":1.2470000000000001},"last":0.93799999999999994}]}]},{"osd":1,"up_from":13,"seq":55834574948,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303164,"kb_used_data":1780,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068676,"statfs":{"total":96636764160,"available":96326324224,"internally_reserved":0,"allocated":1822720,"data_stored":845917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:30:02 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.42999999999999999,"15min":0.45300000000000001},"min":{"1min":0.30599999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.59599999999999997,"5min":0.65500000000000003,"15min":1.2230000000000001},"last":0.314},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.46999999999999997,"15min":0.51400000000000001},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.58899999999999997,"5min":0.71799999999999997,"15min":1.788},"last":0.64500000000000002}]},{"osd":2,"last update":"Fri Oct 6 02:30:02 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.44,"15min":0.50600000000000001},"min":{"1min":0.33800000000000002,"5min":0.307,"15min":0.28299999999999997},"max":{"1min":0.53800000000000003,"5min":0.60899999999999999,"15min":2.0840000000000001},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.504,"15min":0.54700000000000004},"min":{"1min":0.38700000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":1.6699999999999999},"last":0.45300000000000001}]},{"osd":3,"last update":"Fri Oct 6 02:30:02 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.59799999999999998,"15min":0.60999999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.72499999999999998,"5min":0.89900000000000002,"15min":1.266},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.54200000000000004,"15min":0.62},"min":{"1min":0.46899999999999997,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.69999999999999996,"5min":0.79400000000000004,"15min":1.861},"last":0.41999999999999998}]},{"osd":4,"last update":"Fri Oct 6 02:30:02 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.77300000000000002,"15min":0.83799999999999997},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.55900000000000005},"max":{"1min":0.93999999999999995,"5min":1.3580000000000001,"15min":1.9279999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.78400000000000003,"15min":0.84099999999999997},"min":{"1min":0.57399999999999995,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.90200000000000002,"5min":1.4339999999999999,"15min":2.0179999999999998},"last":0.71099999999999997}]},{"osd":5,"last update":"Fri Oct 6 02:30:02 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.73699999999999999,"15min":0.83699999999999997},"min":{"1min":0.54400000000000004,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.874,"5min":1.3069999999999999,"15min":1.9830000000000001},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.748,"15min":0.80700000000000005},"min":{"1min":0.55400000000000005,"5min":0.53600000000000003,"15min":0.47299999999999998},"max":{"1min":0.91300000000000003,"5min":1.327,"15min":1.3380000000000001},"last":0.67200000000000004}]},{"osd":6,"last update":"Fri Oct 6 02:30:02 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.78400000000000003,"15min":0.84799999999999998},"min":{"1min":0.60899999999999999,"5min":0.52700000000000002,"15min":0.45600000000000002},"max":{"1min":1.012,"5min":1.0609999999999999,"15min":1.829},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.80000000000000004,"15min":0.88900000000000001},"min":{"1min":0.61099999999999999,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.93100000000000005,"5min":1.351,"15min":2.052},"last":0.74299999999999999}]},{"osd":7,"last update":"Fri Oct 6 02:30:02 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.76300000000000001,"15min":0.81999999999999995},"min":{"1min":0.59399999999999997,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.90400000000000003,"5min":1.3120000000000001,"15min":1.3340000000000001},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.77800000000000002,"15min":0.85699999999999998},"min":{"1min":0.51600000000000001,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.018,"5min":1.2110000000000001,"15min":1.895},"last":0.58799999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574949,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303740,"kb_used_data":2356,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068100,"statfs":{"total":96636764160,"available":96325734400,"internally_reserved":0,"allocated":2412544,"data_stored":1434853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 02:30:14 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.44900000000000001,"15min":0.45600000000000002},"min":{"1min":0.30199999999999999,"5min":0.28899999999999998,"15min":0.28000000000000003},"max":{"1min":2.6120000000000001,"5min":2.6120000000000001,"15min":2.6120000000000001},"last":0.376},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.48699999999999999,"15min":0.48399999999999999},"min":{"1min":0.32000000000000001,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":3.4199999999999999,"5min":3.4199999999999999,"15min":3.4199999999999999},"last":0.51400000000000001}]},{"osd":2,"last update":"Fri Oct 6 02:30:14 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.52900000000000003,"15min":0.52000000000000002},"min":{"1min":0.375,"5min":0.35999999999999999,"15min":0.34699999999999998},"max":{"1min":2.5699999999999998,"5min":3.0339999999999998,"15min":3.0339999999999998},"last":0.432},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.52700000000000002,"15min":0.52300000000000002},"min":{"1min":0.36499999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":3.343,"5min":3.343,"15min":3.343},"last":0.57599999999999996}]},{"osd":3,"last update":"Fri Oct 6 02:30:14 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.50900000000000001,"15min":0.50800000000000001},"min":{"1min":0.33900000000000002,"5min":0.31,"15min":0.31},"max":{"1min":3.6579999999999999,"5min":3.6579999999999999,"15min":3.6579999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.59699999999999998,"15min":0.59599999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":2.6890000000000001,"5min":2.6890000000000001,"15min":2.6890000000000001},"last":0.56899999999999995}]},{"osd":4,"last update":"Fri Oct 6 02:30:14 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.77900000000000003,"15min":0.79800000000000004},"min":{"1min":0.56100000000000005,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":3.2999999999999998,"5min":3.2999999999999998,"15min":3.2999999999999998},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.76700000000000002,"15min":0.79500000000000004},"min":{"1min":0.51600000000000001,"5min":0.51000000000000001,"15min":0.48499999999999999},"max":{"1min":2.2930000000000001,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":0.93899999999999995}]},{"osd":5,"last update":"Fri Oct 6 02:30:14 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.76000000000000001,"15min":0.78000000000000003},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.012,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":1.038},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.80400000000000005,"15min":0.80200000000000005},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":2.5070000000000001,"5min":3.1110000000000002,"15min":3.1110000000000002},"last":1.02}]},{"osd":6,"last update":"Fri Oct 6 02:30:14 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":2.6459999999999999,"5min":3.1600000000000001,"15min":3.1600000000000001},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.78400000000000003,"15min":0.81999999999999995},"min":{"1min":0.56499999999999995,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":2.3250000000000002,"5min":2.3250000000000002,"15min":2.3250000000000002},"last":0.98199999999999998}]},{"osd":7,"last update":"Fri Oct 6 02:30:14 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.82699999999999996,"15min":0.82899999999999996},"min":{"1min":0.621,"5min":0.56899999999999995,"15min":0.56000000000000005},"max":{"1min":3.3809999999999998,"5min":3.3809999999999998,"15min":3.3809999999999998},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.78000000000000003,"15min":0.81799999999999995},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.53300000000000003},"max":{"1min":2.3740000000000001,"5min":2.3740000000000001,"15min":2.3740000000000001},"last":0.98999999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574950,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303152,"kb_used_data":1768,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068688,"statfs":{"total":96636764160,"available":96326336512,"internally_reserved":0,"allocated":1810432,"data_stored":844381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:30:10 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.43099999999999999,"15min":0.437},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.502,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.307},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.47299999999999998,"15min":0.48199999999999998},"min":{"1min":0.35799999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.64100000000000001,"5min":0.88700000000000001,"15min":1.0800000000000001},"last":0.36899999999999999}]},{"osd":1,"last update":"Fri Oct 6 02:30:10 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.47799999999999998,"15min":0.498},"min":{"1min":0.36299999999999999,"5min":0.33600000000000002,"15min":0.32400000000000001},"max":{"1min":0.55500000000000005,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.52700000000000002,"15min":0.58999999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34599999999999997},"max":{"1min":0.66000000000000003,"5min":1.778,"15min":1.778},"last":0.40000000000000002}]},{"osd":3,"last update":"Fri Oct 6 02:30:10 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.50900000000000001,"15min":0.52900000000000003},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.79500000000000004,"5min":0.83699999999999997,"15min":0.97499999999999998},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.59299999999999997,"15min":0.61399999999999999},"min":{"1min":0.48099999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.77100000000000002,"5min":0.84699999999999998,"15min":1.028},"last":0.45300000000000001}]},{"osd":4,"last update":"Fri Oct 6 02:30:10 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.751,"15min":0.77300000000000002},"min":{"1min":0.51500000000000001,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.97499999999999998,"5min":1.109,"15min":1.1319999999999999},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.78400000000000003,"15min":0.83499999999999996},"min":{"1min":0.57999999999999996,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.96099999999999997,"5min":1.907,"15min":1.907},"last":0.57199999999999995}]},{"osd":5,"last update":"Fri Oct 6 02:30:10 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.73999999999999999,"15min":0.79000000000000004},"min":{"1min":0.53200000000000003,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.97199999999999998,"5min":1.109,"15min":1.1850000000000001},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.76300000000000001,"15min":0.81000000000000005},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0129999999999999,"5min":1.153,"15min":1.258},"last":0.72299999999999998}]},{"osd":6,"last update":"Fri Oct 6 02:30:10 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.79500000000000004,"15min":0.83799999999999997},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.98699999999999999,"5min":1.827,"15min":1.827},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.77500000000000002,"15min":0.79300000000000004},"min":{"1min":0.60299999999999998,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0660000000000001,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.57999999999999996}]},{"osd":7,"last update":"Fri Oct 6 02:30:10 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.80500000000000005,"15min":0.86799999999999999},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0369999999999999,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.78600000000000003,"15min":0.82399999999999995},"min":{"1min":0.66100000000000003,"5min":0.55300000000000005,"15min":0.54800000000000004},"max":{"1min":1.01,"5min":1.125,"15min":1.1479999999999999},"last":0.624}]}]},{"osd":3,"up_from":13,"seq":55834574947,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303160,"kb_used_data":1776,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068680,"statfs":{"total":96636764160,"available":96326328320,"internally_reserved":0,"allocated":1818624,"data_stored":839824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:30:10 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.443,"15min":0.49399999999999999},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.56200000000000006,"5min":1.0489999999999999,"15min":2.024},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.49199999999999999,"15min":0.51500000000000001},"min":{"1min":0.311,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.754,"5min":0.84699999999999998,"15min":2.181},"last":0.31900000000000001}]},{"osd":1,"last update":"Fri Oct 6 02:30:10 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.46899999999999997,"15min":0.64100000000000001},"min":{"1min":0.34799999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.65100000000000002,"5min":0.66100000000000003,"15min":4.492},"last":0.38},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.52600000000000002,"15min":0.68700000000000006},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.76200000000000001,"5min":1.079,"15min":4.569},"last":0.53600000000000003}]},{"osd":2,"last update":"Fri Oct 6 02:30:10 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.627,"15min":0.65500000000000003},"min":{"1min":0.372,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.82599999999999996,"5min":1.024,"15min":2.0870000000000002},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.59299999999999997,"15min":0.63700000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.76800000000000002,"5min":0.94199999999999995,"15min":2.4260000000000002},"last":0.42999999999999999}]},{"osd":4,"last update":"Fri Oct 6 02:30:10 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.73799999999999999,"15min":0.81100000000000005},"min":{"1min":0.55000000000000004,"5min":0.502,"15min":0.44800000000000001},"max":{"1min":0.94199999999999995,"5min":1.0149999999999999,"15min":2.214},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.76400000000000001,"15min":0.80000000000000004},"min":{"1min":0.54400000000000004,"5min":0.50800000000000001,"15min":0.5},"max":{"1min":1.034,"5min":1.079,"15min":1.2869999999999999},"last":0.85299999999999998}]},{"osd":5,"last update":"Fri Oct 6 02:30:10 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.76300000000000001,"15min":0.80900000000000005},"min":{"1min":0.497,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.077,"5min":1.077,"15min":1.3029999999999999},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.752,"5min":0.76000000000000001,"15min":0.83499999999999996},"min":{"1min":0.60099999999999998,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.0029999999999999,"5min":1.0580000000000001,"15min":2.242},"last":0.71499999999999997}]},{"osd":6,"last update":"Fri Oct 6 02:30:10 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.77400000000000002,"15min":0.81100000000000005},"min":{"1min":0.625,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.98199999999999998,"5min":1.0489999999999999,"15min":1.357},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.80300000000000005,"15min":0.82499999999999996},"min":{"1min":0.60599999999999998,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.014,"5min":1.0529999999999999,"15min":1.3100000000000001},"last":0.69699999999999995}]},{"osd":7,"last update":"Fri Oct 6 02:30:10 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.78200000000000003,"15min":0.84399999999999997},"min":{"1min":0.57299999999999995,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.88400000000000001,"5min":1.077,"15min":2.27},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.79800000000000004,"15min":0.84399999999999997},"min":{"1min":0.57499999999999996,"5min":0.55600000000000005,"15min":0.53200000000000003},"max":{"1min":1.0029999999999999,"5min":1.0760000000000001,"15min":1.325},"last":0.877}]}]},{"osd":4,"up_from":13,"seq":55834574949,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303156,"kb_used_data":1772,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068684,"statfs":{"total":96636764160,"available":96326332416,"internally_reserved":0,"allocated":1814528,"data_stored":838448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:30:06 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68100000000000005,"15min":0.95799999999999996},"min":{"1min":0.46200000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":12.337},"last":0.751},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68400000000000005,"15min":0.88200000000000001},"min":{"1min":0.495,"5min":0.44400000000000001,"15min":0.438},"max":{"1min":0.95299999999999996,"5min":1.3520000000000001,"15min":9.6539999999999999},"last":0.66200000000000003}]},{"osd":1,"last update":"Fri Oct 6 02:30:06 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.68000000000000005,"15min":0.69399999999999995},"min":{"1min":0.47799999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.042,"5min":1.129,"15min":3.181},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.70999999999999996,"15min":0.98299999999999998},"min":{"1min":0.46300000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.98799999999999999,"5min":0.995,"15min":12.946999999999999},"last":0.69999999999999996}]},{"osd":2,"last update":"Fri Oct 6 02:30:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.71599999999999997,"15min":1.006},"min":{"1min":0.47999999999999998,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.1319999999999999,"5min":1.2989999999999999,"15min":12.377000000000001},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.745,"5min":0.73799999999999999,"15min":0.85599999999999998},"min":{"1min":0.56899999999999995,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.93700000000000006,"5min":1.0529999999999999,"15min":6.3220000000000001},"last":0.626}]},{"osd":3,"last update":"Fri Oct 6 02:30:06 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72899999999999998,"15min":0.95599999999999996},"min":{"1min":0.55300000000000005,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.96299999999999997,"5min":1.5229999999999999,"15min":9.7720000000000002},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71799999999999997,"15min":0.76800000000000002},"min":{"1min":0.54600000000000004,"5min":0.498,"15min":0.47999999999999998},"max":{"1min":0.90700000000000003,"5min":1.212,"15min":3.0840000000000001},"last":0.76700000000000002}]},{"osd":5,"last update":"Fri Oct 6 02:30:06 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.68700000000000006,"15min":0.82899999999999996},"min":{"1min":0.42099999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.99199999999999999,"5min":1.02,"15min":6.2690000000000001},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.745,"5min":0.71199999999999997,"15min":0.997},"min":{"1min":0.45700000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.0589999999999999,"5min":1.4359999999999999,"15min":10.769},"last":0.72699999999999998}]},{"osd":6,"last update":"Fri Oct 6 02:30:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.68799999999999994,"15min":0.753},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.95699999999999996,"5min":1.4590000000000001,"15min":3.1349999999999998},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.76900000000000002,"15min":0.94999999999999996},"min":{"1min":0.47999999999999998,"5min":0.47799999999999998,"15min":0.45700000000000002},"max":{"1min":1.1579999999999999,"5min":1.3839999999999999,"15min":8.0899999999999999},"last":0.79600000000000004}]},{"osd":7,"last update":"Fri Oct 6 02:30:06 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.76800000000000002,"15min":1.032},"min":{"1min":0.50900000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.1870000000000001,"5min":1.494,"15min":9.7089999999999996},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.76600000000000001,"15min":0.97099999999999997},"min":{"1min":0.44400000000000001,"5min":0.374,"15min":0.374},"max":{"1min":1.133,"5min":1.2669999999999999,"15min":7.6719999999999997},"last":0.84899999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574949,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303160,"kb_used_data":1776,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068680,"statfs":{"total":96636764160,"available":96326328320,"internally_reserved":0,"allocated":1818624,"data_stored":844505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:30:04 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63900000000000001,"15min":0.627},"min":{"1min":0.40100000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.88200000000000001,"5min":0.97599999999999998,"15min":1.4279999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.626,"5min":0.66000000000000003,"15min":0.61599999999999999},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.38700000000000001},"max":{"1min":0.92700000000000005,"5min":1.099,"15min":1.099},"last":0.55100000000000005}]},{"osd":1,"last update":"Fri Oct 6 02:30:04 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.67000000000000004,"15min":0.65100000000000002},"min":{"1min":0.44500000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.85599999999999998,"5min":1.0660000000000001,"15min":1.089},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.68999999999999995,"15min":0.67500000000000004},"min":{"1min":0.49099999999999999,"5min":0.40699999999999997,"15min":0.40500000000000003},"max":{"1min":0.93600000000000005,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.65000000000000002}]},{"osd":2,"last update":"Fri Oct 6 02:30:04 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.69799999999999995,"15min":0.67300000000000004},"min":{"1min":0.41299999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.88500000000000001,"5min":1.016,"15min":1.016},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.63,"5min":0.68899999999999995,"15min":0.68100000000000005},"min":{"1min":0.48799999999999999,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.80500000000000005,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.56999999999999995}]},{"osd":3,"last update":"Fri Oct 6 02:30:04 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.752,"15min":0.73199999999999998},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.48799999999999999},"max":{"1min":0.98499999999999999,"5min":1.159,"15min":1.159},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.753,"5min":0.79000000000000004,"15min":0.76900000000000002},"min":{"1min":0.56999999999999995,"5min":0.56200000000000006,"15min":0.54100000000000004},"max":{"1min":0.86799999999999999,"5min":1.194,"15min":1.194},"last":0.62}]},{"osd":4,"last update":"Fri Oct 6 02:30:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.65200000000000002,"15min":0.67500000000000004},"min":{"1min":0.439,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.93000000000000005,"5min":1.012,"15min":1.1499999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.66600000000000004,"15min":0.68500000000000005},"min":{"1min":0.498,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.92700000000000005,"5min":1.25,"15min":1.25},"last":0.60699999999999998}]},{"osd":6,"last update":"Fri Oct 6 02:30:04 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.72799999999999998,"15min":0.74299999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.40300000000000002},"max":{"1min":0.95699999999999996,"5min":1.0549999999999999,"15min":1.1719999999999999},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.80100000000000005,"15min":0.81899999999999995},"min":{"1min":0.502,"5min":0.373,"15min":0.373},"max":{"1min":0.98299999999999998,"5min":1.1200000000000001,"15min":1.2170000000000001},"last":0.755}]},{"osd":7,"last update":"Fri Oct 6 02:30:04 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.747,"15min":0.76500000000000001},"min":{"1min":0.57199999999999995,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.0169999999999999,"5min":1.23,"15min":1.23},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79800000000000004,"15min":0.82899999999999996},"min":{"1min":0.58499999999999996,"5min":0.437,"15min":0.437},"max":{"1min":0.93300000000000005,"5min":1.1040000000000001,"15min":1.5640000000000001},"last":0.69499999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T02:30:56.946 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T02:30:56.946 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T02:30:57.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.175+0000 7f9edab9d700 1 Processor -- start 2023-10-06T02:30:57.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.175+0000 7f9edab9d700 1 -- start start 2023-10-06T02:30:57.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.176+0000 7f9edab9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9ed413fb20 0x7f9ed415a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:57.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.176+0000 7f9edab9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9ed413fef0 0x7f9ed40c7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:57.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.176+0000 7f9edab9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9ed406aaa0 0x7f9ed40c7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:57.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.176+0000 7f9edab9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ed40c8410 con 0x7f9ed413fef0 2023-10-06T02:30:57.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.176+0000 7f9edab9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ed415ae70 con 0x7f9ed406aaa0 2023-10-06T02:30:57.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.176+0000 7f9edab9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed406af60 con 0x7f9ed413fb20 2023-10-06T02:30:57.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.177+0000 7f9ed939a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9ed413fef0 0x7f9ed40c7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:57.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.177+0000 7f9ed939a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9ed413fef0 0x7f9ed40c7990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54826/0 (socket says 172.21.15.31:54826) 2023-10-06T02:30:57.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.177+0000 7f9ed939a700 1 -- 172.21.15.31:0/1849289164 learned_addr learned my addr 172.21.15.31:0/1849289164 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:57.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.177+0000 7f9ed9b9b700 1 --2- 172.21.15.31:0/1849289164 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9ed413fb20 0x7f9ed415a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:57.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.177+0000 7f9eda39c700 1 --2- 172.21.15.31:0/1849289164 >> v2:172.21.15.62:3300/0 conn(0x7f9ed406aaa0 0x7f9ed40c7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:57.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.177+0000 7f9ed939a700 1 -- 172.21.15.31:0/1849289164 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9ed413fb20 msgr2=0x7f9ed415a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:57.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.177+0000 7f9ed939a700 1 --2- 172.21.15.31:0/1849289164 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9ed413fb20 0x7f9ed415a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:57.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.177+0000 7f9ed939a700 1 -- 172.21.15.31:0/1849289164 >> v2:172.21.15.62:3300/0 conn(0x7f9ed406aaa0 msgr2=0x7f9ed40c7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:57.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.177+0000 7f9ed939a700 1 --2- 172.21.15.31:0/1849289164 >> v2:172.21.15.62:3300/0 conn(0x7f9ed406aaa0 0x7f9ed40c7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:57.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.177+0000 7f9ed939a700 1 -- 172.21.15.31:0/1849289164 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ed406b240 con 0x7f9ed413fef0 2023-10-06T02:30:57.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.177+0000 7f9eda39c700 1 --2- 172.21.15.31:0/1849289164 >> v2:172.21.15.62:3300/0 conn(0x7f9ed406aaa0 0x7f9ed40c7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:30:57.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.178+0000 7f9ed939a700 1 --2- 172.21.15.31:0/1849289164 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ed413fef0 0x7f9ed40c7990 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4156920 tx=0x7f9ec4009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8182e67debcb292a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:57.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.178+0000 7f9ed8b99700 1 -- 172.21.15.31:0/1849289164 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec4005c10 con 0x7f9ed413fef0 2023-10-06T02:30:57.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.178+0000 7f9ed8b99700 1 -- 172.21.15.31:0/1849289164 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ec4005d70 con 0x7f9ed413fef0 2023-10-06T02:30:57.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.178+0000 7f9ed8b99700 1 -- 172.21.15.31:0/1849289164 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec400b880 con 0x7f9ed413fef0 2023-10-06T02:30:57.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.178+0000 7f9edab9d700 1 -- 172.21.15.31:0/1849289164 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ed413fef0 msgr2=0x7f9ed40c7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:57.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.178+0000 7f9edab9d700 1 --2- 172.21.15.31:0/1849289164 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ed413fef0 0x7f9ed40c7990 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4156920 tx=0x7f9ec4009220 comp rx=0 tx=0).stop 2023-10-06T02:30:57.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.178+0000 7f9edab9d700 1 -- 172.21.15.31:0/1849289164 shutdown_connections 2023-10-06T02:30:57.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.178+0000 7f9edab9d700 1 --2- 172.21.15.31:0/1849289164 >> v2:172.21.15.62:3300/0 conn(0x7f9ed406aaa0 0x7f9ed40c7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:57.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.178+0000 7f9edab9d700 1 --2- 172.21.15.31:0/1849289164 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9ed413fb20 0x7f9ed415a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:57.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.178+0000 7f9edab9d700 1 --2- 172.21.15.31:0/1849289164 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ed413fef0 0x7f9ed40c7990 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:57.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.178+0000 7f9edab9d700 1 -- 172.21.15.31:0/1849289164 >> 172.21.15.31:0/1849289164 conn(0x7f9ed406d170 msgr2=0x7f9ed40cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:57.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.179+0000 7f9edab9d700 1 -- 172.21.15.31:0/1849289164 shutdown_connections 2023-10-06T02:30:57.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.179+0000 7f9edab9d700 1 -- 172.21.15.31:0/1849289164 wait complete. 2023-10-06T02:30:57.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.180+0000 7f9edab9d700 1 Processor -- start 2023-10-06T02:30:57.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.180+0000 7f9edab9d700 1 -- start start 2023-10-06T02:30:57.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.181+0000 7f9edab9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9ed406aaa0 0x7f9ed413b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:57.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.181+0000 7f9edab9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ed413fb20 0x7f9ed4169850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:57.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.181+0000 7f9edab9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ed413fef0 0x7f9ed40acd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:57.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.181+0000 7f9edab9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed41570b0 con 0x7f9ed413fb20 2023-10-06T02:30:57.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.181+0000 7f9edab9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed4157620 con 0x7f9ed413fef0 2023-10-06T02:30:57.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.181+0000 7f9edab9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed416a310 con 0x7f9ed406aaa0 2023-10-06T02:30:57.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.181+0000 7f9ed939a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ed413fb20 0x7f9ed4169850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:57.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.181+0000 7f9ed939a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ed413fb20 0x7f9ed4169850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54834/0 (socket says 172.21.15.31:54834) 2023-10-06T02:30:57.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.181+0000 7f9ed939a700 1 -- 172.21.15.31:0/37643524 learned_addr learned my addr 172.21.15.31:0/37643524 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:57.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.181+0000 7f9eda39c700 1 --2- 172.21.15.31:0/37643524 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ed413fef0 0x7f9ed40acd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:57.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.181+0000 7f9ed9b9b700 1 --2- 172.21.15.31:0/37643524 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9ed406aaa0 0x7f9ed413b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:57.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.182+0000 7f9ed939a700 1 -- 172.21.15.31:0/37643524 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9ed406aaa0 msgr2=0x7f9ed413b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:57.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.182+0000 7f9ed939a700 1 --2- 172.21.15.31:0/37643524 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9ed406aaa0 0x7f9ed413b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:57.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.182+0000 7f9ed939a700 1 -- 172.21.15.31:0/37643524 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ed413fef0 msgr2=0x7f9ed40acd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:57.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.182+0000 7f9ed939a700 1 --2- 172.21.15.31:0/37643524 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ed413fef0 0x7f9ed40acd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:57.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.182+0000 7f9ed939a700 1 -- 172.21.15.31:0/37643524 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ec4009480 con 0x7f9ed413fb20 2023-10-06T02:30:57.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.182+0000 7f9ed9b9b700 1 --2- 172.21.15.31:0/37643524 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9ed406aaa0 0x7f9ed413b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:30:57.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.182+0000 7f9ed939a700 1 --2- 172.21.15.31:0/37643524 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ed413fb20 0x7f9ed4169850 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4071320 tx=0x7f9ec4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:57.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.182+0000 7f9ec9ffb700 1 -- 172.21.15.31:0/37643524 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec4011a80 con 0x7f9ed413fb20 2023-10-06T02:30:57.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.182+0000 7f9ec9ffb700 1 -- 172.21.15.31:0/37643524 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ec4011be0 con 0x7f9ed413fb20 2023-10-06T02:30:57.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.182+0000 7f9ec9ffb700 1 -- 172.21.15.31:0/37643524 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec401b430 con 0x7f9ed413fb20 2023-10-06T02:30:57.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.182+0000 7f9edab9d700 1 -- 172.21.15.31:0/37643524 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ed416a5f0 con 0x7f9ed413fb20 2023-10-06T02:30:57.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.182+0000 7f9edab9d700 1 -- 172.21.15.31:0/37643524 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ed4156d60 con 0x7f9ed413fb20 2023-10-06T02:30:57.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.183+0000 7f9edab9d700 1 -- 172.21.15.31:0/37643524 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ed41577b0 con 0x7f9ed413fb20 2023-10-06T02:30:57.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.184+0000 7f9ec9ffb700 1 -- 172.21.15.31:0/37643524 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f9ec401b860 con 0x7f9ed413fb20 2023-10-06T02:30:57.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.185+0000 7f9ec9ffb700 1 --2- 172.21.15.31:0/37643524 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9eb80674d0 0x7f9eb8069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:57.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.185+0000 7f9ec9ffb700 1 -- 172.21.15.31:0/37643524 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f9ec4086b40 con 0x7f9ed413fb20 2023-10-06T02:30:57.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.186+0000 7f9ed9b9b700 1 --2- 172.21.15.31:0/37643524 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9eb80674d0 0x7f9eb8069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:57.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.186+0000 7f9ed9b9b700 1 --2- 172.21.15.31:0/37643524 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9eb80674d0 0x7f9eb8069980 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0006fd0 tx=0x7f9ed0008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:57.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.189+0000 7f9ec9ffb700 1 -- 172.21.15.31:0/37643524 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9ec4053380 con 0x7f9ed413fb20 2023-10-06T02:30:57.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.470+0000 7f9edab9d700 1 -- 172.21.15.31:0/37643524 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f9ed404c3c0 con 0x7f9ed413fb20 2023-10-06T02:30:57.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.475+0000 7f9ec9ffb700 1 -- 172.21.15.31:0/37643524 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v22) v1 ==== 190+0+0 (secure 0 0 0) 0x7f9ec402a020 con 0x7f9ed413fb20 2023-10-06T02:30:57.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.477+0000 7f9edab9d700 1 -- 172.21.15.31:0/37643524 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9eb80674d0 msgr2=0x7f9eb8069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:57.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.477+0000 7f9edab9d700 1 --2- 172.21.15.31:0/37643524 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9eb80674d0 0x7f9eb8069980 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0006fd0 tx=0x7f9ed0008040 comp rx=0 tx=0).stop 2023-10-06T02:30:57.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.477+0000 7f9edab9d700 1 -- 172.21.15.31:0/37643524 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ed413fb20 msgr2=0x7f9ed4169850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:57.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.477+0000 7f9edab9d700 1 --2- 172.21.15.31:0/37643524 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ed413fb20 0x7f9ed4169850 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4071320 tx=0x7f9ec4016000 comp rx=0 tx=0).stop 2023-10-06T02:30:57.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.477+0000 7f9edab9d700 1 -- 172.21.15.31:0/37643524 shutdown_connections 2023-10-06T02:30:57.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.477+0000 7f9edab9d700 1 --2- 172.21.15.31:0/37643524 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ed413fef0 0x7f9ed40acd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:57.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.477+0000 7f9edab9d700 1 --2- 172.21.15.31:0/37643524 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9eb80674d0 0x7f9eb8069980 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:57.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.477+0000 7f9edab9d700 1 --2- 172.21.15.31:0/37643524 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9ed406aaa0 0x7f9ed413b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:57.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.477+0000 7f9edab9d700 1 --2- 172.21.15.31:0/37643524 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ed413fb20 0x7f9ed4169850 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:57.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.477+0000 7f9edab9d700 1 -- 172.21.15.31:0/37643524 >> 172.21.15.31:0/37643524 conn(0x7f9ed406d170 msgr2=0x7f9ed413c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:57.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.477+0000 7f9edab9d700 1 -- 172.21.15.31:0/37643524 shutdown_connections 2023-10-06T02:30:57.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.477+0000 7f9edab9d700 1 -- 172.21.15.31:0/37643524 wait complete. 2023-10-06T02:30:57.479 INFO:teuthology.orchestra.run.smithi031.stderr:updated caps for client.0 2023-10-06T02:30:57.495 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:30:57.639 INFO:tasks.ceph.mds.d.smithi062.stderr:2023-10-06T02:30:57.638+0000 7f491b5f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T02:30:57.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.683+0000 7fb501f9d700 1 Processor -- start 2023-10-06T02:30:57.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.683+0000 7fb501f9d700 1 -- start start 2023-10-06T02:30:57.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.684+0000 7fb501f9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb4fc159640 0x7fb4fc06d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:57.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.684+0000 7fb501f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb4fc159a10 0x7fb4fc0ca7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:57.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.684+0000 7fb501f9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4fc0c7420 con 0x7fb4fc070410 2023-10-06T02:30:57.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.684+0000 7fb501f9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4fc06d5a0 con 0x7fb4fc159640 2023-10-06T02:30:57.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.684+0000 7fb501f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4fc06d820 con 0x7fb4fc159a10 2023-10-06T02:30:57.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.685+0000 7fb50179c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fb4fc070410 0x7fb4fc06a2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:47942/0 (socket says 172.21.15.31:47942) 2023-10-06T02:30:57.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.685+0000 7fb50179c700 1 -- 172.21.15.31:0/3727662509 learned_addr learned my addr 172.21.15.31:0/3727662509 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:30:57.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.685+0000 7fb500f9b700 1 --2- 172.21.15.31:0/3727662509 >> v2:172.21.15.62:3300/0 conn(0x7fb4fc159640 0x7fb4fc06d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:57.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.685+0000 7fb4fbfff700 1 --2- 172.21.15.31:0/3727662509 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb4fc159a10 0x7fb4fc0ca7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:57.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.685+0000 7fb4fb7fe700 1 -- 172.21.15.31:0/3727662509 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1708230110 0 0) 0x7fb4fc0c7420 con 0x7fb4fc070410 2023-10-06T02:30:57.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.685+0000 7fb4fb7fe700 1 -- 172.21.15.31:0/3727662509 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4ec003570 con 0x7fb4fc070410 2023-10-06T02:30:57.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.685+0000 7fb500f9b700 1 -- 172.21.15.31:0/3727662509 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb4fc159a10 msgr2=0x7fb4fc0ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:57.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.685+0000 7fb500f9b700 1 --2- 172.21.15.31:0/3727662509 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb4fc159a10 0x7fb4fc0ca7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:57.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.685+0000 7fb500f9b700 1 -- 172.21.15.31:0/3727662509 >> v1:172.21.15.31:6789/0 conn(0x7fb4fc070410 legacy=0x7fb4fc06a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:57.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.685+0000 7fb500f9b700 1 -- 172.21.15.31:0/3727662509 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4fc06db00 con 0x7fb4fc159640 2023-10-06T02:30:57.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.686+0000 7fb500f9b700 1 --2- 172.21.15.31:0/3727662509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4fc159640 0x7fb4fc06d060 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fb4f00098e0 tx=0x7fb4f00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26aa340bc5d598c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:57.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.686+0000 7fb4fb7fe700 1 -- 172.21.15.31:0/3727662509 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f0011630 con 0x7fb4fc159640 2023-10-06T02:30:57.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.686+0000 7fb4fb7fe700 1 -- 172.21.15.31:0/3727662509 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4f0011790 con 0x7fb4fc159640 2023-10-06T02:30:57.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.686+0000 7fb4fb7fe700 1 -- 172.21.15.31:0/3727662509 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f0011b30 con 0x7fb4fc159640 2023-10-06T02:30:57.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.686+0000 7fb501f9d700 1 -- 172.21.15.31:0/3727662509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4fc159640 msgr2=0x7fb4fc06d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:57.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.686+0000 7fb501f9d700 1 --2- 172.21.15.31:0/3727662509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4fc159640 0x7fb4fc06d060 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fb4f00098e0 tx=0x7fb4f00091a0 comp rx=0 tx=0).stop 2023-10-06T02:30:57.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.686+0000 7fb501f9d700 1 -- 172.21.15.31:0/3727662509 shutdown_connections 2023-10-06T02:30:57.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.686+0000 7fb501f9d700 1 --2- 172.21.15.31:0/3727662509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4fc159640 0x7fb4fc06d060 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:57.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.687+0000 7fb501f9d700 1 --2- 172.21.15.31:0/3727662509 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb4fc159a10 0x7fb4fc0ca7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:57.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.687+0000 7fb501f9d700 1 -- 172.21.15.31:0/3727662509 >> 172.21.15.31:0/3727662509 conn(0x7fb4fc06e720 msgr2=0x7fb4fc06eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:57.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.687+0000 7fb501f9d700 1 -- 172.21.15.31:0/3727662509 shutdown_connections 2023-10-06T02:30:57.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.687+0000 7fb501f9d700 1 -- 172.21.15.31:0/3727662509 wait complete. 2023-10-06T02:30:57.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.688+0000 7fb501f9d700 1 Processor -- start 2023-10-06T02:30:57.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.688+0000 7fb501f9d700 1 -- start start 2023-10-06T02:30:57.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.689+0000 7fb501f9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4fc070410 0x7fb4fc16ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:57.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.689+0000 7fb501f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb4fc159640 0x7fb4fc16b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:57.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.689+0000 7fb501f9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4fc159a10 0x7fb4fc1544d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:57.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.689+0000 7fb501f9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4fc159f50 con 0x7fb4fc070410 2023-10-06T02:30:57.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.689+0000 7fb501f9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4fc0c4b20 con 0x7fb4fc159a10 2023-10-06T02:30:57.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.689+0000 7fb501f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4fc0b1b50 con 0x7fb4fc159640 2023-10-06T02:30:57.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.689+0000 7fb500f9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4fc070410 0x7fb4fc16ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:57.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.689+0000 7fb500f9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4fc070410 0x7fb4fc16ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54848/0 (socket says 172.21.15.31:54848) 2023-10-06T02:30:57.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.689+0000 7fb500f9b700 1 -- 172.21.15.31:0/701247201 learned_addr learned my addr 172.21.15.31:0/701247201 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:57.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.689+0000 7fb4fbfff700 1 --2- 172.21.15.31:0/701247201 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb4fc159640 0x7fb4fc16b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:57.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.689+0000 7fb50179c700 1 --2- 172.21.15.31:0/701247201 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4fc159a10 0x7fb4fc1544d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:57.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.690+0000 7fb500f9b700 1 -- 172.21.15.31:0/701247201 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb4fc159640 msgr2=0x7fb4fc16b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:57.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.690+0000 7fb500f9b700 1 --2- 172.21.15.31:0/701247201 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb4fc159640 0x7fb4fc16b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:57.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.690+0000 7fb500f9b700 1 -- 172.21.15.31:0/701247201 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4fc159a10 msgr2=0x7fb4fc1544d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:57.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.690+0000 7fb500f9b700 1 --2- 172.21.15.31:0/701247201 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4fc159a10 0x7fb4fc1544d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:57.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.690+0000 7fb500f9b700 1 -- 172.21.15.31:0/701247201 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4f0009420 con 0x7fb4fc070410 2023-10-06T02:30:57.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.690+0000 7fb50179c700 1 --2- 172.21.15.31:0/701247201 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4fc159a10 0x7fb4fc1544d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:30:57.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.690+0000 7fb500f9b700 1 --2- 172.21.15.31:0/701247201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4fc070410 0x7fb4fc16ab60 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fb4f0009ed0 tx=0x7fb4f0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:57.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.690+0000 7fb4f8ff9700 1 -- 172.21.15.31:0/701247201 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f0000d10 con 0x7fb4fc070410 2023-10-06T02:30:57.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.690+0000 7fb4f8ff9700 1 -- 172.21.15.31:0/701247201 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4f0011b10 con 0x7fb4fc070410 2023-10-06T02:30:57.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.690+0000 7fb4f8ff9700 1 -- 172.21.15.31:0/701247201 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f0025640 con 0x7fb4fc070410 2023-10-06T02:30:57.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.690+0000 7fb501f9d700 1 -- 172.21.15.31:0/701247201 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4fc169fc0 con 0x7fb4fc070410 2023-10-06T02:30:57.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.690+0000 7fb501f9d700 1 -- 172.21.15.31:0/701247201 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4fc0c3f10 con 0x7fb4fc070410 2023-10-06T02:30:57.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.691+0000 7fb501f9d700 1 -- 172.21.15.31:0/701247201 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4fc04c3c0 con 0x7fb4fc070410 2023-10-06T02:30:57.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.692+0000 7fb4f8ff9700 1 -- 172.21.15.31:0/701247201 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7fb4f0025a30 con 0x7fb4fc070410 2023-10-06T02:30:57.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.694+0000 7fb4f8ff9700 1 --2- 172.21.15.31:0/701247201 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb4e00674d0 0x7fb4e0069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:57.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.694+0000 7fb4f8ff9700 1 -- 172.21.15.31:0/701247201 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fb4f00869b0 con 0x7fb4fc070410 2023-10-06T02:30:57.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.694+0000 7fb4fbfff700 1 --2- 172.21.15.31:0/701247201 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb4e00674d0 0x7fb4e0069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:57.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.695+0000 7fb4fbfff700 1 --2- 172.21.15.31:0/701247201 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb4e00674d0 0x7fb4e0069980 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc06e160 tx=0x7fb4e80091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:57.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.698+0000 7fb4f8ff9700 1 -- 172.21.15.31:0/701247201 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb4f00531f0 con 0x7fb4fc070410 2023-10-06T02:30:57.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.947+0000 7fb501f9d700 1 -- 172.21.15.31:0/701247201 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb4fc0cb2f0 con 0x7fb4fc070410 2023-10-06T02:30:57.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.947+0000 7fb4f8ff9700 1 -- 172.21.15.31:0/701247201 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) v1 ==== 95+0+4603 (secure 0 0 0) 0x7fb4f0056810 con 0x7fb4fc070410 2023-10-06T02:30:57.948 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:30:57.948 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":100,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6923,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/4169296779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":4169296779},{"type":"v1","addr":"172.21.15.31:6835","nonce":4169296779}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95},{"gid":6926,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/3032245108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3032245108},{"type":"v1","addr":"172.21.15.62:6834","nonce":3032245108}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95},{"gid":8469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6833/1522769465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1522769465},{"type":"v1","addr":"172.21.15.31:6833","nonce":1522769465}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:30:51.153736+0000","modified":"2023-10-06T02:30:55.096858+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8487":{"gid":8487,"name":"d","rank":0,"incarnation":96,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/2236458886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2236458886},{"type":"v1","addr":"172.21.15.62:6835","nonce":2236458886}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2023-10-06T02:30:57.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.950+0000 7fb501f9d700 1 -- 172.21.15.31:0/701247201 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb4e00674d0 msgr2=0x7fb4e0069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:57.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.950+0000 7fb501f9d700 1 --2- 172.21.15.31:0/701247201 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb4e00674d0 0x7fb4e0069980 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc06e160 tx=0x7fb4e80091a0 comp rx=0 tx=0).stop 2023-10-06T02:30:57.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.950+0000 7fb501f9d700 1 -- 172.21.15.31:0/701247201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4fc070410 msgr2=0x7fb4fc16ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:57.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.950+0000 7fb501f9d700 1 --2- 172.21.15.31:0/701247201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4fc070410 0x7fb4fc16ab60 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fb4f0009ed0 tx=0x7fb4f0017000 comp rx=0 tx=0).stop 2023-10-06T02:30:57.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.950+0000 7fb501f9d700 1 -- 172.21.15.31:0/701247201 shutdown_connections 2023-10-06T02:30:57.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.950+0000 7fb501f9d700 1 --2- 172.21.15.31:0/701247201 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4fc159a10 0x7fb4fc1544d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:57.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.950+0000 7fb501f9d700 1 --2- 172.21.15.31:0/701247201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4fc070410 0x7fb4fc16ab60 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:57.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.950+0000 7fb501f9d700 1 --2- 172.21.15.31:0/701247201 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb4e00674d0 0x7fb4e0069980 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:57.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.950+0000 7fb501f9d700 1 --2- 172.21.15.31:0/701247201 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb4fc159640 0x7fb4fc16b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:57.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.950+0000 7fb501f9d700 1 -- 172.21.15.31:0/701247201 >> 172.21.15.31:0/701247201 conn(0x7fb4fc06e720 msgr2=0x7fb4fc06ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:57.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.950+0000 7fb501f9d700 1 -- 172.21.15.31:0/701247201 shutdown_connections 2023-10-06T02:30:57.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:57.950+0000 7fb501f9d700 1 -- 172.21.15.31:0/701247201 wait complete. 2023-10-06T02:30:57.952 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 100 2023-10-06T02:30:58.370 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 100, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T02:30:51.153736+0000', 'modified': '2023-10-06T02:30:55.096858+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8487}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8487': {'gid': 8487, 'name': 'd', 'rank': 0, 'incarnation': 96, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.62:6835/2236458886', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6833', 'nonce': 2236458886}, {'type': 'v1', 'addr': '172.21.15.62:6835', 'nonce': 2236458886}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T02:30:58.370 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T02:30:58.371 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-10-06T02:30:58.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.561+0000 7f721f59e700 1 Processor -- start 2023-10-06T02:30:58.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.561+0000 7f721f59e700 1 -- start start 2023-10-06T02:30:58.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.563+0000 7f721f59e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7220142c00 0x7f722006b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:58.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.563+0000 7f721f59e700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f72200c7f90 0x7f722006c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:58.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.563+0000 7f721f59e700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72200ca480 con 0x7f722006bd20 2023-10-06T02:30:58.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.563+0000 7f721f59e700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f722015e240 con 0x7f7220142c00 2023-10-06T02:30:58.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.563+0000 7f721f59e700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f722006b350 con 0x7f72200c7f90 2023-10-06T02:30:58.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.563+0000 7f721e59c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7220142c00 0x7f722006b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:58.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.563+0000 7f721e59c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7220142c00 0x7f722006b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36380/0 (socket says 172.21.15.31:36380) 2023-10-06T02:30:58.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.563+0000 7f721e59c700 1 -- 172.21.15.31:0/303720147 learned_addr learned my addr 172.21.15.31:0/303720147 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:58.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.563+0000 7f721dd9b700 1 --1- 172.21.15.31:0/303720147 >> v1:172.21.15.31:6789/0 conn(0x7f722006bd20 0x7f72200c7880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:36930/0 (socket says 172.21.15.31:36930) 2023-10-06T02:30:58.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.563+0000 7f721ed9d700 1 --2- 172.21.15.31:0/303720147 >> v2:172.21.15.62:3300/0 conn(0x7f72200c7f90 0x7f722006c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:58.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.563+0000 7f721e59c700 1 -- 172.21.15.31:0/303720147 >> v2:172.21.15.62:3300/0 conn(0x7f72200c7f90 msgr2=0x7f722006c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:58.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.563+0000 7f721e59c700 1 --2- 172.21.15.31:0/303720147 >> v2:172.21.15.62:3300/0 conn(0x7f72200c7f90 0x7f722006c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:58.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.563+0000 7f721e59c700 1 -- 172.21.15.31:0/303720147 >> v1:172.21.15.31:6789/0 conn(0x7f722006bd20 legacy=0x7f72200c7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:58.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.563+0000 7f721e59c700 1 -- 172.21.15.31:0/303720147 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f722015df60 con 0x7f7220142c00 2023-10-06T02:30:58.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.564+0000 7f721e59c700 1 --2- 172.21.15.31:0/303720147 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7220142c00 0x7f722006b7e0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f722006d780 tx=0x7f7210009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=764620b393030ff3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:58.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.564+0000 7f721d59a700 1 -- 172.21.15.31:0/303720147 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7210009e70 con 0x7f7220142c00 2023-10-06T02:30:58.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.564+0000 7f721d59a700 1 -- 172.21.15.31:0/303720147 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f721000b860 con 0x7f7220142c00 2023-10-06T02:30:58.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.564+0000 7f721d59a700 1 -- 172.21.15.31:0/303720147 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f721000bc20 con 0x7f7220142c00 2023-10-06T02:30:58.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.564+0000 7f721f59e700 1 -- 172.21.15.31:0/303720147 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7220142c00 msgr2=0x7f722006b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:58.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.564+0000 7f721f59e700 1 --2- 172.21.15.31:0/303720147 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7220142c00 0x7f722006b7e0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f722006d780 tx=0x7f7210009250 comp rx=0 tx=0).stop 2023-10-06T02:30:58.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.565+0000 7f721f59e700 1 -- 172.21.15.31:0/303720147 shutdown_connections 2023-10-06T02:30:58.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.565+0000 7f721f59e700 1 --2- 172.21.15.31:0/303720147 >> v2:172.21.15.62:3300/0 conn(0x7f72200c7f90 0x7f722006c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:58.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.565+0000 7f721f59e700 1 --2- 172.21.15.31:0/303720147 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7220142c00 0x7f722006b7e0 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:58.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.565+0000 7f721f59e700 1 -- 172.21.15.31:0/303720147 >> 172.21.15.31:0/303720147 conn(0x7f72200675a0 msgr2=0x7f72200679a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:58.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.565+0000 7f721f59e700 1 -- 172.21.15.31:0/303720147 shutdown_connections 2023-10-06T02:30:58.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.565+0000 7f721f59e700 1 -- 172.21.15.31:0/303720147 wait complete. 2023-10-06T02:30:58.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.566+0000 7f721f59e700 1 Processor -- start 2023-10-06T02:30:58.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.566+0000 7f721f59e700 1 -- start start 2023-10-06T02:30:58.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.567+0000 7f721f59e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f722006bd20 0x7f7220159190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:58.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.567+0000 7f721f59e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72200c7f90 0x7f72200c61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:58.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.567+0000 7f721f59e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7220142c00 0x7f722015b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:58.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.567+0000 7f721f59e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f722015b310 con 0x7f7220142c00 2023-10-06T02:30:58.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.567+0000 7f721f59e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72200c47c0 con 0x7f72200c7f90 2023-10-06T02:30:58.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.567+0000 7f721f59e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72201696b0 con 0x7f722006bd20 2023-10-06T02:30:58.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.567+0000 7f721ed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7220142c00 0x7f722015b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:58.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.567+0000 7f721ed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7220142c00 0x7f722015b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36390/0 (socket says 172.21.15.31:36390) 2023-10-06T02:30:58.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.567+0000 7f721ed9d700 1 -- 172.21.15.31:0/2146993487 learned_addr learned my addr 172.21.15.31:0/2146993487 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:58.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.568+0000 7f721dd9b700 1 --2- 172.21.15.31:0/2146993487 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72200c7f90 0x7f72200c61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:58.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.568+0000 7f721e59c700 1 --2- 172.21.15.31:0/2146993487 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f722006bd20 0x7f7220159190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:58.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.568+0000 7f721ed9d700 1 -- 172.21.15.31:0/2146993487 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f722006bd20 msgr2=0x7f7220159190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:58.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.568+0000 7f721ed9d700 1 --2- 172.21.15.31:0/2146993487 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f722006bd20 0x7f7220159190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:58.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.568+0000 7f721ed9d700 1 -- 172.21.15.31:0/2146993487 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72200c7f90 msgr2=0x7f72200c61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:58.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.568+0000 7f721ed9d700 1 --2- 172.21.15.31:0/2146993487 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72200c7f90 0x7f72200c61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:58.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.568+0000 7f721ed9d700 1 -- 172.21.15.31:0/2146993487 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72100094f0 con 0x7f7220142c00 2023-10-06T02:30:58.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.568+0000 7f721e59c700 1 --2- 172.21.15.31:0/2146993487 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f722006bd20 0x7f7220159190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:30:58.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.568+0000 7f721ed9d700 1 --2- 172.21.15.31:0/2146993487 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7220142c00 0x7f722015b590 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f7218009ec0 tx=0x7f721800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:58.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.568+0000 7f720e7fc700 1 -- 172.21.15.31:0/2146993487 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f721800cc20 con 0x7f7220142c00 2023-10-06T02:30:58.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.568+0000 7f720e7fc700 1 -- 172.21.15.31:0/2146993487 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f721800f040 con 0x7f7220142c00 2023-10-06T02:30:58.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.568+0000 7f721f59e700 1 -- 172.21.15.31:0/2146993487 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72200c4490 con 0x7f7220142c00 2023-10-06T02:30:58.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.568+0000 7f720e7fc700 1 -- 172.21.15.31:0/2146993487 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7218014480 con 0x7f7220142c00 2023-10-06T02:30:58.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.568+0000 7f721f59e700 1 -- 172.21.15.31:0/2146993487 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f722015b030 con 0x7f7220142c00 2023-10-06T02:30:58.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.570+0000 7f720e7fc700 1 -- 172.21.15.31:0/2146993487 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f7218014830 con 0x7f7220142c00 2023-10-06T02:30:58.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.572+0000 7f720e7fc700 1 --2- 172.21.15.31:0/2146993487 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f72000674d0 0x7f7200069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:58.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.572+0000 7f720e7fc700 1 -- 172.21.15.31:0/2146993487 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f7218085f50 con 0x7f7220142c00 2023-10-06T02:30:58.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.572+0000 7f721e59c700 1 --2- 172.21.15.31:0/2146993487 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f72000674d0 0x7f7200069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:58.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.573+0000 7f721e59c700 1 --2- 172.21.15.31:0/2146993487 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f72000674d0 0x7f7200069980 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f722006d780 tx=0x7f7210017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:58.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.573+0000 7f723311bb80 1 Processor -- start 2023-10-06T02:30:58.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.573+0000 7f723311bb80 1 -- start start 2023-10-06T02:30:58.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.574+0000 7f723311bb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d64ba27e50 0x55d64ba02cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:58.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.574+0000 7f723311bb80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d64ba082c0 0x55d64ba03200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:58.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.574+0000 7f723311bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d64ba09a10 0x55d64ba03740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:58.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.574+0000 7f723311bb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d64b95da70 con 0x55d64ba27e50 2023-10-06T02:30:58.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.574+0000 7f723311bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d64b95b270 con 0x55d64ba09a10 2023-10-06T02:30:58.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.574+0000 7f723311bb80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d64b957060 con 0x55d64ba082c0 2023-10-06T02:30:58.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.574+0000 7f721e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d64ba27e50 0x55d64ba02cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:58.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.574+0000 7f721e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d64ba27e50 0x55d64ba02cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36406/0 (socket says 172.21.15.31:36406) 2023-10-06T02:30:58.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.574+0000 7f721e59c700 1 -- 172.21.15.31:0/3526941276 learned_addr learned my addr 172.21.15.31:0/3526941276 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:58.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.574+0000 7f721dd9b700 1 --2- 172.21.15.31:0/3526941276 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d64ba082c0 0x55d64ba03200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:58.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.574+0000 7f721ed9d700 1 --2- 172.21.15.31:0/3526941276 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d64ba09a10 0x55d64ba03740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:58.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.575+0000 7f721e59c700 1 -- 172.21.15.31:0/3526941276 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d64ba082c0 msgr2=0x55d64ba03200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:58.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.575+0000 7f721e59c700 1 --2- 172.21.15.31:0/3526941276 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d64ba082c0 0x55d64ba03200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:58.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.575+0000 7f721e59c700 1 -- 172.21.15.31:0/3526941276 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d64ba09a10 msgr2=0x55d64ba03740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:58.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.575+0000 7f721e59c700 1 --2- 172.21.15.31:0/3526941276 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d64ba09a10 0x55d64ba03740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:58.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.575+0000 7f721e59c700 1 -- 172.21.15.31:0/3526941276 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d64b7f1270 con 0x55d64ba27e50 2023-10-06T02:30:58.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.575+0000 7f721ed9d700 1 --2- 172.21.15.31:0/3526941276 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d64ba09a10 0x55d64ba03740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:30:58.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.575+0000 7f721e59c700 1 --2- 172.21.15.31:0/3526941276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d64ba27e50 0x55d64ba02cc0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f722006f290 tx=0x7f7210021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24ce36c8a9a51882 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:58.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.575+0000 7f71ff7fe700 1 -- 172.21.15.31:0/3526941276 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f721002d070 con 0x55d64ba27e50 2023-10-06T02:30:58.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.575+0000 7f71ff7fe700 1 -- 172.21.15.31:0/3526941276 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7210000cf0 con 0x55d64ba27e50 2023-10-06T02:30:58.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.575+0000 7f71ff7fe700 1 -- 172.21.15.31:0/3526941276 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7210021070 con 0x55d64ba27e50 2023-10-06T02:30:58.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.576+0000 7f723311bb80 1 -- 172.21.15.31:0/3526941276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d64ba27e50 msgr2=0x55d64ba02cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:58.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.576+0000 7f723311bb80 1 --2- 172.21.15.31:0/3526941276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d64ba27e50 0x55d64ba02cc0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f722006f290 tx=0x7f7210021000 comp rx=0 tx=0).stop 2023-10-06T02:30:58.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.576+0000 7f723311bb80 1 -- 172.21.15.31:0/3526941276 shutdown_connections 2023-10-06T02:30:58.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.576+0000 7f723311bb80 1 --2- 172.21.15.31:0/3526941276 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d64ba09a10 0x55d64ba03740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:58.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.576+0000 7f723311bb80 1 --2- 172.21.15.31:0/3526941276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d64ba27e50 0x55d64ba02cc0 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:58.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.576+0000 7f723311bb80 1 --2- 172.21.15.31:0/3526941276 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d64ba082c0 0x55d64ba03200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:58.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.576+0000 7f723311bb80 1 -- 172.21.15.31:0/3526941276 >> 172.21.15.31:0/3526941276 conn(0x55d64ba07e80 msgr2=0x55d64ba086b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:58.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.576+0000 7f723311bb80 1 -- 172.21.15.31:0/3526941276 shutdown_connections 2023-10-06T02:30:58.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.576+0000 7f723311bb80 1 -- 172.21.15.31:0/3526941276 wait complete. 2023-10-06T02:30:58.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.577+0000 7f723311bb80 1 -- start start 2023-10-06T02:30:58.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.577+0000 7f723311bb80 1 Processor -- start 2023-10-06T02:30:58.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.578+0000 7f723311bb80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d64ba082c0 0x55d64ba511e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:58.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.578+0000 7f723311bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d64ba09a10 0x55d64ba51720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:58.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.578+0000 7f723311bb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d64ba27e50 0x55d64ba206a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:58.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.578+0000 7f723311bb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d64b902890 con 0x55d64ba27e50 2023-10-06T02:30:58.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.578+0000 7f723311bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d64b91dea0 con 0x55d64ba09a10 2023-10-06T02:30:58.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.578+0000 7f723311bb80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d64b7d9e30 con 0x55d64ba082c0 2023-10-06T02:30:58.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.578+0000 7f723311bb80 10 client.0 _ll_register_callbacks cb 0x55d64b9ca020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T02:30:58.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.578+0000 7f721ed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d64ba27e50 0x55d64ba206a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:58.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.578+0000 7f721dd9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d64ba09a10 0x55d64ba51720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:58.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.578+0000 7f721e59c700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d64ba082c0 0x55d64ba511e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:58.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.578+0000 7f721ed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d64ba27e50 0x55d64ba206a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36418/0 (socket says 172.21.15.31:36418) 2023-10-06T02:30:58.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.578+0000 7f721ed9d700 1 -- 172.21.15.31:0/1414952042 learned_addr learned my addr 172.21.15.31:0/1414952042 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:58.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.579+0000 7f721ed9d700 1 -- 172.21.15.31:0/1414952042 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d64ba082c0 msgr2=0x55d64ba511e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:58.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.579+0000 7f721ed9d700 1 --2- 172.21.15.31:0/1414952042 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d64ba082c0 0x55d64ba511e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:58.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.579+0000 7f721ed9d700 1 -- 172.21.15.31:0/1414952042 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d64ba09a10 msgr2=0x55d64ba51720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:58.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.579+0000 7f721ed9d700 1 --2- 172.21.15.31:0/1414952042 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d64ba09a10 0x55d64ba51720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:58.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.579+0000 7f721ed9d700 1 -- 172.21.15.31:0/1414952042 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f722015df60 con 0x55d64ba27e50 2023-10-06T02:30:58.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.579+0000 7f721ed9d700 1 --2- 172.21.15.31:0/1414952042 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d64ba27e50 0x55d64ba206a0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f7218052840 tx=0x7f7218056110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92f32529664f5cc6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:58.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.579+0000 7f71fcff9700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T02:30:58.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.579+0000 7f71fcff9700 1 -- 172.21.15.31:0/1414952042 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7218083890 con 0x55d64ba27e50 2023-10-06T02:30:58.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.579+0000 7f71fcff9700 1 -- 172.21.15.31:0/1414952042 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(144..144 src has 1..144) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f721805d2e0 con 0x55d64ba27e50 2023-10-06T02:30:58.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.579+0000 7f723311bb80 1 -- 172.21.15.31:0/1414952042 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d64b9fb940 con 0x55d64ba27e50 2023-10-06T02:30:58.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.579+0000 7f71fcff9700 1 -- 172.21.15.31:0/1414952042 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=100) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7218059610 con 0x55d64ba27e50 2023-10-06T02:30:58.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.579+0000 7f723311bb80 10 client.8631 fetch_fsmap learned FSMap version 100 2023-10-06T02:30:58.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.579+0000 7f723311bb80 1 -- 172.21.15.31:0/1414952042 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=100,monmap=2+}) v3 -- 0x55d64b793fe0 con 0x55d64ba27e50 2023-10-06T02:30:58.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.580+0000 7f71fcff9700 1 -- 172.21.15.31:0/1414952042 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d64b9fb940 con 0x55d64ba27e50 2023-10-06T02:30:58.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.580+0000 7f71fcff9700 1 -- 172.21.15.31:0/1414952042 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 100) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f721805ce50 con 0x55d64ba27e50 2023-10-06T02:30:58.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.580+0000 7f723311bb80 10 client.8631 fetch_fsmap finished waiting for FSMap version 100 2023-10-06T02:30:58.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.580+0000 7f723311bb80 10 client.8631 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-06T02:30:58.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.580+0000 7f723311bb80 20 client.8631 populate_metadata read hostname 'smithi031' 2023-10-06T02:30:58.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.580+0000 7f723311bb80 1 --2- 172.21.15.31:0/1414952042 >> [v2:172.21.15.62:6833/2236458886,v1:172.21.15.62:6835/2236458886] conn(0x55d64ba31240 0x55d64ba31610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:58.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.580+0000 7f723311bb80 4 client.8631 mds_command: new command op to 8487 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T02:30:58.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.580+0000 7f723311bb80 1 -- 172.21.15.31:0/1414952042 --> [v2:172.21.15.62:6833/2236458886,v1:172.21.15.62:6835/2236458886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d64b9d8870 con 0x55d64ba31240 2023-10-06T02:30:58.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.580+0000 7f721e59c700 1 --2- 172.21.15.31:0/1414952042 >> [v2:172.21.15.62:6833/2236458886,v1:172.21.15.62:6835/2236458886] conn(0x55d64ba31240 0x55d64ba31610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:58.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.581+0000 7f721e59c700 1 --2- 172.21.15.31:0/1414952042 >> [v2:172.21.15.62:6833/2236458886,v1:172.21.15.62:6835/2236458886] conn(0x55d64ba31240 0x55d64ba31610 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f68a78f41fefa18e server_cookie=4bcf64ed8a9a0983 in_seq=0 out_seq=0 2023-10-06T02:30:58.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.581+0000 7f71fcff9700 10 client.8631 ms_handle_connect on v2:172.21.15.62:6833/2236458886 2023-10-06T02:30:58.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.582+0000 7f71fcff9700 1 -- 172.21.15.31:0/1414952042 <== mds.0 v2:172.21.15.62:6833/2236458886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55d64b9d8870 con 0x55d64ba31240 2023-10-06T02:30:58.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.582+0000 7f71fcff9700 10 client.8631 handle_command_reply: tid=0 2023-10-06T02:30:58.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.582+0000 7f723311bb80 1 client.8631 shutdown 2023-10-06T02:30:58.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.582+0000 7f723311bb80 1 -- 172.21.15.31:0/1414952042 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d64ba27e50 msgr2=0x55d64ba206a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:58.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.582+0000 7f723311bb80 1 --2- 172.21.15.31:0/1414952042 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d64ba27e50 0x55d64ba206a0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f7218052840 tx=0x7f7218056110 comp rx=0 tx=0).stop 2023-10-06T02:30:58.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.582+0000 7f723311bb80 1 -- 172.21.15.31:0/1414952042 shutdown_connections 2023-10-06T02:30:58.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.582+0000 7f723311bb80 1 --2- 172.21.15.31:0/1414952042 >> [v2:172.21.15.62:6833/2236458886,v1:172.21.15.62:6835/2236458886] conn(0x55d64ba31240 0x55d64ba31610 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:58.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.582+0000 7f723311bb80 1 --2- 172.21.15.31:0/1414952042 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d64ba27e50 0x55d64ba206a0 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:58.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.582+0000 7f723311bb80 1 --2- 172.21.15.31:0/1414952042 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d64ba082c0 0x55d64ba511e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:58.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.582+0000 7f723311bb80 1 --2- 172.21.15.31:0/1414952042 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d64ba09a10 0x55d64ba51720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:58.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.582+0000 7f71fcff9700 0 client.8631 ms_handle_reset on v2:172.21.15.62:6833/2236458886 2023-10-06T02:30:58.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.582+0000 7f723311bb80 1 -- 172.21.15.31:0/1414952042 >> 172.21.15.31:0/1414952042 conn(0x55d64ba07e80 msgr2=0x55d64ba27510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:58.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.582+0000 7f723311bb80 1 -- 172.21.15.31:0/1414952042 shutdown_connections 2023-10-06T02:30:58.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.582+0000 7f723311bb80 1 -- 172.21.15.31:0/1414952042 wait complete. 2023-10-06T02:30:58.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.582+0000 7f723311bb80 20 client.8631 trim_cache size 0 max 16384 2023-10-06T02:30:58.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.600+0000 7f723311bb80 1 Processor -- start 2023-10-06T02:30:58.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.600+0000 7f723311bb80 1 -- start start 2023-10-06T02:30:58.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.601+0000 7f723311bb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d64ba082c0 0x55d64ba31720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:58.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.601+0000 7f723311bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d64ba09a10 0x55d64ba35c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:58.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.601+0000 7f723311bb80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d64ba27e50 0x55d64ba331c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:58.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.601+0000 7f723311bb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d64ba06c20 con 0x55d64ba082c0 2023-10-06T02:30:58.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.601+0000 7f723311bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d64ba20a10 con 0x55d64ba09a10 2023-10-06T02:30:58.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.601+0000 7f723311bb80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d64ba36150 con 0x55d64ba27e50 2023-10-06T02:30:58.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.601+0000 7f721e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d64ba082c0 0x55d64ba31720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:58.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.601+0000 7f721e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d64ba082c0 0x55d64ba31720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36428/0 (socket says 172.21.15.31:36428) 2023-10-06T02:30:58.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.601+0000 7f721e59c700 1 -- 172.21.15.31:0/1783109080 learned_addr learned my addr 172.21.15.31:0/1783109080 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:58.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.601+0000 7f721e59c700 1 -- 172.21.15.31:0/1783109080 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d64ba27e50 msgr2=0x55d64ba331c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:58.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.601+0000 7f721e59c700 1 --2- 172.21.15.31:0/1783109080 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d64ba27e50 0x55d64ba331c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:58.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.601+0000 7f721e59c700 1 -- 172.21.15.31:0/1783109080 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d64ba09a10 msgr2=0x55d64ba35c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:58.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.601+0000 7f721e59c700 1 --2- 172.21.15.31:0/1783109080 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d64ba09a10 0x55d64ba35c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:58.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.601+0000 7f721e59c700 1 -- 172.21.15.31:0/1783109080 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d64b925780 con 0x55d64ba082c0 2023-10-06T02:30:58.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.602+0000 7f721e59c700 1 --2- 172.21.15.31:0/1783109080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d64ba082c0 0x55d64ba31720 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f7210006fd0 tx=0x7f7210014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d077840b6b47268 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:58.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.602+0000 7f71fffff700 1 -- 172.21.15.31:0/1783109080 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f721002d040 con 0x55d64ba082c0 2023-10-06T02:30:58.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.602+0000 7f71fffff700 1 -- 172.21.15.31:0/1783109080 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f721001a950 con 0x55d64ba082c0 2023-10-06T02:30:58.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.602+0000 7f71fffff700 1 -- 172.21.15.31:0/1783109080 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7210021040 con 0x55d64ba082c0 2023-10-06T02:30:58.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.602+0000 7f723311bb80 1 -- 172.21.15.31:0/1783109080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d64ba082c0 msgr2=0x55d64ba31720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:58.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.602+0000 7f723311bb80 1 --2- 172.21.15.31:0/1783109080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d64ba082c0 0x55d64ba31720 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f7210006fd0 tx=0x7f7210014000 comp rx=0 tx=0).stop 2023-10-06T02:30:58.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.602+0000 7f723311bb80 1 -- 172.21.15.31:0/1783109080 shutdown_connections 2023-10-06T02:30:58.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.602+0000 7f723311bb80 1 --2- 172.21.15.31:0/1783109080 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d64ba27e50 0x55d64ba331c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:58.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.602+0000 7f723311bb80 1 --2- 172.21.15.31:0/1783109080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d64ba082c0 0x55d64ba31720 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:58.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.602+0000 7f723311bb80 1 --2- 172.21.15.31:0/1783109080 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d64ba09a10 0x55d64ba35c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:58.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.602+0000 7f723311bb80 1 -- 172.21.15.31:0/1783109080 >> 172.21.15.31:0/1783109080 conn(0x55d64ba07e80 msgr2=0x55d64ba04ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:58.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.602+0000 7f723311bb80 1 -- 172.21.15.31:0/1783109080 shutdown_connections 2023-10-06T02:30:58.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.602+0000 7f723311bb80 1 -- 172.21.15.31:0/1783109080 wait complete. 2023-10-06T02:30:58.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.603+0000 7f723311bb80 1 -- start start 2023-10-06T02:30:58.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.603+0000 7f723311bb80 1 Processor -- start 2023-10-06T02:30:58.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.604+0000 7f723311bb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d64ba082c0 0x55d64baa99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:58.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.604+0000 7f723311bb80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d64ba09a10 0x55d64b9fd400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:58.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.604+0000 7f723311bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d64ba27e50 0x55d64b9fd940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:58.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.604+0000 7f723311bb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d64ba50850 con 0x55d64ba082c0 2023-10-06T02:30:58.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.604+0000 7f723311bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d64ba089e0 con 0x55d64ba27e50 2023-10-06T02:30:58.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.604+0000 7f723311bb80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d64ba50040 con 0x55d64ba09a10 2023-10-06T02:30:58.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.604+0000 7f723311bb80 10 client.0 _ll_register_callbacks cb 0x55d64b9ca020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T02:30:58.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.605+0000 7f721e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d64ba082c0 0x55d64baa99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:58.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.605+0000 7f721e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d64ba082c0 0x55d64baa99b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36442/0 (socket says 172.21.15.31:36442) 2023-10-06T02:30:58.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.605+0000 7f721e59c700 1 -- 172.21.15.31:0/1268369852 learned_addr learned my addr 172.21.15.31:0/1268369852 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:58.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.605+0000 7f721dd9b700 1 --2- 172.21.15.31:0/1268369852 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d64ba09a10 0x55d64b9fd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:58.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.605+0000 7f721ed9d700 1 --2- 172.21.15.31:0/1268369852 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d64ba27e50 0x55d64b9fd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:58.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.605+0000 7f721e59c700 1 -- 172.21.15.31:0/1268369852 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d64ba09a10 msgr2=0x55d64b9fd400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:58.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.605+0000 7f721e59c700 1 --2- 172.21.15.31:0/1268369852 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d64ba09a10 0x55d64b9fd400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:58.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.605+0000 7f721e59c700 1 -- 172.21.15.31:0/1268369852 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d64ba27e50 msgr2=0x55d64b9fd940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:58.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.605+0000 7f721e59c700 1 --2- 172.21.15.31:0/1268369852 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d64ba27e50 0x55d64b9fd940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:58.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.605+0000 7f721e59c700 1 -- 172.21.15.31:0/1268369852 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d64b7f1270 con 0x55d64ba082c0 2023-10-06T02:30:58.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.605+0000 7f721ed9d700 1 --2- 172.21.15.31:0/1268369852 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d64ba27e50 0x55d64b9fd940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:30:58.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.605+0000 7f721e59c700 1 --2- 172.21.15.31:0/1268369852 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d64ba082c0 0x55d64baa99b0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f72100093a0 tx=0x7f7210041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc3d558d4c84f1ae server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:58.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.605+0000 7f71fcff9700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T02:30:58.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.605+0000 7f71fcff9700 1 -- 172.21.15.31:0/1268369852 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7210021070 con 0x55d64ba082c0 2023-10-06T02:30:58.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.605+0000 7f71fcff9700 1 -- 172.21.15.31:0/1268369852 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(144..144 src has 1..144) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f7210040890 con 0x55d64ba082c0 2023-10-06T02:30:58.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.605+0000 7f723311bb80 1 -- 172.21.15.31:0/1268369852 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7210000cf0 con 0x55d64ba082c0 2023-10-06T02:30:58.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.606+0000 7f71fcff9700 1 -- 172.21.15.31:0/1268369852 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=100) v2 ==== 24+0+0 (secure 0 0 0) 0x7f721002d040 con 0x55d64ba082c0 2023-10-06T02:30:58.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.606+0000 7f723311bb80 10 client.8637 fetch_fsmap learned FSMap version 100 2023-10-06T02:30:58.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.606+0000 7f723311bb80 1 -- 172.21.15.31:0/1268369852 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=100,monmap=2+}) v3 -- 0x55d64b90c470 con 0x55d64ba082c0 2023-10-06T02:30:58.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.606+0000 7f71fcff9700 1 -- 172.21.15.31:0/1268369852 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7210000cf0 con 0x55d64ba082c0 2023-10-06T02:30:58.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.606+0000 7f71fcff9700 1 -- 172.21.15.31:0/1268369852 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 100) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f7210040c60 con 0x55d64ba082c0 2023-10-06T02:30:58.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.606+0000 7f723311bb80 10 client.8637 fetch_fsmap finished waiting for FSMap version 100 2023-10-06T02:30:58.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.606+0000 7f723311bb80 10 client.8637 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-06T02:30:58.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.606+0000 7f723311bb80 20 client.8637 populate_metadata read hostname 'smithi031' 2023-10-06T02:30:58.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.606+0000 7f723311bb80 1 --2- 172.21.15.31:0/1268369852 >> [v2:172.21.15.62:6833/2236458886,v1:172.21.15.62:6835/2236458886] conn(0x55d64ba5c4d0 0x55d64ba5c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:58.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.606+0000 7f723311bb80 4 client.8637 mds_command: new command op to 8487 tid=0[{"prefix": "status"}] 2023-10-06T02:30:58.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.606+0000 7f723311bb80 1 -- 172.21.15.31:0/1268369852 --> [v2:172.21.15.62:6833/2236458886,v1:172.21.15.62:6835/2236458886] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d64b8f9e00 con 0x55d64ba5c4d0 2023-10-06T02:30:58.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.606+0000 7f721dd9b700 1 --2- 172.21.15.31:0/1268369852 >> [v2:172.21.15.62:6833/2236458886,v1:172.21.15.62:6835/2236458886] conn(0x55d64ba5c4d0 0x55d64ba5c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:58.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.607+0000 7f721dd9b700 1 --2- 172.21.15.31:0/1268369852 >> [v2:172.21.15.62:6833/2236458886,v1:172.21.15.62:6835/2236458886] conn(0x55d64ba5c4d0 0x55d64ba5c8a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=350baa944dab4e7 server_cookie=d8b395cc09941fbd in_seq=0 out_seq=0 2023-10-06T02:30:58.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.607+0000 7f71fcff9700 10 client.8637 ms_handle_connect on v2:172.21.15.62:6833/2236458886 2023-10-06T02:30:58.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.607+0000 7f71fcff9700 1 -- 172.21.15.31:0/1268369852 <== mds.0 v2:172.21.15.62:6833/2236458886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x55d64b8f9e00 con 0x55d64ba5c4d0 2023-10-06T02:30:58.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.607+0000 7f71fcff9700 10 client.8637 handle_command_reply: tid=0 2023-10-06T02:30:58.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.607+0000 7f723311bb80 1 client.8637 shutdown 2023-10-06T02:30:58.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.607+0000 7f723311bb80 1 -- 172.21.15.31:0/1268369852 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d64ba082c0 msgr2=0x55d64baa99b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:58.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.607+0000 7f723311bb80 1 --2- 172.21.15.31:0/1268369852 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d64ba082c0 0x55d64baa99b0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f72100093a0 tx=0x7f7210041000 comp rx=0 tx=0).stop 2023-10-06T02:30:58.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.607+0000 7f723311bb80 1 -- 172.21.15.31:0/1268369852 shutdown_connections 2023-10-06T02:30:58.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.607+0000 7f723311bb80 1 --2- 172.21.15.31:0/1268369852 >> [v2:172.21.15.62:6833/2236458886,v1:172.21.15.62:6835/2236458886] conn(0x55d64ba5c4d0 0x55d64ba5c8a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:58.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.607+0000 7f723311bb80 1 --2- 172.21.15.31:0/1268369852 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d64ba27e50 0x55d64b9fd940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:58.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.607+0000 7f723311bb80 1 --2- 172.21.15.31:0/1268369852 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d64ba082c0 0x55d64baa99b0 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:58.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.607+0000 7f71fcff9700 0 client.8637 ms_handle_reset on v2:172.21.15.62:6833/2236458886 2023-10-06T02:30:58.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.607+0000 7f723311bb80 1 --2- 172.21.15.31:0/1268369852 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d64ba09a10 0x55d64b9fd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:58.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.607+0000 7f723311bb80 1 -- 172.21.15.31:0/1268369852 >> 172.21.15.31:0/1268369852 conn(0x55d64ba07e80 msgr2=0x55d64ba31720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:58.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.608+0000 7f723311bb80 1 -- 172.21.15.31:0/1268369852 shutdown_connections 2023-10-06T02:30:58.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.608+0000 7f723311bb80 1 -- 172.21.15.31:0/1268369852 wait complete. 2023-10-06T02:30:58.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.608+0000 7f723311bb80 20 client.8637 trim_cache size 0 max 16384 2023-10-06T02:30:58.614 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T02:30:58.614 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "d6fcf512-874b-4259-b90a-30ae070f00a9", 2023-10-06T02:30:58.614 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 0, 2023-10-06T02:30:58.614 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 8487, 2023-10-06T02:30:58.615 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2023-10-06T02:30:58.615 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2023-10-06T02:30:58.615 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2023-10-06T02:30:58.615 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 6.5237217940000001, 2023-10-06T02:30:58.615 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 100, 2023-10-06T02:30:58.615 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 144, 2023-10-06T02:30:58.615 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 144, 2023-10-06T02:30:58.615 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 12.683581415000001 2023-10-06T02:30:58.615 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T02:30:58.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.608+0000 7f721f59e700 1 -- 172.21.15.31:0/2146993487 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f72000674d0 msgr2=0x7f7200069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:58.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.608+0000 7f721f59e700 1 --2- 172.21.15.31:0/2146993487 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f72000674d0 0x7f7200069980 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f722006d780 tx=0x7f7210017000 comp rx=0 tx=0).stop 2023-10-06T02:30:58.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.608+0000 7f721f59e700 1 -- 172.21.15.31:0/2146993487 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7220142c00 msgr2=0x7f722015b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:58.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.608+0000 7f721f59e700 1 --2- 172.21.15.31:0/2146993487 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7220142c00 0x7f722015b590 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f7218009ec0 tx=0x7f721800b040 comp rx=0 tx=0).stop 2023-10-06T02:30:58.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.608+0000 7f721f59e700 1 -- 172.21.15.31:0/2146993487 shutdown_connections 2023-10-06T02:30:58.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.608+0000 7f721f59e700 1 --2- 172.21.15.31:0/2146993487 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7220142c00 0x7f722015b590 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:58.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.608+0000 7f721f59e700 1 --2- 172.21.15.31:0/2146993487 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f72000674d0 0x7f7200069980 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:58.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.608+0000 7f721f59e700 1 --2- 172.21.15.31:0/2146993487 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f722006bd20 0x7f7220159190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:58.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.608+0000 7f721f59e700 1 --2- 172.21.15.31:0/2146993487 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72200c7f90 0x7f72200c61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:58.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.608+0000 7f721f59e700 1 -- 172.21.15.31:0/2146993487 >> 172.21.15.31:0/2146993487 conn(0x7f72200675a0 msgr2=0x7f7220140e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:58.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.609+0000 7f721f59e700 1 -- 172.21.15.31:0/2146993487 shutdown_connections 2023-10-06T02:30:58.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.609+0000 7f721f59e700 1 -- 172.21.15.31:0/2146993487 wait complete. 2023-10-06T02:30:58.626 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T02:30:58.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.815+0000 7fa23d99d700 1 Processor -- start 2023-10-06T02:30:58.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.815+0000 7fa23d99d700 1 -- start start 2023-10-06T02:30:58.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.816+0000 7fa23d99d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa23806e0a0 0x7fa2380c6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:58.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.816+0000 7fa23d99d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa23806f3a0 0x7fa23814fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:58.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.816+0000 7fa23d99d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa23806a490 con 0x7fa23806e470 2023-10-06T02:30:58.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.816+0000 7fa23d99d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa23806b930 con 0x7fa23806f3a0 2023-10-06T02:30:58.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.816+0000 7fa23d99d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2380c7490 con 0x7fa23806e0a0 2023-10-06T02:30:58.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.817+0000 7fa237fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa23806f3a0 0x7fa23814fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:58.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.817+0000 7fa237fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa23806f3a0 0x7fa23814fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36446/0 (socket says 172.21.15.31:36446) 2023-10-06T02:30:58.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.817+0000 7fa237fff700 1 -- 172.21.15.31:0/133324306 learned_addr learned my addr 172.21.15.31:0/133324306 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:58.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.817+0000 7fa23c99b700 1 --2- 172.21.15.31:0/133324306 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa23806e0a0 0x7fa2380c6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:58.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.817+0000 7fa237fff700 1 -- 172.21.15.31:0/133324306 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa23806e0a0 msgr2=0x7fa2380c6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:30:58.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.817+0000 7fa237fff700 1 --2- 172.21.15.31:0/133324306 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa23806e0a0 0x7fa2380c6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:58.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.817+0000 7fa237fff700 1 -- 172.21.15.31:0/133324306 >> v1:172.21.15.62:6789/0 conn(0x7fa23806e470 legacy=0x7fa238159740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:58.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.817+0000 7fa237fff700 1 -- 172.21.15.31:0/133324306 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2380c7770 con 0x7fa23806f3a0 2023-10-06T02:30:58.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.817+0000 7fa237fff700 1 --2- 172.21.15.31:0/133324306 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa23806f3a0 0x7fa23814fc70 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fa22400bc30 tx=0x7fa22400b4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=642dad2554cfaab2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:58.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.817+0000 7fa2377fe700 1 -- 172.21.15.31:0/133324306 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22400dc50 con 0x7fa23806f3a0 2023-10-06T02:30:58.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.818+0000 7fa2377fe700 1 -- 172.21.15.31:0/133324306 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa22400ddb0 con 0x7fa23806f3a0 2023-10-06T02:30:58.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.818+0000 7fa2377fe700 1 -- 172.21.15.31:0/133324306 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa224005c30 con 0x7fa23806f3a0 2023-10-06T02:30:58.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.818+0000 7fa23d99d700 1 -- 172.21.15.31:0/133324306 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa23806f3a0 msgr2=0x7fa23814fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:58.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.818+0000 7fa23d99d700 1 --2- 172.21.15.31:0/133324306 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa23806f3a0 0x7fa23814fc70 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fa22400bc30 tx=0x7fa22400b4c0 comp rx=0 tx=0).stop 2023-10-06T02:30:58.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.818+0000 7fa23d99d700 1 -- 172.21.15.31:0/133324306 shutdown_connections 2023-10-06T02:30:58.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.818+0000 7fa23d99d700 1 --2- 172.21.15.31:0/133324306 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa23806e0a0 0x7fa2380c6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:58.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.818+0000 7fa23d99d700 1 --2- 172.21.15.31:0/133324306 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa23806f3a0 0x7fa23814fc70 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:58.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.818+0000 7fa23d99d700 1 -- 172.21.15.31:0/133324306 >> 172.21.15.31:0/133324306 conn(0x7fa23806d170 msgr2=0x7fa2380cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:58.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.818+0000 7fa23d99d700 1 -- 172.21.15.31:0/133324306 shutdown_connections 2023-10-06T02:30:58.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.818+0000 7fa23d99d700 1 -- 172.21.15.31:0/133324306 wait complete. 2023-10-06T02:30:58.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.820+0000 7fa23d99d700 1 Processor -- start 2023-10-06T02:30:58.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.820+0000 7fa23d99d700 1 -- start start 2023-10-06T02:30:58.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.821+0000 7fa23d99d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa23806e0a0 0x7fa2380c2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:58.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.821+0000 7fa23d99d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa23806e470 0x7fa238157a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:58.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.821+0000 7fa23d99d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa23806f3a0 0x7fa238157fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:58.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.821+0000 7fa23d99d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2381590f0 con 0x7fa23806f3a0 2023-10-06T02:30:58.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.821+0000 7fa23d99d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2380c19d0 con 0x7fa23806e0a0 2023-10-06T02:30:58.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.821+0000 7fa23d99d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2380c1cb0 con 0x7fa23806e470 2023-10-06T02:30:58.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.821+0000 7fa23d19c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa23806f3a0 0x7fa238157fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:58.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.821+0000 7fa23d19c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa23806f3a0 0x7fa238157fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36458/0 (socket says 172.21.15.31:36458) 2023-10-06T02:30:58.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.821+0000 7fa23d19c700 1 -- 172.21.15.31:0/3035538137 learned_addr learned my addr 172.21.15.31:0/3035538137 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:30:58.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.821+0000 7fa237fff700 1 --2- 172.21.15.31:0/3035538137 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa23806e470 0x7fa238157a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:58.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.821+0000 7fa23c99b700 1 --2- 172.21.15.31:0/3035538137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa23806e0a0 0x7fa2380c2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:58.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.821+0000 7fa23d19c700 1 -- 172.21.15.31:0/3035538137 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa23806e470 msgr2=0x7fa238157a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:58.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.821+0000 7fa23d19c700 1 --2- 172.21.15.31:0/3035538137 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa23806e470 0x7fa238157a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:58.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.821+0000 7fa23d19c700 1 -- 172.21.15.31:0/3035538137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa23806e0a0 msgr2=0x7fa2380c2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:58.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.821+0000 7fa23d19c700 1 --2- 172.21.15.31:0/3035538137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa23806e0a0 0x7fa2380c2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:58.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.821+0000 7fa23d19c700 1 -- 172.21.15.31:0/3035538137 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa22400b720 con 0x7fa23806f3a0 2023-10-06T02:30:58.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.822+0000 7fa23d19c700 1 --2- 172.21.15.31:0/3035538137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa23806f3a0 0x7fa238157fd0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fa23000c8e0 tx=0x7fa230001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:58.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.822+0000 7fa234ff9700 1 -- 172.21.15.31:0/3035538137 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa230012630 con 0x7fa23806f3a0 2023-10-06T02:30:58.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.822+0000 7fa234ff9700 1 -- 172.21.15.31:0/3035538137 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2300127b0 con 0x7fa23806f3a0 2023-10-06T02:30:58.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.822+0000 7fa234ff9700 1 -- 172.21.15.31:0/3035538137 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa230012b70 con 0x7fa23806f3a0 2023-10-06T02:30:58.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.822+0000 7fa23d99d700 1 -- 172.21.15.31:0/3035538137 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2380c1f90 con 0x7fa23806f3a0 2023-10-06T02:30:58.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.822+0000 7fa23d99d700 1 -- 172.21.15.31:0/3035538137 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2380c2550 con 0x7fa23806f3a0 2023-10-06T02:30:58.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.823+0000 7fa23d99d700 1 -- 172.21.15.31:0/3035538137 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2380c2740 con 0x7fa23806f3a0 2023-10-06T02:30:58.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.824+0000 7fa234ff9700 1 -- 172.21.15.31:0/3035538137 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7fa230003030 con 0x7fa23806f3a0 2023-10-06T02:30:58.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.825+0000 7fa234ff9700 1 --2- 172.21.15.31:0/3035538137 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa21c0674d0 0x7fa21c069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:30:58.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.825+0000 7fa234ff9700 1 -- 172.21.15.31:0/3035538137 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fa230085000 con 0x7fa23806f3a0 2023-10-06T02:30:58.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.825+0000 7fa23c99b700 1 --2- 172.21.15.31:0/3035538137 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa21c0674d0 0x7fa21c069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:30:58.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.826+0000 7fa23c99b700 1 --2- 172.21.15.31:0/3035538137 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa21c0674d0 0x7fa21c069980 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fa22c006fd0 tx=0x7fa22c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:30:58.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:58.829+0000 7fa234ff9700 1 -- 172.21.15.31:0/3035538137 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa230051840 con 0x7fa23806f3a0 2023-10-06T02:30:59.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:59.078+0000 7fa23d99d700 1 -- 172.21.15.31:0/3035538137 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa23804c3c0 con 0x7fa23806f3a0 2023-10-06T02:30:59.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:59.078+0000 7fa234ff9700 1 -- 172.21.15.31:0/3035538137 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v100) v1 ==== 53+0+73 (secure 0 0 0) 0x7fa230054e60 con 0x7fa23806f3a0 2023-10-06T02:30:59.079 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T02:30:59.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:59.081+0000 7fa23d99d700 1 -- 172.21.15.31:0/3035538137 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa21c0674d0 msgr2=0x7fa21c069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:59.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:59.081+0000 7fa23d99d700 1 --2- 172.21.15.31:0/3035538137 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa21c0674d0 0x7fa21c069980 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fa22c006fd0 tx=0x7fa22c008040 comp rx=0 tx=0).stop 2023-10-06T02:30:59.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:59.081+0000 7fa23d99d700 1 -- 172.21.15.31:0/3035538137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa23806f3a0 msgr2=0x7fa238157fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:30:59.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:59.081+0000 7fa23d99d700 1 --2- 172.21.15.31:0/3035538137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa23806f3a0 0x7fa238157fd0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fa23000c8e0 tx=0x7fa230001710 comp rx=0 tx=0).stop 2023-10-06T02:30:59.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:59.081+0000 7fa23d99d700 1 -- 172.21.15.31:0/3035538137 shutdown_connections 2023-10-06T02:30:59.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:59.081+0000 7fa23d99d700 1 --2- 172.21.15.31:0/3035538137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa23806f3a0 0x7fa238157fd0 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:59.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:59.081+0000 7fa23d99d700 1 --2- 172.21.15.31:0/3035538137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa23806e0a0 0x7fa2380c2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:59.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:59.081+0000 7fa23d99d700 1 --2- 172.21.15.31:0/3035538137 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa21c0674d0 0x7fa21c069980 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:59.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:59.081+0000 7fa23d99d700 1 --2- 172.21.15.31:0/3035538137 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa23806e470 0x7fa238157a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:30:59.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:59.081+0000 7fa23d99d700 1 -- 172.21.15.31:0/3035538137 >> 172.21.15.31:0/3035538137 conn(0x7fa23806d170 msgr2=0x7fa23814b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:30:59.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:59.081+0000 7fa23d99d700 1 -- 172.21.15.31:0/3035538137 shutdown_connections 2023-10-06T02:30:59.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:30:59.081+0000 7fa23d99d700 1 -- 172.21.15.31:0/3035538137 wait complete. 2023-10-06T02:30:59.099 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T02:30:59.099 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T02:30:59.099 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi031.front.sepia.ceph.com 2023-10-06T02:30:59.099 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T02:30:59.099 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T02:30:59.099 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T02:30:59.099 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T02:30:59.099 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T02:30:59.099 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T02:30:59.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:30:59.100 DEBUG:teuthology.orchestra.run.smithi031:> ip addr 2023-10-06T02:30:59.137 INFO:teuthology.orchestra.run.smithi031.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T02:30:59.137 INFO:teuthology.orchestra.run.smithi031.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T02:30:59.137 INFO:teuthology.orchestra.run.smithi031.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T02:30:59.137 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:30:59.137 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 ::1/128 scope host 2023-10-06T02:30:59.138 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:30:59.138 INFO:teuthology.orchestra.run.smithi031.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T02:30:59.138 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cc brd ff:ff:ff:ff:ff:ff 2023-10-06T02:30:59.138 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f0 2023-10-06T02:30:59.138 INFO:teuthology.orchestra.run.smithi031.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T02:30:59.138 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cd brd ff:ff:ff:ff:ff:ff 2023-10-06T02:30:59.138 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f1 2023-10-06T02:30:59.138 INFO:teuthology.orchestra.run.smithi031.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T02:30:59.138 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:ba brd ff:ff:ff:ff:ff:ff 2023-10-06T02:30:59.138 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f0 2023-10-06T02:30:59.138 INFO:teuthology.orchestra.run.smithi031.stdout: inet 172.21.15.31/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T02:30:59.138 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft 41797sec preferred_lft 41797sec 2023-10-06T02:30:59.139 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::ae1f:6bff:fea5:caba/64 scope link 2023-10-06T02:30:59.139 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:30:59.139 INFO:teuthology.orchestra.run.smithi031.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T02:30:59.139 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:bb brd ff:ff:ff:ff:ff:ff 2023-10-06T02:30:59.139 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f1 2023-10-06T02:30:59.139 INFO:teuthology.orchestra.run.smithi031.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T02:30:59.139 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 16:d7:88:2d:d0:c2 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:30:59.139 INFO:teuthology.orchestra.run.smithi031.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T02:30:59.139 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:30:59.140 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::408:52ff:fe3c:ba8f/64 scope link 2023-10-06T02:30:59.140 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:30:59.140 INFO:teuthology.orchestra.run.smithi031.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T02:30:59.140 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 16:d7:88:2d:d0:c2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T02:30:59.140 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::14d7:88ff:fe2d:d0c2/64 scope link 2023-10-06T02:30:59.140 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:30:59.140 INFO:teuthology.orchestra.run.smithi031.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T02:30:59.140 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 82:16:f1:00:cf:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T02:30:59.140 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::8016:f1ff:fe00:cf2f/64 scope link 2023-10-06T02:30:59.140 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:30:59.140 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:30:59.141 DEBUG:teuthology.orchestra.run.smithi031:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T02:30:59.238 INFO:teuthology.orchestra.run.smithi031.stdout:1 2023-10-06T02:30:59.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:30:59.241 DEBUG:teuthology.orchestra.run.smithi031:> route 2023-10-06T02:30:59.280 INFO:teuthology.orchestra.run.smithi031.stdout:Kernel IP routing table 2023-10-06T02:30:59.280 INFO:teuthology.orchestra.run.smithi031.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T02:30:59.280 INFO:teuthology.orchestra.run.smithi031.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T02:30:59.280 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T02:30:59.281 INFO:teuthology.orchestra.run.smithi031.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T02:30:59.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:30:59.282 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2023-10-06T02:30:59.282 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2023-10-06T02:30:59.282 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T02:30:59.283 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T02:30:59.283 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T02:30:59.283 DEBUG:teuthology.orchestra.run.smithi031:> ') 2023-10-06T02:30:59.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:30:59.482 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2023-10-06T02:30:59.519 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T02:30:59.519 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T02:30:59.519 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T02:30:59.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:30:59.520 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set brx.0 up 2023-10-06T02:30:59.621 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T02:30:59.621 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T02:30:59.658 INFO:teuthology.orchestra.run.smithi031.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T02:30:59.658 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T02:30:59.658 DEBUG:teuthology.orchestra.run.smithi031:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T02:30:59.734 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-06T02:30:59.734 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T02:30:59.735 DEBUG:teuthology.orchestra.run.smithi031:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T02:30:59.866 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T02:30:59.867 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T02:30:59.894 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T02:30:59.894 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T02:30:59.895 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.62:3300,172.21.15.62:3301" 2023-10-06T02:30:59.895 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: trying mount with new device syntax: 0@d6fcf512-874b-4259-b90a-30ae070f00a9.cephfs=/ 2023-10-06T02:30:59.895 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.31:3300/172.21.15.62:3300/172.21.15.62:3301" will pass to kernel 2023-10-06T02:30:59.901 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T02:30:59.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:30:59.902 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T02:30:59.964 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T02:31:00.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:00.152+0000 7ffa3ffff700 1 Processor -- start 2023-10-06T02:31:00.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:00.152+0000 7ffa3ffff700 1 -- start start 2023-10-06T02:31:00.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:00.153+0000 7ffa3ffff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ffa400accb0 0x7ffa400aaed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:00.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:00.153+0000 7ffa3ffff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa400ad780 con 0x7ffa400cbea0 2023-10-06T02:31:00.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:00.153+0000 7ffa3ffff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa400cee60 con 0x7ffa400cc340 2023-10-06T02:31:00.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:00.153+0000 7ffa3ffff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa400cf140 con 0x7ffa400accb0 2023-10-06T02:31:00.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:00.153+0000 7ffa3effd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ffa400accb0 0x7ffa400aaed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:00.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:00.153+0000 7ffa3e7fc700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7ffa400cbea0 0x7ffa4006e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:36942/0 (socket says 172.21.15.31:36942) 2023-10-06T02:31:00.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:00.153+0000 7ffa3e7fc700 1 -- 172.21.15.31:0/3045656115 learned_addr learned my addr 172.21.15.31:0/3045656115 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:31:00.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:00.154+0000 7ffa3effd700 1 -- 172.21.15.31:0/3045656115 >> v1:172.21.15.62:6789/0 conn(0x7ffa400cc340 legacy=0x7ffa4006ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:00.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:00.154+0000 7ffa3effd700 1 -- 172.21.15.31:0/3045656115 >> v1:172.21.15.31:6789/0 conn(0x7ffa400cbea0 legacy=0x7ffa4006e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:00.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:00.154+0000 7ffa3effd700 1 -- 172.21.15.31:0/3045656115 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa4006d850 con 0x7ffa400accb0 2023-10-06T02:31:00.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:00.154+0000 7ffa3dffb700 1 -- 172.21.15.31:0/3045656115 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 237455973 0 0) 0x7ffa400ad780 con 0x7ffa400cbea0 2023-10-06T02:31:00.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:00.154+0000 7ffa3effd700 1 --2- 172.21.15.31:0/3045656115 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa400accb0 0x7ffa400aaed0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7ffa300098e0 tx=0x7ffa300091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=945c55f522af9b8b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:00.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:00.154+0000 7ffa3dffb700 1 -- 172.21.15.31:0/3045656115 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa30009e80 con 0x7ffa400accb0 2023-10-06T02:31:00.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:00.154+0000 7ffa3dffb700 1 -- 172.21.15.31:0/3045656115 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa3000b860 con 0x7ffa400accb0 2023-10-06T02:31:00.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:00.154+0000 7ffa3dffb700 1 -- 172.21.15.31:0/3045656115 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa3000bbe0 con 0x7ffa400accb0 2023-10-06T02:31:00.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:00.155+0000 7ffa3ffff700 1 -- 172.21.15.31:0/3045656115 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa400accb0 msgr2=0x7ffa400aaed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:00.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:00.155+0000 7ffa3ffff700 1 --2- 172.21.15.31:0/3045656115 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa400accb0 0x7ffa400aaed0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7ffa300098e0 tx=0x7ffa300091a0 comp rx=0 tx=0).stop 2023-10-06T02:31:00.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:00.155+0000 7ffa3ffff700 1 -- 172.21.15.31:0/3045656115 shutdown_connections 2023-10-06T02:31:00.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:00.155+0000 7ffa3ffff700 1 --2- 172.21.15.31:0/3045656115 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa400accb0 0x7ffa400aaed0 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:00.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:00.155+0000 7ffa3ffff700 1 -- 172.21.15.31:0/3045656115 >> 172.21.15.31:0/3045656115 conn(0x7ffa4006ce80 msgr2=0x7ffa400ce370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:00.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:00.155+0000 7ffa3ffff700 1 -- 172.21.15.31:0/3045656115 shutdown_connections 2023-10-06T02:31:00.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:00.155+0000 7ffa3ffff700 1 -- 172.21.15.31:0/3045656115 wait complete. 2023-10-06T02:31:00.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:00.156+0000 7ffa3ffff700 1 Processor -- start 2023-10-06T02:31:00.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:00.157+0000 7ffa3ffff700 1 -- start start 2023-10-06T02:31:00.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:00.157+0000 7ffa3ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa400accb0 0x7ffa4016aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:00.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:00.157+0000 7ffa3ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa400cbea0 0x7ffa401579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:00.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:00.157+0000 7ffa3ffff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffa400cc340 0x7ffa400b2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:00.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:00.157+0000 7ffa3ffff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa40157ee0 con 0x7ffa400accb0 2023-10-06T02:31:00.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:00.157+0000 7ffa3ffff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4016a5b0 con 0x7ffa400cbea0 2023-10-06T02:31:00.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:00.157+0000 7ffa3ffff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa400b3960 con 0x7ffa400cc340 2023-10-06T02:31:00.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:00.158+0000 7ffa3effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa400accb0 0x7ffa4016aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:00.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:00.158+0000 7ffa3effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa400accb0 0x7ffa4016aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36480/0 (socket says 172.21.15.31:36480) 2023-10-06T02:31:00.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:00.158+0000 7ffa3effd700 1 -- 172.21.15.31:0/3823832838 learned_addr learned my addr 172.21.15.31:0/3823832838 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:00.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:00.158+0000 7ffa3e7fc700 1 --2- 172.21.15.31:0/3823832838 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa400cbea0 0x7ffa401579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:00.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:00.158+0000 7ffa3effd700 1 -- 172.21.15.31:0/3823832838 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffa400cc340 msgr2=0x7ffa400b2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:00.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:00.158+0000 7ffa3f7fe700 1 --2- 172.21.15.31:0/3823832838 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffa400cc340 0x7ffa400b2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:00.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:00.158+0000 7ffa3effd700 1 --2- 172.21.15.31:0/3823832838 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffa400cc340 0x7ffa400b2ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:00.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:00.158+0000 7ffa3effd700 1 -- 172.21.15.31:0/3823832838 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa400cbea0 msgr2=0x7ffa401579a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:00.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:00.158+0000 7ffa3effd700 1 --2- 172.21.15.31:0/3823832838 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa400cbea0 0x7ffa401579a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:00.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:00.158+0000 7ffa3effd700 1 -- 172.21.15.31:0/3823832838 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa30009420 con 0x7ffa400accb0 2023-10-06T02:31:00.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:00.158+0000 7ffa3e7fc700 1 --2- 172.21.15.31:0/3823832838 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa400cbea0 0x7ffa401579a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:31:00.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:00.158+0000 7ffa3f7fe700 1 --2- 172.21.15.31:0/3823832838 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffa400cc340 0x7ffa400b2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:31:00.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:00.158+0000 7ffa3effd700 1 --2- 172.21.15.31:0/3823832838 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa400accb0 0x7ffa4016aa40 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7ffa3000b3c0 tx=0x7ffa30017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:00.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:00.159+0000 7ffa26ffd700 1 -- 172.21.15.31:0/3823832838 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa30011420 con 0x7ffa400accb0 2023-10-06T02:31:00.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:00.159+0000 7ffa26ffd700 1 -- 172.21.15.31:0/3823832838 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa30011580 con 0x7ffa400accb0 2023-10-06T02:31:00.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:00.159+0000 7ffa3ffff700 1 -- 172.21.15.31:0/3823832838 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa400b3c40 con 0x7ffa400accb0 2023-10-06T02:31:00.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:00.159+0000 7ffa26ffd700 1 -- 172.21.15.31:0/3823832838 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa30011920 con 0x7ffa400accb0 2023-10-06T02:31:00.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:00.159+0000 7ffa3ffff700 1 -- 172.21.15.31:0/3823832838 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa4016a240 con 0x7ffa400accb0 2023-10-06T02:31:00.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:00.160+0000 7ffa3ffff700 1 -- 172.21.15.31:0/3823832838 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa4004c3c0 con 0x7ffa400accb0 2023-10-06T02:31:00.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:00.161+0000 7ffa26ffd700 1 -- 172.21.15.31:0/3823832838 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7ffa30011c90 con 0x7ffa400accb0 2023-10-06T02:31:00.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:00.162+0000 7ffa26ffd700 1 --2- 172.21.15.31:0/3823832838 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ffa2c0674d0 0x7ffa2c069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:00.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:00.162+0000 7ffa26ffd700 1 -- 172.21.15.31:0/3823832838 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6086+0+0 (secure 0 0 0) 0x7ffa30086e10 con 0x7ffa400accb0 2023-10-06T02:31:00.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:00.162+0000 7ffa3e7fc700 1 --2- 172.21.15.31:0/3823832838 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ffa2c0674d0 0x7ffa2c069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:00.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:00.163+0000 7ffa3e7fc700 1 --2- 172.21.15.31:0/3823832838 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ffa2c0674d0 0x7ffa2c069980 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ffa400caad0 tx=0x7ffa34009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:00.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:00.165+0000 7ffa26ffd700 1 -- 172.21.15.31:0/3823832838 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffa30053750 con 0x7ffa400accb0 2023-10-06T02:31:00.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:00.412+0000 7ffa3ffff700 1 -- 172.21.15.31:0/3823832838 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ffa4006d4b0 con 0x7ffa400accb0 2023-10-06T02:31:00.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:00.412+0000 7ffa26ffd700 1 -- 172.21.15.31:0/3823832838 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v100) v1 ==== 53+0+73 (secure 0 0 0) 0x7ffa30015020 con 0x7ffa400accb0 2023-10-06T02:31:00.414 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T02:31:00.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:00.414+0000 7ffa3ffff700 1 -- 172.21.15.31:0/3823832838 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ffa2c0674d0 msgr2=0x7ffa2c069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:00.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:00.414+0000 7ffa3ffff700 1 --2- 172.21.15.31:0/3823832838 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ffa2c0674d0 0x7ffa2c069980 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ffa400caad0 tx=0x7ffa34009040 comp rx=0 tx=0).stop 2023-10-06T02:31:00.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:00.414+0000 7ffa3ffff700 1 -- 172.21.15.31:0/3823832838 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa400accb0 msgr2=0x7ffa4016aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:00.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:00.414+0000 7ffa3ffff700 1 --2- 172.21.15.31:0/3823832838 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa400accb0 0x7ffa4016aa40 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7ffa3000b3c0 tx=0x7ffa30017000 comp rx=0 tx=0).stop 2023-10-06T02:31:00.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:00.415+0000 7ffa3ffff700 1 -- 172.21.15.31:0/3823832838 shutdown_connections 2023-10-06T02:31:00.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:00.415+0000 7ffa3ffff700 1 --2- 172.21.15.31:0/3823832838 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ffa2c0674d0 0x7ffa2c069980 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:00.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:00.415+0000 7ffa3ffff700 1 --2- 172.21.15.31:0/3823832838 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffa400cc340 0x7ffa400b2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:00.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:00.415+0000 7ffa3ffff700 1 --2- 172.21.15.31:0/3823832838 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa400accb0 0x7ffa4016aa40 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:00.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:00.415+0000 7ffa3ffff700 1 --2- 172.21.15.31:0/3823832838 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa400cbea0 0x7ffa401579a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:00.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:00.415+0000 7ffa3ffff700 1 -- 172.21.15.31:0/3823832838 >> 172.21.15.31:0/3823832838 conn(0x7ffa4006ce80 msgr2=0x7ffa401596e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:00.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:00.415+0000 7ffa3ffff700 1 -- 172.21.15.31:0/3823832838 shutdown_connections 2023-10-06T02:31:00.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:00.415+0000 7ffa3ffff700 1 -- 172.21.15.31:0/3823832838 wait complete. 2023-10-06T02:31:00.432 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T02:31:00.432 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T02:31:00.432 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi062.front.sepia.ceph.com 2023-10-06T02:31:00.432 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-06T02:31:00.432 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-06T02:31:00.433 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T02:31:00.433 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T02:31:00.433 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T02:31:00.433 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T02:31:00.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:31:00.433 DEBUG:teuthology.orchestra.run.smithi062:> ip addr 2023-10-06T02:31:00.476 INFO:teuthology.orchestra.run.smithi062.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T02:31:00.476 INFO:teuthology.orchestra.run.smithi062.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T02:31:00.476 INFO:teuthology.orchestra.run.smithi062.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T02:31:00.476 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:31:00.476 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 ::1/128 scope host 2023-10-06T02:31:00.477 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:31:00.477 INFO:teuthology.orchestra.run.smithi062.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T02:31:00.477 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:14 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:31:00.477 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f0 2023-10-06T02:31:00.477 INFO:teuthology.orchestra.run.smithi062.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T02:31:00.477 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:15 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:31:00.477 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f1 2023-10-06T02:31:00.478 INFO:teuthology.orchestra.run.smithi062.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T02:31:00.478 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c2 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:31:00.478 INFO:teuthology.orchestra.run.smithi062.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T02:31:00.478 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c3 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:31:00.478 INFO:teuthology.orchestra.run.smithi062.stdout: inet 172.21.15.62/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T02:31:00.478 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft 41789sec preferred_lft 41789sec 2023-10-06T02:31:00.478 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::ec4:7aff:febb:d1c3/64 scope link 2023-10-06T02:31:00.478 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:31:00.479 INFO:teuthology.orchestra.run.smithi062.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T02:31:00.479 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether ae:42:a8:dd:f7:9b brd ff:ff:ff:ff:ff:ff 2023-10-06T02:31:00.479 INFO:teuthology.orchestra.run.smithi062.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T02:31:00.479 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:31:00.479 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::1098:a4ff:fe0d:389/64 scope link 2023-10-06T02:31:00.479 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:31:00.479 INFO:teuthology.orchestra.run.smithi062.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T02:31:00.479 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether fe:60:2b:8e:af:28 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T02:31:00.480 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::fc60:2bff:fe8e:af28/64 scope link 2023-10-06T02:31:00.480 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:31:00.480 INFO:teuthology.orchestra.run.smithi062.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T02:31:00.480 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether ae:42:a8:dd:f7:9b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T02:31:00.480 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::ac42:a8ff:fedd:f79b/64 scope link 2023-10-06T02:31:00.480 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:31:00.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:31:00.481 DEBUG:teuthology.orchestra.run.smithi062:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T02:31:00.581 INFO:teuthology.orchestra.run.smithi062.stdout:1 2023-10-06T02:31:00.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:31:00.584 DEBUG:teuthology.orchestra.run.smithi062:> route 2023-10-06T02:31:00.624 INFO:teuthology.orchestra.run.smithi062.stdout:Kernel IP routing table 2023-10-06T02:31:00.624 INFO:teuthology.orchestra.run.smithi062.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T02:31:00.624 INFO:teuthology.orchestra.run.smithi062.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T02:31:00.624 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T02:31:00.625 INFO:teuthology.orchestra.run.smithi062.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T02:31:00.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:31:00.625 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2023-10-06T02:31:00.625 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2023-10-06T02:31:00.625 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T02:31:00.625 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T02:31:00.625 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T02:31:00.625 DEBUG:teuthology.orchestra.run.smithi062:> ') 2023-10-06T02:31:00.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:31:00.824 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2023-10-06T02:31:00.860 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-06T02:31:00.860 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T02:31:00.861 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T02:31:00.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:31:00.861 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set brx.0 up 2023-10-06T02:31:00.963 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T02:31:00.964 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-06T02:31:01.000 INFO:teuthology.orchestra.run.smithi062.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-06T02:31:01.000 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T02:31:01.001 DEBUG:teuthology.orchestra.run.smithi062:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-06T02:31:01.076 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-06T02:31:01.076 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T02:31:01.076 DEBUG:teuthology.orchestra.run.smithi062:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T02:31:01.210 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T02:31:01.210 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T02:31:01.238 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T02:31:01.238 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T02:31:01.238 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.62:3300,172.21.15.62:3301" 2023-10-06T02:31:01.238 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: trying mount with new device syntax: 1@d6fcf512-874b-4259-b90a-30ae070f00a9.cephfs=/ 2023-10-06T02:31:01.238 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.31:3300/172.21.15.62:3300/172.21.15.62:3301" will pass to kernel 2023-10-06T02:31:01.243 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T02:31:01.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:31:01.243 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-06T02:31:01.305 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-10-06T02:31:01.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:01.505+0000 7fe31759e700 1 Processor -- start 2023-10-06T02:31:01.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:01.506+0000 7fe31759e700 1 -- start start 2023-10-06T02:31:01.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:01.507+0000 7fe31759e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe3180c2a90 0x7fe318156aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:01.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:01.507+0000 7fe31759e700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe3180c85d0 0x7fe3180cc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:01.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:01.507+0000 7fe31759e700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3181579b0 con 0x7fe3180c8200 2023-10-06T02:31:01.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:01.507+0000 7fe31759e700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3180c1390 con 0x7fe3180c85d0 2023-10-06T02:31:01.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:01.507+0000 7fe31759e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3181573f0 con 0x7fe3180c2a90 2023-10-06T02:31:01.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:01.507+0000 7fe31659c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fe3180c8200 0x7fe3180c1820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:36946/0 (socket says 172.21.15.31:36946) 2023-10-06T02:31:01.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:01.507+0000 7fe31659c700 1 -- 172.21.15.31:0/3241899126 learned_addr learned my addr 172.21.15.31:0/3241899126 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:31:01.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:01.507+0000 7fe315d9b700 1 --2- 172.21.15.31:0/3241899126 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe3180c2a90 0x7fe318156aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:01.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:01.507+0000 7fe316d9d700 1 --2- 172.21.15.31:0/3241899126 >> v2:172.21.15.62:3300/0 conn(0x7fe3180c85d0 0x7fe3180cc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:01.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:01.508+0000 7fe31559a700 1 -- 172.21.15.31:0/3241899126 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4102879494 0 0) 0x7fe3181579b0 con 0x7fe3180c8200 2023-10-06T02:31:01.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:01.508+0000 7fe31559a700 1 -- 172.21.15.31:0/3241899126 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe300003570 con 0x7fe3180c8200 2023-10-06T02:31:01.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:01.508+0000 7fe316d9d700 1 -- 172.21.15.31:0/3241899126 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe3180c2a90 msgr2=0x7fe318156aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:01.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:01.508+0000 7fe316d9d700 1 --2- 172.21.15.31:0/3241899126 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe3180c2a90 0x7fe318156aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:01.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:01.508+0000 7fe316d9d700 1 -- 172.21.15.31:0/3241899126 >> v1:172.21.15.31:6789/0 conn(0x7fe3180c8200 legacy=0x7fe3180c1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:01.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:01.508+0000 7fe316d9d700 1 -- 172.21.15.31:0/3241899126 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3181576d0 con 0x7fe3180c85d0 2023-10-06T02:31:01.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:01.508+0000 7fe316d9d700 1 --2- 172.21.15.31:0/3241899126 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3180c85d0 0x7fe3180cc2f0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fe310009e60 tx=0x7fe310009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0e4749f29ba694f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:01.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:01.508+0000 7fe31559a700 1 -- 172.21.15.31:0/3241899126 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe31000e070 con 0x7fe3180c85d0 2023-10-06T02:31:01.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:01.509+0000 7fe31559a700 1 -- 172.21.15.31:0/3241899126 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe31000bab0 con 0x7fe3180c85d0 2023-10-06T02:31:01.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:01.509+0000 7fe31559a700 1 -- 172.21.15.31:0/3241899126 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe31000be70 con 0x7fe3180c85d0 2023-10-06T02:31:01.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:01.509+0000 7fe31759e700 1 -- 172.21.15.31:0/3241899126 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3180c85d0 msgr2=0x7fe3180cc2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:01.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:01.509+0000 7fe31759e700 1 --2- 172.21.15.31:0/3241899126 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3180c85d0 0x7fe3180cc2f0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fe310009e60 tx=0x7fe310009780 comp rx=0 tx=0).stop 2023-10-06T02:31:01.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:01.509+0000 7fe31759e700 1 -- 172.21.15.31:0/3241899126 shutdown_connections 2023-10-06T02:31:01.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:01.509+0000 7fe31759e700 1 --2- 172.21.15.31:0/3241899126 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3180c85d0 0x7fe3180cc2f0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:01.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:01.509+0000 7fe31759e700 1 --2- 172.21.15.31:0/3241899126 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe3180c2a90 0x7fe318156aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:01.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:01.509+0000 7fe31759e700 1 -- 172.21.15.31:0/3241899126 >> 172.21.15.31:0/3241899126 conn(0x7fe31806c6a0 msgr2=0x7fe31806caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:01.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:01.509+0000 7fe31759e700 1 -- 172.21.15.31:0/3241899126 shutdown_connections 2023-10-06T02:31:01.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:01.509+0000 7fe31759e700 1 -- 172.21.15.31:0/3241899126 wait complete. 2023-10-06T02:31:01.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:01.511+0000 7fe31759e700 1 Processor -- start 2023-10-06T02:31:01.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:01.511+0000 7fe31759e700 1 -- start start 2023-10-06T02:31:01.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:01.512+0000 7fe31759e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe3180c2a90 0x7fe31815a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:01.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:01.512+0000 7fe31759e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3180c8200 0x7fe3180c3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:01.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:01.512+0000 7fe31759e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3180c85d0 0x7fe31814f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:01.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:01.512+0000 7fe31759e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3180c5570 con 0x7fe3180c85d0 2023-10-06T02:31:01.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:01.512+0000 7fe31759e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3180c5700 con 0x7fe3180c8200 2023-10-06T02:31:01.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:01.512+0000 7fe31759e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3180c4100 con 0x7fe3180c2a90 2023-10-06T02:31:01.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:01.512+0000 7fe316d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3180c85d0 0x7fe31814f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:01.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:01.512+0000 7fe316d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3180c85d0 0x7fe31814f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36488/0 (socket says 172.21.15.31:36488) 2023-10-06T02:31:01.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:01.512+0000 7fe316d9d700 1 -- 172.21.15.31:0/2868585829 learned_addr learned my addr 172.21.15.31:0/2868585829 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:01.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:01.512+0000 7fe31659c700 1 --2- 172.21.15.31:0/2868585829 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe3180c2a90 0x7fe31815a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:01.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:01.512+0000 7fe315d9b700 1 --2- 172.21.15.31:0/2868585829 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3180c8200 0x7fe3180c3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:01.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:01.512+0000 7fe316d9d700 1 -- 172.21.15.31:0/2868585829 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe3180c2a90 msgr2=0x7fe31815a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:01.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:01.512+0000 7fe316d9d700 1 --2- 172.21.15.31:0/2868585829 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe3180c2a90 0x7fe31815a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:01.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:01.512+0000 7fe316d9d700 1 -- 172.21.15.31:0/2868585829 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3180c8200 msgr2=0x7fe3180c3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:01.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:01.512+0000 7fe316d9d700 1 --2- 172.21.15.31:0/2868585829 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3180c8200 0x7fe3180c3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:01.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:01.512+0000 7fe316d9d700 1 -- 172.21.15.31:0/2868585829 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe310009a20 con 0x7fe3180c85d0 2023-10-06T02:31:01.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:01.513+0000 7fe316d9d700 1 --2- 172.21.15.31:0/2868585829 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3180c85d0 0x7fe31814f620 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fe3180697c0 tx=0x7fe310019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:01.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:01.513+0000 7fe3067fc700 1 -- 172.21.15.31:0/2868585829 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe310027470 con 0x7fe3180c85d0 2023-10-06T02:31:01.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:01.513+0000 7fe3067fc700 1 -- 172.21.15.31:0/2868585829 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3100275d0 con 0x7fe3180c85d0 2023-10-06T02:31:01.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:01.513+0000 7fe31759e700 1 -- 172.21.15.31:0/2868585829 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe31815a720 con 0x7fe3180c85d0 2023-10-06T02:31:01.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:01.513+0000 7fe3067fc700 1 -- 172.21.15.31:0/2868585829 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe310027990 con 0x7fe3180c85d0 2023-10-06T02:31:01.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:01.513+0000 7fe31759e700 1 -- 172.21.15.31:0/2868585829 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe31815ace0 con 0x7fe3180c85d0 2023-10-06T02:31:01.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:01.514+0000 7fe31759e700 1 -- 172.21.15.31:0/2868585829 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3180c5870 con 0x7fe3180c85d0 2023-10-06T02:31:01.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:01.515+0000 7fe3067fc700 1 -- 172.21.15.31:0/2868585829 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7fe310024070 con 0x7fe3180c85d0 2023-10-06T02:31:01.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:01.516+0000 7fe3067fc700 1 --2- 172.21.15.31:0/2868585829 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe2f8067500 0x7fe2f80699b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:01.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:01.516+0000 7fe3067fc700 1 -- 172.21.15.31:0/2868585829 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fe310087b10 con 0x7fe3180c85d0 2023-10-06T02:31:01.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:01.516+0000 7fe31659c700 1 --2- 172.21.15.31:0/2868585829 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe2f8067500 0x7fe2f80699b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:01.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:01.517+0000 7fe31659c700 1 --2- 172.21.15.31:0/2868585829 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe2f8067500 0x7fe2f80699b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe3080058d0 tx=0x7fe308009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:01.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:01.520+0000 7fe3067fc700 1 -- 172.21.15.31:0/2868585829 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe3180c5870 con 0x7fe3180c85d0 2023-10-06T02:31:01.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:01.766+0000 7fe31759e700 1 -- 172.21.15.31:0/2868585829 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe3180c5870 con 0x7fe3180c85d0 2023-10-06T02:31:02.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.096+0000 7fe3067fc700 1 -- 172.21.15.31:0/2868585829 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v101) v1 ==== 136+0+0 (secure 0 0 0) 0x7fe3180c5870 con 0x7fe3180c85d0 2023-10-06T02:31:02.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.098+0000 7fe31759e700 1 -- 172.21.15.31:0/2868585829 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe2f8067500 msgr2=0x7fe2f80699b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:02.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.098+0000 7fe31759e700 1 --2- 172.21.15.31:0/2868585829 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe2f8067500 0x7fe2f80699b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe3080058d0 tx=0x7fe308009040 comp rx=0 tx=0).stop 2023-10-06T02:31:02.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.098+0000 7fe31759e700 1 -- 172.21.15.31:0/2868585829 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3180c85d0 msgr2=0x7fe31814f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:02.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.098+0000 7fe31759e700 1 --2- 172.21.15.31:0/2868585829 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3180c85d0 0x7fe31814f620 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fe3180697c0 tx=0x7fe310019000 comp rx=0 tx=0).stop 2023-10-06T02:31:02.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.098+0000 7fe31759e700 1 -- 172.21.15.31:0/2868585829 shutdown_connections 2023-10-06T02:31:02.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.098+0000 7fe31759e700 1 --2- 172.21.15.31:0/2868585829 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3180c85d0 0x7fe31814f620 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:02.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.098+0000 7fe31759e700 1 --2- 172.21.15.31:0/2868585829 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe2f8067500 0x7fe2f80699b0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:02.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.098+0000 7fe31759e700 1 --2- 172.21.15.31:0/2868585829 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe3180c2a90 0x7fe31815a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:02.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.098+0000 7fe31759e700 1 --2- 172.21.15.31:0/2868585829 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3180c8200 0x7fe3180c3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:02.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.098+0000 7fe31759e700 1 -- 172.21.15.31:0/2868585829 >> 172.21.15.31:0/2868585829 conn(0x7fe31806c6a0 msgr2=0x7fe3180ca050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:02.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.099+0000 7fe31759e700 1 -- 172.21.15.31:0/2868585829 shutdown_connections 2023-10-06T02:31:02.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.099+0000 7fe31759e700 1 -- 172.21.15.31:0/2868585829 wait complete. 2023-10-06T02:31:02.118 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-06T02:31:02.193 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T02:31:02.197 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T02:31:02.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.306+0000 7f686359e700 1 Processor -- start 2023-10-06T02:31:02.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.306+0000 7f686359e700 1 -- start start 2023-10-06T02:31:02.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.307+0000 7f686359e700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f686406e0a0 0x7f68640c6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:02.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.307+0000 7f686359e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f686406f3a0 0x7f686414fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:02.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.307+0000 7f686359e700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f686406a490 con 0x7f686406e470 2023-10-06T02:31:02.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.307+0000 7f686359e700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f686406b930 con 0x7f686406f3a0 2023-10-06T02:31:02.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.307+0000 7f686359e700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68640c7490 con 0x7f686406e0a0 2023-10-06T02:31:02.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.308+0000 7f6861d9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f686406f3a0 0x7f686414fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:02.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.308+0000 7f6861d9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f686406f3a0 0x7f686414fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36498/0 (socket says 172.21.15.31:36498) 2023-10-06T02:31:02.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.308+0000 7f6861d9b700 1 -- 172.21.15.31:0/949851643 learned_addr learned my addr 172.21.15.31:0/949851643 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:02.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.308+0000 7f6862d9d700 1 --1- 172.21.15.31:0/949851643 >> v1:172.21.15.31:6789/0 conn(0x7f686406e470 0x7f6864159740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:36948/0 (socket says 172.21.15.31:36948) 2023-10-06T02:31:02.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.308+0000 7f686259c700 1 --2- 172.21.15.31:0/949851643 >> v2:172.21.15.62:3300/0 conn(0x7f686406e0a0 0x7f68640c6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:02.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.308+0000 7f6861d9b700 1 -- 172.21.15.31:0/949851643 >> v2:172.21.15.62:3300/0 conn(0x7f686406e0a0 msgr2=0x7f68640c6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:02.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.308+0000 7f6861d9b700 1 --2- 172.21.15.31:0/949851643 >> v2:172.21.15.62:3300/0 conn(0x7f686406e0a0 0x7f68640c6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:02.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.308+0000 7f6861d9b700 1 -- 172.21.15.31:0/949851643 >> v1:172.21.15.31:6789/0 conn(0x7f686406e470 legacy=0x7f6864159740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:02.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.308+0000 7f6861d9b700 1 -- 172.21.15.31:0/949851643 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68640c7770 con 0x7f686406f3a0 2023-10-06T02:31:02.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.308+0000 7f6861d9b700 1 --2- 172.21.15.31:0/949851643 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f686406f3a0 0x7f686414fc70 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f6864069230 tx=0x7f6858009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffb7969b5d26f3e3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:02.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.309+0000 7f686159a700 1 -- 172.21.15.31:0/949851643 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6858005c10 con 0x7f686406f3a0 2023-10-06T02:31:02.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.309+0000 7f686159a700 1 -- 172.21.15.31:0/949851643 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6858005d70 con 0x7f686406f3a0 2023-10-06T02:31:02.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.309+0000 7f686159a700 1 -- 172.21.15.31:0/949851643 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6858006260 con 0x7f686406f3a0 2023-10-06T02:31:02.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.309+0000 7f686359e700 1 -- 172.21.15.31:0/949851643 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f686406f3a0 msgr2=0x7f686414fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:02.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.309+0000 7f686359e700 1 --2- 172.21.15.31:0/949851643 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f686406f3a0 0x7f686414fc70 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f6864069230 tx=0x7f6858009220 comp rx=0 tx=0).stop 2023-10-06T02:31:02.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.309+0000 7f686359e700 1 -- 172.21.15.31:0/949851643 shutdown_connections 2023-10-06T02:31:02.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.309+0000 7f686359e700 1 --2- 172.21.15.31:0/949851643 >> v2:172.21.15.62:3300/0 conn(0x7f686406e0a0 0x7f68640c6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:02.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.309+0000 7f686359e700 1 --2- 172.21.15.31:0/949851643 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f686406f3a0 0x7f686414fc70 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:02.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.309+0000 7f686359e700 1 -- 172.21.15.31:0/949851643 >> 172.21.15.31:0/949851643 conn(0x7f686406d170 msgr2=0x7f68640cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:02.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.309+0000 7f686359e700 1 -- 172.21.15.31:0/949851643 shutdown_connections 2023-10-06T02:31:02.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.309+0000 7f686359e700 1 -- 172.21.15.31:0/949851643 wait complete. 2023-10-06T02:31:02.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.311+0000 7f686359e700 1 Processor -- start 2023-10-06T02:31:02.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.311+0000 7f686359e700 1 -- start start 2023-10-06T02:31:02.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.312+0000 7f686359e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f686406e0a0 0x7f68640b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:02.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.312+0000 7f686359e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f686406e470 0x7f686414b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:02.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.312+0000 7f686359e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f686406f3a0 0x7f686416d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:02.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.312+0000 7f686359e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f686414c440 con 0x7f686406e0a0 2023-10-06T02:31:02.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.312+0000 7f686359e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6864157d40 con 0x7f686406f3a0 2023-10-06T02:31:02.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.312+0000 7f686359e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6864156ea0 con 0x7f686406e470 2023-10-06T02:31:02.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.312+0000 7f686259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f686406e0a0 0x7f68640b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:02.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.312+0000 7f686259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f686406e0a0 0x7f68640b1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36504/0 (socket says 172.21.15.31:36504) 2023-10-06T02:31:02.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.312+0000 7f686259c700 1 -- 172.21.15.31:0/2079773636 learned_addr learned my addr 172.21.15.31:0/2079773636 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:02.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.312+0000 7f686259c700 1 -- 172.21.15.31:0/2079773636 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f686406e470 msgr2=0x7f686414b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:02.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.312+0000 7f686259c700 1 --2- 172.21.15.31:0/2079773636 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f686406e470 0x7f686414b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:02.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.312+0000 7f686259c700 1 -- 172.21.15.31:0/2079773636 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f686406f3a0 msgr2=0x7f686416d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:02.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.312+0000 7f6862d9d700 1 --2- 172.21.15.31:0/2079773636 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f686406f3a0 0x7f686416d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:02.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.312+0000 7f686259c700 1 --2- 172.21.15.31:0/2079773636 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f686406f3a0 0x7f686416d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:02.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.312+0000 7f686259c700 1 -- 172.21.15.31:0/2079773636 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6858009480 con 0x7f686406e0a0 2023-10-06T02:31:02.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.312+0000 7f6862d9d700 1 --2- 172.21.15.31:0/2079773636 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f686406f3a0 0x7f686416d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:31:02.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.313+0000 7f686259c700 1 --2- 172.21.15.31:0/2079773636 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f686406e0a0 0x7f68640b1a10 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f685400ec80 tx=0x7f68540091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:02.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.313+0000 7f68527fc700 1 -- 172.21.15.31:0/2079773636 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f685400c9d0 con 0x7f686406e0a0 2023-10-06T02:31:02.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.313+0000 7f68527fc700 1 -- 172.21.15.31:0/2079773636 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f685400cb30 con 0x7f686406e0a0 2023-10-06T02:31:02.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.313+0000 7f686359e700 1 -- 172.21.15.31:0/2079773636 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6864157180 con 0x7f686406e0a0 2023-10-06T02:31:02.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.313+0000 7f68527fc700 1 -- 172.21.15.31:0/2079773636 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6854013470 con 0x7f686406e0a0 2023-10-06T02:31:02.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.313+0000 7f686359e700 1 -- 172.21.15.31:0/2079773636 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6864156b90 con 0x7f686406e0a0 2023-10-06T02:31:02.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.314+0000 7f686359e700 1 -- 172.21.15.31:0/2079773636 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6864158020 con 0x7f686406e0a0 2023-10-06T02:31:02.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.315+0000 7f68527fc700 1 -- 172.21.15.31:0/2079773636 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f6854013820 con 0x7f686406e0a0 2023-10-06T02:31:02.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.316+0000 7f68527fc700 1 --2- 172.21.15.31:0/2079773636 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f68440674d0 0x7f6844069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:02.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.316+0000 7f68527fc700 1 -- 172.21.15.31:0/2079773636 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f68540854d0 con 0x7f686406e0a0 2023-10-06T02:31:02.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.316+0000 7f6861d9b700 1 --2- 172.21.15.31:0/2079773636 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f68440674d0 0x7f6844069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:02.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.317+0000 7f6861d9b700 1 --2- 172.21.15.31:0/2079773636 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f68440674d0 0x7f6844069980 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f686416d410 tx=0x7f6858016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:02.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.320+0000 7f68527fc700 1 -- 172.21.15.31:0/2079773636 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6854051d10 con 0x7f686406e0a0 2023-10-06T02:31:02.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.554+0000 7f686359e700 1 -- 172.21.15.31:0/2079773636 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f686404c3c0 con 0x7f68440674d0 2023-10-06T02:31:02.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.556+0000 7f68527fc700 1 -- 172.21.15.31:0/2079773636 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f686404c3c0 con 0x7f68440674d0 2023-10-06T02:31:02.556 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2023-10-06T02:31:02.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:31:02.557 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephfs" 2023-10-06T02:31:02.557 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:31:02.557 INFO:teuthology.orchestra.run.smithi031.stdout:] 2023-10-06T02:31:02.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.558+0000 7f686359e700 1 -- 172.21.15.31:0/2079773636 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f68440674d0 msgr2=0x7f6844069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:02.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.558+0000 7f686359e700 1 --2- 172.21.15.31:0/2079773636 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f68440674d0 0x7f6844069980 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f686416d410 tx=0x7f6858016000 comp rx=0 tx=0).stop 2023-10-06T02:31:02.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.558+0000 7f686359e700 1 -- 172.21.15.31:0/2079773636 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f686406e0a0 msgr2=0x7f68640b1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:02.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.558+0000 7f686359e700 1 --2- 172.21.15.31:0/2079773636 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f686406e0a0 0x7f68640b1a10 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f685400ec80 tx=0x7f68540091a0 comp rx=0 tx=0).stop 2023-10-06T02:31:02.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.559+0000 7f686359e700 1 -- 172.21.15.31:0/2079773636 shutdown_connections 2023-10-06T02:31:02.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.559+0000 7f686359e700 1 --2- 172.21.15.31:0/2079773636 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f686406f3a0 0x7f686416d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:02.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.559+0000 7f686359e700 1 --2- 172.21.15.31:0/2079773636 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f686406e0a0 0x7f68640b1a10 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:02.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.559+0000 7f686359e700 1 --2- 172.21.15.31:0/2079773636 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f68440674d0 0x7f6844069980 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:02.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.559+0000 7f686359e700 1 --2- 172.21.15.31:0/2079773636 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f686406e470 0x7f686414b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:02.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.559+0000 7f686359e700 1 -- 172.21.15.31:0/2079773636 >> 172.21.15.31:0/2079773636 conn(0x7f686406d170 msgr2=0x7f686414da20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:02.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.559+0000 7f686359e700 1 -- 172.21.15.31:0/2079773636 shutdown_connections 2023-10-06T02:31:02.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.559+0000 7f686359e700 1 -- 172.21.15.31:0/2079773636 wait complete. 2023-10-06T02:31:02.577 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T02:31:02.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.807+0000 7f72a919d700 1 Processor -- start 2023-10-06T02:31:02.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.807+0000 7f72a919d700 1 -- start start 2023-10-06T02:31:02.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.808+0000 7f72a919d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f72a4159550 0x7f72a40ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:02.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.808+0000 7f72a919d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72a40acc00 con 0x7f72a4159920 2023-10-06T02:31:02.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.808+0000 7f72a919d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72a40abff0 con 0x7f72a415dfd0 2023-10-06T02:31:02.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.808+0000 7f72a919d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72a40ac2d0 con 0x7f72a4159550 2023-10-06T02:31:02.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.809+0000 7f72a3fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f72a4159550 0x7f72a40ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:02.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.809+0000 7f72a899c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f72a4159920 0x7f72a40ad090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:36964/0 (socket says 172.21.15.31:36964) 2023-10-06T02:31:02.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.809+0000 7f72a899c700 1 -- 172.21.15.31:0/2927838020 learned_addr learned my addr 172.21.15.31:0/2927838020 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:31:02.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.809+0000 7f72a3fff700 1 -- 172.21.15.31:0/2927838020 >> v1:172.21.15.62:6789/0 conn(0x7f72a415dfd0 legacy=0x7f72a40aedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:02.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.809+0000 7f72a3fff700 1 -- 172.21.15.31:0/2927838020 >> v1:172.21.15.31:6789/0 conn(0x7f72a4159920 legacy=0x7f72a40ad090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:02.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.809+0000 7f72a3fff700 1 -- 172.21.15.31:0/2927838020 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72a40ac5b0 con 0x7f72a4159550 2023-10-06T02:31:02.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.809+0000 7f72a2ffd700 1 -- 172.21.15.31:0/2927838020 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 263921605 0 0) 0x7f72a40acc00 con 0x7f72a4159920 2023-10-06T02:31:02.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.809+0000 7f72a3fff700 1 --2- 172.21.15.31:0/2927838020 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72a4159550 0x7f72a40ceac0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f72940098e0 tx=0x7f72940091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfedba7c612913ac server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:02.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.810+0000 7f72a2ffd700 1 -- 172.21.15.31:0/2927838020 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7294009e80 con 0x7f72a4159550 2023-10-06T02:31:02.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.810+0000 7f72a2ffd700 1 -- 172.21.15.31:0/2927838020 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f729400b860 con 0x7f72a4159550 2023-10-06T02:31:02.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.810+0000 7f72a2ffd700 1 -- 172.21.15.31:0/2927838020 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f729400bbe0 con 0x7f72a4159550 2023-10-06T02:31:02.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.810+0000 7f72a919d700 1 -- 172.21.15.31:0/2927838020 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72a4159550 msgr2=0x7f72a40ceac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:02.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.810+0000 7f72a919d700 1 --2- 172.21.15.31:0/2927838020 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72a4159550 0x7f72a40ceac0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f72940098e0 tx=0x7f72940091a0 comp rx=0 tx=0).stop 2023-10-06T02:31:02.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.810+0000 7f72a919d700 1 -- 172.21.15.31:0/2927838020 shutdown_connections 2023-10-06T02:31:02.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.810+0000 7f72a919d700 1 --2- 172.21.15.31:0/2927838020 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72a4159550 0x7f72a40ceac0 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:02.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.810+0000 7f72a919d700 1 -- 172.21.15.31:0/2927838020 >> 172.21.15.31:0/2927838020 conn(0x7f72a406ce80 msgr2=0x7f72a40ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:02.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.810+0000 7f72a919d700 1 -- 172.21.15.31:0/2927838020 shutdown_connections 2023-10-06T02:31:02.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.810+0000 7f72a919d700 1 -- 172.21.15.31:0/2927838020 wait complete. 2023-10-06T02:31:02.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.812+0000 7f72a919d700 1 Processor -- start 2023-10-06T02:31:02.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.812+0000 7f72a919d700 1 -- start start 2023-10-06T02:31:02.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.813+0000 7f72a919d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72a4159550 0x7f72a40c8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:02.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.813+0000 7f72a919d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72a4159920 0x7f72a416d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:02.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.813+0000 7f72a919d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f72a415dfd0 0x7f72a4148590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:02.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.813+0000 7f72a919d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a40cc4b0 con 0x7f72a4159550 2023-10-06T02:31:02.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.813+0000 7f72a919d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a40c9910 con 0x7f72a4159920 2023-10-06T02:31:02.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.813+0000 7f72a919d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72a40c9bf0 con 0x7f72a415dfd0 2023-10-06T02:31:02.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.813+0000 7f72a3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72a4159550 0x7f72a40c8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:02.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.813+0000 7f72a3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72a4159550 0x7f72a40c8fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36532/0 (socket says 172.21.15.31:36532) 2023-10-06T02:31:02.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.813+0000 7f72a3fff700 1 -- 172.21.15.31:0/317249668 learned_addr learned my addr 172.21.15.31:0/317249668 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:02.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.813+0000 7f72a37fe700 1 --2- 172.21.15.31:0/317249668 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72a4159920 0x7f72a416d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:02.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.813+0000 7f72a899c700 1 --2- 172.21.15.31:0/317249668 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f72a415dfd0 0x7f72a4148590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:02.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.813+0000 7f72a3fff700 1 -- 172.21.15.31:0/317249668 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f72a415dfd0 msgr2=0x7f72a4148590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:02.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.813+0000 7f72a3fff700 1 --2- 172.21.15.31:0/317249668 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f72a415dfd0 0x7f72a4148590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:02.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.813+0000 7f72a3fff700 1 -- 172.21.15.31:0/317249668 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72a4159920 msgr2=0x7f72a416d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:02.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.813+0000 7f72a3fff700 1 --2- 172.21.15.31:0/317249668 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72a4159920 0x7f72a416d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:02.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.813+0000 7f72a3fff700 1 -- 172.21.15.31:0/317249668 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7294009420 con 0x7f72a4159550 2023-10-06T02:31:02.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.813+0000 7f72a37fe700 1 --2- 172.21.15.31:0/317249668 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72a4159920 0x7f72a416d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:31:02.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.814+0000 7f72a899c700 1 --2- 172.21.15.31:0/317249668 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f72a415dfd0 0x7f72a4148590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:31:02.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.814+0000 7f72a3fff700 1 --2- 172.21.15.31:0/317249668 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72a4159550 0x7f72a40c8fc0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f72a406ad70 tx=0x7f7294017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:02.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.814+0000 7f728bfff700 1 -- 172.21.15.31:0/317249668 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f729401f070 con 0x7f72a4159550 2023-10-06T02:31:02.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.814+0000 7f728bfff700 1 -- 172.21.15.31:0/317249668 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f729400b9d0 con 0x7f72a4159550 2023-10-06T02:31:02.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.814+0000 7f72a919d700 1 -- 172.21.15.31:0/317249668 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72a40c9ed0 con 0x7f72a4159550 2023-10-06T02:31:02.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.814+0000 7f728bfff700 1 -- 172.21.15.31:0/317249668 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7294025670 con 0x7f72a4159550 2023-10-06T02:31:02.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.814+0000 7f72a919d700 1 -- 172.21.15.31:0/317249668 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72a40c9500 con 0x7f72a4159550 2023-10-06T02:31:02.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.815+0000 7f72a919d700 1 -- 172.21.15.31:0/317249668 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72a404c3c0 con 0x7f72a4159550 2023-10-06T02:31:02.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.816+0000 7f728bfff700 1 -- 172.21.15.31:0/317249668 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f7294023070 con 0x7f72a4159550 2023-10-06T02:31:02.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.817+0000 7f728bfff700 1 --2- 172.21.15.31:0/317249668 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f72900674d0 0x7f7290069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:02.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.817+0000 7f728bfff700 1 -- 172.21.15.31:0/317249668 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f7294086730 con 0x7f72a4159550 2023-10-06T02:31:02.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.817+0000 7f72a37fe700 1 --2- 172.21.15.31:0/317249668 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f72900674d0 0x7f7290069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:02.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.818+0000 7f72a37fe700 1 --2- 172.21.15.31:0/317249668 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f72900674d0 0x7f7290069980 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7298007fd0 tx=0x7f7298009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:02.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:02.821+0000 7f728bfff700 1 -- 172.21.15.31:0/317249668 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7294052f70 con 0x7f72a4159550 2023-10-06T02:31:03.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.025+0000 7f72a919d700 1 -- 172.21.15.31:0/317249668 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f72a40cab90 con 0x7f72a4159550 2023-10-06T02:31:03.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.025+0000 7f728bfff700 1 -- 172.21.15.31:0/317249668 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v26)=0 v26) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7294056590 con 0x7f72a4159550 2023-10-06T02:31:03.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.027+0000 7f72a919d700 1 -- 172.21.15.31:0/317249668 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f72900674d0 msgr2=0x7f7290069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:03.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.027+0000 7f72a919d700 1 --2- 172.21.15.31:0/317249668 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f72900674d0 0x7f7290069980 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7298007fd0 tx=0x7f7298009040 comp rx=0 tx=0).stop 2023-10-06T02:31:03.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.027+0000 7f72a919d700 1 -- 172.21.15.31:0/317249668 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72a4159550 msgr2=0x7f72a40c8fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:03.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.027+0000 7f72a919d700 1 --2- 172.21.15.31:0/317249668 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72a4159550 0x7f72a40c8fc0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f72a406ad70 tx=0x7f7294017000 comp rx=0 tx=0).stop 2023-10-06T02:31:03.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.027+0000 7f72a919d700 1 -- 172.21.15.31:0/317249668 shutdown_connections 2023-10-06T02:31:03.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.027+0000 7f72a919d700 1 --2- 172.21.15.31:0/317249668 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f72900674d0 0x7f7290069980 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:03.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.027+0000 7f72a919d700 1 --2- 172.21.15.31:0/317249668 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f72a415dfd0 0x7f72a4148590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:03.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.027+0000 7f72a919d700 1 --2- 172.21.15.31:0/317249668 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72a4159550 0x7f72a40c8fc0 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:03.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.027+0000 7f72a919d700 1 --2- 172.21.15.31:0/317249668 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72a4159920 0x7f72a416d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:03.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.027+0000 7f72a919d700 1 -- 172.21.15.31:0/317249668 >> 172.21.15.31:0/317249668 conn(0x7f72a406ce80 msgr2=0x7f72a40aed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:03.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.028+0000 7f72a919d700 1 -- 172.21.15.31:0/317249668 shutdown_connections 2023-10-06T02:31:03.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.028+0000 7f72a919d700 1 -- 172.21.15.31:0/317249668 wait complete. 2023-10-06T02:31:03.046 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:31:03.046 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/_nogroup/subvolume_0000000000652135) 2023-10-06T02:31:03.119 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:31:03.119 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/group_0000000000911578/subvolume_0000000000652136) 2023-10-06T02:31:03.185 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000652135 2023-10-06T02:31:03.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.375+0000 7fc42259d700 1 Processor -- start 2023-10-06T02:31:03.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.375+0000 7fc42259d700 1 -- start start 2023-10-06T02:31:03.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.376+0000 7fc42259d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc41c0c7be0 0x7fc41c06abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:03.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.376+0000 7fc42259d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc41c13f780 0x7fc41c13fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:03.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.376+0000 7fc42259d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc41c1400e0 con 0x7fc41c0c7fb0 2023-10-06T02:31:03.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.376+0000 7fc42259d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc41c06bf10 con 0x7fc41c13f780 2023-10-06T02:31:03.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.376+0000 7fc42259d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc41c06b290 con 0x7fc41c0c7be0 2023-10-06T02:31:03.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.376+0000 7fc421d9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc41c13f780 0x7fc41c13fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:03.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.376+0000 7fc421d9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc41c13f780 0x7fc41c13fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36538/0 (socket says 172.21.15.31:36538) 2023-10-06T02:31:03.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.376+0000 7fc421d9c700 1 -- 172.21.15.31:0/515684316 learned_addr learned my addr 172.21.15.31:0/515684316 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:03.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.377+0000 7fc42159b700 1 --2- 172.21.15.31:0/515684316 >> v2:172.21.15.62:3300/0 conn(0x7fc41c0c7be0 0x7fc41c06abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:03.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.377+0000 7fc421d9c700 1 -- 172.21.15.31:0/515684316 >> v2:172.21.15.62:3300/0 conn(0x7fc41c0c7be0 msgr2=0x7fc41c06abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:03.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.377+0000 7fc421d9c700 1 --2- 172.21.15.31:0/515684316 >> v2:172.21.15.62:3300/0 conn(0x7fc41c0c7be0 0x7fc41c06abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:03.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.377+0000 7fc421d9c700 1 -- 172.21.15.31:0/515684316 >> v1:172.21.15.62:6789/0 conn(0x7fc41c0c7fb0 legacy=0x7fc41c15a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:03.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.377+0000 7fc421d9c700 1 -- 172.21.15.31:0/515684316 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc41c06b570 con 0x7fc41c13f780 2023-10-06T02:31:03.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.377+0000 7fc421d9c700 1 --2- 172.21.15.31:0/515684316 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc41c13f780 0x7fc41c13fba0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fc41c0c8b00 tx=0x7fc414009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=308395605b7a4475 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:03.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.377+0000 7fc413fff700 1 -- 172.21.15.31:0/515684316 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc41400e070 con 0x7fc41c13f780 2023-10-06T02:31:03.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.378+0000 7fc413fff700 1 -- 172.21.15.31:0/515684316 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc41400ba20 con 0x7fc41c13f780 2023-10-06T02:31:03.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.378+0000 7fc413fff700 1 -- 172.21.15.31:0/515684316 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4140092e0 con 0x7fc41c13f780 2023-10-06T02:31:03.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.378+0000 7fc42259d700 1 -- 172.21.15.31:0/515684316 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc41c13f780 msgr2=0x7fc41c13fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:03.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.378+0000 7fc42259d700 1 --2- 172.21.15.31:0/515684316 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc41c13f780 0x7fc41c13fba0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fc41c0c8b00 tx=0x7fc414009840 comp rx=0 tx=0).stop 2023-10-06T02:31:03.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.378+0000 7fc42259d700 1 -- 172.21.15.31:0/515684316 shutdown_connections 2023-10-06T02:31:03.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.378+0000 7fc42259d700 1 --2- 172.21.15.31:0/515684316 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc41c13f780 0x7fc41c13fba0 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:03.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.378+0000 7fc42259d700 1 --2- 172.21.15.31:0/515684316 >> v2:172.21.15.62:3300/0 conn(0x7fc41c0c7be0 0x7fc41c06abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:03.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.378+0000 7fc42259d700 1 -- 172.21.15.31:0/515684316 >> 172.21.15.31:0/515684316 conn(0x7fc41c06d110 msgr2=0x7fc41c0cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:03.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.378+0000 7fc42259d700 1 -- 172.21.15.31:0/515684316 shutdown_connections 2023-10-06T02:31:03.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.378+0000 7fc42259d700 1 -- 172.21.15.31:0/515684316 wait complete. 2023-10-06T02:31:03.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.380+0000 7fc42259d700 1 Processor -- start 2023-10-06T02:31:03.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.380+0000 7fc42259d700 1 -- start start 2023-10-06T02:31:03.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.381+0000 7fc42259d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc41c0c7be0 0x7fc41c0c2910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:03.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.381+0000 7fc42259d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc41c0c7fb0 0x7fc41c155c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:03.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.381+0000 7fc42259d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc41c13f780 0x7fc41c0ca920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:03.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.381+0000 7fc42259d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc41c0c2e50 con 0x7fc41c13f780 2023-10-06T02:31:03.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.381+0000 7fc42259d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc41c0c1440 con 0x7fc41c0c7fb0 2023-10-06T02:31:03.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.381+0000 7fc42259d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc41c0c1720 con 0x7fc41c0c7be0 2023-10-06T02:31:03.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.381+0000 7fc421d9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc41c13f780 0x7fc41c0ca920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:03.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.381+0000 7fc421d9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc41c13f780 0x7fc41c0ca920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36540/0 (socket says 172.21.15.31:36540) 2023-10-06T02:31:03.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.381+0000 7fc421d9c700 1 -- 172.21.15.31:0/2595176857 learned_addr learned my addr 172.21.15.31:0/2595176857 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:03.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.381+0000 7fc421d9c700 1 -- 172.21.15.31:0/2595176857 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc41c0c7be0 msgr2=0x7fc41c0c2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:03.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.381+0000 7fc421d9c700 1 --2- 172.21.15.31:0/2595176857 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc41c0c7be0 0x7fc41c0c2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:03.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.381+0000 7fc421d9c700 1 -- 172.21.15.31:0/2595176857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc41c0c7fb0 msgr2=0x7fc41c155c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:03.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.381+0000 7fc421d9c700 1 --2- 172.21.15.31:0/2595176857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc41c0c7fb0 0x7fc41c155c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:03.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.381+0000 7fc421d9c700 1 -- 172.21.15.31:0/2595176857 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc414009ae0 con 0x7fc41c13f780 2023-10-06T02:31:03.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.382+0000 7fc421d9c700 1 --2- 172.21.15.31:0/2595176857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc41c13f780 0x7fc41c0ca920 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7fc414009880 tx=0x7fc414018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:03.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.382+0000 7fc4117fa700 1 -- 172.21.15.31:0/2595176857 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc414008be0 con 0x7fc41c13f780 2023-10-06T02:31:03.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.382+0000 7fc4117fa700 1 -- 172.21.15.31:0/2595176857 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc414008d40 con 0x7fc41c13f780 2023-10-06T02:31:03.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.382+0000 7fc42259d700 1 -- 172.21.15.31:0/2595176857 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc41c0c1a00 con 0x7fc41c13f780 2023-10-06T02:31:03.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.382+0000 7fc4117fa700 1 -- 172.21.15.31:0/2595176857 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4140256b0 con 0x7fc41c13f780 2023-10-06T02:31:03.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.382+0000 7fc42259d700 1 -- 172.21.15.31:0/2595176857 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc41c1582c0 con 0x7fc41c13f780 2023-10-06T02:31:03.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.383+0000 7fc42259d700 1 -- 172.21.15.31:0/2595176857 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc41c158540 con 0x7fc41c13f780 2023-10-06T02:31:03.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.384+0000 7fc4117fa700 1 -- 172.21.15.31:0/2595176857 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7fc414003040 con 0x7fc41c13f780 2023-10-06T02:31:03.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.385+0000 7fc4117fa700 1 --2- 172.21.15.31:0/2595176857 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc4000674d0 0x7fc400069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:03.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.385+0000 7fc4117fa700 1 -- 172.21.15.31:0/2595176857 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fc414086ea0 con 0x7fc41c13f780 2023-10-06T02:31:03.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.385+0000 7fc42159b700 1 --2- 172.21.15.31:0/2595176857 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc4000674d0 0x7fc400069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:03.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.386+0000 7fc42159b700 1 --2- 172.21.15.31:0/2595176857 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc4000674d0 0x7fc400069980 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc418009bb0 tx=0x7fc4180091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:03.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.389+0000 7fc4117fa700 1 -- 172.21.15.31:0/2595176857 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc4140535d0 con 0x7fc41c13f780 2023-10-06T02:31:03.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.626+0000 7fc42259d700 1 -- 172.21.15.31:0/2595176857 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000652135", "target": ["mon-mgr", ""]}) v1 -- 0x7fc41c0c1890 con 0x7fc4000674d0 2023-10-06T02:31:03.635 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:31:03.634+0000 7f656537d700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:31:03.635 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:31:03.634+0000 7f656537d700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:31:03.635 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:31:03.634+0000 7f656537d700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:31:03.635 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:31:03.634+0000 7f656537d700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:31:03.635 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:31:03.634+0000 7f656537d700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:31:03.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.669+0000 7fc4117fa700 1 -- 172.21.15.31:0/2595176857 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+45 (secure 0 0 0) 0x7fc41c0c1890 con 0x7fc4000674d0 2023-10-06T02:31:03.670 INFO:teuthology.orchestra.run.smithi031.stdout:/volumes/_nogroup/subvolume_0000000000652135 2023-10-06T02:31:03.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.671+0000 7fc42259d700 1 -- 172.21.15.31:0/2595176857 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc4000674d0 msgr2=0x7fc400069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:03.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.671+0000 7fc42259d700 1 --2- 172.21.15.31:0/2595176857 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc4000674d0 0x7fc400069980 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc418009bb0 tx=0x7fc4180091a0 comp rx=0 tx=0).stop 2023-10-06T02:31:03.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.671+0000 7fc42259d700 1 -- 172.21.15.31:0/2595176857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc41c13f780 msgr2=0x7fc41c0ca920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:03.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.671+0000 7fc42259d700 1 --2- 172.21.15.31:0/2595176857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc41c13f780 0x7fc41c0ca920 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7fc414009880 tx=0x7fc414018000 comp rx=0 tx=0).stop 2023-10-06T02:31:03.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.671+0000 7fc42259d700 1 -- 172.21.15.31:0/2595176857 shutdown_connections 2023-10-06T02:31:03.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.671+0000 7fc42259d700 1 --2- 172.21.15.31:0/2595176857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc41c13f780 0x7fc41c0ca920 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:03.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.671+0000 7fc42259d700 1 --2- 172.21.15.31:0/2595176857 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc4000674d0 0x7fc400069980 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:03.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.671+0000 7fc42259d700 1 --2- 172.21.15.31:0/2595176857 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc41c0c7be0 0x7fc41c0c2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:03.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.671+0000 7fc42259d700 1 --2- 172.21.15.31:0/2595176857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc41c0c7fb0 0x7fc41c155c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:03.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.671+0000 7fc42259d700 1 -- 172.21.15.31:0/2595176857 >> 172.21.15.31:0/2595176857 conn(0x7fc41c06d110 msgr2=0x7fc41c0b16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:03.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.671+0000 7fc42259d700 1 -- 172.21.15.31:0/2595176857 shutdown_connections 2023-10-06T02:31:03.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.671+0000 7fc42259d700 1 -- 172.21.15.31:0/2595176857 wait complete. 2023-10-06T02:31:03.689 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000652136 group_0000000000911578 2023-10-06T02:31:03.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.880+0000 7fc4a359e700 1 Processor -- start 2023-10-06T02:31:03.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.880+0000 7fc4a359e700 1 -- start start 2023-10-06T02:31:03.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.881+0000 7fc4a359e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc4a40c2a90 0x7fc4a4156aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:03.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.881+0000 7fc4a359e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc4a40c85d0 0x7fc4a40cc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:03.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.881+0000 7fc4a359e700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4a41579b0 con 0x7fc4a40c8200 2023-10-06T02:31:03.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.881+0000 7fc4a359e700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4a40c1390 con 0x7fc4a40c85d0 2023-10-06T02:31:03.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.882+0000 7fc4a359e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a41573f0 con 0x7fc4a40c2a90 2023-10-06T02:31:03.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.882+0000 7fc4a2d9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc4a40c85d0 0x7fc4a40cc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:03.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.882+0000 7fc4a2d9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc4a40c85d0 0x7fc4a40cc2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36554/0 (socket says 172.21.15.31:36554) 2023-10-06T02:31:03.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.882+0000 7fc4a2d9d700 1 -- 172.21.15.31:0/2472279271 learned_addr learned my addr 172.21.15.31:0/2472279271 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:03.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.882+0000 7fc4a1d9b700 1 --2- 172.21.15.31:0/2472279271 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc4a40c2a90 0x7fc4a4156aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:03.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.882+0000 7fc4a2d9d700 1 -- 172.21.15.31:0/2472279271 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc4a40c2a90 msgr2=0x7fc4a4156aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:03.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.882+0000 7fc4a2d9d700 1 --2- 172.21.15.31:0/2472279271 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc4a40c2a90 0x7fc4a4156aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:03.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.882+0000 7fc4a2d9d700 1 -- 172.21.15.31:0/2472279271 >> v1:172.21.15.62:6789/0 conn(0x7fc4a40c8200 legacy=0x7fc4a40c1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:03.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.882+0000 7fc4a2d9d700 1 -- 172.21.15.31:0/2472279271 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4a41576d0 con 0x7fc4a40c85d0 2023-10-06T02:31:03.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.882+0000 7fc4a1d9b700 1 --2- 172.21.15.31:0/2472279271 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc4a40c2a90 0x7fc4a4156aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:31:03.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.883+0000 7fc4a2d9d700 1 --2- 172.21.15.31:0/2472279271 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4a40c85d0 0x7fc4a40cc2f0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7fc49c009f20 tx=0x7fc49c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6929a62fb6160c58 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:03.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.883+0000 7fc4a159a700 1 -- 172.21.15.31:0/2472279271 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc49c00e070 con 0x7fc4a40c85d0 2023-10-06T02:31:03.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.883+0000 7fc4a159a700 1 -- 172.21.15.31:0/2472279271 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc49c00ba40 con 0x7fc4a40c85d0 2023-10-06T02:31:03.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.883+0000 7fc4a159a700 1 -- 172.21.15.31:0/2472279271 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc49c0092e0 con 0x7fc4a40c85d0 2023-10-06T02:31:03.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.883+0000 7fc4a359e700 1 -- 172.21.15.31:0/2472279271 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4a40c85d0 msgr2=0x7fc4a40cc2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:03.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.883+0000 7fc4a359e700 1 --2- 172.21.15.31:0/2472279271 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4a40c85d0 0x7fc4a40cc2f0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7fc49c009f20 tx=0x7fc49c009840 comp rx=0 tx=0).stop 2023-10-06T02:31:03.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.883+0000 7fc4a359e700 1 -- 172.21.15.31:0/2472279271 shutdown_connections 2023-10-06T02:31:03.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.883+0000 7fc4a359e700 1 --2- 172.21.15.31:0/2472279271 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4a40c85d0 0x7fc4a40cc2f0 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:03.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.883+0000 7fc4a359e700 1 --2- 172.21.15.31:0/2472279271 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc4a40c2a90 0x7fc4a4156aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:03.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.883+0000 7fc4a359e700 1 -- 172.21.15.31:0/2472279271 >> 172.21.15.31:0/2472279271 conn(0x7fc4a406c6a0 msgr2=0x7fc4a406caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:03.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.884+0000 7fc4a359e700 1 -- 172.21.15.31:0/2472279271 shutdown_connections 2023-10-06T02:31:03.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.884+0000 7fc4a359e700 1 -- 172.21.15.31:0/2472279271 wait complete. 2023-10-06T02:31:03.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.885+0000 7fc4a359e700 1 Processor -- start 2023-10-06T02:31:03.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.885+0000 7fc4a359e700 1 -- start start 2023-10-06T02:31:03.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.886+0000 7fc4a359e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc4a40c2a90 0x7fc4a4146fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:03.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.886+0000 7fc4a359e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4a40c8200 0x7fc4a4149fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:03.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.886+0000 7fc4a359e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4a40c85d0 0x7fc4a40cc490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:03.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.886+0000 7fc4a359e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a40cc000 con 0x7fc4a40c8200 2023-10-06T02:31:03.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.886+0000 7fc4a359e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a40cb430 con 0x7fc4a40c85d0 2023-10-06T02:31:03.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.886+0000 7fc4a359e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a40cb6d0 con 0x7fc4a40c2a90 2023-10-06T02:31:03.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.886+0000 7fc4a1d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4a40c8200 0x7fc4a4149fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:03.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.886+0000 7fc4a1d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4a40c8200 0x7fc4a4149fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36568/0 (socket says 172.21.15.31:36568) 2023-10-06T02:31:03.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.886+0000 7fc4a1d9b700 1 -- 172.21.15.31:0/1083427006 learned_addr learned my addr 172.21.15.31:0/1083427006 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:03.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.886+0000 7fc4a259c700 1 --2- 172.21.15.31:0/1083427006 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc4a40c2a90 0x7fc4a4146fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:03.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.887+0000 7fc4a2d9d700 1 --2- 172.21.15.31:0/1083427006 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4a40c85d0 0x7fc4a40cc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:03.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.887+0000 7fc4a1d9b700 1 -- 172.21.15.31:0/1083427006 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc4a40c2a90 msgr2=0x7fc4a4146fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:03.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.887+0000 7fc4a1d9b700 1 --2- 172.21.15.31:0/1083427006 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc4a40c2a90 0x7fc4a4146fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:03.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.887+0000 7fc4a1d9b700 1 -- 172.21.15.31:0/1083427006 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4a40c85d0 msgr2=0x7fc4a40cc490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:03.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.887+0000 7fc4a1d9b700 1 --2- 172.21.15.31:0/1083427006 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4a40c85d0 0x7fc4a40cc490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:03.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.887+0000 7fc4a1d9b700 1 -- 172.21.15.31:0/1083427006 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc49c009ae0 con 0x7fc4a40c8200 2023-10-06T02:31:03.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.887+0000 7fc4a259c700 1 --2- 172.21.15.31:0/1083427006 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc4a40c2a90 0x7fc4a4146fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:31:03.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.887+0000 7fc4a1d9b700 1 --2- 172.21.15.31:0/1083427006 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4a40c8200 0x7fc4a4149fd0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7fc49800c830 tx=0x7fc498008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:03.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.887+0000 7fc4927fc700 1 -- 172.21.15.31:0/1083427006 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc498011600 con 0x7fc4a40c8200 2023-10-06T02:31:03.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.887+0000 7fc4927fc700 1 -- 172.21.15.31:0/1083427006 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc498011760 con 0x7fc4a40c8200 2023-10-06T02:31:03.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.887+0000 7fc4927fc700 1 -- 172.21.15.31:0/1083427006 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc498011b20 con 0x7fc4a40c8200 2023-10-06T02:31:03.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.887+0000 7fc4a359e700 1 -- 172.21.15.31:0/1083427006 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4a40cb9b0 con 0x7fc4a40c8200 2023-10-06T02:31:03.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.887+0000 7fc4a359e700 1 -- 172.21.15.31:0/1083427006 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4a4147500 con 0x7fc4a40c8200 2023-10-06T02:31:03.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.888+0000 7fc4a359e700 1 -- 172.21.15.31:0/1083427006 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4a40cc170 con 0x7fc4a40c8200 2023-10-06T02:31:03.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.889+0000 7fc4927fc700 1 -- 172.21.15.31:0/1083427006 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 85250+0+0 (secure 0 0 0) 0x7fc4980193b0 con 0x7fc4a40c8200 2023-10-06T02:31:03.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.891+0000 7fc4927fc700 1 --2- 172.21.15.31:0/1083427006 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc4840674d0 0x7fc484069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:03.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.891+0000 7fc4927fc700 1 -- 172.21.15.31:0/1083427006 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fc498085250 con 0x7fc4a40c8200 2023-10-06T02:31:03.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.892+0000 7fc4a259c700 1 --2- 172.21.15.31:0/1083427006 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc4840674d0 0x7fc484069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:03.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.892+0000 7fc4a259c700 1 --2- 172.21.15.31:0/1083427006 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc4840674d0 0x7fc484069980 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc494007fd0 tx=0x7fc494009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:03.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:03.895+0000 7fc4927fc700 1 -- 172.21.15.31:0/1083427006 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc498051980 con 0x7fc4a40c8200 2023-10-06T02:31:04.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.115+0000 7fc4927fc700 1 -- 172.21.15.31:0/1083427006 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mgrmap(e 23) v1 ==== 85301+0+0 (secure 0 0 0) 0x7fc498051d50 con 0x7fc4a40c8200 2023-10-06T02:31:04.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.132+0000 7fc4a359e700 1 -- 172.21.15.31:0/1083427006 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000652136", "group_name": "group_0000000000911578", "target": ["mon-mgr", ""]}) v1 -- 0x7fc4a404c3c0 con 0x7fc4840674d0 2023-10-06T02:31:04.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.152+0000 7fc4927fc700 1 -- 172.21.15.31:0/1083427006 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+59 (secure 0 0 0) 0x7fc4a404c3c0 con 0x7fc4840674d0 2023-10-06T02:31:04.284 INFO:teuthology.orchestra.run.smithi031.stdout:/volumes/group_0000000000911578/subvolume_0000000000652136 2023-10-06T02:31:04.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.154+0000 7fc4a359e700 1 -- 172.21.15.31:0/1083427006 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc4840674d0 msgr2=0x7fc484069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:04.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.154+0000 7fc4a359e700 1 --2- 172.21.15.31:0/1083427006 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc4840674d0 0x7fc484069980 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc494007fd0 tx=0x7fc494009040 comp rx=0 tx=0).stop 2023-10-06T02:31:04.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.154+0000 7fc4a359e700 1 -- 172.21.15.31:0/1083427006 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4a40c8200 msgr2=0x7fc4a4149fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:04.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.154+0000 7fc4a359e700 1 --2- 172.21.15.31:0/1083427006 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4a40c8200 0x7fc4a4149fd0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7fc49800c830 tx=0x7fc498008040 comp rx=0 tx=0).stop 2023-10-06T02:31:04.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.154+0000 7fc4a359e700 1 -- 172.21.15.31:0/1083427006 shutdown_connections 2023-10-06T02:31:04.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.154+0000 7fc4a359e700 1 --2- 172.21.15.31:0/1083427006 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4a40c85d0 0x7fc4a40cc490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:04.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.154+0000 7fc4a359e700 1 --2- 172.21.15.31:0/1083427006 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc4840674d0 0x7fc484069980 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:04.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.154+0000 7fc4a359e700 1 --2- 172.21.15.31:0/1083427006 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc4a40c2a90 0x7fc4a4146fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:04.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.154+0000 7fc4a359e700 1 --2- 172.21.15.31:0/1083427006 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4a40c8200 0x7fc4a4149fd0 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:04.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.154+0000 7fc4a359e700 1 -- 172.21.15.31:0/1083427006 >> 172.21.15.31:0/1083427006 conn(0x7fc4a406c6a0 msgr2=0x7fc4a4148e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:04.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.154+0000 7fc4a359e700 1 -- 172.21.15.31:0/1083427006 shutdown_connections 2023-10-06T02:31:04.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.154+0000 7fc4a359e700 1 -- 172.21.15.31:0/1083427006 wait complete. 2023-10-06T02:31:04.287 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000652135 2023-10-06T02:31:04.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.478+0000 7f9e6339d700 1 Processor -- start 2023-10-06T02:31:04.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.479+0000 7f9e6339d700 1 -- start start 2023-10-06T02:31:04.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.480+0000 7f9e6339d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9e5c0c7be0 0x7f9e5c06abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:04.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.480+0000 7f9e6339d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9e5c0c7fb0 0x7f9e5c15a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:04.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.480+0000 7f9e6339d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e5c15aed0 con 0x7f9e5c13f780 2023-10-06T02:31:04.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.480+0000 7f9e6339d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e5c06bf10 con 0x7f9e5c0c7be0 2023-10-06T02:31:04.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.480+0000 7f9e6339d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e5c06b290 con 0x7f9e5c0c7fb0 2023-10-06T02:31:04.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.480+0000 7f9e6239b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9e5c0c7be0 0x7f9e5c06abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:04.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.480+0000 7f9e6239b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9e5c0c7be0 0x7f9e5c06abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36572/0 (socket says 172.21.15.31:36572) 2023-10-06T02:31:04.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.480+0000 7f9e6239b700 1 -- 172.21.15.31:0/3265592896 learned_addr learned my addr 172.21.15.31:0/3265592896 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:04.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.480+0000 7f9e62b9c700 1 --1- 172.21.15.31:0/3265592896 >> v1:172.21.15.31:6789/0 conn(0x7f9e5c13f780 0x7f9e5c13fb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:36970/0 (socket says 172.21.15.31:36970) 2023-10-06T02:31:04.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.480+0000 7f9e61b9a700 1 --2- 172.21.15.31:0/3265592896 >> v2:172.21.15.62:3300/0 conn(0x7f9e5c0c7fb0 0x7f9e5c15a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:04.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.481+0000 7f9e6239b700 1 -- 172.21.15.31:0/3265592896 >> v2:172.21.15.62:3300/0 conn(0x7f9e5c0c7fb0 msgr2=0x7f9e5c15a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:04.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.481+0000 7f9e6239b700 1 --2- 172.21.15.31:0/3265592896 >> v2:172.21.15.62:3300/0 conn(0x7f9e5c0c7fb0 0x7f9e5c15a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:04.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.481+0000 7f9e6239b700 1 -- 172.21.15.31:0/3265592896 >> v1:172.21.15.31:6789/0 conn(0x7f9e5c13f780 legacy=0x7f9e5c13fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:04.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.481+0000 7f9e6239b700 1 -- 172.21.15.31:0/3265592896 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e5c06b570 con 0x7f9e5c0c7be0 2023-10-06T02:31:04.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.481+0000 7f9e6239b700 1 --2- 172.21.15.31:0/3265592896 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e5c0c7be0 0x7f9e5c06abd0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c0aec50 tx=0x7f9e5800b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbfb701ca10f9a9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:04.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.481+0000 7f9e61399700 1 -- 172.21.15.31:0/3265592896 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e5800dc50 con 0x7f9e5c0c7be0 2023-10-06T02:31:04.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.481+0000 7f9e61399700 1 -- 172.21.15.31:0/3265592896 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e5800ddb0 con 0x7f9e5c0c7be0 2023-10-06T02:31:04.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.481+0000 7f9e61399700 1 -- 172.21.15.31:0/3265592896 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e580196d0 con 0x7f9e5c0c7be0 2023-10-06T02:31:04.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.482+0000 7f9e6339d700 1 -- 172.21.15.31:0/3265592896 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e5c0c7be0 msgr2=0x7f9e5c06abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:04.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.482+0000 7f9e6339d700 1 --2- 172.21.15.31:0/3265592896 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e5c0c7be0 0x7f9e5c06abd0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c0aec50 tx=0x7f9e5800b4f0 comp rx=0 tx=0).stop 2023-10-06T02:31:04.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.482+0000 7f9e6339d700 1 -- 172.21.15.31:0/3265592896 shutdown_connections 2023-10-06T02:31:04.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.482+0000 7f9e6339d700 1 --2- 172.21.15.31:0/3265592896 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e5c0c7be0 0x7f9e5c06abd0 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:04.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.482+0000 7f9e6339d700 1 --2- 172.21.15.31:0/3265592896 >> v2:172.21.15.62:3300/0 conn(0x7f9e5c0c7fb0 0x7f9e5c15a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:04.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.482+0000 7f9e6339d700 1 -- 172.21.15.31:0/3265592896 >> 172.21.15.31:0/3265592896 conn(0x7f9e5c06d110 msgr2=0x7f9e5c0cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:04.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.482+0000 7f9e6339d700 1 -- 172.21.15.31:0/3265592896 shutdown_connections 2023-10-06T02:31:04.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.482+0000 7f9e6339d700 1 -- 172.21.15.31:0/3265592896 wait complete. 2023-10-06T02:31:04.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.483+0000 7f9e6339d700 1 Processor -- start 2023-10-06T02:31:04.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.483+0000 7f9e6339d700 1 -- start start 2023-10-06T02:31:04.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.484+0000 7f9e6339d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9e5c0c7be0 0x7f9e5c0c4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:04.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.484+0000 7f9e6339d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e5c0c7fb0 0x7f9e5c156a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:04.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.484+0000 7f9e6339d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e5c13f780 0x7f9e5c13c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:04.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.484+0000 7f9e6339d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5c157b00 con 0x7f9e5c0c7fb0 2023-10-06T02:31:04.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.484+0000 7f9e6339d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5c158060 con 0x7f9e5c13f780 2023-10-06T02:31:04.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.484+0000 7f9e6339d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5c156fc0 con 0x7f9e5c0c7be0 2023-10-06T02:31:04.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.485+0000 7f9e61b9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e5c0c7fb0 0x7f9e5c156a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:04.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.485+0000 7f9e61b9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e5c0c7fb0 0x7f9e5c156a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36574/0 (socket says 172.21.15.31:36574) 2023-10-06T02:31:04.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.485+0000 7f9e61b9a700 1 -- 172.21.15.31:0/633613032 learned_addr learned my addr 172.21.15.31:0/633613032 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:04.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.485+0000 7f9e62b9c700 1 --2- 172.21.15.31:0/633613032 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e5c13f780 0x7f9e5c13c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:04.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.485+0000 7f9e6239b700 1 --2- 172.21.15.31:0/633613032 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9e5c0c7be0 0x7f9e5c0c4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:04.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.485+0000 7f9e61b9a700 1 -- 172.21.15.31:0/633613032 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9e5c0c7be0 msgr2=0x7f9e5c0c4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:04.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.485+0000 7f9e61b9a700 1 --2- 172.21.15.31:0/633613032 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9e5c0c7be0 0x7f9e5c0c4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:04.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.485+0000 7f9e61b9a700 1 -- 172.21.15.31:0/633613032 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e5c13f780 msgr2=0x7f9e5c13c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:04.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.485+0000 7f9e61b9a700 1 --2- 172.21.15.31:0/633613032 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e5c13f780 0x7f9e5c13c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:04.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.485+0000 7f9e61b9a700 1 -- 172.21.15.31:0/633613032 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e5800b790 con 0x7f9e5c0c7fb0 2023-10-06T02:31:04.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.485+0000 7f9e61b9a700 1 --2- 172.21.15.31:0/633613032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e5c0c7fb0 0x7f9e5c156a80 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c00d890 tx=0x7f9e4c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:04.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.486+0000 7f9e527fc700 1 -- 172.21.15.31:0/633613032 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c0125f0 con 0x7f9e5c0c7fb0 2023-10-06T02:31:04.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.486+0000 7f9e527fc700 1 -- 172.21.15.31:0/633613032 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e4c012750 con 0x7f9e5c0c7fb0 2023-10-06T02:31:04.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.486+0000 7f9e6339d700 1 -- 172.21.15.31:0/633613032 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e5c157260 con 0x7f9e5c0c7fb0 2023-10-06T02:31:04.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.486+0000 7f9e527fc700 1 -- 172.21.15.31:0/633613032 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c012af0 con 0x7f9e5c0c7fb0 2023-10-06T02:31:04.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.486+0000 7f9e6339d700 1 -- 172.21.15.31:0/633613032 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e5c157820 con 0x7f9e5c0c7fb0 2023-10-06T02:31:04.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.487+0000 7f9e6339d700 1 -- 172.21.15.31:0/633613032 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e5c157e30 con 0x7f9e5c0c7fb0 2023-10-06T02:31:04.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.488+0000 7f9e527fc700 1 -- 172.21.15.31:0/633613032 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) v1 ==== 85301+0+0 (secure 0 0 0) 0x7f9e4c01e460 con 0x7f9e5c0c7fb0 2023-10-06T02:31:04.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.489+0000 7f9e527fc700 1 --2- 172.21.15.31:0/633613032 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9e40067560 0x7f9e40069a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:04.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.489+0000 7f9e527fc700 1 -- 172.21.15.31:0/633613032 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f9e4c0857d0 con 0x7f9e5c0c7fb0 2023-10-06T02:31:04.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.489+0000 7f9e6239b700 1 --2- 172.21.15.31:0/633613032 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9e40067560 0x7f9e40069a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:04.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.490+0000 7f9e6239b700 1 --2- 172.21.15.31:0/633613032 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9e40067560 0x7f9e40069a10 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c0aec50 tx=0x7f9e5801b000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:04.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.493+0000 7f9e527fc700 1 -- 172.21.15.31:0/633613032 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9e4c051f40 con 0x7f9e5c0c7fb0 2023-10-06T02:31:04.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.730+0000 7f9e6339d700 1 -- 172.21.15.31:0/633613032 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000652135", "target": ["mon-mgr", ""]}) v1 -- 0x7f9e5c1581d0 con 0x7f9e40067560 2023-10-06T02:31:04.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.736+0000 7f9e527fc700 1 -- 172.21.15.31:0/633613032 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+45 (secure 0 0 0) 0x7f9e5c1581d0 con 0x7f9e40067560 2023-10-06T02:31:04.736 INFO:teuthology.orchestra.run.smithi031.stdout:/volumes/_nogroup/subvolume_0000000000652135 2023-10-06T02:31:04.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.738+0000 7f9e6339d700 1 -- 172.21.15.31:0/633613032 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9e40067560 msgr2=0x7f9e40069a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:04.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.738+0000 7f9e6339d700 1 --2- 172.21.15.31:0/633613032 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9e40067560 0x7f9e40069a10 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c0aec50 tx=0x7f9e5801b000 comp rx=0 tx=0).stop 2023-10-06T02:31:04.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.738+0000 7f9e6339d700 1 -- 172.21.15.31:0/633613032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e5c0c7fb0 msgr2=0x7f9e5c156a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:04.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.738+0000 7f9e6339d700 1 --2- 172.21.15.31:0/633613032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e5c0c7fb0 0x7f9e5c156a80 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c00d890 tx=0x7f9e4c008040 comp rx=0 tx=0).stop 2023-10-06T02:31:04.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.738+0000 7f9e6339d700 1 -- 172.21.15.31:0/633613032 shutdown_connections 2023-10-06T02:31:04.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.738+0000 7f9e6339d700 1 --2- 172.21.15.31:0/633613032 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e5c13f780 0x7f9e5c13c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:04.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.738+0000 7f9e6339d700 1 --2- 172.21.15.31:0/633613032 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9e40067560 0x7f9e40069a10 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:04.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.738+0000 7f9e6339d700 1 --2- 172.21.15.31:0/633613032 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9e5c0c7be0 0x7f9e5c0c4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:04.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.738+0000 7f9e6339d700 1 --2- 172.21.15.31:0/633613032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e5c0c7fb0 0x7f9e5c156a80 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:04.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.738+0000 7f9e6339d700 1 -- 172.21.15.31:0/633613032 >> 172.21.15.31:0/633613032 conn(0x7f9e5c06d110 msgr2=0x7f9e5c13e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:04.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.738+0000 7f9e6339d700 1 -- 172.21.15.31:0/633613032 shutdown_connections 2023-10-06T02:31:04.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:04.738+0000 7f9e6339d700 1 -- 172.21.15.31:0/633613032 wait complete. 2023-10-06T02:31:04.756 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:31:04.756 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_legacy/5a83dca7143a6f6599dd0a650ae10a62.meta) 2023-10-06T02:31:04.825 INFO:teuthology.orchestra.run.smithi031.stdout:[GLOBAL] 2023-10-06T02:31:04.912 INFO:teuthology.orchestra.run.smithi031.stdout:version = 1 2023-10-06T02:31:04.912 INFO:teuthology.orchestra.run.smithi031.stdout:type = subvolume 2023-10-06T02:31:04.913 INFO:teuthology.orchestra.run.smithi031.stdout:path = /volumes/_nogroup/subvolume_0000000000652135 2023-10-06T02:31:04.913 INFO:teuthology.orchestra.run.smithi031.stdout:state = complete 2023-10-06T02:31:04.913 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:31:04.914 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000652136 group_0000000000911578 2023-10-06T02:31:05.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.106+0000 7f01ff59e700 1 Processor -- start 2023-10-06T02:31:05.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.106+0000 7f01ff59e700 1 -- start start 2023-10-06T02:31:05.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.107+0000 7f01ff59e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f02000c8200 0x7f02000c1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:05.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.107+0000 7f01ff59e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f02000c2a90 0x7f0200156aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:05.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.107+0000 7f01ff59e700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f02000c85d0 0x7f02000cc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:05.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.107+0000 7f01ff59e700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02001579b0 con 0x7f02000c8200 2023-10-06T02:31:05.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.107+0000 7f01ff59e700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02000c1390 con 0x7f02000c85d0 2023-10-06T02:31:05.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.107+0000 7f01ff59e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02001573f0 con 0x7f02000c2a90 2023-10-06T02:31:05.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.107+0000 7f01fe59c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f02000c8200 0x7f02000c1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:05.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.108+0000 7f01fe59c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f02000c8200 0x7f02000c1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36582/0 (socket says 172.21.15.31:36582) 2023-10-06T02:31:05.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.108+0000 7f01fe59c700 1 -- 172.21.15.31:0/871246414 learned_addr learned my addr 172.21.15.31:0/871246414 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:05.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.108+0000 7f01fdd9b700 1 --2- 172.21.15.31:0/871246414 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f02000c2a90 0x7f0200156aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:05.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.108+0000 7f01fed9d700 1 --2- 172.21.15.31:0/871246414 >> v2:172.21.15.62:3300/0 conn(0x7f02000c85d0 0x7f02000cc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:05.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.108+0000 7f01fe59c700 1 -- 172.21.15.31:0/871246414 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f02000c2a90 msgr2=0x7f0200156aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:05.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.108+0000 7f01fe59c700 1 --2- 172.21.15.31:0/871246414 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f02000c2a90 0x7f0200156aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:05.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.108+0000 7f01fe59c700 1 -- 172.21.15.31:0/871246414 >> v2:172.21.15.62:3300/0 conn(0x7f02000c85d0 msgr2=0x7f02000cc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:05.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.108+0000 7f01fe59c700 1 --2- 172.21.15.31:0/871246414 >> v2:172.21.15.62:3300/0 conn(0x7f02000c85d0 0x7f02000cc2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:05.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.108+0000 7f01fe59c700 1 -- 172.21.15.31:0/871246414 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02001576d0 con 0x7f02000c8200 2023-10-06T02:31:05.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.108+0000 7f01fe59c700 1 --2- 172.21.15.31:0/871246414 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02000c8200 0x7f02000c1820 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f020006d5e0 tx=0x7f01f0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3d0559e560c4513 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:05.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.108+0000 7f01fd59a700 1 -- 172.21.15.31:0/871246414 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f000b860 con 0x7f02000c8200 2023-10-06T02:31:05.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.109+0000 7f01fd59a700 1 -- 172.21.15.31:0/871246414 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01f000b9c0 con 0x7f02000c8200 2023-10-06T02:31:05.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.109+0000 7f01fd59a700 1 -- 172.21.15.31:0/871246414 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f000bd60 con 0x7f02000c8200 2023-10-06T02:31:05.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.109+0000 7f01ff59e700 1 -- 172.21.15.31:0/871246414 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02000c8200 msgr2=0x7f02000c1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:05.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.109+0000 7f01ff59e700 1 --2- 172.21.15.31:0/871246414 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02000c8200 0x7f02000c1820 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f020006d5e0 tx=0x7f01f0009250 comp rx=0 tx=0).stop 2023-10-06T02:31:05.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.109+0000 7f01ff59e700 1 -- 172.21.15.31:0/871246414 shutdown_connections 2023-10-06T02:31:05.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.109+0000 7f01ff59e700 1 --2- 172.21.15.31:0/871246414 >> v2:172.21.15.62:3300/0 conn(0x7f02000c85d0 0x7f02000cc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:05.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.109+0000 7f01ff59e700 1 --2- 172.21.15.31:0/871246414 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02000c8200 0x7f02000c1820 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:05.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.109+0000 7f01ff59e700 1 --2- 172.21.15.31:0/871246414 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f02000c2a90 0x7f0200156aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:05.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.109+0000 7f01ff59e700 1 -- 172.21.15.31:0/871246414 >> 172.21.15.31:0/871246414 conn(0x7f020006c6a0 msgr2=0x7f020006caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:05.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.109+0000 7f01ff59e700 1 -- 172.21.15.31:0/871246414 shutdown_connections 2023-10-06T02:31:05.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.109+0000 7f01ff59e700 1 -- 172.21.15.31:0/871246414 wait complete. 2023-10-06T02:31:05.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.111+0000 7f01ff59e700 1 Processor -- start 2023-10-06T02:31:05.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.111+0000 7f01ff59e700 1 -- start start 2023-10-06T02:31:05.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.112+0000 7f01ff59e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02000c2a90 0x7f02000c3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:05.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.112+0000 7f01ff59e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02000c8200 0x7f020015b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:05.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.112+0000 7f01ff59e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f02000c85d0 0x7f020013b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:05.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.112+0000 7f01ff59e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f020015ca20 con 0x7f02000c2a90 2023-10-06T02:31:05.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.112+0000 7f01ff59e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f020015c460 con 0x7f02000c8200 2023-10-06T02:31:05.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.112+0000 7f01ff59e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f020015c740 con 0x7f02000c85d0 2023-10-06T02:31:05.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.112+0000 7f01fe59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02000c2a90 0x7f02000c3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:05.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.112+0000 7f01fe59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02000c2a90 0x7f02000c3450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36590/0 (socket says 172.21.15.31:36590) 2023-10-06T02:31:05.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.112+0000 7f01fe59c700 1 -- 172.21.15.31:0/2206877629 learned_addr learned my addr 172.21.15.31:0/2206877629 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:05.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.112+0000 7f01fdd9b700 1 --2- 172.21.15.31:0/2206877629 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02000c8200 0x7f020015b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:05.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.112+0000 7f01fed9d700 1 --2- 172.21.15.31:0/2206877629 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f02000c85d0 0x7f020013b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:05.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.112+0000 7f01fe59c700 1 -- 172.21.15.31:0/2206877629 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f02000c85d0 msgr2=0x7f020013b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:05.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.112+0000 7f01fe59c700 1 --2- 172.21.15.31:0/2206877629 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f02000c85d0 0x7f020013b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:05.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.112+0000 7f01fe59c700 1 -- 172.21.15.31:0/2206877629 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02000c8200 msgr2=0x7f020015b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:05.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.112+0000 7f01fe59c700 1 --2- 172.21.15.31:0/2206877629 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02000c8200 0x7f020015b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:05.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.112+0000 7f01fe59c700 1 -- 172.21.15.31:0/2206877629 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01f00094f0 con 0x7f02000c2a90 2023-10-06T02:31:05.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.112+0000 7f01fed9d700 1 --2- 172.21.15.31:0/2206877629 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f02000c85d0 0x7f020013b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:31:05.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.113+0000 7f01fe59c700 1 --2- 172.21.15.31:0/2206877629 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02000c2a90 0x7f02000c3450 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f02000697c0 tx=0x7f01f0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:05.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.113+0000 7f01ee7fc700 1 -- 172.21.15.31:0/2206877629 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f0024440 con 0x7f02000c2a90 2023-10-06T02:31:05.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.113+0000 7f01ee7fc700 1 -- 172.21.15.31:0/2206877629 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01f00245a0 con 0x7f02000c2a90 2023-10-06T02:31:05.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.113+0000 7f01ee7fc700 1 -- 172.21.15.31:0/2206877629 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f0024960 con 0x7f02000c2a90 2023-10-06T02:31:05.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.113+0000 7f01ff59e700 1 -- 172.21.15.31:0/2206877629 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02000acbe0 con 0x7f02000c2a90 2023-10-06T02:31:05.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.113+0000 7f01ff59e700 1 -- 172.21.15.31:0/2206877629 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02000ac3d0 con 0x7f02000c2a90 2023-10-06T02:31:05.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.114+0000 7f01ff59e700 1 -- 172.21.15.31:0/2206877629 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f020004c3c0 con 0x7f02000c2a90 2023-10-06T02:31:05.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.115+0000 7f01ee7fc700 1 -- 172.21.15.31:0/2206877629 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) v1 ==== 85301+0+0 (secure 0 0 0) 0x7f01f001a420 con 0x7f02000c2a90 2023-10-06T02:31:05.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.116+0000 7f01ee7fc700 1 --2- 172.21.15.31:0/2206877629 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f01e0067560 0x7f01e0069a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:05.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.116+0000 7f01ee7fc700 1 -- 172.21.15.31:0/2206877629 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f01f0087120 con 0x7f02000c2a90 2023-10-06T02:31:05.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.117+0000 7f01fdd9b700 1 --2- 172.21.15.31:0/2206877629 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f01e0067560 0x7f01e0069a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:05.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.117+0000 7f01fdd9b700 1 --2- 172.21.15.31:0/2206877629 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f01e0067560 0x7f01e0069a10 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f020014dfe0 tx=0x7f01f4008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:05.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.121+0000 7f01ee7fc700 1 -- 172.21.15.31:0/2206877629 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f01f0053890 con 0x7f02000c2a90 2023-10-06T02:31:05.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.359+0000 7f01ff59e700 1 -- 172.21.15.31:0/2206877629 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000652136", "group_name": "group_0000000000911578", "target": ["mon-mgr", ""]}) v1 -- 0x7f02000cc000 con 0x7f01e0067560 2023-10-06T02:31:05.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.366+0000 7f01ee7fc700 1 -- 172.21.15.31:0/2206877629 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+59 (secure 0 0 0) 0x7f02000cc000 con 0x7f01e0067560 2023-10-06T02:31:05.367 INFO:teuthology.orchestra.run.smithi031.stdout:/volumes/group_0000000000911578/subvolume_0000000000652136 2023-10-06T02:31:05.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.369+0000 7f01ff59e700 1 -- 172.21.15.31:0/2206877629 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f01e0067560 msgr2=0x7f01e0069a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:05.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.369+0000 7f01ff59e700 1 --2- 172.21.15.31:0/2206877629 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f01e0067560 0x7f01e0069a10 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f020014dfe0 tx=0x7f01f4008040 comp rx=0 tx=0).stop 2023-10-06T02:31:05.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.369+0000 7f01ff59e700 1 -- 172.21.15.31:0/2206877629 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02000c2a90 msgr2=0x7f02000c3450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:05.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.369+0000 7f01ff59e700 1 --2- 172.21.15.31:0/2206877629 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02000c2a90 0x7f02000c3450 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f02000697c0 tx=0x7f01f0017000 comp rx=0 tx=0).stop 2023-10-06T02:31:05.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.369+0000 7f01ff59e700 1 -- 172.21.15.31:0/2206877629 shutdown_connections 2023-10-06T02:31:05.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.369+0000 7f01ff59e700 1 --2- 172.21.15.31:0/2206877629 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f01e0067560 0x7f01e0069a10 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:05.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.369+0000 7f01ff59e700 1 --2- 172.21.15.31:0/2206877629 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f02000c85d0 0x7f020013b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:05.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.369+0000 7f01ff59e700 1 --2- 172.21.15.31:0/2206877629 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02000c2a90 0x7f02000c3450 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:05.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.369+0000 7f01ff59e700 1 --2- 172.21.15.31:0/2206877629 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02000c8200 0x7f020015b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:05.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.369+0000 7f01ff59e700 1 -- 172.21.15.31:0/2206877629 >> 172.21.15.31:0/2206877629 conn(0x7f020006c6a0 msgr2=0x7f020014fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:05.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.369+0000 7f01ff59e700 1 -- 172.21.15.31:0/2206877629 shutdown_connections 2023-10-06T02:31:05.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.369+0000 7f01ff59e700 1 -- 172.21.15.31:0/2206877629 wait complete. 2023-10-06T02:31:05.387 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:31:05.387 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_legacy/277ad30f959f68688bf8872ef0650e97.meta) 2023-10-06T02:31:05.453 INFO:teuthology.orchestra.run.smithi031.stdout:[GLOBAL] 2023-10-06T02:31:05.453 INFO:teuthology.orchestra.run.smithi031.stdout:version = 1 2023-10-06T02:31:05.453 INFO:teuthology.orchestra.run.smithi031.stdout:type = subvolume 2023-10-06T02:31:05.453 INFO:teuthology.orchestra.run.smithi031.stdout:path = /volumes/group_0000000000911578/subvolume_0000000000652136 2023-10-06T02:31:05.453 INFO:teuthology.orchestra.run.smithi031.stdout:state = complete 2023-10-06T02:31:05.454 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:31:05.456 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000652135 2023-10-06T02:31:05.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.645+0000 7f1656f9d700 1 Processor -- start 2023-10-06T02:31:05.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.645+0000 7f1656f9d700 1 -- start start 2023-10-06T02:31:05.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.647+0000 7f1656f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1650142c00 0x7f165006b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:05.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.647+0000 7f1656f9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f16500c7f90 0x7f165006c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:05.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.647+0000 7f1656f9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16500ca480 con 0x7f165006bd20 2023-10-06T02:31:05.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.647+0000 7f1656f9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f165015e240 con 0x7f16500c7f90 2023-10-06T02:31:05.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.647+0000 7f1656f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f165006b350 con 0x7f1650142c00 2023-10-06T02:31:05.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.647+0000 7f1655f9b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1650142c00 0x7f165006b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:05.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.647+0000 7f165679c700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f16500c7f90 0x7f165006c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:05.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.647+0000 7f165579a700 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f165006bd20 0x7f16500c7880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:53494/0 (socket says 172.21.15.31:53494) 2023-10-06T02:31:05.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.647+0000 7f165579a700 1 -- 172.21.15.31:0/3442093624 learned_addr learned my addr 172.21.15.31:0/3442093624 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:31:05.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.648+0000 7f1654f99700 1 -- 172.21.15.31:0/3442093624 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1790759598 0 0) 0x7f16500ca480 con 0x7f165006bd20 2023-10-06T02:31:05.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.648+0000 7f1654f99700 1 -- 172.21.15.31:0/3442093624 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f163c003570 con 0x7f165006bd20 2023-10-06T02:31:05.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.648+0000 7f165679c700 1 -- 172.21.15.31:0/3442093624 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1650142c00 msgr2=0x7f165006b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:05.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.648+0000 7f165679c700 1 --2- 172.21.15.31:0/3442093624 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1650142c00 0x7f165006b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:05.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.648+0000 7f165679c700 1 -- 172.21.15.31:0/3442093624 >> v1:172.21.15.62:6789/0 conn(0x7f165006bd20 legacy=0x7f16500c7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:05.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.648+0000 7f165679c700 1 -- 172.21.15.31:0/3442093624 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f165015df60 con 0x7f16500c7f90 2023-10-06T02:31:05.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.648+0000 7f1655f9b700 1 --2- 172.21.15.31:0/3442093624 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1650142c00 0x7f165006b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:31:05.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.649+0000 7f165679c700 1 --2- 172.21.15.31:0/3442093624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16500c7f90 0x7f165006c6a0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f1648009e90 tx=0x7f1648009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1fba37b9768278b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:05.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.649+0000 7f1654f99700 1 -- 172.21.15.31:0/3442093624 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f164800e070 con 0x7f16500c7f90 2023-10-06T02:31:05.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.649+0000 7f1654f99700 1 -- 172.21.15.31:0/3442093624 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f164800bb40 con 0x7f16500c7f90 2023-10-06T02:31:05.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.649+0000 7f1654f99700 1 -- 172.21.15.31:0/3442093624 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1648012430 con 0x7f16500c7f90 2023-10-06T02:31:05.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.649+0000 7f1656f9d700 1 -- 172.21.15.31:0/3442093624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16500c7f90 msgr2=0x7f165006c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:05.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.649+0000 7f1656f9d700 1 --2- 172.21.15.31:0/3442093624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16500c7f90 0x7f165006c6a0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f1648009e90 tx=0x7f1648009780 comp rx=0 tx=0).stop 2023-10-06T02:31:05.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.649+0000 7f1656f9d700 1 -- 172.21.15.31:0/3442093624 shutdown_connections 2023-10-06T02:31:05.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.649+0000 7f1656f9d700 1 --2- 172.21.15.31:0/3442093624 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1650142c00 0x7f165006b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:05.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.649+0000 7f1656f9d700 1 --2- 172.21.15.31:0/3442093624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16500c7f90 0x7f165006c6a0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:05.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.649+0000 7f1656f9d700 1 -- 172.21.15.31:0/3442093624 >> 172.21.15.31:0/3442093624 conn(0x7f16500675a0 msgr2=0x7f16500679a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:05.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.650+0000 7f1656f9d700 1 -- 172.21.15.31:0/3442093624 shutdown_connections 2023-10-06T02:31:05.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.650+0000 7f1656f9d700 1 -- 172.21.15.31:0/3442093624 wait complete. 2023-10-06T02:31:05.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.651+0000 7f1656f9d700 1 Processor -- start 2023-10-06T02:31:05.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.651+0000 7f1656f9d700 1 -- start start 2023-10-06T02:31:05.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.652+0000 7f1656f9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f165006bd20 0x7f1650171a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:05.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.652+0000 7f1656f9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16500c7f90 0x7f16500b2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:05.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.652+0000 7f1656f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1650142c00 0x7f16500b1290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:05.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.652+0000 7f1656f9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16500b17d0 con 0x7f16500c7f90 2023-10-06T02:31:05.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.652+0000 7f1656f9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16500b2610 con 0x7f165006bd20 2023-10-06T02:31:05.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.652+0000 7f1656f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16500b19e0 con 0x7f1650142c00 2023-10-06T02:31:05.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.652+0000 7f165579a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16500c7f90 0x7f16500b2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:05.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.652+0000 7f165579a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16500c7f90 0x7f16500b2aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36604/0 (socket says 172.21.15.31:36604) 2023-10-06T02:31:05.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.652+0000 7f165579a700 1 -- 172.21.15.31:0/1547746647 learned_addr learned my addr 172.21.15.31:0/1547746647 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:05.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.653+0000 7f165679c700 1 --2- 172.21.15.31:0/1547746647 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1650142c00 0x7f16500b1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:05.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.653+0000 7f1655f9b700 1 --2- 172.21.15.31:0/1547746647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f165006bd20 0x7f1650171a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:05.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.653+0000 7f165579a700 1 -- 172.21.15.31:0/1547746647 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1650142c00 msgr2=0x7f16500b1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:05.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.653+0000 7f165579a700 1 --2- 172.21.15.31:0/1547746647 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1650142c00 0x7f16500b1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:05.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.653+0000 7f165579a700 1 -- 172.21.15.31:0/1547746647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f165006bd20 msgr2=0x7f1650171a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:05.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.653+0000 7f165579a700 1 --2- 172.21.15.31:0/1547746647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f165006bd20 0x7f1650171a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:05.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.653+0000 7f165579a700 1 -- 172.21.15.31:0/1547746647 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f164c009420 con 0x7f16500c7f90 2023-10-06T02:31:05.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.653+0000 7f165579a700 1 --2- 172.21.15.31:0/1547746647 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16500c7f90 0x7f16500b2aa0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f164000c7c0 tx=0x7f1640007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:05.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.653+0000 7f1645ffb700 1 -- 172.21.15.31:0/1547746647 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1640005cd0 con 0x7f16500c7f90 2023-10-06T02:31:05.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.653+0000 7f1645ffb700 1 -- 172.21.15.31:0/1547746647 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1640005e30 con 0x7f16500c7f90 2023-10-06T02:31:05.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.653+0000 7f1645ffb700 1 -- 172.21.15.31:0/1547746647 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1640007320 con 0x7f16500c7f90 2023-10-06T02:31:05.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.653+0000 7f1656f9d700 1 -- 172.21.15.31:0/1547746647 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1648009a20 con 0x7f16500c7f90 2023-10-06T02:31:05.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.654+0000 7f1656f9d700 1 -- 172.21.15.31:0/1547746647 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16500b1fc0 con 0x7f16500c7f90 2023-10-06T02:31:05.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.655+0000 7f1656f9d700 1 -- 172.21.15.31:0/1547746647 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f165004c3c0 con 0x7f16500c7f90 2023-10-06T02:31:05.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.655+0000 7f1645ffb700 1 -- 172.21.15.31:0/1547746647 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) v1 ==== 85301+0+0 (secure 0 0 0) 0x7f164001b3b0 con 0x7f16500c7f90 2023-10-06T02:31:05.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.656+0000 7f1645ffb700 1 --2- 172.21.15.31:0/1547746647 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f1634067560 0x7f1634069a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:05.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.656+0000 7f1645ffb700 1 -- 172.21.15.31:0/1547746647 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f1640084800 con 0x7f16500c7f90 2023-10-06T02:31:05.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.657+0000 7f1655f9b700 1 --2- 172.21.15.31:0/1547746647 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f1634067560 0x7f1634069a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:05.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.658+0000 7f1655f9b700 1 --2- 172.21.15.31:0/1547746647 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f1634067560 0x7f1634069a10 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f164c009ea0 tx=0x7f164c0091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:05.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.660+0000 7f1645ffb700 1 -- 172.21.15.31:0/1547746647 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1640051070 con 0x7f16500c7f90 2023-10-06T02:31:05.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.894+0000 7f1656f9d700 1 -- 172.21.15.31:0/1547746647 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000652135", "target": ["mon-mgr", ""]}) v1 -- 0x7f165006ad00 con 0x7f1634067560 2023-10-06T02:31:05.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.914+0000 7f1645ffb700 1 -- 172.21.15.31:0/1547746647 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f165006ad00 con 0x7f1634067560 2023-10-06T02:31:05.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.917+0000 7f1656f9d700 1 -- 172.21.15.31:0/1547746647 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f1634067560 msgr2=0x7f1634069a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:05.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.917+0000 7f1656f9d700 1 --2- 172.21.15.31:0/1547746647 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f1634067560 0x7f1634069a10 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f164c009ea0 tx=0x7f164c0091a0 comp rx=0 tx=0).stop 2023-10-06T02:31:05.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.917+0000 7f1656f9d700 1 -- 172.21.15.31:0/1547746647 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16500c7f90 msgr2=0x7f16500b2aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:05.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.917+0000 7f1656f9d700 1 --2- 172.21.15.31:0/1547746647 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16500c7f90 0x7f16500b2aa0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f164000c7c0 tx=0x7f1640007e00 comp rx=0 tx=0).stop 2023-10-06T02:31:05.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.917+0000 7f1656f9d700 1 -- 172.21.15.31:0/1547746647 shutdown_connections 2023-10-06T02:31:05.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.917+0000 7f1656f9d700 1 --2- 172.21.15.31:0/1547746647 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f1634067560 0x7f1634069a10 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:05.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.917+0000 7f1656f9d700 1 --2- 172.21.15.31:0/1547746647 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1650142c00 0x7f16500b1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:05.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.917+0000 7f1656f9d700 1 --2- 172.21.15.31:0/1547746647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f165006bd20 0x7f1650171a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:05.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.917+0000 7f1656f9d700 1 --2- 172.21.15.31:0/1547746647 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16500c7f90 0x7f16500b2aa0 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:05.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.917+0000 7f1656f9d700 1 -- 172.21.15.31:0/1547746647 >> 172.21.15.31:0/1547746647 conn(0x7f16500675a0 msgr2=0x7f16500c90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:05.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.917+0000 7f1656f9d700 1 -- 172.21.15.31:0/1547746647 shutdown_connections 2023-10-06T02:31:05.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:05.917+0000 7f1656f9d700 1 -- 172.21.15.31:0/1547746647 wait complete. 2023-10-06T02:31:05.922 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:31:05.921+0000 7f6567381700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:31:05.923 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:31:05.921+0000 7f6567381700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:31:05.923 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:31:05.921+0000 7f6567381700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:31:05.923 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:31:05.921+0000 7f6567381700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:31:05.923 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:31:05.921+0000 7f6567381700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:31:05.937 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000652136 group_0000000000911578 2023-10-06T02:31:05.942 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:31:05.941+0000 7f6567b82700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:31:05.942 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:31:05.941+0000 7f6567b82700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:31:05.942 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:31:05.941+0000 7f6567b82700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:31:05.943 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:31:05.941+0000 7f6567b82700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:31:05.943 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:31:05.941+0000 7f6567b82700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:31:06.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.128+0000 7f8e47fff700 1 Processor -- start 2023-10-06T02:31:06.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.129+0000 7f8e47fff700 1 -- start start 2023-10-06T02:31:06.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.130+0000 7f8e47fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8e480cb4b0 0x7f8e480cb880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:06.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.130+0000 7f8e47fff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e480ceca0 con 0x7f8e480b32b0 2023-10-06T02:31:06.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.130+0000 7f8e47fff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e480c5290 con 0x7f8e480cff40 2023-10-06T02:31:06.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.130+0000 7f8e47fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e480c5e10 con 0x7f8e480cb4b0 2023-10-06T02:31:06.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.130+0000 7f8e467fc700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f8e480b32b0 0x7f8e480cf830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:36982/0 (socket says 172.21.15.31:36982) 2023-10-06T02:31:06.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.130+0000 7f8e467fc700 1 -- 172.21.15.31:0/1218005364 learned_addr learned my addr 172.21.15.31:0/1218005364 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:31:06.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.130+0000 7f8e46ffd700 1 --2- 172.21.15.31:0/1218005364 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8e480cb4b0 0x7f8e480cb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:06.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.130+0000 7f8e45ffb700 1 -- 172.21.15.31:0/1218005364 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2883753495 0 0) 0x7f8e480ceca0 con 0x7f8e480b32b0 2023-10-06T02:31:06.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.131+0000 7f8e45ffb700 1 -- 172.21.15.31:0/1218005364 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e30003570 con 0x7f8e480b32b0 2023-10-06T02:31:06.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.131+0000 7f8e45ffb700 1 -- 172.21.15.31:0/1218005364 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3342791099 0 0) 0x7f8e480c5290 con 0x7f8e480cff40 2023-10-06T02:31:06.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.131+0000 7f8e45ffb700 1 -- 172.21.15.31:0/1218005364 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e480ceca0 con 0x7f8e480cff40 2023-10-06T02:31:06.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.131+0000 7f8e46ffd700 1 -- 172.21.15.31:0/1218005364 >> v1:172.21.15.62:6789/0 conn(0x7f8e480cff40 legacy=0x7f8e480ccef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:06.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.131+0000 7f8e46ffd700 1 -- 172.21.15.31:0/1218005364 >> v1:172.21.15.31:6789/0 conn(0x7f8e480b32b0 legacy=0x7f8e480cf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:06.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.131+0000 7f8e46ffd700 1 -- 172.21.15.31:0/1218005364 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e480cbdc0 con 0x7f8e480cb4b0 2023-10-06T02:31:06.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.131+0000 7f8e46ffd700 1 --2- 172.21.15.31:0/1218005364 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8e480cb4b0 0x7f8e480cb880 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8e38009930 tx=0x7f8e38009120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=caba3643639b9a73 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:06.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.131+0000 7f8e45ffb700 1 -- 172.21.15.31:0/1218005364 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e38005c10 con 0x7f8e480cb4b0 2023-10-06T02:31:06.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.131+0000 7f8e45ffb700 1 -- 172.21.15.31:0/1218005364 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e38005d70 con 0x7f8e480cb4b0 2023-10-06T02:31:06.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.131+0000 7f8e45ffb700 1 -- 172.21.15.31:0/1218005364 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e38006260 con 0x7f8e480cb4b0 2023-10-06T02:31:06.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.132+0000 7f8e47fff700 1 -- 172.21.15.31:0/1218005364 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8e480cb4b0 msgr2=0x7f8e480cb880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:06.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.132+0000 7f8e47fff700 1 --2- 172.21.15.31:0/1218005364 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8e480cb4b0 0x7f8e480cb880 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8e38009930 tx=0x7f8e38009120 comp rx=0 tx=0).stop 2023-10-06T02:31:06.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.132+0000 7f8e47fff700 1 -- 172.21.15.31:0/1218005364 shutdown_connections 2023-10-06T02:31:06.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.132+0000 7f8e47fff700 1 --2- 172.21.15.31:0/1218005364 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8e480cb4b0 0x7f8e480cb880 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:06.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.132+0000 7f8e47fff700 1 -- 172.21.15.31:0/1218005364 >> 172.21.15.31:0/1218005364 conn(0x7f8e48099740 msgr2=0x7f8e48070b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:06.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.132+0000 7f8e47fff700 1 -- 172.21.15.31:0/1218005364 shutdown_connections 2023-10-06T02:31:06.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.132+0000 7f8e47fff700 1 -- 172.21.15.31:0/1218005364 wait complete. 2023-10-06T02:31:06.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.133+0000 7f8e47fff700 1 Processor -- start 2023-10-06T02:31:06.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.134+0000 7f8e47fff700 1 -- start start 2023-10-06T02:31:06.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.134+0000 7f8e47fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e480b32b0 0x7f8e4816cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:06.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.134+0000 7f8e47fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8e480cb4b0 0x7f8e48153360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:06.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.134+0000 7f8e47fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e480cff40 0x7f8e481538a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:06.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.135+0000 7f8e47fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4816ab90 con 0x7f8e480b32b0 2023-10-06T02:31:06.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.135+0000 7f8e47fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e48159050 con 0x7f8e480cff40 2023-10-06T02:31:06.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.135+0000 7f8e47fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e48169f80 con 0x7f8e480cb4b0 2023-10-06T02:31:06.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.135+0000 7f8e46ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e480b32b0 0x7f8e4816cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:06.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.135+0000 7f8e46ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e480b32b0 0x7f8e4816cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36608/0 (socket says 172.21.15.31:36608) 2023-10-06T02:31:06.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.135+0000 7f8e46ffd700 1 -- 172.21.15.31:0/2124479985 learned_addr learned my addr 172.21.15.31:0/2124479985 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:06.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.135+0000 7f8e477fe700 1 --2- 172.21.15.31:0/2124479985 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e480cff40 0x7f8e481538a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:06.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.135+0000 7f8e467fc700 1 --2- 172.21.15.31:0/2124479985 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8e480cb4b0 0x7f8e48153360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:06.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.135+0000 7f8e46ffd700 1 -- 172.21.15.31:0/2124479985 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8e480cb4b0 msgr2=0x7f8e48153360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:06.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.135+0000 7f8e46ffd700 1 --2- 172.21.15.31:0/2124479985 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8e480cb4b0 0x7f8e48153360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:06.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.135+0000 7f8e46ffd700 1 -- 172.21.15.31:0/2124479985 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e480cff40 msgr2=0x7f8e481538a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:06.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.135+0000 7f8e46ffd700 1 --2- 172.21.15.31:0/2124479985 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e480cff40 0x7f8e481538a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:06.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.135+0000 7f8e46ffd700 1 -- 172.21.15.31:0/2124479985 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e380093c0 con 0x7f8e480b32b0 2023-10-06T02:31:06.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.135+0000 7f8e467fc700 1 --2- 172.21.15.31:0/2124479985 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8e480cb4b0 0x7f8e48153360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:31:06.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.135+0000 7f8e477fe700 1 --2- 172.21.15.31:0/2124479985 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e480cff40 0x7f8e481538a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:31:06.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.135+0000 7f8e46ffd700 1 --2- 172.21.15.31:0/2124479985 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e480b32b0 0x7f8e4816cc80 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f8e38009850 tx=0x7f8e38013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:06.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.136+0000 7f8e2effd700 1 -- 172.21.15.31:0/2124479985 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e3801d910 con 0x7f8e480b32b0 2023-10-06T02:31:06.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.136+0000 7f8e2effd700 1 -- 172.21.15.31:0/2124479985 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e3801dad0 con 0x7f8e480b32b0 2023-10-06T02:31:06.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.136+0000 7f8e2effd700 1 -- 172.21.15.31:0/2124479985 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e3801de90 con 0x7f8e480b32b0 2023-10-06T02:31:06.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.136+0000 7f8e47fff700 1 -- 172.21.15.31:0/2124479985 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e4816a1f0 con 0x7f8e480b32b0 2023-10-06T02:31:06.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.136+0000 7f8e47fff700 1 -- 172.21.15.31:0/2124479985 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e4816a7b0 con 0x7f8e480b32b0 2023-10-06T02:31:06.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.137+0000 7f8e47fff700 1 -- 172.21.15.31:0/2124479985 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e4806d1c0 con 0x7f8e480b32b0 2023-10-06T02:31:06.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.138+0000 7f8e2effd700 1 -- 172.21.15.31:0/2124479985 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) v1 ==== 85301+0+0 (secure 0 0 0) 0x7f8e3801b690 con 0x7f8e480b32b0 2023-10-06T02:31:06.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.139+0000 7f8e2effd700 1 --2- 172.21.15.31:0/2124479985 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f8e34067560 0x7f8e34069a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:06.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.139+0000 7f8e2effd700 1 -- 172.21.15.31:0/2124479985 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f8e38087350 con 0x7f8e480b32b0 2023-10-06T02:31:06.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.139+0000 7f8e467fc700 1 --2- 172.21.15.31:0/2124479985 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f8e34067560 0x7f8e34069a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:06.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.140+0000 7f8e467fc700 1 --2- 172.21.15.31:0/2124479985 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f8e34067560 0x7f8e34069a10 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c007fd0 tx=0x7f8e3c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:06.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.143+0000 7f8e2effd700 1 -- 172.21.15.31:0/2124479985 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8e38053ac0 con 0x7f8e480b32b0 2023-10-06T02:31:06.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.377+0000 7f8e47fff700 1 -- 172.21.15.31:0/2124479985 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000652136", "group_name": "group_0000000000911578", "target": ["mon-mgr", ""]}) v1 -- 0x7f8e4816ae00 con 0x7f8e34067560 2023-10-06T02:31:06.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.397+0000 7f8e2effd700 1 -- 172.21.15.31:0/2124479985 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f8e4816ae00 con 0x7f8e34067560 2023-10-06T02:31:06.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.399+0000 7f8e47fff700 1 -- 172.21.15.31:0/2124479985 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f8e34067560 msgr2=0x7f8e34069a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:06.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.399+0000 7f8e47fff700 1 --2- 172.21.15.31:0/2124479985 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f8e34067560 0x7f8e34069a10 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c007fd0 tx=0x7f8e3c009040 comp rx=0 tx=0).stop 2023-10-06T02:31:06.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.399+0000 7f8e47fff700 1 -- 172.21.15.31:0/2124479985 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e480b32b0 msgr2=0x7f8e4816cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:06.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.399+0000 7f8e47fff700 1 --2- 172.21.15.31:0/2124479985 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e480b32b0 0x7f8e4816cc80 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f8e38009850 tx=0x7f8e38013000 comp rx=0 tx=0).stop 2023-10-06T02:31:06.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.400+0000 7f8e47fff700 1 -- 172.21.15.31:0/2124479985 shutdown_connections 2023-10-06T02:31:06.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.400+0000 7f8e47fff700 1 --2- 172.21.15.31:0/2124479985 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e480cff40 0x7f8e481538a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:06.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.400+0000 7f8e47fff700 1 --2- 172.21.15.31:0/2124479985 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e480b32b0 0x7f8e4816cc80 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:06.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.400+0000 7f8e47fff700 1 --2- 172.21.15.31:0/2124479985 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f8e34067560 0x7f8e34069a10 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:06.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.400+0000 7f8e47fff700 1 --2- 172.21.15.31:0/2124479985 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8e480cb4b0 0x7f8e48153360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:06.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.400+0000 7f8e47fff700 1 -- 172.21.15.31:0/2124479985 >> 172.21.15.31:0/2124479985 conn(0x7f8e48099740 msgr2=0x7f8e4806e450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:06.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.400+0000 7f8e47fff700 1 -- 172.21.15.31:0/2124479985 shutdown_connections 2023-10-06T02:31:06.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.400+0000 7f8e47fff700 1 -- 172.21.15.31:0/2124479985 wait complete. 2023-10-06T02:31:06.418 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:31:06.418 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2023-10-06T02:31:06.493 INFO:teuthology.orchestra.run.smithi031.stdout:2 2023-10-06T02:31:06.495 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm cephfs group_0000000000911578 2023-10-06T02:31:06.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.685+0000 7f989759e700 1 Processor -- start 2023-10-06T02:31:06.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.685+0000 7f989759e700 1 -- start start 2023-10-06T02:31:06.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.687+0000 7f989759e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9898159550 0x7f98980ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:06.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.687+0000 7f989759e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f989815dfd0 0x7f98980aedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:06.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.687+0000 7f989759e700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9898159920 0x7f98980ad090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:06.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.687+0000 7f989759e700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98980acc00 con 0x7f989815dfd0 2023-10-06T02:31:06.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.687+0000 7f989759e700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98980abff0 con 0x7f9898159920 2023-10-06T02:31:06.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.687+0000 7f989759e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98980ac2d0 con 0x7f9898159550 2023-10-06T02:31:06.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.687+0000 7f9895d9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f989815dfd0 0x7f98980aedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:06.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.687+0000 7f9895d9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f989815dfd0 0x7f98980aedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36616/0 (socket says 172.21.15.31:36616) 2023-10-06T02:31:06.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.687+0000 7f9895d9b700 1 -- 172.21.15.31:0/762331720 learned_addr learned my addr 172.21.15.31:0/762331720 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:06.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.687+0000 7f989659c700 1 --2- 172.21.15.31:0/762331720 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9898159550 0x7f98980ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:06.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.687+0000 7f9896d9d700 1 --2- 172.21.15.31:0/762331720 >> v2:172.21.15.62:3300/0 conn(0x7f9898159920 0x7f98980ad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:06.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.687+0000 7f9895d9b700 1 -- 172.21.15.31:0/762331720 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9898159550 msgr2=0x7f98980ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:06.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.687+0000 7f9895d9b700 1 --2- 172.21.15.31:0/762331720 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9898159550 0x7f98980ceac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:06.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.687+0000 7f9895d9b700 1 -- 172.21.15.31:0/762331720 >> v2:172.21.15.62:3300/0 conn(0x7f9898159920 msgr2=0x7f98980ad090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:06.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.687+0000 7f9895d9b700 1 --2- 172.21.15.31:0/762331720 >> v2:172.21.15.62:3300/0 conn(0x7f9898159920 0x7f98980ad090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:06.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.687+0000 7f9895d9b700 1 -- 172.21.15.31:0/762331720 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98980ac5b0 con 0x7f989815dfd0 2023-10-06T02:31:06.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.687+0000 7f989659c700 1 --2- 172.21.15.31:0/762331720 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9898159550 0x7f98980ceac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:31:06.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.687+0000 7f9896d9d700 1 --2- 172.21.15.31:0/762331720 >> v2:172.21.15.62:3300/0 conn(0x7f9898159920 0x7f98980ad090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:31:06.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.688+0000 7f9895d9b700 1 --2- 172.21.15.31:0/762331720 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f989815dfd0 0x7f98980aedb0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f989806c180 tx=0x7f988c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dab7f0485923ff4c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:06.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.688+0000 7f989559a700 1 -- 172.21.15.31:0/762331720 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f988c00b860 con 0x7f989815dfd0 2023-10-06T02:31:06.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.688+0000 7f989559a700 1 -- 172.21.15.31:0/762331720 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f988c00b9c0 con 0x7f989815dfd0 2023-10-06T02:31:06.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.688+0000 7f989559a700 1 -- 172.21.15.31:0/762331720 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f988c00bd60 con 0x7f989815dfd0 2023-10-06T02:31:06.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.688+0000 7f989759e700 1 -- 172.21.15.31:0/762331720 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f989815dfd0 msgr2=0x7f98980aedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:06.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.688+0000 7f989759e700 1 --2- 172.21.15.31:0/762331720 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f989815dfd0 0x7f98980aedb0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f989806c180 tx=0x7f988c009250 comp rx=0 tx=0).stop 2023-10-06T02:31:06.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.689+0000 7f989759e700 1 -- 172.21.15.31:0/762331720 shutdown_connections 2023-10-06T02:31:06.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.689+0000 7f989759e700 1 --2- 172.21.15.31:0/762331720 >> v2:172.21.15.62:3300/0 conn(0x7f9898159920 0x7f98980ad090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:06.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.689+0000 7f989759e700 1 --2- 172.21.15.31:0/762331720 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9898159550 0x7f98980ceac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:06.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.689+0000 7f989759e700 1 --2- 172.21.15.31:0/762331720 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f989815dfd0 0x7f98980aedb0 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:06.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.689+0000 7f989759e700 1 -- 172.21.15.31:0/762331720 >> 172.21.15.31:0/762331720 conn(0x7f989806ce80 msgr2=0x7f98980ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:06.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.689+0000 7f989759e700 1 -- 172.21.15.31:0/762331720 shutdown_connections 2023-10-06T02:31:06.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.689+0000 7f989759e700 1 -- 172.21.15.31:0/762331720 wait complete. 2023-10-06T02:31:06.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.690+0000 7f989759e700 1 Processor -- start 2023-10-06T02:31:06.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.690+0000 7f989759e700 1 -- start start 2023-10-06T02:31:06.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.691+0000 7f989759e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9898159550 0x7f989814cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:06.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.691+0000 7f989759e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9898159920 0x7f989814f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:06.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.691+0000 7f989759e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f989815dfd0 0x7f989814f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:06.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.691+0000 7f989759e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f989814d460 con 0x7f9898159920 2023-10-06T02:31:06.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.691+0000 7f989759e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98980b1410 con 0x7f989815dfd0 2023-10-06T02:31:06.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.691+0000 7f989759e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98980cd4a0 con 0x7f9898159550 2023-10-06T02:31:06.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.691+0000 7f9895d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9898159920 0x7f989814f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:06.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.691+0000 7f9895d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9898159920 0x7f989814f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36628/0 (socket says 172.21.15.31:36628) 2023-10-06T02:31:06.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.691+0000 7f9895d9b700 1 -- 172.21.15.31:0/1285130058 learned_addr learned my addr 172.21.15.31:0/1285130058 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:06.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.691+0000 7f989659c700 1 --2- 172.21.15.31:0/1285130058 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9898159550 0x7f989814cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:06.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.692+0000 7f9896d9d700 1 --2- 172.21.15.31:0/1285130058 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f989815dfd0 0x7f989814f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:06.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.692+0000 7f9895d9b700 1 -- 172.21.15.31:0/1285130058 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9898159550 msgr2=0x7f989814cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:06.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.692+0000 7f9895d9b700 1 --2- 172.21.15.31:0/1285130058 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9898159550 0x7f989814cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:06.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.692+0000 7f9895d9b700 1 -- 172.21.15.31:0/1285130058 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f989815dfd0 msgr2=0x7f989814f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:06.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.692+0000 7f9895d9b700 1 --2- 172.21.15.31:0/1285130058 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f989815dfd0 0x7f989814f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:06.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.692+0000 7f9895d9b700 1 -- 172.21.15.31:0/1285130058 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f988c0094f0 con 0x7f9898159920 2023-10-06T02:31:06.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.692+0000 7f9895d9b700 1 --2- 172.21.15.31:0/1285130058 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9898159920 0x7f989814f100 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f988c009290 tx=0x7f988c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:06.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.692+0000 7f98867fc700 1 -- 172.21.15.31:0/1285130058 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f988c011e40 con 0x7f9898159920 2023-10-06T02:31:06.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.692+0000 7f98867fc700 1 -- 172.21.15.31:0/1285130058 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f988c01f070 con 0x7f9898159920 2023-10-06T02:31:06.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.692+0000 7f989759e700 1 -- 172.21.15.31:0/1285130058 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98980cd780 con 0x7f9898159920 2023-10-06T02:31:06.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.692+0000 7f98867fc700 1 -- 172.21.15.31:0/1285130058 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f988c0266d0 con 0x7f9898159920 2023-10-06T02:31:06.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.693+0000 7f989759e700 1 -- 172.21.15.31:0/1285130058 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98980cdd40 con 0x7f9898159920 2023-10-06T02:31:06.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.693+0000 7f989759e700 1 -- 172.21.15.31:0/1285130058 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f989804c3c0 con 0x7f9898159920 2023-10-06T02:31:06.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.694+0000 7f98867fc700 1 -- 172.21.15.31:0/1285130058 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) v1 ==== 85301+0+0 (secure 0 0 0) 0x7f988c026a40 con 0x7f9898159920 2023-10-06T02:31:06.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.695+0000 7f98867fc700 1 --2- 172.21.15.31:0/1285130058 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9878067560 0x7f9878069a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:06.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.695+0000 7f98867fc700 1 -- 172.21.15.31:0/1285130058 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f988c021070 con 0x7f9898159920 2023-10-06T02:31:06.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.696+0000 7f989659c700 1 --2- 172.21.15.31:0/1285130058 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9878067560 0x7f9878069a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:06.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.696+0000 7f989659c700 1 --2- 172.21.15.31:0/1285130058 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9878067560 0x7f9878069a10 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9888006f00 tx=0x7f9888008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:06.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.699+0000 7f98867fc700 1 -- 172.21.15.31:0/1285130058 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f988c053680 con 0x7f9898159920 2023-10-06T02:31:06.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.936+0000 7f989759e700 1 -- 172.21.15.31:0/1285130058 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "group_name": "group_0000000000911578", "target": ["mon-mgr", ""]}) v1 -- 0x7f98980aedb0 con 0x7f9878067560 2023-10-06T02:31:06.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.941+0000 7f98867fc700 1 -- 172.21.15.31:0/1285130058 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f98980aedb0 con 0x7f9878067560 2023-10-06T02:31:06.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.943+0000 7f989759e700 1 -- 172.21.15.31:0/1285130058 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9878067560 msgr2=0x7f9878069a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:06.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.943+0000 7f989759e700 1 --2- 172.21.15.31:0/1285130058 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9878067560 0x7f9878069a10 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9888006f00 tx=0x7f9888008040 comp rx=0 tx=0).stop 2023-10-06T02:31:06.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.943+0000 7f989759e700 1 -- 172.21.15.31:0/1285130058 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9898159920 msgr2=0x7f989814f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:06.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.943+0000 7f989759e700 1 --2- 172.21.15.31:0/1285130058 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9898159920 0x7f989814f100 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f988c009290 tx=0x7f988c017000 comp rx=0 tx=0).stop 2023-10-06T02:31:06.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.944+0000 7f989759e700 1 -- 172.21.15.31:0/1285130058 shutdown_connections 2023-10-06T02:31:06.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.944+0000 7f989759e700 1 --2- 172.21.15.31:0/1285130058 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f989815dfd0 0x7f989814f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:06.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.944+0000 7f989759e700 1 --2- 172.21.15.31:0/1285130058 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9878067560 0x7f9878069a10 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:06.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.944+0000 7f989759e700 1 --2- 172.21.15.31:0/1285130058 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9898159550 0x7f989814cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:06.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.944+0000 7f989759e700 1 --2- 172.21.15.31:0/1285130058 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9898159920 0x7f989814f100 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:06.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.944+0000 7f989759e700 1 -- 172.21.15.31:0/1285130058 >> 172.21.15.31:0/1285130058 conn(0x7f989806ce80 msgr2=0x7f9898068e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:06.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.944+0000 7f989759e700 1 -- 172.21.15.31:0/1285130058 shutdown_connections 2023-10-06T02:31:06.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:06.944+0000 7f989759e700 1 -- 172.21.15.31:0/1285130058 wait complete. 2023-10-06T02:31:06.961 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T02:31:06.962 DEBUG:teuthology.orchestra.run.smithi031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T02:31:07.045 DEBUG:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T02:31:07.130 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T02:31:07.130 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T02:31:07.130 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:31:07.130 DEBUG:teuthology.orchestra.run.smithi031:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T02:31:07.216 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi031.front.sepia.ceph.com 2023-10-06T02:31:07.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:31:07.217 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T02:31:07.254 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-06T02:31:07.254 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:31:07.254 DEBUG:teuthology.orchestra.run.smithi062:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-06T02:31:07.341 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi062.front.sepia.ceph.com 2023-10-06T02:31:07.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:31:07.341 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T02:31:07.381 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:31:07.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:07.568+0000 7f7bacd9d700 1 Processor -- start 2023-10-06T02:31:07.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:07.568+0000 7f7bacd9d700 1 -- start start 2023-10-06T02:31:07.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:07.570+0000 7f7bacd9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7ba8140a70 0x7f7ba80c6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:07.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:07.570+0000 7f7bacd9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7ba8140e40 0x7f7ba80c7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:07.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:07.570+0000 7f7bacd9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ba80c6d50 con 0x7f7ba80c8330 2023-10-06T02:31:07.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:07.570+0000 7f7bacd9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ba80c7030 con 0x7f7ba8140e40 2023-10-06T02:31:07.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:07.570+0000 7f7bacd9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba80c7310 con 0x7f7ba8140a70 2023-10-06T02:31:07.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:07.570+0000 7f7ba7fff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f7ba80c8330 0x7f7ba80aeef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:36984/0 (socket says 172.21.15.31:36984) 2023-10-06T02:31:07.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:07.570+0000 7f7ba7fff700 1 -- 172.21.15.31:0/3223164391 learned_addr learned my addr 172.21.15.31:0/3223164391 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:31:07.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:07.570+0000 7f7ba6ffd700 1 --2- 172.21.15.31:0/3223164391 >> v2:172.21.15.62:3300/0 conn(0x7f7ba8140e40 0x7f7ba80c7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:07.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:07.570+0000 7f7ba77fe700 1 --2- 172.21.15.31:0/3223164391 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7ba8140a70 0x7f7ba80c6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:07.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:07.571+0000 7f7ba67fc700 1 -- 172.21.15.31:0/3223164391 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 228309578 0 0) 0x7f7ba80c6d50 con 0x7f7ba80c8330 2023-10-06T02:31:07.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:07.571+0000 7f7ba67fc700 1 -- 172.21.15.31:0/3223164391 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b90003570 con 0x7f7ba80c8330 2023-10-06T02:31:07.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:07.571+0000 7f7ba6ffd700 1 -- 172.21.15.31:0/3223164391 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7ba8140a70 msgr2=0x7f7ba80c6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:07.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:07.571+0000 7f7ba6ffd700 1 --2- 172.21.15.31:0/3223164391 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7ba8140a70 0x7f7ba80c6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:07.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:07.571+0000 7f7ba6ffd700 1 -- 172.21.15.31:0/3223164391 >> v1:172.21.15.31:6789/0 conn(0x7f7ba80c8330 legacy=0x7f7ba80aeef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:07.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:07.571+0000 7f7ba6ffd700 1 -- 172.21.15.31:0/3223164391 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ba80c75f0 con 0x7f7ba8140e40 2023-10-06T02:31:07.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:07.571+0000 7f7ba77fe700 1 --2- 172.21.15.31:0/3223164391 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7ba8140a70 0x7f7ba80c6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:31:07.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:07.571+0000 7f7ba6ffd700 1 --2- 172.21.15.31:0/3223164391 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ba8140e40 0x7f7ba80c7df0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c0098d0 tx=0x7f7b9c009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20182e4ca6af2bd5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:07.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:07.571+0000 7f7ba67fc700 1 -- 172.21.15.31:0/3223164391 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9c005c10 con 0x7f7ba8140e40 2023-10-06T02:31:07.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:07.571+0000 7f7ba67fc700 1 -- 172.21.15.31:0/3223164391 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b9c005d70 con 0x7f7ba8140e40 2023-10-06T02:31:07.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:07.571+0000 7f7ba67fc700 1 -- 172.21.15.31:0/3223164391 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9c006260 con 0x7f7ba8140e40 2023-10-06T02:31:07.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:07.572+0000 7f7bacd9d700 1 -- 172.21.15.31:0/3223164391 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ba8140e40 msgr2=0x7f7ba80c7df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:07.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:07.572+0000 7f7bacd9d700 1 --2- 172.21.15.31:0/3223164391 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ba8140e40 0x7f7ba80c7df0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c0098d0 tx=0x7f7b9c009120 comp rx=0 tx=0).stop 2023-10-06T02:31:07.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:07.572+0000 7f7bacd9d700 1 -- 172.21.15.31:0/3223164391 shutdown_connections 2023-10-06T02:31:07.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:07.572+0000 7f7bacd9d700 1 --2- 172.21.15.31:0/3223164391 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7ba8140a70 0x7f7ba80c6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:07.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:07.572+0000 7f7bacd9d700 1 --2- 172.21.15.31:0/3223164391 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ba8140e40 0x7f7ba80c7df0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:07.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:07.572+0000 7f7bacd9d700 1 -- 172.21.15.31:0/3223164391 >> 172.21.15.31:0/3223164391 conn(0x7f7ba806d170 msgr2=0x7f7ba80cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:07.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:07.572+0000 7f7bacd9d700 1 -- 172.21.15.31:0/3223164391 shutdown_connections 2023-10-06T02:31:07.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:07.572+0000 7f7bacd9d700 1 -- 172.21.15.31:0/3223164391 wait complete. 2023-10-06T02:31:07.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:07.574+0000 7f7bacd9d700 1 Processor -- start 2023-10-06T02:31:07.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:07.574+0000 7f7bacd9d700 1 -- start start 2023-10-06T02:31:07.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:07.575+0000 7f7bacd9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7ba80c8330 0x7f7ba80c3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:07.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:07.575+0000 7f7bacd9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ba8140a70 0x7f7ba813ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:07.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:07.575+0000 7f7bacd9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ba8140e40 0x7f7ba813d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:07.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:07.575+0000 7f7bacd9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba80ca990 con 0x7f7ba8140e40 2023-10-06T02:31:07.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:07.575+0000 7f7bacd9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba80ca660 con 0x7f7ba8140a70 2023-10-06T02:31:07.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:07.575+0000 7f7bacd9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba80c4690 con 0x7f7ba80c8330 2023-10-06T02:31:07.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:07.575+0000 7f7ba7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ba8140e40 0x7f7ba813d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:07.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:07.575+0000 7f7ba7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ba8140e40 0x7f7ba813d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36634/0 (socket says 172.21.15.31:36634) 2023-10-06T02:31:07.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:07.575+0000 7f7ba7fff700 1 -- 172.21.15.31:0/175769796 learned_addr learned my addr 172.21.15.31:0/175769796 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:07.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:07.575+0000 7f7ba77fe700 1 --2- 172.21.15.31:0/175769796 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7ba80c8330 0x7f7ba80c3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:07.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:07.575+0000 7f7ba6ffd700 1 --2- 172.21.15.31:0/175769796 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ba8140a70 0x7f7ba813ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:07.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:07.575+0000 7f7ba7fff700 1 -- 172.21.15.31:0/175769796 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7ba80c8330 msgr2=0x7f7ba80c3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:07.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:07.575+0000 7f7ba7fff700 1 --2- 172.21.15.31:0/175769796 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7ba80c8330 0x7f7ba80c3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:07.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:07.575+0000 7f7ba7fff700 1 -- 172.21.15.31:0/175769796 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ba8140a70 msgr2=0x7f7ba813ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:07.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:07.575+0000 7f7ba7fff700 1 --2- 172.21.15.31:0/175769796 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ba8140a70 0x7f7ba813ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:07.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:07.575+0000 7f7ba7fff700 1 -- 172.21.15.31:0/175769796 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b98009420 con 0x7f7ba8140e40 2023-10-06T02:31:07.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:07.576+0000 7f7ba7fff700 1 --2- 172.21.15.31:0/175769796 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ba8140e40 0x7f7ba813d930 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f7ba000cc40 tx=0x7f7ba0001aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:07.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:07.576+0000 7f7b8f7fe700 1 -- 172.21.15.31:0/175769796 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba000a8b0 con 0x7f7ba8140e40 2023-10-06T02:31:07.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:07.576+0000 7f7b8f7fe700 1 -- 172.21.15.31:0/175769796 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ba000aa10 con 0x7f7ba8140e40 2023-10-06T02:31:07.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:07.576+0000 7f7bacd9d700 1 -- 172.21.15.31:0/175769796 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b9c0093c0 con 0x7f7ba8140e40 2023-10-06T02:31:07.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:07.576+0000 7f7b8f7fe700 1 -- 172.21.15.31:0/175769796 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba000ad90 con 0x7f7ba8140e40 2023-10-06T02:31:07.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:07.576+0000 7f7bacd9d700 1 -- 172.21.15.31:0/175769796 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ba80c43b0 con 0x7f7ba8140e40 2023-10-06T02:31:07.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:07.577+0000 7f7bacd9d700 1 -- 172.21.15.31:0/175769796 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ba80ca4c0 con 0x7f7ba8140e40 2023-10-06T02:31:07.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:07.578+0000 7f7b8f7fe700 1 -- 172.21.15.31:0/175769796 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) v1 ==== 85301+0+0 (secure 0 0 0) 0x7f7ba0007300 con 0x7f7ba8140e40 2023-10-06T02:31:07.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:07.579+0000 7f7b8f7fe700 1 --2- 172.21.15.31:0/175769796 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7b94067560 0x7f7b94069a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:07.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:07.579+0000 7f7b8f7fe700 1 -- 172.21.15.31:0/175769796 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f7ba0085630 con 0x7f7ba8140e40 2023-10-06T02:31:07.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:07.579+0000 7f7ba77fe700 1 --2- 172.21.15.31:0/175769796 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7b94067560 0x7f7b94069a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:07.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:07.580+0000 7f7ba77fe700 1 --2- 172.21.15.31:0/175769796 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7b94067560 0x7f7b94069a10 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f7b98009f40 tx=0x7f7b980091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:07.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:07.583+0000 7f7b8f7fe700 1 -- 172.21.15.31:0/175769796 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7ba80ca4c0 con 0x7f7ba8140e40 2023-10-06T02:31:07.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:07.835+0000 7f7bacd9d700 1 -- 172.21.15.31:0/175769796 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7ba80ca4c0 con 0x7f7ba8140e40 2023-10-06T02:31:07.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:07.836+0000 7f7b8f7fe700 1 -- 172.21.15.31:0/175769796 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) v1 ==== 95+0+4603 (secure 0 0 0) 0x7f7ba80ca4c0 con 0x7f7ba8140e40 2023-10-06T02:31:07.836 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:31:07.836 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":101,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6923,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/4169296779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":4169296779},{"type":"v1","addr":"172.21.15.31:6835","nonce":4169296779}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95},{"gid":6926,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/3032245108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3032245108},{"type":"v1","addr":"172.21.15.62:6834","nonce":3032245108}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95},{"gid":8469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6833/1522769465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1522769465},{"type":"v1","addr":"172.21.15.31:6833","nonce":1522769465}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:30:51.153736+0000","modified":"2023-10-06T02:30:55.096858+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8487":{"gid":8487,"name":"d","rank":0,"incarnation":96,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/2236458886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2236458886},{"type":"v1","addr":"172.21.15.62:6835","nonce":2236458886}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2023-10-06T02:31:07.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:07.838+0000 7f7bacd9d700 1 -- 172.21.15.31:0/175769796 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7b94067560 msgr2=0x7f7b94069a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:07.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:07.838+0000 7f7bacd9d700 1 --2- 172.21.15.31:0/175769796 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7b94067560 0x7f7b94069a10 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f7b98009f40 tx=0x7f7b980091a0 comp rx=0 tx=0).stop 2023-10-06T02:31:07.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:07.838+0000 7f7bacd9d700 1 -- 172.21.15.31:0/175769796 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ba8140e40 msgr2=0x7f7ba813d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:07.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:07.838+0000 7f7bacd9d700 1 --2- 172.21.15.31:0/175769796 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ba8140e40 0x7f7ba813d930 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f7ba000cc40 tx=0x7f7ba0001aa0 comp rx=0 tx=0).stop 2023-10-06T02:31:07.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:07.838+0000 7f7bacd9d700 1 -- 172.21.15.31:0/175769796 shutdown_connections 2023-10-06T02:31:07.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:07.838+0000 7f7bacd9d700 1 --2- 172.21.15.31:0/175769796 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ba8140e40 0x7f7ba813d930 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:07.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:07.838+0000 7f7bacd9d700 1 --2- 172.21.15.31:0/175769796 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7b94067560 0x7f7b94069a10 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:07.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:07.838+0000 7f7bacd9d700 1 --2- 172.21.15.31:0/175769796 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7ba80c8330 0x7f7ba80c3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:07.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:07.838+0000 7f7bacd9d700 1 --2- 172.21.15.31:0/175769796 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ba8140a70 0x7f7ba813ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:07.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:07.838+0000 7f7bacd9d700 1 -- 172.21.15.31:0/175769796 >> 172.21.15.31:0/175769796 conn(0x7f7ba806d170 msgr2=0x7f7ba80c2970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:07.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:07.838+0000 7f7bacd9d700 1 -- 172.21.15.31:0/175769796 shutdown_connections 2023-10-06T02:31:07.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:07.838+0000 7f7bacd9d700 1 -- 172.21.15.31:0/175769796 wait complete. 2023-10-06T02:31:07.840 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 101 2023-10-06T02:31:07.857 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:31:08.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.044+0000 7f92f7fff700 1 Processor -- start 2023-10-06T02:31:08.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.044+0000 7f92f7fff700 1 -- start start 2023-10-06T02:31:08.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.045+0000 7f92f7fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f92f8153c30 0x7f92f815a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:08.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.045+0000 7f92f7fff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f92f815de50 0x7f92f80cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:08.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.045+0000 7f92f7fff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92f80cf3c0 con 0x7f92f80cef40 2023-10-06T02:31:08.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.045+0000 7f92f7fff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92f815afd0 con 0x7f92f8153c30 2023-10-06T02:31:08.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.045+0000 7f92f7fff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92f815b2b0 con 0x7f92f815de50 2023-10-06T02:31:08.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.045+0000 7f92f6ffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f92f8153c30 0x7f92f815a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:08.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.046+0000 7f92f6ffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f92f8153c30 0x7f92f815a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37384/0 (socket says 172.21.15.31:37384) 2023-10-06T02:31:08.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.046+0000 7f92f6ffd700 1 -- 172.21.15.31:0/603336954 learned_addr learned my addr 172.21.15.31:0/603336954 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:08.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.046+0000 7f92f67fc700 1 --2- 172.21.15.31:0/603336954 >> v2:172.21.15.62:3300/0 conn(0x7f92f815de50 0x7f92f80cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:08.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.046+0000 7f92f6ffd700 1 -- 172.21.15.31:0/603336954 >> v2:172.21.15.62:3300/0 conn(0x7f92f815de50 msgr2=0x7f92f80cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:08.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.046+0000 7f92f6ffd700 1 --2- 172.21.15.31:0/603336954 >> v2:172.21.15.62:3300/0 conn(0x7f92f815de50 0x7f92f80cea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:08.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.046+0000 7f92f6ffd700 1 -- 172.21.15.31:0/603336954 >> v1:172.21.15.62:6789/0 conn(0x7f92f80cef40 legacy=0x7f92f806d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:08.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.046+0000 7f92f6ffd700 1 -- 172.21.15.31:0/603336954 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92f815b590 con 0x7f92f8153c30 2023-10-06T02:31:08.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.046+0000 7f92f6ffd700 1 --2- 172.21.15.31:0/603336954 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92f8153c30 0x7f92f815a5e0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f92f80713a0 tx=0x7f92e8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e192f99eea3bfcd5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:08.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.047+0000 7f92f5ffb700 1 -- 172.21.15.31:0/603336954 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e800b860 con 0x7f92f8153c30 2023-10-06T02:31:08.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.047+0000 7f92f5ffb700 1 -- 172.21.15.31:0/603336954 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92e800b9c0 con 0x7f92f8153c30 2023-10-06T02:31:08.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.047+0000 7f92f5ffb700 1 -- 172.21.15.31:0/603336954 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e800bd60 con 0x7f92f8153c30 2023-10-06T02:31:08.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.047+0000 7f92f7fff700 1 -- 172.21.15.31:0/603336954 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92f8153c30 msgr2=0x7f92f815a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:08.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.047+0000 7f92f7fff700 1 --2- 172.21.15.31:0/603336954 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92f8153c30 0x7f92f815a5e0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f92f80713a0 tx=0x7f92e8009250 comp rx=0 tx=0).stop 2023-10-06T02:31:08.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.047+0000 7f92f7fff700 1 -- 172.21.15.31:0/603336954 shutdown_connections 2023-10-06T02:31:08.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.047+0000 7f92f7fff700 1 --2- 172.21.15.31:0/603336954 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92f8153c30 0x7f92f815a5e0 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:08.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.047+0000 7f92f7fff700 1 --2- 172.21.15.31:0/603336954 >> v2:172.21.15.62:3300/0 conn(0x7f92f815de50 0x7f92f80cea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:08.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.047+0000 7f92f7fff700 1 -- 172.21.15.31:0/603336954 >> 172.21.15.31:0/603336954 conn(0x7f92f80703a0 msgr2=0x7f92f80707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:08.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.047+0000 7f92f7fff700 1 -- 172.21.15.31:0/603336954 shutdown_connections 2023-10-06T02:31:08.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.047+0000 7f92f7fff700 1 -- 172.21.15.31:0/603336954 wait complete. 2023-10-06T02:31:08.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.049+0000 7f92f7fff700 1 Processor -- start 2023-10-06T02:31:08.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.049+0000 7f92f7fff700 1 -- start start 2023-10-06T02:31:08.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.050+0000 7f92f7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92f80cef40 0x7f92f80c7d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:08.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.050+0000 7f92f7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92f8153c30 0x7f92f80b3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:08.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.050+0000 7f92f7fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f92f815de50 0x7f92f80b5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:08.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.050+0000 7f92f7fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f80b3690 con 0x7f92f80cef40 2023-10-06T02:31:08.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.050+0000 7f92f7fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f80c8f60 con 0x7f92f8153c30 2023-10-06T02:31:08.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.050+0000 7f92f7fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92f80c8350 con 0x7f92f815de50 2023-10-06T02:31:08.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.050+0000 7f92f6ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92f80cef40 0x7f92f80c7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:08.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.050+0000 7f92f6ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92f80cef40 0x7f92f80c7d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37394/0 (socket says 172.21.15.31:37394) 2023-10-06T02:31:08.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.050+0000 7f92f6ffd700 1 -- 172.21.15.31:0/397843060 learned_addr learned my addr 172.21.15.31:0/397843060 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:08.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.051+0000 7f92f6ffd700 1 -- 172.21.15.31:0/397843060 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f92f815de50 msgr2=0x7f92f80b5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:08.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.051+0000 7f92f6ffd700 1 --2- 172.21.15.31:0/397843060 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f92f815de50 0x7f92f80b5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:08.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.051+0000 7f92f6ffd700 1 -- 172.21.15.31:0/397843060 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92f8153c30 msgr2=0x7f92f80b3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:08.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.051+0000 7f92f67fc700 1 --2- 172.21.15.31:0/397843060 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92f8153c30 0x7f92f80b3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:08.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.051+0000 7f92f6ffd700 1 --2- 172.21.15.31:0/397843060 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92f8153c30 0x7f92f80b3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:08.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.051+0000 7f92f6ffd700 1 -- 172.21.15.31:0/397843060 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92e80094f0 con 0x7f92f80cef40 2023-10-06T02:31:08.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.051+0000 7f92f67fc700 1 --2- 172.21.15.31:0/397843060 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92f8153c30 0x7f92f80b3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:31:08.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.051+0000 7f92f6ffd700 1 --2- 172.21.15.31:0/397843060 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92f80cef40 0x7f92f80c7d70 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f92f806eb00 tx=0x7f92e8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:08.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.051+0000 7f92deffd700 1 -- 172.21.15.31:0/397843060 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e8011e60 con 0x7f92f80cef40 2023-10-06T02:31:08.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.051+0000 7f92deffd700 1 -- 172.21.15.31:0/397843060 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92e801a630 con 0x7f92f80cef40 2023-10-06T02:31:08.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.051+0000 7f92deffd700 1 -- 172.21.15.31:0/397843060 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e801a9f0 con 0x7f92f80cef40 2023-10-06T02:31:08.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.051+0000 7f92f7fff700 1 -- 172.21.15.31:0/397843060 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92f80c8630 con 0x7f92f80cef40 2023-10-06T02:31:08.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.051+0000 7f92f7fff700 1 -- 172.21.15.31:0/397843060 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92f80c8bf0 con 0x7f92f80cef40 2023-10-06T02:31:08.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.053+0000 7f92f7fff700 1 -- 172.21.15.31:0/397843060 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92f804c3c0 con 0x7f92f80cef40 2023-10-06T02:31:08.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.053+0000 7f92deffd700 1 -- 172.21.15.31:0/397843060 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) v1 ==== 85301+0+0 (secure 0 0 0) 0x7f92e802d440 con 0x7f92f80cef40 2023-10-06T02:31:08.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.054+0000 7f92deffd700 1 --2- 172.21.15.31:0/397843060 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f92e4067560 0x7f92e4069a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:08.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.055+0000 7f92deffd700 1 -- 172.21.15.31:0/397843060 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f92e8086ce0 con 0x7f92f80cef40 2023-10-06T02:31:08.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.055+0000 7f92f67fc700 1 --2- 172.21.15.31:0/397843060 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f92e4067560 0x7f92e4069a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:08.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.056+0000 7f92f67fc700 1 --2- 172.21.15.31:0/397843060 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f92e4067560 0x7f92e4069a10 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f92f80c7c90 tx=0x7f92ec008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:08.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.058+0000 7f92deffd700 1 -- 172.21.15.31:0/397843060 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f92e8053490 con 0x7f92f80cef40 2023-10-06T02:31:08.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.143+0000 7f92deffd700 1 -- 172.21.15.31:0/397843060 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mgrmap(e 24) v1 ==== 85403+0+0 (secure 0 0 0) 0x7f92e8053860 con 0x7f92f80cef40 2023-10-06T02:31:08.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.269+0000 7f92f7fff700 1 -- 172.21.15.31:0/397843060 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f92f80b6300 con 0x7f92f80cef40 2023-10-06T02:31:08.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.270+0000 7f92deffd700 1 -- 172.21.15.31:0/397843060 <== mon.0 v2:172.21.15.31:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v144) v1 ==== 74+0+16876 (secure 0 0 0) 0x7f92e804f970 con 0x7f92f80cef40 2023-10-06T02:31:08.271 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:31:08.271 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":144,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:30:52.650397+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:30:49.325914+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":24,"pool_name":"cephfs_data","create_time":"2023-10-06T02:30:50.127083+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":20,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:31:08.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.272+0000 7f92f7fff700 1 -- 172.21.15.31:0/397843060 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f92e4067560 msgr2=0x7f92e4069a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:08.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.272+0000 7f92f7fff700 1 --2- 172.21.15.31:0/397843060 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f92e4067560 0x7f92e4069a10 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f92f80c7c90 tx=0x7f92ec008040 comp rx=0 tx=0).stop 2023-10-06T02:31:08.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.272+0000 7f92f7fff700 1 -- 172.21.15.31:0/397843060 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92f80cef40 msgr2=0x7f92f80c7d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:08.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.272+0000 7f92f7fff700 1 --2- 172.21.15.31:0/397843060 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92f80cef40 0x7f92f80c7d70 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f92f806eb00 tx=0x7f92e8017000 comp rx=0 tx=0).stop 2023-10-06T02:31:08.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.272+0000 7f92f7fff700 1 -- 172.21.15.31:0/397843060 shutdown_connections 2023-10-06T02:31:08.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.272+0000 7f92f7fff700 1 --2- 172.21.15.31:0/397843060 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f92e4067560 0x7f92e4069a10 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:08.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.272+0000 7f92f7fff700 1 --2- 172.21.15.31:0/397843060 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f92f815de50 0x7f92f80b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:08.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.272+0000 7f92f7fff700 1 --2- 172.21.15.31:0/397843060 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92f80cef40 0x7f92f80c7d70 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:08.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.272+0000 7f92f7fff700 1 --2- 172.21.15.31:0/397843060 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92f8153c30 0x7f92f80b3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:08.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.272+0000 7f92f7fff700 1 -- 172.21.15.31:0/397843060 >> 172.21.15.31:0/397843060 conn(0x7f92f80703a0 msgr2=0x7f92f8151010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:08.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.273+0000 7f92f7fff700 1 -- 172.21.15.31:0/397843060 shutdown_connections 2023-10-06T02:31:08.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.273+0000 7f92f7fff700 1 -- 172.21.15.31:0/397843060 wait complete. 2023-10-06T02:31:08.291 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T02:22:53.183425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T02:23:00.046967+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 23, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T02:30:49.325914+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '143', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 24, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T02:30:50.127083+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '144', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T02:31:08.291 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T02:31:08.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.480+0000 7f152519d700 1 Processor -- start 2023-10-06T02:31:08.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.481+0000 7f152519d700 1 -- start start 2023-10-06T02:31:08.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.482+0000 7f152519d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1520159540 0x7f15200c1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:08.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.482+0000 7f152519d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f1520159910 0x7f15200677b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:08.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.482+0000 7f152519d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1520068250 con 0x7f1520067cf0 2023-10-06T02:31:08.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.482+0000 7f152519d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1520068530 con 0x7f1520159910 2023-10-06T02:31:08.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.482+0000 7f152519d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1520068810 con 0x7f1520159540 2023-10-06T02:31:08.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.482+0000 7f152499c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f1520067cf0 0x7f15200c6be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49710/0 (socket says 172.21.15.31:49710) 2023-10-06T02:31:08.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.482+0000 7f152499c700 1 -- 172.21.15.31:0/1380820636 learned_addr learned my addr 172.21.15.31:0/1380820636 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:31:08.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.482+0000 7f151ffff700 1 --2- 172.21.15.31:0/1380820636 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1520159540 0x7f15200c1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:08.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.482+0000 7f151f7fe700 1 --2- 172.21.15.31:0/1380820636 >> v2:172.21.15.62:3300/0 conn(0x7f1520159910 0x7f15200677b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:08.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.483+0000 7f151effd700 1 -- 172.21.15.31:0/1380820636 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1929850147 0 0) 0x7f1520068250 con 0x7f1520067cf0 2023-10-06T02:31:08.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.483+0000 7f151effd700 1 -- 172.21.15.31:0/1380820636 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1508003570 con 0x7f1520067cf0 2023-10-06T02:31:08.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.483+0000 7f151ffff700 1 -- 172.21.15.31:0/1380820636 >> v2:172.21.15.62:3300/0 conn(0x7f1520159910 msgr2=0x7f15200677b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:08.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.483+0000 7f151ffff700 1 --2- 172.21.15.31:0/1380820636 >> v2:172.21.15.62:3300/0 conn(0x7f1520159910 0x7f15200677b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:08.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.483+0000 7f151ffff700 1 -- 172.21.15.31:0/1380820636 >> v1:172.21.15.31:6789/0 conn(0x7f1520067cf0 legacy=0x7f15200c6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:08.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.483+0000 7f151ffff700 1 -- 172.21.15.31:0/1380820636 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15200a9ca0 con 0x7f1520159540 2023-10-06T02:31:08.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.483+0000 7f151ffff700 1 --2- 172.21.15.31:0/1380820636 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1520159540 0x7f15200c1340 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f15100098e0 tx=0x7f15100091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7b6ce5cc6745995 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:08.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.483+0000 7f151effd700 1 -- 172.21.15.31:0/1380820636 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1510011630 con 0x7f1520159540 2023-10-06T02:31:08.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.484+0000 7f151effd700 1 -- 172.21.15.31:0/1380820636 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1510011790 con 0x7f1520159540 2023-10-06T02:31:08.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.484+0000 7f151effd700 1 -- 172.21.15.31:0/1380820636 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1510011b30 con 0x7f1520159540 2023-10-06T02:31:08.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.484+0000 7f152519d700 1 -- 172.21.15.31:0/1380820636 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1520159540 msgr2=0x7f15200c1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:08.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.484+0000 7f152519d700 1 --2- 172.21.15.31:0/1380820636 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1520159540 0x7f15200c1340 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f15100098e0 tx=0x7f15100091a0 comp rx=0 tx=0).stop 2023-10-06T02:31:08.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.484+0000 7f152519d700 1 -- 172.21.15.31:0/1380820636 shutdown_connections 2023-10-06T02:31:08.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.484+0000 7f152519d700 1 --2- 172.21.15.31:0/1380820636 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1520159540 0x7f15200c1340 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:08.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.484+0000 7f152519d700 1 --2- 172.21.15.31:0/1380820636 >> v2:172.21.15.62:3300/0 conn(0x7f1520159910 0x7f15200677b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:08.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.484+0000 7f152519d700 1 -- 172.21.15.31:0/1380820636 >> 172.21.15.31:0/1380820636 conn(0x7f152006f330 msgr2=0x7f15200cc5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:08.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.484+0000 7f152519d700 1 -- 172.21.15.31:0/1380820636 shutdown_connections 2023-10-06T02:31:08.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.484+0000 7f152519d700 1 -- 172.21.15.31:0/1380820636 wait complete. 2023-10-06T02:31:08.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.486+0000 7f152519d700 1 Processor -- start 2023-10-06T02:31:08.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.486+0000 7f152519d700 1 -- start start 2023-10-06T02:31:08.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.487+0000 7f152519d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1520067cf0 0x7f15200ae740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:08.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.487+0000 7f152519d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1520159540 0x7f152016d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:08.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.487+0000 7f152519d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1520159910 0x7f15200ac090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:08.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.487+0000 7f152519d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15200add50 con 0x7f1520159910 2023-10-06T02:31:08.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.487+0000 7f152519d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15200ae2b0 con 0x7f1520067cf0 2023-10-06T02:31:08.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.487+0000 7f152519d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f152016e240 con 0x7f1520159540 2023-10-06T02:31:08.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.487+0000 7f152499c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1520159910 0x7f15200ac090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:08.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.487+0000 7f152499c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1520159910 0x7f15200ac090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37410/0 (socket says 172.21.15.31:37410) 2023-10-06T02:31:08.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.487+0000 7f152499c700 1 -- 172.21.15.31:0/2248673191 learned_addr learned my addr 172.21.15.31:0/2248673191 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:08.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.487+0000 7f151ffff700 1 --2- 172.21.15.31:0/2248673191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1520067cf0 0x7f15200ae740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:08.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.487+0000 7f151f7fe700 1 --2- 172.21.15.31:0/2248673191 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1520159540 0x7f152016d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:08.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.488+0000 7f152499c700 1 -- 172.21.15.31:0/2248673191 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1520159540 msgr2=0x7f152016d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:08.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.488+0000 7f152499c700 1 --2- 172.21.15.31:0/2248673191 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1520159540 0x7f152016d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:08.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.488+0000 7f152499c700 1 -- 172.21.15.31:0/2248673191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1520067cf0 msgr2=0x7f15200ae740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:08.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.488+0000 7f152499c700 1 --2- 172.21.15.31:0/2248673191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1520067cf0 0x7f15200ae740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:08.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.488+0000 7f152499c700 1 -- 172.21.15.31:0/2248673191 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1510009420 con 0x7f1520159910 2023-10-06T02:31:08.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.488+0000 7f151ffff700 1 --2- 172.21.15.31:0/2248673191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1520067cf0 0x7f15200ae740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:31:08.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.488+0000 7f152499c700 1 --2- 172.21.15.31:0/2248673191 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1520159910 0x7f15200ac090 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f1518001460 tx=0x7f1518001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:08.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.488+0000 7f1507fff700 1 -- 172.21.15.31:0/2248673191 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f151800ce80 con 0x7f1520159910 2023-10-06T02:31:08.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.488+0000 7f1507fff700 1 -- 172.21.15.31:0/2248673191 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f151800ab70 con 0x7f1520159910 2023-10-06T02:31:08.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.488+0000 7f1507fff700 1 -- 172.21.15.31:0/2248673191 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15180070f0 con 0x7f1520159910 2023-10-06T02:31:08.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.488+0000 7f152519d700 1 -- 172.21.15.31:0/2248673191 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15200adf80 con 0x7f1520159910 2023-10-06T02:31:08.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.488+0000 7f152519d700 1 -- 172.21.15.31:0/2248673191 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15200aec80 con 0x7f1520159910 2023-10-06T02:31:08.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.489+0000 7f152519d700 1 -- 172.21.15.31:0/2248673191 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15200ae460 con 0x7f1520159910 2023-10-06T02:31:08.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.490+0000 7f1507fff700 1 -- 172.21.15.31:0/2248673191 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 85403+0+0 (secure 0 0 0) 0x7f1518007520 con 0x7f1520159910 2023-10-06T02:31:08.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.491+0000 7f1507fff700 1 --2- 172.21.15.31:0/2248673191 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f150c067680 0x7f150c069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:08.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.491+0000 7f1507fff700 1 -- 172.21.15.31:0/2248673191 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f1518085cf0 con 0x7f1520159910 2023-10-06T02:31:08.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.491+0000 7f151ffff700 1 --2- 172.21.15.31:0/2248673191 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f150c067680 0x7f150c069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:08.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.492+0000 7f151ffff700 1 --2- 172.21.15.31:0/2248673191 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f150c067680 0x7f150c069b30 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f15100091e0 tx=0x7f1510017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:08.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.495+0000 7f1507fff700 1 -- 172.21.15.31:0/2248673191 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f15200ae460 con 0x7f1520159910 2023-10-06T02:31:08.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.696+0000 7f152519d700 1 -- 172.21.15.31:0/2248673191 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f15200ae420 con 0x7f1520159910 2023-10-06T02:31:08.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.697+0000 7f1507fff700 1 -- 172.21.15.31:0/2248673191 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v144) v1 ==== 93+0+10 (secure 0 0 0) 0x7f15180830b0 con 0x7f1520159910 2023-10-06T02:31:08.698 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-10-06T02:31:08.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.699+0000 7f152519d700 1 -- 172.21.15.31:0/2248673191 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f150c067680 msgr2=0x7f150c069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:08.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.699+0000 7f152519d700 1 --2- 172.21.15.31:0/2248673191 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f150c067680 0x7f150c069b30 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f15100091e0 tx=0x7f1510017000 comp rx=0 tx=0).stop 2023-10-06T02:31:08.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.699+0000 7f152519d700 1 -- 172.21.15.31:0/2248673191 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1520159910 msgr2=0x7f15200ac090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:08.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.699+0000 7f152519d700 1 --2- 172.21.15.31:0/2248673191 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1520159910 0x7f15200ac090 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f1518001460 tx=0x7f1518001ae0 comp rx=0 tx=0).stop 2023-10-06T02:31:08.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.699+0000 7f152519d700 1 -- 172.21.15.31:0/2248673191 shutdown_connections 2023-10-06T02:31:08.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.699+0000 7f152519d700 1 --2- 172.21.15.31:0/2248673191 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1520159910 0x7f15200ac090 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:08.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.699+0000 7f152519d700 1 --2- 172.21.15.31:0/2248673191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1520067cf0 0x7f15200ae740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:08.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.699+0000 7f152519d700 1 --2- 172.21.15.31:0/2248673191 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f150c067680 0x7f150c069b30 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:08.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.699+0000 7f152519d700 1 --2- 172.21.15.31:0/2248673191 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1520159540 0x7f152016d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:08.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.699+0000 7f152519d700 1 -- 172.21.15.31:0/2248673191 >> 172.21.15.31:0/2248673191 conn(0x7f152006f330 msgr2=0x7f152015b000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:08.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.699+0000 7f152519d700 1 -- 172.21.15.31:0/2248673191 shutdown_connections 2023-10-06T02:31:08.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.699+0000 7f152519d700 1 -- 172.21.15.31:0/2248673191 wait complete. 2023-10-06T02:31:08.717 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T02:31:08.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.907+0000 7f9db5f9d700 1 Processor -- start 2023-10-06T02:31:08.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.907+0000 7f9db5f9d700 1 -- start start 2023-10-06T02:31:08.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.909+0000 7f9db5f9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9db0155890 0x7f9db00c7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:08.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.909+0000 7f9db5f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9db0155c60 0x7f9db006a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:08.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.909+0000 7f9db5f9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9db006de50 0x7f9db006e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:08.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.909+0000 7f9db5f9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9db006e810 con 0x7f9db006de50 2023-10-06T02:31:08.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.909+0000 7f9db5f9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9db00c7e70 con 0x7f9db0155890 2023-10-06T02:31:08.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.909+0000 7f9db5f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9db006b790 con 0x7f9db0155c60 2023-10-06T02:31:08.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.909+0000 7f9db579c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9db006de50 0x7f9db006e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:08.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.909+0000 7f9db579c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9db006de50 0x7f9db006e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37418/0 (socket says 172.21.15.31:37418) 2023-10-06T02:31:08.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.909+0000 7f9db579c700 1 -- 172.21.15.31:0/3100183751 learned_addr learned my addr 172.21.15.31:0/3100183751 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:08.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.909+0000 7f9daffff700 1 --2- 172.21.15.31:0/3100183751 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9db0155c60 0x7f9db006a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:08.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.909+0000 7f9db4f9b700 1 --2- 172.21.15.31:0/3100183751 >> v2:172.21.15.62:3300/0 conn(0x7f9db0155890 0x7f9db00c7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:08.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.910+0000 7f9db579c700 1 -- 172.21.15.31:0/3100183751 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9db0155c60 msgr2=0x7f9db006a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:08.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.910+0000 7f9db579c700 1 --2- 172.21.15.31:0/3100183751 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9db0155c60 0x7f9db006a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:08.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.910+0000 7f9db579c700 1 -- 172.21.15.31:0/3100183751 >> v2:172.21.15.62:3300/0 conn(0x7f9db0155890 msgr2=0x7f9db00c7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:08.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.910+0000 7f9db579c700 1 --2- 172.21.15.31:0/3100183751 >> v2:172.21.15.62:3300/0 conn(0x7f9db0155890 0x7f9db00c7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:08.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.910+0000 7f9db579c700 1 -- 172.21.15.31:0/3100183751 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9db006a8e0 con 0x7f9db006de50 2023-10-06T02:31:08.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.910+0000 7f9db579c700 1 --2- 172.21.15.31:0/3100183751 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9db006de50 0x7f9db006e2d0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f9db00c9be0 tx=0x7f9da8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=518652c84d31e1ae server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:08.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.910+0000 7f9daf7fe700 1 -- 172.21.15.31:0/3100183751 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da80092e0 con 0x7f9db006de50 2023-10-06T02:31:08.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.910+0000 7f9daf7fe700 1 -- 172.21.15.31:0/3100183751 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9da800c970 con 0x7f9db006de50 2023-10-06T02:31:08.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.910+0000 7f9daf7fe700 1 -- 172.21.15.31:0/3100183751 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da800cd30 con 0x7f9db006de50 2023-10-06T02:31:08.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.911+0000 7f9db5f9d700 1 -- 172.21.15.31:0/3100183751 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9db006de50 msgr2=0x7f9db006e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:08.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.911+0000 7f9db5f9d700 1 --2- 172.21.15.31:0/3100183751 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9db006de50 0x7f9db006e2d0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f9db00c9be0 tx=0x7f9da8009840 comp rx=0 tx=0).stop 2023-10-06T02:31:08.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.911+0000 7f9db5f9d700 1 -- 172.21.15.31:0/3100183751 shutdown_connections 2023-10-06T02:31:08.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.911+0000 7f9db5f9d700 1 --2- 172.21.15.31:0/3100183751 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9db006de50 0x7f9db006e2d0 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:08.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.911+0000 7f9db5f9d700 1 --2- 172.21.15.31:0/3100183751 >> v2:172.21.15.62:3300/0 conn(0x7f9db0155890 0x7f9db00c7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:08.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.911+0000 7f9db5f9d700 1 --2- 172.21.15.31:0/3100183751 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9db0155c60 0x7f9db006a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:08.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.911+0000 7f9db5f9d700 1 -- 172.21.15.31:0/3100183751 >> 172.21.15.31:0/3100183751 conn(0x7f9db006d110 msgr2=0x7f9db00cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:08.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.911+0000 7f9db5f9d700 1 -- 172.21.15.31:0/3100183751 shutdown_connections 2023-10-06T02:31:08.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.911+0000 7f9db5f9d700 1 -- 172.21.15.31:0/3100183751 wait complete. 2023-10-06T02:31:08.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.912+0000 7f9db5f9d700 1 Processor -- start 2023-10-06T02:31:08.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.913+0000 7f9db5f9d700 1 -- start start 2023-10-06T02:31:08.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.913+0000 7f9db5f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9db006de50 0x7f9db00a98e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:08.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.913+0000 7f9db5f9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9db0155890 0x7f9db0169270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:08.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.913+0000 7f9db5f9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9db0155c60 0x7f9db01697b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:08.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.913+0000 7f9db5f9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9db00c26b0 con 0x7f9db0155890 2023-10-06T02:31:08.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.913+0000 7f9db5f9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9db00c1aa0 con 0x7f9db0155c60 2023-10-06T02:31:08.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.913+0000 7f9db5f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9db00c1d80 con 0x7f9db006de50 2023-10-06T02:31:08.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.914+0000 7f9daffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9db0155890 0x7f9db0169270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:08.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.914+0000 7f9daffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9db0155890 0x7f9db0169270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37424/0 (socket says 172.21.15.31:37424) 2023-10-06T02:31:08.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.914+0000 7f9daffff700 1 -- 172.21.15.31:0/987170659 learned_addr learned my addr 172.21.15.31:0/987170659 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:08.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.914+0000 7f9db4f9b700 1 --2- 172.21.15.31:0/987170659 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9db006de50 0x7f9db00a98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:08.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.914+0000 7f9db579c700 1 --2- 172.21.15.31:0/987170659 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9db0155c60 0x7f9db01697b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:08.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.914+0000 7f9daffff700 1 -- 172.21.15.31:0/987170659 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9db006de50 msgr2=0x7f9db00a98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:08.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.914+0000 7f9daffff700 1 --2- 172.21.15.31:0/987170659 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9db006de50 0x7f9db00a98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:08.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.914+0000 7f9daffff700 1 -- 172.21.15.31:0/987170659 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9db0155c60 msgr2=0x7f9db01697b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:08.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.914+0000 7f9daffff700 1 --2- 172.21.15.31:0/987170659 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9db0155c60 0x7f9db01697b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:08.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.914+0000 7f9daffff700 1 -- 172.21.15.31:0/987170659 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9da800d040 con 0x7f9db0155890 2023-10-06T02:31:08.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.914+0000 7f9daffff700 1 --2- 172.21.15.31:0/987170659 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9db0155890 0x7f9db0169270 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c00ec80 tx=0x7f9d9c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:08.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.915+0000 7f9dacff9700 1 -- 172.21.15.31:0/987170659 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c00c9d0 con 0x7f9db0155890 2023-10-06T02:31:08.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.915+0000 7f9dacff9700 1 -- 172.21.15.31:0/987170659 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d9c00cb50 con 0x7f9db0155890 2023-10-06T02:31:08.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.915+0000 7f9dacff9700 1 -- 172.21.15.31:0/987170659 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c013470 con 0x7f9db0155890 2023-10-06T02:31:08.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.915+0000 7f9db5f9d700 1 -- 172.21.15.31:0/987170659 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9db00c2060 con 0x7f9db0155890 2023-10-06T02:31:08.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.915+0000 7f9db5f9d700 1 -- 172.21.15.31:0/987170659 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9db00a9e20 con 0x7f9db0155890 2023-10-06T02:31:08.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.916+0000 7f9db5f9d700 1 -- 172.21.15.31:0/987170659 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9db00c2860 con 0x7f9db0155890 2023-10-06T02:31:08.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.917+0000 7f9dacff9700 1 -- 172.21.15.31:0/987170659 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 85403+0+0 (secure 0 0 0) 0x7f9d9c013860 con 0x7f9db0155890 2023-10-06T02:31:08.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.918+0000 7f9dacff9700 1 --2- 172.21.15.31:0/987170659 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9d94067680 0x7f9d94069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:08.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.918+0000 7f9dacff9700 1 -- 172.21.15.31:0/987170659 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f9d9c085c20 con 0x7f9db0155890 2023-10-06T02:31:08.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.918+0000 7f9db4f9b700 1 --2- 172.21.15.31:0/987170659 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9d94067680 0x7f9d94069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:08.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.919+0000 7f9db4f9b700 1 --2- 172.21.15.31:0/987170659 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9d94067680 0x7f9d94069b30 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f9db01542d0 tx=0x7f9da4008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:08.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:08.922+0000 7f9dacff9700 1 -- 172.21.15.31:0/987170659 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9d9c082fe0 con 0x7f9db0155890 2023-10-06T02:31:09.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.123+0000 7f9db5f9d700 1 -- 172.21.15.31:0/987170659 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f9db004c3c0 con 0x7f9db0155890 2023-10-06T02:31:09.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.123+0000 7f9dacff9700 1 -- 172.21.15.31:0/987170659 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v144) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9d9c0170a0 con 0x7f9db0155890 2023-10-06T02:31:09.124 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 8 2023-10-06T02:31:09.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.126+0000 7f9db5f9d700 1 -- 172.21.15.31:0/987170659 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9d94067680 msgr2=0x7f9d94069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:09.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.126+0000 7f9db5f9d700 1 --2- 172.21.15.31:0/987170659 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9d94067680 0x7f9d94069b30 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f9db01542d0 tx=0x7f9da4008040 comp rx=0 tx=0).stop 2023-10-06T02:31:09.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.126+0000 7f9db5f9d700 1 -- 172.21.15.31:0/987170659 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9db0155890 msgr2=0x7f9db0169270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:09.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.126+0000 7f9db5f9d700 1 --2- 172.21.15.31:0/987170659 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9db0155890 0x7f9db0169270 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c00ec80 tx=0x7f9d9c0091a0 comp rx=0 tx=0).stop 2023-10-06T02:31:09.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.126+0000 7f9db5f9d700 1 -- 172.21.15.31:0/987170659 shutdown_connections 2023-10-06T02:31:09.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.126+0000 7f9db5f9d700 1 --2- 172.21.15.31:0/987170659 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9db0155c60 0x7f9db01697b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:09.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.126+0000 7f9db5f9d700 1 --2- 172.21.15.31:0/987170659 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9d94067680 0x7f9d94069b30 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:09.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.126+0000 7f9db5f9d700 1 --2- 172.21.15.31:0/987170659 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9db006de50 0x7f9db00a98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:09.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.126+0000 7f9db5f9d700 1 --2- 172.21.15.31:0/987170659 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9db0155890 0x7f9db0169270 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:09.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.126+0000 7f9db5f9d700 1 -- 172.21.15.31:0/987170659 >> 172.21.15.31:0/987170659 conn(0x7f9db006d110 msgr2=0x7f9db00b0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:09.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.126+0000 7f9db5f9d700 1 -- 172.21.15.31:0/987170659 shutdown_connections 2023-10-06T02:31:09.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.126+0000 7f9db5f9d700 1 -- 172.21.15.31:0/987170659 wait complete. 2023-10-06T02:31:09.145 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T02:31:09.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.335+0000 7ff9e559d700 1 Processor -- start 2023-10-06T02:31:09.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.335+0000 7ff9e559d700 1 -- start start 2023-10-06T02:31:09.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.336+0000 7ff9e559d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff9e0153dd0 0x7ff9e015de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:09.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.336+0000 7ff9e559d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff9e00cea00 0x7ff9e00ceea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:09.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.336+0000 7ff9e559d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9e015ac50 con 0x7ff9e0153a00 2023-10-06T02:31:09.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.336+0000 7ff9e559d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9e015d010 con 0x7ff9e00cea00 2023-10-06T02:31:09.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.336+0000 7ff9e559d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9e00cf3e0 con 0x7ff9e0153dd0 2023-10-06T02:31:09.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.337+0000 7ff9e4d9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff9e00cea00 0x7ff9e00ceea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:09.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.337+0000 7ff9e4d9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff9e00cea00 0x7ff9e00ceea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37426/0 (socket says 172.21.15.31:37426) 2023-10-06T02:31:09.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.337+0000 7ff9e4d9c700 1 -- 172.21.15.31:0/1621202744 learned_addr learned my addr 172.21.15.31:0/1621202744 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:09.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.337+0000 7ff9e4d9c700 1 -- 172.21.15.31:0/1621202744 >> v2:172.21.15.62:3300/0 conn(0x7ff9e0153dd0 msgr2=0x7ff9e015de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:09.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.337+0000 7ff9e4d9c700 1 --2- 172.21.15.31:0/1621202744 >> v2:172.21.15.62:3300/0 conn(0x7ff9e0153dd0 0x7ff9e015de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:09.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.337+0000 7ff9e4d9c700 1 -- 172.21.15.31:0/1621202744 >> v1:172.21.15.31:6789/0 conn(0x7ff9e0153a00 legacy=0x7ff9e015a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:09.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.337+0000 7ff9e4d9c700 1 -- 172.21.15.31:0/1621202744 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9e015af60 con 0x7ff9e00cea00 2023-10-06T02:31:09.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.337+0000 7ff9e4d9c700 1 --2- 172.21.15.31:0/1621202744 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9e00cea00 0x7ff9e00ceea0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7ff9e00b3070 tx=0x7ff9d8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e73502d2f5e91ab server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:09.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.337+0000 7ff9deffd700 1 -- 172.21.15.31:0/1621202744 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d800e070 con 0x7ff9e00cea00 2023-10-06T02:31:09.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.338+0000 7ff9deffd700 1 -- 172.21.15.31:0/1621202744 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9d800ba50 con 0x7ff9e00cea00 2023-10-06T02:31:09.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.338+0000 7ff9deffd700 1 -- 172.21.15.31:0/1621202744 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d80092e0 con 0x7ff9e00cea00 2023-10-06T02:31:09.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.338+0000 7ff9e559d700 1 -- 172.21.15.31:0/1621202744 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9e00cea00 msgr2=0x7ff9e00ceea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:09.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.338+0000 7ff9e559d700 1 --2- 172.21.15.31:0/1621202744 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9e00cea00 0x7ff9e00ceea0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7ff9e00b3070 tx=0x7ff9d8009840 comp rx=0 tx=0).stop 2023-10-06T02:31:09.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.338+0000 7ff9e559d700 1 -- 172.21.15.31:0/1621202744 shutdown_connections 2023-10-06T02:31:09.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.338+0000 7ff9e559d700 1 --2- 172.21.15.31:0/1621202744 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9e00cea00 0x7ff9e00ceea0 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:09.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.338+0000 7ff9e559d700 1 --2- 172.21.15.31:0/1621202744 >> v2:172.21.15.62:3300/0 conn(0x7ff9e0153dd0 0x7ff9e015de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:09.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.338+0000 7ff9e559d700 1 -- 172.21.15.31:0/1621202744 >> 172.21.15.31:0/1621202744 conn(0x7ff9e00703a0 msgr2=0x7ff9e00707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:09.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.338+0000 7ff9e559d700 1 -- 172.21.15.31:0/1621202744 shutdown_connections 2023-10-06T02:31:09.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.338+0000 7ff9e559d700 1 -- 172.21.15.31:0/1621202744 wait complete. 2023-10-06T02:31:09.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.340+0000 7ff9e559d700 1 Processor -- start 2023-10-06T02:31:09.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.340+0000 7ff9e559d700 1 -- start start 2023-10-06T02:31:09.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.341+0000 7ff9e559d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff9e00cea00 0x7ff9e0157550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:09.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.341+0000 7ff9e559d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9e0153a00 0x7ff9e0158810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:09.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.341+0000 7ff9e559d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff9e0153dd0 0x7ff9e0152f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:09.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.341+0000 7ff9e559d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e0158d50 con 0x7ff9e0153a00 2023-10-06T02:31:09.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.341+0000 7ff9e559d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e0157b30 con 0x7ff9e00cea00 2023-10-06T02:31:09.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.341+0000 7ff9e559d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e0157e10 con 0x7ff9e0153dd0 2023-10-06T02:31:09.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.341+0000 7ff9df7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9e0153a00 0x7ff9e0158810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:09.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.341+0000 7ff9df7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9e0153a00 0x7ff9e0158810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37434/0 (socket says 172.21.15.31:37434) 2023-10-06T02:31:09.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.341+0000 7ff9df7fe700 1 -- 172.21.15.31:0/1982436798 learned_addr learned my addr 172.21.15.31:0/1982436798 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:09.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.341+0000 7ff9dffff700 1 --2- 172.21.15.31:0/1982436798 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff9e00cea00 0x7ff9e0157550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:09.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.341+0000 7ff9e4d9c700 1 --2- 172.21.15.31:0/1982436798 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff9e0153dd0 0x7ff9e0152f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:09.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.341+0000 7ff9df7fe700 1 -- 172.21.15.31:0/1982436798 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff9e0153dd0 msgr2=0x7ff9e0152f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:09.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.341+0000 7ff9df7fe700 1 --2- 172.21.15.31:0/1982436798 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff9e0153dd0 0x7ff9e0152f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:09.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.341+0000 7ff9df7fe700 1 -- 172.21.15.31:0/1982436798 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff9e00cea00 msgr2=0x7ff9e0157550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:09.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.341+0000 7ff9df7fe700 1 --2- 172.21.15.31:0/1982436798 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff9e00cea00 0x7ff9e0157550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:09.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.341+0000 7ff9df7fe700 1 -- 172.21.15.31:0/1982436798 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9d8009ae0 con 0x7ff9e0153a00 2023-10-06T02:31:09.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.342+0000 7ff9df7fe700 1 --2- 172.21.15.31:0/1982436798 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9e0153a00 0x7ff9e0158810 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7ff9d400b5e0 tx=0x7ff9d40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:09.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.342+0000 7ff9c7fff700 1 -- 172.21.15.31:0/1982436798 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d400f510 con 0x7ff9e0153a00 2023-10-06T02:31:09.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.342+0000 7ff9c7fff700 1 -- 172.21.15.31:0/1982436798 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9d400f690 con 0x7ff9e0153a00 2023-10-06T02:31:09.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.342+0000 7ff9e559d700 1 -- 172.21.15.31:0/1982436798 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9e01580f0 con 0x7ff9e0153a00 2023-10-06T02:31:09.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.342+0000 7ff9c7fff700 1 -- 172.21.15.31:0/1982436798 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d400fa50 con 0x7ff9e0153a00 2023-10-06T02:31:09.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.342+0000 7ff9e559d700 1 -- 172.21.15.31:0/1982436798 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9e01543e0 con 0x7ff9e0153a00 2023-10-06T02:31:09.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.343+0000 7ff9e559d700 1 -- 172.21.15.31:0/1982436798 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9e004c3c0 con 0x7ff9e0153a00 2023-10-06T02:31:09.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.344+0000 7ff9c7fff700 1 -- 172.21.15.31:0/1982436798 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 85403+0+0 (secure 0 0 0) 0x7ff9d401a400 con 0x7ff9e0153a00 2023-10-06T02:31:09.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.345+0000 7ff9c7fff700 1 --2- 172.21.15.31:0/1982436798 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff9cc067680 0x7ff9cc069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:09.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.345+0000 7ff9c7fff700 1 -- 172.21.15.31:0/1982436798 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6086+0+0 (secure 0 0 0) 0x7ff9d4085400 con 0x7ff9e0153a00 2023-10-06T02:31:09.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.345+0000 7ff9dffff700 1 --2- 172.21.15.31:0/1982436798 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff9cc067680 0x7ff9cc069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:09.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.346+0000 7ff9dffff700 1 --2- 172.21.15.31:0/1982436798 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff9cc067680 0x7ff9cc069b30 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0003860 tx=0x7ff9d0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:09.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.349+0000 7ff9c7fff700 1 -- 172.21.15.31:0/1982436798 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff9d40827c0 con 0x7ff9e0153a00 2023-10-06T02:31:09.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.549+0000 7ff9e559d700 1 -- 172.21.15.31:0/1982436798 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff9e015b240 con 0x7ff9e0153a00 2023-10-06T02:31:09.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.550+0000 7ff9c7fff700 1 -- 172.21.15.31:0/1982436798 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v144) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff9d4051630 con 0x7ff9e0153a00 2023-10-06T02:31:09.551 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-10-06T02:31:09.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.552+0000 7ff9e559d700 1 -- 172.21.15.31:0/1982436798 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff9cc067680 msgr2=0x7ff9cc069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:09.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.552+0000 7ff9e559d700 1 --2- 172.21.15.31:0/1982436798 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff9cc067680 0x7ff9cc069b30 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0003860 tx=0x7ff9d0009040 comp rx=0 tx=0).stop 2023-10-06T02:31:09.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.552+0000 7ff9e559d700 1 -- 172.21.15.31:0/1982436798 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9e0153a00 msgr2=0x7ff9e0158810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:09.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.552+0000 7ff9e559d700 1 --2- 172.21.15.31:0/1982436798 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9e0153a00 0x7ff9e0158810 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7ff9d400b5e0 tx=0x7ff9d40091a0 comp rx=0 tx=0).stop 2023-10-06T02:31:09.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.552+0000 7ff9e559d700 1 -- 172.21.15.31:0/1982436798 shutdown_connections 2023-10-06T02:31:09.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.552+0000 7ff9e559d700 1 --2- 172.21.15.31:0/1982436798 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff9cc067680 0x7ff9cc069b30 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:09.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.552+0000 7ff9e559d700 1 --2- 172.21.15.31:0/1982436798 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff9e0153dd0 0x7ff9e0152f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:09.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.552+0000 7ff9e559d700 1 --2- 172.21.15.31:0/1982436798 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff9e00cea00 0x7ff9e0157550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:09.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.552+0000 7ff9e559d700 1 --2- 172.21.15.31:0/1982436798 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9e0153a00 0x7ff9e0158810 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:09.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.552+0000 7ff9e559d700 1 -- 172.21.15.31:0/1982436798 >> 172.21.15.31:0/1982436798 conn(0x7ff9e00703a0 msgr2=0x7ff9e00d0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:09.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.552+0000 7ff9e559d700 1 -- 172.21.15.31:0/1982436798 shutdown_connections 2023-10-06T02:31:09.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.553+0000 7ff9e559d700 1 -- 172.21.15.31:0/1982436798 wait complete. 2023-10-06T02:31:09.571 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T02:31:09.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.759+0000 7fc8fd99d700 1 Processor -- start 2023-10-06T02:31:09.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.760+0000 7fc8fd99d700 1 -- start start 2023-10-06T02:31:09.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.761+0000 7fc8fd99d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc8f80c7fb0 0x7fc8f806abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:09.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.761+0000 7fc8fd99d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8f806b290 con 0x7fc8f813f780 2023-10-06T02:31:09.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.761+0000 7fc8fd99d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8f806b570 con 0x7fc8f80c7be0 2023-10-06T02:31:09.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.761+0000 7fc8fd99d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f806bf10 con 0x7fc8f80c7fb0 2023-10-06T02:31:09.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.761+0000 7fc8fd19c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fc8f813f780 0x7fc8f80c9eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49724/0 (socket says 172.21.15.31:49724) 2023-10-06T02:31:09.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.761+0000 7fc8fd19c700 1 -- 172.21.15.31:0/3610926347 learned_addr learned my addr 172.21.15.31:0/3610926347 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:31:09.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.761+0000 7fc8f7fff700 1 --2- 172.21.15.31:0/3610926347 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc8f80c7fb0 0x7fc8f806abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:09.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.762+0000 7fc8f77fe700 1 -- 172.21.15.31:0/3610926347 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1361837891 0 0) 0x7fc8f806b290 con 0x7fc8f813f780 2023-10-06T02:31:09.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.762+0000 7fc8f77fe700 1 -- 172.21.15.31:0/3610926347 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8e8003570 con 0x7fc8f813f780 2023-10-06T02:31:09.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.762+0000 7fc8f77fe700 1 -- 172.21.15.31:0/3610926347 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2529625627 0 0) 0x7fc8f806b570 con 0x7fc8f80c7be0 2023-10-06T02:31:09.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.762+0000 7fc8f77fe700 1 -- 172.21.15.31:0/3610926347 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8f806b290 con 0x7fc8f80c7be0 2023-10-06T02:31:09.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.762+0000 7fc8f7fff700 1 -- 172.21.15.31:0/3610926347 >> v1:172.21.15.62:6789/0 conn(0x7fc8f80c7be0 legacy=0x7fc8f815a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:09.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.762+0000 7fc8f7fff700 1 -- 172.21.15.31:0/3610926347 >> v1:172.21.15.31:6789/0 conn(0x7fc8f813f780 legacy=0x7fc8f80c9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:09.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.762+0000 7fc8f7fff700 1 -- 172.21.15.31:0/3610926347 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8f806b850 con 0x7fc8f80c7fb0 2023-10-06T02:31:09.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.762+0000 7fc8f7fff700 1 --2- 172.21.15.31:0/3610926347 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc8f80c7fb0 0x7fc8f806abd0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc8e4009910 tx=0x7fc8e40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c3f0651aa748acd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:09.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.762+0000 7fc8f77fe700 1 -- 172.21.15.31:0/3610926347 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e4011630 con 0x7fc8f80c7fb0 2023-10-06T02:31:09.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.763+0000 7fc8f77fe700 1 -- 172.21.15.31:0/3610926347 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8e4011790 con 0x7fc8f80c7fb0 2023-10-06T02:31:09.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.763+0000 7fc8f77fe700 1 -- 172.21.15.31:0/3610926347 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e4011b30 con 0x7fc8f80c7fb0 2023-10-06T02:31:09.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.763+0000 7fc8fd99d700 1 -- 172.21.15.31:0/3610926347 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc8f80c7fb0 msgr2=0x7fc8f806abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:09.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.763+0000 7fc8fd99d700 1 --2- 172.21.15.31:0/3610926347 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc8f80c7fb0 0x7fc8f806abd0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc8e4009910 tx=0x7fc8e40091a0 comp rx=0 tx=0).stop 2023-10-06T02:31:09.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.763+0000 7fc8fd99d700 1 -- 172.21.15.31:0/3610926347 shutdown_connections 2023-10-06T02:31:09.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.763+0000 7fc8fd99d700 1 --2- 172.21.15.31:0/3610926347 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc8f80c7fb0 0x7fc8f806abd0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:09.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.763+0000 7fc8fd99d700 1 -- 172.21.15.31:0/3610926347 >> 172.21.15.31:0/3610926347 conn(0x7fc8f806d110 msgr2=0x7fc8f80cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:09.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.763+0000 7fc8fd99d700 1 -- 172.21.15.31:0/3610926347 shutdown_connections 2023-10-06T02:31:09.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.763+0000 7fc8fd99d700 1 -- 172.21.15.31:0/3610926347 wait complete. 2023-10-06T02:31:09.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.765+0000 7fc8fd99d700 1 Processor -- start 2023-10-06T02:31:09.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.765+0000 7fc8fd99d700 1 -- start start 2023-10-06T02:31:09.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.766+0000 7fc8fd99d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8f80c7be0 0x7fc8f80a54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:09.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.766+0000 7fc8fd99d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8f80c7fb0 0x7fc8f815bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:09.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.766+0000 7fc8fd99d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc8f813f780 0x7fc8f815c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:09.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.766+0000 7fc8fd99d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f815cf60 con 0x7fc8f80c7fb0 2023-10-06T02:31:09.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.766+0000 7fc8fd99d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f8169520 con 0x7fc8f80c7be0 2023-10-06T02:31:09.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.766+0000 7fc8fd99d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f80a5a10 con 0x7fc8f813f780 2023-10-06T02:31:09.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.766+0000 7fc8f7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8f80c7fb0 0x7fc8f815bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:09.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.766+0000 7fc8f7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8f80c7fb0 0x7fc8f815bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37442/0 (socket says 172.21.15.31:37442) 2023-10-06T02:31:09.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.766+0000 7fc8f7fff700 1 -- 172.21.15.31:0/2703546596 learned_addr learned my addr 172.21.15.31:0/2703546596 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:09.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.766+0000 7fc8fd19c700 1 --2- 172.21.15.31:0/2703546596 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc8f813f780 0x7fc8f815c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:09.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.766+0000 7fc8f7fff700 1 -- 172.21.15.31:0/2703546596 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc8f813f780 msgr2=0x7fc8f815c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:09.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.766+0000 7fc8f7fff700 1 --2- 172.21.15.31:0/2703546596 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc8f813f780 0x7fc8f815c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:09.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.766+0000 7fc8f7fff700 1 -- 172.21.15.31:0/2703546596 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8f80c7be0 msgr2=0x7fc8f80a54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:09.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.766+0000 7fc8fc99b700 1 --2- 172.21.15.31:0/2703546596 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8f80c7be0 0x7fc8f80a54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:09.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.766+0000 7fc8f7fff700 1 --2- 172.21.15.31:0/2703546596 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8f80c7be0 0x7fc8f80a54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:09.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.766+0000 7fc8f7fff700 1 -- 172.21.15.31:0/2703546596 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8e4009420 con 0x7fc8f80c7fb0 2023-10-06T02:31:09.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.766+0000 7fc8fd19c700 1 --2- 172.21.15.31:0/2703546596 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc8f813f780 0x7fc8f815c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:31:09.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.766+0000 7fc8fc99b700 1 --2- 172.21.15.31:0/2703546596 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8f80c7be0 0x7fc8f80a54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:31:09.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.767+0000 7fc8f7fff700 1 --2- 172.21.15.31:0/2703546596 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8f80c7fb0 0x7fc8f815bbd0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fc8e4009fd0 tx=0x7fc8e4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:09.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.767+0000 7fc8f4ff9700 1 -- 172.21.15.31:0/2703546596 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e401f070 con 0x7fc8f80c7fb0 2023-10-06T02:31:09.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.767+0000 7fc8f4ff9700 1 -- 172.21.15.31:0/2703546596 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8e4000bc0 con 0x7fc8f80c7fb0 2023-10-06T02:31:09.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.767+0000 7fc8fd99d700 1 -- 172.21.15.31:0/2703546596 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8f80c1d70 con 0x7fc8f80c7fb0 2023-10-06T02:31:09.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.767+0000 7fc8f4ff9700 1 -- 172.21.15.31:0/2703546596 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e4025420 con 0x7fc8f80c7fb0 2023-10-06T02:31:09.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.767+0000 7fc8fd99d700 1 -- 172.21.15.31:0/2703546596 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8f80c2330 con 0x7fc8f80c7fb0 2023-10-06T02:31:09.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.768+0000 7fc8fd99d700 1 -- 172.21.15.31:0/2703546596 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8f81696e0 con 0x7fc8f80c7fb0 2023-10-06T02:31:09.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.769+0000 7fc8f4ff9700 1 -- 172.21.15.31:0/2703546596 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 85403+0+0 (secure 0 0 0) 0x7fc8e4023070 con 0x7fc8f80c7fb0 2023-10-06T02:31:09.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.770+0000 7fc8f4ff9700 1 --2- 172.21.15.31:0/2703546596 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc8dc067680 0x7fc8dc069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:09.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.770+0000 7fc8f4ff9700 1 -- 172.21.15.31:0/2703546596 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fc8e4086c60 con 0x7fc8f80c7fb0 2023-10-06T02:31:09.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.770+0000 7fc8fc99b700 1 --2- 172.21.15.31:0/2703546596 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc8dc067680 0x7fc8dc069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:09.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.771+0000 7fc8fc99b700 1 --2- 172.21.15.31:0/2703546596 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc8dc067680 0x7fc8dc069b30 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec007fd0 tx=0x7fc8ec009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:09.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.774+0000 7fc8f4ff9700 1 -- 172.21.15.31:0/2703546596 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc8f81696e0 con 0x7fc8f80c7fb0 2023-10-06T02:31:09.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.978+0000 7fc8fd99d700 1 -- 172.21.15.31:0/2703546596 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc8f81696e0 con 0x7fc8f80c7fb0 2023-10-06T02:31:09.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.978+0000 7fc8f4ff9700 1 -- 172.21.15.31:0/2703546596 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v144) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc8f81696e0 con 0x7fc8f80c7fb0 2023-10-06T02:31:09.981 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-10-06T02:31:09.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.980+0000 7fc8fd99d700 1 -- 172.21.15.31:0/2703546596 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc8dc067680 msgr2=0x7fc8dc069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:09.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.980+0000 7fc8fd99d700 1 --2- 172.21.15.31:0/2703546596 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc8dc067680 0x7fc8dc069b30 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec007fd0 tx=0x7fc8ec009040 comp rx=0 tx=0).stop 2023-10-06T02:31:09.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.980+0000 7fc8fd99d700 1 -- 172.21.15.31:0/2703546596 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8f80c7fb0 msgr2=0x7fc8f815bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:09.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.980+0000 7fc8fd99d700 1 --2- 172.21.15.31:0/2703546596 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8f80c7fb0 0x7fc8f815bbd0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fc8e4009fd0 tx=0x7fc8e4017000 comp rx=0 tx=0).stop 2023-10-06T02:31:09.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.981+0000 7fc8fd99d700 1 -- 172.21.15.31:0/2703546596 shutdown_connections 2023-10-06T02:31:09.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.981+0000 7fc8fd99d700 1 --2- 172.21.15.31:0/2703546596 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc8dc067680 0x7fc8dc069b30 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:09.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.981+0000 7fc8fd99d700 1 --2- 172.21.15.31:0/2703546596 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc8f813f780 0x7fc8f815c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:09.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.981+0000 7fc8fd99d700 1 --2- 172.21.15.31:0/2703546596 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8f80c7be0 0x7fc8f80a54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:09.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.981+0000 7fc8fd99d700 1 --2- 172.21.15.31:0/2703546596 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8f80c7fb0 0x7fc8f815bbd0 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:09.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.981+0000 7fc8fd99d700 1 -- 172.21.15.31:0/2703546596 >> 172.21.15.31:0/2703546596 conn(0x7fc8f806d110 msgr2=0x7fc8f80c9db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:09.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.981+0000 7fc8fd99d700 1 -- 172.21.15.31:0/2703546596 shutdown_connections 2023-10-06T02:31:09.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:09.981+0000 7fc8fd99d700 1 -- 172.21.15.31:0/2703546596 wait complete. 2023-10-06T02:31:09.999 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:31:10.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.186+0000 7fb8e239d700 1 Processor -- start 2023-10-06T02:31:10.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.186+0000 7fb8e239d700 1 -- start start 2023-10-06T02:31:10.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.188+0000 7fb8e239d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb8dc06e0a0 0x7fb8dc0c6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:10.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.188+0000 7fb8e239d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb8dc150380 0x7fb8dc06a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:10.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.188+0000 7fb8e239d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8dc06a490 con 0x7fb8dc06f3a0 2023-10-06T02:31:10.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.188+0000 7fb8e239d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8dc0c7490 con 0x7fb8dc150380 2023-10-06T02:31:10.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.188+0000 7fb8e239d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8dc0c7770 con 0x7fb8dc06e0a0 2023-10-06T02:31:10.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.188+0000 7fb8e1b9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb8dc150380 0x7fb8dc06a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:10.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.188+0000 7fb8e1b9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb8dc150380 0x7fb8dc06a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37450/0 (socket says 172.21.15.31:37450) 2023-10-06T02:31:10.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.188+0000 7fb8e1b9c700 1 -- 172.21.15.31:0/1125593766 learned_addr learned my addr 172.21.15.31:0/1125593766 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:10.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.188+0000 7fb8e139b700 1 --2- 172.21.15.31:0/1125593766 >> v2:172.21.15.62:3300/0 conn(0x7fb8dc06e0a0 0x7fb8dc0c6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:10.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.188+0000 7fb8e1b9c700 1 -- 172.21.15.31:0/1125593766 >> v2:172.21.15.62:3300/0 conn(0x7fb8dc06e0a0 msgr2=0x7fb8dc0c6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:10.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.188+0000 7fb8e1b9c700 1 --2- 172.21.15.31:0/1125593766 >> v2:172.21.15.62:3300/0 conn(0x7fb8dc06e0a0 0x7fb8dc0c6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:10.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.188+0000 7fb8e1b9c700 1 -- 172.21.15.31:0/1125593766 >> v1:172.21.15.62:6789/0 conn(0x7fb8dc06f3a0 legacy=0x7fb8dc14fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:10.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.189+0000 7fb8e1b9c700 1 -- 172.21.15.31:0/1125593766 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8dc0c7a50 con 0x7fb8dc150380 2023-10-06T02:31:10.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.189+0000 7fb8e1b9c700 1 --2- 172.21.15.31:0/1125593766 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8dc150380 0x7fb8dc06a920 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc069230 tx=0x7fb8d400a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81eb3491e9e5f75d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:10.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.189+0000 7fb8d3fff700 1 -- 172.21.15.31:0/1125593766 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d400f070 con 0x7fb8dc150380 2023-10-06T02:31:10.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.189+0000 7fb8d3fff700 1 -- 172.21.15.31:0/1125593766 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8d400cdc0 con 0x7fb8dc150380 2023-10-06T02:31:10.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.189+0000 7fb8d3fff700 1 -- 172.21.15.31:0/1125593766 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d400a320 con 0x7fb8dc150380 2023-10-06T02:31:10.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.189+0000 7fb8e239d700 1 -- 172.21.15.31:0/1125593766 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8dc150380 msgr2=0x7fb8dc06a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:10.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.189+0000 7fb8e239d700 1 --2- 172.21.15.31:0/1125593766 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8dc150380 0x7fb8dc06a920 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc069230 tx=0x7fb8d400a950 comp rx=0 tx=0).stop 2023-10-06T02:31:10.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.190+0000 7fb8e239d700 1 -- 172.21.15.31:0/1125593766 shutdown_connections 2023-10-06T02:31:10.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.190+0000 7fb8e239d700 1 --2- 172.21.15.31:0/1125593766 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8dc150380 0x7fb8dc06a920 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:10.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.190+0000 7fb8e239d700 1 --2- 172.21.15.31:0/1125593766 >> v2:172.21.15.62:3300/0 conn(0x7fb8dc06e0a0 0x7fb8dc0c6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:10.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.190+0000 7fb8e239d700 1 -- 172.21.15.31:0/1125593766 >> 172.21.15.31:0/1125593766 conn(0x7fb8dc06d170 msgr2=0x7fb8dc0cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:10.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.190+0000 7fb8e239d700 1 -- 172.21.15.31:0/1125593766 shutdown_connections 2023-10-06T02:31:10.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.190+0000 7fb8e239d700 1 -- 172.21.15.31:0/1125593766 wait complete. 2023-10-06T02:31:10.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.191+0000 7fb8e239d700 1 Processor -- start 2023-10-06T02:31:10.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.191+0000 7fb8e239d700 1 -- start start 2023-10-06T02:31:10.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.192+0000 7fb8e239d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb8dc06e0a0 0x7fb8dc149a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:10.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.192+0000 7fb8e239d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8dc06f3a0 0x7fb8dc0b0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:10.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.192+0000 7fb8e239d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8dc150380 0x7fb8dc15ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:10.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.192+0000 7fb8e239d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8dc149fb0 con 0x7fb8dc150380 2023-10-06T02:31:10.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.192+0000 7fb8e239d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8dc167dc0 con 0x7fb8dc06f3a0 2023-10-06T02:31:10.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.192+0000 7fb8e239d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8dc0b1190 con 0x7fb8dc06e0a0 2023-10-06T02:31:10.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.193+0000 7fb8e1b9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8dc150380 0x7fb8dc15ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:10.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.193+0000 7fb8e139b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb8dc06e0a0 0x7fb8dc149a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:10.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.193+0000 7fb8e1b9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8dc150380 0x7fb8dc15ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37462/0 (socket says 172.21.15.31:37462) 2023-10-06T02:31:10.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.193+0000 7fb8e1b9c700 1 -- 172.21.15.31:0/2069264192 learned_addr learned my addr 172.21.15.31:0/2069264192 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:10.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.193+0000 7fb8e0b9a700 1 --2- 172.21.15.31:0/2069264192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8dc06f3a0 0x7fb8dc0b0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:10.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.193+0000 7fb8e1b9c700 1 -- 172.21.15.31:0/2069264192 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb8dc06e0a0 msgr2=0x7fb8dc149a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:10.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.193+0000 7fb8e1b9c700 1 --2- 172.21.15.31:0/2069264192 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb8dc06e0a0 0x7fb8dc149a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:10.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.193+0000 7fb8e1b9c700 1 -- 172.21.15.31:0/2069264192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8dc06f3a0 msgr2=0x7fb8dc0b0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:10.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.193+0000 7fb8e1b9c700 1 --2- 172.21.15.31:0/2069264192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8dc06f3a0 0x7fb8dc0b0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:10.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.193+0000 7fb8e1b9c700 1 -- 172.21.15.31:0/2069264192 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8d400abf0 con 0x7fb8dc150380 2023-10-06T02:31:10.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.193+0000 7fb8e0b9a700 1 --2- 172.21.15.31:0/2069264192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8dc06f3a0 0x7fb8dc0b0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:31:10.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.193+0000 7fb8e1b9c700 1 --2- 172.21.15.31:0/2069264192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8dc150380 0x7fb8dc15ae00 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7fb8d4013000 tx=0x7fb8d4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:10.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.193+0000 7fb8d17fa700 1 -- 172.21.15.31:0/2069264192 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d40082a0 con 0x7fb8dc150380 2023-10-06T02:31:10.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.194+0000 7fb8d17fa700 1 -- 172.21.15.31:0/2069264192 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8d401d440 con 0x7fb8dc150380 2023-10-06T02:31:10.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.194+0000 7fb8d17fa700 1 -- 172.21.15.31:0/2069264192 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d401d800 con 0x7fb8dc150380 2023-10-06T02:31:10.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.194+0000 7fb8e239d700 1 -- 172.21.15.31:0/2069264192 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8dc0b1470 con 0x7fb8dc150380 2023-10-06T02:31:10.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.194+0000 7fb8e239d700 1 -- 172.21.15.31:0/2069264192 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8dc0b1a30 con 0x7fb8dc150380 2023-10-06T02:31:10.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.195+0000 7fb8e239d700 1 -- 172.21.15.31:0/2069264192 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8dc06a820 con 0x7fb8dc150380 2023-10-06T02:31:10.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.196+0000 7fb8d17fa700 1 -- 172.21.15.31:0/2069264192 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 85403+0+0 (secure 0 0 0) 0x7fb8d4003040 con 0x7fb8dc150380 2023-10-06T02:31:10.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.196+0000 7fb8d17fa700 1 --2- 172.21.15.31:0/2069264192 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb8c0067680 0x7fb8c0069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:10.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.196+0000 7fb8d17fa700 1 -- 172.21.15.31:0/2069264192 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fb8d40876d0 con 0x7fb8dc150380 2023-10-06T02:31:10.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.197+0000 7fb8e139b700 1 --2- 172.21.15.31:0/2069264192 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb8c0067680 0x7fb8c0069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:10.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.197+0000 7fb8e139b700 1 --2- 172.21.15.31:0/2069264192 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb8c0067680 0x7fb8c0069b30 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb8d8006fd0 tx=0x7fb8d8008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:10.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.201+0000 7fb8d17fa700 1 -- 172.21.15.31:0/2069264192 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb8d4084980 con 0x7fb8dc150380 2023-10-06T02:31:10.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.449+0000 7fb8e239d700 1 -- 172.21.15.31:0/2069264192 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb8dc04c3c0 con 0x7fb8dc150380 2023-10-06T02:31:10.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.450+0000 7fb8d17fa700 1 -- 172.21.15.31:0/2069264192 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) v1 ==== 95+0+4603 (secure 0 0 0) 0x7fb8d4053770 con 0x7fb8dc150380 2023-10-06T02:31:10.450 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:31:10.451 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":101,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6923,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/4169296779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":4169296779},{"type":"v1","addr":"172.21.15.31:6835","nonce":4169296779}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95},{"gid":6926,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/3032245108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3032245108},{"type":"v1","addr":"172.21.15.62:6834","nonce":3032245108}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95},{"gid":8469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6833/1522769465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1522769465},{"type":"v1","addr":"172.21.15.31:6833","nonce":1522769465}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:30:51.153736+0000","modified":"2023-10-06T02:30:55.096858+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8487":{"gid":8487,"name":"d","rank":0,"incarnation":96,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/2236458886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2236458886},{"type":"v1","addr":"172.21.15.62:6835","nonce":2236458886}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2023-10-06T02:31:10.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.452+0000 7fb8e239d700 1 -- 172.21.15.31:0/2069264192 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb8c0067680 msgr2=0x7fb8c0069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:10.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.452+0000 7fb8e239d700 1 --2- 172.21.15.31:0/2069264192 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb8c0067680 0x7fb8c0069b30 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb8d8006fd0 tx=0x7fb8d8008040 comp rx=0 tx=0).stop 2023-10-06T02:31:10.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.452+0000 7fb8e239d700 1 -- 172.21.15.31:0/2069264192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8dc150380 msgr2=0x7fb8dc15ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:10.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.452+0000 7fb8e239d700 1 --2- 172.21.15.31:0/2069264192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8dc150380 0x7fb8dc15ae00 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7fb8d4013000 tx=0x7fb8d4019000 comp rx=0 tx=0).stop 2023-10-06T02:31:10.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.452+0000 7fb8e239d700 1 -- 172.21.15.31:0/2069264192 shutdown_connections 2023-10-06T02:31:10.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.452+0000 7fb8e239d700 1 --2- 172.21.15.31:0/2069264192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8dc150380 0x7fb8dc15ae00 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:10.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.452+0000 7fb8e239d700 1 --2- 172.21.15.31:0/2069264192 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb8c0067680 0x7fb8c0069b30 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:10.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.452+0000 7fb8e239d700 1 --2- 172.21.15.31:0/2069264192 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb8dc06e0a0 0x7fb8dc149a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:10.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.452+0000 7fb8e239d700 1 --2- 172.21.15.31:0/2069264192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8dc06f3a0 0x7fb8dc0b0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:10.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.452+0000 7fb8e239d700 1 -- 172.21.15.31:0/2069264192 >> 172.21.15.31:0/2069264192 conn(0x7fb8dc06d170 msgr2=0x7fb8dc14dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:10.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.452+0000 7fb8e239d700 1 -- 172.21.15.31:0/2069264192 shutdown_connections 2023-10-06T02:31:10.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.452+0000 7fb8e239d700 1 -- 172.21.15.31:0/2069264192 wait complete. 2023-10-06T02:31:10.454 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 101 2023-10-06T02:31:10.470 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:31:10.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.658+0000 7f827179d700 1 Processor -- start 2023-10-06T02:31:10.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.658+0000 7f827179d700 1 -- start start 2023-10-06T02:31:10.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.659+0000 7f827179d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f826c153c30 0x7f826c15a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:10.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.659+0000 7f827179d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f826c0cf110 0x7f826c15c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:10.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.659+0000 7f827179d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f826c15afd0 con 0x7f826c15de50 2023-10-06T02:31:10.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.659+0000 7f827179d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f826c15b2b0 con 0x7f826c0cf110 2023-10-06T02:31:10.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.659+0000 7f827179d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f826c15b590 con 0x7f826c153c30 2023-10-06T02:31:10.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.660+0000 7f8270f9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f826c0cf110 0x7f826c15c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:10.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.660+0000 7f8270f9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f826c0cf110 0x7f826c15c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37464/0 (socket says 172.21.15.31:37464) 2023-10-06T02:31:10.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.660+0000 7f8270f9c700 1 -- 172.21.15.31:0/2583061603 learned_addr learned my addr 172.21.15.31:0/2583061603 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:10.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.660+0000 7f826bfff700 1 --2- 172.21.15.31:0/2583061603 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f826c153c30 0x7f826c15a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:10.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.660+0000 7f8270f9c700 1 -- 172.21.15.31:0/2583061603 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f826c153c30 msgr2=0x7f826c15a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:10.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.660+0000 7f8270f9c700 1 --2- 172.21.15.31:0/2583061603 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f826c153c30 0x7f826c15a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:10.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.660+0000 7f8270f9c700 1 -- 172.21.15.31:0/2583061603 >> v1:172.21.15.62:6789/0 conn(0x7f826c15de50 legacy=0x7f826c0cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:10.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.660+0000 7f8270f9c700 1 -- 172.21.15.31:0/2583061603 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f826c15b870 con 0x7f826c0cf110 2023-10-06T02:31:10.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.660+0000 7f8270f9c700 1 --2- 172.21.15.31:0/2583061603 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f826c0cf110 0x7f826c15c070 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f8264009f20 tx=0x7f8264009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a96c4c5be463e192 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:10.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.661+0000 7f826affd700 1 -- 172.21.15.31:0/2583061603 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f826400e070 con 0x7f826c0cf110 2023-10-06T02:31:10.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.661+0000 7f826affd700 1 -- 172.21.15.31:0/2583061603 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f826400ba80 con 0x7f826c0cf110 2023-10-06T02:31:10.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.661+0000 7f826affd700 1 -- 172.21.15.31:0/2583061603 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82640092e0 con 0x7f826c0cf110 2023-10-06T02:31:10.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.661+0000 7f827179d700 1 -- 172.21.15.31:0/2583061603 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f826c0cf110 msgr2=0x7f826c15c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:10.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.661+0000 7f827179d700 1 --2- 172.21.15.31:0/2583061603 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f826c0cf110 0x7f826c15c070 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f8264009f20 tx=0x7f8264009840 comp rx=0 tx=0).stop 2023-10-06T02:31:10.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.661+0000 7f827179d700 1 -- 172.21.15.31:0/2583061603 shutdown_connections 2023-10-06T02:31:10.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.661+0000 7f827179d700 1 --2- 172.21.15.31:0/2583061603 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f826c0cf110 0x7f826c15c070 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:10.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.661+0000 7f827179d700 1 --2- 172.21.15.31:0/2583061603 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f826c153c30 0x7f826c15a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:10.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.661+0000 7f827179d700 1 -- 172.21.15.31:0/2583061603 >> 172.21.15.31:0/2583061603 conn(0x7f826c0703a0 msgr2=0x7f826c0707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:10.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.661+0000 7f827179d700 1 -- 172.21.15.31:0/2583061603 shutdown_connections 2023-10-06T02:31:10.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.661+0000 7f827179d700 1 -- 172.21.15.31:0/2583061603 wait complete. 2023-10-06T02:31:10.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.663+0000 7f827179d700 1 Processor -- start 2023-10-06T02:31:10.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.663+0000 7f827179d700 1 -- start start 2023-10-06T02:31:10.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.664+0000 7f827179d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f826c0cf110 0x7f826c0c9f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:10.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.664+0000 7f827179d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f826c153c30 0x7f826c153020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:10.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.664+0000 7f827179d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f826c15de50 0x7f826c0b5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:10.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.664+0000 7f827179d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f826c0ca440 con 0x7f826c153c30 2023-10-06T02:31:10.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.664+0000 7f827179d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f826c0c9a70 con 0x7f826c15de50 2023-10-06T02:31:10.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.664+0000 7f827179d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f826c0c8e60 con 0x7f826c0cf110 2023-10-06T02:31:10.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.664+0000 7f826b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f826c153c30 0x7f826c153020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:10.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.664+0000 7f826b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f826c153c30 0x7f826c153020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37474/0 (socket says 172.21.15.31:37474) 2023-10-06T02:31:10.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.664+0000 7f826b7fe700 1 -- 172.21.15.31:0/3114611885 learned_addr learned my addr 172.21.15.31:0/3114611885 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:10.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.664+0000 7f8270f9c700 1 --2- 172.21.15.31:0/3114611885 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f826c15de50 0x7f826c0b5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:10.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.664+0000 7f826bfff700 1 --2- 172.21.15.31:0/3114611885 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f826c0cf110 0x7f826c0c9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:10.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.665+0000 7f826b7fe700 1 -- 172.21.15.31:0/3114611885 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f826c0cf110 msgr2=0x7f826c0c9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:10.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.665+0000 7f826b7fe700 1 --2- 172.21.15.31:0/3114611885 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f826c0cf110 0x7f826c0c9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:10.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.665+0000 7f826b7fe700 1 -- 172.21.15.31:0/3114611885 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f826c15de50 msgr2=0x7f826c0b5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:10.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.665+0000 7f826b7fe700 1 --2- 172.21.15.31:0/3114611885 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f826c15de50 0x7f826c0b5bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:10.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.665+0000 7f826b7fe700 1 -- 172.21.15.31:0/3114611885 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8264009ae0 con 0x7f826c153c30 2023-10-06T02:31:10.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.665+0000 7f826bfff700 1 --2- 172.21.15.31:0/3114611885 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f826c0cf110 0x7f826c0c9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:31:10.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.665+0000 7f826b7fe700 1 --2- 172.21.15.31:0/3114611885 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f826c153c30 0x7f826c153020 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f826000c4e0 tx=0x7f826000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:10.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.665+0000 7f8253fff700 1 -- 172.21.15.31:0/3114611885 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8260011590 con 0x7f826c153c30 2023-10-06T02:31:10.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.665+0000 7f8253fff700 1 -- 172.21.15.31:0/3114611885 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8260011710 con 0x7f826c153c30 2023-10-06T02:31:10.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.665+0000 7f827179d700 1 -- 172.21.15.31:0/3114611885 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f826c0c9140 con 0x7f826c153c30 2023-10-06T02:31:10.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.665+0000 7f8253fff700 1 -- 172.21.15.31:0/3114611885 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8260011ad0 con 0x7f826c153c30 2023-10-06T02:31:10.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.665+0000 7f827179d700 1 -- 172.21.15.31:0/3114611885 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f826c0c9700 con 0x7f826c153c30 2023-10-06T02:31:10.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.666+0000 7f827179d700 1 -- 172.21.15.31:0/3114611885 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f826c0c9be0 con 0x7f826c153c30 2023-10-06T02:31:10.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.667+0000 7f8253fff700 1 -- 172.21.15.31:0/3114611885 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 85403+0+0 (secure 0 0 0) 0x7f826001b3b0 con 0x7f826c153c30 2023-10-06T02:31:10.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.668+0000 7f8253fff700 1 --2- 172.21.15.31:0/3114611885 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f8258067680 0x7f8258069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:10.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.668+0000 7f8253fff700 1 -- 172.21.15.31:0/3114611885 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f8260084cf0 con 0x7f826c153c30 2023-10-06T02:31:10.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.669+0000 7f826bfff700 1 --2- 172.21.15.31:0/3114611885 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f8258067680 0x7f8258069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:10.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.669+0000 7f826bfff700 1 --2- 172.21.15.31:0/3114611885 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f8258067680 0x7f8258069b30 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f825c006fd0 tx=0x7f825c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:10.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.672+0000 7f8253fff700 1 -- 172.21.15.31:0/3114611885 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f82600820b0 con 0x7f826c153c30 2023-10-06T02:31:10.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.878+0000 7f827179d700 1 -- 172.21.15.31:0/3114611885 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f826c04c3c0 con 0x7f826c153c30 2023-10-06T02:31:10.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.879+0000 7f8253fff700 1 -- 172.21.15.31:0/3114611885 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v144) v1 ==== 74+0+16876 (secure 0 0 0) 0x7f8260017070 con 0x7f826c153c30 2023-10-06T02:31:10.880 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:31:10.880 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":144,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:30:52.650397+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:30:49.325914+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":24,"pool_name":"cephfs_data","create_time":"2023-10-06T02:30:50.127083+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":20,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:31:10.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.881+0000 7f827179d700 1 -- 172.21.15.31:0/3114611885 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f8258067680 msgr2=0x7f8258069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:10.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.881+0000 7f827179d700 1 --2- 172.21.15.31:0/3114611885 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f8258067680 0x7f8258069b30 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f825c006fd0 tx=0x7f825c008040 comp rx=0 tx=0).stop 2023-10-06T02:31:10.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.881+0000 7f827179d700 1 -- 172.21.15.31:0/3114611885 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f826c153c30 msgr2=0x7f826c153020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:10.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.881+0000 7f827179d700 1 --2- 172.21.15.31:0/3114611885 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f826c153c30 0x7f826c153020 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f826000c4e0 tx=0x7f826000c8a0 comp rx=0 tx=0).stop 2023-10-06T02:31:10.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.881+0000 7f827179d700 1 -- 172.21.15.31:0/3114611885 shutdown_connections 2023-10-06T02:31:10.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.881+0000 7f827179d700 1 --2- 172.21.15.31:0/3114611885 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f826c15de50 0x7f826c0b5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:10.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.881+0000 7f827179d700 1 --2- 172.21.15.31:0/3114611885 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f8258067680 0x7f8258069b30 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:10.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.881+0000 7f827179d700 1 --2- 172.21.15.31:0/3114611885 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f826c0cf110 0x7f826c0c9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:10.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.881+0000 7f827179d700 1 --2- 172.21.15.31:0/3114611885 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f826c153c30 0x7f826c153020 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:10.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.881+0000 7f827179d700 1 -- 172.21.15.31:0/3114611885 >> 172.21.15.31:0/3114611885 conn(0x7f826c0703a0 msgr2=0x7f826c070770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:10.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.881+0000 7f827179d700 1 -- 172.21.15.31:0/3114611885 shutdown_connections 2023-10-06T02:31:10.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:10.881+0000 7f827179d700 1 -- 172.21.15.31:0/3114611885 wait complete. 2023-10-06T02:31:10.900 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T02:31:10.900 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:31:11.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.088+0000 7f9322f9d700 1 Processor -- start 2023-10-06T02:31:11.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.088+0000 7f9322f9d700 1 -- start start 2023-10-06T02:31:11.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.089+0000 7f9322f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f931c13fb20 0x7f931c15a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:11.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.089+0000 7f9322f9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f931c06aaa0 0x7f931c0c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:11.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.089+0000 7f9322f9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f931c15ae70 con 0x7f931c13fef0 2023-10-06T02:31:11.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.089+0000 7f9322f9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f931c06af60 con 0x7f931c06aaa0 2023-10-06T02:31:11.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.089+0000 7f9322f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f931c06b240 con 0x7f931c13fb20 2023-10-06T02:31:11.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.090+0000 7f9321f9b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f931c13fb20 0x7f931c15a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:11.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.090+0000 7f932279c700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f931c06aaa0 0x7f931c0c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:11.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.090+0000 7f9321f9b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f931c13fb20 0x7f931c15a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:35664/0 (socket says 172.21.15.31:35664) 2023-10-06T02:31:11.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.090+0000 7f9321f9b700 1 -- 172.21.15.31:0/3532885233 learned_addr learned my addr 172.21.15.31:0/3532885233 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:11.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.090+0000 7f932179a700 1 --1- 172.21.15.31:0/3532885233 >> v1:172.21.15.62:6789/0 conn(0x7f931c13fef0 0x7f931c0c7990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:57430/0 (socket says 172.21.15.31:57430) 2023-10-06T02:31:11.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.090+0000 7f9321f9b700 1 -- 172.21.15.31:0/3532885233 >> v2:172.21.15.62:3300/0 conn(0x7f931c06aaa0 msgr2=0x7f931c0c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:11.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.090+0000 7f9321f9b700 1 --2- 172.21.15.31:0/3532885233 >> v2:172.21.15.62:3300/0 conn(0x7f931c06aaa0 0x7f931c0c9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:11.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.090+0000 7f9321f9b700 1 -- 172.21.15.31:0/3532885233 >> v1:172.21.15.62:6789/0 conn(0x7f931c13fef0 legacy=0x7f931c0c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:11.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.090+0000 7f9321f9b700 1 -- 172.21.15.31:0/3532885233 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f931c06b520 con 0x7f931c13fb20 2023-10-06T02:31:11.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.091+0000 7f9321f9b700 1 --2- 172.21.15.31:0/3532885233 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f931c13fb20 0x7f931c15a930 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f93180099b0 tx=0x7f9318009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70634341481db0f6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:11.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.091+0000 7f9320f99700 1 -- 172.21.15.31:0/3532885233 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9318009ea0 con 0x7f931c13fb20 2023-10-06T02:31:11.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.091+0000 7f9320f99700 1 -- 172.21.15.31:0/3532885233 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f931800b860 con 0x7f931c13fb20 2023-10-06T02:31:11.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.091+0000 7f9320f99700 1 -- 172.21.15.31:0/3532885233 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931800bc00 con 0x7f931c13fb20 2023-10-06T02:31:11.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.091+0000 7f9322f9d700 1 -- 172.21.15.31:0/3532885233 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f931c13fb20 msgr2=0x7f931c15a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:11.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.091+0000 7f9322f9d700 1 --2- 172.21.15.31:0/3532885233 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f931c13fb20 0x7f931c15a930 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f93180099b0 tx=0x7f9318009250 comp rx=0 tx=0).stop 2023-10-06T02:31:11.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.091+0000 7f9322f9d700 1 -- 172.21.15.31:0/3532885233 shutdown_connections 2023-10-06T02:31:11.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.091+0000 7f9322f9d700 1 --2- 172.21.15.31:0/3532885233 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f931c13fb20 0x7f931c15a930 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:11.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.091+0000 7f9322f9d700 1 --2- 172.21.15.31:0/3532885233 >> v2:172.21.15.62:3300/0 conn(0x7f931c06aaa0 0x7f931c0c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:11.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.091+0000 7f9322f9d700 1 -- 172.21.15.31:0/3532885233 >> 172.21.15.31:0/3532885233 conn(0x7f931c06d170 msgr2=0x7f931c0cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:11.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.092+0000 7f9322f9d700 1 -- 172.21.15.31:0/3532885233 shutdown_connections 2023-10-06T02:31:11.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.092+0000 7f9322f9d700 1 -- 172.21.15.31:0/3532885233 wait complete. 2023-10-06T02:31:11.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.093+0000 7f9322f9d700 1 Processor -- start 2023-10-06T02:31:11.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.093+0000 7f9322f9d700 1 -- start start 2023-10-06T02:31:11.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.094+0000 7f9322f9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f931c06aaa0 0x7f931c16dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:11.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.094+0000 7f9322f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f931c13fb20 0x7f931c135960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:11.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.094+0000 7f9322f9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f931c13fef0 0x7f931c133d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:11.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.094+0000 7f9322f9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f931c158c40 con 0x7f931c06aaa0 2023-10-06T02:31:11.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.094+0000 7f9322f9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f931c156c30 con 0x7f931c13fef0 2023-10-06T02:31:11.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.094+0000 7f9322f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f931c16e1a0 con 0x7f931c13fb20 2023-10-06T02:31:11.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.094+0000 7f9321f9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f931c06aaa0 0x7f931c16dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:11.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.094+0000 7f9321f9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f931c06aaa0 0x7f931c16dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37480/0 (socket says 172.21.15.31:37480) 2023-10-06T02:31:11.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.094+0000 7f9321f9b700 1 -- 172.21.15.31:0/1477789191 learned_addr learned my addr 172.21.15.31:0/1477789191 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:11.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.094+0000 7f932179a700 1 --2- 172.21.15.31:0/1477789191 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f931c13fb20 0x7f931c135960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:11.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.094+0000 7f932279c700 1 --2- 172.21.15.31:0/1477789191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f931c13fef0 0x7f931c133d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:11.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.095+0000 7f9321f9b700 1 -- 172.21.15.31:0/1477789191 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f931c13fb20 msgr2=0x7f931c135960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:11.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.095+0000 7f9321f9b700 1 --2- 172.21.15.31:0/1477789191 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f931c13fb20 0x7f931c135960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:11.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.095+0000 7f9321f9b700 1 -- 172.21.15.31:0/1477789191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f931c13fef0 msgr2=0x7f931c133d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:11.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.095+0000 7f9321f9b700 1 --2- 172.21.15.31:0/1477789191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f931c13fef0 0x7f931c133d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:11.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.095+0000 7f9321f9b700 1 -- 172.21.15.31:0/1477789191 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93180094f0 con 0x7f931c06aaa0 2023-10-06T02:31:11.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.095+0000 7f9321f9b700 1 --2- 172.21.15.31:0/1477789191 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f931c06aaa0 0x7f931c16dc60 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f931800b3c0 tx=0x7f9318017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:11.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.095+0000 7f9311ffb700 1 -- 172.21.15.31:0/1477789191 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931800bbe0 con 0x7f931c06aaa0 2023-10-06T02:31:11.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.095+0000 7f9311ffb700 1 -- 172.21.15.31:0/1477789191 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9318000c90 con 0x7f931c06aaa0 2023-10-06T02:31:11.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.095+0000 7f9322f9d700 1 -- 172.21.15.31:0/1477789191 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f931c135600 con 0x7f931c06aaa0 2023-10-06T02:31:11.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.095+0000 7f9311ffb700 1 -- 172.21.15.31:0/1477789191 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9318011650 con 0x7f931c06aaa0 2023-10-06T02:31:11.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.096+0000 7f9322f9d700 1 -- 172.21.15.31:0/1477789191 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f931c1588d0 con 0x7f931c06aaa0 2023-10-06T02:31:11.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.097+0000 7f9322f9d700 1 -- 172.21.15.31:0/1477789191 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f931c135f00 con 0x7f931c06aaa0 2023-10-06T02:31:11.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.098+0000 7f9311ffb700 1 -- 172.21.15.31:0/1477789191 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 85403+0+0 (secure 0 0 0) 0x7f9318011a80 con 0x7f931c06aaa0 2023-10-06T02:31:11.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.098+0000 7f9311ffb700 1 --2- 172.21.15.31:0/1477789191 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9300067680 0x7f9300069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:11.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.098+0000 7f9311ffb700 1 -- 172.21.15.31:0/1477789191 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f93180870b0 con 0x7f931c06aaa0 2023-10-06T02:31:11.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.099+0000 7f932179a700 1 --2- 172.21.15.31:0/1477789191 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9300067680 0x7f9300069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:11.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.100+0000 7f932179a700 1 --2- 172.21.15.31:0/1477789191 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9300067680 0x7f9300069b30 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f931c138240 tx=0x7f930c007f00 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:11.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.103+0000 7f9311ffb700 1 -- 172.21.15.31:0/1477789191 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9318084470 con 0x7f931c06aaa0 2023-10-06T02:31:11.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.350+0000 7f9322f9d700 1 -- 172.21.15.31:0/1477789191 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f931c04c3c0 con 0x7f931c06aaa0 2023-10-06T02:31:11.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.351+0000 7f9311ffb700 1 -- 172.21.15.31:0/1477789191 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) v1 ==== 95+0+4603 (secure 0 0 0) 0x7f9318053260 con 0x7f931c06aaa0 2023-10-06T02:31:11.352 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:31:11.352 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":101,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6923,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/4169296779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":4169296779},{"type":"v1","addr":"172.21.15.31:6835","nonce":4169296779}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95},{"gid":6926,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/3032245108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3032245108},{"type":"v1","addr":"172.21.15.62:6834","nonce":3032245108}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95},{"gid":8469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6833/1522769465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1522769465},{"type":"v1","addr":"172.21.15.31:6833","nonce":1522769465}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:30:51.153736+0000","modified":"2023-10-06T02:30:55.096858+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8487":{"gid":8487,"name":"d","rank":0,"incarnation":96,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/2236458886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2236458886},{"type":"v1","addr":"172.21.15.62:6835","nonce":2236458886}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2023-10-06T02:31:11.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.353+0000 7f9322f9d700 1 -- 172.21.15.31:0/1477789191 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9300067680 msgr2=0x7f9300069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:11.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.353+0000 7f9322f9d700 1 --2- 172.21.15.31:0/1477789191 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9300067680 0x7f9300069b30 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f931c138240 tx=0x7f930c007f00 comp rx=0 tx=0).stop 2023-10-06T02:31:11.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.353+0000 7f9322f9d700 1 -- 172.21.15.31:0/1477789191 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f931c06aaa0 msgr2=0x7f931c16dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:11.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.353+0000 7f9322f9d700 1 --2- 172.21.15.31:0/1477789191 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f931c06aaa0 0x7f931c16dc60 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f931800b3c0 tx=0x7f9318017000 comp rx=0 tx=0).stop 2023-10-06T02:31:11.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.353+0000 7f9322f9d700 1 -- 172.21.15.31:0/1477789191 shutdown_connections 2023-10-06T02:31:11.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.353+0000 7f9322f9d700 1 --2- 172.21.15.31:0/1477789191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f931c13fef0 0x7f931c133d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:11.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.353+0000 7f9322f9d700 1 --2- 172.21.15.31:0/1477789191 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f931c06aaa0 0x7f931c16dc60 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:11.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.353+0000 7f9322f9d700 1 --2- 172.21.15.31:0/1477789191 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9300067680 0x7f9300069b30 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:11.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.353+0000 7f9322f9d700 1 --2- 172.21.15.31:0/1477789191 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f931c13fb20 0x7f931c135960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:11.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.353+0000 7f9322f9d700 1 -- 172.21.15.31:0/1477789191 >> 172.21.15.31:0/1477789191 conn(0x7f931c06d170 msgr2=0x7f931c0c2460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:11.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.354+0000 7f9322f9d700 1 -- 172.21.15.31:0/1477789191 shutdown_connections 2023-10-06T02:31:11.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.354+0000 7f9322f9d700 1 -- 172.21.15.31:0/1477789191 wait complete. 2023-10-06T02:31:11.356 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 101 2023-10-06T02:31:11.372 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:31:11.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.558+0000 7fc1a6b9d700 1 Processor -- start 2023-10-06T02:31:11.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.558+0000 7fc1a6b9d700 1 -- start start 2023-10-06T02:31:11.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.559+0000 7fc1a6b9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc1a0153c30 0x7fc1a015a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:11.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.559+0000 7fc1a6b9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc1a00cf110 0x7fc1a015c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:11.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.559+0000 7fc1a6b9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1a015afd0 con 0x7fc1a015de50 2023-10-06T02:31:11.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.559+0000 7fc1a6b9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1a015b2b0 con 0x7fc1a00cf110 2023-10-06T02:31:11.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.559+0000 7fc1a6b9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a015b590 con 0x7fc1a0153c30 2023-10-06T02:31:11.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.560+0000 7fc1a539a700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fc1a015de50 0x7fc1a00cea00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49734/0 (socket says 172.21.15.31:49734) 2023-10-06T02:31:11.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.560+0000 7fc1a539a700 1 -- 172.21.15.31:0/3901431854 learned_addr learned my addr 172.21.15.31:0/3901431854 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:31:11.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.560+0000 7fc1a639c700 1 --2- 172.21.15.31:0/3901431854 >> v2:172.21.15.62:3300/0 conn(0x7fc1a00cf110 0x7fc1a015c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:11.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.560+0000 7fc1a5b9b700 1 --2- 172.21.15.31:0/3901431854 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc1a0153c30 0x7fc1a015a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:11.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.560+0000 7fc1a4b99700 1 -- 172.21.15.31:0/3901431854 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2207607366 0 0) 0x7fc1a015afd0 con 0x7fc1a015de50 2023-10-06T02:31:11.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.560+0000 7fc1a4b99700 1 -- 172.21.15.31:0/3901431854 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc18c003570 con 0x7fc1a015de50 2023-10-06T02:31:11.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.560+0000 7fc1a639c700 1 -- 172.21.15.31:0/3901431854 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc1a0153c30 msgr2=0x7fc1a015a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:11.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.560+0000 7fc1a639c700 1 --2- 172.21.15.31:0/3901431854 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc1a0153c30 0x7fc1a015a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:11.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.560+0000 7fc1a639c700 1 -- 172.21.15.31:0/3901431854 >> v1:172.21.15.31:6789/0 conn(0x7fc1a015de50 legacy=0x7fc1a00cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:11.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.560+0000 7fc1a639c700 1 -- 172.21.15.31:0/3901431854 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1a015b870 con 0x7fc1a00cf110 2023-10-06T02:31:11.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.560+0000 7fc1a5b9b700 1 --2- 172.21.15.31:0/3901431854 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc1a0153c30 0x7fc1a015a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:31:11.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.561+0000 7fc1a639c700 1 --2- 172.21.15.31:0/3901431854 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1a00cf110 0x7fc1a015c070 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fc198009e60 tx=0x7fc198009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7e4613285c4c082 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:11.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.561+0000 7fc1a4b99700 1 -- 172.21.15.31:0/3901431854 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19800e070 con 0x7fc1a00cf110 2023-10-06T02:31:11.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.561+0000 7fc1a4b99700 1 -- 172.21.15.31:0/3901431854 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc19800bab0 con 0x7fc1a00cf110 2023-10-06T02:31:11.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.561+0000 7fc1a4b99700 1 -- 172.21.15.31:0/3901431854 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19800be70 con 0x7fc1a00cf110 2023-10-06T02:31:11.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.561+0000 7fc1a6b9d700 1 -- 172.21.15.31:0/3901431854 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1a00cf110 msgr2=0x7fc1a015c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:11.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.561+0000 7fc1a6b9d700 1 --2- 172.21.15.31:0/3901431854 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1a00cf110 0x7fc1a015c070 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fc198009e60 tx=0x7fc198009780 comp rx=0 tx=0).stop 2023-10-06T02:31:11.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.562+0000 7fc1a6b9d700 1 -- 172.21.15.31:0/3901431854 shutdown_connections 2023-10-06T02:31:11.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.562+0000 7fc1a6b9d700 1 --2- 172.21.15.31:0/3901431854 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1a00cf110 0x7fc1a015c070 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:11.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.562+0000 7fc1a6b9d700 1 --2- 172.21.15.31:0/3901431854 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc1a0153c30 0x7fc1a015a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:11.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.562+0000 7fc1a6b9d700 1 -- 172.21.15.31:0/3901431854 >> 172.21.15.31:0/3901431854 conn(0x7fc1a00703a0 msgr2=0x7fc1a00707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:11.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.562+0000 7fc1a6b9d700 1 -- 172.21.15.31:0/3901431854 shutdown_connections 2023-10-06T02:31:11.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.562+0000 7fc1a6b9d700 1 -- 172.21.15.31:0/3901431854 wait complete. 2023-10-06T02:31:11.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.563+0000 7fc1a6b9d700 1 Processor -- start 2023-10-06T02:31:11.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.563+0000 7fc1a6b9d700 1 -- start start 2023-10-06T02:31:11.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.564+0000 7fc1a6b9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1a00cf110 0x7fc1a0146020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:11.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.564+0000 7fc1a6b9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc1a0153c30 0x7fc1a00c8800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:11.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.564+0000 7fc1a6b9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1a015de50 0x7fc1a00b3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:11.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.564+0000 7fc1a6b9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a00b59a0 con 0x7fc1a00cf110 2023-10-06T02:31:11.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.564+0000 7fc1a6b9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a0146560 con 0x7fc1a015de50 2023-10-06T02:31:11.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.564+0000 7fc1a6b9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a016d350 con 0x7fc1a0153c30 2023-10-06T02:31:11.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.564+0000 7fc1a5b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1a00cf110 0x7fc1a0146020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:11.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.564+0000 7fc1a5b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1a00cf110 0x7fc1a0146020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37494/0 (socket says 172.21.15.31:37494) 2023-10-06T02:31:11.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.564+0000 7fc1a5b9b700 1 -- 172.21.15.31:0/3714193243 learned_addr learned my addr 172.21.15.31:0/3714193243 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:11.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.564+0000 7fc1a539a700 1 --2- 172.21.15.31:0/3714193243 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc1a0153c30 0x7fc1a00c8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:11.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.565+0000 7fc1a639c700 1 --2- 172.21.15.31:0/3714193243 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1a015de50 0x7fc1a00b3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:11.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.565+0000 7fc1a5b9b700 1 -- 172.21.15.31:0/3714193243 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc1a0153c30 msgr2=0x7fc1a00c8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:11.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.565+0000 7fc1a5b9b700 1 --2- 172.21.15.31:0/3714193243 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc1a0153c30 0x7fc1a00c8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:11.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.565+0000 7fc1a5b9b700 1 -- 172.21.15.31:0/3714193243 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1a015de50 msgr2=0x7fc1a00b3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:11.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.565+0000 7fc1a5b9b700 1 --2- 172.21.15.31:0/3714193243 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1a015de50 0x7fc1a00b3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:11.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.565+0000 7fc1a5b9b700 1 -- 172.21.15.31:0/3714193243 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc19c009420 con 0x7fc1a00cf110 2023-10-06T02:31:11.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.565+0000 7fc1a5b9b700 1 --2- 172.21.15.31:0/3714193243 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1a00cf110 0x7fc1a0146020 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fc19c00ecb0 tx=0x7fc19c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:11.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.565+0000 7fc195ffb700 1 -- 172.21.15.31:0/3714193243 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19c00ee60 con 0x7fc1a00cf110 2023-10-06T02:31:11.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.565+0000 7fc195ffb700 1 -- 172.21.15.31:0/3714193243 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc19c00cb80 con 0x7fc1a00cf110 2023-10-06T02:31:11.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.565+0000 7fc195ffb700 1 -- 172.21.15.31:0/3714193243 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19c013470 con 0x7fc1a00cf110 2023-10-06T02:31:11.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.565+0000 7fc1a6b9d700 1 -- 172.21.15.31:0/3714193243 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc198009a20 con 0x7fc1a00cf110 2023-10-06T02:31:11.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.565+0000 7fc1a6b9d700 1 -- 172.21.15.31:0/3714193243 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1a016a7c0 con 0x7fc1a00cf110 2023-10-06T02:31:11.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.566+0000 7fc1a6b9d700 1 -- 172.21.15.31:0/3714193243 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1a015dab0 con 0x7fc1a00cf110 2023-10-06T02:31:11.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.567+0000 7fc195ffb700 1 -- 172.21.15.31:0/3714193243 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 85403+0+0 (secure 0 0 0) 0x7fc19c013820 con 0x7fc1a00cf110 2023-10-06T02:31:11.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.568+0000 7fc195ffb700 1 --2- 172.21.15.31:0/3714193243 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc184067680 0x7fc184069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:11.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.568+0000 7fc195ffb700 1 -- 172.21.15.31:0/3714193243 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fc19c085c70 con 0x7fc1a00cf110 2023-10-06T02:31:11.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.569+0000 7fc1a539a700 1 --2- 172.21.15.31:0/3714193243 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc184067680 0x7fc184069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:11.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.569+0000 7fc1a539a700 1 --2- 172.21.15.31:0/3714193243 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc184067680 0x7fc184069b30 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc1a00ce6b0 tx=0x7fc190009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:11.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.572+0000 7fc195ffb700 1 -- 172.21.15.31:0/3714193243 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc19c083030 con 0x7fc1a00cf110 2023-10-06T02:31:11.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.817+0000 7fc1a6b9d700 1 -- 172.21.15.31:0/3714193243 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc1a00cfbe0 con 0x7fc1a00cf110 2023-10-06T02:31:11.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.817+0000 7fc195ffb700 1 -- 172.21.15.31:0/3714193243 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) v1 ==== 95+0+4603 (secure 0 0 0) 0x7fc19c017090 con 0x7fc1a00cf110 2023-10-06T02:31:11.818 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:31:11.818 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":101,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6923,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/4169296779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":4169296779},{"type":"v1","addr":"172.21.15.31:6835","nonce":4169296779}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95},{"gid":6926,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/3032245108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3032245108},{"type":"v1","addr":"172.21.15.62:6834","nonce":3032245108}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95},{"gid":8469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6833/1522769465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1522769465},{"type":"v1","addr":"172.21.15.31:6833","nonce":1522769465}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:30:51.153736+0000","modified":"2023-10-06T02:30:55.096858+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8487":{"gid":8487,"name":"d","rank":0,"incarnation":96,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/2236458886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2236458886},{"type":"v1","addr":"172.21.15.62:6835","nonce":2236458886}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2023-10-06T02:31:11.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.819+0000 7fc1a6b9d700 1 -- 172.21.15.31:0/3714193243 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc184067680 msgr2=0x7fc184069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:11.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.819+0000 7fc1a6b9d700 1 --2- 172.21.15.31:0/3714193243 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc184067680 0x7fc184069b30 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc1a00ce6b0 tx=0x7fc190009040 comp rx=0 tx=0).stop 2023-10-06T02:31:11.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.819+0000 7fc1a6b9d700 1 -- 172.21.15.31:0/3714193243 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1a00cf110 msgr2=0x7fc1a0146020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:11.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.819+0000 7fc1a6b9d700 1 --2- 172.21.15.31:0/3714193243 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1a00cf110 0x7fc1a0146020 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fc19c00ecb0 tx=0x7fc19c0091a0 comp rx=0 tx=0).stop 2023-10-06T02:31:11.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.820+0000 7fc1a6b9d700 1 -- 172.21.15.31:0/3714193243 shutdown_connections 2023-10-06T02:31:11.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.820+0000 7fc1a6b9d700 1 --2- 172.21.15.31:0/3714193243 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1a015de50 0x7fc1a00b3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:11.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.820+0000 7fc1a6b9d700 1 --2- 172.21.15.31:0/3714193243 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1a00cf110 0x7fc1a0146020 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:11.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.820+0000 7fc1a6b9d700 1 --2- 172.21.15.31:0/3714193243 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc184067680 0x7fc184069b30 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:11.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.820+0000 7fc1a6b9d700 1 --2- 172.21.15.31:0/3714193243 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc1a0153c30 0x7fc1a00c8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:11.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.820+0000 7fc1a6b9d700 1 -- 172.21.15.31:0/3714193243 >> 172.21.15.31:0/3714193243 conn(0x7fc1a00703a0 msgr2=0x7fc1a0070770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:11.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.820+0000 7fc1a6b9d700 1 -- 172.21.15.31:0/3714193243 shutdown_connections 2023-10-06T02:31:11.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:11.820+0000 7fc1a6b9d700 1 -- 172.21.15.31:0/3714193243 wait complete. 2023-10-06T02:31:11.822 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 101 2023-10-06T02:31:11.838 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:31:12.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.025+0000 7fa7fb39d700 1 Processor -- start 2023-10-06T02:31:12.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.025+0000 7fa7fb39d700 1 -- start start 2023-10-06T02:31:12.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.027+0000 7fa7fb39d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa7f4142c00 0x7fa7f406b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:12.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.027+0000 7fa7fb39d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa7f406bd20 0x7fa7f40c7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:12.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.027+0000 7fa7fb39d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7f40ca480 con 0x7fa7f4142fd0 2023-10-06T02:31:12.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.027+0000 7fa7fb39d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7f415e240 con 0x7fa7f4142c00 2023-10-06T02:31:12.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.027+0000 7fa7fb39d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7f406b350 con 0x7fa7f406bd20 2023-10-06T02:31:12.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.027+0000 7fa7fa39b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa7f4142c00 0x7fa7f406b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:12.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.027+0000 7fa7fa39b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa7f4142c00 0x7fa7f406b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37510/0 (socket says 172.21.15.31:37510) 2023-10-06T02:31:12.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.027+0000 7fa7fa39b700 1 -- 172.21.15.31:0/344277437 learned_addr learned my addr 172.21.15.31:0/344277437 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:12.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.027+0000 7fa7f9b9a700 1 --2- 172.21.15.31:0/344277437 >> v2:172.21.15.62:3300/0 conn(0x7fa7f406bd20 0x7fa7f40c7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:12.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.027+0000 7fa7fa39b700 1 -- 172.21.15.31:0/344277437 >> v2:172.21.15.62:3300/0 conn(0x7fa7f406bd20 msgr2=0x7fa7f40c7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:12.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.027+0000 7fa7fa39b700 1 --2- 172.21.15.31:0/344277437 >> v2:172.21.15.62:3300/0 conn(0x7fa7f406bd20 0x7fa7f40c7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:12.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.027+0000 7fa7fa39b700 1 -- 172.21.15.31:0/344277437 >> v1:172.21.15.62:6789/0 conn(0x7fa7f4142fd0 legacy=0x7fa7f406c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:12.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.027+0000 7fa7fa39b700 1 -- 172.21.15.31:0/344277437 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7f415df60 con 0x7fa7f4142c00 2023-10-06T02:31:12.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.028+0000 7fa7fa39b700 1 --2- 172.21.15.31:0/344277437 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7f4142c00 0x7fa7f406b7e0 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7fa7f406d750 tx=0x7fa7f0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4034b02d8563bfd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:12.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.028+0000 7fa7f9399700 1 -- 172.21.15.31:0/344277437 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f000b860 con 0x7fa7f4142c00 2023-10-06T02:31:12.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.028+0000 7fa7f9399700 1 -- 172.21.15.31:0/344277437 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7f000b9c0 con 0x7fa7f4142c00 2023-10-06T02:31:12.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.028+0000 7fa7f9399700 1 -- 172.21.15.31:0/344277437 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f000bd60 con 0x7fa7f4142c00 2023-10-06T02:31:12.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.028+0000 7fa7fb39d700 1 -- 172.21.15.31:0/344277437 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7f4142c00 msgr2=0x7fa7f406b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:12.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.028+0000 7fa7fb39d700 1 --2- 172.21.15.31:0/344277437 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7f4142c00 0x7fa7f406b7e0 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7fa7f406d750 tx=0x7fa7f0009250 comp rx=0 tx=0).stop 2023-10-06T02:31:12.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.029+0000 7fa7fb39d700 1 -- 172.21.15.31:0/344277437 shutdown_connections 2023-10-06T02:31:12.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.029+0000 7fa7fb39d700 1 --2- 172.21.15.31:0/344277437 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7f4142c00 0x7fa7f406b7e0 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:12.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.029+0000 7fa7fb39d700 1 --2- 172.21.15.31:0/344277437 >> v2:172.21.15.62:3300/0 conn(0x7fa7f406bd20 0x7fa7f40c7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:12.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.029+0000 7fa7fb39d700 1 -- 172.21.15.31:0/344277437 >> 172.21.15.31:0/344277437 conn(0x7fa7f40675a0 msgr2=0x7fa7f40679a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:12.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.029+0000 7fa7fb39d700 1 -- 172.21.15.31:0/344277437 shutdown_connections 2023-10-06T02:31:12.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.029+0000 7fa7fb39d700 1 -- 172.21.15.31:0/344277437 wait complete. 2023-10-06T02:31:12.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.030+0000 7fa7fb39d700 1 Processor -- start 2023-10-06T02:31:12.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.030+0000 7fa7fb39d700 1 -- start start 2023-10-06T02:31:12.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.031+0000 7fa7fb39d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7f406bd20 0x7fa7f40b0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:12.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.031+0000 7fa7fb39d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa7f4142c00 0x7fa7f413ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:12.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.031+0000 7fa7fb39d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7f4142fd0 0x7fa7f40c62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:12.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.031+0000 7fa7fb39d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f40b06d0 con 0x7fa7f4142fd0 2023-10-06T02:31:12.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.031+0000 7fa7fb39d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f40b0c70 con 0x7fa7f406bd20 2023-10-06T02:31:12.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.031+0000 7fa7fb39d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f40c6af0 con 0x7fa7f4142c00 2023-10-06T02:31:12.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.031+0000 7fa7fab9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7f4142fd0 0x7fa7f40c62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:12.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.031+0000 7fa7fab9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7f4142fd0 0x7fa7f40c62a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37512/0 (socket says 172.21.15.31:37512) 2023-10-06T02:31:12.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.031+0000 7fa7fab9c700 1 -- 172.21.15.31:0/2047110125 learned_addr learned my addr 172.21.15.31:0/2047110125 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:12.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.031+0000 7fa7f9b9a700 1 --2- 172.21.15.31:0/2047110125 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa7f4142c00 0x7fa7f413ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:12.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.031+0000 7fa7fa39b700 1 --2- 172.21.15.31:0/2047110125 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7f406bd20 0x7fa7f40b0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:12.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.032+0000 7fa7fab9c700 1 -- 172.21.15.31:0/2047110125 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa7f4142c00 msgr2=0x7fa7f413ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:12.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.032+0000 7fa7fab9c700 1 --2- 172.21.15.31:0/2047110125 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa7f4142c00 0x7fa7f413ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:12.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.032+0000 7fa7fab9c700 1 -- 172.21.15.31:0/2047110125 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7f406bd20 msgr2=0x7fa7f40b0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:12.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.032+0000 7fa7fab9c700 1 --2- 172.21.15.31:0/2047110125 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7f406bd20 0x7fa7f40b0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:12.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.032+0000 7fa7fab9c700 1 -- 172.21.15.31:0/2047110125 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7f00094f0 con 0x7fa7f4142fd0 2023-10-06T02:31:12.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.032+0000 7fa7fab9c700 1 --2- 172.21.15.31:0/2047110125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7f4142fd0 0x7fa7f40c62a0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec00c840 tx=0x7fa7ec001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:12.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.032+0000 7fa7ea7fc700 1 -- 172.21.15.31:0/2047110125 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec012630 con 0x7fa7f4142fd0 2023-10-06T02:31:12.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.032+0000 7fa7ea7fc700 1 -- 172.21.15.31:0/2047110125 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7ec0127b0 con 0x7fa7f4142fd0 2023-10-06T02:31:12.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.032+0000 7fa7ea7fc700 1 -- 172.21.15.31:0/2047110125 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec012b70 con 0x7fa7f4142fd0 2023-10-06T02:31:12.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.032+0000 7fa7fb39d700 1 -- 172.21.15.31:0/2047110125 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7f40c6dd0 con 0x7fa7f4142fd0 2023-10-06T02:31:12.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.032+0000 7fa7fb39d700 1 -- 172.21.15.31:0/2047110125 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7f413d3d0 con 0x7fa7f4142fd0 2023-10-06T02:31:12.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.034+0000 7fa7fb39d700 1 -- 172.21.15.31:0/2047110125 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7f40b0e00 con 0x7fa7f4142fd0 2023-10-06T02:31:12.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.034+0000 7fa7ea7fc700 1 -- 172.21.15.31:0/2047110125 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 85403+0+0 (secure 0 0 0) 0x7fa7ec01a3a0 con 0x7fa7f4142fd0 2023-10-06T02:31:12.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.035+0000 7fa7ea7fc700 1 --2- 172.21.15.31:0/2047110125 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa7d8067680 0x7fa7d8069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:12.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.035+0000 7fa7ea7fc700 1 -- 172.21.15.31:0/2047110125 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fa7ec083060 con 0x7fa7f4142fd0 2023-10-06T02:31:12.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.036+0000 7fa7fa39b700 1 --2- 172.21.15.31:0/2047110125 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa7d8067680 0x7fa7d8069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:12.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.036+0000 7fa7fa39b700 1 --2- 172.21.15.31:0/2047110125 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa7d8067680 0x7fa7d8069b30 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fa7f406d750 tx=0x7fa7f0017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:12.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.039+0000 7fa7ea7fc700 1 -- 172.21.15.31:0/2047110125 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa7ec082d70 con 0x7fa7f4142fd0 2023-10-06T02:31:12.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.246+0000 7fa7fb39d700 1 -- 172.21.15.31:0/2047110125 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa7f404c3c0 con 0x7fa7f4142fd0 2023-10-06T02:31:12.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.247+0000 7fa7ea7fc700 1 -- 172.21.15.31:0/2047110125 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v144) v1 ==== 74+0+16876 (secure 0 0 0) 0x7fa7ec0555f0 con 0x7fa7f4142fd0 2023-10-06T02:31:12.248 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:31:12.248 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":144,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:30:52.650397+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:30:49.325914+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":24,"pool_name":"cephfs_data","create_time":"2023-10-06T02:30:50.127083+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":20,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:31:12.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.249+0000 7fa7fb39d700 1 -- 172.21.15.31:0/2047110125 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa7d8067680 msgr2=0x7fa7d8069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:12.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.249+0000 7fa7fb39d700 1 --2- 172.21.15.31:0/2047110125 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa7d8067680 0x7fa7d8069b30 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fa7f406d750 tx=0x7fa7f0017000 comp rx=0 tx=0).stop 2023-10-06T02:31:12.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.249+0000 7fa7fb39d700 1 -- 172.21.15.31:0/2047110125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7f4142fd0 msgr2=0x7fa7f40c62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:12.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.249+0000 7fa7fb39d700 1 --2- 172.21.15.31:0/2047110125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7f4142fd0 0x7fa7f40c62a0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec00c840 tx=0x7fa7ec001670 comp rx=0 tx=0).stop 2023-10-06T02:31:12.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.249+0000 7fa7fb39d700 1 -- 172.21.15.31:0/2047110125 shutdown_connections 2023-10-06T02:31:12.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.249+0000 7fa7fb39d700 1 --2- 172.21.15.31:0/2047110125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7f4142fd0 0x7fa7f40c62a0 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:12.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.249+0000 7fa7fb39d700 1 --2- 172.21.15.31:0/2047110125 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7f406bd20 0x7fa7f40b0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:12.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.249+0000 7fa7fb39d700 1 --2- 172.21.15.31:0/2047110125 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa7d8067680 0x7fa7d8069b30 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:12.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.249+0000 7fa7fb39d700 1 --2- 172.21.15.31:0/2047110125 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa7f4142c00 0x7fa7f413ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:12.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.249+0000 7fa7fb39d700 1 -- 172.21.15.31:0/2047110125 >> 172.21.15.31:0/2047110125 conn(0x7fa7f40675a0 msgr2=0x7fa7f406c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:12.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.249+0000 7fa7fb39d700 1 -- 172.21.15.31:0/2047110125 shutdown_connections 2023-10-06T02:31:12.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.250+0000 7fa7fb39d700 1 -- 172.21.15.31:0/2047110125 wait complete. 2023-10-06T02:31:12.268 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T02:31:12.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.456+0000 7fd0fe19d700 1 Processor -- start 2023-10-06T02:31:12.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.456+0000 7fd0fe19d700 1 -- start start 2023-10-06T02:31:12.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.457+0000 7fd0fe19d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd0f8153c30 0x7fd0f815a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:12.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.457+0000 7fd0fe19d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd0f80cf110 0x7fd0f815c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:12.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.457+0000 7fd0fe19d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0f815afd0 con 0x7fd0f815de50 2023-10-06T02:31:12.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.457+0000 7fd0fe19d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0f815b2b0 con 0x7fd0f8153c30 2023-10-06T02:31:12.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.457+0000 7fd0fe19d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f815b590 con 0x7fd0f80cf110 2023-10-06T02:31:12.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.458+0000 7fd0fc99a700 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fd0f815de50 0x7fd0f80cea00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:57448/0 (socket says 172.21.15.31:57448) 2023-10-06T02:31:12.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.458+0000 7fd0fc99a700 1 -- 172.21.15.31:0/10894392 learned_addr learned my addr 172.21.15.31:0/10894392 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:31:12.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.458+0000 7fd0fd99c700 1 --2- 172.21.15.31:0/10894392 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd0f80cf110 0x7fd0f815c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:12.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.458+0000 7fd0fd19b700 1 --2- 172.21.15.31:0/10894392 >> v2:172.21.15.62:3300/0 conn(0x7fd0f8153c30 0x7fd0f815a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:12.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.458+0000 7fd0effff700 1 -- 172.21.15.31:0/10894392 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3229265514 0 0) 0x7fd0f815afd0 con 0x7fd0f815de50 2023-10-06T02:31:12.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.459+0000 7fd0effff700 1 -- 172.21.15.31:0/10894392 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0e4003570 con 0x7fd0f815de50 2023-10-06T02:31:12.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.459+0000 7fd0fd19b700 1 -- 172.21.15.31:0/10894392 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd0f80cf110 msgr2=0x7fd0f815c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:12.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.459+0000 7fd0fd19b700 1 --2- 172.21.15.31:0/10894392 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd0f80cf110 0x7fd0f815c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:12.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.459+0000 7fd0fd19b700 1 -- 172.21.15.31:0/10894392 >> v1:172.21.15.62:6789/0 conn(0x7fd0f815de50 legacy=0x7fd0f80cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:12.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.459+0000 7fd0fd19b700 1 -- 172.21.15.31:0/10894392 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0f815b870 con 0x7fd0f8153c30 2023-10-06T02:31:12.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.459+0000 7fd0fd99c700 1 --2- 172.21.15.31:0/10894392 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd0f80cf110 0x7fd0f815c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:31:12.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.459+0000 7fd0fd19b700 1 --2- 172.21.15.31:0/10894392 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0f8153c30 0x7fd0f815a5e0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fd0f40098e0 tx=0x7fd0f40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66e2554c8f147f64 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:12.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.459+0000 7fd0effff700 1 -- 172.21.15.31:0/10894392 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f4011630 con 0x7fd0f8153c30 2023-10-06T02:31:12.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.459+0000 7fd0effff700 1 -- 172.21.15.31:0/10894392 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0f4011790 con 0x7fd0f8153c30 2023-10-06T02:31:12.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.459+0000 7fd0effff700 1 -- 172.21.15.31:0/10894392 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f4011b30 con 0x7fd0f8153c30 2023-10-06T02:31:12.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.460+0000 7fd0fe19d700 1 -- 172.21.15.31:0/10894392 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0f8153c30 msgr2=0x7fd0f815a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:12.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.460+0000 7fd0fe19d700 1 --2- 172.21.15.31:0/10894392 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0f8153c30 0x7fd0f815a5e0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fd0f40098e0 tx=0x7fd0f40091a0 comp rx=0 tx=0).stop 2023-10-06T02:31:12.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.460+0000 7fd0fe19d700 1 -- 172.21.15.31:0/10894392 shutdown_connections 2023-10-06T02:31:12.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.460+0000 7fd0fe19d700 1 --2- 172.21.15.31:0/10894392 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd0f80cf110 0x7fd0f815c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:12.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.460+0000 7fd0fe19d700 1 --2- 172.21.15.31:0/10894392 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0f8153c30 0x7fd0f815a5e0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:12.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.460+0000 7fd0fe19d700 1 -- 172.21.15.31:0/10894392 >> 172.21.15.31:0/10894392 conn(0x7fd0f80703a0 msgr2=0x7fd0f80707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:12.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.460+0000 7fd0fe19d700 1 -- 172.21.15.31:0/10894392 shutdown_connections 2023-10-06T02:31:12.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.460+0000 7fd0fe19d700 1 -- 172.21.15.31:0/10894392 wait complete. 2023-10-06T02:31:12.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.461+0000 7fd0fe19d700 1 Processor -- start 2023-10-06T02:31:12.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.461+0000 7fd0fe19d700 1 -- start start 2023-10-06T02:31:12.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.462+0000 7fd0fe19d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0f80cf110 0x7fd0f80b32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:12.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.462+0000 7fd0fe19d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd0f8153c30 0x7fd0f80c7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:12.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.462+0000 7fd0fe19d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0f815de50 0x7fd0f80b1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:12.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.462+0000 7fd0fe19d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f80b0a50 con 0x7fd0f80cf110 2023-10-06T02:31:12.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.462+0000 7fd0fe19d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f80b0cf0 con 0x7fd0f815de50 2023-10-06T02:31:12.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.462+0000 7fd0fe19d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f80b0fd0 con 0x7fd0f8153c30 2023-10-06T02:31:12.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.463+0000 7fd0fd19b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0f80cf110 0x7fd0f80b32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:12.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.463+0000 7fd0fd19b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0f80cf110 0x7fd0f80b32d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37524/0 (socket says 172.21.15.31:37524) 2023-10-06T02:31:12.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.463+0000 7fd0fd19b700 1 -- 172.21.15.31:0/1280669058 learned_addr learned my addr 172.21.15.31:0/1280669058 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:12.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.463+0000 7fd0fd99c700 1 --2- 172.21.15.31:0/1280669058 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0f815de50 0x7fd0f80b1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:12.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.463+0000 7fd0fc99a700 1 --2- 172.21.15.31:0/1280669058 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd0f8153c30 0x7fd0f80c7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:12.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.463+0000 7fd0fd19b700 1 -- 172.21.15.31:0/1280669058 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd0f8153c30 msgr2=0x7fd0f80c7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:12.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.463+0000 7fd0fd19b700 1 --2- 172.21.15.31:0/1280669058 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd0f8153c30 0x7fd0f80c7ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:12.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.463+0000 7fd0fd19b700 1 -- 172.21.15.31:0/1280669058 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0f815de50 msgr2=0x7fd0f80b1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:12.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.463+0000 7fd0fd19b700 1 --2- 172.21.15.31:0/1280669058 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0f815de50 0x7fd0f80b1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:12.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.463+0000 7fd0fd19b700 1 -- 172.21.15.31:0/1280669058 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0f0009a20 con 0x7fd0f80cf110 2023-10-06T02:31:12.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.463+0000 7fd0fd19b700 1 --2- 172.21.15.31:0/1280669058 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0f80cf110 0x7fd0f80b32d0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7fd0f80cf5c0 tx=0x7fd0f4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:12.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.463+0000 7fd0ed7fa700 1 -- 172.21.15.31:0/1280669058 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f4011b10 con 0x7fd0f80cf110 2023-10-06T02:31:12.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.464+0000 7fd0ed7fa700 1 -- 172.21.15.31:0/1280669058 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0f40253f0 con 0x7fd0f80cf110 2023-10-06T02:31:12.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.464+0000 7fd0ed7fa700 1 -- 172.21.15.31:0/1280669058 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f4025790 con 0x7fd0f80cf110 2023-10-06T02:31:12.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.464+0000 7fd0fe19d700 1 -- 172.21.15.31:0/1280669058 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0f4009420 con 0x7fd0f80cf110 2023-10-06T02:31:12.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.464+0000 7fd0fe19d700 1 -- 172.21.15.31:0/1280669058 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0f80b1590 con 0x7fd0f80cf110 2023-10-06T02:31:12.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.465+0000 7fd0fe19d700 1 -- 172.21.15.31:0/1280669058 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0f804c3c0 con 0x7fd0f80cf110 2023-10-06T02:31:12.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.465+0000 7fd0ed7fa700 1 -- 172.21.15.31:0/1280669058 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 85403+0+0 (secure 0 0 0) 0x7fd0f4025b80 con 0x7fd0f80cf110 2023-10-06T02:31:12.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.466+0000 7fd0ed7fa700 1 --2- 172.21.15.31:0/1280669058 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd0dc067680 0x7fd0dc069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:12.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.466+0000 7fd0ed7fa700 1 -- 172.21.15.31:0/1280669058 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fd0f4087160 con 0x7fd0f80cf110 2023-10-06T02:31:12.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.467+0000 7fd0fc99a700 1 --2- 172.21.15.31:0/1280669058 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd0dc067680 0x7fd0dc069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:12.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.467+0000 7fd0fc99a700 1 --2- 172.21.15.31:0/1280669058 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd0dc067680 0x7fd0dc069b30 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fd0f8153790 tx=0x7fd0e8009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:12.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.471+0000 7fd0ed7fa700 1 -- 172.21.15.31:0/1280669058 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd0f4084520 con 0x7fd0f80cf110 2023-10-06T02:31:12.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.717+0000 7fd0fe19d700 1 -- 172.21.15.31:0/1280669058 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fd0f815bb50 con 0x7fd0f80cf110 2023-10-06T02:31:12.718 INFO:tasks.ceph.mon.a.smithi031.stderr:2023-10-06T02:31:12.718+0000 7f6de01f1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T02:31:12.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.723+0000 7fd0ed7fa700 1 -- 172.21.15.31:0/1280669058 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v102) v1 ==== 161+0+0 (secure 0 0 0) 0x7fd0f4053310 con 0x7fd0f80cf110 2023-10-06T02:31:12.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.725+0000 7fd0fe19d700 1 -- 172.21.15.31:0/1280669058 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd0dc067680 msgr2=0x7fd0dc069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:12.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.725+0000 7fd0fe19d700 1 --2- 172.21.15.31:0/1280669058 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd0dc067680 0x7fd0dc069b30 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fd0f8153790 tx=0x7fd0e8009040 comp rx=0 tx=0).stop 2023-10-06T02:31:12.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.725+0000 7fd0fe19d700 1 -- 172.21.15.31:0/1280669058 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0f80cf110 msgr2=0x7fd0f80b32d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:12.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.725+0000 7fd0fe19d700 1 --2- 172.21.15.31:0/1280669058 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0f80cf110 0x7fd0f80b32d0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7fd0f80cf5c0 tx=0x7fd0f4017000 comp rx=0 tx=0).stop 2023-10-06T02:31:12.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.725+0000 7fd0fe19d700 1 -- 172.21.15.31:0/1280669058 shutdown_connections 2023-10-06T02:31:12.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.725+0000 7fd0fe19d700 1 --2- 172.21.15.31:0/1280669058 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0f815de50 0x7fd0f80b1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:12.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.725+0000 7fd0fe19d700 1 --2- 172.21.15.31:0/1280669058 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0f80cf110 0x7fd0f80b32d0 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:12.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.725+0000 7fd0fe19d700 1 --2- 172.21.15.31:0/1280669058 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd0dc067680 0x7fd0dc069b30 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:12.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.725+0000 7fd0fe19d700 1 --2- 172.21.15.31:0/1280669058 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd0f8153c30 0x7fd0f80c7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:12.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.725+0000 7fd0fe19d700 1 -- 172.21.15.31:0/1280669058 >> 172.21.15.31:0/1280669058 conn(0x7fd0f80703a0 msgr2=0x7fd0f8070a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:12.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.725+0000 7fd0fe19d700 1 -- 172.21.15.31:0/1280669058 shutdown_connections 2023-10-06T02:31:12.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.725+0000 7fd0fe19d700 1 -- 172.21.15.31:0/1280669058 wait complete. 2023-10-06T02:31:12.729 INFO:teuthology.orchestra.run.smithi031.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T02:31:12.744 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T02:31:12.790 INFO:tasks.ceph.mds.d.smithi062.stdout:starting mds.d at 2023-10-06T02:31:12.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.933+0000 7f39c7fff700 1 Processor -- start 2023-10-06T02:31:12.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.933+0000 7f39c7fff700 1 -- start start 2023-10-06T02:31:12.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.935+0000 7f39c7fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39c80aee60 0x7f39c80cdd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:12.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.935+0000 7f39c7fff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f39c80af230 0x7f39c815a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:12.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.935+0000 7f39c7fff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39c80b2c30 con 0x7f39c80b23c0 2023-10-06T02:31:12.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.935+0000 7f39c7fff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39c8159bf0 con 0x7f39c80af230 2023-10-06T02:31:12.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.935+0000 7f39c7fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39c80b2f10 con 0x7f39c80aee60 2023-10-06T02:31:12.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.935+0000 7f39c77fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f39c80b23c0 0x7f39c81536e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49736/0 (socket says 172.21.15.31:49736) 2023-10-06T02:31:12.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.935+0000 7f39c77fe700 1 -- 172.21.15.31:0/825758317 learned_addr learned my addr 172.21.15.31:0/825758317 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:31:12.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.935+0000 7f39c6ffd700 1 --2- 172.21.15.31:0/825758317 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39c80aee60 0x7f39c80cdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:12.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.935+0000 7f39c67fc700 1 --2- 172.21.15.31:0/825758317 >> v2:172.21.15.62:3300/0 conn(0x7f39c80af230 0x7f39c815a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:12.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.935+0000 7f39c5ffb700 1 -- 172.21.15.31:0/825758317 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2718648240 0 0) 0x7f39c80b2c30 con 0x7f39c80b23c0 2023-10-06T02:31:12.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.936+0000 7f39c5ffb700 1 -- 172.21.15.31:0/825758317 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39b0003570 con 0x7f39c80b23c0 2023-10-06T02:31:12.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.936+0000 7f39c6ffd700 1 -- 172.21.15.31:0/825758317 >> v2:172.21.15.62:3300/0 conn(0x7f39c80af230 msgr2=0x7f39c815a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:12.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.936+0000 7f39c6ffd700 1 --2- 172.21.15.31:0/825758317 >> v2:172.21.15.62:3300/0 conn(0x7f39c80af230 0x7f39c815a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:12.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.936+0000 7f39c6ffd700 1 -- 172.21.15.31:0/825758317 >> v1:172.21.15.31:6789/0 conn(0x7f39c80b23c0 legacy=0x7f39c81536e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:12.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.936+0000 7f39c6ffd700 1 -- 172.21.15.31:0/825758317 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39c8159e00 con 0x7f39c80aee60 2023-10-06T02:31:12.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.936+0000 7f39c6ffd700 1 --2- 172.21.15.31:0/825758317 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39c80aee60 0x7f39c80cdd00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f39b80098e0 tx=0x7f39b80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dac75b70c16eb291 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:12.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.936+0000 7f39c5ffb700 1 -- 172.21.15.31:0/825758317 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b8011630 con 0x7f39c80aee60 2023-10-06T02:31:12.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.936+0000 7f39c5ffb700 1 -- 172.21.15.31:0/825758317 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39b8011790 con 0x7f39c80aee60 2023-10-06T02:31:12.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.936+0000 7f39c5ffb700 1 -- 172.21.15.31:0/825758317 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b8011b30 con 0x7f39c80aee60 2023-10-06T02:31:12.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.937+0000 7f39c7fff700 1 -- 172.21.15.31:0/825758317 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39c80aee60 msgr2=0x7f39c80cdd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:12.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.937+0000 7f39c7fff700 1 --2- 172.21.15.31:0/825758317 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39c80aee60 0x7f39c80cdd00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f39b80098e0 tx=0x7f39b80091a0 comp rx=0 tx=0).stop 2023-10-06T02:31:12.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.937+0000 7f39c7fff700 1 -- 172.21.15.31:0/825758317 shutdown_connections 2023-10-06T02:31:12.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.937+0000 7f39c7fff700 1 --2- 172.21.15.31:0/825758317 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39c80aee60 0x7f39c80cdd00 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:12.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.937+0000 7f39c7fff700 1 --2- 172.21.15.31:0/825758317 >> v2:172.21.15.62:3300/0 conn(0x7f39c80af230 0x7f39c815a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:12.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.937+0000 7f39c7fff700 1 -- 172.21.15.31:0/825758317 >> 172.21.15.31:0/825758317 conn(0x7f39c80675a0 msgr2=0x7f39c80679a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:12.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.937+0000 7f39c7fff700 1 -- 172.21.15.31:0/825758317 shutdown_connections 2023-10-06T02:31:12.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.937+0000 7f39c7fff700 1 -- 172.21.15.31:0/825758317 wait complete. 2023-10-06T02:31:12.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.939+0000 7f39c7fff700 1 Processor -- start 2023-10-06T02:31:12.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.939+0000 7f39c7fff700 1 -- start start 2023-10-06T02:31:12.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.940+0000 7f39c7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39c80aee60 0x7f39c8157a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:12.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.940+0000 7f39c7fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39c80af230 0x7f39c8151060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:12.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.940+0000 7f39c7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39c80b23c0 0x7f39c814cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:12.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.940+0000 7f39c7fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39c80a9a00 con 0x7f39c80b23c0 2023-10-06T02:31:12.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.940+0000 7f39c7fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39c8151b20 con 0x7f39c80aee60 2023-10-06T02:31:12.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.940+0000 7f39c7fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39c8169f40 con 0x7f39c80af230 2023-10-06T02:31:12.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.940+0000 7f39c77fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39c80b23c0 0x7f39c814cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:12.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.940+0000 7f39c6ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39c80aee60 0x7f39c8157a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:12.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.940+0000 7f39c67fc700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39c80af230 0x7f39c8151060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:12.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.940+0000 7f39c6ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39c80aee60 0x7f39c8157a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:51238/0 (socket says 172.21.15.31:51238) 2023-10-06T02:31:12.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.940+0000 7f39c77fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39c80b23c0 0x7f39c814cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37540/0 (socket says 172.21.15.31:37540) 2023-10-06T02:31:12.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.940+0000 7f39c6ffd700 1 -- 172.21.15.31:0/1545996198 learned_addr learned my addr 172.21.15.31:0/1545996198 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:12.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.940+0000 7f39c77fe700 1 -- 172.21.15.31:0/1545996198 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39c80af230 msgr2=0x7f39c8151060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:12.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.940+0000 7f39c77fe700 1 --2- 172.21.15.31:0/1545996198 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39c80af230 0x7f39c8151060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:12.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.940+0000 7f39c77fe700 1 -- 172.21.15.31:0/1545996198 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39c80aee60 msgr2=0x7f39c8157a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:12.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.940+0000 7f39c77fe700 1 --2- 172.21.15.31:0/1545996198 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39c80aee60 0x7f39c8157a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:12.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.940+0000 7f39c77fe700 1 -- 172.21.15.31:0/1545996198 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39b8009420 con 0x7f39c80b23c0 2023-10-06T02:31:12.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.941+0000 7f39c77fe700 1 --2- 172.21.15.31:0/1545996198 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39c80b23c0 0x7f39c814cf00 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f39c000cb90 tx=0x7f39c0001980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:12.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.941+0000 7f39aeffd700 1 -- 172.21.15.31:0/1545996198 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c000a7e0 con 0x7f39c80b23c0 2023-10-06T02:31:12.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.941+0000 7f39aeffd700 1 -- 172.21.15.31:0/1545996198 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39c000a940 con 0x7f39c80b23c0 2023-10-06T02:31:12.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.941+0000 7f39aeffd700 1 -- 172.21.15.31:0/1545996198 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c000acc0 con 0x7f39c80b23c0 2023-10-06T02:31:12.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.941+0000 7f39c7fff700 1 -- 172.21.15.31:0/1545996198 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39c816a220 con 0x7f39c80b23c0 2023-10-06T02:31:12.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.941+0000 7f39c7fff700 1 -- 172.21.15.31:0/1545996198 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39c816a5b0 con 0x7f39c80b23c0 2023-10-06T02:31:12.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.942+0000 7f39c7fff700 1 -- 172.21.15.31:0/1545996198 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39c8069b50 con 0x7f39c80b23c0 2023-10-06T02:31:12.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.943+0000 7f39aeffd700 1 -- 172.21.15.31:0/1545996198 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 85403+0+0 (secure 0 0 0) 0x7f39c0007300 con 0x7f39c80b23c0 2023-10-06T02:31:12.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.945+0000 7f39aeffd700 1 --2- 172.21.15.31:0/1545996198 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f39b4067680 0x7f39b4069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:12.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.945+0000 7f39aeffd700 1 -- 172.21.15.31:0/1545996198 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6172+0+0 (secure 0 0 0) 0x7f39c0085350 con 0x7f39c80b23c0 2023-10-06T02:31:12.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.945+0000 7f39c6ffd700 1 --2- 172.21.15.31:0/1545996198 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f39b4067680 0x7f39b4069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:12.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.946+0000 7f39c6ffd700 1 --2- 172.21.15.31:0/1545996198 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f39b4067680 0x7f39b4069b30 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f39c8159af0 tx=0x7f39b8017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:12.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:12.949+0000 7f39aeffd700 1 -- 172.21.15.31:0/1545996198 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f39c004b5a0 con 0x7f39c80b23c0 2023-10-06T02:31:13.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:13.200+0000 7f39c7fff700 1 -- 172.21.15.31:0/1545996198 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f39c816aae0 con 0x7f39c80b23c0 2023-10-06T02:31:13.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:13.725+0000 7f39aeffd700 1 -- 172.21.15.31:0/1545996198 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v103) v1 ==== 104+0+0 (secure 0 0 0) 0x7f39c00826c0 con 0x7f39c80b23c0 2023-10-06T02:31:13.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:13.728+0000 7f39c7fff700 1 -- 172.21.15.31:0/1545996198 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f39b4067680 msgr2=0x7f39b4069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:13.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:13.728+0000 7f39c7fff700 1 --2- 172.21.15.31:0/1545996198 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f39b4067680 0x7f39b4069b30 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f39c8159af0 tx=0x7f39b8017000 comp rx=0 tx=0).stop 2023-10-06T02:31:13.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:13.728+0000 7f39c7fff700 1 -- 172.21.15.31:0/1545996198 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39c80b23c0 msgr2=0x7f39c814cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:13.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:13.728+0000 7f39c7fff700 1 --2- 172.21.15.31:0/1545996198 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39c80b23c0 0x7f39c814cf00 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f39c000cb90 tx=0x7f39c0001980 comp rx=0 tx=0).stop 2023-10-06T02:31:13.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:13.728+0000 7f39c7fff700 1 -- 172.21.15.31:0/1545996198 shutdown_connections 2023-10-06T02:31:13.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:13.728+0000 7f39c7fff700 1 --2- 172.21.15.31:0/1545996198 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39c80b23c0 0x7f39c814cf00 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:13.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:13.728+0000 7f39c7fff700 1 --2- 172.21.15.31:0/1545996198 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39c80aee60 0x7f39c8157a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:13.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:13.728+0000 7f39c7fff700 1 --2- 172.21.15.31:0/1545996198 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f39b4067680 0x7f39b4069b30 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:13.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:13.728+0000 7f39c7fff700 1 --2- 172.21.15.31:0/1545996198 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39c80af230 0x7f39c8151060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:13.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:13.728+0000 7f39c7fff700 1 -- 172.21.15.31:0/1545996198 >> 172.21.15.31:0/1545996198 conn(0x7f39c80675a0 msgr2=0x7f39c80acc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:13.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:13.728+0000 7f39c7fff700 1 -- 172.21.15.31:0/1545996198 shutdown_connections 2023-10-06T02:31:13.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:13.728+0000 7f39c7fff700 1 -- 172.21.15.31:0/1545996198 wait complete. 2023-10-06T02:31:13.749 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T02:31:13.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:13.943+0000 7f0437fff700 1 Processor -- start 2023-10-06T02:31:13.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:13.944+0000 7f0437fff700 1 -- start start 2023-10-06T02:31:13.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:13.945+0000 7f0437fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0438153ac0 0x7f043815a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:13.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:13.945+0000 7f0437fff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f043815abf0 con 0x7f04380cacb0 2023-10-06T02:31:13.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:13.945+0000 7f0437fff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0438070940 con 0x7f0438153e90 2023-10-06T02:31:13.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:13.945+0000 7f0437fff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f043806a330 con 0x7f0438153ac0 2023-10-06T02:31:13.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:13.945+0000 7f0436ffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0438153ac0 0x7f043815a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:13.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:13.945+0000 7f04377fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f04380cacb0 0x7f043806c6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49750/0 (socket says 172.21.15.31:49750) 2023-10-06T02:31:13.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:13.945+0000 7f04377fe700 1 -- 172.21.15.31:0/4043888955 learned_addr learned my addr 172.21.15.31:0/4043888955 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:31:13.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:13.946+0000 7f0435ffb700 1 -- 172.21.15.31:0/4043888955 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1191746694 0 0) 0x7f043815abf0 con 0x7f04380cacb0 2023-10-06T02:31:13.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:13.946+0000 7f0436ffd700 1 -- 172.21.15.31:0/4043888955 >> v1:172.21.15.62:6789/0 conn(0x7f0438153e90 legacy=0x7f04380ca5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:13.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:13.946+0000 7f0436ffd700 1 -- 172.21.15.31:0/4043888955 >> v1:172.21.15.31:6789/0 conn(0x7f04380cacb0 legacy=0x7f043806c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:13.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:13.946+0000 7f0436ffd700 1 -- 172.21.15.31:0/4043888955 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f043806a610 con 0x7f0438153ac0 2023-10-06T02:31:13.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:13.946+0000 7f0436ffd700 1 --2- 172.21.15.31:0/4043888955 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0438153ac0 0x7f043815a6b0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f04280098e0 tx=0x7f04280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b0706fb689d21c3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:13.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:13.946+0000 7f0435ffb700 1 -- 172.21.15.31:0/4043888955 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0428009e80 con 0x7f0438153ac0 2023-10-06T02:31:13.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:13.946+0000 7f0435ffb700 1 -- 172.21.15.31:0/4043888955 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f042800b860 con 0x7f0438153ac0 2023-10-06T02:31:13.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:13.946+0000 7f0435ffb700 1 -- 172.21.15.31:0/4043888955 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f042800bbe0 con 0x7f0438153ac0 2023-10-06T02:31:13.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:13.947+0000 7f0437fff700 1 -- 172.21.15.31:0/4043888955 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0438153ac0 msgr2=0x7f043815a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:13.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:13.947+0000 7f0437fff700 1 --2- 172.21.15.31:0/4043888955 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0438153ac0 0x7f043815a6b0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f04280098e0 tx=0x7f04280091a0 comp rx=0 tx=0).stop 2023-10-06T02:31:13.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:13.947+0000 7f0437fff700 1 -- 172.21.15.31:0/4043888955 shutdown_connections 2023-10-06T02:31:13.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:13.947+0000 7f0437fff700 1 --2- 172.21.15.31:0/4043888955 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0438153ac0 0x7f043815a6b0 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:13.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:13.947+0000 7f0437fff700 1 -- 172.21.15.31:0/4043888955 >> 172.21.15.31:0/4043888955 conn(0x7f043806e790 msgr2=0x7f043806eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:13.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:13.947+0000 7f0437fff700 1 -- 172.21.15.31:0/4043888955 shutdown_connections 2023-10-06T02:31:13.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:13.947+0000 7f0437fff700 1 -- 172.21.15.31:0/4043888955 wait complete. 2023-10-06T02:31:13.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:13.948+0000 7f0437fff700 1 Processor -- start 2023-10-06T02:31:13.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:13.949+0000 7f0437fff700 1 -- start start 2023-10-06T02:31:13.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:13.949+0000 7f0437fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04380cacb0 0x7f043814e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:13.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:13.950+0000 7f0437fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0438153ac0 0x7f043816a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:13.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:13.950+0000 7f0437fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0438153e90 0x7f04380c8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:13.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:13.950+0000 7f0437fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f043815dc60 con 0x7f0438153e90 2023-10-06T02:31:13.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:13.950+0000 7f0437fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04380c33d0 con 0x7f04380cacb0 2023-10-06T02:31:13.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:13.950+0000 7f0437fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f043814ed80 con 0x7f0438153ac0 2023-10-06T02:31:13.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:13.950+0000 7f04377fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0438153e90 0x7f04380c8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:13.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:13.950+0000 7f04377fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0438153e90 0x7f04380c8fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37554/0 (socket says 172.21.15.31:37554) 2023-10-06T02:31:13.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:13.950+0000 7f04377fe700 1 -- 172.21.15.31:0/1067074453 learned_addr learned my addr 172.21.15.31:0/1067074453 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:13.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:13.950+0000 7f0436ffd700 1 --2- 172.21.15.31:0/1067074453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04380cacb0 0x7f043814e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:13.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:13.950+0000 7f04367fc700 1 --2- 172.21.15.31:0/1067074453 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0438153ac0 0x7f043816a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:13.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:13.950+0000 7f04377fe700 1 -- 172.21.15.31:0/1067074453 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0438153ac0 msgr2=0x7f043816a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:13.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:13.950+0000 7f04377fe700 1 --2- 172.21.15.31:0/1067074453 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0438153ac0 0x7f043816a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:13.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:13.950+0000 7f04377fe700 1 -- 172.21.15.31:0/1067074453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04380cacb0 msgr2=0x7f043814e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:13.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:13.950+0000 7f04377fe700 1 --2- 172.21.15.31:0/1067074453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04380cacb0 0x7f043814e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:13.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:13.950+0000 7f04377fe700 1 -- 172.21.15.31:0/1067074453 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0428009420 con 0x7f0438153e90 2023-10-06T02:31:13.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:13.950+0000 7f0436ffd700 1 --2- 172.21.15.31:0/1067074453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04380cacb0 0x7f043814e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:31:13.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:13.951+0000 7f04377fe700 1 --2- 172.21.15.31:0/1067074453 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0438153e90 0x7f04380c8fc0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f043000ccc0 tx=0x7f0430001a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:13.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:13.951+0000 7f041effd700 1 -- 172.21.15.31:0/1067074453 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f043000ab10 con 0x7f0438153e90 2023-10-06T02:31:13.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:13.951+0000 7f041effd700 1 -- 172.21.15.31:0/1067074453 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f043000ac70 con 0x7f0438153e90 2023-10-06T02:31:13.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:13.951+0000 7f041effd700 1 -- 172.21.15.31:0/1067074453 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04300070f0 con 0x7f0438153e90 2023-10-06T02:31:13.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:13.951+0000 7f0437fff700 1 -- 172.21.15.31:0/1067074453 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f043816aaf0 con 0x7f0438153e90 2023-10-06T02:31:13.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:13.951+0000 7f0437fff700 1 -- 172.21.15.31:0/1067074453 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f043816ae30 con 0x7f0438153e90 2023-10-06T02:31:13.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:13.952+0000 7f0437fff700 1 -- 172.21.15.31:0/1067074453 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f043804c3c0 con 0x7f0438153e90 2023-10-06T02:31:13.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:13.953+0000 7f041effd700 1 -- 172.21.15.31:0/1067074453 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 85403+0+0 (secure 0 0 0) 0x7f04300074e0 con 0x7f0438153e90 2023-10-06T02:31:13.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:13.954+0000 7f041effd700 1 --2- 172.21.15.31:0/1067074453 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0424067680 0x7f0424069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:13.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:13.954+0000 7f041effd700 1 -- 172.21.15.31:0/1067074453 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6172+0+0 (secure 0 0 0) 0x7f0430085b30 con 0x7f0438153e90 2023-10-06T02:31:13.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:13.955+0000 7f0436ffd700 1 --2- 172.21.15.31:0/1067074453 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0424067680 0x7f0424069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:13.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:13.955+0000 7f0436ffd700 1 --2- 172.21.15.31:0/1067074453 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0424067680 0x7f0424069b30 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f04280098e0 tx=0x7f0428017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:13.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:13.958+0000 7f041effd700 1 -- 172.21.15.31:0/1067074453 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0430082ea0 con 0x7f0438153e90 2023-10-06T02:31:14.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:14.160+0000 7f0437fff700 1 -- 172.21.15.31:0/1067074453 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f043816aaf0 con 0x7f0438153e90 2023-10-06T02:31:15.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:15.169+0000 7f041effd700 1 -- 172.21.15.31:0/1067074453 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v146) v1 ==== 181+0+0 (secure 0 0 0) 0x7f043816aaf0 con 0x7f0438153e90 2023-10-06T02:31:15.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:15.173+0000 7f0437fff700 1 -- 172.21.15.31:0/1067074453 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0424067680 msgr2=0x7f0424069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:15.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:15.173+0000 7f0437fff700 1 --2- 172.21.15.31:0/1067074453 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0424067680 0x7f0424069b30 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f04280098e0 tx=0x7f0428017000 comp rx=0 tx=0).stop 2023-10-06T02:31:15.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:15.173+0000 7f0437fff700 1 -- 172.21.15.31:0/1067074453 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0438153e90 msgr2=0x7f04380c8fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:15.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:15.173+0000 7f0437fff700 1 --2- 172.21.15.31:0/1067074453 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0438153e90 0x7f04380c8fc0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f043000ccc0 tx=0x7f0430001a20 comp rx=0 tx=0).stop 2023-10-06T02:31:15.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:15.174+0000 7f0437fff700 1 -- 172.21.15.31:0/1067074453 shutdown_connections 2023-10-06T02:31:15.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:15.174+0000 7f0437fff700 1 --2- 172.21.15.31:0/1067074453 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0438153e90 0x7f04380c8fc0 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:15.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:15.174+0000 7f0437fff700 1 --2- 172.21.15.31:0/1067074453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04380cacb0 0x7f043814e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:15.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:15.174+0000 7f0437fff700 1 --2- 172.21.15.31:0/1067074453 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0424067680 0x7f0424069b30 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:15.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:15.174+0000 7f0437fff700 1 --2- 172.21.15.31:0/1067074453 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0438153ac0 0x7f043816a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:15.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:15.174+0000 7f0437fff700 1 -- 172.21.15.31:0/1067074453 >> 172.21.15.31:0/1067074453 conn(0x7f043806e790 msgr2=0x7f04380c3a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:15.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:15.174+0000 7f0437fff700 1 -- 172.21.15.31:0/1067074453 shutdown_connections 2023-10-06T02:31:15.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:15.174+0000 7f0437fff700 1 -- 172.21.15.31:0/1067074453 wait complete. 2023-10-06T02:31:15.176 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' removed 2023-10-06T02:31:15.195 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T02:31:15.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:15.389+0000 7f979319d700 1 Processor -- start 2023-10-06T02:31:15.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:15.389+0000 7f979319d700 1 -- start start 2023-10-06T02:31:15.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:15.391+0000 7f979319d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f978c153ac0 0x7f978c15a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:15.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:15.391+0000 7f979319d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f978c153e90 0x7f978c0ca5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:15.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:15.391+0000 7f979319d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f978c15abf0 con 0x7f978c0703a0 2023-10-06T02:31:15.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:15.391+0000 7f979319d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f978c070940 con 0x7f978c153e90 2023-10-06T02:31:15.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:15.391+0000 7f979319d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f978c06a330 con 0x7f978c153ac0 2023-10-06T02:31:15.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:15.391+0000 7f979299c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f978c0703a0 0x7f978c15d260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49754/0 (socket says 172.21.15.31:49754) 2023-10-06T02:31:15.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:15.391+0000 7f979299c700 1 -- 172.21.15.31:0/1016637737 learned_addr learned my addr 172.21.15.31:0/1016637737 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:31:15.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:15.391+0000 7f979199a700 1 --2- 172.21.15.31:0/1016637737 >> v2:172.21.15.31:3300/0 conn(0x7f978c153e90 0x7f978c0ca5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:15.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:15.391+0000 7f979219b700 1 --2- 172.21.15.31:0/1016637737 >> v2:172.21.15.62:3300/0 conn(0x7f978c153ac0 0x7f978c15a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:15.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:15.392+0000 7f9791199700 1 -- 172.21.15.31:0/1016637737 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4165909813 0 0) 0x7f978c15abf0 con 0x7f978c0703a0 2023-10-06T02:31:15.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:15.392+0000 7f9791199700 1 -- 172.21.15.31:0/1016637737 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9778003570 con 0x7f978c0703a0 2023-10-06T02:31:15.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:15.392+0000 7f979199a700 1 -- 172.21.15.31:0/1016637737 >> v2:172.21.15.62:3300/0 conn(0x7f978c153ac0 msgr2=0x7f978c15a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:15.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:15.392+0000 7f979199a700 1 --2- 172.21.15.31:0/1016637737 >> v2:172.21.15.62:3300/0 conn(0x7f978c153ac0 0x7f978c15a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:15.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:15.392+0000 7f979199a700 1 -- 172.21.15.31:0/1016637737 >> v1:172.21.15.31:6789/0 conn(0x7f978c0703a0 legacy=0x7f978c15d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:15.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:15.392+0000 7f979199a700 1 -- 172.21.15.31:0/1016637737 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f978c06a610 con 0x7f978c153e90 2023-10-06T02:31:15.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:15.392+0000 7f979199a700 1 --2- 172.21.15.31:0/1016637737 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f978c153e90 0x7f978c0ca5a0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f978c0cc930 tx=0x7f977c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6814e79a48907385 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:15.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:15.392+0000 7f9791199700 1 -- 172.21.15.31:0/1016637737 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f977c011630 con 0x7f978c153e90 2023-10-06T02:31:15.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:15.392+0000 7f9791199700 1 -- 172.21.15.31:0/1016637737 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f977c0117b0 con 0x7f978c153e90 2023-10-06T02:31:15.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:15.392+0000 7f9791199700 1 -- 172.21.15.31:0/1016637737 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f977c011b30 con 0x7f978c153e90 2023-10-06T02:31:15.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:15.393+0000 7f979319d700 1 -- 172.21.15.31:0/1016637737 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f978c153e90 msgr2=0x7f978c0ca5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:15.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:15.393+0000 7f979319d700 1 --2- 172.21.15.31:0/1016637737 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f978c153e90 0x7f978c0ca5a0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f978c0cc930 tx=0x7f977c0091a0 comp rx=0 tx=0).stop 2023-10-06T02:31:15.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:15.393+0000 7f979319d700 1 -- 172.21.15.31:0/1016637737 shutdown_connections 2023-10-06T02:31:15.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:15.393+0000 7f979319d700 1 --2- 172.21.15.31:0/1016637737 >> v2:172.21.15.62:3300/0 conn(0x7f978c153ac0 0x7f978c15a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:15.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:15.393+0000 7f979319d700 1 --2- 172.21.15.31:0/1016637737 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f978c153e90 0x7f978c0ca5a0 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:15.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:15.393+0000 7f979319d700 1 -- 172.21.15.31:0/1016637737 >> 172.21.15.31:0/1016637737 conn(0x7f978c06e790 msgr2=0x7f978c06eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:15.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:15.393+0000 7f979319d700 1 -- 172.21.15.31:0/1016637737 shutdown_connections 2023-10-06T02:31:15.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:15.393+0000 7f979319d700 1 -- 172.21.15.31:0/1016637737 wait complete. 2023-10-06T02:31:15.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:15.394+0000 7f979319d700 1 Processor -- start 2023-10-06T02:31:15.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:15.395+0000 7f979319d700 1 -- start start 2023-10-06T02:31:15.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:15.395+0000 7f979319d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f978c0703a0 0x7f978c0b1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:15.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:15.395+0000 7f979319d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f978c153ac0 0x7f978c14e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:15.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:15.395+0000 7f979319d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f978c153e90 0x7f978c15d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:15.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:15.395+0000 7f979319d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f978c0b19d0 con 0x7f978c0703a0 2023-10-06T02:31:15.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:15.395+0000 7f979319d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f978c0b0e40 con 0x7f978c153ac0 2023-10-06T02:31:15.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:15.395+0000 7f979319d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f978c0b10a0 con 0x7f978c153e90 2023-10-06T02:31:15.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:15.396+0000 7f979219b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f978c0703a0 0x7f978c0b1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:15.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:15.396+0000 7f979219b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f978c0703a0 0x7f978c0b1e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37574/0 (socket says 172.21.15.31:37574) 2023-10-06T02:31:15.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:15.396+0000 7f979219b700 1 -- 172.21.15.31:0/3194241525 learned_addr learned my addr 172.21.15.31:0/3194241525 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:15.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:15.396+0000 7f979299c700 1 --2- 172.21.15.31:0/3194241525 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f978c153e90 0x7f978c15d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:15.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:15.396+0000 7f979199a700 1 --2- 172.21.15.31:0/3194241525 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f978c153ac0 0x7f978c14e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:15.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:15.396+0000 7f979219b700 1 -- 172.21.15.31:0/3194241525 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f978c153e90 msgr2=0x7f978c15d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:15.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:15.396+0000 7f979219b700 1 --2- 172.21.15.31:0/3194241525 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f978c153e90 0x7f978c15d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:15.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:15.396+0000 7f979219b700 1 -- 172.21.15.31:0/3194241525 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f978c153ac0 msgr2=0x7f978c14e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:15.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:15.396+0000 7f979219b700 1 --2- 172.21.15.31:0/3194241525 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f978c153ac0 0x7f978c14e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:15.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:15.396+0000 7f979219b700 1 -- 172.21.15.31:0/3194241525 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f977c009420 con 0x7f978c0703a0 2023-10-06T02:31:15.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:15.396+0000 7f979299c700 1 --2- 172.21.15.31:0/3194241525 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f978c153e90 0x7f978c15d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:31:15.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:15.396+0000 7f979199a700 1 --2- 172.21.15.31:0/3194241525 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f978c153ac0 0x7f978c14e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:31:15.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:15.397+0000 7f979219b700 1 --2- 172.21.15.31:0/3194241525 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f978c0703a0 0x7f978c0b1e60 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f978800ec80 tx=0x7f97880091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:15.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:15.397+0000 7f97827fc700 1 -- 172.21.15.31:0/3194241525 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f978800ca60 con 0x7f978c0703a0 2023-10-06T02:31:15.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:15.397+0000 7f97827fc700 1 -- 172.21.15.31:0/3194241525 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f978800cbc0 con 0x7f978c0703a0 2023-10-06T02:31:15.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:15.397+0000 7f97827fc700 1 -- 172.21.15.31:0/3194241525 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9788013470 con 0x7f978c0703a0 2023-10-06T02:31:15.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:15.397+0000 7f979319d700 1 -- 172.21.15.31:0/3194241525 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f978c0b1380 con 0x7f978c0703a0 2023-10-06T02:31:15.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:15.397+0000 7f979319d700 1 -- 172.21.15.31:0/3194241525 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f978c14eac0 con 0x7f978c0703a0 2023-10-06T02:31:15.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:15.398+0000 7f979319d700 1 -- 172.21.15.31:0/3194241525 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f978c04c3c0 con 0x7f978c0703a0 2023-10-06T02:31:15.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:15.399+0000 7f97827fc700 1 -- 172.21.15.31:0/3194241525 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 85403+0+0 (secure 0 0 0) 0x7f9788013820 con 0x7f978c0703a0 2023-10-06T02:31:15.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:15.400+0000 7f97827fc700 1 --2- 172.21.15.31:0/3194241525 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f97700676b0 0x7f9770069b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:15.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:15.400+0000 7f97827fc700 1 -- 172.21.15.31:0/3194241525 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 5734+0+0 (secure 0 0 0) 0x7f9788085b00 con 0x7f978c0703a0 2023-10-06T02:31:15.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:15.400+0000 7f979199a700 1 --2- 172.21.15.31:0/3194241525 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f97700676b0 0x7f9770069b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:15.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:15.401+0000 7f979199a700 1 --2- 172.21.15.31:0/3194241525 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f97700676b0 0x7f9770069b60 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f978c1542e0 tx=0x7f977c017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:15.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:15.405+0000 7f97827fc700 1 -- 172.21.15.31:0/3194241525 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9788083020 con 0x7f978c0703a0 2023-10-06T02:31:15.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:15.605+0000 7f979319d700 1 -- 172.21.15.31:0/3194241525 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f978c159820 con 0x7f978c0703a0 2023-10-06T02:31:16.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.179+0000 7f97827fc700 1 -- 172.21.15.31:0/3194241525 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v147) v1 ==== 169+0+0 (secure 0 0 0) 0x7f97880170d0 con 0x7f978c0703a0 2023-10-06T02:31:16.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.183+0000 7f979319d700 1 -- 172.21.15.31:0/3194241525 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f97700676b0 msgr2=0x7f9770069b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:16.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.183+0000 7f979319d700 1 --2- 172.21.15.31:0/3194241525 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f97700676b0 0x7f9770069b60 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f978c1542e0 tx=0x7f977c017000 comp rx=0 tx=0).stop 2023-10-06T02:31:16.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.183+0000 7f979319d700 1 -- 172.21.15.31:0/3194241525 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f978c0703a0 msgr2=0x7f978c0b1e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:16.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.183+0000 7f979319d700 1 --2- 172.21.15.31:0/3194241525 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f978c0703a0 0x7f978c0b1e60 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f978800ec80 tx=0x7f97880091a0 comp rx=0 tx=0).stop 2023-10-06T02:31:16.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.183+0000 7f979319d700 1 -- 172.21.15.31:0/3194241525 shutdown_connections 2023-10-06T02:31:16.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.183+0000 7f979319d700 1 --2- 172.21.15.31:0/3194241525 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f97700676b0 0x7f9770069b60 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:16.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.183+0000 7f979319d700 1 --2- 172.21.15.31:0/3194241525 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f978c153e90 0x7f978c15d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:16.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.183+0000 7f979319d700 1 --2- 172.21.15.31:0/3194241525 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f978c0703a0 0x7f978c0b1e60 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:16.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.183+0000 7f979319d700 1 --2- 172.21.15.31:0/3194241525 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f978c153ac0 0x7f978c14e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:16.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.183+0000 7f979319d700 1 -- 172.21.15.31:0/3194241525 >> 172.21.15.31:0/3194241525 conn(0x7f978c06e790 msgr2=0x7f978c06db40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:16.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.184+0000 7f979319d700 1 -- 172.21.15.31:0/3194241525 shutdown_connections 2023-10-06T02:31:16.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.184+0000 7f979319d700 1 -- 172.21.15.31:0/3194241525 wait complete. 2023-10-06T02:31:16.186 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' removed 2023-10-06T02:31:16.205 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T02:31:16.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.405+0000 7fcf449fe700 1 Processor -- start 2023-10-06T02:31:16.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.405+0000 7fcf449fe700 1 -- start start 2023-10-06T02:31:16.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.406+0000 7fcf449fe700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fcf400c7be0 0x7fcf4006abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:16.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.406+0000 7fcf449fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcf4013f780 0x7fcf4013fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:16.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.406+0000 7fcf449fe700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf40140120 con 0x7fcf400c7fb0 2023-10-06T02:31:16.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.406+0000 7fcf449fe700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf4006bf10 con 0x7fcf400c7be0 2023-10-06T02:31:16.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.406+0000 7fcf449fe700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf4006b290 con 0x7fcf4013f780 2023-10-06T02:31:16.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.407+0000 7fcf3e59c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fcf400c7fb0 0x7fcf4015a990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49770/0 (socket says 172.21.15.31:49770) 2023-10-06T02:31:16.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.407+0000 7fcf3e59c700 1 -- 172.21.15.31:0/541892277 learned_addr learned my addr 172.21.15.31:0/541892277 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:31:16.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.407+0000 7fcf3ed9d700 1 --2- 172.21.15.31:0/541892277 >> v2:172.21.15.62:3300/0 conn(0x7fcf400c7be0 0x7fcf4006abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:16.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.407+0000 7fcf3f59e700 1 --2- 172.21.15.31:0/541892277 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcf4013f780 0x7fcf4013fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:16.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.407+0000 7fcf3dd9b700 1 -- 172.21.15.31:0/541892277 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2210486267 0 0) 0x7fcf34001110 con 0x7fcf400c7fb0 2023-10-06T02:31:16.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.407+0000 7fcf3dd9b700 1 -- 172.21.15.31:0/541892277 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf28003570 con 0x7fcf400c7fb0 2023-10-06T02:31:16.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.407+0000 7fcf3dd9b700 1 -- 172.21.15.31:0/541892277 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 405954882 0 0) 0x7fcf28003570 con 0x7fcf400c7fb0 2023-10-06T02:31:16.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.408+0000 7fcf3dd9b700 1 -- 172.21.15.31:0/541892277 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcf4013f780 msgr2=0x7fcf4013fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:16.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.408+0000 7fcf3dd9b700 1 --2- 172.21.15.31:0/541892277 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcf4013f780 0x7fcf4013fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:16.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.408+0000 7fcf3dd9b700 1 -- 172.21.15.31:0/541892277 >> v2:172.21.15.62:3300/0 conn(0x7fcf400c7be0 msgr2=0x7fcf4006abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:16.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.408+0000 7fcf3dd9b700 1 --2- 172.21.15.31:0/541892277 >> v2:172.21.15.62:3300/0 conn(0x7fcf400c7be0 0x7fcf4006abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:16.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.408+0000 7fcf3dd9b700 1 -- 172.21.15.31:0/541892277 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf4006b570 con 0x7fcf400c7fb0 2023-10-06T02:31:16.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.408+0000 7fcf3dd9b700 1 -- 172.21.15.31:0/541892277 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7fcf34001600 con 0x7fcf400c7fb0 2023-10-06T02:31:16.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.408+0000 7fcf3dd9b700 1 -- 172.21.15.31:0/541892277 >> v1:172.21.15.31:6789/0 conn(0x7fcf400c7fb0 legacy=0x7fcf4015a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:16.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.408+0000 7fcf3dd9b700 1 --2- 172.21.15.31:0/541892277 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf28003a60 0x7fcf28003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:16.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.408+0000 7fcf3dd9b700 1 --2- 172.21.15.31:0/541892277 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcf28004810 0x7fcf28004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:16.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.408+0000 7fcf3dd9b700 1 --2- 172.21.15.31:0/541892277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcf280051c0 0x7fcf2800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:16.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.408+0000 7fcf3dd9b700 1 -- 172.21.15.31:0/541892277 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf4006bf10 con 0x7fcf28003a60 2023-10-06T02:31:16.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.408+0000 7fcf3dd9b700 1 -- 172.21.15.31:0/541892277 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf4006b290 con 0x7fcf280051c0 2023-10-06T02:31:16.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.408+0000 7fcf3dd9b700 1 -- 172.21.15.31:0/541892277 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2800bfa0 con 0x7fcf28004810 2023-10-06T02:31:16.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.408+0000 7fcf3ed9d700 1 --2- 172.21.15.31:0/541892277 >> v2:172.21.15.62:3300/0 conn(0x7fcf400c7be0 0x7fcf4006abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:31:16.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.408+0000 7fcf3f59e700 1 --2- 172.21.15.31:0/541892277 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcf4013f780 0x7fcf4013fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:31:16.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.408+0000 7fcf3ed9d700 1 --2- 172.21.15.31:0/541892277 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf28003a60 0x7fcf28003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:16.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.408+0000 7fcf3e59c700 1 --2- 172.21.15.31:0/541892277 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcf28004810 0x7fcf28004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:16.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.408+0000 7fcf3f59e700 1 --2- 172.21.15.31:0/541892277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcf280051c0 0x7fcf2800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:16.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.408+0000 7fcf3ed9d700 1 -- 172.21.15.31:0/541892277 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcf28004810 msgr2=0x7fcf28004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:16.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.408+0000 7fcf3ed9d700 1 --2- 172.21.15.31:0/541892277 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcf28004810 0x7fcf28004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:16.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.408+0000 7fcf3ed9d700 1 -- 172.21.15.31:0/541892277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcf280051c0 msgr2=0x7fcf2800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:16.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.408+0000 7fcf3ed9d700 1 --2- 172.21.15.31:0/541892277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcf280051c0 0x7fcf2800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:16.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.408+0000 7fcf3ed9d700 1 -- 172.21.15.31:0/541892277 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf28003570 con 0x7fcf28003a60 2023-10-06T02:31:16.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.409+0000 7fcf3ed9d700 1 --2- 172.21.15.31:0/541892277 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf28003a60 0x7fcf28003e70 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7fcf30009db0 tx=0x7fcf300091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee834fe023d87d8a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:16.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.409+0000 7fcf3dd9b700 1 -- 172.21.15.31:0/541892277 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf30009740 con 0x7fcf28003a60 2023-10-06T02:31:16.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.409+0000 7fcf3dd9b700 1 -- 172.21.15.31:0/541892277 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf3000e920 con 0x7fcf28003a60 2023-10-06T02:31:16.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.409+0000 7fcf3dd9b700 1 -- 172.21.15.31:0/541892277 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf3000eca0 con 0x7fcf28003a60 2023-10-06T02:31:16.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.409+0000 7fcf449fe700 1 -- 172.21.15.31:0/541892277 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf28003a60 msgr2=0x7fcf28003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:16.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.409+0000 7fcf449fe700 1 --2- 172.21.15.31:0/541892277 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf28003a60 0x7fcf28003e70 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7fcf30009db0 tx=0x7fcf300091a0 comp rx=0 tx=0).stop 2023-10-06T02:31:16.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.409+0000 7fcf3f59e700 1 -- 172.21.15.31:0/541892277 reap_dead start 2023-10-06T02:31:16.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.410+0000 7fcf449fe700 1 -- 172.21.15.31:0/541892277 shutdown_connections 2023-10-06T02:31:16.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.410+0000 7fcf449fe700 1 -- 172.21.15.31:0/541892277 >> 172.21.15.31:0/541892277 conn(0x7fcf4006d110 msgr2=0x7fcf400cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:16.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.410+0000 7fcf449fe700 1 -- 172.21.15.31:0/541892277 shutdown_connections 2023-10-06T02:31:16.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.410+0000 7fcf449fe700 1 -- 172.21.15.31:0/541892277 wait complete. 2023-10-06T02:31:16.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.411+0000 7fcf449fe700 1 Processor -- start 2023-10-06T02:31:16.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.411+0000 7fcf449fe700 1 -- start start 2023-10-06T02:31:16.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.412+0000 7fcf449fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcf28003a60 0x7fcf40168180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:16.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.412+0000 7fcf449fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcf4013ce70 0x7fcf400c3d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:16.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.412+0000 7fcf449fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf400c4c40 0x7fcf400c5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:16.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.412+0000 7fcf449fe700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf40159ba0 con 0x7fcf400c4c40 2023-10-06T02:31:16.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.412+0000 7fcf449fe700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf400c38d0 con 0x7fcf4013ce70 2023-10-06T02:31:16.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.412+0000 7fcf449fe700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf400c55d0 con 0x7fcf28003a60 2023-10-06T02:31:16.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.412+0000 7fcf3f59e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf400c4c40 0x7fcf400c5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:16.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.412+0000 7fcf3ed9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcf28003a60 0x7fcf40168180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:16.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.412+0000 7fcf3ed9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcf28003a60 0x7fcf40168180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:35810/0 (socket says 172.21.15.31:35810) 2023-10-06T02:31:16.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.412+0000 7fcf3ed9d700 1 -- 172.21.15.31:0/3413380155 learned_addr learned my addr 172.21.15.31:0/3413380155 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:16.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.412+0000 7fcf3e59c700 1 --2- 172.21.15.31:0/3413380155 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcf4013ce70 0x7fcf400c3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:16.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.413+0000 7fcf3f59e700 1 -- 172.21.15.31:0/3413380155 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcf28003a60 msgr2=0x7fcf40168180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:16.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.413+0000 7fcf3f59e700 1 --2- 172.21.15.31:0/3413380155 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcf28003a60 0x7fcf40168180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:16.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.413+0000 7fcf3f59e700 1 -- 172.21.15.31:0/3413380155 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcf4013ce70 msgr2=0x7fcf400c3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:16.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.413+0000 7fcf3f59e700 1 --2- 172.21.15.31:0/3413380155 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcf4013ce70 0x7fcf400c3d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:16.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.413+0000 7fcf3f59e700 1 -- 172.21.15.31:0/3413380155 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf401686c0 con 0x7fcf400c4c40 2023-10-06T02:31:16.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.413+0000 7fcf3ed9d700 1 --2- 172.21.15.31:0/3413380155 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcf28003a60 0x7fcf40168180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:31:16.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.413+0000 7fcf3f59e700 1 --2- 172.21.15.31:0/3413380155 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf400c4c40 0x7fcf400c5090 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7fcf400ccac0 tx=0x7fcf3800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:16.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.413+0000 7fcf26ffd700 1 -- 172.21.15.31:0/3413380155 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf3800e040 con 0x7fcf400c4c40 2023-10-06T02:31:16.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.413+0000 7fcf26ffd700 1 -- 172.21.15.31:0/3413380155 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf3800f040 con 0x7fcf400c4c40 2023-10-06T02:31:16.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.413+0000 7fcf26ffd700 1 -- 172.21.15.31:0/3413380155 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf3800fd90 con 0x7fcf400c4c40 2023-10-06T02:31:16.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.413+0000 7fcf449fe700 1 -- 172.21.15.31:0/3413380155 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf400c42a0 con 0x7fcf400c4c40 2023-10-06T02:31:16.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.414+0000 7fcf449fe700 1 -- 172.21.15.31:0/3413380155 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf401598c0 con 0x7fcf400c4c40 2023-10-06T02:31:16.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.415+0000 7fcf449fe700 1 -- 172.21.15.31:0/3413380155 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf400c3a60 con 0x7fcf400c4c40 2023-10-06T02:31:16.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.415+0000 7fcf26ffd700 1 -- 172.21.15.31:0/3413380155 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 85403+0+0 (secure 0 0 0) 0x7fcf38015640 con 0x7fcf400c4c40 2023-10-06T02:31:16.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.417+0000 7fcf26ffd700 1 --2- 172.21.15.31:0/3413380155 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fcf2c067680 0x7fcf2c069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:16.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.417+0000 7fcf26ffd700 1 -- 172.21.15.31:0/3413380155 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 5320+0+0 (secure 0 0 0) 0x7fcf38082dd0 con 0x7fcf400c4c40 2023-10-06T02:31:16.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.418+0000 7fcf3ed9d700 1 --2- 172.21.15.31:0/3413380155 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fcf2c067680 0x7fcf2c069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:16.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.419+0000 7fcf3ed9d700 1 --2- 172.21.15.31:0/3413380155 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fcf2c067680 0x7fcf2c069b30 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fcf300093f0 tx=0x7fcf30019000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:16.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.421+0000 7fcf26ffd700 1 -- 172.21.15.31:0/3413380155 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcf38082ae0 con 0x7fcf400c4c40 2023-10-06T02:31:16.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.624+0000 7fcf449fe700 1 -- 172.21.15.31:0/3413380155 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fcf4004c3c0 con 0x7fcf400c4c40 2023-10-06T02:31:16.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.628+0000 7fcf26ffd700 1 -- 172.21.15.31:0/3413380155 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v27) v1 ==== 104+0+0 (secure 0 0 0) 0x7fcf380518d0 con 0x7fcf400c4c40 2023-10-06T02:31:16.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.630+0000 7fcf449fe700 1 -- 172.21.15.31:0/3413380155 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fcf2c067680 msgr2=0x7fcf2c069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:16.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.630+0000 7fcf449fe700 1 --2- 172.21.15.31:0/3413380155 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fcf2c067680 0x7fcf2c069b30 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fcf300093f0 tx=0x7fcf30019000 comp rx=0 tx=0).stop 2023-10-06T02:31:16.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.630+0000 7fcf449fe700 1 -- 172.21.15.31:0/3413380155 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf400c4c40 msgr2=0x7fcf400c5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:16.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.630+0000 7fcf449fe700 1 --2- 172.21.15.31:0/3413380155 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf400c4c40 0x7fcf400c5090 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7fcf400ccac0 tx=0x7fcf3800b040 comp rx=0 tx=0).stop 2023-10-06T02:31:16.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.630+0000 7fcf449fe700 1 -- 172.21.15.31:0/3413380155 shutdown_connections 2023-10-06T02:31:16.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.630+0000 7fcf449fe700 1 --2- 172.21.15.31:0/3413380155 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf400c4c40 0x7fcf400c5090 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:16.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.630+0000 7fcf449fe700 1 --2- 172.21.15.31:0/3413380155 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fcf2c067680 0x7fcf2c069b30 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:16.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.630+0000 7fcf449fe700 1 --2- 172.21.15.31:0/3413380155 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcf28003a60 0x7fcf40168180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:16.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.630+0000 7fcf449fe700 1 --2- 172.21.15.31:0/3413380155 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcf4013ce70 0x7fcf400c3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:16.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.630+0000 7fcf449fe700 1 -- 172.21.15.31:0/3413380155 >> 172.21.15.31:0/3413380155 conn(0x7fcf4006d110 msgr2=0x7fcf4013d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:16.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.630+0000 7fcf449fe700 1 -- 172.21.15.31:0/3413380155 shutdown_connections 2023-10-06T02:31:16.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.631+0000 7fcf449fe700 1 -- 172.21.15.31:0/3413380155 wait complete. 2023-10-06T02:31:16.648 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1' 2023-10-06T02:31:16.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.835+0000 7fcbb119d700 1 Processor -- start 2023-10-06T02:31:16.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.835+0000 7fcbb119d700 1 -- start start 2023-10-06T02:31:16.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.836+0000 7fcbb119d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fcbac153c30 0x7fcbac15a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:16.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.836+0000 7fcbb119d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcbac0cf110 0x7fcbac15c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:16.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.836+0000 7fcbb119d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbac15afd0 con 0x7fcbac15de50 2023-10-06T02:31:16.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.836+0000 7fcbb119d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbac15b2b0 con 0x7fcbac153c30 2023-10-06T02:31:16.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.836+0000 7fcbb119d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbac15b590 con 0x7fcbac0cf110 2023-10-06T02:31:16.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.836+0000 7fcbab7fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fcbac15de50 0x7fcbac0cea00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49774/0 (socket says 172.21.15.31:49774) 2023-10-06T02:31:16.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.836+0000 7fcbab7fe700 1 -- 172.21.15.31:0/2263918573 learned_addr learned my addr 172.21.15.31:0/2263918573 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:31:16.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.837+0000 7fcbabfff700 1 --2- 172.21.15.31:0/2263918573 >> v2:172.21.15.62:3300/0 conn(0x7fcbac153c30 0x7fcbac15a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:16.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.837+0000 7fcbb099c700 1 --2- 172.21.15.31:0/2263918573 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcbac0cf110 0x7fcbac15c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:16.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.837+0000 7fcbaaffd700 1 -- 172.21.15.31:0/2263918573 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1976887845 0 0) 0x7fcbac15afd0 con 0x7fcbac15de50 2023-10-06T02:31:16.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.837+0000 7fcbaaffd700 1 -- 172.21.15.31:0/2263918573 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb94003570 con 0x7fcbac15de50 2023-10-06T02:31:16.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.837+0000 7fcbaaffd700 1 -- 172.21.15.31:0/2263918573 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 350320208 0 0) 0x7fcb94003570 con 0x7fcbac15de50 2023-10-06T02:31:16.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.837+0000 7fcbaaffd700 1 -- 172.21.15.31:0/2263918573 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcbac0cf110 msgr2=0x7fcbac15c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:16.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.837+0000 7fcbaaffd700 1 --2- 172.21.15.31:0/2263918573 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcbac0cf110 0x7fcbac15c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:16.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.837+0000 7fcbaaffd700 1 -- 172.21.15.31:0/2263918573 >> v2:172.21.15.62:3300/0 conn(0x7fcbac153c30 msgr2=0x7fcbac15a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:16.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.837+0000 7fcbaaffd700 1 --2- 172.21.15.31:0/2263918573 >> v2:172.21.15.62:3300/0 conn(0x7fcbac153c30 0x7fcbac15a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:16.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.837+0000 7fcbaaffd700 1 -- 172.21.15.31:0/2263918573 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbac15b870 con 0x7fcbac15de50 2023-10-06T02:31:16.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.837+0000 7fcbaaffd700 1 -- 172.21.15.31:0/2263918573 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7fcba0001470 con 0x7fcbac15de50 2023-10-06T02:31:16.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.837+0000 7fcbaaffd700 1 -- 172.21.15.31:0/2263918573 >> v1:172.21.15.31:6789/0 conn(0x7fcbac15de50 legacy=0x7fcbac0cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:16.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.837+0000 7fcbaaffd700 1 --2- 172.21.15.31:0/2263918573 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb94003ae0 0x7fcb94003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:16.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.837+0000 7fcbaaffd700 1 --2- 172.21.15.31:0/2263918573 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcbac0cf110 0x7fcb94004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:16.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.837+0000 7fcbaaffd700 1 --2- 172.21.15.31:0/2263918573 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb94004d50 0x7fcb940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:16.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.837+0000 7fcbaaffd700 1 -- 172.21.15.31:0/2263918573 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbac15b2b0 con 0x7fcb94003ae0 2023-10-06T02:31:16.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.837+0000 7fcbaaffd700 1 -- 172.21.15.31:0/2263918573 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbac15b590 con 0x7fcb94004d50 2023-10-06T02:31:16.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.837+0000 7fcbaaffd700 1 -- 172.21.15.31:0/2263918573 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb9400bb40 con 0x7fcbac0cf110 2023-10-06T02:31:16.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.837+0000 7fcbabfff700 1 --2- 172.21.15.31:0/2263918573 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb94003ae0 0x7fcb94003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:16.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.838+0000 7fcbab7fe700 1 --2- 172.21.15.31:0/2263918573 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcbac0cf110 0x7fcb94004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:16.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.838+0000 7fcbb099c700 1 --2- 172.21.15.31:0/2263918573 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb94004d50 0x7fcb940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:16.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.838+0000 7fcbabfff700 1 -- 172.21.15.31:0/2263918573 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcbac0cf110 msgr2=0x7fcb94004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:16.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.838+0000 7fcbabfff700 1 --2- 172.21.15.31:0/2263918573 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcbac0cf110 0x7fcb94004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:16.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.838+0000 7fcbabfff700 1 -- 172.21.15.31:0/2263918573 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb94004d50 msgr2=0x7fcb940051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:16.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.838+0000 7fcbabfff700 1 --2- 172.21.15.31:0/2263918573 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb94004d50 0x7fcb940051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:16.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.838+0000 7fcbabfff700 1 -- 172.21.15.31:0/2263918573 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb94003570 con 0x7fcb94003ae0 2023-10-06T02:31:16.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.838+0000 7fcbb099c700 1 --2- 172.21.15.31:0/2263918573 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb94004d50 0x7fcb940051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:31:16.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.838+0000 7fcbabfff700 1 --2- 172.21.15.31:0/2263918573 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb94003ae0 0x7fcb94003ef0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c009c00 tx=0x7fcb9c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26f28ee2e13be038 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:16.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.838+0000 7fcbaaffd700 1 -- 172.21.15.31:0/2263918573 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb9c00e6b0 con 0x7fcb94003ae0 2023-10-06T02:31:16.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.838+0000 7fcbaaffd700 1 -- 172.21.15.31:0/2263918573 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb9c00e810 con 0x7fcb94003ae0 2023-10-06T02:31:16.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.838+0000 7fcbaaffd700 1 -- 172.21.15.31:0/2263918573 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb9c00ebd0 con 0x7fcb94003ae0 2023-10-06T02:31:16.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.839+0000 7fcbb119d700 1 -- 172.21.15.31:0/2263918573 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb94003ae0 msgr2=0x7fcb94003ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:16.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.839+0000 7fcbb119d700 1 --2- 172.21.15.31:0/2263918573 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb94003ae0 0x7fcb94003ef0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c009c00 tx=0x7fcb9c0091a0 comp rx=0 tx=0).stop 2023-10-06T02:31:16.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.839+0000 7fcbb099c700 1 -- 172.21.15.31:0/2263918573 reap_dead start 2023-10-06T02:31:16.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.839+0000 7fcbb119d700 1 -- 172.21.15.31:0/2263918573 shutdown_connections 2023-10-06T02:31:16.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.839+0000 7fcbb119d700 1 -- 172.21.15.31:0/2263918573 >> 172.21.15.31:0/2263918573 conn(0x7fcbac0703a0 msgr2=0x7fcbac0707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:16.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.839+0000 7fcbb119d700 1 -- 172.21.15.31:0/2263918573 shutdown_connections 2023-10-06T02:31:16.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.839+0000 7fcbb119d700 1 -- 172.21.15.31:0/2263918573 wait complete. 2023-10-06T02:31:16.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.840+0000 7fcbb119d700 1 Processor -- start 2023-10-06T02:31:16.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.841+0000 7fcbb119d700 1 -- start start 2023-10-06T02:31:16.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.841+0000 7fcbb119d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb94003ae0 0x7fcbac140b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:16.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.841+0000 7fcbb119d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcbac0b5ab0 0x7fcbac1411e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:16.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.841+0000 7fcbb119d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcbac141720 0x7fcbac0b3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:16.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.841+0000 7fcbb119d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbac0b4a10 con 0x7fcb94003ae0 2023-10-06T02:31:16.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.841+0000 7fcbb119d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbac0b4cf0 con 0x7fcbac141720 2023-10-06T02:31:16.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.841+0000 7fcbb119d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbac0b4fd0 con 0x7fcbac0b5ab0 2023-10-06T02:31:16.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.842+0000 7fcbabfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb94003ae0 0x7fcbac140b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:16.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.842+0000 7fcbabfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb94003ae0 0x7fcbac140b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37608/0 (socket says 172.21.15.31:37608) 2023-10-06T02:31:16.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.842+0000 7fcbabfff700 1 -- 172.21.15.31:0/470152108 learned_addr learned my addr 172.21.15.31:0/470152108 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:16.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.842+0000 7fcbab7fe700 1 --2- 172.21.15.31:0/470152108 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcbac0b5ab0 0x7fcbac1411e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:16.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.842+0000 7fcbb099c700 1 --2- 172.21.15.31:0/470152108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcbac141720 0x7fcbac0b3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:16.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.842+0000 7fcbabfff700 1 -- 172.21.15.31:0/470152108 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcbac0b5ab0 msgr2=0x7fcbac1411e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:16.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.842+0000 7fcbabfff700 1 --2- 172.21.15.31:0/470152108 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcbac0b5ab0 0x7fcbac1411e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:16.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.842+0000 7fcbabfff700 1 -- 172.21.15.31:0/470152108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcbac141720 msgr2=0x7fcbac0b3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:16.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.842+0000 7fcbabfff700 1 --2- 172.21.15.31:0/470152108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcbac141720 0x7fcbac0b3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:16.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.842+0000 7fcbabfff700 1 -- 172.21.15.31:0/470152108 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbac0b52b0 con 0x7fcb94003ae0 2023-10-06T02:31:16.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.842+0000 7fcbab7fe700 1 --2- 172.21.15.31:0/470152108 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcbac0b5ab0 0x7fcbac1411e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:31:16.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.842+0000 7fcbb099c700 1 --2- 172.21.15.31:0/470152108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcbac141720 0x7fcbac0b3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:31:16.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.842+0000 7fcbabfff700 1 --2- 172.21.15.31:0/470152108 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb94003ae0 0x7fcbac140b00 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c00e4b0 tx=0x7fcb9c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:16.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.843+0000 7fcb93fff700 1 -- 172.21.15.31:0/470152108 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb9c022070 con 0x7fcb94003ae0 2023-10-06T02:31:16.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.843+0000 7fcb93fff700 1 -- 172.21.15.31:0/470152108 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb9c00e980 con 0x7fcb94003ae0 2023-10-06T02:31:16.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.843+0000 7fcb93fff700 1 -- 172.21.15.31:0/470152108 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb9c027470 con 0x7fcb94003ae0 2023-10-06T02:31:16.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.843+0000 7fcbb119d700 1 -- 172.21.15.31:0/470152108 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbac0b5590 con 0x7fcb94003ae0 2023-10-06T02:31:16.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.843+0000 7fcbb119d700 1 -- 172.21.15.31:0/470152108 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbac0b5f20 con 0x7fcb94003ae0 2023-10-06T02:31:16.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.844+0000 7fcbb119d700 1 -- 172.21.15.31:0/470152108 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbac04c3c0 con 0x7fcb94003ae0 2023-10-06T02:31:16.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.845+0000 7fcb93fff700 1 -- 172.21.15.31:0/470152108 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 85403+0+0 (secure 0 0 0) 0x7fcb9c01c450 con 0x7fcb94003ae0 2023-10-06T02:31:16.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.846+0000 7fcb93fff700 1 --2- 172.21.15.31:0/470152108 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fcb98067680 0x7fcb98069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:16.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.846+0000 7fcb93fff700 1 -- 172.21.15.31:0/470152108 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 5320+0+0 (secure 0 0 0) 0x7fcb9c087120 con 0x7fcb94003ae0 2023-10-06T02:31:16.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.847+0000 7fcbab7fe700 1 --2- 172.21.15.31:0/470152108 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fcb98067680 0x7fcb98069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:16.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.847+0000 7fcbab7fe700 1 --2- 172.21.15.31:0/470152108 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fcb98067680 0x7fcb98069b30 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fcba0001ed0 tx=0x7fcba0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:16.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:16.850+0000 7fcb93fff700 1 -- 172.21.15.31:0/470152108 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcb9c0847e0 con 0x7fcb94003ae0 2023-10-06T02:31:17.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:17.133+0000 7fcbb119d700 1 -- 172.21.15.31:0/470152108 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1"]} v 0) v1 -- 0x7fcbac15bb50 con 0x7fcb94003ae0 2023-10-06T02:31:17.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:17.187+0000 7fcb93fff700 1 -- 172.21.15.31:0/470152108 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1"]}]=0 v458) v1 ==== 149+0+0 (secure 0 0 0) 0x7fcb9c0170d0 con 0x7fcb94003ae0 2023-10-06T02:31:17.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:17.190+0000 7fcbb119d700 1 -- 172.21.15.31:0/470152108 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fcb98067680 msgr2=0x7fcb98069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:17.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:17.190+0000 7fcbb119d700 1 --2- 172.21.15.31:0/470152108 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fcb98067680 0x7fcb98069b30 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fcba0001ed0 tx=0x7fcba0009040 comp rx=0 tx=0).stop 2023-10-06T02:31:17.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:17.190+0000 7fcbb119d700 1 -- 172.21.15.31:0/470152108 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb94003ae0 msgr2=0x7fcbac140b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:17.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:17.190+0000 7fcbb119d700 1 --2- 172.21.15.31:0/470152108 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb94003ae0 0x7fcbac140b00 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c00e4b0 tx=0x7fcb9c01a000 comp rx=0 tx=0).stop 2023-10-06T02:31:17.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:17.190+0000 7fcbb119d700 1 -- 172.21.15.31:0/470152108 shutdown_connections 2023-10-06T02:31:17.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:17.190+0000 7fcbb119d700 1 --2- 172.21.15.31:0/470152108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcbac141720 0x7fcbac0b3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:17.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:17.190+0000 7fcbb119d700 1 --2- 172.21.15.31:0/470152108 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb94003ae0 0x7fcbac140b00 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:17.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:17.190+0000 7fcbb119d700 1 --2- 172.21.15.31:0/470152108 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fcb98067680 0x7fcb98069b30 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:17.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:17.190+0000 7fcbb119d700 1 --2- 172.21.15.31:0/470152108 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcbac0b5ab0 0x7fcbac1411e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:17.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:17.190+0000 7fcbb119d700 1 -- 172.21.15.31:0/470152108 >> 172.21.15.31:0/470152108 conn(0x7fcbac0703a0 msgr2=0x7fcbac0c69b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:17.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:17.190+0000 7fcbb119d700 1 -- 172.21.15.31:0/470152108 shutdown_connections 2023-10-06T02:31:17.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:17.190+0000 7fcbb119d700 1 -- 172.21.15.31:0/470152108 wait complete. 2023-10-06T02:31:17.208 INFO:tasks.cephfs_test_runner:poor man's upgrade test -- rather than going through a full upgrade cycle, ... ok 2023-10-06T02:31:17.208 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_upgrade_v1_to_v2 (tasks.cephfs.test_volumes.TestMisc) poor man's upgrade test -- theme continues... 2023-10-06T02:31:17.208 INFO:tasks.cephfs_test_runner:test_subvolume_upgrade_v1_to_v2 (tasks.cephfs.test_volumes.TestMisc) 2023-10-06T02:31:17.209 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2' 2023-10-06T02:31:17.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:17.397+0000 7f790e79d700 1 Processor -- start 2023-10-06T02:31:17.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:17.397+0000 7f790e79d700 1 -- start start 2023-10-06T02:31:17.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:17.398+0000 7f790e79d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7908159640 0x7f790806d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:17.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:17.398+0000 7f790e79d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f79080caec0 0x7f79080c78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:17.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:17.398+0000 7f790e79d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79080c7420 con 0x7f7908159a10 2023-10-06T02:31:17.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:17.398+0000 7f790e79d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f790806d5a0 con 0x7f79080caec0 2023-10-06T02:31:17.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:17.398+0000 7f790e79d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f790806d820 con 0x7f7908159640 2023-10-06T02:31:17.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:17.399+0000 7f790df9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f79080caec0 0x7f79080c78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:17.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:17.399+0000 7f790df9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f79080caec0 0x7f79080c78b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37620/0 (socket says 172.21.15.31:37620) 2023-10-06T02:31:17.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:17.399+0000 7f790df9c700 1 -- 172.21.15.31:0/4231460501 learned_addr learned my addr 172.21.15.31:0/4231460501 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:17.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:17.399+0000 7f790d79b700 1 --2- 172.21.15.31:0/4231460501 >> v2:172.21.15.62:3300/0 conn(0x7f7908159640 0x7f790806d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:17.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:17.399+0000 7f790df9c700 1 -- 172.21.15.31:0/4231460501 >> v2:172.21.15.62:3300/0 conn(0x7f7908159640 msgr2=0x7f790806d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:17.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:17.399+0000 7f790df9c700 1 --2- 172.21.15.31:0/4231460501 >> v2:172.21.15.62:3300/0 conn(0x7f7908159640 0x7f790806d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:17.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:17.399+0000 7f790df9c700 1 -- 172.21.15.31:0/4231460501 >> v1:172.21.15.62:6789/0 conn(0x7f7908159a10 legacy=0x7f79080ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:17.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:17.399+0000 7f790df9c700 1 -- 172.21.15.31:0/4231460501 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f790806db00 con 0x7f79080caec0 2023-10-06T02:31:17.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:17.400+0000 7f790df9c700 1 --2- 172.21.15.31:0/4231460501 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79080caec0 0x7f79080c78b0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f7900009f20 tx=0x7f7900009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfe5a733bcf5672f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:17.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:17.400+0000 7f78fffff700 1 -- 172.21.15.31:0/4231460501 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f790000e070 con 0x7f79080caec0 2023-10-06T02:31:17.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:17.400+0000 7f78fffff700 1 -- 172.21.15.31:0/4231460501 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f790000ba40 con 0x7f79080caec0 2023-10-06T02:31:17.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:17.400+0000 7f78fffff700 1 -- 172.21.15.31:0/4231460501 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79000092e0 con 0x7f79080caec0 2023-10-06T02:31:17.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:17.400+0000 7f790e79d700 1 -- 172.21.15.31:0/4231460501 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79080caec0 msgr2=0x7f79080c78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:17.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:17.400+0000 7f790e79d700 1 --2- 172.21.15.31:0/4231460501 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79080caec0 0x7f79080c78b0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f7900009f20 tx=0x7f7900009840 comp rx=0 tx=0).stop 2023-10-06T02:31:17.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:17.400+0000 7f790e79d700 1 -- 172.21.15.31:0/4231460501 shutdown_connections 2023-10-06T02:31:17.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:17.400+0000 7f790e79d700 1 --2- 172.21.15.31:0/4231460501 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79080caec0 0x7f79080c78b0 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:17.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:17.400+0000 7f790e79d700 1 --2- 172.21.15.31:0/4231460501 >> v2:172.21.15.62:3300/0 conn(0x7f7908159640 0x7f790806d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:17.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:17.400+0000 7f790e79d700 1 -- 172.21.15.31:0/4231460501 >> 172.21.15.31:0/4231460501 conn(0x7f790806e720 msgr2=0x7f790806eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:17.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:17.401+0000 7f790e79d700 1 -- 172.21.15.31:0/4231460501 shutdown_connections 2023-10-06T02:31:17.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:17.401+0000 7f790e79d700 1 -- 172.21.15.31:0/4231460501 wait complete. 2023-10-06T02:31:17.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:17.402+0000 7f790e79d700 1 Processor -- start 2023-10-06T02:31:17.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:17.402+0000 7f790e79d700 1 -- start start 2023-10-06T02:31:17.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:17.403+0000 7f790e79d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79080caec0 0x7f790816dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:17.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:17.403+0000 7f790e79d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7908159640 0x7f79080c3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:17.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:17.403+0000 7f790e79d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7908159a10 0x7f790814efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:17.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:17.403+0000 7f790e79d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7908151800 con 0x7f79080caec0 2023-10-06T02:31:17.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:17.403+0000 7f790e79d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79080c69d0 con 0x7f7908159a10 2023-10-06T02:31:17.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:17.403+0000 7f790e79d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79081511b0 con 0x7f7908159640 2023-10-06T02:31:17.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:17.403+0000 7f790d79b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79080caec0 0x7f790816dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:17.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:17.403+0000 7f790d79b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79080caec0 0x7f790816dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37622/0 (socket says 172.21.15.31:37622) 2023-10-06T02:31:17.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:17.403+0000 7f790d79b700 1 -- 172.21.15.31:0/2461870657 learned_addr learned my addr 172.21.15.31:0/2461870657 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:17.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:17.403+0000 7f790df9c700 1 --2- 172.21.15.31:0/2461870657 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7908159a10 0x7f790814efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:17.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:17.403+0000 7f790cf9a700 1 --2- 172.21.15.31:0/2461870657 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7908159640 0x7f79080c3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:17.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:17.404+0000 7f790d79b700 1 -- 172.21.15.31:0/2461870657 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7908159640 msgr2=0x7f79080c3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:17.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:17.404+0000 7f790d79b700 1 --2- 172.21.15.31:0/2461870657 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7908159640 0x7f79080c3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:17.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:17.404+0000 7f790d79b700 1 -- 172.21.15.31:0/2461870657 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7908159a10 msgr2=0x7f790814efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:17.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:17.404+0000 7f790d79b700 1 --2- 172.21.15.31:0/2461870657 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7908159a10 0x7f790814efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:17.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:17.404+0000 7f790d79b700 1 -- 172.21.15.31:0/2461870657 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7900009ae0 con 0x7f79080caec0 2023-10-06T02:31:17.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:17.404+0000 7f790cf9a700 1 --2- 172.21.15.31:0/2461870657 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7908159640 0x7f79080c3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:31:17.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:17.404+0000 7f790df9c700 1 --2- 172.21.15.31:0/2461870657 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7908159a10 0x7f790814efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:31:17.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:17.404+0000 7f790d79b700 1 --2- 172.21.15.31:0/2461870657 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79080caec0 0x7f790816dd90 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f790400d890 tx=0x7f7904008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:17.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:17.404+0000 7f78fd7fa700 1 -- 172.21.15.31:0/2461870657 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79040125f0 con 0x7f79080caec0 2023-10-06T02:31:17.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:17.404+0000 7f78fd7fa700 1 -- 172.21.15.31:0/2461870657 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7904012770 con 0x7f79080caec0 2023-10-06T02:31:17.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:17.404+0000 7f78fd7fa700 1 -- 172.21.15.31:0/2461870657 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7904012b30 con 0x7f79080caec0 2023-10-06T02:31:17.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:17.404+0000 7f790e79d700 1 -- 172.21.15.31:0/2461870657 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7908151490 con 0x7f79080caec0 2023-10-06T02:31:17.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:17.404+0000 7f790e79d700 1 -- 172.21.15.31:0/2461870657 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79080c3fc0 con 0x7f79080caec0 2023-10-06T02:31:17.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:17.405+0000 7f790e79d700 1 -- 172.21.15.31:0/2461870657 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79080c4180 con 0x7f79080caec0 2023-10-06T02:31:17.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:17.406+0000 7f78fd7fa700 1 -- 172.21.15.31:0/2461870657 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 85403+0+0 (secure 0 0 0) 0x7f790401e460 con 0x7f79080caec0 2023-10-06T02:31:17.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:17.407+0000 7f78fd7fa700 1 --2- 172.21.15.31:0/2461870657 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f78ec067680 0x7f78ec069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:17.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:17.408+0000 7f78fd7fa700 1 -- 172.21.15.31:0/2461870657 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 5320+0+0 (secure 0 0 0) 0x7f7904085c10 con 0x7f79080caec0 2023-10-06T02:31:17.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:17.408+0000 7f790cf9a700 1 --2- 172.21.15.31:0/2461870657 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f78ec067680 0x7f78ec069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:17.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:17.409+0000 7f790cf9a700 1 --2- 172.21.15.31:0/2461870657 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f78ec067680 0x7f78ec069b30 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f7908156c10 tx=0x7f78f8009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:17.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:17.412+0000 7f78fd7fa700 1 -- 172.21.15.31:0/2461870657 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f79040832d0 con 0x7f79080caec0 2023-10-06T02:31:17.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:17.696+0000 7f790e79d700 1 -- 172.21.15.31:0/2461870657 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2"]} v 0) v1 -- 0x7f790804c3c0 con 0x7f79080caec0 2023-10-06T02:31:18.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.190+0000 7f78fd7fa700 1 -- 172.21.15.31:0/2461870657 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2"]}]=0 v459) v1 ==== 148+0+0 (secure 0 0 0) 0x7f790804c3c0 con 0x7f79080caec0 2023-10-06T02:31:18.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.192+0000 7f790e79d700 1 -- 172.21.15.31:0/2461870657 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f78ec067680 msgr2=0x7f78ec069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:18.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.192+0000 7f790e79d700 1 --2- 172.21.15.31:0/2461870657 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f78ec067680 0x7f78ec069b30 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f7908156c10 tx=0x7f78f8009040 comp rx=0 tx=0).stop 2023-10-06T02:31:18.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.192+0000 7f790e79d700 1 -- 172.21.15.31:0/2461870657 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79080caec0 msgr2=0x7f790816dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:18.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.192+0000 7f790e79d700 1 --2- 172.21.15.31:0/2461870657 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79080caec0 0x7f790816dd90 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f790400d890 tx=0x7f7904008040 comp rx=0 tx=0).stop 2023-10-06T02:31:18.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.193+0000 7f790e79d700 1 -- 172.21.15.31:0/2461870657 shutdown_connections 2023-10-06T02:31:18.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.193+0000 7f790e79d700 1 --2- 172.21.15.31:0/2461870657 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7908159a10 0x7f790814efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:18.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.193+0000 7f790e79d700 1 --2- 172.21.15.31:0/2461870657 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79080caec0 0x7f790816dd90 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:18.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.193+0000 7f790e79d700 1 --2- 172.21.15.31:0/2461870657 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f78ec067680 0x7f78ec069b30 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:18.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.193+0000 7f790e79d700 1 --2- 172.21.15.31:0/2461870657 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7908159640 0x7f79080c3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:18.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.193+0000 7f790e79d700 1 -- 172.21.15.31:0/2461870657 >> 172.21.15.31:0/2461870657 conn(0x7f790806e720 msgr2=0x7f7908157160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:18.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.193+0000 7f790e79d700 1 -- 172.21.15.31:0/2461870657 shutdown_connections 2023-10-06T02:31:18.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.193+0000 7f790e79d700 1 -- 172.21.15.31:0/2461870657 wait complete. 2023-10-06T02:31:18.212 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T02:31:18.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.403+0000 7fd215d9d700 1 Processor -- start 2023-10-06T02:31:18.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.403+0000 7fd215d9d700 1 -- start start 2023-10-06T02:31:18.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.404+0000 7fd215d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd210153a00 0x7fd210153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:18.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.404+0000 7fd215d9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd21015de50 0x7fd21015a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:18.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.404+0000 7fd215d9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd2100cea00 0x7fd2100cee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:18.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.404+0000 7fd215d9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd21015af60 con 0x7fd2100cea00 2023-10-06T02:31:18.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.404+0000 7fd215d9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd21015b240 con 0x7fd21015de50 2023-10-06T02:31:18.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.404+0000 7fd215d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd21015b520 con 0x7fd210153a00 2023-10-06T02:31:18.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.405+0000 7fd21559c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd2100cea00 0x7fd2100cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:18.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.405+0000 7fd21559c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd2100cea00 0x7fd2100cee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59626/0 (socket says 172.21.15.31:59626) 2023-10-06T02:31:18.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.405+0000 7fd21559c700 1 -- 172.21.15.31:0/3416882245 learned_addr learned my addr 172.21.15.31:0/3416882245 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:18.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.405+0000 7fd214d9b700 1 --2- 172.21.15.31:0/3416882245 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd210153a00 0x7fd210153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:18.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.405+0000 7fd20ffff700 1 --2- 172.21.15.31:0/3416882245 >> v2:172.21.15.62:3300/0 conn(0x7fd21015de50 0x7fd21015a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:18.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.405+0000 7fd21559c700 1 -- 172.21.15.31:0/3416882245 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd210153a00 msgr2=0x7fd210153dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:18.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.405+0000 7fd21559c700 1 --2- 172.21.15.31:0/3416882245 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd210153a00 0x7fd210153dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:18.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.405+0000 7fd21559c700 1 -- 172.21.15.31:0/3416882245 >> v2:172.21.15.62:3300/0 conn(0x7fd21015de50 msgr2=0x7fd21015a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:18.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.405+0000 7fd21559c700 1 --2- 172.21.15.31:0/3416882245 >> v2:172.21.15.62:3300/0 conn(0x7fd21015de50 0x7fd21015a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:18.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.405+0000 7fd21559c700 1 -- 172.21.15.31:0/3416882245 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd21015b800 con 0x7fd2100cea00 2023-10-06T02:31:18.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.405+0000 7fd214d9b700 1 --2- 172.21.15.31:0/3416882245 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd210153a00 0x7fd210153dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:31:18.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.405+0000 7fd20ffff700 1 --2- 172.21.15.31:0/3416882245 >> v2:172.21.15.62:3300/0 conn(0x7fd21015de50 0x7fd21015a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:31:18.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.405+0000 7fd21559c700 1 --2- 172.21.15.31:0/3416882245 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2100cea00 0x7fd2100cee40 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7fd21006e800 tx=0x7fd208009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=927716af470abf03 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:18.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.406+0000 7fd20f7fe700 1 -- 172.21.15.31:0/3416882245 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd20800e070 con 0x7fd2100cea00 2023-10-06T02:31:18.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.406+0000 7fd20f7fe700 1 -- 172.21.15.31:0/3416882245 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd20800ba50 con 0x7fd2100cea00 2023-10-06T02:31:18.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.406+0000 7fd20f7fe700 1 -- 172.21.15.31:0/3416882245 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2080092e0 con 0x7fd2100cea00 2023-10-06T02:31:18.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.406+0000 7fd215d9d700 1 -- 172.21.15.31:0/3416882245 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2100cea00 msgr2=0x7fd2100cee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:18.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.406+0000 7fd215d9d700 1 --2- 172.21.15.31:0/3416882245 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2100cea00 0x7fd2100cee40 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7fd21006e800 tx=0x7fd208009840 comp rx=0 tx=0).stop 2023-10-06T02:31:18.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.406+0000 7fd215d9d700 1 -- 172.21.15.31:0/3416882245 shutdown_connections 2023-10-06T02:31:18.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.406+0000 7fd215d9d700 1 --2- 172.21.15.31:0/3416882245 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2100cea00 0x7fd2100cee40 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:18.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.406+0000 7fd215d9d700 1 --2- 172.21.15.31:0/3416882245 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd210153a00 0x7fd210153dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:18.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.406+0000 7fd215d9d700 1 --2- 172.21.15.31:0/3416882245 >> v2:172.21.15.62:3300/0 conn(0x7fd21015de50 0x7fd21015a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:18.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.406+0000 7fd215d9d700 1 -- 172.21.15.31:0/3416882245 >> 172.21.15.31:0/3416882245 conn(0x7fd2100703a0 msgr2=0x7fd2100707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:18.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.406+0000 7fd215d9d700 1 -- 172.21.15.31:0/3416882245 shutdown_connections 2023-10-06T02:31:18.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.407+0000 7fd215d9d700 1 -- 172.21.15.31:0/3416882245 wait complete. 2023-10-06T02:31:18.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.408+0000 7fd215d9d700 1 Processor -- start 2023-10-06T02:31:18.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.408+0000 7fd215d9d700 1 -- start start 2023-10-06T02:31:18.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.409+0000 7fd215d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd2100cea00 0x7fd2100c7da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:18.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.409+0000 7fd215d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd210153a00 0x7fd2100cd480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:18.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.409+0000 7fd215d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd21015de50 0x7fd2100c9a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:18.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.409+0000 7fd215d9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2100cc400 con 0x7fd21015de50 2023-10-06T02:31:18.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.409+0000 7fd215d9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2100cc6e0 con 0x7fd210153a00 2023-10-06T02:31:18.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.409+0000 7fd215d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2100cc9c0 con 0x7fd2100cea00 2023-10-06T02:31:18.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.409+0000 7fd21559c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd21015de50 0x7fd2100c9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:18.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.409+0000 7fd21559c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd21015de50 0x7fd2100c9a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59634/0 (socket says 172.21.15.31:59634) 2023-10-06T02:31:18.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.409+0000 7fd21559c700 1 -- 172.21.15.31:0/3368219867 learned_addr learned my addr 172.21.15.31:0/3368219867 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:18.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.409+0000 7fd20ffff700 1 --2- 172.21.15.31:0/3368219867 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd210153a00 0x7fd2100cd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:18.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.409+0000 7fd214d9b700 1 --2- 172.21.15.31:0/3368219867 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd2100cea00 0x7fd2100c7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:18.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.410+0000 7fd21559c700 1 -- 172.21.15.31:0/3368219867 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd2100cea00 msgr2=0x7fd2100c7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:18.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.410+0000 7fd21559c700 1 --2- 172.21.15.31:0/3368219867 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd2100cea00 0x7fd2100c7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:18.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.410+0000 7fd21559c700 1 -- 172.21.15.31:0/3368219867 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd210153a00 msgr2=0x7fd2100cd480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:18.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.410+0000 7fd21559c700 1 --2- 172.21.15.31:0/3368219867 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd210153a00 0x7fd2100cd480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:18.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.410+0000 7fd21559c700 1 -- 172.21.15.31:0/3368219867 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd208009ae0 con 0x7fd21015de50 2023-10-06T02:31:18.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.410+0000 7fd214d9b700 1 --2- 172.21.15.31:0/3368219867 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd2100cea00 0x7fd2100c7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:31:18.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.410+0000 7fd21559c700 1 --2- 172.21.15.31:0/3368219867 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd21015de50 0x7fd2100c9a70 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7fd21006eb00 tx=0x7fd208018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:18.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.410+0000 7fd20cff9700 1 -- 172.21.15.31:0/3368219867 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2080032f0 con 0x7fd21015de50 2023-10-06T02:31:18.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.410+0000 7fd20cff9700 1 -- 172.21.15.31:0/3368219867 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd208008dd0 con 0x7fd21015de50 2023-10-06T02:31:18.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.410+0000 7fd20cff9700 1 -- 172.21.15.31:0/3368219867 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd208026690 con 0x7fd21015de50 2023-10-06T02:31:18.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.410+0000 7fd215d9d700 1 -- 172.21.15.31:0/3368219867 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2100ccca0 con 0x7fd21015de50 2023-10-06T02:31:18.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.411+0000 7fd215d9d700 1 -- 172.21.15.31:0/3368219867 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2100c82e0 con 0x7fd21015de50 2023-10-06T02:31:18.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.412+0000 7fd215d9d700 1 -- 172.21.15.31:0/3368219867 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2100cd2f0 con 0x7fd21015de50 2023-10-06T02:31:18.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.412+0000 7fd20cff9700 1 -- 172.21.15.31:0/3368219867 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 85403+0+0 (secure 0 0 0) 0x7fd208003040 con 0x7fd21015de50 2023-10-06T02:31:18.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.413+0000 7fd20cff9700 1 --2- 172.21.15.31:0/3368219867 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd1f4067680 0x7fd1f4069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:18.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.413+0000 7fd20cff9700 1 -- 172.21.15.31:0/3368219867 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 5320+0+0 (secure 0 0 0) 0x7fd208087770 con 0x7fd21015de50 2023-10-06T02:31:18.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.414+0000 7fd214d9b700 1 --2- 172.21.15.31:0/3368219867 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd1f4067680 0x7fd1f4069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:18.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.414+0000 7fd214d9b700 1 --2- 172.21.15.31:0/3368219867 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd1f4067680 0x7fd1f4069b30 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd20400ab50 tx=0x7fd204008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:18.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.418+0000 7fd20cff9700 1 -- 172.21.15.31:0/3368219867 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd208084d20 con 0x7fd21015de50 2023-10-06T02:31:18.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.629+0000 7fd215d9d700 1 -- 172.21.15.31:0/3368219867 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fd2100c84d0 con 0x7fd21015de50 2023-10-06T02:31:18.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.633+0000 7fd20cff9700 1 -- 172.21.15.31:0/3368219867 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v28)=0 v28) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd208053b10 con 0x7fd21015de50 2023-10-06T02:31:18.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.636+0000 7fd215d9d700 1 -- 172.21.15.31:0/3368219867 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd1f4067680 msgr2=0x7fd1f4069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:18.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.636+0000 7fd215d9d700 1 --2- 172.21.15.31:0/3368219867 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd1f4067680 0x7fd1f4069b30 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd20400ab50 tx=0x7fd204008040 comp rx=0 tx=0).stop 2023-10-06T02:31:18.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.636+0000 7fd215d9d700 1 -- 172.21.15.31:0/3368219867 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd21015de50 msgr2=0x7fd2100c9a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:18.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.636+0000 7fd215d9d700 1 --2- 172.21.15.31:0/3368219867 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd21015de50 0x7fd2100c9a70 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7fd21006eb00 tx=0x7fd208018000 comp rx=0 tx=0).stop 2023-10-06T02:31:18.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.636+0000 7fd215d9d700 1 -- 172.21.15.31:0/3368219867 shutdown_connections 2023-10-06T02:31:18.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.636+0000 7fd215d9d700 1 --2- 172.21.15.31:0/3368219867 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd21015de50 0x7fd2100c9a70 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:18.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.636+0000 7fd215d9d700 1 --2- 172.21.15.31:0/3368219867 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd1f4067680 0x7fd1f4069b30 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:18.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.636+0000 7fd215d9d700 1 --2- 172.21.15.31:0/3368219867 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd2100cea00 0x7fd2100c7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:18.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.636+0000 7fd215d9d700 1 --2- 172.21.15.31:0/3368219867 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd210153a00 0x7fd2100cd480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:18.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.636+0000 7fd215d9d700 1 -- 172.21.15.31:0/3368219867 >> 172.21.15.31:0/3368219867 conn(0x7fd2100703a0 msgr2=0x7fd2100b1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:18.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.636+0000 7fd215d9d700 1 -- 172.21.15.31:0/3368219867 shutdown_connections 2023-10-06T02:31:18.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:18.636+0000 7fd215d9d700 1 -- 172.21.15.31:0/3368219867 wait complete. 2023-10-06T02:31:18.654 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T02:31:18.654 DEBUG:teuthology.orchestra.run.smithi031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T02:31:18.737 DEBUG:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T02:31:18.829 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T02:31:18.829 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T02:31:18.829 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:31:19.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.016+0000 7fcac259d700 1 Processor -- start 2023-10-06T02:31:19.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.016+0000 7fcac259d700 1 -- start start 2023-10-06T02:31:19.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.017+0000 7fcac259d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fcabc13fb20 0x7fcabc15a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:19.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.017+0000 7fcac259d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcabc06aaa0 0x7fcabc0c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:19.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.017+0000 7fcac259d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcabc15ae70 con 0x7fcabc13fef0 2023-10-06T02:31:19.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.017+0000 7fcac259d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcabc06af60 con 0x7fcabc13fb20 2023-10-06T02:31:19.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.017+0000 7fcac259d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcabc06b240 con 0x7fcabc06aaa0 2023-10-06T02:31:19.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.018+0000 7fcac159b700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fcabc13fb20 0x7fcabc15a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:19.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.018+0000 7fcac159b700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fcabc13fb20 0x7fcabc15a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:33070/0 (socket says 172.21.15.31:33070) 2023-10-06T02:31:19.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.018+0000 7fcac159b700 1 -- 172.21.15.31:0/3287937233 learned_addr learned my addr 172.21.15.31:0/3287937233 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:19.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.018+0000 7fcac1d9c700 1 --2- 172.21.15.31:0/3287937233 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcabc06aaa0 0x7fcabc0c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:19.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.018+0000 7fcac159b700 1 -- 172.21.15.31:0/3287937233 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcabc06aaa0 msgr2=0x7fcabc0c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:19.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.018+0000 7fcac159b700 1 --2- 172.21.15.31:0/3287937233 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcabc06aaa0 0x7fcabc0c9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:19.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.018+0000 7fcac159b700 1 -- 172.21.15.31:0/3287937233 >> v1:172.21.15.62:6789/0 conn(0x7fcabc13fef0 legacy=0x7fcabc0c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:19.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.018+0000 7fcac159b700 1 -- 172.21.15.31:0/3287937233 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcabc06b520 con 0x7fcabc13fb20 2023-10-06T02:31:19.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.018+0000 7fcac1d9c700 1 --2- 172.21.15.31:0/3287937233 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcabc06aaa0 0x7fcabc0c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:31:19.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.019+0000 7fcac159b700 1 --2- 172.21.15.31:0/3287937233 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcabc13fb20 0x7fcabc15a930 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fcab80099b0 tx=0x7fcab8009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=367430a132796e57 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:19.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.019+0000 7fcab3fff700 1 -- 172.21.15.31:0/3287937233 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab8009ea0 con 0x7fcabc13fb20 2023-10-06T02:31:19.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.019+0000 7fcab3fff700 1 -- 172.21.15.31:0/3287937233 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcab800b860 con 0x7fcabc13fb20 2023-10-06T02:31:19.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.019+0000 7fcab3fff700 1 -- 172.21.15.31:0/3287937233 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab800bc00 con 0x7fcabc13fb20 2023-10-06T02:31:19.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.019+0000 7fcac259d700 1 -- 172.21.15.31:0/3287937233 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcabc13fb20 msgr2=0x7fcabc15a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:19.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.019+0000 7fcac259d700 1 --2- 172.21.15.31:0/3287937233 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcabc13fb20 0x7fcabc15a930 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fcab80099b0 tx=0x7fcab8009250 comp rx=0 tx=0).stop 2023-10-06T02:31:19.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.020+0000 7fcac259d700 1 -- 172.21.15.31:0/3287937233 shutdown_connections 2023-10-06T02:31:19.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.020+0000 7fcac259d700 1 --2- 172.21.15.31:0/3287937233 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcabc06aaa0 0x7fcabc0c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:19.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.020+0000 7fcac259d700 1 --2- 172.21.15.31:0/3287937233 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcabc13fb20 0x7fcabc15a930 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:19.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.020+0000 7fcac259d700 1 -- 172.21.15.31:0/3287937233 >> 172.21.15.31:0/3287937233 conn(0x7fcabc06d170 msgr2=0x7fcabc0cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:19.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.020+0000 7fcac259d700 1 -- 172.21.15.31:0/3287937233 shutdown_connections 2023-10-06T02:31:19.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.020+0000 7fcac259d700 1 -- 172.21.15.31:0/3287937233 wait complete. 2023-10-06T02:31:19.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.021+0000 7fcac259d700 1 Processor -- start 2023-10-06T02:31:19.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.021+0000 7fcac259d700 1 -- start start 2023-10-06T02:31:19.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.022+0000 7fcac259d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcabc06aaa0 0x7fcabc0ab740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:19.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.022+0000 7fcac259d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcabc13fb20 0x7fcabc155bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:19.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.022+0000 7fcac259d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcabc13fef0 0x7fcabc0c22f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:19.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.022+0000 7fcac259d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcabc157560 con 0x7fcabc06aaa0 2023-10-06T02:31:19.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.022+0000 7fcac259d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcabc157820 con 0x7fcabc13fef0 2023-10-06T02:31:19.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.022+0000 7fcac259d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcabc157b00 con 0x7fcabc13fb20 2023-10-06T02:31:19.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.022+0000 7fcac159b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcabc06aaa0 0x7fcabc0ab740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:19.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.022+0000 7fcac159b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcabc06aaa0 0x7fcabc0ab740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59650/0 (socket says 172.21.15.31:59650) 2023-10-06T02:31:19.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.022+0000 7fcac159b700 1 -- 172.21.15.31:0/3692676578 learned_addr learned my addr 172.21.15.31:0/3692676578 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:19.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.023+0000 7fcac1d9c700 1 --2- 172.21.15.31:0/3692676578 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcabc13fef0 0x7fcabc0c22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:19.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.023+0000 7fcac0d9a700 1 --2- 172.21.15.31:0/3692676578 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcabc13fb20 0x7fcabc155bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:19.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.023+0000 7fcac159b700 1 -- 172.21.15.31:0/3692676578 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcabc13fb20 msgr2=0x7fcabc155bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:19.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.023+0000 7fcac159b700 1 --2- 172.21.15.31:0/3692676578 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcabc13fb20 0x7fcabc155bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:19.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.023+0000 7fcac159b700 1 -- 172.21.15.31:0/3692676578 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcabc13fef0 msgr2=0x7fcabc0c22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:19.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.023+0000 7fcac159b700 1 --2- 172.21.15.31:0/3692676578 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcabc13fef0 0x7fcabc0c22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:19.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.023+0000 7fcac159b700 1 -- 172.21.15.31:0/3692676578 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcab4009a20 con 0x7fcabc06aaa0 2023-10-06T02:31:19.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.023+0000 7fcac159b700 1 --2- 172.21.15.31:0/3692676578 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcabc06aaa0 0x7fcabc0ab740 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fcabc071320 tx=0x7fcab8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:19.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.023+0000 7fcab17fa700 1 -- 172.21.15.31:0/3692676578 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab800b9d0 con 0x7fcabc06aaa0 2023-10-06T02:31:19.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.023+0000 7fcab17fa700 1 -- 172.21.15.31:0/3692676578 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcab800bb50 con 0x7fcabc06aaa0 2023-10-06T02:31:19.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.023+0000 7fcac259d700 1 -- 172.21.15.31:0/3692676578 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcab80094f0 con 0x7fcabc06aaa0 2023-10-06T02:31:19.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.023+0000 7fcab17fa700 1 -- 172.21.15.31:0/3692676578 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab8025670 con 0x7fcabc06aaa0 2023-10-06T02:31:19.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.024+0000 7fcac259d700 1 -- 172.21.15.31:0/3692676578 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcabc1580c0 con 0x7fcabc06aaa0 2023-10-06T02:31:19.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.025+0000 7fcac259d700 1 -- 172.21.15.31:0/3692676578 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcabc04c3c0 con 0x7fcabc06aaa0 2023-10-06T02:31:19.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.026+0000 7fcab17fa700 1 -- 172.21.15.31:0/3692676578 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 85403+0+0 (secure 0 0 0) 0x7fcab8023070 con 0x7fcabc06aaa0 2023-10-06T02:31:19.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.026+0000 7fcab17fa700 1 --2- 172.21.15.31:0/3692676578 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fcaa0067680 0x7fcaa0069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:19.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.026+0000 7fcab17fa700 1 -- 172.21.15.31:0/3692676578 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 5320+0+0 (secure 0 0 0) 0x7fcab8084400 con 0x7fcabc06aaa0 2023-10-06T02:31:19.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.027+0000 7fcac0d9a700 1 --2- 172.21.15.31:0/3692676578 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fcaa0067680 0x7fcaa0069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:19.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.028+0000 7fcac0d9a700 1 --2- 172.21.15.31:0/3692676578 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fcaa0067680 0x7fcaa0069b30 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fcabc0c8170 tx=0x7fcaac007eb0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:19.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.031+0000 7fcab17fa700 1 -- 172.21.15.31:0/3692676578 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcab8084110 con 0x7fcabc06aaa0 2023-10-06T02:31:19.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.279+0000 7fcac259d700 1 -- 172.21.15.31:0/3692676578 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcabc0ca160 con 0x7fcabc06aaa0 2023-10-06T02:31:19.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.280+0000 7fcab17fa700 1 -- 172.21.15.31:0/3692676578 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) v1 ==== 95+0+3531 (secure 0 0 0) 0x7fcab802ade0 con 0x7fcabc06aaa0 2023-10-06T02:31:19.280 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:31:19.281 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":103,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6923,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/4169296779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":4169296779},{"type":"v1","addr":"172.21.15.31:6835","nonce":4169296779}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95},{"gid":6926,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/3032245108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3032245108},{"type":"v1","addr":"172.21.15.62:6834","nonce":3032245108}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95},{"gid":8266,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3481660822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":3481660822},{"type":"v1","addr":"172.21.15.62:6835","nonce":3481660822}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":103},{"gid":8469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6833/1522769465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1522769465},{"type":"v1","addr":"172.21.15.31:6833","nonce":1522769465}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94}],"filesystems":[]} 2023-10-06T02:31:19.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.282+0000 7fcac259d700 1 -- 172.21.15.31:0/3692676578 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fcaa0067680 msgr2=0x7fcaa0069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:19.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.282+0000 7fcac259d700 1 --2- 172.21.15.31:0/3692676578 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fcaa0067680 0x7fcaa0069b30 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fcabc0c8170 tx=0x7fcaac007eb0 comp rx=0 tx=0).stop 2023-10-06T02:31:19.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.282+0000 7fcac259d700 1 -- 172.21.15.31:0/3692676578 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcabc06aaa0 msgr2=0x7fcabc0ab740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:19.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.282+0000 7fcac259d700 1 --2- 172.21.15.31:0/3692676578 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcabc06aaa0 0x7fcabc0ab740 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fcabc071320 tx=0x7fcab8017000 comp rx=0 tx=0).stop 2023-10-06T02:31:19.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.282+0000 7fcac259d700 1 -- 172.21.15.31:0/3692676578 shutdown_connections 2023-10-06T02:31:19.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.282+0000 7fcac259d700 1 --2- 172.21.15.31:0/3692676578 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcabc13fef0 0x7fcabc0c22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:19.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.282+0000 7fcac259d700 1 --2- 172.21.15.31:0/3692676578 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcabc06aaa0 0x7fcabc0ab740 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:19.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.282+0000 7fcac259d700 1 --2- 172.21.15.31:0/3692676578 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fcaa0067680 0x7fcaa0069b30 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:19.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.282+0000 7fcac259d700 1 --2- 172.21.15.31:0/3692676578 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcabc13fb20 0x7fcabc155bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:19.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.282+0000 7fcac259d700 1 -- 172.21.15.31:0/3692676578 >> 172.21.15.31:0/3692676578 conn(0x7fcabc06d170 msgr2=0x7fcabc13ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:19.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.282+0000 7fcac259d700 1 -- 172.21.15.31:0/3692676578 shutdown_connections 2023-10-06T02:31:19.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.282+0000 7fcac259d700 1 -- 172.21.15.31:0/3692676578 wait complete. 2023-10-06T02:31:19.285 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 103 2023-10-06T02:31:19.301 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T02:31:19.301 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T02:31:19.301 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T02:31:19.301 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T02:31:19.302 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T02:31:19.302 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T02:31:19.302 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T02:31:19.302 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T02:31:19.302 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T02:31:19.302 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T02:31:19.303 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T02:31:19.303 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T02:31:19.303 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T02:31:19.303 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T02:31:19.303 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T02:31:19.303 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T02:31:19.317 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T02:31:19.317 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T02:31:19.318 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T02:31:19.318 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T02:31:19.321 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T02:31:19.321 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T02:31:19.322 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T02:31:19.322 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T02:31:19.364 INFO:tasks.ceph.mds.c:Started 2023-10-06T02:31:19.364 DEBUG:teuthology.parallel:result is None 2023-10-06T02:31:19.369 INFO:tasks.ceph.mds.a:Started 2023-10-06T02:31:19.370 DEBUG:teuthology.parallel:result is None 2023-10-06T02:31:19.372 INFO:tasks.ceph.mds.b:Started 2023-10-06T02:31:19.373 DEBUG:teuthology.parallel:result is None 2023-10-06T02:31:19.375 INFO:tasks.ceph.mds.d:Started 2023-10-06T02:31:19.375 DEBUG:teuthology.parallel:result is None 2023-10-06T02:31:19.376 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T02:31:19.506 INFO:tasks.ceph.mds.c.smithi031.stdout:starting mds.c at 2023-10-06T02:31:19.510 INFO:tasks.ceph.mds.a.smithi031.stdout:starting mds.a at 2023-10-06T02:31:19.520 INFO:tasks.ceph.mds.b.smithi062.stdout:starting mds.b at 2023-10-06T02:31:19.521 INFO:tasks.ceph.mds.d.smithi062.stdout:starting mds.d at 2023-10-06T02:31:19.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.575+0000 7f826d79d700 1 Processor -- start 2023-10-06T02:31:19.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.575+0000 7f826d79d700 1 -- start start 2023-10-06T02:31:19.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.576+0000 7f826d79d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f826815de50 0x7f82680cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:19.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.576+0000 7f826d79d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82680cf3c0 con 0x7f8268153c30 2023-10-06T02:31:19.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.576+0000 7f826d79d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f826815afd0 con 0x7f82680cef40 2023-10-06T02:31:19.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.576+0000 7f826d79d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f826815b2b0 con 0x7f826815de50 2023-10-06T02:31:19.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.576+0000 7f8267fff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f8268153c30 0x7f826815a5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:39430/0 (socket says 172.21.15.31:39430) 2023-10-06T02:31:19.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.576+0000 7f8267fff700 1 -- 172.21.15.31:0/4137387811 learned_addr learned my addr 172.21.15.31:0/4137387811 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:31:19.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.576+0000 7f82677fe700 1 --2- 172.21.15.31:0/4137387811 >> v2:172.21.15.31:3300/0 conn(0x7f826815de50 0x7f82680cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:19.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.577+0000 7f8266ffd700 1 -- 172.21.15.31:0/4137387811 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3727738215 0 0) 0x7f82680cf3c0 con 0x7f8268153c30 2023-10-06T02:31:19.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.577+0000 7f8266ffd700 1 -- 172.21.15.31:0/4137387811 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8250003570 con 0x7f8268153c30 2023-10-06T02:31:19.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.577+0000 7f8266ffd700 1 -- 172.21.15.31:0/4137387811 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2256217489 0 0) 0x7f826815afd0 con 0x7f82680cef40 2023-10-06T02:31:19.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.577+0000 7f82677fe700 1 -- 172.21.15.31:0/4137387811 >> v1:172.21.15.62:6789/0 conn(0x7f82680cef40 legacy=0x7f826815c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:19.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.577+0000 7f82677fe700 1 -- 172.21.15.31:0/4137387811 >> v1:172.21.15.31:6789/0 conn(0x7f8268153c30 legacy=0x7f826815a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:19.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.577+0000 7f82677fe700 1 -- 172.21.15.31:0/4137387811 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f826815b590 con 0x7f826815de50 2023-10-06T02:31:19.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.577+0000 7f82677fe700 1 --2- 172.21.15.31:0/4137387811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f826815de50 0x7f82680cea00 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f825c0098e0 tx=0x7f825c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=916fd375438aa131 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:19.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.577+0000 7f8266ffd700 1 -- 172.21.15.31:0/4137387811 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825c00b640 con 0x7f826815de50 2023-10-06T02:31:19.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.577+0000 7f8266ffd700 1 -- 172.21.15.31:0/4137387811 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f825c00b7a0 con 0x7f826815de50 2023-10-06T02:31:19.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.577+0000 7f8266ffd700 1 -- 172.21.15.31:0/4137387811 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825c00bb20 con 0x7f826815de50 2023-10-06T02:31:19.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.578+0000 7f826d79d700 1 -- 172.21.15.31:0/4137387811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f826815de50 msgr2=0x7f82680cea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:19.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.578+0000 7f826d79d700 1 --2- 172.21.15.31:0/4137387811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f826815de50 0x7f82680cea00 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f825c0098e0 tx=0x7f825c0091a0 comp rx=0 tx=0).stop 2023-10-06T02:31:19.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.578+0000 7f826d79d700 1 -- 172.21.15.31:0/4137387811 shutdown_connections 2023-10-06T02:31:19.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.578+0000 7f826d79d700 1 --2- 172.21.15.31:0/4137387811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f826815de50 0x7f82680cea00 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:19.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.578+0000 7f826d79d700 1 -- 172.21.15.31:0/4137387811 >> 172.21.15.31:0/4137387811 conn(0x7f82680703a0 msgr2=0x7f82680707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:19.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.578+0000 7f826d79d700 1 -- 172.21.15.31:0/4137387811 shutdown_connections 2023-10-06T02:31:19.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.578+0000 7f826d79d700 1 -- 172.21.15.31:0/4137387811 wait complete. 2023-10-06T02:31:19.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.579+0000 7f826d79d700 1 Processor -- start 2023-10-06T02:31:19.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.580+0000 7f826d79d700 1 -- start start 2023-10-06T02:31:19.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.580+0000 7f826d79d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82680cef40 0x7f82680b5d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:19.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.580+0000 7f826d79d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8268153c30 0x7f82681585c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:19.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.580+0000 7f826d79d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f826815de50 0x7f8268169600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:19.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.581+0000 7f826d79d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82680b58f0 con 0x7f8268153c30 2023-10-06T02:31:19.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.581+0000 7f826d79d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82680b62c0 con 0x7f82680cef40 2023-10-06T02:31:19.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.581+0000 7f826d79d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f826816a1e0 con 0x7f826815de50 2023-10-06T02:31:19.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.581+0000 7f82677fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8268153c30 0x7f82681585c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:19.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.581+0000 7f82677fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8268153c30 0x7f82681585c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59692/0 (socket says 172.21.15.31:59692) 2023-10-06T02:31:19.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.581+0000 7f82677fe700 1 -- 172.21.15.31:0/1948012957 learned_addr learned my addr 172.21.15.31:0/1948012957 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:19.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.581+0000 7f8267fff700 1 --2- 172.21.15.31:0/1948012957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82680cef40 0x7f82680b5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:19.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.581+0000 7f826cf9c700 1 --2- 172.21.15.31:0/1948012957 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f826815de50 0x7f8268169600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:19.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.581+0000 7f82677fe700 1 -- 172.21.15.31:0/1948012957 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f826815de50 msgr2=0x7f8268169600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:19.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.581+0000 7f82677fe700 1 --2- 172.21.15.31:0/1948012957 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f826815de50 0x7f8268169600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:19.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.581+0000 7f82677fe700 1 -- 172.21.15.31:0/1948012957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82680cef40 msgr2=0x7f82680b5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:19.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.581+0000 7f82677fe700 1 --2- 172.21.15.31:0/1948012957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82680cef40 0x7f82680b5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:19.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.581+0000 7f82677fe700 1 -- 172.21.15.31:0/1948012957 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f825c009420 con 0x7f8268153c30 2023-10-06T02:31:19.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.581+0000 7f8267fff700 1 --2- 172.21.15.31:0/1948012957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82680cef40 0x7f82680b5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:31:19.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.582+0000 7f82677fe700 1 --2- 172.21.15.31:0/1948012957 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8268153c30 0x7f82681585c0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f826806eb00 tx=0x7f825c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:19.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.582+0000 7f824ffff700 1 -- 172.21.15.31:0/1948012957 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825c00bb20 con 0x7f8268153c30 2023-10-06T02:31:19.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.582+0000 7f824ffff700 1 -- 172.21.15.31:0/1948012957 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f825c01a430 con 0x7f8268153c30 2023-10-06T02:31:19.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.582+0000 7f826d79d700 1 -- 172.21.15.31:0/1948012957 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f826816a4c0 con 0x7f8268153c30 2023-10-06T02:31:19.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.582+0000 7f824ffff700 1 -- 172.21.15.31:0/1948012957 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825c01a7f0 con 0x7f8268153c30 2023-10-06T02:31:19.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.582+0000 7f826d79d700 1 -- 172.21.15.31:0/1948012957 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82680b5630 con 0x7f8268153c30 2023-10-06T02:31:19.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.583+0000 7f826d79d700 1 -- 172.21.15.31:0/1948012957 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82680cdab0 con 0x7f8268153c30 2023-10-06T02:31:19.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.584+0000 7f824ffff700 1 -- 172.21.15.31:0/1948012957 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 85403+0+0 (secure 0 0 0) 0x7f825c01abe0 con 0x7f8268153c30 2023-10-06T02:31:19.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.585+0000 7f824ffff700 1 --2- 172.21.15.31:0/1948012957 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f8254067680 0x7f8254069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:19.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.585+0000 7f824ffff700 1 -- 172.21.15.31:0/1948012957 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 5320+0+0 (secure 0 0 0) 0x7f825c087360 con 0x7f8268153c30 2023-10-06T02:31:19.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.585+0000 7f8267fff700 1 --2- 172.21.15.31:0/1948012957 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f8254067680 0x7f8254069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:19.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.586+0000 7f8267fff700 1 --2- 172.21.15.31:0/1948012957 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f8254067680 0x7f8254069b30 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f82580058d0 tx=0x7f8258009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:19.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.589+0000 7f824ffff700 1 -- 172.21.15.31:0/1948012957 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f825c084a20 con 0x7f8268153c30 2023-10-06T02:31:19.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:19.806+0000 7f826d79d700 1 -- 172.21.15.31:0/1948012957 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f826804c3c0 con 0x7f8268153c30 2023-10-06T02:31:21.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.119+0000 7f824ffff700 1 -- 172.21.15.31:0/1948012957 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v152) v1 ==== 97+0+0 (secure 0 0 0) 0x7f825c053810 con 0x7f8268153c30 2023-10-06T02:31:21.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.122+0000 7f826d79d700 1 -- 172.21.15.31:0/1948012957 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f8254067680 msgr2=0x7f8254069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:21.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.122+0000 7f826d79d700 1 --2- 172.21.15.31:0/1948012957 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f8254067680 0x7f8254069b30 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f82580058d0 tx=0x7f8258009040 comp rx=0 tx=0).stop 2023-10-06T02:31:21.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.122+0000 7f826d79d700 1 -- 172.21.15.31:0/1948012957 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8268153c30 msgr2=0x7f82681585c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:21.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.122+0000 7f826d79d700 1 --2- 172.21.15.31:0/1948012957 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8268153c30 0x7f82681585c0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f826806eb00 tx=0x7f825c017000 comp rx=0 tx=0).stop 2023-10-06T02:31:21.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.123+0000 7f826d79d700 1 -- 172.21.15.31:0/1948012957 shutdown_connections 2023-10-06T02:31:21.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.123+0000 7f826d79d700 1 --2- 172.21.15.31:0/1948012957 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f8254067680 0x7f8254069b30 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:21.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.123+0000 7f826d79d700 1 --2- 172.21.15.31:0/1948012957 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f826815de50 0x7f8268169600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:21.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.123+0000 7f826d79d700 1 --2- 172.21.15.31:0/1948012957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82680cef40 0x7f82680b5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:21.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.123+0000 7f826d79d700 1 --2- 172.21.15.31:0/1948012957 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8268153c30 0x7f82681585c0 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:21.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.123+0000 7f826d79d700 1 -- 172.21.15.31:0/1948012957 >> 172.21.15.31:0/1948012957 conn(0x7f82680703a0 msgr2=0x7f82680b0310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:21.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.123+0000 7f826d79d700 1 -- 172.21.15.31:0/1948012957 shutdown_connections 2023-10-06T02:31:21.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.123+0000 7f826d79d700 1 -- 172.21.15.31:0/1948012957 wait complete. 2023-10-06T02:31:21.125 INFO:teuthology.orchestra.run.smithi031.stderr: removed all blocklist entries 2023-10-06T02:31:21.144 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T02:31:21.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.335+0000 7f970739d700 1 Processor -- start 2023-10-06T02:31:21.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.336+0000 7f970739d700 1 -- start start 2023-10-06T02:31:21.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.337+0000 7f970739d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f970013fb20 0x7f970015a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:21.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.337+0000 7f970739d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f970013fef0 0x7f97000c7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:21.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.337+0000 7f970739d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f970006aaa0 0x7f97000c7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:21.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.337+0000 7f970739d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97000c8410 con 0x7f970013fef0 2023-10-06T02:31:21.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.337+0000 7f970739d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f970015ae70 con 0x7f970013fb20 2023-10-06T02:31:21.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.337+0000 7f970739d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f970006af60 con 0x7f970006aaa0 2023-10-06T02:31:21.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.337+0000 7f9705b9a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f970013fef0 0x7f97000c7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:21.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.337+0000 7f9705b9a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f970013fef0 0x7f97000c7990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59694/0 (socket says 172.21.15.31:59694) 2023-10-06T02:31:21.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.337+0000 7f9705b9a700 1 -- 172.21.15.31:0/3534110623 learned_addr learned my addr 172.21.15.31:0/3534110623 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:21.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.337+0000 7f970639b700 1 --2- 172.21.15.31:0/3534110623 >> v2:172.21.15.62:3300/0 conn(0x7f970013fb20 0x7f970015a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:21.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.337+0000 7f9706b9c700 1 --2- 172.21.15.31:0/3534110623 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f970006aaa0 0x7f97000c7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:21.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.338+0000 7f9705b9a700 1 -- 172.21.15.31:0/3534110623 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f970006aaa0 msgr2=0x7f97000c7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:21.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.338+0000 7f9705b9a700 1 --2- 172.21.15.31:0/3534110623 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f970006aaa0 0x7f97000c7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:21.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.338+0000 7f9705b9a700 1 -- 172.21.15.31:0/3534110623 >> v2:172.21.15.62:3300/0 conn(0x7f970013fb20 msgr2=0x7f970015a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:21.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.338+0000 7f9705b9a700 1 --2- 172.21.15.31:0/3534110623 >> v2:172.21.15.62:3300/0 conn(0x7f970013fb20 0x7f970015a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:21.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.338+0000 7f9705b9a700 1 -- 172.21.15.31:0/3534110623 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f970006b240 con 0x7f970013fef0 2023-10-06T02:31:21.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.338+0000 7f9705b9a700 1 --2- 172.21.15.31:0/3534110623 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f970013fef0 0x7f97000c7990 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f97000695b0 tx=0x7f96f0009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9609fb45fa79d9ae server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:21.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.338+0000 7f9705399700 1 -- 172.21.15.31:0/3534110623 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f0005c10 con 0x7f970013fef0 2023-10-06T02:31:21.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.338+0000 7f9705399700 1 -- 172.21.15.31:0/3534110623 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96f0005d70 con 0x7f970013fef0 2023-10-06T02:31:21.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.338+0000 7f9705399700 1 -- 172.21.15.31:0/3534110623 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f000b880 con 0x7f970013fef0 2023-10-06T02:31:21.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.339+0000 7f970739d700 1 -- 172.21.15.31:0/3534110623 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f970013fef0 msgr2=0x7f97000c7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:21.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.339+0000 7f970739d700 1 --2- 172.21.15.31:0/3534110623 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f970013fef0 0x7f97000c7990 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f97000695b0 tx=0x7f96f0009220 comp rx=0 tx=0).stop 2023-10-06T02:31:21.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.339+0000 7f970739d700 1 -- 172.21.15.31:0/3534110623 shutdown_connections 2023-10-06T02:31:21.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.339+0000 7f970739d700 1 --2- 172.21.15.31:0/3534110623 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f970006aaa0 0x7f97000c7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:21.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.339+0000 7f970739d700 1 --2- 172.21.15.31:0/3534110623 >> v2:172.21.15.62:3300/0 conn(0x7f970013fb20 0x7f970015a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:21.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.339+0000 7f970739d700 1 --2- 172.21.15.31:0/3534110623 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f970013fef0 0x7f97000c7990 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:21.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.339+0000 7f970739d700 1 -- 172.21.15.31:0/3534110623 >> 172.21.15.31:0/3534110623 conn(0x7f970006d170 msgr2=0x7f97000cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:21.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.339+0000 7f970739d700 1 -- 172.21.15.31:0/3534110623 shutdown_connections 2023-10-06T02:31:21.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.339+0000 7f970739d700 1 -- 172.21.15.31:0/3534110623 wait complete. 2023-10-06T02:31:21.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.340+0000 7f970739d700 1 Processor -- start 2023-10-06T02:31:21.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.340+0000 7f970739d700 1 -- start start 2023-10-06T02:31:21.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.341+0000 7f970739d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f970006aaa0 0x7f97000ae190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:21.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.341+0000 7f970739d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f970013fb20 0x7f9700138ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:21.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.341+0000 7f970739d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f970013fef0 0x7f970013e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:21.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.341+0000 7f970739d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97000ae6d0 con 0x7f970013fb20 2023-10-06T02:31:21.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.341+0000 7f970739d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97000add00 con 0x7f970013fef0 2023-10-06T02:31:21.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.341+0000 7f970739d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9700152770 con 0x7f970006aaa0 2023-10-06T02:31:21.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.342+0000 7f9705b9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f970013fb20 0x7f9700138ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:21.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.342+0000 7f9705b9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f970013fb20 0x7f9700138ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59702/0 (socket says 172.21.15.31:59702) 2023-10-06T02:31:21.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.342+0000 7f9705b9a700 1 -- 172.21.15.31:0/1450769841 learned_addr learned my addr 172.21.15.31:0/1450769841 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:21.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.342+0000 7f970639b700 1 --2- 172.21.15.31:0/1450769841 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f970006aaa0 0x7f97000ae190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:21.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.342+0000 7f9706b9c700 1 --2- 172.21.15.31:0/1450769841 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f970013fef0 0x7f970013e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:21.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.342+0000 7f9705b9a700 1 -- 172.21.15.31:0/1450769841 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f970006aaa0 msgr2=0x7f97000ae190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:21.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.342+0000 7f9705b9a700 1 --2- 172.21.15.31:0/1450769841 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f970006aaa0 0x7f97000ae190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:21.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.342+0000 7f9705b9a700 1 -- 172.21.15.31:0/1450769841 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f970013fef0 msgr2=0x7f970013e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:21.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.342+0000 7f9705b9a700 1 --2- 172.21.15.31:0/1450769841 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f970013fef0 0x7f970013e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:21.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.342+0000 7f9705b9a700 1 -- 172.21.15.31:0/1450769841 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96f0009480 con 0x7f970013fb20 2023-10-06T02:31:21.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.342+0000 7f970639b700 1 --2- 172.21.15.31:0/1450769841 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f970006aaa0 0x7f97000ae190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:31:21.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.342+0000 7f9706b9c700 1 --2- 172.21.15.31:0/1450769841 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f970013fef0 0x7f970013e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:31:21.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.342+0000 7f9705b9a700 1 --2- 172.21.15.31:0/1450769841 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f970013fb20 0x7f9700138ab0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f96f0007fd0 tx=0x7f96f0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:21.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.343+0000 7f96f67fc700 1 -- 172.21.15.31:0/1450769841 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f00118e0 con 0x7f970013fb20 2023-10-06T02:31:21.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.343+0000 7f96f67fc700 1 -- 172.21.15.31:0/1450769841 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96f0020070 con 0x7f970013fb20 2023-10-06T02:31:21.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.343+0000 7f96f67fc700 1 -- 172.21.15.31:0/1450769841 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f0011cb0 con 0x7f970013fb20 2023-10-06T02:31:21.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.343+0000 7f970739d700 1 -- 172.21.15.31:0/1450769841 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97000ad6b0 con 0x7f970013fb20 2023-10-06T02:31:21.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.343+0000 7f970739d700 1 -- 172.21.15.31:0/1450769841 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9700138ff0 con 0x7f970013fb20 2023-10-06T02:31:21.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.344+0000 7f970739d700 1 -- 172.21.15.31:0/1450769841 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97000adeb0 con 0x7f970013fb20 2023-10-06T02:31:21.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.345+0000 7f96f67fc700 1 -- 172.21.15.31:0/1450769841 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 85403+0+0 (secure 0 0 0) 0x7f96f0022030 con 0x7f970013fb20 2023-10-06T02:31:21.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.346+0000 7f96f67fc700 1 --2- 172.21.15.31:0/1450769841 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f96e4067680 0x7f96e4069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:21.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.346+0000 7f96f67fc700 1 -- 172.21.15.31:0/1450769841 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5234+0+0 (secure 0 0 0) 0x7f96f00871f0 con 0x7f970013fb20 2023-10-06T02:31:21.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.346+0000 7f970639b700 1 --2- 172.21.15.31:0/1450769841 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f96e4067680 0x7f96e4069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:21.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.347+0000 7f970639b700 1 --2- 172.21.15.31:0/1450769841 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f96e4067680 0x7f96e4069b30 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f96fc006fd0 tx=0x7f96fc008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:21.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.350+0000 7f96f67fc700 1 -- 172.21.15.31:0/1450769841 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f96f00b85a0 con 0x7f970013fb20 2023-10-06T02:31:21.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.628+0000 7f970739d700 1 -- 172.21.15.31:0/1450769841 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f970004c3c0 con 0x7f970013fb20 2023-10-06T02:31:21.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.629+0000 7f96f67fc700 1 -- 172.21.15.31:0/1450769841 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v22) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f97000adeb0 con 0x7f970013fb20 2023-10-06T02:31:21.629 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:31:21.629 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T02:31:21.630 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_dump": [ 2023-10-06T02:31:21.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:31:21.630 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.a", 2023-10-06T02:31:21.630 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBLbx9lbjrOGxAAVoSOoDaZb1temL41F4Nong==", 2023-10-06T02:31:21.630 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:31:21.630 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:31:21.630 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T02:31:21.630 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T02:31:21.630 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:31:21.631 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:31:21.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:31:21.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:31:21.631 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.b", 2023-10-06T02:31:21.631 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBLbx9lFgsTMRAATTklzroiyQbTLiZc9j1BHQ==", 2023-10-06T02:31:21.631 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:31:21.631 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:31:21.631 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T02:31:21.631 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T02:31:21.632 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:31:21.632 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:31:21.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:31:21.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:31:21.632 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.c", 2023-10-06T02:31:21.632 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBLbx9lezVtJhAA8wK7+dJ1OmIc+iXJkexK0w==", 2023-10-06T02:31:21.632 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:31:21.632 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:31:21.633 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T02:31:21.633 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T02:31:21.633 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:31:21.633 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:31:21.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:31:21.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:31:21.633 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.d", 2023-10-06T02:31:21.633 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBMbx9l74gWABAApBu2nVv12rGA+OBfBnEc3A==", 2023-10-06T02:31:21.634 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:31:21.634 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:31:21.634 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T02:31:21.634 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T02:31:21.634 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:31:21.634 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:31:21.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:31:21.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:31:21.635 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.0", 2023-10-06T02:31:21.635 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBPbx9lgj6+MBAAXSUD0GCG+rO/9DQBZYt/8w==", 2023-10-06T02:31:21.635 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:31:21.635 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:31:21.635 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:31:21.635 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:31:21.636 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:31:21.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:31:21.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:31:21.636 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.1", 2023-10-06T02:31:21.636 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBSbx9lXhMTNBAAeEdPnMP71ZGb6k2Ddg5raQ==", 2023-10-06T02:31:21.636 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:31:21.636 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:31:21.636 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:31:21.636 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:31:21.637 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:31:21.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:31:21.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:31:21.637 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.2", 2023-10-06T02:31:21.637 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBVbx9l4Ly0NhAATYXYDAjYuP53zjqK0yMboA==", 2023-10-06T02:31:21.637 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:31:21.637 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:31:21.638 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:31:21.638 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:31:21.638 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:31:21.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:31:21.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:31:21.638 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.3", 2023-10-06T02:31:21.638 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBZbx9lzsi4ARAAbGnNV0QyL/7itzy827+chg==", 2023-10-06T02:31:21.639 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:31:21.639 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:31:21.639 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:31:21.639 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:31:21.639 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:31:21.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:31:21.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:31:21.639 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.4", 2023-10-06T02:31:21.639 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBfbx9lW4OSEhAAj7HnVh/yq12zzCaXd6BHCA==", 2023-10-06T02:31:21.640 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:31:21.640 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:31:21.640 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:31:21.640 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:31:21.640 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:31:21.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:31:21.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:31:21.641 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.5", 2023-10-06T02:31:21.641 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBibx9lBcaIFRAAV8wq8QjsCOkI9Pzf/E+9bA==", 2023-10-06T02:31:21.641 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:31:21.641 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:31:21.641 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:31:21.641 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:31:21.641 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:31:21.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:31:21.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:31:21.642 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.6", 2023-10-06T02:31:21.642 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBlbx9lA7S+FxAA38mlSAurT+DpwMzH53zFcw==", 2023-10-06T02:31:21.642 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:31:21.642 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:31:21.642 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:31:21.642 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:31:21.642 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:31:21.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:31:21.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:31:21.643 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.7", 2023-10-06T02:31:21.643 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBobx9l5RkdHhAAxn0Z8IAE/o9oe2LRp3O9gg==", 2023-10-06T02:31:21.643 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:31:21.643 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:31:21.643 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:31:21.643 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:31:21.644 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:31:21.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:31:21.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:31:21.644 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.0", 2023-10-06T02:31:21.644 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBMbx9lh44fCRAA7sMISvk1t7dEJg4ZcBiMcA==", 2023-10-06T02:31:21.644 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:31:21.644 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:31:21.644 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow r", 2023-10-06T02:31:21.644 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T02:31:21.645 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:31:21.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:31:21.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:31:21.645 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.1", 2023-10-06T02:31:21.645 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBMbx9lP0zqFhAAlov2y7ipSvS3AkOdQkhg6g==", 2023-10-06T02:31:21.645 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:31:21.645 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:31:21.645 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-10-06T02:31:21.646 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-10-06T02:31:21.646 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-10-06T02:31:21.646 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:31:21.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:31:21.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:31:21.646 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.2", 2023-10-06T02:31:21.646 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBMbx9lpxYAEBAACtiI1XgQXM8yGGcpFyNK1w==", 2023-10-06T02:31:21.646 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:31:21.647 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:31:21.647 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-10-06T02:31:21.647 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-10-06T02:31:21.647 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-10-06T02:31:21.647 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:31:21.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:31:21.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:31:21.647 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.3", 2023-10-06T02:31:21.648 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBMbx9l81/PHRAAYFOoaezu0pwE0NdtnlYfVA==", 2023-10-06T02:31:21.648 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:31:21.648 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:31:21.648 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-10-06T02:31:21.648 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-10-06T02:31:21.648 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-10-06T02:31:21.648 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:31:21.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:31:21.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:31:21.649 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.admin", 2023-10-06T02:31:21.649 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBKbx9liFl1JBAAFyyHYNLVx1/Ex5a2MKm8rA==", 2023-10-06T02:31:21.649 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:31:21.649 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-10-06T02:31:21.649 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T02:31:21.649 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T02:31:21.649 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:31:21.650 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:31:21.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:31:21.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:31:21.650 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mgr.x", 2023-10-06T02:31:21.650 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBLbx9lTBbIFBAAX2wsioKL4Nx4qz/qwsCsNA==", 2023-10-06T02:31:21.650 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:31:21.650 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-10-06T02:31:21.650 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile mgr", 2023-10-06T02:31:21.651 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:31:21.651 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:31:21.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:31:21.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:31:21.651 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mgr.y", 2023-10-06T02:31:21.651 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBLbx9l4+oLDRAA5YVo3q6FI9e9W/rJYROIsg==", 2023-10-06T02:31:21.651 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:31:21.651 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-10-06T02:31:21.652 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile mgr", 2023-10-06T02:31:21.652 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:31:21.652 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:31:21.652 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:31:21.652 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-10-06T02:31:21.652 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T02:31:21.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.631+0000 7f970739d700 1 -- 172.21.15.31:0/1450769841 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f96e4067680 msgr2=0x7f96e4069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:21.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.631+0000 7f970739d700 1 --2- 172.21.15.31:0/1450769841 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f96e4067680 0x7f96e4069b30 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f96fc006fd0 tx=0x7f96fc008040 comp rx=0 tx=0).stop 2023-10-06T02:31:21.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.631+0000 7f970739d700 1 -- 172.21.15.31:0/1450769841 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f970013fb20 msgr2=0x7f9700138ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:21.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.631+0000 7f970739d700 1 --2- 172.21.15.31:0/1450769841 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f970013fb20 0x7f9700138ab0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f96f0007fd0 tx=0x7f96f0016000 comp rx=0 tx=0).stop 2023-10-06T02:31:21.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.631+0000 7f970739d700 1 -- 172.21.15.31:0/1450769841 shutdown_connections 2023-10-06T02:31:21.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.631+0000 7f970739d700 1 --2- 172.21.15.31:0/1450769841 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f970013fef0 0x7f970013e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:21.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.631+0000 7f970739d700 1 --2- 172.21.15.31:0/1450769841 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f96e4067680 0x7f96e4069b30 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:21.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.631+0000 7f970739d700 1 --2- 172.21.15.31:0/1450769841 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f970006aaa0 0x7f97000ae190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:21.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.631+0000 7f970739d700 1 --2- 172.21.15.31:0/1450769841 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f970013fb20 0x7f9700138ab0 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:21.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.631+0000 7f970739d700 1 -- 172.21.15.31:0/1450769841 >> 172.21.15.31:0/1450769841 conn(0x7f970006d170 msgr2=0x7f97000b0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:21.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.631+0000 7f970739d700 1 -- 172.21.15.31:0/1450769841 shutdown_connections 2023-10-06T02:31:21.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.631+0000 7f970739d700 1 -- 172.21.15.31:0/1450769841 wait complete. 2023-10-06T02:31:21.656 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:31:21.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.842+0000 7f9dbe59d700 1 Processor -- start 2023-10-06T02:31:21.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.842+0000 7f9dbe59d700 1 -- start start 2023-10-06T02:31:21.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.844+0000 7f9dbe59d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9db813fb20 0x7f9db815a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:21.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.844+0000 7f9dbe59d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9db806aaa0 0x7f9db80c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:21.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.844+0000 7f9dbe59d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9db815ae70 con 0x7f9db813fef0 2023-10-06T02:31:21.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.844+0000 7f9dbe59d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9db806af60 con 0x7f9db813fb20 2023-10-06T02:31:21.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.844+0000 7f9dbe59d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9db806b240 con 0x7f9db806aaa0 2023-10-06T02:31:21.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.844+0000 7f9dbd59b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9db813fb20 0x7f9db815a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:21.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.844+0000 7f9dbd59b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9db813fb20 0x7f9db815a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59712/0 (socket says 172.21.15.31:59712) 2023-10-06T02:31:21.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.844+0000 7f9dbd59b700 1 -- 172.21.15.31:0/3560060506 learned_addr learned my addr 172.21.15.31:0/3560060506 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:21.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.844+0000 7f9dbdd9c700 1 --2- 172.21.15.31:0/3560060506 >> v2:172.21.15.62:3300/0 conn(0x7f9db806aaa0 0x7f9db80c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:21.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.844+0000 7f9dbd59b700 1 -- 172.21.15.31:0/3560060506 >> v2:172.21.15.62:3300/0 conn(0x7f9db806aaa0 msgr2=0x7f9db80c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:21.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.844+0000 7f9dbd59b700 1 --2- 172.21.15.31:0/3560060506 >> v2:172.21.15.62:3300/0 conn(0x7f9db806aaa0 0x7f9db80c9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:21.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.844+0000 7f9dbd59b700 1 -- 172.21.15.31:0/3560060506 >> v1:172.21.15.62:6789/0 conn(0x7f9db813fef0 legacy=0x7f9db80c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:21.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.844+0000 7f9dbd59b700 1 -- 172.21.15.31:0/3560060506 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9db806b520 con 0x7f9db813fb20 2023-10-06T02:31:21.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.844+0000 7f9dbdd9c700 1 --2- 172.21.15.31:0/3560060506 >> v2:172.21.15.62:3300/0 conn(0x7f9db806aaa0 0x7f9db80c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:31:21.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.845+0000 7f9dbd59b700 1 --2- 172.21.15.31:0/3560060506 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9db813fb20 0x7f9db815a930 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f9db8069d40 tx=0x7f9db4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69e7b132aca7c4f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:21.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.845+0000 7f9daffff700 1 -- 172.21.15.31:0/3560060506 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db400b820 con 0x7f9db813fb20 2023-10-06T02:31:21.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.845+0000 7f9daffff700 1 -- 172.21.15.31:0/3560060506 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9db400b980 con 0x7f9db813fb20 2023-10-06T02:31:21.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.845+0000 7f9daffff700 1 -- 172.21.15.31:0/3560060506 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db400bd20 con 0x7f9db813fb20 2023-10-06T02:31:21.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.845+0000 7f9dbe59d700 1 -- 172.21.15.31:0/3560060506 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9db813fb20 msgr2=0x7f9db815a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:21.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.845+0000 7f9dbe59d700 1 --2- 172.21.15.31:0/3560060506 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9db813fb20 0x7f9db815a930 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f9db8069d40 tx=0x7f9db4009250 comp rx=0 tx=0).stop 2023-10-06T02:31:21.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.846+0000 7f9dbe59d700 1 -- 172.21.15.31:0/3560060506 shutdown_connections 2023-10-06T02:31:21.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.846+0000 7f9dbe59d700 1 --2- 172.21.15.31:0/3560060506 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9db813fb20 0x7f9db815a930 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:21.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.846+0000 7f9dbe59d700 1 --2- 172.21.15.31:0/3560060506 >> v2:172.21.15.62:3300/0 conn(0x7f9db806aaa0 0x7f9db80c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:21.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.846+0000 7f9dbe59d700 1 -- 172.21.15.31:0/3560060506 >> 172.21.15.31:0/3560060506 conn(0x7f9db806d170 msgr2=0x7f9db80cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:21.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.846+0000 7f9dbe59d700 1 -- 172.21.15.31:0/3560060506 shutdown_connections 2023-10-06T02:31:21.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.846+0000 7f9dbe59d700 1 -- 172.21.15.31:0/3560060506 wait complete. 2023-10-06T02:31:21.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.847+0000 7f9dbe59d700 1 Processor -- start 2023-10-06T02:31:21.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.847+0000 7f9dbe59d700 1 -- start start 2023-10-06T02:31:21.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.848+0000 7f9dbe59d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9db806aaa0 0x7f9db816cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:21.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.848+0000 7f9dbe59d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9db813fb20 0x7f9db80cc670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:21.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.848+0000 7f9dbe59d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9db813fef0 0x7f9db81339b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:21.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.848+0000 7f9dbe59d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9db8157df0 con 0x7f9db806aaa0 2023-10-06T02:31:21.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.848+0000 7f9dbe59d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9db80ccbb0 con 0x7f9db813fb20 2023-10-06T02:31:21.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.848+0000 7f9dbe59d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9db816d620 con 0x7f9db813fef0 2023-10-06T02:31:21.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.848+0000 7f9dbd59b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9db806aaa0 0x7f9db816cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:21.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.848+0000 7f9dbd59b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9db806aaa0 0x7f9db816cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59716/0 (socket says 172.21.15.31:59716) 2023-10-06T02:31:21.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.849+0000 7f9dbd59b700 1 -- 172.21.15.31:0/1342392991 learned_addr learned my addr 172.21.15.31:0/1342392991 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:21.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.849+0000 7f9dbdd9c700 1 --2- 172.21.15.31:0/1342392991 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9db813fef0 0x7f9db81339b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:21.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.849+0000 7f9dbcd9a700 1 --2- 172.21.15.31:0/1342392991 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9db813fb20 0x7f9db80cc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:21.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.849+0000 7f9dbd59b700 1 -- 172.21.15.31:0/1342392991 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9db813fef0 msgr2=0x7f9db81339b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:21.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.849+0000 7f9dbd59b700 1 --2- 172.21.15.31:0/1342392991 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9db813fef0 0x7f9db81339b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:21.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.849+0000 7f9dbd59b700 1 -- 172.21.15.31:0/1342392991 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9db813fb20 msgr2=0x7f9db80cc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:21.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.849+0000 7f9dbd59b700 1 --2- 172.21.15.31:0/1342392991 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9db813fb20 0x7f9db80cc670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:21.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.849+0000 7f9dbd59b700 1 -- 172.21.15.31:0/1342392991 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9db40094f0 con 0x7f9db806aaa0 2023-10-06T02:31:21.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.849+0000 7f9dbcd9a700 1 --2- 172.21.15.31:0/1342392991 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9db813fb20 0x7f9db80cc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:31:21.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.849+0000 7f9dbdd9c700 1 --2- 172.21.15.31:0/1342392991 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9db813fef0 0x7f9db81339b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:31:21.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.849+0000 7f9dbd59b700 1 --2- 172.21.15.31:0/1342392991 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9db806aaa0 0x7f9db816cf90 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f9db4009290 tx=0x7f9db4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:21.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.849+0000 7f9dad7fa700 1 -- 172.21.15.31:0/1342392991 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db4011e20 con 0x7f9db806aaa0 2023-10-06T02:31:21.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.849+0000 7f9dad7fa700 1 -- 172.21.15.31:0/1342392991 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9db4020070 con 0x7f9db806aaa0 2023-10-06T02:31:21.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.850+0000 7f9dbe59d700 1 -- 172.21.15.31:0/1342392991 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9db816d900 con 0x7f9db806aaa0 2023-10-06T02:31:21.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.850+0000 7f9dad7fa700 1 -- 172.21.15.31:0/1342392991 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db401b6a0 con 0x7f9db806aaa0 2023-10-06T02:31:21.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.850+0000 7f9dbe59d700 1 -- 172.21.15.31:0/1342392991 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9db8133ef0 con 0x7f9db806aaa0 2023-10-06T02:31:21.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.851+0000 7f9dbe59d700 1 -- 172.21.15.31:0/1342392991 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9db8157bc0 con 0x7f9db806aaa0 2023-10-06T02:31:21.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.851+0000 7f9dad7fa700 1 -- 172.21.15.31:0/1342392991 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 85403+0+0 (secure 0 0 0) 0x7f9db4022030 con 0x7f9db806aaa0 2023-10-06T02:31:21.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.852+0000 7f9dad7fa700 1 --2- 172.21.15.31:0/1342392991 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9d9c067680 0x7f9d9c069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:21.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.852+0000 7f9dad7fa700 1 -- 172.21.15.31:0/1342392991 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5234+0+0 (secure 0 0 0) 0x7f9db4086fd0 con 0x7f9db806aaa0 2023-10-06T02:31:21.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.853+0000 7f9dbcd9a700 1 --2- 172.21.15.31:0/1342392991 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9d9c067680 0x7f9d9c069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:21.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.854+0000 7f9dbcd9a700 1 --2- 172.21.15.31:0/1342392991 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9d9c067680 0x7f9d9c069b30 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f9db80cd200 tx=0x7f9da8008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:21.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:21.856+0000 7f9dad7fa700 1 -- 172.21.15.31:0/1342392991 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9db8157bc0 con 0x7f9db806aaa0 2023-10-06T02:31:22.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.061+0000 7f9dbe59d700 1 -- 172.21.15.31:0/1342392991 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9db815ae70 con 0x7f9db806aaa0 2023-10-06T02:31:22.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.062+0000 7f9dad7fa700 1 -- 172.21.15.31:0/1342392991 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v152) v1 ==== 74+0+13656 (secure 0 0 0) 0x7f9db815ae70 con 0x7f9db806aaa0 2023-10-06T02:31:22.063 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:31:22.063 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":152,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:31:21.114152+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":20,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:31:22.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.064+0000 7f9dbe59d700 1 -- 172.21.15.31:0/1342392991 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9d9c067680 msgr2=0x7f9d9c069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:22.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.064+0000 7f9dbe59d700 1 --2- 172.21.15.31:0/1342392991 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9d9c067680 0x7f9d9c069b30 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f9db80cd200 tx=0x7f9da8008040 comp rx=0 tx=0).stop 2023-10-06T02:31:22.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.064+0000 7f9dbe59d700 1 -- 172.21.15.31:0/1342392991 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9db806aaa0 msgr2=0x7f9db816cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:22.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.064+0000 7f9dbe59d700 1 --2- 172.21.15.31:0/1342392991 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9db806aaa0 0x7f9db816cf90 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f9db4009290 tx=0x7f9db4016000 comp rx=0 tx=0).stop 2023-10-06T02:31:22.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.064+0000 7f9dbe59d700 1 -- 172.21.15.31:0/1342392991 shutdown_connections 2023-10-06T02:31:22.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.064+0000 7f9dbe59d700 1 --2- 172.21.15.31:0/1342392991 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9d9c067680 0x7f9d9c069b30 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:22.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.064+0000 7f9dbe59d700 1 --2- 172.21.15.31:0/1342392991 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9db813fef0 0x7f9db81339b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:22.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.064+0000 7f9dbe59d700 1 --2- 172.21.15.31:0/1342392991 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9db806aaa0 0x7f9db816cf90 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:22.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.064+0000 7f9dbe59d700 1 --2- 172.21.15.31:0/1342392991 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9db813fb20 0x7f9db80cc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:22.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.064+0000 7f9dbe59d700 1 -- 172.21.15.31:0/1342392991 >> 172.21.15.31:0/1342392991 conn(0x7f9db806d170 msgr2=0x7f9db80c25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:22.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.065+0000 7f9dbe59d700 1 -- 172.21.15.31:0/1342392991 shutdown_connections 2023-10-06T02:31:22.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.065+0000 7f9dbe59d700 1 -- 172.21.15.31:0/1342392991 wait complete. 2023-10-06T02:31:22.083 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T02:22:53.183425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T02:23:00.046967+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-10-06T02:31:22.083 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T02:31:22.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.272+0000 7fa4d2d9d700 1 Processor -- start 2023-10-06T02:31:22.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.272+0000 7fa4d2d9d700 1 -- start start 2023-10-06T02:31:22.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.274+0000 7fa4d2d9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa4cc140a70 0x7fa4cc069b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:22.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.274+0000 7fa4d2d9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4cc15b640 con 0x7fa4cc0c7c60 2023-10-06T02:31:22.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.274+0000 7fa4d2d9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4cc15b920 con 0x7fa4cc06de50 2023-10-06T02:31:22.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.274+0000 7fa4d2d9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4cc0c6800 con 0x7fa4cc140a70 2023-10-06T02:31:22.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.274+0000 7fa4d259c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fa4cc0c7c60 0x7fa4cc158190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:39434/0 (socket says 172.21.15.31:39434) 2023-10-06T02:31:22.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.274+0000 7fa4d259c700 1 -- 172.21.15.31:0/472083102 learned_addr learned my addr 172.21.15.31:0/472083102 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:31:22.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.274+0000 7fa4d1d9b700 1 --2- 172.21.15.31:0/472083102 >> v2:172.21.15.62:3300/0 conn(0x7fa4cc140a70 0x7fa4cc069b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:22.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.274+0000 7fa4d0d99700 1 -- 172.21.15.31:0/472083102 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1285798090 0 0) 0x7fa4cc15b640 con 0x7fa4cc0c7c60 2023-10-06T02:31:22.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.275+0000 7fa4d0d99700 1 -- 172.21.15.31:0/472083102 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa4b8003570 con 0x7fa4cc0c7c60 2023-10-06T02:31:22.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.275+0000 7fa4d0d99700 1 -- 172.21.15.31:0/472083102 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 68486214 0 0) 0x7fa4cc15b920 con 0x7fa4cc06de50 2023-10-06T02:31:22.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.275+0000 7fa4d0d99700 1 -- 172.21.15.31:0/472083102 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa4cc15b640 con 0x7fa4cc06de50 2023-10-06T02:31:22.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.275+0000 7fa4d0d99700 1 -- 172.21.15.31:0/472083102 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3156851273 0 0) 0x7fa4b8003570 con 0x7fa4cc0c7c60 2023-10-06T02:31:22.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.275+0000 7fa4d0d99700 1 -- 172.21.15.31:0/472083102 >> v2:172.21.15.62:3300/0 conn(0x7fa4cc140a70 msgr2=0x7fa4cc069b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:22.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.275+0000 7fa4d0d99700 1 --2- 172.21.15.31:0/472083102 >> v2:172.21.15.62:3300/0 conn(0x7fa4cc140a70 0x7fa4cc069b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:22.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.275+0000 7fa4d0d99700 1 -- 172.21.15.31:0/472083102 >> v1:172.21.15.62:6789/0 conn(0x7fa4cc06de50 legacy=0x7fa4cc157a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:22.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.275+0000 7fa4d0d99700 1 -- 172.21.15.31:0/472083102 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4cc15bc00 con 0x7fa4cc0c7c60 2023-10-06T02:31:22.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.275+0000 7fa4d0d99700 1 -- 172.21.15.31:0/472083102 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7fa4c40017f0 con 0x7fa4cc0c7c60 2023-10-06T02:31:22.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.275+0000 7fa4d0d99700 1 -- 172.21.15.31:0/472083102 >> v1:172.21.15.31:6789/0 conn(0x7fa4cc0c7c60 legacy=0x7fa4cc158190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:22.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.275+0000 7fa4d0d99700 1 --2- 172.21.15.31:0/472083102 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa4b8003bf0 0x7fa4b8003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:22.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.275+0000 7fa4d0d99700 1 --2- 172.21.15.31:0/472083102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4b8004810 0x7fa4b8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:22.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.275+0000 7fa4d0d99700 1 --2- 172.21.15.31:0/472083102 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4b80051c0 0x7fa4b800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:22.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.275+0000 7fa4d0d99700 1 -- 172.21.15.31:0/472083102 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4cc0c6800 con 0x7fa4b80051c0 2023-10-06T02:31:22.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.275+0000 7fa4d0d99700 1 -- 172.21.15.31:0/472083102 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b800c0b0 con 0x7fa4b8004810 2023-10-06T02:31:22.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.275+0000 7fa4d0d99700 1 -- 172.21.15.31:0/472083102 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b800c270 con 0x7fa4b8003bf0 2023-10-06T02:31:22.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.275+0000 7fa4d259c700 1 --2- 172.21.15.31:0/472083102 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4b80051c0 0x7fa4b800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:22.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.275+0000 7fa4d1d9b700 1 --2- 172.21.15.31:0/472083102 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa4b8003bf0 0x7fa4b8003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:22.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.275+0000 7fa4d159a700 1 --2- 172.21.15.31:0/472083102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4b8004810 0x7fa4b8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:22.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.276+0000 7fa4d259c700 1 -- 172.21.15.31:0/472083102 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa4b8003bf0 msgr2=0x7fa4b8003fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:22.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.276+0000 7fa4d259c700 1 --2- 172.21.15.31:0/472083102 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa4b8003bf0 0x7fa4b8003fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:22.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.276+0000 7fa4d259c700 1 -- 172.21.15.31:0/472083102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4b8004810 msgr2=0x7fa4b8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:22.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.276+0000 7fa4d259c700 1 --2- 172.21.15.31:0/472083102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4b8004810 0x7fa4b8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:22.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.276+0000 7fa4d259c700 1 -- 172.21.15.31:0/472083102 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4b8003570 con 0x7fa4b80051c0 2023-10-06T02:31:22.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.276+0000 7fa4d259c700 1 -- 172.21.15.31:0/472083102 reap_dead start 2023-10-06T02:31:22.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.276+0000 7fa4d259c700 1 --2- 172.21.15.31:0/472083102 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4b80051c0 0x7fa4b800ba60 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7fa4c400a800 tx=0x7fa4c40020a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21217728b3a6d07d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:22.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.276+0000 7fa4d0d99700 1 -- 172.21.15.31:0/472083102 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c40071a0 con 0x7fa4b80051c0 2023-10-06T02:31:22.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.276+0000 7fa4d0d99700 1 -- 172.21.15.31:0/472083102 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4c4007320 con 0x7fa4b80051c0 2023-10-06T02:31:22.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.276+0000 7fa4d0d99700 1 -- 172.21.15.31:0/472083102 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c40076e0 con 0x7fa4b80051c0 2023-10-06T02:31:22.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.277+0000 7fa4d2d9d700 1 -- 172.21.15.31:0/472083102 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4b80051c0 msgr2=0x7fa4b800ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:22.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.277+0000 7fa4d2d9d700 1 --2- 172.21.15.31:0/472083102 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4b80051c0 0x7fa4b800ba60 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7fa4c400a800 tx=0x7fa4c40020a0 comp rx=0 tx=0).stop 2023-10-06T02:31:22.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.277+0000 7fa4d2d9d700 1 -- 172.21.15.31:0/472083102 shutdown_connections 2023-10-06T02:31:22.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.277+0000 7fa4d2d9d700 1 --2- 172.21.15.31:0/472083102 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4b80051c0 0x7fa4b800ba60 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:22.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.277+0000 7fa4d2d9d700 1 -- 172.21.15.31:0/472083102 >> 172.21.15.31:0/472083102 conn(0x7fa4cc06d110 msgr2=0x7fa4cc0cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:22.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.277+0000 7fa4d2d9d700 1 -- 172.21.15.31:0/472083102 shutdown_connections 2023-10-06T02:31:22.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.277+0000 7fa4d2d9d700 1 -- 172.21.15.31:0/472083102 wait complete. 2023-10-06T02:31:22.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.278+0000 7fa4d2d9d700 1 Processor -- start 2023-10-06T02:31:22.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.278+0000 7fa4d2d9d700 1 -- start start 2023-10-06T02:31:22.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.279+0000 7fa4d2d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa4b80051c0 0x7fa4cc0c3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:22.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.279+0000 7fa4d2d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4cc13db70 0x7fa4cc0b1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:22.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.279+0000 7fa4d2d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4cc0b1560 0x7fa4cc1698a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:22.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.279+0000 7fa4d2d9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4cc0b1a00 con 0x7fa4cc0b1560 2023-10-06T02:31:22.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.279+0000 7fa4d2d9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4cc0c4450 con 0x7fa4cc13db70 2023-10-06T02:31:22.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.279+0000 7fa4d2d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4cc0c31e0 con 0x7fa4b80051c0 2023-10-06T02:31:22.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.280+0000 7fa4d259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4cc0b1560 0x7fa4cc1698a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:22.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.280+0000 7fa4d259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4cc0b1560 0x7fa4cc1698a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59740/0 (socket says 172.21.15.31:59740) 2023-10-06T02:31:22.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.280+0000 7fa4d259c700 1 -- 172.21.15.31:0/3364401541 learned_addr learned my addr 172.21.15.31:0/3364401541 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:22.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.280+0000 7fa4d159a700 1 --2- 172.21.15.31:0/3364401541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4cc13db70 0x7fa4cc0b1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:22.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.280+0000 7fa4d1d9b700 1 --2- 172.21.15.31:0/3364401541 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa4b80051c0 0x7fa4cc0c3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:22.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.280+0000 7fa4d259c700 1 -- 172.21.15.31:0/3364401541 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa4b80051c0 msgr2=0x7fa4cc0c3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:22.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.280+0000 7fa4d259c700 1 --2- 172.21.15.31:0/3364401541 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa4b80051c0 0x7fa4cc0c3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:22.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.280+0000 7fa4d259c700 1 -- 172.21.15.31:0/3364401541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4cc13db70 msgr2=0x7fa4cc0b1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:22.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.280+0000 7fa4d259c700 1 --2- 172.21.15.31:0/3364401541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4cc13db70 0x7fa4cc0b1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:22.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.280+0000 7fa4d259c700 1 -- 172.21.15.31:0/3364401541 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4cc0c3a80 con 0x7fa4cc0b1560 2023-10-06T02:31:22.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.280+0000 7fa4d259c700 1 --2- 172.21.15.31:0/3364401541 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4cc0b1560 0x7fa4cc1698a0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc0c80f0 tx=0x7fa4c4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:22.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.281+0000 7fa4c1ffb700 1 -- 172.21.15.31:0/3364401541 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c4001de0 con 0x7fa4cc0b1560 2023-10-06T02:31:22.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.281+0000 7fa4c1ffb700 1 -- 172.21.15.31:0/3364401541 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4c40020a0 con 0x7fa4cc0b1560 2023-10-06T02:31:22.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.281+0000 7fa4c1ffb700 1 -- 172.21.15.31:0/3364401541 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c40016f0 con 0x7fa4cc0b1560 2023-10-06T02:31:22.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.281+0000 7fa4d2d9d700 1 -- 172.21.15.31:0/3364401541 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4cc16a5d0 con 0x7fa4cc0b1560 2023-10-06T02:31:22.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.281+0000 7fa4d2d9d700 1 -- 172.21.15.31:0/3364401541 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4cc0c3710 con 0x7fa4cc0b1560 2023-10-06T02:31:22.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.282+0000 7fa4d2d9d700 1 -- 172.21.15.31:0/3364401541 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4cc0648e0 con 0x7fa4cc0b1560 2023-10-06T02:31:22.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.283+0000 7fa4c1ffb700 1 -- 172.21.15.31:0/3364401541 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 85403+0+0 (secure 0 0 0) 0x7fa4c4004bb0 con 0x7fa4cc0b1560 2023-10-06T02:31:22.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.283+0000 7fa4c1ffb700 1 --2- 172.21.15.31:0/3364401541 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa4b0067680 0x7fa4b0069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:22.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.284+0000 7fa4c1ffb700 1 -- 172.21.15.31:0/3364401541 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5234+0+0 (secure 0 0 0) 0x7fa4c40874b0 con 0x7fa4cc0b1560 2023-10-06T02:31:22.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.284+0000 7fa4d1d9b700 1 --2- 172.21.15.31:0/3364401541 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa4b0067680 0x7fa4b0069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:22.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.285+0000 7fa4d1d9b700 1 --2- 172.21.15.31:0/3364401541 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa4b0067680 0x7fa4b0069b30 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fa4c8006f00 tx=0x7fa4c80091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:22.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.288+0000 7fa4c1ffb700 1 -- 172.21.15.31:0/3364401541 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa4cc0648e0 con 0x7fa4cc0b1560 2023-10-06T02:31:22.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.489+0000 7fa4d2d9d700 1 -- 172.21.15.31:0/3364401541 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa4cc0648e0 con 0x7fa4cc0b1560 2023-10-06T02:31:22.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.490+0000 7fa4c1ffb700 1 -- 172.21.15.31:0/3364401541 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v152) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa4cc0648e0 con 0x7fa4cc0b1560 2023-10-06T02:31:22.491 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-10-06T02:31:22.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.492+0000 7fa4d2d9d700 1 -- 172.21.15.31:0/3364401541 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa4b0067680 msgr2=0x7fa4b0069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:22.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.492+0000 7fa4d2d9d700 1 --2- 172.21.15.31:0/3364401541 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa4b0067680 0x7fa4b0069b30 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fa4c8006f00 tx=0x7fa4c80091a0 comp rx=0 tx=0).stop 2023-10-06T02:31:22.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.492+0000 7fa4d2d9d700 1 -- 172.21.15.31:0/3364401541 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4cc0b1560 msgr2=0x7fa4cc1698a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:22.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.492+0000 7fa4d2d9d700 1 --2- 172.21.15.31:0/3364401541 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4cc0b1560 0x7fa4cc1698a0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc0c80f0 tx=0x7fa4c4016000 comp rx=0 tx=0).stop 2023-10-06T02:31:22.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.492+0000 7fa4d2d9d700 1 -- 172.21.15.31:0/3364401541 shutdown_connections 2023-10-06T02:31:22.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.492+0000 7fa4d2d9d700 1 --2- 172.21.15.31:0/3364401541 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4cc0b1560 0x7fa4cc1698a0 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:22.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.492+0000 7fa4d2d9d700 1 --2- 172.21.15.31:0/3364401541 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa4b0067680 0x7fa4b0069b30 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:22.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.492+0000 7fa4d2d9d700 1 --2- 172.21.15.31:0/3364401541 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa4b80051c0 0x7fa4cc0c3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:22.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.492+0000 7fa4d2d9d700 1 --2- 172.21.15.31:0/3364401541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4cc13db70 0x7fa4cc0b1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:22.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.492+0000 7fa4d2d9d700 1 -- 172.21.15.31:0/3364401541 >> 172.21.15.31:0/3364401541 conn(0x7fa4cc06d110 msgr2=0x7fa4cc0ca6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:22.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.492+0000 7fa4d2d9d700 1 -- 172.21.15.31:0/3364401541 shutdown_connections 2023-10-06T02:31:22.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.492+0000 7fa4d2d9d700 1 -- 172.21.15.31:0/3364401541 wait complete. 2023-10-06T02:31:22.510 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T02:31:22.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.700+0000 7fbec8d9d700 1 Processor -- start 2023-10-06T02:31:22.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.700+0000 7fbec8d9d700 1 -- start start 2023-10-06T02:31:22.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.702+0000 7fbec8d9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fbec4140a70 0x7fbec4069b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:22.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.702+0000 7fbec8d9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbec415b640 con 0x7fbec40c7c60 2023-10-06T02:31:22.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.702+0000 7fbec8d9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbec415b920 con 0x7fbec406de50 2023-10-06T02:31:22.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.702+0000 7fbec8d9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbec40c6800 con 0x7fbec4140a70 2023-10-06T02:31:22.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.702+0000 7fbec37fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fbec4140a70 0x7fbec4069b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:22.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.702+0000 7fbec37fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fbec4140a70 0x7fbec4069b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59752/0 (socket says 172.21.15.31:59752) 2023-10-06T02:31:22.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.702+0000 7fbec37fe700 1 -- 172.21.15.31:0/3081797070 learned_addr learned my addr 172.21.15.31:0/3081797070 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:22.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.703+0000 7fbec37fe700 1 -- 172.21.15.31:0/3081797070 >> v1:172.21.15.62:6789/0 conn(0x7fbec406de50 legacy=0x7fbec4157a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:22.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.703+0000 7fbec37fe700 1 -- 172.21.15.31:0/3081797070 >> v1:172.21.15.31:6789/0 conn(0x7fbec40c7c60 legacy=0x7fbec4158190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:22.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.703+0000 7fbec37fe700 1 -- 172.21.15.31:0/3081797070 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbec415bc00 con 0x7fbec4140a70 2023-10-06T02:31:22.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.703+0000 7fbec37fe700 1 --2- 172.21.15.31:0/3081797070 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbec4140a70 0x7fbec4069b00 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fbeb40099b0 tx=0x7fbeb4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e897a69e561cc463 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:22.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.703+0000 7fbec27fc700 1 -- 172.21.15.31:0/3081797070 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb400b860 con 0x7fbec4140a70 2023-10-06T02:31:22.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.703+0000 7fbec27fc700 1 -- 172.21.15.31:0/3081797070 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbeb400b9c0 con 0x7fbec4140a70 2023-10-06T02:31:22.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.703+0000 7fbec27fc700 1 -- 172.21.15.31:0/3081797070 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb400bd60 con 0x7fbec4140a70 2023-10-06T02:31:22.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.704+0000 7fbec8d9d700 1 -- 172.21.15.31:0/3081797070 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbec4140a70 msgr2=0x7fbec4069b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:22.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.704+0000 7fbec8d9d700 1 --2- 172.21.15.31:0/3081797070 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbec4140a70 0x7fbec4069b00 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fbeb40099b0 tx=0x7fbeb4009250 comp rx=0 tx=0).stop 2023-10-06T02:31:22.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.704+0000 7fbec8d9d700 1 -- 172.21.15.31:0/3081797070 shutdown_connections 2023-10-06T02:31:22.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.704+0000 7fbec8d9d700 1 --2- 172.21.15.31:0/3081797070 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbec4140a70 0x7fbec4069b00 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:22.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.704+0000 7fbec8d9d700 1 -- 172.21.15.31:0/3081797070 >> 172.21.15.31:0/3081797070 conn(0x7fbec406d110 msgr2=0x7fbec40cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:22.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.704+0000 7fbec8d9d700 1 -- 172.21.15.31:0/3081797070 shutdown_connections 2023-10-06T02:31:22.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.704+0000 7fbec8d9d700 1 -- 172.21.15.31:0/3081797070 wait complete. 2023-10-06T02:31:22.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.705+0000 7fbec8d9d700 1 Processor -- start 2023-10-06T02:31:22.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.705+0000 7fbec8d9d700 1 -- start start 2023-10-06T02:31:22.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.706+0000 7fbec8d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbec406de50 0x7fbec40aef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:22.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.706+0000 7fbec8d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbec40c7c60 0x7fbec413bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:22.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.706+0000 7fbec8d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbec4140a70 0x7fbec413c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:22.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.706+0000 7fbec8d9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbec415ab60 con 0x7fbec4140a70 2023-10-06T02:31:22.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.706+0000 7fbec8d9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbec40af4c0 con 0x7fbec406de50 2023-10-06T02:31:22.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.706+0000 7fbec8d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbec4159f50 con 0x7fbec40c7c60 2023-10-06T02:31:22.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.707+0000 7fbec3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbec4140a70 0x7fbec413c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:22.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.707+0000 7fbec3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbec4140a70 0x7fbec413c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59768/0 (socket says 172.21.15.31:59768) 2023-10-06T02:31:22.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.707+0000 7fbec3fff700 1 -- 172.21.15.31:0/3614171033 learned_addr learned my addr 172.21.15.31:0/3614171033 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:22.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.707+0000 7fbec37fe700 1 --2- 172.21.15.31:0/3614171033 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbec406de50 0x7fbec40aef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:22.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.707+0000 7fbec2ffd700 1 --2- 172.21.15.31:0/3614171033 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbec40c7c60 0x7fbec413bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:22.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.707+0000 7fbec3fff700 1 -- 172.21.15.31:0/3614171033 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbec40c7c60 msgr2=0x7fbec413bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:22.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.707+0000 7fbec3fff700 1 --2- 172.21.15.31:0/3614171033 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbec40c7c60 0x7fbec413bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:22.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.707+0000 7fbec3fff700 1 -- 172.21.15.31:0/3614171033 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbec406de50 msgr2=0x7fbec40aef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:22.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.707+0000 7fbec3fff700 1 --2- 172.21.15.31:0/3614171033 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbec406de50 0x7fbec40aef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:22.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.707+0000 7fbec3fff700 1 -- 172.21.15.31:0/3614171033 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeb40094f0 con 0x7fbec4140a70 2023-10-06T02:31:22.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.707+0000 7fbec37fe700 1 --2- 172.21.15.31:0/3614171033 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbec406de50 0x7fbec40aef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:31:22.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.707+0000 7fbec2ffd700 1 --2- 172.21.15.31:0/3614171033 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbec40c7c60 0x7fbec413bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:31:22.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.707+0000 7fbec3fff700 1 --2- 172.21.15.31:0/3614171033 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbec4140a70 0x7fbec413c490 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7fbebc00c910 tx=0x7fbebc001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:22.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.708+0000 7fbeab7fe700 1 -- 172.21.15.31:0/3614171033 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbebc00a7a0 con 0x7fbec4140a70 2023-10-06T02:31:22.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.708+0000 7fbeab7fe700 1 -- 172.21.15.31:0/3614171033 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbebc00a900 con 0x7fbec4140a70 2023-10-06T02:31:22.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.708+0000 7fbec8d9d700 1 -- 172.21.15.31:0/3614171033 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbec415a230 con 0x7fbec4140a70 2023-10-06T02:31:22.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.708+0000 7fbeab7fe700 1 -- 172.21.15.31:0/3614171033 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbebc00acc0 con 0x7fbec4140a70 2023-10-06T02:31:22.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.708+0000 7fbec8d9d700 1 -- 172.21.15.31:0/3614171033 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbec415a7f0 con 0x7fbec4140a70 2023-10-06T02:31:22.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.709+0000 7fbec8d9d700 1 -- 172.21.15.31:0/3614171033 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbec404c3c0 con 0x7fbec4140a70 2023-10-06T02:31:22.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.710+0000 7fbeab7fe700 1 -- 172.21.15.31:0/3614171033 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 85403+0+0 (secure 0 0 0) 0x7fbebc012630 con 0x7fbec4140a70 2023-10-06T02:31:22.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.710+0000 7fbeab7fe700 1 --2- 172.21.15.31:0/3614171033 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fbeb0067680 0x7fbeb0069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:22.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.711+0000 7fbeab7fe700 1 -- 172.21.15.31:0/3614171033 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5234+0+0 (secure 0 0 0) 0x7fbebc0857e0 con 0x7fbec4140a70 2023-10-06T02:31:22.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.711+0000 7fbec37fe700 1 --2- 172.21.15.31:0/3614171033 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fbeb0067680 0x7fbeb0069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:22.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.712+0000 7fbec37fe700 1 --2- 172.21.15.31:0/3614171033 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fbeb0067680 0x7fbeb0069b30 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fbeb40099b0 tx=0x7fbeb4017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:22.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.714+0000 7fbeab7fe700 1 -- 172.21.15.31:0/3614171033 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbebc082ef0 con 0x7fbec4140a70 2023-10-06T02:31:22.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.917+0000 7fbec8d9d700 1 -- 172.21.15.31:0/3614171033 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fbec415bee0 con 0x7fbec4140a70 2023-10-06T02:31:22.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.917+0000 7fbeab7fe700 1 -- 172.21.15.31:0/3614171033 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v152) v1 ==== 92+0+10 (secure 0 0 0) 0x7fbebc051ce0 con 0x7fbec4140a70 2023-10-06T02:31:22.919 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 8 2023-10-06T02:31:22.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.920+0000 7fbec8d9d700 1 -- 172.21.15.31:0/3614171033 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fbeb0067680 msgr2=0x7fbeb0069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:22.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.920+0000 7fbec8d9d700 1 --2- 172.21.15.31:0/3614171033 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fbeb0067680 0x7fbeb0069b30 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fbeb40099b0 tx=0x7fbeb4017000 comp rx=0 tx=0).stop 2023-10-06T02:31:22.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.920+0000 7fbec8d9d700 1 -- 172.21.15.31:0/3614171033 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbec4140a70 msgr2=0x7fbec413c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:22.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.920+0000 7fbec8d9d700 1 --2- 172.21.15.31:0/3614171033 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbec4140a70 0x7fbec413c490 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7fbebc00c910 tx=0x7fbebc001710 comp rx=0 tx=0).stop 2023-10-06T02:31:22.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.920+0000 7fbec8d9d700 1 -- 172.21.15.31:0/3614171033 shutdown_connections 2023-10-06T02:31:22.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.920+0000 7fbec8d9d700 1 --2- 172.21.15.31:0/3614171033 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbec4140a70 0x7fbec413c490 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:22.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.920+0000 7fbec8d9d700 1 --2- 172.21.15.31:0/3614171033 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbec406de50 0x7fbec40aef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:22.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.920+0000 7fbec8d9d700 1 --2- 172.21.15.31:0/3614171033 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fbeb0067680 0x7fbeb0069b30 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:22.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.920+0000 7fbec8d9d700 1 --2- 172.21.15.31:0/3614171033 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbec40c7c60 0x7fbec413bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:22.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.920+0000 7fbec8d9d700 1 -- 172.21.15.31:0/3614171033 >> 172.21.15.31:0/3614171033 conn(0x7fbec406d110 msgr2=0x7fbec40c0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:22.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.920+0000 7fbec8d9d700 1 -- 172.21.15.31:0/3614171033 shutdown_connections 2023-10-06T02:31:22.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:22.920+0000 7fbec8d9d700 1 -- 172.21.15.31:0/3614171033 wait complete. 2023-10-06T02:31:22.938 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T02:31:23.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.124+0000 7f6354d9d700 1 Processor -- start 2023-10-06T02:31:23.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.124+0000 7f6354d9d700 1 -- start start 2023-10-06T02:31:23.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.125+0000 7f6354d9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f6350140e40 0x7f63500c6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:23.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.125+0000 7f6354d9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f635006de50 0x7f63500aeef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:23.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.125+0000 7f6354d9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63500c6d50 con 0x7f6350140a70 2023-10-06T02:31:23.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.125+0000 7f6354d9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63500c7030 con 0x7f635006de50 2023-10-06T02:31:23.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.125+0000 7f6354d9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63500c7310 con 0x7f6350140e40 2023-10-06T02:31:23.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.125+0000 7f634ffff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f635006de50 0x7f63500aeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:23.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.126+0000 7f634ffff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f635006de50 0x7f63500aeef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59772/0 (socket says 172.21.15.31:59772) 2023-10-06T02:31:23.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.126+0000 7f634ffff700 1 -- 172.21.15.31:0/1834296658 learned_addr learned my addr 172.21.15.31:0/1834296658 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:23.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.126+0000 7f634f7fe700 1 --1- 172.21.15.31:0/1834296658 >> v1:172.21.15.31:6789/0 conn(0x7f6350140a70 0x7f63500c7df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:39464/0 (socket says 172.21.15.31:39464) 2023-10-06T02:31:23.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.126+0000 7f634effd700 1 --2- 172.21.15.31:0/1834296658 >> v2:172.21.15.62:3300/0 conn(0x7f6350140e40 0x7f63500c6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:23.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.126+0000 7f634ffff700 1 -- 172.21.15.31:0/1834296658 >> v2:172.21.15.62:3300/0 conn(0x7f6350140e40 msgr2=0x7f63500c6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:23.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.126+0000 7f634ffff700 1 --2- 172.21.15.31:0/1834296658 >> v2:172.21.15.62:3300/0 conn(0x7f6350140e40 0x7f63500c6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:23.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.126+0000 7f634ffff700 1 -- 172.21.15.31:0/1834296658 >> v1:172.21.15.31:6789/0 conn(0x7f6350140a70 legacy=0x7f63500c7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:23.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.126+0000 7f634ffff700 1 -- 172.21.15.31:0/1834296658 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63500c75f0 con 0x7f635006de50 2023-10-06T02:31:23.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.126+0000 7f634ffff700 1 --2- 172.21.15.31:0/1834296658 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f635006de50 0x7f63500aeef0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f63500cc0e0 tx=0x7f6348009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59d5f907764a356a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:23.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.126+0000 7f634e7fc700 1 -- 172.21.15.31:0/1834296658 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f634800e070 con 0x7f635006de50 2023-10-06T02:31:23.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.127+0000 7f634e7fc700 1 -- 172.21.15.31:0/1834296658 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f634800ba20 con 0x7f635006de50 2023-10-06T02:31:23.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.127+0000 7f634e7fc700 1 -- 172.21.15.31:0/1834296658 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63480092e0 con 0x7f635006de50 2023-10-06T02:31:23.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.127+0000 7f6354d9d700 1 -- 172.21.15.31:0/1834296658 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f635006de50 msgr2=0x7f63500aeef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:23.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.127+0000 7f6354d9d700 1 --2- 172.21.15.31:0/1834296658 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f635006de50 0x7f63500aeef0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f63500cc0e0 tx=0x7f6348009840 comp rx=0 tx=0).stop 2023-10-06T02:31:23.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.127+0000 7f6354d9d700 1 -- 172.21.15.31:0/1834296658 shutdown_connections 2023-10-06T02:31:23.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.127+0000 7f6354d9d700 1 --2- 172.21.15.31:0/1834296658 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f635006de50 0x7f63500aeef0 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:23.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.127+0000 7f6354d9d700 1 --2- 172.21.15.31:0/1834296658 >> v2:172.21.15.62:3300/0 conn(0x7f6350140e40 0x7f63500c6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:23.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.127+0000 7f6354d9d700 1 -- 172.21.15.31:0/1834296658 >> 172.21.15.31:0/1834296658 conn(0x7f635006d170 msgr2=0x7f63500cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:23.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.127+0000 7f6354d9d700 1 -- 172.21.15.31:0/1834296658 shutdown_connections 2023-10-06T02:31:23.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.127+0000 7f6354d9d700 1 -- 172.21.15.31:0/1834296658 wait complete. 2023-10-06T02:31:23.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.129+0000 7f6354d9d700 1 Processor -- start 2023-10-06T02:31:23.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.129+0000 7f6354d9d700 1 -- start start 2023-10-06T02:31:23.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.130+0000 7f6354d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f635006de50 0x7f635013e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:23.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.130+0000 7f6354d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6350140a70 0x7f635016a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:23.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.130+0000 7f6354d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6350140e40 0x7f635016d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:23.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.130+0000 7f6354d9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63500c56f0 con 0x7f635006de50 2023-10-06T02:31:23.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.130+0000 7f6354d9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f635016d4f0 con 0x7f6350140a70 2023-10-06T02:31:23.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.130+0000 7f6354d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f635016a9d0 con 0x7f6350140e40 2023-10-06T02:31:23.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.130+0000 7f634f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f635006de50 0x7f635013e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:23.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.130+0000 7f634f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f635006de50 0x7f635013e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59784/0 (socket says 172.21.15.31:59784) 2023-10-06T02:31:23.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.130+0000 7f634f7fe700 1 -- 172.21.15.31:0/2008504606 learned_addr learned my addr 172.21.15.31:0/2008504606 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:23.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.130+0000 7f634effd700 1 --2- 172.21.15.31:0/2008504606 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6350140a70 0x7f635016a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:23.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.130+0000 7f634ffff700 1 --2- 172.21.15.31:0/2008504606 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6350140e40 0x7f635016d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:23.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.131+0000 7f634f7fe700 1 -- 172.21.15.31:0/2008504606 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6350140e40 msgr2=0x7f635016d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:23.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.131+0000 7f634f7fe700 1 --2- 172.21.15.31:0/2008504606 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6350140e40 0x7f635016d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:23.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.131+0000 7f634f7fe700 1 -- 172.21.15.31:0/2008504606 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6350140a70 msgr2=0x7f635016a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:23.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.131+0000 7f634f7fe700 1 --2- 172.21.15.31:0/2008504606 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6350140a70 0x7f635016a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:23.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.131+0000 7f634f7fe700 1 -- 172.21.15.31:0/2008504606 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6348009ae0 con 0x7f635006de50 2023-10-06T02:31:23.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.131+0000 7f634f7fe700 1 --2- 172.21.15.31:0/2008504606 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f635006de50 0x7f635013e4f0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f634000c7c0 tx=0x7f6340007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:23.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.131+0000 7f63377fe700 1 -- 172.21.15.31:0/2008504606 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6340005a70 con 0x7f635006de50 2023-10-06T02:31:23.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.131+0000 7f63377fe700 1 -- 172.21.15.31:0/2008504606 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6340005bf0 con 0x7f635006de50 2023-10-06T02:31:23.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.131+0000 7f6354d9d700 1 -- 172.21.15.31:0/2008504606 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63500c5130 con 0x7f635006de50 2023-10-06T02:31:23.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.131+0000 7f63377fe700 1 -- 172.21.15.31:0/2008504606 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63400070d0 con 0x7f635006de50 2023-10-06T02:31:23.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.131+0000 7f6354d9d700 1 -- 172.21.15.31:0/2008504606 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f635016d180 con 0x7f635006de50 2023-10-06T02:31:23.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.132+0000 7f6354d9d700 1 -- 172.21.15.31:0/2008504606 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f635004c3c0 con 0x7f635006de50 2023-10-06T02:31:23.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.133+0000 7f63377fe700 1 -- 172.21.15.31:0/2008504606 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 85403+0+0 (secure 0 0 0) 0x7f6340007480 con 0x7f635006de50 2023-10-06T02:31:23.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.134+0000 7f63377fe700 1 --2- 172.21.15.31:0/2008504606 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f633c067680 0x7f633c069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:23.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.134+0000 7f63377fe700 1 -- 172.21.15.31:0/2008504606 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5234+0+0 (secure 0 0 0) 0x7f63400848e0 con 0x7f635006de50 2023-10-06T02:31:23.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.135+0000 7f634effd700 1 --2- 172.21.15.31:0/2008504606 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f633c067680 0x7f633c069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:23.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.135+0000 7f634effd700 1 --2- 172.21.15.31:0/2008504606 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f633c067680 0x7f633c069b30 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f63500c2dd0 tx=0x7f6344009120 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:23.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.138+0000 7f63377fe700 1 -- 172.21.15.31:0/2008504606 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6340081ff0 con 0x7f635006de50 2023-10-06T02:31:23.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.342+0000 7f6354d9d700 1 -- 172.21.15.31:0/2008504606 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f635016d7d0 con 0x7f635006de50 2023-10-06T02:31:23.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.343+0000 7f63377fe700 1 -- 172.21.15.31:0/2008504606 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v152) v1 ==== 84+0+126 (secure 0 0 0) 0x7f6340017090 con 0x7f635006de50 2023-10-06T02:31:23.344 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:31:23.344 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2023-10-06T02:31:23.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:31:23.344 INFO:teuthology.orchestra.run.smithi031.stdout: "poolnum": 1, 2023-10-06T02:31:23.344 INFO:teuthology.orchestra.run.smithi031.stdout: "poolname": ".mgr" 2023-10-06T02:31:23.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:31:23.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:31:23.344 INFO:teuthology.orchestra.run.smithi031.stdout: "poolnum": 2, 2023-10-06T02:31:23.344 INFO:teuthology.orchestra.run.smithi031.stdout: "poolname": "rbd" 2023-10-06T02:31:23.344 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:31:23.344 INFO:teuthology.orchestra.run.smithi031.stdout:] 2023-10-06T02:31:23.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.345+0000 7f6354d9d700 1 -- 172.21.15.31:0/2008504606 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f633c067680 msgr2=0x7f633c069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:23.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.345+0000 7f6354d9d700 1 --2- 172.21.15.31:0/2008504606 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f633c067680 0x7f633c069b30 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f63500c2dd0 tx=0x7f6344009120 comp rx=0 tx=0).stop 2023-10-06T02:31:23.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.345+0000 7f6354d9d700 1 -- 172.21.15.31:0/2008504606 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f635006de50 msgr2=0x7f635013e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:23.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.345+0000 7f6354d9d700 1 --2- 172.21.15.31:0/2008504606 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f635006de50 0x7f635013e4f0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f634000c7c0 tx=0x7f6340007f70 comp rx=0 tx=0).stop 2023-10-06T02:31:23.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.345+0000 7f6354d9d700 1 -- 172.21.15.31:0/2008504606 shutdown_connections 2023-10-06T02:31:23.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.345+0000 7f6354d9d700 1 --2- 172.21.15.31:0/2008504606 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f633c067680 0x7f633c069b30 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:23.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.345+0000 7f6354d9d700 1 --2- 172.21.15.31:0/2008504606 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6350140e40 0x7f635016d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:23.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.345+0000 7f6354d9d700 1 --2- 172.21.15.31:0/2008504606 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f635006de50 0x7f635013e4f0 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:23.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.345+0000 7f6354d9d700 1 --2- 172.21.15.31:0/2008504606 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6350140a70 0x7f635016a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:23.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.345+0000 7f6354d9d700 1 -- 172.21.15.31:0/2008504606 >> 172.21.15.31:0/2008504606 conn(0x7f635006d170 msgr2=0x7f63500c0ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:23.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.345+0000 7f6354d9d700 1 -- 172.21.15.31:0/2008504606 shutdown_connections 2023-10-06T02:31:23.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.345+0000 7f6354d9d700 1 -- 172.21.15.31:0/2008504606 wait complete. 2023-10-06T02:31:23.363 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T02:31:23.363 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T02:31:23.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.554+0000 7f0ec7fff700 1 Processor -- start 2023-10-06T02:31:23.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.554+0000 7f0ec7fff700 1 -- start start 2023-10-06T02:31:23.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.555+0000 7f0ec7fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0ec8142bb0 0x7f0ec80c8920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:23.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.555+0000 7f0ec7fff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0ec8142f80 0x7f0ec815d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:23.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.555+0000 7f0ec7fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0ec815d750 0x7f0ec80cbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:23.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.555+0000 7f0ec7fff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ec806e0e0 con 0x7f0ec8142bb0 2023-10-06T02:31:23.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.555+0000 7f0ec7fff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ec815dc60 con 0x7f0ec8142f80 2023-10-06T02:31:23.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.555+0000 7f0ec7fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec815df40 con 0x7f0ec815d750 2023-10-06T02:31:23.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.556+0000 7f0ec6ffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0ec8142bb0 0x7f0ec80c8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:23.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.556+0000 7f0ec6ffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0ec8142bb0 0x7f0ec80c8920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59796/0 (socket says 172.21.15.31:59796) 2023-10-06T02:31:23.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.556+0000 7f0ec6ffd700 1 -- 172.21.15.31:0/3339783308 learned_addr learned my addr 172.21.15.31:0/3339783308 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:23.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.556+0000 7f0ec67fc700 1 --2- 172.21.15.31:0/3339783308 >> v2:172.21.15.62:3300/0 conn(0x7f0ec8142f80 0x7f0ec815d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:23.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.556+0000 7f0ec77fe700 1 --2- 172.21.15.31:0/3339783308 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0ec815d750 0x7f0ec80cbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:23.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.556+0000 7f0ec6ffd700 1 -- 172.21.15.31:0/3339783308 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0ec815d750 msgr2=0x7f0ec80cbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:23.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.556+0000 7f0ec6ffd700 1 --2- 172.21.15.31:0/3339783308 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0ec815d750 0x7f0ec80cbeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:23.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.556+0000 7f0ec6ffd700 1 -- 172.21.15.31:0/3339783308 >> v2:172.21.15.62:3300/0 conn(0x7f0ec8142f80 msgr2=0x7f0ec815d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:23.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.556+0000 7f0ec6ffd700 1 --2- 172.21.15.31:0/3339783308 >> v2:172.21.15.62:3300/0 conn(0x7f0ec8142f80 0x7f0ec815d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:23.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.556+0000 7f0ec6ffd700 1 -- 172.21.15.31:0/3339783308 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ec806dd70 con 0x7f0ec8142bb0 2023-10-06T02:31:23.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.556+0000 7f0ec77fe700 1 --2- 172.21.15.31:0/3339783308 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0ec815d750 0x7f0ec80cbeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:31:23.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.556+0000 7f0ec67fc700 1 --2- 172.21.15.31:0/3339783308 >> v2:172.21.15.62:3300/0 conn(0x7f0ec8142f80 0x7f0ec815d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:31:23.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.556+0000 7f0ec6ffd700 1 --2- 172.21.15.31:0/3339783308 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ec8142bb0 0x7f0ec80c8920 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f0ec806bb90 tx=0x7f0eb8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa886f363c5cdc60 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:23.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.557+0000 7f0ec5ffb700 1 -- 172.21.15.31:0/3339783308 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb800b860 con 0x7f0ec8142bb0 2023-10-06T02:31:23.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.557+0000 7f0ec5ffb700 1 -- 172.21.15.31:0/3339783308 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eb800b9c0 con 0x7f0ec8142bb0 2023-10-06T02:31:23.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.557+0000 7f0ec5ffb700 1 -- 172.21.15.31:0/3339783308 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb800bd80 con 0x7f0ec8142bb0 2023-10-06T02:31:23.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.557+0000 7f0ec7fff700 1 -- 172.21.15.31:0/3339783308 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ec8142bb0 msgr2=0x7f0ec80c8920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:23.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.557+0000 7f0ec7fff700 1 --2- 172.21.15.31:0/3339783308 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ec8142bb0 0x7f0ec80c8920 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f0ec806bb90 tx=0x7f0eb8009250 comp rx=0 tx=0).stop 2023-10-06T02:31:23.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.557+0000 7f0ec7fff700 1 -- 172.21.15.31:0/3339783308 shutdown_connections 2023-10-06T02:31:23.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.557+0000 7f0ec7fff700 1 --2- 172.21.15.31:0/3339783308 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0ec815d750 0x7f0ec80cbeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:23.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.557+0000 7f0ec7fff700 1 --2- 172.21.15.31:0/3339783308 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ec8142bb0 0x7f0ec80c8920 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:23.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.557+0000 7f0ec7fff700 1 --2- 172.21.15.31:0/3339783308 >> v2:172.21.15.62:3300/0 conn(0x7f0ec8142f80 0x7f0ec815d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:23.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.557+0000 7f0ec7fff700 1 -- 172.21.15.31:0/3339783308 >> 172.21.15.31:0/3339783308 conn(0x7f0ec8067bf0 msgr2=0x7f0ec8067ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:23.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.557+0000 7f0ec7fff700 1 -- 172.21.15.31:0/3339783308 shutdown_connections 2023-10-06T02:31:23.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.557+0000 7f0ec7fff700 1 -- 172.21.15.31:0/3339783308 wait complete. 2023-10-06T02:31:23.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.559+0000 7f0ec7fff700 1 Processor -- start 2023-10-06T02:31:23.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.559+0000 7f0ec7fff700 1 -- start start 2023-10-06T02:31:23.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.560+0000 7f0ec7fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0ec8142bb0 0x7f0ec8158eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:23.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.560+0000 7f0ec7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ec8142f80 0x7f0ec815a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:23.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.560+0000 7f0ec7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ec815d750 0x7f0ec81691a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:23.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.560+0000 7f0ec7fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec80b2430 con 0x7f0ec8142f80 2023-10-06T02:31:23.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.560+0000 7f0ec7fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec815aee0 con 0x7f0ec815d750 2023-10-06T02:31:23.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.560+0000 7f0ec7fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec815b1c0 con 0x7f0ec8142bb0 2023-10-06T02:31:23.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.560+0000 7f0ec67fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ec8142f80 0x7f0ec815a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:23.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.560+0000 7f0ec67fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ec8142f80 0x7f0ec815a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59812/0 (socket says 172.21.15.31:59812) 2023-10-06T02:31:23.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.560+0000 7f0ec67fc700 1 -- 172.21.15.31:0/3083077907 learned_addr learned my addr 172.21.15.31:0/3083077907 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:23.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.560+0000 7f0ec6ffd700 1 --2- 172.21.15.31:0/3083077907 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0ec8142bb0 0x7f0ec8158eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:23.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.560+0000 7f0ec77fe700 1 --2- 172.21.15.31:0/3083077907 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ec815d750 0x7f0ec81691a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:23.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.561+0000 7f0ec67fc700 1 -- 172.21.15.31:0/3083077907 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0ec8142bb0 msgr2=0x7f0ec8158eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:23.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.561+0000 7f0ec67fc700 1 --2- 172.21.15.31:0/3083077907 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0ec8142bb0 0x7f0ec8158eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:23.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.561+0000 7f0ec67fc700 1 -- 172.21.15.31:0/3083077907 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ec815d750 msgr2=0x7f0ec81691a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:23.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.561+0000 7f0ec67fc700 1 --2- 172.21.15.31:0/3083077907 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ec815d750 0x7f0ec81691a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:23.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.561+0000 7f0ec67fc700 1 -- 172.21.15.31:0/3083077907 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eb80094f0 con 0x7f0ec8142f80 2023-10-06T02:31:23.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.561+0000 7f0ec67fc700 1 --2- 172.21.15.31:0/3083077907 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ec8142f80 0x7f0ec815a490 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc00c880 tx=0x7f0ebc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:23.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.561+0000 7f0eaeffd700 1 -- 172.21.15.31:0/3083077907 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ebc011600 con 0x7f0ec8142f80 2023-10-06T02:31:23.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.561+0000 7f0eaeffd700 1 -- 172.21.15.31:0/3083077907 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ebc011760 con 0x7f0ec8142f80 2023-10-06T02:31:23.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.561+0000 7f0ec7fff700 1 -- 172.21.15.31:0/3083077907 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ec80b26f0 con 0x7f0ec8142f80 2023-10-06T02:31:23.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.561+0000 7f0eaeffd700 1 -- 172.21.15.31:0/3083077907 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ebc011ae0 con 0x7f0ec8142f80 2023-10-06T02:31:23.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.561+0000 7f0ec7fff700 1 -- 172.21.15.31:0/3083077907 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ec81593f0 con 0x7f0ec8142f80 2023-10-06T02:31:23.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.562+0000 7f0ec7fff700 1 -- 172.21.15.31:0/3083077907 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ec80b2c10 con 0x7f0ec8142f80 2023-10-06T02:31:23.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.563+0000 7f0eaeffd700 1 -- 172.21.15.31:0/3083077907 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 85403+0+0 (secure 0 0 0) 0x7f0ebc010350 con 0x7f0ec8142f80 2023-10-06T02:31:23.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.565+0000 7f0eaeffd700 1 --2- 172.21.15.31:0/3083077907 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0eb4067680 0x7f0eb4069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:23.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.565+0000 7f0eaeffd700 1 -- 172.21.15.31:0/3083077907 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5234+0+0 (secure 0 0 0) 0x7f0ebc0854e0 con 0x7f0ec8142f80 2023-10-06T02:31:23.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.565+0000 7f0ec6ffd700 1 --2- 172.21.15.31:0/3083077907 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0eb4067680 0x7f0eb4069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:23.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.566+0000 7f0ec6ffd700 1 --2- 172.21.15.31:0/3083077907 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0eb4067680 0x7f0eb4069b30 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f0ec806bb90 tx=0x7f0eb8017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:23.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.569+0000 7f0eaeffd700 1 -- 172.21.15.31:0/3083077907 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0ec80b2c10 con 0x7f0ec8142f80 2023-10-06T02:31:23.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:23.774+0000 7f0ec7fff700 1 -- 172.21.15.31:0/3083077907 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f0ec80b2c10 con 0x7f0ec8142f80 2023-10-06T02:31:24.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:24.216+0000 7f0eaeffd700 1 -- 172.21.15.31:0/3083077907 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v153) v1 ==== 138+0+0 (secure 0 0 0) 0x7f0ec80b2c10 con 0x7f0ec8142f80 2023-10-06T02:31:24.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:24.221+0000 7f0ec7fff700 1 -- 172.21.15.31:0/3083077907 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0eb4067680 msgr2=0x7f0eb4069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:24.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:24.221+0000 7f0ec7fff700 1 --2- 172.21.15.31:0/3083077907 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0eb4067680 0x7f0eb4069b30 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f0ec806bb90 tx=0x7f0eb8017000 comp rx=0 tx=0).stop 2023-10-06T02:31:24.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:24.221+0000 7f0ec7fff700 1 -- 172.21.15.31:0/3083077907 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ec8142f80 msgr2=0x7f0ec815a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:24.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:24.221+0000 7f0ec7fff700 1 --2- 172.21.15.31:0/3083077907 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ec8142f80 0x7f0ec815a490 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc00c880 tx=0x7f0ebc008040 comp rx=0 tx=0).stop 2023-10-06T02:31:24.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:24.222+0000 7f0ec7fff700 1 -- 172.21.15.31:0/3083077907 shutdown_connections 2023-10-06T02:31:24.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:24.222+0000 7f0ec7fff700 1 --2- 172.21.15.31:0/3083077907 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ec815d750 0x7f0ec81691a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:24.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:24.222+0000 7f0ec7fff700 1 --2- 172.21.15.31:0/3083077907 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0eb4067680 0x7f0eb4069b30 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:24.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:24.222+0000 7f0ec7fff700 1 --2- 172.21.15.31:0/3083077907 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0ec8142bb0 0x7f0ec8158eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:24.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:24.222+0000 7f0ec7fff700 1 --2- 172.21.15.31:0/3083077907 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ec8142f80 0x7f0ec815a490 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:24.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:24.222+0000 7f0ec7fff700 1 -- 172.21.15.31:0/3083077907 >> 172.21.15.31:0/3083077907 conn(0x7f0ec8067bf0 msgr2=0x7f0ec80cbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:24.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:24.222+0000 7f0ec7fff700 1 -- 172.21.15.31:0/3083077907 shutdown_connections 2023-10-06T02:31:24.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:24.223+0000 7f0ec7fff700 1 -- 172.21.15.31:0/3083077907 wait complete. 2023-10-06T02:31:24.225 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' created 2023-10-06T02:31:24.243 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T02:31:24.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:24.438+0000 7fd2b2b9d700 1 Processor -- start 2023-10-06T02:31:24.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:24.438+0000 7fd2b2b9d700 1 -- start start 2023-10-06T02:31:24.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:24.439+0000 7fd2b2b9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd2ac142c00 0x7fd2ac06b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:24.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:24.439+0000 7fd2b2b9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd2ac0c7f90 0x7fd2ac06c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:24.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:24.440+0000 7fd2b2b9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2ac0ca480 con 0x7fd2ac06bd20 2023-10-06T02:31:24.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:24.440+0000 7fd2b2b9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2ac15e240 con 0x7fd2ac0c7f90 2023-10-06T02:31:24.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:24.440+0000 7fd2b2b9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2ac06b350 con 0x7fd2ac142c00 2023-10-06T02:31:24.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:24.440+0000 7fd2b239c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd2ac0c7f90 0x7fd2ac06c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:24.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:24.440+0000 7fd2b239c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd2ac0c7f90 0x7fd2ac06c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59820/0 (socket says 172.21.15.31:59820) 2023-10-06T02:31:24.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:24.440+0000 7fd2b239c700 1 -- 172.21.15.31:0/4288369095 learned_addr learned my addr 172.21.15.31:0/4288369095 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:24.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:24.440+0000 7fd2b1b9b700 1 --2- 172.21.15.31:0/4288369095 >> v2:172.21.15.62:3300/0 conn(0x7fd2ac142c00 0x7fd2ac06b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:24.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:24.440+0000 7fd2b239c700 1 -- 172.21.15.31:0/4288369095 >> v2:172.21.15.62:3300/0 conn(0x7fd2ac142c00 msgr2=0x7fd2ac06b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:24.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:24.440+0000 7fd2b239c700 1 --2- 172.21.15.31:0/4288369095 >> v2:172.21.15.62:3300/0 conn(0x7fd2ac142c00 0x7fd2ac06b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:24.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:24.440+0000 7fd2b239c700 1 -- 172.21.15.31:0/4288369095 >> v1:172.21.15.31:6789/0 conn(0x7fd2ac06bd20 legacy=0x7fd2ac0c7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:24.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:24.440+0000 7fd2b239c700 1 -- 172.21.15.31:0/4288369095 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2ac15df60 con 0x7fd2ac0c7f90 2023-10-06T02:31:24.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:24.441+0000 7fd2b239c700 1 --2- 172.21.15.31:0/4288369095 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2ac0c7f90 0x7fd2ac06c6a0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac06f490 tx=0x7fd2a4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84f23bcae2a03bf2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:24.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:24.441+0000 7fd2b0b99700 1 -- 172.21.15.31:0/4288369095 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a400e070 con 0x7fd2ac0c7f90 2023-10-06T02:31:24.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:24.441+0000 7fd2b0b99700 1 -- 172.21.15.31:0/4288369095 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2a400ba20 con 0x7fd2ac0c7f90 2023-10-06T02:31:24.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:24.441+0000 7fd2b0b99700 1 -- 172.21.15.31:0/4288369095 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a40092e0 con 0x7fd2ac0c7f90 2023-10-06T02:31:24.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:24.441+0000 7fd2b2b9d700 1 -- 172.21.15.31:0/4288369095 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2ac0c7f90 msgr2=0x7fd2ac06c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:24.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:24.441+0000 7fd2b2b9d700 1 --2- 172.21.15.31:0/4288369095 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2ac0c7f90 0x7fd2ac06c6a0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac06f490 tx=0x7fd2a4009840 comp rx=0 tx=0).stop 2023-10-06T02:31:24.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:24.442+0000 7fd2b2b9d700 1 -- 172.21.15.31:0/4288369095 shutdown_connections 2023-10-06T02:31:24.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:24.442+0000 7fd2b2b9d700 1 --2- 172.21.15.31:0/4288369095 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2ac0c7f90 0x7fd2ac06c6a0 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:24.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:24.442+0000 7fd2b2b9d700 1 --2- 172.21.15.31:0/4288369095 >> v2:172.21.15.62:3300/0 conn(0x7fd2ac142c00 0x7fd2ac06b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:24.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:24.442+0000 7fd2b2b9d700 1 -- 172.21.15.31:0/4288369095 >> 172.21.15.31:0/4288369095 conn(0x7fd2ac0675a0 msgr2=0x7fd2ac0679a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:24.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:24.442+0000 7fd2b2b9d700 1 -- 172.21.15.31:0/4288369095 shutdown_connections 2023-10-06T02:31:24.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:24.442+0000 7fd2b2b9d700 1 -- 172.21.15.31:0/4288369095 wait complete. 2023-10-06T02:31:24.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:24.443+0000 7fd2b2b9d700 1 Processor -- start 2023-10-06T02:31:24.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:24.443+0000 7fd2b2b9d700 1 -- start start 2023-10-06T02:31:24.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:24.444+0000 7fd2b2b9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2ac06bd20 0x7fd2ac16d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:24.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:24.444+0000 7fd2b2b9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd2ac0c7f90 0x7fd2ac158f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:24.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:24.444+0000 7fd2b2b9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd2ac142c00 0x7fd2ac0b2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:24.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:24.444+0000 7fd2b2b9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ac0b1ea0 con 0x7fd2ac06bd20 2023-10-06T02:31:24.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:24.444+0000 7fd2b2b9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ac16dd50 con 0x7fd2ac142c00 2023-10-06T02:31:24.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:24.444+0000 7fd2b2b9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ac0b0fb0 con 0x7fd2ac0c7f90 2023-10-06T02:31:24.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:24.444+0000 7fd2b1b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2ac06bd20 0x7fd2ac16d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:24.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:24.444+0000 7fd2b1b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2ac06bd20 0x7fd2ac16d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59822/0 (socket says 172.21.15.31:59822) 2023-10-06T02:31:24.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:24.444+0000 7fd2b1b9b700 1 -- 172.21.15.31:0/1350794916 learned_addr learned my addr 172.21.15.31:0/1350794916 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:24.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:24.444+0000 7fd2b239c700 1 --2- 172.21.15.31:0/1350794916 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd2ac142c00 0x7fd2ac0b2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:24.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:24.445+0000 7fd2b139a700 1 --2- 172.21.15.31:0/1350794916 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd2ac0c7f90 0x7fd2ac158f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:24.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:24.445+0000 7fd2b1b9b700 1 -- 172.21.15.31:0/1350794916 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd2ac0c7f90 msgr2=0x7fd2ac158f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:24.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:24.445+0000 7fd2b1b9b700 1 --2- 172.21.15.31:0/1350794916 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd2ac0c7f90 0x7fd2ac158f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:24.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:24.445+0000 7fd2b1b9b700 1 -- 172.21.15.31:0/1350794916 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd2ac142c00 msgr2=0x7fd2ac0b2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:24.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:24.445+0000 7fd2b1b9b700 1 --2- 172.21.15.31:0/1350794916 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd2ac142c00 0x7fd2ac0b2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:24.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:24.445+0000 7fd2b1b9b700 1 -- 172.21.15.31:0/1350794916 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2a4009ae0 con 0x7fd2ac06bd20 2023-10-06T02:31:24.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:24.445+0000 7fd2b239c700 1 --2- 172.21.15.31:0/1350794916 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd2ac142c00 0x7fd2ac0b2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:31:24.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:24.445+0000 7fd2b1b9b700 1 --2- 172.21.15.31:0/1350794916 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2ac06bd20 0x7fd2ac16d810 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7fd2a800e930 tx=0x7fd2a80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:24.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:24.445+0000 7fd2a1ffb700 1 -- 172.21.15.31:0/1350794916 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a800eed0 con 0x7fd2ac06bd20 2023-10-06T02:31:24.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:24.445+0000 7fd2a1ffb700 1 -- 172.21.15.31:0/1350794916 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2a800c850 con 0x7fd2ac06bd20 2023-10-06T02:31:24.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:24.445+0000 7fd2b2b9d700 1 -- 172.21.15.31:0/1350794916 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2ac0b1290 con 0x7fd2ac06bd20 2023-10-06T02:31:24.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:24.445+0000 7fd2a1ffb700 1 -- 172.21.15.31:0/1350794916 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a800cbf0 con 0x7fd2ac06bd20 2023-10-06T02:31:24.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:24.445+0000 7fd2b2b9d700 1 -- 172.21.15.31:0/1350794916 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2ac0b1850 con 0x7fd2ac06bd20 2023-10-06T02:31:24.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:24.447+0000 7fd2b2b9d700 1 -- 172.21.15.31:0/1350794916 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2ac0b2180 con 0x7fd2ac06bd20 2023-10-06T02:31:24.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:24.448+0000 7fd2a1ffb700 1 -- 172.21.15.31:0/1350794916 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 85403+0+0 (secure 0 0 0) 0x7fd2a8013470 con 0x7fd2ac06bd20 2023-10-06T02:31:24.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:24.449+0000 7fd2a1ffb700 1 --2- 172.21.15.31:0/1350794916 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd290067680 0x7fd290069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:24.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:24.449+0000 7fd2a1ffb700 1 -- 172.21.15.31:0/1350794916 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 5604+0+0 (secure 0 0 0) 0x7fd2a8086070 con 0x7fd2ac06bd20 2023-10-06T02:31:24.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:24.449+0000 7fd2b139a700 1 --2- 172.21.15.31:0/1350794916 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd290067680 0x7fd290069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:24.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:24.450+0000 7fd2b139a700 1 --2- 172.21.15.31:0/1350794916 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd290067680 0x7fd290069b30 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac0c4840 tx=0x7fd29c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:24.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:24.452+0000 7fd2a1ffb700 1 -- 172.21.15.31:0/1350794916 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd2a8083610 con 0x7fd2ac06bd20 2023-10-06T02:31:24.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:24.657+0000 7fd2b2b9d700 1 -- 172.21.15.31:0/1350794916 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fd2ac06ad00 con 0x7fd2ac06bd20 2023-10-06T02:31:25.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:25.219+0000 7fd2a1ffb700 1 -- 172.21.15.31:0/1350794916 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v154) v1 ==== 170+0+0 (secure 0 0 0) 0x7fd2a80170d0 con 0x7fd2ac06bd20 2023-10-06T02:31:25.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:25.226+0000 7fd2b2b9d700 1 -- 172.21.15.31:0/1350794916 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd290067680 msgr2=0x7fd290069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:25.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:25.226+0000 7fd2b2b9d700 1 --2- 172.21.15.31:0/1350794916 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd290067680 0x7fd290069b30 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac0c4840 tx=0x7fd29c009040 comp rx=0 tx=0).stop 2023-10-06T02:31:25.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:25.226+0000 7fd2b2b9d700 1 -- 172.21.15.31:0/1350794916 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2ac06bd20 msgr2=0x7fd2ac16d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:25.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:25.226+0000 7fd2b2b9d700 1 --2- 172.21.15.31:0/1350794916 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2ac06bd20 0x7fd2ac16d810 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7fd2a800e930 tx=0x7fd2a80091a0 comp rx=0 tx=0).stop 2023-10-06T02:31:25.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:25.227+0000 7fd2b2b9d700 1 -- 172.21.15.31:0/1350794916 shutdown_connections 2023-10-06T02:31:25.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:25.227+0000 7fd2b2b9d700 1 --2- 172.21.15.31:0/1350794916 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd2ac142c00 0x7fd2ac0b2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:25.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:25.227+0000 7fd2b2b9d700 1 --2- 172.21.15.31:0/1350794916 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2ac06bd20 0x7fd2ac16d810 secure :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7fd2a800e930 tx=0x7fd2a80091a0 comp rx=0 tx=0).stop 2023-10-06T02:31:25.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:25.227+0000 7fd2b2b9d700 1 --2- 172.21.15.31:0/1350794916 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd290067680 0x7fd290069b30 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:25.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:25.227+0000 7fd2b2b9d700 1 --2- 172.21.15.31:0/1350794916 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd2ac0c7f90 0x7fd2ac158f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:25.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:25.227+0000 7fd2b2b9d700 1 -- 172.21.15.31:0/1350794916 >> 172.21.15.31:0/1350794916 conn(0x7fd2ac0675a0 msgr2=0x7fd2ac0c2e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:25.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:25.229+0000 7fd2b2b9d700 1 -- 172.21.15.31:0/1350794916 shutdown_connections 2023-10-06T02:31:25.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:25.230+0000 7fd2b2b9d700 1 -- 172.21.15.31:0/1350794916 wait complete. 2023-10-06T02:31:25.237 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' created 2023-10-06T02:31:25.262 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T02:31:25.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:25.495+0000 7fc10b59e700 1 Processor -- start 2023-10-06T02:31:25.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:25.495+0000 7fc10b59e700 1 -- start start 2023-10-06T02:31:25.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:25.497+0000 7fc10b59e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc10c0c7fb0 0x7fc10c06abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:25.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:25.497+0000 7fc10b59e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc10c13f780 0x7fc10c13fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:25.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:25.497+0000 7fc10b59e700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc10c140120 con 0x7fc10c0c7be0 2023-10-06T02:31:25.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:25.497+0000 7fc10b59e700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc10c06bf10 con 0x7fc10c13f780 2023-10-06T02:31:25.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:25.497+0000 7fc10b59e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc10c06b290 con 0x7fc10c0c7fb0 2023-10-06T02:31:25.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:25.497+0000 7fc10ad9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc10c13f780 0x7fc10c13fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:25.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:25.497+0000 7fc10ad9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc10c13f780 0x7fc10c13fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59824/0 (socket says 172.21.15.31:59824) 2023-10-06T02:31:25.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:25.497+0000 7fc10ad9d700 1 -- 172.21.15.31:0/1518068816 learned_addr learned my addr 172.21.15.31:0/1518068816 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:25.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:25.497+0000 7fc10a59c700 1 --1- 172.21.15.31:0/1518068816 >> v1:172.21.15.31:6789/0 conn(0x7fc10c0c7be0 0x7fc10c15a990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:39484/0 (socket says 172.21.15.31:39484) 2023-10-06T02:31:25.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:25.497+0000 7fc109d9b700 1 --2- 172.21.15.31:0/1518068816 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc10c0c7fb0 0x7fc10c06abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:25.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:25.498+0000 7fc10ad9d700 1 -- 172.21.15.31:0/1518068816 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc10c0c7fb0 msgr2=0x7fc10c06abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:25.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:25.498+0000 7fc10ad9d700 1 --2- 172.21.15.31:0/1518068816 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc10c0c7fb0 0x7fc10c06abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:25.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:25.498+0000 7fc10ad9d700 1 -- 172.21.15.31:0/1518068816 >> v1:172.21.15.31:6789/0 conn(0x7fc10c0c7be0 legacy=0x7fc10c15a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:25.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:25.498+0000 7fc10ad9d700 1 -- 172.21.15.31:0/1518068816 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc10c06b570 con 0x7fc10c13f780 2023-10-06T02:31:25.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:25.498+0000 7fc10ad9d700 1 --2- 172.21.15.31:0/1518068816 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc10c13f780 0x7fc10c13fbe0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7fc104009f20 tx=0x7fc104009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a746183038a646a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:25.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:25.498+0000 7fc10959a700 1 -- 172.21.15.31:0/1518068816 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc10400e070 con 0x7fc10c13f780 2023-10-06T02:31:25.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:25.498+0000 7fc10959a700 1 -- 172.21.15.31:0/1518068816 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc10400b9d0 con 0x7fc10c13f780 2023-10-06T02:31:25.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:25.498+0000 7fc10959a700 1 -- 172.21.15.31:0/1518068816 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1040092e0 con 0x7fc10c13f780 2023-10-06T02:31:25.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:25.499+0000 7fc10b59e700 1 -- 172.21.15.31:0/1518068816 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc10c13f780 msgr2=0x7fc10c13fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:25.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:25.499+0000 7fc10b59e700 1 --2- 172.21.15.31:0/1518068816 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc10c13f780 0x7fc10c13fbe0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7fc104009f20 tx=0x7fc104009840 comp rx=0 tx=0).stop 2023-10-06T02:31:25.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:25.499+0000 7fc10b59e700 1 -- 172.21.15.31:0/1518068816 shutdown_connections 2023-10-06T02:31:25.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:25.499+0000 7fc10b59e700 1 --2- 172.21.15.31:0/1518068816 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc10c13f780 0x7fc10c13fbe0 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:25.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:25.499+0000 7fc10b59e700 1 --2- 172.21.15.31:0/1518068816 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc10c0c7fb0 0x7fc10c06abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:25.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:25.499+0000 7fc10b59e700 1 -- 172.21.15.31:0/1518068816 >> 172.21.15.31:0/1518068816 conn(0x7fc10c06d110 msgr2=0x7fc10c0cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:25.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:25.499+0000 7fc10b59e700 1 -- 172.21.15.31:0/1518068816 shutdown_connections 2023-10-06T02:31:25.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:25.499+0000 7fc10b59e700 1 -- 172.21.15.31:0/1518068816 wait complete. 2023-10-06T02:31:25.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:25.500+0000 7fc10b59e700 1 Processor -- start 2023-10-06T02:31:25.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:25.500+0000 7fc10b59e700 1 -- start start 2023-10-06T02:31:25.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:25.501+0000 7fc10b59e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc10c0c7be0 0x7fc10c155df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:25.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:25.501+0000 7fc10b59e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc10c0c7fb0 0x7fc10c0c53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:25.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:25.501+0000 7fc10b59e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc10c13f780 0x7fc10c0c5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:25.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:25.501+0000 7fc10b59e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc10c157550 con 0x7fc10c13f780 2023-10-06T02:31:25.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:25.501+0000 7fc10b59e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc10c156330 con 0x7fc10c0c7fb0 2023-10-06T02:31:25.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:25.501+0000 7fc10b59e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc10c156c20 con 0x7fc10c0c7be0 2023-10-06T02:31:25.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:25.501+0000 7fc10ad9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc10c13f780 0x7fc10c0c5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:25.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:25.501+0000 7fc10ad9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc10c13f780 0x7fc10c0c5930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59834/0 (socket says 172.21.15.31:59834) 2023-10-06T02:31:25.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:25.501+0000 7fc10ad9d700 1 -- 172.21.15.31:0/3352429532 learned_addr learned my addr 172.21.15.31:0/3352429532 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:25.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:25.502+0000 7fc109d9b700 1 --2- 172.21.15.31:0/3352429532 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc10c0c7fb0 0x7fc10c0c53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:25.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:25.502+0000 7fc10a59c700 1 --2- 172.21.15.31:0/3352429532 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc10c0c7be0 0x7fc10c155df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:25.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:25.502+0000 7fc10ad9d700 1 -- 172.21.15.31:0/3352429532 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc10c0c7be0 msgr2=0x7fc10c155df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:25.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:25.502+0000 7fc10ad9d700 1 --2- 172.21.15.31:0/3352429532 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc10c0c7be0 0x7fc10c155df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:25.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:25.502+0000 7fc10ad9d700 1 -- 172.21.15.31:0/3352429532 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc10c0c7fb0 msgr2=0x7fc10c0c53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:25.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:25.502+0000 7fc10ad9d700 1 --2- 172.21.15.31:0/3352429532 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc10c0c7fb0 0x7fc10c0c53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:25.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:25.502+0000 7fc10ad9d700 1 -- 172.21.15.31:0/3352429532 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc104009ae0 con 0x7fc10c13f780 2023-10-06T02:31:25.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:25.502+0000 7fc10ad9d700 1 --2- 172.21.15.31:0/3352429532 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc10c13f780 0x7fc10c0c5930 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7fc10c0693f0 tx=0x7fc104018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:25.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:25.502+0000 7fc0fa7fc700 1 -- 172.21.15.31:0/3352429532 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc104008b10 con 0x7fc10c13f780 2023-10-06T02:31:25.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:25.503+0000 7fc0fa7fc700 1 -- 172.21.15.31:0/3352429532 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc104008cb0 con 0x7fc10c13f780 2023-10-06T02:31:25.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:25.503+0000 7fc0fa7fc700 1 -- 172.21.15.31:0/3352429532 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc10401c660 con 0x7fc10c13f780 2023-10-06T02:31:25.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:25.503+0000 7fc10b59e700 1 -- 172.21.15.31:0/3352429532 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc10c156f00 con 0x7fc10c13f780 2023-10-06T02:31:25.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:25.503+0000 7fc10b59e700 1 -- 172.21.15.31:0/3352429532 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc10c1579e0 con 0x7fc10c13f780 2023-10-06T02:31:25.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:25.504+0000 7fc10b59e700 1 -- 172.21.15.31:0/3352429532 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc10c1576c0 con 0x7fc10c13f780 2023-10-06T02:31:25.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:25.505+0000 7fc0fa7fc700 1 -- 172.21.15.31:0/3352429532 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 85403+0+0 (secure 0 0 0) 0x7fc104003040 con 0x7fc10c13f780 2023-10-06T02:31:25.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:25.506+0000 7fc0fa7fc700 1 --2- 172.21.15.31:0/3352429532 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc0ec067680 0x7fc0ec069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:25.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:25.506+0000 7fc0fa7fc700 1 -- 172.21.15.31:0/3352429532 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5986+0+0 (secure 0 0 0) 0x7fc104087450 con 0x7fc10c13f780 2023-10-06T02:31:25.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:25.506+0000 7fc10a59c700 1 --2- 172.21.15.31:0/3352429532 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc0ec067680 0x7fc0ec069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:25.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:25.507+0000 7fc10a59c700 1 --2- 172.21.15.31:0/3352429532 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc0ec067680 0x7fc0ec069b30 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fc10c0ca310 tx=0x7fc0fc007eb0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:25.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:25.509+0000 7fc0fa7fc700 1 -- 172.21.15.31:0/3352429532 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc104084b70 con 0x7fc10c13f780 2023-10-06T02:31:25.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:25.754+0000 7fc10b59e700 1 -- 172.21.15.31:0/3352429532 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fc10c04c3c0 con 0x7fc10c13f780 2023-10-06T02:31:26.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.257+0000 7fc0fa7fc700 1 -- 172.21.15.31:0/3352429532 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 25 and data pool 26 v105) v1 ==== 174+0+0 (secure 0 0 0) 0x7fc1040539e0 con 0x7fc10c13f780 2023-10-06T02:31:26.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.264+0000 7fc10b59e700 1 -- 172.21.15.31:0/3352429532 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc0ec067680 msgr2=0x7fc0ec069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:26.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.264+0000 7fc10b59e700 1 --2- 172.21.15.31:0/3352429532 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc0ec067680 0x7fc0ec069b30 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fc10c0ca310 tx=0x7fc0fc007eb0 comp rx=0 tx=0).stop 2023-10-06T02:31:26.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.264+0000 7fc10b59e700 1 -- 172.21.15.31:0/3352429532 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc10c13f780 msgr2=0x7fc10c0c5930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:26.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.264+0000 7fc10b59e700 1 --2- 172.21.15.31:0/3352429532 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc10c13f780 0x7fc10c0c5930 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7fc10c0693f0 tx=0x7fc104018000 comp rx=0 tx=0).stop 2023-10-06T02:31:26.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.265+0000 7fc10b59e700 1 -- 172.21.15.31:0/3352429532 shutdown_connections 2023-10-06T02:31:26.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.265+0000 7fc10b59e700 1 --2- 172.21.15.31:0/3352429532 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc10c13f780 0x7fc10c0c5930 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:26.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.265+0000 7fc10b59e700 1 --2- 172.21.15.31:0/3352429532 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc0ec067680 0x7fc0ec069b30 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:26.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.265+0000 7fc10b59e700 1 --2- 172.21.15.31:0/3352429532 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc10c0c7be0 0x7fc10c155df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:26.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.265+0000 7fc10b59e700 1 --2- 172.21.15.31:0/3352429532 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc10c0c7fb0 0x7fc10c0c53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:26.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.265+0000 7fc10b59e700 1 -- 172.21.15.31:0/3352429532 >> 172.21.15.31:0/3352429532 conn(0x7fc10c06d110 msgr2=0x7fc10c0c0ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:26.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.266+0000 7fc10b59e700 1 -- 172.21.15.31:0/3352429532 shutdown_connections 2023-10-06T02:31:26.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.268+0000 7fc10b59e700 1 -- 172.21.15.31:0/3352429532 wait complete. 2023-10-06T02:31:26.270 INFO:teuthology.orchestra.run.smithi031.stderr:new fs with metadata pool 25 and data pool 26 2023-10-06T02:31:26.291 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:31:26.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.477+0000 7fe4aaf9d700 1 Processor -- start 2023-10-06T02:31:26.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.478+0000 7fe4aaf9d700 1 -- start start 2023-10-06T02:31:26.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.479+0000 7fe4aaf9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe4a413fb20 0x7fe4a415a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:26.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.479+0000 7fe4aaf9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe4a413fef0 0x7fe4a40c7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:26.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.479+0000 7fe4aaf9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe4a406aaa0 0x7fe4a40c7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:26.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.479+0000 7fe4aaf9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4a40c8410 con 0x7fe4a406aaa0 2023-10-06T02:31:26.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.479+0000 7fe4aaf9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4a415ae70 con 0x7fe4a413fb20 2023-10-06T02:31:26.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.479+0000 7fe4aaf9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a406af60 con 0x7fe4a413fef0 2023-10-06T02:31:26.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.479+0000 7fe4aa79c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe4a406aaa0 0x7fe4a40c7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:26.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.479+0000 7fe4aa79c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe4a406aaa0 0x7fe4a40c7ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59836/0 (socket says 172.21.15.31:59836) 2023-10-06T02:31:26.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.479+0000 7fe4aa79c700 1 -- 172.21.15.31:0/3652896064 learned_addr learned my addr 172.21.15.31:0/3652896064 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:26.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.480+0000 7fe4a9f9b700 1 --2- 172.21.15.31:0/3652896064 >> v2:172.21.15.62:3300/0 conn(0x7fe4a413fb20 0x7fe4a415a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:26.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.480+0000 7fe4a979a700 1 --2- 172.21.15.31:0/3652896064 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe4a413fef0 0x7fe4a40c7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:26.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.480+0000 7fe4aa79c700 1 -- 172.21.15.31:0/3652896064 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe4a413fef0 msgr2=0x7fe4a40c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:26.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.480+0000 7fe4aa79c700 1 --2- 172.21.15.31:0/3652896064 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe4a413fef0 0x7fe4a40c7990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:26.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.480+0000 7fe4aa79c700 1 -- 172.21.15.31:0/3652896064 >> v2:172.21.15.62:3300/0 conn(0x7fe4a413fb20 msgr2=0x7fe4a415a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:26.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.480+0000 7fe4aa79c700 1 --2- 172.21.15.31:0/3652896064 >> v2:172.21.15.62:3300/0 conn(0x7fe4a413fb20 0x7fe4a415a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:26.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.480+0000 7fe4aa79c700 1 -- 172.21.15.31:0/3652896064 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4a406b240 con 0x7fe4a406aaa0 2023-10-06T02:31:26.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.480+0000 7fe4a9f9b700 1 --2- 172.21.15.31:0/3652896064 >> v2:172.21.15.62:3300/0 conn(0x7fe4a413fb20 0x7fe4a415a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:31:26.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.480+0000 7fe4aa79c700 1 --2- 172.21.15.31:0/3652896064 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4a406aaa0 0x7fe4a40c7ed0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7fe4a40695b0 tx=0x7fe49c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5a593163c23cf85 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:26.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.480+0000 7fe4a8f99700 1 -- 172.21.15.31:0/3652896064 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe49c00e070 con 0x7fe4a406aaa0 2023-10-06T02:31:26.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.480+0000 7fe4a8f99700 1 -- 172.21.15.31:0/3652896064 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe49c00ba50 con 0x7fe4a406aaa0 2023-10-06T02:31:26.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.480+0000 7fe4a8f99700 1 -- 172.21.15.31:0/3652896064 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe49c0092e0 con 0x7fe4a406aaa0 2023-10-06T02:31:26.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.481+0000 7fe4aaf9d700 1 -- 172.21.15.31:0/3652896064 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4a406aaa0 msgr2=0x7fe4a40c7ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:26.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.481+0000 7fe4aaf9d700 1 --2- 172.21.15.31:0/3652896064 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4a406aaa0 0x7fe4a40c7ed0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7fe4a40695b0 tx=0x7fe49c009840 comp rx=0 tx=0).stop 2023-10-06T02:31:26.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.481+0000 7fe4aaf9d700 1 -- 172.21.15.31:0/3652896064 shutdown_connections 2023-10-06T02:31:26.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.481+0000 7fe4aaf9d700 1 --2- 172.21.15.31:0/3652896064 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4a406aaa0 0x7fe4a40c7ed0 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:26.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.481+0000 7fe4aaf9d700 1 --2- 172.21.15.31:0/3652896064 >> v2:172.21.15.62:3300/0 conn(0x7fe4a413fb20 0x7fe4a415a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:26.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.481+0000 7fe4aaf9d700 1 --2- 172.21.15.31:0/3652896064 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe4a413fef0 0x7fe4a40c7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:26.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.481+0000 7fe4aaf9d700 1 -- 172.21.15.31:0/3652896064 >> 172.21.15.31:0/3652896064 conn(0x7fe4a406d170 msgr2=0x7fe4a40cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:26.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.481+0000 7fe4aaf9d700 1 -- 172.21.15.31:0/3652896064 shutdown_connections 2023-10-06T02:31:26.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.481+0000 7fe4aaf9d700 1 -- 172.21.15.31:0/3652896064 wait complete. 2023-10-06T02:31:26.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.483+0000 7fe4aaf9d700 1 Processor -- start 2023-10-06T02:31:26.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.483+0000 7fe4aaf9d700 1 -- start start 2023-10-06T02:31:26.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.484+0000 7fe4aaf9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe4a406aaa0 0x7fe4a4155cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:26.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.484+0000 7fe4aaf9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4a413fb20 0x7fe4a40c75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:26.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.484+0000 7fe4aaf9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4a413fef0 0x7fe4a40c7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:26.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.484+0000 7fe4aaf9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a415c8c0 con 0x7fe4a413fef0 2023-10-06T02:31:26.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.484+0000 7fe4aaf9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a40c2a00 con 0x7fe4a413fb20 2023-10-06T02:31:26.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.484+0000 7fe4aaf9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a415bc40 con 0x7fe4a406aaa0 2023-10-06T02:31:26.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.484+0000 7fe4aa79c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4a413fef0 0x7fe4a40c7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:26.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.484+0000 7fe4aa79c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4a413fef0 0x7fe4a40c7b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59846/0 (socket says 172.21.15.31:59846) 2023-10-06T02:31:26.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.484+0000 7fe4aa79c700 1 -- 172.21.15.31:0/1807528045 learned_addr learned my addr 172.21.15.31:0/1807528045 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:26.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.484+0000 7fe4a979a700 1 --2- 172.21.15.31:0/1807528045 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4a413fb20 0x7fe4a40c75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:26.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.484+0000 7fe4a9f9b700 1 --2- 172.21.15.31:0/1807528045 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe4a406aaa0 0x7fe4a4155cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:26.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.484+0000 7fe4aa79c700 1 -- 172.21.15.31:0/1807528045 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe4a406aaa0 msgr2=0x7fe4a4155cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:26.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.484+0000 7fe4aa79c700 1 --2- 172.21.15.31:0/1807528045 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe4a406aaa0 0x7fe4a4155cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:26.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.484+0000 7fe4aa79c700 1 -- 172.21.15.31:0/1807528045 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4a413fb20 msgr2=0x7fe4a40c75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:26.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.484+0000 7fe4aa79c700 1 --2- 172.21.15.31:0/1807528045 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4a413fb20 0x7fe4a40c75f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:26.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.484+0000 7fe4aa79c700 1 -- 172.21.15.31:0/1807528045 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe49c009ae0 con 0x7fe4a413fef0 2023-10-06T02:31:26.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.484+0000 7fe4a979a700 1 --2- 172.21.15.31:0/1807528045 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4a413fb20 0x7fe4a40c75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:31:26.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.484+0000 7fe4a9f9b700 1 --2- 172.21.15.31:0/1807528045 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe4a406aaa0 0x7fe4a4155cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:31:26.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.485+0000 7fe4aa79c700 1 --2- 172.21.15.31:0/1807528045 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4a413fef0 0x7fe4a40c7b30 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7fe4a4071320 tx=0x7fe49c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:26.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.485+0000 7fe499ffb700 1 -- 172.21.15.31:0/1807528045 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe49c0032f0 con 0x7fe4a413fef0 2023-10-06T02:31:26.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.485+0000 7fe499ffb700 1 -- 172.21.15.31:0/1807528045 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe49c008e40 con 0x7fe4a413fef0 2023-10-06T02:31:26.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.485+0000 7fe499ffb700 1 -- 172.21.15.31:0/1807528045 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe49c01c660 con 0x7fe4a413fef0 2023-10-06T02:31:26.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.485+0000 7fe4aaf9d700 1 -- 172.21.15.31:0/1807528045 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4a415bf20 con 0x7fe4a413fef0 2023-10-06T02:31:26.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.485+0000 7fe4aaf9d700 1 -- 172.21.15.31:0/1807528045 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4a415c4e0 con 0x7fe4a413fef0 2023-10-06T02:31:26.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.486+0000 7fe4aaf9d700 1 -- 172.21.15.31:0/1807528045 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4a415ca00 con 0x7fe4a413fef0 2023-10-06T02:31:26.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.487+0000 7fe499ffb700 1 -- 172.21.15.31:0/1807528045 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 85403+0+0 (secure 0 0 0) 0x7fe49c003040 con 0x7fe4a413fef0 2023-10-06T02:31:26.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.488+0000 7fe499ffb700 1 --2- 172.21.15.31:0/1807528045 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe488067680 0x7fe488069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:26.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.488+0000 7fe499ffb700 1 -- 172.21.15.31:0/1807528045 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fe49c087200 con 0x7fe4a413fef0 2023-10-06T02:31:26.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.488+0000 7fe4a9f9b700 1 --2- 172.21.15.31:0/1807528045 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe488067680 0x7fe488069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:26.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.489+0000 7fe4a9f9b700 1 --2- 172.21.15.31:0/1807528045 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe488067680 0x7fe488069b30 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fe4a0006f00 tx=0x7fe4a0008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:26.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.492+0000 7fe499ffb700 1 -- 172.21.15.31:0/1807528045 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe4a415ca00 con 0x7fe4a413fef0 2023-10-06T02:31:26.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.694+0000 7fe4aaf9d700 1 -- 172.21.15.31:0/1807528045 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe4a415ca00 con 0x7fe4a413fef0 2023-10-06T02:31:26.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.695+0000 7fe499ffb700 1 -- 172.21.15.31:0/1807528045 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v155) v1 ==== 74+0+16889 (secure 0 0 0) 0x7fe4a415ca00 con 0x7fe4a413fef0 2023-10-06T02:31:26.696 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:31:26.697 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":155,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:31:25.755459+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:31:23.814443+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2023-10-06T02:31:24.697823+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":20,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:31:26.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.697+0000 7fe4aaf9d700 1 -- 172.21.15.31:0/1807528045 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe488067680 msgr2=0x7fe488069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:26.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.697+0000 7fe4aaf9d700 1 --2- 172.21.15.31:0/1807528045 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe488067680 0x7fe488069b30 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fe4a0006f00 tx=0x7fe4a0008040 comp rx=0 tx=0).stop 2023-10-06T02:31:26.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.697+0000 7fe4aaf9d700 1 -- 172.21.15.31:0/1807528045 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4a413fef0 msgr2=0x7fe4a40c7b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:26.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.697+0000 7fe4aaf9d700 1 --2- 172.21.15.31:0/1807528045 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4a413fef0 0x7fe4a40c7b30 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7fe4a4071320 tx=0x7fe49c018000 comp rx=0 tx=0).stop 2023-10-06T02:31:26.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.697+0000 7fe4aaf9d700 1 -- 172.21.15.31:0/1807528045 shutdown_connections 2023-10-06T02:31:26.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.697+0000 7fe4aaf9d700 1 --2- 172.21.15.31:0/1807528045 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4a413fef0 0x7fe4a40c7b30 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:26.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.697+0000 7fe4aaf9d700 1 --2- 172.21.15.31:0/1807528045 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe488067680 0x7fe488069b30 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:26.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.697+0000 7fe4aaf9d700 1 --2- 172.21.15.31:0/1807528045 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe4a406aaa0 0x7fe4a4155cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:26.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.697+0000 7fe4aaf9d700 1 --2- 172.21.15.31:0/1807528045 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4a413fb20 0x7fe4a40c75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:26.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.697+0000 7fe4aaf9d700 1 -- 172.21.15.31:0/1807528045 >> 172.21.15.31:0/1807528045 conn(0x7fe4a406d170 msgr2=0x7fe4a40cb130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:26.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.697+0000 7fe4aaf9d700 1 -- 172.21.15.31:0/1807528045 shutdown_connections 2023-10-06T02:31:26.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.697+0000 7fe4aaf9d700 1 -- 172.21.15.31:0/1807528045 wait complete. 2023-10-06T02:31:26.716 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:31:26.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.902+0000 7feb64d9d700 1 Processor -- start 2023-10-06T02:31:26.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.902+0000 7feb64d9d700 1 -- start start 2023-10-06T02:31:26.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.904+0000 7feb64d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feb6006e0a0 0x7feb600c6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:26.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.904+0000 7feb64d9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7feb60150380 0x7feb6006a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:26.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.904+0000 7feb64d9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb6006a490 con 0x7feb6006f3a0 2023-10-06T02:31:26.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.904+0000 7feb64d9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb600c7490 con 0x7feb60150380 2023-10-06T02:31:26.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.904+0000 7feb64d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb600c7770 con 0x7feb6006e0a0 2023-10-06T02:31:26.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.904+0000 7feb5ffff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7feb60150380 0x7feb6006a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:26.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.904+0000 7feb5ffff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7feb60150380 0x7feb6006a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59862/0 (socket says 172.21.15.31:59862) 2023-10-06T02:31:26.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.904+0000 7feb5ffff700 1 -- 172.21.15.31:0/433459389 learned_addr learned my addr 172.21.15.31:0/433459389 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:26.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.904+0000 7feb5f7fe700 1 --2- 172.21.15.31:0/433459389 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feb6006e0a0 0x7feb600c6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:26.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.905+0000 7feb5ffff700 1 -- 172.21.15.31:0/433459389 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feb6006e0a0 msgr2=0x7feb600c6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:26.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.905+0000 7feb5ffff700 1 --2- 172.21.15.31:0/433459389 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feb6006e0a0 0x7feb600c6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:26.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.905+0000 7feb5ffff700 1 -- 172.21.15.31:0/433459389 >> v1:172.21.15.31:6789/0 conn(0x7feb6006f3a0 legacy=0x7feb6014fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:26.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.905+0000 7feb5ffff700 1 -- 172.21.15.31:0/433459389 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb600c7a50 con 0x7feb60150380 2023-10-06T02:31:26.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.905+0000 7feb5ffff700 1 --2- 172.21.15.31:0/433459389 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb60150380 0x7feb6006a920 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7feb5800c0d0 tx=0x7feb5800a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=114fcdcfd7634189 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:26.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.905+0000 7feb5e7fc700 1 -- 172.21.15.31:0/433459389 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb5800f070 con 0x7feb60150380 2023-10-06T02:31:26.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.905+0000 7feb5e7fc700 1 -- 172.21.15.31:0/433459389 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb5800a2e0 con 0x7feb60150380 2023-10-06T02:31:26.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.905+0000 7feb5e7fc700 1 -- 172.21.15.31:0/433459389 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb5800cd80 con 0x7feb60150380 2023-10-06T02:31:26.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.906+0000 7feb64d9d700 1 -- 172.21.15.31:0/433459389 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb60150380 msgr2=0x7feb6006a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:26.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.906+0000 7feb64d9d700 1 --2- 172.21.15.31:0/433459389 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb60150380 0x7feb6006a920 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7feb5800c0d0 tx=0x7feb5800a950 comp rx=0 tx=0).stop 2023-10-06T02:31:26.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.906+0000 7feb64d9d700 1 -- 172.21.15.31:0/433459389 shutdown_connections 2023-10-06T02:31:26.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.906+0000 7feb64d9d700 1 --2- 172.21.15.31:0/433459389 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb60150380 0x7feb6006a920 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:26.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.906+0000 7feb64d9d700 1 --2- 172.21.15.31:0/433459389 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feb6006e0a0 0x7feb600c6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:26.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.906+0000 7feb64d9d700 1 -- 172.21.15.31:0/433459389 >> 172.21.15.31:0/433459389 conn(0x7feb6006d170 msgr2=0x7feb600cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:26.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.906+0000 7feb64d9d700 1 -- 172.21.15.31:0/433459389 shutdown_connections 2023-10-06T02:31:26.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.906+0000 7feb64d9d700 1 -- 172.21.15.31:0/433459389 wait complete. 2023-10-06T02:31:26.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.907+0000 7feb64d9d700 1 Processor -- start 2023-10-06T02:31:26.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.907+0000 7feb64d9d700 1 -- start start 2023-10-06T02:31:26.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.908+0000 7feb64d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb6006e0a0 0x7feb600c3b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:26.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.908+0000 7feb64d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb6006f3a0 0x7feb600c40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:26.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.908+0000 7feb64d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feb60150380 0x7feb60169530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:26.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.908+0000 7feb64d9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb600b1ba0 con 0x7feb6006e0a0 2023-10-06T02:31:26.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.908+0000 7feb64d9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb6016a6d0 con 0x7feb6006f3a0 2023-10-06T02:31:26.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.908+0000 7feb64d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb6016a9b0 con 0x7feb60150380 2023-10-06T02:31:26.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.909+0000 7feb5f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb6006e0a0 0x7feb600c3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:26.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.909+0000 7feb5f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb6006e0a0 0x7feb600c3b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59872/0 (socket says 172.21.15.31:59872) 2023-10-06T02:31:26.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.909+0000 7feb5f7fe700 1 -- 172.21.15.31:0/3758472857 learned_addr learned my addr 172.21.15.31:0/3758472857 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:26.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.909+0000 7feb5effd700 1 --2- 172.21.15.31:0/3758472857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb6006f3a0 0x7feb600c40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:26.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.909+0000 7feb5ffff700 1 --2- 172.21.15.31:0/3758472857 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feb60150380 0x7feb60169530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:26.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.909+0000 7feb5f7fe700 1 -- 172.21.15.31:0/3758472857 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feb60150380 msgr2=0x7feb60169530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:26.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.909+0000 7feb5f7fe700 1 --2- 172.21.15.31:0/3758472857 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feb60150380 0x7feb60169530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:26.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.909+0000 7feb5f7fe700 1 -- 172.21.15.31:0/3758472857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb6006f3a0 msgr2=0x7feb600c40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:26.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.909+0000 7feb5f7fe700 1 --2- 172.21.15.31:0/3758472857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb6006f3a0 0x7feb600c40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:26.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.909+0000 7feb5f7fe700 1 -- 172.21.15.31:0/3758472857 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb5800abf0 con 0x7feb6006e0a0 2023-10-06T02:31:26.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.909+0000 7feb5effd700 1 --2- 172.21.15.31:0/3758472857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb6006f3a0 0x7feb600c40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:31:26.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.909+0000 7feb5ffff700 1 --2- 172.21.15.31:0/3758472857 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feb60150380 0x7feb60169530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:31:26.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.909+0000 7feb5f7fe700 1 --2- 172.21.15.31:0/3758472857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb6006e0a0 0x7feb600c3b90 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7feb5000ece0 tx=0x7feb500091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:26.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.910+0000 7feb477fe700 1 -- 172.21.15.31:0/3758472857 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb5000c9d0 con 0x7feb6006e0a0 2023-10-06T02:31:26.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.910+0000 7feb477fe700 1 -- 172.21.15.31:0/3758472857 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb5000cb30 con 0x7feb6006e0a0 2023-10-06T02:31:26.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.910+0000 7feb477fe700 1 -- 172.21.15.31:0/3758472857 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb5000ced0 con 0x7feb6006e0a0 2023-10-06T02:31:26.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.910+0000 7feb64d9d700 1 -- 172.21.15.31:0/3758472857 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb600c4610 con 0x7feb6006e0a0 2023-10-06T02:31:26.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.910+0000 7feb64d9d700 1 -- 172.21.15.31:0/3758472857 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb600c4bc0 con 0x7feb6006e0a0 2023-10-06T02:31:26.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.911+0000 7feb64d9d700 1 -- 172.21.15.31:0/3758472857 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb6016ac90 con 0x7feb6006e0a0 2023-10-06T02:31:26.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.912+0000 7feb477fe700 1 -- 172.21.15.31:0/3758472857 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 85403+0+0 (secure 0 0 0) 0x7feb50013680 con 0x7feb6006e0a0 2023-10-06T02:31:26.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.913+0000 7feb477fe700 1 --2- 172.21.15.31:0/3758472857 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7feb4c067680 0x7feb4c069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:26.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.913+0000 7feb477fe700 1 -- 172.21.15.31:0/3758472857 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6086+0+0 (secure 0 0 0) 0x7feb50085c40 con 0x7feb6006e0a0 2023-10-06T02:31:26.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.913+0000 7feb5effd700 1 --2- 172.21.15.31:0/3758472857 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7feb4c067680 0x7feb4c069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:26.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.914+0000 7feb5effd700 1 --2- 172.21.15.31:0/3758472857 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7feb4c067680 0x7feb4c069b30 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7feb600ca250 tx=0x7feb54009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:26.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:26.917+0000 7feb477fe700 1 -- 172.21.15.31:0/3758472857 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7feb50083000 con 0x7feb6006e0a0 2023-10-06T02:31:27.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:27.121+0000 7feb64d9d700 1 -- 172.21.15.31:0/3758472857 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feb6004c3c0 con 0x7feb6006e0a0 2023-10-06T02:31:27.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:27.121+0000 7feb477fe700 1 -- 172.21.15.31:0/3758472857 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v155) v1 ==== 74+0+16889 (secure 0 0 0) 0x7feb50017090 con 0x7feb6006e0a0 2023-10-06T02:31:27.123 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:31:27.123 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":155,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:31:25.755459+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:31:23.814443+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2023-10-06T02:31:24.697823+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":20,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:31:27.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:27.124+0000 7feb64d9d700 1 -- 172.21.15.31:0/3758472857 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7feb4c067680 msgr2=0x7feb4c069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:27.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:27.124+0000 7feb64d9d700 1 --2- 172.21.15.31:0/3758472857 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7feb4c067680 0x7feb4c069b30 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7feb600ca250 tx=0x7feb54009040 comp rx=0 tx=0).stop 2023-10-06T02:31:27.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:27.124+0000 7feb64d9d700 1 -- 172.21.15.31:0/3758472857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb6006e0a0 msgr2=0x7feb600c3b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:27.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:27.124+0000 7feb64d9d700 1 --2- 172.21.15.31:0/3758472857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb6006e0a0 0x7feb600c3b90 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7feb5000ece0 tx=0x7feb500091a0 comp rx=0 tx=0).stop 2023-10-06T02:31:27.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:27.124+0000 7feb64d9d700 1 -- 172.21.15.31:0/3758472857 shutdown_connections 2023-10-06T02:31:27.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:27.124+0000 7feb64d9d700 1 --2- 172.21.15.31:0/3758472857 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7feb4c067680 0x7feb4c069b30 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:27.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:27.124+0000 7feb64d9d700 1 --2- 172.21.15.31:0/3758472857 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feb60150380 0x7feb60169530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:27.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:27.124+0000 7feb64d9d700 1 --2- 172.21.15.31:0/3758472857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb6006e0a0 0x7feb600c3b90 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:27.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:27.124+0000 7feb64d9d700 1 --2- 172.21.15.31:0/3758472857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb6006f3a0 0x7feb600c40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:27.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:27.124+0000 7feb64d9d700 1 -- 172.21.15.31:0/3758472857 >> 172.21.15.31:0/3758472857 conn(0x7feb6006d170 msgr2=0x7feb600cabe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:27.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:27.124+0000 7feb64d9d700 1 -- 172.21.15.31:0/3758472857 shutdown_connections 2023-10-06T02:31:27.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:27.124+0000 7feb64d9d700 1 -- 172.21.15.31:0/3758472857 wait complete. 2023-10-06T02:31:27.143 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T02:31:27.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:27.335+0000 7f9ca09fe700 1 Processor -- start 2023-10-06T02:31:27.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:27.335+0000 7f9ca09fe700 1 -- start start 2023-10-06T02:31:27.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:27.336+0000 7f9ca09fe700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9c9c0c7be0 0x7f9c9c06abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:27.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:27.336+0000 7f9ca09fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9c9c0c7fb0 0x7f9c9c15a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:27.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:27.336+0000 7f9ca09fe700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c9c15aed0 con 0x7f9c9c13f780 2023-10-06T02:31:27.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:27.336+0000 7f9ca09fe700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c9c06bf10 con 0x7f9c9c0c7fb0 2023-10-06T02:31:27.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:27.336+0000 7f9ca09fe700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c9c06b290 con 0x7f9c9c0c7be0 2023-10-06T02:31:27.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:27.336+0000 7f9c9a59c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9c9c0c7fb0 0x7f9c9c15a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:27.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:27.336+0000 7f9c9a59c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9c9c0c7fb0 0x7f9c9c15a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59888/0 (socket says 172.21.15.31:59888) 2023-10-06T02:31:27.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:27.336+0000 7f9c9a59c700 1 -- 172.21.15.31:0/531364619 learned_addr learned my addr 172.21.15.31:0/531364619 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:27.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:27.337+0000 7f9c9ad9d700 1 --2- 172.21.15.31:0/531364619 >> v2:172.21.15.62:3300/0 conn(0x7f9c9c0c7be0 0x7f9c9c06abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:27.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:27.337+0000 7f9c9a59c700 1 -- 172.21.15.31:0/531364619 >> v2:172.21.15.62:3300/0 conn(0x7f9c9c0c7be0 msgr2=0x7f9c9c06abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:27.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:27.337+0000 7f9c9a59c700 1 --2- 172.21.15.31:0/531364619 >> v2:172.21.15.62:3300/0 conn(0x7f9c9c0c7be0 0x7f9c9c06abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:27.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:27.337+0000 7f9c9a59c700 1 -- 172.21.15.31:0/531364619 >> v1:172.21.15.62:6789/0 conn(0x7f9c9c13f780 legacy=0x7f9c9c13fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:27.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:27.337+0000 7f9c9a59c700 1 -- 172.21.15.31:0/531364619 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c9c06b570 con 0x7f9c9c0c7fb0 2023-10-06T02:31:27.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:27.337+0000 7f9c9ad9d700 1 --2- 172.21.15.31:0/531364619 >> v2:172.21.15.62:3300/0 conn(0x7f9c9c0c7be0 0x7f9c9c06abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:31:27.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:27.337+0000 7f9c9a59c700 1 --2- 172.21.15.31:0/531364619 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c9c0c7fb0 0x7f9c9c15a990 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c0c8b00 tx=0x7f9c90009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a40ac1c8ec5a880 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:27.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:27.337+0000 7f9c99d9b700 1 -- 172.21.15.31:0/531364619 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c9000b820 con 0x7f9c9c0c7fb0 2023-10-06T02:31:27.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:27.337+0000 7f9c99d9b700 1 -- 172.21.15.31:0/531364619 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c9000b980 con 0x7f9c9c0c7fb0 2023-10-06T02:31:27.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:27.337+0000 7f9c99d9b700 1 -- 172.21.15.31:0/531364619 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c9000bd20 con 0x7f9c9c0c7fb0 2023-10-06T02:31:27.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:27.338+0000 7f9ca09fe700 1 -- 172.21.15.31:0/531364619 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c9c0c7fb0 msgr2=0x7f9c9c15a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:27.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:27.338+0000 7f9ca09fe700 1 --2- 172.21.15.31:0/531364619 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c9c0c7fb0 0x7f9c9c15a990 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c0c8b00 tx=0x7f9c90009250 comp rx=0 tx=0).stop 2023-10-06T02:31:27.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:27.338+0000 7f9ca09fe700 1 -- 172.21.15.31:0/531364619 shutdown_connections 2023-10-06T02:31:27.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:27.338+0000 7f9ca09fe700 1 --2- 172.21.15.31:0/531364619 >> v2:172.21.15.62:3300/0 conn(0x7f9c9c0c7be0 0x7f9c9c06abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:27.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:27.338+0000 7f9ca09fe700 1 --2- 172.21.15.31:0/531364619 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c9c0c7fb0 0x7f9c9c15a990 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:27.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:27.338+0000 7f9ca09fe700 1 -- 172.21.15.31:0/531364619 >> 172.21.15.31:0/531364619 conn(0x7f9c9c06d110 msgr2=0x7f9c9c0cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:27.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:27.338+0000 7f9ca09fe700 1 -- 172.21.15.31:0/531364619 shutdown_connections 2023-10-06T02:31:27.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:27.338+0000 7f9ca09fe700 1 -- 172.21.15.31:0/531364619 wait complete. 2023-10-06T02:31:27.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:27.340+0000 7f9ca09fe700 1 Processor -- start 2023-10-06T02:31:27.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:27.340+0000 7f9ca09fe700 1 -- start start 2023-10-06T02:31:27.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:27.341+0000 7f9ca09fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c9c0c7be0 0x7f9c9c154f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:27.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:27.341+0000 7f9ca09fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c9c0c7fb0 0x7f9c9c1589b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:27.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:27.341+0000 7f9ca09fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9c9c13f780 0x7f9c9c0c5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:27.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:27.341+0000 7f9ca09fe700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c9c158650 con 0x7f9c9c0c7fb0 2023-10-06T02:31:27.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:27.341+0000 7f9ca09fe700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c9c155490 con 0x7f9c9c0c7be0 2023-10-06T02:31:27.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:27.341+0000 7f9ca09fe700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c9c0c4860 con 0x7f9c9c13f780 2023-10-06T02:31:27.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:27.341+0000 7f9c9a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c9c0c7fb0 0x7f9c9c1589b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:27.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:27.341+0000 7f9c9a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c9c0c7fb0 0x7f9c9c1589b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59898/0 (socket says 172.21.15.31:59898) 2023-10-06T02:31:27.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:27.341+0000 7f9c9a59c700 1 -- 172.21.15.31:0/3503244700 learned_addr learned my addr 172.21.15.31:0/3503244700 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:27.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:27.341+0000 7f9c9ad9d700 1 --2- 172.21.15.31:0/3503244700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c9c0c7be0 0x7f9c9c154f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:27.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:27.341+0000 7f9c9b59e700 1 --2- 172.21.15.31:0/3503244700 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9c9c13f780 0x7f9c9c0c5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:27.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:27.341+0000 7f9c9a59c700 1 -- 172.21.15.31:0/3503244700 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9c9c13f780 msgr2=0x7f9c9c0c5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:27.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:27.341+0000 7f9c9a59c700 1 --2- 172.21.15.31:0/3503244700 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9c9c13f780 0x7f9c9c0c5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:27.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:27.341+0000 7f9c9a59c700 1 -- 172.21.15.31:0/3503244700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c9c0c7be0 msgr2=0x7f9c9c154f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:27.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:27.341+0000 7f9c9a59c700 1 --2- 172.21.15.31:0/3503244700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c9c0c7be0 0x7f9c9c154f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:27.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:27.341+0000 7f9c9a59c700 1 -- 172.21.15.31:0/3503244700 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c900094f0 con 0x7f9c9c0c7fb0 2023-10-06T02:31:27.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:27.341+0000 7f9c9b59e700 1 --2- 172.21.15.31:0/3503244700 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9c9c13f780 0x7f9c9c0c5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:31:27.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:27.342+0000 7f9c9a59c700 1 --2- 172.21.15.31:0/3503244700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c9c0c7fb0 0x7f9c9c1589b0 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f9c90009290 tx=0x7f9c90016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:27.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:27.342+0000 7f9c82ffd700 1 -- 172.21.15.31:0/3503244700 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c90011d70 con 0x7f9c9c0c7fb0 2023-10-06T02:31:27.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:27.342+0000 7f9c82ffd700 1 -- 172.21.15.31:0/3503244700 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c90020070 con 0x7f9c9c0c7fb0 2023-10-06T02:31:27.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:27.342+0000 7f9c82ffd700 1 -- 172.21.15.31:0/3503244700 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c9001b6a0 con 0x7f9c9c0c7fb0 2023-10-06T02:31:27.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:27.342+0000 7f9ca09fe700 1 -- 172.21.15.31:0/3503244700 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c9c0c4b40 con 0x7f9c9c0c7fb0 2023-10-06T02:31:27.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:27.342+0000 7f9ca09fe700 1 -- 172.21.15.31:0/3503244700 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c9c0c5100 con 0x7f9c9c0c7fb0 2023-10-06T02:31:27.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:27.343+0000 7f9ca09fe700 1 -- 172.21.15.31:0/3503244700 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c9c04c3c0 con 0x7f9c9c0c7fb0 2023-10-06T02:31:27.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:27.344+0000 7f9c82ffd700 1 -- 172.21.15.31:0/3503244700 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 85403+0+0 (secure 0 0 0) 0x7f9c90022030 con 0x7f9c9c0c7fb0 2023-10-06T02:31:27.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:27.345+0000 7f9c82ffd700 1 --2- 172.21.15.31:0/3503244700 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9c88067680 0x7f9c88069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:27.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:27.345+0000 7f9c82ffd700 1 -- 172.21.15.31:0/3503244700 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f9c90087420 con 0x7f9c9c0c7fb0 2023-10-06T02:31:27.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:27.346+0000 7f9c9ad9d700 1 --2- 172.21.15.31:0/3503244700 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9c88067680 0x7f9c88069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:27.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:27.346+0000 7f9c9ad9d700 1 --2- 172.21.15.31:0/3503244700 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9c88067680 0x7f9c88069b30 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c156680 tx=0x7f9c8c006f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:27.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:27.349+0000 7f9c82ffd700 1 -- 172.21.15.31:0/3503244700 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9c90084bf0 con 0x7f9c9c0c7fb0 2023-10-06T02:31:27.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:27.598+0000 7f9ca09fe700 1 -- 172.21.15.31:0/3503244700 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f9c9c04c3c0 con 0x7f9c9c0c7fb0 2023-10-06T02:31:28.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.266+0000 7f9c82ffd700 1 -- 172.21.15.31:0/3503244700 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v108) v1 ==== 118+0+0 (secure 0 0 0) 0x7f9c900539e0 con 0x7f9c9c0c7fb0 2023-10-06T02:31:28.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.269+0000 7f9ca09fe700 1 -- 172.21.15.31:0/3503244700 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9c88067680 msgr2=0x7f9c88069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:28.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.269+0000 7f9ca09fe700 1 --2- 172.21.15.31:0/3503244700 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9c88067680 0x7f9c88069b30 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c156680 tx=0x7f9c8c006f90 comp rx=0 tx=0).stop 2023-10-06T02:31:28.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.269+0000 7f9ca09fe700 1 -- 172.21.15.31:0/3503244700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c9c0c7fb0 msgr2=0x7f9c9c1589b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:28.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.269+0000 7f9ca09fe700 1 --2- 172.21.15.31:0/3503244700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c9c0c7fb0 0x7f9c9c1589b0 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f9c90009290 tx=0x7f9c90016000 comp rx=0 tx=0).stop 2023-10-06T02:31:28.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.269+0000 7f9ca09fe700 1 -- 172.21.15.31:0/3503244700 shutdown_connections 2023-10-06T02:31:28.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.269+0000 7f9ca09fe700 1 --2- 172.21.15.31:0/3503244700 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9c88067680 0x7f9c88069b30 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:28.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.269+0000 7f9ca09fe700 1 --2- 172.21.15.31:0/3503244700 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9c9c13f780 0x7f9c9c0c5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:28.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.269+0000 7f9ca09fe700 1 --2- 172.21.15.31:0/3503244700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c9c0c7be0 0x7f9c9c154f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:28.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.269+0000 7f9ca09fe700 1 --2- 172.21.15.31:0/3503244700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c9c0c7fb0 0x7f9c9c1589b0 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:28.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.269+0000 7f9ca09fe700 1 -- 172.21.15.31:0/3503244700 >> 172.21.15.31:0/3503244700 conn(0x7f9c9c06d110 msgr2=0x7f9c9c0c8b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:28.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.270+0000 7f9ca09fe700 1 -- 172.21.15.31:0/3503244700 shutdown_connections 2023-10-06T02:31:28.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.270+0000 7f9ca09fe700 1 -- 172.21.15.31:0/3503244700 wait complete. 2023-10-06T02:31:28.291 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T02:31:28.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.484+0000 7fab4ffff700 1 Processor -- start 2023-10-06T02:31:28.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.484+0000 7fab4ffff700 1 -- start start 2023-10-06T02:31:28.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.485+0000 7fab4ffff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fab50156670 0x7fab50067f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:28.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.486+0000 7fab4ffff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab5006ad00 0x7fab5006b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:28.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.486+0000 7fab4ffff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab500c8060 con 0x7fab50069a00 2023-10-06T02:31:28.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.486+0000 7fab4ffff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab50156a40 con 0x7fab50156670 2023-10-06T02:31:28.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.486+0000 7fab4ffff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab5006b690 con 0x7fab5006ad00 2023-10-06T02:31:28.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.486+0000 7fab4f7fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fab50069a00 0x7fab50069e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58644/0 (socket says 172.21.15.31:58644) 2023-10-06T02:31:28.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.486+0000 7fab4f7fe700 1 -- 172.21.15.31:0/228118724 learned_addr learned my addr 172.21.15.31:0/228118724 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:31:28.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.486+0000 7fab4effd700 1 --2- 172.21.15.31:0/228118724 >> v2:172.21.15.62:3300/0 conn(0x7fab50156670 0x7fab50067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:28.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.486+0000 7fab4e7fc700 1 --2- 172.21.15.31:0/228118724 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab5006ad00 0x7fab5006b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:28.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.486+0000 7fab4dffb700 1 -- 172.21.15.31:0/228118724 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1594041779 0 0) 0x7fab500c8060 con 0x7fab50069a00 2023-10-06T02:31:28.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.486+0000 7fab4dffb700 1 -- 172.21.15.31:0/228118724 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab38003570 con 0x7fab50069a00 2023-10-06T02:31:28.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.487+0000 7fab4dffb700 1 -- 172.21.15.31:0/228118724 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4058079124 0 0) 0x7fab38003570 con 0x7fab50069a00 2023-10-06T02:31:28.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.487+0000 7fab4dffb700 1 -- 172.21.15.31:0/228118724 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab5006ad00 msgr2=0x7fab5006b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:28.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.487+0000 7fab4dffb700 1 --2- 172.21.15.31:0/228118724 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab5006ad00 0x7fab5006b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:28.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.487+0000 7fab4dffb700 1 -- 172.21.15.31:0/228118724 >> v2:172.21.15.62:3300/0 conn(0x7fab50156670 msgr2=0x7fab50067f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:28.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.487+0000 7fab4dffb700 1 --2- 172.21.15.31:0/228118724 >> v2:172.21.15.62:3300/0 conn(0x7fab50156670 0x7fab50067f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:28.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.487+0000 7fab4dffb700 1 -- 172.21.15.31:0/228118724 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab500abaa0 con 0x7fab50069a00 2023-10-06T02:31:28.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.487+0000 7fab4dffb700 1 -- 172.21.15.31:0/228118724 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7fab480017f0 con 0x7fab50069a00 2023-10-06T02:31:28.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.487+0000 7fab4effd700 1 --2- 172.21.15.31:0/228118724 >> v2:172.21.15.62:3300/0 conn(0x7fab50156670 0x7fab50067f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:31:28.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.487+0000 7fab4dffb700 1 -- 172.21.15.31:0/228118724 >> v1:172.21.15.31:6789/0 conn(0x7fab50069a00 legacy=0x7fab50069e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:28.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.487+0000 7fab4dffb700 1 --2- 172.21.15.31:0/228118724 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab5006ad00 0x7fab38003ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:28.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.487+0000 7fab4dffb700 1 --2- 172.21.15.31:0/228118724 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab380040b0 0x7fab38004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:28.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.487+0000 7fab4dffb700 1 --2- 172.21.15.31:0/228118724 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab38004d50 0x7fab380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:28.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.487+0000 7fab4dffb700 1 -- 172.21.15.31:0/228118724 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab50156a40 con 0x7fab38004d50 2023-10-06T02:31:28.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.487+0000 7fab4dffb700 1 -- 172.21.15.31:0/228118724 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5006b690 con 0x7fab380040b0 2023-10-06T02:31:28.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.487+0000 7fab4dffb700 1 -- 172.21.15.31:0/228118724 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab3800dd70 con 0x7fab5006ad00 2023-10-06T02:31:28.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.487+0000 7fab4f7fe700 1 --2- 172.21.15.31:0/228118724 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab38004d50 0x7fab380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:28.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.487+0000 7fab4effd700 1 --2- 172.21.15.31:0/228118724 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab5006ad00 0x7fab38003ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:28.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.487+0000 7fab4e7fc700 1 --2- 172.21.15.31:0/228118724 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab380040b0 0x7fab38004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:28.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.488+0000 7fab4f7fe700 1 -- 172.21.15.31:0/228118724 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab5006ad00 msgr2=0x7fab38003ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:28.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.488+0000 7fab4f7fe700 1 --2- 172.21.15.31:0/228118724 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab5006ad00 0x7fab38003ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:28.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.488+0000 7fab4f7fe700 1 -- 172.21.15.31:0/228118724 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab380040b0 msgr2=0x7fab38004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:28.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.488+0000 7fab4f7fe700 1 --2- 172.21.15.31:0/228118724 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab380040b0 0x7fab38004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:28.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.488+0000 7fab4f7fe700 1 -- 172.21.15.31:0/228118724 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab38003570 con 0x7fab38004d50 2023-10-06T02:31:28.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.488+0000 7fab4f7fe700 1 --2- 172.21.15.31:0/228118724 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab38004d50 0x7fab380051c0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7fab4800a870 tx=0x7fab48002140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a980102e874f33ff server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:28.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.488+0000 7fab4dffb700 1 -- 172.21.15.31:0/228118724 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab4800a230 con 0x7fab38004d50 2023-10-06T02:31:28.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.488+0000 7fab4dffb700 1 -- 172.21.15.31:0/228118724 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab480070f0 con 0x7fab38004d50 2023-10-06T02:31:28.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.488+0000 7fab4dffb700 1 -- 172.21.15.31:0/228118724 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab480074b0 con 0x7fab38004d50 2023-10-06T02:31:28.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.489+0000 7fab4ffff700 1 -- 172.21.15.31:0/228118724 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab38004d50 msgr2=0x7fab380051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:28.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.489+0000 7fab4ffff700 1 --2- 172.21.15.31:0/228118724 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab38004d50 0x7fab380051c0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7fab4800a870 tx=0x7fab48002140 comp rx=0 tx=0).stop 2023-10-06T02:31:28.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.489+0000 7fab4f7fe700 1 -- 172.21.15.31:0/228118724 reap_dead start 2023-10-06T02:31:28.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.489+0000 7fab4ffff700 1 -- 172.21.15.31:0/228118724 shutdown_connections 2023-10-06T02:31:28.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.489+0000 7fab4ffff700 1 -- 172.21.15.31:0/228118724 >> 172.21.15.31:0/228118724 conn(0x7fab500c9600 msgr2=0x7fab500cc0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:28.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.489+0000 7fab4ffff700 1 -- 172.21.15.31:0/228118724 shutdown_connections 2023-10-06T02:31:28.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.489+0000 7fab4ffff700 1 -- 172.21.15.31:0/228118724 wait complete. 2023-10-06T02:31:28.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.490+0000 7fab4ffff700 1 Processor -- start 2023-10-06T02:31:28.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.490+0000 7fab4ffff700 1 -- start start 2023-10-06T02:31:28.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.491+0000 7fab4ffff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab5014d4e0 0x7fab500afe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:28.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.491+0000 7fab4ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab5014d8b0 0x7fab500adc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:28.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.491+0000 7fab4ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab500b03c0 0x7fab500ae170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:28.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.491+0000 7fab4ffff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab501693c0 con 0x7fab5014d8b0 2023-10-06T02:31:28.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.491+0000 7fab4ffff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab500b0940 con 0x7fab500b03c0 2023-10-06T02:31:28.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.491+0000 7fab4ffff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab500b0c20 con 0x7fab5014d4e0 2023-10-06T02:31:28.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.491+0000 7fab4e7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab5014d8b0 0x7fab500adc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:28.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.491+0000 7fab4e7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab5014d8b0 0x7fab500adc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51490/0 (socket says 172.21.15.31:51490) 2023-10-06T02:31:28.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.491+0000 7fab4e7fc700 1 -- 172.21.15.31:0/410458306 learned_addr learned my addr 172.21.15.31:0/410458306 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:28.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.492+0000 7fab4f7fe700 1 --2- 172.21.15.31:0/410458306 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab500b03c0 0x7fab500ae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:28.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.492+0000 7fab4effd700 1 --2- 172.21.15.31:0/410458306 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab5014d4e0 0x7fab500afe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:28.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.492+0000 7fab4e7fc700 1 -- 172.21.15.31:0/410458306 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab5014d4e0 msgr2=0x7fab500afe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:28.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.492+0000 7fab4e7fc700 1 --2- 172.21.15.31:0/410458306 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab5014d4e0 0x7fab500afe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:28.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.492+0000 7fab4e7fc700 1 -- 172.21.15.31:0/410458306 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab500b03c0 msgr2=0x7fab500ae170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:28.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.492+0000 7fab4e7fc700 1 --2- 172.21.15.31:0/410458306 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab500b03c0 0x7fab500ae170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:28.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.492+0000 7fab4e7fc700 1 -- 172.21.15.31:0/410458306 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab500b0f00 con 0x7fab5014d8b0 2023-10-06T02:31:28.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.492+0000 7fab4f7fe700 1 --2- 172.21.15.31:0/410458306 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab500b03c0 0x7fab500ae170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:31:28.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.492+0000 7fab4e7fc700 1 --2- 172.21.15.31:0/410458306 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab5014d8b0 0x7fab500adc30 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fab44006ac0 tx=0x7fab440091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:28.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.492+0000 7fab36ffd700 1 -- 172.21.15.31:0/410458306 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab4400fa10 con 0x7fab5014d8b0 2023-10-06T02:31:28.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.492+0000 7fab36ffd700 1 -- 172.21.15.31:0/410458306 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab4400fb70 con 0x7fab5014d8b0 2023-10-06T02:31:28.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.492+0000 7fab36ffd700 1 -- 172.21.15.31:0/410458306 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab44014440 con 0x7fab5014d8b0 2023-10-06T02:31:28.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.492+0000 7fab4ffff700 1 -- 172.21.15.31:0/410458306 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab50169610 con 0x7fab5014d8b0 2023-10-06T02:31:28.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.492+0000 7fab4ffff700 1 -- 172.21.15.31:0/410458306 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab500bf950 con 0x7fab5014d8b0 2023-10-06T02:31:28.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.493+0000 7fab4ffff700 1 -- 172.21.15.31:0/410458306 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab50068d40 con 0x7fab5014d8b0 2023-10-06T02:31:28.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.494+0000 7fab36ffd700 1 -- 172.21.15.31:0/410458306 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 85403+0+0 (secure 0 0 0) 0x7fab44014870 con 0x7fab5014d8b0 2023-10-06T02:31:28.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.496+0000 7fab36ffd700 1 --2- 172.21.15.31:0/410458306 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fab3c067680 0x7fab3c069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:28.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.496+0000 7fab36ffd700 1 -- 172.21.15.31:0/410458306 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fab44018030 con 0x7fab5014d8b0 2023-10-06T02:31:28.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.496+0000 7fab4effd700 1 --2- 172.21.15.31:0/410458306 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fab3c067680 0x7fab3c069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:28.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.497+0000 7fab4effd700 1 --2- 172.21.15.31:0/410458306 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fab3c067680 0x7fab3c069b30 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fab40006f00 tx=0x7fab40008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:28.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.499+0000 7fab36ffd700 1 -- 172.21.15.31:0/410458306 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fab440836d0 con 0x7fab5014d8b0 2023-10-06T02:31:28.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:28.748+0000 7fab4ffff700 1 -- 172.21.15.31:0/410458306 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fab500c7450 con 0x7fab5014d8b0 2023-10-06T02:31:29.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.272+0000 7fab36ffd700 1 -- 172.21.15.31:0/410458306 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v109) v1 ==== 122+0+0 (secure 0 0 0) 0x7fab44022070 con 0x7fab5014d8b0 2023-10-06T02:31:29.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.274+0000 7fab4ffff700 1 -- 172.21.15.31:0/410458306 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fab3c067680 msgr2=0x7fab3c069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:29.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.274+0000 7fab4ffff700 1 --2- 172.21.15.31:0/410458306 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fab3c067680 0x7fab3c069b30 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fab40006f00 tx=0x7fab40008040 comp rx=0 tx=0).stop 2023-10-06T02:31:29.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.274+0000 7fab4ffff700 1 -- 172.21.15.31:0/410458306 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab5014d8b0 msgr2=0x7fab500adc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:29.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.274+0000 7fab4ffff700 1 --2- 172.21.15.31:0/410458306 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab5014d8b0 0x7fab500adc30 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fab44006ac0 tx=0x7fab440091a0 comp rx=0 tx=0).stop 2023-10-06T02:31:29.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.275+0000 7fab4ffff700 1 -- 172.21.15.31:0/410458306 shutdown_connections 2023-10-06T02:31:29.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.275+0000 7fab4ffff700 1 --2- 172.21.15.31:0/410458306 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab500b03c0 0x7fab500ae170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:29.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.275+0000 7fab4ffff700 1 --2- 172.21.15.31:0/410458306 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fab3c067680 0x7fab3c069b30 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:29.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.275+0000 7fab4ffff700 1 --2- 172.21.15.31:0/410458306 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab5014d4e0 0x7fab500afe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:29.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.275+0000 7fab4ffff700 1 --2- 172.21.15.31:0/410458306 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab5014d8b0 0x7fab500adc30 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:29.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.275+0000 7fab4ffff700 1 -- 172.21.15.31:0/410458306 >> 172.21.15.31:0/410458306 conn(0x7fab500c9600 msgr2=0x7fab50155aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:29.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.275+0000 7fab4ffff700 1 -- 172.21.15.31:0/410458306 shutdown_connections 2023-10-06T02:31:29.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.275+0000 7fab4ffff700 1 -- 172.21.15.31:0/410458306 wait complete. 2023-10-06T02:31:29.295 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:31:29.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.483+0000 7fbbb359e700 1 Processor -- start 2023-10-06T02:31:29.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.483+0000 7fbbb359e700 1 -- start start 2023-10-06T02:31:29.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.484+0000 7fbbb359e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbbb415de50 0x7fbbb40cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:29.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.484+0000 7fbbb359e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fbbb40cef40 0x7fbbb415c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:29.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.484+0000 7fbbb359e700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbb415afd0 con 0x7fbbb4153c30 2023-10-06T02:31:29.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.484+0000 7fbbb359e700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbb415b2b0 con 0x7fbbb40cef40 2023-10-06T02:31:29.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.484+0000 7fbbb359e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb415b590 con 0x7fbbb415de50 2023-10-06T02:31:29.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.484+0000 7fbbb2d9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fbbb40cef40 0x7fbbb415c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:29.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.485+0000 7fbbb2d9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fbbb40cef40 0x7fbbb415c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51498/0 (socket says 172.21.15.31:51498) 2023-10-06T02:31:29.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.485+0000 7fbbb2d9d700 1 -- 172.21.15.31:0/1861722456 learned_addr learned my addr 172.21.15.31:0/1861722456 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:29.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.485+0000 7fbbb1d9b700 1 --2- 172.21.15.31:0/1861722456 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbbb415de50 0x7fbbb40cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:29.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.485+0000 7fbbb2d9d700 1 -- 172.21.15.31:0/1861722456 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbbb415de50 msgr2=0x7fbbb40cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:29.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.485+0000 7fbbb2d9d700 1 --2- 172.21.15.31:0/1861722456 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbbb415de50 0x7fbbb40cea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:29.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.485+0000 7fbbb2d9d700 1 -- 172.21.15.31:0/1861722456 >> v1:172.21.15.62:6789/0 conn(0x7fbbb4153c30 legacy=0x7fbbb415a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:29.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.485+0000 7fbbb2d9d700 1 -- 172.21.15.31:0/1861722456 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbb415b870 con 0x7fbbb40cef40 2023-10-06T02:31:29.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.485+0000 7fbbb159a700 1 -- 172.21.15.31:0/1861722456 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 888003290 0 0) 0x7fbbb415afd0 con 0x7fbbb4153c30 2023-10-06T02:31:29.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.485+0000 7fbbb2d9d700 1 --2- 172.21.15.31:0/1861722456 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbbb40cef40 0x7fbbb415c070 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fbbac009f20 tx=0x7fbbac009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e46b77170504ee16 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:29.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.486+0000 7fbbb159a700 1 -- 172.21.15.31:0/1861722456 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbac00e070 con 0x7fbbb40cef40 2023-10-06T02:31:29.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.486+0000 7fbbb159a700 1 -- 172.21.15.31:0/1861722456 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbac00b9d0 con 0x7fbbb40cef40 2023-10-06T02:31:29.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.486+0000 7fbbb159a700 1 -- 172.21.15.31:0/1861722456 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbac0092e0 con 0x7fbbb40cef40 2023-10-06T02:31:29.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.486+0000 7fbbb359e700 1 -- 172.21.15.31:0/1861722456 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbbb40cef40 msgr2=0x7fbbb415c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:29.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.486+0000 7fbbb359e700 1 --2- 172.21.15.31:0/1861722456 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbbb40cef40 0x7fbbb415c070 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fbbac009f20 tx=0x7fbbac009840 comp rx=0 tx=0).stop 2023-10-06T02:31:29.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.486+0000 7fbbb359e700 1 -- 172.21.15.31:0/1861722456 shutdown_connections 2023-10-06T02:31:29.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.486+0000 7fbbb359e700 1 --2- 172.21.15.31:0/1861722456 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbbb40cef40 0x7fbbb415c070 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:29.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.486+0000 7fbbb359e700 1 --2- 172.21.15.31:0/1861722456 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbbb415de50 0x7fbbb40cea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:29.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.486+0000 7fbbb359e700 1 -- 172.21.15.31:0/1861722456 >> 172.21.15.31:0/1861722456 conn(0x7fbbb40703a0 msgr2=0x7fbbb40707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:29.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.486+0000 7fbbb359e700 1 -- 172.21.15.31:0/1861722456 shutdown_connections 2023-10-06T02:31:29.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.486+0000 7fbbb359e700 1 -- 172.21.15.31:0/1861722456 wait complete. 2023-10-06T02:31:29.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.488+0000 7fbbb359e700 1 Processor -- start 2023-10-06T02:31:29.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.488+0000 7fbbb359e700 1 -- start start 2023-10-06T02:31:29.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.489+0000 7fbbb359e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbb40cef40 0x7fbbb4151ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:29.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.489+0000 7fbbb359e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbbb4153c30 0x7fbbb40c8f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:29.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.489+0000 7fbbb359e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbbb415de50 0x7fbbb40b1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:29.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.489+0000 7fbbb359e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb40b44f0 con 0x7fbbb4153c30 2023-10-06T02:31:29.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.489+0000 7fbbb359e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb40b4230 con 0x7fbbb40cef40 2023-10-06T02:31:29.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.489+0000 7fbbb359e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb40c97c0 con 0x7fbbb415de50 2023-10-06T02:31:29.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.489+0000 7fbbb1d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbbb4153c30 0x7fbbb40c8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:29.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.489+0000 7fbbb1d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbbb4153c30 0x7fbbb40c8f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51512/0 (socket says 172.21.15.31:51512) 2023-10-06T02:31:29.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.489+0000 7fbbb1d9b700 1 -- 172.21.15.31:0/2108717509 learned_addr learned my addr 172.21.15.31:0/2108717509 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:29.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.489+0000 7fbbb2d9d700 1 --2- 172.21.15.31:0/2108717509 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbbb415de50 0x7fbbb40b1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:29.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.489+0000 7fbbb259c700 1 --2- 172.21.15.31:0/2108717509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbb40cef40 0x7fbbb4151ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:29.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.490+0000 7fbbb1d9b700 1 -- 172.21.15.31:0/2108717509 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbbb415de50 msgr2=0x7fbbb40b1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:29.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.490+0000 7fbbb1d9b700 1 --2- 172.21.15.31:0/2108717509 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbbb415de50 0x7fbbb40b1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:29.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.490+0000 7fbbb1d9b700 1 -- 172.21.15.31:0/2108717509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbb40cef40 msgr2=0x7fbbb4151ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:29.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.490+0000 7fbbb1d9b700 1 --2- 172.21.15.31:0/2108717509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbb40cef40 0x7fbbb4151ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:29.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.490+0000 7fbbb1d9b700 1 -- 172.21.15.31:0/2108717509 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbac009ae0 con 0x7fbbb4153c30 2023-10-06T02:31:29.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.490+0000 7fbbb1d9b700 1 --2- 172.21.15.31:0/2108717509 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbbb4153c30 0x7fbbb40c8f90 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fbba800ecb0 tx=0x7fbba80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:29.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.490+0000 7fbba27fc700 1 -- 172.21.15.31:0/2108717509 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba800c9d0 con 0x7fbbb4153c30 2023-10-06T02:31:29.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.490+0000 7fbba27fc700 1 -- 172.21.15.31:0/2108717509 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbba800cb50 con 0x7fbbb4153c30 2023-10-06T02:31:29.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.490+0000 7fbbb359e700 1 -- 172.21.15.31:0/2108717509 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbb40c9aa0 con 0x7fbbb4153c30 2023-10-06T02:31:29.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.490+0000 7fbba27fc700 1 -- 172.21.15.31:0/2108717509 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba8013470 con 0x7fbbb4153c30 2023-10-06T02:31:29.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.490+0000 7fbbb359e700 1 -- 172.21.15.31:0/2108717509 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbb4152420 con 0x7fbbb4153c30 2023-10-06T02:31:29.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.491+0000 7fbbb359e700 1 -- 172.21.15.31:0/2108717509 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbb40b4660 con 0x7fbbb4153c30 2023-10-06T02:31:29.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.492+0000 7fbba27fc700 1 -- 172.21.15.31:0/2108717509 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 85403+0+0 (secure 0 0 0) 0x7fbba8013860 con 0x7fbbb4153c30 2023-10-06T02:31:29.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.493+0000 7fbba27fc700 1 --2- 172.21.15.31:0/2108717509 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fbb94067680 0x7fbb94069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:29.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.493+0000 7fbba27fc700 1 -- 172.21.15.31:0/2108717509 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fbba8085c10 con 0x7fbbb4153c30 2023-10-06T02:31:29.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.494+0000 7fbbb259c700 1 --2- 172.21.15.31:0/2108717509 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fbb94067680 0x7fbb94069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:29.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.494+0000 7fbbb259c700 1 --2- 172.21.15.31:0/2108717509 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fbb94067680 0x7fbb94069b30 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fbba40011e0 tx=0x7fbba4009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:29.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.497+0000 7fbba27fc700 1 -- 172.21.15.31:0/2108717509 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbbb40b4660 con 0x7fbbb4153c30 2023-10-06T02:31:29.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.747+0000 7fbbb359e700 1 -- 172.21.15.31:0/2108717509 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbbb40b4630 con 0x7fbbb4153c30 2023-10-06T02:31:29.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.747+0000 7fbba27fc700 1 -- 172.21.15.31:0/2108717509 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 109 v109) v1 ==== 95+0+4604 (secure 0 0 0) 0x7fbba8017090 con 0x7fbbb4153c30 2023-10-06T02:31:29.749 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:31:29.749 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":109,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7178,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/3218480514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3218480514},{"type":"v1","addr":"172.21.15.62:6834","nonce":3218480514}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104},{"gid":8323,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6833/832192876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":832192876},{"type":"v1","addr":"172.21.15.31:6833","nonce":832192876}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104},{"gid":8877,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/2374603733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":2374603733},{"type":"v1","addr":"172.21.15.31:6835","nonce":2374603733}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104}],"filesystems":[{"mdsmap":{"epoch":109,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:31:25.755801+0000","modified":"2023-10-06T02:31:29.266902+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8889},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8889":{"gid":8889,"name":"d","rank":0,"incarnation":105,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/1184425321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":1184425321},{"type":"v1","addr":"172.21.15.62:6835","nonce":1184425321}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2023-10-06T02:31:29.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.750+0000 7fbbb359e700 1 -- 172.21.15.31:0/2108717509 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fbb94067680 msgr2=0x7fbb94069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:29.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.750+0000 7fbbb359e700 1 --2- 172.21.15.31:0/2108717509 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fbb94067680 0x7fbb94069b30 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fbba40011e0 tx=0x7fbba4009040 comp rx=0 tx=0).stop 2023-10-06T02:31:29.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.750+0000 7fbbb359e700 1 -- 172.21.15.31:0/2108717509 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbbb4153c30 msgr2=0x7fbbb40c8f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:29.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.750+0000 7fbbb359e700 1 --2- 172.21.15.31:0/2108717509 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbbb4153c30 0x7fbbb40c8f90 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fbba800ecb0 tx=0x7fbba80091a0 comp rx=0 tx=0).stop 2023-10-06T02:31:29.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.750+0000 7fbbb359e700 1 -- 172.21.15.31:0/2108717509 shutdown_connections 2023-10-06T02:31:29.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.750+0000 7fbbb359e700 1 --2- 172.21.15.31:0/2108717509 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fbb94067680 0x7fbb94069b30 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:29.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.750+0000 7fbbb359e700 1 --2- 172.21.15.31:0/2108717509 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbbb415de50 0x7fbbb40b1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:29.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.750+0000 7fbbb359e700 1 --2- 172.21.15.31:0/2108717509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbb40cef40 0x7fbbb4151ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:29.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.750+0000 7fbbb359e700 1 --2- 172.21.15.31:0/2108717509 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbbb4153c30 0x7fbbb40c8f90 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:29.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.750+0000 7fbbb359e700 1 -- 172.21.15.31:0/2108717509 >> 172.21.15.31:0/2108717509 conn(0x7fbbb40703a0 msgr2=0x7fbbb40c6d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:29.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.750+0000 7fbbb359e700 1 -- 172.21.15.31:0/2108717509 shutdown_connections 2023-10-06T02:31:29.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.750+0000 7fbbb359e700 1 -- 172.21.15.31:0/2108717509 wait complete. 2023-10-06T02:31:29.753 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 109 2023-10-06T02:31:29.770 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:31:29.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.954+0000 7fdc83fff700 1 Processor -- start 2023-10-06T02:31:29.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.954+0000 7fdc83fff700 1 -- start start 2023-10-06T02:31:29.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.955+0000 7fdc83fff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fdc8413fef0 0x7fdc8415a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:29.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.956+0000 7fdc83fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fdc8406aaa0 0x7fdc840c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:29.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.956+0000 7fdc83fff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc8415ae70 con 0x7fdc8413fb20 2023-10-06T02:31:29.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.956+0000 7fdc83fff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc8406af60 con 0x7fdc8406aaa0 2023-10-06T02:31:29.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.956+0000 7fdc83fff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc8406b240 con 0x7fdc8413fef0 2023-10-06T02:31:29.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.956+0000 7fdc837fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fdc8406aaa0 0x7fdc840c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:29.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.956+0000 7fdc827fc700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fdc8413fef0 0x7fdc8415a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:29.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.956+0000 7fdc837fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fdc8406aaa0 0x7fdc840c9d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51526/0 (socket says 172.21.15.31:51526) 2023-10-06T02:31:29.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.956+0000 7fdc837fe700 1 -- 172.21.15.31:0/3184128181 learned_addr learned my addr 172.21.15.31:0/3184128181 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:29.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.956+0000 7fdc837fe700 1 -- 172.21.15.31:0/3184128181 >> v2:172.21.15.62:3300/0 conn(0x7fdc8413fef0 msgr2=0x7fdc8415a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:29.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.956+0000 7fdc837fe700 1 --2- 172.21.15.31:0/3184128181 >> v2:172.21.15.62:3300/0 conn(0x7fdc8413fef0 0x7fdc8415a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:29.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.956+0000 7fdc837fe700 1 -- 172.21.15.31:0/3184128181 >> v1:172.21.15.62:6789/0 conn(0x7fdc8413fb20 legacy=0x7fdc840c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:29.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.956+0000 7fdc837fe700 1 -- 172.21.15.31:0/3184128181 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc8406b520 con 0x7fdc8406aaa0 2023-10-06T02:31:29.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.957+0000 7fdc837fe700 1 --2- 172.21.15.31:0/3184128181 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc8406aaa0 0x7fdc840c9d00 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7fdc8406a5e0 tx=0x7fdc7c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=514e4eff1310acb5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:29.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.957+0000 7fdc81ffb700 1 -- 172.21.15.31:0/3184128181 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc7c00e070 con 0x7fdc8406aaa0 2023-10-06T02:31:29.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.957+0000 7fdc81ffb700 1 -- 172.21.15.31:0/3184128181 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc7c00ba20 con 0x7fdc8406aaa0 2023-10-06T02:31:29.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.957+0000 7fdc81ffb700 1 -- 172.21.15.31:0/3184128181 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc7c0092e0 con 0x7fdc8406aaa0 2023-10-06T02:31:29.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.957+0000 7fdc83fff700 1 -- 172.21.15.31:0/3184128181 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc8406aaa0 msgr2=0x7fdc840c9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:29.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.957+0000 7fdc83fff700 1 --2- 172.21.15.31:0/3184128181 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc8406aaa0 0x7fdc840c9d00 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7fdc8406a5e0 tx=0x7fdc7c009840 comp rx=0 tx=0).stop 2023-10-06T02:31:29.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.957+0000 7fdc83fff700 1 -- 172.21.15.31:0/3184128181 shutdown_connections 2023-10-06T02:31:29.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.957+0000 7fdc83fff700 1 --2- 172.21.15.31:0/3184128181 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc8406aaa0 0x7fdc840c9d00 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:29.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.957+0000 7fdc83fff700 1 --2- 172.21.15.31:0/3184128181 >> v2:172.21.15.62:3300/0 conn(0x7fdc8413fef0 0x7fdc8415a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:29.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.958+0000 7fdc83fff700 1 -- 172.21.15.31:0/3184128181 >> 172.21.15.31:0/3184128181 conn(0x7fdc8406d170 msgr2=0x7fdc840cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:29.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.958+0000 7fdc83fff700 1 -- 172.21.15.31:0/3184128181 shutdown_connections 2023-10-06T02:31:29.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.958+0000 7fdc83fff700 1 -- 172.21.15.31:0/3184128181 wait complete. 2023-10-06T02:31:29.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.959+0000 7fdc83fff700 1 Processor -- start 2023-10-06T02:31:29.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.959+0000 7fdc83fff700 1 -- start start 2023-10-06T02:31:29.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.960+0000 7fdc83fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdc8406aaa0 0x7fdc84158540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:29.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.960+0000 7fdc83fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc8413fb20 0x7fdc84155bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:29.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.960+0000 7fdc83fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc8413fef0 0x7fdc84156110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:29.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.960+0000 7fdc83fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc84136ab0 con 0x7fdc8413fef0 2023-10-06T02:31:29.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.960+0000 7fdc83fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc840ad160 con 0x7fdc8413fb20 2023-10-06T02:31:29.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.960+0000 7fdc83fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc840ad440 con 0x7fdc8406aaa0 2023-10-06T02:31:29.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.960+0000 7fdc837fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc8413fef0 0x7fdc84156110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:29.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.961+0000 7fdc837fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc8413fef0 0x7fdc84156110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51534/0 (socket says 172.21.15.31:51534) 2023-10-06T02:31:29.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.961+0000 7fdc82ffd700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdc8406aaa0 0x7fdc84158540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:29.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.961+0000 7fdc837fe700 1 -- 172.21.15.31:0/4207572907 learned_addr learned my addr 172.21.15.31:0/4207572907 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:29.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.961+0000 7fdc827fc700 1 --2- 172.21.15.31:0/4207572907 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc8413fb20 0x7fdc84155bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:29.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.961+0000 7fdc837fe700 1 -- 172.21.15.31:0/4207572907 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdc8406aaa0 msgr2=0x7fdc84158540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:29.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.961+0000 7fdc837fe700 1 --2- 172.21.15.31:0/4207572907 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdc8406aaa0 0x7fdc84158540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:29.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.961+0000 7fdc837fe700 1 -- 172.21.15.31:0/4207572907 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc8413fb20 msgr2=0x7fdc84155bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:29.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.961+0000 7fdc837fe700 1 --2- 172.21.15.31:0/4207572907 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc8413fb20 0x7fdc84155bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:29.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.961+0000 7fdc837fe700 1 -- 172.21.15.31:0/4207572907 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc7c009ae0 con 0x7fdc8413fef0 2023-10-06T02:31:29.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.961+0000 7fdc837fe700 1 --2- 172.21.15.31:0/4207572907 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc8413fef0 0x7fdc84156110 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c009880 tx=0x7fdc7c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:29.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.961+0000 7fdc6affd700 1 -- 172.21.15.31:0/4207572907 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc7c008a80 con 0x7fdc8413fef0 2023-10-06T02:31:29.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.961+0000 7fdc6affd700 1 -- 172.21.15.31:0/4207572907 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc7c008be0 con 0x7fdc8413fef0 2023-10-06T02:31:29.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.961+0000 7fdc6affd700 1 -- 172.21.15.31:0/4207572907 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc7c026460 con 0x7fdc8413fef0 2023-10-06T02:31:29.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.961+0000 7fdc83fff700 1 -- 172.21.15.31:0/4207572907 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc840ad720 con 0x7fdc8413fef0 2023-10-06T02:31:29.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.962+0000 7fdc83fff700 1 -- 172.21.15.31:0/4207572907 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc84158a80 con 0x7fdc8413fef0 2023-10-06T02:31:29.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.963+0000 7fdc83fff700 1 -- 172.21.15.31:0/4207572907 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc840aceb0 con 0x7fdc8413fef0 2023-10-06T02:31:29.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.963+0000 7fdc6affd700 1 -- 172.21.15.31:0/4207572907 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 85403+0+0 (secure 0 0 0) 0x7fdc7c003040 con 0x7fdc8413fef0 2023-10-06T02:31:29.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.964+0000 7fdc6affd700 1 --2- 172.21.15.31:0/4207572907 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fdc70067680 0x7fdc70069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:29.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.965+0000 7fdc6affd700 1 -- 172.21.15.31:0/4207572907 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fdc7c087750 con 0x7fdc8413fef0 2023-10-06T02:31:29.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.965+0000 7fdc82ffd700 1 --2- 172.21.15.31:0/4207572907 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fdc70067680 0x7fdc70069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:29.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.966+0000 7fdc82ffd700 1 --2- 172.21.15.31:0/4207572907 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fdc70067680 0x7fdc70069b30 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fdc840c3030 tx=0x7fdc74009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:29.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:29.969+0000 7fdc6affd700 1 -- 172.21.15.31:0/4207572907 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdc7c084a00 con 0x7fdc8413fef0 2023-10-06T02:31:30.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.173+0000 7fdc83fff700 1 -- 172.21.15.31:0/4207572907 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdc8404c3c0 con 0x7fdc8413fef0 2023-10-06T02:31:30.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.174+0000 7fdc6affd700 1 -- 172.21.15.31:0/4207572907 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v156) v1 ==== 74+0+16876 (secure 0 0 0) 0x7fdc7c0537f0 con 0x7fdc8413fef0 2023-10-06T02:31:30.176 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:31:30.176 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":156,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:31:27.235515+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:31:23.814443+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2023-10-06T02:31:24.697823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"156","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":20,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:31:30.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.176+0000 7fdc83fff700 1 -- 172.21.15.31:0/4207572907 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fdc70067680 msgr2=0x7fdc70069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:30.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.176+0000 7fdc83fff700 1 --2- 172.21.15.31:0/4207572907 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fdc70067680 0x7fdc70069b30 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fdc840c3030 tx=0x7fdc74009040 comp rx=0 tx=0).stop 2023-10-06T02:31:30.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.176+0000 7fdc83fff700 1 -- 172.21.15.31:0/4207572907 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc8413fef0 msgr2=0x7fdc84156110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:30.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.176+0000 7fdc83fff700 1 --2- 172.21.15.31:0/4207572907 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc8413fef0 0x7fdc84156110 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c009880 tx=0x7fdc7c018000 comp rx=0 tx=0).stop 2023-10-06T02:31:30.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.177+0000 7fdc83fff700 1 -- 172.21.15.31:0/4207572907 shutdown_connections 2023-10-06T02:31:30.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.177+0000 7fdc83fff700 1 --2- 172.21.15.31:0/4207572907 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc8413fef0 0x7fdc84156110 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:30.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.177+0000 7fdc83fff700 1 --2- 172.21.15.31:0/4207572907 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fdc70067680 0x7fdc70069b30 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:30.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.177+0000 7fdc83fff700 1 --2- 172.21.15.31:0/4207572907 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdc8406aaa0 0x7fdc84158540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:30.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.177+0000 7fdc83fff700 1 --2- 172.21.15.31:0/4207572907 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc8413fb20 0x7fdc84155bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:30.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.177+0000 7fdc83fff700 1 -- 172.21.15.31:0/4207572907 >> 172.21.15.31:0/4207572907 conn(0x7fdc8406d170 msgr2=0x7fdc840c9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:30.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.177+0000 7fdc83fff700 1 -- 172.21.15.31:0/4207572907 shutdown_connections 2023-10-06T02:31:30.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.177+0000 7fdc83fff700 1 -- 172.21.15.31:0/4207572907 wait complete. 2023-10-06T02:31:30.196 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T02:31:30.196 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T02:31:30.206 INFO:tasks.ceph.mds.d.smithi062.stderr:2023-10-06T02:31:30.205+0000 7f30d4bf1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T02:31:30.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.382+0000 7f56aef9d700 1 Processor -- start 2023-10-06T02:31:30.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.382+0000 7f56aef9d700 1 -- start start 2023-10-06T02:31:30.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.383+0000 7f56aef9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f56a8142c00 0x7f56a806b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:30.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.383+0000 7f56aef9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56a80ca480 con 0x7f56a80c7f90 2023-10-06T02:31:30.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.383+0000 7f56aef9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56a815e240 con 0x7f56a806bd20 2023-10-06T02:31:30.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.383+0000 7f56aef9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56a806b350 con 0x7f56a8142c00 2023-10-06T02:31:30.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.384+0000 7f56ae79c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f56a80c7f90 0x7f56a806c6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58658/0 (socket says 172.21.15.31:58658) 2023-10-06T02:31:30.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.384+0000 7f56ae79c700 1 -- 172.21.15.31:0/1682842838 learned_addr learned my addr 172.21.15.31:0/1682842838 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:31:30.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.384+0000 7f56adf9b700 1 --2- 172.21.15.31:0/1682842838 >> v2:172.21.15.62:3300/0 conn(0x7f56a8142c00 0x7f56a806b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:30.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.384+0000 7f56acf99700 1 -- 172.21.15.31:0/1682842838 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1584695974 0 0) 0x7f56a80ca480 con 0x7f56a80c7f90 2023-10-06T02:31:30.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.384+0000 7f56acf99700 1 -- 172.21.15.31:0/1682842838 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5694003570 con 0x7f56a80c7f90 2023-10-06T02:31:30.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.384+0000 7f56acf99700 1 -- 172.21.15.31:0/1682842838 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1344953835 0 0) 0x7f56a815e240 con 0x7f56a806bd20 2023-10-06T02:31:30.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.384+0000 7f56acf99700 1 -- 172.21.15.31:0/1682842838 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56a80ca480 con 0x7f56a806bd20 2023-10-06T02:31:30.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.385+0000 7f56acf99700 1 -- 172.21.15.31:0/1682842838 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 912522768 0 0) 0x7f5694003570 con 0x7f56a80c7f90 2023-10-06T02:31:30.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.385+0000 7f56adf9b700 1 -- 172.21.15.31:0/1682842838 >> v1:172.21.15.62:6789/0 conn(0x7f56a806bd20 legacy=0x7f56a80c7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:30.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.385+0000 7f56adf9b700 1 -- 172.21.15.31:0/1682842838 >> v1:172.21.15.31:6789/0 conn(0x7f56a80c7f90 legacy=0x7f56a806c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:30.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.385+0000 7f56adf9b700 1 -- 172.21.15.31:0/1682842838 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56a815df60 con 0x7f56a8142c00 2023-10-06T02:31:30.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.385+0000 7f56adf9b700 1 --2- 172.21.15.31:0/1682842838 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56a8142c00 0x7f56a806b7e0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f56a4009860 tx=0x7f56a40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c75dc2679b988568 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:30.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.385+0000 7f56acf99700 1 -- 172.21.15.31:0/1682842838 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a00017f0 con 0x7f56a8142c00 2023-10-06T02:31:30.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.385+0000 7f56acf99700 1 -- 172.21.15.31:0/1682842838 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56a4011420 con 0x7f56a8142c00 2023-10-06T02:31:30.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.385+0000 7f56acf99700 1 -- 172.21.15.31:0/1682842838 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56980013f0 con 0x7f56a8142c00 2023-10-06T02:31:30.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.386+0000 7f56aef9d700 1 -- 172.21.15.31:0/1682842838 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56a8142c00 msgr2=0x7f56a806b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:30.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.386+0000 7f56aef9d700 1 --2- 172.21.15.31:0/1682842838 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56a8142c00 0x7f56a806b7e0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f56a4009860 tx=0x7f56a40091a0 comp rx=0 tx=0).stop 2023-10-06T02:31:30.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.386+0000 7f56aef9d700 1 -- 172.21.15.31:0/1682842838 shutdown_connections 2023-10-06T02:31:30.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.386+0000 7f56aef9d700 1 --2- 172.21.15.31:0/1682842838 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56a8142c00 0x7f56a806b7e0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:30.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.386+0000 7f56aef9d700 1 -- 172.21.15.31:0/1682842838 >> 172.21.15.31:0/1682842838 conn(0x7f56a80675a0 msgr2=0x7f56a80679a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:30.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.386+0000 7f56aef9d700 1 -- 172.21.15.31:0/1682842838 shutdown_connections 2023-10-06T02:31:30.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.386+0000 7f56aef9d700 1 -- 172.21.15.31:0/1682842838 wait complete. 2023-10-06T02:31:30.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.387+0000 7f56aef9d700 1 Processor -- start 2023-10-06T02:31:30.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.388+0000 7f56aef9d700 1 -- start start 2023-10-06T02:31:30.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.388+0000 7f56aef9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56a806bd20 0x7f56a813fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:30.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.388+0000 7f56aef9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56a80c7f90 0x7f56a80b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:30.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.389+0000 7f56aef9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f56a8142c00 0x7f56a80b2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:30.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.389+0000 7f56aef9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56a80c52c0 con 0x7f56a80c7f90 2023-10-06T02:31:30.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.389+0000 7f56aef9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56a80c5510 con 0x7f56a806bd20 2023-10-06T02:31:30.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.389+0000 7f56aef9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56a80c57f0 con 0x7f56a8142c00 2023-10-06T02:31:30.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.389+0000 7f56ad79a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56a80c7f90 0x7f56a80b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:30.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.389+0000 7f56ad79a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56a80c7f90 0x7f56a80b1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51550/0 (socket says 172.21.15.31:51550) 2023-10-06T02:31:30.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.389+0000 7f56ad79a700 1 -- 172.21.15.31:0/2742255398 learned_addr learned my addr 172.21.15.31:0/2742255398 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:30.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.389+0000 7f56adf9b700 1 --2- 172.21.15.31:0/2742255398 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56a806bd20 0x7f56a813fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:30.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.389+0000 7f56ae79c700 1 --2- 172.21.15.31:0/2742255398 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f56a8142c00 0x7f56a80b2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:30.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.389+0000 7f56ad79a700 1 -- 172.21.15.31:0/2742255398 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f56a8142c00 msgr2=0x7f56a80b2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:30.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.389+0000 7f56ad79a700 1 --2- 172.21.15.31:0/2742255398 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f56a8142c00 0x7f56a80b2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:30.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.389+0000 7f56ad79a700 1 -- 172.21.15.31:0/2742255398 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56a806bd20 msgr2=0x7f56a813fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:30.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.389+0000 7f56ad79a700 1 --2- 172.21.15.31:0/2742255398 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56a806bd20 0x7f56a813fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:30.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.389+0000 7f56ad79a700 1 -- 172.21.15.31:0/2742255398 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56a4009420 con 0x7f56a80c7f90 2023-10-06T02:31:30.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.389+0000 7f56adf9b700 1 --2- 172.21.15.31:0/2742255398 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56a806bd20 0x7f56a813fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:31:30.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.390+0000 7f56ad79a700 1 --2- 172.21.15.31:0/2742255398 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56a80c7f90 0x7f56a80b1ff0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f569800c810 tx=0x7f56980011a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:30.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.390+0000 7f569dffb700 1 -- 172.21.15.31:0/2742255398 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f569800a520 con 0x7f56a80c7f90 2023-10-06T02:31:30.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.390+0000 7f569dffb700 1 -- 172.21.15.31:0/2742255398 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f569800a680 con 0x7f56a80c7f90 2023-10-06T02:31:30.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.390+0000 7f569dffb700 1 -- 172.21.15.31:0/2742255398 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f569800aa40 con 0x7f56a80c7f90 2023-10-06T02:31:30.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.390+0000 7f56aef9d700 1 -- 172.21.15.31:0/2742255398 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56a80c5ad0 con 0x7f56a80c7f90 2023-10-06T02:31:30.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.390+0000 7f56aef9d700 1 -- 172.21.15.31:0/2742255398 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56a81402e0 con 0x7f56a80c7f90 2023-10-06T02:31:30.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.391+0000 7f56aef9d700 1 -- 172.21.15.31:0/2742255398 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56a80c5ff0 con 0x7f56a80c7f90 2023-10-06T02:31:30.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.392+0000 7f569dffb700 1 -- 172.21.15.31:0/2742255398 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 85403+0+0 (secure 0 0 0) 0x7f569801b400 con 0x7f56a80c7f90 2023-10-06T02:31:30.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.393+0000 7f569dffb700 1 --2- 172.21.15.31:0/2742255398 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f568c067680 0x7f568c069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:30.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.393+0000 7f569dffb700 1 -- 172.21.15.31:0/2742255398 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f56980855f0 con 0x7f56a80c7f90 2023-10-06T02:31:30.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.393+0000 7f56adf9b700 1 --2- 172.21.15.31:0/2742255398 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f568c067680 0x7f568c069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:30.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.394+0000 7f56adf9b700 1 --2- 172.21.15.31:0/2742255398 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f568c067680 0x7f568c069b30 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f56a40030c0 tx=0x7f56a4017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:30.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.397+0000 7f569dffb700 1 -- 172.21.15.31:0/2742255398 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f56a80c5ff0 con 0x7f56a80c7f90 2023-10-06T02:31:30.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.596+0000 7f56aef9d700 1 -- 172.21.15.31:0/2742255398 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f56a80c5680 con 0x7f568c067680 2023-10-06T02:31:30.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.607+0000 7f569dffb700 1 -- 172.21.15.31:0/2742255398 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311554 (secure 0 0 0) 0x7f56a80c5680 con 0x7f568c067680 2023-10-06T02:31:30.612 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:31:30.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.611+0000 7f56aef9d700 1 -- 172.21.15.31:0/2742255398 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f568c067680 msgr2=0x7f568c069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:30.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.611+0000 7f56aef9d700 1 --2- 172.21.15.31:0/2742255398 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f568c067680 0x7f568c069b30 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f56a40030c0 tx=0x7f56a4017000 comp rx=0 tx=0).stop 2023-10-06T02:31:30.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.611+0000 7f56aef9d700 1 -- 172.21.15.31:0/2742255398 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56a80c7f90 msgr2=0x7f56a80b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:30.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.611+0000 7f56aef9d700 1 --2- 172.21.15.31:0/2742255398 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56a80c7f90 0x7f56a80b1ff0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f569800c810 tx=0x7f56980011a0 comp rx=0 tx=0).stop 2023-10-06T02:31:30.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.611+0000 7f56aef9d700 1 -- 172.21.15.31:0/2742255398 shutdown_connections 2023-10-06T02:31:30.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.611+0000 7f56aef9d700 1 --2- 172.21.15.31:0/2742255398 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f568c067680 0x7f568c069b30 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:30.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.611+0000 7f56aef9d700 1 --2- 172.21.15.31:0/2742255398 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f56a8142c00 0x7f56a80b2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:30.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.611+0000 7f56aef9d700 1 --2- 172.21.15.31:0/2742255398 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56a806bd20 0x7f56a813fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:30.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.611+0000 7f56aef9d700 1 --2- 172.21.15.31:0/2742255398 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56a80c7f90 0x7f56a80b1ff0 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:30.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.611+0000 7f56aef9d700 1 -- 172.21.15.31:0/2742255398 >> 172.21.15.31:0/2742255398 conn(0x7f56a80675a0 msgr2=0x7f56a80c3bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:30.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.611+0000 7f56aef9d700 1 -- 172.21.15.31:0/2742255398 shutdown_connections 2023-10-06T02:31:30.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.611+0000 7f56aef9d700 1 -- 172.21.15.31:0/2742255398 wait complete. 2023-10-06T02:31:30.616 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-10-06T02:31:30.630 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":200,"stamp":"2023-10-06T02:31:30.512905+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":223,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":166,"ondisk_log_size":166,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":178,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2427656,"kb_used_data":16584,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752547064,"statfs":{"total":773094113280,"available":770608193536,"internally_reserved":0,"allocated":16982016,"data_stored":8523940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001632"},"pg_stats":[{"pgid":"26.3f","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.267990+0000","last_change":"2023-10-06T02:31:26.265201+0000","last_active":"2023-10-06T02:31:26.267990+0000","last_peered":"2023-10-06T02:31:26.267990+0000","last_clean":"2023-10-06T02:31:26.267990+0000","last_became_active":"2023-10-06T02:31:26.264924+0000","last_became_peered":"2023-10-06T02:31:26.264924+0000","last_unstale":"2023-10-06T02:31:26.267990+0000","last_undegraded":"2023-10-06T02:31:26.267990+0000","last_fullsized":"2023-10-06T02:31:26.267990+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:10:36.589251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3e","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.269621+0000","last_change":"2023-10-06T02:31:26.267681+0000","last_active":"2023-10-06T02:31:26.269621+0000","last_peered":"2023-10-06T02:31:26.269621+0000","last_clean":"2023-10-06T02:31:26.269621+0000","last_became_active":"2023-10-06T02:31:26.267405+0000","last_became_peered":"2023-10-06T02:31:26.267405+0000","last_unstale":"2023-10-06T02:31:26.269621+0000","last_undegraded":"2023-10-06T02:31:26.269621+0000","last_fullsized":"2023-10-06T02:31:26.269621+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:25:17.234017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3d","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.257654+0000","last_change":"2023-10-06T02:31:26.250873+0000","last_active":"2023-10-06T02:31:26.257654+0000","last_peered":"2023-10-06T02:31:26.257654+0000","last_clean":"2023-10-06T02:31:26.257654+0000","last_became_active":"2023-10-06T02:31:26.250538+0000","last_became_peered":"2023-10-06T02:31:26.250538+0000","last_unstale":"2023-10-06T02:31:26.257654+0000","last_undegraded":"2023-10-06T02:31:26.257654+0000","last_fullsized":"2023-10-06T02:31:26.257654+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:25:17.311269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.3c","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.257586+0000","last_change":"2023-10-06T02:31:26.251973+0000","last_active":"2023-10-06T02:31:26.257586+0000","last_peered":"2023-10-06T02:31:26.257586+0000","last_clean":"2023-10-06T02:31:26.257586+0000","last_became_active":"2023-10-06T02:31:26.251699+0000","last_became_peered":"2023-10-06T02:31:26.251699+0000","last_unstale":"2023-10-06T02:31:26.257586+0000","last_undegraded":"2023-10-06T02:31:26.257586+0000","last_fullsized":"2023-10-06T02:31:26.257586+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:17:04.188102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3b","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.260742+0000","last_change":"2023-10-06T02:31:26.256241+0000","last_active":"2023-10-06T02:31:26.260742+0000","last_peered":"2023-10-06T02:31:26.260742+0000","last_clean":"2023-10-06T02:31:26.260742+0000","last_became_active":"2023-10-06T02:31:26.255959+0000","last_became_peered":"2023-10-06T02:31:26.255959+0000","last_unstale":"2023-10-06T02:31:26.260742+0000","last_undegraded":"2023-10-06T02:31:26.260742+0000","last_fullsized":"2023-10-06T02:31:26.260742+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:45:34.829656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.3a","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.260757+0000","last_change":"2023-10-06T02:31:26.256130+0000","last_active":"2023-10-06T02:31:26.260757+0000","last_peered":"2023-10-06T02:31:26.260757+0000","last_clean":"2023-10-06T02:31:26.260757+0000","last_became_active":"2023-10-06T02:31:26.255805+0000","last_became_peered":"2023-10-06T02:31:26.255805+0000","last_unstale":"2023-10-06T02:31:26.260757+0000","last_undegraded":"2023-10-06T02:31:26.260757+0000","last_fullsized":"2023-10-06T02:31:26.260757+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:40:40.851662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.39","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.266551+0000","last_change":"2023-10-06T02:31:26.263200+0000","last_active":"2023-10-06T02:31:26.266551+0000","last_peered":"2023-10-06T02:31:26.266551+0000","last_clean":"2023-10-06T02:31:26.266551+0000","last_became_active":"2023-10-06T02:31:26.262943+0000","last_became_peered":"2023-10-06T02:31:26.262943+0000","last_unstale":"2023-10-06T02:31:26.266551+0000","last_undegraded":"2023-10-06T02:31:26.266551+0000","last_fullsized":"2023-10-06T02:31:26.266551+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:04:38.216053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.38","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.267385+0000","last_change":"2023-10-06T02:31:26.261673+0000","last_active":"2023-10-06T02:31:26.267385+0000","last_peered":"2023-10-06T02:31:26.267385+0000","last_clean":"2023-10-06T02:31:26.267385+0000","last_became_active":"2023-10-06T02:31:26.261364+0000","last_became_peered":"2023-10-06T02:31:26.261364+0000","last_unstale":"2023-10-06T02:31:26.267385+0000","last_undegraded":"2023-10-06T02:31:26.267385+0000","last_fullsized":"2023-10-06T02:31:26.267385+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:08:43.058251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.37","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.267969+0000","last_change":"2023-10-06T02:31:26.264821+0000","last_active":"2023-10-06T02:31:26.267969+0000","last_peered":"2023-10-06T02:31:26.267969+0000","last_clean":"2023-10-06T02:31:26.267969+0000","last_became_active":"2023-10-06T02:31:26.260543+0000","last_became_peered":"2023-10-06T02:31:26.260543+0000","last_unstale":"2023-10-06T02:31:26.267969+0000","last_undegraded":"2023-10-06T02:31:26.267969+0000","last_fullsized":"2023-10-06T02:31:26.267969+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:52:31.748249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.36","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.260751+0000","last_change":"2023-10-06T02:31:26.256860+0000","last_active":"2023-10-06T02:31:26.260751+0000","last_peered":"2023-10-06T02:31:26.260751+0000","last_clean":"2023-10-06T02:31:26.260751+0000","last_became_active":"2023-10-06T02:31:26.256588+0000","last_became_peered":"2023-10-06T02:31:26.256588+0000","last_unstale":"2023-10-06T02:31:26.260751+0000","last_undegraded":"2023-10-06T02:31:26.260751+0000","last_fullsized":"2023-10-06T02:31:26.260751+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:53:41.055784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.35","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.256837+0000","last_change":"2023-10-06T02:31:26.244042+0000","last_active":"2023-10-06T02:31:26.256837+0000","last_peered":"2023-10-06T02:31:26.256837+0000","last_clean":"2023-10-06T02:31:26.256837+0000","last_became_active":"2023-10-06T02:31:26.243653+0000","last_became_peered":"2023-10-06T02:31:26.243653+0000","last_unstale":"2023-10-06T02:31:26.256837+0000","last_undegraded":"2023-10-06T02:31:26.256837+0000","last_fullsized":"2023-10-06T02:31:26.256837+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:25:17.234017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.34","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.263701+0000","last_change":"2023-10-06T02:31:26.260676+0000","last_active":"2023-10-06T02:31:26.263701+0000","last_peered":"2023-10-06T02:31:26.263701+0000","last_clean":"2023-10-06T02:31:26.263701+0000","last_became_active":"2023-10-06T02:31:26.260285+0000","last_became_peered":"2023-10-06T02:31:26.260285+0000","last_unstale":"2023-10-06T02:31:26.263701+0000","last_undegraded":"2023-10-06T02:31:26.263701+0000","last_fullsized":"2023-10-06T02:31:26.263701+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:29:15.508209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.33","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.266711+0000","last_change":"2023-10-06T02:31:26.262919+0000","last_active":"2023-10-06T02:31:26.266711+0000","last_peered":"2023-10-06T02:31:26.266711+0000","last_clean":"2023-10-06T02:31:26.266711+0000","last_became_active":"2023-10-06T02:31:26.262570+0000","last_became_peered":"2023-10-06T02:31:26.262570+0000","last_unstale":"2023-10-06T02:31:26.266711+0000","last_undegraded":"2023-10-06T02:31:26.266711+0000","last_fullsized":"2023-10-06T02:31:26.266711+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:31:30.918438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.32","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.258168+0000","last_change":"2023-10-06T02:31:26.254903+0000","last_active":"2023-10-06T02:31:26.258168+0000","last_peered":"2023-10-06T02:31:26.258168+0000","last_clean":"2023-10-06T02:31:26.258168+0000","last_became_active":"2023-10-06T02:31:26.254617+0000","last_became_peered":"2023-10-06T02:31:26.254617+0000","last_unstale":"2023-10-06T02:31:26.258168+0000","last_undegraded":"2023-10-06T02:31:26.258168+0000","last_fullsized":"2023-10-06T02:31:26.258168+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:07:33.997738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.31","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.266929+0000","last_change":"2023-10-06T02:31:26.264463+0000","last_active":"2023-10-06T02:31:26.266929+0000","last_peered":"2023-10-06T02:31:26.266929+0000","last_clean":"2023-10-06T02:31:26.266929+0000","last_became_active":"2023-10-06T02:31:26.258882+0000","last_became_peered":"2023-10-06T02:31:26.258882+0000","last_unstale":"2023-10-06T02:31:26.266929+0000","last_undegraded":"2023-10-06T02:31:26.266929+0000","last_fullsized":"2023-10-06T02:31:26.266929+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:21:40.138974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.30","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.260702+0000","last_change":"2023-10-06T02:31:26.255233+0000","last_active":"2023-10-06T02:31:26.260702+0000","last_peered":"2023-10-06T02:31:26.260702+0000","last_clean":"2023-10-06T02:31:26.260702+0000","last_became_active":"2023-10-06T02:31:26.254974+0000","last_became_peered":"2023-10-06T02:31:26.254974+0000","last_unstale":"2023-10-06T02:31:26.260702+0000","last_undegraded":"2023-10-06T02:31:26.260702+0000","last_fullsized":"2023-10-06T02:31:26.260702+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:18:38.740278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2f","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.264419+0000","last_change":"2023-10-06T02:31:26.261279+0000","last_active":"2023-10-06T02:31:26.264419+0000","last_peered":"2023-10-06T02:31:26.264419+0000","last_clean":"2023-10-06T02:31:26.264419+0000","last_became_active":"2023-10-06T02:31:26.260982+0000","last_became_peered":"2023-10-06T02:31:26.260982+0000","last_unstale":"2023-10-06T02:31:26.264419+0000","last_undegraded":"2023-10-06T02:31:26.264419+0000","last_fullsized":"2023-10-06T02:31:26.264419+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:08:16.984980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2e","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.265431+0000","last_change":"2023-10-06T02:31:26.261700+0000","last_active":"2023-10-06T02:31:26.265431+0000","last_peered":"2023-10-06T02:31:26.265431+0000","last_clean":"2023-10-06T02:31:26.265431+0000","last_became_active":"2023-10-06T02:31:26.261387+0000","last_became_peered":"2023-10-06T02:31:26.261387+0000","last_unstale":"2023-10-06T02:31:26.265431+0000","last_undegraded":"2023-10-06T02:31:26.265431+0000","last_fullsized":"2023-10-06T02:31:26.265431+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:06:51.338576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2d","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.263668+0000","last_change":"2023-10-06T02:31:26.256852+0000","last_active":"2023-10-06T02:31:26.263668+0000","last_peered":"2023-10-06T02:31:26.263668+0000","last_clean":"2023-10-06T02:31:26.263668+0000","last_became_active":"2023-10-06T02:31:26.256485+0000","last_became_peered":"2023-10-06T02:31:26.256485+0000","last_unstale":"2023-10-06T02:31:26.263668+0000","last_undegraded":"2023-10-06T02:31:26.263668+0000","last_fullsized":"2023-10-06T02:31:26.263668+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:38:03.520905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2c","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.267286+0000","last_change":"2023-10-06T02:31:26.261634+0000","last_active":"2023-10-06T02:31:26.267286+0000","last_peered":"2023-10-06T02:31:26.267286+0000","last_clean":"2023-10-06T02:31:26.267286+0000","last_became_active":"2023-10-06T02:31:26.261365+0000","last_became_peered":"2023-10-06T02:31:26.261365+0000","last_unstale":"2023-10-06T02:31:26.267286+0000","last_undegraded":"2023-10-06T02:31:26.267286+0000","last_fullsized":"2023-10-06T02:31:26.267286+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:32:49.083484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2b","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.262183+0000","last_change":"2023-10-06T02:31:26.254306+0000","last_active":"2023-10-06T02:31:26.262183+0000","last_peered":"2023-10-06T02:31:26.262183+0000","last_clean":"2023-10-06T02:31:26.262183+0000","last_became_active":"2023-10-06T02:31:26.252527+0000","last_became_peered":"2023-10-06T02:31:26.252527+0000","last_unstale":"2023-10-06T02:31:26.262183+0000","last_undegraded":"2023-10-06T02:31:26.262183+0000","last_fullsized":"2023-10-06T02:31:26.262183+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:52:03.877573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.2a","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.264010+0000","last_change":"2023-10-06T02:31:26.259526+0000","last_active":"2023-10-06T02:31:26.264010+0000","last_peered":"2023-10-06T02:31:26.264010+0000","last_clean":"2023-10-06T02:31:26.264010+0000","last_became_active":"2023-10-06T02:31:26.258779+0000","last_became_peered":"2023-10-06T02:31:26.258779+0000","last_unstale":"2023-10-06T02:31:26.264010+0000","last_undegraded":"2023-10-06T02:31:26.264010+0000","last_fullsized":"2023-10-06T02:31:26.264010+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:16:42.707675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.29","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.270281+0000","last_change":"2023-10-06T02:31:26.267009+0000","last_active":"2023-10-06T02:31:26.270281+0000","last_peered":"2023-10-06T02:31:26.270281+0000","last_clean":"2023-10-06T02:31:26.270281+0000","last_became_active":"2023-10-06T02:31:26.266730+0000","last_became_peered":"2023-10-06T02:31:26.266730+0000","last_unstale":"2023-10-06T02:31:26.270281+0000","last_undegraded":"2023-10-06T02:31:26.270281+0000","last_fullsized":"2023-10-06T02:31:26.270281+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:54:52.162681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.28","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.266168+0000","last_change":"2023-10-06T02:31:26.262949+0000","last_active":"2023-10-06T02:31:26.266168+0000","last_peered":"2023-10-06T02:31:26.266168+0000","last_clean":"2023-10-06T02:31:26.266168+0000","last_became_active":"2023-10-06T02:31:26.262604+0000","last_became_peered":"2023-10-06T02:31:26.262604+0000","last_unstale":"2023-10-06T02:31:26.266168+0000","last_undegraded":"2023-10-06T02:31:26.266168+0000","last_fullsized":"2023-10-06T02:31:26.266168+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:06:51.338576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.27","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.264238+0000","last_change":"2023-10-06T02:31:26.257598+0000","last_active":"2023-10-06T02:31:26.264238+0000","last_peered":"2023-10-06T02:31:26.264238+0000","last_clean":"2023-10-06T02:31:26.264238+0000","last_became_active":"2023-10-06T02:31:26.257237+0000","last_became_peered":"2023-10-06T02:31:26.257237+0000","last_unstale":"2023-10-06T02:31:26.264238+0000","last_undegraded":"2023-10-06T02:31:26.264238+0000","last_fullsized":"2023-10-06T02:31:26.264238+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:42:10.460130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.26","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.269912+0000","last_change":"2023-10-06T02:31:26.261508+0000","last_active":"2023-10-06T02:31:26.269912+0000","last_peered":"2023-10-06T02:31:26.269912+0000","last_clean":"2023-10-06T02:31:26.269912+0000","last_became_active":"2023-10-06T02:31:26.261210+0000","last_became_peered":"2023-10-06T02:31:26.261210+0000","last_unstale":"2023-10-06T02:31:26.269912+0000","last_undegraded":"2023-10-06T02:31:26.269912+0000","last_fullsized":"2023-10-06T02:31:26.269912+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:16:42.707675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.25","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.270115+0000","last_change":"2023-10-06T02:31:26.266150+0000","last_active":"2023-10-06T02:31:26.270115+0000","last_peered":"2023-10-06T02:31:26.270115+0000","last_clean":"2023-10-06T02:31:26.270115+0000","last_became_active":"2023-10-06T02:31:26.265862+0000","last_became_peered":"2023-10-06T02:31:26.265862+0000","last_unstale":"2023-10-06T02:31:26.270115+0000","last_undegraded":"2023-10-06T02:31:26.270115+0000","last_fullsized":"2023-10-06T02:31:26.270115+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:23:05.322592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.24","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.261744+0000","last_change":"2023-10-06T02:31:26.256284+0000","last_active":"2023-10-06T02:31:26.261744+0000","last_peered":"2023-10-06T02:31:26.261744+0000","last_clean":"2023-10-06T02:31:26.261744+0000","last_became_active":"2023-10-06T02:31:26.255978+0000","last_became_peered":"2023-10-06T02:31:26.255978+0000","last_unstale":"2023-10-06T02:31:26.261744+0000","last_undegraded":"2023-10-06T02:31:26.261744+0000","last_fullsized":"2023-10-06T02:31:26.261744+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:24:11.363292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.23","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.269993+0000","last_change":"2023-10-06T02:31:26.260168+0000","last_active":"2023-10-06T02:31:26.269993+0000","last_peered":"2023-10-06T02:31:26.269993+0000","last_clean":"2023-10-06T02:31:26.269993+0000","last_became_active":"2023-10-06T02:31:26.259743+0000","last_became_peered":"2023-10-06T02:31:26.259743+0000","last_unstale":"2023-10-06T02:31:26.269993+0000","last_undegraded":"2023-10-06T02:31:26.269993+0000","last_fullsized":"2023-10-06T02:31:26.269993+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:27:31.472908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.22","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.266229+0000","last_change":"2023-10-06T02:31:26.255194+0000","last_active":"2023-10-06T02:31:26.266229+0000","last_peered":"2023-10-06T02:31:26.266229+0000","last_clean":"2023-10-06T02:31:26.266229+0000","last_became_active":"2023-10-06T02:31:26.254396+0000","last_became_peered":"2023-10-06T02:31:26.254396+0000","last_unstale":"2023-10-06T02:31:26.266229+0000","last_undegraded":"2023-10-06T02:31:26.266229+0000","last_fullsized":"2023-10-06T02:31:26.266229+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:09:25.643431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.21","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.262258+0000","last_change":"2023-10-06T02:31:26.253779+0000","last_active":"2023-10-06T02:31:26.262258+0000","last_peered":"2023-10-06T02:31:26.262258+0000","last_clean":"2023-10-06T02:31:26.262258+0000","last_became_active":"2023-10-06T02:31:26.253512+0000","last_became_peered":"2023-10-06T02:31:26.253512+0000","last_unstale":"2023-10-06T02:31:26.262258+0000","last_undegraded":"2023-10-06T02:31:26.262258+0000","last_fullsized":"2023-10-06T02:31:26.262258+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:01:14.251053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.20","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.260675+0000","last_change":"2023-10-06T02:31:26.254349+0000","last_active":"2023-10-06T02:31:26.260675+0000","last_peered":"2023-10-06T02:31:26.260675+0000","last_clean":"2023-10-06T02:31:26.260675+0000","last_became_active":"2023-10-06T02:31:26.254049+0000","last_became_peered":"2023-10-06T02:31:26.254049+0000","last_unstale":"2023-10-06T02:31:26.260675+0000","last_undegraded":"2023-10-06T02:31:26.260675+0000","last_fullsized":"2023-10-06T02:31:26.260675+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:29:15.508209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.14","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.270339+0000","last_change":"2023-10-06T02:31:26.267533+0000","last_active":"2023-10-06T02:31:26.270339+0000","last_peered":"2023-10-06T02:31:26.270339+0000","last_clean":"2023-10-06T02:31:26.270339+0000","last_became_active":"2023-10-06T02:31:26.267238+0000","last_became_peered":"2023-10-06T02:31:26.267238+0000","last_unstale":"2023-10-06T02:31:26.270339+0000","last_undegraded":"2023-10-06T02:31:26.270339+0000","last_fullsized":"2023-10-06T02:31:26.270339+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:49:16.077340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.246287+0000","last_change":"2023-10-06T02:31:25.235636+0000","last_active":"2023-10-06T02:31:26.246287+0000","last_peered":"2023-10-06T02:31:26.246287+0000","last_clean":"2023-10-06T02:31:26.246287+0000","last_became_active":"2023-10-06T02:31:25.233839+0000","last_became_peered":"2023-10-06T02:31:25.233839+0000","last_unstale":"2023-10-06T02:31:26.246287+0000","last_undegraded":"2023-10-06T02:31:26.246287+0000","last_fullsized":"2023-10-06T02:31:26.246287+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_clean_scrub_stamp":"2023-10-06T02:31:24.211070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:26:39.899368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.15","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.263304+0000","last_change":"2023-10-06T02:31:26.260505+0000","last_active":"2023-10-06T02:31:26.263304+0000","last_peered":"2023-10-06T02:31:26.263304+0000","last_clean":"2023-10-06T02:31:26.263304+0000","last_became_active":"2023-10-06T02:31:26.260223+0000","last_became_peered":"2023-10-06T02:31:26.260223+0000","last_unstale":"2023-10-06T02:31:26.263304+0000","last_undegraded":"2023-10-06T02:31:26.263304+0000","last_fullsized":"2023-10-06T02:31:26.263304+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:42:34.992024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.250873+0000","last_change":"2023-10-06T02:31:25.246162+0000","last_active":"2023-10-06T02:31:26.250873+0000","last_peered":"2023-10-06T02:31:26.250873+0000","last_clean":"2023-10-06T02:31:26.250873+0000","last_became_active":"2023-10-06T02:31:25.245821+0000","last_became_peered":"2023-10-06T02:31:25.245821+0000","last_unstale":"2023-10-06T02:31:26.250873+0000","last_undegraded":"2023-10-06T02:31:26.250873+0000","last_fullsized":"2023-10-06T02:31:26.250873+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_clean_scrub_stamp":"2023-10-06T02:31:24.211070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:03:05.907292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.265412+0000","last_change":"2023-10-06T02:31:26.258365+0000","last_active":"2023-10-06T02:31:26.265412+0000","last_peered":"2023-10-06T02:31:26.265412+0000","last_clean":"2023-10-06T02:31:26.265412+0000","last_became_active":"2023-10-06T02:31:26.257997+0000","last_became_peered":"2023-10-06T02:31:26.257997+0000","last_unstale":"2023-10-06T02:31:26.265412+0000","last_undegraded":"2023-10-06T02:31:26.265412+0000","last_fullsized":"2023-10-06T02:31:26.265412+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:03:29.054970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.15","version":"155'3","reported_seq":13,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.275523+0000","last_change":"2023-10-06T02:31:25.242416+0000","last_active":"2023-10-06T02:31:26.275523+0000","last_peered":"2023-10-06T02:31:26.275523+0000","last_clean":"2023-10-06T02:31:26.275523+0000","last_became_active":"2023-10-06T02:31:25.242136+0000","last_became_peered":"2023-10-06T02:31:25.242136+0000","last_unstale":"2023-10-06T02:31:26.275523+0000","last_undegraded":"2023-10-06T02:31:26.275523+0000","last_fullsized":"2023-10-06T02:31:26.275523+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_clean_scrub_stamp":"2023-10-06T02:31:24.211070+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:08:08.801557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.269695+0000","last_change":"2023-10-06T02:31:26.267624+0000","last_active":"2023-10-06T02:31:26.269695+0000","last_peered":"2023-10-06T02:31:26.269695+0000","last_clean":"2023-10-06T02:31:26.269695+0000","last_became_active":"2023-10-06T02:31:26.267261+0000","last_became_peered":"2023-10-06T02:31:26.267261+0000","last_unstale":"2023-10-06T02:31:26.269695+0000","last_undegraded":"2023-10-06T02:31:26.269695+0000","last_fullsized":"2023-10-06T02:31:26.269695+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:27:31.472908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.14","version":"155'6","reported_seq":16,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.281821+0000","last_change":"2023-10-06T02:31:25.231015+0000","last_active":"2023-10-06T02:31:26.281821+0000","last_peered":"2023-10-06T02:31:26.281821+0000","last_clean":"2023-10-06T02:31:26.281821+0000","last_became_active":"2023-10-06T02:31:25.230074+0000","last_became_peered":"2023-10-06T02:31:25.230074+0000","last_unstale":"2023-10-06T02:31:26.281821+0000","last_undegraded":"2023-10-06T02:31:26.281821+0000","last_fullsized":"2023-10-06T02:31:26.281821+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_clean_scrub_stamp":"2023-10-06T02:31:24.211070+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:49:46.744608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.10","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.262624+0000","last_change":"2023-10-06T02:31:26.256692+0000","last_active":"2023-10-06T02:31:26.262624+0000","last_peered":"2023-10-06T02:31:26.262624+0000","last_clean":"2023-10-06T02:31:26.262624+0000","last_became_active":"2023-10-06T02:31:26.256421+0000","last_became_peered":"2023-10-06T02:31:26.256421+0000","last_unstale":"2023-10-06T02:31:26.262624+0000","last_undegraded":"2023-10-06T02:31:26.262624+0000","last_fullsized":"2023-10-06T02:31:26.262624+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:35:27.025630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.13","version":"155'2","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.278320+0000","last_change":"2023-10-06T02:31:25.233452+0000","last_active":"2023-10-06T02:31:26.278320+0000","last_peered":"2023-10-06T02:31:26.278320+0000","last_clean":"2023-10-06T02:31:26.278320+0000","last_became_active":"2023-10-06T02:31:25.232924+0000","last_became_peered":"2023-10-06T02:31:25.232924+0000","last_unstale":"2023-10-06T02:31:26.278320+0000","last_undegraded":"2023-10-06T02:31:26.278320+0000","last_fullsized":"2023-10-06T02:31:26.278320+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_clean_scrub_stamp":"2023-10-06T02:31:24.211070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:06:51.278627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.261402+0000","last_change":"2023-10-06T02:31:26.254653+0000","last_active":"2023-10-06T02:31:26.261402+0000","last_peered":"2023-10-06T02:31:26.261402+0000","last_clean":"2023-10-06T02:31:26.261402+0000","last_became_active":"2023-10-06T02:31:26.254383+0000","last_became_peered":"2023-10-06T02:31:26.254383+0000","last_unstale":"2023-10-06T02:31:26.261402+0000","last_undegraded":"2023-10-06T02:31:26.261402+0000","last_fullsized":"2023-10-06T02:31:26.261402+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:14:58.219720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.251072+0000","last_change":"2023-10-06T02:31:25.245963+0000","last_active":"2023-10-06T02:31:26.251072+0000","last_peered":"2023-10-06T02:31:26.251072+0000","last_clean":"2023-10-06T02:31:26.251072+0000","last_became_active":"2023-10-06T02:31:25.245628+0000","last_became_peered":"2023-10-06T02:31:25.245628+0000","last_unstale":"2023-10-06T02:31:26.251072+0000","last_undegraded":"2023-10-06T02:31:26.251072+0000","last_fullsized":"2023-10-06T02:31:26.251072+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_clean_scrub_stamp":"2023-10-06T02:31:24.211070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:40:16.238993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.264963+0000","last_change":"2023-10-06T02:31:26.257635+0000","last_active":"2023-10-06T02:31:26.264963+0000","last_peered":"2023-10-06T02:31:26.264963+0000","last_clean":"2023-10-06T02:31:26.264963+0000","last_became_active":"2023-10-06T02:31:26.257339+0000","last_became_peered":"2023-10-06T02:31:26.257339+0000","last_unstale":"2023-10-06T02:31:26.264963+0000","last_undegraded":"2023-10-06T02:31:26.264963+0000","last_fullsized":"2023-10-06T02:31:26.264963+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:01:04.378232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.244965+0000","last_change":"2023-10-06T02:31:25.227976+0000","last_active":"2023-10-06T02:31:26.244965+0000","last_peered":"2023-10-06T02:31:26.244965+0000","last_clean":"2023-10-06T02:31:26.244965+0000","last_became_active":"2023-10-06T02:31:25.227709+0000","last_became_peered":"2023-10-06T02:31:25.227709+0000","last_unstale":"2023-10-06T02:31:26.244965+0000","last_undegraded":"2023-10-06T02:31:26.244965+0000","last_fullsized":"2023-10-06T02:31:26.244965+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_clean_scrub_stamp":"2023-10-06T02:31:24.211070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:11.225751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.13","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.265869+0000","last_change":"2023-10-06T02:31:26.257788+0000","last_active":"2023-10-06T02:31:26.265869+0000","last_peered":"2023-10-06T02:31:26.265869+0000","last_clean":"2023-10-06T02:31:26.265869+0000","last_became_active":"2023-10-06T02:31:26.257493+0000","last_became_peered":"2023-10-06T02:31:26.257493+0000","last_unstale":"2023-10-06T02:31:26.265869+0000","last_undegraded":"2023-10-06T02:31:26.265869+0000","last_fullsized":"2023-10-06T02:31:26.265869+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:27:31.472908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.10","version":"155'8","reported_seq":18,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.280459+0000","last_change":"2023-10-06T02:31:25.232435+0000","last_active":"2023-10-06T02:31:26.280459+0000","last_peered":"2023-10-06T02:31:26.280459+0000","last_clean":"2023-10-06T02:31:26.280459+0000","last_became_active":"2023-10-06T02:31:25.232107+0000","last_became_peered":"2023-10-06T02:31:25.232107+0000","last_unstale":"2023-10-06T02:31:26.280459+0000","last_undegraded":"2023-10-06T02:31:26.280459+0000","last_fullsized":"2023-10-06T02:31:26.280459+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_clean_scrub_stamp":"2023-10-06T02:31:24.211070+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:49:18.537141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.257523+0000","last_change":"2023-10-06T02:31:26.253710+0000","last_active":"2023-10-06T02:31:26.257523+0000","last_peered":"2023-10-06T02:31:26.257523+0000","last_clean":"2023-10-06T02:31:26.257523+0000","last_became_active":"2023-10-06T02:31:26.253418+0000","last_became_peered":"2023-10-06T02:31:26.253418+0000","last_unstale":"2023-10-06T02:31:26.257523+0000","last_undegraded":"2023-10-06T02:31:26.257523+0000","last_fullsized":"2023-10-06T02:31:26.257523+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:26:55.194055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"155'1","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.277188+0000","last_change":"2023-10-06T02:31:25.232877+0000","last_active":"2023-10-06T02:31:26.277188+0000","last_peered":"2023-10-06T02:31:26.277188+0000","last_clean":"2023-10-06T02:31:26.277188+0000","last_became_active":"2023-10-06T02:31:25.232601+0000","last_became_peered":"2023-10-06T02:31:25.232601+0000","last_unstale":"2023-10-06T02:31:26.277188+0000","last_undegraded":"2023-10-06T02:31:26.277188+0000","last_fullsized":"2023-10-06T02:31:26.277188+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_clean_scrub_stamp":"2023-10-06T02:31:24.211070+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:32:10.362038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.d","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.262187+0000","last_change":"2023-10-06T02:31:26.260404+0000","last_active":"2023-10-06T02:31:26.262187+0000","last_peered":"2023-10-06T02:31:26.262187+0000","last_clean":"2023-10-06T02:31:26.262187+0000","last_became_active":"2023-10-06T02:31:26.260138+0000","last_became_peered":"2023-10-06T02:31:26.260138+0000","last_unstale":"2023-10-06T02:31:26.262187+0000","last_undegraded":"2023-10-06T02:31:26.262187+0000","last_fullsized":"2023-10-06T02:31:26.262187+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:19:31.367437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.e","version":"155'2","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.275922+0000","last_change":"2023-10-06T02:31:25.235127+0000","last_active":"2023-10-06T02:31:26.275922+0000","last_peered":"2023-10-06T02:31:26.275922+0000","last_clean":"2023-10-06T02:31:26.275922+0000","last_became_active":"2023-10-06T02:31:25.234876+0000","last_became_peered":"2023-10-06T02:31:25.234876+0000","last_unstale":"2023-10-06T02:31:26.275922+0000","last_undegraded":"2023-10-06T02:31:26.275922+0000","last_fullsized":"2023-10-06T02:31:26.275922+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_clean_scrub_stamp":"2023-10-06T02:31:24.211070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:01:07.384356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.e","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.269892+0000","last_change":"2023-10-06T02:31:26.260580+0000","last_active":"2023-10-06T02:31:26.269892+0000","last_peered":"2023-10-06T02:31:26.269892+0000","last_clean":"2023-10-06T02:31:26.269892+0000","last_became_active":"2023-10-06T02:31:26.260281+0000","last_became_peered":"2023-10-06T02:31:26.260281+0000","last_unstale":"2023-10-06T02:31:26.269892+0000","last_undegraded":"2023-10-06T02:31:26.269892+0000","last_fullsized":"2023-10-06T02:31:26.269892+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:10:36.589251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.d","version":"155'1","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.278092+0000","last_change":"2023-10-06T02:31:25.240303+0000","last_active":"2023-10-06T02:31:26.278092+0000","last_peered":"2023-10-06T02:31:26.278092+0000","last_clean":"2023-10-06T02:31:26.278092+0000","last_became_active":"2023-10-06T02:31:25.239964+0000","last_became_peered":"2023-10-06T02:31:25.239964+0000","last_unstale":"2023-10-06T02:31:26.278092+0000","last_undegraded":"2023-10-06T02:31:26.278092+0000","last_fullsized":"2023-10-06T02:31:26.278092+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_clean_scrub_stamp":"2023-10-06T02:31:24.211070+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:25:01.375389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.f","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.269781+0000","last_change":"2023-10-06T02:31:26.268110+0000","last_active":"2023-10-06T02:31:26.269781+0000","last_peered":"2023-10-06T02:31:26.269781+0000","last_clean":"2023-10-06T02:31:26.269781+0000","last_became_active":"2023-10-06T02:31:26.267769+0000","last_became_peered":"2023-10-06T02:31:26.267769+0000","last_unstale":"2023-10-06T02:31:26.269781+0000","last_undegraded":"2023-10-06T02:31:26.269781+0000","last_fullsized":"2023-10-06T02:31:26.269781+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:09:25.643431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.239901+0000","last_change":"2023-10-06T02:31:25.239545+0000","last_active":"2023-10-06T02:31:26.239901+0000","last_peered":"2023-10-06T02:31:26.239901+0000","last_clean":"2023-10-06T02:31:26.239901+0000","last_became_active":"2023-10-06T02:31:25.239256+0000","last_became_peered":"2023-10-06T02:31:25.239256+0000","last_unstale":"2023-10-06T02:31:26.239901+0000","last_undegraded":"2023-10-06T02:31:26.239901+0000","last_fullsized":"2023-10-06T02:31:26.239901+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_clean_scrub_stamp":"2023-10-06T02:31:24.211070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:16:13.065247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.264335+0000","last_change":"2023-10-06T02:31:26.260199+0000","last_active":"2023-10-06T02:31:26.264335+0000","last_peered":"2023-10-06T02:31:26.264335+0000","last_clean":"2023-10-06T02:31:26.264335+0000","last_became_active":"2023-10-06T02:31:26.259898+0000","last_became_peered":"2023-10-06T02:31:26.259898+0000","last_unstale":"2023-10-06T02:31:26.264335+0000","last_undegraded":"2023-10-06T02:31:26.264335+0000","last_fullsized":"2023-10-06T02:31:26.264335+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:49:58.442772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"155'2","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.277853+0000","last_change":"2023-10-06T02:31:25.242559+0000","last_active":"2023-10-06T02:31:26.277853+0000","last_peered":"2023-10-06T02:31:26.277853+0000","last_clean":"2023-10-06T02:31:26.277853+0000","last_became_active":"2023-10-06T02:31:25.242245+0000","last_became_peered":"2023-10-06T02:31:25.242245+0000","last_unstale":"2023-10-06T02:31:26.277853+0000","last_undegraded":"2023-10-06T02:31:26.277853+0000","last_fullsized":"2023-10-06T02:31:26.277853+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_clean_scrub_stamp":"2023-10-06T02:31:24.211070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:40:16.238993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.263797+0000","last_change":"2023-10-06T02:31:26.260197+0000","last_active":"2023-10-06T02:31:26.263797+0000","last_peered":"2023-10-06T02:31:26.263797+0000","last_clean":"2023-10-06T02:31:26.263797+0000","last_became_active":"2023-10-06T02:31:26.258913+0000","last_became_peered":"2023-10-06T02:31:26.258913+0000","last_unstale":"2023-10-06T02:31:26.263797+0000","last_undegraded":"2023-10-06T02:31:26.263797+0000","last_fullsized":"2023-10-06T02:31:26.263797+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:08:36.844212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.244503+0000","last_change":"2023-10-06T02:31:25.227427+0000","last_active":"2023-10-06T02:31:26.244503+0000","last_peered":"2023-10-06T02:31:26.244503+0000","last_clean":"2023-10-06T02:31:26.244503+0000","last_became_active":"2023-10-06T02:31:25.226896+0000","last_became_peered":"2023-10-06T02:31:25.226896+0000","last_unstale":"2023-10-06T02:31:26.244503+0000","last_undegraded":"2023-10-06T02:31:26.244503+0000","last_fullsized":"2023-10-06T02:31:26.244503+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_clean_scrub_stamp":"2023-10-06T02:31:24.211070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:37:47.966193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.266447+0000","last_change":"2023-10-06T02:31:26.261736+0000","last_active":"2023-10-06T02:31:26.266447+0000","last_peered":"2023-10-06T02:31:26.266447+0000","last_clean":"2023-10-06T02:31:26.266447+0000","last_became_active":"2023-10-06T02:31:26.261465+0000","last_became_peered":"2023-10-06T02:31:26.261465+0000","last_unstale":"2023-10-06T02:31:26.266447+0000","last_undegraded":"2023-10-06T02:31:26.266447+0000","last_fullsized":"2023-10-06T02:31:26.266447+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:33:46.837429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.239158+0000","last_change":"2023-10-06T02:31:25.242026+0000","last_active":"2023-10-06T02:31:26.239158+0000","last_peered":"2023-10-06T02:31:26.239158+0000","last_clean":"2023-10-06T02:31:26.239158+0000","last_became_active":"2023-10-06T02:31:25.241765+0000","last_became_peered":"2023-10-06T02:31:25.241765+0000","last_unstale":"2023-10-06T02:31:26.239158+0000","last_undegraded":"2023-10-06T02:31:26.239158+0000","last_fullsized":"2023-10-06T02:31:26.239158+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_clean_scrub_stamp":"2023-10-06T02:31:24.211070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:29:03.369149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.b","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.261452+0000","last_change":"2023-10-06T02:31:26.258018+0000","last_active":"2023-10-06T02:31:26.261452+0000","last_peered":"2023-10-06T02:31:26.261452+0000","last_clean":"2023-10-06T02:31:26.261452+0000","last_became_active":"2023-10-06T02:31:26.257732+0000","last_became_peered":"2023-10-06T02:31:26.257732+0000","last_unstale":"2023-10-06T02:31:26.261452+0000","last_undegraded":"2023-10-06T02:31:26.261452+0000","last_fullsized":"2023-10-06T02:31:26.261452+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:26:58.631262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.239559+0000","last_change":"2023-10-06T02:31:25.241731+0000","last_active":"2023-10-06T02:31:26.239559+0000","last_peered":"2023-10-06T02:31:26.239559+0000","last_clean":"2023-10-06T02:31:26.239559+0000","last_became_active":"2023-10-06T02:31:25.241439+0000","last_became_peered":"2023-10-06T02:31:25.241439+0000","last_unstale":"2023-10-06T02:31:26.239559+0000","last_undegraded":"2023-10-06T02:31:26.239559+0000","last_fullsized":"2023-10-06T02:31:26.239559+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_clean_scrub_stamp":"2023-10-06T02:31:24.211070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:29:03.369149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.248037+0000","last_change":"2023-10-06T02:31:26.245158+0000","last_active":"2023-10-06T02:31:26.248037+0000","last_peered":"2023-10-06T02:31:26.248037+0000","last_clean":"2023-10-06T02:31:26.248037+0000","last_became_active":"2023-10-06T02:31:26.244876+0000","last_became_peered":"2023-10-06T02:31:26.244876+0000","last_unstale":"2023-10-06T02:31:26.248037+0000","last_undegraded":"2023-10-06T02:31:26.248037+0000","last_fullsized":"2023-10-06T02:31:26.248037+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:40:55.719210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.7","version":"155'2","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.275652+0000","last_change":"2023-10-06T02:31:25.239291+0000","last_active":"2023-10-06T02:31:26.275652+0000","last_peered":"2023-10-06T02:31:26.275652+0000","last_clean":"2023-10-06T02:31:26.275652+0000","last_became_active":"2023-10-06T02:31:25.239024+0000","last_became_peered":"2023-10-06T02:31:25.239024+0000","last_unstale":"2023-10-06T02:31:26.275652+0000","last_undegraded":"2023-10-06T02:31:26.275652+0000","last_fullsized":"2023-10-06T02:31:26.275652+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_clean_scrub_stamp":"2023-10-06T02:31:24.211070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:54:51.160175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.264824+0000","last_change":"2023-10-06T02:31:26.259555+0000","last_active":"2023-10-06T02:31:26.264824+0000","last_peered":"2023-10-06T02:31:26.264824+0000","last_clean":"2023-10-06T02:31:26.264824+0000","last_became_active":"2023-10-06T02:31:26.259261+0000","last_became_peered":"2023-10-06T02:31:26.259261+0000","last_unstale":"2023-10-06T02:31:26.264824+0000","last_undegraded":"2023-10-06T02:31:26.264824+0000","last_fullsized":"2023-10-06T02:31:26.264824+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:18:02.982727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.239576+0000","last_change":"2023-10-06T02:31:25.241396+0000","last_active":"2023-10-06T02:31:26.239576+0000","last_peered":"2023-10-06T02:31:26.239576+0000","last_clean":"2023-10-06T02:31:26.239576+0000","last_became_active":"2023-10-06T02:31:25.241108+0000","last_became_peered":"2023-10-06T02:31:25.241108+0000","last_unstale":"2023-10-06T02:31:26.239576+0000","last_undegraded":"2023-10-06T02:31:26.239576+0000","last_fullsized":"2023-10-06T02:31:26.239576+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_clean_scrub_stamp":"2023-10-06T02:31:24.211070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:54:43.986442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.270329+0000","last_change":"2023-10-06T02:31:26.267947+0000","last_active":"2023-10-06T02:31:26.270329+0000","last_peered":"2023-10-06T02:31:26.270329+0000","last_clean":"2023-10-06T02:31:26.270329+0000","last_became_active":"2023-10-06T02:31:26.267621+0000","last_became_peered":"2023-10-06T02:31:26.267621+0000","last_unstale":"2023-10-06T02:31:26.270329+0000","last_undegraded":"2023-10-06T02:31:26.270329+0000","last_fullsized":"2023-10-06T02:31:26.270329+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:57.836609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.249709+0000","last_change":"2023-10-06T02:31:25.240770+0000","last_active":"2023-10-06T02:31:26.249709+0000","last_peered":"2023-10-06T02:31:26.249709+0000","last_clean":"2023-10-06T02:31:26.249709+0000","last_became_active":"2023-10-06T02:31:25.240376+0000","last_became_peered":"2023-10-06T02:31:25.240376+0000","last_unstale":"2023-10-06T02:31:26.249709+0000","last_undegraded":"2023-10-06T02:31:26.249709+0000","last_fullsized":"2023-10-06T02:31:26.249709+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_clean_scrub_stamp":"2023-10-06T02:31:24.211070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:01:07.384356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.7","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.249627+0000","last_change":"2023-10-06T02:31:26.245037+0000","last_active":"2023-10-06T02:31:26.249627+0000","last_peered":"2023-10-06T02:31:26.249627+0000","last_clean":"2023-10-06T02:31:26.249627+0000","last_became_active":"2023-10-06T02:31:26.244741+0000","last_became_peered":"2023-10-06T02:31:26.244741+0000","last_unstale":"2023-10-06T02:31:26.249627+0000","last_undegraded":"2023-10-06T02:31:26.249627+0000","last_fullsized":"2023-10-06T02:31:26.249627+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:39:43.017395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.249571+0000","last_change":"2023-10-06T02:31:25.247052+0000","last_active":"2023-10-06T02:31:26.249571+0000","last_peered":"2023-10-06T02:31:26.249571+0000","last_clean":"2023-10-06T02:31:26.249571+0000","last_became_active":"2023-10-06T02:31:25.246750+0000","last_became_peered":"2023-10-06T02:31:25.246750+0000","last_unstale":"2023-10-06T02:31:26.249571+0000","last_undegraded":"2023-10-06T02:31:26.249571+0000","last_fullsized":"2023-10-06T02:31:26.249571+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_clean_scrub_stamp":"2023-10-06T02:31:24.211070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:08:08.801557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.258497+0000","last_change":"2023-10-06T02:31:26.251288+0000","last_active":"2023-10-06T02:31:26.258497+0000","last_peered":"2023-10-06T02:31:26.258497+0000","last_clean":"2023-10-06T02:31:26.258497+0000","last_became_active":"2023-10-06T02:31:26.250957+0000","last_became_peered":"2023-10-06T02:31:26.250957+0000","last_unstale":"2023-10-06T02:31:26.258497+0000","last_undegraded":"2023-10-06T02:31:26.258497+0000","last_fullsized":"2023-10-06T02:31:26.258497+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:22:34.110436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.249240+0000","last_change":"2023-10-06T02:31:25.237989+0000","last_active":"2023-10-06T02:31:26.249240+0000","last_peered":"2023-10-06T02:31:26.249240+0000","last_clean":"2023-10-06T02:31:26.249240+0000","last_became_active":"2023-10-06T02:31:25.237714+0000","last_became_peered":"2023-10-06T02:31:25.237714+0000","last_unstale":"2023-10-06T02:31:26.249240+0000","last_undegraded":"2023-10-06T02:31:26.249240+0000","last_fullsized":"2023-10-06T02:31:26.249240+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_clean_scrub_stamp":"2023-10-06T02:31:24.211070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:09:24.640925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.257033+0000","last_change":"2023-10-06T02:31:26.252230+0000","last_active":"2023-10-06T02:31:26.257033+0000","last_peered":"2023-10-06T02:31:26.257033+0000","last_clean":"2023-10-06T02:31:26.257033+0000","last_became_active":"2023-10-06T02:31:26.251879+0000","last_became_peered":"2023-10-06T02:31:26.251879+0000","last_unstale":"2023-10-06T02:31:26.257033+0000","last_undegraded":"2023-10-06T02:31:26.257033+0000","last_fullsized":"2023-10-06T02:31:26.257033+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:18:33.267912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.248720+0000","last_change":"2023-10-06T02:31:25.243778+0000","last_active":"2023-10-06T02:31:26.248720+0000","last_peered":"2023-10-06T02:31:26.248720+0000","last_clean":"2023-10-06T02:31:26.248720+0000","last_became_active":"2023-10-06T02:31:25.243475+0000","last_became_peered":"2023-10-06T02:31:25.243475+0000","last_unstale":"2023-10-06T02:31:26.248720+0000","last_undegraded":"2023-10-06T02:31:26.248720+0000","last_fullsized":"2023-10-06T02:31:26.248720+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_clean_scrub_stamp":"2023-10-06T02:31:24.211070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:16:41.705169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.264315+0000","last_change":"2023-10-06T02:31:26.259854+0000","last_active":"2023-10-06T02:31:26.264315+0000","last_peered":"2023-10-06T02:31:26.264315+0000","last_clean":"2023-10-06T02:31:26.264315+0000","last_became_active":"2023-10-06T02:31:26.259226+0000","last_became_peered":"2023-10-06T02:31:26.259226+0000","last_unstale":"2023-10-06T02:31:26.264315+0000","last_undegraded":"2023-10-06T02:31:26.264315+0000","last_fullsized":"2023-10-06T02:31:26.264315+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:12:45.858237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.239903+0000","last_change":"2023-10-06T02:31:25.236085+0000","last_active":"2023-10-06T02:31:26.239903+0000","last_peered":"2023-10-06T02:31:26.239903+0000","last_clean":"2023-10-06T02:31:26.239903+0000","last_became_active":"2023-10-06T02:31:25.235827+0000","last_became_peered":"2023-10-06T02:31:25.235827+0000","last_unstale":"2023-10-06T02:31:26.239903+0000","last_undegraded":"2023-10-06T02:31:26.239903+0000","last_fullsized":"2023-10-06T02:31:26.239903+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_clean_scrub_stamp":"2023-10-06T02:31:24.211070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:25:01.375389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.268856+0000","last_change":"2023-10-06T02:31:26.264440+0000","last_active":"2023-10-06T02:31:26.268856+0000","last_peered":"2023-10-06T02:31:26.268856+0000","last_clean":"2023-10-06T02:31:26.268856+0000","last_became_active":"2023-10-06T02:31:26.263802+0000","last_became_peered":"2023-10-06T02:31:26.263802+0000","last_unstale":"2023-10-06T02:31:26.268856+0000","last_undegraded":"2023-10-06T02:31:26.268856+0000","last_fullsized":"2023-10-06T02:31:26.268856+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:53:31.912429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.239666+0000","last_change":"2023-10-06T02:31:25.246312+0000","last_active":"2023-10-06T02:31:26.239666+0000","last_peered":"2023-10-06T02:31:26.239666+0000","last_clean":"2023-10-06T02:31:26.239666+0000","last_became_active":"2023-10-06T02:31:25.246039+0000","last_became_peered":"2023-10-06T02:31:25.246039+0000","last_unstale":"2023-10-06T02:31:26.239666+0000","last_undegraded":"2023-10-06T02:31:26.239666+0000","last_fullsized":"2023-10-06T02:31:26.239666+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_clean_scrub_stamp":"2023-10-06T02:31:24.211070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:45:40.445239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.1f","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.245899+0000","last_change":"2023-10-06T02:31:26.244615+0000","last_active":"2023-10-06T02:31:26.245899+0000","last_peered":"2023-10-06T02:31:26.245899+0000","last_clean":"2023-10-06T02:31:26.245899+0000","last_became_active":"2023-10-06T02:31:26.244294+0000","last_became_peered":"2023-10-06T02:31:26.244294+0000","last_unstale":"2023-10-06T02:31:26.245899+0000","last_undegraded":"2023-10-06T02:31:26.245899+0000","last_fullsized":"2023-10-06T02:31:26.245899+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:30.357921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1c","version":"155'2","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.279892+0000","last_change":"2023-10-06T02:31:25.231065+0000","last_active":"2023-10-06T02:31:26.279892+0000","last_peered":"2023-10-06T02:31:26.279892+0000","last_clean":"2023-10-06T02:31:26.279892+0000","last_became_active":"2023-10-06T02:31:25.230438+0000","last_became_peered":"2023-10-06T02:31:25.230438+0000","last_unstale":"2023-10-06T02:31:26.279892+0000","last_undegraded":"2023-10-06T02:31:26.279892+0000","last_fullsized":"2023-10-06T02:31:26.279892+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_clean_scrub_stamp":"2023-10-06T02:31:24.211070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:53:48.222531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":124,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.239350+0000","last_change":"2023-10-06T02:23:01.992935+0000","last_active":"2023-10-06T02:31:26.239350+0000","last_peered":"2023-10-06T02:31:26.239350+0000","last_clean":"2023-10-06T02:31:26.239350+0000","last_became_active":"2023-10-06T02:23:01.992568+0000","last_became_peered":"2023-10-06T02:23:01.992568+0000","last_unstale":"2023-10-06T02:31:26.239350+0000","last_undegraded":"2023-10-06T02:31:26.239350+0000","last_fullsized":"2023-10-06T02:31:26.239350+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:06.212251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.1e","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.269959+0000","last_change":"2023-10-06T02:31:26.261651+0000","last_active":"2023-10-06T02:31:26.269959+0000","last_peered":"2023-10-06T02:31:26.269959+0000","last_clean":"2023-10-06T02:31:26.269959+0000","last_became_active":"2023-10-06T02:31:26.261365+0000","last_became_peered":"2023-10-06T02:31:26.261365+0000","last_unstale":"2023-10-06T02:31:26.269959+0000","last_undegraded":"2023-10-06T02:31:26.269959+0000","last_fullsized":"2023-10-06T02:31:26.269959+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:06:51.338576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1d","version":"155'2","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.277227+0000","last_change":"2023-10-06T02:31:25.235638+0000","last_active":"2023-10-06T02:31:26.277227+0000","last_peered":"2023-10-06T02:31:26.277227+0000","last_clean":"2023-10-06T02:31:26.277227+0000","last_became_active":"2023-10-06T02:31:25.235378+0000","last_became_peered":"2023-10-06T02:31:25.235378+0000","last_unstale":"2023-10-06T02:31:26.277227+0000","last_undegraded":"2023-10-06T02:31:26.277227+0000","last_fullsized":"2023-10-06T02:31:26.277227+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_clean_scrub_stamp":"2023-10-06T02:31:24.211070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:46:02.102704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":125,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.250973+0000","last_change":"2023-10-06T02:23:01.989934+0000","last_active":"2023-10-06T02:31:26.250973+0000","last_peered":"2023-10-06T02:31:26.250973+0000","last_clean":"2023-10-06T02:31:26.250973+0000","last_became_active":"2023-10-06T02:23:01.989706+0000","last_became_peered":"2023-10-06T02:23:01.989706+0000","last_unstale":"2023-10-06T02:31:26.250973+0000","last_undegraded":"2023-10-06T02:31:26.250973+0000","last_fullsized":"2023-10-06T02:31:26.250973+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:06.212251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.1d","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.258334+0000","last_change":"2023-10-06T02:31:26.255636+0000","last_active":"2023-10-06T02:31:26.258334+0000","last_peered":"2023-10-06T02:31:26.258334+0000","last_clean":"2023-10-06T02:31:26.258334+0000","last_became_active":"2023-10-06T02:31:26.255277+0000","last_became_peered":"2023-10-06T02:31:26.255277+0000","last_unstale":"2023-10-06T02:31:26.258334+0000","last_undegraded":"2023-10-06T02:31:26.258334+0000","last_fullsized":"2023-10-06T02:31:26.258334+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:59:02.106529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"155'1","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.280240+0000","last_change":"2023-10-06T02:31:25.243505+0000","last_active":"2023-10-06T02:31:26.280240+0000","last_peered":"2023-10-06T02:31:26.280240+0000","last_clean":"2023-10-06T02:31:26.280240+0000","last_became_active":"2023-10-06T02:31:25.243181+0000","last_became_peered":"2023-10-06T02:31:25.243181+0000","last_unstale":"2023-10-06T02:31:26.280240+0000","last_undegraded":"2023-10-06T02:31:26.280240+0000","last_fullsized":"2023-10-06T02:31:26.280240+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_clean_scrub_stamp":"2023-10-06T02:31:24.211070+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:54:51.160175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":119,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.248316+0000","last_change":"2023-10-06T02:23:02.054069+0000","last_active":"2023-10-06T02:31:26.248316+0000","last_peered":"2023-10-06T02:31:26.248316+0000","last_clean":"2023-10-06T02:31:26.248316+0000","last_became_active":"2023-10-06T02:23:02.053739+0000","last_became_peered":"2023-10-06T02:23:02.053739+0000","last_unstale":"2023-10-06T02:31:26.248316+0000","last_undegraded":"2023-10-06T02:31:26.248316+0000","last_fullsized":"2023-10-06T02:31:26.248316+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:57:48.822608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.267702+0000","last_change":"2023-10-06T02:31:26.265761+0000","last_active":"2023-10-06T02:31:26.267702+0000","last_peered":"2023-10-06T02:31:26.267702+0000","last_clean":"2023-10-06T02:31:26.267702+0000","last_became_active":"2023-10-06T02:31:26.265416+0000","last_became_peered":"2023-10-06T02:31:26.265416+0000","last_unstale":"2023-10-06T02:31:26.267702+0000","last_undegraded":"2023-10-06T02:31:26.267702+0000","last_fullsized":"2023-10-06T02:31:26.267702+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:25:17.234017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.1b","version":"155'1","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.280203+0000","last_change":"2023-10-06T02:31:25.244371+0000","last_active":"2023-10-06T02:31:26.280203+0000","last_peered":"2023-10-06T02:31:26.280203+0000","last_clean":"2023-10-06T02:31:26.280203+0000","last_became_active":"2023-10-06T02:31:25.243583+0000","last_became_peered":"2023-10-06T02:31:25.243583+0000","last_unstale":"2023-10-06T02:31:26.280203+0000","last_undegraded":"2023-10-06T02:31:26.280203+0000","last_fullsized":"2023-10-06T02:31:26.280203+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_clean_scrub_stamp":"2023-10-06T02:31:24.211070+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:40:02.130196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":119,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.250562+0000","last_change":"2023-10-06T02:23:01.990773+0000","last_active":"2023-10-06T02:31:26.250562+0000","last_peered":"2023-10-06T02:31:26.250562+0000","last_clean":"2023-10-06T02:31:26.250562+0000","last_became_active":"2023-10-06T02:23:01.990511+0000","last_became_peered":"2023-10-06T02:23:01.990511+0000","last_unstale":"2023-10-06T02:31:26.250562+0000","last_undegraded":"2023-10-06T02:31:26.250562+0000","last_fullsized":"2023-10-06T02:31:26.250562+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:42:54.797731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1b","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.265088+0000","last_change":"2023-10-06T02:31:26.259210+0000","last_active":"2023-10-06T02:31:26.265088+0000","last_peered":"2023-10-06T02:31:26.265088+0000","last_clean":"2023-10-06T02:31:26.265088+0000","last_became_active":"2023-10-06T02:31:26.258923+0000","last_became_peered":"2023-10-06T02:31:26.258923+0000","last_unstale":"2023-10-06T02:31:26.265088+0000","last_undegraded":"2023-10-06T02:31:26.265088+0000","last_fullsized":"2023-10-06T02:31:26.265088+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:35:10.584911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.244477+0000","last_change":"2023-10-06T02:31:25.235892+0000","last_active":"2023-10-06T02:31:26.244477+0000","last_peered":"2023-10-06T02:31:26.244477+0000","last_clean":"2023-10-06T02:31:26.244477+0000","last_became_active":"2023-10-06T02:31:25.235626+0000","last_became_peered":"2023-10-06T02:31:25.235626+0000","last_unstale":"2023-10-06T02:31:26.244477+0000","last_undegraded":"2023-10-06T02:31:26.244477+0000","last_fullsized":"2023-10-06T02:31:26.244477+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_clean_scrub_stamp":"2023-10-06T02:31:24.211070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:33:18.206678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"76'128","reported_seq":311,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.250649+0000","last_change":"2023-10-06T02:22:55.032519+0000","last_active":"2023-10-06T02:31:26.250649+0000","last_peered":"2023-10-06T02:31:26.250649+0000","last_clean":"2023-10-06T02:31:26.250649+0000","last_became_active":"2023-10-06T02:22:55.031367+0000","last_became_peered":"2023-10-06T02:22:55.031367+0000","last_unstale":"2023-10-06T02:31:26.250649+0000","last_undegraded":"2023-10-06T02:31:26.250649+0000","last_fullsized":"2023-10-06T02:31:26.250649+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:22:53.309097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:22:53.309097+0000","last_clean_scrub_stamp":"2023-10-06T02:22:53.309097+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:53:46.520909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":146,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.241040+0000","last_change":"2023-10-06T02:23:01.988140+0000","last_active":"2023-10-06T02:31:26.241040+0000","last_peered":"2023-10-06T02:31:26.241040+0000","last_clean":"2023-10-06T02:31:26.241040+0000","last_became_active":"2023-10-06T02:23:01.987769+0000","last_became_peered":"2023-10-06T02:23:01.987769+0000","last_unstale":"2023-10-06T02:31:26.241040+0000","last_undegraded":"2023-10-06T02:31:26.241040+0000","last_fullsized":"2023-10-06T02:31:26.241040+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:20:08.391912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.264621+0000","last_change":"2023-10-06T02:31:26.254985+0000","last_active":"2023-10-06T02:31:26.264621+0000","last_peered":"2023-10-06T02:31:26.264621+0000","last_clean":"2023-10-06T02:31:26.264621+0000","last_became_active":"2023-10-06T02:31:26.254597+0000","last_became_peered":"2023-10-06T02:31:26.254597+0000","last_unstale":"2023-10-06T02:31:26.264621+0000","last_undegraded":"2023-10-06T02:31:26.264621+0000","last_fullsized":"2023-10-06T02:31:26.264621+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:55:50.761151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1a","version":"155'1","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.275434+0000","last_change":"2023-10-06T02:31:25.234189+0000","last_active":"2023-10-06T02:31:26.275434+0000","last_peered":"2023-10-06T02:31:26.275434+0000","last_clean":"2023-10-06T02:31:26.275434+0000","last_became_active":"2023-10-06T02:31:25.233908+0000","last_became_peered":"2023-10-06T02:31:25.233908+0000","last_unstale":"2023-10-06T02:31:26.275434+0000","last_undegraded":"2023-10-06T02:31:26.275434+0000","last_fullsized":"2023-10-06T02:31:26.275434+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_clean_scrub_stamp":"2023-10-06T02:31:24.211070+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:40:02.130196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":134,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.239982+0000","last_change":"2023-10-06T02:23:02.054338+0000","last_active":"2023-10-06T02:31:26.239982+0000","last_peered":"2023-10-06T02:31:26.239982+0000","last_clean":"2023-10-06T02:31:26.239982+0000","last_became_active":"2023-10-06T02:23:02.053991+0000","last_became_peered":"2023-10-06T02:23:02.053991+0000","last_unstale":"2023-10-06T02:31:26.239982+0000","last_undegraded":"2023-10-06T02:31:26.239982+0000","last_fullsized":"2023-10-06T02:31:26.239982+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:39:30.192838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.1a","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.269807+0000","last_change":"2023-10-06T02:31:26.268448+0000","last_active":"2023-10-06T02:31:26.269807+0000","last_peered":"2023-10-06T02:31:26.269807+0000","last_clean":"2023-10-06T02:31:26.269807+0000","last_became_active":"2023-10-06T02:31:26.268093+0000","last_became_peered":"2023-10-06T02:31:26.268093+0000","last_unstale":"2023-10-06T02:31:26.269807+0000","last_undegraded":"2023-10-06T02:31:26.269807+0000","last_fullsized":"2023-10-06T02:31:26.269807+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:18:33.267912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.248295+0000","last_change":"2023-10-06T02:31:25.238923+0000","last_active":"2023-10-06T02:31:26.248295+0000","last_peered":"2023-10-06T02:31:26.248295+0000","last_clean":"2023-10-06T02:31:26.248295+0000","last_became_active":"2023-10-06T02:31:25.238641+0000","last_became_peered":"2023-10-06T02:31:25.238641+0000","last_unstale":"2023-10-06T02:31:26.248295+0000","last_undegraded":"2023-10-06T02:31:26.248295+0000","last_fullsized":"2023-10-06T02:31:26.248295+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_clean_scrub_stamp":"2023-10-06T02:31:24.211070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:45:30.418711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":146,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.240017+0000","last_change":"2023-10-06T02:23:01.988631+0000","last_active":"2023-10-06T02:31:26.240017+0000","last_peered":"2023-10-06T02:31:26.240017+0000","last_clean":"2023-10-06T02:31:26.240017+0000","last_became_active":"2023-10-06T02:23:01.988303+0000","last_became_peered":"2023-10-06T02:23:01.988303+0000","last_unstale":"2023-10-06T02:31:26.240017+0000","last_undegraded":"2023-10-06T02:31:26.240017+0000","last_fullsized":"2023-10-06T02:31:26.240017+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:38.211466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1c","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.263651+0000","last_change":"2023-10-06T02:31:26.259907+0000","last_active":"2023-10-06T02:31:26.263651+0000","last_peered":"2023-10-06T02:31:26.263651+0000","last_clean":"2023-10-06T02:31:26.263651+0000","last_became_active":"2023-10-06T02:31:26.259641+0000","last_became_peered":"2023-10-06T02:31:26.259641+0000","last_unstale":"2023-10-06T02:31:26.263651+0000","last_undegraded":"2023-10-06T02:31:26.263651+0000","last_fullsized":"2023-10-06T02:31:26.263651+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:39:16.981754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1f","version":"155'4","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.275796+0000","last_change":"2023-10-06T02:31:25.227308+0000","last_active":"2023-10-06T02:31:26.275796+0000","last_peered":"2023-10-06T02:31:26.275796+0000","last_clean":"2023-10-06T02:31:26.275796+0000","last_became_active":"2023-10-06T02:31:25.226372+0000","last_became_peered":"2023-10-06T02:31:25.226372+0000","last_unstale":"2023-10-06T02:31:26.275796+0000","last_undegraded":"2023-10-06T02:31:26.275796+0000","last_fullsized":"2023-10-06T02:31:26.275796+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_clean_scrub_stamp":"2023-10-06T02:31:24.211070+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:26:39.899368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":125,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.250065+0000","last_change":"2023-10-06T02:23:02.054039+0000","last_active":"2023-10-06T02:31:26.250065+0000","last_peered":"2023-10-06T02:31:26.250065+0000","last_clean":"2023-10-06T02:31:26.250065+0000","last_became_active":"2023-10-06T02:23:02.053742+0000","last_became_peered":"2023-10-06T02:23:02.053742+0000","last_unstale":"2023-10-06T02:31:26.250065+0000","last_undegraded":"2023-10-06T02:31:26.250065+0000","last_fullsized":"2023-10-06T02:31:26.250065+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:58:18.846475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":25,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574956,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303884,"kb_used_data":2500,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067956,"statfs":{"total":96636764160,"available":96325586944,"internally_reserved":0,"allocated":2560000,"data_stored":1507692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:31:14 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.622,"15min":0.63300000000000001},"min":{"1min":0.35299999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.78300000000000003,"5min":1.169,"15min":1.169},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.624,"15min":0.61199999999999999},"min":{"1min":0.43099999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.91800000000000004,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.75600000000000001}]},{"osd":1,"last update":"Fri Oct 6 02:31:14 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71399999999999997,"15min":0.69699999999999995},"min":{"1min":0.57899999999999996,"5min":0.45600000000000002,"15min":0.40300000000000002},"max":{"1min":0.86199999999999999,"5min":1.1459999999999999,"15min":1.147},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65900000000000003,"15min":0.67100000000000004},"min":{"1min":0.52100000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83899999999999997,"5min":1.129,"15min":1.129},"last":0.77700000000000002}]},{"osd":2,"last update":"Fri Oct 6 02:31:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.65500000000000003,"15min":0.68300000000000005},"min":{"1min":0.46700000000000003,"5min":0.375,"15min":0.375},"max":{"1min":0.81699999999999995,"5min":1.238,"15min":1.238},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.68700000000000006,"15min":0.69099999999999995},"min":{"1min":0.55000000000000004,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.93700000000000006,"5min":1.242,"15min":1.242},"last":0.67900000000000005}]},{"osd":3,"last update":"Fri Oct 6 02:31:14 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.70799999999999996,"15min":0.72399999999999998},"min":{"1min":0.45500000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.84299999999999997,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.75700000000000001,"15min":0.77200000000000002},"min":{"1min":0.61399999999999999,"5min":0.54500000000000004,"15min":0.497},"max":{"1min":0.95199999999999996,"5min":1.383,"15min":1.383},"last":0.82399999999999995}]},{"osd":4,"last update":"Fri Oct 6 02:31:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.60299999999999998,"15min":0.66700000000000004},"min":{"1min":0.41799999999999998,"5min":0.41099999999999998,"15min":0.35099999999999998},"max":{"1min":0.91700000000000004,"5min":1.1120000000000001,"15min":1.2849999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.67300000000000004,"15min":0.73899999999999999},"min":{"1min":0.40899999999999997,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.96299999999999997,"5min":1.51,"15min":1.51},"last":0.71099999999999997}]},{"osd":5,"last update":"Fri Oct 6 02:31:14 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.70099999999999996,"15min":0.72499999999999998},"min":{"1min":0.503,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.91900000000000004,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.67000000000000004,"15min":0.71599999999999997},"min":{"1min":0.48999999999999999,"5min":0.442,"15min":0.442},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0289999999999999},"last":0.73299999999999998}]},{"osd":6,"last update":"Fri Oct 6 02:31:14 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.752,"15min":0.77800000000000002},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.93400000000000005,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.77100000000000002,"15min":0.80300000000000005},"min":{"1min":0.628,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.90500000000000003,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.90400000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574956,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303308,"kb_used_data":1924,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068532,"statfs":{"total":96636764160,"available":96326176768,"internally_reserved":0,"allocated":1970176,"data_stored":917394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:31:09 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.67400000000000004,"15min":0.66700000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.72099999999999997,"15min":0.70399999999999996},"min":{"1min":0.498,"5min":0.374,"15min":0.374},"max":{"1min":1.04,"5min":1.04,"15min":1.119},"last":0.51000000000000001}]},{"osd":1,"last update":"Fri Oct 6 02:31:09 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.66800000000000004,"15min":0.65000000000000002},"min":{"1min":0.48699999999999999,"5min":0.433,"15min":0.40699999999999997},"max":{"1min":0.83799999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.66600000000000004,"15min":0.66700000000000004},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.94499999999999995,"5min":0.997,"15min":0.997},"last":0.56999999999999995}]},{"osd":2,"last update":"Fri Oct 6 02:31:09 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.71599999999999997,"15min":0.70599999999999996},"min":{"1min":0.51000000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.002,"5min":1.0449999999999999,"15min":1.137},"last":0.748},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.72799999999999998,"15min":0.71799999999999997},"min":{"1min":0.47799999999999998,"5min":0.45300000000000001,"15min":0.40300000000000002},"max":{"1min":0.98699999999999999,"5min":1.077,"15min":1.077},"last":0.71299999999999997}]},{"osd":3,"last update":"Fri Oct 6 02:31:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.70099999999999996,"15min":0.67200000000000004},"min":{"1min":0.51500000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.82099999999999995,"5min":1.056,"15min":1.056},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.71399999999999997,"15min":0.70699999999999996},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.45900000000000002},"max":{"1min":0.91600000000000004,"5min":0.97999999999999998,"15min":0.98499999999999999},"last":0.69499999999999995}]},{"osd":4,"last update":"Fri Oct 6 02:31:09 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.66500000000000004,"15min":0.64500000000000002},"min":{"1min":0.377,"5min":0.36199999999999999,"15min":0.34300000000000003},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":1.095},"last":0.63},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.71299999999999997,"15min":0.67500000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.998,"5min":1.002,"15min":1.29},"last":0.65300000000000002}]},{"osd":5,"last update":"Fri Oct 6 02:31:09 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.69799999999999995,"15min":0.69999999999999996},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.39700000000000002},"max":{"1min":0.98699999999999999,"5min":1.014,"15min":1.2929999999999999},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.71999999999999997,"15min":0.70899999999999996},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.81799999999999995}]},{"osd":7,"last update":"Fri Oct 6 02:31:09 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.755,"15min":0.751},"min":{"1min":0.44700000000000001,"5min":0.41499999999999998,"15min":0.36799999999999999},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.458},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.77700000000000002,"15min":0.76500000000000001},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.45900000000000002},"max":{"1min":1.0429999999999999,"5min":1.103,"15min":1.2470000000000001},"last":0.875}]}]},{"osd":1,"up_from":13,"seq":55834574955,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303304,"kb_used_data":1920,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068536,"statfs":{"total":96636764160,"available":96326180864,"internally_reserved":0,"allocated":1966080,"data_stored":917324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:31:05 2023","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.40999999999999998,"15min":0.44700000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.46999999999999997,"5min":0.63600000000000001,"15min":1.2230000000000001},"last":0.38},{"interface":"front","average":{"1min":0.434,"5min":0.45600000000000002,"15min":0.50600000000000001},"min":{"1min":0.32800000000000001,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.64500000000000002,"5min":0.71799999999999997,"15min":1.788},"last":0.60099999999999998}]},{"osd":2,"last update":"Fri Oct 6 02:31:05 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.433,"15min":0.498},"min":{"1min":0.35199999999999998,"5min":0.307,"15min":0.28299999999999997},"max":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":2.0840000000000001},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.49099999999999999,"15min":0.54000000000000004},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.60599999999999998,"5min":0.88100000000000001,"15min":1.6699999999999999},"last":0.5}]},{"osd":3,"last update":"Fri Oct 6 02:31:05 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.58099999999999996,"15min":0.60599999999999998},"min":{"1min":0.43099999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.67000000000000004,"5min":0.89900000000000002,"15min":1.266},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.53500000000000003,"15min":0.60999999999999999},"min":{"1min":0.41599999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.77900000000000003,"5min":0.79400000000000004,"15min":1.861},"last":0.436}]},{"osd":4,"last update":"Fri Oct 6 02:31:05 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.74299999999999999,"15min":0.82699999999999996},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.006,"5min":1.3580000000000001,"15min":1.9279999999999999},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.752,"15min":0.82999999999999996},"min":{"1min":0.46899999999999997,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.98999999999999999,"5min":1.4339999999999999,"15min":2.0179999999999998},"last":0.73199999999999998}]},{"osd":5,"last update":"Fri Oct 6 02:31:05 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.71199999999999997,"15min":0.82099999999999995},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.016,"5min":1.3069999999999999,"15min":1.9830000000000001},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.72399999999999998,"15min":0.79700000000000004},"min":{"1min":0.54800000000000004,"5min":0.53600000000000003,"15min":0.47299999999999998},"max":{"1min":0.999,"5min":1.327,"15min":1.3380000000000001},"last":0.80900000000000005}]},{"osd":6,"last update":"Fri Oct 6 02:31:05 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.75600000000000001,"15min":0.83699999999999997},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.45600000000000002},"max":{"1min":0.98299999999999998,"5min":1.0609999999999999,"15min":1.829},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.77600000000000002,"15min":0.876},"min":{"1min":0.57899999999999996,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.94699999999999995,"5min":1.351,"15min":2.052},"last":0.67400000000000004}]},{"osd":7,"last update":"Fri Oct 6 02:31:05 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.73299999999999998,"15min":0.80900000000000005},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.93899999999999995,"5min":1.3120000000000001,"15min":1.3340000000000001},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.751,"15min":0.84399999999999997},"min":{"1min":0.52700000000000002,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.0369999999999999,"5min":1.2110000000000001,"15min":1.895},"last":0.88200000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574956,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303900,"kb_used_data":2516,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067940,"statfs":{"total":96636764160,"available":96325570560,"internally_reserved":0,"allocated":2576384,"data_stored":1508392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 02:31:15 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.44600000000000001,"15min":0.45300000000000001},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.76400000000000001,"5min":2.6120000000000001,"15min":2.6120000000000001},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.46999999999999997,"15min":0.47999999999999998},"min":{"1min":0.30599999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.77900000000000003,"5min":3.4199999999999999,"15min":3.4199999999999999},"last":0.438}]},{"osd":2,"last update":"Fri Oct 6 02:31:15 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.52000000000000002,"15min":0.51500000000000001},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.58199999999999996,"5min":3.0339999999999998,"15min":3.0339999999999998},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.70699999999999996,"5min":3.343,"15min":3.343},"last":0.58799999999999997}]},{"osd":3,"last update":"Fri Oct 6 02:31:15 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.505,"15min":0.50600000000000001},"min":{"1min":0.34399999999999997,"5min":0.31,"15min":0.31},"max":{"1min":0.79400000000000004,"5min":3.6579999999999999,"15min":3.6579999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.59099999999999997,"15min":0.59399999999999997},"min":{"1min":0.443,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.82099999999999995,"5min":2.6890000000000001,"15min":2.6890000000000001},"last":0.65700000000000003}]},{"osd":4,"last update":"Fri Oct 6 02:31:15 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.76400000000000001,"15min":0.79200000000000004},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.94199999999999995,"5min":3.2999999999999998,"15min":3.2999999999999998},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.755,"15min":0.78900000000000003},"min":{"1min":0.54400000000000004,"5min":0.51000000000000001,"15min":0.48499999999999999},"max":{"1min":1.085,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":0.89700000000000002}]},{"osd":5,"last update":"Fri Oct 6 02:31:15 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.74299999999999999,"15min":0.77600000000000002},"min":{"1min":0.51200000000000001,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.038,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.79000000000000004,"15min":0.79900000000000004},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.071,"5min":3.1110000000000002,"15min":3.1110000000000002},"last":0.81599999999999995}]},{"osd":6,"last update":"Fri Oct 6 02:31:15 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.79600000000000004,"15min":0.81599999999999995},"min":{"1min":0.52900000000000003,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.103,"5min":3.1600000000000001,"15min":3.1600000000000001},"last":0.878},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.76900000000000002,"15min":0.81299999999999994},"min":{"1min":0.56200000000000006,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.01,"5min":2.3250000000000002,"15min":2.3250000000000002},"last":0.90900000000000003}]},{"osd":7,"last update":"Fri Oct 6 02:31:15 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.81399999999999995,"15min":0.82399999999999995},"min":{"1min":0.57999999999999996,"5min":0.56899999999999995,"15min":0.56000000000000005},"max":{"1min":1.048,"5min":3.3809999999999998,"15min":3.3809999999999998},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.753,"5min":0.77200000000000002,"15min":0.81200000000000006},"min":{"1min":0.56100000000000005,"5min":0.54700000000000004,"15min":0.53300000000000003},"max":{"1min":1.022,"5min":2.3740000000000001,"15min":2.3740000000000001},"last":0.92400000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574957,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303308,"kb_used_data":1924,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068532,"statfs":{"total":96636764160,"available":96326176768,"internally_reserved":0,"allocated":1970176,"data_stored":917878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:31:14 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.41999999999999998,"15min":0.435},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.63500000000000001,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.46500000000000002,"15min":0.47999999999999998},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.65300000000000002,"5min":0.88700000000000001,"15min":1.0800000000000001},"last":0.63500000000000001}]},{"osd":1,"last update":"Fri Oct 6 02:31:14 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47499999999999998,"15min":0.496},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.32400000000000001},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.51000000000000001,"15min":0.57899999999999996},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.622,"5min":1.778,"15min":1.778},"last":0.47499999999999998}]},{"osd":3,"last update":"Fri Oct 6 02:31:14 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.499,"15min":0.52500000000000002},"min":{"1min":0.33700000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.68400000000000005,"5min":0.83699999999999997,"15min":0.97499999999999998},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.57799999999999996,"15min":0.60899999999999999},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":1.028},"last":0.49399999999999999}]},{"osd":4,"last update":"Fri Oct 6 02:31:14 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.746,"15min":0.76900000000000002},"min":{"1min":0.496,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.1319999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.76200000000000001,"15min":0.82499999999999996},"min":{"1min":0.51200000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.95099999999999996,"5min":1.907,"15min":1.907},"last":0.747}]},{"osd":5,"last update":"Fri Oct 6 02:31:14 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.73299999999999998,"15min":0.78200000000000003},"min":{"1min":0.47599999999999998,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.024,"5min":1.109,"15min":1.1850000000000001},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.745,"5min":0.752,"15min":0.80400000000000005},"min":{"1min":0.56200000000000006,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.258},"last":0.624}]},{"osd":6,"last update":"Fri Oct 6 02:31:14 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.78500000000000003,"15min":0.83099999999999996},"min":{"1min":0.48399999999999999,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.1100000000000001,"5min":1.827,"15min":1.827},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.747,"5min":0.76700000000000002,"15min":0.78900000000000003},"min":{"1min":0.56100000000000005,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0269999999999999,"5min":1.0660000000000001,"15min":1.1659999999999999},"last":0.60899999999999999}]},{"osd":7,"last update":"Fri Oct 6 02:31:14 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.79400000000000004,"15min":0.86099999999999999},"min":{"1min":0.57099999999999995,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0649999999999999,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.751,"5min":0.77900000000000003,"15min":0.81799999999999995},"min":{"1min":0.56100000000000005,"5min":0.55300000000000005,"15min":0.54800000000000004},"max":{"1min":1.0269999999999999,"5min":1.125,"15min":1.1479999999999999},"last":0.755}]}]},{"osd":3,"up_from":13,"seq":55834574954,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303312,"kb_used_data":1928,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068528,"statfs":{"total":96636764160,"available":96326172672,"internally_reserved":0,"allocated":1974272,"data_stored":917484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:31:12 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.434,"15min":0.48799999999999999},"min":{"1min":0.29099999999999998,"5min":0.248,"15min":0.248},"max":{"1min":0.60799999999999998,"5min":1.0489999999999999,"15min":2.024},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.441,"5min":0.46800000000000003,"15min":0.50900000000000001},"min":{"1min":0.31900000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.53300000000000003,"5min":0.754,"15min":2.181},"last":0.33200000000000002}]},{"osd":1,"last update":"Fri Oct 6 02:31:12 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.45800000000000002,"15min":0.62},"min":{"1min":0.33400000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.61299999999999999,"5min":0.66100000000000003,"15min":4.492},"last":0.443},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.51600000000000001,"15min":0.66800000000000004},"min":{"1min":0.377,"5min":0.313,"15min":0.313},"max":{"1min":0.67100000000000004,"5min":1.079,"15min":4.569},"last":0.48299999999999998}]},{"osd":2,"last update":"Fri Oct 6 02:31:12 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.59699999999999998,"15min":0.64800000000000002},"min":{"1min":0.46899999999999997,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.79400000000000004,"5min":1.024,"15min":2.0870000000000002},"last":0.626},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.56699999999999995,"15min":0.627},"min":{"1min":0.42799999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.72999999999999998,"5min":0.94199999999999995,"15min":2.4260000000000002},"last":0.50800000000000001}]},{"osd":4,"last update":"Fri Oct 6 02:31:12 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73299999999999998,"15min":0.80200000000000005},"min":{"1min":0.58199999999999996,"5min":0.55000000000000004,"15min":0.44800000000000001},"max":{"1min":0.97299999999999998,"5min":1.0149999999999999,"15min":2.214},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.75,"15min":0.79600000000000004},"min":{"1min":0.53000000000000003,"5min":0.50800000000000001,"15min":0.5},"max":{"1min":1.042,"5min":1.042,"15min":1.2869999999999999},"last":0.83299999999999996}]},{"osd":5,"last update":"Fri Oct 6 02:31:12 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.747,"15min":0.80300000000000005},"min":{"1min":0.55300000000000005,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.97499999999999998,"5min":1.077,"15min":1.3029999999999999},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.754,"5min":0.75600000000000001,"15min":0.82599999999999996},"min":{"1min":0.57999999999999996,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.0549999999999999,"5min":1.0580000000000001,"15min":2.242},"last":0.79500000000000004}]},{"osd":6,"last update":"Fri Oct 6 02:31:12 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.77300000000000002,"15min":0.80500000000000005},"min":{"1min":0.58199999999999996,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.96199999999999997,"5min":1.0489999999999999,"15min":1.357},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.78800000000000003,"15min":0.82099999999999995},"min":{"1min":0.64900000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0049999999999999,"5min":1.0529999999999999,"15min":1.3100000000000001},"last":0.92200000000000004}]},{"osd":7,"last update":"Fri Oct 6 02:31:12 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.76600000000000001,"15min":0.83399999999999996},"min":{"1min":0.54400000000000004,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.98399999999999999,"5min":1.077,"15min":2.27},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.78600000000000003,"15min":0.83899999999999997},"min":{"1min":0.57599999999999996,"5min":0.55600000000000005,"15min":0.53200000000000003},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.325},"last":0.76700000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574956,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303320,"kb_used_data":1936,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068520,"statfs":{"total":96636764160,"available":96326164480,"internally_reserved":0,"allocated":1982464,"data_stored":918894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:31:06 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.65200000000000002,"15min":0.92200000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.84799999999999998,"5min":1.0760000000000001,"15min":12.337},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.67800000000000005,"15min":0.85899999999999999},"min":{"1min":0.48499999999999999,"5min":0.44400000000000001,"15min":0.438},"max":{"1min":0.94299999999999995,"5min":1.3520000000000001,"15min":9.6539999999999999},"last":0.77200000000000002}]},{"osd":1,"last update":"Fri Oct 6 02:31:06 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.66300000000000003,"15min":0.68899999999999995},"min":{"1min":0.42099999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.97899999999999998,"5min":1.129,"15min":3.181},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.70199999999999996,"15min":0.94999999999999996},"min":{"1min":0.45300000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.92700000000000005,"5min":0.995,"15min":12.946999999999999},"last":0.52400000000000002}]},{"osd":2,"last update":"Fri Oct 6 02:31:06 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.69999999999999996,"15min":0.96799999999999997},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.89200000000000002,"5min":1.2989999999999999,"15min":12.377000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.72699999999999998,"15min":0.83899999999999997},"min":{"1min":0.51600000000000001,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.90800000000000003,"5min":1.0529999999999999,"15min":6.3220000000000001},"last":0.63800000000000001}]},{"osd":3,"last update":"Fri Oct 6 02:31:06 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72599999999999998,"15min":0.93100000000000005},"min":{"1min":0.54900000000000004,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.91100000000000003,"5min":1.5229999999999999,"15min":9.7720000000000002},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.70999999999999996,"15min":0.76000000000000001},"min":{"1min":0.53400000000000003,"5min":0.498,"15min":0.47999999999999998},"max":{"1min":1.0389999999999999,"5min":1.212,"15min":3.0840000000000001},"last":0.56799999999999995}]},{"osd":5,"last update":"Fri Oct 6 02:31:06 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.70599999999999996,"15min":0.81499999999999995},"min":{"1min":0.54000000000000004,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.96299999999999997,"5min":1.02,"15min":6.2690000000000001},"last":0.753},{"interface":"front","average":{"1min":0.751,"5min":0.73999999999999999,"15min":0.96799999999999997},"min":{"1min":0.53900000000000003,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.95199999999999996,"5min":1.4359999999999999,"15min":10.769},"last":0.878}]},{"osd":6,"last update":"Fri Oct 6 02:31:06 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.70999999999999996,"15min":0.749},"min":{"1min":0.46100000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.1919999999999999,"5min":1.4590000000000001,"15min":3.1349999999999998},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79100000000000004,"15min":0.93100000000000005},"min":{"1min":0.66700000000000004,"5min":0.47799999999999998,"15min":0.45700000000000002},"max":{"1min":1.008,"5min":1.3839999999999999,"15min":8.0899999999999999},"last":0.79800000000000004}]},{"osd":7,"last update":"Fri Oct 6 02:31:06 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.79600000000000004,"15min":1.006},"min":{"1min":0.60999999999999999,"5min":0.501,"15min":0.47599999999999998},"max":{"1min":1.024,"5min":1.494,"15min":9.7089999999999996},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.78000000000000003,"15min":0.94899999999999995},"min":{"1min":0.55900000000000005,"5min":0.374,"15min":0.374},"max":{"1min":1.1399999999999999,"5min":1.2669999999999999,"15min":7.6719999999999997},"last":0.85499999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574956,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303320,"kb_used_data":1936,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068520,"statfs":{"total":96636764160,"available":96326164480,"internally_reserved":0,"allocated":1982464,"data_stored":918882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:31:05 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.624,"15min":0.627},"min":{"1min":0.38100000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.82799999999999996,"5min":0.97599999999999998,"15min":1.4279999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.63,"15min":0.61799999999999999},"min":{"1min":0.436,"5min":0.40300000000000002,"15min":0.38700000000000001},"max":{"1min":1.0409999999999999,"5min":1.099,"15min":1.099},"last":0.64300000000000002}]},{"osd":1,"last update":"Fri Oct 6 02:31:05 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.627,"15min":0.64500000000000002},"min":{"1min":0.45400000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.64700000000000002,"5min":1.0660000000000001,"15min":1.089},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.442,"5min":0.40699999999999997,"15min":0.40500000000000003},"max":{"1min":1.0649999999999999,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.84699999999999998}]},{"osd":2,"last update":"Fri Oct 6 02:31:05 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.66200000000000003,"15min":0.67000000000000004},"min":{"1min":0.48799999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.84899999999999998,"5min":1.016,"15min":1.016},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.66300000000000003,"15min":0.67900000000000005},"min":{"1min":0.53300000000000003,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.88400000000000001,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.90900000000000003}]},{"osd":3,"last update":"Fri Oct 6 02:31:05 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.71999999999999997,"15min":0.72699999999999998},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.93999999999999995,"5min":1.159,"15min":1.159},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.76300000000000001,"15min":0.76700000000000002},"min":{"1min":0.58899999999999997,"5min":0.56200000000000006,"15min":0.54100000000000004},"max":{"1min":1.0149999999999999,"5min":1.194,"15min":1.194},"last":0.86399999999999999}]},{"osd":4,"last update":"Fri Oct 6 02:31:05 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67000000000000004,"15min":0.67500000000000004},"min":{"1min":0.40799999999999997,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68300000000000005,"15min":0.68500000000000005},"min":{"1min":0.44700000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.1519999999999999,"5min":1.25,"15min":1.25},"last":0.46600000000000003}]},{"osd":6,"last update":"Fri Oct 6 02:31:05 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.71699999999999997,"15min":0.73699999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.40300000000000002},"max":{"1min":0.88200000000000001,"5min":1.0549999999999999,"15min":1.1719999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.80300000000000005,"15min":0.81799999999999995},"min":{"1min":0.60399999999999998,"5min":0.373,"15min":0.373},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.62}]},{"osd":7,"last update":"Fri Oct 6 02:31:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.745,"15min":0.76100000000000001},"min":{"1min":0.48499999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.093,"5min":1.23,"15min":1.23},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.79200000000000004,"15min":0.82399999999999995},"min":{"1min":0.47599999999999998,"5min":0.46999999999999997,"15min":0.437},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.5640000000000001},"last":0.66500000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T02:31:30.637 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T02:31:30.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.855+0000 7f093979d700 1 Processor -- start 2023-10-06T02:31:30.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.855+0000 7f093979d700 1 -- start start 2023-10-06T02:31:30.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.856+0000 7f093979d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0934140e40 0x7f09340c6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:30.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.857+0000 7f093979d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f093406de50 0x7f09340aeef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:30.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.857+0000 7f093979d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09340c6d50 con 0x7f0934140a70 2023-10-06T02:31:30.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.857+0000 7f093979d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09340c7030 con 0x7f0934140e40 2023-10-06T02:31:30.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.857+0000 7f093979d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09340c7310 con 0x7f093406de50 2023-10-06T02:31:30.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.857+0000 7f09337fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0934140e40 0x7f09340c6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:30.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.857+0000 7f09337fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0934140e40 0x7f09340c6770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51554/0 (socket says 172.21.15.31:51554) 2023-10-06T02:31:30.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.857+0000 7f09337fe700 1 -- 172.21.15.31:0/378195321 learned_addr learned my addr 172.21.15.31:0/378195321 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:30.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.857+0000 7f0938f9c700 1 --2- 172.21.15.31:0/378195321 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f093406de50 0x7f09340aeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:30.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.857+0000 7f09337fe700 1 -- 172.21.15.31:0/378195321 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f093406de50 msgr2=0x7f09340aeef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:30.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.857+0000 7f09337fe700 1 --2- 172.21.15.31:0/378195321 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f093406de50 0x7f09340aeef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:30.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.857+0000 7f09337fe700 1 -- 172.21.15.31:0/378195321 >> v1:172.21.15.62:6789/0 conn(0x7f0934140a70 legacy=0x7f09340c7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:30.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.857+0000 7f09337fe700 1 -- 172.21.15.31:0/378195321 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09340c75f0 con 0x7f0934140e40 2023-10-06T02:31:30.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.858+0000 7f09337fe700 1 --2- 172.21.15.31:0/378195321 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0934140e40 0x7f09340c6770 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f0928009990 tx=0x7f0928009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a31dd7d2ec13ceda server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:30.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.858+0000 7f0932ffd700 1 -- 172.21.15.31:0/378195321 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0928009e80 con 0x7f0934140e40 2023-10-06T02:31:30.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.858+0000 7f0932ffd700 1 -- 172.21.15.31:0/378195321 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0928005c10 con 0x7f0934140e40 2023-10-06T02:31:30.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.858+0000 7f0932ffd700 1 -- 172.21.15.31:0/378195321 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f092800b650 con 0x7f0934140e40 2023-10-06T02:31:30.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.858+0000 7f093979d700 1 -- 172.21.15.31:0/378195321 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0934140e40 msgr2=0x7f09340c6770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:30.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.858+0000 7f093979d700 1 --2- 172.21.15.31:0/378195321 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0934140e40 0x7f09340c6770 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f0928009990 tx=0x7f0928009220 comp rx=0 tx=0).stop 2023-10-06T02:31:30.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.858+0000 7f093979d700 1 -- 172.21.15.31:0/378195321 shutdown_connections 2023-10-06T02:31:30.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.858+0000 7f093979d700 1 --2- 172.21.15.31:0/378195321 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f093406de50 0x7f09340aeef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:30.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.858+0000 7f093979d700 1 --2- 172.21.15.31:0/378195321 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0934140e40 0x7f09340c6770 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:30.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.858+0000 7f093979d700 1 -- 172.21.15.31:0/378195321 >> 172.21.15.31:0/378195321 conn(0x7f093406d170 msgr2=0x7f09340cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:30.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.859+0000 7f093979d700 1 -- 172.21.15.31:0/378195321 shutdown_connections 2023-10-06T02:31:30.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.859+0000 7f093979d700 1 -- 172.21.15.31:0/378195321 wait complete. 2023-10-06T02:31:30.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.860+0000 7f093979d700 1 Processor -- start 2023-10-06T02:31:30.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.860+0000 7f093979d700 1 -- start start 2023-10-06T02:31:30.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.861+0000 7f093979d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f093406de50 0x7f093416a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:30.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.861+0000 7f093979d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0934140a70 0x7f093413e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:30.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.861+0000 7f093979d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0934140e40 0x7f09341582f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:30.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.861+0000 7f093979d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0934158c60 con 0x7f0934140a70 2023-10-06T02:31:30.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.861+0000 7f093979d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09340c3cb0 con 0x7f093406de50 2023-10-06T02:31:30.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.861+0000 7f093979d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f093413ec90 con 0x7f0934140e40 2023-10-06T02:31:30.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.861+0000 7f09337fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0934140a70 0x7f093413e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:30.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.861+0000 7f09337fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0934140a70 0x7f093413e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51558/0 (socket says 172.21.15.31:51558) 2023-10-06T02:31:30.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.861+0000 7f09337fe700 1 -- 172.21.15.31:0/1334132158 learned_addr learned my addr 172.21.15.31:0/1334132158 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:30.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.862+0000 7f0933fff700 1 --2- 172.21.15.31:0/1334132158 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f093406de50 0x7f093416a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:30.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.862+0000 7f0938f9c700 1 --2- 172.21.15.31:0/1334132158 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0934140e40 0x7f09341582f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:30.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.862+0000 7f09337fe700 1 -- 172.21.15.31:0/1334132158 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0934140e40 msgr2=0x7f09341582f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:30.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.862+0000 7f09337fe700 1 --2- 172.21.15.31:0/1334132158 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0934140e40 0x7f09341582f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:30.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.862+0000 7f09337fe700 1 -- 172.21.15.31:0/1334132158 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f093406de50 msgr2=0x7f093416a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:30.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.862+0000 7f09337fe700 1 --2- 172.21.15.31:0/1334132158 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f093406de50 0x7f093416a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:30.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.862+0000 7f09337fe700 1 -- 172.21.15.31:0/1334132158 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0928009480 con 0x7f0934140a70 2023-10-06T02:31:30.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.862+0000 7f0933fff700 1 --2- 172.21.15.31:0/1334132158 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f093406de50 0x7f093416a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:31:30.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.862+0000 7f09337fe700 1 --2- 172.21.15.31:0/1334132158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0934140a70 0x7f093413e750 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f092800b3c0 tx=0x7f0928016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:30.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.862+0000 7f091bfff700 1 -- 172.21.15.31:0/1334132158 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09280119f0 con 0x7f0934140a70 2023-10-06T02:31:30.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.863+0000 7f091bfff700 1 -- 172.21.15.31:0/1334132158 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0928020070 con 0x7f0934140a70 2023-10-06T02:31:30.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.863+0000 7f091bfff700 1 -- 172.21.15.31:0/1334132158 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0928011d80 con 0x7f0934140a70 2023-10-06T02:31:30.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.863+0000 7f093979d700 1 -- 172.21.15.31:0/1334132158 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09340c3380 con 0x7f0934140a70 2023-10-06T02:31:30.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.863+0000 7f093979d700 1 -- 172.21.15.31:0/1334132158 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09340c3940 con 0x7f0934140a70 2023-10-06T02:31:30.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.864+0000 7f093979d700 1 -- 172.21.15.31:0/1334132158 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09340c3e40 con 0x7f0934140a70 2023-10-06T02:31:30.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.864+0000 7f091bfff700 1 -- 172.21.15.31:0/1334132158 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 85403+0+0 (secure 0 0 0) 0x7f0928022030 con 0x7f0934140a70 2023-10-06T02:31:30.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.865+0000 7f091bfff700 1 --2- 172.21.15.31:0/1334132158 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0920067680 0x7f0920069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:30.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.866+0000 7f091bfff700 1 -- 172.21.15.31:0/1334132158 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f0928087380 con 0x7f0934140a70 2023-10-06T02:31:30.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.866+0000 7f0933fff700 1 --2- 172.21.15.31:0/1334132158 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0920067680 0x7f0920069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:30.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.867+0000 7f0933fff700 1 --2- 172.21.15.31:0/1334132158 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0920067680 0x7f0920069b30 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f0924007fd0 tx=0x7f0924009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:30.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:30.870+0000 7f091bfff700 1 -- 172.21.15.31:0/1334132158 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0928084740 con 0x7f0934140a70 2023-10-06T02:31:31.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.069+0000 7f093979d700 1 -- 172.21.15.31:0/1334132158 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f093404c3c0 con 0x7f0920067680 2023-10-06T02:31:31.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.079+0000 7f091bfff700 1 -- 172.21.15.31:0/1334132158 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311554 (secure 0 0 0) 0x7f093404c3c0 con 0x7f0920067680 2023-10-06T02:31:31.085 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:31:31.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.083+0000 7f093979d700 1 -- 172.21.15.31:0/1334132158 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0920067680 msgr2=0x7f0920069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:31.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.083+0000 7f093979d700 1 --2- 172.21.15.31:0/1334132158 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0920067680 0x7f0920069b30 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f0924007fd0 tx=0x7f0924009040 comp rx=0 tx=0).stop 2023-10-06T02:31:31.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.083+0000 7f093979d700 1 -- 172.21.15.31:0/1334132158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0934140a70 msgr2=0x7f093413e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:31.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.083+0000 7f093979d700 1 --2- 172.21.15.31:0/1334132158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0934140a70 0x7f093413e750 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f092800b3c0 tx=0x7f0928016000 comp rx=0 tx=0).stop 2023-10-06T02:31:31.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.084+0000 7f093979d700 1 -- 172.21.15.31:0/1334132158 shutdown_connections 2023-10-06T02:31:31.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.084+0000 7f093979d700 1 --2- 172.21.15.31:0/1334132158 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0920067680 0x7f0920069b30 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:31.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.084+0000 7f093979d700 1 --2- 172.21.15.31:0/1334132158 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0934140e40 0x7f09341582f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:31.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.084+0000 7f093979d700 1 --2- 172.21.15.31:0/1334132158 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f093406de50 0x7f093416a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:31.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.084+0000 7f093979d700 1 --2- 172.21.15.31:0/1334132158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0934140a70 0x7f093413e750 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:31.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.084+0000 7f093979d700 1 -- 172.21.15.31:0/1334132158 >> 172.21.15.31:0/1334132158 conn(0x7f093406d170 msgr2=0x7f09340c0ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:31.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.084+0000 7f093979d700 1 -- 172.21.15.31:0/1334132158 shutdown_connections 2023-10-06T02:31:31.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.084+0000 7f093979d700 1 -- 172.21.15.31:0/1334132158 wait complete. 2023-10-06T02:31:31.088 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-10-06T02:31:31.103 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":200,"stamp":"2023-10-06T02:31:30.512905+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":223,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":166,"ondisk_log_size":166,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":178,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2427656,"kb_used_data":16584,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752547064,"statfs":{"total":773094113280,"available":770608193536,"internally_reserved":0,"allocated":16982016,"data_stored":8523940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001632"},"pg_stats":[{"pgid":"26.3f","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.267990+0000","last_change":"2023-10-06T02:31:26.265201+0000","last_active":"2023-10-06T02:31:26.267990+0000","last_peered":"2023-10-06T02:31:26.267990+0000","last_clean":"2023-10-06T02:31:26.267990+0000","last_became_active":"2023-10-06T02:31:26.264924+0000","last_became_peered":"2023-10-06T02:31:26.264924+0000","last_unstale":"2023-10-06T02:31:26.267990+0000","last_undegraded":"2023-10-06T02:31:26.267990+0000","last_fullsized":"2023-10-06T02:31:26.267990+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:10:36.589251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3e","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.269621+0000","last_change":"2023-10-06T02:31:26.267681+0000","last_active":"2023-10-06T02:31:26.269621+0000","last_peered":"2023-10-06T02:31:26.269621+0000","last_clean":"2023-10-06T02:31:26.269621+0000","last_became_active":"2023-10-06T02:31:26.267405+0000","last_became_peered":"2023-10-06T02:31:26.267405+0000","last_unstale":"2023-10-06T02:31:26.269621+0000","last_undegraded":"2023-10-06T02:31:26.269621+0000","last_fullsized":"2023-10-06T02:31:26.269621+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:25:17.234017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3d","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.257654+0000","last_change":"2023-10-06T02:31:26.250873+0000","last_active":"2023-10-06T02:31:26.257654+0000","last_peered":"2023-10-06T02:31:26.257654+0000","last_clean":"2023-10-06T02:31:26.257654+0000","last_became_active":"2023-10-06T02:31:26.250538+0000","last_became_peered":"2023-10-06T02:31:26.250538+0000","last_unstale":"2023-10-06T02:31:26.257654+0000","last_undegraded":"2023-10-06T02:31:26.257654+0000","last_fullsized":"2023-10-06T02:31:26.257654+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:25:17.311269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.3c","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.257586+0000","last_change":"2023-10-06T02:31:26.251973+0000","last_active":"2023-10-06T02:31:26.257586+0000","last_peered":"2023-10-06T02:31:26.257586+0000","last_clean":"2023-10-06T02:31:26.257586+0000","last_became_active":"2023-10-06T02:31:26.251699+0000","last_became_peered":"2023-10-06T02:31:26.251699+0000","last_unstale":"2023-10-06T02:31:26.257586+0000","last_undegraded":"2023-10-06T02:31:26.257586+0000","last_fullsized":"2023-10-06T02:31:26.257586+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:17:04.188102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3b","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.260742+0000","last_change":"2023-10-06T02:31:26.256241+0000","last_active":"2023-10-06T02:31:26.260742+0000","last_peered":"2023-10-06T02:31:26.260742+0000","last_clean":"2023-10-06T02:31:26.260742+0000","last_became_active":"2023-10-06T02:31:26.255959+0000","last_became_peered":"2023-10-06T02:31:26.255959+0000","last_unstale":"2023-10-06T02:31:26.260742+0000","last_undegraded":"2023-10-06T02:31:26.260742+0000","last_fullsized":"2023-10-06T02:31:26.260742+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:45:34.829656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.3a","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.260757+0000","last_change":"2023-10-06T02:31:26.256130+0000","last_active":"2023-10-06T02:31:26.260757+0000","last_peered":"2023-10-06T02:31:26.260757+0000","last_clean":"2023-10-06T02:31:26.260757+0000","last_became_active":"2023-10-06T02:31:26.255805+0000","last_became_peered":"2023-10-06T02:31:26.255805+0000","last_unstale":"2023-10-06T02:31:26.260757+0000","last_undegraded":"2023-10-06T02:31:26.260757+0000","last_fullsized":"2023-10-06T02:31:26.260757+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:40:40.851662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.39","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.266551+0000","last_change":"2023-10-06T02:31:26.263200+0000","last_active":"2023-10-06T02:31:26.266551+0000","last_peered":"2023-10-06T02:31:26.266551+0000","last_clean":"2023-10-06T02:31:26.266551+0000","last_became_active":"2023-10-06T02:31:26.262943+0000","last_became_peered":"2023-10-06T02:31:26.262943+0000","last_unstale":"2023-10-06T02:31:26.266551+0000","last_undegraded":"2023-10-06T02:31:26.266551+0000","last_fullsized":"2023-10-06T02:31:26.266551+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:04:38.216053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.38","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.267385+0000","last_change":"2023-10-06T02:31:26.261673+0000","last_active":"2023-10-06T02:31:26.267385+0000","last_peered":"2023-10-06T02:31:26.267385+0000","last_clean":"2023-10-06T02:31:26.267385+0000","last_became_active":"2023-10-06T02:31:26.261364+0000","last_became_peered":"2023-10-06T02:31:26.261364+0000","last_unstale":"2023-10-06T02:31:26.267385+0000","last_undegraded":"2023-10-06T02:31:26.267385+0000","last_fullsized":"2023-10-06T02:31:26.267385+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:08:43.058251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.37","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.267969+0000","last_change":"2023-10-06T02:31:26.264821+0000","last_active":"2023-10-06T02:31:26.267969+0000","last_peered":"2023-10-06T02:31:26.267969+0000","last_clean":"2023-10-06T02:31:26.267969+0000","last_became_active":"2023-10-06T02:31:26.260543+0000","last_became_peered":"2023-10-06T02:31:26.260543+0000","last_unstale":"2023-10-06T02:31:26.267969+0000","last_undegraded":"2023-10-06T02:31:26.267969+0000","last_fullsized":"2023-10-06T02:31:26.267969+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:52:31.748249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.36","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.260751+0000","last_change":"2023-10-06T02:31:26.256860+0000","last_active":"2023-10-06T02:31:26.260751+0000","last_peered":"2023-10-06T02:31:26.260751+0000","last_clean":"2023-10-06T02:31:26.260751+0000","last_became_active":"2023-10-06T02:31:26.256588+0000","last_became_peered":"2023-10-06T02:31:26.256588+0000","last_unstale":"2023-10-06T02:31:26.260751+0000","last_undegraded":"2023-10-06T02:31:26.260751+0000","last_fullsized":"2023-10-06T02:31:26.260751+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:53:41.055784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.35","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.256837+0000","last_change":"2023-10-06T02:31:26.244042+0000","last_active":"2023-10-06T02:31:26.256837+0000","last_peered":"2023-10-06T02:31:26.256837+0000","last_clean":"2023-10-06T02:31:26.256837+0000","last_became_active":"2023-10-06T02:31:26.243653+0000","last_became_peered":"2023-10-06T02:31:26.243653+0000","last_unstale":"2023-10-06T02:31:26.256837+0000","last_undegraded":"2023-10-06T02:31:26.256837+0000","last_fullsized":"2023-10-06T02:31:26.256837+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:25:17.234017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.34","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.263701+0000","last_change":"2023-10-06T02:31:26.260676+0000","last_active":"2023-10-06T02:31:26.263701+0000","last_peered":"2023-10-06T02:31:26.263701+0000","last_clean":"2023-10-06T02:31:26.263701+0000","last_became_active":"2023-10-06T02:31:26.260285+0000","last_became_peered":"2023-10-06T02:31:26.260285+0000","last_unstale":"2023-10-06T02:31:26.263701+0000","last_undegraded":"2023-10-06T02:31:26.263701+0000","last_fullsized":"2023-10-06T02:31:26.263701+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:29:15.508209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.33","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.266711+0000","last_change":"2023-10-06T02:31:26.262919+0000","last_active":"2023-10-06T02:31:26.266711+0000","last_peered":"2023-10-06T02:31:26.266711+0000","last_clean":"2023-10-06T02:31:26.266711+0000","last_became_active":"2023-10-06T02:31:26.262570+0000","last_became_peered":"2023-10-06T02:31:26.262570+0000","last_unstale":"2023-10-06T02:31:26.266711+0000","last_undegraded":"2023-10-06T02:31:26.266711+0000","last_fullsized":"2023-10-06T02:31:26.266711+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:31:30.918438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.32","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.258168+0000","last_change":"2023-10-06T02:31:26.254903+0000","last_active":"2023-10-06T02:31:26.258168+0000","last_peered":"2023-10-06T02:31:26.258168+0000","last_clean":"2023-10-06T02:31:26.258168+0000","last_became_active":"2023-10-06T02:31:26.254617+0000","last_became_peered":"2023-10-06T02:31:26.254617+0000","last_unstale":"2023-10-06T02:31:26.258168+0000","last_undegraded":"2023-10-06T02:31:26.258168+0000","last_fullsized":"2023-10-06T02:31:26.258168+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:07:33.997738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.31","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.266929+0000","last_change":"2023-10-06T02:31:26.264463+0000","last_active":"2023-10-06T02:31:26.266929+0000","last_peered":"2023-10-06T02:31:26.266929+0000","last_clean":"2023-10-06T02:31:26.266929+0000","last_became_active":"2023-10-06T02:31:26.258882+0000","last_became_peered":"2023-10-06T02:31:26.258882+0000","last_unstale":"2023-10-06T02:31:26.266929+0000","last_undegraded":"2023-10-06T02:31:26.266929+0000","last_fullsized":"2023-10-06T02:31:26.266929+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:21:40.138974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.30","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.260702+0000","last_change":"2023-10-06T02:31:26.255233+0000","last_active":"2023-10-06T02:31:26.260702+0000","last_peered":"2023-10-06T02:31:26.260702+0000","last_clean":"2023-10-06T02:31:26.260702+0000","last_became_active":"2023-10-06T02:31:26.254974+0000","last_became_peered":"2023-10-06T02:31:26.254974+0000","last_unstale":"2023-10-06T02:31:26.260702+0000","last_undegraded":"2023-10-06T02:31:26.260702+0000","last_fullsized":"2023-10-06T02:31:26.260702+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:18:38.740278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2f","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.264419+0000","last_change":"2023-10-06T02:31:26.261279+0000","last_active":"2023-10-06T02:31:26.264419+0000","last_peered":"2023-10-06T02:31:26.264419+0000","last_clean":"2023-10-06T02:31:26.264419+0000","last_became_active":"2023-10-06T02:31:26.260982+0000","last_became_peered":"2023-10-06T02:31:26.260982+0000","last_unstale":"2023-10-06T02:31:26.264419+0000","last_undegraded":"2023-10-06T02:31:26.264419+0000","last_fullsized":"2023-10-06T02:31:26.264419+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:08:16.984980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2e","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.265431+0000","last_change":"2023-10-06T02:31:26.261700+0000","last_active":"2023-10-06T02:31:26.265431+0000","last_peered":"2023-10-06T02:31:26.265431+0000","last_clean":"2023-10-06T02:31:26.265431+0000","last_became_active":"2023-10-06T02:31:26.261387+0000","last_became_peered":"2023-10-06T02:31:26.261387+0000","last_unstale":"2023-10-06T02:31:26.265431+0000","last_undegraded":"2023-10-06T02:31:26.265431+0000","last_fullsized":"2023-10-06T02:31:26.265431+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:06:51.338576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2d","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.263668+0000","last_change":"2023-10-06T02:31:26.256852+0000","last_active":"2023-10-06T02:31:26.263668+0000","last_peered":"2023-10-06T02:31:26.263668+0000","last_clean":"2023-10-06T02:31:26.263668+0000","last_became_active":"2023-10-06T02:31:26.256485+0000","last_became_peered":"2023-10-06T02:31:26.256485+0000","last_unstale":"2023-10-06T02:31:26.263668+0000","last_undegraded":"2023-10-06T02:31:26.263668+0000","last_fullsized":"2023-10-06T02:31:26.263668+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:38:03.520905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2c","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.267286+0000","last_change":"2023-10-06T02:31:26.261634+0000","last_active":"2023-10-06T02:31:26.267286+0000","last_peered":"2023-10-06T02:31:26.267286+0000","last_clean":"2023-10-06T02:31:26.267286+0000","last_became_active":"2023-10-06T02:31:26.261365+0000","last_became_peered":"2023-10-06T02:31:26.261365+0000","last_unstale":"2023-10-06T02:31:26.267286+0000","last_undegraded":"2023-10-06T02:31:26.267286+0000","last_fullsized":"2023-10-06T02:31:26.267286+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:32:49.083484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2b","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.262183+0000","last_change":"2023-10-06T02:31:26.254306+0000","last_active":"2023-10-06T02:31:26.262183+0000","last_peered":"2023-10-06T02:31:26.262183+0000","last_clean":"2023-10-06T02:31:26.262183+0000","last_became_active":"2023-10-06T02:31:26.252527+0000","last_became_peered":"2023-10-06T02:31:26.252527+0000","last_unstale":"2023-10-06T02:31:26.262183+0000","last_undegraded":"2023-10-06T02:31:26.262183+0000","last_fullsized":"2023-10-06T02:31:26.262183+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:52:03.877573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.2a","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.264010+0000","last_change":"2023-10-06T02:31:26.259526+0000","last_active":"2023-10-06T02:31:26.264010+0000","last_peered":"2023-10-06T02:31:26.264010+0000","last_clean":"2023-10-06T02:31:26.264010+0000","last_became_active":"2023-10-06T02:31:26.258779+0000","last_became_peered":"2023-10-06T02:31:26.258779+0000","last_unstale":"2023-10-06T02:31:26.264010+0000","last_undegraded":"2023-10-06T02:31:26.264010+0000","last_fullsized":"2023-10-06T02:31:26.264010+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:16:42.707675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.29","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.270281+0000","last_change":"2023-10-06T02:31:26.267009+0000","last_active":"2023-10-06T02:31:26.270281+0000","last_peered":"2023-10-06T02:31:26.270281+0000","last_clean":"2023-10-06T02:31:26.270281+0000","last_became_active":"2023-10-06T02:31:26.266730+0000","last_became_peered":"2023-10-06T02:31:26.266730+0000","last_unstale":"2023-10-06T02:31:26.270281+0000","last_undegraded":"2023-10-06T02:31:26.270281+0000","last_fullsized":"2023-10-06T02:31:26.270281+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:54:52.162681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.28","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.266168+0000","last_change":"2023-10-06T02:31:26.262949+0000","last_active":"2023-10-06T02:31:26.266168+0000","last_peered":"2023-10-06T02:31:26.266168+0000","last_clean":"2023-10-06T02:31:26.266168+0000","last_became_active":"2023-10-06T02:31:26.262604+0000","last_became_peered":"2023-10-06T02:31:26.262604+0000","last_unstale":"2023-10-06T02:31:26.266168+0000","last_undegraded":"2023-10-06T02:31:26.266168+0000","last_fullsized":"2023-10-06T02:31:26.266168+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:06:51.338576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.27","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.264238+0000","last_change":"2023-10-06T02:31:26.257598+0000","last_active":"2023-10-06T02:31:26.264238+0000","last_peered":"2023-10-06T02:31:26.264238+0000","last_clean":"2023-10-06T02:31:26.264238+0000","last_became_active":"2023-10-06T02:31:26.257237+0000","last_became_peered":"2023-10-06T02:31:26.257237+0000","last_unstale":"2023-10-06T02:31:26.264238+0000","last_undegraded":"2023-10-06T02:31:26.264238+0000","last_fullsized":"2023-10-06T02:31:26.264238+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:42:10.460130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.26","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.269912+0000","last_change":"2023-10-06T02:31:26.261508+0000","last_active":"2023-10-06T02:31:26.269912+0000","last_peered":"2023-10-06T02:31:26.269912+0000","last_clean":"2023-10-06T02:31:26.269912+0000","last_became_active":"2023-10-06T02:31:26.261210+0000","last_became_peered":"2023-10-06T02:31:26.261210+0000","last_unstale":"2023-10-06T02:31:26.269912+0000","last_undegraded":"2023-10-06T02:31:26.269912+0000","last_fullsized":"2023-10-06T02:31:26.269912+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:16:42.707675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.25","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.270115+0000","last_change":"2023-10-06T02:31:26.266150+0000","last_active":"2023-10-06T02:31:26.270115+0000","last_peered":"2023-10-06T02:31:26.270115+0000","last_clean":"2023-10-06T02:31:26.270115+0000","last_became_active":"2023-10-06T02:31:26.265862+0000","last_became_peered":"2023-10-06T02:31:26.265862+0000","last_unstale":"2023-10-06T02:31:26.270115+0000","last_undegraded":"2023-10-06T02:31:26.270115+0000","last_fullsized":"2023-10-06T02:31:26.270115+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:23:05.322592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.24","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.261744+0000","last_change":"2023-10-06T02:31:26.256284+0000","last_active":"2023-10-06T02:31:26.261744+0000","last_peered":"2023-10-06T02:31:26.261744+0000","last_clean":"2023-10-06T02:31:26.261744+0000","last_became_active":"2023-10-06T02:31:26.255978+0000","last_became_peered":"2023-10-06T02:31:26.255978+0000","last_unstale":"2023-10-06T02:31:26.261744+0000","last_undegraded":"2023-10-06T02:31:26.261744+0000","last_fullsized":"2023-10-06T02:31:26.261744+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:24:11.363292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.23","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.269993+0000","last_change":"2023-10-06T02:31:26.260168+0000","last_active":"2023-10-06T02:31:26.269993+0000","last_peered":"2023-10-06T02:31:26.269993+0000","last_clean":"2023-10-06T02:31:26.269993+0000","last_became_active":"2023-10-06T02:31:26.259743+0000","last_became_peered":"2023-10-06T02:31:26.259743+0000","last_unstale":"2023-10-06T02:31:26.269993+0000","last_undegraded":"2023-10-06T02:31:26.269993+0000","last_fullsized":"2023-10-06T02:31:26.269993+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:27:31.472908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.22","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.266229+0000","last_change":"2023-10-06T02:31:26.255194+0000","last_active":"2023-10-06T02:31:26.266229+0000","last_peered":"2023-10-06T02:31:26.266229+0000","last_clean":"2023-10-06T02:31:26.266229+0000","last_became_active":"2023-10-06T02:31:26.254396+0000","last_became_peered":"2023-10-06T02:31:26.254396+0000","last_unstale":"2023-10-06T02:31:26.266229+0000","last_undegraded":"2023-10-06T02:31:26.266229+0000","last_fullsized":"2023-10-06T02:31:26.266229+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:09:25.643431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.21","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.262258+0000","last_change":"2023-10-06T02:31:26.253779+0000","last_active":"2023-10-06T02:31:26.262258+0000","last_peered":"2023-10-06T02:31:26.262258+0000","last_clean":"2023-10-06T02:31:26.262258+0000","last_became_active":"2023-10-06T02:31:26.253512+0000","last_became_peered":"2023-10-06T02:31:26.253512+0000","last_unstale":"2023-10-06T02:31:26.262258+0000","last_undegraded":"2023-10-06T02:31:26.262258+0000","last_fullsized":"2023-10-06T02:31:26.262258+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:01:14.251053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.20","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.260675+0000","last_change":"2023-10-06T02:31:26.254349+0000","last_active":"2023-10-06T02:31:26.260675+0000","last_peered":"2023-10-06T02:31:26.260675+0000","last_clean":"2023-10-06T02:31:26.260675+0000","last_became_active":"2023-10-06T02:31:26.254049+0000","last_became_peered":"2023-10-06T02:31:26.254049+0000","last_unstale":"2023-10-06T02:31:26.260675+0000","last_undegraded":"2023-10-06T02:31:26.260675+0000","last_fullsized":"2023-10-06T02:31:26.260675+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:29:15.508209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.14","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.270339+0000","last_change":"2023-10-06T02:31:26.267533+0000","last_active":"2023-10-06T02:31:26.270339+0000","last_peered":"2023-10-06T02:31:26.270339+0000","last_clean":"2023-10-06T02:31:26.270339+0000","last_became_active":"2023-10-06T02:31:26.267238+0000","last_became_peered":"2023-10-06T02:31:26.267238+0000","last_unstale":"2023-10-06T02:31:26.270339+0000","last_undegraded":"2023-10-06T02:31:26.270339+0000","last_fullsized":"2023-10-06T02:31:26.270339+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:49:16.077340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.246287+0000","last_change":"2023-10-06T02:31:25.235636+0000","last_active":"2023-10-06T02:31:26.246287+0000","last_peered":"2023-10-06T02:31:26.246287+0000","last_clean":"2023-10-06T02:31:26.246287+0000","last_became_active":"2023-10-06T02:31:25.233839+0000","last_became_peered":"2023-10-06T02:31:25.233839+0000","last_unstale":"2023-10-06T02:31:26.246287+0000","last_undegraded":"2023-10-06T02:31:26.246287+0000","last_fullsized":"2023-10-06T02:31:26.246287+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_clean_scrub_stamp":"2023-10-06T02:31:24.211070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:26:39.899368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.15","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.263304+0000","last_change":"2023-10-06T02:31:26.260505+0000","last_active":"2023-10-06T02:31:26.263304+0000","last_peered":"2023-10-06T02:31:26.263304+0000","last_clean":"2023-10-06T02:31:26.263304+0000","last_became_active":"2023-10-06T02:31:26.260223+0000","last_became_peered":"2023-10-06T02:31:26.260223+0000","last_unstale":"2023-10-06T02:31:26.263304+0000","last_undegraded":"2023-10-06T02:31:26.263304+0000","last_fullsized":"2023-10-06T02:31:26.263304+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:42:34.992024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.250873+0000","last_change":"2023-10-06T02:31:25.246162+0000","last_active":"2023-10-06T02:31:26.250873+0000","last_peered":"2023-10-06T02:31:26.250873+0000","last_clean":"2023-10-06T02:31:26.250873+0000","last_became_active":"2023-10-06T02:31:25.245821+0000","last_became_peered":"2023-10-06T02:31:25.245821+0000","last_unstale":"2023-10-06T02:31:26.250873+0000","last_undegraded":"2023-10-06T02:31:26.250873+0000","last_fullsized":"2023-10-06T02:31:26.250873+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_clean_scrub_stamp":"2023-10-06T02:31:24.211070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:03:05.907292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.265412+0000","last_change":"2023-10-06T02:31:26.258365+0000","last_active":"2023-10-06T02:31:26.265412+0000","last_peered":"2023-10-06T02:31:26.265412+0000","last_clean":"2023-10-06T02:31:26.265412+0000","last_became_active":"2023-10-06T02:31:26.257997+0000","last_became_peered":"2023-10-06T02:31:26.257997+0000","last_unstale":"2023-10-06T02:31:26.265412+0000","last_undegraded":"2023-10-06T02:31:26.265412+0000","last_fullsized":"2023-10-06T02:31:26.265412+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:03:29.054970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.15","version":"155'3","reported_seq":13,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.275523+0000","last_change":"2023-10-06T02:31:25.242416+0000","last_active":"2023-10-06T02:31:26.275523+0000","last_peered":"2023-10-06T02:31:26.275523+0000","last_clean":"2023-10-06T02:31:26.275523+0000","last_became_active":"2023-10-06T02:31:25.242136+0000","last_became_peered":"2023-10-06T02:31:25.242136+0000","last_unstale":"2023-10-06T02:31:26.275523+0000","last_undegraded":"2023-10-06T02:31:26.275523+0000","last_fullsized":"2023-10-06T02:31:26.275523+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_clean_scrub_stamp":"2023-10-06T02:31:24.211070+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:08:08.801557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.269695+0000","last_change":"2023-10-06T02:31:26.267624+0000","last_active":"2023-10-06T02:31:26.269695+0000","last_peered":"2023-10-06T02:31:26.269695+0000","last_clean":"2023-10-06T02:31:26.269695+0000","last_became_active":"2023-10-06T02:31:26.267261+0000","last_became_peered":"2023-10-06T02:31:26.267261+0000","last_unstale":"2023-10-06T02:31:26.269695+0000","last_undegraded":"2023-10-06T02:31:26.269695+0000","last_fullsized":"2023-10-06T02:31:26.269695+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:27:31.472908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.14","version":"155'6","reported_seq":16,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.281821+0000","last_change":"2023-10-06T02:31:25.231015+0000","last_active":"2023-10-06T02:31:26.281821+0000","last_peered":"2023-10-06T02:31:26.281821+0000","last_clean":"2023-10-06T02:31:26.281821+0000","last_became_active":"2023-10-06T02:31:25.230074+0000","last_became_peered":"2023-10-06T02:31:25.230074+0000","last_unstale":"2023-10-06T02:31:26.281821+0000","last_undegraded":"2023-10-06T02:31:26.281821+0000","last_fullsized":"2023-10-06T02:31:26.281821+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_clean_scrub_stamp":"2023-10-06T02:31:24.211070+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:49:46.744608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.10","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.262624+0000","last_change":"2023-10-06T02:31:26.256692+0000","last_active":"2023-10-06T02:31:26.262624+0000","last_peered":"2023-10-06T02:31:26.262624+0000","last_clean":"2023-10-06T02:31:26.262624+0000","last_became_active":"2023-10-06T02:31:26.256421+0000","last_became_peered":"2023-10-06T02:31:26.256421+0000","last_unstale":"2023-10-06T02:31:26.262624+0000","last_undegraded":"2023-10-06T02:31:26.262624+0000","last_fullsized":"2023-10-06T02:31:26.262624+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:35:27.025630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.13","version":"155'2","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.278320+0000","last_change":"2023-10-06T02:31:25.233452+0000","last_active":"2023-10-06T02:31:26.278320+0000","last_peered":"2023-10-06T02:31:26.278320+0000","last_clean":"2023-10-06T02:31:26.278320+0000","last_became_active":"2023-10-06T02:31:25.232924+0000","last_became_peered":"2023-10-06T02:31:25.232924+0000","last_unstale":"2023-10-06T02:31:26.278320+0000","last_undegraded":"2023-10-06T02:31:26.278320+0000","last_fullsized":"2023-10-06T02:31:26.278320+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_clean_scrub_stamp":"2023-10-06T02:31:24.211070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:06:51.278627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.261402+0000","last_change":"2023-10-06T02:31:26.254653+0000","last_active":"2023-10-06T02:31:26.261402+0000","last_peered":"2023-10-06T02:31:26.261402+0000","last_clean":"2023-10-06T02:31:26.261402+0000","last_became_active":"2023-10-06T02:31:26.254383+0000","last_became_peered":"2023-10-06T02:31:26.254383+0000","last_unstale":"2023-10-06T02:31:26.261402+0000","last_undegraded":"2023-10-06T02:31:26.261402+0000","last_fullsized":"2023-10-06T02:31:26.261402+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:14:58.219720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.251072+0000","last_change":"2023-10-06T02:31:25.245963+0000","last_active":"2023-10-06T02:31:26.251072+0000","last_peered":"2023-10-06T02:31:26.251072+0000","last_clean":"2023-10-06T02:31:26.251072+0000","last_became_active":"2023-10-06T02:31:25.245628+0000","last_became_peered":"2023-10-06T02:31:25.245628+0000","last_unstale":"2023-10-06T02:31:26.251072+0000","last_undegraded":"2023-10-06T02:31:26.251072+0000","last_fullsized":"2023-10-06T02:31:26.251072+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_clean_scrub_stamp":"2023-10-06T02:31:24.211070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:40:16.238993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.264963+0000","last_change":"2023-10-06T02:31:26.257635+0000","last_active":"2023-10-06T02:31:26.264963+0000","last_peered":"2023-10-06T02:31:26.264963+0000","last_clean":"2023-10-06T02:31:26.264963+0000","last_became_active":"2023-10-06T02:31:26.257339+0000","last_became_peered":"2023-10-06T02:31:26.257339+0000","last_unstale":"2023-10-06T02:31:26.264963+0000","last_undegraded":"2023-10-06T02:31:26.264963+0000","last_fullsized":"2023-10-06T02:31:26.264963+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:01:04.378232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.244965+0000","last_change":"2023-10-06T02:31:25.227976+0000","last_active":"2023-10-06T02:31:26.244965+0000","last_peered":"2023-10-06T02:31:26.244965+0000","last_clean":"2023-10-06T02:31:26.244965+0000","last_became_active":"2023-10-06T02:31:25.227709+0000","last_became_peered":"2023-10-06T02:31:25.227709+0000","last_unstale":"2023-10-06T02:31:26.244965+0000","last_undegraded":"2023-10-06T02:31:26.244965+0000","last_fullsized":"2023-10-06T02:31:26.244965+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_clean_scrub_stamp":"2023-10-06T02:31:24.211070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:11.225751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.13","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.265869+0000","last_change":"2023-10-06T02:31:26.257788+0000","last_active":"2023-10-06T02:31:26.265869+0000","last_peered":"2023-10-06T02:31:26.265869+0000","last_clean":"2023-10-06T02:31:26.265869+0000","last_became_active":"2023-10-06T02:31:26.257493+0000","last_became_peered":"2023-10-06T02:31:26.257493+0000","last_unstale":"2023-10-06T02:31:26.265869+0000","last_undegraded":"2023-10-06T02:31:26.265869+0000","last_fullsized":"2023-10-06T02:31:26.265869+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:27:31.472908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.10","version":"155'8","reported_seq":18,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.280459+0000","last_change":"2023-10-06T02:31:25.232435+0000","last_active":"2023-10-06T02:31:26.280459+0000","last_peered":"2023-10-06T02:31:26.280459+0000","last_clean":"2023-10-06T02:31:26.280459+0000","last_became_active":"2023-10-06T02:31:25.232107+0000","last_became_peered":"2023-10-06T02:31:25.232107+0000","last_unstale":"2023-10-06T02:31:26.280459+0000","last_undegraded":"2023-10-06T02:31:26.280459+0000","last_fullsized":"2023-10-06T02:31:26.280459+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_clean_scrub_stamp":"2023-10-06T02:31:24.211070+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:49:18.537141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.257523+0000","last_change":"2023-10-06T02:31:26.253710+0000","last_active":"2023-10-06T02:31:26.257523+0000","last_peered":"2023-10-06T02:31:26.257523+0000","last_clean":"2023-10-06T02:31:26.257523+0000","last_became_active":"2023-10-06T02:31:26.253418+0000","last_became_peered":"2023-10-06T02:31:26.253418+0000","last_unstale":"2023-10-06T02:31:26.257523+0000","last_undegraded":"2023-10-06T02:31:26.257523+0000","last_fullsized":"2023-10-06T02:31:26.257523+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:26:55.194055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"155'1","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.277188+0000","last_change":"2023-10-06T02:31:25.232877+0000","last_active":"2023-10-06T02:31:26.277188+0000","last_peered":"2023-10-06T02:31:26.277188+0000","last_clean":"2023-10-06T02:31:26.277188+0000","last_became_active":"2023-10-06T02:31:25.232601+0000","last_became_peered":"2023-10-06T02:31:25.232601+0000","last_unstale":"2023-10-06T02:31:26.277188+0000","last_undegraded":"2023-10-06T02:31:26.277188+0000","last_fullsized":"2023-10-06T02:31:26.277188+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_clean_scrub_stamp":"2023-10-06T02:31:24.211070+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:32:10.362038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.d","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.262187+0000","last_change":"2023-10-06T02:31:26.260404+0000","last_active":"2023-10-06T02:31:26.262187+0000","last_peered":"2023-10-06T02:31:26.262187+0000","last_clean":"2023-10-06T02:31:26.262187+0000","last_became_active":"2023-10-06T02:31:26.260138+0000","last_became_peered":"2023-10-06T02:31:26.260138+0000","last_unstale":"2023-10-06T02:31:26.262187+0000","last_undegraded":"2023-10-06T02:31:26.262187+0000","last_fullsized":"2023-10-06T02:31:26.262187+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:19:31.367437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.e","version":"155'2","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.275922+0000","last_change":"2023-10-06T02:31:25.235127+0000","last_active":"2023-10-06T02:31:26.275922+0000","last_peered":"2023-10-06T02:31:26.275922+0000","last_clean":"2023-10-06T02:31:26.275922+0000","last_became_active":"2023-10-06T02:31:25.234876+0000","last_became_peered":"2023-10-06T02:31:25.234876+0000","last_unstale":"2023-10-06T02:31:26.275922+0000","last_undegraded":"2023-10-06T02:31:26.275922+0000","last_fullsized":"2023-10-06T02:31:26.275922+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_clean_scrub_stamp":"2023-10-06T02:31:24.211070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:01:07.384356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.e","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.269892+0000","last_change":"2023-10-06T02:31:26.260580+0000","last_active":"2023-10-06T02:31:26.269892+0000","last_peered":"2023-10-06T02:31:26.269892+0000","last_clean":"2023-10-06T02:31:26.269892+0000","last_became_active":"2023-10-06T02:31:26.260281+0000","last_became_peered":"2023-10-06T02:31:26.260281+0000","last_unstale":"2023-10-06T02:31:26.269892+0000","last_undegraded":"2023-10-06T02:31:26.269892+0000","last_fullsized":"2023-10-06T02:31:26.269892+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:10:36.589251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.d","version":"155'1","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.278092+0000","last_change":"2023-10-06T02:31:25.240303+0000","last_active":"2023-10-06T02:31:26.278092+0000","last_peered":"2023-10-06T02:31:26.278092+0000","last_clean":"2023-10-06T02:31:26.278092+0000","last_became_active":"2023-10-06T02:31:25.239964+0000","last_became_peered":"2023-10-06T02:31:25.239964+0000","last_unstale":"2023-10-06T02:31:26.278092+0000","last_undegraded":"2023-10-06T02:31:26.278092+0000","last_fullsized":"2023-10-06T02:31:26.278092+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_clean_scrub_stamp":"2023-10-06T02:31:24.211070+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:25:01.375389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.f","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.269781+0000","last_change":"2023-10-06T02:31:26.268110+0000","last_active":"2023-10-06T02:31:26.269781+0000","last_peered":"2023-10-06T02:31:26.269781+0000","last_clean":"2023-10-06T02:31:26.269781+0000","last_became_active":"2023-10-06T02:31:26.267769+0000","last_became_peered":"2023-10-06T02:31:26.267769+0000","last_unstale":"2023-10-06T02:31:26.269781+0000","last_undegraded":"2023-10-06T02:31:26.269781+0000","last_fullsized":"2023-10-06T02:31:26.269781+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:09:25.643431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.239901+0000","last_change":"2023-10-06T02:31:25.239545+0000","last_active":"2023-10-06T02:31:26.239901+0000","last_peered":"2023-10-06T02:31:26.239901+0000","last_clean":"2023-10-06T02:31:26.239901+0000","last_became_active":"2023-10-06T02:31:25.239256+0000","last_became_peered":"2023-10-06T02:31:25.239256+0000","last_unstale":"2023-10-06T02:31:26.239901+0000","last_undegraded":"2023-10-06T02:31:26.239901+0000","last_fullsized":"2023-10-06T02:31:26.239901+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_clean_scrub_stamp":"2023-10-06T02:31:24.211070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:16:13.065247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.264335+0000","last_change":"2023-10-06T02:31:26.260199+0000","last_active":"2023-10-06T02:31:26.264335+0000","last_peered":"2023-10-06T02:31:26.264335+0000","last_clean":"2023-10-06T02:31:26.264335+0000","last_became_active":"2023-10-06T02:31:26.259898+0000","last_became_peered":"2023-10-06T02:31:26.259898+0000","last_unstale":"2023-10-06T02:31:26.264335+0000","last_undegraded":"2023-10-06T02:31:26.264335+0000","last_fullsized":"2023-10-06T02:31:26.264335+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:49:58.442772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"155'2","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.277853+0000","last_change":"2023-10-06T02:31:25.242559+0000","last_active":"2023-10-06T02:31:26.277853+0000","last_peered":"2023-10-06T02:31:26.277853+0000","last_clean":"2023-10-06T02:31:26.277853+0000","last_became_active":"2023-10-06T02:31:25.242245+0000","last_became_peered":"2023-10-06T02:31:25.242245+0000","last_unstale":"2023-10-06T02:31:26.277853+0000","last_undegraded":"2023-10-06T02:31:26.277853+0000","last_fullsized":"2023-10-06T02:31:26.277853+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_clean_scrub_stamp":"2023-10-06T02:31:24.211070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:40:16.238993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.263797+0000","last_change":"2023-10-06T02:31:26.260197+0000","last_active":"2023-10-06T02:31:26.263797+0000","last_peered":"2023-10-06T02:31:26.263797+0000","last_clean":"2023-10-06T02:31:26.263797+0000","last_became_active":"2023-10-06T02:31:26.258913+0000","last_became_peered":"2023-10-06T02:31:26.258913+0000","last_unstale":"2023-10-06T02:31:26.263797+0000","last_undegraded":"2023-10-06T02:31:26.263797+0000","last_fullsized":"2023-10-06T02:31:26.263797+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:08:36.844212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.244503+0000","last_change":"2023-10-06T02:31:25.227427+0000","last_active":"2023-10-06T02:31:26.244503+0000","last_peered":"2023-10-06T02:31:26.244503+0000","last_clean":"2023-10-06T02:31:26.244503+0000","last_became_active":"2023-10-06T02:31:25.226896+0000","last_became_peered":"2023-10-06T02:31:25.226896+0000","last_unstale":"2023-10-06T02:31:26.244503+0000","last_undegraded":"2023-10-06T02:31:26.244503+0000","last_fullsized":"2023-10-06T02:31:26.244503+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_clean_scrub_stamp":"2023-10-06T02:31:24.211070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:37:47.966193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.266447+0000","last_change":"2023-10-06T02:31:26.261736+0000","last_active":"2023-10-06T02:31:26.266447+0000","last_peered":"2023-10-06T02:31:26.266447+0000","last_clean":"2023-10-06T02:31:26.266447+0000","last_became_active":"2023-10-06T02:31:26.261465+0000","last_became_peered":"2023-10-06T02:31:26.261465+0000","last_unstale":"2023-10-06T02:31:26.266447+0000","last_undegraded":"2023-10-06T02:31:26.266447+0000","last_fullsized":"2023-10-06T02:31:26.266447+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:33:46.837429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.239158+0000","last_change":"2023-10-06T02:31:25.242026+0000","last_active":"2023-10-06T02:31:26.239158+0000","last_peered":"2023-10-06T02:31:26.239158+0000","last_clean":"2023-10-06T02:31:26.239158+0000","last_became_active":"2023-10-06T02:31:25.241765+0000","last_became_peered":"2023-10-06T02:31:25.241765+0000","last_unstale":"2023-10-06T02:31:26.239158+0000","last_undegraded":"2023-10-06T02:31:26.239158+0000","last_fullsized":"2023-10-06T02:31:26.239158+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_clean_scrub_stamp":"2023-10-06T02:31:24.211070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:29:03.369149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.b","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.261452+0000","last_change":"2023-10-06T02:31:26.258018+0000","last_active":"2023-10-06T02:31:26.261452+0000","last_peered":"2023-10-06T02:31:26.261452+0000","last_clean":"2023-10-06T02:31:26.261452+0000","last_became_active":"2023-10-06T02:31:26.257732+0000","last_became_peered":"2023-10-06T02:31:26.257732+0000","last_unstale":"2023-10-06T02:31:26.261452+0000","last_undegraded":"2023-10-06T02:31:26.261452+0000","last_fullsized":"2023-10-06T02:31:26.261452+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:26:58.631262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.239559+0000","last_change":"2023-10-06T02:31:25.241731+0000","last_active":"2023-10-06T02:31:26.239559+0000","last_peered":"2023-10-06T02:31:26.239559+0000","last_clean":"2023-10-06T02:31:26.239559+0000","last_became_active":"2023-10-06T02:31:25.241439+0000","last_became_peered":"2023-10-06T02:31:25.241439+0000","last_unstale":"2023-10-06T02:31:26.239559+0000","last_undegraded":"2023-10-06T02:31:26.239559+0000","last_fullsized":"2023-10-06T02:31:26.239559+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_clean_scrub_stamp":"2023-10-06T02:31:24.211070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:29:03.369149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.248037+0000","last_change":"2023-10-06T02:31:26.245158+0000","last_active":"2023-10-06T02:31:26.248037+0000","last_peered":"2023-10-06T02:31:26.248037+0000","last_clean":"2023-10-06T02:31:26.248037+0000","last_became_active":"2023-10-06T02:31:26.244876+0000","last_became_peered":"2023-10-06T02:31:26.244876+0000","last_unstale":"2023-10-06T02:31:26.248037+0000","last_undegraded":"2023-10-06T02:31:26.248037+0000","last_fullsized":"2023-10-06T02:31:26.248037+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:40:55.719210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.7","version":"155'2","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.275652+0000","last_change":"2023-10-06T02:31:25.239291+0000","last_active":"2023-10-06T02:31:26.275652+0000","last_peered":"2023-10-06T02:31:26.275652+0000","last_clean":"2023-10-06T02:31:26.275652+0000","last_became_active":"2023-10-06T02:31:25.239024+0000","last_became_peered":"2023-10-06T02:31:25.239024+0000","last_unstale":"2023-10-06T02:31:26.275652+0000","last_undegraded":"2023-10-06T02:31:26.275652+0000","last_fullsized":"2023-10-06T02:31:26.275652+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_clean_scrub_stamp":"2023-10-06T02:31:24.211070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:54:51.160175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.264824+0000","last_change":"2023-10-06T02:31:26.259555+0000","last_active":"2023-10-06T02:31:26.264824+0000","last_peered":"2023-10-06T02:31:26.264824+0000","last_clean":"2023-10-06T02:31:26.264824+0000","last_became_active":"2023-10-06T02:31:26.259261+0000","last_became_peered":"2023-10-06T02:31:26.259261+0000","last_unstale":"2023-10-06T02:31:26.264824+0000","last_undegraded":"2023-10-06T02:31:26.264824+0000","last_fullsized":"2023-10-06T02:31:26.264824+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:18:02.982727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.239576+0000","last_change":"2023-10-06T02:31:25.241396+0000","last_active":"2023-10-06T02:31:26.239576+0000","last_peered":"2023-10-06T02:31:26.239576+0000","last_clean":"2023-10-06T02:31:26.239576+0000","last_became_active":"2023-10-06T02:31:25.241108+0000","last_became_peered":"2023-10-06T02:31:25.241108+0000","last_unstale":"2023-10-06T02:31:26.239576+0000","last_undegraded":"2023-10-06T02:31:26.239576+0000","last_fullsized":"2023-10-06T02:31:26.239576+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_clean_scrub_stamp":"2023-10-06T02:31:24.211070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:54:43.986442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.270329+0000","last_change":"2023-10-06T02:31:26.267947+0000","last_active":"2023-10-06T02:31:26.270329+0000","last_peered":"2023-10-06T02:31:26.270329+0000","last_clean":"2023-10-06T02:31:26.270329+0000","last_became_active":"2023-10-06T02:31:26.267621+0000","last_became_peered":"2023-10-06T02:31:26.267621+0000","last_unstale":"2023-10-06T02:31:26.270329+0000","last_undegraded":"2023-10-06T02:31:26.270329+0000","last_fullsized":"2023-10-06T02:31:26.270329+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:57.836609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.249709+0000","last_change":"2023-10-06T02:31:25.240770+0000","last_active":"2023-10-06T02:31:26.249709+0000","last_peered":"2023-10-06T02:31:26.249709+0000","last_clean":"2023-10-06T02:31:26.249709+0000","last_became_active":"2023-10-06T02:31:25.240376+0000","last_became_peered":"2023-10-06T02:31:25.240376+0000","last_unstale":"2023-10-06T02:31:26.249709+0000","last_undegraded":"2023-10-06T02:31:26.249709+0000","last_fullsized":"2023-10-06T02:31:26.249709+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_clean_scrub_stamp":"2023-10-06T02:31:24.211070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:01:07.384356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.7","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.249627+0000","last_change":"2023-10-06T02:31:26.245037+0000","last_active":"2023-10-06T02:31:26.249627+0000","last_peered":"2023-10-06T02:31:26.249627+0000","last_clean":"2023-10-06T02:31:26.249627+0000","last_became_active":"2023-10-06T02:31:26.244741+0000","last_became_peered":"2023-10-06T02:31:26.244741+0000","last_unstale":"2023-10-06T02:31:26.249627+0000","last_undegraded":"2023-10-06T02:31:26.249627+0000","last_fullsized":"2023-10-06T02:31:26.249627+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:39:43.017395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.249571+0000","last_change":"2023-10-06T02:31:25.247052+0000","last_active":"2023-10-06T02:31:26.249571+0000","last_peered":"2023-10-06T02:31:26.249571+0000","last_clean":"2023-10-06T02:31:26.249571+0000","last_became_active":"2023-10-06T02:31:25.246750+0000","last_became_peered":"2023-10-06T02:31:25.246750+0000","last_unstale":"2023-10-06T02:31:26.249571+0000","last_undegraded":"2023-10-06T02:31:26.249571+0000","last_fullsized":"2023-10-06T02:31:26.249571+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_clean_scrub_stamp":"2023-10-06T02:31:24.211070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:08:08.801557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.258497+0000","last_change":"2023-10-06T02:31:26.251288+0000","last_active":"2023-10-06T02:31:26.258497+0000","last_peered":"2023-10-06T02:31:26.258497+0000","last_clean":"2023-10-06T02:31:26.258497+0000","last_became_active":"2023-10-06T02:31:26.250957+0000","last_became_peered":"2023-10-06T02:31:26.250957+0000","last_unstale":"2023-10-06T02:31:26.258497+0000","last_undegraded":"2023-10-06T02:31:26.258497+0000","last_fullsized":"2023-10-06T02:31:26.258497+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:22:34.110436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.249240+0000","last_change":"2023-10-06T02:31:25.237989+0000","last_active":"2023-10-06T02:31:26.249240+0000","last_peered":"2023-10-06T02:31:26.249240+0000","last_clean":"2023-10-06T02:31:26.249240+0000","last_became_active":"2023-10-06T02:31:25.237714+0000","last_became_peered":"2023-10-06T02:31:25.237714+0000","last_unstale":"2023-10-06T02:31:26.249240+0000","last_undegraded":"2023-10-06T02:31:26.249240+0000","last_fullsized":"2023-10-06T02:31:26.249240+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_clean_scrub_stamp":"2023-10-06T02:31:24.211070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:09:24.640925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.257033+0000","last_change":"2023-10-06T02:31:26.252230+0000","last_active":"2023-10-06T02:31:26.257033+0000","last_peered":"2023-10-06T02:31:26.257033+0000","last_clean":"2023-10-06T02:31:26.257033+0000","last_became_active":"2023-10-06T02:31:26.251879+0000","last_became_peered":"2023-10-06T02:31:26.251879+0000","last_unstale":"2023-10-06T02:31:26.257033+0000","last_undegraded":"2023-10-06T02:31:26.257033+0000","last_fullsized":"2023-10-06T02:31:26.257033+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:18:33.267912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.248720+0000","last_change":"2023-10-06T02:31:25.243778+0000","last_active":"2023-10-06T02:31:26.248720+0000","last_peered":"2023-10-06T02:31:26.248720+0000","last_clean":"2023-10-06T02:31:26.248720+0000","last_became_active":"2023-10-06T02:31:25.243475+0000","last_became_peered":"2023-10-06T02:31:25.243475+0000","last_unstale":"2023-10-06T02:31:26.248720+0000","last_undegraded":"2023-10-06T02:31:26.248720+0000","last_fullsized":"2023-10-06T02:31:26.248720+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_clean_scrub_stamp":"2023-10-06T02:31:24.211070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:16:41.705169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.264315+0000","last_change":"2023-10-06T02:31:26.259854+0000","last_active":"2023-10-06T02:31:26.264315+0000","last_peered":"2023-10-06T02:31:26.264315+0000","last_clean":"2023-10-06T02:31:26.264315+0000","last_became_active":"2023-10-06T02:31:26.259226+0000","last_became_peered":"2023-10-06T02:31:26.259226+0000","last_unstale":"2023-10-06T02:31:26.264315+0000","last_undegraded":"2023-10-06T02:31:26.264315+0000","last_fullsized":"2023-10-06T02:31:26.264315+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:12:45.858237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.239903+0000","last_change":"2023-10-06T02:31:25.236085+0000","last_active":"2023-10-06T02:31:26.239903+0000","last_peered":"2023-10-06T02:31:26.239903+0000","last_clean":"2023-10-06T02:31:26.239903+0000","last_became_active":"2023-10-06T02:31:25.235827+0000","last_became_peered":"2023-10-06T02:31:25.235827+0000","last_unstale":"2023-10-06T02:31:26.239903+0000","last_undegraded":"2023-10-06T02:31:26.239903+0000","last_fullsized":"2023-10-06T02:31:26.239903+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_clean_scrub_stamp":"2023-10-06T02:31:24.211070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:25:01.375389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.268856+0000","last_change":"2023-10-06T02:31:26.264440+0000","last_active":"2023-10-06T02:31:26.268856+0000","last_peered":"2023-10-06T02:31:26.268856+0000","last_clean":"2023-10-06T02:31:26.268856+0000","last_became_active":"2023-10-06T02:31:26.263802+0000","last_became_peered":"2023-10-06T02:31:26.263802+0000","last_unstale":"2023-10-06T02:31:26.268856+0000","last_undegraded":"2023-10-06T02:31:26.268856+0000","last_fullsized":"2023-10-06T02:31:26.268856+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:53:31.912429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.239666+0000","last_change":"2023-10-06T02:31:25.246312+0000","last_active":"2023-10-06T02:31:26.239666+0000","last_peered":"2023-10-06T02:31:26.239666+0000","last_clean":"2023-10-06T02:31:26.239666+0000","last_became_active":"2023-10-06T02:31:25.246039+0000","last_became_peered":"2023-10-06T02:31:25.246039+0000","last_unstale":"2023-10-06T02:31:26.239666+0000","last_undegraded":"2023-10-06T02:31:26.239666+0000","last_fullsized":"2023-10-06T02:31:26.239666+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_clean_scrub_stamp":"2023-10-06T02:31:24.211070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:45:40.445239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.1f","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.245899+0000","last_change":"2023-10-06T02:31:26.244615+0000","last_active":"2023-10-06T02:31:26.245899+0000","last_peered":"2023-10-06T02:31:26.245899+0000","last_clean":"2023-10-06T02:31:26.245899+0000","last_became_active":"2023-10-06T02:31:26.244294+0000","last_became_peered":"2023-10-06T02:31:26.244294+0000","last_unstale":"2023-10-06T02:31:26.245899+0000","last_undegraded":"2023-10-06T02:31:26.245899+0000","last_fullsized":"2023-10-06T02:31:26.245899+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:30.357921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1c","version":"155'2","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.279892+0000","last_change":"2023-10-06T02:31:25.231065+0000","last_active":"2023-10-06T02:31:26.279892+0000","last_peered":"2023-10-06T02:31:26.279892+0000","last_clean":"2023-10-06T02:31:26.279892+0000","last_became_active":"2023-10-06T02:31:25.230438+0000","last_became_peered":"2023-10-06T02:31:25.230438+0000","last_unstale":"2023-10-06T02:31:26.279892+0000","last_undegraded":"2023-10-06T02:31:26.279892+0000","last_fullsized":"2023-10-06T02:31:26.279892+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_clean_scrub_stamp":"2023-10-06T02:31:24.211070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:53:48.222531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":124,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.239350+0000","last_change":"2023-10-06T02:23:01.992935+0000","last_active":"2023-10-06T02:31:26.239350+0000","last_peered":"2023-10-06T02:31:26.239350+0000","last_clean":"2023-10-06T02:31:26.239350+0000","last_became_active":"2023-10-06T02:23:01.992568+0000","last_became_peered":"2023-10-06T02:23:01.992568+0000","last_unstale":"2023-10-06T02:31:26.239350+0000","last_undegraded":"2023-10-06T02:31:26.239350+0000","last_fullsized":"2023-10-06T02:31:26.239350+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:06.212251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.1e","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.269959+0000","last_change":"2023-10-06T02:31:26.261651+0000","last_active":"2023-10-06T02:31:26.269959+0000","last_peered":"2023-10-06T02:31:26.269959+0000","last_clean":"2023-10-06T02:31:26.269959+0000","last_became_active":"2023-10-06T02:31:26.261365+0000","last_became_peered":"2023-10-06T02:31:26.261365+0000","last_unstale":"2023-10-06T02:31:26.269959+0000","last_undegraded":"2023-10-06T02:31:26.269959+0000","last_fullsized":"2023-10-06T02:31:26.269959+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:06:51.338576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1d","version":"155'2","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.277227+0000","last_change":"2023-10-06T02:31:25.235638+0000","last_active":"2023-10-06T02:31:26.277227+0000","last_peered":"2023-10-06T02:31:26.277227+0000","last_clean":"2023-10-06T02:31:26.277227+0000","last_became_active":"2023-10-06T02:31:25.235378+0000","last_became_peered":"2023-10-06T02:31:25.235378+0000","last_unstale":"2023-10-06T02:31:26.277227+0000","last_undegraded":"2023-10-06T02:31:26.277227+0000","last_fullsized":"2023-10-06T02:31:26.277227+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_clean_scrub_stamp":"2023-10-06T02:31:24.211070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:46:02.102704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":125,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.250973+0000","last_change":"2023-10-06T02:23:01.989934+0000","last_active":"2023-10-06T02:31:26.250973+0000","last_peered":"2023-10-06T02:31:26.250973+0000","last_clean":"2023-10-06T02:31:26.250973+0000","last_became_active":"2023-10-06T02:23:01.989706+0000","last_became_peered":"2023-10-06T02:23:01.989706+0000","last_unstale":"2023-10-06T02:31:26.250973+0000","last_undegraded":"2023-10-06T02:31:26.250973+0000","last_fullsized":"2023-10-06T02:31:26.250973+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:06.212251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.1d","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.258334+0000","last_change":"2023-10-06T02:31:26.255636+0000","last_active":"2023-10-06T02:31:26.258334+0000","last_peered":"2023-10-06T02:31:26.258334+0000","last_clean":"2023-10-06T02:31:26.258334+0000","last_became_active":"2023-10-06T02:31:26.255277+0000","last_became_peered":"2023-10-06T02:31:26.255277+0000","last_unstale":"2023-10-06T02:31:26.258334+0000","last_undegraded":"2023-10-06T02:31:26.258334+0000","last_fullsized":"2023-10-06T02:31:26.258334+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:59:02.106529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"155'1","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.280240+0000","last_change":"2023-10-06T02:31:25.243505+0000","last_active":"2023-10-06T02:31:26.280240+0000","last_peered":"2023-10-06T02:31:26.280240+0000","last_clean":"2023-10-06T02:31:26.280240+0000","last_became_active":"2023-10-06T02:31:25.243181+0000","last_became_peered":"2023-10-06T02:31:25.243181+0000","last_unstale":"2023-10-06T02:31:26.280240+0000","last_undegraded":"2023-10-06T02:31:26.280240+0000","last_fullsized":"2023-10-06T02:31:26.280240+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_clean_scrub_stamp":"2023-10-06T02:31:24.211070+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:54:51.160175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":119,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.248316+0000","last_change":"2023-10-06T02:23:02.054069+0000","last_active":"2023-10-06T02:31:26.248316+0000","last_peered":"2023-10-06T02:31:26.248316+0000","last_clean":"2023-10-06T02:31:26.248316+0000","last_became_active":"2023-10-06T02:23:02.053739+0000","last_became_peered":"2023-10-06T02:23:02.053739+0000","last_unstale":"2023-10-06T02:31:26.248316+0000","last_undegraded":"2023-10-06T02:31:26.248316+0000","last_fullsized":"2023-10-06T02:31:26.248316+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:57:48.822608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.267702+0000","last_change":"2023-10-06T02:31:26.265761+0000","last_active":"2023-10-06T02:31:26.267702+0000","last_peered":"2023-10-06T02:31:26.267702+0000","last_clean":"2023-10-06T02:31:26.267702+0000","last_became_active":"2023-10-06T02:31:26.265416+0000","last_became_peered":"2023-10-06T02:31:26.265416+0000","last_unstale":"2023-10-06T02:31:26.267702+0000","last_undegraded":"2023-10-06T02:31:26.267702+0000","last_fullsized":"2023-10-06T02:31:26.267702+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:25:17.234017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.1b","version":"155'1","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.280203+0000","last_change":"2023-10-06T02:31:25.244371+0000","last_active":"2023-10-06T02:31:26.280203+0000","last_peered":"2023-10-06T02:31:26.280203+0000","last_clean":"2023-10-06T02:31:26.280203+0000","last_became_active":"2023-10-06T02:31:25.243583+0000","last_became_peered":"2023-10-06T02:31:25.243583+0000","last_unstale":"2023-10-06T02:31:26.280203+0000","last_undegraded":"2023-10-06T02:31:26.280203+0000","last_fullsized":"2023-10-06T02:31:26.280203+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_clean_scrub_stamp":"2023-10-06T02:31:24.211070+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:40:02.130196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":119,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.250562+0000","last_change":"2023-10-06T02:23:01.990773+0000","last_active":"2023-10-06T02:31:26.250562+0000","last_peered":"2023-10-06T02:31:26.250562+0000","last_clean":"2023-10-06T02:31:26.250562+0000","last_became_active":"2023-10-06T02:23:01.990511+0000","last_became_peered":"2023-10-06T02:23:01.990511+0000","last_unstale":"2023-10-06T02:31:26.250562+0000","last_undegraded":"2023-10-06T02:31:26.250562+0000","last_fullsized":"2023-10-06T02:31:26.250562+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:42:54.797731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1b","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.265088+0000","last_change":"2023-10-06T02:31:26.259210+0000","last_active":"2023-10-06T02:31:26.265088+0000","last_peered":"2023-10-06T02:31:26.265088+0000","last_clean":"2023-10-06T02:31:26.265088+0000","last_became_active":"2023-10-06T02:31:26.258923+0000","last_became_peered":"2023-10-06T02:31:26.258923+0000","last_unstale":"2023-10-06T02:31:26.265088+0000","last_undegraded":"2023-10-06T02:31:26.265088+0000","last_fullsized":"2023-10-06T02:31:26.265088+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:35:10.584911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.244477+0000","last_change":"2023-10-06T02:31:25.235892+0000","last_active":"2023-10-06T02:31:26.244477+0000","last_peered":"2023-10-06T02:31:26.244477+0000","last_clean":"2023-10-06T02:31:26.244477+0000","last_became_active":"2023-10-06T02:31:25.235626+0000","last_became_peered":"2023-10-06T02:31:25.235626+0000","last_unstale":"2023-10-06T02:31:26.244477+0000","last_undegraded":"2023-10-06T02:31:26.244477+0000","last_fullsized":"2023-10-06T02:31:26.244477+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_clean_scrub_stamp":"2023-10-06T02:31:24.211070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:33:18.206678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"76'128","reported_seq":311,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.250649+0000","last_change":"2023-10-06T02:22:55.032519+0000","last_active":"2023-10-06T02:31:26.250649+0000","last_peered":"2023-10-06T02:31:26.250649+0000","last_clean":"2023-10-06T02:31:26.250649+0000","last_became_active":"2023-10-06T02:22:55.031367+0000","last_became_peered":"2023-10-06T02:22:55.031367+0000","last_unstale":"2023-10-06T02:31:26.250649+0000","last_undegraded":"2023-10-06T02:31:26.250649+0000","last_fullsized":"2023-10-06T02:31:26.250649+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:22:53.309097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:22:53.309097+0000","last_clean_scrub_stamp":"2023-10-06T02:22:53.309097+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:53:46.520909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":146,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.241040+0000","last_change":"2023-10-06T02:23:01.988140+0000","last_active":"2023-10-06T02:31:26.241040+0000","last_peered":"2023-10-06T02:31:26.241040+0000","last_clean":"2023-10-06T02:31:26.241040+0000","last_became_active":"2023-10-06T02:23:01.987769+0000","last_became_peered":"2023-10-06T02:23:01.987769+0000","last_unstale":"2023-10-06T02:31:26.241040+0000","last_undegraded":"2023-10-06T02:31:26.241040+0000","last_fullsized":"2023-10-06T02:31:26.241040+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:20:08.391912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.264621+0000","last_change":"2023-10-06T02:31:26.254985+0000","last_active":"2023-10-06T02:31:26.264621+0000","last_peered":"2023-10-06T02:31:26.264621+0000","last_clean":"2023-10-06T02:31:26.264621+0000","last_became_active":"2023-10-06T02:31:26.254597+0000","last_became_peered":"2023-10-06T02:31:26.254597+0000","last_unstale":"2023-10-06T02:31:26.264621+0000","last_undegraded":"2023-10-06T02:31:26.264621+0000","last_fullsized":"2023-10-06T02:31:26.264621+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:55:50.761151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1a","version":"155'1","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.275434+0000","last_change":"2023-10-06T02:31:25.234189+0000","last_active":"2023-10-06T02:31:26.275434+0000","last_peered":"2023-10-06T02:31:26.275434+0000","last_clean":"2023-10-06T02:31:26.275434+0000","last_became_active":"2023-10-06T02:31:25.233908+0000","last_became_peered":"2023-10-06T02:31:25.233908+0000","last_unstale":"2023-10-06T02:31:26.275434+0000","last_undegraded":"2023-10-06T02:31:26.275434+0000","last_fullsized":"2023-10-06T02:31:26.275434+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_clean_scrub_stamp":"2023-10-06T02:31:24.211070+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:40:02.130196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":134,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.239982+0000","last_change":"2023-10-06T02:23:02.054338+0000","last_active":"2023-10-06T02:31:26.239982+0000","last_peered":"2023-10-06T02:31:26.239982+0000","last_clean":"2023-10-06T02:31:26.239982+0000","last_became_active":"2023-10-06T02:23:02.053991+0000","last_became_peered":"2023-10-06T02:23:02.053991+0000","last_unstale":"2023-10-06T02:31:26.239982+0000","last_undegraded":"2023-10-06T02:31:26.239982+0000","last_fullsized":"2023-10-06T02:31:26.239982+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:39:30.192838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.1a","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.269807+0000","last_change":"2023-10-06T02:31:26.268448+0000","last_active":"2023-10-06T02:31:26.269807+0000","last_peered":"2023-10-06T02:31:26.269807+0000","last_clean":"2023-10-06T02:31:26.269807+0000","last_became_active":"2023-10-06T02:31:26.268093+0000","last_became_peered":"2023-10-06T02:31:26.268093+0000","last_unstale":"2023-10-06T02:31:26.269807+0000","last_undegraded":"2023-10-06T02:31:26.269807+0000","last_fullsized":"2023-10-06T02:31:26.269807+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:18:33.267912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.248295+0000","last_change":"2023-10-06T02:31:25.238923+0000","last_active":"2023-10-06T02:31:26.248295+0000","last_peered":"2023-10-06T02:31:26.248295+0000","last_clean":"2023-10-06T02:31:26.248295+0000","last_became_active":"2023-10-06T02:31:25.238641+0000","last_became_peered":"2023-10-06T02:31:25.238641+0000","last_unstale":"2023-10-06T02:31:26.248295+0000","last_undegraded":"2023-10-06T02:31:26.248295+0000","last_fullsized":"2023-10-06T02:31:26.248295+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_clean_scrub_stamp":"2023-10-06T02:31:24.211070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:45:30.418711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":146,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.240017+0000","last_change":"2023-10-06T02:23:01.988631+0000","last_active":"2023-10-06T02:31:26.240017+0000","last_peered":"2023-10-06T02:31:26.240017+0000","last_clean":"2023-10-06T02:31:26.240017+0000","last_became_active":"2023-10-06T02:23:01.988303+0000","last_became_peered":"2023-10-06T02:23:01.988303+0000","last_unstale":"2023-10-06T02:31:26.240017+0000","last_undegraded":"2023-10-06T02:31:26.240017+0000","last_fullsized":"2023-10-06T02:31:26.240017+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:38.211466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1c","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.263651+0000","last_change":"2023-10-06T02:31:26.259907+0000","last_active":"2023-10-06T02:31:26.263651+0000","last_peered":"2023-10-06T02:31:26.263651+0000","last_clean":"2023-10-06T02:31:26.263651+0000","last_became_active":"2023-10-06T02:31:26.259641+0000","last_became_peered":"2023-10-06T02:31:26.259641+0000","last_unstale":"2023-10-06T02:31:26.263651+0000","last_undegraded":"2023-10-06T02:31:26.263651+0000","last_fullsized":"2023-10-06T02:31:26.263651+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:25.213576+0000","last_clean_scrub_stamp":"2023-10-06T02:31:25.213576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:39:16.981754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1f","version":"155'4","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.275796+0000","last_change":"2023-10-06T02:31:25.227308+0000","last_active":"2023-10-06T02:31:26.275796+0000","last_peered":"2023-10-06T02:31:26.275796+0000","last_clean":"2023-10-06T02:31:26.275796+0000","last_became_active":"2023-10-06T02:31:25.226372+0000","last_became_peered":"2023-10-06T02:31:25.226372+0000","last_unstale":"2023-10-06T02:31:26.275796+0000","last_undegraded":"2023-10-06T02:31:26.275796+0000","last_fullsized":"2023-10-06T02:31:26.275796+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:31:24.211070+0000","last_clean_scrub_stamp":"2023-10-06T02:31:24.211070+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:26:39.899368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":125,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T02:31:26.250065+0000","last_change":"2023-10-06T02:23:02.054039+0000","last_active":"2023-10-06T02:31:26.250065+0000","last_peered":"2023-10-06T02:31:26.250065+0000","last_clean":"2023-10-06T02:31:26.250065+0000","last_became_active":"2023-10-06T02:23:02.053742+0000","last_became_peered":"2023-10-06T02:23:02.053742+0000","last_unstale":"2023-10-06T02:31:26.250065+0000","last_undegraded":"2023-10-06T02:31:26.250065+0000","last_fullsized":"2023-10-06T02:31:26.250065+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:58:18.846475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":25,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574956,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303884,"kb_used_data":2500,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067956,"statfs":{"total":96636764160,"available":96325586944,"internally_reserved":0,"allocated":2560000,"data_stored":1507692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:31:14 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.622,"15min":0.63300000000000001},"min":{"1min":0.35299999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.78300000000000003,"5min":1.169,"15min":1.169},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.624,"15min":0.61199999999999999},"min":{"1min":0.43099999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.91800000000000004,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.75600000000000001}]},{"osd":1,"last update":"Fri Oct 6 02:31:14 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71399999999999997,"15min":0.69699999999999995},"min":{"1min":0.57899999999999996,"5min":0.45600000000000002,"15min":0.40300000000000002},"max":{"1min":0.86199999999999999,"5min":1.1459999999999999,"15min":1.147},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65900000000000003,"15min":0.67100000000000004},"min":{"1min":0.52100000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83899999999999997,"5min":1.129,"15min":1.129},"last":0.77700000000000002}]},{"osd":2,"last update":"Fri Oct 6 02:31:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.65500000000000003,"15min":0.68300000000000005},"min":{"1min":0.46700000000000003,"5min":0.375,"15min":0.375},"max":{"1min":0.81699999999999995,"5min":1.238,"15min":1.238},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.68700000000000006,"15min":0.69099999999999995},"min":{"1min":0.55000000000000004,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.93700000000000006,"5min":1.242,"15min":1.242},"last":0.67900000000000005}]},{"osd":3,"last update":"Fri Oct 6 02:31:14 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.70799999999999996,"15min":0.72399999999999998},"min":{"1min":0.45500000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.84299999999999997,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.75700000000000001,"15min":0.77200000000000002},"min":{"1min":0.61399999999999999,"5min":0.54500000000000004,"15min":0.497},"max":{"1min":0.95199999999999996,"5min":1.383,"15min":1.383},"last":0.82399999999999995}]},{"osd":4,"last update":"Fri Oct 6 02:31:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.60299999999999998,"15min":0.66700000000000004},"min":{"1min":0.41799999999999998,"5min":0.41099999999999998,"15min":0.35099999999999998},"max":{"1min":0.91700000000000004,"5min":1.1120000000000001,"15min":1.2849999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.67300000000000004,"15min":0.73899999999999999},"min":{"1min":0.40899999999999997,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.96299999999999997,"5min":1.51,"15min":1.51},"last":0.71099999999999997}]},{"osd":5,"last update":"Fri Oct 6 02:31:14 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.70099999999999996,"15min":0.72499999999999998},"min":{"1min":0.503,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.91900000000000004,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.67000000000000004,"15min":0.71599999999999997},"min":{"1min":0.48999999999999999,"5min":0.442,"15min":0.442},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0289999999999999},"last":0.73299999999999998}]},{"osd":6,"last update":"Fri Oct 6 02:31:14 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.752,"15min":0.77800000000000002},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.93400000000000005,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.77100000000000002,"15min":0.80300000000000005},"min":{"1min":0.628,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.90500000000000003,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.90400000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574956,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303308,"kb_used_data":1924,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068532,"statfs":{"total":96636764160,"available":96326176768,"internally_reserved":0,"allocated":1970176,"data_stored":917394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:31:09 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.67400000000000004,"15min":0.66700000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.72099999999999997,"15min":0.70399999999999996},"min":{"1min":0.498,"5min":0.374,"15min":0.374},"max":{"1min":1.04,"5min":1.04,"15min":1.119},"last":0.51000000000000001}]},{"osd":1,"last update":"Fri Oct 6 02:31:09 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.66800000000000004,"15min":0.65000000000000002},"min":{"1min":0.48699999999999999,"5min":0.433,"15min":0.40699999999999997},"max":{"1min":0.83799999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.66600000000000004,"15min":0.66700000000000004},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.94499999999999995,"5min":0.997,"15min":0.997},"last":0.56999999999999995}]},{"osd":2,"last update":"Fri Oct 6 02:31:09 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.71599999999999997,"15min":0.70599999999999996},"min":{"1min":0.51000000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.002,"5min":1.0449999999999999,"15min":1.137},"last":0.748},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.72799999999999998,"15min":0.71799999999999997},"min":{"1min":0.47799999999999998,"5min":0.45300000000000001,"15min":0.40300000000000002},"max":{"1min":0.98699999999999999,"5min":1.077,"15min":1.077},"last":0.71299999999999997}]},{"osd":3,"last update":"Fri Oct 6 02:31:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.70099999999999996,"15min":0.67200000000000004},"min":{"1min":0.51500000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.82099999999999995,"5min":1.056,"15min":1.056},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.71399999999999997,"15min":0.70699999999999996},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.45900000000000002},"max":{"1min":0.91600000000000004,"5min":0.97999999999999998,"15min":0.98499999999999999},"last":0.69499999999999995}]},{"osd":4,"last update":"Fri Oct 6 02:31:09 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.66500000000000004,"15min":0.64500000000000002},"min":{"1min":0.377,"5min":0.36199999999999999,"15min":0.34300000000000003},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":1.095},"last":0.63},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.71299999999999997,"15min":0.67500000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.998,"5min":1.002,"15min":1.29},"last":0.65300000000000002}]},{"osd":5,"last update":"Fri Oct 6 02:31:09 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.69799999999999995,"15min":0.69999999999999996},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.39700000000000002},"max":{"1min":0.98699999999999999,"5min":1.014,"15min":1.2929999999999999},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.71999999999999997,"15min":0.70899999999999996},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.81799999999999995}]},{"osd":7,"last update":"Fri Oct 6 02:31:09 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.755,"15min":0.751},"min":{"1min":0.44700000000000001,"5min":0.41499999999999998,"15min":0.36799999999999999},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.458},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.77700000000000002,"15min":0.76500000000000001},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.45900000000000002},"max":{"1min":1.0429999999999999,"5min":1.103,"15min":1.2470000000000001},"last":0.875}]}]},{"osd":1,"up_from":13,"seq":55834574955,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303304,"kb_used_data":1920,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068536,"statfs":{"total":96636764160,"available":96326180864,"internally_reserved":0,"allocated":1966080,"data_stored":917324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:31:05 2023","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.40999999999999998,"15min":0.44700000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.46999999999999997,"5min":0.63600000000000001,"15min":1.2230000000000001},"last":0.38},{"interface":"front","average":{"1min":0.434,"5min":0.45600000000000002,"15min":0.50600000000000001},"min":{"1min":0.32800000000000001,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.64500000000000002,"5min":0.71799999999999997,"15min":1.788},"last":0.60099999999999998}]},{"osd":2,"last update":"Fri Oct 6 02:31:05 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.433,"15min":0.498},"min":{"1min":0.35199999999999998,"5min":0.307,"15min":0.28299999999999997},"max":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":2.0840000000000001},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.49099999999999999,"15min":0.54000000000000004},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.60599999999999998,"5min":0.88100000000000001,"15min":1.6699999999999999},"last":0.5}]},{"osd":3,"last update":"Fri Oct 6 02:31:05 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.58099999999999996,"15min":0.60599999999999998},"min":{"1min":0.43099999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.67000000000000004,"5min":0.89900000000000002,"15min":1.266},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.53500000000000003,"15min":0.60999999999999999},"min":{"1min":0.41599999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.77900000000000003,"5min":0.79400000000000004,"15min":1.861},"last":0.436}]},{"osd":4,"last update":"Fri Oct 6 02:31:05 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.74299999999999999,"15min":0.82699999999999996},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.006,"5min":1.3580000000000001,"15min":1.9279999999999999},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.752,"15min":0.82999999999999996},"min":{"1min":0.46899999999999997,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.98999999999999999,"5min":1.4339999999999999,"15min":2.0179999999999998},"last":0.73199999999999998}]},{"osd":5,"last update":"Fri Oct 6 02:31:05 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.71199999999999997,"15min":0.82099999999999995},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.016,"5min":1.3069999999999999,"15min":1.9830000000000001},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.72399999999999998,"15min":0.79700000000000004},"min":{"1min":0.54800000000000004,"5min":0.53600000000000003,"15min":0.47299999999999998},"max":{"1min":0.999,"5min":1.327,"15min":1.3380000000000001},"last":0.80900000000000005}]},{"osd":6,"last update":"Fri Oct 6 02:31:05 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.75600000000000001,"15min":0.83699999999999997},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.45600000000000002},"max":{"1min":0.98299999999999998,"5min":1.0609999999999999,"15min":1.829},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.77600000000000002,"15min":0.876},"min":{"1min":0.57899999999999996,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.94699999999999995,"5min":1.351,"15min":2.052},"last":0.67400000000000004}]},{"osd":7,"last update":"Fri Oct 6 02:31:05 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.73299999999999998,"15min":0.80900000000000005},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.93899999999999995,"5min":1.3120000000000001,"15min":1.3340000000000001},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.751,"15min":0.84399999999999997},"min":{"1min":0.52700000000000002,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.0369999999999999,"5min":1.2110000000000001,"15min":1.895},"last":0.88200000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574956,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303900,"kb_used_data":2516,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067940,"statfs":{"total":96636764160,"available":96325570560,"internally_reserved":0,"allocated":2576384,"data_stored":1508392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 02:31:15 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.44600000000000001,"15min":0.45300000000000001},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.76400000000000001,"5min":2.6120000000000001,"15min":2.6120000000000001},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.46999999999999997,"15min":0.47999999999999998},"min":{"1min":0.30599999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.77900000000000003,"5min":3.4199999999999999,"15min":3.4199999999999999},"last":0.438}]},{"osd":2,"last update":"Fri Oct 6 02:31:15 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.52000000000000002,"15min":0.51500000000000001},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.58199999999999996,"5min":3.0339999999999998,"15min":3.0339999999999998},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.70699999999999996,"5min":3.343,"15min":3.343},"last":0.58799999999999997}]},{"osd":3,"last update":"Fri Oct 6 02:31:15 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.505,"15min":0.50600000000000001},"min":{"1min":0.34399999999999997,"5min":0.31,"15min":0.31},"max":{"1min":0.79400000000000004,"5min":3.6579999999999999,"15min":3.6579999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.59099999999999997,"15min":0.59399999999999997},"min":{"1min":0.443,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.82099999999999995,"5min":2.6890000000000001,"15min":2.6890000000000001},"last":0.65700000000000003}]},{"osd":4,"last update":"Fri Oct 6 02:31:15 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.76400000000000001,"15min":0.79200000000000004},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.94199999999999995,"5min":3.2999999999999998,"15min":3.2999999999999998},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.755,"15min":0.78900000000000003},"min":{"1min":0.54400000000000004,"5min":0.51000000000000001,"15min":0.48499999999999999},"max":{"1min":1.085,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":0.89700000000000002}]},{"osd":5,"last update":"Fri Oct 6 02:31:15 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.74299999999999999,"15min":0.77600000000000002},"min":{"1min":0.51200000000000001,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.038,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.79000000000000004,"15min":0.79900000000000004},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.071,"5min":3.1110000000000002,"15min":3.1110000000000002},"last":0.81599999999999995}]},{"osd":6,"last update":"Fri Oct 6 02:31:15 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.79600000000000004,"15min":0.81599999999999995},"min":{"1min":0.52900000000000003,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.103,"5min":3.1600000000000001,"15min":3.1600000000000001},"last":0.878},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.76900000000000002,"15min":0.81299999999999994},"min":{"1min":0.56200000000000006,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.01,"5min":2.3250000000000002,"15min":2.3250000000000002},"last":0.90900000000000003}]},{"osd":7,"last update":"Fri Oct 6 02:31:15 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.81399999999999995,"15min":0.82399999999999995},"min":{"1min":0.57999999999999996,"5min":0.56899999999999995,"15min":0.56000000000000005},"max":{"1min":1.048,"5min":3.3809999999999998,"15min":3.3809999999999998},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.753,"5min":0.77200000000000002,"15min":0.81200000000000006},"min":{"1min":0.56100000000000005,"5min":0.54700000000000004,"15min":0.53300000000000003},"max":{"1min":1.022,"5min":2.3740000000000001,"15min":2.3740000000000001},"last":0.92400000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574957,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303308,"kb_used_data":1924,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068532,"statfs":{"total":96636764160,"available":96326176768,"internally_reserved":0,"allocated":1970176,"data_stored":917878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:31:14 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.41999999999999998,"15min":0.435},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.63500000000000001,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.46500000000000002,"15min":0.47999999999999998},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.65300000000000002,"5min":0.88700000000000001,"15min":1.0800000000000001},"last":0.63500000000000001}]},{"osd":1,"last update":"Fri Oct 6 02:31:14 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47499999999999998,"15min":0.496},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.32400000000000001},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.51000000000000001,"15min":0.57899999999999996},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.622,"5min":1.778,"15min":1.778},"last":0.47499999999999998}]},{"osd":3,"last update":"Fri Oct 6 02:31:14 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.499,"15min":0.52500000000000002},"min":{"1min":0.33700000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.68400000000000005,"5min":0.83699999999999997,"15min":0.97499999999999998},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.57799999999999996,"15min":0.60899999999999999},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":1.028},"last":0.49399999999999999}]},{"osd":4,"last update":"Fri Oct 6 02:31:14 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.746,"15min":0.76900000000000002},"min":{"1min":0.496,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.1319999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.76200000000000001,"15min":0.82499999999999996},"min":{"1min":0.51200000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.95099999999999996,"5min":1.907,"15min":1.907},"last":0.747}]},{"osd":5,"last update":"Fri Oct 6 02:31:14 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.73299999999999998,"15min":0.78200000000000003},"min":{"1min":0.47599999999999998,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.024,"5min":1.109,"15min":1.1850000000000001},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.745,"5min":0.752,"15min":0.80400000000000005},"min":{"1min":0.56200000000000006,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.258},"last":0.624}]},{"osd":6,"last update":"Fri Oct 6 02:31:14 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.78500000000000003,"15min":0.83099999999999996},"min":{"1min":0.48399999999999999,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.1100000000000001,"5min":1.827,"15min":1.827},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.747,"5min":0.76700000000000002,"15min":0.78900000000000003},"min":{"1min":0.56100000000000005,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0269999999999999,"5min":1.0660000000000001,"15min":1.1659999999999999},"last":0.60899999999999999}]},{"osd":7,"last update":"Fri Oct 6 02:31:14 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.79400000000000004,"15min":0.86099999999999999},"min":{"1min":0.57099999999999995,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0649999999999999,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.751,"5min":0.77900000000000003,"15min":0.81799999999999995},"min":{"1min":0.56100000000000005,"5min":0.55300000000000005,"15min":0.54800000000000004},"max":{"1min":1.0269999999999999,"5min":1.125,"15min":1.1479999999999999},"last":0.755}]}]},{"osd":3,"up_from":13,"seq":55834574954,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303312,"kb_used_data":1928,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068528,"statfs":{"total":96636764160,"available":96326172672,"internally_reserved":0,"allocated":1974272,"data_stored":917484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:31:12 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.434,"15min":0.48799999999999999},"min":{"1min":0.29099999999999998,"5min":0.248,"15min":0.248},"max":{"1min":0.60799999999999998,"5min":1.0489999999999999,"15min":2.024},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.441,"5min":0.46800000000000003,"15min":0.50900000000000001},"min":{"1min":0.31900000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.53300000000000003,"5min":0.754,"15min":2.181},"last":0.33200000000000002}]},{"osd":1,"last update":"Fri Oct 6 02:31:12 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.45800000000000002,"15min":0.62},"min":{"1min":0.33400000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.61299999999999999,"5min":0.66100000000000003,"15min":4.492},"last":0.443},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.51600000000000001,"15min":0.66800000000000004},"min":{"1min":0.377,"5min":0.313,"15min":0.313},"max":{"1min":0.67100000000000004,"5min":1.079,"15min":4.569},"last":0.48299999999999998}]},{"osd":2,"last update":"Fri Oct 6 02:31:12 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.59699999999999998,"15min":0.64800000000000002},"min":{"1min":0.46899999999999997,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.79400000000000004,"5min":1.024,"15min":2.0870000000000002},"last":0.626},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.56699999999999995,"15min":0.627},"min":{"1min":0.42799999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.72999999999999998,"5min":0.94199999999999995,"15min":2.4260000000000002},"last":0.50800000000000001}]},{"osd":4,"last update":"Fri Oct 6 02:31:12 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73299999999999998,"15min":0.80200000000000005},"min":{"1min":0.58199999999999996,"5min":0.55000000000000004,"15min":0.44800000000000001},"max":{"1min":0.97299999999999998,"5min":1.0149999999999999,"15min":2.214},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.75,"15min":0.79600000000000004},"min":{"1min":0.53000000000000003,"5min":0.50800000000000001,"15min":0.5},"max":{"1min":1.042,"5min":1.042,"15min":1.2869999999999999},"last":0.83299999999999996}]},{"osd":5,"last update":"Fri Oct 6 02:31:12 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.747,"15min":0.80300000000000005},"min":{"1min":0.55300000000000005,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.97499999999999998,"5min":1.077,"15min":1.3029999999999999},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.754,"5min":0.75600000000000001,"15min":0.82599999999999996},"min":{"1min":0.57999999999999996,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.0549999999999999,"5min":1.0580000000000001,"15min":2.242},"last":0.79500000000000004}]},{"osd":6,"last update":"Fri Oct 6 02:31:12 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.77300000000000002,"15min":0.80500000000000005},"min":{"1min":0.58199999999999996,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.96199999999999997,"5min":1.0489999999999999,"15min":1.357},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.78800000000000003,"15min":0.82099999999999995},"min":{"1min":0.64900000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0049999999999999,"5min":1.0529999999999999,"15min":1.3100000000000001},"last":0.92200000000000004}]},{"osd":7,"last update":"Fri Oct 6 02:31:12 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.76600000000000001,"15min":0.83399999999999996},"min":{"1min":0.54400000000000004,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.98399999999999999,"5min":1.077,"15min":2.27},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.78600000000000003,"15min":0.83899999999999997},"min":{"1min":0.57599999999999996,"5min":0.55600000000000005,"15min":0.53200000000000003},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.325},"last":0.76700000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574956,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303320,"kb_used_data":1936,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068520,"statfs":{"total":96636764160,"available":96326164480,"internally_reserved":0,"allocated":1982464,"data_stored":918894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:31:06 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.65200000000000002,"15min":0.92200000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.84799999999999998,"5min":1.0760000000000001,"15min":12.337},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.67800000000000005,"15min":0.85899999999999999},"min":{"1min":0.48499999999999999,"5min":0.44400000000000001,"15min":0.438},"max":{"1min":0.94299999999999995,"5min":1.3520000000000001,"15min":9.6539999999999999},"last":0.77200000000000002}]},{"osd":1,"last update":"Fri Oct 6 02:31:06 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.66300000000000003,"15min":0.68899999999999995},"min":{"1min":0.42099999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.97899999999999998,"5min":1.129,"15min":3.181},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.70199999999999996,"15min":0.94999999999999996},"min":{"1min":0.45300000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.92700000000000005,"5min":0.995,"15min":12.946999999999999},"last":0.52400000000000002}]},{"osd":2,"last update":"Fri Oct 6 02:31:06 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.69999999999999996,"15min":0.96799999999999997},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.89200000000000002,"5min":1.2989999999999999,"15min":12.377000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.72699999999999998,"15min":0.83899999999999997},"min":{"1min":0.51600000000000001,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.90800000000000003,"5min":1.0529999999999999,"15min":6.3220000000000001},"last":0.63800000000000001}]},{"osd":3,"last update":"Fri Oct 6 02:31:06 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72599999999999998,"15min":0.93100000000000005},"min":{"1min":0.54900000000000004,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.91100000000000003,"5min":1.5229999999999999,"15min":9.7720000000000002},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.70999999999999996,"15min":0.76000000000000001},"min":{"1min":0.53400000000000003,"5min":0.498,"15min":0.47999999999999998},"max":{"1min":1.0389999999999999,"5min":1.212,"15min":3.0840000000000001},"last":0.56799999999999995}]},{"osd":5,"last update":"Fri Oct 6 02:31:06 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.70599999999999996,"15min":0.81499999999999995},"min":{"1min":0.54000000000000004,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.96299999999999997,"5min":1.02,"15min":6.2690000000000001},"last":0.753},{"interface":"front","average":{"1min":0.751,"5min":0.73999999999999999,"15min":0.96799999999999997},"min":{"1min":0.53900000000000003,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.95199999999999996,"5min":1.4359999999999999,"15min":10.769},"last":0.878}]},{"osd":6,"last update":"Fri Oct 6 02:31:06 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.70999999999999996,"15min":0.749},"min":{"1min":0.46100000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.1919999999999999,"5min":1.4590000000000001,"15min":3.1349999999999998},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79100000000000004,"15min":0.93100000000000005},"min":{"1min":0.66700000000000004,"5min":0.47799999999999998,"15min":0.45700000000000002},"max":{"1min":1.008,"5min":1.3839999999999999,"15min":8.0899999999999999},"last":0.79800000000000004}]},{"osd":7,"last update":"Fri Oct 6 02:31:06 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.79600000000000004,"15min":1.006},"min":{"1min":0.60999999999999999,"5min":0.501,"15min":0.47599999999999998},"max":{"1min":1.024,"5min":1.494,"15min":9.7089999999999996},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.78000000000000003,"15min":0.94899999999999995},"min":{"1min":0.55900000000000005,"5min":0.374,"15min":0.374},"max":{"1min":1.1399999999999999,"5min":1.2669999999999999,"15min":7.6719999999999997},"last":0.85499999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574956,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303320,"kb_used_data":1936,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068520,"statfs":{"total":96636764160,"available":96326164480,"internally_reserved":0,"allocated":1982464,"data_stored":918882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:31:05 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.624,"15min":0.627},"min":{"1min":0.38100000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.82799999999999996,"5min":0.97599999999999998,"15min":1.4279999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.63,"15min":0.61799999999999999},"min":{"1min":0.436,"5min":0.40300000000000002,"15min":0.38700000000000001},"max":{"1min":1.0409999999999999,"5min":1.099,"15min":1.099},"last":0.64300000000000002}]},{"osd":1,"last update":"Fri Oct 6 02:31:05 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.627,"15min":0.64500000000000002},"min":{"1min":0.45400000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.64700000000000002,"5min":1.0660000000000001,"15min":1.089},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.442,"5min":0.40699999999999997,"15min":0.40500000000000003},"max":{"1min":1.0649999999999999,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.84699999999999998}]},{"osd":2,"last update":"Fri Oct 6 02:31:05 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.66200000000000003,"15min":0.67000000000000004},"min":{"1min":0.48799999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.84899999999999998,"5min":1.016,"15min":1.016},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.66300000000000003,"15min":0.67900000000000005},"min":{"1min":0.53300000000000003,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.88400000000000001,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.90900000000000003}]},{"osd":3,"last update":"Fri Oct 6 02:31:05 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.71999999999999997,"15min":0.72699999999999998},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.93999999999999995,"5min":1.159,"15min":1.159},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.76300000000000001,"15min":0.76700000000000002},"min":{"1min":0.58899999999999997,"5min":0.56200000000000006,"15min":0.54100000000000004},"max":{"1min":1.0149999999999999,"5min":1.194,"15min":1.194},"last":0.86399999999999999}]},{"osd":4,"last update":"Fri Oct 6 02:31:05 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67000000000000004,"15min":0.67500000000000004},"min":{"1min":0.40799999999999997,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68300000000000005,"15min":0.68500000000000005},"min":{"1min":0.44700000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.1519999999999999,"5min":1.25,"15min":1.25},"last":0.46600000000000003}]},{"osd":6,"last update":"Fri Oct 6 02:31:05 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.71699999999999997,"15min":0.73699999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.40300000000000002},"max":{"1min":0.88200000000000001,"5min":1.0549999999999999,"15min":1.1719999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.80300000000000005,"15min":0.81799999999999995},"min":{"1min":0.60399999999999998,"5min":0.373,"15min":0.373},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.62}]},{"osd":7,"last update":"Fri Oct 6 02:31:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.745,"15min":0.76100000000000001},"min":{"1min":0.48499999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.093,"5min":1.23,"15min":1.23},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.79200000000000004,"15min":0.82399999999999995},"min":{"1min":0.47599999999999998,"5min":0.46999999999999997,"15min":0.437},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.5640000000000001},"last":0.66500000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T02:31:31.113 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T02:31:31.114 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T02:31:31.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.338+0000 7f2c6b59e700 1 Processor -- start 2023-10-06T02:31:31.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.338+0000 7f2c6b59e700 1 -- start start 2023-10-06T02:31:31.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.339+0000 7f2c6b59e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2c6c13fb20 0x7f2c6c15a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:31.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.339+0000 7f2c6b59e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2c6c13fef0 0x7f2c6c0c7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:31.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.339+0000 7f2c6b59e700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c6c15ae70 con 0x7f2c6c06aaa0 2023-10-06T02:31:31.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.339+0000 7f2c6b59e700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c6c06af60 con 0x7f2c6c13fef0 2023-10-06T02:31:31.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.339+0000 7f2c6b59e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6c06b240 con 0x7f2c6c13fb20 2023-10-06T02:31:31.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.340+0000 7f2c69d9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2c6c13fef0 0x7f2c6c0c7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:31.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.340+0000 7f2c69d9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2c6c13fef0 0x7f2c6c0c7990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51568/0 (socket says 172.21.15.31:51568) 2023-10-06T02:31:31.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.340+0000 7f2c69d9b700 1 -- 172.21.15.31:0/1862062279 learned_addr learned my addr 172.21.15.31:0/1862062279 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:31.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.340+0000 7f2c6ad9d700 1 --1- 172.21.15.31:0/1862062279 >> v1:172.21.15.31:6789/0 conn(0x7f2c6c06aaa0 0x7f2c6c0c6770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58674/0 (socket says 172.21.15.31:58674) 2023-10-06T02:31:31.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.340+0000 7f2c6a59c700 1 --2- 172.21.15.31:0/1862062279 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2c6c13fb20 0x7f2c6c15a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:31.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.340+0000 7f2c69d9b700 1 -- 172.21.15.31:0/1862062279 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2c6c13fb20 msgr2=0x7f2c6c15a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:31.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.340+0000 7f2c69d9b700 1 --2- 172.21.15.31:0/1862062279 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2c6c13fb20 0x7f2c6c15a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:31.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.340+0000 7f2c69d9b700 1 -- 172.21.15.31:0/1862062279 >> v1:172.21.15.31:6789/0 conn(0x7f2c6c06aaa0 legacy=0x7f2c6c0c6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:31.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.340+0000 7f2c69d9b700 1 -- 172.21.15.31:0/1862062279 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c6c06b520 con 0x7f2c6c13fef0 2023-10-06T02:31:31.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.341+0000 7f2c69d9b700 1 --2- 172.21.15.31:0/1862062279 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c6c13fef0 0x7f2c6c0c7990 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f2c60009990 tx=0x7f2c60009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80f805fbb2c3bd08 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:31.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.341+0000 7f2c6959a700 1 -- 172.21.15.31:0/1862062279 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c60005c10 con 0x7f2c6c13fef0 2023-10-06T02:31:31.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.341+0000 7f2c6959a700 1 -- 172.21.15.31:0/1862062279 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c60005d70 con 0x7f2c6c13fef0 2023-10-06T02:31:31.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.341+0000 7f2c6959a700 1 -- 172.21.15.31:0/1862062279 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c6000b880 con 0x7f2c6c13fef0 2023-10-06T02:31:31.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.341+0000 7f2c6b59e700 1 -- 172.21.15.31:0/1862062279 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c6c13fef0 msgr2=0x7f2c6c0c7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:31.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.341+0000 7f2c6b59e700 1 --2- 172.21.15.31:0/1862062279 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c6c13fef0 0x7f2c6c0c7990 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f2c60009990 tx=0x7f2c60009220 comp rx=0 tx=0).stop 2023-10-06T02:31:31.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.341+0000 7f2c6b59e700 1 -- 172.21.15.31:0/1862062279 shutdown_connections 2023-10-06T02:31:31.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.341+0000 7f2c6b59e700 1 --2- 172.21.15.31:0/1862062279 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2c6c13fb20 0x7f2c6c15a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:31.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.341+0000 7f2c6b59e700 1 --2- 172.21.15.31:0/1862062279 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c6c13fef0 0x7f2c6c0c7990 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:31.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.342+0000 7f2c6b59e700 1 -- 172.21.15.31:0/1862062279 >> 172.21.15.31:0/1862062279 conn(0x7f2c6c06d170 msgr2=0x7f2c6c0cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:31.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.342+0000 7f2c6b59e700 1 -- 172.21.15.31:0/1862062279 shutdown_connections 2023-10-06T02:31:31.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.342+0000 7f2c6b59e700 1 -- 172.21.15.31:0/1862062279 wait complete. 2023-10-06T02:31:31.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.343+0000 7f2c6b59e700 1 Processor -- start 2023-10-06T02:31:31.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.343+0000 7f2c6b59e700 1 -- start start 2023-10-06T02:31:31.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.344+0000 7f2c6b59e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c6c06aaa0 0x7f2c6c150600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:31.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.344+0000 7f2c6b59e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c6c13fb20 0x7f2c6c137d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:31.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.344+0000 7f2c6b59e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2c6c13fef0 0x7f2c6c158880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:31.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.344+0000 7f2c6b59e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6c138260 con 0x7f2c6c13fb20 2023-10-06T02:31:31.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.344+0000 7f2c6b59e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6c159340 con 0x7f2c6c06aaa0 2023-10-06T02:31:31.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.344+0000 7f2c6b59e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6c14dc90 con 0x7f2c6c13fef0 2023-10-06T02:31:31.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.344+0000 7f2c69d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c6c13fb20 0x7f2c6c137d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:31.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.344+0000 7f2c69d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c6c13fb20 0x7f2c6c137d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51572/0 (socket says 172.21.15.31:51572) 2023-10-06T02:31:31.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.344+0000 7f2c69d9b700 1 -- 172.21.15.31:0/4207681349 learned_addr learned my addr 172.21.15.31:0/4207681349 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:31.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.345+0000 7f2c6ad9d700 1 --2- 172.21.15.31:0/4207681349 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2c6c13fef0 0x7f2c6c158880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:31.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.345+0000 7f2c6a59c700 1 --2- 172.21.15.31:0/4207681349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c6c06aaa0 0x7f2c6c150600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:31.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.345+0000 7f2c69d9b700 1 -- 172.21.15.31:0/4207681349 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2c6c13fef0 msgr2=0x7f2c6c158880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:31.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.345+0000 7f2c69d9b700 1 --2- 172.21.15.31:0/4207681349 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2c6c13fef0 0x7f2c6c158880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:31.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.345+0000 7f2c69d9b700 1 -- 172.21.15.31:0/4207681349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c6c06aaa0 msgr2=0x7f2c6c150600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:31.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.345+0000 7f2c69d9b700 1 --2- 172.21.15.31:0/4207681349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c6c06aaa0 0x7f2c6c150600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:31.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.345+0000 7f2c69d9b700 1 -- 172.21.15.31:0/4207681349 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c60009480 con 0x7f2c6c13fb20 2023-10-06T02:31:31.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.345+0000 7f2c6a59c700 1 --2- 172.21.15.31:0/4207681349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c6c06aaa0 0x7f2c6c150600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:31:31.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.345+0000 7f2c6ad9d700 1 --2- 172.21.15.31:0/4207681349 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2c6c13fef0 0x7f2c6c158880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:31:31.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.345+0000 7f2c69d9b700 1 --2- 172.21.15.31:0/4207681349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c6c13fb20 0x7f2c6c137d20 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f2c60005d40 tx=0x7f2c60016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:31.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.345+0000 7f2c5a7fc700 1 -- 172.21.15.31:0/4207681349 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c60006b60 con 0x7f2c6c13fb20 2023-10-06T02:31:31.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.345+0000 7f2c5a7fc700 1 -- 172.21.15.31:0/4207681349 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c60011a60 con 0x7f2c6c13fb20 2023-10-06T02:31:31.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.345+0000 7f2c5a7fc700 1 -- 172.21.15.31:0/4207681349 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c60011de0 con 0x7f2c6c13fb20 2023-10-06T02:31:31.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.345+0000 7f2c6b59e700 1 -- 172.21.15.31:0/4207681349 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c6c158dc0 con 0x7f2c6c13fb20 2023-10-06T02:31:31.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.345+0000 7f2c6b59e700 1 -- 172.21.15.31:0/4207681349 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c6c14d640 con 0x7f2c6c13fb20 2023-10-06T02:31:31.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.346+0000 7f2c6b59e700 1 -- 172.21.15.31:0/4207681349 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c6c04c3c0 con 0x7f2c6c13fb20 2023-10-06T02:31:31.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.347+0000 7f2c5a7fc700 1 -- 172.21.15.31:0/4207681349 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 85403+0+0 (secure 0 0 0) 0x7f2c6001c620 con 0x7f2c6c13fb20 2023-10-06T02:31:31.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.349+0000 7f2c5a7fc700 1 --2- 172.21.15.31:0/4207681349 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f2c4c067680 0x7f2c4c069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:31.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.349+0000 7f2c5a7fc700 1 -- 172.21.15.31:0/4207681349 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f2c60087280 con 0x7f2c6c13fb20 2023-10-06T02:31:31.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.350+0000 7f2c6a59c700 1 --2- 172.21.15.31:0/4207681349 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f2c4c067680 0x7f2c4c069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:31.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.350+0000 7f2c6a59c700 1 --2- 172.21.15.31:0/4207681349 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f2c4c067680 0x7f2c4c069b30 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c006fd0 tx=0x7f2c5c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:31.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.353+0000 7f2c5a7fc700 1 -- 172.21.15.31:0/4207681349 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2c60084640 con 0x7f2c6c13fb20 2023-10-06T02:31:31.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.636+0000 7f2c6b59e700 1 -- 172.21.15.31:0/4207681349 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f2c6c0cc990 con 0x7f2c6c13fb20 2023-10-06T02:31:31.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.640+0000 7f2c5a7fc700 1 -- 172.21.15.31:0/4207681349 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v23) v1 ==== 190+0+0 (secure 0 0 0) 0x7f2c600534b0 con 0x7f2c6c13fb20 2023-10-06T02:31:31.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.642+0000 7f2c6b59e700 1 -- 172.21.15.31:0/4207681349 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f2c4c067680 msgr2=0x7f2c4c069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:31.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.642+0000 7f2c6b59e700 1 --2- 172.21.15.31:0/4207681349 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f2c4c067680 0x7f2c4c069b30 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c006fd0 tx=0x7f2c5c008040 comp rx=0 tx=0).stop 2023-10-06T02:31:31.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.642+0000 7f2c6b59e700 1 -- 172.21.15.31:0/4207681349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c6c13fb20 msgr2=0x7f2c6c137d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:31.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.643+0000 7f2c6b59e700 1 --2- 172.21.15.31:0/4207681349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c6c13fb20 0x7f2c6c137d20 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f2c60005d40 tx=0x7f2c60016000 comp rx=0 tx=0).stop 2023-10-06T02:31:31.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.643+0000 7f2c6b59e700 1 -- 172.21.15.31:0/4207681349 shutdown_connections 2023-10-06T02:31:31.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.643+0000 7f2c6b59e700 1 --2- 172.21.15.31:0/4207681349 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f2c4c067680 0x7f2c4c069b30 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:31.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.643+0000 7f2c6b59e700 1 --2- 172.21.15.31:0/4207681349 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2c6c13fef0 0x7f2c6c158880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:31.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.643+0000 7f2c6b59e700 1 --2- 172.21.15.31:0/4207681349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c6c06aaa0 0x7f2c6c150600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:31.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.643+0000 7f2c6b59e700 1 --2- 172.21.15.31:0/4207681349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c6c13fb20 0x7f2c6c137d20 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:31.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.643+0000 7f2c6b59e700 1 -- 172.21.15.31:0/4207681349 >> 172.21.15.31:0/4207681349 conn(0x7f2c6c06d170 msgr2=0x7f2c6c0c20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:31.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.643+0000 7f2c6b59e700 1 -- 172.21.15.31:0/4207681349 shutdown_connections 2023-10-06T02:31:31.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.643+0000 7f2c6b59e700 1 -- 172.21.15.31:0/4207681349 wait complete. 2023-10-06T02:31:31.646 INFO:teuthology.orchestra.run.smithi031.stderr:updated caps for client.0 2023-10-06T02:31:31.663 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:31:31.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.850+0000 7fdedf59e700 1 Processor -- start 2023-10-06T02:31:31.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.850+0000 7fdedf59e700 1 -- start start 2023-10-06T02:31:31.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.851+0000 7fdedf59e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdee0159a10 0x7fdee00ca7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:31.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.852+0000 7fdedf59e700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdee00c7420 con 0x7fdee0070410 2023-10-06T02:31:31.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.852+0000 7fdedf59e700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdee006d820 con 0x7fdee0159640 2023-10-06T02:31:31.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.852+0000 7fdedf59e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdee006db00 con 0x7fdee0159a10 2023-10-06T02:31:31.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.852+0000 7fdeded9d700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fdee0070410 0x7fdee00c78b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58680/0 (socket says 172.21.15.31:58680) 2023-10-06T02:31:31.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.852+0000 7fdeded9d700 1 -- 172.21.15.31:0/1987093614 learned_addr learned my addr 172.21.15.31:0/1987093614 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:31:31.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.852+0000 7fdeddd9b700 1 --2- 172.21.15.31:0/1987093614 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdee0159a10 0x7fdee00ca7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:31.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.852+0000 7fdedd59a700 1 -- 172.21.15.31:0/1987093614 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1839653251 0 0) 0x7fdee00c7420 con 0x7fdee0070410 2023-10-06T02:31:31.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.852+0000 7fdedd59a700 1 -- 172.21.15.31:0/1987093614 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdec8003570 con 0x7fdee0070410 2023-10-06T02:31:31.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.852+0000 7fdedd59a700 1 -- 172.21.15.31:0/1987093614 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1590122404 0 0) 0x7fdee006d820 con 0x7fdee0159640 2023-10-06T02:31:31.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.852+0000 7fdedd59a700 1 -- 172.21.15.31:0/1987093614 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdee00c7420 con 0x7fdee0159640 2023-10-06T02:31:31.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.852+0000 7fdedd59a700 1 -- 172.21.15.31:0/1987093614 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 290415361 0 0) 0x7fdec8003570 con 0x7fdee0070410 2023-10-06T02:31:31.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.853+0000 7fdedd59a700 1 -- 172.21.15.31:0/1987093614 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdee0159a10 msgr2=0x7fdee00ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:31.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.853+0000 7fdedd59a700 1 --2- 172.21.15.31:0/1987093614 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdee0159a10 0x7fdee00ca7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:31.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.853+0000 7fdedd59a700 1 -- 172.21.15.31:0/1987093614 >> v1:172.21.15.62:6789/0 conn(0x7fdee0159640 legacy=0x7fdee006d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:31.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.853+0000 7fdedd59a700 1 -- 172.21.15.31:0/1987093614 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdee006dde0 con 0x7fdee0070410 2023-10-06T02:31:31.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.853+0000 7fdedd59a700 1 -- 172.21.15.31:0/1987093614 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7fded80017f0 con 0x7fdee0070410 2023-10-06T02:31:31.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.853+0000 7fdedd59a700 1 -- 172.21.15.31:0/1987093614 >> v1:172.21.15.31:6789/0 conn(0x7fdee0070410 legacy=0x7fdee00c78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:31.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.853+0000 7fdedd59a700 1 --2- 172.21.15.31:0/1987093614 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdec8003c20 0x7fdec8004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:31.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.853+0000 7fdedd59a700 1 --2- 172.21.15.31:0/1987093614 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdec8004810 0x7fdec8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:31.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.853+0000 7fdedd59a700 1 --2- 172.21.15.31:0/1987093614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdec80051c0 0x7fdec800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:31.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.853+0000 7fdedd59a700 1 -- 172.21.15.31:0/1987093614 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee006db00 con 0x7fdec8004810 2023-10-06T02:31:31.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.853+0000 7fdedd59a700 1 -- 172.21.15.31:0/1987093614 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdec800c020 con 0x7fdec80051c0 2023-10-06T02:31:31.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.853+0000 7fdedd59a700 1 -- 172.21.15.31:0/1987093614 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdec800c1e0 con 0x7fdec8003c20 2023-10-06T02:31:31.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.853+0000 7fdeddd9b700 1 --2- 172.21.15.31:0/1987093614 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdee0159a10 0x7fdee00ca7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:31:31.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.853+0000 7fdeddd9b700 1 --2- 172.21.15.31:0/1987093614 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdec8004810 0x7fdec8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:31.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.853+0000 7fdeded9d700 1 --2- 172.21.15.31:0/1987093614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdec80051c0 0x7fdec800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:31.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.853+0000 7fdede59c700 1 --2- 172.21.15.31:0/1987093614 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdec8003c20 0x7fdec8004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:31.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.853+0000 7fdeddd9b700 1 -- 172.21.15.31:0/1987093614 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdec8003c20 msgr2=0x7fdec8004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:31.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.853+0000 7fdeddd9b700 1 --2- 172.21.15.31:0/1987093614 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdec8003c20 0x7fdec8004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:31.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.853+0000 7fdeddd9b700 1 -- 172.21.15.31:0/1987093614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdec80051c0 msgr2=0x7fdec800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:31.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.853+0000 7fdeddd9b700 1 --2- 172.21.15.31:0/1987093614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdec80051c0 0x7fdec800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:31.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.854+0000 7fdeddd9b700 1 -- 172.21.15.31:0/1987093614 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdec8003570 con 0x7fdec8004810 2023-10-06T02:31:31.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.854+0000 7fdeddd9b700 1 --2- 172.21.15.31:0/1987093614 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdec8004810 0x7fdec8004c80 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7fdee015d370 tx=0x7fded40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae79f00a551782b2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:31.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.854+0000 7fdedd59a700 1 -- 172.21.15.31:0/1987093614 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded400e710 con 0x7fdec8004810 2023-10-06T02:31:31.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.854+0000 7fdedd59a700 1 -- 172.21.15.31:0/1987093614 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fded400e870 con 0x7fdec8004810 2023-10-06T02:31:31.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.854+0000 7fdedd59a700 1 -- 172.21.15.31:0/1987093614 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded400ec10 con 0x7fdec8004810 2023-10-06T02:31:31.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.854+0000 7fdedf59e700 1 -- 172.21.15.31:0/1987093614 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdec8004810 msgr2=0x7fdec8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:31.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.854+0000 7fdedf59e700 1 --2- 172.21.15.31:0/1987093614 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdec8004810 0x7fdec8004c80 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7fdee015d370 tx=0x7fded40091a0 comp rx=0 tx=0).stop 2023-10-06T02:31:31.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.854+0000 7fdeded9d700 1 -- 172.21.15.31:0/1987093614 reap_dead start 2023-10-06T02:31:31.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.855+0000 7fdedf59e700 1 -- 172.21.15.31:0/1987093614 shutdown_connections 2023-10-06T02:31:31.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.855+0000 7fdedf59e700 1 -- 172.21.15.31:0/1987093614 >> 172.21.15.31:0/1987093614 conn(0x7fdee006e720 msgr2=0x7fdee006eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:31.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.855+0000 7fdedf59e700 1 -- 172.21.15.31:0/1987093614 shutdown_connections 2023-10-06T02:31:31.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.855+0000 7fdedf59e700 1 -- 172.21.15.31:0/1987093614 wait complete. 2023-10-06T02:31:31.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.856+0000 7fdedf59e700 1 Processor -- start 2023-10-06T02:31:31.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.856+0000 7fdedf59e700 1 -- start start 2023-10-06T02:31:31.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.857+0000 7fdedf59e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdec8004810 0x7fdee00b24b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:31.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.857+0000 7fdedf59e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdee00b03c0 0x7fdee00b0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:31.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.857+0000 7fdedf59e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdee00c89d0 0x7fdee00c3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:31.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.857+0000 7fdedf59e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee00b2020 con 0x7fdee00b03c0 2023-10-06T02:31:31.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.857+0000 7fdedf59e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee00b1410 con 0x7fdec8004810 2023-10-06T02:31:31.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.857+0000 7fdedf59e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdee00b16f0 con 0x7fdee00c89d0 2023-10-06T02:31:31.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.857+0000 7fdeddd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdee00b03c0 0x7fdee00b0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:31.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.857+0000 7fdeddd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdee00b03c0 0x7fdee00b0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51596/0 (socket says 172.21.15.31:51596) 2023-10-06T02:31:31.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.857+0000 7fdeddd9b700 1 -- 172.21.15.31:0/4041799948 learned_addr learned my addr 172.21.15.31:0/4041799948 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:31.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.857+0000 7fdeded9d700 1 --2- 172.21.15.31:0/4041799948 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdee00c89d0 0x7fdee00c3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:31.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.857+0000 7fdede59c700 1 --2- 172.21.15.31:0/4041799948 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdec8004810 0x7fdee00b24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:31.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.858+0000 7fdeddd9b700 1 -- 172.21.15.31:0/4041799948 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdee00c89d0 msgr2=0x7fdee00c3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:31.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.858+0000 7fdeddd9b700 1 --2- 172.21.15.31:0/4041799948 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdee00c89d0 0x7fdee00c3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:31.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.858+0000 7fdeddd9b700 1 -- 172.21.15.31:0/4041799948 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdec8004810 msgr2=0x7fdee00b24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:31.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.858+0000 7fdeddd9b700 1 --2- 172.21.15.31:0/4041799948 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdec8004810 0x7fdee00b24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:31.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.858+0000 7fdeddd9b700 1 -- 172.21.15.31:0/4041799948 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdee00b19d0 con 0x7fdee00b03c0 2023-10-06T02:31:31.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.858+0000 7fdeddd9b700 1 --2- 172.21.15.31:0/4041799948 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdee00b03c0 0x7fdee00b0be0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7fdec800bff0 tx=0x7fded4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:31.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.858+0000 7fdece7fc700 1 -- 172.21.15.31:0/4041799948 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded4022070 con 0x7fdee00b03c0 2023-10-06T02:31:31.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.858+0000 7fdece7fc700 1 -- 172.21.15.31:0/4041799948 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fded401c420 con 0x7fdee00b03c0 2023-10-06T02:31:31.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.858+0000 7fdedf59e700 1 -- 172.21.15.31:0/4041799948 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdee00b1cb0 con 0x7fdee00b03c0 2023-10-06T02:31:31.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.858+0000 7fdece7fc700 1 -- 172.21.15.31:0/4041799948 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded401c7e0 con 0x7fdee00b03c0 2023-10-06T02:31:31.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.858+0000 7fdedf59e700 1 -- 172.21.15.31:0/4041799948 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdee00c8e40 con 0x7fdee00b03c0 2023-10-06T02:31:31.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.859+0000 7fdedf59e700 1 -- 172.21.15.31:0/4041799948 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdee00c9120 con 0x7fdee00b03c0 2023-10-06T02:31:31.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.860+0000 7fdece7fc700 1 -- 172.21.15.31:0/4041799948 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 85403+0+0 (secure 0 0 0) 0x7fded401cb90 con 0x7fdee00b03c0 2023-10-06T02:31:31.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.861+0000 7fdece7fc700 1 --2- 172.21.15.31:0/4041799948 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fdec0067680 0x7fdec0069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:31.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.862+0000 7fdece7fc700 1 -- 172.21.15.31:0/4041799948 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fded40870f0 con 0x7fdee00b03c0 2023-10-06T02:31:31.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.862+0000 7fdede59c700 1 --2- 172.21.15.31:0/4041799948 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fdec0067680 0x7fdec0069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:31.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.862+0000 7fdede59c700 1 --2- 172.21.15.31:0/4041799948 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fdec0067680 0x7fdec0069b30 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fded0003a00 tx=0x7fded0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:31.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:31.865+0000 7fdece7fc700 1 -- 172.21.15.31:0/4041799948 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdee00c9120 con 0x7fdee00b03c0 2023-10-06T02:31:32.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.112+0000 7fdedf59e700 1 -- 172.21.15.31:0/4041799948 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdee00c9120 con 0x7fdee00b03c0 2023-10-06T02:31:32.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.113+0000 7fdece7fc700 1 -- 172.21.15.31:0/4041799948 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 109 v109) v1 ==== 95+0+4604 (secure 0 0 0) 0x7fdee00c9120 con 0x7fdee00b03c0 2023-10-06T02:31:32.114 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:31:32.115 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":109,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7178,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/3218480514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3218480514},{"type":"v1","addr":"172.21.15.62:6834","nonce":3218480514}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104},{"gid":8323,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6833/832192876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":832192876},{"type":"v1","addr":"172.21.15.31:6833","nonce":832192876}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104},{"gid":8877,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/2374603733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":2374603733},{"type":"v1","addr":"172.21.15.31:6835","nonce":2374603733}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104}],"filesystems":[{"mdsmap":{"epoch":109,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:31:25.755801+0000","modified":"2023-10-06T02:31:29.266902+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8889},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8889":{"gid":8889,"name":"d","rank":0,"incarnation":105,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/1184425321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":1184425321},{"type":"v1","addr":"172.21.15.62:6835","nonce":1184425321}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2023-10-06T02:31:32.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.115+0000 7fdedf59e700 1 -- 172.21.15.31:0/4041799948 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fdec0067680 msgr2=0x7fdec0069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:32.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.115+0000 7fdedf59e700 1 --2- 172.21.15.31:0/4041799948 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fdec0067680 0x7fdec0069b30 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fded0003a00 tx=0x7fded0009040 comp rx=0 tx=0).stop 2023-10-06T02:31:32.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.115+0000 7fdedf59e700 1 -- 172.21.15.31:0/4041799948 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdee00b03c0 msgr2=0x7fdee00b0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:32.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.115+0000 7fdedf59e700 1 --2- 172.21.15.31:0/4041799948 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdee00b03c0 0x7fdee00b0be0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7fdec800bff0 tx=0x7fded4019000 comp rx=0 tx=0).stop 2023-10-06T02:31:32.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.115+0000 7fdedf59e700 1 -- 172.21.15.31:0/4041799948 shutdown_connections 2023-10-06T02:31:32.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.115+0000 7fdedf59e700 1 --2- 172.21.15.31:0/4041799948 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fdec0067680 0x7fdec0069b30 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:32.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.115+0000 7fdedf59e700 1 --2- 172.21.15.31:0/4041799948 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdee00c89d0 0x7fdee00c3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:32.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.115+0000 7fdedf59e700 1 --2- 172.21.15.31:0/4041799948 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdec8004810 0x7fdee00b24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:32.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.115+0000 7fdedf59e700 1 --2- 172.21.15.31:0/4041799948 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdee00b03c0 0x7fdee00b0be0 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:32.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.115+0000 7fdedf59e700 1 -- 172.21.15.31:0/4041799948 >> 172.21.15.31:0/4041799948 conn(0x7fdee006e720 msgr2=0x7fdee0155990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:32.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.116+0000 7fdedf59e700 1 -- 172.21.15.31:0/4041799948 shutdown_connections 2023-10-06T02:31:32.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.116+0000 7fdedf59e700 1 -- 172.21.15.31:0/4041799948 wait complete. 2023-10-06T02:31:32.119 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 109 2023-10-06T02:31:32.300 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T02:31:32.305 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T02:31:32.540 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 109, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T02:31:25.755801+0000', 'modified': '2023-10-06T02:31:29.266902+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8889}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8889': {'gid': 8889, 'name': 'd', 'rank': 0, 'incarnation': 105, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.62:6835/1184425321', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6833', 'nonce': 1184425321}, {'type': 'v1', 'addr': '172.21.15.62:6835', 'nonce': 1184425321}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [26], 'metadata_pool': 25, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T02:31:32.540 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T02:31:32.540 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-10-06T02:31:32.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.727+0000 7efd37fff700 1 Processor -- start 2023-10-06T02:31:32.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.727+0000 7efd37fff700 1 -- start start 2023-10-06T02:31:32.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.728+0000 7efd37fff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7efd38153c30 0x7efd3815a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:32.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.728+0000 7efd37fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efd3815de50 0x7efd380cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:32.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.728+0000 7efd37fff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd3815afd0 con 0x7efd380cef40 2023-10-06T02:31:32.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.728+0000 7efd37fff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd3815b2b0 con 0x7efd38153c30 2023-10-06T02:31:32.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.728+0000 7efd37fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd3815b590 con 0x7efd3815de50 2023-10-06T02:31:32.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.729+0000 7efd377fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7efd380cef40 0x7efd3806d790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58684/0 (socket says 172.21.15.31:58684) 2023-10-06T02:31:32.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.729+0000 7efd377fe700 1 -- 172.21.15.31:0/1836889850 learned_addr learned my addr 172.21.15.31:0/1836889850 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:31:32.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.729+0000 7efd367fc700 1 --2- 172.21.15.31:0/1836889850 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efd3815de50 0x7efd380cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:32.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.729+0000 7efd36ffd700 1 --2- 172.21.15.31:0/1836889850 >> v2:172.21.15.62:3300/0 conn(0x7efd38153c30 0x7efd3815a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:32.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.729+0000 7efd35ffb700 1 -- 172.21.15.31:0/1836889850 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2157916907 0 0) 0x7efd3815afd0 con 0x7efd380cef40 2023-10-06T02:31:32.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.729+0000 7efd35ffb700 1 -- 172.21.15.31:0/1836889850 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd20003570 con 0x7efd380cef40 2023-10-06T02:31:32.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.729+0000 7efd35ffb700 1 -- 172.21.15.31:0/1836889850 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2056929591 0 0) 0x7efd20003570 con 0x7efd380cef40 2023-10-06T02:31:32.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.729+0000 7efd35ffb700 1 -- 172.21.15.31:0/1836889850 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efd3815de50 msgr2=0x7efd380cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:32.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.729+0000 7efd35ffb700 1 --2- 172.21.15.31:0/1836889850 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efd3815de50 0x7efd380cea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:32.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.729+0000 7efd35ffb700 1 -- 172.21.15.31:0/1836889850 >> v2:172.21.15.62:3300/0 conn(0x7efd38153c30 msgr2=0x7efd3815a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:32.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.729+0000 7efd35ffb700 1 --2- 172.21.15.31:0/1836889850 >> v2:172.21.15.62:3300/0 conn(0x7efd38153c30 0x7efd3815a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:32.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.729+0000 7efd35ffb700 1 -- 172.21.15.31:0/1836889850 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd3815b870 con 0x7efd380cef40 2023-10-06T02:31:32.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.729+0000 7efd35ffb700 1 -- 172.21.15.31:0/1836889850 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7efd300017f0 con 0x7efd380cef40 2023-10-06T02:31:32.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.729+0000 7efd35ffb700 1 -- 172.21.15.31:0/1836889850 >> v1:172.21.15.31:6789/0 conn(0x7efd380cef40 legacy=0x7efd3806d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:32.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.729+0000 7efd35ffb700 1 --2- 172.21.15.31:0/1836889850 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd20003a60 0x7efd20003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:32.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.729+0000 7efd35ffb700 1 --2- 172.21.15.31:0/1836889850 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd20004810 0x7efd20004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:32.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.730+0000 7efd35ffb700 1 --2- 172.21.15.31:0/1836889850 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efd3815de50 0x7efd200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:32.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.730+0000 7efd35ffb700 1 -- 172.21.15.31:0/1836889850 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd3815b2b0 con 0x7efd20004810 2023-10-06T02:31:32.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.730+0000 7efd35ffb700 1 -- 172.21.15.31:0/1836889850 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd3815b590 con 0x7efd20003a60 2023-10-06T02:31:32.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.730+0000 7efd35ffb700 1 -- 172.21.15.31:0/1836889850 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd2000ba60 con 0x7efd3815de50 2023-10-06T02:31:32.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.730+0000 7efd36ffd700 1 --2- 172.21.15.31:0/1836889850 >> v2:172.21.15.62:3300/0 conn(0x7efd38153c30 0x7efd3815a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:31:32.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.730+0000 7efd367fc700 1 --2- 172.21.15.31:0/1836889850 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd20004810 0x7efd20004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:32.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.730+0000 7efd377fe700 1 --2- 172.21.15.31:0/1836889850 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efd3815de50 0x7efd200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:32.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.730+0000 7efd36ffd700 1 --2- 172.21.15.31:0/1836889850 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd20003a60 0x7efd20003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:32.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.730+0000 7efd367fc700 1 -- 172.21.15.31:0/1836889850 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efd3815de50 msgr2=0x7efd200051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:32.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.730+0000 7efd367fc700 1 --2- 172.21.15.31:0/1836889850 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efd3815de50 0x7efd200051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:32.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.730+0000 7efd367fc700 1 -- 172.21.15.31:0/1836889850 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd20003a60 msgr2=0x7efd20003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:32.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.730+0000 7efd367fc700 1 --2- 172.21.15.31:0/1836889850 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd20003a60 0x7efd20003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:32.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.730+0000 7efd367fc700 1 -- 172.21.15.31:0/1836889850 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd20003570 con 0x7efd20004810 2023-10-06T02:31:32.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.730+0000 7efd377fe700 1 --2- 172.21.15.31:0/1836889850 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efd3815de50 0x7efd200051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:31:32.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.730+0000 7efd36ffd700 1 --2- 172.21.15.31:0/1836889850 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd20003a60 0x7efd20003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:31:32.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.731+0000 7efd367fc700 1 --2- 172.21.15.31:0/1836889850 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd20004810 0x7efd20004c80 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7efd2c009c30 tx=0x7efd2c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f26f95172d927aa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:32.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.731+0000 7efd35ffb700 1 -- 172.21.15.31:0/1836889850 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd2c00e710 con 0x7efd20004810 2023-10-06T02:31:32.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.731+0000 7efd35ffb700 1 -- 172.21.15.31:0/1836889850 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd2c00e870 con 0x7efd20004810 2023-10-06T02:31:32.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.731+0000 7efd35ffb700 1 -- 172.21.15.31:0/1836889850 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd2c00ec30 con 0x7efd20004810 2023-10-06T02:31:32.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.731+0000 7efd37fff700 1 -- 172.21.15.31:0/1836889850 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd20004810 msgr2=0x7efd20004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:32.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.731+0000 7efd37fff700 1 --2- 172.21.15.31:0/1836889850 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd20004810 0x7efd20004c80 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7efd2c009c30 tx=0x7efd2c0091a0 comp rx=0 tx=0).stop 2023-10-06T02:31:32.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.731+0000 7efd377fe700 1 -- 172.21.15.31:0/1836889850 reap_dead start 2023-10-06T02:31:32.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.731+0000 7efd37fff700 1 -- 172.21.15.31:0/1836889850 shutdown_connections 2023-10-06T02:31:32.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.731+0000 7efd37fff700 1 -- 172.21.15.31:0/1836889850 >> 172.21.15.31:0/1836889850 conn(0x7efd380703a0 msgr2=0x7efd380707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:32.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.731+0000 7efd37fff700 1 -- 172.21.15.31:0/1836889850 shutdown_connections 2023-10-06T02:31:32.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.731+0000 7efd37fff700 1 -- 172.21.15.31:0/1836889850 wait complete. 2023-10-06T02:31:32.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.733+0000 7efd37fff700 1 Processor -- start 2023-10-06T02:31:32.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.733+0000 7efd37fff700 1 -- start start 2023-10-06T02:31:32.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.734+0000 7efd37fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd20004810 0x7efd380c9330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:32.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.734+0000 7efd37fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efd380b4650 0x7efd381530f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:32.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.734+0000 7efd37fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd380b17f0 0x7efd380b1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:32.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.734+0000 7efd37fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd380b21d0 con 0x7efd20004810 2023-10-06T02:31:32.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.734+0000 7efd37fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd381699e0 con 0x7efd380b17f0 2023-10-06T02:31:32.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.734+0000 7efd37fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd380b4ac0 con 0x7efd380b4650 2023-10-06T02:31:32.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.734+0000 7efd36ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd20004810 0x7efd380c9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:32.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.734+0000 7efd36ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd20004810 0x7efd380c9330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51612/0 (socket says 172.21.15.31:51612) 2023-10-06T02:31:32.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.734+0000 7efd36ffd700 1 -- 172.21.15.31:0/408947316 learned_addr learned my addr 172.21.15.31:0/408947316 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:32.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.734+0000 7efd377fe700 1 --2- 172.21.15.31:0/408947316 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd380b17f0 0x7efd380b1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:32.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.734+0000 7efd367fc700 1 --2- 172.21.15.31:0/408947316 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efd380b4650 0x7efd381530f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:32.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.735+0000 7efd36ffd700 1 -- 172.21.15.31:0/408947316 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efd380b4650 msgr2=0x7efd381530f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:32.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.735+0000 7efd36ffd700 1 --2- 172.21.15.31:0/408947316 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efd380b4650 0x7efd381530f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:32.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.735+0000 7efd36ffd700 1 -- 172.21.15.31:0/408947316 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd380b17f0 msgr2=0x7efd380b1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:32.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.735+0000 7efd36ffd700 1 --2- 172.21.15.31:0/408947316 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd380b17f0 0x7efd380b1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:32.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.735+0000 7efd36ffd700 1 -- 172.21.15.31:0/408947316 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd380c9960 con 0x7efd20004810 2023-10-06T02:31:32.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.735+0000 7efd367fc700 1 --2- 172.21.15.31:0/408947316 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efd380b4650 0x7efd381530f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:31:32.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.735+0000 7efd36ffd700 1 --2- 172.21.15.31:0/408947316 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd20004810 0x7efd380c9330 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7efd28009fd0 tx=0x7efd28008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:32.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.735+0000 7efd1effd700 1 -- 172.21.15.31:0/408947316 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd2800f880 con 0x7efd20004810 2023-10-06T02:31:32.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.735+0000 7efd1effd700 1 -- 172.21.15.31:0/408947316 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd2800f9e0 con 0x7efd20004810 2023-10-06T02:31:32.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.735+0000 7efd1effd700 1 -- 172.21.15.31:0/408947316 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd2800fd60 con 0x7efd20004810 2023-10-06T02:31:32.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.735+0000 7efd37fff700 1 -- 172.21.15.31:0/408947316 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd380c9c40 con 0x7efd20004810 2023-10-06T02:31:32.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.735+0000 7efd37fff700 1 -- 172.21.15.31:0/408947316 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd380ca200 con 0x7efd20004810 2023-10-06T02:31:32.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.738+0000 7efd1effd700 1 -- 172.21.15.31:0/408947316 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 85403+0+0 (secure 0 0 0) 0x7efd28015670 con 0x7efd20004810 2023-10-06T02:31:32.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.739+0000 7efd1effd700 1 --2- 172.21.15.31:0/408947316 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7efd24067680 0x7efd24069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:32.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.739+0000 7efd1effd700 1 -- 172.21.15.31:0/408947316 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 6086+0+0 (secure 0 0 0) 0x7efd28085df0 con 0x7efd20004810 2023-10-06T02:31:32.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.740+0000 7efd367fc700 1 --2- 172.21.15.31:0/408947316 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7efd24067680 0x7efd24069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:32.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.740+0000 7efd367fc700 1 --2- 172.21.15.31:0/408947316 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7efd24067680 0x7efd24069b30 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7efd380aff10 tx=0x7efd2c01a000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:32.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.741+0000 7efd4bc0db80 1 Processor -- start 2023-10-06T02:31:32.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.741+0000 7efd4bc0db80 1 -- start start 2023-10-06T02:31:32.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.742+0000 7efd4bc0db80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559518b9ce50 0x559518b77cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:32.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.742+0000 7efd4bc0db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559518b7d2c0 0x559518b78200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:32.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.742+0000 7efd4bc0db80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559518b7ea10 0x559518b78740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:32.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.742+0000 7efd4bc0db80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x559518ad2a70 con 0x559518b7ea10 2023-10-06T02:31:32.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.742+0000 7efd4bc0db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x559518ad0270 con 0x559518b7d2c0 2023-10-06T02:31:32.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.742+0000 7efd4bc0db80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x559518acc060 con 0x559518b9ce50 2023-10-06T02:31:32.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.742+0000 7efd377fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559518b7ea10 0x559518b78740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:32.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.742+0000 7efd377fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559518b7ea10 0x559518b78740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51622/0 (socket says 172.21.15.31:51622) 2023-10-06T02:31:32.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.742+0000 7efd377fe700 1 -- 172.21.15.31:0/1071609350 learned_addr learned my addr 172.21.15.31:0/1071609350 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:32.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.742+0000 7efd367fc700 1 --2- 172.21.15.31:0/1071609350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559518b7d2c0 0x559518b78200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:32.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.742+0000 7efd36ffd700 1 --2- 172.21.15.31:0/1071609350 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559518b9ce50 0x559518b77cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:32.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.743+0000 7efd377fe700 1 -- 172.21.15.31:0/1071609350 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559518b9ce50 msgr2=0x559518b77cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:32.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.743+0000 7efd377fe700 1 --2- 172.21.15.31:0/1071609350 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559518b9ce50 0x559518b77cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:32.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.743+0000 7efd377fe700 1 -- 172.21.15.31:0/1071609350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559518b7d2c0 msgr2=0x559518b78200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:32.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.743+0000 7efd377fe700 1 --2- 172.21.15.31:0/1071609350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559518b7d2c0 0x559518b78200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:32.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.743+0000 7efd377fe700 1 -- 172.21.15.31:0/1071609350 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559518966270 con 0x559518b7ea10 2023-10-06T02:31:32.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.743+0000 7efd377fe700 1 --2- 172.21.15.31:0/1071609350 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559518b7ea10 0x559518b78740 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7efd3815b6d0 tx=0x7efd30001540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5de099a1118b7414 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:32.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.743+0000 7efd17fff700 1 -- 172.21.15.31:0/1071609350 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd30004e90 con 0x559518b7ea10 2023-10-06T02:31:32.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.743+0000 7efd17fff700 1 -- 172.21.15.31:0/1071609350 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd30007820 con 0x559518b7ea10 2023-10-06T02:31:32.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.743+0000 7efd17fff700 1 -- 172.21.15.31:0/1071609350 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd300143d0 con 0x559518b7ea10 2023-10-06T02:31:32.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.744+0000 7efd4bc0db80 1 -- 172.21.15.31:0/1071609350 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559518b7ea10 msgr2=0x559518b78740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:32.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.744+0000 7efd4bc0db80 1 --2- 172.21.15.31:0/1071609350 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559518b7ea10 0x559518b78740 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7efd3815b6d0 tx=0x7efd30001540 comp rx=0 tx=0).stop 2023-10-06T02:31:32.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.744+0000 7efd4bc0db80 1 -- 172.21.15.31:0/1071609350 shutdown_connections 2023-10-06T02:31:32.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.744+0000 7efd4bc0db80 1 --2- 172.21.15.31:0/1071609350 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559518b7ea10 0x559518b78740 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:32.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.744+0000 7efd4bc0db80 1 --2- 172.21.15.31:0/1071609350 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559518b9ce50 0x559518b77cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:32.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.744+0000 7efd4bc0db80 1 --2- 172.21.15.31:0/1071609350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559518b7d2c0 0x559518b78200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:32.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.744+0000 7efd4bc0db80 1 -- 172.21.15.31:0/1071609350 >> 172.21.15.31:0/1071609350 conn(0x559518b7ce80 msgr2=0x559518b7d6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:32.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.744+0000 7efd4bc0db80 1 -- 172.21.15.31:0/1071609350 shutdown_connections 2023-10-06T02:31:32.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.744+0000 7efd4bc0db80 1 -- 172.21.15.31:0/1071609350 wait complete. 2023-10-06T02:31:32.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.745+0000 7efd4bc0db80 1 -- start start 2023-10-06T02:31:32.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.745+0000 7efd4bc0db80 1 Processor -- start 2023-10-06T02:31:32.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.746+0000 7efd4bc0db80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559518b7d2c0 0x559518b9a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:32.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.746+0000 7efd4bc0db80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559518b7ea10 0x559518b9d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:32.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.746+0000 7efd4bc0db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559518b9ce50 0x559518baa520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:32.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.746+0000 7efd4bc0db80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x559518a77890 con 0x559518b7ea10 2023-10-06T02:31:32.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.746+0000 7efd4bc0db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x559518a92ea0 con 0x559518b9ce50 2023-10-06T02:31:32.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.746+0000 7efd4bc0db80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x55951894ee30 con 0x559518b7d2c0 2023-10-06T02:31:32.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.746+0000 7efd4bc0db80 10 client.0 _ll_register_callbacks cb 0x559518b3f020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T02:31:32.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.746+0000 7efd367fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559518b7ea10 0x559518b9d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:32.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.746+0000 7efd367fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559518b7ea10 0x559518b9d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51630/0 (socket says 172.21.15.31:51630) 2023-10-06T02:31:32.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.746+0000 7efd367fc700 1 -- 172.21.15.31:0/2949595056 learned_addr learned my addr 172.21.15.31:0/2949595056 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:32.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.746+0000 7efd36ffd700 1 --2- 172.21.15.31:0/2949595056 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559518b7d2c0 0x559518b9a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:32.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.746+0000 7efd377fe700 1 --2- 172.21.15.31:0/2949595056 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559518b9ce50 0x559518baa520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:32.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.747+0000 7efd367fc700 1 -- 172.21.15.31:0/2949595056 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559518b7d2c0 msgr2=0x559518b9a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:32.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.747+0000 7efd367fc700 1 --2- 172.21.15.31:0/2949595056 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559518b7d2c0 0x559518b9a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:32.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.747+0000 7efd367fc700 1 -- 172.21.15.31:0/2949595056 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559518b9ce50 msgr2=0x559518baa520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:32.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.747+0000 7efd367fc700 1 --2- 172.21.15.31:0/2949595056 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559518b9ce50 0x559518baa520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:32.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.747+0000 7efd367fc700 1 -- 172.21.15.31:0/2949595056 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd3000f040 con 0x559518b7ea10 2023-10-06T02:31:32.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.747+0000 7efd367fc700 1 --2- 172.21.15.31:0/2949595056 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559518b7ea10 0x559518b9d2d0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7efd20004700 tx=0x7efd2c020050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e446c6c63449637 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:32.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.747+0000 7efd157fa700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T02:31:32.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.747+0000 7efd157fa700 1 -- 172.21.15.31:0/2949595056 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd2c013960 con 0x559518b7ea10 2023-10-06T02:31:32.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.747+0000 7efd157fa700 1 -- 172.21.15.31:0/2949595056 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(156..156 src has 1..156) v4 ==== 6086+0+0 (secure 0 0 0) 0x7efd2c013ac0 con 0x559518b7ea10 2023-10-06T02:31:32.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.747+0000 7efd4bc0db80 1 -- 172.21.15.31:0/2949595056 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559518b70940 con 0x559518b7ea10 2023-10-06T02:31:32.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.747+0000 7efd157fa700 1 -- 172.21.15.31:0/2949595056 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) v2 ==== 24+0+0 (secure 0 0 0) 0x7efd2c005a00 con 0x559518b7ea10 2023-10-06T02:31:32.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.747+0000 7efd4bc0db80 10 client.9036 fetch_fsmap learned FSMap version 109 2023-10-06T02:31:32.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.747+0000 7efd4bc0db80 1 -- 172.21.15.31:0/2949595056 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) v3 -- 0x559518908fe0 con 0x559518b7ea10 2023-10-06T02:31:32.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.748+0000 7efd157fa700 1 -- 172.21.15.31:0/2949595056 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559518b70940 con 0x559518b7ea10 2023-10-06T02:31:32.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.748+0000 7efd157fa700 1 -- 172.21.15.31:0/2949595056 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 109) v1 ==== 3100+0+0 (secure 0 0 0) 0x7efd2c005b60 con 0x559518b7ea10 2023-10-06T02:31:32.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.748+0000 7efd4bc0db80 10 client.9036 fetch_fsmap finished waiting for FSMap version 109 2023-10-06T02:31:32.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.748+0000 7efd4bc0db80 10 client.9036 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-06T02:31:32.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.748+0000 7efd4bc0db80 20 client.9036 populate_metadata read hostname 'smithi031' 2023-10-06T02:31:32.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.748+0000 7efd4bc0db80 1 --2- 172.21.15.31:0/2949595056 >> [v2:172.21.15.62:6833/1184425321,v1:172.21.15.62:6835/1184425321] conn(0x559518bc2e20 0x559518bc31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:32.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.748+0000 7efd4bc0db80 4 client.9036 mds_command: new command op to 8889 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T02:31:32.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.748+0000 7efd4bc0db80 1 -- 172.21.15.31:0/2949595056 --> [v2:172.21.15.62:6833/1184425321,v1:172.21.15.62:6835/1184425321] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559518b4d870 con 0x559518bc2e20 2023-10-06T02:31:32.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.748+0000 7efd36ffd700 1 --2- 172.21.15.31:0/2949595056 >> [v2:172.21.15.62:6833/1184425321,v1:172.21.15.62:6835/1184425321] conn(0x559518bc2e20 0x559518bc31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:32.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.749+0000 7efd36ffd700 1 --2- 172.21.15.31:0/2949595056 >> [v2:172.21.15.62:6833/1184425321,v1:172.21.15.62:6835/1184425321] conn(0x559518bc2e20 0x559518bc31f0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f91d517603459fb server_cookie=53af7a42fc4c86c in_seq=0 out_seq=0 2023-10-06T02:31:32.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.749+0000 7efd157fa700 10 client.9036 ms_handle_connect on v2:172.21.15.62:6833/1184425321 2023-10-06T02:31:32.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.750+0000 7efd157fa700 1 -- 172.21.15.31:0/2949595056 <== mds.0 v2:172.21.15.62:6833/1184425321 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x559518b4d870 con 0x559518bc2e20 2023-10-06T02:31:32.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.750+0000 7efd157fa700 10 client.9036 handle_command_reply: tid=0 2023-10-06T02:31:32.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.750+0000 7efd4bc0db80 1 client.9036 shutdown 2023-10-06T02:31:32.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.750+0000 7efd4bc0db80 1 -- 172.21.15.31:0/2949595056 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559518b7ea10 msgr2=0x559518b9d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:32.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.750+0000 7efd4bc0db80 1 --2- 172.21.15.31:0/2949595056 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559518b7ea10 0x559518b9d2d0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7efd20004700 tx=0x7efd2c020050 comp rx=0 tx=0).stop 2023-10-06T02:31:32.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.750+0000 7efd4bc0db80 1 -- 172.21.15.31:0/2949595056 shutdown_connections 2023-10-06T02:31:32.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.750+0000 7efd4bc0db80 1 --2- 172.21.15.31:0/2949595056 >> [v2:172.21.15.62:6833/1184425321,v1:172.21.15.62:6835/1184425321] conn(0x559518bc2e20 0x559518bc31f0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:32.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.750+0000 7efd4bc0db80 1 --2- 172.21.15.31:0/2949595056 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559518b9ce50 0x559518baa520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:32.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.750+0000 7efd4bc0db80 1 --2- 172.21.15.31:0/2949595056 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559518b7d2c0 0x559518b9a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:32.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.750+0000 7efd4bc0db80 1 --2- 172.21.15.31:0/2949595056 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559518b7ea10 0x559518b9d2d0 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:32.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.750+0000 7efd157fa700 0 client.9036 ms_handle_reset on v2:172.21.15.62:6833/1184425321 2023-10-06T02:31:32.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.750+0000 7efd4bc0db80 1 -- 172.21.15.31:0/2949595056 >> 172.21.15.31:0/2949595056 conn(0x559518b7ce80 msgr2=0x559518b7bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:32.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.750+0000 7efd4bc0db80 1 -- 172.21.15.31:0/2949595056 shutdown_connections 2023-10-06T02:31:32.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.750+0000 7efd4bc0db80 1 -- 172.21.15.31:0/2949595056 wait complete. 2023-10-06T02:31:32.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.750+0000 7efd4bc0db80 20 client.9036 trim_cache size 0 max 16384 2023-10-06T02:31:32.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.768+0000 7efd4bc0db80 1 Processor -- start 2023-10-06T02:31:32.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.768+0000 7efd4bc0db80 1 -- start start 2023-10-06T02:31:32.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.769+0000 7efd4bc0db80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559518b7d2c0 0x559518bc3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:32.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.769+0000 7efd4bc0db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559518b7ea10 0x559518b7d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:32.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.769+0000 7efd4bc0db80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559518b9ce50 0x559518bc4950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:32.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.769+0000 7efd4bc0db80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x559518b99e70 con 0x559518b7d2c0 2023-10-06T02:31:32.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.769+0000 7efd4bc0db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x559518babf50 con 0x559518b7ea10 2023-10-06T02:31:32.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.769+0000 7efd4bc0db80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x559518ba5500 con 0x559518b9ce50 2023-10-06T02:31:32.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.769+0000 7efd36ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559518b7d2c0 0x559518bc3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:32.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.769+0000 7efd36ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559518b7d2c0 0x559518bc3bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51634/0 (socket says 172.21.15.31:51634) 2023-10-06T02:31:32.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.769+0000 7efd36ffd700 1 -- 172.21.15.31:0/2935483687 learned_addr learned my addr 172.21.15.31:0/2935483687 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:32.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.769+0000 7efd367fc700 1 --2- 172.21.15.31:0/2935483687 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559518b7ea10 0x559518b7d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:32.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.769+0000 7efd36ffd700 1 -- 172.21.15.31:0/2935483687 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559518b9ce50 msgr2=0x559518bc4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:32.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.769+0000 7efd377fe700 1 --2- 172.21.15.31:0/2935483687 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559518b9ce50 0x559518bc4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:32.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.769+0000 7efd36ffd700 1 --2- 172.21.15.31:0/2935483687 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559518b9ce50 0x559518bc4950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:32.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.769+0000 7efd36ffd700 1 -- 172.21.15.31:0/2935483687 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559518b7ea10 msgr2=0x559518b7d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:32.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.769+0000 7efd36ffd700 1 --2- 172.21.15.31:0/2935483687 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559518b7ea10 0x559518b7d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:32.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.769+0000 7efd36ffd700 1 -- 172.21.15.31:0/2935483687 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559518a9a780 con 0x559518b7d2c0 2023-10-06T02:31:32.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.769+0000 7efd367fc700 1 --2- 172.21.15.31:0/2935483687 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559518b7ea10 0x559518b7d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:31:32.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.770+0000 7efd36ffd700 1 --2- 172.21.15.31:0/2935483687 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559518b7d2c0 0x559518bc3bf0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7efd28052570 tx=0x7efd2805b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=487080740241ce6f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:32.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.770+0000 7efd1cff9700 1 -- 172.21.15.31:0/2935483687 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd28056070 con 0x559518b7d2c0 2023-10-06T02:31:32.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.770+0000 7efd1cff9700 1 -- 172.21.15.31:0/2935483687 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd280592c0 con 0x559518b7d2c0 2023-10-06T02:31:32.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.770+0000 7efd1cff9700 1 -- 172.21.15.31:0/2935483687 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd2805d010 con 0x559518b7d2c0 2023-10-06T02:31:32.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.770+0000 7efd4bc0db80 1 -- 172.21.15.31:0/2935483687 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559518b7d2c0 msgr2=0x559518bc3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:32.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.770+0000 7efd4bc0db80 1 --2- 172.21.15.31:0/2935483687 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559518b7d2c0 0x559518bc3bf0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7efd28052570 tx=0x7efd2805b040 comp rx=0 tx=0).stop 2023-10-06T02:31:32.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.770+0000 7efd4bc0db80 1 -- 172.21.15.31:0/2935483687 shutdown_connections 2023-10-06T02:31:32.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.770+0000 7efd4bc0db80 1 --2- 172.21.15.31:0/2935483687 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559518b9ce50 0x559518bc4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:32.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.770+0000 7efd4bc0db80 1 --2- 172.21.15.31:0/2935483687 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559518b7d2c0 0x559518bc3bf0 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:32.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.771+0000 7efd4bc0db80 1 --2- 172.21.15.31:0/2935483687 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559518b7ea10 0x559518b7d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:32.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.771+0000 7efd4bc0db80 1 -- 172.21.15.31:0/2935483687 >> 172.21.15.31:0/2935483687 conn(0x559518b7ce80 msgr2=0x559518ba4df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:32.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.771+0000 7efd4bc0db80 1 -- 172.21.15.31:0/2935483687 shutdown_connections 2023-10-06T02:31:32.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.771+0000 7efd4bc0db80 1 -- 172.21.15.31:0/2935483687 wait complete. 2023-10-06T02:31:32.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.771+0000 7efd4bc0db80 1 -- start start 2023-10-06T02:31:32.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.771+0000 7efd4bc0db80 1 Processor -- start 2023-10-06T02:31:32.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.772+0000 7efd4bc0db80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559518b7d2c0 0x559518c2a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:32.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.772+0000 7efd4bc0db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559518b7ea10 0x559518b9ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:32.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.772+0000 7efd4bc0db80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559518b9ce50 0x559518b9b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:32.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.772+0000 7efd4bc0db80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x559518c28ce0 con 0x559518b9ce50 2023-10-06T02:31:32.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.772+0000 7efd4bc0db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x559518c278d0 con 0x559518b7ea10 2023-10-06T02:31:32.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.772+0000 7efd4bc0db80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x559518c27b50 con 0x559518b7d2c0 2023-10-06T02:31:32.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.772+0000 7efd4bc0db80 10 client.0 _ll_register_callbacks cb 0x559518b3f020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T02:31:32.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.773+0000 7efd377fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559518b9ce50 0x559518b9b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:32.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.773+0000 7efd36ffd700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559518b7d2c0 0x559518c2a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:32.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.773+0000 7efd377fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559518b9ce50 0x559518b9b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51638/0 (socket says 172.21.15.31:51638) 2023-10-06T02:31:32.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.773+0000 7efd377fe700 1 -- 172.21.15.31:0/1722798418 learned_addr learned my addr 172.21.15.31:0/1722798418 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:32.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.773+0000 7efd367fc700 1 --2- 172.21.15.31:0/1722798418 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559518b7ea10 0x559518b9ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:32.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.773+0000 7efd377fe700 1 -- 172.21.15.31:0/1722798418 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559518b7d2c0 msgr2=0x559518c2a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:32.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.773+0000 7efd377fe700 1 --2- 172.21.15.31:0/1722798418 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559518b7d2c0 0x559518c2a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:32.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.773+0000 7efd377fe700 1 -- 172.21.15.31:0/1722798418 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559518b7ea10 msgr2=0x559518b9ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:32.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.773+0000 7efd377fe700 1 --2- 172.21.15.31:0/1722798418 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559518b7ea10 0x559518b9ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:32.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.773+0000 7efd377fe700 1 -- 172.21.15.31:0/1722798418 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd380ca200 con 0x559518b9ce50 2023-10-06T02:31:32.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.773+0000 7efd367fc700 1 --2- 172.21.15.31:0/1722798418 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559518b7ea10 0x559518b9ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:31:32.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.773+0000 7efd377fe700 1 --2- 172.21.15.31:0/1722798418 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559518b9ce50 0x559518b9b2b0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7efd30001580 tx=0x7efd30019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a63b921508fbb0c6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:32.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.773+0000 7efd157fa700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T02:31:32.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.774+0000 7efd157fa700 1 -- 172.21.15.31:0/1722798418 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd300143d0 con 0x559518b9ce50 2023-10-06T02:31:32.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.774+0000 7efd157fa700 1 -- 172.21.15.31:0/1722798418 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(156..156 src has 1..156) v4 ==== 6086+0+0 (secure 0 0 0) 0x7efd300149d0 con 0x559518b9ce50 2023-10-06T02:31:32.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.774+0000 7efd4bc0db80 1 -- 172.21.15.31:0/1722798418 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efd30007820 con 0x559518b9ce50 2023-10-06T02:31:32.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.774+0000 7efd157fa700 1 -- 172.21.15.31:0/1722798418 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) v2 ==== 24+0+0 (secure 0 0 0) 0x7efd30014da0 con 0x559518b9ce50 2023-10-06T02:31:32.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.774+0000 7efd4bc0db80 10 client.9042 fetch_fsmap learned FSMap version 109 2023-10-06T02:31:32.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.774+0000 7efd4bc0db80 1 -- 172.21.15.31:0/1722798418 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) v3 -- 0x559518a81470 con 0x559518b9ce50 2023-10-06T02:31:32.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.774+0000 7efd157fa700 1 -- 172.21.15.31:0/1722798418 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd30032070 con 0x559518b9ce50 2023-10-06T02:31:32.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.774+0000 7efd157fa700 1 -- 172.21.15.31:0/1722798418 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 109) v1 ==== 3100+0+0 (secure 0 0 0) 0x7efd30007050 con 0x559518b9ce50 2023-10-06T02:31:32.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.774+0000 7efd4bc0db80 10 client.9042 fetch_fsmap finished waiting for FSMap version 109 2023-10-06T02:31:32.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.774+0000 7efd4bc0db80 10 client.9042 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-06T02:31:32.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.774+0000 7efd4bc0db80 20 client.9042 populate_metadata read hostname 'smithi031' 2023-10-06T02:31:32.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.774+0000 7efd4bc0db80 1 --2- 172.21.15.31:0/1722798418 >> [v2:172.21.15.62:6833/1184425321,v1:172.21.15.62:6835/1184425321] conn(0x559518bd0220 0x559518bd05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:32.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.774+0000 7efd4bc0db80 4 client.9042 mds_command: new command op to 8889 tid=0[{"prefix": "status"}] 2023-10-06T02:31:32.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.774+0000 7efd4bc0db80 1 -- 172.21.15.31:0/1722798418 --> [v2:172.21.15.62:6833/1184425321,v1:172.21.15.62:6835/1184425321] -- command(tid 0: {"prefix": "status"}) v1 -- 0x559518a6ee00 con 0x559518bd0220 2023-10-06T02:31:32.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.775+0000 7efd36ffd700 1 --2- 172.21.15.31:0/1722798418 >> [v2:172.21.15.62:6833/1184425321,v1:172.21.15.62:6835/1184425321] conn(0x559518bd0220 0x559518bd05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:32.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.775+0000 7efd36ffd700 1 --2- 172.21.15.31:0/1722798418 >> [v2:172.21.15.62:6833/1184425321,v1:172.21.15.62:6835/1184425321] conn(0x559518bd0220 0x559518bd05f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52ded04e7901bdcb server_cookie=47e9ab87dc533c07 in_seq=0 out_seq=0 2023-10-06T02:31:32.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.775+0000 7efd157fa700 10 client.9042 ms_handle_connect on v2:172.21.15.62:6833/1184425321 2023-10-06T02:31:32.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.776+0000 7efd157fa700 1 -- 172.21.15.31:0/1722798418 <== mds.0 v2:172.21.15.62:6833/1184425321 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x559518a6ee00 con 0x559518bd0220 2023-10-06T02:31:32.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.776+0000 7efd157fa700 10 client.9042 handle_command_reply: tid=0 2023-10-06T02:31:32.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.776+0000 7efd4bc0db80 1 client.9042 shutdown 2023-10-06T02:31:32.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.776+0000 7efd4bc0db80 1 -- 172.21.15.31:0/1722798418 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559518b9ce50 msgr2=0x559518b9b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:32.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.776+0000 7efd4bc0db80 1 --2- 172.21.15.31:0/1722798418 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559518b9ce50 0x559518b9b2b0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7efd30001580 tx=0x7efd30019000 comp rx=0 tx=0).stop 2023-10-06T02:31:32.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.776+0000 7efd4bc0db80 1 -- 172.21.15.31:0/1722798418 shutdown_connections 2023-10-06T02:31:32.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.776+0000 7efd4bc0db80 1 --2- 172.21.15.31:0/1722798418 >> [v2:172.21.15.62:6833/1184425321,v1:172.21.15.62:6835/1184425321] conn(0x559518bd0220 0x559518bd05f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:32.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.776+0000 7efd4bc0db80 1 --2- 172.21.15.31:0/1722798418 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559518b9ce50 0x559518b9b2b0 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:32.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.776+0000 7efd4bc0db80 1 --2- 172.21.15.31:0/1722798418 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559518b7d2c0 0x559518c2a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:32.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.776+0000 7efd157fa700 0 client.9042 ms_handle_reset on v2:172.21.15.62:6833/1184425321 2023-10-06T02:31:32.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.776+0000 7efd4bc0db80 1 --2- 172.21.15.31:0/1722798418 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559518b7ea10 0x559518b9ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:32.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.776+0000 7efd4bc0db80 1 -- 172.21.15.31:0/1722798418 >> 172.21.15.31:0/1722798418 conn(0x559518b7ce80 msgr2=0x559518ba4df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:32.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.776+0000 7efd4bc0db80 1 -- 172.21.15.31:0/1722798418 shutdown_connections 2023-10-06T02:31:32.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.776+0000 7efd4bc0db80 1 -- 172.21.15.31:0/1722798418 wait complete. 2023-10-06T02:31:32.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.776+0000 7efd4bc0db80 20 client.9042 trim_cache size 0 max 16384 2023-10-06T02:31:32.783 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T02:31:32.783 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "d6fcf512-874b-4259-b90a-30ae070f00a9", 2023-10-06T02:31:32.783 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 0, 2023-10-06T02:31:32.783 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 8889, 2023-10-06T02:31:32.783 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2023-10-06T02:31:32.783 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2023-10-06T02:31:32.783 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2023-10-06T02:31:32.784 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 6.5252896419999997, 2023-10-06T02:31:32.784 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 109, 2023-10-06T02:31:32.784 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 156, 2023-10-06T02:31:32.784 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 156, 2023-10-06T02:31:32.784 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 13.253049280999999 2023-10-06T02:31:32.784 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T02:31:32.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.777+0000 7efd37fff700 1 -- 172.21.15.31:0/408947316 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7efd24067680 msgr2=0x7efd24069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:32.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.777+0000 7efd37fff700 1 --2- 172.21.15.31:0/408947316 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7efd24067680 0x7efd24069b30 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7efd380aff10 tx=0x7efd2c01a000 comp rx=0 tx=0).stop 2023-10-06T02:31:32.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.777+0000 7efd37fff700 1 -- 172.21.15.31:0/408947316 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd20004810 msgr2=0x7efd380c9330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:32.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.777+0000 7efd37fff700 1 --2- 172.21.15.31:0/408947316 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd20004810 0x7efd380c9330 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7efd28009fd0 tx=0x7efd28008040 comp rx=0 tx=0).stop 2023-10-06T02:31:32.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.777+0000 7efd37fff700 1 -- 172.21.15.31:0/408947316 shutdown_connections 2023-10-06T02:31:32.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.777+0000 7efd37fff700 1 --2- 172.21.15.31:0/408947316 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd380b17f0 0x7efd380b1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:32.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.777+0000 7efd37fff700 1 --2- 172.21.15.31:0/408947316 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd20004810 0x7efd380c9330 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:32.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.777+0000 7efd37fff700 1 --2- 172.21.15.31:0/408947316 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7efd24067680 0x7efd24069b30 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:32.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.777+0000 7efd37fff700 1 --2- 172.21.15.31:0/408947316 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efd380b4650 0x7efd381530f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:32.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.777+0000 7efd37fff700 1 -- 172.21.15.31:0/408947316 >> 172.21.15.31:0/408947316 conn(0x7efd380703a0 msgr2=0x7efd380c64f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:32.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.777+0000 7efd37fff700 1 -- 172.21.15.31:0/408947316 shutdown_connections 2023-10-06T02:31:32.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.777+0000 7efd37fff700 1 -- 172.21.15.31:0/408947316 wait complete. 2023-10-06T02:31:32.795 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T02:31:32.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.982+0000 7f22f4f9d700 1 Processor -- start 2023-10-06T02:31:32.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.982+0000 7f22f4f9d700 1 -- start start 2023-10-06T02:31:32.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.983+0000 7f22f4f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f22f013fb20 0x7f22f015a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:32.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.983+0000 7f22f4f9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f22f006aaa0 0x7f22f00c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:32.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.983+0000 7f22f4f9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22f015ae70 con 0x7f22f013fef0 2023-10-06T02:31:32.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.983+0000 7f22f4f9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22f006af60 con 0x7f22f006aaa0 2023-10-06T02:31:32.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.983+0000 7f22f4f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22f006b240 con 0x7f22f013fb20 2023-10-06T02:31:32.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.984+0000 7f22eeffd700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f22f013fef0 0x7f22f00c7990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58686/0 (socket says 172.21.15.31:58686) 2023-10-06T02:31:32.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.984+0000 7f22eeffd700 1 -- 172.21.15.31:0/1492162341 learned_addr learned my addr 172.21.15.31:0/1492162341 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:31:32.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.984+0000 7f22effff700 1 --2- 172.21.15.31:0/1492162341 >> v2:172.21.15.62:3300/0 conn(0x7f22f006aaa0 0x7f22f00c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:32.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.984+0000 7f22ef7fe700 1 --2- 172.21.15.31:0/1492162341 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f22f013fb20 0x7f22f015a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:32.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.984+0000 7f22ee7fc700 1 -- 172.21.15.31:0/1492162341 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2042606145 0 0) 0x7f22e4001290 con 0x7f22f013fef0 2023-10-06T02:31:32.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.984+0000 7f22ee7fc700 1 -- 172.21.15.31:0/1492162341 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22d8003570 con 0x7f22f013fef0 2023-10-06T02:31:32.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.984+0000 7f22effff700 1 -- 172.21.15.31:0/1492162341 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f22f013fb20 msgr2=0x7f22f015a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:32.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.984+0000 7f22effff700 1 --2- 172.21.15.31:0/1492162341 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f22f013fb20 0x7f22f015a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:32.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.984+0000 7f22effff700 1 -- 172.21.15.31:0/1492162341 >> v1:172.21.15.31:6789/0 conn(0x7f22f013fef0 legacy=0x7f22f00c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:32.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.984+0000 7f22effff700 1 -- 172.21.15.31:0/1492162341 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22f006b520 con 0x7f22f006aaa0 2023-10-06T02:31:32.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.985+0000 7f22effff700 1 --2- 172.21.15.31:0/1492162341 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22f006aaa0 0x7f22f00c9d00 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f22e800d0d0 tx=0x7f22e800b990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5986e90bea41e3e6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:32.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.985+0000 7f22ee7fc700 1 -- 172.21.15.31:0/1492162341 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e8010070 con 0x7f22f006aaa0 2023-10-06T02:31:32.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.985+0000 7f22ee7fc700 1 -- 172.21.15.31:0/1492162341 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22e800b2e0 con 0x7f22f006aaa0 2023-10-06T02:31:32.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.985+0000 7f22ee7fc700 1 -- 172.21.15.31:0/1492162341 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e800dcd0 con 0x7f22f006aaa0 2023-10-06T02:31:32.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.985+0000 7f22f4f9d700 1 -- 172.21.15.31:0/1492162341 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22f006aaa0 msgr2=0x7f22f00c9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:32.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.985+0000 7f22f4f9d700 1 --2- 172.21.15.31:0/1492162341 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22f006aaa0 0x7f22f00c9d00 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f22e800d0d0 tx=0x7f22e800b990 comp rx=0 tx=0).stop 2023-10-06T02:31:32.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.986+0000 7f22f4f9d700 1 -- 172.21.15.31:0/1492162341 shutdown_connections 2023-10-06T02:31:32.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.986+0000 7f22f4f9d700 1 --2- 172.21.15.31:0/1492162341 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22f006aaa0 0x7f22f00c9d00 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:32.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.986+0000 7f22f4f9d700 1 --2- 172.21.15.31:0/1492162341 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f22f013fb20 0x7f22f015a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:32.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.986+0000 7f22f4f9d700 1 -- 172.21.15.31:0/1492162341 >> 172.21.15.31:0/1492162341 conn(0x7f22f006d170 msgr2=0x7f22f00cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:32.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.986+0000 7f22f4f9d700 1 -- 172.21.15.31:0/1492162341 shutdown_connections 2023-10-06T02:31:32.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.986+0000 7f22f4f9d700 1 -- 172.21.15.31:0/1492162341 wait complete. 2023-10-06T02:31:32.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.987+0000 7f22f4f9d700 1 Processor -- start 2023-10-06T02:31:32.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.987+0000 7f22f4f9d700 1 -- start start 2023-10-06T02:31:32.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.988+0000 7f22f4f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f22f006aaa0 0x7f22f00c0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:32.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.988+0000 7f22f4f9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22f013fb20 0x7f22f00c1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:32.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.988+0000 7f22f4f9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22f013fef0 0x7f22f00b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:32.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.988+0000 7f22f4f9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22f00af190 con 0x7f22f013fb20 2023-10-06T02:31:32.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.988+0000 7f22f4f9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22f00afd10 con 0x7f22f013fef0 2023-10-06T02:31:32.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.988+0000 7f22f4f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22f00b01a0 con 0x7f22f006aaa0 2023-10-06T02:31:32.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.988+0000 7f22eeffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22f013fb20 0x7f22f00c1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:32.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.989+0000 7f22eeffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22f013fb20 0x7f22f00c1410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51646/0 (socket says 172.21.15.31:51646) 2023-10-06T02:31:32.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.989+0000 7f22eeffd700 1 -- 172.21.15.31:0/1466719988 learned_addr learned my addr 172.21.15.31:0/1466719988 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:32.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.989+0000 7f22ef7fe700 1 --2- 172.21.15.31:0/1466719988 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f22f006aaa0 0x7f22f00c0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:32.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.989+0000 7f22effff700 1 --2- 172.21.15.31:0/1466719988 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22f013fef0 0x7f22f00b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:32.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.989+0000 7f22eeffd700 1 -- 172.21.15.31:0/1466719988 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f22f006aaa0 msgr2=0x7f22f00c0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:32.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.989+0000 7f22eeffd700 1 --2- 172.21.15.31:0/1466719988 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f22f006aaa0 0x7f22f00c0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:32.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.989+0000 7f22eeffd700 1 -- 172.21.15.31:0/1466719988 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22f013fef0 msgr2=0x7f22f00b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:32.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.989+0000 7f22eeffd700 1 --2- 172.21.15.31:0/1466719988 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22f013fef0 0x7f22f00b12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:32.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.989+0000 7f22eeffd700 1 -- 172.21.15.31:0/1466719988 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22e800bc30 con 0x7f22f013fb20 2023-10-06T02:31:32.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.989+0000 7f22effff700 1 --2- 172.21.15.31:0/1466719988 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22f013fef0 0x7f22f00b12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:31:32.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.989+0000 7f22eeffd700 1 --2- 172.21.15.31:0/1466719988 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22f013fb20 0x7f22f00c1410 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f22e400c7c0 tx=0x7f22e4007ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:32.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.989+0000 7f22d77fe700 1 -- 172.21.15.31:0/1466719988 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e4007300 con 0x7f22f013fb20 2023-10-06T02:31:32.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.989+0000 7f22d77fe700 1 -- 172.21.15.31:0/1466719988 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22e4007460 con 0x7f22f013fb20 2023-10-06T02:31:32.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.989+0000 7f22d77fe700 1 -- 172.21.15.31:0/1466719988 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e40057f0 con 0x7f22f013fb20 2023-10-06T02:31:32.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.989+0000 7f22f4f9d700 1 -- 172.21.15.31:0/1466719988 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22f00af3e0 con 0x7f22f013fb20 2023-10-06T02:31:32.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.990+0000 7f22f4f9d700 1 -- 172.21.15.31:0/1466719988 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22f00af9a0 con 0x7f22f013fb20 2023-10-06T02:31:32.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.991+0000 7f22f4f9d700 1 -- 172.21.15.31:0/1466719988 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22f00afe80 con 0x7f22f013fb20 2023-10-06T02:31:32.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.992+0000 7f22d77fe700 1 -- 172.21.15.31:0/1466719988 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 85403+0+0 (secure 0 0 0) 0x7f22e4005ba0 con 0x7f22f013fb20 2023-10-06T02:31:32.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.992+0000 7f22d77fe700 1 --2- 172.21.15.31:0/1466719988 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f22dc067680 0x7f22dc069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:32.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.993+0000 7f22d77fe700 1 -- 172.21.15.31:0/1466719988 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f22e40851b0 con 0x7f22f013fb20 2023-10-06T02:31:32.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.993+0000 7f22ef7fe700 1 --2- 172.21.15.31:0/1466719988 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f22dc067680 0x7f22dc069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:32.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.993+0000 7f22ef7fe700 1 --2- 172.21.15.31:0/1466719988 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f22dc067680 0x7f22dc069b30 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f22e0009cb0 tx=0x7f22e0008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:32.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:32.996+0000 7f22d77fe700 1 -- 172.21.15.31:0/1466719988 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f22e4082570 con 0x7f22f013fb20 2023-10-06T02:31:33.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:33.251+0000 7f22f4f9d700 1 -- 172.21.15.31:0/1466719988 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f22f004c3c0 con 0x7f22f013fb20 2023-10-06T02:31:33.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:33.251+0000 7f22d77fe700 1 -- 172.21.15.31:0/1466719988 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v109) v1 ==== 53+0+73 (secure 0 0 0) 0x7f22e40513e0 con 0x7f22f013fb20 2023-10-06T02:31:33.253 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T02:31:33.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:33.253+0000 7f22f4f9d700 1 -- 172.21.15.31:0/1466719988 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f22dc067680 msgr2=0x7f22dc069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:33.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:33.253+0000 7f22f4f9d700 1 --2- 172.21.15.31:0/1466719988 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f22dc067680 0x7f22dc069b30 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f22e0009cb0 tx=0x7f22e0008040 comp rx=0 tx=0).stop 2023-10-06T02:31:33.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:33.253+0000 7f22f4f9d700 1 -- 172.21.15.31:0/1466719988 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22f013fb20 msgr2=0x7f22f00c1410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:33.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:33.253+0000 7f22f4f9d700 1 --2- 172.21.15.31:0/1466719988 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22f013fb20 0x7f22f00c1410 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f22e400c7c0 tx=0x7f22e4007ae0 comp rx=0 tx=0).stop 2023-10-06T02:31:33.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:33.254+0000 7f22f4f9d700 1 -- 172.21.15.31:0/1466719988 shutdown_connections 2023-10-06T02:31:33.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:33.254+0000 7f22f4f9d700 1 --2- 172.21.15.31:0/1466719988 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22f013fef0 0x7f22f00b12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:33.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:33.254+0000 7f22f4f9d700 1 --2- 172.21.15.31:0/1466719988 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f22dc067680 0x7f22dc069b30 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:33.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:33.254+0000 7f22f4f9d700 1 --2- 172.21.15.31:0/1466719988 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f22f006aaa0 0x7f22f00c0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:33.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:33.254+0000 7f22f4f9d700 1 --2- 172.21.15.31:0/1466719988 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22f013fb20 0x7f22f00c1410 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:33.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:33.254+0000 7f22f4f9d700 1 -- 172.21.15.31:0/1466719988 >> 172.21.15.31:0/1466719988 conn(0x7f22f006d170 msgr2=0x7f22f006e670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:33.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:33.254+0000 7f22f4f9d700 1 -- 172.21.15.31:0/1466719988 shutdown_connections 2023-10-06T02:31:33.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:33.254+0000 7f22f4f9d700 1 -- 172.21.15.31:0/1466719988 wait complete. 2023-10-06T02:31:33.273 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T02:31:33.273 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T02:31:33.273 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi031.front.sepia.ceph.com 2023-10-06T02:31:33.273 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T02:31:33.273 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T02:31:33.273 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T02:31:33.273 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T02:31:33.273 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T02:31:33.273 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T02:31:33.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:31:33.274 DEBUG:teuthology.orchestra.run.smithi031:> ip addr 2023-10-06T02:31:33.311 INFO:teuthology.orchestra.run.smithi031.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T02:31:33.312 INFO:teuthology.orchestra.run.smithi031.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T02:31:33.312 INFO:teuthology.orchestra.run.smithi031.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T02:31:33.312 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:31:33.312 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 ::1/128 scope host 2023-10-06T02:31:33.312 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:31:33.312 INFO:teuthology.orchestra.run.smithi031.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T02:31:33.312 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cc brd ff:ff:ff:ff:ff:ff 2023-10-06T02:31:33.312 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f0 2023-10-06T02:31:33.312 INFO:teuthology.orchestra.run.smithi031.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T02:31:33.312 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cd brd ff:ff:ff:ff:ff:ff 2023-10-06T02:31:33.312 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f1 2023-10-06T02:31:33.312 INFO:teuthology.orchestra.run.smithi031.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T02:31:33.313 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:ba brd ff:ff:ff:ff:ff:ff 2023-10-06T02:31:33.313 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f0 2023-10-06T02:31:33.313 INFO:teuthology.orchestra.run.smithi031.stdout: inet 172.21.15.31/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T02:31:33.313 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft 41763sec preferred_lft 41763sec 2023-10-06T02:31:33.313 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::ae1f:6bff:fea5:caba/64 scope link 2023-10-06T02:31:33.313 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:31:33.313 INFO:teuthology.orchestra.run.smithi031.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T02:31:33.313 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:bb brd ff:ff:ff:ff:ff:ff 2023-10-06T02:31:33.313 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f1 2023-10-06T02:31:33.313 INFO:teuthology.orchestra.run.smithi031.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T02:31:33.313 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 16:d7:88:2d:d0:c2 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:31:33.313 INFO:teuthology.orchestra.run.smithi031.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T02:31:33.314 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:31:33.314 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::408:52ff:fe3c:ba8f/64 scope link 2023-10-06T02:31:33.314 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:31:33.314 INFO:teuthology.orchestra.run.smithi031.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T02:31:33.314 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 16:d7:88:2d:d0:c2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T02:31:33.314 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::14d7:88ff:fe2d:d0c2/64 scope link 2023-10-06T02:31:33.314 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:31:33.314 INFO:teuthology.orchestra.run.smithi031.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T02:31:33.314 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 82:16:f1:00:cf:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T02:31:33.314 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::8016:f1ff:fe00:cf2f/64 scope link 2023-10-06T02:31:33.314 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:31:33.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:31:33.315 DEBUG:teuthology.orchestra.run.smithi031:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T02:31:33.412 INFO:teuthology.orchestra.run.smithi031.stdout:1 2023-10-06T02:31:33.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:31:33.415 DEBUG:teuthology.orchestra.run.smithi031:> route 2023-10-06T02:31:33.453 INFO:teuthology.orchestra.run.smithi031.stdout:Kernel IP routing table 2023-10-06T02:31:33.454 INFO:teuthology.orchestra.run.smithi031.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T02:31:33.454 INFO:teuthology.orchestra.run.smithi031.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T02:31:33.454 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T02:31:33.454 INFO:teuthology.orchestra.run.smithi031.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T02:31:33.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:31:33.455 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2023-10-06T02:31:33.455 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2023-10-06T02:31:33.455 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T02:31:33.455 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T02:31:33.455 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T02:31:33.455 DEBUG:teuthology.orchestra.run.smithi031:> ') 2023-10-06T02:31:33.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:31:33.653 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2023-10-06T02:31:33.690 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T02:31:33.690 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T02:31:33.690 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T02:31:33.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:31:33.690 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set brx.0 up 2023-10-06T02:31:33.794 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T02:31:33.795 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T02:31:33.831 INFO:teuthology.orchestra.run.smithi031.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T02:31:33.831 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T02:31:33.831 DEBUG:teuthology.orchestra.run.smithi031:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T02:31:33.907 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-06T02:31:33.907 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T02:31:33.908 DEBUG:teuthology.orchestra.run.smithi031:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T02:31:34.040 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T02:31:34.040 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T02:31:34.068 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T02:31:34.069 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T02:31:34.069 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.62:3300,172.21.15.62:3301" 2023-10-06T02:31:34.069 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: trying mount with new device syntax: 0@d6fcf512-874b-4259-b90a-30ae070f00a9.cephfs=/ 2023-10-06T02:31:34.069 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.31:3300/172.21.15.62:3300/172.21.15.62:3301" will pass to kernel 2023-10-06T02:31:34.074 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T02:31:34.074 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:31:34.074 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T02:31:34.137 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T02:31:34.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:34.327+0000 7fe74d39d700 1 Processor -- start 2023-10-06T02:31:34.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:34.327+0000 7fe74d39d700 1 -- start start 2023-10-06T02:31:34.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:34.328+0000 7fe74d39d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe748159640 0x7fe74806d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:34.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:34.328+0000 7fe74d39d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7480c7420 con 0x7fe7480caec0 2023-10-06T02:31:34.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:34.328+0000 7fe74d39d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe74806d5a0 con 0x7fe748159a10 2023-10-06T02:31:34.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:34.328+0000 7fe74d39d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe74806d820 con 0x7fe748159640 2023-10-06T02:31:34.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:34.329+0000 7fe747fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe748159640 0x7fe74806d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:34.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:34.329+0000 7fe747fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe748159640 0x7fe74806d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51662/0 (socket says 172.21.15.31:51662) 2023-10-06T02:31:34.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:34.329+0000 7fe747fff700 1 -- 172.21.15.31:0/1833211425 learned_addr learned my addr 172.21.15.31:0/1833211425 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:34.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:34.329+0000 7fe74cb9c700 1 --1- 172.21.15.31:0/1833211425 >> v1:172.21.15.31:6789/0 conn(0x7fe7480caec0 0x7fe7480c78b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58696/0 (socket says 172.21.15.31:58696) 2023-10-06T02:31:34.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:34.329+0000 7fe747fff700 1 -- 172.21.15.31:0/1833211425 >> v1:172.21.15.62:6789/0 conn(0x7fe748159a10 legacy=0x7fe7480ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:34.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:34.329+0000 7fe747fff700 1 -- 172.21.15.31:0/1833211425 >> v1:172.21.15.31:6789/0 conn(0x7fe7480caec0 legacy=0x7fe7480c78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:34.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:34.329+0000 7fe747fff700 1 -- 172.21.15.31:0/1833211425 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe74806db00 con 0x7fe748159640 2023-10-06T02:31:34.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:34.329+0000 7fe747fff700 1 --2- 172.21.15.31:0/1833211425 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe748159640 0x7fe74806d060 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7fe7380099e0 tx=0x7fe738009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1003235cbc00e050 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:34.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:34.330+0000 7fe746ffd700 1 -- 172.21.15.31:0/1833211425 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe73800b820 con 0x7fe748159640 2023-10-06T02:31:34.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:34.330+0000 7fe746ffd700 1 -- 172.21.15.31:0/1833211425 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe73800b980 con 0x7fe748159640 2023-10-06T02:31:34.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:34.330+0000 7fe746ffd700 1 -- 172.21.15.31:0/1833211425 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe73800bd20 con 0x7fe748159640 2023-10-06T02:31:34.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:34.330+0000 7fe74d39d700 1 -- 172.21.15.31:0/1833211425 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe748159640 msgr2=0x7fe74806d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:34.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:34.330+0000 7fe74d39d700 1 --2- 172.21.15.31:0/1833211425 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe748159640 0x7fe74806d060 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7fe7380099e0 tx=0x7fe738009250 comp rx=0 tx=0).stop 2023-10-06T02:31:34.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:34.330+0000 7fe74d39d700 1 -- 172.21.15.31:0/1833211425 shutdown_connections 2023-10-06T02:31:34.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:34.330+0000 7fe74d39d700 1 --2- 172.21.15.31:0/1833211425 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe748159640 0x7fe74806d060 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:34.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:34.330+0000 7fe74d39d700 1 -- 172.21.15.31:0/1833211425 >> 172.21.15.31:0/1833211425 conn(0x7fe74806e720 msgr2=0x7fe74806eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:34.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:34.331+0000 7fe74d39d700 1 -- 172.21.15.31:0/1833211425 shutdown_connections 2023-10-06T02:31:34.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:34.331+0000 7fe74d39d700 1 -- 172.21.15.31:0/1833211425 wait complete. 2023-10-06T02:31:34.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:34.332+0000 7fe74d39d700 1 Processor -- start 2023-10-06T02:31:34.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:34.332+0000 7fe74d39d700 1 -- start start 2023-10-06T02:31:34.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:34.333+0000 7fe74d39d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7480caec0 0x7fe74814ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:34.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:34.333+0000 7fe74d39d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe748159640 0x7fe74806eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:34.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:34.333+0000 7fe74d39d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe748159a10 0x7fe74806f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:34.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:34.333+0000 7fe74d39d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe74815f370 con 0x7fe7480caec0 2023-10-06T02:31:34.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:34.333+0000 7fe74d39d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe74815d3a0 con 0x7fe748159640 2023-10-06T02:31:34.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:34.333+0000 7fe74d39d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe74816dc50 con 0x7fe748159a10 2023-10-06T02:31:34.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:34.333+0000 7fe747fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7480caec0 0x7fe74814ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:34.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:34.333+0000 7fe747fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7480caec0 0x7fe74814ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51668/0 (socket says 172.21.15.31:51668) 2023-10-06T02:31:34.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:34.333+0000 7fe747fff700 1 -- 172.21.15.31:0/3614506331 learned_addr learned my addr 172.21.15.31:0/3614506331 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:34.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:34.334+0000 7fe74cb9c700 1 --2- 172.21.15.31:0/3614506331 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe748159a10 0x7fe74806f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:34.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:34.333+0000 7fe7477fe700 1 --2- 172.21.15.31:0/3614506331 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe748159640 0x7fe74806eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:34.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:34.334+0000 7fe747fff700 1 -- 172.21.15.31:0/3614506331 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe748159a10 msgr2=0x7fe74806f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:34.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:34.334+0000 7fe747fff700 1 --2- 172.21.15.31:0/3614506331 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe748159a10 0x7fe74806f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:34.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:34.334+0000 7fe747fff700 1 -- 172.21.15.31:0/3614506331 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe748159640 msgr2=0x7fe74806eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:34.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:34.334+0000 7fe747fff700 1 --2- 172.21.15.31:0/3614506331 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe748159640 0x7fe74806eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:34.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:34.334+0000 7fe747fff700 1 -- 172.21.15.31:0/3614506331 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7380094f0 con 0x7fe7480caec0 2023-10-06T02:31:34.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:34.334+0000 7fe747fff700 1 --2- 172.21.15.31:0/3614506331 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7480caec0 0x7fe74814ac00 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7fe7380094c0 tx=0x7fe738017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:34.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:34.334+0000 7fe72ffff700 1 -- 172.21.15.31:0/3614506331 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe738024440 con 0x7fe7480caec0 2023-10-06T02:31:34.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:34.334+0000 7fe72ffff700 1 -- 172.21.15.31:0/3614506331 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7380245a0 con 0x7fe7480caec0 2023-10-06T02:31:34.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:34.334+0000 7fe72ffff700 1 -- 172.21.15.31:0/3614506331 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe738024960 con 0x7fe7480caec0 2023-10-06T02:31:34.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:34.334+0000 7fe74d39d700 1 -- 172.21.15.31:0/3614506331 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe74814b140 con 0x7fe7480caec0 2023-10-06T02:31:34.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:34.334+0000 7fe74d39d700 1 -- 172.21.15.31:0/3614506331 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe74816d600 con 0x7fe7480caec0 2023-10-06T02:31:34.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:34.335+0000 7fe74d39d700 1 -- 172.21.15.31:0/3614506331 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe74804c3c0 con 0x7fe7480caec0 2023-10-06T02:31:34.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:34.336+0000 7fe72ffff700 1 -- 172.21.15.31:0/3614506331 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 85403+0+0 (secure 0 0 0) 0x7fe73801a420 con 0x7fe7480caec0 2023-10-06T02:31:34.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:34.337+0000 7fe72ffff700 1 --2- 172.21.15.31:0/3614506331 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe734067680 0x7fe734069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:34.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:34.337+0000 7fe72ffff700 1 -- 172.21.15.31:0/3614506331 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fe738087460 con 0x7fe7480caec0 2023-10-06T02:31:34.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:34.338+0000 7fe7477fe700 1 --2- 172.21.15.31:0/3614506331 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe734067680 0x7fe734069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:34.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:34.338+0000 7fe7477fe700 1 --2- 172.21.15.31:0/3614506331 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe734067680 0x7fe734069b30 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fe74816d150 tx=0x7fe73c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:34.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:34.342+0000 7fe72ffff700 1 -- 172.21.15.31:0/3614506331 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe738084c30 con 0x7fe7480caec0 2023-10-06T02:31:34.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:34.592+0000 7fe74d39d700 1 -- 172.21.15.31:0/3614506331 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe7480cb2f0 con 0x7fe7480caec0 2023-10-06T02:31:34.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:34.593+0000 7fe72ffff700 1 -- 172.21.15.31:0/3614506331 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v109) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe738053a20 con 0x7fe7480caec0 2023-10-06T02:31:34.595 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T02:31:34.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:34.595+0000 7fe74d39d700 1 -- 172.21.15.31:0/3614506331 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe734067680 msgr2=0x7fe734069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:34.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:34.595+0000 7fe74d39d700 1 --2- 172.21.15.31:0/3614506331 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe734067680 0x7fe734069b30 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fe74816d150 tx=0x7fe73c009040 comp rx=0 tx=0).stop 2023-10-06T02:31:34.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:34.595+0000 7fe74d39d700 1 -- 172.21.15.31:0/3614506331 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7480caec0 msgr2=0x7fe74814ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:34.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:34.595+0000 7fe74d39d700 1 --2- 172.21.15.31:0/3614506331 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7480caec0 0x7fe74814ac00 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7fe7380094c0 tx=0x7fe738017000 comp rx=0 tx=0).stop 2023-10-06T02:31:34.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:34.595+0000 7fe74d39d700 1 -- 172.21.15.31:0/3614506331 shutdown_connections 2023-10-06T02:31:34.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:34.595+0000 7fe74d39d700 1 --2- 172.21.15.31:0/3614506331 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe734067680 0x7fe734069b30 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:34.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:34.595+0000 7fe74d39d700 1 --2- 172.21.15.31:0/3614506331 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe748159a10 0x7fe74806f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:34.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:34.595+0000 7fe74d39d700 1 --2- 172.21.15.31:0/3614506331 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7480caec0 0x7fe74814ac00 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:34.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:34.595+0000 7fe74d39d700 1 --2- 172.21.15.31:0/3614506331 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe748159640 0x7fe74806eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:34.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:34.595+0000 7fe74d39d700 1 -- 172.21.15.31:0/3614506331 >> 172.21.15.31:0/3614506331 conn(0x7fe74806e720 msgr2=0x7fe74814bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:34.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:34.595+0000 7fe74d39d700 1 -- 172.21.15.31:0/3614506331 shutdown_connections 2023-10-06T02:31:34.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:34.596+0000 7fe74d39d700 1 -- 172.21.15.31:0/3614506331 wait complete. 2023-10-06T02:31:34.614 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T02:31:34.614 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T02:31:34.614 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi062.front.sepia.ceph.com 2023-10-06T02:31:34.614 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-06T02:31:34.614 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-06T02:31:34.614 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T02:31:34.615 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T02:31:34.615 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T02:31:34.615 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T02:31:34.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:31:34.615 DEBUG:teuthology.orchestra.run.smithi062:> ip addr 2023-10-06T02:31:34.657 INFO:teuthology.orchestra.run.smithi062.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T02:31:34.657 INFO:teuthology.orchestra.run.smithi062.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T02:31:34.657 INFO:teuthology.orchestra.run.smithi062.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T02:31:34.657 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:31:34.657 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 ::1/128 scope host 2023-10-06T02:31:34.657 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:31:34.657 INFO:teuthology.orchestra.run.smithi062.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T02:31:34.657 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:14 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:31:34.658 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f0 2023-10-06T02:31:34.658 INFO:teuthology.orchestra.run.smithi062.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T02:31:34.658 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:15 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:31:34.658 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f1 2023-10-06T02:31:34.658 INFO:teuthology.orchestra.run.smithi062.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T02:31:34.658 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c2 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:31:34.658 INFO:teuthology.orchestra.run.smithi062.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T02:31:34.658 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c3 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:31:34.658 INFO:teuthology.orchestra.run.smithi062.stdout: inet 172.21.15.62/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T02:31:34.658 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft 41754sec preferred_lft 41754sec 2023-10-06T02:31:34.658 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::ec4:7aff:febb:d1c3/64 scope link 2023-10-06T02:31:34.659 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:31:34.659 INFO:teuthology.orchestra.run.smithi062.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T02:31:34.659 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether ae:42:a8:dd:f7:9b brd ff:ff:ff:ff:ff:ff 2023-10-06T02:31:34.659 INFO:teuthology.orchestra.run.smithi062.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T02:31:34.659 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:31:34.659 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::1098:a4ff:fe0d:389/64 scope link 2023-10-06T02:31:34.659 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:31:34.659 INFO:teuthology.orchestra.run.smithi062.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T02:31:34.659 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether fe:60:2b:8e:af:28 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T02:31:34.659 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::fc60:2bff:fe8e:af28/64 scope link 2023-10-06T02:31:34.659 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:31:34.660 INFO:teuthology.orchestra.run.smithi062.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T02:31:34.660 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether ae:42:a8:dd:f7:9b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T02:31:34.660 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::ac42:a8ff:fedd:f79b/64 scope link 2023-10-06T02:31:34.660 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:31:34.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:31:34.660 DEBUG:teuthology.orchestra.run.smithi062:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T02:31:34.761 INFO:teuthology.orchestra.run.smithi062.stdout:1 2023-10-06T02:31:34.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:31:34.765 DEBUG:teuthology.orchestra.run.smithi062:> route 2023-10-06T02:31:34.804 INFO:teuthology.orchestra.run.smithi062.stdout:Kernel IP routing table 2023-10-06T02:31:34.804 INFO:teuthology.orchestra.run.smithi062.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T02:31:34.804 INFO:teuthology.orchestra.run.smithi062.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T02:31:34.804 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T02:31:34.805 INFO:teuthology.orchestra.run.smithi062.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T02:31:34.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:31:34.806 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2023-10-06T02:31:34.806 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2023-10-06T02:31:34.806 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T02:31:34.806 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T02:31:34.807 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T02:31:34.807 DEBUG:teuthology.orchestra.run.smithi062:> ') 2023-10-06T02:31:35.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:31:35.006 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2023-10-06T02:31:35.042 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-06T02:31:35.042 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T02:31:35.043 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T02:31:35.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:31:35.043 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set brx.0 up 2023-10-06T02:31:35.148 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T02:31:35.148 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-06T02:31:35.186 INFO:teuthology.orchestra.run.smithi062.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-06T02:31:35.186 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T02:31:35.187 DEBUG:teuthology.orchestra.run.smithi062:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-06T02:31:35.263 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-06T02:31:35.263 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T02:31:35.263 DEBUG:teuthology.orchestra.run.smithi062:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T02:31:35.395 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T02:31:35.396 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T02:31:35.423 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T02:31:35.423 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T02:31:35.423 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.62:3300,172.21.15.62:3301" 2023-10-06T02:31:35.424 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: trying mount with new device syntax: 1@d6fcf512-874b-4259-b90a-30ae070f00a9.cephfs=/ 2023-10-06T02:31:35.424 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.31:3300/172.21.15.62:3300/172.21.15.62:3301" will pass to kernel 2023-10-06T02:31:35.428 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T02:31:35.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:31:35.429 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-06T02:31:35.490 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-10-06T02:31:35.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:35.692+0000 7f953959d700 1 Processor -- start 2023-10-06T02:31:35.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:35.692+0000 7f953959d700 1 -- start start 2023-10-06T02:31:35.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:35.693+0000 7f953959d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f95340c8200 0x7f95340c1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:35.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:35.693+0000 7f953959d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f95340c2a90 0x7f9534156aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:35.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:35.693+0000 7f953959d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95341579b0 con 0x7f95340c85d0 2023-10-06T02:31:35.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:35.693+0000 7f953959d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95340c1390 con 0x7f95340c8200 2023-10-06T02:31:35.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:35.693+0000 7f953959d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95341573f0 con 0x7f95340c2a90 2023-10-06T02:31:35.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:35.694+0000 7f9533fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f95340c8200 0x7f95340c1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:35.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:35.694+0000 7f9533fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f95340c8200 0x7f95340c1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51674/0 (socket says 172.21.15.31:51674) 2023-10-06T02:31:35.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:35.694+0000 7f9533fff700 1 -- 172.21.15.31:0/2716221043 learned_addr learned my addr 172.21.15.31:0/2716221043 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:35.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:35.694+0000 7f95337fe700 1 --2- 172.21.15.31:0/2716221043 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f95340c2a90 0x7f9534156aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:35.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:35.694+0000 7f9533fff700 1 -- 172.21.15.31:0/2716221043 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f95340c2a90 msgr2=0x7f9534156aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:35.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:35.694+0000 7f9533fff700 1 --2- 172.21.15.31:0/2716221043 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f95340c2a90 0x7f9534156aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:35.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:35.694+0000 7f9533fff700 1 -- 172.21.15.31:0/2716221043 >> v1:172.21.15.62:6789/0 conn(0x7f95340c85d0 legacy=0x7f95340cc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:35.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:35.694+0000 7f9533fff700 1 -- 172.21.15.31:0/2716221043 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95341576d0 con 0x7f95340c8200 2023-10-06T02:31:35.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:35.694+0000 7f9533fff700 1 --2- 172.21.15.31:0/2716221043 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95340c8200 0x7f95340c1820 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f95240099b0 tx=0x7f9524009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f0a29b9cc4bf7e9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:35.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:35.695+0000 7f9532ffd700 1 -- 172.21.15.31:0/2716221043 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f952400b860 con 0x7f95340c8200 2023-10-06T02:31:35.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:35.695+0000 7f9532ffd700 1 -- 172.21.15.31:0/2716221043 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f952400b9c0 con 0x7f95340c8200 2023-10-06T02:31:35.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:35.695+0000 7f9532ffd700 1 -- 172.21.15.31:0/2716221043 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f952400bd60 con 0x7f95340c8200 2023-10-06T02:31:35.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:35.695+0000 7f953959d700 1 -- 172.21.15.31:0/2716221043 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95340c8200 msgr2=0x7f95340c1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:35.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:35.695+0000 7f953959d700 1 --2- 172.21.15.31:0/2716221043 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95340c8200 0x7f95340c1820 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f95240099b0 tx=0x7f9524009250 comp rx=0 tx=0).stop 2023-10-06T02:31:35.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:35.695+0000 7f953959d700 1 -- 172.21.15.31:0/2716221043 shutdown_connections 2023-10-06T02:31:35.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:35.695+0000 7f953959d700 1 --2- 172.21.15.31:0/2716221043 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95340c8200 0x7f95340c1820 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:35.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:35.695+0000 7f953959d700 1 --2- 172.21.15.31:0/2716221043 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f95340c2a90 0x7f9534156aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:35.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:35.695+0000 7f953959d700 1 -- 172.21.15.31:0/2716221043 >> 172.21.15.31:0/2716221043 conn(0x7f953406c6a0 msgr2=0x7f953406caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:35.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:35.695+0000 7f953959d700 1 -- 172.21.15.31:0/2716221043 shutdown_connections 2023-10-06T02:31:35.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:35.695+0000 7f953959d700 1 -- 172.21.15.31:0/2716221043 wait complete. 2023-10-06T02:31:35.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:35.697+0000 7f953959d700 1 Processor -- start 2023-10-06T02:31:35.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:35.697+0000 7f953959d700 1 -- start start 2023-10-06T02:31:35.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:35.698+0000 7f953959d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f95340c2a90 0x7f95341444e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:35.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:35.698+0000 7f953959d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95340c8200 0x7f95340af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:35.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:35.698+0000 7f953959d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95340c85d0 0x7f95340afe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:35.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:35.698+0000 7f953959d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9534144ff0 con 0x7f95340c8200 2023-10-06T02:31:35.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:35.698+0000 7f953959d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95340b04a0 con 0x7f95340c85d0 2023-10-06T02:31:35.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:35.698+0000 7f953959d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95340b0780 con 0x7f95340c2a90 2023-10-06T02:31:35.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:35.698+0000 7f95337fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95340c8200 0x7f95340af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:35.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:35.698+0000 7f95337fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95340c8200 0x7f95340af8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51684/0 (socket says 172.21.15.31:51684) 2023-10-06T02:31:35.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:35.698+0000 7f95337fe700 1 -- 172.21.15.31:0/2704238220 learned_addr learned my addr 172.21.15.31:0/2704238220 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:35.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:35.698+0000 7f9533fff700 1 --2- 172.21.15.31:0/2704238220 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f95340c2a90 0x7f95341444e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:35.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:35.698+0000 7f9538d9c700 1 --2- 172.21.15.31:0/2704238220 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95340c85d0 0x7f95340afe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:35.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:35.699+0000 7f95337fe700 1 -- 172.21.15.31:0/2704238220 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f95340c2a90 msgr2=0x7f95341444e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:35.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:35.699+0000 7f95337fe700 1 --2- 172.21.15.31:0/2704238220 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f95340c2a90 0x7f95341444e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:35.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:35.699+0000 7f95337fe700 1 -- 172.21.15.31:0/2704238220 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95340c85d0 msgr2=0x7f95340afe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:35.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:35.699+0000 7f95337fe700 1 --2- 172.21.15.31:0/2704238220 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95340c85d0 0x7f95340afe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:35.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:35.699+0000 7f95337fe700 1 -- 172.21.15.31:0/2704238220 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95240094f0 con 0x7f95340c8200 2023-10-06T02:31:35.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:35.699+0000 7f95337fe700 1 --2- 172.21.15.31:0/2704238220 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95340c8200 0x7f95340af8d0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f952800d890 tx=0x7f9528008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:35.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:35.699+0000 7f951bfff700 1 -- 172.21.15.31:0/2704238220 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95280125f0 con 0x7f95340c8200 2023-10-06T02:31:35.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:35.699+0000 7f951bfff700 1 -- 172.21.15.31:0/2704238220 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9528012770 con 0x7f95340c8200 2023-10-06T02:31:35.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:35.699+0000 7f953959d700 1 -- 172.21.15.31:0/2704238220 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9534149580 con 0x7f95340c8200 2023-10-06T02:31:35.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:35.699+0000 7f951bfff700 1 -- 172.21.15.31:0/2704238220 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9528012b30 con 0x7f95340c8200 2023-10-06T02:31:35.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:35.699+0000 7f953959d700 1 -- 172.21.15.31:0/2704238220 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9534144a20 con 0x7f95340c8200 2023-10-06T02:31:35.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:35.700+0000 7f953959d700 1 -- 172.21.15.31:0/2704238220 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9534149aa0 con 0x7f95340c8200 2023-10-06T02:31:35.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:35.701+0000 7f951bfff700 1 -- 172.21.15.31:0/2704238220 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 85403+0+0 (secure 0 0 0) 0x7f952801e460 con 0x7f95340c8200 2023-10-06T02:31:35.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:35.702+0000 7f951bfff700 1 --2- 172.21.15.31:0/2704238220 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9520067680 0x7f9520069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:35.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:35.702+0000 7f951bfff700 1 -- 172.21.15.31:0/2704238220 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f9528083650 con 0x7f95340c8200 2023-10-06T02:31:35.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:35.702+0000 7f9533fff700 1 --2- 172.21.15.31:0/2704238220 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9520067680 0x7f9520069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:35.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:35.703+0000 7f9533fff700 1 --2- 172.21.15.31:0/2704238220 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9520067680 0x7f9520069b30 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f95240099b0 tx=0x7f9524017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:35.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:35.706+0000 7f951bfff700 1 -- 172.21.15.31:0/2704238220 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9534149aa0 con 0x7f95340c8200 2023-10-06T02:31:35.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:35.958+0000 7f953959d700 1 -- 172.21.15.31:0/2704238220 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f9534149aa0 con 0x7f95340c8200 2023-10-06T02:31:36.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:36.575+0000 7f951bfff700 1 -- 172.21.15.31:0/2704238220 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v110) v1 ==== 136+0+0 (secure 0 0 0) 0x7f9534149aa0 con 0x7f95340c8200 2023-10-06T02:31:36.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:36.577+0000 7f953959d700 1 -- 172.21.15.31:0/2704238220 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9520067680 msgr2=0x7f9520069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:36.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:36.577+0000 7f953959d700 1 --2- 172.21.15.31:0/2704238220 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9520067680 0x7f9520069b30 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f95240099b0 tx=0x7f9524017000 comp rx=0 tx=0).stop 2023-10-06T02:31:36.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:36.577+0000 7f953959d700 1 -- 172.21.15.31:0/2704238220 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95340c8200 msgr2=0x7f95340af8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:36.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:36.577+0000 7f953959d700 1 --2- 172.21.15.31:0/2704238220 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95340c8200 0x7f95340af8d0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f952800d890 tx=0x7f9528008040 comp rx=0 tx=0).stop 2023-10-06T02:31:36.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:36.577+0000 7f953959d700 1 -- 172.21.15.31:0/2704238220 shutdown_connections 2023-10-06T02:31:36.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:36.578+0000 7f953959d700 1 --2- 172.21.15.31:0/2704238220 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95340c85d0 0x7f95340afe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:36.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:36.578+0000 7f953959d700 1 --2- 172.21.15.31:0/2704238220 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9520067680 0x7f9520069b30 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:36.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:36.578+0000 7f953959d700 1 --2- 172.21.15.31:0/2704238220 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f95340c2a90 0x7f95341444e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:36.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:36.578+0000 7f953959d700 1 --2- 172.21.15.31:0/2704238220 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95340c8200 0x7f95340af8d0 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:36.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:36.578+0000 7f953959d700 1 -- 172.21.15.31:0/2704238220 >> 172.21.15.31:0/2704238220 conn(0x7f953406c6a0 msgr2=0x7f953414fb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:36.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:36.578+0000 7f953959d700 1 -- 172.21.15.31:0/2704238220 shutdown_connections 2023-10-06T02:31:36.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:36.578+0000 7f953959d700 1 -- 172.21.15.31:0/2704238220 wait complete. 2023-10-06T02:31:36.599 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-06T02:31:36.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:36.788+0000 7f050f39d700 1 Processor -- start 2023-10-06T02:31:36.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:36.788+0000 7f050f39d700 1 -- start start 2023-10-06T02:31:36.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:36.789+0000 7f050f39d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f050813fb20 0x7f050815a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:36.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:36.789+0000 7f050f39d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f050813fef0 0x7f05080c7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:36.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:36.789+0000 7f050f39d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f050815ae70 con 0x7f050806aaa0 2023-10-06T02:31:36.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:36.789+0000 7f050f39d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f050806af60 con 0x7f050813fef0 2023-10-06T02:31:36.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:36.789+0000 7f050f39d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f050806b240 con 0x7f050813fb20 2023-10-06T02:31:36.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:36.789+0000 7f050db9a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f050813fef0 0x7f05080c7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:36.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:36.790+0000 7f050db9a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f050813fef0 0x7f05080c7990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51698/0 (socket says 172.21.15.31:51698) 2023-10-06T02:31:36.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:36.790+0000 7f050db9a700 1 -- 172.21.15.31:0/2783537701 learned_addr learned my addr 172.21.15.31:0/2783537701 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:36.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:36.790+0000 7f050e39b700 1 --2- 172.21.15.31:0/2783537701 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f050813fb20 0x7f050815a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:36.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:36.790+0000 7f050db9a700 1 -- 172.21.15.31:0/2783537701 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f050813fb20 msgr2=0x7f050815a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:36.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:36.790+0000 7f050db9a700 1 --2- 172.21.15.31:0/2783537701 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f050813fb20 0x7f050815a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:36.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:36.790+0000 7f050db9a700 1 -- 172.21.15.31:0/2783537701 >> v1:172.21.15.62:6789/0 conn(0x7f050806aaa0 legacy=0x7f05080c6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:36.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:36.790+0000 7f050db9a700 1 -- 172.21.15.31:0/2783537701 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f050806b520 con 0x7f050813fef0 2023-10-06T02:31:36.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:36.790+0000 7f050e39b700 1 --2- 172.21.15.31:0/2783537701 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f050813fb20 0x7f050815a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:31:36.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:36.790+0000 7f050db9a700 1 --2- 172.21.15.31:0/2783537701 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f050813fef0 0x7f05080c7990 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f04f8009990 tx=0x7f04f8009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81dd38e3be112d03 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:36.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:36.790+0000 7f050d399700 1 -- 172.21.15.31:0/2783537701 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04f8005c10 con 0x7f050813fef0 2023-10-06T02:31:36.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:36.790+0000 7f050d399700 1 -- 172.21.15.31:0/2783537701 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04f8005d70 con 0x7f050813fef0 2023-10-06T02:31:36.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:36.790+0000 7f050d399700 1 -- 172.21.15.31:0/2783537701 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04f800b880 con 0x7f050813fef0 2023-10-06T02:31:36.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:36.791+0000 7f050f39d700 1 -- 172.21.15.31:0/2783537701 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f050813fef0 msgr2=0x7f05080c7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:36.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:36.791+0000 7f050f39d700 1 --2- 172.21.15.31:0/2783537701 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f050813fef0 0x7f05080c7990 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f04f8009990 tx=0x7f04f8009220 comp rx=0 tx=0).stop 2023-10-06T02:31:36.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:36.791+0000 7f050f39d700 1 -- 172.21.15.31:0/2783537701 shutdown_connections 2023-10-06T02:31:36.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:36.791+0000 7f050f39d700 1 --2- 172.21.15.31:0/2783537701 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f050813fb20 0x7f050815a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:36.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:36.791+0000 7f050f39d700 1 --2- 172.21.15.31:0/2783537701 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f050813fef0 0x7f05080c7990 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:36.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:36.791+0000 7f050f39d700 1 -- 172.21.15.31:0/2783537701 >> 172.21.15.31:0/2783537701 conn(0x7f050806d170 msgr2=0x7f05080cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:36.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:36.791+0000 7f050f39d700 1 -- 172.21.15.31:0/2783537701 shutdown_connections 2023-10-06T02:31:36.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:36.791+0000 7f050f39d700 1 -- 172.21.15.31:0/2783537701 wait complete. 2023-10-06T02:31:36.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:36.793+0000 7f050f39d700 1 Processor -- start 2023-10-06T02:31:36.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:36.793+0000 7f050f39d700 1 -- start start 2023-10-06T02:31:36.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:36.794+0000 7f050f39d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f050806aaa0 0x7f05080abeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:36.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:36.794+0000 7f050f39d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f050813fb20 0x7f050813e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:36.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:36.794+0000 7f050f39d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f050813fef0 0x7f050813b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:36.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:36.794+0000 7f050f39d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f050813e1b0 con 0x7f050813fb20 2023-10-06T02:31:36.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:36.794+0000 7f050f39d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f050813d840 con 0x7f050806aaa0 2023-10-06T02:31:36.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:36.794+0000 7f050f39d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f050813db20 con 0x7f050813fef0 2023-10-06T02:31:36.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:36.794+0000 7f050db9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f050813fb20 0x7f050813e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:36.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:36.794+0000 7f050db9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f050813fb20 0x7f050813e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51702/0 (socket says 172.21.15.31:51702) 2023-10-06T02:31:36.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:36.794+0000 7f050db9a700 1 -- 172.21.15.31:0/727703176 learned_addr learned my addr 172.21.15.31:0/727703176 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:36.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:36.794+0000 7f050e39b700 1 --2- 172.21.15.31:0/727703176 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f050806aaa0 0x7f05080abeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:36.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:36.794+0000 7f050eb9c700 1 --2- 172.21.15.31:0/727703176 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f050813fef0 0x7f050813b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:36.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:36.794+0000 7f050db9a700 1 -- 172.21.15.31:0/727703176 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f050813fef0 msgr2=0x7f050813b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:36.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:36.794+0000 7f050db9a700 1 --2- 172.21.15.31:0/727703176 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f050813fef0 0x7f050813b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:36.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:36.794+0000 7f050db9a700 1 -- 172.21.15.31:0/727703176 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f050806aaa0 msgr2=0x7f05080abeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:36.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:36.794+0000 7f050db9a700 1 --2- 172.21.15.31:0/727703176 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f050806aaa0 0x7f05080abeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:36.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:36.794+0000 7f050db9a700 1 -- 172.21.15.31:0/727703176 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04f8009480 con 0x7f050813fb20 2023-10-06T02:31:36.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:36.794+0000 7f050db9a700 1 --2- 172.21.15.31:0/727703176 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f050813fb20 0x7f050813e990 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f04f8005d40 tx=0x7f04f8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:36.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:36.795+0000 7f04fe7fc700 1 -- 172.21.15.31:0/727703176 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04f8011a60 con 0x7f050813fb20 2023-10-06T02:31:36.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:36.795+0000 7f04fe7fc700 1 -- 172.21.15.31:0/727703176 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04f8011bc0 con 0x7f050813fb20 2023-10-06T02:31:36.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:36.795+0000 7f04fe7fc700 1 -- 172.21.15.31:0/727703176 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04f801b430 con 0x7f050813fb20 2023-10-06T02:31:36.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:36.795+0000 7f050f39d700 1 -- 172.21.15.31:0/727703176 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f050813e470 con 0x7f050813fb20 2023-10-06T02:31:36.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:36.795+0000 7f050f39d700 1 -- 172.21.15.31:0/727703176 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f050813eed0 con 0x7f050813fb20 2023-10-06T02:31:36.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:36.796+0000 7f050f39d700 1 -- 172.21.15.31:0/727703176 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f050806f420 con 0x7f050813fb20 2023-10-06T02:31:36.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:36.797+0000 7f04fe7fc700 1 -- 172.21.15.31:0/727703176 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 85403+0+0 (secure 0 0 0) 0x7f04f801b860 con 0x7f050813fb20 2023-10-06T02:31:36.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:36.798+0000 7f04fe7fc700 1 --2- 172.21.15.31:0/727703176 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f04ec067680 0x7f04ec069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:36.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:36.798+0000 7f04fe7fc700 1 -- 172.21.15.31:0/727703176 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f04f8087210 con 0x7f050813fb20 2023-10-06T02:31:36.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:36.798+0000 7f050e39b700 1 --2- 172.21.15.31:0/727703176 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f04ec067680 0x7f04ec069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:36.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:36.799+0000 7f050e39b700 1 --2- 172.21.15.31:0/727703176 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f04ec067680 0x7f04ec069b30 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0504006f00 tx=0x7f0504008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:36.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:36.802+0000 7f04fe7fc700 1 -- 172.21.15.31:0/727703176 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f04f80845d0 con 0x7f050813fb20 2023-10-06T02:31:37.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.036+0000 7f050f39d700 1 -- 172.21.15.31:0/727703176 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f050813e2f0 con 0x7f04ec067680 2023-10-06T02:31:37.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.039+0000 7f04fe7fc700 1 -- 172.21.15.31:0/727703176 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f0504010640 con 0x7f04ec067680 2023-10-06T02:31:37.040 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2023-10-06T02:31:37.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:31:37.040 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephfs" 2023-10-06T02:31:37.040 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:31:37.040 INFO:teuthology.orchestra.run.smithi031.stdout:] 2023-10-06T02:31:37.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.041+0000 7f050f39d700 1 -- 172.21.15.31:0/727703176 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f04ec067680 msgr2=0x7f04ec069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:37.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.041+0000 7f050f39d700 1 --2- 172.21.15.31:0/727703176 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f04ec067680 0x7f04ec069b30 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0504006f00 tx=0x7f0504008040 comp rx=0 tx=0).stop 2023-10-06T02:31:37.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.041+0000 7f050f39d700 1 -- 172.21.15.31:0/727703176 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f050813fb20 msgr2=0x7f050813e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:37.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.041+0000 7f050f39d700 1 --2- 172.21.15.31:0/727703176 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f050813fb20 0x7f050813e990 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f04f8005d40 tx=0x7f04f8016000 comp rx=0 tx=0).stop 2023-10-06T02:31:37.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.041+0000 7f050f39d700 1 -- 172.21.15.31:0/727703176 shutdown_connections 2023-10-06T02:31:37.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.041+0000 7f050f39d700 1 --2- 172.21.15.31:0/727703176 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f04ec067680 0x7f04ec069b30 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:37.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.041+0000 7f050f39d700 1 --2- 172.21.15.31:0/727703176 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f050813fef0 0x7f050813b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:37.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.041+0000 7f050f39d700 1 --2- 172.21.15.31:0/727703176 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f050806aaa0 0x7f05080abeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:37.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.041+0000 7f050f39d700 1 --2- 172.21.15.31:0/727703176 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f050813fb20 0x7f050813e990 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:37.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.041+0000 7f050f39d700 1 -- 172.21.15.31:0/727703176 >> 172.21.15.31:0/727703176 conn(0x7f050806d170 msgr2=0x7f05080aef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:37.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.041+0000 7f050f39d700 1 -- 172.21.15.31:0/727703176 shutdown_connections 2023-10-06T02:31:37.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.041+0000 7f050f39d700 1 -- 172.21.15.31:0/727703176 wait complete. 2023-10-06T02:31:37.060 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T02:31:37.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.248+0000 7fcc03fff700 1 Processor -- start 2023-10-06T02:31:37.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.249+0000 7fcc03fff700 1 -- start start 2023-10-06T02:31:37.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.250+0000 7fcc03fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcc040c7fb0 0x7fcc0406abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:37.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.250+0000 7fcc03fff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fcc0413f780 0x7fcc0413fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:37.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.250+0000 7fcc03fff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc04140120 con 0x7fcc040c7be0 2023-10-06T02:31:37.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.250+0000 7fcc03fff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc0406bf10 con 0x7fcc0413f780 2023-10-06T02:31:37.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.250+0000 7fcc03fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0406b290 con 0x7fcc040c7fb0 2023-10-06T02:31:37.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.250+0000 7fcc037fe700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fcc0413f780 0x7fcc0413fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:37.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.250+0000 7fcc027fc700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcc040c7fb0 0x7fcc0406abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:37.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.251+0000 7fcc02ffd700 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fcc040c7be0 0x7fcc0415a990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:54996/0 (socket says 172.21.15.31:54996) 2023-10-06T02:31:37.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.251+0000 7fcc02ffd700 1 -- 172.21.15.31:0/2178324245 learned_addr learned my addr 172.21.15.31:0/2178324245 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:31:37.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.251+0000 7fcc01ffb700 1 -- 172.21.15.31:0/2178324245 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2010715817 0 0) 0x7fcbf4001110 con 0x7fcc040c7be0 2023-10-06T02:31:37.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.251+0000 7fcc01ffb700 1 -- 172.21.15.31:0/2178324245 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcbec003570 con 0x7fcc040c7be0 2023-10-06T02:31:37.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.251+0000 7fcc027fc700 1 -- 172.21.15.31:0/2178324245 >> v2:172.21.15.62:3300/0 conn(0x7fcc0413f780 msgr2=0x7fcc0413fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:37.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.251+0000 7fcc027fc700 1 --2- 172.21.15.31:0/2178324245 >> v2:172.21.15.62:3300/0 conn(0x7fcc0413f780 0x7fcc0413fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:37.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.251+0000 7fcc027fc700 1 -- 172.21.15.31:0/2178324245 >> v1:172.21.15.62:6789/0 conn(0x7fcc040c7be0 legacy=0x7fcc0415a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:37.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.251+0000 7fcc027fc700 1 -- 172.21.15.31:0/2178324245 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc0406b570 con 0x7fcc040c7fb0 2023-10-06T02:31:37.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.251+0000 7fcc037fe700 1 --2- 172.21.15.31:0/2178324245 >> v2:172.21.15.62:3300/0 conn(0x7fcc0413f780 0x7fcc0413fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:31:37.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.252+0000 7fcc027fc700 1 --2- 172.21.15.31:0/2178324245 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcc040c7fb0 0x7fcc0406abd0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fcbf80098e0 tx=0x7fcbf80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39911c0465492526 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:37.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.252+0000 7fcc01ffb700 1 -- 172.21.15.31:0/2178324245 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf8011630 con 0x7fcc040c7fb0 2023-10-06T02:31:37.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.252+0000 7fcc01ffb700 1 -- 172.21.15.31:0/2178324245 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbf8011790 con 0x7fcc040c7fb0 2023-10-06T02:31:37.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.252+0000 7fcc01ffb700 1 -- 172.21.15.31:0/2178324245 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf8011b30 con 0x7fcc040c7fb0 2023-10-06T02:31:37.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.252+0000 7fcc03fff700 1 -- 172.21.15.31:0/2178324245 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcc040c7fb0 msgr2=0x7fcc0406abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:37.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.252+0000 7fcc03fff700 1 --2- 172.21.15.31:0/2178324245 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcc040c7fb0 0x7fcc0406abd0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fcbf80098e0 tx=0x7fcbf80091a0 comp rx=0 tx=0).stop 2023-10-06T02:31:37.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.252+0000 7fcc03fff700 1 -- 172.21.15.31:0/2178324245 shutdown_connections 2023-10-06T02:31:37.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.252+0000 7fcc03fff700 1 --2- 172.21.15.31:0/2178324245 >> v2:172.21.15.62:3300/0 conn(0x7fcc0413f780 0x7fcc0413fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:37.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.252+0000 7fcc03fff700 1 --2- 172.21.15.31:0/2178324245 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcc040c7fb0 0x7fcc0406abd0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:37.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.252+0000 7fcc03fff700 1 -- 172.21.15.31:0/2178324245 >> 172.21.15.31:0/2178324245 conn(0x7fcc0406d110 msgr2=0x7fcc040cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:37.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.253+0000 7fcc03fff700 1 -- 172.21.15.31:0/2178324245 shutdown_connections 2023-10-06T02:31:37.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.253+0000 7fcc03fff700 1 -- 172.21.15.31:0/2178324245 wait complete. 2023-10-06T02:31:37.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.254+0000 7fcc03fff700 1 Processor -- start 2023-10-06T02:31:37.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.254+0000 7fcc03fff700 1 -- start start 2023-10-06T02:31:37.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.255+0000 7fcc03fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc040c7be0 0x7fcc04155c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:37.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.255+0000 7fcc03fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc040c7fb0 0x7fcc0413e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:37.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.255+0000 7fcc03fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcc0413f780 0x7fcc0413c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:37.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.255+0000 7fcc03fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0413d3a0 con 0x7fcc040c7be0 2023-10-06T02:31:37.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.255+0000 7fcc03fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0413d680 con 0x7fcc040c7fb0 2023-10-06T02:31:37.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.255+0000 7fcc03fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0413d9a0 con 0x7fcc0413f780 2023-10-06T02:31:37.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.255+0000 7fcc02ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc040c7be0 0x7fcc04155c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:37.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.255+0000 7fcc027fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc040c7fb0 0x7fcc0413e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:37.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.255+0000 7fcc02ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc040c7be0 0x7fcc04155c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51710/0 (socket says 172.21.15.31:51710) 2023-10-06T02:31:37.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.256+0000 7fcc02ffd700 1 -- 172.21.15.31:0/3697239990 learned_addr learned my addr 172.21.15.31:0/3697239990 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:37.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.256+0000 7fcc037fe700 1 --2- 172.21.15.31:0/3697239990 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcc0413f780 0x7fcc0413c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:37.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.256+0000 7fcc02ffd700 1 -- 172.21.15.31:0/3697239990 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcc0413f780 msgr2=0x7fcc0413c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:37.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.256+0000 7fcc02ffd700 1 --2- 172.21.15.31:0/3697239990 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcc0413f780 0x7fcc0413c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:37.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.256+0000 7fcc02ffd700 1 -- 172.21.15.31:0/3697239990 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc040c7fb0 msgr2=0x7fcc0413e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:37.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.256+0000 7fcc02ffd700 1 --2- 172.21.15.31:0/3697239990 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc040c7fb0 0x7fcc0413e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:37.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.256+0000 7fcc02ffd700 1 -- 172.21.15.31:0/3697239990 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbfc009a20 con 0x7fcc040c7be0 2023-10-06T02:31:37.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.256+0000 7fcc02ffd700 1 --2- 172.21.15.31:0/3697239990 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc040c7be0 0x7fcc04155c60 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7fcbf400c7c0 tx=0x7fcbf4007c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:37.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.256+0000 7fcbeaffd700 1 -- 172.21.15.31:0/3697239990 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf4005b30 con 0x7fcc040c7be0 2023-10-06T02:31:37.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.256+0000 7fcbeaffd700 1 -- 172.21.15.31:0/3697239990 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbf4005cb0 con 0x7fcc040c7be0 2023-10-06T02:31:37.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.256+0000 7fcbeaffd700 1 -- 172.21.15.31:0/3697239990 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf4007320 con 0x7fcc040c7be0 2023-10-06T02:31:37.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.257+0000 7fcc03fff700 1 -- 172.21.15.31:0/3697239990 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbf8009420 con 0x7fcc040c7be0 2023-10-06T02:31:37.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.257+0000 7fcc03fff700 1 -- 172.21.15.31:0/3697239990 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc0413df60 con 0x7fcc040c7be0 2023-10-06T02:31:37.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.257+0000 7fcc03fff700 1 -- 172.21.15.31:0/3697239990 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc0404c3c0 con 0x7fcc040c7be0 2023-10-06T02:31:37.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.259+0000 7fcbeaffd700 1 -- 172.21.15.31:0/3697239990 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 85403+0+0 (secure 0 0 0) 0x7fcbf401b3b0 con 0x7fcc040c7be0 2023-10-06T02:31:37.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.260+0000 7fcbeaffd700 1 --2- 172.21.15.31:0/3697239990 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fcbf0067680 0x7fcbf0069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:37.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.260+0000 7fcbeaffd700 1 -- 172.21.15.31:0/3697239990 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fcbf4084b10 con 0x7fcc040c7be0 2023-10-06T02:31:37.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.260+0000 7fcc027fc700 1 --2- 172.21.15.31:0/3697239990 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fcbf0067680 0x7fcbf0069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:37.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.261+0000 7fcc027fc700 1 --2- 172.21.15.31:0/3697239990 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fcbf0067680 0x7fcbf0069b30 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fcbf80091e0 tx=0x7fcbf8017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:37.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.263+0000 7fcbeaffd700 1 -- 172.21.15.31:0/3697239990 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcbf4081ed0 con 0x7fcc040c7be0 2023-10-06T02:31:37.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.465+0000 7fcc03fff700 1 -- 172.21.15.31:0/3697239990 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fcc04140120 con 0x7fcc040c7be0 2023-10-06T02:31:37.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.466+0000 7fcbeaffd700 1 -- 172.21.15.31:0/3697239990 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v28)=0 v28) v1 ==== 122+0+0 (secure 0 0 0) 0x7fcc04140120 con 0x7fcc040c7be0 2023-10-06T02:31:37.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.468+0000 7fcc03fff700 1 -- 172.21.15.31:0/3697239990 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fcbf0067680 msgr2=0x7fcbf0069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:37.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.468+0000 7fcc03fff700 1 --2- 172.21.15.31:0/3697239990 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fcbf0067680 0x7fcbf0069b30 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fcbf80091e0 tx=0x7fcbf8017000 comp rx=0 tx=0).stop 2023-10-06T02:31:37.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.468+0000 7fcc03fff700 1 -- 172.21.15.31:0/3697239990 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc040c7be0 msgr2=0x7fcc04155c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:37.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.468+0000 7fcc03fff700 1 --2- 172.21.15.31:0/3697239990 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc040c7be0 0x7fcc04155c60 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7fcbf400c7c0 tx=0x7fcbf4007c70 comp rx=0 tx=0).stop 2023-10-06T02:31:37.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.468+0000 7fcc03fff700 1 -- 172.21.15.31:0/3697239990 shutdown_connections 2023-10-06T02:31:37.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.468+0000 7fcc03fff700 1 --2- 172.21.15.31:0/3697239990 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fcbf0067680 0x7fcbf0069b30 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:37.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.468+0000 7fcc03fff700 1 --2- 172.21.15.31:0/3697239990 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcc0413f780 0x7fcc0413c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:37.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.468+0000 7fcc03fff700 1 --2- 172.21.15.31:0/3697239990 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc040c7be0 0x7fcc04155c60 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:37.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.468+0000 7fcc03fff700 1 --2- 172.21.15.31:0/3697239990 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc040c7fb0 0x7fcc0413e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:37.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.468+0000 7fcc03fff700 1 -- 172.21.15.31:0/3697239990 >> 172.21.15.31:0/3697239990 conn(0x7fcc0406d110 msgr2=0x7fcc040c94a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:37.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.468+0000 7fcc03fff700 1 -- 172.21.15.31:0/3697239990 shutdown_connections 2023-10-06T02:31:37.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:37.468+0000 7fcc03fff700 1 -- 172.21.15.31:0/3697239990 wait complete. 2023-10-06T02:31:37.487 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:31:37.487 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvolume_0000000000024848/d345f41e-f146-4ab9-9d43-4c480aff6fd8) 2023-10-06T02:31:37.558 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:31:37.559 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2023-10-06T02:31:37.598 INFO:teuthology.orchestra.run.smithi031.stdout:cephfs_data 2023-10-06T02:31:37.598 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:31:37.599 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/_nogroup/subvolume_0000000000024848/d345f41e-f146-4ab9-9d43-4c480aff6fd8) 2023-10-06T02:31:37.706 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T02:31:37.707 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/_nogroup/subvolume_0000000000024848/.meta 2023-10-06T02:31:37.773 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:31:37.773 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/group_0000000000109489/subvolume_0000000000024849/06983d16-fa80-447b-835a-845df9d91482) 2023-10-06T02:31:37.881 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:31:37.881 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2023-10-06T02:31:37.919 INFO:teuthology.orchestra.run.smithi031.stdout:cephfs_data 2023-10-06T02:31:37.920 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:31:37.920 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/group_0000000000109489/subvolume_0000000000024849/06983d16-fa80-447b-835a-845df9d91482) 2023-10-06T02:31:38.027 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T02:31:38.027 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/group_0000000000109489/subvolume_0000000000024849/.meta 2023-10-06T02:31:38.090 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000024848 2023-10-06T02:31:38.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.319+0000 7f70b5d9d700 1 Processor -- start 2023-10-06T02:31:38.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.319+0000 7f70b5d9d700 1 -- start start 2023-10-06T02:31:38.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.321+0000 7f70b5d9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f70b006e560 0x7f70b00c7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:38.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.321+0000 7f70b5d9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70b00c7e70 con 0x7f70b0155c60 2023-10-06T02:31:38.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.321+0000 7f70b5d9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70b006b790 con 0x7f70b0155890 2023-10-06T02:31:38.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.321+0000 7f70b5d9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70b006ab80 con 0x7f70b006e560 2023-10-06T02:31:38.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.321+0000 7f70affff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f70b0155c60 0x7f70b006de50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:53726/0 (socket says 172.21.15.31:53726) 2023-10-06T02:31:38.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.321+0000 7f70affff700 1 -- 172.21.15.31:0/1147061878 learned_addr learned my addr 172.21.15.31:0/1147061878 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:31:38.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.321+0000 7f70b559c700 1 --2- 172.21.15.31:0/1147061878 >> v2:172.21.15.62:3300/0 conn(0x7f70b006e560 0x7f70b00c7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:38.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.321+0000 7f70af7fe700 1 -- 172.21.15.31:0/1147061878 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3436627216 0 0) 0x7f70b00c7e70 con 0x7f70b0155c60 2023-10-06T02:31:38.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.321+0000 7f70af7fe700 1 -- 172.21.15.31:0/1147061878 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70a0004680 con 0x7f70b0155c60 2023-10-06T02:31:38.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.322+0000 7f70af7fe700 1 -- 172.21.15.31:0/1147061878 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3446278611 0 0) 0x7f70b006b790 con 0x7f70b0155890 2023-10-06T02:31:38.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.322+0000 7f70af7fe700 1 -- 172.21.15.31:0/1147061878 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70b00c7e70 con 0x7f70b0155890 2023-10-06T02:31:38.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.322+0000 7f70af7fe700 1 -- 172.21.15.31:0/1147061878 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1743635387 0 0) 0x7f70a0004680 con 0x7f70b0155c60 2023-10-06T02:31:38.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.322+0000 7f70af7fe700 1 -- 172.21.15.31:0/1147061878 >> v2:172.21.15.62:3300/0 conn(0x7f70b006e560 msgr2=0x7f70b00c7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:38.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.322+0000 7f70af7fe700 1 --2- 172.21.15.31:0/1147061878 >> v2:172.21.15.62:3300/0 conn(0x7f70b006e560 0x7f70b00c7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:38.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.322+0000 7f70af7fe700 1 -- 172.21.15.31:0/1147061878 >> v1:172.21.15.62:6789/0 conn(0x7f70b0155890 legacy=0x7f70b006a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:38.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.322+0000 7f70af7fe700 1 -- 172.21.15.31:0/1147061878 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70b006ae60 con 0x7f70b0155c60 2023-10-06T02:31:38.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.322+0000 7f70af7fe700 1 -- 172.21.15.31:0/1147061878 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7f709c001470 con 0x7f70b0155c60 2023-10-06T02:31:38.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.322+0000 7f70af7fe700 1 -- 172.21.15.31:0/1147061878 >> v1:172.21.15.31:6789/0 conn(0x7f70b0155c60 legacy=0x7f70b006de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:38.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.322+0000 7f70af7fe700 1 --2- 172.21.15.31:0/1147061878 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70a0004d00 0x7f70a00050f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:38.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.322+0000 7f70af7fe700 1 --2- 172.21.15.31:0/1147061878 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70a0005920 0x7f70a0005d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:38.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.322+0000 7f70af7fe700 1 --2- 172.21.15.31:0/1147061878 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f70a00062d0 0x7f70a0003490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:38.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.322+0000 7f70af7fe700 1 -- 172.21.15.31:0/1147061878 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b006ab80 con 0x7f70a0005920 2023-10-06T02:31:38.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.322+0000 7f70af7fe700 1 -- 172.21.15.31:0/1147061878 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a0003bb0 con 0x7f70a0004d00 2023-10-06T02:31:38.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.322+0000 7f70af7fe700 1 -- 172.21.15.31:0/1147061878 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70a0003d70 con 0x7f70a00062d0 2023-10-06T02:31:38.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.322+0000 7f70affff700 1 --2- 172.21.15.31:0/1147061878 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70a0005920 0x7f70a0005d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:38.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.322+0000 7f70b4d9b700 1 --2- 172.21.15.31:0/1147061878 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70a0004d00 0x7f70a00050f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:38.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.322+0000 7f70b559c700 1 --2- 172.21.15.31:0/1147061878 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f70a00062d0 0x7f70a0003490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:38.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.323+0000 7f70affff700 1 -- 172.21.15.31:0/1147061878 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f70a00062d0 msgr2=0x7f70a0003490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:38.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.323+0000 7f70affff700 1 --2- 172.21.15.31:0/1147061878 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f70a00062d0 0x7f70a0003490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:38.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.323+0000 7f70affff700 1 -- 172.21.15.31:0/1147061878 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70a0004d00 msgr2=0x7f70a00050f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:38.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.323+0000 7f70affff700 1 --2- 172.21.15.31:0/1147061878 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70a0004d00 0x7f70a00050f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:38.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.323+0000 7f70affff700 1 -- 172.21.15.31:0/1147061878 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70a0004680 con 0x7f70a0005920 2023-10-06T02:31:38.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.323+0000 7f70b4d9b700 1 --2- 172.21.15.31:0/1147061878 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70a0004d00 0x7f70a00050f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:31:38.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.323+0000 7f70b559c700 1 -- 172.21.15.31:0/1147061878 reap_dead start 2023-10-06T02:31:38.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.323+0000 7f70affff700 1 --2- 172.21.15.31:0/1147061878 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70a0005920 0x7f70a0005d90 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f709c00c2d0 tx=0x7f709c001e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e43730abb5df904d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:38.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.323+0000 7f70af7fe700 1 -- 172.21.15.31:0/1147061878 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f709c00cbc0 con 0x7f70a0005920 2023-10-06T02:31:38.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.323+0000 7f70af7fe700 1 -- 172.21.15.31:0/1147061878 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f709c00cd20 con 0x7f70a0005920 2023-10-06T02:31:38.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.323+0000 7f70af7fe700 1 -- 172.21.15.31:0/1147061878 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f709c005b70 con 0x7f70a0005920 2023-10-06T02:31:38.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.324+0000 7f70b5d9d700 1 -- 172.21.15.31:0/1147061878 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70a0005920 msgr2=0x7f70a0005d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:38.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.324+0000 7f70b5d9d700 1 --2- 172.21.15.31:0/1147061878 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70a0005920 0x7f70a0005d90 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f709c00c2d0 tx=0x7f709c001e10 comp rx=0 tx=0).stop 2023-10-06T02:31:38.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.324+0000 7f70b5d9d700 1 -- 172.21.15.31:0/1147061878 shutdown_connections 2023-10-06T02:31:38.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.324+0000 7f70b5d9d700 1 --2- 172.21.15.31:0/1147061878 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70a0005920 0x7f70a0005d90 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:38.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.324+0000 7f70b5d9d700 1 -- 172.21.15.31:0/1147061878 >> 172.21.15.31:0/1147061878 conn(0x7f70b006d110 msgr2=0x7f70b00cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:38.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.324+0000 7f70b5d9d700 1 -- 172.21.15.31:0/1147061878 shutdown_connections 2023-10-06T02:31:38.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.324+0000 7f70b5d9d700 1 -- 172.21.15.31:0/1147061878 wait complete. 2023-10-06T02:31:38.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.325+0000 7f70b5d9d700 1 Processor -- start 2023-10-06T02:31:38.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.325+0000 7f70b5d9d700 1 -- start start 2023-10-06T02:31:38.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.326+0000 7f70b5d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f70a0005920 0x7f70b0150b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:38.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.326+0000 7f70b5d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70b01573c0 0x7f70b00c3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:38.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.326+0000 7f70b5d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70b0157830 0x7f70b00c4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:38.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.326+0000 7f70b5d9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b0158a10 con 0x7f70b01573c0 2023-10-06T02:31:38.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.326+0000 7f70b5d9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b0158ea0 con 0x7f70b0157830 2023-10-06T02:31:38.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.326+0000 7f70b5d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70b0157e00 con 0x7f70a0005920 2023-10-06T02:31:38.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.326+0000 7f70affff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70b01573c0 0x7f70b00c3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:38.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.327+0000 7f70affff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70b01573c0 0x7f70b00c3b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38778/0 (socket says 172.21.15.31:38778) 2023-10-06T02:31:38.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.327+0000 7f70affff700 1 -- 172.21.15.31:0/1271678303 learned_addr learned my addr 172.21.15.31:0/1271678303 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:38.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.327+0000 7f70b4d9b700 1 --2- 172.21.15.31:0/1271678303 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f70a0005920 0x7f70b0150b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:38.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.327+0000 7f70b559c700 1 --2- 172.21.15.31:0/1271678303 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70b0157830 0x7f70b00c4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:38.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.327+0000 7f70affff700 1 -- 172.21.15.31:0/1271678303 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f70a0005920 msgr2=0x7f70b0150b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:38.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.327+0000 7f70affff700 1 --2- 172.21.15.31:0/1271678303 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f70a0005920 0x7f70b0150b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:38.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.327+0000 7f70affff700 1 -- 172.21.15.31:0/1271678303 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70b0157830 msgr2=0x7f70b00c4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:38.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.327+0000 7f70affff700 1 --2- 172.21.15.31:0/1271678303 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70b0157830 0x7f70b00c4070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:38.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.327+0000 7f70affff700 1 -- 172.21.15.31:0/1271678303 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70b01580e0 con 0x7f70b01573c0 2023-10-06T02:31:38.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.327+0000 7f70b559c700 1 --2- 172.21.15.31:0/1271678303 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70b0157830 0x7f70b00c4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:31:38.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.327+0000 7f70affff700 1 --2- 172.21.15.31:0/1271678303 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70b01573c0 0x7f70b00c3b30 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f70a0003b80 tx=0x7f709c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:38.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.327+0000 7f70acff9700 1 -- 172.21.15.31:0/1271678303 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f709c022070 con 0x7f70b01573c0 2023-10-06T02:31:38.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.327+0000 7f70acff9700 1 -- 172.21.15.31:0/1271678303 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f709c000fe0 con 0x7f70b01573c0 2023-10-06T02:31:38.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.327+0000 7f70b5d9d700 1 -- 172.21.15.31:0/1271678303 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70b01583c0 con 0x7f70b01573c0 2023-10-06T02:31:38.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.327+0000 7f70acff9700 1 -- 172.21.15.31:0/1271678303 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f709c007510 con 0x7f70b01573c0 2023-10-06T02:31:38.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.328+0000 7f70b5d9d700 1 -- 172.21.15.31:0/1271678303 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70b0151040 con 0x7f70b01573c0 2023-10-06T02:31:38.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.329+0000 7f70b5d9d700 1 -- 172.21.15.31:0/1271678303 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70b0158b80 con 0x7f70b01573c0 2023-10-06T02:31:38.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.329+0000 7f70acff9700 1 -- 172.21.15.31:0/1271678303 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 85403+0+0 (secure 0 0 0) 0x7f709c01e430 con 0x7f70b01573c0 2023-10-06T02:31:38.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.331+0000 7f70acff9700 1 --2- 172.21.15.31:0/1271678303 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7094067680 0x7f7094069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:38.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.331+0000 7f70acff9700 1 -- 172.21.15.31:0/1271678303 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f709c086a20 con 0x7f70b01573c0 2023-10-06T02:31:38.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.331+0000 7f70b4d9b700 1 --2- 172.21.15.31:0/1271678303 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7094067680 0x7f7094069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:38.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.332+0000 7f70b4d9b700 1 --2- 172.21.15.31:0/1271678303 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7094067680 0x7f7094069b30 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f70a4000c00 tx=0x7f70a4007d30 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:38.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.335+0000 7f70acff9700 1 -- 172.21.15.31:0/1271678303 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f709c083de0 con 0x7f70b01573c0 2023-10-06T02:31:38.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.570+0000 7f70b5d9d700 1 -- 172.21.15.31:0/1271678303 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000024848", "target": ["mon-mgr", ""]}) v1 -- 0x7f70b004c3c0 con 0x7f7094067680 2023-10-06T02:31:38.580 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:31:38.579+0000 7f656537d700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:31:38.580 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:31:38.579+0000 7f656537d700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:31:38.581 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:31:38.579+0000 7f656537d700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:31:38.581 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:31:38.579+0000 7f656537d700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:31:38.581 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:31:38.579+0000 7f656537d700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:31:38.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.642+0000 7f70acff9700 1 -- 172.21.15.31:0/1271678303 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+82 (secure 0 0 0) 0x7f70b004c3c0 con 0x7f7094067680 2023-10-06T02:31:38.644 INFO:teuthology.orchestra.run.smithi031.stdout:/volumes/_nogroup/subvolume_0000000000024848/d345f41e-f146-4ab9-9d43-4c480aff6fd8 2023-10-06T02:31:38.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.644+0000 7f70b5d9d700 1 -- 172.21.15.31:0/1271678303 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7094067680 msgr2=0x7f7094069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:38.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.644+0000 7f70b5d9d700 1 --2- 172.21.15.31:0/1271678303 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7094067680 0x7f7094069b30 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f70a4000c00 tx=0x7f70a4007d30 comp rx=0 tx=0).stop 2023-10-06T02:31:38.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.644+0000 7f70b5d9d700 1 -- 172.21.15.31:0/1271678303 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70b01573c0 msgr2=0x7f70b00c3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:38.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.644+0000 7f70b5d9d700 1 --2- 172.21.15.31:0/1271678303 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70b01573c0 0x7f70b00c3b30 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f70a0003b80 tx=0x7f709c015000 comp rx=0 tx=0).stop 2023-10-06T02:31:38.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.644+0000 7f70b5d9d700 1 -- 172.21.15.31:0/1271678303 shutdown_connections 2023-10-06T02:31:38.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.644+0000 7f70b5d9d700 1 --2- 172.21.15.31:0/1271678303 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70b0157830 0x7f70b00c4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:38.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.644+0000 7f70b5d9d700 1 --2- 172.21.15.31:0/1271678303 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7094067680 0x7f7094069b30 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:38.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.644+0000 7f70b5d9d700 1 --2- 172.21.15.31:0/1271678303 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f70a0005920 0x7f70b0150b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:38.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.644+0000 7f70b5d9d700 1 --2- 172.21.15.31:0/1271678303 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70b01573c0 0x7f70b00c3b30 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:38.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.644+0000 7f70b5d9d700 1 -- 172.21.15.31:0/1271678303 >> 172.21.15.31:0/1271678303 conn(0x7f70b006d110 msgr2=0x7f70b00c0ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:38.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.644+0000 7f70b5d9d700 1 -- 172.21.15.31:0/1271678303 shutdown_connections 2023-10-06T02:31:38.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.644+0000 7f70b5d9d700 1 -- 172.21.15.31:0/1271678303 wait complete. 2023-10-06T02:31:38.663 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000024849 group_0000000000109489 2023-10-06T02:31:38.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.854+0000 7f6ed0b9d700 1 Processor -- start 2023-10-06T02:31:38.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.854+0000 7f6ed0b9d700 1 -- start start 2023-10-06T02:31:38.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.855+0000 7f6ed0b9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6ecc0c8da0 0x7f6ecc0c1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:38.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.856+0000 7f6ed0b9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6ecc0c9170 0x7f6ecc0c1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:38.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.856+0000 7f6ed0b9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f6ecc15c5e0 0x7f6ecc15ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:38.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.856+0000 7f6ed0b9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ecc069b00 con 0x7f6ecc0c9170 2023-10-06T02:31:38.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.856+0000 7f6ed0b9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ecc0c6d50 con 0x7f6ecc15c5e0 2023-10-06T02:31:38.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.856+0000 7f6ed0b9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ecc0c6ff0 con 0x7f6ecc0c8da0 2023-10-06T02:31:38.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.856+0000 7f6ecaffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6ecc0c9170 0x7f6ecc0c1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:38.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.856+0000 7f6ecaffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6ecc0c9170 0x7f6ecc0c1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38794/0 (socket says 172.21.15.31:38794) 2023-10-06T02:31:38.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.856+0000 7f6ecaffd700 1 -- 172.21.15.31:0/3363099928 learned_addr learned my addr 172.21.15.31:0/3363099928 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:38.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.856+0000 7f6ecbfff700 1 --2- 172.21.15.31:0/3363099928 >> v2:172.21.15.62:3300/0 conn(0x7f6ecc15c5e0 0x7f6ecc15ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:38.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.856+0000 7f6ecb7fe700 1 --2- 172.21.15.31:0/3363099928 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6ecc0c8da0 0x7f6ecc0c1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:38.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.856+0000 7f6ecaffd700 1 -- 172.21.15.31:0/3363099928 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6ecc0c8da0 msgr2=0x7f6ecc0c1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:38.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.856+0000 7f6ecaffd700 1 --2- 172.21.15.31:0/3363099928 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6ecc0c8da0 0x7f6ecc0c1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:38.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.856+0000 7f6ecaffd700 1 -- 172.21.15.31:0/3363099928 >> v2:172.21.15.62:3300/0 conn(0x7f6ecc15c5e0 msgr2=0x7f6ecc15ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:38.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.856+0000 7f6ecaffd700 1 --2- 172.21.15.31:0/3363099928 >> v2:172.21.15.62:3300/0 conn(0x7f6ecc15c5e0 0x7f6ecc15ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:38.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.857+0000 7f6ecaffd700 1 -- 172.21.15.31:0/3363099928 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ecc0c2370 con 0x7f6ecc0c9170 2023-10-06T02:31:38.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.857+0000 7f6ecaffd700 1 --2- 172.21.15.31:0/3363099928 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ecc0c9170 0x7f6ecc0c1580 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc0b2150 tx=0x7f6ec0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e29e078e5fa97a2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:38.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.857+0000 7f6eca7fc700 1 -- 172.21.15.31:0/3363099928 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ec000b640 con 0x7f6ecc0c9170 2023-10-06T02:31:38.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.857+0000 7f6eca7fc700 1 -- 172.21.15.31:0/3363099928 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ec000b7a0 con 0x7f6ecc0c9170 2023-10-06T02:31:38.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.857+0000 7f6eca7fc700 1 -- 172.21.15.31:0/3363099928 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ec000bb60 con 0x7f6ecc0c9170 2023-10-06T02:31:38.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.858+0000 7f6ed0b9d700 1 -- 172.21.15.31:0/3363099928 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ecc0c9170 msgr2=0x7f6ecc0c1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:38.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.858+0000 7f6ed0b9d700 1 --2- 172.21.15.31:0/3363099928 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ecc0c9170 0x7f6ecc0c1580 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc0b2150 tx=0x7f6ec0009250 comp rx=0 tx=0).stop 2023-10-06T02:31:38.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.858+0000 7f6ed0b9d700 1 -- 172.21.15.31:0/3363099928 shutdown_connections 2023-10-06T02:31:38.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.858+0000 7f6ed0b9d700 1 --2- 172.21.15.31:0/3363099928 >> v2:172.21.15.62:3300/0 conn(0x7f6ecc15c5e0 0x7f6ecc15ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:38.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.858+0000 7f6ed0b9d700 1 --2- 172.21.15.31:0/3363099928 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6ecc0c8da0 0x7f6ecc0c1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:38.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.858+0000 7f6ed0b9d700 1 --2- 172.21.15.31:0/3363099928 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ecc0c9170 0x7f6ecc0c1580 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:38.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.858+0000 7f6ed0b9d700 1 -- 172.21.15.31:0/3363099928 >> 172.21.15.31:0/3363099928 conn(0x7f6ecc06c6a0 msgr2=0x7f6ecc06caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:38.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.858+0000 7f6ed0b9d700 1 -- 172.21.15.31:0/3363099928 shutdown_connections 2023-10-06T02:31:38.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.858+0000 7f6ed0b9d700 1 -- 172.21.15.31:0/3363099928 wait complete. 2023-10-06T02:31:38.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.859+0000 7f6ed0b9d700 1 Processor -- start 2023-10-06T02:31:38.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.859+0000 7f6ed0b9d700 1 -- start start 2023-10-06T02:31:38.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.860+0000 7f6ed0b9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6ecc0c8da0 0x7f6ecc157150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:38.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.860+0000 7f6ed0b9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ecc0c9170 0x7f6ecc15b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:38.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.860+0000 7f6ed0b9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ecc15c5e0 0x7f6ecc15b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:38.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.860+0000 7f6ed0b9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ecc15a3d0 con 0x7f6ecc0c9170 2023-10-06T02:31:38.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.860+0000 7f6ed0b9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ecc159850 con 0x7f6ecc15c5e0 2023-10-06T02:31:38.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.860+0000 7f6ed0b9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ecc159b30 con 0x7f6ecc0c8da0 2023-10-06T02:31:38.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.860+0000 7f6ecaffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ecc0c9170 0x7f6ecc15b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:38.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.860+0000 7f6ecaffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ecc0c9170 0x7f6ecc15b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38808/0 (socket says 172.21.15.31:38808) 2023-10-06T02:31:38.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.860+0000 7f6ecaffd700 1 -- 172.21.15.31:0/197694217 learned_addr learned my addr 172.21.15.31:0/197694217 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:38.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.861+0000 7f6ecb7fe700 1 --2- 172.21.15.31:0/197694217 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6ecc0c8da0 0x7f6ecc157150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:38.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.861+0000 7f6ecbfff700 1 --2- 172.21.15.31:0/197694217 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ecc15c5e0 0x7f6ecc15b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:38.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.861+0000 7f6ecaffd700 1 -- 172.21.15.31:0/197694217 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6ecc0c8da0 msgr2=0x7f6ecc157150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:38.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.861+0000 7f6ecaffd700 1 --2- 172.21.15.31:0/197694217 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6ecc0c8da0 0x7f6ecc157150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:38.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.861+0000 7f6ecaffd700 1 -- 172.21.15.31:0/197694217 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ecc15c5e0 msgr2=0x7f6ecc15b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:38.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.861+0000 7f6ecaffd700 1 --2- 172.21.15.31:0/197694217 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ecc15c5e0 0x7f6ecc15b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:38.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.861+0000 7f6ecaffd700 1 -- 172.21.15.31:0/197694217 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ec00094f0 con 0x7f6ecc0c9170 2023-10-06T02:31:38.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.861+0000 7f6ecbfff700 1 --2- 172.21.15.31:0/197694217 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ecc15c5e0 0x7f6ecc15b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:31:38.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.861+0000 7f6ecaffd700 1 --2- 172.21.15.31:0/197694217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ecc0c9170 0x7f6ecc15b2a0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc069720 tx=0x7f6ec0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:38.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.861+0000 7f6eb37fe700 1 -- 172.21.15.31:0/197694217 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ec0000d10 con 0x7f6ecc0c9170 2023-10-06T02:31:38.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.861+0000 7f6eb37fe700 1 -- 172.21.15.31:0/197694217 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ec000bb20 con 0x7f6ecc0c9170 2023-10-06T02:31:38.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.861+0000 7f6ed0b9d700 1 -- 172.21.15.31:0/197694217 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ecc159e10 con 0x7f6ecc0c9170 2023-10-06T02:31:38.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.861+0000 7f6eb37fe700 1 -- 172.21.15.31:0/197694217 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ec0011690 con 0x7f6ecc0c9170 2023-10-06T02:31:38.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.862+0000 7f6ed0b9d700 1 -- 172.21.15.31:0/197694217 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ecc157690 con 0x7f6ecc0c9170 2023-10-06T02:31:38.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.863+0000 7f6ed0b9d700 1 -- 172.21.15.31:0/197694217 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ecc04c3c0 con 0x7f6ecc0c9170 2023-10-06T02:31:38.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.864+0000 7f6eb37fe700 1 -- 172.21.15.31:0/197694217 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 85403+0+0 (secure 0 0 0) 0x7f6ec0011a00 con 0x7f6ecc0c9170 2023-10-06T02:31:38.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.864+0000 7f6eb37fe700 1 --2- 172.21.15.31:0/197694217 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6eb8067680 0x7f6eb8069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:38.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.865+0000 7f6eb37fe700 1 -- 172.21.15.31:0/197694217 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f6ec0086cf0 con 0x7f6ecc0c9170 2023-10-06T02:31:38.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.865+0000 7f6ecb7fe700 1 --2- 172.21.15.31:0/197694217 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6eb8067680 0x7f6eb8069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:38.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.866+0000 7f6ecb7fe700 1 --2- 172.21.15.31:0/197694217 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6eb8067680 0x7f6eb8069b30 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc06f730 tx=0x7f6ebc0091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:38.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:38.869+0000 7f6eb37fe700 1 -- 172.21.15.31:0/197694217 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6ec00840b0 con 0x7f6ecc0c9170 2023-10-06T02:31:39.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.109+0000 7f6ed0b9d700 1 -- 172.21.15.31:0/197694217 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000024849", "group_name": "group_0000000000109489", "target": ["mon-mgr", ""]}) v1 -- 0x7f6ecc0a9b20 con 0x7f6eb8067680 2023-10-06T02:31:39.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.166+0000 7f6eb37fe700 1 -- 172.21.15.31:0/197694217 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+96 (secure 0 0 0) 0x7f6ecc0a9b20 con 0x7f6eb8067680 2023-10-06T02:31:39.168 INFO:teuthology.orchestra.run.smithi031.stdout:/volumes/group_0000000000109489/subvolume_0000000000024849/06983d16-fa80-447b-835a-845df9d91482 2023-10-06T02:31:39.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.169+0000 7f6ed0b9d700 1 -- 172.21.15.31:0/197694217 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6eb8067680 msgr2=0x7f6eb8069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:39.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.169+0000 7f6ed0b9d700 1 --2- 172.21.15.31:0/197694217 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6eb8067680 0x7f6eb8069b30 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc06f730 tx=0x7f6ebc0091a0 comp rx=0 tx=0).stop 2023-10-06T02:31:39.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.169+0000 7f6ed0b9d700 1 -- 172.21.15.31:0/197694217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ecc0c9170 msgr2=0x7f6ecc15b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:39.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.169+0000 7f6ed0b9d700 1 --2- 172.21.15.31:0/197694217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ecc0c9170 0x7f6ecc15b2a0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc069720 tx=0x7f6ec0017000 comp rx=0 tx=0).stop 2023-10-06T02:31:39.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.169+0000 7f6ed0b9d700 1 -- 172.21.15.31:0/197694217 shutdown_connections 2023-10-06T02:31:39.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.169+0000 7f6ed0b9d700 1 --2- 172.21.15.31:0/197694217 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ecc15c5e0 0x7f6ecc15b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:39.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.169+0000 7f6ed0b9d700 1 --2- 172.21.15.31:0/197694217 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6eb8067680 0x7f6eb8069b30 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:39.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.169+0000 7f6ed0b9d700 1 --2- 172.21.15.31:0/197694217 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6ecc0c8da0 0x7f6ecc157150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:39.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.169+0000 7f6ed0b9d700 1 --2- 172.21.15.31:0/197694217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ecc0c9170 0x7f6ecc15b2a0 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:39.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.169+0000 7f6ed0b9d700 1 -- 172.21.15.31:0/197694217 >> 172.21.15.31:0/197694217 conn(0x7f6ecc06c6a0 msgr2=0x7f6ecc0cb370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:39.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.169+0000 7f6ed0b9d700 1 -- 172.21.15.31:0/197694217 shutdown_connections 2023-10-06T02:31:39.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.169+0000 7f6ed0b9d700 1 -- 172.21.15.31:0/197694217 wait complete. 2023-10-06T02:31:39.188 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:31:39.189 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvolume_0000000000024848/.meta) 2023-10-06T02:31:39.251 INFO:teuthology.orchestra.run.smithi031.stdout:[GLOBAL] 2023-10-06T02:31:39.251 INFO:teuthology.orchestra.run.smithi031.stdout:version = 2 2023-10-06T02:31:39.252 INFO:teuthology.orchestra.run.smithi031.stdout:type = subvolume 2023-10-06T02:31:39.252 INFO:teuthology.orchestra.run.smithi031.stdout:path = /volumes/_nogroup/subvolume_0000000000024848/d345f41e-f146-4ab9-9d43-4c480aff6fd8 2023-10-06T02:31:39.252 INFO:teuthology.orchestra.run.smithi031.stdout:state = complete 2023-10-06T02:31:39.252 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:31:39.254 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:31:39.255 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/group_0000000000109489/subvolume_0000000000024849/.meta) 2023-10-06T02:31:39.317 INFO:teuthology.orchestra.run.smithi031.stdout:[GLOBAL] 2023-10-06T02:31:39.317 INFO:teuthology.orchestra.run.smithi031.stdout:version = 2 2023-10-06T02:31:39.317 INFO:teuthology.orchestra.run.smithi031.stdout:type = subvolume 2023-10-06T02:31:39.317 INFO:teuthology.orchestra.run.smithi031.stdout:path = /volumes/group_0000000000109489/subvolume_0000000000024849/06983d16-fa80-447b-835a-845df9d91482 2023-10-06T02:31:39.318 INFO:teuthology.orchestra.run.smithi031.stdout:state = complete 2023-10-06T02:31:39.318 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:31:39.320 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000024848 2023-10-06T02:31:39.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.508+0000 7fc962f9d700 1 Processor -- start 2023-10-06T02:31:39.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.509+0000 7fc962f9d700 1 -- start start 2023-10-06T02:31:39.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.510+0000 7fc962f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc95c0c7be0 0x7fc95c06abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:39.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.510+0000 7fc962f9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc95c13f780 0x7fc95c13fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:39.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.510+0000 7fc962f9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc95c06b290 con 0x7fc95c0c7fb0 2023-10-06T02:31:39.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.510+0000 7fc962f9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc95c06bf10 con 0x7fc95c13f780 2023-10-06T02:31:39.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.510+0000 7fc962f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc95c140140 con 0x7fc95c0c7be0 2023-10-06T02:31:39.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.510+0000 7fc96279c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc95c13f780 0x7fc95c13fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:39.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.510+0000 7fc96179a700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fc95c0c7fb0 0x7fc95c15a990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:53738/0 (socket says 172.21.15.31:53738) 2023-10-06T02:31:39.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.510+0000 7fc96179a700 1 -- 172.21.15.31:0/1651528519 learned_addr learned my addr 172.21.15.31:0/1651528519 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:31:39.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.511+0000 7fc961f9b700 1 --2- 172.21.15.31:0/1651528519 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc95c0c7be0 0x7fc95c06abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:39.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.511+0000 7fc96279c700 1 -- 172.21.15.31:0/1651528519 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc95c0c7be0 msgr2=0x7fc95c06abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:39.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.511+0000 7fc96279c700 1 --2- 172.21.15.31:0/1651528519 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc95c0c7be0 0x7fc95c06abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:39.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.511+0000 7fc96279c700 1 -- 172.21.15.31:0/1651528519 >> v1:172.21.15.31:6789/0 conn(0x7fc95c0c7fb0 legacy=0x7fc95c15a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:39.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.511+0000 7fc96279c700 1 -- 172.21.15.31:0/1651528519 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc95c06b570 con 0x7fc95c13f780 2023-10-06T02:31:39.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.511+0000 7fc961f9b700 1 --2- 172.21.15.31:0/1651528519 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc95c0c7be0 0x7fc95c06abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:31:39.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.511+0000 7fc960f99700 1 -- 172.21.15.31:0/1651528519 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1104165046 0 0) 0x7fc95c06b290 con 0x7fc95c0c7fb0 2023-10-06T02:31:39.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.511+0000 7fc96279c700 1 --2- 172.21.15.31:0/1651528519 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc95c13f780 0x7fc95c13fc00 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7fc954009e90 tx=0x7fc954009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6c1f4a9f4466f62 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:39.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.511+0000 7fc960f99700 1 -- 172.21.15.31:0/1651528519 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc95400e070 con 0x7fc95c13f780 2023-10-06T02:31:39.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.511+0000 7fc960f99700 1 -- 172.21.15.31:0/1651528519 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc95400bbf0 con 0x7fc95c13f780 2023-10-06T02:31:39.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.511+0000 7fc960f99700 1 -- 172.21.15.31:0/1651528519 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc954012430 con 0x7fc95c13f780 2023-10-06T02:31:39.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.512+0000 7fc962f9d700 1 -- 172.21.15.31:0/1651528519 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc95c13f780 msgr2=0x7fc95c13fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:39.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.512+0000 7fc962f9d700 1 --2- 172.21.15.31:0/1651528519 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc95c13f780 0x7fc95c13fc00 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7fc954009e90 tx=0x7fc954009780 comp rx=0 tx=0).stop 2023-10-06T02:31:39.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.512+0000 7fc962f9d700 1 -- 172.21.15.31:0/1651528519 shutdown_connections 2023-10-06T02:31:39.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.512+0000 7fc962f9d700 1 --2- 172.21.15.31:0/1651528519 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc95c13f780 0x7fc95c13fc00 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:39.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.512+0000 7fc962f9d700 1 --2- 172.21.15.31:0/1651528519 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc95c0c7be0 0x7fc95c06abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:39.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.512+0000 7fc962f9d700 1 -- 172.21.15.31:0/1651528519 >> 172.21.15.31:0/1651528519 conn(0x7fc95c06d110 msgr2=0x7fc95c0cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:39.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.512+0000 7fc962f9d700 1 -- 172.21.15.31:0/1651528519 shutdown_connections 2023-10-06T02:31:39.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.512+0000 7fc962f9d700 1 -- 172.21.15.31:0/1651528519 wait complete. 2023-10-06T02:31:39.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.513+0000 7fc962f9d700 1 Processor -- start 2023-10-06T02:31:39.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.514+0000 7fc962f9d700 1 -- start start 2023-10-06T02:31:39.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.514+0000 7fc962f9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc95c0c7be0 0x7fc95c0c31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:39.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.514+0000 7fc962f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc95c0c7fb0 0x7fc95c13e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:39.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.515+0000 7fc962f9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc95c13f780 0x7fc95c1558a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:39.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.515+0000 7fc962f9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc95c0c5e70 con 0x7fc95c0c7be0 2023-10-06T02:31:39.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.515+0000 7fc962f9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc95c0c3720 con 0x7fc95c13f780 2023-10-06T02:31:39.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.515+0000 7fc962f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc95c0c5260 con 0x7fc95c0c7fb0 2023-10-06T02:31:39.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.515+0000 7fc961f9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc95c0c7be0 0x7fc95c0c31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:39.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.515+0000 7fc961f9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc95c0c7be0 0x7fc95c0c31e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38826/0 (socket says 172.21.15.31:38826) 2023-10-06T02:31:39.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.515+0000 7fc961f9b700 1 -- 172.21.15.31:0/3682078935 learned_addr learned my addr 172.21.15.31:0/3682078935 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:39.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.515+0000 7fc96179a700 1 --2- 172.21.15.31:0/3682078935 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc95c0c7fb0 0x7fc95c13e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:39.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.515+0000 7fc96279c700 1 --2- 172.21.15.31:0/3682078935 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc95c13f780 0x7fc95c1558a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:39.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.515+0000 7fc961f9b700 1 -- 172.21.15.31:0/3682078935 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc95c0c7fb0 msgr2=0x7fc95c13e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:39.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.515+0000 7fc961f9b700 1 --2- 172.21.15.31:0/3682078935 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc95c0c7fb0 0x7fc95c13e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:39.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.515+0000 7fc961f9b700 1 -- 172.21.15.31:0/3682078935 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc95c13f780 msgr2=0x7fc95c1558a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:39.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.515+0000 7fc961f9b700 1 --2- 172.21.15.31:0/3682078935 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc95c13f780 0x7fc95c1558a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:39.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.515+0000 7fc961f9b700 1 -- 172.21.15.31:0/3682078935 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc954009a20 con 0x7fc95c0c7be0 2023-10-06T02:31:39.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.515+0000 7fc961f9b700 1 --2- 172.21.15.31:0/3682078935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc95c0c7be0 0x7fc95c0c31e0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7fc95800e8d0 tx=0x7fc958008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:39.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.516+0000 7fc951ffb700 1 -- 172.21.15.31:0/3682078935 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc95800ee70 con 0x7fc95c0c7be0 2023-10-06T02:31:39.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.516+0000 7fc951ffb700 1 -- 172.21.15.31:0/3682078935 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc95800c5e0 con 0x7fc95c0c7be0 2023-10-06T02:31:39.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.516+0000 7fc951ffb700 1 -- 172.21.15.31:0/3682078935 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc95800c980 con 0x7fc95c0c7be0 2023-10-06T02:31:39.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.516+0000 7fc962f9d700 1 -- 172.21.15.31:0/3682078935 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc95c0c5540 con 0x7fc95c0c7be0 2023-10-06T02:31:39.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.516+0000 7fc962f9d700 1 -- 172.21.15.31:0/3682078935 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc95c0c5b00 con 0x7fc95c0c7be0 2023-10-06T02:31:39.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.517+0000 7fc962f9d700 1 -- 172.21.15.31:0/3682078935 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc95c04c3c0 con 0x7fc95c0c7be0 2023-10-06T02:31:39.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.518+0000 7fc951ffb700 1 -- 172.21.15.31:0/3682078935 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 85403+0+0 (secure 0 0 0) 0x7fc958013410 con 0x7fc95c0c7be0 2023-10-06T02:31:39.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.519+0000 7fc951ffb700 1 --2- 172.21.15.31:0/3682078935 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc940067680 0x7fc940069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:39.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.519+0000 7fc951ffb700 1 -- 172.21.15.31:0/3682078935 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fc958085830 con 0x7fc95c0c7be0 2023-10-06T02:31:39.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.519+0000 7fc96179a700 1 --2- 172.21.15.31:0/3682078935 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc940067680 0x7fc940069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:39.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.520+0000 7fc96179a700 1 --2- 172.21.15.31:0/3682078935 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc940067680 0x7fc940069b30 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fc95c0b20c0 tx=0x7fc94c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:39.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.523+0000 7fc951ffb700 1 -- 172.21.15.31:0/3682078935 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc958082ae0 con 0x7fc95c0c7be0 2023-10-06T02:31:39.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.764+0000 7fc962f9d700 1 -- 172.21.15.31:0/3682078935 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000024848", "target": ["mon-mgr", ""]}) v1 -- 0x7fc95c0c5fb0 con 0x7fc940067680 2023-10-06T02:31:39.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.783+0000 7fc951ffb700 1 -- 172.21.15.31:0/3682078935 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fc94c0014f0 con 0x7fc940067680 2023-10-06T02:31:39.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.785+0000 7fc962f9d700 1 -- 172.21.15.31:0/3682078935 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc940067680 msgr2=0x7fc940069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:39.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.785+0000 7fc962f9d700 1 --2- 172.21.15.31:0/3682078935 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc940067680 0x7fc940069b30 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fc95c0b20c0 tx=0x7fc94c009040 comp rx=0 tx=0).stop 2023-10-06T02:31:39.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.785+0000 7fc962f9d700 1 -- 172.21.15.31:0/3682078935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc95c0c7be0 msgr2=0x7fc95c0c31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:39.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.785+0000 7fc962f9d700 1 --2- 172.21.15.31:0/3682078935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc95c0c7be0 0x7fc95c0c31e0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7fc95800e8d0 tx=0x7fc958008040 comp rx=0 tx=0).stop 2023-10-06T02:31:39.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.785+0000 7fc962f9d700 1 -- 172.21.15.31:0/3682078935 shutdown_connections 2023-10-06T02:31:39.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.785+0000 7fc962f9d700 1 --2- 172.21.15.31:0/3682078935 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc95c13f780 0x7fc95c1558a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:39.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.785+0000 7fc962f9d700 1 --2- 172.21.15.31:0/3682078935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc95c0c7be0 0x7fc95c0c31e0 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:39.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.785+0000 7fc962f9d700 1 --2- 172.21.15.31:0/3682078935 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc940067680 0x7fc940069b30 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:39.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.785+0000 7fc962f9d700 1 --2- 172.21.15.31:0/3682078935 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc95c0c7fb0 0x7fc95c13e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:39.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.785+0000 7fc962f9d700 1 -- 172.21.15.31:0/3682078935 >> 172.21.15.31:0/3682078935 conn(0x7fc95c06d110 msgr2=0x7fc95c0ca650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:39.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.785+0000 7fc962f9d700 1 -- 172.21.15.31:0/3682078935 shutdown_connections 2023-10-06T02:31:39.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.785+0000 7fc962f9d700 1 -- 172.21.15.31:0/3682078935 wait complete. 2023-10-06T02:31:39.791 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:31:39.791+0000 7f6567b82700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:31:39.792 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:31:39.791+0000 7f6567b82700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:31:39.792 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:31:39.791+0000 7f6567b82700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:31:39.792 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:31:39.791+0000 7f6567b82700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:31:39.792 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:31:39.791+0000 7f6567b82700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:31:39.804 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000024849 group_0000000000109489 2023-10-06T02:31:39.809 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:31:39.808+0000 7f6567381700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:31:39.809 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:31:39.808+0000 7f6567381700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:31:39.810 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:31:39.808+0000 7f6567381700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:31:39.810 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:31:39.808+0000 7f6567381700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:31:39.810 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:31:39.808+0000 7f6567381700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:31:39.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.993+0000 7f8a5b19d700 1 Processor -- start 2023-10-06T02:31:39.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.994+0000 7f8a5b19d700 1 -- start start 2023-10-06T02:31:39.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.995+0000 7f8a5b19d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8a540c8200 0x7f8a540c1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:39.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.995+0000 7f8a5b19d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8a540c2a90 0x7f8a54156aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:39.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.995+0000 7f8a5b19d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a541579b0 con 0x7f8a540c85d0 2023-10-06T02:31:39.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.995+0000 7f8a5b19d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a540c1390 con 0x7f8a540c8200 2023-10-06T02:31:39.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.995+0000 7f8a5b19d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a541573f0 con 0x7f8a540c2a90 2023-10-06T02:31:39.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.995+0000 7f8a5a99c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f8a540c85d0 0x7f8a540cc2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:53750/0 (socket says 172.21.15.31:53750) 2023-10-06T02:31:39.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.995+0000 7f8a5a19b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8a540c8200 0x7f8a540c1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:39.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.995+0000 7f8a5a99c700 1 -- 172.21.15.31:0/1188563608 learned_addr learned my addr 172.21.15.31:0/1188563608 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:31:39.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.996+0000 7f8a5999a700 1 --2- 172.21.15.31:0/1188563608 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8a540c2a90 0x7f8a54156aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:39.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.996+0000 7f8a5a19b700 1 -- 172.21.15.31:0/1188563608 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8a540c2a90 msgr2=0x7f8a54156aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:39.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.996+0000 7f8a59199700 1 -- 172.21.15.31:0/1188563608 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2793661794 0 0) 0x7f8a541579b0 con 0x7f8a540c85d0 2023-10-06T02:31:39.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.996+0000 7f8a5a19b700 1 --2- 172.21.15.31:0/1188563608 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8a540c2a90 0x7f8a54156aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:39.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.996+0000 7f8a5a19b700 1 -- 172.21.15.31:0/1188563608 >> v1:172.21.15.31:6789/0 conn(0x7f8a540c85d0 legacy=0x7f8a540cc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:39.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.996+0000 7f8a5a19b700 1 -- 172.21.15.31:0/1188563608 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a541576d0 con 0x7f8a540c8200 2023-10-06T02:31:39.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.996+0000 7f8a5a19b700 1 --2- 172.21.15.31:0/1188563608 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a540c8200 0x7f8a540c1820 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f8a500098e0 tx=0x7f8a500091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=468020c3f1cf1392 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:39.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.996+0000 7f8a59199700 1 -- 172.21.15.31:0/1188563608 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a50009e80 con 0x7f8a540c8200 2023-10-06T02:31:39.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.996+0000 7f8a59199700 1 -- 172.21.15.31:0/1188563608 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a5000b860 con 0x7f8a540c8200 2023-10-06T02:31:40.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.996+0000 7f8a59199700 1 -- 172.21.15.31:0/1188563608 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a5000bbe0 con 0x7f8a540c8200 2023-10-06T02:31:40.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.997+0000 7f8a5b19d700 1 -- 172.21.15.31:0/1188563608 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a540c8200 msgr2=0x7f8a540c1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:40.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.997+0000 7f8a5b19d700 1 --2- 172.21.15.31:0/1188563608 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a540c8200 0x7f8a540c1820 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f8a500098e0 tx=0x7f8a500091a0 comp rx=0 tx=0).stop 2023-10-06T02:31:40.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.997+0000 7f8a5b19d700 1 -- 172.21.15.31:0/1188563608 shutdown_connections 2023-10-06T02:31:40.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.997+0000 7f8a5b19d700 1 --2- 172.21.15.31:0/1188563608 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a540c8200 0x7f8a540c1820 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:40.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.997+0000 7f8a5b19d700 1 --2- 172.21.15.31:0/1188563608 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8a540c2a90 0x7f8a54156aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:40.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.997+0000 7f8a5b19d700 1 -- 172.21.15.31:0/1188563608 >> 172.21.15.31:0/1188563608 conn(0x7f8a5406c6a0 msgr2=0x7f8a5406caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:40.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.997+0000 7f8a5b19d700 1 -- 172.21.15.31:0/1188563608 shutdown_connections 2023-10-06T02:31:40.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.997+0000 7f8a5b19d700 1 -- 172.21.15.31:0/1188563608 wait complete. 2023-10-06T02:31:40.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.998+0000 7f8a5b19d700 1 Processor -- start 2023-10-06T02:31:40.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.998+0000 7f8a5b19d700 1 -- start start 2023-10-06T02:31:40.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.999+0000 7f8a5b19d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a540c2a90 0x7f8a5414dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:40.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.999+0000 7f8a5b19d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8a540c8200 0x7f8a540ad160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:40.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.999+0000 7f8a5b19d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a540c85d0 0x7f8a5406b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:40.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.999+0000 7f8a5b19d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a540adc90 con 0x7f8a540c2a90 2023-10-06T02:31:40.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.999+0000 7f8a5b19d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a5414e170 con 0x7f8a540c85d0 2023-10-06T02:31:40.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:39.999+0000 7f8a5b19d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a5414f680 con 0x7f8a540c8200 2023-10-06T02:31:40.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:40.000+0000 7f8a5a19b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a540c2a90 0x7f8a5414dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:40.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:40.000+0000 7f8a5a19b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a540c2a90 0x7f8a5414dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38836/0 (socket says 172.21.15.31:38836) 2023-10-06T02:31:40.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:40.000+0000 7f8a5a19b700 1 -- 172.21.15.31:0/3517688467 learned_addr learned my addr 172.21.15.31:0/3517688467 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:40.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:40.000+0000 7f8a5999a700 1 --2- 172.21.15.31:0/3517688467 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8a540c8200 0x7f8a540ad160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:40.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:40.000+0000 7f8a5a99c700 1 --2- 172.21.15.31:0/3517688467 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a540c85d0 0x7f8a5406b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:40.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:40.000+0000 7f8a5a19b700 1 -- 172.21.15.31:0/3517688467 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8a540c8200 msgr2=0x7f8a540ad160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:40.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:40.000+0000 7f8a5a19b700 1 --2- 172.21.15.31:0/3517688467 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8a540c8200 0x7f8a540ad160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:40.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:40.000+0000 7f8a5a19b700 1 -- 172.21.15.31:0/3517688467 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a540c85d0 msgr2=0x7f8a5406b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:40.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:40.000+0000 7f8a5a19b700 1 --2- 172.21.15.31:0/3517688467 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a540c85d0 0x7f8a5406b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:40.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:40.000+0000 7f8a5a19b700 1 -- 172.21.15.31:0/3517688467 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a50009420 con 0x7f8a540c2a90 2023-10-06T02:31:40.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:40.000+0000 7f8a5a99c700 1 --2- 172.21.15.31:0/3517688467 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a540c85d0 0x7f8a5406b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:31:40.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:40.000+0000 7f8a5a19b700 1 --2- 172.21.15.31:0/3517688467 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a540c2a90 0x7f8a5414dc30 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f8a540697c0 tx=0x7f8a50017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:40.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:40.001+0000 7f8a4a7fc700 1 -- 172.21.15.31:0/3517688467 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a5000bbe0 con 0x7f8a540c2a90 2023-10-06T02:31:40.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:40.001+0000 7f8a4a7fc700 1 -- 172.21.15.31:0/3517688467 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a50011420 con 0x7f8a540c2a90 2023-10-06T02:31:40.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:40.001+0000 7f8a5b19d700 1 -- 172.21.15.31:0/3517688467 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a540ae650 con 0x7f8a540c2a90 2023-10-06T02:31:40.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:40.001+0000 7f8a4a7fc700 1 -- 172.21.15.31:0/3517688467 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a500117e0 con 0x7f8a540c2a90 2023-10-06T02:31:40.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:40.001+0000 7f8a5b19d700 1 -- 172.21.15.31:0/3517688467 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a540ae000 con 0x7f8a540c2a90 2023-10-06T02:31:40.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:40.002+0000 7f8a5b19d700 1 -- 172.21.15.31:0/3517688467 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a5404c3c0 con 0x7f8a540c2a90 2023-10-06T02:31:40.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:40.003+0000 7f8a4a7fc700 1 -- 172.21.15.31:0/3517688467 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 85403+0+0 (secure 0 0 0) 0x7f8a50011b90 con 0x7f8a540c2a90 2023-10-06T02:31:40.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:40.003+0000 7f8a4a7fc700 1 --2- 172.21.15.31:0/3517688467 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f8a38067680 0x7f8a38069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:40.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:40.003+0000 7f8a4a7fc700 1 -- 172.21.15.31:0/3517688467 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6173+0+0 (secure 0 0 0) 0x7f8a50087210 con 0x7f8a540c2a90 2023-10-06T02:31:40.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:40.004+0000 7f8a5999a700 1 --2- 172.21.15.31:0/3517688467 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f8a38067680 0x7f8a38069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:40.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:40.004+0000 7f8a5999a700 1 --2- 172.21.15.31:0/3517688467 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f8a38067680 0x7f8a38069b30 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f8a540cb1f0 tx=0x7f8a44008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:40.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:40.008+0000 7f8a4a7fc700 1 -- 172.21.15.31:0/3517688467 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8a50084580 con 0x7f8a540c2a90 2023-10-06T02:31:40.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:40.242+0000 7f8a5b19d700 1 -- 172.21.15.31:0/3517688467 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000024849", "group_name": "group_0000000000109489", "target": ["mon-mgr", ""]}) v1 -- 0x7f8a540681b0 con 0x7f8a38067680 2023-10-06T02:31:40.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:40.603+0000 7f8a4a7fc700 1 -- 172.21.15.31:0/3517688467 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mgrmap(e 25) v1 ==== 85556+0+0 (secure 0 0 0) 0x7f8a50053740 con 0x7f8a540c2a90 2023-10-06T02:31:49.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:49.539+0000 7f8a4a7fc700 1 -- 172.21.15.31:0/3517688467 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f8a540681b0 con 0x7f8a38067680 2023-10-06T02:31:49.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:49.542+0000 7f8a5b19d700 1 -- 172.21.15.31:0/3517688467 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f8a38067680 msgr2=0x7f8a38069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:49.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:49.542+0000 7f8a5b19d700 1 --2- 172.21.15.31:0/3517688467 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f8a38067680 0x7f8a38069b30 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f8a540cb1f0 tx=0x7f8a44008040 comp rx=0 tx=0).stop 2023-10-06T02:31:49.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:49.542+0000 7f8a5b19d700 1 -- 172.21.15.31:0/3517688467 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a540c2a90 msgr2=0x7f8a5414dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:49.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:49.542+0000 7f8a5b19d700 1 --2- 172.21.15.31:0/3517688467 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a540c2a90 0x7f8a5414dc30 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f8a540697c0 tx=0x7f8a50017000 comp rx=0 tx=0).stop 2023-10-06T02:31:49.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:49.542+0000 7f8a5b19d700 1 -- 172.21.15.31:0/3517688467 shutdown_connections 2023-10-06T02:31:49.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:49.542+0000 7f8a5b19d700 1 --2- 172.21.15.31:0/3517688467 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a540c85d0 0x7f8a5406b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:49.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:49.542+0000 7f8a5b19d700 1 --2- 172.21.15.31:0/3517688467 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a540c2a90 0x7f8a5414dc30 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:49.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:49.542+0000 7f8a5b19d700 1 --2- 172.21.15.31:0/3517688467 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f8a38067680 0x7f8a38069b30 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:49.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:49.542+0000 7f8a5b19d700 1 --2- 172.21.15.31:0/3517688467 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8a540c8200 0x7f8a540ad160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:49.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:49.542+0000 7f8a5b19d700 1 -- 172.21.15.31:0/3517688467 >> 172.21.15.31:0/3517688467 conn(0x7f8a5406c6a0 msgr2=0x7f8a540c9510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:49.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:49.542+0000 7f8a5b19d700 1 -- 172.21.15.31:0/3517688467 shutdown_connections 2023-10-06T02:31:49.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:49.542+0000 7f8a5b19d700 1 -- 172.21.15.31:0/3517688467 wait complete. 2023-10-06T02:31:49.566 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:31:49.566 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2023-10-06T02:31:49.648 INFO:teuthology.orchestra.run.smithi031.stdout:2 2023-10-06T02:31:49.649 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T02:31:49.649 DEBUG:teuthology.orchestra.run.smithi031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T02:31:49.742 DEBUG:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T02:31:49.830 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T02:31:49.831 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T02:31:49.831 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:31:49.831 DEBUG:teuthology.orchestra.run.smithi031:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T02:31:49.938 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi031.front.sepia.ceph.com 2023-10-06T02:31:49.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:31:49.939 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T02:31:49.981 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-06T02:31:49.981 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:31:49.981 DEBUG:teuthology.orchestra.run.smithi062:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-06T02:31:50.067 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi062.front.sepia.ceph.com 2023-10-06T02:31:50.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:31:50.067 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T02:31:50.105 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:31:50.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.295+0000 7ff2d999d700 1 Processor -- start 2023-10-06T02:31:50.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.295+0000 7ff2d999d700 1 -- start start 2023-10-06T02:31:50.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.297+0000 7ff2d999d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff2d413fb20 0x7ff2d415a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:50.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.297+0000 7ff2d999d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff2d413fef0 0x7ff2d40c7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:50.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.297+0000 7ff2d999d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff2d406aaa0 0x7ff2d40c7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:50.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.297+0000 7ff2d999d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2d40c8410 con 0x7ff2d406aaa0 2023-10-06T02:31:50.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.297+0000 7ff2d999d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2d415ae70 con 0x7ff2d413fb20 2023-10-06T02:31:50.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.297+0000 7ff2d999d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d406af60 con 0x7ff2d413fef0 2023-10-06T02:31:50.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.297+0000 7ff2d919c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff2d406aaa0 0x7ff2d40c7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:50.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.297+0000 7ff2d919c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff2d406aaa0 0x7ff2d40c7ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56730/0 (socket says 172.21.15.31:56730) 2023-10-06T02:31:50.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.297+0000 7ff2d919c700 1 -- 172.21.15.31:0/3249080465 learned_addr learned my addr 172.21.15.31:0/3249080465 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:50.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.297+0000 7ff2d899b700 1 --2- 172.21.15.31:0/3249080465 >> v2:172.21.15.62:3300/0 conn(0x7ff2d413fb20 0x7ff2d415a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:50.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.297+0000 7ff2d3fff700 1 --2- 172.21.15.31:0/3249080465 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff2d413fef0 0x7ff2d40c7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:50.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.297+0000 7ff2d919c700 1 -- 172.21.15.31:0/3249080465 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff2d413fef0 msgr2=0x7ff2d40c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:50.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.297+0000 7ff2d919c700 1 --2- 172.21.15.31:0/3249080465 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff2d413fef0 0x7ff2d40c7990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:50.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.297+0000 7ff2d919c700 1 -- 172.21.15.31:0/3249080465 >> v2:172.21.15.62:3300/0 conn(0x7ff2d413fb20 msgr2=0x7ff2d415a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:50.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.297+0000 7ff2d919c700 1 --2- 172.21.15.31:0/3249080465 >> v2:172.21.15.62:3300/0 conn(0x7ff2d413fb20 0x7ff2d415a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:50.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.297+0000 7ff2d919c700 1 -- 172.21.15.31:0/3249080465 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2d406b240 con 0x7ff2d406aaa0 2023-10-06T02:31:50.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.298+0000 7ff2d919c700 1 --2- 172.21.15.31:0/3249080465 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2d406aaa0 0x7ff2d40c7ed0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7ff2d40695b0 tx=0x7ff2cc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da3c0912ce02c8d6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:50.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.298+0000 7ff2d37fe700 1 -- 172.21.15.31:0/3249080465 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2cc00e070 con 0x7ff2d406aaa0 2023-10-06T02:31:50.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.298+0000 7ff2d37fe700 1 -- 172.21.15.31:0/3249080465 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2cc00b970 con 0x7ff2d406aaa0 2023-10-06T02:31:50.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.298+0000 7ff2d37fe700 1 -- 172.21.15.31:0/3249080465 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2cc0092e0 con 0x7ff2d406aaa0 2023-10-06T02:31:50.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.298+0000 7ff2d999d700 1 -- 172.21.15.31:0/3249080465 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2d406aaa0 msgr2=0x7ff2d40c7ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:50.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.298+0000 7ff2d999d700 1 --2- 172.21.15.31:0/3249080465 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2d406aaa0 0x7ff2d40c7ed0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7ff2d40695b0 tx=0x7ff2cc009840 comp rx=0 tx=0).stop 2023-10-06T02:31:50.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.299+0000 7ff2d999d700 1 -- 172.21.15.31:0/3249080465 shutdown_connections 2023-10-06T02:31:50.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.299+0000 7ff2d999d700 1 --2- 172.21.15.31:0/3249080465 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2d406aaa0 0x7ff2d40c7ed0 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:50.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.299+0000 7ff2d999d700 1 --2- 172.21.15.31:0/3249080465 >> v2:172.21.15.62:3300/0 conn(0x7ff2d413fb20 0x7ff2d415a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:50.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.299+0000 7ff2d999d700 1 --2- 172.21.15.31:0/3249080465 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff2d413fef0 0x7ff2d40c7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:50.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.299+0000 7ff2d999d700 1 -- 172.21.15.31:0/3249080465 >> 172.21.15.31:0/3249080465 conn(0x7ff2d406d170 msgr2=0x7ff2d40cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:50.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.299+0000 7ff2d999d700 1 -- 172.21.15.31:0/3249080465 shutdown_connections 2023-10-06T02:31:50.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.299+0000 7ff2d999d700 1 -- 172.21.15.31:0/3249080465 wait complete. 2023-10-06T02:31:50.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.300+0000 7ff2d999d700 1 Processor -- start 2023-10-06T02:31:50.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.300+0000 7ff2d999d700 1 -- start start 2023-10-06T02:31:50.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.301+0000 7ff2d999d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2d406aaa0 0x7ff2d4155bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:50.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.301+0000 7ff2d999d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff2d413fb20 0x7ff2d415b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:50.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.301+0000 7ff2d999d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2d413fef0 0x7ff2d4158d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:50.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.301+0000 7ff2d999d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d4157060 con 0x7ff2d406aaa0 2023-10-06T02:31:50.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.301+0000 7ff2d999d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d4157800 con 0x7ff2d413fef0 2023-10-06T02:31:50.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.301+0000 7ff2d999d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d4157240 con 0x7ff2d413fb20 2023-10-06T02:31:50.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.301+0000 7ff2d899b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2d406aaa0 0x7ff2d4155bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:50.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.301+0000 7ff2d899b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2d406aaa0 0x7ff2d4155bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56736/0 (socket says 172.21.15.31:56736) 2023-10-06T02:31:50.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.301+0000 7ff2d899b700 1 -- 172.21.15.31:0/557988296 learned_addr learned my addr 172.21.15.31:0/557988296 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:50.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.302+0000 7ff2d3fff700 1 --2- 172.21.15.31:0/557988296 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff2d413fb20 0x7ff2d415b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:50.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.302+0000 7ff2d919c700 1 --2- 172.21.15.31:0/557988296 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2d413fef0 0x7ff2d4158d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:50.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.302+0000 7ff2d899b700 1 -- 172.21.15.31:0/557988296 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff2d413fb20 msgr2=0x7ff2d415b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:50.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.302+0000 7ff2d899b700 1 --2- 172.21.15.31:0/557988296 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff2d413fb20 0x7ff2d415b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:50.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.302+0000 7ff2d899b700 1 -- 172.21.15.31:0/557988296 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2d413fef0 msgr2=0x7ff2d4158d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:50.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.302+0000 7ff2d899b700 1 --2- 172.21.15.31:0/557988296 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2d413fef0 0x7ff2d4158d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:50.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.302+0000 7ff2d899b700 1 -- 172.21.15.31:0/557988296 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2cc009ae0 con 0x7ff2d406aaa0 2023-10-06T02:31:50.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.302+0000 7ff2d899b700 1 --2- 172.21.15.31:0/557988296 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2d406aaa0 0x7ff2d4155bc0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7ff2c800ecb0 tx=0x7ff2c80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:50.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.302+0000 7ff2d0ff9700 1 -- 172.21.15.31:0/557988296 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c800c850 con 0x7ff2d406aaa0 2023-10-06T02:31:50.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.302+0000 7ff2d0ff9700 1 -- 172.21.15.31:0/557988296 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2c800c9b0 con 0x7ff2d406aaa0 2023-10-06T02:31:50.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.302+0000 7ff2d999d700 1 -- 172.21.15.31:0/557988296 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2d4157520 con 0x7ff2d406aaa0 2023-10-06T02:31:50.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.302+0000 7ff2d0ff9700 1 -- 172.21.15.31:0/557988296 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c800cd50 con 0x7ff2d406aaa0 2023-10-06T02:31:50.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.303+0000 7ff2d999d700 1 -- 172.21.15.31:0/557988296 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2d4158870 con 0x7ff2d406aaa0 2023-10-06T02:31:50.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.304+0000 7ff2d999d700 1 -- 172.21.15.31:0/557988296 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2d4158b60 con 0x7ff2d406aaa0 2023-10-06T02:31:50.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.304+0000 7ff2d0ff9700 1 -- 172.21.15.31:0/557988296 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85556+0+0 (secure 0 0 0) 0x7ff2c8013680 con 0x7ff2d406aaa0 2023-10-06T02:31:50.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.305+0000 7ff2d0ff9700 1 --2- 172.21.15.31:0/557988296 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff2b8067830 0x7ff2b8069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:50.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.305+0000 7ff2d0ff9700 1 -- 172.21.15.31:0/557988296 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6173+0+0 (secure 0 0 0) 0x7ff2c80832e0 con 0x7ff2d406aaa0 2023-10-06T02:31:50.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.306+0000 7ff2d3fff700 1 --2- 172.21.15.31:0/557988296 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff2b8067830 0x7ff2b8069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:50.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.306+0000 7ff2d3fff700 1 --2- 172.21.15.31:0/557988296 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff2b8067830 0x7ff2b8069ce0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff2d40b1bd0 tx=0x7ff2c0009120 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:50.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.310+0000 7ff2d0ff9700 1 -- 172.21.15.31:0/557988296 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff2c8051d30 con 0x7ff2d406aaa0 2023-10-06T02:31:50.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.556+0000 7ff2d999d700 1 -- 172.21.15.31:0/557988296 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff2d4159ca0 con 0x7ff2d406aaa0 2023-10-06T02:31:50.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.556+0000 7ff2d0ff9700 1 -- 172.21.15.31:0/557988296 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 110 v110) v1 ==== 95+0+4604 (secure 0 0 0) 0x7ff2d4159ca0 con 0x7ff2d406aaa0 2023-10-06T02:31:50.559 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:31:50.559 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":110,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7178,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/3218480514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3218480514},{"type":"v1","addr":"172.21.15.62:6834","nonce":3218480514}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104},{"gid":8323,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6833/832192876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":832192876},{"type":"v1","addr":"172.21.15.31:6833","nonce":832192876}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104},{"gid":8877,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/2374603733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":2374603733},{"type":"v1","addr":"172.21.15.31:6835","nonce":2374603733}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104}],"filesystems":[{"mdsmap":{"epoch":109,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:31:25.755801+0000","modified":"2023-10-06T02:31:29.266902+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8889},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8889":{"gid":8889,"name":"d","rank":0,"incarnation":105,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/1184425321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":1184425321},{"type":"v1","addr":"172.21.15.62:6835","nonce":1184425321}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2023-10-06T02:31:50.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.559+0000 7ff2d999d700 1 -- 172.21.15.31:0/557988296 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff2b8067830 msgr2=0x7ff2b8069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:50.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.559+0000 7ff2d999d700 1 --2- 172.21.15.31:0/557988296 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff2b8067830 0x7ff2b8069ce0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff2d40b1bd0 tx=0x7ff2c0009120 comp rx=0 tx=0).stop 2023-10-06T02:31:50.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.559+0000 7ff2d999d700 1 -- 172.21.15.31:0/557988296 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2d406aaa0 msgr2=0x7ff2d4155bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:50.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.559+0000 7ff2d999d700 1 --2- 172.21.15.31:0/557988296 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2d406aaa0 0x7ff2d4155bc0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7ff2c800ecb0 tx=0x7ff2c80091a0 comp rx=0 tx=0).stop 2023-10-06T02:31:50.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.559+0000 7ff2d999d700 1 -- 172.21.15.31:0/557988296 shutdown_connections 2023-10-06T02:31:50.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.559+0000 7ff2d999d700 1 --2- 172.21.15.31:0/557988296 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2d413fef0 0x7ff2d4158d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:50.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.559+0000 7ff2d999d700 1 --2- 172.21.15.31:0/557988296 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2d406aaa0 0x7ff2d4155bc0 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:50.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.559+0000 7ff2d999d700 1 --2- 172.21.15.31:0/557988296 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff2b8067830 0x7ff2b8069ce0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:50.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.559+0000 7ff2d999d700 1 --2- 172.21.15.31:0/557988296 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff2d413fb20 0x7ff2d415b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:50.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.559+0000 7ff2d999d700 1 -- 172.21.15.31:0/557988296 >> 172.21.15.31:0/557988296 conn(0x7ff2d406d170 msgr2=0x7ff2d40c0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:50.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.559+0000 7ff2d999d700 1 -- 172.21.15.31:0/557988296 shutdown_connections 2023-10-06T02:31:50.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.559+0000 7ff2d999d700 1 -- 172.21.15.31:0/557988296 wait complete. 2023-10-06T02:31:50.562 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 110 2023-10-06T02:31:50.578 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:31:50.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.768+0000 7fc16af9d700 1 Processor -- start 2023-10-06T02:31:50.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.768+0000 7fc16af9d700 1 -- start start 2023-10-06T02:31:50.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.770+0000 7fc16af9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc164159920 0x7fc1640ad090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:50.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.770+0000 7fc16af9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1640acc00 con 0x7fc16415dfd0 2023-10-06T02:31:50.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.770+0000 7fc16af9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1640abff0 con 0x7fc164159550 2023-10-06T02:31:50.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.770+0000 7fc16af9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1640ac2d0 con 0x7fc164159920 2023-10-06T02:31:50.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.770+0000 7fc16979a700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fc16415dfd0 0x7fc1640aedb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46500/0 (socket says 172.21.15.31:46500) 2023-10-06T02:31:50.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.770+0000 7fc16979a700 1 -- 172.21.15.31:0/88431066 learned_addr learned my addr 172.21.15.31:0/88431066 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:31:50.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.770+0000 7fc16a79c700 1 --2- 172.21.15.31:0/88431066 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc164159920 0x7fc1640ad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:50.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.770+0000 7fc168f99700 1 -- 172.21.15.31:0/88431066 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 928770421 0 0) 0x7fc1640acc00 con 0x7fc16415dfd0 2023-10-06T02:31:50.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.770+0000 7fc168f99700 1 -- 172.21.15.31:0/88431066 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc150003570 con 0x7fc16415dfd0 2023-10-06T02:31:50.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.770+0000 7fc168f99700 1 -- 172.21.15.31:0/88431066 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 206318937 0 0) 0x7fc1640abff0 con 0x7fc164159550 2023-10-06T02:31:50.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.770+0000 7fc168f99700 1 -- 172.21.15.31:0/88431066 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1640acc00 con 0x7fc164159550 2023-10-06T02:31:50.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.771+0000 7fc168f99700 1 -- 172.21.15.31:0/88431066 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 994066982 0 0) 0x7fc150003570 con 0x7fc16415dfd0 2023-10-06T02:31:50.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.771+0000 7fc168f99700 1 -- 172.21.15.31:0/88431066 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc164159920 msgr2=0x7fc1640ad090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:50.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.771+0000 7fc168f99700 1 --2- 172.21.15.31:0/88431066 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc164159920 0x7fc1640ad090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:50.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.771+0000 7fc168f99700 1 -- 172.21.15.31:0/88431066 >> v1:172.21.15.62:6789/0 conn(0x7fc164159550 legacy=0x7fc1640ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:50.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.771+0000 7fc168f99700 1 -- 172.21.15.31:0/88431066 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1640ac5b0 con 0x7fc16415dfd0 2023-10-06T02:31:50.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.771+0000 7fc168f99700 1 -- 172.21.15.31:0/88431066 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7fc154001470 con 0x7fc16415dfd0 2023-10-06T02:31:50.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.771+0000 7fc168f99700 1 -- 172.21.15.31:0/88431066 >> v1:172.21.15.31:6789/0 conn(0x7fc16415dfd0 legacy=0x7fc1640aedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:50.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.771+0000 7fc168f99700 1 --2- 172.21.15.31:0/88431066 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc150003c20 0x7fc150004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:50.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.771+0000 7fc168f99700 1 --2- 172.21.15.31:0/88431066 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc164159920 0x7fc150004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:50.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.771+0000 7fc168f99700 1 --2- 172.21.15.31:0/88431066 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc150004d50 0x7fc1500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:50.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.771+0000 7fc168f99700 1 -- 172.21.15.31:0/88431066 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1640ac2d0 con 0x7fc150003c20 2023-10-06T02:31:50.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.771+0000 7fc168f99700 1 -- 172.21.15.31:0/88431066 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc15000bbf0 con 0x7fc150004d50 2023-10-06T02:31:50.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.771+0000 7fc168f99700 1 -- 172.21.15.31:0/88431066 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc15000bdb0 con 0x7fc164159920 2023-10-06T02:31:50.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.771+0000 7fc169f9b700 1 --2- 172.21.15.31:0/88431066 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc150003c20 0x7fc150004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:50.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.771+0000 7fc16979a700 1 --2- 172.21.15.31:0/88431066 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc164159920 0x7fc150004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:50.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.771+0000 7fc16a79c700 1 --2- 172.21.15.31:0/88431066 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc150004d50 0x7fc1500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:50.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.771+0000 7fc169f9b700 1 -- 172.21.15.31:0/88431066 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc164159920 msgr2=0x7fc150004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:50.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.771+0000 7fc169f9b700 1 --2- 172.21.15.31:0/88431066 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc164159920 0x7fc150004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:50.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.771+0000 7fc169f9b700 1 -- 172.21.15.31:0/88431066 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc150004d50 msgr2=0x7fc1500051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:50.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.771+0000 7fc169f9b700 1 --2- 172.21.15.31:0/88431066 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc150004d50 0x7fc1500051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:50.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.771+0000 7fc169f9b700 1 -- 172.21.15.31:0/88431066 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc150003570 con 0x7fc150003c20 2023-10-06T02:31:50.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.771+0000 7fc16a79c700 1 --2- 172.21.15.31:0/88431066 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc150004d50 0x7fc1500051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:31:50.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.772+0000 7fc169f9b700 1 --2- 172.21.15.31:0/88431066 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc150003c20 0x7fc150004010 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7fc1600019e0 tx=0x7fc160007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e643352b73a65f67 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:50.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.772+0000 7fc168f99700 1 -- 172.21.15.31:0/88431066 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc16000c5d0 con 0x7fc150003c20 2023-10-06T02:31:50.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.772+0000 7fc168f99700 1 -- 172.21.15.31:0/88431066 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc16000c730 con 0x7fc150003c20 2023-10-06T02:31:50.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.772+0000 7fc168f99700 1 -- 172.21.15.31:0/88431066 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc16000cad0 con 0x7fc150003c20 2023-10-06T02:31:50.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.772+0000 7fc16af9d700 1 -- 172.21.15.31:0/88431066 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc150003c20 msgr2=0x7fc150004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:50.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.772+0000 7fc16af9d700 1 --2- 172.21.15.31:0/88431066 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc150003c20 0x7fc150004010 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7fc1600019e0 tx=0x7fc160007d30 comp rx=0 tx=0).stop 2023-10-06T02:31:50.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.772+0000 7fc16a79c700 1 -- 172.21.15.31:0/88431066 reap_dead start 2023-10-06T02:31:50.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.773+0000 7fc16af9d700 1 -- 172.21.15.31:0/88431066 shutdown_connections 2023-10-06T02:31:50.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.773+0000 7fc16af9d700 1 -- 172.21.15.31:0/88431066 >> 172.21.15.31:0/88431066 conn(0x7fc16406ce80 msgr2=0x7fc1640ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:50.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.773+0000 7fc16af9d700 1 -- 172.21.15.31:0/88431066 shutdown_connections 2023-10-06T02:31:50.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.773+0000 7fc16af9d700 1 -- 172.21.15.31:0/88431066 wait complete. 2023-10-06T02:31:50.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.774+0000 7fc16af9d700 1 Processor -- start 2023-10-06T02:31:50.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.774+0000 7fc16af9d700 1 -- start start 2023-10-06T02:31:50.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.775+0000 7fc16af9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc150003c20 0x7fc1640c9df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:50.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.775+0000 7fc16af9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc164159dd0 0x7fc16416a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:50.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.775+0000 7fc16af9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc164150610 0x7fc16415bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:50.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.775+0000 7fc16af9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1640ca330 con 0x7fc150003c20 2023-10-06T02:31:50.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.775+0000 7fc16af9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1640c7560 con 0x7fc164159dd0 2023-10-06T02:31:50.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.775+0000 7fc16af9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc16416adc0 con 0x7fc164150610 2023-10-06T02:31:50.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.775+0000 7fc169f9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc150003c20 0x7fc1640c9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:50.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.775+0000 7fc169f9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc150003c20 0x7fc1640c9df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56756/0 (socket says 172.21.15.31:56756) 2023-10-06T02:31:50.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.775+0000 7fc169f9b700 1 -- 172.21.15.31:0/3828749488 learned_addr learned my addr 172.21.15.31:0/3828749488 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:50.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.775+0000 7fc16a79c700 1 --2- 172.21.15.31:0/3828749488 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc164150610 0x7fc16415bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:50.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.775+0000 7fc16979a700 1 --2- 172.21.15.31:0/3828749488 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc164159dd0 0x7fc16416a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:50.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.776+0000 7fc169f9b700 1 -- 172.21.15.31:0/3828749488 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc164150610 msgr2=0x7fc16415bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:50.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.776+0000 7fc169f9b700 1 --2- 172.21.15.31:0/3828749488 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc164150610 0x7fc16415bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:50.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.776+0000 7fc169f9b700 1 -- 172.21.15.31:0/3828749488 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc164159dd0 msgr2=0x7fc16416a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:50.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.776+0000 7fc169f9b700 1 --2- 172.21.15.31:0/3828749488 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc164159dd0 0x7fc16416a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:50.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.776+0000 7fc169f9b700 1 -- 172.21.15.31:0/3828749488 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1640c77b0 con 0x7fc150003c20 2023-10-06T02:31:50.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.776+0000 7fc169f9b700 1 --2- 172.21.15.31:0/3828749488 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc150003c20 0x7fc1640c9df0 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7fc160012f00 tx=0x7fc160015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:50.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.776+0000 7fc159ffb700 1 -- 172.21.15.31:0/3828749488 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc16000c5d0 con 0x7fc150003c20 2023-10-06T02:31:50.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.776+0000 7fc159ffb700 1 -- 172.21.15.31:0/3828749488 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc16000cab0 con 0x7fc150003c20 2023-10-06T02:31:50.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.776+0000 7fc159ffb700 1 -- 172.21.15.31:0/3828749488 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc160006a50 con 0x7fc150003c20 2023-10-06T02:31:50.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.776+0000 7fc16af9d700 1 -- 172.21.15.31:0/3828749488 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1640c7a90 con 0x7fc150003c20 2023-10-06T02:31:50.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.776+0000 7fc16af9d700 1 -- 172.21.15.31:0/3828749488 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1640c8050 con 0x7fc150003c20 2023-10-06T02:31:50.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.777+0000 7fc16af9d700 1 -- 172.21.15.31:0/3828749488 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1640c8270 con 0x7fc150003c20 2023-10-06T02:31:50.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.778+0000 7fc159ffb700 1 -- 172.21.15.31:0/3828749488 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85556+0+0 (secure 0 0 0) 0x7fc160019070 con 0x7fc150003c20 2023-10-06T02:31:50.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.780+0000 7fc159ffb700 1 --2- 172.21.15.31:0/3828749488 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc148067830 0x7fc148069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:50.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.780+0000 7fc159ffb700 1 -- 172.21.15.31:0/3828749488 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6173+0+0 (secure 0 0 0) 0x7fc1600841e0 con 0x7fc150003c20 2023-10-06T02:31:50.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.780+0000 7fc16979a700 1 --2- 172.21.15.31:0/3828749488 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc148067830 0x7fc148069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:50.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.781+0000 7fc16979a700 1 --2- 172.21.15.31:0/3828749488 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc148067830 0x7fc148069ce0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc154001bb0 tx=0x7fc154001d20 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:50.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.784+0000 7fc159ffb700 1 -- 172.21.15.31:0/3828749488 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc160052e20 con 0x7fc150003c20 2023-10-06T02:31:50.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.987+0000 7fc16af9d700 1 -- 172.21.15.31:0/3828749488 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc16404c3c0 con 0x7fc150003c20 2023-10-06T02:31:50.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.987+0000 7fc159ffb700 1 -- 172.21.15.31:0/3828749488 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v158) v1 ==== 74+0+17021 (secure 0 0 0) 0x7fc160023090 con 0x7fc150003c20 2023-10-06T02:31:50.990 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:31:50.990 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":158,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:31:40.588975+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:31:23.814443+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2023-10-06T02:31:24.697823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"156","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"26.f","mappings":[{"from":1,"to":0}]},{"pgid":"26.17","mappings":[{"from":1,"to":0}]},{"pgid":"26.27","mappings":[{"from":3,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":20,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:31:50.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.990+0000 7fc16af9d700 1 -- 172.21.15.31:0/3828749488 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc148067830 msgr2=0x7fc148069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:50.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.990+0000 7fc16af9d700 1 --2- 172.21.15.31:0/3828749488 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc148067830 0x7fc148069ce0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc154001bb0 tx=0x7fc154001d20 comp rx=0 tx=0).stop 2023-10-06T02:31:50.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.990+0000 7fc16af9d700 1 -- 172.21.15.31:0/3828749488 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc150003c20 msgr2=0x7fc1640c9df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:50.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.990+0000 7fc16af9d700 1 --2- 172.21.15.31:0/3828749488 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc150003c20 0x7fc1640c9df0 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7fc160012f00 tx=0x7fc160015000 comp rx=0 tx=0).stop 2023-10-06T02:31:50.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.990+0000 7fc16af9d700 1 -- 172.21.15.31:0/3828749488 shutdown_connections 2023-10-06T02:31:50.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.990+0000 7fc16af9d700 1 --2- 172.21.15.31:0/3828749488 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc148067830 0x7fc148069ce0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:50.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.990+0000 7fc16af9d700 1 --2- 172.21.15.31:0/3828749488 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc164150610 0x7fc16415bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:50.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.990+0000 7fc16af9d700 1 --2- 172.21.15.31:0/3828749488 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc150003c20 0x7fc1640c9df0 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:50.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.990+0000 7fc16af9d700 1 --2- 172.21.15.31:0/3828749488 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc164159dd0 0x7fc16416a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:50.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.990+0000 7fc16af9d700 1 -- 172.21.15.31:0/3828749488 >> 172.21.15.31:0/3828749488 conn(0x7fc16406ce80 msgr2=0x7fc164152970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:50.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.990+0000 7fc16af9d700 1 -- 172.21.15.31:0/3828749488 shutdown_connections 2023-10-06T02:31:50.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:50.990+0000 7fc16af9d700 1 -- 172.21.15.31:0/3828749488 wait complete. 2023-10-06T02:31:51.009 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T02:22:53.183425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T02:23:00.046967+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 25, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T02:31:23.814443+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '155', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 26, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T02:31:24.697823+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '156', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T02:31:51.009 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T02:31:51.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.199+0000 7f5f7359e700 1 Processor -- start 2023-10-06T02:31:51.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.199+0000 7f5f7359e700 1 -- start start 2023-10-06T02:31:51.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.201+0000 7f5f7359e700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f5f740c7fb0 0x7f5f7406abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:51.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.201+0000 7f5f7359e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5f7413f780 0x7f5f7413fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:51.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.201+0000 7f5f7359e700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f74140120 con 0x7f5f740c7be0 2023-10-06T02:31:51.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.201+0000 7f5f7359e700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f7406bf10 con 0x7f5f740c7fb0 2023-10-06T02:31:51.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.201+0000 7f5f7359e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7406b290 con 0x7f5f7413f780 2023-10-06T02:31:51.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.201+0000 7f5f72d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5f7413f780 0x7f5f7413fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:51.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.201+0000 7f5f7259c700 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f5f740c7be0 0x7f5f7415a990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:41742/0 (socket says 172.21.15.31:41742) 2023-10-06T02:31:51.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.201+0000 7f5f72d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5f7413f780 0x7f5f7413fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:47754/0 (socket says 172.21.15.31:47754) 2023-10-06T02:31:51.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.201+0000 7f5f7259c700 1 -- 172.21.15.31:0/3278218954 learned_addr learned my addr 172.21.15.31:0/3278218954 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:31:51.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.201+0000 7f5f71d9b700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f5f740c7fb0 0x7f5f7406abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:51.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.202+0000 7f5f7159a700 1 -- 172.21.15.31:0/3278218954 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 227904157 0 0) 0x7f5f64001110 con 0x7f5f740c7be0 2023-10-06T02:31:51.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.202+0000 7f5f7159a700 1 -- 172.21.15.31:0/3278218954 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f5c003570 con 0x7f5f740c7be0 2023-10-06T02:31:51.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.202+0000 7f5f72d9d700 1 -- 172.21.15.31:0/3278218954 >> v2:172.21.15.62:3300/0 conn(0x7f5f740c7fb0 msgr2=0x7f5f7406abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:51.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.202+0000 7f5f72d9d700 1 --2- 172.21.15.31:0/3278218954 >> v2:172.21.15.62:3300/0 conn(0x7f5f740c7fb0 0x7f5f7406abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:51.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.202+0000 7f5f72d9d700 1 -- 172.21.15.31:0/3278218954 >> v1:172.21.15.62:6789/0 conn(0x7f5f740c7be0 legacy=0x7f5f7415a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:51.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.202+0000 7f5f72d9d700 1 -- 172.21.15.31:0/3278218954 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f7406b570 con 0x7f5f7413f780 2023-10-06T02:31:51.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.202+0000 7f5f71d9b700 1 --2- 172.21.15.31:0/3278218954 >> v2:172.21.15.62:3300/0 conn(0x7f5f740c7fb0 0x7f5f7406abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:31:51.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.202+0000 7f5f72d9d700 1 --2- 172.21.15.31:0/3278218954 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5f7413f780 0x7f5f7413fbe0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c009e60 tx=0x7f5f6c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3287ad82090264b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:51.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.202+0000 7f5f7159a700 1 -- 172.21.15.31:0/3278218954 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c00e070 con 0x7f5f7413f780 2023-10-06T02:31:51.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.202+0000 7f5f7159a700 1 -- 172.21.15.31:0/3278218954 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f6c00bab0 con 0x7f5f7413f780 2023-10-06T02:31:51.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.202+0000 7f5f7159a700 1 -- 172.21.15.31:0/3278218954 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c00be70 con 0x7f5f7413f780 2023-10-06T02:31:51.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.203+0000 7f5f7359e700 1 -- 172.21.15.31:0/3278218954 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5f7413f780 msgr2=0x7f5f7413fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:51.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.203+0000 7f5f7359e700 1 --2- 172.21.15.31:0/3278218954 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5f7413f780 0x7f5f7413fbe0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c009e60 tx=0x7f5f6c009780 comp rx=0 tx=0).stop 2023-10-06T02:31:51.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.203+0000 7f5f7359e700 1 -- 172.21.15.31:0/3278218954 shutdown_connections 2023-10-06T02:31:51.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.203+0000 7f5f7359e700 1 --2- 172.21.15.31:0/3278218954 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5f7413f780 0x7f5f7413fbe0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:51.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.203+0000 7f5f7359e700 1 --2- 172.21.15.31:0/3278218954 >> v2:172.21.15.62:3300/0 conn(0x7f5f740c7fb0 0x7f5f7406abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:51.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.203+0000 7f5f7359e700 1 -- 172.21.15.31:0/3278218954 >> 172.21.15.31:0/3278218954 conn(0x7f5f7406d110 msgr2=0x7f5f740cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:51.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.203+0000 7f5f7359e700 1 -- 172.21.15.31:0/3278218954 shutdown_connections 2023-10-06T02:31:51.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.203+0000 7f5f7359e700 1 -- 172.21.15.31:0/3278218954 wait complete. 2023-10-06T02:31:51.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.204+0000 7f5f7359e700 1 Processor -- start 2023-10-06T02:31:51.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.205+0000 7f5f7359e700 1 -- start start 2023-10-06T02:31:51.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.205+0000 7f5f7359e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5f740c7be0 0x7f5f740c3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:51.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.205+0000 7f5f7359e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f740c7fb0 0x7f5f74155e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:51.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.205+0000 7f5f7359e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5f7413f780 0x7f5f74158a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:51.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.206+0000 7f5f7359e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f740c4460 con 0x7f5f740c7fb0 2023-10-06T02:31:51.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.206+0000 7f5f7359e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f740c4740 con 0x7f5f740c7be0 2023-10-06T02:31:51.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.206+0000 7f5f7359e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f740c4a20 con 0x7f5f7413f780 2023-10-06T02:31:51.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.206+0000 7f5f71d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f740c7fb0 0x7f5f74155e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:51.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.206+0000 7f5f71d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f740c7fb0 0x7f5f74155e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56766/0 (socket says 172.21.15.31:56766) 2023-10-06T02:31:51.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.206+0000 7f5f71d9b700 1 -- 172.21.15.31:0/1971802811 learned_addr learned my addr 172.21.15.31:0/1971802811 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:51.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.206+0000 7f5f7259c700 1 --2- 172.21.15.31:0/1971802811 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5f740c7be0 0x7f5f740c3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:51.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.206+0000 7f5f72d9d700 1 --2- 172.21.15.31:0/1971802811 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5f7413f780 0x7f5f74158a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:51.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.206+0000 7f5f71d9b700 1 -- 172.21.15.31:0/1971802811 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5f7413f780 msgr2=0x7f5f74158a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:51.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.206+0000 7f5f71d9b700 1 --2- 172.21.15.31:0/1971802811 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5f7413f780 0x7f5f74158a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:51.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.206+0000 7f5f71d9b700 1 -- 172.21.15.31:0/1971802811 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5f740c7be0 msgr2=0x7f5f740c3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:51.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.206+0000 7f5f71d9b700 1 --2- 172.21.15.31:0/1971802811 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5f740c7be0 0x7f5f740c3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:51.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.206+0000 7f5f71d9b700 1 -- 172.21.15.31:0/1971802811 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f68009420 con 0x7f5f740c7fb0 2023-10-06T02:31:51.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.207+0000 7f5f71d9b700 1 --2- 172.21.15.31:0/1971802811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f740c7fb0 0x7f5f74155e10 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f5f6800ec80 tx=0x7f5f680091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:51.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.207+0000 7f5f627fc700 1 -- 172.21.15.31:0/1971802811 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6800c9d0 con 0x7f5f740c7fb0 2023-10-06T02:31:51.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.207+0000 7f5f627fc700 1 -- 172.21.15.31:0/1971802811 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f6800cb30 con 0x7f5f740c7fb0 2023-10-06T02:31:51.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.207+0000 7f5f627fc700 1 -- 172.21.15.31:0/1971802811 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6800ceb0 con 0x7f5f740c7fb0 2023-10-06T02:31:51.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.207+0000 7f5f7359e700 1 -- 172.21.15.31:0/1971802811 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f6c009a20 con 0x7f5f740c7fb0 2023-10-06T02:31:51.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.207+0000 7f5f7359e700 1 -- 172.21.15.31:0/1971802811 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f7413bfe0 con 0x7f5f740c7fb0 2023-10-06T02:31:51.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.208+0000 7f5f7359e700 1 -- 172.21.15.31:0/1971802811 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f7413c380 con 0x7f5f740c7fb0 2023-10-06T02:31:51.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.209+0000 7f5f627fc700 1 -- 172.21.15.31:0/1971802811 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85556+0+0 (secure 0 0 0) 0x7f5f68013680 con 0x7f5f740c7fb0 2023-10-06T02:31:51.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.211+0000 7f5f627fc700 1 --2- 172.21.15.31:0/1971802811 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5f54067830 0x7f5f54069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:51.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.211+0000 7f5f627fc700 1 -- 172.21.15.31:0/1971802811 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6173+0+0 (secure 0 0 0) 0x7f5f68083500 con 0x7f5f740c7fb0 2023-10-06T02:31:51.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.212+0000 7f5f7259c700 1 --2- 172.21.15.31:0/1971802811 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5f54067830 0x7f5f54069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:51.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.212+0000 7f5f7259c700 1 --2- 172.21.15.31:0/1971802811 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5f54067830 0x7f5f54069ce0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f5f64001270 tx=0x7f5f64009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:51.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.215+0000 7f5f627fc700 1 -- 172.21.15.31:0/1971802811 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5f68051f50 con 0x7f5f740c7fb0 2023-10-06T02:31:51.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.415+0000 7f5f7359e700 1 -- 172.21.15.31:0/1971802811 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5f7404c3c0 con 0x7f5f740c7fb0 2023-10-06T02:31:51.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.416+0000 7f5f627fc700 1 -- 172.21.15.31:0/1971802811 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v158) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5f68055570 con 0x7f5f740c7fb0 2023-10-06T02:31:51.417 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-10-06T02:31:51.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.418+0000 7f5f7359e700 1 -- 172.21.15.31:0/1971802811 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5f54067830 msgr2=0x7f5f54069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:51.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.418+0000 7f5f7359e700 1 --2- 172.21.15.31:0/1971802811 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5f54067830 0x7f5f54069ce0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f5f64001270 tx=0x7f5f64009040 comp rx=0 tx=0).stop 2023-10-06T02:31:51.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.418+0000 7f5f7359e700 1 -- 172.21.15.31:0/1971802811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f740c7fb0 msgr2=0x7f5f74155e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:51.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.418+0000 7f5f7359e700 1 --2- 172.21.15.31:0/1971802811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f740c7fb0 0x7f5f74155e10 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f5f6800ec80 tx=0x7f5f680091a0 comp rx=0 tx=0).stop 2023-10-06T02:31:51.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.418+0000 7f5f7359e700 1 -- 172.21.15.31:0/1971802811 shutdown_connections 2023-10-06T02:31:51.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.418+0000 7f5f7359e700 1 --2- 172.21.15.31:0/1971802811 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5f54067830 0x7f5f54069ce0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:51.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.418+0000 7f5f7359e700 1 --2- 172.21.15.31:0/1971802811 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5f7413f780 0x7f5f74158a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:51.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.418+0000 7f5f7359e700 1 --2- 172.21.15.31:0/1971802811 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5f740c7be0 0x7f5f740c3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:51.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.418+0000 7f5f7359e700 1 --2- 172.21.15.31:0/1971802811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f740c7fb0 0x7f5f74155e10 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:51.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.418+0000 7f5f7359e700 1 -- 172.21.15.31:0/1971802811 >> 172.21.15.31:0/1971802811 conn(0x7f5f7406d110 msgr2=0x7f5f740cae40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:51.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.418+0000 7f5f7359e700 1 -- 172.21.15.31:0/1971802811 shutdown_connections 2023-10-06T02:31:51.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.418+0000 7f5f7359e700 1 -- 172.21.15.31:0/1971802811 wait complete. 2023-10-06T02:31:51.437 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T02:31:51.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.625+0000 7efe7259d700 1 Processor -- start 2023-10-06T02:31:51.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.625+0000 7efe7259d700 1 -- start start 2023-10-06T02:31:51.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.626+0000 7efe7259d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7efe6c153a00 0x7efe6c153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:51.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.626+0000 7efe7259d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efe6c0cea00 0x7efe6c0cee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:51.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.626+0000 7efe7259d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe6c15ac50 con 0x7efe6c15de50 2023-10-06T02:31:51.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.626+0000 7efe7259d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe6c15d010 con 0x7efe6c153a00 2023-10-06T02:31:51.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.627+0000 7efe7259d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe6c0cf3c0 con 0x7efe6c0cea00 2023-10-06T02:31:51.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.627+0000 7efe71d9c700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efe6c0cea00 0x7efe6c0cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:51.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.627+0000 7efe70d9a700 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7efe6c15de50 0x7efe6c15a540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:41758/0 (socket says 172.21.15.31:41758) 2023-10-06T02:31:51.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.627+0000 7efe7159b700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7efe6c153a00 0x7efe6c153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:51.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.627+0000 7efe70d9a700 1 -- 172.21.15.31:0/1662914468 learned_addr learned my addr 172.21.15.31:0/1662914468 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:31:51.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.627+0000 7efe63fff700 1 -- 172.21.15.31:0/1662914468 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 750516414 0 0) 0x7efe6c15ac50 con 0x7efe6c15de50 2023-10-06T02:31:51.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.627+0000 7efe63fff700 1 -- 172.21.15.31:0/1662914468 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe58003570 con 0x7efe6c15de50 2023-10-06T02:31:51.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.627+0000 7efe71d9c700 1 -- 172.21.15.31:0/1662914468 >> v2:172.21.15.62:3300/0 conn(0x7efe6c153a00 msgr2=0x7efe6c153dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:51.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.627+0000 7efe71d9c700 1 --2- 172.21.15.31:0/1662914468 >> v2:172.21.15.62:3300/0 conn(0x7efe6c153a00 0x7efe6c153dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:51.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.628+0000 7efe71d9c700 1 -- 172.21.15.31:0/1662914468 >> v1:172.21.15.62:6789/0 conn(0x7efe6c15de50 legacy=0x7efe6c15a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:51.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.628+0000 7efe71d9c700 1 -- 172.21.15.31:0/1662914468 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe6c15af60 con 0x7efe6c0cea00 2023-10-06T02:31:51.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.628+0000 7efe71d9c700 1 --2- 172.21.15.31:0/1662914468 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efe6c0cea00 0x7efe6c0cee80 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7efe64009e60 tx=0x7efe64009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7382f04664cf469c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:51.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.628+0000 7efe63fff700 1 -- 172.21.15.31:0/1662914468 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe6400e070 con 0x7efe6c0cea00 2023-10-06T02:31:51.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.628+0000 7efe63fff700 1 -- 172.21.15.31:0/1662914468 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe6400bab0 con 0x7efe6c0cea00 2023-10-06T02:31:51.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.628+0000 7efe63fff700 1 -- 172.21.15.31:0/1662914468 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe6400be70 con 0x7efe6c0cea00 2023-10-06T02:31:51.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.629+0000 7efe7259d700 1 -- 172.21.15.31:0/1662914468 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efe6c0cea00 msgr2=0x7efe6c0cee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:51.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.629+0000 7efe7259d700 1 --2- 172.21.15.31:0/1662914468 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efe6c0cea00 0x7efe6c0cee80 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7efe64009e60 tx=0x7efe64009780 comp rx=0 tx=0).stop 2023-10-06T02:31:51.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.629+0000 7efe7259d700 1 -- 172.21.15.31:0/1662914468 shutdown_connections 2023-10-06T02:31:51.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.629+0000 7efe7259d700 1 --2- 172.21.15.31:0/1662914468 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efe6c0cea00 0x7efe6c0cee80 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:51.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.629+0000 7efe7259d700 1 --2- 172.21.15.31:0/1662914468 >> v2:172.21.15.62:3300/0 conn(0x7efe6c153a00 0x7efe6c153dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:51.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.629+0000 7efe7259d700 1 -- 172.21.15.31:0/1662914468 >> 172.21.15.31:0/1662914468 conn(0x7efe6c0703a0 msgr2=0x7efe6c0707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:51.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.629+0000 7efe7259d700 1 -- 172.21.15.31:0/1662914468 shutdown_connections 2023-10-06T02:31:51.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.629+0000 7efe7259d700 1 -- 172.21.15.31:0/1662914468 wait complete. 2023-10-06T02:31:51.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.630+0000 7efe7259d700 1 Processor -- start 2023-10-06T02:31:51.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.630+0000 7efe7259d700 1 -- start start 2023-10-06T02:31:51.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.631+0000 7efe7259d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe6c0cea00 0x7efe6c152ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:51.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.631+0000 7efe7259d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe6c153a00 0x7efe6c0b32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:51.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.631+0000 7efe7259d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efe6c15de50 0x7efe6c158a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:51.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.631+0000 7efe7259d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe6c0c80b0 con 0x7efe6c0cea00 2023-10-06T02:31:51.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.631+0000 7efe7259d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe6c0c7780 con 0x7efe6c153a00 2023-10-06T02:31:51.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.631+0000 7efe7259d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe6c0c7a60 con 0x7efe6c15de50 2023-10-06T02:31:51.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.631+0000 7efe7159b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe6c0cea00 0x7efe6c152ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:51.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.632+0000 7efe7159b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe6c0cea00 0x7efe6c152ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56768/0 (socket says 172.21.15.31:56768) 2023-10-06T02:31:51.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.632+0000 7efe7159b700 1 -- 172.21.15.31:0/3373235889 learned_addr learned my addr 172.21.15.31:0/3373235889 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:51.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.632+0000 7efe71d9c700 1 --2- 172.21.15.31:0/3373235889 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efe6c15de50 0x7efe6c158a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:51.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.632+0000 7efe70d9a700 1 --2- 172.21.15.31:0/3373235889 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe6c153a00 0x7efe6c0b32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:51.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.632+0000 7efe7159b700 1 -- 172.21.15.31:0/3373235889 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efe6c15de50 msgr2=0x7efe6c158a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:51.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.632+0000 7efe7159b700 1 --2- 172.21.15.31:0/3373235889 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efe6c15de50 0x7efe6c158a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:51.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.632+0000 7efe7159b700 1 -- 172.21.15.31:0/3373235889 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe6c153a00 msgr2=0x7efe6c0b32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:51.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.632+0000 7efe7159b700 1 --2- 172.21.15.31:0/3373235889 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe6c153a00 0x7efe6c0b32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:51.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.632+0000 7efe7159b700 1 -- 172.21.15.31:0/3373235889 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe64009a20 con 0x7efe6c0cea00 2023-10-06T02:31:51.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.632+0000 7efe7159b700 1 --2- 172.21.15.31:0/3373235889 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe6c0cea00 0x7efe6c152ff0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7efe6800ec80 tx=0x7efe680091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:51.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.632+0000 7efe617fa700 1 -- 172.21.15.31:0/3373235889 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe6800c8e0 con 0x7efe6c0cea00 2023-10-06T02:31:51.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.633+0000 7efe617fa700 1 -- 172.21.15.31:0/3373235889 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe6800ca40 con 0x7efe6c0cea00 2023-10-06T02:31:51.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.633+0000 7efe7259d700 1 -- 172.21.15.31:0/3373235889 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe6c0c7d40 con 0x7efe6c0cea00 2023-10-06T02:31:51.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.633+0000 7efe617fa700 1 -- 172.21.15.31:0/3373235889 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe6800cde0 con 0x7efe6c0cea00 2023-10-06T02:31:51.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.633+0000 7efe7259d700 1 -- 172.21.15.31:0/3373235889 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe6c0b3810 con 0x7efe6c0cea00 2023-10-06T02:31:51.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.634+0000 7efe7259d700 1 -- 172.21.15.31:0/3373235889 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe6c04c3c0 con 0x7efe6c0cea00 2023-10-06T02:31:51.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.635+0000 7efe617fa700 1 -- 172.21.15.31:0/3373235889 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85556+0+0 (secure 0 0 0) 0x7efe68013680 con 0x7efe6c0cea00 2023-10-06T02:31:51.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.635+0000 7efe617fa700 1 --2- 172.21.15.31:0/3373235889 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7efe50067830 0x7efe50069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:51.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.636+0000 7efe617fa700 1 -- 172.21.15.31:0/3373235889 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6173+0+0 (secure 0 0 0) 0x7efe68083320 con 0x7efe6c0cea00 2023-10-06T02:31:51.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.636+0000 7efe70d9a700 1 --2- 172.21.15.31:0/3373235889 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7efe50067830 0x7efe50069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:51.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.637+0000 7efe70d9a700 1 --2- 172.21.15.31:0/3373235889 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7efe50067830 0x7efe50069ce0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7efe6c0ca470 tx=0x7efe5c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:51.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.640+0000 7efe617fa700 1 -- 172.21.15.31:0/3373235889 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efe68051dd0 con 0x7efe6c0cea00 2023-10-06T02:31:51.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.842+0000 7efe7259d700 1 -- 172.21.15.31:0/3373235889 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7efe6c0caef0 con 0x7efe6c0cea00 2023-10-06T02:31:51.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.843+0000 7efe617fa700 1 -- 172.21.15.31:0/3373235889 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v158) v1 ==== 92+0+10 (secure 0 0 0) 0x7efe680170a0 con 0x7efe6c0cea00 2023-10-06T02:31:51.845 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 8 2023-10-06T02:31:51.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.845+0000 7efe7259d700 1 -- 172.21.15.31:0/3373235889 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7efe50067830 msgr2=0x7efe50069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:51.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.845+0000 7efe7259d700 1 --2- 172.21.15.31:0/3373235889 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7efe50067830 0x7efe50069ce0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7efe6c0ca470 tx=0x7efe5c009040 comp rx=0 tx=0).stop 2023-10-06T02:31:51.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.845+0000 7efe7259d700 1 -- 172.21.15.31:0/3373235889 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe6c0cea00 msgr2=0x7efe6c152ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:51.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.845+0000 7efe7259d700 1 --2- 172.21.15.31:0/3373235889 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe6c0cea00 0x7efe6c152ff0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7efe6800ec80 tx=0x7efe680091a0 comp rx=0 tx=0).stop 2023-10-06T02:31:51.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.845+0000 7efe7259d700 1 -- 172.21.15.31:0/3373235889 shutdown_connections 2023-10-06T02:31:51.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.845+0000 7efe7259d700 1 --2- 172.21.15.31:0/3373235889 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7efe50067830 0x7efe50069ce0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:51.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.845+0000 7efe7259d700 1 --2- 172.21.15.31:0/3373235889 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efe6c15de50 0x7efe6c158a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:51.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.845+0000 7efe7259d700 1 --2- 172.21.15.31:0/3373235889 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe6c0cea00 0x7efe6c152ff0 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:51.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.845+0000 7efe7259d700 1 --2- 172.21.15.31:0/3373235889 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe6c153a00 0x7efe6c0b32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:51.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.845+0000 7efe7259d700 1 -- 172.21.15.31:0/3373235889 >> 172.21.15.31:0/3373235889 conn(0x7efe6c0703a0 msgr2=0x7efe6c15bf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:51.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.845+0000 7efe7259d700 1 -- 172.21.15.31:0/3373235889 shutdown_connections 2023-10-06T02:31:51.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:51.845+0000 7efe7259d700 1 -- 172.21.15.31:0/3373235889 wait complete. 2023-10-06T02:31:51.864 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T02:31:52.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.054+0000 7f07f1f9d700 1 Processor -- start 2023-10-06T02:31:52.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.054+0000 7f07f1f9d700 1 -- start start 2023-10-06T02:31:52.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.055+0000 7f07f1f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f07ec155890 0x7f07ec0c7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:52.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.055+0000 7f07f1f9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f07ec155c60 0x7f07ec06a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:52.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.055+0000 7f07f1f9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f07ec06de50 0x7f07ec06e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:52.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.055+0000 7f07f1f9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07ec06e810 con 0x7f07ec06de50 2023-10-06T02:31:52.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.055+0000 7f07f1f9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07ec0c7e70 con 0x7f07ec155c60 2023-10-06T02:31:52.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.055+0000 7f07f1f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07ec06b790 con 0x7f07ec155890 2023-10-06T02:31:52.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.056+0000 7f07f179c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f07ec06de50 0x7f07ec06e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:52.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.056+0000 7f07f179c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f07ec06de50 0x7f07ec06e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56770/0 (socket says 172.21.15.31:56770) 2023-10-06T02:31:52.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.056+0000 7f07f179c700 1 -- 172.21.15.31:0/689917355 learned_addr learned my addr 172.21.15.31:0/689917355 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:52.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.056+0000 7f07f0f9b700 1 --2- 172.21.15.31:0/689917355 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f07ec155890 0x7f07ec0c7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:52.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.056+0000 7f07ebfff700 1 --2- 172.21.15.31:0/689917355 >> v2:172.21.15.62:3300/0 conn(0x7f07ec155c60 0x7f07ec06a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:52.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.056+0000 7f07f179c700 1 -- 172.21.15.31:0/689917355 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f07ec155890 msgr2=0x7f07ec0c7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:52.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.056+0000 7f07f179c700 1 --2- 172.21.15.31:0/689917355 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f07ec155890 0x7f07ec0c7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:52.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.056+0000 7f07f179c700 1 -- 172.21.15.31:0/689917355 >> v2:172.21.15.62:3300/0 conn(0x7f07ec155c60 msgr2=0x7f07ec06a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:52.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.056+0000 7f07f179c700 1 --2- 172.21.15.31:0/689917355 >> v2:172.21.15.62:3300/0 conn(0x7f07ec155c60 0x7f07ec06a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:52.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.056+0000 7f07f179c700 1 -- 172.21.15.31:0/689917355 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07ec06a8e0 con 0x7f07ec06de50 2023-10-06T02:31:52.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.056+0000 7f07f179c700 1 --2- 172.21.15.31:0/689917355 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07ec06de50 0x7f07ec06e2d0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f07ec0c9c10 tx=0x7f07e4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80c1073bfe821af5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:52.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.057+0000 7f07eb7fe700 1 -- 172.21.15.31:0/689917355 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e400e070 con 0x7f07ec06de50 2023-10-06T02:31:52.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.057+0000 7f07eb7fe700 1 -- 172.21.15.31:0/689917355 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07e400ba40 con 0x7f07ec06de50 2023-10-06T02:31:52.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.057+0000 7f07eb7fe700 1 -- 172.21.15.31:0/689917355 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e40092e0 con 0x7f07ec06de50 2023-10-06T02:31:52.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.057+0000 7f07f1f9d700 1 -- 172.21.15.31:0/689917355 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07ec06de50 msgr2=0x7f07ec06e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:52.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.057+0000 7f07f1f9d700 1 --2- 172.21.15.31:0/689917355 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07ec06de50 0x7f07ec06e2d0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f07ec0c9c10 tx=0x7f07e4009840 comp rx=0 tx=0).stop 2023-10-06T02:31:52.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.057+0000 7f07f1f9d700 1 -- 172.21.15.31:0/689917355 shutdown_connections 2023-10-06T02:31:52.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.057+0000 7f07f1f9d700 1 --2- 172.21.15.31:0/689917355 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07ec06de50 0x7f07ec06e2d0 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:52.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.057+0000 7f07f1f9d700 1 --2- 172.21.15.31:0/689917355 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f07ec155890 0x7f07ec0c7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:52.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.057+0000 7f07f1f9d700 1 --2- 172.21.15.31:0/689917355 >> v2:172.21.15.62:3300/0 conn(0x7f07ec155c60 0x7f07ec06a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:52.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.057+0000 7f07f1f9d700 1 -- 172.21.15.31:0/689917355 >> 172.21.15.31:0/689917355 conn(0x7f07ec06d110 msgr2=0x7f07ec0cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:52.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.057+0000 7f07f1f9d700 1 -- 172.21.15.31:0/689917355 shutdown_connections 2023-10-06T02:31:52.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.057+0000 7f07f1f9d700 1 -- 172.21.15.31:0/689917355 wait complete. 2023-10-06T02:31:52.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.059+0000 7f07f1f9d700 1 Processor -- start 2023-10-06T02:31:52.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.059+0000 7f07f1f9d700 1 -- start start 2023-10-06T02:31:52.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.060+0000 7f07f1f9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07ec06de50 0x7f07ec0b0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:52.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.060+0000 7f07f1f9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07ec155890 0x7f07ec0c5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:52.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.060+0000 7f07f1f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f07ec155c60 0x7f07ec0a98e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:52.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.060+0000 7f07f1f9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07ec16ab30 con 0x7f07ec155890 2023-10-06T02:31:52.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.060+0000 7f07f1f9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07ec0b0dc0 con 0x7f07ec06de50 2023-10-06T02:31:52.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.060+0000 7f07f1f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07ec0b1090 con 0x7f07ec155c60 2023-10-06T02:31:52.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.060+0000 7f07ebfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07ec155890 0x7f07ec0c5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:52.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.060+0000 7f07ebfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07ec155890 0x7f07ec0c5de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56786/0 (socket says 172.21.15.31:56786) 2023-10-06T02:31:52.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.060+0000 7f07ebfff700 1 -- 172.21.15.31:0/3199430515 learned_addr learned my addr 172.21.15.31:0/3199430515 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:52.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.060+0000 7f07f0f9b700 1 --2- 172.21.15.31:0/3199430515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07ec06de50 0x7f07ec0b0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:52.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.060+0000 7f07f179c700 1 --2- 172.21.15.31:0/3199430515 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f07ec155c60 0x7f07ec0a98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:52.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.060+0000 7f07ebfff700 1 -- 172.21.15.31:0/3199430515 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f07ec155c60 msgr2=0x7f07ec0a98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:52.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.060+0000 7f07ebfff700 1 --2- 172.21.15.31:0/3199430515 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f07ec155c60 0x7f07ec0a98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:52.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.061+0000 7f07ebfff700 1 -- 172.21.15.31:0/3199430515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07ec06de50 msgr2=0x7f07ec0b0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:52.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.061+0000 7f07ebfff700 1 --2- 172.21.15.31:0/3199430515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07ec06de50 0x7f07ec0b0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:52.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.061+0000 7f07ebfff700 1 -- 172.21.15.31:0/3199430515 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07e4009ae0 con 0x7f07ec155890 2023-10-06T02:31:52.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.061+0000 7f07f0f9b700 1 --2- 172.21.15.31:0/3199430515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07ec06de50 0x7f07ec0b0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:31:52.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.061+0000 7f07ebfff700 1 --2- 172.21.15.31:0/3199430515 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07ec155890 0x7f07ec0c5de0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f07d800d890 tx=0x7f07d8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:52.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.061+0000 7f07e8ff9700 1 -- 172.21.15.31:0/3199430515 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d800b600 con 0x7f07ec155890 2023-10-06T02:31:52.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.061+0000 7f07e8ff9700 1 -- 172.21.15.31:0/3199430515 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07d800b760 con 0x7f07ec155890 2023-10-06T02:31:52.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.061+0000 7f07f1f9d700 1 -- 172.21.15.31:0/3199430515 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07ec0b1370 con 0x7f07ec155890 2023-10-06T02:31:52.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.061+0000 7f07e8ff9700 1 -- 172.21.15.31:0/3199430515 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d800bae0 con 0x7f07ec155890 2023-10-06T02:31:52.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.061+0000 7f07f1f9d700 1 -- 172.21.15.31:0/3199430515 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07ec16ada0 con 0x7f07ec155890 2023-10-06T02:31:52.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.062+0000 7f07f1f9d700 1 -- 172.21.15.31:0/3199430515 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07ec0c72d0 con 0x7f07ec155890 2023-10-06T02:31:52.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.063+0000 7f07e8ff9700 1 -- 172.21.15.31:0/3199430515 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85556+0+0 (secure 0 0 0) 0x7f07d801e460 con 0x7f07ec155890 2023-10-06T02:31:52.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.064+0000 7f07e8ff9700 1 --2- 172.21.15.31:0/3199430515 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f07d0067830 0x7f07d0069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:52.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.064+0000 7f07e8ff9700 1 -- 172.21.15.31:0/3199430515 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6173+0+0 (secure 0 0 0) 0x7f07d8083730 con 0x7f07ec155890 2023-10-06T02:31:52.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.064+0000 7f07f0f9b700 1 --2- 172.21.15.31:0/3199430515 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f07d0067830 0x7f07d0069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:52.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.065+0000 7f07f0f9b700 1 --2- 172.21.15.31:0/3199430515 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f07d0067830 0x7f07d0069ce0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f07e0006fd0 tx=0x7f07e0008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:52.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.068+0000 7f07e8ff9700 1 -- 172.21.15.31:0/3199430515 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f07d8052180 con 0x7f07ec155890 2023-10-06T02:31:52.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.270+0000 7f07f1f9d700 1 -- 172.21.15.31:0/3199430515 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f07ec0ccdc0 con 0x7f07ec155890 2023-10-06T02:31:52.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.270+0000 7f07e8ff9700 1 -- 172.21.15.31:0/3199430515 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v158) v1 ==== 104+0+11 (secure 0 0 0) 0x7f07d80557a0 con 0x7f07ec155890 2023-10-06T02:31:52.272 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-10-06T02:31:52.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.272+0000 7f07f1f9d700 1 -- 172.21.15.31:0/3199430515 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f07d0067830 msgr2=0x7f07d0069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:52.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.272+0000 7f07f1f9d700 1 --2- 172.21.15.31:0/3199430515 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f07d0067830 0x7f07d0069ce0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f07e0006fd0 tx=0x7f07e0008040 comp rx=0 tx=0).stop 2023-10-06T02:31:52.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.273+0000 7f07f1f9d700 1 -- 172.21.15.31:0/3199430515 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07ec155890 msgr2=0x7f07ec0c5de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:52.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.273+0000 7f07f1f9d700 1 --2- 172.21.15.31:0/3199430515 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07ec155890 0x7f07ec0c5de0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f07d800d890 tx=0x7f07d8008040 comp rx=0 tx=0).stop 2023-10-06T02:31:52.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.273+0000 7f07f1f9d700 1 -- 172.21.15.31:0/3199430515 shutdown_connections 2023-10-06T02:31:52.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.273+0000 7f07f1f9d700 1 --2- 172.21.15.31:0/3199430515 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f07d0067830 0x7f07d0069ce0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:52.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.273+0000 7f07f1f9d700 1 --2- 172.21.15.31:0/3199430515 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f07ec155c60 0x7f07ec0a98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:52.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.273+0000 7f07f1f9d700 1 --2- 172.21.15.31:0/3199430515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07ec06de50 0x7f07ec0b0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:52.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.273+0000 7f07f1f9d700 1 --2- 172.21.15.31:0/3199430515 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07ec155890 0x7f07ec0c5de0 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:52.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.273+0000 7f07f1f9d700 1 -- 172.21.15.31:0/3199430515 >> 172.21.15.31:0/3199430515 conn(0x7f07ec06d110 msgr2=0x7f07ec0c1090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:52.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.273+0000 7f07f1f9d700 1 -- 172.21.15.31:0/3199430515 shutdown_connections 2023-10-06T02:31:52.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.273+0000 7f07f1f9d700 1 -- 172.21.15.31:0/3199430515 wait complete. 2023-10-06T02:31:52.292 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T02:31:52.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.480+0000 7f94e37ff700 1 Processor -- start 2023-10-06T02:31:52.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.481+0000 7f94e37ff700 1 -- start start 2023-10-06T02:31:52.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.482+0000 7f94e37ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f94dc0c8800 0x7f94dc0c94e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:52.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.482+0000 7f94e37ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f94dc0b1150 0x7f94dc0ae050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:52.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.482+0000 7f94e37ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94dc0adbc0 con 0x7f94dc0c8bd0 2023-10-06T02:31:52.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.482+0000 7f94e37ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94dc0c9d30 con 0x7f94dc0c8800 2023-10-06T02:31:52.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.482+0000 7f94e37ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94dc0acfb0 con 0x7f94dc0b1150 2023-10-06T02:31:52.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.482+0000 7f94e159b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f94dc0c8800 0x7f94dc0c94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:52.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.482+0000 7f94e159b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f94dc0c8800 0x7f94dc0c94e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56792/0 (socket says 172.21.15.31:56792) 2023-10-06T02:31:52.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.482+0000 7f94e159b700 1 -- 172.21.15.31:0/1204662734 learned_addr learned my addr 172.21.15.31:0/1204662734 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:52.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.482+0000 7f94e1d9c700 1 --2- 172.21.15.31:0/1204662734 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f94dc0b1150 0x7f94dc0ae050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:52.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.483+0000 7f94e159b700 1 -- 172.21.15.31:0/1204662734 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f94dc0b1150 msgr2=0x7f94dc0ae050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:52.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.483+0000 7f94e159b700 1 --2- 172.21.15.31:0/1204662734 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f94dc0b1150 0x7f94dc0ae050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:52.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.483+0000 7f94e159b700 1 -- 172.21.15.31:0/1204662734 >> v1:172.21.15.62:6789/0 conn(0x7f94dc0c8bd0 legacy=0x7f94dc0b0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:52.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.483+0000 7f94e159b700 1 -- 172.21.15.31:0/1204662734 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94dc0ad290 con 0x7f94dc0c8800 2023-10-06T02:31:52.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.483+0000 7f94e159b700 1 --2- 172.21.15.31:0/1204662734 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94dc0c8800 0x7f94dc0c94e0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f94d80099b0 tx=0x7f94d8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cf8fe8a0a3bfa53 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:52.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.483+0000 7f94d3fff700 1 -- 172.21.15.31:0/1204662734 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d800b860 con 0x7f94dc0c8800 2023-10-06T02:31:52.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.483+0000 7f94d3fff700 1 -- 172.21.15.31:0/1204662734 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94d800b9c0 con 0x7f94dc0c8800 2023-10-06T02:31:52.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.483+0000 7f94d3fff700 1 -- 172.21.15.31:0/1204662734 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d800bd60 con 0x7f94dc0c8800 2023-10-06T02:31:52.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.484+0000 7f94e37ff700 1 -- 172.21.15.31:0/1204662734 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94dc0c8800 msgr2=0x7f94dc0c94e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:52.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.484+0000 7f94e37ff700 1 --2- 172.21.15.31:0/1204662734 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94dc0c8800 0x7f94dc0c94e0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f94d80099b0 tx=0x7f94d8009250 comp rx=0 tx=0).stop 2023-10-06T02:31:52.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.484+0000 7f94e37ff700 1 -- 172.21.15.31:0/1204662734 shutdown_connections 2023-10-06T02:31:52.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.484+0000 7f94e37ff700 1 --2- 172.21.15.31:0/1204662734 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f94dc0b1150 0x7f94dc0ae050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:52.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.484+0000 7f94e37ff700 1 --2- 172.21.15.31:0/1204662734 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94dc0c8800 0x7f94dc0c94e0 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:52.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.484+0000 7f94e37ff700 1 -- 172.21.15.31:0/1204662734 >> 172.21.15.31:0/1204662734 conn(0x7f94dc06d170 msgr2=0x7f94dc0cc400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:52.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.484+0000 7f94e37ff700 1 -- 172.21.15.31:0/1204662734 shutdown_connections 2023-10-06T02:31:52.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.484+0000 7f94e37ff700 1 -- 172.21.15.31:0/1204662734 wait complete. 2023-10-06T02:31:52.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.485+0000 7f94e37ff700 1 Processor -- start 2023-10-06T02:31:52.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.486+0000 7f94e37ff700 1 -- start start 2023-10-06T02:31:52.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.487+0000 7f94e37ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94dc0b1150 0x7f94dc153020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:52.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.487+0000 7f94e37ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f94dc0c8800 0x7f94dc0c8080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:52.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.487+0000 7f94e37ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94dc0c8bd0 0x7f94dc0abd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:52.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.487+0000 7f94e37ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94dc169fb0 con 0x7f94dc0c8bd0 2023-10-06T02:31:52.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.487+0000 7f94e37ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94dc0aa270 con 0x7f94dc0b1150 2023-10-06T02:31:52.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.487+0000 7f94e37ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94dc169c80 con 0x7f94dc0c8800 2023-10-06T02:31:52.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.487+0000 7f94e1d9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94dc0c8bd0 0x7f94dc0abd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:52.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.487+0000 7f94e1d9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94dc0c8bd0 0x7f94dc0abd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56798/0 (socket says 172.21.15.31:56798) 2023-10-06T02:31:52.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.487+0000 7f94e1d9c700 1 -- 172.21.15.31:0/3611949608 learned_addr learned my addr 172.21.15.31:0/3611949608 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:52.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.487+0000 7f94e159b700 1 --2- 172.21.15.31:0/3611949608 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94dc0b1150 0x7f94dc153020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:52.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.487+0000 7f94e0d9a700 1 --2- 172.21.15.31:0/3611949608 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f94dc0c8800 0x7f94dc0c8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:52.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.487+0000 7f94e1d9c700 1 -- 172.21.15.31:0/3611949608 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f94dc0c8800 msgr2=0x7f94dc0c8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:52.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.487+0000 7f94e1d9c700 1 --2- 172.21.15.31:0/3611949608 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f94dc0c8800 0x7f94dc0c8080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:52.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.487+0000 7f94e1d9c700 1 -- 172.21.15.31:0/3611949608 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94dc0b1150 msgr2=0x7f94dc153020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:52.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.487+0000 7f94e1d9c700 1 --2- 172.21.15.31:0/3611949608 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94dc0b1150 0x7f94dc153020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:52.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.487+0000 7f94e1d9c700 1 -- 172.21.15.31:0/3611949608 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94d80094f0 con 0x7f94dc0c8bd0 2023-10-06T02:31:52.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.488+0000 7f94e1d9c700 1 --2- 172.21.15.31:0/3611949608 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94dc0c8bd0 0x7f94dc0abd70 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f94d4007ef0 tx=0x7f94d4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:52.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.488+0000 7f94d17fa700 1 -- 172.21.15.31:0/3611949608 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d400eb30 con 0x7f94dc0c8bd0 2023-10-06T02:31:52.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.488+0000 7f94d17fa700 1 -- 172.21.15.31:0/3611949608 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94d400ec90 con 0x7f94dc0c8bd0 2023-10-06T02:31:52.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.488+0000 7f94d17fa700 1 -- 172.21.15.31:0/3611949608 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d4014670 con 0x7f94dc0c8bd0 2023-10-06T02:31:52.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.488+0000 7f94e37ff700 1 -- 172.21.15.31:0/3611949608 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94dc0aa510 con 0x7f94dc0c8bd0 2023-10-06T02:31:52.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.488+0000 7f94e37ff700 1 -- 172.21.15.31:0/3611949608 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94dc0aa7f0 con 0x7f94dc0c8bd0 2023-10-06T02:31:52.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.489+0000 7f94e37ff700 1 -- 172.21.15.31:0/3611949608 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94dc16a120 con 0x7f94dc0c8bd0 2023-10-06T02:31:52.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.490+0000 7f94d17fa700 1 -- 172.21.15.31:0/3611949608 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85556+0+0 (secure 0 0 0) 0x7f94d4014a60 con 0x7f94dc0c8bd0 2023-10-06T02:31:52.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.491+0000 7f94d17fa700 1 --2- 172.21.15.31:0/3611949608 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f94c0067830 0x7f94c0069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:52.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.491+0000 7f94d17fa700 1 -- 172.21.15.31:0/3611949608 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6173+0+0 (secure 0 0 0) 0x7f94d40833d0 con 0x7f94dc0c8bd0 2023-10-06T02:31:52.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.491+0000 7f94e159b700 1 --2- 172.21.15.31:0/3611949608 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f94c0067830 0x7f94c0069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:52.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.492+0000 7f94e159b700 1 --2- 172.21.15.31:0/3611949608 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f94c0067830 0x7f94c0069ce0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f94d80099b0 tx=0x7f94d8017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:52.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.495+0000 7f94d17fa700 1 -- 172.21.15.31:0/3611949608 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f94d4052070 con 0x7f94dc0c8bd0 2023-10-06T02:31:52.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.696+0000 7f94e37ff700 1 -- 172.21.15.31:0/3611949608 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f94dc04c3c0 con 0x7f94dc0c8bd0 2023-10-06T02:31:52.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.697+0000 7f94d17fa700 1 -- 172.21.15.31:0/3611949608 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v158) v1 ==== 100+0+11 (secure 0 0 0) 0x7f94d4009980 con 0x7f94dc0c8bd0 2023-10-06T02:31:52.720 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-10-06T02:31:52.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.699+0000 7f94e37ff700 1 -- 172.21.15.31:0/3611949608 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f94c0067830 msgr2=0x7f94c0069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:52.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.699+0000 7f94e37ff700 1 --2- 172.21.15.31:0/3611949608 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f94c0067830 0x7f94c0069ce0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f94d80099b0 tx=0x7f94d8017000 comp rx=0 tx=0).stop 2023-10-06T02:31:52.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.699+0000 7f94e37ff700 1 -- 172.21.15.31:0/3611949608 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94dc0c8bd0 msgr2=0x7f94dc0abd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:52.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.699+0000 7f94e37ff700 1 --2- 172.21.15.31:0/3611949608 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94dc0c8bd0 0x7f94dc0abd70 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f94d4007ef0 tx=0x7f94d4009840 comp rx=0 tx=0).stop 2023-10-06T02:31:52.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.699+0000 7f94e37ff700 1 -- 172.21.15.31:0/3611949608 shutdown_connections 2023-10-06T02:31:52.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.699+0000 7f94e37ff700 1 --2- 172.21.15.31:0/3611949608 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94dc0c8bd0 0x7f94dc0abd70 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:52.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.699+0000 7f94e37ff700 1 --2- 172.21.15.31:0/3611949608 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94dc0b1150 0x7f94dc153020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:52.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.699+0000 7f94e37ff700 1 --2- 172.21.15.31:0/3611949608 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f94c0067830 0x7f94c0069ce0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:52.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.699+0000 7f94e37ff700 1 --2- 172.21.15.31:0/3611949608 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f94dc0c8800 0x7f94dc0c8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:52.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.699+0000 7f94e37ff700 1 -- 172.21.15.31:0/3611949608 >> 172.21.15.31:0/3611949608 conn(0x7f94dc06d170 msgr2=0x7f94dc0af1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:52.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.699+0000 7f94e37ff700 1 -- 172.21.15.31:0/3611949608 shutdown_connections 2023-10-06T02:31:52.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.699+0000 7f94e37ff700 1 -- 172.21.15.31:0/3611949608 wait complete. 2023-10-06T02:31:52.722 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:31:52.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.964+0000 7f54a939d700 1 Processor -- start 2023-10-06T02:31:52.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.964+0000 7f54a939d700 1 -- start start 2023-10-06T02:31:52.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.965+0000 7f54a939d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f54a4140a70 0x7f54a40c6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:52.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.965+0000 7f54a939d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f54a406de50 0x7f54a40aeef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:52.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.965+0000 7f54a939d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54a40c6d50 con 0x7f54a4140e40 2023-10-06T02:31:52.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.965+0000 7f54a939d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54a40c7030 con 0x7f54a406de50 2023-10-06T02:31:52.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.965+0000 7f54a939d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54a40c7310 con 0x7f54a4140a70 2023-10-06T02:31:52.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.965+0000 7f54a8b9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f54a406de50 0x7f54a40aeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:52.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.966+0000 7f54a8b9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f54a406de50 0x7f54a40aeef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56806/0 (socket says 172.21.15.31:56806) 2023-10-06T02:31:52.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.966+0000 7f54a8b9c700 1 -- 172.21.15.31:0/2236585439 learned_addr learned my addr 172.21.15.31:0/2236585439 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:52.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.966+0000 7f54a37fe700 1 --1- 172.21.15.31:0/2236585439 >> v1:172.21.15.62:6789/0 conn(0x7f54a4140e40 0x7f54a40c7df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:41766/0 (socket says 172.21.15.31:41766) 2023-10-06T02:31:52.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.966+0000 7f54a3fff700 1 --2- 172.21.15.31:0/2236585439 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f54a4140a70 0x7f54a40c6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:52.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.966+0000 7f54a8b9c700 1 -- 172.21.15.31:0/2236585439 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f54a4140a70 msgr2=0x7f54a40c6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:52.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.966+0000 7f54a8b9c700 1 --2- 172.21.15.31:0/2236585439 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f54a4140a70 0x7f54a40c6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:52.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.966+0000 7f54a8b9c700 1 -- 172.21.15.31:0/2236585439 >> v1:172.21.15.62:6789/0 conn(0x7f54a4140e40 legacy=0x7f54a40c7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:52.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.966+0000 7f54a8b9c700 1 -- 172.21.15.31:0/2236585439 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54a40c75f0 con 0x7f54a406de50 2023-10-06T02:31:52.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.966+0000 7f54a8b9c700 1 --2- 172.21.15.31:0/2236585439 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54a406de50 0x7f54a40aeef0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f549c009f20 tx=0x7f549c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3b8d30cc770126c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:52.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.966+0000 7f54a2ffd700 1 -- 172.21.15.31:0/2236585439 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549c00e070 con 0x7f54a406de50 2023-10-06T02:31:52.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.966+0000 7f54a2ffd700 1 -- 172.21.15.31:0/2236585439 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f549c00b9d0 con 0x7f54a406de50 2023-10-06T02:31:52.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.966+0000 7f54a2ffd700 1 -- 172.21.15.31:0/2236585439 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549c0092e0 con 0x7f54a406de50 2023-10-06T02:31:52.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.967+0000 7f54a939d700 1 -- 172.21.15.31:0/2236585439 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54a406de50 msgr2=0x7f54a40aeef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:53.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.967+0000 7f54a939d700 1 --2- 172.21.15.31:0/2236585439 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54a406de50 0x7f54a40aeef0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f549c009f20 tx=0x7f549c009840 comp rx=0 tx=0).stop 2023-10-06T02:31:53.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.967+0000 7f54a939d700 1 -- 172.21.15.31:0/2236585439 shutdown_connections 2023-10-06T02:31:53.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.967+0000 7f54a939d700 1 --2- 172.21.15.31:0/2236585439 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54a406de50 0x7f54a40aeef0 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:53.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.967+0000 7f54a939d700 1 --2- 172.21.15.31:0/2236585439 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f54a4140a70 0x7f54a40c6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:53.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.967+0000 7f54a939d700 1 -- 172.21.15.31:0/2236585439 >> 172.21.15.31:0/2236585439 conn(0x7f54a406d170 msgr2=0x7f54a40cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:53.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.967+0000 7f54a939d700 1 -- 172.21.15.31:0/2236585439 shutdown_connections 2023-10-06T02:31:53.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.967+0000 7f54a939d700 1 -- 172.21.15.31:0/2236585439 wait complete. 2023-10-06T02:31:53.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.969+0000 7f54a939d700 1 Processor -- start 2023-10-06T02:31:53.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.969+0000 7f54a939d700 1 -- start start 2023-10-06T02:31:53.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.970+0000 7f54a939d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f54a406de50 0x7f54a40b1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:53.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.970+0000 7f54a939d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54a4140a70 0x7f54a4132920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:53.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.970+0000 7f54a939d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54a4140e40 0x7f54a4069ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:53.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.970+0000 7f54a939d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54a40b0ec0 con 0x7f54a4140e40 2023-10-06T02:31:53.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.970+0000 7f54a939d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54a413f000 con 0x7f54a4140a70 2023-10-06T02:31:53.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.970+0000 7f54a939d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54a40b1ec0 con 0x7f54a406de50 2023-10-06T02:31:53.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.970+0000 7f54a8b9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54a4140e40 0x7f54a4069ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:53.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.970+0000 7f54a8b9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54a4140e40 0x7f54a4069ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56818/0 (socket says 172.21.15.31:56818) 2023-10-06T02:31:53.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.970+0000 7f54a8b9c700 1 -- 172.21.15.31:0/1591509130 learned_addr learned my addr 172.21.15.31:0/1591509130 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:53.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.970+0000 7f54a3fff700 1 --2- 172.21.15.31:0/1591509130 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f54a406de50 0x7f54a40b1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:53.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.970+0000 7f54a37fe700 1 --2- 172.21.15.31:0/1591509130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54a4140a70 0x7f54a4132920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:53.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.970+0000 7f54a8b9c700 1 -- 172.21.15.31:0/1591509130 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f54a406de50 msgr2=0x7f54a40b1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:53.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.970+0000 7f54a8b9c700 1 --2- 172.21.15.31:0/1591509130 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f54a406de50 0x7f54a40b1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:53.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.970+0000 7f54a8b9c700 1 -- 172.21.15.31:0/1591509130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54a4140a70 msgr2=0x7f54a4132920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:53.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.970+0000 7f54a8b9c700 1 --2- 172.21.15.31:0/1591509130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54a4140a70 0x7f54a4132920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:53.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.970+0000 7f54a8b9c700 1 -- 172.21.15.31:0/1591509130 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f549c009ae0 con 0x7f54a4140e40 2023-10-06T02:31:53.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.971+0000 7f54a8b9c700 1 --2- 172.21.15.31:0/1591509130 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54a4140e40 0x7f54a4069ac0 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f549c00b610 tx=0x7f549c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:53.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.971+0000 7f548bfff700 1 -- 172.21.15.31:0/1591509130 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549c008ad0 con 0x7f54a4140e40 2023-10-06T02:31:53.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.971+0000 7f548bfff700 1 -- 172.21.15.31:0/1591509130 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f549c008c30 con 0x7f54a4140e40 2023-10-06T02:31:53.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.971+0000 7f548bfff700 1 -- 172.21.15.31:0/1591509130 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549c026460 con 0x7f54a4140e40 2023-10-06T02:31:53.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.971+0000 7f54a939d700 1 -- 172.21.15.31:0/1591509130 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54a40b0340 con 0x7f54a4140e40 2023-10-06T02:31:53.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.971+0000 7f54a939d700 1 -- 172.21.15.31:0/1591509130 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54a40b0900 con 0x7f54a4140e40 2023-10-06T02:31:53.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.972+0000 7f54a939d700 1 -- 172.21.15.31:0/1591509130 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54a404c3c0 con 0x7f54a4140e40 2023-10-06T02:31:53.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.973+0000 7f548bfff700 1 -- 172.21.15.31:0/1591509130 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85556+0+0 (secure 0 0 0) 0x7f549c003040 con 0x7f54a4140e40 2023-10-06T02:31:53.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.974+0000 7f548bfff700 1 --2- 172.21.15.31:0/1591509130 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5490067830 0x7f5490069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:53.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.974+0000 7f548bfff700 1 -- 172.21.15.31:0/1591509130 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6173+0+0 (secure 0 0 0) 0x7f549c0846c0 con 0x7f54a4140e40 2023-10-06T02:31:53.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.974+0000 7f54a3fff700 1 --2- 172.21.15.31:0/1591509130 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5490067830 0x7f5490069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:53.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.975+0000 7f54a3fff700 1 --2- 172.21.15.31:0/1591509130 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5490067830 0x7f5490069ce0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f5494009db0 tx=0x7f54940091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:53.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:52.978+0000 7f548bfff700 1 -- 172.21.15.31:0/1591509130 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f549c053300 con 0x7f54a4140e40 2023-10-06T02:31:53.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.224+0000 7f54a939d700 1 -- 172.21.15.31:0/1591509130 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f54a40681e0 con 0x7f54a4140e40 2023-10-06T02:31:53.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.225+0000 7f548bfff700 1 -- 172.21.15.31:0/1591509130 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 110 v110) v1 ==== 95+0+4604 (secure 0 0 0) 0x7f549c056920 con 0x7f54a4140e40 2023-10-06T02:31:53.244 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:31:53.244 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":110,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7178,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/3218480514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3218480514},{"type":"v1","addr":"172.21.15.62:6834","nonce":3218480514}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104},{"gid":8323,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6833/832192876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":832192876},{"type":"v1","addr":"172.21.15.31:6833","nonce":832192876}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104},{"gid":8877,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/2374603733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":2374603733},{"type":"v1","addr":"172.21.15.31:6835","nonce":2374603733}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104}],"filesystems":[{"mdsmap":{"epoch":109,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:31:25.755801+0000","modified":"2023-10-06T02:31:29.266902+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8889},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8889":{"gid":8889,"name":"d","rank":0,"incarnation":105,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/1184425321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":1184425321},{"type":"v1","addr":"172.21.15.62:6835","nonce":1184425321}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2023-10-06T02:31:53.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.227+0000 7f54a939d700 1 -- 172.21.15.31:0/1591509130 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5490067830 msgr2=0x7f5490069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:53.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.227+0000 7f54a939d700 1 --2- 172.21.15.31:0/1591509130 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5490067830 0x7f5490069ce0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f5494009db0 tx=0x7f54940091a0 comp rx=0 tx=0).stop 2023-10-06T02:31:53.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.227+0000 7f54a939d700 1 -- 172.21.15.31:0/1591509130 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54a4140e40 msgr2=0x7f54a4069ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:53.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.227+0000 7f54a939d700 1 --2- 172.21.15.31:0/1591509130 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54a4140e40 0x7f54a4069ac0 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f549c00b610 tx=0x7f549c018000 comp rx=0 tx=0).stop 2023-10-06T02:31:53.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.227+0000 7f54a939d700 1 -- 172.21.15.31:0/1591509130 shutdown_connections 2023-10-06T02:31:53.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.227+0000 7f54a939d700 1 --2- 172.21.15.31:0/1591509130 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54a4140e40 0x7f54a4069ac0 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:53.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.227+0000 7f54a939d700 1 --2- 172.21.15.31:0/1591509130 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5490067830 0x7f5490069ce0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:53.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.227+0000 7f54a939d700 1 --2- 172.21.15.31:0/1591509130 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f54a406de50 0x7f54a40b1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:53.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.227+0000 7f54a939d700 1 --2- 172.21.15.31:0/1591509130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54a4140a70 0x7f54a4132920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:53.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.227+0000 7f54a939d700 1 -- 172.21.15.31:0/1591509130 >> 172.21.15.31:0/1591509130 conn(0x7f54a406d170 msgr2=0x7f54a40c9d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:53.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.227+0000 7f54a939d700 1 -- 172.21.15.31:0/1591509130 shutdown_connections 2023-10-06T02:31:53.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.227+0000 7f54a939d700 1 -- 172.21.15.31:0/1591509130 wait complete. 2023-10-06T02:31:53.246 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 110 2023-10-06T02:31:53.247 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:31:53.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.433+0000 7ff94a39d700 1 Processor -- start 2023-10-06T02:31:53.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.433+0000 7ff94a39d700 1 -- start start 2023-10-06T02:31:53.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.435+0000 7ff94a39d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff94413fef0 0x7ff94415a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:53.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.435+0000 7ff94a39d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff94406aaa0 0x7ff9440c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:53.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.435+0000 7ff94a39d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff94415ae70 con 0x7ff94413fb20 2023-10-06T02:31:53.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.435+0000 7ff94a39d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff94406af60 con 0x7ff94413fef0 2023-10-06T02:31:53.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.435+0000 7ff94a39d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff94406b240 con 0x7ff94406aaa0 2023-10-06T02:31:53.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.435+0000 7ff948b9a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff94413fef0 0x7ff94415a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:53.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.435+0000 7ff948b9a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff94413fef0 0x7ff94415a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56820/0 (socket says 172.21.15.31:56820) 2023-10-06T02:31:53.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.435+0000 7ff948b9a700 1 -- 172.21.15.31:0/1710547514 learned_addr learned my addr 172.21.15.31:0/1710547514 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:53.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.435+0000 7ff949b9c700 1 --2- 172.21.15.31:0/1710547514 >> v2:172.21.15.62:3300/0 conn(0x7ff94406aaa0 0x7ff9440c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:53.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.436+0000 7ff948b9a700 1 -- 172.21.15.31:0/1710547514 >> v2:172.21.15.62:3300/0 conn(0x7ff94406aaa0 msgr2=0x7ff9440c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:53.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.436+0000 7ff948b9a700 1 --2- 172.21.15.31:0/1710547514 >> v2:172.21.15.62:3300/0 conn(0x7ff94406aaa0 0x7ff9440c9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:53.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.436+0000 7ff948b9a700 1 -- 172.21.15.31:0/1710547514 >> v1:172.21.15.62:6789/0 conn(0x7ff94413fb20 legacy=0x7ff9440c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:53.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.436+0000 7ff948b9a700 1 -- 172.21.15.31:0/1710547514 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff94406b520 con 0x7ff94413fef0 2023-10-06T02:31:53.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.436+0000 7ff93bfff700 1 -- 172.21.15.31:0/1710547514 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 604310047 0 0) 0x7ff940001110 con 0x7ff94413fb20 2023-10-06T02:31:53.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.436+0000 7ff948b9a700 1 --2- 172.21.15.31:0/1710547514 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff94413fef0 0x7ff94415a930 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7ff944069d40 tx=0x7ff93400b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df63d51428e88b0f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:53.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.436+0000 7ff93bfff700 1 -- 172.21.15.31:0/1710547514 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff93400db70 con 0x7ff94413fef0 2023-10-06T02:31:53.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.436+0000 7ff93bfff700 1 -- 172.21.15.31:0/1710547514 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff93400dcd0 con 0x7ff94413fef0 2023-10-06T02:31:53.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.436+0000 7ff93bfff700 1 -- 172.21.15.31:0/1710547514 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9340196b0 con 0x7ff94413fef0 2023-10-06T02:31:53.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.437+0000 7ff94a39d700 1 -- 172.21.15.31:0/1710547514 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff94413fef0 msgr2=0x7ff94415a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:53.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.437+0000 7ff94a39d700 1 --2- 172.21.15.31:0/1710547514 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff94413fef0 0x7ff94415a930 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7ff944069d40 tx=0x7ff93400b4f0 comp rx=0 tx=0).stop 2023-10-06T02:31:53.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.437+0000 7ff94a39d700 1 -- 172.21.15.31:0/1710547514 shutdown_connections 2023-10-06T02:31:53.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.437+0000 7ff94a39d700 1 --2- 172.21.15.31:0/1710547514 >> v2:172.21.15.62:3300/0 conn(0x7ff94406aaa0 0x7ff9440c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:53.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.437+0000 7ff94a39d700 1 --2- 172.21.15.31:0/1710547514 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff94413fef0 0x7ff94415a930 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:53.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.437+0000 7ff94a39d700 1 -- 172.21.15.31:0/1710547514 >> 172.21.15.31:0/1710547514 conn(0x7ff94406d170 msgr2=0x7ff9440cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:53.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.437+0000 7ff94a39d700 1 -- 172.21.15.31:0/1710547514 shutdown_connections 2023-10-06T02:31:53.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.437+0000 7ff94a39d700 1 -- 172.21.15.31:0/1710547514 wait complete. 2023-10-06T02:31:53.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.438+0000 7ff94a39d700 1 Processor -- start 2023-10-06T02:31:53.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.438+0000 7ff94a39d700 1 -- start start 2023-10-06T02:31:53.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.439+0000 7ff94a39d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff94406aaa0 0x7ff9440aae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:53.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.439+0000 7ff94a39d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff94413fb20 0x7ff9440c75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:53.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.439+0000 7ff94a39d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff94413fef0 0x7ff9440c7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:53.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.439+0000 7ff94a39d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9440ab350 con 0x7ff94413fef0 2023-10-06T02:31:53.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.439+0000 7ff94a39d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9440ad390 con 0x7ff94406aaa0 2023-10-06T02:31:53.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.439+0000 7ff94a39d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9440ad670 con 0x7ff94413fb20 2023-10-06T02:31:53.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.440+0000 7ff949b9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff94413fef0 0x7ff9440c7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:53.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.440+0000 7ff949b9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff94413fef0 0x7ff9440c7b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56836/0 (socket says 172.21.15.31:56836) 2023-10-06T02:31:53.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.440+0000 7ff949b9c700 1 -- 172.21.15.31:0/384543776 learned_addr learned my addr 172.21.15.31:0/384543776 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:53.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.440+0000 7ff948b9a700 1 --2- 172.21.15.31:0/384543776 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff94413fb20 0x7ff9440c75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:53.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.440+0000 7ff94939b700 1 --2- 172.21.15.31:0/384543776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff94406aaa0 0x7ff9440aae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:53.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.440+0000 7ff949b9c700 1 -- 172.21.15.31:0/384543776 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff94413fb20 msgr2=0x7ff9440c75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:53.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.440+0000 7ff949b9c700 1 --2- 172.21.15.31:0/384543776 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff94413fb20 0x7ff9440c75f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:53.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.440+0000 7ff949b9c700 1 -- 172.21.15.31:0/384543776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff94406aaa0 msgr2=0x7ff9440aae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:53.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.440+0000 7ff949b9c700 1 --2- 172.21.15.31:0/384543776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff94406aaa0 0x7ff9440aae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:53.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.440+0000 7ff949b9c700 1 -- 172.21.15.31:0/384543776 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff93400b730 con 0x7ff94413fef0 2023-10-06T02:31:53.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.440+0000 7ff948b9a700 1 --2- 172.21.15.31:0/384543776 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff94413fb20 0x7ff9440c75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:31:53.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.440+0000 7ff949b9c700 1 --2- 172.21.15.31:0/384543776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff94413fef0 0x7ff9440c7b30 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7ff93c009ef0 tx=0x7ff93c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:53.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.441+0000 7ff9397fa700 1 -- 172.21.15.31:0/384543776 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff93c00cc20 con 0x7ff94413fef0 2023-10-06T02:31:53.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.441+0000 7ff9397fa700 1 -- 172.21.15.31:0/384543776 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff93c00f040 con 0x7ff94413fef0 2023-10-06T02:31:53.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.441+0000 7ff94a39d700 1 -- 172.21.15.31:0/384543776 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9440ad950 con 0x7ff94413fef0 2023-10-06T02:31:53.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.441+0000 7ff9397fa700 1 -- 172.21.15.31:0/384543776 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff93c014480 con 0x7ff94413fef0 2023-10-06T02:31:53.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.441+0000 7ff94a39d700 1 -- 172.21.15.31:0/384543776 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9440adf10 con 0x7ff94413fef0 2023-10-06T02:31:53.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.442+0000 7ff94a39d700 1 -- 172.21.15.31:0/384543776 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9440ae130 con 0x7ff94413fef0 2023-10-06T02:31:53.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.443+0000 7ff9397fa700 1 -- 172.21.15.31:0/384543776 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85556+0+0 (secure 0 0 0) 0x7ff93c014830 con 0x7ff94413fef0 2023-10-06T02:31:53.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.443+0000 7ff9397fa700 1 --2- 172.21.15.31:0/384543776 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff928067830 0x7ff928069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:53.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.444+0000 7ff9397fa700 1 -- 172.21.15.31:0/384543776 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6173+0+0 (secure 0 0 0) 0x7ff93c083be0 con 0x7ff94413fef0 2023-10-06T02:31:53.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.444+0000 7ff94939b700 1 --2- 172.21.15.31:0/384543776 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff928067830 0x7ff928069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:53.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.445+0000 7ff94939b700 1 --2- 172.21.15.31:0/384543776 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff928067830 0x7ff928069ce0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff940001370 tx=0x7ff940007ba0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:53.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.447+0000 7ff9397fa700 1 -- 172.21.15.31:0/384543776 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff93c052820 con 0x7ff94413fef0 2023-10-06T02:31:53.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.651+0000 7ff94a39d700 1 -- 172.21.15.31:0/384543776 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff9440648e0 con 0x7ff94413fef0 2023-10-06T02:31:53.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.652+0000 7ff9397fa700 1 -- 172.21.15.31:0/384543776 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v158) v1 ==== 74+0+17021 (secure 0 0 0) 0x7ff9440648e0 con 0x7ff94413fef0 2023-10-06T02:31:53.654 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:31:53.654 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":158,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:31:40.588975+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:31:23.814443+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2023-10-06T02:31:24.697823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"156","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"26.f","mappings":[{"from":1,"to":0}]},{"pgid":"26.17","mappings":[{"from":1,"to":0}]},{"pgid":"26.27","mappings":[{"from":3,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":20,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:31:53.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.654+0000 7ff94a39d700 1 -- 172.21.15.31:0/384543776 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff928067830 msgr2=0x7ff928069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:53.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.654+0000 7ff94a39d700 1 --2- 172.21.15.31:0/384543776 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff928067830 0x7ff928069ce0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff940001370 tx=0x7ff940007ba0 comp rx=0 tx=0).stop 2023-10-06T02:31:53.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.654+0000 7ff94a39d700 1 -- 172.21.15.31:0/384543776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff94413fef0 msgr2=0x7ff9440c7b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:53.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.654+0000 7ff94a39d700 1 --2- 172.21.15.31:0/384543776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff94413fef0 0x7ff9440c7b30 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7ff93c009ef0 tx=0x7ff93c00b040 comp rx=0 tx=0).stop 2023-10-06T02:31:53.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.654+0000 7ff94a39d700 1 -- 172.21.15.31:0/384543776 shutdown_connections 2023-10-06T02:31:53.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.654+0000 7ff94a39d700 1 --2- 172.21.15.31:0/384543776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff94413fef0 0x7ff9440c7b30 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:53.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.654+0000 7ff94a39d700 1 --2- 172.21.15.31:0/384543776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff94406aaa0 0x7ff9440aae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:53.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.654+0000 7ff94a39d700 1 --2- 172.21.15.31:0/384543776 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff928067830 0x7ff928069ce0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:53.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.654+0000 7ff94a39d700 1 --2- 172.21.15.31:0/384543776 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff94413fb20 0x7ff9440c75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:53.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.654+0000 7ff94a39d700 1 -- 172.21.15.31:0/384543776 >> 172.21.15.31:0/384543776 conn(0x7ff94406d170 msgr2=0x7ff94413e810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:53.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.654+0000 7ff94a39d700 1 -- 172.21.15.31:0/384543776 shutdown_connections 2023-10-06T02:31:53.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.654+0000 7ff94a39d700 1 -- 172.21.15.31:0/384543776 wait complete. 2023-10-06T02:31:53.673 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T02:31:53.674 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:31:53.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.859+0000 7f39e679d700 1 Processor -- start 2023-10-06T02:31:53.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.859+0000 7f39e679d700 1 -- start start 2023-10-06T02:31:53.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.860+0000 7f39e679d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39e0140a70 0x7f39e00c6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:53.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.860+0000 7f39e679d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f39e006de50 0x7f39e00aeef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:53.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.860+0000 7f39e679d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39e00c6d50 con 0x7f39e0140e40 2023-10-06T02:31:53.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.860+0000 7f39e679d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39e00c7030 con 0x7f39e006de50 2023-10-06T02:31:53.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.861+0000 7f39e679d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39e00c7310 con 0x7f39e0140a70 2023-10-06T02:31:53.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.861+0000 7f39e5f9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f39e006de50 0x7f39e00aeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:53.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.861+0000 7f39e5f9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f39e006de50 0x7f39e00aeef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56844/0 (socket says 172.21.15.31:56844) 2023-10-06T02:31:53.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.861+0000 7f39e5f9c700 1 -- 172.21.15.31:0/1618295302 learned_addr learned my addr 172.21.15.31:0/1618295302 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:53.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.861+0000 7f39e579b700 1 --2- 172.21.15.31:0/1618295302 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39e0140a70 0x7f39e00c6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:53.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.861+0000 7f39e5f9c700 1 -- 172.21.15.31:0/1618295302 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39e0140a70 msgr2=0x7f39e00c6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:53.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.861+0000 7f39e5f9c700 1 --2- 172.21.15.31:0/1618295302 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39e0140a70 0x7f39e00c6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:53.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.861+0000 7f39e5f9c700 1 -- 172.21.15.31:0/1618295302 >> v1:172.21.15.31:6789/0 conn(0x7f39e0140e40 legacy=0x7f39e00c7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:53.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.861+0000 7f39e5f9c700 1 -- 172.21.15.31:0/1618295302 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39e00c75f0 con 0x7f39e006de50 2023-10-06T02:31:53.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.861+0000 7f39e579b700 1 --2- 172.21.15.31:0/1618295302 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39e0140a70 0x7f39e00c6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:31:53.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.861+0000 7f39e5f9c700 1 --2- 172.21.15.31:0/1618295302 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39e006de50 0x7f39e00aeef0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f39d8009f20 tx=0x7f39d8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acd988c4c7e0d4ca server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:53.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.862+0000 7f39d7fff700 1 -- 172.21.15.31:0/1618295302 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d800e070 con 0x7f39e006de50 2023-10-06T02:31:53.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.862+0000 7f39d7fff700 1 -- 172.21.15.31:0/1618295302 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39d800ba80 con 0x7f39e006de50 2023-10-06T02:31:53.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.862+0000 7f39d7fff700 1 -- 172.21.15.31:0/1618295302 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d80092e0 con 0x7f39e006de50 2023-10-06T02:31:53.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.862+0000 7f39e679d700 1 -- 172.21.15.31:0/1618295302 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39e006de50 msgr2=0x7f39e00aeef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:53.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.862+0000 7f39e679d700 1 --2- 172.21.15.31:0/1618295302 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39e006de50 0x7f39e00aeef0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f39d8009f20 tx=0x7f39d8009840 comp rx=0 tx=0).stop 2023-10-06T02:31:53.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.862+0000 7f39e679d700 1 -- 172.21.15.31:0/1618295302 shutdown_connections 2023-10-06T02:31:53.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.862+0000 7f39e679d700 1 --2- 172.21.15.31:0/1618295302 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39e006de50 0x7f39e00aeef0 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:53.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.862+0000 7f39e679d700 1 --2- 172.21.15.31:0/1618295302 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39e0140a70 0x7f39e00c6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:53.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.862+0000 7f39e679d700 1 -- 172.21.15.31:0/1618295302 >> 172.21.15.31:0/1618295302 conn(0x7f39e006d170 msgr2=0x7f39e00cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:53.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.862+0000 7f39e679d700 1 -- 172.21.15.31:0/1618295302 shutdown_connections 2023-10-06T02:31:53.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.862+0000 7f39e679d700 1 -- 172.21.15.31:0/1618295302 wait complete. 2023-10-06T02:31:53.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.864+0000 7f39e679d700 1 Processor -- start 2023-10-06T02:31:53.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.864+0000 7f39e679d700 1 -- start start 2023-10-06T02:31:53.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.865+0000 7f39e679d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39e006de50 0x7f39e0156d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:53.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.865+0000 7f39e679d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39e0140a70 0x7f39e00c2450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:53.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.865+0000 7f39e679d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39e0140e40 0x7f39e00c2990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:53.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.865+0000 7f39e679d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e00a9950 con 0x7f39e0140a70 2023-10-06T02:31:53.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.865+0000 7f39e679d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e00c31e0 con 0x7f39e0140e40 2023-10-06T02:31:53.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.865+0000 7f39e679d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39e00c3a40 con 0x7f39e006de50 2023-10-06T02:31:53.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.865+0000 7f39e4f9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39e0140a70 0x7f39e00c2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:53.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.865+0000 7f39e4f9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39e0140a70 0x7f39e00c2450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56850/0 (socket says 172.21.15.31:56850) 2023-10-06T02:31:53.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.865+0000 7f39e4f9a700 1 -- 172.21.15.31:0/3785498407 learned_addr learned my addr 172.21.15.31:0/3785498407 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:53.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.865+0000 7f39e5f9c700 1 --2- 172.21.15.31:0/3785498407 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39e0140e40 0x7f39e00c2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:53.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.865+0000 7f39e579b700 1 --2- 172.21.15.31:0/3785498407 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39e006de50 0x7f39e0156d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:53.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.866+0000 7f39e4f9a700 1 -- 172.21.15.31:0/3785498407 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39e006de50 msgr2=0x7f39e0156d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:53.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.866+0000 7f39e4f9a700 1 --2- 172.21.15.31:0/3785498407 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39e006de50 0x7f39e0156d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:53.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.866+0000 7f39e4f9a700 1 -- 172.21.15.31:0/3785498407 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39e0140e40 msgr2=0x7f39e00c2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:53.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.866+0000 7f39e4f9a700 1 --2- 172.21.15.31:0/3785498407 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39e0140e40 0x7f39e00c2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:53.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.866+0000 7f39e4f9a700 1 -- 172.21.15.31:0/3785498407 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39d8009ae0 con 0x7f39e0140a70 2023-10-06T02:31:53.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.866+0000 7f39e4f9a700 1 --2- 172.21.15.31:0/3785498407 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39e0140a70 0x7f39e00c2450 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f39d000c540 tx=0x7f39d0007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:53.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.866+0000 7f39d57fa700 1 -- 172.21.15.31:0/3785498407 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d000f470 con 0x7f39e0140a70 2023-10-06T02:31:53.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.866+0000 7f39d57fa700 1 -- 172.21.15.31:0/3785498407 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39d000f5d0 con 0x7f39e0140a70 2023-10-06T02:31:53.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.866+0000 7f39d57fa700 1 -- 172.21.15.31:0/3785498407 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d000f990 con 0x7f39e0140a70 2023-10-06T02:31:53.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.866+0000 7f39e679d700 1 -- 172.21.15.31:0/3785498407 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39e016dfa0 con 0x7f39e0140a70 2023-10-06T02:31:53.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.866+0000 7f39e679d700 1 -- 172.21.15.31:0/3785498407 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39e00c36d0 con 0x7f39e0140a70 2023-10-06T02:31:53.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.867+0000 7f39e679d700 1 -- 172.21.15.31:0/3785498407 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39e004c3c0 con 0x7f39e0140a70 2023-10-06T02:31:53.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.868+0000 7f39d57fa700 1 -- 172.21.15.31:0/3785498407 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85556+0+0 (secure 0 0 0) 0x7f39d000fd00 con 0x7f39e0140a70 2023-10-06T02:31:53.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.870+0000 7f39d57fa700 1 --2- 172.21.15.31:0/3785498407 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f39c4067830 0x7f39c4069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:53.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.870+0000 7f39d57fa700 1 -- 172.21.15.31:0/3785498407 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6173+0+0 (secure 0 0 0) 0x7f39d0082310 con 0x7f39e0140a70 2023-10-06T02:31:53.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.870+0000 7f39e579b700 1 --2- 172.21.15.31:0/3785498407 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f39c4067830 0x7f39c4069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:53.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.871+0000 7f39e579b700 1 --2- 172.21.15.31:0/3785498407 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f39c4067830 0x7f39c4069ce0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f39dc00adc0 tx=0x7f39dc006f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:53.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:53.874+0000 7f39d57fa700 1 -- 172.21.15.31:0/3785498407 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f39d0050f50 con 0x7f39e0140a70 2023-10-06T02:31:54.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.117+0000 7f39e679d700 1 -- 172.21.15.31:0/3785498407 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f39e00c3b80 con 0x7f39e0140a70 2023-10-06T02:31:54.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.117+0000 7f39d57fa700 1 -- 172.21.15.31:0/3785498407 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 110 v110) v1 ==== 95+0+4604 (secure 0 0 0) 0x7f39d0017090 con 0x7f39e0140a70 2023-10-06T02:31:54.121 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:31:54.121 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":110,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7178,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/3218480514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3218480514},{"type":"v1","addr":"172.21.15.62:6834","nonce":3218480514}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104},{"gid":8323,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6833/832192876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":832192876},{"type":"v1","addr":"172.21.15.31:6833","nonce":832192876}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104},{"gid":8877,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/2374603733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":2374603733},{"type":"v1","addr":"172.21.15.31:6835","nonce":2374603733}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104}],"filesystems":[{"mdsmap":{"epoch":109,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:31:25.755801+0000","modified":"2023-10-06T02:31:29.266902+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8889},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8889":{"gid":8889,"name":"d","rank":0,"incarnation":105,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/1184425321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":1184425321},{"type":"v1","addr":"172.21.15.62:6835","nonce":1184425321}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2023-10-06T02:31:54.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.120+0000 7f39e679d700 1 -- 172.21.15.31:0/3785498407 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f39c4067830 msgr2=0x7f39c4069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:54.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.120+0000 7f39e679d700 1 --2- 172.21.15.31:0/3785498407 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f39c4067830 0x7f39c4069ce0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f39dc00adc0 tx=0x7f39dc006f90 comp rx=0 tx=0).stop 2023-10-06T02:31:54.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.120+0000 7f39e679d700 1 -- 172.21.15.31:0/3785498407 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39e0140a70 msgr2=0x7f39e00c2450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:54.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.120+0000 7f39e679d700 1 --2- 172.21.15.31:0/3785498407 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39e0140a70 0x7f39e00c2450 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f39d000c540 tx=0x7f39d0007f60 comp rx=0 tx=0).stop 2023-10-06T02:31:54.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.120+0000 7f39e679d700 1 -- 172.21.15.31:0/3785498407 shutdown_connections 2023-10-06T02:31:54.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.120+0000 7f39e679d700 1 --2- 172.21.15.31:0/3785498407 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39e0140e40 0x7f39e00c2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:54.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.120+0000 7f39e679d700 1 --2- 172.21.15.31:0/3785498407 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f39c4067830 0x7f39c4069ce0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:54.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.120+0000 7f39e679d700 1 --2- 172.21.15.31:0/3785498407 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39e006de50 0x7f39e0156d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:54.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.120+0000 7f39e679d700 1 --2- 172.21.15.31:0/3785498407 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39e0140a70 0x7f39e00c2450 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:54.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.120+0000 7f39e679d700 1 -- 172.21.15.31:0/3785498407 >> 172.21.15.31:0/3785498407 conn(0x7f39e006d170 msgr2=0x7f39e006e330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:54.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.120+0000 7f39e679d700 1 -- 172.21.15.31:0/3785498407 shutdown_connections 2023-10-06T02:31:54.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.120+0000 7f39e679d700 1 -- 172.21.15.31:0/3785498407 wait complete. 2023-10-06T02:31:54.124 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 110 2023-10-06T02:31:54.139 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:31:54.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.358+0000 7f40da99d700 1 Processor -- start 2023-10-06T02:31:54.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.358+0000 7f40da99d700 1 -- start start 2023-10-06T02:31:54.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.360+0000 7f40da99d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f40d4140a70 0x7f40d40c6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:54.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.360+0000 7f40da99d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f40d4140e40 0x7f40d40c7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:54.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.360+0000 7f40da99d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40d40c6d50 con 0x7f40d40c8330 2023-10-06T02:31:54.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.360+0000 7f40da99d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40d40c7030 con 0x7f40d4140a70 2023-10-06T02:31:54.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.360+0000 7f40da99d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40d40c7310 con 0x7f40d4140e40 2023-10-06T02:31:54.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.360+0000 7f40d999b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f40d4140a70 0x7f40d40c6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:54.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.360+0000 7f40d999b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f40d4140a70 0x7f40d40c6770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56854/0 (socket says 172.21.15.31:56854) 2023-10-06T02:31:54.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.360+0000 7f40d999b700 1 -- 172.21.15.31:0/2027704895 learned_addr learned my addr 172.21.15.31:0/2027704895 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:54.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.360+0000 7f40d999b700 1 -- 172.21.15.31:0/2027704895 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f40d4140e40 msgr2=0x7f40d40c7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:54.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.360+0000 7f40d919a700 1 --2- 172.21.15.31:0/2027704895 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f40d4140e40 0x7f40d40c7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:54.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.360+0000 7f40d999b700 1 --2- 172.21.15.31:0/2027704895 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f40d4140e40 0x7f40d40c7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:54.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.360+0000 7f40d999b700 1 -- 172.21.15.31:0/2027704895 >> v1:172.21.15.62:6789/0 conn(0x7f40d40c8330 legacy=0x7f40d40aeef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:54.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.361+0000 7f40d999b700 1 -- 172.21.15.31:0/2027704895 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40d40c75f0 con 0x7f40d4140a70 2023-10-06T02:31:54.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.361+0000 7f40d919a700 1 --2- 172.21.15.31:0/2027704895 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f40d4140e40 0x7f40d40c7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:31:54.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.361+0000 7f40d999b700 1 --2- 172.21.15.31:0/2027704895 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40d4140a70 0x7f40d40c6770 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f40d00099b0 tx=0x7f40d0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=404c2cf81109303d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:54.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.361+0000 7f40d8999700 1 -- 172.21.15.31:0/2027704895 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d000b860 con 0x7f40d4140a70 2023-10-06T02:31:54.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.361+0000 7f40d8999700 1 -- 172.21.15.31:0/2027704895 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40d000b9c0 con 0x7f40d4140a70 2023-10-06T02:31:54.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.361+0000 7f40d8999700 1 -- 172.21.15.31:0/2027704895 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d000bd60 con 0x7f40d4140a70 2023-10-06T02:31:54.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.362+0000 7f40da99d700 1 -- 172.21.15.31:0/2027704895 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40d4140a70 msgr2=0x7f40d40c6770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:54.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.362+0000 7f40da99d700 1 --2- 172.21.15.31:0/2027704895 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40d4140a70 0x7f40d40c6770 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f40d00099b0 tx=0x7f40d0009250 comp rx=0 tx=0).stop 2023-10-06T02:31:54.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.362+0000 7f40da99d700 1 -- 172.21.15.31:0/2027704895 shutdown_connections 2023-10-06T02:31:54.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.362+0000 7f40da99d700 1 --2- 172.21.15.31:0/2027704895 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40d4140a70 0x7f40d40c6770 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:54.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.362+0000 7f40da99d700 1 --2- 172.21.15.31:0/2027704895 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f40d4140e40 0x7f40d40c7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:54.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.362+0000 7f40da99d700 1 -- 172.21.15.31:0/2027704895 >> 172.21.15.31:0/2027704895 conn(0x7f40d406d170 msgr2=0x7f40d40cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:54.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.362+0000 7f40da99d700 1 -- 172.21.15.31:0/2027704895 shutdown_connections 2023-10-06T02:31:54.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.362+0000 7f40da99d700 1 -- 172.21.15.31:0/2027704895 wait complete. 2023-10-06T02:31:54.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.363+0000 7f40da99d700 1 Processor -- start 2023-10-06T02:31:54.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.363+0000 7f40da99d700 1 -- start start 2023-10-06T02:31:54.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.364+0000 7f40da99d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f40d40c8330 0x7f40d4156d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:54.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.364+0000 7f40da99d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40d4140a70 0x7f40d41590b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:54.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.364+0000 7f40da99d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40d4140e40 0x7f40d4167ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:54.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.364+0000 7f40da99d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d4157cb0 con 0x7f40d4140e40 2023-10-06T02:31:54.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.364+0000 7f40da99d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d4157980 con 0x7f40d4140a70 2023-10-06T02:31:54.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.364+0000 7f40da99d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40d4168440 con 0x7f40d40c8330 2023-10-06T02:31:54.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.365+0000 7f40da19c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40d4140e40 0x7f40d4167ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:54.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.365+0000 7f40da19c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40d4140e40 0x7f40d4167ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56862/0 (socket says 172.21.15.31:56862) 2023-10-06T02:31:54.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.365+0000 7f40da19c700 1 -- 172.21.15.31:0/4132717415 learned_addr learned my addr 172.21.15.31:0/4132717415 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:54.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.365+0000 7f40d999b700 1 --2- 172.21.15.31:0/4132717415 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f40d40c8330 0x7f40d4156d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:54.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.365+0000 7f40d919a700 1 --2- 172.21.15.31:0/4132717415 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40d4140a70 0x7f40d41590b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:54.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.365+0000 7f40da19c700 1 -- 172.21.15.31:0/4132717415 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f40d40c8330 msgr2=0x7f40d4156d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:54.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.365+0000 7f40da19c700 1 --2- 172.21.15.31:0/4132717415 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f40d40c8330 0x7f40d4156d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:54.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.365+0000 7f40da19c700 1 -- 172.21.15.31:0/4132717415 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40d4140a70 msgr2=0x7f40d41590b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:54.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.365+0000 7f40da19c700 1 --2- 172.21.15.31:0/4132717415 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40d4140a70 0x7f40d41590b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:54.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.365+0000 7f40da19c700 1 -- 172.21.15.31:0/4132717415 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40d00094f0 con 0x7f40d4140e40 2023-10-06T02:31:54.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.365+0000 7f40da19c700 1 --2- 172.21.15.31:0/4132717415 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40d4140e40 0x7f40d4167ad0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f40cc00c910 tx=0x7f40cc0016f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:54.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.366+0000 7f40c9ffb700 1 -- 172.21.15.31:0/4132717415 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40cc012630 con 0x7f40d4140e40 2023-10-06T02:31:54.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.366+0000 7f40c9ffb700 1 -- 172.21.15.31:0/4132717415 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40cc012790 con 0x7f40d4140e40 2023-10-06T02:31:54.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.366+0000 7f40c9ffb700 1 -- 172.21.15.31:0/4132717415 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40cc012b50 con 0x7f40d4140e40 2023-10-06T02:31:54.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.366+0000 7f40da99d700 1 -- 172.21.15.31:0/4132717415 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40d4168720 con 0x7f40d4140e40 2023-10-06T02:31:54.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.366+0000 7f40da99d700 1 -- 172.21.15.31:0/4132717415 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40d4168010 con 0x7f40d4140e40 2023-10-06T02:31:54.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.367+0000 7f40da99d700 1 -- 172.21.15.31:0/4132717415 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40d404c3c0 con 0x7f40d4140e40 2023-10-06T02:31:54.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.368+0000 7f40c9ffb700 1 -- 172.21.15.31:0/4132717415 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85556+0+0 (secure 0 0 0) 0x7f40cc01b450 con 0x7f40d4140e40 2023-10-06T02:31:54.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.368+0000 7f40c9ffb700 1 --2- 172.21.15.31:0/4132717415 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f40b8067860 0x7f40b8069d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:54.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.369+0000 7f40c9ffb700 1 -- 172.21.15.31:0/4132717415 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6173+0+0 (secure 0 0 0) 0x7f40cc082fe0 con 0x7f40d4140e40 2023-10-06T02:31:54.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.369+0000 7f40d999b700 1 --2- 172.21.15.31:0/4132717415 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f40b8067860 0x7f40b8069d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:54.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.370+0000 7f40d999b700 1 --2- 172.21.15.31:0/4132717415 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f40b8067860 0x7f40b8069d10 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f40d00099b0 tx=0x7f40d0017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:54.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.372+0000 7f40c9ffb700 1 -- 172.21.15.31:0/4132717415 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f40cc051c20 con 0x7f40d4140e40 2023-10-06T02:31:54.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.622+0000 7f40da99d700 1 -- 172.21.15.31:0/4132717415 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f40d40c78d0 con 0x7f40d4140e40 2023-10-06T02:31:54.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.623+0000 7f40c9ffb700 1 -- 172.21.15.31:0/4132717415 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 110 v110) v1 ==== 95+0+4604 (secure 0 0 0) 0x7f40cc016090 con 0x7f40d4140e40 2023-10-06T02:31:54.624 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:31:54.625 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":110,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7178,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/3218480514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3218480514},{"type":"v1","addr":"172.21.15.62:6834","nonce":3218480514}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104},{"gid":8323,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6833/832192876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":832192876},{"type":"v1","addr":"172.21.15.31:6833","nonce":832192876}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104},{"gid":8877,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/2374603733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":2374603733},{"type":"v1","addr":"172.21.15.31:6835","nonce":2374603733}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104}],"filesystems":[{"mdsmap":{"epoch":109,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:31:25.755801+0000","modified":"2023-10-06T02:31:29.266902+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8889},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8889":{"gid":8889,"name":"d","rank":0,"incarnation":105,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/1184425321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":1184425321},{"type":"v1","addr":"172.21.15.62:6835","nonce":1184425321}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2023-10-06T02:31:54.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.626+0000 7f40da99d700 1 -- 172.21.15.31:0/4132717415 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f40b8067860 msgr2=0x7f40b8069d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:54.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.626+0000 7f40da99d700 1 --2- 172.21.15.31:0/4132717415 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f40b8067860 0x7f40b8069d10 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f40d00099b0 tx=0x7f40d0017000 comp rx=0 tx=0).stop 2023-10-06T02:31:54.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.626+0000 7f40da99d700 1 -- 172.21.15.31:0/4132717415 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40d4140e40 msgr2=0x7f40d4167ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:54.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.626+0000 7f40da99d700 1 --2- 172.21.15.31:0/4132717415 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40d4140e40 0x7f40d4167ad0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f40cc00c910 tx=0x7f40cc0016f0 comp rx=0 tx=0).stop 2023-10-06T02:31:54.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.626+0000 7f40da99d700 1 -- 172.21.15.31:0/4132717415 shutdown_connections 2023-10-06T02:31:54.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.626+0000 7f40da99d700 1 --2- 172.21.15.31:0/4132717415 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40d4140e40 0x7f40d4167ad0 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:54.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.626+0000 7f40da99d700 1 --2- 172.21.15.31:0/4132717415 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f40b8067860 0x7f40b8069d10 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:54.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.626+0000 7f40da99d700 1 --2- 172.21.15.31:0/4132717415 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f40d40c8330 0x7f40d4156d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:54.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.626+0000 7f40da99d700 1 --2- 172.21.15.31:0/4132717415 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40d4140a70 0x7f40d41590b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:54.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.626+0000 7f40da99d700 1 -- 172.21.15.31:0/4132717415 >> 172.21.15.31:0/4132717415 conn(0x7f40d406d170 msgr2=0x7f40d40c27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:54.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.626+0000 7f40da99d700 1 -- 172.21.15.31:0/4132717415 shutdown_connections 2023-10-06T02:31:54.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.626+0000 7f40da99d700 1 -- 172.21.15.31:0/4132717415 wait complete. 2023-10-06T02:31:54.631 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 110 2023-10-06T02:31:54.645 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:31:54.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.830+0000 7f2d5159d700 1 Processor -- start 2023-10-06T02:31:54.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.830+0000 7f2d5159d700 1 -- start start 2023-10-06T02:31:54.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.831+0000 7f2d5159d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2d4c153c30 0x7f2d4c15a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:54.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.831+0000 7f2d5159d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2d4c15de50 0x7f2d4c0cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:54.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.831+0000 7f2d5159d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d4c0cf3c0 con 0x7f2d4c0cef40 2023-10-06T02:31:54.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.831+0000 7f2d5159d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d4c15afd0 con 0x7f2d4c15de50 2023-10-06T02:31:54.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.831+0000 7f2d5159d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d4c15b2b0 con 0x7f2d4c153c30 2023-10-06T02:31:54.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.832+0000 7f2d4b7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2d4c15de50 0x7f2d4c0cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:54.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.832+0000 7f2d4b7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2d4c15de50 0x7f2d4c0cea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56868/0 (socket says 172.21.15.31:56868) 2023-10-06T02:31:54.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.832+0000 7f2d4b7fe700 1 -- 172.21.15.31:0/2312767566 learned_addr learned my addr 172.21.15.31:0/2312767566 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:54.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.832+0000 7f2d4bfff700 1 --2- 172.21.15.31:0/2312767566 >> v2:172.21.15.62:3300/0 conn(0x7f2d4c153c30 0x7f2d4c15a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:54.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.832+0000 7f2d4b7fe700 1 -- 172.21.15.31:0/2312767566 >> v2:172.21.15.62:3300/0 conn(0x7f2d4c153c30 msgr2=0x7f2d4c15a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:54.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.832+0000 7f2d4b7fe700 1 --2- 172.21.15.31:0/2312767566 >> v2:172.21.15.62:3300/0 conn(0x7f2d4c153c30 0x7f2d4c15a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:54.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.832+0000 7f2d4b7fe700 1 -- 172.21.15.31:0/2312767566 >> v1:172.21.15.62:6789/0 conn(0x7f2d4c0cef40 legacy=0x7f2d4c06d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:54.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.832+0000 7f2d4b7fe700 1 -- 172.21.15.31:0/2312767566 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d4c15b590 con 0x7f2d4c15de50 2023-10-06T02:31:54.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.832+0000 7f2d4bfff700 1 --2- 172.21.15.31:0/2312767566 >> v2:172.21.15.62:3300/0 conn(0x7f2d4c153c30 0x7f2d4c15a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:31:54.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.832+0000 7f2d4b7fe700 1 --2- 172.21.15.31:0/2312767566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d4c15de50 0x7f2d4c0cea00 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c06ead0 tx=0x7f2d40009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91cfeb230b6f423a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:54.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.833+0000 7f2d4affd700 1 -- 172.21.15.31:0/2312767566 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d4000b860 con 0x7f2d4c15de50 2023-10-06T02:31:54.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.833+0000 7f2d4affd700 1 -- 172.21.15.31:0/2312767566 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d4000b9c0 con 0x7f2d4c15de50 2023-10-06T02:31:54.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.833+0000 7f2d4affd700 1 -- 172.21.15.31:0/2312767566 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d4000bd60 con 0x7f2d4c15de50 2023-10-06T02:31:54.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.833+0000 7f2d5159d700 1 -- 172.21.15.31:0/2312767566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d4c15de50 msgr2=0x7f2d4c0cea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:54.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.833+0000 7f2d5159d700 1 --2- 172.21.15.31:0/2312767566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d4c15de50 0x7f2d4c0cea00 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c06ead0 tx=0x7f2d40009250 comp rx=0 tx=0).stop 2023-10-06T02:31:54.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.833+0000 7f2d5159d700 1 -- 172.21.15.31:0/2312767566 shutdown_connections 2023-10-06T02:31:54.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.833+0000 7f2d5159d700 1 --2- 172.21.15.31:0/2312767566 >> v2:172.21.15.62:3300/0 conn(0x7f2d4c153c30 0x7f2d4c15a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:54.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.833+0000 7f2d5159d700 1 --2- 172.21.15.31:0/2312767566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d4c15de50 0x7f2d4c0cea00 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:54.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.833+0000 7f2d5159d700 1 -- 172.21.15.31:0/2312767566 >> 172.21.15.31:0/2312767566 conn(0x7f2d4c0703a0 msgr2=0x7f2d4c0707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:54.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.833+0000 7f2d5159d700 1 -- 172.21.15.31:0/2312767566 shutdown_connections 2023-10-06T02:31:54.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.833+0000 7f2d5159d700 1 -- 172.21.15.31:0/2312767566 wait complete. 2023-10-06T02:31:54.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.835+0000 7f2d5159d700 1 Processor -- start 2023-10-06T02:31:54.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.835+0000 7f2d5159d700 1 -- start start 2023-10-06T02:31:54.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.836+0000 7f2d5159d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d4c0cef40 0x7f2d4c13e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:54.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.836+0000 7f2d5159d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d4c153c30 0x7f2d4c0b1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:54.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.836+0000 7f2d5159d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2d4c15de50 0x7f2d4c0b32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:54.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.836+0000 7f2d5159d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4c0b61a0 con 0x7f2d4c153c30 2023-10-06T02:31:54.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.836+0000 7f2d5159d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4c0c8bf0 con 0x7f2d4c0cef40 2023-10-06T02:31:54.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.836+0000 7f2d5159d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4c13edf0 con 0x7f2d4c15de50 2023-10-06T02:31:54.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.836+0000 7f2d4b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d4c153c30 0x7f2d4c0b1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:54.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.836+0000 7f2d4b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d4c153c30 0x7f2d4c0b1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56870/0 (socket says 172.21.15.31:56870) 2023-10-06T02:31:54.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.836+0000 7f2d4b7fe700 1 -- 172.21.15.31:0/1150134147 learned_addr learned my addr 172.21.15.31:0/1150134147 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:54.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.836+0000 7f2d4bfff700 1 --2- 172.21.15.31:0/1150134147 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d4c0cef40 0x7f2d4c13e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:54.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.836+0000 7f2d50d9c700 1 --2- 172.21.15.31:0/1150134147 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2d4c15de50 0x7f2d4c0b32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:54.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.836+0000 7f2d4b7fe700 1 -- 172.21.15.31:0/1150134147 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2d4c15de50 msgr2=0x7f2d4c0b32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:54.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.836+0000 7f2d4b7fe700 1 --2- 172.21.15.31:0/1150134147 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2d4c15de50 0x7f2d4c0b32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:54.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.836+0000 7f2d4b7fe700 1 -- 172.21.15.31:0/1150134147 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d4c0cef40 msgr2=0x7f2d4c13e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:54.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.836+0000 7f2d4b7fe700 1 --2- 172.21.15.31:0/1150134147 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d4c0cef40 0x7f2d4c13e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:54.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.836+0000 7f2d4b7fe700 1 -- 172.21.15.31:0/1150134147 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d400094f0 con 0x7f2d4c153c30 2023-10-06T02:31:54.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.836+0000 7f2d50d9c700 1 --2- 172.21.15.31:0/1150134147 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2d4c15de50 0x7f2d4c0b32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:31:54.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.836+0000 7f2d4bfff700 1 --2- 172.21.15.31:0/1150134147 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d4c0cef40 0x7f2d4c13e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:31:54.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.837+0000 7f2d4b7fe700 1 --2- 172.21.15.31:0/1150134147 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d4c153c30 0x7f2d4c0b1640 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f2d400090b0 tx=0x7f2d40017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:54.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.837+0000 7f2d33fff700 1 -- 172.21.15.31:0/1150134147 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d40011e60 con 0x7f2d4c153c30 2023-10-06T02:31:54.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.837+0000 7f2d33fff700 1 -- 172.21.15.31:0/1150134147 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d4001a630 con 0x7f2d4c153c30 2023-10-06T02:31:54.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.837+0000 7f2d33fff700 1 -- 172.21.15.31:0/1150134147 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d4001aa10 con 0x7f2d4c153c30 2023-10-06T02:31:54.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.837+0000 7f2d5159d700 1 -- 172.21.15.31:0/1150134147 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d4c0c7fe0 con 0x7f2d4c153c30 2023-10-06T02:31:54.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.837+0000 7f2d5159d700 1 -- 172.21.15.31:0/1150134147 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d4c0c85a0 con 0x7f2d4c153c30 2023-10-06T02:31:54.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.838+0000 7f2d5159d700 1 -- 172.21.15.31:0/1150134147 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d4c0b2c30 con 0x7f2d4c153c30 2023-10-06T02:31:54.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.839+0000 7f2d33fff700 1 -- 172.21.15.31:0/1150134147 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85556+0+0 (secure 0 0 0) 0x7f2d4002d440 con 0x7f2d4c153c30 2023-10-06T02:31:54.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.840+0000 7f2d33fff700 1 --2- 172.21.15.31:0/1150134147 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f2d38067830 0x7f2d38069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:54.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.840+0000 7f2d33fff700 1 -- 172.21.15.31:0/1150134147 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6173+0+0 (secure 0 0 0) 0x7f2d40084a10 con 0x7f2d4c153c30 2023-10-06T02:31:54.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.840+0000 7f2d4bfff700 1 --2- 172.21.15.31:0/1150134147 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f2d38067830 0x7f2d38069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:54.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.841+0000 7f2d4bfff700 1 --2- 172.21.15.31:0/1150134147 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f2d38067830 0x7f2d38069ce0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c006f00 tx=0x7f2d3c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:54.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:54.844+0000 7f2d33fff700 1 -- 172.21.15.31:0/1150134147 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2d40053650 con 0x7f2d4c153c30 2023-10-06T02:31:55.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.048+0000 7f2d5159d700 1 -- 172.21.15.31:0/1150134147 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2d4c0c8d80 con 0x7f2d4c153c30 2023-10-06T02:31:55.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.049+0000 7f2d33fff700 1 -- 172.21.15.31:0/1150134147 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v158) v1 ==== 74+0+17021 (secure 0 0 0) 0x7f2d40056c70 con 0x7f2d4c153c30 2023-10-06T02:31:55.051 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:31:55.051 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":158,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:31:40.588975+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:31:23.814443+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2023-10-06T02:31:24.697823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"156","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"26.f","mappings":[{"from":1,"to":0}]},{"pgid":"26.17","mappings":[{"from":1,"to":0}]},{"pgid":"26.27","mappings":[{"from":3,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":20,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:31:55.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.051+0000 7f2d5159d700 1 -- 172.21.15.31:0/1150134147 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f2d38067830 msgr2=0x7f2d38069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:55.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.051+0000 7f2d5159d700 1 --2- 172.21.15.31:0/1150134147 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f2d38067830 0x7f2d38069ce0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c006f00 tx=0x7f2d3c008040 comp rx=0 tx=0).stop 2023-10-06T02:31:55.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.051+0000 7f2d5159d700 1 -- 172.21.15.31:0/1150134147 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d4c153c30 msgr2=0x7f2d4c0b1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:55.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.051+0000 7f2d5159d700 1 --2- 172.21.15.31:0/1150134147 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d4c153c30 0x7f2d4c0b1640 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f2d400090b0 tx=0x7f2d40017000 comp rx=0 tx=0).stop 2023-10-06T02:31:55.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.051+0000 7f2d5159d700 1 -- 172.21.15.31:0/1150134147 shutdown_connections 2023-10-06T02:31:55.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.051+0000 7f2d5159d700 1 --2- 172.21.15.31:0/1150134147 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f2d38067830 0x7f2d38069ce0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:55.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.051+0000 7f2d5159d700 1 --2- 172.21.15.31:0/1150134147 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2d4c15de50 0x7f2d4c0b32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:55.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.051+0000 7f2d5159d700 1 --2- 172.21.15.31:0/1150134147 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d4c0cef40 0x7f2d4c13e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:55.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.051+0000 7f2d5159d700 1 --2- 172.21.15.31:0/1150134147 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d4c153c30 0x7f2d4c0b1640 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:55.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.051+0000 7f2d5159d700 1 -- 172.21.15.31:0/1150134147 >> 172.21.15.31:0/1150134147 conn(0x7f2d4c0703a0 msgr2=0x7f2d4c153280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:55.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.051+0000 7f2d5159d700 1 -- 172.21.15.31:0/1150134147 shutdown_connections 2023-10-06T02:31:55.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.051+0000 7f2d5159d700 1 -- 172.21.15.31:0/1150134147 wait complete. 2023-10-06T02:31:55.071 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T02:31:55.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.281+0000 7f618d19d700 1 Processor -- start 2023-10-06T02:31:55.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.282+0000 7f618d19d700 1 -- start start 2023-10-06T02:31:55.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.283+0000 7f618d19d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f6188142c00 0x7f618806b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:55.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.283+0000 7f618d19d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f618806bd20 0x7f61880c7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:55.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.283+0000 7f618d19d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61880ca480 con 0x7f6188142fd0 2023-10-06T02:31:55.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.283+0000 7f618d19d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f618815e240 con 0x7f6188142c00 2023-10-06T02:31:55.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.283+0000 7f618d19d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f618806b350 con 0x7f618806bd20 2023-10-06T02:31:55.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.283+0000 7f61877fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f618806bd20 0x7f61880c7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:55.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.283+0000 7f61877fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f618806bd20 0x7f61880c7880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:47928/0 (socket says 172.21.15.31:47928) 2023-10-06T02:31:55.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.283+0000 7f61877fe700 1 -- 172.21.15.31:0/764186303 learned_addr learned my addr 172.21.15.31:0/764186303 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:55.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.283+0000 7f6187fff700 1 --2- 172.21.15.31:0/764186303 >> v2:172.21.15.62:3300/0 conn(0x7f6188142c00 0x7f618806b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:55.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.284+0000 7f61877fe700 1 -- 172.21.15.31:0/764186303 >> v2:172.21.15.62:3300/0 conn(0x7f6188142c00 msgr2=0x7f618806b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:55.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.284+0000 7f61877fe700 1 --2- 172.21.15.31:0/764186303 >> v2:172.21.15.62:3300/0 conn(0x7f6188142c00 0x7f618806b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:55.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.284+0000 7f61877fe700 1 -- 172.21.15.31:0/764186303 >> v1:172.21.15.62:6789/0 conn(0x7f6188142fd0 legacy=0x7f618806c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:55.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.284+0000 7f61877fe700 1 -- 172.21.15.31:0/764186303 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f618815df60 con 0x7f618806bd20 2023-10-06T02:31:55.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.284+0000 7f6187fff700 1 --2- 172.21.15.31:0/764186303 >> v2:172.21.15.62:3300/0 conn(0x7f6188142c00 0x7f618806b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:31:55.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.284+0000 7f61877fe700 1 --2- 172.21.15.31:0/764186303 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f618806bd20 0x7f61880c7880 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f617c0099b0 tx=0x7f617c009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cafc2479338fd6ad server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:55.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.284+0000 7f6186ffd700 1 -- 172.21.15.31:0/764186303 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f617c009ea0 con 0x7f618806bd20 2023-10-06T02:31:55.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.285+0000 7f6186ffd700 1 -- 172.21.15.31:0/764186303 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f617c00b860 con 0x7f618806bd20 2023-10-06T02:31:55.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.285+0000 7f6186ffd700 1 -- 172.21.15.31:0/764186303 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f617c00bc20 con 0x7f618806bd20 2023-10-06T02:31:55.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.285+0000 7f618d19d700 1 -- 172.21.15.31:0/764186303 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f618806bd20 msgr2=0x7f61880c7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:55.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.285+0000 7f618d19d700 1 --2- 172.21.15.31:0/764186303 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f618806bd20 0x7f61880c7880 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f617c0099b0 tx=0x7f617c009250 comp rx=0 tx=0).stop 2023-10-06T02:31:55.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.285+0000 7f618d19d700 1 -- 172.21.15.31:0/764186303 shutdown_connections 2023-10-06T02:31:55.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.285+0000 7f618d19d700 1 --2- 172.21.15.31:0/764186303 >> v2:172.21.15.62:3300/0 conn(0x7f6188142c00 0x7f618806b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:55.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.285+0000 7f618d19d700 1 --2- 172.21.15.31:0/764186303 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f618806bd20 0x7f61880c7880 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:55.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.285+0000 7f618d19d700 1 -- 172.21.15.31:0/764186303 >> 172.21.15.31:0/764186303 conn(0x7f61880675a0 msgr2=0x7f61880679a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:55.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.285+0000 7f618d19d700 1 -- 172.21.15.31:0/764186303 shutdown_connections 2023-10-06T02:31:55.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.285+0000 7f618d19d700 1 -- 172.21.15.31:0/764186303 wait complete. 2023-10-06T02:31:55.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.287+0000 7f618d19d700 1 Processor -- start 2023-10-06T02:31:55.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.287+0000 7f618d19d700 1 -- start start 2023-10-06T02:31:55.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.288+0000 7f618d19d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f618806bd20 0x7f6188159e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:55.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.288+0000 7f618d19d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6188142c00 0x7f6188157f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:55.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.288+0000 7f618d19d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6188142fd0 0x7f6188140000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:55.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.288+0000 7f618d19d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f618813ae10 con 0x7f6188142fd0 2023-10-06T02:31:55.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.288+0000 7f618d19d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61880b2900 con 0x7f618806bd20 2023-10-06T02:31:55.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.288+0000 7f618d19d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f618813a7c0 con 0x7f6188142c00 2023-10-06T02:31:55.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.288+0000 7f618c99c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6188142fd0 0x7f6188140000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:55.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.288+0000 7f618c99c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6188142fd0 0x7f6188140000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56876/0 (socket says 172.21.15.31:56876) 2023-10-06T02:31:55.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.288+0000 7f618c99c700 1 -- 172.21.15.31:0/775813808 learned_addr learned my addr 172.21.15.31:0/775813808 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:55.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.288+0000 7f6187fff700 1 --2- 172.21.15.31:0/775813808 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f618806bd20 0x7f6188159e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:55.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.288+0000 7f61877fe700 1 --2- 172.21.15.31:0/775813808 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6188142c00 0x7f6188157f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:55.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.288+0000 7f618c99c700 1 -- 172.21.15.31:0/775813808 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6188142c00 msgr2=0x7f6188157f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:55.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.288+0000 7f618c99c700 1 --2- 172.21.15.31:0/775813808 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6188142c00 0x7f6188157f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:55.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.288+0000 7f618c99c700 1 -- 172.21.15.31:0/775813808 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f618806bd20 msgr2=0x7f6188159e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:55.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.288+0000 7f618c99c700 1 --2- 172.21.15.31:0/775813808 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f618806bd20 0x7f6188159e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:55.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.288+0000 7f618c99c700 1 -- 172.21.15.31:0/775813808 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6178009420 con 0x7f6188142fd0 2023-10-06T02:31:55.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.289+0000 7f618c99c700 1 --2- 172.21.15.31:0/775813808 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6188142fd0 0x7f6188140000 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f618000c840 tx=0x7f6180001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:55.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.289+0000 7f616ffff700 1 -- 172.21.15.31:0/775813808 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6180012630 con 0x7f6188142fd0 2023-10-06T02:31:55.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.289+0000 7f616ffff700 1 -- 172.21.15.31:0/775813808 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61800127b0 con 0x7f6188142fd0 2023-10-06T02:31:55.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.289+0000 7f618d19d700 1 -- 172.21.15.31:0/775813808 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f617c0094f0 con 0x7f6188142fd0 2023-10-06T02:31:55.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.289+0000 7f616ffff700 1 -- 172.21.15.31:0/775813808 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6180012b70 con 0x7f6188142fd0 2023-10-06T02:31:55.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.289+0000 7f618d19d700 1 -- 172.21.15.31:0/775813808 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f618815a360 con 0x7f6188142fd0 2023-10-06T02:31:55.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.290+0000 7f618d19d700 1 -- 172.21.15.31:0/775813808 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f618813a620 con 0x7f6188142fd0 2023-10-06T02:31:55.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.291+0000 7f616ffff700 1 -- 172.21.15.31:0/775813808 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85556+0+0 (secure 0 0 0) 0x7f618001a3a0 con 0x7f6188142fd0 2023-10-06T02:31:55.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.292+0000 7f616ffff700 1 --2- 172.21.15.31:0/775813808 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6174067830 0x7f6174069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:55.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.292+0000 7f616ffff700 1 -- 172.21.15.31:0/775813808 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6173+0+0 (secure 0 0 0) 0x7f6180082fb0 con 0x7f6188142fd0 2023-10-06T02:31:55.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.292+0000 7f6187fff700 1 --2- 172.21.15.31:0/775813808 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6174067830 0x7f6174069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:55.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.293+0000 7f6187fff700 1 --2- 172.21.15.31:0/775813808 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6174067830 0x7f6174069ce0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f6178009f40 tx=0x7f61780091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:55.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.296+0000 7f616ffff700 1 -- 172.21.15.31:0/775813808 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f618813a620 con 0x7f6188142fd0 2023-10-06T02:31:55.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.544+0000 7f618d19d700 1 -- 172.21.15.31:0/775813808 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f618813a620 con 0x7f6188142fd0 2023-10-06T02:31:55.546 INFO:tasks.ceph.mon.a.smithi031.stderr:2023-10-06T02:31:55.545+0000 7f6de01f1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T02:31:55.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.550+0000 7f616ffff700 1 -- 172.21.15.31:0/775813808 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v111) v1 ==== 161+0+0 (secure 0 0 0) 0x7f618813a620 con 0x7f6188142fd0 2023-10-06T02:31:55.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.552+0000 7f618d19d700 1 -- 172.21.15.31:0/775813808 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6174067830 msgr2=0x7f6174069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:55.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.552+0000 7f618d19d700 1 --2- 172.21.15.31:0/775813808 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6174067830 0x7f6174069ce0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f6178009f40 tx=0x7f61780091a0 comp rx=0 tx=0).stop 2023-10-06T02:31:55.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.552+0000 7f618d19d700 1 -- 172.21.15.31:0/775813808 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6188142fd0 msgr2=0x7f6188140000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:55.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.552+0000 7f618d19d700 1 --2- 172.21.15.31:0/775813808 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6188142fd0 0x7f6188140000 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f618000c840 tx=0x7f6180001670 comp rx=0 tx=0).stop 2023-10-06T02:31:55.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.552+0000 7f618d19d700 1 -- 172.21.15.31:0/775813808 shutdown_connections 2023-10-06T02:31:55.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.552+0000 7f618d19d700 1 --2- 172.21.15.31:0/775813808 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6188142fd0 0x7f6188140000 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:55.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.553+0000 7f618d19d700 1 --2- 172.21.15.31:0/775813808 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f618806bd20 0x7f6188159e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:55.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.553+0000 7f618d19d700 1 --2- 172.21.15.31:0/775813808 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6174067830 0x7f6174069ce0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:55.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.553+0000 7f618d19d700 1 --2- 172.21.15.31:0/775813808 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6188142c00 0x7f6188157f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:55.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.553+0000 7f618d19d700 1 -- 172.21.15.31:0/775813808 >> 172.21.15.31:0/775813808 conn(0x7f61880675a0 msgr2=0x7f6188158e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:55.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.553+0000 7f618d19d700 1 -- 172.21.15.31:0/775813808 shutdown_connections 2023-10-06T02:31:55.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.553+0000 7f618d19d700 1 -- 172.21.15.31:0/775813808 wait complete. 2023-10-06T02:31:55.556 INFO:teuthology.orchestra.run.smithi031.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T02:31:55.572 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T02:31:55.617 INFO:tasks.ceph.mds.d.smithi062.stdout:starting mds.d at 2023-10-06T02:31:55.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.761+0000 7f0fef19d700 1 Processor -- start 2023-10-06T02:31:55.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.762+0000 7f0fef19d700 1 -- start start 2023-10-06T02:31:55.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.763+0000 7f0fef19d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0fe80aaed0 0x7f0fe806e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:55.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.763+0000 7f0fef19d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fe80ca980 con 0x7f0fe80accb0 2023-10-06T02:31:55.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.763+0000 7f0fef19d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fe80ad780 con 0x7f0fe80cc5b0 2023-10-06T02:31:55.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.763+0000 7f0fef19d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fe80ab2d0 con 0x7f0fe80aaed0 2023-10-06T02:31:55.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.763+0000 7f0fee19b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f0fe80accb0 0x7f0fe80cbea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46524/0 (socket says 172.21.15.31:46524) 2023-10-06T02:31:55.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.763+0000 7f0fee19b700 1 -- 172.21.15.31:0/4260237032 learned_addr learned my addr 172.21.15.31:0/4260237032 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:31:55.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.763+0000 7f0fee99c700 1 --2- 172.21.15.31:0/4260237032 >> v2:172.21.15.31:3300/0 conn(0x7f0fe80aaed0 0x7f0fe806e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:55.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.763+0000 7f0fed199700 1 -- 172.21.15.31:0/4260237032 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3328326768 0 0) 0x7f0fe80ca980 con 0x7f0fe80accb0 2023-10-06T02:31:55.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.763+0000 7f0fed199700 1 -- 172.21.15.31:0/4260237032 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0fd4003570 con 0x7f0fe80accb0 2023-10-06T02:31:55.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.764+0000 7f0fee99c700 1 -- 172.21.15.31:0/4260237032 >> v1:172.21.15.62:6789/0 conn(0x7f0fe80cc5b0 legacy=0x7f0fe80ca270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:55.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.764+0000 7f0fee99c700 1 -- 172.21.15.31:0/4260237032 >> v1:172.21.15.31:6789/0 conn(0x7f0fe80accb0 legacy=0x7f0fe80cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:55.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.764+0000 7f0fee99c700 1 -- 172.21.15.31:0/4260237032 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fe80cee60 con 0x7f0fe80aaed0 2023-10-06T02:31:55.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.764+0000 7f0fee99c700 1 --2- 172.21.15.31:0/4260237032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0fe80aaed0 0x7f0fe806e330 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f0fe0009e60 tx=0x7f0fe0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=305a47132c77f963 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:55.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.764+0000 7f0fed199700 1 -- 172.21.15.31:0/4260237032 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe000e070 con 0x7f0fe80aaed0 2023-10-06T02:31:55.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.764+0000 7f0fed199700 1 -- 172.21.15.31:0/4260237032 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fe000bb60 con 0x7f0fe80aaed0 2023-10-06T02:31:55.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.764+0000 7f0fed199700 1 -- 172.21.15.31:0/4260237032 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe0012430 con 0x7f0fe80aaed0 2023-10-06T02:31:55.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.765+0000 7f0fef19d700 1 -- 172.21.15.31:0/4260237032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0fe80aaed0 msgr2=0x7f0fe806e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:55.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.765+0000 7f0fef19d700 1 --2- 172.21.15.31:0/4260237032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0fe80aaed0 0x7f0fe806e330 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f0fe0009e60 tx=0x7f0fe0009780 comp rx=0 tx=0).stop 2023-10-06T02:31:55.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.765+0000 7f0fef19d700 1 -- 172.21.15.31:0/4260237032 shutdown_connections 2023-10-06T02:31:55.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.765+0000 7f0fef19d700 1 --2- 172.21.15.31:0/4260237032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0fe80aaed0 0x7f0fe806e330 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:55.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.765+0000 7f0fef19d700 1 -- 172.21.15.31:0/4260237032 >> 172.21.15.31:0/4260237032 conn(0x7f0fe806ce80 msgr2=0x7f0fe80ce370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:55.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.765+0000 7f0fef19d700 1 -- 172.21.15.31:0/4260237032 shutdown_connections 2023-10-06T02:31:55.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.765+0000 7f0fef19d700 1 -- 172.21.15.31:0/4260237032 wait complete. 2023-10-06T02:31:55.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.766+0000 7f0fef19d700 1 Processor -- start 2023-10-06T02:31:55.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.767+0000 7f0fef19d700 1 -- start start 2023-10-06T02:31:55.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.767+0000 7f0fef19d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0fe80aaed0 0x7f0fe8169910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:55.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.767+0000 7f0fef19d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0fe80accb0 0x7f0fe8157600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:55.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.767+0000 7f0fef19d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0fe80cc5b0 0x7f0fe80ca300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:55.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.767+0000 7f0fef19d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe80c2d30 con 0x7f0fe80accb0 2023-10-06T02:31:55.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.767+0000 7f0fef19d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe80c3ad0 con 0x7f0fe80cc5b0 2023-10-06T02:31:55.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.767+0000 7f0fef19d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe80c2a50 con 0x7f0fe80aaed0 2023-10-06T02:31:55.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.768+0000 7f0fed99a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0fe80accb0 0x7f0fe8157600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:55.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.768+0000 7f0fed99a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0fe80accb0 0x7f0fe8157600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56888/0 (socket says 172.21.15.31:56888) 2023-10-06T02:31:55.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.768+0000 7f0fed99a700 1 -- 172.21.15.31:0/857317467 learned_addr learned my addr 172.21.15.31:0/857317467 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:55.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.768+0000 7f0fee19b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0fe80aaed0 0x7f0fe8169910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:55.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.768+0000 7f0fee99c700 1 --2- 172.21.15.31:0/857317467 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0fe80cc5b0 0x7f0fe80ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:55.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.768+0000 7f0fed99a700 1 -- 172.21.15.31:0/857317467 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0fe80aaed0 msgr2=0x7f0fe8169910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:55.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.768+0000 7f0fed99a700 1 --2- 172.21.15.31:0/857317467 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0fe80aaed0 0x7f0fe8169910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:55.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.768+0000 7f0fed99a700 1 -- 172.21.15.31:0/857317467 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0fe80cc5b0 msgr2=0x7f0fe80ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:55.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.768+0000 7f0fed99a700 1 --2- 172.21.15.31:0/857317467 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0fe80cc5b0 0x7f0fe80ca300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:55.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.768+0000 7f0fed99a700 1 -- 172.21.15.31:0/857317467 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fe0009a20 con 0x7f0fe80accb0 2023-10-06T02:31:55.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.768+0000 7f0fed99a700 1 --2- 172.21.15.31:0/857317467 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0fe80accb0 0x7f0fe8157600 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f0fd800c590 tx=0x7f0fd8007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:55.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.769+0000 7f0fde7fc700 1 -- 172.21.15.31:0/857317467 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd8011590 con 0x7f0fe80accb0 2023-10-06T02:31:55.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.769+0000 7f0fde7fc700 1 -- 172.21.15.31:0/857317467 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fd8011710 con 0x7f0fe80accb0 2023-10-06T02:31:55.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.769+0000 7f0fde7fc700 1 -- 172.21.15.31:0/857317467 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd8011ad0 con 0x7f0fe80accb0 2023-10-06T02:31:55.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.769+0000 7f0fef19d700 1 -- 172.21.15.31:0/857317467 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fe80c3480 con 0x7f0fe80accb0 2023-10-06T02:31:55.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.769+0000 7f0fef19d700 1 -- 172.21.15.31:0/857317467 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fe8157b40 con 0x7f0fe80accb0 2023-10-06T02:31:55.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.770+0000 7f0fef19d700 1 -- 172.21.15.31:0/857317467 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fe80c3c40 con 0x7f0fe80accb0 2023-10-06T02:31:55.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.771+0000 7f0fde7fc700 1 -- 172.21.15.31:0/857317467 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85556+0+0 (secure 0 0 0) 0x7f0fd801a360 con 0x7f0fe80accb0 2023-10-06T02:31:55.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.772+0000 7f0fde7fc700 1 --2- 172.21.15.31:0/857317467 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0fcc067830 0x7f0fcc069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:55.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.772+0000 7f0fde7fc700 1 -- 172.21.15.31:0/857317467 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6259+0+0 (secure 0 0 0) 0x7f0fd8081ce0 con 0x7f0fe80accb0 2023-10-06T02:31:55.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.772+0000 7f0fee19b700 1 --2- 172.21.15.31:0/857317467 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0fcc067830 0x7f0fcc069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:55.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.773+0000 7f0fee19b700 1 --2- 172.21.15.31:0/857317467 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0fcc067830 0x7f0fcc069ce0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f0fe4007fd0 tx=0x7f0fe4009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:55.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:55.776+0000 7f0fde7fc700 1 -- 172.21.15.31:0/857317467 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0fd8050920 con 0x7f0fe80accb0 2023-10-06T02:31:56.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:56.022+0000 7f0fef19d700 1 -- 172.21.15.31:0/857317467 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f0fe804c3c0 con 0x7f0fe80accb0 2023-10-06T02:31:56.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:56.550+0000 7f0fde7fc700 1 -- 172.21.15.31:0/857317467 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v112) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0fd8053f40 con 0x7f0fe80accb0 2023-10-06T02:31:56.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:56.553+0000 7f0fef19d700 1 -- 172.21.15.31:0/857317467 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0fcc067830 msgr2=0x7f0fcc069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:56.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:56.553+0000 7f0fef19d700 1 --2- 172.21.15.31:0/857317467 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0fcc067830 0x7f0fcc069ce0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f0fe4007fd0 tx=0x7f0fe4009040 comp rx=0 tx=0).stop 2023-10-06T02:31:56.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:56.553+0000 7f0fef19d700 1 -- 172.21.15.31:0/857317467 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0fe80accb0 msgr2=0x7f0fe8157600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:56.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:56.553+0000 7f0fef19d700 1 --2- 172.21.15.31:0/857317467 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0fe80accb0 0x7f0fe8157600 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f0fd800c590 tx=0x7f0fd8007f50 comp rx=0 tx=0).stop 2023-10-06T02:31:56.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:56.553+0000 7f0fef19d700 1 -- 172.21.15.31:0/857317467 shutdown_connections 2023-10-06T02:31:56.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:56.553+0000 7f0fef19d700 1 --2- 172.21.15.31:0/857317467 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0fe80cc5b0 0x7f0fe80ca300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:56.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:56.553+0000 7f0fef19d700 1 --2- 172.21.15.31:0/857317467 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0fcc067830 0x7f0fcc069ce0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:56.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:56.553+0000 7f0fef19d700 1 --2- 172.21.15.31:0/857317467 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0fe80aaed0 0x7f0fe8169910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:56.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:56.553+0000 7f0fef19d700 1 --2- 172.21.15.31:0/857317467 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0fe80accb0 0x7f0fe8157600 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:56.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:56.553+0000 7f0fef19d700 1 -- 172.21.15.31:0/857317467 >> 172.21.15.31:0/857317467 conn(0x7f0fe806ce80 msgr2=0x7f0fe8159550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:56.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:56.553+0000 7f0fef19d700 1 -- 172.21.15.31:0/857317467 shutdown_connections 2023-10-06T02:31:56.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:56.553+0000 7f0fef19d700 1 -- 172.21.15.31:0/857317467 wait complete. 2023-10-06T02:31:56.575 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T02:31:56.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:56.767+0000 7f40a359e700 1 Processor -- start 2023-10-06T02:31:56.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:56.767+0000 7f40a359e700 1 -- start start 2023-10-06T02:31:56.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:56.768+0000 7f40a359e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f40a4153660 0x7f40a40c0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:56.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:56.768+0000 7f40a359e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f40a40c9090 0x7f40a4156d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:56.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:56.768+0000 7f40a359e700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40a40c2200 con 0x7f40a4153a30 2023-10-06T02:31:56.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:56.768+0000 7f40a359e700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40a4157250 con 0x7f40a40c9090 2023-10-06T02:31:56.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:56.768+0000 7f40a359e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40a40c15f0 con 0x7f40a4153660 2023-10-06T02:31:56.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:56.768+0000 7f40a2d9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f40a40c9090 0x7f40a4156d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:56.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:56.768+0000 7f40a1d9b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f40a4153a30 0x7f40a40c8980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46538/0 (socket says 172.21.15.31:46538) 2023-10-06T02:31:56.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:56.768+0000 7f40a1d9b700 1 -- 172.21.15.31:0/2962160993 learned_addr learned my addr 172.21.15.31:0/2962160993 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:31:56.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:56.769+0000 7f40a259c700 1 --2- 172.21.15.31:0/2962160993 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f40a4153660 0x7f40a40c0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:56.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:56.769+0000 7f40a2d9d700 1 -- 172.21.15.31:0/2962160993 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f40a4153660 msgr2=0x7f40a40c0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:56.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:56.769+0000 7f40a2d9d700 1 --2- 172.21.15.31:0/2962160993 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f40a4153660 0x7f40a40c0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:56.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:56.769+0000 7f40a2d9d700 1 -- 172.21.15.31:0/2962160993 >> v1:172.21.15.31:6789/0 conn(0x7f40a4153a30 legacy=0x7f40a40c8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:56.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:56.769+0000 7f40a2d9d700 1 -- 172.21.15.31:0/2962160993 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40a40c18d0 con 0x7f40a40c9090 2023-10-06T02:31:56.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:56.769+0000 7f40a159a700 1 -- 172.21.15.31:0/2962160993 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2303244815 0 0) 0x7f40a40c2200 con 0x7f40a4153a30 2023-10-06T02:31:56.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:56.769+0000 7f40a2d9d700 1 --2- 172.21.15.31:0/2962160993 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40a40c9090 0x7f40a4156d10 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f409c009e60 tx=0x7f409c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1bd617d49a530e1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:56.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:56.769+0000 7f40a159a700 1 -- 172.21.15.31:0/2962160993 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409c00e070 con 0x7f40a40c9090 2023-10-06T02:31:56.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:56.769+0000 7f40a159a700 1 -- 172.21.15.31:0/2962160993 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f409c00bc30 con 0x7f40a40c9090 2023-10-06T02:31:56.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:56.770+0000 7f40a159a700 1 -- 172.21.15.31:0/2962160993 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409c012430 con 0x7f40a40c9090 2023-10-06T02:31:56.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:56.770+0000 7f40a359e700 1 -- 172.21.15.31:0/2962160993 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40a40c9090 msgr2=0x7f40a4156d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:56.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:56.770+0000 7f40a359e700 1 --2- 172.21.15.31:0/2962160993 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40a40c9090 0x7f40a4156d10 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f409c009e60 tx=0x7f409c009780 comp rx=0 tx=0).stop 2023-10-06T02:31:56.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:56.770+0000 7f40a359e700 1 -- 172.21.15.31:0/2962160993 shutdown_connections 2023-10-06T02:31:56.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:56.770+0000 7f40a359e700 1 --2- 172.21.15.31:0/2962160993 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40a40c9090 0x7f40a4156d10 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:56.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:56.770+0000 7f40a359e700 1 --2- 172.21.15.31:0/2962160993 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f40a4153660 0x7f40a40c0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:56.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:56.770+0000 7f40a359e700 1 -- 172.21.15.31:0/2962160993 >> 172.21.15.31:0/2962160993 conn(0x7f40a406c6a0 msgr2=0x7f40a406caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:56.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:56.770+0000 7f40a359e700 1 -- 172.21.15.31:0/2962160993 shutdown_connections 2023-10-06T02:31:56.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:56.770+0000 7f40a359e700 1 -- 172.21.15.31:0/2962160993 wait complete. 2023-10-06T02:31:56.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:56.771+0000 7f40a359e700 1 Processor -- start 2023-10-06T02:31:56.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:56.772+0000 7f40a359e700 1 -- start start 2023-10-06T02:31:56.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:56.773+0000 7f40a359e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40a40c9090 0x7f40a415a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:56.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:56.773+0000 7f40a359e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f40a4153660 0x7f40a4158160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:56.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:56.773+0000 7f40a359e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40a4153a30 0x7f40a414fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:56.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:56.773+0000 7f40a359e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a41595c0 con 0x7f40a4153a30 2023-10-06T02:31:56.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:56.773+0000 7f40a359e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a41589b0 con 0x7f40a40c9090 2023-10-06T02:31:56.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:56.773+0000 7f40a359e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40a4158c90 con 0x7f40a4153660 2023-10-06T02:31:56.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:56.773+0000 7f40a2d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40a4153a30 0x7f40a414fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:56.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:56.773+0000 7f40a2d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40a4153a30 0x7f40a414fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56900/0 (socket says 172.21.15.31:56900) 2023-10-06T02:31:56.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:56.773+0000 7f40a2d9d700 1 -- 172.21.15.31:0/4194618272 learned_addr learned my addr 172.21.15.31:0/4194618272 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:56.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:56.773+0000 7f40a259c700 1 --2- 172.21.15.31:0/4194618272 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40a40c9090 0x7f40a415a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:56.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:56.773+0000 7f40a1d9b700 1 --2- 172.21.15.31:0/4194618272 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f40a4153660 0x7f40a4158160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:56.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:56.773+0000 7f40a2d9d700 1 -- 172.21.15.31:0/4194618272 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f40a4153660 msgr2=0x7f40a4158160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:56.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:56.773+0000 7f40a2d9d700 1 --2- 172.21.15.31:0/4194618272 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f40a4153660 0x7f40a4158160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:56.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:56.773+0000 7f40a2d9d700 1 -- 172.21.15.31:0/4194618272 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40a40c9090 msgr2=0x7f40a415a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:56.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:56.773+0000 7f40a2d9d700 1 --2- 172.21.15.31:0/4194618272 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40a40c9090 0x7f40a415a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:56.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:56.773+0000 7f40a2d9d700 1 -- 172.21.15.31:0/4194618272 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f409c009a20 con 0x7f40a4153a30 2023-10-06T02:31:56.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:56.773+0000 7f40a259c700 1 --2- 172.21.15.31:0/4194618272 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40a40c9090 0x7f40a415a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:31:56.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:56.773+0000 7f40a1d9b700 1 --2- 172.21.15.31:0/4194618272 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f40a4153660 0x7f40a4158160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:31:56.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:56.773+0000 7f40a2d9d700 1 --2- 172.21.15.31:0/4194618272 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40a4153a30 0x7f40a414fd20 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f40a4067f90 tx=0x7f409c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:56.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:56.774+0000 7f40927fc700 1 -- 172.21.15.31:0/4194618272 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409c01d630 con 0x7f40a4153a30 2023-10-06T02:31:56.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:56.774+0000 7f40927fc700 1 -- 172.21.15.31:0/4194618272 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f409c01d7b0 con 0x7f40a4153a30 2023-10-06T02:31:56.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:56.774+0000 7f40927fc700 1 -- 172.21.15.31:0/4194618272 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409c01db70 con 0x7f40a4153a30 2023-10-06T02:31:56.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:56.774+0000 7f40a359e700 1 -- 172.21.15.31:0/4194618272 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40a4158f70 con 0x7f40a4153a30 2023-10-06T02:31:56.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:56.774+0000 7f40a359e700 1 -- 172.21.15.31:0/4194618272 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40a41586a0 con 0x7f40a4153a30 2023-10-06T02:31:56.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:56.775+0000 7f40a359e700 1 -- 172.21.15.31:0/4194618272 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40a4159770 con 0x7f40a4153a30 2023-10-06T02:31:56.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:56.776+0000 7f40927fc700 1 -- 172.21.15.31:0/4194618272 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85556+0+0 (secure 0 0 0) 0x7f409c024070 con 0x7f40a4153a30 2023-10-06T02:31:56.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:56.777+0000 7f40927fc700 1 --2- 172.21.15.31:0/4194618272 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f4084067860 0x7f4084069d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:56.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:56.777+0000 7f40927fc700 1 -- 172.21.15.31:0/4194618272 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6259+0+0 (secure 0 0 0) 0x7f409c0856e0 con 0x7f40a4153a30 2023-10-06T02:31:56.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:56.777+0000 7f40a259c700 1 --2- 172.21.15.31:0/4194618272 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f4084067860 0x7f4084069d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:56.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:56.778+0000 7f40a259c700 1 --2- 172.21.15.31:0/4194618272 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f4084067860 0x7f4084069d10 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f4094006fd0 tx=0x7f4094008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:56.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:56.781+0000 7f40927fc700 1 -- 172.21.15.31:0/4194618272 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f40a4159770 con 0x7f40a4153a30 2023-10-06T02:31:56.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:56.981+0000 7f40a359e700 1 -- 172.21.15.31:0/4194618272 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f40a4159770 con 0x7f40a4153a30 2023-10-06T02:31:57.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:57.553+0000 7f40927fc700 1 -- 172.21.15.31:0/4194618272 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v160) v1 ==== 181+0+0 (secure 0 0 0) 0x7f40a4159770 con 0x7f40a4153a30 2023-10-06T02:31:57.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:57.558+0000 7f40a359e700 1 -- 172.21.15.31:0/4194618272 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f4084067860 msgr2=0x7f4084069d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:57.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:57.558+0000 7f40a359e700 1 --2- 172.21.15.31:0/4194618272 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f4084067860 0x7f4084069d10 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f4094006fd0 tx=0x7f4094008040 comp rx=0 tx=0).stop 2023-10-06T02:31:57.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:57.558+0000 7f40a359e700 1 -- 172.21.15.31:0/4194618272 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40a4153a30 msgr2=0x7f40a414fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:57.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:57.558+0000 7f40a359e700 1 --2- 172.21.15.31:0/4194618272 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40a4153a30 0x7f40a414fd20 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f40a4067f90 tx=0x7f409c019000 comp rx=0 tx=0).stop 2023-10-06T02:31:57.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:57.558+0000 7f40a359e700 1 -- 172.21.15.31:0/4194618272 shutdown_connections 2023-10-06T02:31:57.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:57.558+0000 7f40a359e700 1 --2- 172.21.15.31:0/4194618272 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40a4153a30 0x7f40a414fd20 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:57.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:57.558+0000 7f40a359e700 1 --2- 172.21.15.31:0/4194618272 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40a40c9090 0x7f40a415a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:57.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:57.558+0000 7f40a359e700 1 --2- 172.21.15.31:0/4194618272 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f4084067860 0x7f4084069d10 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:57.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:57.558+0000 7f40a359e700 1 --2- 172.21.15.31:0/4194618272 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f40a4153660 0x7f40a4158160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:57.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:57.558+0000 7f40a359e700 1 -- 172.21.15.31:0/4194618272 >> 172.21.15.31:0/4194618272 conn(0x7f40a406c6a0 msgr2=0x7f40a40caf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:57.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:57.558+0000 7f40a359e700 1 -- 172.21.15.31:0/4194618272 shutdown_connections 2023-10-06T02:31:57.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:57.558+0000 7f40a359e700 1 -- 172.21.15.31:0/4194618272 wait complete. 2023-10-06T02:31:57.561 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' removed 2023-10-06T02:31:57.580 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T02:31:57.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:57.776+0000 7fce83fff700 1 Processor -- start 2023-10-06T02:31:57.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:57.776+0000 7fce83fff700 1 -- start start 2023-10-06T02:31:57.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:57.778+0000 7fce83fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fce84157750 0x7fce840ae080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:57.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:57.778+0000 7fce83fff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce840c8680 con 0x7fce8415b1a0 2023-10-06T02:31:57.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:57.778+0000 7fce83fff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce8415b6a0 con 0x7fce84157380 2023-10-06T02:31:57.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:57.778+0000 7fce83fff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce8406a7f0 con 0x7fce84157750 2023-10-06T02:31:57.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:57.778+0000 7fce837fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fce84157750 0x7fce840ae080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:57.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:57.778+0000 7fce827fc700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fce8415b1a0 0x7fce840c6b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46550/0 (socket says 172.21.15.31:46550) 2023-10-06T02:31:57.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:57.778+0000 7fce827fc700 1 -- 172.21.15.31:0/3348958160 learned_addr learned my addr 172.21.15.31:0/3348958160 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:31:57.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:57.778+0000 7fce81ffb700 1 -- 172.21.15.31:0/3348958160 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1859236384 0 0) 0x7fce840c8680 con 0x7fce8415b1a0 2023-10-06T02:31:57.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:57.778+0000 7fce837fe700 1 -- 172.21.15.31:0/3348958160 >> v1:172.21.15.62:6789/0 conn(0x7fce84157380 legacy=0x7fce841585f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:57.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:57.778+0000 7fce837fe700 1 -- 172.21.15.31:0/3348958160 >> v1:172.21.15.31:6789/0 conn(0x7fce8415b1a0 legacy=0x7fce840c6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:57.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:57.779+0000 7fce837fe700 1 -- 172.21.15.31:0/3348958160 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce8415b980 con 0x7fce84157750 2023-10-06T02:31:57.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:57.779+0000 7fce837fe700 1 --2- 172.21.15.31:0/3348958160 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce84157750 0x7fce840ae080 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7fce7c009e60 tx=0x7fce7c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4ec95f39d225dc8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:57.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:57.779+0000 7fce81ffb700 1 -- 172.21.15.31:0/3348958160 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce7c00e070 con 0x7fce84157750 2023-10-06T02:31:57.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:57.779+0000 7fce81ffb700 1 -- 172.21.15.31:0/3348958160 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce7c00bc30 con 0x7fce84157750 2023-10-06T02:31:57.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:57.779+0000 7fce81ffb700 1 -- 172.21.15.31:0/3348958160 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce7c012430 con 0x7fce84157750 2023-10-06T02:31:57.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:57.780+0000 7fce83fff700 1 -- 172.21.15.31:0/3348958160 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce84157750 msgr2=0x7fce840ae080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:57.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:57.780+0000 7fce83fff700 1 --2- 172.21.15.31:0/3348958160 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce84157750 0x7fce840ae080 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7fce7c009e60 tx=0x7fce7c009780 comp rx=0 tx=0).stop 2023-10-06T02:31:57.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:57.780+0000 7fce83fff700 1 -- 172.21.15.31:0/3348958160 shutdown_connections 2023-10-06T02:31:57.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:57.780+0000 7fce83fff700 1 --2- 172.21.15.31:0/3348958160 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce84157750 0x7fce840ae080 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:57.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:57.780+0000 7fce83fff700 1 -- 172.21.15.31:0/3348958160 >> 172.21.15.31:0/3348958160 conn(0x7fce8406c6a0 msgr2=0x7fce8406caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:57.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:57.780+0000 7fce83fff700 1 -- 172.21.15.31:0/3348958160 shutdown_connections 2023-10-06T02:31:57.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:57.780+0000 7fce83fff700 1 -- 172.21.15.31:0/3348958160 wait complete. 2023-10-06T02:31:57.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:57.781+0000 7fce83fff700 1 Processor -- start 2023-10-06T02:31:57.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:57.781+0000 7fce83fff700 1 -- start start 2023-10-06T02:31:57.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:57.782+0000 7fce83fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce84157380 0x7fce841497f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:57.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:57.782+0000 7fce83fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce84157750 0x7fce840aa570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:57.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:57.782+0000 7fce83fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce8415b1a0 0x7fce84152b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:57.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:57.782+0000 7fce83fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce840c2fa0 con 0x7fce84157380 2023-10-06T02:31:57.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:57.782+0000 7fce83fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce8414ec10 con 0x7fce84157750 2023-10-06T02:31:57.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:57.782+0000 7fce83fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce8415a5a0 con 0x7fce8415b1a0 2023-10-06T02:31:57.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:57.783+0000 7fce82ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce84157380 0x7fce841497f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:57.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:57.783+0000 7fce82ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce84157380 0x7fce841497f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56920/0 (socket says 172.21.15.31:56920) 2023-10-06T02:31:57.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:57.783+0000 7fce82ffd700 1 -- 172.21.15.31:0/100087599 learned_addr learned my addr 172.21.15.31:0/100087599 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:57.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:57.783+0000 7fce827fc700 1 --2- 172.21.15.31:0/100087599 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce84157750 0x7fce840aa570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:57.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:57.783+0000 7fce837fe700 1 --2- 172.21.15.31:0/100087599 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce8415b1a0 0x7fce84152b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:57.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:57.783+0000 7fce82ffd700 1 -- 172.21.15.31:0/100087599 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce8415b1a0 msgr2=0x7fce84152b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:57.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:57.783+0000 7fce82ffd700 1 --2- 172.21.15.31:0/100087599 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce8415b1a0 0x7fce84152b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:57.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:57.783+0000 7fce82ffd700 1 -- 172.21.15.31:0/100087599 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce84157750 msgr2=0x7fce840aa570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:57.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:57.783+0000 7fce82ffd700 1 --2- 172.21.15.31:0/100087599 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce84157750 0x7fce840aa570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:57.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:57.783+0000 7fce82ffd700 1 -- 172.21.15.31:0/100087599 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce7c009a20 con 0x7fce84157380 2023-10-06T02:31:57.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:57.783+0000 7fce82ffd700 1 --2- 172.21.15.31:0/100087599 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce84157380 0x7fce841497f0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7fce7400c590 tx=0x7fce7400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:57.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:57.784+0000 7fce6affd700 1 -- 172.21.15.31:0/100087599 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce74011590 con 0x7fce84157380 2023-10-06T02:31:57.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:57.784+0000 7fce6affd700 1 -- 172.21.15.31:0/100087599 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce740116f0 con 0x7fce84157380 2023-10-06T02:31:57.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:57.784+0000 7fce83fff700 1 -- 172.21.15.31:0/100087599 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce8415a880 con 0x7fce84157380 2023-10-06T02:31:57.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:57.784+0000 7fce6affd700 1 -- 172.21.15.31:0/100087599 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce74011ab0 con 0x7fce84157380 2023-10-06T02:31:57.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:57.784+0000 7fce83fff700 1 -- 172.21.15.31:0/100087599 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce8415ae40 con 0x7fce84157380 2023-10-06T02:31:57.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:57.785+0000 7fce83fff700 1 -- 172.21.15.31:0/100087599 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce8404c3c0 con 0x7fce84157380 2023-10-06T02:31:57.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:57.786+0000 7fce6affd700 1 -- 172.21.15.31:0/100087599 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85556+0+0 (secure 0 0 0) 0x7fce7401b3b0 con 0x7fce84157380 2023-10-06T02:31:57.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:57.787+0000 7fce6affd700 1 --2- 172.21.15.31:0/100087599 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fce70067830 0x7fce70069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:57.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:57.787+0000 7fce6affd700 1 -- 172.21.15.31:0/100087599 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 5821+0+0 (secure 0 0 0) 0x7fce74082440 con 0x7fce84157380 2023-10-06T02:31:57.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:57.787+0000 7fce827fc700 1 --2- 172.21.15.31:0/100087599 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fce70067830 0x7fce70069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:57.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:57.788+0000 7fce827fc700 1 --2- 172.21.15.31:0/100087599 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fce70067830 0x7fce70069ce0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fce84153e80 tx=0x7fce78009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:57.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:57.791+0000 7fce6affd700 1 -- 172.21.15.31:0/100087599 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fce74051080 con 0x7fce84157380 2023-10-06T02:31:57.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:57.994+0000 7fce83fff700 1 -- 172.21.15.31:0/100087599 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fce840c6940 con 0x7fce84157380 2023-10-06T02:31:58.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:58.562+0000 7fce6affd700 1 -- 172.21.15.31:0/100087599 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v161) v1 ==== 169+0+0 (secure 0 0 0) 0x7fce740170d0 con 0x7fce84157380 2023-10-06T02:31:58.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:58.566+0000 7fce83fff700 1 -- 172.21.15.31:0/100087599 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fce70067830 msgr2=0x7fce70069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:58.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:58.566+0000 7fce83fff700 1 --2- 172.21.15.31:0/100087599 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fce70067830 0x7fce70069ce0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fce84153e80 tx=0x7fce78009040 comp rx=0 tx=0).stop 2023-10-06T02:31:58.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:58.566+0000 7fce83fff700 1 -- 172.21.15.31:0/100087599 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce84157380 msgr2=0x7fce841497f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:58.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:58.566+0000 7fce83fff700 1 --2- 172.21.15.31:0/100087599 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce84157380 0x7fce841497f0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7fce7400c590 tx=0x7fce7400c950 comp rx=0 tx=0).stop 2023-10-06T02:31:58.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:58.566+0000 7fce83fff700 1 -- 172.21.15.31:0/100087599 shutdown_connections 2023-10-06T02:31:58.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:58.566+0000 7fce83fff700 1 --2- 172.21.15.31:0/100087599 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fce70067830 0x7fce70069ce0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:58.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:58.566+0000 7fce83fff700 1 --2- 172.21.15.31:0/100087599 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce8415b1a0 0x7fce84152b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:58.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:58.566+0000 7fce83fff700 1 --2- 172.21.15.31:0/100087599 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce84157380 0x7fce841497f0 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:58.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:58.566+0000 7fce83fff700 1 --2- 172.21.15.31:0/100087599 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce84157750 0x7fce840aa570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:58.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:58.566+0000 7fce83fff700 1 -- 172.21.15.31:0/100087599 >> 172.21.15.31:0/100087599 conn(0x7fce8406c6a0 msgr2=0x7fce84150720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:58.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:58.566+0000 7fce83fff700 1 -- 172.21.15.31:0/100087599 shutdown_connections 2023-10-06T02:31:58.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:58.566+0000 7fce83fff700 1 -- 172.21.15.31:0/100087599 wait complete. 2023-10-06T02:31:58.569 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' removed 2023-10-06T02:31:58.588 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T02:31:58.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:58.776+0000 7fd22cd9d700 1 Processor -- start 2023-10-06T02:31:58.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:58.777+0000 7fd22cd9d700 1 -- start start 2023-10-06T02:31:58.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:58.778+0000 7fd22cd9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd228140a70 0x7fd228069b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:58.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:58.778+0000 7fd22cd9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd2280c7c60 0x7fd22806a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:58.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:58.778+0000 7fd22cd9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd22815b640 con 0x7fd22806de50 2023-10-06T02:31:58.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:58.778+0000 7fd22cd9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2280c6800 con 0x7fd228140a70 2023-10-06T02:31:58.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:58.778+0000 7fd22cd9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd22815b920 con 0x7fd2280c7c60 2023-10-06T02:31:58.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:58.778+0000 7fd2277fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd228140a70 0x7fd228069b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:58.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:58.778+0000 7fd2277fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd228140a70 0x7fd228069b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37322/0 (socket says 172.21.15.31:37322) 2023-10-06T02:31:58.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:58.778+0000 7fd2277fe700 1 -- 172.21.15.31:0/2627033694 learned_addr learned my addr 172.21.15.31:0/2627033694 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:58.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:58.778+0000 7fd227fff700 1 --2- 172.21.15.31:0/2627033694 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd2280c7c60 0x7fd22806a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:58.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:58.779+0000 7fd2277fe700 1 -- 172.21.15.31:0/2627033694 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd2280c7c60 msgr2=0x7fd22806a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:58.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:58.779+0000 7fd2277fe700 1 --2- 172.21.15.31:0/2627033694 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd2280c7c60 0x7fd22806a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:58.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:58.779+0000 7fd2277fe700 1 -- 172.21.15.31:0/2627033694 >> v1:172.21.15.62:6789/0 conn(0x7fd22806de50 legacy=0x7fd228157a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:31:58.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:58.779+0000 7fd2277fe700 1 -- 172.21.15.31:0/2627033694 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd22815bc00 con 0x7fd228140a70 2023-10-06T02:31:58.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:58.779+0000 7fd227fff700 1 --2- 172.21.15.31:0/2627033694 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd2280c7c60 0x7fd22806a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:31:58.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:58.779+0000 7fd2277fe700 1 --2- 172.21.15.31:0/2627033694 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd228140a70 0x7fd228069b00 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7fd2180099b0 tx=0x7fd218009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb399e33ba9f7655 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:58.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:58.779+0000 7fd2267fc700 1 -- 172.21.15.31:0/2627033694 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd21800b820 con 0x7fd228140a70 2023-10-06T02:31:58.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:58.779+0000 7fd2267fc700 1 -- 172.21.15.31:0/2627033694 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd21800b980 con 0x7fd228140a70 2023-10-06T02:31:58.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:58.779+0000 7fd2267fc700 1 -- 172.21.15.31:0/2627033694 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd21800bd20 con 0x7fd228140a70 2023-10-06T02:31:58.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:58.780+0000 7fd22cd9d700 1 -- 172.21.15.31:0/2627033694 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd228140a70 msgr2=0x7fd228069b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:58.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:58.780+0000 7fd22cd9d700 1 --2- 172.21.15.31:0/2627033694 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd228140a70 0x7fd228069b00 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7fd2180099b0 tx=0x7fd218009250 comp rx=0 tx=0).stop 2023-10-06T02:31:58.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:58.780+0000 7fd22cd9d700 1 -- 172.21.15.31:0/2627033694 shutdown_connections 2023-10-06T02:31:58.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:58.780+0000 7fd22cd9d700 1 --2- 172.21.15.31:0/2627033694 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd2280c7c60 0x7fd22806a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:58.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:58.780+0000 7fd22cd9d700 1 --2- 172.21.15.31:0/2627033694 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd228140a70 0x7fd228069b00 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:58.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:58.780+0000 7fd22cd9d700 1 -- 172.21.15.31:0/2627033694 >> 172.21.15.31:0/2627033694 conn(0x7fd22806d110 msgr2=0x7fd2280cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:58.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:58.780+0000 7fd22cd9d700 1 -- 172.21.15.31:0/2627033694 shutdown_connections 2023-10-06T02:31:58.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:58.780+0000 7fd22cd9d700 1 -- 172.21.15.31:0/2627033694 wait complete. 2023-10-06T02:31:58.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:58.781+0000 7fd22cd9d700 1 Processor -- start 2023-10-06T02:31:58.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:58.781+0000 7fd22cd9d700 1 -- start start 2023-10-06T02:31:58.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:58.782+0000 7fd22cd9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd22806de50 0x7fd22813ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:58.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:58.782+0000 7fd22cd9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd2280c7c60 0x7fd2280c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:58.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:58.782+0000 7fd22cd9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd228140a70 0x7fd2280ca240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:58.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:58.782+0000 7fd22cd9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2280ca870 con 0x7fd228140a70 2023-10-06T02:31:58.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:58.782+0000 7fd22cd9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2280cab50 con 0x7fd22806de50 2023-10-06T02:31:58.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:58.782+0000 7fd22cd9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2280b00f0 con 0x7fd2280c7c60 2023-10-06T02:31:58.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:58.783+0000 7fd227fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd228140a70 0x7fd2280ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:58.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:58.783+0000 7fd227fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd228140a70 0x7fd2280ca240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37334/0 (socket says 172.21.15.31:37334) 2023-10-06T02:31:58.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:58.783+0000 7fd227fff700 1 -- 172.21.15.31:0/3757145795 learned_addr learned my addr 172.21.15.31:0/3757145795 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:58.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:58.783+0000 7fd2277fe700 1 --2- 172.21.15.31:0/3757145795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd22806de50 0x7fd22813ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:58.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:58.783+0000 7fd226ffd700 1 --2- 172.21.15.31:0/3757145795 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd2280c7c60 0x7fd2280c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:58.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:58.783+0000 7fd227fff700 1 -- 172.21.15.31:0/3757145795 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd2280c7c60 msgr2=0x7fd2280c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:58.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:58.783+0000 7fd227fff700 1 --2- 172.21.15.31:0/3757145795 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd2280c7c60 0x7fd2280c9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:58.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:58.783+0000 7fd227fff700 1 -- 172.21.15.31:0/3757145795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd22806de50 msgr2=0x7fd22813ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:58.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:58.783+0000 7fd227fff700 1 --2- 172.21.15.31:0/3757145795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd22806de50 0x7fd22813ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:58.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:58.783+0000 7fd227fff700 1 -- 172.21.15.31:0/3757145795 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2180094f0 con 0x7fd228140a70 2023-10-06T02:31:58.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:58.783+0000 7fd226ffd700 1 --2- 172.21.15.31:0/3757145795 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd2280c7c60 0x7fd2280c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:31:58.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:58.783+0000 7fd227fff700 1 --2- 172.21.15.31:0/3757145795 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd228140a70 0x7fd2280ca240 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7fd22000a380 tx=0x7fd220007900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:58.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:58.784+0000 7fd20f7fe700 1 -- 172.21.15.31:0/3757145795 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd220015440 con 0x7fd228140a70 2023-10-06T02:31:58.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:58.784+0000 7fd20f7fe700 1 -- 172.21.15.31:0/3757145795 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2200155a0 con 0x7fd228140a70 2023-10-06T02:31:58.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:58.784+0000 7fd20f7fe700 1 -- 172.21.15.31:0/3757145795 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd220015960 con 0x7fd228140a70 2023-10-06T02:31:58.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:58.784+0000 7fd22cd9d700 1 -- 172.21.15.31:0/3757145795 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2280b03d0 con 0x7fd228140a70 2023-10-06T02:31:58.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:58.784+0000 7fd22cd9d700 1 -- 172.21.15.31:0/3757145795 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd228157c00 con 0x7fd228140a70 2023-10-06T02:31:58.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:58.785+0000 7fd22cd9d700 1 -- 172.21.15.31:0/3757145795 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2280648e0 con 0x7fd228140a70 2023-10-06T02:31:58.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:58.786+0000 7fd20f7fe700 1 -- 172.21.15.31:0/3757145795 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85556+0+0 (secure 0 0 0) 0x7fd22000d440 con 0x7fd228140a70 2023-10-06T02:31:58.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:58.786+0000 7fd20f7fe700 1 --2- 172.21.15.31:0/3757145795 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd214067830 0x7fd214069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:58.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:58.786+0000 7fd20f7fe700 1 -- 172.21.15.31:0/3757145795 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5320+0+0 (secure 0 0 0) 0x7fd220083a20 con 0x7fd228140a70 2023-10-06T02:31:58.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:58.787+0000 7fd2277fe700 1 --2- 172.21.15.31:0/3757145795 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd214067830 0x7fd214069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:58.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:58.788+0000 7fd2277fe700 1 --2- 172.21.15.31:0/3757145795 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd214067830 0x7fd214069ce0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fd2180099b0 tx=0x7fd218016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:58.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:58.790+0000 7fd20f7fe700 1 -- 172.21.15.31:0/3757145795 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd2280648e0 con 0x7fd228140a70 2023-10-06T02:31:59.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.001+0000 7fd22cd9d700 1 -- 172.21.15.31:0/3757145795 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fd2280648e0 con 0x7fd228140a70 2023-10-06T02:31:59.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.006+0000 7fd20f7fe700 1 -- 172.21.15.31:0/3757145795 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v29) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd2280648e0 con 0x7fd228140a70 2023-10-06T02:31:59.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.008+0000 7fd22cd9d700 1 -- 172.21.15.31:0/3757145795 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd214067830 msgr2=0x7fd214069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:59.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.008+0000 7fd22cd9d700 1 --2- 172.21.15.31:0/3757145795 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd214067830 0x7fd214069ce0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fd2180099b0 tx=0x7fd218016000 comp rx=0 tx=0).stop 2023-10-06T02:31:59.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.008+0000 7fd22cd9d700 1 -- 172.21.15.31:0/3757145795 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd228140a70 msgr2=0x7fd2280ca240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:59.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.008+0000 7fd22cd9d700 1 --2- 172.21.15.31:0/3757145795 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd228140a70 0x7fd2280ca240 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7fd22000a380 tx=0x7fd220007900 comp rx=0 tx=0).stop 2023-10-06T02:31:59.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.008+0000 7fd22cd9d700 1 -- 172.21.15.31:0/3757145795 shutdown_connections 2023-10-06T02:31:59.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.008+0000 7fd22cd9d700 1 --2- 172.21.15.31:0/3757145795 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd228140a70 0x7fd2280ca240 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:59.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.008+0000 7fd22cd9d700 1 --2- 172.21.15.31:0/3757145795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd22806de50 0x7fd22813ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:59.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.008+0000 7fd22cd9d700 1 --2- 172.21.15.31:0/3757145795 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd214067830 0x7fd214069ce0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:59.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.008+0000 7fd22cd9d700 1 --2- 172.21.15.31:0/3757145795 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd2280c7c60 0x7fd2280c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:59.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.008+0000 7fd22cd9d700 1 -- 172.21.15.31:0/3757145795 >> 172.21.15.31:0/3757145795 conn(0x7fd22806d110 msgr2=0x7fd22813fae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:59.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.008+0000 7fd22cd9d700 1 -- 172.21.15.31:0/3757145795 shutdown_connections 2023-10-06T02:31:59.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.008+0000 7fd22cd9d700 1 -- 172.21.15.31:0/3757145795 wait complete. 2023-10-06T02:31:59.027 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2' 2023-10-06T02:31:59.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.219+0000 7f3636d9d700 1 Processor -- start 2023-10-06T02:31:59.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.220+0000 7f3636d9d700 1 -- start start 2023-10-06T02:31:59.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.221+0000 7f3636d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3630153c30 0x7f363015a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:59.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.221+0000 7f3636d9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f363015afd0 con 0x7f363015de50 2023-10-06T02:31:59.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.221+0000 7f3636d9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f363015b2b0 con 0x7f36300cf110 2023-10-06T02:31:59.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.221+0000 7f3636d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f363015b590 con 0x7f3630153c30 2023-10-06T02:31:59.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.221+0000 7f363559a700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f363015de50 0x7f36300cea00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:43870/0 (socket says 172.21.15.31:43870) 2023-10-06T02:31:59.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.221+0000 7f363559a700 1 -- 172.21.15.31:0/2295817979 learned_addr learned my addr 172.21.15.31:0/2295817979 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:31:59.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.222+0000 7f3635d9b700 1 --2- 172.21.15.31:0/2295817979 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3630153c30 0x7f363015a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:59.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.222+0000 7f3634d99700 1 -- 172.21.15.31:0/2295817979 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2578678042 0 0) 0x7f363015afd0 con 0x7f363015de50 2023-10-06T02:31:59.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.222+0000 7f3634d99700 1 -- 172.21.15.31:0/2295817979 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f361c003570 con 0x7f363015de50 2023-10-06T02:31:59.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.222+0000 7f3635d9b700 1 -- 172.21.15.31:0/2295817979 >> v1:172.21.15.62:6789/0 conn(0x7f36300cf110 legacy=0x7f363015c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:59.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.222+0000 7f3634d99700 1 -- 172.21.15.31:0/2295817979 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1970167829 0 0) 0x7f363015b2b0 con 0x7f36300cf110 2023-10-06T02:31:59.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.222+0000 7f3635d9b700 1 -- 172.21.15.31:0/2295817979 >> v1:172.21.15.31:6789/0 conn(0x7f363015de50 legacy=0x7f36300cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:59.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.222+0000 7f3635d9b700 1 -- 172.21.15.31:0/2295817979 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f363015b870 con 0x7f3630153c30 2023-10-06T02:31:59.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.222+0000 7f3635d9b700 1 --2- 172.21.15.31:0/2295817979 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3630153c30 0x7f363015a5e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f362c009910 tx=0x7f362c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a897068a041ad3b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:59.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.223+0000 7f3634d99700 1 -- 172.21.15.31:0/2295817979 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f362c00b680 con 0x7f3630153c30 2023-10-06T02:31:59.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.223+0000 7f3634d99700 1 -- 172.21.15.31:0/2295817979 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f362c00b7e0 con 0x7f3630153c30 2023-10-06T02:31:59.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.223+0000 7f3634d99700 1 -- 172.21.15.31:0/2295817979 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f362c00bba0 con 0x7f3630153c30 2023-10-06T02:31:59.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.223+0000 7f3636d9d700 1 -- 172.21.15.31:0/2295817979 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3630153c30 msgr2=0x7f363015a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:59.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.223+0000 7f3636d9d700 1 --2- 172.21.15.31:0/2295817979 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3630153c30 0x7f363015a5e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f362c009910 tx=0x7f362c0091a0 comp rx=0 tx=0).stop 2023-10-06T02:31:59.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.223+0000 7f3636d9d700 1 -- 172.21.15.31:0/2295817979 shutdown_connections 2023-10-06T02:31:59.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.223+0000 7f3636d9d700 1 --2- 172.21.15.31:0/2295817979 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3630153c30 0x7f363015a5e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:59.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.223+0000 7f3636d9d700 1 -- 172.21.15.31:0/2295817979 >> 172.21.15.31:0/2295817979 conn(0x7f36300703a0 msgr2=0x7f36300707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:59.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.223+0000 7f3636d9d700 1 -- 172.21.15.31:0/2295817979 shutdown_connections 2023-10-06T02:31:59.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.223+0000 7f3636d9d700 1 -- 172.21.15.31:0/2295817979 wait complete. 2023-10-06T02:31:59.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.225+0000 7f3636d9d700 1 Processor -- start 2023-10-06T02:31:59.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.225+0000 7f3636d9d700 1 -- start start 2023-10-06T02:31:59.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.226+0000 7f3636d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f36300cf110 0x7f36300b5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:59.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.226+0000 7f3636d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3630153c30 0x7f3630152f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:59.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.226+0000 7f3636d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f363015de50 0x7f36301698f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:59.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.226+0000 7f3636d9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36300c9530 con 0x7f36300cf110 2023-10-06T02:31:59.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.226+0000 7f3636d9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36300b60f0 con 0x7f363015de50 2023-10-06T02:31:59.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.226+0000 7f3636d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36300c8920 con 0x7f3630153c30 2023-10-06T02:31:59.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.226+0000 7f3635d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f36300cf110 0x7f36300b5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:59.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.226+0000 7f3635d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f36300cf110 0x7f36300b5bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37340/0 (socket says 172.21.15.31:37340) 2023-10-06T02:31:59.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.226+0000 7f3635d9b700 1 -- 172.21.15.31:0/1748649143 learned_addr learned my addr 172.21.15.31:0/1748649143 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:59.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.226+0000 7f363559a700 1 --2- 172.21.15.31:0/1748649143 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3630153c30 0x7f3630152f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:59.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.226+0000 7f363659c700 1 --2- 172.21.15.31:0/1748649143 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f363015de50 0x7f36301698f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:59.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.227+0000 7f3635d9b700 1 -- 172.21.15.31:0/1748649143 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3630153c30 msgr2=0x7f3630152f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:59.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.227+0000 7f3635d9b700 1 --2- 172.21.15.31:0/1748649143 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3630153c30 0x7f3630152f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:59.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.227+0000 7f3635d9b700 1 -- 172.21.15.31:0/1748649143 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f363015de50 msgr2=0x7f36301698f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:59.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.227+0000 7f3635d9b700 1 --2- 172.21.15.31:0/1748649143 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f363015de50 0x7f36301698f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:59.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.227+0000 7f3635d9b700 1 -- 172.21.15.31:0/1748649143 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f362c009420 con 0x7f36300cf110 2023-10-06T02:31:59.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.227+0000 7f363659c700 1 --2- 172.21.15.31:0/1748649143 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f363015de50 0x7f36301698f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:31:59.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.227+0000 7f3635d9b700 1 --2- 172.21.15.31:0/1748649143 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f36300cf110 0x7f36300b5bb0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f3630159ca0 tx=0x7f362c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:59.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.227+0000 7f3625ffb700 1 -- 172.21.15.31:0/1748649143 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f362c00bb60 con 0x7f36300cf110 2023-10-06T02:31:59.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.227+0000 7f3625ffb700 1 -- 172.21.15.31:0/1748649143 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f362c027450 con 0x7f36300cf110 2023-10-06T02:31:59.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.227+0000 7f3625ffb700 1 -- 172.21.15.31:0/1748649143 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f362c0277f0 con 0x7f36300cf110 2023-10-06T02:31:59.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.227+0000 7f3636d9d700 1 -- 172.21.15.31:0/1748649143 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36300c8c00 con 0x7f36300cf110 2023-10-06T02:31:59.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.227+0000 7f3636d9d700 1 -- 172.21.15.31:0/1748649143 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36300c91c0 con 0x7f36300cf110 2023-10-06T02:31:59.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.228+0000 7f3636d9d700 1 -- 172.21.15.31:0/1748649143 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f363004c3c0 con 0x7f36300cf110 2023-10-06T02:31:59.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.229+0000 7f3625ffb700 1 -- 172.21.15.31:0/1748649143 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85556+0+0 (secure 0 0 0) 0x7f362c01a430 con 0x7f36300cf110 2023-10-06T02:31:59.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.230+0000 7f3625ffb700 1 --2- 172.21.15.31:0/1748649143 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f3614067830 0x7f3614069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:59.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.230+0000 7f3625ffb700 1 -- 172.21.15.31:0/1748649143 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5320+0+0 (secure 0 0 0) 0x7f362c087500 con 0x7f36300cf110 2023-10-06T02:31:59.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.230+0000 7f363559a700 1 --2- 172.21.15.31:0/1748649143 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f3614067830 0x7f3614069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:59.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.231+0000 7f363559a700 1 --2- 172.21.15.31:0/1748649143 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f3614067830 0x7f3614069ce0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f363015bad0 tx=0x7f3620009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:59.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.235+0000 7f3625ffb700 1 -- 172.21.15.31:0/1748649143 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f362c0539d0 con 0x7f36300cf110 2023-10-06T02:31:59.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.516+0000 7f3636d9d700 1 -- 172.21.15.31:0/1748649143 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2"]} v 0) v1 -- 0x7f363015bb50 con 0x7f36300cf110 2023-10-06T02:31:59.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.561+0000 7f3625ffb700 1 -- 172.21.15.31:0/1748649143 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2"]}]=0 v494) v1 ==== 145+0+0 (secure 0 0 0) 0x7f362c056ff0 con 0x7f36300cf110 2023-10-06T02:31:59.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.564+0000 7f3636d9d700 1 -- 172.21.15.31:0/1748649143 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f3614067830 msgr2=0x7f3614069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:59.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.564+0000 7f3636d9d700 1 --2- 172.21.15.31:0/1748649143 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f3614067830 0x7f3614069ce0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f363015bad0 tx=0x7f3620009040 comp rx=0 tx=0).stop 2023-10-06T02:31:59.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.564+0000 7f3636d9d700 1 -- 172.21.15.31:0/1748649143 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f36300cf110 msgr2=0x7f36300b5bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:59.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.564+0000 7f3636d9d700 1 --2- 172.21.15.31:0/1748649143 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f36300cf110 0x7f36300b5bb0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f3630159ca0 tx=0x7f362c017000 comp rx=0 tx=0).stop 2023-10-06T02:31:59.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.564+0000 7f3636d9d700 1 -- 172.21.15.31:0/1748649143 shutdown_connections 2023-10-06T02:31:59.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.564+0000 7f3636d9d700 1 --2- 172.21.15.31:0/1748649143 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f363015de50 0x7f36301698f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:59.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.564+0000 7f3636d9d700 1 --2- 172.21.15.31:0/1748649143 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f36300cf110 0x7f36300b5bb0 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:59.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.564+0000 7f3636d9d700 1 --2- 172.21.15.31:0/1748649143 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f3614067830 0x7f3614069ce0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:59.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.564+0000 7f3636d9d700 1 --2- 172.21.15.31:0/1748649143 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3630153c30 0x7f3630152f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:59.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.564+0000 7f3636d9d700 1 -- 172.21.15.31:0/1748649143 >> 172.21.15.31:0/1748649143 conn(0x7f36300703a0 msgr2=0x7f36300d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:59.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.564+0000 7f3636d9d700 1 -- 172.21.15.31:0/1748649143 shutdown_connections 2023-10-06T02:31:59.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.564+0000 7f3636d9d700 1 -- 172.21.15.31:0/1748649143 wait complete. 2023-10-06T02:31:59.583 INFO:tasks.cephfs_test_runner:poor man's upgrade test -- theme continues... ... ok 2023-10-06T02:31:59.583 INFO:tasks.cephfs_test_runner:Starting test: test_unparseable_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) Validate unparseable text .meta file on legacy subvol root doesn't break the system 2023-10-06T02:31:59.583 INFO:tasks.cephfs_test_runner:test_unparseable_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) 2023-10-06T02:31:59.584 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade' 2023-10-06T02:31:59.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.769+0000 7f604b59e700 1 Processor -- start 2023-10-06T02:31:59.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.771+0000 7f604b59e700 1 -- start start 2023-10-06T02:31:59.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.772+0000 7f604b59e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f604c153c30 0x7f604c15a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:59.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.772+0000 7f604b59e700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f604c15afd0 con 0x7f604c0cf110 2023-10-06T02:31:59.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.772+0000 7f604b59e700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f604c15b2b0 con 0x7f604c15de50 2023-10-06T02:31:59.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.772+0000 7f604b59e700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f604c15b590 con 0x7f604c153c30 2023-10-06T02:31:59.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.772+0000 7f604a59c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f604c153c30 0x7f604c15a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:59.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.772+0000 7f604ad9d700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f604c0cf110 0x7f604c15c070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:43874/0 (socket says 172.21.15.31:43874) 2023-10-06T02:31:59.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.772+0000 7f604ad9d700 1 -- 172.21.15.31:0/1321102212 learned_addr learned my addr 172.21.15.31:0/1321102212 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:31:59.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.772+0000 7f604a59c700 1 -- 172.21.15.31:0/1321102212 >> v1:172.21.15.62:6789/0 conn(0x7f604c15de50 legacy=0x7f604c0cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:59.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.772+0000 7f604a59c700 1 -- 172.21.15.31:0/1321102212 >> v1:172.21.15.31:6789/0 conn(0x7f604c0cf110 legacy=0x7f604c15c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:59.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.772+0000 7f604a59c700 1 -- 172.21.15.31:0/1321102212 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f604c15b870 con 0x7f604c153c30 2023-10-06T02:31:59.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.772+0000 7f604a59c700 1 --2- 172.21.15.31:0/1321102212 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f604c153c30 0x7f604c15a5e0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f603c0098e0 tx=0x7f603c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88af53703a62c61f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:59.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.772+0000 7f604959a700 1 -- 172.21.15.31:0/1321102212 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f603c009e80 con 0x7f604c153c30 2023-10-06T02:31:59.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.772+0000 7f604959a700 1 -- 172.21.15.31:0/1321102212 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f603c00b860 con 0x7f604c153c30 2023-10-06T02:31:59.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.772+0000 7f604959a700 1 -- 172.21.15.31:0/1321102212 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f603c00bbe0 con 0x7f604c153c30 2023-10-06T02:31:59.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.773+0000 7f604b59e700 1 -- 172.21.15.31:0/1321102212 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f604c153c30 msgr2=0x7f604c15a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:59.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.773+0000 7f604b59e700 1 --2- 172.21.15.31:0/1321102212 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f604c153c30 0x7f604c15a5e0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f603c0098e0 tx=0x7f603c0091a0 comp rx=0 tx=0).stop 2023-10-06T02:31:59.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.773+0000 7f604b59e700 1 -- 172.21.15.31:0/1321102212 shutdown_connections 2023-10-06T02:31:59.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.773+0000 7f604b59e700 1 --2- 172.21.15.31:0/1321102212 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f604c153c30 0x7f604c15a5e0 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:59.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.773+0000 7f604b59e700 1 -- 172.21.15.31:0/1321102212 >> 172.21.15.31:0/1321102212 conn(0x7f604c0703a0 msgr2=0x7f604c0707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:31:59.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.773+0000 7f604b59e700 1 -- 172.21.15.31:0/1321102212 shutdown_connections 2023-10-06T02:31:59.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.773+0000 7f604b59e700 1 -- 172.21.15.31:0/1321102212 wait complete. 2023-10-06T02:31:59.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.774+0000 7f604b59e700 1 Processor -- start 2023-10-06T02:31:59.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.775+0000 7f604b59e700 1 -- start start 2023-10-06T02:31:59.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.775+0000 7f604b59e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f604c0cf110 0x7f604c0b5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:59.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.775+0000 7f604b59e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f604c153c30 0x7f604c0c9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:59.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.775+0000 7f604b59e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f604c15de50 0x7f604c0cc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:59.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.775+0000 7f604b59e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f604c169f60 con 0x7f604c153c30 2023-10-06T02:31:59.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.775+0000 7f604b59e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f604c0b4110 con 0x7f604c15de50 2023-10-06T02:31:59.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.775+0000 7f604b59e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f604c0c91c0 con 0x7f604c0cf110 2023-10-06T02:31:59.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.776+0000 7f6049d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f604c153c30 0x7f604c0c9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:59.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.776+0000 7f6049d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f604c153c30 0x7f604c0c9ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37354/0 (socket says 172.21.15.31:37354) 2023-10-06T02:31:59.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.776+0000 7f6049d9b700 1 -- 172.21.15.31:0/2709252147 learned_addr learned my addr 172.21.15.31:0/2709252147 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:31:59.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.776+0000 7f6049d9b700 1 -- 172.21.15.31:0/2709252147 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f604c0cf110 msgr2=0x7f604c0b5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:59.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.776+0000 7f6049d9b700 1 --2- 172.21.15.31:0/2709252147 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f604c0cf110 0x7f604c0b5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:59.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.776+0000 7f6049d9b700 1 -- 172.21.15.31:0/2709252147 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f604c15de50 msgr2=0x7f604c0cc4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:31:59.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.776+0000 7f6049d9b700 1 --2- 172.21.15.31:0/2709252147 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f604c15de50 0x7f604c0cc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:31:59.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.776+0000 7f6049d9b700 1 -- 172.21.15.31:0/2709252147 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f603c009420 con 0x7f604c153c30 2023-10-06T02:31:59.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.776+0000 7f6049d9b700 1 --2- 172.21.15.31:0/2709252147 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f604c153c30 0x7f604c0c9ca0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f604000c4e0 tx=0x7f604000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:59.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.776+0000 7f603a7fc700 1 -- 172.21.15.31:0/2709252147 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6040011590 con 0x7f604c153c30 2023-10-06T02:31:59.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.777+0000 7f603a7fc700 1 -- 172.21.15.31:0/2709252147 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6040011710 con 0x7f604c153c30 2023-10-06T02:31:59.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.777+0000 7f604b59e700 1 -- 172.21.15.31:0/2709252147 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f604c0c94a0 con 0x7f604c153c30 2023-10-06T02:31:59.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.777+0000 7f603a7fc700 1 -- 172.21.15.31:0/2709252147 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6040011ad0 con 0x7f604c153c30 2023-10-06T02:31:59.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.777+0000 7f604b59e700 1 -- 172.21.15.31:0/2709252147 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f604c0b43f0 con 0x7f604c153c30 2023-10-06T02:31:59.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.778+0000 7f604b59e700 1 -- 172.21.15.31:0/2709252147 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f604c0c99c0 con 0x7f604c153c30 2023-10-06T02:31:59.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.780+0000 7f603a7fc700 1 -- 172.21.15.31:0/2709252147 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85556+0+0 (secure 0 0 0) 0x7f604001b3b0 con 0x7f604c153c30 2023-10-06T02:31:59.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.781+0000 7f603a7fc700 1 --2- 172.21.15.31:0/2709252147 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f602c067830 0x7f602c069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:31:59.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.781+0000 7f603a7fc700 1 -- 172.21.15.31:0/2709252147 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5320+0+0 (secure 0 0 0) 0x7f60400823c0 con 0x7f604c153c30 2023-10-06T02:31:59.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.781+0000 7f604a59c700 1 --2- 172.21.15.31:0/2709252147 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f602c067830 0x7f602c069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:31:59.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.782+0000 7f604a59c700 1 --2- 172.21.15.31:0/2709252147 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f602c067830 0x7f602c069ce0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f603c0091e0 tx=0x7f603c017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:31:59.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:31:59.784+0000 7f603a7fc700 1 -- 172.21.15.31:0/2709252147 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f604c0c99c0 con 0x7f604c153c30 2023-10-06T02:32:00.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:00.069+0000 7f604b59e700 1 -- 172.21.15.31:0/2709252147 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade"]} v 0) v1 -- 0x7f604c0c99c0 con 0x7f604c153c30 2023-10-06T02:32:00.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:00.564+0000 7f603a7fc700 1 -- 172.21.15.31:0/2709252147 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade"]}]=0 v495) v1 ==== 166+0+0 (secure 0 0 0) 0x7f604c0c99c0 con 0x7f604c153c30 2023-10-06T02:32:00.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:00.566+0000 7f604b59e700 1 -- 172.21.15.31:0/2709252147 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f602c067830 msgr2=0x7f602c069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:00.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:00.566+0000 7f604b59e700 1 --2- 172.21.15.31:0/2709252147 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f602c067830 0x7f602c069ce0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f603c0091e0 tx=0x7f603c017000 comp rx=0 tx=0).stop 2023-10-06T02:32:00.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:00.566+0000 7f604b59e700 1 -- 172.21.15.31:0/2709252147 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f604c153c30 msgr2=0x7f604c0c9ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:00.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:00.566+0000 7f604b59e700 1 --2- 172.21.15.31:0/2709252147 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f604c153c30 0x7f604c0c9ca0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f604000c4e0 tx=0x7f604000c8a0 comp rx=0 tx=0).stop 2023-10-06T02:32:00.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:00.566+0000 7f604b59e700 1 -- 172.21.15.31:0/2709252147 shutdown_connections 2023-10-06T02:32:00.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:00.566+0000 7f604b59e700 1 --2- 172.21.15.31:0/2709252147 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f604c15de50 0x7f604c0cc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:00.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:00.566+0000 7f604b59e700 1 --2- 172.21.15.31:0/2709252147 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f602c067830 0x7f602c069ce0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:00.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:00.566+0000 7f604b59e700 1 --2- 172.21.15.31:0/2709252147 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f604c0cf110 0x7f604c0b5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:00.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:00.566+0000 7f604b59e700 1 --2- 172.21.15.31:0/2709252147 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f604c153c30 0x7f604c0c9ca0 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:00.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:00.566+0000 7f604b59e700 1 -- 172.21.15.31:0/2709252147 >> 172.21.15.31:0/2709252147 conn(0x7f604c0703a0 msgr2=0x7f604c151010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:00.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:00.567+0000 7f604b59e700 1 -- 172.21.15.31:0/2709252147 shutdown_connections 2023-10-06T02:32:00.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:00.567+0000 7f604b59e700 1 -- 172.21.15.31:0/2709252147 wait complete. 2023-10-06T02:32:00.587 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T02:32:00.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:00.779+0000 7fc0be59d700 1 Processor -- start 2023-10-06T02:32:00.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:00.779+0000 7fc0be59d700 1 -- start start 2023-10-06T02:32:00.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:00.781+0000 7fc0be59d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc0b8142c00 0x7fc0b806b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:00.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:00.781+0000 7fc0be59d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc0b80c7f90 0x7fc0b806c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:00.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:00.781+0000 7fc0be59d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0b80ca480 con 0x7fc0b806bd20 2023-10-06T02:32:00.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:00.781+0000 7fc0be59d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0b815e240 con 0x7fc0b80c7f90 2023-10-06T02:32:00.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:00.781+0000 7fc0be59d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0b806b350 con 0x7fc0b8142c00 2023-10-06T02:32:00.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:00.781+0000 7fc0bdd9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc0b80c7f90 0x7fc0b806c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:00.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:00.781+0000 7fc0bcd9a700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fc0b806bd20 0x7fc0b80c7880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:43876/0 (socket says 172.21.15.31:43876) 2023-10-06T02:32:00.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:00.781+0000 7fc0bcd9a700 1 -- 172.21.15.31:0/3321625850 learned_addr learned my addr 172.21.15.31:0/3321625850 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:32:00.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:00.781+0000 7fc0bd59b700 1 --2- 172.21.15.31:0/3321625850 >> v2:172.21.15.62:3300/0 conn(0x7fc0b8142c00 0x7fc0b806b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:00.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:00.781+0000 7fc0bdd9c700 1 -- 172.21.15.31:0/3321625850 >> v2:172.21.15.62:3300/0 conn(0x7fc0b8142c00 msgr2=0x7fc0b806b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:00.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:00.781+0000 7fc0bdd9c700 1 --2- 172.21.15.31:0/3321625850 >> v2:172.21.15.62:3300/0 conn(0x7fc0b8142c00 0x7fc0b806b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:00.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:00.781+0000 7fc0bdd9c700 1 -- 172.21.15.31:0/3321625850 >> v1:172.21.15.31:6789/0 conn(0x7fc0b806bd20 legacy=0x7fc0b80c7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:00.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:00.781+0000 7fc0bdd9c700 1 -- 172.21.15.31:0/3321625850 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0b815df60 con 0x7fc0b80c7f90 2023-10-06T02:32:00.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:00.781+0000 7fc0affff700 1 -- 172.21.15.31:0/3321625850 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 995320648 0 0) 0x7fc0b80ca480 con 0x7fc0b806bd20 2023-10-06T02:32:00.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:00.782+0000 7fc0bdd9c700 1 --2- 172.21.15.31:0/3321625850 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0b80c7f90 0x7fc0b806c6a0 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7fc0b806f490 tx=0x7fc0b0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2634f584252b1ca server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:00.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:00.782+0000 7fc0affff700 1 -- 172.21.15.31:0/3321625850 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b000e070 con 0x7fc0b80c7f90 2023-10-06T02:32:00.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:00.782+0000 7fc0affff700 1 -- 172.21.15.31:0/3321625850 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0b000bb60 con 0x7fc0b80c7f90 2023-10-06T02:32:00.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:00.782+0000 7fc0affff700 1 -- 172.21.15.31:0/3321625850 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b0012430 con 0x7fc0b80c7f90 2023-10-06T02:32:00.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:00.782+0000 7fc0be59d700 1 -- 172.21.15.31:0/3321625850 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0b80c7f90 msgr2=0x7fc0b806c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:00.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:00.782+0000 7fc0be59d700 1 --2- 172.21.15.31:0/3321625850 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0b80c7f90 0x7fc0b806c6a0 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7fc0b806f490 tx=0x7fc0b0009780 comp rx=0 tx=0).stop 2023-10-06T02:32:00.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:00.783+0000 7fc0be59d700 1 -- 172.21.15.31:0/3321625850 shutdown_connections 2023-10-06T02:32:00.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:00.783+0000 7fc0be59d700 1 --2- 172.21.15.31:0/3321625850 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0b80c7f90 0x7fc0b806c6a0 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:00.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:00.783+0000 7fc0be59d700 1 --2- 172.21.15.31:0/3321625850 >> v2:172.21.15.62:3300/0 conn(0x7fc0b8142c00 0x7fc0b806b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:00.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:00.783+0000 7fc0be59d700 1 -- 172.21.15.31:0/3321625850 >> 172.21.15.31:0/3321625850 conn(0x7fc0b80675a0 msgr2=0x7fc0b80679a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:00.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:00.783+0000 7fc0be59d700 1 -- 172.21.15.31:0/3321625850 shutdown_connections 2023-10-06T02:32:00.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:00.783+0000 7fc0be59d700 1 -- 172.21.15.31:0/3321625850 wait complete. 2023-10-06T02:32:00.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:00.784+0000 7fc0be59d700 1 Processor -- start 2023-10-06T02:32:00.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:00.784+0000 7fc0be59d700 1 -- start start 2023-10-06T02:32:00.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:00.785+0000 7fc0be59d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0b806bd20 0x7fc0b80b2780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:00.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:00.785+0000 7fc0be59d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0b80c7f90 0x7fc0b80c6b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:00.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:00.785+0000 7fc0be59d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc0b8142c00 0x7fc0b80c35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:00.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:00.785+0000 7fc0be59d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b80c66d0 con 0x7fc0b806bd20 2023-10-06T02:32:00.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:00.785+0000 7fc0be59d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b80c5ac0 con 0x7fc0b80c7f90 2023-10-06T02:32:00.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:00.785+0000 7fc0be59d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b80c5da0 con 0x7fc0b8142c00 2023-10-06T02:32:00.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:00.785+0000 7fc0bd59b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0b806bd20 0x7fc0b80b2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:00.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:00.785+0000 7fc0bd59b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0b806bd20 0x7fc0b80b2780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37366/0 (socket says 172.21.15.31:37366) 2023-10-06T02:32:00.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:00.786+0000 7fc0bd59b700 1 -- 172.21.15.31:0/1200895394 learned_addr learned my addr 172.21.15.31:0/1200895394 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:00.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:00.786+0000 7fc0bdd9c700 1 --2- 172.21.15.31:0/1200895394 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc0b8142c00 0x7fc0b80c35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:00.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:00.786+0000 7fc0bcd9a700 1 --2- 172.21.15.31:0/1200895394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0b80c7f90 0x7fc0b80c6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:00.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:00.786+0000 7fc0bd59b700 1 -- 172.21.15.31:0/1200895394 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc0b8142c00 msgr2=0x7fc0b80c35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:00.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:00.786+0000 7fc0bd59b700 1 --2- 172.21.15.31:0/1200895394 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc0b8142c00 0x7fc0b80c35b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:00.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:00.786+0000 7fc0bd59b700 1 -- 172.21.15.31:0/1200895394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0b80c7f90 msgr2=0x7fc0b80c6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:00.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:00.786+0000 7fc0bd59b700 1 --2- 172.21.15.31:0/1200895394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0b80c7f90 0x7fc0b80c6b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:00.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:00.786+0000 7fc0bd59b700 1 -- 172.21.15.31:0/1200895394 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0b0009a20 con 0x7fc0b806bd20 2023-10-06T02:32:00.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:00.786+0000 7fc0bd59b700 1 --2- 172.21.15.31:0/1200895394 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0b806bd20 0x7fc0b80b2780 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7fc0b400e930 tx=0x7fc0b40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:00.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:00.786+0000 7fc0ad7fa700 1 -- 172.21.15.31:0/1200895394 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b4013680 con 0x7fc0b806bd20 2023-10-06T02:32:00.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:00.787+0000 7fc0ad7fa700 1 -- 172.21.15.31:0/1200895394 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0b40137e0 con 0x7fc0b806bd20 2023-10-06T02:32:00.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:00.787+0000 7fc0ad7fa700 1 -- 172.21.15.31:0/1200895394 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b4013b80 con 0x7fc0b806bd20 2023-10-06T02:32:00.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:00.787+0000 7fc0be59d700 1 -- 172.21.15.31:0/1200895394 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0b80c6080 con 0x7fc0b806bd20 2023-10-06T02:32:00.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:00.787+0000 7fc0be59d700 1 -- 172.21.15.31:0/1200895394 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0b80c70a0 con 0x7fc0b806bd20 2023-10-06T02:32:00.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:00.788+0000 7fc0be59d700 1 -- 172.21.15.31:0/1200895394 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0b804c3c0 con 0x7fc0b806bd20 2023-10-06T02:32:00.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:00.789+0000 7fc0ad7fa700 1 -- 172.21.15.31:0/1200895394 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85556+0+0 (secure 0 0 0) 0x7fc0b400c3e0 con 0x7fc0b806bd20 2023-10-06T02:32:00.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:00.790+0000 7fc0ad7fa700 1 --2- 172.21.15.31:0/1200895394 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc09c067830 0x7fc09c069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:00.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:00.790+0000 7fc0ad7fa700 1 -- 172.21.15.31:0/1200895394 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5320+0+0 (secure 0 0 0) 0x7fc0b4083130 con 0x7fc0b806bd20 2023-10-06T02:32:00.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:00.790+0000 7fc0bcd9a700 1 --2- 172.21.15.31:0/1200895394 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc09c067830 0x7fc09c069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:00.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:00.791+0000 7fc0bcd9a700 1 --2- 172.21.15.31:0/1200895394 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc09c067830 0x7fc09c069ce0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fc0b80b2570 tx=0x7fc0a8007d30 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:00.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:00.794+0000 7fc0ad7fa700 1 -- 172.21.15.31:0/1200895394 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc0b4051b80 con 0x7fc0b806bd20 2023-10-06T02:32:00.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:00.999+0000 7fc0be59d700 1 -- 172.21.15.31:0/1200895394 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fc0b806ad00 con 0x7fc0b806bd20 2023-10-06T02:32:01.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.003+0000 7fc0ad7fa700 1 -- 172.21.15.31:0/1200895394 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v30)=0 v30) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc0b40551a0 con 0x7fc0b806bd20 2023-10-06T02:32:01.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.006+0000 7fc0be59d700 1 -- 172.21.15.31:0/1200895394 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc09c067830 msgr2=0x7fc09c069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:01.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.006+0000 7fc0be59d700 1 --2- 172.21.15.31:0/1200895394 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc09c067830 0x7fc09c069ce0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fc0b80b2570 tx=0x7fc0a8007d30 comp rx=0 tx=0).stop 2023-10-06T02:32:01.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.006+0000 7fc0be59d700 1 -- 172.21.15.31:0/1200895394 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0b806bd20 msgr2=0x7fc0b80b2780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:01.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.006+0000 7fc0be59d700 1 --2- 172.21.15.31:0/1200895394 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0b806bd20 0x7fc0b80b2780 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7fc0b400e930 tx=0x7fc0b40091a0 comp rx=0 tx=0).stop 2023-10-06T02:32:01.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.006+0000 7fc0be59d700 1 -- 172.21.15.31:0/1200895394 shutdown_connections 2023-10-06T02:32:01.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.006+0000 7fc0be59d700 1 --2- 172.21.15.31:0/1200895394 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc09c067830 0x7fc09c069ce0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:01.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.006+0000 7fc0be59d700 1 --2- 172.21.15.31:0/1200895394 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc0b8142c00 0x7fc0b80c35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:01.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.006+0000 7fc0be59d700 1 --2- 172.21.15.31:0/1200895394 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0b806bd20 0x7fc0b80b2780 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:01.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.006+0000 7fc0be59d700 1 --2- 172.21.15.31:0/1200895394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0b80c7f90 0x7fc0b80c6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:01.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.006+0000 7fc0be59d700 1 -- 172.21.15.31:0/1200895394 >> 172.21.15.31:0/1200895394 conn(0x7fc0b80675a0 msgr2=0x7fc0b80c9310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:01.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.006+0000 7fc0be59d700 1 -- 172.21.15.31:0/1200895394 shutdown_connections 2023-10-06T02:32:01.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.006+0000 7fc0be59d700 1 -- 172.21.15.31:0/1200895394 wait complete. 2023-10-06T02:32:01.024 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T02:32:01.024 DEBUG:teuthology.orchestra.run.smithi031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T02:32:01.123 DEBUG:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T02:32:01.221 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T02:32:01.221 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T02:32:01.222 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:32:01.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.408+0000 7f345259d700 1 Processor -- start 2023-10-06T02:32:01.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.408+0000 7f345259d700 1 -- start start 2023-10-06T02:32:01.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.409+0000 7f345259d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f344c13fb20 0x7f344c15a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:01.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.409+0000 7f345259d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f344c13fef0 0x7f344c0c7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:01.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.409+0000 7f345259d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f344c15ae70 con 0x7f344c06aaa0 2023-10-06T02:32:01.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.409+0000 7f345259d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f344c06af60 con 0x7f344c13fef0 2023-10-06T02:32:01.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.409+0000 7f345259d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f344c06b240 con 0x7f344c13fb20 2023-10-06T02:32:01.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.410+0000 7f345159b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f344c13fb20 0x7f344c15a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:01.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.410+0000 7f3450d9a700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f344c13fef0 0x7f344c0c7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:01.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.410+0000 7f345159b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f344c13fb20 0x7f344c15a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:35240/0 (socket says 172.21.15.31:35240) 2023-10-06T02:32:01.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.410+0000 7f345159b700 1 -- 172.21.15.31:0/2502572021 learned_addr learned my addr 172.21.15.31:0/2502572021 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:01.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.410+0000 7f3451d9c700 1 --1- 172.21.15.31:0/2502572021 >> v1:172.21.15.62:6789/0 conn(0x7f344c06aaa0 0x7f344c0c6770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:46256/0 (socket says 172.21.15.31:46256) 2023-10-06T02:32:01.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.410+0000 7f3443fff700 1 -- 172.21.15.31:0/2502572021 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2619139724 0 0) 0x7f3444001490 con 0x7f344c06aaa0 2023-10-06T02:32:01.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.410+0000 7f3443fff700 1 -- 172.21.15.31:0/2502572021 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3438003570 con 0x7f344c06aaa0 2023-10-06T02:32:01.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.410+0000 7f345159b700 1 -- 172.21.15.31:0/2502572021 >> v2:172.21.15.62:3300/0 conn(0x7f344c13fef0 msgr2=0x7f344c0c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:01.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.410+0000 7f345159b700 1 --2- 172.21.15.31:0/2502572021 >> v2:172.21.15.62:3300/0 conn(0x7f344c13fef0 0x7f344c0c7990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:01.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.410+0000 7f345159b700 1 -- 172.21.15.31:0/2502572021 >> v1:172.21.15.62:6789/0 conn(0x7f344c06aaa0 legacy=0x7f344c0c6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:01.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.410+0000 7f345159b700 1 -- 172.21.15.31:0/2502572021 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f344c06b520 con 0x7f344c13fb20 2023-10-06T02:32:01.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.410+0000 7f3450d9a700 1 --2- 172.21.15.31:0/2502572021 >> v2:172.21.15.62:3300/0 conn(0x7f344c13fef0 0x7f344c0c7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:32:01.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.411+0000 7f345159b700 1 --2- 172.21.15.31:0/2502572021 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f344c13fb20 0x7f344c15a930 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f34480099b0 tx=0x7f3448009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3cb81d2f44287fd0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:01.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.411+0000 7f3443fff700 1 -- 172.21.15.31:0/2502572021 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f344800b680 con 0x7f344c13fb20 2023-10-06T02:32:01.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.411+0000 7f3443fff700 1 -- 172.21.15.31:0/2502572021 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f344800b7e0 con 0x7f344c13fb20 2023-10-06T02:32:01.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.411+0000 7f3443fff700 1 -- 172.21.15.31:0/2502572021 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f344800bb80 con 0x7f344c13fb20 2023-10-06T02:32:01.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.412+0000 7f345259d700 1 -- 172.21.15.31:0/2502572021 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f344c13fb20 msgr2=0x7f344c15a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:01.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.412+0000 7f345259d700 1 --2- 172.21.15.31:0/2502572021 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f344c13fb20 0x7f344c15a930 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f34480099b0 tx=0x7f3448009250 comp rx=0 tx=0).stop 2023-10-06T02:32:01.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.412+0000 7f345259d700 1 -- 172.21.15.31:0/2502572021 shutdown_connections 2023-10-06T02:32:01.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.412+0000 7f345259d700 1 --2- 172.21.15.31:0/2502572021 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f344c13fb20 0x7f344c15a930 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:01.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.412+0000 7f345259d700 1 --2- 172.21.15.31:0/2502572021 >> v2:172.21.15.62:3300/0 conn(0x7f344c13fef0 0x7f344c0c7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:01.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.412+0000 7f345259d700 1 -- 172.21.15.31:0/2502572021 >> 172.21.15.31:0/2502572021 conn(0x7f344c06d170 msgr2=0x7f344c0cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:01.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.412+0000 7f345259d700 1 -- 172.21.15.31:0/2502572021 shutdown_connections 2023-10-06T02:32:01.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.412+0000 7f345259d700 1 -- 172.21.15.31:0/2502572021 wait complete. 2023-10-06T02:32:01.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.413+0000 7f345259d700 1 Processor -- start 2023-10-06T02:32:01.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.413+0000 7f345259d700 1 -- start start 2023-10-06T02:32:01.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.414+0000 7f345259d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f344c06aaa0 0x7f344c0ac3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:01.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.414+0000 7f345259d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f344c13fb20 0x7f344c1560a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:01.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.414+0000 7f345259d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f344c13fef0 0x7f344c1370a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:01.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.414+0000 7f345259d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f344c157ee0 con 0x7f344c06aaa0 2023-10-06T02:32:01.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.414+0000 7f345259d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f344c1642d0 con 0x7f344c13fef0 2023-10-06T02:32:01.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.414+0000 7f345259d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f344c135610 con 0x7f344c13fb20 2023-10-06T02:32:01.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.414+0000 7f345159b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f344c06aaa0 0x7f344c0ac3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:01.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.415+0000 7f345159b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f344c06aaa0 0x7f344c0ac3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37378/0 (socket says 172.21.15.31:37378) 2023-10-06T02:32:01.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.415+0000 7f345159b700 1 -- 172.21.15.31:0/4101480374 learned_addr learned my addr 172.21.15.31:0/4101480374 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:01.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.415+0000 7f3450d9a700 1 --2- 172.21.15.31:0/4101480374 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f344c13fb20 0x7f344c1560a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:01.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.415+0000 7f3451d9c700 1 --2- 172.21.15.31:0/4101480374 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f344c13fef0 0x7f344c1370a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:01.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.415+0000 7f345159b700 1 -- 172.21.15.31:0/4101480374 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f344c13fb20 msgr2=0x7f344c1560a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:01.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.415+0000 7f345159b700 1 --2- 172.21.15.31:0/4101480374 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f344c13fb20 0x7f344c1560a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:01.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.415+0000 7f345159b700 1 -- 172.21.15.31:0/4101480374 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f344c13fef0 msgr2=0x7f344c1370a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:01.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.415+0000 7f345159b700 1 --2- 172.21.15.31:0/4101480374 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f344c13fef0 0x7f344c1370a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:01.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.415+0000 7f345159b700 1 -- 172.21.15.31:0/4101480374 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f343c0093c0 con 0x7f344c06aaa0 2023-10-06T02:32:01.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.415+0000 7f345159b700 1 --2- 172.21.15.31:0/4101480374 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f344c06aaa0 0x7f344c0ac3d0 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f3448009290 tx=0x7f3448017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:01.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.415+0000 7f34417fa700 1 -- 172.21.15.31:0/4101480374 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3448002f90 con 0x7f344c06aaa0 2023-10-06T02:32:01.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.416+0000 7f34417fa700 1 -- 172.21.15.31:0/4101480374 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f344800b950 con 0x7f344c06aaa0 2023-10-06T02:32:01.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.416+0000 7f34417fa700 1 -- 172.21.15.31:0/4101480374 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f344801a430 con 0x7f344c06aaa0 2023-10-06T02:32:01.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.416+0000 7f345259d700 1 -- 172.21.15.31:0/4101480374 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34480094f0 con 0x7f344c06aaa0 2023-10-06T02:32:01.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.416+0000 7f345259d700 1 -- 172.21.15.31:0/4101480374 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f344c157b70 con 0x7f344c06aaa0 2023-10-06T02:32:01.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.417+0000 7f345259d700 1 -- 172.21.15.31:0/4101480374 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f344c04c3c0 con 0x7f344c06aaa0 2023-10-06T02:32:01.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.418+0000 7f34417fa700 1 -- 172.21.15.31:0/4101480374 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85556+0+0 (secure 0 0 0) 0x7f344801a7e0 con 0x7f344c06aaa0 2023-10-06T02:32:01.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.418+0000 7f34417fa700 1 --2- 172.21.15.31:0/4101480374 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f3430067830 0x7f3430069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:01.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.418+0000 7f34417fa700 1 -- 172.21.15.31:0/4101480374 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5320+0+0 (secure 0 0 0) 0x7f34480849c0 con 0x7f344c06aaa0 2023-10-06T02:32:01.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.419+0000 7f3450d9a700 1 --2- 172.21.15.31:0/4101480374 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f3430067830 0x7f3430069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:01.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.419+0000 7f3450d9a700 1 --2- 172.21.15.31:0/4101480374 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f3430067830 0x7f3430069ce0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f344c16ab60 tx=0x7f343c009120 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:01.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.422+0000 7f34417fa700 1 -- 172.21.15.31:0/4101480374 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3448053600 con 0x7f344c06aaa0 2023-10-06T02:32:01.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.675+0000 7f345259d700 1 -- 172.21.15.31:0/4101480374 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f344c0c1740 con 0x7f344c06aaa0 2023-10-06T02:32:01.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.676+0000 7f34417fa700 1 -- 172.21.15.31:0/4101480374 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) v1 ==== 95+0+3531 (secure 0 0 0) 0x7f3448015de0 con 0x7f344c06aaa0 2023-10-06T02:32:01.677 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:32:01.677 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":112,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7178,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/3218480514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3218480514},{"type":"v1","addr":"172.21.15.62:6834","nonce":3218480514}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104},{"gid":8323,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6833/832192876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":832192876},{"type":"v1","addr":"172.21.15.31:6833","nonce":832192876}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104},{"gid":8614,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1229141759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":1229141759},{"type":"v1","addr":"172.21.15.62:6835","nonce":1229141759}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":112},{"gid":8877,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/2374603733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":2374603733},{"type":"v1","addr":"172.21.15.31:6835","nonce":2374603733}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104}],"filesystems":[]} 2023-10-06T02:32:01.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.679+0000 7f345259d700 1 -- 172.21.15.31:0/4101480374 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f3430067830 msgr2=0x7f3430069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:01.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.679+0000 7f345259d700 1 --2- 172.21.15.31:0/4101480374 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f3430067830 0x7f3430069ce0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f344c16ab60 tx=0x7f343c009120 comp rx=0 tx=0).stop 2023-10-06T02:32:01.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.679+0000 7f345259d700 1 -- 172.21.15.31:0/4101480374 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f344c06aaa0 msgr2=0x7f344c0ac3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:01.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.679+0000 7f345259d700 1 --2- 172.21.15.31:0/4101480374 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f344c06aaa0 0x7f344c0ac3d0 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f3448009290 tx=0x7f3448017000 comp rx=0 tx=0).stop 2023-10-06T02:32:01.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.679+0000 7f345259d700 1 -- 172.21.15.31:0/4101480374 shutdown_connections 2023-10-06T02:32:01.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.679+0000 7f345259d700 1 --2- 172.21.15.31:0/4101480374 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f344c13fef0 0x7f344c1370a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:01.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.679+0000 7f345259d700 1 --2- 172.21.15.31:0/4101480374 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f344c06aaa0 0x7f344c0ac3d0 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:01.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.679+0000 7f345259d700 1 --2- 172.21.15.31:0/4101480374 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f3430067830 0x7f3430069ce0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:01.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.679+0000 7f345259d700 1 --2- 172.21.15.31:0/4101480374 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f344c13fb20 0x7f344c1560a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:01.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.679+0000 7f345259d700 1 -- 172.21.15.31:0/4101480374 >> 172.21.15.31:0/4101480374 conn(0x7f344c06d170 msgr2=0x7f344c0c1f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:01.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.679+0000 7f345259d700 1 -- 172.21.15.31:0/4101480374 shutdown_connections 2023-10-06T02:32:01.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.679+0000 7f345259d700 1 -- 172.21.15.31:0/4101480374 wait complete. 2023-10-06T02:32:01.681 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 112 2023-10-06T02:32:01.697 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T02:32:01.697 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T02:32:01.698 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T02:32:01.698 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T02:32:01.698 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T02:32:01.698 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T02:32:01.698 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T02:32:01.698 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T02:32:01.698 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T02:32:01.699 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T02:32:01.699 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T02:32:01.699 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T02:32:01.699 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T02:32:01.699 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T02:32:01.699 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T02:32:01.699 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T02:32:01.710 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T02:32:01.711 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T02:32:01.711 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T02:32:01.711 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T02:32:01.715 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T02:32:01.715 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T02:32:01.716 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T02:32:01.716 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T02:32:01.755 INFO:tasks.ceph.mds.a:Started 2023-10-06T02:32:01.755 DEBUG:teuthology.parallel:result is None 2023-10-06T02:32:01.758 INFO:tasks.ceph.mds.c:Started 2023-10-06T02:32:01.758 DEBUG:teuthology.parallel:result is None 2023-10-06T02:32:01.760 INFO:tasks.ceph.mds.b:Started 2023-10-06T02:32:01.761 DEBUG:teuthology.parallel:result is None 2023-10-06T02:32:01.763 INFO:tasks.ceph.mds.d:Started 2023-10-06T02:32:01.764 DEBUG:teuthology.parallel:result is None 2023-10-06T02:32:01.764 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T02:32:01.893 INFO:tasks.ceph.mds.c.smithi031.stdout:starting mds.c at 2023-10-06T02:32:01.894 INFO:tasks.ceph.mds.a.smithi031.stdout:starting mds.a at 2023-10-06T02:32:01.903 INFO:tasks.ceph.mds.d.smithi062.stdout:starting mds.d at 2023-10-06T02:32:01.903 INFO:tasks.ceph.mds.b.smithi062.stdout:starting mds.b at 2023-10-06T02:32:01.905 INFO:tasks.ceph.mds.d.smithi062.stderr:2023-10-06T02:32:01.905+0000 7f5cd8dfd700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2023-10-06T02:32:01.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.960+0000 7f374b5ff700 1 Processor -- start 2023-10-06T02:32:01.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.961+0000 7f374b5ff700 1 -- start start 2023-10-06T02:32:01.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.962+0000 7f374b5ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f374415a5d0 0x7f3744156e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:01.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.962+0000 7f374b5ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f374415ae40 con 0x7f37440cb5a0 2023-10-06T02:32:01.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.962+0000 7f374b5ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f374415ab60 con 0x7f37440cb970 2023-10-06T02:32:01.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.962+0000 7f374b5ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37440ce0f0 con 0x7f374415a5d0 2023-10-06T02:32:01.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.962+0000 7f374939b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f37440cb5a0 0x7f37440cd9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:43900/0 (socket says 172.21.15.31:43900) 2023-10-06T02:32:01.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.962+0000 7f374939b700 1 -- 172.21.15.31:0/2973038486 learned_addr learned my addr 172.21.15.31:0/2973038486 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:32:01.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.962+0000 7f373bfff700 1 -- 172.21.15.31:0/2973038486 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2184787374 0 0) 0x7f374415ae40 con 0x7f37440cb5a0 2023-10-06T02:32:01.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.962+0000 7f3749b9c700 1 --2- 172.21.15.31:0/2973038486 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f374415a5d0 0x7f3744156e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:01.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.963+0000 7f373bfff700 1 -- 172.21.15.31:0/2973038486 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3730003570 con 0x7f37440cb5a0 2023-10-06T02:32:01.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.963+0000 7f373bfff700 1 -- 172.21.15.31:0/2973038486 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1936013344 0 0) 0x7f3730003570 con 0x7f37440cb5a0 2023-10-06T02:32:01.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.963+0000 7f373bfff700 1 -- 172.21.15.31:0/2973038486 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f374415a5d0 msgr2=0x7f3744156e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:01.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.963+0000 7f373bfff700 1 --2- 172.21.15.31:0/2973038486 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f374415a5d0 0x7f3744156e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:01.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.963+0000 7f373bfff700 1 -- 172.21.15.31:0/2973038486 >> v1:172.21.15.62:6789/0 conn(0x7f37440cb970 legacy=0x7f3744159ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:01.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.963+0000 7f373bfff700 1 -- 172.21.15.31:0/2973038486 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37440ccf00 con 0x7f37440cb5a0 2023-10-06T02:32:01.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.963+0000 7f373bfff700 1 -- 172.21.15.31:0/2973038486 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7f3740001470 con 0x7f37440cb5a0 2023-10-06T02:32:01.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.963+0000 7f373bfff700 1 -- 172.21.15.31:0/2973038486 >> v1:172.21.15.31:6789/0 conn(0x7f37440cb5a0 legacy=0x7f37440cd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:01.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.963+0000 7f373bfff700 1 --2- 172.21.15.31:0/2973038486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3730003a90 0x7f3730003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:01.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.963+0000 7f373bfff700 1 --2- 172.21.15.31:0/2973038486 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3730004810 0x7f3730004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:01.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.963+0000 7f373bfff700 1 --2- 172.21.15.31:0/2973038486 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f374415a5d0 0x7f37300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:01.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.963+0000 7f373bfff700 1 -- 172.21.15.31:0/2973038486 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37440ce0f0 con 0x7f3730004810 2023-10-06T02:32:01.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.963+0000 7f373bfff700 1 -- 172.21.15.31:0/2973038486 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f373000bb40 con 0x7f3730003a90 2023-10-06T02:32:01.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.963+0000 7f373bfff700 1 -- 172.21.15.31:0/2973038486 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f373000bd00 con 0x7f374415a5d0 2023-10-06T02:32:01.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.963+0000 7f374939b700 1 --2- 172.21.15.31:0/2973038486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3730003a90 0x7f3730003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:01.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.963+0000 7f3749b9c700 1 --2- 172.21.15.31:0/2973038486 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f374415a5d0 0x7f37300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:01.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.963+0000 7f3748b9a700 1 --2- 172.21.15.31:0/2973038486 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3730004810 0x7f3730004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:01.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.964+0000 7f374939b700 1 -- 172.21.15.31:0/2973038486 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f374415a5d0 msgr2=0x7f37300051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:01.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.964+0000 7f374939b700 1 --2- 172.21.15.31:0/2973038486 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f374415a5d0 0x7f37300051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:01.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.964+0000 7f374939b700 1 -- 172.21.15.31:0/2973038486 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3730004810 msgr2=0x7f3730004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:01.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.964+0000 7f374939b700 1 --2- 172.21.15.31:0/2973038486 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3730004810 0x7f3730004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:01.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.964+0000 7f374939b700 1 -- 172.21.15.31:0/2973038486 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3730003570 con 0x7f3730003a90 2023-10-06T02:32:01.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.964+0000 7f3748b9a700 1 --2- 172.21.15.31:0/2973038486 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3730004810 0x7f3730004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:32:01.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.964+0000 7f374939b700 1 --2- 172.21.15.31:0/2973038486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3730003a90 0x7f3730003e60 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f374000c2d0 tx=0x7f3740001da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a8858ec90eb68ca server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:01.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.964+0000 7f373bfff700 1 -- 172.21.15.31:0/2973038486 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f374000cba0 con 0x7f3730003a90 2023-10-06T02:32:01.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.964+0000 7f373bfff700 1 -- 172.21.15.31:0/2973038486 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f374000cd20 con 0x7f3730003a90 2023-10-06T02:32:01.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.964+0000 7f373bfff700 1 -- 172.21.15.31:0/2973038486 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3740005b70 con 0x7f3730003a90 2023-10-06T02:32:01.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.965+0000 7f374b5ff700 1 -- 172.21.15.31:0/2973038486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3730003a90 msgr2=0x7f3730003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:01.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.965+0000 7f374b5ff700 1 --2- 172.21.15.31:0/2973038486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3730003a90 0x7f3730003e60 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f374000c2d0 tx=0x7f3740001da0 comp rx=0 tx=0).stop 2023-10-06T02:32:01.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.965+0000 7f3749b9c700 1 -- 172.21.15.31:0/2973038486 reap_dead start 2023-10-06T02:32:01.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.965+0000 7f374b5ff700 1 -- 172.21.15.31:0/2973038486 shutdown_connections 2023-10-06T02:32:01.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.965+0000 7f374b5ff700 1 -- 172.21.15.31:0/2973038486 >> 172.21.15.31:0/2973038486 conn(0x7f3744070e30 msgr2=0x7f37440d0710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:01.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.965+0000 7f374b5ff700 1 -- 172.21.15.31:0/2973038486 shutdown_connections 2023-10-06T02:32:01.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.965+0000 7f374b5ff700 1 -- 172.21.15.31:0/2973038486 wait complete. 2023-10-06T02:32:01.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.966+0000 7f374b5ff700 1 Processor -- start 2023-10-06T02:32:01.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.967+0000 7f374b5ff700 1 -- start start 2023-10-06T02:32:01.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.967+0000 7f374b5ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3730003a90 0x7f374415b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:01.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.967+0000 7f374b5ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37440b34b0 0x7f37440c9b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:01.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.967+0000 7f374b5ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37440b3920 0x7f3744147da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:01.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.967+0000 7f374b5ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37440b53e0 con 0x7f37440b3920 2023-10-06T02:32:01.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.967+0000 7f374b5ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f374415b5b0 con 0x7f37440b34b0 2023-10-06T02:32:01.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.967+0000 7f374b5ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37440b3e20 con 0x7f3730003a90 2023-10-06T02:32:01.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.968+0000 7f3749b9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37440b3920 0x7f3744147da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:01.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.968+0000 7f3749b9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37440b3920 0x7f3744147da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37436/0 (socket says 172.21.15.31:37436) 2023-10-06T02:32:01.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.968+0000 7f3749b9c700 1 -- 172.21.15.31:0/3249295857 learned_addr learned my addr 172.21.15.31:0/3249295857 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:01.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.968+0000 7f3748b9a700 1 --2- 172.21.15.31:0/3249295857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37440b34b0 0x7f37440c9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:01.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.968+0000 7f374939b700 1 --2- 172.21.15.31:0/3249295857 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3730003a90 0x7f374415b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:01.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.968+0000 7f3749b9c700 1 -- 172.21.15.31:0/3249295857 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3730003a90 msgr2=0x7f374415b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:01.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.968+0000 7f3749b9c700 1 --2- 172.21.15.31:0/3249295857 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3730003a90 0x7f374415b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:01.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.968+0000 7f3749b9c700 1 -- 172.21.15.31:0/3249295857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37440b34b0 msgr2=0x7f37440c9b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:01.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.968+0000 7f3749b9c700 1 --2- 172.21.15.31:0/3249295857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37440b34b0 0x7f37440c9b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:01.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.968+0000 7f3749b9c700 1 -- 172.21.15.31:0/3249295857 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37440b4100 con 0x7f37440b3920 2023-10-06T02:32:01.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.968+0000 7f3749b9c700 1 --2- 172.21.15.31:0/3249295857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37440b3920 0x7f3744147da0 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f3744064ad0 tx=0x7f373c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:01.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.969+0000 7f37397fa700 1 -- 172.21.15.31:0/3249295857 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f373c00e040 con 0x7f37440b3920 2023-10-06T02:32:01.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.969+0000 7f37397fa700 1 -- 172.21.15.31:0/3249295857 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f373c00f040 con 0x7f37440b3920 2023-10-06T02:32:01.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.969+0000 7f37397fa700 1 -- 172.21.15.31:0/3249295857 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f373c00fd90 con 0x7f37440b3920 2023-10-06T02:32:01.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.969+0000 7f374b5ff700 1 -- 172.21.15.31:0/3249295857 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37440b43e0 con 0x7f37440b3920 2023-10-06T02:32:01.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.969+0000 7f374b5ff700 1 -- 172.21.15.31:0/3249295857 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37440ca0a0 con 0x7f37440b3920 2023-10-06T02:32:01.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.970+0000 7f374b5ff700 1 -- 172.21.15.31:0/3249295857 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37440ca320 con 0x7f37440b3920 2023-10-06T02:32:01.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.971+0000 7f37397fa700 1 -- 172.21.15.31:0/3249295857 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85556+0+0 (secure 0 0 0) 0x7f373c015640 con 0x7f37440b3920 2023-10-06T02:32:01.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.972+0000 7f37397fa700 1 --2- 172.21.15.31:0/3249295857 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f3728067830 0x7f3728069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:01.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.972+0000 7f37397fa700 1 -- 172.21.15.31:0/3249295857 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5320+0+0 (secure 0 0 0) 0x7f373c083710 con 0x7f37440b3920 2023-10-06T02:32:01.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.973+0000 7f374939b700 1 --2- 172.21.15.31:0/3249295857 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f3728067830 0x7f3728069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:01.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.973+0000 7f374939b700 1 --2- 172.21.15.31:0/3249295857 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f3728067830 0x7f3728069ce0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f3740007c20 tx=0x7f3740015000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:01.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:01.976+0000 7f37397fa700 1 -- 172.21.15.31:0/3249295857 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f373c052350 con 0x7f37440b3920 2023-10-06T02:32:02.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:02.195+0000 7f374b5ff700 1 -- 172.21.15.31:0/3249295857 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f374404c3c0 con 0x7f37440b3920 2023-10-06T02:32:02.416 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T02:32:02.421 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T02:32:02.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:02.585+0000 7f37397fa700 1 -- 172.21.15.31:0/3249295857 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v163) v1 ==== 97+0+0 (secure 0 0 0) 0x7f373c019080 con 0x7f37440b3920 2023-10-06T02:32:02.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:02.588+0000 7f374b5ff700 1 -- 172.21.15.31:0/3249295857 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f3728067830 msgr2=0x7f3728069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:02.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:02.588+0000 7f374b5ff700 1 --2- 172.21.15.31:0/3249295857 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f3728067830 0x7f3728069ce0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f3740007c20 tx=0x7f3740015000 comp rx=0 tx=0).stop 2023-10-06T02:32:02.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:02.588+0000 7f374b5ff700 1 -- 172.21.15.31:0/3249295857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37440b3920 msgr2=0x7f3744147da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:02.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:02.588+0000 7f374b5ff700 1 --2- 172.21.15.31:0/3249295857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37440b3920 0x7f3744147da0 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f3744064ad0 tx=0x7f373c00b040 comp rx=0 tx=0).stop 2023-10-06T02:32:02.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:02.588+0000 7f374b5ff700 1 -- 172.21.15.31:0/3249295857 shutdown_connections 2023-10-06T02:32:02.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:02.588+0000 7f374b5ff700 1 --2- 172.21.15.31:0/3249295857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37440b3920 0x7f3744147da0 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:02.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:02.588+0000 7f374b5ff700 1 --2- 172.21.15.31:0/3249295857 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f3728067830 0x7f3728069ce0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:02.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:02.588+0000 7f374b5ff700 1 --2- 172.21.15.31:0/3249295857 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3730003a90 0x7f374415b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:02.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:02.588+0000 7f374b5ff700 1 --2- 172.21.15.31:0/3249295857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37440b34b0 0x7f37440c9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:02.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:02.588+0000 7f374b5ff700 1 -- 172.21.15.31:0/3249295857 >> 172.21.15.31:0/3249295857 conn(0x7f3744070e30 msgr2=0x7f37440c54b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:02.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:02.588+0000 7f374b5ff700 1 -- 172.21.15.31:0/3249295857 shutdown_connections 2023-10-06T02:32:02.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:02.588+0000 7f374b5ff700 1 -- 172.21.15.31:0/3249295857 wait complete. 2023-10-06T02:32:02.590 INFO:teuthology.orchestra.run.smithi031.stderr: removed all blocklist entries 2023-10-06T02:32:02.608 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T02:32:02.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:02.795+0000 7f1c99f9d700 1 Processor -- start 2023-10-06T02:32:02.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:02.795+0000 7f1c99f9d700 1 -- start start 2023-10-06T02:32:02.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:02.796+0000 7f1c99f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1c9413fef0 0x7f1c9415a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:02.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:02.796+0000 7f1c99f9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c9415ae70 con 0x7f1c9406aaa0 2023-10-06T02:32:02.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:02.796+0000 7f1c99f9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c9406af60 con 0x7f1c9413fb20 2023-10-06T02:32:02.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:02.796+0000 7f1c99f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9406b240 con 0x7f1c9413fef0 2023-10-06T02:32:02.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:02.797+0000 7f1c9979c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f1c9406aaa0 0x7f1c940c9d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:43914/0 (socket says 172.21.15.31:43914) 2023-10-06T02:32:02.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:02.797+0000 7f1c9979c700 1 -- 172.21.15.31:0/3352975773 learned_addr learned my addr 172.21.15.31:0/3352975773 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:32:02.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:02.797+0000 7f1c93fff700 1 --2- 172.21.15.31:0/3352975773 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1c9413fef0 0x7f1c9415a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:02.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:02.797+0000 7f1c937fe700 1 -- 172.21.15.31:0/3352975773 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 583763420 0 0) 0x7f1c9406af60 con 0x7f1c9413fb20 2023-10-06T02:32:02.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:02.797+0000 7f1c937fe700 1 -- 172.21.15.31:0/3352975773 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c84003570 con 0x7f1c9413fb20 2023-10-06T02:32:02.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:02.797+0000 7f1c937fe700 1 -- 172.21.15.31:0/3352975773 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 383647780 0 0) 0x7f1c8c001490 con 0x7f1c9406aaa0 2023-10-06T02:32:02.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:02.797+0000 7f1c937fe700 1 -- 172.21.15.31:0/3352975773 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c9406af60 con 0x7f1c9406aaa0 2023-10-06T02:32:02.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:02.797+0000 7f1c93fff700 1 -- 172.21.15.31:0/3352975773 >> v1:172.21.15.62:6789/0 conn(0x7f1c9413fb20 legacy=0x7f1c940c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:02.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:02.797+0000 7f1c93fff700 1 -- 172.21.15.31:0/3352975773 >> v1:172.21.15.31:6789/0 conn(0x7f1c9406aaa0 legacy=0x7f1c940c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:02.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:02.797+0000 7f1c93fff700 1 -- 172.21.15.31:0/3352975773 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c9406b520 con 0x7f1c9413fef0 2023-10-06T02:32:02.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:02.798+0000 7f1c93fff700 1 --2- 172.21.15.31:0/3352975773 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1c9413fef0 0x7f1c9415a930 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1c80009900 tx=0x7f1c80009120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8395e1dfbc45c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:02.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:02.798+0000 7f1c937fe700 1 -- 172.21.15.31:0/3352975773 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c80005c10 con 0x7f1c9413fef0 2023-10-06T02:32:02.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:02.798+0000 7f1c937fe700 1 -- 172.21.15.31:0/3352975773 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c80005d70 con 0x7f1c9413fef0 2023-10-06T02:32:02.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:02.798+0000 7f1c937fe700 1 -- 172.21.15.31:0/3352975773 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c80006260 con 0x7f1c9413fef0 2023-10-06T02:32:02.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:02.798+0000 7f1c99f9d700 1 -- 172.21.15.31:0/3352975773 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1c9413fef0 msgr2=0x7f1c9415a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:02.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:02.798+0000 7f1c99f9d700 1 --2- 172.21.15.31:0/3352975773 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1c9413fef0 0x7f1c9415a930 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1c80009900 tx=0x7f1c80009120 comp rx=0 tx=0).stop 2023-10-06T02:32:02.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:02.799+0000 7f1c99f9d700 1 -- 172.21.15.31:0/3352975773 shutdown_connections 2023-10-06T02:32:02.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:02.799+0000 7f1c99f9d700 1 --2- 172.21.15.31:0/3352975773 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1c9413fef0 0x7f1c9415a930 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:02.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:02.799+0000 7f1c99f9d700 1 -- 172.21.15.31:0/3352975773 >> 172.21.15.31:0/3352975773 conn(0x7f1c9406d170 msgr2=0x7f1c940cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:02.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:02.799+0000 7f1c99f9d700 1 -- 172.21.15.31:0/3352975773 shutdown_connections 2023-10-06T02:32:02.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:02.799+0000 7f1c99f9d700 1 -- 172.21.15.31:0/3352975773 wait complete. 2023-10-06T02:32:02.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:02.800+0000 7f1c99f9d700 1 Processor -- start 2023-10-06T02:32:02.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:02.800+0000 7f1c99f9d700 1 -- start start 2023-10-06T02:32:02.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:02.801+0000 7f1c99f9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c9406aaa0 0x7f1c940c0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:02.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:02.801+0000 7f1c99f9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c9413fb20 0x7f1c940c2970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:02.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:02.801+0000 7f1c99f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1c9413fef0 0x7f1c940c7840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:02.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:02.801+0000 7f1c99f9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c940c1410 con 0x7f1c9406aaa0 2023-10-06T02:32:02.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:02.801+0000 7f1c99f9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9416a500 con 0x7f1c9413fb20 2023-10-06T02:32:02.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:02.801+0000 7f1c99f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c940c2eb0 con 0x7f1c9413fef0 2023-10-06T02:32:02.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:02.801+0000 7f1c98f9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c9406aaa0 0x7f1c940c0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:02.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:02.801+0000 7f1c98f9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c9406aaa0 0x7f1c940c0ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37444/0 (socket says 172.21.15.31:37444) 2023-10-06T02:32:02.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:02.801+0000 7f1c98f9b700 1 -- 172.21.15.31:0/1580920904 learned_addr learned my addr 172.21.15.31:0/1580920904 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:02.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:02.802+0000 7f1c9979c700 1 --2- 172.21.15.31:0/1580920904 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1c9413fef0 0x7f1c940c7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:02.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:02.802+0000 7f1c93fff700 1 --2- 172.21.15.31:0/1580920904 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c9413fb20 0x7f1c940c2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:02.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:02.802+0000 7f1c98f9b700 1 -- 172.21.15.31:0/1580920904 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1c9413fef0 msgr2=0x7f1c940c7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:02.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:02.802+0000 7f1c98f9b700 1 --2- 172.21.15.31:0/1580920904 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1c9413fef0 0x7f1c940c7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:02.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:02.802+0000 7f1c98f9b700 1 -- 172.21.15.31:0/1580920904 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c9413fb20 msgr2=0x7f1c940c2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:02.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:02.802+0000 7f1c98f9b700 1 --2- 172.21.15.31:0/1580920904 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c9413fb20 0x7f1c940c2970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:02.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:02.802+0000 7f1c98f9b700 1 -- 172.21.15.31:0/1580920904 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c800093c0 con 0x7f1c9406aaa0 2023-10-06T02:32:02.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:02.802+0000 7f1c98f9b700 1 --2- 172.21.15.31:0/1580920904 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c9406aaa0 0x7f1c940c0ed0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f1c8800c790 tx=0x7f1c88007eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:02.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:02.802+0000 7f1c90ff9700 1 -- 172.21.15.31:0/1580920904 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c88011590 con 0x7f1c9406aaa0 2023-10-06T02:32:02.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:02.802+0000 7f1c90ff9700 1 -- 172.21.15.31:0/1580920904 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c88011710 con 0x7f1c9406aaa0 2023-10-06T02:32:02.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:02.802+0000 7f1c99f9d700 1 -- 172.21.15.31:0/1580920904 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c94169980 con 0x7f1c9406aaa0 2023-10-06T02:32:02.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:02.802+0000 7f1c90ff9700 1 -- 172.21.15.31:0/1580920904 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c88011ad0 con 0x7f1c9406aaa0 2023-10-06T02:32:02.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:02.802+0000 7f1c99f9d700 1 -- 172.21.15.31:0/1580920904 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c94169f40 con 0x7f1c9406aaa0 2023-10-06T02:32:02.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:02.803+0000 7f1c99f9d700 1 -- 172.21.15.31:0/1580920904 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c9404c3c0 con 0x7f1c9406aaa0 2023-10-06T02:32:02.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:02.804+0000 7f1c90ff9700 1 -- 172.21.15.31:0/1580920904 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85556+0+0 (secure 0 0 0) 0x7f1c88012420 con 0x7f1c9406aaa0 2023-10-06T02:32:02.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:02.805+0000 7f1c90ff9700 1 --2- 172.21.15.31:0/1580920904 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f1c78067830 0x7f1c78069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:02.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:02.805+0000 7f1c90ff9700 1 -- 172.21.15.31:0/1580920904 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5234+0+0 (secure 0 0 0) 0x7f1c88082a30 con 0x7f1c9406aaa0 2023-10-06T02:32:02.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:02.806+0000 7f1c93fff700 1 --2- 172.21.15.31:0/1580920904 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f1c78067830 0x7f1c78069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:02.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:02.806+0000 7f1c93fff700 1 --2- 172.21.15.31:0/1580920904 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f1c78067830 0x7f1c78069ce0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f1c94069580 tx=0x7f1c80016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:02.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:02.809+0000 7f1c90ff9700 1 -- 172.21.15.31:0/1580920904 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1c880516d0 con 0x7f1c9406aaa0 2023-10-06T02:32:03.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.094+0000 7f1c99f9d700 1 -- 172.21.15.31:0/1580920904 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f1c940af120 con 0x7f1c9406aaa0 2023-10-06T02:32:03.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.095+0000 7f1c90ff9700 1 -- 172.21.15.31:0/1580920904 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v23) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f1c88054cf0 con 0x7f1c9406aaa0 2023-10-06T02:32:03.096 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:32:03.096 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T02:32:03.096 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_dump": [ 2023-10-06T02:32:03.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:32:03.096 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.a", 2023-10-06T02:32:03.096 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBLbx9lbjrOGxAAVoSOoDaZb1temL41F4Nong==", 2023-10-06T02:32:03.096 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:32:03.096 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:32:03.097 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T02:32:03.097 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T02:32:03.097 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:32:03.097 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:32:03.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:32:03.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:32:03.097 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.b", 2023-10-06T02:32:03.097 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBLbx9lFgsTMRAATTklzroiyQbTLiZc9j1BHQ==", 2023-10-06T02:32:03.097 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:32:03.097 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:32:03.097 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T02:32:03.098 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T02:32:03.098 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:32:03.098 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:32:03.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:32:03.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:32:03.098 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.c", 2023-10-06T02:32:03.098 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBLbx9lezVtJhAA8wK7+dJ1OmIc+iXJkexK0w==", 2023-10-06T02:32:03.098 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:32:03.098 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:32:03.098 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T02:32:03.098 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T02:32:03.099 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:32:03.099 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:32:03.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:32:03.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:32:03.099 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.d", 2023-10-06T02:32:03.099 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBMbx9l74gWABAApBu2nVv12rGA+OBfBnEc3A==", 2023-10-06T02:32:03.099 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:32:03.099 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:32:03.099 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T02:32:03.100 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T02:32:03.100 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:32:03.100 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:32:03.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:32:03.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:32:03.100 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.0", 2023-10-06T02:32:03.100 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBPbx9lgj6+MBAAXSUD0GCG+rO/9DQBZYt/8w==", 2023-10-06T02:32:03.100 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:32:03.100 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:32:03.100 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:32:03.101 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:32:03.101 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:32:03.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:32:03.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:32:03.101 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.1", 2023-10-06T02:32:03.101 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBSbx9lXhMTNBAAeEdPnMP71ZGb6k2Ddg5raQ==", 2023-10-06T02:32:03.101 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:32:03.101 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:32:03.101 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:32:03.101 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:32:03.101 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:32:03.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:32:03.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:32:03.102 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.2", 2023-10-06T02:32:03.102 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBVbx9l4Ly0NhAATYXYDAjYuP53zjqK0yMboA==", 2023-10-06T02:32:03.102 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:32:03.102 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:32:03.102 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:32:03.102 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:32:03.102 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:32:03.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:32:03.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:32:03.103 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.3", 2023-10-06T02:32:03.103 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBZbx9lzsi4ARAAbGnNV0QyL/7itzy827+chg==", 2023-10-06T02:32:03.103 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:32:03.103 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:32:03.103 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:32:03.103 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:32:03.103 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:32:03.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:32:03.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:32:03.103 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.4", 2023-10-06T02:32:03.104 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBfbx9lW4OSEhAAj7HnVh/yq12zzCaXd6BHCA==", 2023-10-06T02:32:03.104 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:32:03.104 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:32:03.104 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:32:03.104 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:32:03.104 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:32:03.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:32:03.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:32:03.104 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.5", 2023-10-06T02:32:03.105 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBibx9lBcaIFRAAV8wq8QjsCOkI9Pzf/E+9bA==", 2023-10-06T02:32:03.105 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:32:03.105 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:32:03.105 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:32:03.105 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:32:03.105 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:32:03.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:32:03.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:32:03.105 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.6", 2023-10-06T02:32:03.105 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBlbx9lA7S+FxAA38mlSAurT+DpwMzH53zFcw==", 2023-10-06T02:32:03.105 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:32:03.106 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:32:03.106 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:32:03.106 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:32:03.106 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:32:03.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:32:03.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:32:03.106 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.7", 2023-10-06T02:32:03.106 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBobx9l5RkdHhAAxn0Z8IAE/o9oe2LRp3O9gg==", 2023-10-06T02:32:03.106 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:32:03.107 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T02:32:03.107 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T02:32:03.107 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:32:03.107 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:32:03.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:32:03.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:32:03.107 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.0", 2023-10-06T02:32:03.107 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBMbx9lh44fCRAA7sMISvk1t7dEJg4ZcBiMcA==", 2023-10-06T02:32:03.107 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:32:03.107 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:32:03.108 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow r", 2023-10-06T02:32:03.108 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T02:32:03.108 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:32:03.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:32:03.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:32:03.108 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.1", 2023-10-06T02:32:03.108 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBMbx9lP0zqFhAAlov2y7ipSvS3AkOdQkhg6g==", 2023-10-06T02:32:03.108 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:32:03.108 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:32:03.109 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-10-06T02:32:03.109 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-10-06T02:32:03.109 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-10-06T02:32:03.109 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:32:03.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:32:03.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:32:03.109 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.2", 2023-10-06T02:32:03.109 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBMbx9lpxYAEBAACtiI1XgQXM8yGGcpFyNK1w==", 2023-10-06T02:32:03.109 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:32:03.109 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:32:03.110 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-10-06T02:32:03.110 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-10-06T02:32:03.110 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-10-06T02:32:03.110 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:32:03.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:32:03.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:32:03.110 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.3", 2023-10-06T02:32:03.111 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBMbx9l81/PHRAAYFOoaezu0pwE0NdtnlYfVA==", 2023-10-06T02:32:03.111 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:32:03.111 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T02:32:03.111 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-10-06T02:32:03.111 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-10-06T02:32:03.111 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-10-06T02:32:03.111 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:32:03.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:32:03.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:32:03.111 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.admin", 2023-10-06T02:32:03.111 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBKbx9liFl1JBAAFyyHYNLVx1/Ex5a2MKm8rA==", 2023-10-06T02:32:03.112 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:32:03.112 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-10-06T02:32:03.112 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T02:32:03.112 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T02:32:03.112 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:32:03.112 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:32:03.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:32:03.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:32:03.112 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mgr.x", 2023-10-06T02:32:03.112 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBLbx9lTBbIFBAAX2wsioKL4Nx4qz/qwsCsNA==", 2023-10-06T02:32:03.112 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:32:03.112 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-10-06T02:32:03.113 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile mgr", 2023-10-06T02:32:03.113 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:32:03.113 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:32:03.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:32:03.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:32:03.113 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mgr.y", 2023-10-06T02:32:03.113 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBLbx9l4+oLDRAA5YVo3q6FI9e9W/rJYROIsg==", 2023-10-06T02:32:03.113 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T02:32:03.113 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-10-06T02:32:03.113 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile mgr", 2023-10-06T02:32:03.113 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T02:32:03.113 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:32:03.114 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:32:03.114 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-10-06T02:32:03.114 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T02:32:03.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.097+0000 7f1c99f9d700 1 -- 172.21.15.31:0/1580920904 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f1c78067830 msgr2=0x7f1c78069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:03.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.097+0000 7f1c99f9d700 1 --2- 172.21.15.31:0/1580920904 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f1c78067830 0x7f1c78069ce0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f1c94069580 tx=0x7f1c80016000 comp rx=0 tx=0).stop 2023-10-06T02:32:03.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.097+0000 7f1c99f9d700 1 -- 172.21.15.31:0/1580920904 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c9406aaa0 msgr2=0x7f1c940c0ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:03.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.097+0000 7f1c99f9d700 1 --2- 172.21.15.31:0/1580920904 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c9406aaa0 0x7f1c940c0ed0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f1c8800c790 tx=0x7f1c88007eb0 comp rx=0 tx=0).stop 2023-10-06T02:32:03.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.098+0000 7f1c99f9d700 1 -- 172.21.15.31:0/1580920904 shutdown_connections 2023-10-06T02:32:03.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.098+0000 7f1c99f9d700 1 --2- 172.21.15.31:0/1580920904 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f1c78067830 0x7f1c78069ce0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:03.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.098+0000 7f1c99f9d700 1 --2- 172.21.15.31:0/1580920904 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1c9413fef0 0x7f1c940c7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:03.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.098+0000 7f1c99f9d700 1 --2- 172.21.15.31:0/1580920904 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c9406aaa0 0x7f1c940c0ed0 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:03.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.098+0000 7f1c99f9d700 1 --2- 172.21.15.31:0/1580920904 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c9413fb20 0x7f1c940c2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:03.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.098+0000 7f1c99f9d700 1 -- 172.21.15.31:0/1580920904 >> 172.21.15.31:0/1580920904 conn(0x7f1c9406d170 msgr2=0x7f1c9413e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:03.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.098+0000 7f1c99f9d700 1 -- 172.21.15.31:0/1580920904 shutdown_connections 2023-10-06T02:32:03.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.098+0000 7f1c99f9d700 1 -- 172.21.15.31:0/1580920904 wait complete. 2023-10-06T02:32:03.117 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:32:03.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.303+0000 7f6649d9d700 1 Processor -- start 2023-10-06T02:32:03.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.304+0000 7f6649d9d700 1 -- start start 2023-10-06T02:32:03.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.305+0000 7f6649d9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f6644150380 0x7f6644160ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:03.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.305+0000 7f6649d9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f664406a490 con 0x7f664406e0a0 2023-10-06T02:32:03.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.305+0000 7f6649d9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66440c7230 con 0x7f664406f3a0 2023-10-06T02:32:03.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.305+0000 7f6649d9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f664406b930 con 0x7f6644150380 2023-10-06T02:32:03.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.305+0000 7f6648d9b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f664406e0a0 0x7f66440c6b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:43918/0 (socket says 172.21.15.31:43918) 2023-10-06T02:32:03.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.305+0000 7f6648d9b700 1 -- 172.21.15.31:0/3754666585 learned_addr learned my addr 172.21.15.31:0/3754666585 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:32:03.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.305+0000 7f664959c700 1 --2- 172.21.15.31:0/3754666585 >> v2:172.21.15.62:3300/0 conn(0x7f6644150380 0x7f6644160ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:03.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.306+0000 7f66437fe700 1 -- 172.21.15.31:0/3754666585 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1290138261 0 0) 0x7f664406a490 con 0x7f664406e0a0 2023-10-06T02:32:03.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.306+0000 7f66437fe700 1 -- 172.21.15.31:0/3754666585 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6634003570 con 0x7f664406e0a0 2023-10-06T02:32:03.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.306+0000 7f66437fe700 1 -- 172.21.15.31:0/3754666585 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1723589129 0 0) 0x7f66440c7230 con 0x7f664406f3a0 2023-10-06T02:32:03.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.306+0000 7f66437fe700 1 -- 172.21.15.31:0/3754666585 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f664406a490 con 0x7f664406f3a0 2023-10-06T02:32:03.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.306+0000 7f66437fe700 1 -- 172.21.15.31:0/3754666585 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3804584074 0 0) 0x7f6634003570 con 0x7f664406e0a0 2023-10-06T02:32:03.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.306+0000 7f66437fe700 1 -- 172.21.15.31:0/3754666585 >> v2:172.21.15.62:3300/0 conn(0x7f6644150380 msgr2=0x7f6644160ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:03.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.306+0000 7f66437fe700 1 --2- 172.21.15.31:0/3754666585 >> v2:172.21.15.62:3300/0 conn(0x7f6644150380 0x7f6644160ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:03.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.306+0000 7f66437fe700 1 -- 172.21.15.31:0/3754666585 >> v1:172.21.15.62:6789/0 conn(0x7f664406f3a0 legacy=0x7f664414fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:03.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.306+0000 7f66437fe700 1 -- 172.21.15.31:0/3754666585 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66440c7490 con 0x7f664406e0a0 2023-10-06T02:32:03.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.306+0000 7f66437fe700 1 -- 172.21.15.31:0/3754666585 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7f6638001470 con 0x7f664406e0a0 2023-10-06T02:32:03.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.306+0000 7f66437fe700 1 -- 172.21.15.31:0/3754666585 >> v1:172.21.15.31:6789/0 conn(0x7f664406e0a0 legacy=0x7f66440c6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:03.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.306+0000 7f66437fe700 1 --2- 172.21.15.31:0/3754666585 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6634003bf0 0x7f6634003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:03.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.306+0000 7f66437fe700 1 --2- 172.21.15.31:0/3754666585 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6634004810 0x7f6634004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:03.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.306+0000 7f66437fe700 1 --2- 172.21.15.31:0/3754666585 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66340051c0 0x7f663400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:03.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.306+0000 7f66437fe700 1 -- 172.21.15.31:0/3754666585 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f664406b930 con 0x7f6634003bf0 2023-10-06T02:32:03.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.306+0000 7f66437fe700 1 -- 172.21.15.31:0/3754666585 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f663400c0b0 con 0x7f66340051c0 2023-10-06T02:32:03.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.306+0000 7f66437fe700 1 -- 172.21.15.31:0/3754666585 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f663400c270 con 0x7f6634004810 2023-10-06T02:32:03.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.306+0000 7f6648d9b700 1 --2- 172.21.15.31:0/3754666585 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6634003bf0 0x7f6634003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:03.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.307+0000 7f664959c700 1 --2- 172.21.15.31:0/3754666585 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66340051c0 0x7f663400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:03.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.307+0000 7f6643fff700 1 --2- 172.21.15.31:0/3754666585 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6634004810 0x7f6634004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:03.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.307+0000 7f6648d9b700 1 -- 172.21.15.31:0/3754666585 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6634004810 msgr2=0x7f6634004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:03.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.307+0000 7f6648d9b700 1 --2- 172.21.15.31:0/3754666585 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6634004810 0x7f6634004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:03.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.307+0000 7f6648d9b700 1 -- 172.21.15.31:0/3754666585 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66340051c0 msgr2=0x7f663400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:03.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.307+0000 7f6648d9b700 1 --2- 172.21.15.31:0/3754666585 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66340051c0 0x7f663400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:03.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.307+0000 7f6648d9b700 1 -- 172.21.15.31:0/3754666585 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6634003570 con 0x7f6634003bf0 2023-10-06T02:32:03.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.307+0000 7f664959c700 1 -- 172.21.15.31:0/3754666585 reap_dead start 2023-10-06T02:32:03.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.307+0000 7f6643fff700 1 --2- 172.21.15.31:0/3754666585 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6634004810 0x7f6634004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:32:03.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.307+0000 7f6648d9b700 1 --2- 172.21.15.31:0/3754666585 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6634003bf0 0x7f6634003fe0 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f663800c2d0 tx=0x7f6638001e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e042e5350eb85ac server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:03.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.307+0000 7f66437fe700 1 -- 172.21.15.31:0/3754666585 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f663800cba0 con 0x7f6634003bf0 2023-10-06T02:32:03.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.307+0000 7f66437fe700 1 -- 172.21.15.31:0/3754666585 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f663800cd00 con 0x7f6634003bf0 2023-10-06T02:32:03.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.307+0000 7f66437fe700 1 -- 172.21.15.31:0/3754666585 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6638005b70 con 0x7f6634003bf0 2023-10-06T02:32:03.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.308+0000 7f6649d9d700 1 -- 172.21.15.31:0/3754666585 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6634003bf0 msgr2=0x7f6634003fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:03.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.308+0000 7f6649d9d700 1 --2- 172.21.15.31:0/3754666585 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6634003bf0 0x7f6634003fe0 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f663800c2d0 tx=0x7f6638001e10 comp rx=0 tx=0).stop 2023-10-06T02:32:03.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.308+0000 7f6649d9d700 1 -- 172.21.15.31:0/3754666585 shutdown_connections 2023-10-06T02:32:03.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.308+0000 7f6649d9d700 1 --2- 172.21.15.31:0/3754666585 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6634003bf0 0x7f6634003fe0 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:03.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.308+0000 7f6649d9d700 1 -- 172.21.15.31:0/3754666585 >> 172.21.15.31:0/3754666585 conn(0x7f664406d170 msgr2=0x7f66440cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:03.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.308+0000 7f6649d9d700 1 -- 172.21.15.31:0/3754666585 shutdown_connections 2023-10-06T02:32:03.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.308+0000 7f6649d9d700 1 -- 172.21.15.31:0/3754666585 wait complete. 2023-10-06T02:32:03.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.309+0000 7f6649d9d700 1 Processor -- start 2023-10-06T02:32:03.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.310+0000 7f6649d9d700 1 -- start start 2023-10-06T02:32:03.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.310+0000 7f6649d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6634003bf0 0x7f6644138770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:03.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.311+0000 7f6649d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66440b15d0 0x7f66440b1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:03.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.311+0000 7f6649d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6644154c00 0x7f6644155070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:03.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.311+0000 7f6649d9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66440b0b80 con 0x7f66440b15d0 2023-10-06T02:32:03.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.311+0000 7f6649d9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66440afce0 con 0x7f6644154c00 2023-10-06T02:32:03.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.311+0000 7f6649d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66440affc0 con 0x7f6634003bf0 2023-10-06T02:32:03.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.311+0000 7f6643fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66440b15d0 0x7f66440b1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:03.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.311+0000 7f6643fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66440b15d0 0x7f66440b1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37468/0 (socket says 172.21.15.31:37468) 2023-10-06T02:32:03.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.311+0000 7f6643fff700 1 -- 172.21.15.31:0/2443400219 learned_addr learned my addr 172.21.15.31:0/2443400219 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:03.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.311+0000 7f6643fff700 1 -- 172.21.15.31:0/2443400219 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6634003bf0 msgr2=0x7f6644138770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:03.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.311+0000 7f6643fff700 1 --2- 172.21.15.31:0/2443400219 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6634003bf0 0x7f6644138770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:03.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.311+0000 7f6643fff700 1 -- 172.21.15.31:0/2443400219 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6644154c00 msgr2=0x7f6644155070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:03.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.311+0000 7f6643fff700 1 --2- 172.21.15.31:0/2443400219 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6644154c00 0x7f6644155070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:03.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.311+0000 7f6643fff700 1 -- 172.21.15.31:0/2443400219 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66440b0dd0 con 0x7f66440b15d0 2023-10-06T02:32:03.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.311+0000 7f6643fff700 1 --2- 172.21.15.31:0/2443400219 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66440b15d0 0x7f66440b1a40 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f6630000c00 tx=0x7f6630009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:03.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.312+0000 7f6640ff9700 1 -- 172.21.15.31:0/2443400219 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6630012660 con 0x7f66440b15d0 2023-10-06T02:32:03.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.312+0000 7f6640ff9700 1 -- 172.21.15.31:0/2443400219 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66300127e0 con 0x7f66440b15d0 2023-10-06T02:32:03.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.312+0000 7f6640ff9700 1 -- 172.21.15.31:0/2443400219 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6630012ba0 con 0x7f66440b15d0 2023-10-06T02:32:03.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.312+0000 7f6649d9d700 1 -- 172.21.15.31:0/2443400219 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66440b10b0 con 0x7f66440b15d0 2023-10-06T02:32:03.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.312+0000 7f6649d9d700 1 -- 172.21.15.31:0/2443400219 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66440c5330 con 0x7f66440b15d0 2023-10-06T02:32:03.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.313+0000 7f6649d9d700 1 -- 172.21.15.31:0/2443400219 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66440b12a0 con 0x7f66440b15d0 2023-10-06T02:32:03.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.314+0000 7f6640ff9700 1 -- 172.21.15.31:0/2443400219 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85556+0+0 (secure 0 0 0) 0x7f6630013420 con 0x7f66440b15d0 2023-10-06T02:32:03.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.315+0000 7f6640ff9700 1 --2- 172.21.15.31:0/2443400219 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6628067830 0x7f6628069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:03.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.315+0000 7f6640ff9700 1 -- 172.21.15.31:0/2443400219 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5234+0+0 (secure 0 0 0) 0x7f6630083310 con 0x7f66440b15d0 2023-10-06T02:32:03.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.315+0000 7f6648d9b700 1 --2- 172.21.15.31:0/2443400219 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6628067830 0x7f6628069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:03.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.316+0000 7f6648d9b700 1 --2- 172.21.15.31:0/2443400219 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6628067830 0x7f6628069ce0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f663800c2d0 tx=0x7f6638015000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:03.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.319+0000 7f6640ff9700 1 -- 172.21.15.31:0/2443400219 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6630051dc0 con 0x7f66440b15d0 2023-10-06T02:32:03.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.525+0000 7f6649d9d700 1 -- 172.21.15.31:0/2443400219 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f664404c3c0 con 0x7f66440b15d0 2023-10-06T02:32:03.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.526+0000 7f6640ff9700 1 -- 172.21.15.31:0/2443400219 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v165) v1 ==== 74+0+13656 (secure 0 0 0) 0x7f66300553e0 con 0x7f66440b15d0 2023-10-06T02:32:03.527 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:32:03.527 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":165,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:32:02.593757+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":20,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:32:03.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.528+0000 7f6649d9d700 1 -- 172.21.15.31:0/2443400219 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6628067830 msgr2=0x7f6628069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:03.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.528+0000 7f6649d9d700 1 --2- 172.21.15.31:0/2443400219 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6628067830 0x7f6628069ce0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f663800c2d0 tx=0x7f6638015000 comp rx=0 tx=0).stop 2023-10-06T02:32:03.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.528+0000 7f6649d9d700 1 -- 172.21.15.31:0/2443400219 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66440b15d0 msgr2=0x7f66440b1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:03.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.528+0000 7f6649d9d700 1 --2- 172.21.15.31:0/2443400219 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66440b15d0 0x7f66440b1a40 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f6630000c00 tx=0x7f6630009040 comp rx=0 tx=0).stop 2023-10-06T02:32:03.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.529+0000 7f6649d9d700 1 -- 172.21.15.31:0/2443400219 shutdown_connections 2023-10-06T02:32:03.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.529+0000 7f6649d9d700 1 --2- 172.21.15.31:0/2443400219 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6644154c00 0x7f6644155070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:03.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.529+0000 7f6649d9d700 1 --2- 172.21.15.31:0/2443400219 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6628067830 0x7f6628069ce0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:03.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.529+0000 7f6649d9d700 1 --2- 172.21.15.31:0/2443400219 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6634003bf0 0x7f6644138770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:03.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.529+0000 7f6649d9d700 1 --2- 172.21.15.31:0/2443400219 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66440b15d0 0x7f66440b1a40 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:03.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.529+0000 7f6649d9d700 1 -- 172.21.15.31:0/2443400219 >> 172.21.15.31:0/2443400219 conn(0x7f664406d170 msgr2=0x7f66440c2210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:03.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.529+0000 7f6649d9d700 1 -- 172.21.15.31:0/2443400219 shutdown_connections 2023-10-06T02:32:03.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.529+0000 7f6649d9d700 1 -- 172.21.15.31:0/2443400219 wait complete. 2023-10-06T02:32:03.547 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T02:22:53.183425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T02:23:00.046967+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-10-06T02:32:03.547 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T02:32:03.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.738+0000 7fb73e19d700 1 Processor -- start 2023-10-06T02:32:03.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.738+0000 7fb73e19d700 1 -- start start 2023-10-06T02:32:03.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.739+0000 7fb73e19d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb73806de50 0x7fb7380c7360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:03.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.739+0000 7fb73e19d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb73815b640 con 0x7fb738140a70 2023-10-06T02:32:03.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.739+0000 7fb73e19d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb73815b920 con 0x7fb73806a210 2023-10-06T02:32:03.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.739+0000 7fb73e19d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7380c6800 con 0x7fb73806de50 2023-10-06T02:32:03.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.740+0000 7fb73d19b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fb738140a70 0x7fb738069b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:43924/0 (socket says 172.21.15.31:43924) 2023-10-06T02:32:03.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.740+0000 7fb73d19b700 1 -- 172.21.15.31:0/986310317 learned_addr learned my addr 172.21.15.31:0/986310317 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:32:03.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.740+0000 7fb73d99c700 1 --2- 172.21.15.31:0/986310317 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb73806de50 0x7fb7380c7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:03.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.740+0000 7fb72ffff700 1 -- 172.21.15.31:0/986310317 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2998427307 0 0) 0x7fb73815b640 con 0x7fb738140a70 2023-10-06T02:32:03.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.740+0000 7fb72ffff700 1 -- 172.21.15.31:0/986310317 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb724003570 con 0x7fb738140a70 2023-10-06T02:32:03.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.740+0000 7fb72ffff700 1 -- 172.21.15.31:0/986310317 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1370291648 0 0) 0x7fb73815b920 con 0x7fb73806a210 2023-10-06T02:32:03.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.740+0000 7fb72ffff700 1 -- 172.21.15.31:0/986310317 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb73815b640 con 0x7fb73806a210 2023-10-06T02:32:03.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.740+0000 7fb73d99c700 1 -- 172.21.15.31:0/986310317 >> v1:172.21.15.62:6789/0 conn(0x7fb73806a210 legacy=0x7fb7380c6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:03.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.740+0000 7fb73d99c700 1 -- 172.21.15.31:0/986310317 >> v1:172.21.15.31:6789/0 conn(0x7fb738140a70 legacy=0x7fb738069b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:03.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.740+0000 7fb73d99c700 1 -- 172.21.15.31:0/986310317 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb73815bc00 con 0x7fb73806de50 2023-10-06T02:32:03.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.741+0000 7fb73d99c700 1 --2- 172.21.15.31:0/986310317 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb73806de50 0x7fb7380c7360 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb730009e90 tx=0x7fb730009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e33165cf736457a9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:03.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.741+0000 7fb72ffff700 1 -- 172.21.15.31:0/986310317 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb73000e070 con 0x7fb73806de50 2023-10-06T02:32:03.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.741+0000 7fb72ffff700 1 -- 172.21.15.31:0/986310317 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb73000bb80 con 0x7fb73806de50 2023-10-06T02:32:03.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.741+0000 7fb72ffff700 1 -- 172.21.15.31:0/986310317 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb730012430 con 0x7fb73806de50 2023-10-06T02:32:03.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.742+0000 7fb73e19d700 1 -- 172.21.15.31:0/986310317 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb73806de50 msgr2=0x7fb7380c7360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:03.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.742+0000 7fb73e19d700 1 --2- 172.21.15.31:0/986310317 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb73806de50 0x7fb7380c7360 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb730009e90 tx=0x7fb730009780 comp rx=0 tx=0).stop 2023-10-06T02:32:03.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.742+0000 7fb73e19d700 1 -- 172.21.15.31:0/986310317 shutdown_connections 2023-10-06T02:32:03.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.742+0000 7fb73e19d700 1 --2- 172.21.15.31:0/986310317 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb73806de50 0x7fb7380c7360 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:03.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.742+0000 7fb73e19d700 1 -- 172.21.15.31:0/986310317 >> 172.21.15.31:0/986310317 conn(0x7fb73806d110 msgr2=0x7fb7380cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:03.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.742+0000 7fb73e19d700 1 -- 172.21.15.31:0/986310317 shutdown_connections 2023-10-06T02:32:03.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.742+0000 7fb73e19d700 1 -- 172.21.15.31:0/986310317 wait complete. 2023-10-06T02:32:03.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.743+0000 7fb73e19d700 1 Processor -- start 2023-10-06T02:32:03.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.743+0000 7fb73e19d700 1 -- start start 2023-10-06T02:32:03.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.744+0000 7fb73e19d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb73806a210 0x7fb7380b1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:03.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.744+0000 7fb73e19d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb73806de50 0x7fb73815a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:03.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.744+0000 7fb73e19d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb738140a70 0x7fb7380affa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:03.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.744+0000 7fb73e19d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7380cbdb0 con 0x7fb73806a210 2023-10-06T02:32:03.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.744+0000 7fb73e19d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7380cb510 con 0x7fb738140a70 2023-10-06T02:32:03.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.744+0000 7fb73e19d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7380adfc0 con 0x7fb73806de50 2023-10-06T02:32:03.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.744+0000 7fb73d19b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb73806a210 0x7fb7380b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:03.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.745+0000 7fb73d19b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb73806a210 0x7fb7380b1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37470/0 (socket says 172.21.15.31:37470) 2023-10-06T02:32:03.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.745+0000 7fb73d19b700 1 -- 172.21.15.31:0/1276151595 learned_addr learned my addr 172.21.15.31:0/1276151595 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:03.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.745+0000 7fb73c99a700 1 --2- 172.21.15.31:0/1276151595 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb73806de50 0x7fb73815a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:03.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.745+0000 7fb73d99c700 1 --2- 172.21.15.31:0/1276151595 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb738140a70 0x7fb7380affa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:03.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.745+0000 7fb73d19b700 1 -- 172.21.15.31:0/1276151595 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb73806de50 msgr2=0x7fb73815a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:03.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.745+0000 7fb73d19b700 1 --2- 172.21.15.31:0/1276151595 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb73806de50 0x7fb73815a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:03.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.745+0000 7fb73d19b700 1 -- 172.21.15.31:0/1276151595 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb738140a70 msgr2=0x7fb7380affa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:03.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.745+0000 7fb73d19b700 1 --2- 172.21.15.31:0/1276151595 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb738140a70 0x7fb7380affa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:03.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.745+0000 7fb73d19b700 1 -- 172.21.15.31:0/1276151595 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb730009a20 con 0x7fb73806a210 2023-10-06T02:32:03.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.745+0000 7fb73c99a700 1 --2- 172.21.15.31:0/1276151595 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb73806de50 0x7fb73815a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:32:03.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.745+0000 7fb73d19b700 1 --2- 172.21.15.31:0/1276151595 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb73806a210 0x7fb7380b1820 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7fb73400c7c0 tx=0x7fb734007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:03.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.745+0000 7fb72d7fa700 1 -- 172.21.15.31:0/1276151595 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb734005b30 con 0x7fb73806a210 2023-10-06T02:32:03.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.745+0000 7fb72d7fa700 1 -- 172.21.15.31:0/1276151595 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb734005c90 con 0x7fb73806a210 2023-10-06T02:32:03.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.745+0000 7fb73e19d700 1 -- 172.21.15.31:0/1276151595 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb73815ad30 con 0x7fb73806a210 2023-10-06T02:32:03.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.745+0000 7fb72d7fa700 1 -- 172.21.15.31:0/1276151595 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb734007300 con 0x7fb73806a210 2023-10-06T02:32:03.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.746+0000 7fb73e19d700 1 -- 172.21.15.31:0/1276151595 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7380cb760 con 0x7fb73806a210 2023-10-06T02:32:03.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.747+0000 7fb73e19d700 1 -- 172.21.15.31:0/1276151595 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb73804c3c0 con 0x7fb73806a210 2023-10-06T02:32:03.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.748+0000 7fb72d7fa700 1 -- 172.21.15.31:0/1276151595 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85556+0+0 (secure 0 0 0) 0x7fb734007670 con 0x7fb73806a210 2023-10-06T02:32:03.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.748+0000 7fb72d7fa700 1 --2- 172.21.15.31:0/1276151595 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb71c067830 0x7fb71c069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:03.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.748+0000 7fb72d7fa700 1 -- 172.21.15.31:0/1276151595 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5234+0+0 (secure 0 0 0) 0x7fb7340823c0 con 0x7fb73806a210 2023-10-06T02:32:03.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.749+0000 7fb73c99a700 1 --2- 172.21.15.31:0/1276151595 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb71c067830 0x7fb71c069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:03.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.749+0000 7fb73c99a700 1 --2- 172.21.15.31:0/1276151595 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb71c067830 0x7fb71c069ce0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb7380ae630 tx=0x7fb728009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:03.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.752+0000 7fb72d7fa700 1 -- 172.21.15.31:0/1276151595 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb734051000 con 0x7fb73806a210 2023-10-06T02:32:03.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.952+0000 7fb73e19d700 1 -- 172.21.15.31:0/1276151595 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb73815bee0 con 0x7fb73806a210 2023-10-06T02:32:03.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.953+0000 7fb72d7fa700 1 -- 172.21.15.31:0/1276151595 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v165) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb7340170a0 con 0x7fb73806a210 2023-10-06T02:32:03.953 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-10-06T02:32:03.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.955+0000 7fb73e19d700 1 -- 172.21.15.31:0/1276151595 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb71c067830 msgr2=0x7fb71c069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:03.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.955+0000 7fb73e19d700 1 --2- 172.21.15.31:0/1276151595 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb71c067830 0x7fb71c069ce0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb7380ae630 tx=0x7fb728009040 comp rx=0 tx=0).stop 2023-10-06T02:32:03.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.955+0000 7fb73e19d700 1 -- 172.21.15.31:0/1276151595 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb73806a210 msgr2=0x7fb7380b1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:03.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.955+0000 7fb73e19d700 1 --2- 172.21.15.31:0/1276151595 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb73806a210 0x7fb7380b1820 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7fb73400c7c0 tx=0x7fb734007ea0 comp rx=0 tx=0).stop 2023-10-06T02:32:03.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.955+0000 7fb73e19d700 1 -- 172.21.15.31:0/1276151595 shutdown_connections 2023-10-06T02:32:03.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.955+0000 7fb73e19d700 1 --2- 172.21.15.31:0/1276151595 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb738140a70 0x7fb7380affa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:03.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.955+0000 7fb73e19d700 1 --2- 172.21.15.31:0/1276151595 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb73806a210 0x7fb7380b1820 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:03.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.955+0000 7fb73e19d700 1 --2- 172.21.15.31:0/1276151595 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb71c067830 0x7fb71c069ce0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:03.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.955+0000 7fb73e19d700 1 --2- 172.21.15.31:0/1276151595 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb73806de50 0x7fb73815a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:03.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.955+0000 7fb73e19d700 1 -- 172.21.15.31:0/1276151595 >> 172.21.15.31:0/1276151595 conn(0x7fb73806d110 msgr2=0x7fb73813e3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:03.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.955+0000 7fb73e19d700 1 -- 172.21.15.31:0/1276151595 shutdown_connections 2023-10-06T02:32:03.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:03.955+0000 7fb73e19d700 1 -- 172.21.15.31:0/1276151595 wait complete. 2023-10-06T02:32:03.973 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T02:32:04.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.162+0000 7f61b4b9d700 1 Processor -- start 2023-10-06T02:32:04.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.162+0000 7f61b4b9d700 1 -- start start 2023-10-06T02:32:04.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.164+0000 7f61b4b9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f61b0159540 0x7f61b00c1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:04.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.164+0000 7f61b4b9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f61b0159910 0x7f61b00677b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:04.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.164+0000 7f61b4b9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61b0068250 con 0x7f61b0067cf0 2023-10-06T02:32:04.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.164+0000 7f61b4b9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61b0068530 con 0x7f61b0159910 2023-10-06T02:32:04.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.164+0000 7f61b4b9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61b0068810 con 0x7f61b0159540 2023-10-06T02:32:04.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.164+0000 7f61af7fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f61b0159540 0x7f61b00c1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:04.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.164+0000 7f61affff700 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f61b0067cf0 0x7f61b00c6be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:46286/0 (socket says 172.21.15.31:46286) 2023-10-06T02:32:04.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.164+0000 7f61affff700 1 -- 172.21.15.31:0/407412402 learned_addr learned my addr 172.21.15.31:0/407412402 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:32:04.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.164+0000 7f61aeffd700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f61b0159910 0x7f61b00677b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:04.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.165+0000 7f61ae7fc700 1 -- 172.21.15.31:0/407412402 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2167227678 0 0) 0x7f61b0068250 con 0x7f61b0067cf0 2023-10-06T02:32:04.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.165+0000 7f61ae7fc700 1 -- 172.21.15.31:0/407412402 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6198003570 con 0x7f61b0067cf0 2023-10-06T02:32:04.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.165+0000 7f61af7fe700 1 -- 172.21.15.31:0/407412402 >> v2:172.21.15.62:3300/0 conn(0x7f61b0159910 msgr2=0x7f61b00677b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:04.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.165+0000 7f61af7fe700 1 --2- 172.21.15.31:0/407412402 >> v2:172.21.15.62:3300/0 conn(0x7f61b0159910 0x7f61b00677b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:04.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.165+0000 7f61af7fe700 1 -- 172.21.15.31:0/407412402 >> v1:172.21.15.62:6789/0 conn(0x7f61b0067cf0 legacy=0x7f61b00c6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:04.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.165+0000 7f61af7fe700 1 -- 172.21.15.31:0/407412402 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61b00a9ca0 con 0x7f61b0159540 2023-10-06T02:32:04.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.165+0000 7f61af7fe700 1 --2- 172.21.15.31:0/407412402 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f61b0159540 0x7f61b00c1340 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f61a00098e0 tx=0x7f61a00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f699cb53fb9a5d8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:04.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.165+0000 7f61ae7fc700 1 -- 172.21.15.31:0/407412402 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a0011630 con 0x7f61b0159540 2023-10-06T02:32:04.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.165+0000 7f61ae7fc700 1 -- 172.21.15.31:0/407412402 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61a0011790 con 0x7f61b0159540 2023-10-06T02:32:04.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.165+0000 7f61ae7fc700 1 -- 172.21.15.31:0/407412402 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a0011b30 con 0x7f61b0159540 2023-10-06T02:32:04.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.166+0000 7f61b4b9d700 1 -- 172.21.15.31:0/407412402 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f61b0159540 msgr2=0x7f61b00c1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:04.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.166+0000 7f61b4b9d700 1 --2- 172.21.15.31:0/407412402 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f61b0159540 0x7f61b00c1340 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f61a00098e0 tx=0x7f61a00091a0 comp rx=0 tx=0).stop 2023-10-06T02:32:04.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.166+0000 7f61b4b9d700 1 -- 172.21.15.31:0/407412402 shutdown_connections 2023-10-06T02:32:04.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.166+0000 7f61b4b9d700 1 --2- 172.21.15.31:0/407412402 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f61b0159540 0x7f61b00c1340 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:04.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.166+0000 7f61b4b9d700 1 --2- 172.21.15.31:0/407412402 >> v2:172.21.15.62:3300/0 conn(0x7f61b0159910 0x7f61b00677b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:04.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.166+0000 7f61b4b9d700 1 -- 172.21.15.31:0/407412402 >> 172.21.15.31:0/407412402 conn(0x7f61b006f330 msgr2=0x7f61b00cc5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:04.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.166+0000 7f61b4b9d700 1 -- 172.21.15.31:0/407412402 shutdown_connections 2023-10-06T02:32:04.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.166+0000 7f61b4b9d700 1 -- 172.21.15.31:0/407412402 wait complete. 2023-10-06T02:32:04.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.167+0000 7f61b4b9d700 1 Processor -- start 2023-10-06T02:32:04.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.168+0000 7f61b4b9d700 1 -- start start 2023-10-06T02:32:04.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.168+0000 7f61b4b9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f61b0067cf0 0x7f61b00ad8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:04.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.168+0000 7f61b4b9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f61b0159540 0x7f61b00ade00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:04.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.168+0000 7f61b4b9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61b0159910 0x7f61b00ae340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:04.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.168+0000 7f61b4b9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61b00b0860 con 0x7f61b0159910 2023-10-06T02:32:04.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.168+0000 7f61b4b9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61b00b02c0 con 0x7f61b0067cf0 2023-10-06T02:32:04.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.168+0000 7f61b4b9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61b0158b40 con 0x7f61b0159540 2023-10-06T02:32:04.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.169+0000 7f61affff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61b0159910 0x7f61b00ae340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:04.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.169+0000 7f61affff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61b0159910 0x7f61b00ae340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37472/0 (socket says 172.21.15.31:37472) 2023-10-06T02:32:04.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.169+0000 7f61affff700 1 -- 172.21.15.31:0/1601658737 learned_addr learned my addr 172.21.15.31:0/1601658737 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:04.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.169+0000 7f61af7fe700 1 --2- 172.21.15.31:0/1601658737 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f61b0067cf0 0x7f61b00ad8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:04.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.169+0000 7f61aeffd700 1 --2- 172.21.15.31:0/1601658737 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f61b0159540 0x7f61b00ade00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:04.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.169+0000 7f61affff700 1 -- 172.21.15.31:0/1601658737 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f61b0159540 msgr2=0x7f61b00ade00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:04.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.169+0000 7f61affff700 1 --2- 172.21.15.31:0/1601658737 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f61b0159540 0x7f61b00ade00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:04.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.169+0000 7f61affff700 1 -- 172.21.15.31:0/1601658737 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f61b0067cf0 msgr2=0x7f61b00ad8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:04.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.169+0000 7f61affff700 1 --2- 172.21.15.31:0/1601658737 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f61b0067cf0 0x7f61b00ad8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:04.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.169+0000 7f61affff700 1 -- 172.21.15.31:0/1601658737 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61a0009420 con 0x7f61b0159910 2023-10-06T02:32:04.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.169+0000 7f61affff700 1 --2- 172.21.15.31:0/1601658737 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61b0159910 0x7f61b00ae340 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f61a8001460 tx=0x7f61a8001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:04.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.170+0000 7f61977fe700 1 -- 172.21.15.31:0/1601658737 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a800a940 con 0x7f61b0159910 2023-10-06T02:32:04.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.170+0000 7f61977fe700 1 -- 172.21.15.31:0/1601658737 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61a800aaa0 con 0x7f61b0159910 2023-10-06T02:32:04.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.170+0000 7f61977fe700 1 -- 172.21.15.31:0/1601658737 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a800ae20 con 0x7f61b0159910 2023-10-06T02:32:04.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.170+0000 7f61b4b9d700 1 -- 172.21.15.31:0/1601658737 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61b00b0580 con 0x7f61b0159910 2023-10-06T02:32:04.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.170+0000 7f61b4b9d700 1 -- 172.21.15.31:0/1601658737 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61b00b1910 con 0x7f61b0159910 2023-10-06T02:32:04.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.171+0000 7f61b4b9d700 1 -- 172.21.15.31:0/1601658737 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61b00b1460 con 0x7f61b0159910 2023-10-06T02:32:04.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.172+0000 7f61977fe700 1 -- 172.21.15.31:0/1601658737 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85556+0+0 (secure 0 0 0) 0x7f61a8007300 con 0x7f61b0159910 2023-10-06T02:32:04.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.172+0000 7f61977fe700 1 --2- 172.21.15.31:0/1601658737 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f619c067830 0x7f619c069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:04.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.173+0000 7f61977fe700 1 -- 172.21.15.31:0/1601658737 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5234+0+0 (secure 0 0 0) 0x7f61a8083270 con 0x7f61b0159910 2023-10-06T02:32:04.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.173+0000 7f61af7fe700 1 --2- 172.21.15.31:0/1601658737 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f619c067830 0x7f619c069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:04.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.174+0000 7f61af7fe700 1 --2- 172.21.15.31:0/1601658737 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f619c067830 0x7f619c069ce0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f61a00091e0 tx=0x7f61a0017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:04.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.176+0000 7f61977fe700 1 -- 172.21.15.31:0/1601658737 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f61a8051f90 con 0x7f61b0159910 2023-10-06T02:32:04.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.379+0000 7f61b4b9d700 1 -- 172.21.15.31:0/1601658737 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f61b004c3c0 con 0x7f61b0159910 2023-10-06T02:32:04.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.379+0000 7f61977fe700 1 -- 172.21.15.31:0/1601658737 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v165) v1 ==== 92+0+10 (secure 0 0 0) 0x7f61a80555b0 con 0x7f61b0159910 2023-10-06T02:32:04.380 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 8 2023-10-06T02:32:04.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.382+0000 7f61b4b9d700 1 -- 172.21.15.31:0/1601658737 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f619c067830 msgr2=0x7f619c069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:04.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.382+0000 7f61b4b9d700 1 --2- 172.21.15.31:0/1601658737 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f619c067830 0x7f619c069ce0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f61a00091e0 tx=0x7f61a0017000 comp rx=0 tx=0).stop 2023-10-06T02:32:04.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.382+0000 7f61b4b9d700 1 -- 172.21.15.31:0/1601658737 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61b0159910 msgr2=0x7f61b00ae340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:04.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.382+0000 7f61b4b9d700 1 --2- 172.21.15.31:0/1601658737 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61b0159910 0x7f61b00ae340 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f61a8001460 tx=0x7f61a8001ae0 comp rx=0 tx=0).stop 2023-10-06T02:32:04.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.382+0000 7f61b4b9d700 1 -- 172.21.15.31:0/1601658737 shutdown_connections 2023-10-06T02:32:04.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.382+0000 7f61b4b9d700 1 --2- 172.21.15.31:0/1601658737 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61b0159910 0x7f61b00ae340 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:04.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.382+0000 7f61b4b9d700 1 --2- 172.21.15.31:0/1601658737 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f61b0067cf0 0x7f61b00ad8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:04.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.382+0000 7f61b4b9d700 1 --2- 172.21.15.31:0/1601658737 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f619c067830 0x7f619c069ce0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:04.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.382+0000 7f61b4b9d700 1 --2- 172.21.15.31:0/1601658737 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f61b0159540 0x7f61b00ade00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:04.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.382+0000 7f61b4b9d700 1 -- 172.21.15.31:0/1601658737 >> 172.21.15.31:0/1601658737 conn(0x7f61b006f330 msgr2=0x7f61b014f0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:04.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.382+0000 7f61b4b9d700 1 -- 172.21.15.31:0/1601658737 shutdown_connections 2023-10-06T02:32:04.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.382+0000 7f61b4b9d700 1 -- 172.21.15.31:0/1601658737 wait complete. 2023-10-06T02:32:04.400 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T02:32:04.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.588+0000 7f55f1f9d700 1 Processor -- start 2023-10-06T02:32:04.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.588+0000 7f55f1f9d700 1 -- start start 2023-10-06T02:32:04.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.590+0000 7f55f1f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55ec153c30 0x7f55ec15a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:04.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.590+0000 7f55f1f9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f55ec0cf110 0x7f55ec15c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:04.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.590+0000 7f55f1f9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55ec15afd0 con 0x7f55ec15de50 2023-10-06T02:32:04.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.590+0000 7f55f1f9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55ec15b2b0 con 0x7f55ec0cf110 2023-10-06T02:32:04.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.590+0000 7f55f1f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55ec15b590 con 0x7f55ec153c30 2023-10-06T02:32:04.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.590+0000 7f55ebfff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f55ec15de50 0x7f55ec0cea00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:43932/0 (socket says 172.21.15.31:43932) 2023-10-06T02:32:04.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.590+0000 7f55f0f9b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55ec153c30 0x7f55ec15a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:04.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.590+0000 7f55ebfff700 1 -- 172.21.15.31:0/1505020250 learned_addr learned my addr 172.21.15.31:0/1505020250 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:32:04.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.590+0000 7f55f179c700 1 --2- 172.21.15.31:0/1505020250 >> v2:172.21.15.62:3300/0 conn(0x7f55ec0cf110 0x7f55ec15c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:04.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.590+0000 7f55eb7fe700 1 -- 172.21.15.31:0/1505020250 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2949931398 0 0) 0x7f55ec15afd0 con 0x7f55ec15de50 2023-10-06T02:32:04.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.590+0000 7f55eb7fe700 1 -- 172.21.15.31:0/1505020250 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55dc003570 con 0x7f55ec15de50 2023-10-06T02:32:04.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.590+0000 7f55f0f9b700 1 -- 172.21.15.31:0/1505020250 >> v2:172.21.15.62:3300/0 conn(0x7f55ec0cf110 msgr2=0x7f55ec15c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:04.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.590+0000 7f55f0f9b700 1 --2- 172.21.15.31:0/1505020250 >> v2:172.21.15.62:3300/0 conn(0x7f55ec0cf110 0x7f55ec15c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:04.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.590+0000 7f55f0f9b700 1 -- 172.21.15.31:0/1505020250 >> v1:172.21.15.31:6789/0 conn(0x7f55ec15de50 legacy=0x7f55ec0cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:04.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.590+0000 7f55f0f9b700 1 -- 172.21.15.31:0/1505020250 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55ec15b870 con 0x7f55ec153c30 2023-10-06T02:32:04.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.591+0000 7f55f0f9b700 1 --2- 172.21.15.31:0/1505020250 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55ec153c30 0x7f55ec15a5e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f55e00098e0 tx=0x7f55e00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9edc3f129ff362fd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:04.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.591+0000 7f55eb7fe700 1 -- 172.21.15.31:0/1505020250 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e0011630 con 0x7f55ec153c30 2023-10-06T02:32:04.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.591+0000 7f55eb7fe700 1 -- 172.21.15.31:0/1505020250 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55e0011790 con 0x7f55ec153c30 2023-10-06T02:32:04.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.591+0000 7f55eb7fe700 1 -- 172.21.15.31:0/1505020250 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e0011b30 con 0x7f55ec153c30 2023-10-06T02:32:04.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.592+0000 7f55f1f9d700 1 -- 172.21.15.31:0/1505020250 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55ec153c30 msgr2=0x7f55ec15a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:04.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.592+0000 7f55f1f9d700 1 --2- 172.21.15.31:0/1505020250 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55ec153c30 0x7f55ec15a5e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f55e00098e0 tx=0x7f55e00091a0 comp rx=0 tx=0).stop 2023-10-06T02:32:04.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.592+0000 7f55f1f9d700 1 -- 172.21.15.31:0/1505020250 shutdown_connections 2023-10-06T02:32:04.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.592+0000 7f55f1f9d700 1 --2- 172.21.15.31:0/1505020250 >> v2:172.21.15.62:3300/0 conn(0x7f55ec0cf110 0x7f55ec15c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:04.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.592+0000 7f55f1f9d700 1 --2- 172.21.15.31:0/1505020250 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55ec153c30 0x7f55ec15a5e0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:04.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.592+0000 7f55f1f9d700 1 -- 172.21.15.31:0/1505020250 >> 172.21.15.31:0/1505020250 conn(0x7f55ec0703a0 msgr2=0x7f55ec0707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:04.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.592+0000 7f55f1f9d700 1 -- 172.21.15.31:0/1505020250 shutdown_connections 2023-10-06T02:32:04.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.592+0000 7f55f1f9d700 1 -- 172.21.15.31:0/1505020250 wait complete. 2023-10-06T02:32:04.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.593+0000 7f55f1f9d700 1 Processor -- start 2023-10-06T02:32:04.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.593+0000 7f55f1f9d700 1 -- start start 2023-10-06T02:32:04.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.594+0000 7f55f1f9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55ec0cf110 0x7f55ec0b32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:04.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.594+0000 7f55f1f9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55ec153c30 0x7f55ec0b5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:04.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.594+0000 7f55f1f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55ec15de50 0x7f55ec0c9e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:04.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.594+0000 7f55f1f9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55ec0b5a50 con 0x7f55ec153c30 2023-10-06T02:32:04.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.594+0000 7f55f1f9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55ec0b45e0 con 0x7f55ec0cf110 2023-10-06T02:32:04.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.594+0000 7f55f1f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55ec0b39d0 con 0x7f55ec15de50 2023-10-06T02:32:04.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.595+0000 7f55ebfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55ec153c30 0x7f55ec0b5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:04.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.595+0000 7f55ebfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55ec153c30 0x7f55ec0b5510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37478/0 (socket says 172.21.15.31:37478) 2023-10-06T02:32:04.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.595+0000 7f55ebfff700 1 -- 172.21.15.31:0/859184196 learned_addr learned my addr 172.21.15.31:0/859184196 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:04.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.595+0000 7f55f0f9b700 1 --2- 172.21.15.31:0/859184196 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55ec0cf110 0x7f55ec0b32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:04.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.595+0000 7f55f179c700 1 --2- 172.21.15.31:0/859184196 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55ec15de50 0x7f55ec0c9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:04.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.595+0000 7f55ebfff700 1 -- 172.21.15.31:0/859184196 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55ec15de50 msgr2=0x7f55ec0c9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:04.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.595+0000 7f55ebfff700 1 --2- 172.21.15.31:0/859184196 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55ec15de50 0x7f55ec0c9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:04.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.595+0000 7f55ebfff700 1 -- 172.21.15.31:0/859184196 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55ec0cf110 msgr2=0x7f55ec0b32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:04.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.595+0000 7f55ebfff700 1 --2- 172.21.15.31:0/859184196 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55ec0cf110 0x7f55ec0b32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:04.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.595+0000 7f55ebfff700 1 -- 172.21.15.31:0/859184196 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55e0009420 con 0x7f55ec153c30 2023-10-06T02:32:04.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.595+0000 7f55f0f9b700 1 --2- 172.21.15.31:0/859184196 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55ec0cf110 0x7f55ec0b32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:32:04.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.595+0000 7f55ebfff700 1 --2- 172.21.15.31:0/859184196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55ec153c30 0x7f55ec0b5510 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f55d800c590 tx=0x7f55d8007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:04.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.596+0000 7f55e8ff9700 1 -- 172.21.15.31:0/859184196 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d8005b30 con 0x7f55ec153c30 2023-10-06T02:32:04.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.596+0000 7f55e8ff9700 1 -- 172.21.15.31:0/859184196 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55d8005cb0 con 0x7f55ec153c30 2023-10-06T02:32:04.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.596+0000 7f55e8ff9700 1 -- 172.21.15.31:0/859184196 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d8007320 con 0x7f55ec153c30 2023-10-06T02:32:04.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.596+0000 7f55f1f9d700 1 -- 172.21.15.31:0/859184196 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55ec0b3cb0 con 0x7f55ec153c30 2023-10-06T02:32:04.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.596+0000 7f55f1f9d700 1 -- 172.21.15.31:0/859184196 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55ec0b4270 con 0x7f55ec153c30 2023-10-06T02:32:04.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.597+0000 7f55f1f9d700 1 -- 172.21.15.31:0/859184196 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55ec04c3c0 con 0x7f55ec153c30 2023-10-06T02:32:04.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.598+0000 7f55e8ff9700 1 -- 172.21.15.31:0/859184196 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85556+0+0 (secure 0 0 0) 0x7f55d801b3b0 con 0x7f55ec153c30 2023-10-06T02:32:04.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.598+0000 7f55e8ff9700 1 --2- 172.21.15.31:0/859184196 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f55d0067830 0x7f55d0069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:04.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.599+0000 7f55e8ff9700 1 -- 172.21.15.31:0/859184196 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5234+0+0 (secure 0 0 0) 0x7f55d8082310 con 0x7f55ec153c30 2023-10-06T02:32:04.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.599+0000 7f55f0f9b700 1 --2- 172.21.15.31:0/859184196 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f55d0067830 0x7f55d0069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:04.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.600+0000 7f55f0f9b700 1 --2- 172.21.15.31:0/859184196 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f55d0067830 0x7f55d0069ce0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f55e00091e0 tx=0x7f55e0017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:04.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.603+0000 7f55e8ff9700 1 -- 172.21.15.31:0/859184196 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f55d8050f50 con 0x7f55ec153c30 2023-10-06T02:32:04.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.807+0000 7f55f1f9d700 1 -- 172.21.15.31:0/859184196 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f55ec15bb50 con 0x7f55ec153c30 2023-10-06T02:32:04.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.807+0000 7f55e8ff9700 1 -- 172.21.15.31:0/859184196 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v165) v1 ==== 84+0+126 (secure 0 0 0) 0x7f55d8017090 con 0x7f55ec153c30 2023-10-06T02:32:04.808 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:32:04.808 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2023-10-06T02:32:04.808 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:32:04.809 INFO:teuthology.orchestra.run.smithi031.stdout: "poolnum": 1, 2023-10-06T02:32:04.809 INFO:teuthology.orchestra.run.smithi031.stdout: "poolname": ".mgr" 2023-10-06T02:32:04.809 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T02:32:04.809 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:32:04.809 INFO:teuthology.orchestra.run.smithi031.stdout: "poolnum": 2, 2023-10-06T02:32:04.809 INFO:teuthology.orchestra.run.smithi031.stdout: "poolname": "rbd" 2023-10-06T02:32:04.809 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:32:04.809 INFO:teuthology.orchestra.run.smithi031.stdout:] 2023-10-06T02:32:04.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.810+0000 7f55f1f9d700 1 -- 172.21.15.31:0/859184196 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f55d0067830 msgr2=0x7f55d0069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:04.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.810+0000 7f55f1f9d700 1 --2- 172.21.15.31:0/859184196 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f55d0067830 0x7f55d0069ce0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f55e00091e0 tx=0x7f55e0017000 comp rx=0 tx=0).stop 2023-10-06T02:32:04.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.810+0000 7f55f1f9d700 1 -- 172.21.15.31:0/859184196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55ec153c30 msgr2=0x7f55ec0b5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:04.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.810+0000 7f55f1f9d700 1 --2- 172.21.15.31:0/859184196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55ec153c30 0x7f55ec0b5510 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f55d800c590 tx=0x7f55d8007e00 comp rx=0 tx=0).stop 2023-10-06T02:32:04.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.810+0000 7f55f1f9d700 1 -- 172.21.15.31:0/859184196 shutdown_connections 2023-10-06T02:32:04.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.810+0000 7f55f1f9d700 1 --2- 172.21.15.31:0/859184196 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f55d0067830 0x7f55d0069ce0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:04.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.810+0000 7f55f1f9d700 1 --2- 172.21.15.31:0/859184196 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55ec15de50 0x7f55ec0c9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:04.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.810+0000 7f55f1f9d700 1 --2- 172.21.15.31:0/859184196 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55ec0cf110 0x7f55ec0b32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:04.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.810+0000 7f55f1f9d700 1 --2- 172.21.15.31:0/859184196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55ec153c30 0x7f55ec0b5510 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:04.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.810+0000 7f55f1f9d700 1 -- 172.21.15.31:0/859184196 >> 172.21.15.31:0/859184196 conn(0x7f55ec0703a0 msgr2=0x7f55ec0ce960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:04.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.810+0000 7f55f1f9d700 1 -- 172.21.15.31:0/859184196 shutdown_connections 2023-10-06T02:32:04.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:04.810+0000 7f55f1f9d700 1 -- 172.21.15.31:0/859184196 wait complete. 2023-10-06T02:32:04.828 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T02:32:04.828 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T02:32:05.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:05.019+0000 7f121679d700 1 Processor -- start 2023-10-06T02:32:05.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:05.019+0000 7f121679d700 1 -- start start 2023-10-06T02:32:05.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:05.020+0000 7f121679d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f12100cb450 0x7f121015dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:05.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:05.020+0000 7f121679d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1210143070 0x7f12100cbf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:05.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:05.020+0000 7f121679d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f121015d8d0 con 0x7f1210142ca0 2023-10-06T02:32:05.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:05.020+0000 7f121679d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f121015d2f0 con 0x7f1210143070 2023-10-06T02:32:05.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:05.020+0000 7f121679d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12100cc630 con 0x7f12100cb450 2023-10-06T02:32:05.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:05.021+0000 7f1215f9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1210143070 0x7f12100cbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:05.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:05.021+0000 7f1215f9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1210143070 0x7f12100cbf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37488/0 (socket says 172.21.15.31:37488) 2023-10-06T02:32:05.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:05.021+0000 7f1215f9c700 1 -- 172.21.15.31:0/988328429 learned_addr learned my addr 172.21.15.31:0/988328429 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:05.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:05.021+0000 7f1214f9a700 1 --2- 172.21.15.31:0/988328429 >> v2:172.21.15.62:3300/0 conn(0x7f12100cb450 0x7f121015dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:05.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:05.021+0000 7f1215f9c700 1 -- 172.21.15.31:0/988328429 >> v2:172.21.15.62:3300/0 conn(0x7f12100cb450 msgr2=0x7f121015dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:05.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:05.021+0000 7f1215f9c700 1 --2- 172.21.15.31:0/988328429 >> v2:172.21.15.62:3300/0 conn(0x7f12100cb450 0x7f121015dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:05.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:05.021+0000 7f1215f9c700 1 -- 172.21.15.31:0/988328429 >> v1:172.21.15.62:6789/0 conn(0x7f1210142ca0 legacy=0x7f12100cad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:05.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:05.021+0000 7f1215f9c700 1 -- 172.21.15.31:0/988328429 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f121015d560 con 0x7f1210143070 2023-10-06T02:32:05.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:05.021+0000 7f1215f9c700 1 --2- 172.21.15.31:0/988328429 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1210143070 0x7f12100cbf30 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f1210068d50 tx=0x7f1208009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e01d1bcd2da7a847 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:05.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:05.021+0000 7f1207fff700 1 -- 172.21.15.31:0/988328429 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f120800e070 con 0x7f1210143070 2023-10-06T02:32:05.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:05.022+0000 7f1207fff700 1 -- 172.21.15.31:0/988328429 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f120800ba50 con 0x7f1210143070 2023-10-06T02:32:05.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:05.022+0000 7f1207fff700 1 -- 172.21.15.31:0/988328429 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12080092e0 con 0x7f1210143070 2023-10-06T02:32:05.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:05.022+0000 7f121679d700 1 -- 172.21.15.31:0/988328429 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1210143070 msgr2=0x7f12100cbf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:05.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:05.022+0000 7f121679d700 1 --2- 172.21.15.31:0/988328429 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1210143070 0x7f12100cbf30 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f1210068d50 tx=0x7f1208009840 comp rx=0 tx=0).stop 2023-10-06T02:32:05.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:05.022+0000 7f121679d700 1 -- 172.21.15.31:0/988328429 shutdown_connections 2023-10-06T02:32:05.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:05.022+0000 7f121679d700 1 --2- 172.21.15.31:0/988328429 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1210143070 0x7f12100cbf30 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:05.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:05.022+0000 7f121679d700 1 --2- 172.21.15.31:0/988328429 >> v2:172.21.15.62:3300/0 conn(0x7f12100cb450 0x7f121015dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:05.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:05.022+0000 7f121679d700 1 -- 172.21.15.31:0/988328429 >> 172.21.15.31:0/988328429 conn(0x7f1210067c80 msgr2=0x7f1210068080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:05.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:05.022+0000 7f121679d700 1 -- 172.21.15.31:0/988328429 shutdown_connections 2023-10-06T02:32:05.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:05.022+0000 7f121679d700 1 -- 172.21.15.31:0/988328429 wait complete. 2023-10-06T02:32:05.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:05.024+0000 7f121679d700 1 Processor -- start 2023-10-06T02:32:05.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:05.024+0000 7f121679d700 1 -- start start 2023-10-06T02:32:05.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:05.025+0000 7f121679d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f12100cb450 0x7f121013fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:05.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:05.025+0000 7f121679d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1210142ca0 0x7f12100b1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:05.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:05.025+0000 7f121679d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1210143070 0x7f12100694d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:05.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:05.025+0000 7f121679d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12100b2460 con 0x7f1210143070 2023-10-06T02:32:05.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:05.025+0000 7f121679d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1210169770 con 0x7f1210142ca0 2023-10-06T02:32:05.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:05.025+0000 7f121679d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12100b1bc0 con 0x7f12100cb450 2023-10-06T02:32:05.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:05.025+0000 7f1215f9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1210143070 0x7f12100694d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:05.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:05.025+0000 7f1215f9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1210143070 0x7f12100694d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37500/0 (socket says 172.21.15.31:37500) 2023-10-06T02:32:05.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:05.025+0000 7f1215f9c700 1 -- 172.21.15.31:0/3211398776 learned_addr learned my addr 172.21.15.31:0/3211398776 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:05.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:05.025+0000 7f121579b700 1 --2- 172.21.15.31:0/3211398776 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f12100cb450 0x7f121013fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:05.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:05.025+0000 7f1214f9a700 1 --2- 172.21.15.31:0/3211398776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1210142ca0 0x7f12100b1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:05.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:05.025+0000 7f1215f9c700 1 -- 172.21.15.31:0/3211398776 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f12100cb450 msgr2=0x7f121013fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:05.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:05.025+0000 7f1215f9c700 1 --2- 172.21.15.31:0/3211398776 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f12100cb450 0x7f121013fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:05.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:05.025+0000 7f1215f9c700 1 -- 172.21.15.31:0/3211398776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1210142ca0 msgr2=0x7f12100b1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:05.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:05.025+0000 7f1215f9c700 1 --2- 172.21.15.31:0/3211398776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1210142ca0 0x7f12100b1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:05.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:05.025+0000 7f1215f9c700 1 -- 172.21.15.31:0/3211398776 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1208009ae0 con 0x7f1210143070 2023-10-06T02:32:05.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:05.025+0000 7f1214f9a700 1 --2- 172.21.15.31:0/3211398776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1210142ca0 0x7f12100b1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:32:05.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:05.026+0000 7f1215f9c700 1 --2- 172.21.15.31:0/3211398776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1210143070 0x7f12100694d0 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f12100691a0 tx=0x7f1208018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:05.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:05.026+0000 7f12057fa700 1 -- 172.21.15.31:0/3211398776 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1208008e80 con 0x7f1210143070 2023-10-06T02:32:05.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:05.026+0000 7f12057fa700 1 -- 172.21.15.31:0/3211398776 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f120800be60 con 0x7f1210143070 2023-10-06T02:32:05.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:05.026+0000 7f12057fa700 1 -- 172.21.15.31:0/3211398776 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1208026690 con 0x7f1210143070 2023-10-06T02:32:05.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:05.026+0000 7f121679d700 1 -- 172.21.15.31:0/3211398776 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12100b1ea0 con 0x7f1210143070 2023-10-06T02:32:05.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:05.026+0000 7f121679d700 1 -- 172.21.15.31:0/3211398776 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1210169960 con 0x7f1210143070 2023-10-06T02:32:05.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:05.027+0000 7f121679d700 1 -- 172.21.15.31:0/3211398776 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1210169ba0 con 0x7f1210143070 2023-10-06T02:32:05.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:05.028+0000 7f12057fa700 1 -- 172.21.15.31:0/3211398776 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85556+0+0 (secure 0 0 0) 0x7f1208003040 con 0x7f1210143070 2023-10-06T02:32:05.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:05.029+0000 7f12057fa700 1 --2- 172.21.15.31:0/3211398776 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f11f4067830 0x7f11f4069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:05.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:05.029+0000 7f12057fa700 1 -- 172.21.15.31:0/3211398776 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5234+0+0 (secure 0 0 0) 0x7f1208084e70 con 0x7f1210143070 2023-10-06T02:32:05.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:05.029+0000 7f121579b700 1 --2- 172.21.15.31:0/3211398776 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f11f4067830 0x7f11f4069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:05.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:05.030+0000 7f121579b700 1 --2- 172.21.15.31:0/3211398776 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f11f4067830 0x7f11f4069ce0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f120c003860 tx=0x7f120c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:05.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:05.033+0000 7f12057fa700 1 -- 172.21.15.31:0/3211398776 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1208053ab0 con 0x7f1210143070 2023-10-06T02:32:05.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:05.235+0000 7f121679d700 1 -- 172.21.15.31:0/3211398776 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f121004c3c0 con 0x7f1210143070 2023-10-06T02:32:06.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:06.105+0000 7f12057fa700 1 -- 172.21.15.31:0/3211398776 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v166) v1 ==== 138+0+0 (secure 0 0 0) 0x7f1208009940 con 0x7f1210143070 2023-10-06T02:32:06.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:06.112+0000 7f121679d700 1 -- 172.21.15.31:0/3211398776 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f11f4067830 msgr2=0x7f11f4069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:06.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:06.112+0000 7f121679d700 1 --2- 172.21.15.31:0/3211398776 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f11f4067830 0x7f11f4069ce0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f120c003860 tx=0x7f120c009040 comp rx=0 tx=0).stop 2023-10-06T02:32:06.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:06.112+0000 7f121679d700 1 -- 172.21.15.31:0/3211398776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1210143070 msgr2=0x7f12100694d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:06.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:06.112+0000 7f121679d700 1 --2- 172.21.15.31:0/3211398776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1210143070 0x7f12100694d0 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f12100691a0 tx=0x7f1208018000 comp rx=0 tx=0).stop 2023-10-06T02:32:06.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:06.112+0000 7f121679d700 1 -- 172.21.15.31:0/3211398776 shutdown_connections 2023-10-06T02:32:06.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:06.112+0000 7f121679d700 1 --2- 172.21.15.31:0/3211398776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1210143070 0x7f12100694d0 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:06.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:06.112+0000 7f121679d700 1 --2- 172.21.15.31:0/3211398776 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f11f4067830 0x7f11f4069ce0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:06.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:06.112+0000 7f121679d700 1 --2- 172.21.15.31:0/3211398776 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f12100cb450 0x7f121013fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:06.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:06.112+0000 7f121679d700 1 --2- 172.21.15.31:0/3211398776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1210142ca0 0x7f12100b1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:06.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:06.113+0000 7f121679d700 1 -- 172.21.15.31:0/3211398776 >> 172.21.15.31:0/3211398776 conn(0x7f1210067c80 msgr2=0x7f1210141cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:06.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:06.113+0000 7f121679d700 1 -- 172.21.15.31:0/3211398776 shutdown_connections 2023-10-06T02:32:06.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:06.113+0000 7f121679d700 1 -- 172.21.15.31:0/3211398776 wait complete. 2023-10-06T02:32:06.116 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' created 2023-10-06T02:32:06.135 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T02:32:06.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:06.334+0000 7fc00939d700 1 Processor -- start 2023-10-06T02:32:06.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:06.334+0000 7fc00939d700 1 -- start start 2023-10-06T02:32:06.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:06.335+0000 7fc00939d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc00413fb20 0x7fc00415a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:06.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:06.336+0000 7fc00939d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc00413fef0 0x7fc0040c7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:06.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:06.336+0000 7fc00939d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc00415ae70 con 0x7fc00406aaa0 2023-10-06T02:32:06.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:06.336+0000 7fc00939d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc00406af60 con 0x7fc00413fef0 2023-10-06T02:32:06.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:06.336+0000 7fc00939d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc00406b240 con 0x7fc00413fb20 2023-10-06T02:32:06.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:06.336+0000 7fc0037fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc00413fef0 0x7fc0040c7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:06.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:06.336+0000 7fc008b9c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fc00406aaa0 0x7fc0040c6770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:43944/0 (socket says 172.21.15.31:43944) 2023-10-06T02:32:06.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:06.336+0000 7fc008b9c700 1 -- 172.21.15.31:0/915963974 learned_addr learned my addr 172.21.15.31:0/915963974 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:32:06.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:06.336+0000 7fc003fff700 1 --2- 172.21.15.31:0/915963974 >> v2:172.21.15.62:3300/0 conn(0x7fc00413fb20 0x7fc00415a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:06.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:06.336+0000 7fc002ffd700 1 -- 172.21.15.31:0/915963974 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 493957574 0 0) 0x7fbffc001490 con 0x7fc00406aaa0 2023-10-06T02:32:06.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:06.336+0000 7fc0037fe700 1 -- 172.21.15.31:0/915963974 >> v2:172.21.15.62:3300/0 conn(0x7fc00413fb20 msgr2=0x7fc00415a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:06.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:06.336+0000 7fc0037fe700 1 --2- 172.21.15.31:0/915963974 >> v2:172.21.15.62:3300/0 conn(0x7fc00413fb20 0x7fc00415a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:06.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:06.336+0000 7fc0037fe700 1 -- 172.21.15.31:0/915963974 >> v1:172.21.15.31:6789/0 conn(0x7fc00406aaa0 legacy=0x7fc0040c6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:06.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:06.336+0000 7fc0037fe700 1 -- 172.21.15.31:0/915963974 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc00406b520 con 0x7fc00413fef0 2023-10-06T02:32:06.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:06.336+0000 7fc003fff700 1 --2- 172.21.15.31:0/915963974 >> v2:172.21.15.62:3300/0 conn(0x7fc00413fb20 0x7fc00415a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:32:06.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:06.337+0000 7fc0037fe700 1 --2- 172.21.15.31:0/915963974 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc00413fef0 0x7fc0040c7990 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7fc0040695b0 tx=0x7fbff8009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef559246789a2383 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:06.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:06.337+0000 7fc002ffd700 1 -- 172.21.15.31:0/915963974 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff8005c10 con 0x7fc00413fef0 2023-10-06T02:32:06.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:06.337+0000 7fc002ffd700 1 -- 172.21.15.31:0/915963974 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbff8005d70 con 0x7fc00413fef0 2023-10-06T02:32:06.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:06.337+0000 7fc002ffd700 1 -- 172.21.15.31:0/915963974 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff8006260 con 0x7fc00413fef0 2023-10-06T02:32:06.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:06.337+0000 7fc00939d700 1 -- 172.21.15.31:0/915963974 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc00413fef0 msgr2=0x7fc0040c7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:06.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:06.337+0000 7fc00939d700 1 --2- 172.21.15.31:0/915963974 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc00413fef0 0x7fc0040c7990 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7fc0040695b0 tx=0x7fbff8009120 comp rx=0 tx=0).stop 2023-10-06T02:32:06.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:06.338+0000 7fc00939d700 1 -- 172.21.15.31:0/915963974 shutdown_connections 2023-10-06T02:32:06.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:06.338+0000 7fc00939d700 1 --2- 172.21.15.31:0/915963974 >> v2:172.21.15.62:3300/0 conn(0x7fc00413fb20 0x7fc00415a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:06.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:06.338+0000 7fc00939d700 1 --2- 172.21.15.31:0/915963974 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc00413fef0 0x7fc0040c7990 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:06.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:06.338+0000 7fc00939d700 1 -- 172.21.15.31:0/915963974 >> 172.21.15.31:0/915963974 conn(0x7fc00406d170 msgr2=0x7fc0040cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:06.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:06.338+0000 7fc00939d700 1 -- 172.21.15.31:0/915963974 shutdown_connections 2023-10-06T02:32:06.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:06.338+0000 7fc00939d700 1 -- 172.21.15.31:0/915963974 wait complete. 2023-10-06T02:32:06.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:06.339+0000 7fc00939d700 1 Processor -- start 2023-10-06T02:32:06.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:06.339+0000 7fc00939d700 1 -- start start 2023-10-06T02:32:06.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:06.340+0000 7fc00939d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc00406aaa0 0x7fc0040b16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:06.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:06.340+0000 7fc00939d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc00413fb20 0x7fc00414d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:06.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:06.340+0000 7fc00939d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc00413fef0 0x7fc004136b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:06.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:06.340+0000 7fc00939d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc004156e20 con 0x7fc00406aaa0 2023-10-06T02:32:06.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:06.340+0000 7fc00939d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc004156b80 con 0x7fc00413fb20 2023-10-06T02:32:06.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:06.340+0000 7fc00939d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc00413ea10 con 0x7fc00413fef0 2023-10-06T02:32:06.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:06.340+0000 7fc003fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc00406aaa0 0x7fc0040b16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:06.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:06.340+0000 7fc003fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc00406aaa0 0x7fc0040b16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37526/0 (socket says 172.21.15.31:37526) 2023-10-06T02:32:06.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:06.340+0000 7fc003fff700 1 -- 172.21.15.31:0/915895492 learned_addr learned my addr 172.21.15.31:0/915895492 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:06.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:06.341+0000 7fc0037fe700 1 --2- 172.21.15.31:0/915895492 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc00413fb20 0x7fc00414d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:06.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:06.341+0000 7fc008b9c700 1 --2- 172.21.15.31:0/915895492 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc00413fef0 0x7fc004136b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:06.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:06.341+0000 7fc003fff700 1 -- 172.21.15.31:0/915895492 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc00413fef0 msgr2=0x7fc004136b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:06.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:06.341+0000 7fc003fff700 1 --2- 172.21.15.31:0/915895492 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc00413fef0 0x7fc004136b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:06.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:06.341+0000 7fc003fff700 1 -- 172.21.15.31:0/915895492 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc00413fb20 msgr2=0x7fc00414d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:06.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:06.341+0000 7fc003fff700 1 --2- 172.21.15.31:0/915895492 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc00413fb20 0x7fc00414d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:06.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:06.341+0000 7fc003fff700 1 -- 172.21.15.31:0/915895492 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbff80093c0 con 0x7fc00406aaa0 2023-10-06T02:32:06.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:06.341+0000 7fc003fff700 1 --2- 172.21.15.31:0/915895492 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc00406aaa0 0x7fc0040b16e0 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7fbff400e900 tx=0x7fbff4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:06.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:06.341+0000 7fbfebfff700 1 -- 172.21.15.31:0/915895492 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff4013410 con 0x7fc00406aaa0 2023-10-06T02:32:06.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:06.341+0000 7fbfebfff700 1 -- 172.21.15.31:0/915895492 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbff4013570 con 0x7fc00406aaa0 2023-10-06T02:32:06.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:06.341+0000 7fc00939d700 1 -- 172.21.15.31:0/915895492 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc00413ecb0 con 0x7fc00406aaa0 2023-10-06T02:32:06.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:06.341+0000 7fbfebfff700 1 -- 172.21.15.31:0/915895492 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff4013910 con 0x7fc00406aaa0 2023-10-06T02:32:06.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:06.341+0000 7fc00939d700 1 -- 172.21.15.31:0/915895492 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0040b1dc0 con 0x7fc00406aaa0 2023-10-06T02:32:06.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:06.342+0000 7fc00939d700 1 -- 172.21.15.31:0/915895492 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc00404c3c0 con 0x7fc00406aaa0 2023-10-06T02:32:06.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:06.344+0000 7fbfebfff700 1 -- 172.21.15.31:0/915895492 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85556+0+0 (secure 0 0 0) 0x7fbff4013d00 con 0x7fc00406aaa0 2023-10-06T02:32:06.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:06.344+0000 7fbfebfff700 1 --2- 172.21.15.31:0/915895492 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fbff0067830 0x7fbff0069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:06.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:06.345+0000 7fbfebfff700 1 -- 172.21.15.31:0/915895492 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5604+0+0 (secure 0 0 0) 0x7fbff4082dc0 con 0x7fc00406aaa0 2023-10-06T02:32:06.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:06.345+0000 7fc0037fe700 1 --2- 172.21.15.31:0/915895492 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fbff0067830 0x7fbff0069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:06.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:06.346+0000 7fc0037fe700 1 --2- 172.21.15.31:0/915895492 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fbff0067830 0x7fbff0069ce0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fc0040695b0 tx=0x7fbff8016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:06.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:06.349+0000 7fbfebfff700 1 -- 172.21.15.31:0/915895492 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbff40552a0 con 0x7fc00406aaa0 2023-10-06T02:32:06.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:06.551+0000 7fc00939d700 1 -- 172.21.15.31:0/915895492 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fc00406b800 con 0x7fc00406aaa0 2023-10-06T02:32:07.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.113+0000 7fbfebfff700 1 -- 172.21.15.31:0/915895492 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v167) v1 ==== 170+0+0 (secure 0 0 0) 0x7fbff4054e30 con 0x7fc00406aaa0 2023-10-06T02:32:07.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.117+0000 7fc00939d700 1 -- 172.21.15.31:0/915895492 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fbff0067830 msgr2=0x7fbff0069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:07.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.117+0000 7fc00939d700 1 --2- 172.21.15.31:0/915895492 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fbff0067830 0x7fbff0069ce0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fc0040695b0 tx=0x7fbff8016000 comp rx=0 tx=0).stop 2023-10-06T02:32:07.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.117+0000 7fc00939d700 1 -- 172.21.15.31:0/915895492 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc00406aaa0 msgr2=0x7fc0040b16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:07.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.117+0000 7fc00939d700 1 --2- 172.21.15.31:0/915895492 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc00406aaa0 0x7fc0040b16e0 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7fbff400e900 tx=0x7fbff4008040 comp rx=0 tx=0).stop 2023-10-06T02:32:07.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.119+0000 7fc00939d700 1 -- 172.21.15.31:0/915895492 shutdown_connections 2023-10-06T02:32:07.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.119+0000 7fc00939d700 1 --2- 172.21.15.31:0/915895492 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fbff0067830 0x7fbff0069ce0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:07.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.119+0000 7fc00939d700 1 --2- 172.21.15.31:0/915895492 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc00413fef0 0x7fc004136b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:07.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.119+0000 7fc00939d700 1 --2- 172.21.15.31:0/915895492 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc00406aaa0 0x7fc0040b16e0 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:07.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.119+0000 7fc00939d700 1 --2- 172.21.15.31:0/915895492 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc00413fb20 0x7fc00414d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:07.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.119+0000 7fc00939d700 1 -- 172.21.15.31:0/915895492 >> 172.21.15.31:0/915895492 conn(0x7fc00406d170 msgr2=0x7fc0040cbd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:07.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.126+0000 7fc00939d700 1 -- 172.21.15.31:0/915895492 shutdown_connections 2023-10-06T02:32:07.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.126+0000 7fc00939d700 1 -- 172.21.15.31:0/915895492 wait complete. 2023-10-06T02:32:07.132 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' created 2023-10-06T02:32:07.154 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T02:32:07.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.348+0000 7f255919d700 1 Processor -- start 2023-10-06T02:32:07.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.348+0000 7f255919d700 1 -- start start 2023-10-06T02:32:07.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.349+0000 7f255919d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f25540cf110 0x7f255415de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:07.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.349+0000 7f255919d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f255415ac50 con 0x7f2554153dd0 2023-10-06T02:32:07.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.349+0000 7f255919d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f255415d010 con 0x7f2554153a00 2023-10-06T02:32:07.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.349+0000 7f255919d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f255415af60 con 0x7f25540cf110 2023-10-06T02:32:07.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.349+0000 7f25537fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f2554153dd0 0x7f25540cea00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:43954/0 (socket says 172.21.15.31:43954) 2023-10-06T02:32:07.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.349+0000 7f25537fe700 1 -- 172.21.15.31:0/4076692561 learned_addr learned my addr 172.21.15.31:0/4076692561 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:32:07.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.350+0000 7f255899c700 1 --2- 172.21.15.31:0/4076692561 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f25540cf110 0x7f255415de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:07.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.350+0000 7f2552ffd700 1 -- 172.21.15.31:0/4076692561 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 27929719 0 0) 0x7f255415ac50 con 0x7f2554153dd0 2023-10-06T02:32:07.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.350+0000 7f2552ffd700 1 -- 172.21.15.31:0/4076692561 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f253c003570 con 0x7f2554153dd0 2023-10-06T02:32:07.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.350+0000 7f2552ffd700 1 -- 172.21.15.31:0/4076692561 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2688856228 0 0) 0x7f255415d010 con 0x7f2554153a00 2023-10-06T02:32:07.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.350+0000 7f2552ffd700 1 -- 172.21.15.31:0/4076692561 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f255415ac50 con 0x7f2554153a00 2023-10-06T02:32:07.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.350+0000 7f2552ffd700 1 -- 172.21.15.31:0/4076692561 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 363764314 0 0) 0x7f253c003570 con 0x7f2554153dd0 2023-10-06T02:32:07.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.350+0000 7f2552ffd700 1 -- 172.21.15.31:0/4076692561 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f25540cf110 msgr2=0x7f255415de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:07.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.350+0000 7f2552ffd700 1 --2- 172.21.15.31:0/4076692561 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f25540cf110 0x7f255415de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:07.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.350+0000 7f2552ffd700 1 -- 172.21.15.31:0/4076692561 >> v1:172.21.15.62:6789/0 conn(0x7f2554153a00 legacy=0x7f255415a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:07.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.350+0000 7f2552ffd700 1 -- 172.21.15.31:0/4076692561 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f255415b240 con 0x7f2554153dd0 2023-10-06T02:32:07.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.350+0000 7f2552ffd700 1 -- 172.21.15.31:0/4076692561 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7f2548001470 con 0x7f2554153dd0 2023-10-06T02:32:07.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.350+0000 7f2552ffd700 1 -- 172.21.15.31:0/4076692561 >> v1:172.21.15.31:6789/0 conn(0x7f2554153dd0 legacy=0x7f25540cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:07.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.350+0000 7f2552ffd700 1 --2- 172.21.15.31:0/4076692561 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f25540cf110 0x7f253c003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:07.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.350+0000 7f2552ffd700 1 --2- 172.21.15.31:0/4076692561 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f253c0041d0 0x7f253c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:07.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.350+0000 7f2552ffd700 1 --2- 172.21.15.31:0/4076692561 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f253c004d50 0x7f253c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:07.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.350+0000 7f2552ffd700 1 -- 172.21.15.31:0/4076692561 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f255415af60 con 0x7f253c0041d0 2023-10-06T02:32:07.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.350+0000 7f2552ffd700 1 -- 172.21.15.31:0/4076692561 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f253c00bbf0 con 0x7f253c004d50 2023-10-06T02:32:07.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.350+0000 7f2552ffd700 1 -- 172.21.15.31:0/4076692561 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f253c00bdb0 con 0x7f25540cf110 2023-10-06T02:32:07.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.351+0000 7f25537fe700 1 --2- 172.21.15.31:0/4076692561 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f253c0041d0 0x7f253c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:07.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.351+0000 7f2553fff700 1 --2- 172.21.15.31:0/4076692561 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f25540cf110 0x7f253c003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:07.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.351+0000 7f255899c700 1 --2- 172.21.15.31:0/4076692561 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f253c004d50 0x7f253c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:07.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.351+0000 7f25537fe700 1 -- 172.21.15.31:0/4076692561 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f25540cf110 msgr2=0x7f253c003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:07.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.351+0000 7f25537fe700 1 --2- 172.21.15.31:0/4076692561 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f25540cf110 0x7f253c003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:07.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.351+0000 7f25537fe700 1 -- 172.21.15.31:0/4076692561 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f253c004d50 msgr2=0x7f253c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:07.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.351+0000 7f25537fe700 1 --2- 172.21.15.31:0/4076692561 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f253c004d50 0x7f253c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:07.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.351+0000 7f25537fe700 1 -- 172.21.15.31:0/4076692561 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f253c003570 con 0x7f253c0041d0 2023-10-06T02:32:07.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.351+0000 7f25537fe700 1 --2- 172.21.15.31:0/4076692561 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f253c0041d0 0x7f253c004810 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f254800c210 tx=0x7f2548001d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9049a2d963d48bf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:07.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.352+0000 7f2552ffd700 1 -- 172.21.15.31:0/4076692561 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f254800cc10 con 0x7f253c0041d0 2023-10-06T02:32:07.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.352+0000 7f2552ffd700 1 -- 172.21.15.31:0/4076692561 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f254800cd70 con 0x7f253c0041d0 2023-10-06T02:32:07.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.352+0000 7f2552ffd700 1 -- 172.21.15.31:0/4076692561 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2548005b70 con 0x7f253c0041d0 2023-10-06T02:32:07.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.352+0000 7f255919d700 1 -- 172.21.15.31:0/4076692561 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f253c0041d0 msgr2=0x7f253c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:07.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.352+0000 7f255919d700 1 --2- 172.21.15.31:0/4076692561 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f253c0041d0 0x7f253c004810 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f254800c210 tx=0x7f2548001d40 comp rx=0 tx=0).stop 2023-10-06T02:32:07.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.352+0000 7f255899c700 1 -- 172.21.15.31:0/4076692561 reap_dead start 2023-10-06T02:32:07.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.352+0000 7f255919d700 1 -- 172.21.15.31:0/4076692561 shutdown_connections 2023-10-06T02:32:07.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.352+0000 7f255919d700 1 -- 172.21.15.31:0/4076692561 >> 172.21.15.31:0/4076692561 conn(0x7f25540703a0 msgr2=0x7f25540707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:07.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.352+0000 7f255919d700 1 -- 172.21.15.31:0/4076692561 shutdown_connections 2023-10-06T02:32:07.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.352+0000 7f255919d700 1 -- 172.21.15.31:0/4076692561 wait complete. 2023-10-06T02:32:07.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.354+0000 7f255919d700 1 Processor -- start 2023-10-06T02:32:07.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.354+0000 7f255919d700 1 -- start start 2023-10-06T02:32:07.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.355+0000 7f255919d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f253c0041d0 0x7f25540c75e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:07.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.355+0000 7f255919d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f25540ca150 0x7f25540b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:07.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.355+0000 7f255919d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2554143f00 0x7f255415c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:07.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.355+0000 7f255919d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25540b2730 con 0x7f2554143f00 2023-10-06T02:32:07.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.355+0000 7f255919d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2554152d20 con 0x7f253c0041d0 2023-10-06T02:32:07.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.355+0000 7f255919d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25540c99e0 con 0x7f25540ca150 2023-10-06T02:32:07.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.355+0000 7f255899c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2554143f00 0x7f255415c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:07.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.355+0000 7f255899c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2554143f00 0x7f255415c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37556/0 (socket says 172.21.15.31:37556) 2023-10-06T02:32:07.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.355+0000 7f255899c700 1 -- 172.21.15.31:0/1670064072 learned_addr learned my addr 172.21.15.31:0/1670064072 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:07.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.356+0000 7f25537fe700 1 --2- 172.21.15.31:0/1670064072 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f25540ca150 0x7f25540b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:07.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.356+0000 7f2553fff700 1 --2- 172.21.15.31:0/1670064072 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f253c0041d0 0x7f25540c75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:07.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.356+0000 7f255899c700 1 -- 172.21.15.31:0/1670064072 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f25540ca150 msgr2=0x7f25540b21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:07.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.356+0000 7f255899c700 1 --2- 172.21.15.31:0/1670064072 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f25540ca150 0x7f25540b21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:07.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.356+0000 7f255899c700 1 -- 172.21.15.31:0/1670064072 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f253c0041d0 msgr2=0x7f25540c75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:07.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.356+0000 7f255899c700 1 --2- 172.21.15.31:0/1670064072 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f253c0041d0 0x7f25540c75e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:07.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.356+0000 7f255899c700 1 -- 172.21.15.31:0/1670064072 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25540c9cc0 con 0x7f2554143f00 2023-10-06T02:32:07.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.356+0000 7f2553fff700 1 --2- 172.21.15.31:0/1670064072 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f253c0041d0 0x7f25540c75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:32:07.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.356+0000 7f25537fe700 1 --2- 172.21.15.31:0/1670064072 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f25540ca150 0x7f25540b21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:32:07.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.356+0000 7f255899c700 1 --2- 172.21.15.31:0/1670064072 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2554143f00 0x7f255415c8a0 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f254c0094d0 tx=0x7f254c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:07.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.356+0000 7f253bfff700 1 -- 172.21.15.31:0/1670064072 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f254c00e040 con 0x7f2554143f00 2023-10-06T02:32:07.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.356+0000 7f253bfff700 1 -- 172.21.15.31:0/1670064072 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f254c00f040 con 0x7f2554143f00 2023-10-06T02:32:07.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.356+0000 7f253bfff700 1 -- 172.21.15.31:0/1670064072 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f254c00fe20 con 0x7f2554143f00 2023-10-06T02:32:07.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.356+0000 7f255919d700 1 -- 172.21.15.31:0/1670064072 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2554152f70 con 0x7f2554143f00 2023-10-06T02:32:07.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.356+0000 7f255919d700 1 -- 172.21.15.31:0/1670064072 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2554144380 con 0x7f2554143f00 2023-10-06T02:32:07.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.358+0000 7f255919d700 1 -- 172.21.15.31:0/1670064072 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25540c9e50 con 0x7f2554143f00 2023-10-06T02:32:07.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.359+0000 7f253bfff700 1 -- 172.21.15.31:0/1670064072 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85556+0+0 (secure 0 0 0) 0x7f254c015640 con 0x7f2554143f00 2023-10-06T02:32:07.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.360+0000 7f253bfff700 1 --2- 172.21.15.31:0/1670064072 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f2540067830 0x7f2540069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:07.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.360+0000 7f253bfff700 1 -- 172.21.15.31:0/1670064072 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 5986+0+0 (secure 0 0 0) 0x7f254c083770 con 0x7f2554143f00 2023-10-06T02:32:07.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.361+0000 7f2553fff700 1 --2- 172.21.15.31:0/1670064072 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f2540067830 0x7f2540069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:07.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.361+0000 7f2553fff700 1 --2- 172.21.15.31:0/1670064072 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f2540067830 0x7f2540069ce0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f2544000c00 tx=0x7f2544009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:07.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.365+0000 7f253bfff700 1 -- 172.21.15.31:0/1670064072 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f254c0523b0 con 0x7f2554143f00 2023-10-06T02:32:07.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:07.612+0000 7f255919d700 1 -- 172.21.15.31:0/1670064072 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f255404c3c0 con 0x7f2554143f00 2023-10-06T02:32:08.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.117+0000 7f253bfff700 1 -- 172.21.15.31:0/1670064072 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 27 and data pool 28 v115) v1 ==== 174+0+0 (secure 0 0 0) 0x7f254c0559d0 con 0x7f2554143f00 2023-10-06T02:32:08.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.131+0000 7f255919d700 1 -- 172.21.15.31:0/1670064072 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f2540067830 msgr2=0x7f2540069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:08.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.131+0000 7f255919d700 1 --2- 172.21.15.31:0/1670064072 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f2540067830 0x7f2540069ce0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f2544000c00 tx=0x7f2544009040 comp rx=0 tx=0).stop 2023-10-06T02:32:08.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.131+0000 7f255919d700 1 -- 172.21.15.31:0/1670064072 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2554143f00 msgr2=0x7f255415c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:08.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.131+0000 7f255919d700 1 --2- 172.21.15.31:0/1670064072 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2554143f00 0x7f255415c8a0 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f254c0094d0 tx=0x7f254c00b040 comp rx=0 tx=0).stop 2023-10-06T02:32:08.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.132+0000 7f255919d700 1 -- 172.21.15.31:0/1670064072 shutdown_connections 2023-10-06T02:32:08.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.132+0000 7f255919d700 1 --2- 172.21.15.31:0/1670064072 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2554143f00 0x7f255415c8a0 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:08.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.132+0000 7f255919d700 1 --2- 172.21.15.31:0/1670064072 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f253c0041d0 0x7f25540c75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:08.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.132+0000 7f255919d700 1 --2- 172.21.15.31:0/1670064072 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f2540067830 0x7f2540069ce0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:08.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.132+0000 7f255919d700 1 --2- 172.21.15.31:0/1670064072 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f25540ca150 0x7f25540b21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:08.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.132+0000 7f255919d700 1 -- 172.21.15.31:0/1670064072 >> 172.21.15.31:0/1670064072 conn(0x7f25540703a0 msgr2=0x7f25540709b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:08.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.134+0000 7f255919d700 1 -- 172.21.15.31:0/1670064072 shutdown_connections 2023-10-06T02:32:08.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.135+0000 7f255919d700 1 -- 172.21.15.31:0/1670064072 wait complete. 2023-10-06T02:32:08.142 INFO:teuthology.orchestra.run.smithi031.stderr:new fs with metadata pool 27 and data pool 28 2023-10-06T02:32:08.165 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:32:08.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.352+0000 7f0f94d9d700 1 Processor -- start 2023-10-06T02:32:08.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.352+0000 7f0f94d9d700 1 -- start start 2023-10-06T02:32:08.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.353+0000 7f0f94d9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0f90153c30 0x7f0f9015a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:08.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.353+0000 7f0f94d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0f900cf110 0x7f0f9015c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:08.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.353+0000 7f0f94d9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f9015afd0 con 0x7f0f9015de50 2023-10-06T02:32:08.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.353+0000 7f0f94d9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f9015b2b0 con 0x7f0f90153c30 2023-10-06T02:32:08.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.353+0000 7f0f94d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9015b590 con 0x7f0f900cf110 2023-10-06T02:32:08.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.354+0000 7f0f8effd700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f0f9015de50 0x7f0f900cea00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:47698/0 (socket says 172.21.15.31:47698) 2023-10-06T02:32:08.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.354+0000 7f0f8effd700 1 -- 172.21.15.31:0/3821480629 learned_addr learned my addr 172.21.15.31:0/3821480629 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:32:08.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.354+0000 7f0f8f7fe700 1 --2- 172.21.15.31:0/3821480629 >> v2:172.21.15.62:3300/0 conn(0x7f0f90153c30 0x7f0f9015a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:08.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.354+0000 7f0f8ffff700 1 --2- 172.21.15.31:0/3821480629 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0f900cf110 0x7f0f9015c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:08.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.354+0000 7f0f8e7fc700 1 -- 172.21.15.31:0/3821480629 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4119500065 0 0) 0x7f0f9015afd0 con 0x7f0f9015de50 2023-10-06T02:32:08.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.354+0000 7f0f8e7fc700 1 -- 172.21.15.31:0/3821480629 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f78003570 con 0x7f0f9015de50 2023-10-06T02:32:08.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.354+0000 7f0f8f7fe700 1 -- 172.21.15.31:0/3821480629 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0f900cf110 msgr2=0x7f0f9015c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:08.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.354+0000 7f0f8f7fe700 1 --2- 172.21.15.31:0/3821480629 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0f900cf110 0x7f0f9015c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:08.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.354+0000 7f0f8f7fe700 1 -- 172.21.15.31:0/3821480629 >> v1:172.21.15.31:6789/0 conn(0x7f0f9015de50 legacy=0x7f0f900cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:08.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.354+0000 7f0f8f7fe700 1 -- 172.21.15.31:0/3821480629 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f9015b870 con 0x7f0f90153c30 2023-10-06T02:32:08.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.354+0000 7f0f8ffff700 1 --2- 172.21.15.31:0/3821480629 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0f900cf110 0x7f0f9015c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:32:08.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.355+0000 7f0f8f7fe700 1 --2- 172.21.15.31:0/3821480629 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0f90153c30 0x7f0f9015a5e0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f0f800098e0 tx=0x7f0f800091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed3bb14a1efde2de server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:08.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.355+0000 7f0f8e7fc700 1 -- 172.21.15.31:0/3821480629 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f80011630 con 0x7f0f90153c30 2023-10-06T02:32:08.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.355+0000 7f0f8e7fc700 1 -- 172.21.15.31:0/3821480629 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f80011790 con 0x7f0f90153c30 2023-10-06T02:32:08.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.355+0000 7f0f8e7fc700 1 -- 172.21.15.31:0/3821480629 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f80011b30 con 0x7f0f90153c30 2023-10-06T02:32:08.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.356+0000 7f0f94d9d700 1 -- 172.21.15.31:0/3821480629 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0f90153c30 msgr2=0x7f0f9015a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:08.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.356+0000 7f0f94d9d700 1 --2- 172.21.15.31:0/3821480629 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0f90153c30 0x7f0f9015a5e0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f0f800098e0 tx=0x7f0f800091a0 comp rx=0 tx=0).stop 2023-10-06T02:32:08.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.356+0000 7f0f94d9d700 1 -- 172.21.15.31:0/3821480629 shutdown_connections 2023-10-06T02:32:08.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.356+0000 7f0f94d9d700 1 --2- 172.21.15.31:0/3821480629 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0f90153c30 0x7f0f9015a5e0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:08.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.356+0000 7f0f94d9d700 1 --2- 172.21.15.31:0/3821480629 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0f900cf110 0x7f0f9015c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:08.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.356+0000 7f0f94d9d700 1 -- 172.21.15.31:0/3821480629 >> 172.21.15.31:0/3821480629 conn(0x7f0f900703a0 msgr2=0x7f0f900707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:08.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.356+0000 7f0f94d9d700 1 -- 172.21.15.31:0/3821480629 shutdown_connections 2023-10-06T02:32:08.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.356+0000 7f0f94d9d700 1 -- 172.21.15.31:0/3821480629 wait complete. 2023-10-06T02:32:08.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.357+0000 7f0f94d9d700 1 Processor -- start 2023-10-06T02:32:08.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.357+0000 7f0f94d9d700 1 -- start start 2023-10-06T02:32:08.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.358+0000 7f0f94d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0f900cf110 0x7f0f900b32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:08.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.358+0000 7f0f94d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0f90153c30 0x7f0f900c7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:08.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.358+0000 7f0f94d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f9015de50 0x7f0f900b1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:08.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.358+0000 7f0f94d9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f900b0a50 con 0x7f0f9015de50 2023-10-06T02:32:08.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.358+0000 7f0f94d9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f900b0cf0 con 0x7f0f900cf110 2023-10-06T02:32:08.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.358+0000 7f0f94d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f900b0fd0 con 0x7f0f90153c30 2023-10-06T02:32:08.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.358+0000 7f0f8ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f9015de50 0x7f0f900b1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:08.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.358+0000 7f0f8ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f9015de50 0x7f0f900b1d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40986/0 (socket says 172.21.15.31:40986) 2023-10-06T02:32:08.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.358+0000 7f0f8ffff700 1 -- 172.21.15.31:0/870546536 learned_addr learned my addr 172.21.15.31:0/870546536 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:08.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.359+0000 7f0f8effd700 1 --2- 172.21.15.31:0/870546536 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0f90153c30 0x7f0f900c7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:08.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.359+0000 7f0f8f7fe700 1 --2- 172.21.15.31:0/870546536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0f900cf110 0x7f0f900b32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:08.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.359+0000 7f0f8ffff700 1 -- 172.21.15.31:0/870546536 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0f90153c30 msgr2=0x7f0f900c7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:08.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.359+0000 7f0f8ffff700 1 --2- 172.21.15.31:0/870546536 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0f90153c30 0x7f0f900c7ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:08.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.359+0000 7f0f8ffff700 1 -- 172.21.15.31:0/870546536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0f900cf110 msgr2=0x7f0f900b32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:08.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.359+0000 7f0f8ffff700 1 --2- 172.21.15.31:0/870546536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0f900cf110 0x7f0f900b32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:08.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.359+0000 7f0f8ffff700 1 -- 172.21.15.31:0/870546536 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f88009a20 con 0x7f0f9015de50 2023-10-06T02:32:08.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.359+0000 7f0f8ffff700 1 --2- 172.21.15.31:0/870546536 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f9015de50 0x7f0f900b1d90 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f0f88009fc0 tx=0x7f0f8800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:08.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.359+0000 7f0f777fe700 1 -- 172.21.15.31:0/870546536 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f8800cbd0 con 0x7f0f9015de50 2023-10-06T02:32:08.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.359+0000 7f0f777fe700 1 -- 172.21.15.31:0/870546536 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f8800f040 con 0x7f0f9015de50 2023-10-06T02:32:08.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.360+0000 7f0f94d9d700 1 -- 172.21.15.31:0/870546536 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f80009420 con 0x7f0f9015de50 2023-10-06T02:32:08.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.360+0000 7f0f777fe700 1 -- 172.21.15.31:0/870546536 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f88014480 con 0x7f0f9015de50 2023-10-06T02:32:08.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.360+0000 7f0f94d9d700 1 -- 172.21.15.31:0/870546536 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f900b1590 con 0x7f0f9015de50 2023-10-06T02:32:08.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.361+0000 7f0f94d9d700 1 -- 172.21.15.31:0/870546536 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f9004c3c0 con 0x7f0f9015de50 2023-10-06T02:32:08.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.362+0000 7f0f777fe700 1 -- 172.21.15.31:0/870546536 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85556+0+0 (secure 0 0 0) 0x7f0f88003020 con 0x7f0f9015de50 2023-10-06T02:32:08.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.362+0000 7f0f777fe700 1 --2- 172.21.15.31:0/870546536 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0f7c067830 0x7f0f7c069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:08.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.363+0000 7f0f777fe700 1 -- 172.21.15.31:0/870546536 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f0f88083750 con 0x7f0f9015de50 2023-10-06T02:32:08.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.363+0000 7f0f8f7fe700 1 --2- 172.21.15.31:0/870546536 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0f7c067830 0x7f0f7c069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:08.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.364+0000 7f0f8f7fe700 1 --2- 172.21.15.31:0/870546536 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0f7c067830 0x7f0f7c069ce0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f0f800091e0 tx=0x7f0f80017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:08.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.367+0000 7f0f777fe700 1 -- 172.21.15.31:0/870546536 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0f880523f0 con 0x7f0f9015de50 2023-10-06T02:32:08.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.572+0000 7f0f94d9d700 1 -- 172.21.15.31:0/870546536 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0f9015bb50 con 0x7f0f9015de50 2023-10-06T02:32:08.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.573+0000 7f0f777fe700 1 -- 172.21.15.31:0/870546536 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v168) v1 ==== 74+0+16889 (secure 0 0 0) 0x7f0f88018090 con 0x7f0f9015de50 2023-10-06T02:32:08.574 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:32:08.574 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":168,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:32:07.613045+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:32:05.275284+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"168","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":28,"pool_name":"cephfs_data","create_time":"2023-10-06T02:32:06.591889+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"168","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":20,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:32:08.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.575+0000 7f0f94d9d700 1 -- 172.21.15.31:0/870546536 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0f7c067830 msgr2=0x7f0f7c069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:08.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.575+0000 7f0f94d9d700 1 --2- 172.21.15.31:0/870546536 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0f7c067830 0x7f0f7c069ce0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f0f800091e0 tx=0x7f0f80017000 comp rx=0 tx=0).stop 2023-10-06T02:32:08.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.575+0000 7f0f94d9d700 1 -- 172.21.15.31:0/870546536 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f9015de50 msgr2=0x7f0f900b1d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:08.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.575+0000 7f0f94d9d700 1 --2- 172.21.15.31:0/870546536 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f9015de50 0x7f0f900b1d90 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f0f88009fc0 tx=0x7f0f8800b040 comp rx=0 tx=0).stop 2023-10-06T02:32:08.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.576+0000 7f0f94d9d700 1 -- 172.21.15.31:0/870546536 shutdown_connections 2023-10-06T02:32:08.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.576+0000 7f0f94d9d700 1 --2- 172.21.15.31:0/870546536 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f9015de50 0x7f0f900b1d90 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:08.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.576+0000 7f0f94d9d700 1 --2- 172.21.15.31:0/870546536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0f900cf110 0x7f0f900b32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:08.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.576+0000 7f0f94d9d700 1 --2- 172.21.15.31:0/870546536 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f0f7c067830 0x7f0f7c069ce0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:08.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.576+0000 7f0f94d9d700 1 --2- 172.21.15.31:0/870546536 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0f90153c30 0x7f0f900c7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:08.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.576+0000 7f0f94d9d700 1 -- 172.21.15.31:0/870546536 >> 172.21.15.31:0/870546536 conn(0x7f0f900703a0 msgr2=0x7f0f90070a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:08.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.576+0000 7f0f94d9d700 1 -- 172.21.15.31:0/870546536 shutdown_connections 2023-10-06T02:32:08.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.576+0000 7f0f94d9d700 1 -- 172.21.15.31:0/870546536 wait complete. 2023-10-06T02:32:08.594 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:32:08.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.780+0000 7facc4d9d700 1 Processor -- start 2023-10-06T02:32:08.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.780+0000 7facc4d9d700 1 -- start start 2023-10-06T02:32:08.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.782+0000 7facc4d9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7facc0159aa0 0x7facc00703a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:08.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.782+0000 7facc4d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7facc00ca7b0 0x7facc00cac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:08.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.782+0000 7facc4d9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facc006d3e0 con 0x7facc01596d0 2023-10-06T02:32:08.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.782+0000 7facc4d9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facc00cb170 con 0x7facc0159aa0 2023-10-06T02:32:08.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.782+0000 7facc4d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facc006c7d0 con 0x7facc00ca7b0 2023-10-06T02:32:08.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.782+0000 7facbf7fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7facc01596d0 0x7facc006d870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:47714/0 (socket says 172.21.15.31:47714) 2023-10-06T02:32:08.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.782+0000 7facbf7fe700 1 -- 172.21.15.31:0/4159132512 learned_addr learned my addr 172.21.15.31:0/4159132512 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:32:08.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.782+0000 7facbeffd700 1 --2- 172.21.15.31:0/4159132512 >> v2:172.21.15.31:3300/0 conn(0x7facc0159aa0 0x7facc00703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:08.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.782+0000 7facbffff700 1 --2- 172.21.15.31:0/4159132512 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7facc00ca7b0 0x7facc00cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:08.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.782+0000 7facbe7fc700 1 -- 172.21.15.31:0/4159132512 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3124103381 0 0) 0x7facc006d3e0 con 0x7facc01596d0 2023-10-06T02:32:08.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.783+0000 7facbe7fc700 1 -- 172.21.15.31:0/4159132512 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faca8003570 con 0x7facc01596d0 2023-10-06T02:32:08.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.783+0000 7facbeffd700 1 -- 172.21.15.31:0/4159132512 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7facc00ca7b0 msgr2=0x7facc00cac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:08.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.783+0000 7facbeffd700 1 --2- 172.21.15.31:0/4159132512 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7facc00ca7b0 0x7facc00cac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:08.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.783+0000 7facbeffd700 1 -- 172.21.15.31:0/4159132512 >> v1:172.21.15.31:6789/0 conn(0x7facc01596d0 legacy=0x7facc006d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:08.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.783+0000 7facbeffd700 1 -- 172.21.15.31:0/4159132512 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facc006cab0 con 0x7facc0159aa0 2023-10-06T02:32:08.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.783+0000 7facbeffd700 1 --2- 172.21.15.31:0/4159132512 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facc0159aa0 0x7facc00703a0 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7facb40098e0 tx=0x7facb40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba210094b0d8b2f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:08.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.783+0000 7facbe7fc700 1 -- 172.21.15.31:0/4159132512 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb4011630 con 0x7facc0159aa0 2023-10-06T02:32:08.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.783+0000 7facbe7fc700 1 -- 172.21.15.31:0/4159132512 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facb4011790 con 0x7facc0159aa0 2023-10-06T02:32:08.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.783+0000 7facbe7fc700 1 -- 172.21.15.31:0/4159132512 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb4011b30 con 0x7facc0159aa0 2023-10-06T02:32:08.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.784+0000 7facc4d9d700 1 -- 172.21.15.31:0/4159132512 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facc0159aa0 msgr2=0x7facc00703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:08.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.784+0000 7facc4d9d700 1 --2- 172.21.15.31:0/4159132512 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facc0159aa0 0x7facc00703a0 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7facb40098e0 tx=0x7facb40091a0 comp rx=0 tx=0).stop 2023-10-06T02:32:08.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.784+0000 7facc4d9d700 1 -- 172.21.15.31:0/4159132512 shutdown_connections 2023-10-06T02:32:08.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.784+0000 7facc4d9d700 1 --2- 172.21.15.31:0/4159132512 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7facc00ca7b0 0x7facc00cac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:08.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.784+0000 7facc4d9d700 1 --2- 172.21.15.31:0/4159132512 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facc0159aa0 0x7facc00703a0 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:08.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.784+0000 7facc4d9d700 1 -- 172.21.15.31:0/4159132512 >> 172.21.15.31:0/4159132512 conn(0x7facc006e720 msgr2=0x7facc006eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:08.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.784+0000 7facc4d9d700 1 -- 172.21.15.31:0/4159132512 shutdown_connections 2023-10-06T02:32:08.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.784+0000 7facc4d9d700 1 -- 172.21.15.31:0/4159132512 wait complete. 2023-10-06T02:32:08.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.785+0000 7facc4d9d700 1 Processor -- start 2023-10-06T02:32:08.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.786+0000 7facc4d9d700 1 -- start start 2023-10-06T02:32:08.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.786+0000 7facc4d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7facc00ca7b0 0x7facc00b2020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:08.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.786+0000 7facc4d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7facc01596d0 0x7facc0157b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:08.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.786+0000 7facc4d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facc0159aa0 0x7facc00af7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:08.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.786+0000 7facc4d9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc00c5180 con 0x7facc0159aa0 2023-10-06T02:32:08.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.786+0000 7facc4d9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc00c4570 con 0x7facc01596d0 2023-10-06T02:32:08.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.786+0000 7facc4d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facc00c4850 con 0x7facc00ca7b0 2023-10-06T02:32:08.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.787+0000 7facbffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facc0159aa0 0x7facc00af7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:08.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.787+0000 7facbffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facc0159aa0 0x7facc00af7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41006/0 (socket says 172.21.15.31:41006) 2023-10-06T02:32:08.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.787+0000 7facbffff700 1 -- 172.21.15.31:0/961037733 learned_addr learned my addr 172.21.15.31:0/961037733 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:08.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.787+0000 7facbf7fe700 1 --2- 172.21.15.31:0/961037733 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7facc00ca7b0 0x7facc00b2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:08.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.787+0000 7facbeffd700 1 --2- 172.21.15.31:0/961037733 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7facc01596d0 0x7facc0157b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:08.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.787+0000 7facbffff700 1 -- 172.21.15.31:0/961037733 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7facc00ca7b0 msgr2=0x7facc00b2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:08.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.787+0000 7facbffff700 1 --2- 172.21.15.31:0/961037733 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7facc00ca7b0 0x7facc00b2020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:08.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.787+0000 7facbffff700 1 -- 172.21.15.31:0/961037733 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7facc01596d0 msgr2=0x7facc0157b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:08.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.787+0000 7facbffff700 1 --2- 172.21.15.31:0/961037733 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7facc01596d0 0x7facc0157b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:08.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.787+0000 7facbffff700 1 -- 172.21.15.31:0/961037733 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facb4009420 con 0x7facc0159aa0 2023-10-06T02:32:08.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.787+0000 7facbffff700 1 --2- 172.21.15.31:0/961037733 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facc0159aa0 0x7facc00af7f0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7facb8009e90 tx=0x7facb800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:08.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.787+0000 7faca77fe700 1 -- 172.21.15.31:0/961037733 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb800cc20 con 0x7facc0159aa0 2023-10-06T02:32:08.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.788+0000 7faca77fe700 1 -- 172.21.15.31:0/961037733 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facb800f040 con 0x7facc0159aa0 2023-10-06T02:32:08.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.788+0000 7facc4d9d700 1 -- 172.21.15.31:0/961037733 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facc00c4b30 con 0x7facc0159aa0 2023-10-06T02:32:08.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.788+0000 7faca77fe700 1 -- 172.21.15.31:0/961037733 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb8014480 con 0x7facc0159aa0 2023-10-06T02:32:08.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.788+0000 7facc4d9d700 1 -- 172.21.15.31:0/961037733 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facc00b2560 con 0x7facc0159aa0 2023-10-06T02:32:08.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.789+0000 7facc4d9d700 1 -- 172.21.15.31:0/961037733 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facc004c3c0 con 0x7facc0159aa0 2023-10-06T02:32:08.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.790+0000 7faca77fe700 1 -- 172.21.15.31:0/961037733 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85556+0+0 (secure 0 0 0) 0x7facb8014830 con 0x7facc0159aa0 2023-10-06T02:32:08.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.791+0000 7faca77fe700 1 --2- 172.21.15.31:0/961037733 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7facac067830 0x7facac069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:08.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.791+0000 7faca77fe700 1 -- 172.21.15.31:0/961037733 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 6086+0+0 (secure 0 0 0) 0x7facb80867f0 con 0x7facc0159aa0 2023-10-06T02:32:08.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.791+0000 7facbf7fe700 1 --2- 172.21.15.31:0/961037733 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7facac067830 0x7facac069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:08.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.792+0000 7facbf7fe700 1 --2- 172.21.15.31:0/961037733 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7facac067830 0x7facac069ce0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7facb00058d0 tx=0x7facb0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:08.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.794+0000 7faca77fe700 1 -- 172.21.15.31:0/961037733 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7facb80567d0 con 0x7facc0159aa0 2023-10-06T02:32:08.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.998+0000 7facc4d9d700 1 -- 172.21.15.31:0/961037733 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7facc00c52c0 con 0x7facc0159aa0 2023-10-06T02:32:08.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:08.999+0000 7faca77fe700 1 -- 172.21.15.31:0/961037733 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v168) v1 ==== 74+0+16889 (secure 0 0 0) 0x7facb8018070 con 0x7facc0159aa0 2023-10-06T02:32:08.999 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:32:09.000 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":168,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:32:07.613045+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:32:05.275284+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"168","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":28,"pool_name":"cephfs_data","create_time":"2023-10-06T02:32:06.591889+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"168","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":20,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:32:09.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:09.001+0000 7facc4d9d700 1 -- 172.21.15.31:0/961037733 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7facac067830 msgr2=0x7facac069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:09.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:09.001+0000 7facc4d9d700 1 --2- 172.21.15.31:0/961037733 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7facac067830 0x7facac069ce0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7facb00058d0 tx=0x7facb0009040 comp rx=0 tx=0).stop 2023-10-06T02:32:09.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:09.001+0000 7facc4d9d700 1 -- 172.21.15.31:0/961037733 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facc0159aa0 msgr2=0x7facc00af7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:09.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:09.001+0000 7facc4d9d700 1 --2- 172.21.15.31:0/961037733 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facc0159aa0 0x7facc00af7f0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7facb8009e90 tx=0x7facb800b040 comp rx=0 tx=0).stop 2023-10-06T02:32:09.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:09.001+0000 7facc4d9d700 1 -- 172.21.15.31:0/961037733 shutdown_connections 2023-10-06T02:32:09.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:09.001+0000 7facc4d9d700 1 --2- 172.21.15.31:0/961037733 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facc0159aa0 0x7facc00af7f0 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:09.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:09.001+0000 7facc4d9d700 1 --2- 172.21.15.31:0/961037733 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7facac067830 0x7facac069ce0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:09.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:09.001+0000 7facc4d9d700 1 --2- 172.21.15.31:0/961037733 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7facc00ca7b0 0x7facc00b2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:09.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:09.001+0000 7facc4d9d700 1 --2- 172.21.15.31:0/961037733 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7facc01596d0 0x7facc0157b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:09.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:09.001+0000 7facc4d9d700 1 -- 172.21.15.31:0/961037733 >> 172.21.15.31:0/961037733 conn(0x7facc006e720 msgr2=0x7facc015b490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:09.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:09.001+0000 7facc4d9d700 1 -- 172.21.15.31:0/961037733 shutdown_connections 2023-10-06T02:32:09.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:09.001+0000 7facc4d9d700 1 -- 172.21.15.31:0/961037733 wait complete. 2023-10-06T02:32:09.020 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T02:32:09.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:09.212+0000 7f1e7599d700 1 Processor -- start 2023-10-06T02:32:09.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:09.212+0000 7f1e7599d700 1 -- start start 2023-10-06T02:32:09.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:09.214+0000 7f1e7599d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e70153a00 0x7f1e70153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:09.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:09.214+0000 7f1e7599d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1e700cea00 0x7f1e700cee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:09.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:09.214+0000 7f1e7599d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e7015ac50 con 0x7f1e7015de50 2023-10-06T02:32:09.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:09.214+0000 7f1e7599d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e7015d010 con 0x7f1e700cea00 2023-10-06T02:32:09.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:09.214+0000 7f1e7599d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e7015af60 con 0x7f1e70153a00 2023-10-06T02:32:09.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:09.214+0000 7f1e7519c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1e700cea00 0x7f1e700cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:09.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:09.214+0000 7f1e7519c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1e700cea00 0x7f1e700cee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41018/0 (socket says 172.21.15.31:41018) 2023-10-06T02:32:09.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:09.214+0000 7f1e7519c700 1 -- 172.21.15.31:0/4256983943 learned_addr learned my addr 172.21.15.31:0/4256983943 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:09.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:09.214+0000 7f1e7499b700 1 --2- 172.21.15.31:0/4256983943 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e70153a00 0x7f1e70153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:09.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:09.215+0000 7f1e7519c700 1 -- 172.21.15.31:0/4256983943 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e70153a00 msgr2=0x7f1e70153dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:09.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:09.215+0000 7f1e7519c700 1 --2- 172.21.15.31:0/4256983943 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e70153a00 0x7f1e70153dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:09.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:09.215+0000 7f1e7519c700 1 -- 172.21.15.31:0/4256983943 >> v1:172.21.15.62:6789/0 conn(0x7f1e7015de50 legacy=0x7f1e7015a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:09.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:09.215+0000 7f1e7519c700 1 -- 172.21.15.31:0/4256983943 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e7015b240 con 0x7f1e700cea00 2023-10-06T02:32:09.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:09.215+0000 7f1e7519c700 1 --2- 172.21.15.31:0/4256983943 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e700cea00 0x7f1e700cee40 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f1e68009f20 tx=0x7f1e68009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19785297ef0c03b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:09.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:09.215+0000 7f1e6f7fe700 1 -- 172.21.15.31:0/4256983943 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e6800e070 con 0x7f1e700cea00 2023-10-06T02:32:09.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:09.215+0000 7f1e6f7fe700 1 -- 172.21.15.31:0/4256983943 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e6800b9d0 con 0x7f1e700cea00 2023-10-06T02:32:09.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:09.215+0000 7f1e6f7fe700 1 -- 172.21.15.31:0/4256983943 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e680092e0 con 0x7f1e700cea00 2023-10-06T02:32:09.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:09.216+0000 7f1e7599d700 1 -- 172.21.15.31:0/4256983943 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e700cea00 msgr2=0x7f1e700cee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:09.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:09.216+0000 7f1e7599d700 1 --2- 172.21.15.31:0/4256983943 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e700cea00 0x7f1e700cee40 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f1e68009f20 tx=0x7f1e68009840 comp rx=0 tx=0).stop 2023-10-06T02:32:09.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:09.216+0000 7f1e7599d700 1 -- 172.21.15.31:0/4256983943 shutdown_connections 2023-10-06T02:32:09.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:09.216+0000 7f1e7599d700 1 --2- 172.21.15.31:0/4256983943 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e700cea00 0x7f1e700cee40 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:09.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:09.216+0000 7f1e7599d700 1 --2- 172.21.15.31:0/4256983943 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e70153a00 0x7f1e70153dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:09.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:09.216+0000 7f1e7599d700 1 -- 172.21.15.31:0/4256983943 >> 172.21.15.31:0/4256983943 conn(0x7f1e700703a0 msgr2=0x7f1e700707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:09.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:09.216+0000 7f1e7599d700 1 -- 172.21.15.31:0/4256983943 shutdown_connections 2023-10-06T02:32:09.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:09.216+0000 7f1e7599d700 1 -- 172.21.15.31:0/4256983943 wait complete. 2023-10-06T02:32:09.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:09.217+0000 7f1e7599d700 1 Processor -- start 2023-10-06T02:32:09.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:09.217+0000 7f1e7599d700 1 -- start start 2023-10-06T02:32:09.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:09.218+0000 7f1e7599d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e700cea00 0x7f1e70152190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:09.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:09.218+0000 7f1e7599d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e70153a00 0x7f1e70150aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:09.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:09.218+0000 7f1e7599d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e7015de50 0x7f1e700c7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:09.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:09.218+0000 7f1e7599d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e700c8820 con 0x7f1e700cea00 2023-10-06T02:32:09.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:09.218+0000 7f1e7599d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e700c8b00 con 0x7f1e70153a00 2023-10-06T02:32:09.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:09.218+0000 7f1e7599d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e700c8de0 con 0x7f1e7015de50 2023-10-06T02:32:09.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:09.218+0000 7f1e7499b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e700cea00 0x7f1e70152190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:09.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:09.218+0000 7f1e7499b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e700cea00 0x7f1e70152190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41026/0 (socket says 172.21.15.31:41026) 2023-10-06T02:32:09.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:09.218+0000 7f1e7499b700 1 -- 172.21.15.31:0/3259750082 learned_addr learned my addr 172.21.15.31:0/3259750082 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:09.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:09.219+0000 7f1e7499b700 1 -- 172.21.15.31:0/3259750082 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e7015de50 msgr2=0x7f1e700c7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:09.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:09.219+0000 7f1e7499b700 1 --2- 172.21.15.31:0/3259750082 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e7015de50 0x7f1e700c7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:09.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:09.219+0000 7f1e7499b700 1 -- 172.21.15.31:0/3259750082 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e70153a00 msgr2=0x7f1e70150aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:09.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:09.219+0000 7f1e6ffff700 1 --2- 172.21.15.31:0/3259750082 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e70153a00 0x7f1e70150aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:09.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:09.219+0000 7f1e7499b700 1 --2- 172.21.15.31:0/3259750082 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e70153a00 0x7f1e70150aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:09.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:09.219+0000 7f1e7499b700 1 -- 172.21.15.31:0/3259750082 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e68009ae0 con 0x7f1e700cea00 2023-10-06T02:32:09.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:09.219+0000 7f1e6ffff700 1 --2- 172.21.15.31:0/3259750082 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e70153a00 0x7f1e70150aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:32:09.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:09.219+0000 7f1e7499b700 1 --2- 172.21.15.31:0/3259750082 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e700cea00 0x7f1e70152190 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f1e6400ec80 tx=0x7f1e640091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:09.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:09.219+0000 7f1e6cff9700 1 -- 172.21.15.31:0/3259750082 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e6400c9d0 con 0x7f1e700cea00 2023-10-06T02:32:09.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:09.219+0000 7f1e6cff9700 1 -- 172.21.15.31:0/3259750082 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e6400cb30 con 0x7f1e700cea00 2023-10-06T02:32:09.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:09.219+0000 7f1e6cff9700 1 -- 172.21.15.31:0/3259750082 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e64013470 con 0x7f1e700cea00 2023-10-06T02:32:09.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:09.219+0000 7f1e7599d700 1 -- 172.21.15.31:0/3259750082 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e700c90c0 con 0x7f1e700cea00 2023-10-06T02:32:09.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:09.220+0000 7f1e7599d700 1 -- 172.21.15.31:0/3259750082 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e700b33e0 con 0x7f1e700cea00 2023-10-06T02:32:09.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:09.221+0000 7f1e7599d700 1 -- 172.21.15.31:0/3259750082 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e7004c3c0 con 0x7f1e700cea00 2023-10-06T02:32:09.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:09.221+0000 7f1e6cff9700 1 -- 172.21.15.31:0/3259750082 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85556+0+0 (secure 0 0 0) 0x7f1e64013820 con 0x7f1e700cea00 2023-10-06T02:32:09.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:09.222+0000 7f1e6cff9700 1 --2- 172.21.15.31:0/3259750082 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f1e54067830 0x7f1e54069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:09.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:09.222+0000 7f1e6cff9700 1 -- 172.21.15.31:0/3259750082 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f1e640833f0 con 0x7f1e700cea00 2023-10-06T02:32:09.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:09.223+0000 7f1e6ffff700 1 --2- 172.21.15.31:0/3259750082 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f1e54067830 0x7f1e54069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:09.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:09.224+0000 7f1e6ffff700 1 --2- 172.21.15.31:0/3259750082 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f1e54067830 0x7f1e54069ce0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f1e7015b4a0 tx=0x7f1e5c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:09.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:09.227+0000 7f1e6cff9700 1 -- 172.21.15.31:0/3259750082 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1e6408b020 con 0x7f1e700cea00 2023-10-06T02:32:09.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:09.475+0000 7f1e7599d700 1 -- 172.21.15.31:0/3259750082 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f1e7015b520 con 0x7f1e700cea00 2023-10-06T02:32:10.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:10.144+0000 7f1e6cff9700 1 -- 172.21.15.31:0/3259750082 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v118) v1 ==== 118+0+0 (secure 0 0 0) 0x7f1e640170b0 con 0x7f1e700cea00 2023-10-06T02:32:10.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:10.147+0000 7f1e7599d700 1 -- 172.21.15.31:0/3259750082 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f1e54067830 msgr2=0x7f1e54069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:10.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:10.147+0000 7f1e7599d700 1 --2- 172.21.15.31:0/3259750082 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f1e54067830 0x7f1e54069ce0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f1e7015b4a0 tx=0x7f1e5c009040 comp rx=0 tx=0).stop 2023-10-06T02:32:10.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:10.147+0000 7f1e7599d700 1 -- 172.21.15.31:0/3259750082 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e700cea00 msgr2=0x7f1e70152190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:10.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:10.147+0000 7f1e7599d700 1 --2- 172.21.15.31:0/3259750082 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e700cea00 0x7f1e70152190 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f1e6400ec80 tx=0x7f1e640091a0 comp rx=0 tx=0).stop 2023-10-06T02:32:10.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:10.147+0000 7f1e7599d700 1 -- 172.21.15.31:0/3259750082 shutdown_connections 2023-10-06T02:32:10.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:10.147+0000 7f1e7599d700 1 --2- 172.21.15.31:0/3259750082 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f1e54067830 0x7f1e54069ce0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:10.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:10.147+0000 7f1e7599d700 1 --2- 172.21.15.31:0/3259750082 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e7015de50 0x7f1e700c7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:10.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:10.147+0000 7f1e7599d700 1 --2- 172.21.15.31:0/3259750082 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e700cea00 0x7f1e70152190 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:10.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:10.147+0000 7f1e7599d700 1 --2- 172.21.15.31:0/3259750082 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e70153a00 0x7f1e70150aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:10.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:10.147+0000 7f1e7599d700 1 -- 172.21.15.31:0/3259750082 >> 172.21.15.31:0/3259750082 conn(0x7f1e700703a0 msgr2=0x7f1e700d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:10.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:10.148+0000 7f1e7599d700 1 -- 172.21.15.31:0/3259750082 shutdown_connections 2023-10-06T02:32:10.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:10.148+0000 7f1e7599d700 1 -- 172.21.15.31:0/3259750082 wait complete. 2023-10-06T02:32:10.169 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T02:32:10.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:10.363+0000 7fb36959d700 1 Processor -- start 2023-10-06T02:32:10.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:10.364+0000 7fb36959d700 1 -- start start 2023-10-06T02:32:10.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:10.365+0000 7fb36959d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb364155890 0x7fb3640c7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:10.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:10.365+0000 7fb36959d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb36406de50 0x7fb36406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:10.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:10.365+0000 7fb36959d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3640c7e70 con 0x7fb364155c60 2023-10-06T02:32:10.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:10.365+0000 7fb36959d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb36406e830 con 0x7fb36406de50 2023-10-06T02:32:10.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:10.365+0000 7fb36959d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb36406b790 con 0x7fb364155890 2023-10-06T02:32:10.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:10.365+0000 7fb368d9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb36406de50 0x7fb36406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:10.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:10.365+0000 7fb368d9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb36406de50 0x7fb36406e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41028/0 (socket says 172.21.15.31:41028) 2023-10-06T02:32:10.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:10.365+0000 7fb368d9c700 1 -- 172.21.15.31:0/680787695 learned_addr learned my addr 172.21.15.31:0/680787695 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:10.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:10.365+0000 7fb363fff700 1 --2- 172.21.15.31:0/680787695 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb364155890 0x7fb3640c7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:10.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:10.366+0000 7fb368d9c700 1 -- 172.21.15.31:0/680787695 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb364155890 msgr2=0x7fb3640c7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:10.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:10.366+0000 7fb368d9c700 1 --2- 172.21.15.31:0/680787695 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb364155890 0x7fb3640c7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:10.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:10.366+0000 7fb368d9c700 1 -- 172.21.15.31:0/680787695 >> v1:172.21.15.62:6789/0 conn(0x7fb364155c60 legacy=0x7fb36406a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:10.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:10.366+0000 7fb368d9c700 1 -- 172.21.15.31:0/680787695 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb36406ab80 con 0x7fb36406de50 2023-10-06T02:32:10.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:10.366+0000 7fb368d9c700 1 --2- 172.21.15.31:0/680787695 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb36406de50 0x7fb36406e2f0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7fb35c009f20 tx=0x7fb35c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c79862c22c992f4d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:10.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:10.366+0000 7fb362ffd700 1 -- 172.21.15.31:0/680787695 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb35c00e070 con 0x7fb36406de50 2023-10-06T02:32:10.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:10.366+0000 7fb362ffd700 1 -- 172.21.15.31:0/680787695 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb35c00b9d0 con 0x7fb36406de50 2023-10-06T02:32:10.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:10.366+0000 7fb362ffd700 1 -- 172.21.15.31:0/680787695 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb35c0092e0 con 0x7fb36406de50 2023-10-06T02:32:10.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:10.367+0000 7fb36959d700 1 -- 172.21.15.31:0/680787695 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb36406de50 msgr2=0x7fb36406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:10.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:10.367+0000 7fb36959d700 1 --2- 172.21.15.31:0/680787695 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb36406de50 0x7fb36406e2f0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7fb35c009f20 tx=0x7fb35c009840 comp rx=0 tx=0).stop 2023-10-06T02:32:10.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:10.367+0000 7fb36959d700 1 -- 172.21.15.31:0/680787695 shutdown_connections 2023-10-06T02:32:10.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:10.367+0000 7fb36959d700 1 --2- 172.21.15.31:0/680787695 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb36406de50 0x7fb36406e2f0 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:10.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:10.367+0000 7fb36959d700 1 --2- 172.21.15.31:0/680787695 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb364155890 0x7fb3640c7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:10.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:10.367+0000 7fb36959d700 1 -- 172.21.15.31:0/680787695 >> 172.21.15.31:0/680787695 conn(0x7fb36406d110 msgr2=0x7fb3640cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:10.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:10.367+0000 7fb36959d700 1 -- 172.21.15.31:0/680787695 shutdown_connections 2023-10-06T02:32:10.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:10.367+0000 7fb36959d700 1 -- 172.21.15.31:0/680787695 wait complete. 2023-10-06T02:32:10.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:10.368+0000 7fb36959d700 1 Processor -- start 2023-10-06T02:32:10.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:10.369+0000 7fb36959d700 1 -- start start 2023-10-06T02:32:10.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:10.370+0000 7fb36959d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb36406de50 0x7fb3640c3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:10.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:10.370+0000 7fb36959d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb364155890 0x7fb3640a9950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:10.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:10.370+0000 7fb36959d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb364155c60 0x7fb3641467a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:10.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:10.370+0000 7fb36959d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3640c40e0 con 0x7fb364155890 2023-10-06T02:32:10.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:10.370+0000 7fb36959d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3640a9e90 con 0x7fb36406de50 2023-10-06T02:32:10.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:10.370+0000 7fb36959d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3640c4cc0 con 0x7fb364155c60 2023-10-06T02:32:10.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:10.370+0000 7fb3637fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb364155890 0x7fb3640a9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:10.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:10.370+0000 7fb3637fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb364155890 0x7fb3640a9950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41042/0 (socket says 172.21.15.31:41042) 2023-10-06T02:32:10.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:10.370+0000 7fb3637fe700 1 -- 172.21.15.31:0/248417403 learned_addr learned my addr 172.21.15.31:0/248417403 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:10.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:10.370+0000 7fb3637fe700 1 -- 172.21.15.31:0/248417403 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb364155c60 msgr2=0x7fb3641467a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:10.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:10.370+0000 7fb3637fe700 1 --2- 172.21.15.31:0/248417403 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb364155c60 0x7fb3641467a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:10.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:10.370+0000 7fb3637fe700 1 -- 172.21.15.31:0/248417403 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb36406de50 msgr2=0x7fb3640c3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:10.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:10.370+0000 7fb3637fe700 1 --2- 172.21.15.31:0/248417403 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb36406de50 0x7fb3640c3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:10.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:10.370+0000 7fb3637fe700 1 -- 172.21.15.31:0/248417403 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb35c009ae0 con 0x7fb364155890 2023-10-06T02:32:10.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:10.370+0000 7fb3637fe700 1 --2- 172.21.15.31:0/248417403 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb364155890 0x7fb3640a9950 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7fb35800c590 tx=0x7fb358007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:10.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:10.371+0000 7fb34bfff700 1 -- 172.21.15.31:0/248417403 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb358011590 con 0x7fb364155890 2023-10-06T02:32:10.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:10.371+0000 7fb34bfff700 1 -- 172.21.15.31:0/248417403 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3580116f0 con 0x7fb364155890 2023-10-06T02:32:10.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:10.371+0000 7fb34bfff700 1 -- 172.21.15.31:0/248417403 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb358011ab0 con 0x7fb364155890 2023-10-06T02:32:10.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:10.371+0000 7fb36959d700 1 -- 172.21.15.31:0/248417403 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3640c4fa0 con 0x7fb364155890 2023-10-06T02:32:10.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:10.371+0000 7fb36959d700 1 -- 172.21.15.31:0/248417403 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3640c5560 con 0x7fb364155890 2023-10-06T02:32:10.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:10.372+0000 7fb36959d700 1 -- 172.21.15.31:0/248417403 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3640c5780 con 0x7fb364155890 2023-10-06T02:32:10.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:10.373+0000 7fb34bfff700 1 -- 172.21.15.31:0/248417403 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85556+0+0 (secure 0 0 0) 0x7fb35801a360 con 0x7fb364155890 2023-10-06T02:32:10.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:10.374+0000 7fb34bfff700 1 --2- 172.21.15.31:0/248417403 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb350067830 0x7fb350069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:10.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:10.374+0000 7fb34bfff700 1 -- 172.21.15.31:0/248417403 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fb358084830 con 0x7fb364155890 2023-10-06T02:32:10.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:10.375+0000 7fb363fff700 1 --2- 172.21.15.31:0/248417403 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb350067830 0x7fb350069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:10.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:10.375+0000 7fb363fff700 1 --2- 172.21.15.31:0/248417403 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb350067830 0x7fb350069ce0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fb354009bb0 tx=0x7fb3540091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:10.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:10.378+0000 7fb34bfff700 1 -- 172.21.15.31:0/248417403 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb358050a80 con 0x7fb364155890 2023-10-06T02:32:10.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:10.629+0000 7fb36959d700 1 -- 172.21.15.31:0/248417403 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fb3640c4a10 con 0x7fb364155890 2023-10-06T02:32:11.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.150+0000 7fb34bfff700 1 -- 172.21.15.31:0/248417403 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v119) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb3580170c0 con 0x7fb364155890 2023-10-06T02:32:11.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.153+0000 7fb36959d700 1 -- 172.21.15.31:0/248417403 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb350067830 msgr2=0x7fb350069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:11.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.153+0000 7fb36959d700 1 --2- 172.21.15.31:0/248417403 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb350067830 0x7fb350069ce0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fb354009bb0 tx=0x7fb3540091a0 comp rx=0 tx=0).stop 2023-10-06T02:32:11.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.153+0000 7fb36959d700 1 -- 172.21.15.31:0/248417403 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb364155890 msgr2=0x7fb3640a9950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:11.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.153+0000 7fb36959d700 1 --2- 172.21.15.31:0/248417403 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb364155890 0x7fb3640a9950 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7fb35800c590 tx=0x7fb358007f50 comp rx=0 tx=0).stop 2023-10-06T02:32:11.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.153+0000 7fb36959d700 1 -- 172.21.15.31:0/248417403 shutdown_connections 2023-10-06T02:32:11.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.153+0000 7fb36959d700 1 --2- 172.21.15.31:0/248417403 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb350067830 0x7fb350069ce0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:11.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.153+0000 7fb36959d700 1 --2- 172.21.15.31:0/248417403 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb364155c60 0x7fb3641467a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:11.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.153+0000 7fb36959d700 1 --2- 172.21.15.31:0/248417403 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb36406de50 0x7fb3640c3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:11.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.153+0000 7fb36959d700 1 --2- 172.21.15.31:0/248417403 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb364155890 0x7fb3640a9950 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:11.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.153+0000 7fb36959d700 1 -- 172.21.15.31:0/248417403 >> 172.21.15.31:0/248417403 conn(0x7fb36406d110 msgr2=0x7fb3640cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:11.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.153+0000 7fb36959d700 1 -- 172.21.15.31:0/248417403 shutdown_connections 2023-10-06T02:32:11.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.153+0000 7fb36959d700 1 -- 172.21.15.31:0/248417403 wait complete. 2023-10-06T02:32:11.175 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:32:11.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.368+0000 7fa0dffff700 1 Processor -- start 2023-10-06T02:32:11.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.368+0000 7fa0dffff700 1 -- start start 2023-10-06T02:32:11.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.369+0000 7fa0dffff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa0e006e0a0 0x7fa0e00c6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:11.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.369+0000 7fa0dffff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa0e0150380 0x7fa0e006a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:11.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.369+0000 7fa0dffff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0e006a490 con 0x7fa0e006f3a0 2023-10-06T02:32:11.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.369+0000 7fa0dffff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0e00c7490 con 0x7fa0e0150380 2023-10-06T02:32:11.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.369+0000 7fa0dffff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e00c7770 con 0x7fa0e006e0a0 2023-10-06T02:32:11.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.369+0000 7fa0df7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa0e0150380 0x7fa0e006a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:11.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.370+0000 7fa0df7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa0e0150380 0x7fa0e006a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41056/0 (socket says 172.21.15.31:41056) 2023-10-06T02:32:11.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.370+0000 7fa0df7fe700 1 -- 172.21.15.31:0/4178511274 learned_addr learned my addr 172.21.15.31:0/4178511274 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:11.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.370+0000 7fa0deffd700 1 --2- 172.21.15.31:0/4178511274 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa0e006e0a0 0x7fa0e00c6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:11.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.370+0000 7fa0df7fe700 1 -- 172.21.15.31:0/4178511274 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa0e006e0a0 msgr2=0x7fa0e00c6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:11.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.370+0000 7fa0df7fe700 1 --2- 172.21.15.31:0/4178511274 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa0e006e0a0 0x7fa0e00c6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:11.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.370+0000 7fa0df7fe700 1 -- 172.21.15.31:0/4178511274 >> v1:172.21.15.62:6789/0 conn(0x7fa0e006f3a0 legacy=0x7fa0e014fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:11.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.370+0000 7fa0df7fe700 1 -- 172.21.15.31:0/4178511274 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0e00c7a50 con 0x7fa0e0150380 2023-10-06T02:32:11.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.370+0000 7fa0ddffb700 1 -- 172.21.15.31:0/4178511274 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 312100469 0 0) 0x7fa0e006a490 con 0x7fa0e006f3a0 2023-10-06T02:32:11.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.370+0000 7fa0df7fe700 1 --2- 172.21.15.31:0/4178511274 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0e0150380 0x7fa0e006a920 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7fa0d8009f20 tx=0x7fa0d8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc734c947986eeeb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:11.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.370+0000 7fa0ddffb700 1 -- 172.21.15.31:0/4178511274 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d800e070 con 0x7fa0e0150380 2023-10-06T02:32:11.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.371+0000 7fa0ddffb700 1 -- 172.21.15.31:0/4178511274 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0d800b9d0 con 0x7fa0e0150380 2023-10-06T02:32:11.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.371+0000 7fa0ddffb700 1 -- 172.21.15.31:0/4178511274 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d80092e0 con 0x7fa0e0150380 2023-10-06T02:32:11.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.371+0000 7fa0dffff700 1 -- 172.21.15.31:0/4178511274 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0e0150380 msgr2=0x7fa0e006a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:11.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.371+0000 7fa0dffff700 1 --2- 172.21.15.31:0/4178511274 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0e0150380 0x7fa0e006a920 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7fa0d8009f20 tx=0x7fa0d8009840 comp rx=0 tx=0).stop 2023-10-06T02:32:11.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.371+0000 7fa0dffff700 1 -- 172.21.15.31:0/4178511274 shutdown_connections 2023-10-06T02:32:11.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.371+0000 7fa0dffff700 1 --2- 172.21.15.31:0/4178511274 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0e0150380 0x7fa0e006a920 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:11.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.371+0000 7fa0dffff700 1 --2- 172.21.15.31:0/4178511274 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa0e006e0a0 0x7fa0e00c6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:11.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.371+0000 7fa0dffff700 1 -- 172.21.15.31:0/4178511274 >> 172.21.15.31:0/4178511274 conn(0x7fa0e006d170 msgr2=0x7fa0e00cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:11.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.371+0000 7fa0dffff700 1 -- 172.21.15.31:0/4178511274 shutdown_connections 2023-10-06T02:32:11.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.371+0000 7fa0dffff700 1 -- 172.21.15.31:0/4178511274 wait complete. 2023-10-06T02:32:11.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.373+0000 7fa0dffff700 1 Processor -- start 2023-10-06T02:32:11.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.373+0000 7fa0dffff700 1 -- start start 2023-10-06T02:32:11.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.374+0000 7fa0dffff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa0e006e0a0 0x7fa0e014be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:11.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.374+0000 7fa0dffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0e006f3a0 0x7fa0e00c31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:11.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.374+0000 7fa0dffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0e0150380 0x7fa0e0157bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:11.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.374+0000 7fa0dffff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e014c3c0 con 0x7fa0e0150380 2023-10-06T02:32:11.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.374+0000 7fa0dffff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e0156250 con 0x7fa0e006f3a0 2023-10-06T02:32:11.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.374+0000 7fa0dffff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e01681f0 con 0x7fa0e006e0a0 2023-10-06T02:32:11.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.374+0000 7fa0df7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0e0150380 0x7fa0e0157bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:11.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.374+0000 7fa0df7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0e0150380 0x7fa0e0157bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41064/0 (socket says 172.21.15.31:41064) 2023-10-06T02:32:11.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.374+0000 7fa0df7fe700 1 -- 172.21.15.31:0/2570777398 learned_addr learned my addr 172.21.15.31:0/2570777398 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:11.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.374+0000 7fa0deffd700 1 --2- 172.21.15.31:0/2570777398 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa0e006e0a0 0x7fa0e014be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:11.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.374+0000 7fa0de7fc700 1 --2- 172.21.15.31:0/2570777398 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0e006f3a0 0x7fa0e00c31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:11.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.374+0000 7fa0df7fe700 1 -- 172.21.15.31:0/2570777398 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa0e006e0a0 msgr2=0x7fa0e014be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:11.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.374+0000 7fa0df7fe700 1 --2- 172.21.15.31:0/2570777398 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa0e006e0a0 0x7fa0e014be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:11.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.374+0000 7fa0df7fe700 1 -- 172.21.15.31:0/2570777398 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0e006f3a0 msgr2=0x7fa0e00c31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:11.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.374+0000 7fa0df7fe700 1 --2- 172.21.15.31:0/2570777398 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0e006f3a0 0x7fa0e00c31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:11.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.374+0000 7fa0df7fe700 1 -- 172.21.15.31:0/2570777398 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0d8009ae0 con 0x7fa0e0150380 2023-10-06T02:32:11.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.374+0000 7fa0deffd700 1 --2- 172.21.15.31:0/2570777398 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa0e006e0a0 0x7fa0e014be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:32:11.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.375+0000 7fa0df7fe700 1 --2- 172.21.15.31:0/2570777398 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0e0150380 0x7fa0e0157bf0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7fa0d8012000 tx=0x7fa0d8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:11.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.375+0000 7fa0c6ffd700 1 -- 172.21.15.31:0/2570777398 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d8008c70 con 0x7fa0e0150380 2023-10-06T02:32:11.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.375+0000 7fa0c6ffd700 1 -- 172.21.15.31:0/2570777398 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0d8008df0 con 0x7fa0e0150380 2023-10-06T02:32:11.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.375+0000 7fa0c6ffd700 1 -- 172.21.15.31:0/2570777398 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d80256b0 con 0x7fa0e0150380 2023-10-06T02:32:11.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.375+0000 7fa0dffff700 1 -- 172.21.15.31:0/2570777398 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0e01564f0 con 0x7fa0e0150380 2023-10-06T02:32:11.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.375+0000 7fa0dffff700 1 -- 172.21.15.31:0/2570777398 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0e01684d0 con 0x7fa0e0150380 2023-10-06T02:32:11.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.376+0000 7fa0dffff700 1 -- 172.21.15.31:0/2570777398 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0e004c3c0 con 0x7fa0e0150380 2023-10-06T02:32:11.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.377+0000 7fa0c6ffd700 1 -- 172.21.15.31:0/2570777398 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85556+0+0 (secure 0 0 0) 0x7fa0d8003040 con 0x7fa0e0150380 2023-10-06T02:32:11.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.378+0000 7fa0c6ffd700 1 --2- 172.21.15.31:0/2570777398 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa0cc067830 0x7fa0cc069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:11.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.378+0000 7fa0c6ffd700 1 -- 172.21.15.31:0/2570777398 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fa0d8084470 con 0x7fa0e0150380 2023-10-06T02:32:11.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.379+0000 7fa0deffd700 1 --2- 172.21.15.31:0/2570777398 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa0cc067830 0x7fa0cc069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:11.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.379+0000 7fa0deffd700 1 --2- 172.21.15.31:0/2570777398 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa0cc067830 0x7fa0cc069ce0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fa0e0159590 tx=0x7fa0d0008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:11.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.382+0000 7fa0c6ffd700 1 -- 172.21.15.31:0/2570777398 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa0d8053160 con 0x7fa0e0150380 2023-10-06T02:32:11.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.630+0000 7fa0dffff700 1 -- 172.21.15.31:0/2570777398 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa0e00c7d30 con 0x7fa0e0150380 2023-10-06T02:32:11.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.631+0000 7fa0c6ffd700 1 -- 172.21.15.31:0/2570777398 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 119 v119) v1 ==== 95+0+4601 (secure 0 0 0) 0x7fa0d8056780 con 0x7fa0e0150380 2023-10-06T02:32:11.632 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:32:11.632 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":119,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7445,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/324072138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":324072138},{"type":"v1","addr":"172.21.15.31:6835","nonce":324072138}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":114},{"gid":7448,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/3160488596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3160488596},{"type":"v1","addr":"172.21.15.62:6834","nonce":3160488596}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":114},{"gid":8677,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/632134516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":632134516},{"type":"v1","addr":"172.21.15.62:6835","nonce":632134516}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":114}],"filesystems":[{"mdsmap":{"epoch":119,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:32:07.613387+0000","modified":"2023-10-06T02:32:11.143622+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9267},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9267":{"gid":9267,"name":"a","rank":0,"incarnation":115,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6834/3741373122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":3741373122},{"type":"v1","addr":"172.21.15.31:6834","nonce":3741373122}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2023-10-06T02:32:11.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.633+0000 7fa0dffff700 1 -- 172.21.15.31:0/2570777398 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa0cc067830 msgr2=0x7fa0cc069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:11.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.633+0000 7fa0dffff700 1 --2- 172.21.15.31:0/2570777398 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa0cc067830 0x7fa0cc069ce0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fa0e0159590 tx=0x7fa0d0008040 comp rx=0 tx=0).stop 2023-10-06T02:32:11.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.633+0000 7fa0dffff700 1 -- 172.21.15.31:0/2570777398 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0e0150380 msgr2=0x7fa0e0157bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:11.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.633+0000 7fa0dffff700 1 --2- 172.21.15.31:0/2570777398 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0e0150380 0x7fa0e0157bf0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7fa0d8012000 tx=0x7fa0d8018000 comp rx=0 tx=0).stop 2023-10-06T02:32:11.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.633+0000 7fa0dffff700 1 -- 172.21.15.31:0/2570777398 shutdown_connections 2023-10-06T02:32:11.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.633+0000 7fa0dffff700 1 --2- 172.21.15.31:0/2570777398 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0e0150380 0x7fa0e0157bf0 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:11.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.633+0000 7fa0dffff700 1 --2- 172.21.15.31:0/2570777398 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa0cc067830 0x7fa0cc069ce0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:11.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.633+0000 7fa0dffff700 1 --2- 172.21.15.31:0/2570777398 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa0e006e0a0 0x7fa0e014be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:11.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.633+0000 7fa0dffff700 1 --2- 172.21.15.31:0/2570777398 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0e006f3a0 0x7fa0e00c31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:11.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.633+0000 7fa0dffff700 1 -- 172.21.15.31:0/2570777398 >> 172.21.15.31:0/2570777398 conn(0x7fa0e006d170 msgr2=0x7fa0e00cc980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:11.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.633+0000 7fa0dffff700 1 -- 172.21.15.31:0/2570777398 shutdown_connections 2023-10-06T02:32:11.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.633+0000 7fa0dffff700 1 -- 172.21.15.31:0/2570777398 wait complete. 2023-10-06T02:32:11.635 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 119 2023-10-06T02:32:11.651 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:32:11.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.837+0000 7f53bf59e700 1 Processor -- start 2023-10-06T02:32:11.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.837+0000 7f53bf59e700 1 -- start start 2023-10-06T02:32:11.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.838+0000 7f53bf59e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f53c0159540 0x7f53c00c1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:11.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.838+0000 7f53bf59e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f53c00c6be0 0x7f53c00c7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:11.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.838+0000 7f53bf59e700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53c00c7580 con 0x7f53c0159910 2023-10-06T02:32:11.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.838+0000 7f53bf59e700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53c0068250 con 0x7f53c00c6be0 2023-10-06T02:32:11.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.838+0000 7f53bf59e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53c0068530 con 0x7f53c0159540 2023-10-06T02:32:11.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.839+0000 7f53bed9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f53c00c6be0 0x7f53c00c7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:11.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.839+0000 7f53bdd9b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f53c0159910 0x7f53c00677b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:47722/0 (socket says 172.21.15.31:47722) 2023-10-06T02:32:11.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.839+0000 7f53bdd9b700 1 -- 172.21.15.31:0/251728244 learned_addr learned my addr 172.21.15.31:0/251728244 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:32:11.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.839+0000 7f53be59c700 1 --2- 172.21.15.31:0/251728244 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f53c0159540 0x7f53c00c1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:11.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.839+0000 7f53bd59a700 1 -- 172.21.15.31:0/251728244 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 641633507 0 0) 0x7f53b4001110 con 0x7f53c0159910 2023-10-06T02:32:11.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.839+0000 7f53bd59a700 1 -- 172.21.15.31:0/251728244 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53a8003570 con 0x7f53c0159910 2023-10-06T02:32:11.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.839+0000 7f53bed9d700 1 -- 172.21.15.31:0/251728244 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f53c0159540 msgr2=0x7f53c00c1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:11.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.839+0000 7f53bed9d700 1 --2- 172.21.15.31:0/251728244 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f53c0159540 0x7f53c00c1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:11.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.839+0000 7f53bed9d700 1 -- 172.21.15.31:0/251728244 >> v1:172.21.15.31:6789/0 conn(0x7f53c0159910 legacy=0x7f53c00677b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:11.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.839+0000 7f53bed9d700 1 -- 172.21.15.31:0/251728244 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53c0068810 con 0x7f53c00c6be0 2023-10-06T02:32:11.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.839+0000 7f53be59c700 1 --2- 172.21.15.31:0/251728244 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f53c0159540 0x7f53c00c1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:32:11.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.840+0000 7f53bed9d700 1 --2- 172.21.15.31:0/251728244 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53c00c6be0 0x7f53c00c7040 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f53b8009e60 tx=0x7f53b8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8d366c9fab7cec1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:11.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.840+0000 7f53bd59a700 1 -- 172.21.15.31:0/251728244 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b800e070 con 0x7f53c00c6be0 2023-10-06T02:32:11.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.840+0000 7f53bd59a700 1 -- 172.21.15.31:0/251728244 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53b800bb60 con 0x7f53c00c6be0 2023-10-06T02:32:11.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.840+0000 7f53bd59a700 1 -- 172.21.15.31:0/251728244 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b8012430 con 0x7f53c00c6be0 2023-10-06T02:32:11.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.840+0000 7f53bf59e700 1 -- 172.21.15.31:0/251728244 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53c00c6be0 msgr2=0x7f53c00c7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:11.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.840+0000 7f53bf59e700 1 --2- 172.21.15.31:0/251728244 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53c00c6be0 0x7f53c00c7040 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f53b8009e60 tx=0x7f53b8009780 comp rx=0 tx=0).stop 2023-10-06T02:32:11.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.840+0000 7f53bf59e700 1 -- 172.21.15.31:0/251728244 shutdown_connections 2023-10-06T02:32:11.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.840+0000 7f53bf59e700 1 --2- 172.21.15.31:0/251728244 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53c00c6be0 0x7f53c00c7040 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:11.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.840+0000 7f53bf59e700 1 --2- 172.21.15.31:0/251728244 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f53c0159540 0x7f53c00c1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:11.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.840+0000 7f53bf59e700 1 -- 172.21.15.31:0/251728244 >> 172.21.15.31:0/251728244 conn(0x7f53c006f310 msgr2=0x7f53c00cc5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:11.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.840+0000 7f53bf59e700 1 -- 172.21.15.31:0/251728244 shutdown_connections 2023-10-06T02:32:11.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.841+0000 7f53bf59e700 1 -- 172.21.15.31:0/251728244 wait complete. 2023-10-06T02:32:11.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.842+0000 7f53bf59e700 1 Processor -- start 2023-10-06T02:32:11.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.842+0000 7f53bf59e700 1 -- start start 2023-10-06T02:32:11.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.843+0000 7f53bf59e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f53c00c6be0 0x7f53c014d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:11.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.843+0000 7f53bf59e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53c0159540 0x7f53c014cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:11.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.843+0000 7f53bf59e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53c0159910 0x7f53c00b0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:11.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.843+0000 7f53bf59e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c0169400 con 0x7f53c0159540 2023-10-06T02:32:11.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.843+0000 7f53bf59e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c014d2c0 con 0x7f53c0159910 2023-10-06T02:32:11.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.843+0000 7f53bf59e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53c014dbe0 con 0x7f53c00c6be0 2023-10-06T02:32:11.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.843+0000 7f53bdd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53c0159540 0x7f53c014cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:11.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.843+0000 7f53bdd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53c0159540 0x7f53c014cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41076/0 (socket says 172.21.15.31:41076) 2023-10-06T02:32:11.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.843+0000 7f53bdd9b700 1 -- 172.21.15.31:0/2866322518 learned_addr learned my addr 172.21.15.31:0/2866322518 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:11.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.843+0000 7f53bed9d700 1 --2- 172.21.15.31:0/2866322518 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53c0159910 0x7f53c00b0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:11.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.843+0000 7f53be59c700 1 --2- 172.21.15.31:0/2866322518 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f53c00c6be0 0x7f53c014d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:11.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.844+0000 7f53bdd9b700 1 -- 172.21.15.31:0/2866322518 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f53c00c6be0 msgr2=0x7f53c014d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:11.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.844+0000 7f53bdd9b700 1 --2- 172.21.15.31:0/2866322518 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f53c00c6be0 0x7f53c014d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:11.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.844+0000 7f53bdd9b700 1 -- 172.21.15.31:0/2866322518 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53c0159910 msgr2=0x7f53c00b0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:11.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.844+0000 7f53bdd9b700 1 --2- 172.21.15.31:0/2866322518 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53c0159910 0x7f53c00b0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:11.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.844+0000 7f53bdd9b700 1 -- 172.21.15.31:0/2866322518 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53b8009a20 con 0x7f53c0159540 2023-10-06T02:32:11.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.844+0000 7f53bed9d700 1 --2- 172.21.15.31:0/2866322518 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53c0159910 0x7f53c00b0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:32:11.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.844+0000 7f53bdd9b700 1 --2- 172.21.15.31:0/2866322518 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53c0159540 0x7f53c014cd80 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f53b400c7c0 tx=0x7f53b4007c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:11.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.844+0000 7f53ae7fc700 1 -- 172.21.15.31:0/2866322518 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b4005b30 con 0x7f53c0159540 2023-10-06T02:32:11.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.844+0000 7f53ae7fc700 1 -- 172.21.15.31:0/2866322518 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53b4005c90 con 0x7f53c0159540 2023-10-06T02:32:11.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.844+0000 7f53bf59e700 1 -- 172.21.15.31:0/2866322518 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53c014dec0 con 0x7f53c0159540 2023-10-06T02:32:11.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.844+0000 7f53ae7fc700 1 -- 172.21.15.31:0/2866322518 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b4007340 con 0x7f53c0159540 2023-10-06T02:32:11.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.844+0000 7f53bf59e700 1 -- 172.21.15.31:0/2866322518 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53c014e480 con 0x7f53c0159540 2023-10-06T02:32:11.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.845+0000 7f53bf59e700 1 -- 172.21.15.31:0/2866322518 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53c004c3c0 con 0x7f53c0159540 2023-10-06T02:32:11.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.846+0000 7f53ae7fc700 1 -- 172.21.15.31:0/2866322518 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85556+0+0 (secure 0 0 0) 0x7f53b40076f0 con 0x7f53c0159540 2023-10-06T02:32:11.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.847+0000 7f53ae7fc700 1 --2- 172.21.15.31:0/2866322518 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f53a0067830 0x7f53a0069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:11.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.847+0000 7f53ae7fc700 1 -- 172.21.15.31:0/2866322518 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f53b40827e0 con 0x7f53c0159540 2023-10-06T02:32:11.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.848+0000 7f53be59c700 1 --2- 172.21.15.31:0/2866322518 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f53a0067830 0x7f53a0069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:11.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.848+0000 7f53be59c700 1 --2- 172.21.15.31:0/2866322518 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f53a0067830 0x7f53a0069ce0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f53b0006f00 tx=0x7f53b0008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:11.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:11.851+0000 7f53ae7fc700 1 -- 172.21.15.31:0/2866322518 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f53b40514a0 con 0x7f53c0159540 2023-10-06T02:32:12.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.057+0000 7f53bf59e700 1 -- 172.21.15.31:0/2866322518 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f53c00c7580 con 0x7f53c0159540 2023-10-06T02:32:12.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.058+0000 7f53ae7fc700 1 -- 172.21.15.31:0/2866322518 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v169) v1 ==== 74+0+16876 (secure 0 0 0) 0x7f53c00c7580 con 0x7f53c0159540 2023-10-06T02:32:12.059 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:32:12.059 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":169,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:32:09.112135+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:32:05.275284+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"168","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":28,"pool_name":"cephfs_data","create_time":"2023-10-06T02:32:06.591889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"169","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":20,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:32:12.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.060+0000 7f53bf59e700 1 -- 172.21.15.31:0/2866322518 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f53a0067830 msgr2=0x7f53a0069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:12.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.060+0000 7f53bf59e700 1 --2- 172.21.15.31:0/2866322518 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f53a0067830 0x7f53a0069ce0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f53b0006f00 tx=0x7f53b0008040 comp rx=0 tx=0).stop 2023-10-06T02:32:12.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.060+0000 7f53bf59e700 1 -- 172.21.15.31:0/2866322518 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53c0159540 msgr2=0x7f53c014cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:12.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.060+0000 7f53bf59e700 1 --2- 172.21.15.31:0/2866322518 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53c0159540 0x7f53c014cd80 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f53b400c7c0 tx=0x7f53b4007c70 comp rx=0 tx=0).stop 2023-10-06T02:32:12.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.060+0000 7f53bf59e700 1 -- 172.21.15.31:0/2866322518 shutdown_connections 2023-10-06T02:32:12.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.060+0000 7f53bf59e700 1 --2- 172.21.15.31:0/2866322518 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53c0159910 0x7f53c00b0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:12.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.060+0000 7f53bf59e700 1 --2- 172.21.15.31:0/2866322518 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f53a0067830 0x7f53a0069ce0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:12.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.060+0000 7f53bf59e700 1 --2- 172.21.15.31:0/2866322518 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f53c00c6be0 0x7f53c014d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:12.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.060+0000 7f53bf59e700 1 --2- 172.21.15.31:0/2866322518 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53c0159540 0x7f53c014cd80 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:12.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.060+0000 7f53bf59e700 1 -- 172.21.15.31:0/2866322518 >> 172.21.15.31:0/2866322518 conn(0x7f53c006f310 msgr2=0x7f53c015a0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:12.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.060+0000 7f53bf59e700 1 -- 172.21.15.31:0/2866322518 shutdown_connections 2023-10-06T02:32:12.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.060+0000 7f53bf59e700 1 -- 172.21.15.31:0/2866322518 wait complete. 2023-10-06T02:32:12.078 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T02:32:12.078 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T02:32:12.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.265+0000 7fc95299d700 1 Processor -- start 2023-10-06T02:32:12.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.265+0000 7fc95299d700 1 -- start start 2023-10-06T02:32:12.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.267+0000 7fc95299d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc94c142c00 0x7fc94c06b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:12.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.267+0000 7fc95299d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc94c06bd20 0x7fc94c0c7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:12.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.267+0000 7fc95299d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc94c0ca480 con 0x7fc94c142fd0 2023-10-06T02:32:12.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.267+0000 7fc95299d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc94c15e240 con 0x7fc94c142c00 2023-10-06T02:32:12.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.267+0000 7fc95299d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc94c06b350 con 0x7fc94c06bd20 2023-10-06T02:32:12.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.267+0000 7fc95219c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fc94c142fd0 0x7fc94c06c6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:47732/0 (socket says 172.21.15.31:47732) 2023-10-06T02:32:12.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.267+0000 7fc95219c700 1 -- 172.21.15.31:0/790259544 learned_addr learned my addr 172.21.15.31:0/790259544 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:32:12.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.267+0000 7fc95199b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc94c142c00 0x7fc94c06b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:12.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.267+0000 7fc95119a700 1 --2- 172.21.15.31:0/790259544 >> v2:172.21.15.62:3300/0 conn(0x7fc94c06bd20 0x7fc94c0c7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:12.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.267+0000 7fc950999700 1 -- 172.21.15.31:0/790259544 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1288115066 0 0) 0x7fc94c0ca480 con 0x7fc94c142fd0 2023-10-06T02:32:12.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.267+0000 7fc95199b700 1 -- 172.21.15.31:0/790259544 >> v2:172.21.15.62:3300/0 conn(0x7fc94c06bd20 msgr2=0x7fc94c0c7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:12.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.267+0000 7fc95199b700 1 --2- 172.21.15.31:0/790259544 >> v2:172.21.15.62:3300/0 conn(0x7fc94c06bd20 0x7fc94c0c7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:12.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.267+0000 7fc95199b700 1 -- 172.21.15.31:0/790259544 >> v1:172.21.15.31:6789/0 conn(0x7fc94c142fd0 legacy=0x7fc94c06c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:12.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.267+0000 7fc95199b700 1 -- 172.21.15.31:0/790259544 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc94c15df60 con 0x7fc94c142c00 2023-10-06T02:32:12.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.268+0000 7fc95199b700 1 --2- 172.21.15.31:0/790259544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc94c142c00 0x7fc94c06b7e0 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7fc94c06d750 tx=0x7fc9480091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e9075d1820e0d20 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:12.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.268+0000 7fc950999700 1 -- 172.21.15.31:0/790259544 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94800b6b0 con 0x7fc94c142c00 2023-10-06T02:32:12.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.268+0000 7fc950999700 1 -- 172.21.15.31:0/790259544 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc94800b810 con 0x7fc94c142c00 2023-10-06T02:32:12.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.268+0000 7fc950999700 1 -- 172.21.15.31:0/790259544 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94800bbb0 con 0x7fc94c142c00 2023-10-06T02:32:12.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.268+0000 7fc95299d700 1 -- 172.21.15.31:0/790259544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc94c142c00 msgr2=0x7fc94c06b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:12.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.268+0000 7fc95299d700 1 --2- 172.21.15.31:0/790259544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc94c142c00 0x7fc94c06b7e0 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7fc94c06d750 tx=0x7fc9480091a0 comp rx=0 tx=0).stop 2023-10-06T02:32:12.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.269+0000 7fc95299d700 1 -- 172.21.15.31:0/790259544 shutdown_connections 2023-10-06T02:32:12.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.269+0000 7fc95299d700 1 --2- 172.21.15.31:0/790259544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc94c142c00 0x7fc94c06b7e0 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:12.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.269+0000 7fc95299d700 1 --2- 172.21.15.31:0/790259544 >> v2:172.21.15.62:3300/0 conn(0x7fc94c06bd20 0x7fc94c0c7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:12.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.269+0000 7fc95299d700 1 -- 172.21.15.31:0/790259544 >> 172.21.15.31:0/790259544 conn(0x7fc94c0675a0 msgr2=0x7fc94c0679a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:12.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.269+0000 7fc95299d700 1 -- 172.21.15.31:0/790259544 shutdown_connections 2023-10-06T02:32:12.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.269+0000 7fc95299d700 1 -- 172.21.15.31:0/790259544 wait complete. 2023-10-06T02:32:12.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.270+0000 7fc95299d700 1 Processor -- start 2023-10-06T02:32:12.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.270+0000 7fc95299d700 1 -- start start 2023-10-06T02:32:12.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.271+0000 7fc95299d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc94c06bd20 0x7fc94c12e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:12.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.271+0000 7fc95299d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc94c142c00 0x7fc94c159050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:12.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.271+0000 7fc95299d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc94c142fd0 0x7fc94c141100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:12.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.271+0000 7fc95299d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc94c159590 con 0x7fc94c142fd0 2023-10-06T02:32:12.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.271+0000 7fc95299d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc94c0c5490 con 0x7fc94c142c00 2023-10-06T02:32:12.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.271+0000 7fc95299d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc94c0c6010 con 0x7fc94c06bd20 2023-10-06T02:32:12.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.272+0000 7fc95219c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc94c142fd0 0x7fc94c141100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:12.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.272+0000 7fc95219c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc94c142fd0 0x7fc94c141100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41090/0 (socket says 172.21.15.31:41090) 2023-10-06T02:32:12.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.272+0000 7fc95219c700 1 -- 172.21.15.31:0/3074432942 learned_addr learned my addr 172.21.15.31:0/3074432942 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:12.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.272+0000 7fc95119a700 1 --2- 172.21.15.31:0/3074432942 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc94c142c00 0x7fc94c159050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:12.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.272+0000 7fc95199b700 1 --2- 172.21.15.31:0/3074432942 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc94c06bd20 0x7fc94c12e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:12.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.272+0000 7fc95219c700 1 -- 172.21.15.31:0/3074432942 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc94c06bd20 msgr2=0x7fc94c12e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:12.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.272+0000 7fc95219c700 1 --2- 172.21.15.31:0/3074432942 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc94c06bd20 0x7fc94c12e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:12.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.272+0000 7fc95219c700 1 -- 172.21.15.31:0/3074432942 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc94c142c00 msgr2=0x7fc94c159050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:12.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.272+0000 7fc95219c700 1 --2- 172.21.15.31:0/3074432942 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc94c142c00 0x7fc94c159050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:12.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.272+0000 7fc95219c700 1 -- 172.21.15.31:0/3074432942 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc948009420 con 0x7fc94c142fd0 2023-10-06T02:32:12.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.272+0000 7fc95119a700 1 --2- 172.21.15.31:0/3074432942 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc94c142c00 0x7fc94c159050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:32:12.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.272+0000 7fc95199b700 1 --2- 172.21.15.31:0/3074432942 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc94c06bd20 0x7fc94c12e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:32:12.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.272+0000 7fc95219c700 1 --2- 172.21.15.31:0/3074432942 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc94c142fd0 0x7fc94c141100 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7fc94400cc50 tx=0x7fc9440019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:12.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.272+0000 7fc941ffb700 1 -- 172.21.15.31:0/3074432942 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94400ce60 con 0x7fc94c142fd0 2023-10-06T02:32:12.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.272+0000 7fc941ffb700 1 -- 172.21.15.31:0/3074432942 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc94400aaf0 con 0x7fc94c142fd0 2023-10-06T02:32:12.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.272+0000 7fc95299d700 1 -- 172.21.15.31:0/3074432942 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc94c0c56e0 con 0x7fc94c142fd0 2023-10-06T02:32:12.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.273+0000 7fc941ffb700 1 -- 172.21.15.31:0/3074432942 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9440070f0 con 0x7fc94c142fd0 2023-10-06T02:32:12.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.273+0000 7fc95299d700 1 -- 172.21.15.31:0/3074432942 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc94c0c5ca0 con 0x7fc94c142fd0 2023-10-06T02:32:12.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.273+0000 7fc95299d700 1 -- 172.21.15.31:0/3074432942 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc94c0c61a0 con 0x7fc94c142fd0 2023-10-06T02:32:12.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.275+0000 7fc941ffb700 1 -- 172.21.15.31:0/3074432942 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85556+0+0 (secure 0 0 0) 0x7fc9440074e0 con 0x7fc94c142fd0 2023-10-06T02:32:12.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.275+0000 7fc941ffb700 1 --2- 172.21.15.31:0/3074432942 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc930067830 0x7fc930069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:12.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.275+0000 7fc941ffb700 1 -- 172.21.15.31:0/3074432942 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fc944083250 con 0x7fc94c142fd0 2023-10-06T02:32:12.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.276+0000 7fc95199b700 1 --2- 172.21.15.31:0/3074432942 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc930067830 0x7fc930069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:12.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.277+0000 7fc95199b700 1 --2- 172.21.15.31:0/3074432942 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc930067830 0x7fc930069ce0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fc9480093f0 tx=0x7fc948017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:12.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.279+0000 7fc941ffb700 1 -- 172.21.15.31:0/3074432942 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc944051f10 con 0x7fc94c142fd0 2023-10-06T02:32:12.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.476+0000 7fc95299d700 1 -- 172.21.15.31:0/3074432942 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc94c04c3c0 con 0x7fc930067830 2023-10-06T02:32:12.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.486+0000 7fc941ffb700 1 -- 172.21.15.31:0/3074432942 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311412 (secure 0 0 0) 0x7fc94c04c3c0 con 0x7fc930067830 2023-10-06T02:32:12.491 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:32:12.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.490+0000 7fc95299d700 1 -- 172.21.15.31:0/3074432942 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc930067830 msgr2=0x7fc930069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:12.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.490+0000 7fc95299d700 1 --2- 172.21.15.31:0/3074432942 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc930067830 0x7fc930069ce0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fc9480093f0 tx=0x7fc948017000 comp rx=0 tx=0).stop 2023-10-06T02:32:12.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.490+0000 7fc95299d700 1 -- 172.21.15.31:0/3074432942 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc94c142fd0 msgr2=0x7fc94c141100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:12.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.490+0000 7fc95299d700 1 --2- 172.21.15.31:0/3074432942 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc94c142fd0 0x7fc94c141100 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7fc94400cc50 tx=0x7fc9440019b0 comp rx=0 tx=0).stop 2023-10-06T02:32:12.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.491+0000 7fc95299d700 1 -- 172.21.15.31:0/3074432942 shutdown_connections 2023-10-06T02:32:12.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.491+0000 7fc95299d700 1 --2- 172.21.15.31:0/3074432942 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc94c142fd0 0x7fc94c141100 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:12.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.491+0000 7fc95299d700 1 --2- 172.21.15.31:0/3074432942 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc930067830 0x7fc930069ce0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:12.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.491+0000 7fc95299d700 1 --2- 172.21.15.31:0/3074432942 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc94c06bd20 0x7fc94c12e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:12.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.491+0000 7fc95299d700 1 --2- 172.21.15.31:0/3074432942 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc94c142c00 0x7fc94c159050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:12.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.491+0000 7fc95299d700 1 -- 172.21.15.31:0/3074432942 >> 172.21.15.31:0/3074432942 conn(0x7fc94c0675a0 msgr2=0x7fc94c0c7840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:12.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.491+0000 7fc95299d700 1 -- 172.21.15.31:0/3074432942 shutdown_connections 2023-10-06T02:32:12.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.491+0000 7fc95299d700 1 -- 172.21.15.31:0/3074432942 wait complete. 2023-10-06T02:32:12.494 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-10-06T02:32:12.509 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":233,"stamp":"2023-10-06T02:32:10.523158+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":223,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":166,"ondisk_log_size":166,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":108,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2428872,"kb_used_data":17800,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752545848,"statfs":{"total":773094113280,"available":770606948352,"internally_reserved":0,"allocated":18227200,"data_stored":9113124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.418737"},"pg_stats":[{"pgid":"28.3f","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.150309+0000","last_change":"2023-10-06T02:32:08.147428+0000","last_active":"2023-10-06T02:32:08.150309+0000","last_peered":"2023-10-06T02:32:08.150309+0000","last_clean":"2023-10-06T02:32:08.150309+0000","last_became_active":"2023-10-06T02:32:08.147118+0000","last_became_peered":"2023-10-06T02:32:08.147118+0000","last_unstale":"2023-10-06T02:32:08.150309+0000","last_undegraded":"2023-10-06T02:32:08.150309+0000","last_fullsized":"2023-10-06T02:32:08.150309+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:33:05.856323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.3e","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.139802+0000","last_change":"2023-10-06T02:32:08.136941+0000","last_active":"2023-10-06T02:32:08.139802+0000","last_peered":"2023-10-06T02:32:08.139802+0000","last_clean":"2023-10-06T02:32:08.139802+0000","last_became_active":"2023-10-06T02:32:08.136676+0000","last_became_peered":"2023-10-06T02:32:08.136676+0000","last_unstale":"2023-10-06T02:32:08.139802+0000","last_undegraded":"2023-10-06T02:32:08.139802+0000","last_fullsized":"2023-10-06T02:32:08.139802+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:20:56.793118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.3d","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.143184+0000","last_change":"2023-10-06T02:32:08.143184+0000","last_active":"2023-10-06T02:32:08.143184+0000","last_peered":"2023-10-06T02:32:08.143184+0000","last_clean":"2023-10-06T02:32:08.143184+0000","last_became_active":"2023-10-06T02:32:08.142920+0000","last_became_peered":"2023-10-06T02:32:08.142920+0000","last_unstale":"2023-10-06T02:32:08.143184+0000","last_undegraded":"2023-10-06T02:32:08.143184+0000","last_fullsized":"2023-10-06T02:32:08.143184+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:18:01.461302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.3c","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.138525+0000","last_change":"2023-10-06T02:32:08.138525+0000","last_active":"2023-10-06T02:32:08.138525+0000","last_peered":"2023-10-06T02:32:08.138525+0000","last_clean":"2023-10-06T02:32:08.138525+0000","last_became_active":"2023-10-06T02:32:08.138206+0000","last_became_peered":"2023-10-06T02:32:08.138206+0000","last_unstale":"2023-10-06T02:32:08.138525+0000","last_undegraded":"2023-10-06T02:32:08.138525+0000","last_fullsized":"2023-10-06T02:32:08.138525+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:59:25.898666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.3b","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.142156+0000","last_change":"2023-10-06T02:32:08.142156+0000","last_active":"2023-10-06T02:32:08.142156+0000","last_peered":"2023-10-06T02:32:08.142156+0000","last_clean":"2023-10-06T02:32:08.142156+0000","last_became_active":"2023-10-06T02:32:08.141849+0000","last_became_peered":"2023-10-06T02:32:08.141849+0000","last_unstale":"2023-10-06T02:32:08.142156+0000","last_undegraded":"2023-10-06T02:32:08.142156+0000","last_fullsized":"2023-10-06T02:32:08.142156+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:27:40.521385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.3a","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.137802+0000","last_change":"2023-10-06T02:32:08.137802+0000","last_active":"2023-10-06T02:32:08.137802+0000","last_peered":"2023-10-06T02:32:08.137802+0000","last_clean":"2023-10-06T02:32:08.137802+0000","last_became_active":"2023-10-06T02:32:08.137516+0000","last_became_peered":"2023-10-06T02:32:08.137516+0000","last_unstale":"2023-10-06T02:32:08.137802+0000","last_undegraded":"2023-10-06T02:32:08.137802+0000","last_fullsized":"2023-10-06T02:32:08.137802+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:11:20.474130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.39","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.142055+0000","last_change":"2023-10-06T02:32:08.142055+0000","last_active":"2023-10-06T02:32:08.142055+0000","last_peered":"2023-10-06T02:32:08.142055+0000","last_clean":"2023-10-06T02:32:08.142055+0000","last_became_active":"2023-10-06T02:32:08.141751+0000","last_became_peered":"2023-10-06T02:32:08.141751+0000","last_unstale":"2023-10-06T02:32:08.142055+0000","last_undegraded":"2023-10-06T02:32:08.142055+0000","last_fullsized":"2023-10-06T02:32:08.142055+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:43:16.882147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.38","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.136360+0000","last_change":"2023-10-06T02:32:08.136360+0000","last_active":"2023-10-06T02:32:08.136360+0000","last_peered":"2023-10-06T02:32:08.136360+0000","last_clean":"2023-10-06T02:32:08.136360+0000","last_became_active":"2023-10-06T02:32:08.136088+0000","last_became_peered":"2023-10-06T02:32:08.136088+0000","last_unstale":"2023-10-06T02:32:08.136360+0000","last_undegraded":"2023-10-06T02:32:08.136360+0000","last_fullsized":"2023-10-06T02:32:08.136360+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:30:07.358659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.37","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.150304+0000","last_change":"2023-10-06T02:32:08.150304+0000","last_active":"2023-10-06T02:32:08.150304+0000","last_peered":"2023-10-06T02:32:08.150304+0000","last_clean":"2023-10-06T02:32:08.150304+0000","last_became_active":"2023-10-06T02:32:08.149835+0000","last_became_peered":"2023-10-06T02:32:08.149835+0000","last_unstale":"2023-10-06T02:32:08.150304+0000","last_undegraded":"2023-10-06T02:32:08.150304+0000","last_fullsized":"2023-10-06T02:32:08.150304+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:57:42.946510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.36","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.140956+0000","last_change":"2023-10-06T02:32:08.140956+0000","last_active":"2023-10-06T02:32:08.140956+0000","last_peered":"2023-10-06T02:32:08.140956+0000","last_clean":"2023-10-06T02:32:08.140956+0000","last_became_active":"2023-10-06T02:32:08.140700+0000","last_became_peered":"2023-10-06T02:32:08.140700+0000","last_unstale":"2023-10-06T02:32:08.140956+0000","last_undegraded":"2023-10-06T02:32:08.140956+0000","last_fullsized":"2023-10-06T02:32:08.140956+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:21:02.172715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.35","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.144570+0000","last_change":"2023-10-06T02:32:08.144570+0000","last_active":"2023-10-06T02:32:08.144570+0000","last_peered":"2023-10-06T02:32:08.144570+0000","last_clean":"2023-10-06T02:32:08.144570+0000","last_became_active":"2023-10-06T02:32:08.144277+0000","last_became_peered":"2023-10-06T02:32:08.144277+0000","last_unstale":"2023-10-06T02:32:08.144570+0000","last_undegraded":"2023-10-06T02:32:08.144570+0000","last_fullsized":"2023-10-06T02:32:08.144570+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:50:59.413536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.34","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.135521+0000","last_change":"2023-10-06T02:32:08.135521+0000","last_active":"2023-10-06T02:32:08.135521+0000","last_peered":"2023-10-06T02:32:08.135521+0000","last_clean":"2023-10-06T02:32:08.135521+0000","last_became_active":"2023-10-06T02:32:08.135206+0000","last_became_peered":"2023-10-06T02:32:08.135206+0000","last_unstale":"2023-10-06T02:32:08.135521+0000","last_undegraded":"2023-10-06T02:32:08.135521+0000","last_fullsized":"2023-10-06T02:32:08.135521+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:53:01.956748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.33","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.141398+0000","last_change":"2023-10-06T02:32:08.141398+0000","last_active":"2023-10-06T02:32:08.141398+0000","last_peered":"2023-10-06T02:32:08.141398+0000","last_clean":"2023-10-06T02:32:08.141398+0000","last_became_active":"2023-10-06T02:32:08.138550+0000","last_became_peered":"2023-10-06T02:32:08.138550+0000","last_unstale":"2023-10-06T02:32:08.141398+0000","last_undegraded":"2023-10-06T02:32:08.141398+0000","last_fullsized":"2023-10-06T02:32:08.141398+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:45:32.024530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.32","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.125227+0000","last_change":"2023-10-06T02:32:08.125227+0000","last_active":"2023-10-06T02:32:08.125227+0000","last_peered":"2023-10-06T02:32:08.125227+0000","last_clean":"2023-10-06T02:32:08.125227+0000","last_became_active":"2023-10-06T02:32:08.124744+0000","last_became_peered":"2023-10-06T02:32:08.124744+0000","last_unstale":"2023-10-06T02:32:08.125227+0000","last_undegraded":"2023-10-06T02:32:08.125227+0000","last_fullsized":"2023-10-06T02:32:08.125227+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:54:22.945907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.31","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.144984+0000","last_change":"2023-10-06T02:32:08.138975+0000","last_active":"2023-10-06T02:32:08.144984+0000","last_peered":"2023-10-06T02:32:08.144984+0000","last_clean":"2023-10-06T02:32:08.144984+0000","last_became_active":"2023-10-06T02:32:08.138559+0000","last_became_peered":"2023-10-06T02:32:08.138559+0000","last_unstale":"2023-10-06T02:32:08.144984+0000","last_undegraded":"2023-10-06T02:32:08.144984+0000","last_fullsized":"2023-10-06T02:32:08.144984+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:38:53.342204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.30","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.130358+0000","last_change":"2023-10-06T02:32:08.130358+0000","last_active":"2023-10-06T02:32:08.130358+0000","last_peered":"2023-10-06T02:32:08.130358+0000","last_clean":"2023-10-06T02:32:08.130358+0000","last_became_active":"2023-10-06T02:32:08.130078+0000","last_became_peered":"2023-10-06T02:32:08.130078+0000","last_unstale":"2023-10-06T02:32:08.130358+0000","last_undegraded":"2023-10-06T02:32:08.130358+0000","last_fullsized":"2023-10-06T02:32:08.130358+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:44:24.680238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.2f","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.149701+0000","last_change":"2023-10-06T02:32:08.149701+0000","last_active":"2023-10-06T02:32:08.149701+0000","last_peered":"2023-10-06T02:32:08.149701+0000","last_clean":"2023-10-06T02:32:08.149701+0000","last_became_active":"2023-10-06T02:32:08.149301+0000","last_became_peered":"2023-10-06T02:32:08.149301+0000","last_unstale":"2023-10-06T02:32:08.149701+0000","last_undegraded":"2023-10-06T02:32:08.149701+0000","last_fullsized":"2023-10-06T02:32:08.149701+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:28:32.152527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.2e","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.137505+0000","last_change":"2023-10-06T02:32:08.137505+0000","last_active":"2023-10-06T02:32:08.137505+0000","last_peered":"2023-10-06T02:32:08.137505+0000","last_clean":"2023-10-06T02:32:08.137505+0000","last_became_active":"2023-10-06T02:32:08.137248+0000","last_became_peered":"2023-10-06T02:32:08.137248+0000","last_unstale":"2023-10-06T02:32:08.137505+0000","last_undegraded":"2023-10-06T02:32:08.137505+0000","last_fullsized":"2023-10-06T02:32:08.137505+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:25.607604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.2d","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.135477+0000","last_change":"2023-10-06T02:32:08.135477+0000","last_active":"2023-10-06T02:32:08.135477+0000","last_peered":"2023-10-06T02:32:08.135477+0000","last_clean":"2023-10-06T02:32:08.135477+0000","last_became_active":"2023-10-06T02:32:08.135037+0000","last_became_peered":"2023-10-06T02:32:08.135037+0000","last_unstale":"2023-10-06T02:32:08.135477+0000","last_undegraded":"2023-10-06T02:32:08.135477+0000","last_fullsized":"2023-10-06T02:32:08.135477+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:32:28.584125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2c","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.143599+0000","last_change":"2023-10-06T02:32:08.143599+0000","last_active":"2023-10-06T02:32:08.143599+0000","last_peered":"2023-10-06T02:32:08.143599+0000","last_clean":"2023-10-06T02:32:08.143599+0000","last_became_active":"2023-10-06T02:32:08.143323+0000","last_became_peered":"2023-10-06T02:32:08.143323+0000","last_unstale":"2023-10-06T02:32:08.143599+0000","last_undegraded":"2023-10-06T02:32:08.143599+0000","last_fullsized":"2023-10-06T02:32:08.143599+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:18.734334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2b","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.141554+0000","last_change":"2023-10-06T02:32:08.141554+0000","last_active":"2023-10-06T02:32:08.141554+0000","last_peered":"2023-10-06T02:32:08.141554+0000","last_clean":"2023-10-06T02:32:08.141554+0000","last_became_active":"2023-10-06T02:32:08.141291+0000","last_became_peered":"2023-10-06T02:32:08.141291+0000","last_unstale":"2023-10-06T02:32:08.141554+0000","last_undegraded":"2023-10-06T02:32:08.141554+0000","last_fullsized":"2023-10-06T02:32:08.141554+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:17:46.078225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2a","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.125040+0000","last_change":"2023-10-06T02:32:08.125040+0000","last_active":"2023-10-06T02:32:08.125040+0000","last_peered":"2023-10-06T02:32:08.125040+0000","last_clean":"2023-10-06T02:32:08.125040+0000","last_became_active":"2023-10-06T02:32:08.124385+0000","last_became_peered":"2023-10-06T02:32:08.124385+0000","last_unstale":"2023-10-06T02:32:08.125040+0000","last_undegraded":"2023-10-06T02:32:08.125040+0000","last_fullsized":"2023-10-06T02:32:08.125040+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:53:01.956748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.29","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.130237+0000","last_change":"2023-10-06T02:32:08.128695+0000","last_active":"2023-10-06T02:32:08.130237+0000","last_peered":"2023-10-06T02:32:08.130237+0000","last_clean":"2023-10-06T02:32:08.130237+0000","last_became_active":"2023-10-06T02:32:08.128358+0000","last_became_peered":"2023-10-06T02:32:08.128358+0000","last_unstale":"2023-10-06T02:32:08.130237+0000","last_undegraded":"2023-10-06T02:32:08.130237+0000","last_fullsized":"2023-10-06T02:32:08.130237+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:45:47.719619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.28","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.123502+0000","last_change":"2023-10-06T02:32:08.123502+0000","last_active":"2023-10-06T02:32:08.123502+0000","last_peered":"2023-10-06T02:32:08.123502+0000","last_clean":"2023-10-06T02:32:08.123502+0000","last_became_active":"2023-10-06T02:32:08.123178+0000","last_became_peered":"2023-10-06T02:32:08.123178+0000","last_unstale":"2023-10-06T02:32:08.123502+0000","last_undegraded":"2023-10-06T02:32:08.123502+0000","last_fullsized":"2023-10-06T02:32:08.123502+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:54:22.945907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.27","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.138182+0000","last_change":"2023-10-06T02:32:08.138182+0000","last_active":"2023-10-06T02:32:08.138182+0000","last_peered":"2023-10-06T02:32:08.138182+0000","last_clean":"2023-10-06T02:32:08.138182+0000","last_became_active":"2023-10-06T02:32:08.137931+0000","last_became_peered":"2023-10-06T02:32:08.137931+0000","last_unstale":"2023-10-06T02:32:08.138182+0000","last_undegraded":"2023-10-06T02:32:08.138182+0000","last_fullsized":"2023-10-06T02:32:08.138182+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:14:29.040980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.26","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.148522+0000","last_change":"2023-10-06T02:32:08.146953+0000","last_active":"2023-10-06T02:32:08.148522+0000","last_peered":"2023-10-06T02:32:08.148522+0000","last_clean":"2023-10-06T02:32:08.148522+0000","last_became_active":"2023-10-06T02:32:08.146663+0000","last_became_peered":"2023-10-06T02:32:08.146663+0000","last_unstale":"2023-10-06T02:32:08.148522+0000","last_undegraded":"2023-10-06T02:32:08.148522+0000","last_fullsized":"2023-10-06T02:32:08.148522+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:50:59.413536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.25","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.143998+0000","last_change":"2023-10-06T02:32:08.143998+0000","last_active":"2023-10-06T02:32:08.143998+0000","last_peered":"2023-10-06T02:32:08.143998+0000","last_clean":"2023-10-06T02:32:08.143998+0000","last_became_active":"2023-10-06T02:32:08.143647+0000","last_became_peered":"2023-10-06T02:32:08.143647+0000","last_unstale":"2023-10-06T02:32:08.143998+0000","last_undegraded":"2023-10-06T02:32:08.143998+0000","last_fullsized":"2023-10-06T02:32:08.143998+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:34:14.560103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.24","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.138504+0000","last_change":"2023-10-06T02:32:08.138504+0000","last_active":"2023-10-06T02:32:08.138504+0000","last_peered":"2023-10-06T02:32:08.138504+0000","last_clean":"2023-10-06T02:32:08.138504+0000","last_became_active":"2023-10-06T02:32:08.138193+0000","last_became_peered":"2023-10-06T02:32:08.138193+0000","last_unstale":"2023-10-06T02:32:08.138504+0000","last_undegraded":"2023-10-06T02:32:08.138504+0000","last_fullsized":"2023-10-06T02:32:08.138504+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:38:52.612898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.23","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.140613+0000","last_change":"2023-10-06T02:32:08.140613+0000","last_active":"2023-10-06T02:32:08.140613+0000","last_peered":"2023-10-06T02:32:08.140613+0000","last_clean":"2023-10-06T02:32:08.140613+0000","last_became_active":"2023-10-06T02:32:08.140353+0000","last_became_peered":"2023-10-06T02:32:08.140353+0000","last_unstale":"2023-10-06T02:32:08.140613+0000","last_undegraded":"2023-10-06T02:32:08.140613+0000","last_fullsized":"2023-10-06T02:32:08.140613+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:55:31.162096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.22","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.145007+0000","last_change":"2023-10-06T02:32:08.139781+0000","last_active":"2023-10-06T02:32:08.145007+0000","last_peered":"2023-10-06T02:32:08.145007+0000","last_clean":"2023-10-06T02:32:08.145007+0000","last_became_active":"2023-10-06T02:32:08.139464+0000","last_became_peered":"2023-10-06T02:32:08.139464+0000","last_unstale":"2023-10-06T02:32:08.145007+0000","last_undegraded":"2023-10-06T02:32:08.145007+0000","last_fullsized":"2023-10-06T02:32:08.145007+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:23.130385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.21","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.137202+0000","last_change":"2023-10-06T02:32:08.137202+0000","last_active":"2023-10-06T02:32:08.137202+0000","last_peered":"2023-10-06T02:32:08.137202+0000","last_clean":"2023-10-06T02:32:08.137202+0000","last_became_active":"2023-10-06T02:32:08.136906+0000","last_became_peered":"2023-10-06T02:32:08.136906+0000","last_unstale":"2023-10-06T02:32:08.137202+0000","last_undegraded":"2023-10-06T02:32:08.137202+0000","last_fullsized":"2023-10-06T02:32:08.137202+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:10:35.944296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.20","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.143967+0000","last_change":"2023-10-06T02:32:08.143967+0000","last_active":"2023-10-06T02:32:08.143967+0000","last_peered":"2023-10-06T02:32:08.143967+0000","last_clean":"2023-10-06T02:32:08.143967+0000","last_became_active":"2023-10-06T02:32:08.143585+0000","last_became_peered":"2023-10-06T02:32:08.143585+0000","last_unstale":"2023-10-06T02:32:08.143967+0000","last_undegraded":"2023-10-06T02:32:08.143967+0000","last_fullsized":"2023-10-06T02:32:08.143967+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:32:15.396249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.10","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.145377+0000","last_change":"2023-10-06T02:32:08.140389+0000","last_active":"2023-10-06T02:32:08.145377+0000","last_peered":"2023-10-06T02:32:08.145377+0000","last_clean":"2023-10-06T02:32:08.145377+0000","last_became_active":"2023-10-06T02:32:08.140116+0000","last_became_peered":"2023-10-06T02:32:08.140116+0000","last_unstale":"2023-10-06T02:32:08.145377+0000","last_undegraded":"2023-10-06T02:32:08.145377+0000","last_fullsized":"2023-10-06T02:32:08.145377+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:42:28.367217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-06T02:32:07.136325+0000","last_change":"2023-10-06T02:32:07.132344+0000","last_active":"2023-10-06T02:32:07.136325+0000","last_peered":"2023-10-06T02:32:07.136325+0000","last_clean":"2023-10-06T02:32:07.136325+0000","last_became_active":"2023-10-06T02:32:07.132084+0000","last_became_peered":"2023-10-06T02:32:07.132084+0000","last_unstale":"2023-10-06T02:32:07.136325+0000","last_undegraded":"2023-10-06T02:32:07.136325+0000","last_fullsized":"2023-10-06T02:32:07.136325+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_clean_scrub_stamp":"2023-10-06T02:32:06.100240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:01:49.273527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.138085+0000","last_change":"2023-10-06T02:32:08.138085+0000","last_active":"2023-10-06T02:32:08.138085+0000","last_peered":"2023-10-06T02:32:08.138085+0000","last_clean":"2023-10-06T02:32:08.138085+0000","last_became_active":"2023-10-06T02:32:08.137462+0000","last_became_peered":"2023-10-06T02:32:08.137462+0000","last_unstale":"2023-10-06T02:32:08.138085+0000","last_undegraded":"2023-10-06T02:32:08.138085+0000","last_fullsized":"2023-10-06T02:32:08.138085+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:23:50.218257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-06T02:32:07.139608+0000","last_change":"2023-10-06T02:32:07.137545+0000","last_active":"2023-10-06T02:32:07.139608+0000","last_peered":"2023-10-06T02:32:07.139608+0000","last_clean":"2023-10-06T02:32:07.139608+0000","last_became_active":"2023-10-06T02:32:07.137242+0000","last_became_peered":"2023-10-06T02:32:07.137242+0000","last_unstale":"2023-10-06T02:32:07.139608+0000","last_undegraded":"2023-10-06T02:32:07.139608+0000","last_fullsized":"2023-10-06T02:32:07.139608+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_clean_scrub_stamp":"2023-10-06T02:32:06.100240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:25:14.308128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.149330+0000","last_change":"2023-10-06T02:32:08.149330+0000","last_active":"2023-10-06T02:32:08.149330+0000","last_peered":"2023-10-06T02:32:08.149330+0000","last_clean":"2023-10-06T02:32:08.149330+0000","last_became_active":"2023-10-06T02:32:08.149003+0000","last_became_peered":"2023-10-06T02:32:08.149003+0000","last_unstale":"2023-10-06T02:32:08.149330+0000","last_undegraded":"2023-10-06T02:32:08.149330+0000","last_fullsized":"2023-10-06T02:32:08.149330+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:49:18.782095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.15","version":"168'3","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.143048+0000","last_change":"2023-10-06T02:32:07.128223+0000","last_active":"2023-10-06T02:32:08.143048+0000","last_peered":"2023-10-06T02:32:08.143048+0000","last_clean":"2023-10-06T02:32:08.143048+0000","last_became_active":"2023-10-06T02:32:07.127823+0000","last_became_peered":"2023-10-06T02:32:07.127823+0000","last_unstale":"2023-10-06T02:32:08.143048+0000","last_undegraded":"2023-10-06T02:32:08.143048+0000","last_fullsized":"2023-10-06T02:32:08.143048+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_clean_scrub_stamp":"2023-10-06T02:32:06.100240+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:27:05.038453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.13","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.129503+0000","last_change":"2023-10-06T02:32:08.129503+0000","last_active":"2023-10-06T02:32:08.129503+0000","last_peered":"2023-10-06T02:32:08.129503+0000","last_clean":"2023-10-06T02:32:08.129503+0000","last_became_active":"2023-10-06T02:32:08.129233+0000","last_became_peered":"2023-10-06T02:32:08.129233+0000","last_unstale":"2023-10-06T02:32:08.129503+0000","last_undegraded":"2023-10-06T02:32:08.129503+0000","last_fullsized":"2023-10-06T02:32:08.129503+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:15:40.109842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.14","version":"168'6","reported_seq":15,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.145876+0000","last_change":"2023-10-06T02:32:07.141381+0000","last_active":"2023-10-06T02:32:08.145876+0000","last_peered":"2023-10-06T02:32:08.145876+0000","last_clean":"2023-10-06T02:32:08.145876+0000","last_became_active":"2023-10-06T02:32:07.141094+0000","last_became_peered":"2023-10-06T02:32:07.141094+0000","last_unstale":"2023-10-06T02:32:08.145876+0000","last_undegraded":"2023-10-06T02:32:08.145876+0000","last_fullsized":"2023-10-06T02:32:08.145876+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_clean_scrub_stamp":"2023-10-06T02:32:06.100240+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:33.292861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.14","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.139535+0000","last_change":"2023-10-06T02:32:08.135413+0000","last_active":"2023-10-06T02:32:08.139535+0000","last_peered":"2023-10-06T02:32:08.139535+0000","last_clean":"2023-10-06T02:32:08.139535+0000","last_became_active":"2023-10-06T02:32:08.135163+0000","last_became_peered":"2023-10-06T02:32:08.135163+0000","last_unstale":"2023-10-06T02:32:08.139535+0000","last_undegraded":"2023-10-06T02:32:08.139535+0000","last_fullsized":"2023-10-06T02:32:08.139535+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:56:05.639408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.13","version":"168'2","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.146593+0000","last_change":"2023-10-06T02:32:07.125916+0000","last_active":"2023-10-06T02:32:08.146593+0000","last_peered":"2023-10-06T02:32:08.146593+0000","last_clean":"2023-10-06T02:32:08.146593+0000","last_became_active":"2023-10-06T02:32:07.125637+0000","last_became_peered":"2023-10-06T02:32:07.125637+0000","last_unstale":"2023-10-06T02:32:08.146593+0000","last_undegraded":"2023-10-06T02:32:08.146593+0000","last_fullsized":"2023-10-06T02:32:08.146593+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_clean_scrub_stamp":"2023-10-06T02:32:06.100240+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:48:34.026512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.15","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.139830+0000","last_change":"2023-10-06T02:32:08.139830+0000","last_active":"2023-10-06T02:32:08.139830+0000","last_peered":"2023-10-06T02:32:08.139830+0000","last_clean":"2023-10-06T02:32:08.139830+0000","last_became_active":"2023-10-06T02:32:08.139495+0000","last_became_peered":"2023-10-06T02:32:08.139495+0000","last_unstale":"2023-10-06T02:32:08.139830+0000","last_undegraded":"2023-10-06T02:32:08.139830+0000","last_fullsized":"2023-10-06T02:32:08.139830+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:47:51.516673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-06T02:32:07.136118+0000","last_change":"2023-10-06T02:32:07.132876+0000","last_active":"2023-10-06T02:32:07.136118+0000","last_peered":"2023-10-06T02:32:07.136118+0000","last_clean":"2023-10-06T02:32:07.136118+0000","last_became_active":"2023-10-06T02:32:07.132295+0000","last_became_peered":"2023-10-06T02:32:07.132295+0000","last_unstale":"2023-10-06T02:32:07.136118+0000","last_undegraded":"2023-10-06T02:32:07.136118+0000","last_fullsized":"2023-10-06T02:32:07.136118+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_clean_scrub_stamp":"2023-10-06T02:32:06.100240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:27.273877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.143394+0000","last_change":"2023-10-06T02:32:08.143394+0000","last_active":"2023-10-06T02:32:08.143394+0000","last_peered":"2023-10-06T02:32:08.143394+0000","last_clean":"2023-10-06T02:32:08.143394+0000","last_became_active":"2023-10-06T02:32:08.143054+0000","last_became_peered":"2023-10-06T02:32:08.143054+0000","last_unstale":"2023-10-06T02:32:08.143394+0000","last_undegraded":"2023-10-06T02:32:08.143394+0000","last_fullsized":"2023-10-06T02:32:08.143394+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:57:54.863263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-06T02:32:07.132654+0000","last_change":"2023-10-06T02:32:07.128845+0000","last_active":"2023-10-06T02:32:07.132654+0000","last_peered":"2023-10-06T02:32:07.132654+0000","last_clean":"2023-10-06T02:32:07.132654+0000","last_became_active":"2023-10-06T02:32:07.128542+0000","last_became_peered":"2023-10-06T02:32:07.128542+0000","last_unstale":"2023-10-06T02:32:07.132654+0000","last_undegraded":"2023-10-06T02:32:07.132654+0000","last_fullsized":"2023-10-06T02:32:07.132654+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_clean_scrub_stamp":"2023-10-06T02:32:06.100240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:27:18.795673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.148804+0000","last_change":"2023-10-06T02:32:08.148804+0000","last_active":"2023-10-06T02:32:08.148804+0000","last_peered":"2023-10-06T02:32:08.148804+0000","last_clean":"2023-10-06T02:32:08.148804+0000","last_became_active":"2023-10-06T02:32:08.148431+0000","last_became_peered":"2023-10-06T02:32:08.148431+0000","last_unstale":"2023-10-06T02:32:08.148804+0000","last_undegraded":"2023-10-06T02:32:08.148804+0000","last_fullsized":"2023-10-06T02:32:08.148804+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:44:32.474746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.10","version":"168'8","reported_seq":17,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.150266+0000","last_change":"2023-10-06T02:32:07.134817+0000","last_active":"2023-10-06T02:32:08.150266+0000","last_peered":"2023-10-06T02:32:08.150266+0000","last_clean":"2023-10-06T02:32:08.150266+0000","last_became_active":"2023-10-06T02:32:07.134517+0000","last_became_peered":"2023-10-06T02:32:07.134517+0000","last_unstale":"2023-10-06T02:32:08.150266+0000","last_undegraded":"2023-10-06T02:32:08.150266+0000","last_fullsized":"2023-10-06T02:32:08.150266+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_clean_scrub_stamp":"2023-10-06T02:32:06.100240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:38:29.855364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.140270+0000","last_change":"2023-10-06T02:32:08.140270+0000","last_active":"2023-10-06T02:32:08.140270+0000","last_peered":"2023-10-06T02:32:08.140270+0000","last_clean":"2023-10-06T02:32:08.140270+0000","last_became_active":"2023-10-06T02:32:08.139999+0000","last_became_peered":"2023-10-06T02:32:08.139999+0000","last_unstale":"2023-10-06T02:32:08.140270+0000","last_undegraded":"2023-10-06T02:32:08.140270+0000","last_fullsized":"2023-10-06T02:32:08.140270+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:38:36.563684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.f","version":"168'1","reported_seq":11,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.149809+0000","last_change":"2023-10-06T02:32:07.128591+0000","last_active":"2023-10-06T02:32:08.149809+0000","last_peered":"2023-10-06T02:32:08.149809+0000","last_clean":"2023-10-06T02:32:08.149809+0000","last_became_active":"2023-10-06T02:32:07.128323+0000","last_became_peered":"2023-10-06T02:32:07.128323+0000","last_unstale":"2023-10-06T02:32:08.149809+0000","last_undegraded":"2023-10-06T02:32:08.149809+0000","last_fullsized":"2023-10-06T02:32:08.149809+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_clean_scrub_stamp":"2023-10-06T02:32:06.100240+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:50:46.184905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.148497+0000","last_change":"2023-10-06T02:32:08.146338+0000","last_active":"2023-10-06T02:32:08.148497+0000","last_peered":"2023-10-06T02:32:08.148497+0000","last_clean":"2023-10-06T02:32:08.148497+0000","last_became_active":"2023-10-06T02:32:08.145847+0000","last_became_peered":"2023-10-06T02:32:08.145847+0000","last_unstale":"2023-10-06T02:32:08.148497+0000","last_undegraded":"2023-10-06T02:32:08.148497+0000","last_fullsized":"2023-10-06T02:32:08.148497+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:34:14.560103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.e","version":"168'2","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.151272+0000","last_change":"2023-10-06T02:32:07.129750+0000","last_active":"2023-10-06T02:32:08.151272+0000","last_peered":"2023-10-06T02:32:08.151272+0000","last_clean":"2023-10-06T02:32:08.151272+0000","last_became_active":"2023-10-06T02:32:07.129332+0000","last_became_peered":"2023-10-06T02:32:07.129332+0000","last_unstale":"2023-10-06T02:32:08.151272+0000","last_undegraded":"2023-10-06T02:32:08.151272+0000","last_fullsized":"2023-10-06T02:32:08.151272+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_clean_scrub_stamp":"2023-10-06T02:32:06.100240+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:20:11.521570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.133461+0000","last_change":"2023-10-06T02:32:08.131643+0000","last_active":"2023-10-06T02:32:08.133461+0000","last_peered":"2023-10-06T02:32:08.133461+0000","last_clean":"2023-10-06T02:32:08.133461+0000","last_became_active":"2023-10-06T02:32:08.131345+0000","last_became_peered":"2023-10-06T02:32:08.131345+0000","last_unstale":"2023-10-06T02:32:08.133461+0000","last_undegraded":"2023-10-06T02:32:08.133461+0000","last_fullsized":"2023-10-06T02:32:08.133461+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:23:50.218257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.d","version":"168'1","reported_seq":11,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.148925+0000","last_change":"2023-10-06T02:32:07.130842+0000","last_active":"2023-10-06T02:32:08.148925+0000","last_peered":"2023-10-06T02:32:08.148925+0000","last_clean":"2023-10-06T02:32:08.148925+0000","last_became_active":"2023-10-06T02:32:07.130584+0000","last_became_peered":"2023-10-06T02:32:07.130584+0000","last_unstale":"2023-10-06T02:32:08.148925+0000","last_undegraded":"2023-10-06T02:32:08.148925+0000","last_fullsized":"2023-10-06T02:32:08.148925+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_clean_scrub_stamp":"2023-10-06T02:32:06.100240+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:00:51.603032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.b","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.140504+0000","last_change":"2023-10-06T02:32:08.140504+0000","last_active":"2023-10-06T02:32:08.140504+0000","last_peered":"2023-10-06T02:32:08.140504+0000","last_clean":"2023-10-06T02:32:08.140504+0000","last_became_active":"2023-10-06T02:32:08.140240+0000","last_became_peered":"2023-10-06T02:32:08.140240+0000","last_unstale":"2023-10-06T02:32:08.140504+0000","last_undegraded":"2023-10-06T02:32:08.140504+0000","last_fullsized":"2023-10-06T02:32:08.140504+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:34:01.099306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-06T02:32:07.126516+0000","last_change":"2023-10-06T02:32:07.124784+0000","last_active":"2023-10-06T02:32:07.126516+0000","last_peered":"2023-10-06T02:32:07.126516+0000","last_clean":"2023-10-06T02:32:07.126516+0000","last_became_active":"2023-10-06T02:32:07.124455+0000","last_became_peered":"2023-10-06T02:32:07.124455+0000","last_unstale":"2023-10-06T02:32:07.126516+0000","last_undegraded":"2023-10-06T02:32:07.126516+0000","last_fullsized":"2023-10-06T02:32:07.126516+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_clean_scrub_stamp":"2023-10-06T02:32:06.100240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:53.114922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.139874+0000","last_change":"2023-10-06T02:32:08.138347+0000","last_active":"2023-10-06T02:32:08.139874+0000","last_peered":"2023-10-06T02:32:08.139874+0000","last_clean":"2023-10-06T02:32:08.139874+0000","last_became_active":"2023-10-06T02:32:08.137515+0000","last_became_peered":"2023-10-06T02:32:08.137515+0000","last_unstale":"2023-10-06T02:32:08.139874+0000","last_undegraded":"2023-10-06T02:32:08.139874+0000","last_fullsized":"2023-10-06T02:32:08.139874+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:10:41.774263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"168'2","reported_seq":11,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.140883+0000","last_change":"2023-10-06T02:32:07.129048+0000","last_active":"2023-10-06T02:32:08.140883+0000","last_peered":"2023-10-06T02:32:08.140883+0000","last_clean":"2023-10-06T02:32:08.140883+0000","last_became_active":"2023-10-06T02:32:07.128786+0000","last_became_peered":"2023-10-06T02:32:07.128786+0000","last_unstale":"2023-10-06T02:32:08.140883+0000","last_undegraded":"2023-10-06T02:32:08.140883+0000","last_fullsized":"2023-10-06T02:32:08.140883+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_clean_scrub_stamp":"2023-10-06T02:32:06.100240+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:34:00.095848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.d","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.138174+0000","last_change":"2023-10-06T02:32:08.138174+0000","last_active":"2023-10-06T02:32:08.138174+0000","last_peered":"2023-10-06T02:32:08.138174+0000","last_clean":"2023-10-06T02:32:08.138174+0000","last_became_active":"2023-10-06T02:32:08.137500+0000","last_became_peered":"2023-10-06T02:32:08.137500+0000","last_unstale":"2023-10-06T02:32:08.138174+0000","last_undegraded":"2023-10-06T02:32:08.138174+0000","last_fullsized":"2023-10-06T02:32:08.138174+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:30:00.558795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":10,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.118960+0000","last_change":"2023-10-06T02:32:07.131601+0000","last_active":"2023-10-06T02:32:08.118960+0000","last_peered":"2023-10-06T02:32:08.118960+0000","last_clean":"2023-10-06T02:32:08.118960+0000","last_became_active":"2023-10-06T02:32:07.131286+0000","last_became_peered":"2023-10-06T02:32:07.131286+0000","last_unstale":"2023-10-06T02:32:08.118960+0000","last_undegraded":"2023-10-06T02:32:08.118960+0000","last_fullsized":"2023-10-06T02:32:08.118960+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_clean_scrub_stamp":"2023-10-06T02:32:06.100240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:27:05.038453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.e","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.148747+0000","last_change":"2023-10-06T02:32:08.148747+0000","last_active":"2023-10-06T02:32:08.148747+0000","last_peered":"2023-10-06T02:32:08.148747+0000","last_clean":"2023-10-06T02:32:08.148747+0000","last_became_active":"2023-10-06T02:32:08.148449+0000","last_became_peered":"2023-10-06T02:32:08.148449+0000","last_unstale":"2023-10-06T02:32:08.148747+0000","last_undegraded":"2023-10-06T02:32:08.148747+0000","last_fullsized":"2023-10-06T02:32:08.148747+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:26:47.123088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":10,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.129723+0000","last_change":"2023-10-06T02:32:07.135376+0000","last_active":"2023-10-06T02:32:08.129723+0000","last_peered":"2023-10-06T02:32:08.129723+0000","last_clean":"2023-10-06T02:32:08.129723+0000","last_became_active":"2023-10-06T02:32:07.134293+0000","last_became_peered":"2023-10-06T02:32:07.134293+0000","last_unstale":"2023-10-06T02:32:08.129723+0000","last_undegraded":"2023-10-06T02:32:08.129723+0000","last_fullsized":"2023-10-06T02:32:08.129723+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_clean_scrub_stamp":"2023-10-06T02:32:06.100240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:47:09.341380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.f","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.138032+0000","last_change":"2023-10-06T02:32:08.138032+0000","last_active":"2023-10-06T02:32:08.138032+0000","last_peered":"2023-10-06T02:32:08.138032+0000","last_clean":"2023-10-06T02:32:08.138032+0000","last_became_active":"2023-10-06T02:32:08.134106+0000","last_became_peered":"2023-10-06T02:32:08.134106+0000","last_unstale":"2023-10-06T02:32:08.138032+0000","last_undegraded":"2023-10-06T02:32:08.138032+0000","last_fullsized":"2023-10-06T02:32:08.138032+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:34:28.727551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":10,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.118329+0000","last_change":"2023-10-06T02:32:07.132018+0000","last_active":"2023-10-06T02:32:08.118329+0000","last_peered":"2023-10-06T02:32:08.118329+0000","last_clean":"2023-10-06T02:32:08.118329+0000","last_became_active":"2023-10-06T02:32:07.131735+0000","last_became_peered":"2023-10-06T02:32:07.131735+0000","last_unstale":"2023-10-06T02:32:08.118329+0000","last_undegraded":"2023-10-06T02:32:08.118329+0000","last_fullsized":"2023-10-06T02:32:08.118329+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_clean_scrub_stamp":"2023-10-06T02:32:06.100240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:58:21.701178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.139533+0000","last_change":"2023-10-06T02:32:08.139533+0000","last_active":"2023-10-06T02:32:08.139533+0000","last_peered":"2023-10-06T02:32:08.139533+0000","last_clean":"2023-10-06T02:32:08.139533+0000","last_became_active":"2023-10-06T02:32:08.139248+0000","last_became_peered":"2023-10-06T02:32:08.139248+0000","last_unstale":"2023-10-06T02:32:08.139533+0000","last_undegraded":"2023-10-06T02:32:08.139533+0000","last_fullsized":"2023-10-06T02:32:08.139533+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:54.118380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.7","version":"168'2","reported_seq":11,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.149045+0000","last_change":"2023-10-06T02:32:07.132994+0000","last_active":"2023-10-06T02:32:08.149045+0000","last_peered":"2023-10-06T02:32:08.149045+0000","last_clean":"2023-10-06T02:32:08.149045+0000","last_became_active":"2023-10-06T02:32:07.129783+0000","last_became_peered":"2023-10-06T02:32:07.129783+0000","last_unstale":"2023-10-06T02:32:08.149045+0000","last_undegraded":"2023-10-06T02:32:08.149045+0000","last_fullsized":"2023-10-06T02:32:08.149045+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_clean_scrub_stamp":"2023-10-06T02:32:06.100240+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:18:48.153012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.138250+0000","last_change":"2023-10-06T02:32:08.138250+0000","last_active":"2023-10-06T02:32:08.138250+0000","last_peered":"2023-10-06T02:32:08.138250+0000","last_clean":"2023-10-06T02:32:08.138250+0000","last_became_active":"2023-10-06T02:32:08.137837+0000","last_became_peered":"2023-10-06T02:32:08.137837+0000","last_unstale":"2023-10-06T02:32:08.138250+0000","last_undegraded":"2023-10-06T02:32:08.138250+0000","last_fullsized":"2023-10-06T02:32:08.138250+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:50.611252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-06T02:32:07.139099+0000","last_change":"2023-10-06T02:32:07.130291+0000","last_active":"2023-10-06T02:32:07.139099+0000","last_peered":"2023-10-06T02:32:07.139099+0000","last_clean":"2023-10-06T02:32:07.139099+0000","last_became_active":"2023-10-06T02:32:07.130018+0000","last_became_peered":"2023-10-06T02:32:07.130018+0000","last_unstale":"2023-10-06T02:32:07.139099+0000","last_undegraded":"2023-10-06T02:32:07.139099+0000","last_fullsized":"2023-10-06T02:32:07.139099+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_clean_scrub_stamp":"2023-10-06T02:32:06.100240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:22:21.025639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.144249+0000","last_change":"2023-10-06T02:32:08.144249+0000","last_active":"2023-10-06T02:32:08.144249+0000","last_peered":"2023-10-06T02:32:08.144249+0000","last_clean":"2023-10-06T02:32:08.144249+0000","last_became_active":"2023-10-06T02:32:08.144002+0000","last_became_peered":"2023-10-06T02:32:08.144002+0000","last_unstale":"2023-10-06T02:32:08.144249+0000","last_undegraded":"2023-10-06T02:32:08.144249+0000","last_fullsized":"2023-10-06T02:32:08.144249+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:48:18.282510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":10,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.127701+0000","last_change":"2023-10-06T02:32:07.138800+0000","last_active":"2023-10-06T02:32:08.127701+0000","last_peered":"2023-10-06T02:32:08.127701+0000","last_clean":"2023-10-06T02:32:08.127701+0000","last_became_active":"2023-10-06T02:32:07.138483+0000","last_became_peered":"2023-10-06T02:32:07.138483+0000","last_unstale":"2023-10-06T02:32:08.127701+0000","last_undegraded":"2023-10-06T02:32:08.127701+0000","last_fullsized":"2023-10-06T02:32:08.127701+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_clean_scrub_stamp":"2023-10-06T02:32:06.100240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:08:44.456006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.147143+0000","last_change":"2023-10-06T02:32:08.138461+0000","last_active":"2023-10-06T02:32:08.147143+0000","last_peered":"2023-10-06T02:32:08.147143+0000","last_clean":"2023-10-06T02:32:08.147143+0000","last_became_active":"2023-10-06T02:32:08.138055+0000","last_became_peered":"2023-10-06T02:32:08.138055+0000","last_unstale":"2023-10-06T02:32:08.147143+0000","last_undegraded":"2023-10-06T02:32:08.147143+0000","last_fullsized":"2023-10-06T02:32:08.147143+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:18:01.461302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":10,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.118354+0000","last_change":"2023-10-06T02:32:07.133367+0000","last_active":"2023-10-06T02:32:08.118354+0000","last_peered":"2023-10-06T02:32:08.118354+0000","last_clean":"2023-10-06T02:32:08.118354+0000","last_became_active":"2023-10-06T02:32:07.132604+0000","last_became_peered":"2023-10-06T02:32:07.132604+0000","last_unstale":"2023-10-06T02:32:08.118354+0000","last_undegraded":"2023-10-06T02:32:08.118354+0000","last_fullsized":"2023-10-06T02:32:08.118354+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_clean_scrub_stamp":"2023-10-06T02:32:06.100240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:27.273877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.137583+0000","last_change":"2023-10-06T02:32:08.137583+0000","last_active":"2023-10-06T02:32:08.137583+0000","last_peered":"2023-10-06T02:32:08.137583+0000","last_clean":"2023-10-06T02:32:08.137583+0000","last_became_active":"2023-10-06T02:32:08.136950+0000","last_became_peered":"2023-10-06T02:32:08.136950+0000","last_unstale":"2023-10-06T02:32:08.137583+0000","last_undegraded":"2023-10-06T02:32:08.137583+0000","last_fullsized":"2023-10-06T02:32:08.137583+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:56:28.297716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-06T02:32:07.139034+0000","last_change":"2023-10-06T02:32:07.129126+0000","last_active":"2023-10-06T02:32:07.139034+0000","last_peered":"2023-10-06T02:32:07.139034+0000","last_clean":"2023-10-06T02:32:07.139034+0000","last_became_active":"2023-10-06T02:32:07.128851+0000","last_became_peered":"2023-10-06T02:32:07.128851+0000","last_unstale":"2023-10-06T02:32:07.139034+0000","last_undegraded":"2023-10-06T02:32:07.139034+0000","last_fullsized":"2023-10-06T02:32:07.139034+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_clean_scrub_stamp":"2023-10-06T02:32:06.100240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:05:23.675738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.138375+0000","last_change":"2023-10-06T02:32:08.138375+0000","last_active":"2023-10-06T02:32:08.138375+0000","last_peered":"2023-10-06T02:32:08.138375+0000","last_clean":"2023-10-06T02:32:08.138375+0000","last_became_active":"2023-10-06T02:32:08.137817+0000","last_became_peered":"2023-10-06T02:32:08.137817+0000","last_unstale":"2023-10-06T02:32:08.138375+0000","last_undegraded":"2023-10-06T02:32:08.138375+0000","last_fullsized":"2023-10-06T02:32:08.138375+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:22:30.522945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-06T02:32:07.135257+0000","last_change":"2023-10-06T02:32:07.134003+0000","last_active":"2023-10-06T02:32:07.135257+0000","last_peered":"2023-10-06T02:32:07.135257+0000","last_clean":"2023-10-06T02:32:07.135257+0000","last_became_active":"2023-10-06T02:32:07.133596+0000","last_became_peered":"2023-10-06T02:32:07.133596+0000","last_unstale":"2023-10-06T02:32:07.135257+0000","last_undegraded":"2023-10-06T02:32:07.135257+0000","last_fullsized":"2023-10-06T02:32:07.135257+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_clean_scrub_stamp":"2023-10-06T02:32:06.100240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:00:51.603032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.6","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.145520+0000","last_change":"2023-10-06T02:32:08.145520+0000","last_active":"2023-10-06T02:32:08.145520+0000","last_peered":"2023-10-06T02:32:08.145520+0000","last_clean":"2023-10-06T02:32:08.145520+0000","last_became_active":"2023-10-06T02:32:08.145250+0000","last_became_peered":"2023-10-06T02:32:08.145250+0000","last_unstale":"2023-10-06T02:32:08.145520+0000","last_undegraded":"2023-10-06T02:32:08.145520+0000","last_fullsized":"2023-10-06T02:32:08.145520+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:48:53.198148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":10,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.119755+0000","last_change":"2023-10-06T02:32:07.133488+0000","last_active":"2023-10-06T02:32:08.119755+0000","last_peered":"2023-10-06T02:32:08.119755+0000","last_clean":"2023-10-06T02:32:08.119755+0000","last_became_active":"2023-10-06T02:32:07.132869+0000","last_became_peered":"2023-10-06T02:32:07.132869+0000","last_unstale":"2023-10-06T02:32:08.119755+0000","last_undegraded":"2023-10-06T02:32:08.119755+0000","last_fullsized":"2023-10-06T02:32:08.119755+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_clean_scrub_stamp":"2023-10-06T02:32:06.100240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:22:21.226937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.7","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.143903+0000","last_change":"2023-10-06T02:32:08.143903+0000","last_active":"2023-10-06T02:32:08.143903+0000","last_peered":"2023-10-06T02:32:08.143903+0000","last_clean":"2023-10-06T02:32:08.143903+0000","last_became_active":"2023-10-06T02:32:08.143593+0000","last_became_peered":"2023-10-06T02:32:08.143593+0000","last_unstale":"2023-10-06T02:32:08.143903+0000","last_undegraded":"2023-10-06T02:32:08.143903+0000","last_fullsized":"2023-10-06T02:32:08.143903+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:33:05.856323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-06T02:32:07.137160+0000","last_change":"2023-10-06T02:32:07.134437+0000","last_active":"2023-10-06T02:32:07.137160+0000","last_peered":"2023-10-06T02:32:07.137160+0000","last_clean":"2023-10-06T02:32:07.137160+0000","last_became_active":"2023-10-06T02:32:07.134183+0000","last_became_peered":"2023-10-06T02:32:07.134183+0000","last_unstale":"2023-10-06T02:32:07.137160+0000","last_undegraded":"2023-10-06T02:32:07.137160+0000","last_fullsized":"2023-10-06T02:32:07.137160+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_clean_scrub_stamp":"2023-10-06T02:32:06.100240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:27:21.788539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.147267+0000","last_change":"2023-10-06T02:32:08.141501+0000","last_active":"2023-10-06T02:32:08.147267+0000","last_peered":"2023-10-06T02:32:08.147267+0000","last_clean":"2023-10-06T02:32:08.147267+0000","last_became_active":"2023-10-06T02:32:08.141207+0000","last_became_peered":"2023-10-06T02:32:08.141207+0000","last_unstale":"2023-10-06T02:32:08.147267+0000","last_undegraded":"2023-10-06T02:32:08.147267+0000","last_fullsized":"2023-10-06T02:32:08.147267+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:34.296319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1e","version":"168'1","reported_seq":11,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.147238+0000","last_change":"2023-10-06T02:32:07.129597+0000","last_active":"2023-10-06T02:32:08.147238+0000","last_peered":"2023-10-06T02:32:08.147238+0000","last_clean":"2023-10-06T02:32:08.147238+0000","last_became_active":"2023-10-06T02:32:07.128946+0000","last_became_peered":"2023-10-06T02:32:07.128946+0000","last_unstale":"2023-10-06T02:32:08.147238+0000","last_undegraded":"2023-10-06T02:32:08.147238+0000","last_fullsized":"2023-10-06T02:32:08.147238+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_clean_scrub_stamp":"2023-10-06T02:32:06.100240+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:44:00.675705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":134,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-06T02:32:07.116964+0000","last_change":"2023-10-06T02:23:01.992935+0000","last_active":"2023-10-06T02:32:07.116964+0000","last_peered":"2023-10-06T02:32:07.116964+0000","last_clean":"2023-10-06T02:32:07.116964+0000","last_became_active":"2023-10-06T02:23:01.992568+0000","last_became_peered":"2023-10-06T02:23:01.992568+0000","last_unstale":"2023-10-06T02:32:07.116964+0000","last_undegraded":"2023-10-06T02:32:07.116964+0000","last_fullsized":"2023-10-06T02:32:07.116964+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:06.212251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.129549+0000","last_change":"2023-10-06T02:32:08.127932+0000","last_active":"2023-10-06T02:32:08.129549+0000","last_peered":"2023-10-06T02:32:08.129549+0000","last_clean":"2023-10-06T02:32:08.129549+0000","last_became_active":"2023-10-06T02:32:08.127648+0000","last_became_peered":"2023-10-06T02:32:08.127648+0000","last_unstale":"2023-10-06T02:32:08.129549+0000","last_undegraded":"2023-10-06T02:32:08.129549+0000","last_fullsized":"2023-10-06T02:32:08.129549+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:38:40.218933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.1f","version":"168'4","reported_seq":14,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.149392+0000","last_change":"2023-10-06T02:32:07.140369+0000","last_active":"2023-10-06T02:32:08.149392+0000","last_peered":"2023-10-06T02:32:08.149392+0000","last_clean":"2023-10-06T02:32:08.149392+0000","last_became_active":"2023-10-06T02:32:07.140046+0000","last_became_peered":"2023-10-06T02:32:07.140046+0000","last_unstale":"2023-10-06T02:32:08.149392+0000","last_undegraded":"2023-10-06T02:32:08.149392+0000","last_fullsized":"2023-10-06T02:32:08.149392+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_clean_scrub_stamp":"2023-10-06T02:32:06.100240+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:58:00.220771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":135,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-06T02:32:07.126621+0000","last_change":"2023-10-06T02:23:01.989934+0000","last_active":"2023-10-06T02:32:07.126621+0000","last_peered":"2023-10-06T02:32:07.126621+0000","last_clean":"2023-10-06T02:32:07.126621+0000","last_became_active":"2023-10-06T02:23:01.989706+0000","last_became_peered":"2023-10-06T02:23:01.989706+0000","last_unstale":"2023-10-06T02:32:07.126621+0000","last_undegraded":"2023-10-06T02:32:07.126621+0000","last_fullsized":"2023-10-06T02:32:07.126621+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:06.212251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.1b","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.129297+0000","last_change":"2023-10-06T02:32:08.129297+0000","last_active":"2023-10-06T02:32:08.129297+0000","last_peered":"2023-10-06T02:32:08.129297+0000","last_clean":"2023-10-06T02:32:08.129297+0000","last_became_active":"2023-10-06T02:32:08.128885+0000","last_became_peered":"2023-10-06T02:32:08.128885+0000","last_unstale":"2023-10-06T02:32:08.129297+0000","last_undegraded":"2023-10-06T02:32:08.129297+0000","last_fullsized":"2023-10-06T02:32:08.129297+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:20:13.257560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1c","version":"168'2","reported_seq":11,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.144557+0000","last_change":"2023-10-06T02:32:07.133974+0000","last_active":"2023-10-06T02:32:08.144557+0000","last_peered":"2023-10-06T02:32:08.144557+0000","last_clean":"2023-10-06T02:32:08.144557+0000","last_became_active":"2023-10-06T02:32:07.133538+0000","last_became_peered":"2023-10-06T02:32:07.133538+0000","last_unstale":"2023-10-06T02:32:08.144557+0000","last_undegraded":"2023-10-06T02:32:08.144557+0000","last_fullsized":"2023-10-06T02:32:08.144557+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_clean_scrub_stamp":"2023-10-06T02:32:06.100240+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:19:59.093429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":128,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-06T02:32:07.130536+0000","last_change":"2023-10-06T02:23:02.054069+0000","last_active":"2023-10-06T02:32:07.130536+0000","last_peered":"2023-10-06T02:32:07.130536+0000","last_clean":"2023-10-06T02:32:07.130536+0000","last_became_active":"2023-10-06T02:23:02.053739+0000","last_became_peered":"2023-10-06T02:23:02.053739+0000","last_unstale":"2023-10-06T02:32:07.130536+0000","last_undegraded":"2023-10-06T02:32:07.130536+0000","last_fullsized":"2023-10-06T02:32:07.130536+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:57:48.822608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1e","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.146820+0000","last_change":"2023-10-06T02:32:08.143629+0000","last_active":"2023-10-06T02:32:08.146820+0000","last_peered":"2023-10-06T02:32:08.146820+0000","last_clean":"2023-10-06T02:32:08.146820+0000","last_became_active":"2023-10-06T02:32:08.143314+0000","last_became_peered":"2023-10-06T02:32:08.143314+0000","last_unstale":"2023-10-06T02:32:08.146820+0000","last_undegraded":"2023-10-06T02:32:08.146820+0000","last_fullsized":"2023-10-06T02:32:08.146820+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:20:12.525028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-06T02:32:07.139952+0000","last_change":"2023-10-06T02:32:07.136501+0000","last_active":"2023-10-06T02:32:07.139952+0000","last_peered":"2023-10-06T02:32:07.139952+0000","last_clean":"2023-10-06T02:32:07.139952+0000","last_became_active":"2023-10-06T02:32:07.135756+0000","last_became_peered":"2023-10-06T02:32:07.135756+0000","last_unstale":"2023-10-06T02:32:07.139952+0000","last_undegraded":"2023-10-06T02:32:07.139952+0000","last_fullsized":"2023-10-06T02:32:07.139952+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_clean_scrub_stamp":"2023-10-06T02:32:06.100240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:44:23.676780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":128,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-06T02:32:07.126874+0000","last_change":"2023-10-06T02:23:01.990773+0000","last_active":"2023-10-06T02:32:07.126874+0000","last_peered":"2023-10-06T02:32:07.126874+0000","last_clean":"2023-10-06T02:32:07.126874+0000","last_became_active":"2023-10-06T02:23:01.990511+0000","last_became_peered":"2023-10-06T02:23:01.990511+0000","last_unstale":"2023-10-06T02:32:07.126874+0000","last_undegraded":"2023-10-06T02:32:07.126874+0000","last_fullsized":"2023-10-06T02:32:07.126874+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:42:54.797731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1d","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.145670+0000","last_change":"2023-10-06T02:32:08.134710+0000","last_active":"2023-10-06T02:32:08.145670+0000","last_peered":"2023-10-06T02:32:08.145670+0000","last_clean":"2023-10-06T02:32:08.145670+0000","last_became_active":"2023-10-06T02:32:08.133600+0000","last_became_peered":"2023-10-06T02:32:08.133600+0000","last_unstale":"2023-10-06T02:32:08.145670+0000","last_undegraded":"2023-10-06T02:32:08.145670+0000","last_fullsized":"2023-10-06T02:32:08.145670+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:50.611252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1a","version":"168'1","reported_seq":10,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.123003+0000","last_change":"2023-10-06T02:32:07.129097+0000","last_active":"2023-10-06T02:32:08.123003+0000","last_peered":"2023-10-06T02:32:08.123003+0000","last_clean":"2023-10-06T02:32:08.123003+0000","last_became_active":"2023-10-06T02:32:07.128807+0000","last_became_peered":"2023-10-06T02:32:07.128807+0000","last_unstale":"2023-10-06T02:32:08.123003+0000","last_undegraded":"2023-10-06T02:32:08.123003+0000","last_fullsized":"2023-10-06T02:32:08.123003+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_clean_scrub_stamp":"2023-10-06T02:32:06.100240+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:53.451053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"76'128","reported_seq":320,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-06T02:32:07.127016+0000","last_change":"2023-10-06T02:22:55.032519+0000","last_active":"2023-10-06T02:32:07.127016+0000","last_peered":"2023-10-06T02:32:07.127016+0000","last_clean":"2023-10-06T02:32:07.127016+0000","last_became_active":"2023-10-06T02:22:55.031367+0000","last_became_peered":"2023-10-06T02:22:55.031367+0000","last_unstale":"2023-10-06T02:32:07.127016+0000","last_undegraded":"2023-10-06T02:32:07.127016+0000","last_fullsized":"2023-10-06T02:32:07.127016+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:22:53.309097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:22:53.309097+0000","last_clean_scrub_stamp":"2023-10-06T02:22:53.309097+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:53:46.520909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":159,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.118706+0000","last_change":"2023-10-06T02:23:01.988140+0000","last_active":"2023-10-06T02:32:08.118706+0000","last_peered":"2023-10-06T02:32:08.118706+0000","last_clean":"2023-10-06T02:32:08.118706+0000","last_became_active":"2023-10-06T02:23:01.987769+0000","last_became_peered":"2023-10-06T02:23:01.987769+0000","last_unstale":"2023-10-06T02:32:08.118706+0000","last_undegraded":"2023-10-06T02:32:08.118706+0000","last_fullsized":"2023-10-06T02:32:08.118706+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:20:08.391912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.1f","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.145160+0000","last_change":"2023-10-06T02:32:08.145160+0000","last_active":"2023-10-06T02:32:08.145160+0000","last_peered":"2023-10-06T02:32:08.145160+0000","last_clean":"2023-10-06T02:32:08.145160+0000","last_became_active":"2023-10-06T02:32:08.144918+0000","last_became_peered":"2023-10-06T02:32:08.144918+0000","last_unstale":"2023-10-06T02:32:08.145160+0000","last_undegraded":"2023-10-06T02:32:08.145160+0000","last_fullsized":"2023-10-06T02:32:08.145160+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:15:37.227204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-06T02:32:07.136072+0000","last_change":"2023-10-06T02:32:07.132943+0000","last_active":"2023-10-06T02:32:07.136072+0000","last_peered":"2023-10-06T02:32:07.136072+0000","last_clean":"2023-10-06T02:32:07.136072+0000","last_became_active":"2023-10-06T02:32:07.132598+0000","last_became_peered":"2023-10-06T02:32:07.132598+0000","last_unstale":"2023-10-06T02:32:07.136072+0000","last_undegraded":"2023-10-06T02:32:07.136072+0000","last_fullsized":"2023-10-06T02:32:07.136072+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_clean_scrub_stamp":"2023-10-06T02:32:06.100240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:44:00.675705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":145,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-06T02:32:07.112141+0000","last_change":"2023-10-06T02:23:02.054338+0000","last_active":"2023-10-06T02:32:07.112141+0000","last_peered":"2023-10-06T02:32:07.112141+0000","last_clean":"2023-10-06T02:32:07.112141+0000","last_became_active":"2023-10-06T02:23:02.053991+0000","last_became_peered":"2023-10-06T02:23:02.053991+0000","last_unstale":"2023-10-06T02:32:07.112141+0000","last_undegraded":"2023-10-06T02:32:07.112141+0000","last_fullsized":"2023-10-06T02:32:07.112141+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:39:30.192838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.1c","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.145534+0000","last_change":"2023-10-06T02:32:08.140801+0000","last_active":"2023-10-06T02:32:08.145534+0000","last_peered":"2023-10-06T02:32:08.145534+0000","last_clean":"2023-10-06T02:32:08.145534+0000","last_became_active":"2023-10-06T02:32:08.140525+0000","last_became_peered":"2023-10-06T02:32:08.140525+0000","last_unstale":"2023-10-06T02:32:08.145534+0000","last_undegraded":"2023-10-06T02:32:08.145534+0000","last_fullsized":"2023-10-06T02:32:08.145534+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:50:47.188363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1b","version":"168'1","reported_seq":10,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.128535+0000","last_change":"2023-10-06T02:32:07.134198+0000","last_active":"2023-10-06T02:32:08.128535+0000","last_peered":"2023-10-06T02:32:08.128535+0000","last_clean":"2023-10-06T02:32:08.128535+0000","last_became_active":"2023-10-06T02:32:07.133937+0000","last_became_peered":"2023-10-06T02:32:07.133937+0000","last_unstale":"2023-10-06T02:32:08.128535+0000","last_undegraded":"2023-10-06T02:32:08.128535+0000","last_fullsized":"2023-10-06T02:32:08.128535+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_clean_scrub_stamp":"2023-10-06T02:32:06.100240+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:23:49.214799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":159,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.120156+0000","last_change":"2023-10-06T02:23:01.988631+0000","last_active":"2023-10-06T02:32:08.120156+0000","last_peered":"2023-10-06T02:32:08.120156+0000","last_clean":"2023-10-06T02:32:08.120156+0000","last_became_active":"2023-10-06T02:23:01.988303+0000","last_became_peered":"2023-10-06T02:23:01.988303+0000","last_unstale":"2023-10-06T02:32:08.120156+0000","last_undegraded":"2023-10-06T02:32:08.120156+0000","last_fullsized":"2023-10-06T02:32:08.120156+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:38.211466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.1a","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.124115+0000","last_change":"2023-10-06T02:32:08.124115+0000","last_active":"2023-10-06T02:32:08.124115+0000","last_peered":"2023-10-06T02:32:08.124115+0000","last_clean":"2023-10-06T02:32:08.124115+0000","last_became_active":"2023-10-06T02:32:08.123772+0000","last_became_peered":"2023-10-06T02:32:08.123772+0000","last_unstale":"2023-10-06T02:32:08.124115+0000","last_undegraded":"2023-10-06T02:32:08.124115+0000","last_fullsized":"2023-10-06T02:32:08.124115+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:29:57.398331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1d","version":"168'2","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.141999+0000","last_change":"2023-10-06T02:32:07.127620+0000","last_active":"2023-10-06T02:32:08.141999+0000","last_peered":"2023-10-06T02:32:08.141999+0000","last_clean":"2023-10-06T02:32:08.141999+0000","last_became_active":"2023-10-06T02:32:07.127314+0000","last_became_peered":"2023-10-06T02:32:07.127314+0000","last_unstale":"2023-10-06T02:32:08.141999+0000","last_undegraded":"2023-10-06T02:32:08.141999+0000","last_fullsized":"2023-10-06T02:32:08.141999+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_clean_scrub_stamp":"2023-10-06T02:32:06.100240+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:26:50.885561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":135,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-06T02:32:07.126690+0000","last_change":"2023-10-06T02:23:02.054039+0000","last_active":"2023-10-06T02:32:07.126690+0000","last_peered":"2023-10-06T02:32:07.126690+0000","last_clean":"2023-10-06T02:32:07.126690+0000","last_became_active":"2023-10-06T02:23:02.053742+0000","last_became_peered":"2023-10-06T02:23:02.053742+0000","last_unstale":"2023-10-06T02:32:07.126690+0000","last_undegraded":"2023-10-06T02:32:07.126690+0000","last_fullsized":"2023-10-06T02:32:07.126690+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:58:18.846475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":28,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":27,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574964,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304052,"kb_used_data":2668,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067788,"statfs":{"total":96636764160,"available":96325414912,"internally_reserved":0,"allocated":2732032,"data_stored":1582946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:31:14 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.622,"15min":0.63300000000000001},"min":{"1min":0.35299999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.78300000000000003,"5min":1.169,"15min":1.169},"last":7.9160000000000004},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.624,"15min":0.61199999999999999},"min":{"1min":0.43099999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.91800000000000004,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":11.643000000000001}]},{"osd":1,"last update":"Fri Oct 6 02:31:14 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71399999999999997,"15min":0.69699999999999995},"min":{"1min":0.57899999999999996,"5min":0.45600000000000002,"15min":0.40300000000000002},"max":{"1min":0.86199999999999999,"5min":1.1459999999999999,"15min":1.147},"last":1.268},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65900000000000003,"15min":0.67100000000000004},"min":{"1min":0.52100000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83899999999999997,"5min":1.129,"15min":1.129},"last":3.0219999999999998}]},{"osd":2,"last update":"Fri Oct 6 02:31:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.65500000000000003,"15min":0.68300000000000005},"min":{"1min":0.46700000000000003,"5min":0.375,"15min":0.375},"max":{"1min":0.81699999999999995,"5min":1.238,"15min":1.238},"last":3.0070000000000001},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.68700000000000006,"15min":0.69099999999999995},"min":{"1min":0.55000000000000004,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.93700000000000006,"5min":1.242,"15min":1.242},"last":12.118}]},{"osd":3,"last update":"Fri Oct 6 02:31:14 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.70799999999999996,"15min":0.72399999999999998},"min":{"1min":0.45500000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.84299999999999997,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":3.073},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.75700000000000001,"15min":0.77200000000000002},"min":{"1min":0.61399999999999999,"5min":0.54500000000000004,"15min":0.497},"max":{"1min":0.95199999999999996,"5min":1.383,"15min":1.383},"last":3.105}]},{"osd":4,"last update":"Fri Oct 6 02:31:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.60299999999999998,"15min":0.66700000000000004},"min":{"1min":0.41799999999999998,"5min":0.41099999999999998,"15min":0.35099999999999998},"max":{"1min":0.91700000000000004,"5min":1.1120000000000001,"15min":1.2849999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.67300000000000004,"15min":0.73899999999999999},"min":{"1min":0.40899999999999997,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.96299999999999997,"5min":1.51,"15min":1.51},"last":8.0060000000000002}]},{"osd":5,"last update":"Fri Oct 6 02:31:14 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.70099999999999996,"15min":0.72499999999999998},"min":{"1min":0.503,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.91900000000000004,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":3.524},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.67000000000000004,"15min":0.71599999999999997},"min":{"1min":0.48999999999999999,"5min":0.442,"15min":0.442},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0289999999999999},"last":3.258}]},{"osd":6,"last update":"Fri Oct 6 02:31:14 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.752,"15min":0.77800000000000002},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.93400000000000005,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":3.4689999999999999},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.77100000000000002,"15min":0.80300000000000005},"min":{"1min":0.628,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.90500000000000003,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":3.4279999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574964,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303472,"kb_used_data":2088,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068368,"statfs":{"total":96636764160,"available":96326008832,"internally_reserved":0,"allocated":2138112,"data_stored":993062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:31:09 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.67400000000000004,"15min":0.66700000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":1.869},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.72099999999999997,"15min":0.70399999999999996},"min":{"1min":0.498,"5min":0.374,"15min":0.374},"max":{"1min":1.04,"5min":1.04,"15min":1.119},"last":1.27}]},{"osd":1,"last update":"Fri Oct 6 02:31:09 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.66800000000000004,"15min":0.65000000000000002},"min":{"1min":0.48699999999999999,"5min":0.433,"15min":0.40699999999999997},"max":{"1min":0.83799999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.66600000000000004,"15min":0.66700000000000004},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.94499999999999995,"5min":0.997,"15min":0.997},"last":1.927}]},{"osd":2,"last update":"Fri Oct 6 02:31:09 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.71599999999999997,"15min":0.70599999999999996},"min":{"1min":0.51000000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.002,"5min":1.0449999999999999,"15min":1.137},"last":1.218},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.72799999999999998,"15min":0.71799999999999997},"min":{"1min":0.47799999999999998,"5min":0.45300000000000001,"15min":0.40300000000000002},"max":{"1min":0.98699999999999999,"5min":1.077,"15min":1.077},"last":1.3149999999999999}]},{"osd":3,"last update":"Fri Oct 6 02:31:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.70099999999999996,"15min":0.67200000000000004},"min":{"1min":0.51500000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.82099999999999995,"5min":1.056,"15min":1.056},"last":1.3620000000000001},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.71399999999999997,"15min":0.70699999999999996},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.45900000000000002},"max":{"1min":0.91600000000000004,"5min":0.97999999999999998,"15min":0.98499999999999999},"last":0.96299999999999997}]},{"osd":4,"last update":"Fri Oct 6 02:31:09 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.66500000000000004,"15min":0.64500000000000002},"min":{"1min":0.377,"5min":0.36199999999999999,"15min":0.34300000000000003},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":1.095},"last":1.97},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.71299999999999997,"15min":0.67500000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.998,"5min":1.002,"15min":1.29},"last":2.7429999999999999}]},{"osd":5,"last update":"Fri Oct 6 02:31:09 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.69799999999999995,"15min":0.69999999999999996},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.39700000000000002},"max":{"1min":0.98699999999999999,"5min":1.014,"15min":1.2929999999999999},"last":1.091},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.71999999999999997,"15min":0.70899999999999996},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":2.484}]},{"osd":7,"last update":"Fri Oct 6 02:31:09 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.755,"15min":0.751},"min":{"1min":0.44700000000000001,"5min":0.41499999999999998,"15min":0.36799999999999999},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.458},"last":2.7909999999999999},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.77700000000000002,"15min":0.76500000000000001},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.45900000000000002},"max":{"1min":1.0429999999999999,"5min":1.103,"15min":1.2470000000000001},"last":1.0269999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574963,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303456,"kb_used_data":2072,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068384,"statfs":{"total":96636764160,"available":96326025216,"internally_reserved":0,"allocated":2121728,"data_stored":990972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.443},"min":{"1min":0.30399999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.58799999999999997,"5min":0.63600000000000001,"15min":1.2230000000000001},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.45800000000000002,"15min":0.501},"min":{"1min":0.34499999999999997,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.66200000000000003,"5min":0.70099999999999996,"15min":1.788},"last":0.44400000000000001}]},{"osd":2,"last update":"Fri Oct 6 02:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.433,"15min":0.49099999999999999},"min":{"1min":0.33500000000000002,"5min":0.307,"15min":0.28299999999999997},"max":{"1min":0.57799999999999996,"5min":0.61799999999999999,"15min":2.0840000000000001},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.503,"5min":0.495,"15min":0.53500000000000003},"min":{"1min":0.374,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.66800000000000004,"5min":0.88100000000000001,"15min":1.6699999999999999},"last":0.46700000000000003}]},{"osd":3,"last update":"Fri Oct 6 02:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.57999999999999996,"15min":0.60399999999999998},"min":{"1min":0.51200000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.79700000000000004,"5min":0.89900000000000002,"15min":1.266},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.54000000000000004,"15min":0.60299999999999998},"min":{"1min":0.436,"5min":0.40999999999999998,"15min":0.34100000000000003},"max":{"1min":0.70399999999999996,"5min":0.79400000000000004,"15min":1.861},"last":0.48099999999999998}]},{"osd":4,"last update":"Fri Oct 6 02:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.752,"15min":0.82199999999999995},"min":{"1min":0.629,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.97999999999999998,"5min":1.3580000000000001,"15min":1.9279999999999999},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.747,"15min":0.82199999999999995},"min":{"1min":0.53900000000000003,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.95999999999999996,"5min":1.4339999999999999,"15min":2.0179999999999998},"last":0.81899999999999995}]},{"osd":5,"last update":"Fri Oct 6 02:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.71499999999999997,"15min":0.80900000000000005},"min":{"1min":0.57199999999999995,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.91700000000000004,"5min":1.3069999999999999,"15min":1.9830000000000001},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.73199999999999998,"15min":0.79100000000000004},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.47299999999999998},"max":{"1min":0.97299999999999998,"5min":1.327,"15min":1.3380000000000001},"last":0.63}]},{"osd":6,"last update":"Fri Oct 6 02:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.76100000000000001,"15min":0.83299999999999996},"min":{"1min":0.62,"5min":0.52100000000000002,"15min":0.45600000000000002},"max":{"1min":1.038,"5min":1.0609999999999999,"15min":1.829},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.77500000000000002,"15min":0.86499999999999999},"min":{"1min":0.60199999999999998,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.96399999999999997,"5min":1.351,"15min":2.052},"last":0.77300000000000002}]},{"osd":7,"last update":"Fri Oct 6 02:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.73399999999999999,"15min":0.80300000000000005},"min":{"1min":0.59399999999999997,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.97299999999999998,"5min":1.3120000000000001,"15min":1.3340000000000001},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.76200000000000001,"15min":0.83799999999999997},"min":{"1min":0.54600000000000004,"5min":0.51600000000000001,"15min":0.47799999999999998},"max":{"1min":1.0780000000000001,"5min":1.2110000000000001,"15min":1.895},"last":0.80300000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574964,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304040,"kb_used_data":2656,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067800,"statfs":{"total":96636764160,"available":96325427200,"internally_reserved":0,"allocated":2719744,"data_stored":1581430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 02:31:15 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.44600000000000001,"15min":0.45300000000000001},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.76400000000000001,"5min":2.6120000000000001,"15min":2.6120000000000001},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.46999999999999997,"15min":0.47999999999999998},"min":{"1min":0.30599999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.77900000000000003,"5min":3.4199999999999999,"15min":3.4199999999999999},"last":0.33000000000000002}]},{"osd":2,"last update":"Fri Oct 6 02:31:15 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.52000000000000002,"15min":0.51500000000000001},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.58199999999999996,"5min":3.0339999999999998,"15min":3.0339999999999998},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.70699999999999996,"5min":3.343,"15min":3.343},"last":0.377}]},{"osd":3,"last update":"Fri Oct 6 02:31:15 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.505,"15min":0.50600000000000001},"min":{"1min":0.34399999999999997,"5min":0.31,"15min":0.31},"max":{"1min":0.79400000000000004,"5min":3.6579999999999999,"15min":3.6579999999999999},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.59099999999999997,"15min":0.59399999999999997},"min":{"1min":0.443,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.82099999999999995,"5min":2.6890000000000001,"15min":2.6890000000000001},"last":0.505}]},{"osd":4,"last update":"Fri Oct 6 02:31:15 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.76400000000000001,"15min":0.79200000000000004},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.94199999999999995,"5min":3.2999999999999998,"15min":3.2999999999999998},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.755,"15min":0.78900000000000003},"min":{"1min":0.54400000000000004,"5min":0.51000000000000001,"15min":0.48499999999999999},"max":{"1min":1.085,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":0.72199999999999998}]},{"osd":5,"last update":"Fri Oct 6 02:31:15 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.74299999999999999,"15min":0.77600000000000002},"min":{"1min":0.51200000000000001,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.038,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.79000000000000004,"15min":0.79900000000000004},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.071,"5min":3.1110000000000002,"15min":3.1110000000000002},"last":0.66900000000000004}]},{"osd":6,"last update":"Fri Oct 6 02:31:15 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.79600000000000004,"15min":0.81599999999999995},"min":{"1min":0.52900000000000003,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.103,"5min":3.1600000000000001,"15min":3.1600000000000001},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.76900000000000002,"15min":0.81299999999999994},"min":{"1min":0.56200000000000006,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.01,"5min":2.3250000000000002,"15min":2.3250000000000002},"last":0.68500000000000005}]},{"osd":7,"last update":"Fri Oct 6 02:31:15 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.81399999999999995,"15min":0.82399999999999995},"min":{"1min":0.57999999999999996,"5min":0.56899999999999995,"15min":0.56000000000000005},"max":{"1min":1.048,"5min":3.3809999999999998,"15min":3.3809999999999998},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.753,"5min":0.77200000000000002,"15min":0.81200000000000006},"min":{"1min":0.56100000000000005,"5min":0.54700000000000004,"15min":0.53300000000000003},"max":{"1min":1.022,"5min":2.3740000000000001,"15min":2.3740000000000001},"last":0.79100000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574965,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303460,"kb_used_data":2076,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068380,"statfs":{"total":96636764160,"available":96326021120,"internally_reserved":0,"allocated":2125824,"data_stored":991062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:31:14 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.41999999999999998,"15min":0.435},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.63500000000000001,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.46500000000000002,"15min":0.47999999999999998},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.65300000000000002,"5min":0.88700000000000001,"15min":1.0800000000000001},"last":0.42799999999999999}]},{"osd":1,"last update":"Fri Oct 6 02:31:14 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47499999999999998,"15min":0.496},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.32400000000000001},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.51000000000000001,"15min":0.57899999999999996},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.622,"5min":1.778,"15min":1.778},"last":0.45900000000000002}]},{"osd":3,"last update":"Fri Oct 6 02:31:14 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.499,"15min":0.52500000000000002},"min":{"1min":0.33700000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.68400000000000005,"5min":0.83699999999999997,"15min":0.97499999999999998},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.57799999999999996,"15min":0.60899999999999999},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":1.028},"last":0.55000000000000004}]},{"osd":4,"last update":"Fri Oct 6 02:31:14 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.746,"15min":0.76900000000000002},"min":{"1min":0.496,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.1319999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.76200000000000001,"15min":0.82499999999999996},"min":{"1min":0.51200000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.95099999999999996,"5min":1.907,"15min":1.907},"last":0.70999999999999996}]},{"osd":5,"last update":"Fri Oct 6 02:31:14 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.73299999999999998,"15min":0.78200000000000003},"min":{"1min":0.47599999999999998,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.024,"5min":1.109,"15min":1.1850000000000001},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.745,"5min":0.752,"15min":0.80400000000000005},"min":{"1min":0.56200000000000006,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.258},"last":0.63400000000000001}]},{"osd":6,"last update":"Fri Oct 6 02:31:14 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.78500000000000003,"15min":0.83099999999999996},"min":{"1min":0.48399999999999999,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.1100000000000001,"5min":1.827,"15min":1.827},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.747,"5min":0.76700000000000002,"15min":0.78900000000000003},"min":{"1min":0.56100000000000005,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0269999999999999,"5min":1.0660000000000001,"15min":1.1659999999999999},"last":0.58799999999999997}]},{"osd":7,"last update":"Fri Oct 6 02:31:14 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.79400000000000004,"15min":0.86099999999999999},"min":{"1min":0.57099999999999995,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0649999999999999,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.751,"5min":0.77900000000000003,"15min":0.81799999999999995},"min":{"1min":0.56100000000000005,"5min":0.55300000000000005,"15min":0.54800000000000004},"max":{"1min":1.0269999999999999,"5min":1.125,"15min":1.1479999999999999},"last":0.72899999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574962,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303468,"kb_used_data":2084,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068372,"statfs":{"total":96636764160,"available":96326012928,"internally_reserved":0,"allocated":2134016,"data_stored":991098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:31:12 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.434,"15min":0.48799999999999999},"min":{"1min":0.29099999999999998,"5min":0.248,"15min":0.248},"max":{"1min":0.60799999999999998,"5min":1.0489999999999999,"15min":2.024},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.441,"5min":0.46800000000000003,"15min":0.50900000000000001},"min":{"1min":0.31900000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.53300000000000003,"5min":0.754,"15min":2.181},"last":0.39600000000000002}]},{"osd":1,"last update":"Fri Oct 6 02:31:12 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.45800000000000002,"15min":0.62},"min":{"1min":0.33400000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.61299999999999999,"5min":0.66100000000000003,"15min":4.492},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.51600000000000001,"15min":0.66800000000000004},"min":{"1min":0.377,"5min":0.313,"15min":0.313},"max":{"1min":0.67100000000000004,"5min":1.079,"15min":4.569},"last":0.438}]},{"osd":2,"last update":"Fri Oct 6 02:31:12 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.59699999999999998,"15min":0.64800000000000002},"min":{"1min":0.46899999999999997,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.79400000000000004,"5min":1.024,"15min":2.0870000000000002},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.56699999999999995,"15min":0.627},"min":{"1min":0.42799999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.72999999999999998,"5min":0.94199999999999995,"15min":2.4260000000000002},"last":0.49399999999999999}]},{"osd":4,"last update":"Fri Oct 6 02:31:12 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73299999999999998,"15min":0.80200000000000005},"min":{"1min":0.58199999999999996,"5min":0.55000000000000004,"15min":0.44800000000000001},"max":{"1min":0.97299999999999998,"5min":1.0149999999999999,"15min":2.214},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.75,"15min":0.79600000000000004},"min":{"1min":0.53000000000000003,"5min":0.50800000000000001,"15min":0.5},"max":{"1min":1.042,"5min":1.042,"15min":1.2869999999999999},"last":0.55200000000000005}]},{"osd":5,"last update":"Fri Oct 6 02:31:12 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.747,"15min":0.80300000000000005},"min":{"1min":0.55300000000000005,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.97499999999999998,"5min":1.077,"15min":1.3029999999999999},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.754,"5min":0.75600000000000001,"15min":0.82599999999999996},"min":{"1min":0.57999999999999996,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.0549999999999999,"5min":1.0580000000000001,"15min":2.242},"last":0.63900000000000001}]},{"osd":6,"last update":"Fri Oct 6 02:31:12 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.77300000000000002,"15min":0.80500000000000005},"min":{"1min":0.58199999999999996,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.96199999999999997,"5min":1.0489999999999999,"15min":1.357},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.78800000000000003,"15min":0.82099999999999995},"min":{"1min":0.64900000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0049999999999999,"5min":1.0529999999999999,"15min":1.3100000000000001},"last":0.61199999999999999}]},{"osd":7,"last update":"Fri Oct 6 02:31:12 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.76600000000000001,"15min":0.83399999999999996},"min":{"1min":0.54400000000000004,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.98399999999999999,"5min":1.077,"15min":2.27},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.78600000000000003,"15min":0.83899999999999997},"min":{"1min":0.57599999999999996,"5min":0.55600000000000005,"15min":0.53200000000000003},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.325},"last":0.64700000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574964,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303460,"kb_used_data":2076,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068380,"statfs":{"total":96636764160,"available":96326021120,"internally_reserved":0,"allocated":2125824,"data_stored":991526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:31:06 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.65200000000000002,"15min":0.92200000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.84799999999999998,"5min":1.0760000000000001,"15min":12.337},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.67800000000000005,"15min":0.85899999999999999},"min":{"1min":0.48499999999999999,"5min":0.44400000000000001,"15min":0.438},"max":{"1min":0.94299999999999995,"5min":1.3520000000000001,"15min":9.6539999999999999},"last":0.48899999999999999}]},{"osd":1,"last update":"Fri Oct 6 02:31:06 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.66300000000000003,"15min":0.68899999999999995},"min":{"1min":0.42099999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.97899999999999998,"5min":1.129,"15min":3.181},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.70199999999999996,"15min":0.94999999999999996},"min":{"1min":0.45300000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.92700000000000005,"5min":0.995,"15min":12.946999999999999},"last":0.59899999999999998}]},{"osd":2,"last update":"Fri Oct 6 02:31:06 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.69999999999999996,"15min":0.96799999999999997},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.89200000000000002,"5min":1.2989999999999999,"15min":12.377000000000001},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.72699999999999998,"15min":0.83899999999999997},"min":{"1min":0.51600000000000001,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.90800000000000003,"5min":1.0529999999999999,"15min":6.3220000000000001},"last":0.65300000000000002}]},{"osd":3,"last update":"Fri Oct 6 02:31:06 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72599999999999998,"15min":0.93100000000000005},"min":{"1min":0.54900000000000004,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.91100000000000003,"5min":1.5229999999999999,"15min":9.7720000000000002},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.70999999999999996,"15min":0.76000000000000001},"min":{"1min":0.53400000000000003,"5min":0.498,"15min":0.47999999999999998},"max":{"1min":1.0389999999999999,"5min":1.212,"15min":3.0840000000000001},"last":0.66700000000000004}]},{"osd":5,"last update":"Fri Oct 6 02:31:06 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.70599999999999996,"15min":0.81499999999999995},"min":{"1min":0.54000000000000004,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.96299999999999997,"5min":1.02,"15min":6.2690000000000001},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.751,"5min":0.73999999999999999,"15min":0.96799999999999997},"min":{"1min":0.53900000000000003,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.95199999999999996,"5min":1.4359999999999999,"15min":10.769},"last":0.68500000000000005}]},{"osd":6,"last update":"Fri Oct 6 02:31:06 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.70999999999999996,"15min":0.749},"min":{"1min":0.46100000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.1919999999999999,"5min":1.4590000000000001,"15min":3.1349999999999998},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79100000000000004,"15min":0.93100000000000005},"min":{"1min":0.66700000000000004,"5min":0.47799999999999998,"15min":0.45700000000000002},"max":{"1min":1.008,"5min":1.3839999999999999,"15min":8.0899999999999999},"last":0.76100000000000001}]},{"osd":7,"last update":"Fri Oct 6 02:31:06 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.79600000000000004,"15min":1.006},"min":{"1min":0.60999999999999999,"5min":0.501,"15min":0.47599999999999998},"max":{"1min":1.024,"5min":1.494,"15min":9.7089999999999996},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.78000000000000003,"15min":0.94899999999999995},"min":{"1min":0.55900000000000005,"5min":0.374,"15min":0.374},"max":{"1min":1.1399999999999999,"5min":1.2669999999999999,"15min":7.6719999999999997},"last":0.84299999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574964,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303464,"kb_used_data":2080,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068376,"statfs":{"total":96636764160,"available":96326017024,"internally_reserved":0,"allocated":2129920,"data_stored":991028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:32:07 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.63200000000000001,"15min":0.63},"min":{"1min":0.47099999999999997,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.93899999999999995,"5min":0.97599999999999998,"15min":1.4279999999999999},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.629,"15min":0.622},"min":{"1min":0.42599999999999999,"5min":0.40300000000000002,"15min":0.38700000000000001},"max":{"1min":0.89300000000000002,"5min":1.099,"15min":1.099},"last":0.752}]},{"osd":1,"last update":"Fri Oct 6 02:32:07 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.63600000000000001,"15min":0.64900000000000002},"min":{"1min":0.51700000000000002,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.93899999999999995,"5min":1.0660000000000001,"15min":1.089},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.67300000000000004,"15min":0.67700000000000005},"min":{"1min":0.42299999999999999,"5min":0.40699999999999997,"15min":0.40500000000000003},"max":{"1min":0.93899999999999995,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.90500000000000003}]},{"osd":2,"last update":"Fri Oct 6 02:32:07 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.67800000000000005,"15min":0.67600000000000005},"min":{"1min":0.45800000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.67200000000000004,"15min":0.68200000000000005},"min":{"1min":0.58599999999999997,"5min":0.46100000000000002,"15min":0.44600000000000001},"max":{"1min":1.028,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.624}]},{"osd":3,"last update":"Fri Oct 6 02:32:07 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.73299999999999998,"15min":0.73199999999999998},"min":{"1min":0.54000000000000004,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0009999999999999,"5min":1.159,"15min":1.159},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.77200000000000002,"15min":0.77000000000000002},"min":{"1min":0.64200000000000002,"5min":0.56200000000000006,"15min":0.54100000000000004},"max":{"1min":0.97699999999999998,"5min":1.194,"15min":1.194},"last":0.72599999999999998}]},{"osd":4,"last update":"Fri Oct 6 02:32:07 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.69099999999999995,"15min":0.67700000000000005},"min":{"1min":0.46999999999999997,"5min":0.40799999999999997,"15min":0.32000000000000001},"max":{"1min":0.92300000000000004,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.878},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.69099999999999995,"15min":0.68300000000000005},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.35399999999999998},"max":{"1min":0.88600000000000001,"5min":1.25,"15min":1.25},"last":0.64900000000000002}]},{"osd":6,"last update":"Fri Oct 6 02:32:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.71899999999999997,"15min":0.73499999999999999},"min":{"1min":0.45900000000000002,"5min":0.42899999999999999,"15min":0.40300000000000002},"max":{"1min":1.024,"5min":1.024,"15min":1.1719999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.81499999999999995,"15min":0.81799999999999995},"min":{"1min":0.60599999999999998,"5min":0.502,"15min":0.373},"max":{"1min":1.0700000000000001,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.92800000000000005}]},{"osd":7,"last update":"Fri Oct 6 02:32:07 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.754,"15min":0.76000000000000001},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.40799999999999997},"max":{"1min":0.95799999999999996,"5min":1.23,"15min":1.23},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.80200000000000005,"15min":0.82099999999999995},"min":{"1min":0.59599999999999997,"5min":0.47599999999999998,"15min":0.437},"max":{"1min":1.1180000000000001,"5min":1.1339999999999999,"15min":1.5640000000000001},"last":0.80900000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T02:32:12.515 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T02:32:12.583 INFO:tasks.ceph.mds.a.smithi031.stderr:2023-10-06T02:32:12.583+0000 7f14ba7f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T02:32:12.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.736+0000 7fa39699d700 1 Processor -- start 2023-10-06T02:32:12.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.736+0000 7fa39699d700 1 -- start start 2023-10-06T02:32:12.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.737+0000 7fa39699d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa390153c30 0x7fa39015a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:12.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.737+0000 7fa39699d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa39015de50 0x7fa3900cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:12.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.737+0000 7fa39699d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa39015afd0 con 0x7fa3900cef40 2023-10-06T02:32:12.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.737+0000 7fa39699d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa39015b2b0 con 0x7fa390153c30 2023-10-06T02:32:12.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.737+0000 7fa39699d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa39015b590 con 0x7fa39015de50 2023-10-06T02:32:12.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.737+0000 7fa39599b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa390153c30 0x7fa39015a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:12.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.737+0000 7fa39599b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa390153c30 0x7fa39015a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41100/0 (socket says 172.21.15.31:41100) 2023-10-06T02:32:12.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.737+0000 7fa39599b700 1 -- 172.21.15.31:0/4069227986 learned_addr learned my addr 172.21.15.31:0/4069227986 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:12.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.737+0000 7fa39519a700 1 --2- 172.21.15.31:0/4069227986 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa39015de50 0x7fa3900cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:12.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.738+0000 7fa394999700 1 -- 172.21.15.31:0/4069227986 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1918933266 0 0) 0x7fa39015afd0 con 0x7fa3900cef40 2023-10-06T02:32:12.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.738+0000 7fa39519a700 1 -- 172.21.15.31:0/4069227986 >> v2:172.21.15.31:3300/0 conn(0x7fa390153c30 msgr2=0x7fa39015a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:12.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.738+0000 7fa39519a700 1 --2- 172.21.15.31:0/4069227986 >> v2:172.21.15.31:3300/0 conn(0x7fa390153c30 0x7fa39015a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:12.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.738+0000 7fa39519a700 1 -- 172.21.15.31:0/4069227986 >> v1:172.21.15.62:6789/0 conn(0x7fa3900cef40 legacy=0x7fa39006d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:12.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.738+0000 7fa39519a700 1 -- 172.21.15.31:0/4069227986 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa39015b870 con 0x7fa39015de50 2023-10-06T02:32:12.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.738+0000 7fa39519a700 1 --2- 172.21.15.31:0/4069227986 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa39015de50 0x7fa3900cea00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa3800098e0 tx=0x7fa3800091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e8f63cc3586699 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:12.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.739+0000 7fa394999700 1 -- 172.21.15.31:0/4069227986 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa38000b640 con 0x7fa39015de50 2023-10-06T02:32:12.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.739+0000 7fa394999700 1 -- 172.21.15.31:0/4069227986 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa38000b7a0 con 0x7fa39015de50 2023-10-06T02:32:12.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.739+0000 7fa394999700 1 -- 172.21.15.31:0/4069227986 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa38000bb40 con 0x7fa39015de50 2023-10-06T02:32:12.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.739+0000 7fa39699d700 1 -- 172.21.15.31:0/4069227986 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa39015de50 msgr2=0x7fa3900cea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:12.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.739+0000 7fa39699d700 1 --2- 172.21.15.31:0/4069227986 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa39015de50 0x7fa3900cea00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa3800098e0 tx=0x7fa3800091a0 comp rx=0 tx=0).stop 2023-10-06T02:32:12.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.739+0000 7fa39699d700 1 -- 172.21.15.31:0/4069227986 shutdown_connections 2023-10-06T02:32:12.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.739+0000 7fa39699d700 1 --2- 172.21.15.31:0/4069227986 >> v2:172.21.15.31:3300/0 conn(0x7fa390153c30 0x7fa39015a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:12.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.739+0000 7fa39699d700 1 --2- 172.21.15.31:0/4069227986 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa39015de50 0x7fa3900cea00 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:12.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.739+0000 7fa39699d700 1 -- 172.21.15.31:0/4069227986 >> 172.21.15.31:0/4069227986 conn(0x7fa3900703a0 msgr2=0x7fa3900707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:12.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.739+0000 7fa39699d700 1 -- 172.21.15.31:0/4069227986 shutdown_connections 2023-10-06T02:32:12.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.739+0000 7fa39699d700 1 -- 172.21.15.31:0/4069227986 wait complete. 2023-10-06T02:32:12.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.741+0000 7fa39699d700 1 Processor -- start 2023-10-06T02:32:12.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.741+0000 7fa39699d700 1 -- start start 2023-10-06T02:32:12.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.742+0000 7fa39699d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3900cef40 0x7fa390148e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:12.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.742+0000 7fa39699d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa390153c30 0x7fa39015f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:12.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.742+0000 7fa39699d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa39015de50 0x7fa39015fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:12.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.742+0000 7fa39699d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3901521d0 con 0x7fa390153c30 2023-10-06T02:32:12.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.742+0000 7fa39699d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa390151320 con 0x7fa3900cef40 2023-10-06T02:32:12.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.742+0000 7fa39699d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3901515c0 con 0x7fa39015de50 2023-10-06T02:32:12.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.742+0000 7fa39519a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa390153c30 0x7fa39015f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:12.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.742+0000 7fa39519a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa390153c30 0x7fa39015f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41108/0 (socket says 172.21.15.31:41108) 2023-10-06T02:32:12.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.742+0000 7fa39519a700 1 -- 172.21.15.31:0/910865655 learned_addr learned my addr 172.21.15.31:0/910865655 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:12.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.742+0000 7fa39599b700 1 --2- 172.21.15.31:0/910865655 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3900cef40 0x7fa390148e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:12.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.742+0000 7fa39619c700 1 --2- 172.21.15.31:0/910865655 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa39015de50 0x7fa39015fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:12.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.742+0000 7fa39519a700 1 -- 172.21.15.31:0/910865655 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa39015de50 msgr2=0x7fa39015fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:12.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.742+0000 7fa39519a700 1 --2- 172.21.15.31:0/910865655 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa39015de50 0x7fa39015fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:12.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.742+0000 7fa39519a700 1 -- 172.21.15.31:0/910865655 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3900cef40 msgr2=0x7fa390148e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:12.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.742+0000 7fa39519a700 1 --2- 172.21.15.31:0/910865655 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3900cef40 0x7fa390148e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:12.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.742+0000 7fa39519a700 1 -- 172.21.15.31:0/910865655 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa380009420 con 0x7fa390153c30 2023-10-06T02:32:12.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.743+0000 7fa39519a700 1 --2- 172.21.15.31:0/910865655 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa390153c30 0x7fa39015f790 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7fa3800093f0 tx=0x7fa380017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:12.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.743+0000 7fa385ffb700 1 -- 172.21.15.31:0/910865655 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa38000bb20 con 0x7fa390153c30 2023-10-06T02:32:12.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.743+0000 7fa385ffb700 1 -- 172.21.15.31:0/910865655 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa380000c90 con 0x7fa390153c30 2023-10-06T02:32:12.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.743+0000 7fa39699d700 1 -- 172.21.15.31:0/910865655 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3901518a0 con 0x7fa390153c30 2023-10-06T02:32:12.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.743+0000 7fa385ffb700 1 -- 172.21.15.31:0/910865655 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa380011670 con 0x7fa390153c30 2023-10-06T02:32:12.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.743+0000 7fa39699d700 1 -- 172.21.15.31:0/910865655 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa390151e60 con 0x7fa390153c30 2023-10-06T02:32:12.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.744+0000 7fa39699d700 1 -- 172.21.15.31:0/910865655 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa390152380 con 0x7fa390153c30 2023-10-06T02:32:12.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.745+0000 7fa385ffb700 1 -- 172.21.15.31:0/910865655 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85556+0+0 (secure 0 0 0) 0x7fa380011a20 con 0x7fa390153c30 2023-10-06T02:32:12.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.747+0000 7fa385ffb700 1 --2- 172.21.15.31:0/910865655 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa374067830 0x7fa374069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:12.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.747+0000 7fa385ffb700 1 -- 172.21.15.31:0/910865655 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fa380084390 con 0x7fa390153c30 2023-10-06T02:32:12.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.747+0000 7fa39599b700 1 --2- 172.21.15.31:0/910865655 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa374067830 0x7fa374069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:12.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.748+0000 7fa39599b700 1 --2- 172.21.15.31:0/910865655 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa374067830 0x7fa374069ce0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fa38c006d40 tx=0x7fa38c009250 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:12.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.751+0000 7fa385ffb700 1 -- 172.21.15.31:0/910865655 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa390152380 con 0x7fa390153c30 2023-10-06T02:32:12.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.947+0000 7fa39699d700 1 -- 172.21.15.31:0/910865655 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa39004c3c0 con 0x7fa374067830 2023-10-06T02:32:12.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.958+0000 7fa385ffb700 1 -- 172.21.15.31:0/910865655 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311412 (secure 0 0 0) 0x7fa39004c3c0 con 0x7fa374067830 2023-10-06T02:32:12.962 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:32:12.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.962+0000 7fa39699d700 1 -- 172.21.15.31:0/910865655 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa374067830 msgr2=0x7fa374069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:12.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.962+0000 7fa39699d700 1 --2- 172.21.15.31:0/910865655 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa374067830 0x7fa374069ce0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fa38c006d40 tx=0x7fa38c009250 comp rx=0 tx=0).stop 2023-10-06T02:32:12.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.962+0000 7fa39699d700 1 -- 172.21.15.31:0/910865655 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa390153c30 msgr2=0x7fa39015f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:12.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.962+0000 7fa39699d700 1 --2- 172.21.15.31:0/910865655 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa390153c30 0x7fa39015f790 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7fa3800093f0 tx=0x7fa380017000 comp rx=0 tx=0).stop 2023-10-06T02:32:12.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.962+0000 7fa39699d700 1 -- 172.21.15.31:0/910865655 shutdown_connections 2023-10-06T02:32:12.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.962+0000 7fa39699d700 1 --2- 172.21.15.31:0/910865655 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa374067830 0x7fa374069ce0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:12.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.962+0000 7fa39699d700 1 --2- 172.21.15.31:0/910865655 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa39015de50 0x7fa39015fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:12.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.962+0000 7fa39699d700 1 --2- 172.21.15.31:0/910865655 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3900cef40 0x7fa390148e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:12.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.962+0000 7fa39699d700 1 --2- 172.21.15.31:0/910865655 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa390153c30 0x7fa39015f790 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:12.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.962+0000 7fa39699d700 1 -- 172.21.15.31:0/910865655 >> 172.21.15.31:0/910865655 conn(0x7fa3900703a0 msgr2=0x7fa3900c6ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:12.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.962+0000 7fa39699d700 1 -- 172.21.15.31:0/910865655 shutdown_connections 2023-10-06T02:32:12.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:12.962+0000 7fa39699d700 1 -- 172.21.15.31:0/910865655 wait complete. 2023-10-06T02:32:12.965 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-10-06T02:32:12.980 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":234,"stamp":"2023-10-06T02:32:12.523713+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":223,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":166,"ondisk_log_size":166,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":108,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2428872,"kb_used_data":17800,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752545848,"statfs":{"total":773094113280,"available":770606948352,"internally_reserved":0,"allocated":18227200,"data_stored":9113124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001710"},"pg_stats":[{"pgid":"28.3f","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.150309+0000","last_change":"2023-10-06T02:32:08.147428+0000","last_active":"2023-10-06T02:32:08.150309+0000","last_peered":"2023-10-06T02:32:08.150309+0000","last_clean":"2023-10-06T02:32:08.150309+0000","last_became_active":"2023-10-06T02:32:08.147118+0000","last_became_peered":"2023-10-06T02:32:08.147118+0000","last_unstale":"2023-10-06T02:32:08.150309+0000","last_undegraded":"2023-10-06T02:32:08.150309+0000","last_fullsized":"2023-10-06T02:32:08.150309+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:33:05.856323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.3e","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.139802+0000","last_change":"2023-10-06T02:32:08.136941+0000","last_active":"2023-10-06T02:32:08.139802+0000","last_peered":"2023-10-06T02:32:08.139802+0000","last_clean":"2023-10-06T02:32:08.139802+0000","last_became_active":"2023-10-06T02:32:08.136676+0000","last_became_peered":"2023-10-06T02:32:08.136676+0000","last_unstale":"2023-10-06T02:32:08.139802+0000","last_undegraded":"2023-10-06T02:32:08.139802+0000","last_fullsized":"2023-10-06T02:32:08.139802+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:20:56.793118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.3d","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.143184+0000","last_change":"2023-10-06T02:32:08.143184+0000","last_active":"2023-10-06T02:32:08.143184+0000","last_peered":"2023-10-06T02:32:08.143184+0000","last_clean":"2023-10-06T02:32:08.143184+0000","last_became_active":"2023-10-06T02:32:08.142920+0000","last_became_peered":"2023-10-06T02:32:08.142920+0000","last_unstale":"2023-10-06T02:32:08.143184+0000","last_undegraded":"2023-10-06T02:32:08.143184+0000","last_fullsized":"2023-10-06T02:32:08.143184+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:18:01.461302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.3c","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.138525+0000","last_change":"2023-10-06T02:32:08.138525+0000","last_active":"2023-10-06T02:32:08.138525+0000","last_peered":"2023-10-06T02:32:08.138525+0000","last_clean":"2023-10-06T02:32:08.138525+0000","last_became_active":"2023-10-06T02:32:08.138206+0000","last_became_peered":"2023-10-06T02:32:08.138206+0000","last_unstale":"2023-10-06T02:32:08.138525+0000","last_undegraded":"2023-10-06T02:32:08.138525+0000","last_fullsized":"2023-10-06T02:32:08.138525+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:59:25.898666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.3b","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.142156+0000","last_change":"2023-10-06T02:32:08.142156+0000","last_active":"2023-10-06T02:32:08.142156+0000","last_peered":"2023-10-06T02:32:08.142156+0000","last_clean":"2023-10-06T02:32:08.142156+0000","last_became_active":"2023-10-06T02:32:08.141849+0000","last_became_peered":"2023-10-06T02:32:08.141849+0000","last_unstale":"2023-10-06T02:32:08.142156+0000","last_undegraded":"2023-10-06T02:32:08.142156+0000","last_fullsized":"2023-10-06T02:32:08.142156+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:27:40.521385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.3a","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.137802+0000","last_change":"2023-10-06T02:32:08.137802+0000","last_active":"2023-10-06T02:32:08.137802+0000","last_peered":"2023-10-06T02:32:08.137802+0000","last_clean":"2023-10-06T02:32:08.137802+0000","last_became_active":"2023-10-06T02:32:08.137516+0000","last_became_peered":"2023-10-06T02:32:08.137516+0000","last_unstale":"2023-10-06T02:32:08.137802+0000","last_undegraded":"2023-10-06T02:32:08.137802+0000","last_fullsized":"2023-10-06T02:32:08.137802+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:11:20.474130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.39","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.142055+0000","last_change":"2023-10-06T02:32:08.142055+0000","last_active":"2023-10-06T02:32:08.142055+0000","last_peered":"2023-10-06T02:32:08.142055+0000","last_clean":"2023-10-06T02:32:08.142055+0000","last_became_active":"2023-10-06T02:32:08.141751+0000","last_became_peered":"2023-10-06T02:32:08.141751+0000","last_unstale":"2023-10-06T02:32:08.142055+0000","last_undegraded":"2023-10-06T02:32:08.142055+0000","last_fullsized":"2023-10-06T02:32:08.142055+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:43:16.882147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.38","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.136360+0000","last_change":"2023-10-06T02:32:08.136360+0000","last_active":"2023-10-06T02:32:08.136360+0000","last_peered":"2023-10-06T02:32:08.136360+0000","last_clean":"2023-10-06T02:32:08.136360+0000","last_became_active":"2023-10-06T02:32:08.136088+0000","last_became_peered":"2023-10-06T02:32:08.136088+0000","last_unstale":"2023-10-06T02:32:08.136360+0000","last_undegraded":"2023-10-06T02:32:08.136360+0000","last_fullsized":"2023-10-06T02:32:08.136360+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:30:07.358659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.37","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.150304+0000","last_change":"2023-10-06T02:32:08.150304+0000","last_active":"2023-10-06T02:32:08.150304+0000","last_peered":"2023-10-06T02:32:08.150304+0000","last_clean":"2023-10-06T02:32:08.150304+0000","last_became_active":"2023-10-06T02:32:08.149835+0000","last_became_peered":"2023-10-06T02:32:08.149835+0000","last_unstale":"2023-10-06T02:32:08.150304+0000","last_undegraded":"2023-10-06T02:32:08.150304+0000","last_fullsized":"2023-10-06T02:32:08.150304+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:57:42.946510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.36","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.140956+0000","last_change":"2023-10-06T02:32:08.140956+0000","last_active":"2023-10-06T02:32:08.140956+0000","last_peered":"2023-10-06T02:32:08.140956+0000","last_clean":"2023-10-06T02:32:08.140956+0000","last_became_active":"2023-10-06T02:32:08.140700+0000","last_became_peered":"2023-10-06T02:32:08.140700+0000","last_unstale":"2023-10-06T02:32:08.140956+0000","last_undegraded":"2023-10-06T02:32:08.140956+0000","last_fullsized":"2023-10-06T02:32:08.140956+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:21:02.172715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.35","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.144570+0000","last_change":"2023-10-06T02:32:08.144570+0000","last_active":"2023-10-06T02:32:08.144570+0000","last_peered":"2023-10-06T02:32:08.144570+0000","last_clean":"2023-10-06T02:32:08.144570+0000","last_became_active":"2023-10-06T02:32:08.144277+0000","last_became_peered":"2023-10-06T02:32:08.144277+0000","last_unstale":"2023-10-06T02:32:08.144570+0000","last_undegraded":"2023-10-06T02:32:08.144570+0000","last_fullsized":"2023-10-06T02:32:08.144570+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:50:59.413536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.34","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.135521+0000","last_change":"2023-10-06T02:32:08.135521+0000","last_active":"2023-10-06T02:32:08.135521+0000","last_peered":"2023-10-06T02:32:08.135521+0000","last_clean":"2023-10-06T02:32:08.135521+0000","last_became_active":"2023-10-06T02:32:08.135206+0000","last_became_peered":"2023-10-06T02:32:08.135206+0000","last_unstale":"2023-10-06T02:32:08.135521+0000","last_undegraded":"2023-10-06T02:32:08.135521+0000","last_fullsized":"2023-10-06T02:32:08.135521+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:53:01.956748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.33","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.141398+0000","last_change":"2023-10-06T02:32:08.141398+0000","last_active":"2023-10-06T02:32:08.141398+0000","last_peered":"2023-10-06T02:32:08.141398+0000","last_clean":"2023-10-06T02:32:08.141398+0000","last_became_active":"2023-10-06T02:32:08.138550+0000","last_became_peered":"2023-10-06T02:32:08.138550+0000","last_unstale":"2023-10-06T02:32:08.141398+0000","last_undegraded":"2023-10-06T02:32:08.141398+0000","last_fullsized":"2023-10-06T02:32:08.141398+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:45:32.024530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.32","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.125227+0000","last_change":"2023-10-06T02:32:08.125227+0000","last_active":"2023-10-06T02:32:08.125227+0000","last_peered":"2023-10-06T02:32:08.125227+0000","last_clean":"2023-10-06T02:32:08.125227+0000","last_became_active":"2023-10-06T02:32:08.124744+0000","last_became_peered":"2023-10-06T02:32:08.124744+0000","last_unstale":"2023-10-06T02:32:08.125227+0000","last_undegraded":"2023-10-06T02:32:08.125227+0000","last_fullsized":"2023-10-06T02:32:08.125227+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:54:22.945907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.31","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.144984+0000","last_change":"2023-10-06T02:32:08.138975+0000","last_active":"2023-10-06T02:32:08.144984+0000","last_peered":"2023-10-06T02:32:08.144984+0000","last_clean":"2023-10-06T02:32:08.144984+0000","last_became_active":"2023-10-06T02:32:08.138559+0000","last_became_peered":"2023-10-06T02:32:08.138559+0000","last_unstale":"2023-10-06T02:32:08.144984+0000","last_undegraded":"2023-10-06T02:32:08.144984+0000","last_fullsized":"2023-10-06T02:32:08.144984+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:38:53.342204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.30","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.130358+0000","last_change":"2023-10-06T02:32:08.130358+0000","last_active":"2023-10-06T02:32:08.130358+0000","last_peered":"2023-10-06T02:32:08.130358+0000","last_clean":"2023-10-06T02:32:08.130358+0000","last_became_active":"2023-10-06T02:32:08.130078+0000","last_became_peered":"2023-10-06T02:32:08.130078+0000","last_unstale":"2023-10-06T02:32:08.130358+0000","last_undegraded":"2023-10-06T02:32:08.130358+0000","last_fullsized":"2023-10-06T02:32:08.130358+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:44:24.680238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.2f","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.149701+0000","last_change":"2023-10-06T02:32:08.149701+0000","last_active":"2023-10-06T02:32:08.149701+0000","last_peered":"2023-10-06T02:32:08.149701+0000","last_clean":"2023-10-06T02:32:08.149701+0000","last_became_active":"2023-10-06T02:32:08.149301+0000","last_became_peered":"2023-10-06T02:32:08.149301+0000","last_unstale":"2023-10-06T02:32:08.149701+0000","last_undegraded":"2023-10-06T02:32:08.149701+0000","last_fullsized":"2023-10-06T02:32:08.149701+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:28:32.152527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.2e","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.137505+0000","last_change":"2023-10-06T02:32:08.137505+0000","last_active":"2023-10-06T02:32:08.137505+0000","last_peered":"2023-10-06T02:32:08.137505+0000","last_clean":"2023-10-06T02:32:08.137505+0000","last_became_active":"2023-10-06T02:32:08.137248+0000","last_became_peered":"2023-10-06T02:32:08.137248+0000","last_unstale":"2023-10-06T02:32:08.137505+0000","last_undegraded":"2023-10-06T02:32:08.137505+0000","last_fullsized":"2023-10-06T02:32:08.137505+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:25.607604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.2d","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.135477+0000","last_change":"2023-10-06T02:32:08.135477+0000","last_active":"2023-10-06T02:32:08.135477+0000","last_peered":"2023-10-06T02:32:08.135477+0000","last_clean":"2023-10-06T02:32:08.135477+0000","last_became_active":"2023-10-06T02:32:08.135037+0000","last_became_peered":"2023-10-06T02:32:08.135037+0000","last_unstale":"2023-10-06T02:32:08.135477+0000","last_undegraded":"2023-10-06T02:32:08.135477+0000","last_fullsized":"2023-10-06T02:32:08.135477+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:32:28.584125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2c","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.143599+0000","last_change":"2023-10-06T02:32:08.143599+0000","last_active":"2023-10-06T02:32:08.143599+0000","last_peered":"2023-10-06T02:32:08.143599+0000","last_clean":"2023-10-06T02:32:08.143599+0000","last_became_active":"2023-10-06T02:32:08.143323+0000","last_became_peered":"2023-10-06T02:32:08.143323+0000","last_unstale":"2023-10-06T02:32:08.143599+0000","last_undegraded":"2023-10-06T02:32:08.143599+0000","last_fullsized":"2023-10-06T02:32:08.143599+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:18.734334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2b","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.141554+0000","last_change":"2023-10-06T02:32:08.141554+0000","last_active":"2023-10-06T02:32:08.141554+0000","last_peered":"2023-10-06T02:32:08.141554+0000","last_clean":"2023-10-06T02:32:08.141554+0000","last_became_active":"2023-10-06T02:32:08.141291+0000","last_became_peered":"2023-10-06T02:32:08.141291+0000","last_unstale":"2023-10-06T02:32:08.141554+0000","last_undegraded":"2023-10-06T02:32:08.141554+0000","last_fullsized":"2023-10-06T02:32:08.141554+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:17:46.078225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2a","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.125040+0000","last_change":"2023-10-06T02:32:08.125040+0000","last_active":"2023-10-06T02:32:08.125040+0000","last_peered":"2023-10-06T02:32:08.125040+0000","last_clean":"2023-10-06T02:32:08.125040+0000","last_became_active":"2023-10-06T02:32:08.124385+0000","last_became_peered":"2023-10-06T02:32:08.124385+0000","last_unstale":"2023-10-06T02:32:08.125040+0000","last_undegraded":"2023-10-06T02:32:08.125040+0000","last_fullsized":"2023-10-06T02:32:08.125040+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:53:01.956748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.29","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.130237+0000","last_change":"2023-10-06T02:32:08.128695+0000","last_active":"2023-10-06T02:32:08.130237+0000","last_peered":"2023-10-06T02:32:08.130237+0000","last_clean":"2023-10-06T02:32:08.130237+0000","last_became_active":"2023-10-06T02:32:08.128358+0000","last_became_peered":"2023-10-06T02:32:08.128358+0000","last_unstale":"2023-10-06T02:32:08.130237+0000","last_undegraded":"2023-10-06T02:32:08.130237+0000","last_fullsized":"2023-10-06T02:32:08.130237+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:45:47.719619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.28","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.123502+0000","last_change":"2023-10-06T02:32:08.123502+0000","last_active":"2023-10-06T02:32:08.123502+0000","last_peered":"2023-10-06T02:32:08.123502+0000","last_clean":"2023-10-06T02:32:08.123502+0000","last_became_active":"2023-10-06T02:32:08.123178+0000","last_became_peered":"2023-10-06T02:32:08.123178+0000","last_unstale":"2023-10-06T02:32:08.123502+0000","last_undegraded":"2023-10-06T02:32:08.123502+0000","last_fullsized":"2023-10-06T02:32:08.123502+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:54:22.945907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.27","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.138182+0000","last_change":"2023-10-06T02:32:08.138182+0000","last_active":"2023-10-06T02:32:08.138182+0000","last_peered":"2023-10-06T02:32:08.138182+0000","last_clean":"2023-10-06T02:32:08.138182+0000","last_became_active":"2023-10-06T02:32:08.137931+0000","last_became_peered":"2023-10-06T02:32:08.137931+0000","last_unstale":"2023-10-06T02:32:08.138182+0000","last_undegraded":"2023-10-06T02:32:08.138182+0000","last_fullsized":"2023-10-06T02:32:08.138182+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:14:29.040980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.26","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.148522+0000","last_change":"2023-10-06T02:32:08.146953+0000","last_active":"2023-10-06T02:32:08.148522+0000","last_peered":"2023-10-06T02:32:08.148522+0000","last_clean":"2023-10-06T02:32:08.148522+0000","last_became_active":"2023-10-06T02:32:08.146663+0000","last_became_peered":"2023-10-06T02:32:08.146663+0000","last_unstale":"2023-10-06T02:32:08.148522+0000","last_undegraded":"2023-10-06T02:32:08.148522+0000","last_fullsized":"2023-10-06T02:32:08.148522+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:50:59.413536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.25","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.143998+0000","last_change":"2023-10-06T02:32:08.143998+0000","last_active":"2023-10-06T02:32:08.143998+0000","last_peered":"2023-10-06T02:32:08.143998+0000","last_clean":"2023-10-06T02:32:08.143998+0000","last_became_active":"2023-10-06T02:32:08.143647+0000","last_became_peered":"2023-10-06T02:32:08.143647+0000","last_unstale":"2023-10-06T02:32:08.143998+0000","last_undegraded":"2023-10-06T02:32:08.143998+0000","last_fullsized":"2023-10-06T02:32:08.143998+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:34:14.560103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.24","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.138504+0000","last_change":"2023-10-06T02:32:08.138504+0000","last_active":"2023-10-06T02:32:08.138504+0000","last_peered":"2023-10-06T02:32:08.138504+0000","last_clean":"2023-10-06T02:32:08.138504+0000","last_became_active":"2023-10-06T02:32:08.138193+0000","last_became_peered":"2023-10-06T02:32:08.138193+0000","last_unstale":"2023-10-06T02:32:08.138504+0000","last_undegraded":"2023-10-06T02:32:08.138504+0000","last_fullsized":"2023-10-06T02:32:08.138504+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:38:52.612898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.23","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.140613+0000","last_change":"2023-10-06T02:32:08.140613+0000","last_active":"2023-10-06T02:32:08.140613+0000","last_peered":"2023-10-06T02:32:08.140613+0000","last_clean":"2023-10-06T02:32:08.140613+0000","last_became_active":"2023-10-06T02:32:08.140353+0000","last_became_peered":"2023-10-06T02:32:08.140353+0000","last_unstale":"2023-10-06T02:32:08.140613+0000","last_undegraded":"2023-10-06T02:32:08.140613+0000","last_fullsized":"2023-10-06T02:32:08.140613+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:55:31.162096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.22","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.145007+0000","last_change":"2023-10-06T02:32:08.139781+0000","last_active":"2023-10-06T02:32:08.145007+0000","last_peered":"2023-10-06T02:32:08.145007+0000","last_clean":"2023-10-06T02:32:08.145007+0000","last_became_active":"2023-10-06T02:32:08.139464+0000","last_became_peered":"2023-10-06T02:32:08.139464+0000","last_unstale":"2023-10-06T02:32:08.145007+0000","last_undegraded":"2023-10-06T02:32:08.145007+0000","last_fullsized":"2023-10-06T02:32:08.145007+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:23.130385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.21","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.137202+0000","last_change":"2023-10-06T02:32:08.137202+0000","last_active":"2023-10-06T02:32:08.137202+0000","last_peered":"2023-10-06T02:32:08.137202+0000","last_clean":"2023-10-06T02:32:08.137202+0000","last_became_active":"2023-10-06T02:32:08.136906+0000","last_became_peered":"2023-10-06T02:32:08.136906+0000","last_unstale":"2023-10-06T02:32:08.137202+0000","last_undegraded":"2023-10-06T02:32:08.137202+0000","last_fullsized":"2023-10-06T02:32:08.137202+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:10:35.944296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.20","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.143967+0000","last_change":"2023-10-06T02:32:08.143967+0000","last_active":"2023-10-06T02:32:08.143967+0000","last_peered":"2023-10-06T02:32:08.143967+0000","last_clean":"2023-10-06T02:32:08.143967+0000","last_became_active":"2023-10-06T02:32:08.143585+0000","last_became_peered":"2023-10-06T02:32:08.143585+0000","last_unstale":"2023-10-06T02:32:08.143967+0000","last_undegraded":"2023-10-06T02:32:08.143967+0000","last_fullsized":"2023-10-06T02:32:08.143967+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:32:15.396249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.10","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.145377+0000","last_change":"2023-10-06T02:32:08.140389+0000","last_active":"2023-10-06T02:32:08.145377+0000","last_peered":"2023-10-06T02:32:08.145377+0000","last_clean":"2023-10-06T02:32:08.145377+0000","last_became_active":"2023-10-06T02:32:08.140116+0000","last_became_peered":"2023-10-06T02:32:08.140116+0000","last_unstale":"2023-10-06T02:32:08.145377+0000","last_undegraded":"2023-10-06T02:32:08.145377+0000","last_fullsized":"2023-10-06T02:32:08.145377+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:42:28.367217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-06T02:32:07.136325+0000","last_change":"2023-10-06T02:32:07.132344+0000","last_active":"2023-10-06T02:32:07.136325+0000","last_peered":"2023-10-06T02:32:07.136325+0000","last_clean":"2023-10-06T02:32:07.136325+0000","last_became_active":"2023-10-06T02:32:07.132084+0000","last_became_peered":"2023-10-06T02:32:07.132084+0000","last_unstale":"2023-10-06T02:32:07.136325+0000","last_undegraded":"2023-10-06T02:32:07.136325+0000","last_fullsized":"2023-10-06T02:32:07.136325+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_clean_scrub_stamp":"2023-10-06T02:32:06.100240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:01:49.273527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.138085+0000","last_change":"2023-10-06T02:32:08.138085+0000","last_active":"2023-10-06T02:32:08.138085+0000","last_peered":"2023-10-06T02:32:08.138085+0000","last_clean":"2023-10-06T02:32:08.138085+0000","last_became_active":"2023-10-06T02:32:08.137462+0000","last_became_peered":"2023-10-06T02:32:08.137462+0000","last_unstale":"2023-10-06T02:32:08.138085+0000","last_undegraded":"2023-10-06T02:32:08.138085+0000","last_fullsized":"2023-10-06T02:32:08.138085+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:23:50.218257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-06T02:32:07.139608+0000","last_change":"2023-10-06T02:32:07.137545+0000","last_active":"2023-10-06T02:32:07.139608+0000","last_peered":"2023-10-06T02:32:07.139608+0000","last_clean":"2023-10-06T02:32:07.139608+0000","last_became_active":"2023-10-06T02:32:07.137242+0000","last_became_peered":"2023-10-06T02:32:07.137242+0000","last_unstale":"2023-10-06T02:32:07.139608+0000","last_undegraded":"2023-10-06T02:32:07.139608+0000","last_fullsized":"2023-10-06T02:32:07.139608+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_clean_scrub_stamp":"2023-10-06T02:32:06.100240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:25:14.308128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.149330+0000","last_change":"2023-10-06T02:32:08.149330+0000","last_active":"2023-10-06T02:32:08.149330+0000","last_peered":"2023-10-06T02:32:08.149330+0000","last_clean":"2023-10-06T02:32:08.149330+0000","last_became_active":"2023-10-06T02:32:08.149003+0000","last_became_peered":"2023-10-06T02:32:08.149003+0000","last_unstale":"2023-10-06T02:32:08.149330+0000","last_undegraded":"2023-10-06T02:32:08.149330+0000","last_fullsized":"2023-10-06T02:32:08.149330+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:49:18.782095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.15","version":"168'3","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.143048+0000","last_change":"2023-10-06T02:32:07.128223+0000","last_active":"2023-10-06T02:32:08.143048+0000","last_peered":"2023-10-06T02:32:08.143048+0000","last_clean":"2023-10-06T02:32:08.143048+0000","last_became_active":"2023-10-06T02:32:07.127823+0000","last_became_peered":"2023-10-06T02:32:07.127823+0000","last_unstale":"2023-10-06T02:32:08.143048+0000","last_undegraded":"2023-10-06T02:32:08.143048+0000","last_fullsized":"2023-10-06T02:32:08.143048+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_clean_scrub_stamp":"2023-10-06T02:32:06.100240+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:27:05.038453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.13","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.129503+0000","last_change":"2023-10-06T02:32:08.129503+0000","last_active":"2023-10-06T02:32:08.129503+0000","last_peered":"2023-10-06T02:32:08.129503+0000","last_clean":"2023-10-06T02:32:08.129503+0000","last_became_active":"2023-10-06T02:32:08.129233+0000","last_became_peered":"2023-10-06T02:32:08.129233+0000","last_unstale":"2023-10-06T02:32:08.129503+0000","last_undegraded":"2023-10-06T02:32:08.129503+0000","last_fullsized":"2023-10-06T02:32:08.129503+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:15:40.109842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.14","version":"168'6","reported_seq":15,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.145876+0000","last_change":"2023-10-06T02:32:07.141381+0000","last_active":"2023-10-06T02:32:08.145876+0000","last_peered":"2023-10-06T02:32:08.145876+0000","last_clean":"2023-10-06T02:32:08.145876+0000","last_became_active":"2023-10-06T02:32:07.141094+0000","last_became_peered":"2023-10-06T02:32:07.141094+0000","last_unstale":"2023-10-06T02:32:08.145876+0000","last_undegraded":"2023-10-06T02:32:08.145876+0000","last_fullsized":"2023-10-06T02:32:08.145876+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_clean_scrub_stamp":"2023-10-06T02:32:06.100240+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:33.292861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.14","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.139535+0000","last_change":"2023-10-06T02:32:08.135413+0000","last_active":"2023-10-06T02:32:08.139535+0000","last_peered":"2023-10-06T02:32:08.139535+0000","last_clean":"2023-10-06T02:32:08.139535+0000","last_became_active":"2023-10-06T02:32:08.135163+0000","last_became_peered":"2023-10-06T02:32:08.135163+0000","last_unstale":"2023-10-06T02:32:08.139535+0000","last_undegraded":"2023-10-06T02:32:08.139535+0000","last_fullsized":"2023-10-06T02:32:08.139535+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:56:05.639408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.13","version":"168'2","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.146593+0000","last_change":"2023-10-06T02:32:07.125916+0000","last_active":"2023-10-06T02:32:08.146593+0000","last_peered":"2023-10-06T02:32:08.146593+0000","last_clean":"2023-10-06T02:32:08.146593+0000","last_became_active":"2023-10-06T02:32:07.125637+0000","last_became_peered":"2023-10-06T02:32:07.125637+0000","last_unstale":"2023-10-06T02:32:08.146593+0000","last_undegraded":"2023-10-06T02:32:08.146593+0000","last_fullsized":"2023-10-06T02:32:08.146593+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_clean_scrub_stamp":"2023-10-06T02:32:06.100240+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:48:34.026512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.15","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.139830+0000","last_change":"2023-10-06T02:32:08.139830+0000","last_active":"2023-10-06T02:32:08.139830+0000","last_peered":"2023-10-06T02:32:08.139830+0000","last_clean":"2023-10-06T02:32:08.139830+0000","last_became_active":"2023-10-06T02:32:08.139495+0000","last_became_peered":"2023-10-06T02:32:08.139495+0000","last_unstale":"2023-10-06T02:32:08.139830+0000","last_undegraded":"2023-10-06T02:32:08.139830+0000","last_fullsized":"2023-10-06T02:32:08.139830+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:47:51.516673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-06T02:32:07.136118+0000","last_change":"2023-10-06T02:32:07.132876+0000","last_active":"2023-10-06T02:32:07.136118+0000","last_peered":"2023-10-06T02:32:07.136118+0000","last_clean":"2023-10-06T02:32:07.136118+0000","last_became_active":"2023-10-06T02:32:07.132295+0000","last_became_peered":"2023-10-06T02:32:07.132295+0000","last_unstale":"2023-10-06T02:32:07.136118+0000","last_undegraded":"2023-10-06T02:32:07.136118+0000","last_fullsized":"2023-10-06T02:32:07.136118+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_clean_scrub_stamp":"2023-10-06T02:32:06.100240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:27.273877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.143394+0000","last_change":"2023-10-06T02:32:08.143394+0000","last_active":"2023-10-06T02:32:08.143394+0000","last_peered":"2023-10-06T02:32:08.143394+0000","last_clean":"2023-10-06T02:32:08.143394+0000","last_became_active":"2023-10-06T02:32:08.143054+0000","last_became_peered":"2023-10-06T02:32:08.143054+0000","last_unstale":"2023-10-06T02:32:08.143394+0000","last_undegraded":"2023-10-06T02:32:08.143394+0000","last_fullsized":"2023-10-06T02:32:08.143394+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:57:54.863263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-06T02:32:07.132654+0000","last_change":"2023-10-06T02:32:07.128845+0000","last_active":"2023-10-06T02:32:07.132654+0000","last_peered":"2023-10-06T02:32:07.132654+0000","last_clean":"2023-10-06T02:32:07.132654+0000","last_became_active":"2023-10-06T02:32:07.128542+0000","last_became_peered":"2023-10-06T02:32:07.128542+0000","last_unstale":"2023-10-06T02:32:07.132654+0000","last_undegraded":"2023-10-06T02:32:07.132654+0000","last_fullsized":"2023-10-06T02:32:07.132654+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_clean_scrub_stamp":"2023-10-06T02:32:06.100240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:27:18.795673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.148804+0000","last_change":"2023-10-06T02:32:08.148804+0000","last_active":"2023-10-06T02:32:08.148804+0000","last_peered":"2023-10-06T02:32:08.148804+0000","last_clean":"2023-10-06T02:32:08.148804+0000","last_became_active":"2023-10-06T02:32:08.148431+0000","last_became_peered":"2023-10-06T02:32:08.148431+0000","last_unstale":"2023-10-06T02:32:08.148804+0000","last_undegraded":"2023-10-06T02:32:08.148804+0000","last_fullsized":"2023-10-06T02:32:08.148804+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:44:32.474746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.10","version":"168'8","reported_seq":17,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.150266+0000","last_change":"2023-10-06T02:32:07.134817+0000","last_active":"2023-10-06T02:32:08.150266+0000","last_peered":"2023-10-06T02:32:08.150266+0000","last_clean":"2023-10-06T02:32:08.150266+0000","last_became_active":"2023-10-06T02:32:07.134517+0000","last_became_peered":"2023-10-06T02:32:07.134517+0000","last_unstale":"2023-10-06T02:32:08.150266+0000","last_undegraded":"2023-10-06T02:32:08.150266+0000","last_fullsized":"2023-10-06T02:32:08.150266+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_clean_scrub_stamp":"2023-10-06T02:32:06.100240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:38:29.855364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.140270+0000","last_change":"2023-10-06T02:32:08.140270+0000","last_active":"2023-10-06T02:32:08.140270+0000","last_peered":"2023-10-06T02:32:08.140270+0000","last_clean":"2023-10-06T02:32:08.140270+0000","last_became_active":"2023-10-06T02:32:08.139999+0000","last_became_peered":"2023-10-06T02:32:08.139999+0000","last_unstale":"2023-10-06T02:32:08.140270+0000","last_undegraded":"2023-10-06T02:32:08.140270+0000","last_fullsized":"2023-10-06T02:32:08.140270+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:38:36.563684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.f","version":"168'1","reported_seq":11,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.149809+0000","last_change":"2023-10-06T02:32:07.128591+0000","last_active":"2023-10-06T02:32:08.149809+0000","last_peered":"2023-10-06T02:32:08.149809+0000","last_clean":"2023-10-06T02:32:08.149809+0000","last_became_active":"2023-10-06T02:32:07.128323+0000","last_became_peered":"2023-10-06T02:32:07.128323+0000","last_unstale":"2023-10-06T02:32:08.149809+0000","last_undegraded":"2023-10-06T02:32:08.149809+0000","last_fullsized":"2023-10-06T02:32:08.149809+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_clean_scrub_stamp":"2023-10-06T02:32:06.100240+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:50:46.184905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.148497+0000","last_change":"2023-10-06T02:32:08.146338+0000","last_active":"2023-10-06T02:32:08.148497+0000","last_peered":"2023-10-06T02:32:08.148497+0000","last_clean":"2023-10-06T02:32:08.148497+0000","last_became_active":"2023-10-06T02:32:08.145847+0000","last_became_peered":"2023-10-06T02:32:08.145847+0000","last_unstale":"2023-10-06T02:32:08.148497+0000","last_undegraded":"2023-10-06T02:32:08.148497+0000","last_fullsized":"2023-10-06T02:32:08.148497+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:34:14.560103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.e","version":"168'2","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.151272+0000","last_change":"2023-10-06T02:32:07.129750+0000","last_active":"2023-10-06T02:32:08.151272+0000","last_peered":"2023-10-06T02:32:08.151272+0000","last_clean":"2023-10-06T02:32:08.151272+0000","last_became_active":"2023-10-06T02:32:07.129332+0000","last_became_peered":"2023-10-06T02:32:07.129332+0000","last_unstale":"2023-10-06T02:32:08.151272+0000","last_undegraded":"2023-10-06T02:32:08.151272+0000","last_fullsized":"2023-10-06T02:32:08.151272+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_clean_scrub_stamp":"2023-10-06T02:32:06.100240+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:20:11.521570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.133461+0000","last_change":"2023-10-06T02:32:08.131643+0000","last_active":"2023-10-06T02:32:08.133461+0000","last_peered":"2023-10-06T02:32:08.133461+0000","last_clean":"2023-10-06T02:32:08.133461+0000","last_became_active":"2023-10-06T02:32:08.131345+0000","last_became_peered":"2023-10-06T02:32:08.131345+0000","last_unstale":"2023-10-06T02:32:08.133461+0000","last_undegraded":"2023-10-06T02:32:08.133461+0000","last_fullsized":"2023-10-06T02:32:08.133461+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:23:50.218257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.d","version":"168'1","reported_seq":11,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.148925+0000","last_change":"2023-10-06T02:32:07.130842+0000","last_active":"2023-10-06T02:32:08.148925+0000","last_peered":"2023-10-06T02:32:08.148925+0000","last_clean":"2023-10-06T02:32:08.148925+0000","last_became_active":"2023-10-06T02:32:07.130584+0000","last_became_peered":"2023-10-06T02:32:07.130584+0000","last_unstale":"2023-10-06T02:32:08.148925+0000","last_undegraded":"2023-10-06T02:32:08.148925+0000","last_fullsized":"2023-10-06T02:32:08.148925+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_clean_scrub_stamp":"2023-10-06T02:32:06.100240+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:00:51.603032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.b","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.140504+0000","last_change":"2023-10-06T02:32:08.140504+0000","last_active":"2023-10-06T02:32:08.140504+0000","last_peered":"2023-10-06T02:32:08.140504+0000","last_clean":"2023-10-06T02:32:08.140504+0000","last_became_active":"2023-10-06T02:32:08.140240+0000","last_became_peered":"2023-10-06T02:32:08.140240+0000","last_unstale":"2023-10-06T02:32:08.140504+0000","last_undegraded":"2023-10-06T02:32:08.140504+0000","last_fullsized":"2023-10-06T02:32:08.140504+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:34:01.099306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-06T02:32:07.126516+0000","last_change":"2023-10-06T02:32:07.124784+0000","last_active":"2023-10-06T02:32:07.126516+0000","last_peered":"2023-10-06T02:32:07.126516+0000","last_clean":"2023-10-06T02:32:07.126516+0000","last_became_active":"2023-10-06T02:32:07.124455+0000","last_became_peered":"2023-10-06T02:32:07.124455+0000","last_unstale":"2023-10-06T02:32:07.126516+0000","last_undegraded":"2023-10-06T02:32:07.126516+0000","last_fullsized":"2023-10-06T02:32:07.126516+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_clean_scrub_stamp":"2023-10-06T02:32:06.100240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:53.114922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.139874+0000","last_change":"2023-10-06T02:32:08.138347+0000","last_active":"2023-10-06T02:32:08.139874+0000","last_peered":"2023-10-06T02:32:08.139874+0000","last_clean":"2023-10-06T02:32:08.139874+0000","last_became_active":"2023-10-06T02:32:08.137515+0000","last_became_peered":"2023-10-06T02:32:08.137515+0000","last_unstale":"2023-10-06T02:32:08.139874+0000","last_undegraded":"2023-10-06T02:32:08.139874+0000","last_fullsized":"2023-10-06T02:32:08.139874+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:10:41.774263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"168'2","reported_seq":11,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.140883+0000","last_change":"2023-10-06T02:32:07.129048+0000","last_active":"2023-10-06T02:32:08.140883+0000","last_peered":"2023-10-06T02:32:08.140883+0000","last_clean":"2023-10-06T02:32:08.140883+0000","last_became_active":"2023-10-06T02:32:07.128786+0000","last_became_peered":"2023-10-06T02:32:07.128786+0000","last_unstale":"2023-10-06T02:32:08.140883+0000","last_undegraded":"2023-10-06T02:32:08.140883+0000","last_fullsized":"2023-10-06T02:32:08.140883+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_clean_scrub_stamp":"2023-10-06T02:32:06.100240+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:34:00.095848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.d","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.138174+0000","last_change":"2023-10-06T02:32:08.138174+0000","last_active":"2023-10-06T02:32:08.138174+0000","last_peered":"2023-10-06T02:32:08.138174+0000","last_clean":"2023-10-06T02:32:08.138174+0000","last_became_active":"2023-10-06T02:32:08.137500+0000","last_became_peered":"2023-10-06T02:32:08.137500+0000","last_unstale":"2023-10-06T02:32:08.138174+0000","last_undegraded":"2023-10-06T02:32:08.138174+0000","last_fullsized":"2023-10-06T02:32:08.138174+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:30:00.558795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":10,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.118960+0000","last_change":"2023-10-06T02:32:07.131601+0000","last_active":"2023-10-06T02:32:08.118960+0000","last_peered":"2023-10-06T02:32:08.118960+0000","last_clean":"2023-10-06T02:32:08.118960+0000","last_became_active":"2023-10-06T02:32:07.131286+0000","last_became_peered":"2023-10-06T02:32:07.131286+0000","last_unstale":"2023-10-06T02:32:08.118960+0000","last_undegraded":"2023-10-06T02:32:08.118960+0000","last_fullsized":"2023-10-06T02:32:08.118960+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_clean_scrub_stamp":"2023-10-06T02:32:06.100240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:27:05.038453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.e","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.148747+0000","last_change":"2023-10-06T02:32:08.148747+0000","last_active":"2023-10-06T02:32:08.148747+0000","last_peered":"2023-10-06T02:32:08.148747+0000","last_clean":"2023-10-06T02:32:08.148747+0000","last_became_active":"2023-10-06T02:32:08.148449+0000","last_became_peered":"2023-10-06T02:32:08.148449+0000","last_unstale":"2023-10-06T02:32:08.148747+0000","last_undegraded":"2023-10-06T02:32:08.148747+0000","last_fullsized":"2023-10-06T02:32:08.148747+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:26:47.123088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":10,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.129723+0000","last_change":"2023-10-06T02:32:07.135376+0000","last_active":"2023-10-06T02:32:08.129723+0000","last_peered":"2023-10-06T02:32:08.129723+0000","last_clean":"2023-10-06T02:32:08.129723+0000","last_became_active":"2023-10-06T02:32:07.134293+0000","last_became_peered":"2023-10-06T02:32:07.134293+0000","last_unstale":"2023-10-06T02:32:08.129723+0000","last_undegraded":"2023-10-06T02:32:08.129723+0000","last_fullsized":"2023-10-06T02:32:08.129723+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_clean_scrub_stamp":"2023-10-06T02:32:06.100240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:47:09.341380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.f","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.138032+0000","last_change":"2023-10-06T02:32:08.138032+0000","last_active":"2023-10-06T02:32:08.138032+0000","last_peered":"2023-10-06T02:32:08.138032+0000","last_clean":"2023-10-06T02:32:08.138032+0000","last_became_active":"2023-10-06T02:32:08.134106+0000","last_became_peered":"2023-10-06T02:32:08.134106+0000","last_unstale":"2023-10-06T02:32:08.138032+0000","last_undegraded":"2023-10-06T02:32:08.138032+0000","last_fullsized":"2023-10-06T02:32:08.138032+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:34:28.727551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":10,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.118329+0000","last_change":"2023-10-06T02:32:07.132018+0000","last_active":"2023-10-06T02:32:08.118329+0000","last_peered":"2023-10-06T02:32:08.118329+0000","last_clean":"2023-10-06T02:32:08.118329+0000","last_became_active":"2023-10-06T02:32:07.131735+0000","last_became_peered":"2023-10-06T02:32:07.131735+0000","last_unstale":"2023-10-06T02:32:08.118329+0000","last_undegraded":"2023-10-06T02:32:08.118329+0000","last_fullsized":"2023-10-06T02:32:08.118329+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_clean_scrub_stamp":"2023-10-06T02:32:06.100240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:58:21.701178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.139533+0000","last_change":"2023-10-06T02:32:08.139533+0000","last_active":"2023-10-06T02:32:08.139533+0000","last_peered":"2023-10-06T02:32:08.139533+0000","last_clean":"2023-10-06T02:32:08.139533+0000","last_became_active":"2023-10-06T02:32:08.139248+0000","last_became_peered":"2023-10-06T02:32:08.139248+0000","last_unstale":"2023-10-06T02:32:08.139533+0000","last_undegraded":"2023-10-06T02:32:08.139533+0000","last_fullsized":"2023-10-06T02:32:08.139533+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:54.118380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.7","version":"168'2","reported_seq":11,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.149045+0000","last_change":"2023-10-06T02:32:07.132994+0000","last_active":"2023-10-06T02:32:08.149045+0000","last_peered":"2023-10-06T02:32:08.149045+0000","last_clean":"2023-10-06T02:32:08.149045+0000","last_became_active":"2023-10-06T02:32:07.129783+0000","last_became_peered":"2023-10-06T02:32:07.129783+0000","last_unstale":"2023-10-06T02:32:08.149045+0000","last_undegraded":"2023-10-06T02:32:08.149045+0000","last_fullsized":"2023-10-06T02:32:08.149045+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_clean_scrub_stamp":"2023-10-06T02:32:06.100240+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:18:48.153012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.138250+0000","last_change":"2023-10-06T02:32:08.138250+0000","last_active":"2023-10-06T02:32:08.138250+0000","last_peered":"2023-10-06T02:32:08.138250+0000","last_clean":"2023-10-06T02:32:08.138250+0000","last_became_active":"2023-10-06T02:32:08.137837+0000","last_became_peered":"2023-10-06T02:32:08.137837+0000","last_unstale":"2023-10-06T02:32:08.138250+0000","last_undegraded":"2023-10-06T02:32:08.138250+0000","last_fullsized":"2023-10-06T02:32:08.138250+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:50.611252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-06T02:32:07.139099+0000","last_change":"2023-10-06T02:32:07.130291+0000","last_active":"2023-10-06T02:32:07.139099+0000","last_peered":"2023-10-06T02:32:07.139099+0000","last_clean":"2023-10-06T02:32:07.139099+0000","last_became_active":"2023-10-06T02:32:07.130018+0000","last_became_peered":"2023-10-06T02:32:07.130018+0000","last_unstale":"2023-10-06T02:32:07.139099+0000","last_undegraded":"2023-10-06T02:32:07.139099+0000","last_fullsized":"2023-10-06T02:32:07.139099+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_clean_scrub_stamp":"2023-10-06T02:32:06.100240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:22:21.025639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.144249+0000","last_change":"2023-10-06T02:32:08.144249+0000","last_active":"2023-10-06T02:32:08.144249+0000","last_peered":"2023-10-06T02:32:08.144249+0000","last_clean":"2023-10-06T02:32:08.144249+0000","last_became_active":"2023-10-06T02:32:08.144002+0000","last_became_peered":"2023-10-06T02:32:08.144002+0000","last_unstale":"2023-10-06T02:32:08.144249+0000","last_undegraded":"2023-10-06T02:32:08.144249+0000","last_fullsized":"2023-10-06T02:32:08.144249+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:48:18.282510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":10,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.127701+0000","last_change":"2023-10-06T02:32:07.138800+0000","last_active":"2023-10-06T02:32:08.127701+0000","last_peered":"2023-10-06T02:32:08.127701+0000","last_clean":"2023-10-06T02:32:08.127701+0000","last_became_active":"2023-10-06T02:32:07.138483+0000","last_became_peered":"2023-10-06T02:32:07.138483+0000","last_unstale":"2023-10-06T02:32:08.127701+0000","last_undegraded":"2023-10-06T02:32:08.127701+0000","last_fullsized":"2023-10-06T02:32:08.127701+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_clean_scrub_stamp":"2023-10-06T02:32:06.100240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:08:44.456006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.147143+0000","last_change":"2023-10-06T02:32:08.138461+0000","last_active":"2023-10-06T02:32:08.147143+0000","last_peered":"2023-10-06T02:32:08.147143+0000","last_clean":"2023-10-06T02:32:08.147143+0000","last_became_active":"2023-10-06T02:32:08.138055+0000","last_became_peered":"2023-10-06T02:32:08.138055+0000","last_unstale":"2023-10-06T02:32:08.147143+0000","last_undegraded":"2023-10-06T02:32:08.147143+0000","last_fullsized":"2023-10-06T02:32:08.147143+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:18:01.461302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":10,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.118354+0000","last_change":"2023-10-06T02:32:07.133367+0000","last_active":"2023-10-06T02:32:08.118354+0000","last_peered":"2023-10-06T02:32:08.118354+0000","last_clean":"2023-10-06T02:32:08.118354+0000","last_became_active":"2023-10-06T02:32:07.132604+0000","last_became_peered":"2023-10-06T02:32:07.132604+0000","last_unstale":"2023-10-06T02:32:08.118354+0000","last_undegraded":"2023-10-06T02:32:08.118354+0000","last_fullsized":"2023-10-06T02:32:08.118354+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_clean_scrub_stamp":"2023-10-06T02:32:06.100240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:27.273877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.137583+0000","last_change":"2023-10-06T02:32:08.137583+0000","last_active":"2023-10-06T02:32:08.137583+0000","last_peered":"2023-10-06T02:32:08.137583+0000","last_clean":"2023-10-06T02:32:08.137583+0000","last_became_active":"2023-10-06T02:32:08.136950+0000","last_became_peered":"2023-10-06T02:32:08.136950+0000","last_unstale":"2023-10-06T02:32:08.137583+0000","last_undegraded":"2023-10-06T02:32:08.137583+0000","last_fullsized":"2023-10-06T02:32:08.137583+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:56:28.297716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-06T02:32:07.139034+0000","last_change":"2023-10-06T02:32:07.129126+0000","last_active":"2023-10-06T02:32:07.139034+0000","last_peered":"2023-10-06T02:32:07.139034+0000","last_clean":"2023-10-06T02:32:07.139034+0000","last_became_active":"2023-10-06T02:32:07.128851+0000","last_became_peered":"2023-10-06T02:32:07.128851+0000","last_unstale":"2023-10-06T02:32:07.139034+0000","last_undegraded":"2023-10-06T02:32:07.139034+0000","last_fullsized":"2023-10-06T02:32:07.139034+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_clean_scrub_stamp":"2023-10-06T02:32:06.100240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:05:23.675738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.138375+0000","last_change":"2023-10-06T02:32:08.138375+0000","last_active":"2023-10-06T02:32:08.138375+0000","last_peered":"2023-10-06T02:32:08.138375+0000","last_clean":"2023-10-06T02:32:08.138375+0000","last_became_active":"2023-10-06T02:32:08.137817+0000","last_became_peered":"2023-10-06T02:32:08.137817+0000","last_unstale":"2023-10-06T02:32:08.138375+0000","last_undegraded":"2023-10-06T02:32:08.138375+0000","last_fullsized":"2023-10-06T02:32:08.138375+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:22:30.522945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-06T02:32:07.135257+0000","last_change":"2023-10-06T02:32:07.134003+0000","last_active":"2023-10-06T02:32:07.135257+0000","last_peered":"2023-10-06T02:32:07.135257+0000","last_clean":"2023-10-06T02:32:07.135257+0000","last_became_active":"2023-10-06T02:32:07.133596+0000","last_became_peered":"2023-10-06T02:32:07.133596+0000","last_unstale":"2023-10-06T02:32:07.135257+0000","last_undegraded":"2023-10-06T02:32:07.135257+0000","last_fullsized":"2023-10-06T02:32:07.135257+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_clean_scrub_stamp":"2023-10-06T02:32:06.100240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:00:51.603032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.6","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.145520+0000","last_change":"2023-10-06T02:32:08.145520+0000","last_active":"2023-10-06T02:32:08.145520+0000","last_peered":"2023-10-06T02:32:08.145520+0000","last_clean":"2023-10-06T02:32:08.145520+0000","last_became_active":"2023-10-06T02:32:08.145250+0000","last_became_peered":"2023-10-06T02:32:08.145250+0000","last_unstale":"2023-10-06T02:32:08.145520+0000","last_undegraded":"2023-10-06T02:32:08.145520+0000","last_fullsized":"2023-10-06T02:32:08.145520+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:48:53.198148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":10,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.119755+0000","last_change":"2023-10-06T02:32:07.133488+0000","last_active":"2023-10-06T02:32:08.119755+0000","last_peered":"2023-10-06T02:32:08.119755+0000","last_clean":"2023-10-06T02:32:08.119755+0000","last_became_active":"2023-10-06T02:32:07.132869+0000","last_became_peered":"2023-10-06T02:32:07.132869+0000","last_unstale":"2023-10-06T02:32:08.119755+0000","last_undegraded":"2023-10-06T02:32:08.119755+0000","last_fullsized":"2023-10-06T02:32:08.119755+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_clean_scrub_stamp":"2023-10-06T02:32:06.100240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:22:21.226937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.7","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.143903+0000","last_change":"2023-10-06T02:32:08.143903+0000","last_active":"2023-10-06T02:32:08.143903+0000","last_peered":"2023-10-06T02:32:08.143903+0000","last_clean":"2023-10-06T02:32:08.143903+0000","last_became_active":"2023-10-06T02:32:08.143593+0000","last_became_peered":"2023-10-06T02:32:08.143593+0000","last_unstale":"2023-10-06T02:32:08.143903+0000","last_undegraded":"2023-10-06T02:32:08.143903+0000","last_fullsized":"2023-10-06T02:32:08.143903+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:33:05.856323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-06T02:32:07.137160+0000","last_change":"2023-10-06T02:32:07.134437+0000","last_active":"2023-10-06T02:32:07.137160+0000","last_peered":"2023-10-06T02:32:07.137160+0000","last_clean":"2023-10-06T02:32:07.137160+0000","last_became_active":"2023-10-06T02:32:07.134183+0000","last_became_peered":"2023-10-06T02:32:07.134183+0000","last_unstale":"2023-10-06T02:32:07.137160+0000","last_undegraded":"2023-10-06T02:32:07.137160+0000","last_fullsized":"2023-10-06T02:32:07.137160+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_clean_scrub_stamp":"2023-10-06T02:32:06.100240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:27:21.788539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.147267+0000","last_change":"2023-10-06T02:32:08.141501+0000","last_active":"2023-10-06T02:32:08.147267+0000","last_peered":"2023-10-06T02:32:08.147267+0000","last_clean":"2023-10-06T02:32:08.147267+0000","last_became_active":"2023-10-06T02:32:08.141207+0000","last_became_peered":"2023-10-06T02:32:08.141207+0000","last_unstale":"2023-10-06T02:32:08.147267+0000","last_undegraded":"2023-10-06T02:32:08.147267+0000","last_fullsized":"2023-10-06T02:32:08.147267+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:34.296319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1e","version":"168'1","reported_seq":11,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.147238+0000","last_change":"2023-10-06T02:32:07.129597+0000","last_active":"2023-10-06T02:32:08.147238+0000","last_peered":"2023-10-06T02:32:08.147238+0000","last_clean":"2023-10-06T02:32:08.147238+0000","last_became_active":"2023-10-06T02:32:07.128946+0000","last_became_peered":"2023-10-06T02:32:07.128946+0000","last_unstale":"2023-10-06T02:32:08.147238+0000","last_undegraded":"2023-10-06T02:32:08.147238+0000","last_fullsized":"2023-10-06T02:32:08.147238+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_clean_scrub_stamp":"2023-10-06T02:32:06.100240+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:44:00.675705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":134,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-06T02:32:07.116964+0000","last_change":"2023-10-06T02:23:01.992935+0000","last_active":"2023-10-06T02:32:07.116964+0000","last_peered":"2023-10-06T02:32:07.116964+0000","last_clean":"2023-10-06T02:32:07.116964+0000","last_became_active":"2023-10-06T02:23:01.992568+0000","last_became_peered":"2023-10-06T02:23:01.992568+0000","last_unstale":"2023-10-06T02:32:07.116964+0000","last_undegraded":"2023-10-06T02:32:07.116964+0000","last_fullsized":"2023-10-06T02:32:07.116964+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:06.212251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.129549+0000","last_change":"2023-10-06T02:32:08.127932+0000","last_active":"2023-10-06T02:32:08.129549+0000","last_peered":"2023-10-06T02:32:08.129549+0000","last_clean":"2023-10-06T02:32:08.129549+0000","last_became_active":"2023-10-06T02:32:08.127648+0000","last_became_peered":"2023-10-06T02:32:08.127648+0000","last_unstale":"2023-10-06T02:32:08.129549+0000","last_undegraded":"2023-10-06T02:32:08.129549+0000","last_fullsized":"2023-10-06T02:32:08.129549+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:38:40.218933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.1f","version":"168'4","reported_seq":14,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.149392+0000","last_change":"2023-10-06T02:32:07.140369+0000","last_active":"2023-10-06T02:32:08.149392+0000","last_peered":"2023-10-06T02:32:08.149392+0000","last_clean":"2023-10-06T02:32:08.149392+0000","last_became_active":"2023-10-06T02:32:07.140046+0000","last_became_peered":"2023-10-06T02:32:07.140046+0000","last_unstale":"2023-10-06T02:32:08.149392+0000","last_undegraded":"2023-10-06T02:32:08.149392+0000","last_fullsized":"2023-10-06T02:32:08.149392+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_clean_scrub_stamp":"2023-10-06T02:32:06.100240+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:58:00.220771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":135,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-06T02:32:07.126621+0000","last_change":"2023-10-06T02:23:01.989934+0000","last_active":"2023-10-06T02:32:07.126621+0000","last_peered":"2023-10-06T02:32:07.126621+0000","last_clean":"2023-10-06T02:32:07.126621+0000","last_became_active":"2023-10-06T02:23:01.989706+0000","last_became_peered":"2023-10-06T02:23:01.989706+0000","last_unstale":"2023-10-06T02:32:07.126621+0000","last_undegraded":"2023-10-06T02:32:07.126621+0000","last_fullsized":"2023-10-06T02:32:07.126621+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:06.212251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.1b","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.129297+0000","last_change":"2023-10-06T02:32:08.129297+0000","last_active":"2023-10-06T02:32:08.129297+0000","last_peered":"2023-10-06T02:32:08.129297+0000","last_clean":"2023-10-06T02:32:08.129297+0000","last_became_active":"2023-10-06T02:32:08.128885+0000","last_became_peered":"2023-10-06T02:32:08.128885+0000","last_unstale":"2023-10-06T02:32:08.129297+0000","last_undegraded":"2023-10-06T02:32:08.129297+0000","last_fullsized":"2023-10-06T02:32:08.129297+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:20:13.257560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1c","version":"168'2","reported_seq":11,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.144557+0000","last_change":"2023-10-06T02:32:07.133974+0000","last_active":"2023-10-06T02:32:08.144557+0000","last_peered":"2023-10-06T02:32:08.144557+0000","last_clean":"2023-10-06T02:32:08.144557+0000","last_became_active":"2023-10-06T02:32:07.133538+0000","last_became_peered":"2023-10-06T02:32:07.133538+0000","last_unstale":"2023-10-06T02:32:08.144557+0000","last_undegraded":"2023-10-06T02:32:08.144557+0000","last_fullsized":"2023-10-06T02:32:08.144557+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_clean_scrub_stamp":"2023-10-06T02:32:06.100240+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:19:59.093429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":128,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-06T02:32:07.130536+0000","last_change":"2023-10-06T02:23:02.054069+0000","last_active":"2023-10-06T02:32:07.130536+0000","last_peered":"2023-10-06T02:32:07.130536+0000","last_clean":"2023-10-06T02:32:07.130536+0000","last_became_active":"2023-10-06T02:23:02.053739+0000","last_became_peered":"2023-10-06T02:23:02.053739+0000","last_unstale":"2023-10-06T02:32:07.130536+0000","last_undegraded":"2023-10-06T02:32:07.130536+0000","last_fullsized":"2023-10-06T02:32:07.130536+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:57:48.822608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1e","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.146820+0000","last_change":"2023-10-06T02:32:08.143629+0000","last_active":"2023-10-06T02:32:08.146820+0000","last_peered":"2023-10-06T02:32:08.146820+0000","last_clean":"2023-10-06T02:32:08.146820+0000","last_became_active":"2023-10-06T02:32:08.143314+0000","last_became_peered":"2023-10-06T02:32:08.143314+0000","last_unstale":"2023-10-06T02:32:08.146820+0000","last_undegraded":"2023-10-06T02:32:08.146820+0000","last_fullsized":"2023-10-06T02:32:08.146820+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:20:12.525028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-06T02:32:07.139952+0000","last_change":"2023-10-06T02:32:07.136501+0000","last_active":"2023-10-06T02:32:07.139952+0000","last_peered":"2023-10-06T02:32:07.139952+0000","last_clean":"2023-10-06T02:32:07.139952+0000","last_became_active":"2023-10-06T02:32:07.135756+0000","last_became_peered":"2023-10-06T02:32:07.135756+0000","last_unstale":"2023-10-06T02:32:07.139952+0000","last_undegraded":"2023-10-06T02:32:07.139952+0000","last_fullsized":"2023-10-06T02:32:07.139952+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_clean_scrub_stamp":"2023-10-06T02:32:06.100240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:44:23.676780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":128,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-06T02:32:07.126874+0000","last_change":"2023-10-06T02:23:01.990773+0000","last_active":"2023-10-06T02:32:07.126874+0000","last_peered":"2023-10-06T02:32:07.126874+0000","last_clean":"2023-10-06T02:32:07.126874+0000","last_became_active":"2023-10-06T02:23:01.990511+0000","last_became_peered":"2023-10-06T02:23:01.990511+0000","last_unstale":"2023-10-06T02:32:07.126874+0000","last_undegraded":"2023-10-06T02:32:07.126874+0000","last_fullsized":"2023-10-06T02:32:07.126874+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:42:54.797731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1d","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.145670+0000","last_change":"2023-10-06T02:32:08.134710+0000","last_active":"2023-10-06T02:32:08.145670+0000","last_peered":"2023-10-06T02:32:08.145670+0000","last_clean":"2023-10-06T02:32:08.145670+0000","last_became_active":"2023-10-06T02:32:08.133600+0000","last_became_peered":"2023-10-06T02:32:08.133600+0000","last_unstale":"2023-10-06T02:32:08.145670+0000","last_undegraded":"2023-10-06T02:32:08.145670+0000","last_fullsized":"2023-10-06T02:32:08.145670+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:50.611252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1a","version":"168'1","reported_seq":10,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.123003+0000","last_change":"2023-10-06T02:32:07.129097+0000","last_active":"2023-10-06T02:32:08.123003+0000","last_peered":"2023-10-06T02:32:08.123003+0000","last_clean":"2023-10-06T02:32:08.123003+0000","last_became_active":"2023-10-06T02:32:07.128807+0000","last_became_peered":"2023-10-06T02:32:07.128807+0000","last_unstale":"2023-10-06T02:32:08.123003+0000","last_undegraded":"2023-10-06T02:32:08.123003+0000","last_fullsized":"2023-10-06T02:32:08.123003+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_clean_scrub_stamp":"2023-10-06T02:32:06.100240+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:53.451053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"76'128","reported_seq":320,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-06T02:32:07.127016+0000","last_change":"2023-10-06T02:22:55.032519+0000","last_active":"2023-10-06T02:32:07.127016+0000","last_peered":"2023-10-06T02:32:07.127016+0000","last_clean":"2023-10-06T02:32:07.127016+0000","last_became_active":"2023-10-06T02:22:55.031367+0000","last_became_peered":"2023-10-06T02:22:55.031367+0000","last_unstale":"2023-10-06T02:32:07.127016+0000","last_undegraded":"2023-10-06T02:32:07.127016+0000","last_fullsized":"2023-10-06T02:32:07.127016+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:22:53.309097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:22:53.309097+0000","last_clean_scrub_stamp":"2023-10-06T02:22:53.309097+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:53:46.520909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":159,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.118706+0000","last_change":"2023-10-06T02:23:01.988140+0000","last_active":"2023-10-06T02:32:08.118706+0000","last_peered":"2023-10-06T02:32:08.118706+0000","last_clean":"2023-10-06T02:32:08.118706+0000","last_became_active":"2023-10-06T02:23:01.987769+0000","last_became_peered":"2023-10-06T02:23:01.987769+0000","last_unstale":"2023-10-06T02:32:08.118706+0000","last_undegraded":"2023-10-06T02:32:08.118706+0000","last_fullsized":"2023-10-06T02:32:08.118706+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:20:08.391912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.1f","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.145160+0000","last_change":"2023-10-06T02:32:08.145160+0000","last_active":"2023-10-06T02:32:08.145160+0000","last_peered":"2023-10-06T02:32:08.145160+0000","last_clean":"2023-10-06T02:32:08.145160+0000","last_became_active":"2023-10-06T02:32:08.144918+0000","last_became_peered":"2023-10-06T02:32:08.144918+0000","last_unstale":"2023-10-06T02:32:08.145160+0000","last_undegraded":"2023-10-06T02:32:08.145160+0000","last_fullsized":"2023-10-06T02:32:08.145160+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:15:37.227204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-06T02:32:07.136072+0000","last_change":"2023-10-06T02:32:07.132943+0000","last_active":"2023-10-06T02:32:07.136072+0000","last_peered":"2023-10-06T02:32:07.136072+0000","last_clean":"2023-10-06T02:32:07.136072+0000","last_became_active":"2023-10-06T02:32:07.132598+0000","last_became_peered":"2023-10-06T02:32:07.132598+0000","last_unstale":"2023-10-06T02:32:07.136072+0000","last_undegraded":"2023-10-06T02:32:07.136072+0000","last_fullsized":"2023-10-06T02:32:07.136072+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_clean_scrub_stamp":"2023-10-06T02:32:06.100240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:44:00.675705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":145,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-06T02:32:07.112141+0000","last_change":"2023-10-06T02:23:02.054338+0000","last_active":"2023-10-06T02:32:07.112141+0000","last_peered":"2023-10-06T02:32:07.112141+0000","last_clean":"2023-10-06T02:32:07.112141+0000","last_became_active":"2023-10-06T02:23:02.053991+0000","last_became_peered":"2023-10-06T02:23:02.053991+0000","last_unstale":"2023-10-06T02:32:07.112141+0000","last_undegraded":"2023-10-06T02:32:07.112141+0000","last_fullsized":"2023-10-06T02:32:07.112141+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:39:30.192838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.1c","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.145534+0000","last_change":"2023-10-06T02:32:08.140801+0000","last_active":"2023-10-06T02:32:08.145534+0000","last_peered":"2023-10-06T02:32:08.145534+0000","last_clean":"2023-10-06T02:32:08.145534+0000","last_became_active":"2023-10-06T02:32:08.140525+0000","last_became_peered":"2023-10-06T02:32:08.140525+0000","last_unstale":"2023-10-06T02:32:08.145534+0000","last_undegraded":"2023-10-06T02:32:08.145534+0000","last_fullsized":"2023-10-06T02:32:08.145534+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:50:47.188363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1b","version":"168'1","reported_seq":10,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.128535+0000","last_change":"2023-10-06T02:32:07.134198+0000","last_active":"2023-10-06T02:32:08.128535+0000","last_peered":"2023-10-06T02:32:08.128535+0000","last_clean":"2023-10-06T02:32:08.128535+0000","last_became_active":"2023-10-06T02:32:07.133937+0000","last_became_peered":"2023-10-06T02:32:07.133937+0000","last_unstale":"2023-10-06T02:32:08.128535+0000","last_undegraded":"2023-10-06T02:32:08.128535+0000","last_fullsized":"2023-10-06T02:32:08.128535+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_clean_scrub_stamp":"2023-10-06T02:32:06.100240+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:23:49.214799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":159,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.120156+0000","last_change":"2023-10-06T02:23:01.988631+0000","last_active":"2023-10-06T02:32:08.120156+0000","last_peered":"2023-10-06T02:32:08.120156+0000","last_clean":"2023-10-06T02:32:08.120156+0000","last_became_active":"2023-10-06T02:23:01.988303+0000","last_became_peered":"2023-10-06T02:23:01.988303+0000","last_unstale":"2023-10-06T02:32:08.120156+0000","last_undegraded":"2023-10-06T02:32:08.120156+0000","last_fullsized":"2023-10-06T02:32:08.120156+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:38.211466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.1a","version":"0'0","reported_seq":8,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.124115+0000","last_change":"2023-10-06T02:32:08.124115+0000","last_active":"2023-10-06T02:32:08.124115+0000","last_peered":"2023-10-06T02:32:08.124115+0000","last_clean":"2023-10-06T02:32:08.124115+0000","last_became_active":"2023-10-06T02:32:08.123772+0000","last_became_peered":"2023-10-06T02:32:08.123772+0000","last_unstale":"2023-10-06T02:32:08.124115+0000","last_undegraded":"2023-10-06T02:32:08.124115+0000","last_fullsized":"2023-10-06T02:32:08.124115+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:07.103698+0000","last_clean_scrub_stamp":"2023-10-06T02:32:07.103698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:29:57.398331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1d","version":"168'2","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T02:32:08.141999+0000","last_change":"2023-10-06T02:32:07.127620+0000","last_active":"2023-10-06T02:32:08.141999+0000","last_peered":"2023-10-06T02:32:08.141999+0000","last_clean":"2023-10-06T02:32:08.141999+0000","last_became_active":"2023-10-06T02:32:07.127314+0000","last_became_peered":"2023-10-06T02:32:07.127314+0000","last_unstale":"2023-10-06T02:32:08.141999+0000","last_undegraded":"2023-10-06T02:32:08.141999+0000","last_fullsized":"2023-10-06T02:32:08.141999+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:06.100240+0000","last_clean_scrub_stamp":"2023-10-06T02:32:06.100240+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:26:50.885561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":135,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-06T02:32:07.126690+0000","last_change":"2023-10-06T02:23:02.054039+0000","last_active":"2023-10-06T02:32:07.126690+0000","last_peered":"2023-10-06T02:32:07.126690+0000","last_clean":"2023-10-06T02:32:07.126690+0000","last_became_active":"2023-10-06T02:23:02.053742+0000","last_became_peered":"2023-10-06T02:23:02.053742+0000","last_unstale":"2023-10-06T02:32:07.126690+0000","last_undegraded":"2023-10-06T02:32:07.126690+0000","last_fullsized":"2023-10-06T02:32:07.126690+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:58:18.846475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":28,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":27,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574964,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304052,"kb_used_data":2668,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067788,"statfs":{"total":96636764160,"available":96325414912,"internally_reserved":0,"allocated":2732032,"data_stored":1582946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:31:14 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.622,"15min":0.63300000000000001},"min":{"1min":0.35299999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.78300000000000003,"5min":1.169,"15min":1.169},"last":7.9160000000000004},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.624,"15min":0.61199999999999999},"min":{"1min":0.43099999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.91800000000000004,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":11.643000000000001}]},{"osd":1,"last update":"Fri Oct 6 02:31:14 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71399999999999997,"15min":0.69699999999999995},"min":{"1min":0.57899999999999996,"5min":0.45600000000000002,"15min":0.40300000000000002},"max":{"1min":0.86199999999999999,"5min":1.1459999999999999,"15min":1.147},"last":1.268},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65900000000000003,"15min":0.67100000000000004},"min":{"1min":0.52100000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83899999999999997,"5min":1.129,"15min":1.129},"last":3.0219999999999998}]},{"osd":2,"last update":"Fri Oct 6 02:31:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.65500000000000003,"15min":0.68300000000000005},"min":{"1min":0.46700000000000003,"5min":0.375,"15min":0.375},"max":{"1min":0.81699999999999995,"5min":1.238,"15min":1.238},"last":3.0070000000000001},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.68700000000000006,"15min":0.69099999999999995},"min":{"1min":0.55000000000000004,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.93700000000000006,"5min":1.242,"15min":1.242},"last":12.118}]},{"osd":3,"last update":"Fri Oct 6 02:31:14 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.70799999999999996,"15min":0.72399999999999998},"min":{"1min":0.45500000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.84299999999999997,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":3.073},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.75700000000000001,"15min":0.77200000000000002},"min":{"1min":0.61399999999999999,"5min":0.54500000000000004,"15min":0.497},"max":{"1min":0.95199999999999996,"5min":1.383,"15min":1.383},"last":3.105}]},{"osd":4,"last update":"Fri Oct 6 02:31:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.60299999999999998,"15min":0.66700000000000004},"min":{"1min":0.41799999999999998,"5min":0.41099999999999998,"15min":0.35099999999999998},"max":{"1min":0.91700000000000004,"5min":1.1120000000000001,"15min":1.2849999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.67300000000000004,"15min":0.73899999999999999},"min":{"1min":0.40899999999999997,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.96299999999999997,"5min":1.51,"15min":1.51},"last":8.0060000000000002}]},{"osd":5,"last update":"Fri Oct 6 02:31:14 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.70099999999999996,"15min":0.72499999999999998},"min":{"1min":0.503,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.91900000000000004,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":3.524},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.67000000000000004,"15min":0.71599999999999997},"min":{"1min":0.48999999999999999,"5min":0.442,"15min":0.442},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0289999999999999},"last":3.258}]},{"osd":6,"last update":"Fri Oct 6 02:31:14 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.752,"15min":0.77800000000000002},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.93400000000000005,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":3.4689999999999999},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.77100000000000002,"15min":0.80300000000000005},"min":{"1min":0.628,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.90500000000000003,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":3.4279999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574964,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303472,"kb_used_data":2088,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068368,"statfs":{"total":96636764160,"available":96326008832,"internally_reserved":0,"allocated":2138112,"data_stored":993062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:31:09 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.67400000000000004,"15min":0.66700000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":1.869},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.72099999999999997,"15min":0.70399999999999996},"min":{"1min":0.498,"5min":0.374,"15min":0.374},"max":{"1min":1.04,"5min":1.04,"15min":1.119},"last":1.27}]},{"osd":1,"last update":"Fri Oct 6 02:31:09 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.66800000000000004,"15min":0.65000000000000002},"min":{"1min":0.48699999999999999,"5min":0.433,"15min":0.40699999999999997},"max":{"1min":0.83799999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.66600000000000004,"15min":0.66700000000000004},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.94499999999999995,"5min":0.997,"15min":0.997},"last":1.927}]},{"osd":2,"last update":"Fri Oct 6 02:31:09 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.71599999999999997,"15min":0.70599999999999996},"min":{"1min":0.51000000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.002,"5min":1.0449999999999999,"15min":1.137},"last":1.218},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.72799999999999998,"15min":0.71799999999999997},"min":{"1min":0.47799999999999998,"5min":0.45300000000000001,"15min":0.40300000000000002},"max":{"1min":0.98699999999999999,"5min":1.077,"15min":1.077},"last":1.3149999999999999}]},{"osd":3,"last update":"Fri Oct 6 02:31:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.70099999999999996,"15min":0.67200000000000004},"min":{"1min":0.51500000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.82099999999999995,"5min":1.056,"15min":1.056},"last":1.3620000000000001},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.71399999999999997,"15min":0.70699999999999996},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.45900000000000002},"max":{"1min":0.91600000000000004,"5min":0.97999999999999998,"15min":0.98499999999999999},"last":0.96299999999999997}]},{"osd":4,"last update":"Fri Oct 6 02:31:09 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.66500000000000004,"15min":0.64500000000000002},"min":{"1min":0.377,"5min":0.36199999999999999,"15min":0.34300000000000003},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":1.095},"last":1.97},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.71299999999999997,"15min":0.67500000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.998,"5min":1.002,"15min":1.29},"last":2.7429999999999999}]},{"osd":5,"last update":"Fri Oct 6 02:31:09 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.69799999999999995,"15min":0.69999999999999996},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.39700000000000002},"max":{"1min":0.98699999999999999,"5min":1.014,"15min":1.2929999999999999},"last":1.091},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.71999999999999997,"15min":0.70899999999999996},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":2.484}]},{"osd":7,"last update":"Fri Oct 6 02:31:09 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.755,"15min":0.751},"min":{"1min":0.44700000000000001,"5min":0.41499999999999998,"15min":0.36799999999999999},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.458},"last":2.7909999999999999},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.77700000000000002,"15min":0.76500000000000001},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.45900000000000002},"max":{"1min":1.0429999999999999,"5min":1.103,"15min":1.2470000000000001},"last":1.0269999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574963,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303456,"kb_used_data":2072,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068384,"statfs":{"total":96636764160,"available":96326025216,"internally_reserved":0,"allocated":2121728,"data_stored":990972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.443},"min":{"1min":0.30399999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.58799999999999997,"5min":0.63600000000000001,"15min":1.2230000000000001},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.45800000000000002,"15min":0.501},"min":{"1min":0.34499999999999997,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.66200000000000003,"5min":0.70099999999999996,"15min":1.788},"last":0.44400000000000001}]},{"osd":2,"last update":"Fri Oct 6 02:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.433,"15min":0.49099999999999999},"min":{"1min":0.33500000000000002,"5min":0.307,"15min":0.28299999999999997},"max":{"1min":0.57799999999999996,"5min":0.61799999999999999,"15min":2.0840000000000001},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.503,"5min":0.495,"15min":0.53500000000000003},"min":{"1min":0.374,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.66800000000000004,"5min":0.88100000000000001,"15min":1.6699999999999999},"last":0.46700000000000003}]},{"osd":3,"last update":"Fri Oct 6 02:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.57999999999999996,"15min":0.60399999999999998},"min":{"1min":0.51200000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.79700000000000004,"5min":0.89900000000000002,"15min":1.266},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.54000000000000004,"15min":0.60299999999999998},"min":{"1min":0.436,"5min":0.40999999999999998,"15min":0.34100000000000003},"max":{"1min":0.70399999999999996,"5min":0.79400000000000004,"15min":1.861},"last":0.48099999999999998}]},{"osd":4,"last update":"Fri Oct 6 02:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.752,"15min":0.82199999999999995},"min":{"1min":0.629,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.97999999999999998,"5min":1.3580000000000001,"15min":1.9279999999999999},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.747,"15min":0.82199999999999995},"min":{"1min":0.53900000000000003,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.95999999999999996,"5min":1.4339999999999999,"15min":2.0179999999999998},"last":0.81899999999999995}]},{"osd":5,"last update":"Fri Oct 6 02:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.71499999999999997,"15min":0.80900000000000005},"min":{"1min":0.57199999999999995,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.91700000000000004,"5min":1.3069999999999999,"15min":1.9830000000000001},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.73199999999999998,"15min":0.79100000000000004},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.47299999999999998},"max":{"1min":0.97299999999999998,"5min":1.327,"15min":1.3380000000000001},"last":0.63}]},{"osd":6,"last update":"Fri Oct 6 02:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.76100000000000001,"15min":0.83299999999999996},"min":{"1min":0.62,"5min":0.52100000000000002,"15min":0.45600000000000002},"max":{"1min":1.038,"5min":1.0609999999999999,"15min":1.829},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.77500000000000002,"15min":0.86499999999999999},"min":{"1min":0.60199999999999998,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.96399999999999997,"5min":1.351,"15min":2.052},"last":0.77300000000000002}]},{"osd":7,"last update":"Fri Oct 6 02:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.73399999999999999,"15min":0.80300000000000005},"min":{"1min":0.59399999999999997,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.97299999999999998,"5min":1.3120000000000001,"15min":1.3340000000000001},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.76200000000000001,"15min":0.83799999999999997},"min":{"1min":0.54600000000000004,"5min":0.51600000000000001,"15min":0.47799999999999998},"max":{"1min":1.0780000000000001,"5min":1.2110000000000001,"15min":1.895},"last":0.80300000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574964,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304040,"kb_used_data":2656,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067800,"statfs":{"total":96636764160,"available":96325427200,"internally_reserved":0,"allocated":2719744,"data_stored":1581430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 02:31:15 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.44600000000000001,"15min":0.45300000000000001},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.76400000000000001,"5min":2.6120000000000001,"15min":2.6120000000000001},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.46999999999999997,"15min":0.47999999999999998},"min":{"1min":0.30599999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.77900000000000003,"5min":3.4199999999999999,"15min":3.4199999999999999},"last":0.33000000000000002}]},{"osd":2,"last update":"Fri Oct 6 02:31:15 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.52000000000000002,"15min":0.51500000000000001},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.58199999999999996,"5min":3.0339999999999998,"15min":3.0339999999999998},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.70699999999999996,"5min":3.343,"15min":3.343},"last":0.377}]},{"osd":3,"last update":"Fri Oct 6 02:31:15 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.505,"15min":0.50600000000000001},"min":{"1min":0.34399999999999997,"5min":0.31,"15min":0.31},"max":{"1min":0.79400000000000004,"5min":3.6579999999999999,"15min":3.6579999999999999},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.59099999999999997,"15min":0.59399999999999997},"min":{"1min":0.443,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.82099999999999995,"5min":2.6890000000000001,"15min":2.6890000000000001},"last":0.505}]},{"osd":4,"last update":"Fri Oct 6 02:31:15 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.76400000000000001,"15min":0.79200000000000004},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.94199999999999995,"5min":3.2999999999999998,"15min":3.2999999999999998},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.755,"15min":0.78900000000000003},"min":{"1min":0.54400000000000004,"5min":0.51000000000000001,"15min":0.48499999999999999},"max":{"1min":1.085,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":0.72199999999999998}]},{"osd":5,"last update":"Fri Oct 6 02:31:15 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.74299999999999999,"15min":0.77600000000000002},"min":{"1min":0.51200000000000001,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.038,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.79000000000000004,"15min":0.79900000000000004},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.071,"5min":3.1110000000000002,"15min":3.1110000000000002},"last":0.66900000000000004}]},{"osd":6,"last update":"Fri Oct 6 02:31:15 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.79600000000000004,"15min":0.81599999999999995},"min":{"1min":0.52900000000000003,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.103,"5min":3.1600000000000001,"15min":3.1600000000000001},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.76900000000000002,"15min":0.81299999999999994},"min":{"1min":0.56200000000000006,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.01,"5min":2.3250000000000002,"15min":2.3250000000000002},"last":0.68500000000000005}]},{"osd":7,"last update":"Fri Oct 6 02:31:15 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.81399999999999995,"15min":0.82399999999999995},"min":{"1min":0.57999999999999996,"5min":0.56899999999999995,"15min":0.56000000000000005},"max":{"1min":1.048,"5min":3.3809999999999998,"15min":3.3809999999999998},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.753,"5min":0.77200000000000002,"15min":0.81200000000000006},"min":{"1min":0.56100000000000005,"5min":0.54700000000000004,"15min":0.53300000000000003},"max":{"1min":1.022,"5min":2.3740000000000001,"15min":2.3740000000000001},"last":0.79100000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574965,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303460,"kb_used_data":2076,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068380,"statfs":{"total":96636764160,"available":96326021120,"internally_reserved":0,"allocated":2125824,"data_stored":991062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:31:14 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.41999999999999998,"15min":0.435},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.63500000000000001,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.46500000000000002,"15min":0.47999999999999998},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.65300000000000002,"5min":0.88700000000000001,"15min":1.0800000000000001},"last":0.42799999999999999}]},{"osd":1,"last update":"Fri Oct 6 02:31:14 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47499999999999998,"15min":0.496},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.32400000000000001},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.51000000000000001,"15min":0.57899999999999996},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.622,"5min":1.778,"15min":1.778},"last":0.45900000000000002}]},{"osd":3,"last update":"Fri Oct 6 02:31:14 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.499,"15min":0.52500000000000002},"min":{"1min":0.33700000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.68400000000000005,"5min":0.83699999999999997,"15min":0.97499999999999998},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.57799999999999996,"15min":0.60899999999999999},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":1.028},"last":0.55000000000000004}]},{"osd":4,"last update":"Fri Oct 6 02:31:14 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.746,"15min":0.76900000000000002},"min":{"1min":0.496,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.1319999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.76200000000000001,"15min":0.82499999999999996},"min":{"1min":0.51200000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.95099999999999996,"5min":1.907,"15min":1.907},"last":0.70999999999999996}]},{"osd":5,"last update":"Fri Oct 6 02:31:14 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.73299999999999998,"15min":0.78200000000000003},"min":{"1min":0.47599999999999998,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.024,"5min":1.109,"15min":1.1850000000000001},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.745,"5min":0.752,"15min":0.80400000000000005},"min":{"1min":0.56200000000000006,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.258},"last":0.63400000000000001}]},{"osd":6,"last update":"Fri Oct 6 02:31:14 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.78500000000000003,"15min":0.83099999999999996},"min":{"1min":0.48399999999999999,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.1100000000000001,"5min":1.827,"15min":1.827},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.747,"5min":0.76700000000000002,"15min":0.78900000000000003},"min":{"1min":0.56100000000000005,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0269999999999999,"5min":1.0660000000000001,"15min":1.1659999999999999},"last":0.58799999999999997}]},{"osd":7,"last update":"Fri Oct 6 02:31:14 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.79400000000000004,"15min":0.86099999999999999},"min":{"1min":0.57099999999999995,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0649999999999999,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.751,"5min":0.77900000000000003,"15min":0.81799999999999995},"min":{"1min":0.56100000000000005,"5min":0.55300000000000005,"15min":0.54800000000000004},"max":{"1min":1.0269999999999999,"5min":1.125,"15min":1.1479999999999999},"last":0.72899999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574962,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303468,"kb_used_data":2084,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068372,"statfs":{"total":96636764160,"available":96326012928,"internally_reserved":0,"allocated":2134016,"data_stored":991098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:31:12 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.434,"15min":0.48799999999999999},"min":{"1min":0.29099999999999998,"5min":0.248,"15min":0.248},"max":{"1min":0.60799999999999998,"5min":1.0489999999999999,"15min":2.024},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.441,"5min":0.46800000000000003,"15min":0.50900000000000001},"min":{"1min":0.31900000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.53300000000000003,"5min":0.754,"15min":2.181},"last":0.39600000000000002}]},{"osd":1,"last update":"Fri Oct 6 02:31:12 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.45800000000000002,"15min":0.62},"min":{"1min":0.33400000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.61299999999999999,"5min":0.66100000000000003,"15min":4.492},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.51600000000000001,"15min":0.66800000000000004},"min":{"1min":0.377,"5min":0.313,"15min":0.313},"max":{"1min":0.67100000000000004,"5min":1.079,"15min":4.569},"last":0.438}]},{"osd":2,"last update":"Fri Oct 6 02:31:12 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.59699999999999998,"15min":0.64800000000000002},"min":{"1min":0.46899999999999997,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.79400000000000004,"5min":1.024,"15min":2.0870000000000002},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.56699999999999995,"15min":0.627},"min":{"1min":0.42799999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.72999999999999998,"5min":0.94199999999999995,"15min":2.4260000000000002},"last":0.49399999999999999}]},{"osd":4,"last update":"Fri Oct 6 02:31:12 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73299999999999998,"15min":0.80200000000000005},"min":{"1min":0.58199999999999996,"5min":0.55000000000000004,"15min":0.44800000000000001},"max":{"1min":0.97299999999999998,"5min":1.0149999999999999,"15min":2.214},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.75,"15min":0.79600000000000004},"min":{"1min":0.53000000000000003,"5min":0.50800000000000001,"15min":0.5},"max":{"1min":1.042,"5min":1.042,"15min":1.2869999999999999},"last":0.55200000000000005}]},{"osd":5,"last update":"Fri Oct 6 02:31:12 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.747,"15min":0.80300000000000005},"min":{"1min":0.55300000000000005,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.97499999999999998,"5min":1.077,"15min":1.3029999999999999},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.754,"5min":0.75600000000000001,"15min":0.82599999999999996},"min":{"1min":0.57999999999999996,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.0549999999999999,"5min":1.0580000000000001,"15min":2.242},"last":0.63900000000000001}]},{"osd":6,"last update":"Fri Oct 6 02:31:12 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.77300000000000002,"15min":0.80500000000000005},"min":{"1min":0.58199999999999996,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.96199999999999997,"5min":1.0489999999999999,"15min":1.357},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.78800000000000003,"15min":0.82099999999999995},"min":{"1min":0.64900000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0049999999999999,"5min":1.0529999999999999,"15min":1.3100000000000001},"last":0.61199999999999999}]},{"osd":7,"last update":"Fri Oct 6 02:31:12 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.76600000000000001,"15min":0.83399999999999996},"min":{"1min":0.54400000000000004,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.98399999999999999,"5min":1.077,"15min":2.27},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.78600000000000003,"15min":0.83899999999999997},"min":{"1min":0.57599999999999996,"5min":0.55600000000000005,"15min":0.53200000000000003},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.325},"last":0.64700000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574964,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303460,"kb_used_data":2076,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068380,"statfs":{"total":96636764160,"available":96326021120,"internally_reserved":0,"allocated":2125824,"data_stored":991526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:31:06 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.65200000000000002,"15min":0.92200000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.84799999999999998,"5min":1.0760000000000001,"15min":12.337},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.67800000000000005,"15min":0.85899999999999999},"min":{"1min":0.48499999999999999,"5min":0.44400000000000001,"15min":0.438},"max":{"1min":0.94299999999999995,"5min":1.3520000000000001,"15min":9.6539999999999999},"last":0.48899999999999999}]},{"osd":1,"last update":"Fri Oct 6 02:31:06 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.66300000000000003,"15min":0.68899999999999995},"min":{"1min":0.42099999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.97899999999999998,"5min":1.129,"15min":3.181},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.70199999999999996,"15min":0.94999999999999996},"min":{"1min":0.45300000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.92700000000000005,"5min":0.995,"15min":12.946999999999999},"last":0.59899999999999998}]},{"osd":2,"last update":"Fri Oct 6 02:31:06 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.69999999999999996,"15min":0.96799999999999997},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.89200000000000002,"5min":1.2989999999999999,"15min":12.377000000000001},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.72699999999999998,"15min":0.83899999999999997},"min":{"1min":0.51600000000000001,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.90800000000000003,"5min":1.0529999999999999,"15min":6.3220000000000001},"last":0.65300000000000002}]},{"osd":3,"last update":"Fri Oct 6 02:31:06 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72599999999999998,"15min":0.93100000000000005},"min":{"1min":0.54900000000000004,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.91100000000000003,"5min":1.5229999999999999,"15min":9.7720000000000002},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.70999999999999996,"15min":0.76000000000000001},"min":{"1min":0.53400000000000003,"5min":0.498,"15min":0.47999999999999998},"max":{"1min":1.0389999999999999,"5min":1.212,"15min":3.0840000000000001},"last":0.66700000000000004}]},{"osd":5,"last update":"Fri Oct 6 02:31:06 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.70599999999999996,"15min":0.81499999999999995},"min":{"1min":0.54000000000000004,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.96299999999999997,"5min":1.02,"15min":6.2690000000000001},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.751,"5min":0.73999999999999999,"15min":0.96799999999999997},"min":{"1min":0.53900000000000003,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.95199999999999996,"5min":1.4359999999999999,"15min":10.769},"last":0.68500000000000005}]},{"osd":6,"last update":"Fri Oct 6 02:31:06 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.70999999999999996,"15min":0.749},"min":{"1min":0.46100000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.1919999999999999,"5min":1.4590000000000001,"15min":3.1349999999999998},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79100000000000004,"15min":0.93100000000000005},"min":{"1min":0.66700000000000004,"5min":0.47799999999999998,"15min":0.45700000000000002},"max":{"1min":1.008,"5min":1.3839999999999999,"15min":8.0899999999999999},"last":0.76100000000000001}]},{"osd":7,"last update":"Fri Oct 6 02:31:06 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.79600000000000004,"15min":1.006},"min":{"1min":0.60999999999999999,"5min":0.501,"15min":0.47599999999999998},"max":{"1min":1.024,"5min":1.494,"15min":9.7089999999999996},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.78000000000000003,"15min":0.94899999999999995},"min":{"1min":0.55900000000000005,"5min":0.374,"15min":0.374},"max":{"1min":1.1399999999999999,"5min":1.2669999999999999,"15min":7.6719999999999997},"last":0.84299999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574964,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303464,"kb_used_data":2080,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068376,"statfs":{"total":96636764160,"available":96326017024,"internally_reserved":0,"allocated":2129920,"data_stored":991028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:32:07 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.63200000000000001,"15min":0.63},"min":{"1min":0.47099999999999997,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.93899999999999995,"5min":0.97599999999999998,"15min":1.4279999999999999},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.629,"15min":0.622},"min":{"1min":0.42599999999999999,"5min":0.40300000000000002,"15min":0.38700000000000001},"max":{"1min":0.89300000000000002,"5min":1.099,"15min":1.099},"last":0.752}]},{"osd":1,"last update":"Fri Oct 6 02:32:07 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.63600000000000001,"15min":0.64900000000000002},"min":{"1min":0.51700000000000002,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.93899999999999995,"5min":1.0660000000000001,"15min":1.089},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.67300000000000004,"15min":0.67700000000000005},"min":{"1min":0.42299999999999999,"5min":0.40699999999999997,"15min":0.40500000000000003},"max":{"1min":0.93899999999999995,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.90500000000000003}]},{"osd":2,"last update":"Fri Oct 6 02:32:07 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.67800000000000005,"15min":0.67600000000000005},"min":{"1min":0.45800000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.67200000000000004,"15min":0.68200000000000005},"min":{"1min":0.58599999999999997,"5min":0.46100000000000002,"15min":0.44600000000000001},"max":{"1min":1.028,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.624}]},{"osd":3,"last update":"Fri Oct 6 02:32:07 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.73299999999999998,"15min":0.73199999999999998},"min":{"1min":0.54000000000000004,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0009999999999999,"5min":1.159,"15min":1.159},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.77200000000000002,"15min":0.77000000000000002},"min":{"1min":0.64200000000000002,"5min":0.56200000000000006,"15min":0.54100000000000004},"max":{"1min":0.97699999999999998,"5min":1.194,"15min":1.194},"last":0.72599999999999998}]},{"osd":4,"last update":"Fri Oct 6 02:32:07 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.69099999999999995,"15min":0.67700000000000005},"min":{"1min":0.46999999999999997,"5min":0.40799999999999997,"15min":0.32000000000000001},"max":{"1min":0.92300000000000004,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.878},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.69099999999999995,"15min":0.68300000000000005},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.35399999999999998},"max":{"1min":0.88600000000000001,"5min":1.25,"15min":1.25},"last":0.64900000000000002}]},{"osd":6,"last update":"Fri Oct 6 02:32:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.71899999999999997,"15min":0.73499999999999999},"min":{"1min":0.45900000000000002,"5min":0.42899999999999999,"15min":0.40300000000000002},"max":{"1min":1.024,"5min":1.024,"15min":1.1719999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.81499999999999995,"15min":0.81799999999999995},"min":{"1min":0.60599999999999998,"5min":0.502,"15min":0.373},"max":{"1min":1.0700000000000001,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.92800000000000005}]},{"osd":7,"last update":"Fri Oct 6 02:32:07 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.754,"15min":0.76000000000000001},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.40799999999999997},"max":{"1min":0.95799999999999996,"5min":1.23,"15min":1.23},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.80200000000000005,"15min":0.82099999999999995},"min":{"1min":0.59599999999999997,"5min":0.47599999999999998,"15min":0.437},"max":{"1min":1.1180000000000001,"5min":1.1339999999999999,"15min":1.5640000000000001},"last":0.80900000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T02:32:12.986 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T02:32:12.986 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T02:32:13.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.216+0000 7f6b0979d700 1 Processor -- start 2023-10-06T02:32:13.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.216+0000 7f6b0979d700 1 -- start start 2023-10-06T02:32:13.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.217+0000 7f6b0979d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6b040accb0 0x7f6b040aaed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:13.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.217+0000 7f6b0979d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b040ad780 con 0x7f6b040cbea0 2023-10-06T02:32:13.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.217+0000 7f6b0979d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b040cee60 con 0x7f6b040cc310 2023-10-06T02:32:13.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.217+0000 7f6b0979d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b040cf140 con 0x7f6b040accb0 2023-10-06T02:32:13.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.218+0000 7f6b037fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f6b040cbea0 0x7f6b0406e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:47748/0 (socket says 172.21.15.31:47748) 2023-10-06T02:32:13.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.218+0000 7f6b037fe700 1 -- 172.21.15.31:0/1580273532 learned_addr learned my addr 172.21.15.31:0/1580273532 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:32:13.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.218+0000 7f6b03fff700 1 --2- 172.21.15.31:0/1580273532 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6b040accb0 0x7f6b040aaed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:13.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.218+0000 7f6b02ffd700 1 -- 172.21.15.31:0/1580273532 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 640000757 0 0) 0x7f6b040ad780 con 0x7f6b040cbea0 2023-10-06T02:32:13.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.218+0000 7f6b02ffd700 1 -- 172.21.15.31:0/1580273532 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6aec003570 con 0x7f6b040cbea0 2023-10-06T02:32:13.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.218+0000 7f6b02ffd700 1 -- 172.21.15.31:0/1580273532 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1481354549 0 0) 0x7f6aec003570 con 0x7f6b040cbea0 2023-10-06T02:32:13.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.218+0000 7f6b02ffd700 1 -- 172.21.15.31:0/1580273532 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6b040accb0 msgr2=0x7f6b040aaed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:13.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.218+0000 7f6b02ffd700 1 --2- 172.21.15.31:0/1580273532 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6b040accb0 0x7f6b040aaed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:13.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.218+0000 7f6b02ffd700 1 -- 172.21.15.31:0/1580273532 >> v1:172.21.15.62:6789/0 conn(0x7f6b040cc310 legacy=0x7f6b0406ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:13.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.218+0000 7f6b02ffd700 1 -- 172.21.15.31:0/1580273532 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b0406d850 con 0x7f6b040cbea0 2023-10-06T02:32:13.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.218+0000 7f6b02ffd700 1 -- 172.21.15.31:0/1580273532 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7f6af8001470 con 0x7f6b040cbea0 2023-10-06T02:32:13.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.218+0000 7f6b02ffd700 1 -- 172.21.15.31:0/1580273532 >> v1:172.21.15.31:6789/0 conn(0x7f6b040cbea0 legacy=0x7f6b0406e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:13.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.218+0000 7f6b02ffd700 1 --2- 172.21.15.31:0/1580273532 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6b040accb0 0x7f6aec003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:13.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.219+0000 7f6b02ffd700 1 --2- 172.21.15.31:0/1580273532 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6aec004060 0x7f6aec004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:13.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.219+0000 7f6b02ffd700 1 --2- 172.21.15.31:0/1580273532 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6aec004d50 0x7f6aec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:13.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.219+0000 7f6b02ffd700 1 -- 172.21.15.31:0/1580273532 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b040cf140 con 0x7f6aec004d50 2023-10-06T02:32:13.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.219+0000 7f6b02ffd700 1 -- 172.21.15.31:0/1580273532 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aec00bc20 con 0x7f6aec004060 2023-10-06T02:32:13.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.219+0000 7f6b02ffd700 1 -- 172.21.15.31:0/1580273532 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6aec00bde0 con 0x7f6b040accb0 2023-10-06T02:32:13.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.219+0000 7f6b03fff700 1 --2- 172.21.15.31:0/1580273532 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6b040accb0 0x7f6aec003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:13.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.219+0000 7f6b08f9c700 1 --2- 172.21.15.31:0/1580273532 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6aec004d50 0x7f6aec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:13.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.219+0000 7f6b037fe700 1 --2- 172.21.15.31:0/1580273532 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6aec004060 0x7f6aec004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:13.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.219+0000 7f6b08f9c700 1 -- 172.21.15.31:0/1580273532 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6b040accb0 msgr2=0x7f6aec003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:13.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.219+0000 7f6b08f9c700 1 --2- 172.21.15.31:0/1580273532 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6b040accb0 0x7f6aec003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:13.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.219+0000 7f6b08f9c700 1 -- 172.21.15.31:0/1580273532 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6aec004060 msgr2=0x7f6aec004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:13.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.219+0000 7f6b08f9c700 1 --2- 172.21.15.31:0/1580273532 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6aec004060 0x7f6aec004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:13.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.219+0000 7f6b08f9c700 1 -- 172.21.15.31:0/1580273532 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aec003570 con 0x7f6aec004d50 2023-10-06T02:32:13.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.219+0000 7f6b03fff700 1 --2- 172.21.15.31:0/1580273532 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6b040accb0 0x7f6aec003a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:32:13.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.220+0000 7f6b08f9c700 1 --2- 172.21.15.31:0/1580273532 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6aec004d50 0x7f6aec0051c0 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f6afc001fc0 tx=0x7f6afc001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2423e3284f9dc9ee server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:13.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.220+0000 7f6b02ffd700 1 -- 172.21.15.31:0/1580273532 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6afc001950 con 0x7f6aec004d50 2023-10-06T02:32:13.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.220+0000 7f6b02ffd700 1 -- 172.21.15.31:0/1580273532 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6afc00c8b0 con 0x7f6aec004d50 2023-10-06T02:32:13.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.220+0000 7f6b02ffd700 1 -- 172.21.15.31:0/1580273532 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6afc00cc50 con 0x7f6aec004d50 2023-10-06T02:32:13.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.220+0000 7f6b0979d700 1 -- 172.21.15.31:0/1580273532 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6aec004d50 msgr2=0x7f6aec0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:13.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.220+0000 7f6b0979d700 1 --2- 172.21.15.31:0/1580273532 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6aec004d50 0x7f6aec0051c0 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f6afc001fc0 tx=0x7f6afc001640 comp rx=0 tx=0).stop 2023-10-06T02:32:13.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.220+0000 7f6b08f9c700 1 -- 172.21.15.31:0/1580273532 reap_dead start 2023-10-06T02:32:13.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.220+0000 7f6b0979d700 1 -- 172.21.15.31:0/1580273532 shutdown_connections 2023-10-06T02:32:13.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.220+0000 7f6b0979d700 1 -- 172.21.15.31:0/1580273532 >> 172.21.15.31:0/1580273532 conn(0x7f6b0406ce80 msgr2=0x7f6b040ce370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:13.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.220+0000 7f6b0979d700 1 -- 172.21.15.31:0/1580273532 shutdown_connections 2023-10-06T02:32:13.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.220+0000 7f6b0979d700 1 -- 172.21.15.31:0/1580273532 wait complete. 2023-10-06T02:32:13.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.222+0000 7f6b0979d700 1 Processor -- start 2023-10-06T02:32:13.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.222+0000 7f6b0979d700 1 -- start start 2023-10-06T02:32:13.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.223+0000 7f6b0979d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6aec004d50 0x7f6b04154f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:13.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.223+0000 7f6b0979d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b04155810 0x7f6b040b3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:13.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.223+0000 7f6b0979d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b040ae2c0 0x7f6b04067e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:13.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.223+0000 7f6b0979d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b040b34f0 con 0x7f6b04155810 2023-10-06T02:32:13.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.223+0000 7f6b0979d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b040b2ce0 con 0x7f6b040ae2c0 2023-10-06T02:32:13.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.223+0000 7f6b0979d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b041554b0 con 0x7f6aec004d50 2023-10-06T02:32:13.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.223+0000 7f6b037fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b04155810 0x7f6b040b3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:13.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.223+0000 7f6b037fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b04155810 0x7f6b040b3c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41124/0 (socket says 172.21.15.31:41124) 2023-10-06T02:32:13.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.223+0000 7f6b037fe700 1 -- 172.21.15.31:0/1652512771 learned_addr learned my addr 172.21.15.31:0/1652512771 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:13.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.223+0000 7f6b03fff700 1 --2- 172.21.15.31:0/1652512771 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6aec004d50 0x7f6b04154f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:13.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.223+0000 7f6b08f9c700 1 --2- 172.21.15.31:0/1652512771 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b040ae2c0 0x7f6b04067e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:13.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.224+0000 7f6b037fe700 1 -- 172.21.15.31:0/1652512771 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6aec004d50 msgr2=0x7f6b04154f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:13.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.224+0000 7f6b037fe700 1 --2- 172.21.15.31:0/1652512771 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6aec004d50 0x7f6b04154f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:13.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.224+0000 7f6b037fe700 1 -- 172.21.15.31:0/1652512771 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b040ae2c0 msgr2=0x7f6b04067e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:13.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.224+0000 7f6b037fe700 1 --2- 172.21.15.31:0/1652512771 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b040ae2c0 0x7f6b04067e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:13.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.224+0000 7f6b037fe700 1 -- 172.21.15.31:0/1652512771 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b040b2f30 con 0x7f6b04155810 2023-10-06T02:32:13.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.224+0000 7f6b03fff700 1 --2- 172.21.15.31:0/1652512771 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6aec004d50 0x7f6b04154f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:32:13.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.224+0000 7f6b08f9c700 1 --2- 172.21.15.31:0/1652512771 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b040ae2c0 0x7f6b04067e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:32:13.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.224+0000 7f6b037fe700 1 --2- 172.21.15.31:0/1652512771 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b04155810 0x7f6b040b3c20 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f6af8007390 tx=0x7f6af8001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:13.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.224+0000 7f6aebfff700 1 -- 172.21.15.31:0/1652512771 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af800f8d0 con 0x7f6b04155810 2023-10-06T02:32:13.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.224+0000 7f6aebfff700 1 -- 172.21.15.31:0/1652512771 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6af800fa30 con 0x7f6b04155810 2023-10-06T02:32:13.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.224+0000 7f6b0979d700 1 -- 172.21.15.31:0/1652512771 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b040b3210 con 0x7f6b04155810 2023-10-06T02:32:13.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.224+0000 7f6aebfff700 1 -- 172.21.15.31:0/1652512771 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af800fdf0 con 0x7f6b04155810 2023-10-06T02:32:13.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.224+0000 7f6b0979d700 1 -- 172.21.15.31:0/1652512771 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b040adda0 con 0x7f6b04155810 2023-10-06T02:32:13.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.225+0000 7f6b0979d700 1 -- 172.21.15.31:0/1652512771 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b040adfc0 con 0x7f6b04155810 2023-10-06T02:32:13.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.226+0000 7f6aebfff700 1 -- 172.21.15.31:0/1652512771 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85556+0+0 (secure 0 0 0) 0x7f6af8013600 con 0x7f6b04155810 2023-10-06T02:32:13.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.227+0000 7f6aebfff700 1 --2- 172.21.15.31:0/1652512771 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6af0067830 0x7f6af0069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:13.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.227+0000 7f6aebfff700 1 -- 172.21.15.31:0/1652512771 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f6af80832d0 con 0x7f6b04155810 2023-10-06T02:32:13.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.228+0000 7f6b03fff700 1 --2- 172.21.15.31:0/1652512771 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6af0067830 0x7f6af0069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:13.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.228+0000 7f6b03fff700 1 --2- 172.21.15.31:0/1652512771 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6af0067830 0x7f6af0069ce0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f6af4006eb0 tx=0x7f6af4008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:13.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.231+0000 7f6aebfff700 1 -- 172.21.15.31:0/1652512771 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6af8051d20 con 0x7f6b04155810 2023-10-06T02:32:13.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.515+0000 7f6b0979d700 1 -- 172.21.15.31:0/1652512771 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f6b0404c3c0 con 0x7f6b04155810 2023-10-06T02:32:13.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.519+0000 7f6aebfff700 1 -- 172.21.15.31:0/1652512771 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v24) v1 ==== 190+0+0 (secure 0 0 0) 0x7f6af8055340 con 0x7f6b04155810 2023-10-06T02:32:13.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.521+0000 7f6b0979d700 1 -- 172.21.15.31:0/1652512771 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6af0067830 msgr2=0x7f6af0069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:13.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.521+0000 7f6b0979d700 1 --2- 172.21.15.31:0/1652512771 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6af0067830 0x7f6af0069ce0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f6af4006eb0 tx=0x7f6af4008040 comp rx=0 tx=0).stop 2023-10-06T02:32:13.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.521+0000 7f6b0979d700 1 -- 172.21.15.31:0/1652512771 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b04155810 msgr2=0x7f6b040b3c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:13.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.521+0000 7f6b0979d700 1 --2- 172.21.15.31:0/1652512771 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b04155810 0x7f6b040b3c20 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f6af8007390 tx=0x7f6af8001d80 comp rx=0 tx=0).stop 2023-10-06T02:32:13.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.521+0000 7f6b0979d700 1 -- 172.21.15.31:0/1652512771 shutdown_connections 2023-10-06T02:32:13.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.521+0000 7f6b0979d700 1 --2- 172.21.15.31:0/1652512771 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b040ae2c0 0x7f6b04067e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:13.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.521+0000 7f6b0979d700 1 --2- 172.21.15.31:0/1652512771 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6af0067830 0x7f6af0069ce0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:13.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.521+0000 7f6b0979d700 1 --2- 172.21.15.31:0/1652512771 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6aec004d50 0x7f6b04154f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:13.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.521+0000 7f6b0979d700 1 --2- 172.21.15.31:0/1652512771 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b04155810 0x7f6b040b3c20 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:13.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.521+0000 7f6b0979d700 1 -- 172.21.15.31:0/1652512771 >> 172.21.15.31:0/1652512771 conn(0x7f6b0406ce80 msgr2=0x7f6b040ac1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:13.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.522+0000 7f6b0979d700 1 -- 172.21.15.31:0/1652512771 shutdown_connections 2023-10-06T02:32:13.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.522+0000 7f6b0979d700 1 -- 172.21.15.31:0/1652512771 wait complete. 2023-10-06T02:32:13.523 INFO:teuthology.orchestra.run.smithi031.stderr:updated caps for client.0 2023-10-06T02:32:13.539 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:32:13.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.725+0000 7f53db39d700 1 Processor -- start 2023-10-06T02:32:13.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.725+0000 7f53db39d700 1 -- start start 2023-10-06T02:32:13.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.726+0000 7f53db39d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f53d40c7f90 0x7f53d406c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:13.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.726+0000 7f53db39d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53d40ca480 con 0x7f53d4142fd0 2023-10-06T02:32:13.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.726+0000 7f53db39d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53d415e240 con 0x7f53d4142c00 2023-10-06T02:32:13.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.726+0000 7f53db39d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53d406b350 con 0x7f53d40c7f90 2023-10-06T02:32:13.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.727+0000 7f53dab9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f53d40c7f90 0x7f53d406c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:13.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.727+0000 7f53d9b9a700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f53d4142fd0 0x7f53d40c7880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:47750/0 (socket says 172.21.15.31:47750) 2023-10-06T02:32:13.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.727+0000 7f53d9b9a700 1 -- 172.21.15.31:0/2263185887 learned_addr learned my addr 172.21.15.31:0/2263185887 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:32:13.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.727+0000 7f53d9399700 1 -- 172.21.15.31:0/2263185887 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1579019083 0 0) 0x7f53d40ca480 con 0x7f53d4142fd0 2023-10-06T02:32:13.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.727+0000 7f53dab9c700 1 -- 172.21.15.31:0/2263185887 >> v1:172.21.15.62:6789/0 conn(0x7f53d4142c00 legacy=0x7f53d406b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:13.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.727+0000 7f53dab9c700 1 -- 172.21.15.31:0/2263185887 >> v1:172.21.15.31:6789/0 conn(0x7f53d4142fd0 legacy=0x7f53d40c7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:13.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.727+0000 7f53dab9c700 1 -- 172.21.15.31:0/2263185887 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53d415df60 con 0x7f53d40c7f90 2023-10-06T02:32:13.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.727+0000 7f53dab9c700 1 --2- 172.21.15.31:0/2263185887 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53d40c7f90 0x7f53d406c6a0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f53cc009e60 tx=0x7f53cc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c94c6c644b915ca server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:13.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.728+0000 7f53d9399700 1 -- 172.21.15.31:0/2263185887 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53cc00e070 con 0x7f53d40c7f90 2023-10-06T02:32:13.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.728+0000 7f53d9399700 1 -- 172.21.15.31:0/2263185887 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53cc00bc30 con 0x7f53d40c7f90 2023-10-06T02:32:13.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.728+0000 7f53d9399700 1 -- 172.21.15.31:0/2263185887 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53cc012430 con 0x7f53d40c7f90 2023-10-06T02:32:13.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.728+0000 7f53db39d700 1 -- 172.21.15.31:0/2263185887 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53d40c7f90 msgr2=0x7f53d406c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:13.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.728+0000 7f53db39d700 1 --2- 172.21.15.31:0/2263185887 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53d40c7f90 0x7f53d406c6a0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f53cc009e60 tx=0x7f53cc009780 comp rx=0 tx=0).stop 2023-10-06T02:32:13.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.728+0000 7f53db39d700 1 -- 172.21.15.31:0/2263185887 shutdown_connections 2023-10-06T02:32:13.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.728+0000 7f53db39d700 1 --2- 172.21.15.31:0/2263185887 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53d40c7f90 0x7f53d406c6a0 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:13.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.728+0000 7f53db39d700 1 -- 172.21.15.31:0/2263185887 >> 172.21.15.31:0/2263185887 conn(0x7f53d40675a0 msgr2=0x7f53d40679a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:13.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.728+0000 7f53db39d700 1 -- 172.21.15.31:0/2263185887 shutdown_connections 2023-10-06T02:32:13.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.729+0000 7f53db39d700 1 -- 172.21.15.31:0/2263185887 wait complete. 2023-10-06T02:32:13.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.730+0000 7f53db39d700 1 Processor -- start 2023-10-06T02:32:13.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.730+0000 7f53db39d700 1 -- start start 2023-10-06T02:32:13.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.731+0000 7f53db39d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f53d40c7f90 0x7f53d4159210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:13.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.731+0000 7f53db39d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53d4142c00 0x7f53d413ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:13.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.731+0000 7f53db39d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53d4142fd0 0x7f53d4140490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:13.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.731+0000 7f53db39d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53d40b19d0 con 0x7f53d4142c00 2023-10-06T02:32:13.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.731+0000 7f53db39d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53d4169860 con 0x7f53d4142fd0 2023-10-06T02:32:13.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.731+0000 7f53db39d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53d4169b30 con 0x7f53d40c7f90 2023-10-06T02:32:13.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.731+0000 7f53d9b9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53d4142c00 0x7f53d413ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:13.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.732+0000 7f53d9b9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53d4142c00 0x7f53d413ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41144/0 (socket says 172.21.15.31:41144) 2023-10-06T02:32:13.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.732+0000 7f53dab9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53d4142fd0 0x7f53d4140490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:13.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.732+0000 7f53d9b9a700 1 -- 172.21.15.31:0/668867662 learned_addr learned my addr 172.21.15.31:0/668867662 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:13.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.732+0000 7f53da39b700 1 --2- 172.21.15.31:0/668867662 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f53d40c7f90 0x7f53d4159210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:13.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.732+0000 7f53d9b9a700 1 -- 172.21.15.31:0/668867662 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f53d40c7f90 msgr2=0x7f53d4159210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:13.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.732+0000 7f53d9b9a700 1 --2- 172.21.15.31:0/668867662 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f53d40c7f90 0x7f53d4159210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:13.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.732+0000 7f53d9b9a700 1 -- 172.21.15.31:0/668867662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53d4142fd0 msgr2=0x7f53d4140490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:13.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.732+0000 7f53d9b9a700 1 --2- 172.21.15.31:0/668867662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53d4142fd0 0x7f53d4140490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:13.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.732+0000 7f53d9b9a700 1 -- 172.21.15.31:0/668867662 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53cc009a20 con 0x7f53d4142c00 2023-10-06T02:32:13.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.732+0000 7f53d9b9a700 1 --2- 172.21.15.31:0/668867662 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53d4142c00 0x7f53d413ff50 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f53c400c5e0 tx=0x7f53c4001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:13.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.732+0000 7f53ca7fc700 1 -- 172.21.15.31:0/668867662 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c4005b30 con 0x7f53d4142c00 2023-10-06T02:32:13.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.733+0000 7f53ca7fc700 1 -- 172.21.15.31:0/668867662 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53c4005c90 con 0x7f53d4142c00 2023-10-06T02:32:13.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.733+0000 7f53db39d700 1 -- 172.21.15.31:0/668867662 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53d4159750 con 0x7f53d4142c00 2023-10-06T02:32:13.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.733+0000 7f53ca7fc700 1 -- 172.21.15.31:0/668867662 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c4007300 con 0x7f53d4142c00 2023-10-06T02:32:13.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.733+0000 7f53db39d700 1 -- 172.21.15.31:0/668867662 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53d413d110 con 0x7f53d4142c00 2023-10-06T02:32:13.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.734+0000 7f53db39d700 1 -- 172.21.15.31:0/668867662 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53d4169cc0 con 0x7f53d4142c00 2023-10-06T02:32:13.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.735+0000 7f53ca7fc700 1 -- 172.21.15.31:0/668867662 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85556+0+0 (secure 0 0 0) 0x7f53c40076b0 con 0x7f53d4142c00 2023-10-06T02:32:13.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.735+0000 7f53ca7fc700 1 --2- 172.21.15.31:0/668867662 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f53b8067830 0x7f53b8069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:13.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.735+0000 7f53ca7fc700 1 -- 172.21.15.31:0/668867662 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f53c4082220 con 0x7f53d4142c00 2023-10-06T02:32:13.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.736+0000 7f53da39b700 1 --2- 172.21.15.31:0/668867662 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f53b8067830 0x7f53b8069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:13.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.736+0000 7f53da39b700 1 --2- 172.21.15.31:0/668867662 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f53b8067830 0x7f53b8069ce0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f53d0003ac0 tx=0x7f53d0007ea0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:13.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.740+0000 7f53ca7fc700 1 -- 172.21.15.31:0/668867662 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f53c4050e60 con 0x7f53d4142c00 2023-10-06T02:32:13.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.986+0000 7f53db39d700 1 -- 172.21.15.31:0/668867662 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f53d404c3c0 con 0x7f53d4142c00 2023-10-06T02:32:13.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.987+0000 7f53ca7fc700 1 -- 172.21.15.31:0/668867662 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 119 v119) v1 ==== 95+0+4601 (secure 0 0 0) 0x7f53c4017090 con 0x7f53d4142c00 2023-10-06T02:32:13.988 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:32:13.988 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":119,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7445,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/324072138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":324072138},{"type":"v1","addr":"172.21.15.31:6835","nonce":324072138}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":114},{"gid":7448,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/3160488596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3160488596},{"type":"v1","addr":"172.21.15.62:6834","nonce":3160488596}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":114},{"gid":8677,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/632134516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":632134516},{"type":"v1","addr":"172.21.15.62:6835","nonce":632134516}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":114}],"filesystems":[{"mdsmap":{"epoch":119,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:32:07.613387+0000","modified":"2023-10-06T02:32:11.143622+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9267},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9267":{"gid":9267,"name":"a","rank":0,"incarnation":115,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6834/3741373122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":3741373122},{"type":"v1","addr":"172.21.15.31:6834","nonce":3741373122}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2023-10-06T02:32:13.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.989+0000 7f53db39d700 1 -- 172.21.15.31:0/668867662 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f53b8067830 msgr2=0x7f53b8069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:13.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.989+0000 7f53db39d700 1 --2- 172.21.15.31:0/668867662 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f53b8067830 0x7f53b8069ce0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f53d0003ac0 tx=0x7f53d0007ea0 comp rx=0 tx=0).stop 2023-10-06T02:32:13.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.990+0000 7f53db39d700 1 -- 172.21.15.31:0/668867662 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53d4142c00 msgr2=0x7f53d413ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:13.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.990+0000 7f53db39d700 1 --2- 172.21.15.31:0/668867662 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53d4142c00 0x7f53d413ff50 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f53c400c5e0 tx=0x7f53c4001070 comp rx=0 tx=0).stop 2023-10-06T02:32:13.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.990+0000 7f53db39d700 1 -- 172.21.15.31:0/668867662 shutdown_connections 2023-10-06T02:32:13.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.990+0000 7f53db39d700 1 --2- 172.21.15.31:0/668867662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53d4142fd0 0x7f53d4140490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:13.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.990+0000 7f53db39d700 1 --2- 172.21.15.31:0/668867662 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f53b8067830 0x7f53b8069ce0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:13.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.990+0000 7f53db39d700 1 --2- 172.21.15.31:0/668867662 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f53d40c7f90 0x7f53d4159210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:13.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.990+0000 7f53db39d700 1 --2- 172.21.15.31:0/668867662 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53d4142c00 0x7f53d413ff50 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:13.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.990+0000 7f53db39d700 1 -- 172.21.15.31:0/668867662 >> 172.21.15.31:0/668867662 conn(0x7f53d40675a0 msgr2=0x7f53d40c2540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:13.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.990+0000 7f53db39d700 1 -- 172.21.15.31:0/668867662 shutdown_connections 2023-10-06T02:32:13.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:13.990+0000 7f53db39d700 1 -- 172.21.15.31:0/668867662 wait complete. 2023-10-06T02:32:13.992 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 119 2023-10-06T02:32:14.411 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 119, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T02:32:07.613387+0000', 'modified': '2023-10-06T02:32:11.143622+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9267}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9267': {'gid': 9267, 'name': 'a', 'rank': 0, 'incarnation': 115, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.31:6834/3741373122', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6832', 'nonce': 3741373122}, {'type': 'v1', 'addr': '172.21.15.31:6834', 'nonce': 3741373122}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [28], 'metadata_pool': 27, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T02:32:14.411 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T02:32:14.411 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-06T02:32:14.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.601+0000 7fe5dbfff700 1 Processor -- start 2023-10-06T02:32:14.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.602+0000 7fe5dbfff700 1 -- start start 2023-10-06T02:32:14.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.603+0000 7fe5dbfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe5dc0cd9d0 0x7fe5dc0a98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:14.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.603+0000 7fe5dbfff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe5dc0b2d10 0x7fe5dc159250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:14.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.603+0000 7fe5dbfff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5dc0b0110 con 0x7fe5dc0b2940 2023-10-06T02:32:14.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.603+0000 7fe5dbfff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5dc06e730 con 0x7fe5dc0cd9d0 2023-10-06T02:32:14.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.603+0000 7fe5dbfff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5dc0b0920 con 0x7fe5dc0b2d10 2023-10-06T02:32:14.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.603+0000 7fe5da7fc700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe5dc0cd9d0 0x7fe5dc0a98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:14.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.603+0000 7fe5da7fc700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe5dc0cd9d0 0x7fe5dc0a98e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41160/0 (socket says 172.21.15.31:41160) 2023-10-06T02:32:14.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.603+0000 7fe5da7fc700 1 -- 172.21.15.31:0/1613062923 learned_addr learned my addr 172.21.15.31:0/1613062923 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:14.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.603+0000 7fe5db7fe700 1 --2- 172.21.15.31:0/1613062923 >> v2:172.21.15.62:3300/0 conn(0x7fe5dc0b2d10 0x7fe5dc159250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:14.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.604+0000 7fe5da7fc700 1 -- 172.21.15.31:0/1613062923 >> v2:172.21.15.62:3300/0 conn(0x7fe5dc0b2d10 msgr2=0x7fe5dc159250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:14.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.604+0000 7fe5da7fc700 1 --2- 172.21.15.31:0/1613062923 >> v2:172.21.15.62:3300/0 conn(0x7fe5dc0b2d10 0x7fe5dc159250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:14.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.604+0000 7fe5da7fc700 1 -- 172.21.15.31:0/1613062923 >> v1:172.21.15.62:6789/0 conn(0x7fe5dc0b2940 legacy=0x7fe5dc06e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:14.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.604+0000 7fe5da7fc700 1 -- 172.21.15.31:0/1613062923 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5dc06cc70 con 0x7fe5dc0cd9d0 2023-10-06T02:32:14.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.604+0000 7fe5da7fc700 1 --2- 172.21.15.31:0/1613062923 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5dc0cd9d0 0x7fe5dc0a98e0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc0a9e80 tx=0x7fe5d0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be8730f241230e39 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:14.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.604+0000 7fe5d9ffb700 1 -- 172.21.15.31:0/1613062923 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d0009e70 con 0x7fe5dc0cd9d0 2023-10-06T02:32:14.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.604+0000 7fe5d9ffb700 1 -- 172.21.15.31:0/1613062923 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5d000b860 con 0x7fe5dc0cd9d0 2023-10-06T02:32:14.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.604+0000 7fe5d9ffb700 1 -- 172.21.15.31:0/1613062923 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d000bc00 con 0x7fe5dc0cd9d0 2023-10-06T02:32:14.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.605+0000 7fe5dbfff700 1 -- 172.21.15.31:0/1613062923 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5dc0cd9d0 msgr2=0x7fe5dc0a98e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:14.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.605+0000 7fe5dbfff700 1 --2- 172.21.15.31:0/1613062923 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5dc0cd9d0 0x7fe5dc0a98e0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc0a9e80 tx=0x7fe5d0009250 comp rx=0 tx=0).stop 2023-10-06T02:32:14.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.605+0000 7fe5dbfff700 1 -- 172.21.15.31:0/1613062923 shutdown_connections 2023-10-06T02:32:14.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.605+0000 7fe5dbfff700 1 --2- 172.21.15.31:0/1613062923 >> v2:172.21.15.62:3300/0 conn(0x7fe5dc0b2d10 0x7fe5dc159250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:14.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.605+0000 7fe5dbfff700 1 --2- 172.21.15.31:0/1613062923 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5dc0cd9d0 0x7fe5dc0a98e0 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:14.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.605+0000 7fe5dbfff700 1 -- 172.21.15.31:0/1613062923 >> 172.21.15.31:0/1613062923 conn(0x7fe5dc0675a0 msgr2=0x7fe5dc0679a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:14.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.605+0000 7fe5dbfff700 1 -- 172.21.15.31:0/1613062923 shutdown_connections 2023-10-06T02:32:14.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.605+0000 7fe5dbfff700 1 -- 172.21.15.31:0/1613062923 wait complete. 2023-10-06T02:32:14.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.606+0000 7fe5dbfff700 1 Processor -- start 2023-10-06T02:32:14.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.607+0000 7fe5dbfff700 1 -- start start 2023-10-06T02:32:14.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.608+0000 7fe5dbfff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe5dc0b2940 0x7fe5dc0af7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:14.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.608+0000 7fe5dbfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5dc0b2d10 0x7fe5dc146d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:14.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.608+0000 7fe5dbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5dc0cd9d0 0x7fe5dc15b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:14.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.608+0000 7fe5dbfff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5dc15aa30 con 0x7fe5dc0cd9d0 2023-10-06T02:32:14.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.608+0000 7fe5dbfff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5dc1472d0 con 0x7fe5dc0b2d10 2023-10-06T02:32:14.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.608+0000 7fe5dbfff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5dc0ad150 con 0x7fe5dc0b2940 2023-10-06T02:32:14.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.608+0000 7fe5db7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5dc0cd9d0 0x7fe5dc15b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:14.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.608+0000 7fe5db7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5dc0cd9d0 0x7fe5dc15b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41164/0 (socket says 172.21.15.31:41164) 2023-10-06T02:32:14.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.608+0000 7fe5db7fe700 1 -- 172.21.15.31:0/2562322011 learned_addr learned my addr 172.21.15.31:0/2562322011 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:14.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.608+0000 7fe5db7fe700 1 -- 172.21.15.31:0/2562322011 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe5dc0b2940 msgr2=0x7fe5dc0af7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:14.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.608+0000 7fe5da7fc700 1 --2- 172.21.15.31:0/2562322011 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5dc0b2d10 0x7fe5dc146d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:14.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.608+0000 7fe5db7fe700 1 --2- 172.21.15.31:0/2562322011 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe5dc0b2940 0x7fe5dc0af7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:14.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.608+0000 7fe5db7fe700 1 -- 172.21.15.31:0/2562322011 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5dc0b2d10 msgr2=0x7fe5dc146d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:14.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.608+0000 7fe5db7fe700 1 --2- 172.21.15.31:0/2562322011 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5dc0b2d10 0x7fe5dc146d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:14.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.608+0000 7fe5db7fe700 1 -- 172.21.15.31:0/2562322011 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5d00094f0 con 0x7fe5dc0cd9d0 2023-10-06T02:32:14.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.609+0000 7fe5db7fe700 1 --2- 172.21.15.31:0/2562322011 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5dc0cd9d0 0x7fe5dc15b1c0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fe5d4009ec0 tx=0x7fe5d400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:14.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.609+0000 7fe5c2ffd700 1 -- 172.21.15.31:0/2562322011 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d400cc20 con 0x7fe5dc0cd9d0 2023-10-06T02:32:14.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.609+0000 7fe5c2ffd700 1 -- 172.21.15.31:0/2562322011 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5d400f040 con 0x7fe5dc0cd9d0 2023-10-06T02:32:14.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.609+0000 7fe5dbfff700 1 -- 172.21.15.31:0/2562322011 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5dc15a470 con 0x7fe5dc0cd9d0 2023-10-06T02:32:14.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.609+0000 7fe5c2ffd700 1 -- 172.21.15.31:0/2562322011 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d4014480 con 0x7fe5dc0cd9d0 2023-10-06T02:32:14.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.609+0000 7fe5dbfff700 1 -- 172.21.15.31:0/2562322011 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5dc0acde0 con 0x7fe5dc0cd9d0 2023-10-06T02:32:14.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.611+0000 7fe5c2ffd700 1 -- 172.21.15.31:0/2562322011 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85556+0+0 (secure 0 0 0) 0x7fe5d4014830 con 0x7fe5dc0cd9d0 2023-10-06T02:32:14.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.612+0000 7fe5c2ffd700 1 --2- 172.21.15.31:0/2562322011 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe5c8067830 0x7fe5c8069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:14.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.612+0000 7fe5c2ffd700 1 -- 172.21.15.31:0/2562322011 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fe5d40838b0 con 0x7fe5dc0cd9d0 2023-10-06T02:32:14.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.613+0000 7fe5daffd700 1 --2- 172.21.15.31:0/2562322011 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe5c8067830 0x7fe5c8069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:14.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.613+0000 7fe5daffd700 1 --2- 172.21.15.31:0/2562322011 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe5c8067830 0x7fe5c8069ce0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc15cd20 tx=0x7fe5cc007aa0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:14.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.614+0000 7fe5efe8cb80 1 Processor -- start 2023-10-06T02:32:14.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.614+0000 7fe5efe8cb80 1 -- start start 2023-10-06T02:32:14.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.615+0000 7fe5efe8cb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5574742f9e50 0x5574742d4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:14.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.615+0000 7fe5efe8cb80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5574742da2c0 0x5574742d5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:14.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.615+0000 7fe5efe8cb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574742dba10 0x5574742d5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:14.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.615+0000 7fe5efe8cb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55747422fa70 con 0x5574742dba10 2023-10-06T02:32:14.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.615+0000 7fe5efe8cb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55747422d270 con 0x5574742f9e50 2023-10-06T02:32:14.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.615+0000 7fe5efe8cb80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x557474229060 con 0x5574742da2c0 2023-10-06T02:32:14.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.615+0000 7fe5db7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574742dba10 0x5574742d5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:14.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.615+0000 7fe5db7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574742dba10 0x5574742d5740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41180/0 (socket says 172.21.15.31:41180) 2023-10-06T02:32:14.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.615+0000 7fe5db7fe700 1 -- 172.21.15.31:0/1503496399 learned_addr learned my addr 172.21.15.31:0/1503496399 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:14.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.615+0000 7fe5da7fc700 1 --2- 172.21.15.31:0/1503496399 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5574742da2c0 0x5574742d5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:14.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.615+0000 7fe5db7fe700 1 -- 172.21.15.31:0/1503496399 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5574742da2c0 msgr2=0x5574742d5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:14.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.615+0000 7fe5db7fe700 1 --2- 172.21.15.31:0/1503496399 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5574742da2c0 0x5574742d5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:14.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.616+0000 7fe5db7fe700 1 -- 172.21.15.31:0/1503496399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5574742f9e50 msgr2=0x5574742d4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:14.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.616+0000 7fe5db7fe700 1 --2- 172.21.15.31:0/1503496399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5574742f9e50 0x5574742d4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:14.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.616+0000 7fe5db7fe700 1 -- 172.21.15.31:0/1503496399 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5574740c3270 con 0x5574742dba10 2023-10-06T02:32:14.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.616+0000 7fe5da7fc700 1 --2- 172.21.15.31:0/1503496399 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5574742da2c0 0x5574742d5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:32:14.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.616+0000 7fe5db7fe700 1 --2- 172.21.15.31:0/1503496399 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574742dba10 0x5574742d5740 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7fe5d400c730 tx=0x7fe5d40093b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=add898a096b8ff68 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:14.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.616+0000 7fe5bbfff700 1 -- 172.21.15.31:0/1503496399 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d405d150 con 0x5574742dba10 2023-10-06T02:32:14.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.616+0000 7fe5bbfff700 1 -- 172.21.15.31:0/1503496399 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5d405b050 con 0x5574742dba10 2023-10-06T02:32:14.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.616+0000 7fe5bbfff700 1 -- 172.21.15.31:0/1503496399 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d4056410 con 0x5574742dba10 2023-10-06T02:32:14.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.617+0000 7fe5efe8cb80 1 -- 172.21.15.31:0/1503496399 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574742dba10 msgr2=0x5574742d5740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:14.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.617+0000 7fe5efe8cb80 1 --2- 172.21.15.31:0/1503496399 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574742dba10 0x5574742d5740 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7fe5d400c730 tx=0x7fe5d40093b0 comp rx=0 tx=0).stop 2023-10-06T02:32:14.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.617+0000 7fe5efe8cb80 1 -- 172.21.15.31:0/1503496399 shutdown_connections 2023-10-06T02:32:14.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.617+0000 7fe5efe8cb80 1 --2- 172.21.15.31:0/1503496399 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574742dba10 0x5574742d5740 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:14.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.617+0000 7fe5efe8cb80 1 --2- 172.21.15.31:0/1503496399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5574742f9e50 0x5574742d4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:14.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.617+0000 7fe5efe8cb80 1 --2- 172.21.15.31:0/1503496399 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5574742da2c0 0x5574742d5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:14.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.617+0000 7fe5efe8cb80 1 -- 172.21.15.31:0/1503496399 >> 172.21.15.31:0/1503496399 conn(0x5574742d9e80 msgr2=0x5574742da6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:14.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.617+0000 7fe5efe8cb80 1 -- 172.21.15.31:0/1503496399 shutdown_connections 2023-10-06T02:32:14.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.617+0000 7fe5efe8cb80 1 -- 172.21.15.31:0/1503496399 wait complete. 2023-10-06T02:32:14.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.617+0000 7fe5efe8cb80 1 -- start start 2023-10-06T02:32:14.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.618+0000 7fe5efe8cb80 1 Processor -- start 2023-10-06T02:32:14.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.619+0000 7fe5efe8cb80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5574742da2c0 0x5574742fa820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:14.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.619+0000 7fe5efe8cb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574742dba10 0x5574742d5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:14.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.619+0000 7fe5efe8cb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5574742f9e50 0x5574742d5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:14.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.619+0000 7fe5efe8cb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574741d4890 con 0x5574742dba10 2023-10-06T02:32:14.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.619+0000 7fe5efe8cb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574741efea0 con 0x5574742f9e50 2023-10-06T02:32:14.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.619+0000 7fe5efe8cb80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x5574740abe30 con 0x5574742da2c0 2023-10-06T02:32:14.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.619+0000 7fe5efe8cb80 10 client.0 _ll_register_callbacks cb 0x55747429c020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T02:32:14.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.619+0000 7fe5da7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574742dba10 0x5574742d5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:14.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.619+0000 7fe5da7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574742dba10 0x5574742d5010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41192/0 (socket says 172.21.15.31:41192) 2023-10-06T02:32:14.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.619+0000 7fe5da7fc700 1 -- 172.21.15.31:0/3069643224 learned_addr learned my addr 172.21.15.31:0/3069643224 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:14.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.619+0000 7fe5daffd700 1 --2- 172.21.15.31:0/3069643224 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5574742da2c0 0x5574742fa820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:14.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.619+0000 7fe5db7fe700 1 --2- 172.21.15.31:0/3069643224 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5574742f9e50 0x5574742d5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:14.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.619+0000 7fe5da7fc700 1 -- 172.21.15.31:0/3069643224 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5574742da2c0 msgr2=0x5574742fa820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:14.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.619+0000 7fe5da7fc700 1 --2- 172.21.15.31:0/3069643224 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5574742da2c0 0x5574742fa820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:14.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.619+0000 7fe5da7fc700 1 -- 172.21.15.31:0/3069643224 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5574742f9e50 msgr2=0x5574742d5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:14.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.619+0000 7fe5da7fc700 1 --2- 172.21.15.31:0/3069643224 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5574742f9e50 0x5574742d5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:14.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.619+0000 7fe5da7fc700 1 -- 172.21.15.31:0/3069643224 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5dc0acde0 con 0x5574742dba10 2023-10-06T02:32:14.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.619+0000 7fe5db7fe700 1 --2- 172.21.15.31:0/3069643224 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5574742f9e50 0x5574742d5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:32:14.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.620+0000 7fe5da7fc700 1 --2- 172.21.15.31:0/3069643224 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574742dba10 0x5574742d5010 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc06ee30 tx=0x7fe5d0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=becef0428cc68d48 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:14.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.620+0000 7fe5b97fa700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T02:32:14.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.620+0000 7fe5b97fa700 1 -- 172.21.15.31:0/3069643224 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d0020070 con 0x5574742dba10 2023-10-06T02:32:14.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.620+0000 7fe5b97fa700 1 -- 172.21.15.31:0/3069643224 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(169..169 src has 1..169) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fe5d0025470 con 0x5574742dba10 2023-10-06T02:32:14.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.620+0000 7fe5efe8cb80 1 -- 172.21.15.31:0/3069643224 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5574742cd940 con 0x5574742dba10 2023-10-06T02:32:14.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.620+0000 7fe5b97fa700 1 -- 172.21.15.31:0/3069643224 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=119) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe5d0025840 con 0x5574742dba10 2023-10-06T02:32:14.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.620+0000 7fe5efe8cb80 10 client.9423 fetch_fsmap learned FSMap version 119 2023-10-06T02:32:14.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.620+0000 7fe5efe8cb80 1 -- 172.21.15.31:0/3069643224 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=119,monmap=2+}) v3 -- 0x557474065fe0 con 0x5574742dba10 2023-10-06T02:32:14.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.620+0000 7fe5b97fa700 1 -- 172.21.15.31:0/3069643224 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5574742cd940 con 0x5574742dba10 2023-10-06T02:32:14.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.621+0000 7fe5b97fa700 1 -- 172.21.15.31:0/3069643224 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 119) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fe5d00259a0 con 0x5574742dba10 2023-10-06T02:32:14.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.621+0000 7fe5efe8cb80 10 client.9423 fetch_fsmap finished waiting for FSMap version 119 2023-10-06T02:32:14.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.621+0000 7fe5efe8cb80 10 client.9423 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T02:32:14.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.621+0000 7fe5efe8cb80 20 client.9423 populate_metadata read hostname 'smithi031' 2023-10-06T02:32:14.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.621+0000 7fe5efe8cb80 1 --2- 172.21.15.31:0/3069643224 >> [v2:172.21.15.31:6832/3741373122,v1:172.21.15.31:6834/3741373122] conn(0x55747431ff30 0x557474320300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:14.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.621+0000 7fe5efe8cb80 4 client.9423 mds_command: new command op to 9267 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T02:32:14.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.621+0000 7fe5efe8cb80 1 -- 172.21.15.31:0/3069643224 --> [v2:172.21.15.31:6832/3741373122,v1:172.21.15.31:6834/3741373122] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5574742aa870 con 0x55747431ff30 2023-10-06T02:32:14.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.621+0000 7fe5daffd700 1 --2- 172.21.15.31:0/3069643224 >> [v2:172.21.15.31:6832/3741373122,v1:172.21.15.31:6834/3741373122] conn(0x55747431ff30 0x557474320300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:14.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.621+0000 7fe5daffd700 1 --2- 172.21.15.31:0/3069643224 >> [v2:172.21.15.31:6832/3741373122,v1:172.21.15.31:6834/3741373122] conn(0x55747431ff30 0x557474320300 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7eb48b2003636176 server_cookie=f5feb3dbf1ad600f in_seq=0 out_seq=0 2023-10-06T02:32:14.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.621+0000 7fe5b97fa700 10 client.9423 ms_handle_connect on v2:172.21.15.31:6832/3741373122 2023-10-06T02:32:14.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.622+0000 7fe5b97fa700 1 -- 172.21.15.31:0/3069643224 <== mds.0 v2:172.21.15.31:6832/3741373122 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x5574742aa870 con 0x55747431ff30 2023-10-06T02:32:14.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.622+0000 7fe5b97fa700 10 client.9423 handle_command_reply: tid=0 2023-10-06T02:32:14.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.622+0000 7fe5efe8cb80 1 client.9423 shutdown 2023-10-06T02:32:14.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.622+0000 7fe5efe8cb80 1 -- 172.21.15.31:0/3069643224 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574742dba10 msgr2=0x5574742d5010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:14.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.622+0000 7fe5efe8cb80 1 --2- 172.21.15.31:0/3069643224 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574742dba10 0x5574742d5010 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc06ee30 tx=0x7fe5d0017000 comp rx=0 tx=0).stop 2023-10-06T02:32:14.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.622+0000 7fe5efe8cb80 1 -- 172.21.15.31:0/3069643224 shutdown_connections 2023-10-06T02:32:14.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.622+0000 7fe5efe8cb80 1 --2- 172.21.15.31:0/3069643224 >> [v2:172.21.15.31:6832/3741373122,v1:172.21.15.31:6834/3741373122] conn(0x55747431ff30 0x557474320300 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:14.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.623+0000 7fe5efe8cb80 1 --2- 172.21.15.31:0/3069643224 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5574742f9e50 0x5574742d5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:14.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.623+0000 7fe5efe8cb80 1 --2- 172.21.15.31:0/3069643224 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5574742da2c0 0x5574742fa820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:14.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.623+0000 7fe5b97fa700 0 client.9423 ms_handle_reset on v2:172.21.15.31:6832/3741373122 2023-10-06T02:32:14.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.623+0000 7fe5efe8cb80 1 --2- 172.21.15.31:0/3069643224 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574742dba10 0x5574742d5010 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:14.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.623+0000 7fe5efe8cb80 1 -- 172.21.15.31:0/3069643224 >> 172.21.15.31:0/3069643224 conn(0x5574742d9e80 msgr2=0x5574742dad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:14.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.623+0000 7fe5efe8cb80 1 -- 172.21.15.31:0/3069643224 shutdown_connections 2023-10-06T02:32:14.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.623+0000 7fe5efe8cb80 1 -- 172.21.15.31:0/3069643224 wait complete. 2023-10-06T02:32:14.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.623+0000 7fe5efe8cb80 20 client.9423 trim_cache size 0 max 16384 2023-10-06T02:32:14.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.640+0000 7fe5efe8cb80 1 Processor -- start 2023-10-06T02:32:14.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.640+0000 7fe5efe8cb80 1 -- start start 2023-10-06T02:32:14.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.641+0000 7fe5efe8cb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574742da2c0 0x557474320300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:14.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.641+0000 7fe5efe8cb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5574742dba10 0x5574742fa700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:14.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.641+0000 7fe5efe8cb80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5574742f9e50 0x5574742d4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:14.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.641+0000 7fe5efe8cb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574742f9700 con 0x5574742da2c0 2023-10-06T02:32:14.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.641+0000 7fe5efe8cb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557474341ab0 con 0x5574742dba10 2023-10-06T02:32:14.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.641+0000 7fe5efe8cb80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x5574742b82e0 con 0x5574742f9e50 2023-10-06T02:32:14.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.642+0000 7fe5daffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574742da2c0 0x557474320300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:14.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.642+0000 7fe5daffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574742da2c0 0x557474320300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41198/0 (socket says 172.21.15.31:41198) 2023-10-06T02:32:14.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.642+0000 7fe5daffd700 1 -- 172.21.15.31:0/4257734255 learned_addr learned my addr 172.21.15.31:0/4257734255 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:14.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.642+0000 7fe5daffd700 1 -- 172.21.15.31:0/4257734255 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5574742f9e50 msgr2=0x5574742d4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:14.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.642+0000 7fe5daffd700 1 --2- 172.21.15.31:0/4257734255 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5574742f9e50 0x5574742d4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:14.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.642+0000 7fe5da7fc700 1 --2- 172.21.15.31:0/4257734255 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5574742dba10 0x5574742fa700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:14.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.642+0000 7fe5daffd700 1 -- 172.21.15.31:0/4257734255 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5574742dba10 msgr2=0x5574742fa700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:14.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.642+0000 7fe5daffd700 1 --2- 172.21.15.31:0/4257734255 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5574742dba10 0x5574742fa700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:14.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.642+0000 7fe5daffd700 1 -- 172.21.15.31:0/4257734255 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5574742d6270 con 0x5574742da2c0 2023-10-06T02:32:14.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.642+0000 7fe5da7fc700 1 --2- 172.21.15.31:0/4257734255 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5574742dba10 0x5574742fa700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:32:14.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.642+0000 7fe5daffd700 1 --2- 172.21.15.31:0/4257734255 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574742da2c0 0x557474320300 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc016750 tx=0x7fe5cc018040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2e6c9164f9f68b4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:14.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.643+0000 7fe5c0ff9700 1 -- 172.21.15.31:0/4257734255 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5cc01b030 con 0x5574742da2c0 2023-10-06T02:32:14.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.643+0000 7fe5c0ff9700 1 -- 172.21.15.31:0/4257734255 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5cc01c040 con 0x5574742da2c0 2023-10-06T02:32:14.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.643+0000 7fe5c0ff9700 1 -- 172.21.15.31:0/4257734255 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5cc006870 con 0x5574742da2c0 2023-10-06T02:32:14.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.643+0000 7fe5efe8cb80 1 -- 172.21.15.31:0/4257734255 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574742da2c0 msgr2=0x557474320300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:14.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.643+0000 7fe5efe8cb80 1 --2- 172.21.15.31:0/4257734255 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574742da2c0 0x557474320300 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc016750 tx=0x7fe5cc018040 comp rx=0 tx=0).stop 2023-10-06T02:32:14.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.643+0000 7fe5efe8cb80 1 -- 172.21.15.31:0/4257734255 shutdown_connections 2023-10-06T02:32:14.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.643+0000 7fe5efe8cb80 1 --2- 172.21.15.31:0/4257734255 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5574742f9e50 0x5574742d4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:14.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.643+0000 7fe5efe8cb80 1 --2- 172.21.15.31:0/4257734255 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574742da2c0 0x557474320300 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:14.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.643+0000 7fe5efe8cb80 1 --2- 172.21.15.31:0/4257734255 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5574742dba10 0x5574742fa700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:14.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.643+0000 7fe5efe8cb80 1 -- 172.21.15.31:0/4257734255 >> 172.21.15.31:0/4257734255 conn(0x5574742d9e80 msgr2=0x5574742cf400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:14.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.643+0000 7fe5efe8cb80 1 -- 172.21.15.31:0/4257734255 shutdown_connections 2023-10-06T02:32:14.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.643+0000 7fe5efe8cb80 1 -- 172.21.15.31:0/4257734255 wait complete. 2023-10-06T02:32:14.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.644+0000 7fe5efe8cb80 1 -- start start 2023-10-06T02:32:14.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.644+0000 7fe5efe8cb80 1 Processor -- start 2023-10-06T02:32:14.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.645+0000 7fe5efe8cb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5574742da2c0 0x55747437b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:14.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.645+0000 7fe5efe8cb80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5574742dba10 0x557474328c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:14.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.645+0000 7fe5efe8cb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574742f9e50 0x5574742f7f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:14.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.645+0000 7fe5efe8cb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574742db2a0 con 0x5574742f9e50 2023-10-06T02:32:14.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.645+0000 7fe5efe8cb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55747432c560 con 0x5574742da2c0 2023-10-06T02:32:14.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.645+0000 7fe5efe8cb80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x5574742f7680 con 0x5574742dba10 2023-10-06T02:32:14.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.645+0000 7fe5efe8cb80 10 client.0 _ll_register_callbacks cb 0x55747429c020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T02:32:14.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.645+0000 7fe5db7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574742f9e50 0x5574742f7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:14.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.645+0000 7fe5db7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574742f9e50 0x5574742f7f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41206/0 (socket says 172.21.15.31:41206) 2023-10-06T02:32:14.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.645+0000 7fe5db7fe700 1 -- 172.21.15.31:0/506859678 learned_addr learned my addr 172.21.15.31:0/506859678 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:14.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.646+0000 7fe5da7fc700 1 --2- 172.21.15.31:0/506859678 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5574742dba10 0x557474328c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:14.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.646+0000 7fe5daffd700 1 --2- 172.21.15.31:0/506859678 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5574742da2c0 0x55747437b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:14.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.646+0000 7fe5db7fe700 1 -- 172.21.15.31:0/506859678 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5574742dba10 msgr2=0x557474328c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:14.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.646+0000 7fe5db7fe700 1 --2- 172.21.15.31:0/506859678 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5574742dba10 0x557474328c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:14.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.646+0000 7fe5db7fe700 1 -- 172.21.15.31:0/506859678 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5574742da2c0 msgr2=0x55747437b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:14.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.646+0000 7fe5db7fe700 1 --2- 172.21.15.31:0/506859678 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5574742da2c0 0x55747437b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:14.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.646+0000 7fe5db7fe700 1 -- 172.21.15.31:0/506859678 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5dc0b0110 con 0x5574742f9e50 2023-10-06T02:32:14.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.646+0000 7fe5daffd700 1 --2- 172.21.15.31:0/506859678 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5574742da2c0 0x55747437b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:32:14.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.646+0000 7fe5db7fe700 1 --2- 172.21.15.31:0/506859678 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574742f9e50 0x5574742f7f10 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7fe5d4009e90 tx=0x7fe5d406b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de705552c82597a6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:14.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.646+0000 7fe5b97fa700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T02:32:14.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.646+0000 7fe5b97fa700 1 -- 172.21.15.31:0/506859678 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d4059400 con 0x5574742f9e50 2023-10-06T02:32:14.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.646+0000 7fe5efe8cb80 1 -- 172.21.15.31:0/506859678 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe5d405b050 con 0x5574742f9e50 2023-10-06T02:32:14.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.646+0000 7fe5b97fa700 1 -- 172.21.15.31:0/506859678 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(169..169 src has 1..169) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fe5d405d150 con 0x5574742f9e50 2023-10-06T02:32:14.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.646+0000 7fe5b97fa700 1 -- 172.21.15.31:0/506859678 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=119) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe5d405fe90 con 0x5574742f9e50 2023-10-06T02:32:14.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.646+0000 7fe5efe8cb80 10 client.9429 fetch_fsmap learned FSMap version 119 2023-10-06T02:32:14.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.646+0000 7fe5efe8cb80 1 -- 172.21.15.31:0/506859678 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=119,monmap=2+}) v3 -- 0x5574741f7780 con 0x5574742f9e50 2023-10-06T02:32:14.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.647+0000 7fe5b97fa700 1 -- 172.21.15.31:0/506859678 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5d405b050 con 0x5574742f9e50 2023-10-06T02:32:14.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.647+0000 7fe5b97fa700 1 -- 172.21.15.31:0/506859678 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 119) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fe5d405ccc0 con 0x5574742f9e50 2023-10-06T02:32:14.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.647+0000 7fe5efe8cb80 10 client.9429 fetch_fsmap finished waiting for FSMap version 119 2023-10-06T02:32:14.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.647+0000 7fe5efe8cb80 10 client.9429 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T02:32:14.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.647+0000 7fe5efe8cb80 20 client.9429 populate_metadata read hostname 'smithi031' 2023-10-06T02:32:14.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.647+0000 7fe5efe8cb80 1 --2- 172.21.15.31:0/506859678 >> [v2:172.21.15.31:6832/3741373122,v1:172.21.15.31:6834/3741373122] conn(0x55747432f740 0x55747432fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:14.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.647+0000 7fe5efe8cb80 4 client.9429 mds_command: new command op to 9267 tid=0[{"prefix": "status"}] 2023-10-06T02:32:14.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.647+0000 7fe5efe8cb80 1 -- 172.21.15.31:0/506859678 --> [v2:172.21.15.31:6832/3741373122,v1:172.21.15.31:6834/3741373122] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5574741cbe00 con 0x55747432f740 2023-10-06T02:32:14.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.647+0000 7fe5daffd700 1 --2- 172.21.15.31:0/506859678 >> [v2:172.21.15.31:6832/3741373122,v1:172.21.15.31:6834/3741373122] conn(0x55747432f740 0x55747432fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:14.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.648+0000 7fe5b97fa700 10 client.9429 ms_handle_connect on v2:172.21.15.31:6832/3741373122 2023-10-06T02:32:14.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.648+0000 7fe5daffd700 1 --2- 172.21.15.31:0/506859678 >> [v2:172.21.15.31:6832/3741373122,v1:172.21.15.31:6834/3741373122] conn(0x55747432f740 0x55747432fb10 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ddd93d124b2665a server_cookie=85d4815feda3da92 in_seq=0 out_seq=0 2023-10-06T02:32:14.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.648+0000 7fe5b97fa700 1 -- 172.21.15.31:0/506859678 <== mds.0 v2:172.21.15.31:6832/3741373122 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+328 (crc 0 0 0) 0x5574741cbe00 con 0x55747432f740 2023-10-06T02:32:14.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.648+0000 7fe5b97fa700 10 client.9429 handle_command_reply: tid=0 2023-10-06T02:32:14.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.648+0000 7fe5efe8cb80 1 client.9429 shutdown 2023-10-06T02:32:14.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.648+0000 7fe5efe8cb80 1 -- 172.21.15.31:0/506859678 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574742f9e50 msgr2=0x5574742f7f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:14.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.648+0000 7fe5efe8cb80 1 --2- 172.21.15.31:0/506859678 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574742f9e50 0x5574742f7f10 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7fe5d4009e90 tx=0x7fe5d406b000 comp rx=0 tx=0).stop 2023-10-06T02:32:14.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.648+0000 7fe5efe8cb80 1 -- 172.21.15.31:0/506859678 shutdown_connections 2023-10-06T02:32:14.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.648+0000 7fe5efe8cb80 1 --2- 172.21.15.31:0/506859678 >> [v2:172.21.15.31:6832/3741373122,v1:172.21.15.31:6834/3741373122] conn(0x55747432f740 0x55747432fb10 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:14.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.648+0000 7fe5efe8cb80 1 --2- 172.21.15.31:0/506859678 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574742f9e50 0x5574742f7f10 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:14.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.648+0000 7fe5efe8cb80 1 --2- 172.21.15.31:0/506859678 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5574742da2c0 0x55747437b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:14.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.648+0000 7fe5efe8cb80 1 --2- 172.21.15.31:0/506859678 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5574742dba10 0x557474328c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:14.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.648+0000 7fe5b97fa700 0 client.9429 ms_handle_reset on v2:172.21.15.31:6832/3741373122 2023-10-06T02:32:14.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.648+0000 7fe5efe8cb80 1 -- 172.21.15.31:0/506859678 >> 172.21.15.31:0/506859678 conn(0x5574742d9e80 msgr2=0x5574742da690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:14.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.648+0000 7fe5efe8cb80 1 -- 172.21.15.31:0/506859678 shutdown_connections 2023-10-06T02:32:14.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.648+0000 7fe5efe8cb80 1 -- 172.21.15.31:0/506859678 wait complete. 2023-10-06T02:32:14.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.649+0000 7fe5efe8cb80 20 client.9429 trim_cache size 0 max 16384 2023-10-06T02:32:14.655 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T02:32:14.655 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "d6fcf512-874b-4259-b90a-30ae070f00a9", 2023-10-06T02:32:14.655 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 0, 2023-10-06T02:32:14.656 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 9267, 2023-10-06T02:32:14.656 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2023-10-06T02:32:14.656 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2023-10-06T02:32:14.656 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2023-10-06T02:32:14.656 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 6.5359952809999999, 2023-10-06T02:32:14.656 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 119, 2023-10-06T02:32:14.656 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 169, 2023-10-06T02:32:14.656 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 169, 2023-10-06T02:32:14.656 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 12.751663014 2023-10-06T02:32:14.657 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T02:32:14.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.649+0000 7fe5dbfff700 1 -- 172.21.15.31:0/2562322011 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe5c8067830 msgr2=0x7fe5c8069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:14.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.649+0000 7fe5dbfff700 1 --2- 172.21.15.31:0/2562322011 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe5c8067830 0x7fe5c8069ce0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc15cd20 tx=0x7fe5cc007aa0 comp rx=0 tx=0).stop 2023-10-06T02:32:14.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.649+0000 7fe5dbfff700 1 -- 172.21.15.31:0/2562322011 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5dc0cd9d0 msgr2=0x7fe5dc15b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:14.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.649+0000 7fe5dbfff700 1 --2- 172.21.15.31:0/2562322011 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5dc0cd9d0 0x7fe5dc15b1c0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fe5d4009ec0 tx=0x7fe5d400b040 comp rx=0 tx=0).stop 2023-10-06T02:32:14.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.649+0000 7fe5dbfff700 1 -- 172.21.15.31:0/2562322011 shutdown_connections 2023-10-06T02:32:14.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.649+0000 7fe5dbfff700 1 --2- 172.21.15.31:0/2562322011 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5dc0cd9d0 0x7fe5dc15b1c0 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:14.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.649+0000 7fe5dbfff700 1 --2- 172.21.15.31:0/2562322011 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe5c8067830 0x7fe5c8069ce0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:14.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.649+0000 7fe5dbfff700 1 --2- 172.21.15.31:0/2562322011 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe5dc0b2940 0x7fe5dc0af7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:14.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.649+0000 7fe5dbfff700 1 --2- 172.21.15.31:0/2562322011 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5dc0b2d10 0x7fe5dc146d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:14.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.649+0000 7fe5dbfff700 1 -- 172.21.15.31:0/2562322011 >> 172.21.15.31:0/2562322011 conn(0x7fe5dc0675a0 msgr2=0x7fe5dc0aa850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:14.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.649+0000 7fe5dbfff700 1 -- 172.21.15.31:0/2562322011 shutdown_connections 2023-10-06T02:32:14.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.649+0000 7fe5dbfff700 1 -- 172.21.15.31:0/2562322011 wait complete. 2023-10-06T02:32:14.667 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T02:32:14.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.852+0000 7f7d7359e700 1 Processor -- start 2023-10-06T02:32:14.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.853+0000 7f7d7359e700 1 -- start start 2023-10-06T02:32:14.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.854+0000 7f7d7359e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7d7406e0a0 0x7f7d740c6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:14.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.854+0000 7f7d7359e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7d74150380 0x7f7d7406a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:14.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.854+0000 7f7d7359e700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d7406a490 con 0x7f7d7406f3a0 2023-10-06T02:32:14.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.854+0000 7f7d7359e700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d740c7490 con 0x7f7d74150380 2023-10-06T02:32:14.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.854+0000 7f7d7359e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d740c7770 con 0x7f7d7406e0a0 2023-10-06T02:32:14.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.854+0000 7f7d71d9b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f7d7406f3a0 0x7f7d7414fc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:47762/0 (socket says 172.21.15.31:47762) 2023-10-06T02:32:14.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.854+0000 7f7d71d9b700 1 -- 172.21.15.31:0/2695747126 learned_addr learned my addr 172.21.15.31:0/2695747126 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:32:14.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.854+0000 7f7d72d9d700 1 --2- 172.21.15.31:0/2695747126 >> v2:172.21.15.31:3300/0 conn(0x7f7d74150380 0x7f7d7406a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:14.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.854+0000 7f7d7259c700 1 --2- 172.21.15.31:0/2695747126 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7d7406e0a0 0x7f7d740c6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:14.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.855+0000 7f7d7159a700 1 -- 172.21.15.31:0/2695747126 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1276700976 0 0) 0x7f7d7406a490 con 0x7f7d7406f3a0 2023-10-06T02:32:14.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.855+0000 7f7d7159a700 1 -- 172.21.15.31:0/2695747126 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d5c003570 con 0x7f7d7406f3a0 2023-10-06T02:32:14.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.855+0000 7f7d72d9d700 1 -- 172.21.15.31:0/2695747126 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7d7406e0a0 msgr2=0x7f7d740c6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:14.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.855+0000 7f7d72d9d700 1 --2- 172.21.15.31:0/2695747126 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7d7406e0a0 0x7f7d740c6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:14.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.855+0000 7f7d72d9d700 1 -- 172.21.15.31:0/2695747126 >> v1:172.21.15.31:6789/0 conn(0x7f7d7406f3a0 legacy=0x7f7d7414fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:14.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.855+0000 7f7d72d9d700 1 -- 172.21.15.31:0/2695747126 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d740c7a50 con 0x7f7d74150380 2023-10-06T02:32:14.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.855+0000 7f7d72d9d700 1 --2- 172.21.15.31:0/2695747126 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d74150380 0x7f7d7406a920 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c009e60 tx=0x7f7d6c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3e57b67ee7ea9c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:14.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.855+0000 7f7d7159a700 1 -- 172.21.15.31:0/2695747126 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d6c00e070 con 0x7f7d74150380 2023-10-06T02:32:14.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.855+0000 7f7d7159a700 1 -- 172.21.15.31:0/2695747126 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d6c00bab0 con 0x7f7d74150380 2023-10-06T02:32:14.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.855+0000 7f7d7159a700 1 -- 172.21.15.31:0/2695747126 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d6c00be70 con 0x7f7d74150380 2023-10-06T02:32:14.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.856+0000 7f7d7359e700 1 -- 172.21.15.31:0/2695747126 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d74150380 msgr2=0x7f7d7406a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:14.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.856+0000 7f7d7359e700 1 --2- 172.21.15.31:0/2695747126 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d74150380 0x7f7d7406a920 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c009e60 tx=0x7f7d6c009780 comp rx=0 tx=0).stop 2023-10-06T02:32:14.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.856+0000 7f7d7359e700 1 -- 172.21.15.31:0/2695747126 shutdown_connections 2023-10-06T02:32:14.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.856+0000 7f7d7359e700 1 --2- 172.21.15.31:0/2695747126 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d74150380 0x7f7d7406a920 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:14.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.856+0000 7f7d7359e700 1 --2- 172.21.15.31:0/2695747126 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7d7406e0a0 0x7f7d740c6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:14.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.856+0000 7f7d7359e700 1 -- 172.21.15.31:0/2695747126 >> 172.21.15.31:0/2695747126 conn(0x7f7d7406d170 msgr2=0x7f7d740cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:14.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.856+0000 7f7d7359e700 1 -- 172.21.15.31:0/2695747126 shutdown_connections 2023-10-06T02:32:14.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.856+0000 7f7d7359e700 1 -- 172.21.15.31:0/2695747126 wait complete. 2023-10-06T02:32:14.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.857+0000 7f7d7359e700 1 Processor -- start 2023-10-06T02:32:14.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.857+0000 7f7d7359e700 1 -- start start 2023-10-06T02:32:14.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.858+0000 7f7d7359e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d7406e0a0 0x7f7d74141170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:14.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.858+0000 7f7d7359e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7d7406f3a0 0x7f7d74157c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:14.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.858+0000 7f7d7359e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d74150380 0x7f7d74158170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:14.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.858+0000 7f7d7359e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d741589a0 con 0x7f7d7406e0a0 2023-10-06T02:32:14.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.858+0000 7f7d7359e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d74158c80 con 0x7f7d74150380 2023-10-06T02:32:14.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.858+0000 7f7d7359e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d74158f60 con 0x7f7d7406f3a0 2023-10-06T02:32:14.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.859+0000 7f7d7259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d7406e0a0 0x7f7d74141170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:14.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.859+0000 7f7d7259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d7406e0a0 0x7f7d74141170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41222/0 (socket says 172.21.15.31:41222) 2023-10-06T02:32:14.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.859+0000 7f7d7259c700 1 -- 172.21.15.31:0/2499526876 learned_addr learned my addr 172.21.15.31:0/2499526876 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:14.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.859+0000 7f7d71d9b700 1 --2- 172.21.15.31:0/2499526876 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7d7406f3a0 0x7f7d74157c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:14.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.859+0000 7f7d72d9d700 1 --2- 172.21.15.31:0/2499526876 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d74150380 0x7f7d74158170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:14.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.859+0000 7f7d7259c700 1 -- 172.21.15.31:0/2499526876 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7d7406f3a0 msgr2=0x7f7d74157c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:14.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.859+0000 7f7d7259c700 1 --2- 172.21.15.31:0/2499526876 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7d7406f3a0 0x7f7d74157c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:14.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.859+0000 7f7d7259c700 1 -- 172.21.15.31:0/2499526876 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d74150380 msgr2=0x7f7d74158170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:14.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.859+0000 7f7d7259c700 1 --2- 172.21.15.31:0/2499526876 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d74150380 0x7f7d74158170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:14.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.859+0000 7f7d7259c700 1 -- 172.21.15.31:0/2499526876 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d6c009a20 con 0x7f7d7406e0a0 2023-10-06T02:32:14.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.859+0000 7f7d71d9b700 1 --2- 172.21.15.31:0/2499526876 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7d7406f3a0 0x7f7d74157c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:32:14.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.859+0000 7f7d72d9d700 1 --2- 172.21.15.31:0/2499526876 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d74150380 0x7f7d74158170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:32:14.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.859+0000 7f7d7259c700 1 --2- 172.21.15.31:0/2499526876 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d7406e0a0 0x7f7d74141170 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f7d6400ec80 tx=0x7f7d640091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:14.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.859+0000 7f7d627fc700 1 -- 172.21.15.31:0/2499526876 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d6400c9d0 con 0x7f7d7406e0a0 2023-10-06T02:32:14.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.860+0000 7f7d627fc700 1 -- 172.21.15.31:0/2499526876 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d6400cb30 con 0x7f7d7406e0a0 2023-10-06T02:32:14.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.860+0000 7f7d7359e700 1 -- 172.21.15.31:0/2499526876 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d74159240 con 0x7f7d7406e0a0 2023-10-06T02:32:14.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.860+0000 7f7d627fc700 1 -- 172.21.15.31:0/2499526876 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d64013470 con 0x7f7d7406e0a0 2023-10-06T02:32:14.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.860+0000 7f7d7359e700 1 -- 172.21.15.31:0/2499526876 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d74141b60 con 0x7f7d7406e0a0 2023-10-06T02:32:14.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.861+0000 7f7d7359e700 1 -- 172.21.15.31:0/2499526876 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d740c6490 con 0x7f7d7406e0a0 2023-10-06T02:32:14.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.862+0000 7f7d627fc700 1 -- 172.21.15.31:0/2499526876 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85556+0+0 (secure 0 0 0) 0x7f7d64013820 con 0x7f7d7406e0a0 2023-10-06T02:32:14.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.863+0000 7f7d627fc700 1 --2- 172.21.15.31:0/2499526876 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7d54067860 0x7f7d54069d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:14.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.863+0000 7f7d627fc700 1 -- 172.21.15.31:0/2499526876 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f7d640833f0 con 0x7f7d7406e0a0 2023-10-06T02:32:14.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.863+0000 7f7d71d9b700 1 --2- 172.21.15.31:0/2499526876 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7d54067860 0x7f7d54069d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:14.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.864+0000 7f7d71d9b700 1 --2- 172.21.15.31:0/2499526876 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7d54067860 0x7f7d54069d10 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f7d74150070 tx=0x7f7d68009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:14.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:14.866+0000 7f7d627fc700 1 -- 172.21.15.31:0/2499526876 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7d6408b020 con 0x7f7d7406e0a0 2023-10-06T02:32:15.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:15.116+0000 7f7d7359e700 1 -- 172.21.15.31:0/2499526876 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f7d741587c0 con 0x7f7d7406e0a0 2023-10-06T02:32:15.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:15.117+0000 7f7d627fc700 1 -- 172.21.15.31:0/2499526876 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v119) v1 ==== 53+0+73 (secure 0 0 0) 0x7f7d64017070 con 0x7f7d7406e0a0 2023-10-06T02:32:15.117 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T02:32:15.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:15.119+0000 7f7d7359e700 1 -- 172.21.15.31:0/2499526876 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7d54067860 msgr2=0x7f7d54069d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:15.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:15.119+0000 7f7d7359e700 1 --2- 172.21.15.31:0/2499526876 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7d54067860 0x7f7d54069d10 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f7d74150070 tx=0x7f7d68009040 comp rx=0 tx=0).stop 2023-10-06T02:32:15.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:15.119+0000 7f7d7359e700 1 -- 172.21.15.31:0/2499526876 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d7406e0a0 msgr2=0x7f7d74141170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:15.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:15.119+0000 7f7d7359e700 1 --2- 172.21.15.31:0/2499526876 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d7406e0a0 0x7f7d74141170 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f7d6400ec80 tx=0x7f7d640091a0 comp rx=0 tx=0).stop 2023-10-06T02:32:15.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:15.119+0000 7f7d7359e700 1 -- 172.21.15.31:0/2499526876 shutdown_connections 2023-10-06T02:32:15.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:15.119+0000 7f7d7359e700 1 --2- 172.21.15.31:0/2499526876 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d74150380 0x7f7d74158170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:15.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:15.119+0000 7f7d7359e700 1 --2- 172.21.15.31:0/2499526876 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d7406e0a0 0x7f7d74141170 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:15.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:15.119+0000 7f7d7359e700 1 --2- 172.21.15.31:0/2499526876 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7d54067860 0x7f7d54069d10 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:15.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:15.119+0000 7f7d7359e700 1 --2- 172.21.15.31:0/2499526876 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7d7406f3a0 0x7f7d74157c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:15.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:15.119+0000 7f7d7359e700 1 -- 172.21.15.31:0/2499526876 >> 172.21.15.31:0/2499526876 conn(0x7f7d7406d170 msgr2=0x7f7d741463a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:15.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:15.119+0000 7f7d7359e700 1 -- 172.21.15.31:0/2499526876 shutdown_connections 2023-10-06T02:32:15.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:15.119+0000 7f7d7359e700 1 -- 172.21.15.31:0/2499526876 wait complete. 2023-10-06T02:32:15.138 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T02:32:15.138 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T02:32:15.138 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi031.front.sepia.ceph.com 2023-10-06T02:32:15.138 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T02:32:15.138 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T02:32:15.138 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T02:32:15.138 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T02:32:15.138 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T02:32:15.139 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T02:32:15.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:32:15.139 DEBUG:teuthology.orchestra.run.smithi031:> ip addr 2023-10-06T02:32:15.177 INFO:teuthology.orchestra.run.smithi031.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T02:32:15.177 INFO:teuthology.orchestra.run.smithi031.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T02:32:15.177 INFO:teuthology.orchestra.run.smithi031.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T02:32:15.177 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:32:15.177 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 ::1/128 scope host 2023-10-06T02:32:15.177 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:32:15.177 INFO:teuthology.orchestra.run.smithi031.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T02:32:15.177 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cc brd ff:ff:ff:ff:ff:ff 2023-10-06T02:32:15.177 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f0 2023-10-06T02:32:15.177 INFO:teuthology.orchestra.run.smithi031.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T02:32:15.178 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cd brd ff:ff:ff:ff:ff:ff 2023-10-06T02:32:15.178 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f1 2023-10-06T02:32:15.178 INFO:teuthology.orchestra.run.smithi031.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T02:32:15.178 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:ba brd ff:ff:ff:ff:ff:ff 2023-10-06T02:32:15.178 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f0 2023-10-06T02:32:15.178 INFO:teuthology.orchestra.run.smithi031.stdout: inet 172.21.15.31/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T02:32:15.178 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft 41721sec preferred_lft 41721sec 2023-10-06T02:32:15.178 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::ae1f:6bff:fea5:caba/64 scope link 2023-10-06T02:32:15.178 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:32:15.178 INFO:teuthology.orchestra.run.smithi031.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T02:32:15.178 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:bb brd ff:ff:ff:ff:ff:ff 2023-10-06T02:32:15.179 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f1 2023-10-06T02:32:15.179 INFO:teuthology.orchestra.run.smithi031.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T02:32:15.179 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 16:d7:88:2d:d0:c2 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:32:15.179 INFO:teuthology.orchestra.run.smithi031.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T02:32:15.179 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:32:15.179 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::408:52ff:fe3c:ba8f/64 scope link 2023-10-06T02:32:15.179 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:32:15.179 INFO:teuthology.orchestra.run.smithi031.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T02:32:15.179 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 16:d7:88:2d:d0:c2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T02:32:15.179 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::14d7:88ff:fe2d:d0c2/64 scope link 2023-10-06T02:32:15.179 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:32:15.179 INFO:teuthology.orchestra.run.smithi031.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T02:32:15.179 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 82:16:f1:00:cf:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T02:32:15.180 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::8016:f1ff:fe00:cf2f/64 scope link 2023-10-06T02:32:15.180 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:32:15.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:32:15.180 DEBUG:teuthology.orchestra.run.smithi031:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T02:32:15.277 INFO:teuthology.orchestra.run.smithi031.stdout:1 2023-10-06T02:32:15.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:32:15.280 DEBUG:teuthology.orchestra.run.smithi031:> route 2023-10-06T02:32:15.319 INFO:teuthology.orchestra.run.smithi031.stdout:Kernel IP routing table 2023-10-06T02:32:15.319 INFO:teuthology.orchestra.run.smithi031.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T02:32:15.319 INFO:teuthology.orchestra.run.smithi031.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T02:32:15.319 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T02:32:15.319 INFO:teuthology.orchestra.run.smithi031.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T02:32:15.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:32:15.320 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2023-10-06T02:32:15.320 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2023-10-06T02:32:15.321 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T02:32:15.321 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T02:32:15.321 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T02:32:15.321 DEBUG:teuthology.orchestra.run.smithi031:> ') 2023-10-06T02:32:15.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:32:15.520 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2023-10-06T02:32:15.556 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T02:32:15.557 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T02:32:15.557 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T02:32:15.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:32:15.557 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set brx.0 up 2023-10-06T02:32:15.659 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T02:32:15.659 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T02:32:15.696 INFO:teuthology.orchestra.run.smithi031.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T02:32:15.697 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T02:32:15.697 DEBUG:teuthology.orchestra.run.smithi031:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T02:32:15.773 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-06T02:32:15.773 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T02:32:15.773 DEBUG:teuthology.orchestra.run.smithi031:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T02:32:15.905 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T02:32:15.906 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T02:32:15.933 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T02:32:15.933 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T02:32:15.933 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.62:3300,172.21.15.62:3301" 2023-10-06T02:32:15.933 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: trying mount with new device syntax: 0@d6fcf512-874b-4259-b90a-30ae070f00a9.cephfs=/ 2023-10-06T02:32:15.933 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.31:3300/172.21.15.62:3300/172.21.15.62:3301" will pass to kernel 2023-10-06T02:32:15.939 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T02:32:15.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:32:15.939 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T02:32:16.002 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T02:32:16.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.190+0000 7f39f7fff700 1 Processor -- start 2023-10-06T02:32:16.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.190+0000 7f39f7fff700 1 -- start start 2023-10-06T02:32:16.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.192+0000 7f39f7fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39f813fef0 0x7f39f815a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:16.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.192+0000 7f39f7fff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39f815ae70 con 0x7f39f806aaa0 2023-10-06T02:32:16.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.192+0000 7f39f7fff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39f806af60 con 0x7f39f813fb20 2023-10-06T02:32:16.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.192+0000 7f39f7fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39f806b240 con 0x7f39f813fef0 2023-10-06T02:32:16.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.192+0000 7f39f77fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f39f806aaa0 0x7f39f80c9d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:47768/0 (socket says 172.21.15.31:47768) 2023-10-06T02:32:16.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.192+0000 7f39f77fe700 1 -- 172.21.15.31:0/3494022650 learned_addr learned my addr 172.21.15.31:0/3494022650 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:32:16.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.192+0000 7f39f67fc700 1 --2- 172.21.15.31:0/3494022650 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39f813fef0 0x7f39f815a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:16.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.192+0000 7f39f5ffb700 1 -- 172.21.15.31:0/3494022650 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 901666136 0 0) 0x7f39f0001490 con 0x7f39f806aaa0 2023-10-06T02:32:16.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.192+0000 7f39f5ffb700 1 -- 172.21.15.31:0/3494022650 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39e0003570 con 0x7f39f806aaa0 2023-10-06T02:32:16.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.193+0000 7f39f5ffb700 1 -- 172.21.15.31:0/3494022650 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1896945326 0 0) 0x7f39e0003570 con 0x7f39f806aaa0 2023-10-06T02:32:16.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.193+0000 7f39f5ffb700 1 -- 172.21.15.31:0/3494022650 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39f813fef0 msgr2=0x7f39f815a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:16.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.193+0000 7f39f5ffb700 1 --2- 172.21.15.31:0/3494022650 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39f813fef0 0x7f39f815a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:16.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.193+0000 7f39f5ffb700 1 -- 172.21.15.31:0/3494022650 >> v1:172.21.15.62:6789/0 conn(0x7f39f813fb20 legacy=0x7f39f80c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:16.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.193+0000 7f39f5ffb700 1 -- 172.21.15.31:0/3494022650 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39f806b520 con 0x7f39f806aaa0 2023-10-06T02:32:16.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.193+0000 7f39f5ffb700 1 -- 172.21.15.31:0/3494022650 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7f39f0001980 con 0x7f39f806aaa0 2023-10-06T02:32:16.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.193+0000 7f39f5ffb700 1 -- 172.21.15.31:0/3494022650 >> v1:172.21.15.31:6789/0 conn(0x7f39f806aaa0 legacy=0x7f39f80c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:16.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.193+0000 7f39f5ffb700 1 --2- 172.21.15.31:0/3494022650 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39e0003a90 0x7f39e0003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:16.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.193+0000 7f39f5ffb700 1 --2- 172.21.15.31:0/3494022650 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39e0004810 0x7f39e0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:16.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.193+0000 7f39f5ffb700 1 --2- 172.21.15.31:0/3494022650 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39f813fef0 0x7f39e0005270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:16.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.193+0000 7f39f5ffb700 1 -- 172.21.15.31:0/3494022650 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f806b240 con 0x7f39e0004810 2023-10-06T02:32:16.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.193+0000 7f39f5ffb700 1 -- 172.21.15.31:0/3494022650 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e000de00 con 0x7f39e0003a90 2023-10-06T02:32:16.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.193+0000 7f39f5ffb700 1 -- 172.21.15.31:0/3494022650 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39e000dfc0 con 0x7f39f813fef0 2023-10-06T02:32:16.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.193+0000 7f39f67fc700 1 --2- 172.21.15.31:0/3494022650 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39e0004810 0x7f39e0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:16.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.193+0000 7f39f6ffd700 1 --2- 172.21.15.31:0/3494022650 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39e0003a90 0x7f39e0003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:16.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.193+0000 7f39f77fe700 1 --2- 172.21.15.31:0/3494022650 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39f813fef0 0x7f39e0005270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:16.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.194+0000 7f39f67fc700 1 -- 172.21.15.31:0/3494022650 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39f813fef0 msgr2=0x7f39e0005270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:16.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.194+0000 7f39f67fc700 1 --2- 172.21.15.31:0/3494022650 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39f813fef0 0x7f39e0005270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:16.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.194+0000 7f39f67fc700 1 -- 172.21.15.31:0/3494022650 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39e0003a90 msgr2=0x7f39e0003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:16.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.194+0000 7f39f67fc700 1 --2- 172.21.15.31:0/3494022650 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39e0003a90 0x7f39e0003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:16.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.194+0000 7f39f67fc700 1 -- 172.21.15.31:0/3494022650 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39e0003570 con 0x7f39e0004810 2023-10-06T02:32:16.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.194+0000 7f39f67fc700 1 --2- 172.21.15.31:0/3494022650 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39e0004810 0x7f39e0004c80 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f39f8069580 tx=0x7f39ec009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3326dc0cabd6b8e1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:16.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.194+0000 7f39f5ffb700 1 -- 172.21.15.31:0/3494022650 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ec005a00 con 0x7f39e0004810 2023-10-06T02:32:16.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.194+0000 7f39f5ffb700 1 -- 172.21.15.31:0/3494022650 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39ec005b60 con 0x7f39e0004810 2023-10-06T02:32:16.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.194+0000 7f39f5ffb700 1 -- 172.21.15.31:0/3494022650 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ec00e640 con 0x7f39e0004810 2023-10-06T02:32:16.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.195+0000 7f39f7fff700 1 -- 172.21.15.31:0/3494022650 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39e0004810 msgr2=0x7f39e0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:16.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.195+0000 7f39f7fff700 1 --2- 172.21.15.31:0/3494022650 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39e0004810 0x7f39e0004c80 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f39f8069580 tx=0x7f39ec009120 comp rx=0 tx=0).stop 2023-10-06T02:32:16.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.195+0000 7f39f77fe700 1 -- 172.21.15.31:0/3494022650 reap_dead start 2023-10-06T02:32:16.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.195+0000 7f39f7fff700 1 -- 172.21.15.31:0/3494022650 shutdown_connections 2023-10-06T02:32:16.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.195+0000 7f39f7fff700 1 -- 172.21.15.31:0/3494022650 >> 172.21.15.31:0/3494022650 conn(0x7f39f806d170 msgr2=0x7f39f80cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:16.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.195+0000 7f39f7fff700 1 -- 172.21.15.31:0/3494022650 shutdown_connections 2023-10-06T02:32:16.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.195+0000 7f39f7fff700 1 -- 172.21.15.31:0/3494022650 wait complete. 2023-10-06T02:32:16.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.196+0000 7f39f7fff700 1 Processor -- start 2023-10-06T02:32:16.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.196+0000 7f39f7fff700 1 -- start start 2023-10-06T02:32:16.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.197+0000 7f39f7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39e0004810 0x7f39f8154420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:16.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.197+0000 7f39f7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39f815e910 0x7f39f80b0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:16.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.197+0000 7f39f7fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39f8169cd0 0x7f39f80cae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:16.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.197+0000 7f39f7fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f816a1d0 con 0x7f39e0004810 2023-10-06T02:32:16.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.197+0000 7f39f7fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f816a4b0 con 0x7f39f815e910 2023-10-06T02:32:16.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.197+0000 7f39f7fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39f816a790 con 0x7f39f8169cd0 2023-10-06T02:32:16.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.197+0000 7f39f6ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39e0004810 0x7f39f8154420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:16.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.198+0000 7f39f6ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39e0004810 0x7f39f8154420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41238/0 (socket says 172.21.15.31:41238) 2023-10-06T02:32:16.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.198+0000 7f39f6ffd700 1 -- 172.21.15.31:0/1404326729 learned_addr learned my addr 172.21.15.31:0/1404326729 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:16.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.198+0000 7f39f67fc700 1 --2- 172.21.15.31:0/1404326729 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39f815e910 0x7f39f80b0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:16.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.198+0000 7f39f77fe700 1 --2- 172.21.15.31:0/1404326729 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39f8169cd0 0x7f39f80cae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:16.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.198+0000 7f39f6ffd700 1 -- 172.21.15.31:0/1404326729 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39f8169cd0 msgr2=0x7f39f80cae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:16.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.198+0000 7f39f6ffd700 1 --2- 172.21.15.31:0/1404326729 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39f8169cd0 0x7f39f80cae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:16.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.198+0000 7f39f6ffd700 1 -- 172.21.15.31:0/1404326729 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39f815e910 msgr2=0x7f39f80b0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:16.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.198+0000 7f39f6ffd700 1 --2- 172.21.15.31:0/1404326729 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39f815e910 0x7f39f80b0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:16.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.198+0000 7f39f6ffd700 1 -- 172.21.15.31:0/1404326729 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39f816aa70 con 0x7f39e0004810 2023-10-06T02:32:16.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.198+0000 7f39f6ffd700 1 --2- 172.21.15.31:0/1404326729 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39e0004810 0x7f39f8154420 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f39e8002a70 tx=0x7f39e8007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:16.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.198+0000 7f39deffd700 1 -- 172.21.15.31:0/1404326729 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e800f8d0 con 0x7f39e0004810 2023-10-06T02:32:16.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.198+0000 7f39deffd700 1 -- 172.21.15.31:0/1404326729 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39e800fa30 con 0x7f39e0004810 2023-10-06T02:32:16.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.198+0000 7f39f7fff700 1 -- 172.21.15.31:0/1404326729 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39f80aeed0 con 0x7f39e0004810 2023-10-06T02:32:16.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.198+0000 7f39deffd700 1 -- 172.21.15.31:0/1404326729 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e800fdf0 con 0x7f39e0004810 2023-10-06T02:32:16.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.199+0000 7f39f7fff700 1 -- 172.21.15.31:0/1404326729 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39f80cb560 con 0x7f39e0004810 2023-10-06T02:32:16.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.200+0000 7f39f7fff700 1 -- 172.21.15.31:0/1404326729 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39f815d360 con 0x7f39e0004810 2023-10-06T02:32:16.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.200+0000 7f39deffd700 1 -- 172.21.15.31:0/1404326729 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85556+0+0 (secure 0 0 0) 0x7f39e8013600 con 0x7f39e0004810 2023-10-06T02:32:16.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.201+0000 7f39deffd700 1 --2- 172.21.15.31:0/1404326729 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f39e4067830 0x7f39e4069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:16.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.201+0000 7f39deffd700 1 -- 172.21.15.31:0/1404326729 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f39e8083420 con 0x7f39e0004810 2023-10-06T02:32:16.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.202+0000 7f39f67fc700 1 --2- 172.21.15.31:0/1404326729 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f39e4067830 0x7f39e4069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:16.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.203+0000 7f39f67fc700 1 --2- 172.21.15.31:0/1404326729 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f39e4067830 0x7f39e4069ce0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f39ec006e70 tx=0x7f39ec019000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:16.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.206+0000 7f39deffd700 1 -- 172.21.15.31:0/1404326729 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f39e8052060 con 0x7f39e0004810 2023-10-06T02:32:16.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.453+0000 7f39f7fff700 1 -- 172.21.15.31:0/1404326729 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f39f816ac00 con 0x7f39e0004810 2023-10-06T02:32:16.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.453+0000 7f39deffd700 1 -- 172.21.15.31:0/1404326729 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v119) v1 ==== 53+0+73 (secure 0 0 0) 0x7f39e8055680 con 0x7f39e0004810 2023-10-06T02:32:16.454 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T02:32:16.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.456+0000 7f39f7fff700 1 -- 172.21.15.31:0/1404326729 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f39e4067830 msgr2=0x7f39e4069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:16.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.456+0000 7f39f7fff700 1 --2- 172.21.15.31:0/1404326729 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f39e4067830 0x7f39e4069ce0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f39ec006e70 tx=0x7f39ec019000 comp rx=0 tx=0).stop 2023-10-06T02:32:16.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.456+0000 7f39f7fff700 1 -- 172.21.15.31:0/1404326729 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39e0004810 msgr2=0x7f39f8154420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:16.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.456+0000 7f39f7fff700 1 --2- 172.21.15.31:0/1404326729 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39e0004810 0x7f39f8154420 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f39e8002a70 tx=0x7f39e8007d30 comp rx=0 tx=0).stop 2023-10-06T02:32:16.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.456+0000 7f39f7fff700 1 -- 172.21.15.31:0/1404326729 shutdown_connections 2023-10-06T02:32:16.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.456+0000 7f39f7fff700 1 --2- 172.21.15.31:0/1404326729 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f39e4067830 0x7f39e4069ce0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:16.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.456+0000 7f39f7fff700 1 --2- 172.21.15.31:0/1404326729 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39f8169cd0 0x7f39f80cae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:16.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.456+0000 7f39f7fff700 1 --2- 172.21.15.31:0/1404326729 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39e0004810 0x7f39f8154420 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:16.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.456+0000 7f39f7fff700 1 --2- 172.21.15.31:0/1404326729 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39f815e910 0x7f39f80b0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:16.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.456+0000 7f39f7fff700 1 -- 172.21.15.31:0/1404326729 >> 172.21.15.31:0/1404326729 conn(0x7f39f806d170 msgr2=0x7f39f806e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:16.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.456+0000 7f39f7fff700 1 -- 172.21.15.31:0/1404326729 shutdown_connections 2023-10-06T02:32:16.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:16.456+0000 7f39f7fff700 1 -- 172.21.15.31:0/1404326729 wait complete. 2023-10-06T02:32:16.474 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T02:32:16.474 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T02:32:16.474 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi062.front.sepia.ceph.com 2023-10-06T02:32:16.474 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-06T02:32:16.474 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-06T02:32:16.474 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T02:32:16.474 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T02:32:16.474 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T02:32:16.474 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T02:32:16.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:32:16.475 DEBUG:teuthology.orchestra.run.smithi062:> ip addr 2023-10-06T02:32:16.518 INFO:teuthology.orchestra.run.smithi062.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T02:32:16.518 INFO:teuthology.orchestra.run.smithi062.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T02:32:16.518 INFO:teuthology.orchestra.run.smithi062.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T02:32:16.518 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:32:16.518 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 ::1/128 scope host 2023-10-06T02:32:16.518 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:32:16.518 INFO:teuthology.orchestra.run.smithi062.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T02:32:16.518 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:14 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:32:16.519 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f0 2023-10-06T02:32:16.519 INFO:teuthology.orchestra.run.smithi062.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T02:32:16.519 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:15 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:32:16.519 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f1 2023-10-06T02:32:16.519 INFO:teuthology.orchestra.run.smithi062.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T02:32:16.519 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c2 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:32:16.519 INFO:teuthology.orchestra.run.smithi062.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T02:32:16.519 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c3 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:32:16.519 INFO:teuthology.orchestra.run.smithi062.stdout: inet 172.21.15.62/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T02:32:16.519 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft 41713sec preferred_lft 41713sec 2023-10-06T02:32:16.519 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::ec4:7aff:febb:d1c3/64 scope link 2023-10-06T02:32:16.519 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:32:16.520 INFO:teuthology.orchestra.run.smithi062.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T02:32:16.520 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether ae:42:a8:dd:f7:9b brd ff:ff:ff:ff:ff:ff 2023-10-06T02:32:16.520 INFO:teuthology.orchestra.run.smithi062.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T02:32:16.520 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:32:16.520 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::1098:a4ff:fe0d:389/64 scope link 2023-10-06T02:32:16.520 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:32:16.520 INFO:teuthology.orchestra.run.smithi062.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T02:32:16.520 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether fe:60:2b:8e:af:28 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T02:32:16.520 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::fc60:2bff:fe8e:af28/64 scope link 2023-10-06T02:32:16.520 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:32:16.520 INFO:teuthology.orchestra.run.smithi062.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T02:32:16.520 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether ae:42:a8:dd:f7:9b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T02:32:16.521 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::ac42:a8ff:fedd:f79b/64 scope link 2023-10-06T02:32:16.521 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:32:16.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:32:16.521 DEBUG:teuthology.orchestra.run.smithi062:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T02:32:16.624 INFO:teuthology.orchestra.run.smithi062.stdout:1 2023-10-06T02:32:16.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:32:16.627 DEBUG:teuthology.orchestra.run.smithi062:> route 2023-10-06T02:32:16.667 INFO:teuthology.orchestra.run.smithi062.stdout:Kernel IP routing table 2023-10-06T02:32:16.667 INFO:teuthology.orchestra.run.smithi062.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T02:32:16.667 INFO:teuthology.orchestra.run.smithi062.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T02:32:16.667 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T02:32:16.667 INFO:teuthology.orchestra.run.smithi062.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T02:32:16.668 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:32:16.668 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2023-10-06T02:32:16.669 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2023-10-06T02:32:16.669 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T02:32:16.669 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T02:32:16.669 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T02:32:16.669 DEBUG:teuthology.orchestra.run.smithi062:> ') 2023-10-06T02:32:16.868 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:32:16.868 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2023-10-06T02:32:16.905 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-06T02:32:16.905 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T02:32:16.905 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T02:32:16.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:32:16.906 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set brx.0 up 2023-10-06T02:32:17.007 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T02:32:17.008 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-06T02:32:17.044 INFO:teuthology.orchestra.run.smithi062.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-06T02:32:17.044 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T02:32:17.044 DEBUG:teuthology.orchestra.run.smithi062:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-06T02:32:17.119 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-06T02:32:17.119 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T02:32:17.120 DEBUG:teuthology.orchestra.run.smithi062:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T02:32:17.252 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T02:32:17.252 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T02:32:17.279 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T02:32:17.280 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T02:32:17.280 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.62:3300,172.21.15.62:3301" 2023-10-06T02:32:17.280 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: trying mount with new device syntax: 1@d6fcf512-874b-4259-b90a-30ae070f00a9.cephfs=/ 2023-10-06T02:32:17.280 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.31:3300/172.21.15.62:3300/172.21.15.62:3301" will pass to kernel 2023-10-06T02:32:17.284 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T02:32:17.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:32:17.284 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-06T02:32:17.346 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-10-06T02:32:17.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:17.549+0000 7fe5f999d700 1 Processor -- start 2023-10-06T02:32:17.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:17.549+0000 7fe5f999d700 1 -- start start 2023-10-06T02:32:17.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:17.550+0000 7fe5f999d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe5f40c85d0 0x7fe5f40cc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:17.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:17.550+0000 7fe5f999d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5f41571b0 con 0x7fe5f40c2a90 2023-10-06T02:32:17.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:17.550+0000 7fe5f999d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5f41579b0 con 0x7fe5f40c8200 2023-10-06T02:32:17.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:17.550+0000 7fe5f999d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f40c1390 con 0x7fe5f40c85d0 2023-10-06T02:32:17.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:17.551+0000 7fe5f3fff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fe5f40c2a90 0x7fe5f4156aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:47770/0 (socket says 172.21.15.31:47770) 2023-10-06T02:32:17.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:17.551+0000 7fe5f3fff700 1 -- 172.21.15.31:0/1402522411 learned_addr learned my addr 172.21.15.31:0/1402522411 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:32:17.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:17.551+0000 7fe5f919c700 1 --2- 172.21.15.31:0/1402522411 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe5f40c85d0 0x7fe5f40cc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:17.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:17.551+0000 7fe5f37fe700 1 -- 172.21.15.31:0/1402522411 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2457297839 0 0) 0x7fe5e0001110 con 0x7fe5f40c2a90 2023-10-06T02:32:17.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:17.551+0000 7fe5f37fe700 1 -- 172.21.15.31:0/1402522411 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5e4003570 con 0x7fe5f40c2a90 2023-10-06T02:32:17.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:17.551+0000 7fe5f37fe700 1 -- 172.21.15.31:0/1402522411 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1779065770 0 0) 0x7fe5f41579b0 con 0x7fe5f40c8200 2023-10-06T02:32:17.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:17.551+0000 7fe5f37fe700 1 -- 172.21.15.31:0/1402522411 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5e0001110 con 0x7fe5f40c8200 2023-10-06T02:32:17.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:17.551+0000 7fe5f919c700 1 -- 172.21.15.31:0/1402522411 >> v1:172.21.15.62:6789/0 conn(0x7fe5f40c8200 legacy=0x7fe5f40c1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:17.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:17.551+0000 7fe5f919c700 1 -- 172.21.15.31:0/1402522411 >> v1:172.21.15.31:6789/0 conn(0x7fe5f40c2a90 legacy=0x7fe5f4156aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:17.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:17.551+0000 7fe5f919c700 1 -- 172.21.15.31:0/1402522411 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5f41573f0 con 0x7fe5f40c85d0 2023-10-06T02:32:17.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:17.552+0000 7fe5f919c700 1 --2- 172.21.15.31:0/1402522411 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe5f40c85d0 0x7fe5f40cc2f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec009e90 tx=0x7fe5ec009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dfa64296ba25fa8c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:17.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:17.552+0000 7fe5f37fe700 1 -- 172.21.15.31:0/1402522411 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec00e070 con 0x7fe5f40c85d0 2023-10-06T02:32:17.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:17.552+0000 7fe5f37fe700 1 -- 172.21.15.31:0/1402522411 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5ec00bb80 con 0x7fe5f40c85d0 2023-10-06T02:32:17.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:17.552+0000 7fe5f37fe700 1 -- 172.21.15.31:0/1402522411 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec012430 con 0x7fe5f40c85d0 2023-10-06T02:32:17.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:17.553+0000 7fe5f999d700 1 -- 172.21.15.31:0/1402522411 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe5f40c85d0 msgr2=0x7fe5f40cc2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:17.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:17.553+0000 7fe5f999d700 1 --2- 172.21.15.31:0/1402522411 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe5f40c85d0 0x7fe5f40cc2f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec009e90 tx=0x7fe5ec009780 comp rx=0 tx=0).stop 2023-10-06T02:32:17.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:17.553+0000 7fe5f999d700 1 -- 172.21.15.31:0/1402522411 shutdown_connections 2023-10-06T02:32:17.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:17.553+0000 7fe5f999d700 1 --2- 172.21.15.31:0/1402522411 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe5f40c85d0 0x7fe5f40cc2f0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:17.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:17.553+0000 7fe5f999d700 1 -- 172.21.15.31:0/1402522411 >> 172.21.15.31:0/1402522411 conn(0x7fe5f406c6a0 msgr2=0x7fe5f406caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:17.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:17.553+0000 7fe5f999d700 1 -- 172.21.15.31:0/1402522411 shutdown_connections 2023-10-06T02:32:17.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:17.553+0000 7fe5f999d700 1 -- 172.21.15.31:0/1402522411 wait complete. 2023-10-06T02:32:17.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:17.554+0000 7fe5f999d700 1 Processor -- start 2023-10-06T02:32:17.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:17.554+0000 7fe5f999d700 1 -- start start 2023-10-06T02:32:17.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:17.555+0000 7fe5f999d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe5f40c2a90 0x7fe5f4136a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:17.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:17.555+0000 7fe5f999d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5f40c8200 0x7fe5f40c9010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:17.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:17.555+0000 7fe5f999d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5f40c85d0 0x7fe5f40c9550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:17.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:17.555+0000 7fe5f999d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f413ba00 con 0x7fe5f40c8200 2023-10-06T02:32:17.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:17.555+0000 7fe5f999d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f4148240 con 0x7fe5f40c85d0 2023-10-06T02:32:17.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:17.555+0000 7fe5f999d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f413b1f0 con 0x7fe5f40c2a90 2023-10-06T02:32:17.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:17.556+0000 7fe5f3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5f40c8200 0x7fe5f40c9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:17.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:17.556+0000 7fe5f3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5f40c8200 0x7fe5f40c9010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41252/0 (socket says 172.21.15.31:41252) 2023-10-06T02:32:17.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:17.556+0000 7fe5f3fff700 1 -- 172.21.15.31:0/4113139828 learned_addr learned my addr 172.21.15.31:0/4113139828 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:17.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:17.556+0000 7fe5f899b700 1 --2- 172.21.15.31:0/4113139828 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe5f40c2a90 0x7fe5f4136a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:17.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:17.556+0000 7fe5f919c700 1 --2- 172.21.15.31:0/4113139828 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5f40c85d0 0x7fe5f40c9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:17.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:17.556+0000 7fe5f3fff700 1 -- 172.21.15.31:0/4113139828 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe5f40c2a90 msgr2=0x7fe5f4136a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:17.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:17.556+0000 7fe5f3fff700 1 --2- 172.21.15.31:0/4113139828 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe5f40c2a90 0x7fe5f4136a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:17.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:17.556+0000 7fe5f3fff700 1 -- 172.21.15.31:0/4113139828 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5f40c85d0 msgr2=0x7fe5f40c9550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:17.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:17.556+0000 7fe5f3fff700 1 --2- 172.21.15.31:0/4113139828 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5f40c85d0 0x7fe5f40c9550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:17.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:17.556+0000 7fe5f3fff700 1 -- 172.21.15.31:0/4113139828 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5ec009a20 con 0x7fe5f40c8200 2023-10-06T02:32:17.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:17.556+0000 7fe5f3fff700 1 --2- 172.21.15.31:0/4113139828 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5f40c8200 0x7fe5f40c9010 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7fe5e000c810 tx=0x7fe5e0007e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:17.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:17.557+0000 7fe5f0ff9700 1 -- 172.21.15.31:0/4113139828 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e0005b30 con 0x7fe5f40c8200 2023-10-06T02:32:17.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:17.557+0000 7fe5f0ff9700 1 -- 172.21.15.31:0/4113139828 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5e0005c90 con 0x7fe5f40c8200 2023-10-06T02:32:17.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:17.557+0000 7fe5f999d700 1 -- 172.21.15.31:0/4113139828 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5f413c070 con 0x7fe5f40c8200 2023-10-06T02:32:17.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:17.557+0000 7fe5f0ff9700 1 -- 172.21.15.31:0/4113139828 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e0007340 con 0x7fe5f40c8200 2023-10-06T02:32:17.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:17.557+0000 7fe5f999d700 1 -- 172.21.15.31:0/4113139828 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5f413b440 con 0x7fe5f40c8200 2023-10-06T02:32:17.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:17.558+0000 7fe5f999d700 1 -- 172.21.15.31:0/4113139828 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5f40ccef0 con 0x7fe5f40c8200 2023-10-06T02:32:17.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:17.559+0000 7fe5f0ff9700 1 -- 172.21.15.31:0/4113139828 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85556+0+0 (secure 0 0 0) 0x7fe5e001b400 con 0x7fe5f40c8200 2023-10-06T02:32:17.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:17.560+0000 7fe5f0ff9700 1 --2- 172.21.15.31:0/4113139828 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe5d8067830 0x7fe5d8069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:17.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:17.560+0000 7fe5f0ff9700 1 -- 172.21.15.31:0/4113139828 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fe5e0082800 con 0x7fe5f40c8200 2023-10-06T02:32:17.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:17.560+0000 7fe5f899b700 1 --2- 172.21.15.31:0/4113139828 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe5d8067830 0x7fe5d8069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:17.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:17.561+0000 7fe5f899b700 1 --2- 172.21.15.31:0/4113139828 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe5d8067830 0x7fe5d8069ce0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fe5e8007fd0 tx=0x7fe5e8007d50 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:17.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:17.564+0000 7fe5f0ff9700 1 -- 172.21.15.31:0/4113139828 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe5e00514c0 con 0x7fe5f40c8200 2023-10-06T02:32:17.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:17.814+0000 7fe5f999d700 1 -- 172.21.15.31:0/4113139828 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe5f41571b0 con 0x7fe5f40c8200 2023-10-06T02:32:18.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.521+0000 7fe5f0ff9700 1 -- 172.21.15.31:0/4113139828 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v120) v1 ==== 136+0+0 (secure 0 0 0) 0x7fe5f41571b0 con 0x7fe5f40c8200 2023-10-06T02:32:18.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.524+0000 7fe5f999d700 1 -- 172.21.15.31:0/4113139828 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe5d8067830 msgr2=0x7fe5d8069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:18.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.524+0000 7fe5f999d700 1 --2- 172.21.15.31:0/4113139828 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe5d8067830 0x7fe5d8069ce0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fe5e8007fd0 tx=0x7fe5e8007d50 comp rx=0 tx=0).stop 2023-10-06T02:32:18.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.524+0000 7fe5f999d700 1 -- 172.21.15.31:0/4113139828 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5f40c8200 msgr2=0x7fe5f40c9010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:18.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.524+0000 7fe5f999d700 1 --2- 172.21.15.31:0/4113139828 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5f40c8200 0x7fe5f40c9010 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7fe5e000c810 tx=0x7fe5e0007e20 comp rx=0 tx=0).stop 2023-10-06T02:32:18.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.524+0000 7fe5f999d700 1 -- 172.21.15.31:0/4113139828 shutdown_connections 2023-10-06T02:32:18.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.524+0000 7fe5f999d700 1 --2- 172.21.15.31:0/4113139828 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5f40c85d0 0x7fe5f40c9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:18.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.524+0000 7fe5f999d700 1 --2- 172.21.15.31:0/4113139828 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe5d8067830 0x7fe5d8069ce0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:18.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.524+0000 7fe5f999d700 1 --2- 172.21.15.31:0/4113139828 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe5f40c2a90 0x7fe5f4136a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:18.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.524+0000 7fe5f999d700 1 --2- 172.21.15.31:0/4113139828 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5f40c8200 0x7fe5f40c9010 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:18.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.524+0000 7fe5f999d700 1 -- 172.21.15.31:0/4113139828 >> 172.21.15.31:0/4113139828 conn(0x7fe5f406c6a0 msgr2=0x7fe5f40ca050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:18.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.524+0000 7fe5f999d700 1 -- 172.21.15.31:0/4113139828 shutdown_connections 2023-10-06T02:32:18.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.524+0000 7fe5f999d700 1 -- 172.21.15.31:0/4113139828 wait complete. 2023-10-06T02:32:18.545 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-06T02:32:18.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.737+0000 7f5fbb59e700 1 Processor -- start 2023-10-06T02:32:18.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.738+0000 7f5fbb59e700 1 -- start start 2023-10-06T02:32:18.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.739+0000 7f5fbb59e700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f5fbc06e0a0 0x7f5fbc0c6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:18.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.739+0000 7f5fbb59e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5fbc06f3a0 0x7f5fbc14fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:18.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.739+0000 7f5fbb59e700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fbc06a490 con 0x7f5fbc06e470 2023-10-06T02:32:18.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.739+0000 7f5fbb59e700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fbc06b930 con 0x7f5fbc06f3a0 2023-10-06T02:32:18.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.739+0000 7f5fbb59e700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fbc0c7490 con 0x7f5fbc06e0a0 2023-10-06T02:32:18.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.739+0000 7f5fb9d9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5fbc06f3a0 0x7f5fbc14fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:18.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.739+0000 7f5fb9d9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5fbc06f3a0 0x7f5fbc14fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38954/0 (socket says 172.21.15.31:38954) 2023-10-06T02:32:18.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.739+0000 7f5fbad9d700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f5fbc06e470 0x7f5fbc159740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56254/0 (socket says 172.21.15.31:56254) 2023-10-06T02:32:18.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.739+0000 7f5fb9d9b700 1 -- 172.21.15.31:0/99935551 learned_addr learned my addr 172.21.15.31:0/99935551 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:18.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.739+0000 7f5fba59c700 1 --2- 172.21.15.31:0/99935551 >> v2:172.21.15.62:3300/0 conn(0x7f5fbc06e0a0 0x7f5fbc0c6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:18.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.740+0000 7f5fb9d9b700 1 -- 172.21.15.31:0/99935551 >> v2:172.21.15.62:3300/0 conn(0x7f5fbc06e0a0 msgr2=0x7f5fbc0c6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:18.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.740+0000 7f5fb9d9b700 1 --2- 172.21.15.31:0/99935551 >> v2:172.21.15.62:3300/0 conn(0x7f5fbc06e0a0 0x7f5fbc0c6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:18.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.740+0000 7f5fb9d9b700 1 -- 172.21.15.31:0/99935551 >> v1:172.21.15.31:6789/0 conn(0x7f5fbc06e470 legacy=0x7f5fbc159740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:18.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.740+0000 7f5fb9d9b700 1 -- 172.21.15.31:0/99935551 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fbc0c7770 con 0x7f5fbc06f3a0 2023-10-06T02:32:18.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.740+0000 7f5fb9d9b700 1 --2- 172.21.15.31:0/99935551 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5fbc06f3a0 0x7f5fbc14fc70 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc069230 tx=0x7f5fb000a3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fb4c6416bb59814 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:18.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.740+0000 7f5fb959a700 1 -- 172.21.15.31:0/99935551 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb0005c10 con 0x7f5fbc06f3a0 2023-10-06T02:32:18.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.740+0000 7f5fb959a700 1 -- 172.21.15.31:0/99935551 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fb0005d70 con 0x7f5fbc06f3a0 2023-10-06T02:32:18.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.740+0000 7f5fb959a700 1 -- 172.21.15.31:0/99935551 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb000ca60 con 0x7f5fbc06f3a0 2023-10-06T02:32:18.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.741+0000 7f5fbb59e700 1 -- 172.21.15.31:0/99935551 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5fbc06f3a0 msgr2=0x7f5fbc14fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:18.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.741+0000 7f5fbb59e700 1 --2- 172.21.15.31:0/99935551 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5fbc06f3a0 0x7f5fbc14fc70 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc069230 tx=0x7f5fb000a3c0 comp rx=0 tx=0).stop 2023-10-06T02:32:18.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.741+0000 7f5fbb59e700 1 -- 172.21.15.31:0/99935551 shutdown_connections 2023-10-06T02:32:18.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.741+0000 7f5fbb59e700 1 --2- 172.21.15.31:0/99935551 >> v2:172.21.15.62:3300/0 conn(0x7f5fbc06e0a0 0x7f5fbc0c6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:18.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.741+0000 7f5fbb59e700 1 --2- 172.21.15.31:0/99935551 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5fbc06f3a0 0x7f5fbc14fc70 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:18.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.741+0000 7f5fbb59e700 1 -- 172.21.15.31:0/99935551 >> 172.21.15.31:0/99935551 conn(0x7f5fbc06d170 msgr2=0x7f5fbc0cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:18.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.741+0000 7f5fbb59e700 1 -- 172.21.15.31:0/99935551 shutdown_connections 2023-10-06T02:32:18.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.741+0000 7f5fbb59e700 1 -- 172.21.15.31:0/99935551 wait complete. 2023-10-06T02:32:18.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.742+0000 7f5fbb59e700 1 Processor -- start 2023-10-06T02:32:18.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.743+0000 7f5fbb59e700 1 -- start start 2023-10-06T02:32:18.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.743+0000 7f5fbb59e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5fbc06e0a0 0x7f5fbc0aeed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:18.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.743+0000 7f5fbb59e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5fbc06e470 0x7f5fbc0c9c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:18.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.743+0000 7f5fbb59e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5fbc06f3a0 0x7f5fbc0ca190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:18.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.743+0000 7f5fbb59e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fbc0affa0 con 0x7f5fbc06e0a0 2023-10-06T02:32:18.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.743+0000 7f5fbb59e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fbc1571a0 con 0x7f5fbc06e470 2023-10-06T02:32:18.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.743+0000 7f5fbb59e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fbc156e70 con 0x7f5fbc06f3a0 2023-10-06T02:32:18.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.744+0000 7f5fba59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5fbc06e0a0 0x7f5fbc0aeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:18.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.744+0000 7f5fba59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5fbc06e0a0 0x7f5fbc0aeed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38958/0 (socket says 172.21.15.31:38958) 2023-10-06T02:32:18.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.744+0000 7f5fba59c700 1 -- 172.21.15.31:0/884001503 learned_addr learned my addr 172.21.15.31:0/884001503 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:18.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.744+0000 7f5fb9d9b700 1 --2- 172.21.15.31:0/884001503 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5fbc06e470 0x7f5fbc0c9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:18.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.744+0000 7f5fba59c700 1 -- 172.21.15.31:0/884001503 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5fbc06f3a0 msgr2=0x7f5fbc0ca190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:18.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.744+0000 7f5fbad9d700 1 --2- 172.21.15.31:0/884001503 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5fbc06f3a0 0x7f5fbc0ca190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:18.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.744+0000 7f5fba59c700 1 --2- 172.21.15.31:0/884001503 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5fbc06f3a0 0x7f5fbc0ca190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:18.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.744+0000 7f5fba59c700 1 -- 172.21.15.31:0/884001503 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5fbc06e470 msgr2=0x7f5fbc0c9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:18.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.744+0000 7f5fba59c700 1 --2- 172.21.15.31:0/884001503 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5fbc06e470 0x7f5fbc0c9c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:18.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.744+0000 7f5fba59c700 1 -- 172.21.15.31:0/884001503 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fb000a620 con 0x7f5fbc06e0a0 2023-10-06T02:32:18.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.744+0000 7f5fb9d9b700 1 --2- 172.21.15.31:0/884001503 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5fbc06e470 0x7f5fbc0c9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:32:18.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.744+0000 7f5fbad9d700 1 --2- 172.21.15.31:0/884001503 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5fbc06f3a0 0x7f5fbc0ca190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:32:18.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.744+0000 7f5fba59c700 1 --2- 172.21.15.31:0/884001503 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5fbc06e0a0 0x7f5fbc0aeed0 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f5fac00d8c0 tx=0x7f5fac008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:18.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.745+0000 7f5faa7fc700 1 -- 172.21.15.31:0/884001503 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fac00de60 con 0x7f5fbc06e0a0 2023-10-06T02:32:18.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.745+0000 7f5faa7fc700 1 -- 172.21.15.31:0/884001503 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fac00b780 con 0x7f5fbc06e0a0 2023-10-06T02:32:18.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.745+0000 7f5fbb59e700 1 -- 172.21.15.31:0/884001503 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fbc0af700 con 0x7f5fbc06e0a0 2023-10-06T02:32:18.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.745+0000 7f5faa7fc700 1 -- 172.21.15.31:0/884001503 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fac00bb20 con 0x7f5fbc06e0a0 2023-10-06T02:32:18.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.745+0000 7f5fbb59e700 1 -- 172.21.15.31:0/884001503 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fbc0afcc0 con 0x7f5fbc06e0a0 2023-10-06T02:32:18.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.746+0000 7f5fbb59e700 1 -- 172.21.15.31:0/884001503 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fbc04c3c0 con 0x7f5fbc06e0a0 2023-10-06T02:32:18.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.747+0000 7f5faa7fc700 1 -- 172.21.15.31:0/884001503 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85556+0+0 (secure 0 0 0) 0x7f5fac01e460 con 0x7f5fbc06e0a0 2023-10-06T02:32:18.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.748+0000 7f5faa7fc700 1 --2- 172.21.15.31:0/884001503 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5f9c067830 0x7f5f9c069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:18.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.748+0000 7f5faa7fc700 1 -- 172.21.15.31:0/884001503 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f5fac083720 con 0x7f5fbc06e0a0 2023-10-06T02:32:18.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.749+0000 7f5fb9d9b700 1 --2- 172.21.15.31:0/884001503 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5f9c067830 0x7f5f9c069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:18.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.749+0000 7f5fb9d9b700 1 --2- 172.21.15.31:0/884001503 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5f9c067830 0x7f5f9c069ce0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc149770 tx=0x7f5fb0019000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:18.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.752+0000 7f5faa7fc700 1 -- 172.21.15.31:0/884001503 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5fac052170 con 0x7f5fbc06e0a0 2023-10-06T02:32:18.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.982+0000 7f5fbb59e700 1 -- 172.21.15.31:0/884001503 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f5fbc1572e0 con 0x7f5f9c067830 2023-10-06T02:32:18.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.984+0000 7f5faa7fc700 1 -- 172.21.15.31:0/884001503 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f5fb000c760 con 0x7f5f9c067830 2023-10-06T02:32:18.984 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2023-10-06T02:32:18.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T02:32:18.985 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephfs" 2023-10-06T02:32:18.985 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T02:32:18.985 INFO:teuthology.orchestra.run.smithi031.stdout:] 2023-10-06T02:32:18.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.986+0000 7f5fbb59e700 1 -- 172.21.15.31:0/884001503 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5f9c067830 msgr2=0x7f5f9c069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:18.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.986+0000 7f5fbb59e700 1 --2- 172.21.15.31:0/884001503 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5f9c067830 0x7f5f9c069ce0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc149770 tx=0x7f5fb0019000 comp rx=0 tx=0).stop 2023-10-06T02:32:18.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.986+0000 7f5fbb59e700 1 -- 172.21.15.31:0/884001503 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5fbc06e0a0 msgr2=0x7f5fbc0aeed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:18.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.986+0000 7f5fbb59e700 1 --2- 172.21.15.31:0/884001503 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5fbc06e0a0 0x7f5fbc0aeed0 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f5fac00d8c0 tx=0x7f5fac008040 comp rx=0 tx=0).stop 2023-10-06T02:32:18.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.986+0000 7f5fbb59e700 1 -- 172.21.15.31:0/884001503 shutdown_connections 2023-10-06T02:32:18.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.986+0000 7f5fbb59e700 1 --2- 172.21.15.31:0/884001503 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5f9c067830 0x7f5f9c069ce0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:18.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.986+0000 7f5fbb59e700 1 --2- 172.21.15.31:0/884001503 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5fbc06f3a0 0x7f5fbc0ca190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:18.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.986+0000 7f5fbb59e700 1 --2- 172.21.15.31:0/884001503 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5fbc06e0a0 0x7f5fbc0aeed0 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:18.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.986+0000 7f5fbb59e700 1 --2- 172.21.15.31:0/884001503 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5fbc06e470 0x7f5fbc0c9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:18.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.986+0000 7f5fbb59e700 1 -- 172.21.15.31:0/884001503 >> 172.21.15.31:0/884001503 conn(0x7f5fbc06d170 msgr2=0x7f5fbc0c8b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:18.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.987+0000 7f5fbb59e700 1 -- 172.21.15.31:0/884001503 shutdown_connections 2023-10-06T02:32:18.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:18.987+0000 7f5fbb59e700 1 -- 172.21.15.31:0/884001503 wait complete. 2023-10-06T02:32:19.005 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T02:32:19.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:19.195+0000 7fc187fff700 1 Processor -- start 2023-10-06T02:32:19.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:19.195+0000 7fc187fff700 1 -- start start 2023-10-06T02:32:19.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:19.196+0000 7fc187fff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc188156670 0x7fc188067f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:19.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:19.196+0000 7fc187fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc18806ad00 0x7fc18806b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:19.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:19.196+0000 7fc187fff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1880c8060 con 0x7fc188069a00 2023-10-06T02:32:19.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:19.196+0000 7fc187fff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc188156a40 con 0x7fc188156670 2023-10-06T02:32:19.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:19.196+0000 7fc187fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc18806b690 con 0x7fc18806ad00 2023-10-06T02:32:19.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:19.197+0000 7fc1867fc700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc18806ad00 0x7fc18806b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:19.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:19.197+0000 7fc186ffd700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc188156670 0x7fc188067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:19.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:19.197+0000 7fc1877fe700 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fc188069a00 0x7fc188069e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:36454/0 (socket says 172.21.15.31:36454) 2023-10-06T02:32:19.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:19.197+0000 7fc1877fe700 1 -- 172.21.15.31:0/1468955418 learned_addr learned my addr 172.21.15.31:0/1468955418 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:32:19.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:19.197+0000 7fc1867fc700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc18806ad00 0x7fc18806b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:51832/0 (socket says 172.21.15.31:51832) 2023-10-06T02:32:19.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:19.197+0000 7fc1867fc700 1 -- 172.21.15.31:0/1468955418 >> v2:172.21.15.62:3300/0 conn(0x7fc188156670 msgr2=0x7fc188067f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:19.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:19.197+0000 7fc1867fc700 1 --2- 172.21.15.31:0/1468955418 >> v2:172.21.15.62:3300/0 conn(0x7fc188156670 0x7fc188067f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:19.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:19.197+0000 7fc185ffb700 1 -- 172.21.15.31:0/1468955418 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3214171080 0 0) 0x7fc1880c8060 con 0x7fc188069a00 2023-10-06T02:32:19.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:19.197+0000 7fc1867fc700 1 -- 172.21.15.31:0/1468955418 >> v1:172.21.15.62:6789/0 conn(0x7fc188069a00 legacy=0x7fc188069e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:19.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:19.197+0000 7fc1867fc700 1 -- 172.21.15.31:0/1468955418 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1880abaa0 con 0x7fc18806ad00 2023-10-06T02:32:19.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:19.198+0000 7fc1867fc700 1 --2- 172.21.15.31:0/1468955418 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc18806ad00 0x7fc18806b150 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc17c0098e0 tx=0x7fc17c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17112202ef5bb72e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:19.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:19.198+0000 7fc185ffb700 1 -- 172.21.15.31:0/1468955418 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17c00b6d0 con 0x7fc18806ad00 2023-10-06T02:32:19.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:19.198+0000 7fc185ffb700 1 -- 172.21.15.31:0/1468955418 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc17c00b830 con 0x7fc18806ad00 2023-10-06T02:32:19.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:19.198+0000 7fc185ffb700 1 -- 172.21.15.31:0/1468955418 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17c00bbd0 con 0x7fc18806ad00 2023-10-06T02:32:19.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:19.198+0000 7fc187fff700 1 -- 172.21.15.31:0/1468955418 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc18806ad00 msgr2=0x7fc18806b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:19.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:19.198+0000 7fc187fff700 1 --2- 172.21.15.31:0/1468955418 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc18806ad00 0x7fc18806b150 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc17c0098e0 tx=0x7fc17c0091a0 comp rx=0 tx=0).stop 2023-10-06T02:32:19.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:19.199+0000 7fc187fff700 1 -- 172.21.15.31:0/1468955418 shutdown_connections 2023-10-06T02:32:19.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:19.199+0000 7fc187fff700 1 --2- 172.21.15.31:0/1468955418 >> v2:172.21.15.62:3300/0 conn(0x7fc188156670 0x7fc188067f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:19.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:19.199+0000 7fc187fff700 1 --2- 172.21.15.31:0/1468955418 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc18806ad00 0x7fc18806b150 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:19.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:19.199+0000 7fc187fff700 1 -- 172.21.15.31:0/1468955418 >> 172.21.15.31:0/1468955418 conn(0x7fc1880c9600 msgr2=0x7fc1880cc0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:19.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:19.199+0000 7fc187fff700 1 -- 172.21.15.31:0/1468955418 shutdown_connections 2023-10-06T02:32:19.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:19.199+0000 7fc187fff700 1 -- 172.21.15.31:0/1468955418 wait complete. 2023-10-06T02:32:19.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:19.200+0000 7fc187fff700 1 Processor -- start 2023-10-06T02:32:19.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:19.200+0000 7fc187fff700 1 -- start start 2023-10-06T02:32:19.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:19.201+0000 7fc187fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc188069a00 0x7fc188150df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:19.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:19.201+0000 7fc187fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc18806ad00 0x7fc188168570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:19.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:19.201+0000 7fc187fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc188156670 0x7fc188154db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:19.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:19.201+0000 7fc187fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc18814cb20 con 0x7fc188069a00 2023-10-06T02:32:19.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:19.201+0000 7fc187fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc188151330 con 0x7fc188156670 2023-10-06T02:32:19.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:19.201+0000 7fc187fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc18814bf10 con 0x7fc18806ad00 2023-10-06T02:32:19.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:19.201+0000 7fc186ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc188069a00 0x7fc188150df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:19.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:19.201+0000 7fc186ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc188069a00 0x7fc188150df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38962/0 (socket says 172.21.15.31:38962) 2023-10-06T02:32:19.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:19.202+0000 7fc186ffd700 1 -- 172.21.15.31:0/1376996046 learned_addr learned my addr 172.21.15.31:0/1376996046 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:19.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:19.202+0000 7fc1867fc700 1 --2- 172.21.15.31:0/1376996046 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc18806ad00 0x7fc188168570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:19.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:19.202+0000 7fc1877fe700 1 --2- 172.21.15.31:0/1376996046 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc188156670 0x7fc188154db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:19.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:19.202+0000 7fc186ffd700 1 -- 172.21.15.31:0/1376996046 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc18806ad00 msgr2=0x7fc188168570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:19.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:19.202+0000 7fc186ffd700 1 --2- 172.21.15.31:0/1376996046 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc18806ad00 0x7fc188168570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:19.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:19.202+0000 7fc186ffd700 1 -- 172.21.15.31:0/1376996046 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc188156670 msgr2=0x7fc188154db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:19.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:19.202+0000 7fc186ffd700 1 --2- 172.21.15.31:0/1376996046 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc188156670 0x7fc188154db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:19.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:19.202+0000 7fc186ffd700 1 -- 172.21.15.31:0/1376996046 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc17c009420 con 0x7fc188069a00 2023-10-06T02:32:19.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:19.202+0000 7fc186ffd700 1 --2- 172.21.15.31:0/1376996046 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc188069a00 0x7fc188150df0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7fc17800ec80 tx=0x7fc1780091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:19.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:19.202+0000 7fc16effd700 1 -- 172.21.15.31:0/1376996046 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17800c850 con 0x7fc188069a00 2023-10-06T02:32:19.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:19.202+0000 7fc16effd700 1 -- 172.21.15.31:0/1376996046 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc17800c9b0 con 0x7fc188069a00 2023-10-06T02:32:19.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:19.202+0000 7fc16effd700 1 -- 172.21.15.31:0/1376996046 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17800cd50 con 0x7fc188069a00 2023-10-06T02:32:19.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:19.202+0000 7fc187fff700 1 -- 172.21.15.31:0/1376996046 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc18814c1f0 con 0x7fc188069a00 2023-10-06T02:32:19.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:19.203+0000 7fc187fff700 1 -- 172.21.15.31:0/1376996046 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc18814c7b0 con 0x7fc188069a00 2023-10-06T02:32:19.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:19.203+0000 7fc187fff700 1 -- 172.21.15.31:0/1376996046 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc188166a60 con 0x7fc188069a00 2023-10-06T02:32:19.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:19.205+0000 7fc16effd700 1 -- 172.21.15.31:0/1376996046 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85556+0+0 (secure 0 0 0) 0x7fc178013680 con 0x7fc188069a00 2023-10-06T02:32:19.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:19.206+0000 7fc16effd700 1 --2- 172.21.15.31:0/1376996046 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc174067830 0x7fc174069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:19.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:19.206+0000 7fc16effd700 1 -- 172.21.15.31:0/1376996046 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fc1780836b0 con 0x7fc188069a00 2023-10-06T02:32:19.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:19.206+0000 7fc1867fc700 1 --2- 172.21.15.31:0/1376996046 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc174067830 0x7fc174069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:19.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:19.207+0000 7fc1867fc700 1 --2- 172.21.15.31:0/1376996046 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc174067830 0x7fc174069ce0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fc17c0093f0 tx=0x7fc17c017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:19.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:19.209+0000 7fc16effd700 1 -- 172.21.15.31:0/1376996046 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc178052100 con 0x7fc188069a00 2023-10-06T02:32:19.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:19.411+0000 7fc187fff700 1 -- 172.21.15.31:0/1376996046 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fc1880c7450 con 0x7fc188069a00 2023-10-06T02:32:19.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:19.412+0000 7fc16effd700 1 -- 172.21.15.31:0/1376996046 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v30)=0 v30) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc178055720 con 0x7fc188069a00 2023-10-06T02:32:19.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:19.414+0000 7fc187fff700 1 -- 172.21.15.31:0/1376996046 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc174067830 msgr2=0x7fc174069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:19.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:19.414+0000 7fc187fff700 1 --2- 172.21.15.31:0/1376996046 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc174067830 0x7fc174069ce0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fc17c0093f0 tx=0x7fc17c017000 comp rx=0 tx=0).stop 2023-10-06T02:32:19.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:19.414+0000 7fc187fff700 1 -- 172.21.15.31:0/1376996046 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc188069a00 msgr2=0x7fc188150df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:19.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:19.414+0000 7fc187fff700 1 --2- 172.21.15.31:0/1376996046 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc188069a00 0x7fc188150df0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7fc17800ec80 tx=0x7fc1780091a0 comp rx=0 tx=0).stop 2023-10-06T02:32:19.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:19.414+0000 7fc187fff700 1 -- 172.21.15.31:0/1376996046 shutdown_connections 2023-10-06T02:32:19.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:19.414+0000 7fc187fff700 1 --2- 172.21.15.31:0/1376996046 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc188156670 0x7fc188154db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:19.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:19.414+0000 7fc187fff700 1 --2- 172.21.15.31:0/1376996046 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc188069a00 0x7fc188150df0 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:19.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:19.414+0000 7fc187fff700 1 --2- 172.21.15.31:0/1376996046 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc174067830 0x7fc174069ce0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:19.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:19.414+0000 7fc187fff700 1 --2- 172.21.15.31:0/1376996046 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc18806ad00 0x7fc188168570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:19.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:19.414+0000 7fc187fff700 1 -- 172.21.15.31:0/1376996046 >> 172.21.15.31:0/1376996046 conn(0x7fc1880c9600 msgr2=0x7fc18806e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:19.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:19.414+0000 7fc187fff700 1 -- 172.21.15.31:0/1376996046 shutdown_connections 2023-10-06T02:32:19.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:19.415+0000 7fc187fff700 1 -- 172.21.15.31:0/1376996046 wait complete. 2023-10-06T02:32:19.433 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:32:19.433 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/group_0000000000082492/subvolume_0000000000634507) 2023-10-06T02:32:19.503 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:32:19.503 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2023-10-06T02:32:19.542 INFO:teuthology.orchestra.run.smithi031.stdout:cephfs_data 2023-10-06T02:32:19.543 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:32:19.543 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data ./volumes/group_0000000000082492/subvolume_0000000000634507) 2023-10-06T02:32:19.650 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T02:32:19.650 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/home/ubuntu/cephtest/mnt.0/./volumes/group_0000000000082492/subvolume_0000000000634507/.meta 2023-10-06T02:32:19.716 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000634507 group_0000000000082492 2023-10-06T02:32:20.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.311+0000 7fe63ffff700 1 Processor -- start 2023-10-06T02:32:20.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.311+0000 7fe63ffff700 1 -- start start 2023-10-06T02:32:20.588 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:32:20.573+0000 7f656537d700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:32:20.588 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:32:20.573+0000 7f656537d700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:32:20.588 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:32:20.573+0000 7f656537d700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:32:20.588 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:32:20.573+0000 7f656537d700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:32:20.588 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:32:20.573+0000 7f656537d700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:32:20.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.312+0000 7fe63ffff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe6400b24f0 0x7fe6400b0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:20.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.312+0000 7fe63ffff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe640159800 0x7fe6400b4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:20.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.312+0000 7fe63ffff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6400b0c80 con 0x7fe6400b1d40 2023-10-06T02:32:20.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.312+0000 7fe63ffff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6400d10d0 con 0x7fe640159800 2023-10-06T02:32:20.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.312+0000 7fe63ffff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6400b0f60 con 0x7fe6400b24f0 2023-10-06T02:32:20.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.312+0000 7fe63f7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe640159800 0x7fe6400b4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:20.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.312+0000 7fe63f7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe640159800 0x7fe6400b4ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38970/0 (socket says 172.21.15.31:38970) 2023-10-06T02:32:20.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.312+0000 7fe63f7fe700 1 -- 172.21.15.31:0/3007419908 learned_addr learned my addr 172.21.15.31:0/3007419908 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:20.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.313+0000 7fe63effd700 1 --2- 172.21.15.31:0/3007419908 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe6400b24f0 0x7fe6400b0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:20.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.313+0000 7fe63f7fe700 1 -- 172.21.15.31:0/3007419908 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe6400b24f0 msgr2=0x7fe6400b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:20.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.313+0000 7fe63f7fe700 1 --2- 172.21.15.31:0/3007419908 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe6400b24f0 0x7fe6400b0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:20.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.313+0000 7fe63f7fe700 1 -- 172.21.15.31:0/3007419908 >> v1:172.21.15.62:6789/0 conn(0x7fe6400b1d40 legacy=0x7fe6401590f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:20.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.313+0000 7fe63f7fe700 1 -- 172.21.15.31:0/3007419908 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6400b1240 con 0x7fe640159800 2023-10-06T02:32:20.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.313+0000 7fe63f7fe700 1 --2- 172.21.15.31:0/3007419908 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe640159800 0x7fe6400b4ae0 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7fe638009f20 tx=0x7fe638009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d38f0621d77e9553 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:20.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.313+0000 7fe63dffb700 1 -- 172.21.15.31:0/3007419908 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe63800e070 con 0x7fe640159800 2023-10-06T02:32:20.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.313+0000 7fe63dffb700 1 -- 172.21.15.31:0/3007419908 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe63800ba80 con 0x7fe640159800 2023-10-06T02:32:20.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.313+0000 7fe63dffb700 1 -- 172.21.15.31:0/3007419908 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6380092e0 con 0x7fe640159800 2023-10-06T02:32:20.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.314+0000 7fe63ffff700 1 -- 172.21.15.31:0/3007419908 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe640159800 msgr2=0x7fe6400b4ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:20.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.314+0000 7fe63ffff700 1 --2- 172.21.15.31:0/3007419908 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe640159800 0x7fe6400b4ae0 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7fe638009f20 tx=0x7fe638009840 comp rx=0 tx=0).stop 2023-10-06T02:32:20.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.314+0000 7fe63ffff700 1 -- 172.21.15.31:0/3007419908 shutdown_connections 2023-10-06T02:32:20.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.314+0000 7fe63ffff700 1 --2- 172.21.15.31:0/3007419908 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe640159800 0x7fe6400b4ae0 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:20.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.314+0000 7fe63ffff700 1 --2- 172.21.15.31:0/3007419908 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe6400b24f0 0x7fe6400b0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:20.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.314+0000 7fe63ffff700 1 -- 172.21.15.31:0/3007419908 >> 172.21.15.31:0/3007419908 conn(0x7fe6400703a0 msgr2=0x7fe6400707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:20.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.314+0000 7fe63ffff700 1 -- 172.21.15.31:0/3007419908 shutdown_connections 2023-10-06T02:32:20.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.314+0000 7fe63ffff700 1 -- 172.21.15.31:0/3007419908 wait complete. 2023-10-06T02:32:20.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.316+0000 7fe63ffff700 1 Processor -- start 2023-10-06T02:32:20.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.316+0000 7fe63ffff700 1 -- start start 2023-10-06T02:32:20.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.317+0000 7fe63ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6400b1d40 0x7fe64015eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:20.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.317+0000 7fe63ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe6400b24f0 0x7fe6400cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:20.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.317+0000 7fe63ffff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe640159800 0x7fe640169770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:20.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.317+0000 7fe63ffff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe64016ce70 con 0x7fe6400b24f0 2023-10-06T02:32:20.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.317+0000 7fe63ffff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe64015f000 con 0x7fe6400b1d40 2023-10-06T02:32:20.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.317+0000 7fe63ffff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe64015d220 con 0x7fe640159800 2023-10-06T02:32:20.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.317+0000 7fe63e7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe6400b24f0 0x7fe6400cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:20.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.317+0000 7fe63e7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe6400b24f0 0x7fe6400cfb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38986/0 (socket says 172.21.15.31:38986) 2023-10-06T02:32:20.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.317+0000 7fe63e7fc700 1 -- 172.21.15.31:0/2156168000 learned_addr learned my addr 172.21.15.31:0/2156168000 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:20.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.317+0000 7fe63e7fc700 1 -- 172.21.15.31:0/2156168000 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe640159800 msgr2=0x7fe640169770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:20.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.317+0000 7fe63e7fc700 1 --2- 172.21.15.31:0/2156168000 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe640159800 0x7fe640169770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:20.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.317+0000 7fe63e7fc700 1 -- 172.21.15.31:0/2156168000 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6400b1d40 msgr2=0x7fe64015eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:20.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.317+0000 7fe63e7fc700 1 --2- 172.21.15.31:0/2156168000 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6400b1d40 0x7fe64015eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:20.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.317+0000 7fe63e7fc700 1 -- 172.21.15.31:0/2156168000 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe638009ae0 con 0x7fe6400b24f0 2023-10-06T02:32:20.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.317+0000 7fe63e7fc700 1 --2- 172.21.15.31:0/2156168000 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe6400b24f0 0x7fe6400cfb80 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7fe63400c590 tx=0x7fe63400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:20.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.318+0000 7fe626ffd700 1 -- 172.21.15.31:0/2156168000 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe634011590 con 0x7fe6400b24f0 2023-10-06T02:32:20.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.318+0000 7fe626ffd700 1 -- 172.21.15.31:0/2156168000 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6340116f0 con 0x7fe6400b24f0 2023-10-06T02:32:20.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.318+0000 7fe626ffd700 1 -- 172.21.15.31:0/2156168000 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe634011ab0 con 0x7fe6400b24f0 2023-10-06T02:32:20.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.318+0000 7fe63ffff700 1 -- 172.21.15.31:0/2156168000 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe64015d500 con 0x7fe6400b24f0 2023-10-06T02:32:20.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.318+0000 7fe63ffff700 1 -- 172.21.15.31:0/2156168000 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe64015dac0 con 0x7fe6400b24f0 2023-10-06T02:32:20.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.319+0000 7fe63ffff700 1 -- 172.21.15.31:0/2156168000 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe64004c3c0 con 0x7fe6400b24f0 2023-10-06T02:32:20.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.320+0000 7fe626ffd700 1 -- 172.21.15.31:0/2156168000 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85556+0+0 (secure 0 0 0) 0x7fe63401b3b0 con 0x7fe6400b24f0 2023-10-06T02:32:20.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.322+0000 7fe626ffd700 1 --2- 172.21.15.31:0/2156168000 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe62c067830 0x7fe62c069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:20.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.322+0000 7fe626ffd700 1 -- 172.21.15.31:0/2156168000 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fe6340823f0 con 0x7fe6400b24f0 2023-10-06T02:32:20.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.322+0000 7fe63effd700 1 --2- 172.21.15.31:0/2156168000 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe62c067830 0x7fe62c069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:20.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.323+0000 7fe63effd700 1 --2- 172.21.15.31:0/2156168000 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe62c067830 0x7fe62c069ce0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fe630009a60 tx=0x7fe630008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:20.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.326+0000 7fe626ffd700 1 -- 172.21.15.31:0/2156168000 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe63408a020 con 0x7fe6400b24f0 2023-10-06T02:32:20.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.565+0000 7fe63ffff700 1 -- 172.21.15.31:0/2156168000 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000634507", "group_name": "group_0000000000082492", "target": ["mon-mgr", ""]}) v1 -- 0x7fe64016cfb0 con 0x7fe62c067830 2023-10-06T02:32:20.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.617+0000 7fe626ffd700 1 -- 172.21.15.31:0/2156168000 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+59 (secure 0 0 0) 0x7fe64016cfb0 con 0x7fe62c067830 2023-10-06T02:32:20.617 INFO:teuthology.orchestra.run.smithi031.stdout:/volumes/group_0000000000082492/subvolume_0000000000634507 2023-10-06T02:32:20.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.619+0000 7fe63ffff700 1 -- 172.21.15.31:0/2156168000 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe62c067830 msgr2=0x7fe62c069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:20.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.619+0000 7fe63ffff700 1 --2- 172.21.15.31:0/2156168000 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe62c067830 0x7fe62c069ce0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fe630009a60 tx=0x7fe630008040 comp rx=0 tx=0).stop 2023-10-06T02:32:20.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.619+0000 7fe63ffff700 1 -- 172.21.15.31:0/2156168000 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe6400b24f0 msgr2=0x7fe6400cfb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:20.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.619+0000 7fe63ffff700 1 --2- 172.21.15.31:0/2156168000 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe6400b24f0 0x7fe6400cfb80 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7fe63400c590 tx=0x7fe63400c950 comp rx=0 tx=0).stop 2023-10-06T02:32:20.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.619+0000 7fe63ffff700 1 -- 172.21.15.31:0/2156168000 shutdown_connections 2023-10-06T02:32:20.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.619+0000 7fe63ffff700 1 --2- 172.21.15.31:0/2156168000 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe62c067830 0x7fe62c069ce0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:20.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.619+0000 7fe63ffff700 1 --2- 172.21.15.31:0/2156168000 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe640159800 0x7fe640169770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:20.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.619+0000 7fe63ffff700 1 --2- 172.21.15.31:0/2156168000 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6400b1d40 0x7fe64015eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:20.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.619+0000 7fe63ffff700 1 --2- 172.21.15.31:0/2156168000 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe6400b24f0 0x7fe6400cfb80 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:20.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.619+0000 7fe63ffff700 1 -- 172.21.15.31:0/2156168000 >> 172.21.15.31:0/2156168000 conn(0x7fe6400703a0 msgr2=0x7fe6400ce860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:20.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.619+0000 7fe63ffff700 1 -- 172.21.15.31:0/2156168000 shutdown_connections 2023-10-06T02:32:20.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.619+0000 7fe63ffff700 1 -- 172.21.15.31:0/2156168000 wait complete. 2023-10-06T02:32:20.638 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000634507 group_0000000000082492 2023-10-06T02:32:20.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.829+0000 7fd838f9d700 1 Processor -- start 2023-10-06T02:32:20.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.829+0000 7fd838f9d700 1 -- start start 2023-10-06T02:32:21.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.830+0000 7fd838f9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd834157380 0x7fd8341585f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:21.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.830+0000 7fd838f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd834157750 0x7fd8340ae080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:21.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.830+0000 7fd838f9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8340c8680 con 0x7fd83415b1a0 2023-10-06T02:32:21.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.830+0000 7fd838f9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd83406a7f0 con 0x7fd834157380 2023-10-06T02:32:21.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.830+0000 7fd838f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8340c7270 con 0x7fd834157750 2023-10-06T02:32:21.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.831+0000 7fd832ffd700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fd83415b1a0 0x7fd8340c6b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56266/0 (socket says 172.21.15.31:56266) 2023-10-06T02:32:21.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.831+0000 7fd832ffd700 1 -- 172.21.15.31:0/109431358 learned_addr learned my addr 172.21.15.31:0/109431358 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:32:21.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.831+0000 7fd8337fe700 1 --2- 172.21.15.31:0/109431358 >> v2:172.21.15.62:3300/0 conn(0x7fd834157380 0x7fd8341585f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:21.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.831+0000 7fd833fff700 1 --2- 172.21.15.31:0/109431358 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd834157750 0x7fd8340ae080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:21.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.831+0000 7fd8327fc700 1 -- 172.21.15.31:0/109431358 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3456810268 0 0) 0x7fd8340c8680 con 0x7fd83415b1a0 2023-10-06T02:32:21.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.831+0000 7fd8327fc700 1 -- 172.21.15.31:0/109431358 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd81c003570 con 0x7fd83415b1a0 2023-10-06T02:32:21.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.831+0000 7fd8327fc700 1 -- 172.21.15.31:0/109431358 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2153961893 0 0) 0x7fd81c003570 con 0x7fd83415b1a0 2023-10-06T02:32:21.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.831+0000 7fd8327fc700 1 -- 172.21.15.31:0/109431358 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd834157750 msgr2=0x7fd8340ae080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:21.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.831+0000 7fd8327fc700 1 --2- 172.21.15.31:0/109431358 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd834157750 0x7fd8340ae080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:21.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.831+0000 7fd8327fc700 1 -- 172.21.15.31:0/109431358 >> v2:172.21.15.62:3300/0 conn(0x7fd834157380 msgr2=0x7fd8341585f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:21.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.831+0000 7fd8327fc700 1 --2- 172.21.15.31:0/109431358 >> v2:172.21.15.62:3300/0 conn(0x7fd834157380 0x7fd8341585f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:21.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.831+0000 7fd8327fc700 1 -- 172.21.15.31:0/109431358 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd834158b30 con 0x7fd83415b1a0 2023-10-06T02:32:21.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.831+0000 7fd8327fc700 1 -- 172.21.15.31:0/109431358 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7fd828001470 con 0x7fd83415b1a0 2023-10-06T02:32:21.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.831+0000 7fd8337fe700 1 --2- 172.21.15.31:0/109431358 >> v2:172.21.15.62:3300/0 conn(0x7fd834157380 0x7fd8341585f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:32:21.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.832+0000 7fd8327fc700 1 -- 172.21.15.31:0/109431358 >> v1:172.21.15.31:6789/0 conn(0x7fd83415b1a0 legacy=0x7fd8340c6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:21.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.832+0000 7fd8327fc700 1 --2- 172.21.15.31:0/109431358 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd834157750 0x7fd81c003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:21.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.832+0000 7fd8327fc700 1 --2- 172.21.15.31:0/109431358 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd81c004010 0x7fd81c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:21.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.832+0000 7fd8327fc700 1 --2- 172.21.15.31:0/109431358 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd81c004d50 0x7fd81c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:21.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.832+0000 7fd8327fc700 1 -- 172.21.15.31:0/109431358 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd83406a7f0 con 0x7fd81c004010 2023-10-06T02:32:21.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.832+0000 7fd8327fc700 1 -- 172.21.15.31:0/109431358 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8340c7270 con 0x7fd81c004d50 2023-10-06T02:32:21.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.832+0000 7fd8327fc700 1 -- 172.21.15.31:0/109431358 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd81c00bab0 con 0x7fd834157750 2023-10-06T02:32:21.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.832+0000 7fd832ffd700 1 --2- 172.21.15.31:0/109431358 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd81c004010 0x7fd81c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:21.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.832+0000 7fd833fff700 1 --2- 172.21.15.31:0/109431358 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd81c004d50 0x7fd81c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:21.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.832+0000 7fd8337fe700 1 --2- 172.21.15.31:0/109431358 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd834157750 0x7fd81c003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:21.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.832+0000 7fd832ffd700 1 -- 172.21.15.31:0/109431358 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd834157750 msgr2=0x7fd81c003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:21.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.832+0000 7fd832ffd700 1 --2- 172.21.15.31:0/109431358 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd834157750 0x7fd81c003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:21.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.832+0000 7fd832ffd700 1 -- 172.21.15.31:0/109431358 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd81c004d50 msgr2=0x7fd81c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:21.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.832+0000 7fd832ffd700 1 --2- 172.21.15.31:0/109431358 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd81c004d50 0x7fd81c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:21.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.832+0000 7fd832ffd700 1 -- 172.21.15.31:0/109431358 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd81c003570 con 0x7fd81c004010 2023-10-06T02:32:21.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.832+0000 7fd833fff700 1 --2- 172.21.15.31:0/109431358 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd81c004d50 0x7fd81c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:32:21.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.833+0000 7fd832ffd700 1 --2- 172.21.15.31:0/109431358 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd81c004010 0x7fd81c004810 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7fd82800c3c0 tx=0x7fd828001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4194372c4d14c095 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:21.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.833+0000 7fd8327fc700 1 -- 172.21.15.31:0/109431358 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82800cc90 con 0x7fd81c004010 2023-10-06T02:32:21.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.833+0000 7fd8327fc700 1 -- 172.21.15.31:0/109431358 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd82800cdf0 con 0x7fd81c004010 2023-10-06T02:32:21.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.833+0000 7fd8327fc700 1 -- 172.21.15.31:0/109431358 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd828005b50 con 0x7fd81c004010 2023-10-06T02:32:21.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.833+0000 7fd838f9d700 1 -- 172.21.15.31:0/109431358 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd81c004010 msgr2=0x7fd81c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:21.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.833+0000 7fd838f9d700 1 --2- 172.21.15.31:0/109431358 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd81c004010 0x7fd81c004810 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7fd82800c3c0 tx=0x7fd828001e50 comp rx=0 tx=0).stop 2023-10-06T02:32:21.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.833+0000 7fd833fff700 1 -- 172.21.15.31:0/109431358 reap_dead start 2023-10-06T02:32:21.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.833+0000 7fd838f9d700 1 -- 172.21.15.31:0/109431358 shutdown_connections 2023-10-06T02:32:21.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.833+0000 7fd838f9d700 1 -- 172.21.15.31:0/109431358 >> 172.21.15.31:0/109431358 conn(0x7fd83406c6a0 msgr2=0x7fd83406caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:21.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.834+0000 7fd838f9d700 1 -- 172.21.15.31:0/109431358 shutdown_connections 2023-10-06T02:32:21.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.834+0000 7fd838f9d700 1 -- 172.21.15.31:0/109431358 wait complete. 2023-10-06T02:32:21.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.835+0000 7fd838f9d700 1 Processor -- start 2023-10-06T02:32:21.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.835+0000 7fd838f9d700 1 -- start start 2023-10-06T02:32:21.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.836+0000 7fd838f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd81c004010 0x7fd834149d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:21.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.836+0000 7fd838f9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd834159cf0 0x7fd83415a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:21.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.836+0000 7fd838f9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8340ca050 0x7fd8340ca4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:21.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.836+0000 7fd838f9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd83414a290 con 0x7fd834159cf0 2023-10-06T02:32:21.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.836+0000 7fd838f9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd83415a6a0 con 0x7fd8340ca050 2023-10-06T02:32:21.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.836+0000 7fd838f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8340cad40 con 0x7fd81c004010 2023-10-06T02:32:21.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.836+0000 7fd832ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd834159cf0 0x7fd83415a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:21.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.836+0000 7fd832ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd834159cf0 0x7fd83415a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39004/0 (socket says 172.21.15.31:39004) 2023-10-06T02:32:21.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.836+0000 7fd832ffd700 1 -- 172.21.15.31:0/3692345005 learned_addr learned my addr 172.21.15.31:0/3692345005 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:21.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.836+0000 7fd8337fe700 1 --2- 172.21.15.31:0/3692345005 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd81c004010 0x7fd834149d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:21.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.836+0000 7fd833fff700 1 --2- 172.21.15.31:0/3692345005 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8340ca050 0x7fd8340ca4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:21.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.837+0000 7fd832ffd700 1 -- 172.21.15.31:0/3692345005 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd81c004010 msgr2=0x7fd834149d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:21.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.837+0000 7fd832ffd700 1 --2- 172.21.15.31:0/3692345005 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd81c004010 0x7fd834149d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:21.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.837+0000 7fd832ffd700 1 -- 172.21.15.31:0/3692345005 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8340ca050 msgr2=0x7fd8340ca4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:21.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.837+0000 7fd832ffd700 1 --2- 172.21.15.31:0/3692345005 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8340ca050 0x7fd8340ca4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:21.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.837+0000 7fd832ffd700 1 -- 172.21.15.31:0/3692345005 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8340cb020 con 0x7fd834159cf0 2023-10-06T02:32:21.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.837+0000 7fd833fff700 1 --2- 172.21.15.31:0/3692345005 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8340ca050 0x7fd8340ca4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:32:21.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.837+0000 7fd832ffd700 1 --2- 172.21.15.31:0/3692345005 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd834159cf0 0x7fd83415a160 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7fd828005c80 tx=0x7fd828015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:21.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.837+0000 7fd81b7fe700 1 -- 172.21.15.31:0/3692345005 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82800c9e0 con 0x7fd834159cf0 2023-10-06T02:32:21.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.837+0000 7fd81b7fe700 1 -- 172.21.15.31:0/3692345005 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8280072e0 con 0x7fd834159cf0 2023-10-06T02:32:21.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.837+0000 7fd81b7fe700 1 -- 172.21.15.31:0/3692345005 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8280076c0 con 0x7fd834159cf0 2023-10-06T02:32:21.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.837+0000 7fd838f9d700 1 -- 172.21.15.31:0/3692345005 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8340cb300 con 0x7fd834159cf0 2023-10-06T02:32:21.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.837+0000 7fd838f9d700 1 -- 172.21.15.31:0/3692345005 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8340cb8c0 con 0x7fd834159cf0 2023-10-06T02:32:21.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.838+0000 7fd838f9d700 1 -- 172.21.15.31:0/3692345005 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8340adc40 con 0x7fd834159cf0 2023-10-06T02:32:21.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.839+0000 7fd81b7fe700 1 -- 172.21.15.31:0/3692345005 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85556+0+0 (secure 0 0 0) 0x7fd828019030 con 0x7fd834159cf0 2023-10-06T02:32:21.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.840+0000 7fd81b7fe700 1 --2- 172.21.15.31:0/3692345005 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd820067830 0x7fd820069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:21.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.840+0000 7fd81b7fe700 1 -- 172.21.15.31:0/3692345005 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fd828083ff0 con 0x7fd834159cf0 2023-10-06T02:32:21.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.840+0000 7fd8337fe700 1 --2- 172.21.15.31:0/3692345005 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd820067830 0x7fd820069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:21.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.841+0000 7fd8337fe700 1 --2- 172.21.15.31:0/3692345005 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd820067830 0x7fd820069ce0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fd8240093f0 tx=0x7fd824008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:21.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:20.844+0000 7fd81b7fe700 1 -- 172.21.15.31:0/3692345005 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd828052c30 con 0x7fd834159cf0 2023-10-06T02:32:21.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:21.080+0000 7fd838f9d700 1 -- 172.21.15.31:0/3692345005 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000634507", "group_name": "group_0000000000082492", "target": ["mon-mgr", ""]}) v1 -- 0x7fd83414a290 con 0x7fd820067830 2023-10-06T02:32:21.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:21.086+0000 7fd81b7fe700 1 -- 172.21.15.31:0/3692345005 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+59 (secure 0 0 0) 0x7fd83414a290 con 0x7fd820067830 2023-10-06T02:32:21.086 INFO:teuthology.orchestra.run.smithi031.stdout:/volumes/group_0000000000082492/subvolume_0000000000634507 2023-10-06T02:32:21.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:21.088+0000 7fd838f9d700 1 -- 172.21.15.31:0/3692345005 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd820067830 msgr2=0x7fd820069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:21.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:21.088+0000 7fd838f9d700 1 --2- 172.21.15.31:0/3692345005 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd820067830 0x7fd820069ce0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fd8240093f0 tx=0x7fd824008040 comp rx=0 tx=0).stop 2023-10-06T02:32:21.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:21.088+0000 7fd838f9d700 1 -- 172.21.15.31:0/3692345005 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd834159cf0 msgr2=0x7fd83415a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:21.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:21.088+0000 7fd838f9d700 1 --2- 172.21.15.31:0/3692345005 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd834159cf0 0x7fd83415a160 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7fd828005c80 tx=0x7fd828015000 comp rx=0 tx=0).stop 2023-10-06T02:32:21.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:21.088+0000 7fd838f9d700 1 -- 172.21.15.31:0/3692345005 shutdown_connections 2023-10-06T02:32:21.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:21.088+0000 7fd838f9d700 1 --2- 172.21.15.31:0/3692345005 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8340ca050 0x7fd8340ca4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:21.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:21.088+0000 7fd838f9d700 1 --2- 172.21.15.31:0/3692345005 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fd820067830 0x7fd820069ce0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:21.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:21.088+0000 7fd838f9d700 1 --2- 172.21.15.31:0/3692345005 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd81c004010 0x7fd834149d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:21.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:21.088+0000 7fd838f9d700 1 --2- 172.21.15.31:0/3692345005 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd834159cf0 0x7fd83415a160 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:21.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:21.088+0000 7fd838f9d700 1 -- 172.21.15.31:0/3692345005 >> 172.21.15.31:0/3692345005 conn(0x7fd83406c6a0 msgr2=0x7fd8340aa200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:21.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:21.088+0000 7fd838f9d700 1 -- 172.21.15.31:0/3692345005 shutdown_connections 2023-10-06T02:32:21.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:21.088+0000 7fd838f9d700 1 -- 172.21.15.31:0/3692345005 wait complete. 2023-10-06T02:32:21.106 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:32:21.106 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_legacy/d5f1180fee77cfbde3aa18c0f9625ce0.meta) 2023-10-06T02:32:21.905 INFO:teuthology.orchestra.run.smithi031.stdout:[GLOBAL] 2023-10-06T02:32:21.905 INFO:teuthology.orchestra.run.smithi031.stdout:version = 1 2023-10-06T02:32:21.905 INFO:teuthology.orchestra.run.smithi031.stdout:type = subvolume 2023-10-06T02:32:21.905 INFO:teuthology.orchestra.run.smithi031.stdout:path = /volumes/group_0000000000082492/subvolume_0000000000634507 2023-10-06T02:32:21.905 INFO:teuthology.orchestra.run.smithi031.stdout:state = complete 2023-10-06T02:32:21.905 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:32:21.909 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000634507 group_0000000000082492 2023-10-06T02:32:22.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.110+0000 7fa68939d700 1 Processor -- start 2023-10-06T02:32:22.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.110+0000 7fa68939d700 1 -- start start 2023-10-06T02:32:22.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.111+0000 7fa68939d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa6840cb450 0x7fa68415dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:22.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.111+0000 7fa68939d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa684143070 0x7fa6840cbf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:22.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.111+0000 7fa68939d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa68415d8d0 con 0x7fa684142ca0 2023-10-06T02:32:22.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.111+0000 7fa68939d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa68415d2f0 con 0x7fa6840cb450 2023-10-06T02:32:22.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.111+0000 7fa68939d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6840cc630 con 0x7fa684143070 2023-10-06T02:32:22.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.111+0000 7fa6837fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa6840cb450 0x7fa68415dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:22.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.111+0000 7fa6837fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa6840cb450 0x7fa68415dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39006/0 (socket says 172.21.15.31:39006) 2023-10-06T02:32:22.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.111+0000 7fa6837fe700 1 -- 172.21.15.31:0/60710884 learned_addr learned my addr 172.21.15.31:0/60710884 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:22.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.112+0000 7fa688b9c700 1 --2- 172.21.15.31:0/60710884 >> v2:172.21.15.62:3300/0 conn(0x7fa684143070 0x7fa6840cbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:22.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.112+0000 7fa6837fe700 1 -- 172.21.15.31:0/60710884 >> v2:172.21.15.62:3300/0 conn(0x7fa684143070 msgr2=0x7fa6840cbf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:22.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.112+0000 7fa6837fe700 1 --2- 172.21.15.31:0/60710884 >> v2:172.21.15.62:3300/0 conn(0x7fa684143070 0x7fa6840cbf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:22.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.112+0000 7fa6837fe700 1 -- 172.21.15.31:0/60710884 >> v1:172.21.15.62:6789/0 conn(0x7fa684142ca0 legacy=0x7fa6840cad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:22.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.112+0000 7fa6837fe700 1 -- 172.21.15.31:0/60710884 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa68415d560 con 0x7fa6840cb450 2023-10-06T02:32:22.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.112+0000 7fa688b9c700 1 --2- 172.21.15.31:0/60710884 >> v2:172.21.15.62:3300/0 conn(0x7fa684143070 0x7fa6840cbf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:32:22.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.112+0000 7fa6837fe700 1 --2- 172.21.15.31:0/60710884 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6840cb450 0x7fa68415dd60 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7fa68406a730 tx=0x7fa678009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cead1e7fd4a9b57d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:22.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.112+0000 7fa682ffd700 1 -- 172.21.15.31:0/60710884 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa67800b860 con 0x7fa6840cb450 2023-10-06T02:32:22.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.112+0000 7fa682ffd700 1 -- 172.21.15.31:0/60710884 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa67800b9c0 con 0x7fa6840cb450 2023-10-06T02:32:22.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.112+0000 7fa682ffd700 1 -- 172.21.15.31:0/60710884 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa67800bd60 con 0x7fa6840cb450 2023-10-06T02:32:22.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.113+0000 7fa68939d700 1 -- 172.21.15.31:0/60710884 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6840cb450 msgr2=0x7fa68415dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:22.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.113+0000 7fa68939d700 1 --2- 172.21.15.31:0/60710884 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6840cb450 0x7fa68415dd60 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7fa68406a730 tx=0x7fa678009250 comp rx=0 tx=0).stop 2023-10-06T02:32:22.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.113+0000 7fa68939d700 1 -- 172.21.15.31:0/60710884 shutdown_connections 2023-10-06T02:32:22.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.113+0000 7fa68939d700 1 --2- 172.21.15.31:0/60710884 >> v2:172.21.15.62:3300/0 conn(0x7fa684143070 0x7fa6840cbf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:22.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.113+0000 7fa68939d700 1 --2- 172.21.15.31:0/60710884 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6840cb450 0x7fa68415dd60 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:22.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.113+0000 7fa68939d700 1 -- 172.21.15.31:0/60710884 >> 172.21.15.31:0/60710884 conn(0x7fa684067c80 msgr2=0x7fa684068080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:22.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.113+0000 7fa68939d700 1 -- 172.21.15.31:0/60710884 shutdown_connections 2023-10-06T02:32:22.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.113+0000 7fa68939d700 1 -- 172.21.15.31:0/60710884 wait complete. 2023-10-06T02:32:22.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.115+0000 7fa68939d700 1 Processor -- start 2023-10-06T02:32:22.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.115+0000 7fa68939d700 1 -- start start 2023-10-06T02:32:22.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.116+0000 7fa68939d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa6840cb450 0x7fa68413ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:22.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.116+0000 7fa68939d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa684142ca0 0x7fa68413f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:22.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.116+0000 7fa68939d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa684143070 0x7fa6840b3260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:22.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.116+0000 7fa68939d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6840aec30 con 0x7fa684142ca0 2023-10-06T02:32:22.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.116+0000 7fa68939d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6840a9aa0 con 0x7fa684143070 2023-10-06T02:32:22.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.116+0000 7fa68939d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6840b1ac0 con 0x7fa6840cb450 2023-10-06T02:32:22.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.116+0000 7fa6837fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa684142ca0 0x7fa68413f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:22.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.116+0000 7fa6837fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa684142ca0 0x7fa68413f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39010/0 (socket says 172.21.15.31:39010) 2023-10-06T02:32:22.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.116+0000 7fa6837fe700 1 -- 172.21.15.31:0/569335256 learned_addr learned my addr 172.21.15.31:0/569335256 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:22.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.116+0000 7fa683fff700 1 --2- 172.21.15.31:0/569335256 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa6840cb450 0x7fa68413ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:22.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.116+0000 7fa688b9c700 1 --2- 172.21.15.31:0/569335256 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa684143070 0x7fa6840b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:22.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.116+0000 7fa6837fe700 1 -- 172.21.15.31:0/569335256 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa6840cb450 msgr2=0x7fa68413ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:22.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.116+0000 7fa6837fe700 1 --2- 172.21.15.31:0/569335256 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa6840cb450 0x7fa68413ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:22.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.116+0000 7fa6837fe700 1 -- 172.21.15.31:0/569335256 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa684143070 msgr2=0x7fa6840b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:22.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.116+0000 7fa6837fe700 1 --2- 172.21.15.31:0/569335256 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa684143070 0x7fa6840b3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:22.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.116+0000 7fa6837fe700 1 -- 172.21.15.31:0/569335256 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6780094f0 con 0x7fa684142ca0 2023-10-06T02:32:22.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.116+0000 7fa688b9c700 1 --2- 172.21.15.31:0/569335256 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa684143070 0x7fa6840b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:32:22.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.117+0000 7fa6837fe700 1 --2- 172.21.15.31:0/569335256 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa684142ca0 0x7fa68413f180 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7fa678009290 tx=0x7fa678017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:22.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.117+0000 7fa66bfff700 1 -- 172.21.15.31:0/569335256 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa678026480 con 0x7fa684142ca0 2023-10-06T02:32:22.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.117+0000 7fa66bfff700 1 -- 172.21.15.31:0/569335256 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa67801f070 con 0x7fa684142ca0 2023-10-06T02:32:22.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.117+0000 7fa66bfff700 1 -- 172.21.15.31:0/569335256 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa678026850 con 0x7fa684142ca0 2023-10-06T02:32:22.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.117+0000 7fa68939d700 1 -- 172.21.15.31:0/569335256 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6840b1d60 con 0x7fa684142ca0 2023-10-06T02:32:22.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.117+0000 7fa68939d700 1 -- 172.21.15.31:0/569335256 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6840b2320 con 0x7fa684142ca0 2023-10-06T02:32:22.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.118+0000 7fa68939d700 1 -- 172.21.15.31:0/569335256 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa68404c3c0 con 0x7fa684142ca0 2023-10-06T02:32:22.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.119+0000 7fa66bfff700 1 -- 172.21.15.31:0/569335256 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85556+0+0 (secure 0 0 0) 0x7fa67801a430 con 0x7fa684142ca0 2023-10-06T02:32:22.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.120+0000 7fa66bfff700 1 --2- 172.21.15.31:0/569335256 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa670067830 0x7fa670069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:22.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.120+0000 7fa66bfff700 1 -- 172.21.15.31:0/569335256 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fa678021070 con 0x7fa684142ca0 2023-10-06T02:32:22.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.120+0000 7fa683fff700 1 --2- 172.21.15.31:0/569335256 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa670067830 0x7fa670069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:22.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.121+0000 7fa683fff700 1 --2- 172.21.15.31:0/569335256 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa670067830 0x7fa670069ce0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fa674003a80 tx=0x7fa674007ec0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:22.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.124+0000 7fa66bfff700 1 -- 172.21.15.31:0/569335256 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa6780539c0 con 0x7fa684142ca0 2023-10-06T02:32:22.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.357+0000 7fa68939d700 1 -- 172.21.15.31:0/569335256 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000634507", "group_name": "group_0000000000082492", "target": ["mon-mgr", ""]}) v1 -- 0x7fa6840aed70 con 0x7fa670067830 2023-10-06T02:32:22.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.375+0000 7fa66bfff700 1 -- 172.21.15.31:0/569335256 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fa6840aed70 con 0x7fa670067830 2023-10-06T02:32:22.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.377+0000 7fa68939d700 1 -- 172.21.15.31:0/569335256 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa670067830 msgr2=0x7fa670069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:22.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.377+0000 7fa68939d700 1 --2- 172.21.15.31:0/569335256 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa670067830 0x7fa670069ce0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fa674003a80 tx=0x7fa674007ec0 comp rx=0 tx=0).stop 2023-10-06T02:32:22.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.377+0000 7fa68939d700 1 -- 172.21.15.31:0/569335256 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa684142ca0 msgr2=0x7fa68413f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:22.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.377+0000 7fa68939d700 1 --2- 172.21.15.31:0/569335256 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa684142ca0 0x7fa68413f180 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7fa678009290 tx=0x7fa678017000 comp rx=0 tx=0).stop 2023-10-06T02:32:22.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.378+0000 7fa68939d700 1 -- 172.21.15.31:0/569335256 shutdown_connections 2023-10-06T02:32:22.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.378+0000 7fa68939d700 1 --2- 172.21.15.31:0/569335256 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa684143070 0x7fa6840b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:22.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.378+0000 7fa68939d700 1 --2- 172.21.15.31:0/569335256 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa670067830 0x7fa670069ce0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:22.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.378+0000 7fa68939d700 1 --2- 172.21.15.31:0/569335256 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa6840cb450 0x7fa68413ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:22.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.378+0000 7fa68939d700 1 --2- 172.21.15.31:0/569335256 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa684142ca0 0x7fa68413f180 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:22.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.378+0000 7fa68939d700 1 -- 172.21.15.31:0/569335256 >> 172.21.15.31:0/569335256 conn(0x7fa684067c80 msgr2=0x7fa68415c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:22.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.378+0000 7fa68939d700 1 -- 172.21.15.31:0/569335256 shutdown_connections 2023-10-06T02:32:22.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.378+0000 7fa68939d700 1 -- 172.21.15.31:0/569335256 wait complete. 2023-10-06T02:32:22.382 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:32:22.381+0000 7f6567381700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:32:22.383 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:32:22.382+0000 7f6567381700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:32:22.383 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:32:22.382+0000 7f6567381700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:32:22.383 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:32:22.382+0000 7f6567381700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:32:22.383 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:32:22.382+0000 7f6567381700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:32:22.396 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:32:22.396 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2023-10-06T02:32:22.401 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:32:22.399+0000 7f6568383700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:32:22.401 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:32:22.399+0000 7f6568383700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:32:22.401 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:32:22.399+0000 7f6568383700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:32:22.402 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:32:22.399+0000 7f6568383700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:32:22.402 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T02:32:22.399+0000 7f6568383700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T02:32:22.471 INFO:teuthology.orchestra.run.smithi031.stdout:2 2023-10-06T02:32:22.472 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm cephfs group_0000000000082492 2023-10-06T02:32:22.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.663+0000 7f933959d700 1 Processor -- start 2023-10-06T02:32:22.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.663+0000 7f933959d700 1 -- start start 2023-10-06T02:32:22.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.665+0000 7f933959d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f93340cc840 0x7f9334155890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:22.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.665+0000 7f933959d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f93340cccb0 0x7f933406de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:22.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.665+0000 7f933959d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f933406b790 con 0x7f93340c7930 2023-10-06T02:32:22.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.665+0000 7f933959d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f933406ab80 con 0x7f93340cccb0 2023-10-06T02:32:22.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.665+0000 7f933959d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f933406ae60 con 0x7f93340cc840 2023-10-06T02:32:22.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.665+0000 7f9338d9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f93340cccb0 0x7f933406de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:22.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.665+0000 7f9338d9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f93340cccb0 0x7f933406de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39016/0 (socket says 172.21.15.31:39016) 2023-10-06T02:32:22.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.665+0000 7f9338d9c700 1 -- 172.21.15.31:0/4191013705 learned_addr learned my addr 172.21.15.31:0/4191013705 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:22.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.665+0000 7f93337fe700 1 --2- 172.21.15.31:0/4191013705 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f93340cc840 0x7f9334155890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:22.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.665+0000 7f9338d9c700 1 -- 172.21.15.31:0/4191013705 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f93340cc840 msgr2=0x7f9334155890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:22.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.665+0000 7f9338d9c700 1 --2- 172.21.15.31:0/4191013705 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f93340cc840 0x7f9334155890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:22.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.665+0000 7f9338d9c700 1 -- 172.21.15.31:0/4191013705 >> v1:172.21.15.62:6789/0 conn(0x7f93340c7930 legacy=0x7f933406a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:22.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.665+0000 7f9338d9c700 1 -- 172.21.15.31:0/4191013705 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f933406b140 con 0x7f93340cccb0 2023-10-06T02:32:22.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.666+0000 7f9338d9c700 1 --2- 172.21.15.31:0/4191013705 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93340cccb0 0x7f933406de50 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f932c009f20 tx=0x7f932c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae12cd7df09cb40 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:22.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.666+0000 7f9332ffd700 1 -- 172.21.15.31:0/4191013705 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f932c00e070 con 0x7f93340cccb0 2023-10-06T02:32:22.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.666+0000 7f9332ffd700 1 -- 172.21.15.31:0/4191013705 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f932c00b9d0 con 0x7f93340cccb0 2023-10-06T02:32:22.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.666+0000 7f9332ffd700 1 -- 172.21.15.31:0/4191013705 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f932c0092e0 con 0x7f93340cccb0 2023-10-06T02:32:22.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.666+0000 7f933959d700 1 -- 172.21.15.31:0/4191013705 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93340cccb0 msgr2=0x7f933406de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:22.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.666+0000 7f933959d700 1 --2- 172.21.15.31:0/4191013705 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93340cccb0 0x7f933406de50 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f932c009f20 tx=0x7f932c009840 comp rx=0 tx=0).stop 2023-10-06T02:32:22.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.666+0000 7f933959d700 1 -- 172.21.15.31:0/4191013705 shutdown_connections 2023-10-06T02:32:22.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.666+0000 7f933959d700 1 --2- 172.21.15.31:0/4191013705 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93340cccb0 0x7f933406de50 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:22.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.666+0000 7f933959d700 1 --2- 172.21.15.31:0/4191013705 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f93340cc840 0x7f9334155890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:22.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.666+0000 7f933959d700 1 -- 172.21.15.31:0/4191013705 >> 172.21.15.31:0/4191013705 conn(0x7f933406d110 msgr2=0x7f93340cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:22.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.667+0000 7f933959d700 1 -- 172.21.15.31:0/4191013705 shutdown_connections 2023-10-06T02:32:22.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.667+0000 7f933959d700 1 -- 172.21.15.31:0/4191013705 wait complete. 2023-10-06T02:32:22.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.668+0000 7f933959d700 1 Processor -- start 2023-10-06T02:32:22.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.668+0000 7f933959d700 1 -- start start 2023-10-06T02:32:22.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.669+0000 7f933959d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93340c7930 0x7f933415a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:22.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.669+0000 7f933959d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93340cc840 0x7f9334153c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:22.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.669+0000 7f933959d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f93340cccb0 0x7f933415da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:22.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.669+0000 7f933959d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9334156a20 con 0x7f93340c7930 2023-10-06T02:32:22.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.669+0000 7f933959d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f933415acb0 con 0x7f93340cc840 2023-10-06T02:32:22.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.669+0000 7f933959d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9334155e10 con 0x7f93340cccb0 2023-10-06T02:32:22.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.669+0000 7f9333fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93340c7930 0x7f933415a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:22.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.669+0000 7f9333fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93340c7930 0x7f933415a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39020/0 (socket says 172.21.15.31:39020) 2023-10-06T02:32:22.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.669+0000 7f9333fff700 1 -- 172.21.15.31:0/1338975335 learned_addr learned my addr 172.21.15.31:0/1338975335 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:22.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.669+0000 7f9338d9c700 1 --2- 172.21.15.31:0/1338975335 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f93340cccb0 0x7f933415da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:22.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.669+0000 7f93337fe700 1 --2- 172.21.15.31:0/1338975335 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93340cc840 0x7f9334153c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:22.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.670+0000 7f9333fff700 1 -- 172.21.15.31:0/1338975335 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f93340cccb0 msgr2=0x7f933415da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:22.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.670+0000 7f9333fff700 1 --2- 172.21.15.31:0/1338975335 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f93340cccb0 0x7f933415da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:22.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.670+0000 7f9333fff700 1 -- 172.21.15.31:0/1338975335 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93340cc840 msgr2=0x7f9334153c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:22.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.670+0000 7f9333fff700 1 --2- 172.21.15.31:0/1338975335 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93340cc840 0x7f9334153c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:22.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.670+0000 7f9333fff700 1 -- 172.21.15.31:0/1338975335 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f932c009ae0 con 0x7f93340c7930 2023-10-06T02:32:22.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.670+0000 7f9333fff700 1 --2- 172.21.15.31:0/1338975335 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93340c7930 0x7f933415a770 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f932400c510 tx=0x7f932400c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:22.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.670+0000 7f931bfff700 1 -- 172.21.15.31:0/1338975335 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9324011590 con 0x7f93340c7930 2023-10-06T02:32:22.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.670+0000 7f931bfff700 1 -- 172.21.15.31:0/1338975335 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9324011710 con 0x7f93340c7930 2023-10-06T02:32:22.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.670+0000 7f933959d700 1 -- 172.21.15.31:0/1338975335 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93341560f0 con 0x7f93340c7930 2023-10-06T02:32:22.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.670+0000 7f931bfff700 1 -- 172.21.15.31:0/1338975335 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9324011ad0 con 0x7f93340c7930 2023-10-06T02:32:22.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.670+0000 7f933959d700 1 -- 172.21.15.31:0/1338975335 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93341566b0 con 0x7f93340c7930 2023-10-06T02:32:22.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.671+0000 7f933959d700 1 -- 172.21.15.31:0/1338975335 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f933404c3c0 con 0x7f93340c7930 2023-10-06T02:32:22.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.672+0000 7f931bfff700 1 -- 172.21.15.31:0/1338975335 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 26) v1 ==== 85607+0+0 (secure 0 0 0) 0x7f932401b3b0 con 0x7f93340c7930 2023-10-06T02:32:22.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.673+0000 7f931bfff700 1 --2- 172.21.15.31:0/1338975335 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f93200678c0 0x7f9320069d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:22.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.673+0000 7f931bfff700 1 -- 172.21.15.31:0/1338975335 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f9324082430 con 0x7f93340c7930 2023-10-06T02:32:22.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.674+0000 7f93337fe700 1 --2- 172.21.15.31:0/1338975335 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f93200678c0 0x7f9320069d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:22.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.674+0000 7f93337fe700 1 --2- 172.21.15.31:0/1338975335 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f93200678c0 0x7f9320069d70 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f93340c60c0 tx=0x7f93280091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:22.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.677+0000 7f931bfff700 1 -- 172.21.15.31:0/1338975335 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9324050e70 con 0x7f93340c7930 2023-10-06T02:32:22.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.910+0000 7f933959d700 1 -- 172.21.15.31:0/1338975335 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "group_name": "group_0000000000082492", "target": ["mon-mgr", ""]}) v1 -- 0x7f9334156b60 con 0x7f93200678c0 2023-10-06T02:32:22.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.915+0000 7f931bfff700 1 -- 172.21.15.31:0/1338975335 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f9328011a50 con 0x7f93200678c0 2023-10-06T02:32:22.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.917+0000 7f933959d700 1 -- 172.21.15.31:0/1338975335 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f93200678c0 msgr2=0x7f9320069d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:22.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.917+0000 7f933959d700 1 --2- 172.21.15.31:0/1338975335 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f93200678c0 0x7f9320069d70 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f93340c60c0 tx=0x7f93280091a0 comp rx=0 tx=0).stop 2023-10-06T02:32:22.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.917+0000 7f933959d700 1 -- 172.21.15.31:0/1338975335 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93340c7930 msgr2=0x7f933415a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:22.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.917+0000 7f933959d700 1 --2- 172.21.15.31:0/1338975335 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93340c7930 0x7f933415a770 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f932400c510 tx=0x7f932400c820 comp rx=0 tx=0).stop 2023-10-06T02:32:22.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.917+0000 7f933959d700 1 -- 172.21.15.31:0/1338975335 shutdown_connections 2023-10-06T02:32:22.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.917+0000 7f933959d700 1 --2- 172.21.15.31:0/1338975335 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f93200678c0 0x7f9320069d70 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:22.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.917+0000 7f933959d700 1 --2- 172.21.15.31:0/1338975335 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f93340cccb0 0x7f933415da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:22.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.917+0000 7f933959d700 1 --2- 172.21.15.31:0/1338975335 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93340c7930 0x7f933415a770 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:22.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.917+0000 7f933959d700 1 --2- 172.21.15.31:0/1338975335 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93340cc840 0x7f9334153c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:22.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.917+0000 7f933959d700 1 -- 172.21.15.31:0/1338975335 >> 172.21.15.31:0/1338975335 conn(0x7f933406d110 msgr2=0x7f93340af1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:22.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.917+0000 7f933959d700 1 -- 172.21.15.31:0/1338975335 shutdown_connections 2023-10-06T02:32:22.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:22.917+0000 7f933959d700 1 -- 172.21.15.31:0/1338975335 wait complete. 2023-10-06T02:32:22.935 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T02:32:22.935 DEBUG:teuthology.orchestra.run.smithi031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T02:32:23.024 DEBUG:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T02:32:23.113 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T02:32:23.113 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T02:32:23.113 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:32:23.113 DEBUG:teuthology.orchestra.run.smithi031:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T02:32:23.215 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi031.front.sepia.ceph.com 2023-10-06T02:32:23.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:32:23.216 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T02:32:23.259 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-06T02:32:23.259 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:32:23.259 DEBUG:teuthology.orchestra.run.smithi062:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-06T02:32:23.341 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi062.front.sepia.ceph.com 2023-10-06T02:32:23.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:32:23.341 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T02:32:23.379 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:32:23.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:23.566+0000 7f1500f9d700 1 Processor -- start 2023-10-06T02:32:23.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:23.566+0000 7f1500f9d700 1 -- start start 2023-10-06T02:32:23.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:23.567+0000 7f1500f9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f14fc0af600 0x7f14fc0c6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:23.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:23.567+0000 7f1500f9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14fc0c6d50 con 0x7f14fc140a70 2023-10-06T02:32:23.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:23.567+0000 7f1500f9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14fc0c7030 con 0x7f14fc140e40 2023-10-06T02:32:23.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:23.567+0000 7f1500f9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14fc0c7310 con 0x7f14fc0af600 2023-10-06T02:32:23.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:23.568+0000 7f14fb7fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f14fc140a70 0x7f14fc0c7df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56270/0 (socket says 172.21.15.31:56270) 2023-10-06T02:32:23.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:23.568+0000 7f14fb7fe700 1 -- 172.21.15.31:0/4116504385 learned_addr learned my addr 172.21.15.31:0/4116504385 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:32:23.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:23.568+0000 7f14fbfff700 1 --2- 172.21.15.31:0/4116504385 >> v2:172.21.15.62:3300/0 conn(0x7f14fc0af600 0x7f14fc0c6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:23.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:23.568+0000 7f14fa7fc700 1 -- 172.21.15.31:0/4116504385 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2712235747 0 0) 0x7f14fc0c6d50 con 0x7f14fc140a70 2023-10-06T02:32:23.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:23.568+0000 7f14fa7fc700 1 -- 172.21.15.31:0/4116504385 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14e4003570 con 0x7f14fc140a70 2023-10-06T02:32:23.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:23.568+0000 7f14fbfff700 1 -- 172.21.15.31:0/4116504385 >> v1:172.21.15.62:6789/0 conn(0x7f14fc140e40 legacy=0x7f14fc0aeef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:23.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:23.568+0000 7f14fbfff700 1 -- 172.21.15.31:0/4116504385 >> v1:172.21.15.31:6789/0 conn(0x7f14fc140a70 legacy=0x7f14fc0c7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:23.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:23.568+0000 7f14fa7fc700 1 -- 172.21.15.31:0/4116504385 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2780766911 0 0) 0x7f14e4003570 con 0x7f14fc140a70 2023-10-06T02:32:23.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:23.568+0000 7f14fbfff700 1 -- 172.21.15.31:0/4116504385 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14fc0c75f0 con 0x7f14fc0af600 2023-10-06T02:32:23.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:23.569+0000 7f14fbfff700 1 --2- 172.21.15.31:0/4116504385 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14fc0af600 0x7f14fc0c6770 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f14f4009e60 tx=0x7f14f4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fce61788d891be26 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:23.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:23.569+0000 7f14fa7fc700 1 -- 172.21.15.31:0/4116504385 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f400e070 con 0x7f14fc0af600 2023-10-06T02:32:23.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:23.569+0000 7f14fa7fc700 1 -- 172.21.15.31:0/4116504385 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14f400b850 con 0x7f14fc0af600 2023-10-06T02:32:23.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:23.569+0000 7f14fa7fc700 1 -- 172.21.15.31:0/4116504385 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ec001470 con 0x7f14fc0af600 2023-10-06T02:32:23.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:23.569+0000 7f1500f9d700 1 -- 172.21.15.31:0/4116504385 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14fc0af600 msgr2=0x7f14fc0c6770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:23.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:23.569+0000 7f1500f9d700 1 --2- 172.21.15.31:0/4116504385 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14fc0af600 0x7f14fc0c6770 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f14f4009e60 tx=0x7f14f4009780 comp rx=0 tx=0).stop 2023-10-06T02:32:23.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:23.570+0000 7f1500f9d700 1 -- 172.21.15.31:0/4116504385 shutdown_connections 2023-10-06T02:32:23.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:23.570+0000 7f1500f9d700 1 --2- 172.21.15.31:0/4116504385 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14fc0af600 0x7f14fc0c6770 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:23.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:23.570+0000 7f1500f9d700 1 -- 172.21.15.31:0/4116504385 >> 172.21.15.31:0/4116504385 conn(0x7f14fc06d170 msgr2=0x7f14fc0cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:23.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:23.570+0000 7f1500f9d700 1 -- 172.21.15.31:0/4116504385 shutdown_connections 2023-10-06T02:32:23.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:23.570+0000 7f1500f9d700 1 -- 172.21.15.31:0/4116504385 wait complete. 2023-10-06T02:32:23.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:23.571+0000 7f1500f9d700 1 Processor -- start 2023-10-06T02:32:23.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:23.571+0000 7f1500f9d700 1 -- start start 2023-10-06T02:32:23.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:23.572+0000 7f1500f9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14fc0af600 0x7f14fc16a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:23.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:23.572+0000 7f1500f9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14fc140a70 0x7f14fc15db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:23.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:23.572+0000 7f1500f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f14fc140e40 0x7f14fc0b0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:23.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:23.572+0000 7f1500f9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14fc157660 con 0x7f14fc0af600 2023-10-06T02:32:23.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:23.572+0000 7f1500f9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14fc158a30 con 0x7f14fc140a70 2023-10-06T02:32:23.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:23.572+0000 7f1500f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14fc16aa60 con 0x7f14fc140e40 2023-10-06T02:32:23.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:23.572+0000 7f14fb7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14fc0af600 0x7f14fc16a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:23.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:23.572+0000 7f14fb7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14fc0af600 0x7f14fc16a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39022/0 (socket says 172.21.15.31:39022) 2023-10-06T02:32:23.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:23.572+0000 7f14fb7fe700 1 -- 172.21.15.31:0/1609315171 learned_addr learned my addr 172.21.15.31:0/1609315171 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:23.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:23.573+0000 7f14faffd700 1 --2- 172.21.15.31:0/1609315171 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14fc140a70 0x7f14fc15db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:23.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:23.573+0000 7f14fbfff700 1 --2- 172.21.15.31:0/1609315171 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f14fc140e40 0x7f14fc0b0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:23.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:23.573+0000 7f14fb7fe700 1 -- 172.21.15.31:0/1609315171 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f14fc140e40 msgr2=0x7f14fc0b0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:23.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:23.573+0000 7f14fb7fe700 1 --2- 172.21.15.31:0/1609315171 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f14fc140e40 0x7f14fc0b0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:23.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:23.573+0000 7f14fb7fe700 1 -- 172.21.15.31:0/1609315171 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14fc140a70 msgr2=0x7f14fc15db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:23.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:23.573+0000 7f14fb7fe700 1 --2- 172.21.15.31:0/1609315171 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14fc140a70 0x7f14fc15db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:23.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:23.573+0000 7f14fb7fe700 1 -- 172.21.15.31:0/1609315171 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14f4009a20 con 0x7f14fc0af600 2023-10-06T02:32:23.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:23.573+0000 7f14fbfff700 1 --2- 172.21.15.31:0/1609315171 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f14fc140e40 0x7f14fc0b0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:32:23.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:23.573+0000 7f14fb7fe700 1 --2- 172.21.15.31:0/1609315171 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14fc0af600 0x7f14fc16a520 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f14ec00cb90 tx=0x7f14ec001ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:23.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:23.573+0000 7f14e37fe700 1 -- 172.21.15.31:0/1609315171 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ec001d60 con 0x7f14fc0af600 2023-10-06T02:32:23.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:23.573+0000 7f1500f9d700 1 -- 172.21.15.31:0/1609315171 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14fc157e20 con 0x7f14fc0af600 2023-10-06T02:32:23.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:23.573+0000 7f14e37fe700 1 -- 172.21.15.31:0/1609315171 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14ec007360 con 0x7f14fc0af600 2023-10-06T02:32:23.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:23.573+0000 7f14e37fe700 1 -- 172.21.15.31:0/1609315171 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ec007700 con 0x7f14fc0af600 2023-10-06T02:32:23.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:23.573+0000 7f1500f9d700 1 -- 172.21.15.31:0/1609315171 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14fc1583e0 con 0x7f14fc0af600 2023-10-06T02:32:23.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:23.574+0000 7f1500f9d700 1 -- 172.21.15.31:0/1609315171 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14fc04c3c0 con 0x7f14fc0af600 2023-10-06T02:32:23.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:23.575+0000 7f14e37fe700 1 -- 172.21.15.31:0/1609315171 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 26) v1 ==== 85607+0+0 (secure 0 0 0) 0x7f14ec004a40 con 0x7f14fc0af600 2023-10-06T02:32:23.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:23.576+0000 7f14e37fe700 1 --2- 172.21.15.31:0/1609315171 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f14e80678c0 0x7f14e8069d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:23.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:23.576+0000 7f14e37fe700 1 -- 172.21.15.31:0/1609315171 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f14ec082550 con 0x7f14fc0af600 2023-10-06T02:32:23.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:23.576+0000 7f14faffd700 1 --2- 172.21.15.31:0/1609315171 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f14e80678c0 0x7f14e8069d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:23.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:23.577+0000 7f14faffd700 1 --2- 172.21.15.31:0/1609315171 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f14e80678c0 0x7f14e8069d70 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f14fc16e220 tx=0x7f14f0007f10 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:23.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:23.580+0000 7f14e37fe700 1 -- 172.21.15.31:0/1609315171 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f14ec050f90 con 0x7f14fc0af600 2023-10-06T02:32:23.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:23.827+0000 7f1500f9d700 1 -- 172.21.15.31:0/1609315171 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f14fc0c78d0 con 0x7f14fc0af600 2023-10-06T02:32:23.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:23.828+0000 7f14e37fe700 1 -- 172.21.15.31:0/1609315171 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 120 v120) v1 ==== 95+0+4601 (secure 0 0 0) 0x7f14ec0545b0 con 0x7f14fc0af600 2023-10-06T02:32:23.829 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:32:23.829 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":120,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7445,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/324072138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":324072138},{"type":"v1","addr":"172.21.15.31:6835","nonce":324072138}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":114},{"gid":7448,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/3160488596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3160488596},{"type":"v1","addr":"172.21.15.62:6834","nonce":3160488596}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":114},{"gid":8677,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/632134516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":632134516},{"type":"v1","addr":"172.21.15.62:6835","nonce":632134516}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":114}],"filesystems":[{"mdsmap":{"epoch":119,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:32:07.613387+0000","modified":"2023-10-06T02:32:11.143622+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9267},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9267":{"gid":9267,"name":"a","rank":0,"incarnation":115,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6834/3741373122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":3741373122},{"type":"v1","addr":"172.21.15.31:6834","nonce":3741373122}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2023-10-06T02:32:23.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:23.830+0000 7f1500f9d700 1 -- 172.21.15.31:0/1609315171 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f14e80678c0 msgr2=0x7f14e8069d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:23.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:23.830+0000 7f1500f9d700 1 --2- 172.21.15.31:0/1609315171 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f14e80678c0 0x7f14e8069d70 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f14fc16e220 tx=0x7f14f0007f10 comp rx=0 tx=0).stop 2023-10-06T02:32:23.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:23.830+0000 7f1500f9d700 1 -- 172.21.15.31:0/1609315171 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14fc0af600 msgr2=0x7f14fc16a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:23.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:23.830+0000 7f1500f9d700 1 --2- 172.21.15.31:0/1609315171 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14fc0af600 0x7f14fc16a520 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f14ec00cb90 tx=0x7f14ec001ad0 comp rx=0 tx=0).stop 2023-10-06T02:32:23.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:23.830+0000 7f1500f9d700 1 -- 172.21.15.31:0/1609315171 shutdown_connections 2023-10-06T02:32:23.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:23.830+0000 7f1500f9d700 1 --2- 172.21.15.31:0/1609315171 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f14e80678c0 0x7f14e8069d70 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:23.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:23.830+0000 7f1500f9d700 1 --2- 172.21.15.31:0/1609315171 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f14fc140e40 0x7f14fc0b0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:23.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:23.830+0000 7f1500f9d700 1 --2- 172.21.15.31:0/1609315171 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14fc0af600 0x7f14fc16a520 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:23.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:23.830+0000 7f1500f9d700 1 --2- 172.21.15.31:0/1609315171 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14fc140a70 0x7f14fc15db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:23.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:23.830+0000 7f1500f9d700 1 -- 172.21.15.31:0/1609315171 >> 172.21.15.31:0/1609315171 conn(0x7f14fc06d170 msgr2=0x7f14fc13fab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:23.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:23.831+0000 7f1500f9d700 1 -- 172.21.15.31:0/1609315171 shutdown_connections 2023-10-06T02:32:23.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:23.831+0000 7f1500f9d700 1 -- 172.21.15.31:0/1609315171 wait complete. 2023-10-06T02:32:23.834 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 120 2023-10-06T02:32:23.848 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:32:24.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.035+0000 7f6f1919d700 1 Processor -- start 2023-10-06T02:32:24.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.035+0000 7f6f1919d700 1 -- start start 2023-10-06T02:32:24.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.037+0000 7f6f1919d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6f1406aaa0 0x7f6f1415a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:24.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.037+0000 7f6f1919d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f1415ae70 con 0x7f6f1413fb20 2023-10-06T02:32:24.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.037+0000 7f6f1919d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f1406af60 con 0x7f6f1413fef0 2023-10-06T02:32:24.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.037+0000 7f6f1919d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f1406b240 con 0x7f6f1406aaa0 2023-10-06T02:32:24.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.037+0000 7f6f13fff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f6f1413fb20 0x7f6f140c7990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56278/0 (socket says 172.21.15.31:56278) 2023-10-06T02:32:24.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.037+0000 7f6f13fff700 1 -- 172.21.15.31:0/2019715829 learned_addr learned my addr 172.21.15.31:0/2019715829 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:32:24.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.037+0000 7f6f1899c700 1 --2- 172.21.15.31:0/2019715829 >> v2:172.21.15.31:3300/0 conn(0x7f6f1406aaa0 0x7f6f1415a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:24.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.037+0000 7f6f12ffd700 1 -- 172.21.15.31:0/2019715829 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4262415135 0 0) 0x7f6f04001110 con 0x7f6f1413fb20 2023-10-06T02:32:24.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.037+0000 7f6f12ffd700 1 -- 172.21.15.31:0/2019715829 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6efc003570 con 0x7f6f1413fb20 2023-10-06T02:32:24.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.038+0000 7f6f1899c700 1 -- 172.21.15.31:0/2019715829 >> v1:172.21.15.62:6789/0 conn(0x7f6f1413fef0 legacy=0x7f6f140c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:24.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.038+0000 7f6f12ffd700 1 -- 172.21.15.31:0/2019715829 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2465377758 0 0) 0x7f6f1406af60 con 0x7f6f1413fef0 2023-10-06T02:32:24.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.038+0000 7f6f1899c700 1 -- 172.21.15.31:0/2019715829 >> v1:172.21.15.31:6789/0 conn(0x7f6f1413fb20 legacy=0x7f6f140c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:24.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.038+0000 7f6f1899c700 1 -- 172.21.15.31:0/2019715829 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f1406b520 con 0x7f6f1406aaa0 2023-10-06T02:32:24.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.038+0000 7f6f1899c700 1 --2- 172.21.15.31:0/2019715829 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f1406aaa0 0x7f6f1415a930 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c009e60 tx=0x7f6f0c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bcaae58dfcfa78a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:24.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.038+0000 7f6f12ffd700 1 -- 172.21.15.31:0/2019715829 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f0c00e070 con 0x7f6f1406aaa0 2023-10-06T02:32:24.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.038+0000 7f6f12ffd700 1 -- 172.21.15.31:0/2019715829 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f0c00bb60 con 0x7f6f1406aaa0 2023-10-06T02:32:24.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.038+0000 7f6f12ffd700 1 -- 172.21.15.31:0/2019715829 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f0c012430 con 0x7f6f1406aaa0 2023-10-06T02:32:24.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.039+0000 7f6f1919d700 1 -- 172.21.15.31:0/2019715829 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f1406aaa0 msgr2=0x7f6f1415a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:24.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.039+0000 7f6f1919d700 1 --2- 172.21.15.31:0/2019715829 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f1406aaa0 0x7f6f1415a930 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c009e60 tx=0x7f6f0c009780 comp rx=0 tx=0).stop 2023-10-06T02:32:24.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.039+0000 7f6f1919d700 1 -- 172.21.15.31:0/2019715829 shutdown_connections 2023-10-06T02:32:24.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.039+0000 7f6f1919d700 1 --2- 172.21.15.31:0/2019715829 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f1406aaa0 0x7f6f1415a930 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:24.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.039+0000 7f6f1919d700 1 -- 172.21.15.31:0/2019715829 >> 172.21.15.31:0/2019715829 conn(0x7f6f1406d170 msgr2=0x7f6f140cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:24.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.039+0000 7f6f1919d700 1 -- 172.21.15.31:0/2019715829 shutdown_connections 2023-10-06T02:32:24.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.039+0000 7f6f1919d700 1 -- 172.21.15.31:0/2019715829 wait complete. 2023-10-06T02:32:24.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.040+0000 7f6f1919d700 1 Processor -- start 2023-10-06T02:32:24.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.040+0000 7f6f1919d700 1 -- start start 2023-10-06T02:32:24.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.041+0000 7f6f1919d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f1406aaa0 0x7f6f140c2940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:24.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.041+0000 7f6f1919d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6f1413fb20 0x7f6f14159350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:24.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.041+0000 7f6f1919d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f1413fef0 0x7f6f14155c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:24.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.041+0000 7f6f1919d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f14158c50 con 0x7f6f1413fef0 2023-10-06T02:32:24.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.041+0000 7f6f1919d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f14158ec0 con 0x7f6f1406aaa0 2023-10-06T02:32:24.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.041+0000 7f6f1919d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f14156c10 con 0x7f6f1413fb20 2023-10-06T02:32:24.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.042+0000 7f6f1899c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f1413fef0 0x7f6f14155c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:24.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.042+0000 7f6f1899c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f1413fef0 0x7f6f14155c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39028/0 (socket says 172.21.15.31:39028) 2023-10-06T02:32:24.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.042+0000 7f6f1899c700 1 -- 172.21.15.31:0/2438422851 learned_addr learned my addr 172.21.15.31:0/2438422851 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:24.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.042+0000 7f6f13fff700 1 --2- 172.21.15.31:0/2438422851 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f1406aaa0 0x7f6f140c2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:24.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.042+0000 7f6f1899c700 1 -- 172.21.15.31:0/2438422851 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6f1413fb20 msgr2=0x7f6f14159350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:24.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.042+0000 7f6f137fe700 1 --2- 172.21.15.31:0/2438422851 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6f1413fb20 0x7f6f14159350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:24.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.042+0000 7f6f1899c700 1 --2- 172.21.15.31:0/2438422851 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6f1413fb20 0x7f6f14159350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:24.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.042+0000 7f6f1899c700 1 -- 172.21.15.31:0/2438422851 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f1406aaa0 msgr2=0x7f6f140c2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:24.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.042+0000 7f6f1899c700 1 --2- 172.21.15.31:0/2438422851 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f1406aaa0 0x7f6f140c2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:24.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.042+0000 7f6f1899c700 1 -- 172.21.15.31:0/2438422851 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f0c009a20 con 0x7f6f1413fef0 2023-10-06T02:32:24.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.042+0000 7f6f13fff700 1 --2- 172.21.15.31:0/2438422851 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f1406aaa0 0x7f6f140c2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:32:24.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.042+0000 7f6f137fe700 1 --2- 172.21.15.31:0/2438422851 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6f1413fb20 0x7f6f14159350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:32:24.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.042+0000 7f6f1899c700 1 --2- 172.21.15.31:0/2438422851 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f1413fef0 0x7f6f14155c60 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f6f14071320 tx=0x7f6f0c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:24.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.043+0000 7f6efbfff700 1 -- 172.21.15.31:0/2438422851 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f0c026480 con 0x7f6f1413fef0 2023-10-06T02:32:24.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.043+0000 7f6efbfff700 1 -- 172.21.15.31:0/2438422851 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f0c0265e0 con 0x7f6f1413fef0 2023-10-06T02:32:24.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.043+0000 7f6f1919d700 1 -- 172.21.15.31:0/2438422851 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f140ada60 con 0x7f6f1413fef0 2023-10-06T02:32:24.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.043+0000 7f6efbfff700 1 -- 172.21.15.31:0/2438422851 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f0c0269a0 con 0x7f6f1413fef0 2023-10-06T02:32:24.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.043+0000 7f6f1919d700 1 -- 172.21.15.31:0/2438422851 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f141561a0 con 0x7f6f1413fef0 2023-10-06T02:32:24.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.044+0000 7f6f1919d700 1 -- 172.21.15.31:0/2438422851 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f14159070 con 0x7f6f1413fef0 2023-10-06T02:32:24.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.045+0000 7f6efbfff700 1 -- 172.21.15.31:0/2438422851 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 26) v1 ==== 85607+0+0 (secure 0 0 0) 0x7f6f0c01c400 con 0x7f6f1413fef0 2023-10-06T02:32:24.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.045+0000 7f6efbfff700 1 --2- 172.21.15.31:0/2438422851 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6f000678c0 0x7f6f00069d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:24.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.046+0000 7f6efbfff700 1 -- 172.21.15.31:0/2438422851 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f6f0c085200 con 0x7f6f1413fef0 2023-10-06T02:32:24.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.046+0000 7f6f13fff700 1 --2- 172.21.15.31:0/2438422851 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6f000678c0 0x7f6f00069d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:24.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.046+0000 7f6f13fff700 1 --2- 172.21.15.31:0/2438422851 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6f000678c0 0x7f6f00069d70 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f6f04001270 tx=0x7f6f04009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:24.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.050+0000 7f6efbfff700 1 -- 172.21.15.31:0/2438422851 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6f14159070 con 0x7f6f1413fef0 2023-10-06T02:32:24.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.259+0000 7f6f1919d700 1 -- 172.21.15.31:0/2438422851 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6f14159070 con 0x7f6f1413fef0 2023-10-06T02:32:24.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.260+0000 7f6efbfff700 1 -- 172.21.15.31:0/2438422851 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v169) v1 ==== 74+0+16876 (secure 0 0 0) 0x7f6f14159070 con 0x7f6f1413fef0 2023-10-06T02:32:24.261 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:32:24.261 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":169,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:32:09.112135+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:32:05.275284+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"168","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":28,"pool_name":"cephfs_data","create_time":"2023-10-06T02:32:06.591889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"169","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":20,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:32:24.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.262+0000 7f6f1919d700 1 -- 172.21.15.31:0/2438422851 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6f000678c0 msgr2=0x7f6f00069d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:24.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.262+0000 7f6f1919d700 1 --2- 172.21.15.31:0/2438422851 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6f000678c0 0x7f6f00069d70 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f6f04001270 tx=0x7f6f04009040 comp rx=0 tx=0).stop 2023-10-06T02:32:24.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.262+0000 7f6f1919d700 1 -- 172.21.15.31:0/2438422851 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f1413fef0 msgr2=0x7f6f14155c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:24.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.262+0000 7f6f1919d700 1 --2- 172.21.15.31:0/2438422851 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f1413fef0 0x7f6f14155c60 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f6f14071320 tx=0x7f6f0c019000 comp rx=0 tx=0).stop 2023-10-06T02:32:24.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.262+0000 7f6f1919d700 1 -- 172.21.15.31:0/2438422851 shutdown_connections 2023-10-06T02:32:24.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.262+0000 7f6f1919d700 1 --2- 172.21.15.31:0/2438422851 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f1413fef0 0x7f6f14155c60 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:24.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.262+0000 7f6f1919d700 1 --2- 172.21.15.31:0/2438422851 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f1406aaa0 0x7f6f140c2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:24.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.262+0000 7f6f1919d700 1 --2- 172.21.15.31:0/2438422851 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f6f000678c0 0x7f6f00069d70 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:24.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.262+0000 7f6f1919d700 1 --2- 172.21.15.31:0/2438422851 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6f1413fb20 0x7f6f14159350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:24.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.262+0000 7f6f1919d700 1 -- 172.21.15.31:0/2438422851 >> 172.21.15.31:0/2438422851 conn(0x7f6f1406d170 msgr2=0x7f6f1406de50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:24.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.262+0000 7f6f1919d700 1 -- 172.21.15.31:0/2438422851 shutdown_connections 2023-10-06T02:32:24.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.262+0000 7f6f1919d700 1 -- 172.21.15.31:0/2438422851 wait complete. 2023-10-06T02:32:24.280 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T02:22:53.183425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T02:23:00.046967+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 27, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T02:32:05.275284+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '168', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 28, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T02:32:06.591889+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '169', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T02:32:24.281 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T02:32:24.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.472+0000 7f9aa91ff700 1 Processor -- start 2023-10-06T02:32:24.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.472+0000 7f9aa91ff700 1 -- start start 2023-10-06T02:32:24.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.473+0000 7f9aa91ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9aa4155f30 0x7f9aa4156380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:24.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.473+0000 7f9aa91ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9aa41575a0 con 0x7f9aa4155b60 2023-10-06T02:32:24.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.473+0000 7f9aa91ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9aa40b0e40 con 0x7f9aa41599d0 2023-10-06T02:32:24.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.473+0000 7f9aa91ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9aa4156990 con 0x7f9aa4155f30 2023-10-06T02:32:24.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.474+0000 7f9aa2d9d700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f9aa4155b60 0x7f9aa40b0730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56286/0 (socket says 172.21.15.31:56286) 2023-10-06T02:32:24.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.474+0000 7f9aa2d9d700 1 -- 172.21.15.31:0/1586272010 learned_addr learned my addr 172.21.15.31:0/1586272010 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:32:24.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.474+0000 7f9aa259c700 1 --2- 172.21.15.31:0/1586272010 >> v2:172.21.15.62:3300/0 conn(0x7f9aa4155f30 0x7f9aa4156380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:24.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.474+0000 7f9aa1d9b700 1 -- 172.21.15.31:0/1586272010 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2826223344 0 0) 0x7f9aa41575a0 con 0x7f9aa4155b60 2023-10-06T02:32:24.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.474+0000 7f9aa1d9b700 1 -- 172.21.15.31:0/1586272010 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a8c004680 con 0x7f9aa4155b60 2023-10-06T02:32:24.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.474+0000 7f9aa1d9b700 1 -- 172.21.15.31:0/1586272010 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1300542644 0 0) 0x7f9aa40b0e40 con 0x7f9aa41599d0 2023-10-06T02:32:24.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.474+0000 7f9aa1d9b700 1 -- 172.21.15.31:0/1586272010 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9aa41575a0 con 0x7f9aa41599d0 2023-10-06T02:32:24.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.474+0000 7f9aa259c700 1 -- 172.21.15.31:0/1586272010 >> v1:172.21.15.62:6789/0 conn(0x7f9aa41599d0 legacy=0x7f9aa4157a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:24.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.474+0000 7f9aa259c700 1 -- 172.21.15.31:0/1586272010 >> v1:172.21.15.31:6789/0 conn(0x7f9aa4155b60 legacy=0x7f9aa40b0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:24.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.474+0000 7f9aa259c700 1 -- 172.21.15.31:0/1586272010 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aa4156c70 con 0x7f9aa4155f30 2023-10-06T02:32:24.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.475+0000 7f9aa259c700 1 --2- 172.21.15.31:0/1586272010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9aa4155f30 0x7f9aa4156380 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f9a980098e0 tx=0x7f9a980091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1243ed7a0a629de server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:24.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.475+0000 7f9aa1d9b700 1 -- 172.21.15.31:0/1586272010 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a98011630 con 0x7f9aa4155f30 2023-10-06T02:32:24.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.475+0000 7f9aa1d9b700 1 -- 172.21.15.31:0/1586272010 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a98011790 con 0x7f9aa4155f30 2023-10-06T02:32:24.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.475+0000 7f9aa1d9b700 1 -- 172.21.15.31:0/1586272010 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a98011b10 con 0x7f9aa4155f30 2023-10-06T02:32:24.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.475+0000 7f9aa91ff700 1 -- 172.21.15.31:0/1586272010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9aa4155f30 msgr2=0x7f9aa4156380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:24.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.475+0000 7f9aa91ff700 1 --2- 172.21.15.31:0/1586272010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9aa4155f30 0x7f9aa4156380 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f9a980098e0 tx=0x7f9a980091a0 comp rx=0 tx=0).stop 2023-10-06T02:32:24.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.476+0000 7f9aa91ff700 1 -- 172.21.15.31:0/1586272010 shutdown_connections 2023-10-06T02:32:24.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.476+0000 7f9aa91ff700 1 --2- 172.21.15.31:0/1586272010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9aa4155f30 0x7f9aa4156380 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:24.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.476+0000 7f9aa91ff700 1 -- 172.21.15.31:0/1586272010 >> 172.21.15.31:0/1586272010 conn(0x7f9aa406d170 msgr2=0x7f9aa40cc400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:24.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.476+0000 7f9aa91ff700 1 -- 172.21.15.31:0/1586272010 shutdown_connections 2023-10-06T02:32:24.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.476+0000 7f9aa91ff700 1 -- 172.21.15.31:0/1586272010 wait complete. 2023-10-06T02:32:24.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.477+0000 7f9aa91ff700 1 Processor -- start 2023-10-06T02:32:24.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.477+0000 7f9aa91ff700 1 -- start start 2023-10-06T02:32:24.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.478+0000 7f9aa91ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9aa4155b60 0x7f9aa4153d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:24.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.478+0000 7f9aa91ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9aa4155f30 0x7f9aa40c2a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:24.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.478+0000 7f9aa91ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9aa41599d0 0x7f9aa40c2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:24.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.478+0000 7f9aa91ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa40c3ed0 con 0x7f9aa41599d0 2023-10-06T02:32:24.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.478+0000 7f9aa91ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa40c1360 con 0x7f9aa4155b60 2023-10-06T02:32:24.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.478+0000 7f9aa91ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa4154630 con 0x7f9aa4155f30 2023-10-06T02:32:24.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.478+0000 7f9aa359e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9aa41599d0 0x7f9aa40c2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:24.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.478+0000 7f9aa359e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9aa41599d0 0x7f9aa40c2fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39030/0 (socket says 172.21.15.31:39030) 2023-10-06T02:32:24.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.478+0000 7f9aa359e700 1 -- 172.21.15.31:0/3541114103 learned_addr learned my addr 172.21.15.31:0/3541114103 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:24.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.479+0000 7f9aa259c700 1 --2- 172.21.15.31:0/3541114103 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9aa4155f30 0x7f9aa40c2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:24.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.479+0000 7f9aa2d9d700 1 --2- 172.21.15.31:0/3541114103 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9aa4155b60 0x7f9aa4153d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:24.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.479+0000 7f9aa359e700 1 -- 172.21.15.31:0/3541114103 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9aa4155f30 msgr2=0x7f9aa40c2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:24.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.479+0000 7f9aa359e700 1 --2- 172.21.15.31:0/3541114103 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9aa4155f30 0x7f9aa40c2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:24.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.479+0000 7f9aa359e700 1 -- 172.21.15.31:0/3541114103 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9aa4155b60 msgr2=0x7f9aa4153d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:24.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.479+0000 7f9aa359e700 1 --2- 172.21.15.31:0/3541114103 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9aa4155b60 0x7f9aa4153d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:24.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.479+0000 7f9aa359e700 1 -- 172.21.15.31:0/3541114103 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a98009420 con 0x7f9aa41599d0 2023-10-06T02:32:24.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.479+0000 7f9aa259c700 1 --2- 172.21.15.31:0/3541114103 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9aa4155f30 0x7f9aa40c2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:32:24.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.479+0000 7f9aa359e700 1 --2- 172.21.15.31:0/3541114103 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9aa41599d0 0x7f9aa40c2fd0 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c00c820 tx=0x7f9a9c002680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:24.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.479+0000 7f9a8affd700 1 -- 172.21.15.31:0/3541114103 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a9c012630 con 0x7f9aa41599d0 2023-10-06T02:32:24.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.479+0000 7f9a8affd700 1 -- 172.21.15.31:0/3541114103 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a9c012790 con 0x7f9aa41599d0 2023-10-06T02:32:24.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.479+0000 7f9aa91ff700 1 -- 172.21.15.31:0/3541114103 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9aa40c3630 con 0x7f9aa41599d0 2023-10-06T02:32:24.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.479+0000 7f9a8affd700 1 -- 172.21.15.31:0/3541114103 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a9c012b50 con 0x7f9aa41599d0 2023-10-06T02:32:24.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.479+0000 7f9aa91ff700 1 -- 172.21.15.31:0/3541114103 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9aa40c3bf0 con 0x7f9aa41599d0 2023-10-06T02:32:24.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.480+0000 7f9aa91ff700 1 -- 172.21.15.31:0/3541114103 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9aa404c3c0 con 0x7f9aa41599d0 2023-10-06T02:32:24.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.481+0000 7f9a8affd700 1 -- 172.21.15.31:0/3541114103 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 26) v1 ==== 85607+0+0 (secure 0 0 0) 0x7f9a9c01b400 con 0x7f9aa41599d0 2023-10-06T02:32:24.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.482+0000 7f9a8affd700 1 --2- 172.21.15.31:0/3541114103 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9a900678c0 0x7f9a90069d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:24.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.482+0000 7f9a8affd700 1 -- 172.21.15.31:0/3541114103 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f9a9c0834e0 con 0x7f9aa41599d0 2023-10-06T02:32:24.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.483+0000 7f9aa2d9d700 1 --2- 172.21.15.31:0/3541114103 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9a900678c0 0x7f9a90069d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:24.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.483+0000 7f9aa2d9d700 1 --2- 172.21.15.31:0/3541114103 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9a900678c0 0x7f9a90069d70 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f9aa4155220 tx=0x7f9a94009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:24.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.486+0000 7f9a8affd700 1 -- 172.21.15.31:0/3541114103 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9a9c051fd0 con 0x7f9aa41599d0 2023-10-06T02:32:24.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.550+0000 7f9a8affd700 1 -- 172.21.15.31:0/3541114103 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mgrmap(e 27) v1 ==== 85709+0+0 (secure 0 0 0) 0x7f9a9c0523a0 con 0x7f9aa41599d0 2023-10-06T02:32:24.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.691+0000 7f9aa91ff700 1 -- 172.21.15.31:0/3541114103 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9aa4156f50 con 0x7f9aa41599d0 2023-10-06T02:32:24.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.692+0000 7f9a8affd700 1 -- 172.21.15.31:0/3541114103 <== mon.0 v2:172.21.15.31:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v169) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9a9c055a60 con 0x7f9aa41599d0 2023-10-06T02:32:24.693 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-10-06T02:32:24.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.694+0000 7f9aa91ff700 1 -- 172.21.15.31:0/3541114103 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9a900678c0 msgr2=0x7f9a90069d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:24.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.694+0000 7f9aa91ff700 1 --2- 172.21.15.31:0/3541114103 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9a900678c0 0x7f9a90069d70 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f9aa4155220 tx=0x7f9a94009040 comp rx=0 tx=0).stop 2023-10-06T02:32:24.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.694+0000 7f9aa91ff700 1 -- 172.21.15.31:0/3541114103 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9aa41599d0 msgr2=0x7f9aa40c2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:24.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.694+0000 7f9aa91ff700 1 --2- 172.21.15.31:0/3541114103 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9aa41599d0 0x7f9aa40c2fd0 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c00c820 tx=0x7f9a9c002680 comp rx=0 tx=0).stop 2023-10-06T02:32:24.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.694+0000 7f9aa91ff700 1 -- 172.21.15.31:0/3541114103 shutdown_connections 2023-10-06T02:32:24.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.694+0000 7f9aa91ff700 1 --2- 172.21.15.31:0/3541114103 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9aa41599d0 0x7f9aa40c2fd0 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:24.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.694+0000 7f9aa91ff700 1 --2- 172.21.15.31:0/3541114103 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9aa4155b60 0x7f9aa4153d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:24.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.694+0000 7f9aa91ff700 1 --2- 172.21.15.31:0/3541114103 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f9a900678c0 0x7f9a90069d70 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:24.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.694+0000 7f9aa91ff700 1 --2- 172.21.15.31:0/3541114103 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9aa4155f30 0x7f9aa40c2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:24.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.694+0000 7f9aa91ff700 1 -- 172.21.15.31:0/3541114103 >> 172.21.15.31:0/3541114103 conn(0x7f9aa406d170 msgr2=0x7f9aa406ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:24.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.694+0000 7f9aa91ff700 1 -- 172.21.15.31:0/3541114103 shutdown_connections 2023-10-06T02:32:24.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.694+0000 7f9aa91ff700 1 -- 172.21.15.31:0/3541114103 wait complete. 2023-10-06T02:32:24.712 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T02:32:24.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.901+0000 7fb15299d700 1 Processor -- start 2023-10-06T02:32:24.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.901+0000 7fb15299d700 1 -- start start 2023-10-06T02:32:24.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.903+0000 7fb15299d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb14c157480 0x7fb14c06dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:24.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.903+0000 7fb15299d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb14c06c6a0 0x7fb14c0c7010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:24.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.903+0000 7fb15299d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb14c06d7b0 con 0x7fb14c070430 2023-10-06T02:32:24.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.903+0000 7fb15299d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb14c06cba0 con 0x7fb14c157480 2023-10-06T02:32:24.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.903+0000 7fb15299d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb14c06ce80 con 0x7fb14c06c6a0 2023-10-06T02:32:24.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.903+0000 7fb15199b700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb14c157480 0x7fb14c06dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:24.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.903+0000 7fb15119a700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb14c06c6a0 0x7fb14c0c7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:24.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.903+0000 7fb15199b700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb14c157480 0x7fb14c06dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:39260/0 (socket says 172.21.15.31:39260) 2023-10-06T02:32:24.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.903+0000 7fb15199b700 1 -- 172.21.15.31:0/188632020 learned_addr learned my addr 172.21.15.31:0/188632020 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:24.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.904+0000 7fb15199b700 1 -- 172.21.15.31:0/188632020 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb14c06c6a0 msgr2=0x7fb14c0c7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:24.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.904+0000 7fb15199b700 1 --2- 172.21.15.31:0/188632020 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb14c06c6a0 0x7fb14c0c7010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:24.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.904+0000 7fb15199b700 1 -- 172.21.15.31:0/188632020 >> v1:172.21.15.62:6789/0 conn(0x7fb14c070430 legacy=0x7fb14c0af7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:24.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.904+0000 7fb15199b700 1 -- 172.21.15.31:0/188632020 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb14c06d160 con 0x7fb14c157480 2023-10-06T02:32:24.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.904+0000 7fb15199b700 1 --2- 172.21.15.31:0/188632020 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb14c157480 0x7fb14c06dc40 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fb1480099b0 tx=0x7fb148009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af2e72e7e83caff2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:24.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.904+0000 7fb150999700 1 -- 172.21.15.31:0/188632020 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb148009ea0 con 0x7fb14c157480 2023-10-06T02:32:24.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.904+0000 7fb150999700 1 -- 172.21.15.31:0/188632020 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb14800b860 con 0x7fb14c157480 2023-10-06T02:32:24.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.904+0000 7fb150999700 1 -- 172.21.15.31:0/188632020 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb14800bc00 con 0x7fb14c157480 2023-10-06T02:32:24.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.905+0000 7fb15299d700 1 -- 172.21.15.31:0/188632020 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb14c157480 msgr2=0x7fb14c06dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:24.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.905+0000 7fb15299d700 1 --2- 172.21.15.31:0/188632020 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb14c157480 0x7fb14c06dc40 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fb1480099b0 tx=0x7fb148009250 comp rx=0 tx=0).stop 2023-10-06T02:32:24.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.905+0000 7fb15299d700 1 -- 172.21.15.31:0/188632020 shutdown_connections 2023-10-06T02:32:24.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.905+0000 7fb15299d700 1 --2- 172.21.15.31:0/188632020 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb14c157480 0x7fb14c06dc40 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:24.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.905+0000 7fb15299d700 1 --2- 172.21.15.31:0/188632020 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb14c06c6a0 0x7fb14c0c7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:24.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.905+0000 7fb15299d700 1 -- 172.21.15.31:0/188632020 >> 172.21.15.31:0/188632020 conn(0x7fb14c06e720 msgr2=0x7fb14c06eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:24.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.905+0000 7fb15299d700 1 -- 172.21.15.31:0/188632020 shutdown_connections 2023-10-06T02:32:24.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.905+0000 7fb15299d700 1 -- 172.21.15.31:0/188632020 wait complete. 2023-10-06T02:32:24.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.906+0000 7fb15299d700 1 Processor -- start 2023-10-06T02:32:24.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.907+0000 7fb15299d700 1 -- start start 2023-10-06T02:32:24.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.908+0000 7fb15299d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb14c06c6a0 0x7fb14c15ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:24.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.908+0000 7fb15299d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb14c070430 0x7fb14c06a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:24.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.908+0000 7fb15299d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb14c157480 0x7fb14c06a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:24.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.908+0000 7fb15299d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb14c1676f0 con 0x7fb14c070430 2023-10-06T02:32:24.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.908+0000 7fb15299d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb14c15c800 con 0x7fb14c06c6a0 2023-10-06T02:32:24.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.908+0000 7fb15299d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb14c15cae0 con 0x7fb14c157480 2023-10-06T02:32:24.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.908+0000 7fb15119a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb14c070430 0x7fb14c06a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:24.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.908+0000 7fb15119a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb14c070430 0x7fb14c06a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39044/0 (socket says 172.21.15.31:39044) 2023-10-06T02:32:24.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.908+0000 7fb15119a700 1 -- 172.21.15.31:0/4265490146 learned_addr learned my addr 172.21.15.31:0/4265490146 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:24.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.908+0000 7fb15199b700 1 --2- 172.21.15.31:0/4265490146 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb14c06c6a0 0x7fb14c15ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:24.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.908+0000 7fb15219c700 1 --2- 172.21.15.31:0/4265490146 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb14c157480 0x7fb14c06a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:24.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.908+0000 7fb15119a700 1 -- 172.21.15.31:0/4265490146 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb14c157480 msgr2=0x7fb14c06a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:24.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.908+0000 7fb15119a700 1 --2- 172.21.15.31:0/4265490146 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb14c157480 0x7fb14c06a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:24.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.908+0000 7fb15119a700 1 -- 172.21.15.31:0/4265490146 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb14c06c6a0 msgr2=0x7fb14c15ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:24.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.908+0000 7fb15119a700 1 --2- 172.21.15.31:0/4265490146 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb14c06c6a0 0x7fb14c15ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:24.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.908+0000 7fb15119a700 1 -- 172.21.15.31:0/4265490146 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1480094f0 con 0x7fb14c070430 2023-10-06T02:32:24.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.908+0000 7fb15219c700 1 --2- 172.21.15.31:0/4265490146 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb14c157480 0x7fb14c06a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:32:24.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.909+0000 7fb15119a700 1 --2- 172.21.15.31:0/4265490146 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb14c070430 0x7fb14c06a1c0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7fb13c00ec80 tx=0x7fb13c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:24.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.909+0000 7fb141ffb700 1 -- 172.21.15.31:0/4265490146 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb13c00c9d0 con 0x7fb14c070430 2023-10-06T02:32:24.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.909+0000 7fb141ffb700 1 -- 172.21.15.31:0/4265490146 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb13c00cb30 con 0x7fb14c070430 2023-10-06T02:32:24.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.909+0000 7fb141ffb700 1 -- 172.21.15.31:0/4265490146 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb13c013470 con 0x7fb14c070430 2023-10-06T02:32:24.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.909+0000 7fb15299d700 1 -- 172.21.15.31:0/4265490146 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb14c15cdc0 con 0x7fb14c070430 2023-10-06T02:32:24.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.909+0000 7fb15299d700 1 -- 172.21.15.31:0/4265490146 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb14c167c20 con 0x7fb14c070430 2023-10-06T02:32:24.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.910+0000 7fb15299d700 1 -- 172.21.15.31:0/4265490146 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb14c04c3c0 con 0x7fb14c070430 2023-10-06T02:32:24.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.911+0000 7fb141ffb700 1 -- 172.21.15.31:0/4265490146 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) v1 ==== 85709+0+0 (secure 0 0 0) 0x7fb13c013820 con 0x7fb14c070430 2023-10-06T02:32:24.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.912+0000 7fb141ffb700 1 --2- 172.21.15.31:0/4265490146 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb1300679e0 0x7fb130069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:24.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.912+0000 7fb141ffb700 1 -- 172.21.15.31:0/4265490146 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fb13c083290 con 0x7fb14c070430 2023-10-06T02:32:24.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.913+0000 7fb15199b700 1 --2- 172.21.15.31:0/4265490146 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb1300679e0 0x7fb130069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:24.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.913+0000 7fb15199b700 1 --2- 172.21.15.31:0/4265490146 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb1300679e0 0x7fb130069e90 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fb1480099b0 tx=0x7fb148017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:24.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:24.916+0000 7fb141ffb700 1 -- 172.21.15.31:0/4265490146 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb13c051ed0 con 0x7fb14c070430 2023-10-06T02:32:25.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.118+0000 7fb15299d700 1 -- 172.21.15.31:0/4265490146 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb14c0c3360 con 0x7fb14c070430 2023-10-06T02:32:25.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.118+0000 7fb141ffb700 1 -- 172.21.15.31:0/4265490146 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v169) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb13c0554f0 con 0x7fb14c070430 2023-10-06T02:32:25.119 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 8 2023-10-06T02:32:25.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.120+0000 7fb15299d700 1 -- 172.21.15.31:0/4265490146 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb1300679e0 msgr2=0x7fb130069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:25.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.120+0000 7fb15299d700 1 --2- 172.21.15.31:0/4265490146 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb1300679e0 0x7fb130069e90 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fb1480099b0 tx=0x7fb148017000 comp rx=0 tx=0).stop 2023-10-06T02:32:25.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.120+0000 7fb15299d700 1 -- 172.21.15.31:0/4265490146 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb14c070430 msgr2=0x7fb14c06a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:25.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.120+0000 7fb15299d700 1 --2- 172.21.15.31:0/4265490146 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb14c070430 0x7fb14c06a1c0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7fb13c00ec80 tx=0x7fb13c0091a0 comp rx=0 tx=0).stop 2023-10-06T02:32:25.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.121+0000 7fb15299d700 1 -- 172.21.15.31:0/4265490146 shutdown_connections 2023-10-06T02:32:25.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.121+0000 7fb15299d700 1 --2- 172.21.15.31:0/4265490146 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb1300679e0 0x7fb130069e90 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:25.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.121+0000 7fb15299d700 1 --2- 172.21.15.31:0/4265490146 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb14c157480 0x7fb14c06a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:25.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.121+0000 7fb15299d700 1 --2- 172.21.15.31:0/4265490146 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb14c06c6a0 0x7fb14c15ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:25.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.121+0000 7fb15299d700 1 --2- 172.21.15.31:0/4265490146 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb14c070430 0x7fb14c06a1c0 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:25.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.121+0000 7fb15299d700 1 -- 172.21.15.31:0/4265490146 >> 172.21.15.31:0/4265490146 conn(0x7fb14c06e720 msgr2=0x7fb14c0b13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:25.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.121+0000 7fb15299d700 1 -- 172.21.15.31:0/4265490146 shutdown_connections 2023-10-06T02:32:25.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.121+0000 7fb15299d700 1 -- 172.21.15.31:0/4265490146 wait complete. 2023-10-06T02:32:25.140 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T02:32:25.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.328+0000 7fbd5e39d700 1 Processor -- start 2023-10-06T02:32:25.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.328+0000 7fbd5e39d700 1 -- start start 2023-10-06T02:32:25.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.330+0000 7fbd5e39d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fbd580c7fb0 0x7fbd5806abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:25.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.330+0000 7fbd5e39d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd5806b290 con 0x7fbd580c7be0 2023-10-06T02:32:25.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.330+0000 7fbd5e39d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd5806b570 con 0x7fbd5813f780 2023-10-06T02:32:25.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.330+0000 7fbd5e39d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd5806bf10 con 0x7fbd580c7fb0 2023-10-06T02:32:25.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.330+0000 7fbd5cb9a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fbd580c7fb0 0x7fbd5806abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:25.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.330+0000 7fbd5d39b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fbd580c7be0 0x7fbd5815a990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56292/0 (socket says 172.21.15.31:56292) 2023-10-06T02:32:25.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.330+0000 7fbd5d39b700 1 -- 172.21.15.31:0/4048875993 learned_addr learned my addr 172.21.15.31:0/4048875993 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:32:25.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.330+0000 7fbd5cb9a700 1 -- 172.21.15.31:0/4048875993 >> v1:172.21.15.62:6789/0 conn(0x7fbd5813f780 legacy=0x7fbd5813fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:25.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.330+0000 7fbd5cb9a700 1 -- 172.21.15.31:0/4048875993 >> v1:172.21.15.31:6789/0 conn(0x7fbd580c7be0 legacy=0x7fbd5815a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:25.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.331+0000 7fbd5cb9a700 1 -- 172.21.15.31:0/4048875993 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd5806b850 con 0x7fbd580c7fb0 2023-10-06T02:32:25.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.331+0000 7fbd5cb9a700 1 --2- 172.21.15.31:0/4048875993 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd580c7fb0 0x7fbd5806abd0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7fbd4800bb90 tx=0x7fbd4800b430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8629d3322f2bf79d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:25.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.331+0000 7fbd4ffff700 1 -- 172.21.15.31:0/4048875993 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4800dba0 con 0x7fbd580c7fb0 2023-10-06T02:32:25.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.331+0000 7fbd4ffff700 1 -- 172.21.15.31:0/4048875993 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd4800dd00 con 0x7fbd580c7fb0 2023-10-06T02:32:25.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.331+0000 7fbd4ffff700 1 -- 172.21.15.31:0/4048875993 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd480196b0 con 0x7fbd580c7fb0 2023-10-06T02:32:25.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.332+0000 7fbd5e39d700 1 -- 172.21.15.31:0/4048875993 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd580c7fb0 msgr2=0x7fbd5806abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:25.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.332+0000 7fbd5e39d700 1 --2- 172.21.15.31:0/4048875993 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd580c7fb0 0x7fbd5806abd0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7fbd4800bb90 tx=0x7fbd4800b430 comp rx=0 tx=0).stop 2023-10-06T02:32:25.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.332+0000 7fbd5e39d700 1 -- 172.21.15.31:0/4048875993 shutdown_connections 2023-10-06T02:32:25.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.332+0000 7fbd5e39d700 1 --2- 172.21.15.31:0/4048875993 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd580c7fb0 0x7fbd5806abd0 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:25.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.332+0000 7fbd5e39d700 1 -- 172.21.15.31:0/4048875993 >> 172.21.15.31:0/4048875993 conn(0x7fbd5806d110 msgr2=0x7fbd580cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:25.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.332+0000 7fbd5e39d700 1 -- 172.21.15.31:0/4048875993 shutdown_connections 2023-10-06T02:32:25.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.332+0000 7fbd5e39d700 1 -- 172.21.15.31:0/4048875993 wait complete. 2023-10-06T02:32:25.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.333+0000 7fbd5e39d700 1 Processor -- start 2023-10-06T02:32:25.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.333+0000 7fbd5e39d700 1 -- start start 2023-10-06T02:32:25.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.334+0000 7fbd5e39d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbd580c7be0 0x7fbd580ac9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:25.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.334+0000 7fbd5e39d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd580c7fb0 0x7fbd580b0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:25.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.334+0000 7fbd5e39d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd5813f780 0x7fbd5813eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:25.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.334+0000 7fbd5e39d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd580c4560 con 0x7fbd580c7fb0 2023-10-06T02:32:25.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.334+0000 7fbd5e39d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd580b1a70 con 0x7fbd5813f780 2023-10-06T02:32:25.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.334+0000 7fbd5e39d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd580b0fa0 con 0x7fbd580c7be0 2023-10-06T02:32:25.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.335+0000 7fbd5cb9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd580c7fb0 0x7fbd580b0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:25.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.335+0000 7fbd5cb9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd580c7fb0 0x7fbd580b0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39058/0 (socket says 172.21.15.31:39058) 2023-10-06T02:32:25.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.335+0000 7fbd5cb9a700 1 -- 172.21.15.31:0/2424111645 learned_addr learned my addr 172.21.15.31:0/2424111645 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:25.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.335+0000 7fbd5db9c700 1 --2- 172.21.15.31:0/2424111645 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd5813f780 0x7fbd5813eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:25.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.335+0000 7fbd5d39b700 1 --2- 172.21.15.31:0/2424111645 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbd580c7be0 0x7fbd580ac9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:25.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.335+0000 7fbd5cb9a700 1 -- 172.21.15.31:0/2424111645 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbd580c7be0 msgr2=0x7fbd580ac9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:25.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.335+0000 7fbd5cb9a700 1 --2- 172.21.15.31:0/2424111645 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbd580c7be0 0x7fbd580ac9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:25.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.335+0000 7fbd5cb9a700 1 -- 172.21.15.31:0/2424111645 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd5813f780 msgr2=0x7fbd5813eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:25.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.335+0000 7fbd5cb9a700 1 --2- 172.21.15.31:0/2424111645 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd5813f780 0x7fbd5813eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:25.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.335+0000 7fbd5cb9a700 1 -- 172.21.15.31:0/2424111645 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd4800b6d0 con 0x7fbd580c7fb0 2023-10-06T02:32:25.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.335+0000 7fbd5cb9a700 1 --2- 172.21.15.31:0/2424111645 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd580c7fb0 0x7fbd580b0a60 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7fbd4800b6a0 tx=0x7fbd4801b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:25.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.335+0000 7fbd4d7fa700 1 -- 172.21.15.31:0/2424111645 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd480106c0 con 0x7fbd580c7fb0 2023-10-06T02:32:25.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.336+0000 7fbd4d7fa700 1 -- 172.21.15.31:0/2424111645 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd48010840 con 0x7fbd580c7fb0 2023-10-06T02:32:25.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.336+0000 7fbd4d7fa700 1 -- 172.21.15.31:0/2424111645 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd48010c00 con 0x7fbd580c7fb0 2023-10-06T02:32:25.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.336+0000 7fbd5e39d700 1 -- 172.21.15.31:0/2424111645 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd580b11d0 con 0x7fbd580c7fb0 2023-10-06T02:32:25.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.336+0000 7fbd5e39d700 1 -- 172.21.15.31:0/2424111645 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd580b1790 con 0x7fbd580c7fb0 2023-10-06T02:32:25.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.337+0000 7fbd5e39d700 1 -- 172.21.15.31:0/2424111645 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd580c4330 con 0x7fbd580c7fb0 2023-10-06T02:32:25.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.337+0000 7fbd4d7fa700 1 -- 172.21.15.31:0/2424111645 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) v1 ==== 85709+0+0 (secure 0 0 0) 0x7fbd48024420 con 0x7fbd580c7fb0 2023-10-06T02:32:25.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.338+0000 7fbd4d7fa700 1 --2- 172.21.15.31:0/2424111645 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fbd3c0679e0 0x7fbd3c069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:25.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.338+0000 7fbd4d7fa700 1 -- 172.21.15.31:0/2424111645 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fbd48020070 con 0x7fbd580c7fb0 2023-10-06T02:32:25.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.339+0000 7fbd5d39b700 1 --2- 172.21.15.31:0/2424111645 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fbd3c0679e0 0x7fbd3c069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:25.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.339+0000 7fbd5d39b700 1 --2- 172.21.15.31:0/2424111645 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fbd3c0679e0 0x7fbd3c069e90 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fbd54001820 tx=0x7fbd54009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:25.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.343+0000 7fbd4d7fa700 1 -- 172.21.15.31:0/2424111645 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbd580c4330 con 0x7fbd580c7fb0 2023-10-06T02:32:25.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.543+0000 7fbd5e39d700 1 -- 172.21.15.31:0/2424111645 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbd580c4330 con 0x7fbd580c7fb0 2023-10-06T02:32:25.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.544+0000 7fbd4d7fa700 1 -- 172.21.15.31:0/2424111645 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v169) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbd580c4330 con 0x7fbd580c7fb0 2023-10-06T02:32:25.545 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-10-06T02:32:25.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.546+0000 7fbd5e39d700 1 -- 172.21.15.31:0/2424111645 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fbd3c0679e0 msgr2=0x7fbd3c069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:25.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.546+0000 7fbd5e39d700 1 --2- 172.21.15.31:0/2424111645 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fbd3c0679e0 0x7fbd3c069e90 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fbd54001820 tx=0x7fbd54009040 comp rx=0 tx=0).stop 2023-10-06T02:32:25.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.546+0000 7fbd5e39d700 1 -- 172.21.15.31:0/2424111645 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd580c7fb0 msgr2=0x7fbd580b0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:25.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.546+0000 7fbd5e39d700 1 --2- 172.21.15.31:0/2424111645 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd580c7fb0 0x7fbd580b0a60 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7fbd4800b6a0 tx=0x7fbd4801b000 comp rx=0 tx=0).stop 2023-10-06T02:32:25.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.546+0000 7fbd5e39d700 1 -- 172.21.15.31:0/2424111645 shutdown_connections 2023-10-06T02:32:25.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.546+0000 7fbd5e39d700 1 --2- 172.21.15.31:0/2424111645 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd5813f780 0x7fbd5813eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:25.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.546+0000 7fbd5e39d700 1 --2- 172.21.15.31:0/2424111645 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fbd3c0679e0 0x7fbd3c069e90 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:25.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.546+0000 7fbd5e39d700 1 --2- 172.21.15.31:0/2424111645 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbd580c7be0 0x7fbd580ac9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:25.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.546+0000 7fbd5e39d700 1 --2- 172.21.15.31:0/2424111645 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd580c7fb0 0x7fbd580b0a60 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:25.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.546+0000 7fbd5e39d700 1 -- 172.21.15.31:0/2424111645 >> 172.21.15.31:0/2424111645 conn(0x7fbd5806d110 msgr2=0x7fbd580cad20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:25.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.546+0000 7fbd5e39d700 1 -- 172.21.15.31:0/2424111645 shutdown_connections 2023-10-06T02:32:25.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.546+0000 7fbd5e39d700 1 -- 172.21.15.31:0/2424111645 wait complete. 2023-10-06T02:32:25.565 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T02:32:25.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.754+0000 7f2cfd99d700 1 Processor -- start 2023-10-06T02:32:25.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.755+0000 7f2cfd99d700 1 -- start start 2023-10-06T02:32:25.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.756+0000 7f2cfd99d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2cf80c7be0 0x7f2cf806abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:25.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.756+0000 7f2cfd99d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2cf80c7fb0 0x7f2cf815a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:25.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.756+0000 7f2cfd99d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cf815aed0 con 0x7f2cf813f780 2023-10-06T02:32:25.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.756+0000 7f2cfd99d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cf806bf10 con 0x7f2cf80c7fb0 2023-10-06T02:32:25.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.756+0000 7f2cfd99d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf806b290 con 0x7f2cf80c7be0 2023-10-06T02:32:25.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.756+0000 7f2cf7fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2cf80c7fb0 0x7f2cf815a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:25.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.756+0000 7f2cfd19c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f2cf813f780 0x7f2cf80c6770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56304/0 (socket says 172.21.15.31:56304) 2023-10-06T02:32:25.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.756+0000 7f2cf7fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2cf80c7fb0 0x7f2cf815a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39072/0 (socket says 172.21.15.31:39072) 2023-10-06T02:32:25.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.756+0000 7f2cfd19c700 1 -- 172.21.15.31:0/2099166575 learned_addr learned my addr 172.21.15.31:0/2099166575 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:32:25.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.756+0000 7f2cfc99b700 1 --2- 172.21.15.31:0/2099166575 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2cf80c7be0 0x7f2cf806abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:25.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.757+0000 7f2cf7fff700 1 -- 172.21.15.31:0/2099166575 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2cf80c7be0 msgr2=0x7f2cf806abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:25.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.757+0000 7f2cf7fff700 1 --2- 172.21.15.31:0/2099166575 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2cf80c7be0 0x7f2cf806abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:25.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.757+0000 7f2cf7fff700 1 -- 172.21.15.31:0/2099166575 >> v1:172.21.15.31:6789/0 conn(0x7f2cf813f780 legacy=0x7f2cf80c6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:25.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.757+0000 7f2cf7fff700 1 -- 172.21.15.31:0/2099166575 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cf806b570 con 0x7f2cf80c7fb0 2023-10-06T02:32:25.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.757+0000 7f2cf7fff700 1 --2- 172.21.15.31:0/2099166575 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2cf80c7fb0 0x7f2cf815a990 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f2ce40098e0 tx=0x7f2ce40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b323fe2cb37b44f4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:25.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.757+0000 7f2cf77fe700 1 -- 172.21.15.31:0/2099166575 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce4009e80 con 0x7f2cf80c7fb0 2023-10-06T02:32:25.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.757+0000 7f2cf77fe700 1 -- 172.21.15.31:0/2099166575 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ce400b860 con 0x7f2cf80c7fb0 2023-10-06T02:32:25.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.757+0000 7f2cf77fe700 1 -- 172.21.15.31:0/2099166575 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce400bbe0 con 0x7f2cf80c7fb0 2023-10-06T02:32:25.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.758+0000 7f2cfd99d700 1 -- 172.21.15.31:0/2099166575 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2cf80c7fb0 msgr2=0x7f2cf815a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:25.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.758+0000 7f2cfd99d700 1 --2- 172.21.15.31:0/2099166575 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2cf80c7fb0 0x7f2cf815a990 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f2ce40098e0 tx=0x7f2ce40091a0 comp rx=0 tx=0).stop 2023-10-06T02:32:25.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.758+0000 7f2cfd99d700 1 -- 172.21.15.31:0/2099166575 shutdown_connections 2023-10-06T02:32:25.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.758+0000 7f2cfd99d700 1 --2- 172.21.15.31:0/2099166575 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2cf80c7be0 0x7f2cf806abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:25.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.758+0000 7f2cfd99d700 1 --2- 172.21.15.31:0/2099166575 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2cf80c7fb0 0x7f2cf815a990 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:25.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.758+0000 7f2cfd99d700 1 -- 172.21.15.31:0/2099166575 >> 172.21.15.31:0/2099166575 conn(0x7f2cf806d110 msgr2=0x7f2cf80cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:25.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.758+0000 7f2cfd99d700 1 -- 172.21.15.31:0/2099166575 shutdown_connections 2023-10-06T02:32:25.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.758+0000 7f2cfd99d700 1 -- 172.21.15.31:0/2099166575 wait complete. 2023-10-06T02:32:25.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.759+0000 7f2cfd99d700 1 Processor -- start 2023-10-06T02:32:25.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.760+0000 7f2cfd99d700 1 -- start start 2023-10-06T02:32:25.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.761+0000 7f2cfd99d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2cf80c7be0 0x7f2cf813ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:25.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.761+0000 7f2cfd99d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2cf80c7fb0 0x7f2cf80c55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:25.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.761+0000 7f2cfd99d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2cf813f780 0x7f2cf80c5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:25.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.761+0000 7f2cfd99d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf816e210 con 0x7f2cf80c7be0 2023-10-06T02:32:25.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.761+0000 7f2cfd99d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf816df30 con 0x7f2cf80c7fb0 2023-10-06T02:32:25.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.761+0000 7f2cfd99d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf8156c20 con 0x7f2cf813f780 2023-10-06T02:32:25.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.761+0000 7f2cfc99b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2cf80c7be0 0x7f2cf813ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:25.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.761+0000 7f2cfc99b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2cf80c7be0 0x7f2cf813ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39078/0 (socket says 172.21.15.31:39078) 2023-10-06T02:32:25.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.761+0000 7f2cfc99b700 1 -- 172.21.15.31:0/2002014914 learned_addr learned my addr 172.21.15.31:0/2002014914 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:25.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.761+0000 7f2cfd19c700 1 --2- 172.21.15.31:0/2002014914 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2cf813f780 0x7f2cf80c5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:25.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.761+0000 7f2cf7fff700 1 --2- 172.21.15.31:0/2002014914 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2cf80c7fb0 0x7f2cf80c55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:25.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.761+0000 7f2cfc99b700 1 -- 172.21.15.31:0/2002014914 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2cf813f780 msgr2=0x7f2cf80c5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:25.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.761+0000 7f2cfc99b700 1 --2- 172.21.15.31:0/2002014914 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2cf813f780 0x7f2cf80c5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:25.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.761+0000 7f2cfc99b700 1 -- 172.21.15.31:0/2002014914 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2cf80c7fb0 msgr2=0x7f2cf80c55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:25.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.761+0000 7f2cfc99b700 1 --2- 172.21.15.31:0/2002014914 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2cf80c7fb0 0x7f2cf80c55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:25.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.761+0000 7f2cfc99b700 1 -- 172.21.15.31:0/2002014914 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ce4009420 con 0x7f2cf80c7be0 2023-10-06T02:32:25.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.761+0000 7f2cfd19c700 1 --2- 172.21.15.31:0/2002014914 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2cf813f780 0x7f2cf80c5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:32:25.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.761+0000 7f2cf7fff700 1 --2- 172.21.15.31:0/2002014914 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2cf80c7fb0 0x7f2cf80c55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:32:25.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.762+0000 7f2cfc99b700 1 --2- 172.21.15.31:0/2002014914 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2cf80c7be0 0x7f2cf813ea80 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f2cec00d890 tx=0x7f2cec008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:25.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.762+0000 7f2cf4ff9700 1 -- 172.21.15.31:0/2002014914 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cec00b600 con 0x7f2cf80c7be0 2023-10-06T02:32:25.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.762+0000 7f2cf4ff9700 1 -- 172.21.15.31:0/2002014914 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cec00b760 con 0x7f2cf80c7be0 2023-10-06T02:32:25.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.762+0000 7f2cf4ff9700 1 -- 172.21.15.31:0/2002014914 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cec00bae0 con 0x7f2cf80c7be0 2023-10-06T02:32:25.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.762+0000 7f2cfd99d700 1 -- 172.21.15.31:0/2002014914 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cf8156f00 con 0x7f2cf80c7be0 2023-10-06T02:32:25.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.762+0000 7f2cfd99d700 1 -- 172.21.15.31:0/2002014914 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cf813efc0 con 0x7f2cf80c7be0 2023-10-06T02:32:25.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.763+0000 7f2cfd99d700 1 -- 172.21.15.31:0/2002014914 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cf8157550 con 0x7f2cf80c7be0 2023-10-06T02:32:25.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.764+0000 7f2cf4ff9700 1 -- 172.21.15.31:0/2002014914 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) v1 ==== 85709+0+0 (secure 0 0 0) 0x7f2cec01e460 con 0x7f2cf80c7be0 2023-10-06T02:32:25.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.765+0000 7f2cf4ff9700 1 --2- 172.21.15.31:0/2002014914 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f2cdc0679e0 0x7f2cdc069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:25.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.765+0000 7f2cf4ff9700 1 -- 172.21.15.31:0/2002014914 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f2cec0835d0 con 0x7f2cf80c7be0 2023-10-06T02:32:25.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.765+0000 7f2cf7fff700 1 --2- 172.21.15.31:0/2002014914 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f2cdc0679e0 0x7f2cdc069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:25.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.766+0000 7f2cf7fff700 1 --2- 172.21.15.31:0/2002014914 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f2cdc0679e0 0x7f2cdc069e90 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f2cf80b1890 tx=0x7f2ce4019000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:25.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.769+0000 7f2cf4ff9700 1 -- 172.21.15.31:0/2002014914 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2cec052210 con 0x7f2cf80c7be0 2023-10-06T02:32:25.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.972+0000 7f2cfd99d700 1 -- 172.21.15.31:0/2002014914 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2cf8156a80 con 0x7f2cf80c7be0 2023-10-06T02:32:25.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.972+0000 7f2cf4ff9700 1 -- 172.21.15.31:0/2002014914 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v169) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2cf8156a80 con 0x7f2cf80c7be0 2023-10-06T02:32:25.973 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-10-06T02:32:25.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.974+0000 7f2cfd99d700 1 -- 172.21.15.31:0/2002014914 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f2cdc0679e0 msgr2=0x7f2cdc069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:25.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.974+0000 7f2cfd99d700 1 --2- 172.21.15.31:0/2002014914 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f2cdc0679e0 0x7f2cdc069e90 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f2cf80b1890 tx=0x7f2ce4019000 comp rx=0 tx=0).stop 2023-10-06T02:32:25.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.974+0000 7f2cfd99d700 1 -- 172.21.15.31:0/2002014914 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2cf80c7be0 msgr2=0x7f2cf813ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:25.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.974+0000 7f2cfd99d700 1 --2- 172.21.15.31:0/2002014914 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2cf80c7be0 0x7f2cf813ea80 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f2cec00d890 tx=0x7f2cec008040 comp rx=0 tx=0).stop 2023-10-06T02:32:25.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.974+0000 7f2cfd99d700 1 -- 172.21.15.31:0/2002014914 shutdown_connections 2023-10-06T02:32:25.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.974+0000 7f2cfd99d700 1 --2- 172.21.15.31:0/2002014914 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f2cdc0679e0 0x7f2cdc069e90 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:25.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.974+0000 7f2cfd99d700 1 --2- 172.21.15.31:0/2002014914 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2cf813f780 0x7f2cf80c5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:25.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.974+0000 7f2cfd99d700 1 --2- 172.21.15.31:0/2002014914 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2cf80c7be0 0x7f2cf813ea80 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:25.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.974+0000 7f2cfd99d700 1 --2- 172.21.15.31:0/2002014914 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2cf80c7fb0 0x7f2cf80c55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:25.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.974+0000 7f2cfd99d700 1 -- 172.21.15.31:0/2002014914 >> 172.21.15.31:0/2002014914 conn(0x7f2cf806d110 msgr2=0x7f2cf80adfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:25.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.974+0000 7f2cfd99d700 1 -- 172.21.15.31:0/2002014914 shutdown_connections 2023-10-06T02:32:25.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:25.974+0000 7f2cfd99d700 1 -- 172.21.15.31:0/2002014914 wait complete. 2023-10-06T02:32:25.992 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:32:26.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.178+0000 7fe6bffff700 1 Processor -- start 2023-10-06T02:32:26.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.178+0000 7fe6bffff700 1 -- start start 2023-10-06T02:32:26.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.180+0000 7fe6bffff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe6c0140a70 0x7fe6c00c6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:26.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.180+0000 7fe6bffff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe6c0140e40 0x7fe6c00c7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:26.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.180+0000 7fe6bffff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6c00c6d50 con 0x7fe6c00c8330 2023-10-06T02:32:26.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.180+0000 7fe6bffff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6c00c7030 con 0x7fe6c0140a70 2023-10-06T02:32:26.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.180+0000 7fe6bffff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c00c7310 con 0x7fe6c0140e40 2023-10-06T02:32:26.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.180+0000 7fe6bf7fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fe6c00c8330 0x7fe6c00aeef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56312/0 (socket says 172.21.15.31:56312) 2023-10-06T02:32:26.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.180+0000 7fe6bf7fe700 1 -- 172.21.15.31:0/1803155201 learned_addr learned my addr 172.21.15.31:0/1803155201 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:32:26.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.180+0000 7fe6be7fc700 1 --2- 172.21.15.31:0/1803155201 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe6c0140e40 0x7fe6c00c7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:26.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.180+0000 7fe6beffd700 1 --2- 172.21.15.31:0/1803155201 >> v2:172.21.15.62:3300/0 conn(0x7fe6c0140a70 0x7fe6c00c6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:26.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.180+0000 7fe6bdffb700 1 -- 172.21.15.31:0/1803155201 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3696209634 0 0) 0x7fe6c00c6d50 con 0x7fe6c00c8330 2023-10-06T02:32:26.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.181+0000 7fe6bdffb700 1 -- 172.21.15.31:0/1803155201 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6a8003570 con 0x7fe6c00c8330 2023-10-06T02:32:26.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.181+0000 7fe6be7fc700 1 -- 172.21.15.31:0/1803155201 >> v2:172.21.15.62:3300/0 conn(0x7fe6c0140a70 msgr2=0x7fe6c00c6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:26.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.181+0000 7fe6be7fc700 1 --2- 172.21.15.31:0/1803155201 >> v2:172.21.15.62:3300/0 conn(0x7fe6c0140a70 0x7fe6c00c6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:26.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.181+0000 7fe6be7fc700 1 -- 172.21.15.31:0/1803155201 >> v1:172.21.15.31:6789/0 conn(0x7fe6c00c8330 legacy=0x7fe6c00aeef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:26.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.181+0000 7fe6be7fc700 1 -- 172.21.15.31:0/1803155201 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6c00c75f0 con 0x7fe6c0140e40 2023-10-06T02:32:26.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.181+0000 7fe6be7fc700 1 --2- 172.21.15.31:0/1803155201 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe6c0140e40 0x7fe6c00c7df0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe6b40098d0 tx=0x7fe6b4009120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1fd215fa33f9ec34 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:26.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.181+0000 7fe6bdffb700 1 -- 172.21.15.31:0/1803155201 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b4005c10 con 0x7fe6c0140e40 2023-10-06T02:32:26.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.181+0000 7fe6bdffb700 1 -- 172.21.15.31:0/1803155201 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6b4005d70 con 0x7fe6c0140e40 2023-10-06T02:32:26.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.181+0000 7fe6bdffb700 1 -- 172.21.15.31:0/1803155201 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b4006260 con 0x7fe6c0140e40 2023-10-06T02:32:26.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.182+0000 7fe6bffff700 1 -- 172.21.15.31:0/1803155201 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe6c0140e40 msgr2=0x7fe6c00c7df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:26.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.182+0000 7fe6bffff700 1 --2- 172.21.15.31:0/1803155201 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe6c0140e40 0x7fe6c00c7df0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe6b40098d0 tx=0x7fe6b4009120 comp rx=0 tx=0).stop 2023-10-06T02:32:26.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.182+0000 7fe6bffff700 1 -- 172.21.15.31:0/1803155201 shutdown_connections 2023-10-06T02:32:26.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.182+0000 7fe6bffff700 1 --2- 172.21.15.31:0/1803155201 >> v2:172.21.15.62:3300/0 conn(0x7fe6c0140a70 0x7fe6c00c6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:26.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.182+0000 7fe6bffff700 1 --2- 172.21.15.31:0/1803155201 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe6c0140e40 0x7fe6c00c7df0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:26.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.182+0000 7fe6bffff700 1 -- 172.21.15.31:0/1803155201 >> 172.21.15.31:0/1803155201 conn(0x7fe6c006d170 msgr2=0x7fe6c00cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:26.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.182+0000 7fe6bffff700 1 -- 172.21.15.31:0/1803155201 shutdown_connections 2023-10-06T02:32:26.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.182+0000 7fe6bffff700 1 -- 172.21.15.31:0/1803155201 wait complete. 2023-10-06T02:32:26.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.183+0000 7fe6bffff700 1 Processor -- start 2023-10-06T02:32:26.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.184+0000 7fe6bffff700 1 -- start start 2023-10-06T02:32:26.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.185+0000 7fe6bffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe6c00c8330 0x7fe6c00ca980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:26.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.185+0000 7fe6bffff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe6c0140a70 0x7fe6c00c44c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:26.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.185+0000 7fe6bffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6c0140e40 0x7fe6c013c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:26.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.185+0000 7fe6bffff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c00c5a00 con 0x7fe6c00c8330 2023-10-06T02:32:26.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.185+0000 7fe6bffff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c00c4df0 con 0x7fe6c0140e40 2023-10-06T02:32:26.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.185+0000 7fe6bffff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c00c50d0 con 0x7fe6c0140a70 2023-10-06T02:32:26.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.185+0000 7fe6beffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe6c00c8330 0x7fe6c00ca980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:26.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.185+0000 7fe6beffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe6c00c8330 0x7fe6c00ca980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39080/0 (socket says 172.21.15.31:39080) 2023-10-06T02:32:26.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.185+0000 7fe6beffd700 1 -- 172.21.15.31:0/2759892447 learned_addr learned my addr 172.21.15.31:0/2759892447 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:26.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.185+0000 7fe6beffd700 1 -- 172.21.15.31:0/2759892447 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe6c0140a70 msgr2=0x7fe6c00c44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:26.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.185+0000 7fe6beffd700 1 --2- 172.21.15.31:0/2759892447 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe6c0140a70 0x7fe6c00c44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:26.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.185+0000 7fe6beffd700 1 -- 172.21.15.31:0/2759892447 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6c0140e40 msgr2=0x7fe6c013c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:26.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.185+0000 7fe6beffd700 1 --2- 172.21.15.31:0/2759892447 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6c0140e40 0x7fe6c013c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:26.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.185+0000 7fe6beffd700 1 -- 172.21.15.31:0/2759892447 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6b40093c0 con 0x7fe6c00c8330 2023-10-06T02:32:26.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.185+0000 7fe6beffd700 1 --2- 172.21.15.31:0/2759892447 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe6c00c8330 0x7fe6c00ca980 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7fe6b000ec80 tx=0x7fe6b00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:26.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.186+0000 7fe6a6ffd700 1 -- 172.21.15.31:0/2759892447 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b000c9d0 con 0x7fe6c00c8330 2023-10-06T02:32:26.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.186+0000 7fe6a6ffd700 1 -- 172.21.15.31:0/2759892447 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6b000cb30 con 0x7fe6c00c8330 2023-10-06T02:32:26.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.186+0000 7fe6a6ffd700 1 -- 172.21.15.31:0/2759892447 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b0013470 con 0x7fe6c00c8330 2023-10-06T02:32:26.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.186+0000 7fe6bffff700 1 -- 172.21.15.31:0/2759892447 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6c00c53b0 con 0x7fe6c00c8330 2023-10-06T02:32:26.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.186+0000 7fe6bffff700 1 -- 172.21.15.31:0/2759892447 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6c00c4a00 con 0x7fe6c00c8330 2023-10-06T02:32:26.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.187+0000 7fe6bffff700 1 -- 172.21.15.31:0/2759892447 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6c004c3c0 con 0x7fe6c00c8330 2023-10-06T02:32:26.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.188+0000 7fe6a6ffd700 1 -- 172.21.15.31:0/2759892447 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) v1 ==== 85709+0+0 (secure 0 0 0) 0x7fe6b00137e0 con 0x7fe6c00c8330 2023-10-06T02:32:26.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.189+0000 7fe6a6ffd700 1 --2- 172.21.15.31:0/2759892447 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe6ac0679e0 0x7fe6ac069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:26.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.189+0000 7fe6a6ffd700 1 -- 172.21.15.31:0/2759892447 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fe6b0083300 con 0x7fe6c00c8330 2023-10-06T02:32:26.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.189+0000 7fe6be7fc700 1 --2- 172.21.15.31:0/2759892447 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe6ac0679e0 0x7fe6ac069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:26.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.190+0000 7fe6be7fc700 1 --2- 172.21.15.31:0/2759892447 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe6ac0679e0 0x7fe6ac069e90 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fe6c013d120 tx=0x7fe6b4016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:26.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.193+0000 7fe6a6ffd700 1 -- 172.21.15.31:0/2759892447 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe6b0051f40 con 0x7fe6c00c8330 2023-10-06T02:32:26.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.441+0000 7fe6bffff700 1 -- 172.21.15.31:0/2759892447 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe6c00c78d0 con 0x7fe6c00c8330 2023-10-06T02:32:26.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.442+0000 7fe6a6ffd700 1 -- 172.21.15.31:0/2759892447 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 120 v120) v1 ==== 95+0+4601 (secure 0 0 0) 0x7fe6b0055560 con 0x7fe6c00c8330 2023-10-06T02:32:26.442 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:32:26.443 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":120,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7445,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/324072138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":324072138},{"type":"v1","addr":"172.21.15.31:6835","nonce":324072138}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":114},{"gid":7448,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/3160488596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3160488596},{"type":"v1","addr":"172.21.15.62:6834","nonce":3160488596}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":114},{"gid":8677,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/632134516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":632134516},{"type":"v1","addr":"172.21.15.62:6835","nonce":632134516}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":114}],"filesystems":[{"mdsmap":{"epoch":119,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:32:07.613387+0000","modified":"2023-10-06T02:32:11.143622+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9267},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9267":{"gid":9267,"name":"a","rank":0,"incarnation":115,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6834/3741373122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":3741373122},{"type":"v1","addr":"172.21.15.31:6834","nonce":3741373122}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2023-10-06T02:32:26.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.444+0000 7fe6bffff700 1 -- 172.21.15.31:0/2759892447 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe6ac0679e0 msgr2=0x7fe6ac069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:26.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.444+0000 7fe6bffff700 1 --2- 172.21.15.31:0/2759892447 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe6ac0679e0 0x7fe6ac069e90 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fe6c013d120 tx=0x7fe6b4016000 comp rx=0 tx=0).stop 2023-10-06T02:32:26.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.444+0000 7fe6bffff700 1 -- 172.21.15.31:0/2759892447 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe6c00c8330 msgr2=0x7fe6c00ca980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:26.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.444+0000 7fe6bffff700 1 --2- 172.21.15.31:0/2759892447 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe6c00c8330 0x7fe6c00ca980 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7fe6b000ec80 tx=0x7fe6b00091a0 comp rx=0 tx=0).stop 2023-10-06T02:32:26.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.444+0000 7fe6bffff700 1 -- 172.21.15.31:0/2759892447 shutdown_connections 2023-10-06T02:32:26.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.444+0000 7fe6bffff700 1 --2- 172.21.15.31:0/2759892447 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6c0140e40 0x7fe6c013c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:26.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.444+0000 7fe6bffff700 1 --2- 172.21.15.31:0/2759892447 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe6c00c8330 0x7fe6c00ca980 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:26.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.444+0000 7fe6bffff700 1 --2- 172.21.15.31:0/2759892447 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fe6ac0679e0 0x7fe6ac069e90 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:26.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.444+0000 7fe6bffff700 1 --2- 172.21.15.31:0/2759892447 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe6c0140a70 0x7fe6c00c44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:26.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.444+0000 7fe6bffff700 1 -- 172.21.15.31:0/2759892447 >> 172.21.15.31:0/2759892447 conn(0x7fe6c006d170 msgr2=0x7fe6c013d800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:26.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.444+0000 7fe6bffff700 1 -- 172.21.15.31:0/2759892447 shutdown_connections 2023-10-06T02:32:26.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.444+0000 7fe6bffff700 1 -- 172.21.15.31:0/2759892447 wait complete. 2023-10-06T02:32:26.446 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 120 2023-10-06T02:32:26.462 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:32:26.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.650+0000 7f4db919d700 1 Processor -- start 2023-10-06T02:32:26.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.650+0000 7f4db919d700 1 -- start start 2023-10-06T02:32:26.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.651+0000 7f4db919d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4db4153c30 0x7f4db415a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:26.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.651+0000 7f4db919d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4db415de50 0x7f4db40cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:26.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.651+0000 7f4db919d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4db40cf3c0 con 0x7f4db40cef40 2023-10-06T02:32:26.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.651+0000 7f4db919d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4db415afd0 con 0x7f4db4153c30 2023-10-06T02:32:26.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.651+0000 7f4db919d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4db415b2b0 con 0x7f4db415de50 2023-10-06T02:32:26.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.651+0000 7f4db3fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4db4153c30 0x7f4db415a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:26.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.652+0000 7f4db3fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4db4153c30 0x7f4db415a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39094/0 (socket says 172.21.15.31:39094) 2023-10-06T02:32:26.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.652+0000 7f4db3fff700 1 -- 172.21.15.31:0/170066657 learned_addr learned my addr 172.21.15.31:0/170066657 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:26.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.652+0000 7f4db3fff700 1 -- 172.21.15.31:0/170066657 >> v2:172.21.15.62:3300/0 conn(0x7f4db415de50 msgr2=0x7f4db40cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:26.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.652+0000 7f4db3fff700 1 --2- 172.21.15.31:0/170066657 >> v2:172.21.15.62:3300/0 conn(0x7f4db415de50 0x7f4db40cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:26.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.652+0000 7f4db3fff700 1 -- 172.21.15.31:0/170066657 >> v1:172.21.15.62:6789/0 conn(0x7f4db40cef40 legacy=0x7f4db406d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:26.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.652+0000 7f4db3fff700 1 -- 172.21.15.31:0/170066657 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4db415b590 con 0x7f4db4153c30 2023-10-06T02:32:26.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.652+0000 7f4db3fff700 1 --2- 172.21.15.31:0/170066657 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4db4153c30 0x7f4db415a5e0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f4db40713a0 tx=0x7f4da4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de4a82d6e2814ba7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:26.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.652+0000 7f4db2ffd700 1 -- 172.21.15.31:0/170066657 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4da400b820 con 0x7f4db4153c30 2023-10-06T02:32:26.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.652+0000 7f4db2ffd700 1 -- 172.21.15.31:0/170066657 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4da400b980 con 0x7f4db4153c30 2023-10-06T02:32:26.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.652+0000 7f4db2ffd700 1 -- 172.21.15.31:0/170066657 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4da400bd20 con 0x7f4db4153c30 2023-10-06T02:32:26.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.653+0000 7f4db919d700 1 -- 172.21.15.31:0/170066657 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4db4153c30 msgr2=0x7f4db415a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:26.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.653+0000 7f4db919d700 1 --2- 172.21.15.31:0/170066657 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4db4153c30 0x7f4db415a5e0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f4db40713a0 tx=0x7f4da4009250 comp rx=0 tx=0).stop 2023-10-06T02:32:26.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.653+0000 7f4db919d700 1 -- 172.21.15.31:0/170066657 shutdown_connections 2023-10-06T02:32:26.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.653+0000 7f4db919d700 1 --2- 172.21.15.31:0/170066657 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4db4153c30 0x7f4db415a5e0 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:26.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.653+0000 7f4db919d700 1 --2- 172.21.15.31:0/170066657 >> v2:172.21.15.62:3300/0 conn(0x7f4db415de50 0x7f4db40cea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:26.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.653+0000 7f4db919d700 1 -- 172.21.15.31:0/170066657 >> 172.21.15.31:0/170066657 conn(0x7f4db40703a0 msgr2=0x7f4db40707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:26.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.653+0000 7f4db919d700 1 -- 172.21.15.31:0/170066657 shutdown_connections 2023-10-06T02:32:26.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.653+0000 7f4db919d700 1 -- 172.21.15.31:0/170066657 wait complete. 2023-10-06T02:32:26.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.655+0000 7f4db919d700 1 Processor -- start 2023-10-06T02:32:26.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.655+0000 7f4db919d700 1 -- start start 2023-10-06T02:32:26.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.655+0000 7f4db919d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4db40cef40 0x7f4db416a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:26.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.656+0000 7f4db919d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4db4153c30 0x7f4db40b1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:26.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.656+0000 7f4db919d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4db415de50 0x7f4db40b1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:26.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.656+0000 7f4db919d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4db4151d90 con 0x7f4db4153c30 2023-10-06T02:32:26.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.656+0000 7f4db919d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4db40ca0d0 con 0x7f4db415de50 2023-10-06T02:32:26.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.656+0000 7f4db919d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4db416aa50 con 0x7f4db40cef40 2023-10-06T02:32:26.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.656+0000 7f4db37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4db4153c30 0x7f4db40b1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:26.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.656+0000 7f4db37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4db4153c30 0x7f4db40b1590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39104/0 (socket says 172.21.15.31:39104) 2023-10-06T02:32:26.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.656+0000 7f4db37fe700 1 -- 172.21.15.31:0/1387073783 learned_addr learned my addr 172.21.15.31:0/1387073783 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:26.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.656+0000 7f4db3fff700 1 --2- 172.21.15.31:0/1387073783 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4db40cef40 0x7f4db416a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:26.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.656+0000 7f4db899c700 1 --2- 172.21.15.31:0/1387073783 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4db415de50 0x7f4db40b1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:26.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.656+0000 7f4db37fe700 1 -- 172.21.15.31:0/1387073783 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4db40cef40 msgr2=0x7f4db416a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:26.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.656+0000 7f4db37fe700 1 --2- 172.21.15.31:0/1387073783 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4db40cef40 0x7f4db416a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:26.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.656+0000 7f4db37fe700 1 -- 172.21.15.31:0/1387073783 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4db415de50 msgr2=0x7f4db40b1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:26.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.656+0000 7f4db37fe700 1 --2- 172.21.15.31:0/1387073783 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4db415de50 0x7f4db40b1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:26.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.656+0000 7f4db37fe700 1 -- 172.21.15.31:0/1387073783 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4da40094f0 con 0x7f4db4153c30 2023-10-06T02:32:26.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.657+0000 7f4db37fe700 1 --2- 172.21.15.31:0/1387073783 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4db4153c30 0x7f4db40b1590 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f4da800b810 tx=0x7f4da8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:26.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.657+0000 7f4d9bfff700 1 -- 172.21.15.31:0/1387073783 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4da800f510 con 0x7f4db4153c30 2023-10-06T02:32:26.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.657+0000 7f4d9bfff700 1 -- 172.21.15.31:0/1387073783 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4da800f690 con 0x7f4db4153c30 2023-10-06T02:32:26.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.657+0000 7f4d9bfff700 1 -- 172.21.15.31:0/1387073783 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4da800fa50 con 0x7f4db4153c30 2023-10-06T02:32:26.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.657+0000 7f4db919d700 1 -- 172.21.15.31:0/1387073783 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4db40c9520 con 0x7f4db4153c30 2023-10-06T02:32:26.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.657+0000 7f4db919d700 1 -- 172.21.15.31:0/1387073783 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4db40c9a80 con 0x7f4db4153c30 2023-10-06T02:32:26.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.658+0000 7f4db919d700 1 -- 172.21.15.31:0/1387073783 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4db40ca460 con 0x7f4db4153c30 2023-10-06T02:32:26.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.659+0000 7f4d9bfff700 1 -- 172.21.15.31:0/1387073783 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) v1 ==== 85709+0+0 (secure 0 0 0) 0x7f4da801a400 con 0x7f4db4153c30 2023-10-06T02:32:26.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.660+0000 7f4d9bfff700 1 --2- 172.21.15.31:0/1387073783 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f4da00679e0 0x7f4da0069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:26.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.660+0000 7f4d9bfff700 1 -- 172.21.15.31:0/1387073783 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f4da80829a0 con 0x7f4db4153c30 2023-10-06T02:32:26.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.660+0000 7f4db3fff700 1 --2- 172.21.15.31:0/1387073783 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f4da00679e0 0x7f4da0069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:26.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.661+0000 7f4db3fff700 1 --2- 172.21.15.31:0/1387073783 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f4da00679e0 0x7f4da0069e90 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f4db40713a0 tx=0x7f4da4016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:26.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.664+0000 7f4d9bfff700 1 -- 172.21.15.31:0/1387073783 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4da8051660 con 0x7f4db4153c30 2023-10-06T02:32:26.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.866+0000 7f4db919d700 1 -- 172.21.15.31:0/1387073783 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4db404c3c0 con 0x7f4db4153c30 2023-10-06T02:32:26.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.867+0000 7f4d9bfff700 1 -- 172.21.15.31:0/1387073783 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v169) v1 ==== 74+0+16876 (secure 0 0 0) 0x7f4da8054c80 con 0x7f4db4153c30 2023-10-06T02:32:26.868 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:32:26.868 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":169,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:32:09.112135+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:32:05.275284+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"168","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":28,"pool_name":"cephfs_data","create_time":"2023-10-06T02:32:06.591889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"169","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":20,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:32:26.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.869+0000 7f4db919d700 1 -- 172.21.15.31:0/1387073783 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f4da00679e0 msgr2=0x7f4da0069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:26.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.869+0000 7f4db919d700 1 --2- 172.21.15.31:0/1387073783 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f4da00679e0 0x7f4da0069e90 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f4db40713a0 tx=0x7f4da4016000 comp rx=0 tx=0).stop 2023-10-06T02:32:26.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.869+0000 7f4db919d700 1 -- 172.21.15.31:0/1387073783 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4db4153c30 msgr2=0x7f4db40b1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:26.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.869+0000 7f4db919d700 1 --2- 172.21.15.31:0/1387073783 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4db4153c30 0x7f4db40b1590 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f4da800b810 tx=0x7f4da8009250 comp rx=0 tx=0).stop 2023-10-06T02:32:26.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.869+0000 7f4db919d700 1 -- 172.21.15.31:0/1387073783 shutdown_connections 2023-10-06T02:32:26.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.869+0000 7f4db919d700 1 --2- 172.21.15.31:0/1387073783 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4db415de50 0x7f4db40b1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:26.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.869+0000 7f4db919d700 1 --2- 172.21.15.31:0/1387073783 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f4da00679e0 0x7f4da0069e90 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:26.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.869+0000 7f4db919d700 1 --2- 172.21.15.31:0/1387073783 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4db40cef40 0x7f4db416a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:26.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.869+0000 7f4db919d700 1 --2- 172.21.15.31:0/1387073783 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4db4153c30 0x7f4db40b1590 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:26.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.869+0000 7f4db919d700 1 -- 172.21.15.31:0/1387073783 >> 172.21.15.31:0/1387073783 conn(0x7f4db40703a0 msgr2=0x7f4db40c6ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:26.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.869+0000 7f4db919d700 1 -- 172.21.15.31:0/1387073783 shutdown_connections 2023-10-06T02:32:26.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:26.869+0000 7f4db919d700 1 -- 172.21.15.31:0/1387073783 wait complete. 2023-10-06T02:32:26.888 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T02:32:26.888 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:32:27.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.075+0000 7f244d79d700 1 Processor -- start 2023-10-06T02:32:27.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.076+0000 7f244d79d700 1 -- start start 2023-10-06T02:32:27.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.077+0000 7f244d79d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2448159640 0x7f244806d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:27.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.077+0000 7f244d79d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2448159a10 0x7f24480ca7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:27.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.077+0000 7f244d79d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2448070410 0x7f24480cacf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:27.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.077+0000 7f244d79d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24480c7420 con 0x7f2448159640 2023-10-06T02:32:27.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.077+0000 7f244d79d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f244806d5a0 con 0x7f2448070410 2023-10-06T02:32:27.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.077+0000 7f244d79d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f244806d820 con 0x7f2448159a10 2023-10-06T02:32:27.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.077+0000 7f2447fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2448159640 0x7f244806d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:27.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.077+0000 7f2447fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2448159640 0x7f244806d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39114/0 (socket says 172.21.15.31:39114) 2023-10-06T02:32:27.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.077+0000 7f2447fff700 1 -- 172.21.15.31:0/1616982119 learned_addr learned my addr 172.21.15.31:0/1616982119 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:27.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.077+0000 7f24477fe700 1 --2- 172.21.15.31:0/1616982119 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2448159a10 0x7f24480ca7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:27.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.078+0000 7f244cf9c700 1 --2- 172.21.15.31:0/1616982119 >> v2:172.21.15.62:3300/0 conn(0x7f2448070410 0x7f24480cacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:27.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.078+0000 7f2447fff700 1 -- 172.21.15.31:0/1616982119 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2448159a10 msgr2=0x7f24480ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:27.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.078+0000 7f2447fff700 1 --2- 172.21.15.31:0/1616982119 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2448159a10 0x7f24480ca7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:27.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.078+0000 7f2447fff700 1 -- 172.21.15.31:0/1616982119 >> v2:172.21.15.62:3300/0 conn(0x7f2448070410 msgr2=0x7f24480cacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:27.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.078+0000 7f2447fff700 1 --2- 172.21.15.31:0/1616982119 >> v2:172.21.15.62:3300/0 conn(0x7f2448070410 0x7f24480cacf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:27.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.078+0000 7f2447fff700 1 -- 172.21.15.31:0/1616982119 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f244806db00 con 0x7f2448159640 2023-10-06T02:32:27.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.078+0000 7f24477fe700 1 --2- 172.21.15.31:0/1616982119 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2448159a10 0x7f24480ca7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:32:27.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.078+0000 7f244cf9c700 1 --2- 172.21.15.31:0/1616982119 >> v2:172.21.15.62:3300/0 conn(0x7f2448070410 0x7f24480cacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:32:27.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.078+0000 7f2447fff700 1 --2- 172.21.15.31:0/1616982119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2448159640 0x7f244806d060 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f2448070990 tx=0x7f2438009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=222e1920782f8b41 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:27.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.078+0000 7f2446ffd700 1 -- 172.21.15.31:0/1616982119 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243800b860 con 0x7f2448159640 2023-10-06T02:32:27.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.078+0000 7f2446ffd700 1 -- 172.21.15.31:0/1616982119 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f243800b9c0 con 0x7f2448159640 2023-10-06T02:32:27.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.078+0000 7f2446ffd700 1 -- 172.21.15.31:0/1616982119 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243800bd60 con 0x7f2448159640 2023-10-06T02:32:27.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.079+0000 7f244d79d700 1 -- 172.21.15.31:0/1616982119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2448159640 msgr2=0x7f244806d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:27.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.079+0000 7f244d79d700 1 --2- 172.21.15.31:0/1616982119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2448159640 0x7f244806d060 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f2448070990 tx=0x7f2438009250 comp rx=0 tx=0).stop 2023-10-06T02:32:27.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.079+0000 7f244d79d700 1 -- 172.21.15.31:0/1616982119 shutdown_connections 2023-10-06T02:32:27.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.079+0000 7f244d79d700 1 --2- 172.21.15.31:0/1616982119 >> v2:172.21.15.62:3300/0 conn(0x7f2448070410 0x7f24480cacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:27.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.079+0000 7f244d79d700 1 --2- 172.21.15.31:0/1616982119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2448159640 0x7f244806d060 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:27.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.079+0000 7f244d79d700 1 --2- 172.21.15.31:0/1616982119 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2448159a10 0x7f24480ca7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:27.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.079+0000 7f244d79d700 1 -- 172.21.15.31:0/1616982119 >> 172.21.15.31:0/1616982119 conn(0x7f244806e720 msgr2=0x7f244806eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:27.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.079+0000 7f244d79d700 1 -- 172.21.15.31:0/1616982119 shutdown_connections 2023-10-06T02:32:27.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.079+0000 7f244d79d700 1 -- 172.21.15.31:0/1616982119 wait complete. 2023-10-06T02:32:27.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.080+0000 7f244d79d700 1 Processor -- start 2023-10-06T02:32:27.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.080+0000 7f244d79d700 1 -- start start 2023-10-06T02:32:27.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.081+0000 7f244d79d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2448070410 0x7f2448153ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:27.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.081+0000 7f244d79d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2448159640 0x7f244816d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:27.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.081+0000 7f244d79d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2448159a10 0x7f244816dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:27.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.081+0000 7f244d79d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f244816a0e0 con 0x7f2448070410 2023-10-06T02:32:27.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.081+0000 7f244d79d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2448154470 con 0x7f2448159640 2023-10-06T02:32:27.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.081+0000 7f244d79d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2448154750 con 0x7f2448159a10 2023-10-06T02:32:27.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.081+0000 7f2447fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2448070410 0x7f2448153ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:27.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.082+0000 7f2447fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2448070410 0x7f2448153ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39116/0 (socket says 172.21.15.31:39116) 2023-10-06T02:32:27.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.082+0000 7f2447fff700 1 -- 172.21.15.31:0/1597269558 learned_addr learned my addr 172.21.15.31:0/1597269558 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:27.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.082+0000 7f24477fe700 1 --2- 172.21.15.31:0/1597269558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2448159640 0x7f244816d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:27.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.082+0000 7f244cf9c700 1 --2- 172.21.15.31:0/1597269558 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2448159a10 0x7f244816dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:27.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.082+0000 7f2447fff700 1 -- 172.21.15.31:0/1597269558 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2448159a10 msgr2=0x7f244816dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:27.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.082+0000 7f2447fff700 1 --2- 172.21.15.31:0/1597269558 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2448159a10 0x7f244816dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:27.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.082+0000 7f2447fff700 1 -- 172.21.15.31:0/1597269558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2448159640 msgr2=0x7f244816d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:27.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.082+0000 7f2447fff700 1 --2- 172.21.15.31:0/1597269558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2448159640 0x7f244816d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:27.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.082+0000 7f2447fff700 1 -- 172.21.15.31:0/1597269558 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24380094f0 con 0x7f2448070410 2023-10-06T02:32:27.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.082+0000 7f24477fe700 1 --2- 172.21.15.31:0/1597269558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2448159640 0x7f244816d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:32:27.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.082+0000 7f2447fff700 1 --2- 172.21.15.31:0/1597269558 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2448070410 0x7f2448153ab0 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f24480c3820 tx=0x7f2438017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:27.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.082+0000 7f242ffff700 1 -- 172.21.15.31:0/1597269558 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2438024440 con 0x7f2448070410 2023-10-06T02:32:27.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.082+0000 7f242ffff700 1 -- 172.21.15.31:0/1597269558 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24380245a0 con 0x7f2448070410 2023-10-06T02:32:27.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.082+0000 7f242ffff700 1 -- 172.21.15.31:0/1597269558 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2438024960 con 0x7f2448070410 2023-10-06T02:32:27.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.082+0000 7f244d79d700 1 -- 172.21.15.31:0/1597269558 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2448154a30 con 0x7f2448070410 2023-10-06T02:32:27.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.083+0000 7f244d79d700 1 -- 172.21.15.31:0/1597269558 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2448153ff0 con 0x7f2448070410 2023-10-06T02:32:27.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.084+0000 7f244d79d700 1 -- 172.21.15.31:0/1597269558 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24480afbe0 con 0x7f2448070410 2023-10-06T02:32:27.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.084+0000 7f242ffff700 1 -- 172.21.15.31:0/1597269558 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) v1 ==== 85709+0+0 (secure 0 0 0) 0x7f243801a420 con 0x7f2448070410 2023-10-06T02:32:27.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.086+0000 7f242ffff700 1 --2- 172.21.15.31:0/1597269558 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f24340679e0 0x7f2434069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:27.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.086+0000 7f242ffff700 1 -- 172.21.15.31:0/1597269558 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f2438085020 con 0x7f2448070410 2023-10-06T02:32:27.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.086+0000 7f24477fe700 1 --2- 172.21.15.31:0/1597269558 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f24340679e0 0x7f2434069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:27.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.087+0000 7f24477fe700 1 --2- 172.21.15.31:0/1597269558 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f24340679e0 0x7f2434069e90 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f2448155040 tx=0x7f243c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:27.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.090+0000 7f242ffff700 1 -- 172.21.15.31:0/1597269558 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f24480afbe0 con 0x7f2448070410 2023-10-06T02:32:27.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.337+0000 7f244d79d700 1 -- 172.21.15.31:0/1597269558 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f24480afbe0 con 0x7f2448070410 2023-10-06T02:32:27.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.337+0000 7f242ffff700 1 -- 172.21.15.31:0/1597269558 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 120 v120) v1 ==== 95+0+4601 (secure 0 0 0) 0x7f24480afbe0 con 0x7f2448070410 2023-10-06T02:32:27.338 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:32:27.338 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":120,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7445,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/324072138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":324072138},{"type":"v1","addr":"172.21.15.31:6835","nonce":324072138}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":114},{"gid":7448,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/3160488596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3160488596},{"type":"v1","addr":"172.21.15.62:6834","nonce":3160488596}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":114},{"gid":8677,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/632134516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":632134516},{"type":"v1","addr":"172.21.15.62:6835","nonce":632134516}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":114}],"filesystems":[{"mdsmap":{"epoch":119,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:32:07.613387+0000","modified":"2023-10-06T02:32:11.143622+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9267},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9267":{"gid":9267,"name":"a","rank":0,"incarnation":115,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6834/3741373122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":3741373122},{"type":"v1","addr":"172.21.15.31:6834","nonce":3741373122}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2023-10-06T02:32:27.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.340+0000 7f244d79d700 1 -- 172.21.15.31:0/1597269558 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f24340679e0 msgr2=0x7f2434069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:27.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.340+0000 7f244d79d700 1 --2- 172.21.15.31:0/1597269558 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f24340679e0 0x7f2434069e90 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f2448155040 tx=0x7f243c008040 comp rx=0 tx=0).stop 2023-10-06T02:32:27.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.340+0000 7f244d79d700 1 -- 172.21.15.31:0/1597269558 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2448070410 msgr2=0x7f2448153ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:27.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.340+0000 7f244d79d700 1 --2- 172.21.15.31:0/1597269558 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2448070410 0x7f2448153ab0 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f24480c3820 tx=0x7f2438017000 comp rx=0 tx=0).stop 2023-10-06T02:32:27.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.340+0000 7f244d79d700 1 -- 172.21.15.31:0/1597269558 shutdown_connections 2023-10-06T02:32:27.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.340+0000 7f244d79d700 1 --2- 172.21.15.31:0/1597269558 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f24340679e0 0x7f2434069e90 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:27.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.340+0000 7f244d79d700 1 --2- 172.21.15.31:0/1597269558 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2448159a10 0x7f244816dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:27.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.340+0000 7f244d79d700 1 --2- 172.21.15.31:0/1597269558 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2448070410 0x7f2448153ab0 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:27.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.340+0000 7f244d79d700 1 --2- 172.21.15.31:0/1597269558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2448159640 0x7f244816d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:27.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.340+0000 7f244d79d700 1 -- 172.21.15.31:0/1597269558 >> 172.21.15.31:0/1597269558 conn(0x7f244806e720 msgr2=0x7f2448157c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:27.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.340+0000 7f244d79d700 1 -- 172.21.15.31:0/1597269558 shutdown_connections 2023-10-06T02:32:27.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.340+0000 7f244d79d700 1 -- 172.21.15.31:0/1597269558 wait complete. 2023-10-06T02:32:27.342 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 120 2023-10-06T02:32:27.358 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:32:27.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.544+0000 7f5e8319d700 1 Processor -- start 2023-10-06T02:32:27.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.544+0000 7f5e8319d700 1 -- start start 2023-10-06T02:32:27.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.545+0000 7f5e8319d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5e7c15b5d0 0x7f5e7c0c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:27.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.545+0000 7f5e8319d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5e7c0ca240 0x7f5e7c0c6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:27.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.545+0000 7f5e8319d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e7c0ca6b0 con 0x7f5e7c0c6ec0 2023-10-06T02:32:27.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.545+0000 7f5e8319d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e7c0cb4f0 con 0x7f5e7c0ca240 2023-10-06T02:32:27.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.545+0000 7f5e8319d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7c1601e0 con 0x7f5e7c15b5d0 2023-10-06T02:32:27.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.545+0000 7f5e8299c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f5e7c0c6ec0 0x7f5e7c06de50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56326/0 (socket says 172.21.15.31:56326) 2023-10-06T02:32:27.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.545+0000 7f5e8199a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5e7c0ca240 0x7f5e7c0c6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:27.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.545+0000 7f5e8299c700 1 -- 172.21.15.31:0/3346872134 learned_addr learned my addr 172.21.15.31:0/3346872134 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:32:27.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.546+0000 7f5e8219b700 1 --2- 172.21.15.31:0/3346872134 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5e7c15b5d0 0x7f5e7c0c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:27.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.546+0000 7f5e81199700 1 -- 172.21.15.31:0/3346872134 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2850453986 0 0) 0x7f5e7c0ca6b0 con 0x7f5e7c0c6ec0 2023-10-06T02:32:27.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.546+0000 7f5e8199a700 1 -- 172.21.15.31:0/3346872134 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5e7c15b5d0 msgr2=0x7f5e7c0c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:27.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.546+0000 7f5e8199a700 1 --2- 172.21.15.31:0/3346872134 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5e7c15b5d0 0x7f5e7c0c9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:27.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.546+0000 7f5e8199a700 1 -- 172.21.15.31:0/3346872134 >> v1:172.21.15.31:6789/0 conn(0x7f5e7c0c6ec0 legacy=0x7f5e7c06de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:27.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.546+0000 7f5e8199a700 1 -- 172.21.15.31:0/3346872134 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e7c15e390 con 0x7f5e7c0ca240 2023-10-06T02:32:27.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.546+0000 7f5e8199a700 1 --2- 172.21.15.31:0/3346872134 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e7c0ca240 0x7f5e7c0c6980 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c0098d0 tx=0x7f5e6c009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e665cf08b274609 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:27.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.546+0000 7f5e81199700 1 -- 172.21.15.31:0/3346872134 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c005c10 con 0x7f5e7c0ca240 2023-10-06T02:32:27.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.547+0000 7f5e81199700 1 -- 172.21.15.31:0/3346872134 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e6c005d70 con 0x7f5e7c0ca240 2023-10-06T02:32:27.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.547+0000 7f5e81199700 1 -- 172.21.15.31:0/3346872134 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c006280 con 0x7f5e7c0ca240 2023-10-06T02:32:27.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.547+0000 7f5e8319d700 1 -- 172.21.15.31:0/3346872134 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e7c0ca240 msgr2=0x7f5e7c0c6980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:27.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.547+0000 7f5e8319d700 1 --2- 172.21.15.31:0/3346872134 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e7c0ca240 0x7f5e7c0c6980 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c0098d0 tx=0x7f5e6c009120 comp rx=0 tx=0).stop 2023-10-06T02:32:27.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.547+0000 7f5e8319d700 1 -- 172.21.15.31:0/3346872134 shutdown_connections 2023-10-06T02:32:27.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.547+0000 7f5e8319d700 1 --2- 172.21.15.31:0/3346872134 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5e7c15b5d0 0x7f5e7c0c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:27.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.547+0000 7f5e8319d700 1 --2- 172.21.15.31:0/3346872134 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e7c0ca240 0x7f5e7c0c6980 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:27.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.547+0000 7f5e8319d700 1 -- 172.21.15.31:0/3346872134 >> 172.21.15.31:0/3346872134 conn(0x7f5e7c06d170 msgr2=0x7f5e7c0cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:27.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.547+0000 7f5e8319d700 1 -- 172.21.15.31:0/3346872134 shutdown_connections 2023-10-06T02:32:27.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.547+0000 7f5e8319d700 1 -- 172.21.15.31:0/3346872134 wait complete. 2023-10-06T02:32:27.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.549+0000 7f5e8319d700 1 Processor -- start 2023-10-06T02:32:27.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.549+0000 7f5e8319d700 1 -- start start 2023-10-06T02:32:27.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.550+0000 7f5e8319d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e7c0c6ec0 0x7f5e7c0c66e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:27.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.550+0000 7f5e8319d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5e7c0ca240 0x7f5e7c0b17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:27.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.550+0000 7f5e8319d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e7c15b5d0 0x7f5e7c163f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:27.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.550+0000 7f5e8319d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7c1647c0 con 0x7f5e7c15b5d0 2023-10-06T02:32:27.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.550+0000 7f5e8319d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7c0c3cb0 con 0x7f5e7c0c6ec0 2023-10-06T02:32:27.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.550+0000 7f5e8319d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7c0b1d10 con 0x7f5e7c0ca240 2023-10-06T02:32:27.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.550+0000 7f5e8299c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e7c15b5d0 0x7f5e7c163f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:27.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.550+0000 7f5e8299c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e7c15b5d0 0x7f5e7c163f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39136/0 (socket says 172.21.15.31:39136) 2023-10-06T02:32:27.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.550+0000 7f5e8299c700 1 -- 172.21.15.31:0/2291286172 learned_addr learned my addr 172.21.15.31:0/2291286172 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:27.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.550+0000 7f5e8219b700 1 --2- 172.21.15.31:0/2291286172 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e7c0c6ec0 0x7f5e7c0c66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:27.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.550+0000 7f5e8299c700 1 -- 172.21.15.31:0/2291286172 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5e7c0ca240 msgr2=0x7f5e7c0b17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:27.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.550+0000 7f5e8199a700 1 --2- 172.21.15.31:0/2291286172 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5e7c0ca240 0x7f5e7c0b17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:27.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.550+0000 7f5e8299c700 1 --2- 172.21.15.31:0/2291286172 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5e7c0ca240 0x7f5e7c0b17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:27.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.550+0000 7f5e8299c700 1 -- 172.21.15.31:0/2291286172 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e7c0c6ec0 msgr2=0x7f5e7c0c66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:27.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.550+0000 7f5e8299c700 1 --2- 172.21.15.31:0/2291286172 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e7c0c6ec0 0x7f5e7c0c66e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:27.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.550+0000 7f5e8299c700 1 -- 172.21.15.31:0/2291286172 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e6c0093c0 con 0x7f5e7c15b5d0 2023-10-06T02:32:27.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.550+0000 7f5e8219b700 1 --2- 172.21.15.31:0/2291286172 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e7c0c6ec0 0x7f5e7c0c66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:32:27.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.550+0000 7f5e8199a700 1 --2- 172.21.15.31:0/2291286172 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5e7c0ca240 0x7f5e7c0b17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:32:27.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.551+0000 7f5e8299c700 1 --2- 172.21.15.31:0/2291286172 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e7c15b5d0 0x7f5e7c163f50 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f5e7400ccc0 tx=0x7f5e74001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:27.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.551+0000 7f5e727fc700 1 -- 172.21.15.31:0/2291286172 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e7400ab30 con 0x7f5e7c15b5d0 2023-10-06T02:32:27.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.551+0000 7f5e727fc700 1 -- 172.21.15.31:0/2291286172 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e7400ac90 con 0x7f5e7c15b5d0 2023-10-06T02:32:27.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.551+0000 7f5e727fc700 1 -- 172.21.15.31:0/2291286172 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e74007340 con 0x7f5e7c15b5d0 2023-10-06T02:32:27.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.551+0000 7f5e8319d700 1 -- 172.21.15.31:0/2291286172 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e7c0c3380 con 0x7f5e7c15b5d0 2023-10-06T02:32:27.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.551+0000 7f5e8319d700 1 -- 172.21.15.31:0/2291286172 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e7c0c3940 con 0x7f5e7c15b5d0 2023-10-06T02:32:27.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.552+0000 7f5e8319d700 1 -- 172.21.15.31:0/2291286172 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e7c04c3c0 con 0x7f5e7c15b5d0 2023-10-06T02:32:27.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.553+0000 7f5e727fc700 1 -- 172.21.15.31:0/2291286172 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) v1 ==== 85709+0+0 (secure 0 0 0) 0x7f5e740076b0 con 0x7f5e7c15b5d0 2023-10-06T02:32:27.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.554+0000 7f5e727fc700 1 --2- 172.21.15.31:0/2291286172 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5e600679e0 0x7f5e60069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:27.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.554+0000 7f5e727fc700 1 -- 172.21.15.31:0/2291286172 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f5e74083350 con 0x7f5e7c15b5d0 2023-10-06T02:32:27.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.554+0000 7f5e8219b700 1 --2- 172.21.15.31:0/2291286172 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5e600679e0 0x7f5e60069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:27.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.555+0000 7f5e8219b700 1 --2- 172.21.15.31:0/2291286172 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5e600679e0 0x7f5e60069e90 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f5e78006f00 tx=0x7f5e78008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:27.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.558+0000 7f5e727fc700 1 -- 172.21.15.31:0/2291286172 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5e74052010 con 0x7f5e7c15b5d0 2023-10-06T02:32:27.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.803+0000 7f5e8319d700 1 -- 172.21.15.31:0/2291286172 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5e7c1605a0 con 0x7f5e7c15b5d0 2023-10-06T02:32:27.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.803+0000 7f5e727fc700 1 -- 172.21.15.31:0/2291286172 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 120 v120) v1 ==== 95+0+4601 (secure 0 0 0) 0x7f5e74055630 con 0x7f5e7c15b5d0 2023-10-06T02:32:27.804 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:32:27.804 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":120,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7445,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/324072138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":324072138},{"type":"v1","addr":"172.21.15.31:6835","nonce":324072138}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":114},{"gid":7448,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/3160488596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3160488596},{"type":"v1","addr":"172.21.15.62:6834","nonce":3160488596}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":114},{"gid":8677,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/632134516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":632134516},{"type":"v1","addr":"172.21.15.62:6835","nonce":632134516}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":114}],"filesystems":[{"mdsmap":{"epoch":119,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:32:07.613387+0000","modified":"2023-10-06T02:32:11.143622+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9267},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9267":{"gid":9267,"name":"a","rank":0,"incarnation":115,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6834/3741373122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":3741373122},{"type":"v1","addr":"172.21.15.31:6834","nonce":3741373122}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2023-10-06T02:32:27.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.805+0000 7f5e8319d700 1 -- 172.21.15.31:0/2291286172 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5e600679e0 msgr2=0x7f5e60069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:27.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.805+0000 7f5e8319d700 1 --2- 172.21.15.31:0/2291286172 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5e600679e0 0x7f5e60069e90 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f5e78006f00 tx=0x7f5e78008040 comp rx=0 tx=0).stop 2023-10-06T02:32:27.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.805+0000 7f5e8319d700 1 -- 172.21.15.31:0/2291286172 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e7c15b5d0 msgr2=0x7f5e7c163f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:27.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.805+0000 7f5e8319d700 1 --2- 172.21.15.31:0/2291286172 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e7c15b5d0 0x7f5e7c163f50 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f5e7400ccc0 tx=0x7f5e74001a80 comp rx=0 tx=0).stop 2023-10-06T02:32:27.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.806+0000 7f5e8319d700 1 -- 172.21.15.31:0/2291286172 shutdown_connections 2023-10-06T02:32:27.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.806+0000 7f5e8319d700 1 --2- 172.21.15.31:0/2291286172 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e7c15b5d0 0x7f5e7c163f50 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:27.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.806+0000 7f5e8319d700 1 --2- 172.21.15.31:0/2291286172 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e7c0c6ec0 0x7f5e7c0c66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:27.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.806+0000 7f5e8319d700 1 --2- 172.21.15.31:0/2291286172 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f5e600679e0 0x7f5e60069e90 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:27.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.806+0000 7f5e8319d700 1 --2- 172.21.15.31:0/2291286172 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5e7c0ca240 0x7f5e7c0b17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:27.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.806+0000 7f5e8319d700 1 -- 172.21.15.31:0/2291286172 >> 172.21.15.31:0/2291286172 conn(0x7f5e7c06d170 msgr2=0x7f5e7c15ef80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:27.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.806+0000 7f5e8319d700 1 -- 172.21.15.31:0/2291286172 shutdown_connections 2023-10-06T02:32:27.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:27.806+0000 7f5e8319d700 1 -- 172.21.15.31:0/2291286172 wait complete. 2023-10-06T02:32:27.807 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 120 2023-10-06T02:32:27.823 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:32:28.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.010+0000 7fc3de19d700 1 Processor -- start 2023-10-06T02:32:28.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.010+0000 7fc3de19d700 1 -- start start 2023-10-06T02:32:28.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.011+0000 7fc3de19d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc3d8150380 0x7fc3d8160ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:28.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.011+0000 7fc3de19d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3d806a490 con 0x7fc3d806e0a0 2023-10-06T02:32:28.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.011+0000 7fc3de19d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3d80c7230 con 0x7fc3d806f3a0 2023-10-06T02:32:28.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.011+0000 7fc3de19d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3d806b930 con 0x7fc3d8150380 2023-10-06T02:32:28.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.012+0000 7fc3dd99c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc3d8150380 0x7fc3d8160ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:28.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.012+0000 7fc3dd19b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fc3d806e0a0 0x7fc3d80c6b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:43830/0 (socket says 172.21.15.31:43830) 2023-10-06T02:32:28.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.012+0000 7fc3dd19b700 1 -- 172.21.15.31:0/2630970032 learned_addr learned my addr 172.21.15.31:0/2630970032 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:32:28.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.012+0000 7fc3cffff700 1 -- 172.21.15.31:0/2630970032 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 441076546 0 0) 0x7fc3d806a490 con 0x7fc3d806e0a0 2023-10-06T02:32:28.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.012+0000 7fc3cffff700 1 -- 172.21.15.31:0/2630970032 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3c4003570 con 0x7fc3d806e0a0 2023-10-06T02:32:28.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.012+0000 7fc3dd99c700 1 -- 172.21.15.31:0/2630970032 >> v1:172.21.15.62:6789/0 conn(0x7fc3d806f3a0 legacy=0x7fc3d814fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:28.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.012+0000 7fc3dd99c700 1 -- 172.21.15.31:0/2630970032 >> v1:172.21.15.31:6789/0 conn(0x7fc3d806e0a0 legacy=0x7fc3d80c6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:28.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.012+0000 7fc3dd99c700 1 -- 172.21.15.31:0/2630970032 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3d80c7490 con 0x7fc3d8150380 2023-10-06T02:32:28.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.012+0000 7fc3dd99c700 1 --2- 172.21.15.31:0/2630970032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3d8150380 0x7fc3d8160ae0 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7fc3d000af70 tx=0x7fc3d000a890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2712850b456fc18e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:28.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.013+0000 7fc3cffff700 1 -- 172.21.15.31:0/2630970032 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3d000f070 con 0x7fc3d8150380 2023-10-06T02:32:28.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.013+0000 7fc3cffff700 1 -- 172.21.15.31:0/2630970032 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3d000a2e0 con 0x7fc3d8150380 2023-10-06T02:32:28.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.013+0000 7fc3cffff700 1 -- 172.21.15.31:0/2630970032 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3d000cc70 con 0x7fc3d8150380 2023-10-06T02:32:28.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.013+0000 7fc3de19d700 1 -- 172.21.15.31:0/2630970032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3d8150380 msgr2=0x7fc3d8160ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:28.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.013+0000 7fc3de19d700 1 --2- 172.21.15.31:0/2630970032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3d8150380 0x7fc3d8160ae0 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7fc3d000af70 tx=0x7fc3d000a890 comp rx=0 tx=0).stop 2023-10-06T02:32:28.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.013+0000 7fc3de19d700 1 -- 172.21.15.31:0/2630970032 shutdown_connections 2023-10-06T02:32:28.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.013+0000 7fc3de19d700 1 --2- 172.21.15.31:0/2630970032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3d8150380 0x7fc3d8160ae0 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:28.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.013+0000 7fc3de19d700 1 -- 172.21.15.31:0/2630970032 >> 172.21.15.31:0/2630970032 conn(0x7fc3d806d170 msgr2=0x7fc3d80cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:28.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.013+0000 7fc3de19d700 1 -- 172.21.15.31:0/2630970032 shutdown_connections 2023-10-06T02:32:28.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.013+0000 7fc3de19d700 1 -- 172.21.15.31:0/2630970032 wait complete. 2023-10-06T02:32:28.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.015+0000 7fc3de19d700 1 Processor -- start 2023-10-06T02:32:28.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.015+0000 7fc3de19d700 1 -- start start 2023-10-06T02:32:28.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.016+0000 7fc3de19d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc3d806e0a0 0x7fc3d80c6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:28.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.016+0000 7fc3de19d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3d806f3a0 0x7fc3d80c31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:28.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.016+0000 7fc3de19d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3d8150380 0x7fc3d81520f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:28.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.016+0000 7fc3de19d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d80c3720 con 0x7fc3d8150380 2023-10-06T02:32:28.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.016+0000 7fc3de19d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d80c4560 con 0x7fc3d806f3a0 2023-10-06T02:32:28.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.016+0000 7fc3de19d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d80c3950 con 0x7fc3d806e0a0 2023-10-06T02:32:28.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.016+0000 7fc3dd99c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3d8150380 0x7fc3d81520f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:28.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.016+0000 7fc3dd99c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3d8150380 0x7fc3d81520f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58206/0 (socket says 172.21.15.31:58206) 2023-10-06T02:32:28.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.016+0000 7fc3dd99c700 1 -- 172.21.15.31:0/3489986757 learned_addr learned my addr 172.21.15.31:0/3489986757 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:28.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.016+0000 7fc3dd99c700 1 -- 172.21.15.31:0/3489986757 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc3d806e0a0 msgr2=0x7fc3d80c6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:28.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.016+0000 7fc3dc99a700 1 --2- 172.21.15.31:0/3489986757 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3d806f3a0 0x7fc3d80c31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:28.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.016+0000 7fc3dd19b700 1 --2- 172.21.15.31:0/3489986757 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc3d806e0a0 0x7fc3d80c6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:28.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.017+0000 7fc3dd99c700 1 --2- 172.21.15.31:0/3489986757 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc3d806e0a0 0x7fc3d80c6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:28.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.017+0000 7fc3dd99c700 1 -- 172.21.15.31:0/3489986757 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3d806f3a0 msgr2=0x7fc3d80c31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:28.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.017+0000 7fc3dd99c700 1 --2- 172.21.15.31:0/3489986757 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3d806f3a0 0x7fc3d80c31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:28.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.017+0000 7fc3dd99c700 1 -- 172.21.15.31:0/3489986757 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3d000ab30 con 0x7fc3d8150380 2023-10-06T02:32:28.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.017+0000 7fc3dc99a700 1 --2- 172.21.15.31:0/3489986757 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3d806f3a0 0x7fc3d80c31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:32:28.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.017+0000 7fc3dd19b700 1 --2- 172.21.15.31:0/3489986757 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc3d806e0a0 0x7fc3d80c6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:32:28.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.017+0000 7fc3dd99c700 1 --2- 172.21.15.31:0/3489986757 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3d8150380 0x7fc3d81520f0 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7fc3d000ab00 tx=0x7fc3d0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:28.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.017+0000 7fc3cd7fa700 1 -- 172.21.15.31:0/3489986757 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3d0009c70 con 0x7fc3d8150380 2023-10-06T02:32:28.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.017+0000 7fc3cd7fa700 1 -- 172.21.15.31:0/3489986757 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3d0009dd0 con 0x7fc3d8150380 2023-10-06T02:32:28.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.017+0000 7fc3de19d700 1 -- 172.21.15.31:0/3489986757 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3d80c3c30 con 0x7fc3d8150380 2023-10-06T02:32:28.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.017+0000 7fc3cd7fa700 1 -- 172.21.15.31:0/3489986757 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3d001d690 con 0x7fc3d8150380 2023-10-06T02:32:28.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.017+0000 7fc3de19d700 1 -- 172.21.15.31:0/3489986757 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3d80c41f0 con 0x7fc3d8150380 2023-10-06T02:32:28.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.018+0000 7fc3de19d700 1 -- 172.21.15.31:0/3489986757 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3d80c4710 con 0x7fc3d8150380 2023-10-06T02:32:28.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.019+0000 7fc3cd7fa700 1 -- 172.21.15.31:0/3489986757 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) v1 ==== 85709+0+0 (secure 0 0 0) 0x7fc3d001da00 con 0x7fc3d8150380 2023-10-06T02:32:28.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.021+0000 7fc3cd7fa700 1 --2- 172.21.15.31:0/3489986757 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc3bc0679e0 0x7fc3bc069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:28.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.021+0000 7fc3cd7fa700 1 -- 172.21.15.31:0/3489986757 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fc3d00849b0 con 0x7fc3d8150380 2023-10-06T02:32:28.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.021+0000 7fc3dd19b700 1 --2- 172.21.15.31:0/3489986757 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc3bc0679e0 0x7fc3bc069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:28.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.022+0000 7fc3dd19b700 1 --2- 172.21.15.31:0/3489986757 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc3bc0679e0 0x7fc3bc069e90 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fc3d40058d0 tx=0x7fc3d4009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:28.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.025+0000 7fc3cd7fa700 1 -- 172.21.15.31:0/3489986757 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc3d80c4710 con 0x7fc3d8150380 2023-10-06T02:32:28.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.228+0000 7fc3de19d700 1 -- 172.21.15.31:0/3489986757 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc3d80c46d0 con 0x7fc3d8150380 2023-10-06T02:32:28.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.229+0000 7fc3cd7fa700 1 -- 172.21.15.31:0/3489986757 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v169) v1 ==== 74+0+16876 (secure 0 0 0) 0x7fc3d00535f0 con 0x7fc3d8150380 2023-10-06T02:32:28.230 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:32:28.230 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":169,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:32:09.112135+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:32:05.275284+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"168","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":28,"pool_name":"cephfs_data","create_time":"2023-10-06T02:32:06.591889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"169","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":20,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:32:28.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.231+0000 7fc3de19d700 1 -- 172.21.15.31:0/3489986757 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc3bc0679e0 msgr2=0x7fc3bc069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:28.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.231+0000 7fc3de19d700 1 --2- 172.21.15.31:0/3489986757 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc3bc0679e0 0x7fc3bc069e90 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fc3d40058d0 tx=0x7fc3d4009040 comp rx=0 tx=0).stop 2023-10-06T02:32:28.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.231+0000 7fc3de19d700 1 -- 172.21.15.31:0/3489986757 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3d8150380 msgr2=0x7fc3d81520f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:28.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.231+0000 7fc3de19d700 1 --2- 172.21.15.31:0/3489986757 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3d8150380 0x7fc3d81520f0 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7fc3d000ab00 tx=0x7fc3d0019000 comp rx=0 tx=0).stop 2023-10-06T02:32:28.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.231+0000 7fc3de19d700 1 -- 172.21.15.31:0/3489986757 shutdown_connections 2023-10-06T02:32:28.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.231+0000 7fc3de19d700 1 --2- 172.21.15.31:0/3489986757 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3d8150380 0x7fc3d81520f0 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:28.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.231+0000 7fc3de19d700 1 --2- 172.21.15.31:0/3489986757 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fc3bc0679e0 0x7fc3bc069e90 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:28.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.231+0000 7fc3de19d700 1 --2- 172.21.15.31:0/3489986757 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc3d806e0a0 0x7fc3d80c6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:28.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.231+0000 7fc3de19d700 1 --2- 172.21.15.31:0/3489986757 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3d806f3a0 0x7fc3d80c31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:28.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.231+0000 7fc3de19d700 1 -- 172.21.15.31:0/3489986757 >> 172.21.15.31:0/3489986757 conn(0x7fc3d806d170 msgr2=0x7fc3d814da00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:28.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.231+0000 7fc3de19d700 1 -- 172.21.15.31:0/3489986757 shutdown_connections 2023-10-06T02:32:28.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.231+0000 7fc3de19d700 1 -- 172.21.15.31:0/3489986757 wait complete. 2023-10-06T02:32:28.249 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T02:32:28.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.437+0000 7f200159d700 1 Processor -- start 2023-10-06T02:32:28.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.437+0000 7f200159d700 1 -- start start 2023-10-06T02:32:28.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.438+0000 7f200159d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f1ffc13fb20 0x7f1ffc15a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:28.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.438+0000 7f200159d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1ffc13fef0 0x7f1ffc0c7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:28.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.438+0000 7f200159d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ffc15ae70 con 0x7f1ffc06aaa0 2023-10-06T02:32:28.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.438+0000 7f200159d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ffc06af60 con 0x7f1ffc13fb20 2023-10-06T02:32:28.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.438+0000 7f200159d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ffc06b240 con 0x7f1ffc13fef0 2023-10-06T02:32:28.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.439+0000 7f1ffbfff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f1ffc13fb20 0x7f1ffc15a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:28.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.439+0000 7f1ffb7fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1ffc13fef0 0x7f1ffc0c7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:28.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.439+0000 7f2000d9c700 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f1ffc06aaa0 0x7f1ffc0c6770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:37012/0 (socket says 172.21.15.31:37012) 2023-10-06T02:32:28.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.439+0000 7f1ffbfff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f1ffc13fb20 0x7f1ffc15a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:50936/0 (socket says 172.21.15.31:50936) 2023-10-06T02:32:28.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.439+0000 7f2000d9c700 1 -- 172.21.15.31:0/561610721 learned_addr learned my addr 172.21.15.31:0/561610721 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:32:28.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.439+0000 7f1ffb7fe700 1 -- 172.21.15.31:0/561610721 >> v2:172.21.15.62:3300/0 conn(0x7f1ffc13fb20 msgr2=0x7f1ffc15a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:28.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.439+0000 7f1ffb7fe700 1 --2- 172.21.15.31:0/561610721 >> v2:172.21.15.62:3300/0 conn(0x7f1ffc13fb20 0x7f1ffc15a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:28.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.439+0000 7f1ffb7fe700 1 -- 172.21.15.31:0/561610721 >> v1:172.21.15.62:6789/0 conn(0x7f1ffc06aaa0 legacy=0x7f1ffc0c6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:28.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.439+0000 7f1ffaffd700 1 -- 172.21.15.31:0/561610721 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3140542943 0 0) 0x7f1ff4001490 con 0x7f1ffc06aaa0 2023-10-06T02:32:28.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.439+0000 7f1ffb7fe700 1 -- 172.21.15.31:0/561610721 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ffc06b520 con 0x7f1ffc13fef0 2023-10-06T02:32:28.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.440+0000 7f1ffb7fe700 1 --2- 172.21.15.31:0/561610721 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1ffc13fef0 0x7f1ffc0c7990 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1ff00098d0 tx=0x7f1ff0009120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a239868a547d3c2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:28.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.440+0000 7f1ffaffd700 1 -- 172.21.15.31:0/561610721 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff0005c10 con 0x7f1ffc13fef0 2023-10-06T02:32:28.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.440+0000 7f1ffaffd700 1 -- 172.21.15.31:0/561610721 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ff0005d70 con 0x7f1ffc13fef0 2023-10-06T02:32:28.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.440+0000 7f1ffaffd700 1 -- 172.21.15.31:0/561610721 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff0006280 con 0x7f1ffc13fef0 2023-10-06T02:32:28.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.440+0000 7f200159d700 1 -- 172.21.15.31:0/561610721 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1ffc13fef0 msgr2=0x7f1ffc0c7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:28.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.440+0000 7f200159d700 1 --2- 172.21.15.31:0/561610721 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1ffc13fef0 0x7f1ffc0c7990 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1ff00098d0 tx=0x7f1ff0009120 comp rx=0 tx=0).stop 2023-10-06T02:32:28.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.440+0000 7f200159d700 1 -- 172.21.15.31:0/561610721 shutdown_connections 2023-10-06T02:32:28.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.440+0000 7f200159d700 1 --2- 172.21.15.31:0/561610721 >> v2:172.21.15.62:3300/0 conn(0x7f1ffc13fb20 0x7f1ffc15a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:28.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.440+0000 7f200159d700 1 --2- 172.21.15.31:0/561610721 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1ffc13fef0 0x7f1ffc0c7990 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:28.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.440+0000 7f200159d700 1 -- 172.21.15.31:0/561610721 >> 172.21.15.31:0/561610721 conn(0x7f1ffc06d170 msgr2=0x7f1ffc0cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:28.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.441+0000 7f200159d700 1 -- 172.21.15.31:0/561610721 shutdown_connections 2023-10-06T02:32:28.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.441+0000 7f200159d700 1 -- 172.21.15.31:0/561610721 wait complete. 2023-10-06T02:32:28.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.442+0000 7f200159d700 1 Processor -- start 2023-10-06T02:32:28.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.442+0000 7f200159d700 1 -- start start 2023-10-06T02:32:28.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.443+0000 7f200159d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1ffc06aaa0 0x7f1ffc0ade10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:28.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.443+0000 7f200159d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ffc13fb20 0x7f1ffc159830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:28.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.443+0000 7f200159d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ffc13fef0 0x7f1ffc169c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:28.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.443+0000 7f200159d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ffc0ae350 con 0x7f1ffc13fef0 2023-10-06T02:32:28.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.443+0000 7f200159d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ffc1593a0 con 0x7f1ffc13fb20 2023-10-06T02:32:28.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.443+0000 7f200159d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ffc13c4a0 con 0x7f1ffc06aaa0 2023-10-06T02:32:28.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.443+0000 7f2000d9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ffc13fef0 0x7f1ffc169c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:28.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.443+0000 7f2000d9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ffc13fef0 0x7f1ffc169c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58220/0 (socket says 172.21.15.31:58220) 2023-10-06T02:32:28.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.443+0000 7f2000d9c700 1 -- 172.21.15.31:0/3221553311 learned_addr learned my addr 172.21.15.31:0/3221553311 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:28.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.444+0000 7f1ffb7fe700 1 --2- 172.21.15.31:0/3221553311 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ffc13fb20 0x7f1ffc159830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:28.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.444+0000 7f1ffbfff700 1 --2- 172.21.15.31:0/3221553311 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1ffc06aaa0 0x7f1ffc0ade10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:28.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.444+0000 7f2000d9c700 1 -- 172.21.15.31:0/3221553311 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1ffc06aaa0 msgr2=0x7f1ffc0ade10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:28.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.444+0000 7f2000d9c700 1 --2- 172.21.15.31:0/3221553311 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1ffc06aaa0 0x7f1ffc0ade10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:28.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.444+0000 7f2000d9c700 1 -- 172.21.15.31:0/3221553311 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ffc13fb20 msgr2=0x7f1ffc159830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:28.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.444+0000 7f2000d9c700 1 --2- 172.21.15.31:0/3221553311 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ffc13fb20 0x7f1ffc159830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:28.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.444+0000 7f2000d9c700 1 -- 172.21.15.31:0/3221553311 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ff00093c0 con 0x7f1ffc13fef0 2023-10-06T02:32:28.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.444+0000 7f1ffbfff700 1 --2- 172.21.15.31:0/3221553311 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1ffc06aaa0 0x7f1ffc0ade10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:32:28.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.444+0000 7f2000d9c700 1 --2- 172.21.15.31:0/3221553311 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ffc13fef0 0x7f1ffc169c20 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f1ff400cbc0 tx=0x7f1ff4001980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:28.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.444+0000 7f1fe3fff700 1 -- 172.21.15.31:0/3221553311 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff400a940 con 0x7f1ffc13fef0 2023-10-06T02:32:28.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.444+0000 7f1fe3fff700 1 -- 172.21.15.31:0/3221553311 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ff400aaa0 con 0x7f1ffc13fef0 2023-10-06T02:32:28.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.444+0000 7f1fe3fff700 1 -- 172.21.15.31:0/3221553311 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff400ae60 con 0x7f1ffc13fef0 2023-10-06T02:32:28.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.444+0000 7f200159d700 1 -- 172.21.15.31:0/3221553311 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ffc16a7e0 con 0x7f1ffc13fef0 2023-10-06T02:32:28.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.445+0000 7f200159d700 1 -- 172.21.15.31:0/3221553311 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ffc159030 con 0x7f1ffc13fef0 2023-10-06T02:32:28.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.445+0000 7f200159d700 1 -- 172.21.15.31:0/3221553311 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ffc159530 con 0x7f1ffc13fef0 2023-10-06T02:32:28.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.447+0000 7f1fe3fff700 1 -- 172.21.15.31:0/3221553311 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) v1 ==== 85709+0+0 (secure 0 0 0) 0x7f1ff4007300 con 0x7f1ffc13fef0 2023-10-06T02:32:28.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.448+0000 7f1fe3fff700 1 --2- 172.21.15.31:0/3221553311 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f1fe80679e0 0x7f1fe8069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:28.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.448+0000 7f1fe3fff700 1 -- 172.21.15.31:0/3221553311 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f1ff4083160 con 0x7f1ffc13fef0 2023-10-06T02:32:28.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.448+0000 7f1ffbfff700 1 --2- 172.21.15.31:0/3221553311 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f1fe80679e0 0x7f1fe8069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:28.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.449+0000 7f1ffbfff700 1 --2- 172.21.15.31:0/3221553311 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f1fe80679e0 0x7f1fe8069e90 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f1fec006f00 tx=0x7f1fec008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:28.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.452+0000 7f1fe3fff700 1 -- 172.21.15.31:0/3221553311 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1ff4051e20 con 0x7f1ffc13fef0 2023-10-06T02:32:28.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.699+0000 7f200159d700 1 -- 172.21.15.31:0/3221553311 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f1ffc15ae70 con 0x7f1ffc13fef0 2023-10-06T02:32:28.701 INFO:tasks.ceph.mon.a.smithi031.stderr:2023-10-06T02:32:28.700+0000 7f6de01f1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T02:32:28.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.709+0000 7f1fe3fff700 1 -- 172.21.15.31:0/3221553311 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v121) v1 ==== 161+0+0 (secure 0 0 0) 0x7f1ffc15ae70 con 0x7f1ffc13fef0 2023-10-06T02:32:28.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.711+0000 7f200159d700 1 -- 172.21.15.31:0/3221553311 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f1fe80679e0 msgr2=0x7f1fe8069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:28.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.711+0000 7f200159d700 1 --2- 172.21.15.31:0/3221553311 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f1fe80679e0 0x7f1fe8069e90 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f1fec006f00 tx=0x7f1fec008040 comp rx=0 tx=0).stop 2023-10-06T02:32:28.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.711+0000 7f200159d700 1 -- 172.21.15.31:0/3221553311 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ffc13fef0 msgr2=0x7f1ffc169c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:28.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.711+0000 7f200159d700 1 --2- 172.21.15.31:0/3221553311 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ffc13fef0 0x7f1ffc169c20 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f1ff400cbc0 tx=0x7f1ff4001980 comp rx=0 tx=0).stop 2023-10-06T02:32:28.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.711+0000 7f200159d700 1 -- 172.21.15.31:0/3221553311 shutdown_connections 2023-10-06T02:32:28.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.711+0000 7f200159d700 1 --2- 172.21.15.31:0/3221553311 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ffc13fef0 0x7f1ffc169c20 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:28.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.711+0000 7f200159d700 1 --2- 172.21.15.31:0/3221553311 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f1fe80679e0 0x7f1fe8069e90 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:28.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.711+0000 7f200159d700 1 --2- 172.21.15.31:0/3221553311 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1ffc06aaa0 0x7f1ffc0ade10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:28.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.711+0000 7f200159d700 1 --2- 172.21.15.31:0/3221553311 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ffc13fb20 0x7f1ffc159830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:28.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.711+0000 7f200159d700 1 -- 172.21.15.31:0/3221553311 >> 172.21.15.31:0/3221553311 conn(0x7f1ffc06d170 msgr2=0x7f1ffc13ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:28.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.711+0000 7f200159d700 1 -- 172.21.15.31:0/3221553311 shutdown_connections 2023-10-06T02:32:28.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.711+0000 7f200159d700 1 -- 172.21.15.31:0/3221553311 wait complete. 2023-10-06T02:32:28.713 INFO:teuthology.orchestra.run.smithi031.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T02:32:28.731 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T02:32:28.772 INFO:tasks.ceph.mds.a.smithi031.stdout:starting mds.a at 2023-10-06T02:32:28.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.924+0000 7ff02df9d700 1 Processor -- start 2023-10-06T02:32:28.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.924+0000 7ff02df9d700 1 -- start start 2023-10-06T02:32:28.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.926+0000 7ff02df9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff02806e560 0x7ff0280c7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:28.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.926+0000 7ff02df9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0280c7e70 con 0x7ff028155c60 2023-10-06T02:32:28.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.926+0000 7ff02df9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff02806b790 con 0x7ff028155890 2023-10-06T02:32:28.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.926+0000 7ff02df9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff02806ab80 con 0x7ff02806e560 2023-10-06T02:32:28.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.926+0000 7ff027fff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7ff028155c60 0x7ff02806de50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:43838/0 (socket says 172.21.15.31:43838) 2023-10-06T02:32:28.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.926+0000 7ff027fff700 1 -- 172.21.15.31:0/1450070810 learned_addr learned my addr 172.21.15.31:0/1450070810 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:32:28.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.926+0000 7ff02d79c700 1 --2- 172.21.15.31:0/1450070810 >> v2:172.21.15.62:3300/0 conn(0x7ff02806e560 0x7ff0280c7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:28.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.926+0000 7ff0277fe700 1 -- 172.21.15.31:0/1450070810 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1811806501 0 0) 0x7ff0280c7e70 con 0x7ff028155c60 2023-10-06T02:32:28.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.926+0000 7ff0277fe700 1 -- 172.21.15.31:0/1450070810 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff018004680 con 0x7ff028155c60 2023-10-06T02:32:28.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.927+0000 7ff0277fe700 1 -- 172.21.15.31:0/1450070810 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3744924638 0 0) 0x7ff018004680 con 0x7ff028155c60 2023-10-06T02:32:28.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.927+0000 7ff0277fe700 1 -- 172.21.15.31:0/1450070810 >> v2:172.21.15.62:3300/0 conn(0x7ff02806e560 msgr2=0x7ff0280c7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:28.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.927+0000 7ff0277fe700 1 --2- 172.21.15.31:0/1450070810 >> v2:172.21.15.62:3300/0 conn(0x7ff02806e560 0x7ff0280c7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:28.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.927+0000 7ff0277fe700 1 -- 172.21.15.31:0/1450070810 >> v1:172.21.15.62:6789/0 conn(0x7ff028155890 legacy=0x7ff02806a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:28.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.927+0000 7ff0277fe700 1 -- 172.21.15.31:0/1450070810 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff02806ae60 con 0x7ff028155c60 2023-10-06T02:32:28.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.927+0000 7ff0277fe700 1 -- 172.21.15.31:0/1450070810 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7ff014001470 con 0x7ff028155c60 2023-10-06T02:32:28.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.927+0000 7ff0277fe700 1 -- 172.21.15.31:0/1450070810 >> v1:172.21.15.31:6789/0 conn(0x7ff028155c60 legacy=0x7ff02806de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:28.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.927+0000 7ff0277fe700 1 --2- 172.21.15.31:0/1450070810 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff018004bc0 0x7ff018004fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:28.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.927+0000 7ff0277fe700 1 --2- 172.21.15.31:0/1450070810 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff018005920 0x7ff018005d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:28.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.927+0000 7ff0277fe700 1 --2- 172.21.15.31:0/1450070810 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff018006320 0x7ff01800cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:28.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.927+0000 7ff0277fe700 1 -- 172.21.15.31:0/1450070810 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff02806ab80 con 0x7ff018006320 2023-10-06T02:32:28.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.927+0000 7ff0277fe700 1 -- 172.21.15.31:0/1450070810 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff01800d2c0 con 0x7ff018004bc0 2023-10-06T02:32:28.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.927+0000 7ff0277fe700 1 -- 172.21.15.31:0/1450070810 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff01800d480 con 0x7ff018005920 2023-10-06T02:32:28.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.927+0000 7ff02d79c700 1 --2- 172.21.15.31:0/1450070810 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff018006320 0x7ff01800cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:28.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.927+0000 7ff02cf9b700 1 --2- 172.21.15.31:0/1450070810 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff018004bc0 0x7ff018004fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:28.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.927+0000 7ff027fff700 1 --2- 172.21.15.31:0/1450070810 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff018005920 0x7ff018005d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:28.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.928+0000 7ff02d79c700 1 -- 172.21.15.31:0/1450070810 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff018005920 msgr2=0x7ff018005d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:28.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.928+0000 7ff02d79c700 1 --2- 172.21.15.31:0/1450070810 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff018005920 0x7ff018005d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:28.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.928+0000 7ff02d79c700 1 -- 172.21.15.31:0/1450070810 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff018004bc0 msgr2=0x7ff018004fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:28.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.928+0000 7ff02d79c700 1 --2- 172.21.15.31:0/1450070810 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff018004bc0 0x7ff018004fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:28.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.928+0000 7ff02d79c700 1 -- 172.21.15.31:0/1450070810 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff018004680 con 0x7ff018006320 2023-10-06T02:32:28.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.928+0000 7ff027fff700 1 --2- 172.21.15.31:0/1450070810 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff018005920 0x7ff018005d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:32:28.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.928+0000 7ff02d79c700 1 -- 172.21.15.31:0/1450070810 reap_dead start 2023-10-06T02:32:28.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.928+0000 7ff02d79c700 1 --2- 172.21.15.31:0/1450070810 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff018006320 0x7ff01800cb70 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7ff02806f870 tx=0x7ff02000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a1bd9a54c6cf6c6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:28.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.928+0000 7ff0277fe700 1 -- 172.21.15.31:0/1450070810 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff02000ea60 con 0x7ff018006320 2023-10-06T02:32:28.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.928+0000 7ff0277fe700 1 -- 172.21.15.31:0/1450070810 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff02000f040 con 0x7ff018006320 2023-10-06T02:32:28.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.928+0000 7ff0277fe700 1 -- 172.21.15.31:0/1450070810 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff02000ee10 con 0x7ff018006320 2023-10-06T02:32:28.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.929+0000 7ff02df9d700 1 -- 172.21.15.31:0/1450070810 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff018006320 msgr2=0x7ff01800cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:28.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.929+0000 7ff02df9d700 1 --2- 172.21.15.31:0/1450070810 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff018006320 0x7ff01800cb70 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7ff02806f870 tx=0x7ff02000b040 comp rx=0 tx=0).stop 2023-10-06T02:32:28.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.929+0000 7ff02df9d700 1 -- 172.21.15.31:0/1450070810 shutdown_connections 2023-10-06T02:32:28.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.929+0000 7ff02df9d700 1 --2- 172.21.15.31:0/1450070810 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff018006320 0x7ff01800cb70 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:28.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.929+0000 7ff02df9d700 1 -- 172.21.15.31:0/1450070810 >> 172.21.15.31:0/1450070810 conn(0x7ff02806d110 msgr2=0x7ff0280cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:28.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.929+0000 7ff02df9d700 1 -- 172.21.15.31:0/1450070810 shutdown_connections 2023-10-06T02:32:28.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.929+0000 7ff02df9d700 1 -- 172.21.15.31:0/1450070810 wait complete. 2023-10-06T02:32:28.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.930+0000 7ff02df9d700 1 Processor -- start 2023-10-06T02:32:28.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.930+0000 7ff02df9d700 1 -- start start 2023-10-06T02:32:28.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.931+0000 7ff02df9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff018006320 0x7ff028146ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:28.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.931+0000 7ff02df9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0280c5af0 0x7ff0280aa180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:28.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.931+0000 7ff02df9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff0280c5f60 0x7ff0280aa6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:28.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.931+0000 7ff02df9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0280ac0f0 con 0x7ff0280c5af0 2023-10-06T02:32:28.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.931+0000 7ff02df9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0280c64d0 con 0x7ff018006320 2023-10-06T02:32:28.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.931+0000 7ff02df9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0280c67b0 con 0x7ff0280c5f60 2023-10-06T02:32:28.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.932+0000 7ff027fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0280c5af0 0x7ff0280aa180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:28.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.932+0000 7ff027fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0280c5af0 0x7ff0280aa180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58280/0 (socket says 172.21.15.31:58280) 2023-10-06T02:32:28.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.932+0000 7ff027fff700 1 -- 172.21.15.31:0/3511751912 learned_addr learned my addr 172.21.15.31:0/3511751912 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:28.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.932+0000 7ff02cf9b700 1 --2- 172.21.15.31:0/3511751912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff018006320 0x7ff028146ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:28.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.932+0000 7ff02d79c700 1 --2- 172.21.15.31:0/3511751912 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff0280c5f60 0x7ff0280aa6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:28.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.932+0000 7ff027fff700 1 -- 172.21.15.31:0/3511751912 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff0280c5f60 msgr2=0x7ff0280aa6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:28.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.932+0000 7ff027fff700 1 --2- 172.21.15.31:0/3511751912 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff0280c5f60 0x7ff0280aa6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:28.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.932+0000 7ff027fff700 1 -- 172.21.15.31:0/3511751912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff018006320 msgr2=0x7ff028146ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:28.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.932+0000 7ff027fff700 1 --2- 172.21.15.31:0/3511751912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff018006320 0x7ff028146ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:28.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.932+0000 7ff027fff700 1 -- 172.21.15.31:0/3511751912 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0280c6a90 con 0x7ff0280c5af0 2023-10-06T02:32:28.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.932+0000 7ff027fff700 1 --2- 172.21.15.31:0/3511751912 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0280c5af0 0x7ff0280aa180 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7ff01400eb00 tx=0x7ff014000d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:28.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.932+0000 7ff024ff9700 1 -- 172.21.15.31:0/3511751912 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff014005e10 con 0x7ff0280c5af0 2023-10-06T02:32:28.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.933+0000 7ff024ff9700 1 -- 172.21.15.31:0/3511751912 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff014010040 con 0x7ff0280c5af0 2023-10-06T02:32:28.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.933+0000 7ff024ff9700 1 -- 172.21.15.31:0/3511751912 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff01400a760 con 0x7ff0280c5af0 2023-10-06T02:32:28.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.933+0000 7ff02df9d700 1 -- 172.21.15.31:0/3511751912 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0280c6d70 con 0x7ff0280c5af0 2023-10-06T02:32:28.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.933+0000 7ff02df9d700 1 -- 172.21.15.31:0/3511751912 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0280c4340 con 0x7ff0280c5af0 2023-10-06T02:32:28.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.934+0000 7ff02df9d700 1 -- 172.21.15.31:0/3511751912 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff02804c3c0 con 0x7ff0280c5af0 2023-10-06T02:32:28.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.935+0000 7ff024ff9700 1 -- 172.21.15.31:0/3511751912 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) v1 ==== 85709+0+0 (secure 0 0 0) 0x7ff01400ab90 con 0x7ff0280c5af0 2023-10-06T02:32:28.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.935+0000 7ff024ff9700 1 --2- 172.21.15.31:0/3511751912 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff00c0679e0 0x7ff00c069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:28.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.936+0000 7ff024ff9700 1 -- 172.21.15.31:0/3511751912 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 6172+0+0 (secure 0 0 0) 0x7ff014082300 con 0x7ff0280c5af0 2023-10-06T02:32:28.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.936+0000 7ff02cf9b700 1 --2- 172.21.15.31:0/3511751912 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff00c0679e0 0x7ff00c069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:28.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.937+0000 7ff02cf9b700 1 --2- 172.21.15.31:0/3511751912 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff00c0679e0 0x7ff00c069e90 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ff01c007e40 tx=0x7ff01c007d30 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:28.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:28.939+0000 7ff024ff9700 1 -- 172.21.15.31:0/3511751912 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff01404d5a0 con 0x7ff0280c5af0 2023-10-06T02:32:29.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:29.192+0000 7ff02df9d700 1 -- 172.21.15.31:0/3511751912 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff0280cbb00 con 0x7ff0280c5af0 2023-10-06T02:32:29.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:29.707+0000 7ff024ff9700 1 -- 172.21.15.31:0/3511751912 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v122) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff0140190a0 con 0x7ff0280c5af0 2023-10-06T02:32:29.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:29.710+0000 7ff02df9d700 1 -- 172.21.15.31:0/3511751912 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff00c0679e0 msgr2=0x7ff00c069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:29.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:29.710+0000 7ff02df9d700 1 --2- 172.21.15.31:0/3511751912 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff00c0679e0 0x7ff00c069e90 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ff01c007e40 tx=0x7ff01c007d30 comp rx=0 tx=0).stop 2023-10-06T02:32:29.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:29.710+0000 7ff02df9d700 1 -- 172.21.15.31:0/3511751912 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0280c5af0 msgr2=0x7ff0280aa180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:29.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:29.710+0000 7ff02df9d700 1 --2- 172.21.15.31:0/3511751912 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0280c5af0 0x7ff0280aa180 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7ff01400eb00 tx=0x7ff014000d50 comp rx=0 tx=0).stop 2023-10-06T02:32:29.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:29.710+0000 7ff02df9d700 1 -- 172.21.15.31:0/3511751912 shutdown_connections 2023-10-06T02:32:29.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:29.710+0000 7ff02df9d700 1 --2- 172.21.15.31:0/3511751912 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff00c0679e0 0x7ff00c069e90 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:29.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:29.710+0000 7ff02df9d700 1 --2- 172.21.15.31:0/3511751912 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff0280c5f60 0x7ff0280aa6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:29.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:29.710+0000 7ff02df9d700 1 --2- 172.21.15.31:0/3511751912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff018006320 0x7ff028146ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:29.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:29.710+0000 7ff02df9d700 1 --2- 172.21.15.31:0/3511751912 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0280c5af0 0x7ff0280aa180 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:29.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:29.710+0000 7ff02df9d700 1 -- 172.21.15.31:0/3511751912 >> 172.21.15.31:0/3511751912 conn(0x7ff02806d110 msgr2=0x7ff0280c1080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:29.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:29.710+0000 7ff02df9d700 1 -- 172.21.15.31:0/3511751912 shutdown_connections 2023-10-06T02:32:29.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:29.710+0000 7ff02df9d700 1 -- 172.21.15.31:0/3511751912 wait complete. 2023-10-06T02:32:29.731 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T02:32:29.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:29.966+0000 7f77bd79d700 1 Processor -- start 2023-10-06T02:32:29.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:29.966+0000 7f77bd79d700 1 -- start start 2023-10-06T02:32:29.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:29.967+0000 7f77bd79d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f77b81539d0 0x7f77b8153da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:29.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:29.967+0000 7f77bd79d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f77b80cbb80 0x7f77b80ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:29.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:29.967+0000 7f77bd79d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77b80cd180 con 0x7f77b815a740 2023-10-06T02:32:29.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:29.967+0000 7f77bd79d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77b80ce930 con 0x7f77b80cbb80 2023-10-06T02:32:29.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:29.967+0000 7f77bd79d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77b815abe0 con 0x7f77b81539d0 2023-10-06T02:32:29.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:29.968+0000 7f77bcf9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f77b80cbb80 0x7f77b80ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:29.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:29.968+0000 7f77bcf9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f77b80cbb80 0x7f77b80ccc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58294/0 (socket says 172.21.15.31:58294) 2023-10-06T02:32:29.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:29.968+0000 7f77bcf9c700 1 -- 172.21.15.31:0/2268028680 learned_addr learned my addr 172.21.15.31:0/2268028680 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:29.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:29.968+0000 7f77bcf9c700 1 -- 172.21.15.31:0/2268028680 >> v2:172.21.15.62:3300/0 conn(0x7f77b81539d0 msgr2=0x7f77b8153da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:29.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:29.968+0000 7f77b7fff700 1 --2- 172.21.15.31:0/2268028680 >> v2:172.21.15.62:3300/0 conn(0x7f77b81539d0 0x7f77b8153da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:29.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:29.968+0000 7f77bcf9c700 1 --2- 172.21.15.31:0/2268028680 >> v2:172.21.15.62:3300/0 conn(0x7f77b81539d0 0x7f77b8153da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:29.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:29.968+0000 7f77bcf9c700 1 -- 172.21.15.31:0/2268028680 >> v1:172.21.15.62:6789/0 conn(0x7f77b815a740 legacy=0x7f77b80cb470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:29.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:29.968+0000 7f77bcf9c700 1 -- 172.21.15.31:0/2268028680 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77b80cd3e0 con 0x7f77b80cbb80 2023-10-06T02:32:29.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:29.968+0000 7f77b7fff700 1 --2- 172.21.15.31:0/2268028680 >> v2:172.21.15.62:3300/0 conn(0x7f77b81539d0 0x7f77b8153da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:32:29.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:29.968+0000 7f77bcf9c700 1 --2- 172.21.15.31:0/2268028680 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77b80cbb80 0x7f77b80ccc40 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f77b8089470 tx=0x7f77b0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab33a817ebf59bbd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:29.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:29.969+0000 7f77b6ffd700 1 -- 172.21.15.31:0/2268028680 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b000e070 con 0x7f77b80cbb80 2023-10-06T02:32:29.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:29.969+0000 7f77b6ffd700 1 -- 172.21.15.31:0/2268028680 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77b000ba10 con 0x7f77b80cbb80 2023-10-06T02:32:29.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:29.969+0000 7f77b6ffd700 1 -- 172.21.15.31:0/2268028680 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b00092e0 con 0x7f77b80cbb80 2023-10-06T02:32:29.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:29.969+0000 7f77bd79d700 1 -- 172.21.15.31:0/2268028680 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77b80cbb80 msgr2=0x7f77b80ccc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:29.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:29.969+0000 7f77bd79d700 1 --2- 172.21.15.31:0/2268028680 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77b80cbb80 0x7f77b80ccc40 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f77b8089470 tx=0x7f77b0009840 comp rx=0 tx=0).stop 2023-10-06T02:32:29.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:29.969+0000 7f77bd79d700 1 -- 172.21.15.31:0/2268028680 shutdown_connections 2023-10-06T02:32:29.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:29.969+0000 7f77bd79d700 1 --2- 172.21.15.31:0/2268028680 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77b80cbb80 0x7f77b80ccc40 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:29.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:29.969+0000 7f77bd79d700 1 --2- 172.21.15.31:0/2268028680 >> v2:172.21.15.62:3300/0 conn(0x7f77b81539d0 0x7f77b8153da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:29.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:29.969+0000 7f77bd79d700 1 -- 172.21.15.31:0/2268028680 >> 172.21.15.31:0/2268028680 conn(0x7f77b80703a0 msgr2=0x7f77b80707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:29.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:29.969+0000 7f77bd79d700 1 -- 172.21.15.31:0/2268028680 shutdown_connections 2023-10-06T02:32:29.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:29.969+0000 7f77bd79d700 1 -- 172.21.15.31:0/2268028680 wait complete. 2023-10-06T02:32:29.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:29.971+0000 7f77bd79d700 1 Processor -- start 2023-10-06T02:32:29.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:29.971+0000 7f77bd79d700 1 -- start start 2023-10-06T02:32:29.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:29.972+0000 7f77bd79d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77b80cbb80 0x7f77b81471c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:29.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:29.972+0000 7f77bd79d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77b81539d0 0x7f77b814ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:29.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:29.972+0000 7f77bd79d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f77b815a740 0x7f77b8151970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:29.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:29.972+0000 7f77bd79d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77b815e0b0 con 0x7f77b80cbb80 2023-10-06T02:32:29.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:29.972+0000 7f77bd79d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77b80b0430 con 0x7f77b81539d0 2023-10-06T02:32:29.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:29.972+0000 7f77bd79d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77b814f0a0 con 0x7f77b815a740 2023-10-06T02:32:29.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:29.972+0000 7f77b7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77b80cbb80 0x7f77b81471c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:29.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:29.972+0000 7f77b7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77b80cbb80 0x7f77b81471c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58302/0 (socket says 172.21.15.31:58302) 2023-10-06T02:32:29.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:29.972+0000 7f77b7fff700 1 -- 172.21.15.31:0/3190727162 learned_addr learned my addr 172.21.15.31:0/3190727162 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:29.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:29.972+0000 7f77b77fe700 1 --2- 172.21.15.31:0/3190727162 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77b81539d0 0x7f77b814ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:29.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:29.972+0000 7f77bcf9c700 1 --2- 172.21.15.31:0/3190727162 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f77b815a740 0x7f77b8151970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:29.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:29.973+0000 7f77b7fff700 1 -- 172.21.15.31:0/3190727162 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f77b815a740 msgr2=0x7f77b8151970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:29.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:29.973+0000 7f77b7fff700 1 --2- 172.21.15.31:0/3190727162 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f77b815a740 0x7f77b8151970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:29.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:29.973+0000 7f77b7fff700 1 -- 172.21.15.31:0/3190727162 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77b81539d0 msgr2=0x7f77b814ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:29.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:29.973+0000 7f77b7fff700 1 --2- 172.21.15.31:0/3190727162 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77b81539d0 0x7f77b814ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:29.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:29.973+0000 7f77b7fff700 1 -- 172.21.15.31:0/3190727162 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77b0009ae0 con 0x7f77b80cbb80 2023-10-06T02:32:29.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:29.973+0000 7f77b7fff700 1 --2- 172.21.15.31:0/3190727162 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77b80cbb80 0x7f77b81471c0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f77a800c830 tx=0x7f77a8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:29.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:29.973+0000 7f779ffff700 1 -- 172.21.15.31:0/3190727162 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a8011600 con 0x7f77b80cbb80 2023-10-06T02:32:29.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:29.973+0000 7f779ffff700 1 -- 172.21.15.31:0/3190727162 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77a8011760 con 0x7f77b80cbb80 2023-10-06T02:32:29.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:29.973+0000 7f779ffff700 1 -- 172.21.15.31:0/3190727162 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a8011b20 con 0x7f77b80cbb80 2023-10-06T02:32:29.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:29.973+0000 7f77bd79d700 1 -- 172.21.15.31:0/3190727162 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77b814e490 con 0x7f77b80cbb80 2023-10-06T02:32:29.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:29.973+0000 7f77bd79d700 1 -- 172.21.15.31:0/3190727162 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77b814ea50 con 0x7f77b80cbb80 2023-10-06T02:32:29.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:29.974+0000 7f77bd79d700 1 -- 172.21.15.31:0/3190727162 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77b804c3c0 con 0x7f77b80cbb80 2023-10-06T02:32:29.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:29.975+0000 7f779ffff700 1 -- 172.21.15.31:0/3190727162 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) v1 ==== 85709+0+0 (secure 0 0 0) 0x7f77a800a3e0 con 0x7f77b80cbb80 2023-10-06T02:32:29.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:29.976+0000 7f779ffff700 1 --2- 172.21.15.31:0/3190727162 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f77a40679e0 0x7f77a4069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:29.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:29.976+0000 7f779ffff700 1 -- 172.21.15.31:0/3190727162 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 6172+0+0 (secure 0 0 0) 0x7f77a8083010 con 0x7f77b80cbb80 2023-10-06T02:32:29.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:29.976+0000 7f77b77fe700 1 --2- 172.21.15.31:0/3190727162 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f77a40679e0 0x7f77a4069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:29.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:29.977+0000 7f77b77fe700 1 --2- 172.21.15.31:0/3190727162 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f77a40679e0 0x7f77a4069e90 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f77ac006fd0 tx=0x7f77ac008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:29.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:29.980+0000 7f779ffff700 1 -- 172.21.15.31:0/3190727162 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f77a8051c50 con 0x7f77b80cbb80 2023-10-06T02:32:30.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:30.183+0000 7f77bd79d700 1 -- 172.21.15.31:0/3190727162 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f77b80cd6c0 con 0x7f77b80cbb80 2023-10-06T02:32:31.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:31.113+0000 7f779ffff700 1 -- 172.21.15.31:0/3190727162 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v171) v1 ==== 181+0+0 (secure 0 0 0) 0x7f77a80150d0 con 0x7f77b80cbb80 2023-10-06T02:32:31.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:31.117+0000 7f77bd79d700 1 -- 172.21.15.31:0/3190727162 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f77a40679e0 msgr2=0x7f77a4069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:31.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:31.117+0000 7f77bd79d700 1 --2- 172.21.15.31:0/3190727162 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f77a40679e0 0x7f77a4069e90 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f77ac006fd0 tx=0x7f77ac008040 comp rx=0 tx=0).stop 2023-10-06T02:32:31.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:31.117+0000 7f77bd79d700 1 -- 172.21.15.31:0/3190727162 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77b80cbb80 msgr2=0x7f77b81471c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:31.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:31.117+0000 7f77bd79d700 1 --2- 172.21.15.31:0/3190727162 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77b80cbb80 0x7f77b81471c0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f77a800c830 tx=0x7f77a8008040 comp rx=0 tx=0).stop 2023-10-06T02:32:31.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:31.118+0000 7f77bd79d700 1 -- 172.21.15.31:0/3190727162 shutdown_connections 2023-10-06T02:32:31.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:31.118+0000 7f77bd79d700 1 --2- 172.21.15.31:0/3190727162 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f77a40679e0 0x7f77a4069e90 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:31.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:31.118+0000 7f77bd79d700 1 --2- 172.21.15.31:0/3190727162 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f77b815a740 0x7f77b8151970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:31.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:31.118+0000 7f77bd79d700 1 --2- 172.21.15.31:0/3190727162 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77b80cbb80 0x7f77b81471c0 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:31.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:31.118+0000 7f77bd79d700 1 --2- 172.21.15.31:0/3190727162 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77b81539d0 0x7f77b814ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:31.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:31.118+0000 7f77bd79d700 1 -- 172.21.15.31:0/3190727162 >> 172.21.15.31:0/3190727162 conn(0x7f77b80703a0 msgr2=0x7f77b815ae00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:31.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:31.118+0000 7f77bd79d700 1 -- 172.21.15.31:0/3190727162 shutdown_connections 2023-10-06T02:32:31.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:31.118+0000 7f77bd79d700 1 -- 172.21.15.31:0/3190727162 wait complete. 2023-10-06T02:32:31.120 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' removed 2023-10-06T02:32:31.140 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T02:32:31.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:31.336+0000 7fae1b59e700 1 Processor -- start 2023-10-06T02:32:31.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:31.336+0000 7fae1b59e700 1 -- start start 2023-10-06T02:32:31.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:31.337+0000 7fae1b59e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fae1c0c8200 0x7fae1c0c1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:31.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:31.337+0000 7fae1b59e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fae1c0c2a90 0x7fae1c156aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:31.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:31.337+0000 7fae1b59e700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae1c1579b0 con 0x7fae1c0c85d0 2023-10-06T02:32:31.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:31.337+0000 7fae1b59e700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae1c0c1390 con 0x7fae1c0c2a90 2023-10-06T02:32:31.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:31.338+0000 7fae1b59e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae1c1573f0 con 0x7fae1c0c8200 2023-10-06T02:32:31.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:31.338+0000 7fae19d9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fae1c0c2a90 0x7fae1c156aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:31.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:31.338+0000 7fae19d9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fae1c0c2a90 0x7fae1c156aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58308/0 (socket says 172.21.15.31:58308) 2023-10-06T02:32:31.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:31.338+0000 7fae19d9b700 1 -- 172.21.15.31:0/339658767 learned_addr learned my addr 172.21.15.31:0/339658767 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:31.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:31.338+0000 7fae1a59c700 1 --2- 172.21.15.31:0/339658767 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fae1c0c8200 0x7fae1c0c1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:31.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:31.338+0000 7fae19d9b700 1 -- 172.21.15.31:0/339658767 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fae1c0c8200 msgr2=0x7fae1c0c1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:31.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:31.338+0000 7fae19d9b700 1 --2- 172.21.15.31:0/339658767 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fae1c0c8200 0x7fae1c0c1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:31.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:31.338+0000 7fae19d9b700 1 -- 172.21.15.31:0/339658767 >> v1:172.21.15.62:6789/0 conn(0x7fae1c0c85d0 legacy=0x7fae1c0cc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:31.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:31.338+0000 7fae19d9b700 1 -- 172.21.15.31:0/339658767 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae1c1576d0 con 0x7fae1c0c2a90 2023-10-06T02:32:31.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:31.338+0000 7fae1a59c700 1 --2- 172.21.15.31:0/339658767 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fae1c0c8200 0x7fae1c0c1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:32:31.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:31.338+0000 7fae19d9b700 1 --2- 172.21.15.31:0/339658767 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae1c0c2a90 0x7fae1c156aa0 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7fae100099b0 tx=0x7fae10009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fb27539db93e91d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:31.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:31.339+0000 7fae1959a700 1 -- 172.21.15.31:0/339658767 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae1000b820 con 0x7fae1c0c2a90 2023-10-06T02:32:31.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:31.339+0000 7fae1959a700 1 -- 172.21.15.31:0/339658767 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae1000b980 con 0x7fae1c0c2a90 2023-10-06T02:32:31.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:31.339+0000 7fae1959a700 1 -- 172.21.15.31:0/339658767 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae1000bd20 con 0x7fae1c0c2a90 2023-10-06T02:32:31.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:31.339+0000 7fae1b59e700 1 -- 172.21.15.31:0/339658767 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae1c0c2a90 msgr2=0x7fae1c156aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:31.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:31.339+0000 7fae1b59e700 1 --2- 172.21.15.31:0/339658767 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae1c0c2a90 0x7fae1c156aa0 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7fae100099b0 tx=0x7fae10009250 comp rx=0 tx=0).stop 2023-10-06T02:32:31.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:31.339+0000 7fae1b59e700 1 -- 172.21.15.31:0/339658767 shutdown_connections 2023-10-06T02:32:31.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:31.339+0000 7fae1b59e700 1 --2- 172.21.15.31:0/339658767 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fae1c0c8200 0x7fae1c0c1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:31.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:31.339+0000 7fae1b59e700 1 --2- 172.21.15.31:0/339658767 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae1c0c2a90 0x7fae1c156aa0 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:31.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:31.339+0000 7fae1b59e700 1 -- 172.21.15.31:0/339658767 >> 172.21.15.31:0/339658767 conn(0x7fae1c06c6a0 msgr2=0x7fae1c06caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:31.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:31.339+0000 7fae1b59e700 1 -- 172.21.15.31:0/339658767 shutdown_connections 2023-10-06T02:32:31.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:31.339+0000 7fae1b59e700 1 -- 172.21.15.31:0/339658767 wait complete. 2023-10-06T02:32:31.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:31.341+0000 7fae1b59e700 1 Processor -- start 2023-10-06T02:32:31.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:31.341+0000 7fae1b59e700 1 -- start start 2023-10-06T02:32:31.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:31.342+0000 7fae1b59e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae1c0c2a90 0x7fae1c0ad930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:31.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:31.342+0000 7fae1b59e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fae1c0c8200 0x7fae1c0aacb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:31.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:31.342+0000 7fae1b59e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae1c0c85d0 0x7fae1c169c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:31.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:31.342+0000 7fae1b59e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae1c0ad510 con 0x7fae1c0c2a90 2023-10-06T02:32:31.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:31.342+0000 7fae1b59e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae1c0ac890 con 0x7fae1c0c85d0 2023-10-06T02:32:31.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:31.342+0000 7fae1b59e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae1c0acb70 con 0x7fae1c0c8200 2023-10-06T02:32:31.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:31.342+0000 7fae1a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae1c0c2a90 0x7fae1c0ad930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:31.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:31.342+0000 7fae1a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae1c0c2a90 0x7fae1c0ad930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58312/0 (socket says 172.21.15.31:58312) 2023-10-06T02:32:31.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:31.342+0000 7fae1a59c700 1 -- 172.21.15.31:0/1352359134 learned_addr learned my addr 172.21.15.31:0/1352359134 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:31.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:31.343+0000 7fae1ad9d700 1 --2- 172.21.15.31:0/1352359134 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae1c0c85d0 0x7fae1c169c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:31.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:31.343+0000 7fae19d9b700 1 --2- 172.21.15.31:0/1352359134 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fae1c0c8200 0x7fae1c0aacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:31.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:31.343+0000 7fae1a59c700 1 -- 172.21.15.31:0/1352359134 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fae1c0c8200 msgr2=0x7fae1c0aacb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:31.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:31.343+0000 7fae1a59c700 1 --2- 172.21.15.31:0/1352359134 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fae1c0c8200 0x7fae1c0aacb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:31.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:31.343+0000 7fae1a59c700 1 -- 172.21.15.31:0/1352359134 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae1c0c85d0 msgr2=0x7fae1c169c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:31.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:31.343+0000 7fae1a59c700 1 --2- 172.21.15.31:0/1352359134 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae1c0c85d0 0x7fae1c169c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:31.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:31.343+0000 7fae1a59c700 1 -- 172.21.15.31:0/1352359134 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae100094f0 con 0x7fae1c0c2a90 2023-10-06T02:32:31.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:31.343+0000 7fae19d9b700 1 --2- 172.21.15.31:0/1352359134 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fae1c0c8200 0x7fae1c0aacb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:32:31.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:31.343+0000 7fae1ad9d700 1 --2- 172.21.15.31:0/1352359134 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae1c0c85d0 0x7fae1c169c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:32:31.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:31.343+0000 7fae1a59c700 1 --2- 172.21.15.31:0/1352359134 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae1c0c2a90 0x7fae1c0ad930 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7fae0c00c830 tx=0x7fae0c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:31.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:31.343+0000 7fae0a7fc700 1 -- 172.21.15.31:0/1352359134 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0c011600 con 0x7fae1c0c2a90 2023-10-06T02:32:31.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:31.343+0000 7fae0a7fc700 1 -- 172.21.15.31:0/1352359134 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae0c011760 con 0x7fae1c0c2a90 2023-10-06T02:32:31.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:31.343+0000 7fae0a7fc700 1 -- 172.21.15.31:0/1352359134 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0c011b20 con 0x7fae1c0c2a90 2023-10-06T02:32:31.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:31.343+0000 7fae1b59e700 1 -- 172.21.15.31:0/1352359134 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae1c0ace50 con 0x7fae1c0c2a90 2023-10-06T02:32:31.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:31.343+0000 7fae1b59e700 1 -- 172.21.15.31:0/1352359134 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae1c0ab1f0 con 0x7fae1c0c2a90 2023-10-06T02:32:31.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:31.344+0000 7fae1b59e700 1 -- 172.21.15.31:0/1352359134 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae1c04c3c0 con 0x7fae1c0c2a90 2023-10-06T02:32:31.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:31.345+0000 7fae0a7fc700 1 -- 172.21.15.31:0/1352359134 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) v1 ==== 85709+0+0 (secure 0 0 0) 0x7fae0c00a3e0 con 0x7fae1c0c2a90 2023-10-06T02:32:31.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:31.346+0000 7fae0a7fc700 1 --2- 172.21.15.31:0/1352359134 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fadfc0679e0 0x7fadfc069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:31.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:31.346+0000 7fae0a7fc700 1 -- 172.21.15.31:0/1352359134 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 5734+0+0 (secure 0 0 0) 0x7fae0c083380 con 0x7fae1c0c2a90 2023-10-06T02:32:31.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:31.347+0000 7fae19d9b700 1 --2- 172.21.15.31:0/1352359134 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fadfc0679e0 0x7fadfc069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:31.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:31.347+0000 7fae19d9b700 1 --2- 172.21.15.31:0/1352359134 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fadfc0679e0 0x7fadfc069e90 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fae100099b0 tx=0x7fae10016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:31.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:31.350+0000 7fae0a7fc700 1 -- 172.21.15.31:0/1352359134 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fae0c052020 con 0x7fae1c0c2a90 2023-10-06T02:32:31.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:31.552+0000 7fae1b59e700 1 -- 172.21.15.31:0/1352359134 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fae1c0ad780 con 0x7fae1c0c2a90 2023-10-06T02:32:32.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.115+0000 7fae0a7fc700 1 -- 172.21.15.31:0/1352359134 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v172) v1 ==== 169+0+0 (secure 0 0 0) 0x7fae0c055640 con 0x7fae1c0c2a90 2023-10-06T02:32:32.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.120+0000 7fae1b59e700 1 -- 172.21.15.31:0/1352359134 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fadfc0679e0 msgr2=0x7fadfc069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:32.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.120+0000 7fae1b59e700 1 --2- 172.21.15.31:0/1352359134 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fadfc0679e0 0x7fadfc069e90 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fae100099b0 tx=0x7fae10016000 comp rx=0 tx=0).stop 2023-10-06T02:32:32.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.120+0000 7fae1b59e700 1 -- 172.21.15.31:0/1352359134 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae1c0c2a90 msgr2=0x7fae1c0ad930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:32.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.120+0000 7fae1b59e700 1 --2- 172.21.15.31:0/1352359134 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae1c0c2a90 0x7fae1c0ad930 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7fae0c00c830 tx=0x7fae0c008040 comp rx=0 tx=0).stop 2023-10-06T02:32:32.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.120+0000 7fae1b59e700 1 -- 172.21.15.31:0/1352359134 shutdown_connections 2023-10-06T02:32:32.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.120+0000 7fae1b59e700 1 --2- 172.21.15.31:0/1352359134 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae1c0c85d0 0x7fae1c169c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:32.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.120+0000 7fae1b59e700 1 --2- 172.21.15.31:0/1352359134 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae1c0c2a90 0x7fae1c0ad930 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:32.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.120+0000 7fae1b59e700 1 --2- 172.21.15.31:0/1352359134 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fadfc0679e0 0x7fadfc069e90 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:32.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.120+0000 7fae1b59e700 1 --2- 172.21.15.31:0/1352359134 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fae1c0c8200 0x7fae1c0aacb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:32.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.120+0000 7fae1b59e700 1 -- 172.21.15.31:0/1352359134 >> 172.21.15.31:0/1352359134 conn(0x7fae1c06c6a0 msgr2=0x7fae1c0c7600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:32.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.121+0000 7fae1b59e700 1 -- 172.21.15.31:0/1352359134 shutdown_connections 2023-10-06T02:32:32.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.121+0000 7fae1b59e700 1 -- 172.21.15.31:0/1352359134 wait complete. 2023-10-06T02:32:32.124 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' removed 2023-10-06T02:32:32.145 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T02:32:32.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.350+0000 7f55ce19d700 1 Processor -- start 2023-10-06T02:32:32.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.350+0000 7f55ce19d700 1 -- start start 2023-10-06T02:32:32.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.351+0000 7f55ce19d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f55c815dfd0 0x7f55c80aedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:32.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.351+0000 7f55ce19d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55c80acc00 con 0x7f55c8159550 2023-10-06T02:32:32.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.351+0000 7f55ce19d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55c80abff0 con 0x7f55c8159920 2023-10-06T02:32:32.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.351+0000 7f55ce19d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55c80ac2d0 con 0x7f55c815dfd0 2023-10-06T02:32:32.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.351+0000 7f55cd19b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f55c8159550 0x7f55c80ceac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:43842/0 (socket says 172.21.15.31:43842) 2023-10-06T02:32:32.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.351+0000 7f55cd19b700 1 -- 172.21.15.31:0/2266055013 learned_addr learned my addr 172.21.15.31:0/2266055013 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:32:32.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.352+0000 7f55cc99a700 1 --2- 172.21.15.31:0/2266055013 >> v2:172.21.15.62:3300/0 conn(0x7f55c815dfd0 0x7f55c80aedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:32.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.352+0000 7f55bffff700 1 -- 172.21.15.31:0/2266055013 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1339462739 0 0) 0x7f55c80acc00 con 0x7f55c8159550 2023-10-06T02:32:32.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.352+0000 7f55bffff700 1 -- 172.21.15.31:0/2266055013 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55b4003570 con 0x7f55c8159550 2023-10-06T02:32:32.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.352+0000 7f55bffff700 1 -- 172.21.15.31:0/2266055013 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2629508945 0 0) 0x7f55b4003570 con 0x7f55c8159550 2023-10-06T02:32:32.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.352+0000 7f55bffff700 1 -- 172.21.15.31:0/2266055013 >> v2:172.21.15.62:3300/0 conn(0x7f55c815dfd0 msgr2=0x7f55c80aedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:32.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.352+0000 7f55bffff700 1 --2- 172.21.15.31:0/2266055013 >> v2:172.21.15.62:3300/0 conn(0x7f55c815dfd0 0x7f55c80aedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:32.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.352+0000 7f55bffff700 1 -- 172.21.15.31:0/2266055013 >> v1:172.21.15.62:6789/0 conn(0x7f55c8159920 legacy=0x7f55c80ad090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:32.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.352+0000 7f55bffff700 1 -- 172.21.15.31:0/2266055013 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55c80ac5b0 con 0x7f55c8159550 2023-10-06T02:32:32.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.352+0000 7f55bffff700 1 -- 172.21.15.31:0/2266055013 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7f55c4001470 con 0x7f55c8159550 2023-10-06T02:32:32.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.352+0000 7f55bffff700 1 -- 172.21.15.31:0/2266055013 >> v1:172.21.15.31:6789/0 conn(0x7f55c8159550 legacy=0x7f55c80ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:32.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.352+0000 7f55bffff700 1 --2- 172.21.15.31:0/2266055013 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55b4003a60 0x7f55b4003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:32.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.352+0000 7f55bffff700 1 --2- 172.21.15.31:0/2266055013 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55b4004810 0x7f55b4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:32.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.352+0000 7f55bffff700 1 --2- 172.21.15.31:0/2266055013 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55b40051c0 0x7f55b400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:32.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.352+0000 7f55bffff700 1 -- 172.21.15.31:0/2266055013 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c80ac2d0 con 0x7f55b40051c0 2023-10-06T02:32:32.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.352+0000 7f55bffff700 1 -- 172.21.15.31:0/2266055013 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55b400c0b0 con 0x7f55b4004810 2023-10-06T02:32:32.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.352+0000 7f55bffff700 1 -- 172.21.15.31:0/2266055013 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55b400c270 con 0x7f55b4003a60 2023-10-06T02:32:32.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.353+0000 7f55cc99a700 1 --2- 172.21.15.31:0/2266055013 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55b4004810 0x7f55b4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:32.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.353+0000 7f55cd19b700 1 --2- 172.21.15.31:0/2266055013 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55b4003a60 0x7f55b4003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:32.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.353+0000 7f55cd99c700 1 --2- 172.21.15.31:0/2266055013 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55b40051c0 0x7f55b400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:32.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.353+0000 7f55cc99a700 1 -- 172.21.15.31:0/2266055013 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55b4003a60 msgr2=0x7f55b4003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:32.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.353+0000 7f55cc99a700 1 --2- 172.21.15.31:0/2266055013 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55b4003a60 0x7f55b4003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:32.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.353+0000 7f55cc99a700 1 -- 172.21.15.31:0/2266055013 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55b40051c0 msgr2=0x7f55b400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:32.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.353+0000 7f55cc99a700 1 --2- 172.21.15.31:0/2266055013 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55b40051c0 0x7f55b400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:32.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.353+0000 7f55cc99a700 1 -- 172.21.15.31:0/2266055013 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55b4003570 con 0x7f55b4004810 2023-10-06T02:32:32.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.353+0000 7f55cd99c700 1 --2- 172.21.15.31:0/2266055013 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55b40051c0 0x7f55b400ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:32:32.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.353+0000 7f55cd99c700 1 -- 172.21.15.31:0/2266055013 reap_dead start 2023-10-06T02:32:32.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.353+0000 7f55cc99a700 1 --2- 172.21.15.31:0/2266055013 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55b4004810 0x7f55b4004c80 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f55b8009c00 tx=0x7f55b80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae6a67ff5dfc8e74 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:32.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.354+0000 7f55bffff700 1 -- 172.21.15.31:0/2266055013 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b800e710 con 0x7f55b4004810 2023-10-06T02:32:32.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.354+0000 7f55bffff700 1 -- 172.21.15.31:0/2266055013 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55b800e870 con 0x7f55b4004810 2023-10-06T02:32:32.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.354+0000 7f55bffff700 1 -- 172.21.15.31:0/2266055013 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b800ec30 con 0x7f55b4004810 2023-10-06T02:32:32.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.354+0000 7f55ce19d700 1 -- 172.21.15.31:0/2266055013 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55b4004810 msgr2=0x7f55b4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:32.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.354+0000 7f55ce19d700 1 --2- 172.21.15.31:0/2266055013 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55b4004810 0x7f55b4004c80 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f55b8009c00 tx=0x7f55b80091a0 comp rx=0 tx=0).stop 2023-10-06T02:32:32.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.354+0000 7f55ce19d700 1 -- 172.21.15.31:0/2266055013 shutdown_connections 2023-10-06T02:32:32.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.354+0000 7f55ce19d700 1 --2- 172.21.15.31:0/2266055013 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55b4004810 0x7f55b4004c80 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:32.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.354+0000 7f55ce19d700 1 -- 172.21.15.31:0/2266055013 >> 172.21.15.31:0/2266055013 conn(0x7f55c806ce80 msgr2=0x7f55c80ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:32.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.354+0000 7f55ce19d700 1 -- 172.21.15.31:0/2266055013 shutdown_connections 2023-10-06T02:32:32.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.354+0000 7f55ce19d700 1 -- 172.21.15.31:0/2266055013 wait complete. 2023-10-06T02:32:32.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.356+0000 7f55ce19d700 1 Processor -- start 2023-10-06T02:32:32.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.356+0000 7f55ce19d700 1 -- start start 2023-10-06T02:32:32.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.357+0000 7f55ce19d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55b4004810 0x7f55c80cb770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:32.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.357+0000 7f55ce19d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55c80ccfa0 0x7f55c80afb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:32.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.357+0000 7f55ce19d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55c80b0080 0x7f55c814bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:32.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.357+0000 7f55ce19d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c80c8ab0 con 0x7f55c80b0080 2023-10-06T02:32:32.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.357+0000 7f55ce19d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c80c8cc0 con 0x7f55b4004810 2023-10-06T02:32:32.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.357+0000 7f55ce19d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55c80c8fa0 con 0x7f55c80ccfa0 2023-10-06T02:32:32.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.357+0000 7f55cd99c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55c80b0080 0x7f55c814bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:32.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.357+0000 7f55cd99c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55c80b0080 0x7f55c814bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58338/0 (socket says 172.21.15.31:58338) 2023-10-06T02:32:32.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.357+0000 7f55cd99c700 1 -- 172.21.15.31:0/1451491232 learned_addr learned my addr 172.21.15.31:0/1451491232 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:32.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.357+0000 7f55cd19b700 1 --2- 172.21.15.31:0/1451491232 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55b4004810 0x7f55c80cb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:32.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.357+0000 7f55cc99a700 1 --2- 172.21.15.31:0/1451491232 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55c80ccfa0 0x7f55c80afb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:32.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.357+0000 7f55cd99c700 1 -- 172.21.15.31:0/1451491232 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55c80ccfa0 msgr2=0x7f55c80afb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:32.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.357+0000 7f55cd99c700 1 --2- 172.21.15.31:0/1451491232 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55c80ccfa0 0x7f55c80afb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:32.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.357+0000 7f55cd99c700 1 -- 172.21.15.31:0/1451491232 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55b4004810 msgr2=0x7f55c80cb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:32.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.357+0000 7f55cd99c700 1 --2- 172.21.15.31:0/1451491232 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55b4004810 0x7f55c80cb770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:32.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.357+0000 7f55cd99c700 1 -- 172.21.15.31:0/1451491232 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55c80c9280 con 0x7f55c80b0080 2023-10-06T02:32:32.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.358+0000 7f55cd99c700 1 --2- 172.21.15.31:0/1451491232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55c80b0080 0x7f55c814bf80 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f55c806f970 tx=0x7f55c0009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:32.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.358+0000 7f55bd7fa700 1 -- 172.21.15.31:0/1451491232 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c0019070 con 0x7f55c80b0080 2023-10-06T02:32:32.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.358+0000 7f55bd7fa700 1 -- 172.21.15.31:0/1451491232 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55c0006ba0 con 0x7f55c80b0080 2023-10-06T02:32:32.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.358+0000 7f55bd7fa700 1 -- 172.21.15.31:0/1451491232 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c000d9a0 con 0x7f55c80b0080 2023-10-06T02:32:32.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.358+0000 7f55ce19d700 1 -- 172.21.15.31:0/1451491232 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55c80c9560 con 0x7f55c80b0080 2023-10-06T02:32:32.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.358+0000 7f55ce19d700 1 -- 172.21.15.31:0/1451491232 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55c814c4c0 con 0x7f55c80b0080 2023-10-06T02:32:32.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.359+0000 7f55ce19d700 1 -- 172.21.15.31:0/1451491232 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55c80c9a80 con 0x7f55c80b0080 2023-10-06T02:32:32.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.360+0000 7f55bd7fa700 1 -- 172.21.15.31:0/1451491232 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) v1 ==== 85709+0+0 (secure 0 0 0) 0x7f55c0003030 con 0x7f55c80b0080 2023-10-06T02:32:32.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.361+0000 7f55bd7fa700 1 --2- 172.21.15.31:0/1451491232 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f55ac0679e0 0x7f55ac069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:32.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.361+0000 7f55bd7fa700 1 -- 172.21.15.31:0/1451491232 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5320+0+0 (secure 0 0 0) 0x7f55c0082d80 con 0x7f55c80b0080 2023-10-06T02:32:32.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.361+0000 7f55cd19b700 1 --2- 172.21.15.31:0/1451491232 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f55ac0679e0 0x7f55ac069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:32.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.362+0000 7f55cd19b700 1 --2- 172.21.15.31:0/1451491232 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f55ac0679e0 0x7f55ac069e90 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f55c4001050 tx=0x7f55c4009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:32.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.365+0000 7f55bd7fa700 1 -- 172.21.15.31:0/1451491232 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f55c80c9a80 con 0x7f55c80b0080 2023-10-06T02:32:32.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.571+0000 7f55ce19d700 1 -- 172.21.15.31:0/1451491232 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f55c80c9a80 con 0x7f55c80b0080 2023-10-06T02:32:32.573 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T02:32:32.578 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T02:32:32.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.574+0000 7f55bd7fa700 1 -- 172.21.15.31:0/1451491232 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v31) v1 ==== 104+0+0 (secure 0 0 0) 0x7f55c80c9a80 con 0x7f55c80b0080 2023-10-06T02:32:32.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.577+0000 7f55ce19d700 1 -- 172.21.15.31:0/1451491232 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f55ac0679e0 msgr2=0x7f55ac069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:32.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.577+0000 7f55ce19d700 1 --2- 172.21.15.31:0/1451491232 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f55ac0679e0 0x7f55ac069e90 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f55c4001050 tx=0x7f55c4009040 comp rx=0 tx=0).stop 2023-10-06T02:32:32.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.577+0000 7f55ce19d700 1 -- 172.21.15.31:0/1451491232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55c80b0080 msgr2=0x7f55c814bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:32.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.577+0000 7f55ce19d700 1 --2- 172.21.15.31:0/1451491232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55c80b0080 0x7f55c814bf80 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f55c806f970 tx=0x7f55c0009040 comp rx=0 tx=0).stop 2023-10-06T02:32:32.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.577+0000 7f55ce19d700 1 -- 172.21.15.31:0/1451491232 shutdown_connections 2023-10-06T02:32:32.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.577+0000 7f55ce19d700 1 --2- 172.21.15.31:0/1451491232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55c80b0080 0x7f55c814bf80 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:32.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.577+0000 7f55ce19d700 1 --2- 172.21.15.31:0/1451491232 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55b4004810 0x7f55c80cb770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:32.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.577+0000 7f55ce19d700 1 --2- 172.21.15.31:0/1451491232 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f55ac0679e0 0x7f55ac069e90 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:32.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.577+0000 7f55ce19d700 1 --2- 172.21.15.31:0/1451491232 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55c80ccfa0 0x7f55c80afb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:32.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.577+0000 7f55ce19d700 1 -- 172.21.15.31:0/1451491232 >> 172.21.15.31:0/1451491232 conn(0x7f55c806ce80 msgr2=0x7f55c814d230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:32.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.577+0000 7f55ce19d700 1 -- 172.21.15.31:0/1451491232 shutdown_connections 2023-10-06T02:32:32.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.577+0000 7f55ce19d700 1 -- 172.21.15.31:0/1451491232 wait complete. 2023-10-06T02:32:32.596 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade' 2023-10-06T02:32:32.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.822+0000 7f97a319d700 1 Processor -- start 2023-10-06T02:32:32.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.823+0000 7f97a319d700 1 -- start start 2023-10-06T02:32:32.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.824+0000 7f97a319d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f979c13fef0 0x7f979c15a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:32.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.824+0000 7f97a319d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f979c06aaa0 0x7f979c0c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:32.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.824+0000 7f97a319d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f979c15ae70 con 0x7f979c13fb20 2023-10-06T02:32:32.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.824+0000 7f97a319d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f979c06af60 con 0x7f979c06aaa0 2023-10-06T02:32:32.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.824+0000 7f97a319d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f979c06b240 con 0x7f979c13fef0 2023-10-06T02:32:32.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.824+0000 7f97a219b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f979c13fb20 0x7f979c0c7990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:43848/0 (socket says 172.21.15.31:43848) 2023-10-06T02:32:32.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.824+0000 7f97a219b700 1 -- 172.21.15.31:0/1209185465 learned_addr learned my addr 172.21.15.31:0/1209185465 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:32:32.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.824+0000 7f97a299c700 1 --2- 172.21.15.31:0/1209185465 >> v2:172.21.15.31:3300/0 conn(0x7f979c06aaa0 0x7f979c0c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:32.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.824+0000 7f97a199a700 1 --2- 172.21.15.31:0/1209185465 >> v2:172.21.15.62:3300/0 conn(0x7f979c13fef0 0x7f979c15a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:32.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.825+0000 7f97a1199700 1 -- 172.21.15.31:0/1209185465 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3148123025 0 0) 0x7f9798001110 con 0x7f979c13fb20 2023-10-06T02:32:32.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.825+0000 7f97a1199700 1 -- 172.21.15.31:0/1209185465 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9788003570 con 0x7f979c13fb20 2023-10-06T02:32:32.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.825+0000 7f97a299c700 1 -- 172.21.15.31:0/1209185465 >> v2:172.21.15.62:3300/0 conn(0x7f979c13fef0 msgr2=0x7f979c15a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:32.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.825+0000 7f97a299c700 1 --2- 172.21.15.31:0/1209185465 >> v2:172.21.15.62:3300/0 conn(0x7f979c13fef0 0x7f979c15a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:32.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.825+0000 7f97a299c700 1 -- 172.21.15.31:0/1209185465 >> v1:172.21.15.31:6789/0 conn(0x7f979c13fb20 legacy=0x7f979c0c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:32.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.825+0000 7f97a299c700 1 -- 172.21.15.31:0/1209185465 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f979c06b520 con 0x7f979c06aaa0 2023-10-06T02:32:32.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.825+0000 7f97a299c700 1 --2- 172.21.15.31:0/1209185465 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f979c06aaa0 0x7f979c0c9d00 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f979c06a5e0 tx=0x7f9794009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7f8aeaf8f49aad7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:32.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.825+0000 7f97a1199700 1 -- 172.21.15.31:0/1209185465 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f979400e070 con 0x7f979c06aaa0 2023-10-06T02:32:32.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.825+0000 7f97a1199700 1 -- 172.21.15.31:0/1209185465 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f979400bab0 con 0x7f979c06aaa0 2023-10-06T02:32:32.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.825+0000 7f97a1199700 1 -- 172.21.15.31:0/1209185465 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f979400be30 con 0x7f979c06aaa0 2023-10-06T02:32:32.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.826+0000 7f97a319d700 1 -- 172.21.15.31:0/1209185465 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f979c06aaa0 msgr2=0x7f979c0c9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:32.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.826+0000 7f97a319d700 1 --2- 172.21.15.31:0/1209185465 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f979c06aaa0 0x7f979c0c9d00 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f979c06a5e0 tx=0x7f9794009780 comp rx=0 tx=0).stop 2023-10-06T02:32:32.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.826+0000 7f97a319d700 1 -- 172.21.15.31:0/1209185465 shutdown_connections 2023-10-06T02:32:32.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.826+0000 7f97a319d700 1 --2- 172.21.15.31:0/1209185465 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f979c06aaa0 0x7f979c0c9d00 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:32.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.826+0000 7f97a319d700 1 --2- 172.21.15.31:0/1209185465 >> v2:172.21.15.62:3300/0 conn(0x7f979c13fef0 0x7f979c15a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:32.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.826+0000 7f97a319d700 1 -- 172.21.15.31:0/1209185465 >> 172.21.15.31:0/1209185465 conn(0x7f979c06d170 msgr2=0x7f979c0cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:32.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.826+0000 7f97a319d700 1 -- 172.21.15.31:0/1209185465 shutdown_connections 2023-10-06T02:32:32.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.826+0000 7f97a319d700 1 -- 172.21.15.31:0/1209185465 wait complete. 2023-10-06T02:32:32.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.828+0000 7f97a319d700 1 Processor -- start 2023-10-06T02:32:32.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.828+0000 7f97a319d700 1 -- start start 2023-10-06T02:32:32.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.829+0000 7f97a319d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f979c06aaa0 0x7f979c155b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:32.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.829+0000 7f97a319d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f979c13fb20 0x7f979c0ac200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:32.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.829+0000 7f97a319d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f979c13fef0 0x7f979c0adb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:32.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.829+0000 7f97a319d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f979c13b900 con 0x7f979c06aaa0 2023-10-06T02:32:32.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.829+0000 7f97a319d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f979c0acc80 con 0x7f979c13fef0 2023-10-06T02:32:32.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.829+0000 7f97a319d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f979c0acf60 con 0x7f979c13fb20 2023-10-06T02:32:32.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.829+0000 7f97a219b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f979c06aaa0 0x7f979c155b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:32.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.829+0000 7f97a219b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f979c06aaa0 0x7f979c155b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58360/0 (socket says 172.21.15.31:58360) 2023-10-06T02:32:32.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.829+0000 7f97a219b700 1 -- 172.21.15.31:0/2403615645 learned_addr learned my addr 172.21.15.31:0/2403615645 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:32.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.829+0000 7f97a299c700 1 --2- 172.21.15.31:0/2403615645 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f979c13fef0 0x7f979c0adb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:32.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.829+0000 7f97a199a700 1 --2- 172.21.15.31:0/2403615645 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f979c13fb20 0x7f979c0ac200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:32.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.829+0000 7f97a219b700 1 -- 172.21.15.31:0/2403615645 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f979c13fb20 msgr2=0x7f979c0ac200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:32.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.829+0000 7f97a219b700 1 --2- 172.21.15.31:0/2403615645 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f979c13fb20 0x7f979c0ac200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:32.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.829+0000 7f97a219b700 1 -- 172.21.15.31:0/2403615645 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f979c13fef0 msgr2=0x7f979c0adb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:32.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.829+0000 7f97a219b700 1 --2- 172.21.15.31:0/2403615645 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f979c13fef0 0x7f979c0adb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:32.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.829+0000 7f97a219b700 1 -- 172.21.15.31:0/2403615645 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9794009a20 con 0x7f979c06aaa0 2023-10-06T02:32:32.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.829+0000 7f97a299c700 1 --2- 172.21.15.31:0/2403615645 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f979c13fef0 0x7f979c0adb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:32:32.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.830+0000 7f97a219b700 1 --2- 172.21.15.31:0/2403615645 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f979c06aaa0 0x7f979c155b40 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f979800c7c0 tx=0x7f9798007c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:32.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.830+0000 7f97927fc700 1 -- 172.21.15.31:0/2403615645 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9798005b30 con 0x7f979c06aaa0 2023-10-06T02:32:32.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.830+0000 7f97927fc700 1 -- 172.21.15.31:0/2403615645 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9798005c90 con 0x7f979c06aaa0 2023-10-06T02:32:32.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.830+0000 7f97927fc700 1 -- 172.21.15.31:0/2403615645 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9798007320 con 0x7f979c06aaa0 2023-10-06T02:32:32.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.830+0000 7f97a319d700 1 -- 172.21.15.31:0/2403615645 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f979c13bb50 con 0x7f979c06aaa0 2023-10-06T02:32:32.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.830+0000 7f97a319d700 1 -- 172.21.15.31:0/2403615645 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f979c156b30 con 0x7f979c06aaa0 2023-10-06T02:32:32.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.831+0000 7f97a319d700 1 -- 172.21.15.31:0/2403615645 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f979c04c3c0 con 0x7f979c06aaa0 2023-10-06T02:32:32.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.832+0000 7f97927fc700 1 -- 172.21.15.31:0/2403615645 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) v1 ==== 85709+0+0 (secure 0 0 0) 0x7f979801b3b0 con 0x7f979c06aaa0 2023-10-06T02:32:32.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.833+0000 7f97927fc700 1 --2- 172.21.15.31:0/2403615645 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f97800679e0 0x7f9780069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:32.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.833+0000 7f97927fc700 1 -- 172.21.15.31:0/2403615645 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5320+0+0 (secure 0 0 0) 0x7f9798082460 con 0x7f979c06aaa0 2023-10-06T02:32:32.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.833+0000 7f97a199a700 1 --2- 172.21.15.31:0/2403615645 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f97800679e0 0x7f9780069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:32.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.834+0000 7f97a199a700 1 --2- 172.21.15.31:0/2403615645 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f97800679e0 0x7f9780069e90 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f979c0b1e60 tx=0x7f978c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:32.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:32.837+0000 7f97927fc700 1 -- 172.21.15.31:0/2403615645 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9798054980 con 0x7f979c06aaa0 2023-10-06T02:32:33.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:33.123+0000 7f97a319d700 1 -- 172.21.15.31:0/2403615645 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade"]} v 0) v1 -- 0x7f979c15bff0 con 0x7f979c06aaa0 2023-10-06T02:32:34.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:34.121+0000 7f97927fc700 1 -- 172.21.15.31:0/2403615645 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade"]}]=0 v527) v1 ==== 163+0+0 (secure 0 0 0) 0x7f979c15bff0 con 0x7f979c06aaa0 2023-10-06T02:32:34.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:34.124+0000 7f97a319d700 1 -- 172.21.15.31:0/2403615645 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f97800679e0 msgr2=0x7f9780069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:34.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:34.124+0000 7f97a319d700 1 --2- 172.21.15.31:0/2403615645 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f97800679e0 0x7f9780069e90 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f979c0b1e60 tx=0x7f978c008040 comp rx=0 tx=0).stop 2023-10-06T02:32:34.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:34.124+0000 7f97a319d700 1 -- 172.21.15.31:0/2403615645 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f979c06aaa0 msgr2=0x7f979c155b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:34.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:34.124+0000 7f97a319d700 1 --2- 172.21.15.31:0/2403615645 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f979c06aaa0 0x7f979c155b40 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f979800c7c0 tx=0x7f9798007c30 comp rx=0 tx=0).stop 2023-10-06T02:32:34.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:34.124+0000 7f97a319d700 1 -- 172.21.15.31:0/2403615645 shutdown_connections 2023-10-06T02:32:34.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:34.124+0000 7f97a319d700 1 --2- 172.21.15.31:0/2403615645 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f979c13fef0 0x7f979c0adb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:34.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:34.124+0000 7f97a319d700 1 --2- 172.21.15.31:0/2403615645 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f979c06aaa0 0x7f979c155b40 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:34.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:34.124+0000 7f97a319d700 1 --2- 172.21.15.31:0/2403615645 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f97800679e0 0x7f9780069e90 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:34.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:34.124+0000 7f97a319d700 1 --2- 172.21.15.31:0/2403615645 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f979c13fb20 0x7f979c0ac200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:34.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:34.124+0000 7f97a319d700 1 -- 172.21.15.31:0/2403615645 >> 172.21.15.31:0/2403615645 conn(0x7f979c06d170 msgr2=0x7f979c0aeed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:34.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:34.125+0000 7f97a319d700 1 -- 172.21.15.31:0/2403615645 shutdown_connections 2023-10-06T02:32:34.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:34.125+0000 7f97a319d700 1 -- 172.21.15.31:0/2403615645 wait complete. 2023-10-06T02:32:34.147 INFO:tasks.cephfs_test_runner:Validate unparseable text .meta file on legacy subvol root doesn't break the system ... ok 2023-10-06T02:32:34.147 INFO:tasks.cephfs_test_runner: 2023-10-06T02:32:34.147 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-10-06T02:32:34.147 INFO:tasks.cephfs_test_runner:Ran 11 tests in 521.498s 2023-10-06T02:32:34.147 INFO:tasks.cephfs_test_runner: 2023-10-06T02:32:34.147 INFO:tasks.cephfs_test_runner:OK 2023-10-06T02:32:34.147 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-10-06T02:32:34.157 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-10-06T02:32:34.196 INFO:tasks.kclient:Unmounting kernel clients... 2023-10-06T02:32:34.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:32:34.196 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2023-10-06T02:32:34.235 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T02:32:34.235 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T02:32:34.236 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:32:34.236 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T02:32:34.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:32:34.341 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T02:32:34.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:32:34.408 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link delete ceph-brx 2023-10-06T02:32:34.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:32:34.528 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2023-10-06T02:32:34.566 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:32:34.566 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link delete ceph-brx 2023-10-06T02:32:34.675 INFO:teuthology.orchestra.run.smithi031.stderr:Cannot find device "ceph-brx" 2023-10-06T02:32:34.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T02:32:34.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:32:34.678 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2023-10-06T02:32:34.719 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-06T02:32:34.719 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T02:32:34.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:32:34.720 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T02:32:34.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:32:34.829 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T02:32:34.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:32:34.912 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link delete ceph-brx 2023-10-06T02:32:35.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:32:35.034 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2023-10-06T02:32:35.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:32:35.073 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link delete ceph-brx 2023-10-06T02:32:35.181 INFO:teuthology.orchestra.run.smithi062.stderr:Cannot find device "ceph-brx" 2023-10-06T02:32:35.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T02:32:35.184 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-10-06T02:32:35.193 INFO:tasks.ceph.ceph_manager.ceph:Canceling any pending splits or merges... 2023-10-06T02:32:35.193 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:32:35.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.387+0000 7f3e19b9d700 1 Processor -- start 2023-10-06T02:32:35.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.387+0000 7f3e19b9d700 1 -- start start 2023-10-06T02:32:35.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.388+0000 7f3e19b9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3e1406f3a0 0x7f3e1414fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:35.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.388+0000 7f3e19b9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3e1406e470 0x7f3e141501b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:35.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.388+0000 7f3e19b9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e1406a490 con 0x7f3e1406e0a0 2023-10-06T02:32:35.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.388+0000 7f3e19b9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e140c7230 con 0x7f3e1406f3a0 2023-10-06T02:32:35.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.388+0000 7f3e19b9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e140c7490 con 0x7f3e1406e470 2023-10-06T02:32:35.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.388+0000 7f3e18b9b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f3e1406e0a0 0x7f3e140c6b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:43852/0 (socket says 172.21.15.31:43852) 2023-10-06T02:32:35.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.388+0000 7f3e18b9b700 1 -- 172.21.15.31:0/1763890104 learned_addr learned my addr 172.21.15.31:0/1763890104 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:32:35.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.389+0000 7f3e1939c700 1 --2- 172.21.15.31:0/1763890104 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3e1406e470 0x7f3e141501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:35.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.389+0000 7f3e13fff700 1 --2- 172.21.15.31:0/1763890104 >> v2:172.21.15.62:3300/0 conn(0x7f3e1406f3a0 0x7f3e1414fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:35.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.389+0000 7f3e137fe700 1 -- 172.21.15.31:0/1763890104 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 384736983 0 0) 0x7f3e1406a490 con 0x7f3e1406e0a0 2023-10-06T02:32:35.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.389+0000 7f3e137fe700 1 -- 172.21.15.31:0/1763890104 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e04003570 con 0x7f3e1406e0a0 2023-10-06T02:32:35.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.389+0000 7f3e13fff700 1 -- 172.21.15.31:0/1763890104 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3e1406e470 msgr2=0x7f3e141501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:35.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.389+0000 7f3e137fe700 1 -- 172.21.15.31:0/1763890104 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3277354159 0 0) 0x7f3e04003570 con 0x7f3e1406e0a0 2023-10-06T02:32:35.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.389+0000 7f3e13fff700 1 --2- 172.21.15.31:0/1763890104 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3e1406e470 0x7f3e141501b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:35.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.389+0000 7f3e13fff700 1 -- 172.21.15.31:0/1763890104 >> v1:172.21.15.31:6789/0 conn(0x7f3e1406e0a0 legacy=0x7f3e140c6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:35.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.389+0000 7f3e13fff700 1 -- 172.21.15.31:0/1763890104 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e140c7770 con 0x7f3e1406f3a0 2023-10-06T02:32:35.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.390+0000 7f3e13fff700 1 --2- 172.21.15.31:0/1763890104 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e1406f3a0 0x7f3e1414fc70 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f3e000098d0 tx=0x7f3e00009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=719c915775a38693 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:35.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.390+0000 7f3e137fe700 1 -- 172.21.15.31:0/1763890104 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e08001470 con 0x7f3e1406f3a0 2023-10-06T02:32:35.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.390+0000 7f3e137fe700 1 -- 172.21.15.31:0/1763890104 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e00011420 con 0x7f3e1406f3a0 2023-10-06T02:32:35.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.390+0000 7f3e137fe700 1 -- 172.21.15.31:0/1763890104 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e000117a0 con 0x7f3e1406f3a0 2023-10-06T02:32:35.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.390+0000 7f3e19b9d700 1 -- 172.21.15.31:0/1763890104 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e1406f3a0 msgr2=0x7f3e1414fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:35.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.390+0000 7f3e19b9d700 1 --2- 172.21.15.31:0/1763890104 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e1406f3a0 0x7f3e1414fc70 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f3e000098d0 tx=0x7f3e00009120 comp rx=0 tx=0).stop 2023-10-06T02:32:35.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.390+0000 7f3e19b9d700 1 -- 172.21.15.31:0/1763890104 shutdown_connections 2023-10-06T02:32:35.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.390+0000 7f3e19b9d700 1 --2- 172.21.15.31:0/1763890104 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3e1406e470 0x7f3e141501b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:35.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.390+0000 7f3e19b9d700 1 --2- 172.21.15.31:0/1763890104 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e1406f3a0 0x7f3e1414fc70 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:35.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.390+0000 7f3e19b9d700 1 -- 172.21.15.31:0/1763890104 >> 172.21.15.31:0/1763890104 conn(0x7f3e1406d170 msgr2=0x7f3e140cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:35.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.390+0000 7f3e19b9d700 1 -- 172.21.15.31:0/1763890104 shutdown_connections 2023-10-06T02:32:35.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.391+0000 7f3e19b9d700 1 -- 172.21.15.31:0/1763890104 wait complete. 2023-10-06T02:32:35.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.392+0000 7f3e19b9d700 1 Processor -- start 2023-10-06T02:32:35.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.392+0000 7f3e19b9d700 1 -- start start 2023-10-06T02:32:35.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.393+0000 7f3e19b9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e1406e0a0 0x7f3e1414bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:35.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.393+0000 7f3e19b9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e1406e470 0x7f3e1413eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:35.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.393+0000 7f3e19b9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3e1406f3a0 0x7f3e140b16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:35.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.393+0000 7f3e19b9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e140c4800 con 0x7f3e1406e0a0 2023-10-06T02:32:35.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.393+0000 7f3e19b9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e140aa370 con 0x7f3e1406e470 2023-10-06T02:32:35.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.393+0000 7f3e19b9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e140aa650 con 0x7f3e1406f3a0 2023-10-06T02:32:35.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.393+0000 7f3e18b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e1406e0a0 0x7f3e1414bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:35.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.393+0000 7f3e18b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e1406e0a0 0x7f3e1414bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58368/0 (socket says 172.21.15.31:58368) 2023-10-06T02:32:35.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.393+0000 7f3e18b9b700 1 -- 172.21.15.31:0/2894813405 learned_addr learned my addr 172.21.15.31:0/2894813405 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:35.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.393+0000 7f3e1939c700 1 --2- 172.21.15.31:0/2894813405 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3e1406f3a0 0x7f3e140b16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:35.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.393+0000 7f3e13fff700 1 --2- 172.21.15.31:0/2894813405 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e1406e470 0x7f3e1413eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:35.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.394+0000 7f3e18b9b700 1 -- 172.21.15.31:0/2894813405 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3e1406f3a0 msgr2=0x7f3e140b16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:35.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.394+0000 7f3e18b9b700 1 --2- 172.21.15.31:0/2894813405 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3e1406f3a0 0x7f3e140b16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:35.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.394+0000 7f3e18b9b700 1 -- 172.21.15.31:0/2894813405 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e1406e470 msgr2=0x7f3e1413eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:35.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.394+0000 7f3e18b9b700 1 --2- 172.21.15.31:0/2894813405 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e1406e470 0x7f3e1413eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:35.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.394+0000 7f3e18b9b700 1 -- 172.21.15.31:0/2894813405 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e000093c0 con 0x7f3e1406e0a0 2023-10-06T02:32:35.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.394+0000 7f3e18b9b700 1 --2- 172.21.15.31:0/2894813405 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e1406e0a0 0x7f3e1414bc10 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f3e0800cb90 tx=0x7f3e08001aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:35.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.394+0000 7f3e10ff9700 1 -- 172.21.15.31:0/2894813405 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e08007150 con 0x7f3e1406e0a0 2023-10-06T02:32:35.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.394+0000 7f3e10ff9700 1 -- 172.21.15.31:0/2894813405 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e080072f0 con 0x7f3e1406e0a0 2023-10-06T02:32:35.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.394+0000 7f3e10ff9700 1 -- 172.21.15.31:0/2894813405 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e080076b0 con 0x7f3e1406e0a0 2023-10-06T02:32:35.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.394+0000 7f3e19b9d700 1 -- 172.21.15.31:0/2894813405 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e1413f030 con 0x7f3e1406e0a0 2023-10-06T02:32:35.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.394+0000 7f3e19b9d700 1 -- 172.21.15.31:0/2894813405 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e1414c370 con 0x7f3e1406e0a0 2023-10-06T02:32:35.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.396+0000 7f3e19b9d700 1 -- 172.21.15.31:0/2894813405 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e1404c3c0 con 0x7f3e1406e0a0 2023-10-06T02:32:35.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.396+0000 7f3e10ff9700 1 -- 172.21.15.31:0/2894813405 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) v1 ==== 85709+0+0 (secure 0 0 0) 0x7f3e08004a40 con 0x7f3e1406e0a0 2023-10-06T02:32:35.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.397+0000 7f3e10ff9700 1 --2- 172.21.15.31:0/2894813405 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f3df80679e0 0x7f3df8069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:35.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.397+0000 7f3e10ff9700 1 -- 172.21.15.31:0/2894813405 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5320+0+0 (secure 0 0 0) 0x7f3e080826d0 con 0x7f3e1406e0a0 2023-10-06T02:32:35.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.398+0000 7f3e13fff700 1 --2- 172.21.15.31:0/2894813405 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f3df80679e0 0x7f3df8069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:35.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.398+0000 7f3e13fff700 1 --2- 172.21.15.31:0/2894813405 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f3df80679e0 0x7f3df8069e90 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f3e000030c0 tx=0x7f3e00016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:35.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.401+0000 7f3e10ff9700 1 -- 172.21.15.31:0/2894813405 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3e0804c5a0 con 0x7f3e1406e0a0 2023-10-06T02:32:35.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.605+0000 7f3e19b9d700 1 -- 172.21.15.31:0/2894813405 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3e140c7a50 con 0x7f3e1406e0a0 2023-10-06T02:32:35.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.606+0000 7f3e10ff9700 1 -- 172.21.15.31:0/2894813405 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v172) v1 ==== 74+0+13785 (secure 0 0 0) 0x7f3e08051270 con 0x7f3e1406e0a0 2023-10-06T02:32:35.607 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:32:35.607 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":172,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:32:32.111235+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:6834/3741373122":"2023-10-07T02:32:28.700776+0000","172.21.15.31:6832/3741373122":"2023-10-07T02:32:28.700776+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":20,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:32:35.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.608+0000 7f3e19b9d700 1 -- 172.21.15.31:0/2894813405 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f3df80679e0 msgr2=0x7f3df8069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:35.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.608+0000 7f3e19b9d700 1 --2- 172.21.15.31:0/2894813405 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f3df80679e0 0x7f3df8069e90 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f3e000030c0 tx=0x7f3e00016000 comp rx=0 tx=0).stop 2023-10-06T02:32:35.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.608+0000 7f3e19b9d700 1 -- 172.21.15.31:0/2894813405 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e1406e0a0 msgr2=0x7f3e1414bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:35.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.608+0000 7f3e19b9d700 1 --2- 172.21.15.31:0/2894813405 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e1406e0a0 0x7f3e1414bc10 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f3e0800cb90 tx=0x7f3e08001aa0 comp rx=0 tx=0).stop 2023-10-06T02:32:35.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.608+0000 7f3e19b9d700 1 -- 172.21.15.31:0/2894813405 shutdown_connections 2023-10-06T02:32:35.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.608+0000 7f3e19b9d700 1 --2- 172.21.15.31:0/2894813405 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f3df80679e0 0x7f3df8069e90 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:35.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.608+0000 7f3e19b9d700 1 --2- 172.21.15.31:0/2894813405 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3e1406f3a0 0x7f3e140b16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:35.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.608+0000 7f3e19b9d700 1 --2- 172.21.15.31:0/2894813405 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e1406e0a0 0x7f3e1414bc10 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:35.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.608+0000 7f3e19b9d700 1 --2- 172.21.15.31:0/2894813405 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e1406e470 0x7f3e1413eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:35.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.608+0000 7f3e19b9d700 1 -- 172.21.15.31:0/2894813405 >> 172.21.15.31:0/2894813405 conn(0x7f3e1406d170 msgr2=0x7f3e140caec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:35.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.608+0000 7f3e19b9d700 1 -- 172.21.15.31:0/2894813405 shutdown_connections 2023-10-06T02:32:35.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.608+0000 7f3e19b9d700 1 -- 172.21.15.31:0/2894813405 wait complete. 2023-10-06T02:32:35.627 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-06T02:32:35.627 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T02:32:35.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.812+0000 7f1689b9d700 1 Processor -- start 2023-10-06T02:32:35.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.812+0000 7f1689b9d700 1 -- start start 2023-10-06T02:32:35.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.814+0000 7f1689b9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1684159540 0x7f16840c1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:35.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.814+0000 7f1689b9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f16840c6be0 0x7f16840c6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:35.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.814+0000 7f1689b9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16840c7520 con 0x7f1684159910 2023-10-06T02:32:35.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.814+0000 7f1689b9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1684068250 con 0x7f1684159540 2023-10-06T02:32:35.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.814+0000 7f1689b9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1684068530 con 0x7f16840c6be0 2023-10-06T02:32:35.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.814+0000 7f1688b9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1684159540 0x7f16840c1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:35.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.814+0000 7f1683fff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f1684159910 0x7f16840677b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:43862/0 (socket says 172.21.15.31:43862) 2023-10-06T02:32:35.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.814+0000 7f1683fff700 1 -- 172.21.15.31:0/865655377 learned_addr learned my addr 172.21.15.31:0/865655377 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:32:35.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.814+0000 7f168939c700 1 --2- 172.21.15.31:0/865655377 >> v2:172.21.15.62:3300/0 conn(0x7f16840c6be0 0x7f16840c6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:35.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.815+0000 7f16837fe700 1 -- 172.21.15.31:0/865655377 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2125098346 0 0) 0x7f16840c7520 con 0x7f1684159910 2023-10-06T02:32:35.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.815+0000 7f1688b9b700 1 -- 172.21.15.31:0/865655377 >> v2:172.21.15.62:3300/0 conn(0x7f16840c6be0 msgr2=0x7f16840c6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:35.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.815+0000 7f1688b9b700 1 --2- 172.21.15.31:0/865655377 >> v2:172.21.15.62:3300/0 conn(0x7f16840c6be0 0x7f16840c6fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:35.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.815+0000 7f1688b9b700 1 -- 172.21.15.31:0/865655377 >> v1:172.21.15.31:6789/0 conn(0x7f1684159910 legacy=0x7f16840677b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:35.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.815+0000 7f1688b9b700 1 -- 172.21.15.31:0/865655377 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1684068810 con 0x7f1684159540 2023-10-06T02:32:35.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.815+0000 7f1688b9b700 1 --2- 172.21.15.31:0/865655377 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1684159540 0x7f16840c1340 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f1684069ec0 tx=0x7f16780091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d17082261d22784e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:35.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.815+0000 7f16837fe700 1 -- 172.21.15.31:0/865655377 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1678011630 con 0x7f1684159540 2023-10-06T02:32:35.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.815+0000 7f16837fe700 1 -- 172.21.15.31:0/865655377 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1678011790 con 0x7f1684159540 2023-10-06T02:32:35.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.815+0000 7f16837fe700 1 -- 172.21.15.31:0/865655377 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1678011b30 con 0x7f1684159540 2023-10-06T02:32:35.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.816+0000 7f1689b9d700 1 -- 172.21.15.31:0/865655377 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1684159540 msgr2=0x7f16840c1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:35.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.816+0000 7f1689b9d700 1 --2- 172.21.15.31:0/865655377 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1684159540 0x7f16840c1340 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f1684069ec0 tx=0x7f16780091a0 comp rx=0 tx=0).stop 2023-10-06T02:32:35.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.816+0000 7f1689b9d700 1 -- 172.21.15.31:0/865655377 shutdown_connections 2023-10-06T02:32:35.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.816+0000 7f1689b9d700 1 --2- 172.21.15.31:0/865655377 >> v2:172.21.15.62:3300/0 conn(0x7f16840c6be0 0x7f16840c6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:35.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.816+0000 7f1689b9d700 1 --2- 172.21.15.31:0/865655377 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1684159540 0x7f16840c1340 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:35.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.816+0000 7f1689b9d700 1 -- 172.21.15.31:0/865655377 >> 172.21.15.31:0/865655377 conn(0x7f168406f310 msgr2=0x7f16840cc5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:35.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.816+0000 7f1689b9d700 1 -- 172.21.15.31:0/865655377 shutdown_connections 2023-10-06T02:32:35.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.816+0000 7f1689b9d700 1 -- 172.21.15.31:0/865655377 wait complete. 2023-10-06T02:32:35.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.817+0000 7f1689b9d700 1 Processor -- start 2023-10-06T02:32:35.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.817+0000 7f1689b9d700 1 -- start start 2023-10-06T02:32:35.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.818+0000 7f1689b9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16840c6be0 0x7f1684159df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:35.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.818+0000 7f1689b9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1684159540 0x7f16840c4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:35.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.818+0000 7f1689b9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1684159910 0x7f16840c59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:35.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.818+0000 7f1689b9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16840c64e0 con 0x7f16840c6be0 2023-10-06T02:32:35.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.818+0000 7f1689b9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16840af4e0 con 0x7f1684159910 2023-10-06T02:32:35.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.818+0000 7f1689b9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16840c6200 con 0x7f1684159540 2023-10-06T02:32:35.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.819+0000 7f1688b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16840c6be0 0x7f1684159df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:35.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.819+0000 7f1688b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16840c6be0 0x7f1684159df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58386/0 (socket says 172.21.15.31:58386) 2023-10-06T02:32:35.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.819+0000 7f1688b9b700 1 -- 172.21.15.31:0/1152634614 learned_addr learned my addr 172.21.15.31:0/1152634614 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:35.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.819+0000 7f168939c700 1 --2- 172.21.15.31:0/1152634614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1684159910 0x7f16840c59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:35.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.819+0000 7f1683fff700 1 --2- 172.21.15.31:0/1152634614 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1684159540 0x7f16840c4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:35.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.819+0000 7f1688b9b700 1 -- 172.21.15.31:0/1152634614 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1684159540 msgr2=0x7f16840c4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:35.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.819+0000 7f1688b9b700 1 --2- 172.21.15.31:0/1152634614 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1684159540 0x7f16840c4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:35.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.819+0000 7f1688b9b700 1 -- 172.21.15.31:0/1152634614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1684159910 msgr2=0x7f16840c59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:35.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.819+0000 7f1688b9b700 1 --2- 172.21.15.31:0/1152634614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1684159910 0x7f16840c59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:35.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.819+0000 7f1688b9b700 1 -- 172.21.15.31:0/1152634614 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1678009420 con 0x7f16840c6be0 2023-10-06T02:32:35.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.819+0000 7f168939c700 1 --2- 172.21.15.31:0/1152634614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1684159910 0x7f16840c59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:32:35.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.819+0000 7f1688b9b700 1 --2- 172.21.15.31:0/1152634614 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16840c6be0 0x7f1684159df0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f1684071320 tx=0x7f1678017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:35.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.819+0000 7f1680ff9700 1 -- 172.21.15.31:0/1152634614 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1678011b10 con 0x7f16840c6be0 2023-10-06T02:32:35.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.820+0000 7f1680ff9700 1 -- 172.21.15.31:0/1152634614 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16780253f0 con 0x7f16840c6be0 2023-10-06T02:32:35.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.820+0000 7f1680ff9700 1 -- 172.21.15.31:0/1152634614 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1678025790 con 0x7f16840c6be0 2023-10-06T02:32:35.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.820+0000 7f1689b9d700 1 -- 172.21.15.31:0/1152634614 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16840af780 con 0x7f16840c6be0 2023-10-06T02:32:35.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.820+0000 7f1689b9d700 1 -- 172.21.15.31:0/1152634614 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16840c5f10 con 0x7f16840c6be0 2023-10-06T02:32:35.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.821+0000 7f1689b9d700 1 -- 172.21.15.31:0/1152634614 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16840af340 con 0x7f16840c6be0 2023-10-06T02:32:35.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.821+0000 7f1680ff9700 1 -- 172.21.15.31:0/1152634614 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) v1 ==== 85709+0+0 (secure 0 0 0) 0x7f1678025b40 con 0x7f16840c6be0 2023-10-06T02:32:35.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.822+0000 7f1680ff9700 1 --2- 172.21.15.31:0/1152634614 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f16680679e0 0x7f1668069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:35.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.822+0000 7f1680ff9700 1 -- 172.21.15.31:0/1152634614 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5320+0+0 (secure 0 0 0) 0x7f1678084810 con 0x7f16840c6be0 2023-10-06T02:32:35.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.823+0000 7f1683fff700 1 --2- 172.21.15.31:0/1152634614 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f16680679e0 0x7f1668069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:35.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.823+0000 7f1683fff700 1 --2- 172.21.15.31:0/1152634614 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f16680679e0 0x7f1668069e90 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f1684156d10 tx=0x7f1670009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:35.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:35.826+0000 7f1680ff9700 1 -- 172.21.15.31:0/1152634614 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f16840af340 con 0x7f16840c6be0 2023-10-06T02:32:36.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.023+0000 7f1689b9d700 1 -- 172.21.15.31:0/1152634614 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f16840af340 con 0x7f16680679e0 2023-10-06T02:32:36.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.026+0000 7f1680ff9700 1 -- 172.21.15.31:0/1152634614 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72379 (secure 0 0 0) 0x7f16840af340 con 0x7f16680679e0 2023-10-06T02:32:36.029 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:32:36.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.029+0000 7f1689b9d700 1 -- 172.21.15.31:0/1152634614 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f16680679e0 msgr2=0x7f1668069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:36.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.029+0000 7f1689b9d700 1 --2- 172.21.15.31:0/1152634614 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f16680679e0 0x7f1668069e90 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f1684156d10 tx=0x7f1670009040 comp rx=0 tx=0).stop 2023-10-06T02:32:36.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.029+0000 7f1689b9d700 1 -- 172.21.15.31:0/1152634614 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16840c6be0 msgr2=0x7f1684159df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:36.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.029+0000 7f1689b9d700 1 --2- 172.21.15.31:0/1152634614 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16840c6be0 0x7f1684159df0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f1684071320 tx=0x7f1678017000 comp rx=0 tx=0).stop 2023-10-06T02:32:36.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.029+0000 7f1689b9d700 1 -- 172.21.15.31:0/1152634614 shutdown_connections 2023-10-06T02:32:36.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.029+0000 7f1689b9d700 1 --2- 172.21.15.31:0/1152634614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1684159910 0x7f16840c59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:36.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.029+0000 7f1689b9d700 1 --2- 172.21.15.31:0/1152634614 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16840c6be0 0x7f1684159df0 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:36.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.029+0000 7f1689b9d700 1 --2- 172.21.15.31:0/1152634614 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f16680679e0 0x7f1668069e90 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:36.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.029+0000 7f1689b9d700 1 --2- 172.21.15.31:0/1152634614 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1684159540 0x7f16840c4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:36.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.029+0000 7f1689b9d700 1 -- 172.21.15.31:0/1152634614 >> 172.21.15.31:0/1152634614 conn(0x7f168406f310 msgr2=0x7f16840c1900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:36.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.029+0000 7f1689b9d700 1 -- 172.21.15.31:0/1152634614 shutdown_connections 2023-10-06T02:32:36.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.029+0000 7f1689b9d700 1 -- 172.21.15.31:0/1152634614 wait complete. 2023-10-06T02:32:36.035 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-10-06T02:32:36.048 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":248,"stamp":"2023-10-06T02:32:34.529682+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":113,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":2433352,"kb_used_data":18184,"kb_used_omap":0,"kb_used_meta":2415104,"kb_avail":752541368,"statfs":{"total":773094113280,"available":770602360832,"internally_reserved":0,"allocated":18620416,"data_stored":9311912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2473066496},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-55730,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-3,"num_read_kb":-3,"num_write":-73,"num_write_kb":-82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001569"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":137,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-06T02:32:29.366431+0000","last_change":"2023-10-06T02:23:01.992935+0000","last_active":"2023-10-06T02:32:29.366431+0000","last_peered":"2023-10-06T02:32:29.366431+0000","last_clean":"2023-10-06T02:32:29.366431+0000","last_became_active":"2023-10-06T02:23:01.992568+0000","last_became_peered":"2023-10-06T02:23:01.992568+0000","last_unstale":"2023-10-06T02:32:29.366431+0000","last_undegraded":"2023-10-06T02:32:29.366431+0000","last_fullsized":"2023-10-06T02:32:29.366431+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:06.212251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":138,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-06T02:32:28.707783+0000","last_change":"2023-10-06T02:23:01.989934+0000","last_active":"2023-10-06T02:32:28.707783+0000","last_peered":"2023-10-06T02:32:28.707783+0000","last_clean":"2023-10-06T02:32:28.707783+0000","last_became_active":"2023-10-06T02:23:01.989706+0000","last_became_peered":"2023-10-06T02:23:01.989706+0000","last_unstale":"2023-10-06T02:32:28.707783+0000","last_undegraded":"2023-10-06T02:32:28.707783+0000","last_fullsized":"2023-10-06T02:32:28.707783+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:06.212251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":131,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-06T02:32:29.229695+0000","last_change":"2023-10-06T02:23:02.054069+0000","last_active":"2023-10-06T02:32:29.229695+0000","last_peered":"2023-10-06T02:32:29.229695+0000","last_clean":"2023-10-06T02:32:29.229695+0000","last_became_active":"2023-10-06T02:23:02.053739+0000","last_became_peered":"2023-10-06T02:23:02.053739+0000","last_unstale":"2023-10-06T02:32:29.229695+0000","last_undegraded":"2023-10-06T02:32:29.229695+0000","last_fullsized":"2023-10-06T02:32:29.229695+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:57:48.822608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":131,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-06T02:32:29.229181+0000","last_change":"2023-10-06T02:23:01.990773+0000","last_active":"2023-10-06T02:32:29.229181+0000","last_peered":"2023-10-06T02:32:29.229181+0000","last_clean":"2023-10-06T02:32:29.229181+0000","last_became_active":"2023-10-06T02:23:01.990511+0000","last_became_peered":"2023-10-06T02:23:01.990511+0000","last_unstale":"2023-10-06T02:32:29.229181+0000","last_undegraded":"2023-10-06T02:32:29.229181+0000","last_fullsized":"2023-10-06T02:32:29.229181+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:42:54.797731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"76'128","reported_seq":323,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-06T02:32:29.229325+0000","last_change":"2023-10-06T02:22:55.032519+0000","last_active":"2023-10-06T02:32:29.229325+0000","last_peered":"2023-10-06T02:32:29.229325+0000","last_clean":"2023-10-06T02:32:29.229325+0000","last_became_active":"2023-10-06T02:22:55.031367+0000","last_became_peered":"2023-10-06T02:22:55.031367+0000","last_unstale":"2023-10-06T02:32:29.229325+0000","last_undegraded":"2023-10-06T02:32:29.229325+0000","last_fullsized":"2023-10-06T02:32:29.229325+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:22:53.309097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:22:53.309097+0000","last_clean_scrub_stamp":"2023-10-06T02:22:53.309097+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:53:46.520909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":162,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-06T02:32:31.117514+0000","last_change":"2023-10-06T02:23:01.988140+0000","last_active":"2023-10-06T02:32:31.117514+0000","last_peered":"2023-10-06T02:32:31.117514+0000","last_clean":"2023-10-06T02:32:31.117514+0000","last_became_active":"2023-10-06T02:23:01.987769+0000","last_became_peered":"2023-10-06T02:23:01.987769+0000","last_unstale":"2023-10-06T02:32:31.117514+0000","last_undegraded":"2023-10-06T02:32:31.117514+0000","last_fullsized":"2023-10-06T02:32:31.117514+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:20:08.391912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":148,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-06T02:32:28.708751+0000","last_change":"2023-10-06T02:23:02.054338+0000","last_active":"2023-10-06T02:32:28.708751+0000","last_peered":"2023-10-06T02:32:28.708751+0000","last_clean":"2023-10-06T02:32:28.708751+0000","last_became_active":"2023-10-06T02:23:02.053991+0000","last_became_peered":"2023-10-06T02:23:02.053991+0000","last_unstale":"2023-10-06T02:32:28.708751+0000","last_undegraded":"2023-10-06T02:32:28.708751+0000","last_fullsized":"2023-10-06T02:32:28.708751+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:39:30.192838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":162,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-06T02:32:31.115164+0000","last_change":"2023-10-06T02:23:01.988631+0000","last_active":"2023-10-06T02:32:31.115164+0000","last_peered":"2023-10-06T02:32:31.115164+0000","last_clean":"2023-10-06T02:32:31.115164+0000","last_became_active":"2023-10-06T02:23:01.988303+0000","last_became_peered":"2023-10-06T02:23:01.988303+0000","last_unstale":"2023-10-06T02:32:31.115164+0000","last_undegraded":"2023-10-06T02:32:31.115164+0000","last_fullsized":"2023-10-06T02:32:31.115164+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:38.211466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":138,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-06T02:32:28.707568+0000","last_change":"2023-10-06T02:23:02.054039+0000","last_active":"2023-10-06T02:32:28.707568+0000","last_peered":"2023-10-06T02:32:28.707568+0000","last_clean":"2023-10-06T02:32:28.707568+0000","last_became_active":"2023-10-06T02:23:02.053742+0000","last_became_peered":"2023-10-06T02:23:02.053742+0000","last_unstale":"2023-10-06T02:32:28.707568+0000","last_undegraded":"2023-10-06T02:32:28.707568+0000","last_fullsized":"2023-10-06T02:32:28.707568+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:58:18.846475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574969,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304092,"kb_used_data":2708,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067748,"statfs":{"total":96636764160,"available":96325373952,"internally_reserved":0,"allocated":2772992,"data_stored":1606765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:32:15 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.68200000000000005,"15min":0.65300000000000002},"min":{"1min":0.38400000000000001,"5min":0.318,"15min":0.318},"max":{"1min":7.9160000000000004,"5min":7.9160000000000004,"15min":7.9160000000000004},"last":0.495},{"interface":"front","average":{"1min":1.1759999999999999,"5min":0.73099999999999998,"15min":0.65100000000000002},"min":{"1min":0.47199999999999998,"5min":0.379,"15min":0.35499999999999998},"max":{"1min":11.643000000000001,"5min":11.643000000000001,"15min":11.643000000000001},"last":0.69999999999999996}]},{"osd":1,"last update":"Fri Oct 6 02:32:15 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.72099999999999997,"15min":0.70099999999999996},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.40300000000000002},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":1.069},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.441,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":3.0219999999999998,"5min":3.0219999999999998,"15min":3.0219999999999998},"last":0.629}]},{"osd":2,"last update":"Fri Oct 6 02:32:15 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.67500000000000004,"15min":0.68799999999999994},"min":{"1min":0.435,"5min":0.435,"15min":0.375},"max":{"1min":3.0070000000000001,"5min":3.0070000000000001,"15min":3.0070000000000001},"last":0.61499999999999999},{"interface":"front","average":{"1min":1.2310000000000001,"5min":0.80300000000000005,"15min":0.72699999999999998},"min":{"1min":0.54100000000000004,"5min":0.46999999999999997,"15min":0.39800000000000002},"max":{"1min":12.118,"5min":12.118,"15min":12.118},"last":1.0149999999999999}]},{"osd":3,"last update":"Fri Oct 6 02:32:15 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.71799999999999997,"15min":0.72999999999999998},"min":{"1min":0.46100000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":3.073,"5min":3.073,"15min":3.073},"last":1.472},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.78500000000000003,"15min":0.78000000000000003},"min":{"1min":0.55600000000000005,"5min":0.54500000000000004,"15min":0.497},"max":{"1min":3.105,"5min":3.105,"15min":3.105},"last":1.3300000000000001}]},{"osd":4,"last update":"Fri Oct 6 02:32:15 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.61599999999999999,"15min":0.66100000000000003},"min":{"1min":0.42699999999999999,"5min":0.41799999999999998,"15min":0.35099999999999998},"max":{"1min":0.879,"5min":1.1120000000000001,"15min":1.2849999999999999},"last":1.179},{"interface":"front","average":{"1min":1.0189999999999999,"5min":0.745,"15min":0.754},"min":{"1min":0.42499999999999999,"5min":0.40899999999999997,"15min":0.32900000000000001},"max":{"1min":8.0060000000000002,"5min":8.0060000000000002,"15min":8.0060000000000002},"last":1.5349999999999999}]},{"osd":5,"last update":"Fri Oct 6 02:32:15 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.73799999999999999,"15min":0.73299999999999998},"min":{"1min":0.46200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":3.524,"5min":3.524,"15min":3.524},"last":1.2609999999999999},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.70399999999999996,"15min":0.72099999999999997},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.442},"max":{"1min":3.258,"5min":3.258,"15min":3.258},"last":0.98199999999999998}]},{"osd":6,"last update":"Fri Oct 6 02:32:15 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.79200000000000004,"15min":0.78600000000000003},"min":{"1min":0.54300000000000004,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":3.4689999999999999,"5min":3.4689999999999999,"15min":3.4689999999999999},"last":1.117},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.80000000000000004,"15min":0.80900000000000005},"min":{"1min":0.51500000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":3.4279999999999999,"5min":3.4279999999999999,"15min":3.4279999999999999},"last":0.67100000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574969,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303512,"kb_used_data":2128,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068328,"statfs":{"total":96636764160,"available":96325967872,"internally_reserved":0,"allocated":2179072,"data_stored":1016397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:32:10 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.67600000000000005,"15min":0.66900000000000004},"min":{"1min":0.41599999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.869,"5min":1.869,"15min":1.869},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.71799999999999997,"15min":0.70699999999999996},"min":{"1min":0.49199999999999999,"5min":0.374,"15min":0.374},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.627}]},{"osd":1,"last update":"Fri Oct 6 02:32:10 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66700000000000004,"15min":0.65100000000000002},"min":{"1min":0.47299999999999998,"5min":0.45100000000000001,"15min":0.40699999999999997},"max":{"1min":0.91600000000000004,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.67400000000000004,"15min":0.67000000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.927,"5min":1.927,"15min":1.927},"last":0.59899999999999998}]},{"osd":2,"last update":"Fri Oct 6 02:32:10 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.72999999999999998,"15min":0.71299999999999997},"min":{"1min":0.58199999999999996,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.72499999999999998,"15min":0.71799999999999997},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.40300000000000002},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.71599999999999997}]},{"osd":3,"last update":"Fri Oct 6 02:32:10 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.70599999999999996,"15min":0.67900000000000005},"min":{"1min":0.48899999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.71899999999999997,"15min":0.70999999999999996},"min":{"1min":0.58999999999999997,"5min":0.48799999999999999,"15min":0.45900000000000002},"max":{"1min":0.96299999999999997,"5min":0.97999999999999998,"15min":0.98499999999999999},"last":0.70499999999999996}]},{"osd":4,"last update":"Fri Oct 6 02:32:10 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.68200000000000005,"15min":0.65100000000000002},"min":{"1min":0.379,"5min":0.377,"15min":0.34300000000000003},"max":{"1min":1.97,"5min":1.97,"15min":1.97},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.73199999999999998,"15min":0.68400000000000005},"min":{"1min":0.438,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":2.7429999999999999,"5min":2.7429999999999999,"15min":2.7429999999999999},"last":0.79800000000000004}]},{"osd":5,"last update":"Fri Oct 6 02:32:10 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.71699999999999997,"15min":0.70099999999999996},"min":{"1min":0.47399999999999998,"5min":0.42699999999999999,"15min":0.39700000000000002},"max":{"1min":1.091,"5min":1.091,"15min":1.2929999999999999},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.747,"15min":0.71799999999999997},"min":{"1min":0.496,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":2.484,"5min":2.484,"15min":2.484},"last":0.72999999999999998}]},{"osd":7,"last update":"Fri Oct 6 02:32:10 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.78600000000000003,"15min":0.75900000000000001},"min":{"1min":0.498,"5min":0.42399999999999999,"15min":0.36799999999999999},"max":{"1min":2.7909999999999999,"5min":2.7909999999999999,"15min":2.7909999999999999},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.78800000000000003,"15min":0.76900000000000002},"min":{"1min":0.55100000000000005,"5min":0.46700000000000003,"15min":0.45900000000000002},"max":{"1min":1.0269999999999999,"5min":1.103,"15min":1.2470000000000001},"last":0.752}]}]},{"osd":1,"up_from":13,"seq":55834574968,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303512,"kb_used_data":2128,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068328,"statfs":{"total":96636764160,"available":96325967872,"internally_reserved":0,"allocated":2179072,"data_stored":1016397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.443},"min":{"1min":0.30399999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.58799999999999997,"5min":0.63600000000000001,"15min":1.2230000000000001},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.45800000000000002,"15min":0.501},"min":{"1min":0.34499999999999997,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.66200000000000003,"5min":0.70099999999999996,"15min":1.788},"last":0.46999999999999997}]},{"osd":2,"last update":"Fri Oct 6 02:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.433,"15min":0.49099999999999999},"min":{"1min":0.33500000000000002,"5min":0.307,"15min":0.28299999999999997},"max":{"1min":0.57799999999999996,"5min":0.61799999999999999,"15min":2.0840000000000001},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.503,"5min":0.495,"15min":0.53500000000000003},"min":{"1min":0.374,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.66800000000000004,"5min":0.88100000000000001,"15min":1.6699999999999999},"last":0.52800000000000002}]},{"osd":3,"last update":"Fri Oct 6 02:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.57999999999999996,"15min":0.60399999999999998},"min":{"1min":0.51200000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.79700000000000004,"5min":0.89900000000000002,"15min":1.266},"last":0.498},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.54000000000000004,"15min":0.60299999999999998},"min":{"1min":0.436,"5min":0.40999999999999998,"15min":0.34100000000000003},"max":{"1min":0.70399999999999996,"5min":0.79400000000000004,"15min":1.861},"last":0.42699999999999999}]},{"osd":4,"last update":"Fri Oct 6 02:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.752,"15min":0.82199999999999995},"min":{"1min":0.629,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.97999999999999998,"5min":1.3580000000000001,"15min":1.9279999999999999},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.747,"15min":0.82199999999999995},"min":{"1min":0.53900000000000003,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.95999999999999996,"5min":1.4339999999999999,"15min":2.0179999999999998},"last":0.80500000000000005}]},{"osd":5,"last update":"Fri Oct 6 02:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.71499999999999997,"15min":0.80900000000000005},"min":{"1min":0.57199999999999995,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.91700000000000004,"5min":1.3069999999999999,"15min":1.9830000000000001},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.73199999999999998,"15min":0.79100000000000004},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.47299999999999998},"max":{"1min":0.97299999999999998,"5min":1.327,"15min":1.3380000000000001},"last":0.67400000000000004}]},{"osd":6,"last update":"Fri Oct 6 02:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.76100000000000001,"15min":0.83299999999999996},"min":{"1min":0.62,"5min":0.52100000000000002,"15min":0.45600000000000002},"max":{"1min":1.038,"5min":1.0609999999999999,"15min":1.829},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.77500000000000002,"15min":0.86499999999999999},"min":{"1min":0.60199999999999998,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.96399999999999997,"5min":1.351,"15min":2.052},"last":0.65200000000000002}]},{"osd":7,"last update":"Fri Oct 6 02:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.73399999999999999,"15min":0.80300000000000005},"min":{"1min":0.59399999999999997,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.97299999999999998,"5min":1.3120000000000001,"15min":1.3340000000000001},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.76200000000000001,"15min":0.83799999999999997},"min":{"1min":0.54600000000000004,"5min":0.51600000000000001,"15min":0.47799999999999998},"max":{"1min":1.0780000000000001,"5min":1.2110000000000001,"15min":1.895},"last":0.69999999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574969,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304092,"kb_used_data":2708,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067748,"statfs":{"total":96636764160,"available":96325373952,"internally_reserved":0,"allocated":2772992,"data_stored":1606765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 02:32:19 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.44500000000000001,"15min":0.45000000000000001},"min":{"1min":0.27800000000000002,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.59299999999999997,"5min":2.6120000000000001,"15min":2.6120000000000001},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.47399999999999998,"15min":0.47799999999999998},"min":{"1min":0.33000000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.57099999999999995,"5min":3.4199999999999999,"15min":3.4199999999999999},"last":0.437}]},{"osd":2,"last update":"Fri Oct 6 02:32:19 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.52400000000000002,"15min":0.51400000000000001},"min":{"1min":0.34999999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.61699999999999999,"5min":3.0339999999999998,"15min":3.0339999999999998},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.51500000000000001,"15min":0.51800000000000002},"min":{"1min":0.377,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.76800000000000002,"5min":3.343,"15min":3.343},"last":0.53400000000000003}]},{"osd":3,"last update":"Fri Oct 6 02:32:19 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.52000000000000002,"15min":0.50800000000000001},"min":{"1min":0.41599999999999998,"5min":0.31,"15min":0.31},"max":{"1min":1.0549999999999999,"5min":3.6579999999999999,"15min":3.6579999999999999},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.62,"5min":0.59499999999999997,"15min":0.59599999999999997},"min":{"1min":0.47699999999999998,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.996,"5min":2.6890000000000001,"15min":2.6890000000000001},"last":0.59699999999999998}]},{"osd":4,"last update":"Fri Oct 6 02:32:19 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.77100000000000002,"15min":0.78700000000000003},"min":{"1min":0.55500000000000005,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0649999999999999,"5min":3.2999999999999998,"15min":3.2999999999999998},"last":0.98899999999999999},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.753,"15min":0.78200000000000003},"min":{"1min":0.53900000000000003,"5min":0.51000000000000001,"15min":0.48499999999999999},"max":{"1min":0.995,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":0.83899999999999997}]},{"osd":5,"last update":"Fri Oct 6 02:32:19 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.752,"15min":0.77200000000000002},"min":{"1min":0.499,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.1699999999999999,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.78600000000000003,"15min":0.79500000000000004},"min":{"1min":0.58099999999999996,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.006,"5min":3.1110000000000002,"15min":3.1110000000000002},"last":0.82899999999999996}]},{"osd":6,"last update":"Fri Oct 6 02:32:19 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.78800000000000003,"15min":0.81100000000000005},"min":{"1min":0.52800000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.0780000000000001,"5min":3.1600000000000001,"15min":3.1600000000000001},"last":0.871},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76600000000000001,"15min":0.80700000000000005},"min":{"1min":0.59999999999999998,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.1100000000000001,"5min":2.3250000000000002,"15min":2.3250000000000002},"last":0.96199999999999997}]},{"osd":7,"last update":"Fri Oct 6 02:32:19 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.81999999999999995,"15min":0.82199999999999995},"min":{"1min":0.58499999999999996,"5min":0.57999999999999996,"15min":0.56000000000000005},"max":{"1min":1.097,"5min":3.3809999999999998,"15min":3.3809999999999998},"last":1.0109999999999999},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77900000000000003,"15min":0.80700000000000005},"min":{"1min":0.59399999999999997,"5min":0.54700000000000004,"15min":0.53300000000000003},"max":{"1min":1.018,"5min":2.3740000000000001,"15min":2.3740000000000001},"last":0.85899999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574970,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303512,"kb_used_data":2128,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068328,"statfs":{"total":96636764160,"available":96325967872,"internally_reserved":0,"allocated":2179072,"data_stored":1016397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:32:18 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.42399999999999999,"15min":0.434},"min":{"1min":0.28199999999999997,"5min":0.245,"15min":0.245},"max":{"1min":0.59399999999999997,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.44},{"interface":"front","average":{"1min":0.495,"5min":0.46600000000000003,"15min":0.47999999999999998},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":1.0800000000000001},"last":0.497}]},{"osd":1,"last update":"Fri Oct 6 02:32:18 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.47599999999999998,"15min":0.495},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32400000000000001},"max":{"1min":0.754,"5min":1.036,"15min":1.036},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51000000000000001,"15min":0.57099999999999995},"min":{"1min":0.40899999999999997,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.68300000000000005,"5min":1.778,"15min":1.778},"last":0.38400000000000001}]},{"osd":3,"last update":"Fri Oct 6 02:32:18 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.499,"15min":0.52100000000000002},"min":{"1min":0.36199999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.97499999999999998},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.57999999999999996,"15min":0.60699999999999998},"min":{"1min":0.46800000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.91700000000000004,"5min":0.95799999999999996,"15min":1.028},"last":0.57099999999999995}]},{"osd":4,"last update":"Fri Oct 6 02:32:18 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.73899999999999999,"15min":0.76400000000000001},"min":{"1min":0.51600000000000001,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.875,"5min":1.0940000000000001,"15min":1.1319999999999999},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.747,"15min":0.81399999999999995},"min":{"1min":0.497,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.0309999999999999,"5min":1.907,"15min":1.907},"last":0.78000000000000003}]},{"osd":5,"last update":"Fri Oct 6 02:32:18 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.72199999999999998,"15min":0.77400000000000002},"min":{"1min":0.51100000000000001,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.97799999999999998,"5min":1.109,"15min":1.1850000000000001},"last":0.97799999999999998},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.751,"15min":0.79800000000000004},"min":{"1min":0.59199999999999997,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.006,"5min":1.0609999999999999,"15min":1.258},"last":0.85299999999999998}]},{"osd":6,"last update":"Fri Oct 6 02:32:18 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77900000000000003,"15min":0.82399999999999995},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.089,"5min":1.827,"15min":1.827},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.76200000000000001,"15min":0.78500000000000003},"min":{"1min":0.52600000000000002,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0469999999999999,"5min":1.0660000000000001,"15min":1.1659999999999999},"last":0.93899999999999995}]},{"osd":7,"last update":"Fri Oct 6 02:32:18 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.78400000000000003,"15min":0.84899999999999998},"min":{"1min":0.61199999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.139,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.77400000000000002,"15min":0.81200000000000006},"min":{"1min":0.57399999999999995,"5min":0.55300000000000005,"15min":0.54800000000000004},"max":{"1min":1.1020000000000001,"5min":1.125,"15min":1.1479999999999999},"last":0.81599999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574967,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303512,"kb_used_data":2128,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068328,"statfs":{"total":96636764160,"available":96325967872,"internally_reserved":0,"allocated":2179072,"data_stored":1016397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:32:15 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.42699999999999999,"15min":0.48199999999999998},"min":{"1min":0.30299999999999999,"5min":0.248,"15min":0.248},"max":{"1min":0.66500000000000004,"5min":1.0489999999999999,"15min":2.024},"last":0.307},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.45000000000000001,"15min":0.5},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.56000000000000005,"5min":0.754,"15min":2.181},"last":0.42899999999999999}]},{"osd":1,"last update":"Fri Oct 6 02:32:15 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.45000000000000001,"15min":0.59899999999999998},"min":{"1min":0.34999999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.65000000000000002,"5min":0.66100000000000003,"15min":4.492},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51000000000000001,"15min":0.64800000000000002},"min":{"1min":0.40999999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.876,"5min":1.079,"15min":4.569},"last":0.39600000000000002}]},{"osd":2,"last update":"Fri Oct 6 02:32:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.58199999999999996,"15min":0.64000000000000001},"min":{"1min":0.435,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.69499999999999995,"5min":1.024,"15min":2.0870000000000002},"last":0.501},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.55200000000000005,"15min":0.61799999999999999},"min":{"1min":0.441,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.77400000000000002,"5min":0.92100000000000004,"15min":2.4260000000000002},"last":0.60499999999999998}]},{"osd":4,"last update":"Fri Oct 6 02:32:15 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.72999999999999998,"15min":0.79400000000000004},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.44800000000000001},"max":{"1min":1.024,"5min":1.024,"15min":2.214},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.751,"15min":0.79000000000000004},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.97999999999999998,"5min":1.042,"15min":1.2869999999999999},"last":0.72799999999999998}]},{"osd":5,"last update":"Fri Oct 6 02:32:15 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.745,"15min":0.79700000000000004},"min":{"1min":0.58499999999999996,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.05,"5min":1.077,"15min":1.3029999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.75,"15min":0.81799999999999995},"min":{"1min":0.55500000000000005,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.097,"5min":1.097,"15min":2.242},"last":0.61599999999999999}]},{"osd":6,"last update":"Fri Oct 6 02:32:15 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.76700000000000002,"15min":0.80000000000000004},"min":{"1min":0.52300000000000002,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.073,"5min":1.073,"15min":1.357},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.78600000000000003,"15min":0.81799999999999995},"min":{"1min":0.61199999999999999,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.95299999999999996,"5min":1.016,"15min":1.3100000000000001},"last":0.63700000000000001}]},{"osd":7,"last update":"Fri Oct 6 02:32:15 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.82399999999999995},"min":{"1min":0.58099999999999996,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.996,"5min":1.004,"15min":2.27},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.78500000000000003,"15min":0.83199999999999996},"min":{"1min":0.61699999999999999,"5min":0.55800000000000005,"15min":0.53200000000000003},"max":{"1min":1.0009999999999999,"5min":1.0820000000000001,"15min":1.325},"last":0.68000000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574969,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":307608,"kb_used_data":2128,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064232,"statfs":{"total":96636764160,"available":96321773568,"internally_reserved":0,"allocated":2179072,"data_stored":1016397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:32:10 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.65100000000000002,"15min":0.88900000000000001},"min":{"1min":0.45800000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.877,"5min":1.0760000000000001,"15min":12.337},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.66700000000000004,"15min":0.83299999999999996},"min":{"1min":0.45100000000000001,"5min":0.44400000000000001,"15min":0.438},"max":{"1min":0.91900000000000004,"5min":1.3520000000000001,"15min":9.6539999999999999},"last":0.52200000000000002}]},{"osd":1,"last update":"Fri Oct 6 02:32:10 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64500000000000002,"15min":0.68400000000000005},"min":{"1min":0.47499999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.83699999999999997,"5min":1.129,"15min":3.181},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.70399999999999996,"15min":0.91800000000000004},"min":{"1min":0.48299999999999998,"5min":0.45300000000000001,"15min":0.40500000000000003},"max":{"1min":0.86799999999999999,"5min":0.995,"15min":12.946999999999999},"last":0.59099999999999997}]},{"osd":2,"last update":"Fri Oct 6 02:32:10 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.69499999999999995,"15min":0.93300000000000005},"min":{"1min":0.47299999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.84499999999999997,"5min":1.2989999999999999,"15min":12.377000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.73099999999999998,"15min":0.82499999999999996},"min":{"1min":0.53700000000000003,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.91200000000000003,"5min":1.0529999999999999,"15min":6.3220000000000001},"last":0.79800000000000004}]},{"osd":3,"last update":"Fri Oct 6 02:32:10 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.72899999999999998,"15min":0.90400000000000003},"min":{"1min":0.57099999999999995,"5min":0.46200000000000002,"15min":0.42699999999999999},"max":{"1min":0.84199999999999997,"5min":1.5229999999999999,"15min":9.7720000000000002},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.70499999999999996,"15min":0.752},"min":{"1min":0.55800000000000005,"5min":0.52400000000000002,"15min":0.47999999999999998},"max":{"1min":0.88900000000000001,"5min":1.212,"15min":3.0840000000000001},"last":0.74299999999999999}]},{"osd":5,"last update":"Fri Oct 6 02:32:10 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72199999999999998,"15min":0.80200000000000005},"min":{"1min":0.46100000000000002,"5min":0.40400000000000003,"15min":0.35299999999999998},"max":{"1min":0.92200000000000004,"5min":1.02,"15min":6.2690000000000001},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.75600000000000001,"15min":0.93899999999999995},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.33600000000000002},"max":{"1min":0.98199999999999998,"5min":1.4359999999999999,"15min":10.769},"last":0.77500000000000002}]},{"osd":6,"last update":"Fri Oct 6 02:32:10 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71799999999999997,"15min":0.74299999999999999},"min":{"1min":0.46300000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.95299999999999996,"5min":1.4590000000000001,"15min":3.1349999999999998},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.80100000000000005,"15min":0.91100000000000003},"min":{"1min":0.52600000000000002,"5min":0.47799999999999998,"15min":0.45700000000000002},"max":{"1min":0.96799999999999997,"5min":1.3839999999999999,"15min":8.0899999999999999},"last":0.76200000000000001}]},{"osd":7,"last update":"Fri Oct 6 02:32:10 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.80300000000000005,"15min":0.97899999999999998},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.47599999999999998},"max":{"1min":1.02,"5min":1.494,"15min":9.7089999999999996},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.78700000000000003,"15min":0.92800000000000005},"min":{"1min":0.56000000000000005,"5min":0.44400000000000001,"15min":0.374},"max":{"1min":1.034,"5min":1.2669999999999999,"15min":7.6719999999999997},"last":0.79000000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574969,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303512,"kb_used_data":2128,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068328,"statfs":{"total":96636764160,"available":96325967872,"internally_reserved":0,"allocated":2179072,"data_stored":1016397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:32:07 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.63200000000000001,"15min":0.63},"min":{"1min":0.47099999999999997,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.93899999999999995,"5min":0.97599999999999998,"15min":1.4279999999999999},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.629,"15min":0.622},"min":{"1min":0.42599999999999999,"5min":0.40300000000000002,"15min":0.38700000000000001},"max":{"1min":0.89300000000000002,"5min":1.099,"15min":1.099},"last":0.79600000000000004}]},{"osd":1,"last update":"Fri Oct 6 02:32:07 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.63600000000000001,"15min":0.64900000000000002},"min":{"1min":0.51700000000000002,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.93899999999999995,"5min":1.0660000000000001,"15min":1.089},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.67300000000000004,"15min":0.67700000000000005},"min":{"1min":0.42299999999999999,"5min":0.40699999999999997,"15min":0.40500000000000003},"max":{"1min":0.93899999999999995,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.78000000000000003}]},{"osd":2,"last update":"Fri Oct 6 02:32:07 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.67800000000000005,"15min":0.67600000000000005},"min":{"1min":0.45800000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.67200000000000004,"15min":0.68200000000000005},"min":{"1min":0.58599999999999997,"5min":0.46100000000000002,"15min":0.44600000000000001},"max":{"1min":1.028,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.75600000000000001}]},{"osd":3,"last update":"Fri Oct 6 02:32:07 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.73299999999999998,"15min":0.73199999999999998},"min":{"1min":0.54000000000000004,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0009999999999999,"5min":1.159,"15min":1.159},"last":0.433},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.77200000000000002,"15min":0.77000000000000002},"min":{"1min":0.64200000000000002,"5min":0.56200000000000006,"15min":0.54100000000000004},"max":{"1min":0.97699999999999998,"5min":1.194,"15min":1.194},"last":0.81000000000000005}]},{"osd":4,"last update":"Fri Oct 6 02:32:07 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.69099999999999995,"15min":0.67700000000000005},"min":{"1min":0.46999999999999997,"5min":0.40799999999999997,"15min":0.32000000000000001},"max":{"1min":0.92300000000000004,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.62},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.69099999999999995,"15min":0.68300000000000005},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.35399999999999998},"max":{"1min":0.88600000000000001,"5min":1.25,"15min":1.25},"last":0.63800000000000001}]},{"osd":6,"last update":"Fri Oct 6 02:32:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.71899999999999997,"15min":0.73499999999999999},"min":{"1min":0.45900000000000002,"5min":0.42899999999999999,"15min":0.40300000000000002},"max":{"1min":1.024,"5min":1.024,"15min":1.1719999999999999},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.81499999999999995,"15min":0.81799999999999995},"min":{"1min":0.60599999999999998,"5min":0.502,"15min":0.373},"max":{"1min":1.0700000000000001,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.82799999999999996}]},{"osd":7,"last update":"Fri Oct 6 02:32:07 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.754,"15min":0.76000000000000001},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.40799999999999997},"max":{"1min":0.95799999999999996,"5min":1.23,"15min":1.23},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.80200000000000005,"15min":0.82099999999999995},"min":{"1min":0.59599999999999997,"5min":0.47599999999999998,"15min":0.437},"max":{"1min":1.1180000000000001,"5min":1.1339999999999999,"15min":1.5640000000000001},"last":0.85499999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T02:32:36.051 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T02:32:36.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.237+0000 7fae8339d700 1 Processor -- start 2023-10-06T02:32:36.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.237+0000 7fae8339d700 1 -- start start 2023-10-06T02:32:36.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.238+0000 7fae8339d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fae7c15dfd0 0x7fae7c0aedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:36.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.238+0000 7fae8339d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae7c0acc00 con 0x7fae7c159550 2023-10-06T02:32:36.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.238+0000 7fae8339d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae7c0abff0 con 0x7fae7c159920 2023-10-06T02:32:36.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.238+0000 7fae8339d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae7c0ac2d0 con 0x7fae7c15dfd0 2023-10-06T02:32:36.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.239+0000 7fae8239b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fae7c159550 0x7fae7c0ceac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:43866/0 (socket says 172.21.15.31:43866) 2023-10-06T02:32:36.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.239+0000 7fae81b9a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fae7c15dfd0 0x7fae7c0aedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:36.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.239+0000 7fae8239b700 1 -- 172.21.15.31:0/2788291667 learned_addr learned my addr 172.21.15.31:0/2788291667 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:32:36.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.239+0000 7fae81399700 1 -- 172.21.15.31:0/2788291667 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2402535445 0 0) 0x7fae7c0acc00 con 0x7fae7c159550 2023-10-06T02:32:36.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.239+0000 7fae81399700 1 -- 172.21.15.31:0/2788291667 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae68003570 con 0x7fae7c159550 2023-10-06T02:32:36.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.239+0000 7fae81b9a700 1 -- 172.21.15.31:0/2788291667 >> v1:172.21.15.62:6789/0 conn(0x7fae7c159920 legacy=0x7fae7c0ad090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:36.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.239+0000 7fae81b9a700 1 -- 172.21.15.31:0/2788291667 >> v1:172.21.15.31:6789/0 conn(0x7fae7c159550 legacy=0x7fae7c0ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:36.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.239+0000 7fae81b9a700 1 -- 172.21.15.31:0/2788291667 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae7c0ac5b0 con 0x7fae7c15dfd0 2023-10-06T02:32:36.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.239+0000 7fae81b9a700 1 --2- 172.21.15.31:0/2788291667 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae7c15dfd0 0x7fae7c0aedb0 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7fae6c0098e0 tx=0x7fae6c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=682df79222faf94f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:36.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.240+0000 7fae81399700 1 -- 172.21.15.31:0/2788291667 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae6c00b640 con 0x7fae7c15dfd0 2023-10-06T02:32:36.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.240+0000 7fae81399700 1 -- 172.21.15.31:0/2788291667 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae6c00b7a0 con 0x7fae7c15dfd0 2023-10-06T02:32:36.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.240+0000 7fae81399700 1 -- 172.21.15.31:0/2788291667 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae6c00bb40 con 0x7fae7c15dfd0 2023-10-06T02:32:36.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.240+0000 7fae8339d700 1 -- 172.21.15.31:0/2788291667 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae7c15dfd0 msgr2=0x7fae7c0aedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:36.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.240+0000 7fae8339d700 1 --2- 172.21.15.31:0/2788291667 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae7c15dfd0 0x7fae7c0aedb0 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7fae6c0098e0 tx=0x7fae6c0091a0 comp rx=0 tx=0).stop 2023-10-06T02:32:36.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.240+0000 7fae8339d700 1 -- 172.21.15.31:0/2788291667 shutdown_connections 2023-10-06T02:32:36.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.240+0000 7fae8339d700 1 --2- 172.21.15.31:0/2788291667 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae7c15dfd0 0x7fae7c0aedb0 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:36.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.240+0000 7fae8339d700 1 -- 172.21.15.31:0/2788291667 >> 172.21.15.31:0/2788291667 conn(0x7fae7c06ce80 msgr2=0x7fae7c0ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:36.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.240+0000 7fae8339d700 1 -- 172.21.15.31:0/2788291667 shutdown_connections 2023-10-06T02:32:36.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.240+0000 7fae8339d700 1 -- 172.21.15.31:0/2788291667 wait complete. 2023-10-06T02:32:36.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.242+0000 7fae8339d700 1 Processor -- start 2023-10-06T02:32:36.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.242+0000 7fae8339d700 1 -- start start 2023-10-06T02:32:36.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.243+0000 7fae8339d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae7c159550 0x7fae7c0b2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:36.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.243+0000 7fae8339d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae7c159920 0x7fae7c14b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:36.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.243+0000 7fae8339d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fae7c15dfd0 0x7fae7c13cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:36.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.243+0000 7fae8339d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae7c0cddf0 con 0x7fae7c159550 2023-10-06T02:32:36.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.243+0000 7fae8339d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae7c0b4190 con 0x7fae7c159920 2023-10-06T02:32:36.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.243+0000 7fae8339d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae7c0b3580 con 0x7fae7c15dfd0 2023-10-06T02:32:36.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.243+0000 7fae8239b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae7c159550 0x7fae7c0b2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:36.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.243+0000 7fae8239b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae7c159550 0x7fae7c0b2ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58408/0 (socket says 172.21.15.31:58408) 2023-10-06T02:32:36.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.243+0000 7fae8239b700 1 -- 172.21.15.31:0/799763113 learned_addr learned my addr 172.21.15.31:0/799763113 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:36.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.243+0000 7fae81b9a700 1 --2- 172.21.15.31:0/799763113 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae7c159920 0x7fae7c14b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:36.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.243+0000 7fae82b9c700 1 --2- 172.21.15.31:0/799763113 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fae7c15dfd0 0x7fae7c13cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:36.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.244+0000 7fae8239b700 1 -- 172.21.15.31:0/799763113 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fae7c15dfd0 msgr2=0x7fae7c13cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:36.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.244+0000 7fae8239b700 1 --2- 172.21.15.31:0/799763113 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fae7c15dfd0 0x7fae7c13cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:36.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.244+0000 7fae8239b700 1 -- 172.21.15.31:0/799763113 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae7c159920 msgr2=0x7fae7c14b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:36.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.244+0000 7fae8239b700 1 --2- 172.21.15.31:0/799763113 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae7c159920 0x7fae7c14b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:36.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.244+0000 7fae8239b700 1 -- 172.21.15.31:0/799763113 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae6c009420 con 0x7fae7c159550 2023-10-06T02:32:36.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.244+0000 7fae8239b700 1 --2- 172.21.15.31:0/799763113 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae7c159550 0x7fae7c0b2ec0 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7fae7800c7c0 tx=0x7fae78007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:36.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.244+0000 7fae727fc700 1 -- 172.21.15.31:0/799763113 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae78005b30 con 0x7fae7c159550 2023-10-06T02:32:36.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.244+0000 7fae727fc700 1 -- 172.21.15.31:0/799763113 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae78005c90 con 0x7fae7c159550 2023-10-06T02:32:36.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.244+0000 7fae8339d700 1 -- 172.21.15.31:0/799763113 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae7c0b3860 con 0x7fae7c159550 2023-10-06T02:32:36.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.244+0000 7fae727fc700 1 -- 172.21.15.31:0/799763113 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae78007340 con 0x7fae7c159550 2023-10-06T02:32:36.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.244+0000 7fae8339d700 1 -- 172.21.15.31:0/799763113 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae7c0b3e20 con 0x7fae7c159550 2023-10-06T02:32:36.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.245+0000 7fae8339d700 1 -- 172.21.15.31:0/799763113 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae7c04c3c0 con 0x7fae7c159550 2023-10-06T02:32:36.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.246+0000 7fae727fc700 1 -- 172.21.15.31:0/799763113 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) v1 ==== 85709+0+0 (secure 0 0 0) 0x7fae7801b400 con 0x7fae7c159550 2023-10-06T02:32:36.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.247+0000 7fae727fc700 1 --2- 172.21.15.31:0/799763113 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fae600679e0 0x7fae60069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:36.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.247+0000 7fae727fc700 1 -- 172.21.15.31:0/799763113 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5320+0+0 (secure 0 0 0) 0x7fae78082a30 con 0x7fae7c159550 2023-10-06T02:32:36.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.248+0000 7fae81b9a700 1 --2- 172.21.15.31:0/799763113 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fae600679e0 0x7fae60069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:36.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.248+0000 7fae81b9a700 1 --2- 172.21.15.31:0/799763113 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fae600679e0 0x7fae60069e90 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fae6c0093f0 tx=0x7fae6c017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:36.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.251+0000 7fae727fc700 1 -- 172.21.15.31:0/799763113 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fae780516f0 con 0x7fae7c159550 2023-10-06T02:32:36.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.451+0000 7fae8339d700 1 -- 172.21.15.31:0/799763113 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fae7c0aed70 con 0x7fae600679e0 2023-10-06T02:32:36.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.455+0000 7fae727fc700 1 -- 172.21.15.31:0/799763113 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72379 (secure 0 0 0) 0x7fae7c0aed70 con 0x7fae600679e0 2023-10-06T02:32:36.457 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:32:36.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.457+0000 7fae8339d700 1 -- 172.21.15.31:0/799763113 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fae600679e0 msgr2=0x7fae60069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:36.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.457+0000 7fae8339d700 1 --2- 172.21.15.31:0/799763113 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fae600679e0 0x7fae60069e90 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fae6c0093f0 tx=0x7fae6c017000 comp rx=0 tx=0).stop 2023-10-06T02:32:36.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.457+0000 7fae8339d700 1 -- 172.21.15.31:0/799763113 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae7c159550 msgr2=0x7fae7c0b2ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:36.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.457+0000 7fae8339d700 1 --2- 172.21.15.31:0/799763113 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae7c159550 0x7fae7c0b2ec0 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7fae7800c7c0 tx=0x7fae78007e00 comp rx=0 tx=0).stop 2023-10-06T02:32:36.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.458+0000 7fae8339d700 1 -- 172.21.15.31:0/799763113 shutdown_connections 2023-10-06T02:32:36.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.458+0000 7fae8339d700 1 --2- 172.21.15.31:0/799763113 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fae600679e0 0x7fae60069e90 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:36.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.458+0000 7fae8339d700 1 --2- 172.21.15.31:0/799763113 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fae7c15dfd0 0x7fae7c13cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:36.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.458+0000 7fae8339d700 1 --2- 172.21.15.31:0/799763113 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae7c159550 0x7fae7c0b2ec0 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:36.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.458+0000 7fae8339d700 1 --2- 172.21.15.31:0/799763113 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae7c159920 0x7fae7c14b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:36.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.458+0000 7fae8339d700 1 -- 172.21.15.31:0/799763113 >> 172.21.15.31:0/799763113 conn(0x7fae7c06ce80 msgr2=0x7fae7c0ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:36.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.458+0000 7fae8339d700 1 -- 172.21.15.31:0/799763113 shutdown_connections 2023-10-06T02:32:36.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.458+0000 7fae8339d700 1 -- 172.21.15.31:0/799763113 wait complete. 2023-10-06T02:32:36.461 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-10-06T02:32:36.477 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":248,"stamp":"2023-10-06T02:32:34.529682+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":113,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":2433352,"kb_used_data":18184,"kb_used_omap":0,"kb_used_meta":2415104,"kb_avail":752541368,"statfs":{"total":773094113280,"available":770602360832,"internally_reserved":0,"allocated":18620416,"data_stored":9311912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2473066496},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-55730,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-3,"num_read_kb":-3,"num_write":-73,"num_write_kb":-82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001569"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":137,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-06T02:32:29.366431+0000","last_change":"2023-10-06T02:23:01.992935+0000","last_active":"2023-10-06T02:32:29.366431+0000","last_peered":"2023-10-06T02:32:29.366431+0000","last_clean":"2023-10-06T02:32:29.366431+0000","last_became_active":"2023-10-06T02:23:01.992568+0000","last_became_peered":"2023-10-06T02:23:01.992568+0000","last_unstale":"2023-10-06T02:32:29.366431+0000","last_undegraded":"2023-10-06T02:32:29.366431+0000","last_fullsized":"2023-10-06T02:32:29.366431+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:06.212251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":138,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-06T02:32:28.707783+0000","last_change":"2023-10-06T02:23:01.989934+0000","last_active":"2023-10-06T02:32:28.707783+0000","last_peered":"2023-10-06T02:32:28.707783+0000","last_clean":"2023-10-06T02:32:28.707783+0000","last_became_active":"2023-10-06T02:23:01.989706+0000","last_became_peered":"2023-10-06T02:23:01.989706+0000","last_unstale":"2023-10-06T02:32:28.707783+0000","last_undegraded":"2023-10-06T02:32:28.707783+0000","last_fullsized":"2023-10-06T02:32:28.707783+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:06.212251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":131,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-06T02:32:29.229695+0000","last_change":"2023-10-06T02:23:02.054069+0000","last_active":"2023-10-06T02:32:29.229695+0000","last_peered":"2023-10-06T02:32:29.229695+0000","last_clean":"2023-10-06T02:32:29.229695+0000","last_became_active":"2023-10-06T02:23:02.053739+0000","last_became_peered":"2023-10-06T02:23:02.053739+0000","last_unstale":"2023-10-06T02:32:29.229695+0000","last_undegraded":"2023-10-06T02:32:29.229695+0000","last_fullsized":"2023-10-06T02:32:29.229695+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:57:48.822608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":131,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-06T02:32:29.229181+0000","last_change":"2023-10-06T02:23:01.990773+0000","last_active":"2023-10-06T02:32:29.229181+0000","last_peered":"2023-10-06T02:32:29.229181+0000","last_clean":"2023-10-06T02:32:29.229181+0000","last_became_active":"2023-10-06T02:23:01.990511+0000","last_became_peered":"2023-10-06T02:23:01.990511+0000","last_unstale":"2023-10-06T02:32:29.229181+0000","last_undegraded":"2023-10-06T02:32:29.229181+0000","last_fullsized":"2023-10-06T02:32:29.229181+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:42:54.797731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"76'128","reported_seq":323,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-06T02:32:29.229325+0000","last_change":"2023-10-06T02:22:55.032519+0000","last_active":"2023-10-06T02:32:29.229325+0000","last_peered":"2023-10-06T02:32:29.229325+0000","last_clean":"2023-10-06T02:32:29.229325+0000","last_became_active":"2023-10-06T02:22:55.031367+0000","last_became_peered":"2023-10-06T02:22:55.031367+0000","last_unstale":"2023-10-06T02:32:29.229325+0000","last_undegraded":"2023-10-06T02:32:29.229325+0000","last_fullsized":"2023-10-06T02:32:29.229325+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:22:53.309097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:22:53.309097+0000","last_clean_scrub_stamp":"2023-10-06T02:22:53.309097+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:53:46.520909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":162,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-06T02:32:31.117514+0000","last_change":"2023-10-06T02:23:01.988140+0000","last_active":"2023-10-06T02:32:31.117514+0000","last_peered":"2023-10-06T02:32:31.117514+0000","last_clean":"2023-10-06T02:32:31.117514+0000","last_became_active":"2023-10-06T02:23:01.987769+0000","last_became_peered":"2023-10-06T02:23:01.987769+0000","last_unstale":"2023-10-06T02:32:31.117514+0000","last_undegraded":"2023-10-06T02:32:31.117514+0000","last_fullsized":"2023-10-06T02:32:31.117514+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:20:08.391912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":148,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-06T02:32:28.708751+0000","last_change":"2023-10-06T02:23:02.054338+0000","last_active":"2023-10-06T02:32:28.708751+0000","last_peered":"2023-10-06T02:32:28.708751+0000","last_clean":"2023-10-06T02:32:28.708751+0000","last_became_active":"2023-10-06T02:23:02.053991+0000","last_became_peered":"2023-10-06T02:23:02.053991+0000","last_unstale":"2023-10-06T02:32:28.708751+0000","last_undegraded":"2023-10-06T02:32:28.708751+0000","last_fullsized":"2023-10-06T02:32:28.708751+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:39:30.192838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":162,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-06T02:32:31.115164+0000","last_change":"2023-10-06T02:23:01.988631+0000","last_active":"2023-10-06T02:32:31.115164+0000","last_peered":"2023-10-06T02:32:31.115164+0000","last_clean":"2023-10-06T02:32:31.115164+0000","last_became_active":"2023-10-06T02:23:01.988303+0000","last_became_peered":"2023-10-06T02:23:01.988303+0000","last_unstale":"2023-10-06T02:32:31.115164+0000","last_undegraded":"2023-10-06T02:32:31.115164+0000","last_fullsized":"2023-10-06T02:32:31.115164+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:38.211466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":138,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-06T02:32:28.707568+0000","last_change":"2023-10-06T02:23:02.054039+0000","last_active":"2023-10-06T02:32:28.707568+0000","last_peered":"2023-10-06T02:32:28.707568+0000","last_clean":"2023-10-06T02:32:28.707568+0000","last_became_active":"2023-10-06T02:23:02.053742+0000","last_became_peered":"2023-10-06T02:23:02.053742+0000","last_unstale":"2023-10-06T02:32:28.707568+0000","last_undegraded":"2023-10-06T02:32:28.707568+0000","last_fullsized":"2023-10-06T02:32:28.707568+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:58:18.846475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574969,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304092,"kb_used_data":2708,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067748,"statfs":{"total":96636764160,"available":96325373952,"internally_reserved":0,"allocated":2772992,"data_stored":1606765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:32:15 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.68200000000000005,"15min":0.65300000000000002},"min":{"1min":0.38400000000000001,"5min":0.318,"15min":0.318},"max":{"1min":7.9160000000000004,"5min":7.9160000000000004,"15min":7.9160000000000004},"last":0.495},{"interface":"front","average":{"1min":1.1759999999999999,"5min":0.73099999999999998,"15min":0.65100000000000002},"min":{"1min":0.47199999999999998,"5min":0.379,"15min":0.35499999999999998},"max":{"1min":11.643000000000001,"5min":11.643000000000001,"15min":11.643000000000001},"last":0.69999999999999996}]},{"osd":1,"last update":"Fri Oct 6 02:32:15 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.72099999999999997,"15min":0.70099999999999996},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.40300000000000002},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":1.069},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.441,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":3.0219999999999998,"5min":3.0219999999999998,"15min":3.0219999999999998},"last":0.629}]},{"osd":2,"last update":"Fri Oct 6 02:32:15 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.67500000000000004,"15min":0.68799999999999994},"min":{"1min":0.435,"5min":0.435,"15min":0.375},"max":{"1min":3.0070000000000001,"5min":3.0070000000000001,"15min":3.0070000000000001},"last":0.61499999999999999},{"interface":"front","average":{"1min":1.2310000000000001,"5min":0.80300000000000005,"15min":0.72699999999999998},"min":{"1min":0.54100000000000004,"5min":0.46999999999999997,"15min":0.39800000000000002},"max":{"1min":12.118,"5min":12.118,"15min":12.118},"last":1.0149999999999999}]},{"osd":3,"last update":"Fri Oct 6 02:32:15 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.71799999999999997,"15min":0.72999999999999998},"min":{"1min":0.46100000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":3.073,"5min":3.073,"15min":3.073},"last":1.472},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.78500000000000003,"15min":0.78000000000000003},"min":{"1min":0.55600000000000005,"5min":0.54500000000000004,"15min":0.497},"max":{"1min":3.105,"5min":3.105,"15min":3.105},"last":1.3300000000000001}]},{"osd":4,"last update":"Fri Oct 6 02:32:15 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.61599999999999999,"15min":0.66100000000000003},"min":{"1min":0.42699999999999999,"5min":0.41799999999999998,"15min":0.35099999999999998},"max":{"1min":0.879,"5min":1.1120000000000001,"15min":1.2849999999999999},"last":1.179},{"interface":"front","average":{"1min":1.0189999999999999,"5min":0.745,"15min":0.754},"min":{"1min":0.42499999999999999,"5min":0.40899999999999997,"15min":0.32900000000000001},"max":{"1min":8.0060000000000002,"5min":8.0060000000000002,"15min":8.0060000000000002},"last":1.5349999999999999}]},{"osd":5,"last update":"Fri Oct 6 02:32:15 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.73799999999999999,"15min":0.73299999999999998},"min":{"1min":0.46200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":3.524,"5min":3.524,"15min":3.524},"last":1.2609999999999999},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.70399999999999996,"15min":0.72099999999999997},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.442},"max":{"1min":3.258,"5min":3.258,"15min":3.258},"last":0.98199999999999998}]},{"osd":6,"last update":"Fri Oct 6 02:32:15 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.79200000000000004,"15min":0.78600000000000003},"min":{"1min":0.54300000000000004,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":3.4689999999999999,"5min":3.4689999999999999,"15min":3.4689999999999999},"last":1.117},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.80000000000000004,"15min":0.80900000000000005},"min":{"1min":0.51500000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":3.4279999999999999,"5min":3.4279999999999999,"15min":3.4279999999999999},"last":0.67100000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574969,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303512,"kb_used_data":2128,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068328,"statfs":{"total":96636764160,"available":96325967872,"internally_reserved":0,"allocated":2179072,"data_stored":1016397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:32:10 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.67600000000000005,"15min":0.66900000000000004},"min":{"1min":0.41599999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.869,"5min":1.869,"15min":1.869},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.71799999999999997,"15min":0.70699999999999996},"min":{"1min":0.49199999999999999,"5min":0.374,"15min":0.374},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.627}]},{"osd":1,"last update":"Fri Oct 6 02:32:10 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66700000000000004,"15min":0.65100000000000002},"min":{"1min":0.47299999999999998,"5min":0.45100000000000001,"15min":0.40699999999999997},"max":{"1min":0.91600000000000004,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.67400000000000004,"15min":0.67000000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.927,"5min":1.927,"15min":1.927},"last":0.59899999999999998}]},{"osd":2,"last update":"Fri Oct 6 02:32:10 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.72999999999999998,"15min":0.71299999999999997},"min":{"1min":0.58199999999999996,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.72499999999999998,"15min":0.71799999999999997},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.40300000000000002},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.71599999999999997}]},{"osd":3,"last update":"Fri Oct 6 02:32:10 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.70599999999999996,"15min":0.67900000000000005},"min":{"1min":0.48899999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.71899999999999997,"15min":0.70999999999999996},"min":{"1min":0.58999999999999997,"5min":0.48799999999999999,"15min":0.45900000000000002},"max":{"1min":0.96299999999999997,"5min":0.97999999999999998,"15min":0.98499999999999999},"last":0.70499999999999996}]},{"osd":4,"last update":"Fri Oct 6 02:32:10 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.68200000000000005,"15min":0.65100000000000002},"min":{"1min":0.379,"5min":0.377,"15min":0.34300000000000003},"max":{"1min":1.97,"5min":1.97,"15min":1.97},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.73199999999999998,"15min":0.68400000000000005},"min":{"1min":0.438,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":2.7429999999999999,"5min":2.7429999999999999,"15min":2.7429999999999999},"last":0.79800000000000004}]},{"osd":5,"last update":"Fri Oct 6 02:32:10 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.71699999999999997,"15min":0.70099999999999996},"min":{"1min":0.47399999999999998,"5min":0.42699999999999999,"15min":0.39700000000000002},"max":{"1min":1.091,"5min":1.091,"15min":1.2929999999999999},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.747,"15min":0.71799999999999997},"min":{"1min":0.496,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":2.484,"5min":2.484,"15min":2.484},"last":0.72999999999999998}]},{"osd":7,"last update":"Fri Oct 6 02:32:10 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.78600000000000003,"15min":0.75900000000000001},"min":{"1min":0.498,"5min":0.42399999999999999,"15min":0.36799999999999999},"max":{"1min":2.7909999999999999,"5min":2.7909999999999999,"15min":2.7909999999999999},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.78800000000000003,"15min":0.76900000000000002},"min":{"1min":0.55100000000000005,"5min":0.46700000000000003,"15min":0.45900000000000002},"max":{"1min":1.0269999999999999,"5min":1.103,"15min":1.2470000000000001},"last":0.752}]}]},{"osd":1,"up_from":13,"seq":55834574968,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303512,"kb_used_data":2128,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068328,"statfs":{"total":96636764160,"available":96325967872,"internally_reserved":0,"allocated":2179072,"data_stored":1016397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.443},"min":{"1min":0.30399999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.58799999999999997,"5min":0.63600000000000001,"15min":1.2230000000000001},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.45800000000000002,"15min":0.501},"min":{"1min":0.34499999999999997,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.66200000000000003,"5min":0.70099999999999996,"15min":1.788},"last":0.46999999999999997}]},{"osd":2,"last update":"Fri Oct 6 02:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.433,"15min":0.49099999999999999},"min":{"1min":0.33500000000000002,"5min":0.307,"15min":0.28299999999999997},"max":{"1min":0.57799999999999996,"5min":0.61799999999999999,"15min":2.0840000000000001},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.503,"5min":0.495,"15min":0.53500000000000003},"min":{"1min":0.374,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.66800000000000004,"5min":0.88100000000000001,"15min":1.6699999999999999},"last":0.52800000000000002}]},{"osd":3,"last update":"Fri Oct 6 02:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.57999999999999996,"15min":0.60399999999999998},"min":{"1min":0.51200000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.79700000000000004,"5min":0.89900000000000002,"15min":1.266},"last":0.498},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.54000000000000004,"15min":0.60299999999999998},"min":{"1min":0.436,"5min":0.40999999999999998,"15min":0.34100000000000003},"max":{"1min":0.70399999999999996,"5min":0.79400000000000004,"15min":1.861},"last":0.42699999999999999}]},{"osd":4,"last update":"Fri Oct 6 02:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.752,"15min":0.82199999999999995},"min":{"1min":0.629,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.97999999999999998,"5min":1.3580000000000001,"15min":1.9279999999999999},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.747,"15min":0.82199999999999995},"min":{"1min":0.53900000000000003,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.95999999999999996,"5min":1.4339999999999999,"15min":2.0179999999999998},"last":0.80500000000000005}]},{"osd":5,"last update":"Fri Oct 6 02:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.71499999999999997,"15min":0.80900000000000005},"min":{"1min":0.57199999999999995,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.91700000000000004,"5min":1.3069999999999999,"15min":1.9830000000000001},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.73199999999999998,"15min":0.79100000000000004},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.47299999999999998},"max":{"1min":0.97299999999999998,"5min":1.327,"15min":1.3380000000000001},"last":0.67400000000000004}]},{"osd":6,"last update":"Fri Oct 6 02:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.76100000000000001,"15min":0.83299999999999996},"min":{"1min":0.62,"5min":0.52100000000000002,"15min":0.45600000000000002},"max":{"1min":1.038,"5min":1.0609999999999999,"15min":1.829},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.77500000000000002,"15min":0.86499999999999999},"min":{"1min":0.60199999999999998,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.96399999999999997,"5min":1.351,"15min":2.052},"last":0.65200000000000002}]},{"osd":7,"last update":"Fri Oct 6 02:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.73399999999999999,"15min":0.80300000000000005},"min":{"1min":0.59399999999999997,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.97299999999999998,"5min":1.3120000000000001,"15min":1.3340000000000001},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.76200000000000001,"15min":0.83799999999999997},"min":{"1min":0.54600000000000004,"5min":0.51600000000000001,"15min":0.47799999999999998},"max":{"1min":1.0780000000000001,"5min":1.2110000000000001,"15min":1.895},"last":0.69999999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574969,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304092,"kb_used_data":2708,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067748,"statfs":{"total":96636764160,"available":96325373952,"internally_reserved":0,"allocated":2772992,"data_stored":1606765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 02:32:19 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.44500000000000001,"15min":0.45000000000000001},"min":{"1min":0.27800000000000002,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.59299999999999997,"5min":2.6120000000000001,"15min":2.6120000000000001},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.47399999999999998,"15min":0.47799999999999998},"min":{"1min":0.33000000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.57099999999999995,"5min":3.4199999999999999,"15min":3.4199999999999999},"last":0.437}]},{"osd":2,"last update":"Fri Oct 6 02:32:19 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.52400000000000002,"15min":0.51400000000000001},"min":{"1min":0.34999999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.61699999999999999,"5min":3.0339999999999998,"15min":3.0339999999999998},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.51500000000000001,"15min":0.51800000000000002},"min":{"1min":0.377,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.76800000000000002,"5min":3.343,"15min":3.343},"last":0.53400000000000003}]},{"osd":3,"last update":"Fri Oct 6 02:32:19 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.52000000000000002,"15min":0.50800000000000001},"min":{"1min":0.41599999999999998,"5min":0.31,"15min":0.31},"max":{"1min":1.0549999999999999,"5min":3.6579999999999999,"15min":3.6579999999999999},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.62,"5min":0.59499999999999997,"15min":0.59599999999999997},"min":{"1min":0.47699999999999998,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.996,"5min":2.6890000000000001,"15min":2.6890000000000001},"last":0.59699999999999998}]},{"osd":4,"last update":"Fri Oct 6 02:32:19 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.77100000000000002,"15min":0.78700000000000003},"min":{"1min":0.55500000000000005,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0649999999999999,"5min":3.2999999999999998,"15min":3.2999999999999998},"last":0.98899999999999999},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.753,"15min":0.78200000000000003},"min":{"1min":0.53900000000000003,"5min":0.51000000000000001,"15min":0.48499999999999999},"max":{"1min":0.995,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":0.83899999999999997}]},{"osd":5,"last update":"Fri Oct 6 02:32:19 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.752,"15min":0.77200000000000002},"min":{"1min":0.499,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.1699999999999999,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.78600000000000003,"15min":0.79500000000000004},"min":{"1min":0.58099999999999996,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.006,"5min":3.1110000000000002,"15min":3.1110000000000002},"last":0.82899999999999996}]},{"osd":6,"last update":"Fri Oct 6 02:32:19 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.78800000000000003,"15min":0.81100000000000005},"min":{"1min":0.52800000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.0780000000000001,"5min":3.1600000000000001,"15min":3.1600000000000001},"last":0.871},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76600000000000001,"15min":0.80700000000000005},"min":{"1min":0.59999999999999998,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.1100000000000001,"5min":2.3250000000000002,"15min":2.3250000000000002},"last":0.96199999999999997}]},{"osd":7,"last update":"Fri Oct 6 02:32:19 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.81999999999999995,"15min":0.82199999999999995},"min":{"1min":0.58499999999999996,"5min":0.57999999999999996,"15min":0.56000000000000005},"max":{"1min":1.097,"5min":3.3809999999999998,"15min":3.3809999999999998},"last":1.0109999999999999},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77900000000000003,"15min":0.80700000000000005},"min":{"1min":0.59399999999999997,"5min":0.54700000000000004,"15min":0.53300000000000003},"max":{"1min":1.018,"5min":2.3740000000000001,"15min":2.3740000000000001},"last":0.85899999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574970,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303512,"kb_used_data":2128,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068328,"statfs":{"total":96636764160,"available":96325967872,"internally_reserved":0,"allocated":2179072,"data_stored":1016397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:32:18 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.42399999999999999,"15min":0.434},"min":{"1min":0.28199999999999997,"5min":0.245,"15min":0.245},"max":{"1min":0.59399999999999997,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.44},{"interface":"front","average":{"1min":0.495,"5min":0.46600000000000003,"15min":0.47999999999999998},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":1.0800000000000001},"last":0.497}]},{"osd":1,"last update":"Fri Oct 6 02:32:18 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.47599999999999998,"15min":0.495},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32400000000000001},"max":{"1min":0.754,"5min":1.036,"15min":1.036},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51000000000000001,"15min":0.57099999999999995},"min":{"1min":0.40899999999999997,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.68300000000000005,"5min":1.778,"15min":1.778},"last":0.38400000000000001}]},{"osd":3,"last update":"Fri Oct 6 02:32:18 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.499,"15min":0.52100000000000002},"min":{"1min":0.36199999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.97499999999999998},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.57999999999999996,"15min":0.60699999999999998},"min":{"1min":0.46800000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.91700000000000004,"5min":0.95799999999999996,"15min":1.028},"last":0.57099999999999995}]},{"osd":4,"last update":"Fri Oct 6 02:32:18 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.73899999999999999,"15min":0.76400000000000001},"min":{"1min":0.51600000000000001,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.875,"5min":1.0940000000000001,"15min":1.1319999999999999},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.747,"15min":0.81399999999999995},"min":{"1min":0.497,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.0309999999999999,"5min":1.907,"15min":1.907},"last":0.78000000000000003}]},{"osd":5,"last update":"Fri Oct 6 02:32:18 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.72199999999999998,"15min":0.77400000000000002},"min":{"1min":0.51100000000000001,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.97799999999999998,"5min":1.109,"15min":1.1850000000000001},"last":0.97799999999999998},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.751,"15min":0.79800000000000004},"min":{"1min":0.59199999999999997,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.006,"5min":1.0609999999999999,"15min":1.258},"last":0.85299999999999998}]},{"osd":6,"last update":"Fri Oct 6 02:32:18 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77900000000000003,"15min":0.82399999999999995},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.089,"5min":1.827,"15min":1.827},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.76200000000000001,"15min":0.78500000000000003},"min":{"1min":0.52600000000000002,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0469999999999999,"5min":1.0660000000000001,"15min":1.1659999999999999},"last":0.93899999999999995}]},{"osd":7,"last update":"Fri Oct 6 02:32:18 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.78400000000000003,"15min":0.84899999999999998},"min":{"1min":0.61199999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.139,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.77400000000000002,"15min":0.81200000000000006},"min":{"1min":0.57399999999999995,"5min":0.55300000000000005,"15min":0.54800000000000004},"max":{"1min":1.1020000000000001,"5min":1.125,"15min":1.1479999999999999},"last":0.81599999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574967,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303512,"kb_used_data":2128,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068328,"statfs":{"total":96636764160,"available":96325967872,"internally_reserved":0,"allocated":2179072,"data_stored":1016397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:32:15 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.42699999999999999,"15min":0.48199999999999998},"min":{"1min":0.30299999999999999,"5min":0.248,"15min":0.248},"max":{"1min":0.66500000000000004,"5min":1.0489999999999999,"15min":2.024},"last":0.307},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.45000000000000001,"15min":0.5},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.56000000000000005,"5min":0.754,"15min":2.181},"last":0.42899999999999999}]},{"osd":1,"last update":"Fri Oct 6 02:32:15 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.45000000000000001,"15min":0.59899999999999998},"min":{"1min":0.34999999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.65000000000000002,"5min":0.66100000000000003,"15min":4.492},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51000000000000001,"15min":0.64800000000000002},"min":{"1min":0.40999999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.876,"5min":1.079,"15min":4.569},"last":0.39600000000000002}]},{"osd":2,"last update":"Fri Oct 6 02:32:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.58199999999999996,"15min":0.64000000000000001},"min":{"1min":0.435,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.69499999999999995,"5min":1.024,"15min":2.0870000000000002},"last":0.501},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.55200000000000005,"15min":0.61799999999999999},"min":{"1min":0.441,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.77400000000000002,"5min":0.92100000000000004,"15min":2.4260000000000002},"last":0.60499999999999998}]},{"osd":4,"last update":"Fri Oct 6 02:32:15 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.72999999999999998,"15min":0.79400000000000004},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.44800000000000001},"max":{"1min":1.024,"5min":1.024,"15min":2.214},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.751,"15min":0.79000000000000004},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.97999999999999998,"5min":1.042,"15min":1.2869999999999999},"last":0.72799999999999998}]},{"osd":5,"last update":"Fri Oct 6 02:32:15 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.745,"15min":0.79700000000000004},"min":{"1min":0.58499999999999996,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.05,"5min":1.077,"15min":1.3029999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.75,"15min":0.81799999999999995},"min":{"1min":0.55500000000000005,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.097,"5min":1.097,"15min":2.242},"last":0.61599999999999999}]},{"osd":6,"last update":"Fri Oct 6 02:32:15 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.76700000000000002,"15min":0.80000000000000004},"min":{"1min":0.52300000000000002,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.073,"5min":1.073,"15min":1.357},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.78600000000000003,"15min":0.81799999999999995},"min":{"1min":0.61199999999999999,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.95299999999999996,"5min":1.016,"15min":1.3100000000000001},"last":0.63700000000000001}]},{"osd":7,"last update":"Fri Oct 6 02:32:15 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.82399999999999995},"min":{"1min":0.58099999999999996,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.996,"5min":1.004,"15min":2.27},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.78500000000000003,"15min":0.83199999999999996},"min":{"1min":0.61699999999999999,"5min":0.55800000000000005,"15min":0.53200000000000003},"max":{"1min":1.0009999999999999,"5min":1.0820000000000001,"15min":1.325},"last":0.68000000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574969,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":307608,"kb_used_data":2128,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064232,"statfs":{"total":96636764160,"available":96321773568,"internally_reserved":0,"allocated":2179072,"data_stored":1016397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:32:10 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.65100000000000002,"15min":0.88900000000000001},"min":{"1min":0.45800000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.877,"5min":1.0760000000000001,"15min":12.337},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.66700000000000004,"15min":0.83299999999999996},"min":{"1min":0.45100000000000001,"5min":0.44400000000000001,"15min":0.438},"max":{"1min":0.91900000000000004,"5min":1.3520000000000001,"15min":9.6539999999999999},"last":0.52200000000000002}]},{"osd":1,"last update":"Fri Oct 6 02:32:10 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64500000000000002,"15min":0.68400000000000005},"min":{"1min":0.47499999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.83699999999999997,"5min":1.129,"15min":3.181},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.70399999999999996,"15min":0.91800000000000004},"min":{"1min":0.48299999999999998,"5min":0.45300000000000001,"15min":0.40500000000000003},"max":{"1min":0.86799999999999999,"5min":0.995,"15min":12.946999999999999},"last":0.59099999999999997}]},{"osd":2,"last update":"Fri Oct 6 02:32:10 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.69499999999999995,"15min":0.93300000000000005},"min":{"1min":0.47299999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.84499999999999997,"5min":1.2989999999999999,"15min":12.377000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.73099999999999998,"15min":0.82499999999999996},"min":{"1min":0.53700000000000003,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.91200000000000003,"5min":1.0529999999999999,"15min":6.3220000000000001},"last":0.79800000000000004}]},{"osd":3,"last update":"Fri Oct 6 02:32:10 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.72899999999999998,"15min":0.90400000000000003},"min":{"1min":0.57099999999999995,"5min":0.46200000000000002,"15min":0.42699999999999999},"max":{"1min":0.84199999999999997,"5min":1.5229999999999999,"15min":9.7720000000000002},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.70499999999999996,"15min":0.752},"min":{"1min":0.55800000000000005,"5min":0.52400000000000002,"15min":0.47999999999999998},"max":{"1min":0.88900000000000001,"5min":1.212,"15min":3.0840000000000001},"last":0.74299999999999999}]},{"osd":5,"last update":"Fri Oct 6 02:32:10 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72199999999999998,"15min":0.80200000000000005},"min":{"1min":0.46100000000000002,"5min":0.40400000000000003,"15min":0.35299999999999998},"max":{"1min":0.92200000000000004,"5min":1.02,"15min":6.2690000000000001},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.75600000000000001,"15min":0.93899999999999995},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.33600000000000002},"max":{"1min":0.98199999999999998,"5min":1.4359999999999999,"15min":10.769},"last":0.77500000000000002}]},{"osd":6,"last update":"Fri Oct 6 02:32:10 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71799999999999997,"15min":0.74299999999999999},"min":{"1min":0.46300000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.95299999999999996,"5min":1.4590000000000001,"15min":3.1349999999999998},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.80100000000000005,"15min":0.91100000000000003},"min":{"1min":0.52600000000000002,"5min":0.47799999999999998,"15min":0.45700000000000002},"max":{"1min":0.96799999999999997,"5min":1.3839999999999999,"15min":8.0899999999999999},"last":0.76200000000000001}]},{"osd":7,"last update":"Fri Oct 6 02:32:10 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.80300000000000005,"15min":0.97899999999999998},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.47599999999999998},"max":{"1min":1.02,"5min":1.494,"15min":9.7089999999999996},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.78700000000000003,"15min":0.92800000000000005},"min":{"1min":0.56000000000000005,"5min":0.44400000000000001,"15min":0.374},"max":{"1min":1.034,"5min":1.2669999999999999,"15min":7.6719999999999997},"last":0.79000000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574969,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303512,"kb_used_data":2128,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068328,"statfs":{"total":96636764160,"available":96325967872,"internally_reserved":0,"allocated":2179072,"data_stored":1016397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:32:07 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.63200000000000001,"15min":0.63},"min":{"1min":0.47099999999999997,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.93899999999999995,"5min":0.97599999999999998,"15min":1.4279999999999999},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.629,"15min":0.622},"min":{"1min":0.42599999999999999,"5min":0.40300000000000002,"15min":0.38700000000000001},"max":{"1min":0.89300000000000002,"5min":1.099,"15min":1.099},"last":0.79600000000000004}]},{"osd":1,"last update":"Fri Oct 6 02:32:07 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.63600000000000001,"15min":0.64900000000000002},"min":{"1min":0.51700000000000002,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.93899999999999995,"5min":1.0660000000000001,"15min":1.089},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.67300000000000004,"15min":0.67700000000000005},"min":{"1min":0.42299999999999999,"5min":0.40699999999999997,"15min":0.40500000000000003},"max":{"1min":0.93899999999999995,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.78000000000000003}]},{"osd":2,"last update":"Fri Oct 6 02:32:07 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.67800000000000005,"15min":0.67600000000000005},"min":{"1min":0.45800000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.67200000000000004,"15min":0.68200000000000005},"min":{"1min":0.58599999999999997,"5min":0.46100000000000002,"15min":0.44600000000000001},"max":{"1min":1.028,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.75600000000000001}]},{"osd":3,"last update":"Fri Oct 6 02:32:07 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.73299999999999998,"15min":0.73199999999999998},"min":{"1min":0.54000000000000004,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0009999999999999,"5min":1.159,"15min":1.159},"last":0.433},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.77200000000000002,"15min":0.77000000000000002},"min":{"1min":0.64200000000000002,"5min":0.56200000000000006,"15min":0.54100000000000004},"max":{"1min":0.97699999999999998,"5min":1.194,"15min":1.194},"last":0.81000000000000005}]},{"osd":4,"last update":"Fri Oct 6 02:32:07 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.69099999999999995,"15min":0.67700000000000005},"min":{"1min":0.46999999999999997,"5min":0.40799999999999997,"15min":0.32000000000000001},"max":{"1min":0.92300000000000004,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.62},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.69099999999999995,"15min":0.68300000000000005},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.35399999999999998},"max":{"1min":0.88600000000000001,"5min":1.25,"15min":1.25},"last":0.63800000000000001}]},{"osd":6,"last update":"Fri Oct 6 02:32:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.71899999999999997,"15min":0.73499999999999999},"min":{"1min":0.45900000000000002,"5min":0.42899999999999999,"15min":0.40300000000000002},"max":{"1min":1.024,"5min":1.024,"15min":1.1719999999999999},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.81499999999999995,"15min":0.81799999999999995},"min":{"1min":0.60599999999999998,"5min":0.502,"15min":0.373},"max":{"1min":1.0700000000000001,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.82799999999999996}]},{"osd":7,"last update":"Fri Oct 6 02:32:07 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.754,"15min":0.76000000000000001},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.40799999999999997},"max":{"1min":0.95799999999999996,"5min":1.23,"15min":1.23},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.80200000000000005,"15min":0.82099999999999995},"min":{"1min":0.59599999999999997,"5min":0.47599999999999998,"15min":0.437},"max":{"1min":1.1180000000000001,"5min":1.1339999999999999,"15min":1.5640000000000001},"last":0.85499999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T02:32:36.479 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-06T02:32:36.479 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T02:32:36.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.664+0000 7f1e2a19d700 1 Processor -- start 2023-10-06T02:32:36.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.664+0000 7f1e2a19d700 1 -- start start 2023-10-06T02:32:36.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.666+0000 7f1e2a19d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1e2406e0a0 0x7f1e240c6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:36.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.666+0000 7f1e2a19d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f1e24150380 0x7f1e2406a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:36.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.666+0000 7f1e2a19d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e2406a490 con 0x7f1e2406f3a0 2023-10-06T02:32:36.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.666+0000 7f1e2a19d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e240c7490 con 0x7f1e2406e0a0 2023-10-06T02:32:36.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.666+0000 7f1e2a19d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e240c7770 con 0x7f1e24150380 2023-10-06T02:32:36.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.666+0000 7f1e2919b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1e2406e0a0 0x7f1e240c6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:36.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.666+0000 7f1e2899a700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f1e2406f3a0 0x7f1e2414fc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:43874/0 (socket says 172.21.15.31:43874) 2023-10-06T02:32:36.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.666+0000 7f1e2919b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1e2406e0a0 0x7f1e240c6b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58416/0 (socket says 172.21.15.31:58416) 2023-10-06T02:32:36.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.666+0000 7f1e2899a700 1 -- 172.21.15.31:0/1166237165 learned_addr learned my addr 172.21.15.31:0/1166237165 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:32:36.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.666+0000 7f1e2999c700 1 --2- 172.21.15.31:0/1166237165 >> v2:172.21.15.62:3300/0 conn(0x7f1e24150380 0x7f1e2406a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:36.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.666+0000 7f1e2919b700 1 -- 172.21.15.31:0/1166237165 >> v2:172.21.15.62:3300/0 conn(0x7f1e24150380 msgr2=0x7f1e2406a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:36.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.666+0000 7f1e2919b700 1 --2- 172.21.15.31:0/1166237165 >> v2:172.21.15.62:3300/0 conn(0x7f1e24150380 0x7f1e2406a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:36.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.666+0000 7f1e2919b700 1 -- 172.21.15.31:0/1166237165 >> v1:172.21.15.31:6789/0 conn(0x7f1e2406f3a0 legacy=0x7f1e2414fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:36.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.667+0000 7f1e2919b700 1 -- 172.21.15.31:0/1166237165 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e240c7a50 con 0x7f1e2406e0a0 2023-10-06T02:32:36.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.667+0000 7f1e1bfff700 1 -- 172.21.15.31:0/1166237165 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 432552539 0 0) 0x7f1e2406a490 con 0x7f1e2406f3a0 2023-10-06T02:32:36.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.667+0000 7f1e2919b700 1 --2- 172.21.15.31:0/1166237165 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e2406e0a0 0x7f1e240c6b20 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f1e2406a400 tx=0x7f1e200091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26af6c95f346d057 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:36.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.667+0000 7f1e1bfff700 1 -- 172.21.15.31:0/1166237165 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e2000b6b0 con 0x7f1e2406e0a0 2023-10-06T02:32:36.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.667+0000 7f1e1bfff700 1 -- 172.21.15.31:0/1166237165 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e2000b810 con 0x7f1e2406e0a0 2023-10-06T02:32:36.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.667+0000 7f1e1bfff700 1 -- 172.21.15.31:0/1166237165 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e2000bbb0 con 0x7f1e2406e0a0 2023-10-06T02:32:36.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.668+0000 7f1e2a19d700 1 -- 172.21.15.31:0/1166237165 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e2406e0a0 msgr2=0x7f1e240c6b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:36.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.668+0000 7f1e2a19d700 1 --2- 172.21.15.31:0/1166237165 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e2406e0a0 0x7f1e240c6b20 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f1e2406a400 tx=0x7f1e200091a0 comp rx=0 tx=0).stop 2023-10-06T02:32:36.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.668+0000 7f1e2a19d700 1 -- 172.21.15.31:0/1166237165 shutdown_connections 2023-10-06T02:32:36.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.668+0000 7f1e2a19d700 1 --2- 172.21.15.31:0/1166237165 >> v2:172.21.15.62:3300/0 conn(0x7f1e24150380 0x7f1e2406a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:36.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.668+0000 7f1e2a19d700 1 --2- 172.21.15.31:0/1166237165 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e2406e0a0 0x7f1e240c6b20 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:36.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.668+0000 7f1e2a19d700 1 -- 172.21.15.31:0/1166237165 >> 172.21.15.31:0/1166237165 conn(0x7f1e2406d170 msgr2=0x7f1e240cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:36.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.668+0000 7f1e2a19d700 1 -- 172.21.15.31:0/1166237165 shutdown_connections 2023-10-06T02:32:36.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.668+0000 7f1e2a19d700 1 -- 172.21.15.31:0/1166237165 wait complete. 2023-10-06T02:32:36.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.669+0000 7f1e2a19d700 1 Processor -- start 2023-10-06T02:32:36.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.669+0000 7f1e2a19d700 1 -- start start 2023-10-06T02:32:36.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.670+0000 7f1e2a19d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e2406e0a0 0x7f1e240c5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:36.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.670+0000 7f1e2a19d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e2406f3a0 0x7f1e2414bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:36.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.670+0000 7f1e2a19d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e24150380 0x7f1e24156bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:36.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.670+0000 7f1e2a19d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e240afac0 con 0x7f1e2406e0a0 2023-10-06T02:32:36.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.670+0000 7f1e2a19d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e240c59b0 con 0x7f1e24150380 2023-10-06T02:32:36.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.670+0000 7f1e2a19d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e240aeed0 con 0x7f1e2406f3a0 2023-10-06T02:32:36.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.670+0000 7f1e2919b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e2406e0a0 0x7f1e240c5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:36.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.671+0000 7f1e2919b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e2406e0a0 0x7f1e240c5470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58418/0 (socket says 172.21.15.31:58418) 2023-10-06T02:32:36.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.671+0000 7f1e2919b700 1 -- 172.21.15.31:0/3326089598 learned_addr learned my addr 172.21.15.31:0/3326089598 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:36.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.671+0000 7f1e2999c700 1 --2- 172.21.15.31:0/3326089598 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e24150380 0x7f1e24156bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:36.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.671+0000 7f1e2899a700 1 --2- 172.21.15.31:0/3326089598 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e2406f3a0 0x7f1e2414bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:36.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.671+0000 7f1e2919b700 1 -- 172.21.15.31:0/3326089598 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e2406f3a0 msgr2=0x7f1e2414bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:36.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.671+0000 7f1e2919b700 1 --2- 172.21.15.31:0/3326089598 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e2406f3a0 0x7f1e2414bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:36.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.671+0000 7f1e2919b700 1 -- 172.21.15.31:0/3326089598 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e24150380 msgr2=0x7f1e24156bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:36.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.671+0000 7f1e2919b700 1 --2- 172.21.15.31:0/3326089598 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e24150380 0x7f1e24156bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:36.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.671+0000 7f1e2919b700 1 -- 172.21.15.31:0/3326089598 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e20009420 con 0x7f1e2406e0a0 2023-10-06T02:32:36.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.671+0000 7f1e2919b700 1 --2- 172.21.15.31:0/3326089598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e2406e0a0 0x7f1e240c5470 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f1e24071320 tx=0x7f1e20017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:36.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.671+0000 7f1e197fa700 1 -- 172.21.15.31:0/3326089598 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e20011420 con 0x7f1e2406e0a0 2023-10-06T02:32:36.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.671+0000 7f1e197fa700 1 -- 172.21.15.31:0/3326089598 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e20011580 con 0x7f1e2406e0a0 2023-10-06T02:32:36.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.671+0000 7f1e197fa700 1 -- 172.21.15.31:0/3326089598 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e20011940 con 0x7f1e2406e0a0 2023-10-06T02:32:36.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.671+0000 7f1e2a19d700 1 -- 172.21.15.31:0/3326089598 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e240af190 con 0x7f1e2406e0a0 2023-10-06T02:32:36.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.672+0000 7f1e2a19d700 1 -- 172.21.15.31:0/3326089598 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e240af750 con 0x7f1e2406e0a0 2023-10-06T02:32:36.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.672+0000 7f1e2a19d700 1 -- 172.21.15.31:0/3326089598 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e2404c3c0 con 0x7f1e2406e0a0 2023-10-06T02:32:36.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.673+0000 7f1e197fa700 1 -- 172.21.15.31:0/3326089598 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) v1 ==== 85709+0+0 (secure 0 0 0) 0x7f1e20011d30 con 0x7f1e2406e0a0 2023-10-06T02:32:36.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.674+0000 7f1e197fa700 1 --2- 172.21.15.31:0/3326089598 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f1e080679e0 0x7f1e08069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:36.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.674+0000 7f1e197fa700 1 -- 172.21.15.31:0/3326089598 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5320+0+0 (secure 0 0 0) 0x7f1e20084980 con 0x7f1e2406e0a0 2023-10-06T02:32:36.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.675+0000 7f1e2899a700 1 --2- 172.21.15.31:0/3326089598 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f1e080679e0 0x7f1e08069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:36.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.675+0000 7f1e2899a700 1 --2- 172.21.15.31:0/3326089598 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f1e080679e0 0x7f1e08069e90 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f1e2414c550 tx=0x7f1e14009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:36.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.678+0000 7f1e197fa700 1 -- 172.21.15.31:0/3326089598 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1e200535c0 con 0x7f1e2406e0a0 2023-10-06T02:32:36.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.875+0000 7f1e2a19d700 1 -- 172.21.15.31:0/3326089598 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1e240afc00 con 0x7f1e080679e0 2023-10-06T02:32:36.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.878+0000 7f1e197fa700 1 -- 172.21.15.31:0/3326089598 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72379 (secure 0 0 0) 0x7f1e14002d40 con 0x7f1e080679e0 2023-10-06T02:32:36.881 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:32:36.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.881+0000 7f1e2a19d700 1 -- 172.21.15.31:0/3326089598 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f1e080679e0 msgr2=0x7f1e08069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:36.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.881+0000 7f1e2a19d700 1 --2- 172.21.15.31:0/3326089598 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f1e080679e0 0x7f1e08069e90 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f1e2414c550 tx=0x7f1e14009040 comp rx=0 tx=0).stop 2023-10-06T02:32:36.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.881+0000 7f1e2a19d700 1 -- 172.21.15.31:0/3326089598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e2406e0a0 msgr2=0x7f1e240c5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:36.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.881+0000 7f1e2a19d700 1 --2- 172.21.15.31:0/3326089598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e2406e0a0 0x7f1e240c5470 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f1e24071320 tx=0x7f1e20017000 comp rx=0 tx=0).stop 2023-10-06T02:32:36.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.881+0000 7f1e2a19d700 1 -- 172.21.15.31:0/3326089598 shutdown_connections 2023-10-06T02:32:36.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.881+0000 7f1e2a19d700 1 --2- 172.21.15.31:0/3326089598 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e24150380 0x7f1e24156bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:36.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.881+0000 7f1e2a19d700 1 --2- 172.21.15.31:0/3326089598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e2406e0a0 0x7f1e240c5470 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:36.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.881+0000 7f1e2a19d700 1 --2- 172.21.15.31:0/3326089598 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f1e080679e0 0x7f1e08069e90 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:36.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.881+0000 7f1e2a19d700 1 --2- 172.21.15.31:0/3326089598 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e2406f3a0 0x7f1e2414bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:36.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.881+0000 7f1e2a19d700 1 -- 172.21.15.31:0/3326089598 >> 172.21.15.31:0/3326089598 conn(0x7f1e2406d170 msgr2=0x7f1e240c9d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:36.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.881+0000 7f1e2a19d700 1 -- 172.21.15.31:0/3326089598 shutdown_connections 2023-10-06T02:32:36.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:36.881+0000 7f1e2a19d700 1 -- 172.21.15.31:0/3326089598 wait complete. 2023-10-06T02:32:36.884 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-10-06T02:32:36.900 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":249,"stamp":"2023-10-06T02:32:36.530276+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":113,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":2433352,"kb_used_data":18184,"kb_used_omap":0,"kb_used_meta":2415104,"kb_avail":752541368,"statfs":{"total":773094113280,"available":770602360832,"internally_reserved":0,"allocated":18620416,"data_stored":9311912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2473066496},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-55730,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-3,"num_read_kb":-3,"num_write":-73,"num_write_kb":-82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.824454"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":137,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-06T02:32:29.366431+0000","last_change":"2023-10-06T02:23:01.992935+0000","last_active":"2023-10-06T02:32:29.366431+0000","last_peered":"2023-10-06T02:32:29.366431+0000","last_clean":"2023-10-06T02:32:29.366431+0000","last_became_active":"2023-10-06T02:23:01.992568+0000","last_became_peered":"2023-10-06T02:23:01.992568+0000","last_unstale":"2023-10-06T02:32:29.366431+0000","last_undegraded":"2023-10-06T02:32:29.366431+0000","last_fullsized":"2023-10-06T02:32:29.366431+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:06.212251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":138,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-06T02:32:28.707783+0000","last_change":"2023-10-06T02:23:01.989934+0000","last_active":"2023-10-06T02:32:28.707783+0000","last_peered":"2023-10-06T02:32:28.707783+0000","last_clean":"2023-10-06T02:32:28.707783+0000","last_became_active":"2023-10-06T02:23:01.989706+0000","last_became_peered":"2023-10-06T02:23:01.989706+0000","last_unstale":"2023-10-06T02:32:28.707783+0000","last_undegraded":"2023-10-06T02:32:28.707783+0000","last_fullsized":"2023-10-06T02:32:28.707783+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:06.212251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":131,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-06T02:32:29.229695+0000","last_change":"2023-10-06T02:23:02.054069+0000","last_active":"2023-10-06T02:32:29.229695+0000","last_peered":"2023-10-06T02:32:29.229695+0000","last_clean":"2023-10-06T02:32:29.229695+0000","last_became_active":"2023-10-06T02:23:02.053739+0000","last_became_peered":"2023-10-06T02:23:02.053739+0000","last_unstale":"2023-10-06T02:32:29.229695+0000","last_undegraded":"2023-10-06T02:32:29.229695+0000","last_fullsized":"2023-10-06T02:32:29.229695+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:57:48.822608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":131,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-06T02:32:29.229181+0000","last_change":"2023-10-06T02:23:01.990773+0000","last_active":"2023-10-06T02:32:29.229181+0000","last_peered":"2023-10-06T02:32:29.229181+0000","last_clean":"2023-10-06T02:32:29.229181+0000","last_became_active":"2023-10-06T02:23:01.990511+0000","last_became_peered":"2023-10-06T02:23:01.990511+0000","last_unstale":"2023-10-06T02:32:29.229181+0000","last_undegraded":"2023-10-06T02:32:29.229181+0000","last_fullsized":"2023-10-06T02:32:29.229181+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:42:54.797731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"76'128","reported_seq":323,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-06T02:32:29.229325+0000","last_change":"2023-10-06T02:22:55.032519+0000","last_active":"2023-10-06T02:32:29.229325+0000","last_peered":"2023-10-06T02:32:29.229325+0000","last_clean":"2023-10-06T02:32:29.229325+0000","last_became_active":"2023-10-06T02:22:55.031367+0000","last_became_peered":"2023-10-06T02:22:55.031367+0000","last_unstale":"2023-10-06T02:32:29.229325+0000","last_undegraded":"2023-10-06T02:32:29.229325+0000","last_fullsized":"2023-10-06T02:32:29.229325+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:22:53.309097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:22:53.309097+0000","last_clean_scrub_stamp":"2023-10-06T02:22:53.309097+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:53:46.520909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":162,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-06T02:32:31.117514+0000","last_change":"2023-10-06T02:23:01.988140+0000","last_active":"2023-10-06T02:32:31.117514+0000","last_peered":"2023-10-06T02:32:31.117514+0000","last_clean":"2023-10-06T02:32:31.117514+0000","last_became_active":"2023-10-06T02:23:01.987769+0000","last_became_peered":"2023-10-06T02:23:01.987769+0000","last_unstale":"2023-10-06T02:32:31.117514+0000","last_undegraded":"2023-10-06T02:32:31.117514+0000","last_fullsized":"2023-10-06T02:32:31.117514+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:20:08.391912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":148,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-06T02:32:28.708751+0000","last_change":"2023-10-06T02:23:02.054338+0000","last_active":"2023-10-06T02:32:28.708751+0000","last_peered":"2023-10-06T02:32:28.708751+0000","last_clean":"2023-10-06T02:32:28.708751+0000","last_became_active":"2023-10-06T02:23:02.053991+0000","last_became_peered":"2023-10-06T02:23:02.053991+0000","last_unstale":"2023-10-06T02:32:28.708751+0000","last_undegraded":"2023-10-06T02:32:28.708751+0000","last_fullsized":"2023-10-06T02:32:28.708751+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:39:30.192838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":162,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-06T02:32:31.115164+0000","last_change":"2023-10-06T02:23:01.988631+0000","last_active":"2023-10-06T02:32:31.115164+0000","last_peered":"2023-10-06T02:32:31.115164+0000","last_clean":"2023-10-06T02:32:31.115164+0000","last_became_active":"2023-10-06T02:23:01.988303+0000","last_became_peered":"2023-10-06T02:23:01.988303+0000","last_unstale":"2023-10-06T02:32:31.115164+0000","last_undegraded":"2023-10-06T02:32:31.115164+0000","last_fullsized":"2023-10-06T02:32:31.115164+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:38.211466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":138,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-06T02:32:28.707568+0000","last_change":"2023-10-06T02:23:02.054039+0000","last_active":"2023-10-06T02:32:28.707568+0000","last_peered":"2023-10-06T02:32:28.707568+0000","last_clean":"2023-10-06T02:32:28.707568+0000","last_became_active":"2023-10-06T02:23:02.053742+0000","last_became_peered":"2023-10-06T02:23:02.053742+0000","last_unstale":"2023-10-06T02:32:28.707568+0000","last_undegraded":"2023-10-06T02:32:28.707568+0000","last_fullsized":"2023-10-06T02:32:28.707568+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:58:18.846475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574969,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304092,"kb_used_data":2708,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067748,"statfs":{"total":96636764160,"available":96325373952,"internally_reserved":0,"allocated":2772992,"data_stored":1606765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:32:15 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.68200000000000005,"15min":0.65300000000000002},"min":{"1min":0.38400000000000001,"5min":0.318,"15min":0.318},"max":{"1min":7.9160000000000004,"5min":7.9160000000000004,"15min":7.9160000000000004},"last":0.495},{"interface":"front","average":{"1min":1.1759999999999999,"5min":0.73099999999999998,"15min":0.65100000000000002},"min":{"1min":0.47199999999999998,"5min":0.379,"15min":0.35499999999999998},"max":{"1min":11.643000000000001,"5min":11.643000000000001,"15min":11.643000000000001},"last":0.69999999999999996}]},{"osd":1,"last update":"Fri Oct 6 02:32:15 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.72099999999999997,"15min":0.70099999999999996},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.40300000000000002},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":1.069},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.441,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":3.0219999999999998,"5min":3.0219999999999998,"15min":3.0219999999999998},"last":0.629}]},{"osd":2,"last update":"Fri Oct 6 02:32:15 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.67500000000000004,"15min":0.68799999999999994},"min":{"1min":0.435,"5min":0.435,"15min":0.375},"max":{"1min":3.0070000000000001,"5min":3.0070000000000001,"15min":3.0070000000000001},"last":0.61499999999999999},{"interface":"front","average":{"1min":1.2310000000000001,"5min":0.80300000000000005,"15min":0.72699999999999998},"min":{"1min":0.54100000000000004,"5min":0.46999999999999997,"15min":0.39800000000000002},"max":{"1min":12.118,"5min":12.118,"15min":12.118},"last":1.0149999999999999}]},{"osd":3,"last update":"Fri Oct 6 02:32:15 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.71799999999999997,"15min":0.72999999999999998},"min":{"1min":0.46100000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":3.073,"5min":3.073,"15min":3.073},"last":1.472},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.78500000000000003,"15min":0.78000000000000003},"min":{"1min":0.55600000000000005,"5min":0.54500000000000004,"15min":0.497},"max":{"1min":3.105,"5min":3.105,"15min":3.105},"last":1.3300000000000001}]},{"osd":4,"last update":"Fri Oct 6 02:32:15 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.61599999999999999,"15min":0.66100000000000003},"min":{"1min":0.42699999999999999,"5min":0.41799999999999998,"15min":0.35099999999999998},"max":{"1min":0.879,"5min":1.1120000000000001,"15min":1.2849999999999999},"last":1.179},{"interface":"front","average":{"1min":1.0189999999999999,"5min":0.745,"15min":0.754},"min":{"1min":0.42499999999999999,"5min":0.40899999999999997,"15min":0.32900000000000001},"max":{"1min":8.0060000000000002,"5min":8.0060000000000002,"15min":8.0060000000000002},"last":1.5349999999999999}]},{"osd":5,"last update":"Fri Oct 6 02:32:15 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.73799999999999999,"15min":0.73299999999999998},"min":{"1min":0.46200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":3.524,"5min":3.524,"15min":3.524},"last":1.2609999999999999},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.70399999999999996,"15min":0.72099999999999997},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.442},"max":{"1min":3.258,"5min":3.258,"15min":3.258},"last":0.98199999999999998}]},{"osd":6,"last update":"Fri Oct 6 02:32:15 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.79200000000000004,"15min":0.78600000000000003},"min":{"1min":0.54300000000000004,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":3.4689999999999999,"5min":3.4689999999999999,"15min":3.4689999999999999},"last":1.117},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.80000000000000004,"15min":0.80900000000000005},"min":{"1min":0.51500000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":3.4279999999999999,"5min":3.4279999999999999,"15min":3.4279999999999999},"last":0.67100000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574969,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303512,"kb_used_data":2128,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068328,"statfs":{"total":96636764160,"available":96325967872,"internally_reserved":0,"allocated":2179072,"data_stored":1016397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:32:10 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.67600000000000005,"15min":0.66900000000000004},"min":{"1min":0.41599999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.869,"5min":1.869,"15min":1.869},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.71799999999999997,"15min":0.70699999999999996},"min":{"1min":0.49199999999999999,"5min":0.374,"15min":0.374},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.627}]},{"osd":1,"last update":"Fri Oct 6 02:32:10 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66700000000000004,"15min":0.65100000000000002},"min":{"1min":0.47299999999999998,"5min":0.45100000000000001,"15min":0.40699999999999997},"max":{"1min":0.91600000000000004,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.67400000000000004,"15min":0.67000000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.927,"5min":1.927,"15min":1.927},"last":0.59899999999999998}]},{"osd":2,"last update":"Fri Oct 6 02:32:10 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.72999999999999998,"15min":0.71299999999999997},"min":{"1min":0.58199999999999996,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.72499999999999998,"15min":0.71799999999999997},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.40300000000000002},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.71599999999999997}]},{"osd":3,"last update":"Fri Oct 6 02:32:10 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.70599999999999996,"15min":0.67900000000000005},"min":{"1min":0.48899999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.71899999999999997,"15min":0.70999999999999996},"min":{"1min":0.58999999999999997,"5min":0.48799999999999999,"15min":0.45900000000000002},"max":{"1min":0.96299999999999997,"5min":0.97999999999999998,"15min":0.98499999999999999},"last":0.70499999999999996}]},{"osd":4,"last update":"Fri Oct 6 02:32:10 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.68200000000000005,"15min":0.65100000000000002},"min":{"1min":0.379,"5min":0.377,"15min":0.34300000000000003},"max":{"1min":1.97,"5min":1.97,"15min":1.97},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.73199999999999998,"15min":0.68400000000000005},"min":{"1min":0.438,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":2.7429999999999999,"5min":2.7429999999999999,"15min":2.7429999999999999},"last":0.79800000000000004}]},{"osd":5,"last update":"Fri Oct 6 02:32:10 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.71699999999999997,"15min":0.70099999999999996},"min":{"1min":0.47399999999999998,"5min":0.42699999999999999,"15min":0.39700000000000002},"max":{"1min":1.091,"5min":1.091,"15min":1.2929999999999999},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.747,"15min":0.71799999999999997},"min":{"1min":0.496,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":2.484,"5min":2.484,"15min":2.484},"last":0.72999999999999998}]},{"osd":7,"last update":"Fri Oct 6 02:32:10 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.78600000000000003,"15min":0.75900000000000001},"min":{"1min":0.498,"5min":0.42399999999999999,"15min":0.36799999999999999},"max":{"1min":2.7909999999999999,"5min":2.7909999999999999,"15min":2.7909999999999999},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.78800000000000003,"15min":0.76900000000000002},"min":{"1min":0.55100000000000005,"5min":0.46700000000000003,"15min":0.45900000000000002},"max":{"1min":1.0269999999999999,"5min":1.103,"15min":1.2470000000000001},"last":0.752}]}]},{"osd":1,"up_from":13,"seq":55834574968,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303512,"kb_used_data":2128,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068328,"statfs":{"total":96636764160,"available":96325967872,"internally_reserved":0,"allocated":2179072,"data_stored":1016397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.443},"min":{"1min":0.30399999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.58799999999999997,"5min":0.63600000000000001,"15min":1.2230000000000001},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.45800000000000002,"15min":0.501},"min":{"1min":0.34499999999999997,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.66200000000000003,"5min":0.70099999999999996,"15min":1.788},"last":0.46999999999999997}]},{"osd":2,"last update":"Fri Oct 6 02:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.433,"15min":0.49099999999999999},"min":{"1min":0.33500000000000002,"5min":0.307,"15min":0.28299999999999997},"max":{"1min":0.57799999999999996,"5min":0.61799999999999999,"15min":2.0840000000000001},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.503,"5min":0.495,"15min":0.53500000000000003},"min":{"1min":0.374,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.66800000000000004,"5min":0.88100000000000001,"15min":1.6699999999999999},"last":0.52800000000000002}]},{"osd":3,"last update":"Fri Oct 6 02:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.57999999999999996,"15min":0.60399999999999998},"min":{"1min":0.51200000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.79700000000000004,"5min":0.89900000000000002,"15min":1.266},"last":0.498},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.54000000000000004,"15min":0.60299999999999998},"min":{"1min":0.436,"5min":0.40999999999999998,"15min":0.34100000000000003},"max":{"1min":0.70399999999999996,"5min":0.79400000000000004,"15min":1.861},"last":0.42699999999999999}]},{"osd":4,"last update":"Fri Oct 6 02:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.752,"15min":0.82199999999999995},"min":{"1min":0.629,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.97999999999999998,"5min":1.3580000000000001,"15min":1.9279999999999999},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.747,"15min":0.82199999999999995},"min":{"1min":0.53900000000000003,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.95999999999999996,"5min":1.4339999999999999,"15min":2.0179999999999998},"last":0.80500000000000005}]},{"osd":5,"last update":"Fri Oct 6 02:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.71499999999999997,"15min":0.80900000000000005},"min":{"1min":0.57199999999999995,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.91700000000000004,"5min":1.3069999999999999,"15min":1.9830000000000001},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.73199999999999998,"15min":0.79100000000000004},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.47299999999999998},"max":{"1min":0.97299999999999998,"5min":1.327,"15min":1.3380000000000001},"last":0.67400000000000004}]},{"osd":6,"last update":"Fri Oct 6 02:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.76100000000000001,"15min":0.83299999999999996},"min":{"1min":0.62,"5min":0.52100000000000002,"15min":0.45600000000000002},"max":{"1min":1.038,"5min":1.0609999999999999,"15min":1.829},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.77500000000000002,"15min":0.86499999999999999},"min":{"1min":0.60199999999999998,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.96399999999999997,"5min":1.351,"15min":2.052},"last":0.65200000000000002}]},{"osd":7,"last update":"Fri Oct 6 02:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.73399999999999999,"15min":0.80300000000000005},"min":{"1min":0.59399999999999997,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.97299999999999998,"5min":1.3120000000000001,"15min":1.3340000000000001},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.76200000000000001,"15min":0.83799999999999997},"min":{"1min":0.54600000000000004,"5min":0.51600000000000001,"15min":0.47799999999999998},"max":{"1min":1.0780000000000001,"5min":1.2110000000000001,"15min":1.895},"last":0.69999999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574969,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304092,"kb_used_data":2708,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067748,"statfs":{"total":96636764160,"available":96325373952,"internally_reserved":0,"allocated":2772992,"data_stored":1606765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 02:32:19 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.44500000000000001,"15min":0.45000000000000001},"min":{"1min":0.27800000000000002,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.59299999999999997,"5min":2.6120000000000001,"15min":2.6120000000000001},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.47399999999999998,"15min":0.47799999999999998},"min":{"1min":0.33000000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.57099999999999995,"5min":3.4199999999999999,"15min":3.4199999999999999},"last":0.437}]},{"osd":2,"last update":"Fri Oct 6 02:32:19 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.52400000000000002,"15min":0.51400000000000001},"min":{"1min":0.34999999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.61699999999999999,"5min":3.0339999999999998,"15min":3.0339999999999998},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.51500000000000001,"15min":0.51800000000000002},"min":{"1min":0.377,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.76800000000000002,"5min":3.343,"15min":3.343},"last":0.53400000000000003}]},{"osd":3,"last update":"Fri Oct 6 02:32:19 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.52000000000000002,"15min":0.50800000000000001},"min":{"1min":0.41599999999999998,"5min":0.31,"15min":0.31},"max":{"1min":1.0549999999999999,"5min":3.6579999999999999,"15min":3.6579999999999999},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.62,"5min":0.59499999999999997,"15min":0.59599999999999997},"min":{"1min":0.47699999999999998,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.996,"5min":2.6890000000000001,"15min":2.6890000000000001},"last":0.59699999999999998}]},{"osd":4,"last update":"Fri Oct 6 02:32:19 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.77100000000000002,"15min":0.78700000000000003},"min":{"1min":0.55500000000000005,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0649999999999999,"5min":3.2999999999999998,"15min":3.2999999999999998},"last":0.98899999999999999},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.753,"15min":0.78200000000000003},"min":{"1min":0.53900000000000003,"5min":0.51000000000000001,"15min":0.48499999999999999},"max":{"1min":0.995,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":0.83899999999999997}]},{"osd":5,"last update":"Fri Oct 6 02:32:19 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.752,"15min":0.77200000000000002},"min":{"1min":0.499,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.1699999999999999,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.78600000000000003,"15min":0.79500000000000004},"min":{"1min":0.58099999999999996,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.006,"5min":3.1110000000000002,"15min":3.1110000000000002},"last":0.82899999999999996}]},{"osd":6,"last update":"Fri Oct 6 02:32:19 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.78800000000000003,"15min":0.81100000000000005},"min":{"1min":0.52800000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.0780000000000001,"5min":3.1600000000000001,"15min":3.1600000000000001},"last":0.871},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76600000000000001,"15min":0.80700000000000005},"min":{"1min":0.59999999999999998,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.1100000000000001,"5min":2.3250000000000002,"15min":2.3250000000000002},"last":0.96199999999999997}]},{"osd":7,"last update":"Fri Oct 6 02:32:19 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.81999999999999995,"15min":0.82199999999999995},"min":{"1min":0.58499999999999996,"5min":0.57999999999999996,"15min":0.56000000000000005},"max":{"1min":1.097,"5min":3.3809999999999998,"15min":3.3809999999999998},"last":1.0109999999999999},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77900000000000003,"15min":0.80700000000000005},"min":{"1min":0.59399999999999997,"5min":0.54700000000000004,"15min":0.53300000000000003},"max":{"1min":1.018,"5min":2.3740000000000001,"15min":2.3740000000000001},"last":0.85899999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574970,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303512,"kb_used_data":2128,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068328,"statfs":{"total":96636764160,"available":96325967872,"internally_reserved":0,"allocated":2179072,"data_stored":1016397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:32:18 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.42399999999999999,"15min":0.434},"min":{"1min":0.28199999999999997,"5min":0.245,"15min":0.245},"max":{"1min":0.59399999999999997,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.44},{"interface":"front","average":{"1min":0.495,"5min":0.46600000000000003,"15min":0.47999999999999998},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":1.0800000000000001},"last":0.497}]},{"osd":1,"last update":"Fri Oct 6 02:32:18 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.47599999999999998,"15min":0.495},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32400000000000001},"max":{"1min":0.754,"5min":1.036,"15min":1.036},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51000000000000001,"15min":0.57099999999999995},"min":{"1min":0.40899999999999997,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.68300000000000005,"5min":1.778,"15min":1.778},"last":0.38400000000000001}]},{"osd":3,"last update":"Fri Oct 6 02:32:18 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.499,"15min":0.52100000000000002},"min":{"1min":0.36199999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.97499999999999998},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.57999999999999996,"15min":0.60699999999999998},"min":{"1min":0.46800000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.91700000000000004,"5min":0.95799999999999996,"15min":1.028},"last":0.57099999999999995}]},{"osd":4,"last update":"Fri Oct 6 02:32:18 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.73899999999999999,"15min":0.76400000000000001},"min":{"1min":0.51600000000000001,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.875,"5min":1.0940000000000001,"15min":1.1319999999999999},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.747,"15min":0.81399999999999995},"min":{"1min":0.497,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.0309999999999999,"5min":1.907,"15min":1.907},"last":0.78000000000000003}]},{"osd":5,"last update":"Fri Oct 6 02:32:18 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.72199999999999998,"15min":0.77400000000000002},"min":{"1min":0.51100000000000001,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.97799999999999998,"5min":1.109,"15min":1.1850000000000001},"last":0.97799999999999998},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.751,"15min":0.79800000000000004},"min":{"1min":0.59199999999999997,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.006,"5min":1.0609999999999999,"15min":1.258},"last":0.85299999999999998}]},{"osd":6,"last update":"Fri Oct 6 02:32:18 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77900000000000003,"15min":0.82399999999999995},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.089,"5min":1.827,"15min":1.827},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.76200000000000001,"15min":0.78500000000000003},"min":{"1min":0.52600000000000002,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0469999999999999,"5min":1.0660000000000001,"15min":1.1659999999999999},"last":0.93899999999999995}]},{"osd":7,"last update":"Fri Oct 6 02:32:18 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.78400000000000003,"15min":0.84899999999999998},"min":{"1min":0.61199999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.139,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.77400000000000002,"15min":0.81200000000000006},"min":{"1min":0.57399999999999995,"5min":0.55300000000000005,"15min":0.54800000000000004},"max":{"1min":1.1020000000000001,"5min":1.125,"15min":1.1479999999999999},"last":0.81599999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574967,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303512,"kb_used_data":2128,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068328,"statfs":{"total":96636764160,"available":96325967872,"internally_reserved":0,"allocated":2179072,"data_stored":1016397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:32:15 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.42699999999999999,"15min":0.48199999999999998},"min":{"1min":0.30299999999999999,"5min":0.248,"15min":0.248},"max":{"1min":0.66500000000000004,"5min":1.0489999999999999,"15min":2.024},"last":0.307},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.45000000000000001,"15min":0.5},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.56000000000000005,"5min":0.754,"15min":2.181},"last":0.42899999999999999}]},{"osd":1,"last update":"Fri Oct 6 02:32:15 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.45000000000000001,"15min":0.59899999999999998},"min":{"1min":0.34999999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.65000000000000002,"5min":0.66100000000000003,"15min":4.492},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51000000000000001,"15min":0.64800000000000002},"min":{"1min":0.40999999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.876,"5min":1.079,"15min":4.569},"last":0.39600000000000002}]},{"osd":2,"last update":"Fri Oct 6 02:32:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.58199999999999996,"15min":0.64000000000000001},"min":{"1min":0.435,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.69499999999999995,"5min":1.024,"15min":2.0870000000000002},"last":0.501},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.55200000000000005,"15min":0.61799999999999999},"min":{"1min":0.441,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.77400000000000002,"5min":0.92100000000000004,"15min":2.4260000000000002},"last":0.60499999999999998}]},{"osd":4,"last update":"Fri Oct 6 02:32:15 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.72999999999999998,"15min":0.79400000000000004},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.44800000000000001},"max":{"1min":1.024,"5min":1.024,"15min":2.214},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.751,"15min":0.79000000000000004},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.97999999999999998,"5min":1.042,"15min":1.2869999999999999},"last":0.72799999999999998}]},{"osd":5,"last update":"Fri Oct 6 02:32:15 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.745,"15min":0.79700000000000004},"min":{"1min":0.58499999999999996,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.05,"5min":1.077,"15min":1.3029999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.75,"15min":0.81799999999999995},"min":{"1min":0.55500000000000005,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.097,"5min":1.097,"15min":2.242},"last":0.61599999999999999}]},{"osd":6,"last update":"Fri Oct 6 02:32:15 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.76700000000000002,"15min":0.80000000000000004},"min":{"1min":0.52300000000000002,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.073,"5min":1.073,"15min":1.357},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.78600000000000003,"15min":0.81799999999999995},"min":{"1min":0.61199999999999999,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.95299999999999996,"5min":1.016,"15min":1.3100000000000001},"last":0.63700000000000001}]},{"osd":7,"last update":"Fri Oct 6 02:32:15 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.82399999999999995},"min":{"1min":0.58099999999999996,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.996,"5min":1.004,"15min":2.27},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.78500000000000003,"15min":0.83199999999999996},"min":{"1min":0.61699999999999999,"5min":0.55800000000000005,"15min":0.53200000000000003},"max":{"1min":1.0009999999999999,"5min":1.0820000000000001,"15min":1.325},"last":0.68000000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574969,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":307608,"kb_used_data":2128,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064232,"statfs":{"total":96636764160,"available":96321773568,"internally_reserved":0,"allocated":2179072,"data_stored":1016397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:32:10 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.65100000000000002,"15min":0.88900000000000001},"min":{"1min":0.45800000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.877,"5min":1.0760000000000001,"15min":12.337},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.66700000000000004,"15min":0.83299999999999996},"min":{"1min":0.45100000000000001,"5min":0.44400000000000001,"15min":0.438},"max":{"1min":0.91900000000000004,"5min":1.3520000000000001,"15min":9.6539999999999999},"last":0.52200000000000002}]},{"osd":1,"last update":"Fri Oct 6 02:32:10 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64500000000000002,"15min":0.68400000000000005},"min":{"1min":0.47499999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.83699999999999997,"5min":1.129,"15min":3.181},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.70399999999999996,"15min":0.91800000000000004},"min":{"1min":0.48299999999999998,"5min":0.45300000000000001,"15min":0.40500000000000003},"max":{"1min":0.86799999999999999,"5min":0.995,"15min":12.946999999999999},"last":0.59099999999999997}]},{"osd":2,"last update":"Fri Oct 6 02:32:10 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.69499999999999995,"15min":0.93300000000000005},"min":{"1min":0.47299999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.84499999999999997,"5min":1.2989999999999999,"15min":12.377000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.73099999999999998,"15min":0.82499999999999996},"min":{"1min":0.53700000000000003,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.91200000000000003,"5min":1.0529999999999999,"15min":6.3220000000000001},"last":0.79800000000000004}]},{"osd":3,"last update":"Fri Oct 6 02:32:10 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.72899999999999998,"15min":0.90400000000000003},"min":{"1min":0.57099999999999995,"5min":0.46200000000000002,"15min":0.42699999999999999},"max":{"1min":0.84199999999999997,"5min":1.5229999999999999,"15min":9.7720000000000002},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.70499999999999996,"15min":0.752},"min":{"1min":0.55800000000000005,"5min":0.52400000000000002,"15min":0.47999999999999998},"max":{"1min":0.88900000000000001,"5min":1.212,"15min":3.0840000000000001},"last":0.74299999999999999}]},{"osd":5,"last update":"Fri Oct 6 02:32:10 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72199999999999998,"15min":0.80200000000000005},"min":{"1min":0.46100000000000002,"5min":0.40400000000000003,"15min":0.35299999999999998},"max":{"1min":0.92200000000000004,"5min":1.02,"15min":6.2690000000000001},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.75600000000000001,"15min":0.93899999999999995},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.33600000000000002},"max":{"1min":0.98199999999999998,"5min":1.4359999999999999,"15min":10.769},"last":0.77500000000000002}]},{"osd":6,"last update":"Fri Oct 6 02:32:10 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71799999999999997,"15min":0.74299999999999999},"min":{"1min":0.46300000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.95299999999999996,"5min":1.4590000000000001,"15min":3.1349999999999998},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.80100000000000005,"15min":0.91100000000000003},"min":{"1min":0.52600000000000002,"5min":0.47799999999999998,"15min":0.45700000000000002},"max":{"1min":0.96799999999999997,"5min":1.3839999999999999,"15min":8.0899999999999999},"last":0.76200000000000001}]},{"osd":7,"last update":"Fri Oct 6 02:32:10 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.80300000000000005,"15min":0.97899999999999998},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.47599999999999998},"max":{"1min":1.02,"5min":1.494,"15min":9.7089999999999996},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.78700000000000003,"15min":0.92800000000000005},"min":{"1min":0.56000000000000005,"5min":0.44400000000000001,"15min":0.374},"max":{"1min":1.034,"5min":1.2669999999999999,"15min":7.6719999999999997},"last":0.79000000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574969,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303512,"kb_used_data":2128,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068328,"statfs":{"total":96636764160,"available":96325967872,"internally_reserved":0,"allocated":2179072,"data_stored":1016397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:32:07 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.63200000000000001,"15min":0.63},"min":{"1min":0.47099999999999997,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.93899999999999995,"5min":0.97599999999999998,"15min":1.4279999999999999},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.629,"15min":0.622},"min":{"1min":0.42599999999999999,"5min":0.40300000000000002,"15min":0.38700000000000001},"max":{"1min":0.89300000000000002,"5min":1.099,"15min":1.099},"last":0.79600000000000004}]},{"osd":1,"last update":"Fri Oct 6 02:32:07 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.63600000000000001,"15min":0.64900000000000002},"min":{"1min":0.51700000000000002,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.93899999999999995,"5min":1.0660000000000001,"15min":1.089},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.67300000000000004,"15min":0.67700000000000005},"min":{"1min":0.42299999999999999,"5min":0.40699999999999997,"15min":0.40500000000000003},"max":{"1min":0.93899999999999995,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.78000000000000003}]},{"osd":2,"last update":"Fri Oct 6 02:32:07 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.67800000000000005,"15min":0.67600000000000005},"min":{"1min":0.45800000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.67200000000000004,"15min":0.68200000000000005},"min":{"1min":0.58599999999999997,"5min":0.46100000000000002,"15min":0.44600000000000001},"max":{"1min":1.028,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.75600000000000001}]},{"osd":3,"last update":"Fri Oct 6 02:32:07 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.73299999999999998,"15min":0.73199999999999998},"min":{"1min":0.54000000000000004,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0009999999999999,"5min":1.159,"15min":1.159},"last":0.433},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.77200000000000002,"15min":0.77000000000000002},"min":{"1min":0.64200000000000002,"5min":0.56200000000000006,"15min":0.54100000000000004},"max":{"1min":0.97699999999999998,"5min":1.194,"15min":1.194},"last":0.81000000000000005}]},{"osd":4,"last update":"Fri Oct 6 02:32:07 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.69099999999999995,"15min":0.67700000000000005},"min":{"1min":0.46999999999999997,"5min":0.40799999999999997,"15min":0.32000000000000001},"max":{"1min":0.92300000000000004,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.62},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.69099999999999995,"15min":0.68300000000000005},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.35399999999999998},"max":{"1min":0.88600000000000001,"5min":1.25,"15min":1.25},"last":0.63800000000000001}]},{"osd":6,"last update":"Fri Oct 6 02:32:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.71899999999999997,"15min":0.73499999999999999},"min":{"1min":0.45900000000000002,"5min":0.42899999999999999,"15min":0.40300000000000002},"max":{"1min":1.024,"5min":1.024,"15min":1.1719999999999999},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.81499999999999995,"15min":0.81799999999999995},"min":{"1min":0.60599999999999998,"5min":0.502,"15min":0.373},"max":{"1min":1.0700000000000001,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.82799999999999996}]},{"osd":7,"last update":"Fri Oct 6 02:32:07 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.754,"15min":0.76000000000000001},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.40799999999999997},"max":{"1min":0.95799999999999996,"5min":1.23,"15min":1.23},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.80200000000000005,"15min":0.82099999999999995},"min":{"1min":0.59599999999999997,"5min":0.47599999999999998,"15min":0.437},"max":{"1min":1.1180000000000001,"5min":1.1339999999999999,"15min":1.5640000000000001},"last":0.85499999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T02:32:36.902 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:32:37.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:37.087+0000 7fa97d99d700 1 Processor -- start 2023-10-06T02:32:37.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:37.087+0000 7fa97d99d700 1 -- start start 2023-10-06T02:32:37.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:37.088+0000 7fa97d99d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa978142c00 0x7fa97806b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:37.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:37.089+0000 7fa97d99d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa9780c7f90 0x7fa97806c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:37.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:37.089+0000 7fa97d99d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9780ca480 con 0x7fa97806bd20 2023-10-06T02:32:37.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:37.089+0000 7fa97d99d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa97815e240 con 0x7fa9780c7f90 2023-10-06T02:32:37.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:37.089+0000 7fa97d99d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa97806b350 con 0x7fa978142c00 2023-10-06T02:32:37.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:37.089+0000 7fa97d19c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa9780c7f90 0x7fa97806c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:37.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:37.089+0000 7fa97d19c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa9780c7f90 0x7fa97806c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58424/0 (socket says 172.21.15.31:58424) 2023-10-06T02:32:37.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:37.089+0000 7fa97d19c700 1 -- 172.21.15.31:0/2345133795 learned_addr learned my addr 172.21.15.31:0/2345133795 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:37.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:37.089+0000 7fa97c99b700 1 --2- 172.21.15.31:0/2345133795 >> v2:172.21.15.62:3300/0 conn(0x7fa978142c00 0x7fa97806b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:37.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:37.089+0000 7fa97d19c700 1 -- 172.21.15.31:0/2345133795 >> v2:172.21.15.62:3300/0 conn(0x7fa978142c00 msgr2=0x7fa97806b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:37.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:37.089+0000 7fa97d19c700 1 --2- 172.21.15.31:0/2345133795 >> v2:172.21.15.62:3300/0 conn(0x7fa978142c00 0x7fa97806b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:37.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:37.089+0000 7fa97d19c700 1 -- 172.21.15.31:0/2345133795 >> v1:172.21.15.62:6789/0 conn(0x7fa97806bd20 legacy=0x7fa9780c7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:37.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:37.089+0000 7fa97d19c700 1 -- 172.21.15.31:0/2345133795 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa97815df60 con 0x7fa9780c7f90 2023-10-06T02:32:37.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:37.090+0000 7fa97d19c700 1 --2- 172.21.15.31:0/2345133795 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9780c7f90 0x7fa97806c6a0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7fa97806f490 tx=0x7fa970009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49f8ec36c2d57dba server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:37.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:37.090+0000 7fa9777fe700 1 -- 172.21.15.31:0/2345133795 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa97000e070 con 0x7fa9780c7f90 2023-10-06T02:32:37.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:37.090+0000 7fa9777fe700 1 -- 172.21.15.31:0/2345133795 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa97000ba50 con 0x7fa9780c7f90 2023-10-06T02:32:37.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:37.090+0000 7fa9777fe700 1 -- 172.21.15.31:0/2345133795 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9700092e0 con 0x7fa9780c7f90 2023-10-06T02:32:37.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:37.090+0000 7fa97d99d700 1 -- 172.21.15.31:0/2345133795 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9780c7f90 msgr2=0x7fa97806c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:37.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:37.090+0000 7fa97d99d700 1 --2- 172.21.15.31:0/2345133795 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9780c7f90 0x7fa97806c6a0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7fa97806f490 tx=0x7fa970009840 comp rx=0 tx=0).stop 2023-10-06T02:32:37.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:37.091+0000 7fa97d99d700 1 -- 172.21.15.31:0/2345133795 shutdown_connections 2023-10-06T02:32:37.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:37.091+0000 7fa97d99d700 1 --2- 172.21.15.31:0/2345133795 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9780c7f90 0x7fa97806c6a0 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:37.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:37.091+0000 7fa97d99d700 1 --2- 172.21.15.31:0/2345133795 >> v2:172.21.15.62:3300/0 conn(0x7fa978142c00 0x7fa97806b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:37.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:37.091+0000 7fa97d99d700 1 -- 172.21.15.31:0/2345133795 >> 172.21.15.31:0/2345133795 conn(0x7fa9780675a0 msgr2=0x7fa9780679a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:37.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:37.091+0000 7fa97d99d700 1 -- 172.21.15.31:0/2345133795 shutdown_connections 2023-10-06T02:32:37.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:37.091+0000 7fa97d99d700 1 -- 172.21.15.31:0/2345133795 wait complete. 2023-10-06T02:32:37.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:37.092+0000 7fa97d99d700 1 Processor -- start 2023-10-06T02:32:37.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:37.092+0000 7fa97d99d700 1 -- start start 2023-10-06T02:32:37.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:37.093+0000 7fa97d99d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa97806bd20 0x7fa9781403b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:37.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:37.093+0000 7fa97d99d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9780c7f90 0x7fa9780b19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:37.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:37.093+0000 7fa97d99d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa978142c00 0x7fa978158f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:37.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:37.093+0000 7fa97d99d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa978140cf0 con 0x7fa9780c7f90 2023-10-06T02:32:37.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:37.093+0000 7fa97d99d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9780b1f00 con 0x7fa97806bd20 2023-10-06T02:32:37.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:37.093+0000 7fa97d99d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa97815c120 con 0x7fa978142c00 2023-10-06T02:32:37.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:37.093+0000 7fa977fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9780c7f90 0x7fa9780b19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:37.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:37.093+0000 7fa977fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9780c7f90 0x7fa9780b19c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58440/0 (socket says 172.21.15.31:58440) 2023-10-06T02:32:37.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:37.093+0000 7fa977fff700 1 -- 172.21.15.31:0/489205515 learned_addr learned my addr 172.21.15.31:0/489205515 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:37.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:37.094+0000 7fa97c99b700 1 --2- 172.21.15.31:0/489205515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa97806bd20 0x7fa9781403b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:37.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:37.094+0000 7fa97d19c700 1 --2- 172.21.15.31:0/489205515 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa978142c00 0x7fa978158f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:37.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:37.094+0000 7fa977fff700 1 -- 172.21.15.31:0/489205515 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa978142c00 msgr2=0x7fa978158f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:37.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:37.094+0000 7fa977fff700 1 --2- 172.21.15.31:0/489205515 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa978142c00 0x7fa978158f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:37.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:37.094+0000 7fa977fff700 1 -- 172.21.15.31:0/489205515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa97806bd20 msgr2=0x7fa9781403b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:37.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:37.094+0000 7fa977fff700 1 --2- 172.21.15.31:0/489205515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa97806bd20 0x7fa9781403b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:37.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:37.094+0000 7fa977fff700 1 -- 172.21.15.31:0/489205515 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa970009ae0 con 0x7fa9780c7f90 2023-10-06T02:32:37.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:37.094+0000 7fa97d19c700 1 --2- 172.21.15.31:0/489205515 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa978142c00 0x7fa978158f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:32:37.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:37.094+0000 7fa97c99b700 1 --2- 172.21.15.31:0/489205515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa97806bd20 0x7fa9781403b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:32:37.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:37.094+0000 7fa977fff700 1 --2- 172.21.15.31:0/489205515 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9780c7f90 0x7fa9780b19c0 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7fa96400c7c0 tx=0x7fa964007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:37.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:37.094+0000 7fa974ff9700 1 -- 172.21.15.31:0/489205515 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa964011590 con 0x7fa9780c7f90 2023-10-06T02:32:37.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:37.094+0000 7fa974ff9700 1 -- 172.21.15.31:0/489205515 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9640116f0 con 0x7fa9780c7f90 2023-10-06T02:32:37.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:37.094+0000 7fa974ff9700 1 -- 172.21.15.31:0/489205515 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa964011ab0 con 0x7fa9780c7f90 2023-10-06T02:32:37.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:37.094+0000 7fa97d99d700 1 -- 172.21.15.31:0/489205515 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa97815be30 con 0x7fa9780c7f90 2023-10-06T02:32:37.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:37.095+0000 7fa97d99d700 1 -- 172.21.15.31:0/489205515 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9780b2490 con 0x7fa9780c7f90 2023-10-06T02:32:37.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:37.095+0000 7fa97d99d700 1 -- 172.21.15.31:0/489205515 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa97804c3c0 con 0x7fa9780c7f90 2023-10-06T02:32:37.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:37.096+0000 7fa974ff9700 1 -- 172.21.15.31:0/489205515 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) v1 ==== 85709+0+0 (secure 0 0 0) 0x7fa96401b3b0 con 0x7fa9780c7f90 2023-10-06T02:32:37.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:37.097+0000 7fa974ff9700 1 --2- 172.21.15.31:0/489205515 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa95c0679e0 0x7fa95c069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:37.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:37.097+0000 7fa974ff9700 1 -- 172.21.15.31:0/489205515 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5320+0+0 (secure 0 0 0) 0x7fa9640826b0 con 0x7fa9780c7f90 2023-10-06T02:32:37.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:37.098+0000 7fa97c99b700 1 --2- 172.21.15.31:0/489205515 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa95c0679e0 0x7fa95c069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:37.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:37.098+0000 7fa97c99b700 1 --2- 172.21.15.31:0/489205515 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa95c0679e0 0x7fa95c069e90 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fa96c006fd0 tx=0x7fa96c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:37.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:37.101+0000 7fa974ff9700 1 -- 172.21.15.31:0/489205515 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa9640510c0 con 0x7fa9780c7f90 2023-10-06T02:32:37.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:37.306+0000 7fa97d99d700 1 -- 172.21.15.31:0/489205515 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa97806ad00 con 0x7fa9780c7f90 2023-10-06T02:32:37.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:37.307+0000 7fa974ff9700 1 -- 172.21.15.31:0/489205515 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v172) v1 ==== 74+0+13785 (secure 0 0 0) 0x7fa9640546e0 con 0x7fa9780c7f90 2023-10-06T02:32:37.308 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:32:37.308 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":172,"fsid":"d6fcf512-874b-4259-b90a-30ae070f00a9","created":"2023-10-06T02:22:40.182670+0000","modified":"2023-10-06T02:32:32.111235+0000","last_up_change":"2023-10-06T02:22:52.300701+0000","last_in_change":"2023-10-06T02:22:43.630440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:22:53.183425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:23:00.046967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"e9b19574-d60f-480a-bb2d-ab0672dcbccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":85567},{"type":"v1","addr":"172.21.15.31:6801","nonce":85567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":85567},{"type":"v1","addr":"172.21.15.31:6803","nonce":85567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":85567},{"type":"v1","addr":"172.21.15.31:6807","nonce":85567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":85567},{"type":"v1","addr":"172.21.15.31:6805","nonce":85567}]},"public_addr":"172.21.15.31:6801/85567","cluster_addr":"172.21.15.31:6803/85567","heartbeat_back_addr":"172.21.15.31:6807/85567","heartbeat_front_addr":"172.21.15.31:6805/85567","state":["exists","up"]},{"osd":1,"uuid":"5ac13881-4851-43f8-bfab-c9b162300451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":85568},{"type":"v1","addr":"172.21.15.31:6809","nonce":85568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":85568},{"type":"v1","addr":"172.21.15.31:6811","nonce":85568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":85568},{"type":"v1","addr":"172.21.15.31:6816","nonce":85568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":85568},{"type":"v1","addr":"172.21.15.31:6813","nonce":85568}]},"public_addr":"172.21.15.31:6809/85568","cluster_addr":"172.21.15.31:6811/85568","heartbeat_back_addr":"172.21.15.31:6816/85568","heartbeat_front_addr":"172.21.15.31:6813/85568","state":["exists","up"]},{"osd":2,"uuid":"ad28fcc7-3408-4231-81cc-5a01e7a56ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":85579},{"type":"v1","addr":"172.21.15.31:6825","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":85579},{"type":"v1","addr":"172.21.15.31:6827","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":85579},{"type":"v1","addr":"172.21.15.31:6831","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":85579},{"type":"v1","addr":"172.21.15.31:6829","nonce":85579}]},"public_addr":"172.21.15.31:6825/85579","cluster_addr":"172.21.15.31:6827/85579","heartbeat_back_addr":"172.21.15.31:6831/85579","heartbeat_front_addr":"172.21.15.31:6829/85579","state":["exists","up"]},{"osd":3,"uuid":"afb6d8de-115b-4585-acdc-e5c2d6f38269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6815","nonce":85584},{"type":"v1","addr":"172.21.15.31:6817","nonce":85584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":85584},{"type":"v1","addr":"172.21.15.31:6819","nonce":85584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":85584},{"type":"v1","addr":"172.21.15.31:6824","nonce":85584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":85584},{"type":"v1","addr":"172.21.15.31:6821","nonce":85584}]},"public_addr":"172.21.15.31:6817/85584","cluster_addr":"172.21.15.31:6819/85584","heartbeat_back_addr":"172.21.15.31:6824/85584","heartbeat_front_addr":"172.21.15.31:6821/85584","state":["exists","up"]},{"osd":4,"uuid":"909bca1c-f5fd-4efd-b50e-79843e358fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":85799},{"type":"v1","addr":"172.21.15.62:6801","nonce":85799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":85799},{"type":"v1","addr":"172.21.15.62:6803","nonce":85799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":85799},{"type":"v1","addr":"172.21.15.62:6807","nonce":85799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":85799},{"type":"v1","addr":"172.21.15.62:6805","nonce":85799}]},"public_addr":"172.21.15.62:6801/85799","cluster_addr":"172.21.15.62:6803/85799","heartbeat_back_addr":"172.21.15.62:6807/85799","heartbeat_front_addr":"172.21.15.62:6805/85799","state":["exists","up"]},{"osd":5,"uuid":"48d582fb-7f96-4cd2-ae6e-95ab273a5907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":85800},{"type":"v1","addr":"172.21.15.62:6817","nonce":85800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":85800},{"type":"v1","addr":"172.21.15.62:6819","nonce":85800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":85800},{"type":"v1","addr":"172.21.15.62:6823","nonce":85800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":85800},{"type":"v1","addr":"172.21.15.62:6821","nonce":85800}]},"public_addr":"172.21.15.62:6817/85800","cluster_addr":"172.21.15.62:6819/85800","heartbeat_back_addr":"172.21.15.62:6823/85800","heartbeat_front_addr":"172.21.15.62:6821/85800","state":["exists","up"]},{"osd":6,"uuid":"485507f8-ea88-45d4-a0fd-793ea2d7507b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":85798},{"type":"v1","addr":"172.21.15.62:6809","nonce":85798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":85798},{"type":"v1","addr":"172.21.15.62:6811","nonce":85798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":85798},{"type":"v1","addr":"172.21.15.62:6815","nonce":85798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":85798},{"type":"v1","addr":"172.21.15.62:6813","nonce":85798}]},"public_addr":"172.21.15.62:6809/85798","cluster_addr":"172.21.15.62:6811/85798","heartbeat_back_addr":"172.21.15.62:6815/85798","heartbeat_front_addr":"172.21.15.62:6813/85798","state":["exists","up"]},{"osd":7,"uuid":"ce5a1b03-1cb6-4cc3-a303-90859e26d517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":85843},{"type":"v1","addr":"172.21.15.62:6825","nonce":85843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":85843},{"type":"v1","addr":"172.21.15.62:6827","nonce":85843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":85843},{"type":"v1","addr":"172.21.15.62:6831","nonce":85843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":85843},{"type":"v1","addr":"172.21.15.62:6829","nonce":85843}]},"public_addr":"172.21.15.62:6825/85843","cluster_addr":"172.21.15.62:6827/85843","heartbeat_back_addr":"172.21.15.62:6831/85843","heartbeat_front_addr":"172.21.15.62:6829/85843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.905021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.930982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.988576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:49.006475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.896630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.931893+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.900282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:22:48.714476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:6834/3741373122":"2023-10-07T02:32:28.700776+0000","172.21.15.31:6832/3741373122":"2023-10-07T02:32:28.700776+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":20,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:32:37.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:37.309+0000 7fa97d99d700 1 -- 172.21.15.31:0/489205515 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa95c0679e0 msgr2=0x7fa95c069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:37.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:37.309+0000 7fa97d99d700 1 --2- 172.21.15.31:0/489205515 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa95c0679e0 0x7fa95c069e90 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fa96c006fd0 tx=0x7fa96c008040 comp rx=0 tx=0).stop 2023-10-06T02:32:37.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:37.309+0000 7fa97d99d700 1 -- 172.21.15.31:0/489205515 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9780c7f90 msgr2=0x7fa9780b19c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:37.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:37.309+0000 7fa97d99d700 1 --2- 172.21.15.31:0/489205515 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9780c7f90 0x7fa9780b19c0 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7fa96400c7c0 tx=0x7fa964007f50 comp rx=0 tx=0).stop 2023-10-06T02:32:37.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:37.309+0000 7fa97d99d700 1 -- 172.21.15.31:0/489205515 shutdown_connections 2023-10-06T02:32:37.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:37.309+0000 7fa97d99d700 1 --2- 172.21.15.31:0/489205515 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fa95c0679e0 0x7fa95c069e90 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:37.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:37.309+0000 7fa97d99d700 1 --2- 172.21.15.31:0/489205515 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa978142c00 0x7fa978158f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:37.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:37.309+0000 7fa97d99d700 1 --2- 172.21.15.31:0/489205515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa97806bd20 0x7fa9781403b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:37.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:37.309+0000 7fa97d99d700 1 --2- 172.21.15.31:0/489205515 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9780c7f90 0x7fa9780b19c0 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:37.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:37.309+0000 7fa97d99d700 1 -- 172.21.15.31:0/489205515 >> 172.21.15.31:0/489205515 conn(0x7fa9780675a0 msgr2=0x7fa9780c2e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:37.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:37.309+0000 7fa97d99d700 1 -- 172.21.15.31:0/489205515 shutdown_connections 2023-10-06T02:32:37.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:37.309+0000 7fa97d99d700 1 -- 172.21.15.31:0/489205515 wait complete. 2023-10-06T02:32:38.329 INFO:tasks.ceph:Scrubbing osd.0 2023-10-06T02:32:38.329 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-10-06T02:32:38.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.531+0000 7f51e179d700 1 Processor -- start 2023-10-06T02:32:38.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.531+0000 7f51e179d700 1 -- start start 2023-10-06T02:32:38.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.532+0000 7f51e179d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f51dc0cce90 0x7f51dc0cd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:38.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.532+0000 7f51e179d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f51dc0cb440 0x7f51dc0cb860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:38.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.532+0000 7f51e179d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51dc0cd840 con 0x7f51dc159fb0 2023-10-06T02:32:38.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.532+0000 7f51e179d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51dc15e180 con 0x7f51dc0cb440 2023-10-06T02:32:38.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.532+0000 7f51e179d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51dc15d570 con 0x7f51dc0cce90 2023-10-06T02:32:38.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.533+0000 7f51e0f9c700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f51dc0cb440 0x7f51dc0cb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:38.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.533+0000 7f51db7fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f51dc0cce90 0x7f51dc0cd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:38.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.533+0000 7f51e0f9c700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f51dc0cb440 0x7f51dc0cb860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:49668/0 (socket says 172.21.15.31:49668) 2023-10-06T02:32:38.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.533+0000 7f51e0f9c700 1 -- 172.21.15.31:0/189589407 learned_addr learned my addr 172.21.15.31:0/189589407 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:38.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.533+0000 7f51dbfff700 1 --1- 172.21.15.31:0/189589407 >> v1:172.21.15.62:6789/0 conn(0x7f51dc159fb0 0x7f51dc15a380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:51926/0 (socket says 172.21.15.31:51926) 2023-10-06T02:32:38.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.534+0000 7f51db7fe700 1 -- 172.21.15.31:0/189589407 >> v2:172.21.15.62:3300/0 conn(0x7f51dc0cb440 msgr2=0x7f51dc0cb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:38.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.534+0000 7f51db7fe700 1 --2- 172.21.15.31:0/189589407 >> v2:172.21.15.62:3300/0 conn(0x7f51dc0cb440 0x7f51dc0cb860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:38.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.534+0000 7f51db7fe700 1 -- 172.21.15.31:0/189589407 >> v1:172.21.15.62:6789/0 conn(0x7f51dc159fb0 legacy=0x7f51dc15a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:38.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.534+0000 7f51db7fe700 1 -- 172.21.15.31:0/189589407 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51dc15d850 con 0x7f51dc0cce90 2023-10-06T02:32:38.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.534+0000 7f51e0f9c700 1 --2- 172.21.15.31:0/189589407 >> v2:172.21.15.62:3300/0 conn(0x7f51dc0cb440 0x7f51dc0cb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:32:38.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.534+0000 7f51daffd700 1 -- 172.21.15.31:0/189589407 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1483778407 0 0) 0x7f51dc0cd840 con 0x7f51dc159fb0 2023-10-06T02:32:38.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.534+0000 7f51db7fe700 1 --2- 172.21.15.31:0/189589407 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f51dc0cce90 0x7f51dc0cd300 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f51d00098e0 tx=0x7f51d00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4f489cdd137bae3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:38.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.534+0000 7f51daffd700 1 -- 172.21.15.31:0/189589407 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d000b640 con 0x7f51dc0cce90 2023-10-06T02:32:38.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.534+0000 7f51daffd700 1 -- 172.21.15.31:0/189589407 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51d000b7a0 con 0x7f51dc0cce90 2023-10-06T02:32:38.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.534+0000 7f51daffd700 1 -- 172.21.15.31:0/189589407 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d000bb40 con 0x7f51dc0cce90 2023-10-06T02:32:38.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.535+0000 7f51e179d700 1 -- 172.21.15.31:0/189589407 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f51dc0cce90 msgr2=0x7f51dc0cd300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:38.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.535+0000 7f51e179d700 1 --2- 172.21.15.31:0/189589407 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f51dc0cce90 0x7f51dc0cd300 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f51d00098e0 tx=0x7f51d00091a0 comp rx=0 tx=0).stop 2023-10-06T02:32:38.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.535+0000 7f51e179d700 1 -- 172.21.15.31:0/189589407 shutdown_connections 2023-10-06T02:32:38.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.535+0000 7f51e179d700 1 --2- 172.21.15.31:0/189589407 >> v2:172.21.15.62:3300/0 conn(0x7f51dc0cb440 0x7f51dc0cb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:38.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.535+0000 7f51e179d700 1 --2- 172.21.15.31:0/189589407 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f51dc0cce90 0x7f51dc0cd300 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:38.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.535+0000 7f51e179d700 1 -- 172.21.15.31:0/189589407 >> 172.21.15.31:0/189589407 conn(0x7f51dc0703a0 msgr2=0x7f51dc0707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:38.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.535+0000 7f51e179d700 1 -- 172.21.15.31:0/189589407 shutdown_connections 2023-10-06T02:32:38.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.535+0000 7f51e179d700 1 -- 172.21.15.31:0/189589407 wait complete. 2023-10-06T02:32:38.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.536+0000 7f51e179d700 1 Processor -- start 2023-10-06T02:32:38.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.537+0000 7f51e179d700 1 -- start start 2023-10-06T02:32:38.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.537+0000 7f51e179d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f51dc0cb440 0x7f51dc15ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:38.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.537+0000 7f51e179d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51dc0cce90 0x7f51dc13fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:38.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.537+0000 7f51e179d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51dc159fb0 0x7f51dc14e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:38.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.537+0000 7f51e179d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51dc0b0940 con 0x7f51dc159fb0 2023-10-06T02:32:38.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.537+0000 7f51e179d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51dc15f500 con 0x7f51dc0cce90 2023-10-06T02:32:38.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.537+0000 7f51e179d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51dc15f7e0 con 0x7f51dc0cb440 2023-10-06T02:32:38.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.538+0000 7f51e0f9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51dc159fb0 0x7f51dc14e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:38.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.538+0000 7f51e0f9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51dc159fb0 0x7f51dc14e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49930/0 (socket says 172.21.15.31:49930) 2023-10-06T02:32:38.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.538+0000 7f51e0f9c700 1 -- 172.21.15.31:0/3113399553 learned_addr learned my addr 172.21.15.31:0/3113399553 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:38.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.538+0000 7f51db7fe700 1 --2- 172.21.15.31:0/3113399553 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51dc0cce90 0x7f51dc13fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:38.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.538+0000 7f51dbfff700 1 --2- 172.21.15.31:0/3113399553 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f51dc0cb440 0x7f51dc15ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:38.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.538+0000 7f51e0f9c700 1 -- 172.21.15.31:0/3113399553 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f51dc0cb440 msgr2=0x7f51dc15ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:38.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.538+0000 7f51e0f9c700 1 --2- 172.21.15.31:0/3113399553 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f51dc0cb440 0x7f51dc15ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:38.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.538+0000 7f51e0f9c700 1 -- 172.21.15.31:0/3113399553 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51dc0cce90 msgr2=0x7f51dc13fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:38.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.538+0000 7f51e0f9c700 1 --2- 172.21.15.31:0/3113399553 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51dc0cce90 0x7f51dc13fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:38.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.538+0000 7f51e0f9c700 1 -- 172.21.15.31:0/3113399553 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51d4009ae0 con 0x7f51dc159fb0 2023-10-06T02:32:38.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.538+0000 7f51e0f9c700 1 --2- 172.21.15.31:0/3113399553 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51dc159fb0 0x7f51dc14e610 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f51d4009fd0 tx=0x7f51d400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:38.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.539+0000 7f51c3fff700 1 -- 172.21.15.31:0/3113399553 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d4018070 con 0x7f51dc159fb0 2023-10-06T02:32:38.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.539+0000 7f51c3fff700 1 -- 172.21.15.31:0/3113399553 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51d400f040 con 0x7f51dc159fb0 2023-10-06T02:32:38.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.539+0000 7f51e179d700 1 -- 172.21.15.31:0/3113399553 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51d0009420 con 0x7f51dc159fb0 2023-10-06T02:32:38.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.539+0000 7f51c3fff700 1 -- 172.21.15.31:0/3113399553 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d400cbf0 con 0x7f51dc159fb0 2023-10-06T02:32:38.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.539+0000 7f51e179d700 1 -- 172.21.15.31:0/3113399553 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51dc0b0b90 con 0x7f51dc159fb0 2023-10-06T02:32:38.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.540+0000 7f51e179d700 1 -- 172.21.15.31:0/3113399553 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f51dc04c3c0 con 0x7f51dc159fb0 2023-10-06T02:32:38.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.541+0000 7f51c3fff700 1 -- 172.21.15.31:0/3113399553 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) v1 ==== 85709+0+0 (secure 0 0 0) 0x7f51d4003020 con 0x7f51dc159fb0 2023-10-06T02:32:38.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.542+0000 7f51c3fff700 1 --2- 172.21.15.31:0/3113399553 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f51c8067a10 0x7f51c8069ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:38.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.542+0000 7f51c3fff700 1 -- 172.21.15.31:0/3113399553 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5320+0+0 (secure 0 0 0) 0x7f51d4082f70 con 0x7f51dc159fb0 2023-10-06T02:32:38.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.542+0000 7f51c3fff700 1 --2- 172.21.15.31:0/3113399553 >> [v2:172.21.15.31:6800/85567,v1:172.21.15.31:6801/85567] conn(0x7f51c806de80 0x7f51c80702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:38.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.542+0000 7f51c3fff700 1 -- 172.21.15.31:0/3113399553 --> [v2:172.21.15.31:6800/85567,v1:172.21.15.31:6801/85567] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f51c8070980 con 0x7f51c806de80 2023-10-06T02:32:38.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.542+0000 7f51c3fff700 1 -- 172.21.15.31:0/3113399553 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=172) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51d40879b0 con 0x7f51dc159fb0 2023-10-06T02:32:38.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.542+0000 7f51dbfff700 1 --2- 172.21.15.31:0/3113399553 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f51c8067a10 0x7f51c8069ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:38.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.542+0000 7f51db7fe700 1 --2- 172.21.15.31:0/3113399553 >> [v2:172.21.15.31:6800/85567,v1:172.21.15.31:6801/85567] conn(0x7f51c806de80 0x7f51c80702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:38.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.543+0000 7f51dbfff700 1 --2- 172.21.15.31:0/3113399553 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f51c8067a10 0x7f51c8069ec0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f51cc007060 tx=0x7f51cc007ef0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:38.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.543+0000 7f51db7fe700 1 --2- 172.21.15.31:0/3113399553 >> [v2:172.21.15.31:6800/85567,v1:172.21.15.31:6801/85567] conn(0x7f51c806de80 0x7f51c80702b0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:38.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.543+0000 7f51c3fff700 1 -- 172.21.15.31:0/3113399553 <== osd.0 v2:172.21.15.31:6800/85567 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f51c8070980 con 0x7f51c806de80 2023-10-06T02:32:38.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.558+0000 7f51e179d700 1 -- 172.21.15.31:0/3113399553 --> [v2:172.21.15.31:6800/85567,v1:172.21.15.31:6801/85567] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f51dc0b2c40 con 0x7f51c806de80 2023-10-06T02:32:38.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.558+0000 7f51c3fff700 1 -- 172.21.15.31:0/3113399553 <== osd.0 v2:172.21.15.31:6800/85567 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f51dc0b2c40 con 0x7f51c806de80 2023-10-06T02:32:38.560 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T02:32:38.560 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T02:32:38.560 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T02:32:38.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.559+0000 7f51e179d700 1 -- 172.21.15.31:0/3113399553 >> [v2:172.21.15.31:6800/85567,v1:172.21.15.31:6801/85567] conn(0x7f51c806de80 msgr2=0x7f51c80702b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:38.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.559+0000 7f51e179d700 1 --2- 172.21.15.31:0/3113399553 >> [v2:172.21.15.31:6800/85567,v1:172.21.15.31:6801/85567] conn(0x7f51c806de80 0x7f51c80702b0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:38.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.559+0000 7f51e179d700 1 -- 172.21.15.31:0/3113399553 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f51c8067a10 msgr2=0x7f51c8069ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:38.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.559+0000 7f51e179d700 1 --2- 172.21.15.31:0/3113399553 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f51c8067a10 0x7f51c8069ec0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f51cc007060 tx=0x7f51cc007ef0 comp rx=0 tx=0).stop 2023-10-06T02:32:38.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.559+0000 7f51e179d700 1 -- 172.21.15.31:0/3113399553 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51dc159fb0 msgr2=0x7f51dc14e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:38.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.559+0000 7f51e179d700 1 --2- 172.21.15.31:0/3113399553 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51dc159fb0 0x7f51dc14e610 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f51d4009fd0 tx=0x7f51d400b040 comp rx=0 tx=0).stop 2023-10-06T02:32:38.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.559+0000 7f51e0f9c700 1 -- 172.21.15.31:0/3113399553 reap_dead start 2023-10-06T02:32:38.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.559+0000 7f51e179d700 1 -- 172.21.15.31:0/3113399553 shutdown_connections 2023-10-06T02:32:38.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.559+0000 7f51e179d700 1 -- 172.21.15.31:0/3113399553 >> 172.21.15.31:0/3113399553 conn(0x7f51dc0703a0 msgr2=0x7f51dc158fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:38.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.559+0000 7f51e179d700 1 -- 172.21.15.31:0/3113399553 shutdown_connections 2023-10-06T02:32:38.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.559+0000 7f51e179d700 1 -- 172.21.15.31:0/3113399553 wait complete. 2023-10-06T02:32:38.577 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-10-06T02:32:38.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.763+0000 7f644339d700 1 Processor -- start 2023-10-06T02:32:38.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.763+0000 7f644339d700 1 -- start start 2023-10-06T02:32:38.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.765+0000 7f644339d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f643c06aaa0 0x7f643c15a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:38.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.765+0000 7f644339d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f643c15ae70 con 0x7f643c13fef0 2023-10-06T02:32:38.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.765+0000 7f644339d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f643c06af60 con 0x7f643c13fb20 2023-10-06T02:32:38.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.765+0000 7f644339d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f643c06b240 con 0x7f643c06aaa0 2023-10-06T02:32:38.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.765+0000 7f6441b9a700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f643c13fef0 0x7f643c0c9d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:55854/0 (socket says 172.21.15.31:55854) 2023-10-06T02:32:38.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.765+0000 7f6441b9a700 1 -- 172.21.15.31:0/3870103721 learned_addr learned my addr 172.21.15.31:0/3870103721 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:32:38.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.765+0000 7f6442b9c700 1 --2- 172.21.15.31:0/3870103721 >> v2:172.21.15.62:3300/0 conn(0x7f643c06aaa0 0x7f643c15a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:38.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.765+0000 7f6441399700 1 -- 172.21.15.31:0/3870103721 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1669395273 0 0) 0x7f642c001290 con 0x7f643c13fef0 2023-10-06T02:32:38.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.765+0000 7f6441399700 1 -- 172.21.15.31:0/3870103721 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6428003570 con 0x7f643c13fef0 2023-10-06T02:32:38.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.766+0000 7f6441399700 1 -- 172.21.15.31:0/3870103721 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 996323127 0 0) 0x7f643c06af60 con 0x7f643c13fb20 2023-10-06T02:32:38.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.766+0000 7f6441399700 1 -- 172.21.15.31:0/3870103721 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f642c001290 con 0x7f643c13fb20 2023-10-06T02:32:38.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.766+0000 7f6441399700 1 -- 172.21.15.31:0/3870103721 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2639966962 0 0) 0x7f6428003570 con 0x7f643c13fef0 2023-10-06T02:32:38.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.766+0000 7f6442b9c700 1 -- 172.21.15.31:0/3870103721 >> v1:172.21.15.62:6789/0 conn(0x7f643c13fb20 legacy=0x7f643c0c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:38.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.766+0000 7f6442b9c700 1 -- 172.21.15.31:0/3870103721 >> v1:172.21.15.31:6789/0 conn(0x7f643c13fef0 legacy=0x7f643c0c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:38.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.766+0000 7f6442b9c700 1 -- 172.21.15.31:0/3870103721 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f643c06b520 con 0x7f643c06aaa0 2023-10-06T02:32:38.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.766+0000 7f6442b9c700 1 --2- 172.21.15.31:0/3870103721 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f643c06aaa0 0x7f643c15a930 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f6434009e60 tx=0x7f6434009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ed55c0eb7c9c19f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:38.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.766+0000 7f6441399700 1 -- 172.21.15.31:0/3870103721 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f643400e070 con 0x7f643c06aaa0 2023-10-06T02:32:38.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.766+0000 7f6441399700 1 -- 172.21.15.31:0/3870103721 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f643400b7a0 con 0x7f643c06aaa0 2023-10-06T02:32:38.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.766+0000 7f6441399700 1 -- 172.21.15.31:0/3870103721 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f642c001780 con 0x7f643c06aaa0 2023-10-06T02:32:38.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.767+0000 7f644339d700 1 -- 172.21.15.31:0/3870103721 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f643c06aaa0 msgr2=0x7f643c15a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:38.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.767+0000 7f644339d700 1 --2- 172.21.15.31:0/3870103721 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f643c06aaa0 0x7f643c15a930 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f6434009e60 tx=0x7f6434009780 comp rx=0 tx=0).stop 2023-10-06T02:32:38.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.767+0000 7f644339d700 1 -- 172.21.15.31:0/3870103721 shutdown_connections 2023-10-06T02:32:38.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.767+0000 7f644339d700 1 --2- 172.21.15.31:0/3870103721 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f643c06aaa0 0x7f643c15a930 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:38.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.767+0000 7f644339d700 1 -- 172.21.15.31:0/3870103721 >> 172.21.15.31:0/3870103721 conn(0x7f643c06d170 msgr2=0x7f643c0cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:38.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.767+0000 7f644339d700 1 -- 172.21.15.31:0/3870103721 shutdown_connections 2023-10-06T02:32:38.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.767+0000 7f644339d700 1 -- 172.21.15.31:0/3870103721 wait complete. 2023-10-06T02:32:38.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.768+0000 7f644339d700 1 Processor -- start 2023-10-06T02:32:38.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.768+0000 7f644339d700 1 -- start start 2023-10-06T02:32:38.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.769+0000 7f644339d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f643c06aaa0 0x7f643c155ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:38.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.769+0000 7f644339d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f643c13fb20 0x7f643c0ab740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:38.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.769+0000 7f644339d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f643c13fef0 0x7f643c0abc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:38.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.769+0000 7f644339d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f643c159830 con 0x7f643c06aaa0 2023-10-06T02:32:38.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.769+0000 7f644339d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f643c159270 con 0x7f643c13fef0 2023-10-06T02:32:38.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.769+0000 7f644339d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f643c159550 con 0x7f643c13fb20 2023-10-06T02:32:38.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.770+0000 7f644239b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f643c06aaa0 0x7f643c155ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:38.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.770+0000 7f644239b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f643c06aaa0 0x7f643c155ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49942/0 (socket says 172.21.15.31:49942) 2023-10-06T02:32:38.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.770+0000 7f644239b700 1 -- 172.21.15.31:0/1123544847 learned_addr learned my addr 172.21.15.31:0/1123544847 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:38.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.770+0000 7f6441b9a700 1 --2- 172.21.15.31:0/1123544847 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f643c13fb20 0x7f643c0ab740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:38.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.770+0000 7f6442b9c700 1 --2- 172.21.15.31:0/1123544847 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f643c13fef0 0x7f643c0abc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:38.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.770+0000 7f644239b700 1 -- 172.21.15.31:0/1123544847 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f643c13fb20 msgr2=0x7f643c0ab740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:38.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.770+0000 7f644239b700 1 --2- 172.21.15.31:0/1123544847 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f643c13fb20 0x7f643c0ab740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:38.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.770+0000 7f644239b700 1 -- 172.21.15.31:0/1123544847 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f643c13fef0 msgr2=0x7f643c0abc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:38.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.770+0000 7f644239b700 1 --2- 172.21.15.31:0/1123544847 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f643c13fef0 0x7f643c0abc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:38.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.770+0000 7f644239b700 1 -- 172.21.15.31:0/1123544847 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6434009a20 con 0x7f643c06aaa0 2023-10-06T02:32:38.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.770+0000 7f6441b9a700 1 --2- 172.21.15.31:0/1123544847 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f643c13fb20 0x7f643c0ab740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:32:38.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.770+0000 7f6442b9c700 1 --2- 172.21.15.31:0/1123544847 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f643c13fef0 0x7f643c0abc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:32:38.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.770+0000 7f644239b700 1 --2- 172.21.15.31:0/1123544847 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f643c06aaa0 0x7f643c155ad0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f643800c790 tx=0x7f6438007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:38.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.770+0000 7f64327fc700 1 -- 172.21.15.31:0/1123544847 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6438010510 con 0x7f643c06aaa0 2023-10-06T02:32:38.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.770+0000 7f64327fc700 1 -- 172.21.15.31:0/1123544847 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6438010670 con 0x7f643c06aaa0 2023-10-06T02:32:38.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.770+0000 7f64327fc700 1 -- 172.21.15.31:0/1123544847 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6438010a10 con 0x7f643c06aaa0 2023-10-06T02:32:38.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.770+0000 7f644339d700 1 -- 172.21.15.31:0/1123544847 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f643c0af270 con 0x7f643c06aaa0 2023-10-06T02:32:38.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.771+0000 7f644339d700 1 -- 172.21.15.31:0/1123544847 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f643c156a30 con 0x7f643c06aaa0 2023-10-06T02:32:38.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.772+0000 7f644339d700 1 -- 172.21.15.31:0/1123544847 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f643c04c3c0 con 0x7f643c06aaa0 2023-10-06T02:32:38.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.772+0000 7f64327fc700 1 -- 172.21.15.31:0/1123544847 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) v1 ==== 85709+0+0 (secure 0 0 0) 0x7f643801b400 con 0x7f643c06aaa0 2023-10-06T02:32:38.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.773+0000 7f64327fc700 1 --2- 172.21.15.31:0/1123544847 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f64200679e0 0x7f6420069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:38.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.773+0000 7f64327fc700 1 -- 172.21.15.31:0/1123544847 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5320+0+0 (secure 0 0 0) 0x7f6438082d70 con 0x7f643c06aaa0 2023-10-06T02:32:38.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.774+0000 7f6441b9a700 1 --2- 172.21.15.31:0/1123544847 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f64200679e0 0x7f6420069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:38.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.775+0000 7f6441b9a700 1 --2- 172.21.15.31:0/1123544847 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f64200679e0 0x7f6420069e90 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f643c0ca8f0 tx=0x7f642c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:38.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.777+0000 7f64327fc700 1 -- 172.21.15.31:0/1123544847 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f64380519b0 con 0x7f643c06aaa0 2023-10-06T02:32:38.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.978+0000 7f644339d700 1 -- 172.21.15.31:0/1123544847 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f643c06b800 con 0x7f64200679e0 2023-10-06T02:32:38.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.979+0000 7f64327fc700 1 -- 172.21.15.31:0/1123544847 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f643c06b800 con 0x7f64200679e0 2023-10-06T02:32:38.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.981+0000 7f644339d700 1 -- 172.21.15.31:0/1123544847 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f64200679e0 msgr2=0x7f6420069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:38.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.981+0000 7f644339d700 1 --2- 172.21.15.31:0/1123544847 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f64200679e0 0x7f6420069e90 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f643c0ca8f0 tx=0x7f642c009040 comp rx=0 tx=0).stop 2023-10-06T02:32:38.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.981+0000 7f644339d700 1 -- 172.21.15.31:0/1123544847 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f643c06aaa0 msgr2=0x7f643c155ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:38.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.981+0000 7f644339d700 1 --2- 172.21.15.31:0/1123544847 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f643c06aaa0 0x7f643c155ad0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f643800c790 tx=0x7f6438007e10 comp rx=0 tx=0).stop 2023-10-06T02:32:38.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.981+0000 7f644339d700 1 -- 172.21.15.31:0/1123544847 shutdown_connections 2023-10-06T02:32:38.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.981+0000 7f644339d700 1 --2- 172.21.15.31:0/1123544847 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f643c13fef0 0x7f643c0abc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:38.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.981+0000 7f644339d700 1 --2- 172.21.15.31:0/1123544847 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f643c06aaa0 0x7f643c155ad0 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:38.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.981+0000 7f644339d700 1 --2- 172.21.15.31:0/1123544847 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f64200679e0 0x7f6420069e90 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:38.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.981+0000 7f644339d700 1 --2- 172.21.15.31:0/1123544847 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f643c13fb20 0x7f643c0ab740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:38.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.981+0000 7f644339d700 1 -- 172.21.15.31:0/1123544847 >> 172.21.15.31:0/1123544847 conn(0x7f643c06d170 msgr2=0x7f643c0c9280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:38.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.981+0000 7f644339d700 1 -- 172.21.15.31:0/1123544847 shutdown_connections 2023-10-06T02:32:38.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:38.981+0000 7f644339d700 1 -- 172.21.15.31:0/1123544847 wait complete. 2023-10-06T02:32:38.984 INFO:teuthology.orchestra.run.smithi031.stderr:instructed osd(s) 0 to deep-scrub 2023-10-06T02:32:39.000 INFO:tasks.ceph:Scrubbing osd.1 2023-10-06T02:32:39.000 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-10-06T02:32:39.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.189+0000 7f80dcb9d700 1 Processor -- start 2023-10-06T02:32:39.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.189+0000 7f80dcb9d700 1 -- start start 2023-10-06T02:32:39.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.191+0000 7f80dcb9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f80d8157380 0x7f80d81585f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:39.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.191+0000 7f80dcb9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f80d8157750 0x7f80d80ae080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:39.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.191+0000 7f80dcb9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80d80c8680 con 0x7f80d815b1a0 2023-10-06T02:32:39.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.191+0000 7f80dcb9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80d806a7f0 con 0x7f80d8157750 2023-10-06T02:32:39.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.191+0000 7f80dcb9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80d80c7270 con 0x7f80d8157380 2023-10-06T02:32:39.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.191+0000 7f80d7fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f80d8157750 0x7f80d80ae080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:39.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.191+0000 7f80d7fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f80d8157750 0x7f80d80ae080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49948/0 (socket says 172.21.15.31:49948) 2023-10-06T02:32:39.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.191+0000 7f80d7fff700 1 -- 172.21.15.31:0/1047383663 learned_addr learned my addr 172.21.15.31:0/1047383663 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:39.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.191+0000 7f80d77fe700 1 --2- 172.21.15.31:0/1047383663 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f80d8157380 0x7f80d81585f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:39.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.191+0000 7f80d7fff700 1 -- 172.21.15.31:0/1047383663 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f80d8157380 msgr2=0x7f80d81585f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:39.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.191+0000 7f80d7fff700 1 --2- 172.21.15.31:0/1047383663 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f80d8157380 0x7f80d81585f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:39.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.192+0000 7f80d7fff700 1 -- 172.21.15.31:0/1047383663 >> v1:172.21.15.62:6789/0 conn(0x7f80d815b1a0 legacy=0x7f80d80c6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:39.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.192+0000 7f80d7fff700 1 -- 172.21.15.31:0/1047383663 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80d8158b30 con 0x7f80d8157750 2023-10-06T02:32:39.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.192+0000 7f80d7fff700 1 --2- 172.21.15.31:0/1047383663 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80d8157750 0x7f80d80ae080 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f80d0009f20 tx=0x7f80d0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec44f90d9765aae4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:39.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.192+0000 7f80d67fc700 1 -- 172.21.15.31:0/1047383663 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d000e070 con 0x7f80d8157750 2023-10-06T02:32:39.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.192+0000 7f80d67fc700 1 -- 172.21.15.31:0/1047383663 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80d000ba80 con 0x7f80d8157750 2023-10-06T02:32:39.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.192+0000 7f80d67fc700 1 -- 172.21.15.31:0/1047383663 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d00092e0 con 0x7f80d8157750 2023-10-06T02:32:39.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.193+0000 7f80dcb9d700 1 -- 172.21.15.31:0/1047383663 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80d8157750 msgr2=0x7f80d80ae080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:39.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.193+0000 7f80dcb9d700 1 --2- 172.21.15.31:0/1047383663 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80d8157750 0x7f80d80ae080 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f80d0009f20 tx=0x7f80d0009840 comp rx=0 tx=0).stop 2023-10-06T02:32:39.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.193+0000 7f80dcb9d700 1 -- 172.21.15.31:0/1047383663 shutdown_connections 2023-10-06T02:32:39.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.193+0000 7f80dcb9d700 1 --2- 172.21.15.31:0/1047383663 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80d8157750 0x7f80d80ae080 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:39.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.193+0000 7f80dcb9d700 1 --2- 172.21.15.31:0/1047383663 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f80d8157380 0x7f80d81585f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:39.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.193+0000 7f80dcb9d700 1 -- 172.21.15.31:0/1047383663 >> 172.21.15.31:0/1047383663 conn(0x7f80d806c6a0 msgr2=0x7f80d806caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:39.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.193+0000 7f80dcb9d700 1 -- 172.21.15.31:0/1047383663 shutdown_connections 2023-10-06T02:32:39.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.193+0000 7f80dcb9d700 1 -- 172.21.15.31:0/1047383663 wait complete. 2023-10-06T02:32:39.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.194+0000 7f80dcb9d700 1 Processor -- start 2023-10-06T02:32:39.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.194+0000 7f80dcb9d700 1 -- start start 2023-10-06T02:32:39.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.195+0000 7f80dcb9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80d8157380 0x7f80d815aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:39.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.195+0000 7f80dcb9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f80d8157750 0x7f80d8168590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:39.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.195+0000 7f80dcb9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80d815b1a0 0x7f80d816a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:39.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.195+0000 7f80dcb9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80d816a7f0 con 0x7f80d8157380 2023-10-06T02:32:39.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.195+0000 7f80dcb9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80d8167ba0 con 0x7f80d815b1a0 2023-10-06T02:32:39.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.195+0000 7f80dcb9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80d8167e80 con 0x7f80d8157750 2023-10-06T02:32:39.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.196+0000 7f80d77fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80d8157380 0x7f80d815aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:39.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.196+0000 7f80d77fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80d8157380 0x7f80d815aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49952/0 (socket says 172.21.15.31:49952) 2023-10-06T02:32:39.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.196+0000 7f80d77fe700 1 -- 172.21.15.31:0/783622142 learned_addr learned my addr 172.21.15.31:0/783622142 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:39.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.196+0000 7f80d7fff700 1 --2- 172.21.15.31:0/783622142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80d815b1a0 0x7f80d816a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:39.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.196+0000 7f80d6ffd700 1 --2- 172.21.15.31:0/783622142 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f80d8157750 0x7f80d8168590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:39.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.196+0000 7f80d77fe700 1 -- 172.21.15.31:0/783622142 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f80d8157750 msgr2=0x7f80d8168590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:39.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.196+0000 7f80d77fe700 1 --2- 172.21.15.31:0/783622142 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f80d8157750 0x7f80d8168590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:39.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.196+0000 7f80d77fe700 1 -- 172.21.15.31:0/783622142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80d815b1a0 msgr2=0x7f80d816a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:39.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.196+0000 7f80d77fe700 1 --2- 172.21.15.31:0/783622142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80d815b1a0 0x7f80d816a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:39.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.196+0000 7f80d77fe700 1 -- 172.21.15.31:0/783622142 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80d0009ae0 con 0x7f80d8157380 2023-10-06T02:32:39.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.196+0000 7f80d77fe700 1 --2- 172.21.15.31:0/783622142 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80d8157380 0x7f80d815aa90 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f80c800d890 tx=0x7f80c8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:39.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.196+0000 7f80bf7fe700 1 -- 172.21.15.31:0/783622142 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c800b6c0 con 0x7f80d8157380 2023-10-06T02:32:39.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.197+0000 7f80bf7fe700 1 -- 172.21.15.31:0/783622142 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80c800b840 con 0x7f80d8157380 2023-10-06T02:32:39.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.197+0000 7f80dcb9d700 1 -- 172.21.15.31:0/783622142 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80d8168160 con 0x7f80d8157380 2023-10-06T02:32:39.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.197+0000 7f80bf7fe700 1 -- 172.21.15.31:0/783622142 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c800bc00 con 0x7f80d8157380 2023-10-06T02:32:39.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.197+0000 7f80dcb9d700 1 -- 172.21.15.31:0/783622142 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80d81673c0 con 0x7f80d8157380 2023-10-06T02:32:39.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.198+0000 7f80dcb9d700 1 -- 172.21.15.31:0/783622142 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f80d8158d20 con 0x7f80d8157380 2023-10-06T02:32:39.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.198+0000 7f80bf7fe700 1 -- 172.21.15.31:0/783622142 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) v1 ==== 85709+0+0 (secure 0 0 0) 0x7f80c801e460 con 0x7f80d8157380 2023-10-06T02:32:39.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.199+0000 7f80bf7fe700 1 --2- 172.21.15.31:0/783622142 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f80c40679e0 0x7f80c4069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:39.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.199+0000 7f80bf7fe700 1 -- 172.21.15.31:0/783622142 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5320+0+0 (secure 0 0 0) 0x7f80c80836e0 con 0x7f80d8157380 2023-10-06T02:32:39.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.200+0000 7f80bf7fe700 1 --2- 172.21.15.31:0/783622142 >> [v2:172.21.15.31:6808/85568,v1:172.21.15.31:6809/85568] conn(0x7f80c406de50 0x7f80c4070280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:39.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.200+0000 7f80bf7fe700 1 -- 172.21.15.31:0/783622142 --> [v2:172.21.15.31:6808/85568,v1:172.21.15.31:6809/85568] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f80c4070950 con 0x7f80c406de50 2023-10-06T02:32:39.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.200+0000 7f80bf7fe700 1 -- 172.21.15.31:0/783622142 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=172) v2 ==== 24+0+0 (secure 0 0 0) 0x7f80c8088420 con 0x7f80d8157380 2023-10-06T02:32:39.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.200+0000 7f80d6ffd700 1 --2- 172.21.15.31:0/783622142 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f80c40679e0 0x7f80c4069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:39.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.200+0000 7f80d7fff700 1 --2- 172.21.15.31:0/783622142 >> [v2:172.21.15.31:6808/85568,v1:172.21.15.31:6809/85568] conn(0x7f80c406de50 0x7f80c4070280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:39.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.200+0000 7f80d7fff700 1 --2- 172.21.15.31:0/783622142 >> [v2:172.21.15.31:6808/85568,v1:172.21.15.31:6809/85568] conn(0x7f80c406de50 0x7f80c4070280 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:39.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.200+0000 7f80d6ffd700 1 --2- 172.21.15.31:0/783622142 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f80c40679e0 0x7f80c4069e90 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f80d806ef60 tx=0x7f80cc009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:39.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.201+0000 7f80bf7fe700 1 -- 172.21.15.31:0/783622142 <== osd.1 v2:172.21.15.31:6808/85568 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f80c4070950 con 0x7f80c406de50 2023-10-06T02:32:39.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.216+0000 7f80dcb9d700 1 -- 172.21.15.31:0/783622142 --> [v2:172.21.15.31:6808/85568,v1:172.21.15.31:6809/85568] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f80d804c3c0 con 0x7f80c406de50 2023-10-06T02:32:39.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.216+0000 7f80bf7fe700 1 -- 172.21.15.31:0/783622142 <== osd.1 v2:172.21.15.31:6808/85568 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f80d804c3c0 con 0x7f80c406de50 2023-10-06T02:32:39.218 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T02:32:39.218 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T02:32:39.218 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T02:32:39.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.216+0000 7f80dcb9d700 1 -- 172.21.15.31:0/783622142 >> [v2:172.21.15.31:6808/85568,v1:172.21.15.31:6809/85568] conn(0x7f80c406de50 msgr2=0x7f80c4070280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:39.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.216+0000 7f80dcb9d700 1 --2- 172.21.15.31:0/783622142 >> [v2:172.21.15.31:6808/85568,v1:172.21.15.31:6809/85568] conn(0x7f80c406de50 0x7f80c4070280 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:39.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.216+0000 7f80dcb9d700 1 -- 172.21.15.31:0/783622142 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f80c40679e0 msgr2=0x7f80c4069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:39.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.216+0000 7f80dcb9d700 1 --2- 172.21.15.31:0/783622142 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f80c40679e0 0x7f80c4069e90 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f80d806ef60 tx=0x7f80cc009040 comp rx=0 tx=0).stop 2023-10-06T02:32:39.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.216+0000 7f80dcb9d700 1 -- 172.21.15.31:0/783622142 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80d8157380 msgr2=0x7f80d815aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:39.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.216+0000 7f80dcb9d700 1 --2- 172.21.15.31:0/783622142 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80d8157380 0x7f80d815aa90 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f80c800d890 tx=0x7f80c8008040 comp rx=0 tx=0).stop 2023-10-06T02:32:39.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.216+0000 7f80d7fff700 1 -- 172.21.15.31:0/783622142 reap_dead start 2023-10-06T02:32:39.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.217+0000 7f80dcb9d700 1 -- 172.21.15.31:0/783622142 shutdown_connections 2023-10-06T02:32:39.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.217+0000 7f80dcb9d700 1 -- 172.21.15.31:0/783622142 >> 172.21.15.31:0/783622142 conn(0x7f80d806c6a0 msgr2=0x7f80d806e680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:39.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.217+0000 7f80dcb9d700 1 -- 172.21.15.31:0/783622142 shutdown_connections 2023-10-06T02:32:39.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.217+0000 7f80dcb9d700 1 -- 172.21.15.31:0/783622142 wait complete. 2023-10-06T02:32:39.234 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-10-06T02:32:39.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.421+0000 7fbadd99d700 1 Processor -- start 2023-10-06T02:32:39.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.421+0000 7fbadd99d700 1 -- start start 2023-10-06T02:32:39.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.422+0000 7fbadd99d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fbad815de50 0x7fbad80cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:39.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.422+0000 7fbadd99d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fbad80cef40 0x7fbad815c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:39.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.422+0000 7fbadd99d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbad80cf3c0 con 0x7fbad8153c30 2023-10-06T02:32:39.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.422+0000 7fbadd99d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbad815afd0 con 0x7fbad815de50 2023-10-06T02:32:39.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.422+0000 7fbadd99d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbad815b2b0 con 0x7fbad80cef40 2023-10-06T02:32:39.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.423+0000 7fbad7fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fbad815de50 0x7fbad80cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:39.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.423+0000 7fbad7fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fbad815de50 0x7fbad80cea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49956/0 (socket says 172.21.15.31:49956) 2023-10-06T02:32:39.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.423+0000 7fbad7fff700 1 -- 172.21.15.31:0/2182980576 learned_addr learned my addr 172.21.15.31:0/2182980576 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:39.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.423+0000 7fbadd19c700 1 --2- 172.21.15.31:0/2182980576 >> v2:172.21.15.62:3300/0 conn(0x7fbad80cef40 0x7fbad815c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:39.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.423+0000 7fbad7fff700 1 -- 172.21.15.31:0/2182980576 >> v2:172.21.15.62:3300/0 conn(0x7fbad80cef40 msgr2=0x7fbad815c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:39.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.423+0000 7fbad7fff700 1 --2- 172.21.15.31:0/2182980576 >> v2:172.21.15.62:3300/0 conn(0x7fbad80cef40 0x7fbad815c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:39.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.423+0000 7fbad7fff700 1 -- 172.21.15.31:0/2182980576 >> v1:172.21.15.62:6789/0 conn(0x7fbad8153c30 legacy=0x7fbad815a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:39.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.423+0000 7fbad7fff700 1 -- 172.21.15.31:0/2182980576 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbad815b590 con 0x7fbad815de50 2023-10-06T02:32:39.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.423+0000 7fbad7fff700 1 --2- 172.21.15.31:0/2182980576 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbad815de50 0x7fbad80cea00 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7fbad80cdab0 tx=0x7fbac4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d566fbc5e29e849f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:39.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.423+0000 7fbad77fe700 1 -- 172.21.15.31:0/2182980576 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac400b820 con 0x7fbad815de50 2023-10-06T02:32:39.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.424+0000 7fbad77fe700 1 -- 172.21.15.31:0/2182980576 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbac400b980 con 0x7fbad815de50 2023-10-06T02:32:39.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.424+0000 7fbad77fe700 1 -- 172.21.15.31:0/2182980576 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac400bd20 con 0x7fbad815de50 2023-10-06T02:32:39.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.424+0000 7fbadd99d700 1 -- 172.21.15.31:0/2182980576 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbad815de50 msgr2=0x7fbad80cea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:39.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.424+0000 7fbadd99d700 1 --2- 172.21.15.31:0/2182980576 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbad815de50 0x7fbad80cea00 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7fbad80cdab0 tx=0x7fbac4009250 comp rx=0 tx=0).stop 2023-10-06T02:32:39.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.424+0000 7fbadd99d700 1 -- 172.21.15.31:0/2182980576 shutdown_connections 2023-10-06T02:32:39.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.424+0000 7fbadd99d700 1 --2- 172.21.15.31:0/2182980576 >> v2:172.21.15.62:3300/0 conn(0x7fbad80cef40 0x7fbad815c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:39.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.424+0000 7fbadd99d700 1 --2- 172.21.15.31:0/2182980576 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbad815de50 0x7fbad80cea00 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:39.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.424+0000 7fbadd99d700 1 -- 172.21.15.31:0/2182980576 >> 172.21.15.31:0/2182980576 conn(0x7fbad80703a0 msgr2=0x7fbad80707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:39.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.424+0000 7fbadd99d700 1 -- 172.21.15.31:0/2182980576 shutdown_connections 2023-10-06T02:32:39.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.424+0000 7fbadd99d700 1 -- 172.21.15.31:0/2182980576 wait complete. 2023-10-06T02:32:39.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.426+0000 7fbadd99d700 1 Processor -- start 2023-10-06T02:32:39.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.426+0000 7fbadd99d700 1 -- start start 2023-10-06T02:32:39.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.427+0000 7fbadd99d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbad80cef40 0x7fbad80b32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:39.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.427+0000 7fbadd99d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbad8153c30 0x7fbad80d0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:39.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.427+0000 7fbadd99d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbad815de50 0x7fbad8150030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:39.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.427+0000 7fbadd99d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbad80ca3d0 con 0x7fbad8153c30 2023-10-06T02:32:39.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.427+0000 7fbadd99d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbad80b4e40 con 0x7fbad80cef40 2023-10-06T02:32:39.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.427+0000 7fbadd99d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbad80b3810 con 0x7fbad815de50 2023-10-06T02:32:39.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.427+0000 7fbad7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbad8153c30 0x7fbad80d0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:39.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.427+0000 7fbad7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbad8153c30 0x7fbad80d0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49960/0 (socket says 172.21.15.31:49960) 2023-10-06T02:32:39.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.427+0000 7fbad7fff700 1 -- 172.21.15.31:0/3583886007 learned_addr learned my addr 172.21.15.31:0/3583886007 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:39.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.427+0000 7fbadd19c700 1 --2- 172.21.15.31:0/3583886007 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbad815de50 0x7fbad8150030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:39.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.427+0000 7fbad7fff700 1 -- 172.21.15.31:0/3583886007 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbad815de50 msgr2=0x7fbad8150030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:39.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.427+0000 7fbad7fff700 1 --2- 172.21.15.31:0/3583886007 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbad815de50 0x7fbad8150030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:39.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.427+0000 7fbad7fff700 1 -- 172.21.15.31:0/3583886007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbad80cef40 msgr2=0x7fbad80b32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:39.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.427+0000 7fbadc99b700 1 --2- 172.21.15.31:0/3583886007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbad80cef40 0x7fbad80b32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:39.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.427+0000 7fbad7fff700 1 --2- 172.21.15.31:0/3583886007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbad80cef40 0x7fbad80b32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:39.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.427+0000 7fbad7fff700 1 -- 172.21.15.31:0/3583886007 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbac40094f0 con 0x7fbad8153c30 2023-10-06T02:32:39.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.427+0000 7fbadc99b700 1 --2- 172.21.15.31:0/3583886007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbad80cef40 0x7fbad80b32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:32:39.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.428+0000 7fbad7fff700 1 --2- 172.21.15.31:0/3583886007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbad8153c30 0x7fbad80d0610 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7fbac4009290 tx=0x7fbac4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:39.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.428+0000 7fbad4ff9700 1 -- 172.21.15.31:0/3583886007 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac4006bc0 con 0x7fbad8153c30 2023-10-06T02:32:39.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.428+0000 7fbad4ff9700 1 -- 172.21.15.31:0/3583886007 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbac4011e60 con 0x7fbad8153c30 2023-10-06T02:32:39.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.428+0000 7fbad4ff9700 1 -- 172.21.15.31:0/3583886007 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac401c8c0 con 0x7fbad8153c30 2023-10-06T02:32:39.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.428+0000 7fbadd99d700 1 -- 172.21.15.31:0/3583886007 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbad80ca0f0 con 0x7fbad8153c30 2023-10-06T02:32:39.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.428+0000 7fbadd99d700 1 -- 172.21.15.31:0/3583886007 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbad80c75e0 con 0x7fbad8153c30 2023-10-06T02:32:39.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.429+0000 7fbadd99d700 1 -- 172.21.15.31:0/3583886007 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbad80b4fb0 con 0x7fbad8153c30 2023-10-06T02:32:39.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.430+0000 7fbad4ff9700 1 -- 172.21.15.31:0/3583886007 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) v1 ==== 85709+0+0 (secure 0 0 0) 0x7fbac401ccb0 con 0x7fbad8153c30 2023-10-06T02:32:39.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.431+0000 7fbad4ff9700 1 --2- 172.21.15.31:0/3583886007 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fbabc0679e0 0x7fbabc069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:39.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.431+0000 7fbad4ff9700 1 -- 172.21.15.31:0/3583886007 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5320+0+0 (secure 0 0 0) 0x7fbac40857d0 con 0x7fbad8153c30 2023-10-06T02:32:39.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.432+0000 7fbadc99b700 1 --2- 172.21.15.31:0/3583886007 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fbabc0679e0 0x7fbabc069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:39.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.432+0000 7fbadc99b700 1 --2- 172.21.15.31:0/3583886007 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fbabc0679e0 0x7fbabc069e90 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fbacc006fd0 tx=0x7fbacc008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:39.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.436+0000 7fbad4ff9700 1 -- 172.21.15.31:0/3583886007 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbad80b4fb0 con 0x7fbad8153c30 2023-10-06T02:32:39.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.636+0000 7fbadd99d700 1 -- 172.21.15.31:0/3583886007 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7fbad80c77d0 con 0x7fbabc0679e0 2023-10-06T02:32:39.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.637+0000 7fbad4ff9700 1 -- 172.21.15.31:0/3583886007 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fbad80c77d0 con 0x7fbabc0679e0 2023-10-06T02:32:39.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.639+0000 7fbadd99d700 1 -- 172.21.15.31:0/3583886007 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fbabc0679e0 msgr2=0x7fbabc069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:39.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.639+0000 7fbadd99d700 1 --2- 172.21.15.31:0/3583886007 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fbabc0679e0 0x7fbabc069e90 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fbacc006fd0 tx=0x7fbacc008040 comp rx=0 tx=0).stop 2023-10-06T02:32:39.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.639+0000 7fbadd99d700 1 -- 172.21.15.31:0/3583886007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbad8153c30 msgr2=0x7fbad80d0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:39.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.639+0000 7fbadd99d700 1 --2- 172.21.15.31:0/3583886007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbad8153c30 0x7fbad80d0610 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7fbac4009290 tx=0x7fbac4016000 comp rx=0 tx=0).stop 2023-10-06T02:32:39.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.639+0000 7fbadd99d700 1 -- 172.21.15.31:0/3583886007 shutdown_connections 2023-10-06T02:32:39.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.639+0000 7fbadd99d700 1 --2- 172.21.15.31:0/3583886007 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fbabc0679e0 0x7fbabc069e90 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:39.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.639+0000 7fbadd99d700 1 --2- 172.21.15.31:0/3583886007 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbad815de50 0x7fbad8150030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:39.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.639+0000 7fbadd99d700 1 --2- 172.21.15.31:0/3583886007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbad80cef40 0x7fbad80b32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:39.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.639+0000 7fbadd99d700 1 --2- 172.21.15.31:0/3583886007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbad8153c30 0x7fbad80d0610 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:39.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.639+0000 7fbadd99d700 1 -- 172.21.15.31:0/3583886007 >> 172.21.15.31:0/3583886007 conn(0x7fbad80703a0 msgr2=0x7fbad815c6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:39.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.639+0000 7fbadd99d700 1 -- 172.21.15.31:0/3583886007 shutdown_connections 2023-10-06T02:32:39.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.639+0000 7fbadd99d700 1 -- 172.21.15.31:0/3583886007 wait complete. 2023-10-06T02:32:39.643 INFO:teuthology.orchestra.run.smithi031.stderr:instructed osd(s) 1 to deep-scrub 2023-10-06T02:32:39.659 INFO:tasks.ceph:Scrubbing osd.2 2023-10-06T02:32:39.659 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-10-06T02:32:39.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.849+0000 7ff26359d700 1 Processor -- start 2023-10-06T02:32:39.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.850+0000 7ff26359d700 1 -- start start 2023-10-06T02:32:39.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.851+0000 7ff26359d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff25c0c2a90 0x7ff25c156aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:39.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.851+0000 7ff26359d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff25c0c85d0 0x7ff25c0cc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:39.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.851+0000 7ff26359d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff25c1579b0 con 0x7ff25c0c8200 2023-10-06T02:32:39.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.851+0000 7ff26359d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff25c0c1390 con 0x7ff25c0c2a90 2023-10-06T02:32:39.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.851+0000 7ff26359d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff25c1573f0 con 0x7ff25c0c85d0 2023-10-06T02:32:39.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.851+0000 7ff26259b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7ff25c0c8200 0x7ff25c0c1820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:55866/0 (socket says 172.21.15.31:55866) 2023-10-06T02:32:39.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.851+0000 7ff261d9a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff25c0c2a90 0x7ff25c156aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:39.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.851+0000 7ff26259b700 1 -- 172.21.15.31:0/2368411406 learned_addr learned my addr 172.21.15.31:0/2368411406 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:32:39.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.852+0000 7ff262d9c700 1 --2- 172.21.15.31:0/2368411406 >> v2:172.21.15.62:3300/0 conn(0x7ff25c0c85d0 0x7ff25c0cc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:39.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.852+0000 7ff261d9a700 1 -- 172.21.15.31:0/2368411406 >> v2:172.21.15.62:3300/0 conn(0x7ff25c0c85d0 msgr2=0x7ff25c0cc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:39.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.852+0000 7ff261d9a700 1 --2- 172.21.15.31:0/2368411406 >> v2:172.21.15.62:3300/0 conn(0x7ff25c0c85d0 0x7ff25c0cc2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:39.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.852+0000 7ff261d9a700 1 -- 172.21.15.31:0/2368411406 >> v1:172.21.15.31:6789/0 conn(0x7ff25c0c8200 legacy=0x7ff25c0c1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:39.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.852+0000 7ff261d9a700 1 -- 172.21.15.31:0/2368411406 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff25c1576d0 con 0x7ff25c0c2a90 2023-10-06T02:32:39.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.852+0000 7ff261d9a700 1 --2- 172.21.15.31:0/2368411406 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff25c0c2a90 0x7ff25c156aa0 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7ff25c0c1300 tx=0x7ff24c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d6edd1d5d33bb86 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:39.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.852+0000 7ff261599700 1 -- 172.21.15.31:0/2368411406 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff24c00b6b0 con 0x7ff25c0c2a90 2023-10-06T02:32:39.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.852+0000 7ff261599700 1 -- 172.21.15.31:0/2368411406 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff24c00b810 con 0x7ff25c0c2a90 2023-10-06T02:32:39.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.852+0000 7ff261599700 1 -- 172.21.15.31:0/2368411406 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff24c00bbb0 con 0x7ff25c0c2a90 2023-10-06T02:32:39.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.853+0000 7ff26359d700 1 -- 172.21.15.31:0/2368411406 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff25c0c2a90 msgr2=0x7ff25c156aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:39.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.853+0000 7ff26359d700 1 --2- 172.21.15.31:0/2368411406 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff25c0c2a90 0x7ff25c156aa0 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7ff25c0c1300 tx=0x7ff24c0091a0 comp rx=0 tx=0).stop 2023-10-06T02:32:39.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.853+0000 7ff26359d700 1 -- 172.21.15.31:0/2368411406 shutdown_connections 2023-10-06T02:32:39.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.853+0000 7ff26359d700 1 --2- 172.21.15.31:0/2368411406 >> v2:172.21.15.62:3300/0 conn(0x7ff25c0c85d0 0x7ff25c0cc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:39.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.853+0000 7ff26359d700 1 --2- 172.21.15.31:0/2368411406 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff25c0c2a90 0x7ff25c156aa0 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:39.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.853+0000 7ff26359d700 1 -- 172.21.15.31:0/2368411406 >> 172.21.15.31:0/2368411406 conn(0x7ff25c06c6a0 msgr2=0x7ff25c06caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:39.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.853+0000 7ff26359d700 1 -- 172.21.15.31:0/2368411406 shutdown_connections 2023-10-06T02:32:39.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.853+0000 7ff26359d700 1 -- 172.21.15.31:0/2368411406 wait complete. 2023-10-06T02:32:39.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.854+0000 7ff26359d700 1 Processor -- start 2023-10-06T02:32:39.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.855+0000 7ff26359d700 1 -- start start 2023-10-06T02:32:39.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.855+0000 7ff26359d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff25c0c2a90 0x7ff25c0b01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:39.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.855+0000 7ff26359d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff25c0c8200 0x7ff25c13f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:39.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.855+0000 7ff26359d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff25c0c85d0 0x7ff25c0ac470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:39.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.855+0000 7ff26359d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff25c144ec0 con 0x7ff25c0c8200 2023-10-06T02:32:39.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.855+0000 7ff26359d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff25c0ab740 con 0x7ff25c0c2a90 2023-10-06T02:32:39.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.855+0000 7ff26359d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff25c0abfe0 con 0x7ff25c0c85d0 2023-10-06T02:32:39.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.856+0000 7ff261d9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff25c0c8200 0x7ff25c13f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:39.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.856+0000 7ff261d9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff25c0c8200 0x7ff25c13f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49982/0 (socket says 172.21.15.31:49982) 2023-10-06T02:32:39.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.856+0000 7ff261d9a700 1 -- 172.21.15.31:0/545902554 learned_addr learned my addr 172.21.15.31:0/545902554 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:39.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.856+0000 7ff26259b700 1 --2- 172.21.15.31:0/545902554 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff25c0c2a90 0x7ff25c0b01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:39.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.856+0000 7ff262d9c700 1 --2- 172.21.15.31:0/545902554 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff25c0c85d0 0x7ff25c0ac470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:39.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.856+0000 7ff261d9a700 1 -- 172.21.15.31:0/545902554 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff25c0c85d0 msgr2=0x7ff25c0ac470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:39.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.856+0000 7ff261d9a700 1 --2- 172.21.15.31:0/545902554 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff25c0c85d0 0x7ff25c0ac470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:39.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.856+0000 7ff261d9a700 1 -- 172.21.15.31:0/545902554 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff25c0c2a90 msgr2=0x7ff25c0b01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:39.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.856+0000 7ff261d9a700 1 --2- 172.21.15.31:0/545902554 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff25c0c2a90 0x7ff25c0b01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:39.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.856+0000 7ff261d9a700 1 -- 172.21.15.31:0/545902554 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff24c009420 con 0x7ff25c0c8200 2023-10-06T02:32:39.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.856+0000 7ff261d9a700 1 --2- 172.21.15.31:0/545902554 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff25c0c8200 0x7ff25c13f9f0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7ff24c013000 tx=0x7ff24c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:39.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.857+0000 7ff2527fc700 1 -- 172.21.15.31:0/545902554 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff24c011420 con 0x7ff25c0c8200 2023-10-06T02:32:39.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.857+0000 7ff2527fc700 1 -- 172.21.15.31:0/545902554 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff24c011580 con 0x7ff25c0c8200 2023-10-06T02:32:39.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.857+0000 7ff2527fc700 1 -- 172.21.15.31:0/545902554 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff24c011940 con 0x7ff25c0c8200 2023-10-06T02:32:39.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.857+0000 7ff26359d700 1 -- 172.21.15.31:0/545902554 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff25c0b1190 con 0x7ff25c0c8200 2023-10-06T02:32:39.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.857+0000 7ff26359d700 1 -- 172.21.15.31:0/545902554 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff25c0ab990 con 0x7ff25c0c8200 2023-10-06T02:32:39.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.857+0000 7ff26359d700 1 -- 172.21.15.31:0/545902554 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff25c04c3c0 con 0x7ff25c0c8200 2023-10-06T02:32:39.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.859+0000 7ff2527fc700 1 -- 172.21.15.31:0/545902554 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) v1 ==== 85709+0+0 (secure 0 0 0) 0x7ff24c011d30 con 0x7ff25c0c8200 2023-10-06T02:32:39.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.859+0000 7ff2527fc700 1 --2- 172.21.15.31:0/545902554 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff2400679e0 0x7ff240069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:39.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.859+0000 7ff2527fc700 1 -- 172.21.15.31:0/545902554 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5320+0+0 (secure 0 0 0) 0x7ff24c084650 con 0x7ff25c0c8200 2023-10-06T02:32:39.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.860+0000 7ff2527fc700 1 --2- 172.21.15.31:0/545902554 >> [v2:172.21.15.31:6822/85579,v1:172.21.15.31:6825/85579] conn(0x7ff24006de50 0x7ff240070280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:39.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.860+0000 7ff2527fc700 1 -- 172.21.15.31:0/545902554 --> [v2:172.21.15.31:6822/85579,v1:172.21.15.31:6825/85579] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff240070950 con 0x7ff24006de50 2023-10-06T02:32:39.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.860+0000 7ff2527fc700 1 -- 172.21.15.31:0/545902554 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=172) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff24c089390 con 0x7ff25c0c8200 2023-10-06T02:32:39.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.860+0000 7ff26259b700 1 --2- 172.21.15.31:0/545902554 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff2400679e0 0x7ff240069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:39.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.860+0000 7ff262d9c700 1 --2- 172.21.15.31:0/545902554 >> [v2:172.21.15.31:6822/85579,v1:172.21.15.31:6825/85579] conn(0x7ff24006de50 0x7ff240070280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:39.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.860+0000 7ff262d9c700 1 --2- 172.21.15.31:0/545902554 >> [v2:172.21.15.31:6822/85579,v1:172.21.15.31:6825/85579] conn(0x7ff24006de50 0x7ff240070280 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:39.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.861+0000 7ff26259b700 1 --2- 172.21.15.31:0/545902554 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff2400679e0 0x7ff240069e90 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7ff258001820 tx=0x7ff258009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:39.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.861+0000 7ff2527fc700 1 -- 172.21.15.31:0/545902554 <== osd.2 v2:172.21.15.31:6822/85579 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7ff240070950 con 0x7ff24006de50 2023-10-06T02:32:39.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.876+0000 7ff26359d700 1 -- 172.21.15.31:0/545902554 --> [v2:172.21.15.31:6822/85579,v1:172.21.15.31:6825/85579] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7ff25c0c2280 con 0x7ff24006de50 2023-10-06T02:32:39.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.876+0000 7ff2527fc700 1 -- 172.21.15.31:0/545902554 <== osd.2 v2:172.21.15.31:6822/85579 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7ff25c0c2280 con 0x7ff24006de50 2023-10-06T02:32:39.878 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T02:32:39.878 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T02:32:39.878 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T02:32:39.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.876+0000 7ff26359d700 1 -- 172.21.15.31:0/545902554 >> [v2:172.21.15.31:6822/85579,v1:172.21.15.31:6825/85579] conn(0x7ff24006de50 msgr2=0x7ff240070280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:39.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.876+0000 7ff26359d700 1 --2- 172.21.15.31:0/545902554 >> [v2:172.21.15.31:6822/85579,v1:172.21.15.31:6825/85579] conn(0x7ff24006de50 0x7ff240070280 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:39.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.877+0000 7ff26359d700 1 -- 172.21.15.31:0/545902554 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff2400679e0 msgr2=0x7ff240069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:39.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.877+0000 7ff26359d700 1 --2- 172.21.15.31:0/545902554 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff2400679e0 0x7ff240069e90 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7ff258001820 tx=0x7ff258009040 comp rx=0 tx=0).stop 2023-10-06T02:32:39.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.877+0000 7ff26359d700 1 -- 172.21.15.31:0/545902554 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff25c0c8200 msgr2=0x7ff25c13f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:39.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.877+0000 7ff26359d700 1 --2- 172.21.15.31:0/545902554 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff25c0c8200 0x7ff25c13f9f0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7ff24c013000 tx=0x7ff24c017000 comp rx=0 tx=0).stop 2023-10-06T02:32:39.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.877+0000 7ff262d9c700 1 -- 172.21.15.31:0/545902554 reap_dead start 2023-10-06T02:32:39.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.877+0000 7ff26359d700 1 -- 172.21.15.31:0/545902554 shutdown_connections 2023-10-06T02:32:39.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.877+0000 7ff26359d700 1 -- 172.21.15.31:0/545902554 >> 172.21.15.31:0/545902554 conn(0x7ff25c06c6a0 msgr2=0x7ff25c149eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:39.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.877+0000 7ff26359d700 1 -- 172.21.15.31:0/545902554 shutdown_connections 2023-10-06T02:32:39.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:39.877+0000 7ff26359d700 1 -- 172.21.15.31:0/545902554 wait complete. 2023-10-06T02:32:39.894 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-10-06T02:32:40.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.080+0000 7f7c2499d700 1 Processor -- start 2023-10-06T02:32:40.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.080+0000 7f7c2499d700 1 -- start start 2023-10-06T02:32:40.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.081+0000 7f7c2499d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7c20140a70 0x7f7c200c6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:40.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.081+0000 7f7c2499d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c200c6d50 con 0x7f7c2006de50 2023-10-06T02:32:40.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.081+0000 7f7c2499d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c200c7030 con 0x7f7c20140e40 2023-10-06T02:32:40.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.081+0000 7f7c2499d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c200c7310 con 0x7f7c20140a70 2023-10-06T02:32:40.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.082+0000 7f7c1ffff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f7c2006de50 0x7f7c200aeef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:55872/0 (socket says 172.21.15.31:55872) 2023-10-06T02:32:40.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.082+0000 7f7c1ffff700 1 -- 172.21.15.31:0/1072164055 learned_addr learned my addr 172.21.15.31:0/1072164055 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:32:40.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.082+0000 7f7c1f7fe700 1 --2- 172.21.15.31:0/1072164055 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7c20140a70 0x7f7c200c6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:40.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.082+0000 7f7c1e7fc700 1 -- 172.21.15.31:0/1072164055 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2497661038 0 0) 0x7f7c200c6d50 con 0x7f7c2006de50 2023-10-06T02:32:40.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.082+0000 7f7c1e7fc700 1 -- 172.21.15.31:0/1072164055 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c08003570 con 0x7f7c2006de50 2023-10-06T02:32:40.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.082+0000 7f7c1e7fc700 1 -- 172.21.15.31:0/1072164055 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3510019688 0 0) 0x7f7c200c7030 con 0x7f7c20140e40 2023-10-06T02:32:40.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.082+0000 7f7c1e7fc700 1 -- 172.21.15.31:0/1072164055 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c200c6d50 con 0x7f7c20140e40 2023-10-06T02:32:40.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.082+0000 7f7c1f7fe700 1 -- 172.21.15.31:0/1072164055 >> v1:172.21.15.62:6789/0 conn(0x7f7c20140e40 legacy=0x7f7c200c7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:40.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.082+0000 7f7c1f7fe700 1 -- 172.21.15.31:0/1072164055 >> v1:172.21.15.31:6789/0 conn(0x7f7c2006de50 legacy=0x7f7c200aeef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:40.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.082+0000 7f7c1f7fe700 1 -- 172.21.15.31:0/1072164055 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c200c75f0 con 0x7f7c20140a70 2023-10-06T02:32:40.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.083+0000 7f7c1f7fe700 1 --2- 172.21.15.31:0/1072164055 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7c20140a70 0x7f7c200c6770 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7c10009910 tx=0x7f7c100091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ee5bbfd7d9cfb1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:40.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.083+0000 7f7c1e7fc700 1 -- 172.21.15.31:0/1072164055 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c10011630 con 0x7f7c20140a70 2023-10-06T02:32:40.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.083+0000 7f7c1e7fc700 1 -- 172.21.15.31:0/1072164055 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c10011790 con 0x7f7c20140a70 2023-10-06T02:32:40.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.083+0000 7f7c1e7fc700 1 -- 172.21.15.31:0/1072164055 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c10011b30 con 0x7f7c20140a70 2023-10-06T02:32:40.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.084+0000 7f7c2499d700 1 -- 172.21.15.31:0/1072164055 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7c20140a70 msgr2=0x7f7c200c6770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:40.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.084+0000 7f7c2499d700 1 --2- 172.21.15.31:0/1072164055 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7c20140a70 0x7f7c200c6770 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7c10009910 tx=0x7f7c100091a0 comp rx=0 tx=0).stop 2023-10-06T02:32:40.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.084+0000 7f7c2499d700 1 -- 172.21.15.31:0/1072164055 shutdown_connections 2023-10-06T02:32:40.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.084+0000 7f7c2499d700 1 --2- 172.21.15.31:0/1072164055 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7c20140a70 0x7f7c200c6770 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:40.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.084+0000 7f7c2499d700 1 -- 172.21.15.31:0/1072164055 >> 172.21.15.31:0/1072164055 conn(0x7f7c2006d170 msgr2=0x7f7c200cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:40.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.084+0000 7f7c2499d700 1 -- 172.21.15.31:0/1072164055 shutdown_connections 2023-10-06T02:32:40.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.084+0000 7f7c2499d700 1 -- 172.21.15.31:0/1072164055 wait complete. 2023-10-06T02:32:40.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.085+0000 7f7c2499d700 1 Processor -- start 2023-10-06T02:32:40.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.085+0000 7f7c2499d700 1 -- start start 2023-10-06T02:32:40.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.086+0000 7f7c2499d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7c2006de50 0x7f7c20156b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:40.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.086+0000 7f7c2499d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c20140a70 0x7f7c20159a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:40.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.086+0000 7f7c2499d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c20140e40 0x7f7c20159f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:40.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.086+0000 7f7c2499d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c200c60f0 con 0x7f7c20140e40 2023-10-06T02:32:40.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.086+0000 7f7c2499d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c200c54e0 con 0x7f7c20140a70 2023-10-06T02:32:40.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.086+0000 7f7c2499d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c200c57c0 con 0x7f7c2006de50 2023-10-06T02:32:40.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.086+0000 7f7c1ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c20140e40 0x7f7c20159f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:40.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.087+0000 7f7c1ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c20140e40 0x7f7c20159f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49990/0 (socket says 172.21.15.31:49990) 2023-10-06T02:32:40.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.087+0000 7f7c1ffff700 1 -- 172.21.15.31:0/2055984437 learned_addr learned my addr 172.21.15.31:0/2055984437 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:40.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.087+0000 7f7c1effd700 1 --2- 172.21.15.31:0/2055984437 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c20140a70 0x7f7c20159a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:40.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.087+0000 7f7c1f7fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7c2006de50 0x7f7c20156b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:40.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.087+0000 7f7c1ffff700 1 -- 172.21.15.31:0/2055984437 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7c2006de50 msgr2=0x7f7c20156b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:40.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.087+0000 7f7c1ffff700 1 --2- 172.21.15.31:0/2055984437 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7c2006de50 0x7f7c20156b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:40.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.087+0000 7f7c1ffff700 1 -- 172.21.15.31:0/2055984437 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c20140a70 msgr2=0x7f7c20159a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:40.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.087+0000 7f7c1ffff700 1 --2- 172.21.15.31:0/2055984437 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c20140a70 0x7f7c20159a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:40.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.087+0000 7f7c1ffff700 1 -- 172.21.15.31:0/2055984437 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c10009420 con 0x7f7c20140e40 2023-10-06T02:32:40.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.087+0000 7f7c1ffff700 1 --2- 172.21.15.31:0/2055984437 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c20140e40 0x7f7c20159f70 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f7c1800cd00 tx=0x7f7c18001ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:40.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.087+0000 7f7c077fe700 1 -- 172.21.15.31:0/2055984437 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c1800ce00 con 0x7f7c20140e40 2023-10-06T02:32:40.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.088+0000 7f7c077fe700 1 -- 172.21.15.31:0/2055984437 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c1800aac0 con 0x7f7c20140e40 2023-10-06T02:32:40.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.088+0000 7f7c077fe700 1 -- 172.21.15.31:0/2055984437 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c1800ae80 con 0x7f7c20140e40 2023-10-06T02:32:40.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.088+0000 7f7c2499d700 1 -- 172.21.15.31:0/2055984437 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c200c5aa0 con 0x7f7c20140e40 2023-10-06T02:32:40.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.088+0000 7f7c2499d700 1 -- 172.21.15.31:0/2055984437 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c20157040 con 0x7f7c20140e40 2023-10-06T02:32:40.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.089+0000 7f7c2499d700 1 -- 172.21.15.31:0/2055984437 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c2004c3c0 con 0x7f7c20140e40 2023-10-06T02:32:40.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.090+0000 7f7c077fe700 1 -- 172.21.15.31:0/2055984437 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) v1 ==== 85709+0+0 (secure 0 0 0) 0x7f7c18007300 con 0x7f7c20140e40 2023-10-06T02:32:40.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.090+0000 7f7c077fe700 1 --2- 172.21.15.31:0/2055984437 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7c0c0679e0 0x7f7c0c069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:40.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.090+0000 7f7c077fe700 1 -- 172.21.15.31:0/2055984437 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5320+0+0 (secure 0 0 0) 0x7f7c18083270 con 0x7f7c20140e40 2023-10-06T02:32:40.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.091+0000 7f7c1f7fe700 1 --2- 172.21.15.31:0/2055984437 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7c0c0679e0 0x7f7c0c069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:40.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.091+0000 7f7c1f7fe700 1 --2- 172.21.15.31:0/2055984437 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7c0c0679e0 0x7f7c0c069e90 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f7c100091e0 tx=0x7f7c10017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:40.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.094+0000 7f7c077fe700 1 -- 172.21.15.31:0/2055984437 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7c18051f30 con 0x7f7c20140e40 2023-10-06T02:32:40.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.297+0000 7f7c2499d700 1 -- 172.21.15.31:0/2055984437 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f7c200c6230 con 0x7f7c0c0679e0 2023-10-06T02:32:40.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.298+0000 7f7c077fe700 1 -- 172.21.15.31:0/2055984437 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f7c10020040 con 0x7f7c0c0679e0 2023-10-06T02:32:40.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.300+0000 7f7c2499d700 1 -- 172.21.15.31:0/2055984437 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7c0c0679e0 msgr2=0x7f7c0c069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:40.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.300+0000 7f7c2499d700 1 --2- 172.21.15.31:0/2055984437 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7c0c0679e0 0x7f7c0c069e90 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f7c100091e0 tx=0x7f7c10017000 comp rx=0 tx=0).stop 2023-10-06T02:32:40.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.300+0000 7f7c2499d700 1 -- 172.21.15.31:0/2055984437 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c20140e40 msgr2=0x7f7c20159f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:40.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.300+0000 7f7c2499d700 1 --2- 172.21.15.31:0/2055984437 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c20140e40 0x7f7c20159f70 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f7c1800cd00 tx=0x7f7c18001ac0 comp rx=0 tx=0).stop 2023-10-06T02:32:40.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.300+0000 7f7c2499d700 1 -- 172.21.15.31:0/2055984437 shutdown_connections 2023-10-06T02:32:40.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.300+0000 7f7c2499d700 1 --2- 172.21.15.31:0/2055984437 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c20140e40 0x7f7c20159f70 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:40.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.300+0000 7f7c2499d700 1 --2- 172.21.15.31:0/2055984437 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f7c0c0679e0 0x7f7c0c069e90 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:40.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.300+0000 7f7c2499d700 1 --2- 172.21.15.31:0/2055984437 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7c2006de50 0x7f7c20156b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:40.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.300+0000 7f7c2499d700 1 --2- 172.21.15.31:0/2055984437 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c20140a70 0x7f7c20159a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:40.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.300+0000 7f7c2499d700 1 -- 172.21.15.31:0/2055984437 >> 172.21.15.31:0/2055984437 conn(0x7f7c2006d170 msgr2=0x7f7c200b1880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:40.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.300+0000 7f7c2499d700 1 -- 172.21.15.31:0/2055984437 shutdown_connections 2023-10-06T02:32:40.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.300+0000 7f7c2499d700 1 -- 172.21.15.31:0/2055984437 wait complete. 2023-10-06T02:32:40.303 INFO:teuthology.orchestra.run.smithi031.stderr:instructed osd(s) 2 to deep-scrub 2023-10-06T02:32:40.319 INFO:tasks.ceph:Scrubbing osd.3 2023-10-06T02:32:40.320 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-10-06T02:32:40.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.509+0000 7f76e179d700 1 Processor -- start 2023-10-06T02:32:40.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.509+0000 7f76e179d700 1 -- start start 2023-10-06T02:32:40.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.511+0000 7f76e179d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f76dc06c6a0 0x7f76dc06dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:40.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.511+0000 7f76e179d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f76dc0cacb0 0x7f76dc0c7010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:40.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.511+0000 7f76e179d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76dc06d740 con 0x7f76dc159770 2023-10-06T02:32:40.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.511+0000 7f76e179d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76dc159b40 con 0x7f76dc06c6a0 2023-10-06T02:32:40.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.511+0000 7f76e179d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76dc06cb30 con 0x7f76dc0cacb0 2023-10-06T02:32:40.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.511+0000 7f76dbfff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f76dc159770 0x7f76dc0ca5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:55876/0 (socket says 172.21.15.31:55876) 2023-10-06T02:32:40.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.511+0000 7f76dbfff700 1 -- 172.21.15.31:0/3804279857 learned_addr learned my addr 172.21.15.31:0/3804279857 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:32:40.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.511+0000 7f76db7fe700 1 --2- 172.21.15.31:0/3804279857 >> v2:172.21.15.62:3300/0 conn(0x7f76dc06c6a0 0x7f76dc06dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:40.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.511+0000 7f76e0f9c700 1 --2- 172.21.15.31:0/3804279857 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f76dc0cacb0 0x7f76dc0c7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:40.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.511+0000 7f76daffd700 1 -- 172.21.15.31:0/3804279857 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3204610277 0 0) 0x7f76dc06d740 con 0x7f76dc159770 2023-10-06T02:32:40.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.511+0000 7f76daffd700 1 -- 172.21.15.31:0/3804279857 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76c4003570 con 0x7f76dc159770 2023-10-06T02:32:40.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.512+0000 7f76daffd700 1 -- 172.21.15.31:0/3804279857 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2220693182 0 0) 0x7f76c4003570 con 0x7f76dc159770 2023-10-06T02:32:40.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.512+0000 7f76daffd700 1 -- 172.21.15.31:0/3804279857 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f76dc0cacb0 msgr2=0x7f76dc0c7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:40.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.512+0000 7f76daffd700 1 --2- 172.21.15.31:0/3804279857 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f76dc0cacb0 0x7f76dc0c7010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:40.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.512+0000 7f76daffd700 1 -- 172.21.15.31:0/3804279857 >> v2:172.21.15.62:3300/0 conn(0x7f76dc06c6a0 msgr2=0x7f76dc06dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:40.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.512+0000 7f76daffd700 1 --2- 172.21.15.31:0/3804279857 >> v2:172.21.15.62:3300/0 conn(0x7f76dc06c6a0 0x7f76dc06dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:40.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.512+0000 7f76daffd700 1 -- 172.21.15.31:0/3804279857 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76dc06ce10 con 0x7f76dc159770 2023-10-06T02:32:40.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.512+0000 7f76daffd700 1 -- 172.21.15.31:0/3804279857 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7f76cc001470 con 0x7f76dc159770 2023-10-06T02:32:40.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.512+0000 7f76daffd700 1 -- 172.21.15.31:0/3804279857 >> v1:172.21.15.31:6789/0 conn(0x7f76dc159770 legacy=0x7f76dc0ca5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:40.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.512+0000 7f76daffd700 1 --2- 172.21.15.31:0/3804279857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76c4003ae0 0x7f76c4003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:40.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.512+0000 7f76daffd700 1 --2- 172.21.15.31:0/3804279857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76c4004810 0x7f76c4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:40.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.512+0000 7f76daffd700 1 --2- 172.21.15.31:0/3804279857 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f76dc0cacb0 0x7f76c40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:40.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.512+0000 7f76daffd700 1 -- 172.21.15.31:0/3804279857 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76dc159b40 con 0x7f76c4003ae0 2023-10-06T02:32:40.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.512+0000 7f76daffd700 1 -- 172.21.15.31:0/3804279857 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76dc06cb30 con 0x7f76c4004810 2023-10-06T02:32:40.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.512+0000 7f76daffd700 1 -- 172.21.15.31:0/3804279857 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76c400bb10 con 0x7f76dc0cacb0 2023-10-06T02:32:40.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.512+0000 7f76db7fe700 1 --2- 172.21.15.31:0/3804279857 >> v2:172.21.15.62:3300/0 conn(0x7f76dc06c6a0 0x7f76dc06dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:32:40.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.512+0000 7f76dbfff700 1 --2- 172.21.15.31:0/3804279857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76c4003ae0 0x7f76c4003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:40.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.512+0000 7f76e0f9c700 1 --2- 172.21.15.31:0/3804279857 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f76dc0cacb0 0x7f76c40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:40.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.512+0000 7f76db7fe700 1 --2- 172.21.15.31:0/3804279857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76c4004810 0x7f76c4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:40.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.512+0000 7f76dbfff700 1 -- 172.21.15.31:0/3804279857 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f76dc0cacb0 msgr2=0x7f76c40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:40.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.512+0000 7f76dbfff700 1 --2- 172.21.15.31:0/3804279857 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f76dc0cacb0 0x7f76c40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:40.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.512+0000 7f76dbfff700 1 -- 172.21.15.31:0/3804279857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76c4004810 msgr2=0x7f76c4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:40.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.512+0000 7f76dbfff700 1 --2- 172.21.15.31:0/3804279857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76c4004810 0x7f76c4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:40.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.513+0000 7f76dbfff700 1 -- 172.21.15.31:0/3804279857 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76c4003570 con 0x7f76c4003ae0 2023-10-06T02:32:40.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.513+0000 7f76db7fe700 1 --2- 172.21.15.31:0/3804279857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76c4004810 0x7f76c4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:32:40.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.513+0000 7f76dbfff700 1 --2- 172.21.15.31:0/3804279857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76c4003ae0 0x7f76c4003ef0 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f76cc00c2d0 tx=0x7f76cc001dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a42baf624f628dea server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:40.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.513+0000 7f76daffd700 1 -- 172.21.15.31:0/3804279857 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76cc00cba0 con 0x7f76c4003ae0 2023-10-06T02:32:40.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.513+0000 7f76daffd700 1 -- 172.21.15.31:0/3804279857 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76cc00cd00 con 0x7f76c4003ae0 2023-10-06T02:32:40.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.513+0000 7f76daffd700 1 -- 172.21.15.31:0/3804279857 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76cc005b70 con 0x7f76c4003ae0 2023-10-06T02:32:40.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.513+0000 7f76e179d700 1 -- 172.21.15.31:0/3804279857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76c4003ae0 msgr2=0x7f76c4003ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:40.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.513+0000 7f76e179d700 1 --2- 172.21.15.31:0/3804279857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76c4003ae0 0x7f76c4003ef0 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f76cc00c2d0 tx=0x7f76cc001dc0 comp rx=0 tx=0).stop 2023-10-06T02:32:40.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.513+0000 7f76e0f9c700 1 -- 172.21.15.31:0/3804279857 reap_dead start 2023-10-06T02:32:40.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.514+0000 7f76e179d700 1 -- 172.21.15.31:0/3804279857 shutdown_connections 2023-10-06T02:32:40.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.514+0000 7f76e179d700 1 -- 172.21.15.31:0/3804279857 >> 172.21.15.31:0/3804279857 conn(0x7f76dc06e790 msgr2=0x7f76dc06eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:40.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.514+0000 7f76e179d700 1 -- 172.21.15.31:0/3804279857 shutdown_connections 2023-10-06T02:32:40.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.514+0000 7f76e179d700 1 -- 172.21.15.31:0/3804279857 wait complete. 2023-10-06T02:32:40.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.515+0000 7f76e179d700 1 Processor -- start 2023-10-06T02:32:40.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.515+0000 7f76e179d700 1 -- start start 2023-10-06T02:32:40.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.516+0000 7f76e179d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76c4003ae0 0x7f76dc16a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:40.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.516+0000 7f76e179d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f76dc0c5270 0x7f76dc0c3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:40.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.516+0000 7f76e179d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76dc14f850 0x7f76dc0c5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:40.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.516+0000 7f76e179d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76dc0c3fc0 con 0x7f76c4003ae0 2023-10-06T02:32:40.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.516+0000 7f76e179d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76dc155dd0 con 0x7f76dc14f850 2023-10-06T02:32:40.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.516+0000 7f76e179d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76dc0c4de0 con 0x7f76dc0c5270 2023-10-06T02:32:40.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.516+0000 7f76dbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76c4003ae0 0x7f76dc16a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:40.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.516+0000 7f76dbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76c4003ae0 0x7f76dc16a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50014/0 (socket says 172.21.15.31:50014) 2023-10-06T02:32:40.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.517+0000 7f76dbfff700 1 -- 172.21.15.31:0/1721732061 learned_addr learned my addr 172.21.15.31:0/1721732061 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:40.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.517+0000 7f76db7fe700 1 --2- 172.21.15.31:0/1721732061 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f76dc0c5270 0x7f76dc0c3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:40.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.517+0000 7f76e0f9c700 1 --2- 172.21.15.31:0/1721732061 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76dc14f850 0x7f76dc0c5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:40.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.517+0000 7f76dbfff700 1 -- 172.21.15.31:0/1721732061 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f76dc0c5270 msgr2=0x7f76dc0c3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:40.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.517+0000 7f76dbfff700 1 --2- 172.21.15.31:0/1721732061 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f76dc0c5270 0x7f76dc0c3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:40.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.517+0000 7f76dbfff700 1 -- 172.21.15.31:0/1721732061 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76dc14f850 msgr2=0x7f76dc0c5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:40.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.517+0000 7f76dbfff700 1 --2- 172.21.15.31:0/1721732061 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76dc14f850 0x7f76dc0c5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:40.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.517+0000 7f76dbfff700 1 -- 172.21.15.31:0/1721732061 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76dc0c41d0 con 0x7f76c4003ae0 2023-10-06T02:32:40.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.517+0000 7f76dbfff700 1 --2- 172.21.15.31:0/1721732061 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76c4003ae0 0x7f76dc16a600 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f76cc011000 tx=0x7f76cc012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:40.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.517+0000 7f76c3fff700 1 -- 172.21.15.31:0/1721732061 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76cc00ce70 con 0x7f76c4003ae0 2023-10-06T02:32:40.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.517+0000 7f76c3fff700 1 -- 172.21.15.31:0/1721732061 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76cc000f50 con 0x7f76c4003ae0 2023-10-06T02:32:40.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.517+0000 7f76c3fff700 1 -- 172.21.15.31:0/1721732061 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76cc007530 con 0x7f76c4003ae0 2023-10-06T02:32:40.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.517+0000 7f76e179d700 1 -- 172.21.15.31:0/1721732061 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76dc0c44b0 con 0x7f76c4003ae0 2023-10-06T02:32:40.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.518+0000 7f76e179d700 1 -- 172.21.15.31:0/1721732061 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76dc0c4a70 con 0x7f76c4003ae0 2023-10-06T02:32:40.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.518+0000 7f76e179d700 1 -- 172.21.15.31:0/1721732061 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f76dc04c3c0 con 0x7f76c4003ae0 2023-10-06T02:32:40.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.520+0000 7f76c3fff700 1 -- 172.21.15.31:0/1721732061 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) v1 ==== 85709+0+0 (secure 0 0 0) 0x7f76cc019030 con 0x7f76c4003ae0 2023-10-06T02:32:40.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.520+0000 7f76c3fff700 1 --2- 172.21.15.31:0/1721732061 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f76c80679e0 0x7f76c8069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:40.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.520+0000 7f76c3fff700 1 -- 172.21.15.31:0/1721732061 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5320+0+0 (secure 0 0 0) 0x7f76cc084820 con 0x7f76c4003ae0 2023-10-06T02:32:40.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.521+0000 7f76c3fff700 1 --2- 172.21.15.31:0/1721732061 >> [v2:172.21.15.31:6815/85584,v1:172.21.15.31:6817/85584] conn(0x7f76c806de50 0x7f76c8070280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:40.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.521+0000 7f76c3fff700 1 -- 172.21.15.31:0/1721732061 --> [v2:172.21.15.31:6815/85584,v1:172.21.15.31:6817/85584] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f76c8070950 con 0x7f76c806de50 2023-10-06T02:32:40.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.521+0000 7f76c3fff700 1 -- 172.21.15.31:0/1721732061 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=172) v2 ==== 24+0+0 (secure 0 0 0) 0x7f76cc086ee0 con 0x7f76c4003ae0 2023-10-06T02:32:40.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.521+0000 7f76db7fe700 1 --2- 172.21.15.31:0/1721732061 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f76c80679e0 0x7f76c8069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:40.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.521+0000 7f76e0f9c700 1 --2- 172.21.15.31:0/1721732061 >> [v2:172.21.15.31:6815/85584,v1:172.21.15.31:6817/85584] conn(0x7f76c806de50 0x7f76c8070280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:40.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.522+0000 7f76e0f9c700 1 --2- 172.21.15.31:0/1721732061 >> [v2:172.21.15.31:6815/85584,v1:172.21.15.31:6817/85584] conn(0x7f76c806de50 0x7f76c8070280 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:40.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.522+0000 7f76db7fe700 1 --2- 172.21.15.31:0/1721732061 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f76c80679e0 0x7f76c8069e90 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f76d00093f0 tx=0x7f76d0008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:40.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.522+0000 7f76c3fff700 1 -- 172.21.15.31:0/1721732061 <== osd.3 v2:172.21.15.31:6815/85584 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f76c8070950 con 0x7f76c806de50 2023-10-06T02:32:40.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.537+0000 7f76e179d700 1 -- 172.21.15.31:0/1721732061 --> [v2:172.21.15.31:6815/85584,v1:172.21.15.31:6817/85584] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f76dc0c3fc0 con 0x7f76c806de50 2023-10-06T02:32:40.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.537+0000 7f76c3fff700 1 -- 172.21.15.31:0/1721732061 <== osd.3 v2:172.21.15.31:6815/85584 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f76dc0c3fc0 con 0x7f76c806de50 2023-10-06T02:32:40.542 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T02:32:40.542 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T02:32:40.542 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T02:32:40.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.537+0000 7f76e179d700 1 -- 172.21.15.31:0/1721732061 >> [v2:172.21.15.31:6815/85584,v1:172.21.15.31:6817/85584] conn(0x7f76c806de50 msgr2=0x7f76c8070280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:40.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.537+0000 7f76e179d700 1 --2- 172.21.15.31:0/1721732061 >> [v2:172.21.15.31:6815/85584,v1:172.21.15.31:6817/85584] conn(0x7f76c806de50 0x7f76c8070280 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:40.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.538+0000 7f76e179d700 1 -- 172.21.15.31:0/1721732061 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f76c80679e0 msgr2=0x7f76c8069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:40.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.538+0000 7f76e179d700 1 --2- 172.21.15.31:0/1721732061 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f76c80679e0 0x7f76c8069e90 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f76d00093f0 tx=0x7f76d0008040 comp rx=0 tx=0).stop 2023-10-06T02:32:40.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.538+0000 7f76e179d700 1 -- 172.21.15.31:0/1721732061 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76c4003ae0 msgr2=0x7f76dc16a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:40.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.538+0000 7f76e179d700 1 --2- 172.21.15.31:0/1721732061 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76c4003ae0 0x7f76dc16a600 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f76cc011000 tx=0x7f76cc012000 comp rx=0 tx=0).stop 2023-10-06T02:32:40.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.538+0000 7f76e0f9c700 1 -- 172.21.15.31:0/1721732061 reap_dead start 2023-10-06T02:32:40.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.538+0000 7f76e179d700 1 -- 172.21.15.31:0/1721732061 shutdown_connections 2023-10-06T02:32:40.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.538+0000 7f76e179d700 1 -- 172.21.15.31:0/1721732061 >> 172.21.15.31:0/1721732061 conn(0x7f76dc06e790 msgr2=0x7f76dc0af7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:40.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.538+0000 7f76e179d700 1 -- 172.21.15.31:0/1721732061 shutdown_connections 2023-10-06T02:32:40.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.538+0000 7f76e179d700 1 -- 172.21.15.31:0/1721732061 wait complete. 2023-10-06T02:32:40.556 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-10-06T02:32:40.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.742+0000 7f78d5f9d700 1 Processor -- start 2023-10-06T02:32:40.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.742+0000 7f78d5f9d700 1 -- start start 2023-10-06T02:32:40.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.743+0000 7f78d5f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f78d0159550 0x7f78d00ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:40.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.743+0000 7f78d5f9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f78d015dfd0 0x7f78d00aedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:40.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.743+0000 7f78d5f9d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78d00acc00 con 0x7f78d0159920 2023-10-06T02:32:40.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.743+0000 7f78d5f9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78d00abff0 con 0x7f78d015dfd0 2023-10-06T02:32:40.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.743+0000 7f78d5f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78d00ac2d0 con 0x7f78d0159550 2023-10-06T02:32:40.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.743+0000 7f78cffff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f78d015dfd0 0x7f78d00aedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:40.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.743+0000 7f78d4f9b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f78d0159550 0x7f78d00ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:40.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.743+0000 7f78cffff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f78d015dfd0 0x7f78d00aedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:49770/0 (socket says 172.21.15.31:49770) 2023-10-06T02:32:40.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.743+0000 7f78cffff700 1 -- 172.21.15.31:0/1047394413 learned_addr learned my addr 172.21.15.31:0/1047394413 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:40.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.744+0000 7f78cffff700 1 -- 172.21.15.31:0/1047394413 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f78d0159550 msgr2=0x7f78d00ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:40.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.744+0000 7f78cffff700 1 --2- 172.21.15.31:0/1047394413 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f78d0159550 0x7f78d00ceac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:40.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.744+0000 7f78cffff700 1 -- 172.21.15.31:0/1047394413 >> v1:172.21.15.62:6789/0 conn(0x7f78d0159920 legacy=0x7f78d00ad090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:40.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.744+0000 7f78cffff700 1 -- 172.21.15.31:0/1047394413 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78d00ac5b0 con 0x7f78d015dfd0 2023-10-06T02:32:40.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.744+0000 7f78cffff700 1 --2- 172.21.15.31:0/1047394413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78d015dfd0 0x7f78d00aedb0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f78bc0099b0 tx=0x7f78bc009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=befa77445f715863 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:40.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.745+0000 7f78cf7fe700 1 -- 172.21.15.31:0/1047394413 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78bc009ea0 con 0x7f78d015dfd0 2023-10-06T02:32:40.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.745+0000 7f78cf7fe700 1 -- 172.21.15.31:0/1047394413 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78bc00b860 con 0x7f78d015dfd0 2023-10-06T02:32:40.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.745+0000 7f78cf7fe700 1 -- 172.21.15.31:0/1047394413 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78bc00bc00 con 0x7f78d015dfd0 2023-10-06T02:32:40.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.745+0000 7f78d5f9d700 1 -- 172.21.15.31:0/1047394413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78d015dfd0 msgr2=0x7f78d00aedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:40.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.745+0000 7f78d5f9d700 1 --2- 172.21.15.31:0/1047394413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78d015dfd0 0x7f78d00aedb0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f78bc0099b0 tx=0x7f78bc009250 comp rx=0 tx=0).stop 2023-10-06T02:32:40.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.745+0000 7f78d5f9d700 1 -- 172.21.15.31:0/1047394413 shutdown_connections 2023-10-06T02:32:40.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.745+0000 7f78d5f9d700 1 --2- 172.21.15.31:0/1047394413 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f78d0159550 0x7f78d00ceac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:40.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.745+0000 7f78d5f9d700 1 --2- 172.21.15.31:0/1047394413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78d015dfd0 0x7f78d00aedb0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:40.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.745+0000 7f78d5f9d700 1 -- 172.21.15.31:0/1047394413 >> 172.21.15.31:0/1047394413 conn(0x7f78d006ce80 msgr2=0x7f78d00ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:40.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.745+0000 7f78d5f9d700 1 -- 172.21.15.31:0/1047394413 shutdown_connections 2023-10-06T02:32:40.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.745+0000 7f78d5f9d700 1 -- 172.21.15.31:0/1047394413 wait complete. 2023-10-06T02:32:40.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.747+0000 7f78d5f9d700 1 Processor -- start 2023-10-06T02:32:40.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.747+0000 7f78d5f9d700 1 -- start start 2023-10-06T02:32:40.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.748+0000 7f78d5f9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f78d0159550 0x7f78d0098d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:40.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.748+0000 7f78d5f9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78d0159920 0x7f78d00b3160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:40.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.748+0000 7f78d5f9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78d015dfd0 0x7f78d01521a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:40.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.748+0000 7f78d5f9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78d00b39e0 con 0x7f78d015dfd0 2023-10-06T02:32:40.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.748+0000 7f78d5f9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78d00caf30 con 0x7f78d0159920 2023-10-06T02:32:40.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.748+0000 7f78d5f9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78d00cb210 con 0x7f78d0159550 2023-10-06T02:32:40.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.748+0000 7f78d579c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78d015dfd0 0x7f78d01521a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:40.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.748+0000 7f78d579c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78d015dfd0 0x7f78d01521a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50026/0 (socket says 172.21.15.31:50026) 2023-10-06T02:32:40.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.748+0000 7f78d579c700 1 -- 172.21.15.31:0/3363094409 learned_addr learned my addr 172.21.15.31:0/3363094409 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:40.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.748+0000 7f78d4f9b700 1 --2- 172.21.15.31:0/3363094409 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f78d0159550 0x7f78d0098d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:40.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.748+0000 7f78cffff700 1 --2- 172.21.15.31:0/3363094409 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78d0159920 0x7f78d00b3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:40.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.749+0000 7f78d579c700 1 -- 172.21.15.31:0/3363094409 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f78d0159550 msgr2=0x7f78d0098d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:40.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.749+0000 7f78d579c700 1 --2- 172.21.15.31:0/3363094409 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f78d0159550 0x7f78d0098d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:40.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.749+0000 7f78d579c700 1 -- 172.21.15.31:0/3363094409 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78d0159920 msgr2=0x7f78d00b3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:40.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.749+0000 7f78d579c700 1 --2- 172.21.15.31:0/3363094409 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78d0159920 0x7f78d00b3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:40.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.749+0000 7f78d579c700 1 -- 172.21.15.31:0/3363094409 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78bc0094f0 con 0x7f78d015dfd0 2023-10-06T02:32:40.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.749+0000 7f78d579c700 1 --2- 172.21.15.31:0/3363094409 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78d015dfd0 0x7f78d01521a0 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f78c800c8e0 tx=0x7f78c8001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:40.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.749+0000 7f78ccff9700 1 -- 172.21.15.31:0/3363094409 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c8012630 con 0x7f78d015dfd0 2023-10-06T02:32:40.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.749+0000 7f78ccff9700 1 -- 172.21.15.31:0/3363094409 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78c8012790 con 0x7f78d015dfd0 2023-10-06T02:32:40.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.749+0000 7f78ccff9700 1 -- 172.21.15.31:0/3363094409 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c8012b50 con 0x7f78d015dfd0 2023-10-06T02:32:40.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.749+0000 7f78d5f9d700 1 -- 172.21.15.31:0/3363094409 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78d00cb4f0 con 0x7f78d015dfd0 2023-10-06T02:32:40.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.749+0000 7f78d5f9d700 1 -- 172.21.15.31:0/3363094409 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78d01526e0 con 0x7f78d015dfd0 2023-10-06T02:32:40.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.750+0000 7f78d5f9d700 1 -- 172.21.15.31:0/3363094409 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78d004c3c0 con 0x7f78d015dfd0 2023-10-06T02:32:40.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.751+0000 7f78ccff9700 1 -- 172.21.15.31:0/3363094409 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) v1 ==== 85709+0+0 (secure 0 0 0) 0x7f78c801a3a0 con 0x7f78d015dfd0 2023-10-06T02:32:40.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.752+0000 7f78ccff9700 1 --2- 172.21.15.31:0/3363094409 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f78b40679e0 0x7f78b4069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:40.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.752+0000 7f78ccff9700 1 -- 172.21.15.31:0/3363094409 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5320+0+0 (secure 0 0 0) 0x7f78c8083360 con 0x7f78d015dfd0 2023-10-06T02:32:40.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.752+0000 7f78d4f9b700 1 --2- 172.21.15.31:0/3363094409 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f78b40679e0 0x7f78b4069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:40.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.753+0000 7f78d4f9b700 1 --2- 172.21.15.31:0/3363094409 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f78b40679e0 0x7f78b4069e90 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f78c4009d10 tx=0x7f78c40091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:40.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.756+0000 7f78ccff9700 1 -- 172.21.15.31:0/3363094409 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f78c8051d70 con 0x7f78d015dfd0 2023-10-06T02:32:40.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.959+0000 7f78d5f9d700 1 -- 172.21.15.31:0/3363094409 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f78d009bed0 con 0x7f78b40679e0 2023-10-06T02:32:40.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.960+0000 7f78ccff9700 1 -- 172.21.15.31:0/3363094409 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f78d009bed0 con 0x7f78b40679e0 2023-10-06T02:32:40.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.962+0000 7f78d5f9d700 1 -- 172.21.15.31:0/3363094409 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f78b40679e0 msgr2=0x7f78b4069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:40.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.962+0000 7f78d5f9d700 1 --2- 172.21.15.31:0/3363094409 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f78b40679e0 0x7f78b4069e90 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f78c4009d10 tx=0x7f78c40091a0 comp rx=0 tx=0).stop 2023-10-06T02:32:40.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.962+0000 7f78d5f9d700 1 -- 172.21.15.31:0/3363094409 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78d015dfd0 msgr2=0x7f78d01521a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:40.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.963+0000 7f78d5f9d700 1 --2- 172.21.15.31:0/3363094409 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78d015dfd0 0x7f78d01521a0 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f78c800c8e0 tx=0x7f78c8001710 comp rx=0 tx=0).stop 2023-10-06T02:32:40.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.963+0000 7f78d5f9d700 1 -- 172.21.15.31:0/3363094409 shutdown_connections 2023-10-06T02:32:40.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.963+0000 7f78d5f9d700 1 --2- 172.21.15.31:0/3363094409 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78d015dfd0 0x7f78d01521a0 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:40.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.963+0000 7f78d5f9d700 1 --2- 172.21.15.31:0/3363094409 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f78b40679e0 0x7f78b4069e90 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:40.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.963+0000 7f78d5f9d700 1 --2- 172.21.15.31:0/3363094409 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f78d0159550 0x7f78d0098d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:40.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.963+0000 7f78d5f9d700 1 --2- 172.21.15.31:0/3363094409 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78d0159920 0x7f78d00b3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:40.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.963+0000 7f78d5f9d700 1 -- 172.21.15.31:0/3363094409 >> 172.21.15.31:0/3363094409 conn(0x7f78d006ce80 msgr2=0x7f78d006eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:40.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.963+0000 7f78d5f9d700 1 -- 172.21.15.31:0/3363094409 shutdown_connections 2023-10-06T02:32:40.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:40.963+0000 7f78d5f9d700 1 -- 172.21.15.31:0/3363094409 wait complete. 2023-10-06T02:32:40.966 INFO:teuthology.orchestra.run.smithi031.stderr:instructed osd(s) 3 to deep-scrub 2023-10-06T02:32:40.982 INFO:tasks.ceph:Scrubbing osd.4 2023-10-06T02:32:40.982 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-10-06T02:32:41.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.171+0000 7f9497fff700 1 Processor -- start 2023-10-06T02:32:41.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.172+0000 7f9497fff700 1 -- start start 2023-10-06T02:32:41.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.173+0000 7f9497fff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f949815cb90 0x7f94980c7870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:41.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.173+0000 7f9497fff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94980c2fa0 con 0x7f949815dc10 2023-10-06T02:32:41.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.173+0000 7f9497fff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94980c2390 con 0x7f949806cdb0 2023-10-06T02:32:41.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.173+0000 7f9497fff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94980ccdf0 con 0x7f949815cb90 2023-10-06T02:32:41.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.173+0000 7f94967fc700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f949815dc10 0x7f949806c6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:55888/0 (socket says 172.21.15.31:55888) 2023-10-06T02:32:41.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.173+0000 7f94967fc700 1 -- 172.21.15.31:0/1276850628 learned_addr learned my addr 172.21.15.31:0/1276850628 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:32:41.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.173+0000 7f9496ffd700 1 --2- 172.21.15.31:0/1276850628 >> v2:172.21.15.62:3300/0 conn(0x7f949815cb90 0x7f94980c7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:41.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.174+0000 7f9495ffb700 1 -- 172.21.15.31:0/1276850628 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1195877502 0 0) 0x7f94980c2fa0 con 0x7f949815dc10 2023-10-06T02:32:41.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.174+0000 7f9495ffb700 1 -- 172.21.15.31:0/1276850628 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9480003570 con 0x7f949815dc10 2023-10-06T02:32:41.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.174+0000 7f9496ffd700 1 -- 172.21.15.31:0/1276850628 >> v1:172.21.15.62:6789/0 conn(0x7f949806cdb0 legacy=0x7f94980c3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:41.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.174+0000 7f9496ffd700 1 -- 172.21.15.31:0/1276850628 >> v1:172.21.15.31:6789/0 conn(0x7f949815dc10 legacy=0x7f949806c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:41.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.174+0000 7f9496ffd700 1 -- 172.21.15.31:0/1276850628 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94980c2670 con 0x7f949815cb90 2023-10-06T02:32:41.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.174+0000 7f9496ffd700 1 --2- 172.21.15.31:0/1276850628 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f949815cb90 0x7f94980c7870 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f94880098e0 tx=0x7f94880091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b1a1187cc831dc5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:41.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.174+0000 7f9495ffb700 1 -- 172.21.15.31:0/1276850628 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948800b640 con 0x7f949815cb90 2023-10-06T02:32:41.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.175+0000 7f9495ffb700 1 -- 172.21.15.31:0/1276850628 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f948800b7a0 con 0x7f949815cb90 2023-10-06T02:32:41.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.175+0000 7f9495ffb700 1 -- 172.21.15.31:0/1276850628 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948800bb40 con 0x7f949815cb90 2023-10-06T02:32:41.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.175+0000 7f9497fff700 1 -- 172.21.15.31:0/1276850628 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f949815cb90 msgr2=0x7f94980c7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:41.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.175+0000 7f9497fff700 1 --2- 172.21.15.31:0/1276850628 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f949815cb90 0x7f94980c7870 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f94880098e0 tx=0x7f94880091a0 comp rx=0 tx=0).stop 2023-10-06T02:32:41.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.175+0000 7f9497fff700 1 -- 172.21.15.31:0/1276850628 shutdown_connections 2023-10-06T02:32:41.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.175+0000 7f9497fff700 1 --2- 172.21.15.31:0/1276850628 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f949815cb90 0x7f94980c7870 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:41.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.175+0000 7f9497fff700 1 -- 172.21.15.31:0/1276850628 >> 172.21.15.31:0/1276850628 conn(0x7f94980cd230 msgr2=0x7f94980cd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:41.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.175+0000 7f9497fff700 1 -- 172.21.15.31:0/1276850628 shutdown_connections 2023-10-06T02:32:41.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.175+0000 7f9497fff700 1 -- 172.21.15.31:0/1276850628 wait complete. 2023-10-06T02:32:41.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.177+0000 7f9497fff700 1 Processor -- start 2023-10-06T02:32:41.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.177+0000 7f9497fff700 1 -- start start 2023-10-06T02:32:41.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.178+0000 7f9497fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f949806cdb0 0x7f94981544b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:41.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.178+0000 7f9497fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f949815cb90 0x7f9498169f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:41.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.178+0000 7f9497fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f949815dc10 0x7f949816a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:41.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.178+0000 7f9497fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9498169b10 con 0x7f949815dc10 2023-10-06T02:32:41.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.178+0000 7f9497fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94980b2480 con 0x7f949806cdb0 2023-10-06T02:32:41.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.178+0000 7f9497fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94980b2760 con 0x7f949815cb90 2023-10-06T02:32:41.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.178+0000 7f94977fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f949815dc10 0x7f949816a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:41.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.178+0000 7f9496ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f949806cdb0 0x7f94981544b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:41.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.178+0000 7f94977fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f949815dc10 0x7f949816a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50042/0 (socket says 172.21.15.31:50042) 2023-10-06T02:32:41.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.178+0000 7f94977fe700 1 -- 172.21.15.31:0/1784109408 learned_addr learned my addr 172.21.15.31:0/1784109408 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:41.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.178+0000 7f94967fc700 1 --2- 172.21.15.31:0/1784109408 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f949815cb90 0x7f9498169f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:41.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.178+0000 7f94977fe700 1 -- 172.21.15.31:0/1784109408 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f949815cb90 msgr2=0x7f9498169f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:41.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.178+0000 7f94977fe700 1 --2- 172.21.15.31:0/1784109408 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f949815cb90 0x7f9498169f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:41.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.178+0000 7f94977fe700 1 -- 172.21.15.31:0/1784109408 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f949806cdb0 msgr2=0x7f94981544b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:41.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.178+0000 7f94977fe700 1 --2- 172.21.15.31:0/1784109408 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f949806cdb0 0x7f94981544b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:41.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.178+0000 7f94977fe700 1 -- 172.21.15.31:0/1784109408 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9488009420 con 0x7f949815dc10 2023-10-06T02:32:41.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.179+0000 7f94977fe700 1 --2- 172.21.15.31:0/1784109408 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f949815dc10 0x7f949816a4a0 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f949000c910 tx=0x7f94900016f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:41.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.179+0000 7f947effd700 1 -- 172.21.15.31:0/1784109408 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9490012630 con 0x7f949815dc10 2023-10-06T02:32:41.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.179+0000 7f947effd700 1 -- 172.21.15.31:0/1784109408 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9490012790 con 0x7f949815dc10 2023-10-06T02:32:41.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.179+0000 7f947effd700 1 -- 172.21.15.31:0/1784109408 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9490012b50 con 0x7f949815dc10 2023-10-06T02:32:41.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.179+0000 7f9497fff700 1 -- 172.21.15.31:0/1784109408 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94980b2a40 con 0x7f949815dc10 2023-10-06T02:32:41.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.179+0000 7f9497fff700 1 -- 172.21.15.31:0/1784109408 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94980b2d20 con 0x7f949815dc10 2023-10-06T02:32:41.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.180+0000 7f9497fff700 1 -- 172.21.15.31:0/1784109408 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f949804c3c0 con 0x7f949815dc10 2023-10-06T02:32:41.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.181+0000 7f947effd700 1 -- 172.21.15.31:0/1784109408 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) v1 ==== 85709+0+0 (secure 0 0 0) 0x7f949001b450 con 0x7f949815dc10 2023-10-06T02:32:41.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.182+0000 7f947effd700 1 --2- 172.21.15.31:0/1784109408 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f94840679e0 0x7f9484069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:41.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.182+0000 7f947effd700 1 -- 172.21.15.31:0/1784109408 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5320+0+0 (secure 0 0 0) 0x7f9490083070 con 0x7f949815dc10 2023-10-06T02:32:41.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.183+0000 7f947effd700 1 --2- 172.21.15.31:0/1784109408 >> [v2:172.21.15.62:6800/85799,v1:172.21.15.62:6801/85799] conn(0x7f948406de50 0x7f9484070280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:41.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.183+0000 7f947effd700 1 -- 172.21.15.31:0/1784109408 --> [v2:172.21.15.62:6800/85799,v1:172.21.15.62:6801/85799] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9484070950 con 0x7f948406de50 2023-10-06T02:32:41.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.183+0000 7f947effd700 1 -- 172.21.15.31:0/1784109408 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=172) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9490087db0 con 0x7f949815dc10 2023-10-06T02:32:41.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.183+0000 7f9496ffd700 1 --2- 172.21.15.31:0/1784109408 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f94840679e0 0x7f9484069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:41.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.183+0000 7f94967fc700 1 --2- 172.21.15.31:0/1784109408 >> [v2:172.21.15.62:6800/85799,v1:172.21.15.62:6801/85799] conn(0x7f948406de50 0x7f9484070280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:41.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.183+0000 7f9496ffd700 1 --2- 172.21.15.31:0/1784109408 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f94840679e0 0x7f9484069e90 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f94880093f0 tx=0x7f9488017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:41.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.184+0000 7f94967fc700 1 --2- 172.21.15.31:0/1784109408 >> [v2:172.21.15.62:6800/85799,v1:172.21.15.62:6801/85799] conn(0x7f948406de50 0x7f9484070280 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:41.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.185+0000 7f947effd700 1 -- 172.21.15.31:0/1784109408 <== osd.4 v2:172.21.15.62:6800/85799 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f9484070950 con 0x7f948406de50 2023-10-06T02:32:41.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.199+0000 7f9497fff700 1 -- 172.21.15.31:0/1784109408 --> [v2:172.21.15.62:6800/85799,v1:172.21.15.62:6801/85799] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f94980cdef0 con 0x7f948406de50 2023-10-06T02:32:41.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.200+0000 7f947effd700 1 -- 172.21.15.31:0/1784109408 <== osd.4 v2:172.21.15.62:6800/85799 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f94980cdef0 con 0x7f948406de50 2023-10-06T02:32:41.202 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T02:32:41.202 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T02:32:41.202 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T02:32:41.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.201+0000 7f9497fff700 1 -- 172.21.15.31:0/1784109408 >> [v2:172.21.15.62:6800/85799,v1:172.21.15.62:6801/85799] conn(0x7f948406de50 msgr2=0x7f9484070280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:41.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.201+0000 7f9497fff700 1 --2- 172.21.15.31:0/1784109408 >> [v2:172.21.15.62:6800/85799,v1:172.21.15.62:6801/85799] conn(0x7f948406de50 0x7f9484070280 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:41.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.201+0000 7f9497fff700 1 -- 172.21.15.31:0/1784109408 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f94840679e0 msgr2=0x7f9484069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:41.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.201+0000 7f9497fff700 1 --2- 172.21.15.31:0/1784109408 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f94840679e0 0x7f9484069e90 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f94880093f0 tx=0x7f9488017000 comp rx=0 tx=0).stop 2023-10-06T02:32:41.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.201+0000 7f9497fff700 1 -- 172.21.15.31:0/1784109408 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f949815dc10 msgr2=0x7f949816a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:41.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.201+0000 7f9497fff700 1 --2- 172.21.15.31:0/1784109408 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f949815dc10 0x7f949816a4a0 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f949000c910 tx=0x7f94900016f0 comp rx=0 tx=0).stop 2023-10-06T02:32:41.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.201+0000 7f94977fe700 1 -- 172.21.15.31:0/1784109408 reap_dead start 2023-10-06T02:32:41.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.201+0000 7f9497fff700 1 -- 172.21.15.31:0/1784109408 shutdown_connections 2023-10-06T02:32:41.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.201+0000 7f9497fff700 1 -- 172.21.15.31:0/1784109408 >> 172.21.15.31:0/1784109408 conn(0x7f94980cd230 msgr2=0x7f9498150f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:41.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.201+0000 7f9497fff700 1 -- 172.21.15.31:0/1784109408 shutdown_connections 2023-10-06T02:32:41.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.201+0000 7f9497fff700 1 -- 172.21.15.31:0/1784109408 wait complete. 2023-10-06T02:32:41.219 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-10-06T02:32:41.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.405+0000 7ffb6899d700 1 Processor -- start 2023-10-06T02:32:41.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.405+0000 7ffb6899d700 1 -- start start 2023-10-06T02:32:41.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.406+0000 7ffb6899d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ffb64153c30 0x7ffb6415a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:41.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.406+0000 7ffb6899d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ffb640cf110 0x7ffb6415c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:41.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.406+0000 7ffb6899d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb6415afd0 con 0x7ffb6415de50 2023-10-06T02:32:41.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.406+0000 7ffb6899d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb6415b2b0 con 0x7ffb64153c30 2023-10-06T02:32:41.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.406+0000 7ffb6899d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb6415b590 con 0x7ffb640cf110 2023-10-06T02:32:41.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.407+0000 7ffb62ffd700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7ffb6415de50 0x7ffb640cea00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:55890/0 (socket says 172.21.15.31:55890) 2023-10-06T02:32:41.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.407+0000 7ffb62ffd700 1 -- 172.21.15.31:0/2388740234 learned_addr learned my addr 172.21.15.31:0/2388740234 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:32:41.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.407+0000 7ffb637fe700 1 --2- 172.21.15.31:0/2388740234 >> v2:172.21.15.31:3300/0 conn(0x7ffb64153c30 0x7ffb6415a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:41.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.407+0000 7ffb63fff700 1 --2- 172.21.15.31:0/2388740234 >> v2:172.21.15.62:3300/0 conn(0x7ffb640cf110 0x7ffb6415c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:41.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.407+0000 7ffb637fe700 1 -- 172.21.15.31:0/2388740234 >> v2:172.21.15.62:3300/0 conn(0x7ffb640cf110 msgr2=0x7ffb6415c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:41.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.407+0000 7ffb637fe700 1 --2- 172.21.15.31:0/2388740234 >> v2:172.21.15.62:3300/0 conn(0x7ffb640cf110 0x7ffb6415c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:41.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.407+0000 7ffb627fc700 1 -- 172.21.15.31:0/2388740234 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 624445590 0 0) 0x7ffb6415afd0 con 0x7ffb6415de50 2023-10-06T02:32:41.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.407+0000 7ffb637fe700 1 -- 172.21.15.31:0/2388740234 >> v1:172.21.15.31:6789/0 conn(0x7ffb6415de50 legacy=0x7ffb640cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:41.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.407+0000 7ffb637fe700 1 -- 172.21.15.31:0/2388740234 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb6415b870 con 0x7ffb64153c30 2023-10-06T02:32:41.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.407+0000 7ffb637fe700 1 --2- 172.21.15.31:0/2388740234 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb64153c30 0x7ffb6415a5e0 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7ffb640cdab0 tx=0x7ffb540091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af07e01d1ff8601c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:41.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.408+0000 7ffb627fc700 1 -- 172.21.15.31:0/2388740234 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb5400b6b0 con 0x7ffb64153c30 2023-10-06T02:32:41.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.408+0000 7ffb627fc700 1 -- 172.21.15.31:0/2388740234 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb5400b810 con 0x7ffb64153c30 2023-10-06T02:32:41.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.408+0000 7ffb627fc700 1 -- 172.21.15.31:0/2388740234 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb5400bbb0 con 0x7ffb64153c30 2023-10-06T02:32:41.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.408+0000 7ffb6899d700 1 -- 172.21.15.31:0/2388740234 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb64153c30 msgr2=0x7ffb6415a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:41.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.408+0000 7ffb6899d700 1 --2- 172.21.15.31:0/2388740234 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb64153c30 0x7ffb6415a5e0 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7ffb640cdab0 tx=0x7ffb540091a0 comp rx=0 tx=0).stop 2023-10-06T02:32:41.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.408+0000 7ffb6899d700 1 -- 172.21.15.31:0/2388740234 shutdown_connections 2023-10-06T02:32:41.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.408+0000 7ffb6899d700 1 --2- 172.21.15.31:0/2388740234 >> v2:172.21.15.62:3300/0 conn(0x7ffb640cf110 0x7ffb6415c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:41.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.408+0000 7ffb6899d700 1 --2- 172.21.15.31:0/2388740234 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb64153c30 0x7ffb6415a5e0 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:41.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.408+0000 7ffb6899d700 1 -- 172.21.15.31:0/2388740234 >> 172.21.15.31:0/2388740234 conn(0x7ffb640703a0 msgr2=0x7ffb640707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:41.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.408+0000 7ffb6899d700 1 -- 172.21.15.31:0/2388740234 shutdown_connections 2023-10-06T02:32:41.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.408+0000 7ffb6899d700 1 -- 172.21.15.31:0/2388740234 wait complete. 2023-10-06T02:32:41.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.410+0000 7ffb6899d700 1 Processor -- start 2023-10-06T02:32:41.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.410+0000 7ffb6899d700 1 -- start start 2023-10-06T02:32:41.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.411+0000 7ffb6899d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb640cf110 0x7ffb640c9e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:41.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.411+0000 7ffb6899d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffb64153c30 0x7ffb640b5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:41.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.411+0000 7ffb6899d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb6415de50 0x7ffb64152f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:41.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.411+0000 7ffb6899d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb640c8d90 con 0x7ffb640cf110 2023-10-06T02:32:41.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.411+0000 7ffb6899d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb640c9070 con 0x7ffb6415de50 2023-10-06T02:32:41.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.411+0000 7ffb6899d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb640c9350 con 0x7ffb64153c30 2023-10-06T02:32:41.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.411+0000 7ffb637fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb640cf110 0x7ffb640c9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:41.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.411+0000 7ffb637fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb640cf110 0x7ffb640c9e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50058/0 (socket says 172.21.15.31:50058) 2023-10-06T02:32:41.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.411+0000 7ffb637fe700 1 -- 172.21.15.31:0/3426859580 learned_addr learned my addr 172.21.15.31:0/3426859580 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:41.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.411+0000 7ffb637fe700 1 -- 172.21.15.31:0/3426859580 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffb64153c30 msgr2=0x7ffb640b5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:41.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.411+0000 7ffb637fe700 1 --2- 172.21.15.31:0/3426859580 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffb64153c30 0x7ffb640b5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:41.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.411+0000 7ffb637fe700 1 -- 172.21.15.31:0/3426859580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb6415de50 msgr2=0x7ffb64152f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:41.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.411+0000 7ffb637fe700 1 --2- 172.21.15.31:0/3426859580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb6415de50 0x7ffb64152f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:41.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.411+0000 7ffb637fe700 1 -- 172.21.15.31:0/3426859580 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb54009420 con 0x7ffb640cf110 2023-10-06T02:32:41.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.412+0000 7ffb637fe700 1 --2- 172.21.15.31:0/3426859580 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb640cf110 0x7ffb640c9e30 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7ffb6406eb00 tx=0x7ffb54017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:41.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.412+0000 7ffb4b7fe700 1 -- 172.21.15.31:0/3426859580 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb54011420 con 0x7ffb640cf110 2023-10-06T02:32:41.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.412+0000 7ffb4b7fe700 1 -- 172.21.15.31:0/3426859580 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb54011580 con 0x7ffb640cf110 2023-10-06T02:32:41.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.412+0000 7ffb4b7fe700 1 -- 172.21.15.31:0/3426859580 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb54011940 con 0x7ffb640cf110 2023-10-06T02:32:41.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.412+0000 7ffb6899d700 1 -- 172.21.15.31:0/3426859580 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb640c9630 con 0x7ffb640cf110 2023-10-06T02:32:41.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.412+0000 7ffb6899d700 1 -- 172.21.15.31:0/3426859580 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb640ca370 con 0x7ffb640cf110 2023-10-06T02:32:41.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.413+0000 7ffb6899d700 1 -- 172.21.15.31:0/3426859580 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb640c9b30 con 0x7ffb640cf110 2023-10-06T02:32:41.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.414+0000 7ffb4b7fe700 1 -- 172.21.15.31:0/3426859580 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) v1 ==== 85709+0+0 (secure 0 0 0) 0x7ffb54011d30 con 0x7ffb640cf110 2023-10-06T02:32:41.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.415+0000 7ffb4b7fe700 1 --2- 172.21.15.31:0/3426859580 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ffb500679e0 0x7ffb50069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:41.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.415+0000 7ffb4b7fe700 1 -- 172.21.15.31:0/3426859580 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5320+0+0 (secure 0 0 0) 0x7ffb54084930 con 0x7ffb640cf110 2023-10-06T02:32:41.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.415+0000 7ffb62ffd700 1 --2- 172.21.15.31:0/3426859580 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ffb500679e0 0x7ffb50069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:41.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.416+0000 7ffb62ffd700 1 --2- 172.21.15.31:0/3426859580 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ffb500679e0 0x7ffb50069e90 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7ffb640ccf10 tx=0x7ffb58007bd0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:41.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.419+0000 7ffb4b7fe700 1 -- 172.21.15.31:0/3426859580 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffb54053570 con 0x7ffb640cf110 2023-10-06T02:32:41.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.621+0000 7ffb6899d700 1 -- 172.21.15.31:0/3426859580 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7ffb6404c3c0 con 0x7ffb500679e0 2023-10-06T02:32:41.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.622+0000 7ffb4b7fe700 1 -- 172.21.15.31:0/3426859580 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7ffb6404c3c0 con 0x7ffb500679e0 2023-10-06T02:32:41.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.624+0000 7ffb6899d700 1 -- 172.21.15.31:0/3426859580 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ffb500679e0 msgr2=0x7ffb50069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:41.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.624+0000 7ffb6899d700 1 --2- 172.21.15.31:0/3426859580 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ffb500679e0 0x7ffb50069e90 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7ffb640ccf10 tx=0x7ffb58007bd0 comp rx=0 tx=0).stop 2023-10-06T02:32:41.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.624+0000 7ffb6899d700 1 -- 172.21.15.31:0/3426859580 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb640cf110 msgr2=0x7ffb640c9e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:41.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.624+0000 7ffb6899d700 1 --2- 172.21.15.31:0/3426859580 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb640cf110 0x7ffb640c9e30 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7ffb6406eb00 tx=0x7ffb54017000 comp rx=0 tx=0).stop 2023-10-06T02:32:41.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.624+0000 7ffb6899d700 1 -- 172.21.15.31:0/3426859580 shutdown_connections 2023-10-06T02:32:41.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.624+0000 7ffb6899d700 1 --2- 172.21.15.31:0/3426859580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb6415de50 0x7ffb64152f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:41.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.624+0000 7ffb6899d700 1 --2- 172.21.15.31:0/3426859580 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb640cf110 0x7ffb640c9e30 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:41.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.624+0000 7ffb6899d700 1 --2- 172.21.15.31:0/3426859580 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ffb500679e0 0x7ffb50069e90 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:41.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.624+0000 7ffb6899d700 1 --2- 172.21.15.31:0/3426859580 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffb64153c30 0x7ffb640b5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:41.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.624+0000 7ffb6899d700 1 -- 172.21.15.31:0/3426859580 >> 172.21.15.31:0/3426859580 conn(0x7ffb640703a0 msgr2=0x7ffb640d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:41.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.624+0000 7ffb6899d700 1 -- 172.21.15.31:0/3426859580 shutdown_connections 2023-10-06T02:32:41.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.624+0000 7ffb6899d700 1 -- 172.21.15.31:0/3426859580 wait complete. 2023-10-06T02:32:41.627 INFO:teuthology.orchestra.run.smithi031.stderr:instructed osd(s) 4 to deep-scrub 2023-10-06T02:32:41.643 INFO:tasks.ceph:Scrubbing osd.5 2023-10-06T02:32:41.643 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-10-06T02:32:41.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.832+0000 7f1a0cd9d700 1 Processor -- start 2023-10-06T02:32:41.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.832+0000 7f1a0cd9d700 1 -- start start 2023-10-06T02:32:41.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.834+0000 7f1a0cd9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1a080c2a90 0x7f1a08156aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:41.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.834+0000 7f1a0cd9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f1a080c85d0 0x7f1a080cc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:41.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.834+0000 7f1a0cd9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a081579b0 con 0x7f1a080c8200 2023-10-06T02:32:41.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.834+0000 7f1a0cd9d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a080c1390 con 0x7f1a080c2a90 2023-10-06T02:32:41.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.834+0000 7f1a0cd9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a081573f0 con 0x7f1a080c85d0 2023-10-06T02:32:41.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.834+0000 7f1a077fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f1a080c8200 0x7f1a080c1820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:55894/0 (socket says 172.21.15.31:55894) 2023-10-06T02:32:41.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.834+0000 7f1a077fe700 1 -- 172.21.15.31:0/1743205247 learned_addr learned my addr 172.21.15.31:0/1743205247 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:32:41.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.834+0000 7f1a06ffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1a080c2a90 0x7f1a08156aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:41.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.834+0000 7f1a07fff700 1 --2- 172.21.15.31:0/1743205247 >> v2:172.21.15.62:3300/0 conn(0x7f1a080c85d0 0x7f1a080cc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:41.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.835+0000 7f1a06ffd700 1 -- 172.21.15.31:0/1743205247 >> v2:172.21.15.62:3300/0 conn(0x7f1a080c85d0 msgr2=0x7f1a080cc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:41.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.834+0000 7f1a067fc700 1 -- 172.21.15.31:0/1743205247 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2619845626 0 0) 0x7f1a081579b0 con 0x7f1a080c8200 2023-10-06T02:32:41.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.835+0000 7f1a06ffd700 1 --2- 172.21.15.31:0/1743205247 >> v2:172.21.15.62:3300/0 conn(0x7f1a080c85d0 0x7f1a080cc2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:41.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.835+0000 7f1a06ffd700 1 -- 172.21.15.31:0/1743205247 >> v1:172.21.15.31:6789/0 conn(0x7f1a080c8200 legacy=0x7f1a080c1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:41.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.835+0000 7f1a06ffd700 1 -- 172.21.15.31:0/1743205247 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a081576d0 con 0x7f1a080c2a90 2023-10-06T02:32:41.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.835+0000 7f1a06ffd700 1 --2- 172.21.15.31:0/1743205247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a080c2a90 0x7f1a08156aa0 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f1a080c1300 tx=0x7f19fc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6edd5e68d34ebf51 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:41.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.835+0000 7f1a067fc700 1 -- 172.21.15.31:0/1743205247 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19fc00f6b0 con 0x7f1a080c2a90 2023-10-06T02:32:41.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.835+0000 7f1a067fc700 1 -- 172.21.15.31:0/1743205247 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19fc00f810 con 0x7f1a080c2a90 2023-10-06T02:32:41.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.835+0000 7f1a067fc700 1 -- 172.21.15.31:0/1743205247 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19fc00fbb0 con 0x7f1a080c2a90 2023-10-06T02:32:41.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.836+0000 7f1a0cd9d700 1 -- 172.21.15.31:0/1743205247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a080c2a90 msgr2=0x7f1a08156aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:41.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.836+0000 7f1a0cd9d700 1 --2- 172.21.15.31:0/1743205247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a080c2a90 0x7f1a08156aa0 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f1a080c1300 tx=0x7f19fc0091a0 comp rx=0 tx=0).stop 2023-10-06T02:32:41.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.836+0000 7f1a0cd9d700 1 -- 172.21.15.31:0/1743205247 shutdown_connections 2023-10-06T02:32:41.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.836+0000 7f1a0cd9d700 1 --2- 172.21.15.31:0/1743205247 >> v2:172.21.15.62:3300/0 conn(0x7f1a080c85d0 0x7f1a080cc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:41.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.836+0000 7f1a0cd9d700 1 --2- 172.21.15.31:0/1743205247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a080c2a90 0x7f1a08156aa0 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:41.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.836+0000 7f1a0cd9d700 1 -- 172.21.15.31:0/1743205247 >> 172.21.15.31:0/1743205247 conn(0x7f1a0806c6a0 msgr2=0x7f1a0806caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:41.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.836+0000 7f1a0cd9d700 1 -- 172.21.15.31:0/1743205247 shutdown_connections 2023-10-06T02:32:41.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.836+0000 7f1a0cd9d700 1 -- 172.21.15.31:0/1743205247 wait complete. 2023-10-06T02:32:41.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.837+0000 7f1a0cd9d700 1 Processor -- start 2023-10-06T02:32:41.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.837+0000 7f1a0cd9d700 1 -- start start 2023-10-06T02:32:41.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.838+0000 7f1a0cd9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1a080c2a90 0x7f1a08144c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:41.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.838+0000 7f1a0cd9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a080c8200 0x7f1a0814e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:41.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.838+0000 7f1a0cd9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a080c85d0 0x7f1a0814eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:41.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.838+0000 7f1a0cd9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a080b1090 con 0x7f1a080c85d0 2023-10-06T02:32:41.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.838+0000 7f1a0cd9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a08146bf0 con 0x7f1a080c8200 2023-10-06T02:32:41.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.838+0000 7f1a0cd9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a081451b0 con 0x7f1a080c2a90 2023-10-06T02:32:41.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.838+0000 7f1a07fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a080c85d0 0x7f1a0814eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:41.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.839+0000 7f1a07fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a080c85d0 0x7f1a0814eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50068/0 (socket says 172.21.15.31:50068) 2023-10-06T02:32:41.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.839+0000 7f1a07fff700 1 -- 172.21.15.31:0/3721659971 learned_addr learned my addr 172.21.15.31:0/3721659971 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:41.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.839+0000 7f1a077fe700 1 --2- 172.21.15.31:0/3721659971 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1a080c2a90 0x7f1a08144c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:41.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.839+0000 7f1a06ffd700 1 --2- 172.21.15.31:0/3721659971 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a080c8200 0x7f1a0814e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:41.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.839+0000 7f1a07fff700 1 -- 172.21.15.31:0/3721659971 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1a080c2a90 msgr2=0x7f1a08144c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:41.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.839+0000 7f1a07fff700 1 --2- 172.21.15.31:0/3721659971 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1a080c2a90 0x7f1a08144c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:41.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.839+0000 7f1a07fff700 1 -- 172.21.15.31:0/3721659971 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a080c8200 msgr2=0x7f1a0814e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:41.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.839+0000 7f1a07fff700 1 --2- 172.21.15.31:0/3721659971 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a080c8200 0x7f1a0814e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:41.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.839+0000 7f1a07fff700 1 -- 172.21.15.31:0/3721659971 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19fc009420 con 0x7f1a080c85d0 2023-10-06T02:32:41.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.839+0000 7f1a07fff700 1 --2- 172.21.15.31:0/3721659971 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a080c85d0 0x7f1a0814eda0 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f1a00007f20 tx=0x7f1a00009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:41.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.839+0000 7f19ef7fe700 1 -- 172.21.15.31:0/3721659971 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a0000eca0 con 0x7f1a080c85d0 2023-10-06T02:32:41.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.839+0000 7f19ef7fe700 1 -- 172.21.15.31:0/3721659971 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a0000ee20 con 0x7f1a080c85d0 2023-10-06T02:32:41.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.839+0000 7f19ef7fe700 1 -- 172.21.15.31:0/3721659971 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a00014670 con 0x7f1a080c85d0 2023-10-06T02:32:41.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.839+0000 7f1a0cd9d700 1 -- 172.21.15.31:0/3721659971 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a08145fe0 con 0x7f1a080c85d0 2023-10-06T02:32:41.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.840+0000 7f1a0cd9d700 1 -- 172.21.15.31:0/3721659971 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a081465a0 con 0x7f1a080c85d0 2023-10-06T02:32:41.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.840+0000 7f1a0cd9d700 1 -- 172.21.15.31:0/3721659971 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1a0804c3c0 con 0x7f1a080c85d0 2023-10-06T02:32:41.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.842+0000 7f19ef7fe700 1 -- 172.21.15.31:0/3721659971 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) v1 ==== 85709+0+0 (secure 0 0 0) 0x7f1a000149e0 con 0x7f1a080c85d0 2023-10-06T02:32:41.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.842+0000 7f19ef7fe700 1 --2- 172.21.15.31:0/3721659971 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f19f40679e0 0x7f19f4069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:41.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.843+0000 7f19ef7fe700 1 -- 172.21.15.31:0/3721659971 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5320+0+0 (secure 0 0 0) 0x7f1a00083650 con 0x7f1a080c85d0 2023-10-06T02:32:41.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.843+0000 7f19ef7fe700 1 --2- 172.21.15.31:0/3721659971 >> [v2:172.21.15.62:6816/85800,v1:172.21.15.62:6817/85800] conn(0x7f19f406de50 0x7f19f4070280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:41.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.843+0000 7f19ef7fe700 1 -- 172.21.15.31:0/3721659971 --> [v2:172.21.15.62:6816/85800,v1:172.21.15.62:6817/85800] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f19f4070950 con 0x7f19f406de50 2023-10-06T02:32:41.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.843+0000 7f19ef7fe700 1 -- 172.21.15.31:0/3721659971 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=172) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a00088390 con 0x7f1a080c85d0 2023-10-06T02:32:41.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.843+0000 7f1a077fe700 1 --2- 172.21.15.31:0/3721659971 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f19f40679e0 0x7f19f4069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:41.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.843+0000 7f1a06ffd700 1 --2- 172.21.15.31:0/3721659971 >> [v2:172.21.15.62:6816/85800,v1:172.21.15.62:6817/85800] conn(0x7f19f406de50 0x7f19f4070280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:41.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.844+0000 7f1a077fe700 1 --2- 172.21.15.31:0/3721659971 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f19f40679e0 0x7f19f4069e90 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f19f80011e0 tx=0x7f19f8009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:41.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.844+0000 7f1a06ffd700 1 --2- 172.21.15.31:0/3721659971 >> [v2:172.21.15.62:6816/85800,v1:172.21.15.62:6817/85800] conn(0x7f19f406de50 0x7f19f4070280 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:41.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.845+0000 7f19ef7fe700 1 -- 172.21.15.31:0/3721659971 <== osd.5 v2:172.21.15.62:6816/85800 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f19f4070950 con 0x7f19f406de50 2023-10-06T02:32:41.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.859+0000 7f1a0cd9d700 1 -- 172.21.15.31:0/3721659971 --> [v2:172.21.15.62:6816/85800,v1:172.21.15.62:6817/85800] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f1a0806d6b0 con 0x7f19f406de50 2023-10-06T02:32:41.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.860+0000 7f19ef7fe700 1 -- 172.21.15.31:0/3721659971 <== osd.5 v2:172.21.15.62:6816/85800 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f1a0806d6b0 con 0x7f19f406de50 2023-10-06T02:32:41.861 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T02:32:41.862 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T02:32:41.862 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T02:32:41.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.860+0000 7f1a0cd9d700 1 -- 172.21.15.31:0/3721659971 >> [v2:172.21.15.62:6816/85800,v1:172.21.15.62:6817/85800] conn(0x7f19f406de50 msgr2=0x7f19f4070280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:41.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.860+0000 7f1a0cd9d700 1 --2- 172.21.15.31:0/3721659971 >> [v2:172.21.15.62:6816/85800,v1:172.21.15.62:6817/85800] conn(0x7f19f406de50 0x7f19f4070280 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:41.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.860+0000 7f1a0cd9d700 1 -- 172.21.15.31:0/3721659971 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f19f40679e0 msgr2=0x7f19f4069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:41.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.860+0000 7f1a0cd9d700 1 --2- 172.21.15.31:0/3721659971 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f19f40679e0 0x7f19f4069e90 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f19f80011e0 tx=0x7f19f8009040 comp rx=0 tx=0).stop 2023-10-06T02:32:41.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.860+0000 7f1a0cd9d700 1 -- 172.21.15.31:0/3721659971 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a080c85d0 msgr2=0x7f1a0814eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:41.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.860+0000 7f1a0cd9d700 1 --2- 172.21.15.31:0/3721659971 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a080c85d0 0x7f1a0814eda0 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f1a00007f20 tx=0x7f1a00009860 comp rx=0 tx=0).stop 2023-10-06T02:32:41.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.860+0000 7f1a07fff700 1 -- 172.21.15.31:0/3721659971 reap_dead start 2023-10-06T02:32:41.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.860+0000 7f1a0cd9d700 1 -- 172.21.15.31:0/3721659971 shutdown_connections 2023-10-06T02:32:41.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.860+0000 7f1a0cd9d700 1 -- 172.21.15.31:0/3721659971 >> 172.21.15.31:0/3721659971 conn(0x7f1a0806c6a0 msgr2=0x7f1a080cab90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:41.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.861+0000 7f1a0cd9d700 1 -- 172.21.15.31:0/3721659971 shutdown_connections 2023-10-06T02:32:41.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:41.861+0000 7f1a0cd9d700 1 -- 172.21.15.31:0/3721659971 wait complete. 2023-10-06T02:32:41.882 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-10-06T02:32:42.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.072+0000 7f91b539d700 1 Processor -- start 2023-10-06T02:32:42.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.072+0000 7f91b539d700 1 -- start start 2023-10-06T02:32:42.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.074+0000 7f91b539d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f91b00af600 0x7f91b00c6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:42.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.074+0000 7f91b539d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91b00c6d50 con 0x7f91b0140e40 2023-10-06T02:32:42.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.074+0000 7f91b539d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91b00c7030 con 0x7f91b0140a70 2023-10-06T02:32:42.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.074+0000 7f91b539d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91b00c7310 con 0x7f91b00af600 2023-10-06T02:32:42.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.074+0000 7f91af7fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f91b0140e40 0x7f91b00aeef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:55906/0 (socket says 172.21.15.31:55906) 2023-10-06T02:32:42.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.074+0000 7f91af7fe700 1 -- 172.21.15.31:0/695568369 learned_addr learned my addr 172.21.15.31:0/695568369 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:32:42.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.074+0000 7f91b4b9c700 1 --2- 172.21.15.31:0/695568369 >> v2:172.21.15.62:3300/0 conn(0x7f91b00af600 0x7f91b00c6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:42.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.074+0000 7f91aeffd700 1 -- 172.21.15.31:0/695568369 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2612946504 0 0) 0x7f91b00c6d50 con 0x7f91b0140e40 2023-10-06T02:32:42.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.074+0000 7f91aeffd700 1 -- 172.21.15.31:0/695568369 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9198003570 con 0x7f91b0140e40 2023-10-06T02:32:42.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.075+0000 7f91aeffd700 1 -- 172.21.15.31:0/695568369 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 301249375 0 0) 0x7f9198003570 con 0x7f91b0140e40 2023-10-06T02:32:42.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.075+0000 7f91aeffd700 1 -- 172.21.15.31:0/695568369 >> v2:172.21.15.62:3300/0 conn(0x7f91b00af600 msgr2=0x7f91b00c6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:42.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.075+0000 7f91aeffd700 1 --2- 172.21.15.31:0/695568369 >> v2:172.21.15.62:3300/0 conn(0x7f91b00af600 0x7f91b00c6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:42.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.075+0000 7f91aeffd700 1 -- 172.21.15.31:0/695568369 >> v1:172.21.15.62:6789/0 conn(0x7f91b0140a70 legacy=0x7f91b00c7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:42.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.075+0000 7f91aeffd700 1 -- 172.21.15.31:0/695568369 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91b00c75f0 con 0x7f91b0140e40 2023-10-06T02:32:42.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.075+0000 7f91aeffd700 1 -- 172.21.15.31:0/695568369 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1219684029 0 0) 0x7f91a40015f0 con 0x7f91b0140e40 2023-10-06T02:32:42.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.075+0000 7f91aeffd700 1 -- 172.21.15.31:0/695568369 >> v1:172.21.15.31:6789/0 conn(0x7f91b0140e40 legacy=0x7f91b00aeef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:42.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.075+0000 7f91aeffd700 1 --2- 172.21.15.31:0/695568369 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9198003ab0 0x7f9198003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:42.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.075+0000 7f91aeffd700 1 --2- 172.21.15.31:0/695568369 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9198004810 0x7f9198004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:42.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.075+0000 7f91aeffd700 1 --2- 172.21.15.31:0/695568369 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9198005210 0x7f919800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:42.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.075+0000 7f91aeffd700 1 -- 172.21.15.31:0/695568369 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91b00c7310 con 0x7f9198005210 2023-10-06T02:32:42.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.075+0000 7f91aeffd700 1 -- 172.21.15.31:0/695568369 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f919800c130 con 0x7f9198003ab0 2023-10-06T02:32:42.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.075+0000 7f91aeffd700 1 -- 172.21.15.31:0/695568369 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f919800c2f0 con 0x7f9198004810 2023-10-06T02:32:42.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.075+0000 7f91b4b9c700 1 --2- 172.21.15.31:0/695568369 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9198005210 0x7f919800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:42.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.075+0000 7f91af7fe700 1 --2- 172.21.15.31:0/695568369 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9198004810 0x7f9198004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:42.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.075+0000 7f91affff700 1 --2- 172.21.15.31:0/695568369 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9198003ab0 0x7f9198003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:42.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.076+0000 7f91b4b9c700 1 -- 172.21.15.31:0/695568369 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9198004810 msgr2=0x7f9198004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:42.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.076+0000 7f91b4b9c700 1 --2- 172.21.15.31:0/695568369 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9198004810 0x7f9198004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:42.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.076+0000 7f91b4b9c700 1 -- 172.21.15.31:0/695568369 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9198003ab0 msgr2=0x7f9198003ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:42.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.076+0000 7f91b4b9c700 1 --2- 172.21.15.31:0/695568369 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9198003ab0 0x7f9198003ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:42.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.076+0000 7f91b4b9c700 1 -- 172.21.15.31:0/695568369 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9198003570 con 0x7f9198005210 2023-10-06T02:32:42.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.076+0000 7f91b4b9c700 1 -- 172.21.15.31:0/695568369 reap_dead start 2023-10-06T02:32:42.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.076+0000 7f91b4b9c700 1 --2- 172.21.15.31:0/695568369 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9198005210 0x7f919800ba60 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f91a800e210 tx=0x7f91a800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9325bc6258513e74 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:42.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.076+0000 7f91aeffd700 1 -- 172.21.15.31:0/695568369 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a800eac0 con 0x7f9198005210 2023-10-06T02:32:42.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.076+0000 7f91aeffd700 1 -- 172.21.15.31:0/695568369 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91a800f040 con 0x7f9198005210 2023-10-06T02:32:42.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.076+0000 7f91aeffd700 1 -- 172.21.15.31:0/695568369 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a800ee70 con 0x7f9198005210 2023-10-06T02:32:42.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.077+0000 7f91b539d700 1 -- 172.21.15.31:0/695568369 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9198005210 msgr2=0x7f919800ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:42.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.077+0000 7f91b539d700 1 --2- 172.21.15.31:0/695568369 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9198005210 0x7f919800ba60 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f91a800e210 tx=0x7f91a800b040 comp rx=0 tx=0).stop 2023-10-06T02:32:42.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.077+0000 7f91b539d700 1 -- 172.21.15.31:0/695568369 shutdown_connections 2023-10-06T02:32:42.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.077+0000 7f91b539d700 1 --2- 172.21.15.31:0/695568369 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9198005210 0x7f919800ba60 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:42.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.077+0000 7f91b539d700 1 -- 172.21.15.31:0/695568369 >> 172.21.15.31:0/695568369 conn(0x7f91b006d170 msgr2=0x7f91b00cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:42.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.077+0000 7f91b539d700 1 -- 172.21.15.31:0/695568369 shutdown_connections 2023-10-06T02:32:42.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.077+0000 7f91b539d700 1 -- 172.21.15.31:0/695568369 wait complete. 2023-10-06T02:32:42.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.078+0000 7f91b539d700 1 Processor -- start 2023-10-06T02:32:42.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.078+0000 7f91b539d700 1 -- start start 2023-10-06T02:32:42.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.079+0000 7f91b539d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9198005210 0x7f91b01595f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:42.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.079+0000 7f91b539d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f91b0157030 0x7f91b016a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:42.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.079+0000 7f91b539d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91b016a8e0 0x7f91b00c8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:42.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.079+0000 7f91b539d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91b015a520 con 0x7f91b016a8e0 2023-10-06T02:32:42.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.079+0000 7f91b539d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91b016b1f0 con 0x7f9198005210 2023-10-06T02:32:42.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.079+0000 7f91b539d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91b016af10 con 0x7f91b0157030 2023-10-06T02:32:42.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.079+0000 7f91b4b9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91b016a8e0 0x7f91b00c8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:42.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.080+0000 7f91b4b9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91b016a8e0 0x7f91b00c8ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50086/0 (socket says 172.21.15.31:50086) 2023-10-06T02:32:42.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.080+0000 7f91b4b9c700 1 -- 172.21.15.31:0/682650378 learned_addr learned my addr 172.21.15.31:0/682650378 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:42.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.080+0000 7f91affff700 1 --2- 172.21.15.31:0/682650378 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9198005210 0x7f91b01595f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:42.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.080+0000 7f91af7fe700 1 --2- 172.21.15.31:0/682650378 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f91b0157030 0x7f91b016a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:42.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.080+0000 7f91b4b9c700 1 -- 172.21.15.31:0/682650378 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f91b0157030 msgr2=0x7f91b016a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:42.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.080+0000 7f91b4b9c700 1 --2- 172.21.15.31:0/682650378 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f91b0157030 0x7f91b016a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:42.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.080+0000 7f91b4b9c700 1 -- 172.21.15.31:0/682650378 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9198005210 msgr2=0x7f91b01595f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:42.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.080+0000 7f91b4b9c700 1 --2- 172.21.15.31:0/682650378 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9198005210 0x7f91b01595f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:42.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.080+0000 7f91b4b9c700 1 -- 172.21.15.31:0/682650378 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91b0158b10 con 0x7f91b016a8e0 2023-10-06T02:32:42.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.080+0000 7f91b4b9c700 1 --2- 172.21.15.31:0/682650378 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91b016a8e0 0x7f91b00c8ac0 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f91a800afd0 tx=0x7f91a8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:42.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.080+0000 7f9197fff700 1 -- 172.21.15.31:0/682650378 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a801d660 con 0x7f91b016a8e0 2023-10-06T02:32:42.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.080+0000 7f9197fff700 1 -- 172.21.15.31:0/682650378 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91a8022070 con 0x7f91b016a8e0 2023-10-06T02:32:42.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.081+0000 7f9197fff700 1 -- 172.21.15.31:0/682650378 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a801da30 con 0x7f91b016a8e0 2023-10-06T02:32:42.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.081+0000 7f91b539d700 1 -- 172.21.15.31:0/682650378 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91b0158df0 con 0x7f91b016a8e0 2023-10-06T02:32:42.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.081+0000 7f91b539d700 1 -- 172.21.15.31:0/682650378 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91b01574a0 con 0x7f91b016a8e0 2023-10-06T02:32:42.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.082+0000 7f91b539d700 1 -- 172.21.15.31:0/682650378 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91b004c3c0 con 0x7f91b016a8e0 2023-10-06T02:32:42.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.082+0000 7f9197fff700 1 -- 172.21.15.31:0/682650378 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) v1 ==== 85709+0+0 (secure 0 0 0) 0x7f91a8003020 con 0x7f91b016a8e0 2023-10-06T02:32:42.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.083+0000 7f9197fff700 1 --2- 172.21.15.31:0/682650378 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f919c0679e0 0x7f919c069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:42.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.083+0000 7f9197fff700 1 -- 172.21.15.31:0/682650378 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5320+0+0 (secure 0 0 0) 0x7f91a8024070 con 0x7f91b016a8e0 2023-10-06T02:32:42.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.084+0000 7f91affff700 1 --2- 172.21.15.31:0/682650378 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f919c0679e0 0x7f919c069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:42.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.084+0000 7f91affff700 1 --2- 172.21.15.31:0/682650378 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f919c0679e0 0x7f919c069e90 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f91a0000c00 tx=0x7f91a0007d30 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:42.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.087+0000 7f9197fff700 1 -- 172.21.15.31:0/682650378 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f91a8053910 con 0x7f91b016a8e0 2023-10-06T02:32:42.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.290+0000 7f91b539d700 1 -- 172.21.15.31:0/682650378 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f91b00c78d0 con 0x7f919c0679e0 2023-10-06T02:32:42.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.291+0000 7f9197fff700 1 -- 172.21.15.31:0/682650378 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f91b00c78d0 con 0x7f919c0679e0 2023-10-06T02:32:42.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.294+0000 7f91b539d700 1 -- 172.21.15.31:0/682650378 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f919c0679e0 msgr2=0x7f919c069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:42.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.294+0000 7f91b539d700 1 --2- 172.21.15.31:0/682650378 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f919c0679e0 0x7f919c069e90 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f91a0000c00 tx=0x7f91a0007d30 comp rx=0 tx=0).stop 2023-10-06T02:32:42.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.294+0000 7f91b539d700 1 -- 172.21.15.31:0/682650378 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91b016a8e0 msgr2=0x7f91b00c8ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:42.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.294+0000 7f91b539d700 1 --2- 172.21.15.31:0/682650378 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91b016a8e0 0x7f91b00c8ac0 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f91a800afd0 tx=0x7f91a8019000 comp rx=0 tx=0).stop 2023-10-06T02:32:42.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.294+0000 7f91b539d700 1 -- 172.21.15.31:0/682650378 shutdown_connections 2023-10-06T02:32:42.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.294+0000 7f91b539d700 1 --2- 172.21.15.31:0/682650378 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91b016a8e0 0x7f91b00c8ac0 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:42.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.294+0000 7f91b539d700 1 --2- 172.21.15.31:0/682650378 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9198005210 0x7f91b01595f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:42.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.294+0000 7f91b539d700 1 --2- 172.21.15.31:0/682650378 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f919c0679e0 0x7f919c069e90 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:42.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.294+0000 7f91b539d700 1 --2- 172.21.15.31:0/682650378 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f91b0157030 0x7f91b016a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:42.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.294+0000 7f91b539d700 1 -- 172.21.15.31:0/682650378 >> 172.21.15.31:0/682650378 conn(0x7f91b006d170 msgr2=0x7f91b00cabf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:42.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.294+0000 7f91b539d700 1 -- 172.21.15.31:0/682650378 shutdown_connections 2023-10-06T02:32:42.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.294+0000 7f91b539d700 1 -- 172.21.15.31:0/682650378 wait complete. 2023-10-06T02:32:42.298 INFO:teuthology.orchestra.run.smithi031.stderr:instructed osd(s) 5 to deep-scrub 2023-10-06T02:32:42.313 INFO:tasks.ceph:Scrubbing osd.6 2023-10-06T02:32:42.313 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-10-06T02:32:42.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.503+0000 7fb95bfff700 1 Processor -- start 2023-10-06T02:32:42.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.503+0000 7fb95bfff700 1 -- start start 2023-10-06T02:32:42.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.505+0000 7fb95bfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb95c153660 0x7fb95c0c0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:42.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.505+0000 7fb95bfff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb95c153a30 0x7fb95c156d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:42.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.505+0000 7fb95bfff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb95c0c2200 con 0x7fb95c0c8980 2023-10-06T02:32:42.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.505+0000 7fb95bfff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb95c157250 con 0x7fb95c153660 2023-10-06T02:32:42.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.505+0000 7fb95bfff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb95c0c15f0 con 0x7fb95c153a30 2023-10-06T02:32:42.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.505+0000 7fb95affd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb95c153660 0x7fb95c0c0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:42.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.505+0000 7fb95b7fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fb95c0c8980 0x7fb95c0c8e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:55912/0 (socket says 172.21.15.31:55912) 2023-10-06T02:32:42.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.505+0000 7fb95b7fe700 1 -- 172.21.15.31:0/947502395 learned_addr learned my addr 172.21.15.31:0/947502395 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:32:42.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.505+0000 7fb95a7fc700 1 --2- 172.21.15.31:0/947502395 >> v2:172.21.15.62:3300/0 conn(0x7fb95c153a30 0x7fb95c156d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:42.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.506+0000 7fb95affd700 1 -- 172.21.15.31:0/947502395 >> v2:172.21.15.62:3300/0 conn(0x7fb95c153a30 msgr2=0x7fb95c156d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:42.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.506+0000 7fb95affd700 1 --2- 172.21.15.31:0/947502395 >> v2:172.21.15.62:3300/0 conn(0x7fb95c153a30 0x7fb95c156d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:42.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.506+0000 7fb95affd700 1 -- 172.21.15.31:0/947502395 >> v1:172.21.15.31:6789/0 conn(0x7fb95c0c8980 legacy=0x7fb95c0c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:42.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.506+0000 7fb95affd700 1 -- 172.21.15.31:0/947502395 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb95c0c18d0 con 0x7fb95c153660 2023-10-06T02:32:42.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.506+0000 7fb95affd700 1 --2- 172.21.15.31:0/947502395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb95c153660 0x7fb95c0c0ef0 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7fb95c0c31b0 tx=0x7fb94c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7cdedd8423ff7ef server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:42.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.506+0000 7fb959ffb700 1 -- 172.21.15.31:0/947502395 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb94c00b6b0 con 0x7fb95c153660 2023-10-06T02:32:42.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.506+0000 7fb959ffb700 1 -- 172.21.15.31:0/947502395 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb94c00b810 con 0x7fb95c153660 2023-10-06T02:32:42.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.506+0000 7fb959ffb700 1 -- 172.21.15.31:0/947502395 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb94c00bbb0 con 0x7fb95c153660 2023-10-06T02:32:42.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.507+0000 7fb95bfff700 1 -- 172.21.15.31:0/947502395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb95c153660 msgr2=0x7fb95c0c0ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:42.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.507+0000 7fb95bfff700 1 --2- 172.21.15.31:0/947502395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb95c153660 0x7fb95c0c0ef0 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7fb95c0c31b0 tx=0x7fb94c0091a0 comp rx=0 tx=0).stop 2023-10-06T02:32:42.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.507+0000 7fb95bfff700 1 -- 172.21.15.31:0/947502395 shutdown_connections 2023-10-06T02:32:42.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.507+0000 7fb95bfff700 1 --2- 172.21.15.31:0/947502395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb95c153660 0x7fb95c0c0ef0 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:42.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.507+0000 7fb95bfff700 1 --2- 172.21.15.31:0/947502395 >> v2:172.21.15.62:3300/0 conn(0x7fb95c153a30 0x7fb95c156d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:42.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.507+0000 7fb95bfff700 1 -- 172.21.15.31:0/947502395 >> 172.21.15.31:0/947502395 conn(0x7fb95c06c6a0 msgr2=0x7fb95c06caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:42.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.507+0000 7fb95bfff700 1 -- 172.21.15.31:0/947502395 shutdown_connections 2023-10-06T02:32:42.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.507+0000 7fb95bfff700 1 -- 172.21.15.31:0/947502395 wait complete. 2023-10-06T02:32:42.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.508+0000 7fb95bfff700 1 Processor -- start 2023-10-06T02:32:42.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.508+0000 7fb95bfff700 1 -- start start 2023-10-06T02:32:42.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.509+0000 7fb95bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb95c0c8980 0x7fb95c144a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:42.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.509+0000 7fb95bfff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb95c153660 0x7fb95c155fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:42.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.509+0000 7fb95bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb95c153a30 0x7fb95c1564e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:42.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.509+0000 7fb95bfff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb95c15c450 con 0x7fb95c153a30 2023-10-06T02:32:42.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.509+0000 7fb95bfff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb95c15c8e0 con 0x7fb95c0c8980 2023-10-06T02:32:42.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.509+0000 7fb95bfff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb95c15b840 con 0x7fb95c153660 2023-10-06T02:32:42.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.510+0000 7fb95b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb95c153a30 0x7fb95c1564e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:42.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.510+0000 7fb95b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb95c153a30 0x7fb95c1564e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50104/0 (socket says 172.21.15.31:50104) 2023-10-06T02:32:42.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.510+0000 7fb95b7fe700 1 -- 172.21.15.31:0/1356504125 learned_addr learned my addr 172.21.15.31:0/1356504125 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:42.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.510+0000 7fb95affd700 1 --2- 172.21.15.31:0/1356504125 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb95c0c8980 0x7fb95c144a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:42.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.510+0000 7fb95a7fc700 1 --2- 172.21.15.31:0/1356504125 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb95c153660 0x7fb95c155fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:42.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.510+0000 7fb95b7fe700 1 -- 172.21.15.31:0/1356504125 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb95c153660 msgr2=0x7fb95c155fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:42.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.510+0000 7fb95b7fe700 1 --2- 172.21.15.31:0/1356504125 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb95c153660 0x7fb95c155fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:42.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.510+0000 7fb95b7fe700 1 -- 172.21.15.31:0/1356504125 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb95c0c8980 msgr2=0x7fb95c144a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:42.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.510+0000 7fb95b7fe700 1 --2- 172.21.15.31:0/1356504125 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb95c0c8980 0x7fb95c144a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:42.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.510+0000 7fb95b7fe700 1 -- 172.21.15.31:0/1356504125 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb94c009420 con 0x7fb95c153a30 2023-10-06T02:32:42.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.510+0000 7fb95a7fc700 1 --2- 172.21.15.31:0/1356504125 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb95c153660 0x7fb95c155fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:32:42.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.510+0000 7fb95b7fe700 1 --2- 172.21.15.31:0/1356504125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb95c153a30 0x7fb95c1564e0 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7fb95400cc40 tx=0x7fb954001a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:42.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.510+0000 7fb942ffd700 1 -- 172.21.15.31:0/1356504125 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb95400a910 con 0x7fb95c153a30 2023-10-06T02:32:42.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.510+0000 7fb942ffd700 1 -- 172.21.15.31:0/1356504125 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb95400aa70 con 0x7fb95c153a30 2023-10-06T02:32:42.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.511+0000 7fb942ffd700 1 -- 172.21.15.31:0/1356504125 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb95400ae30 con 0x7fb95c153a30 2023-10-06T02:32:42.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.511+0000 7fb95bfff700 1 -- 172.21.15.31:0/1356504125 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb95c15bb20 con 0x7fb95c153a30 2023-10-06T02:32:42.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.511+0000 7fb95bfff700 1 -- 172.21.15.31:0/1356504125 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb95c15c0e0 con 0x7fb95c153a30 2023-10-06T02:32:42.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.511+0000 7fb95bfff700 1 -- 172.21.15.31:0/1356504125 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb95c04c3c0 con 0x7fb95c153a30 2023-10-06T02:32:42.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.513+0000 7fb942ffd700 1 -- 172.21.15.31:0/1356504125 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) v1 ==== 85709+0+0 (secure 0 0 0) 0x7fb954007300 con 0x7fb95c153a30 2023-10-06T02:32:42.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.513+0000 7fb942ffd700 1 --2- 172.21.15.31:0/1356504125 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb9480679e0 0x7fb948069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:42.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.514+0000 7fb942ffd700 1 -- 172.21.15.31:0/1356504125 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5320+0+0 (secure 0 0 0) 0x7fb9540832d0 con 0x7fb95c153a30 2023-10-06T02:32:42.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.514+0000 7fb942ffd700 1 --2- 172.21.15.31:0/1356504125 >> [v2:172.21.15.62:6808/85798,v1:172.21.15.62:6809/85798] conn(0x7fb94806de50 0x7fb948070280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:42.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.514+0000 7fb942ffd700 1 -- 172.21.15.31:0/1356504125 --> [v2:172.21.15.62:6808/85798,v1:172.21.15.62:6809/85798] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb948070950 con 0x7fb94806de50 2023-10-06T02:32:42.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.514+0000 7fb942ffd700 1 -- 172.21.15.31:0/1356504125 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=172) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb95408a220 con 0x7fb95c153a30 2023-10-06T02:32:42.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.514+0000 7fb95affd700 1 --2- 172.21.15.31:0/1356504125 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb9480679e0 0x7fb948069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:42.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.514+0000 7fb95a7fc700 1 --2- 172.21.15.31:0/1356504125 >> [v2:172.21.15.62:6808/85798,v1:172.21.15.62:6809/85798] conn(0x7fb94806de50 0x7fb948070280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:42.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.515+0000 7fb95affd700 1 --2- 172.21.15.31:0/1356504125 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb9480679e0 0x7fb948069e90 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fb94c0093f0 tx=0x7fb94c017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:42.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.515+0000 7fb95a7fc700 1 --2- 172.21.15.31:0/1356504125 >> [v2:172.21.15.62:6808/85798,v1:172.21.15.62:6809/85798] conn(0x7fb94806de50 0x7fb948070280 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:42.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.516+0000 7fb942ffd700 1 -- 172.21.15.31:0/1356504125 <== osd.6 v2:172.21.15.62:6808/85798 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fb948070950 con 0x7fb94806de50 2023-10-06T02:32:42.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.530+0000 7fb95bfff700 1 -- 172.21.15.31:0/1356504125 --> [v2:172.21.15.62:6808/85798,v1:172.21.15.62:6809/85798] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fb95c0c81d0 con 0x7fb94806de50 2023-10-06T02:32:42.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.531+0000 7fb942ffd700 1 -- 172.21.15.31:0/1356504125 <== osd.6 v2:172.21.15.62:6808/85798 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fb95c0c81d0 con 0x7fb94806de50 2023-10-06T02:32:42.532 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T02:32:42.532 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T02:32:42.532 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T02:32:42.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.531+0000 7fb95bfff700 1 -- 172.21.15.31:0/1356504125 >> [v2:172.21.15.62:6808/85798,v1:172.21.15.62:6809/85798] conn(0x7fb94806de50 msgr2=0x7fb948070280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:42.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.531+0000 7fb95bfff700 1 --2- 172.21.15.31:0/1356504125 >> [v2:172.21.15.62:6808/85798,v1:172.21.15.62:6809/85798] conn(0x7fb94806de50 0x7fb948070280 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:42.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.531+0000 7fb95bfff700 1 -- 172.21.15.31:0/1356504125 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb9480679e0 msgr2=0x7fb948069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:42.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.531+0000 7fb95bfff700 1 --2- 172.21.15.31:0/1356504125 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fb9480679e0 0x7fb948069e90 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fb94c0093f0 tx=0x7fb94c017000 comp rx=0 tx=0).stop 2023-10-06T02:32:42.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.531+0000 7fb95bfff700 1 -- 172.21.15.31:0/1356504125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb95c153a30 msgr2=0x7fb95c1564e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:42.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.531+0000 7fb95bfff700 1 --2- 172.21.15.31:0/1356504125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb95c153a30 0x7fb95c1564e0 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7fb95400cc40 tx=0x7fb954001a50 comp rx=0 tx=0).stop 2023-10-06T02:32:42.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.531+0000 7fb95b7fe700 1 -- 172.21.15.31:0/1356504125 reap_dead start 2023-10-06T02:32:42.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.531+0000 7fb95bfff700 1 -- 172.21.15.31:0/1356504125 shutdown_connections 2023-10-06T02:32:42.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.531+0000 7fb95bfff700 1 -- 172.21.15.31:0/1356504125 >> 172.21.15.31:0/1356504125 conn(0x7fb95c06c6a0 msgr2=0x7fb95c0b1770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:42.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.531+0000 7fb95bfff700 1 -- 172.21.15.31:0/1356504125 shutdown_connections 2023-10-06T02:32:42.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.531+0000 7fb95bfff700 1 -- 172.21.15.31:0/1356504125 wait complete. 2023-10-06T02:32:42.549 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-10-06T02:32:42.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.736+0000 7f40ca19d700 1 Processor -- start 2023-10-06T02:32:42.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.736+0000 7f40ca19d700 1 -- start start 2023-10-06T02:32:42.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.738+0000 7f40ca19d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f40c406e0a0 0x7f40c40c6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:42.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.738+0000 7f40ca19d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f40c4150380 0x7f40c406a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:42.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.738+0000 7f40ca19d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40c406a490 con 0x7f40c406f3a0 2023-10-06T02:32:42.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.738+0000 7f40ca19d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40c40c7490 con 0x7f40c4150380 2023-10-06T02:32:42.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.738+0000 7f40ca19d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40c40c7770 con 0x7f40c406e0a0 2023-10-06T02:32:42.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.738+0000 7f40c899a700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f40c406f3a0 0x7f40c414fc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:55918/0 (socket says 172.21.15.31:55918) 2023-10-06T02:32:42.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.738+0000 7f40c899a700 1 -- 172.21.15.31:0/4158328507 learned_addr learned my addr 172.21.15.31:0/4158328507 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:32:42.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.738+0000 7f40c999c700 1 --2- 172.21.15.31:0/4158328507 >> v2:172.21.15.31:3300/0 conn(0x7f40c4150380 0x7f40c406a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:42.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.738+0000 7f40c919b700 1 --2- 172.21.15.31:0/4158328507 >> v2:172.21.15.62:3300/0 conn(0x7f40c406e0a0 0x7f40c40c6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:42.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.738+0000 7f40bbfff700 1 -- 172.21.15.31:0/4158328507 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1421918376 0 0) 0x7f40c406a490 con 0x7f40c406f3a0 2023-10-06T02:32:42.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.738+0000 7f40bbfff700 1 -- 172.21.15.31:0/4158328507 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40b0003570 con 0x7f40c406f3a0 2023-10-06T02:32:42.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.739+0000 7f40c919b700 1 -- 172.21.15.31:0/4158328507 >> v2:172.21.15.31:3300/0 conn(0x7f40c4150380 msgr2=0x7f40c406a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:42.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.739+0000 7f40c919b700 1 --2- 172.21.15.31:0/4158328507 >> v2:172.21.15.31:3300/0 conn(0x7f40c4150380 0x7f40c406a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:42.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.739+0000 7f40c919b700 1 -- 172.21.15.31:0/4158328507 >> v1:172.21.15.31:6789/0 conn(0x7f40c406f3a0 legacy=0x7f40c414fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:42.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.739+0000 7f40c919b700 1 -- 172.21.15.31:0/4158328507 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40c40c7a50 con 0x7f40c406e0a0 2023-10-06T02:32:42.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.739+0000 7f40c999c700 1 --2- 172.21.15.31:0/4158328507 >> v2:172.21.15.31:3300/0 conn(0x7f40c4150380 0x7f40c406a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:32:42.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.739+0000 7f40c919b700 1 --2- 172.21.15.31:0/4158328507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40c406e0a0 0x7f40c40c6b20 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f40c4069230 tx=0x7f40c000a330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88a02efa765a1b99 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:42.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.739+0000 7f40bbfff700 1 -- 172.21.15.31:0/4158328507 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c000aea0 con 0x7f40c406e0a0 2023-10-06T02:32:42.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.739+0000 7f40bbfff700 1 -- 172.21.15.31:0/4158328507 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40c000c970 con 0x7f40c406e0a0 2023-10-06T02:32:42.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.739+0000 7f40bbfff700 1 -- 172.21.15.31:0/4158328507 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c000cd10 con 0x7f40c406e0a0 2023-10-06T02:32:42.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.740+0000 7f40ca19d700 1 -- 172.21.15.31:0/4158328507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40c406e0a0 msgr2=0x7f40c40c6b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:42.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.740+0000 7f40ca19d700 1 --2- 172.21.15.31:0/4158328507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40c406e0a0 0x7f40c40c6b20 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f40c4069230 tx=0x7f40c000a330 comp rx=0 tx=0).stop 2023-10-06T02:32:42.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.740+0000 7f40ca19d700 1 -- 172.21.15.31:0/4158328507 shutdown_connections 2023-10-06T02:32:42.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.740+0000 7f40ca19d700 1 --2- 172.21.15.31:0/4158328507 >> v2:172.21.15.31:3300/0 conn(0x7f40c4150380 0x7f40c406a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:42.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.740+0000 7f40ca19d700 1 --2- 172.21.15.31:0/4158328507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40c406e0a0 0x7f40c40c6b20 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:42.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.740+0000 7f40ca19d700 1 -- 172.21.15.31:0/4158328507 >> 172.21.15.31:0/4158328507 conn(0x7f40c406d170 msgr2=0x7f40c40cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:42.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.740+0000 7f40ca19d700 1 -- 172.21.15.31:0/4158328507 shutdown_connections 2023-10-06T02:32:42.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.740+0000 7f40ca19d700 1 -- 172.21.15.31:0/4158328507 wait complete. 2023-10-06T02:32:42.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.741+0000 7f40ca19d700 1 Processor -- start 2023-10-06T02:32:42.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.741+0000 7f40ca19d700 1 -- start start 2023-10-06T02:32:42.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.742+0000 7f40ca19d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f40c406e0a0 0x7f40c4158e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:42.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.742+0000 7f40ca19d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40c406f3a0 0x7f40c40c17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:42.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.742+0000 7f40ca19d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40c4150380 0x7f40c414ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:42.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.742+0000 7f40ca19d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c40c2be0 con 0x7f40c4150380 2023-10-06T02:32:42.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.742+0000 7f40ca19d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c40c1d30 con 0x7f40c406f3a0 2023-10-06T02:32:42.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.742+0000 7f40ca19d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40c40c1fd0 con 0x7f40c406e0a0 2023-10-06T02:32:42.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.743+0000 7f40c999c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40c4150380 0x7f40c414ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:42.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.743+0000 7f40c999c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40c4150380 0x7f40c414ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50112/0 (socket says 172.21.15.31:50112) 2023-10-06T02:32:42.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.743+0000 7f40c999c700 1 -- 172.21.15.31:0/2651801560 learned_addr learned my addr 172.21.15.31:0/2651801560 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:42.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.743+0000 7f40c899a700 1 --2- 172.21.15.31:0/2651801560 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40c406f3a0 0x7f40c40c17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:42.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.743+0000 7f40c919b700 1 --2- 172.21.15.31:0/2651801560 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f40c406e0a0 0x7f40c4158e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:42.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.743+0000 7f40c999c700 1 -- 172.21.15.31:0/2651801560 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f40c406e0a0 msgr2=0x7f40c4158e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:42.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.743+0000 7f40c999c700 1 --2- 172.21.15.31:0/2651801560 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f40c406e0a0 0x7f40c4158e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:42.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.743+0000 7f40c999c700 1 -- 172.21.15.31:0/2651801560 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40c406f3a0 msgr2=0x7f40c40c17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:42.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.743+0000 7f40c999c700 1 --2- 172.21.15.31:0/2651801560 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40c406f3a0 0x7f40c40c17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:42.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.743+0000 7f40c999c700 1 -- 172.21.15.31:0/2651801560 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40bc009a20 con 0x7f40c4150380 2023-10-06T02:32:42.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.743+0000 7f40c899a700 1 --2- 172.21.15.31:0/2651801560 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40c406f3a0 0x7f40c40c17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:32:42.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.743+0000 7f40c919b700 1 --2- 172.21.15.31:0/2651801560 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f40c406e0a0 0x7f40c4158e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:32:42.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.743+0000 7f40c999c700 1 --2- 172.21.15.31:0/2651801560 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40c4150380 0x7f40c414ae70 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f40bc009f50 tx=0x7f40bc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:42.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.744+0000 7f40b97fa700 1 -- 172.21.15.31:0/2651801560 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40bc00cc50 con 0x7f40c4150380 2023-10-06T02:32:42.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.744+0000 7f40b97fa700 1 -- 172.21.15.31:0/2651801560 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40bc00f040 con 0x7f40c4150380 2023-10-06T02:32:42.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.744+0000 7f40ca19d700 1 -- 172.21.15.31:0/2651801560 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40c000a5d0 con 0x7f40c4150380 2023-10-06T02:32:42.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.744+0000 7f40b97fa700 1 -- 172.21.15.31:0/2651801560 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40bc014480 con 0x7f40c4150380 2023-10-06T02:32:42.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.744+0000 7f40ca19d700 1 -- 172.21.15.31:0/2651801560 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40c40c2590 con 0x7f40c4150380 2023-10-06T02:32:42.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.745+0000 7f40ca19d700 1 -- 172.21.15.31:0/2651801560 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40c40c2d50 con 0x7f40c4150380 2023-10-06T02:32:42.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.746+0000 7f40b97fa700 1 -- 172.21.15.31:0/2651801560 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) v1 ==== 85709+0+0 (secure 0 0 0) 0x7f40bc014870 con 0x7f40c4150380 2023-10-06T02:32:42.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.747+0000 7f40b97fa700 1 --2- 172.21.15.31:0/2651801560 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f40a80679e0 0x7f40a8069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:42.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.747+0000 7f40b97fa700 1 -- 172.21.15.31:0/2651801560 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5320+0+0 (secure 0 0 0) 0x7f40bc083f20 con 0x7f40c4150380 2023-10-06T02:32:42.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.747+0000 7f40c919b700 1 --2- 172.21.15.31:0/2651801560 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f40a80679e0 0x7f40a8069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:42.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.748+0000 7f40c919b700 1 --2- 172.21.15.31:0/2651801560 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f40a80679e0 0x7f40a8069e90 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f40c000a370 tx=0x7f40c001a000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:42.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.751+0000 7f40b97fa700 1 -- 172.21.15.31:0/2651801560 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f40bc052930 con 0x7f40c4150380 2023-10-06T02:32:42.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.953+0000 7f40ca19d700 1 -- 172.21.15.31:0/2651801560 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f40c40648e0 con 0x7f40a80679e0 2023-10-06T02:32:42.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.954+0000 7f40b97fa700 1 -- 172.21.15.31:0/2651801560 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f40c40648e0 con 0x7f40a80679e0 2023-10-06T02:32:42.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.956+0000 7f40ca19d700 1 -- 172.21.15.31:0/2651801560 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f40a80679e0 msgr2=0x7f40a8069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:42.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.956+0000 7f40ca19d700 1 --2- 172.21.15.31:0/2651801560 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f40a80679e0 0x7f40a8069e90 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f40c000a370 tx=0x7f40c001a000 comp rx=0 tx=0).stop 2023-10-06T02:32:42.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.956+0000 7f40ca19d700 1 -- 172.21.15.31:0/2651801560 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40c4150380 msgr2=0x7f40c414ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:42.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.956+0000 7f40ca19d700 1 --2- 172.21.15.31:0/2651801560 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40c4150380 0x7f40c414ae70 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f40bc009f50 tx=0x7f40bc00b040 comp rx=0 tx=0).stop 2023-10-06T02:32:42.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.956+0000 7f40ca19d700 1 -- 172.21.15.31:0/2651801560 shutdown_connections 2023-10-06T02:32:42.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.956+0000 7f40ca19d700 1 --2- 172.21.15.31:0/2651801560 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40c4150380 0x7f40c414ae70 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:42.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.956+0000 7f40ca19d700 1 --2- 172.21.15.31:0/2651801560 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f40a80679e0 0x7f40a8069e90 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:42.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.956+0000 7f40ca19d700 1 --2- 172.21.15.31:0/2651801560 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f40c406e0a0 0x7f40c4158e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:42.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.956+0000 7f40ca19d700 1 --2- 172.21.15.31:0/2651801560 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40c406f3a0 0x7f40c40c17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:42.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.956+0000 7f40ca19d700 1 -- 172.21.15.31:0/2651801560 >> 172.21.15.31:0/2651801560 conn(0x7f40c406d170 msgr2=0x7f40c40c8b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:42.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.956+0000 7f40ca19d700 1 -- 172.21.15.31:0/2651801560 shutdown_connections 2023-10-06T02:32:42.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:42.956+0000 7f40ca19d700 1 -- 172.21.15.31:0/2651801560 wait complete. 2023-10-06T02:32:42.959 INFO:teuthology.orchestra.run.smithi031.stderr:instructed osd(s) 6 to deep-scrub 2023-10-06T02:32:42.975 INFO:tasks.ceph:Scrubbing osd.7 2023-10-06T02:32:42.975 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-10-06T02:32:43.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.165+0000 7f62a919d700 1 Processor -- start 2023-10-06T02:32:43.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.166+0000 7f62a919d700 1 -- start start 2023-10-06T02:32:43.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.167+0000 7f62a919d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f62a40c85d0 0x7f62a40cc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:43.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.167+0000 7f62a919d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62a41571b0 con 0x7f62a40c2a90 2023-10-06T02:32:43.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.167+0000 7f62a919d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62a41579b0 con 0x7f62a40c8200 2023-10-06T02:32:43.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.167+0000 7f62a919d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62a40c1390 con 0x7f62a40c85d0 2023-10-06T02:32:43.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.167+0000 7f62a899c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f62a40c85d0 0x7f62a40cc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:43.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.167+0000 7f62a37fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f62a40c2a90 0x7f62a4156aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:55932/0 (socket says 172.21.15.31:55932) 2023-10-06T02:32:43.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.167+0000 7f62a37fe700 1 -- 172.21.15.31:0/4252090691 learned_addr learned my addr 172.21.15.31:0/4252090691 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:32:43.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.168+0000 7f62a899c700 1 -- 172.21.15.31:0/4252090691 >> v1:172.21.15.62:6789/0 conn(0x7f62a40c8200 legacy=0x7f62a40c1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:43.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.168+0000 7f62a899c700 1 -- 172.21.15.31:0/4252090691 >> v1:172.21.15.31:6789/0 conn(0x7f62a40c2a90 legacy=0x7f62a4156aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:43.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.168+0000 7f62a899c700 1 -- 172.21.15.31:0/4252090691 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62a41573f0 con 0x7f62a40c85d0 2023-10-06T02:32:43.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.168+0000 7f62a2ffd700 1 -- 172.21.15.31:0/4252090691 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2350575727 0 0) 0x7f6298001110 con 0x7f62a40c2a90 2023-10-06T02:32:43.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.168+0000 7f62a899c700 1 --2- 172.21.15.31:0/4252090691 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62a40c85d0 0x7f62a40cc2f0 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f629c009e60 tx=0x7f629c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b58942c1b6fb332 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:43.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.168+0000 7f62a2ffd700 1 -- 172.21.15.31:0/4252090691 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629c00e070 con 0x7f62a40c85d0 2023-10-06T02:32:43.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.168+0000 7f62a2ffd700 1 -- 172.21.15.31:0/4252090691 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f629c00bc30 con 0x7f62a40c85d0 2023-10-06T02:32:43.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.168+0000 7f62a2ffd700 1 -- 172.21.15.31:0/4252090691 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629c012430 con 0x7f62a40c85d0 2023-10-06T02:32:43.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.169+0000 7f62a919d700 1 -- 172.21.15.31:0/4252090691 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62a40c85d0 msgr2=0x7f62a40cc2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:43.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.169+0000 7f62a919d700 1 --2- 172.21.15.31:0/4252090691 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62a40c85d0 0x7f62a40cc2f0 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f629c009e60 tx=0x7f629c009780 comp rx=0 tx=0).stop 2023-10-06T02:32:43.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.169+0000 7f62a919d700 1 -- 172.21.15.31:0/4252090691 shutdown_connections 2023-10-06T02:32:43.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.169+0000 7f62a919d700 1 --2- 172.21.15.31:0/4252090691 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62a40c85d0 0x7f62a40cc2f0 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:43.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.169+0000 7f62a919d700 1 -- 172.21.15.31:0/4252090691 >> 172.21.15.31:0/4252090691 conn(0x7f62a406c6a0 msgr2=0x7f62a406caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:43.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.169+0000 7f62a919d700 1 -- 172.21.15.31:0/4252090691 shutdown_connections 2023-10-06T02:32:43.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.169+0000 7f62a919d700 1 -- 172.21.15.31:0/4252090691 wait complete. 2023-10-06T02:32:43.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.170+0000 7f62a919d700 1 Processor -- start 2023-10-06T02:32:43.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.171+0000 7f62a919d700 1 -- start start 2023-10-06T02:32:43.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.171+0000 7f62a919d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62a40c2a90 0x7f62a4150210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:43.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.171+0000 7f62a919d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f62a40c8200 0x7f62a40ab740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:43.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.172+0000 7f62a919d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62a40c85d0 0x7f62a414a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:43.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.172+0000 7f62a919d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62a40a9cf0 con 0x7f62a40c85d0 2023-10-06T02:32:43.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.172+0000 7f62a919d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62a40abc80 con 0x7f62a40c2a90 2023-10-06T02:32:43.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.172+0000 7f62a919d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62a40ac1a0 con 0x7f62a40c8200 2023-10-06T02:32:43.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.172+0000 7f62a899c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62a40c85d0 0x7f62a414a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:43.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.172+0000 7f62a899c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62a40c85d0 0x7f62a414a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50136/0 (socket says 172.21.15.31:50136) 2023-10-06T02:32:43.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.172+0000 7f62a899c700 1 -- 172.21.15.31:0/3239033720 learned_addr learned my addr 172.21.15.31:0/3239033720 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:43.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.172+0000 7f62a37fe700 1 --2- 172.21.15.31:0/3239033720 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f62a40c8200 0x7f62a40ab740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:43.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.172+0000 7f62a3fff700 1 --2- 172.21.15.31:0/3239033720 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62a40c2a90 0x7f62a4150210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:43.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.172+0000 7f62a899c700 1 -- 172.21.15.31:0/3239033720 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f62a40c8200 msgr2=0x7f62a40ab740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:43.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.172+0000 7f62a899c700 1 --2- 172.21.15.31:0/3239033720 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f62a40c8200 0x7f62a40ab740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:43.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.172+0000 7f62a899c700 1 -- 172.21.15.31:0/3239033720 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62a40c2a90 msgr2=0x7f62a4150210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:43.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.172+0000 7f62a899c700 1 --2- 172.21.15.31:0/3239033720 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62a40c2a90 0x7f62a4150210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:43.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.172+0000 7f62a899c700 1 -- 172.21.15.31:0/3239033720 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f629c009a20 con 0x7f62a40c85d0 2023-10-06T02:32:43.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.172+0000 7f62a37fe700 1 --2- 172.21.15.31:0/3239033720 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f62a40c8200 0x7f62a40ab740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:32:43.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.172+0000 7f62a3fff700 1 --2- 172.21.15.31:0/3239033720 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62a40c2a90 0x7f62a4150210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:32:43.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.173+0000 7f62a899c700 1 --2- 172.21.15.31:0/3239033720 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62a40c85d0 0x7f62a414a7a0 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f62a40697c0 tx=0x7f629c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:43.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.173+0000 7f628bfff700 1 -- 172.21.15.31:0/3239033720 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629c01b630 con 0x7f62a40c85d0 2023-10-06T02:32:43.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.173+0000 7f628bfff700 1 -- 172.21.15.31:0/3239033720 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f629c01b790 con 0x7f62a40c85d0 2023-10-06T02:32:43.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.173+0000 7f628bfff700 1 -- 172.21.15.31:0/3239033720 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629c01bb50 con 0x7f62a40c85d0 2023-10-06T02:32:43.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.173+0000 7f62a919d700 1 -- 172.21.15.31:0/3239033720 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62a40ac480 con 0x7f62a40c85d0 2023-10-06T02:32:43.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.173+0000 7f62a919d700 1 -- 172.21.15.31:0/3239033720 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62a40ac760 con 0x7f62a40c85d0 2023-10-06T02:32:43.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.174+0000 7f62a919d700 1 -- 172.21.15.31:0/3239033720 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f62a404c3c0 con 0x7f62a40c85d0 2023-10-06T02:32:43.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.175+0000 7f628bfff700 1 -- 172.21.15.31:0/3239033720 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) v1 ==== 85709+0+0 (secure 0 0 0) 0x7f629c01c400 con 0x7f62a40c85d0 2023-10-06T02:32:43.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.176+0000 7f628bfff700 1 --2- 172.21.15.31:0/3239033720 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f62900679e0 0x7f6290069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:43.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.176+0000 7f628bfff700 1 -- 172.21.15.31:0/3239033720 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5320+0+0 (secure 0 0 0) 0x7f629c085060 con 0x7f62a40c85d0 2023-10-06T02:32:43.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.176+0000 7f628bfff700 1 --2- 172.21.15.31:0/3239033720 >> [v2:172.21.15.62:6824/85843,v1:172.21.15.62:6825/85843] conn(0x7f629006de50 0x7f6290070280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:43.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.176+0000 7f628bfff700 1 -- 172.21.15.31:0/3239033720 --> [v2:172.21.15.62:6824/85843,v1:172.21.15.62:6825/85843] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6290070950 con 0x7f629006de50 2023-10-06T02:32:43.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.176+0000 7f628bfff700 1 -- 172.21.15.31:0/3239033720 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=172) v2 ==== 24+0+0 (secure 0 0 0) 0x7f629c08c220 con 0x7f62a40c85d0 2023-10-06T02:32:43.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.176+0000 7f62a3fff700 1 --2- 172.21.15.31:0/3239033720 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f62900679e0 0x7f6290069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:43.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.176+0000 7f62a37fe700 1 --2- 172.21.15.31:0/3239033720 >> [v2:172.21.15.62:6824/85843,v1:172.21.15.62:6825/85843] conn(0x7f629006de50 0x7f6290070280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:43.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.177+0000 7f62a3fff700 1 --2- 172.21.15.31:0/3239033720 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f62900679e0 0x7f6290069e90 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f6294001840 tx=0x7f6294009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:43.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.177+0000 7f62a37fe700 1 --2- 172.21.15.31:0/3239033720 >> [v2:172.21.15.62:6824/85843,v1:172.21.15.62:6825/85843] conn(0x7f629006de50 0x7f6290070280 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:43.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.178+0000 7f628bfff700 1 -- 172.21.15.31:0/3239033720 <== osd.7 v2:172.21.15.62:6824/85843 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f6290070950 con 0x7f629006de50 2023-10-06T02:32:43.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.192+0000 7f62a919d700 1 -- 172.21.15.31:0/3239033720 --> [v2:172.21.15.62:6824/85843,v1:172.21.15.62:6825/85843] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f62a40c2280 con 0x7f629006de50 2023-10-06T02:32:43.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.193+0000 7f628bfff700 1 -- 172.21.15.31:0/3239033720 <== osd.7 v2:172.21.15.62:6824/85843 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f62a40c2280 con 0x7f629006de50 2023-10-06T02:32:43.195 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T02:32:43.195 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T02:32:43.195 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T02:32:43.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.193+0000 7f62a919d700 1 -- 172.21.15.31:0/3239033720 >> [v2:172.21.15.62:6824/85843,v1:172.21.15.62:6825/85843] conn(0x7f629006de50 msgr2=0x7f6290070280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:43.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.193+0000 7f62a919d700 1 --2- 172.21.15.31:0/3239033720 >> [v2:172.21.15.62:6824/85843,v1:172.21.15.62:6825/85843] conn(0x7f629006de50 0x7f6290070280 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:43.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.194+0000 7f62a919d700 1 -- 172.21.15.31:0/3239033720 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f62900679e0 msgr2=0x7f6290069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:43.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.194+0000 7f62a919d700 1 --2- 172.21.15.31:0/3239033720 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f62900679e0 0x7f6290069e90 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f6294001840 tx=0x7f6294009040 comp rx=0 tx=0).stop 2023-10-06T02:32:43.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.194+0000 7f62a919d700 1 -- 172.21.15.31:0/3239033720 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62a40c85d0 msgr2=0x7f62a414a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:43.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.194+0000 7f62a919d700 1 --2- 172.21.15.31:0/3239033720 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62a40c85d0 0x7f62a414a7a0 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f62a40697c0 tx=0x7f629c019000 comp rx=0 tx=0).stop 2023-10-06T02:32:43.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.194+0000 7f62a899c700 1 -- 172.21.15.31:0/3239033720 reap_dead start 2023-10-06T02:32:43.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.194+0000 7f62a919d700 1 -- 172.21.15.31:0/3239033720 shutdown_connections 2023-10-06T02:32:43.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.194+0000 7f62a919d700 1 -- 172.21.15.31:0/3239033720 >> 172.21.15.31:0/3239033720 conn(0x7f62a406c6a0 msgr2=0x7f62a40ca050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:43.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.194+0000 7f62a919d700 1 -- 172.21.15.31:0/3239033720 shutdown_connections 2023-10-06T02:32:43.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.194+0000 7f62a919d700 1 -- 172.21.15.31:0/3239033720 wait complete. 2023-10-06T02:32:43.212 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-10-06T02:32:43.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.399+0000 7fab9f59e700 1 Processor -- start 2023-10-06T02:32:43.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.399+0000 7fab9f59e700 1 -- start start 2023-10-06T02:32:43.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.401+0000 7fab9f59e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faba006f3a0 0x7faba014fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:43.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.401+0000 7fab9f59e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7faba006e470 0x7faba01501b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:43.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.401+0000 7fab9f59e700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faba006a490 con 0x7faba006e0a0 2023-10-06T02:32:43.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.401+0000 7fab9f59e700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faba00c7230 con 0x7faba006e470 2023-10-06T02:32:43.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.401+0000 7fab9f59e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faba006b930 con 0x7faba006f3a0 2023-10-06T02:32:43.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.401+0000 7fab9e59c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7faba006e0a0 0x7faba00c6b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:55938/0 (socket says 172.21.15.31:55938) 2023-10-06T02:32:43.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.401+0000 7fab9e59c700 1 -- 172.21.15.31:0/525952458 learned_addr learned my addr 172.21.15.31:0/525952458 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:32:43.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.401+0000 7fab9ed9d700 1 --2- 172.21.15.31:0/525952458 >> v2:172.21.15.31:3300/0 conn(0x7faba006e470 0x7faba01501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:43.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.401+0000 7fab9dd9b700 1 --2- 172.21.15.31:0/525952458 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faba006f3a0 0x7faba014fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:43.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.401+0000 7fab9d59a700 1 -- 172.21.15.31:0/525952458 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 724853481 0 0) 0x7faba006a490 con 0x7faba006e0a0 2023-10-06T02:32:43.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.401+0000 7fab9d59a700 1 -- 172.21.15.31:0/525952458 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab88003570 con 0x7faba006e0a0 2023-10-06T02:32:43.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.402+0000 7fab9ed9d700 1 -- 172.21.15.31:0/525952458 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faba006f3a0 msgr2=0x7faba014fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:43.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.402+0000 7fab9ed9d700 1 --2- 172.21.15.31:0/525952458 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faba006f3a0 0x7faba014fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:43.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.402+0000 7fab9ed9d700 1 -- 172.21.15.31:0/525952458 >> v1:172.21.15.31:6789/0 conn(0x7faba006e0a0 legacy=0x7faba00c6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:43.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.402+0000 7fab9ed9d700 1 -- 172.21.15.31:0/525952458 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faba00c7490 con 0x7faba006e470 2023-10-06T02:32:43.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.402+0000 7fab9ed9d700 1 --2- 172.21.15.31:0/525952458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faba006e470 0x7faba01501b0 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7fab98009e60 tx=0x7fab98009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8719c5213d57004 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:43.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.402+0000 7fab9d59a700 1 -- 172.21.15.31:0/525952458 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9800e070 con 0x7faba006e470 2023-10-06T02:32:43.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.402+0000 7fab9d59a700 1 -- 172.21.15.31:0/525952458 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab9800bab0 con 0x7faba006e470 2023-10-06T02:32:43.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.402+0000 7fab9d59a700 1 -- 172.21.15.31:0/525952458 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9800be70 con 0x7faba006e470 2023-10-06T02:32:43.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.403+0000 7fab9f59e700 1 -- 172.21.15.31:0/525952458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faba006e470 msgr2=0x7faba01501b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:43.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.403+0000 7fab9f59e700 1 --2- 172.21.15.31:0/525952458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faba006e470 0x7faba01501b0 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7fab98009e60 tx=0x7fab98009780 comp rx=0 tx=0).stop 2023-10-06T02:32:43.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.403+0000 7fab9f59e700 1 -- 172.21.15.31:0/525952458 shutdown_connections 2023-10-06T02:32:43.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.403+0000 7fab9f59e700 1 --2- 172.21.15.31:0/525952458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faba006e470 0x7faba01501b0 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:43.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.403+0000 7fab9f59e700 1 --2- 172.21.15.31:0/525952458 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faba006f3a0 0x7faba014fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:43.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.403+0000 7fab9f59e700 1 -- 172.21.15.31:0/525952458 >> 172.21.15.31:0/525952458 conn(0x7faba006d170 msgr2=0x7faba00cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:43.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.403+0000 7fab9f59e700 1 -- 172.21.15.31:0/525952458 shutdown_connections 2023-10-06T02:32:43.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.403+0000 7fab9f59e700 1 -- 172.21.15.31:0/525952458 wait complete. 2023-10-06T02:32:43.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.404+0000 7fab9f59e700 1 Processor -- start 2023-10-06T02:32:43.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.404+0000 7fab9f59e700 1 -- start start 2023-10-06T02:32:43.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.405+0000 7fab9f59e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faba006e0a0 0x7faba0158260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:43.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.405+0000 7fab9f59e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faba006e470 0x7faba01587a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:43.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.405+0000 7fab9f59e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faba006f3a0 0x7faba0169bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:43.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.405+0000 7fab9f59e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faba0156a10 con 0x7faba006f3a0 2023-10-06T02:32:43.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.405+0000 7fab9f59e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faba0156cf0 con 0x7faba006e470 2023-10-06T02:32:43.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.405+0000 7fab9f59e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faba0156fd0 con 0x7faba006e0a0 2023-10-06T02:32:43.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.406+0000 7fab9ed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faba006f3a0 0x7faba0169bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:43.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.406+0000 7fab9e59c700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faba006e0a0 0x7faba0158260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:43.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.406+0000 7fab9ed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faba006f3a0 0x7faba0169bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50160/0 (socket says 172.21.15.31:50160) 2023-10-06T02:32:43.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.406+0000 7fab9ed9d700 1 -- 172.21.15.31:0/3668617348 learned_addr learned my addr 172.21.15.31:0/3668617348 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:43.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.406+0000 7fab9dd9b700 1 --2- 172.21.15.31:0/3668617348 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faba006e470 0x7faba01587a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:43.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.406+0000 7fab9ed9d700 1 -- 172.21.15.31:0/3668617348 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faba006e0a0 msgr2=0x7faba0158260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:43.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.406+0000 7fab9ed9d700 1 --2- 172.21.15.31:0/3668617348 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faba006e0a0 0x7faba0158260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:43.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.406+0000 7fab9ed9d700 1 -- 172.21.15.31:0/3668617348 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faba006e470 msgr2=0x7faba01587a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:43.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.406+0000 7fab9ed9d700 1 --2- 172.21.15.31:0/3668617348 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faba006e470 0x7faba01587a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:43.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.406+0000 7fab9ed9d700 1 -- 172.21.15.31:0/3668617348 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab98009a20 con 0x7faba006f3a0 2023-10-06T02:32:43.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.406+0000 7fab9ed9d700 1 --2- 172.21.15.31:0/3668617348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faba006f3a0 0x7faba0169bb0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7faba0071320 tx=0x7fab98019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:43.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.407+0000 7fab8e7fc700 1 -- 172.21.15.31:0/3668617348 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab98012e10 con 0x7faba006f3a0 2023-10-06T02:32:43.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.407+0000 7fab8e7fc700 1 -- 172.21.15.31:0/3668617348 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab98027470 con 0x7faba006f3a0 2023-10-06T02:32:43.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.407+0000 7fab8e7fc700 1 -- 172.21.15.31:0/3668617348 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab98027830 con 0x7faba006f3a0 2023-10-06T02:32:43.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.407+0000 7fab9f59e700 1 -- 172.21.15.31:0/3668617348 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faba01572b0 con 0x7faba006f3a0 2023-10-06T02:32:43.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.407+0000 7fab9f59e700 1 -- 172.21.15.31:0/3668617348 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faba0157ab0 con 0x7faba006f3a0 2023-10-06T02:32:43.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.408+0000 7fab9f59e700 1 -- 172.21.15.31:0/3668617348 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faba01577d0 con 0x7faba006f3a0 2023-10-06T02:32:43.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.409+0000 7fab8e7fc700 1 -- 172.21.15.31:0/3668617348 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) v1 ==== 85709+0+0 (secure 0 0 0) 0x7fab98024070 con 0x7faba006f3a0 2023-10-06T02:32:43.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.410+0000 7fab8e7fc700 1 --2- 172.21.15.31:0/3668617348 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fab800679e0 0x7fab80069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:43.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.410+0000 7fab8e7fc700 1 -- 172.21.15.31:0/3668617348 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5320+0+0 (secure 0 0 0) 0x7fab98085c30 con 0x7faba006f3a0 2023-10-06T02:32:43.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.410+0000 7fab9e59c700 1 --2- 172.21.15.31:0/3668617348 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fab800679e0 0x7fab80069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:43.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.411+0000 7fab9e59c700 1 --2- 172.21.15.31:0/3668617348 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fab800679e0 0x7fab80069e90 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fab90007fd0 tx=0x7fab90009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:43.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.414+0000 7fab8e7fc700 1 -- 172.21.15.31:0/3668617348 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fab98054580 con 0x7faba006f3a0 2023-10-06T02:32:43.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.615+0000 7fab9f59e700 1 -- 172.21.15.31:0/3668617348 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7faba004c3c0 con 0x7fab800679e0 2023-10-06T02:32:43.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.615+0000 7fab8e7fc700 1 -- 172.21.15.31:0/3668617348 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7faba004c3c0 con 0x7fab800679e0 2023-10-06T02:32:43.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.618+0000 7fab9f59e700 1 -- 172.21.15.31:0/3668617348 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fab800679e0 msgr2=0x7fab80069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:43.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.618+0000 7fab9f59e700 1 --2- 172.21.15.31:0/3668617348 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fab800679e0 0x7fab80069e90 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fab90007fd0 tx=0x7fab90009040 comp rx=0 tx=0).stop 2023-10-06T02:32:43.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.618+0000 7fab9f59e700 1 -- 172.21.15.31:0/3668617348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faba006f3a0 msgr2=0x7faba0169bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:43.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.618+0000 7fab9f59e700 1 --2- 172.21.15.31:0/3668617348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faba006f3a0 0x7faba0169bb0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7faba0071320 tx=0x7fab98019000 comp rx=0 tx=0).stop 2023-10-06T02:32:43.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.618+0000 7fab9f59e700 1 -- 172.21.15.31:0/3668617348 shutdown_connections 2023-10-06T02:32:43.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.618+0000 7fab9f59e700 1 --2- 172.21.15.31:0/3668617348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faba006f3a0 0x7faba0169bb0 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:43.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.618+0000 7fab9f59e700 1 --2- 172.21.15.31:0/3668617348 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7fab800679e0 0x7fab80069e90 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:43.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.618+0000 7fab9f59e700 1 --2- 172.21.15.31:0/3668617348 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faba006e0a0 0x7faba0158260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:43.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.618+0000 7fab9f59e700 1 --2- 172.21.15.31:0/3668617348 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faba006e470 0x7faba01587a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:43.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.618+0000 7fab9f59e700 1 -- 172.21.15.31:0/3668617348 >> 172.21.15.31:0/3668617348 conn(0x7faba006d170 msgr2=0x7faba00c9f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:43.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.618+0000 7fab9f59e700 1 -- 172.21.15.31:0/3668617348 shutdown_connections 2023-10-06T02:32:43.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.618+0000 7fab9f59e700 1 -- 172.21.15.31:0/3668617348 wait complete. 2023-10-06T02:32:43.621 INFO:teuthology.orchestra.run.smithi031.stderr:instructed osd(s) 7 to deep-scrub 2023-10-06T02:32:43.637 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T02:32:43.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.821+0000 7ff93f59e700 1 Processor -- start 2023-10-06T02:32:43.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.821+0000 7ff93f59e700 1 -- start start 2023-10-06T02:32:43.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.822+0000 7ff93f59e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff94013fb20 0x7ff94015a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:43.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.823+0000 7ff93f59e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff94006aaa0 0x7ff9400c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:43.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.823+0000 7ff93f59e700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff94015ae70 con 0x7ff94013fef0 2023-10-06T02:32:43.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.823+0000 7ff93f59e700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff94006af60 con 0x7ff94006aaa0 2023-10-06T02:32:43.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.823+0000 7ff93f59e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff94006b240 con 0x7ff94013fb20 2023-10-06T02:32:43.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.823+0000 7ff93dd9b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7ff94013fef0 0x7ff9400c7990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:55954/0 (socket says 172.21.15.31:55954) 2023-10-06T02:32:43.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.823+0000 7ff93ed9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff94006aaa0 0x7ff9400c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:43.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.823+0000 7ff93dd9b700 1 -- 172.21.15.31:0/2701491013 learned_addr learned my addr 172.21.15.31:0/2701491013 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:32:43.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.823+0000 7ff93e59c700 1 --2- 172.21.15.31:0/2701491013 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff94013fb20 0x7ff94015a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:43.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.823+0000 7ff93ed9d700 1 -- 172.21.15.31:0/2701491013 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff94013fb20 msgr2=0x7ff94015a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:32:43.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.823+0000 7ff93ed9d700 1 --2- 172.21.15.31:0/2701491013 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff94013fb20 0x7ff94015a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:43.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.823+0000 7ff93ed9d700 1 -- 172.21.15.31:0/2701491013 >> v1:172.21.15.31:6789/0 conn(0x7ff94013fef0 legacy=0x7ff9400c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:43.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.823+0000 7ff93ed9d700 1 -- 172.21.15.31:0/2701491013 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff94006b520 con 0x7ff94006aaa0 2023-10-06T02:32:43.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.824+0000 7ff93ed9d700 1 --2- 172.21.15.31:0/2701491013 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff94006aaa0 0x7ff9400c9d00 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7ff938009e60 tx=0x7ff938009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daa014c11ce8752e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:43.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.824+0000 7ff93d59a700 1 -- 172.21.15.31:0/2701491013 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff93800e070 con 0x7ff94006aaa0 2023-10-06T02:32:43.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.824+0000 7ff93d59a700 1 -- 172.21.15.31:0/2701491013 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff93800bc30 con 0x7ff94006aaa0 2023-10-06T02:32:43.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.824+0000 7ff93d59a700 1 -- 172.21.15.31:0/2701491013 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff938012430 con 0x7ff94006aaa0 2023-10-06T02:32:43.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.824+0000 7ff93f59e700 1 -- 172.21.15.31:0/2701491013 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff94006aaa0 msgr2=0x7ff9400c9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:43.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.824+0000 7ff93f59e700 1 --2- 172.21.15.31:0/2701491013 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff94006aaa0 0x7ff9400c9d00 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7ff938009e60 tx=0x7ff938009780 comp rx=0 tx=0).stop 2023-10-06T02:32:43.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.825+0000 7ff93f59e700 1 -- 172.21.15.31:0/2701491013 shutdown_connections 2023-10-06T02:32:43.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.825+0000 7ff93f59e700 1 --2- 172.21.15.31:0/2701491013 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff94006aaa0 0x7ff9400c9d00 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:43.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.825+0000 7ff93f59e700 1 --2- 172.21.15.31:0/2701491013 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff94013fb20 0x7ff94015a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:43.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.825+0000 7ff93f59e700 1 -- 172.21.15.31:0/2701491013 >> 172.21.15.31:0/2701491013 conn(0x7ff94006d170 msgr2=0x7ff9400cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:43.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.825+0000 7ff93f59e700 1 -- 172.21.15.31:0/2701491013 shutdown_connections 2023-10-06T02:32:43.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.825+0000 7ff93f59e700 1 -- 172.21.15.31:0/2701491013 wait complete. 2023-10-06T02:32:43.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.826+0000 7ff93f59e700 1 Processor -- start 2023-10-06T02:32:43.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.826+0000 7ff93f59e700 1 -- start start 2023-10-06T02:32:43.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.827+0000 7ff93f59e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff94006aaa0 0x7ff94013c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:43.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.827+0000 7ff93f59e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff94013fb20 0x7ff9400af890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:43.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.827+0000 7ff93f59e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff94013fef0 0x7ff9401696d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:43.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.827+0000 7ff93f59e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff94013cce0 con 0x7ff94006aaa0 2023-10-06T02:32:43.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.827+0000 7ff93f59e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9400ab740 con 0x7ff94013fef0 2023-10-06T02:32:43.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.827+0000 7ff93f59e700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9400ab990 con 0x7ff94013fb20 2023-10-06T02:32:43.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.827+0000 7ff93e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff94006aaa0 0x7ff94013c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:43.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.827+0000 7ff93e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff94006aaa0 0x7ff94013c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50182/0 (socket says 172.21.15.31:50182) 2023-10-06T02:32:43.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.827+0000 7ff93e59c700 1 -- 172.21.15.31:0/38731441 learned_addr learned my addr 172.21.15.31:0/38731441 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:32:43.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.827+0000 7ff93dd9b700 1 --2- 172.21.15.31:0/38731441 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff94013fb20 0x7ff9400af890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:43.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.828+0000 7ff93ed9d700 1 --2- 172.21.15.31:0/38731441 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff94013fef0 0x7ff9401696d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:43.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.828+0000 7ff93e59c700 1 -- 172.21.15.31:0/38731441 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff94013fb20 msgr2=0x7ff9400af890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:43.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.828+0000 7ff93e59c700 1 --2- 172.21.15.31:0/38731441 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff94013fb20 0x7ff9400af890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:43.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.828+0000 7ff93e59c700 1 -- 172.21.15.31:0/38731441 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff94013fef0 msgr2=0x7ff9401696d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:43.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.828+0000 7ff93e59c700 1 --2- 172.21.15.31:0/38731441 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff94013fef0 0x7ff9401696d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:43.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.828+0000 7ff93e59c700 1 -- 172.21.15.31:0/38731441 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff938009a20 con 0x7ff94006aaa0 2023-10-06T02:32:43.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.828+0000 7ff93e59c700 1 --2- 172.21.15.31:0/38731441 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff94006aaa0 0x7ff94013c7a0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7ff93000d890 tx=0x7ff930008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:43.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.828+0000 7ff92e7fc700 1 -- 172.21.15.31:0/38731441 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff93000b600 con 0x7ff94006aaa0 2023-10-06T02:32:43.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.828+0000 7ff92e7fc700 1 -- 172.21.15.31:0/38731441 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff93000b760 con 0x7ff94006aaa0 2023-10-06T02:32:43.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.828+0000 7ff93f59e700 1 -- 172.21.15.31:0/38731441 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9400abc70 con 0x7ff94006aaa0 2023-10-06T02:32:43.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.828+0000 7ff92e7fc700 1 -- 172.21.15.31:0/38731441 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff93000bae0 con 0x7ff94006aaa0 2023-10-06T02:32:43.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.828+0000 7ff93f59e700 1 -- 172.21.15.31:0/38731441 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9400ac230 con 0x7ff94006aaa0 2023-10-06T02:32:43.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.829+0000 7ff93f59e700 1 -- 172.21.15.31:0/38731441 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9400ac880 con 0x7ff94006aaa0 2023-10-06T02:32:43.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.830+0000 7ff92e7fc700 1 -- 172.21.15.31:0/38731441 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) v1 ==== 85709+0+0 (secure 0 0 0) 0x7ff93001e460 con 0x7ff94006aaa0 2023-10-06T02:32:43.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.831+0000 7ff92e7fc700 1 --2- 172.21.15.31:0/38731441 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff9200679e0 0x7ff920069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:32:43.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.831+0000 7ff92e7fc700 1 -- 172.21.15.31:0/38731441 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5320+0+0 (secure 0 0 0) 0x7ff930083600 con 0x7ff94006aaa0 2023-10-06T02:32:43.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.832+0000 7ff93dd9b700 1 --2- 172.21.15.31:0/38731441 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff9200679e0 0x7ff920069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:32:43.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.832+0000 7ff93dd9b700 1 --2- 172.21.15.31:0/38731441 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff9200679e0 0x7ff920069e90 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7ff9400c9690 tx=0x7ff934009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:32:43.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:43.835+0000 7ff92e7fc700 1 -- 172.21.15.31:0/38731441 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff930052240 con 0x7ff94006aaa0 2023-10-06T02:32:44.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:44.034+0000 7ff93f59e700 1 -- 172.21.15.31:0/38731441 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff94004c3c0 con 0x7ff9200679e0 2023-10-06T02:32:44.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:44.037+0000 7ff92e7fc700 1 -- 172.21.15.31:0/38731441 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72303 (secure 0 0 0) 0x7ff94004c3c0 con 0x7ff9200679e0 2023-10-06T02:32:44.040 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:32:44.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:44.039+0000 7ff93f59e700 1 -- 172.21.15.31:0/38731441 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff9200679e0 msgr2=0x7ff920069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:44.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:44.039+0000 7ff93f59e700 1 --2- 172.21.15.31:0/38731441 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff9200679e0 0x7ff920069e90 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7ff9400c9690 tx=0x7ff934009040 comp rx=0 tx=0).stop 2023-10-06T02:32:44.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:44.039+0000 7ff93f59e700 1 -- 172.21.15.31:0/38731441 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff94006aaa0 msgr2=0x7ff94013c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:32:44.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:44.039+0000 7ff93f59e700 1 --2- 172.21.15.31:0/38731441 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff94006aaa0 0x7ff94013c7a0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7ff93000d890 tx=0x7ff930008040 comp rx=0 tx=0).stop 2023-10-06T02:32:44.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:44.039+0000 7ff93f59e700 1 -- 172.21.15.31:0/38731441 shutdown_connections 2023-10-06T02:32:44.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:44.039+0000 7ff93f59e700 1 --2- 172.21.15.31:0/38731441 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff94013fef0 0x7ff9401696d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:44.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:44.039+0000 7ff93f59e700 1 --2- 172.21.15.31:0/38731441 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff94006aaa0 0x7ff94013c7a0 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:44.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:44.039+0000 7ff93f59e700 1 --2- 172.21.15.31:0/38731441 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff9200679e0 0x7ff920069e90 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:44.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:44.039+0000 7ff93f59e700 1 --2- 172.21.15.31:0/38731441 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff94013fb20 0x7ff9400af890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:32:44.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:44.039+0000 7ff93f59e700 1 -- 172.21.15.31:0/38731441 >> 172.21.15.31:0/38731441 conn(0x7ff94006d170 msgr2=0x7ff9400c2720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:32:44.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:44.040+0000 7ff93f59e700 1 -- 172.21.15.31:0/38731441 shutdown_connections 2023-10-06T02:32:44.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:32:44.040+0000 7ff93f59e700 1 -- 172.21.15.31:0/38731441 wait complete. 2023-10-06T02:32:44.042 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-10-06T02:32:44.058 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":252,"stamp":"2023-10-06T02:32:42.531584+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":2433352,"kb_used_data":18184,"kb_used_omap":0,"kb_used_meta":2415104,"kb_avail":752541368,"statfs":{"total":773094113280,"available":770602360832,"internally_reserved":0,"allocated":18620416,"data_stored":9311912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2473066496},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.417027"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":137,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-06T02:32:29.366431+0000","last_change":"2023-10-06T02:23:01.992935+0000","last_active":"2023-10-06T02:32:29.366431+0000","last_peered":"2023-10-06T02:32:29.366431+0000","last_clean":"2023-10-06T02:32:29.366431+0000","last_became_active":"2023-10-06T02:23:01.992568+0000","last_became_peered":"2023-10-06T02:23:01.992568+0000","last_unstale":"2023-10-06T02:32:29.366431+0000","last_undegraded":"2023-10-06T02:32:29.366431+0000","last_fullsized":"2023-10-06T02:32:29.366431+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:06.212251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":138,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-06T02:32:28.707783+0000","last_change":"2023-10-06T02:23:01.989934+0000","last_active":"2023-10-06T02:32:28.707783+0000","last_peered":"2023-10-06T02:32:28.707783+0000","last_clean":"2023-10-06T02:32:28.707783+0000","last_became_active":"2023-10-06T02:23:01.989706+0000","last_became_peered":"2023-10-06T02:23:01.989706+0000","last_unstale":"2023-10-06T02:32:28.707783+0000","last_undegraded":"2023-10-06T02:32:28.707783+0000","last_fullsized":"2023-10-06T02:32:28.707783+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:06.212251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":131,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-06T02:32:29.229695+0000","last_change":"2023-10-06T02:23:02.054069+0000","last_active":"2023-10-06T02:32:29.229695+0000","last_peered":"2023-10-06T02:32:29.229695+0000","last_clean":"2023-10-06T02:32:29.229695+0000","last_became_active":"2023-10-06T02:23:02.053739+0000","last_became_peered":"2023-10-06T02:23:02.053739+0000","last_unstale":"2023-10-06T02:32:29.229695+0000","last_undegraded":"2023-10-06T02:32:29.229695+0000","last_fullsized":"2023-10-06T02:32:29.229695+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:57:48.822608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":131,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-06T02:32:29.229181+0000","last_change":"2023-10-06T02:23:01.990773+0000","last_active":"2023-10-06T02:32:29.229181+0000","last_peered":"2023-10-06T02:32:29.229181+0000","last_clean":"2023-10-06T02:32:29.229181+0000","last_became_active":"2023-10-06T02:23:01.990511+0000","last_became_peered":"2023-10-06T02:23:01.990511+0000","last_unstale":"2023-10-06T02:32:29.229181+0000","last_undegraded":"2023-10-06T02:32:29.229181+0000","last_fullsized":"2023-10-06T02:32:29.229181+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:42:54.797731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"76'128","reported_seq":323,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-06T02:32:29.229325+0000","last_change":"2023-10-06T02:22:55.032519+0000","last_active":"2023-10-06T02:32:29.229325+0000","last_peered":"2023-10-06T02:32:29.229325+0000","last_clean":"2023-10-06T02:32:29.229325+0000","last_became_active":"2023-10-06T02:22:55.031367+0000","last_became_peered":"2023-10-06T02:22:55.031367+0000","last_unstale":"2023-10-06T02:32:29.229325+0000","last_undegraded":"2023-10-06T02:32:29.229325+0000","last_fullsized":"2023-10-06T02:32:29.229325+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:22:53.309097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:22:53.309097+0000","last_clean_scrub_stamp":"2023-10-06T02:22:53.309097+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:53:46.520909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":162,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-06T02:32:31.117514+0000","last_change":"2023-10-06T02:23:01.988140+0000","last_active":"2023-10-06T02:32:31.117514+0000","last_peered":"2023-10-06T02:32:31.117514+0000","last_clean":"2023-10-06T02:32:31.117514+0000","last_became_active":"2023-10-06T02:23:01.987769+0000","last_became_peered":"2023-10-06T02:23:01.987769+0000","last_unstale":"2023-10-06T02:32:31.117514+0000","last_undegraded":"2023-10-06T02:32:31.117514+0000","last_fullsized":"2023-10-06T02:32:31.117514+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:20:08.391912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":148,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-06T02:32:28.708751+0000","last_change":"2023-10-06T02:23:02.054338+0000","last_active":"2023-10-06T02:32:28.708751+0000","last_peered":"2023-10-06T02:32:28.708751+0000","last_clean":"2023-10-06T02:32:28.708751+0000","last_became_active":"2023-10-06T02:23:02.053991+0000","last_became_peered":"2023-10-06T02:23:02.053991+0000","last_unstale":"2023-10-06T02:32:28.708751+0000","last_undegraded":"2023-10-06T02:32:28.708751+0000","last_fullsized":"2023-10-06T02:32:28.708751+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:39:30.192838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":162,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-06T02:32:31.115164+0000","last_change":"2023-10-06T02:23:01.988631+0000","last_active":"2023-10-06T02:32:31.115164+0000","last_peered":"2023-10-06T02:32:31.115164+0000","last_clean":"2023-10-06T02:32:31.115164+0000","last_became_active":"2023-10-06T02:23:01.988303+0000","last_became_peered":"2023-10-06T02:23:01.988303+0000","last_unstale":"2023-10-06T02:32:31.115164+0000","last_undegraded":"2023-10-06T02:32:31.115164+0000","last_fullsized":"2023-10-06T02:32:31.115164+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:38.211466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":138,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-06T02:32:28.707568+0000","last_change":"2023-10-06T02:23:02.054039+0000","last_active":"2023-10-06T02:32:28.707568+0000","last_peered":"2023-10-06T02:32:28.707568+0000","last_clean":"2023-10-06T02:32:28.707568+0000","last_became_active":"2023-10-06T02:23:02.053742+0000","last_became_peered":"2023-10-06T02:23:02.053742+0000","last_unstale":"2023-10-06T02:32:28.707568+0000","last_undegraded":"2023-10-06T02:32:28.707568+0000","last_fullsized":"2023-10-06T02:32:28.707568+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:23:00.968050+0000","last_clean_scrub_stamp":"2023-10-06T02:23:00.968050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:58:18.846475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574970,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304092,"kb_used_data":2708,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067748,"statfs":{"total":96636764160,"available":96325373952,"internally_reserved":0,"allocated":2772992,"data_stored":1606765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:32:15 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.68200000000000005,"15min":0.65300000000000002},"min":{"1min":0.38400000000000001,"5min":0.318,"15min":0.318},"max":{"1min":7.9160000000000004,"5min":7.9160000000000004,"15min":7.9160000000000004},"last":0.44600000000000001},{"interface":"front","average":{"1min":1.1759999999999999,"5min":0.73099999999999998,"15min":0.65100000000000002},"min":{"1min":0.47199999999999998,"5min":0.379,"15min":0.35499999999999998},"max":{"1min":11.643000000000001,"5min":11.643000000000001,"15min":11.643000000000001},"last":0.66700000000000004}]},{"osd":1,"last update":"Fri Oct 6 02:32:15 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.72099999999999997,"15min":0.70099999999999996},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.40300000000000002},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.441,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":3.0219999999999998,"5min":3.0219999999999998,"15min":3.0219999999999998},"last":0.626}]},{"osd":2,"last update":"Fri Oct 6 02:32:15 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.67500000000000004,"15min":0.68799999999999994},"min":{"1min":0.435,"5min":0.435,"15min":0.375},"max":{"1min":3.0070000000000001,"5min":3.0070000000000001,"15min":3.0070000000000001},"last":0.58799999999999997},{"interface":"front","average":{"1min":1.2310000000000001,"5min":0.80300000000000005,"15min":0.72699999999999998},"min":{"1min":0.54100000000000004,"5min":0.46999999999999997,"15min":0.39800000000000002},"max":{"1min":12.118,"5min":12.118,"15min":12.118},"last":0.60599999999999998}]},{"osd":3,"last update":"Fri Oct 6 02:32:15 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.71799999999999997,"15min":0.72999999999999998},"min":{"1min":0.46100000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":3.073,"5min":3.073,"15min":3.073},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.78500000000000003,"15min":0.78000000000000003},"min":{"1min":0.55600000000000005,"5min":0.54500000000000004,"15min":0.497},"max":{"1min":3.105,"5min":3.105,"15min":3.105},"last":0.64300000000000002}]},{"osd":4,"last update":"Fri Oct 6 02:32:15 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.61599999999999999,"15min":0.66100000000000003},"min":{"1min":0.42699999999999999,"5min":0.41799999999999998,"15min":0.35099999999999998},"max":{"1min":0.879,"5min":1.1120000000000001,"15min":1.2849999999999999},"last":0.50600000000000001},{"interface":"front","average":{"1min":1.0189999999999999,"5min":0.745,"15min":0.754},"min":{"1min":0.42499999999999999,"5min":0.40899999999999997,"15min":0.32900000000000001},"max":{"1min":8.0060000000000002,"5min":8.0060000000000002,"15min":8.0060000000000002},"last":0.48999999999999999}]},{"osd":5,"last update":"Fri Oct 6 02:32:15 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.73799999999999999,"15min":0.73299999999999998},"min":{"1min":0.46200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":3.524,"5min":3.524,"15min":3.524},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.70399999999999996,"15min":0.72099999999999997},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.442},"max":{"1min":3.258,"5min":3.258,"15min":3.258},"last":0.57199999999999995}]},{"osd":6,"last update":"Fri Oct 6 02:32:15 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.79200000000000004,"15min":0.78600000000000003},"min":{"1min":0.54300000000000004,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":3.4689999999999999,"5min":3.4689999999999999,"15min":3.4689999999999999},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.80000000000000004,"15min":0.80900000000000005},"min":{"1min":0.51500000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":3.4279999999999999,"5min":3.4279999999999999,"15min":3.4279999999999999},"last":0.61699999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574970,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303512,"kb_used_data":2128,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068328,"statfs":{"total":96636764160,"available":96325967872,"internally_reserved":0,"allocated":2179072,"data_stored":1016397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:32:10 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.67600000000000005,"15min":0.66900000000000004},"min":{"1min":0.41599999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.869,"5min":1.869,"15min":1.869},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.71799999999999997,"15min":0.70699999999999996},"min":{"1min":0.49199999999999999,"5min":0.374,"15min":0.374},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.87}]},{"osd":1,"last update":"Fri Oct 6 02:32:10 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66700000000000004,"15min":0.65100000000000002},"min":{"1min":0.47299999999999998,"5min":0.45100000000000001,"15min":0.40699999999999997},"max":{"1min":0.91600000000000004,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.67400000000000004,"15min":0.67000000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.927,"5min":1.927,"15min":1.927},"last":0.54700000000000004}]},{"osd":2,"last update":"Fri Oct 6 02:32:10 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.72999999999999998,"15min":0.71299999999999997},"min":{"1min":0.58199999999999996,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.72499999999999998,"15min":0.71799999999999997},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.40300000000000002},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.77000000000000002}]},{"osd":3,"last update":"Fri Oct 6 02:32:10 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.70599999999999996,"15min":0.67900000000000005},"min":{"1min":0.48899999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.71899999999999997,"15min":0.70999999999999996},"min":{"1min":0.58999999999999997,"5min":0.48799999999999999,"15min":0.45900000000000002},"max":{"1min":0.96299999999999997,"5min":0.97999999999999998,"15min":0.98499999999999999},"last":0.82299999999999995}]},{"osd":4,"last update":"Fri Oct 6 02:32:10 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.68200000000000005,"15min":0.65100000000000002},"min":{"1min":0.379,"5min":0.377,"15min":0.34300000000000003},"max":{"1min":1.97,"5min":1.97,"15min":1.97},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.73199999999999998,"15min":0.68400000000000005},"min":{"1min":0.438,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":2.7429999999999999,"5min":2.7429999999999999,"15min":2.7429999999999999},"last":0.92300000000000004}]},{"osd":5,"last update":"Fri Oct 6 02:32:10 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.71699999999999997,"15min":0.70099999999999996},"min":{"1min":0.47399999999999998,"5min":0.42699999999999999,"15min":0.39700000000000002},"max":{"1min":1.091,"5min":1.091,"15min":1.2929999999999999},"last":0.746},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.747,"15min":0.71799999999999997},"min":{"1min":0.496,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":2.484,"5min":2.484,"15min":2.484},"last":0.84999999999999998}]},{"osd":7,"last update":"Fri Oct 6 02:32:10 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.78600000000000003,"15min":0.75900000000000001},"min":{"1min":0.498,"5min":0.42399999999999999,"15min":0.36799999999999999},"max":{"1min":2.7909999999999999,"5min":2.7909999999999999,"15min":2.7909999999999999},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.78800000000000003,"15min":0.76900000000000002},"min":{"1min":0.55100000000000005,"5min":0.46700000000000003,"15min":0.45900000000000002},"max":{"1min":1.0269999999999999,"5min":1.103,"15min":1.2470000000000001},"last":0.86199999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574969,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303512,"kb_used_data":2128,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068328,"statfs":{"total":96636764160,"available":96325967872,"internally_reserved":0,"allocated":2179072,"data_stored":1016397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.443},"min":{"1min":0.30399999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.58799999999999997,"5min":0.63600000000000001,"15min":1.2230000000000001},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.45800000000000002,"15min":0.501},"min":{"1min":0.34499999999999997,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.66200000000000003,"5min":0.70099999999999996,"15min":1.788},"last":0.45800000000000002}]},{"osd":2,"last update":"Fri Oct 6 02:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.433,"15min":0.49099999999999999},"min":{"1min":0.33500000000000002,"5min":0.307,"15min":0.28299999999999997},"max":{"1min":0.57799999999999996,"5min":0.61799999999999999,"15min":2.0840000000000001},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.503,"5min":0.495,"15min":0.53500000000000003},"min":{"1min":0.374,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.66800000000000004,"5min":0.88100000000000001,"15min":1.6699999999999999},"last":0.53100000000000003}]},{"osd":3,"last update":"Fri Oct 6 02:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.57999999999999996,"15min":0.60399999999999998},"min":{"1min":0.51200000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.79700000000000004,"5min":0.89900000000000002,"15min":1.266},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.54000000000000004,"15min":0.60299999999999998},"min":{"1min":0.436,"5min":0.40999999999999998,"15min":0.34100000000000003},"max":{"1min":0.70399999999999996,"5min":0.79400000000000004,"15min":1.861},"last":0.54900000000000004}]},{"osd":4,"last update":"Fri Oct 6 02:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.752,"15min":0.82199999999999995},"min":{"1min":0.629,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.97999999999999998,"5min":1.3580000000000001,"15min":1.9279999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.747,"15min":0.82199999999999995},"min":{"1min":0.53900000000000003,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.95999999999999996,"5min":1.4339999999999999,"15min":2.0179999999999998},"last":0.86099999999999999}]},{"osd":5,"last update":"Fri Oct 6 02:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.71499999999999997,"15min":0.80900000000000005},"min":{"1min":0.57199999999999995,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.91700000000000004,"5min":1.3069999999999999,"15min":1.9830000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.73199999999999998,"15min":0.79100000000000004},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.47299999999999998},"max":{"1min":0.97299999999999998,"5min":1.327,"15min":1.3380000000000001},"last":0.77300000000000002}]},{"osd":6,"last update":"Fri Oct 6 02:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.76100000000000001,"15min":0.83299999999999996},"min":{"1min":0.62,"5min":0.52100000000000002,"15min":0.45600000000000002},"max":{"1min":1.038,"5min":1.0609999999999999,"15min":1.829},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.77500000000000002,"15min":0.86499999999999999},"min":{"1min":0.60199999999999998,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.96399999999999997,"5min":1.351,"15min":2.052},"last":0.83499999999999996}]},{"osd":7,"last update":"Fri Oct 6 02:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.73399999999999999,"15min":0.80300000000000005},"min":{"1min":0.59399999999999997,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.97299999999999998,"5min":1.3120000000000001,"15min":1.3340000000000001},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.76200000000000001,"15min":0.83799999999999997},"min":{"1min":0.54600000000000004,"5min":0.51600000000000001,"15min":0.47799999999999998},"max":{"1min":1.0780000000000001,"5min":1.2110000000000001,"15min":1.895},"last":0.79000000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574970,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304092,"kb_used_data":2708,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067748,"statfs":{"total":96636764160,"available":96325373952,"internally_reserved":0,"allocated":2772992,"data_stored":1606765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 02:32:19 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.44500000000000001,"15min":0.45000000000000001},"min":{"1min":0.27800000000000002,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.59299999999999997,"5min":2.6120000000000001,"15min":2.6120000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.47399999999999998,"15min":0.47799999999999998},"min":{"1min":0.33000000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.57099999999999995,"5min":3.4199999999999999,"15min":3.4199999999999999},"last":0.51800000000000002}]},{"osd":2,"last update":"Fri Oct 6 02:32:19 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.52400000000000002,"15min":0.51400000000000001},"min":{"1min":0.34999999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.61699999999999999,"5min":3.0339999999999998,"15min":3.0339999999999998},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.51500000000000001,"15min":0.51800000000000002},"min":{"1min":0.377,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.76800000000000002,"5min":3.343,"15min":3.343},"last":0.56200000000000006}]},{"osd":3,"last update":"Fri Oct 6 02:32:19 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.52000000000000002,"15min":0.50800000000000001},"min":{"1min":0.41599999999999998,"5min":0.31,"15min":0.31},"max":{"1min":1.0549999999999999,"5min":3.6579999999999999,"15min":3.6579999999999999},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.62,"5min":0.59499999999999997,"15min":0.59599999999999997},"min":{"1min":0.47699999999999998,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.996,"5min":2.6890000000000001,"15min":2.6890000000000001},"last":0.65300000000000002}]},{"osd":4,"last update":"Fri Oct 6 02:32:19 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.77100000000000002,"15min":0.78700000000000003},"min":{"1min":0.55500000000000005,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0649999999999999,"5min":3.2999999999999998,"15min":3.2999999999999998},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.753,"15min":0.78200000000000003},"min":{"1min":0.53900000000000003,"5min":0.51000000000000001,"15min":0.48499999999999999},"max":{"1min":0.995,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":0.626}]},{"osd":5,"last update":"Fri Oct 6 02:32:19 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.752,"15min":0.77200000000000002},"min":{"1min":0.499,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.1699999999999999,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.78600000000000003,"15min":0.79500000000000004},"min":{"1min":0.58099999999999996,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.006,"5min":3.1110000000000002,"15min":3.1110000000000002},"last":0.748}]},{"osd":6,"last update":"Fri Oct 6 02:32:19 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.78800000000000003,"15min":0.81100000000000005},"min":{"1min":0.52800000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.0780000000000001,"5min":3.1600000000000001,"15min":3.1600000000000001},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76600000000000001,"15min":0.80700000000000005},"min":{"1min":0.59999999999999998,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.1100000000000001,"5min":2.3250000000000002,"15min":2.3250000000000002},"last":0.76200000000000001}]},{"osd":7,"last update":"Fri Oct 6 02:32:19 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.81999999999999995,"15min":0.82199999999999995},"min":{"1min":0.58499999999999996,"5min":0.57999999999999996,"15min":0.56000000000000005},"max":{"1min":1.097,"5min":3.3809999999999998,"15min":3.3809999999999998},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77900000000000003,"15min":0.80700000000000005},"min":{"1min":0.59399999999999997,"5min":0.54700000000000004,"15min":0.53300000000000003},"max":{"1min":1.018,"5min":2.3740000000000001,"15min":2.3740000000000001},"last":0.72299999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574971,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303512,"kb_used_data":2128,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068328,"statfs":{"total":96636764160,"available":96325967872,"internally_reserved":0,"allocated":2179072,"data_stored":1016397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:32:18 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.42399999999999999,"15min":0.434},"min":{"1min":0.28199999999999997,"5min":0.245,"15min":0.245},"max":{"1min":0.59399999999999997,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.378},{"interface":"front","average":{"1min":0.495,"5min":0.46600000000000003,"15min":0.47999999999999998},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":1.0800000000000001},"last":0.495}]},{"osd":1,"last update":"Fri Oct 6 02:32:18 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.47599999999999998,"15min":0.495},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32400000000000001},"max":{"1min":0.754,"5min":1.036,"15min":1.036},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51000000000000001,"15min":0.57099999999999995},"min":{"1min":0.40899999999999997,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.68300000000000005,"5min":1.778,"15min":1.778},"last":0.433}]},{"osd":3,"last update":"Fri Oct 6 02:32:18 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.499,"15min":0.52100000000000002},"min":{"1min":0.36199999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.97499999999999998},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.57999999999999996,"15min":0.60699999999999998},"min":{"1min":0.46800000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.91700000000000004,"5min":0.95799999999999996,"15min":1.028},"last":0.54500000000000004}]},{"osd":4,"last update":"Fri Oct 6 02:32:18 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.73899999999999999,"15min":0.76400000000000001},"min":{"1min":0.51600000000000001,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.875,"5min":1.0940000000000001,"15min":1.1319999999999999},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.747,"15min":0.81399999999999995},"min":{"1min":0.497,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.0309999999999999,"5min":1.907,"15min":1.907},"last":0.79000000000000004}]},{"osd":5,"last update":"Fri Oct 6 02:32:18 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.72199999999999998,"15min":0.77400000000000002},"min":{"1min":0.51100000000000001,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.97799999999999998,"5min":1.109,"15min":1.1850000000000001},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.751,"15min":0.79800000000000004},"min":{"1min":0.59199999999999997,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.006,"5min":1.0609999999999999,"15min":1.258},"last":0.77500000000000002}]},{"osd":6,"last update":"Fri Oct 6 02:32:18 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77900000000000003,"15min":0.82399999999999995},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.089,"5min":1.827,"15min":1.827},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.76200000000000001,"15min":0.78500000000000003},"min":{"1min":0.52600000000000002,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0469999999999999,"5min":1.0660000000000001,"15min":1.1659999999999999},"last":0.75600000000000001}]},{"osd":7,"last update":"Fri Oct 6 02:32:18 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.78400000000000003,"15min":0.84899999999999998},"min":{"1min":0.61199999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.139,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.77400000000000002,"15min":0.81200000000000006},"min":{"1min":0.57399999999999995,"5min":0.55300000000000005,"15min":0.54800000000000004},"max":{"1min":1.1020000000000001,"5min":1.125,"15min":1.1479999999999999},"last":0.82299999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574968,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303512,"kb_used_data":2128,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068328,"statfs":{"total":96636764160,"available":96325967872,"internally_reserved":0,"allocated":2179072,"data_stored":1016397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:32:15 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.42699999999999999,"15min":0.48199999999999998},"min":{"1min":0.30299999999999999,"5min":0.248,"15min":0.248},"max":{"1min":0.66500000000000004,"5min":1.0489999999999999,"15min":2.024},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.45000000000000001,"15min":0.5},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.56000000000000005,"5min":0.754,"15min":2.181},"last":0.47799999999999998}]},{"osd":1,"last update":"Fri Oct 6 02:32:15 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.45000000000000001,"15min":0.59899999999999998},"min":{"1min":0.34999999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.65000000000000002,"5min":0.66100000000000003,"15min":4.492},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51000000000000001,"15min":0.64800000000000002},"min":{"1min":0.40999999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.876,"5min":1.079,"15min":4.569},"last":0.68100000000000005}]},{"osd":2,"last update":"Fri Oct 6 02:32:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.58199999999999996,"15min":0.64000000000000001},"min":{"1min":0.435,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.69499999999999995,"5min":1.024,"15min":2.0870000000000002},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.55200000000000005,"15min":0.61799999999999999},"min":{"1min":0.441,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.77400000000000002,"5min":0.92100000000000004,"15min":2.4260000000000002},"last":0.61699999999999999}]},{"osd":4,"last update":"Fri Oct 6 02:32:15 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.72999999999999998,"15min":0.79400000000000004},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.44800000000000001},"max":{"1min":1.024,"5min":1.024,"15min":2.214},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.751,"15min":0.79000000000000004},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.97999999999999998,"5min":1.042,"15min":1.2869999999999999},"last":0.96299999999999997}]},{"osd":5,"last update":"Fri Oct 6 02:32:15 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.745,"15min":0.79700000000000004},"min":{"1min":0.58499999999999996,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.05,"5min":1.077,"15min":1.3029999999999999},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.75,"15min":0.81799999999999995},"min":{"1min":0.55500000000000005,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.097,"5min":1.097,"15min":2.242},"last":1.016}]},{"osd":6,"last update":"Fri Oct 6 02:32:15 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.76700000000000002,"15min":0.80000000000000004},"min":{"1min":0.52300000000000002,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.073,"5min":1.073,"15min":1.357},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.78600000000000003,"15min":0.81799999999999995},"min":{"1min":0.61199999999999999,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.95299999999999996,"5min":1.016,"15min":1.3100000000000001},"last":0.998}]},{"osd":7,"last update":"Fri Oct 6 02:32:15 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.82399999999999995},"min":{"1min":0.58099999999999996,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.996,"5min":1.004,"15min":2.27},"last":0.98099999999999998},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.78500000000000003,"15min":0.83199999999999996},"min":{"1min":0.61699999999999999,"5min":0.55800000000000005,"15min":0.53200000000000003},"max":{"1min":1.0009999999999999,"5min":1.0820000000000001,"15min":1.325},"last":0.92200000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574970,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":307608,"kb_used_data":2128,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064232,"statfs":{"total":96636764160,"available":96321773568,"internally_reserved":0,"allocated":2179072,"data_stored":1016397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:32:10 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.65100000000000002,"15min":0.88900000000000001},"min":{"1min":0.45800000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.877,"5min":1.0760000000000001,"15min":12.337},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.66700000000000004,"15min":0.83299999999999996},"min":{"1min":0.45100000000000001,"5min":0.44400000000000001,"15min":0.438},"max":{"1min":0.91900000000000004,"5min":1.3520000000000001,"15min":9.6539999999999999},"last":0.70199999999999996}]},{"osd":1,"last update":"Fri Oct 6 02:32:10 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64500000000000002,"15min":0.68400000000000005},"min":{"1min":0.47499999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.83699999999999997,"5min":1.129,"15min":3.181},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.70399999999999996,"15min":0.91800000000000004},"min":{"1min":0.48299999999999998,"5min":0.45300000000000001,"15min":0.40500000000000003},"max":{"1min":0.86799999999999999,"5min":0.995,"15min":12.946999999999999},"last":0.53400000000000003}]},{"osd":2,"last update":"Fri Oct 6 02:32:10 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.69499999999999995,"15min":0.93300000000000005},"min":{"1min":0.47299999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.84499999999999997,"5min":1.2989999999999999,"15min":12.377000000000001},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.73099999999999998,"15min":0.82499999999999996},"min":{"1min":0.53700000000000003,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.91200000000000003,"5min":1.0529999999999999,"15min":6.3220000000000001},"last":0.752}]},{"osd":3,"last update":"Fri Oct 6 02:32:10 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.72899999999999998,"15min":0.90400000000000003},"min":{"1min":0.57099999999999995,"5min":0.46200000000000002,"15min":0.42699999999999999},"max":{"1min":0.84199999999999997,"5min":1.5229999999999999,"15min":9.7720000000000002},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.70499999999999996,"15min":0.752},"min":{"1min":0.55800000000000005,"5min":0.52400000000000002,"15min":0.47999999999999998},"max":{"1min":0.88900000000000001,"5min":1.212,"15min":3.0840000000000001},"last":0.67200000000000004}]},{"osd":5,"last update":"Fri Oct 6 02:32:10 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72199999999999998,"15min":0.80200000000000005},"min":{"1min":0.46100000000000002,"5min":0.40400000000000003,"15min":0.35299999999999998},"max":{"1min":0.92200000000000004,"5min":1.02,"15min":6.2690000000000001},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.75600000000000001,"15min":0.93899999999999995},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.33600000000000002},"max":{"1min":0.98199999999999998,"5min":1.4359999999999999,"15min":10.769},"last":0.86199999999999999}]},{"osd":6,"last update":"Fri Oct 6 02:32:10 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71799999999999997,"15min":0.74299999999999999},"min":{"1min":0.46300000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.95299999999999996,"5min":1.4590000000000001,"15min":3.1349999999999998},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.80100000000000005,"15min":0.91100000000000003},"min":{"1min":0.52600000000000002,"5min":0.47799999999999998,"15min":0.45700000000000002},"max":{"1min":0.96799999999999997,"5min":1.3839999999999999,"15min":8.0899999999999999},"last":0.877}]},{"osd":7,"last update":"Fri Oct 6 02:32:10 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.80300000000000005,"15min":0.97899999999999998},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.47599999999999998},"max":{"1min":1.02,"5min":1.494,"15min":9.7089999999999996},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.78700000000000003,"15min":0.92800000000000005},"min":{"1min":0.56000000000000005,"5min":0.44400000000000001,"15min":0.374},"max":{"1min":1.034,"5min":1.2669999999999999,"15min":7.6719999999999997},"last":0.78900000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574970,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303512,"kb_used_data":2128,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068328,"statfs":{"total":96636764160,"available":96325967872,"internally_reserved":0,"allocated":2179072,"data_stored":1016397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:32:07 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.63200000000000001,"15min":0.63},"min":{"1min":0.47099999999999997,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.93899999999999995,"5min":0.97599999999999998,"15min":1.4279999999999999},"last":0.99099999999999999},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.629,"15min":0.622},"min":{"1min":0.42599999999999999,"5min":0.40300000000000002,"15min":0.38700000000000001},"max":{"1min":0.89300000000000002,"5min":1.099,"15min":1.099},"last":0.91700000000000004}]},{"osd":1,"last update":"Fri Oct 6 02:32:07 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.63600000000000001,"15min":0.64900000000000002},"min":{"1min":0.51700000000000002,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.93899999999999995,"5min":1.0660000000000001,"15min":1.089},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.67300000000000004,"15min":0.67700000000000005},"min":{"1min":0.42299999999999999,"5min":0.40699999999999997,"15min":0.40500000000000003},"max":{"1min":0.93899999999999995,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.88200000000000001}]},{"osd":2,"last update":"Fri Oct 6 02:32:07 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.67800000000000005,"15min":0.67600000000000005},"min":{"1min":0.45800000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.67200000000000004,"15min":0.68200000000000005},"min":{"1min":0.58599999999999997,"5min":0.46100000000000002,"15min":0.44600000000000001},"max":{"1min":1.028,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.83799999999999997}]},{"osd":3,"last update":"Fri Oct 6 02:32:07 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.73299999999999998,"15min":0.73199999999999998},"min":{"1min":0.54000000000000004,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0009999999999999,"5min":1.159,"15min":1.159},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.77200000000000002,"15min":0.77000000000000002},"min":{"1min":0.64200000000000002,"5min":0.56200000000000006,"15min":0.54100000000000004},"max":{"1min":0.97699999999999998,"5min":1.194,"15min":1.194},"last":0.90300000000000002}]},{"osd":4,"last update":"Fri Oct 6 02:32:07 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.69099999999999995,"15min":0.67700000000000005},"min":{"1min":0.46999999999999997,"5min":0.40799999999999997,"15min":0.32000000000000001},"max":{"1min":0.92300000000000004,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.749},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.69099999999999995,"15min":0.68300000000000005},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.35399999999999998},"max":{"1min":0.88600000000000001,"5min":1.25,"15min":1.25},"last":0.77600000000000002}]},{"osd":6,"last update":"Fri Oct 6 02:32:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.71899999999999997,"15min":0.73499999999999999},"min":{"1min":0.45900000000000002,"5min":0.42899999999999999,"15min":0.40300000000000002},"max":{"1min":1.024,"5min":1.024,"15min":1.1719999999999999},"last":1.0089999999999999},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.81499999999999995,"15min":0.81799999999999995},"min":{"1min":0.60599999999999998,"5min":0.502,"15min":0.373},"max":{"1min":1.0700000000000001,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":1.0289999999999999}]},{"osd":7,"last update":"Fri Oct 6 02:32:07 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.754,"15min":0.76000000000000001},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.40799999999999997},"max":{"1min":0.95799999999999996,"5min":1.23,"15min":1.23},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.80200000000000005,"15min":0.82099999999999995},"min":{"1min":0.59599999999999997,"5min":0.47599999999999998,"15min":0.437},"max":{"1min":1.1180000000000001,"5min":1.1339999999999999,"15min":1.5640000000000001},"last":1.0589999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T02:32:44.063 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-06T02:23:00.968050+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=2, tm_min=23, tm_sec=0, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=2, tm_min=32, tm_sec=37, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T02:32:44.063 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-06T02:23:00.968050+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=2, tm_min=23, tm_sec=0, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=2, tm_min=32, tm_sec=37, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T02:32:44.064 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-06T02:23:00.968050+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=2, tm_min=23, tm_sec=0, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=2, tm_min=32, tm_sec=37, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T02:32:44.064 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-10-06T02:23:00.968050+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=2, tm_min=23, tm_sec=0, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=2, tm_min=32, tm_sec=37, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T02:32:44.064 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-10-06T02:22:53.309097+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=2, tm_min=22, tm_sec=53, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=2, tm_min=32, tm_sec=37, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T02:32:44.064 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-06T02:23:00.968050+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=2, tm_min=23, tm_sec=0, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=2, tm_min=32, tm_sec=37, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T02:32:44.064 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2023-10-06T02:23:00.968050+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=2, tm_min=23, tm_sec=0, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=2, tm_min=32, tm_sec=37, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T02:32:44.064 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-06T02:23:00.968050+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=2, tm_min=23, tm_sec=0, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=2, tm_min=32, tm_sec=37, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T02:32:44.064 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-06T02:23:00.968050+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=2, tm_min=23, tm_sec=0, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=2, tm_min=32, tm_sec=37, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T02:32:44.065 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-06T02:33:02.698 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T02:33:02.703 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T02:33:04.066 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T02:33:04.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.261+0000 7f12a2d9d700 1 Processor -- start 2023-10-06T02:33:04.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.261+0000 7f12a2d9d700 1 -- start start 2023-10-06T02:33:04.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.263+0000 7f12a2d9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f129c13fb20 0x7f129c15a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:33:04.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.263+0000 7f12a2d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f129c06aaa0 0x7f129c0c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:33:04.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.263+0000 7f12a2d9d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f129c15ae70 con 0x7f129c13fef0 2023-10-06T02:33:04.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.263+0000 7f12a2d9d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f129c06af60 con 0x7f129c13fb20 2023-10-06T02:33:04.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.263+0000 7f12a2d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f129c06b240 con 0x7f129c06aaa0 2023-10-06T02:33:04.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.263+0000 7f12a159a700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f129c13fef0 0x7f129c0c7990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:36858/0 (socket says 172.21.15.31:36858) 2023-10-06T02:33:04.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.263+0000 7f12a159a700 1 -- 172.21.15.31:0/2266106086 learned_addr learned my addr 172.21.15.31:0/2266106086 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T02:33:04.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.263+0000 7f12a1d9b700 1 --2- 172.21.15.31:0/2266106086 >> v2:172.21.15.62:3300/0 conn(0x7f129c13fb20 0x7f129c15a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:33:04.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.263+0000 7f12a259c700 1 --2- 172.21.15.31:0/2266106086 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f129c06aaa0 0x7f129c0c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:33:04.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.263+0000 7f12a0d99700 1 -- 172.21.15.31:0/2266106086 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 823315996 0 0) 0x7f128c001290 con 0x7f129c13fef0 2023-10-06T02:33:04.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.264+0000 7f12a0d99700 1 -- 172.21.15.31:0/2266106086 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1288003570 con 0x7f129c13fef0 2023-10-06T02:33:04.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.264+0000 7f12a1d9b700 1 -- 172.21.15.31:0/2266106086 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f129c06aaa0 msgr2=0x7f129c0c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:33:04.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.264+0000 7f12a1d9b700 1 --2- 172.21.15.31:0/2266106086 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f129c06aaa0 0x7f129c0c9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:33:04.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.264+0000 7f12a1d9b700 1 -- 172.21.15.31:0/2266106086 >> v1:172.21.15.31:6789/0 conn(0x7f129c13fef0 legacy=0x7f129c0c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:33:04.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.264+0000 7f12a1d9b700 1 -- 172.21.15.31:0/2266106086 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f129c06b520 con 0x7f129c13fb20 2023-10-06T02:33:04.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.264+0000 7f12a1d9b700 1 --2- 172.21.15.31:0/2266106086 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f129c13fb20 0x7f129c15a930 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f12980098e0 tx=0x7f12980091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b83e0be8820a34e2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:33:04.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.264+0000 7f12a0d99700 1 -- 172.21.15.31:0/2266106086 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1298011630 con 0x7f129c13fb20 2023-10-06T02:33:04.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.264+0000 7f12a0d99700 1 -- 172.21.15.31:0/2266106086 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1298011790 con 0x7f129c13fb20 2023-10-06T02:33:04.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.264+0000 7f12a0d99700 1 -- 172.21.15.31:0/2266106086 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1298011b30 con 0x7f129c13fb20 2023-10-06T02:33:04.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.265+0000 7f12a2d9d700 1 -- 172.21.15.31:0/2266106086 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f129c13fb20 msgr2=0x7f129c15a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:33:04.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.265+0000 7f12a2d9d700 1 --2- 172.21.15.31:0/2266106086 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f129c13fb20 0x7f129c15a930 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f12980098e0 tx=0x7f12980091a0 comp rx=0 tx=0).stop 2023-10-06T02:33:04.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.265+0000 7f12a2d9d700 1 -- 172.21.15.31:0/2266106086 shutdown_connections 2023-10-06T02:33:04.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.265+0000 7f12a2d9d700 1 --2- 172.21.15.31:0/2266106086 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f129c13fb20 0x7f129c15a930 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:33:04.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.265+0000 7f12a2d9d700 1 --2- 172.21.15.31:0/2266106086 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f129c06aaa0 0x7f129c0c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:33:04.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.265+0000 7f12a2d9d700 1 -- 172.21.15.31:0/2266106086 >> 172.21.15.31:0/2266106086 conn(0x7f129c06d170 msgr2=0x7f129c0cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:33:04.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.265+0000 7f12a2d9d700 1 -- 172.21.15.31:0/2266106086 shutdown_connections 2023-10-06T02:33:04.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.265+0000 7f12a2d9d700 1 -- 172.21.15.31:0/2266106086 wait complete. 2023-10-06T02:33:04.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.266+0000 7f12a2d9d700 1 Processor -- start 2023-10-06T02:33:04.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.267+0000 7f12a2d9d700 1 -- start start 2023-10-06T02:33:04.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.267+0000 7f12a2d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f129c06aaa0 0x7f129c155c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:33:04.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.267+0000 7f12a2d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f129c13fb20 0x7f129c156950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:33:04.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.268+0000 7f12a2d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f129c13fef0 0x7f129c0ac150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:33:04.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.268+0000 7f12a2d9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f129c158000 con 0x7f129c06aaa0 2023-10-06T02:33:04.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.268+0000 7f12a2d9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f129c1573f0 con 0x7f129c13fb20 2023-10-06T02:33:04.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.268+0000 7f12a2d9d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f129c1576d0 con 0x7f129c13fef0 2023-10-06T02:33:04.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.268+0000 7f12a1d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f129c06aaa0 0x7f129c155c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:33:04.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.268+0000 7f12a1d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f129c06aaa0 0x7f129c155c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51692/0 (socket says 172.21.15.31:51692) 2023-10-06T02:33:04.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.268+0000 7f12a1d9b700 1 -- 172.21.15.31:0/4141416067 learned_addr learned my addr 172.21.15.31:0/4141416067 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:33:04.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.268+0000 7f12a1d9b700 1 -- 172.21.15.31:0/4141416067 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f129c13fef0 msgr2=0x7f129c0ac150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:33:04.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.268+0000 7f12a1d9b700 1 --2- 172.21.15.31:0/4141416067 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f129c13fef0 0x7f129c0ac150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:33:04.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.268+0000 7f12a1d9b700 1 -- 172.21.15.31:0/4141416067 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f129c13fb20 msgr2=0x7f129c156950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:33:04.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.268+0000 7f12a159a700 1 --2- 172.21.15.31:0/4141416067 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f129c13fb20 0x7f129c156950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:33:04.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.268+0000 7f12a1d9b700 1 --2- 172.21.15.31:0/4141416067 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f129c13fb20 0x7f129c156950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:33:04.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.268+0000 7f12a1d9b700 1 -- 172.21.15.31:0/4141416067 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1298009420 con 0x7f129c06aaa0 2023-10-06T02:33:04.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.268+0000 7f12a159a700 1 --2- 172.21.15.31:0/4141416067 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f129c13fb20 0x7f129c156950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:33:04.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.269+0000 7f12a1d9b700 1 --2- 172.21.15.31:0/4141416067 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f129c06aaa0 0x7f129c155c60 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f129c071320 tx=0x7f1298017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:33:04.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.269+0000 7f1291ffb700 1 -- 172.21.15.31:0/4141416067 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1298011b10 con 0x7f129c06aaa0 2023-10-06T02:33:04.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.269+0000 7f1291ffb700 1 -- 172.21.15.31:0/4141416067 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12980253f0 con 0x7f129c06aaa0 2023-10-06T02:33:04.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.269+0000 7f12a2d9d700 1 -- 172.21.15.31:0/4141416067 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f129c1579b0 con 0x7f129c06aaa0 2023-10-06T02:33:04.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.269+0000 7f1291ffb700 1 -- 172.21.15.31:0/4141416067 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1298025790 con 0x7f129c06aaa0 2023-10-06T02:33:04.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.269+0000 7f12a2d9d700 1 -- 172.21.15.31:0/4141416067 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f129c1561a0 con 0x7f129c06aaa0 2023-10-06T02:33:04.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.270+0000 7f12a2d9d700 1 -- 172.21.15.31:0/4141416067 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f129c04c3c0 con 0x7f129c06aaa0 2023-10-06T02:33:04.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.271+0000 7f1291ffb700 1 -- 172.21.15.31:0/4141416067 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) v1 ==== 85709+0+0 (secure 0 0 0) 0x7f1298025bc0 con 0x7f129c06aaa0 2023-10-06T02:33:04.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.272+0000 7f1291ffb700 1 --2- 172.21.15.31:0/4141416067 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f12800679e0 0x7f1280069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:33:04.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.272+0000 7f1291ffb700 1 -- 172.21.15.31:0/4141416067 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5320+0+0 (secure 0 0 0) 0x7f1298084860 con 0x7f129c06aaa0 2023-10-06T02:33:04.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.272+0000 7f12a159a700 1 --2- 172.21.15.31:0/4141416067 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f12800679e0 0x7f1280069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:33:04.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.273+0000 7f12a159a700 1 --2- 172.21.15.31:0/4141416067 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f12800679e0 0x7f1280069e90 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f128c0013f0 tx=0x7f128c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:33:04.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.276+0000 7f1291ffb700 1 -- 172.21.15.31:0/4141416067 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f12980b85a0 con 0x7f129c06aaa0 2023-10-06T02:33:04.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.474+0000 7f12a2d9d700 1 -- 172.21.15.31:0/4141416067 --> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f129c158140 con 0x7f12800679e0 2023-10-06T02:33:04.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.479+0000 7f1291ffb700 1 -- 172.21.15.31:0/4141416067 <== mgr.4109 v2:172.21.15.62:6836/85212 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72471 (secure 0 0 0) 0x7f128c002050 con 0x7f12800679e0 2023-10-06T02:33:04.481 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:33:04.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.480+0000 7f12a2d9d700 1 -- 172.21.15.31:0/4141416067 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f12800679e0 msgr2=0x7f1280069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:33:04.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.480+0000 7f12a2d9d700 1 --2- 172.21.15.31:0/4141416067 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f12800679e0 0x7f1280069e90 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f128c0013f0 tx=0x7f128c009040 comp rx=0 tx=0).stop 2023-10-06T02:33:04.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.480+0000 7f12a2d9d700 1 -- 172.21.15.31:0/4141416067 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f129c06aaa0 msgr2=0x7f129c155c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:33:04.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.480+0000 7f12a2d9d700 1 --2- 172.21.15.31:0/4141416067 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f129c06aaa0 0x7f129c155c60 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f129c071320 tx=0x7f1298017000 comp rx=0 tx=0).stop 2023-10-06T02:33:04.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.481+0000 7f12a2d9d700 1 -- 172.21.15.31:0/4141416067 shutdown_connections 2023-10-06T02:33:04.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.481+0000 7f12a2d9d700 1 --2- 172.21.15.31:0/4141416067 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f12800679e0 0x7f1280069e90 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:33:04.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.481+0000 7f12a2d9d700 1 --2- 172.21.15.31:0/4141416067 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f129c13fef0 0x7f129c0ac150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:33:04.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.481+0000 7f12a2d9d700 1 --2- 172.21.15.31:0/4141416067 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f129c06aaa0 0x7f129c155c60 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:33:04.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.481+0000 7f12a2d9d700 1 --2- 172.21.15.31:0/4141416067 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f129c13fb20 0x7f129c156950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:33:04.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.481+0000 7f12a2d9d700 1 -- 172.21.15.31:0/4141416067 >> 172.21.15.31:0/4141416067 conn(0x7f129c06d170 msgr2=0x7f129c06f110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:33:04.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.481+0000 7f12a2d9d700 1 -- 172.21.15.31:0/4141416067 shutdown_connections 2023-10-06T02:33:04.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.481+0000 7f12a2d9d700 1 -- 172.21.15.31:0/4141416067 wait complete. 2023-10-06T02:33:04.484 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-10-06T02:33:04.501 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":262,"stamp":"2023-10-06T02:33:02.536053+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130,"ondisk_log_size":130,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":2433352,"kb_used_data":18184,"kb_used_omap":0,"kb_used_meta":2415104,"kb_avail":752541368,"statfs":{"total":773094113280,"available":770602360832,"internally_reserved":0,"allocated":18620416,"data_stored":9311912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2473066496},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002918"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":148,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T02:32:46.551872+0000","last_change":"2023-10-06T02:32:46.551818+0000","last_active":"2023-10-06T02:32:46.551872+0000","last_peered":"2023-10-06T02:32:46.551872+0000","last_clean":"2023-10-06T02:32:46.551872+0000","last_became_active":"2023-10-06T02:23:01.992568+0000","last_became_peered":"2023-10-06T02:23:01.992568+0000","last_unstale":"2023-10-06T02:32:46.551872+0000","last_undegraded":"2023-10-06T02:32:46.551872+0000","last_fullsized":"2023-10-06T02:32:46.551872+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:46.551702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:46.551702+0000","last_clean_scrub_stamp":"2023-10-06T02:32:46.551702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:32:54.844252+0000","scrub_duration":0.002086676,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":149,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T02:32:39.780249+0000","last_change":"2023-10-06T02:32:39.780222+0000","last_active":"2023-10-06T02:32:39.780249+0000","last_peered":"2023-10-06T02:32:39.780249+0000","last_clean":"2023-10-06T02:32:39.780249+0000","last_became_active":"2023-10-06T02:23:01.989706+0000","last_became_peered":"2023-10-06T02:23:01.989706+0000","last_unstale":"2023-10-06T02:32:39.780249+0000","last_undegraded":"2023-10-06T02:32:39.780249+0000","last_fullsized":"2023-10-06T02:32:39.780249+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:39.780141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:39.780141+0000","last_clean_scrub_stamp":"2023-10-06T02:32:39.780141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:01.717422+0000","scrub_duration":0.0021517400000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":142,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T02:32:54.564742+0000","last_change":"2023-10-06T02:32:54.564708+0000","last_active":"2023-10-06T02:32:54.564742+0000","last_peered":"2023-10-06T02:32:54.564742+0000","last_clean":"2023-10-06T02:32:54.564742+0000","last_became_active":"2023-10-06T02:23:02.053739+0000","last_became_peered":"2023-10-06T02:23:02.053739+0000","last_unstale":"2023-10-06T02:32:54.564742+0000","last_undegraded":"2023-10-06T02:32:54.564742+0000","last_fullsized":"2023-10-06T02:32:54.564742+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:54.564604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:54.564604+0000","last_clean_scrub_stamp":"2023-10-06T02:32:54.564604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:32:10.892502+0000","scrub_duration":0.0026502169999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":142,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T02:32:49.655238+0000","last_change":"2023-10-06T02:32:49.655184+0000","last_active":"2023-10-06T02:32:49.655238+0000","last_peered":"2023-10-06T02:32:49.655238+0000","last_clean":"2023-10-06T02:32:49.655238+0000","last_became_active":"2023-10-06T02:23:01.990511+0000","last_became_peered":"2023-10-06T02:23:01.990511+0000","last_unstale":"2023-10-06T02:32:49.655238+0000","last_undegraded":"2023-10-06T02:32:49.655238+0000","last_fullsized":"2023-10-06T02:32:49.655238+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:49.655080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:49.655080+0000","last_clean_scrub_stamp":"2023-10-06T02:32:49.655080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:37:44.661290+0000","scrub_duration":0.0025414529999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"172'130","reported_seq":336,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T02:32:48.609171+0000","last_change":"2023-10-06T02:32:48.609134+0000","last_active":"2023-10-06T02:32:48.609171+0000","last_peered":"2023-10-06T02:32:48.609171+0000","last_clean":"2023-10-06T02:32:48.609171+0000","last_became_active":"2023-10-06T02:22:55.031367+0000","last_became_peered":"2023-10-06T02:22:55.031367+0000","last_unstale":"2023-10-06T02:32:48.609171+0000","last_undegraded":"2023-10-06T02:32:48.609171+0000","last_fullsized":"2023-10-06T02:32:48.609171+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"172'130","last_scrub_stamp":"2023-10-06T02:32:48.609042+0000","last_deep_scrub":"172'130","last_deep_scrub_stamp":"2023-10-06T02:32:48.609042+0000","last_clean_scrub_stamp":"2023-10-06T02:32:48.609042+0000","objects_scrubbed":2,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:26:58.183653+0000","scrub_duration":0.006337339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":173,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T02:32:43.064083+0000","last_change":"2023-10-06T02:32:43.064048+0000","last_active":"2023-10-06T02:32:43.064083+0000","last_peered":"2023-10-06T02:32:43.064083+0000","last_clean":"2023-10-06T02:32:43.064083+0000","last_became_active":"2023-10-06T02:23:01.987769+0000","last_became_peered":"2023-10-06T02:23:01.987769+0000","last_unstale":"2023-10-06T02:32:43.064083+0000","last_undegraded":"2023-10-06T02:32:43.064083+0000","last_fullsized":"2023-10-06T02:32:43.064083+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:43.063940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:43.063940+0000","last_clean_scrub_stamp":"2023-10-06T02:32:43.063940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:31:41.261429+0000","scrub_duration":0.001969274,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":159,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T02:32:45.434048+0000","last_change":"2023-10-06T02:32:45.434018+0000","last_active":"2023-10-06T02:32:45.434048+0000","last_peered":"2023-10-06T02:32:45.434048+0000","last_clean":"2023-10-06T02:32:45.434048+0000","last_became_active":"2023-10-06T02:23:02.053991+0000","last_became_peered":"2023-10-06T02:23:02.053991+0000","last_unstale":"2023-10-06T02:32:45.434048+0000","last_undegraded":"2023-10-06T02:32:45.434048+0000","last_fullsized":"2023-10-06T02:32:45.434048+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:45.433915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:45.433915+0000","last_clean_scrub_stamp":"2023-10-06T02:32:45.433915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:32:53.726466+0000","scrub_duration":0.0020996230000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":173,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T02:32:48.066776+0000","last_change":"2023-10-06T02:32:48.066726+0000","last_active":"2023-10-06T02:32:48.066776+0000","last_peered":"2023-10-06T02:32:48.066776+0000","last_clean":"2023-10-06T02:32:48.066776+0000","last_became_active":"2023-10-06T02:23:01.988303+0000","last_became_peered":"2023-10-06T02:23:01.988303+0000","last_unstale":"2023-10-06T02:32:48.066776+0000","last_undegraded":"2023-10-06T02:32:48.066776+0000","last_fullsized":"2023-10-06T02:32:48.066776+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:48.066611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:48.066611+0000","last_clean_scrub_stamp":"2023-10-06T02:32:48.066611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:32:43.319400+0000","scrub_duration":0.0023154489999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":149,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T02:32:40.774562+0000","last_change":"2023-10-06T02:32:40.774535+0000","last_active":"2023-10-06T02:32:40.774562+0000","last_peered":"2023-10-06T02:32:40.774562+0000","last_clean":"2023-10-06T02:32:40.774562+0000","last_became_active":"2023-10-06T02:23:02.053742+0000","last_became_peered":"2023-10-06T02:23:02.053742+0000","last_unstale":"2023-10-06T02:32:40.774562+0000","last_undegraded":"2023-10-06T02:32:40.774562+0000","last_fullsized":"2023-10-06T02:32:40.774562+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:32:40.774448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:32:40.774448+0000","last_clean_scrub_stamp":"2023-10-06T02:32:40.774448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:59.278354+0000","scrub_duration":0.0015497670000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130,"ondisk_log_size":130,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574974,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304092,"kb_used_data":2708,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067748,"statfs":{"total":96636764160,"available":96325373952,"internally_reserved":0,"allocated":2772992,"data_stored":1606765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:32:15 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.68200000000000005,"15min":0.65300000000000002},"min":{"1min":0.38400000000000001,"5min":0.318,"15min":0.318},"max":{"1min":7.9160000000000004,"5min":7.9160000000000004,"15min":7.9160000000000004},"last":0.70199999999999996},{"interface":"front","average":{"1min":1.1759999999999999,"5min":0.73099999999999998,"15min":0.65100000000000002},"min":{"1min":0.47199999999999998,"5min":0.379,"15min":0.35499999999999998},"max":{"1min":11.643000000000001,"5min":11.643000000000001,"15min":11.643000000000001},"last":0.71799999999999997}]},{"osd":1,"last update":"Fri Oct 6 02:32:15 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.72099999999999997,"15min":0.70099999999999996},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.40300000000000002},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.441,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":3.0219999999999998,"5min":3.0219999999999998,"15min":3.0219999999999998},"last":0.81100000000000005}]},{"osd":2,"last update":"Fri Oct 6 02:32:15 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.67500000000000004,"15min":0.68799999999999994},"min":{"1min":0.435,"5min":0.435,"15min":0.375},"max":{"1min":3.0070000000000001,"5min":3.0070000000000001,"15min":3.0070000000000001},"last":0.79500000000000004},{"interface":"front","average":{"1min":1.2310000000000001,"5min":0.80300000000000005,"15min":0.72699999999999998},"min":{"1min":0.54100000000000004,"5min":0.46999999999999997,"15min":0.39800000000000002},"max":{"1min":12.118,"5min":12.118,"15min":12.118},"last":0.86099999999999999}]},{"osd":3,"last update":"Fri Oct 6 02:32:15 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.71799999999999997,"15min":0.72999999999999998},"min":{"1min":0.46100000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":3.073,"5min":3.073,"15min":3.073},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.78500000000000003,"15min":0.78000000000000003},"min":{"1min":0.55600000000000005,"5min":0.54500000000000004,"15min":0.497},"max":{"1min":3.105,"5min":3.105,"15min":3.105},"last":0.77000000000000002}]},{"osd":4,"last update":"Fri Oct 6 02:32:15 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.61599999999999999,"15min":0.66100000000000003},"min":{"1min":0.42699999999999999,"5min":0.41799999999999998,"15min":0.35099999999999998},"max":{"1min":0.879,"5min":1.1120000000000001,"15min":1.2849999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":1.0189999999999999,"5min":0.745,"15min":0.754},"min":{"1min":0.42499999999999999,"5min":0.40899999999999997,"15min":0.32900000000000001},"max":{"1min":8.0060000000000002,"5min":8.0060000000000002,"15min":8.0060000000000002},"last":0.59599999999999997}]},{"osd":5,"last update":"Fri Oct 6 02:32:15 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.73799999999999999,"15min":0.73299999999999998},"min":{"1min":0.46200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":3.524,"5min":3.524,"15min":3.524},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.70399999999999996,"15min":0.72099999999999997},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.442},"max":{"1min":3.258,"5min":3.258,"15min":3.258},"last":0.73999999999999999}]},{"osd":6,"last update":"Fri Oct 6 02:32:15 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.79200000000000004,"15min":0.78600000000000003},"min":{"1min":0.54300000000000004,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":3.4689999999999999,"5min":3.4689999999999999,"15min":3.4689999999999999},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.80000000000000004,"15min":0.80900000000000005},"min":{"1min":0.51500000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":3.4279999999999999,"5min":3.4279999999999999,"15min":3.4279999999999999},"last":0.53300000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574974,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303512,"kb_used_data":2128,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068328,"statfs":{"total":96636764160,"available":96325967872,"internally_reserved":0,"allocated":2179072,"data_stored":1016397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:32:10 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.67600000000000005,"15min":0.66900000000000004},"min":{"1min":0.41599999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.869,"5min":1.869,"15min":1.869},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.71799999999999997,"15min":0.70699999999999996},"min":{"1min":0.49199999999999999,"5min":0.374,"15min":0.374},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.877}]},{"osd":1,"last update":"Fri Oct 6 02:32:10 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66700000000000004,"15min":0.65100000000000002},"min":{"1min":0.47299999999999998,"5min":0.45100000000000001,"15min":0.40699999999999997},"max":{"1min":0.91600000000000004,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.67400000000000004,"15min":0.67000000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.927,"5min":1.927,"15min":1.927},"last":0.80800000000000005}]},{"osd":2,"last update":"Fri Oct 6 02:32:10 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.72999999999999998,"15min":0.71299999999999997},"min":{"1min":0.58199999999999996,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.72499999999999998,"15min":0.71799999999999997},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.40300000000000002},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.83899999999999997}]},{"osd":3,"last update":"Fri Oct 6 02:32:10 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.70599999999999996,"15min":0.67900000000000005},"min":{"1min":0.48899999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.71899999999999997,"15min":0.70999999999999996},"min":{"1min":0.58999999999999997,"5min":0.48799999999999999,"15min":0.45900000000000002},"max":{"1min":0.96299999999999997,"5min":0.97999999999999998,"15min":0.98499999999999999},"last":0.85399999999999998}]},{"osd":4,"last update":"Fri Oct 6 02:32:10 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.68200000000000005,"15min":0.65100000000000002},"min":{"1min":0.379,"5min":0.377,"15min":0.34300000000000003},"max":{"1min":1.97,"5min":1.97,"15min":1.97},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.73199999999999998,"15min":0.68400000000000005},"min":{"1min":0.438,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":2.7429999999999999,"5min":2.7429999999999999,"15min":2.7429999999999999},"last":0.61099999999999999}]},{"osd":5,"last update":"Fri Oct 6 02:32:10 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.71699999999999997,"15min":0.70099999999999996},"min":{"1min":0.47399999999999998,"5min":0.42699999999999999,"15min":0.39700000000000002},"max":{"1min":1.091,"5min":1.091,"15min":1.2929999999999999},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.747,"15min":0.71799999999999997},"min":{"1min":0.496,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":2.484,"5min":2.484,"15min":2.484},"last":0.53900000000000003}]},{"osd":7,"last update":"Fri Oct 6 02:32:10 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.78600000000000003,"15min":0.75900000000000001},"min":{"1min":0.498,"5min":0.42399999999999999,"15min":0.36799999999999999},"max":{"1min":2.7909999999999999,"5min":2.7909999999999999,"15min":2.7909999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.78800000000000003,"15min":0.76900000000000002},"min":{"1min":0.55100000000000005,"5min":0.46700000000000003,"15min":0.45900000000000002},"max":{"1min":1.0269999999999999,"5min":1.103,"15min":1.2470000000000001},"last":0.82399999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574973,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303512,"kb_used_data":2128,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068328,"statfs":{"total":96636764160,"available":96325967872,"internally_reserved":0,"allocated":2179072,"data_stored":1016397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.443},"min":{"1min":0.30399999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.58799999999999997,"5min":0.63600000000000001,"15min":1.2230000000000001},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.45800000000000002,"15min":0.501},"min":{"1min":0.34499999999999997,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.66200000000000003,"5min":0.70099999999999996,"15min":1.788},"last":0.41199999999999998}]},{"osd":2,"last update":"Fri Oct 6 02:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.433,"15min":0.49099999999999999},"min":{"1min":0.33500000000000002,"5min":0.307,"15min":0.28299999999999997},"max":{"1min":0.57799999999999996,"5min":0.61799999999999999,"15min":2.0840000000000001},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.503,"5min":0.495,"15min":0.53500000000000003},"min":{"1min":0.374,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.66800000000000004,"5min":0.88100000000000001,"15min":1.6699999999999999},"last":0.44500000000000001}]},{"osd":3,"last update":"Fri Oct 6 02:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.57999999999999996,"15min":0.60399999999999998},"min":{"1min":0.51200000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.79700000000000004,"5min":0.89900000000000002,"15min":1.266},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.54000000000000004,"15min":0.60299999999999998},"min":{"1min":0.436,"5min":0.40999999999999998,"15min":0.34100000000000003},"max":{"1min":0.70399999999999996,"5min":0.79400000000000004,"15min":1.861},"last":0.57099999999999995}]},{"osd":4,"last update":"Fri Oct 6 02:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.752,"15min":0.82199999999999995},"min":{"1min":0.629,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.97999999999999998,"5min":1.3580000000000001,"15min":1.9279999999999999},"last":1.0700000000000001},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.747,"15min":0.82199999999999995},"min":{"1min":0.53900000000000003,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.95999999999999996,"5min":1.4339999999999999,"15min":2.0179999999999998},"last":0.997}]},{"osd":5,"last update":"Fri Oct 6 02:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.71499999999999997,"15min":0.80900000000000005},"min":{"1min":0.57199999999999995,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.91700000000000004,"5min":1.3069999999999999,"15min":1.9830000000000001},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.73199999999999998,"15min":0.79100000000000004},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.47299999999999998},"max":{"1min":0.97299999999999998,"5min":1.327,"15min":1.3380000000000001},"last":0.89900000000000002}]},{"osd":6,"last update":"Fri Oct 6 02:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.76100000000000001,"15min":0.83299999999999996},"min":{"1min":0.62,"5min":0.52100000000000002,"15min":0.45600000000000002},"max":{"1min":1.038,"5min":1.0609999999999999,"15min":1.829},"last":0.98699999999999999},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.77500000000000002,"15min":0.86499999999999999},"min":{"1min":0.60199999999999998,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.96399999999999997,"5min":1.351,"15min":2.052},"last":1.048}]},{"osd":7,"last update":"Fri Oct 6 02:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.73399999999999999,"15min":0.80300000000000005},"min":{"1min":0.59399999999999997,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.97299999999999998,"5min":1.3120000000000001,"15min":1.3340000000000001},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.76200000000000001,"15min":0.83799999999999997},"min":{"1min":0.54600000000000004,"5min":0.51600000000000001,"15min":0.47799999999999998},"max":{"1min":1.0780000000000001,"5min":1.2110000000000001,"15min":1.895},"last":0.91600000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574974,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304092,"kb_used_data":2708,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067748,"statfs":{"total":96636764160,"available":96325373952,"internally_reserved":0,"allocated":2772992,"data_stored":1606765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 02:32:19 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.44500000000000001,"15min":0.45000000000000001},"min":{"1min":0.27800000000000002,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.59299999999999997,"5min":2.6120000000000001,"15min":2.6120000000000001},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.47399999999999998,"15min":0.47799999999999998},"min":{"1min":0.33000000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.57099999999999995,"5min":3.4199999999999999,"15min":3.4199999999999999},"last":0.71999999999999997}]},{"osd":2,"last update":"Fri Oct 6 02:32:19 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.52400000000000002,"15min":0.51400000000000001},"min":{"1min":0.34999999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.61699999999999999,"5min":3.0339999999999998,"15min":3.0339999999999998},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.51500000000000001,"15min":0.51800000000000002},"min":{"1min":0.377,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.76800000000000002,"5min":3.343,"15min":3.343},"last":0.59999999999999998}]},{"osd":3,"last update":"Fri Oct 6 02:32:19 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.52000000000000002,"15min":0.50800000000000001},"min":{"1min":0.41599999999999998,"5min":0.31,"15min":0.31},"max":{"1min":1.0549999999999999,"5min":3.6579999999999999,"15min":3.6579999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.62,"5min":0.59499999999999997,"15min":0.59599999999999997},"min":{"1min":0.47699999999999998,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.996,"5min":2.6890000000000001,"15min":2.6890000000000001},"last":0.93000000000000005}]},{"osd":4,"last update":"Fri Oct 6 02:32:19 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.77100000000000002,"15min":0.78700000000000003},"min":{"1min":0.55500000000000005,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0649999999999999,"5min":3.2999999999999998,"15min":3.2999999999999998},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.753,"15min":0.78200000000000003},"min":{"1min":0.53900000000000003,"5min":0.51000000000000001,"15min":0.48499999999999999},"max":{"1min":0.995,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":0.97899999999999998}]},{"osd":5,"last update":"Fri Oct 6 02:32:19 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.752,"15min":0.77200000000000002},"min":{"1min":0.499,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.1699999999999999,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":1.123},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.78600000000000003,"15min":0.79500000000000004},"min":{"1min":0.58099999999999996,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.006,"5min":3.1110000000000002,"15min":3.1110000000000002},"last":1.113}]},{"osd":6,"last update":"Fri Oct 6 02:32:19 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.78800000000000003,"15min":0.81100000000000005},"min":{"1min":0.52800000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.0780000000000001,"5min":3.1600000000000001,"15min":3.1600000000000001},"last":1.1439999999999999},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76600000000000001,"15min":0.80700000000000005},"min":{"1min":0.59999999999999998,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.1100000000000001,"5min":2.3250000000000002,"15min":2.3250000000000002},"last":0.94799999999999995}]},{"osd":7,"last update":"Fri Oct 6 02:32:19 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.81999999999999995,"15min":0.82199999999999995},"min":{"1min":0.58499999999999996,"5min":0.57999999999999996,"15min":0.56000000000000005},"max":{"1min":1.097,"5min":3.3809999999999998,"15min":3.3809999999999998},"last":1.0840000000000001},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77900000000000003,"15min":0.80700000000000005},"min":{"1min":0.59399999999999997,"5min":0.54700000000000004,"15min":0.53300000000000003},"max":{"1min":1.018,"5min":2.3740000000000001,"15min":2.3740000000000001},"last":1.034}]}]},{"osd":2,"up_from":13,"seq":55834574975,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303512,"kb_used_data":2128,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068328,"statfs":{"total":96636764160,"available":96325967872,"internally_reserved":0,"allocated":2179072,"data_stored":1016397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:32:18 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.42399999999999999,"15min":0.434},"min":{"1min":0.28199999999999997,"5min":0.245,"15min":0.245},"max":{"1min":0.59399999999999997,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.495,"5min":0.46600000000000003,"15min":0.47999999999999998},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":1.0800000000000001},"last":0.47799999999999998}]},{"osd":1,"last update":"Fri Oct 6 02:32:18 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.47599999999999998,"15min":0.495},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32400000000000001},"max":{"1min":0.754,"5min":1.036,"15min":1.036},"last":0.502},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51000000000000001,"15min":0.57099999999999995},"min":{"1min":0.40899999999999997,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.68300000000000005,"5min":1.778,"15min":1.778},"last":0.58299999999999996}]},{"osd":3,"last update":"Fri Oct 6 02:32:18 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.499,"15min":0.52100000000000002},"min":{"1min":0.36199999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.97499999999999998},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.57999999999999996,"15min":0.60699999999999998},"min":{"1min":0.46800000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.91700000000000004,"5min":0.95799999999999996,"15min":1.028},"last":0.56899999999999995}]},{"osd":4,"last update":"Fri Oct 6 02:32:18 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.73899999999999999,"15min":0.76400000000000001},"min":{"1min":0.51600000000000001,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.875,"5min":1.0940000000000001,"15min":1.1319999999999999},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.747,"15min":0.81399999999999995},"min":{"1min":0.497,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.0309999999999999,"5min":1.907,"15min":1.907},"last":0.82799999999999996}]},{"osd":5,"last update":"Fri Oct 6 02:32:18 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.72199999999999998,"15min":0.77400000000000002},"min":{"1min":0.51100000000000001,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.97799999999999998,"5min":1.109,"15min":1.1850000000000001},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.751,"15min":0.79800000000000004},"min":{"1min":0.59199999999999997,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.006,"5min":1.0609999999999999,"15min":1.258},"last":0.90300000000000002}]},{"osd":6,"last update":"Fri Oct 6 02:32:18 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77900000000000003,"15min":0.82399999999999995},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.089,"5min":1.827,"15min":1.827},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.76200000000000001,"15min":0.78500000000000003},"min":{"1min":0.52600000000000002,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0469999999999999,"5min":1.0660000000000001,"15min":1.1659999999999999},"last":0.88400000000000001}]},{"osd":7,"last update":"Fri Oct 6 02:32:18 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.78400000000000003,"15min":0.84899999999999998},"min":{"1min":0.61199999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.139,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.77400000000000002,"15min":0.81200000000000006},"min":{"1min":0.57399999999999995,"5min":0.55300000000000005,"15min":0.54800000000000004},"max":{"1min":1.1020000000000001,"5min":1.125,"15min":1.1479999999999999},"last":0.84699999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574972,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303512,"kb_used_data":2128,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068328,"statfs":{"total":96636764160,"available":96325967872,"internally_reserved":0,"allocated":2179072,"data_stored":1016397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:32:15 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.42699999999999999,"15min":0.48199999999999998},"min":{"1min":0.30299999999999999,"5min":0.248,"15min":0.248},"max":{"1min":0.66500000000000004,"5min":1.0489999999999999,"15min":2.024},"last":0.63},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.45000000000000001,"15min":0.5},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.56000000000000005,"5min":0.754,"15min":2.181},"last":0.65400000000000003}]},{"osd":1,"last update":"Fri Oct 6 02:32:15 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.45000000000000001,"15min":0.59899999999999998},"min":{"1min":0.34999999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.65000000000000002,"5min":0.66100000000000003,"15min":4.492},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51000000000000001,"15min":0.64800000000000002},"min":{"1min":0.40999999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.876,"5min":1.079,"15min":4.569},"last":0.69499999999999995}]},{"osd":2,"last update":"Fri Oct 6 02:32:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.58199999999999996,"15min":0.64000000000000001},"min":{"1min":0.435,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.69499999999999995,"5min":1.024,"15min":2.0870000000000002},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.55200000000000005,"15min":0.61799999999999999},"min":{"1min":0.441,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.77400000000000002,"5min":0.92100000000000004,"15min":2.4260000000000002},"last":0.72199999999999998}]},{"osd":4,"last update":"Fri Oct 6 02:32:15 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.72999999999999998,"15min":0.79400000000000004},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.44800000000000001},"max":{"1min":1.024,"5min":1.024,"15min":2.214},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.751,"15min":0.79000000000000004},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.97999999999999998,"5min":1.042,"15min":1.2869999999999999},"last":1.0249999999999999}]},{"osd":5,"last update":"Fri Oct 6 02:32:15 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.745,"15min":0.79700000000000004},"min":{"1min":0.58499999999999996,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.05,"5min":1.077,"15min":1.3029999999999999},"last":1.1299999999999999},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.75,"15min":0.81799999999999995},"min":{"1min":0.55500000000000005,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.097,"5min":1.097,"15min":2.242},"last":1.105}]},{"osd":6,"last update":"Fri Oct 6 02:32:15 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.76700000000000002,"15min":0.80000000000000004},"min":{"1min":0.52300000000000002,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.073,"5min":1.073,"15min":1.357},"last":0.98999999999999999},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.78600000000000003,"15min":0.81799999999999995},"min":{"1min":0.61199999999999999,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.95299999999999996,"5min":1.016,"15min":1.3100000000000001},"last":1.048}]},{"osd":7,"last update":"Fri Oct 6 02:32:15 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.82399999999999995},"min":{"1min":0.58099999999999996,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.996,"5min":1.004,"15min":2.27},"last":1.0660000000000001},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.78500000000000003,"15min":0.83199999999999996},"min":{"1min":0.61699999999999999,"5min":0.55800000000000005,"15min":0.53200000000000003},"max":{"1min":1.0009999999999999,"5min":1.0820000000000001,"15min":1.325},"last":1.1120000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574974,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":307608,"kb_used_data":2128,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064232,"statfs":{"total":96636764160,"available":96321773568,"internally_reserved":0,"allocated":2179072,"data_stored":1016397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:32:10 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.65100000000000002,"15min":0.88900000000000001},"min":{"1min":0.45800000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.877,"5min":1.0760000000000001,"15min":12.337},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.66700000000000004,"15min":0.83299999999999996},"min":{"1min":0.45100000000000001,"5min":0.44400000000000001,"15min":0.438},"max":{"1min":0.91900000000000004,"5min":1.3520000000000001,"15min":9.6539999999999999},"last":0.76800000000000002}]},{"osd":1,"last update":"Fri Oct 6 02:32:10 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64500000000000002,"15min":0.68400000000000005},"min":{"1min":0.47499999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.83699999999999997,"5min":1.129,"15min":3.181},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.70399999999999996,"15min":0.91800000000000004},"min":{"1min":0.48299999999999998,"5min":0.45300000000000001,"15min":0.40500000000000003},"max":{"1min":0.86799999999999999,"5min":0.995,"15min":12.946999999999999},"last":0.71199999999999997}]},{"osd":2,"last update":"Fri Oct 6 02:32:10 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.69499999999999995,"15min":0.93300000000000005},"min":{"1min":0.47299999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.84499999999999997,"5min":1.2989999999999999,"15min":12.377000000000001},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.73099999999999998,"15min":0.82499999999999996},"min":{"1min":0.53700000000000003,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.91200000000000003,"5min":1.0529999999999999,"15min":6.3220000000000001},"last":0.67200000000000004}]},{"osd":3,"last update":"Fri Oct 6 02:32:10 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.72899999999999998,"15min":0.90400000000000003},"min":{"1min":0.57099999999999995,"5min":0.46200000000000002,"15min":0.42699999999999999},"max":{"1min":0.84199999999999997,"5min":1.5229999999999999,"15min":9.7720000000000002},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.70499999999999996,"15min":0.752},"min":{"1min":0.55800000000000005,"5min":0.52400000000000002,"15min":0.47999999999999998},"max":{"1min":0.88900000000000001,"5min":1.212,"15min":3.0840000000000001},"last":0.65300000000000002}]},{"osd":5,"last update":"Fri Oct 6 02:32:10 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72199999999999998,"15min":0.80200000000000005},"min":{"1min":0.46100000000000002,"5min":0.40400000000000003,"15min":0.35299999999999998},"max":{"1min":0.92200000000000004,"5min":1.02,"15min":6.2690000000000001},"last":0.75},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.75600000000000001,"15min":0.93899999999999995},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.33600000000000002},"max":{"1min":0.98199999999999998,"5min":1.4359999999999999,"15min":10.769},"last":0.60799999999999998}]},{"osd":6,"last update":"Fri Oct 6 02:32:10 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71799999999999997,"15min":0.74299999999999999},"min":{"1min":0.46300000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.95299999999999996,"5min":1.4590000000000001,"15min":3.1349999999999998},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.80100000000000005,"15min":0.91100000000000003},"min":{"1min":0.52600000000000002,"5min":0.47799999999999998,"15min":0.45700000000000002},"max":{"1min":0.96799999999999997,"5min":1.3839999999999999,"15min":8.0899999999999999},"last":0.77600000000000002}]},{"osd":7,"last update":"Fri Oct 6 02:32:10 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.80300000000000005,"15min":0.97899999999999998},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.47599999999999998},"max":{"1min":1.02,"5min":1.494,"15min":9.7089999999999996},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.78700000000000003,"15min":0.92800000000000005},"min":{"1min":0.56000000000000005,"5min":0.44400000000000001,"15min":0.374},"max":{"1min":1.034,"5min":1.2669999999999999,"15min":7.6719999999999997},"last":0.58099999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574974,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303512,"kb_used_data":2128,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068328,"statfs":{"total":96636764160,"available":96325967872,"internally_reserved":0,"allocated":2179072,"data_stored":1016397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:32:07 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.63200000000000001,"15min":0.63},"min":{"1min":0.47099999999999997,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.93899999999999995,"5min":0.97599999999999998,"15min":1.4279999999999999},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.629,"15min":0.622},"min":{"1min":0.42599999999999999,"5min":0.40300000000000002,"15min":0.38700000000000001},"max":{"1min":0.89300000000000002,"5min":1.099,"15min":1.099},"last":0.91800000000000004}]},{"osd":1,"last update":"Fri Oct 6 02:32:07 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.63600000000000001,"15min":0.64900000000000002},"min":{"1min":0.51700000000000002,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.93899999999999995,"5min":1.0660000000000001,"15min":1.089},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.67300000000000004,"15min":0.67700000000000005},"min":{"1min":0.42299999999999999,"5min":0.40699999999999997,"15min":0.40500000000000003},"max":{"1min":0.93899999999999995,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.88500000000000001}]},{"osd":2,"last update":"Fri Oct 6 02:32:07 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.67800000000000005,"15min":0.67600000000000005},"min":{"1min":0.45800000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.67200000000000004,"15min":0.68200000000000005},"min":{"1min":0.58599999999999997,"5min":0.46100000000000002,"15min":0.44600000000000001},"max":{"1min":1.028,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.746}]},{"osd":3,"last update":"Fri Oct 6 02:32:07 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.73299999999999998,"15min":0.73199999999999998},"min":{"1min":0.54000000000000004,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0009999999999999,"5min":1.159,"15min":1.159},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.77200000000000002,"15min":0.77000000000000002},"min":{"1min":0.64200000000000002,"5min":0.56200000000000006,"15min":0.54100000000000004},"max":{"1min":0.97699999999999998,"5min":1.194,"15min":1.194},"last":0.86499999999999999}]},{"osd":4,"last update":"Fri Oct 6 02:32:07 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.69099999999999995,"15min":0.67700000000000005},"min":{"1min":0.46999999999999997,"5min":0.40799999999999997,"15min":0.32000000000000001},"max":{"1min":0.92300000000000004,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.69099999999999995,"15min":0.68300000000000005},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.35399999999999998},"max":{"1min":0.88600000000000001,"5min":1.25,"15min":1.25},"last":0.52500000000000002}]},{"osd":6,"last update":"Fri Oct 6 02:32:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.71899999999999997,"15min":0.73499999999999999},"min":{"1min":0.45900000000000002,"5min":0.42899999999999999,"15min":0.40300000000000002},"max":{"1min":1.024,"5min":1.024,"15min":1.1719999999999999},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.81499999999999995,"15min":0.81799999999999995},"min":{"1min":0.60599999999999998,"5min":0.502,"15min":0.373},"max":{"1min":1.0700000000000001,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.60099999999999998}]},{"osd":7,"last update":"Fri Oct 6 02:32:07 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.754,"15min":0.76000000000000001},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.40799999999999997},"max":{"1min":0.95799999999999996,"5min":1.23,"15min":1.23},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.80200000000000005,"15min":0.82099999999999995},"min":{"1min":0.59599999999999997,"5min":0.47599999999999998,"15min":0.437},"max":{"1min":1.1180000000000001,"5min":1.1339999999999999,"15min":1.5640000000000001},"last":0.57099999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T02:33:04.504 DEBUG:teuthology.orchestra.run.smithi031:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-10-06T02:33:04.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.685+0000 7f4f9759d700 1 Processor -- start 2023-10-06T02:33:04.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.685+0000 7f4f9759d700 1 -- start start 2023-10-06T02:33:04.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.687+0000 7f4f9759d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4f90142c00 0x7f4f9006b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:33:04.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.687+0000 7f4f9759d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4f9006bd20 0x7f4f900c7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:33:04.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.687+0000 7f4f9759d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f900ca480 con 0x7f4f90142fd0 2023-10-06T02:33:04.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.687+0000 7f4f9759d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f9015e240 con 0x7f4f9006bd20 2023-10-06T02:33:04.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.687+0000 7f4f9759d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9006b350 con 0x7f4f90142c00 2023-10-06T02:33:04.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.687+0000 7f4f9659b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4f90142c00 0x7f4f9006b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:33:04.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.687+0000 7f4f9659b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4f90142c00 0x7f4f9006b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:45984/0 (socket says 172.21.15.31:45984) 2023-10-06T02:33:04.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.687+0000 7f4f95d9a700 1 --2- 172.21.15.31:0/2338482349 >> v2:172.21.15.62:3300/0 conn(0x7f4f9006bd20 0x7f4f900c7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:33:04.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.687+0000 7f4f9659b700 1 -- 172.21.15.31:0/2338482349 learned_addr learned my addr 172.21.15.31:0/2338482349 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:33:04.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.688+0000 7f4f95599700 1 -- 172.21.15.31:0/2338482349 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 565008146 0 0) 0x7f4f900ca480 con 0x7f4f90142fd0 2023-10-06T02:33:04.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.688+0000 7f4f95599700 1 -- 172.21.15.31:0/2338482349 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f7c003570 con 0x7f4f90142fd0 2023-10-06T02:33:04.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.688+0000 7f4f9659b700 1 -- 172.21.15.31:0/2338482349 >> v2:172.21.15.62:3300/0 conn(0x7f4f9006bd20 msgr2=0x7f4f900c7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:33:04.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.688+0000 7f4f9659b700 1 --2- 172.21.15.31:0/2338482349 >> v2:172.21.15.62:3300/0 conn(0x7f4f9006bd20 0x7f4f900c7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:33:04.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.688+0000 7f4f9659b700 1 -- 172.21.15.31:0/2338482349 >> v1:172.21.15.62:6789/0 conn(0x7f4f90142fd0 legacy=0x7f4f9006c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:33:04.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.688+0000 7f4f9659b700 1 -- 172.21.15.31:0/2338482349 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f9015df60 con 0x7f4f90142c00 2023-10-06T02:33:04.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.688+0000 7f4f95d9a700 1 --2- 172.21.15.31:0/2338482349 >> v2:172.21.15.62:3300/0 conn(0x7f4f9006bd20 0x7f4f900c7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:33:04.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.688+0000 7f4f9659b700 1 --2- 172.21.15.31:0/2338482349 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4f90142c00 0x7f4f9006b7e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c0099b0 tx=0x7f4f8c009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0ce71753fbeaf04 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:33:04.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.688+0000 7f4f95599700 1 -- 172.21.15.31:0/2338482349 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c00b680 con 0x7f4f90142c00 2023-10-06T02:33:04.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.689+0000 7f4f95599700 1 -- 172.21.15.31:0/2338482349 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f8c00b7e0 con 0x7f4f90142c00 2023-10-06T02:33:04.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.689+0000 7f4f95599700 1 -- 172.21.15.31:0/2338482349 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c00bba0 con 0x7f4f90142c00 2023-10-06T02:33:04.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.689+0000 7f4f9759d700 1 -- 172.21.15.31:0/2338482349 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4f90142c00 msgr2=0x7f4f9006b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:33:04.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.689+0000 7f4f9759d700 1 --2- 172.21.15.31:0/2338482349 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4f90142c00 0x7f4f9006b7e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c0099b0 tx=0x7f4f8c009250 comp rx=0 tx=0).stop 2023-10-06T02:33:04.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.689+0000 7f4f9759d700 1 -- 172.21.15.31:0/2338482349 shutdown_connections 2023-10-06T02:33:04.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.689+0000 7f4f9759d700 1 --2- 172.21.15.31:0/2338482349 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4f90142c00 0x7f4f9006b7e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:33:04.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.689+0000 7f4f9759d700 1 --2- 172.21.15.31:0/2338482349 >> v2:172.21.15.62:3300/0 conn(0x7f4f9006bd20 0x7f4f900c7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:33:04.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.689+0000 7f4f9759d700 1 -- 172.21.15.31:0/2338482349 >> 172.21.15.31:0/2338482349 conn(0x7f4f900675a0 msgr2=0x7f4f900679a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:33:04.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.689+0000 7f4f9759d700 1 -- 172.21.15.31:0/2338482349 shutdown_connections 2023-10-06T02:33:04.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.689+0000 7f4f9759d700 1 -- 172.21.15.31:0/2338482349 wait complete. 2023-10-06T02:33:04.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.691+0000 7f4f9759d700 1 Processor -- start 2023-10-06T02:33:04.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.691+0000 7f4f9759d700 1 -- start start 2023-10-06T02:33:04.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.691+0000 7f4f9759d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4f9006bd20 0x7f4f9013da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:33:04.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.692+0000 7f4f9759d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f90142c00 0x7f4f9016a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:33:04.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.692+0000 7f4f9759d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f90142fd0 0x7f4f900affe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:33:04.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.692+0000 7f4f9759d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f90169c70 con 0x7f4f90142c00 2023-10-06T02:33:04.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.692+0000 7f4f9759d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9013df60 con 0x7f4f90142fd0 2023-10-06T02:33:04.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.692+0000 7f4f9759d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9016aba0 con 0x7f4f9006bd20 2023-10-06T02:33:04.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.692+0000 7f4f95d9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f90142c00 0x7f4f9016a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:33:04.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.692+0000 7f4f95d9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f90142c00 0x7f4f9016a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51698/0 (socket says 172.21.15.31:51698) 2023-10-06T02:33:04.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.692+0000 7f4f95d9a700 1 -- 172.21.15.31:0/2047433576 learned_addr learned my addr 172.21.15.31:0/2047433576 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:33:04.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.692+0000 7f4f9659b700 1 --2- 172.21.15.31:0/2047433576 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4f9006bd20 0x7f4f9013da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:33:04.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.692+0000 7f4f95d9a700 1 -- 172.21.15.31:0/2047433576 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4f9006bd20 msgr2=0x7f4f9013da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:33:04.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.692+0000 7f4f95d9a700 1 --2- 172.21.15.31:0/2047433576 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4f9006bd20 0x7f4f9013da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:33:04.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.692+0000 7f4f95d9a700 1 -- 172.21.15.31:0/2047433576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f90142fd0 msgr2=0x7f4f900affe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:33:04.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.692+0000 7f4f96d9c700 1 --2- 172.21.15.31:0/2047433576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f90142fd0 0x7f4f900affe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:33:04.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.692+0000 7f4f95d9a700 1 --2- 172.21.15.31:0/2047433576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f90142fd0 0x7f4f900affe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:33:04.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.692+0000 7f4f95d9a700 1 -- 172.21.15.31:0/2047433576 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f80009420 con 0x7f4f90142c00 2023-10-06T02:33:04.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.692+0000 7f4f9659b700 1 --2- 172.21.15.31:0/2047433576 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4f9006bd20 0x7f4f9013da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:33:04.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.692+0000 7f4f96d9c700 1 --2- 172.21.15.31:0/2047433576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f90142fd0 0x7f4f900affe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:33:04.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.693+0000 7f4f95d9a700 1 --2- 172.21.15.31:0/2047433576 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f90142c00 0x7f4f9016a660 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f4f8000ecb0 tx=0x7f4f800091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:33:04.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.693+0000 7f4f867fc700 1 -- 172.21.15.31:0/2047433576 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f8000ee60 con 0x7f4f90142c00 2023-10-06T02:33:04.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.693+0000 7f4f867fc700 1 -- 172.21.15.31:0/2047433576 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f8000cbe0 con 0x7f4f90142c00 2023-10-06T02:33:04.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.693+0000 7f4f9759d700 1 -- 172.21.15.31:0/2047433576 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f8c0094f0 con 0x7f4f90142c00 2023-10-06T02:33:04.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.693+0000 7f4f867fc700 1 -- 172.21.15.31:0/2047433576 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f80013470 con 0x7f4f90142c00 2023-10-06T02:33:04.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.693+0000 7f4f9759d700 1 -- 172.21.15.31:0/2047433576 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f9015d580 con 0x7f4f90142c00 2023-10-06T02:33:04.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.694+0000 7f4f9759d700 1 -- 172.21.15.31:0/2047433576 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f9016a4b0 con 0x7f4f90142c00 2023-10-06T02:33:04.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.695+0000 7f4f867fc700 1 -- 172.21.15.31:0/2047433576 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) v1 ==== 85709+0+0 (secure 0 0 0) 0x7f4f80013820 con 0x7f4f90142c00 2023-10-06T02:33:04.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.696+0000 7f4f867fc700 1 --2- 172.21.15.31:0/2047433576 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f4f740679e0 0x7f4f74069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:33:04.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.696+0000 7f4f867fc700 1 -- 172.21.15.31:0/2047433576 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5320+0+0 (secure 0 0 0) 0x7f4f80083300 con 0x7f4f90142c00 2023-10-06T02:33:04.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.696+0000 7f4f9659b700 1 --2- 172.21.15.31:0/2047433576 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f4f740679e0 0x7f4f74069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:33:04.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.697+0000 7f4f9659b700 1 --2- 172.21.15.31:0/2047433576 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f4f740679e0 0x7f4f74069e90 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c0099b0 tx=0x7f4f8c017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:33:04.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.700+0000 7f4f867fc700 1 -- 172.21.15.31:0/2047433576 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4f80051f40 con 0x7f4f90142c00 2023-10-06T02:33:04.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.901+0000 7f4f9759d700 1 -- 172.21.15.31:0/2047433576 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f4f9004c3c0 con 0x7f4f90142c00 2023-10-06T02:33:04.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.907+0000 7f4f867fc700 1 -- 172.21.15.31:0/2047433576 <== mon.0 v2:172.21.15.31:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f4f80051f40 con 0x7f4f90142c00 2023-10-06T02:33:04.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.908+0000 7f4f867fc700 1 -- 172.21.15.31:0/2047433576 <== mon.0 v2:172.21.15.31:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v32)=0 v32) v1 ==== 123+0+0 (secure 0 0 0) 0x7f4f80055560 con 0x7f4f90142c00 2023-10-06T02:33:04.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.910+0000 7f4f9759d700 1 -- 172.21.15.31:0/2047433576 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f4f740679e0 msgr2=0x7f4f74069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:33:04.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.910+0000 7f4f9759d700 1 --2- 172.21.15.31:0/2047433576 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f4f740679e0 0x7f4f74069e90 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c0099b0 tx=0x7f4f8c017000 comp rx=0 tx=0).stop 2023-10-06T02:33:04.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.910+0000 7f4f9759d700 1 -- 172.21.15.31:0/2047433576 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f90142c00 msgr2=0x7f4f9016a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:33:04.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.910+0000 7f4f9759d700 1 --2- 172.21.15.31:0/2047433576 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f90142c00 0x7f4f9016a660 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f4f8000ecb0 tx=0x7f4f800091a0 comp rx=0 tx=0).stop 2023-10-06T02:33:04.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.910+0000 7f4f9759d700 1 -- 172.21.15.31:0/2047433576 shutdown_connections 2023-10-06T02:33:04.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.910+0000 7f4f9759d700 1 --2- 172.21.15.31:0/2047433576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f90142fd0 0x7f4f900affe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:33:04.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.910+0000 7f4f9759d700 1 --2- 172.21.15.31:0/2047433576 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7f4f740679e0 0x7f4f74069e90 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:33:04.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.910+0000 7f4f9759d700 1 --2- 172.21.15.31:0/2047433576 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4f9006bd20 0x7f4f9013da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:33:04.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.910+0000 7f4f9759d700 1 --2- 172.21.15.31:0/2047433576 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f90142c00 0x7f4f9016a660 unknown :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:33:04.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.910+0000 7f4f9759d700 1 -- 172.21.15.31:0/2047433576 >> 172.21.15.31:0/2047433576 conn(0x7f4f900675a0 msgr2=0x7f4f900c78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:33:04.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.911+0000 7f4f9759d700 1 -- 172.21.15.31:0/2047433576 shutdown_connections 2023-10-06T02:33:04.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:04.911+0000 7f4f9759d700 1 -- 172.21.15.31:0/2047433576 wait complete. 2023-10-06T02:33:04.929 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T02:33:04.929 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:33:05.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:05.115+0000 7ffa0e19d700 1 Processor -- start 2023-10-06T02:33:05.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:05.115+0000 7ffa0e19d700 1 -- start start 2023-10-06T02:33:05.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:05.116+0000 7ffa0e19d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ffa08153c30 0x7ffa0815a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:33:05.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:05.116+0000 7ffa0e19d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ffa0815de50 0x7ffa080cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:33:05.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:05.116+0000 7ffa0e19d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffa080cef40 0x7ffa0806d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:33:05.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:05.116+0000 7ffa0e19d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa080cf3c0 con 0x7ffa08153c30 2023-10-06T02:33:05.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:05.116+0000 7ffa0e19d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa0815afd0 con 0x7ffa0815de50 2023-10-06T02:33:05.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:05.116+0000 7ffa0e19d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0815b2b0 con 0x7ffa080cef40 2023-10-06T02:33:05.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:05.117+0000 7ffa0d19b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ffa08153c30 0x7ffa0815a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:33:05.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:05.117+0000 7ffa0d19b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ffa08153c30 0x7ffa0815a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51704/0 (socket says 172.21.15.31:51704) 2023-10-06T02:33:05.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:05.117+0000 7ffa0d19b700 1 -- 172.21.15.31:0/1882199817 learned_addr learned my addr 172.21.15.31:0/1882199817 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:33:05.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:05.117+0000 7ffa0d99c700 1 --2- 172.21.15.31:0/1882199817 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffa080cef40 0x7ffa0806d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:33:05.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:05.117+0000 7ffa0c99a700 1 --2- 172.21.15.31:0/1882199817 >> v2:172.21.15.62:3300/0 conn(0x7ffa0815de50 0x7ffa080cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:33:05.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:05.117+0000 7ffa0d19b700 1 -- 172.21.15.31:0/1882199817 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffa080cef40 msgr2=0x7ffa0806d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:33:05.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:05.117+0000 7ffa0d19b700 1 --2- 172.21.15.31:0/1882199817 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffa080cef40 0x7ffa0806d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:33:05.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:05.117+0000 7ffa0d19b700 1 -- 172.21.15.31:0/1882199817 >> v2:172.21.15.62:3300/0 conn(0x7ffa0815de50 msgr2=0x7ffa080cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:33:05.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:05.117+0000 7ffa0d19b700 1 --2- 172.21.15.31:0/1882199817 >> v2:172.21.15.62:3300/0 conn(0x7ffa0815de50 0x7ffa080cea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:33:05.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:05.117+0000 7ffa0d19b700 1 -- 172.21.15.31:0/1882199817 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa0815b590 con 0x7ffa08153c30 2023-10-06T02:33:05.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:05.117+0000 7ffa0d19b700 1 --2- 172.21.15.31:0/1882199817 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa08153c30 0x7ffa0815a5e0 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7ffa080713a0 tx=0x7ffa04009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ed489df2bfea0fa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:33:05.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:05.118+0000 7ff9fffff700 1 -- 172.21.15.31:0/1882199817 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0400b860 con 0x7ffa08153c30 2023-10-06T02:33:05.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:05.118+0000 7ff9fffff700 1 -- 172.21.15.31:0/1882199817 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7ffa0400b9c0 con 0x7ffa08153c30 2023-10-06T02:33:05.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:05.118+0000 7ff9fffff700 1 -- 172.21.15.31:0/1882199817 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0400bd80 con 0x7ffa08153c30 2023-10-06T02:33:05.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:05.118+0000 7ffa0e19d700 1 -- 172.21.15.31:0/1882199817 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa08153c30 msgr2=0x7ffa0815a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:33:05.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:05.118+0000 7ffa0e19d700 1 --2- 172.21.15.31:0/1882199817 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa08153c30 0x7ffa0815a5e0 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7ffa080713a0 tx=0x7ffa04009250 comp rx=0 tx=0).stop 2023-10-06T02:33:05.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:05.118+0000 7ffa0e19d700 1 -- 172.21.15.31:0/1882199817 shutdown_connections 2023-10-06T02:33:05.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:05.118+0000 7ffa0e19d700 1 --2- 172.21.15.31:0/1882199817 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffa080cef40 0x7ffa0806d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:33:05.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:05.118+0000 7ffa0e19d700 1 --2- 172.21.15.31:0/1882199817 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa08153c30 0x7ffa0815a5e0 unknown :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:33:05.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:05.118+0000 7ffa0e19d700 1 --2- 172.21.15.31:0/1882199817 >> v2:172.21.15.62:3300/0 conn(0x7ffa0815de50 0x7ffa080cea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:33:05.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:05.118+0000 7ffa0e19d700 1 -- 172.21.15.31:0/1882199817 >> 172.21.15.31:0/1882199817 conn(0x7ffa080703a0 msgr2=0x7ffa080707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:33:05.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:05.118+0000 7ffa0e19d700 1 -- 172.21.15.31:0/1882199817 shutdown_connections 2023-10-06T02:33:05.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:05.118+0000 7ffa0e19d700 1 -- 172.21.15.31:0/1882199817 wait complete. 2023-10-06T02:33:05.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:05.120+0000 7ffa0e19d700 1 Processor -- start 2023-10-06T02:33:05.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:05.120+0000 7ffa0e19d700 1 -- start start 2023-10-06T02:33:05.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:05.122+0000 7ffa0e19d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa080cef40 0x7ffa081408d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:33:05.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:05.122+0000 7ffa0e19d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffa08153c30 0x7ffa080b4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:33:05.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:05.122+0000 7ffa0e19d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa0815de50 0x7ffa080d0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:33:05.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:05.122+0000 7ffa0e19d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa080b5a40 con 0x7ffa080cef40 2023-10-06T02:33:05.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:05.122+0000 7ffa0e19d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa080b4e30 con 0x7ffa0815de50 2023-10-06T02:33:05.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:05.122+0000 7ffa0e19d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa080b5110 con 0x7ffa08153c30 2023-10-06T02:33:05.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:05.122+0000 7ffa0d19b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa080cef40 0x7ffa081408d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:33:05.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:05.122+0000 7ffa0d19b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa080cef40 0x7ffa081408d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51720/0 (socket says 172.21.15.31:51720) 2023-10-06T02:33:05.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:05.122+0000 7ffa0d19b700 1 -- 172.21.15.31:0/1821192918 learned_addr learned my addr 172.21.15.31:0/1821192918 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T02:33:05.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:05.122+0000 7ffa0c99a700 1 --2- 172.21.15.31:0/1821192918 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffa08153c30 0x7ffa080b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:33:05.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:05.122+0000 7ffa0d99c700 1 --2- 172.21.15.31:0/1821192918 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa0815de50 0x7ffa080d0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:33:05.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:05.122+0000 7ffa0d19b700 1 -- 172.21.15.31:0/1821192918 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffa08153c30 msgr2=0x7ffa080b4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:33:05.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:05.122+0000 7ffa0d19b700 1 --2- 172.21.15.31:0/1821192918 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffa08153c30 0x7ffa080b4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:33:05.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:05.122+0000 7ffa0d19b700 1 -- 172.21.15.31:0/1821192918 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa0815de50 msgr2=0x7ffa080d0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:33:05.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:05.122+0000 7ffa0d19b700 1 --2- 172.21.15.31:0/1821192918 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa0815de50 0x7ffa080d0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:33:05.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:05.122+0000 7ffa0d19b700 1 -- 172.21.15.31:0/1821192918 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa040094f0 con 0x7ffa080cef40 2023-10-06T02:33:05.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:05.122+0000 7ffa0d19b700 1 --2- 172.21.15.31:0/1821192918 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa080cef40 0x7ffa081408d0 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7ffa0806eb00 tx=0x7ffa04009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:33:05.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:05.122+0000 7ff9fd7fa700 1 -- 172.21.15.31:0/1821192918 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa04011e60 con 0x7ffa080cef40 2023-10-06T02:33:05.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:05.122+0000 7ff9fd7fa700 1 -- 172.21.15.31:0/1821192918 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7ffa0401f070 con 0x7ffa080cef40 2023-10-06T02:33:05.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:05.122+0000 7ffa0e19d700 1 -- 172.21.15.31:0/1821192918 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa080b53f0 con 0x7ffa080cef40 2023-10-06T02:33:05.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:05.122+0000 7ff9fd7fa700 1 -- 172.21.15.31:0/1821192918 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0401a8b0 con 0x7ffa080cef40 2023-10-06T02:33:05.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:05.122+0000 7ffa0e19d700 1 -- 172.21.15.31:0/1821192918 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa08140e10 con 0x7ffa080cef40 2023-10-06T02:33:05.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:05.123+0000 7ffa0e19d700 1 -- 172.21.15.31:0/1821192918 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa0804c3c0 con 0x7ffa080cef40 2023-10-06T02:33:05.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:05.124+0000 7ff9fd7fa700 1 -- 172.21.15.31:0/1821192918 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) v1 ==== 85709+0+0 (secure 0 0 0) 0x7ffa0401ac20 con 0x7ffa080cef40 2023-10-06T02:33:05.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:05.125+0000 7ff9fd7fa700 1 --2- 172.21.15.31:0/1821192918 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff9ec0679e0 0x7ff9ec069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:33:05.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:05.125+0000 7ff9fd7fa700 1 -- 172.21.15.31:0/1821192918 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5320+0+0 (secure 0 0 0) 0x7ffa04021070 con 0x7ffa080cef40 2023-10-06T02:33:05.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:05.125+0000 7ffa0c99a700 1 --2- 172.21.15.31:0/1821192918 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff9ec0679e0 0x7ff9ec069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:33:05.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:05.126+0000 7ffa0c99a700 1 --2- 172.21.15.31:0/1821192918 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff9ec0679e0 0x7ff9ec069e90 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7ffa080c68e0 tx=0x7ff9f8008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:33:05.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:05.130+0000 7ff9fd7fa700 1 -- 172.21.15.31:0/1821192918 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffa04053a20 con 0x7ffa080cef40 2023-10-06T02:33:05.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:05.378+0000 7ffa0e19d700 1 -- 172.21.15.31:0/1821192918 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffa0815b870 con 0x7ffa080cef40 2023-10-06T02:33:05.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:05.378+0000 7ff9fd7fa700 1 -- 172.21.15.31:0/1821192918 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 122 v122) v1 ==== 95+0+3525 (secure 0 0 0) 0x7ffa04015020 con 0x7ffa080cef40 2023-10-06T02:33:05.380 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T02:33:05.380 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":122,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7445,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/324072138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":324072138},{"type":"v1","addr":"172.21.15.31:6835","nonce":324072138}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":114},{"gid":7448,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/3160488596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3160488596},{"type":"v1","addr":"172.21.15.62:6834","nonce":3160488596}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":114},{"gid":8677,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/632134516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":632134516},{"type":"v1","addr":"172.21.15.62:6835","nonce":632134516}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":114},{"gid":9594,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/426190720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":426190720},{"type":"v1","addr":"172.21.15.31:6834","nonce":426190720}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":122}],"filesystems":[]} 2023-10-06T02:33:05.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:05.381+0000 7ffa0e19d700 1 -- 172.21.15.31:0/1821192918 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff9ec0679e0 msgr2=0x7ff9ec069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:33:05.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:05.381+0000 7ffa0e19d700 1 --2- 172.21.15.31:0/1821192918 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff9ec0679e0 0x7ff9ec069e90 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7ffa080c68e0 tx=0x7ff9f8008040 comp rx=0 tx=0).stop 2023-10-06T02:33:05.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:05.381+0000 7ffa0e19d700 1 -- 172.21.15.31:0/1821192918 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa080cef40 msgr2=0x7ffa081408d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:33:05.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:05.381+0000 7ffa0e19d700 1 --2- 172.21.15.31:0/1821192918 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa080cef40 0x7ffa081408d0 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7ffa0806eb00 tx=0x7ffa04009250 comp rx=0 tx=0).stop 2023-10-06T02:33:05.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:05.381+0000 7ffa0e19d700 1 -- 172.21.15.31:0/1821192918 shutdown_connections 2023-10-06T02:33:05.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:05.381+0000 7ffa0e19d700 1 --2- 172.21.15.31:0/1821192918 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa0815de50 0x7ffa080d0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:33:05.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:05.381+0000 7ffa0e19d700 1 --2- 172.21.15.31:0/1821192918 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa080cef40 0x7ffa081408d0 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:33:05.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:05.381+0000 7ffa0e19d700 1 --2- 172.21.15.31:0/1821192918 >> [v2:172.21.15.62:6836/85212,v1:172.21.15.62:6837/85212] conn(0x7ff9ec0679e0 0x7ff9ec069e90 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:33:05.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:05.381+0000 7ffa0e19d700 1 --2- 172.21.15.31:0/1821192918 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffa08153c30 0x7ffa080b4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:33:05.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:05.381+0000 7ffa0e19d700 1 -- 172.21.15.31:0/1821192918 >> 172.21.15.31:0/1821192918 conn(0x7ffa080703a0 msgr2=0x7ffa08153100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:33:05.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:05.381+0000 7ffa0e19d700 1 -- 172.21.15.31:0/1821192918 shutdown_connections 2023-10-06T02:33:05.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T02:33:05.381+0000 7ffa0e19d700 1 -- 172.21.15.31:0/1821192918 wait complete. 2023-10-06T02:33:05.384 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 122 2023-10-06T02:33:05.400 DEBUG:tasks.cephfs.filesystem:already dead... 2023-10-06T02:33:05.400 INFO:teuthology.misc:Shutting down mds daemons... 2023-10-06T02:33:05.400 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T02:33:05.400 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T02:33:05.412 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T02:33:05.413 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T02:33:05.413 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T02:33:05.465 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T02:33:05.466 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T02:33:05.466 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T02:33:05.481 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T02:33:05.481 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T02:33:05.482 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T02:33:05.537 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T02:33:05.537 INFO:teuthology.misc:Shutting down osd daemons... 2023-10-06T02:33:05.537 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-10-06T02:33:05.538 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T02:33:05.574 INFO:tasks.ceph.osd.0:Stopped 2023-10-06T02:33:05.574 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-10-06T02:33:05.574 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T02:33:05.655 INFO:tasks.ceph.osd.1:Stopped 2023-10-06T02:33:05.656 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-10-06T02:33:05.656 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T02:33:05.700 INFO:tasks.ceph.osd.2:Stopped 2023-10-06T02:33:05.700 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-10-06T02:33:05.700 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T02:33:05.748 INFO:tasks.ceph.osd.3:Stopped 2023-10-06T02:33:05.749 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-10-06T02:33:05.749 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T02:33:05.796 INFO:tasks.ceph.osd.4:Stopped 2023-10-06T02:33:05.796 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-10-06T02:33:05.796 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T02:33:05.851 INFO:tasks.ceph.osd.5:Stopped 2023-10-06T02:33:05.852 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-10-06T02:33:05.852 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T02:33:05.899 INFO:tasks.ceph.osd.6:Stopped 2023-10-06T02:33:05.900 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-10-06T02:33:05.900 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T02:33:05.947 INFO:tasks.ceph.osd.7:Stopped 2023-10-06T02:33:05.947 INFO:teuthology.misc:Shutting down mgr daemons... 2023-10-06T02:33:05.948 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-10-06T02:33:05.948 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T02:33:06.008 INFO:tasks.ceph.mgr.y:Stopped 2023-10-06T02:33:06.009 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-10-06T02:33:06.009 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T02:33:06.076 INFO:tasks.ceph.mgr.x:Stopped 2023-10-06T02:33:06.076 INFO:teuthology.misc:Shutting down mon daemons... 2023-10-06T02:33:06.077 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-10-06T02:33:06.077 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T02:33:06.097 INFO:tasks.ceph.mon.a:Stopped 2023-10-06T02:33:06.097 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-10-06T02:33:06.097 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T02:33:06.115 INFO:tasks.ceph.mon.b:Stopped 2023-10-06T02:33:06.115 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-10-06T02:33:06.115 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T02:33:06.173 INFO:tasks.ceph.mon.c:Stopped 2023-10-06T02:33:06.174 INFO:tasks.ceph:Checking cluster log for badness... 2023-10-06T02:33:06.174 DEBUG:teuthology.orchestra.run.smithi031:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | head -n 1 2023-10-06T02:33:06.253 INFO:teuthology.orchestra.run.smithi031.stdout:1696558981.970328 mon.a (mon.0) 200 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-06T02:33:06.254 WARNING:tasks.ceph:Found errors (ERR|WRN|SEC) in cluster log 2023-10-06T02:33:06.254 DEBUG:teuthology.orchestra.run.smithi031:> sudo egrep '\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | head -n 1 2023-10-06T02:33:06.335 DEBUG:teuthology.orchestra.run.smithi031:> sudo egrep '\[ERR\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | head -n 1 2023-10-06T02:33:06.413 DEBUG:teuthology.orchestra.run.smithi031:> sudo egrep '\[WRN\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | head -n 1 2023-10-06T02:33:06.489 INFO:teuthology.orchestra.run.smithi031.stdout:1696558981.970328 mon.a (mon.0) 200 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-06T02:33:06.490 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi031.front.sepia.ceph.com 2023-10-06T02:33:06.490 DEBUG:teuthology.orchestra.run.smithi031:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-10-06T02:33:06.797 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi031.front.sepia.ceph.com 2023-10-06T02:33:06.798 DEBUG:teuthology.orchestra.run.smithi031:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-10-06T02:33:07.012 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi031.front.sepia.ceph.com 2023-10-06T02:33:07.012 DEBUG:teuthology.orchestra.run.smithi031:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-10-06T02:33:07.255 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi031.front.sepia.ceph.com 2023-10-06T02:33:07.255 DEBUG:teuthology.orchestra.run.smithi031:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-10-06T02:33:07.431 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi062.front.sepia.ceph.com 2023-10-06T02:33:07.431 DEBUG:teuthology.orchestra.run.smithi062:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-10-06T02:33:07.755 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi062.front.sepia.ceph.com 2023-10-06T02:33:07.755 DEBUG:teuthology.orchestra.run.smithi062:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-10-06T02:33:07.957 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi062.front.sepia.ceph.com 2023-10-06T02:33:07.957 DEBUG:teuthology.orchestra.run.smithi062:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-10-06T02:33:08.171 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi062.front.sepia.ceph.com 2023-10-06T02:33:08.171 DEBUG:teuthology.orchestra.run.smithi062:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-10-06T02:33:08.350 INFO:tasks.ceph:Archiving mon data... 2023-10-06T02:33:08.351 DEBUG:teuthology.misc:Transferring archived files from smithi031:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412129/data/mon.a.tgz 2023-10-06T02:33:08.351 DEBUG:teuthology.orchestra.run.smithi031:> mktemp 2023-10-06T02:33:08.392 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/tmp.g2ZPbswfwU 2023-10-06T02:33:08.392 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.g2ZPbswfwU 2023-10-06T02:33:08.693 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 0666 /tmp/tmp.g2ZPbswfwU 2023-10-06T02:33:08.793 DEBUG:teuthology.orchestra.remote:smithi031:/tmp/tmp.g2ZPbswfwU is 1MB 2023-10-06T02:33:08.893 DEBUG:teuthology.orchestra.run.smithi031:> rm -fr /tmp/tmp.g2ZPbswfwU 2023-10-06T02:33:08.930 DEBUG:teuthology.misc:Transferring archived files from smithi062:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412129/data/mon.b.tgz 2023-10-06T02:33:08.930 DEBUG:teuthology.orchestra.run.smithi062:> mktemp 2023-10-06T02:33:08.968 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/tmp.8HtaFb4kRe 2023-10-06T02:33:08.968 DEBUG:teuthology.orchestra.run.smithi062:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.8HtaFb4kRe 2023-10-06T02:33:09.325 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 0666 /tmp/tmp.8HtaFb4kRe 2023-10-06T02:33:09.426 DEBUG:teuthology.orchestra.remote:smithi062:/tmp/tmp.8HtaFb4kRe is 1MB 2023-10-06T02:33:09.526 DEBUG:teuthology.orchestra.run.smithi062:> rm -fr /tmp/tmp.8HtaFb4kRe 2023-10-06T02:33:09.563 DEBUG:teuthology.misc:Transferring archived files from smithi062:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412129/data/mon.c.tgz 2023-10-06T02:33:09.563 DEBUG:teuthology.orchestra.run.smithi062:> mktemp 2023-10-06T02:33:09.638 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/tmp.OE78R612DH 2023-10-06T02:33:09.638 DEBUG:teuthology.orchestra.run.smithi062:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.OE78R612DH 2023-10-06T02:33:09.940 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 0666 /tmp/tmp.OE78R612DH 2023-10-06T02:33:10.041 DEBUG:teuthology.orchestra.remote:smithi062:/tmp/tmp.OE78R612DH is 1MB 2023-10-06T02:33:10.139 DEBUG:teuthology.orchestra.run.smithi062:> rm -fr /tmp/tmp.OE78R612DH 2023-10-06T02:33:10.176 INFO:tasks.ceph:Cleaning ceph cluster... 2023-10-06T02:33:10.176 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-06T02:33:10.181 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-06T02:33:10.278 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-10-06T02:33:10.278 DEBUG:teuthology.orchestra.run.smithi031:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-06T02:33:10.283 DEBUG:teuthology.orchestra.run.smithi062:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-06T02:33:10.357 INFO:teuthology.orchestra.run.smithi062.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-06T02:33:10.376 INFO:teuthology.orchestra.run.smithi031.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-06T02:33:10.392 INFO:tasks.ceph:Archiving crash dumps... 2023-10-06T02:33:10.393 DEBUG:teuthology.misc:Transferring archived files from smithi031:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412129/remote/smithi031/crash 2023-10-06T02:33:10.394 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-06T02:33:10.456 DEBUG:teuthology.misc:Transferring archived files from smithi062:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412129/remote/smithi062/crash 2023-10-06T02:33:10.457 DEBUG:teuthology.orchestra.run.smithi062:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-06T02:33:10.518 INFO:tasks.ceph:Shutting down logrotate 2023-10-06T02:33:10.518 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T02:33:10.523 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T02:33:10.616 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-06T02:33:10.677 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-06T02:33:10.738 INFO:tasks.ceph:Compressing logs... 2023-10-06T02:33:10.738 DEBUG:teuthology.orchestra.run.smithi031:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-06T02:33:10.743 DEBUG:teuthology.orchestra.run.smithi062:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-06T02:33:16.306 INFO:tasks.ceph:Archiving logs... 2023-10-06T02:33:16.307 DEBUG:teuthology.misc:Transferring archived files from smithi031:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412129/remote/smithi031/log 2023-10-06T02:33:16.308 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-06T02:33:17.548 DEBUG:teuthology.misc:Transferring archived files from smithi062:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412129/remote/smithi062/log 2023-10-06T02:33:17.549 DEBUG:teuthology.orchestra.run.smithi062:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-06T02:33:18.126 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-10-06T02:33:18.135 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-10-06T02:33:18.135 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-06T02:33:18.140 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-06T02:33:18.266 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-10-06T02:33:18.276 INFO:teuthology.task.clock:Checking final clock skew... 2023-10-06T02:33:18.277 DEBUG:teuthology.orchestra.run.smithi031:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-06T02:33:18.282 DEBUG:teuthology.orchestra.run.smithi062:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-06T02:33:18.314 INFO:teuthology.orchestra.run.smithi031.stderr:bash: ntpq: command not found 2023-10-06T02:33:18.316 INFO:teuthology.orchestra.run.smithi062.stderr:bash: ntpq: command not found 2023-10-06T02:33:18.319 INFO:teuthology.orchestra.run.smithi031.stdout:210 Number of sources = 4 2023-10-06T02:33:18.320 INFO:teuthology.orchestra.run.smithi031.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-06T02:33:18.320 INFO:teuthology.orchestra.run.smithi031.stdout:=============================================================================== 2023-10-06T02:33:18.320 INFO:teuthology.orchestra.run.smithi031.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 46 -251us[ -251us] +/- 91ms 2023-10-06T02:33:18.320 INFO:teuthology.orchestra.run.smithi031.stdout:^* hv02.front.sepia.ceph.com 3 6 377 46 -397us[ -397us] +/- 78ms 2023-10-06T02:33:18.320 INFO:teuthology.orchestra.run.smithi031.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 46 +626us[ +626us] +/- 85ms 2023-10-06T02:33:18.320 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T02:33:18.322 INFO:teuthology.orchestra.run.smithi062.stdout:210 Number of sources = 4 2023-10-06T02:33:18.322 INFO:teuthology.orchestra.run.smithi062.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-06T02:33:18.322 INFO:teuthology.orchestra.run.smithi062.stdout:=============================================================================== 2023-10-06T02:33:18.322 INFO:teuthology.orchestra.run.smithi062.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 45 -223us[ -211us] +/- 91ms 2023-10-06T02:33:18.322 INFO:teuthology.orchestra.run.smithi062.stdout:^* hv02.front.sepia.ceph.com 3 6 377 45 -392us[ -380us] +/- 78ms 2023-10-06T02:33:18.322 INFO:teuthology.orchestra.run.smithi062.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 44 +643us[ +643us] +/- 85ms 2023-10-06T02:33:18.322 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T02:33:18.323 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-10-06T02:33:18.332 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-10-06T02:33:18.332 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-10-06T02:33:18.374 DEBUG:teuthology.orchestra.run.smithi031:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-06T02:33:18.475 DEBUG:teuthology.orchestra.run.smithi062:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-06T02:33:18.646 DEBUG:teuthology.orchestra.run.smithi031:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-06T02:33:18.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T02:33:18.710 DEBUG:teuthology.orchestra.run.smithi062:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-06T02:33:18.780 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T02:33:18.781 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-10-06T02:33:18.791 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-10-06T02:33:18.824 INFO:teuthology.task.internal:Duration was 1539.519668 seconds 2023-10-06T02:33:18.825 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-10-06T02:33:18.875 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-10-06T02:33:18.876 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-06T02:33:18.881 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-06T02:33:18.989 INFO:teuthology.orchestra.run.smithi031.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-06T02:33:18.992 INFO:teuthology.orchestra.run.smithi062.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-06T02:33:19.344 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-10-06T02:33:19.344 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi031.front.sepia.ceph.com 2023-10-06T02:33:19.344 DEBUG:teuthology.orchestra.run.smithi031:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-06T02:33:19.396 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi062.front.sepia.ceph.com 2023-10-06T02:33:19.396 DEBUG:teuthology.orchestra.run.smithi062:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-06T02:33:19.448 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-10-06T02:33:19.448 DEBUG:teuthology.orchestra.run.smithi031:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-06T02:33:19.453 DEBUG:teuthology.orchestra.run.smithi062:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-06T02:33:19.536 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-10-06T02:33:19.536 DEBUG:teuthology.orchestra.run.smithi031:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-06T02:33:19.541 DEBUG:teuthology.orchestra.run.smithi062:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-06T02:33:19.680 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-10-06T02:33:19.691 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-10-06T02:33:19.691 DEBUG:teuthology.orchestra.run.smithi031:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-06T02:33:19.753 DEBUG:teuthology.orchestra.run.smithi062:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-06T02:33:19.816 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-10-06T02:33:19.826 DEBUG:teuthology.orchestra.run.smithi031:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-06T02:33:19.831 DEBUG:teuthology.orchestra.run.smithi062:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-06T02:33:19.888 INFO:teuthology.orchestra.run.smithi031.stdout:kernel.core_pattern = core 2023-10-06T02:33:19.892 INFO:teuthology.orchestra.run.smithi062.stdout:kernel.core_pattern = core 2023-10-06T02:33:19.929 DEBUG:teuthology.orchestra.run.smithi031:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-06T02:33:19.963 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T02:33:19.963 DEBUG:teuthology.orchestra.run.smithi062:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-06T02:33:19.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T02:33:19.998 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-10-06T02:33:20.009 INFO:teuthology.task.internal:Transferring archived files... 2023-10-06T02:33:20.010 DEBUG:teuthology.misc:Transferring archived files from smithi031:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412129/remote/smithi031 2023-10-06T02:33:20.010 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-06T02:33:20.080 DEBUG:teuthology.misc:Transferring archived files from smithi062:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412129/remote/smithi062 2023-10-06T02:33:20.081 DEBUG:teuthology.orchestra.run.smithi062:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-06T02:33:20.155 INFO:teuthology.task.internal:Removing archive directory... 2023-10-06T02:33:20.155 DEBUG:teuthology.orchestra.run.smithi031:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-06T02:33:20.160 DEBUG:teuthology.orchestra.run.smithi062:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-06T02:33:20.196 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-10-06T02:33:20.208 INFO:teuthology.task.internal:Not uploading archives. 2023-10-06T02:33:20.208 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-10-06T02:33:20.233 INFO:teuthology.task.internal:Tidying up after the test... 2023-10-06T02:33:20.234 DEBUG:teuthology.orchestra.run.smithi031:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-06T02:33:20.238 DEBUG:teuthology.orchestra.run.smithi062:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-06T02:33:20.270 INFO:teuthology.orchestra.run.smithi031.stdout: 524301 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 6 02:33 /home/ubuntu/cephtest 2023-10-06T02:33:20.275 INFO:teuthology.orchestra.run.smithi062.stdout: 524301 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 6 02:33 /home/ubuntu/cephtest 2023-10-06T02:33:20.278 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-10-06T02:33:20.289 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-10-06T02:33:20.384 INFO:teuthology.nuke:Checking targets against current locks 2023-10-06T02:33:20.408 DEBUG:teuthology.nuke:shortname: smithi031 2023-10-06T02:33:20.409 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-06T02:33:20.432 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi031.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412129', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 01:58:52.479588', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLZXMnoxnVb2uZqUBuqUws24Kw6G5H9Ueh504kqdn3nWt5HsTr4MrTt+YmJAfIwE8R4h+T5Wag7WSIB7NhjqUsQ='} 2023-10-06T02:33:20.435 DEBUG:teuthology.nuke:shortname: smithi062 2023-10-06T02:33:20.435 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-06T02:33:20.455 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi062.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412129', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 01:58:52.481061', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJ9YBgmw9WCA672tKizZk9aWL2VzsFwWWtEBkVvpPLrsDmh+ytXxJq/7oZwtwZzaHXQZNuxRpkPQY6vUM4uonkY='} 2023-10-06T02:33:20.486 INFO:teuthology.orchestra.console.smithi031:Power off 2023-10-06T02:33:20.486 DEBUG:teuthology.orchestra.console.smithi031:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-06T02:33:20.499 INFO:teuthology.orchestra.console.smithi062:Power off 2023-10-06T02:33:20.499 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-06T02:33:20.508 DEBUG:teuthology.orchestra.console.smithi031:power off output: Chassis Power Control: Down/Off 2023-10-06T02:33:20.509 DEBUG:teuthology.orchestra.console.smithi031:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T02:33:20.520 DEBUG:teuthology.orchestra.console.smithi062:power off output: Chassis Power Control: Down/Off 2023-10-06T02:33:20.520 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T02:33:20.534 DEBUG:teuthology.orchestra.console.smithi031:check power output: Chassis Power is on 2023-10-06T02:33:20.541 DEBUG:teuthology.orchestra.console.smithi062:check power output: Chassis Power is on 2023-10-06T02:33:24.535 DEBUG:teuthology.orchestra.console.smithi031:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T02:33:24.542 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T02:33:24.644 DEBUG:teuthology.orchestra.console.smithi031:check power output: Chassis Power is on 2023-10-06T02:33:24.651 DEBUG:teuthology.orchestra.console.smithi062:check power output: Chassis Power is on 2023-10-06T02:33:28.646 DEBUG:teuthology.orchestra.console.smithi031:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T02:33:28.654 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T02:33:28.756 DEBUG:teuthology.orchestra.console.smithi031:check power output: Chassis Power is off 2023-10-06T02:33:28.764 DEBUG:teuthology.orchestra.console.smithi062:check power output: Chassis Power is off 2023-10-06T02:33:28.857 INFO:teuthology.orchestra.console.smithi031:Power off completed 2023-10-06T02:33:28.865 INFO:teuthology.orchestra.console.smithi062:Power off completed 2023-10-06T02:33:28.970 INFO:teuthology.run:Summary data: client.0-kernel-sha1: fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c client.1-kernel-sha1: fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c description: fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/misc}} duration: 1539.519668340683 failure_reason: '"1696558981.970328 mon.a (mon.0) 200 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log' flavor: default owner: scheduled_yuriw@teuthology success: false 2023-10-06T02:33:28.970 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-06T02:33:29.055 INFO:teuthology.run:FAIL